From xen-devel-bounces@lists.xenproject.org Thu Feb 01 00:06:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 00:06:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674184.1048957 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVKb2-00089t-0P; Thu, 01 Feb 2024 00:06:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674184.1048957; Thu, 01 Feb 2024 00:06:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVKb1-00089m-TD; Thu, 01 Feb 2024 00:06:23 +0000
Received: by outflank-mailman (input) for mailman id 674184;
 Thu, 01 Feb 2024 00:06:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F6Mk=JK=proton.me=johnlpoole@srs-se1.protection.inumbo.net>)
 id 1rVKb0-00089g-3y
 for xen-devel@lists.xen.org; Thu, 01 Feb 2024 00:06:22 +0000
Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch
 [185.70.40.131]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b8c1a7b3-c095-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 01:06:19 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b8c1a7b3-c095-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me;
	s=protonmail; t=1706745976; x=1707005176;
	bh=Zkfr6o6QSmCjTM/cjg8EpuSt+sqLtvN4C2pPrvDN4hM=;
	h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date:
	 Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector;
	b=dlaw3VA6LHfOfu5V1SBYAX0kzy7OrJVDZ4L8v3jViejRL/3VNMshrRFq8PM4aZhxf
	 wdr93jWPS1/7w7dw1B/fN7Uuov0IleBNzlEWGdDV+x5xiHM08Im5xG8/YzPaDi8zcw
	 3n1c1pDg8Uxu8PZBFhuRokauO25Xr4lgce9c8puhtOnOEsM3zxYckL6BC93bYsQ5hu
	 Ait3+og2WYiRj79+4rS2GNXk+y/851HBu5EIV05+Kcyh6wYAjl2Sg/37bONDKYdbTd
	 p8/XW+CfVfp7100d96fZTA6EwTznIp9PWFAldGK/mtkxmB4U13UuZSKmMqTI9KD5TQ
	 YoR1xtqefg5uQ==
Date: Thu, 01 Feb 2024 00:05:58 +0000
To: xen-devel@lists.xen.org
From: "John L. Poole" <johnlpoole@proton.me>
Subject: Nullifying Recently Introduced Xen Headers Check
Message-ID: <fb379476-f6ae-4eb7-b3c6-6f0ad175964d@proton.me>
Feedback-ID: 80499891:user:proton
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Greetings,

Gentoo Linux maintains two packages for Xen:

 =C2=A0 1) app-emulation/xen-tools ("xen-tools")
 =C2=A0 2) app-emulation/xen ("xen")

xen-tools is a prerequisite to xen.=C2=A0 So a failure
to install xen-tools will preclude any attempt to install xen.
I acknowledge that building xen-tools before building xen
is backwards and/or runs contrary to your project's way of
building, but that is what the Gentoo paradigm has been.

In trying to troubleshoot why a Gentoo 4.18.0 ebuild fails,
I ended up downloading your archive and successfully building it.
I then decided to compare the products built to see where
they varied in processing order.=C2=A0 The Xen Project builds xen first,
then the tools.

To see a visualization of the to build systems and the object
files they create matched in chronological order, see:

https://salemdata.us/xen/gentoo-4.18.0/comparison_of_build_orders_Xen_4.18.=
xhtml
 =C2=A0=C2=A0=C2=A0 or the LibreOffice workbook I created the comparison in=
:
https://salemdata.us/xen/gentoo-4.18.0/comparison_of_build_orders_Xen_4.18.=
ods

In March 2023, this commit added some sort of headers check:

https://xenbits.xen.org/gitweb/?p=3Dqemu-xen.git;a=3Dcommit;f=3Dinclude/hw/=
xen/xen_native.h;h=3De2abfe5ec67b69fb310fbeaacf7e68d61d16609e

Specifically, the lines 4-6 in [qemu-xen.git] / include / hw / xen /
xen_native.h:

 =C2=A0=C2=A0 4 #ifdef __XEN_INTERFACE_VERSION__
 =C2=A0=C2=A0 5 #error In Xen native files, include xen_native.h before oth=
er Xen
headers
 =C2=A0=C2=A0 6 #endif

cause Gentoo's build to error out.=C2=A0 See line 24790:

 =C2=A0=C2=A0 5 | #error In Xen native files, include xen_native.h before o=
ther
Xen headers

at https://salemdata.us/xen/xen_tools_20240128_Sun_174740.script.html.

What I have done is create a patch for a draft Gentoo ebuild which nullifie=
s
lines 4-6 by wrapping them in a comment:

 =C2=A0=C2=A0=C2=A0 --- a/tools/qemu-xen/include/hw/xen/xen_native.h
 =C2=A0=C2=A0=C2=A0 +++ b/tools/qemu-xen/include/hw/xen/xen_native.h
 =C2=A0=C2=A0=C2=A0 @@ -1,9 +1,9 @@
 =C2=A0=C2=A0=C2=A0=C2=A0 #ifndef QEMU_HW_XEN_NATIVE_H
 =C2=A0=C2=A0=C2=A0=C2=A0 #define QEMU_HW_XEN_NATIVE_H

 =C2=A0=C2=A0=C2=A0 -#ifdef __XEN_INTERFACE_VERSION__
 =C2=A0=C2=A0=C2=A0 +/* #ifdef __XEN_INTERFACE_VERSION__
 =C2=A0=C2=A0=C2=A0=C2=A0 #error In Xen native files, include xen_native.h =
before other Xen
headers
 =C2=A0=C2=A0=C2=A0 -#endif
 =C2=A0=C2=A0=C2=A0 +#endif */

This patch allows the Gentoo xen-tools ebuild to successfully build.

Question: is there a risk nullifying the above 3 lines in xen_native.h
given Gentoo's backward build paradigm?

Thank you.

John Poole




From xen-devel-bounces@lists.xenproject.org Thu Feb 01 00:15:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 00:15:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674190.1048967 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVKk9-0001Vy-1E; Thu, 01 Feb 2024 00:15:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674190.1048967; Thu, 01 Feb 2024 00:15:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVKk8-0001Vr-Th; Thu, 01 Feb 2024 00:15:48 +0000
Received: by outflank-mailman (input) for mailman id 674190;
 Thu, 01 Feb 2024 00:15:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i6jM=JK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVKk7-0001Vf-Qk
 for xen-devel@lists.xen.org; Thu, 01 Feb 2024 00:15:47 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0bbbb200-c097-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 01:15:46 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-33ae3cc8a6aso231720f8f.2
 for <xen-devel@lists.xen.org>; Wed, 31 Jan 2024 16:15:46 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 m4-20020a5d6244000000b0033ae4a3b285sm12269746wrv.36.2024.01.31.16.15.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 31 Jan 2024 16:15:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0bbbb200-c097-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706746546; x=1707351346; darn=lists.xen.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wB6q9VBrbFIo6ekJmlUp4iZqdgxywdF1k8W9gXCxehw=;
        b=EPgaPeVNemS3jMOtjXCprzzwYgWF4vGtO+u1tI6CB2+UQBEIeJ6fRxwiy91thSJsQy
         Cgavz1WsRarSEGbN6MFldqfh4aV9R3ugrCNiR6iIsx70EGt44hbPZq9DJGz+Mb99wW32
         hdQX8+PhJZ5IUqd9Fd6tHTj8i7a+F9ttZcfV0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706746546; x=1707351346;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wB6q9VBrbFIo6ekJmlUp4iZqdgxywdF1k8W9gXCxehw=;
        b=edcxKaRSC0Zhor8X16zabhRJk2qg+QTHRI2uughIcByq1sjFmn0rk/a0DkLZzpPV18
         jUdmqAbv+ZOrYWiZz6uErfLC3E6+1XNS6UF8TAV864EeCTkkHXOSEAL0XKMzRMJKim0D
         ULDk709Ncu91HxQszb6lFEQYbW6s93h1kDtj0IxKbwIjhyBVZxPShfzjfVhjfawkytId
         GPhmKqwR37ssqIgBXn9+thrYYqjy4PRDGbzIU4kd087iRf70CrhT0jvzxsdG+PZeDBFA
         PYyVsEqV9JONrobCIeQKbGS7ubW+DzSC87vi0jK4RvofXtj+t52R0uiBG+hbgtNBzuP6
         i2FA==
X-Gm-Message-State: AOJu0YxlFX6aBmHZA9zTSKIdp/a8zZtPdig+ALw+WNCpOlMLa/Agr1N5
	ZeWcceCuaxRtLoJLWocL5M3gixe1jCfPKLQSd1L+0EwmKzGc3+GUgMT92+oFMQQ=
X-Google-Smtp-Source: AGHT+IHZl+S8Way3RhddZvdYpiFGMASnHvje/NzNyk855TON3IX/JgvrZNVpAONtf6zrp4V4MaVF+w==
X-Received: by 2002:a5d:4049:0:b0:33a:f51a:f7ba with SMTP id w9-20020a5d4049000000b0033af51af7bamr435736wrp.49.1706746546027;
        Wed, 31 Jan 2024 16:15:46 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUpLTGj+TmMpxpItyXt1HUMuJQgnCzdG5KlQS+4HIeEJGoApfGbq64860OqwH2HtdMGiFKSz2bLkxYpD3K3eCx+r9KdvZGtkB7TCl52l9F74DLxBd8Xbw8tN6Ov4w==
Message-ID: <d0cd33d9-a147-4e6c-9b09-6371a4f25f28@citrix.com>
Date: Thu, 1 Feb 2024 00:15:45 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Nullifying Recently Introduced Xen Headers Check
Content-Language: en-GB
To: "John L. Poole" <johnlpoole@proton.me>, xen-devel@lists.xen.org,
 David Woodhouse <dwmw2@infradead.org>
References: <fb379476-f6ae-4eb7-b3c6-6f0ad175964d@proton.me>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <fb379476-f6ae-4eb7-b3c6-6f0ad175964d@proton.me>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01/02/2024 12:05 am, John L. Poole wrote:
> Greetings,
>
> Gentoo Linux maintains two packages for Xen:
>
>    1) app-emulation/xen-tools ("xen-tools")
>    2) app-emulation/xen ("xen")
>
> xen-tools is a prerequisite to xen.  So a failure
> to install xen-tools will preclude any attempt to install xen.
> I acknowledge that building xen-tools before building xen
> is backwards and/or runs contrary to your project's way of
> building, but that is what the Gentoo paradigm has been.
>
> In trying to troubleshoot why a Gentoo 4.18.0 ebuild fails,
> I ended up downloading your archive and successfully building it.
> I then decided to compare the products built to see where
> they varied in processing order.  The Xen Project builds xen first,
> then the tools.
>
> To see a visualization of the to build systems and the object
> files they create matched in chronological order, see:
>
> https://salemdata.us/xen/gentoo-4.18.0/comparison_of_build_orders_Xen_4.18.xhtml
>      or the LibreOffice workbook I created the comparison in:
> https://salemdata.us/xen/gentoo-4.18.0/comparison_of_build_orders_Xen_4.18.ods
>
> In March 2023, this commit added some sort of headers check:
>
> https://xenbits.xen.org/gitweb/?p=qemu-xen.git;a=commit;f=include/hw/xen/xen_native.h;h=e2abfe5ec67b69fb310fbeaacf7e68d61d16609e
>
> Specifically, the lines 4-6 in [qemu-xen.git] / include / hw / xen /
> xen_native.h:
>
>     4 #ifdef __XEN_INTERFACE_VERSION__
>     5 #error In Xen native files, include xen_native.h before other Xen
> headers
>     6 #endif
>
> cause Gentoo's build to error out.  See line 24790:
>
>     5 | #error In Xen native files, include xen_native.h before other
> Xen headers
>
> at https://salemdata.us/xen/xen_tools_20240128_Sun_174740.script.html.
>
> What I have done is create a patch for a draft Gentoo ebuild which nullifies
> lines 4-6 by wrapping them in a comment:
>
>      --- a/tools/qemu-xen/include/hw/xen/xen_native.h
>      +++ b/tools/qemu-xen/include/hw/xen/xen_native.h
>      @@ -1,9 +1,9 @@
>       #ifndef QEMU_HW_XEN_NATIVE_H
>       #define QEMU_HW_XEN_NATIVE_H
>
>      -#ifdef __XEN_INTERFACE_VERSION__
>      +/* #ifdef __XEN_INTERFACE_VERSION__
>       #error In Xen native files, include xen_native.h before other Xen
> headers
>      -#endif
>      +#endif */
>
> This patch allows the Gentoo xen-tools ebuild to successfully build.
>
> Question: is there a risk nullifying the above 3 lines in xen_native.h
> given Gentoo's backward build paradigm?

CC'ing David who authored that Qemu patch.  He'll be best placed to answer.

However, as a word of warning if you're not aware.  The name
__XEN_INTERFACE_VERSION__ is massively dishonest; read it as if it said
"unstable interfaces".

We're working to stabilise the tools->xen interface, but the nice shiny
future is still a while away.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 00:23:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 00:23:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674195.1048976 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVKro-0003Ia-QD; Thu, 01 Feb 2024 00:23:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674195.1048976; Thu, 01 Feb 2024 00:23:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVKro-0003IT-NO; Thu, 01 Feb 2024 00:23:44 +0000
Received: by outflank-mailman (input) for mailman id 674195;
 Thu, 01 Feb 2024 00:23:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=dNsH=JK=kernel.org=kuba@srs-se1.protection.inumbo.net>)
 id 1rVKrn-0003IN-OO
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 00:23:43 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 26d7b74d-c098-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 01:23:42 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id D31196190F;
 Thu,  1 Feb 2024 00:23:40 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 646C0C433F1;
 Thu,  1 Feb 2024 00:23:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 26d7b74d-c098-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1706747020;
	bh=7gsKG6V5s1dopCQuysbsTz34HGC4+l+xc5j4okSwkgM=;
	h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
	b=lk+5raA/Cc61dnv9T5cuz4gmbXaTbEiX742QUf0zqnvIfq3Fj3SYKTN/uoi3yP+g/
	 pLoelw6V1AJdY5MbbIOAYtjq2qesUk/3GIwLzdEFsN3PUAnwi+SxWwW3Lk9cgZsGoS
	 qlbBbWMb45LrvVgTPSl5JJf+j443ewZ3jZ6m6+9GWMGFXHWIqsoJk6VP5mL+8net8k
	 d2kwVXZdx5SbZTOwXM3ZVtzXTiOiHudjReNRanO2NxpAYRWNjdzws2AHMiA+mEOmcg
	 c0zNjS8GO0+Zn4sPw2NrOEAj6LZ6IyjsmFrxzylSiyQWii7Fsps6xQ8mJ7fjWr3eN7
	 6nJN7zWOFch0g==
Date: Wed, 31 Jan 2024 16:23:36 -0800
From: Jakub Kicinski <kuba@kernel.org>
To: Paul Durrant <paul@xen.org>
Cc: Jan Beulich <jbeulich@suse.com>, "netdev@vger.kernel.org"
 <netdev@vger.kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH net] xen-netback: properly sync TX responses
Message-ID: <20240131162336.7d3ba09e@kernel.org>
In-Reply-To: <980c6c3d-e10e-4459-8565-e8fbde122f00@suse.com>
References: <980c6c3d-e10e-4459-8565-e8fbde122f00@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Mon, 29 Jan 2024 14:03:08 +0100 Jan Beulich wrote:
> Invoking the make_tx_response() / push_tx_responses() pair with no lock
> held would be acceptable only if all such invocations happened from the
> same context (NAPI instance or dealloc thread). Since this isn't the
> case, and since the interface "spec" also doesn't demand that multicast
> operations may only be performed with no in-flight transmits,
> MCAST_{ADD,DEL} processing also needs to acquire the response lock
> around the invocations.
> 
> To prevent similar mistakes going forward, "downgrade" the present
> functions to private helpers of just the two remaining ones using them
> directly, with no forward declarations anymore. This involves renaming
> what so far was make_tx_response(), for the new function of that name
> to serve the new (wrapper) purpose.
> 
> While there,
> - constify the txp parameters,
> - correct xenvif_idx_release()'s status parameter's type,
> - rename {,_}make_tx_response()'s status parameters for consistency with
>   xenvif_idx_release()'s.

Hi Paul, is this one on your TODO list to review or should 
we do our best? :)
-- 
pw-bot: needs-ack


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 00:45:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 00:45:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674204.1048986 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVLD1-0006Yf-Kh; Thu, 01 Feb 2024 00:45:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674204.1048986; Thu, 01 Feb 2024 00:45:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVLD1-0006YY-Hy; Thu, 01 Feb 2024 00:45:39 +0000
Received: by outflank-mailman (input) for mailman id 674204;
 Thu, 01 Feb 2024 00:45:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=M+6X=JK=m5p.com=ehem@srs-se1.protection.inumbo.net>)
 id 1rVLCz-0006YN-TU
 for xen-devel@lists.xen.org; Thu, 01 Feb 2024 00:45:37 +0000
Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 353b2bce-c09b-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 01:45:35 +0100 (CET)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.17.1/8.15.2) with ESMTPS id 4110jL33006593
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Wed, 31 Jan 2024 19:45:27 -0500 (EST) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.17.1/8.15.2/Submit) id 4110jLE7006592;
 Wed, 31 Jan 2024 16:45:21 -0800 (PST) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 353b2bce-c09b-11ee-8a43-1f161083a0e0
Date: Wed, 31 Jan 2024 16:45:21 -0800
From: Elliott Mitchell <ehem+xen@m5p.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xen.org
Subject: Re: Thoughts on current Xen EDAC/MCE situation
Message-ID: <Zbrpoa3b5PuwBDTa@mattapan.m5p.com>
References: <Za7Vr+Rb25y/+MSy@mattapan.m5p.com>
 <111f5551-70fa-4ded-a62a-e0aa967b3c29@suse.com>
 <ZbBDJGKqTU9rdpgD@mattapan.m5p.com>
 <e7d67831-d7b9-48fd-9478-f89e85ce0422@suse.com>
 <ZbEq2JLY25TyLltX@mattapan.m5p.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZbEq2JLY25TyLltX@mattapan.m5p.com>
X-Spam-Status: No, score=0.0 required=10.0 tests=KHOP_HELO_FCRDNS
	autolearn=unavailable autolearn_force=no version=4.0.0
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on mattapan.m5p.com

On Wed, Jan 24, 2024 at 07:20:56AM -0800, Elliott Mitchell wrote:
> On Wed, Jan 24, 2024 at 08:23:15AM +0100, Jan Beulich wrote:
> > 
> > Third, as to Dom0's purposes of having the address: If all it is to use
> > it for is to pass it back to Xen, paths in the respective drivers will
> > necessarily be entirely different for the Xen vs the native cases.
> 
> I'm less than certain of the best place for Xen to intercept MCE events.
> For UE memory events, the simplest approach on Linux might be to wrap the
> memory_failure() function.  Yet for Linux/x86,
> mce_register_decode_chain() also looks like a very good candidate.

I did hope to get some response.

It really does look like, aside from being x86-only,
mce_register_decode_chain() is the ideal hook point.  Xen could forward
NMIs to Domain 0, then intercept them from the decode chain.  For UEs
Xen would mark the event handled, then create a new event for whichever
domain (if any) was effected.


Right now my main concern is several of the Linux MCE/EDAC drivers are
growing `if (cpu_feature_enabled(X86_FEATURE_HYPERVISOR)) return -ENODEV;`
calls.

This approach is being poisoned and will become quite difficult if this
isn't stopped.  The justification found for one instance was that it
"removed one message", with no useful information.  I cannot help
suspecting it involved a hypervisor from Redmond, WA and their engineers
are encouraged to poison interfaces used by others.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Thu Feb 01 03:32:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 03:32:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674208.1048998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVNnm-0004IL-1s; Thu, 01 Feb 2024 03:31:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674208.1048998; Thu, 01 Feb 2024 03:31:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVNnl-0004IE-Tn; Thu, 01 Feb 2024 03:31:45 +0000
Received: by outflank-mailman (input) for mailman id 674208;
 Thu, 01 Feb 2024 03:31:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVNnk-0004I4-AC; Thu, 01 Feb 2024 03:31:44 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVNnk-0005dY-1a; Thu, 01 Feb 2024 03:31:44 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVNnj-0005E8-Hz; Thu, 01 Feb 2024 03:31:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVNnj-0005Ox-Go; Thu, 01 Feb 2024 03:31:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jtC924xqWJRZxlyBAoCbZxTG0cODBUo0a61YtqXnzmk=; b=XvI+y/2Z9Rt++Gb4rlKrNigUCm
	WOZV02tm3sKwBM/ZdxjKr2wtSzS3q6WQoKqHAokzhTEpTN5daIzk6ArtlIUjG2326QnF80z6QY1C3
	0SGXlzBSSt71W2dsOX1EWcW7Xa9BmUzVqOIPc1zlYaYgt1XrspjN2sWVpfT88P3LDMYc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184535-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184535: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-rtds:guest-localmigrate/x10:fail:allowable
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=cc6ba68edf6dcd18c3865e7d7c0f1ed822796426
X-Osstest-Versions-That:
    xen=4a7e71aa085170f1a13976507c8e248f8715f116
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 01 Feb 2024 03:31:43 +0000

flight 184535 xen-unstable real [real]
flight 184546 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184535/
http://logs.test-lab.xenproject.org/osstest/logs/184546/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184546-retest

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds     20 guest-localmigrate/x10   fail REGR. vs. 184527

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184527
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184527
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184527
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184527
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184527
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184527
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184527
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184527
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184527
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184527
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184527
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184527
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  cc6ba68edf6dcd18c3865e7d7c0f1ed822796426
baseline version:
 xen                  4a7e71aa085170f1a13976507c8e248f8715f116

Last test of basis   184527  2024-01-30 06:02:22 Z    1 days
Testing same since   184535  2024-01-31 00:07:55 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     fail    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   4a7e71aa08..cc6ba68edf  cc6ba68edf6dcd18c3865e7d7c0f1ed822796426 -> master


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 04:11:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 04:11:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674217.1049014 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVOPe-000113-4I; Thu, 01 Feb 2024 04:10:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674217.1049014; Thu, 01 Feb 2024 04:10:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVOPe-00010w-1V; Thu, 01 Feb 2024 04:10:54 +0000
Received: by outflank-mailman (input) for mailman id 674217;
 Thu, 01 Feb 2024 04:10:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=RfkL=JK=nxp.com=peng.fan@srs-se1.protection.inumbo.net>)
 id 1rVOPd-00010q-I6
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 04:10:53 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01on062f.outbound.protection.outlook.com
 [2a01:111:f400:fe1f::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e29dc848-c0b7-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 05:10:50 +0100 (CET)
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com (2603:10a6:10:358::11)
 by AM0PR04MB6836.eurprd04.prod.outlook.com (2603:10a6:208:187::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.26; Thu, 1 Feb
 2024 04:10:47 +0000
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::c499:8cef:9bb1:ced6]) by DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::c499:8cef:9bb1:ced6%3]) with mapi id 15.20.7249.023; Thu, 1 Feb 2024
 04:10:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e29dc848-c0b7-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bxuci8oc2L6yTUp7UbN2BLK64Dw/5k1cBW5/hlcVrq0rMwbLBw3diikd+f8N2lTJOHHufMwtOoAcOZkNAxvxvWgNMF2l3mAFE1PvFYNg5Vgnbjv1Up6UCZRw2eQFMoAT4Io/pVgBcEkSa7gpW3/9bfOgTCUposgywM6RslX80CGKWU57VVzN1Ic4YDJ7yuRWNURepnmmF6qKBZKj11kfQrB0gCsleAC5D6rFoT3b6tf6KqOE6LqukgrbFUOT7D2KDI2KW+X7Wd0cRj+nbdt6JXW3pXYse9qf+mhxK2ihX61ow17qsnEqKcwye2WOM5t/JFrc+FhldDWH+enFNAJEfw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=DcVXz3h0TliXCAdcDKfzPkeUDq7pj0aXYjzX7MHA3hI=;
 b=Mal0n89NszSIHOutr+Qf+SJJjcwdsYH2H5yHqr5pemTMJrbeEzMnGvrrJE7SVy/VVYH3B9vXnCidOwAwEI/LBBiKG0rD7NKo6Xc5jECQ8p4C8CyKc/aRr7sdy9JChVmanNQB0rWwzKfb8POb6cyf54/V2rAcYzITVdYhd9odfGsGiSMz7mrcl4NEQaa1uzQiDyouvA4OfWMXr7U8QKDzMcy1tLhcPyH+fuiM4kSJ8moQdPREUZ4D3ptPOtWLfK3tP1YJ9KXIk2KDb4FM+/mRupvecoSauTFjFmhttZoCZOIIV2RbHceJ9R28fRznP1upGDnL+utse3vGdX+VXa+JXw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass
 header.d=nxp.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=DcVXz3h0TliXCAdcDKfzPkeUDq7pj0aXYjzX7MHA3hI=;
 b=CCjZnK1v9DKwnqdLy6gng359AVCmmqQBkb5TgUWTBl/YfemDXb0IlAvzxzORpR1B77humRALy+tuaRvO8bYHnzww6bUnWGZrf5fn/p7OUV6dtDbgNqYk3X9Zr1LoD42hnmzMRi2ONZZkU+BEL3klXY5uwU6GgXLNkJv11hEkh4A=
From: Peng Fan <peng.fan@nxp.com>
To: John Ernberg <john.ernberg@actia.se>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
CC: Jonas Blixt <jonas.blixt@actia.se>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: RE: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Topic: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Index: AQHaVDuqVQ/kgxLZgUK/m8/hE0ez5bDz198AgAA1LICAANMm8A==
Date: Thu, 1 Feb 2024 04:10:46 +0000
Message-ID:
 <DU0PR04MB941760D4A8972440B5312B2088432@DU0PR04MB9417.eurprd04.prod.outlook.com>
References: <20240131114952.305805-1-john.ernberg@actia.se>
 <20240131114952.305805-2-john.ernberg@actia.se>
 <a2f726f5-df4c-4d15-90af-7d59c0f1f513@xen.org>
 <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se>
In-Reply-To: <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=nxp.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DU0PR04MB9417:EE_|AM0PR04MB6836:EE_
x-ms-office365-filtering-correlation-id: 335956e2-f29d-4a32-8aa1-08dc22dbc475
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 sqH9RcPapi5UzcfyjoCF0sDBJaS5dHPs9J9oW0RMSLIHLCq8tLW0mMzytOKOhwQ6c3z04RL6gmJM8nV9kaQ2YSCl3XLFQRZ0WtEVsKnz7BZepBp2bk23QB0D1ah6SrzcVrosRZJAjYqPjbJs9fZNbYfI2j9izaysCjhMswVXJoXtpx0JJzzesHrwdZMHvkp8WJYV2Bw0G1112EP7fq81ALiaEIcCLiYl0uY0zpzAzvGMC+eZK+bcr1AegWgUn3T7SL7sb47jCBqAT2ScqsU+TaPvwkQC9yR7FYwQeQyg2lo2ZKXhZtfJPAc1bKYDf/6qg4+cIZ1UsIbq3RMGTl+X0I2dPT7IzaFueftjxF3cHN7vIkxE0SNwN+MxDv53g2uobrK1l6UwOsvFTHtFWqdQJZmrJR3aAFt3SAo2EW0ApGUepW0O2+1s+bRU+qAEmlGzSSOuVl3D3vF9aDfgOFhNk2CyOVRS1SM61zf77plrm9RuUFAQBfsc51smK3YJFs5WFdo6sDodS4umToOKmDKfOeDbVJzJCgzeVSgG1SQmwza+l1XQG8Kz2otHxrQ6WvrzDN8iHowKgLsRE4uBjdGMlhZDCiN4Fl3bzIbgXg7l4WzDEhCGGpYLWtTIPzH6akiXT7b2wngzWODWCqeOzVWirQ==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU0PR04MB9417.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(366004)(346002)(136003)(396003)(39860400002)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(71200400001)(6506007)(7696005)(53546011)(9686003)(8676002)(26005)(8936002)(4326008)(52536014)(44832011)(2906002)(64756008)(316002)(5660300002)(33656002)(54906003)(66556008)(66476007)(66446008)(86362001)(110136005)(38070700009)(66946007)(76116006)(41300700001)(966005)(45080400002)(478600001)(122000001)(55016003)(83380400001)(38100700002)(32563001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?iso-8859-1?Q?uz/duHf/AIjb52JGboPfCHdKUYcch1AZdVYbf196hs0qUguAfQou5cCLeT?=
 =?iso-8859-1?Q?baHIYTBy6tzDFj5mfyKyWPBDyT20O7XVeKsU/6n4+ZonOjoAXL7orvkkiT?=
 =?iso-8859-1?Q?+N3kpFu1A5rpVIcwB7Ry8+MfF16PIGb9oRpgB3pzrPyjzizAYd+fAqu0Kt?=
 =?iso-8859-1?Q?w4/VyNH29liEz62gjlk5tg3ZnQTTmIUGO22+UEMlgQzJqxwuFDlqpramEu?=
 =?iso-8859-1?Q?eKf5LVpJnyoMnGkAvGvK3rtxzLs76dDGHcOYU4QIl2WZUb/+kTl+YepJ0c?=
 =?iso-8859-1?Q?ZOCihAtfoqS1bWTPoEQE3hTXpBII08YUBQfWCykZilP5B3MXo8NfFAFkjH?=
 =?iso-8859-1?Q?CR3LCZDlJp17+tRsQjxW5mJuyBeSAD3E4VrMgRcuYrrXrzY9tae0PDIHVo?=
 =?iso-8859-1?Q?Zhws0duLG2X2gdLQ1JUyTfN50Tii5HBWYt29flJOlBWinhg4eV7t3616pY?=
 =?iso-8859-1?Q?QgvxL8ZXMhn3k78m7oYQyJXF7bI0s4P50Voemt70evd7Q21+gt7SOg/ktb?=
 =?iso-8859-1?Q?Dth/gh75J+pNN7mJ29VKYBxOpFgzHafWq6JB3kImvLQ5wufW1DZUk6/TFP?=
 =?iso-8859-1?Q?w6FqN4gAePUG8hKx5j7orOuJceHrkSqfHatDn/AIcIQDmDPEKcMdk3uIjW?=
 =?iso-8859-1?Q?6QsL5axxgY5spi+XrRfkE/RCNt8e4xgPyGjaFYluUHT0JVFVn2QGDsvOkP?=
 =?iso-8859-1?Q?PBtAlick8egCt4i2y8DykBHz84tbZxLudfWP8PB8BSEI+Edpx3YNUzFhIp?=
 =?iso-8859-1?Q?xg/fZMuOxY7kcdaJUV3+GmiMZuFD+PSXE715tAtMOvbIF484qPzaudNpLf?=
 =?iso-8859-1?Q?Fs6oOz7F8ghx/CMnjtJVH9bcWVRWdkQmgJg9UqP/q2oI4bJdV7/f2IqSHN?=
 =?iso-8859-1?Q?QedsVGMVpFeRto54c6DCkJhWOe5mHe8CZZ5T8Mv7hxMZhf2gMjPyyEFRS5?=
 =?iso-8859-1?Q?nPFCPrchbTGkJx7rWHPLNczAU1mKPBDM8fC8YiW4J03RYxu9Gega5tKq09?=
 =?iso-8859-1?Q?mCL/zlawo9L6JkcRVwMxzUUZOeZ26cXbHuI9j+NgdiAcO+V4r359GuN0qq?=
 =?iso-8859-1?Q?z8CJPrZyoXR214Mgir81YUEL8oJexk2w3Ov3pLCNRztmZ+3DPvfclN56pQ?=
 =?iso-8859-1?Q?Rl24ZMQy6hQDZ7rdX/+fMPgvsDVj3YAT1m2/cPkeC0CBeqX0moXoHWfGF8?=
 =?iso-8859-1?Q?Bih6P1rdPcp00/Dm0Dwoo5YDjtV8ajumt/RwBRQRJLquKM6aiBRE5OziAe?=
 =?iso-8859-1?Q?8k8Mgpn0uqln47EJtfCmZeaPnx/H6pXzyV4cpNtKeCXwnlnJepC8M2zbW5?=
 =?iso-8859-1?Q?3+KE+KsI+JWeiUFbzNd9uTZpcK210dtDGsktFaO/mRlIkAt1S9e7ozByE0?=
 =?iso-8859-1?Q?lX/Sb37N2UAV+h1+YAD/yb8CKz9mOCJp9kifa0MV6KrLke+VJNNEpkXseH?=
 =?iso-8859-1?Q?pJB1NE8UxW3A7HC0E0bGPvG/MqBA5tmsJG4OIRtPTe8WwzMW0RZAS/sEsg?=
 =?iso-8859-1?Q?u7npJevjaVAfLxd/8dXHf0bOwXJYSF9Hf4bvC2IGuI99OjXyjeofV2gvc+?=
 =?iso-8859-1?Q?Uc+PSvJW/sQxR07kKD0tHvWSOkCYTpp3N5k0IFx7RrEaZ/ZrxlXgeoLdK9?=
 =?iso-8859-1?Q?pWQRyhHPd/4EY=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DU0PR04MB9417.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 335956e2-f29d-4a32-8aa1-08dc22dbc475
X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Feb 2024 04:10:46.8616
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Xwvpi/unVz/Zhwr+n4EvBlXIWxxgOfxTqfYpYQzoC/fimMrUkzd3XudHskmIRRmobFhrB+k/acpyiE6381Bvtw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB6836

> Cc: Jonas Blixt <jonas.blixt@actia.se>; xen-devel@lists.xenproject.org
> Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
>=20
> Hi Julien,
>=20
> On 1/31/24 13:22, Julien Grall wrote:
> > Hi,
> >
> > On 31/01/2024 11:50, John Ernberg wrote:
> >> When using Linux for dom0 there are a bunch of drivers that need to
> >> do SMC SIP calls into the PSCI provider to enable certain hardware
> >> bits like the watchdog.
> >
> > Do you know which protocol this is under the hood. Is this SCMI?
>=20
> I think I confused myself here when I wrote the commit log.
>=20
> The EL3 code in our case is ATF, and it does not appear to be SCMI, nor P=
SCI.
> The register usage of these SMC SIP calls are as follows:
> a0 - service
> a1 - function
> a2-a7 - args
>=20
> In ATF the handler is declared as a runtime service.
>=20
> Would the appropriate commmit message here be something along the lines
> of below?
> """
> When using Linux for dom0 there are a bunch of drivers that need to do
> SMC
> SIP calls into the firmware to enable certain hardware bits like the watc=
hdog.
> """
> >
> >>
> >> Provide a basic platform glue that implements the needed SMC forwardin=
g.
> >>
> >> Signed-off-by: John Ernberg <john.ernberg@actia.se>
> >> ---
> >> NOTE: This is based on code found in NXP Xen tree located here:
> >> https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgi=
t
> >> hub.com%2Fnxp-imx%2Fimx-xen%2Fblob%2Flf-
> 5.10.y_4.13%2Fxen%2Farch%2Far
> >>
> m%2Fplatforms%2Fimx8qm.c&data=3D05%7C02%7Cpeng.fan%40nxp.com%7C
> 573b599a
> >>
> 4b4143ceca1d08dc2271e5be%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C
> 0%7C0%7
> >>
> C638423119777601548%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> wMDAiLCJQI
> >>
> joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=3DZO
> 0TXjL6
> >> g0W7TIZo8x8lTNBXEZW%2BDNcLPndWlEf5D2A%3D&reserved=3D0
> >
> > Anything after --- will be removed while applied to the three. I think
> > this NOTE should be written down in the commit message.
>=20
> Ack.
> >
> > You also possibly want a signed-off-by from Peng as this is his code.
>=20
> @Peng: May I add a sign-off from you?

Yeah. You could add my sign off.

> >
> >>
> >> =A0 xen/arch/arm/platforms/Makefile |=A0 1 +
> >> =A0 xen/arch/arm/platforms/imx8qm.c | 65
> >> +++++++++++++++++++++++++++++++++
> >> =A0 2 files changed, 66 insertions(+)
> >> =A0 create mode 100644 xen/arch/arm/platforms/imx8qm.c
> >>
> >> diff --git a/xen/arch/arm/platforms/Makefile
> >> b/xen/arch/arm/platforms/Makefile index 8632f4115f..bec6e55d1f
> 100644
> >> --- a/xen/arch/arm/platforms/Makefile
> >> +++ b/xen/arch/arm/platforms/Makefile
> >> @@ -9,5 +9,6 @@ obj-$(CONFIG_ALL_PLAT)=A0=A0 +=3D sunxi.o
> >> =A0 obj-$(CONFIG_ALL64_PLAT) +=3D thunderx.o
> >> =A0 obj-$(CONFIG_ALL64_PLAT) +=3D xgene-storm.o
> >> =A0 obj-$(CONFIG_ALL64_PLAT) +=3D brcm-raspberry-pi.o
> >> +obj-$(CONFIG_ALL64_PLAT) +=3D imx8qm.o
> >> =A0 obj-$(CONFIG_MPSOC_PLATFORM)=A0 +=3D xilinx-zynqmp.o
> >> =A0 obj-$(CONFIG_MPSOC_PLATFORM)=A0 +=3D xilinx-zynqmp-eemi.o diff --g=
it
> >> a/xen/arch/arm/platforms/imx8qm.c
> b/xen/arch/arm/platforms/imx8qm.c
> >> new file mode 100644 index 0000000000..a9cd9c3615
> >> --- /dev/null
> >> +++ b/xen/arch/arm/platforms/imx8qm.c
> >> @@ -0,0 +1,65 @@
> >> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> >> +/*
> >> + * xen/arch/arm/platforms/imx8qm.c
> >> + *
> >> + * i.MX 8QM setup
> >> + *
> >> + * Copyright (c) 2016 Freescale Inc.
> >> + * Copyright 2018-2019 NXP
> >> + *
> >> + *
> >> + * Peng Fan <peng.fan@nxp.com>
> >> + */
> >> +
> >> +#include <asm/platform.h>
> >> +#include <asm/smccc.h>
> >> +
> >> +static const char * const imx8qm_dt_compat[] __initconst =3D {
> >> +=A0=A0=A0 "fsl,imx8qm",
> >> +=A0=A0=A0 "fsl,imx8qxp",
> >> +=A0=A0=A0 NULL
> >> +};
> >> +
> >> +static bool imx8qm_smc(struct cpu_user_regs *regs) {
> >
> > Your implementation below will not only forward SMC for dom0 but also
> > for any non-trusted domains. Have you investigated that all the SIP
> > calls are safe to be called by anyone?
>=20
> We use pure virtualized domUs, so we do not expect any calls to this SMC
> interface from the guest. I'll limit it to dom0.

Would you mind to share what features are supported in your DomU?

Pure virtualized, you using xen pv or virtio?

Thanks,
Peng.

> >
> > But even if we restrict to dom0, have you checked that none of the
> > SMCs use buffers?
> I haven't found any such instances in the Linux kernel where a buffer is =
used.
> Adding a call filtering like suggested below additions of such functions =
can be
> discovered and adapted for if they would show up later.
> >
> > Rather than providing a blanket forward, to me it sounds more like you
> > want to provide an allowlist of the SMCs. This is more futureproof and
> > avoid the risk to expose unsafe SMCs to any domain.
> >
> > For an example, you can have a look at the EEMI mediator for Xilinx.
>=20
> Ack. Do you prefer to see only on SMCCC service level or also on function
> level? (a1 register, per description earlier)
> >
> > Cheers,
> >
>=20
> Thanks! // John Ernberg


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 04:51:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 04:51:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674221.1049024 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVP2Y-0006Zp-3r; Thu, 01 Feb 2024 04:51:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674221.1049024; Thu, 01 Feb 2024 04:51:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVP2Y-0006Zi-1D; Thu, 01 Feb 2024 04:51:06 +0000
Received: by outflank-mailman (input) for mailman id 674221;
 Thu, 01 Feb 2024 04:51:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WDVG=JK=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rVP2X-0006Zb-71
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 04:51:05 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20627.outbound.protection.outlook.com
 [2a01:111:f400:7eaa::627])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7eb4801b-c0bd-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 05:51:01 +0100 (CET)
Received: from PH8PR02CA0020.namprd02.prod.outlook.com (2603:10b6:510:2d0::9)
 by BY5PR12MB4243.namprd12.prod.outlook.com (2603:10b6:a03:20f::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.8; Thu, 1 Feb
 2024 04:50:57 +0000
Received: from SN1PEPF0002BA4C.namprd03.prod.outlook.com
 (2603:10b6:510:2d0:cafe::22) by PH8PR02CA0020.outlook.office365.com
 (2603:10b6:510:2d0::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.26 via Frontend
 Transport; Thu, 1 Feb 2024 04:50:57 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SN1PEPF0002BA4C.mail.protection.outlook.com (10.167.242.69) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Thu, 1 Feb 2024 04:50:57 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Wed, 31 Jan
 2024 22:50:56 -0600
Received: from [172.23.114.18] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Wed, 31 Jan 2024 22:50:55 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7eb4801b-c0bd-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DIbXHSsvKKbwgnWF8/gmW5lIfPKu8vhpGid9qZlFs+a88X/IieMq8OiwFhHlnelS+PgEnWYum3Qcoa3SzktP2B6GCBxPkhOuRCBo8pfa/GZV9ti+wLQgUSlu0dgtcbw26WTqju04eDz5AB2ePU5bO/8+ubXm86fRGdWiZsqiR9Cj3Eo8LR9klGYQxQIS6zyQoLjUAliQ8JnnGHo7AVUz8kZ8jfGAwwZSAniCXSHmust6uZMWeK1opqFlwB1aAEnN3luD5+C8e80j5ogamundBAnTO+rOI17JN+oZ3pRHRyhtaT501Bb34Rn1YCIiXIRQTvrBAOXlU2EhWorjyVh5wQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Ii7bxJk70kAfTZiQJJ3NC2wsY5BfdVr76TAoUdYYDxA=;
 b=QLXRfH3Z/3ZU3Yf9dflSYEd0n5H9XvU/iYzv+YFchIuFcP593y7ExfzxObUSlbiwUsUONEVSXzMMo/md/Ke2VWEEO4UDBCoLkL9DkiTCERB9jk3ewgQnQjIp/OtDBJesT0JO5ZRNYgcn37rfgCL0cwS5N57g8PSCHFcJGCkYsvgW2GN2NHcoCV51ZS6FOHBEv0DsdhjZU555ONqSZ0FGXHcREplpgJDDxbbsdbI/XCrxRiv1ZZOY5sIjkvQ27bB6SByM7Cwevy2UWalMBNkV6v8xnIauTobmL3/kKq+ZeDnMCslXROLXQfDqhsUTuTzlVoGlQSNCxqPnmgdgcP8HDg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Ii7bxJk70kAfTZiQJJ3NC2wsY5BfdVr76TAoUdYYDxA=;
 b=Cj77J2EvHz6DKCZrxppEGXAAK5IHPmptx+4FozOmzTuXWFlc4yuHQcSU904FW2DIckaBVA3Aj8O/6NCsInU+vf6WEFL65SR+RUBTJ04ai1nw9zj+w6GhZUV8UUAHd1v2WwpDuC1GQcQcDppv7u2b1/qnkMiuWRvIpbnlhKbGgbM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <48a16cb3-9cb9-4b7d-a950-f190a9a64bc2@amd.com>
Date: Wed, 31 Jan 2024 23:50:55 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 10/15] vpci/header: emulate PCI_COMMAND register for
 guests
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Julien
 Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	<xen-devel@lists.xenproject.org>
References: <20240109215145.430207-1-stewart.hildebrand@amd.com>
 <20240109215145.430207-11-stewart.hildebrand@amd.com>
 <50e79338-b2fd-40e9-b754-4e725e27ceb3@suse.com>
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <50e79338-b2fd-40e9-b754-4e725e27ceb3@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4C:EE_|BY5PR12MB4243:EE_
X-MS-Office365-Filtering-Correlation-Id: e1056df2-3584-4de1-804f-08dc22e16149
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	noopwhAvahLcVlVQN7DyVrhMIWwmcUpbQaEAIZTFLsR4mCgOa/zdZXvq8JcfjW60+rf0wyaQhLqATH6pANyCkMTpYQHVUP69Wsq3ZTiO3+yFbIS6J4Qx2wM1Nka2bfJ7Y/Y0r4Zdv1OdH72zvmzn1T3sYR+mBhoebpIzPT1P44Iyd3lpdpw33p5LUzT6BHn+ecwYaoWKRA7sUmBxvVlC/mRhiCXUjCtS0KSOelhCWiXjC/rvWrICH8/Qqmd3fPYqmtYdWg9RhwJwgwA4m07xjcYii+of6Cvgcfnf/aZ0MHOY74prxNrslcelJHjzHBB2juQZL/i4eRvuQCJsCM0J4RMDd2Nt/bWaj1slWK0icJf0NptURICBSXxPwU34NEW2Z2S6vh0GGBoWbVBvrGDSopSlqr112uyS2SMdmLPBKygWq8fkrCCiFTCnqDM1NrOySqsgyPSq3R0nLs5Sh47UqQAUwZs8Kibm8uK5U2sI7EPFSwNnIPqpgxf/qDTzbPD2yjHAil2QtzEKJPn+1XgfLleepQs5p9UvHg1rlt8IMZto95W11bXouvHqWgAWy/AkfoZ6IHWfWOlBJeXVCtUmufns8/30MeFruxxcBaGM5naYZUA10PxWX2CXiQ51tZ0oWS/rrvTOB46BpLBQcXPr9I7yRKrz5zfSfT9pUv2/JbZcCoiTU3+bKkpwU0CyJwDIMZ/eQif75FdF7c+3ybU7bxM+kEoHamooBtS9mgxJbah4/ftDnm/1ZqoBHFSl7+Fsg25U296dlaiOQ0zTByeEPdOvNObwzOwzT+3ZV3/kdVUOruwkbDP2S+Cc2nnCUI6aWdW51mywp9iRzztWcwKtog==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(136003)(346002)(376002)(396003)(230922051799003)(186009)(82310400011)(451199024)(64100799003)(1800799012)(46966006)(40470700004)(36840700001)(41300700001)(47076005)(36860700001)(8936002)(44832011)(4326008)(2906002)(8676002)(5660300002)(70206006)(7416002)(31696002)(86362001)(70586007)(36756003)(54906003)(16576012)(81166007)(83380400001)(82740400003)(356005)(6916009)(316002)(53546011)(478600001)(966005)(2616005)(336012)(426003)(40480700001)(40460700003)(26005)(31686004)(43740500002)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Feb 2024 04:50:57.4335
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e1056df2-3584-4de1-804f-08dc22e16149
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF0002BA4C.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4243

On 1/25/24 10:43, Jan Beulich wrote:
> On 09.01.2024 22:51, Stewart Hildebrand wrote:
>> --- a/xen/drivers/vpci/header.c
>> +++ b/xen/drivers/vpci/header.c
>> @@ -168,6 +168,9 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
>>      if ( !rom_only )
>>      {
>>          pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
>> +        /* Show DomU that we updated P2M */
>> +        header->guest_cmd &= ~PCI_COMMAND_MEMORY;
>> +        header->guest_cmd |= cmd & PCI_COMMAND_MEMORY;
>>          header->bars_mapped = map;
>>      }
> 
> I don't follow what the comment means to say. The bit in question has no
> real connection to the P2M, and the guest also may have no notion of the
> underlying hypervisor's internals. Likely connected to ...

Indeed. If the comment survives to v13, I'll update it to:

        /* Now that we updated P2M, show DomU change to PCI_COMMAND_MEMORY */

> 
>> @@ -524,9 +527,26 @@ static void cf_check cmd_write(
>>  {
>>      struct vpci_header *header = data;
>>  
>> +    if ( !is_hardware_domain(pdev->domain) )
>> +    {
>> +        const struct vpci *vpci = pdev->vpci;
>> +
>> +        if ( (vpci->msi && vpci->msi->enabled) ||
>> +             (vpci->msix && vpci->msix->enabled) )
>> +            cmd |= PCI_COMMAND_INTX_DISABLE;
>> +
>> +        /*
>> +         * Do not show change to PCI_COMMAND_MEMORY bit until we finish
>> +         * modifying P2M mappings.
>> +         */
>> +        header->guest_cmd = (cmd & ~PCI_COMMAND_MEMORY) |
>> +                            (header->guest_cmd & PCI_COMMAND_MEMORY);
>> +    }
> 
> ... the comment here, but then shouldn't it be that the guest can't even
> issue a 2nd cfg space access until the present write has been carried out?
> Otherwise I'd be inclined to claim that such a partial update is unlikely
> to be spec-conformant.

Due to the raise_softirq() call added in

  3e568fa9e19c ("vpci: fix deferral of long operations")

my current understanding is: when the guest toggles memory decoding, the guest vcpu doesn't resume execution until vpci_process_pending() and modify_decoding() have finished. So I think the guest should see a consistent state of the register, unless it was trying to read from a different vcpu than the one doing the writing.

Regardless, if the guest did have an opportunity to successfully read the partially updated state of the register, I'm not really spotting what part of the spec that would be a violation of. PCIe 6.1 has this description regarding the bit: "When this bit is Set" and "When this bit is Clear" the device will decode (or not) memory accesses. The spec doesn't seem to distinguish whether the host or the device itself is the one to set/clear the bit. One might even try to argue the opposite: allowing the bit to be toggled before the device reflects the change would be a violation of spec. Since the spec is ambiguous in this regard, I don't think either argument is particularly strong.

Chesterton's fence: the logic for deferring the update of PCI_COMMAND_MEMORY in guest_cmd was added between v10 and v11 of this series. I went back to look at the review comments on v10 [1], but the rationale is still not entirely clear to me. At the end of the day, with the information I have at hand, I suspect it would be fine either way (whether updating guest_cmd is deferred or not). If no other info comes to light, I'm leaning toward not deferring because it would be simpler to update the bit right away in cmd_write().

[1] https://lore.kernel.org/xen-devel/ZVy73iJ3E8nJHvgf@macbook.local/

> 
>> @@ -843,6 +885,15 @@ static int cf_check init_header(struct pci_dev *pdev)
>>      if ( cmd & PCI_COMMAND_MEMORY )
>>          pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd & ~PCI_COMMAND_MEMORY);
>>  
>> +    /*
>> +     * Clear PCI_COMMAND_MEMORY and PCI_COMMAND_IO for DomUs, so they will
>> +     * always start with memory decoding disabled and to ensure that we will not
>> +     * call modify_bars() at the end of this function.
>> +     */
>> +    if ( !is_hwdom )
>> +        cmd &= ~(PCI_COMMAND_MEMORY | PCI_COMMAND_IO);
>> +    header->guest_cmd = cmd;
> 
> With PCI_COMMAND_MEMORY clear, the hw reg won't further be written on the
> success return path. Yet wouldn't we better clear PCI_COMMAND_IO also in
> hardware (until we properly support it)?

Yes, I'll clear PCI_COMMAND_IO in hardware too

> 
> I also think the insertion point for the new code isn't well chosen: The
> comment just out of context indicates that the code in context above is
> connected to the subsequent code. Whereas the addition is not.

I'll rearrange it

> 
>> --- a/xen/drivers/vpci/msi.c
>> +++ b/xen/drivers/vpci/msi.c
>> @@ -70,6 +70,15 @@ static void cf_check control_write(
>>  
>>          if ( vpci_msi_arch_enable(msi, pdev, vectors) )
>>              return;
>> +
>> +        /*
>> +         * Make sure domU doesn't enable INTx while enabling MSI.
>> +         */
> 
> Nit: This ought to be a single line comment, just like ...

OK, I'll make it a single line

> 
>> +        if ( !is_hardware_domain(pdev->domain) )
>> +        {
>> +            pci_intx(pdev, false);
>> +            pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
>> +        }
>>      }
>>      else
>>          vpci_msi_arch_disable(msi, pdev);
>> --- a/xen/drivers/vpci/msix.c
>> +++ b/xen/drivers/vpci/msix.c
>> @@ -135,6 +135,13 @@ static void cf_check control_write(
>>          }
>>      }
>>  
>> +    /* Make sure domU doesn't enable INTx while enabling MSI-X. */
>> +    if ( new_enabled && !msix->enabled && !is_hardware_domain(pdev->domain) )
>> +    {
>> +        pci_intx(pdev, false);
>> +        pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
>> +    }
> 
> ... the similar code here has it.
> 
> In both cases, is it really appropriate to set the bit in guest view?

I added this based on Roger's comment at [2]. Roger, what do you think? I don't believe QEMU updates the guest view in this manner.

[2] https://lore.kernel.org/xen-devel/ZLqI65gmNj1XDBm4@MacBook-Air-de-Roger.local/


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 06:57:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 06:57:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674225.1049035 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVR0L-0005dj-Un; Thu, 01 Feb 2024 06:56:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674225.1049035; Thu, 01 Feb 2024 06:56:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVR0L-0005dc-QZ; Thu, 01 Feb 2024 06:56:57 +0000
Received: by outflank-mailman (input) for mailman id 674225;
 Thu, 01 Feb 2024 06:56:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVR0K-0005dQ-Gl; Thu, 01 Feb 2024 06:56:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVR0K-0001UG-9k; Thu, 01 Feb 2024 06:56:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVR0J-0005j4-QC; Thu, 01 Feb 2024 06:56:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVR0J-0004C4-Pb; Thu, 01 Feb 2024 06:56:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=fzfFMqCmBHa+JxmDb2qyVL+C8MkfC3MyFUph87Y9fZY=; b=NauGs1tnS908U0SbmvA4SADTC/
	id/Wu2vVLUaGANUqO47C76Gf818BPr0/wsXkBR1+UbL+HPJtzSH72ycqIF8RMUURqxU6GPpyI8khZ
	FkQyjeNUqFplv4rOVGI8QPj5FqBATOabmSK9Xf4CqZiCX1OlumUty2VXHEpjURllqPi8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184537-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184537: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=2757e91c2b28b704d9a0b586fb60012450110b1a
X-Osstest-Versions-That:
    libvirt=3a3f73ea9f1925ca5e256fa54c5aa451ddeaa19e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 01 Feb 2024 06:56:55 +0000

flight 184537 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184537/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184526
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184526
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184526
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              2757e91c2b28b704d9a0b586fb60012450110b1a
baseline version:
 libvirt              3a3f73ea9f1925ca5e256fa54c5aa451ddeaa19e

Last test of basis   184526  2024-01-30 04:18:50 Z    2 days
Testing same since   184537  2024-01-31 04:24:45 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrea Bolognani <abologna@redhat.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Michal Privoznik <mprivozn@redhat.com>
  Pavel Hrdina <phrdina@redhat.com>
  Purna Pavan Chandra Aekkaladevi <paekkaladevi@linux.microsoft.com>
  김인수 <simmon@nplob.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   3a3f73ea9f..2757e91c2b  2757e91c2b28b704d9a0b586fb60012450110b1a -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 07:39:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 07:39:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674232.1049044 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVRfd-0002hy-55; Thu, 01 Feb 2024 07:39:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674232.1049044; Thu, 01 Feb 2024 07:39:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVRfd-0002hr-2P; Thu, 01 Feb 2024 07:39:37 +0000
Received: by outflank-mailman (input) for mailman id 674232;
 Thu, 01 Feb 2024 07:39:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVRfb-0002hl-Uy
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 07:39:35 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0a8e9613-c0d5-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 08:39:33 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-40e72a567eeso5785675e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 31 Jan 2024 23:39:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 10-20020a05600c22ca00b0040ef8aa4822sm3522786wmg.38.2024.01.31.23.39.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 31 Jan 2024 23:39:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a8e9613-c0d5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706773173; x=1707377973; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=bz5FmuhEJiQexu5mu42jZymD7KqSCxtSIt0Um8+gwT4=;
        b=db6x70WhrqzoW5Ng8Iau5RWM6t9mXmB3xi9kcOmNmjIrGb1Cen8gjHHvzgBcp/TF0H
         P4BS//6BeqqCIHw0tXj55TwHne5wwPup2CthRnmWGVzRBgCGNTKUrdVqFG9viKL8wFNY
         YFGRmo0/OwvdQ34q+QX11+w+MU06hXpvQDxz6IK15iJJSQTgYgcL5PsDqYfuls2fQLuW
         G1AaA920xaPrfQ7w01CHgISagsP71FmkZSsuGCVC4GDiFe/zjiG54eOAzuhw7nGga9y5
         jX8oRgQpWJW6k5lR6STCOij7OUD2bRBMn0uOxHksJtIHEC83P3HDaE+7dXZE+nVvkEqn
         eWXA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706773173; x=1707377973;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bz5FmuhEJiQexu5mu42jZymD7KqSCxtSIt0Um8+gwT4=;
        b=nI5azqFofOsk0vhKLeGGb7ABiRHKji4RqThAGwPNLFikjLag0M0SCOgYbuQsnoWStN
         bZ2zB4Q65uZZsExPWYZo3EuRPwADZ40gE2ws4qLapaqzMYUDRiCezYsuItAfF3gtrLba
         IX1VKbI88q1DWrZ1qNjS85JXUWGz2BGPBzNya0e64bi5GWDJNqG6Mns1vnmJypQtE+vb
         RcKSD6y6/vIYotyvWYYIkVVLmd+0jGgalAVmWOM08GA9XbLYjsPuvC/FxiFqnDSOW5ko
         eHQoqA6T8dYd86Ptmr8PBG0R6vpDKoJ+X5xHAmBNto1mYa60EtIBkJKo2mOMj2PQFost
         +mLQ==
X-Gm-Message-State: AOJu0Yw9JQpl72P5F4aM7aaok3H1ymzt4hmmgVK556tT4e9FgWG2hW04
	t8mswSdSeF2q52E5kQYAoiesDje4AayPc8iJbXNOWd3kTj+w+M/Idi9P21oujXB5LV7LnUlBm0I
	=
X-Google-Smtp-Source: AGHT+IHI+PDZQn2TwuOAWsQO5Xun50aTkX0BHz5ysmzYQWPCRPzhA7d12I0X6nQ4pfD8oCB7K9+R8w==
X-Received: by 2002:a05:600c:538d:b0:40e:66ba:7fff with SMTP id hg13-20020a05600c538d00b0040e66ba7fffmr3798549wmb.19.1706773172939;
        Wed, 31 Jan 2024 23:39:32 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVkJeNTuUFK4VV99ciwxWA8mAtOgklaMsawAgkLQgEbncOZYC7E1zsj0RukpKDSFqGb6ZYlQbFlvq73DZwyvKFeZyqX+aFzsnhD6IGVEYKttQ==
Message-ID: <062053aa-455e-43d6-bc11-e25dc8052878@suse.com>
Date: Thu, 1 Feb 2024 08:39:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] shim: avoid building of vendor IOMMU code
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

There's no use for IOMMU code in the shim. Disable at least the vendor-
specific code, until eventually IOMMU code can be disabled altogether.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/configs/pvshim_defconfig
+++ b/xen/arch/x86/configs/pvshim_defconfig
@@ -21,5 +21,7 @@ CONFIG_EXPERT=y
 # CONFIG_LIVEPATCH is not set
 # CONFIG_SUPPRESS_DUPLICATE_SYMBOL_WARNINGS is not set
 # CONFIG_TRACEBUFFER is not set
+# CONFIG_AMD_IOMMU is not set
+# CONFIG_INTEL_IOMMU is not set
 # CONFIG_DEBUG is not set
 # CONFIG_GDBSX is not set


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 08:15:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 08:15:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674240.1049055 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVSDv-0008P1-3R; Thu, 01 Feb 2024 08:15:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674240.1049055; Thu, 01 Feb 2024 08:15:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVSDv-0008Ou-0O; Thu, 01 Feb 2024 08:15:03 +0000
Received: by outflank-mailman (input) for mailman id 674240;
 Thu, 01 Feb 2024 08:15:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVSDt-0008Oo-EI
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 08:15:01 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fa7cb8c8-c0d9-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 09:14:54 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-40ef64d8955so5840325e9.3
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 00:14:54 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bj26-20020a0560001e1a00b0033ae6530969sm12754030wrb.85.2024.02.01.00.14.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 00:14:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa7cb8c8-c0d9-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706775294; x=1707380094; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=uXX6851uCjtiZLhZz0jMQytjWVPIVDPo0oUudLzGlF4=;
        b=HGFbnaYFM2/SoDxApgOSazElODc2mdLTO6gu/W8FGrRvkrF9UqywRI3rqxHNkpb7e7
         bpa+u8o/ct+nTOdMXi5Huyn8Ix1Dax+9pWGIcnKL5tZtXN7qSRDBLLvGeltFsWemGgIw
         C9yaTSeuLihcBZW+XsNxVdMcBEe34o9pVd93Yt5ieJfRQqiygaplHk6wZSDdKzsWJxgR
         9i2l8aoPz4IeQyBsCnOCaOTITypkrjhFQz1BaF7oGyW8mVcuTBRPYRcFdb6m9T/tT6DO
         1uBSBwAR8M7yAhJ0M1DZ7WOaaDTNpWtHcAmkMh4yLhRFbZKfp8mLrXptrtO/F4kPj1Sg
         r8mQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706775294; x=1707380094;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=uXX6851uCjtiZLhZz0jMQytjWVPIVDPo0oUudLzGlF4=;
        b=dONfjgOhxjArdyKd323HNa8mZR4RuncVvdmfOH1V4gObiK03IUwrLvF3Lm73GMYreH
         C0gtEyAcrdyAn6/c1CCstwsGGJg96mtarTcrPwbuqFNE/fCBIJ7q5IINtTkFqJHHtmN3
         pENBOmXTlLqQW5dq54wWf6vv7hcvI3JSgyzrDf52aEUycjL889FLcQLth+wVlrNSEIrk
         MshujEFNrBGtO0FagieT9d9KhHSZDj9Au5nHIhqLWm0KEwDwfCGJFap6oRInhebfNlQh
         EjtPSBMeK0gEn7TtKevcx/BAbRaTBT8S/NtfQHn3vBvln5Yi0K6suO48zc1m/ZMy3X5n
         RxbA==
X-Gm-Message-State: AOJu0Yyc97C21UXRfYoESoTDliRnnCedT2ruL98gRwW6BeRmGuWxo1x8
	oCyiP6TnlxUgM2S1aEWi+5956l547tAb8ashvjDclB+/fLZY1j+4mqpUfLgFew==
X-Google-Smtp-Source: AGHT+IEkjzxS2zne3m7X6CWeM1TzwLvKuAUxIIpsXH4W91koqNYL/oKEce9niECpnSm7Ha9IyULh2w==
X-Received: by 2002:a05:600c:2943:b0:40f:b456:6877 with SMTP id n3-20020a05600c294300b0040fb4566877mr3106061wmd.18.1706775294283;
        Thu, 01 Feb 2024 00:14:54 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUNg7/nooxIE2+n8r7M2gx0v35/gSb2/6HKOA2x+aPjot/a+czV56G/YcuDy3Y1R5MhjEsIgncVkuM0SseNv73Mo4YtxLX4AtChf+x8fbhBuY8YJg35bbojzmWs2KaCPiXYwjVbLo1SkVnITW9LQru8P5fNII4c1Eq2bYckZwXyZ9Wuds1Vb34AINRE1ORoKcA/dHXtVxz81ddWNN68KDaOwGysrodv7d2nPkP/m9UXy+Gxgu3eU/oN1APyW9p2KHQ6QIIKnPefhYvYzboQ+PxXnnis/6b+6a3evNPMOpgEBAGSPWjD95l7xQ==
Message-ID: <86b25777-788c-4b9a-8166-a6f8174bedc9@suse.com>
Date: Thu, 1 Feb 2024 09:14:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 10/15] vpci/header: emulate PCI_COMMAND register for
 guests
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20240109215145.430207-1-stewart.hildebrand@amd.com>
 <20240109215145.430207-11-stewart.hildebrand@amd.com>
 <50e79338-b2fd-40e9-b754-4e725e27ceb3@suse.com>
 <48a16cb3-9cb9-4b7d-a950-f190a9a64bc2@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <48a16cb3-9cb9-4b7d-a950-f190a9a64bc2@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.02.2024 05:50, Stewart Hildebrand wrote:
> On 1/25/24 10:43, Jan Beulich wrote:
>> On 09.01.2024 22:51, Stewart Hildebrand wrote:
>>> --- a/xen/drivers/vpci/header.c
>>> +++ b/xen/drivers/vpci/header.c
>>> @@ -168,6 +168,9 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
>>>      if ( !rom_only )
>>>      {
>>>          pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
>>> +        /* Show DomU that we updated P2M */
>>> +        header->guest_cmd &= ~PCI_COMMAND_MEMORY;
>>> +        header->guest_cmd |= cmd & PCI_COMMAND_MEMORY;
>>>          header->bars_mapped = map;
>>>      }
>>
>> I don't follow what the comment means to say. The bit in question has no
>> real connection to the P2M, and the guest also may have no notion of the
>> underlying hypervisor's internals. Likely connected to ...
> 
> Indeed. If the comment survives to v13, I'll update it to:
> 
>         /* Now that we updated P2M, show DomU change to PCI_COMMAND_MEMORY */
> 
>>
>>> @@ -524,9 +527,26 @@ static void cf_check cmd_write(
>>>  {
>>>      struct vpci_header *header = data;
>>>  
>>> +    if ( !is_hardware_domain(pdev->domain) )
>>> +    {
>>> +        const struct vpci *vpci = pdev->vpci;
>>> +
>>> +        if ( (vpci->msi && vpci->msi->enabled) ||
>>> +             (vpci->msix && vpci->msix->enabled) )
>>> +            cmd |= PCI_COMMAND_INTX_DISABLE;
>>> +
>>> +        /*
>>> +         * Do not show change to PCI_COMMAND_MEMORY bit until we finish
>>> +         * modifying P2M mappings.
>>> +         */
>>> +        header->guest_cmd = (cmd & ~PCI_COMMAND_MEMORY) |
>>> +                            (header->guest_cmd & PCI_COMMAND_MEMORY);
>>> +    }
>>
>> ... the comment here, but then shouldn't it be that the guest can't even
>> issue a 2nd cfg space access until the present write has been carried out?
>> Otherwise I'd be inclined to claim that such a partial update is unlikely
>> to be spec-conformant.
> 
> Due to the raise_softirq() call added in
> 
>   3e568fa9e19c ("vpci: fix deferral of long operations")
> 
> my current understanding is: when the guest toggles memory decoding, the guest vcpu doesn't resume execution until vpci_process_pending() and modify_decoding() have finished. So I think the guest should see a consistent state of the register, unless it was trying to read from a different vcpu than the one doing the writing.
> 
> Regardless, if the guest did have an opportunity to successfully read the partially updated state of the register, I'm not really spotting what part of the spec that would be a violation of. PCIe 6.1 has this description regarding the bit: "When this bit is Set" and "When this bit is Clear" the device will decode (or not) memory accesses. The spec doesn't seem to distinguish whether the host or the device itself is the one to set/clear the bit. One might even try to argue the opposite: allowing the bit to be toggled before the device reflects the change would be a violation of spec. Since the spec is ambiguous in this regard, I don't think either argument is particularly strong.
> 
> Chesterton's fence: the logic for deferring the update of PCI_COMMAND_MEMORY in guest_cmd was added between v10 and v11 of this series. I went back to look at the review comments on v10 [1], but the rationale is still not entirely clear to me.

Indeed. The only sentence possibly hinting in such a direction would imo
have been "I'm kind of unsure whether we want to fake the guest view by
returning what the guest writes." It's unclear to me whether it really
was meant that way.

> At the end of the day, with the information I have at hand, I suspect it would be fine either way (whether updating guest_cmd is deferred or not). If no other info comes to light, I'm leaning toward not deferring because it would be simpler to update the bit right away in cmd_write().

I'm not sure it would be fine either way. Config space writes aren't
posted writes, so they complete synchronously. IOW whatever internal
state updates are needed in the device, they ought to have finished by
the time the write completes.

> [1] https://lore.kernel.org/xen-devel/ZVy73iJ3E8nJHvgf@macbook.local/
> 
>>[...]
>>> --- a/xen/drivers/vpci/msix.c
>>> +++ b/xen/drivers/vpci/msix.c
>>> @@ -135,6 +135,13 @@ static void cf_check control_write(
>>>          }
>>>      }
>>>  
>>> +    /* Make sure domU doesn't enable INTx while enabling MSI-X. */
>>> +    if ( new_enabled && !msix->enabled && !is_hardware_domain(pdev->domain) )
>>> +    {
>>> +        pci_intx(pdev, false);
>>> +        pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
>>> +    }
>>
>> ... the similar code here has it.
>>
>> In both cases, is it really appropriate to set the bit in guest view?
> 
> I added this based on Roger's comment at [2]. Roger, what do you think? I don't believe QEMU updates the guest view in this manner.
> 
> [2] https://lore.kernel.org/xen-devel/ZLqI65gmNj1XDBm4@MacBook-Air-de-Roger.local/

Leaving this for Roger to answer.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 08:15:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 08:15:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674241.1049065 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVSE7-0000Ed-Ba; Thu, 01 Feb 2024 08:15:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674241.1049065; Thu, 01 Feb 2024 08:15:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVSE7-0000EW-7r; Thu, 01 Feb 2024 08:15:15 +0000
Received: by outflank-mailman (input) for mailman id 674241;
 Thu, 01 Feb 2024 08:15:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cAD+=JK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rVSE5-0008Oo-Pi
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 08:15:13 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 04cf22e6-c0da-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 09:15:11 +0100 (CET)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-51124d43943so908980e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 00:15:12 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 r17-20020a05600c459100b0040e527602c8sm3658249wmo.9.2024.02.01.00.15.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 00:15:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04cf22e6-c0da-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706775311; x=1707380111; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=ftZj8xWkxeuKAOmzAYbWpkD03g2lwYFka9yLaabGBYU=;
        b=fl2m96rzMgAtO6uVBnzWIazZuyEM4Mvjy0Smj8Irzbap8i3/caM+0FjUEeMRgoyCDW
         iduIKdw2FiyhEWuSbC570BF4Cx+547z3o6mb1UJBpWcg2i1pSSy3yGzaouU2dUrEC90t
         vwJ4kBC+Cn6zj+RG39PVFWXC3pPe8cgL6yMZQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706775311; x=1707380111;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ftZj8xWkxeuKAOmzAYbWpkD03g2lwYFka9yLaabGBYU=;
        b=pC/idycX5XQewvJK0xOZ2HX2luKCIHsrAH/y/kHKh82ZAlrkkmvVk6qVlOgKXEABD5
         g7L/5rP0EKmC1YwQyv0DhiIUfdJeLiF8T0X5Ymx3/lPV5FYRSOBT6mQpWz+IYQiNMbAW
         kZPgYr5ykuY0hGN4EoJCdSqNB3JDygVKoxHUcxqQLwhL6By3bkOdWxg5Kd/nNx6M3b/2
         zLZx9Z6tRBytwupGWPBIYI5pxtyE8tHQ4Qbzgoc/sROrB9ESPUShgEhBczf7fAIz0piy
         2WLwEhjYujBfo0QU4sOItoXbhkoGsIV/+FkCc2tQwPmmWC2lxm4j/9KnG2Os2v8uKNsb
         GV3g==
X-Gm-Message-State: AOJu0YwmkdXlTTp5InqrYGXnr5qX29zKQeh/enKQ5+iyluTnBWsuP2Px
	XkTVaUIi3xCUHzSmeDT3RY5mYbCuUEwlViADnZbJL05yec0nUts8ESiQaREUabAjPiI4iL40NrJ
	S
X-Google-Smtp-Source: AGHT+IE8gLuXy+MPg1izu/6bIhKWak2wRqZet0KZ2tIQTi7ixhztfq33TeOjjIOuHvBuVum5CAKS/Q==
X-Received: by 2002:ac2:5fcb:0:b0:511:8a1:4e08 with SMTP id q11-20020ac25fcb000000b0051108a14e08mr1097492lfg.9.1706775311520;
        Thu, 01 Feb 2024 00:15:11 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCU72vY78WMM5Oc05nOMgQHKUahgsw8cAJB74JwC997R/ve+rpdCHw2Iuue/6RSs2TuwxkFyU6X3WUykkvh4w0g/5hUjkUAnJDSwRTmwim84+je9wcTm
Date: Thu, 1 Feb 2024 09:15:10 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] shim: avoid building of vendor IOMMU code
Message-ID: <ZbtTDgXFdaLx6kgx@macbook>
References: <062053aa-455e-43d6-bc11-e25dc8052878@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <062053aa-455e-43d6-bc11-e25dc8052878@suse.com>

On Thu, Feb 01, 2024 at 08:39:31AM +0100, Jan Beulich wrote:
> There's no use for IOMMU code in the shim. Disable at least the vendor-
> specific code, until eventually IOMMU code can be disabled altogether.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

That was easier than I expected :).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 08:21:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 08:21:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674246.1049075 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVSJu-0002OP-1i; Thu, 01 Feb 2024 08:21:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674246.1049075; Thu, 01 Feb 2024 08:21:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVSJt-0002OI-V7; Thu, 01 Feb 2024 08:21:13 +0000
Received: by outflank-mailman (input) for mailman id 674246;
 Thu, 01 Feb 2024 08:16:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Ox/P=JK=proton.me=oxjo@srs-se1.protection.inumbo.net>)
 id 1rVSF6-0000xF-3n
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 08:16:17 +0000
Received: from mail-40141.protonmail.ch (mail-40141.protonmail.ch
 [185.70.40.141]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 293a3220-c0da-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 09:16:12 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 293a3220-c0da-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me;
	s=oat2jhs3tfeuzhvy2ui5tmazwq.protonmail; t=1706775371; x=1707034571;
	bh=m5mnweclGAWQKNx6iCQLtjWxNRC7Y1HIkiDeF/KR1/M=;
	h=Date:To:From:Cc:Subject:Message-ID:Feedback-ID:From:To:Cc:Date:
	 Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector;
	b=GDQn/oHiMp5ogEXCNNEtTx9oWPyUfm9ZE3zaxJlI9up0b62tjRYzzzoRmM/tXHKLa
	 AFEahWDJwSLIw0BBScC/DEt9ddcwFXI6B2KmvtYrXQ/2n9mG7BRitPEwxp6CRufEg/
	 Mr8HEl93tmo6V30acpctzIp/ueDvIFK9vaHxE3XPro00I0D4Y+Sxkh4IooOVYLZIG0
	 GhrmeYqmQ0HFa2O6mgUiriX81gctmWWpYVdwvGbQTLGBOJ9m5r2qoPoEd8alBikOZv
	 C2H/nqesk62SQsucJooGgjudIB32BwxRhGfECHLBW7e51bWOXdB1rw5xT+f+ojFnim
	 DbcO/B1jY5Q9Q==
Date: Thu, 01 Feb 2024 08:16:05 +0000
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: oxjo@proton.me
Cc: "roger.pau@citrix.com" <roger.pau@citrix.com>
Subject: Issue iommu unrecognized on amd computer
Message-ID: <qTUd8Z_TU2_jJzz9kTj2DJK54p4wECUT072VD_1nomh-d8Ej6hC-QNzY6BW6G9bQVdcDJuQbapJ7NlqqOAKd49Iu8rgkyt4E9qdnN_SyZj8=@proton.me>
Feedback-ID: 99814252:user:proton
MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="b1_ahePdPP8ZPOBcsnbMP4jUdSkh5Jox7jBC4XyO2W8"

This is a multi-part message in MIME format.

--b1_ahePdPP8ZPOBcsnbMP4jUdSkh5Jox7jBC4XyO2W8
Content-Type: multipart/alternative;
 boundary="b2_ahePdPP8ZPOBcsnbMP4jUdSkh5Jox7jBC4XyO2W8"

--b2_ahePdPP8ZPOBcsnbMP4jUdSkh5Jox7jBC4XyO2W8
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: base64

SGksCgpGb2xsb3dpbmcgb3VyIGludGVyYWN0aW9uIG9uIG1hdHJpeCwgSSBzZW5kIHlvdSB0aGUg
Ym9vdCBsb2cgYHhsIGRtZXNnYCBvdXRwdXQgb2YgbXkgY29tcHV0ZXIsIGJvb3Rpbmcgd2l0aCBg
aW9tbXU9ZGVidWdgIG9wdGlvbnMgb24gbGludXggNi42LjEzLTIwMC5mYzM5LgoKTXkgaXNzdWUg
aXMgZXZlcnl0aGluZyB3b3JrcyBvbiBxdWJlcyBvbiBteSB0aGlua3BhZCBMMTUgZ2VuIDQgd2l0
aCBBTUQgUnl6ZW4gNyBQcm8gNzczMFUsIGV4Y2VwdCBpb21tdS9hbWQtdmkuCkkgY2FuIGJvb3Qg
YW5kIHVzZSBhIHZtIGJ1dCBubyBwY2kgcGFzc3Rocm91Z2guCgpJIHRyaWVkIHF1YmVzIDQuMiwg
YmFyZSB4ZW4gMTcuMiwgeGVuIDE4IGFuZCBJIGFsd2F5cyBoYXZlIHRoZSBzYW1lIG1lc3NhZ2Vz
IHlvdSBjYW4gc2VlIGluIHRoZSBsb2dzIHdpdGggSVZNRC4KCkknZCBiZSBncmF0ZWZ1bCBpZiB5
b3UgY2FuIHRlbGwgbWUgd2l0aGluIGEgd2VlayBpZiBpdCdzIGFuIGlzc3VlIHlvdSB0aGluayBj
YW4gYmUgZGVhbHQgd2l0aCBvciBubyBzbyBJIGNhbiBkZWNpZGUgd2hhdCBJIGRvIHdpdGggdGhp
cyBjb21wdXRlci4KCkJlc3QgcmVnYXJkcywKT3g=

--b2_ahePdPP8ZPOBcsnbMP4jUdSkh5Jox7jBC4XyO2W8
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64

PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0
cHg7Ij5IaSw8L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7
IGZvbnQtc2l6ZTogMTRweDsiPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJp
YWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsiPkZvbGxvd2luZyBvdXIgaW50ZXJhY3Rp
b24gb24gbWF0cml4LCBJIHNlbmQgeW91IHRoZSBib290IGxvZyBgeGwgZG1lc2dgIG91dHB1dCBv
ZiBteSBjb21wdXRlciwgYm9vdGluZyB3aXRoIGBpb21tdT1kZWJ1Z2Agb3B0aW9ucyBvbiBsaW51
eCA2LjYuMTMtMjAwLmZjMzkuPC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBz
YW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1m
YW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7Ij5NeSBpc3N1ZSBpcyBl
dmVyeXRoaW5nIHdvcmtzIG9uIHF1YmVzIG9uIG15IHRoaW5rcGFkIEwxNSBnZW4gNCB3aXRoIEFN
RCBSeXplbiA3Jm5ic3A7IFBybyA3NzMwVSwgZXhjZXB0IGlvbW11L2FtZC12aS48L2Rpdj48ZGl2
IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsi
PkkgY2FuIGJvb3QgYW5kIHVzZSBhIHZtIGJ1dCBubyBwY2kgcGFzc3Rocm91Z2guPC9kaXY+PGRp
diBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7
Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBm
b250LXNpemU6IDE0cHg7Ij5JIHRyaWVkIHF1YmVzIDQuMiwgYmFyZSB4ZW4gMTcuMiwgeGVuIDE4
IGFuZCBJIGFsd2F5cyBoYXZlIHRoZSBzYW1lIG1lc3NhZ2VzIHlvdSBjYW4gc2VlIGluIHRoZSBs
b2dzIHdpdGggSVZNRC48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMt
c2VyaWY7IGZvbnQtc2l6ZTogMTRweDsiPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWls
eTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsiPkknZCBiZSBncmF0ZWZ1bCBp
ZiB5b3UgY2FuIHRlbGwgbWUgd2l0aGluIGEgd2VlayBpZiBpdCdzIGFuIGlzc3VlIHlvdSB0aGlu
ayBjYW4gYmUgZGVhbHQgd2l0aCBvciBubyBzbyBJIGNhbiBkZWNpZGUgd2hhdCBJIGRvIHdpdGgg
dGhpcyBjb21wdXRlci48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMt
c2VyaWY7IGZvbnQtc2l6ZTogMTRweDsiPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWls
eTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsiPkJlc3QgcmVnYXJkcyw8L2Rp
dj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTog
MTRweDsiPk94PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlm
OyBmb250LXNpemU6IDE0cHg7Ij48YnI+PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFy
aWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7Ij48YnI+PC9kaXY+DQo=


--b2_ahePdPP8ZPOBcsnbMP4jUdSkh5Jox7jBC4XyO2W8--

--b1_ahePdPP8ZPOBcsnbMP4jUdSkh5Jox7jBC4XyO2W8
Content-Type: text/plain; name=boot.log
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=boot.log

IFhlbiA0LjE4LjAKKFhFTikgWGVuIHZlcnNpb24gNC4xOC4wIChtb2NrYnVpbGRAKSAoZ2NjIChH
Q0MpIDEzLjIuMSAyMDIzMTIwNSAoUmVkIEhhdCAxMy4yLjEtNikpIGRlYnVnPW4gV2VkIERlYyAx
MyAxNzo1MDo0MiBVVEMgMjAyMwooWEVOKSBMYXRlc3QgQ2hhbmdlU2V0OiAKKFhFTikgYnVpbGQt
aWQ6IGVhNWQxOTYwYThkMDZmMzFiMDAwMzU1ZjQ2NjIxNjlmZTEyZDA5YTYKKFhFTikgQm9vdGxv
YWRlcjogR1JVQiAyLjA2CihYRU4pIENvbW1hbmQgbGluZTogcGxhY2Vob2xkZXIgbm8tcmVhbC1t
b2RlIGVkZD1vZmYKKFhFTikgWGVuIGltYWdlIGxvYWQgYmFzZSBhZGRyZXNzOiAweGMxYTAwMDAw
CihYRU4pIFZpZGVvIGluZm9ybWF0aW9uOgooWEVOKSAgVkdBIGlzIGdyYXBoaWNzIG1vZGUgMTky
MHgxMDgwLCAzMiBicHAKKFhFTikgIFZCRS9EREMgbWV0aG9kczogbm9uZTsgRURJRCB0cmFuc2Zl
ciB0aW1lOiAwIHNlY29uZHMKKFhFTikgRGlzYyBpbmZvcm1hdGlvbjoKKFhFTikgIEZvdW5kIDAg
TUJSIHNpZ25hdHVyZXMKKFhFTikgIEZvdW5kIDEgRUREIGluZm9ybWF0aW9uIHN0cnVjdHVyZXMK
KFhFTikgQ1BVIFZlbmRvcjogQU1ELCBGYW1pbHkgMjUgKDB4MTkpLCBNb2RlbCA4MCAoMHg1MCks
IFN0ZXBwaW5nIDAgKHJhdyAwMGE1MGYwMCkKKFhFTikgRW5hYmxpbmcgU3VwZXJ2aXNvciBTaGFk
b3cgU3RhY2tzCihYRU4pICAgLSBEaXNhYmxpbmcgUFYzMiBkdWUgdG8gQ0VUCihYRU4pIEVGSSBS
QU0gbWFwOgooWEVOKSAgWzAwMDAwMDAwMDAwMDAwMDAsIDAwMDAwMDAwMDAwOWVmZmZdICh1c2Fi
bGUpCihYRU4pICBbMDAwMDAwMDAwMDA5ZjAwMCwgMDAwMDAwMDAwMDA5ZmZmZl0gKHJlc2VydmVk
KQooWEVOKSAgWzAwMDAwMDAwMDAxMDAwMDAsIDAwMDAwMDAwMDliZmZmZmZdICh1c2FibGUpCihY
RU4pICBbMDAwMDAwMDAwOWMwMDAwMCwgMDAwMDAwMDAwOWRiMGZmZl0gKHJlc2VydmVkKQooWEVO
KSAgWzAwMDAwMDAwMDlkYjEwMDAsIDAwMDAwMDAwMDllZmZmZmZdICh1c2FibGUpCihYRU4pICBb
MDAwMDAwMDAwOWYwMDAwMCwgMDAwMDAwMDAwOWYwZWZmZl0gKEFDUEkgTlZTKQooWEVOKSAgWzAw
MDAwMDAwMDlmMGYwMDAsIDAwMDAwMDAwYzJkNTdmZmZdICh1c2FibGUpCihYRU4pICBbMDAwMDAw
MDBjMmQ1ODAwMCwgMDAwMDAwMDBjOGY1N2ZmZl0gKHJlc2VydmVkKQooWEVOKSAgWzAwMDAwMDAw
YzhmNTgwMDAsIDAwMDAwMDAwYzlmNTdmZmZdIChBQ1BJIE5WUykKKFhFTikgIFswMDAwMDAwMGM5
ZjU4MDAwLCAwMDAwMDAwMGM5ZmQ3ZmZmXSAoQUNQSSBkYXRhKQooWEVOKSAgWzAwMDAwMDAwYzlm
ZDgwMDAsIDAwMDAwMDAwYzlmZDlmZmZdICh1c2FibGUpCihYRU4pICBbMDAwMDAwMDBjOWZkYTAw
MCwgMDAwMDAwMDBjOWZmZmZmZl0gKHJlc2VydmVkKQooWEVOKSAgWzAwMDAwMDAwY2EwMDAwMDAs
IDAwMDAwMDAwY2JmZmZmZmZdICh1c2FibGUpCihYRU4pICBbMDAwMDAwMDBjYzAwMDAwMCwgMDAw
MDAwMDBjZGZmZmZmZl0gKHJlc2VydmVkKQooWEVOKSAgWzAwMDAwMDAwY2YwMDAwMDAsIDAwMDAw
MDAwY2ZmZmZmZmZdIChyZXNlcnZlZCkKKFhFTikgIFswMDAwMDAwMGY4MDAwMDAwLCAwMDAwMDAw
MGZiZmZmZmZmXSAocmVzZXJ2ZWQpCihYRU4pICBbMDAwMDAwMDBmZGMwMDAwMCwgMDAwMDAwMDBm
ZGNmZmZmZl0gKHJlc2VydmVkKQooWEVOKSAgWzAwMDAwMDAwZmVkODAwMDAsIDAwMDAwMDAwZmVk
ODBmZmZdIChyZXNlcnZlZCkKKFhFTikgIFswMDAwMDAwMTAwMDAwMDAwLCAwMDAwMDAwZmVlMmZm
ZmZmXSAodXNhYmxlKQooWEVOKSAgWzAwMDAwMDBmZWUzMDAwMDAsIDAwMDAwMDEwMmZmZmZmZmZd
IChyZXNlcnZlZCkKKFhFTikgQUNQSTogUlNEUCBDOUZENzAxNCwgMDAyNCAocjIgTEVOT1ZPKQoo
WEVOKSBBQ1BJOiBYU0RUIEM5RkQ1MTg4LCAwMTA0IChyMSBMRU5PVk8gVFAtUjI1ICAgICAgIDEx
MzAgUFRFQyAgICAgICAgMikKKFhFTikgQUNQSTogRkFDUCBDMzk2NDAwMCwgMDExNCAocjYgTEVO
T1ZPIFRQLVIyNSAgICAgICAxMTMwIFBURUMgICAgICAgIDIpCihYRU4pIEFDUEk6IERTRFQgQzM5
NEMwMDAsIDEyNzkxIChyMSBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgSU5UTCAyMDE4MDMxMykK
KFhFTikgQUNQSTogRkFDUyBDOURGODAwMCwgMDA0MAooWEVOKSBBQ1BJOiBTU0RUIEM1RTc0MDAw
LCAwMEEyIChyMSBMRU5PVk8gUElEMFNzZHQgICAgICAgIDEgSU5UTCAyMDE4MDMxMykKKFhFTikg
QUNQSTogU1NEVCBDNUU3MzAwMCwgMDc1MiAocjEgTEVOT1ZPIFVzYkNUYWJsICAgICAgICAxIElO
VEwgMjAxODAzMTMpCihYRU4pIEFDUEk6IFNTRFQgQzVFNjYwMDAsIDczNDUgKHIyIExFTk9WTyBU
UC1SMjUgICAgICAgICAgMiBNU0ZUICAyMDAwMDAyKQooWEVOKSBBQ1BJOiBNU0RNIEM1QjdCMDAw
LCAwMDU1IChyMyBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgUFRFQyAgICAgICAgMikKKFhFTikg
QUNQSTogQkFUQiBDNUI2NjAwMCwgMDA0QSAocjIgTEVOT1ZPIFRQLVIyNSAgICAgICAxMTMwIFBU
RUMgICAgICAgIDIpCihYRU4pIEFDUEk6IEhQRVQgQzM5NjMwMDAsIDAwMzggKHIxIExFTk9WTyBU
UC1SMjUgICAgICAgMTEzMCBQVEVDICAgICAgICAyKQooWEVOKSBBQ1BJOiBBUElDIEMzOTYyMDAw
LCAwMTM4IChyMiBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgUFRFQyAgICAgICAgMikKKFhFTikg
QUNQSTogTUNGRyBDMzk2MTAwMCwgMDAzQyAocjEgTEVOT1ZPIFRQLVIyNSAgICAgICAxMTMwIFBU
RUMgICAgICAgIDIpCihYRU4pIEFDUEk6IFNCU1QgQzM5NjAwMDAsIDAwMzAgKHIxIExFTk9WTyBU
UC1SMjUgICAgICAgMTEzMCBQVEVDICAgICAgICAyKQooWEVOKSBBQ1BJOiBXU01UIEMzOTVGMDAw
LCAwMDI4IChyMSBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgUFRFQyAgICAgICAgMikKKFhFTikg
QUNQSTogVkZDVCBDMzkzRTAwMCwgRDg4NCAocjEgTEVOT1ZPIFRQLVIyNSAgICAgICAxMTMwIFBU
RUMgICAgICAgIDIpCihYRU4pIEFDUEk6IFNTRFQgQzM5MzgwMDAsIDUzNTQgKHIyIExFTk9WTyBU
UC1SMjUgICAgICAgICAgMSBBTUQgICAgICAgICAxKQooWEVOKSBBQ1BJOiBDUkFUIEMzOTM3MDAw
LCAwRUMwIChyMSBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgUFRFQyAgICAgICAgMikKKFhFTikg
QUNQSTogQ0RJVCBDMzkzNjAwMCwgMDAyOSAocjEgTEVOT1ZPIFRQLVIyNSAgICAgICAxMTMwIFBU
RUMgICAgICAgIDIpCihYRU4pIEFDUEk6IEZQRFQgQzVCNjcwMDAsIDAwMzQgKHIxIExFTk9WTyBU
UC1SMjUgICAgICAgMTEzMCBQVEVDICAgICAgICAyKQooWEVOKSBBQ1BJOiBTU0RUIEMzOTM1MDAw
LCAwMTQ5IChyMSBMRU5PVk8gVFAtUjI1ICAgICAgICAgIDEgSU5UTCAyMDE4MDMxMykKKFhFTikg
QUNQSTogU1NEVCBDMzkzMzAwMCwgMTRDMyAocjEgTEVOT1ZPIFRQLVIyNSAgICAgICAgICAxIElO
VEwgMjAxODAzMTMpCihYRU4pIEFDUEk6IFNTRFQgQzM5MzEwMDAsIDE1QTggKHIxIExFTk9WTyBU
UC1SMjUgICAgICAgICAgMSBJTlRMIDIwMTgwMzEzKQooWEVOKSBBQ1BJOiBTU0RUIEMzOTJEMDAw
LCAzQjBFIChyMSBMRU5PVk8gVFAtUjI1ICAgICAgICAgIDEgSU5UTCAyMDE4MDMxMykKKFhFTikg
QUNQSTogQkdSVCBDMzkyQzAwMCwgMDAzOCAocjEgTEVOT1ZPIFRQLVIyNSAgICAgICAxMTMwIFBU
RUMgICAgICAgIDIpCihYRU4pIEFDUEk6IFNTRFQgQzM5MkIwMDAsIDAyNEQgKHIxIExFTk9WTyBU
UC1SMjUgICAgICAgICAgMSBJTlRMIDIwMTgwMzEzKQooWEVOKSBBQ1BJOiBTU0RUIEMzOTI5MDAw
LCAxNEM0IChyMSBMRU5PVk8gVFAtUjI1ICAgICAgICAgIDEgSU5UTCAyMDE4MDMxMykKKFhFTikg
QUNQSTogU1NEVCBDMzkyODAwMCwgMEFCNyAocjEgTEVOT1ZPIFRQLVIyNSAgICAgICAgICAxIElO
VEwgMjAxODAzMTMpCihYRU4pIEFDUEk6IFVFRkkgQzlERjcwMDAsIDAwQzYgKHIxIExFTk9WTyBU
UC1SMjUgICAgICAgMTEzMCBQVEVDICAgICAgICAyKQooWEVOKSBBQ1BJOiBJVlJTIEMzOTI3MDAw
LCAwMUU0IChyMiBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgUFRFQyAgICAgICAgMikKKFhFTikg
QUNQSTogU1NEVCBDNUU3MjAwMCwgMDA5MCAocjEgTEVOT1ZPIFRQLVIyNSAgICAgICAgICAxIElO
VEwgMjAxODAzMTMpCihYRU4pIEFDUEk6IFNTRFQgQzVFNzEwMDAsIDA5OEQgKHIxIExFTk9WTyBU
UC1SMjUgICAgICAgICAgMSBJTlRMIDIwMTgwMzEzKQooWEVOKSBTeXN0ZW0gUkFNOiA2NDMwMk1C
ICg2NTg0NTQ3NmtCKQooWEVOKSBObyBOVU1BIGNvbmZpZ3VyYXRpb24gZm91bmQKKFhFTikgRmFr
aW5nIGEgbm9kZSBhdCAwMDAwMDAwMDAwMDAwMDAwLTAwMDAwMDBmZWUzMDAwMDAKKFhFTikgRG9t
YWluIGhlYXAgaW5pdGlhbGlzZWQKKFhFTikgdmVzYWZiOiBmcmFtZWJ1ZmZlciBhdCAweDAwMDAw
MDAwZDAwMDAwMDAsIG1hcHBlZCB0byAweGZmZmY4MmMwMDAyMDEwMDAsIHVzaW5nIDgxMjhrLCB0
b3RhbCA4MTI4awooWEVOKSB2ZXNhZmI6IG1vZGUgaXMgMTkyMHgxMDgweDMyLCBsaW5lbGVuZ3Ro
PTc2ODAsIGZvbnQgOHgxNgooWEVOKSB2ZXNhZmI6IFRydWVjb2xvcjogc2l6ZT04Ojg6ODo4LCBz
aGlmdD0yNDoxNjo4OjAKKFhFTikgU01CSU9TIDMuMyBwcmVzZW50LgooWEVOKSBVc2luZyBBUElD
IGRyaXZlciBkZWZhdWx0CihYRU4pIEFDUEk6IFBNLVRpbWVyIElPIFBvcnQ6IDB4NDA4ICgzMiBi
aXRzKQooWEVOKSBBQ1BJOiB2NSBTTEVFUCBJTkZPOiBjb250cm9sWzE6MF0sIHN0YXR1c1sxOjBd
CihYRU4pIEFDUEk6IFNMRUVQIElORk86IHBtMXhfY250WzE6NDA0LDE6MF0sIHBtMXhfZXZ0WzE6
NDAwLDE6MF0KKFhFTikgQUNQSTogMzIvNjRYIEZBQ1MgYWRkcmVzcyBtaXNtYXRjaCBpbiBGQURU
IC0gYzlkZjgwMDAvMDAwMDAwMDAwMDAwMDAwMCwgdXNpbmcgMzIKKFhFTikgQUNQSTogICAgICAg
ICAgICAgd2FrZXVwX3ZlY1tjOWRmODAwY10sIHZlY19zaXplWzIwXQooWEVOKSBPdmVycmlkaW5n
IEFQSUMgZHJpdmVyIHdpdGggYmlnc21wCihYRU4pIEFDUEk6IElPQVBJQyAoaWRbMHgyMF0gYWRk
cmVzc1sweGZlYzAwMDAwXSBnc2lfYmFzZVswXSkKKFhFTikgSU9BUElDWzBdOiBhcGljX2lkIDMy
LCB2ZXJzaW9uIDMzLCBhZGRyZXNzIDB4ZmVjMDAwMDAsIEdTSSAwLTIzCihYRU4pIEFDUEk6IElP
QVBJQyAoaWRbMHgyMV0gYWRkcmVzc1sweGZlYzAxMDAwXSBnc2lfYmFzZVsyNF0pCihYRU4pIElP
QVBJQ1sxXTogYXBpY19pZCAzMywgdmVyc2lvbiAzMywgYWRkcmVzcyAweGZlYzAxMDAwLCBHU0kg
MjQtNTUKKFhFTikgQUNQSTogSU5UX1NSQ19PVlIgKGJ1cyAwIGJ1c19pcnEgMCBnbG9iYWxfaXJx
IDIgZGZsIGRmbCkKKFhFTikgQUNQSTogSU5UX1NSQ19PVlIgKGJ1cyAwIGJ1c19pcnEgOSBnbG9i
YWxfaXJxIDkgbG93IGxldmVsKQooWEVOKSBBQ1BJOiBIUEVUIGlkOiAweDQzNTM4MjEwIGJhc2U6
IDB4ZmVkMDAwMDAKKFhFTikgUENJOiBNQ0ZHIGNvbmZpZ3VyYXRpb24gMDogYmFzZSBmODAwMDAw
MCBzZWdtZW50IDAwMDAgYnVzZXMgMDAgLSAzZgooWEVOKSBQQ0k6IE1DRkcgYXJlYSBhdCBmODAw
MDAwMCByZXNlcnZlZCBpbiBFODIwCihYRU4pIFBDSTogVXNpbmcgTUNGRyBmb3Igc2VnbWVudCAw
MDAwIGJ1cyAwMC0zZgooWEVOKSBBQ1BJOiBCR1JUOiBpbnZhbGlkYXRpbmcgdjEgaW1hZ2UgYXQg
MHhiZDY1NjAxOAooWEVOKSBVc2luZyBBQ1BJIChNQURUKSBmb3IgU01QIGNvbmZpZ3VyYXRpb24g
aW5mb3JtYXRpb24KKFhFTikgU01QOiBBbGxvd2luZyAxNiBDUFVzICgwIGhvdHBsdWcgQ1BVcykK
KFhFTikgSVJRIGxpbWl0czogNTYgR1NJLCAzMjcyIE1TSS9NU0ktWAooWEVOKSBDUFUwOiAxNjAw
IC4uLiAyMDAwIE1IegooWEVOKSB4c3RhdGU6IHNpemU6IDB4OTg4IGFuZCBzdGF0ZXM6IDB4MjA3
CihYRU4pIENQVTA6IEFNRCBGYW0xOWggbWFjaGluZSBjaGVjayByZXBvcnRpbmcgZW5hYmxlZAoo
WEVOKSBWdWxuZXJhYmxlIHRvIFNSU08sIHdpdGhvdXQgc3VpdGFibGUgbWljcm9jb2RlIHRvIG1p
dGlnYXRlCihYRU4pIFNwZWN1bGF0aXZlIG1pdGlnYXRpb24gZmFjaWxpdGllczoKKFhFTikgICBI
YXJkd2FyZSBoaW50czogU1RJQlBfQUxXQVlTIElCUlNfRkFTVCBJQlJTX1NBTUVfTU9ERQooWEVO
KSAgIEhhcmR3YXJlIGZlYXR1cmVzOiBJQlBCIElCUlMgU1RJQlAgU1NCRCBQU0ZECihYRU4pICAg
Q29tcGlsZWQtaW4gc3VwcG9ydDogSU5ESVJFQ1RfVEhVTksgU0hBRE9XX1BBR0lORwooWEVOKSAg
IFhlbiBzZXR0aW5nczogQlRJLVRodW5rIEpNUCwgU1BFQ19DVFJMOiBJQlJTKyBTVElCUCsgU1NC
RC0gUFNGRC0sIE90aGVyOiBJQlBCLWN0eHQgQlJBTkNIX0hBUkRFTgooWEVOKSAgIFN1cHBvcnQg
Zm9yIEhWTSBWTXM6IE1TUl9TUEVDX0NUUkwgTVNSX1ZJUlRfU1BFQ19DVFJMIFJTQgooWEVOKSAg
IFN1cHBvcnQgZm9yIFBWIFZNczogTm9uZQooWEVOKSAgIFhQVEkgKDY0LWJpdCBQViBvbmx5KTog
RG9tMCBkaXNhYmxlZCwgRG9tVSBkaXNhYmxlZCAod2l0aG91dCBQQ0lEKQooWEVOKSAgIFBWIEwx
VEYgc2hhZG93aW5nOiBEb20wIGRpc2FibGVkLCBEb21VIGRpc2FibGVkCihYRU4pIFVzaW5nIHNj
aGVkdWxlcjogU01QIENyZWRpdCBTY2hlZHVsZXIgcmV2MiAoY3JlZGl0MikKKFhFTikgSW5pdGlh
bGl6aW5nIENyZWRpdDIgc2NoZWR1bGVyCihYRU4pICBsb2FkX3ByZWNpc2lvbl9zaGlmdDogMTgK
KFhFTikgIGxvYWRfd2luZG93X3NoaWZ0OiAzMAooWEVOKSAgdW5kZXJsb2FkX2JhbGFuY2VfdG9s
ZXJhbmNlOiAwCihYRU4pICBvdmVybG9hZF9iYWxhbmNlX3RvbGVyYW5jZTogLTMKKFhFTikgIHJ1
bnF1ZXVlcyBhcnJhbmdlbWVudDogc29ja2V0CihYRU4pICBjYXAgZW5mb3JjZW1lbnQgZ3JhbnVs
YXJpdHk6IDEwbXMKKFhFTikgbG9hZCB0cmFja2luZyB3aW5kb3cgbGVuZ3RoIDEwNzM3NDE4MjQg
bnMKKFhFTikgUGxhdGZvcm0gdGltZXIgaXMgMTQuMzE4TUh6IEhQRVQKKFhFTikgRGV0ZWN0ZWQg
MTk5Ni4yNTkgTUh6IHByb2Nlc3Nvci4KKFhFTikgRnJlZWQgMTAyMGtCIHVudXNlZCBCU1MgbWVt
b3J5CihYRU4pIGFsdCB0YWJsZSBmZmZmODJkMDQwNDViZDk4IC0+IGZmZmY4MmQwNDA0NjkyYTgK
KFhFTikgQU1ELVZpOiBJT01NVSBFeHRlbmRlZCBGZWF0dXJlczoKKFhFTikgLSBQZXJpcGhlcmFs
IFBhZ2UgU2VydmljZSBSZXF1ZXN0CihYRU4pIC0geDJBUElDCihYRU4pIC0gTlggYml0CihYRU4p
IC0gSW52YWxpZGF0ZSBBbGwgQ29tbWFuZAooWEVOKSAtIEd1ZXN0IEFQSUMKKFhFTikgLSBQZXJm
b3JtYW5jZSBDb3VudGVycwooWEVOKSAtIEhvc3QgQWRkcmVzcyBUcmFuc2xhdGlvbiBTaXplOiAw
eDIKKFhFTikgLSBHdWVzdCBBZGRyZXNzIFRyYW5zbGF0aW9uIFNpemU6IDAKKFhFTikgLSBHdWVz
dCBDUjMgUm9vdCBUYWJsZSBMZXZlbDogMHgxCihYRU4pIC0gTWF4aW11bSBQQVNJRDogMHhmCihY
RU4pIC0gU01JIEZpbHRlciBSZWdpc3RlcjogMHgxCihYRU4pIC0gU01JIEZpbHRlciBSZWdpc3Rl
ciBDb3VudDogMHgxCihYRU4pIC0gR3Vlc3QgVmlydHVhbCBBUElDIE1vZGVzOiAweDEKKFhFTikg
LSBEdWFsIFBQUiBMb2c6IDB4MgooWEVOKSAtIER1YWwgRXZlbnQgTG9nOiAweDIKKFhFTikgLSBV
c2VyIC8gU3VwZXJ2aXNvciBQYWdlIFByb3RlY3Rpb24KKFhFTikgLSBEZXZpY2UgVGFibGUgU2Vn
bWVudGF0aW9uOiAweDMKKFhFTikgLSBQUFIgTG9nIE92ZXJmbG93IEVhcmx5IFdhcm5pbmcKKFhF
TikgLSBQUFIgQXV0b21hdGljIFJlc3BvbnNlCihYRU4pIC0gTWVtb3J5IEFjY2VzcyBSb3V0aW5n
IGFuZCBDb250cm9sOiAwCihYRU4pIC0gQmxvY2sgU3RvcE1hcmsgTWVzc2FnZQooWEVOKSAtIFBl
cmZvcm1hbmNlIE9wdGltaXphdGlvbgooWEVOKSAtIE1TSSBDYXBhYmlsaXR5IE1NSU8gQWNjZXNz
CihYRU4pIC0gR3Vlc3QgSS9PIFByb3RlY3Rpb24KKFhFTikgLSBFbmhhbmNlZCBQUFIgSGFuZGxp
bmcKKFhFTikgLSBBdHRyaWJ1dGUgRm9yd2FyZAooWEVOKSAtIEludmFsaWRhdGUgSU9UTEIgVHlw
ZQooWEVOKSAtIFZNIFRhYmxlIFNpemU6IDAKKFhFTikgLSBHdWVzdCBBY2Nlc3MgQml0IFVwZGF0
ZSBEaXNhYmxlCihYRU4pIEFNRC1WaTogV2FybmluZzogSVZNRDogW2M5ZjFjMDAwLGM5ZjQyMDAw
KSBpcyBub3QgKGVudGlyZWx5KSBpbiByZXNlcnZlZCBtZW1vcnkKKFhFTikgQU1ELVZpOiBFcnJv
cjogSVZNRDogcGFnZSBhdCBjOWYxYzAwMCBjYW4ndCBiZSBjb252ZXJ0ZWQKKFhFTikgQU1ELVZp
OiBFcnJvciBpbml0aWFsaXphdGlvbgooWEVOKSBJL08gdmlydHVhbGlzYXRpb24gZGlzYWJsZWQK
KFhFTikgRW5hYmxpbmcgQVBJQyBtb2RlOiAgUGh5c2ljYWwuICBVc2luZyAyIEkvTyBBUElDcwoo
WEVOKSBFTkFCTElORyBJTy1BUElDIElSUXMKKFhFTikgIC0+IFVzaW5nIG5ldyBBQ0sgbWV0aG9k
CihYRU4pIC4uVElNRVI6IHZlY3Rvcj0weEYwIGFwaWMxPTAgcGluMT0yIGFwaWMyPS0xIHBpbjI9
LTEKKFhFTikgQWxsb2NhdGVkIGNvbnNvbGUgcmluZyBvZiA2NCBLaUIuCihYRU4pIEhWTTogQVNJ
RHMgZW5hYmxlZC4KKFhFTikgU1ZNOiBTdXBwb3J0ZWQgYWR2YW5jZWQgZmVhdHVyZXM6CihYRU4p
ICAtIE5lc3RlZCBQYWdlIFRhYmxlcyAoTlBUKQooWEVOKSAgLSBMYXN0IEJyYW5jaCBSZWNvcmQg
KExCUikgVmlydHVhbGlzYXRpb24KKFhFTikgIC0gTmV4dC1SSVAgU2F2ZWQgb24gI1ZNRVhJVAoo
WEVOKSAgLSBWTUNCIENsZWFuIEJpdHMKKFhFTikgIC0gRGVjb2RlQXNzaXN0cwooWEVOKSAgLSBW
aXJ0dWFsIFZNTE9BRC9WTVNBVkUKKFhFTikgIC0gVmlydHVhbCBHSUYKKFhFTikgIC0gUGF1c2Ut
SW50ZXJjZXB0IEZpbHRlcgooWEVOKSAgLSBQYXVzZS1JbnRlcmNlcHQgRmlsdGVyIFRocmVzaG9s
ZAooWEVOKSAgLSBUU0MgUmF0ZSBNU1IKKFhFTikgIC0gTlBUIFN1cGVydmlzb3IgU2hhZG93IFN0
YWNrCihYRU4pICAtIE1TUl9TUEVDX0NUUkwgdmlydHVhbGlzYXRpb24KKFhFTikgSFZNOiBTVk0g
ZW5hYmxlZAooWEVOKSBIVk06IEhhcmR3YXJlIEFzc2lzdGVkIFBhZ2luZyAoSEFQKSBkZXRlY3Rl
ZAooWEVOKSBIVk06IEhBUCBwYWdlIHNpemVzOiA0a0IsIDJNQiwgMUdCCihYRU4pIGFsdCB0YWJs
ZSBmZmZmODJkMDQwNDViZDk4IC0+IGZmZmY4MmQwNDA0NjkyYTgKKFhFTikgQnJvdWdodCB1cCAx
NiBDUFVzCihYRU4pIFNjaGVkdWxpbmcgZ3JhbnVsYXJpdHk6IGNwdSwgMSBDUFUgcGVyIHNjaGVk
LXJlc291cmNlCihYRU4pIEluaXRpYWxpemluZyBDcmVkaXQyIHNjaGVkdWxlcgooWEVOKSAgbG9h
ZF9wcmVjaXNpb25fc2hpZnQ6IDE4CihYRU4pICBsb2FkX3dpbmRvd19zaGlmdDogMzAKKFhFTikg
IHVuZGVybG9hZF9iYWxhbmNlX3RvbGVyYW5jZTogMAooWEVOKSAgb3ZlcmxvYWRfYmFsYW5jZV90
b2xlcmFuY2U6IC0zCihYRU4pICBydW5xdWV1ZXMgYXJyYW5nZW1lbnQ6IHNvY2tldAooWEVOKSAg
Y2FwIGVuZm9yY2VtZW50IGdyYW51bGFyaXR5OiAxMG1zCihYRU4pIGxvYWQgdHJhY2tpbmcgd2lu
ZG93IGxlbmd0aCAxMDczNzQxODI0IG5zCihYRU4pIEFkZGluZyBjcHUgMCB0byBydW5xdWV1ZSAw
CihYRU4pICBGaXJzdCBjcHUgb24gcnVucXVldWUsIGFjdGl2YXRpbmcKKFhFTikgQWRkaW5nIGNw
dSAxIHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNwdSAyIHRvIHJ1bnF1ZXVlIDAKKFhFTikg
QWRkaW5nIGNwdSAzIHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNwdSA0IHRvIHJ1bnF1ZXVl
IDAKKFhFTikgQWRkaW5nIGNwdSA1IHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNwdSA2IHRv
IHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNwdSA3IHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5n
IGNwdSA4IHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNwdSA5IHRvIHJ1bnF1ZXVlIDAKKFhF
TikgQWRkaW5nIGNwdSAxMCB0byBydW5xdWV1ZSAwCihYRU4pIEFkZGluZyBjcHUgMTEgdG8gcnVu
cXVldWUgMAooWEVOKSBBZGRpbmcgY3B1IDEyIHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNw
dSAxMyB0byBydW5xdWV1ZSAwCihYRU4pIEFkZGluZyBjcHUgMTQgdG8gcnVucXVldWUgMAooWEVO
KSBBZGRpbmcgY3B1IDE1IHRvIHJ1bnF1ZXVlIDAKKFhFTikgbWNoZWNrX3BvbGw6IE1hY2hpbmUg
Y2hlY2sgcG9sbGluZyB0aW1lciBzdGFydGVkLgooWEVOKSBOWCAoRXhlY3V0ZSBEaXNhYmxlKSBw
cm90ZWN0aW9uIGFjdGl2ZQooWEVOKSBEb20wIGhhcyBtYXhpbXVtIDEwOTYgUElSUXMKKFhFTikg
KioqIEJ1aWxkaW5nIGEgUFYgRG9tMCAqKioKKFhFTikgIFhlbiAga2VybmVsOiA2NC1iaXQsIGxz
YiwgY29tcGF0MzIKKFhFTikgIERvbTAga2VybmVsOiA2NC1iaXQsIGxzYiwgcGFkZHIgMHgxMDAw
MDAwIC0+IDB4NGEwMDAwMAooWEVOKSBQSFlTSUNBTCBNRU1PUlkgQVJSQU5HRU1FTlQ6CihYRU4p
ICBEb20wIGFsbG9jLjogICAwMDAwMDAwYzMwMDAwMDAwLT4wMDAwMDAwYzM4MDAwMDAwICgxNjIx
OTU2NCBwYWdlcyB0byBiZSBhbGxvY2F0ZWQpCihYRU4pICBJbml0LiByYW1kaXNrOiAwMDAwMDAw
ZmVhZjA1MDAwLT4wMDAwMDAwZmVlMWZmMjhlCihYRU4pIFZJUlRVQUwgTUVNT1JZIEFSUkFOR0VN
RU5UOgooWEVOKSAgTG9hZGVkIGtlcm5lbDogZmZmZmZmZmY4MTAwMDAwMC0+ZmZmZmZmZmY4NGEw
MDAwMAooWEVOKSAgUGh5cy1NYWNoIG1hcDogMDAwMDAwODAwMDAwMDAwMC0+MDAwMDAwODAwN2Mx
ODUzOAooWEVOKSAgU3RhcnQgaW5mbzogICAgZmZmZmZmZmY4NGEwMDAwMC0+ZmZmZmZmZmY4NGEw
MDRiOAooWEVOKSAgUGFnZSB0YWJsZXM6ICAgZmZmZmZmZmY4NGEwMTAwMC0+ZmZmZmZmZmY4NGEy
YTAwMAooWEVOKSAgQm9vdCBzdGFjazogICAgZmZmZmZmZmY4NGEyYTAwMC0+ZmZmZmZmZmY4NGEy
YjAwMAooWEVOKSAgVE9UQUw6ICAgICAgICAgZmZmZmZmZmY4MDAwMDAwMC0+ZmZmZmZmZmY4NGMw
MDAwMAooWEVOKSAgRU5UUlkgQUREUkVTUzogZmZmZmZmZmY4M2FlNzRjMAooWEVOKSBEb20wIGhh
cyBtYXhpbXVtIDE2IFZDUFVzCihYRU4pIEluaXRpYWwgbG93IG1lbW9yeSB2aXJxIHRocmVzaG9s
ZCBzZXQgYXQgMHg0MDAwIHBhZ2VzLgooWEVOKSBTY3J1YmJpbmcgRnJlZSBSQU0gaW4gYmFja2dy
b3VuZAooWEVOKSBTdGQuIExvZ2xldmVsOiBFcnJvcnMsIHdhcm5pbmdzIGFuZCBpbmZvCihYRU4p
IEd1ZXN0IExvZ2xldmVsOiBOb3RoaW5nIChSYXRlLWxpbWl0ZWQ6IEVycm9ycyBhbmQgd2Fybmlu
Z3MpCihYRU4pIFhlbiBpcyByZWxpbnF1aXNoaW5nIFZHQSBjb25zb2xlLgooWEVOKSAqKiogU2Vy
aWFsIGlucHV0IHRvIERPTTAgKHR5cGUgJ0NUUkwtYScgdGhyZWUgdGltZXMgdG8gc3dpdGNoIGlu
cHV0KQooWEVOKSBGcmVlZCA2NDhrQiBpbml0IG1lbW9yeQo=

--b1_ahePdPP8ZPOBcsnbMP4jUdSkh5Jox7jBC4XyO2W8--



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 08:40:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 08:40:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674256.1049086 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVSbz-000576-KK; Thu, 01 Feb 2024 08:39:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674256.1049086; Thu, 01 Feb 2024 08:39:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVSbz-00056z-GA; Thu, 01 Feb 2024 08:39:55 +0000
Received: by outflank-mailman (input) for mailman id 674256;
 Thu, 01 Feb 2024 08:39:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cAD+=JK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rVSby-00056t-4C
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 08:39:54 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 76f01adc-c0dd-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 09:39:53 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-40e80046246so3010285e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 00:39:51 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 g10-20020a05600c4eca00b0040fbba734f3sm1082225wmq.34.2024.02.01.00.39.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 00:39:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 76f01adc-c0dd-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706776790; x=1707381590; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Mmx7Ge44ylq+xRaE54Mmi5lsueMZNlgqIRQYM/UNpUU=;
        b=OmNEBETwYjdQ78BY2VOaVyxirQZhsePrn9jIlvhcHMLq3qHpua71BUUKK9TolNJaIM
         ICUePWoYJ+sten497EIQhayU6+HzPM7RICgEDsW4dnkflBGpmjI0L98/8OM5R9RpzWOp
         b91TgB3ekalm0wqP8YgixA02jfs0HOW1UyL+g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706776790; x=1707381590;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Mmx7Ge44ylq+xRaE54Mmi5lsueMZNlgqIRQYM/UNpUU=;
        b=tp5nUovhg8hSheHR5aULhHepI1RkXg0+w1u9R0XWbIT3ObJCbZTMD19kYjfWYMC7PH
         GPQYZkx/36C5+AvHs+I537C67RkfTa+YyHBZ/HJip22L/PaR0YawnSXfujdReOpnDb0Y
         yUHCA1G3gJHbuboJd0rQGsyBJI0xPWxf3ZeNZs+3fk9RZqWPf8qur6Qm7DmyVbTFVFWV
         qX06nNDYcSZRD5Bg93gmakJ9XUMbU58ZFaJb8jBY98lQxyXlMKeQrE7hgEpmeE+xQrUL
         UxwpWBbmtgAhpiM39y+YIch72cdlQnHYrnJqPK4Ro/gzQfY3p42CrpBLMozgOyMnSO3B
         tJjA==
X-Gm-Message-State: AOJu0Yx1Iuvv8F+cBwLosI7Z5gyqSPHt1gZ7DBrOVLNwfO0vkFGpXaVT
	rIELNccLVJDzAXL9wdk3+Hy0sdkWl9hoVkjsUQ8WGk1Bq86ciZiJSMqjcWkUMXE=
X-Google-Smtp-Source: AGHT+IGW7bgbJkTrR4Yza/IRXj8QJZwTE7Cr91T4ok2nKqWk8e0wQ2sciL8gpCHIKiRmLekUEHkNew==
X-Received: by 2002:a05:600c:1c9a:b0:40e:f972:9901 with SMTP id k26-20020a05600c1c9a00b0040ef9729901mr3736599wms.4.1706776790540;
        Thu, 01 Feb 2024 00:39:50 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXDCVhgfb0o24SN4Wqn/FgnWkJYm9QCYpD/ESYbFl4VjZTFA1lzb2EecVom9k5fEwfP12Y/uppYGzcffTRrzp/icQGv3HjGZpMbEHP1O0bmMpkENMmh/i5WpK5zIezuae0Uf4YZJFj/GIIpi8YgFkSb+ve1+U2VK0dRTY3tLYI0EKqJsftwJxoWx8X1Knfb8tnC76BuOBql1srwLsN7vGSl2wcbMkEqp6rlaRVuZC9dj8g2KKow5t0/nsSr8nJZlls/92TMW3QXD5Uc/pJNuCeeBPj2+nM9/cIi4ZkYdMgG2c7Y3W/85xx3atQ7NFnga1NjCNQMDhJbuqruZaGAYHg7aAFbNfLKQC7xPnI5wPmFp1E3D2qFqwlvqSIyLUOZrN5YudkUPsFIPL0Qk1faQOXY4FjKqsnSsDMGI58pczPV4freLpRkOBnJ+zblZe0frUgjrALk1NdUEoPj3Y9vKQ+JxYOJfr1AnKZs/vkDbp6joC8yuId8ZmXn+YeGnhlWwAZhccdcXntd3R0=
Date: Thu, 1 Feb 2024 09:39:49 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: "Chen, Jiqian" <Jiqian.Chen@amd.com>,
	"Rafael J . Wysocki" <rafael@kernel.org>,
	Len Brown <lenb@kernel.org>, Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
	"Huang, Ray" <Ray.Huang@amd.com>,
	"Ragiadakou, Xenia" <Xenia.Ragiadakou@amd.com>
Subject: Re: [RFC KERNEL PATCH v4 3/3] PCI/sysfs: Add gsi sysfs for pci_dev
Message-ID: <ZbtY1R15pYZz3F3B@macbook>
References: <ZboLq6kZhwpUC_c3@macbook>
 <20240131190014.GA593286@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240131190014.GA593286@bhelgaas>

On Wed, Jan 31, 2024 at 01:00:14PM -0600, Bjorn Helgaas wrote:
> On Wed, Jan 31, 2024 at 09:58:19AM +0100, Roger Pau Monné wrote:
> > On Tue, Jan 30, 2024 at 02:44:03PM -0600, Bjorn Helgaas wrote:
> > > On Tue, Jan 30, 2024 at 10:07:36AM +0100, Roger Pau Monné wrote:
> > > > On Mon, Jan 29, 2024 at 04:01:13PM -0600, Bjorn Helgaas wrote:
> > > > > On Thu, Jan 25, 2024 at 07:17:24AM +0000, Chen, Jiqian wrote:
> > > > > > On 2024/1/24 00:02, Bjorn Helgaas wrote:
> > > > > > > On Tue, Jan 23, 2024 at 10:13:52AM +0000, Chen, Jiqian wrote:
> > > > > > >> On 2024/1/23 07:37, Bjorn Helgaas wrote:
> > > > > > >>> On Fri, Jan 05, 2024 at 02:22:17PM +0800, Jiqian Chen wrote:
> > > > > > >>>> There is a need for some scenarios to use gsi sysfs.
> > > > > > >>>> For example, when xen passthrough a device to dumU, it will
> > > > > > >>>> use gsi to map pirq, but currently userspace can't get gsi
> > > > > > >>>> number.
> > > > > > >>>> So, add gsi sysfs for that and for other potential scenarios.
> > > > > > >> ...
> > > > > > > 
> > > > > > >>> I don't know enough about Xen to know why it needs the GSI in
> > > > > > >>> userspace.  Is this passthrough brand new functionality that can't be
> > > > > > >>> done today because we don't expose the GSI yet?
> > > > > 
> > > > > I assume this must be new functionality, i.e., this kind of
> > > > > passthrough does not work today, right?
> > > > > 
> > > > > > >> has ACPI support and is responsible for detecting and controlling
> > > > > > >> the hardware, also it performs privileged operations such as the
> > > > > > >> creation of normal (unprivileged) domains DomUs. When we give to a
> > > > > > >> DomU direct access to a device, we need also to route the physical
> > > > > > >> interrupts to the DomU. In order to do so Xen needs to setup and map
> > > > > > >> the interrupts appropriately.
> > > > > > > 
> > > > > > > What kernel interfaces are used for this setup and mapping?
> > > > > >
> > > > > > For passthrough devices, the setup and mapping of routing physical
> > > > > > interrupts to DomU are done on Xen hypervisor side, hypervisor only
> > > > > > need userspace to provide the GSI info, see Xen code:
> > > > > > xc_physdev_map_pirq require GSI and then will call hypercall to pass
> > > > > > GSI into hypervisor and then hypervisor will do the mapping and
> > > > > > routing, kernel doesn't do the setup and mapping.
> > > > > 
> > > > > So we have to expose the GSI to userspace not because userspace itself
> > > > > uses it, but so userspace can turn around and pass it back into the
> > > > > kernel?
> > > > 
> > > > No, the point is to pass it back to Xen, which doesn't know the
> > > > mapping between GSIs and PCI devices because it can't execute the ACPI
> > > > AML resource methods that provide such information.
> > > > 
> > > > The (Linux) kernel is just a proxy that forwards the hypercalls from
> > > > user-space tools into Xen.
> > > 
> > > But I guess Xen knows how to interpret a GSI even though it doesn't
> > > have access to AML?
> > 
> > On x86 Xen does know how to map a GSI into an IO-APIC pin, in order
> > configure the RTE as requested.
> 
> IIUC, mapping a GSI to an IO-APIC pin requires information from the
> MADT.  So I guess Xen does use the static ACPI tables, but not the AML
> _PRT methods that would connect a GSI with a PCI device?

Yes, Xen can parse the static tables, and knows the base GSI of
IO-APICs from the MADT.

> I guess this means Xen would not be able to deal with _MAT methods,
> which also contains MADT entries?  I don't know the implications of
> this -- maybe it means Xen might not be able to use with hot-added
> devices?

It's my understanding _MAT will only be present on some very specific
devices (IO-APIC or CPU objects).  Xen doesn't support hotplug of
IO-APICs, but hotplug of CPUs should in principle be supported with
cooperation from the control domain OS (albeit it's not something that
we tests on our CI).  I don't expect however that a CPU object _MAT
method will return IO APIC entries.

> The tables (including DSDT and SSDTS that contain the AML) are exposed
> to userspace via /sys/firmware/acpi/tables/, but of course that
> doesn't mean Xen knows how to interpret the AML, and even if it did,
> Xen probably wouldn't be able to *evaluate* it since that could
> conflict with the host kernel's use of AML.

Indeed, there can only be a single OSPM, and that's the dom0 OS (Linux
in our context).

Getting back to our context though, what would be a suitable place for
exposing the GSI assigned to each device?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 08:59:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 08:59:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674260.1049094 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVSun-00009K-3y; Thu, 01 Feb 2024 08:59:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674260.1049094; Thu, 01 Feb 2024 08:59:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVSun-00009D-1A; Thu, 01 Feb 2024 08:59:21 +0000
Received: by outflank-mailman (input) for mailman id 674260;
 Thu, 01 Feb 2024 08:59:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eRhu=JK=redhat.com=thuth@srs-se1.protection.inumbo.net>)
 id 1rVSum-000097-0u
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 08:59:20 +0000
Received: from us-smtp-delivery-124.mimecast.com
 (us-smtp-delivery-124.mimecast.com [170.10.129.124])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2d97fc85-c0e0-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 09:59:17 +0100 (CET)
Received: from mail-lf1-f72.google.com (mail-lf1-f72.google.com
 [209.85.167.72]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-445-FwDJFd6PPhCM1yzxSKZkVQ-1; Thu, 01 Feb 2024 03:59:15 -0500
Received: by mail-lf1-f72.google.com with SMTP id
 2adb3069b0e04-50ec9529001so559987e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 00:59:14 -0800 (PST)
Received: from [192.168.0.9] (ip-109-43-177-196.web.vodafone.de.
 [109.43.177.196]) by smtp.gmail.com with ESMTPSA id
 t12-20020a05600c450c00b0040e9f7308f4sm3803420wmo.10.2024.02.01.00.59.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 00:59:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d97fc85-c0e0-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1706777956;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=46aTfHzOBMWuZA5PRKaMztJA+fyye2SIsD0CIZHrpW8=;
	b=OaK6VppTSmTVUr2WknCT2e3supYgKeVbVpAFUuOcr5kyqR+7wXQZszsyzR5qmHsw0Ul+kl
	PMRwUVqN2xMh8B5egG/SZaRRdK+NaYK6pvI4odvI7skaiwNJDUATBxlithSy7A/kngyXg6
	fbHKWcf1eGQnkpeV7wY6d0wnMqdrd+Q=
X-MC-Unique: FwDJFd6PPhCM1yzxSKZkVQ-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706777953; x=1707382753;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=46aTfHzOBMWuZA5PRKaMztJA+fyye2SIsD0CIZHrpW8=;
        b=brrNEeXOAnNn8vJ26b0J3odrllNSWHCo4RN/A/Hyxj8JhEeqEAq1USdwIee2eu4d3g
         SeBwxwg4Q11lvb0E73YVkHNH4frwHCFYTYCI6QkMTED1u/Cp2xR0ZGFB/Dym0P53WxeJ
         zuXpmZh0fkunUzIgH+uZazVhJCWc8sxJ52vHclB3I4V79Yuie/F8G02CK4Wz1SwY0iCA
         gDgzg/TboCh27kTTe7QJwTlOFyUa/+2TXEmPuKwU1OaDACTAnJs6A/BuXgfMMPcCKF1K
         Rxq1vHnEwtsvTGCIb1RnTxpmcDSWSk+RVf6v7KxjyjkD+RRn5eCGQOs1WL+DPebQunMJ
         amxQ==
X-Gm-Message-State: AOJu0Yz9imYTTTGOxZ3B40BdqO+ukW5iqPWK0vj5dmcFFYdG6/o0pkxk
	nmy5UXexVFI2FRtqXNZx7BkMU3+78eZlEoC0air7HkxJ3z4I/zqH9NQjRg2BMyGl0dkh08rNmS7
	ptfUzdEQSPVksRL9o6DWEhd5gsobqBhyn5bEF2cwdq8X3orsO0sp9wTBzV/QHn/g3
X-Received: by 2002:a19:ad0c:0:b0:511:319f:82eb with SMTP id t12-20020a19ad0c000000b00511319f82ebmr152468lfc.43.1706777953347;
        Thu, 01 Feb 2024 00:59:13 -0800 (PST)
X-Google-Smtp-Source: AGHT+IHxtZpmfE8n4eP1jKoWvPvn4IfLMH3obU6Ulatqlu1FlZMnyZlnKO4WsRVCTxkqqtQiWLrSIw==
X-Received: by 2002:a19:ad0c:0:b0:511:319f:82eb with SMTP id t12-20020a19ad0c000000b00511319f82ebmr152423lfc.43.1706777953003;
        Thu, 01 Feb 2024 00:59:13 -0800 (PST)
Message-ID: <e3470bee-a714-4259-9bde-df9595618877@redhat.com>
Date: Thu, 1 Feb 2024 09:59:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 37/47] hw/net/lasi_i82596: Re-enable build
To: David Woodhouse <dwmw2@infradead.org>, qemu-devel@nongnu.org
Cc: Richard Henderson <richard.henderson@linaro.org>,
 Beniamino Galvani <b.galvani@gmail.com>,
 Peter Maydell <peter.maydell@linaro.org>,
 Strahinja Jankovic <strahinja.p.jankovic@gmail.com>,
 Niek Linnenbank <nieklinnenbank@gmail.com>, =?UTF-8?Q?C=C3=A9dric_Le_Goater?=
 <clg@kaod.org>, Andrew Jeffery <andrew@codeconstruct.com.au>,
 Joel Stanley <joel@jms.id.au>, Igor Mitsyanko <i.mitsyanko@gmail.com>,
 Jean-Christophe Dubois <jcd@tribudubois.net>,
 Andrey Smirnov <andrew.smirnov@gmail.com>,
 =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 Rob Herring <robh@kernel.org>, Subbaraya Sundeep <sundeep.lkml@gmail.com>,
 Jan Kiszka <jan.kiszka@web.de>, Tyrone Ting <kfting@nuvoton.com>,
 Hao Wu <wuhaotsh@google.com>, Radoslaw Biernacki <rad@semihalf.com>,
 Leif Lindholm <quic_llindhol@quicinc.com>,
 Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Alistair Francis <alistair@alistair23.me>, Helge Deller <deller@gmx.de>,
 Paolo Bonzini <pbonzini@redhat.com>, Eduardo Habkost <eduardo@habkost.net>,
 "Michael S. Tsirkin" <mst@redhat.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Song Gao
 <gaosong@loongson.cn>, Thomas Huth <huth@tuxfamily.org>,
 Laurent Vivier <laurent@vivier.eu>, Huacai Chen <chenhuacai@kernel.org>,
 Jiaxun Yang <jiaxun.yang@flygoat.com>,
 =?UTF-8?Q?Herv=C3=A9_Poussineau?= <hpoussin@reactos.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Aurelien Jarno <aurelien@aurel32.net>, Jason Wang <jasowang@redhat.com>,
 Jia Liu <proljc@gmail.com>, Stafford Horne <shorne@gmail.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Nicholas Piggin <npiggin@gmail.com>,
 Daniel Henrique Barboza <danielhb413@gmail.com>,
 David Gibson <david@gibson.dropbear.id.au>,
 Harsh Prateek Bora <harshpb@linux.ibm.com>, Bin Meng
 <bin.meng@windriver.com>, Palmer Dabbelt <palmer@dabbelt.com>,
 Weiwei Li <liwei1518@gmail.com>, Liu Zhiwei <zhiwei_liu@linux.alibaba.com>,
 Halil Pasic <pasic@linux.ibm.com>,
 Christian Borntraeger <borntraeger@linux.ibm.com>,
 Eric Farman <farman@linux.ibm.com>, David Hildenbrand <david@redhat.com>,
 Ilya Leoshkevich <iii@linux.ibm.com>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Magnus Damm <magnus.damm@gmail.com>, Artyom Tarasenko <atar4qemu@gmail.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 Max Filippov <jcmvbkbc@gmail.com>, qemu-arm@nongnu.org, qemu-ppc@nongnu.org,
 qemu-riscv@nongnu.org, qemu-s390x@nongnu.org,
 xen-devel@lists.xenproject.org, David Woodhouse <dwmw@amazon.co.uk>
References: <20240126173228.394202-1-dwmw2@infradead.org>
 <20240126173228.394202-38-dwmw2@infradead.org>
From: Thomas Huth <thuth@redhat.com>
Autocrypt: addr=thuth@redhat.com; keydata=
 xsFNBFH7eUwBEACzyOXKU+5Pcs6wNpKzrlJwzRl3VGZt95VCdb+FgoU9g11m7FWcOafrVRwU
 yYkTm9+7zBUc0sW5AuPGR/dp3pSLX/yFWsA/UB4nJsHqgDvDU7BImSeiTrnpMOTXb7Arw2a2
 4CflIyFqjCpfDM4MuTmzTjXq4Uov1giGE9X6viNo1pxyEpd7PanlKNnf4PqEQp06X4IgUacW
 tSGj6Gcns1bCuHV8OPWLkf4hkRnu8hdL6i60Yxz4E6TqlrpxsfYwLXgEeswPHOA6Mn4Cso9O
 0lewVYfFfsmokfAVMKWzOl1Sr0KGI5T9CpmRfAiSHpthhHWnECcJFwl72NTi6kUcUzG4se81
 O6n9d/kTj7pzTmBdfwuOZ0YUSqcqs0W+l1NcASSYZQaDoD3/SLk+nqVeCBB4OnYOGhgmIHNW
 0CwMRO/GK+20alxzk//V9GmIM2ACElbfF8+Uug3pqiHkVnKqM7W9/S1NH2qmxB6zMiJUHlTH
 gnVeZX0dgH27mzstcF786uPcdEqS0KJuxh2kk5IvUSL3Qn3ZgmgdxBMyCPciD/1cb7/Ahazr
 3ThHQXSHXkH/aDXdfLsKVuwDzHLVSkdSnZdt5HHh75/NFHxwaTlydgfHmFFwodK8y/TjyiGZ
 zg2Kje38xnz8zKn9iesFBCcONXS7txENTzX0z80WKBhK+XSFJwARAQABzR5UaG9tYXMgSHV0
 aCA8dGh1dGhAcmVkaGF0LmNvbT7CwXgEEwECACIFAlVgX6oCGwMGCwkIBwMCBhUIAgkKCwQW
 AgMBAh4BAheAAAoJEC7Z13T+cC21EbIP/ii9cvT2HHGbFRl8HqGT6+7Wkb+XLMqJBMAIGiQK
 QIP3xk1HPTsLfVG0ao4hy/oYkGNOP8+ubLnZen6Yq3zAFiMhQ44lvgigDYJo3Ve59gfe99KX
 EbtB+X95ODARkq0McR6OAsPNJ7gpEUzfkQUUJTXRDQXfG/FX303Gvk+YU0spm2tsIKPl6AmV
 1CegDljzjycyfJbk418MQmMu2T82kjrkEofUO2a24ed3VGC0/Uz//XCR2ZTo+vBoBUQl41BD
 eFFtoCSrzo3yPFS+w5fkH9NT8ChdpSlbNS32NhYQhJtr9zjWyFRf0Zk+T/1P7ECn6gTEkp5k
 ofFIA4MFBc/fXbaDRtBmPB0N9pqTFApIUI4vuFPPO0JDrII9dLwZ6lO9EKiwuVlvr1wwzsgq
 zJTPBU3qHaUO4d/8G+gD7AL/6T4zi8Jo/GmjBsnYaTzbm94lf0CjXjsOX3seMhaE6WAZOQQG
 tZHAO1kAPWpaxne+wtgMKthyPLNwelLf+xzGvrIKvLX6QuLoWMnWldu22z2ICVnLQChlR9d6
 WW8QFEpo/FK7omuS8KvvopFcOOdlbFMM8Y/8vBgVMSsK6fsYUhruny/PahprPbYGiNIhKqz7
 UvgyZVl4pBFjTaz/SbimTk210vIlkDyy1WuS8Zsn0htv4+jQPgo9rqFE4mipJjy/iboDzsFN
 BFH7eUwBEAC2nzfUeeI8dv0C4qrfCPze6NkryUflEut9WwHhfXCLjtvCjnoGqFelH/PE9NF4
 4VPSCdvD1SSmFVzu6T9qWdcwMSaC+e7G/z0/AhBfqTeosAF5XvKQlAb9ZPkdDr7YN0a1XDfa
 +NgA+JZB4ROyBZFFAwNHT+HCnyzy0v9Sh3BgJJwfpXHH2l3LfncvV8rgFv0bvdr70U+On2XH
 5bApOyW1WpIG5KPJlDdzcQTyptOJ1dnEHfwnABEfzI3dNf63rlxsGouX/NFRRRNqkdClQR3K
 gCwciaXfZ7ir7fF0u1N2UuLsWA8Ei1JrNypk+MRxhbvdQC4tyZCZ8mVDk+QOK6pyK2f4rMf/
 WmqxNTtAVmNuZIwnJdjRMMSs4W4w6N/bRvpqtykSqx7VXcgqtv6eqoDZrNuhGbekQA0sAnCJ
 VPArerAZGArm63o39me/bRUQeQVSxEBmg66yshF9HkcUPGVeC4B0TPwz+HFcVhheo6hoJjLq
 knFOPLRj+0h+ZL+D0GenyqD3CyuyeTT5dGcNU9qT74bdSr20k/CklvI7S9yoQje8BeQAHtdV
 cvO8XCLrpGuw9SgOS7OP5oI26a0548M4KldAY+kqX6XVphEw3/6U1KTf7WxW5zYLTtadjISB
 X9xsRWSU+Yqs3C7oN5TIPSoj9tXMoxZkCIHWvnqGwZ7JhwARAQABwsFfBBgBAgAJBQJR+3lM
 AhsMAAoJEC7Z13T+cC21hPAQAIsBL9MdGpdEpvXs9CYrBkd6tS9mbaSWj6XBDfA1AEdQkBOn
 ZH1Qt7HJesk+qNSnLv6+jP4VwqK5AFMrKJ6IjE7jqgzGxtcZnvSjeDGPF1h2CKZQPpTw890k
 fy18AvgFHkVk2Oylyexw3aOBsXg6ukN44vIFqPoc+YSU0+0QIdYJp/XFsgWxnFIMYwDpxSHS
 5fdDxUjsk3UBHZx+IhFjs2siVZi5wnHIqM7eK9abr2cK2weInTBwXwqVWjsXZ4tq5+jQrwDK
 cvxIcwXdUTLGxc4/Z/VRH1PZSvfQxdxMGmNTGaXVNfdFZjm4fz0mz+OUi6AHC4CZpwnsliGV
 ODqwX8Y1zic9viSTbKS01ZNp175POyWViUk9qisPZB7ypfSIVSEULrL347qY/hm9ahhqmn17
 Ng255syASv3ehvX7iwWDfzXbA0/TVaqwa1YIkec+/8miicV0zMP9siRcYQkyTqSzaTFBBmqD
 oiT+z+/E59qj/EKfyce3sbC9XLjXv3mHMrq1tKX4G7IJGnS989E/fg6crv6NHae9Ckm7+lSs
 IQu4bBP2GxiRQ+NV3iV/KU3ebMRzqIC//DCOxzQNFNJAKldPe/bKZMCxEqtVoRkuJtNdp/5a
 yXFZ6TfE1hGKrDBYAm4vrnZ4CXFSBDllL59cFFOJCkn4Xboj/aVxxJxF30bn
In-Reply-To: <20240126173228.394202-38-dwmw2@infradead.org>
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 26/01/2024 18.25, David Woodhouse wrote:
> From: David Woodhouse <dwmw@amazon.co.uk>
> 
> When converting to the shiny build-system-du-jour, a typo prevented the
> last_i82596 driver from being built. Correct the config option name to
> re-enable the build. And include "sysemu/sysemu.h" so it actually builds.
> 
> Fixes: b1419fa66558 ("meson: convert hw/net")
> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> ---
>   hw/net/lasi_i82596.c | 1 +
>   hw/net/meson.build   | 2 +-
>   2 files changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/net/lasi_i82596.c b/hw/net/lasi_i82596.c
> index 6a3147fe2d..09e830ba5f 100644
> --- a/hw/net/lasi_i82596.c
> +++ b/hw/net/lasi_i82596.c
> @@ -14,6 +14,7 @@
>   #include "qapi/error.h"
>   #include "qemu/timer.h"
>   #include "hw/sysbus.h"
> +#include "sysemu/sysemu.h"
>   #include "net/eth.h"
>   #include "hw/net/lasi_82596.h"
>   #include "hw/net/i82596.h"
> diff --git a/hw/net/meson.build b/hw/net/meson.build
> index 9afceb0619..2b426d3d5a 100644
> --- a/hw/net/meson.build
> +++ b/hw/net/meson.build
> @@ -33,7 +33,7 @@ system_ss.add(when: 'CONFIG_MARVELL_88W8618', if_true: files('mv88w8618_eth.c'))
>   system_ss.add(when: 'CONFIG_CADENCE', if_true: files('cadence_gem.c'))
>   system_ss.add(when: 'CONFIG_STELLARIS_ENET', if_true: files('stellaris_enet.c'))
>   system_ss.add(when: 'CONFIG_LANCE', if_true: files('lance.c'))
> -system_ss.add(when: 'CONFIG_LASI_I82596', if_true: files('lasi_i82596.c'))
> +system_ss.add(when: 'CONFIG_LASI_82596', if_true: files('lasi_i82596.c'))
>   system_ss.add(when: 'CONFIG_I82596_COMMON', if_true: files('i82596.c'))
>   system_ss.add(when: 'CONFIG_SUNHME', if_true: files('sunhme.c'))
>   system_ss.add(when: 'CONFIG_FTGMAC100', if_true: files('ftgmac100.c'))

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2144



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 10:34:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 10:34:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674271.1049108 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUO9-0006Kg-Py; Thu, 01 Feb 2024 10:33:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674271.1049108; Thu, 01 Feb 2024 10:33:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUO9-0006KZ-MG; Thu, 01 Feb 2024 10:33:45 +0000
Received: by outflank-mailman (input) for mailman id 674271;
 Thu, 01 Feb 2024 10:33:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i6jM=JK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVUO8-0006KQ-B6
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 10:33:44 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5f1cbb73-c0ed-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 11:33:43 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-40e80046264so6792725e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 02:33:43 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 g10-20020a05600c4eca00b0040fbba734f3sm1395865wmq.34.2024.02.01.02.33.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 02:33:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f1cbb73-c0ed-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706783622; x=1707388422; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=D8J71cORIIJRoqYhXR8/qnQiyP/PfOcpk4LGA73fgB0=;
        b=ehCcxnN5SUNiax9nWYsD+sK7dGawQ90387sRVjM8UlSA7wS3hUAHsa/Mk+LZ3W/Gkx
         FjwtV7eD5grH7eB+UTyayD1jW+P9IZzWpYqTjCjNqcTGqaglXG5ePiQdhKX22mM6Y7pO
         DMSrpuNPo6xN5Y8kAembfTXKWNaCdtNDeL0Lw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706783622; x=1707388422;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=D8J71cORIIJRoqYhXR8/qnQiyP/PfOcpk4LGA73fgB0=;
        b=Bk9gI8tO1ePfj0P8mmgwxFq/aQaGad6ZOlCkWaYRPoItMZXWuFAFxsQObk+TOEGzum
         FPxgHibMhWf00+Srm1y4gI4Jpm13ex5uMsf87ZPa+/py7BhB0r50Xe/8D5X/TkJbkz5k
         d5JVUD/VySBgvvoufxSKOU3e6uAne0l+qfcJKksjZnFMUg98JR/fOci07uXPfZe5HRxX
         BtJi3CZqjv81UsHDSPK72iGBlZdm6IHjFEwK6AvWhyEcZTramtyPURRTxU98Oyqy1R0l
         X3ZuYfaCnLUoSA0bRIEHPI/g47aMS9syi5pf2Q75Hoe2I/k7uCs879/08MTa9mqjcpKb
         l4mg==
X-Gm-Message-State: AOJu0Yx2PdB7byHM4PH+jcZ7cvC2xKgzHNn8sYZr4WBYLTTrNs+KeJuL
	MHEYhS0b01GRN8o0IISH363FG0/lnPNES2+fAOPe1otkp07j6tcsicm4T4kYdOwir3/M9v856QN
	w
X-Google-Smtp-Source: AGHT+IHFb00cp37ppIbJJKHh4MJRADrSYXz3jmxNOyaAIEpn5stWts3uRs9zYdfitFoGEmO37ivjlQ==
X-Received: by 2002:a05:600c:3d88:b0:40f:b8e7:ad51 with SMTP id bi8-20020a05600c3d8800b0040fb8e7ad51mr1510749wmb.31.1706783622063;
        Thu, 01 Feb 2024 02:33:42 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUBY5YRDQ0F/TzC+6OmFx9c5nokmfEOljkdq0czD5PZFyC5kwbob7tsFyYLMm35OZxSGhC8oiE4dScslf5XPYuCsev3lyimueyppHDkKLIki6MDFxZ8paolBaKusCj9sHbFiKd0zoNYtZINe4w8cYmW27AW3i3pCB6OD7bqnLJj+xd/et6WxsfwnjQx+QjnIA==
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH] xen/bitmap: Consistently use unsigned bits values
Date: Thu,  1 Feb 2024 10:33:39 +0000
Message-Id: <20240201103339.549307-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Right now, most of the static inline helpers take an unsigned nbits quantity,
and most of the library functions take a signed quanity.  Because
BITMAP_LAST_WORD_MASK() is expressed as a divide, the compiler is forced to
emit two different paths to get the correct semantics for signed division.

Swap all signed bit-counts to being unsigned bit-counts for the simple cases.
This includes the return value of bitmap_weight().

Bloat-o-meter for a random x86 build reports:
  add/remove: 0/0 grow/shrink: 8/19 up/down: 167/-413 (-246)

which all comes from compiler not emitting "dead" logic paths for negative bit
counts.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>

Found when investigating negative-shift UBSAN, because each of the library
functions ended up being instrumented.

There is much more wanting cleaning up here, but we have to start somewhere.
Some observations:

 * Various of the boolean-like return values have -1 for zero-length bitmaps.
   I can't spot any callers which care, so this seems like a waste.
 * bitmap_zero() and similar clear predate us switching to use
   __builtin_memset(), because there's no need for bitmap_switch().
 * Should we consolidate 'bits' vs 'nbits'?
 * The internals of these helpers want converting too.  Other helpers need
   more than just a parameter conversion.
---
 xen/common/bitmap.c      | 24 +++++++++++-----------
 xen/include/xen/bitmap.h | 43 ++++++++++++++++++++--------------------
 2 files changed, 34 insertions(+), 33 deletions(-)

diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c
index 7d4551f78283..c57b35f0042c 100644
--- a/xen/common/bitmap.c
+++ b/xen/common/bitmap.c
@@ -55,7 +55,7 @@ static void clamp_last_byte(uint8_t *bp, unsigned int nbits)
 		bp[nbits/8] &= (1U << remainder) - 1;
 }
 
-int __bitmap_empty(const unsigned long *bitmap, int bits)
+int __bitmap_empty(const unsigned long *bitmap, unsigned int bits)
 {
 	int k, lim = bits/BITS_PER_LONG;
 	for (k = 0; k < lim; ++k)
@@ -70,7 +70,7 @@ int __bitmap_empty(const unsigned long *bitmap, int bits)
 }
 EXPORT_SYMBOL(__bitmap_empty);
 
-int __bitmap_full(const unsigned long *bitmap, int bits)
+int __bitmap_full(const unsigned long *bitmap, unsigned int bits)
 {
 	int k, lim = bits/BITS_PER_LONG;
 	for (k = 0; k < lim; ++k)
@@ -86,7 +86,7 @@ int __bitmap_full(const unsigned long *bitmap, int bits)
 EXPORT_SYMBOL(__bitmap_full);
 
 int __bitmap_equal(const unsigned long *bitmap1,
-		const unsigned long *bitmap2, int bits)
+                   const unsigned long *bitmap2, unsigned int bits)
 {
 	int k, lim = bits/BITS_PER_LONG;
 	for (k = 0; k < lim; ++k)
@@ -101,7 +101,7 @@ int __bitmap_equal(const unsigned long *bitmap1,
 }
 EXPORT_SYMBOL(__bitmap_equal);
 
-void __bitmap_complement(unsigned long *dst, const unsigned long *src, int bits)
+void __bitmap_complement(unsigned long *dst, const unsigned long *src, unsigned int bits)
 {
 	int k, lim = bits/BITS_PER_LONG;
 	for (k = 0; k < lim; ++k)
@@ -113,7 +113,7 @@ void __bitmap_complement(unsigned long *dst, const unsigned long *src, int bits)
 EXPORT_SYMBOL(__bitmap_complement);
 
 void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
-				const unsigned long *bitmap2, int bits)
+                  const unsigned long *bitmap2, unsigned int bits)
 {
 	int k;
 	int nr = BITS_TO_LONGS(bits);
@@ -124,7 +124,7 @@ void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
 EXPORT_SYMBOL(__bitmap_and);
 
 void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
-				const unsigned long *bitmap2, int bits)
+                 const unsigned long *bitmap2, unsigned int bits)
 {
 	int k;
 	int nr = BITS_TO_LONGS(bits);
@@ -135,7 +135,7 @@ void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
 EXPORT_SYMBOL(__bitmap_or);
 
 void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
-				const unsigned long *bitmap2, int bits)
+                  const unsigned long *bitmap2, unsigned int bits)
 {
 	int k;
 	int nr = BITS_TO_LONGS(bits);
@@ -146,7 +146,7 @@ void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
 EXPORT_SYMBOL(__bitmap_xor);
 
 void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
-				const unsigned long *bitmap2, int bits)
+                     const unsigned long *bitmap2, unsigned int bits)
 {
 	int k;
 	int nr = BITS_TO_LONGS(bits);
@@ -157,7 +157,7 @@ void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
 EXPORT_SYMBOL(__bitmap_andnot);
 
 int __bitmap_intersects(const unsigned long *bitmap1,
-				const unsigned long *bitmap2, int bits)
+                        const unsigned long *bitmap2, unsigned int bits)
 {
 	int k, lim = bits/BITS_PER_LONG;
 	for (k = 0; k < lim; ++k)
@@ -172,7 +172,7 @@ int __bitmap_intersects(const unsigned long *bitmap1,
 EXPORT_SYMBOL(__bitmap_intersects);
 
 int __bitmap_subset(const unsigned long *bitmap1,
-				const unsigned long *bitmap2, int bits)
+                    const unsigned long *bitmap2, unsigned int bits)
 {
 	int k, lim = bits/BITS_PER_LONG;
 	for (k = 0; k < lim; ++k)
@@ -187,7 +187,7 @@ int __bitmap_subset(const unsigned long *bitmap1,
 EXPORT_SYMBOL(__bitmap_subset);
 
 #if BITS_PER_LONG == 32
-int __bitmap_weight(const unsigned long *bitmap, int bits)
+unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int bits)
 {
 	int k, w = 0, lim = bits/BITS_PER_LONG;
 
@@ -200,7 +200,7 @@ int __bitmap_weight(const unsigned long *bitmap, int bits)
 	return w;
 }
 #else
-int __bitmap_weight(const unsigned long *bitmap, int bits)
+unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int bits)
 {
 	int k, w = 0, lim = bits/BITS_PER_LONG;
 
diff --git a/xen/include/xen/bitmap.h b/xen/include/xen/bitmap.h
index 657390e32ebc..b9f980e91930 100644
--- a/xen/include/xen/bitmap.h
+++ b/xen/include/xen/bitmap.h
@@ -66,25 +66,25 @@
  * lib/bitmap.c provides these functions:
  */
 
-extern int __bitmap_empty(const unsigned long *bitmap, int bits);
-extern int __bitmap_full(const unsigned long *bitmap, int bits);
-extern int __bitmap_equal(const unsigned long *bitmap1,
-                	const unsigned long *bitmap2, int bits);
-extern void __bitmap_complement(unsigned long *dst, const unsigned long *src,
-			int bits);
-extern void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
-			const unsigned long *bitmap2, int bits);
-extern void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
-			const unsigned long *bitmap2, int bits);
-extern void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
-			const unsigned long *bitmap2, int bits);
-extern void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
-			const unsigned long *bitmap2, int bits);
-extern int __bitmap_intersects(const unsigned long *bitmap1,
-			const unsigned long *bitmap2, int bits);
-extern int __bitmap_subset(const unsigned long *bitmap1,
-			const unsigned long *bitmap2, int bits);
-extern int __bitmap_weight(const unsigned long *bitmap, int bits);
+int __bitmap_empty(const unsigned long *bitmap, unsigned int bits);
+int __bitmap_full(const unsigned long *bitmap, unsigned int bits);
+int __bitmap_equal(const unsigned long *bitmap1,
+                   const unsigned long *bitmap2, unsigned int bits);
+void __bitmap_complement(unsigned long *dst, const unsigned long *src,
+                         unsigned int bits);
+void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
+                  const unsigned long *bitmap2, unsigned int bits);
+void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
+                 const unsigned long *bitmap2, unsigned int bits);
+void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
+                  const unsigned long *bitmap2, unsigned int bits);
+void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
+                     const unsigned long *bitmap2, unsigned int bits);
+int __bitmap_intersects(const unsigned long *bitmap1,
+                        const unsigned long *bitmap2, unsigned int bits);
+int __bitmap_subset(const unsigned long *bitmap1,
+                    const unsigned long *bitmap2, unsigned int bits);
+unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int bits);
 extern void __bitmap_set(unsigned long *map, unsigned int start, int len);
 extern void __bitmap_clear(unsigned long *map, unsigned int start, int len);
 
@@ -117,7 +117,7 @@ static inline void bitmap_zero(unsigned long *dst, unsigned int nbits)
 		memset(dst, 0, bitmap_bytes(nbits)));
 }
 
-static inline void bitmap_fill(unsigned long *dst, int nbits)
+static inline void bitmap_fill(unsigned long *dst, unsigned int nbits)
 {
 	size_t nlongs = BITS_TO_LONGS(nbits);
 
@@ -224,7 +224,8 @@ static inline int bitmap_full(const unsigned long *src, unsigned int nbits)
 		return __bitmap_full(src, nbits));
 }
 
-static inline int bitmap_weight(const unsigned long *src, int nbits)
+static inline unsigned int bitmap_weight(const unsigned long *src,
+                                         unsigned int nbits)
 {
 	return __bitmap_weight(src, nbits);
 }
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 10:35:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 10:35:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674275.1049117 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUPZ-0006sh-5t; Thu, 01 Feb 2024 10:35:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674275.1049117; Thu, 01 Feb 2024 10:35:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUPZ-0006sa-39; Thu, 01 Feb 2024 10:35:13 +0000
Received: by outflank-mailman (input) for mailman id 674275;
 Thu, 01 Feb 2024 10:35:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i6jM=JK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVUPW-0006sM-U0
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 10:35:10 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 92445fb3-c0ed-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 11:35:09 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33ae6dfa923so452542f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 02:35:09 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 r6-20020adfca86000000b0033aed7423e8sm10424665wrh.11.2024.02.01.02.35.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 02:35:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92445fb3-c0ed-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706783708; x=1707388508; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=yCiKinlpzlts0E4HThCuxCG1NalXkAwCr46PyLENUPc=;
        b=M9LUOs617+AOXH3uMO0bt3UJqNwDW092MZ1CeIWqNSyBsrMqg8WnSkQsLJc0eIc2wc
         TXh5yxG8GFJwirJgCt9JPTRTlf4dIq9+XJF4Z8ppCfAGWox+IiCcemmh3yxd4B9WXBxM
         xOP3X7CGR24xD7gtp6dlTYvhddsHNpb7ahFoM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706783708; x=1707388508;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=yCiKinlpzlts0E4HThCuxCG1NalXkAwCr46PyLENUPc=;
        b=ENrgldqzaws79gRZqPLmGgwODhgWPAbdPv4TElQMxqrqorizD/DmITir1hsSpKHQHe
         WOATdRPWfBPvaQHEiM5NXw4zw6+iGKWs0c2LYaKPyXHzbOYux/AS1pCn0QIAkAar7tx9
         Cv8La08uzvFblNcSU/8BQwFJKTK2dQpLwKH6z407Yiuvme+wiH3tOU3ox+V5t19RqG4L
         m7xYnvZdINnQxUrUahR5Zfood1S50dlZw1LpqN/vL4CBelrFAf+N/u594AET49PYKWBw
         BExAtyNwfkkCoaJwPftlgtsCipSYshiMAEBHquO8pvBEPqRnhCKyMl4+Qbz6oTcVBjq+
         sCzw==
X-Gm-Message-State: AOJu0YwCGOzc0mqlTiPFBoE8tpzEAZHuf8dvHmYRZYojt8iq2E6jmMJG
	QHGvhqFClBzljekKbV/qNS9zvXnryLJK5lUp3aOmOKp+DBsrWHYcdm7oU+3ivq5OKDBQO1OYhqp
	g
X-Google-Smtp-Source: AGHT+IF3IYhsbx8io/sHFiJrAGGdpXKU7tkn0zeXJuTqKsE29NOH/Gha7ryNWAFLJlbJgM1L9l+h2A==
X-Received: by 2002:a5d:4145:0:b0:33a:e9b5:b14d with SMTP id c5-20020a5d4145000000b0033ae9b5b14dmr2981336wrq.65.1706783708297;
        Thu, 01 Feb 2024 02:35:08 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVyhCfZlgnvrfayBS3hs4UKFkj9Aq4QmIunFjb/f/qZoIKI+kW0GSETBvLqi9tfVYRiop8+ZAnArt+922jtvVmDIp+b98rGgDSJdmJI1LxZLfx2Fbwsw7C8UqItb0WR7NU66PdqUKP8VVzsARoEdK6qmKv/JJSKlpM7+wCPyObsuVrNVLXqc63+Rf/ocEdVTQ==
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH] xen/bitmap: Drop unused headers
Date: Thu,  1 Feb 2024 10:35:06 +0000
Message-Id: <20240201103506.549662-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Nothing in bitmap.h uses lib.h, and there's no point including types.h when we
need to include bitops.h anyway.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
---
 xen/include/xen/bitmap.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/xen/include/xen/bitmap.h b/xen/include/xen/bitmap.h
index b9f980e91930..9f749e3913d8 100644
--- a/xen/include/xen/bitmap.h
+++ b/xen/include/xen/bitmap.h
@@ -3,8 +3,6 @@
 
 #ifndef __ASSEMBLY__
 
-#include <xen/lib.h>
-#include <xen/types.h>
 #include <xen/bitops.h>
 
 /*
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 10:45:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 10:45:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674281.1049127 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUZc-0000VG-4D; Thu, 01 Feb 2024 10:45:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674281.1049127; Thu, 01 Feb 2024 10:45:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUZc-0000V9-1S; Thu, 01 Feb 2024 10:45:36 +0000
Received: by outflank-mailman (input) for mailman id 674281;
 Thu, 01 Feb 2024 10:45:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVUZb-0000V2-Ao
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 10:45:35 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 06d3df5d-c0ef-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 11:45:34 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-558f523c072so1060281a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 02:45:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 dh8-20020a0564021d2800b0055f0b3ec5d8sm3675264edb.36.2024.02.01.02.45.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 02:45:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06d3df5d-c0ef-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706784333; x=1707389133; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=36zQ2znP4kvPGoaJBxKSgCjs1iecbtwJ0jpIQ8VM5sM=;
        b=Di/Rw1OdfHBFgX9FwXAd0M/cffyCS9Gp/mLeBo3gMDfKy0Dt+N0fnCVprCV9GWx9nz
         w+IAslP8tn4fOFqgR6DB5Hw3aQFWJMgGP0wTPk/XAiBGFB4DDDSRRsb4F3ZggNF0M3+S
         QSkqRyvyZ9gUD9ieVLa835Ikleug9zH4gKQCbjwrA2L78icaqWXUSo3dU38B0KR7nUST
         v3Tgmg4Dgbvt/gjnYMlUefthmCyLMGnd4f67xStn9zRL6XDTF+m3OBBzGQeSVDIhszu7
         POmzDh2bEiWTL2N3N7WIoknJ1G0vkxlaxCe1wnypRcpL5SDwJuSlaaQ7R8Bf2+FhLsX+
         CrmQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706784333; x=1707389133;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=36zQ2znP4kvPGoaJBxKSgCjs1iecbtwJ0jpIQ8VM5sM=;
        b=YpCUPLc3ujlZU32ZJmiMy3iqhsImcdwQmILy2zxH9a8oYrqA/qczWyJrnKY1Wa0+Az
         TiZFBLpc2PpstaFmUxh4MDYrf5LMzpWcxD0PDuRfg+WGnRNszuesHt/mlrTcKXTTz2SA
         rHoWEuSq8aFB16wzWXSSwBmcXgVIKmc7bNVHcwejfcCHXxH6P17tO0BWbZexsUxgkswp
         VAoeJwTvqOU6q3Md9W7wamvmY+2HS6VUvq9wPsHv8qHAJX6X61tuEQuG2Te7UfdmciqD
         RXlUlabOiSIF+P7VGUv/kw2uyBcEptbx+L0nExNAt7xt09rOWRzH2B1ki0CEwDxV5BmA
         5Y4Q==
X-Gm-Message-State: AOJu0Yxbp8SI73KmoOEgAi4by0nlZdxSKbfWC0pdXUWAf52DibPDan8k
	Snrok3d3sVn0H5tCwEVgKTfHtKBT++2URq515nUNE49A5TBlVHLfyI1YA9f1pQ==
X-Google-Smtp-Source: AGHT+IHoKcEQI4G5dKMR9mfNE5wCzl3xe9xM2sdmat0isSflfGWkENqPDprgtmfVkSpMmKczQoqYnA==
X-Received: by 2002:a05:6402:3126:b0:55f:7b91:cbd5 with SMTP id dd6-20020a056402312600b0055f7b91cbd5mr3283824edb.8.1706784333612;
        Thu, 01 Feb 2024 02:45:33 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWnqi44md3/rfvumnOnsPaJCA87tDh4NdQ44RQilWZKSX4l1wjKYee6wY5TPWL3POn+igT3rDAnhBFIXKm2kiwdoy/g729lqytUkN1Ogd0HNOh7c/vgqzkioWC83osRUUYjrWLoBbo7MRERvMgMd6CT9uiDyYaR5K3p0ISmszv5XU63
Message-ID: <88b0e4bb-7f1c-4dc6-a22b-42c5dcd2ca46@suse.com>
Date: Thu, 1 Feb 2024 11:45:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/bitmap: Consistently use unsigned bits values
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240201103339.549307-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240201103339.549307-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.02.2024 11:33, Andrew Cooper wrote:
> Right now, most of the static inline helpers take an unsigned nbits quantity,
> and most of the library functions take a signed quanity.  Because
> BITMAP_LAST_WORD_MASK() is expressed as a divide, the compiler is forced to
> emit two different paths to get the correct semantics for signed division.
> 
> Swap all signed bit-counts to being unsigned bit-counts for the simple cases.
> This includes the return value of bitmap_weight().
> 
> Bloat-o-meter for a random x86 build reports:
>   add/remove: 0/0 grow/shrink: 8/19 up/down: 167/-413 (-246)
> 
> which all comes from compiler not emitting "dead" logic paths for negative bit
> counts.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
albeit with a question at the bottom.

> There is much more wanting cleaning up here, but we have to start somewhere.
> Some observations:
> 
>  * Various of the boolean-like return values have -1 for zero-length bitmaps.
>    I can't spot any callers which care, so this seems like a waste.
>  * bitmap_zero() and similar clear predate us switching to use
>    __builtin_memset(), because there's no need for bitmap_switch().
>  * Should we consolidate 'bits' vs 'nbits'?

This looks desirable to me.

>  * The internals of these helpers want converting too.  Other helpers need
>    more than just a parameter conversion.

This may or may not relate to my question; it's not exactly clear what you
mean here.

> --- a/xen/include/xen/bitmap.h
> +++ b/xen/include/xen/bitmap.h
> @@ -66,25 +66,25 @@
>   * lib/bitmap.c provides these functions:
>   */
>  
> -extern int __bitmap_empty(const unsigned long *bitmap, int bits);
> -extern int __bitmap_full(const unsigned long *bitmap, int bits);
> -extern int __bitmap_equal(const unsigned long *bitmap1,
> -                	const unsigned long *bitmap2, int bits);
> -extern void __bitmap_complement(unsigned long *dst, const unsigned long *src,
> -			int bits);
> -extern void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
> -			const unsigned long *bitmap2, int bits);
> -extern void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
> -			const unsigned long *bitmap2, int bits);
> -extern void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
> -			const unsigned long *bitmap2, int bits);
> -extern void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
> -			const unsigned long *bitmap2, int bits);
> -extern int __bitmap_intersects(const unsigned long *bitmap1,
> -			const unsigned long *bitmap2, int bits);
> -extern int __bitmap_subset(const unsigned long *bitmap1,
> -			const unsigned long *bitmap2, int bits);
> -extern int __bitmap_weight(const unsigned long *bitmap, int bits);
> +int __bitmap_empty(const unsigned long *bitmap, unsigned int bits);
> +int __bitmap_full(const unsigned long *bitmap, unsigned int bits);
> +int __bitmap_equal(const unsigned long *bitmap1,
> +                   const unsigned long *bitmap2, unsigned int bits);
> +void __bitmap_complement(unsigned long *dst, const unsigned long *src,
> +                         unsigned int bits);
> +void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
> +                  const unsigned long *bitmap2, unsigned int bits);
> +void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
> +                 const unsigned long *bitmap2, unsigned int bits);
> +void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
> +                  const unsigned long *bitmap2, unsigned int bits);
> +void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
> +                     const unsigned long *bitmap2, unsigned int bits);
> +int __bitmap_intersects(const unsigned long *bitmap1,
> +                        const unsigned long *bitmap2, unsigned int bits);
> +int __bitmap_subset(const unsigned long *bitmap1,
> +                    const unsigned long *bitmap2, unsigned int bits);
> +unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int bits);
>  extern void __bitmap_set(unsigned long *map, unsigned int start, int len);
>  extern void __bitmap_clear(unsigned long *map, unsigned int start, int len);

What about these two, and the subsequent (in the .c file at least)
bitmap_*_region()? That last remark above may mean you deliberately
left them out for now (which is okay - as you say, this is merely a
1st step).

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 10:46:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 10:46:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674283.1049138 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUa7-0000xl-D1; Thu, 01 Feb 2024 10:46:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674283.1049138; Thu, 01 Feb 2024 10:46:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUa7-0000xe-98; Thu, 01 Feb 2024 10:46:07 +0000
Received: by outflank-mailman (input) for mailman id 674283;
 Thu, 01 Feb 2024 10:46:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVUa6-0000wJ-GW
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 10:46:06 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 19a9aead-c0ef-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 11:46:05 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-5112cb7ae27so1400712e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 02:46:05 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 dh8-20020a0564021d2800b0055f0b3ec5d8sm3675264edb.36.2024.02.01.02.46.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 02:46:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19a9aead-c0ef-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706784365; x=1707389165; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HdmZFcuuIPK1Q90rspQKvLqZJBvx+8Rnf5ur+I73X64=;
        b=U8ROuIUHlCd+99sdwiitG1gmKnx7AxbYdA0ahc97jH5zq2dzChb3TedZDAMMyyskj3
         mMyxTAg+Pj4oBO4n1sztYqNqVpV2ISEPDaJFpHXX/WxJ4YHfFj419DZExd/gwZnBy7Qg
         Oz3T75gM8vWD/ocnyew3MCqQDVEnn4nvg26G/GgEYmWibLbPvd8v9UhO/1balcPyU4HR
         vWUVl4mqB+Ku0KYbbl8p6eCBtvQGVLg5akA9YSf+CZBabqDYimTvoR6YxxWMnrKy5Kgc
         kpqYbbym8Wc+JuK8UKz4iwzYlyOLMdbGHj5sn9w/7fpsUGHUS00D+iTlYWZ8/bAMp+a6
         lZLQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706784365; x=1707389165;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HdmZFcuuIPK1Q90rspQKvLqZJBvx+8Rnf5ur+I73X64=;
        b=tiYqlbXqvthpm+KCDMiLMSghpx4nDK6FQGlG0/KPoLqsJnh/68TdK6VIANT4PCYCTg
         2wy4s4VCVihICI705cjfIxfCKz7F6scW7iR+AYox6vQq+HTMyAKPz0C2wynudjvpeHla
         uN8RvssRNlZSQXR9Ch5fmIt7E957kGP55xVHw+afKvHV3gzu//E29HM87j8zrw5QiiCZ
         yN4zNu0g5YgBKs+nUBuuRUiHRcr8nkgvFD0/zEEjVzACZPRZ+2+rCLoeeiMvR7vqIqkR
         gJVMHqIknS1dzjjE0KRsSE1meCVaH+xxL8GQgKdLmhUshzQkYivLfjOB9gERte2BAZ0d
         Laeg==
X-Gm-Message-State: AOJu0Yz6jP5dPhc5EfRIyFr3jaet7GUv9VL6S8jY7aV4tmdDl4ZbZFP/
	uoBSSUSIEeWeRf1Kysq/03lq6zjkvTlJyXVnB8jG3jY2lvdmctC6ZlCQMO0VSg==
X-Google-Smtp-Source: AGHT+IEtw/eyfacwle68fAA8jyfPWhq2Wtd9UuM/AS6Yf6sX+JkhGtqEZMHo1ubXosn5shU/MUCK5Q==
X-Received: by 2002:a05:651c:2105:b0:2d0:7590:7dd6 with SMTP id a5-20020a05651c210500b002d075907dd6mr1340506ljq.5.1706784365185;
        Thu, 01 Feb 2024 02:46:05 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVquoUlq88gZmNPZqxy2SO3va5yUl9GUEiWzOCfruj8wzBq+Gm1asGPTelwtOccYJRG6dsXHB02w2mZ//OTcYd2v0J89gAgJIMIBmrg9XW8IRKRdbytszfQ80E5Vb1mrP2X0/2Gv6wGsy2Wgk1aBobG6KCyio+jgciywaS0vWqZTRn4
Message-ID: <81820a24-03c1-4926-bd0c-5c21ed982271@suse.com>
Date: Thu, 1 Feb 2024 11:46:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/bitmap: Drop unused headers
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240201103506.549662-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240201103506.549662-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.02.2024 11:35, Andrew Cooper wrote:
> Nothing in bitmap.h uses lib.h, and there's no point including types.h when we
> need to include bitops.h anyway.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Thu Feb 01 10:51:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 10:51:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674289.1049148 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUfO-0003Aj-VN; Thu, 01 Feb 2024 10:51:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674289.1049148; Thu, 01 Feb 2024 10:51:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUfO-0003Ac-Ry; Thu, 01 Feb 2024 10:51:34 +0000
Received: by outflank-mailman (input) for mailman id 674289;
 Thu, 01 Feb 2024 10:51:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9oxY=JK=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1rVUfN-00039n-Fr
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 10:51:33 +0000
Received: from mail-ej1-x643.google.com (mail-ej1-x643.google.com
 [2a00:1450:4864:20::643])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dc50aee3-c0ef-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 11:51:32 +0100 (CET)
Received: by mail-ej1-x643.google.com with SMTP id
 a640c23a62f3a-a36126ee41eso94269566b.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 02:51:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc50aee3-c0ef-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1706784692; x=1707389492; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=/e2n3brzjS7sgJpoVZMw+wyx5Jd+q7mbKNa90maiaTQ=;
        b=dhCn9V9PyHw/jMSSThJKcZazcGT0Ygrk9XRlz1ekQnVK/DqKL+ZEyhd1so3LFbKSdK
         8f5vqBuvT6fjQvvR1TvxNKfB4AzVPWJMCZgxkBOq3w4Nq7Lkm5p846MNWjLdM0N8mG/P
         y++MTWL+J1lJgJ6umQGZKPHS3euwd6w76Bgms=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706784692; x=1707389492;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/e2n3brzjS7sgJpoVZMw+wyx5Jd+q7mbKNa90maiaTQ=;
        b=qJobS1l2p+MR6g+GD1nl/X2UtCyEVEzi8JjZA1eX//MwHaQ4/cb4TXfNCNTvyc0mSS
         +DnTobC52jipNJyI4FFQi0VTDIoJbqTRaCY1S1azUhaVLxURfB8oemoB6dierJ2QteGQ
         +3FBLaEHqoR/3lbdBUU1AoBjM4gK4cvwthWF5qVD3Q4FnTRWWf5RttJjN+lECXMhL2/1
         lQJuVe17It5uPBP/54+HLsf9ltxGjqHMQ/v/1vQ5ljP4A73Px5vAe+WTc0hWtEuzAllz
         50eQ13CGc5gQertXwCRS8ycOyeXTJdPM1kmz1b/2YVVxcV5qQXZphJvF9Lra/uBoxyH6
         fOkw==
X-Gm-Message-State: AOJu0Yz7xDIprxD8PnECTrPPIoMQugcP+hURP+xsggOum85h5IsQJAGI
	sjPMV9n/ABUCO+9GtHEMKi63F7GYhcNKHQanVLQHYf459iIzlnQj0muKhsfL8OqIHOn/HT2XvY+
	r6MT7F6EanHNxeDn75kt80cxXtQ50Ofbm0ruTJN84Ju2TsKkXfRIOoUrl
X-Google-Smtp-Source: AGHT+IEXayaWwGkYdvNw1Kt8cebqMKuLrh2e2Ffv7xtSrxXGNYNcl4BoF9N5/TgmA9WxTDbpmM0GBh/Od+Ej/yB/fPI=
X-Received: by 2002:a17:906:6857:b0:a35:5b6:1e0f with SMTP id
 a23-20020a170906685700b00a3505b61e0fmr3210597ejs.71.1706784691644; Thu, 01
 Feb 2024 02:51:31 -0800 (PST)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Thu, 1 Feb 2024 10:50:55 +0000
Message-ID: <CAO-mL=zKzc9BJ5+emfq+L-m5+3t83TYOQmsFNWuKNPvYnJpqcA@mail.gmail.com>
Subject: Upcoming Social Event - Location!
To: xen-devel@lists.xenproject.org, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Content-Type: multipart/alternative; boundary="00000000000040f1ec06104fc74e"

--00000000000040f1ec06104fc74e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi all,

I'm pleased to announce the location of our next social event!
Join and chat with the Xen community, there will also be free food and
drinks.

*Date:* Wednesday 21st February 2024

*Time:* 5:30pm - 9pm (you are welcome to stay later if you wish)

*Location:*
The Portland Arms
129 Chesterton Rd Cambridge
www.theportlandarms.co.uk
T: 01223 357268

*Getting to the venue (more information here
<https://theportlandarms.co.uk/wp/access-info/>):*
There are pay and display spaces (free after 5pm) on Milton Road to the
East of us or opposite on Chesterton Road. The nearest bus stops are:
Chesterton Road, opposite the pub outside Hing Hung restaurant, and on
Milton Road, near Westbrook centre, Citi 1, 9, X9 and The Busway. Cambridge
North rail station is 2.3 miles away and Cambridge Central station is 2.4
miles.

*To anticipate the number of attendees, please email me indicating you wish
to attend. *
*If you have any food allergies, let me know ahead of time. *

Many thanks,
Kelly Choi

Community Manager
Xen Project


---------- Forwarded message ---------
From: Kelly Choi <kelly.choi@cloud.com>
Date: Thu, Nov 23, 2023 at 1:01=E2=80=AFPM
Subject: The Xen Project=E2=80=99s 20th Anniversary - Upcoming Social Event=
!
To: <advisory-board@lists.xenproject.org>, <xen-devel@lists.xenproject.org>=
,
<xen-users@lists.xenproject.org>


*The Xen Project=E2=80=99s 20th Anniversary*

*Let's get together for an informal social, likely to be pizza/drinks and
getting involved with the community. I hope to run these in future
locations to give everyone a chance to attend.*

*Date placeholder: Wednesday 21st February 2024*
*Location: Cambridge*
*Details TBC - **If you're interested, please reply to me directly and I
will add you to the list.*

*Celebrating Two Decades of Innovation*

It=E2=80=99s hard to believe that two decades have passed since the incepti=
on of
the Xen Project, a trailblazing force in the world of open-source
virtualization. As we raise our glasses to commemorate this momentous
occasion, it=E2=80=99s not just a celebration of time but a reflection on t=
he
incredible journey that has defined the Xen Project=E2=80=99s legacy.

*A Legacy of Innovation*

In the year 2003, the Xen Project emerged as a pioneering open-source
hypervisor, laying the groundwork for some of the most influential cloud
infrastructures that shape our digital landscape today. Over the past 20
years, the Xen Project has not only endured but has thrived, continuously
evolving to meet the dynamic demands of the ever-changing tech landscape.

*Driving Technological Frontiers*

>From data center and server virtualization to cloud computing, desktop
virtualization, and fortifying desktop security and hardware appliances,
the Xen Project has been at the forefront of driving technological
innovation. With 20 years of relentless development, it has become
synonymous with reliability, scalability, and adaptability.

*Venturing into New Horizons*

As we celebrate this milestone, we also look forward to the exciting new
territories that the Xen Project is venturing into. From embedded
virtualization to even making strides in the automotive industry, the Xen
Project continues to push boundaries and redefine what=E2=80=99s possible i=
n the
world of open-source virtualization.

*The Annual Event: Xen Project Developer and Design Summit*

At the heart of this remarkable journey is the Xen Project Developer and
Design Summit, an annual gathering of the community=E2=80=99s brilliant min=
ds and
power users. More than just a conference, it=E2=80=99s a celebration of ide=
a
exchange, a showcase of the latest advancements, a platform for sharing
invaluable experiences, and a forum for strategic planning and
collaborative efforts. Be sure to look out for our upcoming event in 2024.

*A Vibrant Community Defining the Future*

Beyond the code and technological achievements, the Xen Project=E2=80=99s s=
trength
lies in its vibrant community. It=E2=80=99s a community that has come toget=
her to
celebrate successes, overcome challenges, and collectively shape the future
of open-source virtualization technology. Even to this day, community
contributions and reviews are still going!

*Looking Ahead*

As we commemorate 20 years of innovation, we also eagerly anticipate the
next chapter in the Xen Project=E2=80=99s journey. With gratitude for the p=
ast and
excitement for the future, we extend our deepest thanks to everyone who has
contributed to this incredible legacy.

Here=E2=80=99s to 20 years of pushing boundaries, fostering collaboration, =
and
shaping the digital landscape.

Happy anniversary, Xen Project! The best is yet to come and I can=E2=80=99t=
 wait to
see what we all achieve.

Many thanks,
Kelly Choi

Open Source Community Manager
XenServer, Cloud Software Group

--00000000000040f1ec06104fc74e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi all,=C2=A0<div><br></div><div>I&#39;m pleased to announ=
ce the location of our next social event!</div><div>Join and chat with the =
Xen community, there will also be free food and drinks.</div><div><br></div=
><div><b>Date:</b> Wednesday 21st February 2024<br><b><br></b></div><div><b=
>Time:</b> 5:30pm - 9pm (you are welcome to stay later if you wish)<br></di=
v><b><div><b><br></b></div>Location:</b><br>The Portland Arms<br>129 Cheste=
rton Rd Cambridge<br><a href=3D"http://www.theportlandarms.co.uk" target=3D=
"_blank">www.theportlandarms.co.uk</a><br>T: 01223 357268<div><br></div><di=
v><b>Getting to the venue (<a href=3D"https://theportlandarms.co.uk/wp/acce=
ss-info/" target=3D"_blank">more information here</a>):</b></div><div>There=
 are pay and display spaces (free after 5pm) on Milton Road to the East of =
us or opposite on Chesterton Road. The nearest bus stops are: Chesterton Ro=
ad, opposite the pub outside Hing Hung restaurant, and on Milton Road, near=
 Westbrook centre, Citi 1, 9, X9 and The Busway. Cambridge North rail stati=
on is 2.3 miles away and Cambridge Central station is 2.4 miles.=C2=A0</div=
><br><div><div><div><b>To anticipate the number of attendees, please email =
me=C2=A0indicating you wish to attend.=C2=A0</b></div><div><b>If you have a=
ny food allergies, let me know ahead of time.=C2=A0</b></div><div><br></div=
><div><div dir=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_si=
gnature"><div dir=3D"ltr"><div>Many thanks,</div><div>Kelly Choi</div><div>=
<br></div><div><div style=3D"color:rgb(136,136,136)">Community Manager</div=
><div style=3D"color:rgb(136,136,136)">Xen Project=C2=A0<br></div></div></d=
iv></div></div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D=
"gmail_attr">---------- Forwarded message ---------<br>From: <strong class=
=3D"gmail_sendername" dir=3D"auto">Kelly Choi</strong> <span dir=3D"auto">&=
lt;<a href=3D"mailto:kelly.choi@cloud.com" target=3D"_blank">kelly.choi@clo=
ud.com</a>&gt;</span><br>Date: Thu, Nov 23, 2023 at 1:01=E2=80=AFPM<br>Subj=
ect: The Xen Project=E2=80=99s 20th Anniversary - Upcoming Social Event! <b=
r>To:  &lt;<a href=3D"mailto:advisory-board@lists.xenproject.org" target=3D=
"_blank">advisory-board@lists.xenproject.org</a>&gt;,  &lt;<a href=3D"mailt=
o:xen-devel@lists.xenproject.org" target=3D"_blank">xen-devel@lists.xenproj=
ect.org</a>&gt;,  &lt;<a href=3D"mailto:xen-users@lists.xenproject.org" tar=
get=3D"_blank">xen-users@lists.xenproject.org</a>&gt;<br></div><br><br><div=
 dir=3D"ltr"><div><b>The Xen Project=E2=80=99s 20th Anniversary</b></div><d=
iv><br></div><div><i>Let&#39;s get=C2=A0together for an informal social, li=
kely to be pizza/drinks and getting involved with the community. I hope to =
run these in future locations to give everyone a chance to attend.</i></div=
><div><i><br></i></div><div><i><b>Date placeholder: Wednesday 21st February=
 2024</b></i></div><div><i><b>Location: Cambridge</b></i></div><div><i><b>D=
etails TBC -=C2=A0</b></i><b><font color=3D"#ff0000"><i>If you&#39;re inter=
ested, please reply to me directly and I will add you to the list.</i></fon=
t></b></div><div><br></div><div><b>Celebrating Two Decades of Innovation</b=
><br></div><div><b><br></b></div><div>It=E2=80=99s hard to believe that two=
 decades have passed since the inception of the Xen Project, a trailblazing=
 force in the world of open-source virtualization. As we raise our glasses =
to commemorate this momentous occasion, it=E2=80=99s not just a celebration=
 of time but a reflection on the incredible journey that has defined the Xe=
n Project=E2=80=99s legacy.<br><b><br>A Legacy of Innovation</b><br><br>In =
the year 2003, the Xen Project emerged as a pioneering open-source hypervis=
or, laying the groundwork for some of the most influential cloud infrastruc=
tures that shape our digital landscape today. Over the past 20 years, the X=
en Project has not only endured but has thrived, continuously evolving to m=
eet the dynamic demands of the ever-changing tech landscape.<br><b><br>Driv=
ing Technological Frontiers</b><br><br>From data center and server virtuali=
zation to cloud computing, desktop virtualization, and fortifying desktop s=
ecurity and hardware appliances, the Xen Project has been at the forefront =
of driving technological innovation. With 20 years of relentless developmen=
t, it has become synonymous with reliability, scalability, and adaptability=
.<br><b><br>Venturing into New Horizons</b><br><br>As we celebrate this mil=
estone, we also look forward to the exciting new territories that the Xen P=
roject is venturing into. From embedded virtualization to even making strid=
es in the automotive industry, the Xen Project continues to push boundaries=
 and redefine what=E2=80=99s possible in the world of open-source virtualiz=
ation.<br><b><br>The Annual Event: Xen Project Developer and Design Summit<=
/b><br><br>At the heart of this remarkable journey is the Xen Project Devel=
oper and Design Summit, an annual gathering of the community=E2=80=99s bril=
liant minds and power users. More than just a conference, it=E2=80=99s a ce=
lebration of idea exchange, a showcase of the latest advancements, a platfo=
rm for sharing invaluable experiences, and a forum for strategic planning a=
nd collaborative efforts. Be sure to look out for our upcoming event in 202=
4.<br><b><br>A Vibrant Community Defining the Future</b><br><br>Beyond the =
code and technological achievements, the Xen Project=E2=80=99s strength lie=
s in its vibrant community. It=E2=80=99s a community that has come together=
 to celebrate successes, overcome challenges, and collectively shape the fu=
ture of open-source virtualization technology. Even to this day, community =
contributions and reviews are still going!<br><br><b>Looking Ahead</b><br><=
br>As we commemorate 20 years of innovation, we also eagerly anticipate the=
 next chapter in the Xen Project=E2=80=99s journey. With gratitude for the =
past and excitement for the future, we extend our deepest thanks to everyon=
e who has contributed to this incredible legacy.<br><br>Here=E2=80=99s to 2=
0 years of pushing boundaries, fostering collaboration, and shaping the dig=
ital landscape.<br><br>Happy anniversary, Xen Project! The best is yet to c=
ome and I can=E2=80=99t wait to see what we all achieve.<br></div><div><br>=
</div><div><div dir=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gma=
il_signature"><div dir=3D"ltr"><div>Many thanks,</div><div>Kelly Choi</div>=
<div><br></div><div><div style=3D"color:rgb(136,136,136)">Open Source Commu=
nity Manager</div><div style=3D"color:rgb(136,136,136)">XenServer, Cloud So=
ftware Group</div></div></div></div></div></div>
</div></div></div></div>

--00000000000040f1ec06104fc74e--


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 10:54:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 10:54:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674322.1049187 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUhg-0005D0-1l; Thu, 01 Feb 2024 10:53:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674322.1049187; Thu, 01 Feb 2024 10:53:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUhf-0005Ct-VQ; Thu, 01 Feb 2024 10:53:55 +0000
Received: by outflank-mailman (input) for mailman id 674322;
 Thu, 01 Feb 2024 10:53:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i6jM=JK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVUhe-0005Cb-OC
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 10:53:54 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2ff1208d-c0f0-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 11:53:52 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33ae4eb360aso460812f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 02:53:52 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 cx18-20020a056000093200b0033935779a23sm15819798wrb.89.2024.02.01.02.53.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 02:53:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ff1208d-c0f0-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706784832; x=1707389632; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=P/7t2W/nuc1bPI8HTUKEuA5/guWmVVdGl+fvEX/9WfI=;
        b=F7AfHFIYSaesCkMjbBcDUbnQQup/J29SqDcBMBr9yriP/QeKJpattuGKiryETd36Gc
         g6i4Cqqidn9nOxq1HtG6ZM2uqYdPf87pIrTR4xAqRbpmxn1dFe7vvGsAMIOzntrfzWxr
         mBBiywqyCes1iCsaLa8kcpRcGhkJd8iLTSWf0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706784832; x=1707389632;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=P/7t2W/nuc1bPI8HTUKEuA5/guWmVVdGl+fvEX/9WfI=;
        b=wHj6zGq7EK00exHYr2GT+wDmgVCZmPvFmu9tMOQJfX2auYacvlyCAebt5qL9z1UUH8
         423cFZnjpE4BatbCrF1+4mLW0nUTJbDQWpiAOki1hooYIx9yj83YGf//taLd7uhTKJg0
         9A6Ci06qhwDguqDNGewC2DgY4JqeZ0aVIFIc0xaD3fsyC4sFclwOkhWXBjvl7UwaryZl
         2HKFPhyk5SirCQRpXOtfr2smSusy0LMZlB/4mQhHfcxNCoIW8N5QVfP9bZvjuSoxP/za
         KORR3dHF3/Q12lh0SEaMwCcUjaLdeSDiP4VMwfI5hevL3+Xz53UaZLFmLDsoaCYBCvj6
         MQyQ==
X-Gm-Message-State: AOJu0Yyq3liQVDqpY1kgANt4K3vTaaXyAVBZ2Vgx67NG+UAHGjd7lbF6
	uCBcWsVRJKe51AcGZ3ntBWHwFJsoG4vKxaomNJqUu/yrUX9aXq4+N+DzD8/RAvQ=
X-Google-Smtp-Source: AGHT+IG0qUNhArOIAClSjVvblXAHmw0o/Fns4eTQZQNNsvsb4fLEMsEMpFPi3EshofloyGkaBfMWmA==
X-Received: by 2002:a5d:4710:0:b0:33a:ee33:ae9b with SMTP id y16-20020a5d4710000000b0033aee33ae9bmr3841202wrq.32.1706784832032;
        Thu, 01 Feb 2024 02:53:52 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUrfGCGQDe/ZzFIomh2xIw9PFGV+W3jUNe4mE3wWF3gp34F77j6V9/1w/zZLFQg3N7a/UtKhcHRCIyk2tA97VsLmjZcvDpm62jCwAvVo712a6Lykv5hQKKVA9SECUD8gnKFvH5RYnyF8VNRUIhQbBMvl9ANtxTcg9k4HZtE9/FXC+pf
Message-ID: <7b6e70a9-f19b-4487-ae92-6480ff0edfd5@citrix.com>
Date: Thu, 1 Feb 2024 10:53:51 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/bitmap: Consistently use unsigned bits values
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240201103339.549307-1-andrew.cooper3@citrix.com>
 <88b0e4bb-7f1c-4dc6-a22b-42c5dcd2ca46@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <88b0e4bb-7f1c-4dc6-a22b-42c5dcd2ca46@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01/02/2024 10:45 am, Jan Beulich wrote:
> On 01.02.2024 11:33, Andrew Cooper wrote:
>> Right now, most of the static inline helpers take an unsigned nbits quantity,
>> and most of the library functions take a signed quanity.  Because
>> BITMAP_LAST_WORD_MASK() is expressed as a divide, the compiler is forced to
>> emit two different paths to get the correct semantics for signed division.
>>
>> Swap all signed bit-counts to being unsigned bit-counts for the simple cases.
>> This includes the return value of bitmap_weight().
>>
>> Bloat-o-meter for a random x86 build reports:
>>   add/remove: 0/0 grow/shrink: 8/19 up/down: 167/-413 (-246)
>>
>> which all comes from compiler not emitting "dead" logic paths for negative bit
>> counts.
>>
>> No functional change.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> albeit with a question at the bottom.
>
>> There is much more wanting cleaning up here, but we have to start somewhere.
>> Some observations:
>>
>>  * Various of the boolean-like return values have -1 for zero-length bitmaps.
>>    I can't spot any callers which care, so this seems like a waste.
>>  * bitmap_zero() and similar clear predate us switching to use
>>    __builtin_memset(), because there's no need for bitmap_switch().
>>  * Should we consolidate 'bits' vs 'nbits'?
> This looks desirable to me.
>
>>  * The internals of these helpers want converting too.  Other helpers need
>>    more than just a parameter conversion.
> This may or may not relate to my question; it's not exactly clear what you
> mean here.
>
>> --- a/xen/include/xen/bitmap.h
>> +++ b/xen/include/xen/bitmap.h
>> @@ -66,25 +66,25 @@
>>   * lib/bitmap.c provides these functions:
>>   */
>>  
>> -extern int __bitmap_empty(const unsigned long *bitmap, int bits);
>> -extern int __bitmap_full(const unsigned long *bitmap, int bits);
>> -extern int __bitmap_equal(const unsigned long *bitmap1,
>> -                	const unsigned long *bitmap2, int bits);
>> -extern void __bitmap_complement(unsigned long *dst, const unsigned long *src,
>> -			int bits);
>> -extern void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
>> -			const unsigned long *bitmap2, int bits);
>> -extern void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
>> -			const unsigned long *bitmap2, int bits);
>> -extern void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
>> -			const unsigned long *bitmap2, int bits);
>> -extern void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
>> -			const unsigned long *bitmap2, int bits);
>> -extern int __bitmap_intersects(const unsigned long *bitmap1,
>> -			const unsigned long *bitmap2, int bits);
>> -extern int __bitmap_subset(const unsigned long *bitmap1,
>> -			const unsigned long *bitmap2, int bits);
>> -extern int __bitmap_weight(const unsigned long *bitmap, int bits);
>> +int __bitmap_empty(const unsigned long *bitmap, unsigned int bits);
>> +int __bitmap_full(const unsigned long *bitmap, unsigned int bits);
>> +int __bitmap_equal(const unsigned long *bitmap1,
>> +                   const unsigned long *bitmap2, unsigned int bits);
>> +void __bitmap_complement(unsigned long *dst, const unsigned long *src,
>> +                         unsigned int bits);
>> +void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
>> +                  const unsigned long *bitmap2, unsigned int bits);
>> +void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
>> +                 const unsigned long *bitmap2, unsigned int bits);
>> +void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
>> +                  const unsigned long *bitmap2, unsigned int bits);
>> +void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
>> +                     const unsigned long *bitmap2, unsigned int bits);
>> +int __bitmap_intersects(const unsigned long *bitmap1,
>> +                        const unsigned long *bitmap2, unsigned int bits);
>> +int __bitmap_subset(const unsigned long *bitmap1,
>> +                    const unsigned long *bitmap2, unsigned int bits);
>> +unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int bits);
>>  extern void __bitmap_set(unsigned long *map, unsigned int start, int len);
>>  extern void __bitmap_clear(unsigned long *map, unsigned int start, int len);
> What about these two, and the subsequent (in the .c file at least)
> bitmap_*_region()? That last remark above may mean you deliberately
> left them out for now (which is okay - as you say, this is merely a
> 1st step).

They want map->bitmap and int len -> unsigned int bits for consistency,
which is more than just a typechange.

I also wasn't totally certain of the correctness of the internal logic. 
I don't have time to investigate further in the immediate future, so
left it unchanged out of caution.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 10:57:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 10:57:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674335.1049198 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUkp-00062W-GQ; Thu, 01 Feb 2024 10:57:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674335.1049198; Thu, 01 Feb 2024 10:57:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVUkp-00062P-Bp; Thu, 01 Feb 2024 10:57:11 +0000
Received: by outflank-mailman (input) for mailman id 674335;
 Thu, 01 Feb 2024 10:57:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVUko-00062J-9f
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 10:57:10 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a4759ebf-c0f0-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 11:57:08 +0100 (CET)
Received: by mail-wr1-x42c.google.com with SMTP id
 ffacd0b85a97d-33b1117bbe4so468501f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 02:57:08 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h15-20020a05600016cf00b0033aedfc5581sm10506517wrf.32.2024.02.01.02.57.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 02:57:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a4759ebf-c0f0-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706785027; x=1707389827; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wHfXA9CWPeygkYwIqkcdbd05IHnuqGa1RECkAr53WVo=;
        b=asFPrVO7b0QjErlBxkbHr6Y6E3/6ovF/W2NHKSt31VKjFD3aiw4LlUxdJsLE4s5ZVH
         4cNAuRwcnSUoQfrZZS6NLDgiJlgYiuQVuasNKQBgGW3xHhRsPu4PgHO3R+T1YmoJItFs
         EXyKYVm9jEG5lXYHW+9N/bv1BAiXtOCqjg3le0qMQBfon2NLZ+rKkEsYcR7yznOgn+yD
         EIrZSBVikXvYcUnK3a/JUjrMX6j1qG+rWPxHWQ9U5GI8x7fGwtj1WqBMNx6qZmK9ya4I
         BzuzRs1r1gmEOhOHVR9X0ZxjUdwE7nQRXLcm7DnLXt/NsdENKLltIGIKu90nsgP//60P
         Ar6g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706785027; x=1707389827;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wHfXA9CWPeygkYwIqkcdbd05IHnuqGa1RECkAr53WVo=;
        b=UFdpmo3QfsiBcFpYTUo6o2eyVIyiQinXGfE4u5acHW3us9PzgQX0gnb4cbs811+GzT
         aSgwBc00ViGmc6LZmUfsu6QAhZEieQM7DNfCBtWwbnK3ym3QRbdT96zT7MrM53hNBFn6
         FHOuuRUcXlyoF5S8caJZaYbm+hdV0o4iach6VZAJDs2YpmvO0LCv2w1gYVd2Rt7ElrA/
         qgWfwqHpTXgLESx/AY7J2eDiYBDDEDQVF3+DdhvEcud4m7rOwKxvxYL+51RGO2IMLc17
         iQDHWhqefSBI5bxz99+rPKzD3GlyQcDxEIll5C0FwLYhBKOkuYjJaXoaJklgkTxgx6ta
         X3Gw==
X-Gm-Message-State: AOJu0Yys7yFUsV6a9VKF9iZY5XJmxuQsDEfWsRSqcG+1NwApytmZ0k+I
	zVnLv9F03Budn4+PtiA2fQ5qmWfkqA3g3xUiOveBKNiI0V0QzH1z+0KCoauFVw==
X-Google-Smtp-Source: AGHT+IFanJa09jb1jj3FfMdUHrewWqzuq4tYduICl2RdJ1jrJaEwQgKLGHt9+Y7Xjaxk4/vTfBTB4Q==
X-Received: by 2002:a05:6000:18ad:b0:33a:f3dd:97e9 with SMTP id b13-20020a05600018ad00b0033af3dd97e9mr3796510wri.40.1706785027455;
        Thu, 01 Feb 2024 02:57:07 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUuDNLxcpFtX35VuilYbBCDQl1DrMo5MLB0ykN0JZyRDKlwIC7hZBaFo2S9UEEDblvWSIQ05zpvYiHc9Rz4I1k5VEbIp9+nuYOjhy/KFsM=
Message-ID: <5cda514b-4b6f-4259-9dbc-bb1462eec13e@suse.com>
Date: Thu, 1 Feb 2024 11:57:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Issue iommu unrecognized on amd computer
Content-Language: en-US
To: oxjo@proton.me
Cc: "roger.pau@citrix.com" <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <qTUd8Z_TU2_jJzz9kTj2DJK54p4wECUT072VD_1nomh-d8Ej6hC-QNzY6BW6G9bQVdcDJuQbapJ7NlqqOAKd49Iu8rgkyt4E9qdnN_SyZj8=@proton.me>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <qTUd8Z_TU2_jJzz9kTj2DJK54p4wECUT072VD_1nomh-d8Ej6hC-QNzY6BW6G9bQVdcDJuQbapJ7NlqqOAKd49Iu8rgkyt4E9qdnN_SyZj8=@proton.me>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.02.2024 09:16, oxjo@proton.me wrote:
> Following our interaction on matrix, I send you the boot log `xl dmesg` output of my computer, booting with `iommu=debug` options on linux 6.6.13-200.fc39.

There's no "iommu=debug" anywhere in sight:

(XEN) Command line: placeholder no-real-mode edd=off

Did you mistakenly put it on the Linux command line?

Nevertheless there are relevant messages already without the extra
verbosity (and without you using a debug build of Xen, which would
be preferable):

(XEN) AMD-Vi: Warning: IVMD: [c9f1c000,c9f42000) is not (entirely) in reserved memory
(XEN) AMD-Vi: Error: IVMD: page at c9f1c000 can't be converted
(XEN) AMD-Vi: Error initialization

Quoting the respective entry from the memory map:

(XEN)  [00000000c8f58000, 00000000c9f57fff] (ACPI NVS)

Both Roger and I look to agree that this condition we have

            if ( !(type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
                           RAM_TYPE_UNUSABLE)) )

is inverted in some way. We merely need to agree in which way it
wants adjusting.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 11:50:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 11:50:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674356.1049206 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVVaD-0002F3-Bw; Thu, 01 Feb 2024 11:50:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674356.1049206; Thu, 01 Feb 2024 11:50:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVVaD-0002Ew-9F; Thu, 01 Feb 2024 11:50:17 +0000
Received: by outflank-mailman (input) for mailman id 674356;
 Thu, 01 Feb 2024 11:50:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cAD+=JK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rVVaB-0002DW-Pu
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 11:50:15 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0f061d62-c0f8-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 12:50:13 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a26ed1e05c7so116894166b.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 03:50:13 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 fj17-20020a1709069c9100b00a36d0dae491sm275054ejc.202.2024.02.01.03.50.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 03:50:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f061d62-c0f8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706788213; x=1707393013; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=/b2nCxVuhiNKnJIcxnoWW7CsS5U/pLdZUbH4D9pjupQ=;
        b=RJT6t6omnWCpwhysmMmIuy8b22WELM6osuIkyrBpCYL7NdW9KLJpq4cCjqcodKXn21
         Bd6eUMdpi8LH/6aFyncqyO4sFGEtV6JHOQ+qcQodJB0TGAZV8RzKqHuRaCYpdTD85uVV
         udEiBiAjJgBVufw5wztPfzFzf7rISVjayjbWs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706788213; x=1707393013;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/b2nCxVuhiNKnJIcxnoWW7CsS5U/pLdZUbH4D9pjupQ=;
        b=dKtxvCVZH11mCgB16oHEKy6tIO9wp6wSjDhH6DIPprky2yygYX2NJL4jp7Ok8SbXox
         eJ6zLNKtnfQbP0iED0CKpFvjs72yXxIXQY4FOLAWw2rKqp3h03+0Dyqrff90xKk3e17Z
         GiPFVQ2EbQbATuCJoTYhiyXmkCuNYa5eLbHJPOBFDtgftvpQ5hHqRrtchs0jMVi4F7rO
         GS+0eRiOlCMeYIYOegBazFPnlFRGdObuj9NU7XJW9S7nGNTi667sOpDhhzroFO0r+28D
         7NoyWstFP4Tdnt5D92i06nZ8EBF/c4USeGGppn2v8tdDub+ARPH4F1t6wj3LOcreHmHq
         IIKQ==
X-Gm-Message-State: AOJu0Ywz3YBzZpxkpvzM/cxSN+TM2xemw8X1bMSvRxKwgqgrxWP6vRoD
	DjZIINNHw+/1I+BNxMuVl8uCjhY+0A9qOQhQJWOMRxYkZmLwUO6zNdD5QMPj9PQ=
X-Google-Smtp-Source: AGHT+IEx/y0d9DNtQS9M0vQ0aGFGc9W/x/9Cnr/JO2o61EjFrMzGGqzlCsG5Kr9w497A1VY4CBEMUQ==
X-Received: by 2002:a17:906:260b:b0:a36:34b6:9ca4 with SMTP id h11-20020a170906260b00b00a3634b69ca4mr1758930ejc.64.1706788212751;
        Thu, 01 Feb 2024 03:50:12 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXVZ+Yq4REdfLsj1fV3tQfEcqzAeR0Cu2JkvNGBlJ3yO5Hj1WF4Uu2XxMDuv/vspIDPCtaTc8BVvGNYDbVsCBruAPuTk/RtKBxNEBTJOQ5Lak4jeIAindJhSH2iboYHv4ts8I2NaZX+2+xbmVEXG4Sv+JIDlAa2yqPyABc4diL8I6rEqw==
Date: Thu, 1 Feb 2024 12:50:11 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	Kevin Tian <kevin.tian@intel.com>,
	Jun Nakajima <jun.nakajima@intel.com>
Subject: Re: [PATCH v4 3/8] VMX: tertiary execution control infrastructure
Message-ID: <ZbuFc-ha-iv6B9ui@macbook>
References: <9dd23064-c79e-4a50-9c71-c0e73b189944@suse.com>
 <aa553449-888f-4e52-85b7-0bc0b7f010b4@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <aa553449-888f-4e52-85b7-0bc0b7f010b4@suse.com>

On Thu, Jan 11, 2024 at 10:00:10AM +0100, Jan Beulich wrote:
> This is a prereq to enabling the MSRLIST feature.
> 
> Note that the PROCBASED_CTLS3 MSR is different from other VMX feature
> reporting MSRs, in that all 64 bits report allowed 1-settings.
> 
> vVMX code is left alone, though, for the time being.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> v2: New.
> 
> --- a/xen/arch/x86/hvm/vmx/vmcs.c
> +++ b/xen/arch/x86/hvm/vmx/vmcs.c
> @@ -164,6 +164,7 @@ static int cf_check parse_ept_param_runt
>  u32 vmx_pin_based_exec_control __read_mostly;
>  u32 vmx_cpu_based_exec_control __read_mostly;
>  u32 vmx_secondary_exec_control __read_mostly;
> +uint64_t vmx_tertiary_exec_control __read_mostly;
>  u32 vmx_vmexit_control __read_mostly;
>  u32 vmx_vmentry_control __read_mostly;
>  u64 vmx_ept_vpid_cap __read_mostly;
> @@ -228,10 +229,32 @@ static u32 adjust_vmx_controls(
>      return ctl;
>  }
>  
> -static bool cap_check(const char *name, u32 expected, u32 saw)
> +static uint64_t adjust_vmx_controls2(
> +    const char *name, uint64_t ctl_min, uint64_t ctl_opt, unsigned int msr,
> +    bool *mismatch)
> +{
> +    uint64_t vmx_msr, ctl = ctl_min | ctl_opt;
> +
> +    rdmsrl(msr, vmx_msr);
> +
> +    ctl &= vmx_msr; /* bit == 0 ==> must be zero */
> +
> +    /* Ensure minimum (required) set of control bits are supported. */
> +    if ( ctl_min & ~ctl )
> +    {
> +        *mismatch = true;
> +        printk("VMX: CPU%u has insufficient %s (%#lx; requires %#lx)\n",
> +               smp_processor_id(), name, ctl, ctl_min);
> +    }
> +
> +    return ctl;
> +}
> +
> +static bool cap_check(
> +    const char *name, unsigned long expected, unsigned long saw)
>  {
>      if ( saw != expected )
> -        printk("VMX %s: saw %#x expected %#x\n", name, saw, expected);
> +        printk("VMX %s: saw %#lx expected %#lx\n", name, saw, expected);
>      return saw != expected;
>  }
>  
> @@ -241,6 +264,7 @@ static int vmx_init_vmcs_config(bool bsp
>      u32 _vmx_pin_based_exec_control;
>      u32 _vmx_cpu_based_exec_control;
>      u32 _vmx_secondary_exec_control = 0;
> +    uint64_t _vmx_tertiary_exec_control = 0;
>      u64 _vmx_ept_vpid_cap = 0;
>      u64 _vmx_misc_cap = 0;
>      u32 _vmx_vmexit_control;
> @@ -274,7 +298,8 @@ static int vmx_init_vmcs_config(bool bsp
>      opt = (CPU_BASED_ACTIVATE_MSR_BITMAP |
>             CPU_BASED_TPR_SHADOW |
>             CPU_BASED_MONITOR_TRAP_FLAG |
> -           CPU_BASED_ACTIVATE_SECONDARY_CONTROLS);
> +           CPU_BASED_ACTIVATE_SECONDARY_CONTROLS |
> +           CPU_BASED_ACTIVATE_TERTIARY_CONTROLS);
>      _vmx_cpu_based_exec_control = adjust_vmx_controls(
>          "CPU-Based Exec Control", min, opt,
>          MSR_IA32_VMX_PROCBASED_CTLS, &mismatch);
> @@ -338,6 +363,15 @@ static int vmx_init_vmcs_config(bool bsp
>              MSR_IA32_VMX_PROCBASED_CTLS2, &mismatch);
>      }
>  
> +    if ( _vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS )
> +    {
> +        uint64_t opt = 0;
> +
> +        _vmx_tertiary_exec_control = adjust_vmx_controls2(
> +            "Tertiary Exec Control", 0, opt,
> +            MSR_IA32_VMX_PROCBASED_CTLS3, &mismatch);
> +    }
> +
>      /* The IA32_VMX_EPT_VPID_CAP MSR exists only when EPT or VPID available */
>      if ( _vmx_secondary_exec_control & (SECONDARY_EXEC_ENABLE_EPT |
>                                          SECONDARY_EXEC_ENABLE_VPID) )
> @@ -468,6 +502,7 @@ static int vmx_init_vmcs_config(bool bsp
>          vmx_pin_based_exec_control = _vmx_pin_based_exec_control;
>          vmx_cpu_based_exec_control = _vmx_cpu_based_exec_control;
>          vmx_secondary_exec_control = _vmx_secondary_exec_control;
> +        vmx_tertiary_exec_control  = _vmx_tertiary_exec_control;
>          vmx_ept_vpid_cap           = _vmx_ept_vpid_cap;
>          vmx_vmexit_control         = _vmx_vmexit_control;
>          vmx_vmentry_control        = _vmx_vmentry_control;
> @@ -503,6 +538,9 @@ static int vmx_init_vmcs_config(bool bsp
>              "Secondary Exec Control",
>              vmx_secondary_exec_control, _vmx_secondary_exec_control);
>          mismatch |= cap_check(
> +            "Tertiary Exec Control",
> +            vmx_tertiary_exec_control, _vmx_tertiary_exec_control);

I know it's done to match the surrounding style, but couldn't you move
the name parameter one line up, and then limit the call to two lines?

(I don't think it will compromise readability).

> +        mismatch |= cap_check(
>              "VMExit Control",
>              vmx_vmexit_control, _vmx_vmexit_control);
>          mismatch |= cap_check(
> @@ -1080,6 +1118,7 @@ static int construct_vmcs(struct vcpu *v
>          v->arch.hvm.vmx.exec_control |= CPU_BASED_RDTSC_EXITING;
>  
>      v->arch.hvm.vmx.secondary_exec_control = vmx_secondary_exec_control;
> +    v->arch.hvm.vmx.tertiary_exec_control  = vmx_tertiary_exec_control;
>  
>      /*
>       * Disable features which we don't want active by default:
> @@ -1134,6 +1173,10 @@ static int construct_vmcs(struct vcpu *v
>          __vmwrite(SECONDARY_VM_EXEC_CONTROL,
>                    v->arch.hvm.vmx.secondary_exec_control);
>  
> +    if ( cpu_has_vmx_tertiary_exec_control )
> +        __vmwrite(TERTIARY_VM_EXEC_CONTROL,
> +                  v->arch.hvm.vmx.tertiary_exec_control);
> +
>      /* MSR access bitmap. */
>      if ( cpu_has_vmx_msr_bitmap )
>      {
> @@ -2068,10 +2111,12 @@ void vmcs_dump_vcpu(struct vcpu *v)
>                 vmr(HOST_PERF_GLOBAL_CTRL));
>  
>      printk("*** Control State ***\n");
> -    printk("PinBased=%08x CPUBased=%08x SecondaryExec=%08x\n",
> +    printk("PinBased=%08x CPUBased=%08x\n",
>             vmr32(PIN_BASED_VM_EXEC_CONTROL),
> -           vmr32(CPU_BASED_VM_EXEC_CONTROL),
> -           vmr32(SECONDARY_VM_EXEC_CONTROL));
> +           vmr32(CPU_BASED_VM_EXEC_CONTROL));
> +    printk("SecondaryExec=%08x TertiaryExec=%08lx\n",

For consistency, shouldn't TertiaryExec use 016 instead of 08 (as it's
a 64bit filed).

> +           vmr32(SECONDARY_VM_EXEC_CONTROL),
> +           vmr(TERTIARY_VM_EXEC_CONTROL));
>      printk("EntryControls=%08x ExitControls=%08x\n", vmentry_ctl, vmexit_ctl);
>      printk("ExceptionBitmap=%08x PFECmask=%08x PFECmatch=%08x\n",
>             vmr32(EXCEPTION_BITMAP),
> --- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
> +++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
> @@ -114,6 +114,7 @@ struct vmx_vcpu {
>      /* Cache of cpu execution control. */
>      u32                  exec_control;
>      u32                  secondary_exec_control;
> +    uint64_t             tertiary_exec_control;
>      u32                  exception_bitmap;
>  
>      uint64_t             shadow_gs;
> @@ -196,6 +197,7 @@ void vmx_vmcs_reload(struct vcpu *v);
>  #define CPU_BASED_RDTSC_EXITING               0x00001000U
>  #define CPU_BASED_CR3_LOAD_EXITING            0x00008000U
>  #define CPU_BASED_CR3_STORE_EXITING           0x00010000U
> +#define CPU_BASED_ACTIVATE_TERTIARY_CONTROLS  0x00020000U
>  #define CPU_BASED_CR8_LOAD_EXITING            0x00080000U
>  #define CPU_BASED_CR8_STORE_EXITING           0x00100000U
>  #define CPU_BASED_TPR_SHADOW                  0x00200000U
> @@ -260,6 +262,13 @@ extern u32 vmx_vmentry_control;
>  #define SECONDARY_EXEC_NOTIFY_VM_EXITING        0x80000000U
>  extern u32 vmx_secondary_exec_control;
>  
> +#define TERTIARY_EXEC_LOADIWKEY_EXITING         BIT(0, UL)
> +#define TERTIARY_EXEC_ENABLE_HLAT               BIT(1, UL)
> +#define TERTIARY_EXEC_EPT_PAGING_WRITE          BIT(2, UL)
> +#define TERTIARY_EXEC_GUEST_PAGING_VERIFY       BIT(3, UL)
> +#define TERTIARY_EXEC_IPI_VIRT                  BIT(4, UL)

While at it, my copy of the SDM also has:

#define TERTIARY_EXEC_VIRT_SPEC_CTRL               BIT(7, UL)

> +extern uint64_t vmx_tertiary_exec_control;
> +
>  #define VMX_EPT_EXEC_ONLY_SUPPORTED                         0x00000001
>  #define VMX_EPT_WALK_LENGTH_4_SUPPORTED                     0x00000040
>  #define VMX_EPT_MEMORY_TYPE_UC                              0x00000100
> @@ -295,6 +304,8 @@ extern u64 vmx_ept_vpid_cap;
>      (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_MSR_BITMAP)
>  #define cpu_has_vmx_secondary_exec_control \
>      (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS)
> +#define cpu_has_vmx_tertiary_exec_control \
> +    (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS)
>  #define cpu_has_vmx_ept \
>      (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_EPT)
>  #define cpu_has_vmx_dt_exiting \
> @@ -422,6 +433,7 @@ enum vmcs_field {
>      VIRT_EXCEPTION_INFO             = 0x0000202a,
>      XSS_EXIT_BITMAP                 = 0x0000202c,
>      TSC_MULTIPLIER                  = 0x00002032,
> +    TERTIARY_VM_EXEC_CONTROL        = 0x00002034,
>      GUEST_PHYSICAL_ADDRESS          = 0x00002400,
>      VMCS_LINK_POINTER               = 0x00002800,
>      GUEST_IA32_DEBUGCTL             = 0x00002802,
> --- a/xen/arch/x86/include/asm/msr-index.h
> +++ b/xen/arch/x86/include/asm/msr-index.h
> @@ -347,6 +347,7 @@
>  #define MSR_IA32_VMX_TRUE_EXIT_CTLS             0x48f
>  #define MSR_IA32_VMX_TRUE_ENTRY_CTLS            0x490
>  #define MSR_IA32_VMX_VMFUNC                     0x491
> +#define MSR_IA32_VMX_PROCBASED_CTLS3            0x492

Shouldn't this be added above the "Legacy MSR constants in need of
cleanup.  No new MSRs below this comment." line?

>  
>  /* K7/K8 MSRs. Not complete. See the architecture manual for a more
>     complete list. */
> --- a/xen/arch/x86/hvm/vmx/vmx.c
> +++ b/xen/arch/x86/hvm/vmx/vmx.c
> @@ -760,6 +760,12 @@ void vmx_update_secondary_exec_control(s
>                    v->arch.hvm.vmx.secondary_exec_control);
>  }
>  
> +void vmx_update_tertiary_exec_control(struct vcpu *v)

const vcpu?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 12:09:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 12:09:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674379.1049217 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVVsa-0005sW-F6; Thu, 01 Feb 2024 12:09:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674379.1049217; Thu, 01 Feb 2024 12:09:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVVsa-0005sP-B7; Thu, 01 Feb 2024 12:09:16 +0000
Received: by outflank-mailman (input) for mailman id 674379;
 Thu, 01 Feb 2024 12:09:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVVsY-0005sJ-Os
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 12:09:14 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b6984565-c0fa-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 13:09:13 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-40fc22f372cso1963085e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 04:09:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m21-20020a05600c4f5500b0040fb03f803esm4285986wmq.24.2024.02.01.04.09.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 04:09:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b6984565-c0fa-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706789353; x=1707394153; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=tH/k2x+4r+PxB4CU+/h65fC4pUhID4rLTMIEaTjk6kc=;
        b=N1ybM8BjZTsNbZJVm2PRgBJ68eC9SFowJseY04ZE6tzDoMtYuyX7MAB019B3hPtys7
         4IcVZj1nxIY/2zPutO5zI2Rf2CumrEiepycntX7/d/4uGXzzO0N4ZhlalIUdtT+v7toE
         jmzLSiJH/wnjtyKj1d2iofJ8+ZPHeyLkYUEBy0i5epGiPvuQuD39EO+EQruuTPtEVkNW
         B/3x07UrjqRumjKibespMtYf64UlrappmbRlGzJdKUi+t4j/ZT8yFg082dn49gcqFv/Y
         pcZZP7fBuIMPDI4DeQntvepG2SsDYDBNRZDAdaLlKnEzhtHFDfFvRE0jNtst42mtdTxv
         n9ug==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706789353; x=1707394153;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=tH/k2x+4r+PxB4CU+/h65fC4pUhID4rLTMIEaTjk6kc=;
        b=Kv+WxlcD2LLq+0Br+U4h9wQyRMB1X3mRZxoilawig/kO0ePCk6s3yIQYgwWojTYFDT
         ow211IFMwQl7nsTBt6GPxe2bPRAaE7IvwZFEQdh+eOTxGHDi31JOPzS1LC+UYBVBruRE
         +cMtlDgLb5rtIpDdJIgA/iniI3KR2CX06lDuEOaqalUhnW++MbzcQJfDD3A+useOvOZB
         UXReXNyo18vAxjv3GehZiRzk95mX5icRA9dLQebaiaOnesTSYvwUOhh+qMzxYywYwgp9
         cy3D1eMaRWvXQ1tbXBEgcfXrHPA1GZlFpPnFMyjJAJfaLM70+swg3MQvxFL6ZAmVHQHq
         PinQ==
X-Gm-Message-State: AOJu0YxGBmyVIVR100qH72SS/DjDNgQ5mt3RbLDh3N0t96PwPICYFMSK
	RT1jrqntFTzc9VSD6hnMWQhb1xgxiDv2jhevQZNt8pOipkG6OuiZg/UJaj4jaA==
X-Google-Smtp-Source: AGHT+IFh1Xug2JOLJu4UIO6rBKLgHJ8iKOfKC4MlaViuy7J+pQcLHi7b+niGlM2CJVtBIroCGY1lMw==
X-Received: by 2002:a05:600c:4288:b0:40e:fbc8:401 with SMTP id v8-20020a05600c428800b0040efbc80401mr4018713wmc.20.1706789352844;
        Thu, 01 Feb 2024 04:09:12 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCURuCvjHFZf6Q2hgZhVvg+SRJ3Jz4SHOk7TUi/MvgeKj79V3sTzZ+/dSh3NnjcK+sLZKofsVwBge2HzDit2M86H4Uw5ujpMuMCpwqIwYQfDbzLTENUpiMlbxTP2q7QOzbVPkfr/wVqmHluT5J3qJ/vHKBq9HKGGe8rq73uGFcirSSJ5Vw==
Message-ID: <56822f9c-6156-4652-9de1-3d599e22c51a@suse.com>
Date: Thu, 1 Feb 2024 13:09:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 3/8] VMX: tertiary execution control infrastructure
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Wei Liu <wl@xen.org>, Kevin Tian <kevin.tian@intel.com>,
 Jun Nakajima <jun.nakajima@intel.com>
References: <9dd23064-c79e-4a50-9c71-c0e73b189944@suse.com>
 <aa553449-888f-4e52-85b7-0bc0b7f010b4@suse.com> <ZbuFc-ha-iv6B9ui@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZbuFc-ha-iv6B9ui@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01.02.2024 12:50, Roger Pau Monné wrote:
> On Thu, Jan 11, 2024 at 10:00:10AM +0100, Jan Beulich wrote:
>> @@ -503,6 +538,9 @@ static int vmx_init_vmcs_config(bool bsp
>>              "Secondary Exec Control",
>>              vmx_secondary_exec_control, _vmx_secondary_exec_control);
>>          mismatch |= cap_check(
>> +            "Tertiary Exec Control",
>> +            vmx_tertiary_exec_control, _vmx_tertiary_exec_control);
> 
> I know it's done to match the surrounding style, but couldn't you move
> the name parameter one line up, and then limit the call to two lines?
> 
> (I don't think it will compromise readability).

You mean like this:

        mismatch |= cap_check("Tertiary Exec Control",
            vmx_tertiary_exec_control, _vmx_tertiary_exec_control);

? No, I view this as a mix of two possible styles. If the string literal
was moved up, the other legitimate style would only be

        mismatch |= cap_check("Tertiary Exec Control",
                              vmx_tertiary_exec_control,
                              _vmx_tertiary_exec_control);

aiui (again extending over 3 lines). Yet none of this is written down
anywhere.

But anyway - consistency with surrounding code trumps here, I think.

>> @@ -2068,10 +2111,12 @@ void vmcs_dump_vcpu(struct vcpu *v)
>>                 vmr(HOST_PERF_GLOBAL_CTRL));
>>  
>>      printk("*** Control State ***\n");
>> -    printk("PinBased=%08x CPUBased=%08x SecondaryExec=%08x\n",
>> +    printk("PinBased=%08x CPUBased=%08x\n",
>>             vmr32(PIN_BASED_VM_EXEC_CONTROL),
>> -           vmr32(CPU_BASED_VM_EXEC_CONTROL),
>> -           vmr32(SECONDARY_VM_EXEC_CONTROL));
>> +           vmr32(CPU_BASED_VM_EXEC_CONTROL));
>> +    printk("SecondaryExec=%08x TertiaryExec=%08lx\n",
> 
> For consistency, shouldn't TertiaryExec use 016 instead of 08 (as it's
> a 64bit filed).

Perhaps, assuming we'll gets bits 32 and populated sooner or later.
However, I view 16-digit literal numbers as hard to read, so I'd be
inclined to insert a separator (e.g. an underscore) between the low
and high halves. Thoughts?

>> @@ -260,6 +262,13 @@ extern u32 vmx_vmentry_control;
>>  #define SECONDARY_EXEC_NOTIFY_VM_EXITING        0x80000000U
>>  extern u32 vmx_secondary_exec_control;
>>  
>> +#define TERTIARY_EXEC_LOADIWKEY_EXITING         BIT(0, UL)
>> +#define TERTIARY_EXEC_ENABLE_HLAT               BIT(1, UL)
>> +#define TERTIARY_EXEC_EPT_PAGING_WRITE          BIT(2, UL)
>> +#define TERTIARY_EXEC_GUEST_PAGING_VERIFY       BIT(3, UL)
>> +#define TERTIARY_EXEC_IPI_VIRT                  BIT(4, UL)
> 
> While at it, my copy of the SDM also has:
> 
> #define TERTIARY_EXEC_VIRT_SPEC_CTRL               BIT(7, UL)

Ah yes, this must have appeared in the over 9 months that have
passed since I originally wrote this patch.

>> --- a/xen/arch/x86/include/asm/msr-index.h
>> +++ b/xen/arch/x86/include/asm/msr-index.h
>> @@ -347,6 +347,7 @@
>>  #define MSR_IA32_VMX_TRUE_EXIT_CTLS             0x48f
>>  #define MSR_IA32_VMX_TRUE_ENTRY_CTLS            0x490
>>  #define MSR_IA32_VMX_VMFUNC                     0x491
>> +#define MSR_IA32_VMX_PROCBASED_CTLS3            0x492
> 
> Shouldn't this be added above the "Legacy MSR constants in need of
> cleanup.  No new MSRs below this comment." line?

Now this is a question I'd like to forward to Andrew. Imo grouping the
new MSR with the other VMX ones is more important than respecting that
comment. But yes, I could of course add yet another patch to move the
entire block up first ...

>> --- a/xen/arch/x86/hvm/vmx/vmx.c
>> +++ b/xen/arch/x86/hvm/vmx/vmx.c
>> @@ -760,6 +760,12 @@ void vmx_update_secondary_exec_control(s
>>                    v->arch.hvm.vmx.secondary_exec_control);
>>  }
>>  
>> +void vmx_update_tertiary_exec_control(struct vcpu *v)
> 
> const vcpu?

Hmm, yes - overly blind copy-and-paste.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 12:11:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 12:11:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674383.1049227 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVVv6-0007k3-Om; Thu, 01 Feb 2024 12:11:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674383.1049227; Thu, 01 Feb 2024 12:11:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVVv6-0007jw-M7; Thu, 01 Feb 2024 12:11:52 +0000
Received: by outflank-mailman (input) for mailman id 674383;
 Thu, 01 Feb 2024 12:11:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TmiS=JK=cloud.com=edwin.torok@srs-se1.protection.inumbo.net>)
 id 1rVVv5-0007fE-3U
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 12:11:51 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1432c2dd-c0fb-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 13:11:50 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-40fb63c40c0so7625455e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 04:11:50 -0800 (PST)
Received: from smtpclient.apple ([90.251.137.254])
 by smtp.gmail.com with ESMTPSA id
 u18-20020a05600c19d200b0040fbc9a3405sm1492197wmq.0.2024.02.01.04.11.49
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 01 Feb 2024 04:11:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1432c2dd-c0fb-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1706789510; x=1707394310; darn=lists.xenproject.org;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XzTNVroynWj1SFNgSOveOmhFes8MZYVA132LR0GQJKg=;
        b=fZ8DN/y3BJBshqeJaP+AqrudKL+ih7InvvvjXizdyFQiQGNlZVbMjjtkXIXqCF6yhy
         +o+4PxCuPcSz5XW+4xKoFz8EJxTwJk3DNAwCPArxMH6ILFPy/tZ5byYEFUyueavOT+Mf
         DSu3IP33V645O9BWqJBTch8EOAP4bt0O6aOWQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706789510; x=1707394310;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=XzTNVroynWj1SFNgSOveOmhFes8MZYVA132LR0GQJKg=;
        b=WD7xIUB2DN1IBcOFhlb+dz5rrNcAlrVlFCSi6Y4oj2P1gxnG4NzmhxPU72eyUiCVyz
         JSG/1AYrMXfgIjyNQWTKgcoA3yeyt8Zu20HS+J6Yw7NeRxsbMZTpbyaXiIpFaldfvBtm
         atU90+gjRKaIKItvHJe4YOGibnQ+DTTFoYycpXgkcXAQQ6mY9MmVc0t2o5fyzFnuxnSc
         sP5V18vHMAX/2gzqbJyipdA8Jm2BCzTSgAtPK4GGAet1o8W0mg3GPKH9mEyYHNSpnmDU
         rLgARcVkW8fgLiaGqM3K2MR9K4NNAWySNqRqv7020fuTQEOgSGJ/QR1g6NXl5Gxgzu0x
         bP2w==
X-Gm-Message-State: AOJu0Yxm4QPakgcfxC4KkmzlSWg8dQTlSIk1bK8TufjQP28PzGpq68VN
	2oUjSaqC+6ImULtx4pekPAe9bkGUQTwkA/e83Vp7c24y2cC694YoRX3S2OkN84E=
X-Google-Smtp-Source: AGHT+IEDKHUB7aduZzxaOJOYrYpySaXvUVlGyfCD8NywdNJcBBIeYNEQYFUNE4nZPcqtNUnz+oDPlw==
X-Received: by 2002:a05:600c:35d3:b0:40e:fc29:f52f with SMTP id r19-20020a05600c35d300b0040efc29f52fmr3808286wmq.21.1706789509759;
        Thu, 01 Feb 2024 04:11:49 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCX8YTFu08UiZIBLLBhhKjDwQAhtWPsu5OdluSrJLI7AjTMvyi/4QJ9LErhP1grHuxDh3O5uwKnN/BvKmPSkf1nTm+NmuWYJiXDwT2LMlZonLpLzXXQw8uPeQ6jKRmyLi1rT2gCG+i40SzzCmmykOvfQUj0qNBvhcaduwj7wKRc4zyUsGZwjBqE99ms86awQztqCe4jhSOun2aKjCwN1bYy5WSBx
Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\))
Subject: Re: [PATCH v1 0/2] tools/ocaml: support OCaml 5.x, drop support for
 <=4.05
From: Edwin Torok <edwin.torok@cloud.com>
In-Reply-To: <b88df473-9a3e-4c83-8629-3b65c6a6190b@citrix.com>
Date: Thu, 1 Feb 2024 12:11:38 +0000
Cc: Christian Lindig <christian.lindig@cloud.com>,
 Xen-devel <xen-devel@lists.xenproject.org>,
 Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 Christian Lindig <christian.lindig@citrix.com>,
 David Scott <dave@recoil.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <2297C847-DE25-4D25-B1B0-57EFB8AA51DE@cloud.com>
References: <cover.1706697216.git.edwin.torok@cloud.com>
 <4E9A23AB-5745-4A71-9013-156B6648025A@cloud.com>
 <b88df473-9a3e-4c83-8629-3b65c6a6190b@citrix.com>
To: Andrew Cooper <Andrew.Cooper3@citrix.com>
X-Mailer: Apple Mail (2.3774.400.31)



> On 31 Jan 2024, at 10:55, Andrew Cooper <Andrew.Cooper3@citrix.com> =
wrote:
>=20
> On 31/01/2024 10:44 am, Christian Lindig wrote:
>>> On 31 Jan 2024, at 10:42, Edwin T=C3=B6r=C3=B6k =
<edwin.torok@cloud.com> wrote:
>>>=20
>>> Fix building oxenstored with OCaml 5.x.
>>> OCaml 5.x has removed some functions that have been deprecated for =
many years,
>>> in order to support OCaml 5.x we need to drop support for OCaml =
4.02.
>>>=20
>>> Tested in gitlab CI (together with my other series):
>>> =
https://gitlab.com/xen-project/people/edwintorok/xen/-/pipelines/115830282=
7
>>>=20
>>> Edwin T=C3=B6r=C3=B6k (2):
>>> oxenstored: fix build on OCaml 5.x
>>> tools/ocaml: bump minimum version to OCaml 4.05
>>>=20
>>> tools/configure               | 2 +-
>>> tools/configure.ac            | 2 +-
>>> tools/ocaml/xenstored/disk.ml | 2 +-
>>> 3 files changed, 3 insertions(+), 3 deletions(-)
>>>=20
>>> --=20
>>> 2.43.0
>>>=20
>> Acked-by: Christian Lindig <christian.lindig@cloud.com>
>=20
> It occurs to me that this is the kind of thing which should get a
> CHANGELOG.md entry these days.  Something like:
>=20
> diff --git a/CHANGELOG.md b/CHANGELOG.md
> index 1f55c9c72d10..fd7c8f5c6b82 100644
> --- a/CHANGELOG.md
> +++ b/CHANGELOG.md
> @@ -9,6 +9,7 @@ The format is based on [Keep a
> Changelog](https://keepachangelog.com/en/1.0.0/)
>  ### Changed
>   - Changed flexible array definitions in public I/O interface headers
> to not
>     use "1" as the number of array elements.
> + - The minimum supported Ocaml toolchain version is now 4.05
>   - On x86:
>     - HVM PIRQs are disabled by default.
>     - Reduce IOMMU setup time for hardware domain.

Sounds good.

Should this be mentioned in =
https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Sending_a_=
Patch_Series?

Best regards,
=E2=80=94Edwin

>=20
>=20
> ought to do.
>=20
> Have we checked to see whether this drops Ocaml from any of the build
> containers ?


I can look into this later, haven=E2=80=99t tried rebuilding the =
containers (the gitlab CI passed though)

Best regards,
=E2=80=94Edwin

>=20
> ~Andrew



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 12:14:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 12:14:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674385.1049237 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVVxY-0008Jn-5U; Thu, 01 Feb 2024 12:14:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674385.1049237; Thu, 01 Feb 2024 12:14:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVVxY-0008Jg-2R; Thu, 01 Feb 2024 12:14:24 +0000
Received: by outflank-mailman (input) for mailman id 674385;
 Thu, 01 Feb 2024 12:14:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVVxW-0008Ja-O1
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 12:14:22 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6e758c66-c0fb-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 13:14:21 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-5112cb7ae27so1564502e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 04:14:21 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v21-20020a05600c445500b0040faf3df118sm4353674wmn.32.2024.02.01.04.14.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 04:14:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e758c66-c0fb-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706789661; x=1707394461; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=poKUmijZjrU6z7E2d33cHjhbfbhs9SLZRbEIBTVo/DE=;
        b=cZz4DL6odwCfbDpN6b/XxqfZKdw4CKVICf6hUvbPbic2FZ/bTxOCFfwkwVk3mbRV54
         2DHERwS6NM+lCe5aAm24vdFw/5Xtx/WEgPYruDpOSnqAGdsXT+p8s82vo8ZEgwmWI9pA
         csBP7rozOVuwrJ4Jg0GSfwAEOvIvSV3GNRPF/cOtmI5QcrTm6Wh2SfQQyen8fZED+2qM
         OF67JZTuUsQd9QpYlSGEWbpSoR4Dyhschywk+mJD1eX2gMkcl+WTsI1HdNwDSkXwiCZC
         g3l65MLRsED8/sSfdCfIUIfIiJCSFmwqJNEvJ00cgdPxbCTtBmNPf7RlQKgMfnJGBkvz
         Onqw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706789661; x=1707394461;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=poKUmijZjrU6z7E2d33cHjhbfbhs9SLZRbEIBTVo/DE=;
        b=rN+zNv+GXnLSrAIZIt3kd0HVfkExc8iXdmv80Q5s1e9kWdJokBwUTHW+iwASE9BRUb
         ikluuiu/a531U97Vjm15bX+YYaGv6wGjfKVXy0Tj4S8fpI6+/B+d+vQj4jQYC7Vz+NJe
         I2KyNryV7/RXaVSW2hT1Ry4rSwOEr8MV7gsIlcUbNFeyvV/bD0Xouor7TOGzsBcKbhQB
         OGQ/NEShhpYn3eXxql3q/qKdeBxhsfG+eBK8fbwIv724bAC/tyn1u5SOkfDMDKODE50U
         g2HNwNmBCQ7f76BFg3oQn91Gd7nFZgbg9iXyxntb2TB9qeylx1DcbmOaBA2dGYfrdcRa
         p3lg==
X-Gm-Message-State: AOJu0YwYFHXSRdDFfcB6IMTkAdL7yilpwnLC9BvCyWGLs3kRO3DdS7AZ
	BDN3+fH6rUXjEzuvOruG6i4NCxXnrCzcRAOfOc5wDDmRwJKiFnFP6Yqkcw02qg==
X-Google-Smtp-Source: AGHT+IHxdOAPTqinLiUUJR1DlPq06pe5+JRkvZDL1us9hsAfrJG1Mfm4gzq7a2zjjxHFmesTNYMzLQ==
X-Received: by 2002:ac2:54b2:0:b0:511:1e60:8157 with SMTP id w18-20020ac254b2000000b005111e608157mr1554336lfk.10.1706789661344;
        Thu, 01 Feb 2024 04:14:21 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXmFke1JplbLXNMHDzsGpMZvvcuwb9pqNH3vnqcW8EWR2ZsBZyL/B79yvrezo5MNQB6EksE2yGCBKpTZtB8bljQhbTAl5YZ16FSx23ONv5sDWBI0NvEkFvXGdAdFDVYST8N+B/u43q0dsazw2VAeTitF8K2SWepHha12YokQbrzHuYoVowL8cLCYGVMXBYVqsq4mpTuBdSXTTL+f8sFi95p77lYzC20d4rD1M3eeK55EYBrVs/IptvSifPU1VspNtuohIgkT5Mbd8yaU82fDRkdh6d9zZhY
Message-ID: <412fd2a2-4382-4131-92a5-cfb399f95f31@suse.com>
Date: Thu, 1 Feb 2024 13:14:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 01/15] xen/common: add cache coloring common code
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-2-carlo.nonato@minervasys.tech>
 <42eabd87-b245-44e3-9945-40b1a5c5eaf5@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <42eabd87-b245-44e3-9945-40b1a5c5eaf5@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 31.01.2024 16:57, Jan Beulich wrote:
> On 29.01.2024 18:17, Carlo Nonato wrote:
>> +Command line parameters
>> +***********************
>> +
>> +More specific documentation is available at `docs/misc/xen-command-line.pandoc`.
>> +
>> ++----------------------+-------------------------------+
>> +| **Parameter**        | **Description**               |
>> ++----------------------+-------------------------------+
>> +| ``llc-coloring``     | enable coloring at runtime    |
>> ++----------------------+-------------------------------+
>> +| ``llc-way-size``     | set the LLC way size          |
>> ++----------------------+-------------------------------+
> 
> As a result of the above, I also find it confusing to specify "way size"
> as a command line option. Cache size, number of ways, and cache line size
> would seem more natural to me.

Or, alternatively, have the number of colors be specifiable directly.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 12:19:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 12:19:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674390.1049246 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVW20-000157-My; Thu, 01 Feb 2024 12:19:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674390.1049246; Thu, 01 Feb 2024 12:19:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVW20-000150-Jh; Thu, 01 Feb 2024 12:19:00 +0000
Received: by outflank-mailman (input) for mailman id 674390;
 Thu, 01 Feb 2024 12:18:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVW1z-00013X-Lz
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 12:18:59 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 12be9131-c0fc-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 13:18:57 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-40eacb4bfa0so7382585e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 04:18:57 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 r7-20020adff107000000b0033b1ab837e1sm482525wro.71.2024.02.01.04.18.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 04:18:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12be9131-c0fc-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706789937; x=1707394737; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=eD5NDrU/PsoBq64jRmzK/ZATALjDnfifROpcrYlE12M=;
        b=JEbxKO2h9635JJWO1hgk+WORuQHq1btg8+a6Rc8/D8/3MC5HMQFI4Rpoz8UKn3mOoA
         Pkbv9mO/S3bweIZrlvmtH2FDsmyOeSwtNP8AjAOkDTERDszxmKFLvdD0Q7ReCBFJEi0S
         fYO8PT41xGpYbTKjtaZtSqoHvubI5hJb14dPaGNegil611mIYyZEg8lLhZKXLsKJxbRA
         QGZgRW40WpeHgy0F/SaWqklnBsoJdb1763Vg6wELjyILUFlaSJKC6mBseWhPZCHBjVOJ
         VFE0gu3yDxv0KcOYYeWpXhMW4AMjQ6FxMe5IeFL8aMMH5siDeXLx7l+pOY+TRH1fnUGa
         exgw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706789937; x=1707394737;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=eD5NDrU/PsoBq64jRmzK/ZATALjDnfifROpcrYlE12M=;
        b=QFVcr/fQ4yfhdmi+XmsuLYtGq3N+1aQaOW7xm2Rv7UPiUVvh4kDqlSau9L7V0KCzWG
         gGekgmNYNC+Sf1f+LlSIiCkrZ7TZpzfj5jgMnUwI51plEWO1nqwfZe7SO9TP0xGDgWqw
         zZXk/19bcyVJFFnM2x+asX/Lu7vdS4gttQoLxV4TPylsLbvqv+F4NPqk5GBKLUJLywxA
         b1Uh/ty6vOlMgbuGW2lz6agX213sRUG+guXSakfl9F3C+U3djneU4ffCzQDBBwfIdIqR
         dgciKlj+xSGT/TNLGawvlThxnVFmwyb6qX9qIojrFC8esODJTz/0Tad2ITogwrZ1lm5c
         6UIg==
X-Gm-Message-State: AOJu0Ywd1r/eimsTb6fheW88ZgBJU/gQO7zOQbIpg/bWD/IK0zYiNHCY
	KXITvSu1kG+lqnJmE5+g0Lw7J6ChNk2uDId99/wG4dTt8RMTjNO5UE8mmeuqTA==
X-Google-Smtp-Source: AGHT+IF/p4o/Xb8wZmChDdC74b5tDL6KyfWTBri+huEg8aWAzIul24aPxXevexGDeI1UJEqM1anbaA==
X-Received: by 2002:a05:600c:5486:b0:40f:2d7:287c with SMTP id iv6-20020a05600c548600b0040f02d7287cmr1475583wmb.8.1706789937022;
        Thu, 01 Feb 2024 04:18:57 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUNUvMIJtlH1LlNw03rIBTiyobe8YrzJIEWgsx2GKoUChIqVmtowcM/clP0WJ7ot75B+qvwekFTWTtkNlpRQqVoda8Vd+dA8JJNav+juC6rbfiBe2ReHsT8TQOYvm4dso10lYz02GaA+0HrylZuXaIoTj28mkfvWx5lgs48Ycml8LPjZyrtNbvlGxsKzQQguvrN4mPlSang167/dH9aCOgsGKa86QQV8sXS/lITIcFkxvkmqYfsoMW8ua+CvaJy4UabjOTJ/X9bxC1Iy4zmpbQJrZhu6C4t
Message-ID: <c6f2c117-c8a7-4356-8421-4781090279c0@suse.com>
Date: Thu, 1 Feb 2024 13:18:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 01/15] xen/common: add cache coloring common code
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-2-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-2-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:17, Carlo Nonato wrote:
> --- /dev/null
> +++ b/docs/misc/cache-coloring.rst
> @@ -0,0 +1,87 @@
> +Xen cache coloring user guide
> +=============================
> +
> +The cache coloring support in Xen allows to reserve Last Level Cache (LLC)
> +partitions for Dom0, DomUs and Xen itself. Currently only ARM64 is supported.
> +
> +To compile LLC coloring support set ``CONFIG_LLC_COLORING=y``.
> +
> +If needed, change the maximum number of colors with
> +``CONFIG_NR_LLC_COLORS=<n>``.
> +
> +Compile Xen and the toolstack and then configure it via
> +`Command line parameters`_.
> +
> +Background
> +**********
> +
> +Cache hierarchy of a modern multi-core CPU typically has first levels dedicated
> +to each core (hence using multiple cache units), while the last level is shared
> +among all of them. Such configuration implies that memory operations on one
> +core (e.g. running a DomU) are able to generate interference on another core
> +(e.g .hosting another DomU). Cache coloring allows eliminating this
> +mutual interference, and thus guaranteeing higher and more predictable
> +performances for memory accesses.

Since you say "eliminating" - what about shared mid-level caches? What about
shared TLBs?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 12:20:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 12:20:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674393.1049257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVW3E-0002qf-4P; Thu, 01 Feb 2024 12:20:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674393.1049257; Thu, 01 Feb 2024 12:20:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVW3E-0002qY-0y; Thu, 01 Feb 2024 12:20:16 +0000
Received: by outflank-mailman (input) for mailman id 674393;
 Thu, 01 Feb 2024 12:20:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rVW3D-0002p3-BB
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 12:20:15 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVW37-0008HE-Be; Thu, 01 Feb 2024 12:20:09 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVW37-0002Mi-4i; Thu, 01 Feb 2024 12:20:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=fRFuwtEwNy6+DYwm/cJrzgMhWIr5XdvdCViTG3PTGDQ=; b=Pd84zAaBe96ZSB1Tx8w5uTdpal
	d/+vtBbEiDAl+GCFp8KKrPijfZJNf92UADMXvu2iPLbabj5SgGwZxpCtWI0vIR/uRog3qRYC6x865
	Rs9yYtxAUi1N6J+TdBtGaBWC8y3shLasWSNt5oWaSuRgHDPjh5RpkuW053pUeaqneVSw=;
Message-ID: <167f0c7a-e037-446c-82f8-2584e35a7af1@xen.org>
Date: Thu, 1 Feb 2024 12:20:06 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Content-Language: en-GB
To: John Ernberg <john.ernberg@actia.se>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Peng Fan <peng.fan@nxp.com>
Cc: Jonas Blixt <jonas.blixt@actia.se>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20240131114952.305805-1-john.ernberg@actia.se>
 <20240131114952.305805-2-john.ernberg@actia.se>
 <a2f726f5-df4c-4d15-90af-7d59c0f1f513@xen.org>
 <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se>
From: Julien Grall <julien@xen.org>
In-Reply-To: <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 31/01/2024 15:32, John Ernberg wrote:
> Hi Julien,

Hi John,

> On 1/31/24 13:22, Julien Grall wrote:
>> Hi,
>>
>> On 31/01/2024 11:50, John Ernberg wrote:
>>> When using Linux for dom0 there are a bunch of drivers that need to do
>>> SMC
>>> SIP calls into the PSCI provider to enable certain hardware bits like the
>>> watchdog.
>>
>> Do you know which protocol this is under the hood. Is this SCMI?
> 
> I think I confused myself here when I wrote the commit log.
> 
> The EL3 code in our case is ATF, and it does not appear to be SCMI, nor
> PSCI. The register usage of these SMC SIP calls are as follows:
> a0 - service
> a1 - function
> a2-a7 - args
> 
> In ATF the handler is declared as a runtime service.
> 
> Would the appropriate commmit message here be something along the lines
> of below?
> """
> When using Linux for dom0 there are a bunch of drivers that need to do   SMC
> SIP calls into the firmware to enable certain hardware bits like the
> watchdog.
> """

It reads better thanks.

[...]

>> But even if we restrict to dom0, have you checked that none of the SMCs
>> use buffers?
> I haven't found any such instances in the Linux kernel where a buffer is
> used. Adding a call filtering like suggested below additions of such
> functions can be discovered and adapted for if they would show up later.
>>
>> Rather than providing a blanket forward, to me it sounds more like you
>> want to provide an allowlist of the SMCs. This is more futureproof and
>> avoid the risk to expose unsafe SMCs to any domain.
>>
>> For an example, you can have a look at the EEMI mediator for Xilinx.
> 
> Ack. Do you prefer to see only on SMCCC service level or also on
> function level? (a1 register, per description earlier)

I am not sure. It will depend on whether it is correct to expose *all* 
the functions within a service level and they have the same format.

If you can't guarantee that, then you will most likely need to allowlist 
at the function level.

Also, do you have a spec in hand that would help to understand which 
service/function is implemented via those SMCs?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 12:30:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 12:30:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674405.1049267 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVWDM-0005Lt-5H; Thu, 01 Feb 2024 12:30:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674405.1049267; Thu, 01 Feb 2024 12:30:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVWDM-0005Lm-25; Thu, 01 Feb 2024 12:30:44 +0000
Received: by outflank-mailman (input) for mailman id 674405;
 Thu, 01 Feb 2024 12:30:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=biZV=JK=gmail.com=euidzero@srs-se1.protection.inumbo.net>)
 id 1rVWDL-0005Lg-3A
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 12:30:43 +0000
Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com
 [2607:f8b0:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b534da19-c0fd-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 13:30:40 +0100 (CET)
Received: by mail-pf1-x42b.google.com with SMTP id
 d2e1a72fcca58-6ddc1b30458so720756b3a.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 04:30:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b534da19-c0fd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706790639; x=1707395439; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=MNNe40aDhpyf7c0R61dDLN1CMdR/83Z6jEtpSATxBS4=;
        b=jSXm+5Dq4ym8TPRAfVDS3fbVV7NYeCBkrEAxim0C6shcE7pnL3xJ3SuLmnnSBmoN//
         y/MVdyF3IBbCek4OD90mZZixbeRh2ev0hg7hydsVYq43aAMHcf6/JfU9nuNJLaFCp2nY
         6GMEI1kjRXNZuA0OPvW0EhY6l+dB2j4mq8PXNuGKwXXLHezNY31+HoTgWLjLcJO7gPPy
         1uvRCWbTyKNfOTIbxmAV2HmSSSgM7cmogiEGBg95CIJi3EPz+UUGUAp/vsf4iZJuaVGB
         t35hgp3uO+9d8CSV/0jrtt/Q2if+TtZu9RtIPRJTeUQD1OJ5BNBVWx6TBAE0u+pDa/r8
         HnXQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706790639; x=1707395439;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=MNNe40aDhpyf7c0R61dDLN1CMdR/83Z6jEtpSATxBS4=;
        b=R/5/UwfH6SLO5nsXycSNzoeBfhQa28Qbd754a0ecPbhzNcfTTZ/LEtkOnJJyvbBLUp
         g4ib+7cVGjy8aD/OUWCVBs3lE2wKBDptHKhqODzKWH3vd9258G4p4niCk57lle7rhZhv
         M+CNO7D+61pPQ1rZOEnaKKlZpDhuOkjAgw2Onr3WpkXzpOuOodaXlg2wYbSyQ50ytsam
         1Is9BznbuZ4cLLs1nZlDmKJamSitHzk5SArI4q28xxmgViCRv8t4TnKHpcbohRT4izI8
         34DtoUnzLZs0GPz3FEu9B7git3NFHF8d3nwRKgST8Bz1eW8TcjbIEtV008MLyB/jf+2B
         Q0QA==
X-Gm-Message-State: AOJu0Yybt4yvrrYReGI92XfZK1Ee5ASWhqq0TG2BUrNvpJFOZ5jrF1fs
	cfrq7Wr9w3qN2g70fgF4bNujYo/SQXwL88BJsiFyi7hSqZsn441Eg4hpvxQ40NunzYnobKKal9+
	2L98Lq4GQqevfSQITxRFT4g41pcSvlQVpOQkZNw==
X-Google-Smtp-Source: AGHT+IGGofdZQxqikVe1bre/Xj1M34eesDnF1lZASV+6c77b+PD/3kvWcUxdr2RYddnUex8ieuor7XAswonzV0qmI4o=
X-Received: by 2002:a05:6a00:2ea7:b0:6db:dc74:d6e with SMTP id
 fd39-20020a056a002ea700b006dbdc740d6emr5098643pfb.17.1706790638574; Thu, 01
 Feb 2024 04:30:38 -0800 (PST)
MIME-Version: 1.0
References: <CAKm-Umas=5=JzooRzPHxUDigKpFK3Ze7cQcG8eR5kWgn-d_7fQ@mail.gmail.com>
 <CAKm-UmaUZ=rj-F3RvMoR7H3OvBHm1iy4PTRZAjyi=MQUVTuYVg@mail.gmail.com>
 <CAKm-Umat77MTh6=GEuH+bf1WRX+_B4-N5hY4xwADALfhj4QCPA@mail.gmail.com>
 <6ba9568a-ff8e-48ad-907f-74303e04d3a2@suse.com> <CAKm-UmZVoUhidEV15Xk+J_XN5a2GsBqMbXL9wqdFdoAFf_u5AA@mail.gmail.com>
 <bcbceba7-42ac-45be-80ab-e3dfe7741ec6@suse.com> <2eafe45b-fd97-472c-a173-459d6ff0b957@suse.com>
 <CAKm-Umbs2D7NHFE+NW2xKOu7-AZhFpH4uzE5QTuQusnGA_eNWA@mail.gmail.com>
 <f539144c-885a-461c-a506-bdb73e626a65@suse.com> <CAKm-UmY126AfdGhWcZ3s9vwN9+ksVRRFEhOu0ZFMoDvxrqOibw@mail.gmail.com>
 <CAKm-UmYt3iV8zOhSmtqMGhi_8T93c_nCbnefs4m3UC+3UABqeQ@mail.gmail.com>
 <CAKm-UmY-KXEAtBagikW4Jvp=SFXtmEg8P62pHfSo3Hr2s-0_-A@mail.gmail.com>
 <CAKm-UmYbMCfXc1Ny0=qfB+UaLSXV9oEHZiSgS=mwKMwmOFGVrQ@mail.gmail.com>
 <77c1c05d-a0a0-4292-9257-9b7fbebee0e3@suse.com> <2859ad22-8714-4628-8142-fc77fc2072c3@amd.com>
 <CAKm-UmZpyGkWXugYTJqU+qqVDyCFEKghtd=NTr2wK5EMCeL9Ww@mail.gmail.com> <214585d5-689d-4ba6-bd48-359428a7ed8f@amd.com>
In-Reply-To: <214585d5-689d-4ba6-bd48-359428a7ed8f@amd.com>
From: =?UTF-8?Q?S=C3=A9bastien_Chaumat?= <euidzero@gmail.com>
Date: Thu, 1 Feb 2024 13:30:26 +0100
Message-ID: <CAKm-UmZoKwre8-G793VqRNFCmzAti1o-0Kp3ZyV_Z5cc0YNiKw@mail.gmail.com>
Subject: Re: [BUG]i2c_hid_acpi broken with 4.17.2 on Framework Laptop 13 AMD
To: Mario Limonciello <mario.limonciello@amd.com>
Cc: Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xenproject.org, 
	Juergen Gross <jgross@suse.com>, 
	=?UTF-8?Q?Marek_Marczykowski=2DG=C3=B3recki?= <marmarek@invisiblethingslab.com>
Content-Type: multipart/alternative; boundary="000000000000b7e0af061051296f"

--000000000000b7e0af061051296f
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I spotted the following warning for IRQ7 (along with IRQ6 and 10)

[    0.686073] fedora kernel: __irq_set_trigger: genirq: No set_type
function for IRQ 7 (IR-IO-APIC)

This comes from kernel/irq/manage.c


int __irq_set_trigger(struct irq_desc *desc, unsigned long flags)
{
struct irq_chip *chip =3D desc->irq_data.chip;
int ret, unmask =3D 0;

if (!chip || !chip->irq_set_type) {
/*
* IRQF_TRIGGER_* but the PIC does not support multiple
* flow-types?
*/
pr_debug("No set_type function for IRQ %d (%s)\n",
irq_desc_get_irq(desc),
chip ? (chip->name ? : "unknown") : "unknown");
return 0;
}

Could this have a role in the IRQ misconfiguration by xen ?




Le lun. 22 janv. 2024 =C3=A0 21:59, Mario Limonciello <mario.limonciello@am=
d.com>
a =C3=A9crit :

> On 1/22/2024 11:06, S=C3=A9bastien Chaumat wrote:
> >
> >
> > Le mer. 17 janv. 2024 =C3=A0 03:20, Mario Limonciello
> > <mario.limonciello@amd.com <mailto:mario.limonciello@amd.com>> a =C3=A9=
crit :
> >
> >     On 1/16/2024 10:18, Jan Beulich wrote:
> >      > On 16.01.2024 16:52, S=C3=A9bastien Chaumat wrote:
> >      >> Le mar. 2 janv. 2024 =C3=A0 21:23, S=C3=A9bastien Chaumat
> >     <euidzero@gmail.com <mailto:euidzero@gmail.com>> a
> >      >> =C3=A9crit :
> >      >>
> >      >>>
> >      >>>   output of gpioinfo
> >      >>>>
> >      >>>> kernel alone :
> >      >>>>
> >      >>>>          line   5: unnamed         input active-low
> >     consumer=3Dinterrupt
> >      >>>>          line  84: unnamed         input active-low
> >     consumer=3Dinterrupt
> >      >>>>
> >      >>>> xen:
> >      >>>>
> >      >>>>          line   5: unnamed         input active-low
> >      >>>>          line  84: unnamed         input active-low
> >      >>>>
> >      >>>> xen with skipping IRQ7 double init :
> >      >>>>
> >      >>>>          line   5: unnamed         input active-low
> >     consumer=3Dinterrupt
> >      >>>>          line  84: unnamed         input active-low
> >      >>>>
> >      >>>>
> >      >>>> So definitely progressing.
> >      >>>>
> >      >>>
> >      >>> Checking /sys/kernel/irq/7
> >      >>>
> >      >>> kernel alone :
> >      >>>   actions: pinctrl_amd
> >      >>>   chip_name: IR-IO-APIC
> >      >>>   hwirq: 7
> >      >>>   name: fasteoi
> >      >>>   per_cpu_count: 0,0,0,0,0,20,0,0,0,0,0,0,0,0,0,0
> >      >>>   type: level
> >      >>>   wakeup: enabled
> >      >>>
> >      >>> xen skipping IRQ7 double init :
> >      >>>
> >      >>> actions: pinctrl_amd
> >      >>>   chip_name: xen-pirq
> >      >>>   hwirq:
> >      >>>   name: ioapic-level
> >      >>>   per_cpu_count: 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0
> >      >>>   type: edge
> >      >>>   wakeup: disabled
> >      >>>
> >      >>> So the skip of IRQ7 in pci_xen_initial_domain() sets the
> >     correct handler
> >      >>>   (IIUC xen uses the ioapic-level and handles the eoi
> >     separately), but not
> >      >>> the correct type (still edge).
> >      >>> I guess this may explains the results above.
> >      >>>
> >      >>>
> >      >>   Mario (in CC) patched the pinctrl_amd to flush pending
> >     interrupt before
> >      >> starting the driver for the GPIO.
> >      >>
> >      >> This helped in  the sense of there's no more pending interrupt
> >     on IRQ7
> >      >> (whatever the handler is, level or edge) but then the touchpad
> >     is not
> >      >> detected by i2c-hid.
> >      >>
> >      >> Is there any work in progress related to the incorrect IRQ
> >     configuration ?
> >      >
> >      > I'm not aware of any. As per my recollection it's still not
> entirely
> >      > clear where in the kernel things go astray. And to be honest I
> don't
> >      > feel comfortable trying to half-blindly address this, e.g. by
> trying
> >      > to circumvent / defer the early setting up of the low 16 IRQs.
> >      >
> >      > Jan
> >
> >     Shot in the dark - but could this be a problem where PCAT_COMPAT fr=
om
> >     the MADT is being ignored causing PIC not to be setup properly in t=
he
> >     Xen case?
> >
> >     See https://lore.kernel.org/all/875y2u5s8g.ffs@tglx/
> >     <https://lore.kernel.org/all/875y2u5s8g.ffs@tglx/> for some context=
.
> >
> > At least we know that no MADT override is found by xen for INT7 as no
> > INT_SRC_OVR message is printed.
> >
> > Do we expect one @Mario Limonciello <mailto:mario.limonciello@amd.com>
> ?
>
> No; the INT_SRV_OVR you'll see on Framework 13 AMD is on IRQ 2 and IRQ 9.
>
>

--000000000000b7e0af061051296f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>I spotted the following warning for IRQ7 (along with =
IRQ6 and 10)<br></div><div><br></div><div><div>[ =C2=A0 =C2=A00.686073] fed=
ora kernel: __irq_set_trigger: genirq: No set_type function for IRQ 7 (IR-I=
O-APIC)</div></div><div><br></div><div>This comes from kernel/irq/manage.c<=
/div><div><br></div><div><br></div><div><div style=3D"color:rgb(0,0,0);back=
ground-color:rgb(255,255,255);font-family:&quot;Droid Sans Mono&quot;,&quot=
;monospace&quot;,monospace;font-weight:normal;font-size:14px;line-height:19=
px;white-space:pre"><div><span style=3D"color:rgb(0,0,255)">int</span><span=
 style=3D"color:rgb(0,0,0)"> </span><span style=3D"color:rgb(116,83,31)">__=
irq_set_trigger</span><span style=3D"color:rgb(0,0,0)">(</span><span style=
=3D"color:rgb(0,0,255)">struct</span><span style=3D"color:rgb(0,0,0)"> </sp=
an><span style=3D"color:rgb(43,145,175)">irq_desc</span><span style=3D"colo=
r:rgb(0,0,0)"> *</span><span style=3D"color:rgb(128,128,128)">desc</span><s=
pan style=3D"color:rgb(0,0,0)">, </span><span style=3D"color:rgb(0,0,255)">=
unsigned</span><span style=3D"color:rgb(0,0,0)"> </span><span style=3D"colo=
r:rgb(0,0,255)">long</span><span style=3D"color:rgb(0,0,0)"> </span><span s=
tyle=3D"color:rgb(128,128,128)">flags</span><span style=3D"color:rgb(0,0,0)=
">)</span></div><div><span style=3D"color:rgb(0,0,0)">{</span></div><div><s=
pan style=3D"color:rgb(0,0,0)">    </span><span style=3D"color:rgb(0,0,255)=
">struct</span><span style=3D"color:rgb(0,0,0)"> </span><span style=3D"colo=
r:rgb(43,145,175)">irq_chip</span><span style=3D"color:rgb(0,0,0)"> *</span=
><span style=3D"color:rgb(31,55,127)">chip</span><span style=3D"color:rgb(0=
,0,0)"> =3D </span><span style=3D"color:rgb(128,128,128)">desc</span><span =
style=3D"color:rgb(0,0,0)">-&gt;</span><span style=3D"color:rgb(0,0,0)">irq=
_data</span><span style=3D"color:rgb(0,0,0)">.</span><span style=3D"color:r=
gb(0,0,0)">chip</span><span style=3D"color:rgb(0,0,0)">;</span></div><div><=
span style=3D"color:rgb(0,0,0)">    </span><span style=3D"color:rgb(0,0,255=
)">int</span><span style=3D"color:rgb(0,0,0)"> </span><span style=3D"color:=
rgb(31,55,127)">ret</span><span style=3D"color:rgb(0,0,0)">, </span><span s=
tyle=3D"color:rgb(31,55,127)">unmask</span><span style=3D"color:rgb(0,0,0)"=
> =3D </span><span style=3D"color:rgb(9,134,88)">0</span><span style=3D"col=
or:rgb(0,0,0)">;</span></div><br><div><span style=3D"color:rgb(0,0,0)">    =
</span><span style=3D"color:rgb(143,8,196)">if</span><span style=3D"color:r=
gb(0,0,0)"> (!</span><span style=3D"color:rgb(31,55,127)">chip</span><span =
style=3D"color:rgb(0,0,0)"> || !</span><span style=3D"color:rgb(31,55,127)"=
>chip</span><span style=3D"color:rgb(0,0,0)">-&gt;</span><span style=3D"col=
or:rgb(0,0,0)">irq_set_type</span><span style=3D"color:rgb(0,0,0)">) {</spa=
n></div><div><span style=3D"color:rgb(0,128,0)">        /*</span></div><div=
><span style=3D"color:rgb(0,128,0)">         * IRQF_TRIGGER_* but the PIC d=
oes not support multiple</span></div><div><span style=3D"color:rgb(0,128,0)=
">         * flow-types?</span></div><div><span style=3D"color:rgb(0,128,0)=
">         */</span></div><div><span style=3D"color:rgb(0,0,0)">        </s=
pan><span style=3D"color:rgb(138,27,255)">pr_debug</span><span style=3D"col=
or:rgb(0,0,0)">(</span><span style=3D"color:rgb(226,31,31)">&quot;</span><s=
pan style=3D"color:rgb(163,21,21)">No set_type function for IRQ </span><spa=
n style=3D"color:rgb(31,55,127)">%d</span><span style=3D"color:rgb(163,21,2=
1)"> (</span><span style=3D"color:rgb(31,55,127)">%s</span><span style=3D"c=
olor:rgb(163,21,21)">)</span><span style=3D"color:rgb(183,118,251)">\n</spa=
n><span style=3D"color:rgb(226,31,31)">&quot;</span><span style=3D"color:rg=
b(0,0,0)">,</span></div><div><span style=3D"color:rgb(0,0,0)">             =
</span><span style=3D"color:rgb(116,83,31)">irq_desc_get_irq</span><span st=
yle=3D"color:rgb(0,0,0)">(</span><span style=3D"color:rgb(128,128,128)">des=
c</span><span style=3D"color:rgb(0,0,0)">),</span></div><div><span style=3D=
"color:rgb(0,0,0)">             </span><span style=3D"color:rgb(31,55,127)"=
>chip</span><span style=3D"color:rgb(0,0,0)"> ? (</span><span style=3D"colo=
r:rgb(31,55,127)">chip</span><span style=3D"color:rgb(0,0,0)">-&gt;</span><=
span style=3D"color:rgb(0,0,0)">name</span><span style=3D"color:rgb(0,0,0)"=
> ? : </span><span style=3D"color:rgb(226,31,31)">&quot;</span><span style=
=3D"color:rgb(163,21,21)">unknown</span><span style=3D"color:rgb(226,31,31)=
">&quot;</span><span style=3D"color:rgb(0,0,0)">) : </span><span style=3D"c=
olor:rgb(226,31,31)">&quot;</span><span style=3D"color:rgb(163,21,21)">unkn=
own</span><span style=3D"color:rgb(226,31,31)">&quot;</span><span style=3D"=
color:rgb(0,0,0)">);</span></div><div><span style=3D"color:rgb(0,0,0)">    =
    </span><span style=3D"color:rgb(143,8,196)">return</span><span style=3D=
"color:rgb(0,0,0)"> </span><span style=3D"color:rgb(9,134,88)">0</span><spa=
n style=3D"color:rgb(0,0,0)">;</span></div><div><span style=3D"color:rgb(0,=
0,0)">    }</span></div></div></div><div><br></div><div>Could this have a r=
ole in the IRQ misconfiguration by xen ?</div><div><br></div><div><br></div=
><div><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=
=3D"gmail_attr">Le=C2=A0lun. 22 janv. 2024 =C3=A0=C2=A021:59, Mario Limonci=
ello &lt;<a href=3D"mailto:mario.limonciello@amd.com">mario.limonciello@amd=
.com</a>&gt; a =C3=A9crit=C2=A0:<br></div><blockquote class=3D"gmail_quote"=
 style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);p=
adding-left:1ex">On 1/22/2024 11:06, S=C3=A9bastien Chaumat wrote:<br>
&gt; <br>
&gt; <br>
&gt; Le=C2=A0mer. 17 janv. 2024 =C3=A0=C2=A003:20, Mario Limonciello <br>
&gt; &lt;<a href=3D"mailto:mario.limonciello@amd.com" target=3D"_blank">mar=
io.limonciello@amd.com</a> &lt;mailto:<a href=3D"mailto:mario.limonciello@a=
md.com" target=3D"_blank">mario.limonciello@amd.com</a>&gt;&gt; a =C3=A9cri=
t=C2=A0:<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0On 1/16/2024 10:18, Jan Beulich wrote:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt; On 16.01.2024 16:52, S=C3=A9bastien Chaumat w=
rote:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt; Le mar. 2 janv. 2024 =C3=A0 21:23, S=C3=
=A9bastien Chaumat<br>
&gt;=C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"mailto:euidzero@gmail.com" target=3D=
"_blank">euidzero@gmail.com</a> &lt;mailto:<a href=3D"mailto:euidzero@gmail=
.com" target=3D"_blank">euidzero@gmail.com</a>&gt;&gt; a<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt; =C3=A9crit :<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0output of gpioinfo<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt; kernel alone :<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 line=C2=A0 =C2=A05: unnamed=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0input active-=
low<br>
&gt;=C2=A0 =C2=A0 =C2=A0consumer=3Dinterrupt<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 line=C2=A0 84: unnamed=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0input active-low<b=
r>
&gt;=C2=A0 =C2=A0 =C2=A0consumer=3Dinterrupt<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt; xen:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 line=C2=A0 =C2=A05: unnamed=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0input active-=
low<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 line=C2=A0 84: unnamed=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0input active-low<b=
r>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt; xen with skipping IRQ7 double ini=
t :<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 line=C2=A0 =C2=A05: unnamed=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0input active-=
low<br>
&gt;=C2=A0 =C2=A0 =C2=A0consumer=3Dinterrupt<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 line=C2=A0 84: unnamed=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0input active-low<b=
r>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt; So definitely progressing.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt; Checking /sys/kernel/irq/7<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt; kernel alone :<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0actions: pinctrl_amd<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0chip_name: IR-IO-APIC<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0hwirq: 7<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0name: fasteoi<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0per_cpu_count: 0,0,0,0,0,=
20,0,0,0,0,0,0,0,0,0,0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0type: level<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0wakeup: enabled<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt; xen skipping IRQ7 double init :<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt; actions: pinctrl_amd<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0chip_name: xen-pirq<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0hwirq:<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0name: ioapic-level<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0per_cpu_count: 0,0,0,0,0,=
0,1,0,0,0,0,0,0,0,0,0<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0type: edge<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0wakeup: disabled<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt; So the skip of IRQ7 in pci_xen_initia=
l_domain() sets the<br>
&gt;=C2=A0 =C2=A0 =C2=A0correct handler<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;=C2=A0 =C2=A0(IIUC xen uses the ioapic=
-level and handles the eoi<br>
&gt;=C2=A0 =C2=A0 =C2=A0separately), but not<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt; the correct type (still edge).<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt; I guess this may explains the results=
 above.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;=C2=A0 =C2=A0Mario (in CC) patched the pin=
ctrl_amd to flush pending<br>
&gt;=C2=A0 =C2=A0 =C2=A0interrupt before<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt; starting the driver for the GPIO.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt; This helped in=C2=A0 the sense of there&#=
39;s no more pending interrupt<br>
&gt;=C2=A0 =C2=A0 =C2=A0on IRQ7<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt; (whatever the handler is, level or edge) =
but then the touchpad<br>
&gt;=C2=A0 =C2=A0 =C2=A0is not<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt; detected by i2c-hid.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;&gt; Is there any work in progress related to =
the incorrect IRQ<br>
&gt;=C2=A0 =C2=A0 =C2=A0configuration ?<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt; I&#39;m not aware of any. As per my recollect=
ion it&#39;s still not entirely<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt; clear where in the kernel things go astray. A=
nd to be honest I don&#39;t<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt; feel comfortable trying to half-blindly addre=
ss this, e.g. by trying<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt; to circumvent / defer the early setting up of=
 the low 16 IRQs.<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0 &gt; Jan<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0Shot in the dark - but could this be a problem wher=
e PCAT_COMPAT from<br>
&gt;=C2=A0 =C2=A0 =C2=A0the MADT is being ignored causing PIC not to be set=
up properly in the<br>
&gt;=C2=A0 =C2=A0 =C2=A0Xen case?<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0See <a href=3D"https://lore.kernel.org/all/875y2u5s=
8g.ffs@tglx/" rel=3D"noreferrer" target=3D"_blank">https://lore.kernel.org/=
all/875y2u5s8g.ffs@tglx/</a><br>
&gt;=C2=A0 =C2=A0 =C2=A0&lt;<a href=3D"https://lore.kernel.org/all/875y2u5s=
8g.ffs@tglx/" rel=3D"noreferrer" target=3D"_blank">https://lore.kernel.org/=
all/875y2u5s8g.ffs@tglx/</a>&gt; for some context.<br>
&gt; <br>
&gt; At least we know that no MADT override is found by xen for INT7 as no =
<br>
&gt; INT_SRC_OVR message is printed.<br>
&gt; <br>
&gt; Do we expect one @Mario Limonciello &lt;mailto:<a href=3D"mailto:mario=
.limonciello@amd.com" target=3D"_blank">mario.limonciello@amd.com</a>&gt;=
=C2=A0 ?<br>
<br>
No; the INT_SRV_OVR you&#39;ll see on Framework 13 AMD is on IRQ 2 and IRQ =
9.<br>
<br>
</blockquote></div>

--000000000000b7e0af061051296f--


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 13:00:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 13:00:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674409.1049277 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVWfX-0003fT-Dn; Thu, 01 Feb 2024 12:59:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674409.1049277; Thu, 01 Feb 2024 12:59:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVWfX-0003fM-9x; Thu, 01 Feb 2024 12:59:51 +0000
Received: by outflank-mailman (input) for mailman id 674409;
 Thu, 01 Feb 2024 12:59:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVWfV-0003do-Ca
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 12:59:49 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c5d893dc-c101-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 13:59:45 +0100 (CET)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a3510d79ae9so108924266b.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 04:59:45 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 un6-20020a170907cb8600b00a36a7f0f087sm853168ejc.222.2024.02.01.04.59.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 04:59:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5d893dc-c101-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706792385; x=1707397185; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=zl2cW3Nf4pRC//h1O/iq1g877AMcMR/l+E8T3gdpu4c=;
        b=WwCQMQpPaJNq3qRfK4HKLccSKMqIzeHhBm3Li6rMDu5+nrBCvvOHoV0s+pY4Q8JhBv
         hN72jbOvfBqzTHXpBqsv+qZuW87AF3Xctz+/8rqGDBWP/4DY+NMaO47xluzDdpKHwRuG
         WN4t8ny1hJnEdxrCI7dmjzMgR3u/7LCvL1+Hd+32scGf0mnfDuLZOqKwxLTGiYyWysfR
         a4exlMcICb9Ffv0Mdk3VZ7R7GtWIeT+b4OE/DfRCgiXm3/dPgsXJB+aYOwyFGhSvrqcz
         4cGsN1hFCpam1YHSvPCkVJqFsUDW0PYQOBkimiKnWCF1bo0Ke8zv4CBVIyW6tBhLZFda
         aKfQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706792385; x=1707397185;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=zl2cW3Nf4pRC//h1O/iq1g877AMcMR/l+E8T3gdpu4c=;
        b=loINMDXLkEvILakdMdGotnWzqKyfBJpPGM+qpWpgUW9PGCffKvoyf4DRuLVdyg/eYP
         MQooYlDVUC3J2kdKuj8OOst7nk8iki0gBj17c0MVFYTaJQe2nVlO+yWBOWePttfdeckd
         B0fdRz8hZCKfFLBfZpAbaEdlKHEJrM6pqe7nDhZo6c04RkTgxmnA1fQVV40KAOivHoHb
         2DhjYckNO1+4NT0eO0F87mmdRvBs4DNBYzBnq9FvDeoV9NLMDW6xMVe7Dzk0UY05YshD
         NrZ5NmlJUKcJvRQmX+z+7gRn+hs8AT2WHLWzaKFdscwEwi4F2byXmL44nfC+lxLqEpTU
         /zmQ==
X-Gm-Message-State: AOJu0YwtMbAZmRygDpYl64t8UGT/OaNckJbt3rOZPGSOaJL2u4wRl++u
	d/xV0mDG41oOBYHqOSK0KgkN7mKVHH14w/mDVuCNiEp2gnI6DJw/WxWJlszFZQ==
X-Google-Smtp-Source: AGHT+IGjwjGs0v1oXiJdD8mOW31gkXxFyk3rQD2SskaB5KSot/lGzXDyFgsWowu6ah8cajrXrFalzg==
X-Received: by 2002:a17:906:a883:b0:a36:5079:d6c9 with SMTP id ha3-20020a170906a88300b00a365079d6c9mr3338004ejb.76.1706792384939;
        Thu, 01 Feb 2024 04:59:44 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXMv61hXbNFRRGNxKyhVFyOzUYucwqmMJh7MEuQw8gspOkU4MzN9MwCijeDbkN5hQ5zqGi/3nGaG+efacx08Z4JOatK1pioa50xojOtAQ1ZZ2f7Y7OK3mJ4Up49Ardy5KLUsN9xijDO5IL3abIN6I68+3z7IArUef5p3J2DkofcdSY81mYQGPor4BabVq6G7hvytIuUFvpTRzEoB5JFCe/aPYAT8cEzqvr2IYxHXcYZtlqRlB69Xh/hgSWbQQlDTuigkcmKDzHZFFCM7kachH4wGiO9ohms
Message-ID: <749799db-de6f-4ca8-bab1-67468f97929b@suse.com>
Date: Thu, 1 Feb 2024 13:59:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 01/15] xen/common: add cache coloring common code
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-2-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-2-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:17, Carlo Nonato wrote:
> --- a/xen/arch/Kconfig
> +++ b/xen/arch/Kconfig
> @@ -31,3 +31,20 @@ config NR_NUMA_NODES
>  	  associated with multiple-nodes management. It is the upper bound of
>  	  the number of NUMA nodes that the scheduler, memory allocation and
>  	  other NUMA-aware components can handle.
> +
> +config LLC_COLORING
> +	bool "Last Level Cache (LLC) coloring" if EXPERT
> +	depends on HAS_LLC_COLORING
> +
> +config NR_LLC_COLORS
> +	int "Maximum number of LLC colors"
> +	range 2 1024

What's the reasoning behind this upper bound? IOW - can something to this
effect be said in the description, please?

> +	default 128
> +	depends on LLC_COLORING
> +	help
> +	  Controls the build-time size of various arrays associated with LLC
> +	  coloring. Refer to cache coloring documentation for how to compute the
> +	  number of colors supported by the platform. This is only an upper
> +	  bound. The runtime value is autocomputed or manually set via cmdline.
> +	  The default value corresponds to an 8 MiB 16-ways LLC, which should be
> +	  more than what needed in the general case.

Aiui while not outright wrong, non-power-of-2 values are meaningless to
specify. Perhaps that is worth mentioning (if not making this a value
that's used as exponent of 2 in the first place)?

As to the default and its description: As said for the documentation,
doesn't what this corresponds to also depend on cache line size? Even
if this was still Arm-specific rather than common code, I'd question
whether now and forever Arm chips may only use one pre-determined cache
line size.

> --- /dev/null
> +++ b/xen/common/llc-coloring.c
> @@ -0,0 +1,87 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Last Level Cache (LLC) coloring common code
> + *
> + * Copyright (C) 2022 Xilinx Inc.
> + */
> +#include <xen/keyhandler.h>
> +#include <xen/llc-coloring.h>
> +#include <xen/param.h>
> +
> +bool __ro_after_init llc_coloring_enabled;
> +boolean_param("llc-coloring", llc_coloring_enabled);

The variable has no use right now afaics, so it's unclear whether (a) it
is legitimately non-static and (b) placed in an appropriate section.

> +/* Size of an LLC way */
> +static unsigned int __ro_after_init llc_way_size;
> +size_param("llc-way-size", llc_way_size);
> +/* Number of colors available in the LLC */
> +static unsigned int __ro_after_init max_nr_colors = CONFIG_NR_LLC_COLORS;
> +
> +static void print_colors(const unsigned int *colors, unsigned int num_colors)
> +{
> +    unsigned int i;
> +
> +    printk("{ ");
> +    for ( i = 0; i < num_colors; i++ ) {

Nit (style): Brace placement.

> +        unsigned int start = colors[i], end = colors[i];
> +
> +        printk("%u", start);
> +
> +        for ( ;
> +              i < num_colors - 1 && colors[i] + 1 == colors[i + 1];

To reduce the number of array accesses, may I suggest to use "end + 1"
here instead of "colors[i] + 1"? (The initializer of "end" could also
be "start", but I guess the compiler will recognize this anyway.) This
would then (imo) also better justify the desire for having "end" in
the first place.

> +              i++, end++ );

Imo for clarity the semicolon want to live on its own line.

> +static void dump_coloring_info(unsigned char key)

This being common code now, I think it would be good practice to have
cf_check here right away, even if for now (for whatever reason) the
feature is meant to be limited to Arm. (Albeit see below for whether
this is to remain that way.)

> +void __init llc_coloring_init(void)
> +{
> +    if ( !llc_way_size && !(llc_way_size = get_llc_way_size()) )
> +        panic("Probed LLC coloring way size is 0 and no custom value found\n");
> +
> +    /*
> +     * The maximum number of colors must be a power of 2 in order to correctly
> +     * map them to bits of an address, so also the LLC way size must be so.
> +     */
> +    if ( llc_way_size & (llc_way_size - 1) )
> +        panic("LLC coloring way size (%u) isn't a power of 2\n", llc_way_size);
> +
> +    max_nr_colors = llc_way_size >> PAGE_SHIFT;

With this unconditionally initialized here, what's the purpose of the
variable's initializer?

> +    if ( max_nr_colors < 2 || max_nr_colors > CONFIG_NR_LLC_COLORS )
> +        panic("Number of LLC colors (%u) not in range [2, %u]\n",
> +              max_nr_colors, CONFIG_NR_LLC_COLORS);

I'm not convinced of panic()ing here (including the earlier two
instances). You could warn, taint, disable, and continue. If you want
to stick to panic(), please justify doing so in the description.

Plus, if you panic(), shouldn't that be limited to llc_coloring_enabled
being true? Or - not visible here, due to the lack of a caller of the
function - is that meant to be taken care of by the caller (to not call
here when the flag is off)? I think it would be cleaner if the check
lived here; quite possibly that would then further permit the flag
variable to become static.

> +    register_keyhandler('K', dump_coloring_info, "dump LLC coloring info", 1);

I'm also not convinced of using a separate key for this little bit of
information. How about attaching this to what 'm' or 'H' produce?

> +    arch_llc_coloring_init();
> +}
> +
> +void domain_dump_llc_colors(const struct domain *d)
> +{
> +    printk("Domain %pd has %u LLC colors: ", d, d->num_llc_colors);

%pd resolves to d<N> - why "Domain" as a prefix? And really - why the
domain identifier in the first place? All surrounding information is
already for this very domain.

> +    print_colors(d->llc_colors, d->num_llc_colors);

Imo this (or perhaps even the entire function) wants skipping when
num_llc_colors is zero, which would in particular also cover the
!llc_coloring_enabled case.

> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -626,6 +626,11 @@ struct domain
>  
>      /* Holding CDF_* constant. Internal flags for domain creation. */
>      unsigned int cdf;
> +
> +#ifdef CONFIG_LLC_COLORING
> +    unsigned const int *llc_colors;

const unsigned int * please.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 13:30:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 13:30:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674415.1049287 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVX9H-0003me-Qx; Thu, 01 Feb 2024 13:30:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674415.1049287; Thu, 01 Feb 2024 13:30:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVX9H-0003mX-NP; Thu, 01 Feb 2024 13:30:35 +0000
Received: by outflank-mailman (input) for mailman id 674415;
 Thu, 01 Feb 2024 13:30:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVX9H-0003mR-0j
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 13:30:35 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 13558d97-c106-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 14:30:33 +0100 (CET)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-55f19a3ca7aso3899898a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 05:30:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ty8-20020a170907c70800b00a353d1a19a9sm6117061ejc.191.2024.02.01.05.30.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 05:30:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13558d97-c106-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706794233; x=1707399033; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1YU4wWiLq7QGViTJ8twYyIsIRAeG5ZtwwK8ol7Dc0W4=;
        b=SGazCaxJT2cSx/06lN6VXMdvWCYGuQ0Vw5ZPuvebC2UU28pP0smR//FbRmAz14iLey
         t48jp7IA9J60klspTK5GELimKepMhiWVbbAhPMY9f9gB8irj9TFSu0GNlv5v/B7G2kez
         NN11hOSB2/v9+pbm4DoboUGANTujMRxroNS1X6tC69iEwEEuqftnJKyZ5sX4wjIVC9V5
         6J64E8OkE51nGg+jUbu9lBHlvaBX+y+NqIzDY2CaA9BCai0oJHLmzMt5cWS34AjU4V+Z
         iXT+2fc+2ybAu0fOg1Mk4Lp0EpQA3Y7RQTTKGt5lOG9d1TCPla6YlN58E5zmszADgqlu
         zH/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706794233; x=1707399033;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1YU4wWiLq7QGViTJ8twYyIsIRAeG5ZtwwK8ol7Dc0W4=;
        b=dUiFUZTCVHFBj+le3vtgbHUOGpGn4cfwxXArpfYyPrjmPXEgNdcbddMWj99fXfQYUr
         xPQyzlAmG3ofoHetktmdSa+BuTxpFMlY2vwREX+YCYeTGnE5UysJI6ZByAFdhGoJebg9
         wKKR1SHjDilkx87j7wmlHdfBLXlhFP6R2Yu2BCp/7xZextwxkp9tgAl9hrsv0fOprHnC
         tYMS4/APCQaCUaVF2DjRSlUugV6qKwL/GTumus4/UuQQKIIX54/4G7UysZcbjDkwG0xR
         gXYz6QyL4yhypbd68z+AZjprX/9v7wqEvUhTSZPG5kc6faVFxqzzCwuSHVOXiPqsCOuG
         Lu+w==
X-Gm-Message-State: AOJu0Yzfz9q5x76b/aPa/wmHQdEkxhAs/eKiZhmK9oIh918MjhKH5h6K
	LVdhs322MLNsOO5xJG4NRTGwRq1HxzbrUbwDOYS65r4XiU4gabgfbP0JOcly4g==
X-Google-Smtp-Source: AGHT+IG0zbt/Kby5a0Ec6Dd4UsqOjpX3KN0wfRbWosqH+TkEDgZfimjmzIjRrxyxy83V9JlR7ln9EQ==
X-Received: by 2002:a17:907:d50b:b0:a36:85b9:cd1b with SMTP id wb11-20020a170907d50b00b00a3685b9cd1bmr4066674ejc.25.1706794232927;
        Thu, 01 Feb 2024 05:30:32 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUvwMi1o4O1O3XGlvYlCzXDYOfiWfPTcXsJzKOvVrT3BOnXf2gCPv6fW/trBSunXLl9f+P+x61r9OstYSLn+Mxc9dor+2jRellnSxHTVaUglgo3mWh0Qjc/tV16PNeN45uj+0844kuxe5DvKlnGbwg9Q1Aj0cT/vCWj1LeNppUlwy9U+Sjco9WYDrfJGlGtJ96UqSw1ZM0283As9U23OPBHmw61neh/J+8JrYHnvSjFQFEtvKg1Rku6oU6gS2qarnXqGLFDNj4XpyME/um0wiovvPG8mFO6nD0UpCHSoBW3kk+uBBlLonRfQJQx+4H5WwsB7VAvodmyuwkV1JrQPPwssJ0L2jgElbaxMWACbZ3VQTaKqMJlxk8FTY5vYAhIKr4LEw==
Message-ID: <cf23d8a8-7111-4014-adc7-93ecd5f110ec@suse.com>
Date: Thu, 1 Feb 2024 14:30:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 04/15] xen/arm: add Dom0 cache coloring support
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-5-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-5-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:18, Carlo Nonato wrote:
> Add a command line parameter to allow the user to set the coloring
> configuration for Dom0.
> A common configuration syntax for cache colors is introduced and
> documented.
> Take the opportunity to also add:
>  - default configuration notion.
>  - function to check well-formed configurations.
> 
> Direct mapping Dom0 isn't possible when coloring is enabled, so
> CDF_directmap flag is removed when creating it.

What implications does this have?

> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -963,6 +963,15 @@ Controls for the dom0 IOMMU setup.
>  
>  Specify a list of IO ports to be excluded from dom0 access.
>  
> +### dom0-llc-colors
> +> `= List of [ <integer> | <integer>-<integer> ]`
> +
> +> Default: `All available LLC colors`
> +
> +Specify dom0 LLC color configuration. This options is available only when
> +`CONFIG_LLC_COLORING` is enabled. If the parameter is not set, all available
> +colors are used.

Even Arm already has a "dom0=" option. Is there a particular reason why
this doesn't become a new sub-option there?

As to meaning: With just a single <integer>, that's still a color value
then (and not a count of colors)? Wouldn't it make sense to have a
simpler variant available where you just say how many, and a suitable
set/range is then picked?

Finally a nit: "This option is ...".

> @@ -2188,10 +2190,16 @@ void __init create_dom0(void)
>              panic("SVE vector length error\n");
>      }
>  
> -    dom0 = domain_create(0, &dom0_cfg, CDF_privileged | CDF_directmap);
> +    if ( !llc_coloring_enabled )
> +        flags |= CDF_directmap;
> +
> +    dom0 = domain_create(0, &dom0_cfg, flags);
>      if ( IS_ERR(dom0) )
>          panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
>  
> +    if ( llc_coloring_enabled && (rc = dom0_set_llc_colors(dom0)) )
> +        panic("Error initializing LLC coloring for domain 0 (rc = %d)", rc);

As for the earlier patch, I find panic()ing here dubious. You can continue
quite fine, with a warning and perhaps again tainting the system.

> --- a/xen/common/llc-coloring.c
> +++ b/xen/common/llc-coloring.c
> @@ -17,6 +17,63 @@ size_param("llc-way-size", llc_way_size);
>  /* Number of colors available in the LLC */
>  static unsigned int __ro_after_init max_nr_colors = CONFIG_NR_LLC_COLORS;
>  
> +static unsigned int __initdata dom0_colors[CONFIG_NR_LLC_COLORS];
> +static unsigned int __initdata dom0_num_colors;
> +
> +/*
> + * Parse the coloring configuration given in the buf string, following the
> + * syntax below.
> + *
> + * COLOR_CONFIGURATION ::= COLOR | RANGE,...,COLOR | RANGE
> + * RANGE               ::= COLOR-COLOR
> + *
> + * Example: "0,2-6,15-16" represents the set of colors: 0,2,3,4,5,6,15,16.
> + */
> +static int parse_color_config(const char *buf, unsigned int *colors,
> +                              unsigned int num_colors, unsigned int *num_parsed)

Is this function going to be re-used? If not, it wants to be __init.
If so, I wonder where the input string is going to come from ...

Also "num_colors" looks to be misnamed - doesn't this specify an
upper bound only?

> +{
> +    const char *s = buf;
> +
> +    if ( !colors || !num_colors )
> +        return -EINVAL;

Why do you check colors but not ...

> +    *num_parsed = 0;

... num_parsed? I think internal functions don't need such NULL checks.

> +    while ( *s != '\0' )
> +    {
> +        if ( *s != ',' )

Hmm, this way you also accept leading/trailing commas as well as multiple
consecutive ones. Elsewhere we're more strict.

> @@ -70,12 +150,85 @@ void __init llc_coloring_init(void)
>      arch_llc_coloring_init();
>  }
>  
> +void domain_llc_coloring_free(struct domain *d)
> +{
> +    xfree(__va(__pa(d->llc_colors)));

This __va(__pa()) trick deserves a comment, I think.

> +}
> +
>  void domain_dump_llc_colors(const struct domain *d)
>  {
>      printk("Domain %pd has %u LLC colors: ", d, d->num_llc_colors);
>      print_colors(d->llc_colors, d->num_llc_colors);
>  }
>  
> +static unsigned int *alloc_colors(unsigned int num_colors)
> +{
> +    unsigned int *colors;
> +
> +    if ( num_colors > max_nr_colors )
> +        return NULL;

Shouldn't check_colors() have made sure of this? If so, convert to
ASSERT()?

> +    colors = xmalloc_array(unsigned int, num_colors);
> +    if ( !colors )
> +        return NULL;

These last two lines are redundant with ...

> +    return colors;

... this one. Question then is whether this is useful at all as a
separate helper function.

> +}
> +
> +static int domain_check_colors(const struct domain *d)
> +{
> +    if ( !d->num_llc_colors )
> +    {
> +        printk(XENLOG_ERR "No LLC color config found for %pd\n", d);
> +        return -ENODATA;
> +    }
> +    else if ( !check_colors(d->llc_colors, d->num_llc_colors) )

I generally recommend against use of "else" in cases like this one.

> +    {
> +        printk(XENLOG_ERR "Bad LLC color config for %pd\n", d);
> +        return -EINVAL;
> +    }
> +
> +    return 0;
> +}
> +
> +static int domain_set_default_colors(struct domain *d)
> +{
> +    unsigned int *colors = alloc_colors(max_nr_colors);
> +    unsigned int i;
> +
> +    if ( !colors )
> +        return -ENOMEM;
> +
> +    printk(XENLOG_WARNING
> +           "LLC color config not found for %pd, using default\n", d);

Leaving open what the default(s) is/are. Judging from ...

> +    for ( i = 0; i < max_nr_colors; i++ )
> +        colors[i] = i;

... this it's simply "all colors". Then perhaps have the message also
say so?

> +    d->llc_colors = colors;
> +    d->num_llc_colors = max_nr_colors;
> +
> +    return 0;
> +}
> +
> +int __init dom0_set_llc_colors(struct domain *d)
> +{
> +    unsigned int *colors;
> +
> +    if ( !dom0_num_colors )
> +        return domain_set_default_colors(d);
> +
> +    colors = alloc_colors(dom0_num_colors);
> +    if ( !colors )
> +        return -ENOMEM;
> +
> +    memcpy(colors, dom0_colors, sizeof(unsigned int) * dom0_num_colors);

sizeof(*colors) or some such please. Plus a check that colors and
dom0_colors are actually of the same type. Alternatively, how about
making dom0_colors[] __ro_after_init? Is this too much of a waste?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 13:32:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 13:32:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674420.1049297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXAr-0004Tc-4r; Thu, 01 Feb 2024 13:32:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674420.1049297; Thu, 01 Feb 2024 13:32:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXAr-0004TV-1W; Thu, 01 Feb 2024 13:32:13 +0000
Received: by outflank-mailman (input) for mailman id 674420;
 Thu, 01 Feb 2024 13:32:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVXAq-0004Sp-Ew
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 13:32:12 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4d5c9c05-c106-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 14:32:10 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a363961b96aso116845466b.3
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 05:32:10 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ty8-20020a170907c70800b00a353d1a19a9sm6117061ejc.191.2024.02.01.05.32.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 05:32:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d5c9c05-c106-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706794330; x=1707399130; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HSbL1ahC/OCghSLFI8J5zchzx+wbFweFqGoB6NFWkk0=;
        b=Ljpcmgj0HULTlfTyQ9R7Ih1X+PniRoaty+B++Yx+LI51aO6024FMTqKoSrInPRokTt
         kjdwW0BfWxuOB0i5nzV3jLF2GR6ZePD7PV56YiZLwRGVCEQrzHdDpnhCKVLqSPLqUavp
         4oEmgxRweQTatxB7ZtiHl6mDj9WD0rLaTTzTnhTFcv0RatWDl6rUJLNxBlxRL3j15voc
         CKsiXH9kKuMVTQXdFFCnIQ6A8A6pu4FVlq96BM2tyJTx5BUvHrASSb54MCyXQeCzEy5I
         lKx43WGf5+GCYKgm96E6B8QLie6VfQomeqtjI44Z5BJGsEMj4GJoihu5kQBh2WLdL4FG
         TnKQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706794330; x=1707399130;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HSbL1ahC/OCghSLFI8J5zchzx+wbFweFqGoB6NFWkk0=;
        b=FxUelkZ8egffsAkwpS8UcrYhtzZE3aXgIaS7tNim1govlgQkPnDNPtq/Memr+WRblb
         KJXr78YBvySGxLd8qEbNEa8uKkakAMxCW5LutUeEqJWrQgdHyAtwEhILjCnkL697gEzo
         5wuCik62CboY0Zu7yVTNb0iRN8HncXCz2y8CynDJSGFi1lZlZbDTQVjJW2nhnWgoXwJp
         vh/evaZJaLmGDcmmSh5oFTltf5r73RYPh59EHVoiWBqpDbPbZCqSm2cLOsB1ZTgEzpE5
         FlxpQCR390kV6tLcyQ1S0A/b08UUlZ/TSB0sZu1QNJ04ktn74uFPuJg5s8aJzDw3RaSE
         1dyQ==
X-Gm-Message-State: AOJu0YxAQ1KCY6tQtgasi3RMSVnar39CBHbKdgndpnuMi3arabTUDi5v
	/o+I8XeS+KfSR+TX1vrj4fRxzDS2daxVKfzkwLLelEB2rc5v/pMHFYGSJzl6nA==
X-Google-Smtp-Source: AGHT+IEYlUuzvJFTDcaly8PjGRwlnOCmuFO1sYrOi/L1Dkaf3e95uQwbB+VihWBnrzyKOJ2wIBdaLQ==
X-Received: by 2002:a17:906:c78b:b0:a33:1d7d:912f with SMTP id cw11-20020a170906c78b00b00a331d7d912fmr4014306ejb.8.1706794330225;
        Thu, 01 Feb 2024 05:32:10 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUXS1Fst4W2B5k+letico01XszACO8b/dq/eXMxAKOsAT/y74D9/t8FH9vpIIg9fhsm6aUhaYGsfyFUcqdE0mZ5Z3gdSPIqIE+8px36tptwSre2bO7nByoFXGPHc5er9fBy4t3Bu7vy26112MA+XewsIM0NTjW7J1/LcojUbRYoLJ2fSLOhN14JCaB9yuZvQufBYG96/mzei2KN2q21ehXCpViGMMMG/wRew2Amr89uPNVQ/x5hdfLinkAyg18ixRbcArcBMc2pKNdhCCqdS1qWW5OswPWE
Message-ID: <5e309560-51a8-4b9a-962d-819470375f62@suse.com>
Date: Thu, 1 Feb 2024 14:32:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 01/15] xen/common: add cache coloring common code
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-2-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-2-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:17, Carlo Nonato wrote:
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -626,6 +626,11 @@ struct domain
>  
>      /* Holding CDF_* constant. Internal flags for domain creation. */
>      unsigned int cdf;
> +
> +#ifdef CONFIG_LLC_COLORING
> +    unsigned const int *llc_colors;
> +    unsigned int num_llc_colors;
> +#endif
>  };

Btw, at this point flipping the order of the two fields will be more
efficient for 64-bit architectures (consuming a padding hole rather
than adding yet another one).

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 13:32:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 13:32:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674423.1049306 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXBR-0004yD-CN; Thu, 01 Feb 2024 13:32:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674423.1049306; Thu, 01 Feb 2024 13:32:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXBR-0004y6-9l; Thu, 01 Feb 2024 13:32:49 +0000
Received: by outflank-mailman (input) for mailman id 674423;
 Thu, 01 Feb 2024 13:32:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+ZK+=JK=gmail.com=dunlapg@srs-se1.protection.inumbo.net>)
 id 1rVXBQ-0004k9-06
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 13:32:48 +0000
Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com
 [2a00:1450:4864:20::436])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 62b1199c-c106-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 14:32:46 +0100 (CET)
Received: by mail-wr1-x436.google.com with SMTP id
 ffacd0b85a97d-33b14702adaso414497f8f.3
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 05:32:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62b1199c-c106-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=umich.edu; s=google-2016-06-03; t=1706794366; x=1707399166; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=q1N1WhUa+zlUda6ded9+L5IGCZd1kPELbcb28F33NvM=;
        b=HMXydHZ2OiqMc/911YWyxB09s8pcu5ffQElF66mzj4TPpSp3s821tvkSUPI1W+buHV
         HldUL4wNmjoLMQCWuIVQb1ERazG39qAX3frvnqIxP+IXk+0o5wARLWi8fr9Nvkp7VnuY
         VlEaBYbPsWgpZBgWhczTE+M0iS/0OE9Np/gVERMPKdT8we0m0d26qHVT5FKINhBVnsNp
         VFrk+DXtGp7tZENpGDFSkl/0zYZT86kpi3Jfk6tjS0/pWyrpUgsxICCJmiWgxHG9aupY
         S9TxkJ3m7Od7QKbvY52wAq3n2caaB8JwCFKtfXVmM/3Qc+PFFMo0mDATJ8aFKXQ46GuI
         Gh4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706794366; x=1707399166;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=q1N1WhUa+zlUda6ded9+L5IGCZd1kPELbcb28F33NvM=;
        b=gVCazNyqwIM74GLQPrxHzdHNIuAtaicuO87DHI/UFpuQ46Gdl5uh/cHdPOsaTKFkzP
         3U9IZ9UJjsuibbOlRsWBHQLvwfg29M8z9KsFtc0axpdIpSdlcJx1uByNGy+3o1kMLgUB
         wWo8+bAU9thrSFVwbYw0bTvG/lN7s8zDtXW2mhmDQEVm7D5c7pI1VAxqu2FfYBWANoro
         Kl9qfNXNeXVOXLlfscArymYoYTpyv6x0FdZAWVkK8F7AtAjA+MQC4ENNCkN9EWoTVE39
         p/SSGQ3QgK76vCsNGqzn6JQVxki2BS8LjzAxhpgVA7OJycboFX0Mhrvgi/AVyspvOt4n
         ueGQ==
X-Gm-Message-State: AOJu0Yy0JPCT9vbbdZExBmv+H55dUXyG2I5EI31wBepyGcJHSWGYSWhp
	EMU+O7Vp18qzEGl1Q8Cr72rtlkKsJvWmuFQ9YW2/rgG+QwQFheAsTMBY5J9+UEKVquCONibuZqQ
	4hDzUk9i+CIuNeTDf5ONAhn9ZtNI=
X-Google-Smtp-Source: AGHT+IHy2GjZWx2D3UbxOtDX7JOyRUDWbtychyj3EAprrf88Hk0njswWMcO7Dgc0SCwAMARJcQsFgQOWPsqvhYV2ejQ=
X-Received: by 2002:a5d:4b04:0:b0:33b:e5f:edec with SMTP id
 v4-20020a5d4b04000000b0033b0e5fedecmr2584518wrq.16.1706794365798; Thu, 01 Feb
 2024 05:32:45 -0800 (PST)
MIME-Version: 1.0
References: <5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com> <7cce89f4-962e-bfbe-7d30-18fea7515bed@suse.com>
In-Reply-To: <7cce89f4-962e-bfbe-7d30-18fea7515bed@suse.com>
From: George Dunlap <dunlapg@umich.edu>
Date: Thu, 1 Feb 2024 13:32:34 +0000
Message-ID: <CAFLBxZZLJMWpf1fCNN4dhoDpYpW6O=V_C==-sCAZy6t4Df_yBw@mail.gmail.com>
Subject: Re: [PATCH v2 2/2] x86/p2m: aid the compiler in folding p2m_is_...()
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: multipart/alternative; boundary="000000000000e0ccbb0610520702"

--000000000000e0ccbb0610520702
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Jun 23, 2022 at 12:54=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wr=
ote:

> By using | instead of || or (in the negated form) && chances increase
> for the compiler to recognize that both predicates can actually be
> folded into an expression requiring just a single branch (via OR-ing
> together the respective P2M_*_TYPES constants).
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>

Sorry for the delay.  Git complains that this patch is malformed:

error: `git apply --index`: error: corrupt patch at line 28

Similar complaint from patchew when it was posted:

https://patchew.org/Xen/5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com/

 -George

--000000000000e0ccbb0610520702
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Thu, Jun 23, 2022 at 12:54=E2=80=
=AFPM Jan Beulich &lt;<a href=3D"mailto:jbeulich@suse.com">jbeulich@suse.co=
m</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin=
:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"=
>By using | instead of || or (in the negated form) &amp;&amp; chances incre=
ase<br>
for the compiler to recognize that both predicates can actually be<br>
folded into an expression requiring just a single branch (via OR-ing<br>
together the respective P2M_*_TYPES constants).<br>
<br>
Signed-off-by: Jan Beulich &lt;<a href=3D"mailto:jbeulich@suse.com" target=
=3D"_blank">jbeulich@suse.com</a>&gt;<br></blockquote><div><br></div><div>S=
orry for the delay.=C2=A0 Git complains that this patch is malformed:</div>=
<div><div><br></div><div>error: `git apply --index`: error: corrupt patch a=
t line 28<br></div><div><br></div><div>Similar complaint from patchew when =
it was posted:</div><div><br></div><div><a href=3D"https://patchew.org/Xen/=
5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com/" target=3D"_blank">https://p=
atchew.org/Xen/5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com/</a></div><fon=
t color=3D"#888888"><div><br></div><div>=C2=A0-George=C2=A0</div></font></d=
iv><div>=C2=A0</div></div></div>

--000000000000e0ccbb0610520702--


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 13:35:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 13:35:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674429.1049317 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXEE-0005bB-Q6; Thu, 01 Feb 2024 13:35:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674429.1049317; Thu, 01 Feb 2024 13:35:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXEE-0005b4-ND; Thu, 01 Feb 2024 13:35:42 +0000
Received: by outflank-mailman (input) for mailman id 674429;
 Thu, 01 Feb 2024 13:35:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rVXED-0005ay-7r
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 13:35:41 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVXE8-00018T-5A; Thu, 01 Feb 2024 13:35:36 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVXE7-0005nq-TT; Thu, 01 Feb 2024 13:35:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=6zsy4WakuTn74XC7bvekR33avbJIFGCKsBaSH8B/Nkk=; b=vgnHXgXOOytksbrFMm8skNetQ8
	C/J63cs7CWC1BoSur++uWVpGw1Ck3fT1PlFMG4PkNTlXR3vMVlHktDKAP07AKz8g3ZHyI+YK7NtAV
	eWzTJjP6tLHtpdyGnLUVyCj6ZgeuWmc1Hi6r3qVecgrb3z2Xz1bEqnpsRDA1FPi1UPUw=;
Message-ID: <12b2d25a-ff80-45d8-ad3e-fca6684508df@xen.org>
Date: Thu, 1 Feb 2024 13:35:33 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 04/15] xen/arm: add Dom0 cache coloring support
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-5-carlo.nonato@minervasys.tech>
 <cf23d8a8-7111-4014-adc7-93ecd5f110ec@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <cf23d8a8-7111-4014-adc7-93ecd5f110ec@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 01/02/2024 13:30, Jan Beulich wrote:
> On 29.01.2024 18:18, Carlo Nonato wrote:
>> Add a command line parameter to allow the user to set the coloring
>> configuration for Dom0.
>> A common configuration syntax for cache colors is introduced and
>> documented.
>> Take the opportunity to also add:
>>   - default configuration notion.
>>   - function to check well-formed configurations.
>>
>> Direct mapping Dom0 isn't possible when coloring is enabled, so
>> CDF_directmap flag is removed when creating it.
> 
> What implications does this have?
> 
>> --- a/docs/misc/xen-command-line.pandoc
>> +++ b/docs/misc/xen-command-line.pandoc
>> @@ -963,6 +963,15 @@ Controls for the dom0 IOMMU setup.
>>   
>>   Specify a list of IO ports to be excluded from dom0 access.
>>   
>> +### dom0-llc-colors
>> +> `= List of [ <integer> | <integer>-<integer> ]`
>> +
>> +> Default: `All available LLC colors`
>> +
>> +Specify dom0 LLC color configuration. This options is available only when
>> +`CONFIG_LLC_COLORING` is enabled. If the parameter is not set, all available
>> +colors are used.
> 
> Even Arm already has a "dom0=" option. Is there a particular reason why
> this doesn't become a new sub-option there?
> 
> As to meaning: With just a single <integer>, that's still a color value
> then (and not a count of colors)? Wouldn't it make sense to have a
> simpler variant available where you just say how many, and a suitable
> set/range is then picked?
> 
> Finally a nit: "This option is ...".
> 
>> @@ -2188,10 +2190,16 @@ void __init create_dom0(void)
>>               panic("SVE vector length error\n");
>>       }
>>   
>> -    dom0 = domain_create(0, &dom0_cfg, CDF_privileged | CDF_directmap);
>> +    if ( !llc_coloring_enabled )
>> +        flags |= CDF_directmap;
>> +
>> +    dom0 = domain_create(0, &dom0_cfg, flags);
>>       if ( IS_ERR(dom0) )
>>           panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
>>   
>> +    if ( llc_coloring_enabled && (rc = dom0_set_llc_colors(dom0)) )
>> +        panic("Error initializing LLC coloring for domain 0 (rc = %d)", rc);
> 
> As for the earlier patch, I find panic()ing here dubious. You can continue
> quite fine, with a warning and perhaps again tainting the system.
There are arguments for both approach. I agree that you can continue but 
technically this is not the configuration you asked. Someone may not 
notice the tainting until it is too late (read they have done 
investigation).

Bear in mind that the user for cache coloring will be in very 
specialized environment. So if you can't enable cache coloring in 
production, then something really wrong has happened and continue to 
boot is probably not right.

This matches the approach for Arm we have been using since the 
beginning. And I will strongly argue to continue this way.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 13:39:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 13:39:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674433.1049326 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXI5-0007Kj-CJ; Thu, 01 Feb 2024 13:39:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674433.1049326; Thu, 01 Feb 2024 13:39:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXI5-0007K2-9F; Thu, 01 Feb 2024 13:39:41 +0000
Received: by outflank-mailman (input) for mailman id 674433;
 Thu, 01 Feb 2024 13:39:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVXI3-0007If-8T
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 13:39:39 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5807a138-c107-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 14:39:38 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-55f2b0c5ae9so1187399a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 05:39:38 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e6-20020a50ec86000000b0055fbc52457fsm415569edr.30.2024.02.01.05.39.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 05:39:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5807a138-c107-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706794778; x=1707399578; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=XxXKa9rZUl8KpqDrwZu8AgUKl5L3iDiVhgOfVsooojw=;
        b=elFg1tMGExKhVZIY52162xXonEH9dbsp0d3WhHbDc+/zqsGP1jIeyXg6vHfK8B9jio
         ivD+aICmCPXzhImNBUYujlH/6VACqxbFDuWnbMxnjReJAWMkB5oxWFsRMftWj49WsgnI
         1keqCfSE5sqiRBL4/cS47NCMXsP7PM4VRYv7jpIzCzTazeYhnrvNGoC2WX0ABbBcSl5h
         nvqUk9y/RNV2/YETIyZNdzFScwTudUz6Ou0/2DUapx74g5JLEu2sLPP66mxDMmv0ChCz
         pgT9ahzCqZs09c7T1+S3b+4qmuV2TGooczIOe19312sycIGwKGm4wYk8LCI8zDtvk2gu
         iY7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706794778; x=1707399578;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=XxXKa9rZUl8KpqDrwZu8AgUKl5L3iDiVhgOfVsooojw=;
        b=q00IMasrLI1x/OwM5f/os7Yp27NvpAkHiCa3UUfCd3/D53skJTirvFMtmLWjJxF2n/
         I3Q7+NFri+z5W0Q4sPfQXo5MmJCCUD/CHI6q39Ean8+P4HdQtQ77SxHkbjn0VwtBgBAj
         yTouZlWnxw4L3qbpVQysJcdDEjnyW3dhIVsPrFXNFyqP8RmjRL3J7s5dKq6JM4+v8SgP
         gwUep5Uoi888zxaCcNOY4tz3p01OODK6tCLPDSl0BEdh+9C2jnMQsWxdHu1YV7RFIZ3E
         VGLvCFQs9Rrsc348y0ivk30D+NaoEM7RducVx0GOeflZEQ9CsYWuWkVDFIjTjEMBDDUp
         vzag==
X-Gm-Message-State: AOJu0Yx2ZSqhVmtMXq2AlLSlieBj91EEdUzQ4qBZImOAoy+oZBsACxlx
	laE/xPY+UAGVrD8BWmMjdpZhqh85jbBiDMdS/vsNxNMSQy+QfloV2jNLqxeORQ==
X-Google-Smtp-Source: AGHT+IGECh5aNBsOigEpT3v2t8BrUbtPJKPmkXoF3D02N6X817HjuwqShtttIA0jixaptbd4Ahmznw==
X-Received: by 2002:aa7:d386:0:b0:55f:64ca:5a6e with SMTP id x6-20020aa7d386000000b0055f64ca5a6emr3120308edq.12.1706794777653;
        Thu, 01 Feb 2024 05:39:37 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVngAvbmO7RP9zivsg+O4UQGLFVvLk02FwXI2a/EG1PyOocMEwhB2trjeybxR1RWKQ5RUqqCVq0B4T+x5z9druc/i4xRXubtgBdwbI4k5El1fawvvNPN+lzrRZvr68WM0h/MYbMmBrsdYY3JtMbyAngNL184PljLmCyJ5DogdG2XcYC+YzlRz1Yu4QjhrnGYnc7iPBGqEpTBbwNfzyeQ+c2G4vHDzxWVTxVYm4YXDeGEAoVvju/shVdxOJS0fuigYGMJbTWSorfPKB2jyy5VaFoD4KXTw/3034xLI4njBax9l8xawNsT+bILoQPTFQ1Br/PkrHUH/rikyV1oeilbfOJ234PjBwHkCqTO3wpMPV1U4aRGJf0Rnzmm7B2pHe0+B9KYX1FB678fcSxFWLDknz4
Message-ID: <02b2905d-cc83-4c39-bc29-2ac8351bfc37@suse.com>
Date: Thu, 1 Feb 2024 14:39:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 04/15] xen/arm: add Dom0 cache coloring support
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org, Carlo Nonato <carlo.nonato@minervasys.tech>
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-5-carlo.nonato@minervasys.tech>
 <cf23d8a8-7111-4014-adc7-93ecd5f110ec@suse.com>
 <12b2d25a-ff80-45d8-ad3e-fca6684508df@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <12b2d25a-ff80-45d8-ad3e-fca6684508df@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.02.2024 14:35, Julien Grall wrote:
> Hi Jan,
> 
> On 01/02/2024 13:30, Jan Beulich wrote:
>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>> Add a command line parameter to allow the user to set the coloring
>>> configuration for Dom0.
>>> A common configuration syntax for cache colors is introduced and
>>> documented.
>>> Take the opportunity to also add:
>>>   - default configuration notion.
>>>   - function to check well-formed configurations.
>>>
>>> Direct mapping Dom0 isn't possible when coloring is enabled, so
>>> CDF_directmap flag is removed when creating it.
>>
>> What implications does this have?
>>
>>> --- a/docs/misc/xen-command-line.pandoc
>>> +++ b/docs/misc/xen-command-line.pandoc
>>> @@ -963,6 +963,15 @@ Controls for the dom0 IOMMU setup.
>>>   
>>>   Specify a list of IO ports to be excluded from dom0 access.
>>>   
>>> +### dom0-llc-colors
>>> +> `= List of [ <integer> | <integer>-<integer> ]`
>>> +
>>> +> Default: `All available LLC colors`
>>> +
>>> +Specify dom0 LLC color configuration. This options is available only when
>>> +`CONFIG_LLC_COLORING` is enabled. If the parameter is not set, all available
>>> +colors are used.
>>
>> Even Arm already has a "dom0=" option. Is there a particular reason why
>> this doesn't become a new sub-option there?
>>
>> As to meaning: With just a single <integer>, that's still a color value
>> then (and not a count of colors)? Wouldn't it make sense to have a
>> simpler variant available where you just say how many, and a suitable
>> set/range is then picked?
>>
>> Finally a nit: "This option is ...".
>>
>>> @@ -2188,10 +2190,16 @@ void __init create_dom0(void)
>>>               panic("SVE vector length error\n");
>>>       }
>>>   
>>> -    dom0 = domain_create(0, &dom0_cfg, CDF_privileged | CDF_directmap);
>>> +    if ( !llc_coloring_enabled )
>>> +        flags |= CDF_directmap;
>>> +
>>> +    dom0 = domain_create(0, &dom0_cfg, flags);
>>>       if ( IS_ERR(dom0) )
>>>           panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
>>>   
>>> +    if ( llc_coloring_enabled && (rc = dom0_set_llc_colors(dom0)) )
>>> +        panic("Error initializing LLC coloring for domain 0 (rc = %d)", rc);
>>
>> As for the earlier patch, I find panic()ing here dubious. You can continue
>> quite fine, with a warning and perhaps again tainting the system.
> There are arguments for both approach.

In which case - perhaps allow for both? With a Kconfig-established
default and a command line option to override?

> I agree that you can continue but 
> technically this is not the configuration you asked. Someone may not 
> notice the tainting until it is too late (read they have done 
> investigation).
> 
> Bear in mind that the user for cache coloring will be in very 
> specialized environment.

s/will/may/ I suppose. People may enable the option without being in
any specialized environment.

> So if you can't enable cache coloring in 
> production, then something really wrong has happened and continue to 
> boot is probably not right.
> 
> This matches the approach for Arm we have been using since the 
> beginning. And I will strongly argue to continue this way.

I'm okay with this, and here (for Arm-specific code) it may even be okay
to do so without further justification. But in the earlier patch where
common code is affected, I'll insist on at least justifying this behavior.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 13:48:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 13:48:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674440.1049336 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXQs-0001GR-69; Thu, 01 Feb 2024 13:48:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674440.1049336; Thu, 01 Feb 2024 13:48:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXQs-0001GK-3Q; Thu, 01 Feb 2024 13:48:46 +0000
Received: by outflank-mailman (input) for mailman id 674440;
 Thu, 01 Feb 2024 13:48:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+ZK+=JK=gmail.com=dunlapg@srs-se1.protection.inumbo.net>)
 id 1rVXQr-0001GE-C0
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 13:48:45 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9d82452d-c108-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 14:48:44 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-337cc8e72f5so586336f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 05:48:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d82452d-c108-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=umich.edu; s=google-2016-06-03; t=1706795324; x=1707400124; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=uAGJ40Mi5YAKkJYTVBw8bZ5DQVfGYXwY1pM2JcXYAro=;
        b=ahqFhiZepKwvOq6s3o0cYYqp0wP7EtlJ4xfGJNiBSaTrhx1XsrN26inzDOedg2scMC
         pWDCN1S5J5YH9X5YXWDOli+yfoKM5+jKtzIpF4H8SufG2WrUVqXhafp8MV9bS2FzwtVr
         3LEqYGMbjnOkAOj8TV+PgK9q1f0YK/KNEUYucnk4QAqelUpxPLjDHmPrL34FsifTD1S7
         DY9ohoAX1E9MBEXr4e+RM2KQnXAJZ9PTzOmooJHWmWvXGkRy2TMiVff3LZNEQiqBnZGE
         rM9W+qpzZGyZFE7IrWtLJbVsWEkEcUr5iajnV1ebp6OT5qaxCH/WVj1b2Z8oUeGkVxB6
         HM9A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706795324; x=1707400124;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=uAGJ40Mi5YAKkJYTVBw8bZ5DQVfGYXwY1pM2JcXYAro=;
        b=WEnn/qqrTFOjGbVbnqa56O16Hr3VtocvUpMxhrVFbdIkMe5p8DU+sdH5QWcNzZihzt
         5pQuAi2vlNDcUiTnljsKGMjsQb54/UbdXPN/XPFRe5Faao6oQsntqRj1yAPQZdbMGTcT
         D7UAm2MzVKE0/lPKG1kem0Ghwx3JOdLU2WL7jMhHQaItmhfwrCr4uAVk+B1lO+5vyDyh
         t8PZLlhZJPktVJrDNS7uGtmsxjnKvkEj5otG7wH5YMPstGXLiFvAXn/ryLDm2DnjAmsa
         TXTAMWBkOaFLosTTdC6+eHM6FX33F4cX/JNX/MmhfU5u2PjHNW99GgspXgz7GoY2BoK+
         my4g==
X-Gm-Message-State: AOJu0Yy6NJqUwcJZgd9sIDtomtYgH+V+ZF0hILGZTa9ZjJc+jAiyVjfg
	ftQ88U+WYHvlq5o5O47/uyxX+LPxxMNalu+InJzGI348f6SWqA7IuHdNVjTUM9dYRVAlvT2DKJn
	awNyoen0/ivrWTN6O18JIaCQerD4=
X-Google-Smtp-Source: AGHT+IE7tozFHaLnynQdgdJw3y9iEQ0VpntF/6OAUIFHKYfZhEIPGYvIBgrm0kz22Dc/kTDHdCp7PYd5aB4dp8NUV+U=
X-Received: by 2002:adf:ee10:0:b0:33a:ecd6:1c71 with SMTP id
 y16-20020adfee10000000b0033aecd61c71mr1714660wrn.56.1706795323475; Thu, 01
 Feb 2024 05:48:43 -0800 (PST)
MIME-Version: 1.0
References: <e9257e96-ede9-2809-9a77-fd4dc206badc@suse.com> <0c61e460-122c-6486-00b7-22a9a24ee1fc@suse.com>
In-Reply-To: <0c61e460-122c-6486-00b7-22a9a24ee1fc@suse.com>
From: George Dunlap <dunlapg@umich.edu>
Date: Thu, 1 Feb 2024 13:48:32 +0000
Message-ID: <CAFLBxZbUeuyLx998fpxeOCGjbgR4C_aLJcN-To9isfOcQ3iYNA@mail.gmail.com>
Subject: Re: [PATCH v2 1/3] x86/PoD: simplify / improve p2m_pod_cache_add()
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: multipart/alternative; boundary="000000000000f5cef90610524009"

--000000000000f5cef90610524009
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Jan 4, 2022 at 9:48=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wrot=
e:

> Avoid recurring MFN -> page or page -> MFN translations. Drop the pretty
> pointless local variable "p". Make sure the MFN logged in a debugging
> error message is actually the offending one. Return negative errno
> values rather than -1 (presently no caller really cares, but imo this
> should change). Adjust style.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>

Sorry, not sure how I managed to miss this:

Reviewed-by: George Dunlap <george.dunlap@cloud.com>

--000000000000f5cef90610524009
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Tue, Jan 4, 2022 at 9:48=E2=80=AFA=
M Jan Beulich &lt;<a href=3D"mailto:jbeulich@suse.com">jbeulich@suse.com</a=
>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px=
 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Avo=
id recurring MFN -&gt; page or page -&gt; MFN translations. Drop the pretty=
<br>
pointless local variable &quot;p&quot;. Make sure the MFN logged in a debug=
ging<br>
error message is actually the offending one. Return negative errno<br>
values rather than -1 (presently no caller really cares, but imo this<br>
should change). Adjust style.<br>
<br>
Signed-off-by: Jan Beulich &lt;<a href=3D"mailto:jbeulich@suse.com" target=
=3D"_blank">jbeulich@suse.com</a>&gt;<br></blockquote><div><br></div><div>S=
orry, not sure how I managed to miss this:</div><div><br></div><div><div>Re=
viewed-by: George Dunlap &lt;<a href=3D"mailto:george.dunlap@cloud.com" tar=
get=3D"_blank">george.dunlap@cloud.com</a>&gt;</div><div class=3D"gmail-yj6=
qo"></div><div class=3D"gmail-adL">=C2=A0</div></div><div class=3D"gmail-ad=
L"><br></div></div></div>

--000000000000f5cef90610524009--


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 13:51:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 13:51:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674444.1049347 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXTb-0003TA-Jf; Thu, 01 Feb 2024 13:51:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674444.1049347; Thu, 01 Feb 2024 13:51:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXTb-0003T3-H0; Thu, 01 Feb 2024 13:51:35 +0000
Received: by outflank-mailman (input) for mailman id 674444;
 Thu, 01 Feb 2024 13:51:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVXTa-0003Sx-M0
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 13:51:34 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 025962d3-c109-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 14:51:33 +0100 (CET)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-33b1a51743fso255695f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 05:51:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 o2-20020a5d4a82000000b0033b1277e95dsm2119858wrq.77.2024.02.01.05.51.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 05:51:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 025962d3-c109-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706795493; x=1707400293; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8RRdQKASc3af7mpTARYqUcV+0lPpVTBJtM4WGMrlCRw=;
        b=AQAMHaPpRg691kwz/SeL4DcMupVqnsJk3jhtovx3ZR1GsqpFv35GZw0Mku4gbmOI9b
         wWSNWbk4N376x1JslwLhA7n8MdJOx9Z8vMhqQumJReGMijTRKVOgntv5LM8VNYybNbQ6
         XcKbcotk9G5k7YZlMf5zK3Cxp2kqKM7ZMy+ZS60fsz6Zz8mbrNefy5P/lrXn+HE2a0L4
         hJNreWQHuRzSWeKIuw9aSWdfQh4OCM9RlI+gurE2ImKBU0proy49C7Q2kulAbiyE5JEw
         Wh0xCCqVWaqowtyDUBdHT9vt98cUbZocRRr86Q90EwT8OVe3H877zIuMiBGCE4mAhAQi
         B9Mg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706795493; x=1707400293;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8RRdQKASc3af7mpTARYqUcV+0lPpVTBJtM4WGMrlCRw=;
        b=dxNVEyA9kD66itUxye3UrO1LtvAzwVbpZvSRDkj0ksxpKIHznYhaa4CHuMdXFm3lir
         +45Pt+ugvlWC0Dtq/oVboRLj0S2d1xoTvLmP9bc6nJnLI4XxdyYnkXyiZsBE4K+faV6K
         6oTTeBBMO6VdOB9NPErRPC3niiwUDxcR+/ypbIK3VpahR+QDHFyDjzrml1F/+Bonot7N
         bpgyRUJrUInOG+1HP+y5Kj2DSF01MK3aLg3mNdDpcMUMvxMUZsSdsAIxR6EfJGjHLtTl
         xetarOIaJd7QKIayr7dEiaXb9FR4F2R60tq3XBt6dM6LzomImry8N0TVb832Nfz40dI3
         qTPQ==
X-Gm-Message-State: AOJu0Yzs+2enboizNcjPLTw+0zpS+X5ACd9y2iOZwI5Qt+/1N+AHVyES
	Yhmyp7mUCoGykF/mW4SLBMpQqC9Q27PwwSla8L90DtNdBXwZlNDznZw/1tuPeQ==
X-Google-Smtp-Source: AGHT+IFMBy8UxQLDea4mgrUx/zzQwIOassxiv2dP8qnvvs6//4Bn0T41OedMDEPWMSkZT6UI6Pd3Fg==
X-Received: by 2002:adf:ce8b:0:b0:33a:eae4:aaf7 with SMTP id r11-20020adfce8b000000b0033aeae4aaf7mr3489557wrn.48.1706795492933;
        Thu, 01 Feb 2024 05:51:32 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVxHvl2mTcAPoZFlqoBOxEKsYLkudUEhv4D2bXChYhJXFEbGOOqDIEcPN/gcwZMR/HfIBcRen8SPC7QV2XjKCQuwK4/fPf+KMW7Uz7xxMW2Fv7CSHUbB5CxRSYhmsDzsGUzqYo6MiSeeglG3M/XEKvldWp8IPzaaG4vsq6bwzZkBcGZI2As2YzcjlLOv4NHyFg9pSjOOBj5RKrC5WseG7andnq9bCeGxHkNjbVSy6bz0q6ok8gBVv7R8bpbhEmF57pJvhfYKMZcgAdPyg+MdRcJdHdN00RN
Message-ID: <49827753-14f1-42cc-8791-27c5400e6e50@suse.com>
Date: Thu, 1 Feb 2024 14:51:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 05/15] xen: extend domctl interface for cache coloring
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-6-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-6-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:18, Carlo Nonato wrote:
> @@ -858,6 +859,16 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>                  __HYPERVISOR_domctl, "h", u_domctl);
>          break;
>  
> +    case XEN_DOMCTL_set_llc_colors:
> +        if ( !llc_coloring_enabled )
> +            break;

With "ret" still being 0, this amounts to "successfully ignored". Ought
to be -EOPNOTSUPP, I guess.

> +        ret = domain_set_llc_colors_domctl(d, &op->u.set_llc_colors);
> +        if ( ret == -EEXIST )
> +            printk(XENLOG_ERR
> +                   "Can't set LLC colors on an already created domain\n");

If at all a dprintk(). But personally I think even that's too much - we
don't do so elsewhere, I don't think.

> --- a/xen/common/llc-coloring.c
> +++ b/xen/common/llc-coloring.c
> @@ -4,6 +4,7 @@
>   *
>   * Copyright (C) 2022 Xilinx Inc.
>   */
> +#include <xen/guest_access.h>
>  #include <xen/keyhandler.h>
>  #include <xen/llc-coloring.h>
>  #include <xen/param.h>
> @@ -229,6 +230,30 @@ int __init dom0_set_llc_colors(struct domain *d)
>      return domain_check_colors(d);
>  }
>  
> +int domain_set_llc_colors_domctl(struct domain *d,
> +                                 const struct xen_domctl_set_llc_colors *config)

What purpose has the "domctl" in the function name?

> +{
> +    unsigned int *colors;
> +
> +    if ( d->num_llc_colors )
> +        return -EEXIST;
> +
> +    if ( !config->num_llc_colors )
> +        return domain_set_default_colors(d);
> +
> +    colors = alloc_colors(config->num_llc_colors);
> +    if ( !colors )
> +        return -ENOMEM;

Hmm, I see here you call the function without first having bounds checked
the input. But in case of too big a value, -ENOMEM is inappropriate, so
such a check wants adding up front anyway.

> +    if ( copy_from_guest(colors, config->llc_colors, config->num_llc_colors) )
> +        return -EFAULT;

There again wants to be a check that the pointed to values are the same,
or at least of the same size. Else you'd need to do element-wise copy.

> +    d->llc_colors = colors;
> +    d->num_llc_colors = config->num_llc_colors;
> +
> +    return domain_check_colors(d);

And if this fails, you leave the domain with the bad settings? Shouldn't
you check and only then store pointer and count?

> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -1190,6 +1190,13 @@ struct xen_domctl_vmtrace_op {
>  typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
>  DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
>  
> +struct xen_domctl_set_llc_colors {
> +    /* IN LLC coloring parameters */
> +    uint32_t num_llc_colors;
> +    uint32_t padding;

I see you've added padding, but: You don't check it to be zero. Plus
the overwhelming majority of padding fields is named "pad".

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 13:55:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 13:55:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674448.1049357 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXX3-00043U-3Q; Thu, 01 Feb 2024 13:55:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674448.1049357; Thu, 01 Feb 2024 13:55:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXX2-00043N-Ve; Thu, 01 Feb 2024 13:55:08 +0000
Received: by outflank-mailman (input) for mailman id 674448;
 Thu, 01 Feb 2024 13:55:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rVXX1-00043G-NL
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 13:55:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVXWw-0001UW-OC; Thu, 01 Feb 2024 13:55:02 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVXWw-0006WY-HF; Thu, 01 Feb 2024 13:55:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=BWz9L10Z3/J3KrmaAbRIqLHSINybu8RaJ2IT/zQ6ddg=; b=PkLahu2Zq8kY+vanGWH6Capr1r
	zrGyRuZoltZ2UmqTB6FDokYyA76pBHhmDi8awyaTnsk34SQWStxN1rvky4ih0BiDhHjAmOleymGvh
	mk8tGyXJlio0HuvxKS+LTihpqa//nnkbaLtL9nkHou53hfrQncrVJT0BduF5ntAvTwGI=;
Message-ID: <4d913f15-f2e7-412d-a3c2-727c83df7ec1@xen.org>
Date: Thu, 1 Feb 2024 13:55:00 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 04/15] xen/arm: add Dom0 cache coloring support
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org, Carlo Nonato <carlo.nonato@minervasys.tech>
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-5-carlo.nonato@minervasys.tech>
 <cf23d8a8-7111-4014-adc7-93ecd5f110ec@suse.com>
 <12b2d25a-ff80-45d8-ad3e-fca6684508df@xen.org>
 <02b2905d-cc83-4c39-bc29-2ac8351bfc37@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <02b2905d-cc83-4c39-bc29-2ac8351bfc37@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 01/02/2024 13:39, Jan Beulich wrote:
> On 01.02.2024 14:35, Julien Grall wrote:
>> Hi Jan,
>>
>> On 01/02/2024 13:30, Jan Beulich wrote:
>>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>>> Add a command line parameter to allow the user to set the coloring
>>>> configuration for Dom0.
>>>> A common configuration syntax for cache colors is introduced and
>>>> documented.
>>>> Take the opportunity to also add:
>>>>    - default configuration notion.
>>>>    - function to check well-formed configurations.
>>>>
>>>> Direct mapping Dom0 isn't possible when coloring is enabled, so
>>>> CDF_directmap flag is removed when creating it.
>>>
>>> What implications does this have?
>>>
>>>> --- a/docs/misc/xen-command-line.pandoc
>>>> +++ b/docs/misc/xen-command-line.pandoc
>>>> @@ -963,6 +963,15 @@ Controls for the dom0 IOMMU setup.
>>>>    
>>>>    Specify a list of IO ports to be excluded from dom0 access.
>>>>    
>>>> +### dom0-llc-colors
>>>> +> `= List of [ <integer> | <integer>-<integer> ]`
>>>> +
>>>> +> Default: `All available LLC colors`
>>>> +
>>>> +Specify dom0 LLC color configuration. This options is available only when
>>>> +`CONFIG_LLC_COLORING` is enabled. If the parameter is not set, all available
>>>> +colors are used.
>>>
>>> Even Arm already has a "dom0=" option. Is there a particular reason why
>>> this doesn't become a new sub-option there?
>>>
>>> As to meaning: With just a single <integer>, that's still a color value
>>> then (and not a count of colors)? Wouldn't it make sense to have a
>>> simpler variant available where you just say how many, and a suitable
>>> set/range is then picked?
>>>
>>> Finally a nit: "This option is ...".
>>>
>>>> @@ -2188,10 +2190,16 @@ void __init create_dom0(void)
>>>>                panic("SVE vector length error\n");
>>>>        }
>>>>    
>>>> -    dom0 = domain_create(0, &dom0_cfg, CDF_privileged | CDF_directmap);
>>>> +    if ( !llc_coloring_enabled )
>>>> +        flags |= CDF_directmap;
>>>> +
>>>> +    dom0 = domain_create(0, &dom0_cfg, flags);
>>>>        if ( IS_ERR(dom0) )
>>>>            panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
>>>>    
>>>> +    if ( llc_coloring_enabled && (rc = dom0_set_llc_colors(dom0)) )
>>>> +        panic("Error initializing LLC coloring for domain 0 (rc = %d)", rc);
>>>
>>> As for the earlier patch, I find panic()ing here dubious. You can continue
>>> quite fine, with a warning and perhaps again tainting the system.
>> There are arguments for both approach.
> 
> In which case - perhaps allow for both? With a Kconfig-established
> default and a command line option to override?

Perhaps. But this is a separate discussion from this series. What Carlo 
has been doing match the surrounding code on Arm.

> 
>> I agree that you can continue but
>> technically this is not the configuration you asked. Someone may not
>> notice the tainting until it is too late (read they have done
>> investigation).
>>
>> Bear in mind that the user for cache coloring will be in very
>> specialized environment.
> 
> s/will/may/ I suppose. People may enable the option without being in
> any specialized environment.

Sure. But again, why would you want to boot with a half broken 
configuration?

In a lot of cases, you are not making a favor to the admin to continue 
to boot. It is easy to say there is a warning in the logs, but this can 
often be overlooked and difficult to diagnostic afterwards. For 
instance, if you think about cache coloring the issue would be latency.

I don't think a lambda users will be able to easily figure out that 
their configuration was wrong.

Futhermore, when you operate at scale, I feel it is better to have an 
early boot crash rather than allowing the system to boot (parsing the 
logs is feasible but IMO risky as they are not stable).

> 
>> So if you can't enable cache coloring in
>> production, then something really wrong has happened and continue to
>> boot is probably not right.
>>
>> This matches the approach for Arm we have been using since the
>> beginning. And I will strongly argue to continue this way.
> 
> I'm okay with this, and here (for Arm-specific code) it may even be okay
> to do so without further justification. But in the earlier patch where
> common code is affected, I'll insist on at least justifying this behavior.

See above for a justification. If someone asks for cache coloring, then 
you most likely don't want to continue without cache coloring.

If you dislike the panic() in common code, then we can simply modify the 
function to return an error and move the panic() in the Arm code. This 
is not my preference, but I am under the impression that we both have 
very diverging view how to handle boot error and it will be hard to 
reconcile them (at least in this series, this can be done afterwards if 
somone fancy to write a series matching what you proposed above). So 
this would be the second best option for me.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 13:57:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 13:57:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674453.1049367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXZi-0004ki-JF; Thu, 01 Feb 2024 13:57:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674453.1049367; Thu, 01 Feb 2024 13:57:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXZi-0004kb-Fr; Thu, 01 Feb 2024 13:57:54 +0000
Received: by outflank-mailman (input) for mailman id 674453;
 Thu, 01 Feb 2024 13:57:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ae5+=JK=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1rVXZg-0004kP-Lv
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 13:57:52 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 (mail-am6eur05on20601.outbound.protection.outlook.com
 [2a01:111:f403:2612::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e3791333-c109-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 14:57:51 +0100 (CET)
Received: from AS4P191CA0024.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:5d9::18)
 by DU0PR08MB8114.eurprd08.prod.outlook.com (2603:10a6:10:3ea::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.24; Thu, 1 Feb
 2024 13:57:42 +0000
Received: from AM3PEPF00009B9B.eurprd04.prod.outlook.com
 (2603:10a6:20b:5d9:cafe::7b) by AS4P191CA0024.outlook.office365.com
 (2603:10a6:20b:5d9::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.26 via Frontend
 Transport; Thu, 1 Feb 2024 13:57:42 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM3PEPF00009B9B.mail.protection.outlook.com (10.167.16.20) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7249.19 via Frontend Transport; Thu, 1 Feb 2024 13:57:42 +0000
Received: ("Tessian outbound 67699c3f02bc:v228");
 Thu, 01 Feb 2024 13:57:41 +0000
Received: from 70e782ea9825.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 8C0B5C65-24A4-470F-8E58-CFC52442F710.1; 
 Thu, 01 Feb 2024 13:57:35 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 70e782ea9825.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 01 Feb 2024 13:57:35 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by AM8PR08MB6436.eurprd08.prod.outlook.com (2603:10a6:20b:365::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34; Thu, 1 Feb
 2024 13:57:32 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::2257:5fcb:b449:fff0]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::2257:5fcb:b449:fff0%7]) with mapi id 15.20.7228.029; Thu, 1 Feb 2024
 13:57:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3791333-c109-11ee-8a43-1f161083a0e0
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=RebIadRljAgFs028uav5E26E4uUm1S5wG3Rqm5TwYCX1y3MZjhIrLu/nQFWLkN2A6zmHagK5h17lH2nIIXBFpKVqFHMGy/WyW13xabJWoKg4cg7DCstAMWl9UCFeVKYyCce8SaqPoQ1nAZQZAP3le0Sq+EX4puccLR+aHTaB9v1l50Sec4bWOw7dDhkAmg4ZCBXNacppCdO0JzPm0TqUY6X5KWT4ix1E/u9HBkYU/qbhWY8hZ+6EgrtkysJoyavMFH/5DAhAz86Dylw2T+9Rh4Qg7Mu6WqczVMnl5ZQiqc287Rs7dTioPZkQ8i5qiqrlhg++Q23GSCqz85nS4KQxVQ==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zPdfunecO54m0x8EuWLfnmbInPCoKzwASsxqOyKq7ZI=;
 b=XwE9IqFSuKf7gmRZMUv74pKtVMLmlcsbtL7Dci8Ce755cCINvl0cNu5Wmu39wUdSFBvKCNbNGa9QjwdHQPxh2GawmYZbIMSqr4aFf1DfwZquNzG0+wOgxEoAHnwvtizehKUsAR4FfddaLsxDU24mRaxn/H+zdL7tfVoUep43mnPhotwnbAKkCgc8NjeRVla02/iCN59Q+LVczULVdErNdXM5jDxQEBD3NnKQvyWUjszJ9p6T3CBzhDpS17bnuAAgKf+Z4f5K/KafttFdmhttUI+jKPGF7dkfUFYtAKUciqHpGCoR76bOVgIJw25c/RsyQel74f9cTR7IznOGTGDwVg==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0
 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zPdfunecO54m0x8EuWLfnmbInPCoKzwASsxqOyKq7ZI=;
 b=UHxWHfO2T3NRY34cQPcWSeMshqjkm+/GJ9rzvgXKmJzZj6vIFxtGvnFa6AciiiGos+pSL1N2u/2+H6XAsX2zdRfEtzAr23iO5J65AXgyPgjqb1AKESTh4ap38AL+tHdiZ3Y+XbP4aT/nY0SMXyuWfXFV3H/yPP1P0CZj8g8b2bQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 6c0886f16c02cd39
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=N5i957EnDgbHQNi6S7QRufRbugdrh0UnaMBiiyOsj4TdpPKpPZAsqMSRnQ6f6gMzF/C+8BWcLFaI0HD4upfqJq4GAf9W9/nH5kE3i+7bllXFaIxorKl9eQe9SfeT6Zxsz54LgxoPCSpye/095LIn+rPYgeYD8QtTeUHa2qzi5Uc3g07+nKUMONuj2RavuMR/z1NlMPAoz8twr2Y0iQtTc5bS7+qVeQW6M0wzqZpYmZnNRM2ftJaT5+gpm6gCfUhK4/CdZUEcmePIIGaWQq7o28Wmez9T+r7qaMgJx787+A/zvh//n1ahU2NrEPtzPWxKwMCUdMk47/PPwmy45kRsKw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zPdfunecO54m0x8EuWLfnmbInPCoKzwASsxqOyKq7ZI=;
 b=YsWkx8amtRh6FupGKgyFvPxgzXwhuuuYKSvzV5s0LfARF87+cvDq/ppCZs+5TiHG71CuMe2SlGEAykkV+AOenfkaZIHl0ldllWBEldzMLRz/DYbd/NAMxxggh4oxRw2h2B90Cym2flZHu/XuelIordQ+NWpAr68BM3KXrIvlrvXSaEJU6h8ZJuYcrtVSA3bU/OGa+LEE8no90lIWsHZ6BDtFcboBbZjSYj1/BizR9jeos1enH91Be3OA2brM7SwfpCelmp67tB4seZqB/SPxJZdiP1PrTj9FyPmsRoa3rKBtTucIkdGyMFfiyW8qrOM5syprTm7pxfq7V1jokEM4Vg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zPdfunecO54m0x8EuWLfnmbInPCoKzwASsxqOyKq7ZI=;
 b=UHxWHfO2T3NRY34cQPcWSeMshqjkm+/GJ9rzvgXKmJzZj6vIFxtGvnFa6AciiiGos+pSL1N2u/2+H6XAsX2zdRfEtzAr23iO5J65AXgyPgjqb1AKESTh4ap38AL+tHdiZ3Y+XbP4aT/nY0SMXyuWfXFV3H/yPP1P0CZj8g8b2bQ=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Jens Wiklander <jens.wiklander@linaro.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, "patches@linaro.org"
	<patches@linaro.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Michal
 Orzel <michal.orzel@amd.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [XEN PATCH v3] xen/arm: ffa: reclaim shared memory on guest
 destroy
Thread-Topic: [XEN PATCH v3] xen/arm: ffa: reclaim shared memory on guest
 destroy
Thread-Index: AQHaSTVKNVT0ywRTlE22Kr4Xf5r90LD1mryA
Date: Thu, 1 Feb 2024 13:57:31 +0000
Message-ID: <6980D75D-544A-4A3B-906B-DC27D68A260B@arm.com>
References: <20240117110618.2183977-1-jens.wiklander@linaro.org>
In-Reply-To: <20240117110618.2183977-1-jens.wiklander@linaro.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.200.91.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|AM8PR08MB6436:EE_|AM3PEPF00009B9B:EE_|DU0PR08MB8114:EE_
X-MS-Office365-Filtering-Correlation-Id: f02cef81-20ea-4208-e491-08dc232dc25e
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Tgv5bWFUrsH84yBUGu4f4tGmbiwJmFkjgxQq7njGjtmIlJRJ5i0PGcZilKgqzZ08BcNdQDCF5m3IO04tWjM3042TZKQd6gwpFFyuiDJlO5/hRJV2aEYix2v6qmIbEkwJtbXOJwZOL7S55RxWmeVepNeCv8GiSSz7FDcLFYBSN4mm6Qq2zNTL4JNm6SpgAJCBCtHHVkeVzjevnnPCN1lx8eC4ZSMvCSBvvdhFR0b36883wZBWIZeCmcdYUG2t3I9IjaJKfIe7lfVbvcDhxumlFURguZQhpT8G6Pq9Eid/d+EehAqIAdl30mZCm0eMChlsO2iOqgMgm6InopjAAlgRPLWuiTaxqg8FFA/AdJgE2tISaEJ7Mga1MAdugEuiT1vj33AF/L84Pw6AIjb0dMV81yLhEZ0S6NSII5aqU9NtLqcsW838MS6HPNK1LMArGpCOnDYlpkc5sNiA6qg2J4i3WjFqHrK+tDUwWJStLg7udeY76XWk5fjyJVSS40BKaYhUJXgp8RWPU04FwQxptchCLll+d+E2r47tKBmLgy9l+2csM4ve2LPNbiCtpo5XFi977vCAC9j0gRHBy3XnggZUFvzZQdbu9G3PFxrIr73Gn6BpXUYKjXPWuqQpn4dAifB5MZjs5mnVATPB40aG8cn/bA==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(136003)(396003)(366004)(39860400002)(376002)(230922051799003)(64100799003)(1800799012)(451199024)(186009)(41300700001)(2616005)(83380400001)(36756003)(6506007)(53546011)(38070700009)(478600001)(71200400001)(122000001)(6512007)(38100700002)(6486002)(54906003)(30864003)(2906002)(66556008)(8936002)(76116006)(66476007)(66446008)(6916009)(316002)(64756008)(86362001)(66946007)(5660300002)(4326008)(33656002)(8676002)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <ABB3DC4E3B824841A0F4843DC1BD1EBE@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR08MB6436
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM3PEPF00009B9B.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	acc22d59-8db2-4f11-f3f5-08dc232dbc43
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	5evtEcsVpog9Rq/wRscMIJeqUydbn4MHjr9z5gMRKfxtosFgfckWt07as6Vk4oYHgzgyPrbpilb8KXy0Aqp9LA6LYPre8WIgBo7y/nARKijvQuD+/o88hX2L/yU0bnMFKPbLRuucZ1Hv9OBOIvT6zR4d2RpJhFwv/G9c0RdLC0ZCGLc1YQULvmmqOqsNRxFpfwY7ka21ByVR/FSjnYLqNXfH6dN2VDh4ULbICFa1ixQ65DEhFG0zBEGuGpCrHruzjRF4+7P0bPpv/yFRGtFpeKOILj1vdbF3kiXZMNsM3b2vUZcPzmA11htAfuWYv/uPYl/+3TYgayMOLU4mqUc3GYxxZns6vYUUZzTb/KlZYoXNN4dV+QGvz6tIpng3tdAsMOlmtYTeUr702OXVjO9gpd76mRDwTmC651x0ORLsWL1i55by3qyBtX83RR1tMCH75BwXXFUUU8w7eG54A/jxxYcp1WgQ22htV+FknPhidHjnzT4TEsUSVB6C2wx6boILTNUFmEgljkfY8T0OwS17Jk3Q1i/dfZzxP8wC5f3DoKC34QfBGtGtF8QWFJrJuhJ8ud9GwyJBD1JSLA+N7XpFp9EUi2E9Y5vLqifEVKrDtsbsesHkik7Z5WMDyjq9cxcYZAmfYBdDyMa2n9B0gqIurClO3PWWJ6/Aza+YzGiMREk6Os5IHKed1C/E0J97puYlS3J6ckji2dhKs53WN8uuRyuM4EzTwYTWm1LhTkElnmP9BhtBHPF8q6+jUiZTmDqL
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(39860400002)(396003)(346002)(136003)(230922051799003)(186009)(1800799012)(451199024)(82310400011)(64100799003)(46966006)(40470700004)(36840700001)(83380400001)(41300700001)(47076005)(6512007)(107886003)(6506007)(336012)(2616005)(8676002)(26005)(356005)(36860700001)(4326008)(81166007)(82740400003)(5660300002)(8936002)(478600001)(54906003)(30864003)(53546011)(2906002)(70586007)(6486002)(70206006)(316002)(6862004)(33656002)(36756003)(86362001)(40460700003)(40480700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Feb 2024 13:57:42.0642
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f02cef81-20ea-4208-e491-08dc232dc25e
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM3PEPF00009B9B.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8114

Hi Jens,

> On 17 Jan 2024, at 12:06, Jens Wiklander <jens.wiklander@linaro.org> wrot=
e:
>=20
> When an FF-A enabled guest is destroyed it may leave behind memory
> shared with SPs. This memory must be reclaimed before it's reused or an
> SP may make changes to memory used by a new unrelated guest. So when the
> domain is teared down add FF-A requests to reclaim all remaining shared
> memory.
>=20
> SPs in the secure world are notified using VM_DESTROYED that a guest has
> been destroyed. An SP is supposed to relinquish all shared memory to allo=
w
> reclaiming the memory. The relinquish operation may need to be delayed if
> the shared memory is for instance part of a DMA operation.
>=20
> The domain reference counter is increased when the first FF-A shared
> memory is registered and the counter is decreased again when the last
> shared memory is reclaimed. If FF-A shared memory registrations remain
> at the end of of ffa_domain_teardown() a timer is set to try to reclaim
> the shared memory every second until the memory is reclaimed.
>=20
> A few minor style fixes with a removed empty line here and an added new
> line there.
>=20
> Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
> ---
>=20
> v3:
> - Mentioning in the commit message that there are some style fixes
> - Addressing review comments
> - Refactor the ffa_domain_teardown() path to let
>  ffa_domain_teardown_continue() do most of the work.
>=20
> v2:
> - Update commit message to match the new implementation
> - Using a per domain bitfield to keep track of which SPs has been notifie=
d
>  with VM_DESTROYED
> - Holding a domain reference counter to keep the domain as a zombie domai=
n
>  while there still is shared memory registrations remaining to be reclaim=
ed
> - Using a timer to retry reclaiming remaining shared memory registrations
> ---
> xen/arch/arm/tee/ffa.c | 253 +++++++++++++++++++++++++++++++++--------
> 1 file changed, 204 insertions(+), 49 deletions(-)
>=20
> diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
> index 0793c1c7585d..80ebbf4f01c6 100644
> --- a/xen/arch/arm/tee/ffa.c
> +++ b/xen/arch/arm/tee/ffa.c
> @@ -54,6 +54,7 @@
> #include <xen/mm.h>
> #include <xen/sched.h>
> #include <xen/sizes.h>
> +#include <xen/timer.h>
> #include <xen/types.h>
>=20
> #include <asm/event.h>
> @@ -144,6 +145,12 @@
>  */
> #define FFA_MAX_SHM_COUNT               32
>=20
> +/*
> + * The time we wait until trying to tear down a domain again if it was
> + * blocked initially.
> + */
> +#define FFA_CTX_TEARDOWN_DELAY          SECONDS(1)
> +
> /* FF-A-1.1-REL0 section 10.9.2 Memory region handle, page 167 */
> #define FFA_HANDLE_HYP_FLAG             BIT(63, ULL)
> #define FFA_HANDLE_INVALID              0xffffffffffffffffULL
> @@ -384,11 +391,6 @@ struct ffa_ctx {
>     unsigned int page_count;
>     /* FF-A version used by the guest */
>     uint32_t guest_vers;
> -    /*
> -     * Number of SPs that we have sent a VM created signal to, used in
> -     * ffa_domain_teardown() to know which SPs need to be signalled.
> -     */
> -    uint16_t create_signal_count;
>     bool rx_is_free;
>     /* Used shared memory objects, struct ffa_shm_mem */
>     struct list_head shm_list;
> @@ -402,6 +404,15 @@ struct ffa_ctx {
>     spinlock_t tx_lock;
>     spinlock_t rx_lock;
>     spinlock_t lock;
> +    /* Used if domain can't be torn down immediately */
> +    struct domain *teardown_d;
> +    struct list_head teardown_list;
> +    s_time_t teardown_expire;
> +    /*
> +     * Used for ffa_domain_teardown() to keep track of which SPs should =
be
> +     * notified that this guest is being destroyed.
> +     */
> +    unsigned long vm_destroy_bitmap[];
> };
>=20
> struct ffa_shm_mem {
> @@ -436,6 +447,12 @@ static void *ffa_tx __read_mostly;
> static DEFINE_SPINLOCK(ffa_rx_buffer_lock);
> static DEFINE_SPINLOCK(ffa_tx_buffer_lock);
>=20
> +
> +/* Used to track domains that could not be torn down immediately. */
> +static struct timer ffa_teardown_timer;
> +static struct list_head ffa_teardown_head;
> +static DEFINE_SPINLOCK(ffa_teardown_lock);
> +
> static bool ffa_get_version(uint32_t *vers)
> {
>     const struct arm_smccc_1_2_regs arg =3D {
> @@ -853,7 +870,6 @@ static int32_t handle_partition_info_get(uint32_t w1,=
 uint32_t w2, uint32_t w3,
>             goto out_rx_release;
>         }
>=20
> -
>         memcpy(ctx->rx, ffa_rx, sz);
>     }
>     ctx->rx_is_free =3D false;
> @@ -992,53 +1008,75 @@ static void put_shm_pages(struct ffa_shm_mem *shm)
>     }
> }
>=20
> -static bool inc_ctx_shm_count(struct ffa_ctx *ctx)
> +static bool inc_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
> {
>     bool ret =3D true;
>=20
>     spin_lock(&ctx->lock);
> +
> +    /*
> +     * If this is the first shm added, increase the domain reference
> +     * counter as we need to keep domain around a bit longer to reclaim =
the
> +     * shared memory in the teardown path.
> +     */
> +    if ( !ctx->shm_count )
> +        get_knownalive_domain(d);
> +
>     if (ctx->shm_count >=3D FFA_MAX_SHM_COUNT)
>         ret =3D false;
>     else
>         ctx->shm_count++;
> +
>     spin_unlock(&ctx->lock);
>=20
>     return ret;
> }
>=20
> -static void dec_ctx_shm_count(struct ffa_ctx *ctx)
> +static void dec_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
> {
>     spin_lock(&ctx->lock);
> +
>     ASSERT(ctx->shm_count > 0);
>     ctx->shm_count--;
> +
> +    /*
> +     * If this was the last shm removed, let go of the domain reference =
we
> +     * took in inc_ctx_shm_count() above.
> +     */
> +    if ( !ctx->shm_count )
> +        put_domain(d);
> +
>     spin_unlock(&ctx->lock);
> }
>=20
> -static struct ffa_shm_mem *alloc_ffa_shm_mem(struct ffa_ctx *ctx,
> +static struct ffa_shm_mem *alloc_ffa_shm_mem(struct domain *d,
>                                              unsigned int page_count)
> {
> +    struct ffa_ctx *ctx =3D d->arch.tee;
>     struct ffa_shm_mem *shm;
>=20
>     if ( page_count >=3D FFA_MAX_SHM_PAGE_COUNT )
>         return NULL;
> -    if ( !inc_ctx_shm_count(ctx) )
> +    if ( !inc_ctx_shm_count(d, ctx) )
>         return NULL;
>=20
>     shm =3D xzalloc_flex_struct(struct ffa_shm_mem, pages, page_count);
>     if ( shm )
>         shm->page_count =3D page_count;
>     else
> -        dec_ctx_shm_count(ctx);
> +        dec_ctx_shm_count(d, ctx);
>=20
>     return shm;
> }
>=20
> -static void free_ffa_shm_mem(struct ffa_ctx *ctx, struct ffa_shm_mem *sh=
m)
> +static void free_ffa_shm_mem(struct domain *d, struct ffa_shm_mem *shm)
> {
> +    struct ffa_ctx *ctx =3D d->arch.tee;
> +
>     if ( !shm )
>         return;
>=20
> -    dec_ctx_shm_count(ctx);
> +    dec_ctx_shm_count(d, ctx);
>     put_shm_pages(shm);
>     xfree(shm);
> }
> @@ -1306,7 +1344,7 @@ static void handle_mem_share(struct cpu_user_regs *=
regs)
>         goto out_unlock;
>     }
>=20
> -    shm =3D alloc_ffa_shm_mem(ctx, page_count);
> +    shm =3D alloc_ffa_shm_mem(d, page_count);
>     if ( !shm )
>     {
>         ret =3D FFA_RET_NO_MEMORY;
> @@ -1350,7 +1388,7 @@ static void handle_mem_share(struct cpu_user_regs *=
regs)
>=20
> out:
>     if ( ret )
> -        free_ffa_shm_mem(ctx, shm);
> +        free_ffa_shm_mem(d, shm);
> out_unlock:
>     spin_unlock(&ctx->tx_lock);
>=20
> @@ -1401,7 +1439,7 @@ static int handle_mem_reclaim(uint64_t handle, uint=
32_t flags)
>     }
>     else
>     {
> -        free_ffa_shm_mem(ctx, shm);
> +        free_ffa_shm_mem(d, shm);
>     }
>=20
>     return ret;
> @@ -1486,6 +1524,41 @@ static bool ffa_handle_call(struct cpu_user_regs *=
regs)
>     }
> }
>=20
> +static bool is_in_subscr_list(const uint16_t *subscr, uint16_t start,
> +                              uint16_t end, uint16_t sp_id)
> +{
> +    unsigned int n;
> +
> +    for ( n =3D start; n < end; n++ )
> +    {
> +        if ( subscr[n] =3D=3D sp_id )
> +            return true;
> +    }
> +
> +    return false;
> +}
> +
> +static void vm_destroy_bitmap_init(struct ffa_ctx *ctx,
> +                                   unsigned int create_signal_count)
> +{
> +    unsigned int n;
> +
> +    for ( n =3D 0; n < subscr_vm_destroyed_count; n++ )
> +    {
> +        /*
> +         * Skip SPs subscribed to the VM created event that never was
> +         * notified of the VM creation due to an error during
> +         * ffa_domain_init().
> +         */
> +        if ( is_in_subscr_list(subscr_vm_created, create_signal_count,
> +                               subscr_vm_created_count,
> +                               subscr_vm_destroyed[n]) )
> +            continue;
> +
> +        set_bit(n, ctx->vm_destroy_bitmap);
> +    }
> +}
> +
> static int ffa_domain_init(struct domain *d)
> {
>     struct ffa_ctx *ctx;
> @@ -1501,11 +1574,14 @@ static int ffa_domain_init(struct domain *d)
>     if ( d->domain_id >=3D UINT16_MAX)
>         return -ERANGE;
>=20
> -    ctx =3D xzalloc(struct ffa_ctx);
> +    ctx =3D xzalloc_flex_struct(struct ffa_ctx, vm_destroy_bitmap,
> +                              BITS_TO_LONGS(subscr_vm_destroyed_count));
>     if ( !ctx )
>         return -ENOMEM;
>=20
>     d->arch.tee =3D ctx;
> +    ctx->teardown_d =3D d;
> +    INIT_LIST_HEAD(&ctx->shm_list);
>=20
>     for ( n =3D 0; n < subscr_vm_created_count; n++ )
>     {
> @@ -1515,65 +1591,141 @@ static int ffa_domain_init(struct domain *d)
>         {
>             printk(XENLOG_ERR "ffa: Failed to report creation of vm_id %u=
 to  %u: res %d\n",
>                    get_vm_id(d), subscr_vm_created[n], res);
> -            ctx->create_signal_count =3D n;
> -            return -EIO;
> +            break;
>         }
>     }
> -    ctx->create_signal_count =3D subscr_vm_created_count;
> -
> -    INIT_LIST_HEAD(&ctx->shm_list);
> +    vm_destroy_bitmap_init(ctx, n);
> +    if ( n !=3D subscr_vm_created_count )
> +        return -EIO;
>=20
>     return 0;
> }
>=20
> -static bool is_in_subscr_list(const uint16_t *subscr, uint16_t start,
> -                              uint16_t end, uint16_t sp_id)
> +static void send_vm_destroyed(struct domain *d)
> {
> +    struct ffa_ctx *ctx =3D d->arch.tee;
>     unsigned int n;
> +    int32_t res;
>=20
> -    for ( n =3D start; n < end; n++ )
> +    for ( n =3D 0; n < subscr_vm_destroyed_count; n++ )
>     {
> -        if ( subscr[n] =3D=3D sp_id )
> -            return true;
> -    }
> +        if ( !test_bit(n, ctx->vm_destroy_bitmap) )
> +            continue;
>=20
> -    return false;
> +        res =3D ffa_direct_req_send_vm(subscr_vm_destroyed[n], get_vm_id=
(d),
> +                                     FFA_MSG_SEND_VM_DESTROYED);
> +

I think here we should dinstinguish cases depending on the error received:
- retry: we should keep the bit set and retry later
- other: unset the bit as there is not much we can do but log it.

The interrupted case can happen, we handle that already in the direct messa=
ge
request but we will get back here if that does not work after some retries =
so
putting it in the other category is ok i think.

> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "%pd: ffa: Failed to report destruction of=
 vm_id %u to %u: res %d\n",
> +                   d, get_vm_id(d), subscr_vm_destroyed[n], res);
> +        }
> +        else
> +        {
> +            clear_bit(n, ctx->vm_destroy_bitmap);
> +        }
> +    }
> }
>=20
> -/* This function is supposed to undo what ffa_domain_init() has done */
> -static int ffa_domain_teardown(struct domain *d)
> +static void reclaim_shms(struct domain *d)
> {
>     struct ffa_ctx *ctx =3D d->arch.tee;
> -    unsigned int n;
> +    struct ffa_shm_mem *shm, *tmp;
>     int32_t res;
>=20
> -    if ( !ctx )
> -        return 0;
> +    list_for_each_entry_safe(shm, tmp, &ctx->shm_list, list)
> +    {
> +        register_t handle_hi;
> +        register_t handle_lo;
>=20
> -    for ( n =3D 0; n < subscr_vm_destroyed_count; n++ )
> +        uint64_to_regpair(&handle_hi, &handle_lo, shm->handle);
> +        res =3D ffa_mem_reclaim(handle_lo, handle_hi, 0);

Same here as for the VM_DESTROYED, there are some error code that we should
not reiterate on as they will also fail next time:
- invalid parameters: stop and log an error, this will always fail
- no memory: for now I would say do the same as invalid parameter with a co=
mment
 maybe because we could retry and memory coud be available next time
- abort: stop and log an error, this is a major fault on the other side
- denied: retry.

> +        if ( res )
> +        {
> +            printk(XENLOG_G_INFO "%pd: ffa: Failed to reclaim handle %#l=
x : %d\n",
> +                   d, shm->handle, res);
> +        }
> +        else
> +        {
> +            printk(XENLOG_G_DEBUG "%pd: ffa: Reclaimed handle %#lx\n",
> +                   d, shm->handle);
> +            list_del(&shm->list);
> +            free_ffa_shm_mem(d, shm);
> +        }
> +    }
> +}
> +
> +static void ffa_domain_teardown_continue(struct ffa_ctx *ctx, bool first=
_time)
> +{
> +    struct ffa_ctx *next_ctx =3D NULL;
> +
> +    send_vm_destroyed(ctx->teardown_d);
> +    reclaim_shms(ctx->teardown_d);
> +
> +    if ( ctx->shm_count )
> +    {

If i do understand right you only retry if there are remaining SHM to recla=
im but what
if you have destroy messages that were not sent successfully, shouldn't we =
reiterate
in that case to send the destroy message again ?

> +        printk(XENLOG_G_INFO "%pd: ffa: Remaining unclaimed handles, ret=
rying\n", ctx->teardown_d);
> +
> +        ctx->teardown_expire =3D NOW() + FFA_CTX_TEARDOWN_DELAY;
> +
> +        spin_lock(&ffa_teardown_lock);
> +        list_add_tail(&ctx->teardown_list, &ffa_teardown_head);
> +        /* Need to set a new timer for the next ctx in line */
> +        next_ctx =3D list_first_entry(&ffa_teardown_head, struct ffa_ctx=
,
> +                                    teardown_list);
> +        spin_unlock(&ffa_teardown_lock);
> +    }
> +    else
>     {
>         /*
> -         * Skip SPs subscribed to the VM created event that never was
> -         * notified of the VM creation due to an error during
> -         * ffa_domain_init().
> +         * domain_destroy() might have been called (via put_domain() in
> +         * reclaim_shms()), so we can't touch the domain structure anymo=
re.
>          */
> -        if ( is_in_subscr_list(subscr_vm_created, ctx->create_signal_cou=
nt,
> -                               subscr_vm_created_count,
> -                               subscr_vm_destroyed[n]) )
> -            continue;
> +        xfree(ctx);
>=20
> -        res =3D ffa_direct_req_send_vm(subscr_vm_destroyed[n], get_vm_id=
(d),
> -                                     FFA_MSG_SEND_VM_DESTROYED);
> -
> -        if ( res )
> -            printk(XENLOG_ERR "ffa: Failed to report destruction of vm_i=
d %u to  %u: res %d\n",
> -                   get_vm_id(d), subscr_vm_destroyed[n], res);
> +        /* Only check if there has been a change to the teardown queue *=
/
> +        if ( !first_time )
> +        {
> +            spin_lock(&ffa_teardown_lock);
> +            next_ctx =3D list_first_entry_or_null(&ffa_teardown_head,
> +                                                struct ffa_ctx, teardown=
_list);
> +            spin_unlock(&ffa_teardown_lock);
> +        }
>     }
>=20
> +    if ( next_ctx )
> +        set_timer(&ffa_teardown_timer, next_ctx->teardown_expire);
> +}
> +
> +static void ffa_teardown_timer_callback(void *arg)
> +{
> +    struct ffa_ctx *ctx;
> +
> +    spin_lock(&ffa_teardown_lock);
> +    ctx =3D list_first_entry_or_null(&ffa_teardown_head, struct ffa_ctx,
> +                                   teardown_list);
> +    if ( ctx )
> +        list_del(&ctx->teardown_list);
> +    spin_unlock(&ffa_teardown_lock);
> +
> +    if ( ctx )
> +        ffa_domain_teardown_continue(ctx, false /* !first_time */);
> +    else
> +        printk(XENLOG_G_ERR "%s: teardown list is empty\n", __func__);
> +}
> +
> +/* This function is supposed to undo what ffa_domain_init() has done */
> +static int ffa_domain_teardown(struct domain *d)
> +{
> +    struct ffa_ctx *ctx =3D d->arch.tee;
> +
> +    if ( !ctx )
> +        return 0;
> +
>     if ( ctx->rx )
>         rxtx_unmap(ctx);
>=20
> -    XFREE(d->arch.tee);
> +    ffa_domain_teardown_continue(ctx, true /* first_time */);
>=20
>     return 0;
> }
> @@ -1739,6 +1891,9 @@ static bool ffa_probe(void)
>     if ( !init_sps() )
>         goto err_free_ffa_tx;
>=20
> +    INIT_LIST_HEAD(&ffa_teardown_head);
> +    init_timer(&ffa_teardown_timer, ffa_teardown_timer_callback, NULL, 0=
);
> +
>     return true;
>=20
> err_free_ffa_tx:
> --=20
> 2.34.1
>=20

Regards
Bertrand




From xen-devel-bounces@lists.xenproject.org Thu Feb 01 14:15:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 14:15:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674460.1049377 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXqC-0001Kk-VS; Thu, 01 Feb 2024 14:14:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674460.1049377; Thu, 01 Feb 2024 14:14:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXqC-0001Kd-Rv; Thu, 01 Feb 2024 14:14:56 +0000
Received: by outflank-mailman (input) for mailman id 674460;
 Thu, 01 Feb 2024 14:14:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVXqA-0001KX-Ta
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 14:14:54 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4443fa92-c10c-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 15:14:52 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a358ec50b7cso177054666b.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 06:14:52 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n11-20020a1709061d0b00b00a31805b4165sm7307095ejh.11.2024.02.01.06.14.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 06:14:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4443fa92-c10c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706796892; x=1707401692; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=372ElkJtaiL6Ckt2N2GRS8UgDvCeS0ipS47J5ex3oPo=;
        b=U5Zzk9cy3F6p1AYSXBhqL5oXqA2X9BDsNV9ogxQes40DqjhOE5PrDq2aDuJ2iF2OMP
         gJff50FwrbgJaJKVboXV5g354hTuGSJPY9+t0juLJc0zqWsGzW+SFAkiCDdkcx9dAPcE
         XirwuZcr+O8DV2hdGcauORW/cg3kL1BkTD1TBYi9efLy35160NVDQKcIYm9gKRhSxdmv
         XVBEWA4cTWRnnznzTv3OpNEl4vkoX/a5n/wcZRkD49VUN1txHkp0kBB0ozAldVYXkPYt
         xAgvtNSIfn/hQoYox7N13DlOOpuZkl5yhMQV6sFWB1fPsNPuUIhqbPnxQzK9XvR9lrib
         ua8A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706796892; x=1707401692;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=372ElkJtaiL6Ckt2N2GRS8UgDvCeS0ipS47J5ex3oPo=;
        b=PF95EWIS2qB8TYydqiWMyoCdqsPGShrTQZInlBP6NcsiVn59SVFL2si9G4GTAfGB1l
         vDtPkZeQr9T19mutbE+kc7lAXDbRjLlnKfMdpu46/5VnOZde77Eey1a6LjjC7F+XOfWf
         xh/BZEolyxeetvZHZc5IRifQntZZUmj5pw8eNZWOtHYD3WHwxVfnA/+wbA+DkzeUDs4B
         vWlLtDIYe70oo6ZR5pBVD0MgxEZXgApqBQdz+RCQpPefF0/ozxMJGklfNfRStcDqjwtz
         1CJ/hUe2g4Ce2khnJy84ONpZ+t14syCAyMefsYS3iG1ibuhwO3GPsu2228HfRxqraLUK
         tKjw==
X-Gm-Message-State: AOJu0YxllbpLXY5BLWCGcEm7vZclV0vLiE26fh4aSMKvhgOaGcJcx1RQ
	9TqFFU0RZTziDpsPrmE5FVUvng7b/5EbFdbowmpiO0zXuZySMITBw9gPPCNgyQ==
X-Google-Smtp-Source: AGHT+IGMs6lhETPsxHHCYuu4paYmqeShsaRHhh6gxYOmsr1GP9cGuFZ6nDDesUE42Ccxf37+yHDpjw==
X-Received: by 2002:a17:906:f884:b0:a29:b31d:1dc6 with SMTP id lg4-20020a170906f88400b00a29b31d1dc6mr4608416ejb.6.1706796892038;
        Thu, 01 Feb 2024 06:14:52 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWrLCoi+13j4zvLjpgdICpGbP9M3IKMweasrcL5I5I6UVnXqoaByYfddo0BHjwYQc4u24nwrcIGpvOj/Sh6sMjpPELEuJOSoBkJ5RCqi50LGoxbAIu6SNLmfUBbzpUqh52JIRdWQ5FYC4LzCg==
Message-ID: <47b5a1be-280c-4e8e-a5c5-6df7da657539@suse.com>
Date: Thu, 1 Feb 2024 15:14:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/2] x86/p2m: aid the compiler in folding p2m_is_...()
Content-Language: en-US
To: George Dunlap <dunlapg@umich.edu>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com>
 <7cce89f4-962e-bfbe-7d30-18fea7515bed@suse.com>
 <CAFLBxZZLJMWpf1fCNN4dhoDpYpW6O=V_C==-sCAZy6t4Df_yBw@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAFLBxZZLJMWpf1fCNN4dhoDpYpW6O=V_C==-sCAZy6t4Df_yBw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01.02.2024 14:32, George Dunlap wrote:
> On Thu, Jun 23, 2022 at 12:54 PM Jan Beulich <jbeulich@suse.com> wrote:
> 
>> By using | instead of || or (in the negated form) && chances increase
>> for the compiler to recognize that both predicates can actually be
>> folded into an expression requiring just a single branch (via OR-ing
>> together the respective P2M_*_TYPES constants).
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>
> 
> Sorry for the delay.  Git complains that this patch is malformed:
> 
> error: `git apply --index`: error: corrupt patch at line 28
> 
> Similar complaint from patchew when it was posted:
> 
> https://patchew.org/Xen/5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com/

Not sure what to say. The patch surely is well-formed. It applies fine
using patch (when not taken from email). When taken from email, patch
mentions that it strips CRs (I'm running my email client on Windows),
but the saved email still applies fine. "git am" indeed is unhappy
when taking the plain file as saved from email, albeit here with an
error different from yours. If I edit the saved email to retain just
the From: and Subject: tags, all is fine.

I can't tell what git doesn't like. The error messages (the one you
see and the one I got) tell me nothing. I'm also not aware of there
being a requirement that patches I send via email need to be
"git am"-able (unlike in xsa.git, where I edit patches enough to be
suitable for that), nor am I aware how I would convince my email
client and/or server to omit whatever git doesn't like or to add
whatever git is missing.

Bottom line - your response would be actionable by me only in so far
as I could switch to using "git send-email". Which I'm afraid I'm not
going to do unless left with no other choice. The way I've been
sending patches has worked well for over 20 years, and for different
projects. (I'm aware Andrew has some special "Jan" command to apply
patches I send, but I don't know any specifics.)

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 14:19:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 14:19:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674464.1049386 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXuy-0003dh-KG; Thu, 01 Feb 2024 14:19:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674464.1049386; Thu, 01 Feb 2024 14:19:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXuy-0003da-HU; Thu, 01 Feb 2024 14:19:52 +0000
Received: by outflank-mailman (input) for mailman id 674464;
 Thu, 01 Feb 2024 14:19:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVXux-0003c4-Od
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 14:19:51 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f5e4368e-c10c-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 15:19:50 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-55a9008c185so1738283a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 06:19:50 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ev20-20020a056402541400b0055a8fe70387sm6856536edb.1.2024.02.01.06.19.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 06:19:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5e4368e-c10c-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706797190; x=1707401990; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xFLBrh7Dpw3f87TJClFlFdPkIP+m+X3DpeBb+FAMqqw=;
        b=PNI7jFTnDkJrprISyHNm5/JCws0Ort9DCoRhyRIgD4jcth7NqGqSqc0CpcxqtX4jYg
         L85KMwpFGpWywHXjAqxOiiTATHd4GJ1E7GwhDyUaJcHGdjYnKgPagKy91DuGUmGEoTks
         sVvrFa7SZPRC/9iQz9fyRaupjxTfbPs0rW6SqpH6TaJQE4QAjfsewHK7IDmQDvGDa2h7
         a2Mi41mWn7+GNf4DCoHBYw3FIBsVb8E4MzhVH1MTK1b9DNwO8TF52QYNaEU+Rq8tP3UI
         eR9r2jN2tRque+hRXHOrJ3XqzZpa9DFDW0MVqA/aMed89nLHRp0IGW3NRrmzRkBjrRYI
         krQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706797190; x=1707401990;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xFLBrh7Dpw3f87TJClFlFdPkIP+m+X3DpeBb+FAMqqw=;
        b=r2HHmU0m9dce4+4KJcA7gQTs6wg3HH/bDh+ITU6AgV3zS+wevAlUX6YkE91Tdu7yh3
         lFWdLbaD/MUYzAePbDi0JJrcXUzwBjGdbgM50IaKU0omLNHE5uNjtoQRt1o/8oeBCom2
         U91F2wGztB/FLDaXWegdinjWu9V4sVAl5pPA9qZ0GZzKJd2x93SU5KQGckdCemguBXxf
         5rwB9CZJApeH1E2TJ0GW/R5/NL7/7g25b6jqud/7d5/ymiAEm43GNzT5k4EiVcbZ+LJW
         c7oTiBLbQv0/Up/xS6pWR36q6RrvmhBA4hbTqHrhMqG+maUj/hCAQKjxW2wvxyDpiYIE
         G6HQ==
X-Gm-Message-State: AOJu0Yz7mpWVjtOu/3FRqcqgCiLZhvBAxcV1zKw3sgq/h9uoSxckT2XV
	pW+RNb/XQCUuc7b3p5dAmH18WayQI5zMCsUQZarWXXMwSmA6FIVjGIYN/jRQNg==
X-Google-Smtp-Source: AGHT+IEjsmIYEgx0oOGNg8gK4dNS2Sc9/aKI1hvXhaj3q0G54l/mXV66zL4dQRKZUW2UtAfPQtp3+Q==
X-Received: by 2002:a50:8e43:0:b0:55f:5197:6314 with SMTP id 3-20020a508e43000000b0055f51976314mr4697303edx.18.1706797190022;
        Thu, 01 Feb 2024 06:19:50 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUBqgQZk9DK8Vom/o14kamV2ZuSDDHgUC0AQp3yBxYwUOGEmXiAYPGzLM3CQ20RXxd9iDYUmoddPqZUGol9geb9pxXvFY5GN4xMTU7yBbmI4kJoTzFb7L1oj1b7BAjpfNDnm+o2X2jtV4xc59RYoPUfyucokPaHc431aTtnsGeIJENOHMBiUSchSrHNsVDMGhPBbbB1uVf1IKIsuh9TGdVQdjz9bB1VYhLOSu95PHqDb4plwZFY5GdjeTJj+kGay3KWy8LrrBJyBooCFZQY+KG4WbsXXEVuWqgVak2BDDPmjt4X33fll2lgJEvRsV2QDWB+pezyb1vXk6rablH5Xe9foxJ/XAGwPFrIfbZrc8L71JakFnlV5S94jhb10dzSbpKuMQ==
Message-ID: <a6ffb4a5-cd83-4c04-9e5a-12d605863b5d@suse.com>
Date: Thu, 1 Feb 2024 15:19:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 07/15] xen/arm: add support for cache coloring
 configuration via device-tree
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-8-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-8-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:18, Carlo Nonato wrote:
> @@ -950,6 +951,11 @@ void __init create_domUs(void)
>  #endif
>          }
>  
> +        dt_property_read_string(node, "llc-colors", &llc_colors_str);
> +        if ( !llc_coloring_enabled && llc_colors_str)
> +            printk(XENLOG_WARNING
> +                   "'llc-colors' found, but LLC coloring is disabled\n");

Why's this just a warning, when ...

> @@ -960,6 +966,11 @@ void __init create_domUs(void)
>              panic("Error creating domain %s (rc = %ld)\n",
>                    dt_node_name(node), PTR_ERR(d));
>  
> +        if ( llc_coloring_enabled &&
> +             (rc = domain_set_llc_colors_from_str(d, llc_colors_str)) )
> +            panic("Error initializing LLC coloring for domain %s (rc = %d)\n",
> +                  dt_node_name(node), rc);

... this results in panic()?

> --- a/xen/common/llc-coloring.c
> +++ b/xen/common/llc-coloring.c
> @@ -254,6 +254,29 @@ int domain_set_llc_colors_domctl(struct domain *d,
>      return domain_check_colors(d);
>  }
>  
> +int domain_set_llc_colors_from_str(struct domain *d, const char *str)

__init ?

> +{
> +    int err;
> +    unsigned int *colors;
> +
> +    if ( !str )
> +        return domain_set_default_colors(d);
> +
> +    colors = alloc_colors(max_nr_colors);
> +    if ( !colors )
> +        return -ENOMEM;
> +
> +    err = parse_color_config(str, colors, max_nr_colors, &d->num_llc_colors);
> +    if ( err )
> +    {
> +        printk(XENLOG_ERR "Error parsing LLC color configuration.");

Nit: No full stop at the end of log messages please.

> +        return err;
> +    }
> +    d->llc_colors = colors;
> +
> +    return domain_check_colors(d);

Same ordering issue as in the earlier patch, I think.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 14:21:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 14:21:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674467.1049397 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXwS-0005T8-Vw; Thu, 01 Feb 2024 14:21:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674467.1049397; Thu, 01 Feb 2024 14:21:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXwS-0005T1-Sv; Thu, 01 Feb 2024 14:21:24 +0000
Received: by outflank-mailman (input) for mailman id 674467;
 Thu, 01 Feb 2024 14:21:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVXwR-0005Sq-M2
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 14:21:23 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2cda519a-c10d-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 15:21:22 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-55efbaca48bso1327702a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 06:21:22 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ev20-20020a056402541400b0055a8fe70387sm6856536edb.1.2024.02.01.06.21.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 06:21:22 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2cda519a-c10d-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706797282; x=1707402082; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9bIqj/x+kRtQHMH7oFV6HTCiWYHXei4IpsdLUZpA3ac=;
        b=O9Lm3MGY1790qmS18+KgOFjBY7j6xnKcmtyor6rVRuDcagJVZix9kPyiuiPkceob0k
         U3tiFSzJ65x8iVS8/dSrBIbNb2HH9DpJ1y61cPSpIrDe6RKOVGRVoxx9YT16BwQc3/SI
         PyVfBpBi19wzY47lelteiuFCSSb1i3hJSzH7IyPkE8sLMfh7cHQi/6R6oiWa/iF0fpwA
         4yOuR3brwEcnABSj7nIwgDjUHg1urpf/eJ8e/3PDfaV5S9xYV+3ldIfrTFJtlJ2oaJxx
         AfXEJi2YBzLdefVeJenOieHuZpF9zHefFoRbSl6v39+NAwA2+wQyGOYvwo9o8me0voMV
         4lJw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706797282; x=1707402082;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9bIqj/x+kRtQHMH7oFV6HTCiWYHXei4IpsdLUZpA3ac=;
        b=HUnHYKdImpq4pe5lCjlqAvXkwOlMLV+bI+y7vCAOnR9hLNUbJtuaaiZKHb3p349hZj
         J1YXqsXukNdqehUzBYMeNU5XNtENvnJGOkvHwA3w+d+ewNJVQ4OZTMiZhnbTJhOaTm2k
         MwvuXyS/Qm3xObX1FAaBbezCRG8YSz3olmVnc9+3rZULbf9JqkIGWA7a/56U2JJp1aUT
         rgPDR8JRRtjIxW4fsbMfdUxhyyYMRbMjifP5HNUffl6w0p64vGu0N52i2sDuOIlxb/Fd
         XbiSFhiAh67por84vPFL00+nBiabHcuSEni9nZVUoj8+rWts5Vz1ZP24WTTERqaUUcFM
         T7ng==
X-Gm-Message-State: AOJu0YyE2GjWmLBi6xddNbwn4C5SMnMJFd7JC62vTIzpZMarA1H43Y4N
	LttyUqKP0bvrbOGnhwUBJ5U7bWLbi5ywRVObnLhc+pJqDTB4eAE1n2wOBcG86g==
X-Google-Smtp-Source: AGHT+IHCDcO+re9/6Et4j8ebOHRgtecY3ijparwiGLDuHCxKvvYmoiX3hLEoeuPhAczJQE0sHExA6Q==
X-Received: by 2002:aa7:dad4:0:b0:55f:9337:7a22 with SMTP id x20-20020aa7dad4000000b0055f93377a22mr3417898eds.12.1706797282349;
        Thu, 01 Feb 2024 06:21:22 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWFab+X+uzE+y11Wu0iDF0hKJmFcX1Tb7+ot0Zuwaty3uexfiBTXOG7YXtL/+WA9U2WN0WYx29MS838tRnolMZ2MUizM5coZc6/QlMK2Ra4et5kigclXvGcKtNezq6lMGZCcn5ppjw+q7vZ5lMEdqqKbjQ8ansIm0crGbiRRRRJZl0C1C+dgJT19ysQuEVtKKNhRtlplQ4v5VTXexPg6PFfBNi012w9hVeWRnzOuYlBZTtB0zeW5Qk=
Message-ID: <a881fe27-0e3e-49ad-8340-71fabf92ea7e@suse.com>
Date: Thu, 1 Feb 2024 15:21:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 08/15] xen/page_alloc: introduce
 init_free_page_fields() helper
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-9-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-9-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:18, Carlo Nonato wrote:
> Introduce a new helper to initialize fields that have different uses for
> free pages.
> 
> Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Thu Feb 01 14:24:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 14:24:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674472.1049407 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXza-0006Qb-DT; Thu, 01 Feb 2024 14:24:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674472.1049407; Thu, 01 Feb 2024 14:24:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVXza-0006QU-AU; Thu, 01 Feb 2024 14:24:38 +0000
Received: by outflank-mailman (input) for mailman id 674472;
 Thu, 01 Feb 2024 14:24:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVXzY-0006Q7-P9
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 14:24:36 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9f4902d4-c10d-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 15:24:34 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-55faa1e3822so1314746a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 06:24:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e8-20020a056402104800b0055ef2c893b3sm4540772edu.52.2024.02.01.06.24.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 06:24:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9f4902d4-c10d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706797474; x=1707402274; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=aTnwzMZUdVn2T6ESwgfPVQgOhtpjh2quV7YR2J9QNh4=;
        b=RLUJPpe5tMzV4VXZDuhVk10fKf155R+rsgpjq6njP0/STq+xzdyn6LopfS8QmilkOf
         S88nMzLpu+j9jkCihCLpngqIhzbWrXMItsf79olejnNPgfcsYDyWDovthH+bmTCRgmoZ
         l7v3NfsB8r9j9vaavVZuyCrhe5hHmdNZiUHvp0OeeUmwjZCBZT0PhjNBrGPBqeth9J8a
         OWbbaEvrMVHQs5JpbOUJrwY2AHNWB1zmVrDTt76DtFIo1bL1KTWlBVsXQylJ573LsuNK
         ONDt1K5Xl6FxwcIJOButWSYxrQgFUA24PPME8S88XJydu0nz94bIqa357InlhDOLlGQ5
         kFHg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706797474; x=1707402274;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=aTnwzMZUdVn2T6ESwgfPVQgOhtpjh2quV7YR2J9QNh4=;
        b=Pm4O7ug8mTJ0uCksGiN/0d/VrmVFzZ/5+DnAjFD2XBE6DI2ZFyd3lLBfDc5AYFxcy2
         ceXI/F0SMeZw0FCq6gwS06/4gxwAzWSsteGzvBBhUTyo8OB6yas6Wy8Jo8wiBDFFdIbG
         Cl03L53eTtKnsr0ItSP/qFtvDnra0JON2y+9LJwdl1lVfkCjUwUqS2HqyknxnlkAj3Gn
         n3bIfl6j9/xxL0FpZy89PV3A9Z/lIVoNHWdjXC3zceSaLpcElkypqVw5C0TOMIIR839g
         k5HDn+bNdutVJZwkdmQimwG1afIQNhP94M3qM5WnBNGsNLMT9j0cHWgQy06m9DIcYQcP
         J6TA==
X-Gm-Message-State: AOJu0YxqUdypb9IOSSgptELrFuSbH75C731w/wZR2jhs4n85r5ElvBfP
	JciwRMVGByIsmu5sSwFV1gy0VPxj1UxF2PCUW0oo28K6ukwYfZ/5yA87TmKMyA==
X-Google-Smtp-Source: AGHT+IFaO3VX6mxNKUT73n4vv/ju0W/5o8JkQK0/kF58SmLXbknCcHVqpnJfHAgpO6g40I5uoL8Abg==
X-Received: by 2002:a05:6402:3126:b0:55f:3fc1:a8b7 with SMTP id dd6-20020a056402312600b0055f3fc1a8b7mr3410268edb.7.1706797474292;
        Thu, 01 Feb 2024 06:24:34 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUqhF/GCPXrKrWIgvYcPZnp4ltseZjPvx8cPAWbjXgVphbTv5YNZNCF9mm0AlT8WBm4TpyB05Hpo1Bf037tCnwm1zTZlaRr77Ch0lxXGPqhs5ziffxId1XNW1evc33K5kN5hC5U0uu5w4bj5z7oceqJmjr5J2Cqn0iOnCoWGNNYW4MokbCS9afZgh+DWuX6uLpHb5vHsN3yYCFWooUCLsh5aSKeXlTRjYsJdsqQ+zY6VMAtN3ElV1c=
Message-ID: <8a3f3c38-b290-417a-86cb-216e36182abe@suse.com>
Date: Thu, 1 Feb 2024 15:24:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 09/15] xen/page_alloc: introduce preserved page flags
 macro
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-10-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-10-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:18, Carlo Nonato wrote:
> PGC_static and PGC_extra are flags that needs to be preserved when assigning
> a page. Define a new macro that groups those flags and use it instead of
> or'ing every time.

While here you say where the "preserving" applies, ...

> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -157,6 +157,8 @@
>  #define PGC_static 0
>  #endif
>  
> +#define PGC_preserved (PGC_extra | PGC_static)

... nothing is said here. From the earlier version I also seem to recall
that the constant was then used outside of assign_pages(). That would
then mean amending whatever comment would be added here.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 14:49:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 14:49:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674477.1049416 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYNQ-00055t-9O; Thu, 01 Feb 2024 14:49:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674477.1049416; Thu, 01 Feb 2024 14:49:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYNQ-00055m-6X; Thu, 01 Feb 2024 14:49:16 +0000
Received: by outflank-mailman (input) for mailman id 674477;
 Thu, 01 Feb 2024 14:49:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qXLx=JK=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rVYNO-00052A-N6
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 14:49:14 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0fdbc749-c111-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 15:49:12 +0100 (CET)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-554fe147ddeso1329235a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 06:49:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0fdbc749-c111-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1706798951; x=1707403751; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=dFrb14sFYxvQL3XHYJxoeThDXXBr7naHSSTej1GaxwM=;
        b=HkshQvKloTCdtnLdkPscnYQAHZr695KnweqO+dQX6FCvifY46YjvCkMeR4FBAGGIDk
         pzRRbAy8FJxua6TLs63JVXGE6HoywShHJKeSBXXshNUY3XVSAhNFs1PO0RYB7bt7kW3K
         eZXMy3YbtV27wFhYdkJoIMyUT095lxt9JPReoe2SLNgxGmPHw1a9m+pRXS6SgsKiCfmx
         Iz7YsyX06Ny90kHawBFxv0ZYClaq2hXBshEXMJi1J7fX4CCQawany2AUf23bd+j+/tfA
         BUzNMllNC9QkOdtIb1EFB/8F5zyOvDH0LxaKlMBWhG1Bfj8njRfuZLuvmlKtAQg/yncn
         NSqA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706798951; x=1707403751;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=dFrb14sFYxvQL3XHYJxoeThDXXBr7naHSSTej1GaxwM=;
        b=it69Y+mahH5/EjHLl2j1naCSXPhZhf51WmXt9VuMlLWTmKZXJdGxA/Bs30F0Pei9HJ
         J6EOQdHiwSjHFBWr/uJhb9Tb21TJxAx+ynH7UxHlKer07gM68MdpNGvoXmtnIIsew5KJ
         a9Un9q4BmB5r/JZNZYuquQykJbyssz9lg8ac48IPrH1q7I4ekLSOj5V7cuJK7l7aaB3u
         QIvnNxFvEupCH0nY76WmxhZo80oldjZ/NeYbsxfjglzOUcmWerXgJIWzomwRlu5O9O58
         RZkhCKHuxB2B6CnxiWahlqtafW/zauoHqDREmbA4WkOJ3XhaAIAhTbGLWDwg0uERdlZV
         53NQ==
X-Gm-Message-State: AOJu0YwsVrP8KC/anWDYyFoRHnaMpkDM76xQ7DwGXhrBMzAns9nSdhUY
	XgRO0lMVPqhulY0czaJXIe7PPSGQvBDrOXBjxKjiMHOwYG8kfeuDH7/wV28YgVvaXtaEdLXBhm0
	mCtU+a/q7673GxMzxvfrftN9jexBbB1/Z1B5S2Q==
X-Google-Smtp-Source: AGHT+IGZeF8OndDYcaG2J0sgatGE/PKwuAz0nBmgVPylZnF2Kzym4hDGoLbgQtOieEEWLKmER+mEc/I1uizfcYgv5JA=
X-Received: by 2002:a05:6402:b4d:b0:55e:ef54:1239 with SMTP id
 bx13-20020a0564020b4d00b0055eef541239mr3840572edb.10.1706798951541; Thu, 01
 Feb 2024 06:49:11 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-10-carlo.nonato@minervasys.tech> <8a3f3c38-b290-417a-86cb-216e36182abe@suse.com>
In-Reply-To: <8a3f3c38-b290-417a-86cb-216e36182abe@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Thu, 1 Feb 2024 15:49:00 +0100
Message-ID: <CAG+AhRU5Yhz_VF02CfaFD5ifvw8cUQbsM7-UiMcRWm+1LR4btg@mail.gmail.com>
Subject: Re: [PATCH v6 09/15] xen/page_alloc: introduce preserved page flags macro
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Thu, Feb 1, 2024 at 3:24=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 29.01.2024 18:18, Carlo Nonato wrote:
> > PGC_static and PGC_extra are flags that needs to be preserved when assi=
gning
> > a page. Define a new macro that groups those flags and use it instead o=
f
> > or'ing every time.
>
> While here you say where the "preserving" applies, ...
>
> > --- a/xen/common/page_alloc.c
> > +++ b/xen/common/page_alloc.c
> > @@ -157,6 +157,8 @@
> >  #define PGC_static 0
> >  #endif
> >
> > +#define PGC_preserved (PGC_extra | PGC_static)
>
> ... nothing is said here. From the earlier version I also seem to recall
> that the constant was then used outside of assign_pages(). That would
> then mean amending whatever comment would be added here.

Yes, but it was used in places where the name didn't fit (to stop merging i=
n
free_heap_pages()) and so I thought it would've been better to use the
constant only for one of the two concepts: only for preserved flags in
assign_pages().

Are you suggesting adding a comment to this #define to clarify its usage?

Thanks.

> Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 15:00:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 15:00:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674481.1049426 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYXn-0000GQ-6u; Thu, 01 Feb 2024 14:59:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674481.1049426; Thu, 01 Feb 2024 14:59:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYXn-0000GJ-4M; Thu, 01 Feb 2024 14:59:59 +0000
Received: by outflank-mailman (input) for mailman id 674481;
 Thu, 01 Feb 2024 14:59:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fxIb=JK=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1rVYXm-0000Ez-FD
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 14:59:58 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 907b3e74-c112-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 15:59:57 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-40eac352733so9228695e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 06:59:57 -0800 (PST)
Received: from fziglio-xenia-fedora.eng.citrite.net ([185.25.67.249])
 by smtp.gmail.com with ESMTPSA id
 t12-20020a05600c450c00b0040d5ae2906esm4737894wmo.30.2024.02.01.06.59.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 06:59:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 907b3e74-c112-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1706799597; x=1707404397; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=wUDi3+dPODjB9qZn1Kktd29eMC32bAroiLdFB6ZmDys=;
        b=K3/SWa1s4kczFXdltkie+9Xp2m8lRB8OPL9lEVfs1vpo+ZOZv2+wFEGSVq0Tdsyoqk
         unAOWIhHzgpH9DLEM1jpkTE3TCN3TEyvnNIwYqchGzPw3OZQrCYOp792H8soevgDgA+m
         23bL7VS37+Otn7I03yuniVsQXmP35zg0yh21s=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706799597; x=1707404397;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=wUDi3+dPODjB9qZn1Kktd29eMC32bAroiLdFB6ZmDys=;
        b=GZUJA2t+HM3JSmHZYG0LabJYyH2OmfMXubI3+ybGGh1IqfEundjlq4pXEOuG2qQuh+
         YQTD5sF26Sm3dWgEyPYZX6MVoqrTirzsyB274UxP4uDUI5j30BdpGPZNxrKO+j5ge85U
         uvXtJ7CKc9mCy0+ICXe69Dp/54yOVhmRB3UXxZrr6hN3AgN6OKemogHwHUTwEC53SoVQ
         Q7rUlBGNueXvRAi/CiU+Lw2fdaC0R/OXn8jW+hW30Hjg3k1hEj0wJvaNBGcn1/xk/Zmo
         Ar01vId1/dVnNmqIiLUCthOfhAItOTgiGQkJnuddcgJTVibPFYFyX/ifIQRsw3KetakJ
         Wv3g==
X-Gm-Message-State: AOJu0YwUJoMfea7Y/Ha6xVeO81rtbpgLulsJQY2dDPGGg6UVud4olB57
	pLaESkloJjXvvUQKSFuonOhK8Z2cbAZRGWtxZQVm9WzrBRibR9bZwuDPNOVX3D7wp9inIITrkma
	I
X-Google-Smtp-Source: AGHT+IGKXDSVancIaYJuUcE2eBx3guV9apdjao5cLNe+I/ZxtXIKceERhfuR5AI+XVeftejihKvdCg==
X-Received: by 2002:a05:600c:314f:b0:40e:4806:f436 with SMTP id h15-20020a05600c314f00b0040e4806f436mr3815617wmo.3.1706799596826;
        Thu, 01 Feb 2024 06:59:56 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUxxooJNPYJ9HJDdLtqIfzjdUuPvjPLr5HQUUTc2xl5zqJiCxdFIyWrnfY+BxsDrFMtGoSnUGK1LSuOL/tCV3h+VGrq+Tz8r+qulh9KkT9uQIMG+/JmRQvr3H9vcSSRU6emzqDG
From: Frediano Ziglio <frediano.ziglio@cloud.com>
To: George Dunlap <george.dunlap@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org,
	Frediano Ziglio <frediano.ziglio@cloud.com>
Subject: [PATCH] Fix some typos in comments
Date: Thu,  1 Feb 2024 14:59:43 +0000
Message-ID: <20240201145943.738531-1-frediano.ziglio@cloud.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
---
 arch/x86/include/arch/processor.h | 2 +-
 include/xen/hvm/params.h          | 2 +-
 include/xtf/console.h             | 2 +-
 include/xtf/extable.h             | 4 ++--
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/arch/processor.h b/arch/x86/include/arch/processor.h
index 0c33545..1c7e655 100644
--- a/arch/x86/include/arch/processor.h
+++ b/arch/x86/include/arch/processor.h
@@ -101,7 +101,7 @@
 #define X86_EXC_TS            10 /* Invalid TSS. */
 #define X86_EXC_NP            11 /* Segment Not Present. */
 #define X86_EXC_SS            12 /* Stack-Segment Fault. */
-#define X86_EXC_GP            13 /* General Porection Fault. */
+#define X86_EXC_GP            13 /* General Protection Fault. */
 #define X86_EXC_PF            14 /* Page Fault. */
 #define X86_EXC_SPV           15 /* PIC Spurious Interrupt Vector. */
 #define X86_EXC_MF            16 /* Maths fault (x87 FPU). */
diff --git a/include/xen/hvm/params.h b/include/xen/hvm/params.h
index 886b986..0b7c05b 100644
--- a/include/xen/hvm/params.h
+++ b/include/xen/hvm/params.h
@@ -1,5 +1,5 @@
 /*
- * Xen public hvm paramter index
+ * Xen public hvm parameter index
  */
 
 #ifndef XEN_PUBLIC_HVM_PARAMS_H
diff --git a/include/xtf/console.h b/include/xtf/console.h
index caec790..16a6a23 100644
--- a/include/xtf/console.h
+++ b/include/xtf/console.h
@@ -11,7 +11,7 @@
 typedef void (*cons_output_cb)(const char *buf, size_t len);
 
 /*
- * Register a console callback.  Several callbacks can be registered for usful
+ * Register a console callback.  Several callbacks can be registered for useful
  * destinations of console text.
  */
 void register_console_callback(cons_output_cb cb);
diff --git a/include/xtf/extable.h b/include/xtf/extable.h
index e93331e..0668b7d 100644
--- a/include/xtf/extable.h
+++ b/include/xtf/extable.h
@@ -4,8 +4,8 @@
  * Exception table support.
  *
  * Allows code to tag an instruction which might fault, and where to jump to
- * in order to recover.  For more complicated recovery, a cusom handler
- * handler can be registerd.
+ * in order to recover.  For more complicated recovery, a custom handler
+ * can be registered.
  */
 #ifndef XTF_EXTABLE_H
 #define XTF_EXTABLE_H
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 15:07:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 15:07:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674488.1049437 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYfH-0001s1-1d; Thu, 01 Feb 2024 15:07:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674488.1049437; Thu, 01 Feb 2024 15:07:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYfG-0001ru-V3; Thu, 01 Feb 2024 15:07:42 +0000
Received: by outflank-mailman (input) for mailman id 674488;
 Thu, 01 Feb 2024 15:07:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Ox/P=JK=proton.me=oxjo@srs-se1.protection.inumbo.net>)
 id 1rVYfF-0001qg-7l
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 15:07:42 +0000
Received: from mail-4318.protonmail.ch (mail-4318.protonmail.ch [185.70.43.18])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a3c32236-c113-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 16:07:39 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3c32236-c113-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me;
	s=protonmail; t=1706800057; x=1707059257;
	bh=6BO9FSjLv6fOZG/S8ewf50JyY4Qtsp7WXzHDVSF0kA0=;
	h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References:
	 Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:
	 Message-ID:BIMI-Selector;
	b=e3fOUv5JPF8xfqxmHkZpu6xyWD79Nvhbxi1diXgQOa6zY5ES89gOBIVNbyKpXeE7H
	 AhVcwb3arI06btTyj7Pk9N/TKr7Iavd2E/aR2bACjKYK9pB+TCzE1Yxm+I9FWscf3L
	 CKUWVBaY6E4WWiyisdm1ITZphfK6uMif0RBxB2KoR5lT3OJiHefg7lL9kX6JnzHi8Y
	 eJH4VB8OsXw8k5HXhrjx+j2eo2T1l3kMrtkOlKuv19oA57fodEFD4DQG8Atpe997Wu
	 YpxctvGyb0iHc480Gtt1MolvRBobMo5M9p0dNf5b1nyxPCfwsTIrXZSbvcpASgnYkn
	 gswpN+aBaRkoQ==
Date: Thu, 01 Feb 2024 15:07:29 +0000
To: Jan Beulich <jbeulich@suse.com>
From: oxjo@proton.me
Cc: "roger.pau@citrix.com" <roger.pau@citrix.com>, "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: Issue iommu unrecognized on amd computer
Message-ID: <-sBcc1pvw26g_MiNhiisTW9NrekKWz2P3_8cIduaPlwkkm_v_GdSvMhlQtNBQU-v77vAP1_9EXWfpcuUdbk64mDIj-FBpyedXPvmkmgXAkc=@proton.me>
In-Reply-To: <5cda514b-4b6f-4259-9dbc-bb1462eec13e@suse.com>
References: <qTUd8Z_TU2_jJzz9kTj2DJK54p4wECUT072VD_1nomh-d8Ej6hC-QNzY6BW6G9bQVdcDJuQbapJ7NlqqOAKd49Iu8rgkyt4E9qdnN_SyZj8=@proton.me> <5cda514b-4b6f-4259-9dbc-bb1462eec13e@suse.com>
Feedback-ID: 99814252:user:proton
MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="b1_GBYj6Y1YZQCBAVgd6bUn1cquVDPuaNM7JB5U3PQFQw"

This is a multi-part message in MIME format.

--b1_GBYj6Y1YZQCBAVgd6bUn1cquVDPuaNM7JB5U3PQFQw
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

I put it on the linux command line indeed, here is the log with iommu=3Ddeb=
ug.

I can install Xen with a debug build or test a fix if needed.
I just need some guides because all I did for know is apt or dnf install Xe=
n.


Le jeudi 1 f=C3=A9vrier 2024 =C3=A0 11:57, Jan Beulich <jbeulich@suse.com> =
a =C3=A9crit=C2=A0:

> On 01.02.2024 09:16, oxjo@proton.me wrote:
>=20
> > Following our interaction on matrix, I send you the boot log `xl dmesg`=
 output of my computer, booting with `iommu=3Ddebug` options on linux 6.6.1=
3-200.fc39.
>=20
>=20
> There's no "iommu=3Ddebug" anywhere in sight:
>=20
> (XEN) Command line: placeholder no-real-mode edd=3Doff
>=20
> Did you mistakenly put it on the Linux command line?
>=20
> Nevertheless there are relevant messages already without the extra
> verbosity (and without you using a debug build of Xen, which would
> be preferable):
>=20
> (XEN) AMD-Vi: Warning: IVMD: [c9f1c000,c9f42000) is not (entirely) in res=
erved memory
> (XEN) AMD-Vi: Error: IVMD: page at c9f1c000 can't be converted
> (XEN) AMD-Vi: Error initialization
>=20
> Quoting the respective entry from the memory map:
>=20
> (XEN) [00000000c8f58000, 00000000c9f57fff] (ACPI NVS)
>=20
> Both Roger and I look to agree that this condition we have
>=20
> if ( !(type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
> RAM_TYPE_UNUSABLE)) )
>=20
> is inverted in some way. We merely need to agree in which way it
> wants adjusting.
>=20
> Jan
--b1_GBYj6Y1YZQCBAVgd6bUn1cquVDPuaNM7JB5U3PQFQw
Content-Type: text/plain; name="boot (1).log"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="boot (1).log"

IFhlbiA0LjE4LjAKKFhFTikgWGVuIHZlcnNpb24gNC4xOC4wIChtb2NrYnVpbGRAKSAoZ2NjIChH
Q0MpIDEzLjIuMSAyMDIzMTIwNSAoUmVkIEhhdCAxMy4yLjEtNikpIGRlYnVnPW4gV2VkIERlYyAx
MyAxNzo1MDo0MiBVVEMgMjAyMwooWEVOKSBMYXRlc3QgQ2hhbmdlU2V0OiAKKFhFTikgYnVpbGQt
aWQ6IGVhNWQxOTYwYThkMDZmMzFiMDAwMzU1ZjQ2NjIxNjlmZTEyZDA5YTYKKFhFTikgQm9vdGxv
YWRlcjogR1JVQiAyLjA2CihYRU4pIENvbW1hbmQgbGluZTogcGxhY2Vob2xkZXIgbm8tcmVhbC1t
b2RlIGVkZD1vZmYgaW9tbXU9ZGVidWcKKFhFTikgWGVuIGltYWdlIGxvYWQgYmFzZSBhZGRyZXNz
OiAweGMxYTAwMDAwCihYRU4pIFZpZGVvIGluZm9ybWF0aW9uOgooWEVOKSAgVkdBIGlzIGdyYXBo
aWNzIG1vZGUgMTkyMHgxMDgwLCAzMiBicHAKKFhFTikgIFZCRS9EREMgbWV0aG9kczogbm9uZTsg
RURJRCB0cmFuc2ZlciB0aW1lOiAwIHNlY29uZHMKKFhFTikgRGlzYyBpbmZvcm1hdGlvbjoKKFhF
TikgIEZvdW5kIDAgTUJSIHNpZ25hdHVyZXMKKFhFTikgIEZvdW5kIDEgRUREIGluZm9ybWF0aW9u
IHN0cnVjdHVyZXMKKFhFTikgQ1BVIFZlbmRvcjogQU1ELCBGYW1pbHkgMjUgKDB4MTkpLCBNb2Rl
bCA4MCAoMHg1MCksIFN0ZXBwaW5nIDAgKHJhdyAwMGE1MGYwMCkKKFhFTikgRW5hYmxpbmcgU3Vw
ZXJ2aXNvciBTaGFkb3cgU3RhY2tzCihYRU4pICAgLSBEaXNhYmxpbmcgUFYzMiBkdWUgdG8gQ0VU
CihYRU4pIEVGSSBSQU0gbWFwOgooWEVOKSAgWzAwMDAwMDAwMDAwMDAwMDAsIDAwMDAwMDAwMDAw
OWVmZmZdICh1c2FibGUpCihYRU4pICBbMDAwMDAwMDAwMDA5ZjAwMCwgMDAwMDAwMDAwMDA5ZmZm
Zl0gKHJlc2VydmVkKQooWEVOKSAgWzAwMDAwMDAwMDAxMDAwMDAsIDAwMDAwMDAwMDliZmZmZmZd
ICh1c2FibGUpCihYRU4pICBbMDAwMDAwMDAwOWMwMDAwMCwgMDAwMDAwMDAwOWRiMGZmZl0gKHJl
c2VydmVkKQooWEVOKSAgWzAwMDAwMDAwMDlkYjEwMDAsIDAwMDAwMDAwMDllZmZmZmZdICh1c2Fi
bGUpCihYRU4pICBbMDAwMDAwMDAwOWYwMDAwMCwgMDAwMDAwMDAwOWYwZWZmZl0gKEFDUEkgTlZT
KQooWEVOKSAgWzAwMDAwMDAwMDlmMGYwMDAsIDAwMDAwMDAwYzJkNTdmZmZdICh1c2FibGUpCihY
RU4pICBbMDAwMDAwMDBjMmQ1ODAwMCwgMDAwMDAwMDBjOGY1N2ZmZl0gKHJlc2VydmVkKQooWEVO
KSAgWzAwMDAwMDAwYzhmNTgwMDAsIDAwMDAwMDAwYzlmNTdmZmZdIChBQ1BJIE5WUykKKFhFTikg
IFswMDAwMDAwMGM5ZjU4MDAwLCAwMDAwMDAwMGM5ZmQ3ZmZmXSAoQUNQSSBkYXRhKQooWEVOKSAg
WzAwMDAwMDAwYzlmZDgwMDAsIDAwMDAwMDAwYzlmZDlmZmZdICh1c2FibGUpCihYRU4pICBbMDAw
MDAwMDBjOWZkYTAwMCwgMDAwMDAwMDBjOWZmZmZmZl0gKHJlc2VydmVkKQooWEVOKSAgWzAwMDAw
MDAwY2EwMDAwMDAsIDAwMDAwMDAwY2JmZmZmZmZdICh1c2FibGUpCihYRU4pICBbMDAwMDAwMDBj
YzAwMDAwMCwgMDAwMDAwMDBjZGZmZmZmZl0gKHJlc2VydmVkKQooWEVOKSAgWzAwMDAwMDAwY2Yw
MDAwMDAsIDAwMDAwMDAwY2ZmZmZmZmZdIChyZXNlcnZlZCkKKFhFTikgIFswMDAwMDAwMGY4MDAw
MDAwLCAwMDAwMDAwMGZiZmZmZmZmXSAocmVzZXJ2ZWQpCihYRU4pICBbMDAwMDAwMDBmZGMwMDAw
MCwgMDAwMDAwMDBmZGNmZmZmZl0gKHJlc2VydmVkKQooWEVOKSAgWzAwMDAwMDAwZmVkODAwMDAs
IDAwMDAwMDAwZmVkODBmZmZdIChyZXNlcnZlZCkKKFhFTikgIFswMDAwMDAwMTAwMDAwMDAwLCAw
MDAwMDAwZmVlMmZmZmZmXSAodXNhYmxlKQooWEVOKSAgWzAwMDAwMDBmZWUzMDAwMDAsIDAwMDAw
MDEwMmZmZmZmZmZdIChyZXNlcnZlZCkKKFhFTikgQUNQSTogUlNEUCBDOUZENzAxNCwgMDAyNCAo
cjIgTEVOT1ZPKQooWEVOKSBBQ1BJOiBYU0RUIEM5RkQ1MTg4LCAwMTA0IChyMSBMRU5PVk8gVFAt
UjI1ICAgICAgIDExMzAgUFRFQyAgICAgICAgMikKKFhFTikgQUNQSTogRkFDUCBDMzk2NDAwMCwg
MDExNCAocjYgTEVOT1ZPIFRQLVIyNSAgICAgICAxMTMwIFBURUMgICAgICAgIDIpCihYRU4pIEFD
UEk6IERTRFQgQzM5NEMwMDAsIDEyNzkxIChyMSBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgSU5U
TCAyMDE4MDMxMykKKFhFTikgQUNQSTogRkFDUyBDOURGODAwMCwgMDA0MAooWEVOKSBBQ1BJOiBT
U0RUIEM1RTc0MDAwLCAwMEEyIChyMSBMRU5PVk8gUElEMFNzZHQgICAgICAgIDEgSU5UTCAyMDE4
MDMxMykKKFhFTikgQUNQSTogU1NEVCBDNUU3MzAwMCwgMDc1MiAocjEgTEVOT1ZPIFVzYkNUYWJs
ICAgICAgICAxIElOVEwgMjAxODAzMTMpCihYRU4pIEFDUEk6IFNTRFQgQzVFNjYwMDAsIDczNDUg
KHIyIExFTk9WTyBUUC1SMjUgICAgICAgICAgMiBNU0ZUICAyMDAwMDAyKQooWEVOKSBBQ1BJOiBN
U0RNIEM1QjdCMDAwLCAwMDU1IChyMyBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgUFRFQyAgICAg
ICAgMikKKFhFTikgQUNQSTogQkFUQiBDNUI2NjAwMCwgMDA0QSAocjIgTEVOT1ZPIFRQLVIyNSAg
ICAgICAxMTMwIFBURUMgICAgICAgIDIpCihYRU4pIEFDUEk6IEhQRVQgQzM5NjMwMDAsIDAwMzgg
KHIxIExFTk9WTyBUUC1SMjUgICAgICAgMTEzMCBQVEVDICAgICAgICAyKQooWEVOKSBBQ1BJOiBB
UElDIEMzOTYyMDAwLCAwMTM4IChyMiBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgUFRFQyAgICAg
ICAgMikKKFhFTikgQUNQSTogTUNGRyBDMzk2MTAwMCwgMDAzQyAocjEgTEVOT1ZPIFRQLVIyNSAg
ICAgICAxMTMwIFBURUMgICAgICAgIDIpCihYRU4pIEFDUEk6IFNCU1QgQzM5NjAwMDAsIDAwMzAg
KHIxIExFTk9WTyBUUC1SMjUgICAgICAgMTEzMCBQVEVDICAgICAgICAyKQooWEVOKSBBQ1BJOiBX
U01UIEMzOTVGMDAwLCAwMDI4IChyMSBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgUFRFQyAgICAg
ICAgMikKKFhFTikgQUNQSTogVkZDVCBDMzkzRTAwMCwgRDg4NCAocjEgTEVOT1ZPIFRQLVIyNSAg
ICAgICAxMTMwIFBURUMgICAgICAgIDIpCihYRU4pIEFDUEk6IFNTRFQgQzM5MzgwMDAsIDUzNTQg
KHIyIExFTk9WTyBUUC1SMjUgICAgICAgICAgMSBBTUQgICAgICAgICAxKQooWEVOKSBBQ1BJOiBD
UkFUIEMzOTM3MDAwLCAwRUMwIChyMSBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgUFRFQyAgICAg
ICAgMikKKFhFTikgQUNQSTogQ0RJVCBDMzkzNjAwMCwgMDAyOSAocjEgTEVOT1ZPIFRQLVIyNSAg
ICAgICAxMTMwIFBURUMgICAgICAgIDIpCihYRU4pIEFDUEk6IEZQRFQgQzVCNjcwMDAsIDAwMzQg
KHIxIExFTk9WTyBUUC1SMjUgICAgICAgMTEzMCBQVEVDICAgICAgICAyKQooWEVOKSBBQ1BJOiBT
U0RUIEMzOTM1MDAwLCAwMTQ5IChyMSBMRU5PVk8gVFAtUjI1ICAgICAgICAgIDEgSU5UTCAyMDE4
MDMxMykKKFhFTikgQUNQSTogU1NEVCBDMzkzMzAwMCwgMTRDMyAocjEgTEVOT1ZPIFRQLVIyNSAg
ICAgICAgICAxIElOVEwgMjAxODAzMTMpCihYRU4pIEFDUEk6IFNTRFQgQzM5MzEwMDAsIDE1QTgg
KHIxIExFTk9WTyBUUC1SMjUgICAgICAgICAgMSBJTlRMIDIwMTgwMzEzKQooWEVOKSBBQ1BJOiBT
U0RUIEMzOTJEMDAwLCAzQjBFIChyMSBMRU5PVk8gVFAtUjI1ICAgICAgICAgIDEgSU5UTCAyMDE4
MDMxMykKKFhFTikgQUNQSTogQkdSVCBDMzkyQzAwMCwgMDAzOCAocjEgTEVOT1ZPIFRQLVIyNSAg
ICAgICAxMTMwIFBURUMgICAgICAgIDIpCihYRU4pIEFDUEk6IFNTRFQgQzM5MkIwMDAsIDAyNEQg
KHIxIExFTk9WTyBUUC1SMjUgICAgICAgICAgMSBJTlRMIDIwMTgwMzEzKQooWEVOKSBBQ1BJOiBT
U0RUIEMzOTI5MDAwLCAxNEM0IChyMSBMRU5PVk8gVFAtUjI1ICAgICAgICAgIDEgSU5UTCAyMDE4
MDMxMykKKFhFTikgQUNQSTogU1NEVCBDMzkyODAwMCwgMEFCNyAocjEgTEVOT1ZPIFRQLVIyNSAg
ICAgICAgICAxIElOVEwgMjAxODAzMTMpCihYRU4pIEFDUEk6IFVFRkkgQzlERjcwMDAsIDAwQzYg
KHIxIExFTk9WTyBUUC1SMjUgICAgICAgMTEzMCBQVEVDICAgICAgICAyKQooWEVOKSBBQ1BJOiBJ
VlJTIEMzOTI3MDAwLCAwMUU0IChyMiBMRU5PVk8gVFAtUjI1ICAgICAgIDExMzAgUFRFQyAgICAg
ICAgMikKKFhFTikgQUNQSTogU1NEVCBDNUU3MjAwMCwgMDA5MCAocjEgTEVOT1ZPIFRQLVIyNSAg
ICAgICAgICAxIElOVEwgMjAxODAzMTMpCihYRU4pIEFDUEk6IFNTRFQgQzVFNzEwMDAsIDA5OEQg
KHIxIExFTk9WTyBUUC1SMjUgICAgICAgICAgMSBJTlRMIDIwMTgwMzEzKQooWEVOKSBTeXN0ZW0g
UkFNOiA2NDMwMk1CICg2NTg0NTQ3NmtCKQooWEVOKSBObyBOVU1BIGNvbmZpZ3VyYXRpb24gZm91
bmQKKFhFTikgRmFraW5nIGEgbm9kZSBhdCAwMDAwMDAwMDAwMDAwMDAwLTAwMDAwMDBmZWUzMDAw
MDAKKFhFTikgRG9tYWluIGhlYXAgaW5pdGlhbGlzZWQKKFhFTikgdmVzYWZiOiBmcmFtZWJ1ZmZl
ciBhdCAweDAwMDAwMDAwZDAwMDAwMDAsIG1hcHBlZCB0byAweGZmZmY4MmMwMDAyMDEwMDAsIHVz
aW5nIDgxMjhrLCB0b3RhbCA4MTI4awooWEVOKSB2ZXNhZmI6IG1vZGUgaXMgMTkyMHgxMDgweDMy
LCBsaW5lbGVuZ3RoPTc2ODAsIGZvbnQgOHgxNgooWEVOKSB2ZXNhZmI6IFRydWVjb2xvcjogc2l6
ZT04Ojg6ODo4LCBzaGlmdD0yNDoxNjo4OjAKKFhFTikgU01CSU9TIDMuMyBwcmVzZW50LgooWEVO
KSBBTUQtVmk6IElWUlMgQmxvY2s6IEZvdW5kIHR5cGUgMHgxMCBmbGFncyAweGIwIGxlbiAweDQ4
IGlkIDB4MgooWEVOKSBBTUQtVmk6IElWUlMgQmxvY2s6IEZvdW5kIHR5cGUgMHgxMSBmbGFncyAw
eGIwIGxlbiAweDU4IGlkIDB4MgooWEVOKSBBTUQtVmk6IFVzaW5nIElWSEQgdHlwZSAweDExCihY
RU4pIEFNRC1WaTogRm91bmQgTVNJIGNhcGFiaWxpdHkgYmxvY2sgYXQgMHg2NAooWEVOKSBVc2lu
ZyBBUElDIGRyaXZlciBkZWZhdWx0CihYRU4pIEFDUEk6IFBNLVRpbWVyIElPIFBvcnQ6IDB4NDA4
ICgzMiBiaXRzKQooWEVOKSBBQ1BJOiB2NSBTTEVFUCBJTkZPOiBjb250cm9sWzE6MF0sIHN0YXR1
c1sxOjBdCihYRU4pIEFDUEk6IFNMRUVQIElORk86IHBtMXhfY250WzE6NDA0LDE6MF0sIHBtMXhf
ZXZ0WzE6NDAwLDE6MF0KKFhFTikgQUNQSTogMzIvNjRYIEZBQ1MgYWRkcmVzcyBtaXNtYXRjaCBp
biBGQURUIC0gYzlkZjgwMDAvMDAwMDAwMDAwMDAwMDAwMCwgdXNpbmcgMzIKKFhFTikgQUNQSTog
ICAgICAgICAgICAgd2FrZXVwX3ZlY1tjOWRmODAwY10sIHZlY19zaXplWzIwXQooWEVOKSBPdmVy
cmlkaW5nIEFQSUMgZHJpdmVyIHdpdGggYmlnc21wCihYRU4pIEFDUEk6IElPQVBJQyAoaWRbMHgy
MF0gYWRkcmVzc1sweGZlYzAwMDAwXSBnc2lfYmFzZVswXSkKKFhFTikgSU9BUElDWzBdOiBhcGlj
X2lkIDMyLCB2ZXJzaW9uIDMzLCBhZGRyZXNzIDB4ZmVjMDAwMDAsIEdTSSAwLTIzCihYRU4pIEFD
UEk6IElPQVBJQyAoaWRbMHgyMV0gYWRkcmVzc1sweGZlYzAxMDAwXSBnc2lfYmFzZVsyNF0pCihY
RU4pIElPQVBJQ1sxXTogYXBpY19pZCAzMywgdmVyc2lvbiAzMywgYWRkcmVzcyAweGZlYzAxMDAw
LCBHU0kgMjQtNTUKKFhFTikgQUNQSTogSU5UX1NSQ19PVlIgKGJ1cyAwIGJ1c19pcnEgMCBnbG9i
YWxfaXJxIDIgZGZsIGRmbCkKKFhFTikgQUNQSTogSU5UX1NSQ19PVlIgKGJ1cyAwIGJ1c19pcnEg
OSBnbG9iYWxfaXJxIDkgbG93IGxldmVsKQooWEVOKSBBQ1BJOiBIUEVUIGlkOiAweDQzNTM4MjEw
IGJhc2U6IDB4ZmVkMDAwMDAKKFhFTikgUENJOiBNQ0ZHIGNvbmZpZ3VyYXRpb24gMDogYmFzZSBm
ODAwMDAwMCBzZWdtZW50IDAwMDAgYnVzZXMgMDAgLSAzZgooWEVOKSBQQ0k6IE1DRkcgYXJlYSBh
dCBmODAwMDAwMCByZXNlcnZlZCBpbiBFODIwCihYRU4pIFBDSTogVXNpbmcgTUNGRyBmb3Igc2Vn
bWVudCAwMDAwIGJ1cyAwMC0zZgooWEVOKSBBQ1BJOiBCR1JUOiBpbnZhbGlkYXRpbmcgdjEgaW1h
Z2UgYXQgMHhiZDY1NjAxOAooWEVOKSBVc2luZyBBQ1BJIChNQURUKSBmb3IgU01QIGNvbmZpZ3Vy
YXRpb24gaW5mb3JtYXRpb24KKFhFTikgU01QOiBBbGxvd2luZyAxNiBDUFVzICgwIGhvdHBsdWcg
Q1BVcykKKFhFTikgSVJRIGxpbWl0czogNTYgR1NJLCAzMjcyIE1TSS9NU0ktWAooWEVOKSBDUFUw
OiAxNjAwIC4uLiAyMDAwIE1IegooWEVOKSB4c3RhdGU6IHNpemU6IDB4OTg4IGFuZCBzdGF0ZXM6
IDB4MjA3CihYRU4pIENQVTA6IEFNRCBGYW0xOWggbWFjaGluZSBjaGVjayByZXBvcnRpbmcgZW5h
YmxlZAooWEVOKSBWdWxuZXJhYmxlIHRvIFNSU08sIHdpdGhvdXQgc3VpdGFibGUgbWljcm9jb2Rl
IHRvIG1pdGlnYXRlCihYRU4pIFNwZWN1bGF0aXZlIG1pdGlnYXRpb24gZmFjaWxpdGllczoKKFhF
TikgICBIYXJkd2FyZSBoaW50czogU1RJQlBfQUxXQVlTIElCUlNfRkFTVCBJQlJTX1NBTUVfTU9E
RQooWEVOKSAgIEhhcmR3YXJlIGZlYXR1cmVzOiBJQlBCIElCUlMgU1RJQlAgU1NCRCBQU0ZECihY
RU4pICAgQ29tcGlsZWQtaW4gc3VwcG9ydDogSU5ESVJFQ1RfVEhVTksgU0hBRE9XX1BBR0lORwoo
WEVOKSAgIFhlbiBzZXR0aW5nczogQlRJLVRodW5rIEpNUCwgU1BFQ19DVFJMOiBJQlJTKyBTVElC
UCsgU1NCRC0gUFNGRC0sIE90aGVyOiBJQlBCLWN0eHQgQlJBTkNIX0hBUkRFTgooWEVOKSAgIFN1
cHBvcnQgZm9yIEhWTSBWTXM6IE1TUl9TUEVDX0NUUkwgTVNSX1ZJUlRfU1BFQ19DVFJMIFJTQgoo
WEVOKSAgIFN1cHBvcnQgZm9yIFBWIFZNczogTm9uZQooWEVOKSAgIFhQVEkgKDY0LWJpdCBQViBv
bmx5KTogRG9tMCBkaXNhYmxlZCwgRG9tVSBkaXNhYmxlZCAod2l0aG91dCBQQ0lEKQooWEVOKSAg
IFBWIEwxVEYgc2hhZG93aW5nOiBEb20wIGRpc2FibGVkLCBEb21VIGRpc2FibGVkCihYRU4pIFVz
aW5nIHNjaGVkdWxlcjogU01QIENyZWRpdCBTY2hlZHVsZXIgcmV2MiAoY3JlZGl0MikKKFhFTikg
SW5pdGlhbGl6aW5nIENyZWRpdDIgc2NoZWR1bGVyCihYRU4pICBsb2FkX3ByZWNpc2lvbl9zaGlm
dDogMTgKKFhFTikgIGxvYWRfd2luZG93X3NoaWZ0OiAzMAooWEVOKSAgdW5kZXJsb2FkX2JhbGFu
Y2VfdG9sZXJhbmNlOiAwCihYRU4pICBvdmVybG9hZF9iYWxhbmNlX3RvbGVyYW5jZTogLTMKKFhF
TikgIHJ1bnF1ZXVlcyBhcnJhbmdlbWVudDogc29ja2V0CihYRU4pICBjYXAgZW5mb3JjZW1lbnQg
Z3JhbnVsYXJpdHk6IDEwbXMKKFhFTikgbG9hZCB0cmFja2luZyB3aW5kb3cgbGVuZ3RoIDEwNzM3
NDE4MjQgbnMKKFhFTikgUGxhdGZvcm0gdGltZXIgaXMgMTQuMzE4TUh6IEhQRVQKKFhFTikgRGV0
ZWN0ZWQgMTk5Ni4yNTggTUh6IHByb2Nlc3Nvci4KKFhFTikgRnJlZWQgMTAyMGtCIHVudXNlZCBC
U1MgbWVtb3J5CihYRU4pIGFsdCB0YWJsZSBmZmZmODJkMDQwNDViZDk4IC0+IGZmZmY4MmQwNDA0
NjkyYTgKKFhFTikgQU1ELVZpOiBJT01NVSBFeHRlbmRlZCBGZWF0dXJlczoKKFhFTikgLSBQZXJp
cGhlcmFsIFBhZ2UgU2VydmljZSBSZXF1ZXN0CihYRU4pIC0geDJBUElDCihYRU4pIC0gTlggYml0
CihYRU4pIC0gSW52YWxpZGF0ZSBBbGwgQ29tbWFuZAooWEVOKSAtIEd1ZXN0IEFQSUMKKFhFTikg
LSBQZXJmb3JtYW5jZSBDb3VudGVycwooWEVOKSAtIEhvc3QgQWRkcmVzcyBUcmFuc2xhdGlvbiBT
aXplOiAweDIKKFhFTikgLSBHdWVzdCBBZGRyZXNzIFRyYW5zbGF0aW9uIFNpemU6IDAKKFhFTikg
LSBHdWVzdCBDUjMgUm9vdCBUYWJsZSBMZXZlbDogMHgxCihYRU4pIC0gTWF4aW11bSBQQVNJRDog
MHhmCihYRU4pIC0gU01JIEZpbHRlciBSZWdpc3RlcjogMHgxCihYRU4pIC0gU01JIEZpbHRlciBS
ZWdpc3RlciBDb3VudDogMHgxCihYRU4pIC0gR3Vlc3QgVmlydHVhbCBBUElDIE1vZGVzOiAweDEK
KFhFTikgLSBEdWFsIFBQUiBMb2c6IDB4MgooWEVOKSAtIER1YWwgRXZlbnQgTG9nOiAweDIKKFhF
TikgLSBVc2VyIC8gU3VwZXJ2aXNvciBQYWdlIFByb3RlY3Rpb24KKFhFTikgLSBEZXZpY2UgVGFi
bGUgU2VnbWVudGF0aW9uOiAweDMKKFhFTikgLSBQUFIgTG9nIE92ZXJmbG93IEVhcmx5IFdhcm5p
bmcKKFhFTikgLSBQUFIgQXV0b21hdGljIFJlc3BvbnNlCihYRU4pIC0gTWVtb3J5IEFjY2VzcyBS
b3V0aW5nIGFuZCBDb250cm9sOiAwCihYRU4pIC0gQmxvY2sgU3RvcE1hcmsgTWVzc2FnZQooWEVO
KSAtIFBlcmZvcm1hbmNlIE9wdGltaXphdGlvbgooWEVOKSAtIE1TSSBDYXBhYmlsaXR5IE1NSU8g
QWNjZXNzCihYRU4pIC0gR3Vlc3QgSS9PIFByb3RlY3Rpb24KKFhFTikgLSBFbmhhbmNlZCBQUFIg
SGFuZGxpbmcKKFhFTikgLSBBdHRyaWJ1dGUgRm9yd2FyZAooWEVOKSAtIEludmFsaWRhdGUgSU9U
TEIgVHlwZQooWEVOKSAtIFZNIFRhYmxlIFNpemU6IDAKKFhFTikgLSBHdWVzdCBBY2Nlc3MgQml0
IFVwZGF0ZSBEaXNhYmxlCihYRU4pIEFNRC1WaTogQUNQSSBUYWJsZToKKFhFTikgQU1ELVZpOiAg
U2lnbmF0dXJlIElWUlMKKFhFTikgQU1ELVZpOiAgTGVuZ3RoIDB4MWU0CihYRU4pIEFNRC1WaTog
IFJldmlzaW9uIDB4MgooWEVOKSBBTUQtVmk6ICBDaGVja1N1bSAweDFhCihYRU4pIEFNRC1WaTog
IE9FTV9JZCBMRU5PVk8KKFhFTikgQU1ELVZpOiAgT0VNX1RhYmxlX0lkIFRQLVIyNSAgCihYRU4p
IEFNRC1WaTogIE9FTV9SZXZpc2lvbiAweDExMzAKKFhFTikgQU1ELVZpOiAgQ3JlYXRvcl9JZCBQ
VEVDCihYRU4pIEFNRC1WaTogIENyZWF0b3JfUmV2aXNpb24gMHgyCihYRU4pIEFNRC1WaTogSVZS
UyBCbG9jazogdHlwZSAweDEwIGZsYWdzIDB4YjAgbGVuIDB4NDggaWQgMHgyCihYRU4pIEFNRC1W
aTogSVZSUyBCbG9jazogdHlwZSAweDExIGZsYWdzIDB4YjAgbGVuIDB4NTggaWQgMHgyCihYRU4p
IEFNRC1WaTogSVZIRCBEZXZpY2UgRW50cnk6IHR5cGUgMHgzIGlkIDB4OCBmbGFncyAwCihYRU4p
IEFNRC1WaTogIERldl9JZCBSYW5nZTogMHg4IC0+IDB4ZmZmZQooWEVOKSBBTUQtVmk6IElWSEQg
RGV2aWNlIEVudHJ5OiB0eXBlIDB4NDMgaWQgMHhmZjAwIGZsYWdzIDAKKFhFTikgQU1ELVZpOiAg
RGV2X0lkIFJhbmdlOiAweGZmMDAgLT4gMHhmZmZmIGFsaWFzIDB4YTUKKFhFTikgQU1ELVZpOiBJ
VkhEIERldmljZSBFbnRyeTogdHlwZSAwIGlkIDAgZmxhZ3MgMAooWEVOKSBBTUQtVmk6IElWSEQg
RGV2aWNlIEVudHJ5OiB0eXBlIDB4NDggaWQgMCBmbGFncyAwCihYRU4pIEFNRC1WaTogSVZIRCBT
cGVjaWFsOiAwMDAwOjAwOjE0LjAgdmFyaWV0eSAweDIgaGFuZGxlIDAKKFhFTikgQU1ELVZpOiBJ
VkhEIERldmljZSBFbnRyeTogdHlwZSAweDQ4IGlkIDAgZmxhZ3MgMHhkNwooWEVOKSBBTUQtVmk6
IElWSEQgU3BlY2lhbDogMDAwMDowMDoxNC4wIHZhcmlldHkgMHgxIGhhbmRsZSAweDIwCihYRU4p
IEFNRC1WaTogSVZIRCBEZXZpY2UgRW50cnk6IHR5cGUgMHg0OCBpZCAwIGZsYWdzIDAKKFhFTikg
QU1ELVZpOiBJVkhEIFNwZWNpYWw6IDAwMDA6MDA6MDAuMSB2YXJpZXR5IDB4MSBoYW5kbGUgMHgy
MQooWEVOKSBBTUQtVmk6IElWUlMgQmxvY2s6IHR5cGUgMHgyMSBmbGFncyAweDggbGVuIDB4MjAg
aWQgMHgxMDAKKFhFTikgQU1ELVZpOiBJVk1EIEJsb2NrOiB0eXBlIDB4MjEgcGh5cyAweGM5ZjFj
MDAwIGxlbiAweDI2MDAwCihYRU4pIEFNRC1WaTogV2FybmluZzogSVZNRDogW2M5ZjFjMDAwLGM5
ZjQyMDAwKSBpcyBub3QgKGVudGlyZWx5KSBpbiByZXNlcnZlZCBtZW1vcnkKKFhFTikgQU1ELVZp
OiBFcnJvcjogSVZNRDogcGFnZSBhdCBjOWYxYzAwMCBjYW4ndCBiZSBjb252ZXJ0ZWQKKFhFTikg
QU1ELVZpOiBFcnJvciBpbml0aWFsaXphdGlvbgooWEVOKSBJL08gdmlydHVhbGlzYXRpb24gZGlz
YWJsZWQKKFhFTikgRW5hYmxpbmcgQVBJQyBtb2RlOiAgUGh5c2ljYWwuICBVc2luZyAyIEkvTyBB
UElDcwooWEVOKSBFTkFCTElORyBJTy1BUElDIElSUXMKKFhFTikgIC0+IFVzaW5nIG5ldyBBQ0sg
bWV0aG9kCihYRU4pIC4uVElNRVI6IHZlY3Rvcj0weEYwIGFwaWMxPTAgcGluMT0yIGFwaWMyPS0x
IHBpbjI9LTEKKFhFTikgQWxsb2NhdGVkIGNvbnNvbGUgcmluZyBvZiA2NCBLaUIuCihYRU4pIEhW
TTogQVNJRHMgZW5hYmxlZC4KKFhFTikgU1ZNOiBTdXBwb3J0ZWQgYWR2YW5jZWQgZmVhdHVyZXM6
CihYRU4pICAtIE5lc3RlZCBQYWdlIFRhYmxlcyAoTlBUKQooWEVOKSAgLSBMYXN0IEJyYW5jaCBS
ZWNvcmQgKExCUikgVmlydHVhbGlzYXRpb24KKFhFTikgIC0gTmV4dC1SSVAgU2F2ZWQgb24gI1ZN
RVhJVAooWEVOKSAgLSBWTUNCIENsZWFuIEJpdHMKKFhFTikgIC0gRGVjb2RlQXNzaXN0cwooWEVO
KSAgLSBWaXJ0dWFsIFZNTE9BRC9WTVNBVkUKKFhFTikgIC0gVmlydHVhbCBHSUYKKFhFTikgIC0g
UGF1c2UtSW50ZXJjZXB0IEZpbHRlcgooWEVOKSAgLSBQYXVzZS1JbnRlcmNlcHQgRmlsdGVyIFRo
cmVzaG9sZAooWEVOKSAgLSBUU0MgUmF0ZSBNU1IKKFhFTikgIC0gTlBUIFN1cGVydmlzb3IgU2hh
ZG93IFN0YWNrCihYRU4pICAtIE1TUl9TUEVDX0NUUkwgdmlydHVhbGlzYXRpb24KKFhFTikgSFZN
OiBTVk0gZW5hYmxlZAooWEVOKSBIVk06IEhhcmR3YXJlIEFzc2lzdGVkIFBhZ2luZyAoSEFQKSBk
ZXRlY3RlZAooWEVOKSBIVk06IEhBUCBwYWdlIHNpemVzOiA0a0IsIDJNQiwgMUdCCihYRU4pIGFs
dCB0YWJsZSBmZmZmODJkMDQwNDViZDk4IC0+IGZmZmY4MmQwNDA0NjkyYTgKKFhFTikgQnJvdWdo
dCB1cCAxNiBDUFVzCihYRU4pIFNjaGVkdWxpbmcgZ3JhbnVsYXJpdHk6IGNwdSwgMSBDUFUgcGVy
IHNjaGVkLXJlc291cmNlCihYRU4pIEluaXRpYWxpemluZyBDcmVkaXQyIHNjaGVkdWxlcgooWEVO
KSAgbG9hZF9wcmVjaXNpb25fc2hpZnQ6IDE4CihYRU4pICBsb2FkX3dpbmRvd19zaGlmdDogMzAK
KFhFTikgIHVuZGVybG9hZF9iYWxhbmNlX3RvbGVyYW5jZTogMAooWEVOKSAgb3ZlcmxvYWRfYmFs
YW5jZV90b2xlcmFuY2U6IC0zCihYRU4pICBydW5xdWV1ZXMgYXJyYW5nZW1lbnQ6IHNvY2tldAoo
WEVOKSAgY2FwIGVuZm9yY2VtZW50IGdyYW51bGFyaXR5OiAxMG1zCihYRU4pIGxvYWQgdHJhY2tp
bmcgd2luZG93IGxlbmd0aCAxMDczNzQxODI0IG5zCihYRU4pIEFkZGluZyBjcHUgMCB0byBydW5x
dWV1ZSAwCihYRU4pICBGaXJzdCBjcHUgb24gcnVucXVldWUsIGFjdGl2YXRpbmcKKFhFTikgQWRk
aW5nIGNwdSAxIHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNwdSAyIHRvIHJ1bnF1ZXVlIDAK
KFhFTikgQWRkaW5nIGNwdSAzIHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNwdSA0IHRvIHJ1
bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNwdSA1IHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNw
dSA2IHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNwdSA3IHRvIHJ1bnF1ZXVlIDAKKFhFTikg
QWRkaW5nIGNwdSA4IHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRkaW5nIGNwdSA5IHRvIHJ1bnF1ZXVl
IDAKKFhFTikgQWRkaW5nIGNwdSAxMCB0byBydW5xdWV1ZSAwCihYRU4pIEFkZGluZyBjcHUgMTEg
dG8gcnVucXVldWUgMAooWEVOKSBBZGRpbmcgY3B1IDEyIHRvIHJ1bnF1ZXVlIDAKKFhFTikgQWRk
aW5nIGNwdSAxMyB0byBydW5xdWV1ZSAwCihYRU4pIEFkZGluZyBjcHUgMTQgdG8gcnVucXVldWUg
MAooWEVOKSBBZGRpbmcgY3B1IDE1IHRvIHJ1bnF1ZXVlIDAKKFhFTikgbWNoZWNrX3BvbGw6IE1h
Y2hpbmUgY2hlY2sgcG9sbGluZyB0aW1lciBzdGFydGVkLgooWEVOKSBOWCAoRXhlY3V0ZSBEaXNh
YmxlKSBwcm90ZWN0aW9uIGFjdGl2ZQooWEVOKSBEb20wIGhhcyBtYXhpbXVtIDEwOTYgUElSUXMK
KFhFTikgKioqIEJ1aWxkaW5nIGEgUFYgRG9tMCAqKioKKFhFTikgIFhlbiAga2VybmVsOiA2NC1i
aXQsIGxzYiwgY29tcGF0MzIKKFhFTikgIERvbTAga2VybmVsOiA2NC1iaXQsIGxzYiwgcGFkZHIg
MHgxMDAwMDAwIC0+IDB4NGEwMDAwMAooWEVOKSBQSFlTSUNBTCBNRU1PUlkgQVJSQU5HRU1FTlQ6
CihYRU4pICBEb20wIGFsbG9jLjogICAwMDAwMDAwYzMwMDAwMDAwLT4wMDAwMDAwYzM4MDAwMDAw
ICgxNjIxOTU2NCBwYWdlcyB0byBiZSBhbGxvY2F0ZWQpCihYRU4pICBJbml0LiByYW1kaXNrOiAw
MDAwMDAwZmVhZjA1MDAwLT4wMDAwMDAwZmVlMWZmMjhlCihYRU4pIFZJUlRVQUwgTUVNT1JZIEFS
UkFOR0VNRU5UOgooWEVOKSAgTG9hZGVkIGtlcm5lbDogZmZmZmZmZmY4MTAwMDAwMC0+ZmZmZmZm
ZmY4NGEwMDAwMAooWEVOKSAgUGh5cy1NYWNoIG1hcDogMDAwMDAwODAwMDAwMDAwMC0+MDAwMDAw
ODAwN2MxODUzOAooWEVOKSAgU3RhcnQgaW5mbzogICAgZmZmZmZmZmY4NGEwMDAwMC0+ZmZmZmZm
ZmY4NGEwMDRiOAooWEVOKSAgUGFnZSB0YWJsZXM6ICAgZmZmZmZmZmY4NGEwMTAwMC0+ZmZmZmZm
ZmY4NGEyYTAwMAooWEVOKSAgQm9vdCBzdGFjazogICAgZmZmZmZmZmY4NGEyYTAwMC0+ZmZmZmZm
ZmY4NGEyYjAwMAooWEVOKSAgVE9UQUw6ICAgICAgICAgZmZmZmZmZmY4MDAwMDAwMC0+ZmZmZmZm
ZmY4NGMwMDAwMAooWEVOKSAgRU5UUlkgQUREUkVTUzogZmZmZmZmZmY4M2FlNzRjMAooWEVOKSBE
b20wIGhhcyBtYXhpbXVtIDE2IFZDUFVzCihYRU4pIEluaXRpYWwgbG93IG1lbW9yeSB2aXJxIHRo
cmVzaG9sZCBzZXQgYXQgMHg0MDAwIHBhZ2VzLgooWEVOKSBTY3J1YmJpbmcgRnJlZSBSQU0gaW4g
YmFja2dyb3VuZAooWEVOKSBTdGQuIExvZ2xldmVsOiBFcnJvcnMsIHdhcm5pbmdzIGFuZCBpbmZv
CihYRU4pIEd1ZXN0IExvZ2xldmVsOiBOb3RoaW5nIChSYXRlLWxpbWl0ZWQ6IEVycm9ycyBhbmQg
d2FybmluZ3MpCihYRU4pIFhlbiBpcyByZWxpbnF1aXNoaW5nIFZHQSBjb25zb2xlLgooWEVOKSAq
KiogU2VyaWFsIGlucHV0IHRvIERPTTAgKHR5cGUgJ0NUUkwtYScgdGhyZWUgdGltZXMgdG8gc3dp
dGNoIGlucHV0KQooWEVOKSBGcmVlZCA2NDhrQiBpbml0IG1lbW9yeQo=

--b1_GBYj6Y1YZQCBAVgd6bUn1cquVDPuaNM7JB5U3PQFQw--



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 15:07:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 15:07:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674489.1049446 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYfQ-00028X-91; Thu, 01 Feb 2024 15:07:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674489.1049446; Thu, 01 Feb 2024 15:07:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYfQ-00028Q-5y; Thu, 01 Feb 2024 15:07:52 +0000
Received: by outflank-mailman (input) for mailman id 674489;
 Thu, 01 Feb 2024 15:07:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i6jM=JK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVYfP-0001qg-At
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 15:07:51 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id aa79f84a-c113-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 16:07:50 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-40fc2a6f8b6so2487435e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 07:07:50 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 f6-20020a05600c154600b0040fa661ee82sm4733650wmg.44.2024.02.01.07.07.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 07:07:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa79f84a-c113-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706800070; x=1707404870; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nGJOcsAA00gDsEgOJ9ltZg1lwRc3fMEvRczR2WdI3fY=;
        b=l7HlO8tWaXBdYQDEjildnw29gx76kmOqCi51aMYTn3bpAYLxYfsmdgJceSUgZfW8+r
         cyVUPryQpXKmdQanJEXk8NyR0g/DG5OeJTptlm4gOs/9uaMHNTJNCy7wL4dPvsmGMoKz
         s88hl8NHJpS1thtedwPmRB+5zalgWCQLEhoJU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706800070; x=1707404870;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nGJOcsAA00gDsEgOJ9ltZg1lwRc3fMEvRczR2WdI3fY=;
        b=tD8/bx3hbUFrypJ8IGfHhPQm22JxNMAtjPC++czkInI+KajXnSzyx+d5q/TU9X7VQ0
         kl1eBUQyhdeQIY5JeqTNiqU00z/qNDGJn446h2NFwxjsftLhPqS0yODaV7pLDWbcXqzv
         1mlIcnrDPKjOZVod0EaOQeCCNtLZrARoA0cnsJ20pSBd8nEymJdzpBvbOpnJkXn4eOe8
         w2UewuMaccxDOfRBtHVl/ucQ6H8rXdAC+HbuTX19eUPzXpPDlVHdDtcG9tx29qeZQl8G
         EHYTxX52I8Vf6zpIFNgNkM5MOXdiJh+sKUeWwIOwj1mgwFdyaTtKkBpH5HM61/EvXHu2
         +HxA==
X-Gm-Message-State: AOJu0YzSADe0ENSnmQx+DIoNRayGt6RY1bjcjw6ftctTh69o8Yp2VG7u
	kdWXbdJYSEQGrchd2Gyem1/mrrnW0s6cEqBKmsh/cWomO4b7rUvMCVa0/wB0Hm0=
X-Google-Smtp-Source: AGHT+IHi1WXIrQV39Nw64NCPfXhk+pvoJVXXgdn0MTzRkbB2glhoZ880ndtYAmDlt1txv+rocLiSFg==
X-Received: by 2002:a05:600c:5612:b0:40e:5ed3:ccf4 with SMTP id jr18-20020a05600c561200b0040e5ed3ccf4mr4677446wmb.16.1706800069963;
        Thu, 01 Feb 2024 07:07:49 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXP4Tu5Jw/IzHJ412fbg0giCN7S6+RIp4f1KngV8xCvhTjyypRdB4OeKw3ZyvPtN3xaCAmgEli6fj8JzB3xCeC2K5TO5Eo0iWA=
Message-ID: <e31aba88-1607-4a74-b2f9-66e754c89e9c@citrix.com>
Date: Thu, 1 Feb 2024 15:07:49 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Fix some typos in comments
Content-Language: en-GB
To: Frediano Ziglio <frediano.ziglio@cloud.com>,
 George Dunlap <george.dunlap@citrix.com>
Cc: xen-devel@lists.xenproject.org
References: <20240201145943.738531-1-frediano.ziglio@cloud.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240201145943.738531-1-frediano.ziglio@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01/02/2024 2:59 pm, Frediano Ziglio wrote:
> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
> ---
>  arch/x86/include/arch/processor.h | 2 +-
>  include/xen/hvm/params.h          | 2 +-
>  include/xtf/console.h             | 2 +-
>  include/xtf/extable.h             | 4 ++--
>  4 files changed, 5 insertions(+), 5 deletions(-)

This looks like an XTF patch?

Considering that I do try to remember to spell check before committing,
I'm somewhat embarrassed by this.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 15:10:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 15:10:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674494.1049456 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYiH-0003sc-Lv; Thu, 01 Feb 2024 15:10:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674494.1049456; Thu, 01 Feb 2024 15:10:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYiH-0003sV-JE; Thu, 01 Feb 2024 15:10:49 +0000
Received: by outflank-mailman (input) for mailman id 674494;
 Thu, 01 Feb 2024 15:10:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVYiG-0003sL-Kw
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 15:10:48 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 13586c00-c114-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 16:10:46 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-556c3f0d6c5so1263139a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 07:10:46 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 fi7-20020a170906da0700b00a36edfb508asm15809ejb.79.2024.02.01.07.10.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 07:10:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13586c00-c114-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706800246; x=1707405046; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MpmtCMKIg7MG7KLzATLN7tvHdDRolI1j9YpQuU+oSQU=;
        b=aRqfkYsJs+etBmF9yFOy6h8zA1aQzCv/EcB3MOG8YiM3uDwvV0eis5dKYd0yk030Mn
         tuEbsRTgDLWHRPF8ziPQmfv+DQYIIXtAi/rR2oYUJ7abm0I28c6qUy6g7vhDMxBJWw/V
         R0xKIYLbq1D6DNhlz+MxdCmfOcF6BCOdOSB+mhMraugoDVF4HXMeX9O2BEIROo0Nctbr
         HI1JNfuW8QnDxZrRBbz+GcKLjp0/klevlZsN3b8FoMpBlJQ6oZ/XkQAUm1KfiRQ2QJSR
         wAC7Q0GR4DSBP5MI/BfJzdTCcbJtx1Rq3zGan3138biKTcmtiiIW4v57LZxJdQXTJr+4
         0fbQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706800246; x=1707405046;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MpmtCMKIg7MG7KLzATLN7tvHdDRolI1j9YpQuU+oSQU=;
        b=nyVNdg7VFP74R2tCjnMTaapOygPGL0dtEr5A87bcH8q5vsT8cl/cLIzPSKSl+USjwx
         YkNtA0kkCvUA6SNArGnBscyD5qzOVA2G+x9k1oA1C8jzA86TzgQu3ZDFQwgWs2LFuAwl
         XOJS1Zs9kHHaAaPgflIsyrlxd3Ibmv40OVqTG/tF4jkSBv66As7OXRYgX70itm4rPLUu
         26UrDO2StIR+bdnyC48aER/mTnEuJkWqEEHk8ND0Bln/yIo1NHxxPdOeJYColdShRqTC
         AWjL7bpe/IT4iDXGKu9G2MSqtsF1F2xTlFdeltunjftkz5NXtlK8pJ2lsvgdXksxDyjD
         C60A==
X-Gm-Message-State: AOJu0YyuH9adlzYjfJiGN3ech3DId3HIt1OSO5Pe3NkusEYkPKUoNiUS
	XjQjo7C9A4frQbBqLzOiTcw9b1OjR+cOiO6vM2dXJrDAhsfKdnChFJMxyLSSSQ==
X-Google-Smtp-Source: AGHT+IFEpnKZ6B3Px3BJI0czRwgs70itxp5h5od8ByQw4MCMAbS+fVkWAyW/nE6DxqtxJDLUmYl4+Q==
X-Received: by 2002:a17:906:c7d2:b0:a36:9f72:2461 with SMTP id dc18-20020a170906c7d200b00a369f722461mr1894807ejb.69.1706800245952;
        Thu, 01 Feb 2024 07:10:45 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUzfX2oWSfXNYZuvfUjh+pvznlFpyEUpMiT0WSDrvE8dnVPoAMp3WPZtAMOoNSQS7LmXJ/EF1CCBpmzXbZf3NkMmHsFJAt7rMEgA4X2HU2h6sD1BVo66/y7DFAf18k2gEDLWCS2Qzby1Nr3kunL7Y4J0tDGwkVQSFfmkV/BPaCBX5+fC+pI4kg5SEWFSThBoH/vzRlIjdV7leLV30skF96fu1+TSn2qpIHFsT0TAhchgVf00Mwb+WSYJtOxOrWuvwrrkkoPOgk1gJmvuooptMY/welzze3JbzRjSZ0At3jRSG09y/gvQbDvLcUu
Message-ID: <b39e8700-62ce-44f1-b1bc-f25bb4f27898@suse.com>
Date: Thu, 1 Feb 2024 16:10:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 12/15] xen/arm: add Xen cache colors command line
 parameter
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Luca Miccio <lucmiccio@gmail.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-13-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-13-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:18, Carlo Nonato wrote:
> --- a/xen/common/llc-coloring.c
> +++ b/xen/common/llc-coloring.c
> @@ -9,6 +9,9 @@
>  #include <xen/llc-coloring.h>
>  #include <xen/param.h>
>  
> +#define XEN_DEFAULT_COLOR       0
> +#define XEN_DEFAULT_NUM_COLORS  1
> +
>  bool __ro_after_init llc_coloring_enabled;
>  boolean_param("llc-coloring", llc_coloring_enabled);
>  
> @@ -21,6 +24,9 @@ static unsigned int __ro_after_init max_nr_colors = CONFIG_NR_LLC_COLORS;
>  static unsigned int __initdata dom0_colors[CONFIG_NR_LLC_COLORS];
>  static unsigned int __initdata dom0_num_colors;
>  
> +static unsigned int __ro_after_init xen_colors[CONFIG_NR_LLC_COLORS];

So unlike for Dom0 here you use the static buffer at runtime.

> +static unsigned int __ro_after_init xen_num_colors;

Taken together, I don't see the value in having XEN_DEFAULT_COLOR:
One can't simply change it and XEN_DEFAULT_NUM_COLORS to have Xen have,
say, 4 colors by default. I think you want to have xen_colors[] have
an initializer, with XEN_DEFAULT_COLOR dropped and XEN_DEFAULT_NUM_COLORS
moved. Or you actually allocate the runtime buffer if a command line
option is found, and just use ARRAY_SIZE() as the default count.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 15:14:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 15:14:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674501.1049466 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYlK-0004lt-6d; Thu, 01 Feb 2024 15:13:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674501.1049466; Thu, 01 Feb 2024 15:13:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVYlK-0004lm-3s; Thu, 01 Feb 2024 15:13:58 +0000
Received: by outflank-mailman (input) for mailman id 674501;
 Thu, 01 Feb 2024 15:13:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVYlI-0004lb-FB
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 15:13:56 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 840f4193-c114-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 16:13:55 +0100 (CET)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2d051fb89fbso14707001fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 07:13:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 fd12-20020a056402388c00b00557d839727esm6884848edb.7.2024.02.01.07.13.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 07:13:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 840f4193-c114-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706800435; x=1707405235; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=aXKi7HcGOdYt1N8wDuA8yYqEyLVPLGkiW5x+mctbQnA=;
        b=bWkIisqtI5SzalPgEt5kCQ7YohYzKYzey2y5EKPQzdI4TmcfSk7Pm1rChK9jdqIZQS
         KVJc7PR3SQVxBggJUm59TqBUf5sPLow3leh86El51BpRRKdHSCvSHZ+OCb5l/QXUCgV8
         JNfWAjoRdibA4PclPlMOgsTSJnSm+6gw+neHEsfH8ap999y5KO2c0mtJD6yv0bXS034A
         kA9uH2jPS5qOaZiVP1TbgNjduI9kKrU83lelUBydZfEtwPCsSioY1h2Dgtteg5oGzgHb
         CIO2zOU5O5+CxcR4V3vK2PZdls2anfjYS3R4AM8X8oobWlQbRB/ispLWVu9LXb07SYrE
         TccA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706800435; x=1707405235;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=aXKi7HcGOdYt1N8wDuA8yYqEyLVPLGkiW5x+mctbQnA=;
        b=MUz7CIGyy3LJhuyH0jsr+2C/kIHzQHcPaNXZYrwhCu/tl+ICjiucP+FgpOEWCy8PhL
         MWoO5bCud2ex+ai6u4arHMXVEpGGQzJl+OO7cFCHFPJaI1WjRTA1DQud7cE2QD/tUQpC
         JgyggVXB0sFviGJipPA2llxP2+Sg6bj4qQkU/8lDfbcUc4f9etdz0ywweUt0KlGwboxQ
         R99lk4wA3skc6sVYGZ69S8pq69DHiajqwK2NlMI+qTNEgTYFsDo1d4tKs60FQsJfnugh
         +t29EGKh3fkbdrZjJpndye9/y3EVLLpG3EMXlA76O9Sfn5FI/1s3q2gr69v5Jn+My3Lz
         b06w==
X-Gm-Message-State: AOJu0YwXec05dMGqoiVFRJ4eRWa3f+0W24HlAqEHhZ9WDwPGNSmTYxOd
	q+yfVTiwWFnXdXIb9rn1Kl5shxPTI2avACSfRVnczO+WVF7mp0z9CpWpp5lX1w==
X-Google-Smtp-Source: AGHT+IEwSjCvoaggmdb4kXFYQ3RqVlua5qOImD5wc0V+Ei6g/Qsou0fiRFdsxrx/AZH+QvWRLeq67A==
X-Received: by 2002:a2e:9858:0:b0:2d0:438d:fb2b with SMTP id e24-20020a2e9858000000b002d0438dfb2bmr3924988ljj.33.1706800434948;
        Thu, 01 Feb 2024 07:13:54 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXCnKVNGws8gZYP1EY2/PjY6v/uPpFtl6vBpS+NUH7YIylRcIrznKZsk39x9ci+NlSxlAgPIS01v9xRBuDifvM23uRKe+uP7rZN2W0MVQnREkGVJJQ0NuNRmqChVhrtl6q3tfWTvoLzR8r84dAFz0OQrgB/df9I4jdimor7gKgAp7GgTg10I/54TdqZaS76eT0t/PnZTx91S7FnK5py6w7YDBrqVomlnlrRuaxkOaHbhzqUE7uEGtw=
Message-ID: <246c970c-7fad-4842-8305-bcee78a52843@suse.com>
Date: Thu, 1 Feb 2024 16:13:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 09/15] xen/page_alloc: introduce preserved page flags
 macro
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-10-carlo.nonato@minervasys.tech>
 <8a3f3c38-b290-417a-86cb-216e36182abe@suse.com>
 <CAG+AhRU5Yhz_VF02CfaFD5ifvw8cUQbsM7-UiMcRWm+1LR4btg@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRU5Yhz_VF02CfaFD5ifvw8cUQbsM7-UiMcRWm+1LR4btg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01.02.2024 15:49, Carlo Nonato wrote:
> On Thu, Feb 1, 2024 at 3:24 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>> PGC_static and PGC_extra are flags that needs to be preserved when assigning
>>> a page. Define a new macro that groups those flags and use it instead of
>>> or'ing every time.
>>
>> While here you say where the "preserving" applies, ...
>>
>>> --- a/xen/common/page_alloc.c
>>> +++ b/xen/common/page_alloc.c
>>> @@ -157,6 +157,8 @@
>>>  #define PGC_static 0
>>>  #endif
>>>
>>> +#define PGC_preserved (PGC_extra | PGC_static)
>>
>> ... nothing is said here. From the earlier version I also seem to recall
>> that the constant was then used outside of assign_pages(). That would
>> then mean amending whatever comment would be added here.
> 
> Yes, but it was used in places where the name didn't fit (to stop merging in
> free_heap_pages()) and so I thought it would've been better to use the
> constant only for one of the two concepts: only for preserved flags in
> assign_pages().
> 
> Are you suggesting adding a comment to this #define to clarify its usage?

Yes. Albeit if I understand the earlier paragraph right, you don't use
it further (anymore). In which case the question would be: Is the patch
still needed, and if so is it rightfully part of this series?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 15:43:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 15:43:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674508.1049477 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZDq-00062u-E9; Thu, 01 Feb 2024 15:43:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674508.1049477; Thu, 01 Feb 2024 15:43:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZDq-00062n-BM; Thu, 01 Feb 2024 15:43:26 +0000
Received: by outflank-mailman (input) for mailman id 674508;
 Thu, 01 Feb 2024 15:43:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Zkz=JK=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rVZDp-00061u-Cu
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 15:43:25 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a240a512-c118-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 16:43:24 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-40e76626170so9304235e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 07:43:24 -0800 (PST)
Received: from [10.95.167.50] (54-240-197-233.amazon.com. [54.240.197.233])
 by smtp.gmail.com with ESMTPSA id
 h12-20020a05600c260c00b0040d87100733sm4683354wma.39.2024.02.01.07.43.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 07:43:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a240a512-c118-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706802203; x=1707407003; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=g2UrnwlOgsItEDKTVLgJCkedsveuSTltHV+AVhMWfwo=;
        b=jMtTrhX8uL6LODSVrmT42juVnhrpOHZKIEYzr1QBh8NbFYFLffvQHgXbnHHTkJvh5l
         LLT+3EvTY4JbfLF9kyggyc7OYtXskXlzrPix2bDe2LJWeOf9cUb0eC7BFtqDPMz14yjH
         UgPy37X1YtM1wtPzalR9XRVlNWXSH6KwXAwQ4dpj5rlWIgUTCAcGeRM3tZ9vU8wmm40c
         0eFNjg1RP+WFUU3Up56Tn8O+5godcbo0C1fQgOichHwMwgddn4ogam4xtpuqJepvGm3l
         frpbswWLyBFz0OZF/R+mltBBL+sEPHmMnA5AETEnnMM1kItLK6W9+5VzYbdhc1pRgyY9
         eq5w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706802203; x=1707407003;
        h=content-transfer-encoding:in-reply-to:organization:content-language
         :references:cc:to:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=g2UrnwlOgsItEDKTVLgJCkedsveuSTltHV+AVhMWfwo=;
        b=pAw/gdwY1Zge1C395g9s1uxPrWs7gfLCsn5XhBCQkhznRI4EshW3EAxSlYVig1Nn4l
         U9Sewp6JHnimMYwe7LGlGbYc+oIqNbAmESzsfGc7xMrF74ZcAkMKeGSfe0AxBnf0j+EC
         Z9IPJq0enkK8AvgIdnu3Vb17rA9KccpvCn+rSG+1iPZA/JwOfY3jZ2N0CUZ2WxNTCWMS
         EepNZXKunqEpoaipcfWMnTbsjws9QRkublNQUavqTTMo8OPESqxakIEZ5lFJXm82JtP0
         qSPzTFdZyV3Y6Qk4Lo80ogeIi+X+XanF8kecRRgdxvmv8B0/z8wdXjlBcX28Noslvewd
         5ksQ==
X-Gm-Message-State: AOJu0YyYwUS/5G4CikH+zb12YVm6e6PM5vUTKvO4Fjj8FY9/2zIbnqUN
	MxZN4/nypod84iWuO0mvYGS4cPQxo/6EkxBpKLqSGLLY4nsNP1P2
X-Google-Smtp-Source: AGHT+IGwEQbXnJ24dLKwBX9KdAuPoFpCEhwu8WSK1M5x2HdTKaFyDg5C3nBYtavqD107Kr1ecR86Gg==
X-Received: by 2002:a5d:440b:0:b0:33b:182f:eb9d with SMTP id z11-20020a5d440b000000b0033b182feb9dmr1261356wrq.43.1706802203356;
        Thu, 01 Feb 2024 07:43:23 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWhQorHyBHjSUpH8dxTdPXUf5nO8Z9/mt5FVm88AJruzfba5njDn7SvUjQAgNPuU7tlZqD0U5nA3VizSmFmtAhGM9Ry5S8SuJSInCWFV5UlbgMbv8KnpMdlcRXnteE3NnhGPS4NFlly26oaQtBBnbptkRA=
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <ee98c944-610a-44a3-a26e-a10c25c12eb8@xen.org>
Date: Thu, 1 Feb 2024 15:43:21 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH net] xen-netback: properly sync TX responses
To: Jakub Kicinski <kuba@kernel.org>
Cc: Jan Beulich <jbeulich@suse.com>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <980c6c3d-e10e-4459-8565-e8fbde122f00@suse.com>
 <20240131162336.7d3ba09e@kernel.org>
Content-Language: en-US
Organization: Xen Project
In-Reply-To: <20240131162336.7d3ba09e@kernel.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 01/02/2024 00:23, Jakub Kicinski wrote:
> On Mon, 29 Jan 2024 14:03:08 +0100 Jan Beulich wrote:
>> Invoking the make_tx_response() / push_tx_responses() pair with no lock
>> held would be acceptable only if all such invocations happened from the
>> same context (NAPI instance or dealloc thread). Since this isn't the
>> case, and since the interface "spec" also doesn't demand that multicast
>> operations may only be performed with no in-flight transmits,
>> MCAST_{ADD,DEL} processing also needs to acquire the response lock
>> around the invocations.
>>
>> To prevent similar mistakes going forward, "downgrade" the present
>> functions to private helpers of just the two remaining ones using them
>> directly, with no forward declarations anymore. This involves renaming
>> what so far was make_tx_response(), for the new function of that name
>> to serve the new (wrapper) purpose.
>>
>> While there,
>> - constify the txp parameters,
>> - correct xenvif_idx_release()'s status parameter's type,
>> - rename {,_}make_tx_response()'s status parameters for consistency with
>>    xenvif_idx_release()'s.
> 
> Hi Paul, is this one on your TODO list to review or should
> we do our best? :)

Sorry for the delay. I'll take a look at this now.

   Paul


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 15:44:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 15:44:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674510.1049487 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZEj-0006XT-NR; Thu, 01 Feb 2024 15:44:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674510.1049487; Thu, 01 Feb 2024 15:44:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZEj-0006XM-Ki; Thu, 01 Feb 2024 15:44:21 +0000
Received: by outflank-mailman (input) for mailman id 674510;
 Thu, 01 Feb 2024 15:44:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVZEi-0006X2-4B; Thu, 01 Feb 2024 15:44:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVZEi-0003UO-35; Thu, 01 Feb 2024 15:44:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVZEh-0001aI-Gl; Thu, 01 Feb 2024 15:44:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVZEh-0000pg-GJ; Thu, 01 Feb 2024 15:44:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jrPTGdghm9eLsf7XVixHjPJv1bXKsmFnKpgqmiZKISA=; b=dLc/vBNMEtgz/ybT5ItLDFPy8B
	DwrIfy011kKW7FjBaNkJPR3GdMIZ2TK9kHJO9d21jPsYuBCBPFQSxFKaN109FurcinZ8oF/CtPX4x
	pY3Mknlvl52rhrDeRthL+RCxhdI9Mvi4/EBWSocWOCuVhkGUgpKSUQ4GZwzmMZqhmzTs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184548-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184548: regressions - FAIL
X-Osstest-Failures:
    libvirt:build-arm64-libvirt:libvirt-build:fail:regression
    libvirt:test-arm64-arm64-libvirt:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=f85a382a0e709afea3a4021de593b1679553a35a
X-Osstest-Versions-That:
    libvirt=2757e91c2b28b704d9a0b586fb60012450110b1a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 01 Feb 2024 15:44:19 +0000

flight 184548 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184548/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-libvirt           6 libvirt-build            fail REGR. vs. 184537

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184537
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184537
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184537
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              f85a382a0e709afea3a4021de593b1679553a35a
baseline version:
 libvirt              2757e91c2b28b704d9a0b586fb60012450110b1a

Last test of basis   184537  2024-01-31 04:24:45 Z    1 days
Testing same since   184548  2024-02-01 06:57:31 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrea Bolognani <abologna@redhat.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Peter Krempa <pkrempa@redhat.com>
  Stefano Brivio <sbrivio@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          fail    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     blocked 
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               blocked 
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 582 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 15:53:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 15:53:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674519.1049500 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZNM-00013m-Ju; Thu, 01 Feb 2024 15:53:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674519.1049500; Thu, 01 Feb 2024 15:53:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZNM-000133-Gn; Thu, 01 Feb 2024 15:53:16 +0000
Received: by outflank-mailman (input) for mailman id 674519;
 Thu, 01 Feb 2024 15:53:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=XFwQ=JK=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVZNL-00012x-PD
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 15:53:15 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 016915aa-c11a-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 16:53:13 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a3566c0309fso145167566b.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 07:53:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bv11-20020a170906b1cb00b00a35e7326b69sm3832284ejb.93.2024.02.01.07.53.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 07:53:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 016915aa-c11a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706802793; x=1707407593; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jnPQD2HqisVQvijjQ142klgke++wMuogZ1JRlvcJlRI=;
        b=KSw0As6M4LuqAuqCalTNb85ZhTpEDUtYhaZxsFwdq40HnXqWueRnQb9CxMhUT6AqBD
         2ly67y400kTJ1qdJnsVTQTezO4gBDXcq/Po+vNGHzVvbMP0+Mf+CDA7NGcEAfpapUHMN
         CYPxygi3GqFsRGZZP/7qMFdqSJjUMDSvVrwidMGwniwuNbzcBf1h6gLArFGT/g2hUB3a
         rRbk2vzCwIApEd1Jd9OOwKp/4ES9q6AIJlwg+juyGcl0G/DcSAFZ0vXHicuXx8vgylHp
         +kERuB6AvhsphDi0lIAeFvGsfC0i/5/flhofWyAW2LKAgMphJWYObwepMt3/GiQJtFNk
         vcSg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706802793; x=1707407593;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jnPQD2HqisVQvijjQ142klgke++wMuogZ1JRlvcJlRI=;
        b=QT8FzUJFumOuy1vZUa4aH2EB9LfWb1V1p2yEwGESU5Gncd0FuYr4tQRiKmhdSywzMZ
         KEfxnc85AZ+/TNSwwcCf6FvmOV2PQ6nHskqC/gulBSjcvhE2qzLHVV16mi5gH65jWVxJ
         cg4XzY2AiKxlscX7GjZesIFOIdMWRxkQEaTrCgbybcwfAyucSuIa/3v1dDpOoOzn+dGI
         wAjYiotOXU+UlqSCyF5+WBGr6PUwbJEg0vDeDWTZotm1fgs1fTGsEY92ke6nhVo17Z7e
         erB88hsdcAaJw5KeoJuX2wzj38FpFA4Plefa/uz9+5SJE4JcF+zpLVUfypm3LoIusIgp
         jPcg==
X-Gm-Message-State: AOJu0YylagojWQBpM1g0c27zhwZ15EkmKIW83rXqSfq8Ae6Px5sN8oK1
	/DU4J5SZbL9TC/nh8MnlrSgwNS2qUzNins/WM+fJInNlVK0lh4JPrYelLKFvpA==
X-Google-Smtp-Source: AGHT+IH5atcEZv0GpexmtmtfD2kn1bhOvJUNo+K8fd1EjAJFL1oU1YyrgFAn8+Z45/5Y8avjunghSQ==
X-Received: by 2002:a17:906:e9a:b0:a2d:a6a7:b3bc with SMTP id p26-20020a1709060e9a00b00a2da6a7b3bcmr3386227ejf.4.1706802792683;
        Thu, 01 Feb 2024 07:53:12 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCURc03naErdxIlak0VCqU9q9pK521sr8P/KDp0LC80O9mFYeQnumZLLigpWFcDJX7+Np7FQcVsgiur4aLUunfU+TtrFUlA50TCdpJWtiTv5UMWEFvqzQhotj4VH2k8EFwHjqz2fD/IqCXPPdo/MbIWN6Q1IEzVoFf02rJfkaezF/Uel1ktroSg5AnLcCyZk6XFKJu5yw3laaVAaVyCzYXFSalw8X0M02CuxY637CnmPKcIo0YvrzEJNrHGiyD2suXVkhnaHCB24rAOt03auaIE9L0oWiX1lI6Xh46FLsvyRKFgYg5XlSqrLBIoIM+JowAyzmJPWtUTwnnEXeII/7LdSPuqOIl5f4jFaSNZx+5/oJpCeackIqZgv1LZsdF+41Um2JQ==
Message-ID: <1739dbb4-5d7e-4d84-8d4b-bfb13c4041d8@suse.com>
Date: Thu, 1 Feb 2024 16:53:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 10/15] xen: add cache coloring allocator for domains
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-11-carlo.nonato@minervasys.tech>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-11-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:18, Carlo Nonato wrote:
> @@ -157,7 +158,11 @@
>  #define PGC_static 0
>  #endif
>  
> -#define PGC_preserved (PGC_extra | PGC_static)
> +#ifndef PGC_colored
> +#define PGC_colored 0
> +#endif
> +
> +#define PGC_preserved (PGC_extra | PGC_static | PGC_colored)
>  
>  #ifndef PGT_TYPE_INFO_INITIALIZER
>  #define PGT_TYPE_INFO_INITIALIZER 0
> @@ -1504,6 +1509,7 @@ static void free_heap_pages(
>              if ( !mfn_valid(page_to_mfn(predecessor)) ||
>                   !page_state_is(predecessor, free) ||
>                   (predecessor->count_info & PGC_static) ||
> +                 (predecessor->count_info & PGC_colored) ||

How about a 2nd #define (e.g. PGC_no_buddy_merge) to use here and ...

>                   (PFN_ORDER(predecessor) != order) ||
>                   (page_to_nid(predecessor) != node) )
>                  break;
> @@ -1528,6 +1534,7 @@ static void free_heap_pages(
>              if ( !mfn_valid(page_to_mfn(successor)) ||
>                   !page_state_is(successor, free) ||
>                   (successor->count_info & PGC_static) ||
> +                 (successor->count_info & PGC_colored) ||

... here? That'll then also avoid me commenting that I don't see why
these two PGC_* checks aren't folded.

> @@ -1943,6 +1950,161 @@ static unsigned long avail_heap_pages(
>      return free_pages;
>  }
>  
> +/*************************
> + * COLORED SIDE-ALLOCATOR
> + *
> + * Pages are grouped by LLC color in lists which are globally referred to as the
> + * color heap. Lists are populated in end_boot_allocator().
> + * After initialization there will be N lists where N is the number of
> + * available colors on the platform.
> + */
> +static struct page_list_head *__ro_after_init _color_heap;
> +static unsigned long *__ro_after_init free_colored_pages;
> +
> +/* Memory required for buddy allocator to work with colored one */
> +#ifdef CONFIG_LLC_COLORING
> +static unsigned long __initdata buddy_alloc_size =
> +    MB(CONFIG_BUDDY_ALLOCATOR_SIZE);
> +size_param("buddy-alloc-size", buddy_alloc_size);
> +
> +#define domain_num_llc_colors(d) (d)->num_llc_colors
> +#define domain_llc_color(d, i)   (d)->llc_colors[(i)]

Nit: Unnecessary parentheses inside the square brackets.

> +#else
> +static unsigned long __initdata buddy_alloc_size;
> +
> +#define domain_num_llc_colors(d) 0
> +#define domain_llc_color(d, i)   0
> +#endif
> +
> +#define color_heap(color) (&_color_heap[color])

Wouldn't this better live next to _color_heap()'s definition?

> +static void free_color_heap_page(struct page_info *pg, bool need_scrub)
> +{
> +    unsigned int color = page_to_llc_color(pg);
> +    struct page_list_head *head = color_heap(color);
> +
> +    spin_lock(&heap_lock);
> +
> +    mark_page_free(pg, page_to_mfn(pg));
> +
> +    if ( need_scrub )
> +    {
> +        pg->count_info |= PGC_need_scrub;
> +        poison_one_page(pg);
> +    }
> +
> +    pg->count_info |= PGC_colored;

The page transiently losing PGC_colored is not an issue then (presumably
because of holding the heap lock)? Did you consider having mark_page_free()
also use PGC_preserved?

> +    free_colored_pages[color]++;
> +    page_list_add(pg, head);
> +
> +    spin_unlock(&heap_lock);
> +}
> +
> +static struct page_info *alloc_color_heap_page(unsigned int memflags,
> +                                               const struct domain *d)
> +{
> +    struct page_info *pg = NULL;
> +    unsigned int i, color = 0;
> +    unsigned long max = 0;
> +    bool need_tlbflush = false;
> +    uint32_t tlbflush_timestamp = 0;
> +    bool scrub = !(memflags & MEMF_no_scrub);
> +
> +    spin_lock(&heap_lock);
> +
> +    for ( i = 0; i < domain_num_llc_colors(d); i++ )
> +    {
> +        unsigned long free = free_colored_pages[domain_llc_color(d, i)];
> +
> +        if ( free > max )
> +        {
> +            color = domain_llc_color(d, i);
> +            pg = page_list_first(color_heap(color));
> +            max = free;
> +        }
> +    }

The apporach is likely fine at least initially, but: By going from where
the most free pages are, you're not necessarily evenly distributing a
domain's pages over the colors it may use, unless the domain uses its
set of colors exclusively.

> +    if ( !pg )
> +    {
> +        spin_unlock(&heap_lock);
> +        return NULL;
> +    }
> +
> +    pg->count_info = PGC_state_inuse | PGC_colored |
> +                     (pg->count_info & PGC_need_scrub);

Isn't PGC_colored already set on the page? Together with ...

> +    free_colored_pages[color]--;
> +    page_list_del(pg, color_heap(color));
> +
> +    if ( !(memflags & MEMF_no_tlbflush) )
> +        accumulate_tlbflush(&need_tlbflush, pg, &tlbflush_timestamp);
> +
> +    init_free_page_fields(pg);
> +
> +    spin_unlock(&heap_lock);
> +
> +    if ( test_and_clear_bit(_PGC_need_scrub, &pg->count_info) && scrub )

... this, can't the above be simplified?

> +        scrub_one_page(pg);
> +    else if ( scrub )
> +        check_one_page(pg);
> +
> +    if ( need_tlbflush )
> +        filtered_flush_tlb_mask(tlbflush_timestamp);
> +
> +    flush_page_to_ram(mfn_x(page_to_mfn(pg)),
> +                      !(memflags & MEMF_no_icache_flush));
> +
> +    return pg;
> +}
> +
> +static void __init init_color_heap_pages(struct page_info *pg,
> +                                         unsigned long nr_pages)
> +{
> +    unsigned int i;
> +    bool need_scrub = opt_bootscrub == BOOTSCRUB_IDLE;
> +
> +    if ( buddy_alloc_size )
> +    {
> +        unsigned long buddy_pages = min(PFN_DOWN(buddy_alloc_size), nr_pages);
> +
> +        init_heap_pages(pg, buddy_pages);
> +        nr_pages -= buddy_pages;
> +        buddy_alloc_size -= buddy_pages << PAGE_SHIFT;
> +        pg += buddy_pages;
> +    }
> +
> +    if ( !_color_heap )
> +    {
> +        unsigned int max_nr_colors = get_max_nr_llc_colors();
> +
> +        _color_heap = xmalloc_array(struct page_list_head, max_nr_colors);
> +        free_colored_pages = xzalloc_array(unsigned long, max_nr_colors);
> +        if ( !_color_heap || !free_colored_pages )
> +            panic("Can't allocate colored heap. Buddy reserved size is too low");
> +
> +        for ( i = 0; i < max_nr_colors; i++ )
> +            INIT_PAGE_LIST_HEAD(color_heap(i));
> +    }
> +
> +    if ( nr_pages )
> +        printk(XENLOG_DEBUG
> +               "Init color heap with %lu pages, start MFN: %"PRI_mfn"\n",
> +               nr_pages, mfn_x(page_to_mfn(pg)));

This message can be issued more than once. Is that really desirable /
necessary?

> @@ -2458,7 +2626,14 @@ struct page_info *alloc_domheap_pages(
>      if ( memflags & MEMF_no_owner )
>          memflags |= MEMF_no_refcount;
>  
> -    if ( !dma_bitsize )
> +    /* Only domains are supported for coloring */
> +    if ( d && llc_coloring_enabled )
> +    {
> +        /* Colored allocation must be done on 0 order */
> +        if ( order || (pg = alloc_color_heap_page(memflags, d)) == NULL )
> +            return NULL;
> +    }

I think I had asked before: What about MEMF_node() or MEMF_bits()
having been used by the caller?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 16:10:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 16:10:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674524.1049510 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZdz-0006IS-4i; Thu, 01 Feb 2024 16:10:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674524.1049510; Thu, 01 Feb 2024 16:10:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZdz-0006IL-1z; Thu, 01 Feb 2024 16:10:27 +0000
Received: by outflank-mailman (input) for mailman id 674524;
 Thu, 01 Feb 2024 16:10:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Zkz=JK=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rVZdx-0006IF-KU
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 16:10:25 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 67de5e1c-c11c-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 17:10:24 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-33b1b21c020so260087f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 08:10:24 -0800 (PST)
Received: from [10.95.167.50] (54-240-197-233.amazon.com. [54.240.197.233])
 by smtp.gmail.com with ESMTPSA id
 a15-20020a056000100f00b0033aee3bfac5sm10679554wrx.16.2024.02.01.08.10.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 08:10:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67de5e1c-c11c-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706803823; x=1707408623; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:references:cc:to
         :content-language:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=yvgKt6vyIvLQ9cao7dV112oZw/spOoZQuBNAwiuVdb8=;
        b=iurUL+SnHD4LNJg78ueat7VSRM0I27B6HsnDT4SEJIvxH/PEv8RIELZIYHnuUvGBaU
         YFFiBb/Z/xNcS+qdHxOKYff2cnRIrmRdrLNqENNExQZduFsVAXq2hRLO22h+4p7itKVn
         5MlB1xjVGSBfXa48azbKEjfHtDp0tgga+bdFgxemtZWQZifD+Gw4ytr7T4TCO6/SKDN6
         mFIiK1NjcQA89/n+p/9H4cQ28JYYdqheFlGQyO05Lk2AF2u65qXE86p5xpJg1fArF/3P
         O2U/z0SuXL2wER7QRyS66Ewg6uwXZLFzSLDKPrbV/Zfh2MI6YqNyt+d4ueB76Uc6PTcs
         amoQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706803823; x=1707408623;
        h=content-transfer-encoding:in-reply-to:organization:references:cc:to
         :content-language:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=yvgKt6vyIvLQ9cao7dV112oZw/spOoZQuBNAwiuVdb8=;
        b=XaiKaafrnVbYZY6CA9euQViexG+uGRvYyct9tw5kc3bq3txlnD04uh9rgsyMMPjMnd
         Ad8BPqCKssprXAP1FC7gYl7rzrGceldlwN0UOYneiFKfV0mGGhQlW3+fGVOn0ljz/hbG
         oL3rk4lw0sF8yqHMDBKoj5T+Ly4QhxeONJMAC6pwWTKfmB3AN6HP2WS/ciryTd/Gm4Cj
         CN4/n3ZgBLiIJEvReITa79OnMtVoAO18kHlB1c1AXLCMs0GPegdv0k/XIVHptV/G699t
         Bhwxwck2ZqHLba2D+f5tGVzb0Gg/rR1aOvbwtc5IDpOQwpMMW7uxgjVN/qk7hNR3xJWv
         Mu7g==
X-Gm-Message-State: AOJu0Yy+BVmpc+Uxt/lp+seeBQ2gUYq4mLDXg8juS+fn6ZqyQMFfiVuN
	U7NDeONnhjhvNauhRcqaTyTRQjhKgSRDS3JNNFi6M6tvOICuuxln
X-Google-Smtp-Source: AGHT+IEJ1m79KnyLHr6GvYPcIVi1Yn5A5Nr4MDBCO78RLOXhhcFiwoz/B1H/xeAutDcnO1f9DXmivw==
X-Received: by 2002:a5d:6b81:0:b0:33a:fec8:e46a with SMTP id n1-20020a5d6b81000000b0033afec8e46amr4267477wrx.11.1706803823477;
        Thu, 01 Feb 2024 08:10:23 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCViOqOOszIhKkGMgLBmqB92mrQNRdR7OnPceOL0Z9LSVRHf4dJA9gJwG7JBTlRP94BQ48gsy/QsdPjZgPY6BhSE176aU/sPB+whQerq9g1p2xjSj4t+4gumT/sTyp1gCIf9yoShBiM=
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <383c1d4a-175f-4e23-b884-b9d9c1dce4da@xen.org>
Date: Thu, 1 Feb 2024 16:10:19 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH net] xen-netback: properly sync TX responses
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>
Cc: Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <980c6c3d-e10e-4459-8565-e8fbde122f00@suse.com>
Organization: Xen Project
In-Reply-To: <980c6c3d-e10e-4459-8565-e8fbde122f00@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 29/01/2024 13:03, Jan Beulich wrote:
> Invoking the make_tx_response() / push_tx_responses() pair with no lock
> held would be acceptable only if all such invocations happened from the
> same context (NAPI instance or dealloc thread). Since this isn't the
> case, and since the interface "spec" also doesn't demand that multicast
> operations may only be performed with no in-flight transmits,
> MCAST_{ADD,DEL} processing also needs to acquire the response lock
> around the invocations.
> 
> To prevent similar mistakes going forward, "downgrade" the present
> functions to private helpers of just the two remaining ones using them
> directly, with no forward declarations anymore. This involves renaming
> what so far was make_tx_response(), for the new function of that name
> to serve the new (wrapper) purpose.
> 
> While there,
> - constify the txp parameters,
> - correct xenvif_idx_release()'s status parameter's type,
> - rename {,_}make_tx_response()'s status parameters for consistency with
>    xenvif_idx_release()'s.
> 
> Fixes: 210c34dcd8d9 ("xen-netback: add support for multicast control")
> Cc: stable@vger.kernel.org
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> Of course this could be split into two or even more separate changes,
> but I think these transformations are best done all in one go.
> 
> It remains questionable whether push_tx_responses() really needs
> invoking after every single _make_tx_response().
> 
> MCAST_{ADD,DEL} are odd also from another perspective: They're supposed
> to come with "dummy requests", with the comment in the public header
> leaving open what that means.

IIRC the only reference I had at the time was Solaris code... I don't 
really there being any documentation of the feature. I think that 
https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/xen/io/xnb.c 
is probably similar to the code I looked at to determine what the 
Solaris frontend expected. So I think 'dummy' means 'ignored'.

> Netback doesn't check dummy-ness (e.g.
> size being zero). Furthermore the description in the public header
> doesn't really make clear that there's a restriction of one such "extra"
> per dummy request. Yet the way xenvif_get_extras() works precludes
> multiple ADDs or multiple DELs in a single dummy request (only the last
> one would be honored afaict). While the way xenvif_tx_build_gops() works
> precludes an ADD and a DEL coming together in a single dummy request
> (the DEL would be ignored).

It appears the Solaris backend never coped with multiple extra_info so 
what the 'correct' semantic would be is unclear.

Anyway...

Reviewed-by: Paul Durrant <paul@xen.org>

> 
> --- a/drivers/net/xen-netback/netback.c
> +++ b/drivers/net/xen-netback/netback.c
> @@ -104,13 +104,12 @@ bool provides_xdp_headroom = true;
>   module_param(provides_xdp_headroom, bool, 0644);
>   
>   static void xenvif_idx_release(struct xenvif_queue *queue, u16 pending_idx,
> -			       u8 status);
> +			       s8 status);
>   
>   static void make_tx_response(struct xenvif_queue *queue,
> -			     struct xen_netif_tx_request *txp,
> +			     const struct xen_netif_tx_request *txp,
>   			     unsigned int extra_count,
> -			     s8       st);
> -static void push_tx_responses(struct xenvif_queue *queue);
> +			     s8 status);
>   
>   static void xenvif_idx_unmap(struct xenvif_queue *queue, u16 pending_idx);
>   
> @@ -208,13 +207,9 @@ static void xenvif_tx_err(struct xenvif_
>   			  unsigned int extra_count, RING_IDX end)
>   {
>   	RING_IDX cons = queue->tx.req_cons;
> -	unsigned long flags;
>   
>   	do {
> -		spin_lock_irqsave(&queue->response_lock, flags);
>   		make_tx_response(queue, txp, extra_count, XEN_NETIF_RSP_ERROR);
> -		push_tx_responses(queue);
> -		spin_unlock_irqrestore(&queue->response_lock, flags);
>   		if (cons == end)
>   			break;
>   		RING_COPY_REQUEST(&queue->tx, cons++, txp);
> @@ -465,12 +460,7 @@ static void xenvif_get_requests(struct x
>   	for (shinfo->nr_frags = 0; nr_slots > 0 && shinfo->nr_frags < MAX_SKB_FRAGS;
>   	     nr_slots--) {
>   		if (unlikely(!txp->size)) {
> -			unsigned long flags;
> -
> -			spin_lock_irqsave(&queue->response_lock, flags);
>   			make_tx_response(queue, txp, 0, XEN_NETIF_RSP_OKAY);
> -			push_tx_responses(queue);
> -			spin_unlock_irqrestore(&queue->response_lock, flags);
>   			++txp;
>   			continue;
>   		}
> @@ -496,14 +486,8 @@ static void xenvif_get_requests(struct x
>   
>   		for (shinfo->nr_frags = 0; shinfo->nr_frags < nr_slots; ++txp) {
>   			if (unlikely(!txp->size)) {
> -				unsigned long flags;
> -
> -				spin_lock_irqsave(&queue->response_lock, flags);
>   				make_tx_response(queue, txp, 0,
>   						 XEN_NETIF_RSP_OKAY);
> -				push_tx_responses(queue);
> -				spin_unlock_irqrestore(&queue->response_lock,
> -						       flags);
>   				continue;
>   			}
>   
> @@ -995,7 +979,6 @@ static void xenvif_tx_build_gops(struct
>   					 (ret == 0) ?
>   					 XEN_NETIF_RSP_OKAY :
>   					 XEN_NETIF_RSP_ERROR);
> -			push_tx_responses(queue);
>   			continue;
>   		}
>   
> @@ -1007,7 +990,6 @@ static void xenvif_tx_build_gops(struct
>   
>   			make_tx_response(queue, &txreq, extra_count,
>   					 XEN_NETIF_RSP_OKAY);
> -			push_tx_responses(queue);
>   			continue;
>   		}
>   
> @@ -1433,8 +1415,35 @@ int xenvif_tx_action(struct xenvif_queue
>   	return work_done;
>   }
>   
> +static void _make_tx_response(struct xenvif_queue *queue,
> +			     const struct xen_netif_tx_request *txp,
> +			     unsigned int extra_count,
> +			     s8 status)
> +{
> +	RING_IDX i = queue->tx.rsp_prod_pvt;
> +	struct xen_netif_tx_response *resp;
> +
> +	resp = RING_GET_RESPONSE(&queue->tx, i);
> +	resp->id     = txp->id;
> +	resp->status = status;
> +
> +	while (extra_count-- != 0)
> +		RING_GET_RESPONSE(&queue->tx, ++i)->status = XEN_NETIF_RSP_NULL;
> +
> +	queue->tx.rsp_prod_pvt = ++i;
> +}
> +
> +static void push_tx_responses(struct xenvif_queue *queue)
> +{
> +	int notify;
> +
> +	RING_PUSH_RESPONSES_AND_CHECK_NOTIFY(&queue->tx, notify);
> +	if (notify)
> +		notify_remote_via_irq(queue->tx_irq);
> +}
> +
>   static void xenvif_idx_release(struct xenvif_queue *queue, u16 pending_idx,
> -			       u8 status)
> +			       s8 status)
>   {
>   	struct pending_tx_info *pending_tx_info;
>   	pending_ring_idx_t index;
> @@ -1444,8 +1453,8 @@ static void xenvif_idx_release(struct xe
>   
>   	spin_lock_irqsave(&queue->response_lock, flags);
>   
> -	make_tx_response(queue, &pending_tx_info->req,
> -			 pending_tx_info->extra_count, status);
> +	_make_tx_response(queue, &pending_tx_info->req,
> +			  pending_tx_info->extra_count, status);
>   
>   	/* Release the pending index before pusing the Tx response so
>   	 * its available before a new Tx request is pushed by the
> @@ -1459,32 +1468,19 @@ static void xenvif_idx_release(struct xe
>   	spin_unlock_irqrestore(&queue->response_lock, flags);
>   }
>   
> -
>   static void make_tx_response(struct xenvif_queue *queue,
> -			     struct xen_netif_tx_request *txp,
> +			     const struct xen_netif_tx_request *txp,
>   			     unsigned int extra_count,
> -			     s8       st)
> +			     s8 status)
>   {
> -	RING_IDX i = queue->tx.rsp_prod_pvt;
> -	struct xen_netif_tx_response *resp;
> -
> -	resp = RING_GET_RESPONSE(&queue->tx, i);
> -	resp->id     = txp->id;
> -	resp->status = st;
> -
> -	while (extra_count-- != 0)
> -		RING_GET_RESPONSE(&queue->tx, ++i)->status = XEN_NETIF_RSP_NULL;
> +	unsigned long flags;
>   
> -	queue->tx.rsp_prod_pvt = ++i;
> -}
> +	spin_lock_irqsave(&queue->response_lock, flags);
>   
> -static void push_tx_responses(struct xenvif_queue *queue)
> -{
> -	int notify;
> +	_make_tx_response(queue, txp, extra_count, status);
> +	push_tx_responses(queue);
>   
> -	RING_PUSH_RESPONSES_AND_CHECK_NOTIFY(&queue->tx, notify);
> -	if (notify)
> -		notify_remote_via_irq(queue->tx_irq);
> +	spin_unlock_irqrestore(&queue->response_lock, flags);
>   }
>   
>   static void xenvif_idx_unmap(struct xenvif_queue *queue, u16 pending_idx)



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 16:15:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 16:15:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674528.1049520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZjD-0007VX-MP; Thu, 01 Feb 2024 16:15:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674528.1049520; Thu, 01 Feb 2024 16:15:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZjD-0007VQ-Io; Thu, 01 Feb 2024 16:15:51 +0000
Received: by outflank-mailman (input) for mailman id 674528;
 Thu, 01 Feb 2024 16:15:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=N1rn=JK=actia.se=john.ernberg@srs-se1.protection.inumbo.net>)
 id 1rVZjC-0007VK-71
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 16:15:50 +0000
Received: from mail.actia.se (mail.actia.se [212.181.117.226])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2942478f-c11d-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 17:15:49 +0100 (CET)
Received: from S036ANL.actianordic.se (10.12.31.117) by S036ANL.actianordic.se
 (10.12.31.117) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 1 Feb
 2024 17:15:47 +0100
Received: from S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69]) by
 S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69%4]) with mapi id
 15.01.2507.035; Thu, 1 Feb 2024 17:15:47 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2942478f-c11d-11ee-8a43-1f161083a0e0
From: John Ernberg <john.ernberg@actia.se>
To: Peng Fan <peng.fan@nxp.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
CC: Jonas Blixt <jonas.blixt@actia.se>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Topic: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Index: AQHaVDuqVQ/kgxLZgUK/m8/hE0ez5bDzxxsAgAA1KwCAANPCAIAAypGA
Date: Thu, 1 Feb 2024 16:15:47 +0000
Message-ID: <71234f8d-c6e1-4294-88ca-22682fd222f5@actia.se>
References: <20240131114952.305805-1-john.ernberg@actia.se>
 <20240131114952.305805-2-john.ernberg@actia.se>
 <a2f726f5-df4c-4d15-90af-7d59c0f1f513@xen.org>
 <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se>
 <DU0PR04MB941760D4A8972440B5312B2088432@DU0PR04MB9417.eurprd04.prod.outlook.com>
In-Reply-To: <DU0PR04MB941760D4A8972440B5312B2088432@DU0PR04MB9417.eurprd04.prod.outlook.com>
Accept-Language: en-US, sv-SE
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.12.12.30]
x-esetresult: clean, is OK
x-esetid: 37303A2958D72955667264
Content-Type: text/plain; charset="utf-8"
Content-ID: <43B2996CBED84A4DA2BBEE67688887BE@actia.se>
Content-Transfer-Encoding: base64
MIME-Version: 1.0

SGkgUGVuZywNCg0KT24gMi8xLzI0IDA1OjEwLCBQZW5nIEZhbiB3cm90ZToNCj4+IENjOiBKb25h
cyBCbGl4dCA8am9uYXMuYmxpeHRAYWN0aWEuc2U+OyB4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVj
dC5vcmcNCj4+IFN1YmplY3Q6IFJlOiBbUEFUQ0ggMS8yXSB4ZW4vYXJtOiBBZGQgaW14OHF7bSx4
fSBwbGF0Zm9ybSBnbHVlDQo+Pg0KPj4gSGkgSnVsaWVuLA0KPj4NCj4+IE9uIDEvMzEvMjQgMTM6
MjIsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4+PiBIaSwNCj4+Pg0KPj4+IE9uIDMxLzAxLzIwMjQg
MTE6NTAsIEpvaG4gRXJuYmVyZyB3cm90ZToNCj4+Pj4gV2hlbiB1c2luZyBMaW51eCBmb3IgZG9t
MCB0aGVyZSBhcmUgYSBidW5jaCBvZiBkcml2ZXJzIHRoYXQgbmVlZCB0bw0KPj4+PiBkbyBTTUMg
U0lQIGNhbGxzIGludG8gdGhlIFBTQ0kgcHJvdmlkZXIgdG8gZW5hYmxlIGNlcnRhaW4gaGFyZHdh
cmUNCj4+Pj4gYml0cyBsaWtlIHRoZSB3YXRjaGRvZy4NCj4+Pg0KPj4+IERvIHlvdSBrbm93IHdo
aWNoIHByb3RvY29sIHRoaXMgaXMgdW5kZXIgdGhlIGhvb2QuIElzIHRoaXMgU0NNST8NCj4+DQo+
PiBJIHRoaW5rIEkgY29uZnVzZWQgbXlzZWxmIGhlcmUgd2hlbiBJIHdyb3RlIHRoZSBjb21taXQg
bG9nLg0KPj4NCj4+IFRoZSBFTDMgY29kZSBpbiBvdXIgY2FzZSBpcyBBVEYsIGFuZCBpdCBkb2Vz
IG5vdCBhcHBlYXIgdG8gYmUgU0NNSSwgbm9yIFBTQ0kuDQo+PiBUaGUgcmVnaXN0ZXIgdXNhZ2Ug
b2YgdGhlc2UgU01DIFNJUCBjYWxscyBhcmUgYXMgZm9sbG93czoNCj4+IGEwIC0gc2VydmljZQ0K
Pj4gYTEgLSBmdW5jdGlvbg0KPj4gYTItYTcgLSBhcmdzDQo+Pg0KPj4gSW4gQVRGIHRoZSBoYW5k
bGVyIGlzIGRlY2xhcmVkIGFzIGEgcnVudGltZSBzZXJ2aWNlLg0KPj4NCj4+IFdvdWxkIHRoZSBh
cHByb3ByaWF0ZSBjb21tbWl0IG1lc3NhZ2UgaGVyZSBiZSBzb21ldGhpbmcgYWxvbmcgdGhlIGxp
bmVzDQo+PiBvZiBiZWxvdz8NCj4+ICIiIg0KPj4gV2hlbiB1c2luZyBMaW51eCBmb3IgZG9tMCB0
aGVyZSBhcmUgYSBidW5jaCBvZiBkcml2ZXJzIHRoYXQgbmVlZCB0byBkbw0KPj4gU01DDQo+PiBT
SVAgY2FsbHMgaW50byB0aGUgZmlybXdhcmUgdG8gZW5hYmxlIGNlcnRhaW4gaGFyZHdhcmUgYml0
cyBsaWtlIHRoZSB3YXRjaGRvZy4NCj4+ICIiIg0KPj4+DQo+Pj4+DQo+Pj4+IFByb3ZpZGUgYSBi
YXNpYyBwbGF0Zm9ybSBnbHVlIHRoYXQgaW1wbGVtZW50cyB0aGUgbmVlZGVkIFNNQyBmb3J3YXJk
aW5nLg0KPj4+Pg0KPj4+PiBTaWduZWQtb2ZmLWJ5OiBKb2huIEVybmJlcmcgPGpvaG4uZXJuYmVy
Z0BhY3RpYS5zZT4NCj4+Pj4gLS0tDQo+Pj4+IE5PVEU6IFRoaXMgaXMgYmFzZWQgb24gY29kZSBm
b3VuZCBpbiBOWFAgWGVuIHRyZWUgbG9jYXRlZCBoZXJlOg0KPj4+PiBodHRwczovL2V1cjAxLnNh
ZmVsaW5rcy5wcm90ZWN0aW9uLm91dGxvb2suY29tLz91cmw9aHR0cHMlM0ElMkYlMkZnaXQNCj4+
Pj4gaHViLmNvbSUyRm54cC1pbXglMkZpbXgteGVuJTJGYmxvYiUyRmxmLQ0KPj4gNS4xMC55XzQu
MTMlMkZ4ZW4lMkZhcmNoJTJGYXINCj4+Pj4NCj4+IG0lMkZwbGF0Zm9ybXMlMkZpbXg4cW0uYyZk
YXRhPTA1JTdDMDIlN0NwZW5nLmZhbiU0MG54cC5jb20lN0MNCj4+IDU3M2I1OTlhDQo+Pj4+DQo+
PiA0YjQxNDNjZWNhMWQwOGRjMjI3MWU1YmUlN0M2ODZlYTFkM2JjMmI0YzZmYTkyY2Q5OWM1YzMw
MTYzNSU3Qw0KPj4gMCU3QzAlNw0KPj4+Pg0KPj4gQzYzODQyMzExOTc3NzYwMTU0OCU3Q1Vua25v
d24lN0NUV0ZwYkdac2IzZDhleUpXSWpvaU1DNHdMakENCj4+IHdNREFpTENKUUkNCj4+Pj4NCj4+
IGpvaVYybHVNeklpTENKQlRpSTZJazFoYVd3aUxDSlhWQ0k2TW4wJTNEJTdDMCU3QyU3QyU3QyZz
ZGF0YT1aTw0KPj4gMFRYakw2DQo+Pj4+IGcwVzdUSVpvOHg4bFROQlhFWlclMkJETmNMUG5kV2xF
ZjVEMkElM0QmcmVzZXJ2ZWQ9MA0KPj4+DQo+Pj4gQW55dGhpbmcgYWZ0ZXIgLS0tIHdpbGwgYmUg
cmVtb3ZlZCB3aGlsZSBhcHBsaWVkIHRvIHRoZSB0aHJlZS4gSSB0aGluaw0KPj4+IHRoaXMgTk9U
RSBzaG91bGQgYmUgd3JpdHRlbiBkb3duIGluIHRoZSBjb21taXQgbWVzc2FnZS4NCj4+DQo+PiBB
Y2suDQo+Pj4NCj4+PiBZb3UgYWxzbyBwb3NzaWJseSB3YW50IGEgc2lnbmVkLW9mZi1ieSBmcm9t
IFBlbmcgYXMgdGhpcyBpcyBoaXMgY29kZS4NCj4+DQo+PiBAUGVuZzogTWF5IEkgYWRkIGEgc2ln
bi1vZmYgZnJvbSB5b3U/DQo+IA0KPiBZZWFoLiBZb3UgY291bGQgYWRkIG15IHNpZ24gb2ZmLg0K
DQpHcmVhdCwgdGhhbmtzIQ0KPiANCj4+Pg0KPj4+Pg0KPj4+PiAgwqAgeGVuL2FyY2gvYXJtL3Bs
YXRmb3Jtcy9NYWtlZmlsZSB8wqAgMSArDQo+Pj4+ICDCoCB4ZW4vYXJjaC9hcm0vcGxhdGZvcm1z
L2lteDhxbS5jIHwgNjUNCj4+Pj4gKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrDQo+
Pj4+ICDCoCAyIGZpbGVzIGNoYW5nZWQsIDY2IGluc2VydGlvbnMoKykNCj4+Pj4gIMKgIGNyZWF0
ZSBtb2RlIDEwMDY0NCB4ZW4vYXJjaC9hcm0vcGxhdGZvcm1zL2lteDhxbS5jDQo+Pj4+DQo+Pj4+
IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGxhdGZvcm1zL01ha2VmaWxlDQo+Pj4+IGIveGVu
L2FyY2gvYXJtL3BsYXRmb3Jtcy9NYWtlZmlsZSBpbmRleCA4NjMyZjQxMTVmLi5iZWM2ZTU1ZDFm
DQo+PiAxMDA2NDQNCj4+Pj4gLS0tIGEveGVuL2FyY2gvYXJtL3BsYXRmb3Jtcy9NYWtlZmlsZQ0K
Pj4+PiArKysgYi94ZW4vYXJjaC9hcm0vcGxhdGZvcm1zL01ha2VmaWxlDQo+Pj4+IEBAIC05LDUg
KzksNiBAQCBvYmotJChDT05GSUdfQUxMX1BMQVQpwqDCoCArPSBzdW54aS5vDQo+Pj4+ICDCoCBv
YmotJChDT05GSUdfQUxMNjRfUExBVCkgKz0gdGh1bmRlcngubw0KPj4+PiAgwqAgb2JqLSQoQ09O
RklHX0FMTDY0X1BMQVQpICs9IHhnZW5lLXN0b3JtLm8NCj4+Pj4gIMKgIG9iai0kKENPTkZJR19B
TEw2NF9QTEFUKSArPSBicmNtLXJhc3BiZXJyeS1waS5vDQo+Pj4+ICtvYmotJChDT05GSUdfQUxM
NjRfUExBVCkgKz0gaW14OHFtLm8NCj4+Pj4gIMKgIG9iai0kKENPTkZJR19NUFNPQ19QTEFURk9S
TSnCoCArPSB4aWxpbngtenlucW1wLm8NCj4+Pj4gIMKgIG9iai0kKENPTkZJR19NUFNPQ19QTEFU
Rk9STSnCoCArPSB4aWxpbngtenlucW1wLWVlbWkubyBkaWZmIC0tZ2l0DQo+Pj4+IGEveGVuL2Fy
Y2gvYXJtL3BsYXRmb3Jtcy9pbXg4cW0uYw0KPj4gYi94ZW4vYXJjaC9hcm0vcGxhdGZvcm1zL2lt
eDhxbS5jDQo+Pj4+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0IGluZGV4IDAwMDAwMDAwMDAuLmE5Y2Q5
YzM2MTUNCj4+Pj4gLS0tIC9kZXYvbnVsbA0KPj4+PiArKysgYi94ZW4vYXJjaC9hcm0vcGxhdGZv
cm1zL2lteDhxbS5jDQo+Pj4+IEBAIC0wLDAgKzEsNjUgQEANCj4+Pj4gKy8qIFNQRFgtTGljZW5z
ZS1JZGVudGlmaWVyOiBHUEwtMi4wLW9yLWxhdGVyICovDQo+Pj4+ICsvKg0KPj4+PiArICogeGVu
L2FyY2gvYXJtL3BsYXRmb3Jtcy9pbXg4cW0uYw0KPj4+PiArICoNCj4+Pj4gKyAqIGkuTVggOFFN
IHNldHVwDQo+Pj4+ICsgKg0KPj4+PiArICogQ29weXJpZ2h0IChjKSAyMDE2IEZyZWVzY2FsZSBJ
bmMuDQo+Pj4+ICsgKiBDb3B5cmlnaHQgMjAxOC0yMDE5IE5YUA0KPj4+PiArICoNCj4+Pj4gKyAq
DQo+Pj4+ICsgKiBQZW5nIEZhbiA8cGVuZy5mYW5AbnhwLmNvbT4NCj4+Pj4gKyAqLw0KPj4+PiAr
DQo+Pj4+ICsjaW5jbHVkZSA8YXNtL3BsYXRmb3JtLmg+DQo+Pj4+ICsjaW5jbHVkZSA8YXNtL3Nt
Y2NjLmg+DQo+Pj4+ICsNCj4+Pj4gK3N0YXRpYyBjb25zdCBjaGFyICogY29uc3QgaW14OHFtX2R0
X2NvbXBhdFtdIF9faW5pdGNvbnN0ID0gew0KPj4+PiArwqDCoMKgICJmc2wsaW14OHFtIiwNCj4+
Pj4gK8KgwqDCoCAiZnNsLGlteDhxeHAiLA0KPj4+PiArwqDCoMKgIE5VTEwNCj4+Pj4gK307DQo+
Pj4+ICsNCj4+Pj4gK3N0YXRpYyBib29sIGlteDhxbV9zbWMoc3RydWN0IGNwdV91c2VyX3JlZ3Mg
KnJlZ3MpIHsNCj4+Pg0KPj4+IFlvdXIgaW1wbGVtZW50YXRpb24gYmVsb3cgd2lsbCBub3Qgb25s
eSBmb3J3YXJkIFNNQyBmb3IgZG9tMCBidXQgYWxzbw0KPj4+IGZvciBhbnkgbm9uLXRydXN0ZWQg
ZG9tYWlucy4gSGF2ZSB5b3UgaW52ZXN0aWdhdGVkIHRoYXQgYWxsIHRoZSBTSVANCj4+PiBjYWxs
cyBhcmUgc2FmZSB0byBiZSBjYWxsZWQgYnkgYW55b25lPw0KPj4NCj4+IFdlIHVzZSBwdXJlIHZp
cnR1YWxpemVkIGRvbVVzLCBzbyB3ZSBkbyBub3QgZXhwZWN0IGFueSBjYWxscyB0byB0aGlzIFNN
Qw0KPj4gaW50ZXJmYWNlIGZyb20gdGhlIGd1ZXN0LiBJJ2xsIGxpbWl0IGl0IHRvIGRvbTAuDQo+
IA0KPiBXb3VsZCB5b3UgbWluZCB0byBzaGFyZSB3aGF0IGZlYXR1cmVzIGFyZSBzdXBwb3J0ZWQg
aW4geW91ciBEb21VPw0KPiANCj4gUHVyZSB2aXJ0dWFsaXplZCwgeW91IHVzaW5nIHhlbiBwdiBv
ciB2aXJ0aW8/DQoNCk5vdCBhdCBhbGwuIFdlJ3JlIGZvcndhcmRpbmcgYmxvY2sgYW5kIG5ldHdv
cmtpbmcgKGFuZCBhZGRpdGlvbmFsbHkgDQp1c2JpcCBvdmVyIG5ldHdvcmtpbmcgZm9yIFVTQiBz
aGFyaW5nKSwgdmlhIHRoZSBYZW4gUFYgZHJpdmVycy4NCj4gDQo+IFRoYW5rcywNCj4gUGVuZy4N
Cj4gDQoNClRoYW5rcyEgLy8gSm9obiBFcm5iZXJn


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 16:17:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 16:17:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674531.1049530 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZkS-0008Bi-VA; Thu, 01 Feb 2024 16:17:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674531.1049530; Thu, 01 Feb 2024 16:17:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZkS-0008Bb-S9; Thu, 01 Feb 2024 16:17:08 +0000
Received: by outflank-mailman (input) for mailman id 674531;
 Thu, 01 Feb 2024 16:17:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=N1rn=JK=actia.se=john.ernberg@srs-se1.protection.inumbo.net>)
 id 1rVZkR-0008BT-8f
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 16:17:07 +0000
Received: from mail.actia.se (mail.actia.se [212.181.117.226])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 579aba30-c11d-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 17:17:06 +0100 (CET)
Received: from S036ANL.actianordic.se (10.12.31.117) by S036ANL.actianordic.se
 (10.12.31.117) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 1 Feb
 2024 17:17:06 +0100
Received: from S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69]) by
 S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69%4]) with mapi id
 15.01.2507.035; Thu, 1 Feb 2024 17:17:06 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 579aba30-c11d-11ee-8a43-1f161083a0e0
From: John Ernberg <john.ernberg@actia.se>
To: Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
	"Michal Orzel" <michal.orzel@amd.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Peng Fan <peng.fan@nxp.com>
CC: Jonas Blixt <jonas.blixt@actia.se>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Topic: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Index: AQHaVDuqVQ/kgxLZgUK/m8/hE0ez5bDzxxsAgAA1KwCAAVx5AIAAQjeA
Date: Thu, 1 Feb 2024 16:17:05 +0000
Message-ID: <a265ea2d-9b5f-4726-9395-448b1b669839@actia.se>
References: <20240131114952.305805-1-john.ernberg@actia.se>
 <20240131114952.305805-2-john.ernberg@actia.se>
 <a2f726f5-df4c-4d15-90af-7d59c0f1f513@xen.org>
 <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se>
 <167f0c7a-e037-446c-82f8-2584e35a7af1@xen.org>
In-Reply-To: <167f0c7a-e037-446c-82f8-2584e35a7af1@xen.org>
Accept-Language: en-US, sv-SE
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.12.12.30]
x-esetresult: clean, is OK
x-esetid: 37303A2958D72955667264
Content-Type: text/plain; charset="utf-8"
Content-ID: <F6C5E070CDA15F49BC5E8D4D4FFE4477@actia.se>
Content-Transfer-Encoding: base64
MIME-Version: 1.0

T24gMi8xLzI0IDEzOjIwLCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+IA0KPiANCj4gT24gMzEvMDEv
MjAyNCAxNTozMiwgSm9obiBFcm5iZXJnIHdyb3RlOg0KPj4gSGkgSnVsaWVuLA0KPiANCj4gSGkg
Sm9obiwNCj4gDQo+PiBPbiAxLzMxLzI0IDEzOjIyLCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+Pj4g
SGksDQo+Pj4NCj4+PiBPbiAzMS8wMS8yMDI0IDExOjUwLCBKb2huIEVybmJlcmcgd3JvdGU6DQo+
Pj4+IFdoZW4gdXNpbmcgTGludXggZm9yIGRvbTAgdGhlcmUgYXJlIGEgYnVuY2ggb2YgZHJpdmVy
cyB0aGF0IG5lZWQgdG8gZG8NCj4+Pj4gU01DDQo+Pj4+IFNJUCBjYWxscyBpbnRvIHRoZSBQU0NJ
IHByb3ZpZGVyIHRvIGVuYWJsZSBjZXJ0YWluIGhhcmR3YXJlIGJpdHMgDQo+Pj4+IGxpa2UgdGhl
DQo+Pj4+IHdhdGNoZG9nLg0KPj4+DQo+Pj4gRG8geW91IGtub3cgd2hpY2ggcHJvdG9jb2wgdGhp
cyBpcyB1bmRlciB0aGUgaG9vZC4gSXMgdGhpcyBTQ01JPw0KPj4NCj4+IEkgdGhpbmsgSSBjb25m
dXNlZCBteXNlbGYgaGVyZSB3aGVuIEkgd3JvdGUgdGhlIGNvbW1pdCBsb2cuDQo+Pg0KPj4gVGhl
IEVMMyBjb2RlIGluIG91ciBjYXNlIGlzIEFURiwgYW5kIGl0IGRvZXMgbm90IGFwcGVhciB0byBi
ZSBTQ01JLCBub3INCj4+IFBTQ0kuIFRoZSByZWdpc3RlciB1c2FnZSBvZiB0aGVzZSBTTUMgU0lQ
IGNhbGxzIGFyZSBhcyBmb2xsb3dzOg0KPj4gYTAgLSBzZXJ2aWNlDQo+PiBhMSAtIGZ1bmN0aW9u
DQo+PiBhMi1hNyAtIGFyZ3MNCj4+DQo+PiBJbiBBVEYgdGhlIGhhbmRsZXIgaXMgZGVjbGFyZWQg
YXMgYSBydW50aW1lIHNlcnZpY2UuDQo+Pg0KPj4gV291bGQgdGhlIGFwcHJvcHJpYXRlIGNvbW1t
aXQgbWVzc2FnZSBoZXJlIGJlIHNvbWV0aGluZyBhbG9uZyB0aGUgbGluZXMNCj4+IG9mIGJlbG93
Pw0KPj4gIiIiDQo+PiBXaGVuIHVzaW5nIExpbnV4IGZvciBkb20wIHRoZXJlIGFyZSBhIGJ1bmNo
IG9mIGRyaXZlcnMgdGhhdCBuZWVkIHRvIA0KPj4gZG/CoMKgIFNNQw0KPj4gU0lQIGNhbGxzIGlu
dG8gdGhlIGZpcm13YXJlIHRvIGVuYWJsZSBjZXJ0YWluIGhhcmR3YXJlIGJpdHMgbGlrZSB0aGUN
Cj4+IHdhdGNoZG9nLg0KPj4gIiIiDQo+IA0KPiBJdCByZWFkcyBiZXR0ZXIgdGhhbmtzLg0KPiAN
Cj4gWy4uLl0NCj4gDQo+Pj4gQnV0IGV2ZW4gaWYgd2UgcmVzdHJpY3QgdG8gZG9tMCwgaGF2ZSB5
b3UgY2hlY2tlZCB0aGF0IG5vbmUgb2YgdGhlIFNNQ3MNCj4+PiB1c2UgYnVmZmVycz8NCj4+IEkg
aGF2ZW4ndCBmb3VuZCBhbnkgc3VjaCBpbnN0YW5jZXMgaW4gdGhlIExpbnV4IGtlcm5lbCB3aGVy
ZSBhIGJ1ZmZlciBpcw0KPj4gdXNlZC4gQWRkaW5nIGEgY2FsbCBmaWx0ZXJpbmcgbGlrZSBzdWdn
ZXN0ZWQgYmVsb3cgYWRkaXRpb25zIG9mIHN1Y2gNCj4+IGZ1bmN0aW9ucyBjYW4gYmUgZGlzY292
ZXJlZCBhbmQgYWRhcHRlZCBmb3IgaWYgdGhleSB3b3VsZCBzaG93IHVwIGxhdGVyLg0KPj4+DQo+
Pj4gUmF0aGVyIHRoYW4gcHJvdmlkaW5nIGEgYmxhbmtldCBmb3J3YXJkLCB0byBtZSBpdCBzb3Vu
ZHMgbW9yZSBsaWtlIHlvdQ0KPj4+IHdhbnQgdG8gcHJvdmlkZSBhbiBhbGxvd2xpc3Qgb2YgdGhl
IFNNQ3MuIFRoaXMgaXMgbW9yZSBmdXR1cmVwcm9vZiBhbmQNCj4+PiBhdm9pZCB0aGUgcmlzayB0
byBleHBvc2UgdW5zYWZlIFNNQ3MgdG8gYW55IGRvbWFpbi4NCj4+Pg0KPj4+IEZvciBhbiBleGFt
cGxlLCB5b3UgY2FuIGhhdmUgYSBsb29rIGF0IHRoZSBFRU1JIG1lZGlhdG9yIGZvciBYaWxpbngu
DQo+Pg0KPj4gQWNrLiBEbyB5b3UgcHJlZmVyIHRvIHNlZSBvbmx5IG9uIFNNQ0NDIHNlcnZpY2Ug
bGV2ZWwgb3IgYWxzbyBvbg0KPj4gZnVuY3Rpb24gbGV2ZWw/IChhMSByZWdpc3RlciwgcGVyIGRl
c2NyaXB0aW9uIGVhcmxpZXIpDQo+IA0KPiBJIGFtIG5vdCBzdXJlLiBJdCB3aWxsIGRlcGVuZCBv
biB3aGV0aGVyIGl0IGlzIGNvcnJlY3QgdG8gZXhwb3NlICphbGwqIA0KPiB0aGUgZnVuY3Rpb25z
IHdpdGhpbiBhIHNlcnZpY2UgbGV2ZWwgYW5kIHRoZXkgaGF2ZSB0aGUgc2FtZSBmb3JtYXQuDQo+
IA0KPiBJZiB5b3UgY2FuJ3QgZ3VhcmFudGVlIHRoYXQsIHRoZW4geW91IHdpbGwgbW9zdCBsaWtl
bHkgbmVlZCB0byBhbGxvd2xpc3QgDQo+IGF0IHRoZSBmdW5jdGlvbiBsZXZlbC4NCj4gDQo+IEFs
c28sIGRvIHlvdSBoYXZlIGEgc3BlYyBpbiBoYW5kIHRoYXQgd291bGQgaGVscCB0byB1bmRlcnN0
YW5kIHdoaWNoIA0KPiBzZXJ2aWNlL2Z1bmN0aW9uIGlzIGltcGxlbWVudGVkIHZpYSB0aG9zZSBT
TUNzPw0KDQpJIGRvbid0IGhhdmUgdGhlIHNwZWMgdW5mb3J0dW5hdGVseSwgYnV0IEkgd2lsbCBh
ZGQgYSBmaWx0ZXIgb24gYm90aCANCnNlcnZpY2UgYW5kIGZ1bmN0aW9uIGZvciBWMiBhbmQgd2Un
bGwgdGFrZSBpdCBmcm9tIHRoZXJlLg0KPiANCj4gQ2hlZXJzLA0KPiANCg0KVGhhbmtzISAvLyBK
b2huIEVybmJlcmc=


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 16:20:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 16:20:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674535.1049541 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZnq-0002Us-F8; Thu, 01 Feb 2024 16:20:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674535.1049541; Thu, 01 Feb 2024 16:20:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZnq-0002Ul-B7; Thu, 01 Feb 2024 16:20:38 +0000
Received: by outflank-mailman (input) for mailman id 674535;
 Thu, 01 Feb 2024 16:20:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Qo6T=JK=kaod.org=clg@ozlabs.org>)
 id 1rVZno-0002UK-Bz
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 16:20:36 +0000
Received: from gandalf.ozlabs.org (mail.ozlabs.org [2404:9400:2221:ea00::3])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d0e4465b-c11d-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 17:20:33 +0100 (CET)
Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76])
 by gandalf.ozlabs.org (Postfix) with ESMTP id 4TQkf30k39z4wcW;
 Fri,  2 Feb 2024 03:20:27 +1100 (AEDT)
Received: from authenticated.ozlabs.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by mail.ozlabs.org (Postfix) with ESMTPSA id 4TQkdZ2ZRRz4wcN;
 Fri,  2 Feb 2024 03:20:02 +1100 (AEDT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0e4465b-c11d-11ee-98f5-efadbce2ee36
Message-ID: <fcd5cf8a-68f7-45c4-a09f-e7870ed7ffa1@kaod.org>
Date: Thu, 1 Feb 2024 17:19:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 22/47] hw/arm/aspeed: use qemu_configure_nic_device()
Content-Language: en-US
To: David Woodhouse <dwmw2@infradead.org>, qemu-devel@nongnu.org
Cc: Richard Henderson <richard.henderson@linaro.org>,
 Beniamino Galvani <b.galvani@gmail.com>,
 Peter Maydell <peter.maydell@linaro.org>,
 Strahinja Jankovic <strahinja.p.jankovic@gmail.com>,
 Niek Linnenbank <nieklinnenbank@gmail.com>,
 Andrew Jeffery <andrew@codeconstruct.com.au>, Joel Stanley <joel@jms.id.au>,
 Igor Mitsyanko <i.mitsyanko@gmail.com>,
 Jean-Christophe Dubois <jcd@tribudubois.net>,
 Andrey Smirnov <andrew.smirnov@gmail.com>,
 =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>,
 Rob Herring <robh@kernel.org>, Subbaraya Sundeep <sundeep.lkml@gmail.com>,
 Jan Kiszka <jan.kiszka@web.de>, Tyrone Ting <kfting@nuvoton.com>,
 Hao Wu <wuhaotsh@google.com>, Radoslaw Biernacki <rad@semihalf.com>,
 Leif Lindholm <quic_llindhol@quicinc.com>,
 Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>,
 "Edgar E. Iglesias" <edgar.iglesias@gmail.com>,
 Alistair Francis <alistair@alistair23.me>, Helge Deller <deller@gmx.de>,
 Paolo Bonzini <pbonzini@redhat.com>, Eduardo Habkost <eduardo@habkost.net>,
 "Michael S. Tsirkin" <mst@redhat.com>,
 Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Song Gao
 <gaosong@loongson.cn>, Thomas Huth <huth@tuxfamily.org>,
 Laurent Vivier <laurent@vivier.eu>, Huacai Chen <chenhuacai@kernel.org>,
 Jiaxun Yang <jiaxun.yang@flygoat.com>,
 =?UTF-8?Q?Herv=C3=A9_Poussineau?= <hpoussin@reactos.org>,
 Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>,
 Aurelien Jarno <aurelien@aurel32.net>, Jason Wang <jasowang@redhat.com>,
 Jia Liu <proljc@gmail.com>, Stafford Horne <shorne@gmail.com>,
 Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>,
 Nicholas Piggin <npiggin@gmail.com>,
 Daniel Henrique Barboza <danielhb413@gmail.com>,
 David Gibson <david@gibson.dropbear.id.au>,
 Harsh Prateek Bora <harshpb@linux.ibm.com>, Bin Meng
 <bin.meng@windriver.com>, Palmer Dabbelt <palmer@dabbelt.com>,
 Weiwei Li <liwei1518@gmail.com>, Liu Zhiwei <zhiwei_liu@linux.alibaba.com>,
 Halil Pasic <pasic@linux.ibm.com>,
 Christian Borntraeger <borntraeger@linux.ibm.com>,
 Eric Farman <farman@linux.ibm.com>, David Hildenbrand <david@redhat.com>,
 Ilya Leoshkevich <iii@linux.ibm.com>,
 Yoshinori Sato <ysato@users.sourceforge.jp>,
 Magnus Damm <magnus.damm@gmail.com>, Artyom Tarasenko <atar4qemu@gmail.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 Max Filippov <jcmvbkbc@gmail.com>, qemu-arm@nongnu.org, qemu-ppc@nongnu.org,
 qemu-riscv@nongnu.org, qemu-s390x@nongnu.org,
 xen-devel@lists.xenproject.org, David Woodhouse <dwmw@amazon.co.uk>
References: <20240126173228.394202-1-dwmw2@infradead.org>
 <20240126173228.394202-23-dwmw2@infradead.org>
From: =?UTF-8?Q?C=C3=A9dric_Le_Goater?= <clg@kaod.org>
In-Reply-To: <20240126173228.394202-23-dwmw2@infradead.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 1/26/24 18:24, David Woodhouse wrote:
> From: David Woodhouse <dwmw@amazon.co.uk>
> 
> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> Acked-by: Cédric Le Goater <clg@kaod.org>

and

Reviewed-by: Cédric Le Goater <clg@kaod.org>

Thanks,

C.


> ---
>   hw/arm/aspeed.c | 9 ++++-----
>   1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/arm/aspeed.c b/hw/arm/aspeed.c
> index cc59176563..bed5e4f40b 100644
> --- a/hw/arm/aspeed.c
> +++ b/hw/arm/aspeed.c
> @@ -356,7 +356,6 @@ static void aspeed_machine_init(MachineState *machine)
>       AspeedMachineClass *amc = ASPEED_MACHINE_GET_CLASS(machine);
>       AspeedSoCClass *sc;
>       int i;
> -    NICInfo *nd = &nd_table[0];
>   
>       bmc->soc = ASPEED_SOC(object_new(amc->soc_name));
>       object_property_add_child(OBJECT(machine), "soc", OBJECT(bmc->soc));
> @@ -371,10 +370,10 @@ static void aspeed_machine_init(MachineState *machine)
>                                &error_fatal);
>   
>       for (i = 0; i < sc->macs_num; i++) {
> -        if ((amc->macs_mask & (1 << i)) && nd->used) {
> -            qemu_check_nic_model(nd, TYPE_FTGMAC100);
> -            qdev_set_nic_properties(DEVICE(&bmc->soc->ftgmac100[i]), nd);
> -            nd++;
> +        if ((amc->macs_mask & (1 << i)) &&
> +            !qemu_configure_nic_device(DEVICE(&bmc->soc->ftgmac100[i]),
> +                                       true, NULL)) {
> +            break; /* No configs left; stop asking */
>           }
>       }
>   



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 16:30:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 16:30:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674542.1049550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZxH-0005mb-H0; Thu, 01 Feb 2024 16:30:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674542.1049550; Thu, 01 Feb 2024 16:30:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVZxH-0005mU-Bd; Thu, 01 Feb 2024 16:30:23 +0000
Received: by outflank-mailman (input) for mailman id 674542;
 Thu, 01 Feb 2024 16:30:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVZxG-0005mK-4s; Thu, 01 Feb 2024 16:30:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVZxG-0004oj-17; Thu, 01 Feb 2024 16:30:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVZxF-0002oH-GD; Thu, 01 Feb 2024 16:30:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVZxF-00081n-Fe; Thu, 01 Feb 2024 16:30:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=9YpjMy9I01TpieSQ6jpM/g+5JXXXsb5ycykMpNe1yVg=; b=qfOlRYyqIB+n4yGMqklx1lcUgd
	94MqpPANMC/+mxsBqJuDWkG//M9f2/xSRB6+j14DWgStexaFAjPP/ijuDi3SbfmWjEPf//0uE1Erq
	EXcCcDL+8dbG3QR3b/W1AbqnjcUo4w8mr+mXvnItFuPvaUhXSTMy6mv9nWRSgTA8PcgM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184545-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184545: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-xsm:xen-boot:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=6764c317b6bb91bd806ef79adf6d9c0e428b191e
X-Osstest-Versions-That:
    linux=861c0981648f5b64c86fd028ee622096eb7af05a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 01 Feb 2024 16:30:21 +0000

flight 184545 linux-linus real [real]
flight 184550 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184545/
http://logs.test-lab.xenproject.org/osstest/logs/184550/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-xsm       8 xen-boot            fail pass in 184550-retest

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm     15 migrate-support-check fail in 184550 never pass
 test-arm64-arm64-xl-xsm 16 saverestore-support-check fail in 184550 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184525
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184525
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184525
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184525
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184525
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184525
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184525
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184525
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                6764c317b6bb91bd806ef79adf6d9c0e428b191e
baseline version:
 linux                861c0981648f5b64c86fd028ee622096eb7af05a

Last test of basis   184525  2024-01-30 04:10:43 Z    2 days
Failing since        184534  2024-01-30 23:45:05 Z    1 days    2 attempts
Testing same since   184545  2024-01-31 22:37:40 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexander Gordeev <agordeev@linux.ibm.com>
  Anders Roxell <anders.roxell@linaro.org>
  Arthur Grillo <arthurgrillo@riseup.net>
  Chunhai Guo <guochunhai@vivo.com>
  Colin Ian King <colin.i.king@gmail.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  David Gow <davidgow@google.com>
  Gao Xiang <hsiangkao@linux.alibaba.com>
  Jingbo Xu <jefflexu@linux.alibaba.com>
  Joe Lawrence <joe.lawrence@redhat.com>
  Kees Cook <keescook@chromium.org>
  Li RongQing <lirongqing@baidu.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Marco Pagani <marpagan@redhat.com>
  Marcos Paulo de Souza <mpdesouza@suse.com>
  Mark Brown <broonie@kernel.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
  Michael Kelley <mhklinux@outlook.com>
  Michael S. Tsirkin <mst@redhat.com>
  Ming Lei <ming.lei@redhat.com>
  Miroslav Benes <mbenes@suse.cz>
  Rae Moar <rmoar@google.com>
  Richard Fitzgerald <rf@opensource.cirrus.com>
  Sathya Prakash Veerichetty <safhya.prakash@broadcom.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Su Hui <suhui@nfschina.com>
  Tyrel Datwyler <tyreld@linux.ibm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   861c0981648f..6764c317b6bb  6764c317b6bb91bd806ef79adf6d9c0e428b191e -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 16:40:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 16:40:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674549.1049560 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVa76-0008KE-CI; Thu, 01 Feb 2024 16:40:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674549.1049560; Thu, 01 Feb 2024 16:40:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVa76-0008K7-9E; Thu, 01 Feb 2024 16:40:32 +0000
Received: by outflank-mailman (input) for mailman id 674549;
 Thu, 01 Feb 2024 16:40:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IdfB=JK=kernel.org=patchwork-bot+netdevbpf@srs-se1.protection.inumbo.net>)
 id 1rVa74-0008Jx-Nm
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 16:40:30 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9b2579f0-c120-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 17:40:29 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id C963C61F9D;
 Thu,  1 Feb 2024 16:40:27 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPS id ED7CAC43399;
 Thu,  1 Feb 2024 16:40:26 +0000 (UTC)
Received: from aws-us-west-2-korg-oddjob-1.ci.codeaurora.org
 (localhost.localdomain [127.0.0.1])
 by aws-us-west-2-korg-oddjob-1.ci.codeaurora.org (Postfix) with ESMTP id
 D639DD8C97B; Thu,  1 Feb 2024 16:40:26 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b2579f0-c120-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1706805627;
	bh=tUGHjQXZc4kPtsp3kBrK8stZgKY4d0Sjj2hXbau2+tw=;
	h=Subject:From:Date:References:In-Reply-To:To:Cc:From;
	b=VfJZ0QH3JSTZZIgOoOIZVp+7MvzJdERSMVdJrd7Nz3wf6BV+idXCMDx4tOcaTmvat
	 fxmPMtckf+EgnLTFPSQgP+8PFLohotnQa8Y73hUITcqIffQ818riTNTFzmzMBDfb+x
	 lqRcmaJib3eQlM6Bs/nkFhdsotdm+9+pU5XwzS5R1XNhZejmtoY2KQX1zi5IBVNhjs
	 frvKdkUdX+klk0Nz3k+cWF4y52HNIuv9tO2vavaZpWzr9Fi04wQMA4jDiQ8OSgFZwz
	 fxk1x53xSh9Yg8N0jUCPDUPEC3/Gx+Hxv1sJYy5QKNuCs+V3fQxgU4dnlaTpml7b4V
	 4VibnUIC34LDg==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Subject: Re: [PATCH net] xen-netback: properly sync TX responses
From: patchwork-bot+netdevbpf@kernel.org
Message-Id: 
 <170680562687.32005.2375881723958020276.git-patchwork-notify@kernel.org>
Date: Thu, 01 Feb 2024 16:40:26 +0000
References: <980c6c3d-e10e-4459-8565-e8fbde122f00@suse.com>
In-Reply-To: <980c6c3d-e10e-4459-8565-e8fbde122f00@suse.com>
To: Jan Beulich <JBeulich@suse.com>
Cc: netdev@vger.kernel.org, wl@xen.org, paul@xen.org,
 xen-devel@lists.xenproject.org

Hello:

This patch was applied to netdev/net.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Mon, 29 Jan 2024 14:03:08 +0100 you wrote:
> Invoking the make_tx_response() / push_tx_responses() pair with no lock
> held would be acceptable only if all such invocations happened from the
> same context (NAPI instance or dealloc thread). Since this isn't the
> case, and since the interface "spec" also doesn't demand that multicast
> operations may only be performed with no in-flight transmits,
> MCAST_{ADD,DEL} processing also needs to acquire the response lock
> around the invocations.
> 
> [...]

Here is the summary with links:
  - [net] xen-netback: properly sync TX responses
    https://git.kernel.org/netdev/net/c/7b55984c96ff

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html




From xen-devel-bounces@lists.xenproject.org Thu Feb 01 16:51:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 16:51:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674553.1049569 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaHx-0002gt-BP; Thu, 01 Feb 2024 16:51:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674553.1049569; Thu, 01 Feb 2024 16:51:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaHx-0002gm-8a; Thu, 01 Feb 2024 16:51:45 +0000
Received: by outflank-mailman (input) for mailman id 674553;
 Thu, 01 Feb 2024 16:51:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=biZV=JK=gmail.com=euidzero@srs-se1.protection.inumbo.net>)
 id 1rVaHw-0002fT-DY
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 16:51:44 +0000
Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com
 [2607:f8b0:4864:20::435])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2c4ffda2-c122-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 17:51:42 +0100 (CET)
Received: by mail-pf1-x435.google.com with SMTP id
 d2e1a72fcca58-6da6b0eb2d4so886546b3a.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 08:51:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c4ffda2-c122-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706806300; x=1707411100; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=pHgO2N+l6o7ljfKXbnAdMBmgfTYXAOxMzwYRNwJOjdc=;
        b=Y+1uv5kVKuqautBdz9r0hgNodRR5rLugkj7q2Wpiio4EbLwxvLHVmTNp/Q4ZJ/zWkY
         y58Cz3mjc+fX9tpeQXM+7RjhCl2q+fk+Jjm2p6ibDDOHWcx76v3ymtQmcr8Loc8Tr9eP
         K81l9EcsSfDoz1kllEsCTkuc60dKO5eggVJWOFzr7d1ZEJ4WkQEvW2x7I3DzrUU1ASOt
         6lhNenZbjcOLVrbcVbbsY5AQ+Ol53FLe0GIoo2XCajukKInM+RcI1dpcFgxPZBE95NQz
         MMeQwulMo3JYy1VsqwuM5Y8gCeBdqaJ8whlIninJX7Z0VxLyPYDfPgrIkonPWxjsKfwl
         cSIQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706806300; x=1707411100;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=pHgO2N+l6o7ljfKXbnAdMBmgfTYXAOxMzwYRNwJOjdc=;
        b=qd0q5UY4GUH2gMBH/MIDfRKX4J5aDKNh1o7J4x8xnIyi4GI7vVl4dyqVCdr4LFXkZm
         nukRwE3eXoELLus6u6Y0ueZzcY9u1xFXFxiOy8iMlTcFrSQfuSuw+kv/+jQX/7k+w01G
         +fnVV9DmBURTeyBl5+HEnKmC1DAwwPtSI5Cik8cUbbWtm4zYaORaJqw78wI0hluWB6wz
         JRVLITT/IfMMzIQOvbwYya3S6VepdaRQX9GDpbJErpn0ZMzdPxEKyVa/IdPDQGp70xGQ
         /Snx2UMw+L8q+f00T7Y8SdYWcfHyXvXZIB7bJgktJNhlfnh+cl1HsHKRXVsWBFZwP1zn
         aAIQ==
X-Gm-Message-State: AOJu0YxqfgztfK+O1x+C2oTRtzlm9Pk8+i27DlY9LklzjaCoQjJGogbf
	U4g9sjOLGQl5d5yaNkiEfw8mrhOK5CgAimqu5dGJBek1/cBJubLLwYfAlQQLShLmNY6HtowaQfq
	biNTJiXkuLAhFaL03JHO+Fp56zF8=
X-Google-Smtp-Source: AGHT+IHaHkuZaTIehF3ZXnPaTmy4AEtFZTyIYcHEjMM+enOgb7CO+yg+2maj9afol4UoDChrKOKvieU9a+/6Yw/rdbU=
X-Received: by 2002:a05:6a00:181c:b0:6dd:c870:3f82 with SMTP id
 y28-20020a056a00181c00b006ddc8703f82mr7032844pfa.4.1706806300407; Thu, 01 Feb
 2024 08:51:40 -0800 (PST)
MIME-Version: 1.0
References: <CAKm-Umas=5=JzooRzPHxUDigKpFK3Ze7cQcG8eR5kWgn-d_7fQ@mail.gmail.com>
 <CAKm-UmaUZ=rj-F3RvMoR7H3OvBHm1iy4PTRZAjyi=MQUVTuYVg@mail.gmail.com>
 <CAKm-Umat77MTh6=GEuH+bf1WRX+_B4-N5hY4xwADALfhj4QCPA@mail.gmail.com>
 <6ba9568a-ff8e-48ad-907f-74303e04d3a2@suse.com> <CAKm-UmZVoUhidEV15Xk+J_XN5a2GsBqMbXL9wqdFdoAFf_u5AA@mail.gmail.com>
 <bcbceba7-42ac-45be-80ab-e3dfe7741ec6@suse.com> <2eafe45b-fd97-472c-a173-459d6ff0b957@suse.com>
 <CAKm-Umbs2D7NHFE+NW2xKOu7-AZhFpH4uzE5QTuQusnGA_eNWA@mail.gmail.com>
 <f539144c-885a-461c-a506-bdb73e626a65@suse.com> <CAKm-UmY126AfdGhWcZ3s9vwN9+ksVRRFEhOu0ZFMoDvxrqOibw@mail.gmail.com>
 <CAKm-UmYt3iV8zOhSmtqMGhi_8T93c_nCbnefs4m3UC+3UABqeQ@mail.gmail.com>
 <CAKm-UmY-KXEAtBagikW4Jvp=SFXtmEg8P62pHfSo3Hr2s-0_-A@mail.gmail.com>
 <CAKm-UmYbMCfXc1Ny0=qfB+UaLSXV9oEHZiSgS=mwKMwmOFGVrQ@mail.gmail.com>
 <77c1c05d-a0a0-4292-9257-9b7fbebee0e3@suse.com> <2859ad22-8714-4628-8142-fc77fc2072c3@amd.com>
 <CAKm-UmZpyGkWXugYTJqU+qqVDyCFEKghtd=NTr2wK5EMCeL9Ww@mail.gmail.com>
 <214585d5-689d-4ba6-bd48-359428a7ed8f@amd.com> <CAKm-UmZoKwre8-G793VqRNFCmzAti1o-0Kp3ZyV_Z5cc0YNiKw@mail.gmail.com>
In-Reply-To: <CAKm-UmZoKwre8-G793VqRNFCmzAti1o-0Kp3ZyV_Z5cc0YNiKw@mail.gmail.com>
From: =?UTF-8?Q?S=C3=A9bastien_Chaumat?= <euidzero@gmail.com>
Date: Thu, 1 Feb 2024 17:51:28 +0100
Message-ID: <CAKm-Umb=kGFqc5je9E3bbfQ0bcbZeY_Ntv5JDmO-vXj3N0MvPg@mail.gmail.com>
Subject: Re: [BUG]i2c_hid_acpi broken with 4.17.2 on Framework Laptop 13 AMD
To: Mario Limonciello <mario.limonciello@amd.com>
Cc: Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xenproject.org, 
	Juergen Gross <jgross@suse.com>, 
	=?UTF-8?Q?Marek_Marczykowski=2DG=C3=B3recki?= <marmarek@invisiblethingslab.com>
Content-Type: multipart/alternative; boundary="0000000000003c7d2d061054cf25"

--0000000000003c7d2d061054cf25
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Le jeu. 1 f=C3=A9vr. 2024 =C3=A0 13:30, S=C3=A9bastien Chaumat <euidzero@gm=
ail.com> a
=C3=A9crit :

> I spotted the following warning for IRQ7 (along with IRQ6 and 10)
>
> [    0.686073] fedora kernel: __irq_set_trigger: genirq: No set_type
> function for IRQ 7 (IR-IO-APIC)
>
> This comes from kernel/irq/manage.c
>
>
> int __irq_set_trigger(struct irq_desc *desc, unsigned long flags)
> {
> struct irq_chip *chip =3D desc->irq_data.chip;
> int ret, unmask =3D 0;
>
> if (!chip || !chip->irq_set_type) {
> /*
> * IRQF_TRIGGER_* but the PIC does not support multiple
> * flow-types?
> */
> pr_debug("No set_type function for IRQ %d (%s)\n",
> irq_desc_get_irq(desc),
> chip ? (chip->name ? : "unknown") : "unknown");
> return 0;
> }
> Could this have a role in the IRQ misconfiguration by xen ?
>

 Things are getting even weirder :

xen 4.18.1-pre (vanilla, no patching of pci_xen_initial_domain()), kernel
6.8.0-rc2) :

checking /sys/kernel/irq/7 :

actions: pinctrl_amd
chip_name: xen-pirq
hwirq:
name: ioapic-edge
per_cpu_count: 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0
type: edge
wakeup: disabled

now checking  xen with xl debug-key i

(XEN)    IRQ:   7 vec:51 IO-APIC-level   status=3D030 aff:{13}/{13}
in-flight=3D0 d0:  7(---)
(XEN)     IRQ  7 Vec 81:
(XEN)       Apic 0x00, Pin  7: vec=3D51 delivery=3DFixed dest=3DP status=3D=
1
polarity=3D1 irr=3D1 trig=3DL mask=3D0 dest_id:0000510d

So even after dom0 is set up, the kernel considers  IRQ7 is of type edge
while xen has it registered as IO-APIC-level.

--0000000000003c7d2d061054cf25
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br><div><div><br></div><div><br></div><d=
iv><br></div><div><br><br></div></div><div><br></div></div><br><div class=
=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">Le=C2=A0jeu. 1 f=C3=
=A9vr. 2024 =C3=A0=C2=A013:30, S=C3=A9bastien Chaumat &lt;<a href=3D"mailto=
:euidzero@gmail.com" target=3D"_blank">euidzero@gmail.com</a>&gt; a =C3=A9c=
rit=C2=A0:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div =
dir=3D"ltr"><div>I spotted the following warning for IRQ7 (along with IRQ6 =
and 10)<br></div><div><br></div><div><div>[ =C2=A0 =C2=A00.686073] fedora k=
ernel: __irq_set_trigger: genirq: No set_type function for IRQ 7 (IR-IO-API=
C)</div></div><div><br></div><div>This comes from kernel/irq/manage.c</div>=
<div><br></div><div><br></div>int __irq_set_trigger(struct irq_desc *desc, =
unsigned long flags)<br>{<br>struct irq_chip *chip =3D desc-&gt;irq_data.ch=
ip;<br>int ret, unmask =3D 0;<br><br>if (!chip || !chip-&gt;irq_set_type) {=
<br>/*<br>* IRQF_TRIGGER_* but the PIC does not support multiple<br>* flow-=
types?<br>*/<br>pr_debug(&quot;No set_type function for IRQ %d (%s)\n&quot;=
,<br>irq_desc_get_irq(desc),<br>chip ? (chip-&gt;name ? : &quot;unknown&quo=
t;) : &quot;unknown&quot;);<br>return 0;<br>}<div>Could this have a role in=
 the IRQ misconfiguration by xen ?</div><div><div></div></div></div></block=
quote><div><br></div><div>=C2=A0Things are getting even weirder : <br></div=
><div><br></div><div>xen 4.18.1-pre (vanilla, no patching of  pci_xen_initi=
al_domain()), kernel 6.8.0-rc2) :<br><div><br></div><div>checking /sys/kern=
el/irq/7 :</div><div><br></div><div>actions: pinctrl_amd<br>chip_name: xen-=
pirq<br>hwirq: <br>name: ioapic-edge<br>per_cpu_count: 0,0,0,0,0,0,1,0,0,0,=
0,0,0,0,0,0<br>type: edge<br>wakeup: disabled</div><div><br></div><div>now =
checking=C2=A0 xen with xl debug-key i</div><div><br></div><div>(XEN) =C2=
=A0 =C2=A0IRQ: =C2=A0 7 vec:51 IO-APIC-level =C2=A0 status=3D030 aff:{13}/{=
13} in-flight=3D0 d0: =C2=A07(---)</div><div><div>(XEN) =C2=A0 =C2=A0 IRQ =
=C2=A07 Vec 81:<br>(XEN) =C2=A0 =C2=A0 =C2=A0 Apic 0x00, Pin =C2=A07: vec=
=3D51 delivery=3DFixed dest=3DP status=3D1 polarity=3D1 irr=3D1 trig=3DL ma=
sk=3D0 dest_id:0000510d</div><div><br></div><div>So even after dom0 is set =
up, the kernel considers=C2=A0 IRQ7 is of type edge while xen has it regist=
ered as IO-APIC-level.</div><div><br></div><div><br></div></div></div></div=
>
</div>

--0000000000003c7d2d061054cf25--


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 16:55:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 16:55:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674557.1049580 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaLT-0003RI-T3; Thu, 01 Feb 2024 16:55:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674557.1049580; Thu, 01 Feb 2024 16:55:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaLT-0003RB-QN; Thu, 01 Feb 2024 16:55:23 +0000
Received: by outflank-mailman (input) for mailman id 674557;
 Thu, 01 Feb 2024 16:55:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=dNcs=JK=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rVaLS-0003R5-G8
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 16:55:22 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aefe8879-c122-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 17:55:20 +0100 (CET)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2cf1288097aso15575301fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 08:55:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aefe8879-c122-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1706806520; x=1707411320; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=RBRoiFhzBwl5VFAP4C+wSNH9W94y9ZIzJo2QLJgsqTM=;
        b=acovdA5UYl4Y7QJUzxQiTAjAWKSE0+ZCpH8ElH6jluz6jwtPOwvKD1Cmeufg48qqHF
         1G1Coz7v58k/sVHmqPLOgZmvRi0PTKGDlI+Arzwx+RJ1ao+wvllsMmiMcBizI36yrb2o
         0kCWG5UbFrrODDID7h0ulWWJHCIcsYrgXsOB8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706806520; x=1707411320;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=RBRoiFhzBwl5VFAP4C+wSNH9W94y9ZIzJo2QLJgsqTM=;
        b=XmtKB6yrP6chDq1twnJ7niwk2pjhMojicbD5bQk/SVAKX27fmEk6/NLVAD8wxcjFNc
         SbiHIRb9fY7/lpVtq43DJI4PrQreq4om/Ms5KCxHfHkIcskdxHYT6bvQ6hlKd8myOABF
         DF3wQFtt6cyRqFqi+uFD6n8JK/6B3EVfi+sJvnlYA4N+9mllnNGUbD/4Xa4lENX1iUUZ
         af7nvVdDyHBofJ11LpgdOkUe4nK+/e7fxrPAi87+efoaw88HmiQV58IQblHLZy5yVmr2
         e6suKcvGSg/Baj0NK1cmbHdmQ6ysrnNKncO3pkoRcF2f0hXIWdVS7fEWiAWA/Q695A+E
         r4kg==
X-Gm-Message-State: AOJu0YyCbm0XJRcZQLKs76MVsc3mMlvJqQpYY/b9noQDTzGg2KIyy0g0
	FCGk6QZU/zvV+X5hizorGMPfsuTOESa+Z3mfdjQBgmo2CLwK38/1aj4Byj6X4/LJP0Q/N9f/7WX
	ohSPEQJbzUWvLqB9efZwgDs690H5aYKgSuxOSkw==
X-Google-Smtp-Source: AGHT+IFMO8fChOZ+UHjzvt1AawyangJW+kQdJkUS6kcWOuEeYgsZ8hD7DskmrNqhrWCEIJFKNgf1UYcIsGa+df8Rytk=
X-Received: by 2002:a2e:b0f7:0:b0:2d0:7dab:d45e with SMTP id
 h23-20020a2eb0f7000000b002d07dabd45emr959137ljl.9.1706806519868; Thu, 01 Feb
 2024 08:55:19 -0800 (PST)
MIME-Version: 1.0
References: <aea51d2e-5da8-4da8-954f-2ee2a43be73e@vates.tech> <1be8f6f1-f977-4743-bd26-362ada960dc6@vates.tech>
In-Reply-To: <1be8f6f1-f977-4743-bd26-362ada960dc6@vates.tech>
From: George Dunlap <george.dunlap@cloud.com>
Date: Thu, 1 Feb 2024 16:55:08 +0000
Message-ID: <CA+zSX=Yu_Epi6fge71iQNkecYOn92XO4KzwRtQNfrchtMaCxzw@mail.gmail.com>
Subject: Re: Next steps for Rust guest agent
To: Yann Dirson <yann.dirson@vates.tech>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Lunfan Zhang <Lunfan.Zhang@citrix.com>, Xihuan Yang <xihuan.yang@citrix.com>, 
	dpsmith@apertussolutions.com, marmarek@invisiblethingslab.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Jan 11, 2024 at 12:27=E2=80=AFPM Yann Dirson <yann.dirson@vates.tec=
h> wrote:
> > - what should be the criteria to advertise it as official Xenproject
> > guest agent ?
>
> What do people think here?

As we discussed at the community call, I think that we should
basically set a date at which we consider this the official Xen
Project guest agent.  Anyone who wants to have input can give it
before then.  Then once you guys think it's ready, we can start to
"market" it to the distros.

Shall we say 29 February, 8 weeks from now?

 -George


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 16:57:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 16:57:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674560.1049590 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaNC-0003yW-7o; Thu, 01 Feb 2024 16:57:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674560.1049590; Thu, 01 Feb 2024 16:57:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaNC-0003yP-4m; Thu, 01 Feb 2024 16:57:10 +0000
Received: by outflank-mailman (input) for mailman id 674560;
 Thu, 01 Feb 2024 16:57:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rVaNA-0003yH-Qi
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 16:57:08 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVaN8-0005Gl-Px; Thu, 01 Feb 2024 16:57:06 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVaN8-0003uN-HP; Thu, 01 Feb 2024 16:57:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=qgZJCfdLDu8JYR0LsPiclE/eQBuhsWgYjnA8lKA8aOc=; b=Zg5EhqUmnKmEqg2nTWgRTCi6UC
	wTGtaWHah0fwFn+1OvePB8NU//0FkGY37MzQ+VOaDMe8Mc+eZfwssbG6LaqALzzpPesRvc44gKYgV
	T/S2Je17Ry9N4/e3LpE19WyVFrQmAR1EqD0++8rRVao7eQLHH1tWKwl5+9KjFDyBYMeg=;
Message-ID: <9e01b86e-e3d2-4eec-93ed-e8828375bb30@xen.org>
Date: Thu, 1 Feb 2024 16:57:04 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Next steps for Rust guest agent
Content-Language: en-GB
To: George Dunlap <george.dunlap@cloud.com>,
 Yann Dirson <yann.dirson@vates.tech>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Lunfan Zhang <Lunfan.Zhang@citrix.com>, Xihuan Yang
 <xihuan.yang@citrix.com>, dpsmith@apertussolutions.com,
 marmarek@invisiblethingslab.com
References: <aea51d2e-5da8-4da8-954f-2ee2a43be73e@vates.tech>
 <1be8f6f1-f977-4743-bd26-362ada960dc6@vates.tech>
 <CA+zSX=Yu_Epi6fge71iQNkecYOn92XO4KzwRtQNfrchtMaCxzw@mail.gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CA+zSX=Yu_Epi6fge71iQNkecYOn92XO4KzwRtQNfrchtMaCxzw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi George,

On 01/02/2024 16:55, George Dunlap wrote:
> On Thu, Jan 11, 2024 at 12:27 PM Yann Dirson <yann.dirson@vates.tech> wrote:
>>> - what should be the criteria to advertise it as official Xenproject
>>> guest agent ?
>>
>> What do people think here?
> 
> As we discussed at the community call, I think that we should
> basically set a date at which we consider this the official Xen
> Project guest agent.  Anyone who wants to have input can give it
> before then.  Then once you guys think it's ready, we can start to
> "market" it to the distros.

+1

> 
> Shall we say 29 February, 8 weeks from now?

This is 4 weeks away. I am fine with that, but checking this is the date 
you intended to set.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 17:02:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 17:02:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674564.1049600 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaS0-0006nL-Q2; Thu, 01 Feb 2024 17:02:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674564.1049600; Thu, 01 Feb 2024 17:02:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaS0-0006nE-Mw; Thu, 01 Feb 2024 17:02:08 +0000
Received: by outflank-mailman (input) for mailman id 674564;
 Thu, 01 Feb 2024 17:02:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=dNcs=JK=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rVaRz-0006n8-B6
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 17:02:07 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a0fa446c-c123-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 18:02:06 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-5112bd13a4fso1135889e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 09:02:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0fa446c-c123-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1706806926; x=1707411726; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=FHF+v7x8/QeBquQOP44XZ2/os7VovEFQTEi/GM6OTH4=;
        b=Wmy1rCxqHgFoAQYJEYhbAE5m3yMqIxu5GNWr6s/0uX70Tyf7VqOKSCTZU60RuaSy4G
         2nuLkTLzQGCfXKTFfh6uGnrhABSXgaf4j5Q5uF4tMt3TRhWv7tAclMqw58dQVpN94x3a
         AkGhJUPXbqaZW1qIyAIFqlHyVHh4j+oAQhGPw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706806926; x=1707411726;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=FHF+v7x8/QeBquQOP44XZ2/os7VovEFQTEi/GM6OTH4=;
        b=sM5Xu/bTexV5tcnbrMV5UBu1ihy7Hc+NjEojaNDHRKveOIgwJJQImC0259MRo+0CA6
         z+61zKamjyO9oe5e/8u7/5wzKqzw5J+uPK/30lbkRQ7m15AfMlBpd1kijgrnRLNJYPoK
         3ubzvicoXYdgy83xZSWR65oUPWa0RqmfYuveytwSibvdwiea5T2w0r47NQzV0Liy0lPw
         dOGM5ztVzK2HxMlhUVsnyvCC2zhAkT65fCQQEN78fnKLyG8a6feKPpMmr5+6hah/T/UL
         P82NB1Rq616y/rRBueAFS7pURrhPYU2JaD2Ve+/jXk/3ky5YFxGHh/As/BBChkJaFwRI
         1sSQ==
X-Gm-Message-State: AOJu0YyzX0MFIWAsFL1rmenEwIxBefTbIsdJ3dygw+VIlasV3Jrpm+qa
	L21ZtL4TaIP3W6WpbVhj5GTzyTev/F3RvWsC3xDVcMCBN+YueKb1eBX0Dl8WoMjgpy9eMLkJ3Ve
	EMDmEtu5/naGUuR4+kfn2jSGlkltxN2IfdGiMLw==
X-Google-Smtp-Source: AGHT+IGxxLinwhS09HN0SxRzAxnFm0ip6c3RZ2uSi9UQHrxxNFviyXmgu7rTp7e9YNMAxeEUr8I2tvpYdgYqHeZk/LI=
X-Received: by 2002:a05:6512:286:b0:510:22c4:fbaf with SMTP id
 j6-20020a056512028600b0051022c4fbafmr1076144lfp.13.1706806925753; Thu, 01 Feb
 2024 09:02:05 -0800 (PST)
MIME-Version: 1.0
References: <aea51d2e-5da8-4da8-954f-2ee2a43be73e@vates.tech>
 <1be8f6f1-f977-4743-bd26-362ada960dc6@vates.tech> <CA+zSX=Yu_Epi6fge71iQNkecYOn92XO4KzwRtQNfrchtMaCxzw@mail.gmail.com>
 <9e01b86e-e3d2-4eec-93ed-e8828375bb30@xen.org>
In-Reply-To: <9e01b86e-e3d2-4eec-93ed-e8828375bb30@xen.org>
From: George Dunlap <george.dunlap@cloud.com>
Date: Thu, 1 Feb 2024 17:01:54 +0000
Message-ID: <CA+zSX=aT8+4sECB+k7for6EWmR=Wjwi5q-0HzvJT0xs1p=xTsA@mail.gmail.com>
Subject: Re: Next steps for Rust guest agent
To: Julien Grall <julien@xen.org>
Cc: Yann Dirson <yann.dirson@vates.tech>, xen-devel@lists.xenproject.org, 
	Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Lunfan Zhang <Lunfan.Zhang@citrix.com>, Xihuan Yang <xihuan.yang@citrix.com>, 
	dpsmith@apertussolutions.com, marmarek@invisiblethingslab.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 1, 2024 at 4:57=E2=80=AFPM Julien Grall <julien@xen.org> wrote:
>
> Hi George,
>
> On 01/02/2024 16:55, George Dunlap wrote:
> > On Thu, Jan 11, 2024 at 12:27=E2=80=AFPM Yann Dirson <yann.dirson@vates=
.tech> wrote:
> >>> - what should be the criteria to advertise it as official Xenproject
> >>> guest agent ?
> >>
> >> What do people think here?
> >
> > As we discussed at the community call, I think that we should
> > basically set a date at which we consider this the official Xen
> > Project guest agent.  Anyone who wants to have input can give it
> > before then.  Then once you guys think it's ready, we can start to
> > "market" it to the distros.
>
> +1
>
> >
> > Shall we say 29 February, 8 weeks from now?
>
> This is 4 weeks away. I am fine with that, but checking this is the date
> you intended to set.

...

I have no idea how that 8 got there... yes, 4 weeks is what I meant to writ=
e.

 -George


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 17:02:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 17:02:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674565.1049609 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaS8-00073L-0E; Thu, 01 Feb 2024 17:02:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674565.1049609; Thu, 01 Feb 2024 17:02:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaS7-00073E-TP; Thu, 01 Feb 2024 17:02:15 +0000
Received: by outflank-mailman (input) for mailman id 674565;
 Thu, 01 Feb 2024 17:02:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cAD+=JK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rVaS5-0006n8-UR
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 17:02:13 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a5210a0e-c123-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 18:02:13 +0100 (CET)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-558f523c072so1708116a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 09:02:13 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 o11-20020a056402444b00b0055d3d1653b3sm6617362edb.31.2024.02.01.09.02.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 09:02:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5210a0e-c123-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706806932; x=1707411732; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Wz8hnYroCYqHv9y7dmqbQbjNnQAdCfNne5v071bTWRE=;
        b=L7qau/I88cftDBnfnPoIVFPpEfVzHfPyGzFntfcQNEdf70FfYlbSUEz8zBckuJVCbk
         gItcX53QfiKGUrpA4qRjiUPddySOzmFtvFypek/Ct8X4xuVdxpy9Ol0PGbKXy3sSTIr2
         dUPXb4WIhm8pegJCw9Wt0qeYl7idr1Te+jj74=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706806932; x=1707411732;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Wz8hnYroCYqHv9y7dmqbQbjNnQAdCfNne5v071bTWRE=;
        b=UTQY1fBtdqB8jiScLoLLt/A1E02nxx5a13pkf+pOhH0YmdV6VDnh/etZR2oID395S3
         FnSPesFlDGQ6sR+fzd4u6puM28wNjGNBNmUQephyVaOCKRJK9ojdFPoSE3KybXEt2wUD
         fiXC89AvVXq8jbw9p/2kirDJ1RE7BrJzwvv9skGmXvhOYMo6R1E+uhakWgDwh1V3vBpO
         V0a3yYvltdN2S6ccGprz9/SnWwAj1N51bzdZUOfLz5rL1GiCwHKoWjXGNQT0+lUcGvkr
         BkOoZ4yrILvW2unRhsrbrlM16jVSdzDPizcbgNuDQYwTt0iabYDoFe3/VpfHQ1/9ZIAn
         zPBg==
X-Gm-Message-State: AOJu0YyuVM0fuvr60th+E4PwoyZT8ZJLDDmb0kraTPs9bapF6lmNKIWj
	K6aJcyPvvK3UHLXcjvlfdxmvTDG9esw4elv69roTyUjwHrw3mK1cIzJ9f3R7buWlFoeBZ6+bAKI
	j
X-Google-Smtp-Source: AGHT+IFr5vIuvATmEU3Lizj2zwmteX8aji47igUJODCkyit0aMkH+H6HN7Dwi3X0Bl5wJUh8C0YAoA==
X-Received: by 2002:aa7:c354:0:b0:55f:6960:4935 with SMTP id j20-20020aa7c354000000b0055f69604935mr4160521edr.25.1706806932627;
        Thu, 01 Feb 2024 09:02:12 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCV9RMEgIwUIj0rcfz0UfGOs/2ICwuB5BCl4gBnTB3w4gK7fy4/e3a0ECQLXSpr0yZXm51FqwEuqN1S4M19xYSTtfARfnLKTBgmnwpnzJMKWBniBKwyJxR1CA9tK0MwZgy8+UkKPB/75QU/ycccohpvFlqUk3kFgjFjMfHMyoUZIXnVdyrP7BXWe8hbtLGkeVcuNQ9/eFSJiD9+aZps=
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	oxjo@proton.me,
	Wei Liu <wl@xen.org>,
	Paul Durrant <paul@xen.org>,
	Kevin Tian <kevin.tian@intel.com>
Subject: [PATCH 0/4] iommu/x86: fixes/improvements for unity range checks
Date: Thu,  1 Feb 2024 18:01:55 +0100
Message-ID: <20240201170159.66330-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

Patch 1 is a bugfix for AMD-Vi IVMD range type checks.

Further patches unify the IVMD/RMRR checks into a common function, and
the last patch tightens up the condition to panic when attempting to
boot on a system that has RMRR or IVMD regions over memory that Xen uses
as RAM.

Thanks, Roger.

Roger Pau Monne (4):
  amd-vi: fix IVMD memory type checks
  iommu/x86: introduce a generic IVMD/RMRR range validity helper
  iommu/vt-d: switch to common RMRR checker
  iommu/x86: make unity range checking more strict

 xen/arch/x86/include/asm/iommu.h         |  3 ++
 xen/drivers/passthrough/amd/iommu_acpi.c | 33 ++---------------
 xen/drivers/passthrough/vtd/dmar.c       | 14 ++------
 xen/drivers/passthrough/x86/iommu.c      | 46 ++++++++++++++++++++++++
 4 files changed, 55 insertions(+), 41 deletions(-)

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 17:02:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 17:02:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674566.1049614 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaS8-00076i-8K; Thu, 01 Feb 2024 17:02:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674566.1049614; Thu, 01 Feb 2024 17:02:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaS8-00075f-3y; Thu, 01 Feb 2024 17:02:16 +0000
Received: by outflank-mailman (input) for mailman id 674566;
 Thu, 01 Feb 2024 17:02:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cAD+=JK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rVaS7-0006n8-1s
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 17:02:15 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a5a8341b-c123-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 18:02:14 +0100 (CET)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-55c2cf644f3so1537824a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 09:02:14 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 h29-20020a0564020e9d00b0055cfb3f948fsm7002015eda.76.2024.02.01.09.02.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 09:02:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a5a8341b-c123-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706806933; x=1707411733; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=gGkbw6NMRmq84Zwm3toJg408+YkNymy6A8aNzx/GF0M=;
        b=WOgFGmp4wAlVZe5KSBYm4fBKBfSBdlk357RV5UIFLOIru8f2k4iKop1hFKpyyOsv+m
         9eTCf2nQTsmJtMbWPob3emXPsseqQ6Tyo163yfi79jv0fUyS3wXsaiHyRLTMkOknuix6
         rwiZdvEC0MCUlF73TCII3pjH6o4OkGBu8gGoM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706806933; x=1707411733;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=gGkbw6NMRmq84Zwm3toJg408+YkNymy6A8aNzx/GF0M=;
        b=cswMIbBaMh+zeqbZqHUXNVTRszQ443nA03LvhFWvvvdrPz+7u5FhMn9ElEF5O7XAhY
         NsqabZco3oqGMFhoIa6AZIRCFOiEIFGaw9R5ZlF363FYOotD1V+AowWPrYBABhgOx1cX
         Ynmv/s6cBqr6xeiaHQfYodbzV0oo3+lC1WmqruMjGhtNFlbA61YRdpnWcs6P8sPuzhGJ
         NCrKUcLu0pNAO8SFQj1U7rveKMimGICvyLGhEIiammNb05MdPw5zdq/oFNzGdlkDMNpY
         gVGZtX52SSktFTnMZDZoQG7kU7/+Pgn9iePeSiWp4ARmdFF5/Dh2ejC4hxr3cFDb/jFH
         J9ng==
X-Gm-Message-State: AOJu0YwBWlLuUPfCz8KVb01b9ZfVFgLOn4NJ18wED09R4Zsrs+DS+hRs
	3mAh+fFmYgXwT7bhvjaC3E+RnGzK2hdMnX7sSgZfUDXKVu57hSgiT2vzHx1C9hLTnb/r2mNRuiE
	M
X-Google-Smtp-Source: AGHT+IGIMAX9xEfzp7fazur889a6R83N+pLnidiTerN578sP+ra2lpf0shXr+EPsnqASXyDum22iXA==
X-Received: by 2002:aa7:d5c6:0:b0:55e:dcdf:50b0 with SMTP id d6-20020aa7d5c6000000b0055edcdf50b0mr3590001eds.15.1706806933771;
        Thu, 01 Feb 2024 09:02:13 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXa7/kBApDrW+9oLGL5JuXgqFjsL7REMMsbCNkqEymxKzmK6rPiWX8gjFOgJ9L8ciruSynzi2zHvP4pAUa5PQrrhSdSTpsLNI5FKIRo7qkN7zxWhxL5+OreNCiZeAN0U91XWHZnH50iqfzKV8M=
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	oxjo@proton.me
Subject: [PATCH 1/4] amd-vi: fix IVMD memory type checks
Date: Thu,  1 Feb 2024 18:01:56 +0100
Message-ID: <20240201170159.66330-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201170159.66330-1-roger.pau@citrix.com>
References: <20240201170159.66330-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The current code that parses the IVMD blocks is relaxed with regard to the
restriction that such unity regions should always fall into memory ranges
marked as reserved in the memory map.

However the type checks for the IVMD addresses are inverted, and as a result
IVMD ranges falling into RAM areas are accepted.  Note that having such ranges
in the first place is a firmware bug, as IVMD should always fall into reserved
ranges.

Fixes: ed6c77ebf0c1 ('AMD/IOMMU: check / convert IVMD ranges for being / to be reserved')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Cc: oxjo@proton.me
---
 xen/drivers/passthrough/amd/iommu_acpi.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/xen/drivers/passthrough/amd/iommu_acpi.c b/xen/drivers/passthrough/amd/iommu_acpi.c
index 2e3b83014beb..ca70f4f3ae2c 100644
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -426,9 +426,14 @@ static int __init parse_ivmd_block(const struct acpi_ivrs_memory *ivmd_block)
                 return -EIO;
             }
 
-            /* Types which won't be handed out are considered good enough. */
-            if ( !(type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
-                           RAM_TYPE_UNUSABLE)) )
+            /*
+             * Types which aren't RAM are considered good enough.
+             * Note that a page being partially RESERVED, ACPI or UNUSABLE will
+             * force Xen into assuming the whole page as having that type in
+             * practice.
+             */
+            if ( type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
+                         RAM_TYPE_UNUSABLE) )
                 continue;
 
             AMD_IOMMU_ERROR("IVMD: page at %lx can't be converted\n", addr);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 17:02:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 17:02:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674567.1049630 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaSB-0007ZD-G6; Thu, 01 Feb 2024 17:02:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674567.1049630; Thu, 01 Feb 2024 17:02:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaSB-0007Z0-Cy; Thu, 01 Feb 2024 17:02:19 +0000
Received: by outflank-mailman (input) for mailman id 674567;
 Thu, 01 Feb 2024 17:02:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cAD+=JK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rVaS9-0007Wp-Vh
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 17:02:17 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a67f37fb-c123-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 18:02:15 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a354408e6bfso393531766b.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 09:02:15 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 x13-20020aa7dacd000000b0055eb8830906sm5799854eds.85.2024.02.01.09.02.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 09:02:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a67f37fb-c123-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706806935; x=1707411735; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=LaBUruy8bPZuvsCRWTLi8TtvG3+zI8gmjxbPMfpVXBQ=;
        b=lQiuQocOtK++8BUqK7RASUFT0dAorXBb8lITr9CLEtfmLuXp2/rbXS91qk7twI26SL
         JIAcGZ05qTqkOMnc96G44qVKFmU6AKFHyje8haxM62B/2DavS9hQVSWRR5UMUiWyUeaU
         YTf2agBBCDfRsbqh3Z8x7R6BWq1q86pl4nG10=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706806935; x=1707411735;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=LaBUruy8bPZuvsCRWTLi8TtvG3+zI8gmjxbPMfpVXBQ=;
        b=ZeKhbMKzxR5HrtUofWqxSMDR4ChAgQs4BI4K61PcZeQO1Lk2YEyi7oLn393Xi3WwGp
         TmRL4voIf0gJeoPqoYTxSJeaQhn8N6nkdUXKJDGvYDrtosYYmALkFpm3ZjOCAEhKIVHf
         rOmiIBeY0r38yHQia+JLApTVtp0yV2ELJAAzfIkJCv3gM0hck3JsCBqemLhZ+CXJN5Vi
         q9VJzUije0PIfZqGlqSe7lf1fG46QMgfyPYkGDpxVcbJqNhTde8hUlNBTaliog1IY6LH
         hKvEwVpudXlvXgKk4xkVgkISWVJ+vzCYB6m5CY9iMINYxAqCQiJ7YKagzc67cgJxQ2pJ
         jn/Q==
X-Gm-Message-State: AOJu0YwCteBCGziwDdqJLCCbZQ/ql3230lBfKZiQ8UZCgG1VxeqNOAdQ
	L+/82K9Xch1H83pI86K+UhIAtX+NpJ7B8HYCMvRNkVtbfGDbHXipUNMQjkfIsGzrBh3ofOIClUf
	Z
X-Google-Smtp-Source: AGHT+IGXtQwRJDikIm3QRBpY+NsFJJ2VzaJ8sihA2KkXMbRjVgXNsY2axsutk7Ypvh3WHZBEw62lBw==
X-Received: by 2002:a17:907:20ee:b0:a36:83b6:385a with SMTP id rh14-20020a17090720ee00b00a3683b6385amr4052264ejb.7.1706806934981;
        Thu, 01 Feb 2024 09:02:14 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXz9y6qNecsuzFbhqWvYu586TWh1h6wyME4CUQvvl+rXnQoYcXO6n+vN0EpxXhLKl8Wiu3E8XwBAwUKxc2D8TZVTyBcQfdIoGy5xyisZNJMxu6cseqarQdNLbEAC101yEK4quN+35e9TF0/CLdY3xHpZRi1M43P
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>,
	Paul Durrant <paul@xen.org>
Subject: [PATCH 2/4] iommu/x86: introduce a generic IVMD/RMRR range validity helper
Date: Thu,  1 Feb 2024 18:01:57 +0100
Message-ID: <20240201170159.66330-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201170159.66330-1-roger.pau@citrix.com>
References: <20240201170159.66330-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

IVMD and RMRR ranges are functionally equivalent, and as so could use the same
validity checker.

Move the IVMD to x86 common IOMMU code and adjust the function to take a pair
of [start, end) mfn parameters.

So far only the AMD-Vi side is adjusted to use the newly introduced helper, the
VT-d side will be adjusted in a further change.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/include/asm/iommu.h         |  3 ++
 xen/drivers/passthrough/amd/iommu_acpi.c | 38 ++------------------
 xen/drivers/passthrough/x86/iommu.c      | 46 ++++++++++++++++++++++++
 3 files changed, 52 insertions(+), 35 deletions(-)

diff --git a/xen/arch/x86/include/asm/iommu.h b/xen/arch/x86/include/asm/iommu.h
index 15a848ddc329..5c7e37331aad 100644
--- a/xen/arch/x86/include/asm/iommu.h
+++ b/xen/arch/x86/include/asm/iommu.h
@@ -135,6 +135,9 @@ struct page_info *__must_check iommu_alloc_pgtable(struct domain_iommu *hd,
                                                    uint64_t contig_mask);
 void iommu_queue_free_pgtable(struct domain_iommu *hd, struct page_info *pg);
 
+/* Check [start, end) unity map range for correctness. */
+bool iommu_unity_region_ok(mfn_t start, mfn_t end);
+
 #endif /* !__ARCH_X86_IOMMU_H__ */
 /*
  * Local variables:
diff --git a/xen/drivers/passthrough/amd/iommu_acpi.c b/xen/drivers/passthrough/amd/iommu_acpi.c
index ca70f4f3ae2c..40468dbbccf3 100644
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -405,41 +405,9 @@ static int __init parse_ivmd_block(const struct acpi_ivrs_memory *ivmd_block)
         return 0;
     }
 
-    if ( !e820_all_mapped(base, limit + PAGE_SIZE, E820_RESERVED) )
-    {
-        paddr_t addr;
-
-        AMD_IOMMU_WARN("IVMD: [%lx,%lx) is not (entirely) in reserved memory\n",
-                       base, limit + PAGE_SIZE);
-
-        for ( addr = base; addr <= limit; addr += PAGE_SIZE )
-        {
-            unsigned int type = page_get_ram_type(maddr_to_mfn(addr));
-
-            if ( type == RAM_TYPE_UNKNOWN )
-            {
-                if ( e820_add_range(addr, addr + PAGE_SIZE,
-                                    E820_RESERVED) )
-                    continue;
-                AMD_IOMMU_ERROR("IVMD: page at %lx couldn't be reserved\n",
-                                addr);
-                return -EIO;
-            }
-
-            /*
-             * Types which aren't RAM are considered good enough.
-             * Note that a page being partially RESERVED, ACPI or UNUSABLE will
-             * force Xen into assuming the whole page as having that type in
-             * practice.
-             */
-            if ( type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
-                         RAM_TYPE_UNUSABLE) )
-                continue;
-
-            AMD_IOMMU_ERROR("IVMD: page at %lx can't be converted\n", addr);
-            return -EIO;
-        }
-    }
+    if ( !iommu_unity_region_ok(maddr_to_mfn(base),
+                                maddr_to_mfn(limit + PAGE_SIZE)) )
+        return -EIO;
 
     if ( ivmd_block->header.flags & ACPI_IVMD_EXCLUSION_RANGE )
         exclusion = true;
diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index c90755ff58fa..63d4cb898218 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -792,6 +792,52 @@ static int __init cf_check adjust_irq_affinities(void)
 }
 __initcall(adjust_irq_affinities);
 
+bool __init iommu_unity_region_ok(mfn_t start, mfn_t end)
+{
+    mfn_t addr;
+
+    if ( e820_all_mapped(mfn_to_maddr(start), mfn_to_maddr(end),
+                         E820_RESERVED) )
+        return true;
+
+    printk(XENLOG_WARNING "IOMMU: [%#" PRI_mfn " ,%#" PRI_mfn
+           ") is not (entirely) in reserved memory\n",
+           mfn_x(start), mfn_x(end));
+
+    for ( addr = start; !mfn_eq(addr, end); mfn_add(addr, 1) )
+    {
+        unsigned int type = page_get_ram_type(addr);
+
+        if ( type == RAM_TYPE_UNKNOWN )
+        {
+            if ( e820_add_range(mfn_to_maddr(addr),
+                                mfn_to_maddr(addr) + PAGE_SIZE, E820_RESERVED) )
+                continue;
+            printk(XENLOG_WARNING
+                   "IOMMU: page at %#" PRI_mfn " couldn't be reserved\n",
+                   mfn_x(addr));
+            return false;
+        }
+
+        /*
+         * Types which aren't RAM are considered good enough.
+         * Note that a page being partially RESERVED, ACPI or UNUSABLE will
+         * force Xen into assuming the whole page as having that type in
+         * practice.
+         */
+        if ( type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
+                     RAM_TYPE_UNUSABLE) )
+            continue;
+
+        printk(XENLOG_WARNING
+               "IOMMU: page at %#" PRI_mfn " can't be converted\n",
+               mfn_x(addr));
+        return false;
+    }
+
+    return true;
+}
+
 /*
  * Local variables:
  * mode: C
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 17:02:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 17:02:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674568.1049635 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaSB-0007c3-Rb; Thu, 01 Feb 2024 17:02:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674568.1049635; Thu, 01 Feb 2024 17:02:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaSB-0007bP-N9; Thu, 01 Feb 2024 17:02:19 +0000
Received: by outflank-mailman (input) for mailman id 674568;
 Thu, 01 Feb 2024 17:02:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cAD+=JK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rVaSA-0007Wp-Lm
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 17:02:18 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a74d3c51-c123-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 18:02:17 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a357cba4a32so159191166b.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 09:02:17 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 hw20-20020a170907a0d400b00a2b1a20e662sm7368698ejc.34.2024.02.01.09.02.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 09:02:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a74d3c51-c123-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706806936; x=1707411736; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=UloEpPKCM7G82luLru9QjRBa08XW6aQ69k/jBsglyGw=;
        b=rNbPd2z6k1d03ehIATXiBtiO+aDVtXpX4K5tQpGZmtFiq/LY33nsoVcRmKD21l4l8u
         RVHSWcClSRMdpzVLEl6yRSzyEahGvdjJMgPsEz4JJ0T7vk8igobml+QAm7bA/JVeCLmP
         UW19WIipZPnMk4+PmfysralqiLq5u9Xgz+YGE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706806936; x=1707411736;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=UloEpPKCM7G82luLru9QjRBa08XW6aQ69k/jBsglyGw=;
        b=JdNWZsxBO+Ii5htEvhm/g9K3ti/lNh1/TeEFyDtlp/c76VXQNsVdvFcHF1682yz9Fy
         0ytg4qj5QH2e9W530nE/pgjgswHrsxeFc6lup7r1ByKTFXK1aOfbLQbOJVahm9S+Rdvh
         9ZCSbQn0LSrwk2Ys2zvHFFZM2q43S5Aej4bpWuVLQnOabtEABrixhQVKmEk21Ex3KMqz
         vKw/VwCuoktGVD2erjnMQHnHf6d0Y5HF8s9jpe5t1nVubAa7rHr/g+diz5KTOlYQ472Z
         U8MUPADt7tS9WUOCob0qsZHrD2o9BnzgePXOehC68OExvNdYZuBgbsCIeMV1g8Y4RPC+
         b/dQ==
X-Gm-Message-State: AOJu0Yx2UVHkzkWxaD7J+dNFsoc99iu9nfVxYTtKr+L+52B0XM+L29Tt
	bGdN7btVXhuun3TX/xHGZJzUlr8/N7Fseh3X+2ELZXWNc6aLIVdjUOzYg7kxmzxZUUGeMLQniIm
	w
X-Google-Smtp-Source: AGHT+IE1ioEOtkHXIsPlEbxeCQ5KNW6gZONtNKo7ovJn0q0tGWZV/t+8E8s+EtazaGjUEAvRtXy5GA==
X-Received: by 2002:a17:906:3515:b0:a36:2dc:1903 with SMTP id r21-20020a170906351500b00a3602dc1903mr2272370eja.68.1706806936197;
        Thu, 01 Feb 2024 09:02:16 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXRcHbpgIR2NyN9m6tWAgLtB7nbJPyYIh6UIkvdWkRs8hEq9Bn5Xp1NNCrN4gxyPsb4MxHHLpBB+H/4soWBdQS6v6r3gQ==
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Kevin Tian <kevin.tian@intel.com>
Subject: [PATCH 3/4] iommu/vt-d: switch to common RMRR checker
Date: Thu,  1 Feb 2024 18:01:58 +0100
Message-ID: <20240201170159.66330-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201170159.66330-1-roger.pau@citrix.com>
References: <20240201170159.66330-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use the newly introduced generic unity map checker.

Also drop the message recommending the usage of iommu_inclusive_mapping: the
ranges would end up being mapped anyway even if some of the checks above
failed, regardless of whether iommu_inclusive_mapping is set.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/drivers/passthrough/vtd/dmar.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/xen/drivers/passthrough/vtd/dmar.c b/xen/drivers/passthrough/vtd/dmar.c
index 07772f178fe6..005b42706a34 100644
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -642,17 +642,9 @@ acpi_parse_one_rmrr(struct acpi_dmar_header *header)
            return -EEXIST;
        }
 
-    /* This check is here simply to detect when RMRR values are
-     * not properly represented in the system memory map and
-     * inform the user
-     */
-    if ( !e820_all_mapped(base_addr, end_addr + 1, E820_RESERVED) &&
-         !e820_all_mapped(base_addr, end_addr + 1, E820_NVS) &&
-         !e820_all_mapped(base_addr, end_addr + 1, E820_ACPI) )
-        printk(XENLOG_WARNING VTDPREFIX
-               " RMRR [%"PRIx64",%"PRIx64"] not in reserved memory;"
-               " need \"iommu_inclusive_mapping=1\"?\n",
-                base_addr, end_addr);
+    if ( !iommu_unity_region_ok(maddr_to_mfn(base_addr),
+                                maddr_to_mfn(end_addr + PAGE_SIZE)) )
+        return -EIO;
 
     rmrru = xzalloc(struct acpi_rmrr_unit);
     if ( !rmrru )
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 17:02:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 17:02:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674569.1049641 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaSC-0007iL-98; Thu, 01 Feb 2024 17:02:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674569.1049641; Thu, 01 Feb 2024 17:02:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaSC-0007gf-1Q; Thu, 01 Feb 2024 17:02:20 +0000
Received: by outflank-mailman (input) for mailman id 674569;
 Thu, 01 Feb 2024 17:02:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cAD+=JK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rVaSA-0006n8-Nk
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 17:02:18 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a7ed1163-c123-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 18:02:18 +0100 (CET)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a359e6fde44so123351766b.3
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 09:02:18 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 cu10-20020a170906ba8a00b00a36a94ecf9dsm1031214ejd.175.2024.02.01.09.02.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 09:02:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a7ed1163-c123-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706806937; x=1707411737; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=MrhtNQnEdn9vrYvKxplKdOmWw8AltePzlQHZOnCcYkE=;
        b=TREy726yqQyCNCvxbN4sQq49dv4v/hR85PC4SXyzmDbx1h2mFnLOECrsNCDdiUhhmA
         Ld82CPHRMEHkuQ/KXqx5wHuWO34L8VdvELZYW0kE0uU5eC+zvBcj6O7HEv3RUcfyTen0
         ygEamOCONVXjYDzw4zJVfMpKvC/jhgpXEzVOg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706806937; x=1707411737;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=MrhtNQnEdn9vrYvKxplKdOmWw8AltePzlQHZOnCcYkE=;
        b=NOdHIKklLmsUQEAdrffdW3Np4z0Z5HI7AW1xc5EDJ4Cotu+Nc3BNl7K4xOL8gxmRK2
         gZtkq7C8N19sVf5PSJjnb88oHGSczinMOQGIj9IffGuJliVhnx16Jh6ATP5LX0adNyzA
         d9rmYZDMc+ao+eCpcGIiFO4N6FYeUw4gcCI/iwbochqGOBKQoIBfRbCUlzGHbKkw6hBE
         rXZVhDxTQhGtSaXgw1VjV1pR+xd6TzT+QuFnBXTJoz7/+NUCN7KkwxfucYk0FhY6asps
         7C8gDHVwW9I6nJnOka95DYwDov6w+qo9SSeRp74Oe0lqLhefqlAQ6i4pVaaAK7dWlJxU
         MQiA==
X-Gm-Message-State: AOJu0YxMKqYGTxHH1nPvzYQQxcwu3tFp3CEmaxX4VSrwQ9voInO55Sc+
	7VXj4NBEd9ENXor97/Jz67XFmxKP9BlApqhM88vfIOPZwAoe6RffUfNH6Tw2h46zk1YKwhGqMHz
	Z
X-Google-Smtp-Source: AGHT+IGc7i6430H4QaIi3+LxQD/GqTuUYou2Jbiu1ErGblVAr8z1YC/aUC4MxNi18CobbC4sV1WzTA==
X-Received: by 2002:a17:906:5f96:b0:a35:b7e6:1c44 with SMTP id a22-20020a1709065f9600b00a35b7e61c44mr4170204eju.75.1706806937437;
        Thu, 01 Feb 2024 09:02:17 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWf11yDgkYofRYjl5BKyx2LV56+rtBBpdJEdCgo5EHCP2ZvruGsNYEccw5XEHxjhlu/nPxBONNJtNXsvcabLciBZDfjnbZ8t8I6raIVlOXf
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>
Subject: [PATCH 4/4] iommu/x86: make unity range checking more strict
Date: Thu,  1 Feb 2024 18:01:59 +0100
Message-ID: <20240201170159.66330-5-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201170159.66330-1-roger.pau@citrix.com>
References: <20240201170159.66330-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Currently when a unity range overlaps with memory being used as RAM by the
hypervisor the result would be that the IOMMU gets disabled.  However that's
not enough, as even with the IOMMU disabled the device will still access the
affected RAM areas.

Note that IVMD or RMRR ranges being placed over RAM is a firmware bug.

Doing so also allows to simplify the code and use a switch over the reported
memory type(s).

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/drivers/passthrough/x86/iommu.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index 63d4cb898218..9b977f84582f 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -806,10 +806,14 @@ bool __init iommu_unity_region_ok(mfn_t start, mfn_t end)
 
     for ( addr = start; !mfn_eq(addr, end); mfn_add(addr, 1) )
     {
-        unsigned int type = page_get_ram_type(addr);
-
-        if ( type == RAM_TYPE_UNKNOWN )
+        /*
+         * Any page that's at least partially of type RESERVED, UNUSABLE or
+         * ACPI will be considered by Xen of being all of that type, and hence
+         * the problematic pages are those that are fully holes or RAM.
+         */
+        switch ( page_get_ram_type(addr) )
         {
+        case RAM_TYPE_UNKNOWN:
             if ( e820_add_range(mfn_to_maddr(addr),
                                 mfn_to_maddr(addr) + PAGE_SIZE, E820_RESERVED) )
                 continue;
@@ -817,7 +821,10 @@ bool __init iommu_unity_region_ok(mfn_t start, mfn_t end)
                    "IOMMU: page at %#" PRI_mfn " couldn't be reserved\n",
                    mfn_x(addr));
             return false;
-        }
+
+        case RAM_TYPE_CONVENTIONAL:
+            panic("IOMMU: page at %#" PRI_mfn " overlaps RAM range\n",
+                  mfn_x(addr));
 
         /*
          * Types which aren't RAM are considered good enough.
@@ -825,14 +832,7 @@ bool __init iommu_unity_region_ok(mfn_t start, mfn_t end)
          * force Xen into assuming the whole page as having that type in
          * practice.
          */
-        if ( type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
-                     RAM_TYPE_UNUSABLE) )
-            continue;
-
-        printk(XENLOG_WARNING
-               "IOMMU: page at %#" PRI_mfn " can't be converted\n",
-               mfn_x(addr));
-        return false;
+        }
     }
 
     return true;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 17:10:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 17:10:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674587.1049660 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaaQ-0002fR-AJ; Thu, 01 Feb 2024 17:10:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674587.1049660; Thu, 01 Feb 2024 17:10:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaaQ-0002fK-7V; Thu, 01 Feb 2024 17:10:50 +0000
Received: by outflank-mailman (input) for mailman id 674587;
 Thu, 01 Feb 2024 17:10:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cAD+=JK=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rVaaO-0002fC-Nk
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 17:10:48 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d7b13bb1-c124-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 18:10:47 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a3566c0309fso158463466b.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 09:10:47 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 xa14-20020a170907b9ce00b00a34ae71e58dsm7473352ejc.147.2024.02.01.09.10.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 09:10:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7b13bb1-c124-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706807447; x=1707412247; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=510KNn1Q+2goZvYgFG10wKrbP7MgTGvopo6XgmHSduA=;
        b=c511vmQT8O1jGlO0HNVMWgEU0bMSxc9Q1AYxGALHb5wOUvaggjdSpg/Z+F+CnTmL/0
         tS2lfxlHJANGcegZ3PMp0AsweM0tQt+vxOOO01cCK6wQfaILfrV3ZX3IUERkDbV/zH0l
         YIs/I1CMmgAj9M9EoXBib43Bfd3+zhtinKYVY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706807447; x=1707412247;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=510KNn1Q+2goZvYgFG10wKrbP7MgTGvopo6XgmHSduA=;
        b=Q6OQNQKpyFHqKhVVLL1nca+V2txI4zW5pkrmTWqjGRfv66+XkUZhoiS/iKUaPNTsZr
         a5DVDnBA2/3wEDYI94gB7xFsoRZz9iouNFdIVjxhMyYutZ8XKsrxHjGWLcAITZDYh4SN
         fa2nbF8aAlpQ3872Taq0UPw4aZz6qYUVQUhzuYaHjPLK5tZs1omlmrY7z2dE8AXWEFi9
         LJsx5Qn7BpNujygJeyjFhVMKRuqU2qLhGhHkAy1LOJhbmAe4qrM6PVJ/PHx/1awNvaQy
         BU77abXQKekXOrgqvIAlik4ZEW068pS/zoyCnJirUFhPQzUHRRU52P/4o6O7oo+Zuv6H
         hdmg==
X-Gm-Message-State: AOJu0Yx4kCy+4VaAcoNJxXBLwpCRjDhIYaQyqCXjoq42QBec+s7218Hc
	Dyzw+EsV9M7C4uqb7ORPHh5Fwuq8PK9k2KvukJJ5U2prviHr+etMQp8scq5LnqU=
X-Google-Smtp-Source: AGHT+IGMB8bk+RwtbqA2kw8RFjlZrVHfrfFT0UtQLSx8nX7wXIHIn6m63pkC0HTAArmKFfW7Xxq61A==
X-Received: by 2002:a17:906:491a:b0:a36:34c7:21a5 with SMTP id b26-20020a170906491a00b00a3634c721a5mr3836938ejq.22.1706807447306;
        Thu, 01 Feb 2024 09:10:47 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCW3rj632b4k6Bly38eru1TfMXrU1Brj/DVB1CG1KkVRxwmfU+NLcJSgsyEWaPQjVgQmnD9E7P+s7lIa61S0YJvXamPbo/Ah8Ik+43p9BIu+4zJ5kZTcWnsxDLKHdQy2tw0QRBFyZhGd2XVcV5og0TTegfh0fOp1Ydj3IIU4/MqxEMaUUs1SiMGV
Date: Thu, 1 Feb 2024 18:10:46 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Wei Liu <wl@xen.org>, Kevin Tian <kevin.tian@intel.com>,
	Jun Nakajima <jun.nakajima@intel.com>
Subject: Re: [PATCH v4 3/8] VMX: tertiary execution control infrastructure
Message-ID: <ZbvQlrj7u-bwoVzy@macbook>
References: <9dd23064-c79e-4a50-9c71-c0e73b189944@suse.com>
 <aa553449-888f-4e52-85b7-0bc0b7f010b4@suse.com>
 <ZbuFc-ha-iv6B9ui@macbook>
 <56822f9c-6156-4652-9de1-3d599e22c51a@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <56822f9c-6156-4652-9de1-3d599e22c51a@suse.com>

On Thu, Feb 01, 2024 at 01:09:11PM +0100, Jan Beulich wrote:
> On 01.02.2024 12:50, Roger Pau Monné wrote:
> > On Thu, Jan 11, 2024 at 10:00:10AM +0100, Jan Beulich wrote:
> >> @@ -503,6 +538,9 @@ static int vmx_init_vmcs_config(bool bsp
> >>              "Secondary Exec Control",
> >>              vmx_secondary_exec_control, _vmx_secondary_exec_control);
> >>          mismatch |= cap_check(
> >> +            "Tertiary Exec Control",
> >> +            vmx_tertiary_exec_control, _vmx_tertiary_exec_control);
> > 
> > I know it's done to match the surrounding style, but couldn't you move
> > the name parameter one line up, and then limit the call to two lines?
> > 
> > (I don't think it will compromise readability).
> 
> You mean like this:
> 
>         mismatch |= cap_check("Tertiary Exec Control",
>             vmx_tertiary_exec_control, _vmx_tertiary_exec_control);
> 
> ? No, I view this as a mix of two possible styles. If the string literal
> was moved up, the other legitimate style would only be
> 
>         mismatch |= cap_check("Tertiary Exec Control",
>                               vmx_tertiary_exec_control,
>                               _vmx_tertiary_exec_control);
> 
> aiui (again extending over 3 lines). Yet none of this is written down
> anywhere.
> 
> But anyway - consistency with surrounding code trumps here, I think.

I was hoping it could still fit on 2 lines, but if you need 3 never
mind then.

> >> @@ -2068,10 +2111,12 @@ void vmcs_dump_vcpu(struct vcpu *v)
> >>                 vmr(HOST_PERF_GLOBAL_CTRL));
> >>  
> >>      printk("*** Control State ***\n");
> >> -    printk("PinBased=%08x CPUBased=%08x SecondaryExec=%08x\n",
> >> +    printk("PinBased=%08x CPUBased=%08x\n",
> >>             vmr32(PIN_BASED_VM_EXEC_CONTROL),
> >> -           vmr32(CPU_BASED_VM_EXEC_CONTROL),
> >> -           vmr32(SECONDARY_VM_EXEC_CONTROL));
> >> +           vmr32(CPU_BASED_VM_EXEC_CONTROL));
> >> +    printk("SecondaryExec=%08x TertiaryExec=%08lx\n",
> > 
> > For consistency, shouldn't TertiaryExec use 016 instead of 08 (as it's
> > a 64bit filed).
> 
> Perhaps, assuming we'll gets bits 32 and populated sooner or later.
> However, I view 16-digit literal numbers as hard to read, so I'd be
> inclined to insert a separator (e.g. an underscore) between the low
> and high halves. Thoughts?

Works for me.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 17:18:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 17:18:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674596.1049670 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVai8-00053Q-3X; Thu, 01 Feb 2024 17:18:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674596.1049670; Thu, 01 Feb 2024 17:18:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVai7-00053J-Vw; Thu, 01 Feb 2024 17:18:47 +0000
Received: by outflank-mailman (input) for mailman id 674596;
 Thu, 01 Feb 2024 17:18:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cG+K=JK=casper.srs.infradead.org=BATV+16c2638fe4346fda554c+7466+infradead.org+dwmw2@srs-se1.protection.inumbo.net>)
 id 1rVai5-00053D-PB
 for xen-devel@lists.xen.org; Thu, 01 Feb 2024 17:18:46 +0000
Received: from casper.infradead.org (casper.infradead.org
 [2001:8b0:10b:1236::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f31d2171-c125-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 18:18:44 +0100 (CET)
Received: from [205.251.233.53] (helo=freeip.amazon.com)
 by casper.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rVai0-0000000GNFh-45si; Thu, 01 Feb 2024 17:18:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f31d2171-c125-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=MIME-Version:Content-Type:References:
	In-Reply-To:Date:To:From:Subject:Message-ID:Sender:Reply-To:Cc:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=T9Fs+1NiNaEOfFjfaUDtfOiyuIOFtkmdu6ETtmgrLlk=; b=GW90F22632pdDYARggo0apvbI8
	AyaoS6wVbNBejq3AP77VoH6ZBGFK4ypoJwnA/AEhgG0hjAWbXYibuM8Deu38EqQSwdmZmy1/VC/hZ
	qTUXmaHaj4lySPAe4Som1GgGS1krpbUD8ppWN32cjIF2yGs5WQXqNIVDDB9+kUm7ZruQJVBLqWqeV
	ZW/eTybKGjIknHtl28vT99syJvMXdhUFrmXVVHB4ia4XRtn2aaNJVYvKVlnY6fiA29KMUru3uweoE
	eKjj8jsARVJrE/0PqKFjosOu0rCGXdDbU3I1p8YPyCfxkHqooCuvkLM58glyLAbieKUdinH2iN83b
	YeccKfag==;
Message-ID: <4b5b1fdadbabd53116d1ab79295b913c27947c4b.camel@infradead.org>
Subject: Re: Nullifying Recently Introduced Xen Headers Check
From: David Woodhouse <dwmw2@infradead.org>
To: "John L. Poole" <johnlpoole@proton.me>, xen-devel@lists.xen.org
Date: Thu, 01 Feb 2024 09:18:40 -0800
In-Reply-To: <fb379476-f6ae-4eb7-b3c6-6f0ad175964d@proton.me>
References: <fb379476-f6ae-4eb7-b3c6-6f0ad175964d@proton.me>
Content-Type: multipart/signed; micalg="sha-256"; protocol="application/pkcs7-signature";
	boundary="=-I6Mgn+WqVKlm+Jzmgs0k"
User-Agent: Evolution 3.44.4-0ubuntu2 
MIME-Version: 1.0
X-SRS-Rewrite: SMTP reverse-path rewritten from <dwmw2@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html


--=-I6Mgn+WqVKlm+Jzmgs0k
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

(Thanks Andy for the explicit cc)

On Thu, 2024-02-01 at 00:05 +0000, John L. Poole wrote:
>=20
> cause Gentoo's build to error out.=C2=A0 See line 24790:
>=20
> =C2=A0=C2=A0=C2=A0 5 | #error In Xen native files, include xen_native.h b=
efore other Xen headers
>=20
> at
> https://salemdata.us/xen/xen_tools_20240128_Sun_174740.script.html.
>=20
> What I have done is create a patch for a draft Gentoo ebuild which
> nullifies lines 4-6 by wrapping them in a comment:

That isn't what the #error told you to do, though.

 24788	In file included from ../qemu-xen/hw/xen/xen-operations.c:16:
 24789	/var/tmp/portage/app-emulation/xen-tools-4.18.0/work/xen-4.18.0/tool=
s/qemu-xen/include/hw/xen/xen_native.h:5:2: error: #error In Xen native fil=
es, include xen_native.h before other Xen headers
 24790	    5 | #error In Xen native files, include xen_native.h before othe=
r Xen headers
 24791	      |  ^~~~~

So it's hw/xen/xen-operations.h which is failing. As far as I can tell
(visually and empirically because it does actually build elsewhere), it
*is* doing what the #error said =E2=80=94 it *is* including xen_native.h be=
fore
any other Xen headers.=20

The first four non-comment lines of xen-operations.c should look
something like this...

  #include "qemu/osdep.h"
  #include "qemu/uuid.h"
  #include "qapi/error.h"

  #include "hw/xen/xen_native.h"

So... did you patch it so it doesn't start like that any more? Or does
one of those first three files (perhaps qemu/osdep.h?) end up bringing
in the Xen interface headers in a way that I didn't anticipate and
which doesn't seem to happen elsewhere?

I didn't cite the full gcc command line from line 24787 of your log
because it's huge. Can you run a variant of that command to just give
me the *preprocessed* output (-E -dD -o xen-operations.i).

--=-I6Mgn+WqVKlm+Jzmgs0k
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Disposition: attachment; filename="smime.p7s"
Content-Transfer-Encoding: base64

MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCCEkQw
ggYQMIID+KADAgECAhBNlCwQ1DvglAnFgS06KwZPMA0GCSqGSIb3DQEBDAUAMIGIMQswCQYDVQQG
EwJVUzETMBEGA1UECBMKTmV3IEplcnNleTEUMBIGA1UEBxMLSmVyc2V5IENpdHkxHjAcBgNVBAoT
FVRoZSBVU0VSVFJVU1QgTmV0d29yazEuMCwGA1UEAxMlVVNFUlRydXN0IFJTQSBDZXJ0aWZpY2F0
aW9uIEF1dGhvcml0eTAeFw0xODExMDIwMDAwMDBaFw0zMDEyMzEyMzU5NTlaMIGWMQswCQYDVQQG
EwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYD
VQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMTNVNlY3RpZ28gUlNBIENsaWVudCBBdXRoZW50
aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
AQEAyjztlApB/975Rrno1jvm2pK/KxBOqhq8gr2+JhwpKirSzZxQgT9tlC7zl6hn1fXjSo5MqXUf
ItMltrMaXqcESJuK8dtK56NCSrq4iDKaKq9NxOXFmqXX2zN8HHGjQ2b2Xv0v1L5Nk1MQPKA19xeW
QcpGEGFUUd0kN+oHox+L9aV1rjfNiCj3bJk6kJaOPabPi2503nn/ITX5e8WfPnGw4VuZ79Khj1YB
rf24k5Ee1sLTHsLtpiK9OjG4iQRBdq6Z/TlVx/hGAez5h36bBJMxqdHLpdwIUkTqT8se3ed0PewD
ch/8kHPo5fZl5u1B0ecpq/sDN/5sCG52Ds+QU5O5EwIDAQABo4IBZDCCAWAwHwYDVR0jBBgwFoAU
U3m/WqorSs9UgOHYm8Cd8rIDZsswHQYDVR0OBBYEFAnA8vwL2pTbX/4r36iZQs/J4K0AMA4GA1Ud
DwEB/wQEAwIBhjASBgNVHRMBAf8ECDAGAQH/AgEAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDARBgNVHSAECjAIMAYGBFUdIAAwUAYDVR0fBEkwRzBFoEOgQYY/aHR0cDovL2NybC51c2Vy
dHJ1c3QuY29tL1VTRVJUcnVzdFJTQUNlcnRpZmljYXRpb25BdXRob3JpdHkuY3JsMHYGCCsGAQUF
BwEBBGowaDA/BggrBgEFBQcwAoYzaHR0cDovL2NydC51c2VydHJ1c3QuY29tL1VTRVJUcnVzdFJT
QUFkZFRydXN0Q0EuY3J0MCUGCCsGAQUFBzABhhlodHRwOi8vb2NzcC51c2VydHJ1c3QuY29tMA0G
CSqGSIb3DQEBDAUAA4ICAQBBRHUAqznCFfXejpVtMnFojADdF9d6HBA4kMjjsb0XMZHztuOCtKF+
xswhh2GqkW5JQrM8zVlU+A2VP72Ky2nlRA1GwmIPgou74TZ/XTarHG8zdMSgaDrkVYzz1g3nIVO9
IHk96VwsacIvBF8JfqIs+8aWH2PfSUrNxP6Ys7U0sZYx4rXD6+cqFq/ZW5BUfClN/rhk2ddQXyn7
kkmka2RQb9d90nmNHdgKrwfQ49mQ2hWQNDkJJIXwKjYA6VUR/fZUFeCUisdDe/0ABLTI+jheXUV1
eoYV7lNwNBKpeHdNuO6Aacb533JlfeUHxvBz9OfYWUiXu09sMAviM11Q0DuMZ5760CdO2VnpsXP4
KxaYIhvqPqUMWqRdWyn7crItNkZeroXaecG03i3mM7dkiPaCkgocBg0EBYsbZDZ8bsG3a08LwEsL
1Ygz3SBsyECa0waq4hOf/Z85F2w2ZpXfP+w8q4ifwO90SGZZV+HR/Jh6rEaVPDRF/CEGVqR1hiuQ
OZ1YL5ezMTX0ZSLwrymUE0pwi/KDaiYB15uswgeIAcA6JzPFf9pLkAFFWs1QNyN++niFhsM47qod
x/PL+5jR87myx5uYdBEQkkDc+lKB1Wct6ucXqm2EmsaQ0M95QjTmy+rDWjkDYdw3Ms6mSWE3Bn7i
5ZgtwCLXgAIe5W8mybM2JzCCBhQwggT8oAMCAQICEQDGvhmWZ0DEAx0oURL6O6l+MA0GCSqGSIb3
DQEBCwUAMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYD
VQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMTNVNlY3RpZ28g
UlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMB4XDTIyMDEwNzAw
MDAwMFoXDTI1MDEwNjIzNTk1OVowJDEiMCAGCSqGSIb3DQEJARYTZHdtdzJAaW5mcmFkZWFkLm9y
ZzCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALQ3GpC2bomUqk+91wLYBzDMcCj5C9m6
oZaHwvmIdXftOgTbCJXADo6G9T7BBAebw2JV38EINgKpy/ZHh7htyAkWYVoFsFPrwHounto8xTsy
SSePMiPlmIdQ10BcVSXMUJ3Juu16GlWOnAMJY2oYfEzmE7uT9YgcBqKCo65pTFmOnR/VVbjJk4K2
xE34GC2nAdUQkPFuyaFisicc6HRMOYXPuF0DuwITEKnjxgNjP+qDrh0db7PAjO1D4d5ftfrsf+kd
RR4gKVGSk8Tz2WwvtLAroJM4nXjNPIBJNT4w/FWWc/5qPHJy2U+eITZ5LLE5s45mX2oPFknWqxBo
bQZ8a9dsZ3dSPZBvE9ZrmtFLrVrN4eo1jsXgAp1+p7bkfqd3BgBEmfsYWlBXO8rVXfvPgLs32VdV
NZxb/CDWPqBsiYv0Hv3HPsz07j5b+/cVoWqyHDKzkaVbxfq/7auNVRmPB3v5SWEsH8xi4Bez2V9U
KxfYCnqsjp8RaC2/khxKt0A552Eaxnz/4ly/2C7wkwTQnBmdlFYhAflWKQ03Ufiu8t3iBE3VJbc2
5oMrglj7TRZrmKq3CkbFnX0fyulB+kHimrt6PIWn7kgyl9aelIl6vtbhMA+l0nfrsORMa4kobqQ5
C5rveVgmcIad67EDa+UqEKy/GltUwlSh6xy+TrK1tzDvAgMBAAGjggHMMIIByDAfBgNVHSMEGDAW
gBQJwPL8C9qU21/+K9+omULPyeCtADAdBgNVHQ4EFgQUzMeDMcimo0oz8o1R1Nver3ZVpSkwDgYD
VR0PAQH/BAQDAgWgMAwGA1UdEwEB/wQCMAAwHQYDVR0lBBYwFAYIKwYBBQUHAwQGCCsGAQUFBwMC
MEAGA1UdIAQ5MDcwNQYMKwYBBAGyMQECAQEBMCUwIwYIKwYBBQUHAgEWF2h0dHBzOi8vc2VjdGln
by5jb20vQ1BTMFoGA1UdHwRTMFEwT6BNoEuGSWh0dHA6Ly9jcmwuc2VjdGlnby5jb20vU2VjdGln
b1JTQUNsaWVudEF1dGhlbnRpY2F0aW9uYW5kU2VjdXJlRW1haWxDQS5jcmwwgYoGCCsGAQUFBwEB
BH4wfDBVBggrBgEFBQcwAoZJaHR0cDovL2NydC5zZWN0aWdvLmNvbS9TZWN0aWdvUlNBQ2xpZW50
QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFpbENBLmNydDAjBggrBgEFBQcwAYYXaHR0cDovL29j
c3Auc2VjdGlnby5jb20wHgYDVR0RBBcwFYETZHdtdzJAaW5mcmFkZWFkLm9yZzANBgkqhkiG9w0B
AQsFAAOCAQEAyW6MUir5dm495teKqAQjDJwuFCi35h4xgnQvQ/fzPXmtR9t54rpmI2TfyvcKgOXp
qa7BGXNFfh1JsqexVkIqZP9uWB2J+uVMD+XZEs/KYNNX2PvIlSPrzIB4Z2wyIGQpaPLlYflrrVFK
v9CjT2zdqvy2maK7HKOQRt3BiJbVG5lRiwbbygldcALEV9ChWFfgSXvrWDZspnU3Gjw/rMHrGnql
Htlyebp3pf3fSS9kzQ1FVtVIDrL6eqhTwJxe+pXSMMqFiN0whpBtXdyDjzBtQTaZJ7zTT/vlehc/
tDuqZwGHm/YJy883Ll+GP3NvOkgaRGWEuYWJJ6hFCkXYjyR9IzCCBhQwggT8oAMCAQICEQDGvhmW
Z0DEAx0oURL6O6l+MA0GCSqGSIb3DQEBCwUAMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3Jl
YXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0
ZWQxPjA8BgNVBAMTNVNlY3RpZ28gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJl
IEVtYWlsIENBMB4XDTIyMDEwNzAwMDAwMFoXDTI1MDEwNjIzNTk1OVowJDEiMCAGCSqGSIb3DQEJ
ARYTZHdtdzJAaW5mcmFkZWFkLm9yZzCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALQ3
GpC2bomUqk+91wLYBzDMcCj5C9m6oZaHwvmIdXftOgTbCJXADo6G9T7BBAebw2JV38EINgKpy/ZH
h7htyAkWYVoFsFPrwHounto8xTsySSePMiPlmIdQ10BcVSXMUJ3Juu16GlWOnAMJY2oYfEzmE7uT
9YgcBqKCo65pTFmOnR/VVbjJk4K2xE34GC2nAdUQkPFuyaFisicc6HRMOYXPuF0DuwITEKnjxgNj
P+qDrh0db7PAjO1D4d5ftfrsf+kdRR4gKVGSk8Tz2WwvtLAroJM4nXjNPIBJNT4w/FWWc/5qPHJy
2U+eITZ5LLE5s45mX2oPFknWqxBobQZ8a9dsZ3dSPZBvE9ZrmtFLrVrN4eo1jsXgAp1+p7bkfqd3
BgBEmfsYWlBXO8rVXfvPgLs32VdVNZxb/CDWPqBsiYv0Hv3HPsz07j5b+/cVoWqyHDKzkaVbxfq/
7auNVRmPB3v5SWEsH8xi4Bez2V9UKxfYCnqsjp8RaC2/khxKt0A552Eaxnz/4ly/2C7wkwTQnBmd
lFYhAflWKQ03Ufiu8t3iBE3VJbc25oMrglj7TRZrmKq3CkbFnX0fyulB+kHimrt6PIWn7kgyl9ae
lIl6vtbhMA+l0nfrsORMa4kobqQ5C5rveVgmcIad67EDa+UqEKy/GltUwlSh6xy+TrK1tzDvAgMB
AAGjggHMMIIByDAfBgNVHSMEGDAWgBQJwPL8C9qU21/+K9+omULPyeCtADAdBgNVHQ4EFgQUzMeD
Mcimo0oz8o1R1Nver3ZVpSkwDgYDVR0PAQH/BAQDAgWgMAwGA1UdEwEB/wQCMAAwHQYDVR0lBBYw
FAYIKwYBBQUHAwQGCCsGAQUFBwMCMEAGA1UdIAQ5MDcwNQYMKwYBBAGyMQECAQEBMCUwIwYIKwYB
BQUHAgEWF2h0dHBzOi8vc2VjdGlnby5jb20vQ1BTMFoGA1UdHwRTMFEwT6BNoEuGSWh0dHA6Ly9j
cmwuc2VjdGlnby5jb20vU2VjdGlnb1JTQUNsaWVudEF1dGhlbnRpY2F0aW9uYW5kU2VjdXJlRW1h
aWxDQS5jcmwwgYoGCCsGAQUFBwEBBH4wfDBVBggrBgEFBQcwAoZJaHR0cDovL2NydC5zZWN0aWdv
LmNvbS9TZWN0aWdvUlNBQ2xpZW50QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFpbENBLmNydDAj
BggrBgEFBQcwAYYXaHR0cDovL29jc3Auc2VjdGlnby5jb20wHgYDVR0RBBcwFYETZHdtdzJAaW5m
cmFkZWFkLm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAyW6MUir5dm495teKqAQjDJwuFCi35h4xgnQv
Q/fzPXmtR9t54rpmI2TfyvcKgOXpqa7BGXNFfh1JsqexVkIqZP9uWB2J+uVMD+XZEs/KYNNX2PvI
lSPrzIB4Z2wyIGQpaPLlYflrrVFKv9CjT2zdqvy2maK7HKOQRt3BiJbVG5lRiwbbygldcALEV9Ch
WFfgSXvrWDZspnU3Gjw/rMHrGnqlHtlyebp3pf3fSS9kzQ1FVtVIDrL6eqhTwJxe+pXSMMqFiN0w
hpBtXdyDjzBtQTaZJ7zTT/vlehc/tDuqZwGHm/YJy883Ll+GP3NvOkgaRGWEuYWJJ6hFCkXYjyR9
IzGCBMcwggTDAgEBMIGsMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVz
dGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMT
NVNlY3RpZ28gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBAhEA
xr4ZlmdAxAMdKFES+jupfjANBglghkgBZQMEAgEFAKCCAeswGAYJKoZIhvcNAQkDMQsGCSqGSIb3
DQEHATAcBgkqhkiG9w0BCQUxDxcNMjQwMjAxMTcxODQwWjAvBgkqhkiG9w0BCQQxIgQg2zS68SBP
FwBC9hLRhAbc2pPgDSz+KkiikomWclfpe4Ewgb0GCSsGAQQBgjcQBDGBrzCBrDCBljELMAkGA1UE
BhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEYMBYG
A1UEChMPU2VjdGlnbyBMaW1pdGVkMT4wPAYDVQQDEzVTZWN0aWdvIFJTQSBDbGllbnQgQXV0aGVu
dGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIRAMa+GZZnQMQDHShREvo7qX4wgb8GCyqGSIb3
DQEJEAILMYGvoIGsMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVy
MRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMTNVNl
Y3RpZ28gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBAhEAxr4Z
lmdAxAMdKFES+jupfjANBgkqhkiG9w0BAQEFAASCAgAm7sjBKVr7pvaQw8DnxjMwOVYu1tpMm2jA
KFEIXr2VYtaBsptTS4w+uQRYXFW4QhdlHUuMfcysouEVrnF3om+HPRZgsVhzcuz+IyAHoMGIu5ch
vl6/nu806VbKs/xr4j4LK/w9biQn+BYJd+K7Zcepfp+ExUk+ftiv6FcT8iwRkjL/FB3INutIZ4X5
Gz2N0IMzHId3IBZ9+iI0o7Y3rSjW1fkLm/1ponOza6Qp4j7znMVia76Xsdzh0grPjL7AyX0omFvt
mihhruLRcCryQY5a6oeYDqgHQFwGqc8PRvMVMc9V8aLTTqd7iolaFoQZGujhKC/MVTSR3TCzSLfW
DsZ03aCCKJzf7A1lOhVvRVPin+bDmVlBAFmgC1mENokIAzaNSQB+7tL/XcmmY1f/i9l7C1bZm0NQ
T8S9YcYdjS3H7q9pgQOEukZNmf8cYGuYgHanhbg3+ZHTbQEKqroTheRgvsufot/NdNaUbVpl12PS
3ijDIf/Pd6y+x4/CJODE2KQi5dzPllXRSGV23QHUmMys/hkVLi8m8J6IfuFrjvbc9HDb3JJzCpQL
rL90JBYeHI0yIPZ/g/A1oPUYK04lTQD/py0kv4fRcsZbhGLVN0BlMGD2j9jySEFnnzRYeGwOJNIZ
MF8JcHBfDDC01fwWuOyds2lM+i2VxNn7/a2XJeuCDQAAAAAAAA==


--=-I6Mgn+WqVKlm+Jzmgs0k--


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 17:36:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 17:36:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674612.1049683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaz6-0002av-Hz; Thu, 01 Feb 2024 17:36:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674612.1049683; Thu, 01 Feb 2024 17:36:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVaz6-0002ao-FL; Thu, 01 Feb 2024 17:36:20 +0000
Received: by outflank-mailman (input) for mailman id 674612;
 Thu, 01 Feb 2024 17:36:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rVaz5-0002ai-8B
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 17:36:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVaz4-00061F-QR; Thu, 01 Feb 2024 17:36:18 +0000
Received: from [15.248.2.31] (helo=[10.24.67.35])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVaz4-0005vb-Hi; Thu, 01 Feb 2024 17:36:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=zsnw957Hfwirsjx1xy39SSyh4Kw8IBkEQYb1cspkB/c=; b=FX7P5C9pdrgsi43y/iDae+DJJC
	mmM2UGQmE7hOuwHFuSTgD2wV1ZRhXtvbITAApdR/BMd+dwVa/pGPptoEZ4vcSqGMmSIucUWP5Upre
	tET1nn5xwTFggQPLuLiYeqrwGKRIFlOJGQNJew0dmlJ3vDZ1CcKZuNsryYu8Cskxq55k=;
Message-ID: <aa433592-56d7-44f5-9f2d-a5288a6fd766@xen.org>
Date: Thu, 1 Feb 2024 17:36:16 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Properly clean update to init_ttbr and
 smp_up_cpu
Content-Language: en-GB
To: Bertrand Marquis <Bertrand.Marquis@arm.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Julien Grall <jgrall@amazon.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
References: <20240130172942.52175-1-julien@xen.org>
 <0D96C966-8DC4-43C5-9F61-0C54A2DC796C@arm.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <0D96C966-8DC4-43C5-9F61-0C54A2DC796C@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 31/01/2024 07:57, Bertrand Marquis wrote:
>> On 30 Jan 2024, at 18:29, Julien Grall <julien@xen.org> wrote:
>>
>> From: Julien Grall <jgrall@amazon.com>
>>
>> Recent rework to the secondary boot code modified how init_ttbr and
>> smp_up_cpu are accessed. Rather than directly accessing them, we
>> are using a pointer to them.
>>
>> The helper clean_dcache() is expected to take the variable in parameter
>> and then clean its content. As we now pass a pointer to the variable,
>> we will clean the area storing the address rather than the content itself.
>>
>> Switch to use clean_dcache_va_range() to avoid casting the pointer.
>>
>> Fixes: a5ed59e62c6f ("arm/mmu: Move init_ttbr to a new section .data.idmap")
>> Fixes: 9a5114074b04 ("arm/smpboot: Move smp_up_cpu to a new section .data.idmap)
>>
>> Reported-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> Signed-off-by: Julien Grall <jgrall@amazon.com>
> 
> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Committed. Thanks.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 18:28:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 18:28:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674663.1049721 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbn0-0008RD-JO; Thu, 01 Feb 2024 18:27:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674663.1049721; Thu, 01 Feb 2024 18:27:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbn0-0008R6-Fp; Thu, 01 Feb 2024 18:27:54 +0000
Received: by outflank-mailman (input) for mailman id 674663;
 Thu, 01 Feb 2024 18:27:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVbmz-0008Qw-Qu; Thu, 01 Feb 2024 18:27:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVbmz-00078j-PR; Thu, 01 Feb 2024 18:27:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVbmz-0006e4-E2; Thu, 01 Feb 2024 18:27:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVbmz-0005P9-DU; Thu, 01 Feb 2024 18:27:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=rX8rLPavRS3L2XzwXivWDx/dMB/0BrCwqoH/MmpsE38=; b=2ZIej4vtwD6sCFLeRJy2TSushi
	pc8iZXHYlIW0rZWysaoUqB1PhgDIuTBMmXmoHJFceJQyu8REFJyaXQVQpfsEeaGWPCK/SOMaIn3FL
	1E9RipsOrCBpLYKF3pIbnNLWZjsNkbUmjNVh6J/Dpvwxm6ZI0WGvr+/NlvYfzotpK+Ls=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184552-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184552: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=bc45f20c01f1711bc56a4bb0955c49c182a5a03a
X-Osstest-Versions-That:
    xen=cc6ba68edf6dcd18c3865e7d7c0f1ed822796426
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 01 Feb 2024 18:27:53 +0000

flight 184552 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184552/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  bc45f20c01f1711bc56a4bb0955c49c182a5a03a
baseline version:
 xen                  cc6ba68edf6dcd18c3865e7d7c0f1ed822796426

Last test of basis   184528  2024-01-30 14:03:48 Z    2 days
Testing same since   184552  2024-02-01 16:00:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Carlo Nonato <carlo.nonato@minervasys.tech>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   cc6ba68edf..bc45f20c01  bc45f20c01f1711bc56a4bb0955c49c182a5a03a -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 18:30:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 18:30:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674668.1049730 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbpf-00020y-VV; Thu, 01 Feb 2024 18:30:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674668.1049730; Thu, 01 Feb 2024 18:30:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbpf-00020r-Sq; Thu, 01 Feb 2024 18:30:39 +0000
Received: by outflank-mailman (input) for mailman id 674668;
 Thu, 01 Feb 2024 18:30:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MzG+=JK=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rVbpe-00020T-ST
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 18:30:38 +0000
Received: from mail-oo1-xc30.google.com (mail-oo1-xc30.google.com
 [2607:f8b0:4864:20::c30])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fe426915-c12f-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 19:30:37 +0100 (CET)
Received: by mail-oo1-xc30.google.com with SMTP id
 006d021491bc7-59a8b9b327aso588674eaf.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 10:30:37 -0800 (PST)
Received: from pm2-ws13.praxislan02.com
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 pf8-20020a056214498800b0068c75141e73sm25824qvb.2.2024.02.01.10.30.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 10:30:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe426915-c12f-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706812235; x=1707417035; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=vJji4XqXHiQq6UqkyTBADvfuoJ+3mi3CXw1t6K2EPpo=;
        b=CfKu3gHMrURnR5VmyAQeZ5dRpF06sIEGjmVfRwShbWJIMWJ1Th3IiVACbKlRYPTF6R
         1fT78Nt0dAkQs3tY4VscgMIzNl0qoxZRTSP/1Xp8csxzlUdTy5pA1/nUkuJ294bDfuCn
         s2+SzMhYwHu3DQQxhsPVXfMoY/7NpC7Jjwa7YLh9JJJtf9LzryxiLAQCpkkA12GCu2tV
         hv/15wqGQ9jnwqRgQrJQE+3zqF5NBhYbnUiFtmO2oezEELeEaleyL+2VUQZ/XYQW8v4C
         n+sQbzbe/3HbTi1T4kvu7cbp+swVMCXHeKJsy25HLxxo44deqvXZ30eE5LBQfqeeQYWU
         ekCQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706812235; x=1707417035;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=vJji4XqXHiQq6UqkyTBADvfuoJ+3mi3CXw1t6K2EPpo=;
        b=qWOHlCCOcFGf+qm8GvdvU+KyISmrEneHBvWfyh1WCJQ052/zlXUng0WowT6VNkCtT+
         lLv5Q1utJ7/956nGUJyAbLyS93QEreMVuL2HBMyrpIu7hlXwmpM1pKUcPSEx9cPFQr7M
         I7ztJ1Yftb1JWC0hG+yRIfrqPlEI+inhgcyTa9Y9nQ7oolpOw0ph2r2DrbS2OX1yj6FX
         MyvdEre0pQ8NmOFekBIKeGikTLKwwYz21VFvtOlay/vXNS47xECffYjufxBjC3nqsAat
         PntEmcuogsNwCxJYpR2i5X9lJI60FtrPeNl5M3CWBZ801y3xzGhnw3RakfsaVYGGNVOH
         yuHw==
X-Gm-Message-State: AOJu0YyL6UVFs/PH7/kTcR9m81xd5Yu18+rYwvmRa2AX2cLSdbNBbqzW
	6EA2m52TZU8BzdM3GmgZIUxkIhebsu5EocNeiN9FhSVX4CblOREMZdtrl1ij
X-Google-Smtp-Source: AGHT+IGV0uyPo9vT0Nu0foAi8DR32p22Pp8jVZK9AlB4genAWqSBAP0FPq2wS6ruxXJ0fQURhtfBsg==
X-Received: by 2002:a05:6358:4b05:b0:175:fc1a:c7df with SMTP id kr5-20020a0563584b0500b00175fc1ac7dfmr5911393rwc.15.1706812235109;
        Thu, 01 Feb 2024 10:30:35 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUHWpEq/EQiJKqWpN75DGqRJxbJeZ11a8UUSu3X/895bYo+YOko02W7JF9qji9NNesW7KkyvNP51IZNKtxVjIunowdVDL0aXFSem43458xbgSa4FEGGUuvCbTcQWtdIXHcrAb7vpWA=
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jandryuk@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH 0/4] libxl: blktap/tapback support
Date: Thu,  1 Feb 2024 13:30:20 -0500
Message-ID: <20240201183024.145424-1-jandryuk@gmail.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch re-introduces blktap support to libxl.  Unlike earlier
versions, it does not link against any blktap library.  libxl changes
are needed to write to the vbd3 backend XenStore nodes.

blktap has three components.  tapdisk is a daemon implementing the disk
IO, NBD (Network Block Device), and Xen PV interfaces.  tap-ctl is a
tool to control tapdisks - creating, starting, stopping and freeing.
tapback manages the XenStore operations and instructs tapdisk to
connect.

It is notable that tapdisk performs the grant and event channel ops, but
doesn't interact with XenStore.  tapback performs XenStore operations
and notifies tapdisks of values and changes.

The flow is: libxl writes to the "vbd3" XenStore nodes and runs the
block-tap script.  The block-tap script runs tap-ctl to create a tapdisk
instance as the physical device.  tapback then sees the tapdisk and
instructs the tapdisk to connect up the PV blkif interface.

This is expected to work without the kernel blktap driver, so the
block-tap script is modified accordingly to write the UNIX NBD path.
(It works with the kernel blktap driver as well - upstream blktap hasn't
removed the blktap driver requirement yet -
https://github.com/xapi-project/blktap/pull/364)

An example command to attach a vhd:
xl block-attach vm 'vdev=xvdf,backendtype=tap,format=vhd,target=/srv/target.vhd'

VHD support is important for OpenXT since there are lots of existing
VHDs which still need supporting.  tapdisk also supports encrypted VHDs
which is not available in QEMU.

blktap's tapback needs minimal changes to work with libxl:
https://github.com/xapi-project/blktap/pull/394

Jason Andryuk (4):
  block-common: Fix same_vm for no targets
  libxl: Add support for blktap vbd3
  hotplug: Update block-tap
  libxl: Support blktap with HVM device model

 docs/man/xl-disk-configuration.5.pod.in   |   4 +-
 tools/hotplug/Linux/block-common.sh       |   8 +-
 tools/hotplug/Linux/block-tap             | 162 ++++++++++++++++++++--
 tools/libs/light/libxl_device.c           |  14 +-
 tools/libs/light/libxl_disk.c             |  37 +++--
 tools/libs/light/libxl_dm.c               |   1 -
 tools/libs/light/libxl_linux.c            |   1 +
 tools/libs/light/libxl_types_internal.idl |   1 +
 tools/libs/light/libxl_utils.c            |   2 +
 9 files changed, 201 insertions(+), 29 deletions(-)

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 18:30:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 18:30:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674669.1049740 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbph-0002Fe-6I; Thu, 01 Feb 2024 18:30:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674669.1049740; Thu, 01 Feb 2024 18:30:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbph-0002FV-3R; Thu, 01 Feb 2024 18:30:41 +0000
Received: by outflank-mailman (input) for mailman id 674669;
 Thu, 01 Feb 2024 18:30:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MzG+=JK=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rVbpg-00020T-Im
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 18:30:40 +0000
Received: from mail-qv1-xf32.google.com (mail-qv1-xf32.google.com
 [2607:f8b0:4864:20::f32])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ffd0bc85-c12f-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 19:30:40 +0100 (CET)
Received: by mail-qv1-xf32.google.com with SMTP id
 6a1803df08f44-6818a9fe380so7291216d6.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 10:30:40 -0800 (PST)
Received: from pm2-ws13.praxislan02.com
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 pf8-20020a056214498800b0068c75141e73sm25824qvb.2.2024.02.01.10.30.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 10:30:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ffd0bc85-c12f-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706812238; x=1707417038; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+F33WLb8SsbhR8svkMBYfGDsCEmngZ8q/7ZaljEVS3M=;
        b=UwPIDUq10isCpumWgi97FEc+KPHMV/lzjQUdUJom8FKLNOJFrxvg3FpDxy5n1gp0SE
         ILP/i+JESIXyEntN9bZ7ZnT0kSF9Lghvd2zwS+Ry47cSDo81pHYekxtpC4VRgBVOFAK/
         a1Yld2ffKmAMfgyouFIDep2M93QDSeBunhSO4rlgMxWgc3obZ0v6hJoL3n1Xv7GSDkMi
         OPccFDprcDcMguOjOwAyltWBf8EWXT58S4poeFpQKBBALojtDoGsdS2+p8gToes8vzkC
         uPGaBEHFBmakozTIu8ss+w5px7ICfrjYndxV+xBhNnPn2AQScBW7p7OEV2VSRyJHvWF4
         S6tA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706812238; x=1707417038;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=+F33WLb8SsbhR8svkMBYfGDsCEmngZ8q/7ZaljEVS3M=;
        b=u0KzJ0SFsLvWzqhs/El9QLGfcKEqRfX3qbSny+BaEjNoJdGm3O6rFscApUIUTrpQzg
         W6G5pQPE6ih1PCBMbbVG/K/gvZT67NSCah02mii74/K2cP4Pn1Vq9zpm3lXgq+gyCSAG
         XPT+z45wV1qjxAu0suGPhimK1LBdPDCB3DRcHFC3M4loetB6+/5KfsDfdG8FSGQ7uZoT
         8cEfIPWkkk+BUwrX29FLYmXHB6X8bSyzUky0xN5IBYJTLxpum/KtzPWWzNTFJCYyO6PF
         HgSfdW8teWeWgdW77qbHMfiuaUTc9vUyG/kXH3eKcwC0FMt+KjUJ3EzdUfqYKwaRF8Gs
         3Ahw==
X-Gm-Message-State: AOJu0YygHjLqaUkQdraYAfzk38lBy+7c1hE0yaC5XANj1aZlbcEJKohG
	3rdjr2+ZCagm4bTxfzDClSljNQ3KCgQtJzOsaqj3mGIhsikuCBauWSv//0im
X-Google-Smtp-Source: AGHT+IEShpw6JudIZoJMi92UdXTK/nz8GoPmUq8wT+imB1lMTJxLIqgl6uF2XZHxabDNwONIF0UnIA==
X-Received: by 2002:ad4:5746:0:b0:686:a472:1205 with SMTP id q6-20020ad45746000000b00686a4721205mr6139692qvx.0.1706812238079;
        Thu, 01 Feb 2024 10:30:38 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVCqkg7ZSjgpD/6/3COlNsVpzNskT++aPAlEYlQcK/J6/X50n8ZfzMbMgXQu2IqIT3VJLJ2pWTsNJ/JK/rhpjASjcHk5PBWTPpYhRO3amEmvhZkTGv+
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jandryuk@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH 1/4] block-common: Fix same_vm for no targets
Date: Thu,  1 Feb 2024 13:30:21 -0500
Message-ID: <20240201183024.145424-2-jandryuk@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201183024.145424-1-jandryuk@gmail.com>
References: <20240201183024.145424-1-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

same_vm is broken when the two main domains do not have targets.  otvm
and targetvm are both missing, which means they get set to -1 and then
converted to empty strings:

++10697+ local targetvm=-1
++10697+ local otvm=-1
++10697+ otvm=
++10697+ othervm=/vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4
++10697+ targetvm=
++10697+ local frontend_uuid=/vm/844dea4e-44f8-4e3e-8145-325132a31ca5

The final comparison returns true since the two empty strings match:

++10697+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o '' = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = '' -o '' = '' ']'

Replace -1 with distinct strings indicating the lack of a value and
remove the collescing to empty stings.  The strings themselves will no
longer match, and that is correct.

++12364+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o 'No target' = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = 'No other target' -o 'No target' = 'No other target' ']'

Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
---
 tools/hotplug/Linux/block-common.sh | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/tools/hotplug/Linux/block-common.sh b/tools/hotplug/Linux/block-common.sh
index f86a88c4eb..5c80237d99 100644
--- a/tools/hotplug/Linux/block-common.sh
+++ b/tools/hotplug/Linux/block-common.sh
@@ -112,14 +112,12 @@ same_vm()
                   "$FRONTEND_UUID")
   local target=$(xenstore_read_default  "/local/domain/$FRONTEND_ID/target"   \
                  "-1")
-  local targetvm=$(xenstore_read_default "/local/domain/$target/vm" "-1")
+  local targetvm=$(xenstore_read_default "/local/domain/$target/vm" "No Target")
   local otarget=$(xenstore_read_default  "/local/domain/$otherdom/target"   \
                  "-1")
   local otvm=$(xenstore_read_default  "/local/domain/$otarget/vm"   \
-                 "-1")
-  otvm=${otvm%-1}
-  othervm=${othervm%-1}
-  targetvm=${targetvm%-1}
+                 "No Other Target")
+
   local frontend_uuid=${FRONTEND_UUID%-1}
   
   [ "$frontend_uuid" = "$othervm" -o "$targetvm" = "$othervm" -o \
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 18:30:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 18:30:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674670.1049750 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbpm-0002Xv-Du; Thu, 01 Feb 2024 18:30:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674670.1049750; Thu, 01 Feb 2024 18:30:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbpm-0002Xm-Al; Thu, 01 Feb 2024 18:30:46 +0000
Received: by outflank-mailman (input) for mailman id 674670;
 Thu, 01 Feb 2024 18:30:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MzG+=JK=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rVbpl-0002Wb-2Q
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 18:30:45 +0000
Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com
 [2607:f8b0:4864:20::730])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 017bb2e7-c130-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 19:30:42 +0100 (CET)
Received: by mail-qk1-x730.google.com with SMTP id
 af79cd13be357-783d8e09a1cso73034085a.3
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 10:30:42 -0800 (PST)
Received: from pm2-ws13.praxislan02.com
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 pf8-20020a056214498800b0068c75141e73sm25824qvb.2.2024.02.01.10.30.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 10:30:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 017bb2e7-c130-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706812241; x=1707417041; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=PjjeoKX1m7W13o8Y7ZaEmX1lMzcVrUhBaLtbtuWZOxs=;
        b=WIQ8by9wMw2KvMGH49gSyXd36QEqZQA622acQ002jIMFMIxVWuGZul50oDt7i5ScWX
         VGBX8r3MhGIaeVc6xObwGA/QW+RORrZD9kH2tI7O+ujnhC9wZzu1BqQMBaPV0VHWEVlb
         hF5iTcRLKum26F2Qj6HhKkqsJf0v79pPU+9V48t/KcBMy2SlGUYKCKKoPOWJbXxRmZMZ
         v77EMJtgugJ0+G9Qd4aH8H5403vFnAXf/mNE5ahd8gINEbJ42b21gq4WEkpI7nkIi3aj
         SS0y37/8HIay1vvm2sEaAWPJ4c6tzXWR+4gnta7HyJE1tLGxJjRkNJcYvACCPh8LA/74
         EF+Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706812241; x=1707417041;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=PjjeoKX1m7W13o8Y7ZaEmX1lMzcVrUhBaLtbtuWZOxs=;
        b=LgMm/hph4edgBqGNgiJnXrh9mdiOpUFAJcWLUbieDmZbCsoDUKBQa/+Iu/yojkN9nw
         M2BLVh7RXypRDDoWUnTQ2dlJZFmm+gMDf20iwvfgI2BylIc/qdsAKIuYsecLMNCjjhxO
         ip+H95wwFFhgRgnEgBvP27LHzwvxAb796hfiHBWHgv06QSR+/J8HBRjOXMpOzsW7LTKr
         cafheP6xhrP5CHgJX4ek3Ysoo0/J77Mzw0Vcq6950MF83GOpl1EvbxOoYs2H/lQW5U2o
         FNR1qW24WI9ESyFaLfrLY4qubAc7ImTkCNXz4IDwEfWnPq6jf4ceJD0LNOCM8bmfOlwE
         PG8g==
X-Gm-Message-State: AOJu0YxuhG1zHXVPq4o3Dcyk3AGkGDmg3WxkRUwoGOSz04gPJvUy/FhZ
	N94oPdX7qO9Nceh/KFNazy3QbvGzhXQ4h/lZttweBuJ1U5nTVb61jVQyqDGC
X-Google-Smtp-Source: AGHT+IFHvs1BSlxFAbLomRF+WK4BCyWVl6PudvCCCdULyTh8v4FtjY//z2fGun7qgHHPllbSX1rsxg==
X-Received: by 2002:ad4:4ea4:0:b0:68c:61bd:7765 with SMTP id ed4-20020ad44ea4000000b0068c61bd7765mr6903583qvb.45.1706812240849;
        Thu, 01 Feb 2024 10:30:40 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUYh1WG/ROL1iDB4lPXW50gxnr9Y/V7AJ3B0CY7+Ub0PhR7RfgTo5nnJIcli8/j7t8CiihQDJFiPSft5ZniYYiK7NPm1J2tGxvtsXgRUsFt6+hfcP8Zy4Ivq4HcVhDdMpnJHprtF7g=
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jandryuk@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH 2/4] libxl: Add support for blktap vbd3
Date: Thu,  1 Feb 2024 13:30:22 -0500
Message-ID: <20240201183024.145424-3-jandryuk@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201183024.145424-1-jandryuk@gmail.com>
References: <20240201183024.145424-1-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch re-introduces blktap support to libxl.  Unlike earlier
versions, it does not link against any blktap library.  libxl changes
are needed to write to the vbd3 backend XenStore nodes.

blktap has three components.  tapdisk is a daemon implementing the disk
IO, NBD (Network Block Device), and Xen PV interfaces.  tap-ctl is a
tool to control tapdisks - creating, starting, stopping and freeing.
tapback manages the XenStore operations and instructs tapdisk to
connect.

It is notable that tapdisk performs the grant and event channel ops, but
doesn't interact with XenStore.  tapback performs XenStore operations
and notifies tapdisks of values and changes.

The flow is: libxl writes to the "vbd3" XenStore nodes and runs the
block-tap script.  The block-tap script runs tap-ctl to create a tapdisk
instance as the physical device.  tapback then sees the tapdisk and
instructs the tapdisk to connect up the PV blkif interface.

This is expected to work without the kernel blktap driver, so the
block-tap script is modified accordingly to write the UNIX NBD path.

backendtype=tap was not fully removed previously, but it would never
succeed since it would hit the hardcoded error in disk_try_backend().
It is reused now.

An example command to attach a vhd:
xl block-attach vm 'vdev=xvdf,backendtype=tap,format=vhd,target=/srv/target.vhd'

Format raw also works to run an "aio:" tapdisk.

Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
---
VHD support is important for OpenXT since there are lots of existing
VHDs which still need supporting.  tapdisk also supports encrypting VHDs
which is not available in QEMU.
---
 docs/man/xl-disk-configuration.5.pod.in   |  4 +++-
 tools/libs/light/libxl_device.c           | 14 ++++++++++----
 tools/libs/light/libxl_disk.c             | 20 +++++++++++++++-----
 tools/libs/light/libxl_linux.c            |  1 +
 tools/libs/light/libxl_types_internal.idl |  1 +
 tools/libs/light/libxl_utils.c            |  2 ++
 6 files changed, 32 insertions(+), 10 deletions(-)

diff --git a/docs/man/xl-disk-configuration.5.pod.in b/docs/man/xl-disk-configuration.5.pod.in
index bc945cc517..cb442bd5b4 100644
--- a/docs/man/xl-disk-configuration.5.pod.in
+++ b/docs/man/xl-disk-configuration.5.pod.in
@@ -232,7 +232,7 @@ Specifies the backend implementation to use
 
 =item Supported values
 
-phy, qdisk, standalone
+phy, qdisk, standalone, tap
 
 =item Mandatory
 
@@ -254,6 +254,8 @@ and "standalone" does not support specifications other than "virtio".
 Normally this option should not be specified, in which case libxl will
 automatically determine the most suitable backend.
 
+"tap" needs blktap's tapback to be running.
+
 
 =item B<script>=I<SCRIPT>
 
diff --git a/tools/libs/light/libxl_device.c b/tools/libs/light/libxl_device.c
index 13da6e0573..ae2b71b0bf 100644
--- a/tools/libs/light/libxl_device.c
+++ b/tools/libs/light/libxl_device.c
@@ -328,9 +328,15 @@ static int disk_try_backend(disk_try_backend_args *a,
         return 0;
 
     case LIBXL_DISK_BACKEND_TAP:
-        LOG(DEBUG, "Disk vdev=%s, backend tap unsuitable because blktap "
-                   "not available", a->disk->vdev);
-        return 0;
+        if (a->disk->format != LIBXL_DISK_FORMAT_RAW &&
+            a->disk->format != LIBXL_DISK_FORMAT_VHD)
+            goto bad_format;
+
+        if (libxl_defbool_val(a->disk->colo_enable))
+            goto bad_colo;
+
+        LOG(DEBUG, "Disk vdev=%s, returning blktap", a->disk->vdev);
+        return backend;
 
     case LIBXL_DISK_BACKEND_QDISK:
         if (a->disk->script) goto bad_script;
@@ -478,7 +484,7 @@ char *libxl__device_disk_string_of_backend(libxl_disk_backend backend)
 {
     switch (backend) {
         case LIBXL_DISK_BACKEND_QDISK: return "qdisk";
-        case LIBXL_DISK_BACKEND_TAP: return "phy";
+        case LIBXL_DISK_BACKEND_TAP: return "vbd3";
         case LIBXL_DISK_BACKEND_PHY: return "phy";
         case LIBXL_DISK_BACKEND_STANDALONE: return "standalone";
         default: return NULL;
diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
index ea3623dd6f..59ff996837 100644
--- a/tools/libs/light/libxl_disk.c
+++ b/tools/libs/light/libxl_disk.c
@@ -56,7 +56,9 @@ static void disk_eject_xswatch_callback(libxl__egc *egc, libxl__ev_xswatch *w,
             "/local/domain/%d/backend/%" TOSTRING(BACKEND_STRING_SIZE)
            "[a-z]/%*d/%*d",
            &disk->backend_domid, backend_type);
-    if (!strcmp(backend_type, "tap") || !strcmp(backend_type, "vbd")) {
+    if (!strcmp(backend_type, "tap") ||
+        !strcmp(backend_type, "vbd") ||
+        !strcmp(backend_type, "vbd3")) {
         disk->backend = LIBXL_DISK_BACKEND_TAP;
     } else if (!strcmp(backend_type, "qdisk")) {
         disk->backend = LIBXL_DISK_BACKEND_QDISK;
@@ -224,7 +226,7 @@ static int libxl__device_from_disk(libxl__gc *gc, uint32_t domid,
             device->backend_kind = LIBXL__DEVICE_KIND_VBD;
             break;
         case LIBXL_DISK_BACKEND_TAP:
-            device->backend_kind = LIBXL__DEVICE_KIND_VBD;
+            device->backend_kind = LIBXL__DEVICE_KIND_VBD3;
             break;
         case LIBXL_DISK_BACKEND_QDISK:
             device->backend_kind = LIBXL__DEVICE_KIND_QDISK;
@@ -368,9 +370,17 @@ static void device_disk_add(libxl__egc *egc, uint32_t domid,
                 assert(device->backend_kind == LIBXL__DEVICE_KIND_VIRTIO_DISK);
                 break;
             case LIBXL_DISK_BACKEND_TAP:
-                LOG(ERROR, "blktap is not supported");
-                rc = ERROR_FAIL;
-                goto out;
+                flexarray_append(back, "params");
+                flexarray_append(back, GCSPRINTF("%s:%s",
+                              libxl__device_disk_string_of_format(disk->format),
+                              disk->pdev_path ? : ""));
+
+                script = libxl__abs_path(gc, disk->script?: "block-tap",
+                                         libxl__xen_script_dir_path());
+                flexarray_append_pair(back, "script", script);
+
+                assert(device->backend_kind == LIBXL__DEVICE_KIND_VBD3);
+                break;
             case LIBXL_DISK_BACKEND_QDISK:
                 flexarray_append(back, "params");
                 flexarray_append(back, GCSPRINTF("%s:%s",
diff --git a/tools/libs/light/libxl_linux.c b/tools/libs/light/libxl_linux.c
index f7c92ba562..0b4c8bd045 100644
--- a/tools/libs/light/libxl_linux.c
+++ b/tools/libs/light/libxl_linux.c
@@ -207,6 +207,7 @@ int libxl__get_hotplug_script_info(libxl__gc *gc, libxl__device *dev,
 
     switch (dev->backend_kind) {
     case LIBXL__DEVICE_KIND_VBD:
+    case LIBXL__DEVICE_KIND_VBD3:
         if (num_exec != 0) {
             LOGD(DEBUG, dev->domid,
                  "num_exec %d, not running hotplug scripts", num_exec);
diff --git a/tools/libs/light/libxl_types_internal.idl b/tools/libs/light/libxl_types_internal.idl
index e24288f1a5..56dccac153 100644
--- a/tools/libs/light/libxl_types_internal.idl
+++ b/tools/libs/light/libxl_types_internal.idl
@@ -34,6 +34,7 @@ libxl__device_kind = Enumeration("device_kind", [
     (16, "VINPUT"),
     (17, "VIRTIO_DISK"),
     (18, "VIRTIO"),
+    (19, "VBD3"),
     ])
 
 libxl__console_backend = Enumeration("console_backend", [
diff --git a/tools/libs/light/libxl_utils.c b/tools/libs/light/libxl_utils.c
index e403bd9bcf..10398a6c86 100644
--- a/tools/libs/light/libxl_utils.c
+++ b/tools/libs/light/libxl_utils.c
@@ -295,6 +295,8 @@ int libxl_string_to_backend(libxl_ctx *ctx, char *s, libxl_disk_backend *backend
         *backend = LIBXL_DISK_BACKEND_PHY;
     } else if (!strcmp(s, "file")) {
         *backend = LIBXL_DISK_BACKEND_TAP;
+    } else if (!strcmp(s, "vbd3")) {
+        *backend = LIBXL_DISK_BACKEND_TAP;
     } else if (!strcmp(s, "qdisk")) {
         *backend = LIBXL_DISK_BACKEND_QDISK;
     } else if (!strcmp(s, "standalone")) {
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 18:30:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 18:30:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674671.1049761 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbpo-0002pc-OD; Thu, 01 Feb 2024 18:30:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674671.1049761; Thu, 01 Feb 2024 18:30:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbpo-0002pO-LI; Thu, 01 Feb 2024 18:30:48 +0000
Received: by outflank-mailman (input) for mailman id 674671;
 Thu, 01 Feb 2024 18:30:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MzG+=JK=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rVbpm-0002Wb-SE
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 18:30:46 +0000
Received: from mail-vk1-xa32.google.com (mail-vk1-xa32.google.com
 [2607:f8b0:4864:20::a32])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 02bc14c8-c130-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 19:30:45 +0100 (CET)
Received: by mail-vk1-xa32.google.com with SMTP id
 71dfb90a1353d-4c009d2053fso59430e0c.3
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 10:30:45 -0800 (PST)
Received: from pm2-ws13.praxislan02.com
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 pf8-20020a056214498800b0068c75141e73sm25824qvb.2.2024.02.01.10.30.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 10:30:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02bc14c8-c130-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706812243; x=1707417043; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=x/WTpgrQ9hdXXsQXJiReaHAHfLb66CO+RLEPfZJBiuA=;
        b=lT4AttseEeqf8kimHVc0cSwdcwGh7MYXgg7ygfsACQp19gyx1AEQ3W5P+7saEY4DW4
         u5rRqsbglRMCCUrA4jmkIQGwz7P2kgQNAjtgyDmXzqPhKIptlg8IBVQAgX4C/JL9FPpt
         Mb6OsV0kulBAVoCLH/OdKNZYicSAeeGsjun7Lx8RNjh4xHGQ58tWdlpzO2ZKTSj3U7zg
         XZVZPCcvMyaZpZCZdl1xaMUyokNR7jSqdKlLBKSZZPjj5LBY7aWw9uMK2jrX3V/qWHk4
         0Rz5MWwa4TrhwWs6J1IgrRMKqg1J/pt1kD7lTFtZUjmEjJOQH5Qxv8sOkX36Oob1X/2h
         lfGg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706812243; x=1707417043;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=x/WTpgrQ9hdXXsQXJiReaHAHfLb66CO+RLEPfZJBiuA=;
        b=cOjmmF/niaq/blT6IOZVZk1i/rM8TXP7rHuYkJr2eDS+uBoGrOJiDCKkzQfKOkVqtc
         +RfGT/GfudD8pkPfprw854/wcKikSGG4NwWzMe3G7sYwB6wPHBlq3RGLGRPdk4HxhlOc
         WcZKB9i1sMruaxWYwJ8UY7SwJ0EeSbGYf/EKc041+ZCKC7SrBGq+K3EtXlunbZYgtq7l
         pPNTo5wxhQU0fwSSTjy8qETJcT8vVCR7cI+8OfvwVDUOue5hJ6ySrEqceTfOd3AZdRPW
         FaT6YOqOjYy1rOLkqXoq07V8Jdos6nqxGk5gMn8g4FmKUzpFbfi9wtN29I5OpChTFQLL
         yDGg==
X-Gm-Message-State: AOJu0YzffBq7tTpttSXGynEbRdw03dn4CJgVQr3VrFA+HeiDm2dHNjt2
	F5tG2gEtMU2y4cNnyWOLiNYW1gFqwa9eZFsTOLlo+yXOw27Y+RN6mZ0/bP0l
X-Google-Smtp-Source: AGHT+IGRKPQeZ5ZOhUWCupkw2VecP6TbVYfEOTHsu6/G9/NRslWr/0BzNAcbU1uCJGRJ6PNU/TjkzA==
X-Received: by 2002:a05:6122:368e:b0:4b7:8d7c:346f with SMTP id ec14-20020a056122368e00b004b78d7c346fmr4904896vkb.9.1706812243220;
        Thu, 01 Feb 2024 10:30:43 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUyp/NEkVKDrtZPeVhzmpv+A9M40Qgqn8LMrsBBunY63KJgNr2pTm8jdli00xi81ZFpJmrw1oR5ql//3fkocsBW7AxeIuT4umM2pj1TWRkPrbi6kDEn
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jandryuk@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH 3/4] hotplug: Update block-tap
Date: Thu,  1 Feb 2024 13:30:23 -0500
Message-ID: <20240201183024.145424-4-jandryuk@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201183024.145424-1-jandryuk@gmail.com>
References: <20240201183024.145424-1-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Implement a sharing check like the regular block script.

Checking tapback inside block-tap is too late since it needs to be
running to transition the backend to InitWait before block-tap is run.

tap-ctl check will be removed when the requirement for the blktap kernel
driver is removed.  Remove it now as it is of limited use.

find_device() needs to be non-fatal allow a sharing check.

Only write physical-device-path because that is all that tapback needs.
Also write_dev doesn't handled files and would incorrectly store
physical-device as 0:0 which would confuse the minor inside tapback
---
 tools/hotplug/Linux/block-tap | 162 ++++++++++++++++++++++++++++++++--
 1 file changed, 153 insertions(+), 9 deletions(-)

diff --git a/tools/hotplug/Linux/block-tap b/tools/hotplug/Linux/block-tap
index 89247921b9..5eca09f0f6 100755
--- a/tools/hotplug/Linux/block-tap
+++ b/tools/hotplug/Linux/block-tap
@@ -37,10 +37,6 @@ check_tools()
     if ! command -v tap-ctl > /dev/null 2>&1; then
         fatal "Unable to find tap-ctl tool"
     fi
-    modprobe blktap
-    if ! tap-ctl check >& /dev/null ; then
-	fatal "Blocktap kernel module not available"
-    fi
 }
 
 # Sets the following global variables based on the params field passed in as
@@ -81,7 +77,105 @@ find_device()
     done
 
     if [ -z "$pid" ] || [ -z "$minor" ]; then
-        fatal "cannot find required parameters"
+        return 1
+    fi
+
+    return 0
+}
+
+count_using()
+{
+  local file="$1"
+  local f
+
+  local i=0
+  local base_path="$XENBUS_BASE_PATH/$XENBUS_TYPE"
+  for dom in $(xenstore-list "$base_path")
+  do
+    for dev in $(xenstore-list "$base_path/$dom")
+    do
+      f=$(xenstore_read_default "$base_path/$dom/$dev/params" "")
+      f=$(echo "$f" | cut -d ":" -f 2)
+
+      if [ -n "$f" ] && [ "$file" = $f ] ; then
+          i=$(( i + 1 ))
+      fi
+    done
+  done
+
+  echo "$i"
+}
+
+# tap_shared is used to determine if a shared tap can be closed
+# Since a stubdom and a guest both use the same tap, it can only
+# be freed when there is a single one left.
+tap_shared() {
+    [ $( count_using "$file" ) -gt 1 ]
+}
+
+check_tap_sharing()
+{
+  local file="$1"
+  local mode="$2"
+  local dev
+
+  local base_path="$XENBUS_BASE_PATH/$XENBUS_TYPE"
+  for dom in $(xenstore-list "$base_path") ; do
+    for dev in $(xenstore-list "$base_path/$dom") ; do
+      f=$(xenstore_read_default "$base_path/$dom/$dev/params" "")
+      f=$(echo "$f" | cut -d ":" -f 2)
+
+      if [ -n "$f" ] && [ "$file" = "$f" ] ; then
+        if [ "$mode" = 'w' ] ; then
+          if ! same_vm $dom ; then
+            echo "guest $f"
+            return
+          fi
+        else
+          local m=$(xenstore_read_default "$base_path/$dom/$dev/mode" "")
+          m=$(canonicalise_mode "$m")
+
+          if [ "$m" = 'w' ] ; then
+            if ! same_vm $dom ; then
+              echo "guest $f"
+              return
+            fi
+          fi
+        fi
+      fi
+    done
+  done
+
+  echo 'ok'
+}
+
+tap_create()
+{
+    if ! minor=$( tap-ctl allocate ) ; then
+        fatal "Could not allocate minor"
+    fi
+
+    # Handle with or without kernel blktap
+    minor=${minor#/run/blktap-control/tapdisk/tapdisk-}
+    minor=${minor#/dev/xen/blktap-2/tapdev}
+
+    # tap-ctl is spawning tapdisk which would hold the _lockfd open.
+    # Ensure it is closed before running tap-ctl spawn, which needs to be
+    # done in a subshell to continue holding the lock in the parent.
+    if ! pid=$( ( eval "exec $_lockfd>&-" ; tap-ctl spawn ) ) ; then
+        tap-ctl free -m "$minor"
+        fatal "Could not spawn tapdisk for $minor"
+    fi
+
+    if ! tap-ctl attach -p "$pid" -m "$minor" ; then
+        tap-ctl free -m "$minor"
+        fatal "Could not attach $pid and $minor"
+    fi
+
+    if ! tap-ctl open -p "$pid" -m "$minor" -a "$target" ; then
+        tap-ctl detach -p "$pid" -m "$minor"
+        tap-ctl free -m "$minor"
+        fatal "Could not open \"$target\""
     fi
 }
 
@@ -89,15 +183,57 @@ find_device()
 # the device
 add()
 {
-    dev=$(tap-ctl create -a $target)
-    write_dev $dev
+    local minor
+    local pid
+    local res
+
+    claim_lock "block"
+
+    if find_device; then
+        result=$( check_tap_sharing "$file" "$mode" )
+        if [ "$result" != "ok" ] ; then
+            do_ebusy "tap $type file $file in use " "$mode" "${result%% *}"
+        fi
+    else
+        tap_create
+    fi
+
+    # Create nbd unix path
+    dev=$( printf "/run/blktap-control/nbd%ld.%d" "$pid" "$minor" )
+
+    xenstore_write "$XENBUS_PATH/pid" "$pid"
+    xenstore_write "$XENBUS_PATH/minor" "$minor"
+    # dev, as a unix socket, would end up with major:minor 0:0 in
+    # physical-device if write_dev were used.  tapback would be thrown off by
+    # that incorrect minor, so just skip writing physical-device.
+    xenstore_write "$XENBUS_PATH/physical-device-path" "$dev"
+
+    success
+
+    release_lock "block"
 }
 
 # Disconnects the device
 remove()
 {
-    find_device
-    do_or_die tap-ctl destroy -p ${pid} -m ${minor} > /dev/null
+    local minor
+    local pid
+
+    claim_lock "block"
+
+    if tap_shared ; then
+        return
+    fi
+
+    minor=$( xenstore_read "$XENBUS_PATH/minor" )
+    pid=$( xenstore_read "$XENBUS_PATH/pid" )
+
+    [ -n "$minor" ] || fatal "minor missing"
+    [ -n "$pid" ] || fatal "pid missing"
+    do_or_die tap-ctl close -p "$pid" -m "$minor" > /dev/null
+    do_or_die tap-ctl detach -p "$pid" -m "$minor" > /dev/null
+
+    release_lock "block"
 }
 
 command=$1
@@ -110,6 +246,14 @@ parse_target "$target"
 
 check_tools || exit 1
 
+mode=$( xenstore_read $XENBUS_PATH/mode )
+mode=$( canonicalise_mode $mode )
+
+# needed for same_vm
+FRONTEND_ID=$(xenstore_read "$XENBUS_PATH/frontend-id")
+FRONTEND_UUID=$(xenstore_read_default \
+                    "/local/domain/$FRONTEND_ID/vm" 'unknown')
+
 case $command in
 add)
     add
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 18:30:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 18:30:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674672.1049771 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbpp-00036K-WD; Thu, 01 Feb 2024 18:30:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674672.1049771; Thu, 01 Feb 2024 18:30:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbpp-000366-SZ; Thu, 01 Feb 2024 18:30:49 +0000
Received: by outflank-mailman (input) for mailman id 674672;
 Thu, 01 Feb 2024 18:30:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MzG+=JK=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rVbpo-00020T-6r
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 18:30:48 +0000
Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com
 [2607:f8b0:4864:20::829])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 044cf4dc-c130-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 19:30:47 +0100 (CET)
Received: by mail-qt1-x829.google.com with SMTP id
 d75a77b69052e-42aa4a9d984so11308851cf.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 10:30:47 -0800 (PST)
Received: from pm2-ws13.praxislan02.com
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 pf8-20020a056214498800b0068c75141e73sm25824qvb.2.2024.02.01.10.30.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 10:30:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 044cf4dc-c130-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706812245; x=1707417045; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=DXQTUemsN74RnAn1dvvGsbLMWeFxaQCoGiu+7AK6fc4=;
        b=SjxheN/pBiJVZYu+sYdtU9tg1XNVwzY0B7Kh+2HmsLLciLH0jtqvy2EBGLXX2be/wg
         8t8+mEvC2IFcxiRqLjH5iMDDVFjZAQFByQOeKQY4W/3WVeiKCyMpUJHewjvyBuXia6Kv
         h9/+ygNJFtE9+G78RYVnv8oSdrhqjX5PLhqpzDCw9dZlPkBhSqFqtaYs3IEA8RdF7oAm
         l4sGB4nUFZkXfa7gwO5+aNvxs9DqKL2bBNXXB31636HJ0jCstFnkpymBDCvQf3LIwfbW
         aZ8Jg2GniWgc5Z64LXwI5cQbektaq5meX+AnnNyIEOwDZBulFp520Mi4pw2ADg3y9dHb
         mwiw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706812245; x=1707417045;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=DXQTUemsN74RnAn1dvvGsbLMWeFxaQCoGiu+7AK6fc4=;
        b=YBHrwK9bOWf0PpuiLY0Ad9Ux1s2q57y0aSp6QrZsZb3XKWfbqBljzPqyvYVffUc766
         VB5sH4rj0l0rQjno0wA/qhlsLJO5A0VfkDYP+rwe/qDoTECBTpc10cHyifD3XdTeNWYv
         krOh9M9y0p1y+he+vCTlfEaMMFdsGAI0vYyaYuxSPlInrgs9JVuA9rngtwB0Wv5rpwTi
         DupH8t6xe4m6t9GlOJN/GU65ZkCVSzj/In85x5J5NVd3DEEHZvHCh9Cagy7hlFGmD5Ys
         UlnKnXkySx0ZIbPfHOsYIpDEG6lGs3Yz5BRyPYKoce6jaQyzYPC9nGNFYuOMNQKXWcDV
         LDwA==
X-Gm-Message-State: AOJu0YwMzOYeGY5lfYgXEBoI5FUztr4kVozuBa2u0P+zadDcXzCUHl9N
	eGsoMUJFzWxsOEG5YvFP+RlrpalZhWQpzeTkcQdRZi40J6di0ybQ8zJ938oz
X-Google-Smtp-Source: AGHT+IEWx00EhNjwiB+5VOyp/liVH8EKonyCySzNRm7Zbu9GgKjuetHeeXTeHp5nysA4xxULR3fIIg==
X-Received: by 2002:ad4:5be2:0:b0:681:24e9:f6fe with SMTP id k2-20020ad45be2000000b0068124e9f6femr5967924qvc.35.1706812245609;
        Thu, 01 Feb 2024 10:30:45 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWYFDkfLuco0XGfZ2n8zscgoQvWNCJdEScd5K+8Z4tSpZGAoRMcrxsEdlXh8eCkD3sD4dIKh1Yows0O6mWbb1ExfPkdNMvWeZrfNSwJROCCT3LUn65Df4Ii+lALcKfJNxc9wJUgnOA=
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Jason Andryuk <jandryuk@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH 4/4] libxl: Support blktap with HVM device model
Date: Thu,  1 Feb 2024 13:30:24 -0500
Message-ID: <20240201183024.145424-5-jandryuk@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201183024.145424-1-jandryuk@gmail.com>
References: <20240201183024.145424-1-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

blktap exposes disks over UNIX socket Network Block Device (NBD).
Modify libxl__device_disk_find_local_path() to provide back the
QEMU-formatted NBD path.  This allows tapdisk to be used for booting an
HVM.

Use the nbd+unix:/// format specified by the protocol at
https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md

Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
---
 tools/libs/light/libxl_disk.c | 17 +++++++++++++----
 tools/libs/light/libxl_dm.c   |  1 -
 2 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
index 59ff996837..b65cad33cc 100644
--- a/tools/libs/light/libxl_disk.c
+++ b/tools/libs/light/libxl_disk.c
@@ -1317,7 +1317,8 @@ char *libxl__device_disk_find_local_path(libxl__gc *gc,
      * If the format isn't raw and / or we're using a script, then see
      * if the script has written a path to the "cooked" node
      */
-    if (disk->script && guest_domid != INVALID_DOMID) {
+    if ((disk->script && guest_domid != INVALID_DOMID) ||
+        disk->backend == LIBXL_DISK_BACKEND_TAP) {
         libxl__device device;
         char *be_path, *pdpath;
         int rc;
@@ -1337,10 +1338,18 @@ char *libxl__device_disk_find_local_path(libxl__gc *gc,
         LOGD(DEBUG, guest_domid, "Attempting to read node %s", pdpath);
         path = libxl__xs_read(gc, XBT_NULL, pdpath);
 
-        if (path)
+        if (path) {
             LOGD(DEBUG, guest_domid, "Accessing cooked block device %s", path);
-        else
-            LOGD(DEBUG, guest_domid, "No physical-device-path, can't access locally.");
+
+            /* tapdisk exposes disks locally over UNIX socket NBD. */
+            if (disk->backend == LIBXL_DISK_BACKEND_TAP) {
+                path = libxl__sprintf(gc, "nbd+unix:///?socket=%s", path);
+                LOGD(DEBUG, guest_domid,
+                     "Directly accessing local TAP target %s", path);
+            }
+        } else
+            LOGD(DEBUG, guest_domid,
+                "No physical-device-path, can't access locally.");
 
         goto out;
     }
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 14b593110f..f7c796011d 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -1915,7 +1915,6 @@ static int libxl__build_device_model_args_new(libxl__gc *gc,
                     continue;
                 }
 
-                assert(disks[i].backend != LIBXL_DISK_BACKEND_TAP);
                 target_path = libxl__device_disk_find_local_path(gc,
                                     guest_domid, &disks[i], true);
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 18:39:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 18:39:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674690.1049780 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbyX-0005Su-S9; Thu, 01 Feb 2024 18:39:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674690.1049780; Thu, 01 Feb 2024 18:39:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVbyX-0005Sn-PS; Thu, 01 Feb 2024 18:39:49 +0000
Received: by outflank-mailman (input) for mailman id 674690;
 Thu, 01 Feb 2024 18:39:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F6Mk=JK=proton.me=johnlpoole@srs-se1.protection.inumbo.net>)
 id 1rVbyW-0005Sh-9L
 for xen-devel@lists.xen.org; Thu, 01 Feb 2024 18:39:49 +0000
Received: from mail-4322.protonmail.ch (mail-4322.protonmail.ch [185.70.43.22])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 44ae1948-c131-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 19:39:45 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 44ae1948-c131-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me;
	s=protonmail; t=1706812782; x=1707071982;
	bh=cPs6kKUUuYzW4X1WmKezrCve1xfZwxiuH9eHp/d/dQc=;
	h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References:
	 Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:
	 Message-ID:BIMI-Selector;
	b=C60h4l9V47Uv8ngHpztHjRH6LgCBr5+wpZvTndIGbQHitu3Z9txpiQQ/+TdETi7Gi
	 4edUt1a7iTqz3iRmaVjmOPEROrCIyoZSMBY2AxVcfnTmaCS1UVmJ6MQokHtqF8OFJU
	 V9wYmX4+iLHn7jhW/J1Jn15oS/AfCfYsAZz+wmTxLdfIDKbRTvnL1ZbOEgGHLyHZcJ
	 /U+9gMdLv3Q83cFnjA76iwoCk9M+MAh2dJX31cG0ipgItA7xfPtz7mmhymzRmL4N01
	 HnZQzgLxeRn5YIlIqkPV0FMwcg2boi3+vKWUPnkLNCJqCkcBueZrt787hzC8Bsg67Y
	 z5ggLVfIvEAyA==
Date: Thu, 01 Feb 2024 18:39:20 +0000
To: David Woodhouse <dwmw2@infradead.org>
From: "John L. Poole" <johnlpoole@proton.me>
Cc: xen-devel@lists.xen.org
Subject: Re: Nullifying Recently Introduced Xen Headers Check
Message-ID: <cj1qg72oASoR0AT_B5_w6AnPm5tf8K-muPdrGWSGYozXJ6N8vaMUI3XMVBx2IoAYlKPAyUTewevznMbj6S0t3plMygEAS8_HtUVubMXo1dA=@proton.me>
In-Reply-To: <4b5b1fdadbabd53116d1ab79295b913c27947c4b.camel@infradead.org>
References: <fb379476-f6ae-4eb7-b3c6-6f0ad175964d@proton.me> <4b5b1fdadbabd53116d1ab79295b913c27947c4b.camel@infradead.org>
Feedback-ID: 80499891:user:proton
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


On Thursday, February 1st, 2024 at 9:18 AM, David Woodhouse <dwmw2@infradea=
d.org> wrote:

> (Thanks Andy for the explicit cc)
>=20
> On Thu, 2024-02-01 at 00:05 +0000, John L. Poole wrote:
>=20
> > cause Gentoo's build to error out. See line 24790:
> >=20
> > 5 | #error In Xen native files, include xen_native.h before other Xen h=
eaders
> >=20
> > at
> > https://salemdata.us/xen/xen_tools_20240128_Sun_174740.script.html.
> >=20
> > What I have done is create a patch for a draft Gentoo ebuild which
> > nullifies lines 4-6 by wrapping them in a comment:
>=20
>=20
> That isn't what the #error told you to do, though.
>=20
> 24788 In file included from ../qemu-xen/hw/xen/xen-operations.c:16:
> 24789 /var/tmp/portage/app-emulation/xen-tools-4.18.0/work/xen-4.18.0/too=
ls/qemu-xen/include/hw/xen/xen_native.h:5:2: error: #error In Xen native fi=
les, include xen_native.h before other Xen headers
> 24790 5 | #error In Xen native files, include xen_native.h before other X=
en headers
> 24791 | ^~~~~
>=20
> So it's hw/xen/xen-operations.h which is failing. As far as I can tell
> (visually and empirically because it does actually build elsewhere), it
> is doing what the #error said =E2=80=94 it is including xen_native.h befo=
re
> any other Xen headers.
>=20
> The first four non-comment lines of xen-operations.c should look
> something like this...
>=20
> #include "qemu/osdep.h"
> #include "qemu/uuid.h"
> #include "qapi/error.h"
>=20
> #include "hw/xen/xen_native.h"
>=20
> So... did you patch it so it doesn't start like that any more? Or does
> one of those first three files (perhaps qemu/osdep.h?) end up bringing
> in the Xen interface headers in a way that I didn't anticipate and
> which doesn't seem to happen elsewhere?
>=20
> I didn't cite the full gcc command line from line 24787 of your log
> because it's huge. Can you run a variant of that command to just give
> me the preprocessed output (-E -dD -o xen-operations.i).

Hi David,

To answer your questions: the patch consists only of this
modification of 3 lines in xen_native.h:

   https://921932.bugs.gentoo.org/attachment.cgi?id=3D883883

Otherwise, the 4.18 code tree is as released.

I'm without the know-how to run a variant of the emerge command
as you have requested, alas.  I just simply removed 3 lines that
were introduced into the 4.18 tree to achieve a successful build.
I posted to this list to learn what the negative impacts might be.=20

While the Xen Project "make" works, the Gentoo emerge
of app-emulation/xen-tools does not unless the three lines are
removed to simulate prior 4.17.3 and earlier code. =20

I suspect the Gentoo approach=20
of building tools first contributes to the problem.

Thank you,

John


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 18:49:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 18:49:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674695.1049790 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVc7c-00019g-NP; Thu, 01 Feb 2024 18:49:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674695.1049790; Thu, 01 Feb 2024 18:49:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVc7c-00019Z-Kl; Thu, 01 Feb 2024 18:49:12 +0000
Received: by outflank-mailman (input) for mailman id 674695;
 Thu, 01 Feb 2024 18:49:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i6jM=JK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVc7b-000187-RR
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 18:49:11 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 957dd787-c132-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 19:49:09 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-33b1d7f7366so43346f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 10:49:09 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 x3-20020adfffc3000000b0033aeda49732sm113134wrs.33.2024.02.01.10.49.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 10:49:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 957dd787-c132-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706813349; x=1707418149; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4kbF8WRlofSOI2ZY6m24DC6V7jABowHISkZjrGUMGUI=;
        b=Tz3Lv9WRljQZoswCRBQWAVkBf5C24PtEAD2R97fsTRXV5OLbRUuHsT7Hjsne8LTJaN
         ABr/M8E5M9sISQxjbNhvcd57KsAQMXRbGuOLTm2VFBbJv7EcEa26cIu740ydpFnO6aMD
         syxpSi6EGkbTXRVbdBY0OvYpq2zMmuUP5ptGA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706813349; x=1707418149;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=4kbF8WRlofSOI2ZY6m24DC6V7jABowHISkZjrGUMGUI=;
        b=UUG9CoJGKfQ+6vr2Ns7j6HWtsI3GVSB3otNtpicxDq/BkDx4b6JHRm1JkN68urEVIV
         SeDU/UzrYvu4gKp85q9UpKJFP23C8oPppcbwmfOD9GB8oEzAXxW5IZFHfsTZUWaaKQpE
         7BobBAcRojedI0jwVolk1RviOMUQ9BTWJOSY0MW1mb3JDJkpVxvkScOgMJQ+qfnhXicR
         iAe/loHPlLDgokxjBCQjF/wOIhcp2MgR0IobY1p7O1q5N94w6ChSvoI1hH3nHpBzST66
         pVcKtVMhUIH81Dc2tBYsaWB5jV8nxe/wlwlfwHJ5fc5wTY3KWAyztjE1yPYfxXppEWLX
         tg1A==
X-Gm-Message-State: AOJu0YxQo1eaSaUVKs94mTIL9U+J7OY4bMNbCE6MlWpjs6cMC0F5cm04
	0HC45OJed/ObLL3Wh/1HxxOKSXzC+Lf3DyRmiOENX/sin0laEHVxUAmP58/QPlg=
X-Google-Smtp-Source: AGHT+IGxBXdIwtTQ6o1KBNbzRmTJqYuy8p/runcsXzhrvzmOrmQOjrXr4tM89NtBCeqiWyDMgHGozQ==
X-Received: by 2002:a5d:58d7:0:b0:33a:e8fd:f43 with SMTP id o23-20020a5d58d7000000b0033ae8fd0f43mr3730115wrf.6.1706813349060;
        Thu, 01 Feb 2024 10:49:09 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCV+65DleEUNL5aYyRpLUeAYyfxikb5GAdjZi4txfWBB0NiIw5VRp4nwPk4VD4l+uINn3SEi7D/0Fua/0eJ9mnGlZA5HRPxI5ahhlY+ExvhaIRA7YmnLozg//o2syCh1G6S6wRlY7qPc2M+Axg5V+OdRAHTfRTN61L/c6A==
Message-ID: <bd5ca25a-027d-4d11-abef-83efcda9baa7@citrix.com>
Date: Thu, 1 Feb 2024 18:49:08 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/4] libxl: blktap/tapback support
Content-Language: en-GB
To: Jason Andryuk <jandryuk@gmail.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>
References: <20240201183024.145424-1-jandryuk@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240201183024.145424-1-jandryuk@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01/02/2024 6:30 pm, Jason Andryuk wrote:
> Jason Andryuk (4):
>   block-common: Fix same_vm for no targets
>   libxl: Add support for blktap vbd3
>   hotplug: Update block-tap
>   libxl: Support blktap with HVM device model

If libxl is learning how to drive tapdisk, then it's time for:

https://github.com/xenserver/xen.pg/blob/XS-8/patches/xentop-vbd3.patch

to make it's way upstream.  Otherwise, there's an absence of disk info
in xentop.

I previously kept it back because vbd3 was specific to XenServer.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 19:08:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 19:08:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674700.1049800 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVcQG-0005vJ-Eh; Thu, 01 Feb 2024 19:08:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674700.1049800; Thu, 01 Feb 2024 19:08:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVcQG-0005vC-C8; Thu, 01 Feb 2024 19:08:28 +0000
Received: by outflank-mailman (input) for mailman id 674700;
 Thu, 01 Feb 2024 19:08:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i6jM=JK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVcQE-0005sx-HE
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 19:08:26 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 459d5eca-c135-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 20:08:24 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33ae3cc8a6aso896970f8f.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 11:08:24 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 m27-20020a056000025b00b0033b11e91c0bsm132728wrz.81.2024.02.01.11.08.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 11:08:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 459d5eca-c135-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706814503; x=1707419303; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=TjUYRwPbanYV4UigJIlfR3BoASsaObFgt/HvfYLJikE=;
        b=uY9kaSkCPWDyoAYBnS7FzZGDnrCu2HR6BRQA6RnmQBfD7VrHgEj6/xykhhGU31ITvv
         4OBO+oVoPJLAqNirHL3hYhi7gSR4YgPFhF4ObCSoj1Y8fM+9ItEDYyVm98KoTSl0+Sqq
         3nLKcKU8XhujvuzPit0ci1j1SXEGesC2za2Zc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706814503; x=1707419303;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=TjUYRwPbanYV4UigJIlfR3BoASsaObFgt/HvfYLJikE=;
        b=deaP2MNM7URqXTa2Ns0+q6u7AUMTLIy7P2ANUw8TBaBZrekzLU6UrdmnzrEx2qCHON
         4iwy91Cd1CMpB2x1JjpKrs5W5X7S/LldgzVEUd9QN3ToORkKFUErxhLDp0iutTxwIUGT
         EpJmadwpjJ7sgH662iDjxVnc0i1HPgMGAxv5O4TuBW0nL6h21/jC2GdB9x+Rl2bcDcUN
         yqd0D0SVZw0YEITI553DI6WBrrPKjrasF5vTiVOF/x40iW2I9r0VSdTTAWEJCZT8JWxJ
         NFYNTcT25/d1uoS+y6hhg0wcYeqUift0Mv/nnGgiBbmw/Bl9BjBY/3HqT2A8iqNeE76L
         J/Mg==
X-Gm-Message-State: AOJu0YxzF9CFchZrGdmJZ0lTBzi2OmYaaiSp99LTaBEtmLV6tjjnY4Ya
	J11W83ow1/I/i8TiGweQjaL1hkPO3BmFaloFC51oXkqq0aJ3iR6rlDqiJroMapOpIu+T+sCRKfs
	7
X-Google-Smtp-Source: AGHT+IEQ0r2joF0nW4AsnurSs3Qf2xAu9Y126gSBYOe3bcQnbD5lvSf/jnR5/xNnegM6kp0C6UWlJg==
X-Received: by 2002:a5d:522e:0:b0:33a:e4ba:16f5 with SMTP id i14-20020a5d522e000000b0033ae4ba16f5mr2608244wra.3.1706814503681;
        Thu, 01 Feb 2024 11:08:23 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCX3+fmlm+QzNwJVSp400Cnnt7C4iBe+aMqlg8m1DSqr2mnDEtztiuvBP1cB46ik6Gja8Vr3tLHPC0t3iGG3I2bJqJnVI6laTg8=
Message-ID: <0f3a09a0-142e-4534-8f14-4d73dac00990@citrix.com>
Date: Thu, 1 Feb 2024 19:08:23 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Fix some typos in comments
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Frediano Ziglio <frediano.ziglio@cloud.com>,
 George Dunlap <george.dunlap@citrix.com>
Cc: xen-devel@lists.xenproject.org
References: <20240201145943.738531-1-frediano.ziglio@cloud.com>
 <e31aba88-1607-4a74-b2f9-66e754c89e9c@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <e31aba88-1607-4a74-b2f9-66e754c89e9c@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01/02/2024 3:07 pm, Andrew Cooper wrote:
> On 01/02/2024 2:59 pm, Frediano Ziglio wrote:
>> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
>> ---
>>  arch/x86/include/arch/processor.h | 2 +-
>>  include/xen/hvm/params.h          | 2 +-
>>  include/xtf/console.h             | 2 +-
>>  include/xtf/extable.h             | 4 ++--
>>  4 files changed, 5 insertions(+), 5 deletions(-)
> This looks like an XTF patch?

Given that it applies cleanly, that will be a yes.

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com> and committed.

Thanks.


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 19:46:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 19:46:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674704.1049811 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVd17-00079W-7m; Thu, 01 Feb 2024 19:46:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674704.1049811; Thu, 01 Feb 2024 19:46:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVd17-00079P-4m; Thu, 01 Feb 2024 19:46:33 +0000
Received: by outflank-mailman (input) for mailman id 674704;
 Thu, 01 Feb 2024 19:46:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i6jM=JK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVd16-00079J-0S
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 19:46:32 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 97ecce65-c13a-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 20:46:30 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-40fc2e36eeeso4438585e9.3
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 11:46:30 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 p14-20020a05600c468e00b0040fbad272f6sm2749569wmo.46.2024.02.01.11.46.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 11:46:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97ecce65-c13a-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706816788; x=1707421588; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bfkwkaqY6LXddZTpjAutD7ZofgChudRBdiKliQIp684=;
        b=kJodZGNaoJ/xpTXoOG+p9tZorYuISKox5tL8+di1Z8Z69TndK9BbKQfxNtxLPbK5Uw
         BelAq78jxgpa7WMQ2TTNhbCl0BNooZaXMSScarfUrBvFPG6nNGlMakWQ9PhGOTV7TtkR
         RW3H6ni/N9sq8y7FM2RgvztAVEWvF6DWYbZTc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706816788; x=1707421588;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bfkwkaqY6LXddZTpjAutD7ZofgChudRBdiKliQIp684=;
        b=TrbiwdsEUueoYFiCDQPPx2U1fF1WvzDOR3WFbofZssJspcHtcl4opSYitJWYENHVDt
         6WpKAtkZektX8WZAbPVh6SA7SJ0Wovz6+zVkD1VU8tM89xB9DUZHfEcbe6q6keMfoeMT
         Dx097D17qui8zqzm+ILcvMndHIREQDb327No4IJPrfwSBK2MCVTIZ6B4TiQWsLWUWuci
         a/4Y8t7fuqMZVfs0cxjtJPbt/ZVDkg1PBKAzFjaBemQrR/xUX6s6g/tiUD1/943dJglQ
         Drmku0RG1RP/6ozy2Jg5ykN3bRgroWhKB6gsG6YbK7F2f5U6DKmaOT1U4iXJ8AA233Jq
         EaSw==
X-Gm-Message-State: AOJu0YxxYV+XcV/ChlPIJGdVWV/FgTqfkTXqIQpsxFtRNX2FKY3LAZ9u
	Z7tzqrBl2PT5vJ2Mguno1Yuccsfq+cPvw+ytsoB4Kuu4FIUSvOda18flwFdM4lyZU4YxKjybTLG
	X
X-Google-Smtp-Source: AGHT+IEJaaNAy11CuMjtIhDZG94zdoM/cT/VjnMvTfACwIfYbdb20LD5RDeMkTWTR4pnlkY/7Rrk/A==
X-Received: by 2002:a05:600c:3504:b0:40e:e834:3d86 with SMTP id h4-20020a05600c350400b0040ee8343d86mr29092wmq.37.1706816788516;
        Thu, 01 Feb 2024 11:46:28 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXOypleYe3f63m/2LIhLM3XdOH2lum/UvKlw0/fdUoCFsOZarZM4b7qt2jWUNm2edi9MXwbvxlDInQ8I7PwudzYrWlycp635ybDBhRlLV2OxIC3WIPneM4deYsKla1Mm2yielMOUWYEJA9gidZS1yKgadoVvsg=
Message-ID: <95df07a3-0fae-4f1b-bca7-04a72a333dff@citrix.com>
Date: Thu, 1 Feb 2024 19:46:27 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/bitmap: Drop unused headers
Content-Language: en-GB
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: George Dunlap <George.Dunlap@citrix.com>, Jan Beulich
 <JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Julien Grall <julien@xen.org>
References: <20240201103506.549662-1-andrew.cooper3@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240201103506.549662-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01/02/2024 10:35 am, Andrew Cooper wrote:
> Nothing in bitmap.h uses lib.h, and there's no point including types.h when we
> need to include bitops.h anyway.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Wei Liu <wl@xen.org>
> CC: Julien Grall <julien@xen.org>
> ---
>  xen/include/xen/bitmap.h | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/xen/include/xen/bitmap.h b/xen/include/xen/bitmap.h
> index b9f980e91930..9f749e3913d8 100644
> --- a/xen/include/xen/bitmap.h
> +++ b/xen/include/xen/bitmap.h
> @@ -3,8 +3,6 @@
>  
>  #ifndef __ASSEMBLY__
>  
> -#include <xen/lib.h>
> -#include <xen/types.h>
>  #include <xen/bitops.h>

Turns out this went too far, and breaks PPC.  Other arches look ok.

https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/6076263594

bitmap.h uses mem{set,cpy}() so needs string.h.  That's a bug in this
patch specifically.

However, cpumask.h transitively picks up IS_ALIGNED() (so needs
macros.h) and ASSERT() which sadly is still in lib.h

I guess moving the mess from bitmap.h to cpumask.h is a (minor) improvement.

But this comes back to the header tangle which prevented moving BUG() in
the first place.  Sadly there's been no reply to my question in the
debugger.h removal, but I'm going to get that committed and then we can
re-evaluate.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 21:12:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 21:12:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674710.1049821 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVeLu-0008NT-8r; Thu, 01 Feb 2024 21:12:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674710.1049821; Thu, 01 Feb 2024 21:12:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVeLu-0008NM-4s; Thu, 01 Feb 2024 21:12:06 +0000
Received: by outflank-mailman (input) for mailman id 674710;
 Thu, 01 Feb 2024 21:12:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i6jM=JK=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVeLt-0008NG-HG
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 21:12:05 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8b9aad2c-c146-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 22:12:03 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-5101cd91017so1703613e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 13:12:03 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 v7-20020a05600c444700b0040e541ddcb1sm612832wmn.33.2024.02.01.13.12.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 13:12:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8b9aad2c-c146-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706821923; x=1707426723; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nSJ2hZZ2VYITCzpQNgQpqsT/bQ+Ovwqx8owNpGRLy7I=;
        b=vVRGOxYrPd+/P57aZn6eefbcQ9dUCBKJG+236rChXEkwzL7EjuhpZpOhO7+voDF3Mm
         IMu1lzrBQPv/oYY7Tj2W5cQQmdO64RhF8QxvX4gRJ9bMLwoCbh93/Oht17dfQXm2KR/n
         /lvzmllFLu4VgvDziE4sv8wJuAL/YPp+QOlEw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706821923; x=1707426723;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nSJ2hZZ2VYITCzpQNgQpqsT/bQ+Ovwqx8owNpGRLy7I=;
        b=QDJzsCz8wAWysnQZgEzuAJbBWIBnUgtYv1x/NV15NNsk3QFMyvMc7+xAPawCZ1/srG
         aPDz1zEoa02xF+bq0Qfb0iW2Jh1EFfOKSy9rM1yHSRZ+huLT994ChXBkieFJFcqID72l
         jS6/O9HOMg0Wp5WxaWcHrDWMQ866iWaebhZEdCQFV7jRRJ4pnIWMk2Qe5j5F6kiohZTG
         KnbGywbQZmaHZD6W/DFzJXVXQbN//G06dvADAa6jF4q2x41Dvu16fzhi+n4AjvVhO3Ij
         nVoJSa4hWUE/8hs9L9Lw8E9RkWFpAl+g22Xmi2GvBYmti26fWnoseUhJ6tANv4b7UhMy
         ch5A==
X-Gm-Message-State: AOJu0YyTh7U1/yIC0PE3tWUvcwGfPH5tnY4x20myKUWRhx02BDkL48EV
	6Kk61nkiGpM710xP2JojOXVxKrDIh98JH1C0eogNL2c602SiLmeKqvhxUEPizsQ=
X-Google-Smtp-Source: AGHT+IF1qYQV3WYRGe2k4kQs/eIQkj1dW54V5m//TWFyxZZPuqL1Q5/RdiTXGTpJLG7kMyTD1xs3WA==
X-Received: by 2002:a05:6512:1103:b0:511:1796:aad6 with SMTP id l3-20020a056512110300b005111796aad6mr2983367lfg.35.1706821922811;
        Thu, 01 Feb 2024 13:12:02 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXJaWVxA3z8SUIiotiHS+oUN5oWQIZU7R+UHnTuUULPQ3waCjMrvQuICCMI6vqFg+C2Ir4B/vQtMAFp/OUk9YSJFKsLlJhxeo5RuIryYA+Fb2iNDHHOA+x94/F4uHEQoIh24qGt
Message-ID: <e42ee8c5-10b0-4369-8cc9-7b298fa68ef7@citrix.com>
Date: Thu, 1 Feb 2024 21:12:02 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86: arrange for ENDBR zapping from
 <vendor>_ctxt_switch_masking()
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <c19cf600-5971-457b-936d-77a035ab6913@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <c19cf600-5971-457b-936d-77a035ab6913@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16/01/2024 4:53 pm, Jan Beulich wrote:
> While altcall is already used for them, the functions want announcing in
> .init.rodata.cf_clobber, even if the resulting static variables aren't
> otherwise used.
>
> While doing this also move ctxt_switch_masking to .data.ro_after_init.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

>
> --- a/xen/arch/x86/cpu/amd.c
> +++ b/xen/arch/x86/cpu/amd.c
> @@ -258,6 +258,11 @@ static void cf_check amd_ctxt_switch_mas
>  #undef LAZY
>  }
>  
> +#ifdef CONFIG_XEN_IBT /* Announce the function to ENDBR clobbering logic. */
> +static const typeof(ctxt_switch_masking) __initconst_cf_clobber __used csm =
> +    amd_ctxt_switch_masking;
> +#endif

If we gain more of these, I suspect we'll want a wrapper for it.

Irritatingly you can't pass parameters into global asm, because the nice
way to do this would be an _ASM_PTR in a pushsection.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 21:40:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 21:40:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674715.1049830 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVenL-0005Jq-Dr; Thu, 01 Feb 2024 21:40:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674715.1049830; Thu, 01 Feb 2024 21:40:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVenL-0005Jj-BA; Thu, 01 Feb 2024 21:40:27 +0000
Received: by outflank-mailman (input) for mailman id 674715;
 Thu, 01 Feb 2024 21:40:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MzG+=JK=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rVenK-0005Jd-Aj
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 21:40:26 +0000
Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com
 [2607:f8b0:4864:20::1136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 812dd29a-c14a-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 22:40:24 +0100 (CET)
Received: by mail-yw1-x1136.google.com with SMTP id
 00721157ae682-6040d9e52b9so14765167b3.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 13:40:24 -0800 (PST)
Received: from pm2-ws13.praxislan02.com
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 h7-20020a05620a21c700b007853eb72c49sm154978qka.46.2024.02.01.13.40.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 13:40:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 812dd29a-c14a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706823622; x=1707428422; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=sOBkl/FI31A16UeVSXNN4wI40FVp19VWKB4FKGEq7Co=;
        b=k1JoC9othNW5gzz/XkYnAXF4rEl5+NZBYdNwP4Uu/BRRbIbF0AIkc6HusIr+Gn57i7
         pJ8I6op4aMUk0iEmDrpqtcKJ4STYJWudp9XKKzaMagUxZ6l/Y+bh6608idQzXk49Q63G
         ZPgE5G1wL3D15V9/BK4YLZbHUy8obMXVILRX8CfpSrTw7NRYb49XLtZMKKzefiDkUT81
         J4PnI6XBzSEJ3BjGB7h6Iy6mMu+E2bLAvqKf+y1ZW7Y+nJEfpw4JMqtRHHk6YsUtXLKt
         mmUWF9UYhHT8B3gkD1dhJzLOJ5QHafewFbyATyVionLtRD1SP5/2UL+8WHAmM4O8W8DB
         z/QQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706823622; x=1707428422;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=sOBkl/FI31A16UeVSXNN4wI40FVp19VWKB4FKGEq7Co=;
        b=ZYHiD9CDzqK72BMFiKMWV1msX+SnPUJdkr+yGUvKBwslkHSO4S3VrhHQfvR47GHiWM
         bTiLzBldw9Mad3yV3F8mLisMBFqpxeHY6jHpcAE4aby43kgXwLlH35nQIIn5w/qH+8O+
         UHJZXksNVzrOhmAY1N4GQANf6Mj21CrCYcoz1uXkwyVJHzKGjPxdyOovWILrqNrhR7rW
         8QDCUNsr4KKvGmPQ5t/zItae/kQhsmwn/rnE1QVsqjTUNimqFcGTJPDMnNoiD2e5QfJP
         yRLgaNF/Q0bx5IsUCAdoinQNLPwS5duBUBVzH+H+AdiLF8VnnqwyWgc04hXgQ1RQogO1
         1u2Q==
X-Gm-Message-State: AOJu0YwTIleOlfVpLTOy+t8o0o7cGSTrqPDS3KKwLOxbGcZqx/0T3oT0
	k2uLOqfq0o85HKbiUiJXuONILw+lQG4WdtwWgpN5nUzTJPlD6eDP5eRix1LR
X-Google-Smtp-Source: AGHT+IHXu0MiyHUQqw/Xz8M9a/POsPkTtJhpLdNaLRvqNDZa39Uf6TbkyqqY5jMPckdRNIhOxZyyvg==
X-Received: by 2002:a81:ad08:0:b0:602:9161:2e73 with SMTP id l8-20020a81ad08000000b0060291612e73mr419591ywh.29.1706823621845;
        Thu, 01 Feb 2024 13:40:21 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCW9l/sZeUwluN+qLJsGDA8KBjAkKBO/JxjefS6jLoalY8FJCijPf3GVLkhcwPr0IY0aHOHSfjxNfvAMrRgsjFRLs7k2lkOMyBNh+5etLeAF4ViLtqqlbBNsM5+y7O++8sGlKBFsl9T6LWgF82dxnFZpcbPKNr+vQtqoqGwUY53VVIe+BKq10oXqWYKJTFMYOnu4VC+FmbX6f0BkpAxRFnJrWJ51te4Z0NfI7xaSwZ8hCJ8hpxZ3HPIyR9pTdbgQUkqCOqNnJn5yvRrU6NTJfjepZh1tmTuANatUvTEG5uAGNAbneF1XlErOYzo=
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: marmarek@invisiblethingslab.com,
	Jason Andryuk <jandryuk@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v2 0/3] libxl: Stubdom cd-rom changing support
Date: Thu,  1 Feb 2024 16:40:01 -0500
Message-ID: <20240201214004.238858-1-jandryuk@gmail.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

These patches enable cd-rom media changing for an HVM with a linux
stubdom.

v1 didn't support these empty drives.  The code came out of OpenXT which
has a hack - null.iso.  null.iso is an ISO with no contents.  OpenXT
doesn't actually eject the cd-rom, it just inserts null.iso.  This patch
set has QEMU present empty media.

The first patch creates an empty file, /run/xen/empty-cdrom.%u, for Phy
drives that are "empty".  The place holder simplifies things because the
block scripts don't work for an empty params.  Even if the scripts were
modified for that, a stubdom will timeout on startup when the empty
disk/blkback never connects.  The empty file works around these issues.

The second patch allows use of Phy backend drives for a cd-rom.  This
works for non-stubdom HVMs.  Actually special casing stubdoms didn't
work.

The third patch expands the cd-rom changing code to support the stubdom
case.

To change the cd-rom medium, libxl will:
 - QMP eject the medium from QEMU
 - block-detach the old PV disk
 - block-attach the new PV disk
 - QMP change the medium to the new PV disk by fdset-id

xl cd-eject follows the above through connecting the new PV disk,
empty-cdrom.%u.  It skips the QMP media change.  This keeps the xenstore
entries which are needed to identify that a cd-rom drive is present.  If
the xenstore entries were removed on eject, libxl wouldn't find the
device (hdc) for a subsequent cd-insert.

The QMP change insert uses fdset-id STUBDOM_FDSET_CD + $disk - 'a'.
That is, hda -> 'a', so
STUBDOM_FDSET_CD + 'a' - 'a' = STUBDOM_FDSET_CD.
For hdc:
STUBDOM_FDSET_CD + 'c' - 'a' = STUBDOM_FDSET_CD + 2.

The stubdom must internally handle adding /dev/xvdc to the appropriate
fdset inside QEMU.

A script like this:
https://github.com/OpenXT/xenclient-oe/blob/master/recipes-core/initrdscripts/initramfs-stubdomain/qemu-xvdc-add-fd.sh

Can be called by busybox mdev configured like this:
https://github.com/OpenXT/xenclient-oe/blob/master/recipes-core/busybox/files/mdev.conf

(OpenXT mdev as the hotplug helper works, but with a ~Qubes stubdom, I
had to run mdev as a daemon, mdev -d.)

Linux locks the cd-rom by default?  That means the QMP eject commands
fail, but then Linux unlocks.  Re-running a second time works.  Windows
doesn't do that.

There are spurious messages sometimes like:
libxl: error: libxl_qmp.c:1837:qmp_ev_parse_error_messages: Domain 5:Could not dup FD for /dev/fdset/8002 flags 0: No such file or directory

libxl doesn't know when the stubdom has setup the fdset.  Since it gets
those errors, it'll retry adding to the fdset.

Jason Andryuk (3):
  libxl: Create empty cdrom file for stubdom
  libxl: Allow Phy backend for CDROM devices
  libxl: Enable stubdom cdrom changing

 docs/misc/stubdom.txt             |  16 ++
 tools/libs/light/libxl_device.c   |  17 +-
 tools/libs/light/libxl_disk.c     | 345 +++++++++++++++++++++++++++---
 tools/libs/light/libxl_domain.c   |   4 +
 tools/libs/light/libxl_internal.h |   1 +
 5 files changed, 344 insertions(+), 39 deletions(-)

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 21:40:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 21:40:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674716.1049841 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVenP-0005Z8-Nk; Thu, 01 Feb 2024 21:40:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674716.1049841; Thu, 01 Feb 2024 21:40:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVenP-0005Yx-L1; Thu, 01 Feb 2024 21:40:31 +0000
Received: by outflank-mailman (input) for mailman id 674716;
 Thu, 01 Feb 2024 21:40:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MzG+=JK=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rVenN-0005YC-PJ
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 21:40:29 +0000
Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com
 [2607:f8b0:4864:20::731])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 83c67fe8-c14a-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 22:40:28 +0100 (CET)
Received: by mail-qk1-x731.google.com with SMTP id
 af79cd13be357-783d4b3a00bso100070885a.3
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 13:40:28 -0800 (PST)
Received: from pm2-ws13.praxislan02.com
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 h7-20020a05620a21c700b007853eb72c49sm154978qka.46.2024.02.01.13.40.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 13:40:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83c67fe8-c14a-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706823627; x=1707428427; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vkexTyneUo8yDBZMkq46yZq63xLJ/JfR3uJQ589GQF4=;
        b=aMbHaPRdMZxgH0MmI0cZagUUrvBf+uCDs2Li3M/7v/JjuaHESdDmjwNSkHTCyS0vu9
         U46JZzZTN8Ka0BfP0AcGJata3CNGz/UXm9OBTtNKOXj3VuRh22DYCAeA/Sch57mV2lyZ
         pvlz6vjBn/2UlmP32a8QedqyP8+i+9upuz1g6IXfTjstnmtnm9CE8ki7tm6KbNvmEw6a
         6R5oEgwnOeJlPQvZA0HFJPOsEIAnVPc0fkyZaApjR1cu2i9FiGUBWKO/IhZUgPDJw8Br
         /esNCGEtVJT+IFQHiMuAdl7WQE0gmc3H5D+m5k5LmsBNZXi6JnvlrJXHNOVntrh8qSV1
         pR3A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706823627; x=1707428427;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=vkexTyneUo8yDBZMkq46yZq63xLJ/JfR3uJQ589GQF4=;
        b=qfM7BGu1yZWOloU9QFdHtCU3pktqt/la2gJPztA1AKCZGJHgQq0ez4yxYiQrIKQSZW
         H0Eku/Nd8jUackv8sNvRafuAxq2jpIslv4p61o290nQfwGvACe/bIisjXzsuU6UFfuoi
         aHzx/f6ukh8v+9Z5EzLADOnLJN8DlWG06bxDIn5gZsreup6K5ln2vKIoPViDK8XT09N3
         6kNg6mzU9wF1Fg5uLiGTtoybVhe9kK2F8KJj4GEq8+GyIbMz2vN+TFxWKid3mnc0KNR4
         1f3i+l9+97oYVYyCbsCIOVKddZ6gHuIA3t2c75nIaIqC6Yw1LcUt+GIspQsyg5qXqC/2
         LJnw==
X-Gm-Message-State: AOJu0YzueLxubb2/RA9J1UQ+6MczHB+NMSnAMa+CNPNRjINqmhylaf9g
	ho4lX/r7vrqdSjmvdRqloLw4lhMG9mXzuNbqqAi3LJ5wKgar/vLVSOVXKkBT
X-Google-Smtp-Source: AGHT+IGLBrIpsduzyOBzC7LSkPVe7PuJfrOhZUbgPczkeBMlBbhnOdd0NAToewP/ECbnKyOeZmEAYw==
X-Received: by 2002:a05:6214:dcc:b0:685:c584:6e8b with SMTP id 12-20020a0562140dcc00b00685c5846e8bmr7595618qvt.0.1706823626707;
        Thu, 01 Feb 2024 13:40:26 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCX34J5GhBe54FX3RBJqs7ZIWuDZVmL3wYyHWNMhm4yVwBjP4NDDQcS4ST2AaUoq/EH7He+c6OGqS3wkKJlg5Rhh3uN4e+jlCVP/jJCzcXOuBXqDeRJMNoCkC4IN3lwXH6Vjdym+ADGppGppFifok3YOG4DAbHttMLfKsw==
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: marmarek@invisiblethingslab.com,
	Jason Andryuk <jandryuk@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 1/3] libxl: Create empty file for Phy cdrom
Date: Thu,  1 Feb 2024 16:40:02 -0500
Message-ID: <20240201214004.238858-2-jandryuk@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201214004.238858-1-jandryuk@gmail.com>
References: <20240201214004.238858-1-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

With a device model stubdom, dom0 exports a PV disk to the stubdom.
Inside the stubdom, QEMU emulates a cdrom to the guest with a
host_device pointing at the PV frontend (/dev/xvdc)

An empty cdrom drive causes problems booting the stubdom.  The PV disk
protocol isn't designed to support no media.  That can be partially
hacked around, but the stubdom kernel waits for all block devices to
transition to Connected.  Since the backend never connects empty media,
stubdom launch times out and it is destroyed.

Empty media and the PV disks not connecting is fine at runtime since the
stubdom keeps running irrespective of the disk state.

Empty media can be worked around my providing an empty file to the
stubdom for the PV disk source.  This works as the disk is exposed as a
zero-size disk.  Dynamically create the empty file as needed and remove
in the stubdom cleanup.

libxl__device_disk_set_backend() needs to allow through these "empty"
disks with a pdev_path.

Fixup the params writing since scripts have trouble with an empty params
field.

This works for non-stubdom HVMs as well.

Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
---
v2:
New to support "empty" cdroms
---
 tools/libs/light/libxl_device.c   |  5 ++++-
 tools/libs/light/libxl_disk.c     | 36 +++++++++++++++++++++++++++----
 tools/libs/light/libxl_domain.c   |  4 ++++
 tools/libs/light/libxl_internal.h |  1 +
 4 files changed, 41 insertions(+), 5 deletions(-)

diff --git a/tools/libs/light/libxl_device.c b/tools/libs/light/libxl_device.c
index 13da6e0573..09d85928d7 100644
--- a/tools/libs/light/libxl_device.c
+++ b/tools/libs/light/libxl_device.c
@@ -421,7 +421,10 @@ int libxl__device_disk_set_backend(libxl__gc *gc, libxl_device_disk *disk) {
             LOG(ERROR, "Disk vdev=%s is empty but not cdrom", disk->vdev);
             return ERROR_INVAL;
         }
-        if (disk->pdev_path != NULL && strcmp(disk->pdev_path, "")) {
+        if (disk->pdev_path != NULL &&
+            (strcmp(disk->pdev_path, "") &&
+             strncmp(disk->pdev_path, LIBXL_STUBDOM_EMPTY_CDROM,
+                     strlen(LIBXL_STUBDOM_EMPTY_CDROM)))) {
             LOG(ERROR,
                 "Disk vdev=%s is empty but an image has been provided: %s",
                 disk->vdev, disk->pdev_path);
diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
index ea3623dd6f..c48e1de659 100644
--- a/tools/libs/light/libxl_disk.c
+++ b/tools/libs/light/libxl_disk.c
@@ -199,6 +199,32 @@ static int libxl__device_disk_setdefault(libxl__gc *gc, uint32_t domid,
         disk->backend = LIBXL_DISK_BACKEND_QDISK;
     }
 
+    if (disk->is_cdrom &&
+        disk->format == LIBXL_DISK_FORMAT_EMPTY &&
+        disk->backend == LIBXL_DISK_BACKEND_PHY &&
+        disk->backend_domid == LIBXL_TOOLSTACK_DOMID) {
+        uint32_t target_domid;
+        int fd;
+
+        if (libxl_is_stubdom(CTX, domid, &target_domid)) {
+            LOGED(DEBUG, domid, "Using target_domid %u", target_domid);
+        } else {
+            target_domid = domid;
+        }
+        free(disk->pdev_path);
+        disk->pdev_path =
+            libxl__sprintf(NOGC, LIBXL_STUBDOM_EMPTY_CDROM ".%u",
+                           target_domid);
+        fd = creat(disk->pdev_path, 0400);
+        if (fd < 0) {
+            LOGED(ERROR, domid, "Failed to create empty cdrom \"%s\"",
+                  disk->pdev_path);
+            return ERROR_FAIL;
+        }
+
+        close(fd);
+    }
+
     rc = libxl__device_disk_set_backend(gc, disk);
     return rc;
 }
@@ -988,7 +1014,7 @@ static void cdrom_insert_ejected(libxl__egc *egc,
     empty = flexarray_make(gc, 4, 1);
     flexarray_append_pair(empty, "type",
                           libxl__device_disk_string_of_backend(disk->backend));
-    flexarray_append_pair(empty, "params", "");
+    flexarray_append_pair(empty, "params", disk->pdev_path ?: "");
 
     for (;;) {
         rc = libxl__xs_transaction_start(gc, &t);
@@ -1164,13 +1190,15 @@ static void cdrom_insert_inserted(libxl__egc *egc,
     insert = flexarray_make(gc, 4, 1);
     flexarray_append_pair(insert, "type",
                       libxl__device_disk_string_of_backend(disk->backend));
-    if (disk->format != LIBXL_DISK_FORMAT_EMPTY)
+    if (disk->backend == LIBXL_DISK_BACKEND_QDISK &&
+        disk->format != LIBXL_DISK_FORMAT_EMPTY) {
         flexarray_append_pair(insert, "params",
                     GCSPRINTF("%s:%s",
                         libxl__device_disk_string_of_format(disk->format),
                         disk->pdev_path));
-    else
-        flexarray_append_pair(insert, "params", "");
+    } else {
+        flexarray_append_pair(insert, "params", disk->pdev_path ?: "");
+    }
 
     for (;;) {
         rc = libxl__xs_transaction_start(gc, &t);
diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c
index 5ee1544d9c..6751fc785f 100644
--- a/tools/libs/light/libxl_domain.c
+++ b/tools/libs/light/libxl_domain.c
@@ -1525,6 +1525,10 @@ static void devices_destroy_cb(libxl__egc *egc,
     if (rc < 0)
         LOGD(ERROR, domid, "libxl__devices_destroy failed");
 
+    /* Remove the file after the hotplug scripts have run.  The scripts won't
+     * run if the file doesn't exist when they are run.  */
+    libxl__remove_file(gc, GCSPRINTF(LIBXL_STUBDOM_EMPTY_CDROM ".%u", domid));
+
     vm_path = libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/vm", dom_path));
     if (vm_path)
         if (!xs_rm(ctx->xsh, XBT_NULL, vm_path))
diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h
index d5732d1c37..094d0df9b1 100644
--- a/tools/libs/light/libxl_internal.h
+++ b/tools/libs/light/libxl_internal.h
@@ -120,6 +120,7 @@
 #define STUBDOM_CONSOLE_SERIAL 3
 #define STUBDOM_SPECIAL_CONSOLES 3
 #define LIBXL_LINUX_STUBDOM_MEM 128
+#define LIBXL_STUBDOM_EMPTY_CDROM XEN_RUN_DIR "/empty-cdrom" /* .$domid */
 #define TAP_DEVICE_SUFFIX "-emu"
 #define DOMID_XS_PATH "domid"
 #define PVSHIM_BASENAME "xen-shim"
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 21:40:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 21:40:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674717.1049851 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVenT-0005ps-36; Thu, 01 Feb 2024 21:40:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674717.1049851; Thu, 01 Feb 2024 21:40:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVenS-0005ph-UT; Thu, 01 Feb 2024 21:40:34 +0000
Received: by outflank-mailman (input) for mailman id 674717;
 Thu, 01 Feb 2024 21:40:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MzG+=JK=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rVenR-0005Jd-Qq
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 21:40:33 +0000
Received: from mail-yw1-x1130.google.com (mail-yw1-x1130.google.com
 [2607:f8b0:4864:20::1130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 85fb3eb9-c14a-11ee-98f5-efadbce2ee36;
 Thu, 01 Feb 2024 22:40:32 +0100 (CET)
Received: by mail-yw1-x1130.google.com with SMTP id
 00721157ae682-6040d9e52b9so14766197b3.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 13:40:32 -0800 (PST)
Received: from pm2-ws13.praxislan02.com
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 h7-20020a05620a21c700b007853eb72c49sm154978qka.46.2024.02.01.13.40.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 13:40:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85fb3eb9-c14a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706823630; x=1707428430; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ucbmMMAbbZ+Pz4AdE+09SKz62vvqmDMnyT9AshCA9EQ=;
        b=Qsibb857iTEPS1EEvQDMv1Wz+TmQkg/EwlPqWKozxR3UH1IYEDkQvSAIaJKCyMrJQc
         R8ZNpsoTXmjJ/zwkGZJBApYJRhgqAJEiHA6+u2s9EafRGxZQ2BwesEGeTe4FovUnbqkc
         2awQVWK59lBmHQbDj7g+TBHgc1yGOW1sbkkltym96DOqMpg25cbjevZjSu9lancgoChk
         7y4e6fkwhDt4NSm9uXFLxOMrkXD2l+Z5rVuMc0YqpDLbnKoAmpR4v8yNFhA8W87pyk08
         Pxv3kTKcSnAFd61f5C7payadPlUYWeIDrgss3DUhKY0M8CwQCajdd9z+woCsXnrPXqXa
         d5Qw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706823630; x=1707428430;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ucbmMMAbbZ+Pz4AdE+09SKz62vvqmDMnyT9AshCA9EQ=;
        b=VIpoGrOzpy7g5nNLGCaMOiBrW52SYDp6wKKB2TPHkmtHTMzdiLOeDO5oNbN6NfpJJs
         aZMDiiTH3ylYltn7WOWka6SHgiRiV45fkPWE1KqNe/doLKRDObSWDlPyITgjeKHSOuBB
         FCHrvgAG7FT61GNNJi5vMA0ORLU3ET8v2W/M04//DdG1ON6ZOItGiAzuYKEox2N9Evw2
         Xb/CVflISCg1inRMGOyLAi9LwAWJbNlxQ0I3yxMCzkr6T7kUsnF850/hsQ88PUivtt2Z
         91mX44L3nR9GDuk8z/PqcS3E0RAIDbO+hPhPwfdpNrc2hji4sDFZURWODamSfrqMxCnN
         I5iw==
X-Gm-Message-State: AOJu0YxBNaTiG1IM5z8MAjpiO/9Vr1vcHkLIHFL/s9L0jKFKErgQcJGd
	Njs9Vbovx1FQGs6Tm5lCHoOc1N0yugTdK4Whm+tSU10FukK8J3AnFhjBrPit
X-Google-Smtp-Source: AGHT+IGyZ3ROrgxWgwd9VmfOUiLAllEP63hV3T4NdJde0vqe3kbDuDiHRW5UFWDUI28W5/SabNCBWw==
X-Received: by 2002:a0d:cc53:0:b0:5ff:6b9e:cdd9 with SMTP id o80-20020a0dcc53000000b005ff6b9ecdd9mr463708ywd.46.1706823630026;
        Thu, 01 Feb 2024 13:40:30 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWt/eemjeAB0YsJRR+bss0JwPO4dcbAfWs7OXwO60kt3BD3y1I2eGHOBZL3aMxPpKLx8YYVZeNUEUmAMMNLGe99eaiJiz5jrUGQwORoA/L3/EG66/nsdnT8Yd4yV+zKju6qm0LYgtGFDem4UWXV9C1+PwFIa2cW4mOc4w==
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: marmarek@invisiblethingslab.com,
	Jason Andryuk <jandryuk@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 2/3] libxl: Allow Phy backend for CDROM devices
Date: Thu,  1 Feb 2024 16:40:03 -0500
Message-ID: <20240201214004.238858-3-jandryuk@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201214004.238858-1-jandryuk@gmail.com>
References: <20240201214004.238858-1-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

A Linux HVM domain ignores PV block devices with type cdrom.  The
Windows PV drivers also ignore device-type != "disk".  Therefore QEMU's
emulated CD-ROM support is used.  This allows ejection and other CD-ROM
features to work.

With a stubdom, QEMU is running in the stubdom.  A PV disk is still
connected into the stubdom, and then QEMU can emulate the CD-ROM into
the guest.  Phy support has been enhanced to provide a placeholder file
forempty disks, so it is usable as a CDROM backend as well.  Allow Phy
to pass the check as well.

(Bypassing just for a linux-based stubdom doesn't work because
libxl__device_disk_setdefault() gets called early in domain creation
before xenstore is populated with relevant information for the stubdom
type.  The build information isn't readily available and won't exist in
some call trees, so it isn't usable either.)

Let disk_try_backend() allow format empty for Phy cdrom drives.

Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
---
v2:
Different approach to pass QDISK requirement check.
---
 tools/libs/light/libxl_device.c | 12 ++++++++----
 tools/libs/light/libxl_disk.c   | 11 +++++++----
 2 files changed, 15 insertions(+), 8 deletions(-)

diff --git a/tools/libs/light/libxl_device.c b/tools/libs/light/libxl_device.c
index 09d85928d7..f73c6705d4 100644
--- a/tools/libs/light/libxl_device.c
+++ b/tools/libs/light/libxl_device.c
@@ -301,13 +301,17 @@ static int disk_try_backend(disk_try_backend_args *a,
 
     switch (backend) {
     case LIBXL_DISK_BACKEND_PHY:
-        if (a->disk->format != LIBXL_DISK_FORMAT_RAW) {
-            goto bad_format;
-        }
-
         if (libxl_defbool_val(a->disk->colo_enable))
             goto bad_colo;
 
+        if (a->disk->is_cdrom && a->disk->format == LIBXL_DISK_FORMAT_EMPTY) {
+            LOG(DEBUG, "Disk vdev=%s is an empty cdrom", a->disk->vdev);
+            return backend;
+        }
+
+        if (a->disk->format != LIBXL_DISK_FORMAT_RAW)
+            goto bad_format;
+
         if (a->disk->backend_domid != LIBXL_TOOLSTACK_DOMID) {
             LOG(DEBUG, "Disk vdev=%s, is using a storage driver domain, "
                        "skipping physical device check", a->disk->vdev);
diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
index c48e1de659..09082ffb58 100644
--- a/tools/libs/light/libxl_disk.c
+++ b/tools/libs/light/libxl_disk.c
@@ -188,15 +188,18 @@ static int libxl__device_disk_setdefault(libxl__gc *gc, uint32_t domid,
         return ERROR_FAIL;
     }
 
-    /* Force Qdisk backend for CDROM devices of guests with a device model. */
+    /* Only allow Qdisk or Phy for CDROM devices. */
     if (disk->is_cdrom != 0 &&
         libxl__domain_type(gc, domid) == LIBXL_DOMAIN_TYPE_HVM) {
+        if (disk->backend == LIBXL_DISK_BACKEND_UNKNOWN)
+            disk->backend = LIBXL_DISK_BACKEND_QDISK;
+
         if (!(disk->backend == LIBXL_DISK_BACKEND_QDISK ||
-              disk->backend == LIBXL_DISK_BACKEND_UNKNOWN)) {
-            LOGD(ERROR, domid, "Backend for CD devices on HVM guests must be Qdisk");
+              disk->backend == LIBXL_DISK_BACKEND_PHY)) {
+            LOGD(ERROR, domid,
+                 "Backend for CD devices on HVM guests must be Qdisk or Phy");
             return ERROR_FAIL;
         }
-        disk->backend = LIBXL_DISK_BACKEND_QDISK;
     }
 
     if (disk->is_cdrom &&
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 21:40:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 21:40:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674718.1049861 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVenW-00067j-9d; Thu, 01 Feb 2024 21:40:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674718.1049861; Thu, 01 Feb 2024 21:40:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVenW-00067a-64; Thu, 01 Feb 2024 21:40:38 +0000
Received: by outflank-mailman (input) for mailman id 674718;
 Thu, 01 Feb 2024 21:40:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=MzG+=JK=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rVenU-0005YC-HZ
 for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 21:40:36 +0000
Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com
 [2607:f8b0:4864:20::735])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 87c20d68-c14a-11ee-8a43-1f161083a0e0;
 Thu, 01 Feb 2024 22:40:35 +0100 (CET)
Received: by mail-qk1-x735.google.com with SMTP id
 af79cd13be357-78407987510so70292385a.3
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 13:40:35 -0800 (PST)
Received: from pm2-ws13.praxislan02.com
 (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com.
 [207.172.141.204]) by smtp.gmail.com with ESMTPSA id
 h7-20020a05620a21c700b007853eb72c49sm154978qka.46.2024.02.01.13.40.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 13:40:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87c20d68-c14a-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706823634; x=1707428434; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wX9UDy1UxiRe7Hnzf1ly24vLwl89/WjFYev/11/D2g4=;
        b=lmwXYq9LM8MDYhc6H6NeZ9DkM9l0o/2eeS8k5YUUgL8cbHEiUOnTzoQjlFf0+IEBMq
         JmHkaiFJ7Sl8IUXmKKNwsIiLxvG5PuFLqfCPTB9TxeYEVNvtHP4ZPM8XJWh0QzwQPC6B
         /cjRhFXvv1XF3dvRUt0OvXI6XZL00NRxpJExoo1wu4YTG2CiJmdlOE6Pk5fjpMw/9JoC
         lBWbsV2ffldjX0vFS5oSjOWTFVrF62EY2f1HFI3NwOqPR9ngjH2HVEM2adjIxBRlZMyt
         IeixCdBnR3rbJyiOMas062WoHg+ixhHFZ+32g2uts4T4GhGLPcVL+0gwUHFU/Rf1C36W
         hXNw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706823634; x=1707428434;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=wX9UDy1UxiRe7Hnzf1ly24vLwl89/WjFYev/11/D2g4=;
        b=vyLGAKyXASIwaYXbeOfDpEUNJ6BbiZXRolNAZU4DThH0a1QxRDwe1Iw81WSN4vKBVC
         H6Az96ABnOC0tXWxhTIJuB6bpt0WJxw9fWC4RaqIUPnZl01JpJb/p/11IOKYP9zrZcmT
         jAKOhFFiUc6utbfJYAYND8x3r6Q+VR+GaTWR7BJ7oqEYzNnsMaQnxTi0BnIDyRDcYdvy
         cplW068nFw1BbnG169fu2Y6uuRCaCnaedZAdDbe6gyymTI65W2ueABPpRHrneawotNkh
         wzcZi5kYKVQU9eQuX6aWZ2MwIvRjEpAs7TTNC0cHqcbG5gtIScJ0hO051nxyMD9/bd8N
         pItA==
X-Gm-Message-State: AOJu0YyKwciZ/efvCYxYDgHlAeRCdR6RMQVkvv/4MRlCqlm59tBnBQAi
	UXI4I24YO3EXdx36nnx87hF6/tWRClAM/jf1Zrn7kSFyy5BHlF2vfwfXINge
X-Google-Smtp-Source: AGHT+IEX178TypQ98U+CByn1kT4kVETUvyWOs9cB5bGAMhTVzG7Y7WLPgidUFDvGOT4uGCSHRj2/1g==
X-Received: by 2002:a05:620a:248b:b0:783:9ab7:374 with SMTP id i11-20020a05620a248b00b007839ab70374mr4495234qkn.8.1706823633759;
        Thu, 01 Feb 2024 13:40:33 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXgOnxKl8woBEDI2950HAGp1QbE3WzPvs7NrYFP6KZwAFr/Z3wtM10l01aHKQCriAJNUMBtPgK9eG5IGrATujMSueXD3gSuQKWYSKR9Td5DtC6cwhkdr+2Sbg27/1XQCtiq+cfNj0sLuSjnNKQHz73oesTxlZDozfKoZX2VjWfzq+GGpGwwKRvn1QwmRWl1zqexnlcZfT652TxoPLlhutevKomA3YfcMp61fC7guhh6brcBdCKLSuMgqKdIEgj+0vjVcy/ayyIuMW1PAIW7jehMB1Qgz4WqtfXmJXHZdrrc43BiK6SX4feM7k4=
From: Jason Andryuk <jandryuk@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: marmarek@invisiblethingslab.com,
	Jason Andryuk <jandryuk@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2 3/3] libxl: Enable stubdom cdrom changing
Date: Thu,  1 Feb 2024 16:40:04 -0500
Message-ID: <20240201214004.238858-4-jandryuk@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240201214004.238858-1-jandryuk@gmail.com>
References: <20240201214004.238858-1-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

To change the cd-rom medium, libxl will:
 - QMP eject the medium from QEMU
 - block-detach the old PV disk
 - block-attach the new PV disk
 - QMP change the medium to the new PV disk by fdset-id

The QMP code is reused, and remove and attach are implemented here.

The stubdom must internally handle adding /dev/xvdc to the appropriate
fdset.  libxl in dom0 doesn't see the result of adding to the fdset as
that is internal to the stubdom, so a delay and retries are added to
around calling cdrom_insert_addfd_cb().

For cd-eject, we still need to attach the empty vbd.  This is necessary
since xenstore is used to determine that hdc exists.  Otherwise after
eject, hdc would be gone and the cd-insert would fail to find the drive
to insert new media.

The ERROR_JSON_CONFIG_EMPTY check in cdrom_insert_inserted() is because
a stubdom don't have a json config.

Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
---
v2:
Only allow for Linux stubdoms (Marek)
Fix cd-eject
Fix errant hard tabs
Move the debug print and into the special case.
---
 docs/misc/stubdom.txt         |  16 ++
 tools/libs/light/libxl_disk.c | 298 +++++++++++++++++++++++++++++++---
 2 files changed, 288 insertions(+), 26 deletions(-)

diff --git a/docs/misc/stubdom.txt b/docs/misc/stubdom.txt
index c717a95d17..1b2380ae8f 100644
--- a/docs/misc/stubdom.txt
+++ b/docs/misc/stubdom.txt
@@ -127,6 +127,22 @@ Limitations:
  - at most 26 emulated disks are supported (more are still available as PV disks)
  - graphics output (VNC/SDL/Spice) not supported
 
+CD-ROM changing:
+
+To change the CD-ROM medium, libxl will:
+ - QMP eject the medium from QEMU
+ - block-detach the old PV disk
+ - block-attach the new PV disk
+ - QMP change the medium to the new PV disk by fdset-id
+
+The QMP change insert uses fdset-id STUBDOM_FDSET_CD + $disk - 'a'.
+That is, hda -> 'a', so
+STUBDOM_FDSET_CD + 'a' - 'a' = STUBDOM_FDSET_CD.
+For hdc:
+STUBDOM_FDSET_CD + 'c' - 'a' = STUBDOM_FDSET_CD + 2.
+
+The stubdom must internally handle adding /dev/xvdc to the appropriate
+fdset.
 
                                    PV-GRUB
                                    =======
diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
index 09082ffb58..6354982c05 100644
--- a/tools/libs/light/libxl_disk.c
+++ b/tools/libs/light/libxl_disk.c
@@ -808,25 +808,46 @@ int libxl_device_disk_getinfo(libxl_ctx *ctx, uint32_t domid,
 
 typedef struct {
     libxl__ao *ao;
+    libxl__ao_device aodev;
+    libxl__ao_device aodev_del;
     libxl_domid domid;
+    libxl_domid disk_domid;
     libxl_device_disk *disk;
     libxl_device_disk disk_saved;
     libxl__ev_slowlock qmp_lock;
     int dm_ver;
     libxl__ev_time time;
+    libxl__ev_time timeout_retry;
     libxl__ev_qmp qmp;
+    int retries;
+    int stubdom_fdset;
 } libxl__cdrom_insert_state;
 
 static void cdrom_insert_lock_acquired(libxl__egc *, libxl__ev_slowlock *,
                                        int rc);
 static void cdrom_insert_qmp_connected(libxl__egc *, libxl__ev_qmp *,
                                        const libxl__json_object *, int rc);
+static void cdrom_insert_stubdom_removefd(libxl__egc *egc, libxl__ev_qmp *qmp,
+                                          const libxl__json_object *response,
+                                          int rc);
+static void cdrom_insert_stubdom_ejected(libxl__egc *egc, libxl__ev_qmp *,
+                                         const libxl__json_object *, int rc);
+static void cdrom_insert_stubdom_disk_ejected_aocomplete(libxl__egc *egc,
+                                                         libxl__ao_device *aodev);
+static void cdrom_insert_stubdom_disk_ejected(libxl__egc *egc, libxl__ev_qmp *,
+                                              const libxl__json_object *,
+                                              int rc);
+static void cdrom_insert_ejected_aodevcb(libxl__egc *egc,
+                                         libxl__ao_device *aodev);
 static void cdrom_insert_ejected(libxl__egc *egc, libxl__ev_qmp *,
                                  const libxl__json_object *, int rc);
 static void cdrom_insert_addfd_cb(libxl__egc *egc, libxl__ev_qmp *,
                                   const libxl__json_object *, int rc);
 static void cdrom_insert_inserted(libxl__egc *egc, libxl__ev_qmp *,
                                   const libxl__json_object *, int rc);
+static void cdrom_insert_addfd_retry(libxl__egc *egc, libxl__ev_time *ev,
+                                     const struct timeval *requested_abs,
+                                     int rc);
 static void cdrom_insert_timout(libxl__egc *egc, libxl__ev_time *ev,
                                 const struct timeval *requested_abs,
                                 int rc);
@@ -842,6 +863,7 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
     libxl_device_disk *disks = NULL;
     int rc;
     libxl__cdrom_insert_state *cis;
+    libxl_domid stubdomid;
 
     GCNEW(cis);
     cis->ao = ao;
@@ -853,6 +875,8 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
     cis->qmp_lock.ao = ao;
     cis->qmp_lock.domid = domid;
     libxl__ev_time_init(&cis->time);
+    libxl__ev_time_init(&cis->timeout_retry);
+    cis->retries = 0;
     libxl__ev_qmp_init(&cis->qmp);
     cis->qmp.ao = ao;
     cis->qmp.domid = domid;
@@ -869,12 +893,6 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
         goto out;
     }
 
-    if (libxl_get_stubdom_id(ctx, domid) != 0) {
-        LOGD(ERROR, domid, "cdrom-insert doesn't work for stub domains");
-        rc = ERROR_INVAL;
-        goto out;
-    }
-
     cis->dm_ver = libxl__device_model_version_running(gc, domid);
     if (cis->dm_ver == -1) {
         LOGD(ERROR, domid, "Cannot determine device model version");
@@ -882,7 +900,22 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
         goto out;
     }
 
-    disks = libxl__device_list(gc, &libxl__disk_devtype, domid, &num);
+    stubdomid = libxl_get_stubdom_id(CTX, cis->domid);
+    if (stubdomid == 0) {
+        cis->disk_domid = domid;
+    } else {
+        cis->disk_domid = stubdomid;
+        disk->backend = LIBXL_DISK_BACKEND_PHY;
+    }
+
+    if (cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL &&
+        stubdomid) {
+        LOGD(ERROR, domid, "cdrom-insert doesn't work for Mini-OS stubdoms");
+        rc = ERROR_INVAL;
+        goto out;
+    }
+
+    disks = libxl__device_list(gc, &libxl__disk_devtype, cis->disk_domid, &num);
     for (i = 0; i < num; i++) {
         if (disks[i].is_cdrom && !strcmp(disk->vdev, disks[i].vdev))
         {
@@ -897,7 +930,7 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
         goto out;
     }
 
-    rc = libxl__device_disk_setdefault(gc, domid, disk, false);
+    rc = libxl__device_disk_setdefault(gc, cis->disk_domid, disk, false);
     if (rc) goto out;
 
     if (!disk->pdev_path) {
@@ -905,6 +938,18 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
         disk->format = LIBXL_DISK_FORMAT_EMPTY;
     }
 
+#define STUBDOM_FDSET_CD 8000
+    if (strncmp(disk->vdev, "hd", 2) == 0) {
+        cis->stubdom_fdset = STUBDOM_FDSET_CD + disk->vdev[2] - 'a';
+    } else if (strncmp(disk->vdev, "xvd", 3) == 0) {
+        cis->stubdom_fdset = STUBDOM_FDSET_CD + disk->vdev[3] - 'a';
+    } else {
+        LOGD(ERROR, cis->domid, "disk->vdev \"%s\" isn't hdX or xvdY",
+             disk->vdev);
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
 out:
     libxl__device_list_free(&libxl__disk_devtype, disks, num);
     if (rc) {
@@ -923,6 +968,7 @@ static void cdrom_insert_lock_acquired(libxl__egc *egc,
     libxl__cdrom_insert_state *cis = CONTAINER_OF(lock, *cis, qmp_lock);
     STATE_AO_GC(cis->ao);
 
+    LOGD(DEBUG, cis->domid, "rc=%d", rc);
     if (rc) goto out;
 
     rc = libxl__ev_time_register_rel(ao, &cis->time,
@@ -971,7 +1017,12 @@ static void cdrom_insert_qmp_connected(libxl__egc *egc, libxl__ev_qmp *qmp,
         QMP_PARAMETERS_SPRINTF(&args, "id", "ide-%i", devid);
     else
         QMP_PARAMETERS_SPRINTF(&args, "device", "ide-%i", devid);
-    qmp->callback = cdrom_insert_ejected;
+
+    if (libxl_get_stubdom_id(CTX, cis->domid))
+        qmp->callback = cdrom_insert_stubdom_removefd;
+    else
+        qmp->callback = cdrom_insert_ejected;
+
     rc = libxl__ev_qmp_send(egc, qmp, "eject", args);
     if (rc) goto out;
     return;
@@ -979,6 +1030,148 @@ out:
     cdrom_insert_done(egc, cis, rc); /* must be last */
 }
 
+static void cdrom_insert_stubdom_removefd(libxl__egc *egc, libxl__ev_qmp *qmp,
+                                          const libxl__json_object *response,
+                                          int rc)
+{
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp);
+    STATE_AO_GC(cis->ao);
+
+    if (rc) goto out;
+
+    /* Only called for qemu-xen/linux stubdom. */
+    assert(cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN);
+    libxl__json_object *args = NULL;
+
+    libxl__qmp_param_add_integer(gc, &args, "fdset-id", cis->stubdom_fdset);
+
+    cis->qmp.callback = cdrom_insert_stubdom_ejected;
+
+    rc = libxl__ev_qmp_send(egc, &cis->qmp, "remove-fd", args);
+    if (rc) goto out;
+
+    return;
+
+out:
+    cdrom_insert_done(egc, cis, rc); /* must be last */
+}
+
+static void cdrom_insert_stubdom_ejected(libxl__egc *egc, libxl__ev_qmp *qmp,
+                                         const libxl__json_object *response,
+                                         int rc)
+{
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp);
+    libxl__device *device;
+    STATE_AO_GC(cis->ao);
+    domid_t stubdomid = libxl_get_stubdom_id(CTX, cis->domid);
+
+    LOGD(DEBUG, cis->domid, "rc=%d", rc);
+    /* cis->stubdom_fdset is initially empty, so remove-fd fails the first
+     * call with:
+     * {"error": {"class": "GenericError",
+     *            "desc": "File descriptor named 'fdset-id:8675' not found"}}
+     * Carry on in that case. */
+    if (rc && rc != ERROR_QMP_GENERIC_ERROR) goto out;
+
+    GCNEW(device);
+    rc = libxl__device_from_disk(gc, stubdomid, cis->disk, device);
+    if (rc != 0) goto out;
+
+    /* block dev eject */
+    /* Below is basically an open coding of:
+     * libxl_device_disk__remove(CTX, cis->domid, cis->disk, 0);
+     * ...since we can't call it from within libxl.
+     */
+    libxl__prepare_ao_device(ao, &cis->aodev_del);
+    cis->aodev_del.action = LIBXL__DEVICE_ACTION_REMOVE;
+    cis->aodev_del.dev = device;
+    cis->aodev_del.callback = cdrom_insert_stubdom_disk_ejected_aocomplete;
+    cis->aodev_del.force.flag = LIBXL__FORCE_OFF;
+    libxl__initiate_device_generic_remove(egc, &cis->aodev_del);
+    return;
+
+ out:
+    cdrom_insert_done(egc, cis, rc); /* must be last */
+}
+
+static void cdrom_insert_stubdom_disk_ejected_aocomplete(libxl__egc *egc,
+                                                         libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(aodev, *cis, aodev_del);
+
+    LOGD(DEBUG, cis->domid, "rc=%d", aodev->rc);
+    if (aodev->rc) {
+        if (aodev->dev) {
+            LOGD(ERROR, aodev->dev->domid, "Unable to %s %s with id %u",
+                        libxl__device_action_to_string(aodev->action),
+                        libxl__device_kind_to_string(aodev->dev->kind),
+                        aodev->dev->devid);
+        } else {
+            LOG(ERROR, "unable to %s device",
+                       libxl__device_action_to_string(aodev->action));
+        }
+        goto out;
+    }
+
+    cdrom_insert_stubdom_disk_ejected(egc, &cis->qmp, NULL, aodev->rc);
+    return;
+
+ out:
+    cdrom_insert_done(egc, cis, aodev->rc);
+}
+
+static void cdrom_insert_stubdom_disk_ejected(libxl__egc *egc,
+                                              libxl__ev_qmp *qmp,
+                                              const libxl__json_object *response,
+                                              int rc)
+{
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp);
+    STATE_AO_GC(cis->ao);
+    domid_t stubdomid = libxl_get_stubdom_id(CTX, cis->domid);
+
+    LOGD(DEBUG, cis->domid, "rc=%d", rc);
+    if (rc) goto out;
+
+    /* block dev insert - this may be inserting an empty disk for eject. */
+    libxl__prepare_ao_device(ao, &cis->aodev);
+    /* set an ao callback to end up in cdrom_insert_ejected */
+    cis->aodev.callback = cdrom_insert_ejected_aodevcb;
+    libxl__device_disk_add(egc, stubdomid, cis->disk, &cis->aodev);
+
+    return;
+
+ out:
+    cdrom_insert_done(egc, cis, rc); /* must be last */
+}
+
+static void cdrom_insert_ejected_aodevcb(libxl__egc *egc,
+                                         libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(aodev, *cis, aodev);
+
+    LOGD(DEBUG, cis->domid, "rc=%d", aodev->rc);
+    if (aodev->rc) {
+        if (aodev->dev) {
+            LOGD(ERROR, aodev->dev->domid, "Unable to %s %s with id %u",
+                        libxl__device_action_to_string(aodev->action),
+                        libxl__device_kind_to_string(aodev->dev->kind),
+                        aodev->dev->devid);
+        } else {
+            LOG(ERROR, "unable to %s device",
+                       libxl__device_action_to_string(aodev->action));
+        }
+        goto out;
+    }
+
+    cdrom_insert_ejected(egc, &cis->qmp, NULL, aodev->rc);
+    return;
+
+ out:
+    cdrom_insert_done(egc, cis, aodev->rc);
+}
+
 static void cdrom_insert_ejected(libxl__egc *egc,
                                  libxl__ev_qmp *qmp,
                                  const libxl__json_object *response,
@@ -1001,9 +1194,10 @@ static void cdrom_insert_ejected(libxl__egc *egc,
 
     libxl_domain_config_init(&d_config);
 
+    LOGD(DEBUG, cis->domid, "rc=%d", rc);
     if (rc) goto out;
 
-    rc = libxl__device_from_disk(gc, domid, disk, &device);
+    rc = libxl__device_from_disk(gc, cis->disk_domid, disk, &device);
     if (rc) goto out;
     be_path = libxl__device_backend_path(gc, &device);
     libxl_path = libxl__device_libxl_path(gc, &device);
@@ -1050,7 +1244,7 @@ static void cdrom_insert_ejected(libxl__egc *egc,
      */
 
     rc = libxl__get_domain_configuration(gc, domid, &d_config);
-    if (rc) goto out;
+    if (rc && rc != ERROR_JSON_CONFIG_EMPTY) goto out;
 
     device_add_domain_config(gc, &d_config, &libxl__disk_devtype,
                              &cis->disk_saved);
@@ -1058,10 +1252,15 @@ static void cdrom_insert_ejected(libxl__egc *egc,
     rc = libxl__dm_check_start(gc, &d_config, domid);
     if (rc) goto out;
 
+    LOGD(DEBUG, cis->domid, "stubdom_id=%d",
+                libxl_get_stubdom_id(CTX, cis->domid));
+    /* A linux stubdom will perform add-fd with calculated stubdom_fdset. */
     if (cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN &&
+        libxl_get_stubdom_id(CTX, cis->domid) == 0 &&
         disk->format != LIBXL_DISK_FORMAT_EMPTY) {
         libxl__json_object *args = NULL;
 
+        LOGD(DEBUG, cis->domid, "Doing qmp add-fd path");
         assert(qmp->payload_fd == -1);
         qmp->payload_fd = open(disk->pdev_path, O_RDONLY);
         if (qmp->payload_fd < 0) {
@@ -1080,20 +1279,30 @@ static void cdrom_insert_ejected(libxl__egc *egc,
         if (rc) goto out;
         has_callback = true;
     } else {
+        LOGD(DEBUG, cis->domid, "Skipping qmp add-fd path");
         has_callback = false;
     }
 
     rc = 0;
 
 out:
+    LOGD(DEBUG, cis->domid, "out label rc=%d", rc);
     libxl__xs_transaction_abort(gc, &t);
     libxl_domain_config_dispose(&d_config);
     if (data_lock) libxl__unlock_file(data_lock);
     if (rc) {
         cdrom_insert_done(egc, cis, rc); /* must be last */
     } else if (!has_callback) {
-        /* Only called if no asynchronous callback are set. */
-        cdrom_insert_inserted(egc, qmp, NULL, 0); /* must be last */
+        if (libxl_get_stubdom_id(CTX, cis->domid) &&
+            disk->format != LIBXL_DISK_FORMAT_EMPTY) {
+            LOGD(DEBUG, cis->domid,
+                 "stubdom %d needs to perform add-fd internally",
+                 libxl_get_stubdom_id(CTX, cis->domid));
+            cdrom_insert_addfd_cb(egc, qmp, NULL, 0); /* must be last */
+        } else  {
+            /* Only called if no asynchronous callback are set. */
+            cdrom_insert_inserted(egc, qmp, NULL, 0); /* must be last */
+        }
     }
 }
 
@@ -1112,17 +1321,24 @@ static void cdrom_insert_addfd_cb(libxl__egc *egc,
     /* convenience aliases */
     libxl_device_disk *disk = cis->disk;
 
-    close(qmp->payload_fd);
-    qmp->payload_fd = -1;
+    LOGD(DEBUG, cis->domid, "rc=%d", rc);
 
     if (rc) goto out;
 
-    o = libxl__json_map_get("fdset-id", response, JSON_INTEGER);
-    if (!o) {
-        rc = ERROR_FAIL;
-        goto out;
+    /* response non-NULL only for non-stubdom */
+    if (response) {
+        close(qmp->payload_fd);
+        qmp->payload_fd = -1;
+
+        o = libxl__json_map_get("fdset-id", response, JSON_INTEGER);
+        if (!o) {
+            rc = ERROR_FAIL;
+            goto out;
+        }
+        fdset = libxl__json_object_get_integer(o);
+    } else {
+        fdset = cis->stubdom_fdset;
     }
-    fdset = libxl__json_object_get_integer(o);
 
     devid = libxl__device_disk_dev_number(disk->vdev, NULL, NULL);
     qmp->callback = cdrom_insert_inserted;
@@ -1135,8 +1351,13 @@ static void cdrom_insert_addfd_cb(libxl__egc *egc,
     if (libxl__qmp_ev_qemu_compare_version(qmp, 2, 8, 0) >= 0) {
         QMP_PARAMETERS_SPRINTF(&args, "id", "ide-%i", devid);
         QMP_PARAMETERS_SPRINTF(&args, "filename", "/dev/fdset/%d", fdset);
-        libxl__qmp_param_add_string(gc, &args, "format",
-            libxl__qemu_disk_format_string(disk->format));
+        if (response) {
+            libxl__qmp_param_add_string(gc, &args, "format",
+                libxl__qemu_disk_format_string(disk->format));
+        } else {
+            /* Stubdom is using blockdev /dev/xvd* */
+            libxl__qmp_param_add_string(gc, &args, "format", "host_device");
+        }
         rc = libxl__ev_qmp_send(egc, qmp, "blockdev-change-medium", args);
     } else {
         QMP_PARAMETERS_SPRINTF(&args, "device", "ide-%i", devid);
@@ -1150,13 +1371,22 @@ out:
         cdrom_insert_done(egc, cis, rc); /* must be last */
 }
 
+static void cdrom_insert_addfd_retry(libxl__egc *egc, libxl__ev_time *ev,
+                                     const struct timeval *requested_abs,
+                                     int rc)
+{
+    libxl__cdrom_insert_state *cis = CONTAINER_OF(ev, *cis, timeout_retry);
+
+    cdrom_insert_addfd_cb(egc, &cis->qmp, NULL, 0);
+}
+
 static void cdrom_insert_inserted(libxl__egc *egc,
                                   libxl__ev_qmp *qmp,
                                   const libxl__json_object *response,
                                   int rc)
 {
-    EGC_GC;
     libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp);
+    STATE_AO_GC(cis->ao);
     libxl__flock *data_lock = NULL;
     libxl_domain_config d_config;
     flexarray_t *insert = NULL;
@@ -1171,9 +1401,22 @@ static void cdrom_insert_inserted(libxl__egc *egc,
 
     libxl_domain_config_init(&d_config);
 
-    if (rc) goto out;
+    LOGD(DEBUG, cis->domid, "rc=%d", rc);
+
+    if (rc) {
+        if (cis->retries++ < 10 ) {
+            LOGD(DEBUG, qmp->domid, "Retrying QMP cdrom change\n");
+            rc = libxl__ev_time_register_rel(ao, &cis->timeout_retry,
+                                             cdrom_insert_addfd_retry, 100);
+            if (rc) goto out;
 
-    rc = libxl__device_from_disk(gc, domid, disk, &device);
+            return;
+        } else {
+            goto out;
+        }
+    }
+
+    rc = libxl__device_from_disk(gc, cis->disk_domid, disk, &device);
     if (rc) goto out;
     be_path = libxl__device_backend_path(gc, &device);
     libxl_path = libxl__device_libxl_path(gc, &device);
@@ -1185,7 +1428,7 @@ static void cdrom_insert_inserted(libxl__egc *egc,
     }
 
     rc = libxl__get_domain_configuration(gc, domid, &d_config);
-    if (rc) goto out;
+    if (rc && rc != ERROR_JSON_CONFIG_EMPTY) goto out;
 
     device_add_domain_config(gc, &d_config, &libxl__disk_devtype,
                              &cis->disk_saved);
@@ -1257,7 +1500,10 @@ static void cdrom_insert_done(libxl__egc *egc,
 {
     EGC_GC;
 
+    LOGD(DEBUG, cis->domid, "rc=%d", rc);
+
     libxl__ev_time_deregister(gc, &cis->time);
+    libxl__ev_time_deregister(gc, &cis->timeout_retry);
     libxl__ev_qmp_dispose(gc, &cis->qmp);
     if (cis->qmp.payload_fd >= 0) close(cis->qmp.payload_fd);
     libxl__ev_slowlock_unlock(gc, &cis->qmp_lock);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 21:52:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 21:52:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674733.1049870 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVeyy-00016O-DZ; Thu, 01 Feb 2024 21:52:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674733.1049870; Thu, 01 Feb 2024 21:52:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVeyy-00016H-AU; Thu, 01 Feb 2024 21:52:28 +0000
Received: by outflank-mailman (input) for mailman id 674733;
 Thu, 01 Feb 2024 21:52:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVeyw-000165-Mk; Thu, 01 Feb 2024 21:52:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVeyw-0002fy-J0; Thu, 01 Feb 2024 21:52:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVeyw-000295-4t; Thu, 01 Feb 2024 21:52:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVeyw-0000bu-43; Thu, 01 Feb 2024 21:52:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OAZGLotUYbbSI2UwJ2NLfB1D9lGxLi6YCChWW9BCSbM=; b=ndRo2hG+/Nw36NqOmuhUWpXXMa
	A4EYOh5bKVy9fXKD+5sopjUVrjjG5tQ2luhAIfWBPHheZDJfKzDrP/i5Asb3Wa9ibX8bFkFEcgwlX
	LVP1AYKgLA4WnpxwW6XhNSUFQpdnyDovqKIhb2YPNGepHrooNgmZvDAEtGbfJkfW3ib0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184547-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184547: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=cc6ba68edf6dcd18c3865e7d7c0f1ed822796426
X-Osstest-Versions-That:
    xen=cc6ba68edf6dcd18c3865e7d7c0f1ed822796426
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 01 Feb 2024 21:52:26 +0000

flight 184547 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184547/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184535
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184535
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184535
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184535
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184535
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184535
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184535
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184535
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184535
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184535
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184535
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184535
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  cc6ba68edf6dcd18c3865e7d7c0f1ed822796426
baseline version:
 xen                  cc6ba68edf6dcd18c3865e7d7c0f1ed822796426

Last test of basis   184547  2024-02-01 03:35:56 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Thu Feb 01 22:17:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 22:17:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674740.1049880 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVfMw-0005X9-AU; Thu, 01 Feb 2024 22:17:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674740.1049880; Thu, 01 Feb 2024 22:17:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVfMw-0005X2-7v; Thu, 01 Feb 2024 22:17:14 +0000
Received: by outflank-mailman (input) for mailman id 674740;
 Thu, 01 Feb 2024 22:17:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVfMv-0005Ws-2r; Thu, 01 Feb 2024 22:17:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVfMu-0003BW-Vl; Thu, 01 Feb 2024 22:17:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVfMu-0003ge-Mf; Thu, 01 Feb 2024 22:17:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVfMu-0006mE-M7; Thu, 01 Feb 2024 22:17:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=O7GqCjqt6gMwofDuYONER2Y6n5th7tDHOWRvMLXzsNQ=; b=O7paYH4FaTNdXoKu+3+kHdWOsS
	7+OQYirz6ZtOxDgFbJVE7S8qAxEyyPjzOBjD1M0HEdudGI4T76n9D1+hArN4gKsxnCqtOkPZZP/Jg
	KRREzWNAt40xCdIVBE+75nOAxd4pixESUGi1XT21giSPuRtOrzE1u0nLwJ/l4PxNcZJ4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184556-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xtf test] 184556: all pass - PUSHED
X-Osstest-Versions-This:
    xtf=d1de27cf787e9c86610b11f6fe13d90f5ca91e27
X-Osstest-Versions-That:
    xtf=5a513bb7a7f08a0ab9aa23312ab9a34240a24d2c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 01 Feb 2024 22:17:12 +0000

flight 184556 xtf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184556/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xtf                  d1de27cf787e9c86610b11f6fe13d90f5ca91e27
baseline version:
 xtf                  5a513bb7a7f08a0ab9aa23312ab9a34240a24d2c

Last test of basis   184341  2024-01-13 14:42:52 Z   19 days
Testing same since   184556  2024-02-01 19:13:08 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Frediano Ziglio <frediano.ziglio@cloud.com>

jobs:
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-amd64-pvops                                            pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xtf.git
   5a513bb..d1de27c  d1de27cf787e9c86610b11f6fe13d90f5ca91e27 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Feb 01 22:44:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 01 Feb 2024 22:44:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674747.1049891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVfnh-00033R-F5; Thu, 01 Feb 2024 22:44:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674747.1049891; Thu, 01 Feb 2024 22:44:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVfnh-00033K-BZ; Thu, 01 Feb 2024 22:44:53 +0000
Received: by outflank-mailman (input) for mailman id 674747;
 Thu, 01 Feb 2024 22:44:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVfng-00033A-Iq; Thu, 01 Feb 2024 22:44:52 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVfng-0003jH-Em; Thu, 01 Feb 2024 22:44:52 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVfnf-0004sn-US; Thu, 01 Feb 2024 22:44:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVfnf-0004D2-Tx; Thu, 01 Feb 2024 22:44:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pFX4ytJVBCCPdUGLN3abIQvQ/CDpwbmpAaMimuMam8c=; b=YQUvMbnWqCZ8wSrXtgOiVPBIAU
	J74ywKy0wM9rbPooMqQ7vtMZNKtVfzqTeb9SNGPIHWTk3avpjSVyXaOjeodlJzY0q9EyDHBk5klqr
	H/hYTeZp5jFgB4fB6i/kEt/jDtSOerifoC5FIhsVrGniyb4VEXQkyJzJyoURDRmsshX4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184555-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184555: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=4c2d50d586717533f6bee4c6340be0432d0a76d0
X-Osstest-Versions-That:
    xen=bc45f20c01f1711bc56a4bb0955c49c182a5a03a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 01 Feb 2024 22:44:51 +0000

flight 184555 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184555/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  4c2d50d586717533f6bee4c6340be0432d0a76d0
baseline version:
 xen                  bc45f20c01f1711bc56a4bb0955c49c182a5a03a

Last test of basis   184552  2024-02-01 16:00:25 Z    0 days
Testing same since   184555  2024-02-01 19:00:28 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Julien Grall <jgrall@amazon.com>
  Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   bc45f20c01..4c2d50d586  4c2d50d586717533f6bee4c6340be0432d0a76d0 -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 00:40:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 00:40:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674759.1049909 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVhav-000328-Cm; Fri, 02 Feb 2024 00:39:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674759.1049909; Fri, 02 Feb 2024 00:39:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVhav-000321-9I; Fri, 02 Feb 2024 00:39:49 +0000
Received: by outflank-mailman (input) for mailman id 674759;
 Fri, 02 Feb 2024 00:39:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WgAc=JL=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVhat-00031v-9g
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 00:39:47 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8ff47a32-c163-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 01:39:45 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-511207d62b6so2372268e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 16:39:45 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 n11-20020a170906378b00b00a36a7f0f087sm298651ejc.222.2024.02.01.16.39.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 01 Feb 2024 16:39:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8ff47a32-c163-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706834384; x=1707439184; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=6EisdkplKjb+6gPzuhq8ok8YmwsZwgkvtvLjmMqQhmg=;
        b=susvAqfe8bZnZnLuHoM1qL1Q9CNoKjbTdyaH4yZ1FeL7MUbQDOjSDGHEvqS5zs5ZM7
         9R7E4ij/iXxR3WjdwdDnlpzyfDGSypFk6CbtWPJY/6nb7kuYljEbFpkXEnQVriNhTWKF
         Dadz4Xw58yz69AV3w+r3pK4dKN5a7zpo9Pu04=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706834384; x=1707439184;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=6EisdkplKjb+6gPzuhq8ok8YmwsZwgkvtvLjmMqQhmg=;
        b=RGuvGZ+LeLq0mBHPVlFSH1WRwtbwAZy/xmjZiuWf6GHV+khINItX9uOA71kyCImSjx
         8b+RVmcm//a+BPDm+ERrEOHsoZtDBH/ASwO5yQf1z1QcIKDK4KAfrl0kUmJAFkD0VsZS
         FWY7y3r5PTeVL2QKDhswvj93NDtVrHTdpWOkwI3CUy8u0Rqwp1f4zG5QlnibgXpoNXlr
         1S4oP/J3p3FAnk2Vvc1XrdwYXIZsyT/xEjpDMxH2DS/lEW+fZYdtBiZkksPpy75MaswO
         01CYrZkpvUCIkYX/j8SAV2wjLNKs9BcouFFp7UtztWoxsm2+VqvcN3nLLy+bVpvNazrX
         xi+g==
X-Gm-Message-State: AOJu0Yyb8kxk1xW4B2DSKxiHn0DP35gWTgEmjVLFCjyGteNV5y9f2o+1
	8rSjuNXPJ/d5QDL1oFJW+grSWB0fg0W35ttKYaxzLx9QFpyg6hXYBPA8pj7J2OjQcJRg727ZPrD
	x
X-Google-Smtp-Source: AGHT+IFpdZknAV/M2vLYEnw5xA1s2f+VaDjY4Kop0dtLfxUXeprQVZtqtiqIJFqKiGo20Izj1EMP9w==
X-Received: by 2002:a05:6512:707:b0:511:3622:508e with SMTP id b7-20020a056512070700b005113622508emr133273lfs.62.1706834384542;
        Thu, 01 Feb 2024 16:39:44 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCU9JRUni8fQW8KcKh/baFTMBFJUgblA7/G9GXFqu91nn3iBWBu1BznpGc8SMHt1M2ITUM2j+Oahx4t0r07ElbTECbVz4/gVuvnrZYMZ+RE5NiRBk4jF7Cx3vrD8FB3dPwrjvKg0DhzhtRa+9aNGj/lvmIesvneKmuGxeHf3NW7vR7pVOlqv53U2LcHHY1jOAw==
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH] xen: Swap order of actions in the FREE*() macros
Date: Fri,  2 Feb 2024 00:39:42 +0000
Message-Id: <20240202003942.647599-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Wherever possible, it is a good idea to NULL out the visible reference to an
object prior to freeing it.  The FREE*() macros already collect together both
parts, making it easy to adjust.

This has a marginal code generation improvement, as some of the calls to the
free() function can be tailcall optimised.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
---
 xen/include/xen/mm.h      | 3 ++-
 xen/include/xen/xmalloc.h | 7 ++++---
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 3d9b2d05a5c8..044f3f3b19c8 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -92,8 +92,9 @@ bool scrub_free_pages(void);
 
 /* Free an allocation, and zero the pointer to it. */
 #define FREE_XENHEAP_PAGES(p, o) do { \
-    free_xenheap_pages(p, o);         \
+    void *_ptr_ = (p);                \
     (p) = NULL;                       \
+    free_xenheap_pages(_ptr_, o);     \
 } while ( false )
 #define FREE_XENHEAP_PAGE(p) FREE_XENHEAP_PAGES(p, 0)
 
diff --git a/xen/include/xen/xmalloc.h b/xen/include/xen/xmalloc.h
index 9ecddbff5e00..1b88a83be879 100644
--- a/xen/include/xen/xmalloc.h
+++ b/xen/include/xen/xmalloc.h
@@ -66,9 +66,10 @@
 extern void xfree(void *p);
 
 /* Free an allocation, and zero the pointer to it. */
-#define XFREE(p) do { \
-    xfree(p);         \
-    (p) = NULL;       \
+#define XFREE(p) do {                       \
+    void *_ptr_ = (p);                      \
+    (p) = NULL;                             \
+    xfree(_ptr_);                           \
 } while ( false )
 
 /* Underlying functions */

base-commit: 3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 02:45:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 02:45:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674766.1049920 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVjXv-0007Px-9I; Fri, 02 Feb 2024 02:44:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674766.1049920; Fri, 02 Feb 2024 02:44:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVjXv-0007Pq-4O; Fri, 02 Feb 2024 02:44:51 +0000
Received: by outflank-mailman (input) for mailman id 674766;
 Fri, 02 Feb 2024 02:44:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVjXu-0007Pg-Ah; Fri, 02 Feb 2024 02:44:50 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVjXu-0004pF-8n; Fri, 02 Feb 2024 02:44:50 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVjXt-00019O-UL; Fri, 02 Feb 2024 02:44:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVjXt-0003PB-T7; Fri, 02 Feb 2024 02:44:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hhW6GWAgvcKa6LAkGlcAGM1bzm6hLjuhyzrydIb6APE=; b=Hl0xkJvRAiD/VUJ9IRAeGkxCqe
	nH/rkxq3nPsrvnplpj2QvEhnIAr19WtmfikI+COYan1iAW9pIitg9AB2xNGn55si6lMclPS+Jravm
	Ckj52K4fLymfFt9sIZOWUVtOiSeo1lLlSyt5H3J3asFfxG+Gxlry5dnqmqjQp1e7PVRs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184560-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184560: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
X-Osstest-Versions-That:
    xen=4c2d50d586717533f6bee4c6340be0432d0a76d0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 02 Feb 2024 02:44:49 +0000

flight 184560 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184560/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
baseline version:
 xen                  4c2d50d586717533f6bee4c6340be0432d0a76d0

Last test of basis   184555  2024-02-01 19:00:28 Z    0 days
Testing same since   184560  2024-02-01 23:00:27 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Christian Lindig <christian.lindig@cloud.com>
  Edwin Török <edwin.torok@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   4c2d50d586..3f819af8a7  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 02:53:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 02:53:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674772.1049930 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVjgL-0000gS-3Z; Fri, 02 Feb 2024 02:53:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674772.1049930; Fri, 02 Feb 2024 02:53:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVjgK-0000gL-VW; Fri, 02 Feb 2024 02:53:32 +0000
Received: by outflank-mailman (input) for mailman id 674772;
 Fri, 02 Feb 2024 02:53:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVjgJ-0000gB-NY; Fri, 02 Feb 2024 02:53:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVjgJ-0004za-KB; Fri, 02 Feb 2024 02:53:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVjgH-0001zU-VS; Fri, 02 Feb 2024 02:53:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVjgH-0006gN-Uw; Fri, 02 Feb 2024 02:53:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=H8c4Dfs5Z7PBHUHy53oxA4h0I7rmKvPoqFELopbcvxo=; b=iOq8Fype+HhVa4pAzn4oKG0yxq
	N8qUA+A8rPr0J6HRNV2R4HOqfKZzR3aU8WLLAl4JZFPNBwVFBkGrgpRmG2aayZSthfJZIHzOj07HP
	RaKF51lTzT4/57LOnA/Gpts7NUfM54ExszQPnEHjXrD4QCbK0r9QiHTTsTP+O6FLn7zA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184558-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184558: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=3656352675bf66c06b65bf85632996d3471073ae
X-Osstest-Versions-That:
    ovmf=97c3f5b8d27230acfc20f479adea64c348750612
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 02 Feb 2024 02:53:29 +0000

flight 184558 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184558/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 3656352675bf66c06b65bf85632996d3471073ae
baseline version:
 ovmf                 97c3f5b8d27230acfc20f479adea64c348750612

Last test of basis   184542  2024-01-31 13:12:38 Z    1 days
Testing same since   184558  2024-02-01 22:41:22 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  MarsX Lin <marsx.lin@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   97c3f5b8d2..3656352675  3656352675bf66c06b65bf85632996d3471073ae -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 03:30:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 03:30:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674778.1049940 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVkFr-0005uv-QR; Fri, 02 Feb 2024 03:30:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674778.1049940; Fri, 02 Feb 2024 03:30:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVkFr-0005uo-Nr; Fri, 02 Feb 2024 03:30:15 +0000
Received: by outflank-mailman (input) for mailman id 674778;
 Fri, 02 Feb 2024 03:30:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rVkFp-0005ue-Rb
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 03:30:13 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2418::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5ed78fb9-c17b-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 04:30:12 +0100 (CET)
Received: from CYZPR05CA0020.namprd05.prod.outlook.com (2603:10b6:930:89::23)
 by DS0PR12MB7559.namprd12.prod.outlook.com (2603:10b6:8:134::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.22; Fri, 2 Feb
 2024 03:30:07 +0000
Received: from CY4PEPF0000EE35.namprd05.prod.outlook.com
 (2603:10b6:930:89:cafe::17) by CYZPR05CA0020.outlook.office365.com
 (2603:10b6:930:89::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.24 via Frontend
 Transport; Fri, 2 Feb 2024 03:30:07 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000EE35.mail.protection.outlook.com (10.167.242.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 03:30:07 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Thu, 1 Feb
 2024 21:30:06 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Thu, 1 Feb
 2024 21:30:06 -0600
Received: from [172.23.114.18] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Thu, 1 Feb 2024 21:30:05 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5ed78fb9-c17b-11ee-8a43-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n7eRCLc3E9JBVUpiEhfvFp0esr7gWFfyr7lPlqBfz5R1SvWmRGcF8THVK0Afp1QnIyohgMllUTQyBZd+j+DQnTFmU6KollX5UerdQ/HYPG/lfqrLuSNFQ5qkBo1wd1GPdsAFvuv7SccUlO/ecwRSQhalNOUTYFZiHw8zF9CPngthLTz+ieTbJMUv4Cui0ISyj1C2ssigS1OJ1i94XJ+HtpYqEbHjJ0YFMMeiw2+efxFOR5U4hYPWv9Ih/5GLYhwdisY5KXZbnL9pBgAdfIGrHNOp7PXZw4LVDhykZZt5uJWkRHKkOBJTS59qBurp5nkPdbtQwCntjwRFuUfd6kLCwQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=v6WFUKV9+5WrvLvThFZR31umrBLy7SAfqc1e6Ju/6XM=;
 b=iWVHX8sQE6YvKeTJSSI330KXUUkIUiBCnXGXEwYgHUVfXcKNkTJCMUcCVg4+QMa+UpFGTTNoHjTOjg4oM+zAPZy8e/ZWWlnMvfkVmC+ShSfKWevbGrV3d8i0OD1CxSb4txlALBuL//gOzV+fH26SJp5zR6rfYuTH+jcRQRnilnIGcrT5IQg2wDhG6X8B3BKkdDsxtD8s4plJQVDNS3gL9SOuVod0tZOijl442B1D/kcjkCUpSb8MkAj0k7DZCa2CJp3SiS++4uecAxWdKWnju9cm/7adFwIw+kCrj+0eBsR3sFzV/JHfkwAcaqO9NX5j0El3f38AMPBbTzLxvkPLAQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=v6WFUKV9+5WrvLvThFZR31umrBLy7SAfqc1e6Ju/6XM=;
 b=HHuKH9pEZp808sHjrvT7izCd3tlhwQL18qDfJ8IJD33ODCx2N3sKewzEng8rEZahitN73IpdTcyGRyG6eqh4rzHRrzmI+IZIn5NTjsHZS6M6HJg8AAtAiqC8tavGhKZwwDvg3XkZeEK6h/KNrnX31zGO1133Go2OvcEphWsvEYM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <3b6af1cb-c234-430e-b92a-2479b42de184@amd.com>
Date: Thu, 1 Feb 2024 22:30:05 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 11/15] vpci: add initial support for virtual PCI bus
 topology
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
	"Julien Grall" <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	<xen-devel@lists.xenproject.org>
References: <20240109215145.430207-1-stewart.hildebrand@amd.com>
 <20240109215145.430207-12-stewart.hildebrand@amd.com>
 <ec947a93-3026-4119-84dc-09785fab8f7f@suse.com>
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <ec947a93-3026-4119-84dc-09785fab8f7f@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE35:EE_|DS0PR12MB7559:EE_
X-MS-Office365-Filtering-Correlation-Id: 3806993a-290d-4001-0523-08dc239f40f0
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	X8dexJi6QM+RF3vURXIr26hFIG0TgCXCmPxNqyY4evCZ5aVhNGgp2RMWMzcRQkOc4/6gX3l5EjqrNHP8aQZOwK8bngTFGkDThv/+Vf70jyvROT6z63L8yZ5uOWrUoubKGjtKxBx+eG/g4icoj8x9NVxELGZX2FnYzAEBt49d0k1uZFDSILaTbZvJOsKWTG/1nAHOi4HqHBI61hr9qDCDOhqc+3Hyhtt4syg90Z9kSmzPNQY2I29aNT47WoE6e3Rd4OnFXmKvwpKmTezipUxPIiD8UX5ywygYNBEK5PdxsytBkQXgwMOiIL/ggVOSTVjiY+ibSAxmQLY5SwQa/R/teIjTVfUn5vObjOUHxImBplELEI5UXW9N/NUSTp6BqPAdfpqTqsV9E4B1a/CaQVf0Zzpc2ynmBHq7EjRnNI3xl9Y9RQH3kpni/8eMUoS0N/CRXEDFtrj1b2HYIWQl56OYojezpKq86qmcsNfVhU+dkPLMg31xNEwCbOY27HF+WpbjcExXh8xpABsD8Xri1l3yQnzc4cKY3/R8BHlUEPLgV1I2yZ43punhGIau20n68OjbTfAU60J7lQQeyG7herXNV6K/C1oUhs2dwWA8GBPikpG4MfVnOQTlH8Olw4g2HWTPIMIH94XU6K0dlod7h/jvPq5q41S4KJbsr1isw2FDjHxV295VqGfldvIAjL+GAzGJG7OcDWVXhzaLl/sSXcQRyuhWjYfTuxyAdPV+CpRYpChhO7ODhV1peEiPMB6jDU7vOUMycbZRhgBXs50mNUQaoXZCgwTlcJudYbbNjt/0iy089A/yaZ0wxIOn8/OKl5Gh
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(136003)(346002)(396003)(39860400002)(230922051799003)(64100799003)(82310400011)(1800799012)(451199024)(186009)(46966006)(36840700001)(40470700004)(41300700001)(83380400001)(47076005)(336012)(26005)(2616005)(36860700001)(5660300002)(81166007)(8936002)(44832011)(7416002)(54906003)(8676002)(16576012)(478600001)(2906002)(82740400003)(70586007)(70206006)(6916009)(53546011)(316002)(4326008)(426003)(31696002)(36756003)(86362001)(356005)(40460700003)(40480700001)(31686004)(36900700001)(43740500002);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 03:30:07.5074
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3806993a-290d-4001-0523-08dc239f40f0
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EE35.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7559

On 1/25/24 11:00, Jan Beulich wrote:
> On 09.01.2024 22:51, Stewart Hildebrand wrote:
>> --- a/xen/drivers/Kconfig
>> +++ b/xen/drivers/Kconfig
>> @@ -15,4 +15,8 @@ source "drivers/video/Kconfig"
>>  config HAS_VPCI
>>  	bool
>>  
>> +config HAS_VPCI_GUEST_SUPPORT
>> +	bool
>> +	depends on HAS_VPCI
> 
> Wouldn't this better be "select", or even just "imply"?

I prefer "select"

> 
>> --- a/xen/drivers/vpci/vpci.c
>> +++ b/xen/drivers/vpci/vpci.c
>> @@ -40,6 +40,49 @@ extern vpci_register_init_t *const __start_vpci_array[];
>>  extern vpci_register_init_t *const __end_vpci_array[];
>>  #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
>>  
>> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
>> +static int add_virtual_device(struct pci_dev *pdev)
>> +{
>> +    struct domain *d = pdev->domain;
>> +    unsigned int new_dev_number;
>> +
>> +    if ( is_hardware_domain(d) )
>> +        return 0;
>> +
>> +    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
>> +
>> +    /*
>> +     * Each PCI bus supports 32 devices/slots at max or up to 256 when
>> +     * there are multi-function ones which are not yet supported.
>> +     */
>> +    if ( pdev->info.is_extfn && !pdev->info.is_virtfn )
>> +    {
>> +        gdprintk(XENLOG_ERR, "%pp: only function 0 passthrough supported\n",
>> +                 &pdev->sbdf);
> 
> The message suggests you ought to check pdev->devfn to have the low
> three bits clear. Yet what you check are two booleans.

I'll check pdev->sbdf.fn

> 
> Further doesn't this require the multi-function bit to be emulated
> clear?

I consider this to be future work. The header type register, where the
bit resides, is not yet emulated for domUs. I have a series in the works
for emulating additional registers (including PCI_HEADER_TYPE), but I'm
planning to wait to submit it until after the current series is finished
so as to not delay the current series any further.

> And finally don't you then also need to disallow assignment of
> devices with phantom functions?

No, I don't think so. My understanding is that there is no configuration
space associated with the phantom SBDFs. There's no special handling
required in vPCI per se, because the phantom function RIDs get mapped
in the IOMMU when the device gets assigned. Future work would include
exposing the PCI Express Capability, including device control register
with the phantom function enable bit. I say this having only done
limited testing of phantom functions on ARM, and by faking it using the
pci-phantom= Xen arg because I don't have a real device with phantom
function capability.

> 
>> --- a/xen/include/xen/sched.h
>> +++ b/xen/include/xen/sched.h
>> @@ -484,6 +484,14 @@ struct domain
>>       * 2. pdev->vpci->lock
>>       */
>>      rwlock_t pci_lock;
>> +#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
>> +    /*
>> +     * The bitmap which shows which device numbers are already used by the
>> +     * virtual PCI bus topology and is used to assign a unique SBDF to the
>> +     * next passed through virtual PCI device.
>> +     */
>> +    DECLARE_BITMAP(vpci_dev_assigned_map, VPCI_MAX_VIRT_DEV);
>> +#endif
>>  #endif
> 
> With this the 2nd #endif would likely better gain a comment.

I will add it. Actually, I see no harm in adding a comment for both of
these #endifs.

> 
>> --- a/xen/include/xen/vpci.h
>> +++ b/xen/include/xen/vpci.h
>> @@ -21,6 +21,13 @@ typedef int vpci_register_init_t(struct pci_dev *dev);
>>  
>>  #define VPCI_ECAM_BDF(addr)     (((addr) & 0x0ffff000) >> 12)
>>  
>> +/*
>> + * Maximum number of devices supported by the virtual bus topology:
>> + * each PCI bus supports 32 devices/slots at max or up to 256 when
>> + * there are multi-function ones which are not yet supported.
>> + */
>> +#define VPCI_MAX_VIRT_DEV       (PCI_SLOT(~0) + 1)
> 
> The limit being this means only bus 0 / seg 0 is supported, which I
> think the comment would better also say. (In add_virtual_device(),
> which has a similar comment, there's then at least a 2nd one saying
> so.)

OK, I'll adjust the comment.


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 05:03:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 05:03:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674785.1049950 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVli5-0001IG-AR; Fri, 02 Feb 2024 05:03:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674785.1049950; Fri, 02 Feb 2024 05:03:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVli5-0001I9-6H; Fri, 02 Feb 2024 05:03:29 +0000
Received: by outflank-mailman (input) for mailman id 674785;
 Fri, 02 Feb 2024 05:03:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVli4-0001Hx-03; Fri, 02 Feb 2024 05:03:28 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVli3-00082K-U2; Fri, 02 Feb 2024 05:03:27 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVli3-0007Vl-Jx; Fri, 02 Feb 2024 05:03:27 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVli3-0000KJ-JZ; Fri, 02 Feb 2024 05:03:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HLKjIG71iBLJBIPvNPKHr/VY/3JrgM2iFFXw0DG0wh0=; b=RMkKFTwbbooxiT9sj8J+zsXRbF
	Moj0Hn3lJb3mdUBdnJG2JRYjL9iHjq5siNaZt+R0oTdixntTiNCJCjQcv5FehTpRaQBidcDlvCsia
	MAOXleEWwavgr8HSBBNchhy0+B/I3utP7b5BpXP1Jq7d0SYqN59QUqkGNDOEbS3w3Jj4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184561-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184561: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=927ea1364d047d54cb67e0f231644f295c032944
X-Osstest-Versions-That:
    ovmf=3656352675bf66c06b65bf85632996d3471073ae
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 02 Feb 2024 05:03:27 +0000

flight 184561 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184561/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 927ea1364d047d54cb67e0f231644f295c032944
baseline version:
 ovmf                 3656352675bf66c06b65bf85632996d3471073ae

Last test of basis   184558  2024-02-01 22:41:22 Z    0 days
Testing same since   184561  2024-02-02 02:55:40 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dongyan Qian <qiandongyan@loongson.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   3656352675..927ea1364d  927ea1364d047d54cb67e0f231644f295c032944 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 05:21:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 05:21:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674795.1049959 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVlzU-00043h-QN; Fri, 02 Feb 2024 05:21:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674795.1049959; Fri, 02 Feb 2024 05:21:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVlzU-00043a-Ni; Fri, 02 Feb 2024 05:21:28 +0000
Received: by outflank-mailman (input) for mailman id 674795;
 Fri, 02 Feb 2024 05:21:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVlzT-00043Q-Cx; Fri, 02 Feb 2024 05:21:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVlzT-0008Ih-Ae; Fri, 02 Feb 2024 05:21:27 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVlzS-00087v-Tc; Fri, 02 Feb 2024 05:21:27 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVlzS-0006X7-TA; Fri, 02 Feb 2024 05:21:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PR036MYZWGx5li+8CxrmZHckMSJJOF6t1Iqx0x6CieQ=; b=Q9bGDXgq8z3F5LBZ5C8zsIovUV
	0c+eprLoCkvj7B8hbNmmIyjVkhrUAlvq/xWZjrqohd39YY+7+/HBtkR0kVv/OUKe/YPNv2Obb7ZVO
	wjWBRfBfifGc4IG9qxgWpqFELrz+22TgIhAJU7fSD2sozKjK9z7g4VDwFF6hPaaBPg+w=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184549-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-6.1 baseline test] 184549: tolerable FAIL
X-Osstest-Failures:
    linux-6.1:test-armhf-armhf-xl:xen-boot:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:xen-boot:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=8fd7f44624538675abadc73f5a44e95016964d22
X-Osstest-Versions-That:
    linux=8fd7f44624538675abadc73f5a44e95016964d22
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 02 Feb 2024 05:21:26 +0000

"Old" tested version had not actually been tested; therefore in this
flight we test it, rather than a new candidate.  The baseline, if
any, is the most recent actually tested revision.

flight 184549 linux-6.1 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184549/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl           8 xen-boot                fail baseline untested
 test-armhf-armhf-xl-arndale   8 xen-boot                fail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop      fail baseline untested
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop      fail baseline untested
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail baseline untested
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check fail baseline untested
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop      fail baseline untested
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop      fail baseline untested
 test-armhf-armhf-libvirt   16 saverestore-support-check fail baseline untested
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail baseline untested
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 linux                8fd7f44624538675abadc73f5a44e95016964d22
baseline version:
 linux                8fd7f44624538675abadc73f5a44e95016964d22

Last test of basis   184549  2024-02-01 11:13:03 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  fail    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 06:57:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 06:57:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674804.1049970 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVnUJ-0006Xp-QH; Fri, 02 Feb 2024 06:57:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674804.1049970; Fri, 02 Feb 2024 06:57:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVnUJ-0006Xi-Lv; Fri, 02 Feb 2024 06:57:23 +0000
Received: by outflank-mailman (input) for mailman id 674804;
 Fri, 02 Feb 2024 06:57:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WBRZ=JL=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVnUH-0006Oy-FE
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 06:57:21 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4d054f50-c198-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 07:57:16 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a2f79e79f0cso241742366b.2
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 22:57:16 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 jl17-20020a17090775d100b00a35e26d86dfsm551834ejc.201.2024.02.01.22.57.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 22:57:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d054f50-c198-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706857036; x=1707461836; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=kZdjfu8oAFQ8X474kws54qQg6FUjkj9qrq3asMu0rq0=;
        b=gFtsKi7XlrUiKuLnxLOvRjbLDLYcBBV3osy8p1EiaJQYG+gWm09+aQXMbDJYSfeivn
         dXC/cXUUMTDVzhRnvlU39O9GPHnUPTxydlTjPkLfAg+uA4dGBzIKvS1fry003hy5kVD5
         Ht+v65qcbf/HX6sXWbITvLBrCufOUtsNIAMVm0fYwg4dxEJYRPSSUw2bK3L8agr+uFkB
         sH3TI1zYtoRXscwgRZ9IeE/ULwY/Y+YYCbqgMLRcOPaPAL+yPRVq9AYDVpYAEmHFSxTq
         O0EauO3l+WA90QCZp1ObJKp0tZYin3pgZCJsOPj/XjTJ+fBT2FXmeahrlaEn+6etVPPc
         WaCg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706857036; x=1707461836;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kZdjfu8oAFQ8X474kws54qQg6FUjkj9qrq3asMu0rq0=;
        b=e1u9QQFXtkliJKDJ5nvKYeNJrXqaphp384Bq7KRXQPEU9yoWfwOGWGr1ZL6bNhfVlT
         wTBalbyovnDDt3+qnUhaPRaA9RIHZhP0glHS6GKdIhcaMIUhkF1zsKrsvLuB8CEkiObD
         VAi0VWxcnyg+y0GBIJJZkIsgDMTfydYYRxWzUtM56RgGxWhGI8estSiszSnMlFnn5KId
         aTJgl0+1ztkFbrTJCxHIvZc35sHePS7eqp3SpKB/eLJlqCkJS42KT9GDsqbBq6J+w779
         8eTWPrE1x5Ysc1+ImIBuw7LAVGEOBj3iHBShPtba+GB7+qKQHDycN/RdyNTAxE1yLir+
         FioA==
X-Gm-Message-State: AOJu0YzHpVYk/E3XiZWUbntrS0NGihd1xb+NCaAOPR5pCcbwsS0AbLUh
	aahpF/0hpWhOXWXEQAaegcMglAMEVv36bpaDssckjbEWcbQCg3SQxgpH15un1w==
X-Google-Smtp-Source: AGHT+IExR5qZZrfPvrU/IqwjJ19q+7TVxlRszalZ3FzsrP3NwvfF7qWgf9e+xw9hJm/JfojtBmkBiQ==
X-Received: by 2002:a17:906:5fd7:b0:a35:ba57:42bc with SMTP id k23-20020a1709065fd700b00a35ba5742bcmr5234974ejv.64.1706857036195;
        Thu, 01 Feb 2024 22:57:16 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCV2wagA81I2EirX3rvEEaNf//FvaM1mOUpFsOxwjO98GHHbKF/3lCYGNlUVHKU5w1Mq4auhl4hOz+jr1e3YbBvOQMfMpt2cgoVl90wXolyEMH/G84OTWBun8pKOV+Oz4wqKsXPcvWGn5c5Jb5LHS1NzgHRd4gKUzp1f1tn1zaXUL8Q7
Message-ID: <27632e93-482d-459d-9dec-12429a02f588@suse.com>
Date: Fri, 2 Feb 2024 07:57:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen: Swap order of actions in the FREE*() macros
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240202003942.647599-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240202003942.647599-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 01:39, Andrew Cooper wrote:
> Wherever possible, it is a good idea to NULL out the visible reference to an
> object prior to freeing it.  The FREE*() macros already collect together both
> parts, making it easy to adjust.
> 
> This has a marginal code generation improvement, as some of the calls to the
> free() function can be tailcall optimised.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

However, ...

> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -92,8 +92,9 @@ bool scrub_free_pages(void);
>  
>  /* Free an allocation, and zero the pointer to it. */
>  #define FREE_XENHEAP_PAGES(p, o) do { \
> -    free_xenheap_pages(p, o);         \
> +    void *_ptr_ = (p);                \

... why a trailing _and_ a leading underscore? Sooner or later we'll
need to get rid of the leading ones anyway aiui (for Misra), here
and elsewhere. With it omitted right away (also below):
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 07:08:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 07:08:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674808.1049980 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVner-0008Mv-PB; Fri, 02 Feb 2024 07:08:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674808.1049980; Fri, 02 Feb 2024 07:08:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVner-0008Mo-LV; Fri, 02 Feb 2024 07:08:17 +0000
Received: by outflank-mailman (input) for mailman id 674808;
 Fri, 02 Feb 2024 07:08:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WBRZ=JL=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVneq-0008Mi-U9
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 07:08:16 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d5c07227-c199-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 08:08:15 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a2f22bfb4e6so245034366b.0
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 23:08:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e25-20020a170906045900b00a3595d6c358sm565046eja.36.2024.02.01.23.08.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 23:08:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5c07227-c199-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706857695; x=1707462495; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=qut6hrSVzJQhMvY4HslmBnFgt2PyOl+xHe7+bdJSOB0=;
        b=YmzKSraLy0CUgF5772CygelCgbswZrSjwASKyYg+fYjOLs2ilZDAd9iIMJeOd9LIkL
         JnbRBCdq8nxsqnMTJV0th4wKs9oDtMbBDPwbEf+rVUozcb4krsBj6WMqXHhlOrZGJ6mn
         tl2+7N0BEq8aNgi3jVP1p0tADJvrmYD8uvaXeJrLyj5X+aQWtcFkCzHKQDLq6/ZEOOa+
         0LPdTCNx8p1vM2yPPOrn/nHncmX8dW/uct40fqAAuZzVic5xK7LMxVBP5N+bwaspXLFc
         IeBLTqPb+ZM6+UdyHmgXrPNcXnGjRX1+TMr+fVRN/tNyUtV99mqGrxldRgzDu86H5PBp
         bS/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706857695; x=1707462495;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=qut6hrSVzJQhMvY4HslmBnFgt2PyOl+xHe7+bdJSOB0=;
        b=SxqRd7lEguaMINR5zpjgfUU5lKx1OorS8f+KQyT0OKQwA/NkC3EZSyp42Z1YFzHLad
         mT9SjEyPP5FLYdvvqzouLJV+jGtCznRU900OT9h48rDk0qOQsrm42OHF/GCwpz9CSLSZ
         ZcmdSecIT50Gsotm0CUx1JI6E8WmVNv3vdntgA/aCreBZOwqpek55Dk87y0PFycoI4JI
         mtBXd0p0wUCCn+Udb4qMeJo0TB3KTO50H6FaHLD2OcOgICClSRZ8cD5yJvYRHDoNd8Eh
         5J0OsdnVHRyfnUhR/5ex/TGCTf7gV23EmRXMYG0RqGxs+b7+tNgy9j+zvxu4CUOU0JGX
         L8lA==
X-Gm-Message-State: AOJu0Yyg8Run4vQxzUMJNJ9v7YkOH73qW/XZWZixplYArWmFzXpPy/K8
	SAuRki9hY74M+jDNnwlbTBu17ZUVDeO2XHbh5qQLa3gbHON3OFKu0zCTvXXBeQ==
X-Google-Smtp-Source: AGHT+IF8QUpHOuWGL26Obv9O3MUPTg9JAFlAEzuov8NWx6XZwsj8bBFrGCfH5gDhYcQegapQChvSMA==
X-Received: by 2002:a17:906:1b57:b0:a31:2119:90d7 with SMTP id p23-20020a1709061b5700b00a31211990d7mr4828947ejg.38.1706857695063;
        Thu, 01 Feb 2024 23:08:15 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCU+wUiKxQTdHpr+cTlfe/Zp7o489Qh6xsY2+6KaAXjdb+DSy05eBkT0eYtDuE+W8pOwslGeewWtxtCTVkXuoiUAzVAComsNRuh17kgWCPfVRN9hOWN+2Ut7mRA+hpjVpoCHqaY8mBm46dfn/IpbG3J1
Message-ID: <578a6845-6046-4d15-88a8-8a34936095a7@suse.com>
Date: Fri, 2 Feb 2024 08:08:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Ping: [PATCH] x86/guest: finish conversion to altcall
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Paul Durrant <paul@xen.org>, Wei Liu <wl@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <65fb766e-ab5a-49ab-a904-01ee9711d42d@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <65fb766e-ab5a-49ab-a904-01ee9711d42d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 17.01.2024 10:31, Jan Beulich wrote:
> While .setup() and .e820_fixup() don't need fiddling with for being run
> only very early, both .ap_setup() and .resume() want converting too:
> This way both pre-filled struct hypervisor_ops instances can become
> __initconst_cf_clobber, thus allowing to eliminate up to 5 more ENDBR
> (configuration dependent) during the 2nd phase of alternatives patching.
> 
> While fiddling with section annotations here, also move "ops" itself to
> .data.ro_after_init.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

May I ask for an ack (or otherwise)?

Thanks, Jan

> --- a/xen/arch/x86/guest/hyperv/hyperv.c
> +++ b/xen/arch/x86/guest/hyperv/hyperv.c
> @@ -207,7 +207,7 @@ static int cf_check flush_tlb(
>      return hyperv_flush_tlb(mask, va, flags);
>  }
>  
> -static const struct hypervisor_ops __initconstrel ops = {
> +static const struct hypervisor_ops __initconst_cf_clobber ops = {
>      .name = "Hyper-V",
>      .setup = setup,
>      .ap_setup = ap_setup,
> --- a/xen/arch/x86/guest/hypervisor.c
> +++ b/xen/arch/x86/guest/hypervisor.c
> @@ -13,7 +13,7 @@
>  #include <asm/cache.h>
>  #include <asm/guest.h>
>  
> -static struct hypervisor_ops __read_mostly ops;
> +static struct hypervisor_ops __ro_after_init ops;
>  
>  const char *__init hypervisor_probe(void)
>  {
> @@ -49,7 +49,7 @@ void __init hypervisor_setup(void)
>  int hypervisor_ap_setup(void)
>  {
>      if ( ops.ap_setup )
> -        return ops.ap_setup();
> +        return alternative_call(ops.ap_setup);
>  
>      return 0;
>  }
> @@ -57,7 +57,7 @@ int hypervisor_ap_setup(void)
>  void hypervisor_resume(void)
>  {
>      if ( ops.resume )
> -        ops.resume();
> +        alternative_vcall(ops.resume);
>  }
>  
>  void __init hypervisor_e820_fixup(void)
> --- a/xen/arch/x86/guest/xen/xen.c
> +++ b/xen/arch/x86/guest/xen/xen.c
> @@ -318,7 +318,7 @@ static int cf_check flush_tlb(
>      return xen_hypercall_hvm_op(HVMOP_flush_tlbs, NULL);
>  }
>  
> -static const struct hypervisor_ops __initconstrel ops = {
> +static const struct hypervisor_ops __initconst_cf_clobber ops = {
>      .name = "Xen",
>      .setup = setup,
>      .ap_setup = ap_setup,



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 07:55:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 07:55:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674852.1050006 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVoOl-0006fB-KZ; Fri, 02 Feb 2024 07:55:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674852.1050006; Fri, 02 Feb 2024 07:55:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVoOl-0006f4-GG; Fri, 02 Feb 2024 07:55:43 +0000
Received: by outflank-mailman (input) for mailman id 674852;
 Fri, 02 Feb 2024 07:55:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WBRZ=JL=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVoOj-0006ey-Q4
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 07:55:41 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7579f229-c1a0-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 08:55:40 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a318ccfe412so197398166b.1
 for <xen-devel@lists.xenproject.org>; Thu, 01 Feb 2024 23:55:40 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 dc20-20020a170906c7d400b00a36cd9624c0sm610402ejb.95.2024.02.01.23.55.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 01 Feb 2024 23:55:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7579f229-c1a0-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706860540; x=1707465340; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=uyMkE39StgLvA3SInVPI0AARfSkny9Aun6vln2isN24=;
        b=caCr4Ta9mmssafIb6p4wAmjnImJ7nBGHRKZfaVxDX3MLoKXx5cC2xK/A5vqGF/63vV
         Q5L2rZS9RR5Igz2C+SWVLrskXbz3yLYAj0fN2c3v4Ml5swRGJaR2FyMLkiiapBGuBmHa
         SsL7ZBSJd7UbwZyZEOjOf28j74ufMSHvYHHhoAuj9kttzxS1Xqifb2RqQTRu1TBWlHNn
         JtzUcfUzBvGjDzf5d9JAGvpGATxpiuhPRazmo/1hRDvBev3SyJ8tNDOK/9x244eXxAwZ
         rCY3aB0xht7Vf6hA9XJbzjWNzljfYO5chCNv8aZjLVwn4wmnWzL21Ar6WJRbbFSVKbH7
         NpbQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706860540; x=1707465340;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=uyMkE39StgLvA3SInVPI0AARfSkny9Aun6vln2isN24=;
        b=mnYKG4kfJUDxtLoRXfj0axSDhf1MrhPEJbaENK29O2qcXt1uSHAeRuvUTdT7MM3yCl
         lu0+NJXEQJMTksycNMlHo3rl3nQO24/iedtcvSdwCL14+Nd9NhawctQz8K1t9duR2Wm6
         VnHAXI2P9upUzGDTRpxZl3xr/X23XqL59/hm1l/HVvjL9K0hvAs7kDUX2umuxEXjQIY4
         TDFVYe+p2Q5gFznIc20k3v3MZMTBDHTkTpTqrqi9R/pY1usXgpildz4vERoIkfMB/eB/
         Jq94ZsCGTkO8BcjqPWJY67UMOG2X5bh4dyYx2E0PIvFQdjisp2AvWbLFKHAxxXnsfKum
         v3qQ==
X-Gm-Message-State: AOJu0YxihOaSD6rPw6dV8plIF7qnk5lUrsXN5OUFbLERNptI4cB+pfS/
	arPXHvhGl3qOXzoY7BLxcoshulTrzt2mySrjxweNGjOFZNR+8+2ZquhfGp7a6Q==
X-Google-Smtp-Source: AGHT+IG4wdKiNu3Gu8m5mk1omt4OcK7bNheqFkPbehysCvBX0GDy6UX6XToxoJbnXJkQK1QxfzXmwA==
X-Received: by 2002:a17:906:7256:b0:a36:c8fa:45a8 with SMTP id n22-20020a170906725600b00a36c8fa45a8mr978261ejk.24.1706860540122;
        Thu, 01 Feb 2024 23:55:40 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWfNhn1syeRgH0Sih+5MJL270DZYyPFO08Dcbc8czQ/EJvByM26xiU5/VVJbjCCTEfz4nCXI+APMIjCy+0V6KFLvryBFiuCXlTLgekNDRA5CUOB0Xt4zqTlKsIQOlLWlPm7+5hz
Message-ID: <90e22c7c-866f-402a-ac47-6eb0e094fce4@suse.com>
Date: Fri, 2 Feb 2024 08:55:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86: arrange for ENDBR zapping from
 <vendor>_ctxt_switch_masking()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <c19cf600-5971-457b-936d-77a035ab6913@suse.com>
 <e42ee8c5-10b0-4369-8cc9-7b298fa68ef7@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e42ee8c5-10b0-4369-8cc9-7b298fa68ef7@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.02.2024 22:12, Andrew Cooper wrote:
> On 16/01/2024 4:53 pm, Jan Beulich wrote:
>> While altcall is already used for them, the functions want announcing in
>> .init.rodata.cf_clobber, even if the resulting static variables aren't
>> otherwise used.
>>
>> While doing this also move ctxt_switch_masking to .data.ro_after_init.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Thanks.

>> --- a/xen/arch/x86/cpu/amd.c
>> +++ b/xen/arch/x86/cpu/amd.c
>> @@ -258,6 +258,11 @@ static void cf_check amd_ctxt_switch_mas
>>  #undef LAZY
>>  }
>>  
>> +#ifdef CONFIG_XEN_IBT /* Announce the function to ENDBR clobbering logic. */
>> +static const typeof(ctxt_switch_masking) __initconst_cf_clobber __used csm =
>> +    amd_ctxt_switch_masking;
>> +#endif
> 
> If we gain more of these, I suspect we'll want a wrapper for it.
> 
> Irritatingly you can't pass parameters into global asm, because the nice
> way to do this would be an _ASM_PTR in a pushsection.

While I'm not convinced resorting to asm() here would indeed be a good thing,
for very many years I've been carrying a gcc change to permit exactly this.
I don't even recall anymore why it wasn't liked upstream.

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 09:02:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 09:02:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674868.1050020 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVpQi-0007vh-Pn; Fri, 02 Feb 2024 09:01:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674868.1050020; Fri, 02 Feb 2024 09:01:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVpQi-0007va-Lx; Fri, 02 Feb 2024 09:01:48 +0000
Received: by outflank-mailman (input) for mailman id 674868;
 Fri, 02 Feb 2024 09:01:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JhER=JL=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rVpQh-0007vO-Cq
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 09:01:47 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b09de6a1-c1a9-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 10:01:45 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-511207d62b6so2811759e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 02 Feb 2024 01:01:45 -0800 (PST)
Received: from [192.168.12.38] (54-240-197-238.amazon.com. [54.240.197.238])
 by smtp.gmail.com with ESMTPSA id
 s15-20020a05600c45cf00b0040e3635ca65sm6854679wmo.2.2024.02.02.01.01.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 02 Feb 2024 01:01:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b09de6a1-c1a9-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706864505; x=1707469305; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:references:cc:to
         :content-language:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=zXErrdCa6Tr/iUNLljuNLnYOqYVTtO+yEzmXtzl8oAg=;
        b=QSpWPwS7NJBl7jNg6qP/NIhIEulmG2TNCdo6uCZ4Hu2JBNZ7zWtiBupLYs5G/uTmY/
         o5QMrBL1dvWjIxcA5u2pOHtGYJtE0Iqr+V0ZgKglyQvbzYfuZcb4IautqKtupxQQFj/R
         DzrvKisECCdHj3lPh5mr9ja7+9/VD1I2Mo8S5F/IZryGB+BKiQbAplGj9SKtyTNApbf4
         BvzlSKvZWy9SwzcX5oQcK3wTTUZiqfcYTSa7wHCrac+bMcn4ofGuhLtyxAdqu1iAHAnw
         Y3MJxITNGUt5fKx6gtjSnBhcYfObUjG09V6vnRW3EeP84LjMs5RveG9XzhE+Bp1QFdi0
         JzlQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706864505; x=1707469305;
        h=content-transfer-encoding:in-reply-to:organization:references:cc:to
         :content-language:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zXErrdCa6Tr/iUNLljuNLnYOqYVTtO+yEzmXtzl8oAg=;
        b=tvRVfxtZIOps90/ikguS94RVXhcp3ltFo2Vy2kCjkPtomE7/l1qAqJee1an5XC8zxA
         G22mxBlAN9ibXSxSCbrKsFiE4FMVPssUzatxz9uSbbEBcd/hht1dTfLxHakpCprKK/W9
         pdd1Cwcrm16bH1naKMy7j6rF8GBVaWhsPtzrY8rhA5Qv4WPb8PwMgJtwambBcpiLto34
         hv62ryXd2OmFL1Pv98VgzUB2yJG+vdVDWUN4+AmqHQKmVlCNdbp91KgJrbF+t1xXQQ+2
         u9wmC2ZCnwgScq4KjeL//FJIhZZsLipVG94RTVeOoBY+hTNB7TtHanBZzpuMe/HEwyzK
         lgkA==
X-Gm-Message-State: AOJu0YxxjFlkcTV2TylB6PKPUrN+WJ0zjqPo+kdgux3yRr4CxpLvGXv1
	GvTM6PLILZffph2MJfquAXZcqTxucBAbyPQiNVQG+CkfVcwI2U2Y
X-Google-Smtp-Source: AGHT+IFbw8InomjsHPmQ7NoKN1mFYFTZy5xPgK4K6WBP0JxM925vC6kFs5ZDi6dJPHmMSPEYDNDBbw==
X-Received: by 2002:a05:6512:2ea:b0:510:c2a:290e with SMTP id m10-20020a05651202ea00b005100c2a290emr3101188lfq.3.1706864504495;
        Fri, 02 Feb 2024 01:01:44 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCV+rHd1vBTlUhXmMIrCW2ZbJAsHc9otguFV2eS0cIHs6LDMRH5ZblyXmVfHscvNscY29hvkr9BDC3/WPcnjhf/WPB5oadNNsYSN7dk35nQr8hC1mrJd9TIZwHLN+GYsZJCAqVYZ7VKMTpAeLcfWiXtK
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <c64deb80-d01c-4c8d-89a6-5b0d37f25bf6@xen.org>
Date: Fri, 2 Feb 2024 09:01:42 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: Ping: [PATCH] x86/guest: finish conversion to altcall
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <65fb766e-ab5a-49ab-a904-01ee9711d42d@suse.com>
 <578a6845-6046-4d15-88a8-8a34936095a7@suse.com>
Organization: Xen Project
In-Reply-To: <578a6845-6046-4d15-88a8-8a34936095a7@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 02/02/2024 07:08, Jan Beulich wrote:
> On 17.01.2024 10:31, Jan Beulich wrote:
>> While .setup() and .e820_fixup() don't need fiddling with for being run
>> only very early, both .ap_setup() and .resume() want converting too:
>> This way both pre-filled struct hypervisor_ops instances can become
>> __initconst_cf_clobber, thus allowing to eliminate up to 5 more ENDBR
>> (configuration dependent) during the 2nd phase of alternatives patching.
>>
>> While fiddling with section annotations here, also move "ops" itself to
>> .data.ro_after_init.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> May I ask for an ack (or otherwise)?
> 
> Thanks, Jan
> 

Yes, no objections.

Acked-by: Paul Durrant <paul@xen.org>

>> --- a/xen/arch/x86/guest/hyperv/hyperv.c
>> +++ b/xen/arch/x86/guest/hyperv/hyperv.c
>> @@ -207,7 +207,7 @@ static int cf_check flush_tlb(
>>       return hyperv_flush_tlb(mask, va, flags);
>>   }
>>   
>> -static const struct hypervisor_ops __initconstrel ops = {
>> +static const struct hypervisor_ops __initconst_cf_clobber ops = {
>>       .name = "Hyper-V",
>>       .setup = setup,
>>       .ap_setup = ap_setup,
>> --- a/xen/arch/x86/guest/hypervisor.c
>> +++ b/xen/arch/x86/guest/hypervisor.c
>> @@ -13,7 +13,7 @@
>>   #include <asm/cache.h>
>>   #include <asm/guest.h>
>>   
>> -static struct hypervisor_ops __read_mostly ops;
>> +static struct hypervisor_ops __ro_after_init ops;
>>   
>>   const char *__init hypervisor_probe(void)
>>   {
>> @@ -49,7 +49,7 @@ void __init hypervisor_setup(void)
>>   int hypervisor_ap_setup(void)
>>   {
>>       if ( ops.ap_setup )
>> -        return ops.ap_setup();
>> +        return alternative_call(ops.ap_setup);
>>   
>>       return 0;
>>   }
>> @@ -57,7 +57,7 @@ int hypervisor_ap_setup(void)
>>   void hypervisor_resume(void)
>>   {
>>       if ( ops.resume )
>> -        ops.resume();
>> +        alternative_vcall(ops.resume);
>>   }
>>   
>>   void __init hypervisor_e820_fixup(void)
>> --- a/xen/arch/x86/guest/xen/xen.c
>> +++ b/xen/arch/x86/guest/xen/xen.c
>> @@ -318,7 +318,7 @@ static int cf_check flush_tlb(
>>       return xen_hypercall_hvm_op(HVMOP_flush_tlbs, NULL);
>>   }
>>   
>> -static const struct hypervisor_ops __initconstrel ops = {
>> +static const struct hypervisor_ops __initconst_cf_clobber ops = {
>>       .name = "Xen",
>>       .setup = setup,
>>       .ap_setup = ap_setup,
> 



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 09:17:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 09:17:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674886.1050034 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVpfq-0001Un-58; Fri, 02 Feb 2024 09:17:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674886.1050034; Fri, 02 Feb 2024 09:17:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVpfq-0001Ug-15; Fri, 02 Feb 2024 09:17:26 +0000
Received: by outflank-mailman (input) for mailman id 674886;
 Fri, 02 Feb 2024 09:17:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVpfo-0001UW-W5; Fri, 02 Feb 2024 09:17:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVpfo-0004XY-Td; Fri, 02 Feb 2024 09:17:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVpfo-0002cZ-F2; Fri, 02 Feb 2024 09:17:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVpfo-0008JR-EL; Fri, 02 Feb 2024 09:17:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dlBMpoIK8dmHbLfgcpm/c8PyDMpP9eecQUUs27LCf6M=; b=nfew3syhCXefCgat1xF1si1dgU
	C4FCEyxaZV8rz12t/HV4LVSRHvQ5OWKk1bvg22GRnS631iavfeQ4F6o0ljeQEATDbAD92LBPB/HVC
	ToaMabqmicWOhOGtDwlVFh2kYDoZfgnvLfb4wkPEMlgV29NeDVOzPfKFtJqLLKI4NFi8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184553-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.18-testing test] 184553: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.18-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=b1fdd7d0e47e0831ac7a99d0417385fc10d3068c
X-Osstest-Versions-That:
    xen=c7ac596a575a05d6ff1e35c3ff98bc4d143712d2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 02 Feb 2024 09:17:24 +0000

flight 184553 xen-4.18-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184553/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184532
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184532
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184532
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184532
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184532
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184532
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184532
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184532
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184532
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184532
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184532
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184532
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  b1fdd7d0e47e0831ac7a99d0417385fc10d3068c
baseline version:
 xen                  c7ac596a575a05d6ff1e35c3ff98bc4d143712d2

Last test of basis   184532  2024-01-30 14:07:14 Z    2 days
Testing same since   184553  2024-02-01 17:08:47 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Michal Orzel <michal.orzel@amd.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   c7ac596a57..b1fdd7d0e4  b1fdd7d0e47e0831ac7a99d0417385fc10d3068c -> stable-4.18


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 09:19:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 09:19:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674893.1050044 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVpi8-0002LO-LC; Fri, 02 Feb 2024 09:19:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674893.1050044; Fri, 02 Feb 2024 09:19:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVpi8-0002LH-IO; Fri, 02 Feb 2024 09:19:48 +0000
Received: by outflank-mailman (input) for mailman id 674893;
 Fri, 02 Feb 2024 09:19:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rVpi7-0002LB-Dl
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 09:19:47 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVpi2-0004bl-7a; Fri, 02 Feb 2024 09:19:42 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVpi2-00077F-0l; Fri, 02 Feb 2024 09:19:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=k3uQIgLom4V7KFF4qgMw0ynkHzKoY5M6cmVrZ+EaNIE=; b=x6AExTe/d862cvjJp0nMC6YY+H
	7SAWhbGp3XDQr77UHECSIPF6sJfoX5kMY8gHQcydF2iFPBVzHRuqhrHAZjNrbfJOxls5hbeu2emub
	U4sxmtCFpaKZr468xSgbYhPjQUiN+MBDqB+OKeIqhsjlst3huANiPLezC26Ttj4uags4=;
Message-ID: <012b5f83-2f9b-4477-965e-07b8506c0052@xen.org>
Date: Fri, 2 Feb 2024 09:19:40 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Content-Language: en-GB
To: John Ernberg <john.ernberg@actia.se>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Peng Fan <peng.fan@nxp.com>
Cc: Jonas Blixt <jonas.blixt@actia.se>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20240131114952.305805-1-john.ernberg@actia.se>
 <20240131114952.305805-2-john.ernberg@actia.se>
 <a2f726f5-df4c-4d15-90af-7d59c0f1f513@xen.org>
 <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se>
 <167f0c7a-e037-446c-82f8-2584e35a7af1@xen.org>
 <a265ea2d-9b5f-4726-9395-448b1b669839@actia.se>
From: Julien Grall <julien@xen.org>
In-Reply-To: <a265ea2d-9b5f-4726-9395-448b1b669839@actia.se>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 01/02/2024 16:17, John Ernberg wrote:
> On 2/1/24 13:20, Julien Grall wrote:
>>
>>
>> On 31/01/2024 15:32, John Ernberg wrote:
>>> Hi Julien,
>>
>> Hi John,
>>
>>> On 1/31/24 13:22, Julien Grall wrote:
>>>> Hi,
>>>>
>>>> On 31/01/2024 11:50, John Ernberg wrote:
>>>>> When using Linux for dom0 there are a bunch of drivers that need to do
>>>>> SMC
>>>>> SIP calls into the PSCI provider to enable certain hardware bits
>>>>> like the
>>>>> watchdog.
>>>>
>>>> Do you know which protocol this is under the hood. Is this SCMI?
>>>
>>> I think I confused myself here when I wrote the commit log.
>>>
>>> The EL3 code in our case is ATF, and it does not appear to be SCMI, nor
>>> PSCI. The register usage of these SMC SIP calls are as follows:
>>> a0 - service
>>> a1 - function
>>> a2-a7 - args
>>>
>>> In ATF the handler is declared as a runtime service.
>>>
>>> Would the appropriate commmit message here be something along the lines
>>> of below?
>>> """
>>> When using Linux for dom0 there are a bunch of drivers that need to
>>> do   SMC
>>> SIP calls into the firmware to enable certain hardware bits like the
>>> watchdog.
>>> """
>>
>> It reads better thanks.
>>
>> [...]
>>
>>>> But even if we restrict to dom0, have you checked that none of the SMCs
>>>> use buffers?
>>> I haven't found any such instances in the Linux kernel where a buffer is
>>> used. Adding a call filtering like suggested below additions of such
>>> functions can be discovered and adapted for if they would show up later.
>>>>
>>>> Rather than providing a blanket forward, to me it sounds more like you
>>>> want to provide an allowlist of the SMCs. This is more futureproof and
>>>> avoid the risk to expose unsafe SMCs to any domain.
>>>>
>>>> For an example, you can have a look at the EEMI mediator for Xilinx.
>>>
>>> Ack. Do you prefer to see only on SMCCC service level or also on
>>> function level? (a1 register, per description earlier)
>>
>> I am not sure. It will depend on whether it is correct to expose *all*
>> the functions within a service level and they have the same format.
>>
>> If you can't guarantee that, then you will most likely need to allowlist
>> at the function level.
>>
>> Also, do you have a spec in hand that would help to understand which
>> service/function is implemented via those SMCs?
> 
> I don't have the spec unfortunately, but I will add a filter on both
> service and function for V2 and we'll take it from there.

@Peng, do you have any specification you could share? How stable is the 
interface?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 09:38:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 09:38:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674922.1050120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVq0S-0006Zo-Bj; Fri, 02 Feb 2024 09:38:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674922.1050120; Fri, 02 Feb 2024 09:38:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVq0S-0006Zf-7d; Fri, 02 Feb 2024 09:38:44 +0000
Received: by outflank-mailman (input) for mailman id 674922;
 Fri, 02 Feb 2024 09:38:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVq0R-0006GX-3x
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 09:38:43 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d8f2270f-c1ae-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 10:38:41 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 97E9E4EE073D;
 Fri,  2 Feb 2024 10:38:39 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8f2270f-c1ae-11ee-8a43-1f161083a0e0
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	sstabellini@kernel.org,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 3/4] xen/x86: address violations of MISRA C:2012 Rule 13.1
Date: Fri,  2 Feb 2024 10:37:46 +0100
Message-Id: <7e54cfeaffea70f567d248779c092c4005e93a36.1706700228.git.maria.celeste.cesario@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
References: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Rule 13.1: Initializer lists shall not contain persistent side effects

This patch moves expressions with side-effects into new variables before
the initializer lists.

No functional changes.

Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
---
 xen/arch/x86/io_apic.c | 9 ++++++---
 xen/arch/x86/mpparse.c | 3 ++-
 xen/arch/x86/setup.c   | 3 ++-
 3 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
index b48a642465..4a6ab85689 100644
--- a/xen/arch/x86/io_apic.c
+++ b/xen/arch/x86/io_apic.c
@@ -2559,9 +2559,12 @@ integer_param("max_gsi_irqs", max_gsi_irqs);
 
 static __init bool bad_ioapic_register(unsigned int idx)
 {
-    union IO_APIC_reg_00 reg_00 = { .raw = io_apic_read(idx, 0) };
-    union IO_APIC_reg_01 reg_01 = { .raw = io_apic_read(idx, 1) };
-    union IO_APIC_reg_02 reg_02 = { .raw = io_apic_read(idx, 2) };
+    uint32_t reg_00_raw = io_apic_read(idx, 0);
+    uint32_t reg_01_raw = io_apic_read(idx, 1);
+    uint32_t reg_02_raw = io_apic_read(idx, 2);
+    union IO_APIC_reg_00 reg_00 = { .raw = reg_00_raw };
+    union IO_APIC_reg_01 reg_01 = { .raw = reg_01_raw };
+    union IO_APIC_reg_02 reg_02 = { .raw = reg_02_raw };
 
     if ( reg_00.raw == -1 && reg_01.raw == -1 && reg_02.raw == -1 )
     {
diff --git a/xen/arch/x86/mpparse.c b/xen/arch/x86/mpparse.c
index d8ccab2449..81a819403b 100644
--- a/xen/arch/x86/mpparse.c
+++ b/xen/arch/x86/mpparse.c
@@ -798,11 +798,12 @@ void __init mp_register_lapic_address (
 
 int mp_register_lapic(u32 id, bool enabled, bool hotplug)
 {
+	u32 apic = apic_read(APIC_LVR);
 	struct mpc_config_processor processor = {
 		.mpc_type = MP_PROCESSOR,
 		/* Note: We don't fill in fields not consumed anywhere. */
 		.mpc_apicid = id,
-		.mpc_apicver = GET_APIC_VERSION(apic_read(APIC_LVR)),
+		.mpc_apicver = GET_APIC_VERSION(apic),
 		.mpc_cpuflag = (enabled ? CPU_ENABLED : 0) |
 			       (id == boot_cpu_physical_apicid ?
 				CPU_BOOTPROCESSOR : 0),
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index ee682dd136..886031d86a 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -885,13 +885,14 @@ static struct domain *__init create_dom0(const module_t *image,
 {
     static char __initdata cmdline[MAX_GUEST_CMDLINE];
 
+    unsigned int max_vcpus = dom0_max_vcpus();
     struct xen_domctl_createdomain dom0_cfg = {
         .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
         .max_evtchn_port = -1,
         .max_grant_frames = -1,
         .max_maptrack_frames = -1,
         .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version),
-        .max_vcpus = dom0_max_vcpus(),
+        .max_vcpus = max_vcpus,
         .arch = {
             .misc_flags = opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0,
         },
-- 
2.40.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 09:38:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 09:38:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674920.1050094 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVq0P-0005vv-Ra; Fri, 02 Feb 2024 09:38:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674920.1050094; Fri, 02 Feb 2024 09:38:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVq0P-0005vk-Nm; Fri, 02 Feb 2024 09:38:41 +0000
Received: by outflank-mailman (input) for mailman id 674920;
 Fri, 02 Feb 2024 09:38:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVq0O-0005sE-Ax
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 09:38:40 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d7913475-c1ae-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 10:38:38 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id A36D54EE073A;
 Fri,  2 Feb 2024 10:38:36 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7913475-c1ae-11ee-98f5-efadbce2ee36
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	sstabellini@kernel.org,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Dario Faggioli <dfaggioli@suse.com>,
	Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
Subject: [PATCH v3 1/4] xen: add SAF deviation for debugging and logging effects
Date: Fri,  2 Feb 2024 10:37:44 +0100
Message-Id: <f3ab1e5f79b1c6d4e2cfaa34644512338766736b.1706700228.git.maria.celeste.cesario@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
References: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Rule 13.1: Initializer lists shall not contain persistent side effects

Effects caused by debug/logging macros and functions (like ASSERT, __bad_atomic_size,
LOG, etc ...) that crash execution or produce logs are not dangerous in initializer
lists. The evaluation order in abnormal conditions is not relevant. Evaluation order
of logging effects is always safe.

Function hvm_get_guest_tsc_fixed (indirectly) performs different side effects.
For example it calls hvm_get_guest_time_fixed that contains an ASSERT and calls
to spin_lock and spin_unlock.

These side effects are not dangerous: they can be executed regardless of the
initializer list evaluation order

This patch deviates violations using SAF commits caused by debug/logging macros and
functions.

Asm volatile statements in initializer lists that do not perform any persistent side
effect are safe: this patch deviates violations caused by uses of the current macro
(that contains an asm volatile) in initializer lists.

No functional changes.

Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>

---
Changes in v3:
- change commit prefix from xen/arm to xen
- specify where saf-3-safe comments are applied in guestcopy.c
- reword SAF text
Changes in v2:
New patch based on the discussion for "xen/arm: address violations of MISRA C:2012 Rule 13.1".
---
 docs/misra/safe.json     | 16 ++++++++++++++++
 xen/arch/arm/device.c    |  1 +
 xen/arch/arm/guestcopy.c | 16 ++++++++++++----
 xen/arch/x86/hvm/hvm.c   |  1 +
 xen/common/sched/core.c  |  3 +++
 5 files changed, 33 insertions(+), 4 deletions(-)

diff --git a/docs/misra/safe.json b/docs/misra/safe.json
index 952324f85c..5539e8dfda 100644
--- a/docs/misra/safe.json
+++ b/docs/misra/safe.json
@@ -28,6 +28,22 @@
         },
         {
             "id": "SAF-3-safe",
+            "analyser": {
+                "eclair": "MC3R1.R13.1"
+            },
+            "name": "MC3R1.R13.1: effects for debugging and logging",
+            "text": "Effects for debugging and loggings reasons that crash execution or produce logs are allowed in initializer lists. The evaluation order in abnormal conditions is not relevant."
+        },
+        {
+            "id": "SAF-4-safe",
+            "analyser": {
+                "eclair": "MC3R1.R13.1"
+            },
+            "name": "MC3R1.R13.1: volatile asm statements that do not perform any persistent side effect",
+            "text": "Volatile asm statement in an initializer list that does not perform persistent side effects is safe."
+        },
+        {
+            "id": "SAF-5-safe",
             "analyser": {},
             "name": "Sentinel",
             "text": "Next ID to be used"
diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
index 1f631d3274..fa331f164d 100644
--- a/xen/arch/arm/device.c
+++ b/xen/arch/arm/device.c
@@ -331,6 +331,7 @@ int handle_device(struct domain *d, struct dt_device_node *dev, p2m_type_t p2mt,
         .p2mt = p2mt,
         .skip_mapping = !own_device ||
                         (is_pci_passthrough_enabled() &&
+                        /* SAF-3-safe effects for debugging/logging reasons are safe */
                         (device_get_class(dev) == DEVICE_PCI_HOSTBRIDGE)),
         .iomem_ranges = iomem_ranges,
         .irq_ranges = irq_ranges
diff --git a/xen/arch/arm/guestcopy.c b/xen/arch/arm/guestcopy.c
index 6716b03561..b75538252a 100644
--- a/xen/arch/arm/guestcopy.c
+++ b/xen/arch/arm/guestcopy.c
@@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf, uint64_t addr, unsigned int len,
 unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
 {
     return copy_guest((void *)from, (vaddr_t)to, len,
-                      GVA_INFO(current), COPY_to_guest | COPY_linear);
+                      /* SAF-4-safe No persistent side effects */
+                      GVA_INFO(current),
+                      COPY_to_guest | COPY_linear);
 }
 
 unsigned long raw_copy_to_guest_flush_dcache(void *to, const void *from,
                                              unsigned int len)
 {
-    return copy_guest((void *)from, (vaddr_t)to, len, GVA_INFO(current),
+    return copy_guest((void *)from, (vaddr_t)to, len,
+                      /* SAF-4-safe No persistent side effects */
+                      GVA_INFO(current),
                       COPY_to_guest | COPY_flush_dcache | COPY_linear);
 }
 
 unsigned long raw_clear_guest(void *to, unsigned int len)
 {
-    return copy_guest(NULL, (vaddr_t)to, len, GVA_INFO(current),
+    return copy_guest(NULL, (vaddr_t)to, len,
+                      /* SAF-4-safe No persistent side effects */
+                      GVA_INFO(current),
                       COPY_to_guest | COPY_linear);
 }
 
 unsigned long raw_copy_from_guest(void *to, const void __user *from,
                                   unsigned int len)
 {
-    return copy_guest(to, (vaddr_t)from, len, GVA_INFO(current),
+    return copy_guest(to, (vaddr_t)from, len,
+                      /* SAF-4-safe No persistent side effects */
+                      GVA_INFO(current),
                       COPY_from_guest | COPY_linear);
 }
 
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index e3bd9157d7..38af437634 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -800,6 +800,7 @@ static int cf_check hvm_save_cpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
 {
     struct segment_register seg;
     struct hvm_hw_cpu ctxt = {
+        /* SAF-3-safe effects for debugging/logging reasons are safe */
         .tsc = hvm_get_guest_tsc_fixed(v, v->domain->arch.hvm.sync_tsc),
         .msr_tsc_aux = v->arch.msrs->tsc_aux,
         .rax = v->arch.user_regs.rax,
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index d177c675c8..9e973fcf31 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -1521,6 +1521,7 @@ long vcpu_yield(void)
 
     SCHED_STAT_CRANK(vcpu_yield);
 
+    /* SAF-4-safe No persistent side effects */
     TRACE_2D(TRC_SCHED_YIELD, current->domain->domain_id, current->vcpu_id);
     raise_softirq(SCHEDULE_SOFTIRQ);
     return 0;
@@ -1899,6 +1900,7 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         if ( copy_from_guest(&sched_shutdown, arg, 1) )
             break;
 
+        /* SAF-4-safe No persistent side effects */
         TRACE_3D(TRC_SCHED_SHUTDOWN,
                  current->domain->domain_id, current->vcpu_id,
                  sched_shutdown.reason);
@@ -1916,6 +1918,7 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         if ( copy_from_guest(&sched_shutdown, arg, 1) )
             break;
 
+        /* SAF-4-safe No persistent side effects */
         TRACE_3D(TRC_SCHED_SHUTDOWN_CODE,
                  d->domain_id, current->vcpu_id, sched_shutdown.reason);
 
-- 
2.40.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 09:38:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 09:38:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674921.1050104 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVq0Q-00063N-9W; Fri, 02 Feb 2024 09:38:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674921.1050104; Fri, 02 Feb 2024 09:38:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVq0Q-000631-0Z; Fri, 02 Feb 2024 09:38:42 +0000
Received: by outflank-mailman (input) for mailman id 674921;
 Fri, 02 Feb 2024 09:38:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVq0P-0005sE-4a
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 09:38:41 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d86c3a85-c1ae-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 10:38:39 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 2BA274EE073C;
 Fri,  2 Feb 2024 10:38:38 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d86c3a85-c1ae-11ee-98f5-efadbce2ee36
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	sstabellini@kernel.org,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>,
	Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
Subject: [PATCH v3 2/4] xen/ns16550: address violations of MISRA C:2012 Rule 13.1
Date: Fri,  2 Feb 2024 10:37:45 +0100
Message-Id: <d1fb1baae24cf0b0b2826ed531abab12950f93cd.1706700228.git.maria.celeste.cesario@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
References: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Rule 13.1: Initializer lists shall not contain persistent side effects

The assignment operation in:

.irq = rc = uart->irq,

is a persistent side effect in a struct initializer list.

This patch assigns rc separately outside the structure.

No functional change.

Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>

---
Changes in v3:
- add assignment of rc;
- use rc as controlling expression in the following if-statement;
- change prefix from xen to xen/ns16550.
Changes in v2:
- avoid assignment of rc;
- drop changes in vcpu_yield(void).
---
 xen/drivers/char/ns16550.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c
index afe3d514b9..97bf098534 100644
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -441,10 +441,12 @@ static void __init cf_check ns16550_init_postirq(struct serial_port *port)
             struct msi_info msi = {
                 .sbdf = PCI_SBDF(0, uart->ps_bdf[0], uart->ps_bdf[1],
                                  uart->ps_bdf[2]),
-                .irq = rc = uart->irq,
+                .irq = uart->irq,
                 .entry_nr = 1
             };
 
+            rc = uart->irq;
+
             if ( rc > 0 )
             {
                 struct msi_desc *msi_desc = NULL;
-- 
2.40.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 09:38:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 09:38:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674919.1050089 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVq0P-0005sc-J8; Fri, 02 Feb 2024 09:38:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674919.1050089; Fri, 02 Feb 2024 09:38:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVq0P-0005sV-FR; Fri, 02 Feb 2024 09:38:41 +0000
Received: by outflank-mailman (input) for mailman id 674919;
 Fri, 02 Feb 2024 09:38:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVq0O-0005sE-3i
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 09:38:40 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d6d50c72-c1ae-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 10:38:37 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 8C67D4EE0737;
 Fri,  2 Feb 2024 10:38:35 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d6d50c72-c1ae-11ee-98f5-efadbce2ee36
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	sstabellini@kernel.org,
	Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Dario Faggioli <dfaggioli@suse.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [PATCH v3 0/4] address violation of MISRA C:2012 Rule 13.1
Date: Fri,  2 Feb 2024 10:37:43 +0100
Message-Id: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>

The Xen sources contain violations of MISRA C:2012 Rule 13.1 whose headline states:
"Initializer lists shall not contain persistent side effects".

The file properties.json containing function and macro properties is introduced, as
stated in v2 discussion. Some functions and macros are found to have properties that
can be exploited by static analyzers. For this reason, the file docs/properties.json
contains all the needed properties. A description of the json file is documented in
docs/properties.rst.

Some persistent effects have been moved outside initializer lists to address violations
of Rule 13.1.

Link to the discussion: https://lore.kernel.org/all/cover.1700844359.git.simone.ballarin@bugseng.com/T/#u


Changes in v3:
- change prefix from xen to xen/ns16550
- add assignment of rc in xen/ns16550
- use rc as controlling expression in the following if-statement
- change commit prefix from xen/arm to xen
- specify where saf-3-safe comments are applied in guestcopy.c
- reword saf comments text

Maria Celeste Cesario (1):
  eclair: add and manage properties

Simone Ballarin (3):
  xen: add SAF deviation for debugging and logging effects
  xen/ns16550: address violations of MISRA C:2012 Rule 13.1
  xen/x86: address violations of MISRA C:2012 Rule 13.1

 .../eclair_analysis/ECLAIR/analysis.ecl       |   1 +
 automation/eclair_analysis/prepare.sh         |   2 +
 docs/misra/safe.json                          |  16 +
 docs/properties.json                          | 841 ++++++++++++++++++
 docs/properties.rst                           |  58 ++
 xen/arch/arm/device.c                         |   1 +
 xen/arch/arm/guestcopy.c                      |  16 +-
 xen/arch/x86/hvm/hvm.c                        |   1 +
 xen/arch/x86/io_apic.c                        |   9 +-
 xen/arch/x86/mpparse.c                        |   3 +-
 xen/arch/x86/setup.c                          |   3 +-
 xen/common/sched/core.c                       |   3 +
 xen/drivers/char/ns16550.c                    |   4 +-
 13 files changed, 948 insertions(+), 10 deletions(-)
 create mode 100644 docs/properties.json
 create mode 100644 docs/properties.rst

-- 
2.40.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 09:38:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 09:38:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674923.1050130 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVq0T-0006q2-If; Fri, 02 Feb 2024 09:38:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674923.1050130; Fri, 02 Feb 2024 09:38:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVq0T-0006pt-EO; Fri, 02 Feb 2024 09:38:45 +0000
Received: by outflank-mailman (input) for mailman id 674923;
 Fri, 02 Feb 2024 09:38:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVq0R-0006GX-Hb
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 09:38:43 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d962f27c-c1ae-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 10:38:41 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 5D9C64EE073E;
 Fri,  2 Feb 2024 10:38:40 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d962f27c-c1ae-11ee-8a43-1f161083a0e0
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	sstabellini@kernel.org,
	Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 4/4] eclair: add and manage properties
Date: Fri,  2 Feb 2024 10:37:47 +0100
Message-Id: <bc9bdf2b16a6131f476e6e1f491cf04629f431cf.1706700228.git.maria.celeste.cesario@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
References: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>

Add JSON file containing properties.
Add rst file containing explanation on how to update properties.json.
Add instruction to eclair_analysis/prepare.sh to parse the JSON file.

Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
---
 .../eclair_analysis/ECLAIR/analysis.ecl       |   1 +
 automation/eclair_analysis/prepare.sh         |   2 +
 docs/properties.json                          | 841 ++++++++++++++++++
 docs/properties.rst                           |  58 ++
 4 files changed, 902 insertions(+)
 create mode 100644 docs/properties.json
 create mode 100644 docs/properties.rst

diff --git a/automation/eclair_analysis/ECLAIR/analysis.ecl b/automation/eclair_analysis/ECLAIR/analysis.ecl
index a604582da3..684c5b0b39 100644
--- a/automation/eclair_analysis/ECLAIR/analysis.ecl
+++ b/automation/eclair_analysis/ECLAIR/analysis.ecl
@@ -30,6 +30,7 @@ if(not(scheduled_analysis),
 -eval_file=deviations.ecl
 -eval_file=call_properties.ecl
 -eval_file=tagging.ecl
+-eval_file=properties.ecl
 -eval_file=concat(set,".ecl")
 
 -doc="Hide reports in external code."
diff --git a/automation/eclair_analysis/prepare.sh b/automation/eclair_analysis/prepare.sh
index fe9d16e48e..47b2a2f32a 100755
--- a/automation/eclair_analysis/prepare.sh
+++ b/automation/eclair_analysis/prepare.sh
@@ -43,4 +43,6 @@ fi
     make -f "${script_dir}/Makefile.prepare" prepare
     # Translate the /* SAF-n-safe */ comments into ECLAIR CBTs
     scripts/xen-analysis.py --run-eclair --no-build --no-clean
+    # Translate function-properties.json into ECLAIR properties
+    python3 ${script_dir}/propertyparser.py
 )
diff --git a/docs/properties.json b/docs/properties.json
new file mode 100644
index 0000000000..74058297b5
--- /dev/null
+++ b/docs/properties.json
@@ -0,0 +1,841 @@
+{
+   "version": "1.0",
+   "content": [
+      {
+         "type": "function",
+         "value": "^printk.*$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^debugtrace_printk.*$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^panic.*$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^domain_crash$",
+         "properties":{
+            "pointee_write": "2..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^(g?d|mm_)?printk$",
+         "properties":{
+            "pointee_write": "2..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^guest_bug_on_failed$",
+         "properties":{
+            "pointee_write": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^spin_lock_init_prof$",
+         "properties":{
+            "pointee_write": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^sched_test_func$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^dev_(info|warn)$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^PAGING_DEBUG$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^ACPI_(WARNING|ERROR|INFO)$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^fdt_get_property_by_offset_.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^read_atomic_size.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^device_tree_get_reg.*$",
+         "properties":{
+            "pointee_write": "4..=always",
+            "pointee_read": "4..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^dt_get_range.*$",
+         "properties":{
+            "pointee_write": "3..=always",
+            "pointee_read": "3..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^parse_static_mem_prop.*$",
+         "properties":{
+            "pointee_write": "2..=always",
+            "pointee_read": "2..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_ttbr_and_gran_64bit.*$",
+         "properties":{
+            "pointee_write": "1..2=always",
+            "pointee_read": "1..2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^hvm_emulate_init_once.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^__vmread.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^hvm_pci_decode_addr.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^vpci_mmcfg_decode_addr.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^x86emul_decode.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^unmap_grant_ref.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^arm_smmu_cmdq_build_cmd.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^pci_size_mem_bar.*$",
+         "properties":{
+            "pointee_write": "4=always",
+            "pointee_read": "4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^_hvm_read_entry.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^hvm_map_guest_frame_rw.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^guest_cpuid.*$",
+         "properties":{
+            "pointee_write": "4=always",
+            "pointee_read": "4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^epte_get_entry_emt.*$",
+         "properties":{
+            "pointee_write": "5=always",
+            "pointee_read": "5=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^mcheck_mca_logout.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^set_field_in_reg_u32.*$",
+         "properties":{
+            "pointee_write": "5=always",
+            "pointee_read": "5=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^alloc_affinity_masks.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^xasprintf.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^find_non_smt.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^call_rcu.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^getdomaininfo.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^MAPPING_(INSERT|SEARCH)\\(.*$",
+         "properties":{
+            "pointee_write": "2..=always",
+            "pointee_read": "2..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^FormatDec.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^FormatHex.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^p2m_get_ioreq_server.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^elf_memset_unchecked.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^set_iommu_pte_present.*$",
+         "properties":{
+            "pointee_write": "7=always",
+            "pointee_read": "7=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^clear_iommu_pte_present.*$",
+         "properties":{
+            "pointee_write": "4=always",
+            "pointee_read": "4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^vcpu_runstate_get.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^va_start.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^sgi_target_init.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_hw_residencies.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^x86_cpu_policy_to_featureset.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^simple_strtou?ll?\\(.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^msi_compose_msg.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^print_tainted.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_hvm_registers.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^x86_insn_modrm.*$",
+         "properties":{
+            "pointee_write": "2..3=always",
+            "pointee_read": "2..3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^cpuid_count_leaf.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^rcu_lock_remote_domain_by_id.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^cpuid_count.*$",
+         "properties":{
+            "pointee_write": "3..=always",
+            "pointee_read": "3..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^.*efi_boot_mem_unused.*$",
+         "properties":{
+            "pointee_write": "1..=always",
+            "pointee_read": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^collect_time_info.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^setup_xstate_comp.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^map_domain_gfn.*$",
+         "properties":{
+            "pointee_read": "5=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^fdt_getprop.*$",
+         "properties":{
+            "pointee_read": "4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^fdt_get_name.*$",
+         "properties":{
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^fdt_get_property.*$",
+         "properties":{
+            "pointee_read": "4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^pci_get_host_bridge_segment.*$",
+         "properties":{
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^dt_get_property.*$",
+         "properties":{
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^dt_property_read_u32.*$",
+         "properties":{
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^dt_device_get_paddr.*$",
+         "properties":{
+            "pointee_read": "3..4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_evtchn_dt_property.*$",
+         "properties":{
+            "pointee_write": "2..3=maybe",
+            "pointee_read": "2..3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^setup_chosen_node.*$",
+         "properties":{
+            "pointee_write": "2..3=maybe",
+            "pointee_read": "2..3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^queue_remove_raw.*$",
+         "properties":{
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^memset$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^va_start$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^memcmp$",
+         "properties":{
+            "pointee_write": "1..2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^memcpy$",
+         "properties":{
+            "pointee_write": "1=always&&2..=never",
+            "pointee_read": "1=never&&2..=always",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_cpu_info.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^pdx_to_pfn.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^is_pci_passthrough_enabled.*$",
+         "properties":{
+            "attribute": "const"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_cycles.*$",
+         "properties":{
+            "attribute": "noeffect"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^msi_gflags.*$",
+         "properties":{
+            "attribute": "const"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^hvm_save_size.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^cpu_has.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^boot_cpu_has.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_cpu_info.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^put_pte_flags.*$",
+         "properties":{
+            "attribute": "const"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^is_pv_cpu.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "description": "Property inferred as a consequence of the semantics of device_tree_get_reg",
+         "type": "function",
+         "value": "^acquire_static_memory_bank.*$",
+         "properties":{
+            "pointee_write": "4..=always",
+            "pointee_read": "4..=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "Property inferred as a consequence of the semantics of dt_set_cell",
+         "type": "function",
+         "value": "^set_interrupt.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description":"Property inferred as a consequence of the semantics of __p2m_get_mem_access",
+         "type": "function",
+         "value": "^p2m_get_mem_access.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "This function has alternative definitions with props {write=always,read=never} and {write=never,read=never}",
+         "type": "function",
+         "value": "^alloc_cpumask_var.*$",
+         "properties":{
+            "pointee_write": "1=maybe",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "Property inferred as a consequence of the semantics of alloc_cpumask_var",
+         "type": "function",
+         "value": "^xenctl_bitmap_to_cpumask.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "The call to bitmap_and causes the pointee of dstp to be always written",
+         "type": "function",
+         "value": "^cpumask_(and|andnot|clear|copy|complement).*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "The call to bitmap_and causes the pointee of dstp to be always written",
+         "type": "function",
+         "value": "^bitmap_(andnot|complement|fill).*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "The .*copy_(to|from).* helpers all have a memcpy-like expectation that the destination is a copy of the source. Furthermore, their uses do initialize the involved variables as needed by futher uses in the caller.",
+         "type": "macro",
+         "value": "^(__)?(raw_)?copy_from_(paddr|guest|compat)(_offset)?$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description":"The .*copy_(to|from).* helpers all have a memcpy-like expectation that the destination is a copy of the source. Furthermore, their uses do initialize the involved variables as needed by futher uses in the caller.",
+         "type": "macro",
+         "value": "^(__)?copy_to_(guest|compat)(_offset)?$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "Functions generated by build_atomic_read cannot be considered pure since the input pointer is volatile, but they do not produce any persistent side effect.",
+         "type": "function",
+         "value": "^read_u(8|16|32|64|int)_atomic.*$",
+         "properties":{
+            "attribute": "noeffect"
+         }
+      },
+      {
+         "description": "Functions generated by TYPE_SAFE are const.",
+         "type": "function",
+         "value": "^(mfn|gfn|pfn)_x\\(.*$",
+         "properties":{
+            "attribute": "const"
+         }
+      },
+      {
+         "description": "Functions generated by TYPE_SAFE are const.",
+         "type": "function",
+         "value": "^_(mfn|gfn|pfn)\\(.*$",
+         "properties":{
+            "attribute": "const"
+         }
+      }
+   ]
+}
diff --git a/docs/properties.rst b/docs/properties.rst
new file mode 100644
index 0000000000..790f1ede1e
--- /dev/null
+++ b/docs/properties.rst
@@ -0,0 +1,58 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Properties list for Xen
+=======================
+
+Some functions and macros are found to have properties relevant to 
+the Xen codebase. For this reason, the file docs/properties.json 
+contains all the needed properties.
+
+Here is an example of the properties.json file::
+
+  {
+     "version": "1.0",
+     "content": [
+        {
+           "description": ""
+           "type": "function",       // required
+           "value:": "^printk*.$",   // required
+           "properties":{
+              "pointee_write": "1..2=never",
+              "pointee_read": "",
+              "taken": ""
+              "attribute": ""
+           }
+        }
+     ]
+  }
+
+Here is an explanation of the fields inside an object of the "content" array:
+
+ - description: a brief description of why the properties apply
+ - type: this is the kind of the element called: it may be either ``macro`` or ``function``
+ - value: must be a regex, starting with ^ and ending with $ and matching function fully 
+   qualified name or macro name.
+ - properties: a list of properties applied to said function.
+   Possible values are:
+
+    - pointee_write: indicate the write use for call arguments that correspond to
+      parameters whose pointee types are non-const
+    - pointee_read: indicate the read use for call arguments that correspond to
+      parameters whose pointee types are non-const
+    - taken: indicates that the specified address arguments may be stored in objects
+      that persist after the function has ceased to exist (excluding the returned value);
+      address arguments not listed are never taken
+    - attribute: attributes a function may have. Possible values are pure, const and noeffect.
+
+   pointee_read and pointee_write use a specific kind of argument, structured as pointee_arg=rw:
+
+    - pointee_arg: argument index for callee. Index 0 refers to the return value,
+      the indices of the arguments start from 1. It can be either a single value or a range.
+    - rw: a value that's either always, maybe or never
+    
+       - always: for pointee_read: argument pointee is expected to be fully initialized at function enter,
+         for pointee_write: argument pointee is fully initialized at function exit
+       - maybe: for pointee_read: argument pointee may be expected to be initialized at function enter,
+         for pointee_write: argument pointee may be written by function body
+       - never: for pointee_read: argument pointee is not expected to be read in the functions's body,
+         for pointee_write: argument pointee is never written by function body
-- 
2.40.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 10:50:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 10:50:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674988.1050160 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVr7X-0001a8-7C; Fri, 02 Feb 2024 10:50:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674988.1050160; Fri, 02 Feb 2024 10:50:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVr7X-0001a1-3z; Fri, 02 Feb 2024 10:50:07 +0000
Received: by outflank-mailman (input) for mailman id 674988;
 Fri, 02 Feb 2024 10:50:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Cnb7=JL=gmail.com=olekstysh@srs-se1.protection.inumbo.net>)
 id 1rVr7V-0001HW-Ck
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 10:50:05 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d2229aa4-c1b8-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 11:50:03 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-50eac018059so2430102e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 02 Feb 2024 02:50:03 -0800 (PST)
Received: from EPUAKYIW03DD.. ([91.123.150.198])
 by smtp.gmail.com with ESMTPSA id
 k14-20020ac2456e000000b005110e924453sm270644lfm.286.2024.02.02.02.50.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 02 Feb 2024 02:50:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2229aa4-c1b8-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706871003; x=1707475803; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=TfsdWJKAZazVraSPY8diG0NYRmfnCFQmfv0c0xO59nM=;
        b=BZTl2YDiifZjOt8MEPRpy5lufKt3UWc1gKAVkXkLmGDsxDgtr8EOu+cG7yifGqOYat
         PezueYkSzwBHkMhwaHpLHp7Bou/CNPlFgeRryv0P8L3BMpdLgAwhydrYC+WtjikgZp7Y
         3fdv6gCGwfaovlu3nBKcxUYao3uBKogatlBIJ5bwIs0AaIBruAiXwhW7tUwyzJBfEh/B
         Tzfi+T9LdHwCh/uI6ZUmq8nyyL486QNvf+KgcrC1YPRKF8I/6PeiOaB3vGkezufAkhIG
         dYpLgbjZXM38WcF6Y2hyOaJ/TSI2/aAc0aK9A9CFkiiRRax39bB00jhjcKM39ECjH63w
         5kcg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706871003; x=1707475803;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=TfsdWJKAZazVraSPY8diG0NYRmfnCFQmfv0c0xO59nM=;
        b=KCDuUvK7gplF+MFWQlv8ZqOMf7UA6mfIogb58KN6kzQfi+xdoQfBWQT+h5p+wuLTdT
         insews0J/Y4gG/H4COAKayztrdkhBxogImo8bwPb77sTWjppxDbcf1wQlW/cGJ47EETi
         dReY72zu6OU650K5aKf8KWfW8K3F6yfdkcPqEw2GZlFfXCY4hPUET+ygGZH6MJcFJt50
         PfqXqGYGH+fFA4XSxnetOJ+WDhQ2jTplNNhmInByOZ8iPL1DxSY1q/tuOYBXP6s7Eu6i
         tshsZ5Wr/sRa9CR3YGh9fDkV6RniVmGps47BglnpkdlFkqWfxj82Wu7GHSpuXfs6oNYu
         r2Xg==
X-Gm-Message-State: AOJu0YzO5uzcm3fvo/aDlrpwXUX6NPPQfHeMaQ94pmMcrp0GMGpvkOQa
	rV3YhBYvPORYpHeifvJfWvqWseEcutE9VgRX7DmPiMt23Eyv4hZcGG4hXQOV
X-Google-Smtp-Source: AGHT+IEJsdTpLw5yxxcHCog1lfDFwhuKJVr+GAko6THzmfwj3QKZeiNYAT69tLMVBJrWwAYYNt+m7A==
X-Received: by 2002:ac2:5e62:0:b0:50e:fd98:8ad0 with SMTP id a2-20020ac25e62000000b0050efd988ad0mr951236lfr.63.1706871002711;
        Fri, 02 Feb 2024 02:50:02 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUFb7gY5eK6aA/parcPv1ltuJIMIwI11j4EuHhKmIP3oDlV5gqwO3r1F0mpLzlxy+L7vc8qt+lrIx7X/kQCrbZj3+7eJwlQp3qoPSREeBZIsFJRQxKx2D0SJL6RNzxhc37jhZAl+bieaUVpe8cu7s0ZpTmKN0Pp4keCqin1qKypc1LyxG2h9YxvKQAtHtMnn36+DICRXnoKbcXP049mB5RkJlb3toUb9Ay/yTWatlY=
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Juergen Gross <jgross@suse.com>,
	Viresh Kumar <viresh.kumar@linaro.org>
Subject: [PATCH] libxl: Add "grant_usage" parameter for virtio disk devices
Date: Fri,  2 Feb 2024 12:49:03 +0200
Message-Id: <20240202104903.1112772-1-olekstysh@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Allow administrators to control whether Xen grant mappings for
the virtio disk devices should be used. By default (when new
parameter is not specified), the existing behavior is retained
(we enable grants if backend-domid != 0).

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
In addition to "libxl: arm: Add grant_usage parameter for virtio devices"
https://github.com/xen-project/xen/commit/c14254065ff4826e34f714e1790eab5217368c38
---
 docs/man/xl-disk-configuration.5.pod.in | 25 +++++++++++++++++++++++++
 tools/golang/xenlight/helpers.gen.go    |  6 ++++++
 tools/golang/xenlight/types.gen.go      |  1 +
 tools/include/libxl.h                   |  7 +++++++
 tools/libs/light/libxl_arm.c            |  4 ++--
 tools/libs/light/libxl_disk.c           | 13 +++++++++++++
 tools/libs/light/libxl_types.idl        |  1 +
 tools/libs/util/libxlu_disk_l.l         |  3 +++
 8 files changed, 58 insertions(+), 2 deletions(-)

diff --git a/docs/man/xl-disk-configuration.5.pod.in b/docs/man/xl-disk-configuration.5.pod.in
index bc945cc517..3c035456d5 100644
--- a/docs/man/xl-disk-configuration.5.pod.in
+++ b/docs/man/xl-disk-configuration.5.pod.in
@@ -404,6 +404,31 @@ Virtio frontend driver (virtio-blk) to be used. Please note, the virtual
 device (vdev) is not passed to the guest in that case, but it still must be
 specified for the internal purposes.
 
+=item B<grant_usage=BOOLEAN>
+
+=over 4
+
+=item Description
+
+Specifies the usage of Xen grants for accessing guest memory. Only applicable
+to specification "virtio".
+
+=item Supported values
+
+If this option is B<true>, the Xen grants are always enabled.
+If this option is B<false>, the Xen grants are always disabled.
+
+=item Mandatory
+
+No
+
+=item Default value
+
+If this option is missing, then the default grant setting will be used,
+i.e. enable grants if backend-domid != 0.
+
+=back
+
 =back
 
 =head1 COLO Parameters
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 35e209ff1b..768ab0f566 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1879,6 +1879,9 @@ x.ActiveDisk = C.GoString(xc.active_disk)
 x.HiddenDisk = C.GoString(xc.hidden_disk)
 if err := x.Trusted.fromC(&xc.trusted);err != nil {
 return fmt.Errorf("converting field Trusted: %v", err)
+}
+if err := x.GrantUsage.fromC(&xc.grant_usage);err != nil {
+return fmt.Errorf("converting field GrantUsage: %v", err)
 }
 
  return nil}
@@ -1927,6 +1930,9 @@ if x.HiddenDisk != "" {
 xc.hidden_disk = C.CString(x.HiddenDisk)}
 if err := x.Trusted.toC(&xc.trusted); err != nil {
 return fmt.Errorf("converting field Trusted: %v", err)
+}
+if err := x.GrantUsage.toC(&xc.grant_usage); err != nil {
+return fmt.Errorf("converting field GrantUsage: %v", err)
 }
 
  return nil
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 7907aa8999..0b712d2aa4 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -740,6 +740,7 @@ ColoExport string
 ActiveDisk string
 HiddenDisk string
 Trusted Defbool
+GrantUsage Defbool
 }
 
 type DeviceNic struct {
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index f1652b1664..2b69e08466 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -578,6 +578,13 @@
  */
 #define LIBXL_HAVE_DEVICE_DISK_SPECIFICATION 1
 
+/*
+ * LIBXL_HAVE_DISK_GRANT_USAGE indicates that the libxl_device_disk
+ * has 'grant_usage' field to specify the usage of Xen grants for
+ * the specification 'virtio'.
+ */
+#define LIBXL_HAVE_DISK_GRANT_USAGE 1
+
 /*
  * LIBXL_HAVE_CONSOLE_ADD_XENSTORE indicates presence of the function
  * libxl_console_add_xenstore() in libxl.
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 1539191774..1cb89fa584 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -1372,12 +1372,12 @@ next_resize:
             libxl_device_disk *disk = &d_config->disks[i];
 
             if (disk->specification == LIBXL_DISK_SPECIFICATION_VIRTIO) {
-                if (disk->backend_domid != LIBXL_TOOLSTACK_DOMID)
+                if (libxl_defbool_val(disk->grant_usage))
                     iommu_needed = true;
 
                 FDT( make_virtio_mmio_node(gc, fdt, disk->base, disk->irq,
                                            disk->backend_domid,
-                                           disk->backend_domid != LIBXL_TOOLSTACK_DOMID) );
+                                           libxl_defbool_val(disk->grant_usage)) );
             }
         }
 
diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
index ea3623dd6f..f39f427091 100644
--- a/tools/libs/light/libxl_disk.c
+++ b/tools/libs/light/libxl_disk.c
@@ -181,6 +181,9 @@ static int libxl__device_disk_setdefault(libxl__gc *gc, uint32_t domid,
             return ERROR_INVAL;
         }
         disk->transport = LIBXL_DISK_TRANSPORT_MMIO;
+
+        libxl_defbool_setdefault(&disk->grant_usage,
+                                 disk->backend_domid != LIBXL_TOOLSTACK_DOMID);
     }
 
     if (hotplug && disk->specification == LIBXL_DISK_SPECIFICATION_VIRTIO) {
@@ -429,6 +432,8 @@ static void device_disk_add(libxl__egc *egc, uint32_t domid,
             flexarray_append(back, libxl__device_disk_string_of_transport(disk->transport));
             flexarray_append_pair(back, "base", GCSPRINTF("%"PRIu64, disk->base));
             flexarray_append_pair(back, "irq", GCSPRINTF("%u", disk->irq));
+            flexarray_append_pair(back, "grant_usage",
+                                  libxl_defbool_val(disk->grant_usage) ? "1" : "0");
         }
 
         flexarray_append(front, "backend-id");
@@ -623,6 +628,14 @@ static int libxl__disk_from_xenstore(libxl__gc *gc, const char *libxl_path,
             goto cleanup;
         }
         disk->irq = strtoul(tmp, NULL, 10);
+
+        tmp = libxl__xs_read(gc, XBT_NULL,
+                             GCSPRINTF("%s/grant_usage", libxl_path));
+        if (!tmp) {
+            LOG(ERROR, "Missing xenstore node %s/grant_usage", libxl_path);
+            goto cleanup;
+        }
+        libxl_defbool_set(&disk->grant_usage, strtoul(tmp, NULL, 0));
     }
 
     disk->vdev = xs_read(ctx->xsh, XBT_NULL,
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 899ad30969..6d76f25528 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -803,6 +803,7 @@ libxl_device_disk = Struct("device_disk", [
     ("active_disk", string),
     ("hidden_disk", string),
     ("trusted", libxl_defbool),
+    ("grant_usage", libxl_defbool),
     ])
 
 libxl_device_nic = Struct("device_nic", [
diff --git a/tools/libs/util/libxlu_disk_l.l b/tools/libs/util/libxlu_disk_l.l
index 6d53c093a3..f37dd443bd 100644
--- a/tools/libs/util/libxlu_disk_l.l
+++ b/tools/libs/util/libxlu_disk_l.l
@@ -220,6 +220,9 @@ hidden-disk=[^,]*,?	{ STRIP(','); SAVESTRING("hidden-disk", hidden_disk, FROMEQU
 trusted,?		{ libxl_defbool_set(&DPC->disk->trusted, true); }
 untrusted,?		{ libxl_defbool_set(&DPC->disk->trusted, false); }
 
+grant_usage=1,?		{ libxl_defbool_set(&DPC->disk->grant_usage, true); }
+grant_usage=0,?		{ libxl_defbool_set(&DPC->disk->grant_usage, false); }
+
  /* the target magic parameter, eats the rest of the string */
 
 target=.*	{ STRIP(','); SAVESTRING("target", pdev_path, FROMEQUALS); }
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 11:03:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 11:03:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674992.1050170 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVrKP-0003Mc-AQ; Fri, 02 Feb 2024 11:03:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674992.1050170; Fri, 02 Feb 2024 11:03:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVrKP-0003MV-7E; Fri, 02 Feb 2024 11:03:25 +0000
Received: by outflank-mailman (input) for mailman id 674992;
 Fri, 02 Feb 2024 11:03:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CNq7=JL=linaro.org=viresh.kumar@srs-se1.protection.inumbo.net>)
 id 1rVrKN-0003MP-Qx
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 11:03:23 +0000
Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com
 [2607:f8b0:4864:20::52f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ac92d973-c1ba-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 12:03:20 +0100 (CET)
Received: by mail-pg1-x52f.google.com with SMTP id
 41be03b00d2f7-53fbf2c42bfso1553866a12.3
 for <xen-devel@lists.xenproject.org>; Fri, 02 Feb 2024 03:03:20 -0800 (PST)
Received: from localhost ([122.172.83.95]) by smtp.gmail.com with ESMTPSA id
 g1-20020aa79f01000000b006dd810cdd91sm1333659pfr.88.2024.02.02.03.03.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 02 Feb 2024 03:03:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac92d973-c1ba-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1706871799; x=1707476599; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=pOzd3P/hbbhllGoqP07nNcV24HbOBqqsBesfyxV3P3A=;
        b=K70qiCjnqwBAPpaAED6JhyHrtszkQm82TbZ5MOsZf5oHIJ1Rfzu70us0nVN98P43Ef
         ytTUkxXwAxCT/5Ivo4Sg8k3IK51XPKZFW4Ej8VChxuLz38HdbzwQkEAUtAB9dEreUUNd
         PlQ0WLdDiSloJWY+HolaOa/IO3DcHazg0vkZjbV2bK083TRSP6qZHxDFmMUkGk5eTi6M
         it7SpRgEHSk77COA2PlPTvvmixTEsopcve9/MmJHfIeC6JRXRQGnDv0K9HfIiW3SwTo6
         nCagkcM9Q+PQzZWMBUJ0+x+kqpwkWStBbU5rSNVEpt0KBe/N9YNfj32hxxQy6x8DQUDP
         SopQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706871799; x=1707476599;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=pOzd3P/hbbhllGoqP07nNcV24HbOBqqsBesfyxV3P3A=;
        b=anWyadvM97Hn8xSM54sRdaVPm0xccGJzRrJ3KMUcH48riJcazjnHMtgBk3MyVQhZ7s
         K0T0JaUAUOezGljjwgRNWXSl5Nr+0rBRmDp+ahArdpK1oJuKa2jxTel6NYLcdKoKcWz0
         p192Znd3KvGmwPugn9TZp0bTwzmtIbJg7rB/3J3ddScWtY8cvxPoVJHqCVyJ7hZM+mhZ
         GLLRejkcYsejjC0yHMN5qGud2vUY1ubNwVapHdvXm+XfO+P3FpAY4WzqLyCdkZ3sDEbc
         xAibT3zqaRXKcuZOCnXTwcsu4G4pbsdc4gLXio9a8Btk+TBiorNjjVZxfg6acIyHxr+r
         RjBA==
X-Gm-Message-State: AOJu0Yx5xgEjGQ1M+mkZMyigxzfhTrULYVXWeT7K+9YI6rX69PKvxMSm
	5RqOd2/n1mtgOVQvhcv5IFe3vhoXGlTFytjiHnhDucvqbNpcj9w9/6RmqyYoni4=
X-Google-Smtp-Source: AGHT+IH8BUE/rhtLmMDcmB6yOpNrU6zurwqymctC8MdQurtg5xR2MhTfGET5Fs5zi0GIl8poLk3Blw==
X-Received: by 2002:a05:6a20:e30b:b0:19e:4004:b6a3 with SMTP id nb11-20020a056a20e30b00b0019e4004b6a3mr5200384pzb.39.1706871799203;
        Fri, 02 Feb 2024 03:03:19 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXxyQTiAXw+BLklx0iNwHwyvVWZRGyyue7VyiFcjB7ZBKfL1i1DifOd2V+tFtJYGEbTSO/gbQSQduA4M33YRCx9+dW+sn7QE0zYvtSLQ+OQeO/Uvozlyf5hdnNYbdKUQ6xKe7Uw7X5Qy4xizMPBu2g2LPZuKLwGp0SIeYYlOKcfQrCnG/MNtOKyitb1PYuMFtQtrwZYAKiVY8w3kEdygtPxgTJdIZ4YR3fzKFVhBwwjLFx0OxafhO8Yaohg7+yujK8BOZ7p3msl545KdZ7b
Date: Fri, 2 Feb 2024 16:33:16 +0530
From: Viresh Kumar <viresh.kumar@linaro.org>
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] libxl: Add "grant_usage" parameter for virtio disk
 devices
Message-ID: <20240202110316.hfzwseow7uhdgtsz@vireshk-i7>
References: <20240202104903.1112772-1-olekstysh@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240202104903.1112772-1-olekstysh@gmail.com>

On 02-02-24, 12:49, Oleksandr Tyshchenko wrote:
> diff --git a/docs/man/xl-disk-configuration.5.pod.in b/docs/man/xl-disk-configuration.5.pod.in
> index bc945cc517..3c035456d5 100644
> --- a/docs/man/xl-disk-configuration.5.pod.in
> +++ b/docs/man/xl-disk-configuration.5.pod.in
> @@ -404,6 +404,31 @@ Virtio frontend driver (virtio-blk) to be used. Please note, the virtual
>  device (vdev) is not passed to the guest in that case, but it still must be
>  specified for the internal purposes.
>  
> +=item B<grant_usage=BOOLEAN>
> +
> +=over 4
> +
> +=item Description
> +
> +Specifies the usage of Xen grants for accessing guest memory. Only applicable
> +to specification "virtio".
> +
> +=item Supported values
> +
> +If this option is B<true>, the Xen grants are always enabled.
> +If this option is B<false>, the Xen grants are always disabled.
> +
> +=item Mandatory
> +
> +No
> +
> +=item Default value
> +
> +If this option is missing, then the default grant setting will be used,
> +i.e. enable grants if backend-domid != 0.
> +
> +=back
> +
>  =back
>  
>  =head1 COLO Parameters

I wonder if there is a way to avoid the duplication here and use the definition
from: docs/man/xl.cfg.5.pod.in somehow ?

-- 
viresh


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 11:07:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 11:07:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.674995.1050180 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVrO1-0003wM-P0; Fri, 02 Feb 2024 11:07:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 674995.1050180; Fri, 02 Feb 2024 11:07:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVrO1-0003wF-M8; Fri, 02 Feb 2024 11:07:09 +0000
Received: by outflank-mailman (input) for mailman id 674995;
 Fri, 02 Feb 2024 11:07:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVrNz-0003w5-QL
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 11:07:08 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3294e83d-c1bb-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 12:07:04 +0100 (CET)
Received: from [192.168.1.9] (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id E42604EE0737;
 Fri,  2 Feb 2024 12:07:03 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3294e83d-c1bb-11ee-98f5-efadbce2ee36
Message-ID: <97949032-6c87-4a02-a239-700ac8468968@bugseng.com>
Date: Fri, 2 Feb 2024 12:07:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 4/4] eclair: add and manage properties
Content-Language: en-US
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>
References: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
 <bc9bdf2b16a6131f476e6e1f491cf04629f431cf.1706700228.git.maria.celeste.cesario@bugseng.com>
From: Simone Ballarin <simone.ballarin@bugseng.com>
Organization: BUGSENG
In-Reply-To: <bc9bdf2b16a6131f476e6e1f491cf04629f431cf.1706700228.git.maria.celeste.cesario@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 02/02/24 10:37, Simone Ballarin wrote:
> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
> 
> Add JSON file containing properties.
> Add rst file containing explanation on how to update properties.json.
> Add instruction to eclair_analysis/prepare.sh to parse the JSON file.
> 
> Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
> ---
>   .../eclair_analysis/ECLAIR/analysis.ecl       |   1 +
>   automation/eclair_analysis/prepare.sh         |   2 +
>   docs/properties.json                          | 841 ++++++++++++++++++
>   docs/properties.rst                           |  58 ++
>   4 files changed, 902 insertions(+)
>   create mode 100644 docs/properties.json
>   create mode 100644 docs/properties.rst
> 
> diff --git a/automation/eclair_analysis/ECLAIR/analysis.ecl b/automation/eclair_analysis/ECLAIR/analysis.ecl
> index a604582da3..684c5b0b39 100644
> --- a/automation/eclair_analysis/ECLAIR/analysis.ecl
> +++ b/automation/eclair_analysis/ECLAIR/analysis.ecl
> @@ -30,6 +30,7 @@ if(not(scheduled_analysis),
>   -eval_file=deviations.ecl
>   -eval_file=call_properties.ecl
>   -eval_file=tagging.ecl
> +-eval_file=properties.ecl
>   -eval_file=concat(set,".ecl")
>   
>   -doc="Hide reports in external code."
> diff --git a/automation/eclair_analysis/prepare.sh b/automation/eclair_analysis/prepare.sh
> index fe9d16e48e..47b2a2f32a 100755
> --- a/automation/eclair_analysis/prepare.sh
> +++ b/automation/eclair_analysis/prepare.sh
> @@ -43,4 +43,6 @@ fi
>       make -f "${script_dir}/Makefile.prepare" prepare
>       # Translate the /* SAF-n-safe */ comments into ECLAIR CBTs
>       scripts/xen-analysis.py --run-eclair --no-build --no-clean
> +    # Translate function-properties.json into ECLAIR properties
> +    python3 ${script_dir}/propertyparser.py
>   )
> diff --git a/docs/properties.json b/docs/properties.json
> new file mode 100644
> index 0000000000..74058297b5
> --- /dev/null
> +++ b/docs/properties.json
> @@ -0,0 +1,841 @@
> +{
> +   "version": "1.0",
> +   "content": [
> +      {
> +         "type": "function",
> +         "value": "^printk.*$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^debugtrace_printk.*$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^panic.*$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^domain_crash$",
> +         "properties":{
> +            "pointee_write": "2..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^(g?d|mm_)?printk$",
> +         "properties":{
> +            "pointee_write": "2..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^guest_bug_on_failed$",
> +         "properties":{
> +            "pointee_write": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^spin_lock_init_prof$",
> +         "properties":{
> +            "pointee_write": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^sched_test_func$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^dev_(info|warn)$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^PAGING_DEBUG$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^ACPI_(WARNING|ERROR|INFO)$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^fdt_get_property_by_offset_.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^read_atomic_size.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^device_tree_get_reg.*$",
> +         "properties":{
> +            "pointee_write": "4..=always",
> +            "pointee_read": "4..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^dt_get_range.*$",
> +         "properties":{
> +            "pointee_write": "3..=always",
> +            "pointee_read": "3..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^parse_static_mem_prop.*$",
> +         "properties":{
> +            "pointee_write": "2..=always",
> +            "pointee_read": "2..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_ttbr_and_gran_64bit.*$",
> +         "properties":{
> +            "pointee_write": "1..2=always",
> +            "pointee_read": "1..2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^hvm_emulate_init_once.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^__vmread.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^hvm_pci_decode_addr.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^vpci_mmcfg_decode_addr.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^x86emul_decode.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^unmap_grant_ref.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^arm_smmu_cmdq_build_cmd.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^pci_size_mem_bar.*$",
> +         "properties":{
> +            "pointee_write": "4=always",
> +            "pointee_read": "4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^_hvm_read_entry.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^hvm_map_guest_frame_rw.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^guest_cpuid.*$",
> +         "properties":{
> +            "pointee_write": "4=always",
> +            "pointee_read": "4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^epte_get_entry_emt.*$",
> +         "properties":{
> +            "pointee_write": "5=always",
> +            "pointee_read": "5=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^mcheck_mca_logout.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^set_field_in_reg_u32.*$",
> +         "properties":{
> +            "pointee_write": "5=always",
> +            "pointee_read": "5=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^alloc_affinity_masks.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^xasprintf.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^find_non_smt.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^call_rcu.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^getdomaininfo.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^MAPPING_(INSERT|SEARCH)\\(.*$",
> +         "properties":{
> +            "pointee_write": "2..=always",
> +            "pointee_read": "2..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^FormatDec.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^FormatHex.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^p2m_get_ioreq_server.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^elf_memset_unchecked.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^set_iommu_pte_present.*$",
> +         "properties":{
> +            "pointee_write": "7=always",
> +            "pointee_read": "7=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^clear_iommu_pte_present.*$",
> +         "properties":{
> +            "pointee_write": "4=always",
> +            "pointee_read": "4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^vcpu_runstate_get.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^va_start.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^sgi_target_init.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_hw_residencies.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^x86_cpu_policy_to_featureset.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^simple_strtou?ll?\\(.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^msi_compose_msg.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^print_tainted.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_hvm_registers.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^x86_insn_modrm.*$",
> +         "properties":{
> +            "pointee_write": "2..3=always",
> +            "pointee_read": "2..3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^cpuid_count_leaf.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^rcu_lock_remote_domain_by_id.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^cpuid_count.*$",
> +         "properties":{
> +            "pointee_write": "3..=always",
> +            "pointee_read": "3..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^.*efi_boot_mem_unused.*$",
> +         "properties":{
> +            "pointee_write": "1..=always",
> +            "pointee_read": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^collect_time_info.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^setup_xstate_comp.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^map_domain_gfn.*$",
> +         "properties":{
> +            "pointee_read": "5=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^fdt_getprop.*$",
> +         "properties":{
> +            "pointee_read": "4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^fdt_get_name.*$",
> +         "properties":{
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^fdt_get_property.*$",
> +         "properties":{
> +            "pointee_read": "4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^pci_get_host_bridge_segment.*$",
> +         "properties":{
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^dt_get_property.*$",
> +         "properties":{
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^dt_property_read_u32.*$",
> +         "properties":{
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^dt_device_get_paddr.*$",
> +         "properties":{
> +            "pointee_read": "3..4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_evtchn_dt_property.*$",
> +         "properties":{
> +            "pointee_write": "2..3=maybe",
> +            "pointee_read": "2..3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^setup_chosen_node.*$",
> +         "properties":{
> +            "pointee_write": "2..3=maybe",
> +            "pointee_read": "2..3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^queue_remove_raw.*$",
> +         "properties":{
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^memset$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^va_start$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^memcmp$",
> +         "properties":{
> +            "pointee_write": "1..2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^memcpy$",
> +         "properties":{
> +            "pointee_write": "1=always&&2..=never",
> +            "pointee_read": "1=never&&2..=always",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_cpu_info.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^pdx_to_pfn.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^is_pci_passthrough_enabled.*$",
> +         "properties":{
> +            "attribute": "const"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_cycles.*$",
> +         "properties":{
> +            "attribute": "noeffect"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^msi_gflags.*$",
> +         "properties":{
> +            "attribute": "const"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^hvm_save_size.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^cpu_has.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^boot_cpu_has.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_cpu_info.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^put_pte_flags.*$",
> +         "properties":{
> +            "attribute": "const"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^is_pv_cpu.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "description": "Property inferred as a consequence of the semantics of device_tree_get_reg",
> +         "type": "function",
> +         "value": "^acquire_static_memory_bank.*$",
> +         "properties":{
> +            "pointee_write": "4..=always",
> +            "pointee_read": "4..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "Property inferred as a consequence of the semantics of dt_set_cell",
> +         "type": "function",
> +         "value": "^set_interrupt.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description":"Property inferred as a consequence of the semantics of __p2m_get_mem_access",
> +         "type": "function",
> +         "value": "^p2m_get_mem_access.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "This function has alternative definitions with props {write=always,read=never} and {write=never,read=never}",
> +         "type": "function",
> +         "value": "^alloc_cpumask_var.*$",
> +         "properties":{
> +            "pointee_write": "1=maybe",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "Property inferred as a consequence of the semantics of alloc_cpumask_var",
> +         "type": "function",
> +         "value": "^xenctl_bitmap_to_cpumask.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "The call to bitmap_and causes the pointee of dstp to be always written",
> +         "type": "function",
> +         "value": "^cpumask_(and|andnot|clear|copy|complement).*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "The call to bitmap_and causes the pointee of dstp to be always written",
> +         "type": "function",
> +         "value": "^bitmap_(andnot|complement|fill).*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "The .*copy_(to|from).* helpers all have a memcpy-like expectation that the destination is a copy of the source. Furthermore, their uses do initialize the involved variables as needed by futher uses in the caller.",
> +         "type": "macro",
> +         "value": "^(__)?(raw_)?copy_from_(paddr|guest|compat)(_offset)?$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description":"The .*copy_(to|from).* helpers all have a memcpy-like expectation that the destination is a copy of the source. Furthermore, their uses do initialize the involved variables as needed by futher uses in the caller.",
> +         "type": "macro",
> +         "value": "^(__)?copy_to_(guest|compat)(_offset)?$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "Functions generated by build_atomic_read cannot be considered pure since the input pointer is volatile, but they do not produce any persistent side effect.",
> +         "type": "function",
> +         "value": "^read_u(8|16|32|64|int)_atomic.*$",
> +         "properties":{
> +            "attribute": "noeffect"
> +         }
> +      },
> +      {
> +         "description": "Functions generated by TYPE_SAFE are const.",
> +         "type": "function",
> +         "value": "^(mfn|gfn|pfn)_x\\(.*$",
> +         "properties":{
> +            "attribute": "const"
> +         }
> +      },
> +      {
> +         "description": "Functions generated by TYPE_SAFE are const.",
> +         "type": "function",
> +         "value": "^_(mfn|gfn|pfn)\\(.*$",
> +         "properties":{
> +            "attribute": "const"
> +         }
> +      }
> +   ]
> +}
> diff --git a/docs/properties.rst b/docs/properties.rst
> new file mode 100644
> index 0000000000..790f1ede1e
> --- /dev/null
> +++ b/docs/properties.rst
> @@ -0,0 +1,58 @@
> +.. SPDX-License-Identifier: CC-BY-4.0
> +
> +Properties list for Xen
> +=======================
> +
> +Some functions and macros are found to have properties relevant to
> +the Xen codebase. For this reason, the file docs/properties.json
> +contains all the needed properties.
> +
> +Here is an example of the properties.json file::
> +
> +  {
> +     "version": "1.0",
> +     "content": [
> +        {
> +           "description": ""
> +           "type": "function",       // required
> +           "value:": "^printk*.$",   // required
> +           "properties":{
> +              "pointee_write": "1..2=never",
> +              "pointee_read": "",
> +              "taken": ""
> +              "attribute": ""
> +           }
> +        }
> +     ]
> +  }
> +
> +Here is an explanation of the fields inside an object of the "content" array:
> +
> + - description: a brief description of why the properties apply
> + - type: this is the kind of the element called: it may be either ``macro`` or ``function``
> + - value: must be a regex, starting with ^ and ending with $ and matching function fully
> +   qualified name or macro name.
> + - properties: a list of properties applied to said function.
> +   Possible values are:
> +
> +    - pointee_write: indicate the write use for call arguments that correspond to
> +      parameters whose pointee types are non-const
> +    - pointee_read: indicate the read use for call arguments that correspond to
> +      parameters whose pointee types are non-const
> +    - taken: indicates that the specified address arguments may be stored in objects
> +      that persist after the function has ceased to exist (excluding the returned value);
> +      address arguments not listed are never taken
> +    - attribute: attributes a function may have. Possible values are pure, const and noeffect.
> +
> +   pointee_read and pointee_write use a specific kind of argument, structured as pointee_arg=rw:
> +
> +    - pointee_arg: argument index for callee. Index 0 refers to the return value,
> +      the indices of the arguments start from 1. It can be either a single value or a range.
> +    - rw: a value that's either always, maybe or never
> +
> +       - always: for pointee_read: argument pointee is expected to be fully initialized at function enter,
> +         for pointee_write: argument pointee is fully initialized at function exit
> +       - maybe: for pointee_read: argument pointee may be expected to be initialized at function enter,
> +         for pointee_write: argument pointee may be written by function body
> +       - never: for pointee_read: argument pointee is not expected to be read in the functions's body,
> +         for pointee_write: argument pointee is never written by function body


The script for converting the JSON in an ECL file (propertyparse.py) is missing,
so the patch as it is incomplete. Please ignore it for the moment I will provide
a v4 with the missing script.

-- 
Simone Ballarin, M.Sc.

Field Application Engineer, BUGSENG (https://bugseng.com)



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 12:13:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 12:13:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675016.1050194 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVsQE-0004tw-Qf; Fri, 02 Feb 2024 12:13:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675016.1050194; Fri, 02 Feb 2024 12:13:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVsQE-0004tp-NI; Fri, 02 Feb 2024 12:13:30 +0000
Received: by outflank-mailman (input) for mailman id 675016;
 Fri, 02 Feb 2024 12:13:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WgAc=JL=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVsQC-0004tj-VP
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 12:13:28 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 780ff4e5-c1c4-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 13:13:26 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a2f22bfb4e6so287742266b.0
 for <xen-devel@lists.xenproject.org>; Fri, 02 Feb 2024 04:13:26 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 c3-20020a170906340300b00a35a659cab5sm832581ejb.179.2024.02.02.04.13.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 02 Feb 2024 04:13:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 780ff4e5-c1c4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706876006; x=1707480806; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=GWFDngQiKydBZrvoC97UkAJNkunw9jXVKneYEqNH8/Q=;
        b=RxIq8CXbCcJCt9NYHuDw3Kg3aLFhTonjKZJnybzLhvSI1/cHlGRf62JAwNweaKMr73
         ikezCEkGm92QUXf8PzPViye+W+4DETImiXgPUT46VwetZ/X4KXoHx+4HiU6R5xo6MCov
         eRddQ5mxvPETMbY52ka/Cmu+XI13VIDKYeg70=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706876006; x=1707480806;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GWFDngQiKydBZrvoC97UkAJNkunw9jXVKneYEqNH8/Q=;
        b=fcnaG9j5HFJScPTPIO+nBsF9sEUam8xrreAvyz+aBM5RsuHQ639v5jdMReZSfa0Bwt
         zs29OGIwci3DeULFo7ZO2+ftf5/X1elMSrfLT+rjQwau4RP6XVG2JXPj5gKIdGG2SmX/
         CnkK0WSgXRdFbaiGc8AEOYmtgIQffpzrUKyj5pav4BEsjuqo8cTpbSLBkoWoou2kms6m
         Z46554qUHzs3WDDbzZ1FC1tUqFxVYnTZdXnDNAtxYrKJw63fC+JJy6o7RNzU83syCqEY
         r/X+Vov1MVwK4Zd1maBvaDTcgddfbIvj2JmaTNaiFf9eOowH3khynysleb3SmOoTNtot
         pJCA==
X-Gm-Message-State: AOJu0YxCDoKuwrkJ74qh0Gr7/TPKNGNd+qJkmIukN8knog7xNIegla4o
	wGYLML61n6CdWlW/fFQR9LcujhLzvuOd8vIqDS/J6LzNRmEANmo3+fTExj6smrU=
X-Google-Smtp-Source: AGHT+IGUcVMzO69A4hRoVddv0iznUXEm8+YyLbNNAZChAsNQrUwEeagWEbvUJ5tQUNcRJzHEw1ylNA==
X-Received: by 2002:a17:906:1c15:b0:a35:7132:26d5 with SMTP id k21-20020a1709061c1500b00a35713226d5mr5384692ejg.63.1706876006313;
        Fri, 02 Feb 2024 04:13:26 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUEtlZqQmBcMybdKlK5eyJuAKksfRctfHGdIvPslYibFdpAklSAstj6g+MpCu0rSZ6GOjmNr+2xx4fPjBaylrybO86H3NQL9bfYe+s+o3RuF0mJ/rYLKuM0lDPRDipwzJfcRQj8
Message-ID: <ac612937-bb9b-4838-a69a-0a6b5967889e@citrix.com>
Date: Fri, 2 Feb 2024 12:13:25 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/CPU: convert vendor hook invocations to altcall
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <27d828d2-fb4b-41fe-82e8-90720bd9ff8f@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <27d828d2-fb4b-41fe-82e8-90720bd9ff8f@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/01/2024 11:00 am, Jan Beulich wrote:
> While not performance critical, these hook invocations still want
> converting: This way all pre-filled struct cpu_dev instances can become
> __initconst_cf_clobber, thus allowing to eliminate further 8 ENDBR
> during the 2nd phase of alternatives patching (besides moving previously
> resident data to .init.*).
>
> Since all use sites need touching anyway, take the opportunity and also
> address a Misra C:2012 Rule 5.5 violation: Rename the this_cpu static
> variable.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

> ---
> With LTO it might end up necessary to tag as __used more than just
> "default_cpu".

Why is it even needed here?

LTO can't rid early_cpu_init() of the default clause, so can't make
default_cpu unreferenced, I don't think.

> Perhaps __used would better be integrated into __initconst_cf_clobber,
> to be independent of the compiler potentially eliding structure
> instances.

Maybe.  I guess the issue here is that the tools really can't see the
connection between being in the clobber section, and alternatives going
and making a modification.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 14:18:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 14:18:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675061.1050219 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVuNI-0004sI-Pg; Fri, 02 Feb 2024 14:18:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675061.1050219; Fri, 02 Feb 2024 14:18:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVuNI-0004sB-N9; Fri, 02 Feb 2024 14:18:36 +0000
Received: by outflank-mailman (input) for mailman id 675061;
 Fri, 02 Feb 2024 14:18:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=WBRZ=JL=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rVuNH-0004s5-Gc
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 14:18:35 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f2acb06e-c1d5-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 15:18:34 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-40e7065b7bdso17761715e9.3
 for <xen-devel@lists.xenproject.org>; Fri, 02 Feb 2024 06:18:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d15-20020adfe88f000000b0033ae50e2c6asm2045835wrm.83.2024.02.02.06.18.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 02 Feb 2024 06:18:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2acb06e-c1d5-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1706883513; x=1707488313; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SOILyzCW1Wg5HSXpticJgJax4Ap7ionOOLE6NJI3tck=;
        b=BE8dHpiNHjUhF3lT9B7CdDemcftqcJ7neOLLdG2JaMm4WzKELOTpwOahTOL3fM/Jm0
         Umtkw+ObJ9hqhZRby/Y5xI0zLsBLG32zffeZfonLEkM2sVoNoIJWiQB4YwOdu0wUOGys
         H1ArdDJ6z0Dq7GaYt5vWgBw4DomJ6fUybMj/9iyJV+9QBvI2QhSvQzfZE9MUE+HMUchg
         AQ4S+b6JlinU+z+ZbgHXVpAObnJywAg6TeF4tpNpyK8PpZkYYacgit1FqvqmMwyCJLgz
         yoBSaWdfxG0J8TBYwmaX8hK1iWhxmbOp6N3tb/iS1ezm6nGLZOPXNeXVf5ZlojKIYBnb
         JswA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706883513; x=1707488313;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SOILyzCW1Wg5HSXpticJgJax4Ap7ionOOLE6NJI3tck=;
        b=p4xtMHrOyBKeUygOMXwPepGjmQd0gXgDoAFmIT3eVHxQqywJaA0o3PC/mcH6eGpUBb
         sGPL0MLrPvLCow+oycvrIsx+N9v2feCn29lgJzdjhOkG1AHd7Y4PuK7j2yHtTBauJseT
         ICUZz/oFj0/zchS5yEPy82OpdlP2EOjTm3aNSlG+2dUGIea0noCLfzVREY1uI7f9ILu9
         jcms4YKHXvemXkcnDwO9pN6tEYd0sH0iSzAEVdosEqUEPCgLjXvViSbN9/I5zAL/WSWB
         j1jE2VsvFCym3dbpXh3lN2d0dVAQcUAJj4gCdD4gOJYkCi+cbnu/vP2kyPG3B9Dlxbki
         5Jgg==
X-Gm-Message-State: AOJu0YzTX24fXKrIQPFP1/bQCYp20oEGKCf5LLRqjt6nIxyLj4fkOz/e
	L7AXgnZFCLIZSw05CiF5AJcoL1hEdrr4tWIN5cQcXjKc7aQYPnQq2kz1nOjG2g==
X-Google-Smtp-Source: AGHT+IERUYrnNY+7x/uaQU/FjXIWwena/40Nz5zy3aYEgQfSfbvY55Nu8dH0/4K0k8oJrqxlgoSWPQ==
X-Received: by 2002:adf:9b94:0:b0:337:c39a:1fbd with SMTP id d20-20020adf9b94000000b00337c39a1fbdmr3977032wrc.42.1706883513446;
        Fri, 02 Feb 2024 06:18:33 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCURzqLoDH8bBm8y8bF9oEdOWHtoDVuxjuXmCCIRAtFhafGqp64T2Z68kbmbghUCW7tNnrPSQPPRVCZUBGi78XvqzY8eIoBTyl+KfOmN2kDjRdPUPRNYG+gtUruJ0/TM7f25Xigf
Message-ID: <9eeb91f9-c33a-4d2c-ace6-55cc035470d6@suse.com>
Date: Fri, 2 Feb 2024 15:18:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/CPU: convert vendor hook invocations to altcall
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <27d828d2-fb4b-41fe-82e8-90720bd9ff8f@suse.com>
 <ac612937-bb9b-4838-a69a-0a6b5967889e@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ac612937-bb9b-4838-a69a-0a6b5967889e@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02.02.2024 13:13, Andrew Cooper wrote:
> On 23/01/2024 11:00 am, Jan Beulich wrote:
>> While not performance critical, these hook invocations still want
>> converting: This way all pre-filled struct cpu_dev instances can become
>> __initconst_cf_clobber, thus allowing to eliminate further 8 ENDBR
>> during the 2nd phase of alternatives patching (besides moving previously
>> resident data to .init.*).
>>
>> Since all use sites need touching anyway, take the opportunity and also
>> address a Misra C:2012 Rule 5.5 violation: Rename the this_cpu static
>> variable.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

Thanks.

>> ---
>> With LTO it might end up necessary to tag as __used more than just
>> "default_cpu".
> 
> Why is it even needed here?
> 
> LTO can't rid early_cpu_init() of the default clause, so can't make
> default_cpu unreferenced, I don't think.

Even without LTO I've actually seen gcc eliminate default_cpu. The
use in early_cpu_init() simply was expanded to assignments, rather
then something memcpy()-like.

>> Perhaps __used would better be integrated into __initconst_cf_clobber,
>> to be independent of the compiler potentially eliding structure
>> instances.
> 
> Maybe.  I guess the issue here is that the tools really can't see the
> connection between being in the clobber section, and alternatives going
> and making a modification.

The const-ness is the issue, I expect. From that the compiler can
infer that it may transform use of the variable into something which
in the end doesn't require the variable anymore. And it can't know
that we use all contributions to that section for a second purpose.

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 14:31:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 14:31:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675065.1050230 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVua3-0007gF-VL; Fri, 02 Feb 2024 14:31:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675065.1050230; Fri, 02 Feb 2024 14:31:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVua3-0007g8-Se; Fri, 02 Feb 2024 14:31:47 +0000
Received: by outflank-mailman (input) for mailman id 675065;
 Fri, 02 Feb 2024 14:31:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tPbJ=JL=epam.com=prvs=8762cb4b7f=oleksandr_tyshchenko@srs-se1.protection.inumbo.net>)
 id 1rVua2-0007g2-FW
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 14:31:46 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c9856b6c-c1d7-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 15:31:44 +0100 (CET)
Received: from pps.filterd (m0174680.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id
 412DAtNn002337; Fri, 2 Feb 2024 14:31:41 GMT
Received: from eur03-am7-obe.outbound.protection.outlook.com
 (mail-am7eur03lp2232.outbound.protection.outlook.com [104.47.51.232])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3w1126r7wy-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Fri, 02 Feb 2024 14:31:40 +0000 (GMT)
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com (2603:10a6:10:ed::15)
 by AS8PR03MB7842.eurprd03.prod.outlook.com (2603:10a6:20b:341::19) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.30; Fri, 2 Feb
 2024 14:31:37 +0000
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167]) by DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167%4]) with mapi id 15.20.7249.027; Fri, 2 Feb 2024
 14:31:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c9856b6c-c1d7-11ee-8a43-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=O9AREyWShnAijfjFnLr86in0QK3cFn8L8Rb4Mrgv0wXgftdHoKdfCVLrjPYFqCekSKawO7J1qY40YlXiG/bu6MVyfTdjYfF2TQvvsuoumT2V/vYaS0p3SwH6H0QkfoUkJJda8W1cVWd3x7m4rSyDCTB0JKqfKDimL/CvN8NICV9D75Ylazolgd8ep9ovxJxJFr8QLX/6IHfyXx0ANHfdT91rXUPF+gdp3TXLGZgnwvYc1WtsqDs/4fW1NQRHENELyrdbZVcooexpotKmV+FghpH4VzU/EkwCsr+cpWLsZkxp+nsisZgnt8G73vc3ie70kRtN6C6h1grSPEFexscyaQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=OG/j2KF9G6OvLae5GXWX3zztvjbeOfZVwZa2+PowQ44=;
 b=UGFn9KtHBxZV+wgRmq7qhx0jMxNE9Mv7jhOJRWSb3hhyBEUDykJmqTtwnALjrTbzI/nveHN3BCpoT+zkfg7bpKf3LxdPwsIxOmUGyS9a9i5iqYLvlq+w6hU//wErhio5WWiSml9/QtJMfQwTXKxYls0Vf+IecYMR1/7BUDj5rWJmASpXp+ZxrtwmYcHqgRkGKEwPtHKKfX7IEVVTZEWEfqUd2sSjOkDl1paJwxAccXayE8lFHZ6WrBtaUTo+zjDKoqGt3wfRQZvgNlFYf9BKW1WPGhAFOmV8yFLGCfUm5HuX9pbyWwF20H968kri4JB/OAvI17Zo9YMzT8aZ5g051A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OG/j2KF9G6OvLae5GXWX3zztvjbeOfZVwZa2+PowQ44=;
 b=WAOPYJunZYc2ZOkqGo0LzYtZFDlTe0DCMjLDN1hXZ2uf4yO+HIttjMdXFIIv//8f1W7iN5QT0/UcOFBjjCrhnlVMoMh4KB40JQadL72LYTQbuwFw1U6+EotWb86pPMurzw0KXH/OvzcoToEgJHqg6qHhM2/swwdJzHyjhaEjMe078YTr4uzH3Bia9LqPaDAhyvkZwmAp1S7B/EyBTiXiEyvl7GuYq8orzHbx11CUBUpJaM97MS0PYv90qveDb2iof5TzIaiMRKGBUuHH+CHuaFwLMgK2uxgjzVDFkQLsAxeh+5IbLzLZgnkEXA138RyohkZ5N9iBc4SN/JthatLKAg==
From: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
To: Viresh Kumar <viresh.kumar@linaro.org>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Wei Liu <wl@xen.org>, Oleksandr Tyshchenko <olekstysh@gmail.com>,
        Anthony
 PERARD <anthony.perard@citrix.com>,
        George Dunlap <george.dunlap@citrix.com>,
        Nick Rosbrook <rosbrookn@gmail.com>, Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] libxl: Add "grant_usage" parameter for virtio disk
 devices
Thread-Topic: [PATCH] libxl: Add "grant_usage" parameter for virtio disk
 devices
Thread-Index: AQHaVcWXJV0m8uwkbEKeSgAhd5N6SLD240wAgAA6LgA=
Date: Fri, 2 Feb 2024 14:31:37 +0000
Message-ID: <ea2fac26-6c14-4df0-9bb9-e311deb32832@epam.com>
References: <20240202104903.1112772-1-olekstysh@gmail.com>
 <20240202110316.hfzwseow7uhdgtsz@vireshk-i7>
In-Reply-To: <20240202110316.hfzwseow7uhdgtsz@vireshk-i7>
Accept-Language: en-US, ru-RU
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DB8PR03MB6108:EE_|AS8PR03MB7842:EE_
x-ms-office365-filtering-correlation-id: b0f813ed-ee05-48ce-1950-08dc23fbaa0d
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 1SNLGnuOtFINIy8UTz+BGAbg/xYWn1pfpXDPO/Jyaji5pqS5rFJClmIoLqY3de46qtqCXfMZz9Jo2zMbW4ZfLWrK4vWw5hA5S65HtGl74XP8vLwJBCxqSvZUAIl9pcVZhwbQu8ITuV7rXtKyi4Oz3NUEumTYnfxekW+ehUykZUM5tnTm95IzMNxCSx7hKBDZj7cSTJ4ssnmxk135GfFi5YfR08+T80GMIUu0jhfuQ/3XvThNtmm5tmjOPwe5hZmJRssYUVaAtR65MtAU7BMBVyyMbAWWzHxe9lCLQkp3k7/0bgkcZHRIiR1C2SmsaDZAQv/XDLSNQ+yL7Gqrp0xvlUAlFs4E4PhQOKQK2XkEPNvrolg1SN8byxR/mEP9b08E9sVnLz17i01hVV87KrBccdSfALh2H+BE97+sFW1PxemMAMBB4wGemgNFoqvTgseb1xjJhVyQX20oPRnKGdpzxVHdFaT5vlnGG+dY07e4m5vQplIDUvZSOHf9VnVJDpjQVeurBV280numcc78PD6tv82NWCC84ToD3TZpxZCR34+ebd1FzWuctPmsE1usTxJuWV01jUDbRgopIZGhn3nx1AB120bCczlHy9SITkPn14GfyxyK980WJD7ZqFSjXI1mgGCP+d3cI0Mv7jnvalLYZA==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR03MB6108.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(39860400002)(136003)(396003)(366004)(346002)(230922051799003)(1800799012)(451199024)(186009)(64100799003)(83380400001)(2616005)(6512007)(38100700002)(26005)(6506007)(122000001)(4326008)(64756008)(8936002)(66946007)(5660300002)(8676002)(55236004)(76116006)(2906002)(54906003)(53546011)(478600001)(71200400001)(316002)(66556008)(66476007)(66446008)(6486002)(110136005)(38070700009)(86362001)(36756003)(41300700001)(31696002)(31686004)(45980500001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?S3A3OWNSRXRMVldLUGdWNVRYdzQzNXZzUmtIcUdTMnMwRzRKbVNNaEU3UTFX?=
 =?utf-8?B?M09OdnJsNnE5Q3VwMnhab0dIdHZYTXdWNVQ3eVlWbzFrNFZFb2ZaUjVYSHN4?=
 =?utf-8?B?cmhvLzRMVjJGQWdzbTNJbHNieWtXRkx2bm1LRkE1MGxmbjF3YlptbkdpcFRC?=
 =?utf-8?B?Rk1JZWYveGhxV2JHOVhERmpLcGlsRFRnY0JkenJMNzI0L1ZHZm1YVlQ1a0Q5?=
 =?utf-8?B?U0pXN3hIaE5pcGdmK1JkNEZKekxNekY1SGZSQ1dVRzVJd1NFbFN2c3dnMmFr?=
 =?utf-8?B?b2FIYjVWRFh3M2hpNFFQYklMNzFZQmpYNjJPVjZpUHpZYjhUYitpYTRoVW5Y?=
 =?utf-8?B?eExXaXU0WHozRTl5UjIxY3pURW00cjBCSGR5VGJnQ3J2a3ZaekRvNmFJRG4r?=
 =?utf-8?B?NVdaQWFXMXBrZTJ6N3FPT0pKVDhMTzJJby9rQkhuSGQ2UURXdkJZUndzblhS?=
 =?utf-8?B?WGlINDJuT0tKaEdqWXhaRGJuNElONHIvQUxQOXIwK0dFajNucXMzcURBNURT?=
 =?utf-8?B?OUxad2lOTkZNZUIxNDBzQ0UvZy9sMVplcE5xQ2VESXcwb1dGenpscTRKekpl?=
 =?utf-8?B?dDlJdGRXaXpEVHBEVS9nN3Z0TGRMNGtIbmVVL1k1RnZka00ySXMzMGl1NzhF?=
 =?utf-8?B?UVdlUXVnUC9VU2UwZEd4NGZlYmhYdlZmdUp5NlNIaWFqUmdBamtmU0hjcTZH?=
 =?utf-8?B?M3U0U2lNb243RXQvdzhObXlJQnZkaWlHWVJYVjVIRmpKc3U5ZzZ5anV2UzB5?=
 =?utf-8?B?S2hHbk16c212L1dDci9ldU80RGM5SVFBOUdiMzFJTURFQWUxem1yTkZxVXJ5?=
 =?utf-8?B?L0Q0MFhlV0hiVWlIWjFvZTI5U21ONDh0b003K0piV0wyTkJyLzZvVSszVGxX?=
 =?utf-8?B?dlhYakUzOGVYcWRJMEFPcS9LUHFVY1duQ1I3UzhuNU1zYnNMZmg4eVFKSmh5?=
 =?utf-8?B?dFp6QUdlVTBoV3FoaE5jOTF3WUREK04vc3RiU0ZFdDVkcjFMdXB0L0tFaXdG?=
 =?utf-8?B?eU44UlNuWWltV0ptTFA2Vjd3YUtoSHFtWFFoaEFaTnJjZ3dWbWQvOTk5YlZn?=
 =?utf-8?B?RVlxQ2YwT2QxQm5yRTJoWjdBck9RRFF6TFoxcTJJb0x1M2tiaFZiMDZHTUoy?=
 =?utf-8?B?MktWczlXVEFXN1N0cW45eVZjOThMdFZnQWRVKzZueklKL0VrK29rODVzR1dq?=
 =?utf-8?B?dzk5Ti90VXBpK21RM0EzVzRySk8zWjZWRnNmRHB4Y29GMzhVWkVZS3l2cmNw?=
 =?utf-8?B?LzFCRmx1c29uY0hFbHJjTzM2QmsvVlQrQStPUTd2TWJHcUF4Sm1oaHgyVVV6?=
 =?utf-8?B?L25jY2ErK1FxR1hkclAyUEZXMFNaTmVaSmt1U04wZ1NrRUpYNm1CbTZ4VEJo?=
 =?utf-8?B?MEE1T1JUcS9oRzRiNk53Z3U2dGU2RktwMEtRYWlRM1BCLzJtMkJ4cEFQVzBC?=
 =?utf-8?B?V0J6dUVmMm9jS3V3WitGaTdZTjRHRzdOSkFVT2E2Q0NUT1E1bkJhd2N0aGxa?=
 =?utf-8?B?ZHV6SSt2NVBoTlRWWjQwZDRuZmw2TCtMTmtJdzJSVEVRM2tMbHdqRCs3b1dw?=
 =?utf-8?B?TG44WHIveE1hWUNXL0lucU5aOVdUQ1JCU3gyMzFxYTJZanJPMnNzRlllUzZC?=
 =?utf-8?B?RmhHWkpCV0RHdG9vWVB0WXpGZ1VPNjhCMklOUEVNOFVmNDlGOWFySHBNSm5u?=
 =?utf-8?B?OWM0OFF3ZnMwZ04yZzd2T1ZkOXptdjVXYk9HQy9RUmtOcHNCMHpkRFd5cHRQ?=
 =?utf-8?B?dnhSbjlwSjdwVEJ6clIyUFF3VkZKc2d0TmFVRWQzL1pSYmtWcHlBUEtsUHlm?=
 =?utf-8?B?cUVIT2VRS3FJb0Y1T1lpYlJCd3I5VS9yR3hIWW1GcFZkK2pxNk51QUlwYzNv?=
 =?utf-8?B?SkVUaTZwU2RiWTBTa0wwQ2pVUE1qMEJiM1hxSDBLMjIwa3p4WFFZQlZFSEZr?=
 =?utf-8?B?NkpETUM4UHRwZENwODVrbzdiYTV6c01ZdDQ0dENwMmJzR1duTTIwOEMxSjBj?=
 =?utf-8?B?aVU4ZHRvcUEzV3VodmY3Y0lPMWVMbVR0NUZBbGdYcFM5eENtVmhJTjc4V0VK?=
 =?utf-8?B?YjVxZmh2WHJncjhpU2J2dWpoMW9IK2JnY0RQOU9DSWNOV2dLb1ZvU0h5RDFH?=
 =?utf-8?B?L25KY3JjVDlkdGViWStUczJtNkhhSkJTWThRVWlialNEdjZpRDVlSS8xWjJL?=
 =?utf-8?Q?8F6vosj6GzOyAMV2Uo+npzM=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <2DE9435BF3074A42B8AE214B2290FA88@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB8PR03MB6108.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b0f813ed-ee05-48ce-1950-08dc23fbaa0d
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Feb 2024 14:31:37.6200
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: bNFuznfAL926AXXLw0JGAtccL7r06pHOZqc/0laWsGa4oAuu8R5OMJOdalPUMiin5mE3nbKkxKns0GqfhOqAyKni+9nXnvrnqU0nsOmpBsM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB7842
X-Proofpoint-GUID: Bbs9mBJR-SQow6WojRApXF-CkidJ65Ax
X-Proofpoint-ORIG-GUID: Bbs9mBJR-SQow6WojRApXF-CkidJ65Ax
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-02_08,2024-01-31_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 bulkscore=0
 clxscore=1011 malwarescore=0 mlxlogscore=999 phishscore=0
 priorityscore=1501 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2401310000 definitions=main-2402020106

DQoNCk9uIDAyLjAyLjI0IDEzOjAzLCBWaXJlc2ggS3VtYXIgd3JvdGU6DQoNCkhlbGxvIFZpcmVz
aA0KDQoNCj4gT24gMDItMDItMjQsIDEyOjQ5LCBPbGVrc2FuZHIgVHlzaGNoZW5rbyB3cm90ZToN
Cj4+IGRpZmYgLS1naXQgYS9kb2NzL21hbi94bC1kaXNrLWNvbmZpZ3VyYXRpb24uNS5wb2QuaW4g
Yi9kb2NzL21hbi94bC1kaXNrLWNvbmZpZ3VyYXRpb24uNS5wb2QuaW4NCj4+IGluZGV4IGJjOTQ1
Y2M1MTcuLjNjMDM1NDU2ZDUgMTAwNjQ0DQo+PiAtLS0gYS9kb2NzL21hbi94bC1kaXNrLWNvbmZp
Z3VyYXRpb24uNS5wb2QuaW4NCj4+ICsrKyBiL2RvY3MvbWFuL3hsLWRpc2stY29uZmlndXJhdGlv
bi41LnBvZC5pbg0KPj4gQEAgLTQwNCw2ICs0MDQsMzEgQEAgVmlydGlvIGZyb250ZW5kIGRyaXZl
ciAodmlydGlvLWJsaykgdG8gYmUgdXNlZC4gUGxlYXNlIG5vdGUsIHRoZSB2aXJ0dWFsDQo+PiAg
IGRldmljZSAodmRldikgaXMgbm90IHBhc3NlZCB0byB0aGUgZ3Vlc3QgaW4gdGhhdCBjYXNlLCBi
dXQgaXQgc3RpbGwgbXVzdCBiZQ0KPj4gICBzcGVjaWZpZWQgZm9yIHRoZSBpbnRlcm5hbCBwdXJw
b3Nlcy4NCj4+ICAgDQo+PiArPWl0ZW0gQjxncmFudF91c2FnZT1CT09MRUFOPg0KPj4gKw0KPj4g
Kz1vdmVyIDQNCj4+ICsNCj4+ICs9aXRlbSBEZXNjcmlwdGlvbg0KPj4gKw0KPj4gK1NwZWNpZmll
cyB0aGUgdXNhZ2Ugb2YgWGVuIGdyYW50cyBmb3IgYWNjZXNzaW5nIGd1ZXN0IG1lbW9yeS4gT25s
eSBhcHBsaWNhYmxlDQo+PiArdG8gc3BlY2lmaWNhdGlvbiAidmlydGlvIi4NCj4+ICsNCj4+ICs9
aXRlbSBTdXBwb3J0ZWQgdmFsdWVzDQo+PiArDQo+PiArSWYgdGhpcyBvcHRpb24gaXMgQjx0cnVl
PiwgdGhlIFhlbiBncmFudHMgYXJlIGFsd2F5cyBlbmFibGVkLg0KPj4gK0lmIHRoaXMgb3B0aW9u
IGlzIEI8ZmFsc2U+LCB0aGUgWGVuIGdyYW50cyBhcmUgYWx3YXlzIGRpc2FibGVkLg0KPj4gKw0K
Pj4gKz1pdGVtIE1hbmRhdG9yeQ0KPj4gKw0KPj4gK05vDQo+PiArDQo+PiArPWl0ZW0gRGVmYXVs
dCB2YWx1ZQ0KPj4gKw0KPj4gK0lmIHRoaXMgb3B0aW9uIGlzIG1pc3NpbmcsIHRoZW4gdGhlIGRl
ZmF1bHQgZ3JhbnQgc2V0dGluZyB3aWxsIGJlIHVzZWQsDQo+PiAraS5lLiBlbmFibGUgZ3JhbnRz
IGlmIGJhY2tlbmQtZG9taWQgIT0gMC4NCj4+ICsNCj4+ICs9YmFjaw0KPj4gKw0KPj4gICA9YmFj
aw0KPj4gICANCj4+ICAgPWhlYWQxIENPTE8gUGFyYW1ldGVycw0KPiANCj4gSSB3b25kZXIgaWYg
dGhlcmUgaXMgYSB3YXkgdG8gYXZvaWQgdGhlIGR1cGxpY2F0aW9uIGhlcmUgYW5kIHVzZSB0aGUg
ZGVmaW5pdGlvbg0KPiBmcm9tOiBkb2NzL21hbi94bC5jZmcuNS5wb2QuaW4gc29tZWhvdyA/DQoN
Cg0KVGhhdCdzIGdvb2QgcG9pbnQuIEkgYW0gbm90IDEwMCUgc3VyZSwgYnV0IGlmIHdlIGNvdWxk
IHVzZSBzb21ldGhpbmcgDQpsaWtlIHRoYXQgaXQgd291bGQgYmUgcmVhbGx5IG5pY2UuIExldCdz
IHNlZSB3aGF0IG90aGVyIHJldmlld2VycyB3aWxsIHNheS4NCg0KDQo9aXRlbSBCPGdyYW50X3Vz
YWdlPUJPT0xFQU4+DQoNCj1vdmVyIDQNCg0KU3BlY2lmaWVzIHRoZSB1c2FnZSBvZiBYZW4gZ3Jh
bnRzIGZvciBhY2Nlc3NpbmcgZ3Vlc3QgbWVtb3J5LiBPbmx5IA0KYXBwbGljYWJsZSB0byBzcGVj
aWZpY2F0aW9uICJ2aXJ0aW8iLiBQbGVhc2Ugc2VlIEI8Z3JhbnRfdXNhZ2U+IGluIA0KTDx4bC5j
ZmcoNSk+IGZvciBtb3JlIGluZm9ybWF0aW9uIG9uIHRoaXMgb3B0aW9uLg0KDQo9YmFjaw0K


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:16:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:16:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675070.1050240 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvGj-0005mm-0P; Fri, 02 Feb 2024 15:15:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675070.1050240; Fri, 02 Feb 2024 15:15:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvGi-0005me-Tb; Fri, 02 Feb 2024 15:15:52 +0000
Received: by outflank-mailman (input) for mailman id 675070;
 Fri, 02 Feb 2024 15:15:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pa3t=JL=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1rVvGh-0005mR-6J
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 15:15:51 +0000
Received: from mail-ej1-x641.google.com (mail-ej1-x641.google.com
 [2a00:1450:4864:20::641])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f2238b82-c1dd-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 16:15:49 +0100 (CET)
Received: by mail-ej1-x641.google.com with SMTP id
 a640c23a62f3a-a271a28aeb4so327411366b.2
 for <xen-devel@lists.xenproject.org>; Fri, 02 Feb 2024 07:15:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2238b82-c1dd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1706886948; x=1707491748; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=TNDypIirJFmg0WiRSLJ9ja2kfhRIBVyVdgbPpTnEuD4=;
        b=GhqOuUwV7/yd4Os85Z/ox9D3IdBKGZZU2sUulsKqMv8Po9f7u+J9XR1hCuswbrQV5e
         +zIQiXX4MqAokKP1wZPxRPcCNtl/MvDYV+Hn7iP/Wxb7/Tbk/isX2ys1BiQMhMd2VbWY
         ZdGNQcG0qeGzeDXT+sF3uib5YReOP/tzl0Bms=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706886948; x=1707491748;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=TNDypIirJFmg0WiRSLJ9ja2kfhRIBVyVdgbPpTnEuD4=;
        b=QlV7aRXdw/CavhTLp5s5OKIR1T8hGCsMcM38vPiyC2ArZWSuqDhsF10vITs51cL9i4
         UCswCbxw3zh0FplJdwE05sJ8M6rstDx4adJgr4pSLhPox5qBe+OUYofiAp3Qm2Lzl+CC
         BHNaZtXj3BlSgTRSoY+o/1eZ83pXoUSnZEFToffvpYJHNPT+lslssP00nlb5f9c+9X67
         RNWshuDPp5dLRp3BaWZsN/vrSMSun71Pw9/XhahUpRkBCmENN32gD3DUv9A/oUyR+u68
         S+pfT0ZKO3EdWY0v2pbqi//lDKLZpQT6giJlWzo71hPozSR44xaYfZ0rQ+Kgxhnkyt4/
         RFbg==
X-Gm-Message-State: AOJu0YysJSEgGnodVFd15MpJh5ncWwH5tRybmFG73AGcKaOgyfsGUGKF
	DNXv8AYJO3tgrhdfD8/1YAJSrDVgB6piwsFCh0gA/Es0yefee7zn+XGDZG+fguWKSbM2tIxyUEE
	t/kKrFUxNvoLxT4lD9Xd6oO8kVrEB/nQ8YTTD+IQlL7KbrJC5EgB4JlEXIC+etg==
X-Google-Smtp-Source: AGHT+IH9rRd0s1x5tRCe+14Sm6gbdVbmZgQvYL/lsjv85Gc3axsCs1i0uEjvBDDFmNYRL7m6eVzmOsFcNRONgivkns4=
X-Received: by 2002:a17:907:7701:b0:a37:3234:3e9f with SMTP id
 kw1-20020a170907770100b00a3732343e9fmr140463ejc.39.1706886948207; Fri, 02 Feb
 2024 07:15:48 -0800 (PST)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Fri, 2 Feb 2024 16:14:00 +0100
Message-ID: <CAO-mL=xKyG0EyFJyUkbkojUVc8-kpL4e9mciP2O0PueCMXnGWQ@mail.gmail.com>
Subject: Community call recording: 1st February 2024
To: xen-devel@lists.xenproject.org
Content-Type: multipart/alternative; boundary="00000000000038634e0610679607"

--00000000000038634e0610679607
Content-Type: text/plain; charset="UTF-8"

Hi all,

Please find the community call recording below.
https://www.youtube.com/watch?v=MJSTvwvNtF8&ab_channel=TheXenProject

These are unlisted and published on our new YouTube channel. Only users
with the specific link below will be able to access the recording.

This has also been saved in the agenda notes.

Many thanks,
Kelly Choi

Community Manager
Xen Project

--00000000000038634e0610679607
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi all,=C2=A0</div><div><br></div><div>Please find th=
e community call recording below.=C2=A0</div><div><a href=3D"https://www.yo=
utube.com/watch?v=3DMJSTvwvNtF8&amp;ab_channel=3DTheXenProject">https://www=
.youtube.com/watch?v=3DMJSTvwvNtF8&amp;ab_channel=3DTheXenProject</a><br></=
div><div><br></div><div>These are unlisted and published on our new YouTube=
 channel. Only users with the specific link below will be able to access th=
e=C2=A0<span class=3D"gmail-il">recording</span>.<br></div><div><br></div><=
div>This has also been saved in the agenda notes.=C2=A0</div><div><br></div=
><div><div dir=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_si=
gnature"><div dir=3D"ltr"><div>Many thanks,</div><div>Kelly Choi</div><div>=
<br></div><div><div style=3D"color:rgb(136,136,136)">Community Manager</div=
><div style=3D"color:rgb(136,136,136)">Xen Project=C2=A0<br></div></div></d=
iv></div></div></div>

--00000000000038634e0610679607--


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:16:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:16:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675076.1050250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvHg-0006Mq-C3; Fri, 02 Feb 2024 15:16:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675076.1050250; Fri, 02 Feb 2024 15:16:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvHg-0006Mj-9P; Fri, 02 Feb 2024 15:16:52 +0000
Received: by outflank-mailman (input) for mailman id 675076;
 Fri, 02 Feb 2024 15:16:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVvHe-0005mR-Gr
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 15:16:50 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 159e6126-c1de-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 16:16:48 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 63D3E4EE0737;
 Fri,  2 Feb 2024 16:16:47 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 159e6126-c1de-11ee-98f5-efadbce2ee36
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	sstabellini@kernel.org,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Dario Faggioli <dfaggioli@suse.com>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [XEN PATCH v4 0/4] address violation of MISRA C:2012 Rule 13.1
Date: Fri,  2 Feb 2024 16:16:01 +0100
Message-Id: <cover.1706886631.git.simone.ballarin@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit


The Xen sources contain violations of MISRA C:2012 Rule 13.1 whose headline states:
"Initializer lists shall not contain persistent side effects".

The file properties.json containing function and macro properties is introduced, as
stated in v2 discussion. Some functions and macros are found to have properties that
can be exploited by static analyzers. For this reason, the file docs/properties.json
contains all the needed properties. A description of the json file is documented in
docs/properties.rst.

Some persistent effects have been moved outside initializer lists to address violations
of Rule 13.1.

Link to the discussion: https://lore.kernel.org/all/cover.1700844359.git.simone.ballarin@bugseng.com/T/#u

Changes in v4:
- added missing script for converting function_macro_properties.json in ECL configurations
  and other related improvements.

Changes in v3:
- change prefix from xen to xen/ns16550
- add assignment of rc in xen/ns16550
- use rc as controlling expression in the following if-statement
- change commit prefix from xen/arm to xen
- specify where saf-3-safe comments are applied in guestcopy.c
- reword saf comments text

Maria Celeste Cesario (1):
  eclair: move function and macro properties outside ECLAIR

Simone Ballarin (3):
  xen: add SAF deviation for debugging and logging effects
  xen/ns16550: address violations of MISRA C:2012 Rule 13.1
  xen/x86: address violations of MISRA C:2012 Rule 13.1

 .../eclair_analysis/ECLAIR/analysis.ecl       |   1 +
 .../ECLAIR/call_properties.ecl                | 128 ---
 automation/eclair_analysis/prepare.sh         |   2 +
 automation/eclair_analysis/propertyparser.py  |  37 +
 docs/function_macro_properties.json           | 841 ++++++++++++++++++
 docs/function_macro_properties.rst            |  58 ++
 docs/misra/safe.json                          |  16 +
 xen/arch/arm/device.c                         |   1 +
 xen/arch/arm/guestcopy.c                      |  16 +-
 xen/arch/x86/hvm/hvm.c                        |   1 +
 xen/arch/x86/io_apic.c                        |   9 +-
 xen/arch/x86/mpparse.c                        |   3 +-
 xen/arch/x86/setup.c                          |   3 +-
 xen/common/sched/core.c                       |   3 +
 xen/drivers/char/ns16550.c                    |   4 +-
 15 files changed, 985 insertions(+), 138 deletions(-)
 delete mode 100644 automation/eclair_analysis/ECLAIR/call_properties.ecl
 create mode 100644 automation/eclair_analysis/propertyparser.py
 create mode 100644 docs/function_macro_properties.json
 create mode 100644 docs/function_macro_properties.rst

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:16:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:16:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675077.1050255 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvHg-0006Pq-NF; Fri, 02 Feb 2024 15:16:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675077.1050255; Fri, 02 Feb 2024 15:16:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvHg-0006PM-Gh; Fri, 02 Feb 2024 15:16:52 +0000
Received: by outflank-mailman (input) for mailman id 675077;
 Fri, 02 Feb 2024 15:16:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVvHe-0006ME-OC
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 15:16:50 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 161edc99-c1de-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 16:16:49 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 6C70D4EE073C;
 Fri,  2 Feb 2024 16:16:48 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 161edc99-c1de-11ee-8a43-1f161083a0e0
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	sstabellini@kernel.org,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Dario Faggioli <dfaggioli@suse.com>,
	Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
Subject: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and logging effects
Date: Fri,  2 Feb 2024 16:16:02 +0100
Message-Id: <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1706886631.git.simone.ballarin@bugseng.com>
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Rule 13.1: Initializer lists shall not contain persistent side effects

Effects caused by debug/logging macros and functions (like ASSERT, __bad_atomic_size,
LOG, etc ...) that crash execution or produce logs are not dangerous in initializer
lists. The evaluation order in abnormal conditions is not relevant. Evaluation order
of logging effects is always safe.

Function hvm_get_guest_tsc_fixed (indirectly) performs different side effects.
For example it calls hvm_get_guest_time_fixed that contains an ASSERT and calls
to spin_lock and spin_unlock.

These side effects are not dangerous: they can be executed regardless of the
initializer list evaluation order

This patch deviates violations using SAF commits caused by debug/logging macros and
functions.

Asm volatile statements in initializer lists that do not perform any persistent side
effect are safe: this patch deviates violations caused by uses of the current macro
(that contains an asm volatile) in initializer lists.

No functional changes.

Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>

---
Changes in v3:
- change commit prefix from xen/arm to xen
- specify where saf-3-safe comments are applied in guestcopy.c
- reword SAF text
Changes in v2:
New patch based on the discussion for "xen/arm: address violations of MISRA C:2012 Rule 13.1".
---
 docs/misra/safe.json     | 16 ++++++++++++++++
 xen/arch/arm/device.c    |  1 +
 xen/arch/arm/guestcopy.c | 16 ++++++++++++----
 xen/arch/x86/hvm/hvm.c   |  1 +
 xen/common/sched/core.c  |  3 +++
 5 files changed, 33 insertions(+), 4 deletions(-)

diff --git a/docs/misra/safe.json b/docs/misra/safe.json
index 952324f85c..5539e8dfda 100644
--- a/docs/misra/safe.json
+++ b/docs/misra/safe.json
@@ -28,6 +28,22 @@
         },
         {
             "id": "SAF-3-safe",
+            "analyser": {
+                "eclair": "MC3R1.R13.1"
+            },
+            "name": "MC3R1.R13.1: effects for debugging and logging",
+            "text": "Effects for debugging and loggings reasons that crash execution or produce logs are allowed in initializer lists. The evaluation order in abnormal conditions is not relevant."
+        },
+        {
+            "id": "SAF-4-safe",
+            "analyser": {
+                "eclair": "MC3R1.R13.1"
+            },
+            "name": "MC3R1.R13.1: volatile asm statements that do not perform any persistent side effect",
+            "text": "Volatile asm statement in an initializer list that does not perform persistent side effects is safe."
+        },
+        {
+            "id": "SAF-5-safe",
             "analyser": {},
             "name": "Sentinel",
             "text": "Next ID to be used"
diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
index 1f631d3274..fa331f164d 100644
--- a/xen/arch/arm/device.c
+++ b/xen/arch/arm/device.c
@@ -331,6 +331,7 @@ int handle_device(struct domain *d, struct dt_device_node *dev, p2m_type_t p2mt,
         .p2mt = p2mt,
         .skip_mapping = !own_device ||
                         (is_pci_passthrough_enabled() &&
+                        /* SAF-3-safe effects for debugging/logging reasons are safe */
                         (device_get_class(dev) == DEVICE_PCI_HOSTBRIDGE)),
         .iomem_ranges = iomem_ranges,
         .irq_ranges = irq_ranges
diff --git a/xen/arch/arm/guestcopy.c b/xen/arch/arm/guestcopy.c
index 6716b03561..b75538252a 100644
--- a/xen/arch/arm/guestcopy.c
+++ b/xen/arch/arm/guestcopy.c
@@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf, uint64_t addr, unsigned int len,
 unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
 {
     return copy_guest((void *)from, (vaddr_t)to, len,
-                      GVA_INFO(current), COPY_to_guest | COPY_linear);
+                      /* SAF-4-safe No persistent side effects */
+                      GVA_INFO(current),
+                      COPY_to_guest | COPY_linear);
 }
 
 unsigned long raw_copy_to_guest_flush_dcache(void *to, const void *from,
                                              unsigned int len)
 {
-    return copy_guest((void *)from, (vaddr_t)to, len, GVA_INFO(current),
+    return copy_guest((void *)from, (vaddr_t)to, len,
+                      /* SAF-4-safe No persistent side effects */
+                      GVA_INFO(current),
                       COPY_to_guest | COPY_flush_dcache | COPY_linear);
 }
 
 unsigned long raw_clear_guest(void *to, unsigned int len)
 {
-    return copy_guest(NULL, (vaddr_t)to, len, GVA_INFO(current),
+    return copy_guest(NULL, (vaddr_t)to, len,
+                      /* SAF-4-safe No persistent side effects */
+                      GVA_INFO(current),
                       COPY_to_guest | COPY_linear);
 }
 
 unsigned long raw_copy_from_guest(void *to, const void __user *from,
                                   unsigned int len)
 {
-    return copy_guest(to, (vaddr_t)from, len, GVA_INFO(current),
+    return copy_guest(to, (vaddr_t)from, len,
+                      /* SAF-4-safe No persistent side effects */
+                      GVA_INFO(current),
                       COPY_from_guest | COPY_linear);
 }
 
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index e8deeb0222..19322fbb56 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -800,6 +800,7 @@ static int cf_check hvm_save_cpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
 {
     struct segment_register seg;
     struct hvm_hw_cpu ctxt = {
+        /* SAF-3-safe effects for debugging/logging reasons are safe */
         .tsc = hvm_get_guest_tsc_fixed(v, v->domain->arch.hvm.sync_tsc),
         .msr_tsc_aux = v->arch.msrs->tsc_aux,
         .rax = v->arch.user_regs.rax,
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index d177c675c8..9e973fcf31 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -1521,6 +1521,7 @@ long vcpu_yield(void)
 
     SCHED_STAT_CRANK(vcpu_yield);
 
+    /* SAF-4-safe No persistent side effects */
     TRACE_2D(TRC_SCHED_YIELD, current->domain->domain_id, current->vcpu_id);
     raise_softirq(SCHEDULE_SOFTIRQ);
     return 0;
@@ -1899,6 +1900,7 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         if ( copy_from_guest(&sched_shutdown, arg, 1) )
             break;
 
+        /* SAF-4-safe No persistent side effects */
         TRACE_3D(TRC_SCHED_SHUTDOWN,
                  current->domain->domain_id, current->vcpu_id,
                  sched_shutdown.reason);
@@ -1916,6 +1918,7 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
         if ( copy_from_guest(&sched_shutdown, arg, 1) )
             break;
 
+        /* SAF-4-safe No persistent side effects */
         TRACE_3D(TRC_SCHED_SHUTDOWN_CODE,
                  d->domain_id, current->vcpu_id, sched_shutdown.reason);
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:16:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:16:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675078.1050262 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvHh-0006T2-2s; Fri, 02 Feb 2024 15:16:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675078.1050262; Fri, 02 Feb 2024 15:16:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvHg-0006RZ-P3; Fri, 02 Feb 2024 15:16:52 +0000
Received: by outflank-mailman (input) for mailman id 675078;
 Fri, 02 Feb 2024 15:16:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVvHf-0005mR-Kl
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 15:16:51 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1681f954-c1de-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 16:16:50 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 55B664EE073F;
 Fri,  2 Feb 2024 16:16:49 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1681f954-c1de-11ee-98f5-efadbce2ee36
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	sstabellini@kernel.org,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>,
	Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
Subject: [XEN PATCH v4 2/4] xen/ns16550: address violations of MISRA C:2012 Rule 13.1
Date: Fri,  2 Feb 2024 16:16:03 +0100
Message-Id: <08f1d4a7475c557da016482025067f54e2e56b99.1706886631.git.simone.ballarin@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1706886631.git.simone.ballarin@bugseng.com>
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Rule 13.1: Initializer lists shall not contain persistent side effects

The assignment operation in:

.irq = rc = uart->irq,

is a persistent side effect in a struct initializer list.

This patch assigns rc separately outside the structure.

No functional change.

Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>

---
Changes in v3:
- add assignment of rc;
- use rc as controlling expression in the following if-statement;
- change prefix from xen to xen/ns16550.
Changes in v2:
- avoid assignment of rc;
- drop changes in vcpu_yield(void).
---
 xen/drivers/char/ns16550.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c
index afe3d514b9..97bf098534 100644
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -441,10 +441,12 @@ static void __init cf_check ns16550_init_postirq(struct serial_port *port)
             struct msi_info msi = {
                 .sbdf = PCI_SBDF(0, uart->ps_bdf[0], uart->ps_bdf[1],
                                  uart->ps_bdf[2]),
-                .irq = rc = uart->irq,
+                .irq = uart->irq,
                 .entry_nr = 1
             };
 
+            rc = uart->irq;
+
             if ( rc > 0 )
             {
                 struct msi_desc *msi_desc = NULL;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:16:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:16:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675079.1050280 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvHi-000724-8D; Fri, 02 Feb 2024 15:16:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675079.1050280; Fri, 02 Feb 2024 15:16:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvHi-00070O-1Z; Fri, 02 Feb 2024 15:16:54 +0000
Received: by outflank-mailman (input) for mailman id 675079;
 Fri, 02 Feb 2024 15:16:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVvHg-0005mR-HE
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 15:16:52 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 16da0948-c1de-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 16:16:50 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 03D944EE0741;
 Fri,  2 Feb 2024 16:16:49 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16da0948-c1de-11ee-98f5-efadbce2ee36
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	sstabellini@kernel.org,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH v4 3/4] xen/x86: address violations of MISRA C:2012 Rule 13.1
Date: Fri,  2 Feb 2024 16:16:04 +0100
Message-Id: <16bb514ac0a5fe0d6e9a2c95279a8200ff4495c6.1706886631.git.simone.ballarin@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1706886631.git.simone.ballarin@bugseng.com>
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Rule 13.1: Initializer lists shall not contain persistent side effects

This patch moves expressions with side-effects into new variables before
the initializer lists.

No functional changes.

Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
---
 xen/arch/x86/io_apic.c | 9 ++++++---
 xen/arch/x86/mpparse.c | 3 ++-
 xen/arch/x86/setup.c   | 3 ++-
 3 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
index b48a642465..4a6ab85689 100644
--- a/xen/arch/x86/io_apic.c
+++ b/xen/arch/x86/io_apic.c
@@ -2559,9 +2559,12 @@ integer_param("max_gsi_irqs", max_gsi_irqs);
 
 static __init bool bad_ioapic_register(unsigned int idx)
 {
-    union IO_APIC_reg_00 reg_00 = { .raw = io_apic_read(idx, 0) };
-    union IO_APIC_reg_01 reg_01 = { .raw = io_apic_read(idx, 1) };
-    union IO_APIC_reg_02 reg_02 = { .raw = io_apic_read(idx, 2) };
+    uint32_t reg_00_raw = io_apic_read(idx, 0);
+    uint32_t reg_01_raw = io_apic_read(idx, 1);
+    uint32_t reg_02_raw = io_apic_read(idx, 2);
+    union IO_APIC_reg_00 reg_00 = { .raw = reg_00_raw };
+    union IO_APIC_reg_01 reg_01 = { .raw = reg_01_raw };
+    union IO_APIC_reg_02 reg_02 = { .raw = reg_02_raw };
 
     if ( reg_00.raw == -1 && reg_01.raw == -1 && reg_02.raw == -1 )
     {
diff --git a/xen/arch/x86/mpparse.c b/xen/arch/x86/mpparse.c
index d8ccab2449..81a819403b 100644
--- a/xen/arch/x86/mpparse.c
+++ b/xen/arch/x86/mpparse.c
@@ -798,11 +798,12 @@ void __init mp_register_lapic_address (
 
 int mp_register_lapic(u32 id, bool enabled, bool hotplug)
 {
+	u32 apic = apic_read(APIC_LVR);
 	struct mpc_config_processor processor = {
 		.mpc_type = MP_PROCESSOR,
 		/* Note: We don't fill in fields not consumed anywhere. */
 		.mpc_apicid = id,
-		.mpc_apicver = GET_APIC_VERSION(apic_read(APIC_LVR)),
+		.mpc_apicver = GET_APIC_VERSION(apic),
 		.mpc_cpuflag = (enabled ? CPU_ENABLED : 0) |
 			       (id == boot_cpu_physical_apicid ?
 				CPU_BOOTPROCESSOR : 0),
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index ee682dd136..886031d86a 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -885,13 +885,14 @@ static struct domain *__init create_dom0(const module_t *image,
 {
     static char __initdata cmdline[MAX_GUEST_CMDLINE];
 
+    unsigned int max_vcpus = dom0_max_vcpus();
     struct xen_domctl_createdomain dom0_cfg = {
         .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
         .max_evtchn_port = -1,
         .max_grant_frames = -1,
         .max_maptrack_frames = -1,
         .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version),
-        .max_vcpus = dom0_max_vcpus(),
+        .max_vcpus = max_vcpus,
         .arch = {
             .misc_flags = opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0,
         },
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:16:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:16:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675080.1050290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvHj-0007Lh-BZ; Fri, 02 Feb 2024 15:16:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675080.1050290; Fri, 02 Feb 2024 15:16:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvHj-0007Ko-88; Fri, 02 Feb 2024 15:16:55 +0000
Received: by outflank-mailman (input) for mailman id 675080;
 Fri, 02 Feb 2024 15:16:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVvHi-0005mR-4D
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 15:16:54 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 174f9951-c1de-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 16:16:51 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 90D444EE0740;
 Fri,  2 Feb 2024 16:16:50 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 174f9951-c1de-11ee-98f5-efadbce2ee36
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	sstabellini@kernel.org,
	Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH v4 4/4] eclair: move function and macro properties outside ECLAIR
Date: Fri,  2 Feb 2024 16:16:05 +0100
Message-Id: <387b160ae93c221c4bc2426605b96b432b26224e.1706886631.git.simone.ballarin@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1706886631.git.simone.ballarin@bugseng.com>
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>

Function and macro properties contained in ECLAIR/call_properties.ecl are of
general interest: this patch moves these annotations in a generaric JSON file
in docs. In this way, they can be exploited for other purposes (i.e. documentation,
other tools).

Add rst file containing explanation on how to update function_macro_properties.json.
Add script to convert the JSON file in ECL configurations.
Remove ECLAIR/call_properties.ecl: the file is now automatically generated from
the JSON file.

Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>

---
Changes in v4:
- add missing script for converting the JSON file in ECL configurations;
- improve commit message;
- remove call_properties.ecs.
---
 .../eclair_analysis/ECLAIR/analysis.ecl       |   1 +
 .../ECLAIR/call_properties.ecl                | 128 ---
 automation/eclair_analysis/prepare.sh         |   2 +
 automation/eclair_analysis/propertyparser.py  |  37 +
 docs/function_macro_properties.json           | 841 ++++++++++++++++++
 docs/function_macro_properties.rst            |  58 ++
 6 files changed, 939 insertions(+), 128 deletions(-)
 delete mode 100644 automation/eclair_analysis/ECLAIR/call_properties.ecl
 create mode 100644 automation/eclair_analysis/propertyparser.py
 create mode 100644 docs/function_macro_properties.json
 create mode 100644 docs/function_macro_properties.rst

diff --git a/automation/eclair_analysis/ECLAIR/analysis.ecl b/automation/eclair_analysis/ECLAIR/analysis.ecl
index a604582da3..684c5b0b39 100644
--- a/automation/eclair_analysis/ECLAIR/analysis.ecl
+++ b/automation/eclair_analysis/ECLAIR/analysis.ecl
@@ -30,6 +30,7 @@ if(not(scheduled_analysis),
 -eval_file=deviations.ecl
 -eval_file=call_properties.ecl
 -eval_file=tagging.ecl
+-eval_file=properties.ecl
 -eval_file=concat(set,".ecl")
 
 -doc="Hide reports in external code."
diff --git a/automation/eclair_analysis/ECLAIR/call_properties.ecl b/automation/eclair_analysis/ECLAIR/call_properties.ecl
deleted file mode 100644
index c2b2a6182e..0000000000
--- a/automation/eclair_analysis/ECLAIR/call_properties.ecl
+++ /dev/null
@@ -1,128 +0,0 @@
-
--call_properties+={"name(printk)", {"pointee_write(1..=never)", "taken()"}}
--call_properties+={"name(debugtrace_printk)", {"pointee_write(1..=never)", "taken()"}}
--call_properties+={"name(panic)", {"pointee_write(1..=never)", "taken()"}}
--call_properties+={"macro(^domain_crash$)", {"pointee_write(2..=never)", "taken()"}}
--call_properties+={"macro(^(g?d|mm_)?printk$)", {"pointee_write(2..=never)", "taken()"}}
--call_properties+={"macro(^guest_bug_on_failed$)", {"pointee_write(1=never)", "taken()"}}
--call_properties+={"macro(^spin_lock_init_prof$)", {"pointee_write(2=never)", "taken()"}}
--call_properties+={"macro(^sched_test_func$)", {"pointee_write(1..=never)", "taken()"}}
--call_properties+={"macro(^dev_(info|warn)$)", {"pointee_write(1..=never)", "taken()"}}
--call_properties+={"macro(^PAGING_DEBUG$)", {"pointee_write(1..=never)", "taken()"}}
--call_properties+={"macro(^ACPI_(WARNING|ERROR|INFO)$)", {"pointee_write(1..=never)", "taken()"}}
--call_properties+={"name(fdt_get_property_by_offset_)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
--call_properties+={"name(read_atomic_size)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(device_tree_get_reg)", {"pointee_write(4..=always)", "pointee_read(4..=never)", "taken()"}}
--call_properties+={"name(dt_get_range)", {"pointee_write(3..=always)", "pointee_read(3..=never)", "taken()"}}
--call_properties+={"name(parse_static_mem_prop)", {"pointee_write(2..=always)", "pointee_read(2..=never)", "taken()"}}
--call_properties+={"name(get_ttbr_and_gran_64bit)", {"pointee_write(1..2=always)", "pointee_read(1..2=never)", "taken()"}}
--call_properties+={"name(hvm_emulate_init_once)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(__vmread)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(hvm_pci_decode_addr)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
--call_properties+={"name(vpci_mmcfg_decode_addr)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
--call_properties+={"name(x86emul_decode)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(unmap_grant_ref)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(arm_smmu_cmdq_build_cmd)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(pci_size_mem_bar)", {"pointee_write(4=always)", "pointee_read(4=never)", "taken()"}}
--call_properties+={"name(_hvm_read_entry)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(hvm_map_guest_frame_rw)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
--call_properties+={"name(guest_cpuid)", {"pointee_write(4=always)", "pointee_read(4=never)", "taken()"}}
--call_properties+={"name(epte_get_entry_emt)", {"pointee_write(5=always)", "pointee_read(5=never)", "taken()"}}
--call_properties+={"name(mcheck_mca_logout)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
--call_properties+={"name(set_field_in_reg_u32)", {"pointee_write(5=always)", "pointee_read(5=never)", "taken()"}}
--call_properties+={"name(alloc_affinity_masks)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(xasprintf)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(find_non_smt)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(call_rcu)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(getdomaininfo)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"^MAPPING_(INSERT|SEARCH)\\(.*$", {"pointee_write(2..=always)", "pointee_read(2..=never)", "taken()"}}
--call_properties+={"name(FormatDec)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(FormatHex)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
--call_properties+={"name(p2m_get_ioreq_server)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(elf_memset_unchecked)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(set_iommu_pte_present)", {"pointee_write(7=always)", "pointee_read(7=never)", "taken()"}}
--call_properties+={"name(clear_iommu_pte_present)", {"pointee_write(4=always)", "pointee_read(4=never)", "taken()"}}
--call_properties+={"name(vcpu_runstate_get)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(va_start)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(sgi_target_init)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(get_hw_residencies)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(x86_cpu_policy_to_featureset)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"^simple_strtou?ll?\\(.*$", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(msi_compose_msg)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
--call_properties+={"name(print_tainted)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(get_hvm_registers)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
--call_properties+={"name(x86_insn_modrm)", {"pointee_write(2..3=always)", "pointee_read(2..3=never)", "taken()"}}
--call_properties+={"name(cpuid_count_leaf)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
--call_properties+={"name(rcu_lock_remote_domain_by_id)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(cpuid_count)", {"pointee_write(3..=always)", "pointee_read(3..=never)", "taken()"}}
--call_properties+={"name(efi_boot_mem_unused)", {"pointee_write(1..=always)", "pointee_read(1..=never)", "taken()"}}
--call_properties+={"name(collect_time_info)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--call_properties+={"name(setup_xstate_comp)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"name(map_domain_gfn)", {"pointee_read(5=never)", "taken()"}}
--call_properties+={"name(fdt_getprop)", {"pointee_read(4=never)", "taken()"}}
--call_properties+={"name(fdt_get_name)", {"pointee_read(3=never)", "taken()"}}
--call_properties+={"name(fdt_get_property)", {"pointee_read(4=never)", "taken()"}}
--call_properties+={"name(pci_get_host_bridge_segment)", {"pointee_read(2=never)", "taken()"}}
--call_properties+={"name(dt_get_property)", {"pointee_read(3=never)", "taken()"}}
--call_properties+={"name(dt_property_read_u32)", {"pointee_read(3=never)", "taken()"}}
--call_properties+={"name(dt_device_get_paddr)", {"pointee_read(3..4=never)", "taken()"}}
--call_properties+={"name(get_evtchn_dt_property)", {"pointee_write(2..3=maybe)", "pointee_read(2..3=never)", "taken()"}}
--call_properties+={"name(setup_chosen_node)", {"pointee_write(2..3=maybe)", "pointee_read(2..3=never)", "taken()"}}
--call_properties+={"name(queue_remove_raw)", {"pointee_read(2=never)", "taken()"}}
--call_properties+={"macro(^memset$)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"macro(^va_start$)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"macro(^memcmp$)", {"pointee_write(1..2=never)", "taken()"}}
--call_properties+={"macro(^memcpy$)", {"pointee_write(1=always&&2..=never)", "pointee_read(1=never&&2..=always)", "taken()"}}
--call_properties+={"name(get_cpu_info)",{pure}}
--call_properties+={"name(pdx_to_pfn)",{pure}}
--call_properties+={"name(is_pci_passthrough_enabled)",{const}}
--call_properties+={"name(get_cycles)", {"noeffect"}}
--call_properties+={"name(msi_gflags)",{const}}
--call_properties+={"name(hvm_save_size)",{pure}}
--call_properties+={"name(cpu_has)",{pure}}
--call_properties+={"name(boot_cpu_has)",{pure}}
--call_properties+={"name(get_cpu_info)",{pure}}
--call_properties+={"name(put_pte_flags)",{const}}
--call_properties+={"name(is_pv_vcpu)",{pure}}
-
--doc_begin="Property inferred as a consequence of the semantics of device_tree_get_reg"
--call_properties+={"name(acquire_static_memory_bank)", {"pointee_write(4..=always)", "pointee_read(4..=never)", "taken()"}}
--doc_end
-
--doc_begin="Property inferred as a consequence of the semantics of dt_set_cell"
--call_properties+={"name(set_interrupt)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--doc_end
-
--doc_begin="Property inferred as a consequence of the semantics of __p2m_get_mem_access"
--call_properties+={"name(p2m_get_mem_access)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
--doc_end
-
--doc_begin="This function has alternative definitions with props {write=always,read=never} and {write=never,read=never}"
--call_properties+={"name(alloc_cpumask_var)", {"pointee_write(1=maybe)", "pointee_read(1=never)", "taken()"}}
--doc_end
-
--doc_begin="Property inferred as a consequence of the semantics of alloc_cpumask_var"
--call_properties+={"name(xenctl_bitmap_to_cpumask)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--doc_end
-
--doc_begin="The call to bitmap_and causes the pointee of dstp to be always written"
--call_properties+={"^cpumask_(and|andnot|clear|copy|complement).*$", {"pointee_write(1=always)", "pointee_read(1=never)" "taken()"}}
--call_properties+={"^bitmap_(andnot|complement|fill).*$", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--doc_end
-
--doc_begin="The .*copy_(to|from).* helpers all have a memcpy-like expectation that the destination is a copy of the source.
-Furthermore, their uses do initialize the involved variables as needed by futher uses in the caller."
--call_properties+={"macro(^(__)?(raw_)?copy_from_(paddr|guest|compat)(_offset)?$)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
--call_properties+={"macro(^(__)?copy_to_(guest|compat)(_offset)?$)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
--doc_end
-
--doc_begin="Functions generated by build_atomic_read cannot be considered pure
-since the input pointer is volatile, but they do not produce any persistent side
-effect."
--call_properties+={"^read_u(8|16|32|64|int)_atomic.*$", {noeffect}}
--doc_end
-
--doc_begin="Functions generated by TYPE_SAFE are const."
--call_properties+={"^(mfn|gfn|pfn)_x\\(.*$",{const}}
--call_properties+={"^_(mfn|gfn|pfn)\\(.*$",{const}}
--doc_end
diff --git a/automation/eclair_analysis/prepare.sh b/automation/eclair_analysis/prepare.sh
index fe9d16e48e..47b2a2f32a 100755
--- a/automation/eclair_analysis/prepare.sh
+++ b/automation/eclair_analysis/prepare.sh
@@ -43,4 +43,6 @@ fi
     make -f "${script_dir}/Makefile.prepare" prepare
     # Translate the /* SAF-n-safe */ comments into ECLAIR CBTs
     scripts/xen-analysis.py --run-eclair --no-build --no-clean
+    # Translate function-properties.json into ECLAIR properties
+    python3 ${script_dir}/propertyparser.py
 )
diff --git a/automation/eclair_analysis/propertyparser.py b/automation/eclair_analysis/propertyparser.py
new file mode 100644
index 0000000000..0d02f505a6
--- /dev/null
+++ b/automation/eclair_analysis/propertyparser.py
@@ -0,0 +1,37 @@
+import json
+import os
+
+script_dir = os.path.dirname(__file__)
+properties_path = os.path.join(script_dir, "../../docs/function_macro_properties.json")
+output_path   = os.path.join(script_dir, "ECLAIR/call_properties.ecl")
+
+with open(properties_path) as fp:
+    properties = json.load(fp)['content']
+
+ecl = open(output_path, 'w')
+
+for record in properties:
+
+    string = "-call_properties+={\""
+    if record['type'] == "function":
+        string += f"{record['value']}\", {{".replace("\\", "\\\\")
+    else:
+        string += f"{record['type']}({record['value']})\", {{".replace("\\", "\\\\")
+
+    i=0
+    for prop in record['properties'].items():
+        if prop[0] == 'attribute':
+            string += prop[1]
+            i+=1
+        else:
+            string += f"\"{prop[0]}({prop[1]})\""
+            i+=1
+
+        if i<len(record['properties']):
+            string += ", "
+        else:
+            string +="}}\n"
+
+    ecl.write(string)
+
+ecl.close()
diff --git a/docs/function_macro_properties.json b/docs/function_macro_properties.json
new file mode 100644
index 0000000000..74058297b5
--- /dev/null
+++ b/docs/function_macro_properties.json
@@ -0,0 +1,841 @@
+{
+   "version": "1.0",
+   "content": [
+      {
+         "type": "function",
+         "value": "^printk.*$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^debugtrace_printk.*$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^panic.*$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^domain_crash$",
+         "properties":{
+            "pointee_write": "2..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^(g?d|mm_)?printk$",
+         "properties":{
+            "pointee_write": "2..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^guest_bug_on_failed$",
+         "properties":{
+            "pointee_write": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^spin_lock_init_prof$",
+         "properties":{
+            "pointee_write": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^sched_test_func$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^dev_(info|warn)$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^PAGING_DEBUG$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^ACPI_(WARNING|ERROR|INFO)$",
+         "properties":{
+            "pointee_write": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^fdt_get_property_by_offset_.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^read_atomic_size.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^device_tree_get_reg.*$",
+         "properties":{
+            "pointee_write": "4..=always",
+            "pointee_read": "4..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^dt_get_range.*$",
+         "properties":{
+            "pointee_write": "3..=always",
+            "pointee_read": "3..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^parse_static_mem_prop.*$",
+         "properties":{
+            "pointee_write": "2..=always",
+            "pointee_read": "2..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_ttbr_and_gran_64bit.*$",
+         "properties":{
+            "pointee_write": "1..2=always",
+            "pointee_read": "1..2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^hvm_emulate_init_once.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^__vmread.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^hvm_pci_decode_addr.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^vpci_mmcfg_decode_addr.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^x86emul_decode.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^unmap_grant_ref.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^arm_smmu_cmdq_build_cmd.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^pci_size_mem_bar.*$",
+         "properties":{
+            "pointee_write": "4=always",
+            "pointee_read": "4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^_hvm_read_entry.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^hvm_map_guest_frame_rw.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^guest_cpuid.*$",
+         "properties":{
+            "pointee_write": "4=always",
+            "pointee_read": "4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^epte_get_entry_emt.*$",
+         "properties":{
+            "pointee_write": "5=always",
+            "pointee_read": "5=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^mcheck_mca_logout.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^set_field_in_reg_u32.*$",
+         "properties":{
+            "pointee_write": "5=always",
+            "pointee_read": "5=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^alloc_affinity_masks.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^xasprintf.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^find_non_smt.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^call_rcu.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^getdomaininfo.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^MAPPING_(INSERT|SEARCH)\\(.*$",
+         "properties":{
+            "pointee_write": "2..=always",
+            "pointee_read": "2..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^FormatDec.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^FormatHex.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^p2m_get_ioreq_server.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^elf_memset_unchecked.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^set_iommu_pte_present.*$",
+         "properties":{
+            "pointee_write": "7=always",
+            "pointee_read": "7=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^clear_iommu_pte_present.*$",
+         "properties":{
+            "pointee_write": "4=always",
+            "pointee_read": "4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^vcpu_runstate_get.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^va_start.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^sgi_target_init.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_hw_residencies.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^x86_cpu_policy_to_featureset.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^simple_strtou?ll?\\(.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^msi_compose_msg.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^print_tainted.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_hvm_registers.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^x86_insn_modrm.*$",
+         "properties":{
+            "pointee_write": "2..3=always",
+            "pointee_read": "2..3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^cpuid_count_leaf.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^rcu_lock_remote_domain_by_id.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^cpuid_count.*$",
+         "properties":{
+            "pointee_write": "3..=always",
+            "pointee_read": "3..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^.*efi_boot_mem_unused.*$",
+         "properties":{
+            "pointee_write": "1..=always",
+            "pointee_read": "1..=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^collect_time_info.*$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^setup_xstate_comp.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^map_domain_gfn.*$",
+         "properties":{
+            "pointee_read": "5=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^fdt_getprop.*$",
+         "properties":{
+            "pointee_read": "4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^fdt_get_name.*$",
+         "properties":{
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^fdt_get_property.*$",
+         "properties":{
+            "pointee_read": "4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^pci_get_host_bridge_segment.*$",
+         "properties":{
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^dt_get_property.*$",
+         "properties":{
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^dt_property_read_u32.*$",
+         "properties":{
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^dt_device_get_paddr.*$",
+         "properties":{
+            "pointee_read": "3..4=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_evtchn_dt_property.*$",
+         "properties":{
+            "pointee_write": "2..3=maybe",
+            "pointee_read": "2..3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^setup_chosen_node.*$",
+         "properties":{
+            "pointee_write": "2..3=maybe",
+            "pointee_read": "2..3=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^queue_remove_raw.*$",
+         "properties":{
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^memset$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^va_start$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^memcmp$",
+         "properties":{
+            "pointee_write": "1..2=never",
+            "taken": ""
+         }
+      },
+      {
+         "type": "macro",
+         "value": "^memcpy$",
+         "properties":{
+            "pointee_write": "1=always&&2..=never",
+            "pointee_read": "1=never&&2..=always",
+            "taken": ""
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_cpu_info.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^pdx_to_pfn.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^is_pci_passthrough_enabled.*$",
+         "properties":{
+            "attribute": "const"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_cycles.*$",
+         "properties":{
+            "attribute": "noeffect"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^msi_gflags.*$",
+         "properties":{
+            "attribute": "const"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^hvm_save_size.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^cpu_has.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^boot_cpu_has.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^get_cpu_info.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^put_pte_flags.*$",
+         "properties":{
+            "attribute": "const"
+         }
+      },
+      {
+         "type": "function",
+         "value": "^is_pv_cpu.*$",
+         "properties":{
+            "attribute": "pure"
+         }
+      },
+      {
+         "description": "Property inferred as a consequence of the semantics of device_tree_get_reg",
+         "type": "function",
+         "value": "^acquire_static_memory_bank.*$",
+         "properties":{
+            "pointee_write": "4..=always",
+            "pointee_read": "4..=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "Property inferred as a consequence of the semantics of dt_set_cell",
+         "type": "function",
+         "value": "^set_interrupt.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description":"Property inferred as a consequence of the semantics of __p2m_get_mem_access",
+         "type": "function",
+         "value": "^p2m_get_mem_access.*$",
+         "properties":{
+            "pointee_write": "3=always",
+            "pointee_read": "3=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "This function has alternative definitions with props {write=always,read=never} and {write=never,read=never}",
+         "type": "function",
+         "value": "^alloc_cpumask_var.*$",
+         "properties":{
+            "pointee_write": "1=maybe",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "Property inferred as a consequence of the semantics of alloc_cpumask_var",
+         "type": "function",
+         "value": "^xenctl_bitmap_to_cpumask.*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "The call to bitmap_and causes the pointee of dstp to be always written",
+         "type": "function",
+         "value": "^cpumask_(and|andnot|clear|copy|complement).*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "The call to bitmap_and causes the pointee of dstp to be always written",
+         "type": "function",
+         "value": "^bitmap_(andnot|complement|fill).*$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "The .*copy_(to|from).* helpers all have a memcpy-like expectation that the destination is a copy of the source. Furthermore, their uses do initialize the involved variables as needed by futher uses in the caller.",
+         "type": "macro",
+         "value": "^(__)?(raw_)?copy_from_(paddr|guest|compat)(_offset)?$",
+         "properties":{
+            "pointee_write": "1=always",
+            "pointee_read": "1=never",
+            "taken": ""
+         }
+      },
+      {
+         "description":"The .*copy_(to|from).* helpers all have a memcpy-like expectation that the destination is a copy of the source. Furthermore, their uses do initialize the involved variables as needed by futher uses in the caller.",
+         "type": "macro",
+         "value": "^(__)?copy_to_(guest|compat)(_offset)?$",
+         "properties":{
+            "pointee_write": "2=always",
+            "pointee_read": "2=never",
+            "taken": ""
+         }
+      },
+      {
+         "description": "Functions generated by build_atomic_read cannot be considered pure since the input pointer is volatile, but they do not produce any persistent side effect.",
+         "type": "function",
+         "value": "^read_u(8|16|32|64|int)_atomic.*$",
+         "properties":{
+            "attribute": "noeffect"
+         }
+      },
+      {
+         "description": "Functions generated by TYPE_SAFE are const.",
+         "type": "function",
+         "value": "^(mfn|gfn|pfn)_x\\(.*$",
+         "properties":{
+            "attribute": "const"
+         }
+      },
+      {
+         "description": "Functions generated by TYPE_SAFE are const.",
+         "type": "function",
+         "value": "^_(mfn|gfn|pfn)\\(.*$",
+         "properties":{
+            "attribute": "const"
+         }
+      }
+   ]
+}
diff --git a/docs/function_macro_properties.rst b/docs/function_macro_properties.rst
new file mode 100644
index 0000000000..ea6fb5cf1f
--- /dev/null
+++ b/docs/function_macro_properties.rst
@@ -0,0 +1,58 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Properties list for Xen
+=======================
+
+Some functions and macros are found to have properties relevant to
+the Xen codebase. For this reason, the file docs/properties.json
+contains all the needed properties.
+
+Here is an example of the properties.json file::
+
+  {
+     "version": "1.0",
+     "content": [
+        {
+           "description": ""
+           "type": "function",       // required
+           "value:": "^printk*.$",   // required
+           "properties":{
+              "pointee_write": "1..2=never",
+              "pointee_read": "",
+              "taken": ""
+              "attribute": ""
+           }
+        }
+     ]
+  }
+
+Here is an explanation of the fields inside an object of the "content" array:
+
+ - description: a brief description of why the properties apply
+ - type: this is the kind of the element called: it may be either ``macro`` or ``function``
+ - value: must be a regex, starting with ^ and ending with $ and matching function fully
+   qualified name or macro name.
+ - properties: a list of properties applied to said function.
+   Possible values are:
+
+    - pointee_write: indicate the write use for call arguments that correspond to
+      parameters whose pointee types are non-const
+    - pointee_read: indicate the read use for call arguments that correspond to
+      parameters whose pointee types are non-const
+    - taken: indicates that the specified address arguments may be stored in objects
+      that persist after the function has ceased to exist (excluding the returned value);
+      address arguments not listed are never taken
+    - attribute: attributes a function may have. Possible values are pure, const and noeffect.
+
+   pointee_read and pointee_write use a specific kind of argument, structured as pointee_arg=rw:
+
+    - pointee_arg: argument index for callee. Index 0 refers to the return value,
+      the indices of the arguments start from 1. It can be either a single value or a range.
+    - rw: a value that's either always, maybe or never
+
+       - always: for pointee_read: argument pointee is expected to be fully read in the function body,
+         for pointee_write: argument pointee is fully initialized at function exit
+       - maybe: for pointee_read: argument pointee may be expected to be read in the function body,
+         for pointee_write: argument pointee may be written by function body
+       - never: for pointee_read: argument pointee is not expected to be read in the function body,
+         for pointee_write: argument pointee is never written by function body
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:21:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:21:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675099.1050300 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvMH-00029k-1f; Fri, 02 Feb 2024 15:21:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675099.1050300; Fri, 02 Feb 2024 15:21:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvMG-00029d-VK; Fri, 02 Feb 2024 15:21:36 +0000
Received: by outflank-mailman (input) for mailman id 675099;
 Fri, 02 Feb 2024 15:21:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYVL=JL=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rVvMF-00029T-3I
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 15:21:35 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bfd3d1d2-c1de-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 16:21:34 +0100 (CET)
Received: from [192.168.1.9] (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 378E04EE0737;
 Fri,  2 Feb 2024 16:21:33 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfd3d1d2-c1de-11ee-8a43-1f161083a0e0
Message-ID: <117db3f7-698b-47d5-905f-5346a21f0b54@bugseng.com>
Date: Fri, 2 Feb 2024 16:21:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 0/4] address violation of MISRA C:2012 Rule 13.1
Content-Language: en-US
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>, Doug Goldstein <cardoe@cardoe.com>
References: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
From: Simone Ballarin <simone.ballarin@bugseng.com>
Organization: BUGSENG
In-Reply-To: <cover.1706700228.git.maria.celeste.cesario@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 02/02/24 10:37, Simone Ballarin wrote:
> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
> 
> The Xen sources contain violations of MISRA C:2012 Rule 13.1 whose headline states:
> "Initializer lists shall not contain persistent side effects".
> 
> The file properties.json containing function and macro properties is introduced, as
> stated in v2 discussion. Some functions and macros are found to have properties that
> can be exploited by static analyzers. For this reason, the file docs/properties.json
> contains all the needed properties. A description of the json file is documented in
> docs/properties.rst.
> 
> Some persistent effects have been moved outside initializer lists to address violations
> of Rule 13.1.
> 
> Link to the discussion: https://lore.kernel.org/all/cover.1700844359.git.simone.ballarin@bugseng.com/T/#u
> 
> 
> Changes in v3:
> - change prefix from xen to xen/ns16550
> - add assignment of rc in xen/ns16550
> - use rc as controlling expression in the following if-statement
> - change commit prefix from xen/arm to xen
> - specify where saf-3-safe comments are applied in guestcopy.c
> - reword saf comments text
> 
> Maria Celeste Cesario (1):
>    eclair: add and manage properties
> 
> Simone Ballarin (3):
>    xen: add SAF deviation for debugging and logging effects
>    xen/ns16550: address violations of MISRA C:2012 Rule 13.1
>    xen/x86: address violations of MISRA C:2012 Rule 13.1
> 
>   .../eclair_analysis/ECLAIR/analysis.ecl       |   1 +
>   automation/eclair_analysis/prepare.sh         |   2 +
>   docs/misra/safe.json                          |  16 +
>   docs/properties.json                          | 841 ++++++++++++++++++
>   docs/properties.rst                           |  58 ++
>   xen/arch/arm/device.c                         |   1 +
>   xen/arch/arm/guestcopy.c                      |  16 +-
>   xen/arch/x86/hvm/hvm.c                        |   1 +
>   xen/arch/x86/io_apic.c                        |   9 +-
>   xen/arch/x86/mpparse.c                        |   3 +-
>   xen/arch/x86/setup.c                          |   3 +-
>   xen/common/sched/core.c                       |   3 +
>   xen/drivers/char/ns16550.c                    |   4 +-
>   13 files changed, 948 insertions(+), 10 deletions(-)
>   create mode 100644 docs/properties.json
>   create mode 100644 docs/properties.rst
> 

Sorry, patch "eclair: add and manage properties" is incomplete,
please ignore this series: v4 has already been submitted.

-- 
Simone Ballarin, M.Sc.

Field Application Engineer, BUGSENG (https://bugseng.com)



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:21:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:21:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675100.1050310 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvMR-0002RA-BG; Fri, 02 Feb 2024 15:21:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675100.1050310; Fri, 02 Feb 2024 15:21:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvMR-0002R1-70; Fri, 02 Feb 2024 15:21:47 +0000
Received: by outflank-mailman (input) for mailman id 675100;
 Fri, 02 Feb 2024 15:21:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVvMP-0002QK-U4; Fri, 02 Feb 2024 15:21:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVvMP-0002rA-PF; Fri, 02 Feb 2024 15:21:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVvMP-0000q4-D9; Fri, 02 Feb 2024 15:21:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVvMP-0000rW-CJ; Fri, 02 Feb 2024 15:21:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ECybsxs5r0HI915GtIUYT2rHMe3toA2w6a2tendvfmY=; b=2/UsaozG9LSpuEDzMNb2T8B+xE
	Vv0M2IbDDtxeyd/3KblkKHwE60ZfIBKyflInSThFyI9T6uxS41ZWzDpSUND2oa22oX7Pm7H0pqvVF
	84gsCegEJg/MEwFR8E5OKyzYj0CcIRhdF4q3nakpV3UZtZbGaURoig5Ny1M98OZrrGqE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184554-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184554: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt-qcow2:host-ping-check-xen:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=5c24e4e9e70822cf49955fc8174bc5efaa93d17f
X-Osstest-Versions-That:
    linux=6764c317b6bb91bd806ef79adf6d9c0e428b191e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 02 Feb 2024 15:21:45 +0000

flight 184554 linux-linus real [real]
flight 184565 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184554/
http://logs.test-lab.xenproject.org/osstest/logs/184565/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-qcow2 10 host-ping-check-xen fail pass in 184565-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail in 184565 like 184545
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check fail in 184565 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184545
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184545
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184545
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184545
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184545
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184545
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184545
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                5c24e4e9e70822cf49955fc8174bc5efaa93d17f
baseline version:
 linux                6764c317b6bb91bd806ef79adf6d9c0e428b191e

Last test of basis   184545  2024-01-31 22:37:40 Z    1 days
Testing same since   184554  2024-02-01 18:42:19 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adam Goldman <adamg@pobox.com>
  Benjamin Tissoires <bentiss@kernel.org>
  Damian Muszynski <damian.muszynski@intel.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Gaurav Jain <gaurav.jain@nxp.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Jason Gerecke <jason.gerecke@wacom.com>
  Jiri Kosina <jkosina@suse.com>
  Johan Hovold <johan+linaro@kernel.org>
  Kai-Heng Feng <kai.heng.feng@canonical.com>
  Kunwu Chan <chentao@kylinos.cn>
  Linus Torvalds <torvalds@linux-foundation.org>
  Mark Brown <broonie@kernel.org>
  Martin Blumenstingl <martin.blumenstingl@googlemail.com>
  Naresh Solanki <naresh.solanki@9elements.com>
  Ondrej Mosnacek <omosnace@redhat.com>
  Patrick Rudolph <patrick.rudolph@9elements.com>
  Paul Moore <paul@paul-moore.com>
  Romain Naour <romain.naour@skf.com>
  Su Hui <suhui@nfschina.com>
  Takashi Sakamoto <o-takashi@sakamocchi.jp>
  Wolfram Sang <wsa+renesas@sang-engineering.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               fail    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   6764c317b6bb..5c24e4e9e708  5c24e4e9e70822cf49955fc8174bc5efaa93d17f -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:28:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:28:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675114.1050321 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvSf-0003lr-2O; Fri, 02 Feb 2024 15:28:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675114.1050321; Fri, 02 Feb 2024 15:28:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvSe-0003lk-Tz; Fri, 02 Feb 2024 15:28:12 +0000
Received: by outflank-mailman (input) for mailman id 675114;
 Fri, 02 Feb 2024 15:28:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=PMC9=JL=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rVvSd-0003la-Jp
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 15:28:11 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ab393269-c1df-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 16:28:09 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 24A2B1F78B;
 Fri,  2 Feb 2024 15:28:08 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id F1C2A139AB;
 Fri,  2 Feb 2024 15:28:07 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id hXY9OQcKvWUCMQAAD6G6ig
 (envelope-from <jgross@suse.com>); Fri, 02 Feb 2024 15:28:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab393269-c1df-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1706887688; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=pjIjVlaZSusmlRZWUjxn8NCoTe4lVPMeSnznLpTIgYw=;
	b=a/QM3GbohoveOW4rlud0d+p/9nxeKImMlocVJ2n2NaLi6ibtwUzA4jJxeGs2aWS6A9wL91
	hvFBxUDJjaQa7NJuVpIEEmlfKtjtDBpdHY6ekfacc8mYlpxwXC5Y/M3rqhRJzJeBYQVl7S
	5ePItdKzgMlLMnm9FQgdmpJVtH93cr8=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1706887688; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=pjIjVlaZSusmlRZWUjxn8NCoTe4lVPMeSnznLpTIgYw=;
	b=a/QM3GbohoveOW4rlud0d+p/9nxeKImMlocVJ2n2NaLi6ibtwUzA4jJxeGs2aWS6A9wL91
	hvFBxUDJjaQa7NJuVpIEEmlfKtjtDBpdHY6ekfacc8mYlpxwXC5Y/M3rqhRJzJeBYQVl7S
	5ePItdKzgMlLMnm9FQgdmpJVtH93cr8=
Message-ID: <d92da5e1-3d96-4073-bf67-7d891aec6411@suse.com>
Date: Fri, 2 Feb 2024 16:28:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 17/33] tools/xl: support new 9pfs backend xen-9pfsd
Content-Language: en-US
From: Juergen Gross <jgross@suse.com>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Jason Andryuk <jandryuk@gmail.com>
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-18-jgross@suse.com>
 <43cb6d28-34da-47b9-b61f-0be8298f9d8b@perard>
 <3ce7fa13-fb64-438d-ba7d-455e2b42854b@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <3ce7fa13-fb64-438d-ba7d-455e2b42854b@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------dzdi0lEtMEVxIOdlW40kPljk"
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b="a/QM3Gbo"
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-3.45 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 TO_DN_SOME(0.00)[];
	 DWL_DNSWL_BLOCKED(0.00)[suse.com:dkim];
	 HAS_ATTACHMENT(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 MID_RHS_MATCH_FROM(0.00)[];
	 BAYES_HAM(-0.05)[60.62%];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[lists.xenproject.org,xen.org,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: -3.45
X-Rspamd-Queue-Id: 24A2B1F78B
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------dzdi0lEtMEVxIOdlW40kPljk
Content-Type: multipart/mixed; boundary="------------LeHDCecJd2s0Zt4QegX5h63U";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Jason Andryuk <jandryuk@gmail.com>
Message-ID: <d92da5e1-3d96-4073-bf67-7d891aec6411@suse.com>
Subject: Re: [PATCH v3 17/33] tools/xl: support new 9pfs backend xen-9pfsd
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-18-jgross@suse.com>
 <43cb6d28-34da-47b9-b61f-0be8298f9d8b@perard>
 <3ce7fa13-fb64-438d-ba7d-455e2b42854b@suse.com>
In-Reply-To: <3ce7fa13-fb64-438d-ba7d-455e2b42854b@suse.com>

--------------LeHDCecJd2s0Zt4QegX5h63U
Content-Type: multipart/mixed; boundary="------------N4xNK2PE9t7wlbCT67WqhYMG"

--------------N4xNK2PE9t7wlbCT67WqhYMG
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMzEuMDEuMjQgMTY6MjAsIErDvHJnZW4gR3Jvw58gd3JvdGU6DQo+IE9uIDE1LjAxLjI0
IDE2OjE0LCBBbnRob255IFBFUkFSRCB3cm90ZToNCj4+IE9uIFRodSwgSmFuIDA0LCAyMDI0
IGF0IDEwOjAwOjM5QU0gKzAxMDAsIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+Pj4gQEAgLTIy
NDIsNiArMjI1NiwyOCBAQCB2b2lkIHBhcnNlX2NvbmZpZ19kYXRhKGNvbnN0IGNoYXIgKmNv
bmZpZ19zb3VyY2UsDQo+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgbGlieGxfc3Ry
aW5nX2xpc3RfZGlzcG9zZSgmcGFpcnMpOw0KPj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
IGlmIChwOS0+dHlwZSA9PSBMSUJYTF9QOV9UWVBFX1VOS05PV04pIHsNCj4+PiArwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHA5LT50eXBlID0gTElCWExfUDlfVFlQRV9RRU1V
Ow0KPj4NCj4+IFRoZSBkZWZhdWx0aW5nIGlzIG5vcm1hbGx5IGRvbmUgaW4gbGlieGwsIHNv
IHRoYXQgaXQgd29ya3MgZm9yIGFsbCB1c2Vycw0KPj4gb2YgbGlieGwuIENhbiB0aGlzIGJl
IGRvbmUgaW5zdGVhZCBpbiBsaWJ4bD8gSG9wZWZ1bGx5LCBpdCdzIGVub3VnaCB0bw0KPj4g
ZG8gaXQgaW4gbGlieGxfX2RldmljZV9wOV9zZXRkZWZhdWx0KCkuDQo+Pg0KPj4gU2FtZSBx
dWVzdGlvbiBmb3IgdGhlIGZvbGxvd3VwIGNoZWNrcyBhbmQgZGVmYXVsdCB2YWx1ZXMuDQo+
IA0KPiBJJ2xsIGxvb2sgaW50byBpdC4NCg0KVHVybnMgb3V0IHRoaXMgd291bGQgbmVlZCBz
b21lIGhhY2ssIGFzIEkgaGF2ZSBubyBhY2Nlc3MgdG8gdGhlIG5hbWUgb2YNCnRoZSBkb21h
aW4gYmVpbmcgY3JlYXRlZCBpbiBsaWJ4bF9fZGV2aWNlX3A5X3NldGRlZmF1bHQoKS4gQW5k
IHRoaXMgaXMgbmVlZGVkDQpmb3Igc2V0dGluZyB0aGUgcGF0aCBkZWZhdWx0IHZhbHVlLg0K
DQpEbyB5b3UgaGF2ZSBhbnkgaWRlYSBob3cgdG8gcmVzb2x2ZSB0aGlzIGlzc3VlLCBvciBh
cmUgeW91IGZpbmUgdG8ga2VlcCB0aGUNCnBhdGNoIGFzIGlzPw0KDQoNCkp1ZXJnZW4NCg0K

--------------N4xNK2PE9t7wlbCT67WqhYMG
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------N4xNK2PE9t7wlbCT67WqhYMG--

--------------LeHDCecJd2s0Zt4QegX5h63U--

--------------dzdi0lEtMEVxIOdlW40kPljk
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmW9CgcFAwAAAAAACgkQsN6d1ii/Ey+9
HQgAhp54WX08lr8qD/hgdqCek/eq/CZUXeQjlrLZSjHUOgxTAi8WHc5Vv4sHhdMZIYBaBxoKcdR3
dTkK6SU8gar82E/N2+ypUfeC6AKAwEGr0Kz+XmwwocL8vlmVXk16imgZaN6XMQ4LEGy8mnF8iqP8
wOZQ0wBLaUHgQU9BIhlssm8OeX1hVf1Ikni2v5yj4eyaOl1nUo/qpJkAD11PiWkayYIqgpNvauPf
MSYgH9gWvNBsAuqTF9sDZ5oZTrcW3o9rBKy1k09lB1/D1HPEd3m1lFwnBa0QRYSZsSsoOMRayPZ6
OUFxSI5xAX3jQU+2BBXmhqukWowlR/E6gndssQ17gQ==
=yAMN
-----END PGP SIGNATURE-----

--------------dzdi0lEtMEVxIOdlW40kPljk--


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:38:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:38:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675120.1050329 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvcT-0005uu-28; Fri, 02 Feb 2024 15:38:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675120.1050329; Fri, 02 Feb 2024 15:38:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvcS-0005un-Vl; Fri, 02 Feb 2024 15:38:20 +0000
Received: by outflank-mailman (input) for mailman id 675120;
 Fri, 02 Feb 2024 15:38:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WgAc=JL=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVvcS-0005uf-8k
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 15:38:20 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 17045854-c1e1-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 16:38:19 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a36597a3104so309152966b.2
 for <xen-devel@lists.xenproject.org>; Fri, 02 Feb 2024 07:38:19 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 tk12-20020a170907c28c00b00a30e1f233b4sm981843ejc.128.2024.02.02.07.38.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 02 Feb 2024 07:38:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17045854-c1e1-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706888298; x=1707493098; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=hpMzOIqZKWVm1pXLHi8lKIHS6AP3wGi+MQkb7Zs99jY=;
        b=kZOrhHHB3I6hkbaymuFvYbjjRdX/ubra9jndzYT1fqPjhhkz2fThHd4/hvQqBmh+b5
         4w0YiGzsaNEmkgV4bY2bWQwzh0z1/R+9PRwyUFWmY0BUJejw74qvmFz1jwVRvxIkrG3a
         U6beDf22yNgUADvZtHMcGXjHGgHwZZAWjzGtU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706888298; x=1707493098;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=hpMzOIqZKWVm1pXLHi8lKIHS6AP3wGi+MQkb7Zs99jY=;
        b=J8Pb6llKC3xbbO3WG2VaOI0bjrbcM0BQweQCPWnQLEnAVNv9J+4d0xgD1B37TB5NlP
         dfaoedHUlD5p/xEXFb0tcCsZuveK6FmqenK71tZLmfa8AWOzFd1rDowshPW8PI+9++kk
         CGL/mX5tOC8kk/wRKyGHfqns2lzHgTuBCu8PTwWWMuLriN1vHJZYlJn7mJuDMLrzuARZ
         qM3A2or6NkKSuiTfZmRzGLcOv2tBJ/pHdqfGLMMvp0KkRTfSQPGM+sjGFlqsIPqb3GT+
         C0Bbb5bzIJRvlfElqrjvDVrj25qOi9ZeOdsEmw910QNpF9/4UGd9T78qDMoO9psY1x3f
         lyQw==
X-Gm-Message-State: AOJu0Ywz171418A4ovhPBi3jnGAAEVxjwC8Suyrv7+l++L7mrvf1daM6
	DS8uiu/kFCm0a2sisrLVlZN9BtdC7iew1+NhIT5aS90CCWnvXrVpScjEx8AgAkKuoRjC1GVjWJT
	O
X-Google-Smtp-Source: AGHT+IGfJf/bSI18m/zGA4x9m8BNqem86Eib2UTZhpa41sPerYxWf8XUt6RUvwl6OA1BynUD8pxhsQ==
X-Received: by 2002:a17:906:58f:b0:a36:4908:1dcc with SMTP id 15-20020a170906058f00b00a3649081dccmr1776938ejn.29.1706888298225;
        Fri, 02 Feb 2024 07:38:18 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVvOIdB/Dz2ctg2y8kd2leNIHcHpFkI8VHvTWsnsonzPoGU3MT7CSbEP9TuxIGgCAuhUUGvD2I6dT9vrHSsfHnSYtEEvYnHwCIM0e9EreYFjkoxJ+xXz9yPxdwuPcPF7JkC+urpRMlMo+jj0hxsQLlQ96XPJByMeD4kXhaxeXjvVVi1kBblU+34NhJ3p/C5rQ==
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH] xen/bitmap: Deduplicate __bitmap_weight() implementations
Date: Fri,  2 Feb 2024 15:38:16 +0000
Message-Id: <20240202153816.772594-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

We have two copies of __bitmap_weight() that differ by whether they make
hweight32() or hweight64() calls, yet we already have hweight_long() which is
the form that __bitmap_weight() wants.

Fix hweight_long() to return unsigned int like all the other hweight helpers,
and fix __bitmap_weight() to used unsigned integers.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
---
 xen/common/bitmap.c      | 21 +++------------------
 xen/include/xen/bitops.h |  2 +-
 2 files changed, 4 insertions(+), 19 deletions(-)

diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c
index c57b35f0042c..9d9ff09f3dde 100644
--- a/xen/common/bitmap.c
+++ b/xen/common/bitmap.c
@@ -186,33 +186,18 @@ int __bitmap_subset(const unsigned long *bitmap1,
 }
 EXPORT_SYMBOL(__bitmap_subset);
 
-#if BITS_PER_LONG == 32
 unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int bits)
 {
-	int k, w = 0, lim = bits/BITS_PER_LONG;
+	unsigned int k, w = 0, lim = bits / BITS_PER_LONG;
 
 	for (k = 0; k < lim; k++)
-		w += hweight32(bitmap[k]);
+		w += hweight_long(bitmap[k]);
 
 	if (bits % BITS_PER_LONG)
-		w += hweight32(bitmap[k] & BITMAP_LAST_WORD_MASK(bits));
+		w += hweight_long(bitmap[k] & BITMAP_LAST_WORD_MASK(bits));
 
 	return w;
 }
-#else
-unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int bits)
-{
-	int k, w = 0, lim = bits/BITS_PER_LONG;
-
-	for (k = 0; k < lim; k++)
-		w += hweight64(bitmap[k]);
-
-	if (bits % BITS_PER_LONG)
-		w += hweight64(bitmap[k] & BITMAP_LAST_WORD_MASK(bits));
-
-	return w;
-}
-#endif
 EXPORT_SYMBOL(__bitmap_weight);
 
 void __bitmap_set(unsigned long *map, unsigned int start, int len)
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index a88d45475c40..2cb7892bcca0 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -199,7 +199,7 @@ static inline unsigned int generic_hweight64(uint64_t w)
     return (w + (w >> 32)) & 0xFF;
 }
 
-static inline unsigned long hweight_long(unsigned long w)
+static inline unsigned int hweight_long(unsigned long w)
 {
     return sizeof(w) == 4 ? generic_hweight32(w) : generic_hweight64(w);
 }
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 15:43:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 15:43:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675123.1050340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvhj-0007uf-LB; Fri, 02 Feb 2024 15:43:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675123.1050340; Fri, 02 Feb 2024 15:43:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVvhj-0007uY-IP; Fri, 02 Feb 2024 15:43:47 +0000
Received: by outflank-mailman (input) for mailman id 675123;
 Fri, 02 Feb 2024 15:43:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=PMC9=JL=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rVvhi-0007uN-Ck
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 15:43:46 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d91520ba-c1e1-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 16:43:45 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 626811F795;
 Fri,  2 Feb 2024 15:43:44 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 344E5139AB;
 Fri,  2 Feb 2024 15:43:44 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id tHGECrANvWU0NQAAD6G6ig
 (envelope-from <jgross@suse.com>); Fri, 02 Feb 2024 15:43:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d91520ba-c1e1-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1706888624; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=HIS7Q8cV+NYLcorvsLrbiSHVDcbRqskySy2I/KGPvcM=;
	b=p3Eq9rrqa4sUFtSpc/MwnbSJ2j0przft+3FsLqZJExebsolNbs1ijzIu5RDryKKkDMybz9
	1vwlETe6lW6oLQZb+q5Qp6osWqgIJndnv2rjM1AoKHv+EB7KH/p2rFP/zu0tjiyW13Fdwx
	AFw8BqbZKA4dU+JDq5QH1QRwMxq7gYI=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1706888624; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=HIS7Q8cV+NYLcorvsLrbiSHVDcbRqskySy2I/KGPvcM=;
	b=p3Eq9rrqa4sUFtSpc/MwnbSJ2j0przft+3FsLqZJExebsolNbs1ijzIu5RDryKKkDMybz9
	1vwlETe6lW6oLQZb+q5Qp6osWqgIJndnv2rjM1AoKHv+EB7KH/p2rFP/zu0tjiyW13Fdwx
	AFw8BqbZKA4dU+JDq5QH1QRwMxq7gYI=
Message-ID: <cba89cf3-a77f-4394-ba19-42cf5a206f77@suse.com>
Date: Fri, 2 Feb 2024 16:43:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 17/33] tools/xl: support new 9pfs backend xen-9pfsd
Content-Language: en-US
From: Juergen Gross <jgross@suse.com>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Jason Andryuk <jandryuk@gmail.com>
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-18-jgross@suse.com>
 <43cb6d28-34da-47b9-b61f-0be8298f9d8b@perard>
 <3ce7fa13-fb64-438d-ba7d-455e2b42854b@suse.com>
 <d92da5e1-3d96-4073-bf67-7d891aec6411@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <d92da5e1-3d96-4073-bf67-7d891aec6411@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------fNQKDuzUFvKKQrqZ3dNqzJwR"
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=p3Eq9rrq
X-Spamd-Result: default: False [-2.25 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 TO_DN_SOME(0.00)[];
	 DWL_DNSWL_BLOCKED(0.00)[suse.com:dkim];
	 HAS_ATTACHMENT(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 MID_RHS_MATCH_FROM(0.00)[];
	 BAYES_HAM(-0.05)[59.50%];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[lists.xenproject.org,xen.org,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 626811F795
X-Spam-Level: 
X-Spam-Score: -2.25
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------fNQKDuzUFvKKQrqZ3dNqzJwR
Content-Type: multipart/mixed; boundary="------------udv00NbYVSiNj5QC2430E3ee";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Jason Andryuk <jandryuk@gmail.com>
Message-ID: <cba89cf3-a77f-4394-ba19-42cf5a206f77@suse.com>
Subject: Re: [PATCH v3 17/33] tools/xl: support new 9pfs backend xen-9pfsd
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-18-jgross@suse.com>
 <43cb6d28-34da-47b9-b61f-0be8298f9d8b@perard>
 <3ce7fa13-fb64-438d-ba7d-455e2b42854b@suse.com>
 <d92da5e1-3d96-4073-bf67-7d891aec6411@suse.com>
In-Reply-To: <d92da5e1-3d96-4073-bf67-7d891aec6411@suse.com>

--------------udv00NbYVSiNj5QC2430E3ee
Content-Type: multipart/mixed; boundary="------------xb0Qaq3CzA4Dop537C8SLoKb"

--------------xb0Qaq3CzA4Dop537C8SLoKb
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMDIuMDIuMjQgMTY6MjgsIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+IE9uIDMxLjAxLjI0
IDE2OjIwLCBKw7xyZ2VuIEdyb8OfIHdyb3RlOg0KPj4gT24gMTUuMDEuMjQgMTY6MTQsIEFu
dGhvbnkgUEVSQVJEIHdyb3RlOg0KPj4+IE9uIFRodSwgSmFuIDA0LCAyMDI0IGF0IDEwOjAw
OjM5QU0gKzAxMDAsIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+Pj4+IEBAIC0yMjQyLDYgKzIy
NTYsMjggQEAgdm9pZCBwYXJzZV9jb25maWdfZGF0YShjb25zdCBjaGFyICpjb25maWdfc291
cmNlLA0KPj4+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBsaWJ4bF9zdHJpbmdfbGlz
dF9kaXNwb3NlKCZwYWlycyk7DQo+Pj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGlmIChw
OS0+dHlwZSA9PSBMSUJYTF9QOV9UWVBFX1VOS05PV04pIHsNCj4+Pj4gK8KgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCBwOS0+dHlwZSA9IExJQlhMX1A5X1RZUEVfUUVNVTsNCj4+
Pg0KPj4+IFRoZSBkZWZhdWx0aW5nIGlzIG5vcm1hbGx5IGRvbmUgaW4gbGlieGwsIHNvIHRo
YXQgaXQgd29ya3MgZm9yIGFsbCB1c2Vycw0KPj4+IG9mIGxpYnhsLiBDYW4gdGhpcyBiZSBk
b25lIGluc3RlYWQgaW4gbGlieGw/IEhvcGVmdWxseSwgaXQncyBlbm91Z2ggdG8NCj4+PiBk
byBpdCBpbiBsaWJ4bF9fZGV2aWNlX3A5X3NldGRlZmF1bHQoKS4NCj4+Pg0KPj4+IFNhbWUg
cXVlc3Rpb24gZm9yIHRoZSBmb2xsb3d1cCBjaGVja3MgYW5kIGRlZmF1bHQgdmFsdWVzLg0K
Pj4NCj4+IEknbGwgbG9vayBpbnRvIGl0Lg0KPiANCj4gVHVybnMgb3V0IHRoaXMgd291bGQg
bmVlZCBzb21lIGhhY2ssIGFzIEkgaGF2ZSBubyBhY2Nlc3MgdG8gdGhlIG5hbWUgb2YNCj4g
dGhlIGRvbWFpbiBiZWluZyBjcmVhdGVkIGluIGxpYnhsX19kZXZpY2VfcDlfc2V0ZGVmYXVs
dCgpLiBBbmQgdGhpcyBpcyBuZWVkZWQNCj4gZm9yIHNldHRpbmcgdGhlIHBhdGggZGVmYXVs
dCB2YWx1ZS4NCj4gDQo+IERvIHlvdSBoYXZlIGFueSBpZGVhIGhvdyB0byByZXNvbHZlIHRo
aXMgaXNzdWUsIG9yIGFyZSB5b3UgZmluZSB0byBrZWVwIHRoZQ0KPiBwYXRjaCBhcyBpcz8N
Cg0KSnVzdCBmb3VuZCBhIGhvcGVmdWxseSBhY2NlcHRhYmxlIHNvbHV0aW9uOiBpZiB0aGUg
dHlwZSBpcyBYRU5fOVBGU0QgYW5kIG5vDQpwYXRoIGhhcyBiZWVuIHNwZWNpZmllZCBJIHNl
dCBpdCBpbiB4bF9wYXJzZS5jLiBUaGUgcmVzdCBvZiB0aGUgZGVmYXVsdA0Kc2V0dGluZ3Mg
YW5kIGNvbnNpc3RlbmN5IGNoZWNrcyBhcmUgZG9uZSBpbiBsaWJ4bC4NCg0KSXMgdGhpcyBv
a2F5IGZvciB5b3U/DQoNCg0KSnVlcmdlbg0K
--------------xb0Qaq3CzA4Dop537C8SLoKb
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------xb0Qaq3CzA4Dop537C8SLoKb--

--------------udv00NbYVSiNj5QC2430E3ee--

--------------fNQKDuzUFvKKQrqZ3dNqzJwR
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmW9Da8FAwAAAAAACgkQsN6d1ii/Ey8Q
Twf/dl5LI+KiNolT5sqqDDzuSdvUOc3hPq7NRfj6a9ISoF+A5GNlrHVVBESXcHENbR3piFXeQgvN
bMiqA6+dhKAqQcOVw5hk3zmje5AktD8qe4WDarmZtneWjDdF0gSe2gfb3ThRQw5O/x2klylv+UOs
phIbatkGI1dt0/cg1yYWYgTvRIE4TAdvHpWLSgYef6JF1X+4unTGN89SVQXTnSktoe06UD1riUiR
vmEu3xPvs74inevezqwuIjeHyOaqjSbfKrLI4Qrh0ZRMvocFqgaKVmKqgLsJh8hAhnFdTD6GWhiP
f90UUTWigKs6TJwKw3hxuUOPJDPIl54QE3JtTqri/Q==
=gOro
-----END PGP SIGNATURE-----

--------------fNQKDuzUFvKKQrqZ3dNqzJwR--


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 16:34:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 16:34:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675137.1050350 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVwUk-0008AX-7g; Fri, 02 Feb 2024 16:34:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675137.1050350; Fri, 02 Feb 2024 16:34:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVwUk-0008AQ-4A; Fri, 02 Feb 2024 16:34:26 +0000
Received: by outflank-mailman (input) for mailman id 675137;
 Fri, 02 Feb 2024 16:34:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=PMC9=JL=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rVwUi-0008AK-Mx
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 16:34:24 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ebc1e8af-c1e8-11ee-8a43-1f161083a0e0;
 Fri, 02 Feb 2024 17:34:23 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 28F3D1F802;
 Fri,  2 Feb 2024 16:34:22 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id E7AD913A60;
 Fri,  2 Feb 2024 16:34:21 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id +8C2No0ZvWXQQQAAD6G6ig
 (envelope-from <jgross@suse.com>); Fri, 02 Feb 2024 16:34:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebc1e8af-c1e8-11ee-8a43-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1706891662; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=PA6OYGu4R6n4SDeUZwkoSJyyE8lzK5shUya70YYhQ14=;
	b=HMeUqiI99OTHg3vyUTKGKpCSQwNxCu/8BdW0bscQA8XwbvlTNaGrOVUGkYV49D2gzklM3C
	IguGQs8e0pPuSqeLXRDFn8f3CyTQ26ShO4E78auE6sw/gBeAoyEHz9mx1BJRBhSoeqxcEk
	Z2ZQH/jeJCuUKZhU4LcDHmgsQQmwFE4=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1706891662; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=PA6OYGu4R6n4SDeUZwkoSJyyE8lzK5shUya70YYhQ14=;
	b=HMeUqiI99OTHg3vyUTKGKpCSQwNxCu/8BdW0bscQA8XwbvlTNaGrOVUGkYV49D2gzklM3C
	IguGQs8e0pPuSqeLXRDFn8f3CyTQ26ShO4E78auE6sw/gBeAoyEHz9mx1BJRBhSoeqxcEk
	Z2ZQH/jeJCuUKZhU4LcDHmgsQQmwFE4=
Message-ID: <fc36ce28-9158-4a61-9452-86569b7721ef@suse.com>
Date: Fri, 2 Feb 2024 17:34:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 21/33] tools: add 9pfs device to xenstore-stubdom
Content-Language: en-US
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, Jason Andryuk <jandryuk@gmail.com>
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-22-jgross@suse.com>
 <55c49bed-4039-449c-bb79-412ea9c2b5e2@perard>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <55c49bed-4039-449c-bb79-412ea9c2b5e2@perard>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------5vFayze0PdxLtTsxg1i30oxx"
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=HMeUqiI9
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.40 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 TO_DN_SOME(0.00)[];
	 HAS_ATTACHMENT(0.00)[];
	 MIME_BASE64_TEXT_BOGUS(1.00)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 MID_RHS_MATCH_FROM(0.00)[];
	 BAYES_HAM(-0.00)[23.14%];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[lists.xenproject.org,xen.org,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: -4.40
X-Rspamd-Queue-Id: 28F3D1F802
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------5vFayze0PdxLtTsxg1i30oxx
Content-Type: multipart/mixed; boundary="------------1UxKPinAv47F1f0AfiAeo459";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, Jason Andryuk <jandryuk@gmail.com>
Message-ID: <fc36ce28-9158-4a61-9452-86569b7721ef@suse.com>
Subject: Re: [PATCH v3 21/33] tools: add 9pfs device to xenstore-stubdom
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-22-jgross@suse.com>
 <55c49bed-4039-449c-bb79-412ea9c2b5e2@perard>
In-Reply-To: <55c49bed-4039-449c-bb79-412ea9c2b5e2@perard>

--------------1UxKPinAv47F1f0AfiAeo459
Content-Type: multipart/mixed; boundary="------------y8GvvfIikntByqRRHYPeM7LG"

--------------y8GvvfIikntByqRRHYPeM7LG
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMTUuMDEuMjQgMTY6MzEsIEFudGhvbnkgUEVSQVJEIHdyb3RlOg0KPiBPbiBUaHUsIEph
biAwNCwgMjAyNCBhdCAxMDowMDo0M0FNICswMTAwLCBKdWVyZ2VuIEdyb3NzIHdyb3RlOg0K
Pj4gQWRkIGEgOXBmcyBkZXZpY2UgdG8gWGVuc3RvcmUgc3R1YmRvbSBpbiBvcmRlciB0byBh
bGxvdyBpdCB0byBkbyBlLmcuDQo+PiBsb2dnaW5nIGludG8gYSBkb20wIGZpbGUuDQo+Pg0K
Pj4gVXNlIHRoZSBmb2xsb3dpbmcgcGFyYW1ldGVycyBmb3IgdGhlIG5ldyBkZXZpY2U6DQo+
Pg0KPj4gLSB0YWcgPSAieGVuIg0KPiANCj4gSXMgaXQgb2sgdG8gaGF2ZSBoZXJlIHRhZyAi
eGVuIiB3aGVuIHRoZSBkZWZhdWx0IHRhZyBpcyAiWGVuIiA/DQoNCkl0IGlzIG9rYXksIGJ1
dCBJIGFncmVlIGl0IHNob3VsZCBiZSAiWGVuIi4NCg0KSSdsbCBjaGFuZ2UgaXQuDQoNCj4g
DQo+PiBkaWZmIC0tZ2l0IGEvdG9vbHMvaW5jbHVkZS9saWJ4bC5oIGIvdG9vbHMvaW5jbHVk
ZS9saWJ4bC5oDQo+PiBpbmRleCA5MDdhYTBhMzMwLi4wMDY5MzI2NGY3IDEwMDY0NA0KPj4g
LS0tIGEvdG9vbHMvaW5jbHVkZS9saWJ4bC5oDQo+PiArKysgYi90b29scy9pbmNsdWRlL2xp
YnhsLmgNCj4+IEBAIC0yMDYwLDYgKzIwNjcsMTYgQEAgaW50IGxpYnhsX2NvbnNvbGVfYWRk
X3hlbnN0b3JlKGxpYnhsX2N0eCAqY3R4LCB1aW50MzJfdCBkb21pZCwgdWludDMyX3QgYmFj
a2VuZCwNCj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IGxpYnhs
X2FzeW5jb3BfaG93ICphb19ob3cpDQo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICBMSUJYTF9FWFRFUk5BTF9DQUxMRVJTX09OTFk7DQo+PiAgIA0KPj4gKy8qIGxpYnhs
X3A5X2FkZF94ZW5zdG9yZSB3cml0ZXMgdGhlIFhlbnN0b3JlIGVudHJpZXMgZm9yIGEgZG9t
YWluJ3MNCj4+ICsgKiBwcmltYXJ5IDlwZnMgZGV2aWNlIGJhc2VkIG9uIGRvbWlkLCBiYWNr
ZW5kIHR5cGUgYW5kIGRldmljZSBwYXJhbWV0ZXJzLg0KPj4gKyAqLw0KPj4gK2ludCBsaWJ4
bF9wOV9hZGRfeGVuc3RvcmUobGlieGxfY3R4ICpjdHgsIHVpbnQzMl90IGRvbWlkLCB1aW50
MzJfdCBiYWNrZW5kLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgbGlieGxfcDlf
dHlwZSB0eXBlLCBjaGFyICp0YWcsIGNoYXIgKnBhdGgsDQo+PiArICAgICAgICAgICAgICAg
ICAgICAgICAgICBjaGFyICpzZWN1cml0eV9tb2RlbCwgdW5zaWduZWQgaW50IG1heF9zcGFj
ZSwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGludCBtYXhfZmls
ZXMsIHVuc2lnbmVkIGludCBtYXhfb3Blbl9maWxlcywNCj4+ICsgICAgICAgICAgICAgICAg
ICAgICAgICAgIGJvb2wgYXV0b19kZWxldGUsIGNvbnN0IGxpYnhsX2FzeW5jb3BfaG93ICph
b19ob3cpDQo+IA0KPiBDb3VsZCB3ZSBzaW1wbHkgcGFzcyBhICJsaWJ4bF9kZXZpY2VfcDkq
IiBpbnN0ZWFkIG9mIGFsbCB0aGVzZQ0KPiBwYXJhbWV0ZXJzPyBJdCB3b3VsZCBhbHNvIG1l
YW4gdGhhdCB3ZSBjYW4gdXBkYXRlIHRoZSBsaXN0IG9mIHBhcmFtZXRlcnMNCj4gd2l0aG91
dCBoYXZpbmcgdG8gY2hhbmdlIHRoZSBmdW5jdGlvbiBwcm90b3R5cGUuDQoNCkZpbmUgd2l0
aCBtZS4NCg0KPiBUaGVzZSBmdW5jdGlvbnMgdGVuZCB0byBiZSBjYWxsZWQgImxpYnhsX2Rl
dmljZV8qX2FkZCgpIiwgaXMgaXQgcG9zc2libGUNCj4gdG8gZm9sbG93IHRoZSBzYW1lIHNj
aGVtYT8gSW4gcGFydGljdWxhciwgSSBkb24ndCBzZWUgYW55dGhpbmcgeGVuc3RvcmUNCj4g
c3BlY2lmaWMgaW4gdGhlIGZ1bmN0aW9uLg0KDQpJdCB3YXMgbWVhbnQgdG8gYmUgc2ltaWxh
ciB0byBsaWJ4bF9jb25zb2xlX2FkZF94ZW5zdG9yZSgpLCB3aGljaCBqdXN0IHdyaXRlcw0K
dGhlIFhlbnN0b3JlIGNvbnRlbnRzIG9mIHRoZSBkZXZpY2UuDQoNCkkgdGhpbmsgeW91IGFy
ZSByaWdodCB0aGF0IGxpYnhsX2RldmljZV85cGZzX2FkZCgpIGlzIGEgYmV0dGVyIG5hbWUu
DQoNCg0KSnVlcmdlbg0K
--------------y8GvvfIikntByqRRHYPeM7LG
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------y8GvvfIikntByqRRHYPeM7LG--

--------------1UxKPinAv47F1f0AfiAeo459--

--------------5vFayze0PdxLtTsxg1i30oxx
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmW9GY0FAwAAAAAACgkQsN6d1ii/Ey8e
uggAmz4AyUMNcNv7oRPcsoiyEdUYVSD4AVkv41cTDGvbiy9zomHTWOR57jzImfh1DX6pmTFTyic9
eNdc9wJE7ushlwNfFHMWEwgFwm5cRoRExLWIHobb93/8P1QhamxjFCcdi6u0pC6ZMFuj0RkZr/jC
TEc2rDayFuH+QLEphJnyCYtlbUGZK9u+l20a9rHyi8y2qNtrMeOQYvbKKlTk79EboK6dnOCwOHYA
onuTOvCSKbkeoOkX2Hxs4LRGDcjQ1EsUfUtNjfyRmmcfS3ivtvVqgm/6dsQRgB20gsMVpZ/ZLgfU
JhnDj8Y/brBRPYGPFwTnGuFGMgZ3Pl3WMTIjjBG08g==
=EjzQ
-----END PGP SIGNATURE-----

--------------5vFayze0PdxLtTsxg1i30oxx--


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 17:31:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 17:31:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675150.1050361 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVxNV-0008E2-K4; Fri, 02 Feb 2024 17:31:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675150.1050361; Fri, 02 Feb 2024 17:31:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVxNV-0008Dv-F4; Fri, 02 Feb 2024 17:31:01 +0000
Received: by outflank-mailman (input) for mailman id 675150;
 Fri, 02 Feb 2024 17:31:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=j24o=JL=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rVxNU-0008Dp-6x
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 17:31:00 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d3fbda7d-c1f0-11ee-8a44-1f161083a0e0;
 Fri, 02 Feb 2024 18:30:58 +0100 (CET)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-51121637524so3467493e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 02 Feb 2024 09:30:58 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 b6-20020ac247e6000000b005112ec0dc45sm357805lfp.228.2024.02.02.09.30.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 02 Feb 2024 09:30:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d3fbda7d-c1f0-11ee-8a44-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706895058; x=1707499858; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=zuFptEy94K8AhZ3deS/adnk3QF8mQaoyFhAN9vlssWo=;
        b=AexlbHJOXwFudoIYo5McIUS8ytreMB/G5sQzChshes1WdjDdEo65ackjoafigiwDzo
         zsFxmHCuwSp9GmJ5b8q/9bN2eDcN6/nRI18JkcI72w02zCJWqviw8T4Q1YbezhlsyO1G
         wb84jH0ffidTTsl6sRnUJOVjb9oWuMAecmTXN++ivRUthkoiYSkA5mPMq3PUxwy3ApLv
         qjtCQhR9jH9Fq7FWdHUycLGe490rWrdP4ThtK2EYTh0BQ1MzCXxlgFDSS+aPAFxNxG9L
         2o3+j0KAtHg2ExDsc93MBJtoOzhiAXFgcqCaAsRTiSGtOkYSRTNKeINwpTp5may54kso
         z8bg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706895058; x=1707499858;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=zuFptEy94K8AhZ3deS/adnk3QF8mQaoyFhAN9vlssWo=;
        b=X1Hq403L9TkLE8gqP0oT6Bfjzo/UmWFECYoapNKUuEm2lJGNoorpGIOfm8wlb2Qhvz
         eD/kBC8zaEA7bcS/sJHFokQ/Lq2JqVZq5gwpHi75XkvFcXO2sf5nD4Z5v9OO+KVoJLnR
         AQW5aQJCoRPcmbRCOlH/Tq4F2ZYgLbJBblRkmbGIgXRJ0X0OJ5ae1WSvFmC/cdbzu320
         lQk32lGITCiQjw9dyl2JrxPKkYLZkbdmoO3UZX1sOWmDmSTQdfiSEraKalR77SrgAn9i
         vjDFJ7khv6v1pb8O9kNNjxbXpibhllPBeev3/l7mY4bREvDtlvwtLotcJEdfKi6AFYJD
         5c6Q==
X-Gm-Message-State: AOJu0YxXm8DTXFRlID1xa0lBXI2Z3LBXdS1gBQGa6OM3NF5rOJ6TzXWn
	k7NLTWuqIZnCE6NtgYdNh7hS5VhOCmrHKG+QsCOlpOPfnQZVFqoZ
X-Google-Smtp-Source: AGHT+IHvDk/T/kmZPYHcq9Y5AU1oA+UiXI/qceQM2mz78TI9vBKaPWmggW6s3Qa8/lzLw/89zI/joA==
X-Received: by 2002:a05:6512:31cb:b0:511:3a70:b954 with SMTP id j11-20020a05651231cb00b005113a70b954mr1366405lfe.18.1706895058002;
        Fri, 02 Feb 2024 09:30:58 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCW+baJqMd+UJJeAx1IRozm5+WDWH1cd3g1rA/R8Dqk2Zx+p0G6UnoC8aiX/LNATvQnvrMUFqexIgB203NK5KBAhBO3WlVfBQ4ldYYCUpIuRAtpeOjUnwBbstKDsZ+LheG8AL+BKffgv6tovQEtqSlX4+iZAZNdmPrLXI6Inzu9ckgtp6jWYfoZhvfzOZ/1gzxg6E9iIt+SaSvBeCrdn0MCiKfe80NBLJqbRf/Enn6UnmER18YZwmDiFNB0sSWcE+g53hCq6trIfJ4C2uDMOnkmMDIMJC6JAC++Ci/27mv+Cjv6llQcYCn0=
Message-ID: <f3858360fba14a2c5b794bdd0400cd7ab8e66f73.camel@gmail.com>
Subject: Re: [PATCH v3 31/34] xen/riscv: add minimal stuff to mm.h to build
 full Xen
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Fri, 02 Feb 2024 19:30:57 +0200
In-Reply-To: <d347c4d9-e93b-4937-8e33-e5fbbdcd6bfb@suse.com>
References: <cover.1703255175.git.oleksii.kurochko@gmail.com>
	 <4411f6af38586074b347cd6005f19f9c670faa74.1703255175.git.oleksii.kurochko@gmail.com>
	 <d347c4d9-e93b-4937-8e33-e5fbbdcd6bfb@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Tue, 2024-01-23 at 14:03 +0100, Jan Beulich wrote:
> On 22.12.2023 16:13, Oleksii Kurochko wrote:
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > Changes in V3:
> > =C2=A0- update the commit message
>=20
> ??? (yet again)
>=20
> > --- a/xen/arch/riscv/include/asm/mm.h
> > +++ b/xen/arch/riscv/include/asm/mm.h
> > @@ -3,8 +3,251 @@
> > =C2=A0#ifndef _ASM_RISCV_MM_H
> > =C2=A0#define _ASM_RISCV_MM_H
> > =C2=A0
> > +#include <public/xen.h>
> > +#include <xen/pdx.h>
> > +#include <xen/types.h>
> > +
> > +#include <asm/page.h>
> > =C2=A0#include <asm/page-bits.h>
> > =C2=A0
> > +#define paddr_to_pdx(pa)=C2=A0=C2=A0=C2=A0 mfn_to_pdx(maddr_to_mfn(pa)=
)
> > +#define gfn_to_gaddr(gfn)=C2=A0=C2=A0 pfn_to_paddr(gfn_x(gfn))
> > +#define gaddr_to_gfn(ga)=C2=A0=C2=A0=C2=A0 _gfn(paddr_to_pfn(ga))
> > +#define mfn_to_maddr(mfn)=C2=A0=C2=A0 pfn_to_paddr(mfn_x(mfn))
> > +#define maddr_to_mfn(ma)=C2=A0=C2=A0=C2=A0 _mfn(paddr_to_pfn(ma))
> > +#define vmap_to_mfn(va)=C2=A0=C2=A0=C2=A0=C2=A0
> > maddr_to_mfn(virt_to_maddr((vaddr_t)va))
> > +#define vmap_to_page(va)=C2=A0=C2=A0=C2=A0 mfn_to_page(vmap_to_mfn(va)=
)
>=20
> Everything you have above ...
>=20
> > +#define paddr_to_pdx(pa)=C2=A0=C2=A0=C2=A0 mfn_to_pdx(maddr_to_mfn(pa)=
)
> > +#define gfn_to_gaddr(gfn)=C2=A0=C2=A0 pfn_to_paddr(gfn_x(gfn))
> > +#define gaddr_to_gfn(ga)=C2=A0=C2=A0=C2=A0 _gfn(paddr_to_pfn(ga))
> > +#define mfn_to_maddr(mfn)=C2=A0=C2=A0 pfn_to_paddr(mfn_x(mfn))
> > +#define maddr_to_mfn(ma)=C2=A0=C2=A0=C2=A0 _mfn(paddr_to_pfn(ma))
> > +#define vmap_to_mfn(va)=C2=A0=C2=A0=C2=A0=C2=A0
> > maddr_to_mfn(virt_to_maddr((vaddr_t)va))
> > +#define vmap_to_page(va)=C2=A0=C2=A0=C2=A0 mfn_to_page(vmap_to_mfn(va)=
)
>=20
> ... appears a 2nd time right afterwards.
>=20
> > +#define virt_to_maddr(va) ((paddr_t)((vaddr_t)(va) & PADDR_MASK))
> > +#define maddr_to_virt(pa) ((void *)((paddr_t)(pa) |
> > DIRECTMAP_VIRT_START))
> > +
> > +/* Convert between Xen-heap virtual addresses and machine frame
> > numbers. */
> > +#define __virt_to_mfn(va) (virt_to_maddr(va) >> PAGE_SHIFT)
> > +#define __mfn_to_virt(mfn) maddr_to_virt((paddr_t)(mfn) <<
> > PAGE_SHIFT)
>=20
> These would imo better use maddr_to_mfn() and mfn_to_maddr(), rather
> than
> kind of open-coding them. The former could also use PFN_DOWN() as an
> alternative.
We can't to as __virt_to_mfn() when is used it is usually wrapped by
_mfn() which expect to have unsigned long as an argument.

>=20
> > +/* Convert between Xen-heap virtual addresses and page-info
> > structures. */
> > +static inline struct page_info *virt_to_page(const void *v)
> > +{
> > +=C2=A0=C2=A0=C2=A0 BUG();
> > +=C2=A0=C2=A0=C2=A0 return NULL;
> > +}
> > +
> > +/*
> > + * We define non-underscored wrappers for above conversion
> > functions.
> > + * These are overriden in various source files while underscored
> > version
> > + * remain intact.
> > + */
> > +#define virt_to_mfn(va)=C2=A0=C2=A0=C2=A0=C2=A0 __virt_to_mfn(va)
> > +#define mfn_to_virt(mfn)=C2=A0=C2=A0=C2=A0 __mfn_to_virt(mfn)
>=20
> Is this really still needed? Would be pretty nice if in a new port we
> could get to start cleanly right away (i.e. by not needing per-file
> overrides, but using type-safe expansions here right away).
We still need __virt_to_mfn and __mfn_to_virt as common code use them:
 * xen/common/xenoprof.c:24:#define virt_to_mfn(va)
mfn(__virt_to_mfn(va))
 * xen/include/xen/domain_page.h:59:#define domain_page_map_to_mfn(ptr)
_mfn(__virt_to_mfn((unsigned long)(ptr)))

~ Oleksii
>=20
> > +struct page_info
> > +{
> > +=C2=A0=C2=A0=C2=A0 /* Each frame can be threaded onto a doubly-linked =
list. */
> > +=C2=A0=C2=A0=C2=A0 struct page_list_entry list;
> > +
> > +=C2=A0=C2=A0=C2=A0 /* Reference count and various PGC_xxx flags and fi=
elds. */
> > +=C2=A0=C2=A0=C2=A0 unsigned long count_info;
> > +
> > +=C2=A0=C2=A0=C2=A0 /* Context-dependent fields follow... */
> > +=C2=A0=C2=A0=C2=A0 union {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Page is in use: ((count_=
info & PGC_count_mask) !=3D 0).
> > */
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* =
Type reference count and various PGT_xxx flags and
> > fields. */
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 uns=
igned long type_info;
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } inuse;
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Page is on a free list: =
((count_info & PGC_count_mask)
> > =3D=3D 0). */
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 union {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 str=
uct {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 /*
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 * Index of the first *possibly* unscrubbed page
> > in the buddy.
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 * One more bit than maximum possible order to
> > accommodate
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 * INVALID_DIRTY_IDX.
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 */
> > +#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1)
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 unsigned long first_dirty:MAX_ORDER + 1;
> > +
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 /* Do TLBs need flushing for safety before next
> > page use? */
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 bool need_tlbflush:1;
> > +
> > +#define BUDDY_NOT_SCRUBBING=C2=A0=C2=A0=C2=A0 0
> > +#define BUDDY_SCRUBBING=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1
> > +#define BUDDY_SCRUB_ABORT=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 2
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 unsigned long scrub_state:2;
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 };
> > +
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 unsigned long val;
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } f=
ree;
>=20
> Indentation is wrong (and thus misleading) for these two lines.
>=20
> > +
> > +=C2=A0=C2=A0=C2=A0 } u;
>=20
> Nit: I don't see the value of the trailing blank line inside the
> union.
>=20
> > +=C2=A0=C2=A0=C2=A0 union {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Page is in use, but not =
as a shadow. */
>=20
> I question the appicability of "shadow" here.
>=20
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* =
Owner of this page (zero if page is anonymous). */
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 str=
uct domain *domain;
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } inuse;
> > +
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Page is on a free list. =
*/
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* =
Order-size of the free chunk this page is the head
> > of. */
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 uns=
igned int order;
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } free;
> > +
> > +=C2=A0=C2=A0=C2=A0 } v;
> > +
> > +=C2=A0=C2=A0=C2=A0 union {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /*
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * Timestamp from 'TLB=
 clock', used to avoid extra safety
> > flushes.
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * Only valid for: a) =
free pages, and b) pages with zero
> > type count
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 uint32_t tlbflush_timestamp=
;
> > +=C2=A0=C2=A0=C2=A0 };
> > +=C2=A0=C2=A0=C2=A0 uint64_t pad;
> > +};
> > +
> > +#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
> > +
> > +/* PDX of the first page in the frame table. */
> > +extern unsigned long frametable_base_pdx;
>=20
> From this I conclude memory on RISC-V systems may not start at (or
> near) 0?
>=20
> > +/* Convert between machine frame numbers and page-info structures.
> > */
> > +#define
> > mfn_to_page(mfn)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 (frame_table + (mfn_to_pdx(mfn) - frametable_base_p=
dx))
> > +#define
> > page_to_mfn(pg)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 pdx_to_mfn((unsigned long)((pg) - frame_table) +
> > frametable_base_pdx)
> > +
> > +static inline void *page_to_virt(const struct page_info *pg)
> > +{
> > +=C2=A0=C2=A0=C2=A0 return mfn_to_virt(mfn_x(page_to_mfn(pg)));
> > +}
> > +
> > +/*
> > + * Common code requires get_page_type and put_page_type.
> > + * We don't care about typecounts so we just do the minimum to
> > make it
> > + * happy.
> > + */
> > +static inline int get_page_type(struct page_info *page, unsigned
> > long type)
> > +{
> > +=C2=A0=C2=A0=C2=A0 return 1;
> > +}
> > +
> > +static inline void put_page_type(struct page_info *page)
> > +{
> > +}
> > +
> > +static inline void put_page_and_type(struct page_info *page)
> > +{
> > +=C2=A0=C2=A0=C2=A0 put_page_type(page);
> > +=C2=A0=C2=A0=C2=A0 put_page(page);
> > +}
> > +
> > +/*
> > + * RISC-V does not have an M2P, but common code expects a handful
> > of
> > + * M2P-related defines and functions. Provide dummy versions of
> > these.
> > + */
> > +#define INVALID_M2P_ENTRY=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (~=
0UL)
> > +#define SHARED_M2P_ENTRY=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 (~0UL - 1UL)
> > +#define SHARED_M2P(_e)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 ((_e) =3D=3D SHARED_M2P_ENTRY)
> > +
> > +/* Xen always owns P2M on RISC-V */
> > +#define set_gpfn_from_mfn(mfn, pfn) do { (void) (mfn),
> > (void)(pfn); } while (0)
>=20
> Nit: Stray blank again after cast.
>=20
> > +#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
>=20
> What's the relation of the comment with these two #define-s?
>=20
> > +#define PDX_GROUP_SHIFT (16 + 5)
> > +
> > +static inline unsigned long domain_get_maximum_gpfn(struct domain
> > *d)
> > +{
> > +=C2=A0=C2=A0=C2=A0 BUG();
> > +=C2=A0=C2=A0=C2=A0 return 0;
> > +}
> > +
> > +static inline long arch_memory_op(int op,
> > XEN_GUEST_HANDLE_PARAM(void) arg)
> > +{
> > +=C2=A0=C2=A0=C2=A0 BUG();
> > +=C2=A0=C2=A0=C2=A0 return 0;
> > +}
> > +
> > +/*
> > + * On RISCV, all the RAM is currently direct mapped in Xen.
> > + * Hence return always true.
> > + */
> > +static inline bool arch_mfns_in_directmap(unsigned long mfn,
> > unsigned long nr)
> > +{
> > +=C2=A0=C2=A0=C2=A0 return true;
> > +}
> > +
> > +#define PG_shift(idx)=C2=A0=C2=A0 (BITS_PER_LONG - (idx))
> > +#define PG_mask(x, idx) (x ## UL << PG_shift(idx))
> > +
> > +#define PGT_none=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 PG_mask(0, 1)=C2=A0 /* no special uses of
> > this page=C2=A0=C2=A0 */
> > +#define PGT_writable_page PG_mask(1, 1)=C2=A0 /* has writable
> > mappings?=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */
> > +#define PGT_type_mask=C2=A0=C2=A0=C2=A0=C2=A0 PG_mask(1, 1)=C2=A0 /* B=
its 31 or
> > 63.=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */
> > +
> > + /* Count of uses of this frame as its current type. */
> > +#define PGT_count_width=C2=A0=C2=A0 PG_shift(2)
> > +#define PGT_count_mask=C2=A0=C2=A0=C2=A0 ((1UL<<PGT_count_width)-1)
>=20
> Nit: Style (missing blanks around binary operators). Also a few more
> times further down.
>=20
> > +/*
> > + * Page needs to be scrubbed. Since this bit can only be set on a
> > page that is
> > + * free (i.e. in PGC_state_free) we can reuse PGC_allocated bit.
> > + */
> > +#define _PGC_need_scrub=C2=A0=C2=A0 _PGC_allocated
> > +#define PGC_need_scrub=C2=A0=C2=A0=C2=A0 PGC_allocated
> > +
> > +//=C2=A0 /* Cleared when the owning guest 'frees' this page. */
>=20
> Why a commented out comment?
>=20
> > +#define _PGC_allocated=C2=A0=C2=A0=C2=A0 PG_shift(1)
> > +#define PGC_allocated=C2=A0=C2=A0=C2=A0=C2=A0 PG_mask(1, 1)
> > +=C2=A0 /* Page is Xen heap? */
> > +#define _PGC_xen_heap=C2=A0=C2=A0=C2=A0=C2=A0 PG_shift(2)
> > +#define PGC_xen_heap=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PG_mask(1, 2)
> > +/* Page is broken? */
> > +#define _PGC_broken=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PG_shift(7)
> > +#define PGC_broken=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PG_mask(1=
, 7)
> > + /* Mutually-exclusive page states: { inuse, offlining, offlined,
> > free }. */
>=20
> Can similar comments in this block please all be similarly indented
> (or not)?
>=20
> > +#define PGC_state=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PG_m=
ask(3, 9)
> > +#define PGC_state_inuse=C2=A0=C2=A0 PG_mask(0, 9)
> > +#define PGC_state_offlining PG_mask(1, 9)
> > +#define PGC_state_offlined PG_mask(2, 9)
> > +#define PGC_state_free=C2=A0=C2=A0=C2=A0 PG_mask(3, 9)
> > +// #define page_state_is(pg, st) (((pg)->count_info&PGC_state) =3D=3D
> > PGC_state_##st)
>=20
> ???
>=20
> > +/* Count of references to this frame. */
> > +#define PGC_count_width=C2=A0=C2=A0 PG_shift(9)
> > +#define PGC_count_mask=C2=A0=C2=A0=C2=A0 ((1UL<<PGC_count_width)-1)
> > +
> > +#define page_state_is(pg, st) (((pg)->count_info&PGC_state) =3D=3D
> > PGC_state_##st)
>=20
> And here it then "properly" appears?
>=20
> > +#define _PGC_extra=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PG_shift(=
10)
> > +#define PGC_extra=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PG_m=
ask(1, 10)
> > +
> > +#define is_xen_heap_page(page) ((page)->count_info & PGC_xen_heap)
> > +#define is_xen_heap_mfn(mfn) \
> > +=C2=A0=C2=A0=C2=A0 (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn=
)))
> > +
> > +#define is_xen_fixed_mfn(mfn)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 ((mfn_to_maddr(mfn) >=3D virt_to_maddr(&_start)) &&=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0 (mfn_to_maddr(mfn) <=3D virt_to_maddr((vaddr_=
t)_end - 1)))
>=20
> Why does _start need prefixing wuth & and _end prefixing with a cast?
> First and foremost both want to be consistent. And then preferably
> with as little extra clutter as possible.
>=20
> > +#define page_get_owner(_p)=C2=A0=C2=A0=C2=A0 (_p)->v.inuse.domain
> > +#define page_set_owner(_p,_d) ((_p)->v.inuse.domain =3D (_d))
> > +
> > +/* TODO: implement */
> > +#define mfn_valid(mfn) ({ (void) (mfn); 0; })
> > +
> > +#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
>=20
> This appeared further up already.
>=20
> > +#define domain_set_alloc_bitsize(d) ((void)0)
>=20
> Better ((void)(d)) ? And then ...
>=20
> > +#define domain_clamp_alloc_bitsize(d, b) (b)
>=20
> ... ((void)(d), (b)) here?
>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 18:00:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 18:00:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675155.1050370 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVxpg-00047a-MN; Fri, 02 Feb 2024 18:00:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675155.1050370; Fri, 02 Feb 2024 18:00:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVxpg-00047T-JR; Fri, 02 Feb 2024 18:00:08 +0000
Received: by outflank-mailman (input) for mailman id 675155;
 Fri, 02 Feb 2024 18:00:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WgAc=JL=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rVxpf-00047J-0P
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 18:00:07 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e53d7317-c1f4-11ee-8a44-1f161083a0e0;
 Fri, 02 Feb 2024 19:00:05 +0100 (CET)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2d08df038b6so5117581fa.2
 for <xen-devel@lists.xenproject.org>; Fri, 02 Feb 2024 10:00:05 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 d13-20020a50fb0d000000b00557d839727esm992380edq.7.2024.02.02.10.00.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 02 Feb 2024 10:00:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e53d7317-c1f4-11ee-8a44-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1706896804; x=1707501604; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=PgteywaViE777WQcL4kR53i7eXhLxOklKrqpmivgd+A=;
        b=Nq2/1kl4T2m0xmuk5ypiJtJ3OVGKKR3jGOGFsgGMAhtYd93MfmcbKUzfJmxGskfRHH
         nbUcqT5IWgpFdNMyRFbzyATGVEH9XjndHNRlEu9rWrTpMbI11LT+BgllSfury0hD2NqQ
         rKAe4pmPHUe5rvOPI8yBWBjukE4VagaipVgFU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706896804; x=1707501604;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=PgteywaViE777WQcL4kR53i7eXhLxOklKrqpmivgd+A=;
        b=HkFAEJgtVCFJ9fGmjaEMNg5LVRyTQp/Y+339RAfbExtZ90gbb/DWnVxMxCVVdqJwlO
         9cpXmvQFeuKr3BuIsfAhFmEJFaRkCfOApEDQNy9IBsUOO4y869eAYE9lztU81uGBm342
         nh7rb2DjjQlmYKPkCh0ZX0EUR3XwU4tWcep8me1136AGpPqrVpm4ejaFSZApK1B7re1Y
         lqZipiRRWtbMqgY6T2rbz/CrboibMIBTtLbN9SbqhdfYRDBLmGYMYgT4qOQvvcBMtk6E
         vgsqjSEVn1tS3p8ZtZJySipjgjUeI7j8gOHP5BijsliWctn6VOMscTxmC+q8sLtRrIIn
         va5g==
X-Gm-Message-State: AOJu0YwRzrEjgGSr5biVl34qYOyRXsN5sVziPmkau8DoYLo0Brxb+eSv
	5YWYXjCtLvMQ4v5HO/EsdeAlpwGLHZ0m7hQoSJ0bg57EyWSbWwTsV9ZuAt1t307UZMmwJYkbwJF
	Q
X-Google-Smtp-Source: AGHT+IEPE86/rtAqTDaTcPB9oCMttE8prHielYbZJ9xophVpMum+83jh1e3V2yZndcqiWtYdWMCMcA==
X-Received: by 2002:a2e:b5ae:0:b0:2cc:6210:1dce with SMTP id f14-20020a2eb5ae000000b002cc62101dcemr1628191ljn.4.1706896804656;
        Fri, 02 Feb 2024 10:00:04 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCV8eKCZlu+wLy5g98L5X9hwQ/TA/q/g3EDNtfijxYf4JzF2J/7FeRSkkpN0NWac8kzygMaFGk1lqtFewmmq/g4L2oN0kUcjwHSqzIGbwUszyXF4yJ5pFQje3rcHYo1eHiH+YmA=
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/ucode: Remove accidentally introduced tabs
Date: Fri,  2 Feb 2024 18:00:03 +0000
Message-Id: <20240202180003.1295735-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Fixes: cf7fe8b72dea ("x86/ucode: Fix stability of the raw CPU Policy rescan")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

I don't know what went wrong here.
---
 xen/arch/x86/cpu/microcode/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index 6f95f7bbe223..1c9f66ea8a0f 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -685,12 +685,12 @@ static long cf_check microcode_update_helper(void *data)
          * Disable CPUID masking if in use, to avoid having current's
          * cpu_policy affect the rescan.
          */
-	if ( ctxt_switch_masking )
+        if ( ctxt_switch_masking )
             alternative_vcall(ctxt_switch_masking, NULL);
 
         calculate_raw_cpu_policy();
 
-	if ( ctxt_switch_masking )
+        if ( ctxt_switch_masking )
             alternative_vcall(ctxt_switch_masking, current);
     }
     else

base-commit: 3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 18:04:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 18:04:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675158.1050380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVxtS-0004gs-70; Fri, 02 Feb 2024 18:04:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675158.1050380; Fri, 02 Feb 2024 18:04:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVxtS-0004gl-3W; Fri, 02 Feb 2024 18:04:02 +0000
Received: by outflank-mailman (input) for mailman id 675158;
 Fri, 02 Feb 2024 18:04:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=j24o=JL=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rVxtR-0004gf-H9
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 18:04:01 +0000
Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com
 [2a00:1450:4864:20::22b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 70fe4910-c1f5-11ee-8a44-1f161083a0e0;
 Fri, 02 Feb 2024 19:04:00 +0100 (CET)
Received: by mail-lj1-x22b.google.com with SMTP id
 38308e7fff4ca-2d07b400bb8so21920131fa.2
 for <xen-devel@lists.xenproject.org>; Fri, 02 Feb 2024 10:04:00 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 f6-20020a05651c02c600b002cd38676699sm320976ljo.66.2024.02.02.10.03.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 02 Feb 2024 10:03:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 70fe4910-c1f5-11ee-8a44-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1706897039; x=1707501839; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=gQoMHnNRwfrV1WDYdAHoQ/fdY6iq16m3Egnrg7QaF4I=;
        b=EKS3dSktXbdcO+KTcZ+T9gsmgDuE6dcxZs0sBOFaAklKN+IPTA3FG7/Kq9g/T3Hfbc
         njbtlvoJ9968S7AvcTRDNgxIIhv7D4lhnX2bzZDG+FHKTKyxWv3/f9giaGIgS6OUsV37
         UHujl2nOuquWOnJNXe7PlZlYHp8ySEW7c6UenaxoruXxYM6yhjhbnxS1lvC8iFo4G8+/
         m3nkU2gDOCXUjYISrsP3pfkOvd+Yc+DZ85NhzZREbJ4oCfSt6V4W6mje6irl3WSX41jf
         3RtqsQNe5fYijixATS9cQBLRbgSi3fKv+UrhRBJWPmZwCHqa/cAM2oVrSnkZBe+sMomK
         asUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706897039; x=1707501839;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=gQoMHnNRwfrV1WDYdAHoQ/fdY6iq16m3Egnrg7QaF4I=;
        b=ZwtHAeoyoB+Gni/xBrAkxozDOmhLTARzvoTdrjNNVfH/Y3iZ6UNLp7jidK9NKgTdpa
         2FFGZDy5PIuHovn31nWZ5u2aaas+DpFfO0uqAVQOh5zQFw8kjeIn7fXwSMDL0k+prPA1
         0rbLLu/TWaGCyhCen03yH2exq0eT6wg1sIlnskqndd5u3RRNlagic0ky59Ma1dQi20Tm
         dqb89ixfNUyMp1TKez/Y5lpfmvRv6WLBEVK8AFUwjmE93+kB61h3098xhFAIkXd795jv
         P5ZjbgbfF4/nelEH3iFqJ7YtUhmoxZseTworbzk+bGXUMXs59558dJa84dqjri+kE1yq
         3clg==
X-Gm-Message-State: AOJu0Yz+5ohkRP0MjmNIa6a3xeMkqIj09pyc+h2QKAKcV3hGOlR7iMPK
	luTeHc0rgISGzqLapoV2GITjucC0AMOga1CnEhHS8WB/90ryO3WoCMF13Tft
X-Google-Smtp-Source: AGHT+IE9l26f3+lWXxhPxw2mzeVtif1AaEHf3ekzfXdffsH9qKf35KINSLIXrZIgblGynA93ntyIaA==
X-Received: by 2002:a2e:a401:0:b0:2ce:5524:1d96 with SMTP id p1-20020a2ea401000000b002ce55241d96mr3801118ljn.51.1706897039259;
        Fri, 02 Feb 2024 10:03:59 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVoOrtM8tvvooUGcNo0U0Rqg7CTOyeHo6fqDTDv28q12SHxtQT3P+5MbAxpBHKgNXYVvTV2+emUk6LroTzp41qINuPIbFNJluR1Raa8QIkeCM7yH9ez8t/oF5BaLV2P96Q2pq16R93mbLBAFqftBTWzNJ+rZqItcCGHMRNzLKndJaq/K+WSmE3fzAJ6MmdZVwUI/chmDRmnRVEYQKlv9tTP+XMSYebbZ/0jmPNSLxYXF+ay4k/j4yLb3bUtUh+p3qyccU9O0kTwxYHTXTbqEqPk1Dx+VJuHqA2WXwGX6bkZDvhyQ6S7DvY5vY5vUK2Icf0o+9rPghDuAuqGSUMf4qXR/Ki9olhsVTmafoF0OIMBW5Vgv5XLoIrsWdyY5Pqmajx5lJpwSxavy31eY3YqLwqVZ4PIKzUNDaEi9Ou7rL1TGG8wEfCp/efTYpL8b5jwPz1Z0or4XweUvubD8dxOVmAyFBb8wsjX1Z9n0mlI+eQfh/S63gokxHY2f+YkZKRagA==
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(), ASSERT_UNREACHABLE() to xen/bug.h
Date: Fri,  2 Feb 2024 20:03:56 +0200
Message-ID: <4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Move the macros mentioned in the commit subject to their appropriate
locations.
Additionally, eliminate the dependency of xen/lib.h from xen/bug.h and
include "xen/bug.h" in files where xen/bug.h macros are utilized.

Most of the changes were made because a file requires macros from xen/bug.h,
except for some files for Arm which require definitions of BUG_OPCODE,
BUG_INSTR, BUG_FN_REG.

xen/lib.h was added to list-sort.c ( otherwise compilation errors related
to {d}printk occur during compilation of list-sort.c. ) as xen/lib.h was
removed from xen/list.h. Since nothing in xen/list.h depends on xen/lib.h
functionality and only xen/bug.h is needed.

cpufeature.h requires the inclusion of <xen/cache.h>;
otherwise, the following error will occur:
ld: common/monitor.o:/build/xen/./arch/x86/include/asm/cpufeature.h:41:
multiple definitions of `__cacheline_aligned';

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/arm/arm32/insn.c                |  3 ++-
 xen/arch/arm/arm64/cpufeature.c          |  1 +
 xen/arch/arm/arm64/insn.c                |  1 +
 xen/arch/arm/cpufeature.c                |  1 +
 xen/arch/arm/include/asm/arm32/cmpxchg.h |  1 +
 xen/arch/arm/include/asm/arm64/cmpxchg.h |  2 ++
 xen/arch/arm/include/asm/regs.h          |  2 +-
 xen/arch/arm/include/asm/vgic.h          |  1 +
 xen/arch/ppc/include/asm/time.h          |  2 +-
 xen/arch/x86/bitops.c                    |  2 +-
 xen/arch/x86/include/asm/cpufeature.h    |  1 +
 xen/arch/x86/include/asm/system.h        |  2 +-
 xen/arch/x86/include/asm/x86_64/page.h   |  2 ++
 xen/arch/x86/x86_emulate/private.h       |  1 +
 xen/common/efi/common-stub.c             |  2 +-
 xen/common/version.c                     |  1 +
 xen/include/public/hvm/save.h            |  2 +-
 xen/include/xen/bug.h                    | 19 +++++++++++++++++++
 xen/include/xen/cpumask.h                |  1 +
 xen/include/xen/device_tree.h            |  1 +
 xen/include/xen/lib.h                    | 19 -------------------
 xen/include/xen/list.h                   |  2 +-
 xen/include/xen/livepatch.h              |  2 ++
 xen/include/xen/mm.h                     |  1 +
 xen/include/xen/param.h                  |  1 +
 xen/lib/list-sort.c                      |  1 +
 xen/xsm/flask/ss/ebitmap.h               |  1 +
 27 files changed, 48 insertions(+), 27 deletions(-)

diff --git a/xen/arch/arm/arm32/insn.c b/xen/arch/arm/arm32/insn.c
index 49953a042a..2a62bb9cce 100644
--- a/xen/arch/arm/arm32/insn.c
+++ b/xen/arch/arm/arm32/insn.c
@@ -13,8 +13,9 @@
   * You should have received a copy of the GNU General Public License
   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
   */
-#include <xen/lib.h>
 #include <xen/bitops.h>
+#include <xen/bug.h>
+#include <xen/lib.h>
 #include <xen/sizes.h>
 #include <asm/insn.h>
 
diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
index b4656ff4d8..864413d9cc 100644
--- a/xen/arch/arm/arm64/cpufeature.c
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -69,6 +69,7 @@
  *   KVM guests.
  */
 
+#include <xen/bug.h>
 #include <xen/types.h>
 #include <xen/kernel.h>
 #include <asm/sysregs.h>
diff --git a/xen/arch/arm/arm64/insn.c b/xen/arch/arm/arm64/insn.c
index 22f2bdebd5..773c3749d1 100644
--- a/xen/arch/arm/arm64/insn.c
+++ b/xen/arch/arm/arm64/insn.c
@@ -18,6 +18,7 @@
  * You should have received a copy of the GNU General Public License
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
+#include <xen/bug.h>
 #include <xen/types.h>
 #include <xen/lib.h>
 #include <xen/errno.h>
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index f43d5cb338..ef77473bf8 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -5,6 +5,7 @@
  * Copyright (C) 2015 ARM Ltd.
  */
 
+#include <xen/bug.h>
 #include <xen/types.h>
 #include <xen/init.h>
 #include <xen/smp.h>
diff --git a/xen/arch/arm/include/asm/arm32/cmpxchg.h b/xen/arch/arm/include/asm/arm32/cmpxchg.h
index 37b2d64eb6..8455eb7cc3 100644
--- a/xen/arch/arm/include/asm/arm32/cmpxchg.h
+++ b/xen/arch/arm/include/asm/arm32/cmpxchg.h
@@ -1,6 +1,7 @@
 #ifndef __ASM_ARM32_CMPXCHG_H
 #define __ASM_ARM32_CMPXCHG_H
 
+#include <xen/bug.h>
 #include <xen/prefetch.h>
 
 extern void __bad_xchg(volatile void *ptr, int size);
diff --git a/xen/arch/arm/include/asm/arm64/cmpxchg.h b/xen/arch/arm/include/asm/arm64/cmpxchg.h
index 031fa6d92a..f160e8e7bc 100644
--- a/xen/arch/arm/include/asm/arm64/cmpxchg.h
+++ b/xen/arch/arm/include/asm/arm64/cmpxchg.h
@@ -1,6 +1,8 @@
 #ifndef __ASM_ARM64_CMPXCHG_H
 #define __ASM_ARM64_CMPXCHG_H
 
+#include <xen/bug.h>
+
 extern void __bad_xchg(volatile void *ptr, int size);
 
 static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size)
diff --git a/xen/arch/arm/include/asm/regs.h b/xen/arch/arm/include/asm/regs.h
index f998aedff5..0d9f239a77 100644
--- a/xen/arch/arm/include/asm/regs.h
+++ b/xen/arch/arm/include/asm/regs.h
@@ -5,7 +5,7 @@
 
 #ifndef __ASSEMBLY__
 
-#include <xen/lib.h>
+#include <xen/bug.h>
 #include <xen/types.h>
 #include <public/xen.h>
 #include <asm/current.h>
diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
index 922779ce14..79b73a0dbb 100644
--- a/xen/arch/arm/include/asm/vgic.h
+++ b/xen/arch/arm/include/asm/vgic.h
@@ -22,6 +22,7 @@
 #include <asm/new_vgic.h>
 #else
 
+#include <xen/bug.h>
 #include <xen/radix-tree.h>
 #include <xen/rbtree.h>
 
diff --git a/xen/arch/ppc/include/asm/time.h b/xen/arch/ppc/include/asm/time.h
index aa9dda82a3..f23eec173e 100644
--- a/xen/arch/ppc/include/asm/time.h
+++ b/xen/arch/ppc/include/asm/time.h
@@ -2,7 +2,7 @@
 #ifndef __ASM_PPC_TIME_H__
 #define __ASM_PPC_TIME_H__
 
-#include <xen/lib.h>
+#include <xen/bug.h>
 #include <asm/processor.h>
 #include <asm/regs.h>
 
diff --git a/xen/arch/x86/bitops.c b/xen/arch/x86/bitops.c
index f6ee71512c..720ab32362 100644
--- a/xen/arch/x86/bitops.c
+++ b/xen/arch/x86/bitops.c
@@ -1,6 +1,6 @@
 
 #include <xen/bitops.h>
-#include <xen/lib.h>
+#include <xen/bug.h>
 
 unsigned int __find_first_bit(
     const unsigned long *addr, unsigned int size)
diff --git a/xen/arch/x86/include/asm/cpufeature.h b/xen/arch/x86/include/asm/cpufeature.h
index 896483a900..be16492c68 100644
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -6,6 +6,7 @@
 #ifndef __ASM_I386_CPUFEATURE_H
 #define __ASM_I386_CPUFEATURE_H
 
+#include <xen/cache.h>
 #include <xen/const.h>
 #include <asm/cpuid.h>
 
diff --git a/xen/arch/x86/include/asm/system.h b/xen/arch/x86/include/asm/system.h
index 6e40a03cb3..debf6bfa17 100644
--- a/xen/arch/x86/include/asm/system.h
+++ b/xen/arch/x86/include/asm/system.h
@@ -1,8 +1,8 @@
 #ifndef __ASM_SYSTEM_H
 #define __ASM_SYSTEM_H
 
-#include <xen/lib.h>
 #include <xen/bitops.h>
+#include <xen/bug.h>
 #include <asm/cpufeature.h>
 #include <asm/x86-defns.h>
 
diff --git a/xen/arch/x86/include/asm/x86_64/page.h b/xen/arch/x86/include/asm/x86_64/page.h
index f49e10475f..19ca64d792 100644
--- a/xen/arch/x86/include/asm/x86_64/page.h
+++ b/xen/arch/x86/include/asm/x86_64/page.h
@@ -19,6 +19,8 @@ static inline unsigned long canonicalise_addr(unsigned long addr)
         return addr & ~CANONICAL_MASK;
 }
 
+#include <xen/bug.h>
+
 #include <xen/types.h>
 
 #include <xen/pdx.h>
diff --git a/xen/arch/x86/x86_emulate/private.h b/xen/arch/x86/x86_emulate/private.h
index 33f2870508..0fa26ba00a 100644
--- a/xen/arch/x86/x86_emulate/private.h
+++ b/xen/arch/x86/x86_emulate/private.h
@@ -8,6 +8,7 @@
 
 #ifdef __XEN__
 
+# include <xen/bug.h>
 # include <xen/kernel.h>
 # include <asm/endbr.h>
 # include <asm/msr-index.h>
diff --git a/xen/common/efi/common-stub.c b/xen/common/efi/common-stub.c
index 5a91fe28cc..77f138a6c5 100644
--- a/xen/common/efi/common-stub.c
+++ b/xen/common/efi/common-stub.c
@@ -1,6 +1,6 @@
+#include <xen/bug.h>
 #include <xen/efi.h>
 #include <xen/errno.h>
-#include <xen/lib.h>
 
 bool efi_enabled(unsigned int feature)
 {
diff --git a/xen/common/version.c b/xen/common/version.c
index e807ef4de0..80869430fc 100644
--- a/xen/common/version.c
+++ b/xen/common/version.c
@@ -1,3 +1,4 @@
+#include <xen/bug.h>
 #include <xen/compile.h>
 #include <xen/init.h>
 #include <xen/errno.h>
diff --git a/xen/include/public/hvm/save.h b/xen/include/public/hvm/save.h
index ff0048e5f8..5561495b27 100644
--- a/xen/include/public/hvm/save.h
+++ b/xen/include/public/hvm/save.h
@@ -51,7 +51,7 @@ struct hvm_save_descriptor {
     struct __HVM_SAVE_TYPE_##_x { _type t; char c[_code]; char cpt[2];};  \
     struct __HVM_SAVE_TYPE_COMPAT_##_x { _ctype t; }
 
-# include <xen/lib.h> /* BUG() */
+# include <xen/bug.h> /* BUG() */
 # define DECLARE_HVM_SAVE_TYPE(_x, _code, _type)                         \
     static inline int __HVM_SAVE_FIX_COMPAT_##_x(void *h, uint32_t size) \
         { BUG(); return -1; }                                            \
diff --git a/xen/include/xen/bug.h b/xen/include/xen/bug.h
index 4bd0be5088..7c150b7032 100644
--- a/xen/include/xen/bug.h
+++ b/xen/include/xen/bug.h
@@ -16,6 +16,7 @@
 
 #ifndef __ASSEMBLY__
 
+#include <xen/compiler.h>
 #include <xen/macros.h>
 #include <xen/types.h>
 
@@ -127,6 +128,24 @@ static void always_inline run_in_exception_handler(
 } while ( false )
 #endif
 
+#define BUG_ON(p)  do { if (unlikely(p)) BUG();  } while (0)
+#define WARN_ON(p)  ({                  \
+    bool ret_warn_on_ = (p);            \
+                                        \
+    if ( unlikely(ret_warn_on_) )       \
+        WARN();                         \
+    unlikely(ret_warn_on_);             \
+})
+
+#ifndef NDEBUG
+#define ASSERT(p) \
+    do { if ( unlikely(!(p)) ) assert_failed(#p); } while (0)
+#define ASSERT_UNREACHABLE() assert_failed("unreachable")
+#else
+#define ASSERT(p) do { if ( 0 && (p) ) {} } while (0)
+#define ASSERT_UNREACHABLE() do { } while (0)
+#endif
+
 #ifdef CONFIG_GENERIC_BUG_FRAME
 
 /*
diff --git a/xen/include/xen/cpumask.h b/xen/include/xen/cpumask.h
index 145e140481..46e75d54ba 100644
--- a/xen/include/xen/cpumask.h
+++ b/xen/include/xen/cpumask.h
@@ -55,6 +55,7 @@
  * for_each_present_cpu(cpu)		for-loop cpu over cpu_present_map
  */
 
+#include <xen/bug.h>
 #include <xen/bitmap.h>
 #include <xen/kernel.h>
 #include <xen/random.h>
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 94a836cb4e..6fe2fa8b21 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -14,6 +14,7 @@
 #include <asm/device.h>
 #include <public/xen.h>
 #include <public/device_tree_defs.h>
+#include <xen/bug.h>
 #include <xen/kernel.h>
 #include <xen/string.h>
 #include <xen/types.h>
diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
index 1793be5b6b..4c755cc275 100644
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -5,31 +5,12 @@
 
 #ifndef __ASSEMBLY__
 
-#include <xen/bug.h>
 #include <xen/inttypes.h>
 #include <xen/stdarg.h>
 #include <xen/types.h>
 #include <xen/xmalloc.h>
 #include <xen/string.h>
 
-#define BUG_ON(p)  do { if (unlikely(p)) BUG();  } while (0)
-#define WARN_ON(p)  ({                  \
-    bool ret_warn_on_ = (p);            \
-                                        \
-    if ( unlikely(ret_warn_on_) )       \
-        WARN();                         \
-    unlikely(ret_warn_on_);             \
-})
-
-#ifndef NDEBUG
-#define ASSERT(p) \
-    do { if ( unlikely(!(p)) ) assert_failed(#p); } while (0)
-#define ASSERT_UNREACHABLE() assert_failed("unreachable")
-#else
-#define ASSERT(p) do { if ( 0 && (p) ) {} } while (0)
-#define ASSERT_UNREACHABLE() do { } while (0)
-#endif
-
 #define __ACCESS_ONCE(x) ({                             \
             (void)(typeof(x))0; /* Scalar typecheck. */ \
             (volatile typeof(x) *)&(x); })
diff --git a/xen/include/xen/list.h b/xen/include/xen/list.h
index dc5a8c461b..b5eab3a1eb 100644
--- a/xen/include/xen/list.h
+++ b/xen/include/xen/list.h
@@ -7,7 +7,7 @@
 #ifndef __XEN_LIST_H__
 #define __XEN_LIST_H__
 
-#include <xen/lib.h>
+#include <xen/bug.h>
 #include <asm/system.h>
 
 /*
diff --git a/xen/include/xen/livepatch.h b/xen/include/xen/livepatch.h
index 45df4bba4f..ad0eae28bd 100644
--- a/xen/include/xen/livepatch.h
+++ b/xen/include/xen/livepatch.h
@@ -18,6 +18,8 @@ struct xen_sysctl_livepatch_op;
 
 #ifdef CONFIG_LIVEPATCH
 
+#include <xen/lib.h>
+
 /*
  * We use alternative and exception table code - which by default are __init
  * only, however we need them during runtime. These macros allows us to build
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 3d9b2d05a5..bb29b352ec 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -61,6 +61,7 @@
 #ifndef __XEN_MM_H__
 #define __XEN_MM_H__
 
+#include <xen/bug.h>
 #include <xen/compiler.h>
 #include <xen/mm-frame.h>
 #include <xen/types.h>
diff --git a/xen/include/xen/param.h b/xen/include/xen/param.h
index 9170455cde..d8471ffe18 100644
--- a/xen/include/xen/param.h
+++ b/xen/include/xen/param.h
@@ -1,6 +1,7 @@
 #ifndef _XEN_PARAM_H
 #define _XEN_PARAM_H
 
+#include <xen/bug.h>
 #include <xen/hypfs.h>
 #include <xen/init.h>
 #include <xen/lib.h>
diff --git a/xen/lib/list-sort.c b/xen/lib/list-sort.c
index de1af2ef8b..251764804c 100644
--- a/xen/lib/list-sort.c
+++ b/xen/lib/list-sort.c
@@ -15,6 +15,7 @@
  * this program; If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <xen/lib.h>
 #include <xen/list.h>
 #include <xen/list_sort.h>
 
diff --git a/xen/xsm/flask/ss/ebitmap.h b/xen/xsm/flask/ss/ebitmap.h
index bb43de891d..7b2da8179a 100644
--- a/xen/xsm/flask/ss/ebitmap.h
+++ b/xen/xsm/flask/ss/ebitmap.h
@@ -15,6 +15,7 @@
 #define _SS_EBITMAP_H_
 
 #include <xen/bitmap.h>
+#include <xen/bug.h>
 
 #define EBITMAP_UNIT_NUMS	((32 - sizeof(void *) - sizeof(u32))	\
 					/ sizeof(unsigned long))
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 18:11:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 18:11:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675164.1050389 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVy0l-0006ab-W8; Fri, 02 Feb 2024 18:11:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675164.1050389; Fri, 02 Feb 2024 18:11:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVy0l-0006aU-TI; Fri, 02 Feb 2024 18:11:35 +0000
Received: by outflank-mailman (input) for mailman id 675164;
 Fri, 02 Feb 2024 18:11:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rVy0l-0006aO-9n
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 18:11:35 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVy0j-0006LV-CG; Fri, 02 Feb 2024 18:11:33 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVy0j-0004Up-54; Fri, 02 Feb 2024 18:11:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=k4ujF5wibe8ZZB+NW0LDju939U6RrWnDNvZLBP9Nx2U=; b=T3SGgQltgNMXhy4lMRLa5Au7ob
	qyHxwvlETDlvS5CSEDdwkvS6vN3yDvwkf3FYsvS0X8d6gJe7Emcln8qcSQymCs4yYUzPheVcQirY1
	1/D1cV5vcXxjbJXUOWW5rZ53ZffPcTiat1K+AEJ0MDYrHpDVc5CfnwGP1dScPlDvWoBA=;
Message-ID: <b85ecee7-65ce-47f0-8e9d-cdc056d337fb@xen.org>
Date: Fri, 2 Feb 2024 18:11:31 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 3/9] x86/smp: drop x86_cpu_to_apicid, use
 cpu_data[cpu].apicid instead
To: Krystian Hebel <krystian.hebel@3mdeb.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <8121d9b472b305be751158aa3af3fed98ff0572e.1699982111.git.krystian.hebel@3mdeb.com>
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
In-Reply-To: <8121d9b472b305be751158aa3af3fed98ff0572e.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 14/11/2023 17:50, Krystian Hebel wrote:
> Both fields held the same data.
> 
> Signed-off-by: Krystian Hebel <krystian.hebel@3mdeb.com>
> ---
>   xen/arch/x86/boot/x86_64.S           |  8 +++++---
>   xen/arch/x86/include/asm/asm_defns.h |  2 +-
>   xen/arch/x86/include/asm/processor.h |  2 ++
>   xen/arch/x86/include/asm/smp.h       |  4 ----
>   xen/arch/x86/numa.c                  | 15 +++++++--------
>   xen/arch/x86/smpboot.c               |  8 ++++----
>   xen/arch/x86/x86_64/asm-offsets.c    |  4 +++-
>   7 files changed, 22 insertions(+), 21 deletions(-)
> 
> diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
> index b85b47b5c1a0..195550b5c0ea 100644
> --- a/xen/arch/x86/boot/x86_64.S
> +++ b/xen/arch/x86/boot/x86_64.S
> @@ -20,15 +20,17 @@ ENTRY(__high_start)
>           jz      .L_stack_set
>   
>           /* APs only: get stack base from APIC ID saved in %esp. */
> -        mov     $-1, %rax
> -        lea     x86_cpu_to_apicid(%rip), %rcx
> +        mov     $0, %rax
> +        lea     cpu_data(%rip), %rcx
> +        /* cpu_data[0] is BSP, skip it. */
>   1:
>           add     $1, %rax
> +        add     $CPUINFO_X86_sizeof, %rcx
>           cmp     $NR_CPUS, %eax
>           jb      2f
>           hlt
>   2:
> -        cmp     %esp, (%rcx, %rax, 4)
> +        cmp     %esp, CPUINFO_X86_apicid(%rcx)
>           jne     1b
>   
>           /* %eax is now Xen CPU index. */

As mentioned in an earlier patch, I think you want to re-order the 
patches. This will avoid to modify twice the same code within the same 
series (it is best to avoid if you can).

> diff --git a/xen/arch/x86/include/asm/asm_defns.h b/xen/arch/x86/include/asm/asm_defns.h
> index baaaccb26e17..6b05d9d140b8 100644
> --- a/xen/arch/x86/include/asm/asm_defns.h
> +++ b/xen/arch/x86/include/asm/asm_defns.h
> @@ -158,7 +158,7 @@ register unsigned long current_stack_pointer asm("rsp");
>   #endif
>   
>   #define CPUINFO_FEATURE_OFFSET(feature)           \
> -    (CPUINFO_features + (cpufeat_word(feature) * 4))
> +    (CPUINFO_X86_features + (cpufeat_word(feature) * 4))
>   
>   #else
>   
> diff --git a/xen/arch/x86/include/asm/processor.h b/xen/arch/x86/include/asm/processor.h
> index b0d2a62c075f..8345d58094da 100644
> --- a/xen/arch/x86/include/asm/processor.h
> +++ b/xen/arch/x86/include/asm/processor.h
> @@ -92,6 +92,8 @@ struct x86_cpu_id {
>   extern struct cpuinfo_x86 cpu_data[];
>   #define current_cpu_data cpu_data[smp_processor_id()]
>   
> +#define cpu_physical_id(cpu)	cpu_data[cpu].apicid
> +
>   extern bool probe_cpuid_faulting(void);
>   extern void ctxt_switch_levelling(const struct vcpu *next);
>   extern void (*ctxt_switch_masking)(const struct vcpu *next);
> diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
> index c0b5d7cdd8dd..94c557491860 100644
> --- a/xen/arch/x86/include/asm/smp.h
> +++ b/xen/arch/x86/include/asm/smp.h
> @@ -39,10 +39,6 @@ extern void (*mtrr_hook) (void);
>   
>   extern void zap_low_mappings(void);
>   
> -extern u32 x86_cpu_to_apicid[];
> -
> -#define cpu_physical_id(cpu)	x86_cpu_to_apicid[cpu]
> -
>   #define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
>   extern void cpu_exit_clear(unsigned int cpu);
>   extern void cpu_uninit(unsigned int cpu);
> diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
> index 39e131cb4f35..91527be5b406 100644
> --- a/xen/arch/x86/numa.c
> +++ b/xen/arch/x86/numa.c
> @@ -54,14 +54,13 @@ bool __init arch_numa_unavailable(void)
>   /*
>    * Setup early cpu_to_node.
>    *
> - * Populate cpu_to_node[] only if x86_cpu_to_apicid[],
> - * and apicid_to_node[] tables have valid entries for a CPU.
> - * This means we skip cpu_to_node[] initialisation for NUMA
> - * emulation and faking node case (when running a kernel compiled
> - * for NUMA on a non NUMA box), which is OK as cpu_to_node[]
> - * is already initialized in a round robin manner at numa_init_array,
> - * prior to this call, and this initialization is good enough
> - * for the fake NUMA cases.
> + * Populate cpu_to_node[] only if cpu_data[], and apicid_to_node[]
> + * tables have valid entries for a CPU. This means we skip
> + * cpu_to_node[] initialisation for NUMA emulation and faking node
> + * case (when running a kernel compiled for NUMA on a non NUMA box),
> + * which is OK as cpu_to_node[] is already initialized in a round
> + * robin manner at numa_init_array, prior to this call, and this
> + * initialization is good enough for the fake NUMA cases.
>    */
>   void __init init_cpu_to_node(void)
>   {
> diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
> index de87c5a41926..f061486e56eb 100644
> --- a/xen/arch/x86/smpboot.c
> +++ b/xen/arch/x86/smpboot.c
> @@ -61,10 +61,8 @@ unsigned int __read_mostly nr_sockets;
>   cpumask_t **__read_mostly socket_cpumask;
>   static cpumask_t *secondary_socket_cpumask;
>   
> -struct cpuinfo_x86 cpu_data[NR_CPUS];
> -
> -u32 x86_cpu_to_apicid[NR_CPUS] __read_mostly =
> -	{ [0 ... NR_CPUS-1] = BAD_APICID };
> +struct cpuinfo_x86 cpu_data[NR_CPUS] =
> +        { [0 ... NR_CPUS-1] .apicid = BAD_APICID };
>   
>   static int cpu_error;
>   static enum cpu_state {
> @@ -81,7 +79,9 @@ void *stack_base[NR_CPUS];
>   
>   void initialize_cpu_data(unsigned int cpu)
>   {
> +    uint32_t apicid = cpu_physical_id(cpu);

It would be probably worth it to add a comment explaining why you save 
apicid. What about?

/* The APIC ID is saved in cpu_data[cpu] which will be overriden below. */

Coding style: Newline after the declaration.

>       cpu_data[cpu] = boot_cpu_data;
> +    cpu_physical_id(cpu) = apicid;
>   }
>   
>   static bool smp_store_cpu_info(unsigned int id)
> diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
> index 57b73a4e6214..e881cd5de0a0 100644
> --- a/xen/arch/x86/x86_64/asm-offsets.c
> +++ b/xen/arch/x86/x86_64/asm-offsets.c
> @@ -159,7 +159,9 @@ void __dummy__(void)
>       OFFSET(IRQSTAT_softirq_pending, irq_cpustat_t, __softirq_pending);
>       BLANK();
>   
> -    OFFSET(CPUINFO_features, struct cpuinfo_x86, x86_capability);
> +    OFFSET(CPUINFO_X86_features, struct cpuinfo_x86, x86_capability);

The rename seems to be unrelated to this patch. Can you clarify?

> +    OFFSET(CPUINFO_X86_apicid, struct cpuinfo_x86, apicid);
> +    DEFINE(CPUINFO_X86_sizeof, sizeof(struct cpuinfo_x86));
>       BLANK();
>   
>       OFFSET(MB_flags, multiboot_info_t, flags);

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 18:24:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 18:24:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675169.1050399 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVyDV-00007H-3T; Fri, 02 Feb 2024 18:24:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675169.1050399; Fri, 02 Feb 2024 18:24:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVyDV-000079-0m; Fri, 02 Feb 2024 18:24:45 +0000
Received: by outflank-mailman (input) for mailman id 675169;
 Fri, 02 Feb 2024 18:24:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rVyDT-000073-Fp
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 18:24:43 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVyDR-0006Zc-Ot; Fri, 02 Feb 2024 18:24:41 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rVyDR-00051G-Ff; Fri, 02 Feb 2024 18:24:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=yt8GOwOCT3ePosG6LIt5BLmiN7sipz1LPFk9gjL90tY=; b=Az8WqVad/yL2/z8I9FNvPzjy50
	rtcGmPq8oqze56gIs92/7g14zEosRd5oIjAptSXd++Et41jdvZ+68oDE4wox3SVa5F93QTHuAOFzl
	tXZueDKKmaRK5NIZnrXkIMrFSAiZynMaLFZDD+cHIxYo7CpU7G8buqVMOs/xE5oKDAzs=;
Message-ID: <d0fc568a-f54e-4480-a903-6407f4e3a5b6@xen.org>
Date: Fri, 2 Feb 2024 18:24:39 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 4/9] x86/smp: move stack_base to cpu_data
Content-Language: en-GB
To: Krystian Hebel <krystian.hebel@3mdeb.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <70e3b7c84a69a7ec52b3ed6314395165c281734c.1699982111.git.krystian.hebel@3mdeb.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <70e3b7c84a69a7ec52b3ed6314395165c281734c.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 14/11/2023 17:50, Krystian Hebel wrote:
> This location is easier to access from assembly. Having it close to
> other data required during initialization has also positive (although
> rather small) impact on prefetching data from RAM.

I understand your goal but...

> 
> Signed-off-by: Krystian Hebel <krystian.hebel@3mdeb.com>
> ---
>   xen/arch/x86/boot/x86_64.S            |  5 ++---
>   xen/arch/x86/include/asm/cpufeature.h |  1 +
>   xen/arch/x86/include/asm/smp.h        |  2 +-
>   xen/arch/x86/setup.c                  |  6 +++---
>   xen/arch/x86/smpboot.c                | 25 +++++++++++++------------
>   xen/arch/x86/traps.c                  |  4 ++--
>   xen/arch/x86/x86_64/asm-offsets.c     |  1 +
>   xen/include/xen/smp.h                 |  2 --
>   8 files changed, 23 insertions(+), 23 deletions(-)
> 
> diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
> index 195550b5c0ea..8d61f270761f 100644
> --- a/xen/arch/x86/boot/x86_64.S
> +++ b/xen/arch/x86/boot/x86_64.S
> @@ -33,9 +33,8 @@ ENTRY(__high_start)
>           cmp     %esp, CPUINFO_X86_apicid(%rcx)
>           jne     1b
>   
> -        /* %eax is now Xen CPU index. */
> -        lea     stack_base(%rip), %rcx
> -        mov     (%rcx, %rax, 8), %rsp
> +        /* %rcx is now cpu_data[cpu], read stack base from it. */
> +        mov     CPUINFO_X86_stack_base(%rcx), %rsp
>   
>           test    %rsp,%rsp
>           jnz     1f
> diff --git a/xen/arch/x86/include/asm/cpufeature.h b/xen/arch/x86/include/asm/cpufeature.h
> index 06e1dd7f3332..ff0e18864cc7 100644
> --- a/xen/arch/x86/include/asm/cpufeature.h
> +++ b/xen/arch/x86/include/asm/cpufeature.h

... cpufeature.h feels a rather odd place for storing the stack. I am 
not entirely sure where else to place. Andrew, Jan, Roger?

> @@ -37,6 +37,7 @@ struct cpuinfo_x86 {
>       unsigned int phys_proc_id;         /* package ID of each logical CPU */
>       unsigned int cpu_core_id;          /* core ID of each logical CPU */
>       unsigned int compute_unit_id;      /* AMD compute unit ID of each logical CPU */
> +    void *stack_base;

AFAICT, this means there will be a padding before stack_base and ...

>       unsigned short x86_clflush_size;

... another one here. Is there any particular reason the new field 
wasn't added at the end?

>   } __cacheline_aligned;
>   
> diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
> index 94c557491860..98739028a6ed 100644
> --- a/xen/arch/x86/include/asm/smp.h
> +++ b/xen/arch/x86/include/asm/smp.h
> @@ -69,7 +69,7 @@ extern cpumask_t **socket_cpumask;
>    * by certain scheduling code only.
>    */
>   #define get_cpu_current(cpu) \
> -    (get_cpu_info_from_stack((unsigned long)stack_base[cpu])->current_vcpu)
> +    (get_cpu_info_from_stack((unsigned long)cpu_data[cpu].stack_base)->current_vcpu)
>   
>   extern unsigned int disabled_cpus;
>   extern bool unaccounted_cpus;
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index a19fe219bbf6..b2c0679725ea 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -798,7 +798,7 @@ static void __init noreturn reinit_bsp_stack(void)
>       /* Update SYSCALL trampolines */
>       percpu_traps_init();
>   
> -    stack_base[0] = stack;
> +    cpu_data[0].stack_base = stack;
>   
>       rc = setup_cpu_root_pgt(0);
>       if ( rc )
> @@ -1959,8 +1959,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
>               /* Set up node_to_cpumask based on cpu_to_node[]. */
>               numa_add_cpu(i);
>   
> -            if ( stack_base[i] == NULL )
> -                stack_base[i] = cpu_alloc_stack(i);
> +            if ( cpu_data[i].stack_base == NULL )
> +                cpu_data[i].stack_base = cpu_alloc_stack(i);
>           }
>   
>           for_each_present_cpu ( i )
> diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
> index f061486e56eb..8ae65ab1769f 100644
> --- a/xen/arch/x86/smpboot.c
> +++ b/xen/arch/x86/smpboot.c
> @@ -75,13 +75,15 @@ static enum cpu_state {
>   } cpu_state;
>   #define set_cpu_state(state) do { smp_mb(); cpu_state = (state); } while (0)
>   
> -void *stack_base[NR_CPUS];
> -
>   void initialize_cpu_data(unsigned int cpu)
>   {
>       uint32_t apicid = cpu_physical_id(cpu);
> +    void *stack = cpu_data[cpu].stack_base;
> +
>       cpu_data[cpu] = boot_cpu_data;
> +
>       cpu_physical_id(cpu) = apicid;
> +    cpu_data[cpu].stack_base = stack;
>   }
>   
>   static bool smp_store_cpu_info(unsigned int id)
> @@ -579,8 +581,6 @@ static int do_boot_cpu(int apicid, int cpu)
>           printk("Booting processor %d/%d eip %lx\n",
>                  cpu, apicid, start_eip);
>   
> -    stack_start = stack_base[cpu] + STACK_SIZE - sizeof(struct cpu_info);
> -

You remove this line because I can't quite figure out where stack_start 
is now set. This is used...

>       /* This grunge runs the startup process for the targeted processor. */
>   
>       set_cpu_state(CPU_STATE_INIT);
> @@ -856,7 +856,7 @@ int setup_cpu_root_pgt(unsigned int cpu)
>   
>       /* Install direct map page table entries for stack, IDT, and TSS. */
>       for ( off = rc = 0; !rc && off < STACK_SIZE; off += PAGE_SIZE )
> -        rc = clone_mapping(__va(__pa(stack_base[cpu])) + off, rpt);
> +        rc = clone_mapping(__va(__pa(cpu_data[cpu].stack_base)) + off, rpt);
>   
>       if ( !rc )
>           rc = clone_mapping(idt_tables[cpu], rpt);
> @@ -1007,10 +1007,10 @@ static void cpu_smpboot_free(unsigned int cpu, bool remove)
>           FREE_XENHEAP_PAGE(per_cpu(gdt, cpu));
>           FREE_XENHEAP_PAGE(idt_tables[cpu]);
>   
> -        if ( stack_base[cpu] )
> +        if ( cpu_data[cpu].stack_base )
>           {
> -            memguard_unguard_stack(stack_base[cpu]);
> -            FREE_XENHEAP_PAGES(stack_base[cpu], STACK_ORDER);
> +            memguard_unguard_stack(cpu_data[cpu].stack_base);
> +            FREE_XENHEAP_PAGES(cpu_data[cpu].stack_base, STACK_ORDER);
>           }
>       }
>   }
> @@ -1044,11 +1044,11 @@ static int cpu_smpboot_alloc(unsigned int cpu)
>       if ( node != NUMA_NO_NODE )
>           memflags = MEMF_node(node);
>   
> -    if ( stack_base[cpu] == NULL &&
> -         (stack_base[cpu] = cpu_alloc_stack(cpu)) == NULL )
> +    if ( cpu_data[cpu].stack_base == NULL &&
> +         (cpu_data[cpu].stack_base = cpu_alloc_stack(cpu)) == NULL )
>               goto out;
>   
> -    info = get_cpu_info_from_stack((unsigned long)stack_base[cpu]);
> +    info = get_cpu_info_from_stack((unsigned long)cpu_data[cpu].stack_base);

... here.

>       info->processor_id = cpu;
>       info->per_cpu_offset = __per_cpu_offset[cpu];
>   
> @@ -1156,7 +1156,8 @@ void __init smp_prepare_cpus(void)
>       boot_cpu_physical_apicid = get_apic_id();
>       cpu_physical_id(0) = boot_cpu_physical_apicid;
>   
> -    stack_base[0] = (void *)((unsigned long)stack_start & ~(STACK_SIZE - 1));
> +    cpu_data[0].stack_base = (void *)
> +             ((unsigned long)stack_start & ~(STACK_SIZE - 1));
>   
>       set_nr_sockets();
>   
> diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
> index e1356f696aba..90d9201d1c52 100644
> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -611,9 +611,9 @@ void show_stack_overflow(unsigned int cpu, const struct cpu_user_regs *regs)
>       unsigned long curr_stack_base = esp & ~(STACK_SIZE - 1);
>       unsigned long esp_top, esp_bottom;
>   
> -    if ( _p(curr_stack_base) != stack_base[cpu] )
> +    if ( _p(curr_stack_base) != cpu_data[cpu].stack_base )
>           printk("Current stack base %p differs from expected %p\n",
> -               _p(curr_stack_base), stack_base[cpu]);
> +               _p(curr_stack_base), cpu_data[cpu].stack_base);
>   
>       esp_bottom = (esp | (STACK_SIZE - 1)) + 1;
>       esp_top    = esp_bottom - PRIMARY_STACK_SIZE;
> diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
> index e881cd5de0a0..d81a30344677 100644
> --- a/xen/arch/x86/x86_64/asm-offsets.c
> +++ b/xen/arch/x86/x86_64/asm-offsets.c
> @@ -161,6 +161,7 @@ void __dummy__(void)
>   
>       OFFSET(CPUINFO_X86_features, struct cpuinfo_x86, x86_capability);
>       OFFSET(CPUINFO_X86_apicid, struct cpuinfo_x86, apicid);
> +    OFFSET(CPUINFO_X86_stack_base, struct cpuinfo_x86, stack_base);
>       DEFINE(CPUINFO_X86_sizeof, sizeof(struct cpuinfo_x86));
>       BLANK();
>   
> diff --git a/xen/include/xen/smp.h b/xen/include/xen/smp.h
> index 0a9219173f0f..994fdc474200 100644
> --- a/xen/include/xen/smp.h
> +++ b/xen/include/xen/smp.h
> @@ -67,8 +67,6 @@ void smp_send_call_function_mask(const cpumask_t *mask);
>   
>   int alloc_cpu_id(void);
>   
> -extern void *stack_base[NR_CPUS];
> -
>   void initialize_cpu_data(unsigned int cpu);
>   int setup_cpu_root_pgt(unsigned int cpu);
>   

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 19:22:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 19:22:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675177.1050410 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVz6t-0000fu-UG; Fri, 02 Feb 2024 19:21:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675177.1050410; Fri, 02 Feb 2024 19:21:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVz6t-0000fm-Qm; Fri, 02 Feb 2024 19:21:59 +0000
Received: by outflank-mailman (input) for mailman id 675177;
 Fri, 02 Feb 2024 19:21:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVz6s-0000fb-Pv; Fri, 02 Feb 2024 19:21:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVz6s-0007V2-NE; Fri, 02 Feb 2024 19:21:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rVz6s-0001k1-8S; Fri, 02 Feb 2024 19:21:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rVz6s-00084U-7n; Fri, 02 Feb 2024 19:21:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zc4dUEtdl/WWKQsQazFZpux/22K8YjTiabgtStfWCjk=; b=yW74UvLDVPdpquKy329DviR+Fi
	wOzsByYtahvirl1kG6aVLF6KZkTQGCl6Akm/lBau/Aubu2EEfDMGMdUuMlfyu3hD4VxNV9jPvlBU0
	WPBjCpHIBj04Z1OOWD5rqQhDNyhpyWoUXjDmxPMXGetLI0vJ4faAVYYFSHNAQZlzOoVA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184557-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184557: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-dom0pvh-xl-intel:guest-localmigrate/x10:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=bc45f20c01f1711bc56a4bb0955c49c182a5a03a
X-Osstest-Versions-That:
    xen=cc6ba68edf6dcd18c3865e7d7c0f1ed822796426
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 02 Feb 2024 19:21:58 +0000

flight 184557 xen-unstable real [real]
flight 184567 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184557/
http://logs.test-lab.xenproject.org/osstest/logs/184567/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-dom0pvh-xl-intel 20 guest-localmigrate/x10 fail pass in 184567-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184547
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184547
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184547
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184547
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184547
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184547
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184547
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184547
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184547
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184547
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184547
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184547
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  bc45f20c01f1711bc56a4bb0955c49c182a5a03a
baseline version:
 xen                  cc6ba68edf6dcd18c3865e7d7c0f1ed822796426

Last test of basis   184547  2024-02-01 03:35:56 Z    1 days
Testing same since   184557  2024-02-01 22:11:04 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Carlo Nonato <carlo.nonato@minervasys.tech>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            fail    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   cc6ba68edf..bc45f20c01  bc45f20c01f1711bc56a4bb0955c49c182a5a03a -> master


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 19:25:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 19:25:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675189.1050439 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVzAH-0001c2-MF; Fri, 02 Feb 2024 19:25:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675189.1050439; Fri, 02 Feb 2024 19:25:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVzAH-0001bv-Ji; Fri, 02 Feb 2024 19:25:29 +0000
Received: by outflank-mailman (input) for mailman id 675189;
 Fri, 02 Feb 2024 19:25:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HwCY=JL=casper.srs.infradead.org=BATV+836f8942eaadfea8137c+7467+infradead.org+dwmw2@srs-se1.protection.inumbo.net>)
 id 1rVzAE-0001bg-TT
 for xen-devel@lists.xen.org; Fri, 02 Feb 2024 19:25:27 +0000
Received: from casper.infradead.org (casper.infradead.org
 [2001:8b0:10b:1236::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cf6f35b7-c200-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 20:25:24 +0100 (CET)
Received: from [205.251.233.110] (helo=freeip.amazon.com)
 by casper.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rVzA8-00000001sgo-442C; Fri, 02 Feb 2024 19:25:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf6f35b7-c200-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=casper.20170209; h=MIME-Version:Content-Type:References:
	In-Reply-To:Date:Cc:To:From:Subject:Message-ID:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=Ag16YAc3/tUsCqDHnNPMn80B+fOZRnMAnvXZ8MlZeCA=; b=lxTUiZwCoH9acOFUip9zBhN1h3
	0ifncYZYwbbG0TX3COn2577QT/LfA9k1itivbESsVUdyj9mkbuVbQdLkKRaR07hkyklOuemoGY4OD
	3ITSklNPrMdrQsA9yoMMGM0+PpidTEYeV4O07TDHLcmXBseGHKlPYxYnQciSN6ZFdM4IijFXZE2mz
	v0di3DCOnt5a7gq43Ioh6mrnahaQN35XZBMpsAqFVckIlL5iOF7ILc9Hnv0uXC9OHDvQRYAfJhvnT
	OiBGilE0h6Bo7kItrNr89mq3ye3sbzmakBmRFKvmQZGJQD7qZMoHcdZUO+aBb3dygaT8QY8IIJeA7
	xG6oYDTg==;
Message-ID: <a8a3c0d82f868869751958250719398a2d042e93.camel@infradead.org>
Subject: Re: Nullifying Recently Introduced Xen Headers Check
From: David Woodhouse <dwmw2@infradead.org>
To: "John L. Poole" <johnlpoole@proton.me>
Cc: xen-devel@lists.xen.org
Date: Fri, 02 Feb 2024 11:25:18 -0800
In-Reply-To: <cj1qg72oASoR0AT_B5_w6AnPm5tf8K-muPdrGWSGYozXJ6N8vaMUI3XMVBx2IoAYlKPAyUTewevznMbj6S0t3plMygEAS8_HtUVubMXo1dA=@proton.me>
References: <fb379476-f6ae-4eb7-b3c6-6f0ad175964d@proton.me>
	 <4b5b1fdadbabd53116d1ab79295b913c27947c4b.camel@infradead.org>
	 <cj1qg72oASoR0AT_B5_w6AnPm5tf8K-muPdrGWSGYozXJ6N8vaMUI3XMVBx2IoAYlKPAyUTewevznMbj6S0t3plMygEAS8_HtUVubMXo1dA=@proton.me>
Content-Type: multipart/signed; micalg="sha-256"; protocol="application/pkcs7-signature";
	boundary="=-vjzs1uokPh3s7tR50m12"
User-Agent: Evolution 3.44.4-0ubuntu2 
MIME-Version: 1.0
X-SRS-Rewrite: SMTP reverse-path rewritten from <dwmw2@infradead.org> by casper.infradead.org. See http://www.infradead.org/rpr.html


--=-vjzs1uokPh3s7tR50m12
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, 2024-02-01 at 18:39 +0000, John L. Poole wrote:
>=20
> While the Xen Project "make" works, the Gentoo emerge
> of app-emulation/xen-tools does not unless the three lines are
> removed to simulate prior 4.17.3 and earlier code.=C2=A0=20
>=20
> I suspect the Gentoo approach=20
> of building tools first contributes to the problem.

The problem is that -D__XEN_INTERFACE_VERSION=3D=E2=80=A6 is being set on t=
he
compiler command line, in ${CFLAGS}.

Clearly it isn't qemu itself which does that. And I think you said
above that invoking the qemu build even from the Xen makefiles directly
is also working fine?=20

So the question is why the Gentoo build is setting that explicitly in
CFLAGS?

--=-vjzs1uokPh3s7tR50m12
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Disposition: attachment; filename="smime.p7s"
Content-Transfer-Encoding: base64

MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCCEkQw
ggYQMIID+KADAgECAhBNlCwQ1DvglAnFgS06KwZPMA0GCSqGSIb3DQEBDAUAMIGIMQswCQYDVQQG
EwJVUzETMBEGA1UECBMKTmV3IEplcnNleTEUMBIGA1UEBxMLSmVyc2V5IENpdHkxHjAcBgNVBAoT
FVRoZSBVU0VSVFJVU1QgTmV0d29yazEuMCwGA1UEAxMlVVNFUlRydXN0IFJTQSBDZXJ0aWZpY2F0
aW9uIEF1dGhvcml0eTAeFw0xODExMDIwMDAwMDBaFw0zMDEyMzEyMzU5NTlaMIGWMQswCQYDVQQG
EwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYD
VQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMTNVNlY3RpZ28gUlNBIENsaWVudCBBdXRoZW50
aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
AQEAyjztlApB/975Rrno1jvm2pK/KxBOqhq8gr2+JhwpKirSzZxQgT9tlC7zl6hn1fXjSo5MqXUf
ItMltrMaXqcESJuK8dtK56NCSrq4iDKaKq9NxOXFmqXX2zN8HHGjQ2b2Xv0v1L5Nk1MQPKA19xeW
QcpGEGFUUd0kN+oHox+L9aV1rjfNiCj3bJk6kJaOPabPi2503nn/ITX5e8WfPnGw4VuZ79Khj1YB
rf24k5Ee1sLTHsLtpiK9OjG4iQRBdq6Z/TlVx/hGAez5h36bBJMxqdHLpdwIUkTqT8se3ed0PewD
ch/8kHPo5fZl5u1B0ecpq/sDN/5sCG52Ds+QU5O5EwIDAQABo4IBZDCCAWAwHwYDVR0jBBgwFoAU
U3m/WqorSs9UgOHYm8Cd8rIDZsswHQYDVR0OBBYEFAnA8vwL2pTbX/4r36iZQs/J4K0AMA4GA1Ud
DwEB/wQEAwIBhjASBgNVHRMBAf8ECDAGAQH/AgEAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDARBgNVHSAECjAIMAYGBFUdIAAwUAYDVR0fBEkwRzBFoEOgQYY/aHR0cDovL2NybC51c2Vy
dHJ1c3QuY29tL1VTRVJUcnVzdFJTQUNlcnRpZmljYXRpb25BdXRob3JpdHkuY3JsMHYGCCsGAQUF
BwEBBGowaDA/BggrBgEFBQcwAoYzaHR0cDovL2NydC51c2VydHJ1c3QuY29tL1VTRVJUcnVzdFJT
QUFkZFRydXN0Q0EuY3J0MCUGCCsGAQUFBzABhhlodHRwOi8vb2NzcC51c2VydHJ1c3QuY29tMA0G
CSqGSIb3DQEBDAUAA4ICAQBBRHUAqznCFfXejpVtMnFojADdF9d6HBA4kMjjsb0XMZHztuOCtKF+
xswhh2GqkW5JQrM8zVlU+A2VP72Ky2nlRA1GwmIPgou74TZ/XTarHG8zdMSgaDrkVYzz1g3nIVO9
IHk96VwsacIvBF8JfqIs+8aWH2PfSUrNxP6Ys7U0sZYx4rXD6+cqFq/ZW5BUfClN/rhk2ddQXyn7
kkmka2RQb9d90nmNHdgKrwfQ49mQ2hWQNDkJJIXwKjYA6VUR/fZUFeCUisdDe/0ABLTI+jheXUV1
eoYV7lNwNBKpeHdNuO6Aacb533JlfeUHxvBz9OfYWUiXu09sMAviM11Q0DuMZ5760CdO2VnpsXP4
KxaYIhvqPqUMWqRdWyn7crItNkZeroXaecG03i3mM7dkiPaCkgocBg0EBYsbZDZ8bsG3a08LwEsL
1Ygz3SBsyECa0waq4hOf/Z85F2w2ZpXfP+w8q4ifwO90SGZZV+HR/Jh6rEaVPDRF/CEGVqR1hiuQ
OZ1YL5ezMTX0ZSLwrymUE0pwi/KDaiYB15uswgeIAcA6JzPFf9pLkAFFWs1QNyN++niFhsM47qod
x/PL+5jR87myx5uYdBEQkkDc+lKB1Wct6ucXqm2EmsaQ0M95QjTmy+rDWjkDYdw3Ms6mSWE3Bn7i
5ZgtwCLXgAIe5W8mybM2JzCCBhQwggT8oAMCAQICEQDGvhmWZ0DEAx0oURL6O6l+MA0GCSqGSIb3
DQEBCwUAMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYD
VQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMTNVNlY3RpZ28g
UlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBMB4XDTIyMDEwNzAw
MDAwMFoXDTI1MDEwNjIzNTk1OVowJDEiMCAGCSqGSIb3DQEJARYTZHdtdzJAaW5mcmFkZWFkLm9y
ZzCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALQ3GpC2bomUqk+91wLYBzDMcCj5C9m6
oZaHwvmIdXftOgTbCJXADo6G9T7BBAebw2JV38EINgKpy/ZHh7htyAkWYVoFsFPrwHounto8xTsy
SSePMiPlmIdQ10BcVSXMUJ3Juu16GlWOnAMJY2oYfEzmE7uT9YgcBqKCo65pTFmOnR/VVbjJk4K2
xE34GC2nAdUQkPFuyaFisicc6HRMOYXPuF0DuwITEKnjxgNjP+qDrh0db7PAjO1D4d5ftfrsf+kd
RR4gKVGSk8Tz2WwvtLAroJM4nXjNPIBJNT4w/FWWc/5qPHJy2U+eITZ5LLE5s45mX2oPFknWqxBo
bQZ8a9dsZ3dSPZBvE9ZrmtFLrVrN4eo1jsXgAp1+p7bkfqd3BgBEmfsYWlBXO8rVXfvPgLs32VdV
NZxb/CDWPqBsiYv0Hv3HPsz07j5b+/cVoWqyHDKzkaVbxfq/7auNVRmPB3v5SWEsH8xi4Bez2V9U
KxfYCnqsjp8RaC2/khxKt0A552Eaxnz/4ly/2C7wkwTQnBmdlFYhAflWKQ03Ufiu8t3iBE3VJbc2
5oMrglj7TRZrmKq3CkbFnX0fyulB+kHimrt6PIWn7kgyl9aelIl6vtbhMA+l0nfrsORMa4kobqQ5
C5rveVgmcIad67EDa+UqEKy/GltUwlSh6xy+TrK1tzDvAgMBAAGjggHMMIIByDAfBgNVHSMEGDAW
gBQJwPL8C9qU21/+K9+omULPyeCtADAdBgNVHQ4EFgQUzMeDMcimo0oz8o1R1Nver3ZVpSkwDgYD
VR0PAQH/BAQDAgWgMAwGA1UdEwEB/wQCMAAwHQYDVR0lBBYwFAYIKwYBBQUHAwQGCCsGAQUFBwMC
MEAGA1UdIAQ5MDcwNQYMKwYBBAGyMQECAQEBMCUwIwYIKwYBBQUHAgEWF2h0dHBzOi8vc2VjdGln
by5jb20vQ1BTMFoGA1UdHwRTMFEwT6BNoEuGSWh0dHA6Ly9jcmwuc2VjdGlnby5jb20vU2VjdGln
b1JTQUNsaWVudEF1dGhlbnRpY2F0aW9uYW5kU2VjdXJlRW1haWxDQS5jcmwwgYoGCCsGAQUFBwEB
BH4wfDBVBggrBgEFBQcwAoZJaHR0cDovL2NydC5zZWN0aWdvLmNvbS9TZWN0aWdvUlNBQ2xpZW50
QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFpbENBLmNydDAjBggrBgEFBQcwAYYXaHR0cDovL29j
c3Auc2VjdGlnby5jb20wHgYDVR0RBBcwFYETZHdtdzJAaW5mcmFkZWFkLm9yZzANBgkqhkiG9w0B
AQsFAAOCAQEAyW6MUir5dm495teKqAQjDJwuFCi35h4xgnQvQ/fzPXmtR9t54rpmI2TfyvcKgOXp
qa7BGXNFfh1JsqexVkIqZP9uWB2J+uVMD+XZEs/KYNNX2PvIlSPrzIB4Z2wyIGQpaPLlYflrrVFK
v9CjT2zdqvy2maK7HKOQRt3BiJbVG5lRiwbbygldcALEV9ChWFfgSXvrWDZspnU3Gjw/rMHrGnql
Htlyebp3pf3fSS9kzQ1FVtVIDrL6eqhTwJxe+pXSMMqFiN0whpBtXdyDjzBtQTaZJ7zTT/vlehc/
tDuqZwGHm/YJy883Ll+GP3NvOkgaRGWEuYWJJ6hFCkXYjyR9IzCCBhQwggT8oAMCAQICEQDGvhmW
Z0DEAx0oURL6O6l+MA0GCSqGSIb3DQEBCwUAMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3Jl
YXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0
ZWQxPjA8BgNVBAMTNVNlY3RpZ28gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJl
IEVtYWlsIENBMB4XDTIyMDEwNzAwMDAwMFoXDTI1MDEwNjIzNTk1OVowJDEiMCAGCSqGSIb3DQEJ
ARYTZHdtdzJAaW5mcmFkZWFkLm9yZzCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALQ3
GpC2bomUqk+91wLYBzDMcCj5C9m6oZaHwvmIdXftOgTbCJXADo6G9T7BBAebw2JV38EINgKpy/ZH
h7htyAkWYVoFsFPrwHounto8xTsySSePMiPlmIdQ10BcVSXMUJ3Juu16GlWOnAMJY2oYfEzmE7uT
9YgcBqKCo65pTFmOnR/VVbjJk4K2xE34GC2nAdUQkPFuyaFisicc6HRMOYXPuF0DuwITEKnjxgNj
P+qDrh0db7PAjO1D4d5ftfrsf+kdRR4gKVGSk8Tz2WwvtLAroJM4nXjNPIBJNT4w/FWWc/5qPHJy
2U+eITZ5LLE5s45mX2oPFknWqxBobQZ8a9dsZ3dSPZBvE9ZrmtFLrVrN4eo1jsXgAp1+p7bkfqd3
BgBEmfsYWlBXO8rVXfvPgLs32VdVNZxb/CDWPqBsiYv0Hv3HPsz07j5b+/cVoWqyHDKzkaVbxfq/
7auNVRmPB3v5SWEsH8xi4Bez2V9UKxfYCnqsjp8RaC2/khxKt0A552Eaxnz/4ly/2C7wkwTQnBmd
lFYhAflWKQ03Ufiu8t3iBE3VJbc25oMrglj7TRZrmKq3CkbFnX0fyulB+kHimrt6PIWn7kgyl9ae
lIl6vtbhMA+l0nfrsORMa4kobqQ5C5rveVgmcIad67EDa+UqEKy/GltUwlSh6xy+TrK1tzDvAgMB
AAGjggHMMIIByDAfBgNVHSMEGDAWgBQJwPL8C9qU21/+K9+omULPyeCtADAdBgNVHQ4EFgQUzMeD
Mcimo0oz8o1R1Nver3ZVpSkwDgYDVR0PAQH/BAQDAgWgMAwGA1UdEwEB/wQCMAAwHQYDVR0lBBYw
FAYIKwYBBQUHAwQGCCsGAQUFBwMCMEAGA1UdIAQ5MDcwNQYMKwYBBAGyMQECAQEBMCUwIwYIKwYB
BQUHAgEWF2h0dHBzOi8vc2VjdGlnby5jb20vQ1BTMFoGA1UdHwRTMFEwT6BNoEuGSWh0dHA6Ly9j
cmwuc2VjdGlnby5jb20vU2VjdGlnb1JTQUNsaWVudEF1dGhlbnRpY2F0aW9uYW5kU2VjdXJlRW1h
aWxDQS5jcmwwgYoGCCsGAQUFBwEBBH4wfDBVBggrBgEFBQcwAoZJaHR0cDovL2NydC5zZWN0aWdv
LmNvbS9TZWN0aWdvUlNBQ2xpZW50QXV0aGVudGljYXRpb25hbmRTZWN1cmVFbWFpbENBLmNydDAj
BggrBgEFBQcwAYYXaHR0cDovL29jc3Auc2VjdGlnby5jb20wHgYDVR0RBBcwFYETZHdtdzJAaW5m
cmFkZWFkLm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAyW6MUir5dm495teKqAQjDJwuFCi35h4xgnQv
Q/fzPXmtR9t54rpmI2TfyvcKgOXpqa7BGXNFfh1JsqexVkIqZP9uWB2J+uVMD+XZEs/KYNNX2PvI
lSPrzIB4Z2wyIGQpaPLlYflrrVFKv9CjT2zdqvy2maK7HKOQRt3BiJbVG5lRiwbbygldcALEV9Ch
WFfgSXvrWDZspnU3Gjw/rMHrGnqlHtlyebp3pf3fSS9kzQ1FVtVIDrL6eqhTwJxe+pXSMMqFiN0w
hpBtXdyDjzBtQTaZJ7zTT/vlehc/tDuqZwGHm/YJy883Ll+GP3NvOkgaRGWEuYWJJ6hFCkXYjyR9
IzGCBMcwggTDAgEBMIGsMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVz
dGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMT
NVNlY3RpZ28gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBAhEA
xr4ZlmdAxAMdKFES+jupfjANBglghkgBZQMEAgEFAKCCAeswGAYJKoZIhvcNAQkDMQsGCSqGSIb3
DQEHATAcBgkqhkiG9w0BCQUxDxcNMjQwMjAyMTkyNTE4WjAvBgkqhkiG9w0BCQQxIgQgU6bMNsjH
7wxQpfJHMGbFH7M3mPgLE8U2keuidzMsMQ0wgb0GCSsGAQQBgjcQBDGBrzCBrDCBljELMAkGA1UE
BhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEYMBYG
A1UEChMPU2VjdGlnbyBMaW1pdGVkMT4wPAYDVQQDEzVTZWN0aWdvIFJTQSBDbGllbnQgQXV0aGVu
dGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIRAMa+GZZnQMQDHShREvo7qX4wgb8GCyqGSIb3
DQEJEAILMYGvoIGsMIGWMQswCQYDVQQGEwJHQjEbMBkGA1UECBMSR3JlYXRlciBNYW5jaGVzdGVy
MRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxPjA8BgNVBAMTNVNl
Y3RpZ28gUlNBIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQgU2VjdXJlIEVtYWlsIENBAhEAxr4Z
lmdAxAMdKFES+jupfjANBgkqhkiG9w0BAQEFAASCAgBFqPNmPhsbOpE8lBTnn7O91GP3BjRfxBrq
oH5GhysCtAv5l8xfZelruDESqPvT1wn+BswfpHsOuBxZ3ca6pzGbTADFkXDIOhh6YPfcrgLqCN/0
uM8gcziZNGlOd1RXDqFnuG2Sii/5sNfqjbXItzcCcT+NcOnl5gcxc17uqadwXzpGMilnUHdn+Xk2
guYbTppZjWn3aPESKBsPVi8Fjt2y0iji4pmJTcGccosXnrodSulE3PiM3I7RI0+tj/1IW6WfP+NK
1LrOWn08XZeU3ynW7URGMwzmJFbiWUlqse7Ws6+haiFZNf41SdlzMDjlFMX/vWBYswV2sT9KAn7y
9HIoIAMxO0mXDB8Op2eoycwqYS53qFdry8gsMaH2FV0ZwBG0uU814O2EJf+hnqzIlsDgclBjnPkF
UE3d+9RmOiY098Ms6XCl7k8NSRtrCbUDzPNJcav5UNMjd+P0FJ4+tCDkGax5Eu9fSppsKgupveuF
609yM6Y9DFofSstGxyM0dHJy71EoLTXdBJ5i72Ly77mLcIBpwxeAbsa4Nnu9c4mW+lwUEdjLy0a+
99HXzzTgW4ffzyNsjB2X9qgdSviYOwPePpod3S0XWxDiiIXP9z2/nuu1LgIviCmfdmK4v+NIH6Pe
ksIb3puCSbq8I421ALeed9eGreOmSCkTeqdjIuPG8gAAAAAAAA==


--=-vjzs1uokPh3s7tR50m12--


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 19:49:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 19:49:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675194.1050449 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVzXR-0005jt-HH; Fri, 02 Feb 2024 19:49:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675194.1050449; Fri, 02 Feb 2024 19:49:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rVzXR-0005jm-En; Fri, 02 Feb 2024 19:49:25 +0000
Received: by outflank-mailman (input) for mailman id 675194;
 Fri, 02 Feb 2024 19:49:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+1zP=JL=proton.me=johnlpoole@srs-se1.protection.inumbo.net>)
 id 1rVzXP-0005jc-5i
 for xen-devel@lists.xen.org; Fri, 02 Feb 2024 19:49:24 +0000
Received: from mail-40134.protonmail.ch (mail-40134.protonmail.ch
 [185.70.40.134]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 28a35320-c204-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 20:49:21 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28a35320-c204-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me;
	s=protonmail; t=1706903360; x=1707162560;
	bh=R+gSnI5dg26Sgj4zkL+FSBJq3xjPpbwjKyqY1GDDlYo=;
	h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References:
	 Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:
	 Message-ID:BIMI-Selector;
	b=DvmXLxi5b/UGCaRY2kZWWiF/3TIeIf7lZu+FWGaitEV0oh1y2c2rMNJt2/UbAEe19
	 lh6o5PV+5usaPm6BPp4ijtmfxpWM1w+Agu6kJIE4oefUjhS/oWvpaiJwR4fzK3UY3J
	 MAn4lPt3J8ij5uz+SYuex+TmvHA7eCUr/dR3Uo5l7bveYTI8ma52SMDi8qnq04lYIR
	 xlgZvWokXp5Pc3htNwB1Ia9L9ZkLp/2suEGjqlRiYsuuKHLhkdvB7HsluFJHG+eLZh
	 NGk/3G+APlt5KJSResGzAunoowZcVWn6JRCg9R4IhK+9QriUbuysgcHGJzqTNnJLVg
	 7n6bF6rKiPkbg==
Date: Fri, 02 Feb 2024 19:48:59 +0000
To: David Woodhouse <dwmw2@infradead.org>
From: "John L. Poole" <johnlpoole@proton.me>
Cc: xen-devel@lists.xen.org
Subject: Re: Nullifying Recently Introduced Xen Headers Check
Message-ID: <ff5abd30-0b64-4c4f-b992-2894cbde7468@proton.me>
In-Reply-To: <a8a3c0d82f868869751958250719398a2d042e93.camel@infradead.org>
References: <fb379476-f6ae-4eb7-b3c6-6f0ad175964d@proton.me> <4b5b1fdadbabd53116d1ab79295b913c27947c4b.camel@infradead.org> <cj1qg72oASoR0AT_B5_w6AnPm5tf8K-muPdrGWSGYozXJ6N8vaMUI3XMVBx2IoAYlKPAyUTewevznMbj6S0t3plMygEAS8_HtUVubMXo1dA=@proton.me> <a8a3c0d82f868869751958250719398a2d042e93.camel@infradead.org>
Feedback-ID: 80499891:user:proton
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


On 2/2/2024 11:25 AM, David Woodhouse wrote:
> On Thu, 2024-02-01 at 18:39 +0000, John L. Poole wrote:
>> While the Xen Project "make" works, the Gentoo emerge
>> of app-emulation/xen-tools does not unless the three lines are
>> removed to simulate prior 4.17.3 and earlier code.
>>
>> I suspect the Gentoo approach
>> of building tools first contributes to the problem.
> The problem is that -D__XEN_INTERFACE_VERSION=3D=E2=80=A6 is being set on=
 the
> compiler command line, in ${CFLAGS}.
>
> Clearly it isn't qemu itself which does that. And I think you said
> above that invoking the qemu build even from the Xen makefiles directly
> is also working fine?
>
> So the question is why the Gentoo build is setting that explicitly in
> CFLAGS?

Correct.

I downloaded

https://downloads.xenproject.org/release/xen/4.18.0/xen-4.18.0.tar.gz,

staged and entered the tree, ".configure", then "make" and everything
worked.=C2=A0 That's when
I decided Gentoo's package/build system was affecting matters and I
later determined that Gentoo
had made the decision to bifurcate the build order and build tools in
its own package,
app-emulation/xen-tools, and then xen in its own package
app-emulation/xen, the
former being a prerequisite to the latter.

I've postured this divergence in the bug: https://bugs.gentoo.org/921932#c2=
1


-- John




From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:06:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:06:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675200.1050460 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW0jr-0008LR-Jp; Fri, 02 Feb 2024 21:06:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675200.1050460; Fri, 02 Feb 2024 21:06:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW0jr-0008LK-Fm; Fri, 02 Feb 2024 21:06:19 +0000
Received: by outflank-mailman (input) for mailman id 675200;
 Fri, 02 Feb 2024 21:06:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rW0jp-0008LA-Kp; Fri, 02 Feb 2024 21:06:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rW0jp-0000tH-F0; Fri, 02 Feb 2024 21:06:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rW0jo-00063A-V9; Fri, 02 Feb 2024 21:06:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rW0jo-0000pM-Uf; Fri, 02 Feb 2024 21:06:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=w6Km7wddHMIGjjFG9HF5gZt0Y4Q6G+MwsqXMIQpN03M=; b=Yzl1uWzcuCoyKuMxh5IMgVvh51
	XLFOjU9CQJmyQ6sf8K5RvElsMrPIw+jWqjEKPzXNZqKgV0SRXmmNvYkuMdvGetQCyxn2CWPI22Lgn
	qP1ElmbJKWjGiZKQRPIQRUtujfqJ4Xiz3CGbms2zzes0nmdpiJsPU3V3Zn0wOMU2ibSU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184562-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184562: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=5e95cedbb2c5d5af049110369c627f90bbdb0799
X-Osstest-Versions-That:
    libvirt=2757e91c2b28b704d9a0b586fb60012450110b1a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 02 Feb 2024 21:06:16 +0000

flight 184562 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184562/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184537
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184537
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184537
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              5e95cedbb2c5d5af049110369c627f90bbdb0799
baseline version:
 libvirt              2757e91c2b28b704d9a0b586fb60012450110b1a

Last test of basis   184537  2024-01-31 04:24:45 Z    2 days
Failing since        184548  2024-02-01 06:57:31 Z    1 days    2 attempts
Testing same since   184562  2024-02-02 04:20:30 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrea Bolognani <abologna@redhat.com>
  Göran Uddeborg <goeran@uddeborg.se>
  Jiri Denemark <jdenemar@redhat.com>
  Peter Krempa <pkrempa@redhat.com>
  Stefano Brivio <sbrivio@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   2757e91c2b..5e95cedbb2  5e95cedbb2c5d5af049110369c627f90bbdb0799 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:34:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:34:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675206.1050470 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Ag-0003tW-Iz; Fri, 02 Feb 2024 21:34:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675206.1050470; Fri, 02 Feb 2024 21:34:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Ag-0003tP-FI; Fri, 02 Feb 2024 21:34:02 +0000
Received: by outflank-mailman (input) for mailman id 675206;
 Fri, 02 Feb 2024 21:34:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1Ae-0003tJ-Pg
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:34:00 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c590aee7-c212-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:33:58 +0100 (CET)
Received: from DM6PR02CA0142.namprd02.prod.outlook.com (2603:10b6:5:332::9) by
 SJ0PR12MB6928.namprd12.prod.outlook.com (2603:10b6:a03:47a::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7202.32; Fri, 2 Feb
 2024 21:33:54 +0000
Received: from CY4PEPF0000E9DB.namprd05.prod.outlook.com
 (2603:10b6:5:332:cafe::67) by DM6PR02CA0142.outlook.office365.com
 (2603:10b6:5:332::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.30 via Frontend
 Transport; Fri, 2 Feb 2024 21:33:54 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9DB.mail.protection.outlook.com (10.167.241.81) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 21:33:53 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:33:53 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:33:52 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:33:51 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c590aee7-c212-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=aqZg1QjxG6gen1ImyHBNAgwJZWsdiV8Ybz9BdoletmRz6J6SFmo8RiV6phBysyVlDqybpjoEtA9JnZn1ihaIbfmDG8tBWpFhZMxhZXtgUd8Nz2R4XZdoGGiw16c4VGQ2chVm/8/97KHvehY3GsEYhyf1CeQIbhCpkkdKg8YfK8X6piJiN+r/TubOr4XfimUOkfEyRKsUXkqYycgOgPnSMp+AYmzXW9c24rZOfJHKnXPJdEtnsbhSngLy1uI0IDKHvGmrNDPWyiRBZ1ivYctqVnlNvBiPDixkbqRUiTRoiOU+BahqJNilTELVNk000VrQA4ni8pOyAW8DBBdlrk1NLw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=IZoKI2DoN+ryFAYH0HfQvHMden9FV5PzUplCnc4MtLE=;
 b=g3s3+ivk6PTR9k15CWvapG/bB1lNwaBAKXbmIAulPol5lrVPPAdqkdVaFHjMG0RXXWQ9W2Ydb3eXwlJqqZqJxyd58Yd5/NngB7G7j9jbdK2YMwizDtDyjXY8UqOEweA2AQ9ungceYsUirwkSTYPJooK26w90pZo81jl6K5ZxB9D0QG8ZY0emJh8JjF3KEDQi0nZeOUEuKzstfySmel4XyqSE184RevUUheLn0kjtP1TtiFiB/uQDiLPd83LLFE4LKBs3hwwJQTyFd2tbVX0nTdBB4CLyQ094PZZ962mj48yJGwfR2hc1SCYOZxvs5SPZRGnVDWLo7ubEJGZmpiNR4g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IZoKI2DoN+ryFAYH0HfQvHMden9FV5PzUplCnc4MtLE=;
 b=v18zJVGPGPyBhzbdp3qTTA8zQoybFVaOk/2E1PTC9QuckMvquo9XZiJ7uc2kGufqdfuyZt4x3r+baTkXuSBIItHKF4o89+AGoLqSJ+ag6xybwmssWNNBaGS/aJhnEFMdsquW+g+1XWRCGH/flKgXOVotOMHRNdHY+n7aF/cccPA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stewart Hildebrand <stewart.hildebrand@amd.com>, Jan Beulich
	<jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Jun Nakajima
	<jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>, Paul Durrant
	<paul@xen.org>, "Daniel P. Smith" <dpsmith@apertussolutions.com>, "Bertrand
 Marquis" <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v13 00/14] PCI devices passthrough on Arm, part 3
Date: Fri, 2 Feb 2024 16:33:04 -0500
Message-ID: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9DB:EE_|SJ0PR12MB6928:EE_
X-MS-Office365-Filtering-Correlation-Id: a8090217-e44a-41c2-9df9-08dc2436a7a2
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	75tiX/DzmopYVrcNiO7o6GccTayUcd19i3hPfxJgnTvCYM878jsu36rlNolnAtmqJXfzHtnk+U0KRnYWy095NQlBvs9KNy2Dac785L3aqhAV5rk6v7e1y2bIoMdmklq+k2Q2dgNxxTpA5VKDYYdEUDmS5zrcZKGeF2bnJRv+ECUGEASzfwYAtmg7XqeHWZJ2VvZv7Lk/N1ldXdYcFt4nEpYiz6C6pgVbGOeKf01Y7XlhI6banw9zNql9szk6T0ihbYH8S6353wgvjnw2ykT7cJt4umFoqaP6YHEwXoFAyESdgFkSObE5pfrZxcww6GZ0DV+xH9ZHCKQpoKApdtAYEomPatXs1FfEeO7HgH61NnYStX8qKl9YgLg9eog1giVeK5xF+ooe5VIt1ss2j+GUPGqOqoVWf4ACGl5XWkMig9BXCtVMJyK25vwAHrefXAflkvMsf7KpJc7akr5qkektC5N5g8pXI5NuFmTBkSDKceNl5hVRadQeoe1O4PvWgJfO5jIyY7Ku1c5Ofb0rqvURE85WWfQXBU7CcO7lC20CQLJP9hJVeh5fsnFE9KGiTHq6RogKkIT1/LrGiIOBI5RbBZCbNWXBqhaeC2RhYPQq3SXtLg44JYSNOqa+pEkKEczvzf/Hokw7QZAJwYftFGISM+pKnLdkZNATELznnFfiRRLdPlFd7j6zEj8J2yYu7Ax/iA58J4dsiqaC/XlvrM7Z+fAxjEZ1Hhy+YyT4VR1jBIUOjAvQvpiO+GjLTCSVs3TDj15zaGF0Ff7U20cp6OcpKQ==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(39860400002)(346002)(396003)(376002)(230922051799003)(186009)(451199024)(64100799003)(82310400011)(1800799012)(36840700001)(40470700004)(46966006)(26005)(336012)(1076003)(426003)(6666004)(478600001)(36860700001)(82740400003)(41300700001)(2616005)(8676002)(44832011)(4326008)(70586007)(6916009)(8936002)(70206006)(5660300002)(54906003)(316002)(2906002)(83380400001)(7416002)(40480700001)(36756003)(356005)(40460700003)(81166007)(86362001)(47076005)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:33:53.7960
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a8090217-e44a-41c2-9df9-08dc2436a7a2
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9DB.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6928

This is next version of vPCI rework. Aim of this series is to prepare
ground for introducing PCI support on ARM platform.

in v13:
 - drop ("xen/arm: vpci: permit access to guest vpci space") as it was
   unnecessary

in v12:
 - I (Stewart) coordinated with Volodomyr to send this whole series. So,
   add my (Stewart) Signed-off-by to all patches.
 - The biggest change is to re-work the PCI_COMMAND register patch.
   Additional feedback has also been addressed - see individual patches.
 - Drop ("pci: msi: pass pdev to pci_enable_msi() function") and
   ("pci: introduce per-domain PCI rwlock") as they were committed
 - Rename ("rangeset: add rangeset_empty() function")
       to ("rangeset: add rangeset_purge() function")
 - Rename ("vpci/header: rework exit path in init_bars")
       to ("vpci/header: rework exit path in init_header()")

in v11:
 - Added my (Volodymyr) Signed-off-by tag to all patches
 - Patch "vpci/header: emulate PCI_COMMAND register for guests" is in
   intermediate state, because it was agreed to rework it once Stewart's
   series on register handling are in.
 - Addressed comments, please see patch descriptions for details.

in v10:

 - Removed patch ("xen/arm: vpci: check guest range"), proper fix
   for the issue is part of ("vpci/header: emulate PCI_COMMAND
   register for guests")
 - Removed patch ("pci/header: reset the command register when adding
   devices")
 - Added patch ("rangeset: add rangeset_empty() function") because
   this function is needed in ("vpci/header: handle p2m range sets
   per BAR")
 - Added ("vpci/header: handle p2m range sets per BAR") which addressed
   an issue discovered by Andrii Chepurnyi during virtio integration
 - Added ("pci: msi: pass pdev to pci_enable_msi() function"), which is
   prereq for ("pci: introduce per-domain PCI rwlock")
 - Fixed "Since v9/v8/... " comments in changelogs to reduce confusion.
   I left "Since" entries for older versions, because they were added
   by original author of the patches.

in v9:

v9 includes addressed commentes from a previous one. Also it
introduces a couple patches from Stewart. This patches are related to
vPCI use on ARM. Patch "vpci/header: rework exit path in init_bars"
was factored-out from "vpci/header: handle p2m range sets per BAR".

in v8:

The biggest change from previous, mistakenly named, v7 series is how
locking is implemented. Instead of d->vpci_rwlock we introduce
d->pci_lock which has broader scope, as it protects not only domain's
vpci state, but domain's list of PCI devices as well.

As we discussed in IRC with Roger, it is not feasible to rework all
the existing code to use the new lock right away. It was agreed that
any write access to d->pdev_list will be protected by **both**
d->pci_lock in write mode and pcidevs_lock(). Read access on other
hand should be protected by either d->pci_lock in read mode or
pcidevs_lock(). It is expected that existing code will use
pcidevs_lock() and new users will use new rw lock. Of course, this
does not mean that new users shall not use pcidevs_lock() when it is
appropriate.

Changes from previous versions are described in each separate patch.

Oleksandr Andrushchenko (11):
  vpci: use per-domain PCI lock to protect vpci structure
  vpci: restrict unhandled read/write operations for guests
  vpci: add hooks for PCI device assign/de-assign
  vpci/header: implement guest BAR register handlers
  rangeset: add RANGESETF_no_print flag
  vpci/header: handle p2m range sets per BAR
  vpci/header: program p2m with guest BAR view
  vpci/header: emulate PCI_COMMAND register for guests
  vpci: add initial support for virtual PCI bus topology
  xen/arm: translate virtual PCI bus topology for guests
  xen/arm: account IO handlers for emulated PCI MSI-X

Volodymyr Babchuk (3):
  vpci/header: rework exit path in init_header()
  rangeset: add rangeset_purge() function
  arm/vpci: honor access size when returning an error

 xen/arch/arm/vpci.c           |  63 ++++-
 xen/arch/x86/hvm/vmsi.c       |  31 ++-
 xen/arch/x86/hvm/vmx/vmx.c    |   2 +-
 xen/arch/x86/irq.c            |   8 +-
 xen/arch/x86/msi.c            |  20 +-
 xen/arch/x86/physdev.c        |   2 +
 xen/common/rangeset.c         |  21 +-
 xen/drivers/Kconfig           |   4 +
 xen/drivers/passthrough/pci.c |  35 ++-
 xen/drivers/vpci/header.c     | 495 +++++++++++++++++++++++++++-------
 xen/drivers/vpci/msi.c        |  37 ++-
 xen/drivers/vpci/msix.c       |  59 +++-
 xen/drivers/vpci/vpci.c       | 125 ++++++++-
 xen/include/xen/pci_regs.h    |   1 +
 xen/include/xen/rangeset.h    |   8 +-
 xen/include/xen/sched.h       |  25 +-
 xen/include/xen/vpci.h        |  45 +++-
 17 files changed, 808 insertions(+), 173 deletions(-)


base-commit: 3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:34:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:34:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675208.1050480 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Aw-0004DL-0m; Fri, 02 Feb 2024 21:34:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675208.1050480; Fri, 02 Feb 2024 21:34:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Av-0004DE-TG; Fri, 02 Feb 2024 21:34:17 +0000
Received: by outflank-mailman (input) for mailman id 675208;
 Fri, 02 Feb 2024 21:34:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1At-0004Ae-In
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:34:15 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2409::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ce299130-c212-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 22:34:13 +0100 (CET)
Received: from CY5P221CA0005.NAMP221.PROD.OUTLOOK.COM (2603:10b6:930:b::17) by
 MW3PR12MB4380.namprd12.prod.outlook.com (2603:10b6:303:5a::19) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7270.11; Fri, 2 Feb 2024 21:34:07 +0000
Received: from CY4PEPF0000E9D7.namprd05.prod.outlook.com
 (2603:10b6:930:b:cafe::bc) by CY5P221CA0005.outlook.office365.com
 (2603:10b6:930:b::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.26 via Frontend
 Transport; Fri, 2 Feb 2024 21:34:07 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D7.mail.protection.outlook.com (10.167.241.78) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 21:34:06 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:34:05 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:34:04 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce299130-c212-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LSA2ldg0jJmljnpoOe9T5Z9S0suH/qKrPwSY2qe4G5jzThpAj8SzhtIYo8+oxFf9VaUCqJ+s3n+0FWF/mSXKgWptEooTEy/OvNMDfviGRmniHxMAXShvPnKg8rtpu47MCgIcNUtETmLifJuXJFxSkEFNmdnInitcACZN8mhD8imYtCjl4+fm/sFUfRLnJ2H3pa49XExPDQFW+IbZsBIv9rUokolQoLFldPiR9j87/lZ7j5ooTeJtuGVjmOCPN44I8kacFG9l73cRB84NDi+B8ezVY/Pv4Sg3SWfXAegzqA+wB9YRRvx3VRZL+6FYlo66Gq7eDtsJOwNaqPueVJvQ5A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=H3tS51EWeLyplN/iiO2R/OIVC2P7UkDQqG6IT45GkZU=;
 b=EnDMgCXzO0MCscxAcKXVGUQ20qmeYutRKKJonjUj0m1vPeScJzR2Mckukr3jKEU2QwpHLZUiXXJi1lfdfGSjmqoc5DtcByJkAG/2LQf6BrA5Q0iRWI7qZCf15y951EiceLRwxL7OxAnhiYnajVBGW3LPJYmEB0kd5XXHVYR90OuDe+4iMqtpSZVKXrBDlc6I8PahmfrEPvpJsWOBAQFqiRNqTDR7cc2Vi1B/gZ+oPMJb8qNnOy5rYgX+14ofGiyRJKP1ojwWRQptw68lJJHxs1h9WtZ7vAcbi5jkJCDmJ4wkVvOvKrNYzQwfHcJwCickRIaluDLsulGEwdowyoHS9A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=H3tS51EWeLyplN/iiO2R/OIVC2P7UkDQqG6IT45GkZU=;
 b=BbVGTg4g8dIi2wsuPfIMOApmaS1tEyrEBInF7Ah8FFYODgeTjqht9bhx6ADNjNRSjTxalhzdIkhiP1VSA5ARLZ6TEERVS9GMVwTBiLgR0v77ly5qy4qlQMlw03hgnw1YciIt7+y1YXZamoRK3MvkFKiiP4ECQoszrpXRDsWWZRI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Jan Beulich
	<jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Jun Nakajima
	<jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>, Paul Durrant
	<paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>
Subject: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci structure
Date: Fri, 2 Feb 2024 16:33:05 -0500
Message-ID: <20240202213321.1920347-2-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D7:EE_|MW3PR12MB4380:EE_
X-MS-Office365-Filtering-Correlation-Id: 00f4f925-5f80-477b-8f2e-08dc2436af56
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wTQslJ6PtW5okJLReliSa9juXs3C+LdSaOTnJVOHZW+3eK3H2Mfx+ZNh6Kgx7V1HwKp4OcD5DQTzZGECRP17EJmkK8SOt/Da+A2AYHvhk0TW9eyjLSisIc9dp1SJJT5qqQrxxZds+jCfMFHK+enOpww5qApAGxy761qjYi6YKwRfW/EpA6OIYB6PXR7aYXW4ZXyrYXFx+OlbT3CnKOAgIOWHKKSHJNLtmDVKCpJfk2kTl2C2hoAoOQnc6zASTqRqd97T87ss062NoDpg+TaIYKQdtSKdxaQjYaaXxzGGrIyfrMk5SmrDavzyA9pwqULYNq+vunF1xDdGcP/eNcKX7v5bymph90oWbcqxZqKKa2bv5m0beohhREpSRFYDaSUQ+YINwHid1Z/Vh0QQcwopXEMzrBPicFstRPj/vls8psCFUk6dLrb5Yfo3PNF3q2efl4xBMZtvaJE2I5WIIHqfUtPmZTW/YBB5iANrGW0WANZFQeeL4ygDB4lAQ3/0NbjCCswO/f8XCo4rUnoAwZW/FzwDF1opqNB8sRmb/nb/6qxE1WiJMPzPgO3OPNBn0wxUhVZpCFCxkRurV8OsG+30Dl6sYKxHvHVgQDqJzEnFArx7XxBG2dYSYcoXfau0bwtmE+J1AjPaF8mVtx/VdX7tAvkomRioXpoyX1NniytY2D2jtObuX87O94EqG3vsl2HWx/ZyQh5UiuhLM5qsUGFFY4gC4lYlso6dzYIZdPNVWHJYeVsAAcsDMetiBdw0jXi4cYi6ziTucP/wTE0NY09E9/nyFMOuisezeWZByPP09aE=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(136003)(39860400002)(376002)(396003)(230922051799003)(82310400011)(64100799003)(186009)(451199024)(1800799012)(36840700001)(46966006)(40470700004)(83380400001)(81166007)(336012)(47076005)(2616005)(26005)(30864003)(36860700001)(426003)(1076003)(8936002)(5660300002)(7416002)(8676002)(44832011)(4326008)(6916009)(316002)(478600001)(2906002)(54906003)(70586007)(70206006)(82740400003)(356005)(36756003)(41300700001)(86362001)(40460700003)(40480700001)(36900700001)(309714004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:34:06.7349
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 00f4f925-5f80-477b-8f2e-08dc2436af56
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D7.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4380

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Use the per-domain PCI read/write lock to protect the presence of the
pci device vpci field. This lock can be used (and in a few cases is used
right away) so that vpci removal can be performed while holding the lock
in write mode. Previously such removal could race with vpci_read for
example.

When taking both d->pci_lock and pdev->vpci->lock, they should be
taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
possible deadlock situations.

1. Per-domain's pci_lock is used to protect pdev->vpci structure
from being removed.

2. Writing the command register and ROM BAR register may trigger
modify_bars to run, which in turn may access multiple pdevs while
checking for the existing BAR's overlap. The overlapping check, if
done under the read lock, requires vpci->lock to be acquired on both
devices being compared, which may produce a deadlock. It is not
possible to upgrade read lock to write lock in such a case. So, in
order to prevent the deadlock, use d->pci_lock in write mode instead.

All other code, which doesn't lead to pdev->vpci destruction and does
not access multiple pdevs at the same time, can still use a
combination of the read lock and pdev->vpci->lock.

3. Drop const qualifier where the new rwlock is used and this is
appropriate.

4. Do not call process_pending_softirqs with any locks held. For that
unlock prior the call and re-acquire the locks after. After
re-acquiring the lock there is no need to check if pdev->vpci exists:
 - in apply_map because of the context it is called (no race condition
   possible)
 - for MSI/MSI-X debug code because it is called at the end of
   pdev->vpci access and no further access to pdev->vpci is made

5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
while accessing pdevs in vpci code.

6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
do not go away. The vPCI functions call several MSI-related functions
which already have existing non-vPCI callers. Change those MSI-related
functions to allow using either pcidevs_lock() or d->pci_lock for
ensuring pdevs do not go away. Holding d->pci_lock in read mode is
sufficient. Note that this pdev protection mechanism does not protect
other state or critical sections. These MSI-related functions already
have other race condition and state protection mechanims (e.g.
d->event_lock and msixtbl RCU), so we deduce that the use of the global
pcidevs_lock() is to ensure that pdevs do not go away. Existing non-vPCI
callers of these MSI-related functions will remain (ab)using the global
pcidevs_lock() to ensure pdevs do not go away so as to minimize changes
to existing non-vPCI call paths.

7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
that pdevs do not go away. The purpose of this wrapper is to aid
readability and document the intent of the pdev protection mechanism.

Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
Changes in v13:
 - hold off adding Roger's R-b tag even though it was provided on v12.2
 - use a wrapper construct to ease readability of odd-looking ASSERTs
 - new placement of ASSERT in __pci_enable_msix(), __pci_enable_msi(),
   and pci_enable_msi(). Rearrange/add pdev NULL check.
 - expand commit description with details about using either
   pcidevs_lock() or d->pci_lock

Changes in v12.2:
 - drop Roger's R-b
 - drop both locks on error paths in vpci_msix_arch_print()
 - add another ASSERT in vpci_msix_arch_print(), to enforce the
   expectation both locks are held before calling vpci_msix_arch_print()
 - move pdev_done label in vpci_dump_msi()
 - update comments in vpci_dump_msi() to say locks (plural)

Changes in v12.1:
 - use read_trylock() in vpci_msix_arch_print()
 - fixup in-code comments (revert double space, use DomXEN) in
   vpci_{read,write}()
 - minor updates in commit message
 - add Roger's R-b

Changes in v12:
 - s/pci_rwlock/pci_lock/ in commit message
 - expand comment about scope of pci_lock in sched.h
 - in vpci_{read,write}, if hwdom is trying to access a device assigned
   to dom_xen, holding hwdom->pci_lock is sufficient (no need to hold
   dom_xen->pci_lock)
 - reintroduce ASSERT in vmx_pi_update_irte()
 - reintroduce ASSERT in __pci_enable_msi{x}()
 - delete note 6. in commit message about removing ASSERTs since we have
   reintroduced them

Changes in v11:
 - Fixed commit message regarding possible spinlocks
 - Removed parameter from allocate_and_map_msi_pirq(), which was added
 in the prev version. Now we are taking pcidevs_lock in
 physdev_map_pirq()
 - Returned ASSERT to pci_enable_msi
 - Fixed case when we took read lock instead of write one
 - Fixed label indentation

Changes in v10:
 - Moved printk pas locked area
 - Returned back ASSERTs
 - Added new parameter to allocate_and_map_msi_pirq() so it knows if
 it should take the global pci lock
 - Added comment about possible improvement in vpci_write
 - Changed ASSERT(rw_is_locked()) to rw_is_write_locked() in
   appropriate places
 - Renamed release_domain_locks() to release_domain_write_locks()
 - moved domain_done label in vpci_dump_msi() to correct place
Changes in v9:
 - extended locked region to protect vpci_remove_device and
   vpci_add_handlers() calls
 - vpci_write() takes lock in the write mode to protect
   potential call to modify_bars()
 - renamed lock releasing function
 - removed ASSERT()s from msi code
 - added trylock in vpci_dump_msi

Changes in v8:
 - changed d->vpci_lock to d->pci_lock
 - introducing d->pci_lock in a separate patch
 - extended locked region in vpci_process_pending
 - removed pcidevs_lockis vpci_dump_msi()
 - removed some changes as they are not needed with
   the new locking scheme
 - added handling for hwdom && dom_xen case
---
 xen/arch/x86/hvm/vmsi.c       | 31 +++++++++++++--------
 xen/arch/x86/hvm/vmx/vmx.c    |  2 +-
 xen/arch/x86/irq.c            |  8 +++---
 xen/arch/x86/msi.c            | 20 +++++++++-----
 xen/arch/x86/physdev.c        |  2 ++
 xen/drivers/passthrough/pci.c |  9 +++---
 xen/drivers/vpci/header.c     | 18 ++++++++++++
 xen/drivers/vpci/msi.c        | 30 +++++++++++++++++---
 xen/drivers/vpci/msix.c       | 52 ++++++++++++++++++++++++++++++-----
 xen/drivers/vpci/vpci.c       | 24 ++++++++++++++--
 xen/include/xen/sched.h       | 15 +++++++++-
 11 files changed, 170 insertions(+), 41 deletions(-)

diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index 128f23636279..f29089178a59 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -468,7 +468,7 @@ int msixtbl_pt_register(struct domain *d, struct pirq *pirq, uint64_t gtable)
     struct msixtbl_entry *entry, *new_entry;
     int r = -EINVAL;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(d));
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     if ( !msixtbl_initialised(d) )
@@ -538,7 +538,7 @@ void msixtbl_pt_unregister(struct domain *d, struct pirq *pirq)
     struct pci_dev *pdev;
     struct msixtbl_entry *entry;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(d));
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     if ( !msixtbl_initialised(d) )
@@ -684,7 +684,7 @@ static int vpci_msi_update(const struct pci_dev *pdev, uint32_t data,
 {
     unsigned int i;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(rw_is_locked(&pdev->domain->pci_lock));
 
     if ( (address & MSI_ADDR_BASE_MASK) != MSI_ADDR_HEADER )
     {
@@ -725,8 +725,8 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev)
     int rc;
 
     ASSERT(msi->arch.pirq != INVALID_PIRQ);
+    ASSERT(rw_is_locked(&pdev->domain->pci_lock));
 
-    pcidevs_lock();
     for ( i = 0; i < msi->vectors && msi->arch.bound; i++ )
     {
         struct xen_domctl_bind_pt_irq unbind = {
@@ -745,7 +745,6 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev)
 
     msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address,
                                        msi->vectors, msi->arch.pirq, msi->mask);
-    pcidevs_unlock();
 }
 
 static int vpci_msi_enable(const struct pci_dev *pdev, unsigned int nr,
@@ -778,15 +777,14 @@ int vpci_msi_arch_enable(struct vpci_msi *msi, const struct pci_dev *pdev,
     int rc;
 
     ASSERT(msi->arch.pirq == INVALID_PIRQ);
+    ASSERT(rw_is_locked(&pdev->domain->pci_lock));
     rc = vpci_msi_enable(pdev, vectors, 0);
     if ( rc < 0 )
         return rc;
     msi->arch.pirq = rc;
 
-    pcidevs_lock();
     msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address, vectors,
                                        msi->arch.pirq, msi->mask);
-    pcidevs_unlock();
 
     return 0;
 }
@@ -797,8 +795,8 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq,
     unsigned int i;
 
     ASSERT(pirq != INVALID_PIRQ);
+    ASSERT(rw_is_locked(&pdev->domain->pci_lock));
 
-    pcidevs_lock();
     for ( i = 0; i < nr && bound; i++ )
     {
         struct xen_domctl_bind_pt_irq bind = {
@@ -814,7 +812,6 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq,
     write_lock(&pdev->domain->event_lock);
     unmap_domain_pirq(pdev->domain, pirq);
     write_unlock(&pdev->domain->event_lock);
-    pcidevs_unlock();
 }
 
 void vpci_msi_arch_disable(struct vpci_msi *msi, const struct pci_dev *pdev)
@@ -854,6 +851,7 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
     int rc;
 
     ASSERT(entry->arch.pirq == INVALID_PIRQ);
+    ASSERT(rw_is_locked(&pdev->domain->pci_lock));
     rc = vpci_msi_enable(pdev, vmsix_entry_nr(pdev->vpci->msix, entry),
                          table_base);
     if ( rc < 0 )
@@ -861,7 +859,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
 
     entry->arch.pirq = rc;
 
-    pcidevs_lock();
     rc = vpci_msi_update(pdev, entry->data, entry->addr, 1, entry->arch.pirq,
                          entry->masked);
     if ( rc )
@@ -869,7 +866,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
         vpci_msi_disable(pdev, entry->arch.pirq, 1, false);
         entry->arch.pirq = INVALID_PIRQ;
     }
-    pcidevs_unlock();
 
     return rc;
 }
@@ -895,6 +891,9 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
 {
     unsigned int i;
 
+    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
+    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));
+
     for ( i = 0; i < msix->max_entries; i++ )
     {
         const struct vpci_msix_entry *entry = &msix->entries[i];
@@ -913,13 +912,23 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
             struct pci_dev *pdev = msix->pdev;
 
             spin_unlock(&msix->pdev->vpci->lock);
+            read_unlock(&pdev->domain->pci_lock);
             process_pending_softirqs();
+
+            if ( !read_trylock(&pdev->domain->pci_lock) )
+                return -EBUSY;
+
             /* NB: we assume that pdev cannot go away for an alive domain. */
             if ( !pdev->vpci || !spin_trylock(&pdev->vpci->lock) )
+            {
+                read_unlock(&pdev->domain->pci_lock);
                 return -EBUSY;
+            }
+
             if ( pdev->vpci->msix != msix )
             {
                 spin_unlock(&pdev->vpci->lock);
+                read_unlock(&pdev->domain->pci_lock);
                 return -EAGAIN;
             }
         }
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 1500dca6039f..5aac4863d30a 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -413,7 +413,7 @@ static int cf_check vmx_pi_update_irte(const struct vcpu *v,
 
     spin_unlock_irq(&desc->lock);
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(msi_desc->dev->domain));
 
     return iommu_update_ire_from_msi(msi_desc, &msi_desc->msg);
 
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index bbae7751e494..e31144d82fd4 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2164,7 +2164,7 @@ int map_domain_pirq(
         struct pci_dev *pdev;
         unsigned int nr = 0;
 
-        ASSERT(pcidevs_locked());
+        ASSERT(pdev_list_is_read_locked(d));
 
         ret = -ENODEV;
         if ( !cpu_has_apic )
@@ -2321,7 +2321,7 @@ int unmap_domain_pirq(struct domain *d, int pirq)
     if ( (pirq < 0) || (pirq >= d->nr_pirqs) )
         return -EINVAL;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(d));
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     info = pirq_info(d, pirq);
@@ -2886,6 +2886,8 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 {
     int irq, pirq, ret;
 
+    ASSERT(pdev_list_is_read_locked(d));
+
     switch ( type )
     {
     case MAP_PIRQ_TYPE_MSI:
@@ -2915,7 +2917,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 
     msi->irq = irq;
 
-    pcidevs_lock();
     /* Verify or get pirq. */
     write_lock(&d->event_lock);
     pirq = allocate_pirq(d, index, *pirq_p, irq, type, &msi->entry_nr);
@@ -2931,7 +2932,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 
  done:
     write_unlock(&d->event_lock);
-    pcidevs_unlock();
     if ( ret )
     {
         switch ( type )
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index 335c0868a225..e008b6789a28 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -602,7 +602,7 @@ static int msi_capability_init(struct pci_dev *dev,
     unsigned int i, mpos;
     uint16_t control;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(dev->domain));
     pos = pci_find_cap_offset(dev->sbdf, PCI_CAP_ID_MSI);
     if ( !pos )
         return -ENODEV;
@@ -771,7 +771,7 @@ static int msix_capability_init(struct pci_dev *dev,
     if ( !pos )
         return -ENODEV;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(dev->domain));
 
     control = pci_conf_read16(dev->sbdf, msix_control_reg(pos));
     /*
@@ -988,11 +988,11 @@ static int __pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi,
 {
     struct msi_desc *old_desc;
 
-    ASSERT(pcidevs_locked());
-
     if ( !pdev )
         return -ENODEV;
 
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
+
     old_desc = find_msi_entry(pdev, msi->irq, PCI_CAP_ID_MSI);
     if ( old_desc )
     {
@@ -1043,9 +1043,12 @@ static int __pci_enable_msix(struct pci_dev *pdev, struct msi_info *msi,
 {
     struct msi_desc *old_desc;
 
-    ASSERT(pcidevs_locked());
+    if ( !pdev )
+        return -ENODEV;
+
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
 
-    if ( !pdev || !pdev->msix )
+    if ( !pdev->msix )
         return -ENODEV;
 
     if ( msi->entry_nr >= pdev->msix->nr_entries )
@@ -1154,7 +1157,10 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off)
 int pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi,
                    struct msi_desc **desc)
 {
-    ASSERT(pcidevs_locked());
+    if ( !pdev )
+        return -ENODEV;
+
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
 
     if ( !use_msi )
         return -EPERM;
diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
index 47c4da0af7e1..369c9e788c1c 100644
--- a/xen/arch/x86/physdev.c
+++ b/xen/arch/x86/physdev.c
@@ -123,7 +123,9 @@ int physdev_map_pirq(domid_t domid, int type, int *index, int *pirq_p,
 
     case MAP_PIRQ_TYPE_MSI:
     case MAP_PIRQ_TYPE_MULTI_MSI:
+        pcidevs_lock();
         ret = allocate_and_map_msi_pirq(d, *index, pirq_p, type, msi);
+        pcidevs_unlock();
         break;
 
     default:
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 47c0eee7bdcc..c97dd4504a7a 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -750,7 +750,6 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
         pdev->domain = hardware_domain;
         write_lock(&hardware_domain->pci_lock);
         list_add(&pdev->domain_list, &hardware_domain->pdev_list);
-        write_unlock(&hardware_domain->pci_lock);
 
         /*
          * For devices not discovered by Xen during boot, add vPCI handlers
@@ -759,18 +758,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
         ret = vpci_add_handlers(pdev);
         if ( ret )
         {
-            printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret);
-            write_lock(&hardware_domain->pci_lock);
             list_del(&pdev->domain_list);
             write_unlock(&hardware_domain->pci_lock);
             pdev->domain = NULL;
+            printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret);
             goto out;
         }
+        write_unlock(&hardware_domain->pci_lock);
         ret = iommu_add_device(pdev);
         if ( ret )
         {
-            vpci_remove_device(pdev);
             write_lock(&hardware_domain->pci_lock);
+            vpci_remove_device(pdev);
             list_del(&pdev->domain_list);
             write_unlock(&hardware_domain->pci_lock);
             pdev->domain = NULL;
@@ -1146,7 +1145,9 @@ static void __hwdom_init setup_one_hwdom_device(const struct setup_hwdom *ctxt,
     } while ( devfn != pdev->devfn &&
               PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn) );
 
+    write_lock(&ctxt->d->pci_lock);
     err = vpci_add_handlers(pdev);
+    write_unlock(&ctxt->d->pci_lock);
     if ( err )
         printk(XENLOG_ERR "setup of vPCI for d%d failed: %d\n",
                ctxt->d->domain_id, err);
diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 58195549d50a..8f5850b8cf6d 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -173,6 +173,7 @@ bool vpci_process_pending(struct vcpu *v)
         if ( rc == -ERESTART )
             return true;
 
+        write_lock(&v->domain->pci_lock);
         spin_lock(&v->vpci.pdev->vpci->lock);
         /* Disable memory decoding unconditionally on failure. */
         modify_decoding(v->vpci.pdev,
@@ -191,6 +192,7 @@ bool vpci_process_pending(struct vcpu *v)
              * failure.
              */
             vpci_remove_device(v->vpci.pdev);
+        write_unlock(&v->domain->pci_lock);
     }
 
     return false;
@@ -202,8 +204,20 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
     struct map_data data = { .d = d, .map = true };
     int rc;
 
+    ASSERT(rw_is_write_locked(&d->pci_lock));
+
     while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART )
+    {
+        /*
+         * It's safe to drop and reacquire the lock in this context
+         * without risking pdev disappearing because devices cannot be
+         * removed until the initial domain has been started.
+         */
+        write_unlock(&d->pci_lock);
         process_pending_softirqs();
+        write_lock(&d->pci_lock);
+    }
+
     rangeset_destroy(mem);
     if ( !rc )
         modify_decoding(pdev, cmd, false);
@@ -244,6 +258,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
     unsigned int i;
     int rc;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !mem )
         return -ENOMEM;
 
@@ -524,6 +540,8 @@ static int cf_check init_header(struct pci_dev *pdev)
     int rc;
     bool mask_cap_list = false;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     switch ( pci_conf_read8(pdev->sbdf, PCI_HEADER_TYPE) & 0x7f )
     {
     case PCI_HEADER_TYPE_NORMAL:
diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c
index a253ccbd7db7..dc71938e23f5 100644
--- a/xen/drivers/vpci/msi.c
+++ b/xen/drivers/vpci/msi.c
@@ -263,7 +263,7 @@ REGISTER_VPCI_INIT(init_msi, VPCI_PRIORITY_LOW);
 
 void vpci_dump_msi(void)
 {
-    const struct domain *d;
+    struct domain *d;
 
     rcu_read_lock(&domlist_read_lock);
     for_each_domain ( d )
@@ -275,6 +275,9 @@ void vpci_dump_msi(void)
 
         printk("vPCI MSI/MSI-X d%d\n", d->domain_id);
 
+        if ( !read_trylock(&d->pci_lock) )
+            continue;
+
         for_each_pdev ( d, pdev )
         {
             const struct vpci_msi *msi;
@@ -313,17 +316,36 @@ void vpci_dump_msi(void)
                 {
                     /*
                      * On error vpci_msix_arch_print will always return without
-                     * holding the lock.
+                     * holding the locks.
                      */
                     printk("unable to print all MSI-X entries: %d\n", rc);
-                    process_pending_softirqs();
-                    continue;
+                    goto pdev_done;
                 }
             }
 
+            /*
+             * Unlock locks to process pending softirqs. This is
+             * potentially unsafe, as d->pdev_list can be changed in
+             * meantime.
+             */
             spin_unlock(&pdev->vpci->lock);
+            read_unlock(&d->pci_lock);
+        pdev_done:
             process_pending_softirqs();
+            if ( !read_trylock(&d->pci_lock) )
+            {
+                printk("unable to access other devices for the domain\n");
+                goto domain_done;
+            }
         }
+        read_unlock(&d->pci_lock);
+    domain_done:
+        /*
+         * We need this label at the end of the loop, but some
+         * compilers might not be happy about label at the end of the
+         * compound statement so we adding an empty statement here.
+         */
+        ;
     }
     rcu_read_unlock(&domlist_read_lock);
 }
diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
index d1126a417da9..b6abab47efdd 100644
--- a/xen/drivers/vpci/msix.c
+++ b/xen/drivers/vpci/msix.c
@@ -147,6 +147,8 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr)
 {
     struct vpci_msix *msix;
 
+    ASSERT(rw_is_locked(&d->pci_lock));
+
     list_for_each_entry ( msix, &d->arch.hvm.msix_tables, next )
     {
         const struct vpci_bar *bars = msix->pdev->vpci->header.bars;
@@ -163,7 +165,13 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr)
 
 static int cf_check msix_accept(struct vcpu *v, unsigned long addr)
 {
-    return !!msix_find(v->domain, addr);
+    int rc;
+
+    read_lock(&v->domain->pci_lock);
+    rc = !!msix_find(v->domain, addr);
+    read_unlock(&v->domain->pci_lock);
+
+    return rc;
 }
 
 static bool access_allowed(const struct pci_dev *pdev, unsigned long addr,
@@ -358,21 +366,35 @@ static int adjacent_read(const struct domain *d, const struct vpci_msix *msix,
 static int cf_check msix_read(
     struct vcpu *v, unsigned long addr, unsigned int len, unsigned long *data)
 {
-    const struct domain *d = v->domain;
-    struct vpci_msix *msix = msix_find(d, addr);
+    struct domain *d = v->domain;
+    struct vpci_msix *msix;
     const struct vpci_msix_entry *entry;
     unsigned int offset;
 
     *data = ~0UL;
 
+    read_lock(&d->pci_lock);
+
+    msix = msix_find(d, addr);
     if ( !msix )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_RETRY;
+    }
 
     if ( adjacent_handle(msix, addr) )
-        return adjacent_read(d, msix, addr, len, data);
+    {
+        int rc = adjacent_read(d, msix, addr, len, data);
+
+        read_unlock(&d->pci_lock);
+        return rc;
+    }
 
     if ( !access_allowed(msix->pdev, addr, len) )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_OKAY;
+    }
 
     spin_lock(&msix->pdev->vpci->lock);
     entry = get_entry(msix, addr);
@@ -404,6 +426,7 @@ static int cf_check msix_read(
         break;
     }
     spin_unlock(&msix->pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     return X86EMUL_OKAY;
 }
@@ -491,19 +514,33 @@ static int adjacent_write(const struct domain *d, const struct vpci_msix *msix,
 static int cf_check msix_write(
     struct vcpu *v, unsigned long addr, unsigned int len, unsigned long data)
 {
-    const struct domain *d = v->domain;
-    struct vpci_msix *msix = msix_find(d, addr);
+    struct domain *d = v->domain;
+    struct vpci_msix *msix;
     struct vpci_msix_entry *entry;
     unsigned int offset;
 
+    read_lock(&d->pci_lock);
+
+    msix = msix_find(d, addr);
     if ( !msix )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_RETRY;
+    }
 
     if ( adjacent_handle(msix, addr) )
-        return adjacent_write(d, msix, addr, len, data);
+    {
+        int rc = adjacent_write(d, msix, addr, len, data);
+
+        read_unlock(&d->pci_lock);
+        return rc;
+    }
 
     if ( !access_allowed(msix->pdev, addr, len) )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_OKAY;
+    }
 
     spin_lock(&msix->pdev->vpci->lock);
     entry = get_entry(msix, addr);
@@ -579,6 +616,7 @@ static int cf_check msix_write(
         break;
     }
     spin_unlock(&msix->pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     return X86EMUL_OKAY;
 }
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 72ef277c4f8e..475272b173f3 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -42,6 +42,8 @@ extern vpci_register_init_t *const __end_vpci_array[];
 
 void vpci_remove_device(struct pci_dev *pdev)
 {
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !has_vpci(pdev->domain) || !pdev->vpci )
         return;
 
@@ -77,6 +79,8 @@ int vpci_add_handlers(struct pci_dev *pdev)
     const unsigned long *ro_map;
     int rc = 0;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !has_vpci(pdev->domain) )
         return 0;
 
@@ -361,7 +365,7 @@ static uint32_t merge_result(uint32_t data, uint32_t new, unsigned int size,
 
 uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
 {
-    const struct domain *d = current->domain;
+    struct domain *d = current->domain;
     const struct pci_dev *pdev;
     const struct vpci_register *r;
     unsigned int data_offset = 0;
@@ -376,12 +380,18 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
     /*
      * Find the PCI dev matching the address, which for hwdom also requires
      * consulting DomXEN.  Passthrough everything that's not trapped.
+     * If this is hwdom and the device is assigned to DomXEN, acquiring hwdom's
+     * pci_lock is sufficient.
      */
+    read_lock(&d->pci_lock);
     pdev = pci_get_pdev(d, sbdf);
     if ( !pdev && is_hardware_domain(d) )
         pdev = pci_get_pdev(dom_xen, sbdf);
     if ( !pdev || !pdev->vpci )
+    {
+        read_unlock(&d->pci_lock);
         return vpci_read_hw(sbdf, reg, size);
+    }
 
     spin_lock(&pdev->vpci->lock);
 
@@ -428,6 +438,7 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
         ASSERT(data_offset < size);
     }
     spin_unlock(&pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     if ( data_offset < size )
     {
@@ -470,7 +481,7 @@ static void vpci_write_helper(const struct pci_dev *pdev,
 void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
                 uint32_t data)
 {
-    const struct domain *d = current->domain;
+    struct domain *d = current->domain;
     const struct pci_dev *pdev;
     const struct vpci_register *r;
     unsigned int data_offset = 0;
@@ -484,7 +495,13 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
     /*
      * Find the PCI dev matching the address, which for hwdom also requires
      * consulting DomXEN.  Passthrough everything that's not trapped.
+     * If this is hwdom and the device is assigned to DomXEN, acquiring hwdom's
+     * pci_lock is sufficient.
+     *
+     * TODO: We need to take pci_locks in exclusive mode only if we
+     * are modifying BARs, so there is a room for improvement.
      */
+    write_lock(&d->pci_lock);
     pdev = pci_get_pdev(d, sbdf);
     if ( !pdev && is_hardware_domain(d) )
         pdev = pci_get_pdev(dom_xen, sbdf);
@@ -493,6 +510,8 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
         /* Ignore writes to read-only devices, which have no ->vpci. */
         const unsigned long *ro_map = pci_get_ro_map(sbdf.seg);
 
+        write_unlock(&d->pci_lock);
+
         if ( !ro_map || !test_bit(sbdf.bdf, ro_map) )
             vpci_write_hw(sbdf, reg, size, data);
         return;
@@ -534,6 +553,7 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
         ASSERT(data_offset < size);
     }
     spin_unlock(&pdev->vpci->lock);
+    write_unlock(&d->pci_lock);
 
     if ( data_offset < size )
         /* Tailing gap, write the remaining. */
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 9da91e0e6244..c3adec1aca3c 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -462,7 +462,8 @@ struct domain
 #ifdef CONFIG_HAS_PCI
     struct list_head pdev_list;
     /*
-     * pci_lock protects access to pdev_list.
+     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
+     * structure from being removed.
      *
      * Any user *reading* from pdev_list, or from devices stored in pdev_list,
      * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,
@@ -628,6 +629,18 @@ struct domain
     unsigned int cdf;
 };
 
+/*
+ * Check for use in ASSERTs to ensure that:
+ *   1. we can *read* d->pdev_list
+ *   2. pdevs (belonging to this domain) do not go away
+ *   3. pdevs (belonging to this domain) do not get assigned to other domains
+ * This check is not suitable for protecting other state or critical regions.
+ */
+#define pdev_list_is_read_locked(d) ({                           \
+        struct domain *d_ = (d);                                 \
+        pcidevs_locked() || (d_ && rw_is_locked(&d_->pci_lock)); \
+    })
+
 static inline struct page_list_head *page_to_list(
     struct domain *d, const struct page_info *pg)
 {
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:34:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:34:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675210.1050490 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1B3-0004cB-D0; Fri, 02 Feb 2024 21:34:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675210.1050490; Fri, 02 Feb 2024 21:34:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1B3-0004c4-9O; Fri, 02 Feb 2024 21:34:25 +0000
Received: by outflank-mailman (input) for mailman id 675210;
 Fri, 02 Feb 2024 21:34:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1B2-0003tJ-ND
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:34:24 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d46e9809-c212-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:34:24 +0100 (CET)
Received: from CY5PR15CA0096.namprd15.prod.outlook.com (2603:10b6:930:7::11)
 by IA0PR12MB8304.namprd12.prod.outlook.com (2603:10b6:208:3dc::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.22; Fri, 2 Feb
 2024 21:34:18 +0000
Received: from CY4PEPF0000E9DA.namprd05.prod.outlook.com
 (2603:10b6:930:7:cafe::28) by CY5PR15CA0096.outlook.office365.com
 (2603:10b6:930:7::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.30 via Frontend
 Transport; Fri, 2 Feb 2024 21:34:18 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9DA.mail.protection.outlook.com (10.167.241.79) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 21:34:17 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:34:17 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Fri, 2 Feb
 2024 13:34:17 -0800
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:34:16 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d46e9809-c212-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gQwFfosgWnZmhUB4LiYM9HAJhM8YvsX3Mw1YVghC3BjWiiE+ZjV0U994eZpHu8MNQ0bOjWAOEjIr2vHoHJIvRQosxAOw2EqMH/XmyWihGNFxyTClFBjg0336ODgJQcQSnVq0LlCFehh8eFP34boG7pyWph72At6gNd0wiJtTe+S6FCrvw471TZMXGl0KFeTyivpQy2JPDuvaII04YnT/FLnHVkYz9vFI3VBhiht/RhynpvhYc15zHpbMp0D9nkzZKqJz+rnxdm/bmQhmR3b9CiwnLhVBaWmqx+pMXdtN+rwh62jCZL0PCqWjF8ChpTh5mWR6Dv77f3nT8N9Z35qzsw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ibej2hdMvib6q8x4h3ru+hAP1vQUr+0gj5LhSbls42E=;
 b=W7AJGwzqrgRZNT17JmXWlx59PI/xEd1a5BHIYD3Ad1AKx0V/0eKJeASCMPCYZfuonu9kVGrbXvjRZX8hhXbdBlhlhwBKU3m2dWYMZW2GHkxIUvVrmRM2acAYPWg2VhkwAbGqkhwYdmza6ZGWxl4+/cLDRqMRdSUhIc9hYUyXtniHeBhMchcjwnHnbUwRK60Shw/bNb+ocSqY7D0Nvkz6XjoZOxZjZTqWUkHxbVru2qZKRXzNcsS07X9hxeDjHC5yurb/8NosWd91oXcWrVOB64atbi4l9lXwreyUOteEeZ9XCgWNSja12ZJdFOi/EN6I3dVIHzGYeErfIfTvBt8C/Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ibej2hdMvib6q8x4h3ru+hAP1vQUr+0gj5LhSbls42E=;
 b=FT3C3ykmAW7vPaoEfBIJ0CkrBY25zzz4/W060pUvLi0IJ3gmLSv0U7yCFx32eVfipDcrf+3+qN6GobFeQa/EOzDq8X0GlLjtvXZZe15CKUHxxpEfx/djCG7slRuOGAXPxBMQ8sjR9uX0eePUYcUIF9ss0z/1h80eCRdj2i3F6ok=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Volodymyr
 Babchuk" <volodymyr_babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>
Subject: [PATCH v13 02/14] vpci: restrict unhandled read/write operations for guests
Date: Fri, 2 Feb 2024 16:33:06 -0500
Message-ID: <20240202213321.1920347-3-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9DA:EE_|IA0PR12MB8304:EE_
X-MS-Office365-Filtering-Correlation-Id: a9963615-368f-4bb0-0a57-08dc2436b5f9
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	JR3gm5O71QtrN9EeQJHiO2C4BzhNccb4XfG+9JqNRjBuUxXH92Tzf0r46OC/hIM8X2TgNzv/7HTWNJyysF1K5fZLTt15EB1UOEtqnLciihyZebkELEb3cHCvK8F9RNxa00fd9Iv0Zs1aBhv7MiJUEWFpNrx7x4Q7EUmllF0gJTPkYtoB2bWVyWBSY0B9KUyzu9c6m9J74MwSXcCL/Lsb7Z48qUxCcvjxLJvbdMiMZ/8US38PBVd2O5Y3+O5zmOAS9zLOHGTZFe/aGgYLe3py7SOnhIjA6g1QMOrj3kxSA93gJQSz0M/vSBVjkGJFZReaQ1XrlOonvIzxwj34H/+irm7rWoetnzlaUDRgrv2qckuOYu9WiS0z9jN4eWSGR0g8lB1KxIS3jVUgnJxhiUOUo+qiMobPr/gJ8JrhmBiTNSnpnaRKdbZNC7sWiZdhPnudzed1A5rkgj/GtOy0MGMZPXzwa38/YpP1VPC1hwwtuO7KshK/sB1fADqJxFTfl/X2Fbmn0AbcDobXYqNkTScGR+deBLTiMZPvXkRlvIptuPIHTcraKS/WZNVEgXFlXX3E2HgMJXyjxVjkORzzx1fUGSQmyr7SFSvV80nXe2n8niGI4kwoRvT1KEnZzUnOVD3LeorsT/thDOGRP7buvZVb630t42otaUxzp+iZ1Gz8foIQzWbu6rpBiOUx0oJNVEzMUtc5WfGnZi9hB1pd4xp5OLbk1WkF52uLnw2zSGEEOjN6LFzsA9QSlF0KkJS46Gu7ysroVT72woTQVGpdow/fSA==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(396003)(376002)(39860400002)(346002)(230922051799003)(186009)(82310400011)(64100799003)(1800799012)(451199024)(46966006)(40470700004)(36840700001)(40480700001)(40460700003)(83380400001)(41300700001)(86362001)(36756003)(26005)(36860700001)(426003)(356005)(81166007)(54906003)(6916009)(336012)(2616005)(1076003)(70586007)(47076005)(44832011)(2906002)(5660300002)(8676002)(4326008)(6666004)(316002)(82740400003)(70206006)(478600001)(8936002)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:34:17.8661
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a9963615-368f-4bb0-0a57-08dc2436b5f9
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9DA.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8304

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

A guest would be able to read and write those registers which are not
emulated and have no respective vPCI handlers, so it will be possible
for it to access the hardware directly.
In order to prevent a guest from reads and writes from/to the unhandled
registers make sure only hardware domain can access the hardware directly
and restrict guests from doing so.

Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
Since v9:
- removed stray formatting change
- added Roger's R-b tag
Since v6:
- do not use is_hwdom parameter for vpci_{read|write}_hw and use
  current->domain internally
- update commit message
New in v6
---
 xen/drivers/vpci/vpci.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 475272b173f3..d545dc633c40 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -268,6 +268,10 @@ static uint32_t vpci_read_hw(pci_sbdf_t sbdf, unsigned int reg,
 {
     uint32_t data;
 
+    /* Guest domains are not allowed to read real hardware. */
+    if ( !is_hardware_domain(current->domain) )
+        return ~(uint32_t)0;
+
     switch ( size )
     {
     case 4:
@@ -311,6 +315,10 @@ static uint32_t vpci_read_hw(pci_sbdf_t sbdf, unsigned int reg,
 static void vpci_write_hw(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
                           uint32_t data)
 {
+    /* Guest domains are not allowed to write real hardware. */
+    if ( !is_hardware_domain(current->domain) )
+        return;
+
     switch ( size )
     {
     case 4:
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:34:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:34:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675213.1050499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1BD-000562-LB; Fri, 02 Feb 2024 21:34:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675213.1050499; Fri, 02 Feb 2024 21:34:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1BD-00055v-Ib; Fri, 02 Feb 2024 21:34:35 +0000
Received: by outflank-mailman (input) for mailman id 675213;
 Fri, 02 Feb 2024 21:34:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1BC-0003tJ-77
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:34:34 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2409::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dae8aeb1-c212-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:34:33 +0100 (CET)
Received: from BYAPR03CA0001.namprd03.prod.outlook.com (2603:10b6:a02:a8::14)
 by MW5PR12MB5600.namprd12.prod.outlook.com (2603:10b6:303:195::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.9; Fri, 2 Feb
 2024 21:34:30 +0000
Received: from SJ1PEPF00001CDC.namprd05.prod.outlook.com
 (2603:10b6:a02:a8:cafe::2f) by BYAPR03CA0001.outlook.office365.com
 (2603:10b6:a02:a8::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.23 via Frontend
 Transport; Fri, 2 Feb 2024 21:34:29 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ1PEPF00001CDC.mail.protection.outlook.com (10.167.242.4) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7202.16 via Frontend Transport; Fri, 2 Feb 2024 21:34:29 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:34:28 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Fri, 2 Feb
 2024 13:34:28 -0800
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:34:27 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dae8aeb1-c212-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=k1IV2fkW4MlwOgrgGzh8fEIhwjZP2sJrnLPO7NpcwlKavBVUBIp9/ewqlF+Rxztd2ShM9SjpnKPlNAnvCC/zrruphJzNM9PYS9fBzVqEYSwYlk46nbTXuYBDcOoWnM6qyExXx346dbZKGhoWS52O6qNuBfc2UqpIH+vwjHIv71gv3cLNbXE+Di771e8QzRHE9im7Ab3bfbMtHfd8bhyVBBUS6S2VYhZT9xdActlDZFKZQQyHkuKhSsWEjrZJth+I8VoHSEGfdNNcU+zD4+Qggn6sr8QcY6qAkGfqqu4IUJPF42wUC+s0SGuPhixmfaSnyJr1aLTGN+NyChVa+KsqtQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=NWmHF1bTVpBmU7drHJ9mWbFr611+spd24HiT6koGFoM=;
 b=C6tx3ucDy0C7nxCqI/LNmR/HZSgUuXc+wfrSxJ8kT/pXiTCkKhopcXI5XockOn/bNzyQDDq/+bYBo47QvXhm75SiAiKzfk6DqaszaeNwakH1DAQpU900g8LI7GZEGEtPjx4QpeKNUAZg4KVrrePRGSVZ5dGAmI7AEjfRaazbDO8NFFkaKPVpzSLh/EcHDs2cIbNr5T68TQ9pChYkxaC5croqYGfD+xEiUfNM5sQkiRkthhMr5+XRc/kZc4OEmDLRe8weLoOu0YyAP6pIpOOvnuAa0NeR2hka9ar4Y5Pv8CZ01BND99FN/fUTkz1jQaffecVkYV6r7ikQgqYT0T60MA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=NWmHF1bTVpBmU7drHJ9mWbFr611+spd24HiT6koGFoM=;
 b=Yg1j149sHsTJl26CvEuTs9SfJCSNb49Klj79OmsEJsqEl1GppmhTD+8zC+qcwPMYlU0g1ds0WSfIk3Mvjn/1fAUZeX5549xWQfQ7WcDBU9ItifTmvMTgsjWsww7+rkWJ2QnmPg1e4nWsLfUBq/GQU/sdAgs0IH4qT6fIMlgCfEM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Jan Beulich
	<jbeulich@suse.com>, Paul Durrant <paul@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Volodymyr
 Babchuk" <volodymyr_babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>
Subject: [PATCH v13 03/14] vpci: add hooks for PCI device assign/de-assign
Date: Fri, 2 Feb 2024 16:33:07 -0500
Message-ID: <20240202213321.1920347-4-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CDC:EE_|MW5PR12MB5600:EE_
X-MS-Office365-Filtering-Correlation-Id: bc76e669-0c30-490a-0edc-08dc2436bcd8
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	26CNyWiHFkKUwHCJUuvydOTrQInd9iUThSL+uq7k910Yh1Pf2BN7NaaAoFu1zbu8O/UrsAoPbBkh05Pt4uK3X7xNlCzURkOKL+JuiGWeY9RsJti0xxQeS4bGUKyNA87jKCsI1prAKTwlNTrh1kCVxP9eGWO6ZdAkh7su6Qhdxyr/O0Cr4N89a2nXp+GNIdhvD2sXXOZSkVU/6bnwC2nOTsW4sqJmV++P2GVlFMDcYPYm86YmVw6xFK+odCcGrwEBhqArnNnb41mnXFVoVlUWvTrP1zAze/bhVNy16H443O5Kz9a45jqtHI0BzyGl/jjNh7hojsvsXg3HFyvBy7/n+Ss24/RZ3/Ymlmz67iHAHbgLRQf8UErnPKJbMpO2rxH006zfF5csn6HsjlSlYZ1NqKwiiCvhNHCsQPPfJppUorBni/d9NQglrVoFFLSxuNd78CNz5fcpS4AY+Gfceb841A3rUYgDR1U1ZN85exqw6srSCmL5j7ArFUMOJhHyx+5LdUlHlffHTeDggxnLP77YBeQBSVD4cAGwBIIbK8z1rW9v/M5JZDFe3kj7HE+Jfqez5Hxk0LFyBntK84sVCwNxeuvZo5RSMJzWeDd77nVhk3zIO5SJr0uDSaUTT/TK50/hQa1jFb4ZWCCCquQmwa/JzLLxUO5Gt63w9GsMc8bwDc3xm0qj9krtIJ6AGW6cvNzR28uVVeMfHhXol/nfkL8K8LpYiQ+9QQ/ZqJqy/pevQtlWBNdvKj6DboBd+y99ckIBxzqifcDpZTxUft1GVgP1zg==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(376002)(396003)(346002)(39860400002)(230922051799003)(82310400011)(64100799003)(186009)(451199024)(1800799012)(36840700001)(40470700004)(46966006)(40460700003)(40480700001)(36756003)(86362001)(478600001)(356005)(426003)(8936002)(82740400003)(1076003)(336012)(2616005)(41300700001)(81166007)(26005)(83380400001)(47076005)(44832011)(54906003)(36860700001)(6916009)(5660300002)(2906002)(6666004)(316002)(4326008)(70206006)(70586007)(8676002)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:34:29.3627
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: bc76e669-0c30-490a-0edc-08dc2436bcd8
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CDC.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR12MB5600

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

When a PCI device gets assigned/de-assigned we need to
initialize/de-initialize vPCI state for the device.

Also, rename vpci_add_handlers() to vpci_assign_device() and
vpci_remove_device() to vpci_deassign_device() to better reflect role
of the functions.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
In v13:
 - Add Jan's A-b
 - Rebase on
   cb4ecb3cc17b ("pci: fail device assignment if phantom functions cannot be assigned")
   and add if ( rc ) goto done; in assign_device()
In v12:
 - Add Roger's R-b
 - Clean up comment in xen/include/xen/vpci.h
 - Add comment in xen/drivers/passthrough/pci.c:deassign_device() to
   clarify vpci_assign_device() call
In v11:
- Call vpci_assign_device() in "deassign_device" if IOMMU call
"reassign_device" was successful.
In v10:
- removed HAS_VPCI_GUEST_SUPPORT checks
- HAS_VPCI_GUEST_SUPPORT config option (in Kconfig) as it is not used
  anywhere
In v9:
- removed previous  vpci_[de]assign_device function and renamed
  existing handlers
- dropped attempts to handle errors in assign_device() function
- do not call vpci_assign_device for dom_io
- use d instead of pdev->domain
- use IS_ENABLED macro
In v8:
- removed vpci_deassign_device
In v6:
- do not pass struct domain to vpci_{assign|deassign}_device as
  pdev->domain can be used
- do not leave the device assigned (pdev->domain == new domain) in case
  vpci_assign_device fails: try to de-assign and if this also fails, then
  crash the domain
In v5:
- do not split code into run_vpci_init
- do not check for is_system_domain in vpci_{de}assign_device
- do not use vpci_remove_device_handlers_locked and re-allocate
  pdev->vpci completely
- make vpci_deassign_device void
In v4:
 - de-assign vPCI from the previous domain on device assignment
 - do not remove handlers in vpci_assign_device as those must not
   exist at that point
In v3:
 - remove toolstack roll-back description from the commit message
   as error are to be handled with proper cleanup in Xen itself
 - remove __must_check
 - remove redundant rc check while assigning devices
 - fix redundant CONFIG_HAS_VPCI check for CONFIG_HAS_VPCI_GUEST_SUPPORT
 - use REGISTER_VPCI_INIT machinery to run required steps on device
   init/assign: add run_vpci_init helper
In v2:
- define CONFIG_HAS_VPCI_GUEST_SUPPORT so dead code is not compiled
  for x86
In v1:
 - constify struct pci_dev where possible
 - do not open code is_system_domain()
 - extended the commit message
---
 xen/drivers/passthrough/pci.c | 28 ++++++++++++++++++++++++----
 xen/drivers/vpci/header.c     |  2 +-
 xen/drivers/vpci/vpci.c       |  6 +++---
 xen/include/xen/vpci.h        | 10 +++++-----
 4 files changed, 33 insertions(+), 13 deletions(-)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index c97dd4504a7a..4c0a836486ec 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -755,7 +755,7 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
          * For devices not discovered by Xen during boot, add vPCI handlers
          * when Dom0 first informs Xen about such devices.
          */
-        ret = vpci_add_handlers(pdev);
+        ret = vpci_assign_device(pdev);
         if ( ret )
         {
             list_del(&pdev->domain_list);
@@ -769,7 +769,7 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
         if ( ret )
         {
             write_lock(&hardware_domain->pci_lock);
-            vpci_remove_device(pdev);
+            vpci_deassign_device(pdev);
             list_del(&pdev->domain_list);
             write_unlock(&hardware_domain->pci_lock);
             pdev->domain = NULL;
@@ -817,7 +817,7 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
     list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
         if ( pdev->bus == bus && pdev->devfn == devfn )
         {
-            vpci_remove_device(pdev);
+            vpci_deassign_device(pdev);
             pci_cleanup_msi(pdev);
             ret = iommu_remove_device(pdev);
             if ( pdev->domain )
@@ -875,6 +875,10 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
             goto out;
     }
 
+    write_lock(&d->pci_lock);
+    vpci_deassign_device(pdev);
+    write_unlock(&d->pci_lock);
+
     devfn = pdev->devfn;
     ret = iommu_call(hd->platform_ops, reassign_device, d, target, devfn,
                      pci_to_dev(pdev));
@@ -886,6 +890,11 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
 
     pdev->fault.count = 0;
 
+    write_lock(&target->pci_lock);
+    /* Re-assign back to hardware_domain */
+    ret = vpci_assign_device(pdev);
+    write_unlock(&target->pci_lock);
+
  out:
     if ( ret )
         printk(XENLOG_G_ERR "%pd: deassign (%pp) failed (%d)\n",
@@ -1146,7 +1155,7 @@ static void __hwdom_init setup_one_hwdom_device(const struct setup_hwdom *ctxt,
               PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn) );
 
     write_lock(&ctxt->d->pci_lock);
-    err = vpci_add_handlers(pdev);
+    err = vpci_assign_device(pdev);
     write_unlock(&ctxt->d->pci_lock);
     if ( err )
         printk(XENLOG_ERR "setup of vPCI for d%d failed: %d\n",
@@ -1476,6 +1485,10 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
     if ( pdev->broken && d != hardware_domain && d != dom_io )
         goto done;
 
+    write_lock(&pdev->domain->pci_lock);
+    vpci_deassign_device(pdev);
+    write_unlock(&pdev->domain->pci_lock);
+
     rc = pdev_msix_assign(d, pdev);
     if ( rc )
         goto done;
@@ -1501,6 +1514,13 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag)
                         pci_to_dev(pdev), flag);
     }
 
+    if ( rc )
+        goto done;
+
+    write_lock(&d->pci_lock);
+    rc = vpci_assign_device(pdev);
+    write_unlock(&d->pci_lock);
+
  done:
     if ( rc )
     {
diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 8f5850b8cf6d..2f2d98ada012 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -191,7 +191,7 @@ bool vpci_process_pending(struct vcpu *v)
              * killed in order to avoid leaking stale p2m mappings on
              * failure.
              */
-            vpci_remove_device(v->vpci.pdev);
+            vpci_deassign_device(v->vpci.pdev);
         write_unlock(&v->domain->pci_lock);
     }
 
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index d545dc633c40..ff4837391786 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -40,7 +40,7 @@ extern vpci_register_init_t *const __start_vpci_array[];
 extern vpci_register_init_t *const __end_vpci_array[];
 #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
 
-void vpci_remove_device(struct pci_dev *pdev)
+void vpci_deassign_device(struct pci_dev *pdev)
 {
     ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
 
@@ -73,7 +73,7 @@ void vpci_remove_device(struct pci_dev *pdev)
     pdev->vpci = NULL;
 }
 
-int vpci_add_handlers(struct pci_dev *pdev)
+int vpci_assign_device(struct pci_dev *pdev)
 {
     unsigned int i;
     const unsigned long *ro_map;
@@ -107,7 +107,7 @@ int vpci_add_handlers(struct pci_dev *pdev)
     }
 
     if ( rc )
-        vpci_remove_device(pdev);
+        vpci_deassign_device(pdev);
 
     return rc;
 }
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index d20c301a3db3..99fe76f08ace 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -25,11 +25,11 @@ typedef int vpci_register_init_t(struct pci_dev *dev);
   static vpci_register_init_t *const x##_entry  \
                __used_section(".data.vpci." p) = x
 
-/* Add vPCI handlers to device. */
-int __must_check vpci_add_handlers(struct pci_dev *pdev);
+/* Assign vPCI to device by adding handlers. */
+int __must_check vpci_assign_device(struct pci_dev *pdev);
 
 /* Remove all handlers and free vpci related structures. */
-void vpci_remove_device(struct pci_dev *pdev);
+void vpci_deassign_device(struct pci_dev *pdev);
 
 /* Add/remove a register handler. */
 int __must_check vpci_add_register_mask(struct vpci *vpci,
@@ -255,12 +255,12 @@ bool vpci_ecam_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int len,
 #else /* !CONFIG_HAS_VPCI */
 struct vpci_vcpu {};
 
-static inline int vpci_add_handlers(struct pci_dev *pdev)
+static inline int vpci_assign_device(struct pci_dev *pdev)
 {
     return 0;
 }
 
-static inline void vpci_remove_device(struct pci_dev *pdev) { }
+static inline void vpci_deassign_device(struct pci_dev *pdev) { }
 
 static inline void vpci_dump_msi(void) { }
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:34:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:34:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675216.1050510 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1BX-0005pJ-Vp; Fri, 02 Feb 2024 21:34:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675216.1050510; Fri, 02 Feb 2024 21:34:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1BX-0005pC-Rf; Fri, 02 Feb 2024 21:34:55 +0000
Received: by outflank-mailman (input) for mailman id 675216;
 Fri, 02 Feb 2024 21:34:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1BW-0004Ae-4G
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:34:54 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e56c383d-c212-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 22:34:52 +0100 (CET)
Received: from BY3PR05CA0011.namprd05.prod.outlook.com (2603:10b6:a03:254::16)
 by IA1PR12MB8538.namprd12.prod.outlook.com (2603:10b6:208:455::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.31; Fri, 2 Feb
 2024 21:34:46 +0000
Received: from SJ1PEPF00001CE0.namprd05.prod.outlook.com
 (2603:10b6:a03:254:cafe::d9) by BY3PR05CA0011.outlook.office365.com
 (2603:10b6:a03:254::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.22 via Frontend
 Transport; Fri, 2 Feb 2024 21:34:46 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ1PEPF00001CE0.mail.protection.outlook.com (10.167.242.8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7202.16 via Frontend Transport; Fri, 2 Feb 2024 21:34:45 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:34:44 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:34:38 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e56c383d-c212-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IYxAPDVpiu3L0g27UXvfFoylhEbrW5YA12f7RZDgiUpmXE++LL+vqScpw9813n1J9x6u0Zpc416QwxFxlDgAki9LfurMdQojXtC/2VOFkOY7mbAFqMkIGd5Zv8GHDT2gZfAlDwl610GnH2Ue7WLwOSdB+yIlmXCXbvSfIKUxFCRWuWq2MhudgnH2FLA/7gNJhmyuwTSdLKqZ7rShKCszihaHEcbvxxpWt+cEYliV3k5/KJo0mbcK02zj6GjUDPZH45lNGU98EFW4SjOeiu0uHLEGkVjYvwMX06QFZdyDNLXw2IuZ76x4TQzdXlfYw7c1lo/G+6BgLpfwgk2dhuq8JQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=xyiyI3xsKrX/cnHhNKfE69AhVBubvl5XKQMwzn7dWtQ=;
 b=OAB+PaOIGfo6v7aGYnYFcZIxCz93zKegteOCTUKtvEQdO9SkU7qIKXg5SJvMvGBXGdoEwc7KBWFX+djcSu1XeXQa4QocQMIjtCOwLWMGFyFhgS0WYGtr5RtDcuHmi/Knw3pntAQnCSQ7SvZoM2NXKRAVKJs7AnuBaEiDXBRz/I3KlBdqian6ms2TE6yBpv/qZBrlc6MEpYtoLDgMfP7lC9xmVu51qeB8odIV70z7auwYxWaP5DUdqAOPYS8u9XqtmDUH32ZvBnytXN5ogcZf38aXYGWbX+UmIj+ZcLewRr9FJ7j3Cm73djceg3EiqzEH4CLdXMGqrx5DtCcdhWzBQQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xyiyI3xsKrX/cnHhNKfE69AhVBubvl5XKQMwzn7dWtQ=;
 b=uDtqpQb/VaXETn246NN0JRJyTTvuJ5fAfdsCYKKPhSNniYLqH5GzUhFC4FwGGeTvDKv93FR7X3j4kdHbBxmO1tPX2Kj/qZY2Upg8D9PmKuZ81kIBDBHWBgWzLWnzjxkdVK5HjWg9dMTICsh6XjYTYV/ioOeluxu957pKIiATDrQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Volodymyr
 Babchuk" <volodymyr_babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>
Subject: [PATCH v13 04/14] vpci/header: rework exit path in init_header()
Date: Fri, 2 Feb 2024 16:33:08 -0500
Message-ID: <20240202213321.1920347-5-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CE0:EE_|IA1PR12MB8538:EE_
X-MS-Office365-Filtering-Correlation-Id: a80af3f4-acae-4c13-479e-08dc2436c68b
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	WSVPJi+gJuL0p7fuaFgh2pUAQxR1q+zSDOX14sKPtYT9STdwb1haXULoO0MxVrNvTmSl7XJX+VYKjhgHXmtyEjKJIWkNFO38IqR9unjkLgmceh0+o7nPyHvnZt4MMHkseOjj2eoABDYJurjF7HtFCBfFhOAdnL116KhIoYnwxjNzbcyzUnD85LJBZIVlryvyzLAl1K8JXF3ri7S1vCZSpWZZ01cgNLmdcFwUP41+YoCtz/donRhqlMV8rpmRBjImUkDPrr7M7T75D8fWeVNMdbMNb5AfvcYF9i+z9FxSundpSzxX3KsbXr0Kl8GjKVDwKCFbFJCLkRppBh13a4uGi2m79fg/6LXFwwsypAyslXCDGVcIO96a0uYMPTJ02+34WSLxHUQNp9vYTTLkGoSGz2jLAXTrM0PrtwYVsrzxswSvpoQyh0Jp3sslALJnDs2yEmaWJ0CBXLfDIcWjn14WYIoqHEKSHHrK/8IlZb7Lf3A9TpVgmHH5qYp2dne5KO+oNvoJgLrHCVU9g6avsr5/B/dsRHNHBxWazJ8ENEJ7C0ujJRe1YP6TzRpVsuY5XvkYB/7quK3LEmZulbkIMLVPY3UnhIbDHV8n3YLs5KDZoetsHXXFZ4lp5nXzbqUxHF6T47EuUvbmnmt/RQ7IcgrWNu/t2uzP2RZHllZxyrmgEucHnUBEe0pLA7ZfA8OOegxYwmWWKpwcgBFFrXwuLcPhT+vy6EZ65x+9iw9C/iux+fVe0jicGO8SAEl9RGSL0hyVVUCBF/d+rzNj1+9GMNbcgg==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(396003)(376002)(136003)(39860400002)(346002)(230922051799003)(186009)(1800799012)(64100799003)(82310400011)(451199024)(40470700004)(36840700001)(46966006)(8676002)(426003)(336012)(2616005)(4326008)(1076003)(26005)(5660300002)(478600001)(41300700001)(6916009)(8936002)(36756003)(316002)(44832011)(86362001)(54906003)(70206006)(70586007)(6666004)(2906002)(82740400003)(81166007)(47076005)(36860700001)(356005)(83380400001)(40480700001)(40460700003)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:34:45.6368
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a80af3f4-acae-4c13-479e-08dc2436c68b
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CE0.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8538

From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>

Introduce "fail" label in init_header() function to have the centralized
error return path. This is the pre-requirement for the future changes
in this function.

This patch does not introduce functional changes.

Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
In v12:
- s/init_bars/init_header/
- Re-order tags
- Fixup scissors line
In v11:
- Do not remove empty line between "goto fail;" and "continue;"
In v10:
- Added Roger's A-b tag.
In v9:
- New in v9
---
 xen/drivers/vpci/header.c | 19 +++++++------------
 1 file changed, 7 insertions(+), 12 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 2f2d98ada012..803fe4bb99a6 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -656,10 +656,7 @@ static int cf_check init_header(struct pci_dev *pdev)
             rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
                                    4, &bars[i]);
             if ( rc )
-            {
-                pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
-                return rc;
-            }
+                goto fail;
 
             continue;
         }
@@ -679,10 +676,7 @@ static int cf_check init_header(struct pci_dev *pdev)
         rc = pci_size_mem_bar(pdev->sbdf, reg, &addr, &size,
                               (i == num_bars - 1) ? PCI_BAR_LAST : 0);
         if ( rc < 0 )
-        {
-            pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
-            return rc;
-        }
+            goto fail;
 
         if ( size == 0 )
         {
@@ -697,10 +691,7 @@ static int cf_check init_header(struct pci_dev *pdev)
         rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, 4,
                                &bars[i]);
         if ( rc )
-        {
-            pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
-            return rc;
-        }
+            goto fail;
     }
 
     /* Check expansion ROM. */
@@ -722,6 +713,10 @@ static int cf_check init_header(struct pci_dev *pdev)
     }
 
     return (cmd & PCI_COMMAND_MEMORY) ? modify_bars(pdev, cmd, false) : 0;
+
+ fail:
+    pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd);
+    return rc;
 }
 REGISTER_VPCI_INIT(init_header, VPCI_PRIORITY_MIDDLE);
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:35:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:35:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675218.1050520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Be-0006Ac-BP; Fri, 02 Feb 2024 21:35:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675218.1050520; Fri, 02 Feb 2024 21:35:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Be-0006AV-8W; Fri, 02 Feb 2024 21:35:02 +0000
Received: by outflank-mailman (input) for mailman id 675218;
 Fri, 02 Feb 2024 21:35:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1Bd-0003tJ-Hr
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:35:01 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ea8d533f-c212-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:35:00 +0100 (CET)
Received: from DS7PR06CA0011.namprd06.prod.outlook.com (2603:10b6:8:2a::6) by
 DM6PR12MB4265.namprd12.prod.outlook.com (2603:10b6:5:211::14) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7270.12; Fri, 2 Feb 2024 21:34:57 +0000
Received: from CY4PEPF0000E9D9.namprd05.prod.outlook.com
 (2603:10b6:8:2a:cafe::d5) by DS7PR06CA0011.outlook.office365.com
 (2603:10b6:8:2a::6) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.31 via Frontend
 Transport; Fri, 2 Feb 2024 21:34:57 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D9.mail.protection.outlook.com (10.167.241.77) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 21:34:56 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:34:56 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:34:55 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea8d533f-c212-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PjJQLFq2KtvqVaN/Tm2Ex8k70gamTp6bwQQWhrzQPMbFuyzFxrU6rex1iFGnPWJqiByAZaRCjiFDnjJjtKAIWevOcMImk9bMcGiQPpX0P/WfltgxWlnK80L21dNRR2MsN2DWRQm4B6im0t+q5P8s3aVx1q7YAvUaIsNxaY6g9748K+6gxTFYDrknGGHzdWyoW1W1iAlmDjqQHra9I3Y2vg5YDXQreEXvS3C1CpS2bj9EdpFLOK9nOm8WtDLbYS50wE19rzpQRI+zSVHkOw4SWhHEgX+g/eX6MFscyEIP1miVJ4Acur6nzYL7+YoZOA4jT1gxLf24zO4n4QwjqOqTXA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=3fv9DTE0P0KLmwQzPHqaJ/yxioeZbVVsJLRZmvugJrA=;
 b=cwd9UEcWrsXAEAap3OK6GoReAC7oF3fzD0SzB6/yhnur054AYY37dBLyJsJOnSgBz+YbV7yLzFiXfU9ELuRc8sOgoqv/RtFiOPFSIdZMhxgVJJd0QDgexQdglaEqwTVqzy4jn++nqFhPlGuOq/zO/7lSbrxf9W5wlL4XbL6jZdiwmXw2ZeSoTLRq70BKvopcD8hrFzEDooAYJNzi/9nbA3DZXN5ivYcLMxLHWTuoc6G3bOM6NLITf57exlbRAAUT6bEHuWpXwndJ7B5aqM5iZDeMYcQ5nvl7XOXCkzcjVaZ5YKIONZbt8Xnw5cclks21ytdxHOWQEZ8JYxEMSa5Tbw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3fv9DTE0P0KLmwQzPHqaJ/yxioeZbVVsJLRZmvugJrA=;
 b=kKFkQXJVa2as7Kwh/DiORASGaAjUdCbXawJE2QeonZBgPCuqid3OynZ+WP/F4rF1u/vx3UYY0oIg21wKuDE0wcbcj7Llry8SYAh2CKpS1djeiChTiRVdQ6mDw7k8TEvI4NEgtj+KLe0mjVizk7gbUTJJN5CU1J7FbuVKx+BUz9w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Volodymyr
 Babchuk" <volodymyr_babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>
Subject: [PATCH v13 05/14] vpci/header: implement guest BAR register handlers
Date: Fri, 2 Feb 2024 16:33:09 -0500
Message-ID: <20240202213321.1920347-6-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D9:EE_|DM6PR12MB4265:EE_
X-MS-Office365-Filtering-Correlation-Id: 6336fe6a-39fa-4bb8-fc01-08dc2436cd2c
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	W33MB+IUMfFJVkt+OdUiGD1Ek3orT1eAO7jFBQVCSL1r/dw6oorWkGUL8Q7F/4VkxbnyOHWyD5f+FvatBV5beImKOMkHYO39EJbM58xCR/2lPBeZd9/atLQn7lR8Vf4ISr9DV/rMtUN0al/WtpbmZIrHlDRG60EPQLyaxkcXHVlTnq4ERa6s7b3S+loJ7orh8e/hJYNmXIXajwtw3vQ/k1BScGSwQICHuoXFdrSZOnF6bnBr+SZ4uTs+NvTV9G6DHfby6g+1xRLy2px+xiTMF3DhsO9nn/3ae5G0/yrBy+smd41+TeRqGw9zE3Ro86GV4ogYhtuxaY1+/sX3tx+yfBQIy8UVfO3ToFc0YxkUHkHNnl6M4vDfW093L4LWPB1i0GpVLh0q2sEkM3e6lhw4q8jrhcHVnVq2k7de56RV7saXNlDb1xyLkLwOXuOFuZFjC292HApbBudLf68plvlBpwvMbCgk/48Sg6zbxGaJetkcY2hCG5IEYhPVIOaGuhs+kqmESvwQyWDdJs+0cCySnfaAoCASoCKs5w77Bu7INSYjw3xMBUT2uOdOKRa33k/Eser1xjmlnK5+CncZzjNAgaeuFTFI4W5Rkpi8qC7NJ7widCsFyEhczMMAHDrSBmp5TP/LOWqTPIsevEDjBOAmP+u0yeh5nUn4WHJ38Q9yvT6wB15LTTpBJQhcZBupZYrPOT1ervRiIKZiF5uIjAhIJ+WwetAuLc7u14RHdi18ZJRTTPwMT9cqOJvK/7Ta5zplUIEXVRsjsHkhHxwMBgUR/A==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(136003)(346002)(376002)(396003)(230273577357003)(230922051799003)(186009)(64100799003)(82310400011)(451199024)(1800799012)(40470700004)(46966006)(36840700001)(40460700003)(40480700001)(41300700001)(86362001)(1076003)(36756003)(478600001)(356005)(2616005)(81166007)(83380400001)(47076005)(26005)(2906002)(54906003)(4326008)(36860700001)(316002)(6666004)(82740400003)(426003)(336012)(44832011)(8936002)(70586007)(70206006)(8676002)(5660300002)(6916009)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:34:56.7785
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6336fe6a-39fa-4bb8-fc01-08dc2436cd2c
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D9.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4265

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Add relevant vpci register handlers when assigning PCI device to a domain
and remove those when de-assigning. This allows having different
handlers for different domains, e.g. hwdom and other guests.

Emulate guest BAR register values: this allows creating a guest view
of the registers and emulates size and properties probe as it is done
during PCI device enumeration by the guest.

All empty, IO and ROM BARs for guests are emulated by returning 0 on
reads and ignoring writes: this BARs are special with this respect as
their lower bits have special meaning, so returning default ~0 on read
may confuse guest OS.

Introduce is_hwdom convenience variable and convert an existing
is_hardware_domain() check.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
In v12:
- Add Roger's R-b
- Get rid of empty_bar_read, use vpci_read_val instead
- Convert an existing is_hardware_domain() check to is_hwdom
- Re-indent usage of ternary operator
- Use ternary operator to avoid re-indenting expansion ROM block
In v11:
- Access guest_addr after adjusting for MEM64_HI bar in
guest_bar_write()
- guest bar handlers renamed and now  _mem_ part to denote
that they are handling only memory BARs
- refuse to update guest BAR address if BAR is enabled
In v10:
- ull -> ULL to be MISRA-compatbile
- Use PAGE_OFFSET() instead of combining with ~PAGE_MASK
- Set type of empty bars to VPCI_BAR_EMPTY
In v9:
- factored-out "fail" label introduction in init_bars()
- replaced #ifdef CONFIG_X86 with IS_ENABLED()
- do not pass bars[i] to empty_bar_read() handler
- store guest's BAR address instead of guests BAR register view
Since v6:
- unify the writing of the PCI_COMMAND register on the
  error path into a label
- do not introduce bar_ignore_access helper and open code
- s/guest_bar_ignore_read/empty_bar_read
- update error message in guest_bar_write
- only setup empty_bar_read for IO if !x86
Since v5:
- make sure that the guest set address has the same page offset
  as the physical address on the host
- remove guest_rom_{read|write} as those just implement the default
  behaviour of the registers not being handled
- adjusted comment for struct vpci.addr field
- add guest handlers for BARs which are not handled and will otherwise
  return ~0 on read and ignore writes. The BARs are special with this
  respect as their lower bits have special meaning, so returning ~0
  doesn't seem to be right
Since v4:
- updated commit message
- s/guest_addr/guest_reg
Since v3:
- squashed two patches: dynamic add/remove handlers and guest BAR
  handler implementation
- fix guest BAR read of the high part of a 64bit BAR (Roger)
- add error handling to vpci_assign_device
- s/dom%pd/%pd
- blank line before return
Since v2:
- remove unneeded ifdefs for CONFIG_HAS_VPCI_GUEST_SUPPORT as more code
  has been eliminated from being built on x86
Since v1:
 - constify struct pci_dev where possible
 - do not open code is_system_domain()
 - simplify some code3. simplify
 - use gdprintk + error code instead of gprintk
 - gate vpci_bar_{add|remove}_handlers with CONFIG_HAS_VPCI_GUEST_SUPPORT,
   so these do not get compiled for x86
 - removed unneeded is_system_domain check
 - re-work guest read/write to be much simpler and do more work on write
   than read which is expected to be called more frequently
 - removed one too obvious comment
---
 xen/drivers/vpci/header.c | 109 +++++++++++++++++++++++++++++++++++---
 xen/include/xen/vpci.h    |   3 ++
 2 files changed, 106 insertions(+), 6 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 803fe4bb99a6..39e11e141b38 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -478,6 +478,69 @@ static void cf_check bar_write(
     pci_conf_write32(pdev->sbdf, reg, val);
 }
 
+static void cf_check guest_mem_bar_write(const struct pci_dev *pdev,
+                                         unsigned int reg, uint32_t val,
+                                         void *data)
+{
+    struct vpci_bar *bar = data;
+    bool hi = false;
+    uint64_t guest_addr;
+
+    if ( bar->type == VPCI_BAR_MEM64_HI )
+    {
+        ASSERT(reg > PCI_BASE_ADDRESS_0);
+        bar--;
+        hi = true;
+    }
+    else
+    {
+        val &= PCI_BASE_ADDRESS_MEM_MASK;
+    }
+
+    guest_addr = bar->guest_addr;
+    guest_addr &= ~(0xffffffffULL << (hi ? 32 : 0));
+    guest_addr |= (uint64_t)val << (hi ? 32 : 0);
+
+    /* Allow guest to size BAR correctly */
+    guest_addr &= ~(bar->size - 1);
+
+    /*
+     * Xen only cares whether the BAR is mapped into the p2m, so allow BAR
+     * writes as long as the BAR is not mapped into the p2m.
+     */
+    if ( bar->enabled )
+    {
+        /* If the value written is the current one avoid printing a warning. */
+        if ( guest_addr != bar->guest_addr )
+            gprintk(XENLOG_WARNING,
+                    "%pp: ignored guest BAR %zu write while mapped\n",
+                    &pdev->sbdf, bar - pdev->vpci->header.bars + hi);
+        return;
+    }
+    bar->guest_addr = guest_addr;
+}
+
+static uint32_t cf_check guest_mem_bar_read(const struct pci_dev *pdev,
+                                            unsigned int reg, void *data)
+{
+    const struct vpci_bar *bar = data;
+    uint32_t reg_val;
+
+    if ( bar->type == VPCI_BAR_MEM64_HI )
+    {
+        ASSERT(reg > PCI_BASE_ADDRESS_0);
+        bar--;
+        return bar->guest_addr >> 32;
+    }
+
+    reg_val = bar->guest_addr;
+    reg_val |= bar->type == VPCI_BAR_MEM32 ? PCI_BASE_ADDRESS_MEM_TYPE_32 :
+                                             PCI_BASE_ADDRESS_MEM_TYPE_64;
+    reg_val |= bar->prefetchable ? PCI_BASE_ADDRESS_MEM_PREFETCH : 0;
+
+    return reg_val;
+}
+
 static void cf_check rom_write(
     const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
 {
@@ -539,6 +602,7 @@ static int cf_check init_header(struct pci_dev *pdev)
     struct vpci_bar *bars = header->bars;
     int rc;
     bool mask_cap_list = false;
+    bool is_hwdom = is_hardware_domain(pdev->domain);
 
     ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
 
@@ -564,7 +628,7 @@ static int cf_check init_header(struct pci_dev *pdev)
     if ( rc )
         return rc;
 
-    if ( !is_hardware_domain(pdev->domain) )
+    if ( !is_hwdom )
     {
         if ( pci_conf_read16(pdev->sbdf, PCI_STATUS) & PCI_STATUS_CAP_LIST )
         {
@@ -653,8 +717,11 @@ static int cf_check init_header(struct pci_dev *pdev)
         if ( i && bars[i - 1].type == VPCI_BAR_MEM64_LO )
         {
             bars[i].type = VPCI_BAR_MEM64_HI;
-            rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg,
-                                   4, &bars[i]);
+            rc = vpci_add_register(pdev->vpci,
+                                   is_hwdom ? vpci_hw_read32
+                                            : guest_mem_bar_read,
+                                   is_hwdom ? bar_write : guest_mem_bar_write,
+                                   reg, 4, &bars[i]);
             if ( rc )
                 goto fail;
 
@@ -665,6 +732,14 @@ static int cf_check init_header(struct pci_dev *pdev)
         if ( (val & PCI_BASE_ADDRESS_SPACE) == PCI_BASE_ADDRESS_SPACE_IO )
         {
             bars[i].type = VPCI_BAR_IO;
+            if ( !IS_ENABLED(CONFIG_X86) && !is_hwdom )
+            {
+                rc = vpci_add_register(pdev->vpci, vpci_read_val, NULL,
+                                       reg, 4, (void *)0);
+                if ( rc )
+                    goto fail;
+            }
+
             continue;
         }
         if ( (val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
@@ -681,6 +756,15 @@ static int cf_check init_header(struct pci_dev *pdev)
         if ( size == 0 )
         {
             bars[i].type = VPCI_BAR_EMPTY;
+
+            if ( !is_hwdom )
+            {
+                rc = vpci_add_register(pdev->vpci, vpci_read_val, NULL,
+                                       reg, 4, (void *)0);
+                if ( rc )
+                    goto fail;
+            }
+
             continue;
         }
 
@@ -688,14 +772,18 @@ static int cf_check init_header(struct pci_dev *pdev)
         bars[i].size = size;
         bars[i].prefetchable = val & PCI_BASE_ADDRESS_MEM_PREFETCH;
 
-        rc = vpci_add_register(pdev->vpci, vpci_hw_read32, bar_write, reg, 4,
-                               &bars[i]);
+        rc = vpci_add_register(pdev->vpci,
+                               is_hwdom ? vpci_hw_read32 : guest_mem_bar_read,
+                               is_hwdom ? bar_write : guest_mem_bar_write,
+                               reg, 4, &bars[i]);
         if ( rc )
             goto fail;
     }
 
     /* Check expansion ROM. */
-    rc = pci_size_mem_bar(pdev->sbdf, rom_reg, &addr, &size, PCI_BAR_ROM);
+    rc = is_hwdom ? pci_size_mem_bar(pdev->sbdf, rom_reg, &addr, &size,
+                                     PCI_BAR_ROM)
+                  : 0;
     if ( rc > 0 && size )
     {
         struct vpci_bar *rom = &header->bars[num_bars];
@@ -711,6 +799,15 @@ static int cf_check init_header(struct pci_dev *pdev)
         if ( rc )
             rom->type = VPCI_BAR_EMPTY;
     }
+    else if ( !is_hwdom )
+    {
+        /* TODO: Check expansion ROM, we do not handle ROM for guests for now */
+        header->bars[num_bars].type = VPCI_BAR_EMPTY;
+        rc = vpci_add_register(pdev->vpci, vpci_read_val, NULL,
+                               rom_reg, 4, (void *)0);
+        if ( rc )
+            goto fail;
+    }
 
     return (cmd & PCI_COMMAND_MEMORY) ? modify_bars(pdev, cmd, false) : 0;
 
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 99fe76f08ace..b0e38a5a1acb 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -87,7 +87,10 @@ struct vpci {
     struct vpci_header {
         /* Information about the PCI BARs of this device. */
         struct vpci_bar {
+            /* Physical (host) address. */
             uint64_t addr;
+            /* Guest address. */
+            uint64_t guest_addr;
             uint64_t size;
             enum {
                 VPCI_BAR_EMPTY,
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:35:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:35:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675222.1050530 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Bq-0006gk-Mm; Fri, 02 Feb 2024 21:35:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675222.1050530; Fri, 02 Feb 2024 21:35:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Bq-0006gb-Jl; Fri, 02 Feb 2024 21:35:14 +0000
Received: by outflank-mailman (input) for mailman id 675222;
 Fri, 02 Feb 2024 21:35:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1Bp-0003tJ-Dn
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:35:13 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f1ae0816-c212-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:35:12 +0100 (CET)
Received: from CY5PR15CA0217.namprd15.prod.outlook.com (2603:10b6:930:88::6)
 by PH0PR12MB7929.namprd12.prod.outlook.com (2603:10b6:510:284::5) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.12; Fri, 2 Feb
 2024 21:35:08 +0000
Received: from CY4PEPF0000E9D8.namprd05.prod.outlook.com
 (2603:10b6:930:88:cafe::19) by CY5PR15CA0217.outlook.office365.com
 (2603:10b6:930:88::6) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.31 via Frontend
 Transport; Fri, 2 Feb 2024 21:35:08 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D8.mail.protection.outlook.com (10.167.241.83) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 21:35:08 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:35:07 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Fri, 2 Feb
 2024 13:35:07 -0800
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:35:06 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1ae0816-c212-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CAXrtD/XwX3PvWAIhRp4bxDCq+MabzGDbD72l1O/6mk0GcPAxDxE43gClQ98r8QDLG4MqzUo2TSnbBRsKQF8+Db4fovSKAlnKPYtstegwEyuOYgeuO9E02V1Jia8IGGUIt7iMTxevbFWFoPjM5UzUo1Jbxh10hxvDp5H2D85DOkZssJG0kQNIymtkZxJ1JMIkIHQLNR/0VtfKVZLS/Xmg7va8BuJOg9uCAjOl6CDelH8GWnvDeHSPDKHAVi/0gRLmy6OJ9rw3LBbYdZQ/lKeTg0YuU62LLmIy1AG3T93LqNCSrNtjdqIb5honhOZO+Xe0mPHJKeu1XKEjw30xZSlZQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=tIlYNCEQ+4nfk+sRGPCKjR3ey13meQvjdP5M2lt1W5E=;
 b=Xi5QYI/t2UdRWUt6hWYnmQPMdGqg1idlzoaBmb8jHeaJ+/bXt5Wk6tksfqhuMQ1O5xK+43AHqEaDuj605Z5/gtUanWyKTMWNZNcltighm9LYUAnKCMp2ILurnMivhX9i2DMB0YNoH38Wt/0l2i1H0LQvo9q0Xr9nzQlp6FhUMpp6lDlxQMzPqtjO/mxKPrRwc7XoX4JCvy8to8q5Mg8k1CACbFn8zv2ZlojjkbMrGLk7V/+XL2n0pRWpQSwwPAFvkB4kUOIhENdjS3ofR92e77ajJJQVlyO8QvylRuOxVdK9u3IhCOLnNcchO3NYhS59EVWuijAArPOujWg2MgC7Bg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tIlYNCEQ+4nfk+sRGPCKjR3ey13meQvjdP5M2lt1W5E=;
 b=rKsCCfAwUSUOprM33XzmL/66B3IyxsDuVmTkXf/ExfcmnvFUU0UstsLaYPEAdWGwFqH/PbwP9nhHA/P3YhfLss8x+HmmppNmNydfsF9uhlUIcyQ2Dpfq8erNmk1834j1XOuzwfCiXm7loWEfZQjhTkqffOiJJIfO7avJCElyU3M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>, Stewart Hildebrand <stewart.hildebrand@amd.com>
Subject: [PATCH v13 06/14] rangeset: add RANGESETF_no_print flag
Date: Fri, 2 Feb 2024 16:33:10 -0500
Message-ID: <20240202213321.1920347-7-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D8:EE_|PH0PR12MB7929:EE_
X-MS-Office365-Filtering-Correlation-Id: df09e7b0-9791-4897-ae4f-08dc2436d424
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	uN8K93cdDoxuD3pHr3+1GeRTsqTTvunKnDa22WBCOP45j+emRvrSLvFxc5SVUi9GgHj2q5Vhh3CpPcwQ1P26Z4pWIUgnxINdld+ALMc0Sm3slYLjQhFCgeIMvuEoU2y+5suvTZJc6kaohBv39G+6+Fwfv8HlRVBZnalaqiQienuspFCfBENetyChFDZMzWIENanUNCfO2Dqmy+8mBymzy6T7t/rk8mpkKghp5jj81vFNZZhi3hBNWUOjBO5O1smNDsUPbm22Zw36Y5ZhluFrTCQmE2mTyaXZnonfYAJMcb+33eSmFfa/ol3OamQXgjDWZbdVEgiDDNpliLN7dN/doES7QUCyXfvVopMoGcbfi71FL4nt89rONSGz8E9fBAQ7l95IiCXN3a4LliN8Fh7ZdXJPOWyVYh7QnxUyOQLQJ7RkCxpHOd84kaW+cE/ESCgMsn8q/g9VDb8qcfOjMjsq5xBuU0aHW+RugIrioBrvSjwFnPQpcxO7XLWDalHOUQpQ3WL+faFbJqME2cG8H1eO2bkzqzDa8RhV0Qi1+w7QOHhknlgwqhocws9f59RMzRUv9kDu8M1KlBFkr4Q5l8A3EDXxvw9rvq5UHY61Oobji3Du1UkRmw9pslPveFdRrVTRJny3AKycdfIpFZTLsH+Oo4JVYb/ZibGrvx9q3x+VrERnwnCrinq4riLEJoPYT4WBZMM/yhk4ELhLTQ0D6yAuny6lx5BbNaPVA82bXQnUmm4zJZsBSF6Iz4kEDpeLJ7TnOWDso789ZZD8XeosY2fJYA==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(136003)(39860400002)(396003)(346002)(230922051799003)(64100799003)(82310400011)(186009)(1800799012)(451199024)(40470700004)(46966006)(36840700001)(478600001)(41300700001)(82740400003)(83380400001)(36860700001)(40460700003)(40480700001)(81166007)(356005)(47076005)(336012)(426003)(6666004)(54906003)(5660300002)(44832011)(2906002)(4326008)(8676002)(26005)(36756003)(8936002)(316002)(6916009)(70586007)(70206006)(86362001)(2616005)(1076003)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:35:08.4837
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: df09e7b0-9791-4897-ae4f-08dc2436d424
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D8.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7929

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

There are range sets which should not be printed, so introduce a flag
which allows marking those as such. Implement relevant logic to skip
such entries while printing.

While at it also simplify the definition of the flags by directly
defining those without helpers.

Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
Since v5:
- comment indentation (Jan)
Since v1:
- update BUG_ON with new flag
- simplify the definition of the flags
---
 xen/common/rangeset.c      | 5 ++++-
 xen/include/xen/rangeset.h | 5 +++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/xen/common/rangeset.c b/xen/common/rangeset.c
index 16a4c3b842e6..0ccd53caac52 100644
--- a/xen/common/rangeset.c
+++ b/xen/common/rangeset.c
@@ -433,7 +433,7 @@ struct rangeset *rangeset_new(
     INIT_LIST_HEAD(&r->range_list);
     r->nr_ranges = -1;
 
-    BUG_ON(flags & ~RANGESETF_prettyprint_hex);
+    BUG_ON(flags & ~(RANGESETF_prettyprint_hex | RANGESETF_no_print));
     r->flags = flags;
 
     safe_strcpy(r->name, name ?: "(no name)");
@@ -575,6 +575,9 @@ void rangeset_domain_printk(
 
     list_for_each_entry ( r, &d->rangesets, rangeset_list )
     {
+        if ( r->flags & RANGESETF_no_print )
+            continue;
+
         printk("    ");
         rangeset_printk(r);
         printk("\n");
diff --git a/xen/include/xen/rangeset.h b/xen/include/xen/rangeset.h
index 8be0722787ed..87bd956962b5 100644
--- a/xen/include/xen/rangeset.h
+++ b/xen/include/xen/rangeset.h
@@ -49,8 +49,9 @@ void rangeset_limit(
 
 /* Flags for passing to rangeset_new(). */
  /* Pretty-print range limits in hexadecimal. */
-#define _RANGESETF_prettyprint_hex 0
-#define RANGESETF_prettyprint_hex  (1U << _RANGESETF_prettyprint_hex)
+#define RANGESETF_prettyprint_hex   (1U << 0)
+ /* Do not print entries marked with this flag. */
+#define RANGESETF_no_print          (1U << 1)
 
 bool __must_check rangeset_is_empty(
     const struct rangeset *r);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:35:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:35:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675227.1050539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1CJ-0007UP-0H; Fri, 02 Feb 2024 21:35:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675227.1050539; Fri, 02 Feb 2024 21:35:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1CI-0007UI-SX; Fri, 02 Feb 2024 21:35:42 +0000
Received: by outflank-mailman (input) for mailman id 675227;
 Fri, 02 Feb 2024 21:35:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1CH-0003tJ-Dj
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:35:41 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 02493886-c213-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:35:40 +0100 (CET)
Received: from CY5PR15CA0220.namprd15.prod.outlook.com (2603:10b6:930:88::22)
 by BL1PR12MB5850.namprd12.prod.outlook.com (2603:10b6:208:395::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.11; Fri, 2 Feb
 2024 21:35:35 +0000
Received: from CY4PEPF0000E9D8.namprd05.prod.outlook.com
 (2603:10b6:930:88:cafe::bc) by CY5PR15CA0220.outlook.office365.com
 (2603:10b6:930:88::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.30 via Frontend
 Transport; Fri, 2 Feb 2024 21:35:35 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D8.mail.protection.outlook.com (10.167.241.83) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 21:35:35 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:35:34 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:35:34 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:35:33 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02493886-c213-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oWw7qFCFgb7Y6CQAyPCxLZ7UyiP5+MEy4Aw84yglAmarrZJG3L/KDmyiPsHsL6WVuos3RNShPZmCUsDALq2f8Ien99sQ3AullPXr/O+GJuqBrAVTbD7Xi53aDFg4ip+Fa+gldfkPiYdE0lP3vvYbarQbF8TZFfgVzFrtzXknYIFW8CqrSOawE4VB/+U0xOiY/V2hvb0Q1LG7IX1Ap5gSFV0cW4peBWxO6ZACPTEJV8ck3SX9xqNM/n9elNhfaMoXtqY00tSH1CUlX+26Yq1V3B7NTShFDHbm6e8dDJX7YfsxWKpFa98VjvMj7kFRqLbtrF0RgiUz3LpTheiV+41jSQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=HDy9jvm+K+yrePwNekCbbD5gVDRQvCSbWFl/2rlmQPI=;
 b=hC7d7WpTLHjV6dPiKMvNeF8LjdJzBaElTejbn1za9K+OlvJ9a6pbDR+4+blBXw31XJqHw9j/pwJl3XNBIiTrv3uULbbW+odFnaGrk+dxFsajXraKCdfcsvyuKC41TD6T6y7M83P1yUWMgk2jOZInQs8nJqKR5mX9xE72WWRqwbmFqtfcE2lIJq+NF4fSTziLyLA2GTgR5BRod8SKvqxeGkPlOvuXZ7y5lb2mFIsp59TnEOPC1tB44DzvL6RHmxSnNsNZIPKmv+H1WTBAh/u7WjTPTNiQMOKWDMZkL7J7neQY3NhhekYqJqeGNvvn8HftNzHPwUBzyjUMYtQSVVTneg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HDy9jvm+K+yrePwNekCbbD5gVDRQvCSbWFl/2rlmQPI=;
 b=fdRW8+qVIWcLlJcdIa94/THaCsB3HshFHmoHwvFAQFqm3lOiwbPT5/3Ecs097ylsW/eqoBQQC1dxO2y0Ev5iaBAAbBQx6nGWQZnEVW2fvApPo1bCj7bpTbCB8X94m2WiS6yTVw7UEw1+Al+2NIlnstbgGmT9UiG+QI9HY6caaPo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Volodymyr
 Babchuk" <volodymyr_babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>
Subject: [PATCH v13 08/14] vpci/header: handle p2m range sets per BAR
Date: Fri, 2 Feb 2024 16:33:12 -0500
Message-ID: <20240202213321.1920347-9-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D8:EE_|BL1PR12MB5850:EE_
X-MS-Office365-Filtering-Correlation-Id: 266ebd28-21d1-45c1-49f7-08dc2436e40f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	g1TZxLMAuXKKeKCZDmkJh4IGyFyQ8okVd3gRN/aif429iN6n4qjOmgNE/d2AtoeT9hqThTA0Vt/qLPZF4/MpiQt55XikR2cvf5egcy8ttGRoYgpWOB7aVg+4GS9lE+EyTdUhvC4N6dxpFjTHaavYgnTSQcw+4qhNvOasLaYc9CF0Ckcb4+jyP6gX12meXjs66M/8UT4fW0fl0Bt9bJxWukcpDlc1NqsCoAukpIRb5c8lW1JnFcR8cYeXO5/KDgbkVhRarkOKaKEB2m7DQbF92d9/uNC3LpCTzfR7h2xrm3Npo/9n0+Xu3CoYZ50mNQrVgD99A7GfOt8dGH+H4tk43rV/3qR3F/sc5KQEU3KeSLNmZKMeyC+gO0NsVt1dRd14Psh06FDDwLCClJeyDPDKTlKXgopa4+StSr2YYB4E+wXgq9iBUUrQG4p9nHkm4yOPEl+DlgfzkxXx7rIEGh/4a5Tb2BbHKkjOJSXMjm4TPzLSd25x7KLujjYwMo+uagJebaJYlVNNxHE7V0ouUbAURkq8uFMnTLdh+elXqQJOQrv35TUz9+y2mlnPnuT1R558UkMtlqGCVV0ZZr9rduEBclPMYz4f+m4+27Z3XiuNc7sW/4wt8yUMs8w1RH8+dKSXzINEYcK0pdVj0DfK9j+Wg+rS7WYy/zxa+2UZ9XqZPXb8uh9C83JRP++oKuLHSQIlAdRwE+/vX7f38+gvcFpSrblxJMRZm2oitUrwKVI8aKOKLUyH5KRI/+HlKMSUX9horh2wEkjUcfTEueMd7Co+3w==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(346002)(136003)(376002)(396003)(230273577357003)(230922051799003)(1800799012)(451199024)(186009)(82310400011)(64100799003)(46966006)(40470700004)(36840700001)(40480700001)(40460700003)(83380400001)(336012)(86362001)(41300700001)(356005)(36756003)(47076005)(426003)(36860700001)(4326008)(5660300002)(26005)(2616005)(81166007)(2906002)(1076003)(70206006)(6666004)(30864003)(316002)(82740400003)(70586007)(6916009)(8936002)(8676002)(44832011)(54906003)(478600001)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:35:35.1559
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 266ebd28-21d1-45c1-49f7-08dc2436e40f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D8.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5850

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Instead of handling a single range set, that contains all the memory
regions of all the BARs and ROM, have them per BAR.
As the range sets are now created when a PCI device is added and destroyed
when it is removed so make them named and accounted.

Note that rangesets were chosen here despite there being only up to
3 separate ranges in each set (typically just 1). But rangeset per BAR
was chosen for the ease of implementation and existing code re-usability.

Also note that error handling of vpci_process_pending() is slightly
modified, and that vPCI handlers are no longer removed if the creation
of the mappings in vpci_process_pending() fails, as that's unlikely to
lead to a functional device in any case.

This is in preparation of making non-identity mappings in p2m for the MMIOs.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
In v12:
- s/rangeset_empty/rangeset_purge/
- change i to num_bars for expansion ROM (purely cosmetic change)
In v11:
- Modified commit message to note changes in error handling in
vpci_process_pending()
- Removed redundant ASSERT() in defer_map. There is no reason to
introduce it in this patch and there is no other patch where
introducing that ASSERT() was appropriate.
- Fixed formatting
- vpci_process_pending() clears v->vpci.pdev if it failed
  checks at the beginning
- Added Roger's R-B tag
In v10:
- Added additional checks to vpci_process_pending()
- vpci_process_pending() now clears rangeset in case of failure
- Fixed locks in vpci_process_pending()
- Fixed coding style issues
- Fixed error handling in init_bars
In v9:
- removed d->vpci.map_pending in favor of checking v->vpci.pdev !=
NULL
- printk -> gprintk
- renamed bar variable to fix shadowing
- fixed bug with iterating on remote device's BARs
- relaxed lock in vpci_process_pending
- removed stale comment
Since v6:
- update according to the new locking scheme
- remove odd fail label in modify_bars
Since v5:
- fix comments
- move rangeset allocation to init_bars and only allocate
  for MAPPABLE BARs
- check for overlap with the already setup BAR ranges
Since v4:
- use named range sets for BARs (Jan)
- changes required by the new locking scheme
- updated commit message (Jan)
Since v3:
- re-work vpci_cancel_pending accordingly to the per-BAR handling
- s/num_mem_ranges/map_pending and s/uint8_t/bool
- ASSERT(bar->mem) in modify_bars
- create and destroy the rangesets on add/remove
---
 xen/drivers/vpci/header.c | 257 ++++++++++++++++++++++++++------------
 xen/drivers/vpci/vpci.c   |   6 +
 xen/include/xen/vpci.h    |   2 +-
 3 files changed, 185 insertions(+), 80 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 39e11e141b38..feccd070ddd0 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -162,63 +162,107 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
 
 bool vpci_process_pending(struct vcpu *v)
 {
-    if ( v->vpci.mem )
+    struct pci_dev *pdev = v->vpci.pdev;
+    struct map_data data = {
+        .d = v->domain,
+        .map = v->vpci.cmd & PCI_COMMAND_MEMORY,
+    };
+    struct vpci_header *header = NULL;
+    unsigned int i;
+
+    if ( !pdev )
+        return false;
+
+    read_lock(&v->domain->pci_lock);
+
+    if ( !pdev->vpci || (v->domain != pdev->domain) )
     {
-        struct map_data data = {
-            .d = v->domain,
-            .map = v->vpci.cmd & PCI_COMMAND_MEMORY,
-        };
-        int rc = rangeset_consume_ranges(v->vpci.mem, map_range, &data);
+        v->vpci.pdev = NULL;
+        read_unlock(&v->domain->pci_lock);
+        return false;
+    }
+
+    header = &pdev->vpci->header;
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+    {
+        struct vpci_bar *bar = &header->bars[i];
+        int rc;
+
+        if ( rangeset_is_empty(bar->mem) )
+            continue;
+
+        rc = rangeset_consume_ranges(bar->mem, map_range, &data);
 
         if ( rc == -ERESTART )
+        {
+            read_unlock(&v->domain->pci_lock);
             return true;
+        }
 
-        write_lock(&v->domain->pci_lock);
-        spin_lock(&v->vpci.pdev->vpci->lock);
-        /* Disable memory decoding unconditionally on failure. */
-        modify_decoding(v->vpci.pdev,
-                        rc ? v->vpci.cmd & ~PCI_COMMAND_MEMORY : v->vpci.cmd,
-                        !rc && v->vpci.rom_only);
-        spin_unlock(&v->vpci.pdev->vpci->lock);
-
-        rangeset_destroy(v->vpci.mem);
-        v->vpci.mem = NULL;
         if ( rc )
-            /*
-             * FIXME: in case of failure remove the device from the domain.
-             * Note that there might still be leftover mappings. While this is
-             * safe for Dom0, for DomUs the domain will likely need to be
-             * killed in order to avoid leaking stale p2m mappings on
-             * failure.
-             */
-            vpci_deassign_device(v->vpci.pdev);
-        write_unlock(&v->domain->pci_lock);
+        {
+            spin_lock(&pdev->vpci->lock);
+            /* Disable memory decoding unconditionally on failure. */
+            modify_decoding(pdev, v->vpci.cmd & ~PCI_COMMAND_MEMORY,
+                            false);
+            spin_unlock(&pdev->vpci->lock);
+
+            /* Clean all the rangesets */
+            for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
+                if ( !rangeset_is_empty(header->bars[i].mem) )
+                     rangeset_purge(header->bars[i].mem);
+
+            v->vpci.pdev = NULL;
+
+            read_unlock(&v->domain->pci_lock);
+
+            if ( !is_hardware_domain(v->domain) )
+                domain_crash(v->domain);
+
+            return false;
+        }
     }
+    v->vpci.pdev = NULL;
+
+    spin_lock(&pdev->vpci->lock);
+    modify_decoding(pdev, v->vpci.cmd, v->vpci.rom_only);
+    spin_unlock(&pdev->vpci->lock);
+
+    read_unlock(&v->domain->pci_lock);
 
     return false;
 }
 
 static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
-                            struct rangeset *mem, uint16_t cmd)
+                            uint16_t cmd)
 {
     struct map_data data = { .d = d, .map = true };
-    int rc;
+    struct vpci_header *header = &pdev->vpci->header;
+    int rc = 0;
+    unsigned int i;
 
     ASSERT(rw_is_write_locked(&d->pci_lock));
 
-    while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART )
+    for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
     {
-        /*
-         * It's safe to drop and reacquire the lock in this context
-         * without risking pdev disappearing because devices cannot be
-         * removed until the initial domain has been started.
-         */
-        write_unlock(&d->pci_lock);
-        process_pending_softirqs();
-        write_lock(&d->pci_lock);
-    }
+        struct vpci_bar *bar = &header->bars[i];
 
-    rangeset_destroy(mem);
+        if ( rangeset_is_empty(bar->mem) )
+            continue;
+
+        while ( (rc = rangeset_consume_ranges(bar->mem, map_range,
+                                              &data)) == -ERESTART )
+        {
+            /*
+             * It's safe to drop and reacquire the lock in this context
+             * without risking pdev disappearing because devices cannot be
+             * removed until the initial domain has been started.
+             */
+            write_unlock(&d->pci_lock);
+            process_pending_softirqs();
+            write_lock(&d->pci_lock);
+        }
+    }
     if ( !rc )
         modify_decoding(pdev, cmd, false);
 
@@ -226,7 +270,7 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
 }
 
 static void defer_map(struct domain *d, struct pci_dev *pdev,
-                      struct rangeset *mem, uint16_t cmd, bool rom_only)
+                      uint16_t cmd, bool rom_only)
 {
     struct vcpu *curr = current;
 
@@ -237,7 +281,6 @@ static void defer_map(struct domain *d, struct pci_dev *pdev,
      * started for the same device if the domain is not well-behaved.
      */
     curr->vpci.pdev = pdev;
-    curr->vpci.mem = mem;
     curr->vpci.cmd = cmd;
     curr->vpci.rom_only = rom_only;
     /*
@@ -251,33 +294,33 @@ static void defer_map(struct domain *d, struct pci_dev *pdev,
 static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
 {
     struct vpci_header *header = &pdev->vpci->header;
-    struct rangeset *mem = rangeset_new(NULL, NULL, 0);
     struct pci_dev *tmp, *dev = NULL;
     const struct domain *d;
     const struct vpci_msix *msix = pdev->vpci->msix;
-    unsigned int i;
+    unsigned int i, j;
     int rc;
 
     ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
 
-    if ( !mem )
-        return -ENOMEM;
-
     /*
-     * Create a rangeset that represents the current device BARs memory region
-     * and compare it against all the currently active BAR memory regions. If
-     * an overlap is found, subtract it from the region to be mapped/unmapped.
+     * Create a rangeset per BAR that represents the current device memory
+     * region and compare it against all the currently active BAR memory
+     * regions. If an overlap is found, subtract it from the region to be
+     * mapped/unmapped.
      *
-     * First fill the rangeset with all the BARs of this device or with the ROM
+     * First fill the rangesets with the BAR of this device or with the ROM
      * BAR only, depending on whether the guest is toggling the memory decode
      * bit of the command register, or the enable bit of the ROM BAR register.
      */
     for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
     {
-        const struct vpci_bar *bar = &header->bars[i];
+        struct vpci_bar *bar = &header->bars[i];
         unsigned long start = PFN_DOWN(bar->addr);
         unsigned long end = PFN_DOWN(bar->addr + bar->size - 1);
 
+        if ( !bar->mem )
+            continue;
+
         if ( !MAPPABLE_BAR(bar) ||
              (rom_only ? bar->type != VPCI_BAR_ROM
                        : (bar->type == VPCI_BAR_ROM && !header->rom_enabled)) ||
@@ -293,14 +336,31 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
             continue;
         }
 
-        rc = rangeset_add_range(mem, start, end);
+        rc = rangeset_add_range(bar->mem, start, end);
         if ( rc )
         {
             printk(XENLOG_G_WARNING "Failed to add [%lx, %lx]: %d\n",
                    start, end, rc);
-            rangeset_destroy(mem);
             return rc;
         }
+
+        /* Check for overlap with the already setup BAR ranges. */
+        for ( j = 0; j < i; j++ )
+        {
+            struct vpci_bar *prev_bar = &header->bars[j];
+
+            if ( rangeset_is_empty(prev_bar->mem) )
+                continue;
+
+            rc = rangeset_remove_range(prev_bar->mem, start, end);
+            if ( rc )
+            {
+                gprintk(XENLOG_WARNING,
+                       "%pp: failed to remove overlapping range [%lx, %lx]: %d\n",
+                        &pdev->sbdf, start, end, rc);
+                return rc;
+            }
+        }
     }
 
     /* Remove any MSIX regions if present. */
@@ -310,14 +370,21 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
         unsigned long end = PFN_DOWN(vmsix_table_addr(pdev->vpci, i) +
                                      vmsix_table_size(pdev->vpci, i) - 1);
 
-        rc = rangeset_remove_range(mem, start, end);
-        if ( rc )
+        for ( j = 0; j < ARRAY_SIZE(header->bars); j++ )
         {
-            printk(XENLOG_G_WARNING
-                   "Failed to remove MSIX table [%lx, %lx]: %d\n",
-                   start, end, rc);
-            rangeset_destroy(mem);
-            return rc;
+            const struct vpci_bar *bar = &header->bars[j];
+
+            if ( rangeset_is_empty(bar->mem) )
+                continue;
+
+            rc = rangeset_remove_range(bar->mem, start, end);
+            if ( rc )
+            {
+                gprintk(XENLOG_WARNING,
+                       "%pp: failed to remove MSIX table [%lx, %lx]: %d\n",
+                        &pdev->sbdf, start, end, rc);
+                return rc;
+            }
         }
     }
 
@@ -357,27 +424,37 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
 
             for ( i = 0; i < ARRAY_SIZE(tmp->vpci->header.bars); i++ )
             {
-                const struct vpci_bar *bar = &tmp->vpci->header.bars[i];
-                unsigned long start = PFN_DOWN(bar->addr);
-                unsigned long end = PFN_DOWN(bar->addr + bar->size - 1);
-
-                if ( !bar->enabled ||
-                     !rangeset_overlaps_range(mem, start, end) ||
-                     /*
-                      * If only the ROM enable bit is toggled check against
-                      * other BARs in the same device for overlaps, but not
-                      * against the same ROM BAR.
-                      */
-                     (rom_only && tmp == pdev && bar->type == VPCI_BAR_ROM) )
+                const struct vpci_bar *remote_bar = &tmp->vpci->header.bars[i];
+                unsigned long start = PFN_DOWN(remote_bar->addr);
+                unsigned long end = PFN_DOWN(remote_bar->addr +
+                                             remote_bar->size - 1);
+
+                if ( !remote_bar->enabled )
                     continue;
 
-                rc = rangeset_remove_range(mem, start, end);
-                if ( rc )
+                for ( j = 0; j < ARRAY_SIZE(header->bars); j++)
                 {
-                    printk(XENLOG_G_WARNING "Failed to remove [%lx, %lx]: %d\n",
-                           start, end, rc);
-                    rangeset_destroy(mem);
-                    return rc;
+                    const struct vpci_bar *bar = &header->bars[j];
+
+                    if ( !rangeset_overlaps_range(bar->mem, start, end) ||
+                         /*
+                          * If only the ROM enable bit is toggled check against
+                          * other BARs in the same device for overlaps, but not
+                          * against the same ROM BAR.
+                          */
+                         (rom_only &&
+                          tmp == pdev &&
+                          bar->type == VPCI_BAR_ROM) )
+                        continue;
+
+                    rc = rangeset_remove_range(bar->mem, start, end);
+                    if ( rc )
+                    {
+                        gprintk(XENLOG_WARNING,
+                                "%pp: failed to remove [%lx, %lx]: %d\n",
+                                &pdev->sbdf, start, end, rc);
+                        return rc;
+                    }
                 }
             }
         }
@@ -401,10 +478,10 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
          * will always be to establish mappings and process all the BARs.
          */
         ASSERT((cmd & PCI_COMMAND_MEMORY) && !rom_only);
-        return apply_map(pdev->domain, pdev, mem, cmd);
+        return apply_map(pdev->domain, pdev, cmd);
     }
 
-    defer_map(dev->domain, dev, mem, cmd, rom_only);
+    defer_map(dev->domain, dev, cmd, rom_only);
 
     return 0;
 }
@@ -593,6 +670,18 @@ static void cf_check rom_write(
         rom->addr = val & PCI_ROM_ADDRESS_MASK;
 }
 
+static int bar_add_rangeset(const struct pci_dev *pdev, struct vpci_bar *bar,
+                            unsigned int i)
+{
+    char str[32];
+
+    snprintf(str, sizeof(str), "%pp:BAR%u", &pdev->sbdf, i);
+
+    bar->mem = rangeset_new(pdev->domain, str, RANGESETF_no_print);
+
+    return !bar->mem ? -ENOMEM : 0;
+}
+
 static int cf_check init_header(struct pci_dev *pdev)
 {
     uint16_t cmd;
@@ -748,6 +837,10 @@ static int cf_check init_header(struct pci_dev *pdev)
         else
             bars[i].type = VPCI_BAR_MEM32;
 
+        rc = bar_add_rangeset(pdev, &bars[i], i);
+        if ( rc )
+            goto fail;
+
         rc = pci_size_mem_bar(pdev->sbdf, reg, &addr, &size,
                               (i == num_bars - 1) ? PCI_BAR_LAST : 0);
         if ( rc < 0 )
@@ -798,6 +891,12 @@ static int cf_check init_header(struct pci_dev *pdev)
                                4, rom);
         if ( rc )
             rom->type = VPCI_BAR_EMPTY;
+        else
+        {
+            rc = bar_add_rangeset(pdev, rom, num_bars);
+            if ( rc )
+                goto fail;
+        }
     }
     else if ( !is_hwdom )
     {
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index ff4837391786..260b72875ee1 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -42,6 +42,8 @@ extern vpci_register_init_t *const __end_vpci_array[];
 
 void vpci_deassign_device(struct pci_dev *pdev)
 {
+    unsigned int i;
+
     ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
 
     if ( !has_vpci(pdev->domain) || !pdev->vpci )
@@ -67,6 +69,10 @@ void vpci_deassign_device(struct pci_dev *pdev)
             if ( pdev->vpci->msix->table[i] )
                 iounmap(pdev->vpci->msix->table[i]);
     }
+
+    for ( i = 0; i < ARRAY_SIZE(pdev->vpci->header.bars); i++ )
+        rangeset_destroy(pdev->vpci->header.bars[i].mem);
+
     xfree(pdev->vpci->msix);
     xfree(pdev->vpci->msi);
     xfree(pdev->vpci);
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index b0e38a5a1acb..817ee9ee7300 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -92,6 +92,7 @@ struct vpci {
             /* Guest address. */
             uint64_t guest_addr;
             uint64_t size;
+            struct rangeset *mem;
             enum {
                 VPCI_BAR_EMPTY,
                 VPCI_BAR_IO,
@@ -176,7 +177,6 @@ struct vpci {
 
 struct vpci_vcpu {
     /* Per-vcpu structure to store state while {un}mapping of PCI BARs. */
-    struct rangeset *mem;
     struct pci_dev *pdev;
     uint16_t cmd;
     bool rom_only : 1;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:37:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:37:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675232.1050550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Dk-0008Ho-Dj; Fri, 02 Feb 2024 21:37:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675232.1050550; Fri, 02 Feb 2024 21:37:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Dk-0008Hh-Aj; Fri, 02 Feb 2024 21:37:12 +0000
Received: by outflank-mailman (input) for mailman id 675232;
 Fri, 02 Feb 2024 21:37:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1Di-0008HK-Ny
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:37:10 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 36d2369e-c213-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 22:37:08 +0100 (CET)
Received: from DM6PR02CA0166.namprd02.prod.outlook.com (2603:10b6:5:332::33)
 by SA0PR12MB4352.namprd12.prod.outlook.com (2603:10b6:806:9c::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.9; Fri, 2 Feb
 2024 21:37:04 +0000
Received: from CY4PEPF0000E9DB.namprd05.prod.outlook.com
 (2603:10b6:5:332:cafe::8) by DM6PR02CA0166.outlook.office365.com
 (2603:10b6:5:332::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.26 via Frontend
 Transport; Fri, 2 Feb 2024 21:37:03 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9DB.mail.protection.outlook.com (10.167.241.81) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 21:37:03 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:37:03 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:37:02 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:36:56 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36d2369e-c213-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=VOwCokeJ+nAkYL5YQpg0yBcovVPhOFz8a3t2YyQCXNDgWZC+K+8Dr0k6eQU9WCr3yBFUlEgEJaHK9fLtMzw2de4XDyge8oc7GgqP58i6snvaHOmDZWakuAWHzVgkOzPuqQ64AeE3yJvKEDcEAtz2t5X0b9HLvyvonS/A3n0MfMiqPt9ERI2jdhtRrsBkh0vzjkHU8Ocz7DKJy0eJxkUKkSAWzGlLqxOuvLSD5l3iepdK+axqF4jLcD8LXWhwxJAyI170bi1OvbyFLwTGxkQK/1UZ8Yb5zdEHF7hcOj2LIKp2XFbeYSa7I2XeIPlC4ljktNDW4evQCOinuP/CpPLl9g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=GP/sADPnaw4TMrLkrvzS2aR+Na6StFaFDNxGi2x57hw=;
 b=DX6OsOgxQt9xx84rX1T8v1+ukzDYU5QSJ1OHkweUcF7y0I+SLELcJYvVnwYbsfCEMX2iZQfO+YnyJJr8fhkalCMcE4h7tLu8evOtddP6MwjGEFoCt/qIgYXm4rB/vjgALNj92KoZLFhUgDbraehnbWqANETDl2mMjMiYhWLSeJR4JCBRfdDvbR2xUHcnT9H96YEFpgYZA9yKaQ6V4of4FASD3FCNSknZVCVUO7HLgPELpBvsHNe7RZbWrXGy3c6v9UYwPqyzmjJuPlu8ikybEUGYSK+TTolkBx6Mj7hxMs3lK91gRYz8qq++qbgmyjyJBCQVKaRsGW5hqnGiIZ696A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GP/sADPnaw4TMrLkrvzS2aR+Na6StFaFDNxGi2x57hw=;
 b=VDB96vADUAr6Y/qYqTv3M41heMpZkBxJjyiMJe6yw1d4pF+GsdBS6jzBCPdpRBMj2iv73d3dMeE0CECizgPJ8uQXmCTsYx+SPdynt3NQKoYYreBOiiKM/QS67NovTfw/WkR0ddXTPYGLXHDCJqDcsBytB4URhT4JnIriW7ZqxlA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <volodymyr_babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>
Subject: [PATCH v13 14/14] arm/vpci: honor access size when returning an error
Date: Fri, 2 Feb 2024 16:33:18 -0500
Message-ID: <20240202213321.1920347-15-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9DB:EE_|SA0PR12MB4352:EE_
X-MS-Office365-Filtering-Correlation-Id: c8852deb-b112-438c-dec6-08dc243718cc
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	/g0GtkjSkbXxLposSJFD3xdRkY3vRTT0d+AqyCBY6ZRKFEEPkf1roKI9wPAaLM1jVIIFORc/gltpxNAzvC4PiCurY9foWBpR7FMmf58OEPxQPizzKi2tYScISHyeysPcPIvQOsfMMdgoV5LklvF84VvKlU36976RrxmnfqY0A1CNEwiy2O7OZBMCaac/hXMhdjwoPP0aA8rMrracOhiLWagWlETWLId5WSa1PjXF9y+OHc21RdLR9UBhT4n4yfRN4uI/DbHI7voAOLlEs6zQdC/ZNlbtfGQiZBDf9chIvT5JNfuAHMyNCFo0IuBFBs2WX9nrfvn968EnZGqSkVCAzY+v94XDjc7GxlEDm2I5KgFprrejhojINDWMd0wuMrKb7W/V0uwfGOVDFpdfrwi54F1ZjADSWF47KbVnvp4sTJ1mlVTczs0dMCLDSVizAKHG6C7wDu0rrmrtCIGE5vQ2ghrgwr4uqgtQKae1BsL+4900Cxzu/xMikSAyzFyk50RjEne+Qbs+/IE2v91VjExReM/OTpV3qyzx4mxzb/w/YHhbiC9/N6LBYcuhPrsREfrVvPJRyNTkSxlElDT6ur+Dr/l39OgcdRukXKz9jFUP4VXxkA5cbTV4bcCMgL61M0RJ//sXBoYKmkyf0b4BP4lm+YqawXq4q5lOpxejuRLpOo/MIvnoA+/yzPqfreVrU2SzPxkqWrm2EGWefDe18rEwr0n4CnD+dgiCREQz640Oba7mAmAELoGr0EdqGoLSf/LluUr97cfgCZHCVtORkH6YCA==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(376002)(346002)(396003)(39860400002)(230922051799003)(451199024)(82310400011)(186009)(64100799003)(1800799012)(36840700001)(40470700004)(46966006)(316002)(70206006)(2616005)(336012)(426003)(26005)(8936002)(41300700001)(36860700001)(6666004)(83380400001)(47076005)(1076003)(8676002)(4326008)(5660300002)(44832011)(2906002)(54906003)(478600001)(36756003)(70586007)(6916009)(86362001)(82740400003)(81166007)(356005)(40480700001)(40460700003)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:37:03.6554
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c8852deb-b112-438c-dec6-08dc243718cc
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9DB.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4352

From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>

Guest can try to read config space using different access sizes: 8,
16, 32, 64 bits. We need to take this into account when we are
returning an error back to MMIO handler, otherwise it is possible to
provide more data than requested: i.e. guest issues LDRB instruction
to read one byte, but we are writing 0xFFFFFFFFFFFFFFFF in the target
register.

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
v9->10:
* New patch in v10.
---
 xen/arch/arm/vpci.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 348ba0fbc860..aaf9d9120c3d 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -41,6 +41,8 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
 {
     struct pci_host_bridge *bridge = p;
     pci_sbdf_t sbdf;
+    const uint8_t access_size = (1 << info->dabt.size) * 8;
+    const uint64_t access_mask = GENMASK_ULL(access_size - 1, 0);
     /* data is needed to prevent a pointer cast on 32bit */
     unsigned long data;
 
@@ -48,7 +50,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
 
     if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
     {
-        *r = ~0UL;
+        *r = access_mask;
         return 1;
     }
 
@@ -59,7 +61,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
         return 1;
     }
 
-    *r = ~0UL;
+    *r = access_mask;
 
     return 0;
 }
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:37:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:37:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675233.1050559 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Ds-0000L2-N3; Fri, 02 Feb 2024 21:37:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675233.1050559; Fri, 02 Feb 2024 21:37:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Ds-0000K8-K5; Fri, 02 Feb 2024 21:37:20 +0000
Received: by outflank-mailman (input) for mailman id 675233;
 Fri, 02 Feb 2024 21:37:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1CT-0003tJ-81
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:35:53 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:240a::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 09c28148-c213-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:35:52 +0100 (CET)
Received: from SJ0PR13CA0064.namprd13.prod.outlook.com (2603:10b6:a03:2c4::9)
 by SA1PR12MB8919.namprd12.prod.outlook.com (2603:10b6:806:38e::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.24; Fri, 2 Feb
 2024 21:35:48 +0000
Received: from SJ1PEPF00001CDD.namprd05.prod.outlook.com
 (2603:10b6:a03:2c4:cafe::43) by SJ0PR13CA0064.outlook.office365.com
 (2603:10b6:a03:2c4::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.23 via Frontend
 Transport; Fri, 2 Feb 2024 21:35:48 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ1PEPF00001CDD.mail.protection.outlook.com (10.167.242.5) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7202.16 via Frontend Transport; Fri, 2 Feb 2024 21:35:47 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:35:46 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:35:45 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09c28148-c213-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IvOct/gWcaV2Y1adym3+GJbEf/FtNIA9bai7PPs4lDnFAN/wCpcgZG/dJ1mqADUEHECkSr0mPbweljWIG5hwo1mP6ijnoQ0+9T/0ugtUis1IsmmgqhpGP3ynxNCEk/sEghrJLlsPcfZn/Q0xMcRgrvk7QDCrgU6SonvViJOPqleMMCvWy06nCcWMVTfsP2STRiDtE7DZu4fp3b9sBXX5rn/PaaZbqFvSMkDnt7p4HBXYAK+cYW28O0GxGsp5dFh/QeHz121pxomK8VcCK413WoMLFTeb7FEXfgkf+BRRKDf8hGHpuuBk6gk7kmIHMDX3jYkY1rnb8ujHR7kNcXZ4mw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=BtY/WrBc7LpwZJnW3+E9MtL7jMXCACUj7apzU1DoyGY=;
 b=j/CO97zn7v7z/t5peFsom/yE4f3QJDs8isdlryGxR71RdMn5UwvVNEWZPrIq5erQqpSeeAOY66mLWOV57LHYjSmt9rEncmtleV0TPOXj+UHPRIYlZr/hEAvfwy3lROxWFEaDvK6ZUZ+HMvYR6ymkMxGl/zhTTd+zB9K8vFO4u8KdO0FEIWta9S/i7Ej5eMHpGedKoej21kQif8++/CQRv85M0ixzISJKI6ONPcceSJiovBOIwLbpUTvO4HfTYQ20LPFAYtLn7HYs/Vfu6WONIG1AyJulDNkbq0R+IRnupCW5L2/v7x19LhR7oRVrfptnZVnGkL1L763OC9p8hBbeuw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=BtY/WrBc7LpwZJnW3+E9MtL7jMXCACUj7apzU1DoyGY=;
 b=427UUaaXuaWL/LHQdkd+Alv5YOI/wFzCULiOsfEhM7FROQUNQNKFi+JpCKcvUSDWBBHStFk5EMkG41jA4vOHz6INTuUsP8ZqZt4u0JTWyUDmYRX1xSS1TlHkIsON9pB+NBN9j4aEOvRHXsA4kL+l1N5SjZUQUTkpnHFu/QWul0Y=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Daniel P.
 Smith" <dpsmith@apertussolutions.com>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>, Stewart Hildebrand <stewart.hildebrand@amd.com>
Subject: [PATCH v13 09/14] vpci/header: program p2m with guest BAR view
Date: Fri, 2 Feb 2024 16:33:13 -0500
Message-ID: <20240202213321.1920347-10-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CDD:EE_|SA1PR12MB8919:EE_
X-MS-Office365-Filtering-Correlation-Id: b1ba2c8c-ddbb-475d-f107-08dc2436eb68
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	hENbE8imIVJ2HS2JJqWAsEV6JyTe5iQDHbGHKofm+lIV0R9JNzosNzoh4LY9WuYY3RPaPcGxnVu3LIH8Px5K5eVtKOrUUChbKhEAadAwTIP3ym5oZpNxpx0LOQ4mqCRN0oCbzJTOPTyawkLtykymvJ7A4sfL3+pvJjMxTtIOulHT0VHqsu5+HMw7Qa6YDvvW3gdo6RR7O60GSeQMJII5lBNF7Ey+s1Hg8MMJxoQ3/rjPDN6dC7V7cFn9fZWNr4PsiSOb5/lro9eaU32lRTRBayRwqYikTD4tX1KkypMcCbSQLpSXtfv1EQFjCKYIKh/THBzZKeod62dWTvM/vRHiokgYN4KrvrNJrHWc4NkviZdtw1XZXZ0DPzR35aYuhs+RtBiw9FBy84KKQ+R6MAvrcd/Ebtj+W+mbdkq84OOJ/3gmcLKgYCZmF4bEPxQjnK1Fd680INqypcZRhj4t7p/LygcSTbHNYAygkXm7p4U5idwG/6978YkVgV50iD+AVEVbRXYjytbzMwfB+ecKMcExx+ToqyQMj1NbWIdvBxVqhSa5tYCibVzLhN/U/Tf/WYjLh+VM7Eg7+QxUE84tT9T5Ij+McwnvAz58fTqM9faYxaCpq8SuqlBMCHLhNdnm/dpo2U99a+oQfJoKsNJfvFS3F2bb5FVtzHvu0qKTCrRX/uJMMGeaxhuK8DfhuHZ9IvsLctcQHSqB6uYWJt4ZKs35wFb2CErhroSGm/AZJMq9JSbI16/V9HtynxAzt+VVNvf/iZZQ9Zssc7WDiSL/EoKAsg==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(346002)(376002)(136003)(396003)(230922051799003)(230273577357003)(451199024)(186009)(1800799012)(82310400011)(64100799003)(36840700001)(46966006)(40470700004)(426003)(336012)(6666004)(2616005)(1076003)(83380400001)(26005)(478600001)(41300700001)(36860700001)(8936002)(8676002)(4326008)(36756003)(6916009)(44832011)(356005)(316002)(40460700003)(40480700001)(2906002)(54906003)(70206006)(5660300002)(70586007)(82740400003)(81166007)(30864003)(47076005)(86362001)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:35:47.4874
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b1ba2c8c-ddbb-475d-f107-08dc2436eb68
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CDD.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8919

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Take into account guest's BAR view and program its p2m accordingly:
gfn is guest's view of the BAR and mfn is the physical BAR value.
This way hardware domain sees physical BAR values and guest sees
emulated ones.

Hardware domain continues getting the BARs identity mapped, while for
domUs the BARs are mapped at the requested guest address without
modifying the BAR address in the device PCI config space.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
---
In v12.3:
- Update arguments passed to permission error prints in map_range()
In v12.2:
- Slightly tweak print format in modify_bars()
In v12.1:
- ASSERT(rangeset_is_empty()) in modify_bars()
- Fixup print format in modify_bars()
- Make comment single line in bar_write()
- Add Roger's R-b
In v12:
- Update guest_addr in rom_write()
- Use unsigned long for start_mfn and map_mfn to reduce mfn_x() calls
- Use existing vmsix_table_*() functions
- Change vmsix_table_base() to use .guest_addr
In v11:
- Add vmsix_guest_table_addr() and vmsix_guest_table_base() functions
  to access guest's view of the VMSIx tables.
- Use MFN (not GFN) to check access permissions
- Move page offset check to this patch
- Call rangeset_remove_range() with correct parameters
In v10:
- Moved GFN variable definition outside the loop in map_range()
- Updated printk error message in map_range()
- Now BAR address is always stored in bar->guest_addr, even for
  HW dom, this removes bunch of ugly is_hwdom() checks in modify_bars()
- vmsix_table_base() now uses .guest_addr instead of .addr
In v9:
- Extended the commit message
- Use bar->guest_addr in modify_bars
- Extended printk error message in map_range
- Moved map_data initialization so .bar can be initialized during declaration
Since v5:
- remove debug print in map_range callback
- remove "identity" from the debug print
Since v4:
- moved start_{gfn|mfn} calculation into map_range
- pass vpci_bar in the map_data instead of start_{gfn|mfn}
- s/guest_addr/guest_reg
Since v3:
- updated comment (Roger)
- removed gfn_add(map->start_gfn, rc); which is wrong
- use v->domain instead of v->vpci.pdev->domain
- removed odd e.g. in comment
- s/d%d/%pd in altered code
- use gdprintk for map/unmap logs
Since v2:
- improve readability for data.start_gfn and restructure ?: construct
Since v1:
 - s/MSI/MSI-X in comments
---
 xen/drivers/vpci/header.c | 83 ++++++++++++++++++++++++++++++---------
 xen/include/xen/vpci.h    |  3 +-
 2 files changed, 66 insertions(+), 20 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index feccd070ddd0..47648c395132 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -34,6 +34,7 @@
 
 struct map_data {
     struct domain *d;
+    const struct vpci_bar *bar;
     bool map;
 };
 
@@ -41,26 +42,37 @@ static int cf_check map_range(
     unsigned long s, unsigned long e, void *data, unsigned long *c)
 {
     const struct map_data *map = data;
+    /* Start address of the BAR as seen by the guest. */
+    unsigned long start_gfn = PFN_DOWN(map->bar->guest_addr);
+    /* Physical start address of the BAR. */
+    unsigned long start_mfn = PFN_DOWN(map->bar->addr);
     int rc;
 
     for ( ; ; )
     {
         unsigned long size = e - s + 1;
+        /*
+         * Ranges to be mapped don't always start at the BAR start address, as
+         * there can be holes or partially consumed ranges. Account for the
+         * offset of the current address from the BAR start.
+         */
+        unsigned long map_mfn = start_mfn + s - start_gfn;
+        unsigned long m_end = map_mfn + size - 1;
 
-        if ( !iomem_access_permitted(map->d, s, e) )
+        if ( !iomem_access_permitted(map->d, map_mfn, m_end) )
         {
             printk(XENLOG_G_WARNING
                    "%pd denied access to MMIO range [%#lx, %#lx]\n",
-                   map->d, s, e);
+                   map->d, map_mfn, m_end);
             return -EPERM;
         }
 
-        rc = xsm_iomem_mapping(XSM_HOOK, map->d, s, e, map->map);
+        rc = xsm_iomem_mapping(XSM_HOOK, map->d, map_mfn, m_end, map->map);
         if ( rc )
         {
             printk(XENLOG_G_WARNING
                    "%pd XSM denied access to MMIO range [%#lx, %#lx]: %d\n",
-                   map->d, s, e, rc);
+                   map->d, map_mfn, m_end, rc);
             return rc;
         }
 
@@ -73,8 +85,8 @@ static int cf_check map_range(
          * - {un}map_mmio_regions doesn't support preemption.
          */
 
-        rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, _mfn(s))
-                      : unmap_mmio_regions(map->d, _gfn(s), size, _mfn(s));
+        rc = map->map ? map_mmio_regions(map->d, _gfn(s), size, _mfn(map_mfn))
+                      : unmap_mmio_regions(map->d, _gfn(s), size, _mfn(map_mfn));
         if ( rc == 0 )
         {
             *c += size;
@@ -83,8 +95,9 @@ static int cf_check map_range(
         if ( rc < 0 )
         {
             printk(XENLOG_G_WARNING
-                   "Failed to identity %smap [%lx, %lx] for d%d: %d\n",
-                   map->map ? "" : "un", s, e, map->d->domain_id, rc);
+                   "Failed to %smap [%lx %lx] -> [%lx %lx] for %pd: %d\n",
+                   map->map ? "" : "un", s, e, map_mfn,
+                   map_mfn + size, map->d, rc);
             break;
         }
         ASSERT(rc < size);
@@ -163,10 +176,6 @@ static void modify_decoding(const struct pci_dev *pdev, uint16_t cmd,
 bool vpci_process_pending(struct vcpu *v)
 {
     struct pci_dev *pdev = v->vpci.pdev;
-    struct map_data data = {
-        .d = v->domain,
-        .map = v->vpci.cmd & PCI_COMMAND_MEMORY,
-    };
     struct vpci_header *header = NULL;
     unsigned int i;
 
@@ -186,6 +195,11 @@ bool vpci_process_pending(struct vcpu *v)
     for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
     {
         struct vpci_bar *bar = &header->bars[i];
+        struct map_data data = {
+            .d = v->domain,
+            .map = v->vpci.cmd & PCI_COMMAND_MEMORY,
+            .bar = bar,
+        };
         int rc;
 
         if ( rangeset_is_empty(bar->mem) )
@@ -236,7 +250,6 @@ bool vpci_process_pending(struct vcpu *v)
 static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
                             uint16_t cmd)
 {
-    struct map_data data = { .d = d, .map = true };
     struct vpci_header *header = &pdev->vpci->header;
     int rc = 0;
     unsigned int i;
@@ -246,6 +259,7 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
     for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
     {
         struct vpci_bar *bar = &header->bars[i];
+        struct map_data data = { .d = d, .map = true, .bar = bar };
 
         if ( rangeset_is_empty(bar->mem) )
             continue;
@@ -311,12 +325,16 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
      * First fill the rangesets with the BAR of this device or with the ROM
      * BAR only, depending on whether the guest is toggling the memory decode
      * bit of the command register, or the enable bit of the ROM BAR register.
+     *
+     * For non-hardware domain we use guest physical addresses.
      */
     for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
     {
         struct vpci_bar *bar = &header->bars[i];
         unsigned long start = PFN_DOWN(bar->addr);
         unsigned long end = PFN_DOWN(bar->addr + bar->size - 1);
+        unsigned long start_guest = PFN_DOWN(bar->guest_addr);
+        unsigned long end_guest = PFN_DOWN(bar->guest_addr + bar->size - 1);
 
         if ( !bar->mem )
             continue;
@@ -336,11 +354,26 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
             continue;
         }
 
-        rc = rangeset_add_range(bar->mem, start, end);
+        ASSERT(rangeset_is_empty(bar->mem));
+
+        /*
+         * Make sure that the guest set address has the same page offset
+         * as the physical address on the host or otherwise things won't work as
+         * expected.
+         */
+        if ( PAGE_OFFSET(bar->guest_addr) != PAGE_OFFSET(bar->addr) )
+        {
+            gprintk(XENLOG_G_WARNING,
+                    "%pp: can't map BAR%u - offset mismatch: %#lx vs %#lx\n",
+                    &pdev->sbdf, i, bar->guest_addr, bar->addr);
+            return -EINVAL;
+        }
+
+        rc = rangeset_add_range(bar->mem, start_guest, end_guest);
         if ( rc )
         {
             printk(XENLOG_G_WARNING "Failed to add [%lx, %lx]: %d\n",
-                   start, end, rc);
+                   start_guest, end_guest, rc);
             return rc;
         }
 
@@ -352,12 +385,12 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
             if ( rangeset_is_empty(prev_bar->mem) )
                 continue;
 
-            rc = rangeset_remove_range(prev_bar->mem, start, end);
+            rc = rangeset_remove_range(prev_bar->mem, start_guest, end_guest);
             if ( rc )
             {
                 gprintk(XENLOG_WARNING,
                        "%pp: failed to remove overlapping range [%lx, %lx]: %d\n",
-                        &pdev->sbdf, start, end, rc);
+                        &pdev->sbdf, start_guest, end_guest, rc);
                 return rc;
             }
         }
@@ -425,8 +458,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
             for ( i = 0; i < ARRAY_SIZE(tmp->vpci->header.bars); i++ )
             {
                 const struct vpci_bar *remote_bar = &tmp->vpci->header.bars[i];
-                unsigned long start = PFN_DOWN(remote_bar->addr);
-                unsigned long end = PFN_DOWN(remote_bar->addr +
+                unsigned long start = PFN_DOWN(remote_bar->guest_addr);
+                unsigned long end = PFN_DOWN(remote_bar->guest_addr +
                                              remote_bar->size - 1);
 
                 if ( !remote_bar->enabled )
@@ -513,6 +546,8 @@ static void cf_check bar_write(
     struct vpci_bar *bar = data;
     bool hi = false;
 
+    ASSERT(is_hardware_domain(pdev->domain));
+
     if ( bar->type == VPCI_BAR_MEM64_HI )
     {
         ASSERT(reg > PCI_BASE_ADDRESS_0);
@@ -543,6 +578,8 @@ static void cf_check bar_write(
      */
     bar->addr &= ~(0xffffffffULL << (hi ? 32 : 0));
     bar->addr |= (uint64_t)val << (hi ? 32 : 0);
+    /* Update guest address, so hardware domain BAR is identity mapped. */
+    bar->guest_addr = bar->addr;
 
     /* Make sure Xen writes back the same value for the BAR RO bits. */
     if ( !hi )
@@ -639,11 +676,14 @@ static void cf_check rom_write(
     }
 
     if ( !rom->enabled )
+    {
         /*
          * If the ROM BAR is not mapped update the address field so the
          * correct address is mapped into the p2m.
          */
         rom->addr = val & PCI_ROM_ADDRESS_MASK;
+        rom->guest_addr = rom->addr;
+    }
 
     if ( !header->bars_mapped || rom->enabled == new_enabled )
     {
@@ -667,7 +707,10 @@ static void cf_check rom_write(
         return;
 
     if ( !new_enabled )
+    {
         rom->addr = val & PCI_ROM_ADDRESS_MASK;
+        rom->guest_addr = rom->addr;
+    }
 }
 
 static int bar_add_rangeset(const struct pci_dev *pdev, struct vpci_bar *bar,
@@ -862,6 +905,7 @@ static int cf_check init_header(struct pci_dev *pdev)
         }
 
         bars[i].addr = addr;
+        bars[i].guest_addr = addr;
         bars[i].size = size;
         bars[i].prefetchable = val & PCI_BASE_ADDRESS_MEM_PREFETCH;
 
@@ -884,6 +928,7 @@ static int cf_check init_header(struct pci_dev *pdev)
         rom->type = VPCI_BAR_ROM;
         rom->size = size;
         rom->addr = addr;
+        rom->guest_addr = addr;
         header->rom_enabled = pci_conf_read32(pdev->sbdf, rom_reg) &
                               PCI_ROM_ADDRESS_ENABLE;
 
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 817ee9ee7300..e89c571890b2 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -216,7 +216,8 @@ int vpci_msix_arch_print(const struct vpci_msix *msix);
  */
 static inline paddr_t vmsix_table_base(const struct vpci *vpci, unsigned int nr)
 {
-    return vpci->header.bars[vpci->msix->tables[nr] & PCI_MSIX_BIRMASK].addr;
+    return vpci->header.bars[vpci->msix->tables[nr] &
+                             PCI_MSIX_BIRMASK].guest_addr;
 }
 
 static inline paddr_t vmsix_table_addr(const struct vpci *vpci, unsigned int nr)
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:37:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:37:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675235.1050570 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Du-0000nU-3z; Fri, 02 Feb 2024 21:37:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675235.1050570; Fri, 02 Feb 2024 21:37:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Du-0000nG-10; Fri, 02 Feb 2024 21:37:22 +0000
Received: by outflank-mailman (input) for mailman id 675235;
 Fri, 02 Feb 2024 21:37:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1DN-0003tJ-W2
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:36:49 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:240a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2be7ad32-c213-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:36:49 +0100 (CET)
Received: from DM6PR02CA0140.namprd02.prod.outlook.com (2603:10b6:5:332::7) by
 SJ0PR12MB6686.namprd12.prod.outlook.com (2603:10b6:a03:479::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34; Fri, 2 Feb
 2024 21:36:45 +0000
Received: from CY4PEPF0000E9DB.namprd05.prod.outlook.com
 (2603:10b6:5:332:cafe::f) by DM6PR02CA0140.outlook.office365.com
 (2603:10b6:5:332::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.23 via Frontend
 Transport; Fri, 2 Feb 2024 21:36:45 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9DB.mail.protection.outlook.com (10.167.241.81) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 21:36:45 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:36:45 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:36:44 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:36:38 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2be7ad32-c213-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EVSXQSO3N1wbviduaw4m+NX851dUp+Bn5GdbSbl1f9NkPiXE/OTQt0D/EOWJVNehf5Sjxv++aoZ5wOB93rNvoKW+Qsc4qYBvfMkJnCZ5qOcuIlYDFE8r4sb3b0HSx5FJKsJ0Ud/PTzrwVnHF54TY3q5zJwoHTMogHAyLTnH/7JmJACBXpXugm56GNaZRROqlxug0zOX5poLGR+h7M55BxVKuv4kwSwTfJ8d1YnzlmUkGWXr7ueop+nuv8OHAwBGvbm1rXNkXB69Jxjxfn+JFkcPfoyMRHgUXsG2OGrgS77bEc2RGn1P8LAxw/Vq8eaLZH6ZFpMBOkhk48Cfm6Klzpg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=PsoOL/G8DIgXV37+ao0tajW6v6ZQDcdP8FQCKLy06D4=;
 b=Uz9QF7TrTujvGsb+B1GRDjC6D2j1ByKYm8vrEzErYgIC15SMtrNo0bQR4XH35WN2Kz6uhbHNh9E4TcJxA8iJ75FsR/OPiHLu2BCVhlRJT3dlt19Udsc7fULUPe+bNUn5ZqhthStp2yxGCA35v1N/Lcq10Ko2fgOYi/2pDm2EOvk+dNLbC6XHWFBMsr50GqeldBmKw2mIpT73/8okYnWu/Cq443CnCi5psuTdgMrDkIKWXKFMIlwFeP8f05KjaKjwtbDAZqfFCmCE2up80X3RHcxJmP5VC22PX3r1u26Ckgp4OPFKu7N4aB9o+DGMni0Hes/ywvEtdvfMyP/Qpb9GyA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PsoOL/G8DIgXV37+ao0tajW6v6ZQDcdP8FQCKLy06D4=;
 b=cKtf/lvYRNTuY/AAUj8hOqEzr1bBc+OvemF3Ct0LRXY7U9SlsdHIwOt2gpcj0AI1mIYs9Pig59wkfblvrQK8kguPhTsXecjRe/5Rqt6usDeatILb92vSW4RwV0zYU3KJV5cczZp+JkFBOw0+sDmhvBzyvmATvBuzDvNDxZl1vp8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand
 Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Julien Grall
	<jgrall@amazon.com>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, Stewart
 Hildebrand <stewart.hildebrand@amd.com>
Subject: [PATCH v13 13/14] xen/arm: account IO handlers for emulated PCI MSI-X
Date: Fri, 2 Feb 2024 16:33:17 -0500
Message-ID: <20240202213321.1920347-14-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9DB:EE_|SJ0PR12MB6686:EE_
X-MS-Office365-Filtering-Correlation-Id: 3c3a574b-cb66-4213-990d-08dc24370e01
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	NfwLVJTBJiWeqB0L159LB+HbaZ/MGzX0kikt5eg8b/pmVJSknxC+yL47d06bnnVTKB3/k8Y9KZSJqmLpoyxBV4DvJJfS4AsQor6X7rY9f3kTPK7E3A9xuKxksTjc4K+FgHD6xJDZVlgZYTSwPVw+kDo1tedBqwg69+L//Dgv+nlGyimAySVjO3eAlgo7yids20tOzioL68IkqQA2TrcDtPX9HE2KeQxwoEYAPUpONfCbGH9w14iuAInYv/VtIPyU/dNsICzCi0Tuopybdft6OVbMakDpUjC8T1gNysNQH1z35vTk2zLE8QWk/lIL6nUZOF3Wdc2B6PhRfQorPKwbnyHizGZHpAaSN0NiiU95bPUO6g4UeAA6deV6wva2tnobtncgyPruN68Z7a+NKFkndxCBhOpwFz/eut3cYg70mVQyzw8S98OYTTY20sExjYf75Q4g5KHplbfQdeImU8SIlT1Ixi2BsMLCv89Wai0bTD8gR8cp/e6nT4oabSt/eudZZOvg1dcNaINT1kNXO0Am6TYGkYwW2spF64qDumw0bcuvCDa02HlTfs0lyAr5gdt8t8alBczi8gXnNCgEHRby3lb64lA0ThjvGySQ6LXlaoi5sEsxdyqYqSRUymuJMz+I/fEdP9ObHq5ohCMI2ErublMbWt1a30m9EE9kS3CByXDe4+fVvmEiKDlNjPeKvkJCl8taCZKycessNU18ZIrAvdAaM0/hb2kbw05Kz+E4UHSQtUWtH2e24+aAoAjGOndz6/1QeGoz8gjh1PIbuJw06A==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(376002)(346002)(396003)(39860400002)(230922051799003)(1800799012)(64100799003)(186009)(82310400011)(451199024)(46966006)(36840700001)(40470700004)(66899024)(336012)(54906003)(47076005)(36860700001)(8936002)(82740400003)(81166007)(41300700001)(36756003)(86362001)(15650500001)(70586007)(6666004)(8676002)(4326008)(316002)(426003)(83380400001)(26005)(44832011)(1076003)(2616005)(6916009)(5660300002)(2906002)(356005)(478600001)(70206006)(40460700003)(40480700001)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:36:45.5461
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3c3a574b-cb66-4213-990d-08dc24370e01
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9DB.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6686

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

At the moment, we always allocate an extra 16 slots for IO handlers
(see MAX_IO_HANDLER). So while adding IO trap handlers for the emulated
MSI-X registers we need to explicitly tell that we have additional IO
handlers, so those are accounted.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
This actually moved here from the part 2 of the prep work for PCI
passthrough on Arm as it seems to be the proper place for it.

Since v5:
- optimize with IS_ENABLED(CONFIG_HAS_PCI_MSI) since VPCI_MAX_VIRT_DEV is
  defined unconditionally
New in v5
---
 xen/arch/arm/vpci.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 7a6a0017d132..348ba0fbc860 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -130,6 +130,8 @@ static int vpci_get_num_handlers_cb(struct domain *d,
 
 unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
 {
+    unsigned int count;
+
     if ( !has_vpci(d) )
         return 0;
 
@@ -150,7 +152,17 @@ unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
      * For guests each host bridge requires one region to cover the
      * configuration space. At the moment, we only expose a single host bridge.
      */
-    return 1;
+    count = 1;
+
+    /*
+     * There's a single MSI-X MMIO handler that deals with both PBA
+     * and MSI-X tables per each PCI device being passed through.
+     * Maximum number of emulated virtual devices is VPCI_MAX_VIRT_DEV.
+     */
+    if ( IS_ENABLED(CONFIG_HAS_PCI_MSI) )
+        count += VPCI_MAX_VIRT_DEV;
+
+    return count;
 }
 
 /*
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:37:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:37:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675237.1050580 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Dv-0001Cy-Gw; Fri, 02 Feb 2024 21:37:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675237.1050580; Fri, 02 Feb 2024 21:37:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Dv-0001CX-BF; Fri, 02 Feb 2024 21:37:23 +0000
Received: by outflank-mailman (input) for mailman id 675237;
 Fri, 02 Feb 2024 21:37:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1C7-0004Ae-4j
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:35:31 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:200a::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fba113e6-c212-11ee-98f5-efadbce2ee36;
 Fri, 02 Feb 2024 22:35:29 +0100 (CET)
Received: from SJ0PR03CA0381.namprd03.prod.outlook.com (2603:10b6:a03:3a1::26)
 by CY5PR12MB6298.namprd12.prod.outlook.com (2603:10b6:930:21::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.13; Fri, 2 Feb
 2024 21:35:25 +0000
Received: from SJ1PEPF00001CE3.namprd05.prod.outlook.com
 (2603:10b6:a03:3a1:cafe::f) by SJ0PR03CA0381.outlook.office365.com
 (2603:10b6:a03:3a1::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.27 via Frontend
 Transport; Fri, 2 Feb 2024 21:35:24 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ1PEPF00001CE3.mail.protection.outlook.com (10.167.242.11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7202.16 via Frontend Transport; Fri, 2 Feb 2024 21:35:24 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:35:23 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:35:17 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fba113e6-c212-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=JfeohXYcssNx4LGdnMYO1l6efKA2Cu4pWzgNugzpbp0WI3n+F98n0GqUP0JLNg7pcfDIR9w4+wIDzaJawO8YRqDlMsEqd1C75+U+Y5zKeUlAoQNywrn4zZkXVaQ2O1DPy9sgOuYrSQnKqxDAu+RnKt+qFC5mnHCikAeH0r/FfPuXFf3YFquVam/ixdfbkRpC3cpSfSONj7sCnnEX4aow0gonPk7gYWmSkLDnYRDPBF/6YFyhFFRoSU8wVFOYWA2lQJWwLVN48eyJ68sar9B9WIZUmpNm3U3ig3c+9VZIvocvWifww/Bbd/zdz5bqfqzS7VJ8sKSJ0B7blP7xanF7NQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Qi56JEzWMgKoMsc50mwUebNiiljtIu6vzi5g/b8KIBY=;
 b=kEb6n/l4euu0BG8KsUgcZjMIxggmkdndvzmYPDqCQZ5dMuqXtMko0tbwAMm9CEgc3XxjM27iBxVSlpTC9803CtGNcPyXYe+JCPPpWTbqb1QsA+eG2ldPEIR6Lpg1OG4GCfkYqJB1rBBqcSzmBrVhEOv4hhUxxf2uLRMHqWQb9/XZejlIRe+2Jn/hCI76rQV6Uv7Z0R3CFT7525Lzgx5AP2XWXi1L8rltAkemmVdj7iFsu2pXQiNXiG5vZoLD/SXsinPnNYzId0c7lTL0NGAjjmGOt/KPMwxVE6zLqKLiKcYOWN1D+ktuXApxWm8VjAaNNCntqxMyBNgxyhPFUOJ6dQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Qi56JEzWMgKoMsc50mwUebNiiljtIu6vzi5g/b8KIBY=;
 b=k4Ys+m/wdNa1Pkyi/my9j9CLIW8jMOAnsTQBw3N+dqAndW7VMbuHc3KbcADrqjTnPgQgIxQKq7rBi9ImZECRkmumgh0HOXKzgDw9bjaDXT6gv/1SBxz53s1TyPdWT3ixpcoblttqHflhNH/U3KqHdNcKAd/0WNl+HeLgE0JxeH8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>, Stewart Hildebrand <stewart.hildebrand@amd.com>
Subject: [PATCH v13 07/14] rangeset: add rangeset_purge() function
Date: Fri, 2 Feb 2024 16:33:11 -0500
Message-ID: <20240202213321.1920347-8-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CE3:EE_|CY5PR12MB6298:EE_
X-MS-Office365-Filtering-Correlation-Id: cbd32703-21c8-4311-cf92-08dc2436dd84
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wA0uZjREnZav4KOm+CgHBzzFX+GbLx8RJW3/lFefgpDJGl5mpchTDav0KXHRhl/N1TJSeg1iC9aOx0EE41f/nC/S+jd5XCHXUQBGgrLn4d4jjjEfbG+T/wzFbwYWiKUBAYpSP0nLLDq1d0wScq8F7g14sGuYVi1Gw8LfXRWZgPpcfhVvsddbgSfCghTMzd/qhjHcGJKsubHplszvEe0Sek5+G0Bh026uVjSVf9KsEm8T04DDwjxyCiAHIGlWLJUGwZxHBMcLKL/zUL9WJCQTFz04o9+Vv9Ydx77/c6kdNkZc1hOoK8s3lXf9V1TPcuUOinXR/XNe9CrAFdBM5AExOHKnRq9lGWEB159TRKUMdDfpQytgtrCmwIrkLrJSqOU4fUrIgynNB2kEiixyn5x09BiW6miaPXuaqm6YoAn9oErMLJ+I0/oI03bOEDM1sQkoGB1ZrjqElR4q7ui/KDozBFz6pMb+EiZLr0+kwP8I9EKchyLkH/xgQD/9+vEa/B50HKgXLu3mGJyL1URtQtH0u3mgn++o0i78pK1oaETuOSBOMAJD4sTYKHeNTerFKztCF9iCJEbHGGTWuILWcbQJ0QlAdtHjtFynE6X4Liy7MLp+7hi04NB8VmGWSyz9UZbKRvbUi9clWbPD0cUs0fvZ29k5dwCgqT0xp2/kvwetHfA4OpTM9UKbMg6T5T22TFUZS4Y+YlUsrske9seX5UsEE5XCcMXsIVnAHjF+xBsPZC/RExBd0TgYPkAlSEF7ejqttL4VLYXtPuOBKIIHFqbnfQ==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(136003)(39860400002)(376002)(396003)(230922051799003)(64100799003)(451199024)(82310400011)(1800799012)(186009)(40470700004)(46966006)(36840700001)(36756003)(2906002)(5660300002)(41300700001)(1076003)(2616005)(83380400001)(336012)(26005)(6916009)(316002)(426003)(54906003)(6666004)(70206006)(70586007)(478600001)(40460700003)(40480700001)(36860700001)(47076005)(82740400003)(356005)(81166007)(8936002)(44832011)(4326008)(8676002)(86362001)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:35:24.1820
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: cbd32703-21c8-4311-cf92-08dc2436dd84
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CE3.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6298

From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>

This function can be used when user wants to remove all rangeset
entries but do not want to destroy rangeset itself.

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v13:
 - Added Jan's A-b
Changes in v12:
 - s/rangeset_empty/rangeset_purge/
Changes in v11:
 - Now the function only empties rangeset, without removing it from
   domain's list

Changes in v10:
 - New in v10. The function is used in "vpci/header: handle p2m range sets per BAR"
---
 xen/common/rangeset.c      | 16 ++++++++++++----
 xen/include/xen/rangeset.h |  3 ++-
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/xen/common/rangeset.c b/xen/common/rangeset.c
index 0ccd53caac52..b75590f90744 100644
--- a/xen/common/rangeset.c
+++ b/xen/common/rangeset.c
@@ -448,11 +448,20 @@ struct rangeset *rangeset_new(
     return r;
 }
 
-void rangeset_destroy(
-    struct rangeset *r)
+void rangeset_purge(struct rangeset *r)
 {
     struct range *x;
 
+    if ( r == NULL )
+        return;
+
+    while ( (x = first_range(r)) != NULL )
+        destroy_range(r, x);
+}
+
+void rangeset_destroy(
+    struct rangeset *r)
+{
     if ( r == NULL )
         return;
 
@@ -463,8 +472,7 @@ void rangeset_destroy(
         spin_unlock(&r->domain->rangesets_lock);
     }
 
-    while ( (x = first_range(r)) != NULL )
-        destroy_range(r, x);
+    rangeset_purge(r);
 
     xfree(r);
 }
diff --git a/xen/include/xen/rangeset.h b/xen/include/xen/rangeset.h
index 87bd956962b5..96c918082501 100644
--- a/xen/include/xen/rangeset.h
+++ b/xen/include/xen/rangeset.h
@@ -56,7 +56,7 @@ void rangeset_limit(
 bool __must_check rangeset_is_empty(
     const struct rangeset *r);
 
-/* Add/claim/remove/query a numeric range. */
+/* Add/claim/remove/query/purge a numeric range. */
 int __must_check rangeset_add_range(
     struct rangeset *r, unsigned long s, unsigned long e);
 int __must_check rangeset_claim_range(struct rangeset *r, unsigned long size,
@@ -70,6 +70,7 @@ bool __must_check rangeset_overlaps_range(
 int rangeset_report_ranges(
     struct rangeset *r, unsigned long s, unsigned long e,
     int (*cb)(unsigned long s, unsigned long e, void *data), void *ctxt);
+void rangeset_purge(struct rangeset *r);
 
 /*
  * Note that the consume function can return an error value apart from
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:37:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:37:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675238.1050589 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Dw-0001d0-Pl; Fri, 02 Feb 2024 21:37:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675238.1050589; Fri, 02 Feb 2024 21:37:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Dw-0001bw-MH; Fri, 02 Feb 2024 21:37:24 +0000
Received: by outflank-mailman (input) for mailman id 675238;
 Fri, 02 Feb 2024 21:37:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1Ce-0003tJ-Uy
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:36:05 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20631.outbound.protection.outlook.com
 [2a01:111:f400:7eaa::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1041dd65-c213-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:36:03 +0100 (CET)
Received: from BY3PR05CA0005.namprd05.prod.outlook.com (2603:10b6:a03:254::10)
 by DS7PR12MB5838.namprd12.prod.outlook.com (2603:10b6:8:79::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.35; Fri, 2 Feb
 2024 21:36:00 +0000
Received: from SJ1PEPF00001CE0.namprd05.prod.outlook.com
 (2603:10b6:a03:254:cafe::2a) by BY3PR05CA0005.outlook.office365.com
 (2603:10b6:a03:254::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.22 via Frontend
 Transport; Fri, 2 Feb 2024 21:36:00 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ1PEPF00001CE0.mail.protection.outlook.com (10.167.242.8) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7202.16 via Frontend Transport; Fri, 2 Feb 2024 21:35:59 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:35:59 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:35:59 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:35:57 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1041dd65-c213-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oNzwinYcJJSme46a4s+2b94tFowL2cHyt5VCYSpQoKQcfmj2TJwRIEklsjENqH//1ghEVxT6jJfsq1APAh2fawq8oTsH1R8HdwTTzNtxejL67M5U7iOY0xf1RsxjzT1m/J2ckE+pDIv4hZsbzSPvWGxv/kewE/cb0i+ccj1ZoA+UYNT1VlyZptSt+VS+qWe6BAB+6+VK7AMjxFsemZigZLvzR/wtCeQMbeUP5THvVjpzIb9LqLjYDMKy12Mowoy43fxxpAxDyxOVkm6WQsKGi/rQuQiLOYGflvPZQdm8ngAefmmc7fhqCA5zVY7lEIKWrYz1HEo0LvHY4QwjJ/ttvg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=fhjSNHqgk5XVfapJjt8Oh6Wc0r8aqkJq3Ia//oNN7uA=;
 b=VRHaH297bBHkpV6L1/K1meALhKCHNchBqDpl8wgRg7u8e6sDNUWat4FdQH+TnCTAI92+SK/DnvmDLQ08HWYy5SpNParYvRuwfyeod22YpDuj4uCL+2USLhUhgODAAPb8xrA1dEk5+rSDLz1DJBaL/+aCDCy1tJRj5TLQo38gzDg5AQFFN1jb+pScRHpq7xRYjn3SEryst7CKIq0+MDMZFXzNN4Xjf8TkM8Jqkhidnu4vxztF7NlAx2otXIfO5fQY93QSWoG+llFH80dgwP78evDwt0Y6pPDOmZyAV+G6wygtlYZz2bQh9girWPOBrCjhL7ADXFIehe0BwallzkE8hw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fhjSNHqgk5XVfapJjt8Oh6Wc0r8aqkJq3Ia//oNN7uA=;
 b=TfKRBo/y1ROvr6RlmamvsDobCH6N1C4xZz72Im24s363UzPya9f/+J79xlXa4n/CPfdCAQpIEpbXWVg+gN9PozKuk/7RmakwSR/lp7GazetnovbfYcr0kIYI/KvZvb/h/p3WpdAkn3pHnMI85KQd6iWHCSvSrofwiSDVDuAxs1U=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>, Stewart Hildebrand <stewart.hildebrand@amd.com>
Subject: [PATCH v13 10/14] vpci/header: emulate PCI_COMMAND register for guests
Date: Fri, 2 Feb 2024 16:33:14 -0500
Message-ID: <20240202213321.1920347-11-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CE0:EE_|DS7PR12MB5838:EE_
X-MS-Office365-Filtering-Correlation-Id: d3b26f68-4d2e-4929-265e-08dc2436f2d4
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ijowIiVS/xfTikK2Q9f1ShhLcqaELW+irl/FfF5KZpv3e7E7iYqkEeaDmn3VUiakugk2W0PM7soeVn3L75KOJquyC6YdEMJsjXCX10PrUYbKkPrb4abuk2H+oW9W8IXgTNI5kicAk2pfggpGlca/4rqz++Fed7BoXi0wkbY3nBHTmrbW+JZLV72nxGob81/G5eI1VnbE0LiB7ZJOdSMMoanzrtUWO93zz4h0MC81g5y4BTwY65B9MFHfZK9sh82QjcOkLTC7L7ZOc1TM6xeZdOJR3R1xt9YF6RR8OwxldG+T62EQ8y1oqZvMkjpEgyhwJwrOAZDzGakMzoKWzRk3nd/TYrbOzE7nrzAFVuddyUHm5ciPJ97dyOIdV7GRiKghESiHJnyiPtiiRYbuCtOXmWLp1BI1oLXTV8GbiMjA5np21mkcf+oCIH1IkJ6oP0X7rGlsW/fOEX92KR1GLzarjsLibV5Df9JicH0s2vzg6jpfKYxPN2GvvpprVu+2dCmiecfYYSEP+vfaJGIZZbutea5IejOOwtoqv3I2Yr4Mvqe0EJCJLYGQWbqVeZpKEkexhHQGfOgL4HLy4ijzLv5X5p0QfIt+WyTn6fCfmork79RWEiNzDK/V7R4VhSqrRsxTj/0LuuO3QT2rELBO1CncEdjAuh0EFLAGYnh0ei7Ye+4ZnhLcYA7MZQDEvPm1H9TNET9afsX8uoTWH+0tRiMS2xitUA24d16tSDyObQkQPKnQYKyvO+TMvBy2cZwIFETJbVnIooc3277cbFUbgVaQtA==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(136003)(346002)(376002)(396003)(230273577357003)(230922051799003)(186009)(82310400011)(64100799003)(451199024)(1800799012)(40470700004)(36840700001)(46966006)(44832011)(82740400003)(54906003)(70586007)(8676002)(86362001)(2906002)(8936002)(30864003)(5660300002)(70206006)(4326008)(7416002)(36860700001)(478600001)(6916009)(36756003)(316002)(81166007)(966005)(6666004)(47076005)(356005)(83380400001)(26005)(336012)(41300700001)(40480700001)(40460700003)(1076003)(2616005)(426003)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:35:59.9341
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d3b26f68-4d2e-4929-265e-08dc2436f2d4
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CE0.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5838

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Xen and/or Dom0 may have put values in PCI_COMMAND which they expect
to remain unaltered. PCI_COMMAND_SERR bit is a good example: while the
guest's (domU) view of this will want to be zero (for now), the host
having set it to 1 should be preserved, or else we'd effectively be
giving the domU control of the bit. Thus, PCI_COMMAND register needs
proper emulation in order to honor host's settings.

According to "PCI LOCAL BUS SPECIFICATION, REV. 3.0", section "6.2.2
Device Control" the reset state of the command register is typically 0,
so when assigning a PCI device use 0 as the initial state for the
guest's (domU) view of the command register.

Here is the full list of command register bits with notes about
PCI/PCIe specification, and how Xen handles the bit. QEMU's behavior is
also documented here since that is our current reference implementation
for PCI passthrough.

PCI_COMMAND_IO (bit 0)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU sets this bit to 1 in
    hardware if an I/O BAR is exposed to the guest.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP for now since we
    don't yet support I/O BARs for domUs.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_MEMORY (bit 1)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU sets this bit to 1 in
    hardware if a Memory BAR is exposed to the guest.
  Xen domU/dom0: We handle writes to this bit by mapping/unmapping BAR
    regions.
  Xen domU: For devices assigned to DomUs, memory decoding will be
    disabled at the time of initialization.

PCI_COMMAND_MASTER (bit 2)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_SPECIAL (bit 3)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_INVALIDATE (bit 4)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_VGA_PALETTE (bit 5)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: Pass through writes to hardware.
  Xen domU/dom0: Pass through writes to hardware.

PCI_COMMAND_PARITY (bit 6)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_WAIT (bit 7)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: hardwire to 0
  QEMU: res_mask
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_SERR (bit 8)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_FAST_BACK (bit 9)
  PCIe 6.1: RO, hardwire to 0
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware.
  Xen domU: (rsvdp_mask) We treat this bit as RsvdP.
  Xen dom0: We allow dom0 to control this bit freely.

PCI_COMMAND_INTX_DISABLE (bit 10)
  PCIe 6.1: RW
  PCI LB 3.0: RW
  QEMU: (emu_mask) QEMU provides an emulated view of this bit. Guest
    writes do not propagate to hardware. QEMU checks if INTx was mapped
    for a device. If it is not, then guest can't control
    PCI_COMMAND_INTX_DISABLE bit.
  Xen domU: We prohibit a guest from enabling INTx if MSI(X) is enabled.
  Xen dom0: We allow dom0 to control this bit freely.

Bits 11-15
  PCIe 6.1: RsvdP
  PCI LB 3.0: Reserved
  QEMU: res_mask
  Xen domU/dom0: rsvdp_mask

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
RFC: There is an unaddressed question for Roger: should we update the
     guest view of the PCI_COMMAND_INTX_DISABLE bit in
     msi.c/msix.c:control_write()? See prior discussion at [1].

[1] https://lore.kernel.org/xen-devel/86b25777-788c-4b9a-8166-a6f8174bedc9@suse.com/

In v13:
- Update right away (don't defer) PCI_COMMAND_MEMORY bit in guest_cmd
  variable in cmd_write()
- Make comment single line in xen/drivers/vpci/msi.c:control_write()
- Rearrange memory decoding disabling snippet in init_header()

In v12:
- Rework patch using vpci_add_register_mask()
- Add bitmask #define in pci_regs.h according to PCIe 6.1 spec, except
  don't add the RO bits because they were RW in PCI LB 3.0 spec.
- Move and expand TODO comment about properly emulating bits
- Update guest_cmd in msi.c/msix.c:control_write()
- Simplify cmd_write(), thanks to rsvdp_mask
- Update commit description

In v11:
- Fix copy-paste mistake: vpci->msi should be vpci->msix
- Handle PCI_COMMAND_IO
- Fix condition for disabling INTx in the MSI-X code
- Show domU changes to only allowed bits
- Show PCI_COMMAND_MEMORY write only after P2M was altered
- Update comments in the code
In v10:
- Added cf_check attribute to guest_cmd_read
- Removed warning about non-zero cmd
- Updated comment MSI code regarding disabling INTX
- Used ternary operator in vpci_add_register() call
- Disable memory decoding for DomUs in init_bars()
In v9:
- Reworked guest_cmd_read
- Added handling for more bits
Since v6:
- fold guest's logic into cmd_write
- implement cmd_read, so we can report emulated INTx state to guests
- introduce header->guest_cmd to hold the emulated state of the
  PCI_COMMAND register for guests
Since v5:
- add additional check for MSI-X enabled while altering INTX bit
- make sure INTx disabled while guests enable MSI/MSI-X
Since v3:
- gate more code on CONFIG_HAS_MSI
- removed logic for the case when MSI/MSI-X not enabled
---
 xen/drivers/vpci/header.c  | 57 ++++++++++++++++++++++++++++++++++----
 xen/drivers/vpci/msi.c     |  7 +++++
 xen/drivers/vpci/msix.c    |  7 +++++
 xen/include/xen/pci_regs.h |  1 +
 xen/include/xen/vpci.h     |  3 ++
 5 files changed, 69 insertions(+), 6 deletions(-)

diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 47648c395132..5d13ea975cc2 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -524,9 +524,21 @@ static void cf_check cmd_write(
 {
     struct vpci_header *header = data;
 
+    if ( !is_hardware_domain(pdev->domain) )
+    {
+        const struct vpci *vpci = pdev->vpci;
+
+        if ( (vpci->msi && vpci->msi->enabled) ||
+             (vpci->msix && vpci->msix->enabled) )
+            cmd |= PCI_COMMAND_INTX_DISABLE;
+
+        header->guest_cmd = cmd;
+    }
+
     /*
      * Let Dom0 play with all the bits directly except for the memory
-     * decoding one.
+     * decoding one. Bits that are not allowed for DomU are already
+     * handled above and by the rsvdp_mask.
      */
     if ( header->bars_mapped != !!(cmd & PCI_COMMAND_MEMORY) )
         /*
@@ -540,6 +552,14 @@ static void cf_check cmd_write(
         pci_conf_write16(pdev->sbdf, reg, cmd);
 }
 
+static uint32_t cf_check guest_cmd_read(
+    const struct pci_dev *pdev, unsigned int reg, void *data)
+{
+    const struct vpci_header *header = data;
+
+    return header->guest_cmd;
+}
+
 static void cf_check bar_write(
     const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data)
 {
@@ -754,9 +774,23 @@ static int cf_check init_header(struct pci_dev *pdev)
         return -EOPNOTSUPP;
     }
 
-    /* Setup a handler for the command register. */
-    rc = vpci_add_register(pdev->vpci, vpci_hw_read16, cmd_write, PCI_COMMAND,
-                           2, header);
+    /*
+     * Setup a handler for the command register.
+     *
+     * TODO: If support for emulated bits is added, re-visit how to handle
+     * PCI_COMMAND_PARITY, PCI_COMMAND_SERR, and PCI_COMMAND_FAST_BACK.
+     */
+    rc = vpci_add_register_mask(pdev->vpci,
+                                is_hwdom ? vpci_hw_read16 : guest_cmd_read,
+                                cmd_write, PCI_COMMAND, 2, header, 0, 0,
+                                PCI_COMMAND_RSVDP_MASK |
+                                    (is_hwdom ? 0
+                                              : PCI_COMMAND_IO |
+                                                PCI_COMMAND_PARITY |
+                                                PCI_COMMAND_WAIT |
+                                                PCI_COMMAND_SERR |
+                                                PCI_COMMAND_FAST_BACK),
+                                0);
     if ( rc )
         return rc;
 
@@ -836,9 +870,20 @@ static int cf_check init_header(struct pci_dev *pdev)
     if ( pdev->ignore_bars )
         return 0;
 
-    /* Disable memory decoding before sizing. */
     cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
-    if ( cmd & PCI_COMMAND_MEMORY )
+
+    /*
+     * Clear PCI_COMMAND_MEMORY and PCI_COMMAND_IO for DomUs, so they will
+     * always start with memory decoding disabled and to ensure that we will not
+     * call modify_bars() at the end of this function.
+     */
+    if ( !is_hwdom )
+        cmd &= ~(PCI_COMMAND_MEMORY | PCI_COMMAND_IO);
+
+    header->guest_cmd = cmd;
+
+    /* Disable memory decoding before sizing. */
+    if ( !is_hwdom || (cmd & PCI_COMMAND_MEMORY) )
         pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd & ~PCI_COMMAND_MEMORY);
 
     for ( i = 0; i < num_bars; i++ )
diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c
index dc71938e23f5..1e747ad9bba8 100644
--- a/xen/drivers/vpci/msi.c
+++ b/xen/drivers/vpci/msi.c
@@ -70,6 +70,13 @@ static void cf_check control_write(
 
         if ( vpci_msi_arch_enable(msi, pdev, vectors) )
             return;
+
+        /* Make sure domU doesn't enable INTx while enabling MSI. */
+        if ( !is_hardware_domain(pdev->domain) )
+        {
+            pci_intx(pdev, false);
+            pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
+        }
     }
     else
         vpci_msi_arch_disable(msi, pdev);
diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
index b6abab47efdd..d4f756ce287a 100644
--- a/xen/drivers/vpci/msix.c
+++ b/xen/drivers/vpci/msix.c
@@ -135,6 +135,13 @@ static void cf_check control_write(
         }
     }
 
+    /* Make sure domU doesn't enable INTx while enabling MSI-X. */
+    if ( new_enabled && !msix->enabled && !is_hardware_domain(pdev->domain) )
+    {
+        pci_intx(pdev, false);
+        pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
+    }
+
     msix->masked = new_masked;
     msix->enabled = new_enabled;
 
diff --git a/xen/include/xen/pci_regs.h b/xen/include/xen/pci_regs.h
index 9909b27425a5..b2f2e43e864d 100644
--- a/xen/include/xen/pci_regs.h
+++ b/xen/include/xen/pci_regs.h
@@ -48,6 +48,7 @@
 #define  PCI_COMMAND_SERR	0x100	/* Enable SERR */
 #define  PCI_COMMAND_FAST_BACK	0x200	/* Enable back-to-back writes */
 #define  PCI_COMMAND_INTX_DISABLE 0x400 /* INTx Emulation Disable */
+#define  PCI_COMMAND_RSVDP_MASK	0xf800
 
 #define PCI_STATUS		0x06	/* 16 bits */
 #define  PCI_STATUS_IMM_READY	0x01	/* Immediate Readiness */
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index e89c571890b2..77320a667e55 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -107,6 +107,9 @@ struct vpci {
         } bars[PCI_HEADER_NORMAL_NR_BARS + 1];
         /* At most 6 BARS + 1 expansion ROM BAR. */
 
+        /* Guest (domU only) view of the PCI_COMMAND register. */
+        uint16_t guest_cmd;
+
         /*
          * Store whether the ROM enable bit is set (doesn't imply ROM BAR
          * is mapped into guest p2m) if there's a ROM BAR on the device.
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:37:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:37:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675239.1050597 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Dx-0001ix-Bi; Fri, 02 Feb 2024 21:37:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675239.1050597; Fri, 02 Feb 2024 21:37:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Dx-0001hC-2M; Fri, 02 Feb 2024 21:37:25 +0000
Received: by outflank-mailman (input) for mailman id 675239;
 Fri, 02 Feb 2024 21:37:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1Cv-0003tJ-8L
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:36:21 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1a1ed3fc-c213-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:36:20 +0100 (CET)
Received: from BYAPR21CA0029.namprd21.prod.outlook.com (2603:10b6:a03:114::39)
 by CH2PR12MB4924.namprd12.prod.outlook.com (2603:10b6:610:6b::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.11; Fri, 2 Feb
 2024 21:36:13 +0000
Received: from SJ1PEPF00001CE1.namprd05.prod.outlook.com
 (2603:10b6:a03:114:cafe::97) by BYAPR21CA0029.outlook.office365.com
 (2603:10b6:a03:114::39) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.6 via Frontend
 Transport; Fri, 2 Feb 2024 21:36:12 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ1PEPF00001CE1.mail.protection.outlook.com (10.167.242.9) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7202.16 via Frontend Transport; Fri, 2 Feb 2024 21:36:12 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:36:11 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Fri, 2 Feb
 2024 13:36:11 -0800
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:36:10 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1a1ed3fc-c213-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IwOOgn2rYg5D/WvmsCUT8qoBhJQrAtgLYklDazN56h+7gozf/U1RXmKI9yOBn5UqYAU5VAakFNeEQklGiMySOeEe27427haZH6cPdzyj84+ZLcraC9IHApv2uHUGUK1ZXXX7JDojZjEvw4vLosNgA14rPUSf6V1RdYfeoJsiCyRaw7nwdlYqja/AFsgGDH4UO+InEeAA88jg2ZOVTGUjouSDuJLhCxV5cRnygN0uEK9rTSghmt3p2jYxHjUktqYMUb49MC0O1cJa4W5ilxL3rVIU4IsRYZcnWLzx7nOTyXpO35yd4WLZHxxLBGd+JjWmMjQy+mVQ4Br+K05jpmAcEQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=G/OcqFAAiB6EoPEXzP0cP8TrQpQFguNrAiXOKI4bieI=;
 b=ct4wHl7v6EYAOeYVrA88XoHXd22z7VpeZbEU571h3N/MsRtBIqrmYgUlgfWjRYtxGiz7ribJn86J6rcBlGtzanWOguvVJUKUQqOTJ4R6lRDQJDzjYTE4VBqcK70t3GMzIwjGWmGWOeIkvRO8YMgddvMfXcCLeXob2y4S6WGn50TkRIIqdVFhibehf3xRpEKEAIdOR2KxEMb8sq12em5i9kVF5wMaq9dzt7CZC9YAR2nJMxREHG1ZajJ7HMlDtPt6XrHiq0DjNY9sQF/bo/ITs10sTJ2dxLwvpU14sL8H2zzknNv6DqQWPHOE9LIEn+UnbzMdY3Xq98exWewfHpnwiw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=G/OcqFAAiB6EoPEXzP0cP8TrQpQFguNrAiXOKI4bieI=;
 b=Ude+pJ4U0Qn5DhphKlbgGg2PKATdoQiMFuxK4mtdj3cw/UCnS4hzOJrEsW2X8pU0aIdOhHwVeQ6AeHBPp4eRglz0Da00i5p4233HxvJERi7ErKnQ3WVuFQ9iR1SFlnF2mGSCOzH1WTdiIcFAy4mzYn9ggVfEgri2WXD6deZ9dnU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Volodymyr
 Babchuk" <volodymyr_babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>
Subject: [PATCH v13 11/14] vpci: add initial support for virtual PCI bus topology
Date: Fri, 2 Feb 2024 16:33:15 -0500
Message-ID: <20240202213321.1920347-12-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CE1:EE_|CH2PR12MB4924:EE_
X-MS-Office365-Filtering-Correlation-Id: 9ce17121-21b0-493f-f0fa-08dc2436fa45
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1LPvKEVU8yLAbZTsAkiEDS9+ieooTes7LRyUKxTsETRdw5IcipVtDLAG/y7h1vtY5UP9KnITXORg1yGxABVX57lU6ctZ/6Ce3VzzTVF3p0BpqtA/vxGIHUjUegat5JZkFLeJgHB218YTmS0UP8npI6PXXm0lGuaCvGVIhlfGD9xXFkNQ35VzqWlRi/BSdQvR/1MfH/cWS90HajWBXS/9GKKrU8Njj3JlQxOj9w7OszD0ky3jQDea7wSJaUT9kw5l7x2pD/uEg+c+I+YrcmqF2KK2Vs2ABINHfov+mjpuyFmPIERDW7ZPHr2hqsalJoFWPmnB8A5jkNoUUo3TvvcMpczlvEd6ptOZKN4kw3KcZqe3UNKcomkdgJTfcTf3fHWKhhe6MqoO0njB4khaI1hGAof2thp8HLpVBwpaFS0SyB9jl/sbJsa2KZzJdIC6cm9vGr0axmC8S6/gyJyYq0SQA/Y0f6XaqEmfJYybM3GQuUU8ZVyJvJjGKk4355IMCA96b/5gSAAmLmyQVpZEm5zm31hLISj/AQfv7qu+ZghCM1TM3Ne3p6FBhWC2qWKcOnrLHll6d7hUcTpPJdZRIhdiG11ES6oHYvT/sF07ThtBmvs01zGPUcsvxed2hed1KhY6s1ZONhuwvxtvKQj/sCKDlr/Jwt9nfqL64gdNWWB9D01s46/Sqj3xyeki+jMJr3cBpN9iUvPta/OBdaN58EicKoUh+wk69gFCiaQGXNu+lDQxp/jmTf7Ne07CYd1XTqF09/ZngU1NwlVOL7/4ldtJHA==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(136003)(346002)(396003)(39860400002)(230922051799003)(186009)(82310400011)(451199024)(64100799003)(1800799012)(36840700001)(40470700004)(46966006)(83380400001)(26005)(1076003)(47076005)(2616005)(36860700001)(426003)(8936002)(70206006)(7416002)(8676002)(44832011)(81166007)(4326008)(82740400003)(2906002)(316002)(54906003)(5660300002)(6916009)(478600001)(70586007)(36756003)(41300700001)(336012)(356005)(86362001)(40460700003)(40480700001)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:36:12.4083
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ce17121-21b0-493f-f0fa-08dc2436fa45
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CE1.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4924

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Assign SBDF to the PCI devices being passed through with bus 0.
The resulting topology is where PCIe devices reside on the bus 0 of the
root complex itself (embedded endpoints).
This implementation is limited to 32 devices which are allowed on
a single PCI bus.

Please note, that at the moment only function 0 of a multifunction
device can be passed through.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
In v13:
- s/depends on/select/ in Kconfig
- check pdev->sbdf.fn instead of two booleans in add_virtual_device()
- comment #endifs in sched.h
- clarify comment about limits in vpci.h with seg/bus limit
In v11:
- Fixed code formatting
- Removed bogus write_unlock() call
- Fixed type for new_dev_number
In v10:
- Removed ASSERT(pcidevs_locked())
- Removed redundant code (local sbdf variable, clearing sbdf during
device removal, etc)
- Added __maybe_unused attribute to "out:" label
- Introduced HAS_VPCI_GUEST_SUPPORT Kconfig option, as this is the
  first patch where it is used (previously was in "vpci: add hooks for
  PCI device assign/de-assign")
In v9:
- Lock in add_virtual_device() replaced with ASSERT (thanks, Stewart)
In v8:
- Added write lock in add_virtual_device
Since v6:
- re-work wrt new locking scheme
- OT: add ASSERT(pcidevs_write_locked()); to add_virtual_device()
Since v5:
- s/vpci_add_virtual_device/add_virtual_device and make it static
- call add_virtual_device from vpci_assign_device and do not use
  REGISTER_VPCI_INIT machinery
- add pcidevs_locked ASSERT
- use DECLARE_BITMAP for vpci_dev_assigned_map
Since v4:
- moved and re-worked guest sbdf initializers
- s/set_bit/__set_bit
- s/clear_bit/__clear_bit
- minor comment fix s/Virtual/Guest/
- added VPCI_MAX_VIRT_DEV constant (PCI_SLOT(~0) + 1) which will be used
  later for counting the number of MMIO handlers required for a guest
  (Julien)
Since v3:
 - make use of VPCI_INIT
 - moved all new code to vpci.c which belongs to it
 - changed open-coded 31 to PCI_SLOT(~0)
 - added comments and code to reject multifunction devices with
   functions other than 0
 - updated comment about vpci_dev_next and made it unsigned int
 - implement roll back in case of error while assigning/deassigning devices
 - s/dom%pd/%pd
Since v2:
 - remove casts that are (a) malformed and (b) unnecessary
 - add new line for better readability
 - remove CONFIG_HAS_VPCI_GUEST_SUPPORT ifdef's as the relevant vPCI
    functions are now completely gated with this config
 - gate common code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
 xen/drivers/Kconfig     |  4 +++
 xen/drivers/vpci/vpci.c | 57 +++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/sched.h | 10 +++++++-
 xen/include/xen/vpci.h  | 12 +++++++++
 4 files changed, 82 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/Kconfig b/xen/drivers/Kconfig
index db94393f47a6..20050e9bb8b3 100644
--- a/xen/drivers/Kconfig
+++ b/xen/drivers/Kconfig
@@ -15,4 +15,8 @@ source "drivers/video/Kconfig"
 config HAS_VPCI
 	bool
 
+config HAS_VPCI_GUEST_SUPPORT
+	bool
+	select HAS_VPCI
+
 endmenu
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 260b72875ee1..3cd142068f4e 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -40,6 +40,49 @@ extern vpci_register_init_t *const __start_vpci_array[];
 extern vpci_register_init_t *const __end_vpci_array[];
 #define NUM_VPCI_INIT (__end_vpci_array - __start_vpci_array)
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+static int add_virtual_device(struct pci_dev *pdev)
+{
+    struct domain *d = pdev->domain;
+    unsigned int new_dev_number;
+
+    if ( is_hardware_domain(d) )
+        return 0;
+
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
+    /*
+     * Each PCI bus supports 32 devices/slots at max or up to 256 when
+     * there are multi-function ones which are not yet supported.
+     */
+    if ( pdev->sbdf.fn )
+    {
+        gdprintk(XENLOG_ERR, "%pp: only function 0 passthrough supported\n",
+                 &pdev->sbdf);
+        return -EOPNOTSUPP;
+    }
+    new_dev_number = find_first_zero_bit(d->vpci_dev_assigned_map,
+                                         VPCI_MAX_VIRT_DEV);
+    if ( new_dev_number == VPCI_MAX_VIRT_DEV )
+        return -ENOSPC;
+
+    __set_bit(new_dev_number, &d->vpci_dev_assigned_map);
+
+    /*
+     * Both segment and bus number are 0:
+     *  - we emulate a single host bridge for the guest, e.g. segment 0
+     *  - with bus 0 the virtual devices are seen as embedded
+     *    endpoints behind the root complex
+     *
+     * TODO: add support for multi-function devices.
+     */
+    pdev->vpci->guest_sbdf = PCI_SBDF(0, 0, new_dev_number, 0);
+
+    return 0;
+}
+
+#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
+
 void vpci_deassign_device(struct pci_dev *pdev)
 {
     unsigned int i;
@@ -49,6 +92,12 @@ void vpci_deassign_device(struct pci_dev *pdev)
     if ( !has_vpci(pdev->domain) || !pdev->vpci )
         return;
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    if ( pdev->vpci->guest_sbdf.sbdf != ~0 )
+        __clear_bit(pdev->vpci->guest_sbdf.dev,
+                    &pdev->domain->vpci_dev_assigned_map);
+#endif
+
     spin_lock(&pdev->vpci->lock);
     while ( !list_empty(&pdev->vpci->handlers) )
     {
@@ -105,6 +154,13 @@ int vpci_assign_device(struct pci_dev *pdev)
     INIT_LIST_HEAD(&pdev->vpci->handlers);
     spin_lock_init(&pdev->vpci->lock);
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    pdev->vpci->guest_sbdf.sbdf = ~0;
+    rc = add_virtual_device(pdev);
+    if ( rc )
+        goto out;
+#endif
+
     for ( i = 0; i < NUM_VPCI_INIT; i++ )
     {
         rc = __start_vpci_array[i](pdev);
@@ -112,6 +168,7 @@ int vpci_assign_device(struct pci_dev *pdev)
             break;
     }
 
+ out: __maybe_unused;
     if ( rc )
         vpci_deassign_device(pdev);
 
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index c3adec1aca3c..daa3c1743047 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -484,7 +484,15 @@ struct domain
      * 2. pdev->vpci->lock
      */
     rwlock_t pci_lock;
-#endif
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    /*
+     * The bitmap which shows which device numbers are already used by the
+     * virtual PCI bus topology and is used to assign a unique SBDF to the
+     * next passed through virtual PCI device.
+     */
+    DECLARE_BITMAP(vpci_dev_assigned_map, VPCI_MAX_VIRT_DEV);
+#endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
+#endif /* CONFIG_HAS_PCI */
 
 #ifdef CONFIG_HAS_PASSTHROUGH
     struct domain_iommu iommu;
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index 77320a667e55..a69c640d065a 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -21,6 +21,14 @@ typedef int vpci_register_init_t(struct pci_dev *dev);
 
 #define VPCI_ECAM_BDF(addr)     (((addr) & 0x0ffff000) >> 12)
 
+/*
+ * Maximum number of devices supported by the virtual bus topology:
+ * each PCI bus supports 32 devices/slots at max or up to 256 when
+ * there are multi-function ones which are not yet supported.
+ * This limit implies only segment 0, bus 0 is supported.
+ */
+#define VPCI_MAX_VIRT_DEV       (PCI_SLOT(~0) + 1)
+
 #define REGISTER_VPCI_INIT(x, p)                \
   static vpci_register_init_t *const x##_entry  \
                __used_section(".data.vpci." p) = x
@@ -175,6 +183,10 @@ struct vpci {
             struct vpci_arch_msix_entry arch;
         } entries[];
     } *msix;
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+    /* Guest SBDF of the device. */
+    pci_sbdf_t guest_sbdf;
+#endif
 #endif
 };
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 21:37:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 21:37:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675241.1050609 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Dy-0002FO-Sw; Fri, 02 Feb 2024 21:37:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675241.1050609; Fri, 02 Feb 2024 21:37:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1Dy-0002Dw-LN; Fri, 02 Feb 2024 21:37:26 +0000
Received: by outflank-mailman (input) for mailman id 675241;
 Fri, 02 Feb 2024 21:37:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Iq6B=JL=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rW1D8-0003tJ-FF
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 21:36:34 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 22084f02-c213-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 22:36:33 +0100 (CET)
Received: from DM6PR02CA0161.namprd02.prod.outlook.com (2603:10b6:5:332::28)
 by SN7PR12MB7370.namprd12.prod.outlook.com (2603:10b6:806:299::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.24; Fri, 2 Feb
 2024 21:36:29 +0000
Received: from CY4PEPF0000E9DB.namprd05.prod.outlook.com
 (2603:10b6:5:332:cafe::78) by DM6PR02CA0161.outlook.office365.com
 (2603:10b6:5:332::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.26 via Frontend
 Transport; Fri, 2 Feb 2024 21:36:29 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9DB.mail.protection.outlook.com (10.167.241.81) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 2 Feb 2024 21:36:29 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:36:28 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 2 Feb
 2024 15:36:27 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 2 Feb 2024 15:36:21 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22084f02-c213-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fGmoGsXwDOzFWKt37IsnRD24EDWaF5PhKPWGQNi5TAC0gAtFWizs3+A95NEYtFB7zSLBcHvHtqolVckoeMcYsLu1lyMLld7qHvWiHh/sjVHATYBlXrUmrC3t/VVUsx59Lr7cy0xggMpfFA5mnjFNZ8Puobo+Nhuw4TSDqwxKRS9GGNeVAnEJs8IFKyii9awFkpw7jLjVNldn6tDFiE6/UdR8870iH/UWckP1uIaCy3bJTV4qRv1Ea1phYWK3lK/oYHeDDdo8RuLZZaYB+nRZWPvRzoKL/+Si4FmCQb7RRjg0hYTFvdjzvbmFbIe/N+suBLFwE/nfnTqdR0RpZUSC1A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=eJMXVdcAYBu7iSzlaGDF0CzLtmxwO7mBpjOdIW/Bo5w=;
 b=jsI752toqx8WVIw2uuGDY23GacvmXh7F4Qpl8cSJvbmnqa7R8wVctj/6cIgif5Oua0d1mbIZgaQSUfKGQUFMwiy5BWHDknIq5OhQK3/whIdK7ZcD1JB/KY7losRRXI+rdQ3lEpaZDlIGXLdPUr8s2ZCgQNuok76y/W3GU5klbDLZ1f4B7dlJQq2touJz3gXRlPtQSTshyiftE8ptClgRkgEYBxWm7cvuasf7bGoUghhI5N2aDZ41eBN13MmwhrXP9RgEZhRkdSRdNn5m1jEWI+Y2zMmGX3bzjYx4aFFbDfUaUCgoFldLQqIgBtjs1xjysK4Yh4oMs2pyk6dBK6G7ng==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eJMXVdcAYBu7iSzlaGDF0CzLtmxwO7mBpjOdIW/Bo5w=;
 b=2dsZIp+++bUGi/IEc67cA+wst93hTfHHqqH7QJ9WzA1tC4bJXErDdxL0IPQOSt3mudt53It1uuioGx+3lGLoUE44phOPOHS2txzJ9rKGRmrq6n4Xhlo5FiSI/o5Kq+s8tvQ95u80k+6VpiUVsE33kPKaOW4fcYsihl2B+8av52c=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand
 Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, "Volodymyr
 Babchuk" <volodymyr_babchuk@epam.com>, Stewart Hildebrand
	<stewart.hildebrand@amd.com>
Subject: [PATCH v13 12/14] xen/arm: translate virtual PCI bus topology for guests
Date: Fri, 2 Feb 2024 16:33:16 -0500
Message-ID: <20240202213321.1920347-13-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9DB:EE_|SN7PR12MB7370:EE_
X-MS-Office365-Filtering-Correlation-Id: 758d98da-fd84-460d-aa11-08dc24370430
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xuRGVq7rbp7nSYM1rtri/KW0RhyGyFeLnalZfu8gWv9MmNx8Q0tNb0f8qOdXo7r4Hfh99sv9KJxbC4DOZtC/++cI+3WdbT+Jh57aYU1os3HGfC72W+dfnQMVi0vbs9e9NFA92/rEo9PuxZMmaLoj6jCDdEHhv1ojG2r47ol76wdV9dP5DRo1OYzNaQ7GH/J/KLJHL9svzNHRXIjL8JVAmMkiUfVh8MWpu7A+GGH5xeg/yxA4rKJZVpn1iAwafbXSr30IwwNvsFaBsvSWpwySn0n4OIQJUIMIM7vHhFcXoyAi1u6xPaUU3CGiS65ms387GEn/IGA9cVXtCirpwWvIfpbRRSG1z8AtEkzlOPohonpVRzAF/4uGlMsIzXIb3Q7KfjwUR3zqv7EwM+2zbA2stlC10s9YmmNZdcgdqQ7xhAa22bpgQtk41SyrI1iANJRpyDAcKdC6J0SXdPY0ycICj/cYExM8zfOutH9V1f6R+FtIBftoBPv1O/L/Ik4x5hMErnz8RGJyWiOR7VghyVaJ+/hwYt9c6d/ErWdEujJhEY/+zCU6ji8xDybynVAVq/IEPi6GBCDQF4dhWf2UxnJthgXkGs1jA+lPCa3XVOXDxMpuC43uuSXcIBE5ulQJwdOwrYopIz4RBAr8VWVOI8pCO32F3lexpl+iNG/RbcTWsEYvYeYEv5HqojNpOEp6xtJaQ33Hp4gVmFO0Z7jljActP+eohTlxh7KPA+IDH6f2EcmVPtrcbgljK4+F/sXpdbjDNdFpTquGOSGzn5melops2w==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(396003)(136003)(346002)(376002)(39860400002)(230922051799003)(64100799003)(1800799012)(82310400011)(451199024)(186009)(40470700004)(46966006)(36840700001)(44832011)(8936002)(2906002)(5660300002)(86362001)(36756003)(356005)(82740400003)(81166007)(478600001)(36860700001)(1076003)(83380400001)(426003)(336012)(26005)(41300700001)(6666004)(2616005)(316002)(8676002)(47076005)(70206006)(6916009)(4326008)(54906003)(70586007)(40460700003)(40480700001)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2024 21:36:29.0773
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 758d98da-fd84-460d-aa11-08dc24370430
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9DB.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7370

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

There are three  originators for the PCI configuration space access:
1. The domain that owns physical host bridge: MMIO handlers are
there so we can update vPCI register handlers with the values
written by the hardware domain, e.g. physical view of the registers
vs guest's view on the configuration space.
2. Guest access to the passed through PCI devices: we need to properly
map virtual bus topology to the physical one, e.g. pass the configuration
space access to the corresponding physical devices.
3. Emulated host PCI bridge access. It doesn't exist in the physical
topology, e.g. it can't be mapped to some physical host bridge.
So, all access to the host bridge itself needs to be trapped and
emulated.

Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
In v11:
- Fixed format issues
- Added ASSERT_UNREACHABLE() to the dummy implementation of
vpci_translate_virtual_device()
- Moved variable in vpci_sbdf_from_gpa(), now it is easier to follow
the logic in the function
Since v9:
- Commend about required lock replaced with ASSERT()
- Style fixes
- call to vpci_translate_virtual_device folded into vpci_sbdf_from_gpa
Since v8:
- locks moved out of vpci_translate_virtual_device()
Since v6:
- add pcidevs locking to vpci_translate_virtual_device
- update wrt to the new locking scheme
Since v5:
- add vpci_translate_virtual_device for #ifndef CONFIG_HAS_VPCI_GUEST_SUPPORT
  case to simplify ifdefery
- add ASSERT(!is_hardware_domain(d)); to vpci_translate_virtual_device
- reset output register on failed virtual SBDF translation
Since v4:
- indentation fixes
- constify struct domain
- updated commit message
- updates to the new locking scheme (pdev->vpci_lock)
Since v3:
- revisit locking
- move code to vpci.c
Since v2:
 - pass struct domain instead of struct vcpu
 - constify arguments where possible
 - gate relevant code with CONFIG_HAS_VPCI_GUEST_SUPPORT
New in v2
---
 xen/arch/arm/vpci.c     | 47 +++++++++++++++++++++++++++++++----------
 xen/drivers/vpci/vpci.c | 24 +++++++++++++++++++++
 xen/include/xen/vpci.h  | 12 +++++++++++
 3 files changed, 72 insertions(+), 11 deletions(-)

diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index 3bc4bb55082a..7a6a0017d132 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -7,31 +7,51 @@
 
 #include <asm/mmio.h>
 
-static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge,
-                                     paddr_t gpa)
+static bool vpci_sbdf_from_gpa(struct domain *d,
+                               const struct pci_host_bridge *bridge,
+                               paddr_t gpa, pci_sbdf_t *sbdf)
 {
-    pci_sbdf_t sbdf;
+    bool translated = true;
+
+    ASSERT(sbdf);
 
     if ( bridge )
     {
-        sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
-        sbdf.seg = bridge->segment;
-        sbdf.bus += bridge->cfg->busn_start;
+        sbdf->sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
+        sbdf->seg = bridge->segment;
+        sbdf->bus += bridge->cfg->busn_start;
     }
     else
-        sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
+    {
+        /*
+         * For the passed through devices we need to map their virtual SBDF
+         * to the physical PCI device being passed through.
+         */
+        sbdf->sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
+        read_lock(&d->pci_lock);
+        translated = vpci_translate_virtual_device(d, sbdf);
+        read_unlock(&d->pci_lock);
+    }
 
-    return sbdf;
+    return translated;
 }
 
 static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
                           register_t *r, void *p)
 {
     struct pci_host_bridge *bridge = p;
-    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    pci_sbdf_t sbdf;
     /* data is needed to prevent a pointer cast on 32bit */
     unsigned long data;
 
+    ASSERT(!bridge == !is_hardware_domain(v->domain));
+
+    if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
+    {
+        *r = ~0UL;
+        return 1;
+    }
+
     if ( vpci_ecam_read(sbdf, ECAM_REG_OFFSET(info->gpa),
                         1U << info->dabt.size, &data) )
     {
@@ -39,7 +59,7 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
         return 1;
     }
 
-    *r = ~0ul;
+    *r = ~0UL;
 
     return 0;
 }
@@ -48,7 +68,12 @@ static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
                            register_t r, void *p)
 {
     struct pci_host_bridge *bridge = p;
-    pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+    pci_sbdf_t sbdf;
+
+    ASSERT(!bridge == !is_hardware_domain(v->domain));
+
+    if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, &sbdf) )
+        return 1;
 
     return vpci_ecam_write(sbdf, ECAM_REG_OFFSET(info->gpa),
                            1U << info->dabt.size, r);
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 3cd142068f4e..6b732bb2358e 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -81,6 +81,30 @@ static int add_virtual_device(struct pci_dev *pdev)
     return 0;
 }
 
+/*
+ * Find the physical device which is mapped to the virtual device
+ * and translate virtual SBDF to the physical one.
+ */
+bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf)
+{
+    const struct pci_dev *pdev;
+
+    ASSERT(!is_hardware_domain(d));
+    ASSERT(rw_is_locked(&d->pci_lock));
+
+    for_each_pdev ( d, pdev )
+    {
+        if ( pdev->vpci && (pdev->vpci->guest_sbdf.sbdf == sbdf->sbdf) )
+        {
+            /* Replace guest SBDF with the physical one. */
+            *sbdf = pdev->sbdf;
+            return true;
+        }
+    }
+
+    return false;
+}
+
 #endif /* CONFIG_HAS_VPCI_GUEST_SUPPORT */
 
 void vpci_deassign_device(struct pci_dev *pdev)
diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h
index a69c640d065a..a23320562ed7 100644
--- a/xen/include/xen/vpci.h
+++ b/xen/include/xen/vpci.h
@@ -303,6 +303,18 @@ static inline bool __must_check vpci_process_pending(struct vcpu *v)
 }
 #endif
 
+#ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT
+bool vpci_translate_virtual_device(const struct domain *d, pci_sbdf_t *sbdf);
+#else
+static inline bool vpci_translate_virtual_device(const struct domain *d,
+                                                 pci_sbdf_t *sbdf)
+{
+    ASSERT_UNREACHABLE();
+
+    return false;
+}
+#endif
+
 #endif
 
 /*
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 02 22:09:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 02 Feb 2024 22:09:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675281.1050620 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1ij-0002am-Kc; Fri, 02 Feb 2024 22:09:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675281.1050620; Fri, 02 Feb 2024 22:09:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW1ij-0002af-H8; Fri, 02 Feb 2024 22:09:13 +0000
Received: by outflank-mailman (input) for mailman id 675281;
 Fri, 02 Feb 2024 22:09:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=L4Ps=JL=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rW1ii-0002aZ-MH
 for xen-devel@lists.xenproject.org; Fri, 02 Feb 2024 22:09:12 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b071b5b3-c217-11ee-8a45-1f161083a0e0;
 Fri, 02 Feb 2024 23:09:10 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 08F69628BB;
 Fri,  2 Feb 2024 22:09:09 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87B62C433C7;
 Fri,  2 Feb 2024 22:09:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b071b5b3-c217-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1706911748;
	bh=mMWoEjvSdbn6OkvFwGWf8jH0jd9qlvcg2vq/sKfxrps=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=cK+ichH1EzOON/enkSBGgl1rMHjts9cADg3+/rPBt+ouCWbZC2u+N+q+GfTNmeAEX
	 ayg/oEtxihUMARuQaJDDymoYjveDy32QZKLISgrwjci0QgdDeieRCSVTPbdvSd770P
	 6L3dFKQqxkwB1VUnP5KlcZQ9tJFXWoIlmBFYqZxTHJbfB8cuA49Z82tu9nhWwOL4sW
	 cd6IsFjj2S1Jvll417Ebov34WHFey4taKQcxrDqs9658M1oQICCpQzjwsBjgpFgrpo
	 f9XWgU/CgJjNEl2qvfEXerp1/CwKfwgKf8f82C2oXGTV0X+rjwIoQ8xQk+fNi88Bif
	 xSRaDYj63sTsw==
Date: Fri, 2 Feb 2024 14:09:06 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: John Ernberg <john.ernberg@actia.se>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Peng Fan <peng.fan@nxp.com>, Jonas Blixt <jonas.blixt@actia.se>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
In-Reply-To: <012b5f83-2f9b-4477-965e-07b8506c0052@xen.org>
Message-ID: <alpine.DEB.2.22.394.2402021401310.292110@ubuntu-linux-20-04-desktop>
References: <20240131114952.305805-1-john.ernberg@actia.se> <20240131114952.305805-2-john.ernberg@actia.se> <a2f726f5-df4c-4d15-90af-7d59c0f1f513@xen.org> <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se> <167f0c7a-e037-446c-82f8-2584e35a7af1@xen.org>
 <a265ea2d-9b5f-4726-9395-448b1b669839@actia.se> <012b5f83-2f9b-4477-965e-07b8506c0052@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1121442091-1706911363=:292110"
Content-ID: <alpine.DEB.2.22.394.2402021402550.292110@ubuntu-linux-20-04-desktop>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1121442091-1706911363=:292110
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.22.394.2402021402551.292110@ubuntu-linux-20-04-desktop>

On Fri, 2 Feb 2024, Julien Grall wrote:
> On 01/02/2024 16:17, John Ernberg wrote:
> > On 2/1/24 13:20, Julien Grall wrote:
> > > 
> > > 
> > > On 31/01/2024 15:32, John Ernberg wrote:
> > > > Hi Julien,
> > > 
> > > Hi John,
> > > 
> > > > On 1/31/24 13:22, Julien Grall wrote:
> > > > > Hi,
> > > > > 
> > > > > On 31/01/2024 11:50, John Ernberg wrote:
> > > > > > When using Linux for dom0 there are a bunch of drivers that need to
> > > > > > do
> > > > > > SMC
> > > > > > SIP calls into the PSCI provider to enable certain hardware bits
> > > > > > like the
> > > > > > watchdog.
> > > > > 
> > > > > Do you know which protocol this is under the hood. Is this SCMI?
> > > > 
> > > > I think I confused myself here when I wrote the commit log.
> > > > 
> > > > The EL3 code in our case is ATF, and it does not appear to be SCMI, nor
> > > > PSCI. The register usage of these SMC SIP calls are as follows:
> > > > a0 - service
> > > > a1 - function
> > > > a2-a7 - args
> > > > 
> > > > In ATF the handler is declared as a runtime service.
> > > > 
> > > > Would the appropriate commmit message here be something along the lines
> > > > of below?
> > > > """
> > > > When using Linux for dom0 there are a bunch of drivers that need to
> > > > do   SMC
> > > > SIP calls into the firmware to enable certain hardware bits like the
> > > > watchdog.
> > > > """
> > > 
> > > It reads better thanks.
> > > 
> > > [...]
> > > 
> > > > > But even if we restrict to dom0, have you checked that none of the
> > > > > SMCs
> > > > > use buffers?
> > > > I haven't found any such instances in the Linux kernel where a buffer is
> > > > used. Adding a call filtering like suggested below additions of such
> > > > functions can be discovered and adapted for if they would show up later.
> > > > > 
> > > > > Rather than providing a blanket forward, to me it sounds more like you
> > > > > want to provide an allowlist of the SMCs. This is more futureproof and
> > > > > avoid the risk to expose unsafe SMCs to any domain.
> > > > > 
> > > > > For an example, you can have a look at the EEMI mediator for Xilinx.
> > > > 
> > > > Ack. Do you prefer to see only on SMCCC service level or also on
> > > > function level? (a1 register, per description earlier)
> > > 
> > > I am not sure. It will depend on whether it is correct to expose *all*
> > > the functions within a service level and they have the same format.
> > > 
> > > If you can't guarantee that, then you will most likely need to allowlist
> > > at the function level.
> > > 
> > > Also, do you have a spec in hand that would help to understand which
> > > service/function is implemented via those SMCs?
> > 
> > I don't have the spec unfortunately, but I will add a filter on both
> > service and function for V2 and we'll take it from there.
> 
> @Peng, do you have any specification you could share? How stable is the
> interface?

Just to add some context to make the reason for the question clearer, if
we have a specification we could check the patch for correctness.
Without it, it is difficult to know if it is doing the right thing.

The other aspect is about expectation of forward and backward
compatibility. Can we guarantee that the next version of Xen and the one
after it will still work against this interface? If not, can we check
for the version of the interface before continuing? If not, can we at
least document that the interface is only known-to-work with specific
firmware versions?

This is basically just to provide the right expectations to users and
ideally to prevent a future version of Xen to break on boot silently
without information.

If we don't have a spec and we don't know if the interface is stable, I
think we should try to detect the version of the interface and print a
warning in Xen if it not a known version.
--8323329-1121442091-1706911363=:292110--


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 02:24:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 02:24:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675293.1050629 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW5h9-0007BU-Ju; Sat, 03 Feb 2024 02:23:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675293.1050629; Sat, 03 Feb 2024 02:23:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW5h9-0007BN-Gu; Sat, 03 Feb 2024 02:23:51 +0000
Received: by outflank-mailman (input) for mailman id 675293;
 Sat, 03 Feb 2024 02:23:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rW5h8-0007BD-93; Sat, 03 Feb 2024 02:23:50 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rW5h8-0002DM-2Z; Sat, 03 Feb 2024 02:23:50 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rW5h7-0004dA-Jg; Sat, 03 Feb 2024 02:23:49 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rW5h7-00037n-JC; Sat, 03 Feb 2024 02:23:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pJtBpto34lVKtJFoXuN/xHASybWeSFcZcNh1R9ezd+E=; b=H2UxSHwT2KmSubnywRzA3HBFg9
	1jlStjCsSkfwPLCqOxrPDySte1loetC1EbK11G9IhXEYbGYyeNEO72u3TPYakujym3Ifok3vppoMi
	svKDFnoTVZY1GG9octWTcUb1vX5CO44uShb45IrOMTwXrQjrcyZDkjYRk80+uS5RKF/A=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184563-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-6.1 test] 184563: regressions - FAIL
X-Osstest-Failures:
    linux-6.1:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:regression
    linux-6.1:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=e5c3b988b827c76f52d0f62343e863b9133a0cd2
X-Osstest-Versions-That:
    linux=8fd7f44624538675abadc73f5a44e95016964d22
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 03 Feb 2024 02:23:49 +0000

flight 184563 linux-6.1 real [real]
flight 184569 linux-6.1 real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184563/
http://logs.test-lab.xenproject.org/osstest/logs/184569/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. vs. 184549

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184549
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184549
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184549
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184549
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184549
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184549
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184549
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184549
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 linux                e5c3b988b827c76f52d0f62343e863b9133a0cd2
baseline version:
 linux                8fd7f44624538675abadc73f5a44e95016964d22

Last test of basis   184549  2024-02-01 11:13:03 Z    1 days
Testing same since   184563  2024-02-02 05:26:08 Z    0 days    1 attempts

------------------------------------------------------------
486 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 17439 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 05:54:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 05:54:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675305.1050649 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW8yh-0007oR-T3; Sat, 03 Feb 2024 05:54:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675305.1050649; Sat, 03 Feb 2024 05:54:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW8yh-0007oK-Pz; Sat, 03 Feb 2024 05:54:11 +0000
Received: by outflank-mailman (input) for mailman id 675305;
 Sat, 03 Feb 2024 05:54:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rW8yf-0007o9-OD; Sat, 03 Feb 2024 05:54:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rW8yf-0007nZ-KJ; Sat, 03 Feb 2024 05:54:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rW8yf-0004nf-6J; Sat, 03 Feb 2024 05:54:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rW8yf-000865-5r; Sat, 03 Feb 2024 05:54:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ctCMyVgYaMwZg5uo1cPmzMQ/M+0Xe4TBZcwHDBugSa4=; b=EALrAuPxr7QiCkrVGXWPfHsX/z
	uD07aXae56u868Hb7hoPLIF4lVwUFgNVmfkzw0bydJVtB8sbtXZ8t/hsFzcgjdH+3xqkp4EFySf9y
	ueIs7DRgFUBDOsuncXGkGUIlq5mY+wy66uzjzd2OIihqb24utnAlxalCACYOCWTcth0w=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184564-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.17-testing test] 184564: tolerable trouble: fail/pass/starved - PUSHED
X-Osstest-Failures:
    xen-4.17-testing:test-amd64-i386-xl-shadow:guest-start/debian.repeat:fail:heisenbug
    xen-4.17-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:hosts-allocate:starved:nonblocking
X-Osstest-Versions-This:
    xen=091466ba55d1e2e75738f751818ace2e3ed08ccf
X-Osstest-Versions-That:
    xen=6b1864afc14d484cdbc9754ce3172ac3dc189846
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 03 Feb 2024 05:54:09 +0000

flight 184564 xen-4.17-testing real [real]
flight 184574 xen-4.17-testing real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184564/
http://logs.test-lab.xenproject.org/osstest/logs/184574/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-shadow 22 guest-start/debian.repeat fail pass in 184574-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184531
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184531
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184531
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184531
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184531
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184531
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184531
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184531
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184531
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184531
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184531
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184531
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm       3 hosts-allocate               starved  n/a

version targeted for testing:
 xen                  091466ba55d1e2e75738f751818ace2e3ed08ccf
baseline version:
 xen                  6b1864afc14d484cdbc9754ce3172ac3dc189846

Last test of basis   184531  2024-01-30 14:07:13 Z    3 days
Testing same since   184564  2024-02-02 07:07:04 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Michal Orzel <michal.orzel@amd.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      starved 
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    fail    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   6b1864afc1..091466ba55  091466ba55d1e2e75738f751818ace2e3ed08ccf -> stable-4.17


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 06:52:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 06:52:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675320.1050690 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW9si-0007F5-Fr; Sat, 03 Feb 2024 06:52:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675320.1050690; Sat, 03 Feb 2024 06:52:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rW9si-0007Ey-CU; Sat, 03 Feb 2024 06:52:04 +0000
Received: by outflank-mailman (input) for mailman id 675320;
 Sat, 03 Feb 2024 06:52:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=do4w=JM=proton.me=oxjo@srs-se1.protection.inumbo.net>)
 id 1rW9sg-0007EZ-AY
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 06:52:03 +0000
Received: from mail-40138.protonmail.ch (mail-40138.protonmail.ch
 [185.70.40.138]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bab46c43-c260-11ee-8a45-1f161083a0e0;
 Sat, 03 Feb 2024 07:52:00 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bab46c43-c260-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me;
	s=pnaxooauwbbdxks7bzwswiseja.protonmail; t=1706943119; x=1707202319;
	bh=EobaSJc23ddaVAh9q16q+fP0pU/dKKyhVwROBXbwYbQ=;
	h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References:
	 Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:
	 Message-ID:BIMI-Selector;
	b=G4sTtgcyEFdrcTTVQEAQHoNzOP/SJHdhZ9e+U1XC/IrCA+yXUz3Vr+GXLy1baMb2H
	 s6pGN8l3eWUIU3EmubBltDw6ZXV//uP3YPFTt4KTbkmI0hyK7Wrh5j88D06CSCjrs7
	 5T8Ua7zPmKWIFyuNsBEAh19+UOfVzmNI89lBjXpeEFVQyMO4AqVzSZIQtFhwJ0f41q
	 smIkWZvOiMz6+srMeR9mihuQcFpOFdwR5lySedoCJXJs9omjFK5NorzLKL6q2vi/LD
	 X+sHdApDAIkD+5MpkiPir67g5bAemc+m6KjLTWULlr9la4FTi5waMM9wD9dfWq9TFh
	 NNLlF+sDCBYYw==
Date: Sat, 03 Feb 2024 06:51:40 +0000
To: Roger Pau Monne <roger.pau@citrix.com>
From: oxjo@proton.me
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 1/4] amd-vi: fix IVMD memory type checks
Message-ID: <3aNB4uE4wIvluDR2_B44k-NdgLmDwQJ5HDZJ5VioTNW0erk7v8LOvQSIhhy2d82r8aj7FtVFznD2fLVwn2fHmNN9uMUa-UVNUPeVV1kondI=@proton.me>
In-Reply-To: <20240201170159.66330-2-roger.pau@citrix.com>
References: <20240201170159.66330-1-roger.pau@citrix.com> <20240201170159.66330-2-roger.pau@citrix.com>
Feedback-ID: 99814252:user:proton
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Thursday, February 1st, 2024 at 18:01, Roger Pau Monne <roger.pau@citrix=
.com> wrote:

> The current code that parses the IVMD blocks is relaxed with regard to th=
e
> restriction that such unity regions should always fall into memory ranges
> marked as reserved in the memory map.
>
> However the type checks for the IVMD addresses are inverted, and as a res=
ult
> IVMD ranges falling into RAM areas are accepted. Note that having such ra=
nges
> in the first place is a firmware bug, as IVMD should always fall into res=
erved
> ranges.
>
> Fixes: ed6c77ebf0c1 ('AMD/IOMMU: check / convert IVMD ranges for being / =
to be reserved')
> Signed-off-by: Roger Pau Monn=C3=A9 roger.pau@citrix.com
>
> ---
> Cc: oxjo@proton.me
> ---
> xen/drivers/passthrough/amd/iommu_acpi.c | 11 ++++++++---
> 1 file changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/xen/drivers/passthrough/amd/iommu_acpi.c b/xen/drivers/passt=
hrough/amd/iommu_acpi.c
> index 2e3b83014beb..ca70f4f3ae2c 100644
> --- a/xen/drivers/passthrough/amd/iommu_acpi.c
> +++ b/xen/drivers/passthrough/amd/iommu_acpi.c
> @@ -426,9 +426,14 @@ static int __init parse_ivmd_block(const struct acpi=
_ivrs_memory ivmd_block)
> return -EIO;
> }
>
> - / Types which won't be handed out are considered good enough. /
> - if ( !(type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
> - RAM_TYPE_UNUSABLE)) )
> + /
> + * Types which aren't RAM are considered good enough.
> + * Note that a page being partially RESERVED, ACPI or UNUSABLE will
> + * force Xen into assuming the whole page as having that type in
> + * practice.
> + */
> + if ( type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
> + RAM_TYPE_UNUSABLE) )
> continue;
>
> AMD_IOMMU_ERROR("IVMD: page at %lx can't be converted\n", addr);

I tested the patch and it resolves the issue.
It eliminates the boot IVMD error message.
AMD-Vi is enabled and pci passthrough works.


Tested-by: oxjo <oxjo@proton.me>


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 10:57:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 10:57:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675344.1050700 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWDiA-0003GH-38; Sat, 03 Feb 2024 10:57:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675344.1050700; Sat, 03 Feb 2024 10:57:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWDi9-0003GA-VT; Sat, 03 Feb 2024 10:57:25 +0000
Received: by outflank-mailman (input) for mailman id 675344;
 Sat, 03 Feb 2024 10:57:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JBn0=JM=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rWDi8-0003G3-4d
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 10:57:24 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 012caa73-c283-11ee-98f5-efadbce2ee36;
 Sat, 03 Feb 2024 11:57:21 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-55a5e7fa471so3899788a12.1
 for <xen-devel@lists.xenproject.org>; Sat, 03 Feb 2024 02:57:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 012caa73-c283-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1706957841; x=1707562641; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=fGsty/kH/g/f05zone2AZ5tXmGe1ChZeNzwKEwhoKGA=;
        b=ijxnYlsbXpC8l7sYfkt39Rd32xUmUf0c72ssCYPKKJ8S5Zj3Axa338qOc8/q3fjeFl
         dVBBd8nhvQEcyDC5BXBQykF/tLyFHiR8bKsmxEMA7q77MMK61cNijU8NDG5CObZyp1ep
         Zj9PKJNRoeqV9SpcKsD4QKR8pwUWFxrILlp1+OaJxxyzojYmkLAZNOBkuyEMmsulYORw
         jUnqfHsB76qBtsTSfPif9t4DMCZUJa34irqSrc6xOOw00/DLD3zqUSIUg7WHpwttkGuQ
         zT2vjEsRVJ6jHdurPLEou9OmthafIO0H3TXm+JRPj2D0uhoP4YABpT4idL5qJRs4oxEN
         HoLQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706957841; x=1707562641;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=fGsty/kH/g/f05zone2AZ5tXmGe1ChZeNzwKEwhoKGA=;
        b=SoJgAxVbZUEFqoOSiWsdFEvNS0VjQpLYROjTHY3acK6+BnMN7SaZHZEHGk5Y+xDDi2
         J6Gc41EWsGoVxiargFFMB9KVK1rgorxhP3j5jzYLJnZi7a9yeT9qld8slgKw5wXurEui
         0GphCAsThvP0jwPpH6AhLiT5Q6+fDF9EgDIaYpgUL4JLIKMUz0aDrHAzxC/ybPkV+kbv
         LUV/AZNxwYwlmE3Am1WHOpgT1uRlYHR+C0RwN3vYvctRfXgtCWD2Df7sQWfincVUQ13G
         trXtg8voaQgTJF5AJLVEzgcwgKZxwuAGwpvQmliuuO3S0897+GfDb/0VUM1MapPlSsOS
         5lfA==
X-Gm-Message-State: AOJu0YyPwbvEUoEsyEM+jQSN69xhgFJHvO9i+qNLpuYFZw7ye8jVPGBO
	Q7mY9S6zFv2YuxmIaxCZz4UR3jsDdbZitSfWOnWPGq4zcIsNqYMrb+NuZ0/hGuNGj6otzXnPfmV
	rzPJhQlFpEc+9i4IJx1MLgENQtX1vAGM+ao5LQw==
X-Google-Smtp-Source: AGHT+IGxOlJQqTT4EJoO7G6v0ZohXm/9L+cN7xvZpOugu9RQQGwd20bWHNtr7buh1aFaHtrjk5Tn4IfFKz0BRfFelNQ=
X-Received: by 2002:aa7:c89a:0:b0:55f:fad9:a91c with SMTP id
 p26-20020aa7c89a000000b0055ffad9a91cmr1453613eds.6.1706957840528; Sat, 03 Feb
 2024 02:57:20 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-2-carlo.nonato@minervasys.tech> <42eabd87-b245-44e3-9945-40b1a5c5eaf5@suse.com>
In-Reply-To: <42eabd87-b245-44e3-9945-40b1a5c5eaf5@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Sat, 3 Feb 2024 11:57:09 +0100
Message-ID: <CAG+AhRVjiqjae+Lwdk71OkF=X93m+CNLFYpK65h+O1kozOgtew@mail.gmail.com>
Subject: Re: [PATCH v6 01/15] xen/common: add cache coloring common code
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Wed, Jan 31, 2024 at 4:57=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 29.01.2024 18:17, Carlo Nonato wrote:
> > Last Level Cache (LLC) coloring allows to partition the cache in smalle=
r
> > chunks called cache colors. Since not all architectures can actually
> > implement it, add a HAS_LLC_COLORING Kconfig and put other options unde=
r
> > xen/arch.
> >
> > LLC colors are a property of the domain, so the domain struct has to be
> > extended.
> >
> > Based on original work from: Luca Miccio <lucmiccio@gmail.com>
> >
> > Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
> > Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
> > ---
> > v6:
> > - moved almost all code in common
> > - moved documentation in this patch
> > - reintroduced range for CONFIG_NR_LLC_COLORS
> > - reintroduced some stub functions to reduce the number of checks on
> >   llc_coloring_enabled
> > - moved domain_llc_coloring_free() in same patch where allocation happe=
ns
> > - turned "d->llc_colors" to pointer-to-const
> > - llc_coloring_init() now returns void and panics if errors are found
> > v5:
> > - used - instead of _ for filenames
> > - removed domain_create_llc_colored()
> > - removed stub functions
> > - coloring domain fields are now #ifdef protected
> > v4:
> > - Kconfig options moved to xen/arch
> > - removed range for CONFIG_NR_LLC_COLORS
> > - added "llc_coloring_enabled" global to later implement the boot-time
> >   switch
> > - added domain_create_llc_colored() to be able to pass colors
> > - added is_domain_llc_colored() macro
> > ---
> >  docs/misc/cache-coloring.rst      | 87 +++++++++++++++++++++++++++++++
> >  docs/misc/xen-command-line.pandoc | 27 ++++++++++
> >  xen/arch/Kconfig                  | 17 ++++++
> >  xen/common/Kconfig                |  3 ++
> >  xen/common/Makefile               |  1 +
> >  xen/common/keyhandler.c           |  3 ++
> >  xen/common/llc-coloring.c         | 87 +++++++++++++++++++++++++++++++
> >  xen/include/xen/llc-coloring.h    | 38 ++++++++++++++
> >  xen/include/xen/sched.h           |  5 ++
> >  9 files changed, 268 insertions(+)
> >  create mode 100644 docs/misc/cache-coloring.rst
> >  create mode 100644 xen/common/llc-coloring.c
> >  create mode 100644 xen/include/xen/llc-coloring.h
> >
> > diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rs=
t
> > new file mode 100644
> > index 0000000000..9fe01e99e1
> > --- /dev/null
> > +++ b/docs/misc/cache-coloring.rst
> > @@ -0,0 +1,87 @@
> > +Xen cache coloring user guide
> > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D
> > +
> > +The cache coloring support in Xen allows to reserve Last Level Cache (=
LLC)
> > +partitions for Dom0, DomUs and Xen itself. Currently only ARM64 is sup=
ported.
> > +
> > +To compile LLC coloring support set ``CONFIG_LLC_COLORING=3Dy``.
> > +
> > +If needed, change the maximum number of colors with
> > +``CONFIG_NR_LLC_COLORS=3D<n>``.
> > +
> > +Compile Xen and the toolstack and then configure it via
> > +`Command line parameters`_.
> > +
> > +Background
> > +**********
> > +
> > +Cache hierarchy of a modern multi-core CPU typically has first levels =
dedicated
> > +to each core (hence using multiple cache units), while the last level =
is shared
> > +among all of them. Such configuration implies that memory operations o=
n one
> > +core (e.g. running a DomU) are able to generate interference on anothe=
r core
> > +(e.g .hosting another DomU). Cache coloring allows eliminating this
> > +mutual interference, and thus guaranteeing higher and more predictable
> > +performances for memory accesses.
> > +The key concept underlying cache coloring is a fragmentation of the me=
mory
> > +space into a set of sub-spaces called colors that are mapped to disjoi=
nt cache
> > +partitions. Technically, the whole memory space is first divided into =
a number
> > +of subsequent regions. Then each region is in turn divided into a numb=
er of
> > +subsequent sub-colors. The generic i-th color is then obtained by all =
the
> > +i-th sub-colors in each region.
> > +
> > +::
> > +
> > +                            Region j            Region j+1
> > +                .....................   ............
> > +                .                     . .
> > +                .                       .
> > +            _ _ _______________ _ _____________________ _ _
> > +                |     |     |     |     |     |     |
> > +                | c_0 | c_1 |     | c_n | c_0 | c_1 |
> > +           _ _ _|_____|_____|_ _ _|_____|_____|_____|_ _ _
> > +                    :                       :
> > +                    :                       :...         ... .
> > +                    :                            color 0
> > +                    :...........................         ... .
> > +                                                :
> > +          . . ..................................:
> > +
> > +There are two pragmatic lesson to be learnt.
> > +
> > +1. If one wants to avoid cache interference between two domains, diffe=
rent
> > +   colors needs to be used for their memory.
> > +
> > +2. Color assignment must privilege contiguity in the partitioning. E.g=
.,
> > +   assigning colors (0,1) to domain I  and (2,3) to domain  J is bette=
r than
> > +   assigning colors (0,2) to I and (1,3) to J.
>
> I can't connect this 2nd point with any of what was said above.

If colors are contiguous then a greater spatial locality is achievable. You
mean we should better explain this?

> > +How to compute the number of colors
> > +***********************************
> > +
> > +To compute the number of available colors for a specific platform, the=
 size of
> > +an LLC way and the page size used by Xen must be known. The first para=
meter can
> > +be found in the processor manual or can be also computed dividing the =
total
> > +cache size by the number of its ways. The second parameter is the mini=
mum
> > +amount of memory that can be mapped by the hypervisor,
>
> I find "amount of memory that can be mapped" quite confusing here. Don't =
you
> really mean the granularity at which memory can be mapped?

Yes that's what I wanted to describe. I'll change it.

> > thus dividing the way
> > +size by the page size, the number of total cache partitions is found. =
So for
> > +example, an Arm Cortex-A53 with a 16-ways associative 1 MiB LLC, can i=
solate up
> > +to 16 colors when pages are 4 KiB in size.
>
> I guess it's a matter of what one's use to, but to me talking of "way siz=
e"
> and how the calculation is described is, well, unusual. What I would star=
t
> from is the smallest entity, i.e. a cache line. Then it would be relevant
> to describe how, after removing the low so many bits to cover for cache l=
ine
> size, the remaining address bits are used to map to a particular set. It
> looks to me as if you're assuming that this mapping is linear, using the
> next so many bits from the address. Afaik this isn't true on various mode=
rn
> CPUs; instead hash functions are used. Without knowing at least certain
> properties of such a hash function, I'm afraid your mapping from address =
to
> color isn't necessarily guaranteeing the promised isolation. The guarante=
e
> may hold for processors you specifically target, but then I think in this
> description it would help if you would fully spell out any assumptions yo=
u
> make on how hardware maps addresses to elements of the cache.

You're right, we are assuming a linear mapping. We are going to review and
extend the documentation in order to fully specify when coloring can be
applied.

About the "way size" it's a way of summarizing all the parameters into one.
We could ask for different cache parameters as you said, but in the end wha=
t
we are interested in is how many partitions is the cache capable of isolate
and how big they are. The answer is, in theory, as many partitions as the
number of sets, each one as big as a cache line, bacause we can't have
isolation inside a set.
Then memory mapping comes into place and the minimum granularity at which
mapping can happen actually lowers the number of partitions.
To recap we can isolate:
    nr_sets * line_size / page_size
Then we simply named:
    way_size =3D nr_sets * line_size
Another way of computing it:
    way_size =3D cache_size / nr_ways

We are ok with having two parameters: cache_size and nr_ways which are even
easier and intuitive to find for a normal user.

> Or, alternatively, have the number of colors be specifiable directly.

This is to be avoided in my opinion since it's more difficult to compute an=
d
it requires more knowledge.

Thanks.

>
> > +Cache layout is probed automatically by looking at the CLIDR_EL1 arm r=
egister.
> > +This means that other system caches that aren't visible there, are ign=
ored.
> > +The possibility of manually setting the way size is left to the user t=
o overcome
> > +failing situations or for debugging/testing purposes. See
> > +`Command line parameters`_ for more information on that.
> > +
> > +Command line parameters
> > +***********************
> > +
> > +More specific documentation is available at `docs/misc/xen-command-lin=
e.pandoc`.
> > +
> > ++----------------------+-------------------------------+
> > +| **Parameter**        | **Description**               |
> > ++----------------------+-------------------------------+
> > +| ``llc-coloring``     | enable coloring at runtime    |
> > ++----------------------+-------------------------------+
> > +| ``llc-way-size``     | set the LLC way size          |
> > ++----------------------+-------------------------------+
>
> As a result of the above, I also find it confusing to specify "way size"
> as a command line option. Cache size, number of ways, and cache line size
> would seem more natural to me.
>
> I'll get to looking nat the actual code later.
>
> Jan


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 10:58:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 10:58:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675346.1050710 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWDiq-0003jG-Ar; Sat, 03 Feb 2024 10:58:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675346.1050710; Sat, 03 Feb 2024 10:58:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWDiq-0003j9-7N; Sat, 03 Feb 2024 10:58:08 +0000
Received: by outflank-mailman (input) for mailman id 675346;
 Sat, 03 Feb 2024 10:58:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JBn0=JM=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rWDip-0003h9-JZ
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 10:58:07 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1bb78267-c283-11ee-98f5-efadbce2ee36;
 Sat, 03 Feb 2024 11:58:05 +0100 (CET)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2d093cb1ef3so9009941fa.3
 for <xen-devel@lists.xenproject.org>; Sat, 03 Feb 2024 02:58:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1bb78267-c283-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1706957885; x=1707562685; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=KDdMAAimSpageKWRimuNsLBbYmOtOFz0XiaG4cD/bQk=;
        b=ExAKcnosm83uXg9SvUMILTSpWU7KIEI3GiNrLPEmqrTwaeQTJW+FDwss1YTP+PRqZs
         1IKw5usqKN4qBd4LO1mIwvCHbxex+sM6XF1Jn1l06p1cxVtWQf6gnX9/GGC03Dl3mpmV
         pZmSoayZcp7gzowX1bUWaSXcMy1n/6PvpGK+oK82HbZQe64BkI21IZQ/NgO7r84mH3Pr
         aHc5fnhuAjly6HSst3hA9num2LDb8oAHOJfTcuqCLMTinYSfnekEzglr26mgVRbsFCv6
         LCOMOj1XRIucHqeWzrlpIde+HfD9At5MKQXsOfI/bTNGf9viLl4AHkPzLmt3/+uIAir8
         Cogg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706957885; x=1707562685;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=KDdMAAimSpageKWRimuNsLBbYmOtOFz0XiaG4cD/bQk=;
        b=DjsrhTvCdcH+XmSdySGUEycQhfYX9On4NzmnVRx7x41hXVCFpzIfy6har+VpO+Yxbm
         BCs4IyjfDOfzf++mLU1rUdJey5DTaYszQcHkKTSTxmestcCsKK7Zyl0irWkV/h9TfZfF
         QCyhrzWe329wDjp4Ad7Iz5gMWmTvtmQHjy0vyNGT8MqZ/XRJ8OSvYnJ4BtWKTvj2ueE/
         iJ0MG2CWXfeG8m26L1wfx+MRNkhqRVNOpExZ1wvlEsdc5NnCmZJnHDXaQZAoG3nGbo/3
         A4mES4Uzu3C0aoY41IeDXDhpJ2VTHvmhcN3ZL7BUy6IupfLITNlChxT53UP7FIKHGtg3
         BePg==
X-Gm-Message-State: AOJu0YzpPp9n+1Se27Gpg9BBEGff4VcYDiRWUMrfdaJP6cSujCQrEQcl
	63B1VQBEfOZ9UL5n2ti98zcIXBPeHUCSmmf0HMfgK4xQO+GpAxHwMsnvwk2pNHu9LT55miw5ClF
	2GlscHqMGzU3dN0eZX83O7uNXYiwQWfrLlti54Q==
X-Google-Smtp-Source: AGHT+IGxlFsF5RYpqn/+fabSdxDEQBFSIQV+PRRLuuqMtWbDgRoYvgPrSFU9Zorx3+fRMvJVSiJ+gV5K6h1B5EVhClc=
X-Received: by 2002:a2e:2201:0:b0:2d0:8eff:431e with SMTP id
 i1-20020a2e2201000000b002d08eff431emr1732528lji.44.1706957885043; Sat, 03 Feb
 2024 02:58:05 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-2-carlo.nonato@minervasys.tech> <749799db-de6f-4ca8-bab1-67468f97929b@suse.com>
In-Reply-To: <749799db-de6f-4ca8-bab1-67468f97929b@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Sat, 3 Feb 2024 11:57:54 +0100
Message-ID: <CAG+AhRV8gPdbm5MK6-g_DO+=KNrakWs82xMCGDMGy05Rxngvtg@mail.gmail.com>
Subject: Re: [PATCH v6 01/15] xen/common: add cache coloring common code
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Thu, Feb 1, 2024 at 1:59=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 29.01.2024 18:17, Carlo Nonato wrote:
> > --- a/xen/arch/Kconfig
> > +++ b/xen/arch/Kconfig
> > @@ -31,3 +31,20 @@ config NR_NUMA_NODES
> >         associated with multiple-nodes management. It is the upper boun=
d of
> >         the number of NUMA nodes that the scheduler, memory allocation =
and
> >         other NUMA-aware components can handle.
> > +
> > +config LLC_COLORING
> > +     bool "Last Level Cache (LLC) coloring" if EXPERT
> > +     depends on HAS_LLC_COLORING
> > +
> > +config NR_LLC_COLORS
> > +     int "Maximum number of LLC colors"
> > +     range 2 1024
>
> What's the reasoning behind this upper bound? IOW - can something to this
> effect be said in the description, please?

The only reason is that this is the number of colors that fit in a 4 KiB pa=
ge.
I don't have any other good way of picking a number here. 1024 is already b=
ig
and probably nobody would use such a configuration. But 512 or 256 would be
equally arbitrary.

> > +     default 128
> > +     depends on LLC_COLORING
> > +     help
> > +       Controls the build-time size of various arrays associated with =
LLC
> > +       coloring. Refer to cache coloring documentation for how to comp=
ute the
> > +       number of colors supported by the platform. This is only an upp=
er
> > +       bound. The runtime value is autocomputed or manually set via cm=
dline.
> > +       The default value corresponds to an 8 MiB 16-ways LLC, which sh=
ould be
> > +       more than what needed in the general case.
>
> Aiui while not outright wrong, non-power-of-2 values are meaningless to
> specify. Perhaps that is worth mentioning (if not making this a value
> that's used as exponent of 2 in the first place)?

Yes, I prefer a better help message.

> As to the default and its description: As said for the documentation,
> doesn't what this corresponds to also depend on cache line size? Even
> if this was still Arm-specific rather than common code, I'd question
> whether now and forever Arm chips may only use one pre-determined cache
> line size.

I hope I answered in the previous mail why the line size (in the specific c=
ase
we are applying coloring to) can be ignored as a parameter in favor of cach=
e
size and number of ways.

> > --- /dev/null
> > +++ b/xen/common/llc-coloring.c
> > @@ -0,0 +1,87 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Last Level Cache (LLC) coloring common code
> > + *
> > + * Copyright (C) 2022 Xilinx Inc.
> > + */
> > +#include <xen/keyhandler.h>
> > +#include <xen/llc-coloring.h>
> > +#include <xen/param.h>
> > +
> > +bool __ro_after_init llc_coloring_enabled;
> > +boolean_param("llc-coloring", llc_coloring_enabled);
>
> The variable has no use right now afaics, so it's unclear whether (a) it
> is legitimately non-static and (b) placed in an appropriate section.

My bad here. The variable should be tested for in llc_coloring_init() and i=
n
domain_dump_llc_colors() (in domain_llc_coloring_free() as well, in later
patches). That change was lost in the rebase of the series.

Anyway per this patch, the global is only accessed from this file while it'=
s
going to be accessed from outside in later patches. In this case what shoul=
d
I do? Declare it static and then make it non-static afterwards?

> > +/* Size of an LLC way */
> > +static unsigned int __ro_after_init llc_way_size;
> > +size_param("llc-way-size", llc_way_size);
> > +/* Number of colors available in the LLC */
> > +static unsigned int __ro_after_init max_nr_colors =3D CONFIG_NR_LLC_CO=
LORS;
> > +
> > +static void print_colors(const unsigned int *colors, unsigned int num_=
colors)
> > +{
> > +    unsigned int i;
> > +
> > +    printk("{ ");
> > +    for ( i =3D 0; i < num_colors; i++ ) {
>
> Nit (style): Brace placement.
>
> > +        unsigned int start =3D colors[i], end =3D colors[i];
> > +
> > +        printk("%u", start);
> > +
> > +        for ( ;
> > +              i < num_colors - 1 && colors[i] + 1 =3D=3D colors[i + 1]=
;
>
> To reduce the number of array accesses, may I suggest to use "end + 1"
> here instead of "colors[i] + 1"? (The initializer of "end" could also
> be "start", but I guess the compiler will recognize this anyway.) This
> would then (imo) also better justify the desire for having "end" in
> the first place.
>
> > +              i++, end++ );
>
> Imo for clarity the semicolon want to live on its own line.
>
> > +static void dump_coloring_info(unsigned char key)
>
> This being common code now, I think it would be good practice to have
> cf_check here right away, even if for now (for whatever reason) the
> feature is meant to be limited to Arm. (Albeit see below for whether
> this is to remain that way.)
>
> > +void __init llc_coloring_init(void)
> > +{
> > +    if ( !llc_way_size && !(llc_way_size =3D get_llc_way_size()) )
> > +        panic("Probed LLC coloring way size is 0 and no custom value f=
ound\n");
> > +
> > +    /*
> > +     * The maximum number of colors must be a power of 2 in order to c=
orrectly
> > +     * map them to bits of an address, so also the LLC way size must b=
e so.
> > +     */
> > +    if ( llc_way_size & (llc_way_size - 1) )
> > +        panic("LLC coloring way size (%u) isn't a power of 2\n", llc_w=
ay_size);
> > +
> > +    max_nr_colors =3D llc_way_size >> PAGE_SHIFT;
>
> With this unconditionally initialized here, what's the purpose of the
> variable's initializer?

Previously I was using the global in parse_color_config() (later introduced=
),
but since now I'm not doing it anymore I can drop the initializer.

> > +    if ( max_nr_colors < 2 || max_nr_colors > CONFIG_NR_LLC_COLORS )
> > +        panic("Number of LLC colors (%u) not in range [2, %u]\n",
> > +              max_nr_colors, CONFIG_NR_LLC_COLORS);
>
> I'm not convinced of panic()ing here (including the earlier two
> instances). You could warn, taint, disable, and continue. If you want
> to stick to panic(), please justify doing so in the description.
>
> Plus, if you panic(), shouldn't that be limited to llc_coloring_enabled
> being true? Or - not visible here, due to the lack of a caller of the
> function - is that meant to be taken care of by the caller (to not call
> here when the flag is off)? I think it would be cleaner if the check
> lived here; quite possibly that would then further permit the flag
> variable to become static.

You're right. As I said here the check on llc_coloring_enabled is missing.
Obviously it's an error doing the initialization no matter what.

> > +    register_keyhandler('K', dump_coloring_info, "dump LLC coloring in=
fo", 1);
>
> I'm also not convinced of using a separate key for this little bit of
> information. How about attaching this to what 'm' or 'H' produce?

Ok. 'm' seems the right place.

> > +    arch_llc_coloring_init();
> > +}
> > +
> > +void domain_dump_llc_colors(const struct domain *d)
> > +{
> > +    printk("Domain %pd has %u LLC colors: ", d, d->num_llc_colors);
>
> %pd resolves to d<N> - why "Domain" as a prefix? And really - why the
> domain identifier in the first place? All surrounding information is
> already for this very domain.
>
> > +    print_colors(d->llc_colors, d->num_llc_colors);
>
> Imo this (or perhaps even the entire function) wants skipping when
> num_llc_colors is zero, which would in particular also cover the
> !llc_coloring_enabled case.

This shouldn't be possible. As I said this function should be a no-op when
!llc_coloring_enabled.

Thanks.

> > --- a/xen/include/xen/sched.h
> > +++ b/xen/include/xen/sched.h
> > @@ -626,6 +626,11 @@ struct domain
> >
> >      /* Holding CDF_* constant. Internal flags for domain creation. */
> >      unsigned int cdf;
> > +
> > +#ifdef CONFIG_LLC_COLORING
> > +    unsigned const int *llc_colors;
>
> const unsigned int * please.
>
> Jan


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 11:31:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 11:31:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675358.1050720 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWEFA-0000qc-VM; Sat, 03 Feb 2024 11:31:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675358.1050720; Sat, 03 Feb 2024 11:31:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWEFA-0000qV-SE; Sat, 03 Feb 2024 11:31:32 +0000
Received: by outflank-mailman (input) for mailman id 675358;
 Sat, 03 Feb 2024 11:31:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JBn0=JM=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rWEF9-0000qJ-CM
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 11:31:31 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c54990be-c287-11ee-8a45-1f161083a0e0;
 Sat, 03 Feb 2024 12:31:30 +0100 (CET)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2d073b54359so34282471fa.0
 for <xen-devel@lists.xenproject.org>; Sat, 03 Feb 2024 03:31:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c54990be-c287-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1706959887; x=1707564687; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iiyj18aRrfUjaARWFPuGN9TX5sHlhFENgj9Qgnt8pDY=;
        b=N7G0XJg2zW3lc/kSGXcgeK3/uzxEhluGOH5tiO/4XFE/CF/dcyug/mFNEm8Vu6jhaQ
         OQ32HXVAZP8Kr8C2XbzDJrta5m4BZa9ruGPhdHUen7j3VJVuwVtsVH+lPQqmAO+zYTvo
         mDIQD2Q0rqVWHe3LkPpA+zkLbl48v0x2Z3v5qmCBfqgaM/GIjmgl56iH6b/wTkWFKiBO
         3+1iX9EcD7H5DLNnZU0eLPiqWAoicyAs/CmWL7SdWtC/Kj4SPVJBx3tEdjgXbixiFIlN
         TSUMDeJ48RhfdtsDX0QGzGvu2O+Kc+mZJRuxCzKNvhOqP5+pv3S2puhBY9RkbbFbx5UD
         vbBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706959887; x=1707564687;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=iiyj18aRrfUjaARWFPuGN9TX5sHlhFENgj9Qgnt8pDY=;
        b=dYcQX/7GFi+gCmB4j6u7iaGX7sa44q9nU6wYVDBKV3Hq1Ytkj/yRGZXRGvZdEhbT8A
         yQC3eCqjKG0GkkfAzwLuQg2oqmoNQLvu7bB/6GE3It8xEldC0COZ7kB0f1a1ckUre4IZ
         T2hYZKhvO7DvnKxmss0h7w4/Y5W8F/zFBV9uNGkCZ5b3Le0k9m3XhjLT3+a3GCL+LyDQ
         hWKYBqH05YSuArdVGjsjxznJmnpl3ywMrbZYJfe3onItmUrsw2smtd4ypMh8PekV/Efb
         g/cbemETMQFXPsqbttbLtFiA217/Kzm4oPkAoatEam03Ef3/BP4vR3sz5kU7QrJ7eg03
         SGaA==
X-Gm-Message-State: AOJu0Yzqnot68wU6Mzaj9ja3TwJ0WvPmJJ/fTiQRxXx7bSH+Q/YetYGU
	ll+VWXSJuPhgoCSZZ+WK3ja7uGqSaxXz6BYumD1lsXx5uoXz6i9E7dltsqfmlQ9vhOsDBdLeUxx
	6g+Nq4dlUnKB9hF0eXWkWcR4CZ5+oGyvj6ne16w==
X-Google-Smtp-Source: AGHT+IF99EqKUWQtk6SXSV/8x9o3bEaJnVobDpePd5hGbz9dX+6eVxFynBcxRSdve8zHx7AWsKhU8FtZIgHn12Ax6MY=
X-Received: by 2002:a05:651c:211a:b0:2d0:6bc4:238c with SMTP id
 a26-20020a05651c211a00b002d06bc4238cmr8514483ljq.42.1706959887565; Sat, 03
 Feb 2024 03:31:27 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-2-carlo.nonato@minervasys.tech> <c6f2c117-c8a7-4356-8421-4781090279c0@suse.com>
In-Reply-To: <c6f2c117-c8a7-4356-8421-4781090279c0@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Sat, 3 Feb 2024 12:31:16 +0100
Message-ID: <CAG+AhRXA_RFLN2nxN7SwtTbwzyFZLteLxG2vsw9MTiJs5VbWAA@mail.gmail.com>
Subject: Re: [PATCH v6 01/15] xen/common: add cache coloring common code
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Thu, Feb 1, 2024 at 1:18=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 29.01.2024 18:17, Carlo Nonato wrote:
> > --- /dev/null
> > +++ b/docs/misc/cache-coloring.rst
> > @@ -0,0 +1,87 @@
> > +Xen cache coloring user guide
> > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D
> > +
> > +The cache coloring support in Xen allows to reserve Last Level Cache (=
LLC)
> > +partitions for Dom0, DomUs and Xen itself. Currently only ARM64 is sup=
ported.
> > +
> > +To compile LLC coloring support set ``CONFIG_LLC_COLORING=3Dy``.
> > +
> > +If needed, change the maximum number of colors with
> > +``CONFIG_NR_LLC_COLORS=3D<n>``.
> > +
> > +Compile Xen and the toolstack and then configure it via
> > +`Command line parameters`_.
> > +
> > +Background
> > +**********
> > +
> > +Cache hierarchy of a modern multi-core CPU typically has first levels =
dedicated
> > +to each core (hence using multiple cache units), while the last level =
is shared
> > +among all of them. Such configuration implies that memory operations o=
n one
> > +core (e.g. running a DomU) are able to generate interference on anothe=
r core
> > +(e.g .hosting another DomU). Cache coloring allows eliminating this
> > +mutual interference, and thus guaranteeing higher and more predictable
> > +performances for memory accesses.
>
> Since you say "eliminating" - what about shared mid-level caches? What ab=
out
> shared TLBs?

Cache coloring can help in reducing the interference, but you're right and
there are other factors to be considered. We will update the documentation =
to
better specify the applicability range and relax the terminology concerning
"eliminating" etc.

Thanks

> Jan


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 11:40:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 11:40:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675362.1050730 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWENK-0001s0-O9; Sat, 03 Feb 2024 11:39:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675362.1050730; Sat, 03 Feb 2024 11:39:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWENK-0001rt-LR; Sat, 03 Feb 2024 11:39:58 +0000
Received: by outflank-mailman (input) for mailman id 675362;
 Sat, 03 Feb 2024 11:39:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JBn0=JM=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rWENJ-0001rn-DQ
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 11:39:57 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f2e4a5c2-c288-11ee-98f5-efadbce2ee36;
 Sat, 03 Feb 2024 12:39:54 +0100 (CET)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2d051fb89fbso38242951fa.2
 for <xen-devel@lists.xenproject.org>; Sat, 03 Feb 2024 03:39:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2e4a5c2-c288-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1706960394; x=1707565194; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=64UaWdYCmmh8PjNcXUOydAJeDT3Q9ZzDW2Q62uynbwA=;
        b=edHMSFNjijgTzQQ0dl0MeE1gpHVFrG2C/Y8mZ2UDQVipKRfUmCA+uNwXKA29UNAnIE
         l89+tGpSBir9DKxnjMP4gpYLySR5nhmtVY5UlghCgQNywKP/cOjiVm7JLBsSMqiJGPEg
         WJ2Z7k4nkZf/ggVFTKVKVM+lIa1+30DOzbKoMSBZElz/c62CfPnovCnGAdPyC2HwEI3Q
         frL+Ro98utTXMyVcjQqcsSYtn9vSZHkMl+Zmt2x+zdMKJ5SXLMVWceYCyAGWPHyXGq8X
         iICHgSKItaGgx7Zo+9c+GSxn96eyJ7ZtZE+/2hEmebo6ebVsY+ZOmP1imZ06crjCXgOw
         SoFQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706960394; x=1707565194;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=64UaWdYCmmh8PjNcXUOydAJeDT3Q9ZzDW2Q62uynbwA=;
        b=E+PkZduoFB8C7k3W4qlUR6iTPpgA2x0f2+IfYZP3NSgNWBKoIP6ueE/V1LTewJQ8L8
         F7TJtPhppmezgmDdYN3doZ7ytQxNzYEI8twrdYRuH1+I8RI88/apzXcn5/01bGPjkHVW
         5Vl9jih1Zh5fxEv+amBl7ZGMJHa31WXPdySdrYyKc1uh9k2hONe1Vm4gHc282BBA+gJn
         Dth0YTTOCztV/gblAl0UHR37+VAxOr7Uo+529YT/OG9U6aUM1st9lW7YZnwnk1v0xdWs
         HEkxriWjQAtSrp0BP7KfZGnHlKjDgde2RL/qweaoEARvkP6ce4N9qq9DjUh+zfO+GntN
         VhgA==
X-Gm-Message-State: AOJu0YwvKMQ5+inYK/3QEAnwV/Dj/bAvdj4KaLIMjpFXLy5yTjUJR6Vh
	aCYu6/vsOMz8bfdfXZtBQAAbtVgSFTe75SQnWSj3mY1oBCYj0MYiwG3zXBQ1uoDjgA1lujgakhd
	vMu3M/Sl3pfsiQLm9Pq40lrcioqC4k+jBgyn3XQ==
X-Google-Smtp-Source: AGHT+IH4cT1re366oKsQSoTImKX+Qigf8CjiOT230T5Kq2eBN6QbmZr0HLSXUCJMHam0CPM9WXMODo3JAXqzRpZiz98=
X-Received: by 2002:a2e:9cc9:0:b0:2d0:9076:e9cc with SMTP id
 g9-20020a2e9cc9000000b002d09076e9ccmr1956581ljj.29.1706960393640; Sat, 03 Feb
 2024 03:39:53 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-5-carlo.nonato@minervasys.tech> <cf23d8a8-7111-4014-adc7-93ecd5f110ec@suse.com>
In-Reply-To: <cf23d8a8-7111-4014-adc7-93ecd5f110ec@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Sat, 3 Feb 2024 12:39:42 +0100
Message-ID: <CAG+AhRW4Ng7wkhB5f2H5ruCmYgium0YXZgA_bu7=T6HErUEg2A@mail.gmail.com>
Subject: Re: [PATCH v6 04/15] xen/arm: add Dom0 cache coloring support
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Marco Solieri <marco.solieri@minervasys.tech>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Thu, Feb 1, 2024 at 2:30=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 29.01.2024 18:18, Carlo Nonato wrote:
> > Add a command line parameter to allow the user to set the coloring
> > configuration for Dom0.
> > A common configuration syntax for cache colors is introduced and
> > documented.
> > Take the opportunity to also add:
> >  - default configuration notion.
> >  - function to check well-formed configurations.
> >
> > Direct mapping Dom0 isn't possible when coloring is enabled, so
> > CDF_directmap flag is removed when creating it.
>
> What implications does this have?

You will need an IOMMU to boot and extra care when assigning guest physical
addresses to Dom0. We have a new patch for that and it should solve what
Michal was pointing out in the cover letter.

> > --- a/docs/misc/xen-command-line.pandoc
> > +++ b/docs/misc/xen-command-line.pandoc
> > @@ -963,6 +963,15 @@ Controls for the dom0 IOMMU setup.
> >
> >  Specify a list of IO ports to be excluded from dom0 access.
> >
> > +### dom0-llc-colors
> > +> `=3D List of [ <integer> | <integer>-<integer> ]`
> > +
> > +> Default: `All available LLC colors`
> > +
> > +Specify dom0 LLC color configuration. This options is available only w=
hen
> > +`CONFIG_LLC_COLORING` is enabled. If the parameter is not set, all ava=
ilable
> > +colors are used.
>
> Even Arm already has a "dom0=3D" option. Is there a particular reason why
> this doesn't become a new sub-option there?

Cause this is a list and I don't think "dom0" option supports it since it's
already a list.

> As to meaning: With just a single <integer>, that's still a color value
> then (and not a count of colors)?

Exactly.

> Wouldn't it make sense to have a
> simpler variant available where you just say how many, and a suitable
> set/range is then picked?

This can be done in the future. It's not a feature that we want to support =
as
of now. For the moment we just want to give the user maximum freedom.

> Finally a nit: "This option is ...".
>
> > @@ -2188,10 +2190,16 @@ void __init create_dom0(void)
> >              panic("SVE vector length error\n");
> >      }
> >
> > -    dom0 =3D domain_create(0, &dom0_cfg, CDF_privileged | CDF_directma=
p);
> > +    if ( !llc_coloring_enabled )
> > +        flags |=3D CDF_directmap;
> > +
> > +    dom0 =3D domain_create(0, &dom0_cfg, flags);
> >      if ( IS_ERR(dom0) )
> >          panic("Error creating domain 0 (rc =3D %ld)\n", PTR_ERR(dom0))=
;
> >
> > +    if ( llc_coloring_enabled && (rc =3D dom0_set_llc_colors(dom0)) )
> > +        panic("Error initializing LLC coloring for domain 0 (rc =3D %d=
)", rc);
>
> As for the earlier patch, I find panic()ing here dubious. You can continu=
e
> quite fine, with a warning and perhaps again tainting the system.
>
> > --- a/xen/common/llc-coloring.c
> > +++ b/xen/common/llc-coloring.c
> > @@ -17,6 +17,63 @@ size_param("llc-way-size", llc_way_size);
> >  /* Number of colors available in the LLC */
> >  static unsigned int __ro_after_init max_nr_colors =3D CONFIG_NR_LLC_CO=
LORS;
> >
> > +static unsigned int __initdata dom0_colors[CONFIG_NR_LLC_COLORS];
> > +static unsigned int __initdata dom0_num_colors;
> > +
> > +/*
> > + * Parse the coloring configuration given in the buf string, following=
 the
> > + * syntax below.
> > + *
> > + * COLOR_CONFIGURATION ::=3D COLOR | RANGE,...,COLOR | RANGE
> > + * RANGE               ::=3D COLOR-COLOR
> > + *
> > + * Example: "0,2-6,15-16" represents the set of colors: 0,2,3,4,5,6,15=
,16.
> > + */
> > +static int parse_color_config(const char *buf, unsigned int *colors,
> > +                              unsigned int num_colors, unsigned int *n=
um_parsed)
>
> Is this function going to be re-used? If not, it wants to be __init.
> If so, I wonder where the input string is going to come from ...

You're right. It needs __init.

> Also "num_colors" looks to be misnamed - doesn't this specify an
> upper bound only?

It's the real size of the colors array. Than the used size will be found in
num_parsed.

> > +{
> > +    const char *s =3D buf;
> > +
> > +    if ( !colors || !num_colors )
> > +        return -EINVAL;
>
> Why do you check colors but not ...
>
> > +    *num_parsed =3D 0;
>
> ... num_parsed? I think internal functions don't need such NULL checks.

Ok I'll drop it.

>
> > +    while ( *s !=3D '\0' )
> > +    {
> > +        if ( *s !=3D ',' )
>
> Hmm, this way you also accept leading/trailing commas as well as multiple
> consecutive ones. Elsewhere we're more strict.
>
> > @@ -70,12 +150,85 @@ void __init llc_coloring_init(void)
> >      arch_llc_coloring_init();
> >  }
> >
> > +void domain_llc_coloring_free(struct domain *d)
> > +{
> > +    xfree(__va(__pa(d->llc_colors)));
>
> This __va(__pa()) trick deserves a comment, I think.
>
> > +}
> > +
> >  void domain_dump_llc_colors(const struct domain *d)
> >  {
> >      printk("Domain %pd has %u LLC colors: ", d, d->num_llc_colors);
> >      print_colors(d->llc_colors, d->num_llc_colors);
> >  }
> >
> > +static unsigned int *alloc_colors(unsigned int num_colors)
> > +{
> > +    unsigned int *colors;
> > +
> > +    if ( num_colors > max_nr_colors )
> > +        return NULL;
>
> Shouldn't check_colors() have made sure of this? If so, convert to
> ASSERT()?
>
> > +    colors =3D xmalloc_array(unsigned int, num_colors);
> > +    if ( !colors )
> > +        return NULL;
>
> These last two lines are redundant with ...
>
> > +    return colors;
>
> ... this one. Question then is whether this is useful at all as a
> separate helper function.

I'm going to remove alloc_colors().

> > +}
> > +
> > +static int domain_check_colors(const struct domain *d)
> > +{
> > +    if ( !d->num_llc_colors )
> > +    {
> > +        printk(XENLOG_ERR "No LLC color config found for %pd\n", d);
> > +        return -ENODATA;
> > +    }
> > +    else if ( !check_colors(d->llc_colors, d->num_llc_colors) )
>
> I generally recommend against use of "else" in cases like this one.
>
> > +    {
> > +        printk(XENLOG_ERR "Bad LLC color config for %pd\n", d);
> > +        return -EINVAL;
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> > +static int domain_set_default_colors(struct domain *d)
> > +{
> > +    unsigned int *colors =3D alloc_colors(max_nr_colors);
> > +    unsigned int i;
> > +
> > +    if ( !colors )
> > +        return -ENOMEM;
> > +
> > +    printk(XENLOG_WARNING
> > +           "LLC color config not found for %pd, using default\n", d);
>
> Leaving open what the default(s) is/are. Judging from ...
>
> > +    for ( i =3D 0; i < max_nr_colors; i++ )
> > +        colors[i] =3D i;
>
> ... this it's simply "all colors". Then perhaps have the message also
> say so?

Yep, ok.

> > +    d->llc_colors =3D colors;
> > +    d->num_llc_colors =3D max_nr_colors;
> > +
> > +    return 0;
> > +}
> > +
> > +int __init dom0_set_llc_colors(struct domain *d)
> > +{
> > +    unsigned int *colors;
> > +
> > +    if ( !dom0_num_colors )
> > +        return domain_set_default_colors(d);
> > +
> > +    colors =3D alloc_colors(dom0_num_colors);
> > +    if ( !colors )
> > +        return -ENOMEM;
> > +
> > +    memcpy(colors, dom0_colors, sizeof(unsigned int) * dom0_num_colors=
);
>
> sizeof(*colors) or some such please. Plus a check that colors and
> dom0_colors are actually of the same type. Alternatively, how about
> making dom0_colors[] __ro_after_init? Is this too much of a waste?

You mean an ASSERT on the two arrays type?

Thanks

> Jan


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 11:41:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 11:41:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675365.1050739 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWEPG-0003FV-2v; Sat, 03 Feb 2024 11:41:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675365.1050739; Sat, 03 Feb 2024 11:41:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWEPF-0003FO-WE; Sat, 03 Feb 2024 11:41:58 +0000
Received: by outflank-mailman (input) for mailman id 675365;
 Sat, 03 Feb 2024 11:41:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JBn0=JM=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rWEPE-0003FG-1y
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 11:41:56 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3a9dbc38-c289-11ee-8a45-1f161083a0e0;
 Sat, 03 Feb 2024 12:41:54 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-55c2cf644f3so3638441a12.1
 for <xen-devel@lists.xenproject.org>; Sat, 03 Feb 2024 03:41:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a9dbc38-c289-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1706960514; x=1707565314; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8+qu75hMXPYuPHwIGXQ06nMziFKRnaeLPjJHX+4ptT4=;
        b=BLfWxqv02FsNR+GQbUKLQFlrnmoXwBp6gaUThS4NgPBpmiJHBoper1ZZ5ii5seKzj4
         jbm/D+oGzoMXTEHeW4IEzdvB/2xKzEdblJqWOXzlRzKTB9hE4aqEPvt8pN02jie+8ro4
         5ZU/QZWTXx3Dm51DWXjDOyqKQ75NWB5eUnOFlj0/LoZFgYELctp3+N3+9NXbQShX3w70
         R8llKsv0xokmPez4L/NdhW1XgCYjc1e4tJH7F3oqVaY3pwjFfky9Vl3s2gOrkTfZtW23
         1xzdRz0lAHQGZ9iMnFg7gqws+TgRvUJwgEXoaANWXjjghs8YHKQkyvSJsaN6X7kw9RJX
         0FRQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706960514; x=1707565314;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8+qu75hMXPYuPHwIGXQ06nMziFKRnaeLPjJHX+4ptT4=;
        b=UZBxyu3p9kgmWlN2Q6fXSih6rU+RRqBCtkTBSmwAey+AB1lRaFK3lzQD+dSoV89z0G
         B6AA/gCWsUp8PkDAApTC/FiCWi5CHKTJCxM3IdM00mj4YeXKllD/cAjR+Z76A26s3wFK
         Vy7ByCjv240EtiCE4UGCFkHsT3fPmxk5flDDUluNFvSBROUF95Bp8gb+ff6N7ftHVm5t
         e5h25RJz5QZBdVIJzkwwvHofAYPXb6GZc8NOjYkiZ+78q4bABlVt2DNrj6V6A2LGolyN
         z6lUSBtjLZ6vK1OejUuHGwTTSzT9zBnh55lrKyQI7Q4zZvfwr0jmHlJMMw/4+irqCnuq
         m2ww==
X-Gm-Message-State: AOJu0Yw2xVCznqZvk4SCRaOace4ug+5tMiKCFurKYqJm5kWKWrh5slI0
	wh4h3XlDu/724dL1J91UcC/5k6Ley5wxlPycupDy+2ZCagTvwgY4LWOBUxbqpzzpIZCdcnWurPd
	YTjkmeKDSgkNfZWtffobjLVjV+fSjftkEKtLOCQ==
X-Google-Smtp-Source: AGHT+IEgX/70d8LGDrSJM8Lac3vjz6z98BriOFQm8mt3amQz/TJb0/25tEufKK1J8IqWzTWz7wj9lSXkh+diwcMjM8U=
X-Received: by 2002:a50:bb48:0:b0:560:14c4:58fe with SMTP id
 y66-20020a50bb48000000b0056014c458femr1108926ede.29.1706960514068; Sat, 03
 Feb 2024 03:41:54 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-6-carlo.nonato@minervasys.tech> <49827753-14f1-42cc-8791-27c5400e6e50@suse.com>
In-Reply-To: <49827753-14f1-42cc-8791-27c5400e6e50@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Sat, 3 Feb 2024 12:41:43 +0100
Message-ID: <CAG+AhRUGdkqhoSL-ihiCGZctH2mkSOi+ap+=a_kKLzJRGugCbw@mail.gmail.com>
Subject: Re: [PATCH v6 05/15] xen: extend domctl interface for cache coloring
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Thu, Feb 1, 2024 at 2:51=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 29.01.2024 18:18, Carlo Nonato wrote:
> > @@ -858,6 +859,16 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t=
) u_domctl)
> >                  __HYPERVISOR_domctl, "h", u_domctl);
> >          break;
> >
> > +    case XEN_DOMCTL_set_llc_colors:
> > +        if ( !llc_coloring_enabled )
> > +            break;
>
> With "ret" still being 0, this amounts to "successfully ignored". Ought
> to be -EOPNOTSUPP, I guess.

Yep.

> > +        ret =3D domain_set_llc_colors_domctl(d, &op->u.set_llc_colors)=
;
> > +        if ( ret =3D=3D -EEXIST )
> > +            printk(XENLOG_ERR
> > +                   "Can't set LLC colors on an already created domain\=
n");
>
> If at all a dprintk(). But personally I think even that's too much - we
> don't do so elsewhere, I don't think.

I'm going to drop the printk.

> > --- a/xen/common/llc-coloring.c
> > +++ b/xen/common/llc-coloring.c
> > @@ -4,6 +4,7 @@
> >   *
> >   * Copyright (C) 2022 Xilinx Inc.
> >   */
> > +#include <xen/guest_access.h>
> >  #include <xen/keyhandler.h>
> >  #include <xen/llc-coloring.h>
> >  #include <xen/param.h>
> > @@ -229,6 +230,30 @@ int __init dom0_set_llc_colors(struct domain *d)
> >      return domain_check_colors(d);
> >  }
> >
> > +int domain_set_llc_colors_domctl(struct domain *d,
> > +                                 const struct xen_domctl_set_llc_color=
s *config)
>
> What purpose has the "domctl" in the function name?

To signal that it's called from domctl. Do you suggest leaving it out?

> > +{
> > +    unsigned int *colors;
> > +
> > +    if ( d->num_llc_colors )
> > +        return -EEXIST;
> > +
> > +    if ( !config->num_llc_colors )
> > +        return domain_set_default_colors(d);
> > +
> > +    colors =3D alloc_colors(config->num_llc_colors);
> > +    if ( !colors )
> > +        return -ENOMEM;
>
> Hmm, I see here you call the function without first having bounds checked
> the input. But in case of too big a value, -ENOMEM is inappropriate, so
> such a check wants adding up front anyway.

Yeah, ok.

> > +    if ( copy_from_guest(colors, config->llc_colors, config->num_llc_c=
olors) )
> > +        return -EFAULT;
>
> There again wants to be a check that the pointed to values are the same,
> or at least of the same size. Else you'd need to do element-wise copy.
>
> > +    d->llc_colors =3D colors;
> > +    d->num_llc_colors =3D config->num_llc_colors;
> > +
> > +    return domain_check_colors(d);
>
> And if this fails, you leave the domain with the bad settings? Shouldn't
> you check and only then store pointer and count?

Yes. I'm gonna change it.

Thanks.

> > --- a/xen/include/public/domctl.h
> > +++ b/xen/include/public/domctl.h
> > @@ -1190,6 +1190,13 @@ struct xen_domctl_vmtrace_op {
> >  typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
> >  DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
> >
> > +struct xen_domctl_set_llc_colors {
> > +    /* IN LLC coloring parameters */
> > +    uint32_t num_llc_colors;
> > +    uint32_t padding;
>
> I see you've added padding, but: You don't check it to be zero. Plus
> the overwhelming majority of padding fields is named "pad".
>
> Jan


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 11:43:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 11:43:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675368.1050750 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWERA-0003nn-Dr; Sat, 03 Feb 2024 11:43:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675368.1050750; Sat, 03 Feb 2024 11:43:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWERA-0003ng-B3; Sat, 03 Feb 2024 11:43:56 +0000
Received: by outflank-mailman (input) for mailman id 675368;
 Sat, 03 Feb 2024 11:43:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JBn0=JM=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rWER9-0003na-08
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 11:43:55 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 812aeecf-c289-11ee-98f5-efadbce2ee36;
 Sat, 03 Feb 2024 12:43:52 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-51117bfd452so5068164e87.3
 for <xen-devel@lists.xenproject.org>; Sat, 03 Feb 2024 03:43:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 812aeecf-c289-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1706960632; x=1707565432; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=xwaSY2p3h1mRDxMHMFJXzP8ZbV1sm8YiOVuurOA3qn4=;
        b=GDaGGzuHLPrO/LMlsKf0Ua0lEoLr2rPKW9Mx89DeS9LqapsyGHYWFfyVB56sUGRMIe
         3hgqqPyyOKyXzmv7SY079iEJo2ED8rTMrHPQ8TAY7HQGZL0Hx24LNwR8l7yGz3HI2kRg
         5TI7zZjk7XWUIKWM279GWx0gFmvMUIWWbzeer5yaRyooe8rMh+acEVOtbFqk2/xb6wpv
         Cidi4zSIufYZqyYqymzfK+jSYS9N4+w1cBATqLiWjPPH1yZTkqXZVjJnN/d7AOQfkYba
         HwYEdt03zg20WFUHc+okOK6Uv1BTZw+OSPc6suzd9KF1gljDd85f7RDjxGraleUeTpGh
         sp7A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706960632; x=1707565432;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=xwaSY2p3h1mRDxMHMFJXzP8ZbV1sm8YiOVuurOA3qn4=;
        b=k+0dfo0Wi3GCuyP9bES/ZvImaaZRIyjvhfwsa0wVim6KWG/tPj/vh3KsHd/5hd1NN9
         HjOr4dqy24UsbIaPf4Gg1Yz0aRlYWEGQrEY5rMa15J6AFt+9XFFHsbFbCEk/RxVlmlAy
         tjOwfzpvVrhrPVjQpcluYBZeAcd4XXfr/8oNuLbvFM5rIxcAtQmoumv9gkepeY0baQ4I
         vlPOzNFHvrI+yVBfxRfqOeXDAPcB45siZib/u7F2WcCsqLYTGhmpv/GjMkNPahzgDsxI
         swAOyWOVui2DLzUYPuqcgdfZCYZb+vcIbktTUVxxGhGBwfd5nx5jI4Gm0UXE3Q2NBxhE
         fC6w==
X-Gm-Message-State: AOJu0Yy+fQcPRaWurw33SUTvgxCOPXxrpegdU121yqIqZ+EldfWyac+p
	RDtMDR0u37szT2enkHSASoCtPuekgbXUHjNHQIzd0uFxBZb+mkVv+koanUq2UTpUTj9uhCjZQHY
	us8pG1j2Hh2QlN3i3qDdtKa9qRcas4etpeWstvQ==
X-Google-Smtp-Source: AGHT+IFOmHuKGzl4AA28PwKv37wNKQEc5eexygy3Ozekw29jrH2WEICYF77aFF75gSdJjwzZSpHNtZb/akmUMeBbU6c=
X-Received: by 2002:a05:6512:308d:b0:511:2f20:3c55 with SMTP id
 z13-20020a056512308d00b005112f203c55mr6685812lfd.12.1706960632357; Sat, 03
 Feb 2024 03:43:52 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-8-carlo.nonato@minervasys.tech> <a6ffb4a5-cd83-4c04-9e5a-12d605863b5d@suse.com>
In-Reply-To: <a6ffb4a5-cd83-4c04-9e5a-12d605863b5d@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Sat, 3 Feb 2024 12:43:41 +0100
Message-ID: <CAG+AhRUG8nNF0cTtFYLu1TrV4fWEbix57Ah_-Zsz=U2yRWcGpw@mail.gmail.com>
Subject: Re: [PATCH v6 07/15] xen/arm: add support for cache coloring
 configuration via device-tree
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, 
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, 
	Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Thu, Feb 1, 2024 at 3:19=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 29.01.2024 18:18, Carlo Nonato wrote:
> > @@ -950,6 +951,11 @@ void __init create_domUs(void)
> >  #endif
> >          }
> >
> > +        dt_property_read_string(node, "llc-colors", &llc_colors_str);
> > +        if ( !llc_coloring_enabled && llc_colors_str)
> > +            printk(XENLOG_WARNING
> > +                   "'llc-colors' found, but LLC coloring is disabled\n=
");
>
> Why's this just a warning, when ...

This informs the user that this configuration will be ignored, but the DomU=
 can
be constructed anyway...

> > @@ -960,6 +966,11 @@ void __init create_domUs(void)
> >              panic("Error creating domain %s (rc =3D %ld)\n",
> >                    dt_node_name(node), PTR_ERR(d));
> >
> > +        if ( llc_coloring_enabled &&
> > +             (rc =3D domain_set_llc_colors_from_str(d, llc_colors_str)=
) )
> > +            panic("Error initializing LLC coloring for domain %s (rc =
=3D %d)\n",
> > +                  dt_node_name(node), rc);
>
> ... this results in panic()?

... while here we can't continue because there's some error in the
configuration and the DomU can't be constructed. Domains must have a valid
coloring configuration.

> > --- a/xen/common/llc-coloring.c
> > +++ b/xen/common/llc-coloring.c
> > @@ -254,6 +254,29 @@ int domain_set_llc_colors_domctl(struct domain *d,
> >      return domain_check_colors(d);
> >  }
> >
> > +int domain_set_llc_colors_from_str(struct domain *d, const char *str)
>
> __init ?

Yes.

Thanks.

> > +{
> > +    int err;
> > +    unsigned int *colors;
> > +
> > +    if ( !str )
> > +        return domain_set_default_colors(d);
> > +
> > +    colors =3D alloc_colors(max_nr_colors);
> > +    if ( !colors )
> > +        return -ENOMEM;
> > +
> > +    err =3D parse_color_config(str, colors, max_nr_colors, &d->num_llc=
_colors);
> > +    if ( err )
> > +    {
> > +        printk(XENLOG_ERR "Error parsing LLC color configuration.");
>
> Nit: No full stop at the end of log messages please.
>
> > +        return err;
> > +    }
> > +    d->llc_colors =3D colors;
> > +
> > +    return domain_check_colors(d);
>
> Same ordering issue as in the earlier patch, I think.
>
> Jan


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 11:59:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 11:59:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675376.1050760 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWEfs-0006YQ-Rg; Sat, 03 Feb 2024 11:59:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675376.1050760; Sat, 03 Feb 2024 11:59:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWEfs-0006YJ-NW; Sat, 03 Feb 2024 11:59:08 +0000
Received: by outflank-mailman (input) for mailman id 675376;
 Sat, 03 Feb 2024 11:59:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWEfr-0006Y9-Fb; Sat, 03 Feb 2024 11:59:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWEfr-000672-DG; Sat, 03 Feb 2024 11:59:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWEfq-0006iF-Vt; Sat, 03 Feb 2024 11:59:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWEfq-0004ts-VU; Sat, 03 Feb 2024 11:59:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=lmMbrpcRdnSzJRgfEO9S/k9Ic53kVtuvOy5faK8rCiI=; b=Hu0RS7viizIncOk+IcAeoYznxI
	AMwucpbt9pC1ko5QiY1iznDJdZ953RJoIXQGTAkIpI2b4Go7dyHNuVRcRp7F5ba8F/A/qlVgSvPT6
	TTLFe0Fm1PYLvKXiZiXeZZMHLokgE/4d5+R+rNXdjiXnJWHfkbAD1rqSgy3+SZjI0LIY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184566-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184566: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-shadow:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=021533194476035883300d60fbb3136426ac8ea5
X-Osstest-Versions-That:
    linux=5c24e4e9e70822cf49955fc8174bc5efaa93d17f
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 03 Feb 2024 11:59:06 +0000

flight 184566 linux-linus real [real]
flight 184575 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184566/
http://logs.test-lab.xenproject.org/osstest/logs/184575/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-shadow 22 guest-start/debian.repeat fail pass in 184575-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail blocked in 184554
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184554
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184554
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184554
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184554
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184554
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184554
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184554
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 linux                021533194476035883300d60fbb3136426ac8ea5
baseline version:
 linux                5c24e4e9e70822cf49955fc8174bc5efaa93d17f

Last test of basis   184554  2024-02-01 18:42:19 Z    1 days
Testing same since   184566  2024-02-02 15:26:30 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aaron Conole <aconole@redhat.com>
  Andrew Halaney <ahalaney@redhat.com>
  Andrew Lunn <andrew@lunn.ch>
  Arınç ÜNAL <arinc.unal@arinc9.com>
  Benjamin Poirier <bpoirier@nvidia.com>
  Breno Leitao <leitao@debian.org>
  Brett Creeley <brett.creeley@amd.com>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Chuck Lever <chuck.lever@oracle.com>
  Daniel Golle <daniel@makrotopia.org>
  David S. Miller <davem@davemloft.net>
  David Wang <00107082@163.com>
  Dmitry Goncharov <dgoncharov@users.sf.net>
  Dmitry Safonov <dima@arista.com>
  Donald Hunter <donald.hunter@gmail.com>
  Eric Dumazet <edumazet@google.com>
  Esben Haabendal <esben@geanix.com>
  Fedor Pchelkin <pchelkin@ispras.ru>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Geetha sowjanya <gakula@marvell.com>
  Hangbin Liu <liuhangbin@gmail.com>
  Helge Deller <deller@gmx.de>
  Helge Deller <deller@kernel.org>
  Horatiu Vultur <horatiu.vultur@microchip.com>
  Hui Zhou <hui.zhou@corigine.com>
  Ido Schimmel <idosch@nvidia.com>
  Jacob Keller <jacob.e.keller@intel.com>
  Jakub Kicinski <kuba@kernel.org>
  Jan Beulich <jbeulich@suse.com>
  Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
  Jozsef Kadlecsik <kadlec@netfilter.org>
  Krishneil Singh  <krishneil.k.singh@intel.com>
  Linus Lüssing <linus.luessing@c0d3.blue>
  Linus Torvalds <torvalds@linux-foundation.org>
  Louis Peens <louis.peens@corigine.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Matthias May <matthias.may@westermo.com>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Michal Vokáč <michal.vokac@ysoft.com>
  Mohammad Nassiri <mnassiri@ciena.com>
  Naama Meir <naamax.meir@linux.intel.com>
  Namjae Jeon <linkinjeon@kernel.org>
  Nathan Chancellor <nathan@kernel.org>
  Nicolas Dichtel <nicolas.dichtel@6wind.com>
  Nikita Zhandarovich <n.zhandarovich@fintech.ru>
  Nikolay Aleksandrov <razor@blackwall.org>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Paolo Abeni <pabeni@redhat.com>
  Parav Pandit <parav@nvidia.com>
  Pavan Kumar Linga <pavan.kumar.linga@intel.com>
  Praveen Kaligineedi <pkaligineedi@google.com>
  Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
  Ryan Schaefer <ryanschf@amazon.com>
  Samasth Norway Ananda <samasth.norway.ananda@oracle.com>
  Simon Horman <horms@kernel.org>
  Simon Wunderlich <sw@simonwunderlich.de>
  Souradeep Chakrabarti <schakrabarti@linux.microsoft.com>
  Sven Eckelmann <sven@narfation.org>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Wen Gu <guwen@linux.alibaba.com>
  Xin Long <lucien.xin@gmail.com>
  Yuezhang Mo <Yuezhang.Mo@sony.com>
  Zhang Bingwu <xtexchooser@duck.com>
  Zhipeng Lu <alexious@zju.edu.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   5c24e4e9e708..021533194476  021533194476035883300d60fbb3136426ac8ea5 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 16:11:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 16:11:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675409.1050783 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWIc2-000619-9V; Sat, 03 Feb 2024 16:11:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675409.1050783; Sat, 03 Feb 2024 16:11:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWIc2-000612-4y; Sat, 03 Feb 2024 16:11:26 +0000
Received: by outflank-mailman (input) for mailman id 675409;
 Sat, 03 Feb 2024 16:11:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWIc0-00060d-Ez; Sat, 03 Feb 2024 16:11:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWIc0-0002uE-Cb; Sat, 03 Feb 2024 16:11:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWIbz-0008FW-SB; Sat, 03 Feb 2024 16:11:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWIbz-0005qz-Rj; Sat, 03 Feb 2024 16:11:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=KgKo5qkwaDpf0esb3icBOWDBst/oNG+W13FOnOLRV9Q=; b=zR/vWeW+vHPtZo2TAqgSVPpTLK
	JGm1GZGbnf+6kvEVnyQCSTTer6ZLwLMUOVFFHM2tP4UlE2HTxAgKiJ+lTCR4BdTTKZcVjuvSjU2dR
	AcDNQ5Ys3ghLElr+kmfqZHKmUqrOrgjeqsjN+iUvSOhzFLW5V0lhzuExtx2ITrnbAW6Q=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184568-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184568: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-dom0pvh-xl-amd:guest-start:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit1:guest-start.2:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
X-Osstest-Versions-That:
    xen=bc45f20c01f1711bc56a4bb0955c49c182a5a03a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 03 Feb 2024 16:11:23 +0000

flight 184568 xen-unstable real [real]
flight 184577 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184568/
http://logs.test-lab.xenproject.org/osstest/logs/184577/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-dom0pvh-xl-amd 14 guest-start      fail pass in 184577-retest
 test-armhf-armhf-xl-credit1  19 guest-start.2       fail pass in 184577-retest
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install fail pass in 184577-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop  fail in 184577 like 184557
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184557
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184557
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184557
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184557
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184557
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184557
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184557
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184557
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184557
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184557
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184557
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
baseline version:
 xen                  bc45f20c01f1711bc56a4bb0955c49c182a5a03a

Last test of basis   184557  2024-02-01 22:11:04 Z    1 days
Testing same since   184568  2024-02-02 19:24:35 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Christian Lindig <christian.lindig@cloud.com>
  Edwin Török <edwin.torok@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   bc45f20c01..3f819af8a7  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc -> master


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 19:30:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 19:30:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675438.1050843 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWLi8-0004ij-8e; Sat, 03 Feb 2024 19:29:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675438.1050843; Sat, 03 Feb 2024 19:29:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWLi8-0004g3-20; Sat, 03 Feb 2024 19:29:56 +0000
Received: by outflank-mailman (input) for mailman id 675438;
 Sat, 03 Feb 2024 18:53:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WIqD=JM=gmail.com=rbmarliere@srs-se1.protection.inumbo.net>)
 id 1rWL8i-0000aF-Bf
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 18:53:20 +0000
Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com
 [2607:f8b0:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7e4fe044-c2c5-11ee-98f5-efadbce2ee36;
 Sat, 03 Feb 2024 19:53:18 +0100 (CET)
Received: by mail-pl1-x62f.google.com with SMTP id
 d9443c01a7336-1d746ce7d13so26478225ad.0
 for <xen-devel@lists.xenproject.org>; Sat, 03 Feb 2024 10:53:18 -0800 (PST)
Received: from mail.marliere.net ([24.199.118.162])
 by smtp.gmail.com with ESMTPSA id
 l5-20020a170902e2c500b001d974ffa1fcsm3058420plc.173.2024.02.03.10.53.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 03 Feb 2024 10:53:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e4fe044-c2c5-11ee-98f5-efadbce2ee36
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706986397; x=1707591197;
        h=cc:to:in-reply-to:references:message-id:content-transfer-encoding
         :mime-version:subject:date:dkim-signature:from:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=UMK6uKXfNz/nsGcon/C8VrTPd2Ab3T3BTLt2EsWBnyE=;
        b=saFRPwXInioejC9THgb1GFaYjwXfKwzzx3foD7cVpDbopbriM/v9ZFu5Q3Ejh4zO8n
         DTh72HFCjeZcLJzSDs2mJNGWVaXjZaWJ+n72mh7Flt2U8weOVnLllUp2OG0enPXC/xrO
         Z2UB/VoZJlFNPGl0Rne9gSYIimAB1dMphhAmrCJGO6Uj4FVfAaOaIdSP/asCul+K9DMv
         TnAc3V/KsIgbmi6YXOCptcn4eJsFXr6hl/KHoAHJmqYToT0+uU8WQWYWAgdCI24hFq5H
         sK5yuc/PY1NNBfimXngFzI8cFRdyqKPWiQLjEpuGko6E2QQm/Zhu1QkrOQGYLXqvSJmE
         p20A==
X-Gm-Message-State: AOJu0YzAEmtXHMPyiph74d9yUzLMM7lcXoT50wkAW1eG39HfggOE0V8k
	YLmDmyFK2x7H0dotAq9ZdxXgSfmKwkQdxiat+Sgsv0Th2C56+rjN
X-Google-Smtp-Source: AGHT+IGbH6ZYJYqCyLdLrsDa8OizYNf648PZ/HHOu7tx8iYG0Ii4xFkddglDF0dNTfItn0AvcwNOSw==
X-Received: by 2002:a17:903:249:b0:1d4:e208:9b0 with SMTP id j9-20020a170903024900b001d4e20809b0mr11130471plh.44.1706986397240;
        Sat, 03 Feb 2024 10:53:17 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCW2Fadp/SGhL3HK6yTH0lKP3rfGEbv8/GAi3tzKf8/aeB0by5STQsj+r3op+wDEmYIuRn3Y/pBERUwnHxVAj4AYKwjYXs2RMwgg8oZjWyGFLFfGNnYidF/TBcQMLaYr33BdKR1jQVFG4GsnUWg1Hw7QkEcXLBHAY2zOJza4ElNmpOGKSOReeWllojdyhoXBArdQTvK9HvU0OGd8+ruzpG4=
From: "Ricardo B. Marliere" <ricardo@marliere.net>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marliere.net;
	s=2023; t=1706986395;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=UMK6uKXfNz/nsGcon/C8VrTPd2Ab3T3BTLt2EsWBnyE=;
	b=d/b91IH79JyEKl4lTtvPDJZkgOP3vWmmAkRT4CCuLWZ0q0cM4YSYxO5x90ynhmRNqoGAb2
	MX5rNMnI6UgqZnWkeyWxjhJFxu6toynOA6u1Xf39buDCaFsawntbhwVEJp4N+4J3Ac2VGr
	5JhntPbRQe4HHwF5nDA3tPxH7mRIfY9eC35llsvDKa1lKbPx5tocjZGIlRdh4fmxIRP4dz
	lDNqKI/O8JqdQhhFkkGn7Ql88HC6qMjbRvdMxfjHIwzMEgQ1Pv8KMtRVhdEHMuj4oHrKqI
	4GnnRAarZ2sl5RD64TTF9uXul3GTgCxT+M5jvICQ2/zjXOW1oyqh6CeYlAXRYw==
Authentication-Results: ORIGINATING;
	auth=pass smtp.auth=ricardo@marliere.net smtp.mailfrom=ricardo@marliere.net
Date: Sat, 03 Feb 2024 15:53:38 -0300
Subject: [PATCH 2/2] xen: balloon: make balloon_subsys const
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-Id: <20240203-bus_cleanup-xen-v1-2-c2f5fe89ed95@marliere.net>
References: <20240203-bus_cleanup-xen-v1-0-c2f5fe89ed95@marliere.net>
In-Reply-To: <20240203-bus_cleanup-xen-v1-0-c2f5fe89ed95@marliere.net>
To: Juergen Gross <jgross@suse.com>, 
 Stefano Stabellini <sstabellini@kernel.org>, 
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
 Greg Kroah-Hartman <gregkh@linuxfoundation.org>, 
 "Ricardo B. Marliere" <ricardo@marliere.net>
X-Developer-Signature: v=1; a=openpgp-sha256; l=931; i=ricardo@marliere.net;
 h=from:subject:message-id; bh=a15mtQr01yfWQWLF5/5WsbY/2R7o79ghPTBzDCe5X0Y=;
 b=owEBbQKS/ZANAwAKAckLinxjhlimAcsmYgBlvouxq6eiZcwBEfL0eKDgR/Tsw3cuRwc1xJx6s
 WHMrCFJCRGJAjMEAAEKAB0WIQQDCo6eQk7jwGVXh+HJC4p8Y4ZYpgUCZb6LsQAKCRDJC4p8Y4ZY
 poA/EACY3HzoQHClLbpnthOER522MUTZO2lQJLXt+WNqaobWySLeseiDeex2CGxWdeGsn4gzV3s
 SCD7J00yLWCX29j3M/BvDTPmePA+mavHsyQ8rQielbuNnr3B5iXLYmHFaBEHd4D2+LC7NolXmwA
 XDAa2sLewEnr7rvIk2PMjlmcHa/KIOh6G7Ls/12Q95339qmdPhDIbZk4NjH2KY8Pr8OAzRw4DpW
 UptByEzu3k1kS59uBT7FfrjvIU4fWmEmlJAWU/LoXKt+AlkbrOoNcKRsXX0Z174cxZlT6u3EHeA
 I2zaEzpITAelotRc53pJpVw4pzdKJoodaRIIF0+Tnolc3K2iclnHCHLVP4ZzSdP4cuSd5yaD/Me
 yyS84r5wQksXW4aqTvmhHDVz/YVK/N2O9wGRo/hQiDzT5LZKnBuXJWYc0zDIrLdI/Bint4kKJDm
 ygxa+XRL8U7KhpMsuSe2vrNMlvt5+r6SVeqQ+3PyxjSEIHm5A6Tlq87EC3InHuN7v+bv6ciEkb6
 WMn5YKTe/VR9rmjUqqIs9KaSOzg5EBPrU5KJAi+EPyEBuUOOYUoNxAaea7dJMRdHqRIhcyvb40J
 khD83wKupnXtMRyuSG2iGRp64bsE1ep9J/UU27/ZdmANXs+kxoaU0PnFf1APqUm18cS3GHxwLRs
 PWecp9qGUFo1M6g==
X-Developer-Key: i=ricardo@marliere.net; a=openpgp;
 fpr=030A8E9E424EE3C0655787E1C90B8A7C638658A6

Now that the driver core can properly handle constant struct bus_type,
move the balloon_subsys variable to be a constant structure as well,
placing it into read-only memory which can not be modified at runtime.

Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Ricardo B. Marliere <ricardo@marliere.net>
---
 drivers/xen/xen-balloon.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/xen/xen-balloon.c b/drivers/xen/xen-balloon.c
index 8cd583db20b1..b293d7652f15 100644
--- a/drivers/xen/xen-balloon.c
+++ b/drivers/xen/xen-balloon.c
@@ -237,7 +237,7 @@ static const struct attribute_group *balloon_groups[] = {
 	NULL
 };
 
-static struct bus_type balloon_subsys = {
+static const struct bus_type balloon_subsys = {
 	.name = BALLOON_CLASS_NAME,
 	.dev_name = BALLOON_CLASS_NAME,
 };

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Sat Feb 03 19:30:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 19:30:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675434.1050832 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWLi7-0004bA-Ow; Sat, 03 Feb 2024 19:29:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675434.1050832; Sat, 03 Feb 2024 19:29:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWLi7-0004b3-L2; Sat, 03 Feb 2024 19:29:55 +0000
Received: by outflank-mailman (input) for mailman id 675434;
 Sat, 03 Feb 2024 18:53:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WIqD=JM=gmail.com=rbmarliere@srs-se1.protection.inumbo.net>)
 id 1rWL8c-0000Zl-GX
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 18:53:14 +0000
Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com
 [2607:f8b0:4864:20::1034])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7a9d7eb3-c2c5-11ee-8a45-1f161083a0e0;
 Sat, 03 Feb 2024 19:53:12 +0100 (CET)
Received: by mail-pj1-x1034.google.com with SMTP id
 98e67ed59e1d1-296043e44caso2556332a91.0
 for <xen-devel@lists.xenproject.org>; Sat, 03 Feb 2024 10:53:12 -0800 (PST)
Received: from mail.marliere.net ([24.199.118.162])
 by smtp.gmail.com with ESMTPSA id
 p1-20020a17090adf8100b0028df5c748e4sm2119387pjv.44.2024.02.03.10.53.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 03 Feb 2024 10:53:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a9d7eb3-c2c5-11ee-8a45-1f161083a0e0
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706986391; x=1707591191;
        h=cc:to:content-transfer-encoding:mime-version:message-id:date
         :subject:dkim-signature:from:x-gm-message-state:from:to:cc:subject
         :date:message-id:reply-to;
        bh=OZiekaIfpVrKOF3qlNauN16zVWw+/1GjqDEklLmvnts=;
        b=kOlYa3fbH+MjFvLmBnPtfcLPgOErXcHEyqa18y3X4o75okE4A5NQkaRku2sRCe3eXD
         xUgxqZNwcttq3DHc0YzKsXoNkmknXV4Bqcrx2xW32xKk7cIoRjA2vAd+CIPlpEcvAm8c
         jdEd75Zec1BAiA2h4SiKPa0OMdIx5UIdOrl3PPkgmXJaMswYPnullBDwp2Q4/mLVUypy
         PtTCWvV1eUJw+k3yCtylFlJXTh+XvT3qI+cxcbgdoDbqcj1Eo+kOEKVdu88H99td+lDB
         x9w1/J/lH2NAnht7pAUINP224r/MqjFthrrvosj4355LuzWl4mHi5hXJ3DUj3tYfqUxb
         r2tg==
X-Gm-Message-State: AOJu0YzHmZWa4mCRSoktqBTT6s3uIZuTh3+CO+Vko7avjoxsqXxZnJO0
	MelshDJ0HkhyHrnn0tkN1qJqVSff+/BM+vQ+vdz21fuURYvtAsngGOjixwgyoO7xPA==
X-Google-Smtp-Source: AGHT+IFqLotwZ+hQeDo37iliFJZepy3q/2BZbRkjOXY4QBZFo73gskAH5bb0qVTSKR5AWfd6OLrGcA==
X-Received: by 2002:a17:90b:1014:b0:295:f310:efd6 with SMTP id gm20-20020a17090b101400b00295f310efd6mr5231697pjb.43.1706986390968;
        Sat, 03 Feb 2024 10:53:10 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXhGFw3RCfKea7l4V9sFQec+KG8sGmKcqPuZBoeIDD3LDJwGnGU4R7RYN5Ty9EMkdB10ONC6MN4vn6dE9GENBhkKSbja5TqgQNwLqRbuovwYC41AJFpjEarUOVfSNZaGMGrTEHsPSfiFbaxUrXIhPzNO/wa5I8+KJ2frC37FZrZDbHMqgBR0E37giN09AYX0TWxIqzPgBKZYDmiWhfHnVM=
From: "Ricardo B. Marliere" <ricardo@marliere.net>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marliere.net;
	s=2023; t=1706986388;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=OZiekaIfpVrKOF3qlNauN16zVWw+/1GjqDEklLmvnts=;
	b=H97NBcnpZQ59hIs5U/nx99cmSh4vblAVkS4yZRJavzuz0qX6V6CYy6vn/R7wctxO8ju9Ql
	oQehq2n+r+IAj0PTYhkGn0st5hFzkmrv5k+Obh3VuBq1MEE1iyrDlaIxVgcAb0fX+6d76L
	DkHb9jM4+UCVTT0eisaM1HdwoR0jVdJ2NojvVOHuY2tEI2QwqLwqukxqok841mTfvzs/hF
	Zrt8+rguJ8HF/Q/rhTMpEdybUUuCgS6bZSElhgZEkKcviiQjLJC15yyvLM6qeMZ+ez0gLu
	KajZ6/8MHRGJxrWMg+JbHJKXsLYmz4XP5OGq91WxayeS7VS5L5r6HaybFMs6kw==
Authentication-Results: ORIGINATING;
	auth=pass smtp.auth=ricardo@marliere.net smtp.mailfrom=ricardo@marliere.net
Subject: [PATCH 0/2] drivers: xen: struct bus_type cleanup
Date: Sat, 03 Feb 2024 15:53:36 -0300
Message-Id: <20240203-bus_cleanup-xen-v1-0-c2f5fe89ed95@marliere.net>
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
X-B4-Tracking: v=1; b=H4sIALCLvmUC/x3M0QpAQBBA0V/RPNtasyh+RdJgMKWh3Uht/t3m8
 TzcGyGwFw7QZhE83xLk0IQiz2DaSFc2MicDWiwtWmfGKwzTzqTXaR5WM1NTUY1Yk6sgVafnRZ7
 /2PXv+wGEV5g+YQAAAA==
To: Juergen Gross <jgross@suse.com>, 
 Stefano Stabellini <sstabellini@kernel.org>, 
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
 Greg Kroah-Hartman <gregkh@linuxfoundation.org>, 
 "Ricardo B. Marliere" <ricardo@marliere.net>
X-Developer-Signature: v=1; a=openpgp-sha256; l=1036; i=ricardo@marliere.net;
 h=from:subject:message-id; bh=DMQ8P/LGJBRCmHwEmxktQacfWtwbjDULAMo73R7wiAQ=;
 b=owEBbQKS/ZANAwAKAckLinxjhlimAcsmYgBlvoux0PW32SbhsfI597K/NU/w+vQgdmNdCHwJK
 LXOgiyVXqGJAjMEAAEKAB0WIQQDCo6eQk7jwGVXh+HJC4p8Y4ZYpgUCZb6LsQAKCRDJC4p8Y4ZY
 pgXqD/9Hgb+Kr23vOwiCVsyz6tS8SSlJymR+rzt8b+oFSLBUAHPA02/D+eCzFgN5gzQmavvwB3o
 3xtsKg8AUqbZNmLvQSnO7ahHqeevzhRjG8vqgxwOBemczrTzh3H/WublHvK1ojsd0aTLI0AMrUt
 xChU1a/m82yZHHslgQTThr8jHjPOW4cjzReI9o/Xl2hOLuXK78IE+a51ZnY1SUI0W+tP6cE9LU7
 /gSZwEftp4SuL+R9rJrwSfeWbSc2dLxuvf/yNiYITEswhQZCNTfxd9wU/IK85cv2K22W5AMTbsY
 ObdpjtWaZZjAwZrt+VU8B462reAEZxRDiFI51TsiED8p9274RmMDC6MQFwPxpRThDSoIGqhvTdb
 jejuZVSPrIOfWdVEP/ERXcgVDjb0mDkEmeYqQb/5hYrVK/cCqcEcIVrylekfh1iGzj5xqgtffqq
 A0lXEgMFT7OC6FtqcwHtxs5GXUCqNYqmHRoZoy4knRlYeSbeypMG4gIYb++4vaeXNLdqV22T0Sa
 I57ls3glisePCcjL8vs/TNJs0WxMVsyTJje2NqcyibPhNk3ZQW0a+ET+4Bk1MrPFR0r/teZSuBX
 Yji3PGK+IL1jbILrn6vl5DKiftOPm+dQ5PqK08Z7MhhtnNXnUjB1BKC+Gq5cz4MQb38zA4Wlw7N
 +bdOFgxWnlcea8Q==
X-Developer-Key: i=ricardo@marliere.net; a=openpgp;
 fpr=030A8E9E424EE3C0655787E1C90B8A7C638658A6

This series is part of an effort to cleanup the users of the driver
core, as can be seen in many recent patches authored by Greg across the
tree (e.g. [1]). Specifically, this series is part of the task of
splitting one of his TODOs [2].

---
[1]: https://lore.kernel.org/lkml/?q=f%3Agregkh%40linuxfoundation.org+s%3A%22make%22+and+s%3A%22const%22
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core.git/commit/?h=bus_cleanup&id=26105f537f0c60eacfeb430abd2e05d7ddcdd8aa

Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Ricardo B. Marliere <ricardo@marliere.net>

---
Ricardo B. Marliere (2):
      xen: pcpu: make xen_pcpu_subsys const
      xen: balloon: make balloon_subsys const

 drivers/xen/pcpu.c        | 2 +-
 drivers/xen/xen-balloon.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
---
base-commit: 2d2db7d40254d5fb53b11ebd703cd1ed0c5de7a1
change-id: 20240203-bus_cleanup-xen-da95a6226a35

Best regards,
-- 
Ricardo B. Marliere <ricardo@marliere.net>



From xen-devel-bounces@lists.xenproject.org Sat Feb 03 19:30:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 19:30:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675436.1050837 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWLi8-0004eQ-0C; Sat, 03 Feb 2024 19:29:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675436.1050837; Sat, 03 Feb 2024 19:29:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWLi7-0004e4-SP; Sat, 03 Feb 2024 19:29:55 +0000
Received: by outflank-mailman (input) for mailman id 675436;
 Sat, 03 Feb 2024 18:53:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WIqD=JM=gmail.com=rbmarliere@srs-se1.protection.inumbo.net>)
 id 1rWL8h-0000aF-1I
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 18:53:19 +0000
Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com
 [2607:f8b0:4864:20::1035])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7c572b2b-c2c5-11ee-98f5-efadbce2ee36;
 Sat, 03 Feb 2024 19:53:15 +0100 (CET)
Received: by mail-pj1-x1035.google.com with SMTP id
 98e67ed59e1d1-295c8b795e2so2334082a91.0
 for <xen-devel@lists.xenproject.org>; Sat, 03 Feb 2024 10:53:15 -0800 (PST)
Received: from mail.marliere.net ([24.199.118.162])
 by smtp.gmail.com with ESMTPSA id
 s15-20020a17090ad48f00b0029636e02c83sm2143181pju.35.2024.02.03.10.53.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 03 Feb 2024 10:53:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c572b2b-c2c5-11ee-98f5-efadbce2ee36
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1706986394; x=1707591194;
        h=cc:to:in-reply-to:references:message-id:content-transfer-encoding
         :mime-version:subject:date:dkim-signature:from:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xhpVXJ+IMs5n5HNm3DBIJeHeLpAjHMNMhg7tgPyvFRk=;
        b=H5XMAFxaQQWmOt17FbGQZka99oeOhDcMJW07MlOBMrFbTqjfr2DkU1e6nX/f0kh8Rb
         zYQF9SPAUR3pY5OtscgkqpS+7aBuCVAXeu0M23UtDnSAr8SWc5WOCYkI7ruui72vBxI7
         S4yMq3WeLFORSPqC8jl8g0HMckUXF01IoizEbmhVNgrdSJINO+ckopijq2Y/sJcbBnK1
         TD6BquEk9HJv51Qrm4+jmdPmiATpeSiJ3500f2VgFK6U1OApHJgCJymIy+kX3ijv+imX
         PAuybluVSogJnrJeMsh5vLYIc5W6/J35XysbLZHPAEv/bQoTPNVK3p8oGRnhpe9lQbQK
         MWYg==
X-Gm-Message-State: AOJu0YzEOHLeDnu5cI7rYBH7zB5wWhG6dg9JcTfQKuk+fnBdojhQWgmc
	X4T32Qo/eUPb1u+5eGByY20EfbYaxD/UGEngY/kWZ6XF19vyElDr
X-Google-Smtp-Source: AGHT+IHNiV9m9PnT9NfWWTLKS5PfvHJH3YAPWwmMb5gceIQF3VaO+pK7Qdw5HKIhnVCxyVRzaLS1KA==
X-Received: by 2002:a17:90a:fd84:b0:295:ab00:e9b9 with SMTP id cx4-20020a17090afd8400b00295ab00e9b9mr8137370pjb.5.1706986393946;
        Sat, 03 Feb 2024 10:53:13 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXU3F//e13WRqDNIvePL9ElC+L1hOrpM9eXVNU4WUyGNILtV/P/hwWPIxHcY/Hvjec4tjvqzgBxTPCeEAppBqwINIMrDzRToVZ0tb5Z/wj6qHZyi4NVR1xqob4Xsz+T70MOVp0cAogc/podyejn4uFBRbE4r4EOOtlGtaKY7e2jZh8oi3f3/uNG4PgqC+bs/xyOW6UAaWOlg2fGjiWqRkI=
From: "Ricardo B. Marliere" <ricardo@marliere.net>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marliere.net;
	s=2023; t=1706986392;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xhpVXJ+IMs5n5HNm3DBIJeHeLpAjHMNMhg7tgPyvFRk=;
	b=HgpwLjLE+5IojjaDPw5lVDBUECMPSExHUMPhE++AL5xcGmYR3TeDz16MbH7zvVuZcuV5RZ
	4CNrmHvySWMapk0sZFaMJeQB6tvCEaW0NIrF7d4w6ZrmiMrby4BgZAvN9+tjz2G++u4r4w
	g1WohaDiXSkJ1PbbGFjguIfXcDkazNeuHAmnsCs4+TAQfGKlZ7bW55I1mz//fZzFDUowaB
	tk1ydtFuveiyVSiNX44BG98XUe9QjlUuP2GlA8LuzUTVUjJx+In/cmZUhiecYr1pC45nAL
	G5A49wyLaW2gu6Xab/A/hPxI/xxjRRLzA32U6yU+UZiHTQntU4nRd9xKIoQPuQ==
Authentication-Results: ORIGINATING;
	auth=pass smtp.auth=ricardo@marliere.net smtp.mailfrom=ricardo@marliere.net
Date: Sat, 03 Feb 2024 15:53:37 -0300
Subject: [PATCH 1/2] xen: pcpu: make xen_pcpu_subsys const
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Message-Id: <20240203-bus_cleanup-xen-v1-1-c2f5fe89ed95@marliere.net>
References: <20240203-bus_cleanup-xen-v1-0-c2f5fe89ed95@marliere.net>
In-Reply-To: <20240203-bus_cleanup-xen-v1-0-c2f5fe89ed95@marliere.net>
To: Juergen Gross <jgross@suse.com>, 
 Stefano Stabellini <sstabellini@kernel.org>, 
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, 
 Greg Kroah-Hartman <gregkh@linuxfoundation.org>, 
 "Ricardo B. Marliere" <ricardo@marliere.net>
X-Developer-Signature: v=1; a=openpgp-sha256; l=846; i=ricardo@marliere.net;
 h=from:subject:message-id; bh=uVbAr2jHnQr3MqspyOfEzRc9k0HTklRcQDcrLwrPX2E=;
 b=owEBbQKS/ZANAwAKAckLinxjhlimAcsmYgBlvouxTV6WmgHhD2cBPYK/XUfsjwDH4uwBWaCje
 UD7Ddr2uBiJAjMEAAEKAB0WIQQDCo6eQk7jwGVXh+HJC4p8Y4ZYpgUCZb6LsQAKCRDJC4p8Y4ZY
 pqGCEACkMVwx/dvxAGqkTeMv3vt97i5rt0L9zEHqvNbgGvxqXH7vIcUCDiuX+ziP0EpEUVdYV7k
 lpXpbodhjS9FFsFGFz+otupP0ETTWlykpYf/fV0R6CVcDHTGeh0KsvlBE7wUod6gN+x3+70cpxV
 EUX/Gy7KTHd/1RUYtnvcWjdNXky9qsWIHZ2gUY29UljGu+pWgXXM0y6N0bWUdI8jOVr/O08mscj
 NWZMFj6l0KmP/xmu/YZtnsvmmG59BTE+1TDPZMwWv8Bc0NBN/Crf+cdpOUHf4Ttp8Pm2kGuarNT
 epDpJ5MtH4kwMI2VrQfnzo7rBSHjPn0o2n1LofwAcPMkcJ8dP6YA9/p3MC+hfScuRVo3pmo44En
 zjZwz2RoD2zzRhDoj1uJqjWwVl6bsxnTOgHp+FRHWnCzFb5QBWGpGYD6RskC7YyrywFqE5cr7cW
 pU4zo1L7Cx+k2hvWPt20+Q6fuF4zike1ItHUAfV1yGVnk5vMDQYUMgYzaDanTJInoosriAsw0mi
 OpefvNjEidNZXxXHfsem3ZpKpUonrVqErVqmKcEgeaT2BKa4DoqQMwyubVpOD8pSA2KOMZyWWpQ
 IBegfAdUOBd2Jv+Tf2UkaszcT1FEawY0F5HjWONwE5U2joStmRWirRS/yA3lTCeB2laJlFHBY9L
 waKgnT3rwMWDgAg==
X-Developer-Key: i=ricardo@marliere.net; a=openpgp;
 fpr=030A8E9E424EE3C0655787E1C90B8A7C638658A6

Now that the driver core can properly handle constant struct bus_type,
move the xen_pcpu_subsys variable to be a constant structure as well,
placing it into read-only memory which can not be modified at runtime.

Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Ricardo B. Marliere <ricardo@marliere.net>
---
 drivers/xen/pcpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/xen/pcpu.c b/drivers/xen/pcpu.c
index 508655273145..c63f317e3df3 100644
--- a/drivers/xen/pcpu.c
+++ b/drivers/xen/pcpu.c
@@ -65,7 +65,7 @@ struct pcpu {
 	uint32_t flags;
 };
 
-static struct bus_type xen_pcpu_subsys = {
+static const struct bus_type xen_pcpu_subsys = {
 	.name = "xen_cpu",
 	.dev_name = "xen_cpu",
 };

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Sat Feb 03 20:07:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 20:07:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675452.1050862 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWMIR-0002bZ-2s; Sat, 03 Feb 2024 20:07:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675452.1050862; Sat, 03 Feb 2024 20:07:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWMIQ-0002bS-Vn; Sat, 03 Feb 2024 20:07:26 +0000
Received: by outflank-mailman (input) for mailman id 675452;
 Sat, 03 Feb 2024 20:07:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWMIP-0002bG-Bs; Sat, 03 Feb 2024 20:07:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWMIP-0001Zt-A7; Sat, 03 Feb 2024 20:07:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWMIO-0000Wa-U4; Sat, 03 Feb 2024 20:07:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWMIO-0001O2-Ta; Sat, 03 Feb 2024 20:07:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LrCysuyGYTAGsWyVxdIADq6yhKZyd7DQDsvC5PDG3so=; b=6anuZIunNNT8iWE0HyM75pYZLw
	1YYCXd14oJN/WIkJiCtNcGJL8nHUqUAlvnaLtNjF0PyoNqN7wSMGsRaRceuAP8opJJMrkosBSinNy
	v9IV9KHANC0nJPOtdenQfXS/pNIJuTFDTX3vuQtmzuux0csNAM1YSkQYGsghtGsEGoV0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184579-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184579: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=a1b98c8f845c165572937149a46e12ca36960617
X-Osstest-Versions-That:
    ovmf=927ea1364d047d54cb67e0f231644f295c032944
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 03 Feb 2024 20:07:24 +0000

flight 184579 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184579/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 a1b98c8f845c165572937149a46e12ca36960617
baseline version:
 ovmf                 927ea1364d047d54cb67e0f231644f295c032944

Last test of basis   184561  2024-02-02 02:55:40 Z    1 days
Testing same since   184579  2024-02-03 17:11:33 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michael Kubacki <michael.kubacki@microsoft.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   927ea1364d..a1b98c8f84  a1b98c8f845c165572937149a46e12ca36960617 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 21:24:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 21:24:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675458.1050871 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWNUN-00047Q-Ko; Sat, 03 Feb 2024 21:23:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675458.1050871; Sat, 03 Feb 2024 21:23:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWNUN-00047J-I3; Sat, 03 Feb 2024 21:23:51 +0000
Received: by outflank-mailman (input) for mailman id 675458;
 Sat, 03 Feb 2024 21:23:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DGSX=JM=linuxfoundation.org=gregkh@srs-se1.protection.inumbo.net>)
 id 1rWNUL-00047C-LF
 for xen-devel@lists.xenproject.org; Sat, 03 Feb 2024 21:23:49 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 82bb4c86-c2da-11ee-98f5-efadbce2ee36;
 Sat, 03 Feb 2024 22:23:46 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 5E44B60BDE;
 Sat,  3 Feb 2024 21:23:44 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id F0292C433C7;
 Sat,  3 Feb 2024 21:23:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82bb4c86-c2da-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org;
	s=korg; t=1706995424;
	bh=2cUK20HpkF3Qs4l/910Mfk2HuHRlZed/HuN6ZaSpsas=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=0zT/dmAGOhDbo3w43Gct1tLu57+LlgLX3XUWVqC6PWSzPvrxVSZHuEL2aVbj6HY7R
	 Gk2B6RlHxh9fIyJKccTMvL/ATtanY+uTuyVh4MHCMLcVAdrCAwNth3otAnQlgM7R/Z
	 odVM+yiI8gwrS2cguP889Ixo+vlsXtfRO1/D/uBQ=
Date: Sat, 3 Feb 2024 13:23:43 -0800
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: "Ricardo B. Marliere" <ricardo@marliere.net>
Cc: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 0/2] drivers: xen: struct bus_type cleanup
Message-ID: <2024020333-unwatched-provolone-dc3c@gregkh>
References: <20240203-bus_cleanup-xen-v1-0-c2f5fe89ed95@marliere.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240203-bus_cleanup-xen-v1-0-c2f5fe89ed95@marliere.net>

On Sat, Feb 03, 2024 at 03:53:36PM -0300, Ricardo B. Marliere wrote:
> This series is part of an effort to cleanup the users of the driver
> core, as can be seen in many recent patches authored by Greg across the
> tree (e.g. [1]). Specifically, this series is part of the task of
> splitting one of his TODOs [2].
> 
> ---
> [1]: https://lore.kernel.org/lkml/?q=f%3Agregkh%40linuxfoundation.org+s%3A%22make%22+and+s%3A%22const%22
> [2]: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core.git/commit/?h=bus_cleanup&id=26105f537f0c60eacfeb430abd2e05d7ddcdd8aa
> 
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Signed-off-by: Ricardo B. Marliere <ricardo@marliere.net>

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 21:57:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 21:57:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675467.1050882 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWO0h-0008Ue-5n; Sat, 03 Feb 2024 21:57:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675467.1050882; Sat, 03 Feb 2024 21:57:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWO0h-0008UX-2o; Sat, 03 Feb 2024 21:57:15 +0000
Received: by outflank-mailman (input) for mailman id 675467;
 Sat, 03 Feb 2024 21:57:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWO0f-0008UN-VX; Sat, 03 Feb 2024 21:57:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWO0f-000603-QW; Sat, 03 Feb 2024 21:57:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWO0f-0006Lu-B1; Sat, 03 Feb 2024 21:57:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWO0f-0000bv-Aa; Sat, 03 Feb 2024 21:57:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=EWAkrE8kmxlb6Cmg4y5Fqlz0q9lxzR/LSzzf1KhzJwY=; b=r8Db3Q66Oq4Iy3S1y/qyG2oknX
	cDJuMPus3LvvaO+WL4KioSIq4it7L5LKAB4HT05bhY5qtp5QlwfooRAs8L84Ow/dX4LnieqENYEzL
	n2BIu+pmnmyyzdDuo9zOd0nImZ2Fai9ULA4w8yDmnNAN8S/bRJwaNKx+eSsKvUtHj4tQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184573-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184573: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=442061583e9dc0e4e3bf314275979051345a4a93
X-Osstest-Versions-That:
    libvirt=5e95cedbb2c5d5af049110369c627f90bbdb0799
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 03 Feb 2024 21:57:13 +0000

flight 184573 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184573/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184562
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184562
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184562
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              442061583e9dc0e4e3bf314275979051345a4a93
baseline version:
 libvirt              5e95cedbb2c5d5af049110369c627f90bbdb0799

Last test of basis   184562  2024-02-02 04:20:30 Z    1 days
Testing same since   184573  2024-02-03 04:24:41 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michal Privoznik <mprivozn@redhat.com>
  Peter Krempa <pkrempa@redhat.com>
  Praveen K Paladugu <prapal@linux.microsoft.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   5e95cedbb2..442061583e  442061583e9dc0e4e3bf314275979051345a4a93 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 22:18:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 22:18:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675473.1050892 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWOLN-0003Nl-U8; Sat, 03 Feb 2024 22:18:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675473.1050892; Sat, 03 Feb 2024 22:18:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWOLN-0003Ne-Pk; Sat, 03 Feb 2024 22:18:37 +0000
Received: by outflank-mailman (input) for mailman id 675473;
 Sat, 03 Feb 2024 22:18:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWOLM-0003NU-OY; Sat, 03 Feb 2024 22:18:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWOLM-0006gF-M1; Sat, 03 Feb 2024 22:18:36 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWOLM-0007ch-Ac; Sat, 03 Feb 2024 22:18:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWOLM-00077T-AJ; Sat, 03 Feb 2024 22:18:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=t8s9QfXBdBFvZfHweumfVuH0GaRcTUH2ENv/OWDzrCc=; b=jxv5/XDOQA8qCkAuVMA2HvtsOb
	jx4xGQIXy1ZY2CR0vuxWRCRFyiqKEMaBUjEINH7FnVG5Aw/RMFfn8Eg2aOTwcH8+lW1AqKJu4nPAY
	XS67vuKO/mR6AuvOBl8EdDJ1VOkIaY4ucwCsTvgm+j+G+rIsRCl2Xas0cWX2gutUfHMg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184570-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-6.1 test] 184570: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-6.1:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-6.1:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=e5c3b988b827c76f52d0f62343e863b9133a0cd2
X-Osstest-Versions-That:
    linux=8fd7f44624538675abadc73f5a44e95016964d22
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 03 Feb 2024 22:18:36 +0000

flight 184570 linux-6.1 real [real]
flight 184581 linux-6.1 real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184570/
http://logs.test-lab.xenproject.org/osstest/logs/184581/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184581-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184549
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184549
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184549
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184549
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184549
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184549
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184549
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184549
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 linux                e5c3b988b827c76f52d0f62343e863b9133a0cd2
baseline version:
 linux                8fd7f44624538675abadc73f5a44e95016964d22

Last test of basis   184549  2024-02-01 11:13:03 Z    2 days
Testing same since   184563  2024-02-02 05:26:08 Z    1 days    2 attempts

------------------------------------------------------------
486 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   8fd7f4462453..e5c3b988b827  e5c3b988b827c76f52d0f62343e863b9133a0cd2 -> tested/linux-6.1


From xen-devel-bounces@lists.xenproject.org Sat Feb 03 22:20:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 03 Feb 2024 22:20:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675478.1050901 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWONe-0004r9-E4; Sat, 03 Feb 2024 22:20:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675478.1050901; Sat, 03 Feb 2024 22:20:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWONe-0004r2-Ay; Sat, 03 Feb 2024 22:20:58 +0000
Received: by outflank-mailman (input) for mailman id 675478;
 Sat, 03 Feb 2024 22:20:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWONd-0004qs-7q; Sat, 03 Feb 2024 22:20:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWONd-0006jU-71; Sat, 03 Feb 2024 22:20:57 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWONc-0007mG-Nr; Sat, 03 Feb 2024 22:20:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWONc-0007RE-NV; Sat, 03 Feb 2024 22:20:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LwUIW0bnnfgyNXSbHtx4jL3Tc81JLnHiFdliE0TuAeE=; b=beQ/E9WEl2UTx38TiYMFSK0jZS
	XYIXUtApRAHnTrnZhMTO0Po2GlBEYStH4j51HJzC7By8hkHezuoyphMsONKyR1CT43VLHTq5ijfGS
	kmLR0DOEPmLpw2a3sw8vfsaSx1xcSa5yIgohd8yy7SC4H4dHNpQHy5t9/umIbOOy0oy0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184580-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184580: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=cd6f2152237713d12723a55aa258c7ae91577dff
X-Osstest-Versions-That:
    ovmf=a1b98c8f845c165572937149a46e12ca36960617
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 03 Feb 2024 22:20:56 +0000

flight 184580 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184580/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 cd6f2152237713d12723a55aa258c7ae91577dff
baseline version:
 ovmf                 a1b98c8f845c165572937149a46e12ca36960617

Last test of basis   184579  2024-02-03 17:11:33 Z    0 days
Testing same since   184580  2024-02-03 20:14:33 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Tom Lendacky <thomas.lendacky@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   a1b98c8f84..cd6f215223  cd6f2152237713d12723a55aa258c7ae91577dff -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sun Feb 04 03:41:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 04 Feb 2024 03:41:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675489.1050912 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWTNZ-0007tT-7f; Sun, 04 Feb 2024 03:41:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675489.1050912; Sun, 04 Feb 2024 03:41:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWTNZ-0007tA-2A; Sun, 04 Feb 2024 03:41:13 +0000
Received: by outflank-mailman (input) for mailman id 675489;
 Sun, 04 Feb 2024 03:41:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWTNX-0007sy-Vg; Sun, 04 Feb 2024 03:41:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWTNX-0003iG-RN; Sun, 04 Feb 2024 03:41:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWTNX-00079f-A7; Sun, 04 Feb 2024 03:41:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWTNX-00009B-9g; Sun, 04 Feb 2024 03:41:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=BVCkm32p8Nn2OwrgWo9LVibIyA5qQEibhVRwk/A46/Y=; b=F09obJZXuOKf052Kr0wzxgaPmU
	M3hOFzCWRwNdQVwB4CWcs+StpxDIvSL9ZfqwqVBGcUu07N0/TzqnodpiGahAlTwwaBvP7gOTvxvRE
	3nqVoXUK13zsqj38SGwBcdqosZWbaZsBsCqiRO3eWCL66U7rr8l6ODKo4X4/UM8CQXVE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184576-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184576: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-vhd:guest-start:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=56897d51886fa7e9f034ff26128eb09f1b811594
X-Osstest-Versions-That:
    linux=021533194476035883300d60fbb3136426ac8ea5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 04 Feb 2024 03:41:11 +0000

flight 184576 linux-linus real [real]
flight 184582 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184576/
http://logs.test-lab.xenproject.org/osstest/logs/184582/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-vhd      13 guest-start         fail pass in 184582-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-vhd     14 migrate-support-check fail in 184582 never pass
 test-armhf-armhf-xl-vhd 15 saverestore-support-check fail in 184582 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184566
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184566
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184566
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184566
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184566
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184566
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184566
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184566
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 linux                56897d51886fa7e9f034ff26128eb09f1b811594
baseline version:
 linux                021533194476035883300d60fbb3136426ac8ea5

Last test of basis   184566  2024-02-02 15:26:30 Z    1 days
Testing same since   184576  2024-02-03 12:02:13 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alan Adamson <alan.adamson@oracle.com>
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Tsoy <alexander@tsoy.me>
  Andreas Gruenbacher <agruenba@redhat.com>
  Andy Chi <andy.chi@canonical.com>
  Ard Biesheuvel <ardb@kernel.org>
  Badal Nilawar <badal.nilawar@intel.com>
  Bagas Sanjaya <bagasdotme@gmail.com>
  Bjorn Helgaas <bhelgaas@google.com>
  Brenton Simpson <appsforartists@google.com>
  Caleb Sander <csander@purestorage.com>
  Chaitanya Kulkarni <kch@nvidia.com>
  Charlene Liu <charlene.liu@amd.com>
  Chen-Yu Tsai <wens@csie.org>
  Chhayly Leang <clw.leang@gmail.com>
  Christian König <christian.koenig@amd.com>
  Conor Dooley <conor.dooley@microchip.com>
  Daniel Bristot de Oliveira <bristot@kernel.org>
  Daniel Miess <daniel.miess@amd.com>
  Daniel Wagner <dwagner@suse.de>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Dave Airlie <airlied@redhat.com>
  David McFarland <corngood@gmail.com>
  David Senoner <seda18@rolmail.net>
  diogo.ivo@tecnico.ulisboa.pt
  Dmitry Osipenko <dmitry.osipenko@collabora.com>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Dmytro Laktyushkin <dmytro.laktyushkin@amd.com>
  Edson Juliano Drosdeck <edson.drosdeck@gmail.com>
  Fangzhi Zuo <jerry.zuo@amd.com>
  Fei Shao <fshao@chromium.org>
  Friedrich Vock <friedrich.vock@gmx.de>
  Guenter Roeck <linux@roeck-us.net>
  Guixin Liu <kanie@linux.alibaba.com>
  Hannes Reinecke <hare@suse.de>
  Hans de Goede <hdegoede@redhat.com>
  Harshit Mogalapalli <harshit.m.mogalapalli@oracle.com>
  Hongyu Jin <hongyu.jin@unisoc.com>
  Israel Rukshin <israelr@nvidia.com>
  Ivan Orlov <ivan.orlov0322@gmail.com>
  Jacob Siverskog <jacob@teenage.engineering>
  Jani Nikula <jani.nikula@intel.com>
  Jason Gunthorpe <jgg@nvidia.com>
  Javier Carrasco <javier.carrasco.cruz@gmail.com>
  Jay Cornwall <jay.cornwall@amd.com>
  Jens Axboe <axboe@kernel.dk>
  Joerg Roedel <jroedel@suse.de>
  Johan Hovold <johan+linaro@kernel.org>
  José Relvas <josemonsantorelvas@gmail.com>
  José Roberto de Souza <jose.souza@intel.com>
  Julian Sikorski <belegdol+github@gmail.com>
  Kailang Yang <kailang@realtek.com>
  Keith Busch <kbusch@kernel.org>
  Kenzo Gomez <kenzo.sgomez@gmail.com>
  Kevin Brodsky <kevin.brodsky@arm.com>
  Konstantin Aladyshev <aladyshev22@gmail.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Lang Yu <Lang.Yu@amd.com>
  Le Ma <le.ma@amd.com>
  Lennert Buytenhek <kernel@wantstofly.org>
  Li RongQing <lirongqing@baidu.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lorenzo Pieralisi <lpieralisi@kernel.org>
  Lucas De Marchi <lucas.demarchi@intel.com>
  Luka Guzenko <l.guzenko@web.de>
  Ma Jun <Jun.Ma2@amd.com>
  Maarten van der Schrieck
  Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
  Marian Postevca <posteuca@mutex.one>
  Mario Limonciello <mario.limonciello@amd.com>
  Mark Brown <broonie@kernel.org>
  Matt Roper <matthew.d.roper@intel.com>
  Matthew Brost <matthew.brost@intel.com>
  Max Gurtovoy <mgurtovoy@nvidia.com>
  Maxime Ripard <mripard@redhat.com>
  Mike Snitzer <snitzer@kernel.org>
  Mikulas Patocka <mpatocka@redhat.com>
  Mukul Joshi <mukul.joshi@amd.com>
  Nicholas Susanto <nicholas.susanto@amd.com>
  Niklas Cassel <cassel@kernel.org>
  Nitin U. Yewale <nyewale@redhat.com>
  Ovidiu Panait <ovidiu.panait@windriver.com>
  Paul Menzel <pmenzel@molgen.mpg.de>
  Philip Yang <Philip.Yang@amd.com>
  Richard Fitzgerald <rf@opensource.cirrus.com>
  Rodrigo Vivi <rodrigo.vivi@intel.com>
  Rui Salvaterra <rsalvaterra@gmail.com>
  Saleemkhan Jamadar <saleemkhan.jamadar@amd.com>
  Sami Tolvanen <samitolvanen@google.com>
  Sean Young <sean@mess.org>
  Sebastian Ott <sebott@redhat.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  Shivaprasad G Bhat <sbhat@linux.ibm.com>
  Sohaib Nadeem <sohaib.nadeem@amd.com>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Sumit Semwal <sumit.semwal@linaro.org>
  Szilard Fabian <szfabian@bluemarch.art>
  T.J. Mercier <tjmercier@google.com>
  Takashi Iwai <tiwai@suse.de>
  Techno Mooney <techno.mooney@gmail.com>
  Thomas Hellstrom <thomas.hellstrom@linux.intel.com>
  Thomas Hellström <thomas.hellstrom@linux.intel.com>
  Tom Chung <chiahsuan.chung@amd.com>
  Venkata Prasad Potturu <venkataprasad.potturu@amd.com>
  Vincent Donnefort <vdonnefort@google.com>
  Vitaly Prosyak <vitaly.prosyak@amd.com>
  Vitaly Rodionov <vitalyr@opensource.cirrus.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Wenjing Liu <wenjing.liu@amd.com>
  Werner Sembach <wse@tuxedocomputers.com>
  Will Deacon <will@kernel.org>
  Yibin Ding <yibin.ding@unisoc.com>
  Yifan Zhang <yifan1.zhang@amd.com>
  Zhenyu Zhang <zhenyzha@redhat.com>
  Zhu Ning <zhuning0077@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   021533194476..56897d51886f  56897d51886fa7e9f034ff26128eb09f1b811594 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun Feb 04 04:54:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 04 Feb 2024 04:54:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675497.1050921 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWUW7-0007FF-GU; Sun, 04 Feb 2024 04:54:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675497.1050921; Sun, 04 Feb 2024 04:54:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWUW7-0007F8-Dg; Sun, 04 Feb 2024 04:54:07 +0000
Received: by outflank-mailman (input) for mailman id 675497;
 Sun, 04 Feb 2024 04:54:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWUW5-0007Ey-CJ; Sun, 04 Feb 2024 04:54:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWUW5-0006OT-Au; Sun, 04 Feb 2024 04:54:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWUW1-0000Tu-U9; Sun, 04 Feb 2024 04:54:02 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWUW1-0000BE-Qz; Sun, 04 Feb 2024 04:54:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ECLmxQTO6r8M7BZYAFkPA65ltEgBpiLD9M+NPR2wL5s=; b=VFGQ7c8dUvaVcpxW3RTnCrw1+t
	vO6np2+c69S5nWpWY7xuq9Xh1gOE3Us14Aipu7Y32tbo+B/LRnwjb5KlFkn43RrjpOStVwOo8r3/0
	2wiA2GKBK/FsXD/NUTkXbTMi6Y1LN84Evp+e5R3HSfYcdkVRnNQRnJaGIXlAuWjrEezs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184583-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184583: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=141dcaed6cc930d83a4f95cb51ebc22f51fcc32c
X-Osstest-Versions-That:
    ovmf=cd6f2152237713d12723a55aa258c7ae91577dff
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 04 Feb 2024 04:54:01 +0000

flight 184583 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184583/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 141dcaed6cc930d83a4f95cb51ebc22f51fcc32c
baseline version:
 ovmf                 cd6f2152237713d12723a55aa258c7ae91577dff

Last test of basis   184580  2024-02-03 20:14:33 Z    0 days
Testing same since   184583  2024-02-04 03:11:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dhaval Sharma <dhaval@rivosinc.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   cd6f215223..141dcaed6c  141dcaed6cc930d83a4f95cb51ebc22f51fcc32c -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sun Feb 04 09:40:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 04 Feb 2024 09:40:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675523.1050932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWYzK-0003xT-5g; Sun, 04 Feb 2024 09:40:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675523.1050932; Sun, 04 Feb 2024 09:40:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWYzK-0003xM-31; Sun, 04 Feb 2024 09:40:34 +0000
Received: by outflank-mailman (input) for mailman id 675523;
 Sun, 04 Feb 2024 09:40:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=uT9t=JN=nxp.com=peng.fan@srs-se1.protection.inumbo.net>)
 id 1rWYzI-0003xF-WC
 for xen-devel@lists.xenproject.org; Sun, 04 Feb 2024 09:40:33 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2611::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6fd552ae-c341-11ee-8a45-1f161083a0e0;
 Sun, 04 Feb 2024 10:40:31 +0100 (CET)
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com (2603:10a6:10:358::11)
 by AS8PR04MB7605.eurprd04.prod.outlook.com (2603:10a6:20b:292::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.23; Sun, 4 Feb
 2024 09:40:28 +0000
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::c499:8cef:9bb1:ced6]) by DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::c499:8cef:9bb1:ced6%3]) with mapi id 15.20.7249.032; Sun, 4 Feb 2024
 09:40:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fd552ae-c341-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KKs2ETrvV5WPzJExL7tKmM/Rq3dW5HIoJIZj0bZYhJi5W/PU63y4UC92/GFCgNvI79AvtTaFvtxzzOarymqqHn0yOW+Wus539N4MVwL/znNb7hmJni/kEpMm7NktosOKz8kqQZEPsj6w5x1b+mUuuaodl6V866sLdwFmaxd3yV1darUSPKa184b5fhtYoJTsorL+q0VzFE1z9Xem5+F74lEZLyguZKhbScEilQsQDCbnlzGS+lyOQVH01k6Qe7jcIEk8kD6isoFRFv5bmWLlocyPxU20ODg8MiHY0VSuLdeHCkfq6Gd1yqQSOgr1Q4y5ANJpEOn8u39o3fADIg0qnA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=55rW2s57Br8Fac3uaZ9y6xYDTKErO+HNI1Kc8z05vEs=;
 b=mehypZW/yq40QIT+Hwz8F4szy/3xmZzlFG2SbcZl0KYOhsZY6Zw2IOI1WCBHG04uaAMWUPW+zpH3oUUBwEr5xuKjqMPNiG32pLX6/XaKxmI3ovqS7BgFMml5E1h7P+MzcpM1OkXL4Ap0EmgkH0EHrso46SdriA7//4Axr0YaWOc4vJiKsQWSAKAkkhwJa2H4TWbIDmazEB1QxzqKLQWpnbwRKwrLMxURmiSS3OdvdkhuNvWpqP1td94yd0Bg3HV4K2dV86rz6PmoCpcrwRzYINZDGhVQOtBi52eUsmqPrM/mTphtfOnwTkKC1qVvyTIbsxx8jd3Nd3FzH+Dt2OXRSw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass
 header.d=nxp.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=55rW2s57Br8Fac3uaZ9y6xYDTKErO+HNI1Kc8z05vEs=;
 b=pETYcx0Sy42dq+Lix2cJamMXs7x5owucsZt0fJdtyLBpmiW8Zrzk6WqQNE7383/qxZgBb/lD8U5xjzn0MXqm+w7L/dsJZGTsevmFdxtEBskKIGmF0ZStCVISXL9ONcS/nliC3APDE+/9KRn4qvWldy3Kp8k1QsViLHn3Ez3ZYos=
From: Peng Fan <peng.fan@nxp.com>
To: Julien Grall <julien@xen.org>, John Ernberg <john.ernberg@actia.se>,
	Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
CC: Jonas Blixt <jonas.blixt@actia.se>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: RE: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Topic: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Index:
 AQHaVDuqVQ/kgxLZgUK/m8/hE0ez5bDz198AgAA1LICAAVx4AIAAQjaAgAEdtQCAAynKwA==
Date: Sun, 4 Feb 2024 09:40:28 +0000
Message-ID:
 <DU0PR04MB94170CB1F77E3D594A6D0E9488402@DU0PR04MB9417.eurprd04.prod.outlook.com>
References: <20240131114952.305805-1-john.ernberg@actia.se>
 <20240131114952.305805-2-john.ernberg@actia.se>
 <a2f726f5-df4c-4d15-90af-7d59c0f1f513@xen.org>
 <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se>
 <167f0c7a-e037-446c-82f8-2584e35a7af1@xen.org>
 <a265ea2d-9b5f-4726-9395-448b1b669839@actia.se>
 <012b5f83-2f9b-4477-965e-07b8506c0052@xen.org>
In-Reply-To: <012b5f83-2f9b-4477-965e-07b8506c0052@xen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=nxp.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DU0PR04MB9417:EE_|AS8PR04MB7605:EE_
x-ms-office365-filtering-correlation-id: e8a18df2-bc08-4a2b-ea61-08dc256552ab
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 eUFoLlSHnoiwHoKcuS4GHRU70DOZq7UTPTTwRTmOYpbmh6EEJv3MzusCIbG7jt7/O+9exiqceocM6KW4H9RbebMII3m6hZXjJKVTlbu6gEmgy6HROyziFMK8gM+pSW41MQUE2KRp3G0ScZuNLDuFTJoagiC0GBcUoGZKw12XbN2kZXLaKAhG3QICgyXqrljt82+RWPhojZQ7Ix+DPEUPUATBANTP66ZkoWyXy1SXNohCHFrOE7JKwZ0pHymIfl0LzzaTBzUp9Kub32XdBZSGwqW3JeQTh+VsbU07MxEU7hTZEbmg9VmhEdcVOID+LwtVCkweqFoR50reFxsBvabuW7Kc3lKBhV43NKYwrKmY8CX3yl+U3G2+CAj5GjCsH6vbgAO0r770b+nIi3DRol2kGbcRoRAbtj/BcWIILvijQH7S44qNMRUq81IfHxGwmbC6TqomJCeJfQFp5Q8jhgdEQN3Q5VS3yf0MjQyVVV4+BZUDcuirx/iBEf6JC9MV919huk/0FkRtlwV1XbcKgAlk9fKt1gLXEYYfnpxm6E4cD/OkhzFIrpbWNdRDe51dstMZC0pERF2ZpUy/gOdXGQjnCb+KbTl3hGfA4HePP/QidYAKamzuysK9e175HBBllbXZsuTBInxk5I9bYvCYkhc+eg==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU0PR04MB9417.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(366004)(136003)(376002)(396003)(346002)(230922051799003)(186009)(64100799003)(1800799012)(451199024)(66556008)(7696005)(66446008)(6506007)(66476007)(478600001)(53546011)(316002)(66946007)(54906003)(9686003)(76116006)(64756008)(8676002)(110136005)(8936002)(26005)(71200400001)(4326008)(52536014)(5660300002)(44832011)(2906002)(83380400001)(41300700001)(38070700009)(122000001)(33656002)(38100700002)(86362001)(55016003)(32563001);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?TVVFQlFnVFpDSnBoZVlVUnlqSk4wSWlzZHBhRW0wUERTYTJFYWcvZ0tUM2Zu?=
 =?utf-8?B?OFlxUFVsOFUvSktLMFBQcnMwQndOZHB1R3RBa1pZdzJXbTIvSmJFTUNtV0FI?=
 =?utf-8?B?TVhoTGxZNFdxYmVpdnIrcW5EN2xXNmF6MSthL3o3RyttdmtEUlBEcFBRVDBt?=
 =?utf-8?B?TDNQaVlNZ3FkZTVGSlhhR0VjMFBJdllGWjFMY2IvdyswbUhQbTQranpxY25H?=
 =?utf-8?B?S0xBSEprelduckpGUWFTcGxkdE5TSHZsaGdibmJ3Tm5kR1FESWp1d1BZYXA1?=
 =?utf-8?B?WmpPbTlOSVROenhUQVdDY242T1NYUEE3akZyZGZZQkpBNWRINWZ1aFdNMEJH?=
 =?utf-8?B?SXc5YkpTWHY1cUFaemtLSHd4WmZWUnlMa3dnUzNlVFZKMW1MbnZGMTVOWi9k?=
 =?utf-8?B?bU55VWhFa2lyc0ZQWWZpQWc2NVVEWkhzS21aRlVudWgya2xsdjhheFhPZVpF?=
 =?utf-8?B?T1ZKaW50SG12S2NKdVBRL0VJZWU2Tm5GcitKZ0N5RDR4eUtnUU8yeFg4WFZM?=
 =?utf-8?B?TUpRWDFWVmk1TDJwbXFaTTdhbXo2Zm5RR1k3UUVrNHV0RThWSVQwQ1AvZjky?=
 =?utf-8?B?SVZDMVIrZTF1dXBKaDJRNDZsTSt3ZVlXZHVUUGNGZUc5b1NUS09xSzBxVHZj?=
 =?utf-8?B?WTJoUGdsOThVOHVwNzNKUEMxeXBkQ2xJb2hmNnExQ3pTajFTRHhMWU85eGVJ?=
 =?utf-8?B?Yy9RNGk0TU1kU0ZQUTFCajJkbS9IM1hYYUtZK1l5TDdRY21xRkNpMFhSR3NU?=
 =?utf-8?B?S0tMUjNXU3NVRWQ0T0pIL3RiTXV5VjN5M0NlNmF0OHhMeTBmZ2dMMjljYTk3?=
 =?utf-8?B?RVFFOG1LTlJvdnd6Y2ZFbFY1TktHTTVrQmhrdW1IMHYxL0ZIaWNMSEt4aC9G?=
 =?utf-8?B?UU4yM0pqcGZ2UTNDcFozU3NCNzhkSnFlZ3FIQlFyRzcweWlHNzVpckxqSkZG?=
 =?utf-8?B?Y25xL2hFVHNMRStBTlBnSnJzVzFrenFtMUI1bFUrYS9GVk5UWmJ1NytLZTFD?=
 =?utf-8?B?Q2lSbXlNYW1ac2pDOUMxME0relpZMDU5UDYwQ0JLT0Frc0RKSHpDU0p3c1BM?=
 =?utf-8?B?cllIZWh4aVFGdm9RNTlnYTc4bWFkTnc4Q0diR1ppSFhxK3YxeHk3Mnh3eFkv?=
 =?utf-8?B?dXhhcVJFTTZueUl5eHR4WnVJSWlKTXRIczNCK1pGOGhtOFJVZXNVYTB5TFlX?=
 =?utf-8?B?bkdKOGxlYzU5SW5pV3J5bm5nQU5uNXMyNERpdXp2bGRMRjR3UVFRdmluQTYv?=
 =?utf-8?B?QUpubS8wcmV3dkhKcGRrUUJLaG0yQ1czSGVlMmVTUVYxR2o4TVAyYlpzRm5H?=
 =?utf-8?B?OWE1NnlYWVQ4YUZyMWhLYXRpdE54U0FZSEZiTzM5VVBXdVYyK1FFWFdSbXkr?=
 =?utf-8?B?M2V3U280aGlwYWV5djRKemh1UXU4VGdpa2xTalAzVzlkMGhJOE9Ma3liM3U2?=
 =?utf-8?B?bjRsSU1pd0dNS081eXR1K1RoT1B2cExQNi93a01uVUozQ092WWhNL3B1aHVB?=
 =?utf-8?B?b3JOdmI3YkVWelpzbTlDa2pCOFlDN2N5QzlpNXVuSEFPd0tsU1YzS0grTU5r?=
 =?utf-8?B?Z3Jqb05xTWJqWXh4T1V4R1orR3NLWUF0MzA0Ym1TQlY4bjRpYkRSS2x4UWFI?=
 =?utf-8?B?QUlhclFFTEhidGNXV3p1b0tHOUdVc2Z1R2dwSk5UL1E5blBlZzNXVU9rTVVo?=
 =?utf-8?B?dGQxQlkxNHlNcm0ybTNYK3Z3b3E0QkpsUzEzQko3UWY4b0d0TSt3Y2JwQ2JF?=
 =?utf-8?B?bnN1eHZub0dUdTZJTkhra0dBY3RsMzF1MHdxN0ZpNUxERFlxcDZxcElacVE3?=
 =?utf-8?B?Qm92TE1NT05hbDBZVXc4T0lmSzdzUlZob3R0SjF2cC8yemU5UGNiYWdnRUNP?=
 =?utf-8?B?RVRFNWhkbU15Z213VWRIUXZTNXV0KzRIV3BkZW93ZDBlVDZrV2xxNE9IbTdr?=
 =?utf-8?B?NWQzeHQyaUhoTXhoaEF4SkxRc0hpckVTZUJjamxXRCs5OUpjeDN4S2RuTUQ5?=
 =?utf-8?B?QVhjdWRRcXdqSUNScU5aVFNlS0EzTWtPd1Z4U21UNjY3aFltUjNtMUFOeWR3?=
 =?utf-8?B?Rkg0MGFwOFcyY3RIUzlvalE5cmtrTDBTZUVDbHVHM2RmNkFSZzB6ZXlLNDRu?=
 =?utf-8?Q?PMPc=3D?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DU0PR04MB9417.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e8a18df2-bc08-4a2b-ea61-08dc256552ab
X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Feb 2024 09:40:28.8188
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: dq6dNCrUvA0tTHExJP6mocgEAwM9hQYMoB2PB+t5suXGodapUEBYA95/4N7qltWl08TxsAdYnZHlkzrAvt2UlQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB7605

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogSnVsaWVuIEdyYWxsIDxq
dWxpZW5AeGVuLm9yZz4NCj4gU2VudDogMjAyNOW5tDLmnIgy5pelIDE3OjIwDQo+IFRvOiBKb2hu
IEVybmJlcmcgPGpvaG4uZXJuYmVyZ0BhY3RpYS5zZT47IFN0ZWZhbm8gU3RhYmVsbGluaQ0KPiA8
c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IEJlcnRyYW5kIE1hcnF1aXMgPGJlcnRyYW5kLm1hcnF1
aXNAYXJtLmNvbT47DQo+IE1pY2hhbCBPcnplbCA8bWljaGFsLm9yemVsQGFtZC5jb20+OyBWb2xv
ZHlteXIgQmFiY2h1aw0KPiA8Vm9sb2R5bXlyX0JhYmNodWtAZXBhbS5jb20+OyBQZW5nIEZhbiA8
cGVuZy5mYW5AbnhwLmNvbT4NCj4gQ2M6IEpvbmFzIEJsaXh0IDxqb25hcy5ibGl4dEBhY3RpYS5z
ZT47IHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZw0KPiBTdWJqZWN0OiBSZTogW1BBVENI
IDEvMl0geGVuL2FybTogQWRkIGlteDhxe20seH0gcGxhdGZvcm0gZ2x1ZQ0KPiANCj4gT24gMDEv
MDIvMjAyNCAxNjoxNywgSm9obiBFcm5iZXJnIHdyb3RlOg0KPiA+IE9uIDIvMS8yNCAxMzoyMCwg
SnVsaWVuIEdyYWxsIHdyb3RlOg0KPiA+Pg0KPiA+Pg0KPiA+PiBPbiAzMS8wMS8yMDI0IDE1OjMy
LCBKb2huIEVybmJlcmcgd3JvdGU6DQo+ID4+PiBIaSBKdWxpZW4sDQo+ID4+DQo+ID4+IEhpIEpv
aG4sDQo+ID4+DQo+ID4+PiBPbiAxLzMxLzI0IDEzOjIyLCBKdWxpZW4gR3JhbGwgd3JvdGU6DQo+
ID4+Pj4gSGksDQo+ID4+Pj4NCj4gPj4+PiBPbiAzMS8wMS8yMDI0IDExOjUwLCBKb2huIEVybmJl
cmcgd3JvdGU6DQo+ID4+Pj4+IFdoZW4gdXNpbmcgTGludXggZm9yIGRvbTAgdGhlcmUgYXJlIGEg
YnVuY2ggb2YgZHJpdmVycyB0aGF0IG5lZWQNCj4gPj4+Pj4gdG8gZG8gU01DIFNJUCBjYWxscyBp
bnRvIHRoZSBQU0NJIHByb3ZpZGVyIHRvIGVuYWJsZSBjZXJ0YWluDQo+ID4+Pj4+IGhhcmR3YXJl
IGJpdHMgbGlrZSB0aGUgd2F0Y2hkb2cuDQo+ID4+Pj4NCj4gPj4+PiBEbyB5b3Uga25vdyB3aGlj
aCBwcm90b2NvbCB0aGlzIGlzIHVuZGVyIHRoZSBob29kLiBJcyB0aGlzIFNDTUk/DQo+ID4+Pg0K
PiA+Pj4gSSB0aGluayBJIGNvbmZ1c2VkIG15c2VsZiBoZXJlIHdoZW4gSSB3cm90ZSB0aGUgY29t
bWl0IGxvZy4NCj4gPj4+DQo+ID4+PiBUaGUgRUwzIGNvZGUgaW4gb3VyIGNhc2UgaXMgQVRGLCBh
bmQgaXQgZG9lcyBub3QgYXBwZWFyIHRvIGJlIFNDTUksDQo+ID4+PiBub3IgUFNDSS4gVGhlIHJl
Z2lzdGVyIHVzYWdlIG9mIHRoZXNlIFNNQyBTSVAgY2FsbHMgYXJlIGFzIGZvbGxvd3M6DQo+ID4+
PiBhMCAtIHNlcnZpY2UNCj4gPj4+IGExIC0gZnVuY3Rpb24NCj4gPj4+IGEyLWE3IC0gYXJncw0K
PiA+Pj4NCj4gPj4+IEluIEFURiB0aGUgaGFuZGxlciBpcyBkZWNsYXJlZCBhcyBhIHJ1bnRpbWUg
c2VydmljZS4NCj4gPj4+DQo+ID4+PiBXb3VsZCB0aGUgYXBwcm9wcmlhdGUgY29tbW1pdCBtZXNz
YWdlIGhlcmUgYmUgc29tZXRoaW5nIGFsb25nIHRoZQ0KPiA+Pj4gbGluZXMgb2YgYmVsb3c/DQo+
ID4+PiAiIiINCj4gPj4+IFdoZW4gdXNpbmcgTGludXggZm9yIGRvbTAgdGhlcmUgYXJlIGEgYnVu
Y2ggb2YgZHJpdmVycyB0aGF0IG5lZWQgdG8NCj4gPj4+IGRvwqDCoCBTTUMgU0lQIGNhbGxzIGlu
dG8gdGhlIGZpcm13YXJlIHRvIGVuYWJsZSBjZXJ0YWluIGhhcmR3YXJlIGJpdHMNCj4gPj4+IGxp
a2UgdGhlIHdhdGNoZG9nLg0KPiA+Pj4gIiIiDQo+ID4+DQo+ID4+IEl0IHJlYWRzIGJldHRlciB0
aGFua3MuDQo+ID4+DQo+ID4+IFsuLi5dDQo+ID4+DQo+ID4+Pj4gQnV0IGV2ZW4gaWYgd2UgcmVz
dHJpY3QgdG8gZG9tMCwgaGF2ZSB5b3UgY2hlY2tlZCB0aGF0IG5vbmUgb2YgdGhlDQo+ID4+Pj4g
U01DcyB1c2UgYnVmZmVycz8NCj4gPj4+IEkgaGF2ZW4ndCBmb3VuZCBhbnkgc3VjaCBpbnN0YW5j
ZXMgaW4gdGhlIExpbnV4IGtlcm5lbCB3aGVyZSBhDQo+ID4+PiBidWZmZXIgaXMgdXNlZC4gQWRk
aW5nIGEgY2FsbCBmaWx0ZXJpbmcgbGlrZSBzdWdnZXN0ZWQgYmVsb3cNCj4gPj4+IGFkZGl0aW9u
cyBvZiBzdWNoIGZ1bmN0aW9ucyBjYW4gYmUgZGlzY292ZXJlZCBhbmQgYWRhcHRlZCBmb3IgaWYg
dGhleQ0KPiB3b3VsZCBzaG93IHVwIGxhdGVyLg0KPiA+Pj4+DQo+ID4+Pj4gUmF0aGVyIHRoYW4g
cHJvdmlkaW5nIGEgYmxhbmtldCBmb3J3YXJkLCB0byBtZSBpdCBzb3VuZHMgbW9yZSBsaWtlDQo+
ID4+Pj4geW91IHdhbnQgdG8gcHJvdmlkZSBhbiBhbGxvd2xpc3Qgb2YgdGhlIFNNQ3MuIFRoaXMg
aXMgbW9yZQ0KPiA+Pj4+IGZ1dHVyZXByb29mIGFuZCBhdm9pZCB0aGUgcmlzayB0byBleHBvc2Ug
dW5zYWZlIFNNQ3MgdG8gYW55IGRvbWFpbi4NCj4gPj4+Pg0KPiA+Pj4+IEZvciBhbiBleGFtcGxl
LCB5b3UgY2FuIGhhdmUgYSBsb29rIGF0IHRoZSBFRU1JIG1lZGlhdG9yIGZvciBYaWxpbnguDQo+
ID4+Pg0KPiA+Pj4gQWNrLiBEbyB5b3UgcHJlZmVyIHRvIHNlZSBvbmx5IG9uIFNNQ0NDIHNlcnZp
Y2UgbGV2ZWwgb3IgYWxzbyBvbg0KPiA+Pj4gZnVuY3Rpb24gbGV2ZWw/IChhMSByZWdpc3Rlciwg
cGVyIGRlc2NyaXB0aW9uIGVhcmxpZXIpDQo+ID4+DQo+ID4+IEkgYW0gbm90IHN1cmUuIEl0IHdp
bGwgZGVwZW5kIG9uIHdoZXRoZXIgaXQgaXMgY29ycmVjdCB0byBleHBvc2UNCj4gPj4gKmFsbCog
dGhlIGZ1bmN0aW9ucyB3aXRoaW4gYSBzZXJ2aWNlIGxldmVsIGFuZCB0aGV5IGhhdmUgdGhlIHNh
bWUgZm9ybWF0Lg0KPiA+Pg0KPiA+PiBJZiB5b3UgY2FuJ3QgZ3VhcmFudGVlIHRoYXQsIHRoZW4g
eW91IHdpbGwgbW9zdCBsaWtlbHkgbmVlZCB0bw0KPiA+PiBhbGxvd2xpc3QgYXQgdGhlIGZ1bmN0
aW9uIGxldmVsLg0KPiA+Pg0KPiA+PiBBbHNvLCBkbyB5b3UgaGF2ZSBhIHNwZWMgaW4gaGFuZCB0
aGF0IHdvdWxkIGhlbHAgdG8gdW5kZXJzdGFuZCB3aGljaA0KPiA+PiBzZXJ2aWNlL2Z1bmN0aW9u
IGlzIGltcGxlbWVudGVkIHZpYSB0aG9zZSBTTUNzPw0KPiA+DQo+ID4gSSBkb24ndCBoYXZlIHRo
ZSBzcGVjIHVuZm9ydHVuYXRlbHksIGJ1dCBJIHdpbGwgYWRkIGEgZmlsdGVyIG9uIGJvdGgNCj4g
PiBzZXJ2aWNlIGFuZCBmdW5jdGlvbiBmb3IgVjIgYW5kIHdlJ2xsIHRha2UgaXQgZnJvbSB0aGVy
ZS4NCj4gDQo+IEBQZW5nLCBkbyB5b3UgaGF2ZSBhbnkgc3BlY2lmaWNhdGlvbiB5b3UgY291bGQg
c2hhcmU/IEhvdyBzdGFibGUgaXMgdGhlDQo+IGludGVyZmFjZT8NCg0KTm8gc3BlY2lmaWNhdGlv
biwgdGhlIHVzZSBpcyBJTVhfU0lQX1ggaW4gbGludXgga2VybmVsIHNvdXJjZS4NCg0KU3VjaCBh
cyBJTVhfU0lQX1JUQywgSU1YX1NJUF9USU1FUg0KDQpJdCBpcyBzdGFibGUgYW5kIG5vIGNoYW5n
ZSwgd2Ugb25seSBhZGQgbmV3IFNJUCBtYWNybyBpZiBuZWVkZWQNCmFuZCBubyBjaGFuZ2UgdGhl
IG1lYW5pbmcgb3IgcmV1c2Ugb2xkIFNJUC4NCg0KUmVnYXJkcywNClBlbmcuDQoNCj4gDQo+IENo
ZWVycywNCj4gDQo+IC0tDQo+IEp1bGllbiBHcmFsbA0K


From xen-devel-bounces@lists.xenproject.org Sun Feb 04 10:21:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 04 Feb 2024 10:21:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675527.1050941 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWZch-0000VH-4A; Sun, 04 Feb 2024 10:21:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675527.1050941; Sun, 04 Feb 2024 10:21:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWZch-0000VA-1d; Sun, 04 Feb 2024 10:21:15 +0000
Received: by outflank-mailman (input) for mailman id 675527;
 Sun, 04 Feb 2024 10:21:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=yZno=JN=rabbit.lu=slack@srs-se1.protection.inumbo.net>)
 id 1rWZcf-0000V4-IY
 for xen-devel@lists.xenproject.org; Sun, 04 Feb 2024 10:21:13 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1df6788b-c347-11ee-8a45-1f161083a0e0;
 Sun, 04 Feb 2024 11:21:11 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-40fd446d4ceso5876675e9.2
 for <xen-devel@lists.xenproject.org>; Sun, 04 Feb 2024 02:21:10 -0800 (PST)
Received: from localhost.localdomain (82-64-138-184.subs.proxad.net.
 [82.64.138.184]) by smtp.googlemail.com with ESMTPSA id
 l30-20020a05600c1d1e00b0040eee852a3dsm5291730wms.10.2024.02.04.02.21.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sun, 04 Feb 2024 02:21:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1df6788b-c347-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=rabbit-lu.20230601.gappssmtp.com; s=20230601; t=1707042070; x=1707646870; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=FXr9s7Gve9HSkvnZjmir4x9yxXlIr/2oW7d2MgJIizE=;
        b=uAqkBH6G4JzJvlvk38i4yaGbtz9gVHhmv0+/je5iq0YFWGDiwQavgAJ0O7ZKhoUYeK
         Yq72cbhMGs9YLM0QjtF5r1VDMXuKuXGbYPrzKXGa7VwsnZQxYnPy3mbXFBsWX/sevcJv
         t0IIu3/Srs5BlftnB0xvDUGe14tXI5yED5v7EFnQFXwvXo8R9mIyp0pvcv4sF4YXjXZY
         5llk65iryueTNELUSjttePAz4CYHyndPjh7uxrezI6WqPGhBuA8MVYi4w77IFTqbsm4o
         nmsc+DP9oUgG4hKXypLHZGK49/fpQqpTv+eB8I5ScFRA9MIv4DdU2tEG2bJVqUFI5H6K
         AgIQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707042070; x=1707646870;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=FXr9s7Gve9HSkvnZjmir4x9yxXlIr/2oW7d2MgJIizE=;
        b=Nw4t5IgL82ul39tOKhQFRjZOdwAnEV+oyT8wq8w262cN373lp/VUlnsgTxlU67ISiu
         p2AEI/DeFQYtQb+1lJFYBy0qkTAcAHOtQgB/rK9Wrdeq/U/72cy/DrBU6tr1jiLGf3xf
         2X/HPLr1hUBLNwaQB5fbgfs5joJHRZ3eZyl44qJiOO/LpjxdAWJP0hn9VgXe18jM5cC6
         LoGEo8RsODCUb+pSJ0upKzSIiGFRDsM6+tDph0vxNJ0/YG2RvN98bn3CJVI6A9/OG9Wi
         SjF9C0YerUfhs2OkazbWDacbDTTgHi27rZpS212+wz4f74pjxWptvUUQG2u0VVGPrztb
         ShuQ==
X-Gm-Message-State: AOJu0Yx6NszHHvaqDNIO/sbpkoupyQnCVPIqe+o4BNovdMd7O+tCzck1
	7u1LbXHmS+DyNwXfUor49VNs346aPGzE6the0Fgss7hRKjGoXP0nhLrSzUKBeo7fFbroOkwcUTl
	KSUA=
X-Google-Smtp-Source: AGHT+IFXauec+v82sndPjUeNUGFJuDh0jNuxkThnYday7jYQTu6Z3KDLWhlvdukLBE141RYkNOPcOQ==
X-Received: by 2002:a05:600c:a01:b0:40f:c1f9:c884 with SMTP id z1-20020a05600c0a0100b0040fc1f9c884mr2465373wmp.39.1707042070275;
        Sun, 04 Feb 2024 02:21:10 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVFLYBAaYhUIBbpb8TXk/ZjTglOToDvWIujNIfIeqjP9qfv6CBN6m2fl9vzm7kPer5M82gXG4f2iPpFkT3O7IdUhVC9HhmS5vEG9pkY1LueliAwp/TJ
From: =?UTF-8?q?zithro=20/=20Cyril=20R=C3=A9bert?= <slack@rabbit.lu>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Cyril=20R=C3=A9bert?= <slack@rabbit.lu>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH] tools/xentop: fix sorting bug for some columns
Date: Sun,  4 Feb 2024 11:19:40 +0100
Message-Id: <1bb93dbf9d09091fa36a3966ad8ffbccdb742166.1707041980.git.slack@rabbit.lu>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Cyril Rébert <slack@rabbit.lu>

Sort doesn't work on columns VBD_OO, VBD_RD, VBD_WR and VBD_RSECT.
Fix by adjusting variables names in compare functions.
Bug fix only. No functional change.

Signed-off-by: Cyril Rébert (zithro) <slack@rabbit.lu>
---
 tools/xentop/xentop.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
index 950e8935c4..545bd5e96d 100644
--- a/tools/xentop/xentop.c
+++ b/tools/xentop/xentop.c
@@ -684,7 +684,7 @@ static int compare_vbd_oo(xenstat_domain *domain1, xenstat_domain *domain2)
 	unsigned long long dom1_vbd_oo = 0, dom2_vbd_oo = 0;
 
 	tot_vbd_reqs(domain1, FIELD_VBD_OO, &dom1_vbd_oo);
-	tot_vbd_reqs(domain1, FIELD_VBD_OO, &dom2_vbd_oo);
+	tot_vbd_reqs(domain2, FIELD_VBD_OO, &dom2_vbd_oo);
 
 	return -compare(dom1_vbd_oo, dom2_vbd_oo);
 }
@@ -711,9 +711,9 @@ static int compare_vbd_rd(xenstat_domain *domain1, xenstat_domain *domain2)
 	unsigned long long dom1_vbd_rd = 0, dom2_vbd_rd = 0;
 
 	tot_vbd_reqs(domain1, FIELD_VBD_RD, &dom1_vbd_rd);
-	tot_vbd_reqs(domain1, FIELD_VBD_RD, &dom2_vbd_rd);
+	tot_vbd_reqs(domain2, FIELD_VBD_RD, &dom2_vbd_rd);
 
-	return -compare(dom1_vbd_rd, dom1_vbd_rd);
+	return -compare(dom1_vbd_rd, dom2_vbd_rd);
 }
 
 /* Prints number of total VBD READ requests statistic */
@@ -738,7 +738,7 @@ static int compare_vbd_wr(xenstat_domain *domain1, xenstat_domain *domain2)
 	unsigned long long dom1_vbd_wr = 0, dom2_vbd_wr = 0;
 
 	tot_vbd_reqs(domain1, FIELD_VBD_WR, &dom1_vbd_wr);
-	tot_vbd_reqs(domain1, FIELD_VBD_WR, &dom2_vbd_wr);
+	tot_vbd_reqs(domain2, FIELD_VBD_WR, &dom2_vbd_wr);
 
 	return -compare(dom1_vbd_wr, dom2_vbd_wr);
 }
@@ -765,7 +765,7 @@ static int compare_vbd_rsect(xenstat_domain *domain1, xenstat_domain *domain2)
 	unsigned long long dom1_vbd_rsect = 0, dom2_vbd_rsect = 0;
 
 	tot_vbd_reqs(domain1, FIELD_VBD_RSECT, &dom1_vbd_rsect);
-	tot_vbd_reqs(domain1, FIELD_VBD_RSECT, &dom2_vbd_rsect);
+	tot_vbd_reqs(domain2, FIELD_VBD_RSECT, &dom2_vbd_rsect);
 
 	return -compare(dom1_vbd_rsect, dom2_vbd_rsect);
 }
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Sun Feb 04 13:07:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 04 Feb 2024 13:07:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675583.1050968 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWcDQ-00030n-La; Sun, 04 Feb 2024 13:07:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675583.1050968; Sun, 04 Feb 2024 13:07:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWcDQ-00030g-HU; Sun, 04 Feb 2024 13:07:20 +0000
Received: by outflank-mailman (input) for mailman id 675583;
 Sun, 04 Feb 2024 13:07:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWcDO-000308-EC; Sun, 04 Feb 2024 13:07:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWcDO-0007zr-Au; Sun, 04 Feb 2024 13:07:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWcDN-0004FD-TY; Sun, 04 Feb 2024 13:07:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWcDN-0004nD-T7; Sun, 04 Feb 2024 13:07:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=4TY1jupZHuw+jCzZfrHq8qq9W9bp8YqCDagJT9zJ2Do=; b=GPkGxpBltni7tMBImYvENErhhL
	aXNlxco+0HvkGw4G4OA5zbVH7z+yiqgEiABOAc6+AYVKoFIHhRFWa/gkgcKORZ1pj2Rry1P4GgGna
	eS6bdXsCUCueD2J30TjccmBx4GMZvHwbIzA9yuq4Eoieba/OKwcRR1D6hqxqElaeyAUU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184578-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184578: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
X-Osstest-Versions-That:
    xen=3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 04 Feb 2024 13:07:17 +0000

flight 184578 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184578/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop      fail blocked in 184568
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184568
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184568
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184568
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184568
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184568
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184568
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184568
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184568
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184568
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184568
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184568
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
baseline version:
 xen                  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc

Last test of basis   184578  2024-02-03 16:15:30 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun Feb 04 16:49:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 04 Feb 2024 16:49:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675608.1050977 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWffX-00011J-Vc; Sun, 04 Feb 2024 16:48:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675608.1050977; Sun, 04 Feb 2024 16:48:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWffX-00011C-T7; Sun, 04 Feb 2024 16:48:35 +0000
Received: by outflank-mailman (input) for mailman id 675608;
 Sun, 04 Feb 2024 16:48:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWffW-000112-1U; Sun, 04 Feb 2024 16:48:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWffV-0003t3-UI; Sun, 04 Feb 2024 16:48:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWffV-0004VE-JI; Sun, 04 Feb 2024 16:48:33 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWffV-0006qV-Iv; Sun, 04 Feb 2024 16:48:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=KgQ6loXEVE6JXvPLzQ0xRSjfGKon+HMIdtLtzsafwTU=; b=G260DKAK0BFCCnWVFTpXb/5aPx
	ZaeUEjU8YpEcAlXpu3mp+QGzril0wzo1syaNEa0Lqv6M3z86vRrLItk0jXgzW6iGgnmMAVstnJghG
	41jMCF59wdmisMxhA2ACW06753Sb91JV5PtqLJx4PK8ddRRJCtXZ084it/UdHZ+KYfoc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184587-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184587: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=056b4bf74bf5cfa2c0c9c5e44a7c5f3f461d17c0
X-Osstest-Versions-That:
    ovmf=141dcaed6cc930d83a4f95cb51ebc22f51fcc32c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 04 Feb 2024 16:48:33 +0000

flight 184587 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184587/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 056b4bf74bf5cfa2c0c9c5e44a7c5f3f461d17c0
baseline version:
 ovmf                 141dcaed6cc930d83a4f95cb51ebc22f51fcc32c

Last test of basis   184583  2024-02-04 03:11:25 Z    0 days
Testing same since   184587  2024-02-04 13:41:21 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Pedro Falcato <pedro.falcato@gmail.com>
  Pierre Gondois <Pierre.Gondois@arm.com>
  PierreGondois <pierre.gondois@arm.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   141dcaed6c..056b4bf74b  056b4bf74bf5cfa2c0c9c5e44a7c5f3f461d17c0 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sun Feb 04 19:32:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 04 Feb 2024 19:32:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675634.1050993 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWiDr-0001ru-7a; Sun, 04 Feb 2024 19:32:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675634.1050993; Sun, 04 Feb 2024 19:32:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWiDr-0001rn-4K; Sun, 04 Feb 2024 19:32:11 +0000
Received: by outflank-mailman (input) for mailman id 675634;
 Sun, 04 Feb 2024 19:32:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWiDq-0001rd-6E; Sun, 04 Feb 2024 19:32:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWiDq-0006oF-1P; Sun, 04 Feb 2024 19:32:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWiDp-00034I-IN; Sun, 04 Feb 2024 19:32:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWiDp-0003Fp-Ho; Sun, 04 Feb 2024 19:32:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/n/bpTC7+Yj7kwnVKewJ78pmFPfk3nVdp1Ecso7TvFM=; b=nZgqaAxArphFbLFJdgltNywDaf
	SEBaF9ZT9pbNVPCoSV/fyjJjpfB/iWs/7Jinj1KGi74jHIWtcR0G21/U1xNzPRmVti2J3ow52oZXi
	eQkNgOwDMxyqZyOOP7/gmo7+Wl5v6mRfv9q6hZRFE3NOD4X8309wEETs9H9+/jPdDN4c=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184584-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184584: regressions - FAIL
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:debian-fixup:fail:regression
    linux-linus:build-arm64-xsm:xen-build:fail:regression
    linux-linus:test-amd64-amd64-freebsd11-amd64:guest-start/freebsd.repeat:fail:regression
    linux-linus:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=b555d191561a7f89b8d2108dff687d9bc4284e48
X-Osstest-Versions-That:
    linux=56897d51886fa7e9f034ff26128eb09f1b811594
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 04 Feb 2024 19:32:09 +0000

flight 184584 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184584/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvhv2-intel 13 debian-fixup          fail REGR. vs. 184576
 build-arm64-xsm               6 xen-build                fail REGR. vs. 184576
 test-amd64-amd64-freebsd11-amd64 21 guest-start/freebsd.repeat fail REGR. vs. 184576

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184576
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184576
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184576
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184576
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184576
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184576
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184576
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184576
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 linux                b555d191561a7f89b8d2108dff687d9bc4284e48
baseline version:
 linux                56897d51886fa7e9f034ff26128eb09f1b811594

Last test of basis   184576  2024-02-03 12:02:13 Z    1 days
Testing same since   184584  2024-02-04 03:45:39 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Arnaldo Carvalho de Melo <acme@kernel.org>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Ian Rogers <irogers@google.com>
  James Clark <james.clark@arm.com>
  Jiri Olsa <jolsa@kernel.org>
  Kan Liang <kan.liang@linux.intel.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Namhyung Kim <namhyung@kernel.org>
  Sun Haiyong <sunhaiyong@loongson.cn>
  Thomas Richter <tmricht@linux.ibm.com>
  Yanteng Si <siyanteng@loongson.cn>
  Yicong Yang <yangyicong@hisilicon.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              fail    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             fail    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              fail    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 892 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Feb 04 22:33:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 04 Feb 2024 22:33:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675656.1051004 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWl2e-0003nO-1b; Sun, 04 Feb 2024 22:32:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675656.1051004; Sun, 04 Feb 2024 22:32:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWl2d-0003nH-TH; Sun, 04 Feb 2024 22:32:47 +0000
Received: by outflank-mailman (input) for mailman id 675656;
 Sun, 04 Feb 2024 22:32:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWl2c-0003n7-FM; Sun, 04 Feb 2024 22:32:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWl2c-0001VG-Cf; Sun, 04 Feb 2024 22:32:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWl2b-0002fR-Su; Sun, 04 Feb 2024 22:32:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWl2b-0004zm-SN; Sun, 04 Feb 2024 22:32:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=n6PD1WFkSlfKd2IYcI3QCMmLy3HmR5WjWdffeF1S9xI=; b=s7I8a+LX6lpv3NMdEZovcTX05Y
	9BcRuzbvmWgW7i9z37Hc+WdsUQSeqYpdyLzGu9Q/HNdEj/rBharBqoVb1XkOCxKhg54vHde6xBoxA
	NXrOHUujvfQtiMEkjqVFjvGsCpdwWIh5nzg42kATXt0pJOtz8u6vDTn+VB1wr6hJbVjA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184586-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184586: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
X-Osstest-Versions-That:
    xen=3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 04 Feb 2024 22:32:45 +0000

flight 184586 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184586/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184578
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184578
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184578
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184578
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184578
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184578
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184578
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184578
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184578
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184578
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184578
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184578
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
baseline version:
 xen                  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc

Last test of basis   184586  2024-02-04 13:11:45 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 04:52:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 04:52:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675666.1051014 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWqxT-0003Pc-KD; Mon, 05 Feb 2024 04:51:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675666.1051014; Mon, 05 Feb 2024 04:51:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWqxT-0003PT-Ey; Mon, 05 Feb 2024 04:51:51 +0000
Received: by outflank-mailman (input) for mailman id 675666;
 Mon, 05 Feb 2024 04:51:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWqxS-0003PJ-Ij; Mon, 05 Feb 2024 04:51:50 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWqxS-0003pi-F9; Mon, 05 Feb 2024 04:51:50 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWqxR-0007li-TK; Mon, 05 Feb 2024 04:51:49 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWqxR-0002cD-Qq; Mon, 05 Feb 2024 04:51:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ABykh2kT9rIBUSNwbVRjOO81qRhOn2WQIToSHRNXsLY=; b=KVz0OBcH4q9B/IZUUiTBe9n04g
	HBSLB1CUGNqfuQS3nhd/o3dvDDwizB1ZgscGMoKb9H8UPt6QR9JquPUc2pjIGCuw5BXJcI81M/EIL
	lhyoSh3fGQ+8d51/vpuPRwr7S82dVww1klPNInhr36SPFu4eLmXQJUPv39cNK++qIze0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184588-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184588: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=54be6c6c5ae8e0d93a6c4641cb7528eb0b6ba478
X-Osstest-Versions-That:
    linux=56897d51886fa7e9f034ff26128eb09f1b811594
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 05 Feb 2024 04:51:49 +0000

flight 184588 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184588/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184576
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184576
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184576
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184576
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184576
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184576
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184576
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184576
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                54be6c6c5ae8e0d93a6c4641cb7528eb0b6ba478
baseline version:
 linux                56897d51886fa7e9f034ff26128eb09f1b811594

Last test of basis   184576  2024-02-03 12:02:13 Z    1 days
Failing since        184584  2024-02-04 03:45:39 Z    1 days    2 attempts
Testing same since   184588  2024-02-04 19:44:13 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Christian A. Ehrhardt" <lk@c--e.de>
  "Darrick J. Wong" <djwong@kernel.org>
  Andi Shyti <andi.shyti@kernel.org>
  Andrey Albershteyn <aalbersh@redhat.com>
  Arnaldo Carvalho de Melo <acme@kernel.org>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Badhri Jagan Sridharan <badhri@google.com>
  Baokun Li <libaokun1@huawei.com>
  Carlos Llamas <cmllamas@google.com>
  Chandan Babu R <chandanbabu@kernel.org>
  Christian A. Ehrhardt <lk@c--e.de>
  Christoph Niedermaier <cniedermaier@dh-electronics.com>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Dan Carpenter <dan.carpenter@linaro.org>
  Darrick J. Wong <djwong@kernel.org>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Ekansh Gupta <quic_ekangupt@quicinc.com>
  Frank Li <Frank.Li@nxp.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guanhua Gao <guanhua.gao@nxp.com>
  Gui-Dong Han <2045gemini@gmail.com>
  Heikki Krogerus <heikki.krogerus@linux.intel.com>
  Horatiu Vultur <horatiu.vultur@microchip.com>
  Hugo Villeneuve <hvilleneuve@dimonoff.com>
  Ian Rogers <irogers@google.com>
  JackBB Wu <wojackbb@gmail.com>
  Jai Luthra <j-luthra@ti.com>
  James Clark <james.clark@arm.com>
  Jiri Olsa <jolsa@kernel.org>
  Johan Hovold <johan@kernel.org>
  Kan Liang <kan.liang@linux.intel.com>
  Kemeng Shi <shikemeng@huaweicloud.com>
  Krishna Kurapati <quic_kriskura@quicinc.com>
  Kunwu Chan <chentao@kylinos.cn>
  Leonard Dallmayr <leonard.dallmayr@mailbox.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Mantas Pucka <mantas@8devices.com>
  Mathias Nyman <mathias.nyman@linux.intel.com>
  Michal Pecio <michal.pecio@gmail.com>
  Michał Pecio <michal.pecio@gmail.com>
  Namhyung Kim <namhyung@kernel.org>
  Oliver Neukum <oneukum@suse.com>
  Paulo Alcantara <pc@manguebit.com>
  Peter Ujfalusi <peter.ujfalusi@gmail.com>
  Prashanth K <quic_prashk@quicinc.com>
  Puliang Lu <puliang.lu@fibocom.com>
  Randy Dunlap <rdunlap@infradead.org>
  Sean Anderson <sean.anderson@seco.com>
  Shyam Prasad N <sprasad@microsoft.com>
  Steve French <stfrench@microsoft.com>
  Sun Haiyong <sunhaiyong@loongson.cn>
  Theodore Ts'o <tytso@mit.edu>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Thomas Richter <tmricht@linux.ibm.com>
  Tony Lindgren <tony@atomide.com>
  Udipto Goswami <quic_ugoswami@quicinc.com>
  Uttkarsh Aggarwal <quic_uaggarwa@quicinc.com>
  Vinod Koul <vkoul@kernel.org>
  Will Deacon <will@kernel.org>
  Wolfram Sang <wsa+renesas@sang-engineering.com>
  Xu Yang <xu.yang_2@nxp.com>
  Yanteng Si <siyanteng@loongson.cn>
  Yicong Yang <yangyicong@hisilicon.com>
  Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
  yuan linyu <yuanlinyu@hihonor.com>
  Zhang Yi <yi.zhang@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   56897d51886f..54be6c6c5ae8  54be6c6c5ae8e0d93a6c4641cb7528eb0b6ba478 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 07:46:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 07:46:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675675.1051023 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWtgY-0004Uu-3B; Mon, 05 Feb 2024 07:46:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675675.1051023; Mon, 05 Feb 2024 07:46:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWtgX-0004Un-WF; Mon, 05 Feb 2024 07:46:34 +0000
Received: by outflank-mailman (input) for mailman id 675675;
 Mon, 05 Feb 2024 07:46:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWtgX-0004Uh-8g
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 07:46:33 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ac4a7774-c3fa-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 08:46:29 +0100 (CET)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-337d05b8942so3156192f8f.3
 for <xen-devel@lists.xenproject.org>; Sun, 04 Feb 2024 23:46:29 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n8-20020a5d4208000000b0033b1b01e4fcsm7449521wrq.96.2024.02.04.23.46.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 04 Feb 2024 23:46:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac4a7774-c3fa-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707119189; x=1707723989; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=TjSQo4C+RU730aORSF5XUTZ6KLXBn/fJU6t9RU4V5Dk=;
        b=fylLfL4Wd6U/u9EYnU2oX6tVbvCKYmvTp7mqMp5TVsCUzqPFTay5BpY8/gLZN7lJ8Z
         99JGveU1p10O6oWBWDJvYzPus3F5Xdfy1EDqZrBXGXkRy2Jey2tOwiMXPbLkZjQTVyJr
         DloxquIXguQHAwwE2BoLDZS7uNfMmOex3Ok9aQfCmUATxCntyPhsJuoqvvP0MAq46lB1
         nPLkPxBK28z2aOxMVagtlG4T1CkeNnue/5VMux5371PC1lUiP5rgDBrp9aqWi+MW/Qhz
         3KTBhy1D5K8iDoYLCDTXoDBuog76DUd9pJbRGTUcdPyycTJpayICAQVEGaADrYd7Drg+
         NMlQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707119189; x=1707723989;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=TjSQo4C+RU730aORSF5XUTZ6KLXBn/fJU6t9RU4V5Dk=;
        b=Oq6DvEDhZ8pZ/2OJ9UF8tAKbG92bl5+w4QaYaKbJs7+PiHnySVyRn/CCJShQ8StpXa
         JRWCNr3ylVoDHIepi0fz81pELT401gBqSd06FwgN6iiaJKYwKJj6HFWTo8z+2l95r+vO
         MO0kfqZjjGxj1dlLPSAHk36EDfB3y0flpNputBySbzNC7dcoAujDOAqZnqCUlWmqbO7j
         Ero9g6Mmoz3FubHLxglmDV//NgA1xIig4YUGe19a0vvoborMiK2JuukY9g87t/E0qnwm
         dLNO8TWOh1wtb3usQiAxTa9Y8HZnlmMjolo+19mDsM5SW50DevTo72fmX4Xc89rRaMe5
         AqyA==
X-Gm-Message-State: AOJu0Yy1Ir2J64LXPcY/edL4X789CU18Po1/VtsbQ6NX+Z9N8F4vr8aO
	gPxEvaVK6z1R8dT4cKhKzMk8VHXmMkRuA1wXO0WBMhlBqGcXQoE3Obpi7oLeHA==
X-Google-Smtp-Source: AGHT+IHMUJDoE9LhZEEGDpkNIsLx6rWaY8eVjKhBLBY4RrCN4WH70whjLLrvSammXfUjJwv84gFiZQ==
X-Received: by 2002:a5d:47aa:0:b0:33b:3dcb:9564 with SMTP id 10-20020a5d47aa000000b0033b3dcb9564mr1405884wrb.2.1707119189008;
        Sun, 04 Feb 2024 23:46:29 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUwnzG7qvOUK/2VXOP17tUvEric+bBn0h+1DvnTlL83ESI0wWf3NOd7cypwG0zz4ukKjimSeOjn1b0iWlf25R/sa/ffh3qD6liBvKQ33qxWwqq85H5hY8j4aJlV9zq6ZiI8J44pJl8+7SPjPow0yn4CT/t1s3zykUNaaWwYcnQCf1CyMWQrlx68NxoE4e67HCLIELgkxAFeXDUlv8sxkHGkV3PwcpkTJa5ph5XDSMZmEyMo4BzVOf+OTeq2CtT17sxW9zP1d8WPeHfYL0EBiIdjOQkoK2PR7muLYVm0u+pOBOdTLHpVhyY=
Message-ID: <44a9a92c-ac4c-43d0-bc1e-685818e4331d@suse.com>
Date: Mon, 5 Feb 2024 08:46:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 31/34] xen/riscv: add minimal stuff to mm.h to build
 full Xen
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1703255175.git.oleksii.kurochko@gmail.com>
 <4411f6af38586074b347cd6005f19f9c670faa74.1703255175.git.oleksii.kurochko@gmail.com>
 <d347c4d9-e93b-4937-8e33-e5fbbdcd6bfb@suse.com>
 <f3858360fba14a2c5b794bdd0400cd7ab8e66f73.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f3858360fba14a2c5b794bdd0400cd7ab8e66f73.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02.02.2024 18:30, Oleksii wrote:
> On Tue, 2024-01-23 at 14:03 +0100, Jan Beulich wrote:
>> On 22.12.2023 16:13, Oleksii Kurochko wrote:
>>> +/* Convert between Xen-heap virtual addresses and machine frame
>>> numbers. */
>>> +#define __virt_to_mfn(va) (virt_to_maddr(va) >> PAGE_SHIFT)
>>> +#define __mfn_to_virt(mfn) maddr_to_virt((paddr_t)(mfn) <<
>>> PAGE_SHIFT)
>>
>> These would imo better use maddr_to_mfn() and mfn_to_maddr(), rather
>> than
>> kind of open-coding them. The former could also use PFN_DOWN() as an
>> alternative.
> We can't to as __virt_to_mfn() when is used it is usually wrapped by
> _mfn() which expect to have unsigned long as an argument.

#define __virt_to_mfn(va)  mfn_x(maddr_to_mfn(virt_to_maddr(va))
#define __mfn_to_virt(mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn)))

?

>>> +/* Convert between Xen-heap virtual addresses and page-info
>>> structures. */
>>> +static inline struct page_info *virt_to_page(const void *v)
>>> +{
>>> +    BUG();
>>> +    return NULL;
>>> +}
>>> +
>>> +/*
>>> + * We define non-underscored wrappers for above conversion
>>> functions.
>>> + * These are overriden in various source files while underscored
>>> version
>>> + * remain intact.
>>> + */
>>> +#define virt_to_mfn(va)     __virt_to_mfn(va)
>>> +#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
>>
>> Is this really still needed? Would be pretty nice if in a new port we
>> could get to start cleanly right away (i.e. by not needing per-file
>> overrides, but using type-safe expansions here right away).
> We still need __virt_to_mfn and __mfn_to_virt as common code use them:
>  * xen/common/xenoprof.c:24:#define virt_to_mfn(va)
> mfn(__virt_to_mfn(va))

Are you meaning to enable this code on RISC-V.

>  * xen/include/xen/domain_page.h:59:#define domain_page_map_to_mfn(ptr)
> _mfn(__virt_to_mfn((unsigned long)(ptr)))

Hmm, yes, we should finally get this sorted. But yeah, not something I want
to ask you to do.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 08:06:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 08:06:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675682.1051033 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWtzY-0007iS-0h; Mon, 05 Feb 2024 08:06:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675682.1051033; Mon, 05 Feb 2024 08:06:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWtzX-0007iL-Tt; Mon, 05 Feb 2024 08:06:11 +0000
Received: by outflank-mailman (input) for mailman id 675682;
 Mon, 05 Feb 2024 08:06:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWtzW-0007iD-0h
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 08:06:10 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 684f5b12-c3fd-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 09:06:04 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 550B41F8A3;
 Mon,  5 Feb 2024 08:06:03 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 29A31136F5;
 Mon,  5 Feb 2024 08:06:03 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 0JdxCOuWwGWKCAAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 08:06:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 684f5b12-c3fd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707120363; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=XC3UG/rC52b26n3G3wrw4PZYIDR1jSqfCbyh0n9R4PA=;
	b=CgQyfnwxzr7VUtnaF8gQ9tNImQKN+j8aVI8nMw6lh6yNUdpn0zsbKq6UH7zDX74GjoDGE1
	EUGgHxOPyRi9BP3V6Nz5Hpu3cHwWK0RadEM9ey2XOornlF7c4Dm92d9RhiiPxTlg7I/r4m
	/gQRzIupuG7DzNZhx2AqktNRxMVgau8=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707120363; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=XC3UG/rC52b26n3G3wrw4PZYIDR1jSqfCbyh0n9R4PA=;
	b=CgQyfnwxzr7VUtnaF8gQ9tNImQKN+j8aVI8nMw6lh6yNUdpn0zsbKq6UH7zDX74GjoDGE1
	EUGgHxOPyRi9BP3V6Nz5Hpu3cHwWK0RadEM9ey2XOornlF7c4Dm92d9RhiiPxTlg7I/r4m
	/gQRzIupuG7DzNZhx2AqktNRxMVgau8=
Message-ID: <034b465b-df1f-4886-a12e-2fe1110a75ea@suse.com>
Date: Mon, 5 Feb 2024 09:06:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 24/33] tools/xenstored: move all log-pipe handling into
 posix.c
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-25-jgross@suse.com>
 <7feae542-5b76-481e-8f2e-8dac91042a03@xen.org>
Content-Language: en-US
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <7feae542-5b76-481e-8f2e-8dac91042a03@xen.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------sGPDc7XGyjRV4V38h78Xa0Gs"
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.19
X-Spamd-Result: default: False [-3.19 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 HAS_ATTACHMENT(0.00)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MIME_BASE64_TEXT(0.10)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 SIGNED_PGP(-2.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[];
	 MIME_UNKNOWN(0.10)[application/pgp-keys]
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------sGPDc7XGyjRV4V38h78Xa0Gs
Content-Type: multipart/mixed; boundary="------------MPJx6rTmdo08wB5X9nmg3BC0";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
Message-ID: <034b465b-df1f-4886-a12e-2fe1110a75ea@suse.com>
Subject: Re: [PATCH v3 24/33] tools/xenstored: move all log-pipe handling into
 posix.c
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-25-jgross@suse.com>
 <7feae542-5b76-481e-8f2e-8dac91042a03@xen.org>
In-Reply-To: <7feae542-5b76-481e-8f2e-8dac91042a03@xen.org>

--------------MPJx6rTmdo08wB5X9nmg3BC0
Content-Type: multipart/mixed; boundary="------------w4OGu0mmAjRAbhcf7fbgeX5K"

--------------w4OGu0mmAjRAbhcf7fbgeX5K
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMjUuMDEuMjQgMTk6NTQsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4gSGkgSnVlcmdlbiwN
Cj4gDQo+IE9uIDA0LzAxLzIwMjQgMDk6MDAsIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+PiBB
bGwgb2YgdGhlIGxvZy1waXBlIGhhbmRsaW5nIGlzIG5lZWRlZCBvbmx5IHdoZW4gcnVubmlu
ZyBhcyBkYWVtb24uDQo+Pg0KPj4gTW92ZSBpdCBpbnRvIHBvc2l4LmMuIFRoaXMgcmVxdWly
ZXMgdG8gaGF2ZSBhIHNlcnZpY2UgZnVuY3Rpb24gaW4gdGhlDQo+PiBtYWluIGV2ZW50IGxv
b3AgZm9yIGhhbmRsaW5nIHRoZSByZWxhdGVkIHJlcXVlc3RzIGFuZCBvbmUgZm9yIHNldHRp
bmcNCj4+IHRoZSBmZHNbXSBhcnJheS4gVXNlIGEgZ2VuZXJpYyBuYW1lIGZvciB0aG9zZSBm
dW5jdGlvbnMsIGFzIHNvY2tldA0KPj4gaGFuZGxpbmcgY2FuIGJlIGFkZGVkIHRvIHRoZW0g
bGF0ZXIsIHRvby4NCj4+DQo+PiBTaWduZWQtb2ZmLWJ5OiBKdWVyZ2VuIEdyb3NzIDxqZ3Jv
c3NAc3VzZS5jb20+DQo+PiAtLS0NCj4+IFYzOg0KPj4gLSBuZXcgcGF0Y2gNCj4+IC0tLQ0K
Pj4gwqAgdG9vbHMveGVuc3RvcmVkL2NvcmUuY8KgwqAgfCA0MSArKysrLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLQ0KPj4gwqAgdG9vbHMveGVuc3RvcmVkL2NvcmUuaMKgwqAg
fCAxMSArKysrKystLS0tDQo+PiDCoCB0b29scy94ZW5zdG9yZWQvbWluaW9zLmMgfCAxNCAr
KysrKystLS0tLS0NCj4+IMKgIHRvb2xzL3hlbnN0b3JlZC9wb3NpeC5jwqAgfCA0NiArKysr
KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKystDQo+PiDCoCA0IGZpbGVzIGNo
YW5nZWQsIDY0IGluc2VydGlvbnMoKyksIDQ4IGRlbGV0aW9ucygtKQ0KPj4NCj4+IGRpZmYg
LS1naXQgYS90b29scy94ZW5zdG9yZWQvY29yZS5jIGIvdG9vbHMveGVuc3RvcmVkL2NvcmUu
Yw0KPj4gaW5kZXggN2I1ZTFkMGMwZi4uNjUwYzFkMTU3NCAxMDA2NDQNCj4+IC0tLSBhL3Rv
b2xzL3hlbnN0b3JlZC9jb3JlLmMNCj4+ICsrKyBiL3Rvb2xzL3hlbnN0b3JlZC9jb3JlLmMN
Cj4+IEBAIC01Niw3ICs1Niw3IEBADQo+PiDCoCBleHRlcm4geGVuZXZ0Y2huX2hhbmRsZSAq
eGNlX2hhbmRsZTsgLyogaW4gZG9tYWluLmMgKi8NCj4+IMKgIHN0YXRpYyBpbnQgeGNlX3Bv
bGxmZF9pZHggPSAtMTsNCj4+IC1zdGF0aWMgc3RydWN0IHBvbGxmZCAqZmRzOw0KPj4gK3N0
cnVjdCBwb2xsZmQgKmZkczsNCj4gDQo+IFRoZSBuYW1lIGlzIHF1aXRlIGdlbmVyaWMuIFNv
LCBJIGFtIG5vdCBzdXJlIEkgbGlrZSB0aGUgaWRlYSB0byBleHBvcnQgaXQuIEhvdyANCj4g
YWJvdXQgcGFzc2luZyAnZmRzJyBhcyBhbiBhcmd1bWVudCBvZiBoYW5kbGVfc3BlY2lhbF9m
ZHMoKT8NCg0KT2theS4NCg0KPiANCj4gVGhpcyBpcyBzdGlsbCBub3QgdmVyeSBncmVhdCAo
d2UgaGF2ZSBhbiBhcnJheSBvZiAidW5rbm93biIgc2l6ZWQgcGFzcykuIEJ1dCBpdCANCj4g
aXMgYmV0dGVyIHRoYW4gZXhwb3J0aW5nIGl0Lg0KPiANCj4gVGhlIHJlc3QgTEdUTS4NCg0K
VGhhbmtzLA0KDQoNCkp1ZXJnZW4NCg0K
--------------w4OGu0mmAjRAbhcf7fbgeX5K
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------w4OGu0mmAjRAbhcf7fbgeX5K--

--------------MPJx6rTmdo08wB5X9nmg3BC0--

--------------sGPDc7XGyjRV4V38h78Xa0Gs
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXAluoFAwAAAAAACgkQsN6d1ii/Ey80
dggAgfiG/WcXnqMtPCuIyKROe3xd8nsn2hxdjnEOTq8AzTUgp4Ob9gfsnaNYeu6+1JmY3fOPeqO1
OwXyBFFV+X31JrkqqpqB0I8xcYs+Fj56s82h3Y3lxXk6tAsHOTU0D7RpgahBjQ2eeZm5lXqzh7eV
PWY0Ujx2iAn3jkvVuL7N5YALxvMJLD1F9XVSlUa/y2wjeCsyD+q3FX1h4WdYfslpTwzg1MyRmSyp
GkS6TTYymaB6keh4+B7GVtkpqhi26/bdcW148wHY78H8p4XDsK27yjr4lr7LysyiBypPnG3Nup/+
4pTcsI04tE3C4Q9dDTexeZORjM8FUuS80E+AckG2SQ==
=Hd+X
-----END PGP SIGNATURE-----

--------------sGPDc7XGyjRV4V38h78Xa0Gs--


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 08:07:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 08:07:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675685.1051043 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWu11-0008Mn-AR; Mon, 05 Feb 2024 08:07:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675685.1051043; Mon, 05 Feb 2024 08:07:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWu11-0008Mg-7s; Mon, 05 Feb 2024 08:07:43 +0000
Received: by outflank-mailman (input) for mailman id 675685;
 Mon, 05 Feb 2024 08:07:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWu10-0008MY-7H
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 08:07:42 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a1c84524-c3fd-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 09:07:40 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 990552200E;
 Mon,  5 Feb 2024 08:07:40 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6FA2A136F5;
 Mon,  5 Feb 2024 08:07:40 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 4BV9GUyXwGWKCAAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 08:07:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1c84524-c3fd-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707120460; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=Y57vnITRo/UnY7/a4vN66d5UT3L9c6cJLW2ysFG+Uuo=;
	b=lrYj5D2GRF8u6hcSXhf7BfbC3MP08WP/0MOwjyrvepyjGc5EUjN+Dd0ci5E1A4yGrCnUzB
	LP346+I3/s7/ePWZyj2qFNSTokJ1CFoYQPQPOvSsfW0xdsNatHXF8WSyZm8Y3CVoPvrqeo
	ZVInSgKznUOFX/jd92ggS6PPxAPx+9Q=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707120460; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=Y57vnITRo/UnY7/a4vN66d5UT3L9c6cJLW2ysFG+Uuo=;
	b=lrYj5D2GRF8u6hcSXhf7BfbC3MP08WP/0MOwjyrvepyjGc5EUjN+Dd0ci5E1A4yGrCnUzB
	LP346+I3/s7/ePWZyj2qFNSTokJ1CFoYQPQPOvSsfW0xdsNatHXF8WSyZm8Y3CVoPvrqeo
	ZVInSgKznUOFX/jd92ggS6PPxAPx+9Q=
Message-ID: <6abe490f-c8c5-4a7c-9ca0-0ef71845d576@suse.com>
Date: Mon, 5 Feb 2024 09:07:40 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 25/33] tools/xenstored: move all socket handling into
 posix.c
Content-Language: en-US
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-26-jgross@suse.com>
 <dcc477bd-b7cf-40e8-85be-3cde5b9cfd61@xen.org>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <dcc477bd-b7cf-40e8-85be-3cde5b9cfd61@xen.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------BWt6rFLR0DQR2ByTci0NsstP"
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -6.19
X-Spamd-Result: default: False [-6.19 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 HAS_ATTACHMENT(0.00)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 BAYES_HAM(-3.00)[100.00%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MIME_BASE64_TEXT(0.10)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 SIGNED_PGP(-2.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[];
	 MIME_UNKNOWN(0.10)[application/pgp-keys]
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------BWt6rFLR0DQR2ByTci0NsstP
Content-Type: multipart/mixed; boundary="------------PKvHsj8IqtIsQKshOKYbgtqo";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
Message-ID: <6abe490f-c8c5-4a7c-9ca0-0ef71845d576@suse.com>
Subject: Re: [PATCH v3 25/33] tools/xenstored: move all socket handling into
 posix.c
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-26-jgross@suse.com>
 <dcc477bd-b7cf-40e8-85be-3cde5b9cfd61@xen.org>
In-Reply-To: <dcc477bd-b7cf-40e8-85be-3cde5b9cfd61@xen.org>

--------------PKvHsj8IqtIsQKshOKYbgtqo
Content-Type: multipart/mixed; boundary="------------iWBJJMLqhrg9TouHb0fJluT8"

--------------iWBJJMLqhrg9TouHb0fJluT8
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMjYuMDEuMjQgMTc6MjIsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4gSGkgSnVlcmdlbiwN
Cj4gDQo+IE9uIDA0LzAxLzIwMjQgMDk6MDAsIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+PiBB
bGwgb2YgdGhlIHNvY2tldCBoYW5kbGluZyBpcyBuZWVkZWQgb25seSB3aGVuIHJ1bm5pbmcg
YXMgZGFlbW9uLg0KPj4NCj4+IE1vdmUgaXQgaW50byBwb3NpeC5jLCBhbGxvd2luZyB0byBy
ZW1vdmUgdGhlIE5PX1NPQ0tFVFMgbWFjcm8uDQo+Pg0KPj4gU2lnbmVkLW9mZi1ieTogSnVl
cmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuY29tPg0KPj4gLS0tDQo+PiBWMzoNCj4+IC0gbmV3
IHBhdGNoDQo+PiAtLS0NCj4+IMKgIHRvb2xzL3hlbnN0b3JlZC9NYWtlZmlsZS5jb21tb24g
fMKgwqAgNCAtDQo+PiDCoCB0b29scy94ZW5zdG9yZWQvY29yZS5jwqDCoMKgwqDCoMKgwqDC
oMKgIHwgMTU2ICstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+PiDCoCB0b29s
cy94ZW5zdG9yZWQvY29yZS5owqDCoMKgwqDCoMKgwqDCoMKgIHzCoMKgIDggKy0NCj4+IMKg
IHRvb2xzL3hlbnN0b3JlZC9kb21haW4uY8KgwqDCoMKgwqDCoMKgIHzCoMKgIDkgKy0NCj4+
IMKgIHRvb2xzL3hlbnN0b3JlZC9taW5pb3MuY8KgwqDCoMKgwqDCoMKgIHzCoMKgIDYgKysN
Cj4+IMKgIHRvb2xzL3hlbnN0b3JlZC9wb3NpeC5jwqDCoMKgwqDCoMKgwqDCoCB8IDE1MyAr
KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrDQo+PiDCoCA2IGZpbGVzIGNoYW5nZWQs
IDE2NyBpbnNlcnRpb25zKCspLCAxNjkgZGVsZXRpb25zKC0pDQo+Pg0KPj4gZGlmZiAtLWdp
dCBhL3Rvb2xzL3hlbnN0b3JlZC9NYWtlZmlsZS5jb21tb24gYi90b29scy94ZW5zdG9yZWQv
TWFrZWZpbGUuY29tbW9uDQo+PiBpbmRleCAxODlhYjgxYjhkLi5lZjYzZWY2NTBjIDEwMDY0
NA0KPj4gLS0tIGEvdG9vbHMveGVuc3RvcmVkL01ha2VmaWxlLmNvbW1vbg0KPj4gKysrIGIv
dG9vbHMveGVuc3RvcmVkL01ha2VmaWxlLmNvbW1vbg0KPj4gQEAgLTE2LDEwICsxNiw2IEBA
IENGTEFHUyArPSAkKENGTEFHU19saWJ4ZW5jdHJsKQ0KPj4gwqAgQ0ZMQUdTICs9ICQoQ0ZM
QUdTX2xpYnhlbmd1ZXN0KQ0KPj4gwqAgQ0ZMQUdTICs9ICQoQ0ZMQUdTX2xpYnhlbnRvb2xj
b3JlKQ0KPj4gLWlmZGVmIENPTkZJR19TVFVCRE9NDQo+PiAtQ0ZMQUdTICs9IC1ETk9fU09D
S0VUUz0xDQo+PiAtZW5kaWYNCj4+IC0NCj4+IMKgICQoWEVOU1RPUkVEX09CSlMteSk6IENG
TEFHUyArPSAkKENGTEFHU19saWJ4ZW5nbnR0YWIpDQo+PiDCoCB4ZW5zdG9yZWQuYTogJChY
RU5TVE9SRURfT0JKUy15KQ0KPj4gZGlmZiAtLWdpdCBhL3Rvb2xzL3hlbnN0b3JlZC9jb3Jl
LmMgYi90b29scy94ZW5zdG9yZWQvY29yZS5jDQo+PiBpbmRleCA2NTBjMWQxNTc0Li44ZmYx
ZmFjY2VmIDEwMDY0NA0KPj4gLS0tIGEvdG9vbHMveGVuc3RvcmVkL2NvcmUuYw0KPj4gKysr
IGIvdG9vbHMveGVuc3RvcmVkL2NvcmUuYw0KPj4gQEAgLTIwLDEwICsyMCw2IEBADQo+PiDC
oCAjaW5jbHVkZSA8c3lzL3R5cGVzLmg+DQo+PiDCoCAjaW5jbHVkZSA8c3lzL3N0YXQuaD4N
Cj4+IMKgICNpbmNsdWRlIDxwb2xsLmg+DQo+PiAtI2lmbmRlZiBOT19TT0NLRVRTDQo+PiAt
I2luY2x1ZGUgPHN5cy9zb2NrZXQuaD4NCj4+IC0jaW5jbHVkZSA8c3lzL3VuLmg+DQo+PiAt
I2VuZGlmDQo+PiDCoCAjaW5jbHVkZSA8c3lzL3RpbWUuaD4NCj4+IMKgICNpbmNsdWRlIDx0
aW1lLmg+DQo+PiDCoCAjaW5jbHVkZSA8dW5pc3RkLmg+DQo+PiBAQCAtNjEsNyArNTcsNyBA
QCBzdGF0aWMgdW5zaWduZWQgaW50IGN1cnJlbnRfYXJyYXlfc2l6ZTsNCj4+IMKgIHN0YXRp
YyB1bnNpZ25lZCBpbnQgbnJfZmRzOw0KPj4gwqAgc3RhdGljIHVuc2lnbmVkIGludCBkZWxh
eWVkX3JlcXVlc3RzOw0KPj4gLXN0YXRpYyBpbnQgc29jayA9IC0xOw0KPj4gK2ludCBzb2Nr
ID0gLTE7DQo+IA0KPiBTaW1pbGFyIGNvbW1lbnQgdG8gJ2Zkcycgb24gdGhlIHByZXZpb3Vz
IHBhdGNoLiBUaGlzIG5hbWUgaXMgcXVpdGUgZ2VuZXJpYy4gQ2FuIA0KPiB3ZSBlaXRoZXIg
cGFzcyBpdCBhcyBhbiBhcmd1bWVudCAob3IgcmV0dXJuKSBvciByZW5hbWUgaXQ/DQoNCldo
YXQgYWJvdXQgdG8gbWFrZSBpdCBsb2NhbCB0byBwb3NpeC5jIGFuZCBhZGRpbmcgYSBnZXRf
c29ja2V0X2ZkKCkgZnVuY3Rpb24NCmZvciBpdHMgb25seSB1c2FnZSBpbiBjb3JlLmMgKHdp
dGggYSByZWxhdGVkIHN0dWIgaW4gbWluaW9zLmMsIG9mIGNvdXJzZSk/DQoNCg0KSnVlcmdl
bg0KDQo=
--------------iWBJJMLqhrg9TouHb0fJluT8
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------iWBJJMLqhrg9TouHb0fJluT8--

--------------PKvHsj8IqtIsQKshOKYbgtqo--

--------------BWt6rFLR0DQR2ByTci0NsstP
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXAl0wFAwAAAAAACgkQsN6d1ii/Ey9B
tQf/Uqix5ge9Sq0y0Myx7Sc52QCMgNCKRsOrGPoiT3J29lMVdVWB5Ab5CZb6YKDc2slVx4yoD42s
D1zN8vW05AVSeP0lzaQo5Iqx/dJxcmRxLGD1g9nud75NMBCUU+9C/Jhr5rf3zL7VSX36BmBKRxpn
KGXstAV+PDmMYiDsAkS7vfsul8Tf3vJGbSNjfVK/lqtI+c9OuooSOnOm6u7oUJLsR0dC6Tk8m4qC
Lzm96gGRkXQgJT7kzAzqvX4wTlJlYTvDgryS+3q+af+5Q/0QgNZE1demar90hiq9ZQZA2WMYPa9m
E3JBtpZapsqH5/8C7hqLvrU3BTxlEkdrXgig99eSOw==
=AVQq
-----END PGP SIGNATURE-----

--------------BWt6rFLR0DQR2ByTci0NsstP--


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 08:41:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 08:41:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675692.1051053 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWuXx-0004mn-0o; Mon, 05 Feb 2024 08:41:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675692.1051053; Mon, 05 Feb 2024 08:41:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWuXw-0004mg-Th; Mon, 05 Feb 2024 08:41:44 +0000
Received: by outflank-mailman (input) for mailman id 675692;
 Mon, 05 Feb 2024 08:41:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWuXv-0004ma-Kj
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 08:41:43 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6299c523-c402-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 09:41:42 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-3392b12dd21so2289051f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 00:41:42 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 t10-20020adff04a000000b003392206c808sm7519516wro.105.2024.02.05.00.41.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 00:41:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6299c523-c402-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707122501; x=1707727301; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dq7SqYO12ah/UBh39KyW+qD89O7v4w0Laqr7DaGhVzQ=;
        b=gllH8eVL+r6WEzkSCinKYJp+/fj//UE//ibBjjztPxHvVtkEGLmFxOuB9GROkFNt7r
         PaD6VDVsz+JyoUst2zggDyq+KzGwVxRg8oAnFyRu9R1w2H9aCs7QUlz5sl63lmOEUYqJ
         BKcCz7O0FD/LvpUc2cUqCdTOuXP3pjvJqKdZEILgnqfu0iRZLU3gBTB4tDnSTD/6bBsr
         KBct5W/U/1bpeB9CuOyFj0AhZp6+i/LeuVTFBeZn7L7aS7pxXUyq3F/1IyGynIRAFpBe
         ltLSfGizK5pDuwuc70Y+PqZl+GHzP+sOctBc5ffD9SHhXNveUmKN1OAxoSTqUX8JilAp
         ZFdA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707122501; x=1707727301;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dq7SqYO12ah/UBh39KyW+qD89O7v4w0Laqr7DaGhVzQ=;
        b=rgNRj5r1f93Hv8XDt/Hpl/ZZrTKZ/KnfoeN/R9e9LJ5g3XxGvaSh7ROOgI351lMJ3l
         MWhC6JHvf5YWeRC3Aub527+ELoJgZyyKwvxq1r9Aoj3CkaItnJ3umYfYwk7Dq4ur6NZG
         lxeCJvlSiqB8UigWbhIF+l2sUbpNjTrKYyg67Heia4AuC4Q8GU9PIxoE75QCH+J7PGNy
         MdHKhYmxIv7tjx3Y/0BSswywlvPfKYVFbwDQyzPxOayNTEnFPYf7Ps3wa+0BGSb4WlPH
         Syb2m1mGSX4bvlkWNjffv3g471AhbM02TWs5Fr4QnLFh2rAG2fVEUo6uXonS0u5odkVo
         ArJw==
X-Gm-Message-State: AOJu0YzTCfnw5MWoUKuUOfC8EFRSCAGHU7M4CAEuLEaFOGLroWuO7cUP
	AgkeBQtQ3vK1N1fMQb3EXqZseNKVPyXHYlNkDSv+oqPlrKf6WIX2LKW0MgaG7Q==
X-Google-Smtp-Source: AGHT+IERYNhqRBhYe/sSFOCCi7QhUyZ+694xveNPxlXhILfy6WXEMtaymLGCNP+C9+uB9m9n1CjYCQ==
X-Received: by 2002:adf:f8ca:0:b0:337:c4c2:8141 with SMTP id f10-20020adff8ca000000b00337c4c28141mr5276900wrq.35.1707122501412;
        Mon, 05 Feb 2024 00:41:41 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVeBzwgnU5fN02zEeBY2VKI5+MTkO0LCEVuDTqWE1JuUvLJkBMjmWdIK9AxU48GP4AgogkvVu2EQ+MomxoBuc9Nxw1xXs02VEvZ/dgd7mR8U3fmk0kn5eTy0PtVWl7wmCnhczT2lXPbjJRPH5QB7Yrn295t7evfa8NW5nkOQRbKLpHScwzWD7No+vJyZ8wNN0TNrz3baTcGBREuyGKkit1pYbe0GHGv94MAJrBkvVwt/qO8GvQnJXSHhLR8jw==
Message-ID: <52e2db77-88c4-47c5-85d7-34b0bb133b69@suse.com>
Date: Mon, 5 Feb 2024 09:41:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 4/9] x86/smp: move stack_base to cpu_data
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Krystian Hebel <krystian.hebel@3mdeb.com>, xen-devel@lists.xenproject.org
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <70e3b7c84a69a7ec52b3ed6314395165c281734c.1699982111.git.krystian.hebel@3mdeb.com>
 <d0fc568a-f54e-4480-a903-6407f4e3a5b6@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d0fc568a-f54e-4480-a903-6407f4e3a5b6@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 19:24, Julien Grall wrote:
> On 14/11/2023 17:50, Krystian Hebel wrote:
>> This location is easier to access from assembly. Having it close to
>> other data required during initialization has also positive (although
>> rather small) impact on prefetching data from RAM.
> 
> I understand your goal but...
> 
>> --- a/xen/arch/x86/include/asm/cpufeature.h
>> +++ b/xen/arch/x86/include/asm/cpufeature.h
> 
> ... cpufeature.h feels a rather odd place for storing the stack. I am 
> not entirely sure where else to place. Andrew, Jan, Roger?

Well, without having looked at the patch/series itself yet, I can only
say that if struct cpuinfo_x86 really is the place to put this
information, then it's unavoidable to have the field added in this
header. That said, it certainly feels like an abuse - there's nothing
in common with other (collected) data in this structure. "Easier to
access from assembly" may be a fair reason, but then I'd expect the
downsides of alternatives to be discussed explicitly. For example, a
simple new array might be as "easily" accessible from assembly.

>> @@ -37,6 +37,7 @@ struct cpuinfo_x86 {
>>       unsigned int phys_proc_id;         /* package ID of each logical CPU */
>>       unsigned int cpu_core_id;          /* core ID of each logical CPU */
>>       unsigned int compute_unit_id;      /* AMD compute unit ID of each logical CPU */
>> +    void *stack_base;
> 
> AFAICT, this means there will be a padding before stack_base and ...
> 
>>       unsigned short x86_clflush_size;
> 
> ... another one here. Is there any particular reason the new field 
> wasn't added at the end?

With ...

>>   } __cacheline_aligned;

... this I'm not exactly sure this is a problem right now (there may
be ample padding space anyway, yet I didn't go count). But I agree
with your comment in principle.

>> --- a/xen/arch/x86/smpboot.c
>> +++ b/xen/arch/x86/smpboot.c
>> @@ -75,13 +75,15 @@ static enum cpu_state {
>>   } cpu_state;
>>   #define set_cpu_state(state) do { smp_mb(); cpu_state = (state); } while (0)
>>   
>> -void *stack_base[NR_CPUS];
>> -
>>   void initialize_cpu_data(unsigned int cpu)
>>   {
>>       uint32_t apicid = cpu_physical_id(cpu);
>> +    void *stack = cpu_data[cpu].stack_base;
>> +
>>       cpu_data[cpu] = boot_cpu_data;
>> +
>>       cpu_physical_id(cpu) = apicid;
>> +    cpu_data[cpu].stack_base = stack;
>>   }
>>   
>>   static bool smp_store_cpu_info(unsigned int id)
>> @@ -579,8 +581,6 @@ static int do_boot_cpu(int apicid, int cpu)
>>           printk("Booting processor %d/%d eip %lx\n",
>>                  cpu, apicid, start_eip);
>>   
>> -    stack_start = stack_base[cpu] + STACK_SIZE - sizeof(struct cpu_info);
>> -
> 
> You remove this line because I can't quite figure out where stack_start 
> is now set. This is used...

This line sets a global variable, which ...

>> @@ -856,7 +856,7 @@ int setup_cpu_root_pgt(unsigned int cpu)
>>   
>>       /* Install direct map page table entries for stack, IDT, and TSS. */
>>       for ( off = rc = 0; !rc && off < STACK_SIZE; off += PAGE_SIZE )
>> -        rc = clone_mapping(__va(__pa(stack_base[cpu])) + off, rpt);
>> +        rc = clone_mapping(__va(__pa(cpu_data[cpu].stack_base)) + off, rpt);
>>   
>>       if ( !rc )
>>           rc = clone_mapping(idt_tables[cpu], rpt);
>> @@ -1007,10 +1007,10 @@ static void cpu_smpboot_free(unsigned int cpu, bool remove)
>>           FREE_XENHEAP_PAGE(per_cpu(gdt, cpu));
>>           FREE_XENHEAP_PAGE(idt_tables[cpu]);
>>   
>> -        if ( stack_base[cpu] )
>> +        if ( cpu_data[cpu].stack_base )
>>           {
>> -            memguard_unguard_stack(stack_base[cpu]);
>> -            FREE_XENHEAP_PAGES(stack_base[cpu], STACK_ORDER);
>> +            memguard_unguard_stack(cpu_data[cpu].stack_base);
>> +            FREE_XENHEAP_PAGES(cpu_data[cpu].stack_base, STACK_ORDER);
>>           }
>>       }
>>   }
>> @@ -1044,11 +1044,11 @@ static int cpu_smpboot_alloc(unsigned int cpu)
>>       if ( node != NUMA_NO_NODE )
>>           memflags = MEMF_node(node);
>>   
>> -    if ( stack_base[cpu] == NULL &&
>> -         (stack_base[cpu] = cpu_alloc_stack(cpu)) == NULL )
>> +    if ( cpu_data[cpu].stack_base == NULL &&
>> +         (cpu_data[cpu].stack_base = cpu_alloc_stack(cpu)) == NULL )
>>               goto out;
>>   
>> -    info = get_cpu_info_from_stack((unsigned long)stack_base[cpu]);
>> +    info = get_cpu_info_from_stack((unsigned long)cpu_data[cpu].stack_base);
> 
> ... here.

... pretty clearly is not used here (anymore). Instead I'd raise the
question of what the remaining purpose of that variable then is.
Looking through updates this patch alone makes to use sites of
stack_start, it's unclear whether the use from assembly code has gone
away already - brief checking suggests it hasn't.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 08:56:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 08:56:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675696.1051063 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWumJ-0006WS-7M; Mon, 05 Feb 2024 08:56:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675696.1051063; Mon, 05 Feb 2024 08:56:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWumJ-0006WL-4E; Mon, 05 Feb 2024 08:56:35 +0000
Received: by outflank-mailman (input) for mailman id 675696;
 Mon, 05 Feb 2024 08:56:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWumH-0006WF-Qc
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 08:56:33 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 74363fa6-c404-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 09:56:30 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-40fcb4ef7bcso21744465e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 00:56:30 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 o13-20020a05600c4fcd00b0040fc56712e8sm7855258wmq.17.2024.02.05.00.56.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 00:56:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74363fa6-c404-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707123390; x=1707728190; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sszY4sOF2GoYgL/RsyF4xdROBXpl0OIMy6KUXcqqU8k=;
        b=du/nSgCH91ovTj0cClVN5K72MYKTezj9niAxXId9ObnzNFjRFTF2rd6AbdTSB1Hjgq
         sGnqahQ8Iea5h5+y8rko4egXypLN0AguIXmvFV5LtxoKFJsIhpF3kTRVMc4Da4ViRcg1
         fhdhSqTygGgFNgFC72h4Xeh1udsGNJwoUKhyO3B4XqQuZ0B88fEwtcD59fF40OA0iOwk
         jBNYtSh5RSC4Mpif6evvu1v/2zx3QBKqnnP7TDcM3L3/CW1WKD/gr4o68TglH1WMKYsd
         5JLRJXNosBhC0Uh/LFDd+RRvHBvhFDtsBvopbbVpkpPupDBvYEE0eLw5umrwyhPEgBsf
         mlaA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707123390; x=1707728190;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sszY4sOF2GoYgL/RsyF4xdROBXpl0OIMy6KUXcqqU8k=;
        b=c9T6Ep4iyXo/2A0vfHNsjwl1RDa/gjyBQu4vfgM2s6py5MbylTAnB46NP7wjzS5Nkn
         DnSP0NFoo9b8vADgYdfLMkF8GLVmyF27majUifbqUe3jMuj5q5++frje8UvfgpcOEXrR
         AytCXDItdP0Dtsk5sNxk3pyU5HRuUtHJyBrDUKYaVY9wsKI6Yi6VefoEp64K2cjiFWn5
         pjut9FHt8ws0iRELtraBKEWWBbkV1EQwmiv+d3IB9+RwmqJKmS58cfoo0oB27IPP/ROQ
         XPwXELKCDTKeLq8p75d0hjKCOIjWEUdi0EokTUWEndaSuGr8c/VHGtP5WM30DKrx0HME
         XRIw==
X-Gm-Message-State: AOJu0YzeW6w6gWueNKySETRX/7rO+B6+MwJrQQjzM2Hr3AcdbVUtAldt
	joDvJyN5EU7GMeUeNtmJa+xTt9GRcag4jlkWKELiQaUlPTFsrXFpJ8foRjci0A==
X-Google-Smtp-Source: AGHT+IFbGtdMEMaOOnK1rbWeOXEwRYXM2EKm0fzDp3scDp2L3DDp4HYpZl66mCdmvObA2xIDoRzOSA==
X-Received: by 2002:a05:600c:6022:b0:40f:c6ab:6c72 with SMTP id az34-20020a05600c602200b0040fc6ab6c72mr4023625wmb.0.1707123389954;
        Mon, 05 Feb 2024 00:56:29 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVqa20xLYiGunQdYxQRvSpZUOaU2t/FSGmdh1TApGPb81wuva5skTPXmCWaW0i/DjYYnOAHgtHVLBQwXN4jLc51xsmoYQDgS+ScQSEmCFKYI8PyfXS96hWSBULqhw6znwPs3wrqIsSh8+Kceyt9PTjWn66qbtS2C72zcxlvMxGTzSZI
Message-ID: <e55d5125-72d7-4608-b7a2-6af73982a69d@suse.com>
Date: Mon, 5 Feb 2024 09:56:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/bitmap: Deduplicate __bitmap_weight() implementations
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240202153816.772594-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240202153816.772594-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 16:38, Andrew Cooper wrote:
> We have two copies of __bitmap_weight() that differ by whether they make
> hweight32() or hweight64() calls, yet we already have hweight_long() which is
> the form that __bitmap_weight() wants.
> 
> Fix hweight_long() to return unsigned int like all the other hweight helpers,
> and fix __bitmap_weight() to used unsigned integers.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon Feb 05 08:57:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 08:57:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675699.1051073 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWunV-00079H-GJ; Mon, 05 Feb 2024 08:57:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675699.1051073; Mon, 05 Feb 2024 08:57:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWunV-00079A-DK; Mon, 05 Feb 2024 08:57:49 +0000
Received: by outflank-mailman (input) for mailman id 675699;
 Mon, 05 Feb 2024 08:57:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWunT-000794-VC
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 08:57:47 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a1ce1dd0-c404-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 09:57:47 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-40fdd11475aso2794435e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 00:57:47 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 o13-20020a05600c4fcd00b0040fc56712e8sm7855258wmq.17.2024.02.05.00.57.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 00:57:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1ce1dd0-c404-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707123466; x=1707728266; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IFg9otBeA8ysHfCQq1m6m0GEFd8ZAtUS1+v10mm+WXc=;
        b=LMK7touVsiP0X03Mr2Cp72Ftvppf8ZJAtfT9hWA789m+Jb4Rr9KnsMLqLDkCTAuYOG
         5oV2m+3qktcN/ycPz5iR98UeLQC1QUqFzJLV9mf7qTIAs8uFaXIHRNpwB13UWLa0RsMd
         2W0ZCCaNLZKb5s/6Q3b0e6VziSE9cAJmOrdaMh+Z7MqCwLriLDb9fe3Q61E5cJKSL54J
         q9zTt6V/VN+4BlGN7vpr3oDTXBPlwmKa7dgUpaVL3vBQaWHk9tK8j40xQ+8UpqIpJfTJ
         P2aVdbmf2WT5J6ouR5A5tO5t5yT1MM0aA7eXHsRdMZxzEL5+2MJ2C+7hS/0wdAZS7hon
         pphw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707123466; x=1707728266;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IFg9otBeA8ysHfCQq1m6m0GEFd8ZAtUS1+v10mm+WXc=;
        b=ZI2OULSdFVWjfrrKt/VR+IOh1OrsFXe56pJSznh1o0LoSvdnF81Xd1w1xg1x6NcFD9
         718h+fk936Ea6T7xTR4XZxyDxfl88ByRr2K5Lib0jdseAJzbw4izZKPcLF81m/+S/7Qf
         P7t8bQLT5nkCBGSU873ddaBIb8aEvyDt9k7oJqUpYn+Xf6Tuo7D/D9qBZvWbW6TYUkds
         x/ISAePllVA+/Jzy4K/Bw9uOyUOyivM7ZhN9aFappwianegG4+kSyFySyChl5TYpw/eG
         mvzr3u00JsYTmsQsO8IwiDQEmA5pvEGfGt+J0S3rGgmlZJLpiWP0JqWJC9RPBFQGNzm9
         PcGw==
X-Gm-Message-State: AOJu0YzDfbquSOXyQaryRJxsCoHdGk0MlZPx2HXO3FT3AYg5Bw5a8SZq
	RWucsM/q5YcJ4WCYHe/kpNOa5TNvPmQUjdttwt2B9gBZMoWckwtbsVeR11qmig==
X-Google-Smtp-Source: AGHT+IEBE+poTjfDzcz8CM8zzsE1Etore0Rh9JzoWw0NLBTK5r2+VRgNuenyaE1rDtWZ68LhUa6p0Q==
X-Received: by 2002:a7b:cb8c:0:b0:40f:afe1:c9f8 with SMTP id m12-20020a7bcb8c000000b0040fafe1c9f8mr4233297wmi.12.1707123466546;
        Mon, 05 Feb 2024 00:57:46 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUvxx7yh3O/gQTAspWRu/wbVDbqLOQai3dxS4/GUgh9ClbN3BWz5QpCA+fJNgslONHm7XxiqOZQ0lI3JsVtuSknjOZQFMqtN0wad0McfIy36m1Cn0Gw+ehfb58=
Message-ID: <ed5ab7f6-1b5a-4953-8add-8a2df41b748c@suse.com>
Date: Mon, 5 Feb 2024 09:57:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/ucode: Remove accidentally introduced tabs
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240202180003.1295735-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240202180003.1295735-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 19:00, Andrew Cooper wrote:
> Fixes: cf7fe8b72dea ("x86/ucode: Fix stability of the raw CPU Policy rescan")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

I had spotted those as odd while backporting, but didn't consider the
issue bad enough to raise it when the change had gone in already anyway.

Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 09:21:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 09:21:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675705.1051083 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvA0-0002ly-C2; Mon, 05 Feb 2024 09:21:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675705.1051083; Mon, 05 Feb 2024 09:21:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvA0-0002lr-8M; Mon, 05 Feb 2024 09:21:04 +0000
Received: by outflank-mailman (input) for mailman id 675705;
 Mon, 05 Feb 2024 09:21:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWv9z-0002ll-PR
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 09:21:03 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e177977b-c407-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 10:21:02 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id BF4A222030;
 Mon,  5 Feb 2024 09:21:01 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 97912136F5;
 Mon,  5 Feb 2024 09:21:01 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id XPt7I32owGWwHAAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 09:21:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e177977b-c407-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707124861; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=Tp3qKGzgjZmnMxQ5Z5H2MgUI9LsT+czRWEroxqF2alo=;
	b=Ctsl2Jfylj/BkryaNlHByhwwiDEknNn9mLuAT4Ih3t/AW3LLvUZjKFnL8dp8wxDVf67X4O
	ZYvcFYum213ovMK0sGIIyK7kq9xu1wxEa3fenB0tAbEM+aY2CJ/cBPyxZhWOn4ZmzgzOKX
	bb4kToS3XGJxd3BWhgHoHH6Qe6eVYf0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707124861; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=Tp3qKGzgjZmnMxQ5Z5H2MgUI9LsT+czRWEroxqF2alo=;
	b=Ctsl2Jfylj/BkryaNlHByhwwiDEknNn9mLuAT4Ih3t/AW3LLvUZjKFnL8dp8wxDVf67X4O
	ZYvcFYum213ovMK0sGIIyK7kq9xu1wxEa3fenB0tAbEM+aY2CJ/cBPyxZhWOn4ZmzgzOKX
	bb4kToS3XGJxd3BWhgHoHH6Qe6eVYf0=
Message-ID: <c4d5b0e0-e726-4ab2-9482-f284f2b92674@suse.com>
Date: Mon, 5 Feb 2024 10:21:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 25/33] tools/xenstored: move all socket handling into
 posix.c
Content-Language: en-US
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-26-jgross@suse.com>
 <dcc477bd-b7cf-40e8-85be-3cde5b9cfd61@xen.org>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <dcc477bd-b7cf-40e8-85be-3cde5b9cfd61@xen.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------K7eZ8ovPuUvz2bJcAVCFUx8q"
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=Ctsl2Jfy
X-Spamd-Result: default: False [-5.20 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 HAS_ATTACHMENT(0.00)[];
	 BAYES_HAM(-3.00)[100.00%];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 MX_GOOD(-0.01)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 SIGNED_PGP(-2.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[];
	 MIME_UNKNOWN(0.10)[application/pgp-keys]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: BF4A222030
X-Spam-Level: 
X-Spam-Score: -5.20
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------K7eZ8ovPuUvz2bJcAVCFUx8q
Content-Type: multipart/mixed; boundary="------------2RAwL0wvq4E7Ep80rjsE4RtN";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
Message-ID: <c4d5b0e0-e726-4ab2-9482-f284f2b92674@suse.com>
Subject: Re: [PATCH v3 25/33] tools/xenstored: move all socket handling into
 posix.c
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-26-jgross@suse.com>
 <dcc477bd-b7cf-40e8-85be-3cde5b9cfd61@xen.org>
In-Reply-To: <dcc477bd-b7cf-40e8-85be-3cde5b9cfd61@xen.org>

--------------2RAwL0wvq4E7Ep80rjsE4RtN
Content-Type: multipart/mixed; boundary="------------0B01UgYYQ45VEXIBX7L0Ul5g"

--------------0B01UgYYQ45VEXIBX7L0Ul5g
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMjYuMDEuMjQgMTc6MjIsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4gSGkgSnVlcmdlbiwN
Cj4gDQo+IE9uIDA0LzAxLzIwMjQgMDk6MDAsIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+PiBB
bGwgb2YgdGhlIHNvY2tldCBoYW5kbGluZyBpcyBuZWVkZWQgb25seSB3aGVuIHJ1bm5pbmcg
YXMgZGFlbW9uLg0KPj4NCj4+IE1vdmUgaXQgaW50byBwb3NpeC5jLCBhbGxvd2luZyB0byBy
ZW1vdmUgdGhlIE5PX1NPQ0tFVFMgbWFjcm8uDQo+Pg0KPj4gU2lnbmVkLW9mZi1ieTogSnVl
cmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuY29tPg0KPj4gLS0tDQo+PiBWMzoNCj4+IC0gbmV3
IHBhdGNoDQo+PiAtLS0NCj4+IMKgIHRvb2xzL3hlbnN0b3JlZC9NYWtlZmlsZS5jb21tb24g
fMKgwqAgNCAtDQo+PiDCoCB0b29scy94ZW5zdG9yZWQvY29yZS5jwqDCoMKgwqDCoMKgwqDC
oMKgIHwgMTU2ICstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+PiDCoCB0b29s
cy94ZW5zdG9yZWQvY29yZS5owqDCoMKgwqDCoMKgwqDCoMKgIHzCoMKgIDggKy0NCj4+IMKg
IHRvb2xzL3hlbnN0b3JlZC9kb21haW4uY8KgwqDCoMKgwqDCoMKgIHzCoMKgIDkgKy0NCj4+
IMKgIHRvb2xzL3hlbnN0b3JlZC9taW5pb3MuY8KgwqDCoMKgwqDCoMKgIHzCoMKgIDYgKysN
Cj4+IMKgIHRvb2xzL3hlbnN0b3JlZC9wb3NpeC5jwqDCoMKgwqDCoMKgwqDCoCB8IDE1MyAr
KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrDQo+PiDCoCA2IGZpbGVzIGNoYW5nZWQs
IDE2NyBpbnNlcnRpb25zKCspLCAxNjkgZGVsZXRpb25zKC0pDQo+Pg0KPj4gZGlmZiAtLWdp
dCBhL3Rvb2xzL3hlbnN0b3JlZC9NYWtlZmlsZS5jb21tb24gYi90b29scy94ZW5zdG9yZWQv
TWFrZWZpbGUuY29tbW9uDQo+PiBpbmRleCAxODlhYjgxYjhkLi5lZjYzZWY2NTBjIDEwMDY0
NA0KPj4gLS0tIGEvdG9vbHMveGVuc3RvcmVkL01ha2VmaWxlLmNvbW1vbg0KPj4gKysrIGIv
dG9vbHMveGVuc3RvcmVkL01ha2VmaWxlLmNvbW1vbg0KPj4gQEAgLTE2LDEwICsxNiw2IEBA
IENGTEFHUyArPSAkKENGTEFHU19saWJ4ZW5jdHJsKQ0KPj4gwqAgQ0ZMQUdTICs9ICQoQ0ZM
QUdTX2xpYnhlbmd1ZXN0KQ0KPj4gwqAgQ0ZMQUdTICs9ICQoQ0ZMQUdTX2xpYnhlbnRvb2xj
b3JlKQ0KPj4gLWlmZGVmIENPTkZJR19TVFVCRE9NDQo+PiAtQ0ZMQUdTICs9IC1ETk9fU09D
S0VUUz0xDQo+PiAtZW5kaWYNCj4+IC0NCj4+IMKgICQoWEVOU1RPUkVEX09CSlMteSk6IENG
TEFHUyArPSAkKENGTEFHU19saWJ4ZW5nbnR0YWIpDQo+PiDCoCB4ZW5zdG9yZWQuYTogJChY
RU5TVE9SRURfT0JKUy15KQ0KPj4gZGlmZiAtLWdpdCBhL3Rvb2xzL3hlbnN0b3JlZC9jb3Jl
LmMgYi90b29scy94ZW5zdG9yZWQvY29yZS5jDQo+PiBpbmRleCA2NTBjMWQxNTc0Li44ZmYx
ZmFjY2VmIDEwMDY0NA0KPj4gLS0tIGEvdG9vbHMveGVuc3RvcmVkL2NvcmUuYw0KPj4gKysr
IGIvdG9vbHMveGVuc3RvcmVkL2NvcmUuYw0KPj4gQEAgLTIwLDEwICsyMCw2IEBADQo+PiDC
oCAjaW5jbHVkZSA8c3lzL3R5cGVzLmg+DQo+PiDCoCAjaW5jbHVkZSA8c3lzL3N0YXQuaD4N
Cj4+IMKgICNpbmNsdWRlIDxwb2xsLmg+DQo+PiAtI2lmbmRlZiBOT19TT0NLRVRTDQo+PiAt
I2luY2x1ZGUgPHN5cy9zb2NrZXQuaD4NCj4+IC0jaW5jbHVkZSA8c3lzL3VuLmg+DQo+PiAt
I2VuZGlmDQo+PiDCoCAjaW5jbHVkZSA8c3lzL3RpbWUuaD4NCj4+IMKgICNpbmNsdWRlIDx0
aW1lLmg+DQo+PiDCoCAjaW5jbHVkZSA8dW5pc3RkLmg+DQo+PiBAQCAtNjEsNyArNTcsNyBA
QCBzdGF0aWMgdW5zaWduZWQgaW50IGN1cnJlbnRfYXJyYXlfc2l6ZTsNCj4+IMKgIHN0YXRp
YyB1bnNpZ25lZCBpbnQgbnJfZmRzOw0KPj4gwqAgc3RhdGljIHVuc2lnbmVkIGludCBkZWxh
eWVkX3JlcXVlc3RzOw0KPj4gLXN0YXRpYyBpbnQgc29jayA9IC0xOw0KPj4gK2ludCBzb2Nr
ID0gLTE7DQo+IA0KPiBTaW1pbGFyIGNvbW1lbnQgdG8gJ2Zkcycgb24gdGhlIHByZXZpb3Vz
IHBhdGNoLiBUaGlzIG5hbWUgaXMgcXVpdGUgZ2VuZXJpYy4gQ2FuIA0KPiB3ZSBlaXRoZXIg
cGFzcyBpdCBhcyBhbiBhcmd1bWVudCAob3IgcmV0dXJuKSBvciByZW5hbWUgaXQ/DQoNClR1
cm5zIG91dCB0aGlzIHBhdGNoIG5vdyBtYWtlcyB0aGUgImZkcyIgc29sdXRpb24gbW9yZSBj
b21wbGljYXRlZDogSSdkIG5lZWQgdG8NCmFkZCBmZHMgYXMgYW4gYWRkaXRpb25hbCBwYXJh
bWV0ZXIgdG8gdGhlIC5jYW5fd3JpdGUoKSBhbmQgLmNhbl9yZWFkKCkgY2FsbGJhY2tzDQpv
ZiBzdHJ1Y3QgaW50ZXJmYWNlX2Z1bmNzLg0KDQpJIGNhbiBkbyB0aGF0LCBidXQgYW4gYWx0
ZXJuYXRpdmUgd291bGQgYmUgdG8ganVzdCByZW5hbWUgZmRzIHRvIGUuZy4gcG9sbF9mZHMu
DQoNCkRvIHlvdSBoYXZlIGFueSBwcmVmZXJlbmNlPw0KDQpJbiBjYXNlIHdlIHBhc3MgZmRz
IGFzIGEgcGFyYW1ldGVyLCBJJ2QgYmUgaW5jbGluZWQgdG8gbWFrZSBpdCBldmVuIGxvY2Fs
IHRvIHRoZQ0KbWFpbigpIGZ1bmN0aW9uLg0KDQoNCkp1ZXJnZW4NCg==
--------------0B01UgYYQ45VEXIBX7L0Ul5g
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------0B01UgYYQ45VEXIBX7L0Ul5g--

--------------2RAwL0wvq4E7Ep80rjsE4RtN--

--------------K7eZ8ovPuUvz2bJcAVCFUx8q
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXAqH0FAwAAAAAACgkQsN6d1ii/Ey/u
MQf9GmdF9/BRxEzqIZpm6TfF9eC7SxPhAZn9TYtjunQURxZq+d3vW56jq3tPNOFFGQfpq6sSetlc
jXgTITkAXfRKhvoYfiMyOp5lOL0BU4Wz7B7sFp7aSk8GGX8VPRW1j7o2k6V6W0kTaBMetogkUOxy
TDe/h2pu+G4HIS4UDFdSKd8Ye/gS79g/ri3BZHP+ox+rbLdnyrmuFJckS8qtVa7iu/DXmmMB0k6Y
c9nMTU0J6JFZWyOgkwGsq5nwvfEeOhi6gxUYxGn048PAgDLff8m95IBICb2Po3UKf+xfFvfHMNHL
PW+f8EbkavGoE6x0TjIFMIPlrP98tAU1F+cO64P2Cg==
=CQn2
-----END PGP SIGNATURE-----

--------------K7eZ8ovPuUvz2bJcAVCFUx8q--


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 09:21:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 09:21:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675706.1051093 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvAF-00035F-MJ; Mon, 05 Feb 2024 09:21:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675706.1051093; Mon, 05 Feb 2024 09:21:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvAF-000356-J2; Mon, 05 Feb 2024 09:21:19 +0000
Received: by outflank-mailman (input) for mailman id 675706;
 Mon, 05 Feb 2024 09:21:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWvAE-00034W-FV
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 09:21:18 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e94ccb89-c407-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 10:21:15 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-33adec41b55so2643538f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 01:21:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z11-20020a5d4c8b000000b0033b3d726d41sm1548046wrs.104.2024.02.05.01.21.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 01:21:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e94ccb89-c407-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707124875; x=1707729675; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=j2Q/2D/uhkEc/IQYeaxz94TrTCI22fhTLhvc0BslLEA=;
        b=ceXwaxlyCW5g5ySp8VEF9RJ+OMGAkSFe7I/P70zyGxeAt878sG3lRe9XD5hxUeP1xy
         uE2XWfWwG8UkPmURlyVxOtX6HbeVa2cDuOgDfGvAK2Ba9nLGUu++22yf/iCXRAyFJbTG
         q8XrajsJH9TDbuRBZR/hl7y6sbjOno2P/DNaZL/ysrHFEmTl0/vOyxSPI6VFpaP7Vqpy
         fbSxQaN80So4FcZ/yN39h40xMSLhqaKIYEDmBKeFfwVkHzaMBhG9g8usf/Kr8XnWTH2m
         lqd190ePS1h44hgi4O9oZemXzif8rvtDtdd+b0YVPEU5i9pq0775HwWP1w354l7S4niR
         JvAw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707124875; x=1707729675;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=j2Q/2D/uhkEc/IQYeaxz94TrTCI22fhTLhvc0BslLEA=;
        b=Mr5Ddrud+7Z12N7tJmd5KpyxX+9piJ/rH3tGfvuQCVuBjhLUNqExBZPzA296HJEYZC
         8r2u8pCUowq7zifnAe6GrpIFmPna5HjYAxXEdNDAuRuE4s6Lda9xoxsjkTbWnyRZp1JV
         mHWy/yoYG1fUPtNu7Czv36DzOMHKLslJ17keMsHpiEmWs3GuWxpHYSfHfCbl3Qqx8HYw
         GhS+8LJHZBdvPcUAQCIV1h1xa0tFKbja+MxXns7zy+za3evRYWvQHt+tL60fi3mdYN6b
         FSwm/Be9SyKK41QW8YFuOlRrYSpe1/6J3Rgw/KxrVGxUfJPDGnM6Leu5Oin+eb33WPe/
         HxZA==
X-Gm-Message-State: AOJu0Yw9Q2B1iU0sUHk5ysREO/aBwVGsGHlVEDq8QUm6QD9CxwMre4rr
	VGol+abTIscSQpAgUPt8+1AVZstYXdxhWnAkHbXANFl+bBaiLF7Tli6X0tXLT1M2WptVpyDKi0s
	=
X-Google-Smtp-Source: AGHT+IFyGq0q46qH+HMAV51w/HNQwcRlTZ6iiKJ49W2J8Te4rvK82boWbnSq4dl8koxPX+FG2rwycQ==
X-Received: by 2002:adf:f009:0:b0:33b:14fa:53ea with SMTP id j9-20020adff009000000b0033b14fa53eamr7071779wro.33.1707124874844;
        Mon, 05 Feb 2024 01:21:14 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXL4JSi5sFMa4vvqImThemsdDuYcoSFErj2K9v8bUigUkhPoS5VM/pyFrpHFw5bW/9gl/bEYVgLBBWBXzg6stdpFPH+SbxVLHHiTOJi2TdoyID02uwzhw3L3IHy4hoV18jzIg30hw6priMcnULHc/Vdgqucxv1nmtr/KhFiwgRFBghNLJlvYpJNJTL/MjBL/AYLADBlUt2D8q023ToVq0F7iIsjFGVPUXhLUpQt6ygcX5X+QQizKmHk6kKSXU54MzVG3ywsSGQvRwQsfzdOx6r2OQ4CvUmU
Message-ID: <101c9449-8eab-477b-8f41-fb6c2c70c97a@suse.com>
Date: Mon, 5 Feb 2024 10:21:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 01/15] xen/common: add cache coloring common code
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-2-carlo.nonato@minervasys.tech>
 <42eabd87-b245-44e3-9945-40b1a5c5eaf5@suse.com>
 <CAG+AhRVjiqjae+Lwdk71OkF=X93m+CNLFYpK65h+O1kozOgtew@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRVjiqjae+Lwdk71OkF=X93m+CNLFYpK65h+O1kozOgtew@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 03.02.2024 11:57, Carlo Nonato wrote:
> On Wed, Jan 31, 2024 at 4:57 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 29.01.2024 18:17, Carlo Nonato wrote:
>>> +Background
>>> +**********
>>> +
>>> +Cache hierarchy of a modern multi-core CPU typically has first levels dedicated
>>> +to each core (hence using multiple cache units), while the last level is shared
>>> +among all of them. Such configuration implies that memory operations on one
>>> +core (e.g. running a DomU) are able to generate interference on another core
>>> +(e.g .hosting another DomU). Cache coloring allows eliminating this
>>> +mutual interference, and thus guaranteeing higher and more predictable
>>> +performances for memory accesses.
>>> +The key concept underlying cache coloring is a fragmentation of the memory
>>> +space into a set of sub-spaces called colors that are mapped to disjoint cache
>>> +partitions. Technically, the whole memory space is first divided into a number
>>> +of subsequent regions. Then each region is in turn divided into a number of
>>> +subsequent sub-colors. The generic i-th color is then obtained by all the
>>> +i-th sub-colors in each region.
>>> +
>>> +::
>>> +
>>> +                            Region j            Region j+1
>>> +                .....................   ............
>>> +                .                     . .
>>> +                .                       .
>>> +            _ _ _______________ _ _____________________ _ _
>>> +                |     |     |     |     |     |     |
>>> +                | c_0 | c_1 |     | c_n | c_0 | c_1 |
>>> +           _ _ _|_____|_____|_ _ _|_____|_____|_____|_ _ _
>>> +                    :                       :
>>> +                    :                       :...         ... .
>>> +                    :                            color 0
>>> +                    :...........................         ... .
>>> +                                                :
>>> +          . . ..................................:
>>> +
>>> +There are two pragmatic lesson to be learnt.
>>> +
>>> +1. If one wants to avoid cache interference between two domains, different
>>> +   colors needs to be used for their memory.
>>> +
>>> +2. Color assignment must privilege contiguity in the partitioning. E.g.,
>>> +   assigning colors (0,1) to domain I  and (2,3) to domain  J is better than
>>> +   assigning colors (0,2) to I and (1,3) to J.
>>
>> I can't connect this 2nd point with any of what was said above.
> 
> If colors are contiguous then a greater spatial locality is achievable. You
> mean we should better explain this?

Yes, but not just that. See how you using "must" in the text contradicts you
now suggesting this is merely an optimization.

>>> +How to compute the number of colors
>>> +***********************************
>>> +
>>> +To compute the number of available colors for a specific platform, the size of
>>> +an LLC way and the page size used by Xen must be known. The first parameter can
>>> +be found in the processor manual or can be also computed dividing the total
>>> +cache size by the number of its ways. The second parameter is the minimum
>>> +amount of memory that can be mapped by the hypervisor,
>>
>> I find "amount of memory that can be mapped" quite confusing here. Don't you
>> really mean the granularity at which memory can be mapped?
> 
> Yes that's what I wanted to describe. I'll change it.
> 
>>> thus dividing the way
>>> +size by the page size, the number of total cache partitions is found. So for
>>> +example, an Arm Cortex-A53 with a 16-ways associative 1 MiB LLC, can isolate up
>>> +to 16 colors when pages are 4 KiB in size.
>>
>> I guess it's a matter of what one's use to, but to me talking of "way size"
>> and how the calculation is described is, well, unusual. What I would start
>> from is the smallest entity, i.e. a cache line. Then it would be relevant
>> to describe how, after removing the low so many bits to cover for cache line
>> size, the remaining address bits are used to map to a particular set. It
>> looks to me as if you're assuming that this mapping is linear, using the
>> next so many bits from the address. Afaik this isn't true on various modern
>> CPUs; instead hash functions are used. Without knowing at least certain
>> properties of such a hash function, I'm afraid your mapping from address to
>> color isn't necessarily guaranteeing the promised isolation. The guarantee
>> may hold for processors you specifically target, but then I think in this
>> description it would help if you would fully spell out any assumptions you
>> make on how hardware maps addresses to elements of the cache.
> 
> You're right, we are assuming a linear mapping. We are going to review and
> extend the documentation in order to fully specify when coloring can be
> applied.
> 
> About the "way size" it's a way of summarizing all the parameters into one.
> We could ask for different cache parameters as you said, but in the end what
> we are interested in is how many partitions is the cache capable of isolate
> and how big they are. The answer is, in theory, as many partitions as the
> number of sets, each one as big as a cache line, bacause we can't have
> isolation inside a set.
> Then memory mapping comes into place and the minimum granularity at which
> mapping can happen actually lowers the number of partitions.
> To recap we can isolate:
>     nr_sets * line_size / page_size
> Then we simply named:
>     way_size = nr_sets * line_size
> Another way of computing it:
>     way_size = cache_size / nr_ways
> 
> We are ok with having two parameters: cache_size and nr_ways which are even
> easier and intuitive to find for a normal user.

Right, that's the aspect I was actually after.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 09:28:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 09:28:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675715.1051103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvGx-0004E0-FA; Mon, 05 Feb 2024 09:28:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675715.1051103; Mon, 05 Feb 2024 09:28:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvGx-0004Dt-Bx; Mon, 05 Feb 2024 09:28:15 +0000
Received: by outflank-mailman (input) for mailman id 675715;
 Mon, 05 Feb 2024 09:28:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWvGw-0004Ct-EW
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 09:28:14 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e25b0ce9-c408-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 10:28:13 +0100 (CET)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-33b406e16d4so262648f8f.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 01:28:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n2-20020a5d4c42000000b0033b39cba5e4sm2739049wrt.116.2024.02.05.01.28.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 01:28:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e25b0ce9-c408-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707125293; x=1707730093; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=NuRT6PO9FlqzfTLIgTmJkdWYkbG+oUe7ID0Ea5F8FhM=;
        b=dQlpDUSoK8AZbC2HhZMsQCHYvuHrlbCR4HQPjHk+2KKfdxXQjOqJyh01b3iNgr3TtT
         x1FPpwQWTLSQYedop9XIiKDOsakc6d+oPrIzF4/Ii3y3j7YXWH8/m4LzImdWTGbZtAcm
         GUNWwrlxWe9ZMKBbG1giEhSSTfC2jr7ENH+857fJdnOXqDEeSpFrTfN9H4Gm7TfDRTAZ
         W1GOLeD0mrrENSYoGnpR3nvBnEXFrUJC0dVOl2IrCt60UHTnIz7Yb/kXR6p6qPo4jJz4
         zgzlIIgH20uiu2r5c3xrGp02EU1PGy6S65ejv8ZzdeNKA4gjgrzQI1D8hKQjpWl15aOF
         ihOw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707125293; x=1707730093;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=NuRT6PO9FlqzfTLIgTmJkdWYkbG+oUe7ID0Ea5F8FhM=;
        b=D5CDvmlRhJjdcECiYi/Y526O4uioiAkckC1YvyaaRSc6GncAdG14NqjqYSVRP8nwkG
         X8vnKlGFsnP7LOcWIjHPxy2oPLtf1LWb+VTiDx/JN03xw/Ib79eugw7lhMpFEqrx5t4F
         fGj85hB/4a5MVPjhlynhkTamKONOo06cRW97yoRoMY4wLD+IKlrrKiZ0pjoJepFmrK/6
         TleiloxR2fG/5DY8qm905qmOHhzehjcbUCN46VnF1zRDCzfQt9nvE48VqktRr31KXTUA
         615m/owP79/zLEbnY2pVUy+NOHIRHcmTXnEIIvo5oIfilMZeFj5anj/lD1zqM7ThkmHs
         WSwg==
X-Gm-Message-State: AOJu0Ywqnhe1O39gL8qVaZ4g9gy4Q8fszgopZT1bin0fneJnaoO7wBFJ
	grU8DQs7kCwqcvRgF2HjhyeKXC9tI/2OxrKTEI+yBw6iRHU35sQ+QL5WEesd6w==
X-Google-Smtp-Source: AGHT+IFc7YV6GyTNBo1TERsFx+qmUms3imNQX8kvzJl7L0pUwmaMOGJqIwjMCX74SVgd+y4M86JoEg==
X-Received: by 2002:a05:6000:22f:b0:33b:3040:df03 with SMTP id l15-20020a056000022f00b0033b3040df03mr2990275wrz.46.1707125292759;
        Mon, 05 Feb 2024 01:28:12 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXeMCyKqtCCJQ2XZ10rVc3LbkRbD+YsHU+P54QqFPcYt1cgnEaLAP4eyhGoHdg8NRaCaKkU2HNyU8LmNLixV9yJlGiDoTkO5d/LQA7E0Z6UMGhr8gVwI5iVoyQ3xqoqWCSVnrN8Iiv8c93d4qegRx94jAMUwBrclwO7RakOWAVi849ipw+2/Zqm5TlR4eB5ZQ+Vk1lskzJEv4y3WDFVXyu36BdONIsSuG6KayrkcRlT42cbmOyyG99xghTYyRdKxxn1mftx4Hlu3QzRS7brxBi92NNUAZtn
Message-ID: <817898fc-b58d-4977-9cb8-977922019ccd@suse.com>
Date: Mon, 5 Feb 2024 10:28:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 01/15] xen/common: add cache coloring common code
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-2-carlo.nonato@minervasys.tech>
 <749799db-de6f-4ca8-bab1-67468f97929b@suse.com>
 <CAG+AhRV8gPdbm5MK6-g_DO+=KNrakWs82xMCGDMGy05Rxngvtg@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRV8gPdbm5MK6-g_DO+=KNrakWs82xMCGDMGy05Rxngvtg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 03.02.2024 11:57, Carlo Nonato wrote:
> On Thu, Feb 1, 2024 at 1:59 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 29.01.2024 18:17, Carlo Nonato wrote:
>>> --- a/xen/arch/Kconfig
>>> +++ b/xen/arch/Kconfig
>>> @@ -31,3 +31,20 @@ config NR_NUMA_NODES
>>>         associated with multiple-nodes management. It is the upper bound of
>>>         the number of NUMA nodes that the scheduler, memory allocation and
>>>         other NUMA-aware components can handle.
>>> +
>>> +config LLC_COLORING
>>> +     bool "Last Level Cache (LLC) coloring" if EXPERT
>>> +     depends on HAS_LLC_COLORING
>>> +
>>> +config NR_LLC_COLORS
>>> +     int "Maximum number of LLC colors"
>>> +     range 2 1024
>>
>> What's the reasoning behind this upper bound? IOW - can something to this
>> effect be said in the description, please?
> 
> The only reason is that this is the number of colors that fit in a 4 KiB page.
> I don't have any other good way of picking a number here. 1024 is already big
> and probably nobody would use such a configuration. But 512 or 256 would be
> equally arbitrary.

And because of this I'm asking that you say in the description how you
arrived at this value. As to fitting in 4k-page: That makes two
assumptions (both true for all ports right now, but liable to be missed if
either changed down the road): PAGE_SIZE == 0x1000 && sizeof(int) == 4.

>>> --- /dev/null
>>> +++ b/xen/common/llc-coloring.c
>>> @@ -0,0 +1,87 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +/*
>>> + * Last Level Cache (LLC) coloring common code
>>> + *
>>> + * Copyright (C) 2022 Xilinx Inc.
>>> + */
>>> +#include <xen/keyhandler.h>
>>> +#include <xen/llc-coloring.h>
>>> +#include <xen/param.h>
>>> +
>>> +bool __ro_after_init llc_coloring_enabled;
>>> +boolean_param("llc-coloring", llc_coloring_enabled);
>>
>> The variable has no use right now afaics, so it's unclear whether (a) it
>> is legitimately non-static and (b) placed in an appropriate section.
> 
> My bad here. The variable should be tested for in llc_coloring_init() and in
> domain_dump_llc_colors() (in domain_llc_coloring_free() as well, in later
> patches). That change was lost in the rebase of the series.
> 
> Anyway per this patch, the global is only accessed from this file while it's
> going to be accessed from outside in later patches. In this case what should
> I do? Declare it static and then make it non-static afterwards?

That would be preferred, considering that there may be an extended time
period between the 1st and 2nd patches going in. Explaining why a
variable is non-static despite not needing to be just yet would be an
alternative, but then you'd also need to justify why transiently
violating the respective Misra guideline is acceptable.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 09:32:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 09:32:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675723.1051113 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvLK-0005lH-2M; Mon, 05 Feb 2024 09:32:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675723.1051113; Mon, 05 Feb 2024 09:32:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvLJ-0005lA-Ub; Mon, 05 Feb 2024 09:32:45 +0000
Received: by outflank-mailman (input) for mailman id 675723;
 Mon, 05 Feb 2024 09:32:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rWvLI-0005l4-OJ
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 09:32:44 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWvLF-0001PD-2z; Mon, 05 Feb 2024 09:32:41 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.102])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWvLE-000206-Sz; Mon, 05 Feb 2024 09:32:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=j7Brr5GINz2LyCnMs4BEdlXOcAEEvLzTCRFN5963xCU=; b=FL4xA8vkCdRkVF6bvs2AdV2aVz
	aMMT/+fJ3Ici15MumkMzzR2lBEbyvec9v4pETfvC4eW2kJ2QIB7jSwyIxSSj9ZNbcjShZOumUcFuE
	GqGIhn1a1L3b+F0W7GWh2Z+zRX+X9SGVBYzRmj9rjbvSD0Njj7dQg3Nje5mkZqSx04ts=;
Message-ID: <a19f4217-4ed7-4065-b86c-805f7d57ba09@xen.org>
Date: Mon, 5 Feb 2024 09:32:38 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 4/9] x86/smp: move stack_base to cpu_data
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Krystian Hebel <krystian.hebel@3mdeb.com>, xen-devel@lists.xenproject.org
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <70e3b7c84a69a7ec52b3ed6314395165c281734c.1699982111.git.krystian.hebel@3mdeb.com>
 <d0fc568a-f54e-4480-a903-6407f4e3a5b6@xen.org>
 <52e2db77-88c4-47c5-85d7-34b0bb133b69@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <52e2db77-88c4-47c5-85d7-34b0bb133b69@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 05/02/2024 08:41, Jan Beulich wrote:
> On 02.02.2024 19:24, Julien Grall wrote:
>>>    static bool smp_store_cpu_info(unsigned int id)
>>> @@ -579,8 +581,6 @@ static int do_boot_cpu(int apicid, int cpu)
>>>            printk("Booting processor %d/%d eip %lx\n",
>>>                   cpu, apicid, start_eip);
>>>    
>>> -    stack_start = stack_base[cpu] + STACK_SIZE - sizeof(struct cpu_info);
>>> -
>>
>> You remove this line because I can't quite figure out where stack_start
>> is now set. This is used...
> 
> This line sets a global variable, which ...
> 
>>> @@ -856,7 +856,7 @@ int setup_cpu_root_pgt(unsigned int cpu)
>>>    
>>>        /* Install direct map page table entries for stack, IDT, and TSS. */
>>>        for ( off = rc = 0; !rc && off < STACK_SIZE; off += PAGE_SIZE )
>>> -        rc = clone_mapping(__va(__pa(stack_base[cpu])) + off, rpt);
>>> +        rc = clone_mapping(__va(__pa(cpu_data[cpu].stack_base)) + off, rpt);
>>>    
>>>        if ( !rc )
>>>            rc = clone_mapping(idt_tables[cpu], rpt);
>>> @@ -1007,10 +1007,10 @@ static void cpu_smpboot_free(unsigned int cpu, bool remove)
>>>            FREE_XENHEAP_PAGE(per_cpu(gdt, cpu));
>>>            FREE_XENHEAP_PAGE(idt_tables[cpu]);
>>>    
>>> -        if ( stack_base[cpu] )
>>> +        if ( cpu_data[cpu].stack_base )
>>>            {
>>> -            memguard_unguard_stack(stack_base[cpu]);
>>> -            FREE_XENHEAP_PAGES(stack_base[cpu], STACK_ORDER);
>>> +            memguard_unguard_stack(cpu_data[cpu].stack_base);
>>> +            FREE_XENHEAP_PAGES(cpu_data[cpu].stack_base, STACK_ORDER);
>>>            }
>>>        }
>>>    }
>>> @@ -1044,11 +1044,11 @@ static int cpu_smpboot_alloc(unsigned int cpu)
>>>        if ( node != NUMA_NO_NODE )
>>>            memflags = MEMF_node(node);
>>>    
>>> -    if ( stack_base[cpu] == NULL &&
>>> -         (stack_base[cpu] = cpu_alloc_stack(cpu)) == NULL )
>>> +    if ( cpu_data[cpu].stack_base == NULL &&
>>> +         (cpu_data[cpu].stack_base = cpu_alloc_stack(cpu)) == NULL )
>>>                goto out;
>>>    
>>> -    info = get_cpu_info_from_stack((unsigned long)stack_base[cpu]);
>>> +    info = get_cpu_info_from_stack((unsigned long)cpu_data[cpu].stack_base);
>>
>> ... here.
> 
> ... pretty clearly is not used here (anymore). Instead I'd raise the
> question of what the remaining purpose of that variable then is.
> Looking through updates this patch alone makes to use sites of
> stack_start, it's unclear whether the use from assembly code has gone
> away already - brief checking suggests it hasn't.

Whoops you are right. I am not sure how I thought this line was (still) 
using stack_start.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 09:35:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 09:35:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675726.1051122 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvNs-0006Jz-De; Mon, 05 Feb 2024 09:35:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675726.1051122; Mon, 05 Feb 2024 09:35:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvNs-0006Js-Aq; Mon, 05 Feb 2024 09:35:24 +0000
Received: by outflank-mailman (input) for mailman id 675726;
 Mon, 05 Feb 2024 09:35:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rWvNq-0006Jm-CZ
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 09:35:22 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWvNp-0001TZ-PK; Mon, 05 Feb 2024 09:35:21 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.102])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWvNp-0002Fp-HJ; Mon, 05 Feb 2024 09:35:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=PP1+qCJlAuvRneLYsJZF3rvQSzbheGF6lMKfJbyQ5Nw=; b=EF+ix0/5o266rNdOct2qWjC/Is
	LNtjevm5gpyp3QDDkBJXzoM97MvHUFkNQesfT0I8133X46XHAHfAu7jO+hh9GEoViH97JCKISB12V
	gfOvhp5LWI1pHf8fQjlTh39sYBAA44trPqGGwVv1nFu50U1blopv4lKeuCZh5tjzirtw=;
Message-ID: <916f4c54-c045-44d9-bff1-bec93936710a@xen.org>
Date: Mon, 5 Feb 2024 09:35:19 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 25/33] tools/xenstored: move all socket handling into
 posix.c
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-26-jgross@suse.com>
 <dcc477bd-b7cf-40e8-85be-3cde5b9cfd61@xen.org>
 <6abe490f-c8c5-4a7c-9ca0-0ef71845d576@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <6abe490f-c8c5-4a7c-9ca0-0ef71845d576@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Juergen,

On 05/02/2024 08:07, Juergen Gross wrote:
> On 26.01.24 17:22, Julien Grall wrote:
>> Hi Juergen,
>>
>> On 04/01/2024 09:00, Juergen Gross wrote:
>>> All of the socket handling is needed only when running as daemon.
>>>
>>> Move it into posix.c, allowing to remove the NO_SOCKETS macro.
>>>
>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>> ---
>>> V3:
>>> - new patch
>>> ---
>>>   tools/xenstored/Makefile.common |   4 -
>>>   tools/xenstored/core.c          | 156 +-------------------------------
>>>   tools/xenstored/core.h          |   8 +-
>>>   tools/xenstored/domain.c        |   9 +-
>>>   tools/xenstored/minios.c        |   6 ++
>>>   tools/xenstored/posix.c         | 153 +++++++++++++++++++++++++++++++
>>>   6 files changed, 167 insertions(+), 169 deletions(-)
>>>
>>> diff --git a/tools/xenstored/Makefile.common 
>>> b/tools/xenstored/Makefile.common
>>> index 189ab81b8d..ef63ef650c 100644
>>> --- a/tools/xenstored/Makefile.common
>>> +++ b/tools/xenstored/Makefile.common
>>> @@ -16,10 +16,6 @@ CFLAGS += $(CFLAGS_libxenctrl)
>>>   CFLAGS += $(CFLAGS_libxenguest)
>>>   CFLAGS += $(CFLAGS_libxentoolcore)
>>> -ifdef CONFIG_STUBDOM
>>> -CFLAGS += -DNO_SOCKETS=1
>>> -endif
>>> -
>>>   $(XENSTORED_OBJS-y): CFLAGS += $(CFLAGS_libxengnttab)
>>>   xenstored.a: $(XENSTORED_OBJS-y)
>>> diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
>>> index 650c1d1574..8ff1faccef 100644
>>> --- a/tools/xenstored/core.c
>>> +++ b/tools/xenstored/core.c
>>> @@ -20,10 +20,6 @@
>>>   #include <sys/types.h>
>>>   #include <sys/stat.h>
>>>   #include <poll.h>
>>> -#ifndef NO_SOCKETS
>>> -#include <sys/socket.h>
>>> -#include <sys/un.h>
>>> -#endif
>>>   #include <sys/time.h>
>>>   #include <time.h>
>>>   #include <unistd.h>
>>> @@ -61,7 +57,7 @@ static unsigned int current_array_size;
>>>   static unsigned int nr_fds;
>>>   static unsigned int delayed_requests;
>>> -static int sock = -1;
>>> +int sock = -1;
>>
>> Similar comment to 'fds' on the previous patch. This name is quite 
>> generic. Can we either pass it as an argument (or return) or rename it?
> 
> What about to make it local to posix.c and adding a get_socket_fd() 
> function
> for its only usage in core.c (with a related stub in minios.c, of course)?

That looks fine to me in principle.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 09:35:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 09:35:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675728.1051133 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvO9-0006lS-OH; Mon, 05 Feb 2024 09:35:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675728.1051133; Mon, 05 Feb 2024 09:35:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvO9-0006lL-Lj; Mon, 05 Feb 2024 09:35:41 +0000
Received: by outflank-mailman (input) for mailman id 675728;
 Mon, 05 Feb 2024 09:35:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWvO9-0006ih-4F
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 09:35:41 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ebf56445-c409-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 10:35:38 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-40fd72f71f8so5522105e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 01:35:38 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 je21-20020a05600c1f9500b0040efb490814sm7947906wmb.27.2024.02.05.01.35.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 01:35:38 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebf56445-c409-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707125738; x=1707730538; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4uMjQ6jp5wYZEe3NmQUyqQlqUL6pV9sndahTuPWK+ck=;
        b=MHqhTWR4+RjMOqsP4k8P6NdCKopI30/CNLDnr9lTfy3IA+YfUG2wbxtxn1Qui1togy
         IAYMdon5vk0PZS8hOpMiGKQaMEXPNafALOMWYApeGfQ0JnTmSd4qEgXW0FV8kmTLQtXV
         seXegV7W9LHFRX3LcrpmLtkSQtKfdn/zJvt1Uo54MldzaT1P+7YmKhN2Dj2woJ+MT0mX
         uVPPdCjkwaXEUEEqFdO6WCmHLdmXmQitCMOsIL8sZNCHK1EpTQFfBM9IY/kGV6zwAwWR
         8BmTNGblPxqPCVK9EJfTb9IijVFWhMfo7+wOxbCSxBuj02bvRsmjDF6OVHN6TrRpoclf
         5UUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707125738; x=1707730538;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=4uMjQ6jp5wYZEe3NmQUyqQlqUL6pV9sndahTuPWK+ck=;
        b=kxyAvseCnh444C/w26L98VbTrakD3zN8KRGdlkAUWTDlKH6TLXylPbF2sCj3h6haId
         9x0jNwBrLD2zBOXdcym+759FiHeqKlfKzXgkNJFup0kwsqMIsekQgKsppEG2pFD7RgoH
         U6VtClRlFExNxNE+0BA3QdsCzUisBsyy7xc8gWZAWzzn0yuRMx9Zdy8RjekgFhdTMQ+l
         5bXjcDkMJMqyK9w3hfsOl2vSYCH5rJkK050Osl49AHPvzejBf41pfPeGpGt+7oT886RS
         695Hxn/ldYXybPquUpZgeN2isR6wTlmJJskbz3mExIEABEJIa674KOD6Chorh2iztR2g
         KwFQ==
X-Gm-Message-State: AOJu0YwLe1LFWl7ilkBDA5PuHfJC6XPQOQW+VOkce+KruCJ3IiTcP5gQ
	GSHOVdfk3zlh8COL2UBM4H5fJWG/mT5V7OJn+jyNOzHGqwPZRo0EiJireH0jqQ==
X-Google-Smtp-Source: AGHT+IERZtHpeul+KmV0AOgCD71wnHdvIeZoqZ+jSKG/7ErB7HzscEWJG5vA8pKwCSnm4c5MrKAa9Q==
X-Received: by 2002:a05:600c:5248:b0:40f:b2ba:b893 with SMTP id fc8-20020a05600c524800b0040fb2bab893mr3806760wmb.1.1707125738312;
        Mon, 05 Feb 2024 01:35:38 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCULU1Ae+ltZSEb02wJqGuY4oW7VxxkEnkrKVGjzvS5NBYDaS77u6HAqBcQ7KtTlZn5KqUccX4FaxrtrhVMXWuzeufL7qLcBM3ctEnEAN7iFyPJ5SHQpT7FVq+vRUBJzRWnzCHlb2R0O2B+MtbYCQeiHYL0iAsAqTaR2Wz9ZU5w6iBDnC2y1n0jaitrPoU06KVc/xho7qaddaXFQf0E0TOwa/JFgqwaK0+lN6vzCcRUSkDIgPkkzf1MnjwFK4dgE/mVd4Yv4DjZM4E/rvAJII5p83ymN0LVflHJ4PPGpqfpiKrK7pNwwGA84tgMA6GIcvIKqZQnZdf5+fTBxQdjGMuX+SETciTGdIZAbCGjoHkgKFEUYJB/n9EaIe+tSXHEqfay3Fg==
Message-ID: <82e01801-2fef-4a5b-b343-2cd4a99af016@suse.com>
Date: Mon, 5 Feb 2024 10:35:40 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 04/15] xen/arm: add Dom0 cache coloring support
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-5-carlo.nonato@minervasys.tech>
 <cf23d8a8-7111-4014-adc7-93ecd5f110ec@suse.com>
 <CAG+AhRW4Ng7wkhB5f2H5ruCmYgium0YXZgA_bu7=T6HErUEg2A@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRW4Ng7wkhB5f2H5ruCmYgium0YXZgA_bu7=T6HErUEg2A@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 03.02.2024 12:39, Carlo Nonato wrote:
> On Thu, Feb 1, 2024 at 2:30 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>> --- a/xen/common/llc-coloring.c
>>> +++ b/xen/common/llc-coloring.c
>>> @@ -17,6 +17,63 @@ size_param("llc-way-size", llc_way_size);
>>>  /* Number of colors available in the LLC */
>>>  static unsigned int __ro_after_init max_nr_colors = CONFIG_NR_LLC_COLORS;
>>>
>>> +static unsigned int __initdata dom0_colors[CONFIG_NR_LLC_COLORS];
>>> +static unsigned int __initdata dom0_num_colors;
>>> +
>>> +/*
>>> + * Parse the coloring configuration given in the buf string, following the
>>> + * syntax below.
>>> + *
>>> + * COLOR_CONFIGURATION ::= COLOR | RANGE,...,COLOR | RANGE
>>> + * RANGE               ::= COLOR-COLOR
>>> + *
>>> + * Example: "0,2-6,15-16" represents the set of colors: 0,2,3,4,5,6,15,16.
>>> + */
>>> +static int parse_color_config(const char *buf, unsigned int *colors,
>>> +                              unsigned int num_colors, unsigned int *num_parsed)
>>
>> Is this function going to be re-used? If not, it wants to be __init.
>> If so, I wonder where the input string is going to come from ...
> 
> You're right. It needs __init.

Am I misremembering to have spotted a non-init use in a later patch?

>> Also "num_colors" looks to be misnamed - doesn't this specify an
>> upper bound only?
> 
> It's the real size of the colors array.

Hence my remark: It is _not_ the number of colors.

>>> +int __init dom0_set_llc_colors(struct domain *d)
>>> +{
>>> +    unsigned int *colors;
>>> +
>>> +    if ( !dom0_num_colors )
>>> +        return domain_set_default_colors(d);
>>> +
>>> +    colors = alloc_colors(dom0_num_colors);
>>> +    if ( !colors )
>>> +        return -ENOMEM;
>>> +
>>> +    memcpy(colors, dom0_colors, sizeof(unsigned int) * dom0_num_colors);
>>
>> sizeof(*colors) or some such please. Plus a check that colors and
>> dom0_colors are actually of the same type. Alternatively, how about
>> making dom0_colors[] __ro_after_init? Is this too much of a waste?
> 
> You mean an ASSERT on the two arrays type?

I don't think you can use ASSERT() for such very well. It's runtime
check, when here we want a build-time one. I'd therefore rather see
it be something like

   (void)(colors == dom0_colors);

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 09:36:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 09:36:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675733.1051142 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvPF-0007Ov-13; Mon, 05 Feb 2024 09:36:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675733.1051142; Mon, 05 Feb 2024 09:36:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvPE-0007Oo-UT; Mon, 05 Feb 2024 09:36:48 +0000
Received: by outflank-mailman (input) for mailman id 675733;
 Mon, 05 Feb 2024 09:36:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rWvPD-0007Oe-Gt
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 09:36:47 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWvPC-0001V9-Oh; Mon, 05 Feb 2024 09:36:46 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.102])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWvPC-0002Fp-Jr; Mon, 05 Feb 2024 09:36:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=env0YJKk9R293kThFXtsw6WZIYkIC2PNcd6ob0Oe2VA=; b=BZeVq+8buNt3fcy9K7+heOrYSF
	l1IZp1F02mCLCuOQrZZTWsMwGukBKi4xhwNXIYqeSFEFjzAT9ekfBOzDuJhKnRW5qMGVSAY+CuBJd
	YoxwbtH4VEh1dW06FiaJsosLOC6JZ6heErr0qL9fTBFKFszLeC7RSaM4dXUv5cn2x7Dc=;
Message-ID: <fb0e6d99-8cfd-4e99-b9e9-07aac9321b46@xen.org>
Date: Mon, 5 Feb 2024 09:36:45 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 25/33] tools/xenstored: move all socket handling into
 posix.c
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240104090055.27323-1-jgross@suse.com>
 <20240104090055.27323-26-jgross@suse.com>
 <dcc477bd-b7cf-40e8-85be-3cde5b9cfd61@xen.org>
 <c4d5b0e0-e726-4ab2-9482-f284f2b92674@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <c4d5b0e0-e726-4ab2-9482-f284f2b92674@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Juergen,

On 05/02/2024 09:21, Juergen Gross wrote:
> On 26.01.24 17:22, Julien Grall wrote:
>> Hi Juergen,
>>
>> On 04/01/2024 09:00, Juergen Gross wrote:
>>> All of the socket handling is needed only when running as daemon.
>>>
>>> Move it into posix.c, allowing to remove the NO_SOCKETS macro.
>>>
>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>> ---
>>> V3:
>>> - new patch
>>> ---
>>>   tools/xenstored/Makefile.common |   4 -
>>>   tools/xenstored/core.c          | 156 +-------------------------------
>>>   tools/xenstored/core.h          |   8 +-
>>>   tools/xenstored/domain.c        |   9 +-
>>>   tools/xenstored/minios.c        |   6 ++
>>>   tools/xenstored/posix.c         | 153 +++++++++++++++++++++++++++++++
>>>   6 files changed, 167 insertions(+), 169 deletions(-)
>>>
>>> diff --git a/tools/xenstored/Makefile.common 
>>> b/tools/xenstored/Makefile.common
>>> index 189ab81b8d..ef63ef650c 100644
>>> --- a/tools/xenstored/Makefile.common
>>> +++ b/tools/xenstored/Makefile.common
>>> @@ -16,10 +16,6 @@ CFLAGS += $(CFLAGS_libxenctrl)
>>>   CFLAGS += $(CFLAGS_libxenguest)
>>>   CFLAGS += $(CFLAGS_libxentoolcore)
>>> -ifdef CONFIG_STUBDOM
>>> -CFLAGS += -DNO_SOCKETS=1
>>> -endif
>>> -
>>>   $(XENSTORED_OBJS-y): CFLAGS += $(CFLAGS_libxengnttab)
>>>   xenstored.a: $(XENSTORED_OBJS-y)
>>> diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
>>> index 650c1d1574..8ff1faccef 100644
>>> --- a/tools/xenstored/core.c
>>> +++ b/tools/xenstored/core.c
>>> @@ -20,10 +20,6 @@
>>>   #include <sys/types.h>
>>>   #include <sys/stat.h>
>>>   #include <poll.h>
>>> -#ifndef NO_SOCKETS
>>> -#include <sys/socket.h>
>>> -#include <sys/un.h>
>>> -#endif
>>>   #include <sys/time.h>
>>>   #include <time.h>
>>>   #include <unistd.h>
>>> @@ -61,7 +57,7 @@ static unsigned int current_array_size;
>>>   static unsigned int nr_fds;
>>>   static unsigned int delayed_requests;
>>> -static int sock = -1;
>>> +int sock = -1;
>>
>> Similar comment to 'fds' on the previous patch. This name is quite 
>> generic. Can we either pass it as an argument (or return) or rename it?
> 
> Turns out this patch now makes the "fds" solution more complicated: I'd 
> need to
> add fds as an additional parameter to the .can_write() and .can_read() 
> callbacks
> of struct interface_funcs.
> 
> I can do that, but an alternative would be to just rename fds to e.g. 
> poll_fds.
> 
> Do you have any preference?
The last one seems to be the simplest approach. And I am fine with the 
renaming. This can be improved in the future.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 09:37:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 09:37:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675735.1051152 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvPw-00088W-8p; Mon, 05 Feb 2024 09:37:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675735.1051152; Mon, 05 Feb 2024 09:37:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvPw-00088P-6B; Mon, 05 Feb 2024 09:37:32 +0000
Received: by outflank-mailman (input) for mailman id 675735;
 Mon, 05 Feb 2024 09:37:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWvPu-0007pV-Us
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 09:37:30 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2db1c08d-c40a-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 10:37:29 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-51121637524so6285462e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 01:37:29 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h11-20020a05600c314b00b0040fde7250f4sm350606wmo.38.2024.02.05.01.37.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 01:37:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2db1c08d-c40a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707125848; x=1707730648; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=QAvm6dqaFKH/KmVmlkXN3f2MJxKOho4bem3WUKORwKU=;
        b=MR8SpM4sv4YHO7l6NsGPueq9le3wShrnbh1YC3LKRIgFhoIJk+1Ps3zo+I0L1WrQvg
         T6dvK3EXcJAGpYLG/5+opfu59+5bNk+SUhxuLp4CU9vwil8Wxvmh8Dma7VaRkHBqTrJB
         rOU3zj7j9xVi3DvY7TCfapdlDV5p9fxGRfxojcLwJVoA1IgHybsoT88JcF4JwvJAJB32
         8YFE9gEi/qLhzHMp1Nt5jXdunTV4zarQRMdkLTcCdx6olvBrA7iuH+X4eXUhPuWaM0Yx
         pevgJW/ePyXYlkGFtHadeI/7sDbGnoRKSDSh7+51IOmDdC37dZbBeCSfWAd14+ugveAs
         njqA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707125848; x=1707730648;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=QAvm6dqaFKH/KmVmlkXN3f2MJxKOho4bem3WUKORwKU=;
        b=oxV7zgYmOJ1UOsd9NM/1dpXN3vOaWb2fm7EIhe6xpFHjSaa+DOFeN+YOgXfz07rVLA
         yKnJmbVFcSk2YlvpFdW0o//tOe1XvT0dVJmQNiET6En6/aVz5MV2nnuhEuNPoopVJzeB
         bcwPjX7NJmKEmeAGIycRicx5ll6t8wYcOcv/nTc/HLDaniIPdQsqxlD+zD6WvbTxhBBF
         +fHNWpdrx47zpVkaBUYIJ3MKduqCcOabLHP695rolNOxht3X2IX5szRp5GApwsNqlqR5
         jYdQ5xngLrmSCP5+AfV4XzX5/DUMGLrYccaKUmr8t6Dcce0SbkMUWwozA6AhBxz9VhPI
         1E4g==
X-Gm-Message-State: AOJu0YxcPTWVb97+NjUsgjRtQK63E1bRSrOCBg1YbyiT/U5VfuktSeoB
	dk1+qlGaMJg/z9OpkgwPlWt1Gf3xcqP5or+iml1ILMOfO1W1Hj8OtBQt1jePUA==
X-Google-Smtp-Source: AGHT+IE1M8ZaHQnOtfnGdyJpOMNL3DFNWKXiwEf660AjIaAY6Xr9yEUYuEDrXQzfAVK04Za2ajnQDA==
X-Received: by 2002:a05:6512:1153:b0:511:53fa:b971 with SMTP id m19-20020a056512115300b0051153fab971mr809998lfg.9.1707125848585;
        Mon, 05 Feb 2024 01:37:28 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVJSNQHLtlc4LZRlQmHYJAvVOO84UHHJ8Qm9MxCWZcVPAUgJTxTBwta3Lgk1MMmAT2QLWSOvrp8NjvLl1KfT9US8w9IWLccbxBr45XKsqI+U4YV35XpdBUZXT5gcMyKyWXjYhQ2C/bx7CsVxTl48H/DyyLxff3+8gO+awzegREfZFedxq9UBkeGBj5e6waNfBp0LQRRtiPHwYJp1gyJ0XK4RQYLsW+GHvcw6BLvQAjNHMq/69Wwa1+rCowywUmCKku3eyG5RQSHPcgXcOV9P0XaU3mLWudZ
Message-ID: <275107b0-bcee-489b-92e6-449a4ac627d0@suse.com>
Date: Mon, 5 Feb 2024 10:37:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 05/15] xen: extend domctl interface for cache coloring
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-6-carlo.nonato@minervasys.tech>
 <49827753-14f1-42cc-8791-27c5400e6e50@suse.com>
 <CAG+AhRUGdkqhoSL-ihiCGZctH2mkSOi+ap+=a_kKLzJRGugCbw@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRUGdkqhoSL-ihiCGZctH2mkSOi+ap+=a_kKLzJRGugCbw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 03.02.2024 12:41, Carlo Nonato wrote:
> On Thu, Feb 1, 2024 at 2:51 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>> --- a/xen/common/llc-coloring.c
>>> +++ b/xen/common/llc-coloring.c
>>> @@ -4,6 +4,7 @@
>>>   *
>>>   * Copyright (C) 2022 Xilinx Inc.
>>>   */
>>> +#include <xen/guest_access.h>
>>>  #include <xen/keyhandler.h>
>>>  #include <xen/llc-coloring.h>
>>>  #include <xen/param.h>
>>> @@ -229,6 +230,30 @@ int __init dom0_set_llc_colors(struct domain *d)
>>>      return domain_check_colors(d);
>>>  }
>>>
>>> +int domain_set_llc_colors_domctl(struct domain *d,
>>> +                                 const struct xen_domctl_set_llc_colors *config)
>>
>> What purpose has the "domctl" in the function name?
> 
> To signal that it's called from domctl. Do you suggest leaving it out?

Yes. Names want to be descriptive, but also not be overly long. Imo.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 09:39:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 09:39:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675739.1051163 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvSF-0000kb-L2; Mon, 05 Feb 2024 09:39:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675739.1051163; Mon, 05 Feb 2024 09:39:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvSF-0000kU-HN; Mon, 05 Feb 2024 09:39:55 +0000
Received: by outflank-mailman (input) for mailman id 675739;
 Mon, 05 Feb 2024 09:39:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWvSE-0000kO-Fn
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 09:39:54 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 83b1ebc9-c40a-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 10:39:53 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-51025cafb51so6276957e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 01:39:53 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ay2-20020a05600c1e0200b0040fdb17e66csm2725405wmb.4.2024.02.05.01.39.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 01:39:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83b1ebc9-c40a-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707125993; x=1707730793; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HFtPtIsiMgcY2W79MV1g1dHWCE1ow2sFdFokur2UR5M=;
        b=NuyMVVS20DbhxKU4FXC+GxRBjiF3vGGP2zfgu77he2zya1JX0uFdeik/DOi8eJL0/6
         zhAm1bvDSdRGaWHJrCfGucxwGhxH0otOi08yQwc+YI4Gb56pqwQQ/ce+8UD2z948VK9k
         BX32diXVryARbqkEjeHTCHAPp+SFgCVGHPuJVNW6n65NU9uRswzxQgMnzsmV4gWtSQzq
         8zc1SG+4aTfg5D0l2Q/5pYnL2C92PGfmg4WeJI7/brp/yDDXqnZo3thagazQ0TzVicHq
         5qihbzEyhs3O/0KG5WgobKpvjmOk/KEUopEsYdPGgzaYFBjQxshA/m35TS05V1bzSidw
         hnWw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707125993; x=1707730793;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HFtPtIsiMgcY2W79MV1g1dHWCE1ow2sFdFokur2UR5M=;
        b=r+gVS+kc0S5PKJgqipEyvMnw5fSw7kDUWAjqN5XwCaAYPl4KLbm/tEkxKB64qoz9fr
         GYtAdpH1TZbejiVuuEupIl4PT4zaeQkdu0SeF9Zg7ByCqt5O71aW4/J5aLrG578+NDu/
         hTAuSJ6Hs5an2J4cUxxYAMycowXDj3B+0ZaggTFxAnfuYnjulM1etVK5OKYCWijsBQ+4
         sa9HrCNAgyswj7sEMrfAl8Fp3IIYzMigLlu3vZx3zgVZdR8/M3CR1kkDIcTgaaoBMQTy
         OjnwZmlpWGFS8wcjCtRDDnKQwdZWv9/1lKv21IQy5vBdLJwuKdrpT28iwO8tIy2EblcP
         gn2Q==
X-Gm-Message-State: AOJu0YzuJf7iFuKv0Xkh7c4Q+pqMblsr0u52v5F8ZddSIToV5EMryffu
	Qak68n/u5Lz5QkmHcyRKldAnzBoSi5H7wu8KU8rrGMhHWhZrllGlrZKL8JFfpg==
X-Google-Smtp-Source: AGHT+IFQXeyjawxy0vjE4cMv5Sg+C7Z38vPKSwitDcAxSdyAcT+fN/Cntt4BeyMCVqfFbQxHtl+E7w==
X-Received: by 2002:a05:6512:31ca:b0:50e:7c70:f8da with SMTP id j10-20020a05651231ca00b0050e7c70f8damr6378955lfe.16.1707125993015;
        Mon, 05 Feb 2024 01:39:53 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWRFRxn6duz6IXN0MpPZ+hOwnG4yqPpZahDgctq4h8oZrlpcJaH7r8iEXuGeAzjNo7QO36jt/dFRcwLUUtU+78RGcVdndBpoE9hFvIZ/1klDWNVVJ/XxJZLmBLmvSNmZ8WUP7gyXI5vx9fo/Jymdm2Fx2/9CY8d41nwDz4KJbEFkKaECRhgqqz5tl7oghXYehOpyzNMoI7e4ZqMHES/VaumDOLoFtiblAIg+RPIYkMe72eng7a5xAPKJRnRvB5SSFLW/K810HU7AwNPMCgok1WZUqa1piZ0oJajzKkN57BfA9KC3VXW3nTg3wWd5Z9EjlKZKW3LPhlVYubXUyluzsEepTabye79R1qVt7AKO9Tng+IQeRnzgX04mHaJNdjrleIYpQ==
Message-ID: <26815df2-bfe5-4767-b1a6-94666e535f4e@suse.com>
Date: Mon, 5 Feb 2024 10:39:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 07/15] xen/arm: add support for cache coloring
 configuration via device-tree
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-8-carlo.nonato@minervasys.tech>
 <a6ffb4a5-cd83-4c04-9e5a-12d605863b5d@suse.com>
 <CAG+AhRUG8nNF0cTtFYLu1TrV4fWEbix57Ah_-Zsz=U2yRWcGpw@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRUG8nNF0cTtFYLu1TrV4fWEbix57Ah_-Zsz=U2yRWcGpw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 03.02.2024 12:43, Carlo Nonato wrote:
> On Thu, Feb 1, 2024 at 3:19 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>> @@ -950,6 +951,11 @@ void __init create_domUs(void)
>>>  #endif
>>>          }
>>>
>>> +        dt_property_read_string(node, "llc-colors", &llc_colors_str);
>>> +        if ( !llc_coloring_enabled && llc_colors_str)
>>> +            printk(XENLOG_WARNING
>>> +                   "'llc-colors' found, but LLC coloring is disabled\n");
>>
>> Why's this just a warning, when ...
> 
> This informs the user that this configuration will be ignored, but the DomU can
> be constructed anyway...

Yet that's a violation of the principle that Julien had outlined when
discussing whether to panic() in such cases. The property indicates to
me that the domain ought to be run with coloring enabled, i.e. not much
different from ...

>>> @@ -960,6 +966,11 @@ void __init create_domUs(void)
>>>              panic("Error creating domain %s (rc = %ld)\n",
>>>                    dt_node_name(node), PTR_ERR(d));
>>>
>>> +        if ( llc_coloring_enabled &&
>>> +             (rc = domain_set_llc_colors_from_str(d, llc_colors_str)) )
>>> +            panic("Error initializing LLC coloring for domain %s (rc = %d)\n",
>>> +                  dt_node_name(node), rc);
>>
>> ... this results in panic()?
> 
> ... while here we can't continue because there's some error in the
> configuration and the DomU can't be constructed. Domains must have a valid
> coloring configuration.

... the request not being possible to fulfill here.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:08:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:08:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675751.1051174 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvte-0005ou-Tx; Mon, 05 Feb 2024 10:08:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675751.1051174; Mon, 05 Feb 2024 10:08:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvte-0005on-PZ; Mon, 05 Feb 2024 10:08:14 +0000
Received: by outflank-mailman (input) for mailman id 675751;
 Mon, 05 Feb 2024 10:08:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VsQK=JO=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rWvtd-0005oh-21
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:08:13 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20621.outbound.protection.outlook.com
 [2a01:111:f400:7eaa::621])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 770f01e9-c40e-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:08:11 +0100 (CET)
Received: from SA0PR11CA0084.namprd11.prod.outlook.com (2603:10b6:806:d2::29)
 by DS7PR12MB9044.namprd12.prod.outlook.com (2603:10b6:8:e3::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.17; Mon, 5 Feb
 2024 10:08:07 +0000
Received: from SA2PEPF000015C9.namprd03.prod.outlook.com
 (2603:10b6:806:d2:cafe::dd) by SA0PR11CA0084.outlook.office365.com
 (2603:10b6:806:d2::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7159.18 via Frontend
 Transport; Mon, 5 Feb 2024 10:08:07 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF000015C9.mail.protection.outlook.com (10.167.241.199) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Mon, 5 Feb 2024 10:08:07 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 5 Feb
 2024 04:08:07 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 5 Feb
 2024 04:08:06 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Mon, 5 Feb 2024 04:08:04 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 770f01e9-c40e-11ee-8a45-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OjBIg7XzZ2YIUEJ/NT/J6rKc21Lfg06xyBcnMMpGqe36Ir/aePLT3uj8l0R5nSbPddCif2LJ5Vm4sC7f1Iz21PMbiP8+M15OHtUZBtsaEUiUpyZkEjbdDKglPyXgTfosFHL/oZPr6xKh8zgHGMK8LlWm08PtLTsg/CsYV+kJw+p8xT6qokZVQqxyMN0QwaANFMr6hIIvXf5KQNA09ELYDdljYeGTNpGVujJJ+soF20D+7PyEtMOghKmkcGmzUCGr6UNygZWj3IRCAesnjMuHAPWvaRZSTKETL1hJ6SAY4r0Enquc3atAYFQM+SEkAaEFmpYUnTyZKoObcSWUsr4IfQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Zt3qmdHsgtzsLYbX8ogEy8B8gjkmIE8jC21cSZOWJ1o=;
 b=XXIJbLAEFFwpsCfUoZHB7pkCyWXxHln5RpBgBMh6sh96JT7t7IVIiOkjv+qYLV9hIpqjfKpVeBAgRyVVUAzo5wsskrJCgKzCsfw4Gmg9+GP6Vum/8g+V7royyo1Q80DaMNGOGqPuoOkJpjzh8qyMKA4lawUToGwXLoo94ENfs0N8cpcC8NAfUqRORKKOxf2QtvwKHbpZ+dIHr7nMYwuRAqnAGfK0Lo8fc+n67JvbqUc8cKeIZKHiWgEHmKO1s8QAZ48gjkkbDaNgpCxHt2glMeoFnkm0/psabOynfMV6TcJGpVsx5dVteuEUraHrjJPSf//BtRV49idL5h3dtWjJMg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Zt3qmdHsgtzsLYbX8ogEy8B8gjkmIE8jC21cSZOWJ1o=;
 b=AfKCKa+k8tvU0gQrhGqZvv+VPvXdNJFGfUceHcOOi7QEGt3rnAU4zqKnQUnoY0RGv58NLm8x3dbigKH2wx7bhdHmxbzPfJnP4LgkIzGXiSu8I+hR9MWMQMILNNa0PfKpr70K9IyddTCQmT+S8qXunxez6qxkVdXNyfb1skgUXRM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <8867c445-2a34-4506-95e7-8eca4d91a364@amd.com>
Date: Mon, 5 Feb 2024 11:08:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 07/15] xen/arm: add support for cache coloring
 configuration via device-tree
To: Jan Beulich <jbeulich@suse.com>, Carlo Nonato
	<carlo.nonato@minervasys.tech>
CC: <andrea.bastoni@minervasys.tech>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, Marco Solieri
	<marco.solieri@minervasys.tech>, <xen-devel@lists.xenproject.org>
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-8-carlo.nonato@minervasys.tech>
 <a6ffb4a5-cd83-4c04-9e5a-12d605863b5d@suse.com>
 <CAG+AhRUG8nNF0cTtFYLu1TrV4fWEbix57Ah_-Zsz=U2yRWcGpw@mail.gmail.com>
 <26815df2-bfe5-4767-b1a6-94666e535f4e@suse.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <26815df2-bfe5-4767-b1a6-94666e535f4e@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF000015C9:EE_|DS7PR12MB9044:EE_
X-MS-Office365-Filtering-Correlation-Id: 04571602-143a-4ad2-6e08-08dc263259cc
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	FKFgWoZfNxCCEaPPxUy0U3qMsCRVb74QCOzrruZNq04/7vzOsptYRQ4YbY+PtH4wv+C4/lcnUqJ6mfKG7JOjvBnA+cagMCA4xML79sDrXZCXIUcJSeNDM3rJ21fluwIRLMIRHI7c33cg4pk+8fgDGpbfr2OB5LKkaf8LYNW669unthJd818P6delkLl62GOwDLw9ce3TjAvOEFBF9+hahrWHwRyjE98r/xIuLaKbskfJGXoyUjPg1OKl3X3QmnNzSzMzPUEEoEV5VCgPbmSA2McA7kWpyNcOc/1VoVRFLa3GzGWgVSeudbUY6HiZpr9PRwYG+KIW2qbt3emlIeU3fXdU6aI7Ys+c0ztoP9pvogEEvIdZu6lurvwOw5lOObsG2xk3MMt73ARZZ/kg3F/xOT6vwJNTd2/96NpmdG5G2ABDXljAiQN+Av//MoHw6WUd85ktRsPRjJrfajYKmXR4nh+D7F4N48G+V24DODAJZgmFgp56E6YOuVu4emhFy4exq4//r/CfclraPiRAicmNcfnehdbbed8zT5QARJk8tqG4IlnWr0JPI7nwq1vNwzeB2oybX23h+FUF/SGqZRRDUioWCKQ6mCXAV9izAHIsq7Gn2G0uN+Qo3oYuji2487548zpm0qrBNGPycc8gOsOJe3Zh+TqYcsI2lhhcHnareW0JJfGytSGv93wsUk/FUthi7C0/jWTgAAdrmmORNFiH9GO5DVlM7y/cTi6JwKwVYYWmKObvoAM1H1w98btjTvPQBUTjtZD7rR/zaAQTO80aE577bZpPiJ262jWgXl8qIMFxn52z2Z5aZzY2tqbXT2UA
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(396003)(39860400002)(136003)(376002)(346002)(230922051799003)(230273577357003)(82310400011)(451199024)(186009)(64100799003)(1800799012)(40470700004)(36840700001)(46966006)(41300700001)(8676002)(4326008)(5660300002)(31696002)(8936002)(44832011)(86362001)(54906003)(16576012)(316002)(478600001)(70586007)(70206006)(110136005)(356005)(2906002)(7416002)(36756003)(47076005)(36860700001)(82740400003)(81166007)(2616005)(53546011)(83380400001)(26005)(426003)(336012)(40460700003)(40480700001)(31686004)(43740500002)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Feb 2024 10:08:07.5932
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 04571602-143a-4ad2-6e08-08dc263259cc
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF000015C9.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB9044



On 05/02/2024 10:39, Jan Beulich wrote:
> 
> 
> On 03.02.2024 12:43, Carlo Nonato wrote:
>> On Thu, Feb 1, 2024 at 3:19 PM Jan Beulich <jbeulich@suse.com> wrote:
>>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>>> @@ -950,6 +951,11 @@ void __init create_domUs(void)
>>>>  #endif
>>>>          }
>>>>
>>>> +        dt_property_read_string(node, "llc-colors", &llc_colors_str);
>>>> +        if ( !llc_coloring_enabled && llc_colors_str)
>>>> +            printk(XENLOG_WARNING
>>>> +                   "'llc-colors' found, but LLC coloring is disabled\n");
>>>
>>> Why's this just a warning, when ...
>>
>> This informs the user that this configuration will be ignored, but the DomU can
>> be constructed anyway...
> 
> Yet that's a violation of the principle that Julien had outlined when
> discussing whether to panic() in such cases. The property indicates to
> me that the domain ought to be run with coloring enabled, i.e. not much
> different from ...
> 
>>>> @@ -960,6 +966,11 @@ void __init create_domUs(void)
>>>>              panic("Error creating domain %s (rc = %ld)\n",
>>>>                    dt_node_name(node), PTR_ERR(d));
>>>>
>>>> +        if ( llc_coloring_enabled &&
>>>> +             (rc = domain_set_llc_colors_from_str(d, llc_colors_str)) )
>>>> +            panic("Error initializing LLC coloring for domain %s (rc = %d)\n",
>>>> +                  dt_node_name(node), rc);
>>>
>>> ... this results in panic()?
>>
>> ... while here we can't continue because there's some error in the
>> configuration and the DomU can't be constructed. Domains must have a valid
>> coloring configuration.
> 
> ... the request not being possible to fulfill here.
+1
If the user requests a certain functionality which cannot be fulfilled, we shall panic.
Take a look at e.g. sve, static-shmem, vpl011.

~Michal


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:13:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:13:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675756.1051183 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvyf-0007rc-Jo; Mon, 05 Feb 2024 10:13:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675756.1051183; Mon, 05 Feb 2024 10:13:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWvyf-0007rV-HF; Mon, 05 Feb 2024 10:13:25 +0000
Received: by outflank-mailman (input) for mailman id 675756;
 Mon, 05 Feb 2024 10:13:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rWvyf-0007rP-0K
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:13:25 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWvyV-0002Gx-NU; Mon, 05 Feb 2024 10:13:15 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWvyV-0004Yg-FP; Mon, 05 Feb 2024 10:13:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=J7JoeYQ0DQnkWnozzirMu5yZ/gg+y8pD8sZa9vnAbzM=; b=R4SONskWvj3JoG3HZbtwSEvWmr
	6NZ9Kslz4Xzj/FqYiMdYLKirngGpnk+TqEuD0dtKlc9tR14U0NKJtEmOTYURA23vy11kkK1BxLkBD
	pvM5wFNGoORLB2B7lxEwVvOYei+MxSbGD/HCqsaC0sLP+3UH8amdHFVfhw/sZ6THiNok=;
Message-ID: <5a0c8f09-4f01-45e9-892c-86342c0d0ec5@xen.org>
Date: Mon, 5 Feb 2024 10:13:13 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
To: Peng Fan <peng.fan@nxp.com>, John Ernberg <john.ernberg@actia.se>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Cc: Jonas Blixt <jonas.blixt@actia.se>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <20240131114952.305805-1-john.ernberg@actia.se>
 <20240131114952.305805-2-john.ernberg@actia.se>
 <a2f726f5-df4c-4d15-90af-7d59c0f1f513@xen.org>
 <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se>
 <167f0c7a-e037-446c-82f8-2584e35a7af1@xen.org>
 <a265ea2d-9b5f-4726-9395-448b1b669839@actia.se>
 <012b5f83-2f9b-4477-965e-07b8506c0052@xen.org>
 <DU0PR04MB94170CB1F77E3D594A6D0E9488402@DU0PR04MB9417.eurprd04.prod.outlook.com>
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
In-Reply-To: <DU0PR04MB94170CB1F77E3D594A6D0E9488402@DU0PR04MB9417.eurprd04.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 04/02/2024 09:40, Peng Fan wrote:
> 
> 
>> -----Original Message-----
>> From: Julien Grall <julien@xen.org>
>> Sent: 2024年2月2日 17:20
>> To: John Ernberg <john.ernberg@actia.se>; Stefano Stabellini
>> <sstabellini@kernel.org>; Bertrand Marquis <bertrand.marquis@arm.com>;
>> Michal Orzel <michal.orzel@amd.com>; Volodymyr Babchuk
>> <Volodymyr_Babchuk@epam.com>; Peng Fan <peng.fan@nxp.com>
>> Cc: Jonas Blixt <jonas.blixt@actia.se>; xen-devel@lists.xenproject.org
>> Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
>>
>> On 01/02/2024 16:17, John Ernberg wrote:
>>> On 2/1/24 13:20, Julien Grall wrote:
>>>>
>>>>
>>>> On 31/01/2024 15:32, John Ernberg wrote:
>>>>> Hi Julien,
>>>>
>>>> Hi John,
>>>>
>>>>> On 1/31/24 13:22, Julien Grall wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On 31/01/2024 11:50, John Ernberg wrote:
>>>>>>> When using Linux for dom0 there are a bunch of drivers that need
>>>>>>> to do SMC SIP calls into the PSCI provider to enable certain
>>>>>>> hardware bits like the watchdog.
>>>>>>
>>>>>> Do you know which protocol this is under the hood. Is this SCMI?
>>>>>
>>>>> I think I confused myself here when I wrote the commit log.
>>>>>
>>>>> The EL3 code in our case is ATF, and it does not appear to be SCMI,
>>>>> nor PSCI. The register usage of these SMC SIP calls are as follows:
>>>>> a0 - service
>>>>> a1 - function
>>>>> a2-a7 - args
>>>>>
>>>>> In ATF the handler is declared as a runtime service.
>>>>>
>>>>> Would the appropriate commmit message here be something along the
>>>>> lines of below?
>>>>> """
>>>>> When using Linux for dom0 there are a bunch of drivers that need to
>>>>> do   SMC SIP calls into the firmware to enable certain hardware bits
>>>>> like the watchdog.
>>>>> """
>>>>
>>>> It reads better thanks.
>>>>
>>>> [...]
>>>>
>>>>>> But even if we restrict to dom0, have you checked that none of the
>>>>>> SMCs use buffers?
>>>>> I haven't found any such instances in the Linux kernel where a
>>>>> buffer is used. Adding a call filtering like suggested below
>>>>> additions of such functions can be discovered and adapted for if they
>> would show up later.
>>>>>>
>>>>>> Rather than providing a blanket forward, to me it sounds more like
>>>>>> you want to provide an allowlist of the SMCs. This is more
>>>>>> futureproof and avoid the risk to expose unsafe SMCs to any domain.
>>>>>>
>>>>>> For an example, you can have a look at the EEMI mediator for Xilinx.
>>>>>
>>>>> Ack. Do you prefer to see only on SMCCC service level or also on
>>>>> function level? (a1 register, per description earlier)
>>>>
>>>> I am not sure. It will depend on whether it is correct to expose
>>>> *all* the functions within a service level and they have the same format.
>>>>
>>>> If you can't guarantee that, then you will most likely need to
>>>> allowlist at the function level.
>>>>
>>>> Also, do you have a spec in hand that would help to understand which
>>>> service/function is implemented via those SMCs?
>>>
>>> I don't have the spec unfortunately, but I will add a filter on both
>>> service and function for V2 and we'll take it from there.
>>
>> @Peng, do you have any specification you could share? How stable is the
>> interface?
> 
> No specification, the use is IMX_SIP_X in linux kernel source.
> 
> Such as IMX_SIP_RTC, IMX_SIP_TIMER
> 
> It is stable and no change, we only add new SIP macro if needed
> and no change the meaning or reuse old SIP.

Thanks for the answer. It is a bit unfortunate there are no 
specification, but at least they are stable.

I have searched IMX_SIP in Linux, there doesn't seem many so we could 
allowlist them (see more below). Do you know if there are more necessary 
that are not yet upstreamed in Linux?


Looking through the list, there are some that probably want a bit more 
discussion on whether we want to expose them:
   * IMX_SIP_CPUFREQ: Right now, dom0 is not aware of the full system. 
So it may take wrong decision.
   * IMX_SIP_DDR_DVFS: Some operation seems to take the number of online 
CPUs. Dom0 doesn't know that
   * IMX_SIP_TIMER_*:  This seems to be related to the watchdog. 
Shouldn't dom0 rely on the watchdog provided by Xen instead? So those 
call will be used by Xen.

Also, what happen if we don't expose those SMC to dom0?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:21:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:21:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675759.1051193 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWw5z-0001Yb-AU; Mon, 05 Feb 2024 10:20:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675759.1051193; Mon, 05 Feb 2024 10:20:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWw5z-0001YU-7d; Mon, 05 Feb 2024 10:20:59 +0000
Received: by outflank-mailman (input) for mailman id 675759;
 Mon, 05 Feb 2024 10:20:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GYrN=JO=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rWw5x-0001YO-Od
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:20:57 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3f17d1d4-c410-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:20:55 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-557dcb0f870so5482646a12.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 02:20:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f17d1d4-c410-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1707128455; x=1707733255; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nFSvxJvVHBhNYhugdUop+64G9e1+rgXM4OFSzDZShnk=;
        b=epyJ5G3CP1dHDKzCH480BLwKoaBFxSOy/CJq/hZg5HkTTYiJI53qNvXjZ+qTTujA0c
         qjFTs6aeuHmfHC6h/q87HcRshF30jO2QoEqmtuiTyh+L+u5RD7wXpfxsss4mR1R2d8aw
         gHuBijXVj1sUA3o9/jKC2BUgEoOd2U4gy0sDT3QVrAOCZyECbX14mvlYnO58S86em2h+
         fZAPgVH93SJQQ5/b8g4TLdWWhn2Bc1p4KMTx4KYrwHsOXdU4QhfEKgKNeqrK43XFIpiw
         FlQ7hWEPFTnvq84AemrQPrNcoHWMer0gnDWZCuOMPptiEwbiSfLdovaedQJMAxBoenqy
         iwTw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707128455; x=1707733255;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=nFSvxJvVHBhNYhugdUop+64G9e1+rgXM4OFSzDZShnk=;
        b=BeB0KA5+xFCgevT1aZ5LXa62bJegYyhVfgB2yulW8/JgYtA2qUlEgeL1kMkB+qU+9+
         UqB0PI/xd+Ad7S//3oV0MtfGg4vp5VJp6SUDu5RZ2EzsTWz1ZnGgqi3XMDRJq8sngPbc
         20Yd9zXoeaPxY5FiWT1i7du25E4O1D5CGi33lOoUQn6qmU/S9IgGmYCSMlEhtEIK3uY3
         1pxubvxe+zAfGB7vpx4jl3Qx6rqyd0Osm4H+1qXxPlIC4TFlzobsXjk9XoTktRm42qF7
         QxOTNKE2vhSW78ayFnn/fhkChazmE1qhcfeGivBZW6UxUND4qlqC8s7SWQ/NeI4ftNRQ
         o4ag==
X-Gm-Message-State: AOJu0YzSpRTjLuUnDryP29DH9sy75Yh/ypPr++sP6xQ6N1pWr6SgkF/A
	8x1n/Ht73Q18RDc5Rcg96KwESooHcZTXAftlVpuKuS8DbPL6vBH7EhXVuPGM13dsQ2RbCeTyG8u
	oaqxcojDi35RDgpmQO3nWxivArCgOe/HZ/omd4Q==
X-Google-Smtp-Source: AGHT+IFoMjpt0lJmyJWTNcLsN9OjnM2LLpneZg7//GnpHtSmZhIn1tnXmpwZ4yGZ4ZnYKgucmiej022il7YZKi43MAM=
X-Received: by 2002:aa7:d804:0:b0:55f:fed8:ea2f with SMTP id
 v4-20020aa7d804000000b0055ffed8ea2fmr4169510edq.25.1707128454743; Mon, 05 Feb
 2024 02:20:54 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-5-carlo.nonato@minervasys.tech> <cf23d8a8-7111-4014-adc7-93ecd5f110ec@suse.com>
 <CAG+AhRW4Ng7wkhB5f2H5ruCmYgium0YXZgA_bu7=T6HErUEg2A@mail.gmail.com> <82e01801-2fef-4a5b-b343-2cd4a99af016@suse.com>
In-Reply-To: <82e01801-2fef-4a5b-b343-2cd4a99af016@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Mon, 5 Feb 2024 11:20:43 +0100
Message-ID: <CAG+AhRUasNChBjPU5doEfbTHvY75yUt_NJ-p4PUo-RcjpFsnsA@mail.gmail.com>
Subject: Re: [PATCH v6 04/15] xen/arm: add Dom0 cache coloring support
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Marco Solieri <marco.solieri@minervasys.tech>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Mon, Feb 5, 2024 at 10:35=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 03.02.2024 12:39, Carlo Nonato wrote:
> > On Thu, Feb 1, 2024 at 2:30=E2=80=AFPM Jan Beulich <jbeulich@suse.com> =
wrote:
> >> On 29.01.2024 18:18, Carlo Nonato wrote:
> >>> --- a/xen/common/llc-coloring.c
> >>> +++ b/xen/common/llc-coloring.c
> >>> @@ -17,6 +17,63 @@ size_param("llc-way-size", llc_way_size);
> >>>  /* Number of colors available in the LLC */
> >>>  static unsigned int __ro_after_init max_nr_colors =3D CONFIG_NR_LLC_=
COLORS;
> >>>
> >>> +static unsigned int __initdata dom0_colors[CONFIG_NR_LLC_COLORS];
> >>> +static unsigned int __initdata dom0_num_colors;
> >>> +
> >>> +/*
> >>> + * Parse the coloring configuration given in the buf string, followi=
ng the
> >>> + * syntax below.
> >>> + *
> >>> + * COLOR_CONFIGURATION ::=3D COLOR | RANGE,...,COLOR | RANGE
> >>> + * RANGE               ::=3D COLOR-COLOR
> >>> + *
> >>> + * Example: "0,2-6,15-16" represents the set of colors: 0,2,3,4,5,6,=
15,16.
> >>> + */
> >>> +static int parse_color_config(const char *buf, unsigned int *colors,
> >>> +                              unsigned int num_colors, unsigned int =
*num_parsed)
> >>
> >> Is this function going to be re-used? If not, it wants to be __init.
> >> If so, I wonder where the input string is going to come from ...
> >
> > You're right. It needs __init.
>
> Am I misremembering to have spotted a non-init use in a later patch?

The only usage of the function other than for parameter parsing is in
domain_set_llc_colors_from_str() which is in turn used in dom0less domain
creation, so from another __init function.

Thanks.

> >> Also "num_colors" looks to be misnamed - doesn't this specify an
> >> upper bound only?
> >
> > It's the real size of the colors array.
>
> Hence my remark: It is _not_ the number of colors.
>
> >>> +int __init dom0_set_llc_colors(struct domain *d)
> >>> +{
> >>> +    unsigned int *colors;
> >>> +
> >>> +    if ( !dom0_num_colors )
> >>> +        return domain_set_default_colors(d);
> >>> +
> >>> +    colors =3D alloc_colors(dom0_num_colors);
> >>> +    if ( !colors )
> >>> +        return -ENOMEM;
> >>> +
> >>> +    memcpy(colors, dom0_colors, sizeof(unsigned int) * dom0_num_colo=
rs);
> >>
> >> sizeof(*colors) or some such please. Plus a check that colors and
> >> dom0_colors are actually of the same type. Alternatively, how about
> >> making dom0_colors[] __ro_after_init? Is this too much of a waste?
> >
> > You mean an ASSERT on the two arrays type?
>
> I don't think you can use ASSERT() for such very well. It's runtime
> check, when here we want a build-time one. I'd therefore rather see
> it be something like
>
>    (void)(colors =3D=3D dom0_colors);
>
> Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:33:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:33:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675764.1051203 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwIF-0003lL-Cc; Mon, 05 Feb 2024 10:33:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675764.1051203; Mon, 05 Feb 2024 10:33:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwIF-0003lE-9o; Mon, 05 Feb 2024 10:33:39 +0000
Received: by outflank-mailman (input) for mailman id 675764;
 Mon, 05 Feb 2024 10:33:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W3Sh=JO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rWwID-0003ks-GV
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:33:37 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 04c78a58-c412-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:33:36 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-40fc22f372cso32350365e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 02:33:36 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 j17-20020a05600c1c1100b0040e88d1422esm8233097wms.31.2024.02.05.02.33.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 02:33:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04c78a58-c412-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707129216; x=1707734016; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=p9WUHKMRlDO1RXRhiCjOMGqsymAumuPJ2v+QEN13b6c=;
        b=PeBsjPPCtcNGJ4hp5P2Wn7OdIjiRreqgEXwqs1b3Pc7msf0aCFXR4trNMZjzDDzmXg
         SeilCpUR/1jUWJqrEg1DqywsgPQUNjXhgG4U4e7bpEmZ5P/fKPrlQOPor5u3gMpNnEEJ
         GWcwCC4eNd17sUA9B9rUpdo8L7+lR5k8tuY9k=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707129216; x=1707734016;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=p9WUHKMRlDO1RXRhiCjOMGqsymAumuPJ2v+QEN13b6c=;
        b=MOl5BBLzD0PvC6Xm9ZfPNo/QEsn/wcshNxYwXpQ8IaUDvuBIDmQ1UMLU2jQVnP7o4E
         S8DtCHqIylk/vBMFZaC4kaU3o/EAS0cbWD7OfDKIRPe073jlch+wCDxoBlV0KJ6nHyFi
         UQxCxPAkd7bBCVfc+kAaD85a403iBq8+SBAZLUSgaIIsLnM1dCU2LXKIRUvYu2GopQfo
         I30WF1tTZXnjJRDHIHdLYfRHYDVaX4FiJGP5QBgGczoBgFFg3egxfBq12sO4HDjwH+a5
         JKW9mfarbASBSOeOGq1T928L2dgjOz+XgOYOTaQwqEwrWwA2fthsDxQafm7bWm3payjW
         HiAg==
X-Gm-Message-State: AOJu0YyIC6ephbz9UTr+RuVK6rd7dD9wQG/bA9fsplHS7iLty8OmvgoK
	g3l+3T9kjht7/KlNvGiGBLw7ue1o/GoBQF0RksWAy3wkQzZw7KpDsybR61Avjf+jvNb6zZhmqgo
	O
X-Google-Smtp-Source: AGHT+IH59QdIYJYfzETA4uJONxxaETXUnFYaAdv3Twms0dtvYpWQK04cJXCrbetD/040190RJDHBhA==
X-Received: by 2002:a05:600c:35d3:b0:40e:ccf8:7539 with SMTP id r19-20020a05600c35d300b0040eccf87539mr4257504wmq.34.1707129215909;
        Mon, 05 Feb 2024 02:33:35 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVoI8FTxA3pVpFOezhwfbxzqEsVm5mGJdsPVcH7bMd++33s2ScO6JMJZMaS2734u0IRGwgFNahBiXjWBeNcox9ExCaeh/UJI4NYKGiHUGpf5A48SNJXEl7mCGM=
Message-ID: <2fb3ba7a-563f-4969-a745-4c6959b3ef02@citrix.com>
Date: Mon, 5 Feb 2024 10:33:34 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/ucode: Remove accidentally introduced tabs
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240202180003.1295735-1-andrew.cooper3@citrix.com>
 <ed5ab7f6-1b5a-4953-8add-8a2df41b748c@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ed5ab7f6-1b5a-4953-8add-8a2df41b748c@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 05/02/2024 8:57 am, Jan Beulich wrote:
> On 02.02.2024 19:00, Andrew Cooper wrote:
>> Fixes: cf7fe8b72dea ("x86/ucode: Fix stability of the raw CPU Policy rescan")
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> I had spotted those as odd while backporting, but didn't consider the
> issue bad enough to raise it when the change had gone in already anyway.
>
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks.  I noticed too on backports to 4.17.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:42:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:42:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675768.1051213 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwQY-0005vU-6w; Mon, 05 Feb 2024 10:42:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675768.1051213; Mon, 05 Feb 2024 10:42:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwQY-0005vN-33; Mon, 05 Feb 2024 10:42:14 +0000
Received: by outflank-mailman (input) for mailman id 675768;
 Mon, 05 Feb 2024 10:42:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rWwQW-0005vH-TA
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:42:12 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWwQW-0002iD-Ng; Mon, 05 Feb 2024 10:42:12 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWwQW-0005u3-HE; Mon, 05 Feb 2024 10:42:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=HRvNL3oNBnTz6X7s0C07QdIHNQBkiOBTkUsJs7Sizb0=; b=dSwOTD4QdLxTgQTMUr1ZmmYjj4
	CxfWfgo4SzpJed4h7mCRb+pibcezhmDH8KKHO9nFMk8qZ6xpDlo8qgq8tZntuDzRUTO49ckwGpL5u
	2hVwxghri1WBKpvowIpa1mFTLoTQvcXbJOo00SL103KW1Cke1sw7mkA10hXWOqJILPWc=;
Message-ID: <e82e5a15-88e8-4d24-b872-548b608cebbc@xen.org>
Date: Mon, 5 Feb 2024 10:42:10 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH] xen/arm: improve handling of load/store instruction
 decoding
Content-Language: en-GB
To: =?UTF-8?Q?Alex_Benn=C3=A9e?= <alex.bennee@linaro.org>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, bertrand.marquis@arm.com,
 Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
References: <20240131175043.1488886-1-alex.bennee@linaro.org>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240131175043.1488886-1-alex.bennee@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Alex,

On 31/01/2024 17:50, Alex Bennée wrote:
> While debugging VirtIO on Arm we ran into a warning due to memory
> being memcpy'd across MMIO space. While the bug was in the mappings
> the warning was a little confusing:
> 
>    (XEN) d47v2 Rn should not be equal to Rt except for r31
>    (XEN) d47v2 unhandled Arm instruction 0x3d800000
>    (XEN) d47v2 Unable to decode instruction
> 
> The Rn == Rt warning is only applicable to single register load/stores
> so add some verification steps before to weed out unexpected accesses.
> 
> I updated the Arm ARM reference to the online instruction decoding
> table which will hopefully be more stable than the Arm ARM section
> numbers.

I am not sure if the links to the Arm websites are stable. But from 
past, experience, URL tends to disappear after a while. This is why we 
went with the section + the Arm spec.

This also has the advantage that we can check any differences between 
version. So my preference is to stick the Arm ARM reference. Bertrand, 
Michal, Stefano, any opinions?

Anyway, the idea looks fine to me. I left mostly some style comments.

> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
> ---
>   xen/arch/arm/decode.c | 20 ++++++++++++++++++++
>   xen/arch/arm/decode.h | 38 +++++++++++++++++++++++++++++++++++---
>   2 files changed, 55 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/arm/decode.c b/xen/arch/arm/decode.c
> index 2537dbebc1..824025c24c 100644
> --- a/xen/arch/arm/decode.c
> +++ b/xen/arch/arm/decode.c
> @@ -87,6 +87,26 @@ static int decode_arm64(register_t pc, mmio_info_t *info)
>           return 1;
>       }
>   
> +    /*
> +     * Check this is a load/store of some sort
> +     */

Coding style: This is a single line comment, so the preferred format is:

/* .... */

> +    if ( (opcode.top_level.op1 & 0b0101) != 0b0100 )

NIT: We are trying to avoid opcoding value in Xen. Can you add some  define?

> +    {
> +        gprintk(XENLOG_ERR, "Not a load/store instruction op1=%d",

Does the value need to be signed?

> +                opcode.top_level.op1);
> +        goto bad_loadstore;
> +    }
> +
> +    /*
> +     * We are only expecting single register load/stores
> +     */

Same here.

> +    if ( (opcode.ld_st.op0 & 0b0011) != 0b0011 )
> +    {
> +        gprintk(XENLOG_ERR, "Not single register load/store op0=%d",

Same remark as above.

> +                opcode.ld_st.op0);
> +        goto bad_loadstore;
> +    }
> +
>       /*
>        * Refer Arm v8 ARM DDI 0487G.b, Page - C6-1107
>        * "Shared decode for all encodings" (under ldr immediate)
> diff --git a/xen/arch/arm/decode.h b/xen/arch/arm/decode.h
> index 13db8ac968..b1580178eb 100644
> --- a/xen/arch/arm/decode.h
> +++ b/xen/arch/arm/decode.h
> @@ -24,9 +24,27 @@
>   #include <asm/processor.h>
>   
>   /*
> - * Refer to the ARMv8 ARM (DDI 0487G.b), Section C4.1.4 Loads and Stores
> - * Page 318 specifies the following bit pattern for
> - * "load/store register (immediate post-indexed)".
> + * From:
> + * https://developer.arm.com/documentation/ddi0602/2023-12/Index-by-Encoding
> + *
> + * Top level encoding:
> + *
> + *   31  30  29 28  25 24                                             0
> + * ___________________________________________________________________
> + * |op0 | x  x |  op1 |                                               |
> + * |____|______|______|_______________________________________________|
> + *
> + * op0 = 0 is reserved
> + * op1 = x1x0 for Loads and Stores
> + *
> + * Loads and Stores
> + *
> + *  31    28 27   26   25  24             9 8                        0
> + * ___________________________________________________________________
> + * |  op0   | 1 | op1 | 0 |       op2      |                          |
> + * |________|___|_____|___|________________|__________________________|
> + *
> + * Load/store register (immediate post-indexed)
>    *
>    * 31 30 29  27 26 25  23   21 20              11   9         4       0
>    * ___________________________________________________________________
> @@ -35,6 +53,20 @@
>    */
>   union instr {
>       uint32_t value;
> +    struct {
> +        unsigned int ign2:25;
> +        unsigned int op1:4;     /* instruction class */
> +        unsigned int ign1:2;
> +        unsigned int op0:1;     /* value = 1b */
> +    } top_level;
> +    struct {
> +        unsigned int ign1:9;
> +        unsigned int op2:15;
> +        unsigned int fixed1:1; /* value = 0b */
> +        unsigned int op1:1;
> +        unsigned int fixed2:1; /* value = 1b */
> +        unsigned int op0:4;
> +    } ld_st;
>       struct {
>           unsigned int rt:5;     /* Rt register */
>           unsigned int rn:5;     /* Rn register */

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:45:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:45:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675771.1051222 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwTk-0006Uq-K9; Mon, 05 Feb 2024 10:45:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675771.1051222; Mon, 05 Feb 2024 10:45:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwTk-0006Uj-HY; Mon, 05 Feb 2024 10:45:32 +0000
Received: by outflank-mailman (input) for mailman id 675771;
 Mon, 05 Feb 2024 10:45:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8nl=JO=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1rWwTj-0006Ud-Kt
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:45:31 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ae75e977-c413-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:45:30 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-5606f2107ebso1045610a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 02:45:30 -0800 (PST)
Received: from localhost.localdomain ([185.25.64.249])
 by smtp.gmail.com with ESMTPSA id
 st10-20020a170907c08a00b00a36f7891799sm4137629ejc.193.2024.02.05.02.45.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 02:45:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae75e977-c413-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707129930; x=1707734730; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=KFRLF/eDxgbHDTI7/GHRl3I+hTFf/BFZxPyQ0bGdHe8=;
        b=QaQkvvKu40zmNYr3wK07W236H9j17Li3ElDA9v76rktEmIfdMXlUKJHJlgNadnmJ8u
         FTjjyAM9gHv7KhHVwS5NFKoGyvIXmGkeUhc+jhmW9qisaGdVjlrL3hiOnnKijuirEVcZ
         6ryjyOGfA3lCx4ZieD75YzJwUkTMyPvE6+Qjo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707129930; x=1707734730;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=KFRLF/eDxgbHDTI7/GHRl3I+hTFf/BFZxPyQ0bGdHe8=;
        b=TVDLDPHSUQa/ErGebz7YX64U+QTiF0oYbhSIjX9nxzbSvkyX+wabeUXGSqqFuKkgUS
         2YxoV9SRytOOD5qbQ/+IQhfioJtnvqFHQyZryXdS4XS8RUceS9GrmNmm8ulCI0gKS83V
         Qwfdt0LwpC6IoTStlfVTRTzMsv367ktU5ILv1ZO5nP1b3ogxCjsdRnJVrj4CHqhebJfW
         neyPmLUFyC/GAcCOqf0lwY5H3UmK+4Xi2UjQ5YcOrx3A/XfCYsQya4fZd7BrGvt3omuS
         hOVMGeZVrVR6iTjpy3FLw4vlTuzkQGydF1oxEgf+yhKsOjuz7BpUDMGaxzn/x7jPUY93
         85OQ==
X-Gm-Message-State: AOJu0Yw5H4UA0n8qhbvvcV8Ua0Rwq2daItmLRZ1McApX29OE/XHvh9f0
	oFLxpa0gD6QS4O4F0zasI7VXuYwqF4lgteZkJsRzp//Eev5lK9rj+MQfmvXudNRUehcH2HguKu8
	fc4ev7g==
X-Google-Smtp-Source: AGHT+IHnzcS/aHFsA8Lz6lv0ncTPBBlXB9hZV/8pptAbo5+iwwJyGv0h9ZJFqEAj3VnHRjBASzhXeA==
X-Received: by 2002:a17:906:5647:b0:a34:9d98:ac1f with SMTP id v7-20020a170906564700b00a349d98ac1fmr6926315ejr.24.1707129930225;
        Mon, 05 Feb 2024 02:45:30 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWHQYT6OFTXRTQpovzE1OQK79KJxFz9IJZd+StjlMDMHgTfvS7pKpWFLc8ExpKIcf73WbY26B3pu3NcIvD7ZK95+WrpUOH4YQCzTkl6YrJveqzV2Vt8x+SVGWJcxHCnNbKdMSabqhqQBvemBqedibcwz6GSHmRGj/eREFEvnkWPCCBTvnO6u6Nem95vwSVLyjrcO/F0TDn/AXYn4tGZ40F4ECUVaaLJX63tHsU8wkIwCizL/M5vzNfvYwJC8pWofCCIbiqw6yHKbZ8O9oH6pGmLIOk=
From: Frediano Ziglio <frediano.ziglio@cloud.com>
To: Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Dario Faggioli <dfaggioli@suse.com>,
	"Daniel P . Smith" <dpsmith@apertussolutions.com>
Cc: xen-devel@lists.xenproject.org,
	Frediano Ziglio <frediano.ziglio@cloud.com>
Subject: [PATCH 1/5] Constify some parameters
Date: Mon,  5 Feb 2024 10:45:04 +0000
Message-Id: <20240205104504.14830-1-frediano.ziglio@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Make clean they are not changed in the functions.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
---
 xen/arch/x86/pv/callback.c | 4 ++--
 xen/common/sched/compat.c  | 2 +-
 xen/common/sched/core.c    | 2 +-
 xen/xsm/flask/flask_op.c   | 8 ++++----
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/pv/callback.c b/xen/arch/x86/pv/callback.c
index 17829304fe..caec4fb16f 100644
--- a/xen/arch/x86/pv/callback.c
+++ b/xen/arch/x86/pv/callback.c
@@ -48,7 +48,7 @@ static void unregister_guest_nmi_callback(void)
     memset(t, 0, sizeof(*t));
 }
 
-static long register_guest_callback(struct callback_register *reg)
+static long register_guest_callback(const struct callback_register *reg)
 {
     long ret = 0;
     struct vcpu *curr = current;
@@ -102,7 +102,7 @@ static long register_guest_callback(struct callback_register *reg)
     return ret;
 }
 
-static long unregister_guest_callback(struct callback_unregister *unreg)
+static long unregister_guest_callback(const struct callback_unregister *unreg)
 {
     long ret;
 
diff --git a/xen/common/sched/compat.c b/xen/common/sched/compat.c
index dd97593630..a02204ec9a 100644
--- a/xen/common/sched/compat.c
+++ b/xen/common/sched/compat.c
@@ -26,7 +26,7 @@ CHECK_sched_shutdown;
 CHECK_sched_remote_shutdown;
 #undef xen_sched_remote_shutdown
 
-static int compat_poll(struct compat_sched_poll *compat)
+static int compat_poll(const struct compat_sched_poll *compat)
 {
     struct sched_poll native;
 
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index d177c675c8..c5db373972 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -1431,7 +1431,7 @@ static void vcpu_block_enable_events(void)
     vcpu_block();
 }
 
-static long do_poll(struct sched_poll *sched_poll)
+static long do_poll(const struct sched_poll *sched_poll)
 {
     struct vcpu   *v = current;
     struct domain *d = v->domain;
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index b866e8d05f..ea7dd10dc8 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -469,7 +469,7 @@ static int flask_security_load(struct xen_flask_load *load)
     return ret;
 }
 
-static int flask_devicetree_label(struct xen_flask_devicetree_label *arg)
+static int flask_devicetree_label(const struct xen_flask_devicetree_label *arg)
 {
     int rv;
     char *buf;
@@ -492,7 +492,7 @@ static int flask_devicetree_label(struct xen_flask_devicetree_label *arg)
 
 #ifndef COMPAT
 
-static int flask_ocontext_del(struct xen_flask_ocontext *arg)
+static int flask_ocontext_del(const struct xen_flask_ocontext *arg)
 {
     int rv;
 
@@ -506,7 +506,7 @@ static int flask_ocontext_del(struct xen_flask_ocontext *arg)
     return security_ocontext_del(arg->ocon, arg->low, arg->high);
 }
 
-static int flask_ocontext_add(struct xen_flask_ocontext *arg)
+static int flask_ocontext_add(const struct xen_flask_ocontext *arg)
 {
     int rv;
 
@@ -550,7 +550,7 @@ static int flask_get_peer_sid(struct xen_flask_peersid *arg)
     return rv;
 }
 
-static int flask_relabel_domain(struct xen_flask_relabel *arg)
+static int flask_relabel_domain(const struct xen_flask_relabel *arg)
 {
     int rc;
     struct domain *d;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:46:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:46:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675775.1051233 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwV6-00074Z-2v; Mon, 05 Feb 2024 10:46:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675775.1051233; Mon, 05 Feb 2024 10:46:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwV6-00074Q-0C; Mon, 05 Feb 2024 10:46:56 +0000
Received: by outflank-mailman (input) for mailman id 675775;
 Mon, 05 Feb 2024 10:46:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8nl=JO=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1rWwV4-00074G-BA
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:46:54 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dfca4a3a-c413-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:46:53 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-56001d49cc5so2785901a12.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 02:46:53 -0800 (PST)
Received: from localhost.localdomain ([185.25.64.249])
 by smtp.gmail.com with ESMTPSA id
 ty24-20020a170907c71800b00a372a739731sm3409598ejc.45.2024.02.05.02.46.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 02:46:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dfca4a3a-c413-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707130013; x=1707734813; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=SzenSDt43d9dKKfeFM0SDMDstPJFaTNqj96pw7JED8k=;
        b=W/p+LcO90q9l0VU7eVvx2W0lx0DImxZAmXC5LLtLOWpTmI1jZYbP6s32gsZMtFYj0Z
         Le/cEnYozgQJMjCDgUba0DKrFmoCRaQuavg5sXrfc2YQndCReuipWYqOIP5h8yZs3QIh
         zU3V0iDYNlmktFWKyfb7h61+Iw5EL5dgGfFD8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707130013; x=1707734813;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=SzenSDt43d9dKKfeFM0SDMDstPJFaTNqj96pw7JED8k=;
        b=FuFiBcwuT+6TSv5R6inl13J9h4MFTGn4dKQlC0blsrq4nQ/U69ksvB+PS5OZ2RtQZH
         rsV00hQPQGe0DDYHA1C4rK8skQLyQUSBhp21lyZP8EucO5AkUmqKs9H544TgcNGQH/du
         OSo8N0dfeBvJtTWvnzMWz9V/uXCSiHbU520HyRRW8L6S5SHsE3oOOY23hXg42O7PzlUr
         Uo6q9CjHwmqCraBYCr/nRrbYnreyGWhIsORiBI3nHnZGJp9xsOM65nOI0ElfhkqREDOy
         sSZH1DWuwO2mtG2sk1lysBiyhNYEiXV5ZeqQYx014iyFKMiA2GfHfgRRUfkMHj6NyjdZ
         5Asw==
X-Gm-Message-State: AOJu0YysJSrIQP40BgqWEwhuRbJX6bWpjyH94KHsdM3uEQzEbGkACfVu
	+yS13vZcQzLSFkvTzdTJu858feCHHkVa4NjytzA5aPNjbDr47OYy5gm2wklKScE=
X-Google-Smtp-Source: AGHT+IGpt7ONVCe4L9IOR0v5GCjP3NS8ipucmpxnpP8IN6nrT22e82WK+MJKeNOvIaSiGD1ieUkC8Q==
X-Received: by 2002:a17:907:77d0:b0:a37:152e:2e61 with SMTP id kz16-20020a17090777d000b00a37152e2e61mr6562814ejc.71.1707130012924;
        Mon, 05 Feb 2024 02:46:52 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUq9xZccYn5t6aR5xhQOBUOB6URWktyY2MLQj+RFNhncs7AC8XGpc5ZJ8q8b9AxJqq5FOjIoJruXSeyy9HFIHwAyLA2/Wb0UZeYhZBFOVtidxKUAlRXBX9Xh9zkx2V4/ekj5veA5U3wuNvo/MYAOJD55/LB0Ul0cvMY4aVOg4Lsheg8QKMR6kjRtw8oe5/wMFhwcQ41l1yzTAfqgdNsLbQQ9U1AQgJuAAK4aw==
From: Frediano Ziglio <frediano.ziglio@cloud.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Cc: xen-devel@lists.xenproject.org,
	Frediano Ziglio <frediano.ziglio@cloud.com>
Subject: [PATCH 2/5] Remove useless assignment
Date: Mon,  5 Feb 2024 10:46:37 +0000
Message-Id: <20240205104637.14868-1-frediano.ziglio@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The variable is assigned later, that value is never used.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
---
 xen/common/event_channel.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
index a7a004a084..15aec5dcbb 100644
--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -324,8 +324,6 @@ int evtchn_alloc_unbound(evtchn_alloc_unbound_t *alloc, evtchn_port_t port)
         goto out;
     }
 
-    rc = 0;
-
     chn = evtchn_from_port(d, port);
 
     rc = xsm_evtchn_unbound(XSM_TARGET, d, chn, alloc->remote_dom);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:48:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:48:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675778.1051244 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwW8-0007n2-Dk; Mon, 05 Feb 2024 10:48:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675778.1051244; Mon, 05 Feb 2024 10:48:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwW8-0007mv-8v; Mon, 05 Feb 2024 10:48:00 +0000
Received: by outflank-mailman (input) for mailman id 675778;
 Mon, 05 Feb 2024 10:47:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8nl=JO=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1rWwW6-0007md-Qe
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:47:58 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0523a1f2-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:47:56 +0100 (CET)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-511538be947so476423e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 02:47:56 -0800 (PST)
Received: from localhost.localdomain ([185.25.64.249])
 by smtp.gmail.com with ESMTPSA id
 vg7-20020a170907d30700b00a354a5d2c39sm4188301ejc.31.2024.02.05.02.47.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 02:47:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0523a1f2-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707130075; x=1707734875; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=/qYpJWJ6YrcOns7diE+XyCUeN2tM6yt9XGqkK0qPIH4=;
        b=l0P1LythzGj619cydLWyxJigwYnXpbhFvA3dDp/D8uzu/Ar0Bw6FKUt7COc88aV6Lm
         telLIbj/cYo1Q8z6hmGZLoM0RKLyTRDxIiVAZsnp9cbTcahuomMvDCTvaFtgnAOtAugp
         V3TXNgas21klU3CAkir3+MifqT2c7bfN6kUtE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707130075; x=1707734875;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=/qYpJWJ6YrcOns7diE+XyCUeN2tM6yt9XGqkK0qPIH4=;
        b=HYNTSNo1Ba0Dam0OChJHzrL/tXyAIyn1hR8yXraXfS+JiWOuzADn3QdyKovfZruFSn
         O+PwqJrCrNjN9DuDcELMXdiugZ/p8HVqzkvPkLhskGgIvbAxBkVATDQcLqsn2SLzx/TW
         5YnGJoXab0VaGIO8GIx8GoRj0DBuR+dWmK5y3Zkon46CxwCI0Qy3xMdOYeHK1pmNPBED
         Y9wG+BXbnZ2JZYicsMcNSTDQEs4JrCfEFTDwcnRVfXtMNkSubnWU7nxD3yrxsfR9HvkG
         eVfZ0YpdpRagDp8o+vQy/hFxOKtt+qK9nl4b1nDHjyRWk3l5iDMkbQVuOWNlhkk29n8V
         ObyQ==
X-Gm-Message-State: AOJu0YxBG5x2nkB13Sio5W17+3Mi8aUlmZLnVOPWTFOeJy3cLmmzFkTJ
	dSsy141IOF/hMvjrie/1TOmqDyreLL+rzOnUd1FEU09zCRiGpkeHwqexpQEjRNA=
X-Google-Smtp-Source: AGHT+IFjkCT9bPQ+NJkTyB8Yw50hamladXzXCjyeZSV28bhMrO5ypS+sRo/VpPqf781XBVZi81ypYw==
X-Received: by 2002:a05:6512:21ce:b0:511:add:cfe9 with SMTP id d14-20020a05651221ce00b005110addcfe9mr6991715lft.26.1707130075494;
        Mon, 05 Feb 2024 02:47:55 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWPMEAR/GZxphCk5ZD6P8CiDrKyXeA8r/3M9Ldir3dqtahAJXXBP/QdSnf4VW9aiwxDkBcL08qVdLA2bMuAlvZXSz2z1G0FD18ntWniP+zWVS9emXF1OoK6PrM+rYsY4dtmVBqwifzOotQKh1DqDVqcoXWjSsf7bFVmWmcgyas5I8Xuq11Qe0yEF7EqF9Ka18IYpuAxauGiQKrkLyw7kxbm
From: Frediano Ziglio <frediano.ziglio@cloud.com>
To: Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Cc: xen-devel@lists.xenproject.org,
	Frediano Ziglio <frediano.ziglio@cloud.com>
Subject: [PATCH 3/5] Use more specific constant
Date: Mon,  5 Feb 2024 10:47:40 +0000
Message-Id: <20240205104740.14905-1-frediano.ziglio@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

__HYPERVISOR_arch_1 and __HYPERVISOR_paging_domctl_cont for x86
have the same value but this function is handling
"paging_domctl_cont" hypercall so using the latter mnemonic in
the code is more clear.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
---
 xen/arch/x86/mm/paging.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/mm/paging.c b/xen/arch/x86/mm/paging.c
index 541c2ea9b2..2961ca734d 100644
--- a/xen/arch/x86/mm/paging.c
+++ b/xen/arch/x86/mm/paging.c
@@ -783,7 +783,7 @@ long do_paging_domctl_cont(
     rcu_unlock_domain(d);
 
     if ( ret == -ERESTART )
-        ret = hypercall_create_continuation(__HYPERVISOR_arch_1,
+        ret = hypercall_create_continuation(__HYPERVISOR_paging_domctl_cont,
                                             "h", u_domctl);
     else if ( __copy_field_to_guest(u_domctl, &op, u.shadow_op) )
         ret = -EFAULT;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:49:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:49:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675781.1051252 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwX5-0000Hv-Lv; Mon, 05 Feb 2024 10:48:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675781.1051252; Mon, 05 Feb 2024 10:48:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwX5-0000Ho-J2; Mon, 05 Feb 2024 10:48:59 +0000
Received: by outflank-mailman (input) for mailman id 675781;
 Mon, 05 Feb 2024 10:48:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8nl=JO=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1rWwX4-0007md-7l
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:48:58 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 28a759ff-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:48:55 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a30e445602cso1093350066b.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 02:48:55 -0800 (PST)
Received: from localhost.localdomain ([185.25.64.249])
 by smtp.gmail.com with ESMTPSA id
 f23-20020a1709067f9700b00a37b0d09d9esm1201965ejr.119.2024.02.05.02.48.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 02:48:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28a759ff-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707130135; x=1707734935; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=wyOiV3AtGDkj74WIW4cpDGcIhzL87xknPQ5xMGnZjFU=;
        b=BkI7HZb+4tjFY6J2grE41eVeBiL1g9SX21GRd5BaK9d8L878fr/VBpXRD1twtggbzT
         Swo0WVXAvpyjvM+h6OV9dIjhXdRaMeeM6AhcMjXShZzsf9JIj3exCZgIK2cdzcJN6+QP
         R6PnyAQiNr7ikqJwKipIkPkoYlRazHSLcIUv8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707130135; x=1707734935;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=wyOiV3AtGDkj74WIW4cpDGcIhzL87xknPQ5xMGnZjFU=;
        b=DVJXX/K8vnotIvpB63Fpld/MsmAmas/tne5GUYR1Zc/Xt9pDzwy/L5nqlXUI+EHSUM
         Jv6CuW1DOl+1CZIpGxDOOHUQnp8m3G+gqztp2zWT7Od7QZSpCAoJAOI1nhtEV87IjL/P
         /wWjBC3/NlGx0jbl/WzGPH0vsabMMmF7OArFpL6TsssE5vI+5atFR6zYbCRuAcggmk0e
         k/iiPHMZGY3duK6I3BQFnqDi5yqaAN4uVZ1NKNDyABYVm+eLZ2gycWn5rRKnAiolg2uC
         kDoZ/GpQzTbpWxPWTrlHvEh4Igt4iB0j0yuvn5T/C/kSolYL/GE22Yv4wz6vUqg9/U/Z
         2stA==
X-Gm-Message-State: AOJu0YzS+dNurY7E0D/nPsqkj4ZyGQFgs8eZ5joe8uXQsiQhD4OFwkOI
	8s9kB+6P7OtxYgzhHU1Jab+aoDEqbaVyc4NtHaewRazkE8DGE7HxIsAuG8VuVac=
X-Google-Smtp-Source: AGHT+IHPLlbZu4hWveArh5WSuBHr1gOP3+SDiU520//wgpfaMV+kcP36tWgTHecNQ37wF0+k7GpUXw==
X-Received: by 2002:a17:907:7893:b0:a37:8574:357a with SMTP id ku19-20020a170907789300b00a378574357amr3791718ejc.4.1707130135261;
        Mon, 05 Feb 2024 02:48:55 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUVRlAMaIT1cI3q/pqat/a+scZbTiTUAU99J6Qy9e6AOqD0zjH5Jn+KZT8lmMdND00XdJqlLV+MqCccIDR+OhI/Hq+E/5JwNM4oCuKPdVlKI145C0NNblESX4jDyqErin+lxCXnEd3DZBiywKF1Z7KDi3CiQqfnLbGety0jw09DvbNdFoj8MQ==
From: Frediano Ziglio <frediano.ziglio@cloud.com>
To: Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Cc: xen-devel@lists.xenproject.org,
	Frediano Ziglio <frediano.ziglio@cloud.com>
Subject: [PATCH 4/5] Reduce assembly code size of exception entry points
Date: Mon,  5 Feb 2024 10:48:40 +0000
Message-Id: <20240205104840.14942-1-frediano.ziglio@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

We just pushed a 8-bytes zero and exception constants are
small so we can just write a single byte saving 3 bytes for
instruction.
With ENDBR64 this reduces the size of the entry point from 32 to 16
bytes (due to alignment).

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
---
 xen/arch/x86/x86_64/entry.S | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index ecdd6e5b47..b6a157d96a 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -898,28 +898,28 @@ END(handle_exception)
 FUNC(entry_DE)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_DE, 4(%rsp)
+        movb  $X86_EXC_DE, 4(%rsp)
         jmp   handle_exception
 END(entry_DE)
 
 FUNC(entry_MF)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_MF, 4(%rsp)
+        movb  $X86_EXC_MF, 4(%rsp)
         jmp   handle_exception
 END(entry_MF)
 
 FUNC(entry_XM)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_XM, 4(%rsp)
+        movb  $X86_EXC_XM, 4(%rsp)
         jmp   handle_exception
 END(entry_XM)
 
 FUNC(entry_NM)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_NM, 4(%rsp)
+        movb  $X86_EXC_NM, 4(%rsp)
         jmp   handle_exception
 END(entry_NM)
 
@@ -933,28 +933,28 @@ END(entry_DB)
 FUNC(entry_BP)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_BP, 4(%rsp)
+        movb  $X86_EXC_BP, 4(%rsp)
         jmp   handle_exception
 END(entry_BP)
 
 FUNC(entry_OF)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_OF, 4(%rsp)
+        movb  $X86_EXC_OF, 4(%rsp)
         jmp   handle_exception
 END(entry_OF)
 
 FUNC(entry_BR)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_BR, 4(%rsp)
+        movb  $X86_EXC_BR, 4(%rsp)
         jmp   handle_exception
 END(entry_BR)
 
 FUNC(entry_UD)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_UD, 4(%rsp)
+        movb  $X86_EXC_UD, 4(%rsp)
         jmp   handle_exception
 END(entry_UD)
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:49:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:49:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675783.1051262 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwXi-0000qw-Tw; Mon, 05 Feb 2024 10:49:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675783.1051262; Mon, 05 Feb 2024 10:49:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwXi-0000qp-RK; Mon, 05 Feb 2024 10:49:38 +0000
Received: by outflank-mailman (input) for mailman id 675783;
 Mon, 05 Feb 2024 10:49:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/8nl=JO=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1rWwXh-0000qL-Fv
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:49:37 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3fe642b0-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:49:34 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a26ed1e05c7so591060266b.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 02:49:34 -0800 (PST)
Received: from localhost.localdomain ([185.25.64.249])
 by smtp.gmail.com with ESMTPSA id
 gw14-20020a170906f14e00b00a35200615d7sm4185891ejb.1.2024.02.05.02.49.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 02:49:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3fe642b0-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707130174; x=1707734974; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=mBomYnkbGBCYxlJkVuCw9TZfVRMHdHHWQ2aYnOe5VSg=;
        b=ZduG/EtDOdSskHflS5AKn3Gsdi5nr9ak+nM1EZnRN5gQ1XWKoanTHXbKky5PPv6GWI
         Y/HHrgncFKxowRdXwku3TDxCV3MlyNuprcLD4m4WEH5BLmI4y6peWiXUhOyralhSgH5+
         NM2ehKTCa7qP1keSXw0MR1cHhvNz7TS58V5w8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707130174; x=1707734974;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=mBomYnkbGBCYxlJkVuCw9TZfVRMHdHHWQ2aYnOe5VSg=;
        b=rl9NITtgicDjw0dkiSZiKG+UdyA36VtpqKkNdlHLnMQOksFiIebcOj24+uZ0QwCwCw
         oDiWlCsT2EJWhQHjY25j9EoHPHOLhBHGyRqhQ+zsBFGDU0XNw54jPyyBW+pZ7pv4H+Y6
         IWAWmTk2NYk2ZsLBM4wl3vjS9CXsWuGxcEnX7ZB6y99LfFdkXLPugF472SH9ME6TP9Lt
         RWRHm3fHVFzk2scMWO43OXfcXNNiR+dgj9z5ki131rw1rHiU1c51SIewLyxUAfvsq06+
         YHOzIVDFbLyboSVhYDXJ24w/9Tx/GeCd5/ikTH7DEgLErPXlVJxz7SXEA5dNuuS2ne4w
         zUwg==
X-Gm-Message-State: AOJu0YzzU0nT9gVoThxzlNlgO7+5kgPLQIfTtfGaJ0Xl0T4tdNhQ52nB
	+rFn3e7T+if6uKhM8kUL73ci13kRecjkZgXHD2HwxYkqCrusPExlrwcUCR+r5+0=
X-Google-Smtp-Source: AGHT+IFNsmFy2DMoBL9yJr/PSVQr8mahMXw42/lGkbmjRScCrA4JYifVpRvOfnB40FPJmbY7xHmwxQ==
X-Received: by 2002:a17:906:19b:b0:a36:fd11:984 with SMTP id 27-20020a170906019b00b00a36fd110984mr7367475ejb.5.1707130174197;
        Mon, 05 Feb 2024 02:49:34 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCU+iuSb1rAJAXFuQGCIAm+fXUH1AlN8Iv0eaJtxcnaxhEdZBBLhd+y/NOcNxuUEefeRWFW9iDeQVWDFV/EAlB2c6x4RSGx1ItvJ+XdE1vpjFQhAe9k20MfR9D03mDipcc9CkV8FYMNgcvIx82Pozv3S7Rr8GBzOlXmh3mdThzD5g7K3vJnW0w==
From: Frediano Ziglio <frediano.ziglio@cloud.com>
To: Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Cc: xen-devel@lists.xenproject.org,
	Frediano Ziglio <frediano.ziglio@cloud.com>
Subject: [PATCH 5/5] Reduce assembly instruction size
Date: Mon,  5 Feb 2024 10:49:19 +0000
Message-Id: <20240205104919.14983-1-frediano.ziglio@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Use 32 bit versions in all cases, not only for registers till 8th.
This reduces the encoding from (example with r14):

     49 c7 c6 ff 7f 00 00    mov    $0x7fff,%r14

to

     41 be ff 7f 00 00       mov    $0x7fff,%r14d

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
---
 xen/arch/x86/include/asm/asm_defns.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/include/asm/asm_defns.h b/xen/arch/x86/include/asm/asm_defns.h
index 51bf1cf1e6..a9a6c21c76 100644
--- a/xen/arch/x86/include/asm/asm_defns.h
+++ b/xen/arch/x86/include/asm/asm_defns.h
@@ -136,7 +136,7 @@ register unsigned long current_stack_pointer asm("rsp");
 #define STACK_CPUINFO_FIELD(field) (1 - CPUINFO_sizeof + CPUINFO_##field)
 #define GET_STACK_END(reg)                        \
         .if .Lr##reg >= 8;                        \
-        movq $STACK_SIZE-1, %r##reg;              \
+        movl $STACK_SIZE-1, %r##reg##d;           \
         .else;                                    \
         movl $STACK_SIZE-1, %e##reg;              \
         .endif;                                   \
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:50:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:50:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675786.1051273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYC-0002CK-68; Mon, 05 Feb 2024 10:50:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675786.1051273; Mon, 05 Feb 2024 10:50:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYC-0002CD-2e; Mon, 05 Feb 2024 10:50:08 +0000
Received: by outflank-mailman (input) for mailman id 675786;
 Mon, 05 Feb 2024 10:50:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwYA-0000qL-RY
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:50:06 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 51db2365-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:50:05 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 5768621EBE;
 Mon,  5 Feb 2024 10:50:04 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 12B13132DD;
 Mon,  5 Feb 2024 10:50:04 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id WfoEA1y9wGXmNQAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:50:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51db2365-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130204; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=qLlIl7OjjYNVRH5ZQzVIOZ7jF0V85ktb0LIDG0Y7Z5o=;
	b=Kdf2O3RzkrsHXaXkzB+tVTd0QNLVXzu//kkL54Sdcv+Z+IcbPV/hIRH86tsD+Wc8ze7FGc
	a9q6x7FerhFdeVjZ8IwKakbOmr0NJZU1D/B3xTE3pqw0qY8vjR1gBHyfRnYbsQYZNOPiYL
	KNZ7/OhBiHOhDDXtCQhOyA3xVPK+yl0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130204; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=qLlIl7OjjYNVRH5ZQzVIOZ7jF0V85ktb0LIDG0Y7Z5o=;
	b=Kdf2O3RzkrsHXaXkzB+tVTd0QNLVXzu//kkL54Sdcv+Z+IcbPV/hIRH86tsD+Wc8ze7FGc
	a9q6x7FerhFdeVjZ8IwKakbOmr0NJZU1D/B3xTE3pqw0qY8vjR1gBHyfRnYbsQYZNOPiYL
	KNZ7/OhBiHOhDDXtCQhOyA3xVPK+yl0=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Julien Grall <julien@xen.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>
Subject: [PATCH v4 00/32] tools: enable xenstore-stubdom to use 9pfs
Date: Mon,  5 Feb 2024 11:49:29 +0100
Message-Id: <20240205105001.24171-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: 0.70
X-Spamd-Result: default: False [0.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,ens-lyon.org];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

This series is adding 9pfs support to Xenstore-stubdom, enabling it
to do logging to a dom0 directory.

This is a prerequisite for the final goal to add live update support
to Xenstore-stubdom, as it enables the stubdom to store its state in
a dom0 file.

The 9pfs backend is a new daemon written from scratch. Using a
dedicated 9pfs daemon has several advantages:

- it is using much less resources than a full blown qemu process
- it can serve multiple guests (the idea is to use it for other
  infrastructure domains, like qemu-stubdom or driver domains, too)
- it is designed to support several security enhancements, like
  limiting the number of files for a guest, or limiting the allocated
  file system space
- it doesn't support file links (neither hard nor soft links) or
  referencing parent directories via "..", minimizing the risk that
  a guest can "escape" from its home directory

Note that for now the daemon only contains the minimal needed
functionality to do logging from Xenstore-stubdom. I didn't want to
add all the 9pfs commands and security add-ons in the beginning, in
order to avoid needless efforts in case the idea of the daemon is
being rejected.

Changes in V4:
- patch 2 of V3 was applied
- added support of reading directories
- addressed review comments

Changes in V3:
- new patches 1, 23-25
- addressed review comments

Changes in V2:
- support of multiple rings per device
- xenlogd->xen-9pfsd rename
- addressed review comments
- fixed some bugs

Juergen Gross (32):
  tools: add access macros for unaligned data
  tools: add a new xen logging daemon
  tools/xen-9pfsd: connect to frontend
  tools/xen-9pfsd: add transport layer
  tools/xen-9pfsd: add 9pfs response generation support
  tools/xen-9pfsd: add 9pfs version request support
  tools/xen-9pfsd: add 9pfs attach request support
  tools/xen-9pfsd: add 9pfs walk request support
  tools/xen-9pfsd: add 9pfs open request support
  tools/xen-9pfsd: add 9pfs clunk request support
  tools/xen-9pfsd: add 9pfs create request support
  tools/xen-9pfsd: add 9pfs stat request support
  tools/xen-9pfsd: add 9pfs write request support
  tools/xen-9pfsd: add 9pfs read request support
  tools/libs/light: add backend type for 9pfs PV devices
  tools/xl: support new 9pfs backend xen_9pfsd
  tools/helpers: allocate xenstore event channel for xenstore stubdom
  tools/xenstored: rename xenbus_evtchn()
  stubdom: extend xenstore stubdom configs
  tools: add 9pfs device to xenstore-stubdom
  tools/xenstored: add early_init() function
  tools/xenstored: move systemd handling to posix.c
  tools/xenstored: move all log-pipe handling into posix.c
  tools/xenstored: move all socket handling into posix.c
  tools/xenstored: get own domid in stubdom case
  tools/xenstored: rework ring page (un)map functions
  tools/xenstored: split domain_init()
  tools/xenstored: map stubdom interface
  tools/xenstored: mount 9pfs device in stubdom
  tools/xenstored: add helpers for filename handling
  tools/xenstored: support complete log capabilities in stubdom
  tools/xenstored: have a single do_control_memreport()

 docs/man/xl.cfg.5.pod.in                      |   36 +-
 stubdom/xenstore-minios.cfg                   |    2 +-
 stubdom/xenstorepvh-minios.cfg                |    2 +-
 tools/Makefile                                |    1 +
 tools/golang/xenlight/helpers.gen.go          |   16 +
 tools/golang/xenlight/types.gen.go            |   13 +
 tools/helpers/init-xenstore-domain.c          |   14 +
 .../Linux/init.d/sysconfig.xencommons.in      |    1 -
 tools/hotplug/Linux/launch-xenstore.in        |    1 +
 tools/include/libxl.h                         |   22 +
 tools/include/xen-tools/common-macros.h       |   17 +
 tools/libs/light/libxl_9pfs.c                 |  193 ++-
 tools/libs/light/libxl_create.c               |    4 +-
 tools/libs/light/libxl_dm.c                   |    2 +-
 tools/libs/light/libxl_types.idl              |   11 +
 tools/libs/light/libxl_types_internal.idl     |    1 +
 tools/xen-9pfsd/.gitignore                    |    1 +
 tools/xen-9pfsd/Makefile                      |   38 +
 tools/xen-9pfsd/io.c                          | 1510 +++++++++++++++++
 tools/xen-9pfsd/xen-9pfsd.c                   |  800 +++++++++
 tools/xen-9pfsd/xen-9pfsd.h                   |   99 ++
 tools/xenstored/Makefile.common               |    4 -
 tools/xenstored/control.c                     |   29 +-
 tools/xenstored/core.c                        |  274 +--
 tools/xenstored/core.h                        |   36 +-
 tools/xenstored/domain.c                      |   81 +-
 tools/xenstored/domain.h                      |    2 +
 tools/xenstored/lu_daemon.c                   |    4 +-
 tools/xenstored/minios.c                      |  102 +-
 tools/xenstored/posix.c                       |  263 ++-
 tools/xl/xl_parse.c                           |   23 +-
 31 files changed, 3261 insertions(+), 341 deletions(-)
 create mode 100644 tools/xen-9pfsd/.gitignore
 create mode 100644 tools/xen-9pfsd/Makefile
 create mode 100644 tools/xen-9pfsd/io.c
 create mode 100644 tools/xen-9pfsd/xen-9pfsd.c
 create mode 100644 tools/xen-9pfsd/xen-9pfsd.h

-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:50:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:50:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675787.1051282 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYH-0002Tm-HW; Mon, 05 Feb 2024 10:50:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675787.1051282; Mon, 05 Feb 2024 10:50:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYH-0002Td-Em; Mon, 05 Feb 2024 10:50:13 +0000
Received: by outflank-mailman (input) for mailman id 675787;
 Mon, 05 Feb 2024 10:50:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwYF-0000qL-WE
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:50:12 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5512502b-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:50:10 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id F16B81F8BA;
 Mon,  5 Feb 2024 10:50:09 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BC91B132DD;
 Mon,  5 Feb 2024 10:50:09 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id hKaCLGG9wGUBNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:50:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5512502b-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v4 01/32] tools: add access macros for unaligned data
Date: Mon,  5 Feb 2024 11:49:30 +0100
Message-Id: <20240205105001.24171-2-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: F16B81F8BA
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add the basic access macros for unaligned data to common-macros.h.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
V3:
- new patch
---
 tools/include/xen-tools/common-macros.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/tools/include/xen-tools/common-macros.h b/tools/include/xen-tools/common-macros.h
index e5ed603904..81fba2e9f5 100644
--- a/tools/include/xen-tools/common-macros.h
+++ b/tools/include/xen-tools/common-macros.h
@@ -79,6 +79,10 @@
 #define __must_check __attribute__((__warn_unused_result__))
 #endif
 
+#ifndef __packed
+#define __packed __attribute__((__packed__))
+#endif
+
 #define container_of(ptr, type, member) ({              \
     typeof(((type *)0)->member) *mptr__ = (ptr);        \
     (type *)((char *)mptr__ - offsetof(type, member));  \
@@ -87,4 +91,17 @@
 #define __AC(X, Y)   (X ## Y)
 #define _AC(X, Y)    __AC(X, Y)
 
+#define get_unaligned_t(type, ptr) ({                               \
+    const struct { type x; } __packed *ptr_ = (typeof(ptr_))(ptr);  \
+    ptr_->x;                                                        \
+})
+
+#define put_unaligned_t(type, val, ptr) do {                        \
+    struct { type x; } __packed *ptr_ = (typeof(ptr_))(ptr);        \
+    ptr_->x = val;                                                  \
+} while (0)
+
+#define get_unaligned(ptr)      get_unaligned_t(typeof(*(ptr)), ptr)
+#define put_unaligned(val, ptr) put_unaligned_t(typeof(*(ptr)), val, ptr)
+
 #endif	/* __XEN_TOOLS_COMMON_MACROS__ */
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:50:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:50:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675788.1051292 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYN-0002oV-Qe; Mon, 05 Feb 2024 10:50:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675788.1051292; Mon, 05 Feb 2024 10:50:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYN-0002oL-NW; Mon, 05 Feb 2024 10:50:19 +0000
Received: by outflank-mailman (input) for mailman id 675788;
 Mon, 05 Feb 2024 10:50:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwYL-0000qL-OG
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:50:17 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 58701518-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:50:15 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 9B52921EBE;
 Mon,  5 Feb 2024 10:50:15 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5F828132DD;
 Mon,  5 Feb 2024 10:50:15 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id mW3SFWe9wGUKNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:50:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58701518-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v4 02/32] tools: add a new xen logging daemon
Date: Mon,  5 Feb 2024 11:49:31 +0100
Message-Id: <20240205105001.24171-3-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 9B52921EBE
X-Spam-Flag: NO

Add "xen-9pfsd", a new logging daemon meant to support infrastructure
domains (e.g. xenstore-stubdom) to access files in dom0.

For now only add the code needed for starting the daemon and
registering it with Xenstore via a new "libxl/xen-9pfs/state" node by
writing the "running" state to it.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- rename from xenlogd to xen-9pfsd (Andrew Cooper)
- use a backend domain local Xenstore node (Jason Andryuk)
- use "volatile" for stop_me (Andrew Cooper)
V3:
- fix SPDX Identifier (Andrew Cooper)
V4:
- add strerror() test to error logging (Andrew Cooper)
- don't handle "daemon already running" as error (Anthony Perard)
---
 tools/Makefile              |   1 +
 tools/xen-9pfsd/.gitignore  |   1 +
 tools/xen-9pfsd/Makefile    |  38 ++++++++++
 tools/xen-9pfsd/xen-9pfsd.c | 147 ++++++++++++++++++++++++++++++++++++
 4 files changed, 187 insertions(+)
 create mode 100644 tools/xen-9pfsd/.gitignore
 create mode 100644 tools/xen-9pfsd/Makefile
 create mode 100644 tools/xen-9pfsd/xen-9pfsd.c

diff --git a/tools/Makefile b/tools/Makefile
index 3a510663a0..f8faa3a902 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -32,6 +32,7 @@ SUBDIRS-y += xenpmd
 SUBDIRS-$(CONFIG_GOLANG) += golang
 SUBDIRS-y += xl
 SUBDIRS-y += helpers
+SUBDIRS-y += xen-9pfsd
 SUBDIRS-$(CONFIG_X86) += xenpaging
 SUBDIRS-$(CONFIG_X86) += debugger
 SUBDIRS-$(CONFIG_TESTS) += tests
diff --git a/tools/xen-9pfsd/.gitignore b/tools/xen-9pfsd/.gitignore
new file mode 100644
index 0000000000..d0c2d223ef
--- /dev/null
+++ b/tools/xen-9pfsd/.gitignore
@@ -0,0 +1 @@
+/xen-9pfsd
diff --git a/tools/xen-9pfsd/Makefile b/tools/xen-9pfsd/Makefile
new file mode 100644
index 0000000000..4e35202f08
--- /dev/null
+++ b/tools/xen-9pfsd/Makefile
@@ -0,0 +1,38 @@
+#
+# tools/xen-9pfsd/Makefile
+#
+
+XEN_ROOT = $(CURDIR)/../..
+include $(XEN_ROOT)/tools/Rules.mk
+
+CFLAGS += $(PTHREAD_CFLAGS)
+LDFLAGS += $(PTHREAD_LDFLAGS)
+
+TARGETS := xen-9pfsd
+
+XEN-9PFSD_OBJS = xen-9pfsd.o
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenstore)
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenevtchn)
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
+xen-9pfsd: LDLIBS += $(call xenlibs-ldlibs,store evtchn gnttab)
+
+.PHONY: all
+all: $(TARGETS)
+
+xen-9pfsd: $(XEN-9PFSD_OBJS)
+	$(CC) $(LDFLAGS) -o $@ $(XEN-9PFSD_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
+
+.PHONY: install
+install: all
+	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
+	for i in $(TARGETS); do $(INSTALL_PROG) $$i $(DESTDIR)$(LIBEXEC_BIN); done
+
+.PHONY: uninstall
+uninstall:
+	for i in $(TARGETS); do rm -f $(DESTDIR)$(LIBEXEC_BIN)/$$i; done
+
+.PHONY: clean
+clean:
+	$(RM) *.o $(TARGETS) $(DEPS_RM)
+
+distclean: clean
diff --git a/tools/xen-9pfsd/xen-9pfsd.c b/tools/xen-9pfsd/xen-9pfsd.c
new file mode 100644
index 0000000000..6939d01574
--- /dev/null
+++ b/tools/xen-9pfsd/xen-9pfsd.c
@@ -0,0 +1,147 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+/*
+ * xen-9pfsd - Xen 9pfs daemon
+ *
+ * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
+ *
+ * Daemon to enable guests to access a directory of the dom0 file system.
+ * Access is made via the 9pfs protocol (xen-9pfsd acts as a PV 9pfs backend).
+ *
+ * Usage: xen-9pfsd
+ *
+ * xen-9pfsd does NOT support writing any links (neither soft links nor hard
+ * links), and it is accepting only canonicalized file paths in order to
+ * avoid the possibility to "escape" from the guest specific directory.
+ *
+ * The backend device string is "xen_9pfs", the tag used for mounting the
+ * 9pfs device is "Xen".
+ *
+ * As an additional security measure the maximum file space used by the guest
+ * can be limited by the backend Xenstore node "max-size" specifying the size
+ * in MBytes. This size includes the size of the root directory of the guest.
+ */
+
+#include <err.h>
+#include <errno.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <syslog.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <xenevtchn.h>
+#include <xengnttab.h>
+#include <xenstore.h>
+
+static volatile bool stop_me;
+static bool daemon_running;
+static struct xs_handle *xs;
+static xengnttab_handle *xg;
+static xenevtchn_handle *xe;
+
+static void handle_stop(int sig)
+{
+    stop_me = true;
+}
+
+static void close_all(void)
+{
+    if ( daemon_running )
+        xs_rm(xs, XBT_NULL, "libxl/xen-9pfs");
+    if ( xe )
+        xenevtchn_close(xe);
+    if ( xg )
+        xengnttab_close(xg);
+    if ( xs )
+        xs_close(xs);
+    closelog();
+}
+
+static void do_err(const char *msg)
+{
+    syslog(LOG_ALERT, "%s, errno = %d, %s", msg, errno, strerror(errno));
+    close_all();
+    exit(1);
+}
+
+static void xen_connect(void)
+{
+    xs_transaction_t t;
+    char *val;
+    unsigned int len;
+
+    xs = xs_open(0);
+    if ( xs == NULL )
+        do_err("xs_open() failed");
+
+    xg = xengnttab_open(NULL, 0);
+    if ( xg == NULL )
+        do_err("xengnttab_open() failed");
+
+    xe = xenevtchn_open(NULL, 0);
+    if ( xe == NULL )
+        do_err("xenevtchn_open() failed");
+
+    while ( true )
+    {
+        t = xs_transaction_start(xs);
+        if ( t == XBT_NULL )
+            do_err("xs_transaction_start() failed");
+
+        val = xs_read(xs, t, "libxl/xen-9pfs/state", &len);
+        if ( val )
+        {
+            free(val);
+            xs_transaction_end(xs, t, true);
+            syslog(LOG_INFO, "daemon already running");
+            close_all();
+            exit(0);
+        }
+
+        if ( !xs_write(xs, t, "libxl/xen-9pfs/state", "running",
+                       strlen("running")) )
+        {
+            xs_transaction_end(xs, t, true);
+            do_err("xs_write() failed writing state");
+        }
+
+        if ( xs_transaction_end(xs, t, false) )
+            break;
+        if ( errno != EAGAIN )
+            do_err("xs_transaction_end() failed");
+    }
+
+    daemon_running = true;
+}
+
+int main(int argc, char *argv[])
+{
+    struct sigaction act = { .sa_handler = handle_stop, };
+    int syslog_mask = LOG_MASK(LOG_WARNING) | LOG_MASK(LOG_ERR) |
+                      LOG_MASK(LOG_CRIT) | LOG_MASK(LOG_ALERT) |
+                      LOG_MASK(LOG_EMERG);
+
+    umask(027);
+    if ( getenv("XEN_9PFSD_VERBOSE") )
+        syslog_mask |= LOG_MASK(LOG_NOTICE) | LOG_MASK(LOG_INFO);
+    openlog("xen-9pfsd", LOG_CONS, LOG_DAEMON);
+    setlogmask(syslog_mask);
+
+    sigemptyset(&act.sa_mask);
+    sigaction(SIGHUP, &act, NULL);
+
+    xen_connect();
+
+    while ( !stop_me )
+    {
+        sleep(60);
+    }
+
+    close_all();
+
+    return 0;
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:50:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:50:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675791.1051303 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYS-0003EZ-1Z; Mon, 05 Feb 2024 10:50:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675791.1051303; Mon, 05 Feb 2024 10:50:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYR-0003EQ-U9; Mon, 05 Feb 2024 10:50:23 +0000
Received: by outflank-mailman (input) for mailman id 675791;
 Mon, 05 Feb 2024 10:50:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwYQ-0002wU-Nh
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:50:22 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5bc82005-c414-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:50:21 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 417491F8BA;
 Mon,  5 Feb 2024 10:50:21 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0C3E3132DD;
 Mon,  5 Feb 2024 10:50:21 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id D65tAW29wGUONgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:50:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5bc82005-c414-11ee-8a45-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v4 03/32] tools/xen-9pfsd: connect to frontend
Date: Mon,  5 Feb 2024 11:49:32 +0100
Message-Id: <20240205105001.24171-4-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 417491F8BA
X-Spam-Flag: NO

Add the code for connecting to frontends to xenlogd.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- support multiple rings per device (Jason Andryuk)
- don't set .revents initially (Jason Andryuk)
- call poll() with infinite timeout (Jason Andryuk)
- take mutex before calling pthread_cond_signal()
V3:
- fix SPDX indentifier (Andrew Cooper)
- better validation of host path (Jason Andryuk)
- don't hard-code dom0 in backend nodes (Jason Andryuk)
- use bool instead of int for some functions' return types
- open root directory (Jason Andryuk)
---
 tools/xen-9pfsd/Makefile    |   2 +-
 tools/xen-9pfsd/io.c        |  45 +++
 tools/xen-9pfsd/xen-9pfsd.c | 653 +++++++++++++++++++++++++++++++++++-
 tools/xen-9pfsd/xen-9pfsd.h |  61 ++++
 4 files changed, 757 insertions(+), 4 deletions(-)
 create mode 100644 tools/xen-9pfsd/io.c
 create mode 100644 tools/xen-9pfsd/xen-9pfsd.h

diff --git a/tools/xen-9pfsd/Makefile b/tools/xen-9pfsd/Makefile
index 4e35202f08..2023b19c62 100644
--- a/tools/xen-9pfsd/Makefile
+++ b/tools/xen-9pfsd/Makefile
@@ -10,7 +10,7 @@ LDFLAGS += $(PTHREAD_LDFLAGS)
 
 TARGETS := xen-9pfsd
 
-XEN-9PFSD_OBJS = xen-9pfsd.o
+XEN-9PFSD_OBJS = xen-9pfsd.o io.o
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenstore)
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenevtchn)
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
new file mode 100644
index 0000000000..eb7c136e09
--- /dev/null
+++ b/tools/xen-9pfsd/io.c
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+/*
+ * xen-9pfsd - Xen 9pfs daemon
+ *
+ * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
+ *
+ * I/O thread handling.
+ */
+
+#include <stdbool.h>
+#include <string.h>
+#include <syslog.h>
+
+#include "xen-9pfsd.h"
+
+static bool io_work_pending(struct ring *ring)
+{
+    if ( ring->stop_thread )
+        return true;
+    return false;
+}
+
+void *io_thread(void *arg)
+{
+    struct ring *ring = arg;
+
+    while ( !ring->stop_thread )
+    {
+        pthread_mutex_lock(&ring->mutex);
+        if ( !io_work_pending(ring) )
+        {
+            if ( xenevtchn_unmask(xe, ring->evtchn) < 0 )
+                syslog(LOG_WARNING, "xenevtchn_unmask() failed");
+            pthread_cond_wait(&ring->cond, &ring->mutex);
+        }
+        pthread_mutex_unlock(&ring->mutex);
+
+        /* TODO: I/O handling. */
+    }
+
+    ring->thread_active = false;
+
+    return NULL;
+}
diff --git a/tools/xen-9pfsd/xen-9pfsd.c b/tools/xen-9pfsd/xen-9pfsd.c
index 6939d01574..73b6c3a30e 100644
--- a/tools/xen-9pfsd/xen-9pfsd.c
+++ b/tools/xen-9pfsd/xen-9pfsd.c
@@ -24,34 +24,632 @@
 
 #include <err.h>
 #include <errno.h>
+#include <fcntl.h>
+#include <poll.h>
+#include <pthread.h>
 #include <signal.h>
 #include <stdbool.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <sys/mman.h>
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <unistd.h>
-#include <xenevtchn.h>
 #include <xengnttab.h>
 #include <xenstore.h>
 
+#include "xen-9pfsd.h"
+
+/*
+ * List of currently known devices.
+ * The list itself is modified only in the main thread. When a device is being
+ * removed its memory needs to be freed after the I/O thread (if existing)
+ * has stopped.
+ */
+static XEN_TAILQ_HEAD(devhead, device) devs = XEN_TAILQ_HEAD_INITIALIZER(devs);
+
+struct path {
+    char path[100];
+};
+
 static volatile bool stop_me;
 static bool daemon_running;
 static struct xs_handle *xs;
 static xengnttab_handle *xg;
-static xenevtchn_handle *xe;
+static unsigned int now;
+
+xenevtchn_handle *xe;
 
 static void handle_stop(int sig)
 {
     stop_me = true;
 }
 
+static int check_host_path(device *device)
+{
+    struct stat statbuf;
+    char *path, *p;
+    int ret = 1;
+
+    if ( !device->host_path )
+        return 1;
+
+    /* Path must be absolute. */
+    if ( device->host_path[0] != '/' )
+        return 1;
+
+    /* No double "/". */
+    if ( strstr(device->host_path, "//") )
+        return 1;
+
+    /* No trailing "/" (includes refusing to share "/"). */
+    if ( device->host_path[strlen(device->host_path) - 1] == '/' )
+        return 1;
+
+    path = strdup(device->host_path);
+    if ( !path )
+    {
+        syslog(LOG_CRIT, "memory allocation failure!");
+        return 1;
+    }
+
+    for ( p = path; p; )
+    {
+        p = strchr(p + 1, '/');
+        if ( p )
+            *p = 0;
+        if ( !stat(path, &statbuf) )
+        {
+            if ( !(statbuf.st_mode & S_IFDIR) )
+                break;
+            if ( !p )
+            {
+                ret = 0;
+                break;
+            }
+            *p = '/';
+            continue;
+        }
+        if ( mkdir(path, 0777) )
+            break;
+        if ( p )
+            *p = '/';
+    }
+
+    free(path);
+    return ret;
+}
+
+static void construct_frontend_path(device *device, const char *node,
+                                    struct path *p)
+{
+    snprintf(p->path, sizeof(p->path), "/local/domain/%u/device/9pfs/%u/%s",
+             device->domid, device->devid, node);
+}
+
+static void construct_backend_path(device *device, const char *node,
+                                   struct path *p)
+{
+    snprintf(p->path, sizeof(p->path), "backend/xen_9pfs/%u/%u/%s",
+             device->domid, device->devid, node);
+}
+
+static char *read_backend_node(device *device, const char *node)
+{
+    struct path p;
+    char *val;
+    unsigned int len;
+
+    construct_backend_path(device, node, &p);
+    val = xs_read(xs, XBT_NULL, p.path, &len);
+
+    return val;
+}
+
+static unsigned int uint_from_string(char *string, unsigned int def)
+{
+    unsigned long val;
+    char *end;
+
+    if ( !string )
+        return def;
+
+    val = strtoul(string, &end, 10);
+    if ( *end || val > UINT_MAX )
+        val = def;
+    free(string);
+
+    return val;
+}
+
+static unsigned int read_backend_node_uint(device *device, const char *node,
+                                           unsigned int def)
+{
+    return uint_from_string(read_backend_node(device, node), def);
+}
+
+static unsigned int read_frontend_node_uint(device *device, const char *node,
+                                            unsigned int def)
+{
+    struct path p;
+    unsigned int len;
+
+    construct_frontend_path(device, node, &p);
+
+    return uint_from_string(xs_read(xs, XBT_NULL, p.path, &len), def);
+}
+
+static bool write_backend_node(device *device, const char *node,
+                               const char *val)
+{
+    struct path p;
+    unsigned int num_perms;
+    struct xs_permissions *old_perms;
+    struct xs_permissions perms[2] = {
+        { .id = 0, .perms = XS_PERM_NONE },
+        { .id = device->domid, .perms = XS_PERM_READ }
+    };
+
+    construct_backend_path(device, node, &p);
+    if ( !xs_write(xs, XBT_NULL, p.path, val, strlen(val)) )
+    {
+        syslog(LOG_ERR, "error writing bacḱend node \"%s\" for device %u/%u",
+               node, device->domid, device->devid);
+        return false;
+    }
+
+    old_perms = xs_get_permissions(xs, XBT_NULL, p.path, &num_perms);
+    if ( !old_perms )
+    {
+        syslog(LOG_ERR, "error getting permissions for \"%s\"", p.path);
+        return false;
+    }
+
+    perms[0] = old_perms[0];
+    free(old_perms);
+    if ( !xs_set_permissions(xs, XBT_NULL, p.path, perms, 2) )
+    {
+        syslog(LOG_ERR, "error setting permissions for \"%s\"", p.path);
+        return false;
+    }
+
+    return true;
+}
+
+static bool write_backend_node_uint(device *device, const char *node,
+                                   unsigned int val)
+{
+    char str[12];
+
+    snprintf(str, sizeof(str), "%u", val);
+
+    return write_backend_node(device, node, str);
+}
+
+static bool write_backend_state(device *device, enum xenbus_state state)
+{
+    struct path p;
+    char val[2];
+
+    snprintf(val, sizeof(val), "%u", state);
+    construct_backend_path(device, "state", &p);
+    if ( !xs_write(xs, XBT_NULL, p.path, val, 1) )
+    {
+        syslog(LOG_ERR, "error writing backend state %u for device %u/%u",
+               state, device->domid, device->devid);
+        return false;
+    }
+
+    device->backend_state = state;
+
+    return true;
+}
+
+static device *find_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+
+    XEN_TAILQ_FOREACH( device, &devs, list )
+    {
+        if ( domid == device->domid && devid == device->devid )
+            return device;
+    }
+
+    return NULL;
+}
+
+static void free_device(device *device)
+{
+    char token[20];
+    struct path p;
+
+    construct_frontend_path(device, "state", &p);
+    snprintf(token, sizeof(token), "%u/%u", device->domid, device->devid);
+    xs_unwatch(xs, p.path, token);
+
+    if ( device->root_fd >= 0 )
+        close(device->root_fd);
+
+    free(device->host_path);
+    free(device);
+}
+
+static device *new_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+    char token[20];
+    struct path p;
+    char *val;
+
+    device = calloc(1, sizeof(*device));
+    if ( !device )
+    {
+        syslog(LOG_CRIT, "Got no memory for new device %u/%u", domid, devid);
+        return NULL;
+    }
+
+    device->domid = domid;
+    device->devid = devid;
+    device->root_fd = -1;
+
+    construct_frontend_path(device, "state", &p);
+    snprintf(token, sizeof(token), "%u/%u", domid, devid);
+    if ( !xs_watch(xs, p.path, token) )
+    {
+        syslog(LOG_ERR, "Setting up watch for device %u/%u failed",
+               domid, devid);
+        free(device);
+        return NULL;
+    }
+
+    val = read_backend_node(device, "security_model");
+    if ( !val || strcmp(val, "none") )
+    {
+        syslog(LOG_ERR, "Security model \"%s\" for device %u/%u invalid.",
+               val, domid, devid);
+        free(val);
+        goto err;
+    }
+    free(val);
+
+    device->max_space = read_backend_node_uint(device, "max-space", 0);
+    device->max_files = read_backend_node_uint(device, "max-files", 0);
+    device->max_open_files =
+        read_backend_node_uint(device, "max-open-files", 0)
+        ?: MAX_OPEN_FILES_DEFAULT;
+    device->auto_delete = read_backend_node_uint(device, "auto-delete", 0);
+
+    device->host_path = read_backend_node(device, "path");
+    if ( check_host_path(device) )
+    {
+        syslog(LOG_ERR, "Host path \"%s\" for device %u/%u invalid.",
+               device->host_path, domid, devid);
+        goto err;
+    }
+    device->root_fd = open(device->host_path, O_RDONLY | O_DIRECTORY);
+    if ( device->root_fd < 0 )
+        goto err;
+
+    if ( !write_backend_node(device, "versions", "1") )
+        goto err;
+    if ( !write_backend_node_uint(device, "max-rings", MAX_RINGS) )
+        goto err;
+    if ( !write_backend_node_uint(device, "max-ring-page-order",
+                                 MAX_RING_ORDER) )
+        goto err;
+
+    if ( !write_backend_state(device, XenbusStateInitWait) )
+        goto err;
+
+    XEN_TAILQ_INSERT_TAIL(&devs, device, list);
+    syslog(LOG_INFO, "New device %u/%u added", domid, devid);
+
+    return device;
+
+ err:
+    free_device(device);
+    return NULL;
+}
+
+static void disconnect_ring(struct ring *ring)
+{
+    if ( !ring )
+        return;
+
+    if ( ring->thread_active )
+    {
+        ring->stop_thread = true;
+        pthread_cond_signal(&ring->cond);
+        pthread_join(ring->thread, NULL);
+        ring->stop_thread = false;
+    }
+
+    if ( ring->data.in )
+    {
+        xengnttab_unmap(xg, ring->data.in, 1 << ring->ring_order);
+        ring->data.in = NULL;
+    }
+    if ( ring->intf )
+    {
+        xengnttab_unmap(xg, ring->intf, 1 );
+        ring->intf = NULL;
+    }
+
+    if ( ring->evtchn )
+    {
+        xenevtchn_unbind(xe, ring->evtchn);
+        ring->evtchn = 0;
+    }
+
+    pthread_mutex_destroy(&ring->mutex);
+    pthread_cond_destroy(&ring->cond);
+}
+
+static void disconnect_guest(device *device)
+{
+    unsigned int ring_idx;
+
+    for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+    {
+        disconnect_ring(device->ring[ring_idx]);
+        free(device->ring[ring_idx]);
+        device->ring[ring_idx] = NULL;
+    }
+
+    device->num_rings = 0;
+}
+
+static void close_device(device *device, enum xenbus_state state)
+{
+    disconnect_guest(device);
+    write_backend_state(device, state);
+}
+
+static void connect_err(device *device, const char *msg)
+{
+    syslog(LOG_WARNING, "%s", msg);
+    close_device(device, XenbusStateClosed);
+}
+
+static void connect_device(device *device)
+{
+    unsigned int val;
+    unsigned int ring_idx;
+    char node[20];
+    struct ring *ring;
+    xenevtchn_port_or_error_t evtchn;
+
+    val = read_frontend_node_uint(device, "version", 0);
+    if ( val != 1 )
+        return connect_err(device, "frontend specifies illegal version");
+    device->num_rings = read_frontend_node_uint(device, "num-rings", 0);
+    if ( device->num_rings < 1 || device->num_rings > MAX_RINGS )
+        return connect_err(device, "frontend specifies illegal ring number");
+
+    for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+    {
+        ring = calloc(1, sizeof(*ring));
+        if ( !ring )
+            return connect_err(device, "could not allocate ring memory");
+        device->ring[ring_idx] = ring;
+        ring->device = device;
+        pthread_cond_init(&ring->cond, NULL);
+        pthread_mutex_init(&ring->mutex, NULL);
+
+        snprintf(node, sizeof(node), "event-channel-%u", ring_idx);
+        val = read_frontend_node_uint(device, node, 0);
+        if ( val == 0 )
+            return connect_err(device, "frontend specifies illegal evtchn");
+        evtchn = xenevtchn_bind_interdomain(xe, device->domid, val);
+        if ( evtchn < 0 )
+            return connect_err(device, "could not bind to event channel");
+        ring->evtchn = evtchn;
+
+        snprintf(node, sizeof(node), "ring-ref%u", ring_idx);
+        val = read_frontend_node_uint(device, node, 0);
+        if ( val == 0 )
+            return connect_err(device,
+                               "frontend specifies illegal grant for ring");
+        ring->intf = xengnttab_map_grant_ref(xg, device->domid, val,
+                                             PROT_READ | PROT_WRITE);
+        if ( !ring->intf )
+            return connect_err(device, "could not map interface page");
+        ring->ring_order = ring->intf->ring_order;
+        if ( ring->ring_order > MAX_RING_ORDER || ring->ring_order < 1 )
+            return connect_err(device, "frontend specifies illegal ring order");
+        ring->ring_size = XEN_FLEX_RING_SIZE(ring->ring_order);
+        ring->data.in = xengnttab_map_domain_grant_refs(xg,
+                                                        1 << ring->ring_order,
+                                                        device->domid,
+                                                        ring->intf->ref,
+                                                        PROT_READ | PROT_WRITE);
+        if ( !ring->data.in )
+            return connect_err(device, "could not map ring pages");
+        ring->data.out = ring->data.in + ring->ring_size;
+
+        if ( pthread_create(&ring->thread, NULL, io_thread, ring) )
+            return connect_err(device, "could not start I/O thread");
+        ring->thread_active = true;
+    }
+
+    write_backend_state(device, XenbusStateConnected);
+}
+
+static void remove_device(device *device)
+{
+    XEN_TAILQ_REMOVE(&devs, device, list);
+
+    disconnect_guest(device);
+    free_device(device);
+}
+
+static void remove_all_devices(void)
+{
+    device *device, *tmp;
+
+    XEN_TAILQ_FOREACH_SAFE( device, &devs, list, tmp )
+        remove_device(device);
+}
+
+static void frontend_changed(device *device)
+{
+    struct path p;
+    char *state, *end;
+    unsigned int len;
+    unsigned long new_state;
+
+    construct_frontend_path(device, "state", &p);
+    state = xs_read(xs, XBT_NULL, p.path, &len);
+    if ( !state )
+    {
+        close_device(device, XenbusStateClosed);
+        return;
+    }
+
+    new_state = strtoul(state, &end, 10);
+    if ( *end || new_state > XenbusStateReconfigured )
+    {
+        syslog(LOG_WARNING, "unknown state \"%s\" seen for device %u/%u", state,
+               device->domid, device->devid);
+        new_state = XenbusStateUnknown;
+    }
+    free(state);
+
+    if ( new_state == device->frontend_state )
+        return;
+
+    switch ( new_state )
+    {
+    case XenbusStateInitialising:
+        break;
+
+    case XenbusStateInitialised:
+        connect_device(device);
+        break;
+
+    case XenbusStateConnected:
+        break;
+
+    case XenbusStateClosing:
+        close_device(device, XenbusStateClosing);
+        break;
+
+    case XenbusStateClosed:
+        close_device(device, XenbusStateClosed);
+        break;
+
+    default:
+        syslog(LOG_WARNING, "not handled frontend state %lu for device %u/%u",
+               new_state, device->domid, device->devid);
+        break;
+    }
+
+    device->frontend_state = new_state;
+}
+
+static void check_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+
+    device = find_device(domid, devid);
+    if ( !device )
+    {
+        device = new_device(domid, devid);
+        if ( !device )
+            return;
+    }
+
+    device->last_seen = now;
+}
+
+static void scan_backend(void)
+{
+    char **doms;
+    unsigned int n_doms, dom;
+    char **devices;
+    unsigned int n_devs, dev;
+    char dom_path[24];
+    unsigned long domid, devid;
+    char *end;
+    device *device, *tmp;
+
+    now++;
+
+    doms = xs_directory(xs, XBT_NULL, "backend/xen_9pfs", &n_doms);
+    if ( doms == NULL )
+        return;
+
+    for ( dom = 0; dom < n_doms; dom++ )
+    {
+        errno = 0;
+        domid = strtoul(doms[dom], &end, 10);
+        if ( errno || *end || domid >= DOMID_FIRST_RESERVED )
+            continue;
+
+        snprintf(dom_path, sizeof(dom_path), "backend/xen_9pfs/%lu", domid);
+        devices = xs_directory(xs, XBT_NULL, dom_path, &n_devs);
+
+        for ( dev = 0; dev < n_devs; dev++ )
+        {
+            errno = 0;
+            devid = strtoul(devices[dev], &end, 10);
+            if ( errno || *end || devid > UINT_MAX )
+                continue;
+
+            check_device(domid, devid);
+        }
+
+        free(devices);
+    }
+
+    free(doms);
+
+    XEN_TAILQ_FOREACH_SAFE( device, &devs, list, tmp )
+    {
+        if ( device->last_seen != now )
+            remove_device(device);
+    }
+}
+
+static void handle_watch(char *path, char *token)
+{
+    unsigned int domid, devid;
+    device *device;
+
+    if ( !strcmp(token, "main") )
+    {
+        scan_backend();
+        return;
+    }
+
+    if ( sscanf(token, "%u/%u", &domid, &devid) != 2 )
+    {
+        syslog(LOG_WARNING, "unknown watch event %s %s", path, token);
+        return;
+    }
+
+    device = find_device(domid, devid);
+    if ( !device )
+    {
+        syslog(LOG_WARNING, "watch event for unknown device %u/%u",
+               domid, devid);
+        return;
+    }
+
+    frontend_changed(device);
+}
+
 static void close_all(void)
 {
     if ( daemon_running )
+    {
         xs_rm(xs, XBT_NULL, "libxl/xen-9pfs");
+        xs_unwatch(xs, "backend/xen_9pfs", "main");
+
+        remove_all_devices();
+    }
     if ( xe )
         xenevtchn_close(xe);
     if ( xg )
@@ -68,6 +666,33 @@ static void do_err(const char *msg)
     exit(1);
 }
 
+static void handle_event(void)
+{
+    xenevtchn_port_or_error_t evtchn;
+    device *device;
+    struct ring *ring;
+    unsigned int ring_idx;
+
+    evtchn = xenevtchn_pending(xe);
+    if ( evtchn < 0 )
+        do_err("xenevtchn_pending() failed");
+
+    XEN_TAILQ_FOREACH( device, &devs, list )
+    {
+        for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+        {
+            ring = device->ring[ring_idx];
+            if ( ring && ring->evtchn == evtchn )
+            {
+                pthread_mutex_lock(&ring->mutex);
+                pthread_cond_signal(&ring->cond);
+                pthread_mutex_unlock(&ring->mutex);
+                return;
+            }
+        }
+    }
+}
+
 static void xen_connect(void)
 {
     xs_transaction_t t;
@@ -124,6 +749,11 @@ int main(int argc, char *argv[])
     int syslog_mask = LOG_MASK(LOG_WARNING) | LOG_MASK(LOG_ERR) |
                       LOG_MASK(LOG_CRIT) | LOG_MASK(LOG_ALERT) |
                       LOG_MASK(LOG_EMERG);
+    char **watch;
+    struct pollfd p[2] = {
+        { .events = POLLIN },
+        { .events = POLLIN }
+    };
 
     umask(027);
     if ( getenv("XEN_9PFSD_VERBOSE") )
@@ -136,9 +766,26 @@ int main(int argc, char *argv[])
 
     xen_connect();
 
+    if ( !xs_watch(xs, "backend/xen_9pfs", "main") )
+        do_err("xs_watch() in main thread failed");
+    p[0].fd = xs_fileno(xs);
+    p[1].fd = xenevtchn_fd(xe);
+
+    scan_backend();
+
     while ( !stop_me )
     {
-        sleep(60);
+        while ( (p[0].revents & POLLIN) &&
+                (watch = xs_check_watch(xs)) != NULL )
+        {
+            handle_watch(watch[XS_WATCH_PATH], watch[XS_WATCH_TOKEN]);
+            free(watch);
+        }
+
+        if ( p[1].revents & POLLIN )
+            handle_event();
+
+        poll(p, 2, -1);
     }
 
     close_all();
diff --git a/tools/xen-9pfsd/xen-9pfsd.h b/tools/xen-9pfsd/xen-9pfsd.h
new file mode 100644
index 0000000000..ff3562164a
--- /dev/null
+++ b/tools/xen-9pfsd/xen-9pfsd.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef XEN_9PFSD_H
+#define XEN_9PFSD_H
+
+#include <pthread.h>
+#include <stdbool.h>
+#include <xenevtchn.h>
+#include <xen_list.h>
+#include <xen/xen.h>
+#include <xen/io/xenbus.h>
+#include <xen/io/9pfs.h>
+
+#define MAX_RINGS                4
+#define MAX_RING_ORDER           9
+#define MAX_OPEN_FILES_DEFAULT   5
+
+typedef struct device device;
+
+struct ring {
+    device *device;
+    pthread_t thread;
+    bool thread_active;
+    bool stop_thread;
+    pthread_cond_t cond;
+    pthread_mutex_t mutex;
+
+    evtchn_port_t evtchn;
+    struct xen_9pfs_data_intf *intf;
+    unsigned int ring_order;
+    RING_IDX ring_size;
+    struct xen_9pfs_data data;
+};
+
+struct device {
+    /* Admin data. */
+    XEN_TAILQ_ENTRY(device) list;
+    unsigned int last_seen;    /* Set in scan_backend(). */
+    unsigned int domid;
+    unsigned int devid;
+
+    /* Tool side configuration data. */
+    char *host_path;
+    unsigned int max_space;
+    unsigned int max_files;
+    unsigned int max_open_files;
+    bool auto_delete;
+
+    /* Connection data. */
+    enum xenbus_state backend_state;
+    enum xenbus_state frontend_state;
+    unsigned int num_rings;
+    struct ring *ring[MAX_RINGS];
+    int root_fd;
+};
+
+extern xenevtchn_handle *xe;
+
+void *io_thread(void *arg);
+
+#endif /* XEN_9PFSD_H */
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:50:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:50:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675793.1051313 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYY-0003kf-DZ; Mon, 05 Feb 2024 10:50:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675793.1051313; Mon, 05 Feb 2024 10:50:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYY-0003kY-Al; Mon, 05 Feb 2024 10:50:30 +0000
Received: by outflank-mailman (input) for mailman id 675793;
 Mon, 05 Feb 2024 10:50:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwYX-0000qL-5b
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:50:29 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5f40cfe4-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:50:27 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id D582221EBE;
 Mon,  5 Feb 2024 10:50:26 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A35E8132DD;
 Mon,  5 Feb 2024 10:50:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 94lrJnK9wGUUNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:50:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f40cfe4-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130227; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GzcbwuhHDQyavc9oDP7uqfqZzwZ8UtCeX4PMuuKBycU=;
	b=PTh7ztC19I2BTF/e4pIDbls+TG1KMv8JFYQocDhFAbVH7ujVFUv5gUub7GcE8f0XhtNY2r
	gbb83rrcaRQJMFPwsTJ1Yo33IhwxRYQQWheOk7fGp8IlnjcNtvE8WE8OpEI9OFQSsJDVwT
	m87+2jbN9oJ5iSBv/v8ZEVa/cjdq/AY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130226; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GzcbwuhHDQyavc9oDP7uqfqZzwZ8UtCeX4PMuuKBycU=;
	b=TkdsZHGXVL5zRimtIaMPf6U0mYDbJRQ9r9my5q+Zu2LkWS4/1DotO3qgnzL0xw9C/ZrGlI
	hXhRSzICXk30ofsNnPDFRcpmQ/R/9CbUiuUF//osXNQ0d48uwt2c4yCmuGXHpA025phJLA
	uodpC7Y0flqA9QriJ9/0T5N4Q940Xcc=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v4 04/32] tools/xen-9pfsd: add transport layer
Date: Mon,  5 Feb 2024 11:49:33 +0100
Message-Id: <20240205105001.24171-5-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add the transport layer of 9pfs. This is basically the infrastructure
to receive requests from the frontend and to send the related answers
via the rings.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- rename put_request_bytes() (Jason Andryuk)
- rename get_request_bytes() and put_response_bytes() len parameter
  (Jason Andryuk)
- don't unmask event channel if error indicator is set (Jason Andryuk)
---
 tools/xen-9pfsd/io.c        | 143 +++++++++++++++++++++++++++++++++++-
 tools/xen-9pfsd/xen-9pfsd.h |  16 ++++
 2 files changed, 156 insertions(+), 3 deletions(-)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index eb7c136e09..4312a62dfe 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -6,39 +6,176 @@
  * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
  *
  * I/O thread handling.
+ *
+ * Only handle one request at a time, pushing out the complete response
+ * before looking for the next request.
  */
 
 #include <stdbool.h>
+#include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <xenctrl.h>           /* For cpu barriers. */
+#include <xen-tools/common-macros.h>
 
 #include "xen-9pfsd.h"
 
+/*
+ * Note that the ring names "in" and "out" are from the frontend's
+ * perspective, so the "in" ring will be used for responses to the frontend,
+ * while the "out" ring is used for requests from the frontend to the
+ * backend.
+ */
+static unsigned int ring_in_free(struct ring *ring)
+{
+    unsigned int queued;
+
+    queued = xen_9pfs_queued(ring->prod_pvt_in, ring->intf->in_cons,
+                             ring->ring_size);
+    xen_rmb();
+
+    return ring->ring_size - queued;
+}
+
+static unsigned int ring_out_data(struct ring *ring)
+{
+    unsigned int queued;
+
+    queued = xen_9pfs_queued(ring->intf->out_prod, ring->cons_pvt_out,
+                             ring->ring_size);
+    xen_rmb();
+
+    return queued;
+}
+
+static unsigned int get_request_bytes(struct ring *ring, unsigned int off,
+                                      unsigned int total_len)
+{
+    unsigned int size;
+    unsigned int out_data = ring_out_data(ring);
+    RING_IDX prod, cons;
+
+    size = min(total_len - off, out_data);
+    prod = xen_9pfs_mask(ring->intf->out_prod, ring->ring_size);
+    cons = xen_9pfs_mask(ring->cons_pvt_out, ring->ring_size);
+    xen_9pfs_read_packet(ring->buffer + off, ring->data.out, size,
+                         prod, &cons, ring->ring_size);
+
+    xen_rmb();           /* Read data out before setting visible consumer. */
+    ring->cons_pvt_out += size;
+    ring->intf->out_cons = ring->cons_pvt_out;
+
+    /* Signal that more space is available now. */
+    xenevtchn_notify(xe, ring->evtchn);
+
+    return size;
+}
+
+static unsigned int put_response_bytes(struct ring *ring, unsigned int off,
+                                       unsigned int total_len)
+{
+    unsigned int size;
+    unsigned int in_data = ring_in_free(ring);
+    RING_IDX prod, cons;
+
+    size = min(total_len - off, in_data);
+    prod = xen_9pfs_mask(ring->prod_pvt_in, ring->ring_size);
+    cons = xen_9pfs_mask(ring->intf->in_cons, ring->ring_size);
+    xen_9pfs_write_packet(ring->data.in, ring->buffer + off, size,
+                          &prod, cons, ring->ring_size);
+
+    xen_wmb();           /* Write data out before setting visible producer. */
+    ring->prod_pvt_in += size;
+    ring->intf->in_prod = ring->prod_pvt_in;
+
+    return size;
+}
+
 static bool io_work_pending(struct ring *ring)
 {
     if ( ring->stop_thread )
         return true;
-    return false;
+    if ( ring->error )
+        return false;
+    return ring->handle_response ? ring_in_free(ring) : ring_out_data(ring);
 }
 
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
+    unsigned int count = 0;
+    struct p9_header hdr;
+    bool in_hdr = true;
+
+    ring->max_size = ring->ring_size;
+    ring->buffer = malloc(ring->max_size);
+    if ( !ring->buffer )
+    {
+        syslog(LOG_CRIT, "memory allocation failure!");
+        return NULL;
+    }
 
     while ( !ring->stop_thread )
     {
         pthread_mutex_lock(&ring->mutex);
         if ( !io_work_pending(ring) )
         {
-            if ( xenevtchn_unmask(xe, ring->evtchn) < 0 )
+            if ( !ring->error && xenevtchn_unmask(xe, ring->evtchn) < 0 )
                 syslog(LOG_WARNING, "xenevtchn_unmask() failed");
             pthread_cond_wait(&ring->cond, &ring->mutex);
         }
         pthread_mutex_unlock(&ring->mutex);
 
-        /* TODO: I/O handling. */
+        if ( ring->stop_thread || ring->error )
+            continue;
+
+        if ( !ring->handle_response )
+        {
+            if ( in_hdr )
+            {
+                count += get_request_bytes(ring, count, sizeof(hdr));
+                if ( count != sizeof(hdr) )
+                    continue;
+                hdr = *(struct p9_header *)ring->buffer;
+                if ( hdr.size > ring->max_size || hdr.size < sizeof(hdr) )
+                {
+                    syslog(LOG_ERR, "%u.%u specified illegal request length %u",
+                           ring->device->domid, ring->device->devid, hdr.size);
+                    ring->error = true;
+                    continue;
+                }
+                in_hdr = false;
+            }
+
+            count += get_request_bytes(ring, count, hdr.size);
+            if ( count < hdr.size )
+                continue;
+
+            /* TODO: handle request (will rewrite hdr.size). */
+
+            ring->handle_response = true;
+            hdr.size = ((struct p9_header *)ring->buffer)->size;
+            count = 0;
+        }
+
+        if ( ring->handle_response )
+        {
+            count += put_response_bytes(ring, count, hdr.size);
+
+            if ( count == hdr.size )
+            {
+                /* Signal presence of response. */
+                xenevtchn_notify(xe, ring->evtchn);
+
+                ring->handle_response = false;
+                in_hdr = true;
+                count = 0;
+            }
+        }
     }
 
+    free(ring->buffer);
+
     ring->thread_active = false;
 
     return NULL;
diff --git a/tools/xen-9pfsd/xen-9pfsd.h b/tools/xen-9pfsd/xen-9pfsd.h
index ff3562164a..d587f59a32 100644
--- a/tools/xen-9pfsd/xen-9pfsd.h
+++ b/tools/xen-9pfsd/xen-9pfsd.h
@@ -15,6 +15,12 @@
 #define MAX_RING_ORDER           9
 #define MAX_OPEN_FILES_DEFAULT   5
 
+struct p9_header {
+    uint32_t size;
+    uint8_t cmd;
+    uint16_t tag;
+} __attribute__((packed));
+
 typedef struct device device;
 
 struct ring {
@@ -29,7 +35,17 @@ struct ring {
     struct xen_9pfs_data_intf *intf;
     unsigned int ring_order;
     RING_IDX ring_size;
+
+    /* Transport layer data. */
     struct xen_9pfs_data data;
+    RING_IDX prod_pvt_in;
+    RING_IDX cons_pvt_out;
+
+    /* Request and response handling. */
+    uint32_t max_size;
+    bool error;             /* Protocol error - stop processing. */
+    bool handle_response;   /* Main loop now handling response. */
+    void *buffer;           /* Request/response buffer. */
 };
 
 struct device {
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:50:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:50:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675796.1051323 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYd-0004EO-Ly; Mon, 05 Feb 2024 10:50:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675796.1051323; Mon, 05 Feb 2024 10:50:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYd-0004ED-IZ; Mon, 05 Feb 2024 10:50:35 +0000
Received: by outflank-mailman (input) for mailman id 675796;
 Mon, 05 Feb 2024 10:50:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwYc-0000qL-KK
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:50:34 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6275bc70-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:50:32 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 6FDE822100;
 Mon,  5 Feb 2024 10:50:32 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 453CD132DD;
 Mon,  5 Feb 2024 10:50:32 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id PVFnD3i9wGUZNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:50:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6275bc70-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4 05/32] tools/xen-9pfsd: add 9pfs response generation support
Date: Mon,  5 Feb 2024 11:49:34 +0100
Message-Id: <20240205105001.24171-6-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 6FDE822100
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add support for generation a 9pfs protocol response via a format based
approach.

Strings are stored in a per device string buffer and they are
referenced via their offset in this buffer. This allows to avoid
having to dynamically allocate memory for each single string.

As a first user of the response handling add a generic p9_error()
function which will be used to return any error to the client.

Add all format parsing variants in order to avoid additional code churn
later when adding the users of those variants. Prepare a special case
for the "read" case already (format character 'D'): in order to avoid
adding another buffer for read data support doing the read I/O directly
into the response buffer.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- check parameter size limits (Jason Andryuk)
V3:
- use new unaligned access macros (Jason Andryuk)
V4:
- use recursion in fill_buffer() as a preparation for reading dirs
---
 tools/xen-9pfsd/io.c        | 217 +++++++++++++++++++++++++++++++++++-
 tools/xen-9pfsd/xen-9pfsd.h |   3 +
 2 files changed, 219 insertions(+), 1 deletion(-)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index 4312a62dfe..4a44c70c4d 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -11,6 +11,7 @@
  * before looking for the next request.
  */
 
+#include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
@@ -20,6 +21,16 @@
 
 #include "xen-9pfsd.h"
 
+/* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
+#define P9_CMD_ERROR      107
+
+struct p9_qid {
+    uint8_t type;
+#define QID_TYPE_DIR      0x80
+    uint32_t version;
+    uint64_t path;
+};
+
 /*
  * Note that the ring names "in" and "out" are from the frontend's
  * perspective, so the "in" ring will be used for responses to the frontend,
@@ -100,6 +111,200 @@ static bool io_work_pending(struct ring *ring)
     return ring->handle_response ? ring_in_free(ring) : ring_out_data(ring);
 }
 
+static void fmt_err(const char *fmt)
+{
+    syslog(LOG_CRIT, "illegal format %s passed to fill_buffer()", fmt);
+    exit(1);
+}
+
+/*
+ * Fill buffer with response data.
+ * fmt is a sequence of format characters. Supported characters are:
+ * a: an array (2 bytes number of elements + the following format as elements)
+ *    The number of elements is passed in the first unsigned int parameter, the
+ *    next parameter is a pointer to an array of elements as denoted by the next
+ *    format character.
+ * b: 1 byte unsigned integer
+ * u: 2 byte unsigned integer
+ *    The parameter is a pointer to a uint16_t value
+ * D: Data blob (4 byte length + <length> bytes)
+ *    2 parameters are consumed, first an unsigned int for the length, then a
+ *    pointer to the first uint8_t value.
+ *    No array support.
+ * L: 8 byte unsigned integer
+ *    The parameter is a pointer to a uint64_t value
+ * Q: Qid (struct p9_qid)
+ * S: String (2 byte length + <length> characters)
+ *    The length is obtained via strlen() of the parameter, being a pointer
+ *    to the first character of the string
+ * U: 4 byte unsigned integer
+ *    The parameter is a pointer to a uint32_t value
+ */
+static void fill_buffer_at(void **data, const char *fmt, ...);
+static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
+{
+    const char *f;
+    const void *par;
+    const char *str_val;
+    const struct p9_qid *qid;
+    unsigned int len;
+    unsigned int array_sz = 0;
+    unsigned int elem_sz = 0;
+
+    for ( f = fmt; *f; f++ )
+    {
+        if ( !array_sz )
+            par = va_arg(ap, const void *);
+        else
+        {
+            par += elem_sz;
+            array_sz--;
+        }
+
+        switch ( *f )
+        {
+        case 'a':
+            f++;
+            if ( !*f || array_sz )
+                fmt_err(fmt);
+            array_sz = *(const unsigned int *)par;
+            if ( array_sz > 0xffff )
+            {
+                syslog(LOG_CRIT, "array size %u in fill_buffer()", array_sz);
+                exit(1);
+            }
+            put_unaligned(array_sz, (uint16_t *)*data);
+            *data += sizeof(uint16_t);
+            par = va_arg(ap, const void *);
+            elem_sz = 0;
+            break;
+
+        case 'b':
+            put_unaligned(*(const uint8_t *)par, (uint8_t *)*data);
+            elem_sz = sizeof(uint8_t);
+            *data += sizeof(uint8_t);
+            break;
+
+        case 'u':
+            put_unaligned(*(const uint16_t *)par, (uint16_t *)*data);
+            elem_sz = sizeof(uint16_t);
+            *data += sizeof(uint16_t);
+            break;
+
+        case 'D':
+            if ( array_sz )
+                fmt_err(fmt);
+            len = *(const unsigned int *)par;
+            put_unaligned(len, (uint32_t *)*data);
+            *data += sizeof(uint32_t);
+            par = va_arg(ap, const void *);
+            if ( *data != par )
+                memcpy(*data, par, len);
+            *data += len;
+            break;
+
+        case 'L':
+            put_unaligned(*(const uint64_t *)par, (uint64_t *)*data);
+            elem_sz = sizeof(uint64_t);
+            *data += sizeof(uint64_t);
+            break;
+
+        case 'Q':
+            qid = par;
+            elem_sz = sizeof(*qid);
+            fill_buffer_at(data, "bUL", &qid->type, &qid->version, &qid->path);
+            break;
+
+        case 'S':
+            str_val = par;
+            elem_sz = sizeof(str_val);
+            len = strlen(str_val);
+            if ( len > 0xffff )
+            {
+                syslog(LOG_CRIT, "string length %u in fill_buffer()", len);
+                exit(1);
+            }
+            put_unaligned(len, (uint16_t *)*data);
+            *data += sizeof(uint16_t);
+            memcpy(*data, str_val, len);
+            *data += len;
+            break;
+
+        case 'U':
+            put_unaligned(*(const uint32_t *)par, (uint32_t *)*data);
+            elem_sz = sizeof(uint32_t);
+            *data += sizeof(uint32_t);
+            break;
+
+        default:
+            fmt_err(fmt);
+        }
+
+        if ( array_sz )
+            f--;
+    }
+}
+
+static void fill_buffer_at(void **data, const char *fmt, ...)
+{
+    va_list ap;
+
+    va_start(ap, fmt);
+    vfill_buffer_at(data, fmt, ap);
+    va_end(ap);
+}
+
+static void fill_buffer(struct ring *ring, uint8_t cmd, uint16_t tag,
+                        const char *fmt, ...)
+{
+    struct p9_header *hdr = ring->buffer;
+    void *data = hdr + 1;
+    va_list ap;
+
+    hdr->cmd = cmd;
+    hdr->tag = tag;
+
+    va_start(ap, fmt);
+    vfill_buffer_at(&data, fmt, ap);
+    va_end(ap);
+
+    hdr->size = data - ring->buffer;
+}
+
+static unsigned int add_string(struct ring *ring, const char *str,
+                               unsigned int len)
+{
+    char *tmp;
+    unsigned int ret;
+
+    if ( ring->str_used + len + 1 > ring->str_size )
+    {
+        tmp = realloc(ring->str, ring->str_used + len + 1);
+        if ( !tmp )
+            return ~0;
+        ring->str = tmp;
+        ring->str_size = ring->str_used + len + 1;
+    }
+
+    ret = ring->str_used;
+    memcpy(ring->str + ret, str, len);
+    ring->str_used += len;
+    ring->str[ring->str_used++] = 0;
+
+    return ret;
+}
+
+static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
+{
+    unsigned int erroff;
+
+    strerror_r(err, ring->buffer, ring->ring_size);
+    erroff = add_string(ring, ring->buffer, strlen(ring->buffer));
+    fill_buffer(ring, P9_CMD_ERROR, tag, "SU",
+                erroff != ~0 ? ring->str + erroff : "cannot allocate memory",
+                &err);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -151,7 +356,16 @@ void *io_thread(void *arg)
             if ( count < hdr.size )
                 continue;
 
-            /* TODO: handle request (will rewrite hdr.size). */
+            ring->str_used = 0;
+
+            switch ( hdr.cmd )
+            {
+            default:
+                syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
+                       ring->device->domid, ring->device->devid, hdr.cmd);
+                p9_error(ring, hdr.tag, EOPNOTSUPP);
+                break;
+            }
 
             ring->handle_response = true;
             hdr.size = ((struct p9_header *)ring->buffer)->size;
@@ -174,6 +388,7 @@ void *io_thread(void *arg)
         }
     }
 
+    free(ring->str);
     free(ring->buffer);
 
     ring->thread_active = false;
diff --git a/tools/xen-9pfsd/xen-9pfsd.h b/tools/xen-9pfsd/xen-9pfsd.h
index d587f59a32..0cde0d2bb8 100644
--- a/tools/xen-9pfsd/xen-9pfsd.h
+++ b/tools/xen-9pfsd/xen-9pfsd.h
@@ -46,6 +46,9 @@ struct ring {
     bool error;             /* Protocol error - stop processing. */
     bool handle_response;   /* Main loop now handling response. */
     void *buffer;           /* Request/response buffer. */
+    char *str;              /* String work space. */
+    unsigned int str_size;  /* Size of *str. */
+    unsigned int str_used;  /* Currently used size of *str. */
 };
 
 struct device {
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:50:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:50:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675799.1051333 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYj-0004nO-Ss; Mon, 05 Feb 2024 10:50:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675799.1051333; Mon, 05 Feb 2024 10:50:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwYj-0004nC-Px; Mon, 05 Feb 2024 10:50:41 +0000
Received: by outflank-mailman (input) for mailman id 675799;
 Mon, 05 Feb 2024 10:50:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwYi-0000qL-Eb
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:50:40 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 65f51b28-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:50:38 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 1ED821F8BA;
 Mon,  5 Feb 2024 10:50:38 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DAF68132DD;
 Mon,  5 Feb 2024 10:50:37 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id Oq28M329wGUjNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:50:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65f51b28-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130238; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hRIG3K7WCdgf9Z1PIv4Opjhz+muswiZWfpgWnc+2iLg=;
	b=UDpSx3gHoJ28Rag8I1uLRwN5+my/9UWih3NG0D+C1ZzTJqCX4GQi3cJV2bVhZeBjIEsT2b
	CNc10NjvPJsnu1H0WoxazDOxkhE1gx3lua7Jv2qEGFGJvV/r593B89mvEJmnvHr/Vn6dRA
	8O1bWK34oq0hTxdUd9IECxd3VF39RNE=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130238; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=hRIG3K7WCdgf9Z1PIv4Opjhz+muswiZWfpgWnc+2iLg=;
	b=UDpSx3gHoJ28Rag8I1uLRwN5+my/9UWih3NG0D+C1ZzTJqCX4GQi3cJV2bVhZeBjIEsT2b
	CNc10NjvPJsnu1H0WoxazDOxkhE1gx3lua7Jv2qEGFGJvV/r593B89mvEJmnvHr/Vn6dRA
	8O1bWK34oq0hTxdUd9IECxd3VF39RNE=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v4 06/32] tools/xen-9pfsd: add 9pfs version request support
Date: Mon,  5 Feb 2024 11:49:35 +0100
Message-Id: <20240205105001.24171-7-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-2.10 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: -2.10

Add the version request of the 9pfs protocol. For the version use the
"9P2000.u" variant, as it is supported by Mini-OS and Linux.

For the request parsing add all format items needed even in future in
order to avoid code churn for those additions later.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V3:
- use unaligned helper macros (Jason Andryuk)
---
 tools/xen-9pfsd/io.c | 201 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 201 insertions(+)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index 4a44c70c4d..839dd1112c 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -22,8 +22,12 @@
 #include "xen-9pfsd.h"
 
 /* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
+#define P9_CMD_VERSION    100
 #define P9_CMD_ERROR      107
 
+#define P9_MIN_MSIZE      2048
+#define P9_VERSION        "9P2000.u"
+
 struct p9_qid {
     uint8_t type;
 #define QID_TYPE_DIR      0x80
@@ -294,6 +298,169 @@ static unsigned int add_string(struct ring *ring, const char *str,
     return ret;
 }
 
+static bool chk_data(struct ring *ring, void *data, unsigned int len)
+{
+    struct p9_header *hdr = ring->buffer;
+
+    if ( data + len <= ring->buffer + hdr->size )
+        return true;
+
+    errno = E2BIG;
+
+    return false;
+}
+
+static bool fill_data_elem(void **par, void **array, unsigned int *array_sz,
+                           unsigned int elem_sz, void *data)
+{
+    if ( *array_sz && !*array )
+    {
+        *array = calloc(*array_sz, elem_sz);
+        if ( !*array )
+            return false;
+        *par = *array;
+    }
+
+    memcpy(*par, data, elem_sz);
+
+    if ( *array_sz )
+    {
+        *par += elem_sz;
+        *array_sz -= 1;
+    }
+
+    return true;
+}
+
+/*
+ * Fill variables with request data.
+ * fmt is a sequence of format characters. Supported characters are:
+ * a: an array (2 bytes number of elements + the following format as elements)
+ *    The number of elements is stored in the first unsigned int parameter, the
+ *    next parameter is a pointer to an array of elements as denoted by the next
+ *    format character. The array is allocated dynamically.
+ * b: 1 byte unsigned integer
+ *    The value is stored in the next parameter with type uint8_t.
+ * D: Data blob (4 byte length + <length> bytes)
+ *    2 parameters are consumed, first an unsigned int for the length, then a
+ *    pointer to the first uint8_t value.
+ *    No array support.
+ * L: 8 byte unsigned integer
+ *    The value is stored in the next parameter with type uint64_t.
+ * S: String (2 byte length + <length> characters)
+ *    The 0-terminated string is stored in device->str + off, off is stored in
+ *    the next parameter with type unsigned int.
+ * U: 4 byte unsigned integer
+ *    The value is stored in the next parameter with type uint32_t.
+ *
+ * Return value: number of filled variables, errno will be set in case of
+ *   error.
+ */
+static int fill_data(struct ring *ring, const char *fmt, ...)
+{
+    struct p9_header *hdr = ring->buffer;
+    void *data = hdr + 1;
+    void *par;
+    unsigned int pars = 0;
+    const char *f;
+    va_list ap;
+    unsigned int len;
+    unsigned int str_off;
+    unsigned int array_sz = 0;
+    void **array = NULL;
+
+    va_start(ap, fmt);
+
+    for ( f = fmt; *f; f++ )
+    {
+        if ( !array_sz )
+            par = va_arg(ap, void *);
+
+        switch ( *f )
+        {
+        case 'a':
+            f++;
+            if ( !*f || array_sz )
+                fmt_err(fmt);
+            if ( !chk_data(ring, data, sizeof(uint16_t)) )
+                return pars;
+            array_sz = get_unaligned((uint16_t *)data);
+            data += sizeof(uint16_t);
+            *(unsigned int *)par = array_sz;
+            array = va_arg(ap, void **);
+            *array = NULL;
+            break;
+
+        case 'b':
+            if ( !chk_data(ring, data, sizeof(uint8_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint8_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint8_t);
+            break;
+
+        case 'D':
+            if ( array_sz )
+                fmt_err(fmt);
+            if ( !chk_data(ring, data, sizeof(uint32_t)) )
+                return pars;
+            len = get_unaligned((uint32_t *)data);
+            data += sizeof(uint32_t);
+            *(unsigned int *)par = len;
+            par = va_arg(ap, void *);
+            if ( !chk_data(ring, data, len) )
+                return pars;
+            memcpy(par, data, len);
+            data += len;
+            break;
+
+        case 'L':
+            if ( !chk_data(ring, data, sizeof(uint64_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint64_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint64_t);
+            break;
+
+        case 'S':
+            if ( !chk_data(ring, data, sizeof(uint16_t)) )
+                return pars;
+            len = get_unaligned((uint16_t *)data);
+            data += sizeof(uint16_t);
+            if ( !chk_data(ring, data, len) )
+                return pars;
+            str_off = add_string(ring, data, len);
+            if ( str_off == ~0 )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(unsigned int),
+                                 &str_off) )
+                return pars;
+            data += len;
+            break;
+
+        case 'U':
+            if ( !chk_data(ring, data, sizeof(uint32_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint32_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint32_t);
+            break;
+
+        default:
+            fmt_err(fmt);
+        }
+
+        if ( array_sz )
+            f--;
+        pars++;
+    }
+
+    return pars;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -305,6 +472,36 @@ static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
                 &err);
 }
 
+static void p9_version(struct ring *ring, struct p9_header *hdr)
+{
+    uint32_t max_size;
+    unsigned int off;
+    char *version;
+    int ret;
+
+    ret = fill_data(ring, "US", &max_size, &off);
+    if ( ret != 2 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    if ( max_size < P9_MIN_MSIZE )
+    {
+        p9_error(ring, hdr->tag, EMSGSIZE);
+        return;
+    }
+
+    if ( max_size < ring->max_size )
+        ring->max_size = max_size;
+
+    version = ring->str + off;
+    if ( strcmp(version, P9_VERSION) )
+        version = "unknown";
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "US", &ring->max_size, version);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -360,6 +557,10 @@ void *io_thread(void *arg)
 
             switch ( hdr.cmd )
             {
+            case P9_CMD_VERSION:
+                p9_version(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:52:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:52:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675812.1051343 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwaX-00064m-CR; Mon, 05 Feb 2024 10:52:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675812.1051343; Mon, 05 Feb 2024 10:52:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwaX-00064d-95; Mon, 05 Feb 2024 10:52:33 +0000
Received: by outflank-mailman (input) for mailman id 675812;
 Mon, 05 Feb 2024 10:52:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwaW-00064Q-Le
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:52:32 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a8c4cb0b-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:52:30 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 5255922285;
 Mon,  5 Feb 2024 10:52:30 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 14A4A132DD;
 Mon,  5 Feb 2024 10:52:30 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id QOmNA+69wGXmNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8c4cb0b-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130350; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=sBN45VY796bPT208E7z1cKUSQ/jSCHWuzgN6GPy8ckw=;
	b=XPLQmfsImAnW4qPMeUaZMNYUKSCWkTag5wGDPSYySdjzxWw0LTfK7czpaKIGUbR2VOgO/U
	w/sayaMlagAz0M9Fclx3CRPeO7wYiBdhqS4ushlEwZ2F8eoSOVBWGaxr7mgKgLiW+ZIqQ4
	BWDxSi9ki62/PDIMB4SWLuhbqmMLaTY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130350; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=sBN45VY796bPT208E7z1cKUSQ/jSCHWuzgN6GPy8ckw=;
	b=XPLQmfsImAnW4qPMeUaZMNYUKSCWkTag5wGDPSYySdjzxWw0LTfK7czpaKIGUbR2VOgO/U
	w/sayaMlagAz0M9Fclx3CRPeO7wYiBdhqS4ushlEwZ2F8eoSOVBWGaxr7mgKgLiW+ZIqQ4
	BWDxSi9ki62/PDIMB4SWLuhbqmMLaTY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v4 26/32] tools/xenstored: rework ring page (un)map functions
Date: Mon,  5 Feb 2024 11:49:55 +0100
Message-Id: <20240205105001.24171-27-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [0.89 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.01)[50.16%]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: 0.89

When [un]mapping the ring page of a Xenstore client, different actions
are required for "normal" guests and dom0. Today this distinction is
made at call site.

Move this distinction into [un]map_interface() instead, avoiding code
duplication and preparing special handling for [un]mapping the stub
domain's ring page.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
 tools/xenstored/domain.c | 31 +++++++++++++------------------
 1 file changed, 13 insertions(+), 18 deletions(-)

diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index 040df98b8c..af80a8221d 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -497,14 +497,20 @@ static const struct interface_funcs domain_funcs = {
 
 static void *map_interface(domid_t domid)
 {
+	if (domid == xenbus_master_domid())
+		return xenbus_map();
+
 	return xengnttab_map_grant_ref(*xgt_handle, domid,
 				       GNTTAB_RESERVED_XENSTORE,
 				       PROT_READ|PROT_WRITE);
 }
 
-static void unmap_interface(void *interface)
+static void unmap_interface(domid_t domid, void *interface)
 {
-	xengnttab_unmap(*xgt_handle, interface, 1);
+	if (domid == xenbus_master_domid())
+		unmap_xenbus(interface);
+	else
+		xengnttab_unmap(*xgt_handle, interface, 1);
 }
 
 static int domain_tree_remove_sub(const void *ctx, struct connection *conn,
@@ -594,14 +600,8 @@ static int destroy_domain(void *_domain)
 			eprintf("> Unbinding port %i failed!\n", domain->port);
 	}
 
-	if (domain->interface) {
-		/* Domain 0 was mapped by dom0_init, so it must be unmapped
-		   using munmap() and not the grant unmap call. */
-		if (domain->domid == dom0_domid)
-			unmap_xenbus(domain->interface);
-		else
-			unmap_interface(domain->interface);
-	}
+	if (domain->interface)
+		unmap_interface(domain->domid, domain->interface);
 
 	fire_special_watches("@releaseDomain");
 
@@ -966,18 +966,13 @@ static struct domain *introduce_domain(const void *ctx,
 		return NULL;
 
 	if (!domain->introduced) {
-		interface = is_master_domain ? xenbus_map()
-					     : map_interface(domid);
+		interface = map_interface(domid);
 		if (!interface && !restore)
 			return NULL;
 		if (new_domain(domain, port, restore)) {
 			rc = errno;
-			if (interface) {
-				if (is_master_domain)
-					unmap_xenbus(interface);
-				else
-					unmap_interface(interface);
-			}
+			if (interface)
+				unmap_interface(domid, interface);
 			errno = rc;
 			return NULL;
 		}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:52:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:52:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675813.1051353 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwad-0006M9-KC; Mon, 05 Feb 2024 10:52:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675813.1051353; Mon, 05 Feb 2024 10:52:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwad-0006M0-Gk; Mon, 05 Feb 2024 10:52:39 +0000
Received: by outflank-mailman (input) for mailman id 675813;
 Mon, 05 Feb 2024 10:52:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwac-00064Q-28
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:52:38 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ac30caae-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:52:36 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 1279222100;
 Mon,  5 Feb 2024 10:52:36 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BDF5C132DD;
 Mon,  5 Feb 2024 10:52:35 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id CPKmLPO9wGXuNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac30caae-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130356; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=WeNBiDTNS7TMepXj/jHm53gvlyAKdRYh91nCqLjvqLA=;
	b=ihzYIvnwhErawe9X0SPXCpAAKCwJ5kImARwcHVfOvAYWrMKt8N3m6pcy/ylnYFzPUrDUc8
	fJGFWcsoYkpqv6UQQY1e5LpK/jgZ2kC2Qu+nfUerL6NaPTNOnbNDa4C6iZVz5ImDz+3pTK
	kuZqJjCB6DEJ06yHzi/BIfO2Uy+VYvI=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130356; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=WeNBiDTNS7TMepXj/jHm53gvlyAKdRYh91nCqLjvqLA=;
	b=ihzYIvnwhErawe9X0SPXCpAAKCwJ5kImARwcHVfOvAYWrMKt8N3m6pcy/ylnYFzPUrDUc8
	fJGFWcsoYkpqv6UQQY1e5LpK/jgZ2kC2Qu+nfUerL6NaPTNOnbNDa4C6iZVz5ImDz+3pTK
	kuZqJjCB6DEJ06yHzi/BIfO2Uy+VYvI=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v4 27/32] tools/xenstored: split domain_init()
Date: Mon,  5 Feb 2024 11:49:56 +0100
Message-Id: <20240205105001.24171-28-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Today domain_init() is called either just before calling dom0_init()
in case no live update is being performed, or it is called after
reading the global state from read_state_global(), as the event
channel fd is needed.

Split up domain_init() into a preparation part which can be called
unconditionally, and in a part setting up the event channel handle.

Note that there is no chance that chk_domain_generation() can be
called now before xc_handle has been setup, so there is no need for
the related special case anymore.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- rename new function to domain_early_init() (Julien Grall)
---
 tools/xenstored/core.c   |  2 ++
 tools/xenstored/domain.c | 12 ++++++------
 tools/xenstored/domain.h |  1 +
 3 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 9eb3cdf446..3bf2ec7734 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -2742,6 +2742,8 @@ int main(int argc, char *argv[])
 
 	talloc_enable_null_tracking();
 
+	domain_early_init();
+
 	/* Listen to hypervisor. */
 	if (!live_update) {
 		domain_init(-1);
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index af80a8221d..621b415088 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -1224,10 +1224,8 @@ static int domeq_fn(const void *key1, const void *key2)
 	return *(const unsigned int *)key1 == *(const unsigned int *)key2;
 }
 
-void domain_init(int evtfd)
+void domain_early_init(void)
 {
-	int rc;
-
 	/* Start with a random rather low domain count for the hashtable. */
 	domhash = create_hashtable(NULL, "domains", domhash_fn, domeq_fn, 0);
 	if (!domhash)
@@ -1258,6 +1256,11 @@ void domain_init(int evtfd)
 	xengnttab_set_max_grants(*xgt_handle, DOMID_FIRST_RESERVED);
 
 	talloc_set_destructor(xgt_handle, close_xgt_handle);
+}
+
+void domain_init(int evtfd)
+{
+	int rc;
 
 	if (evtfd < 0)
 		xce_handle = xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
@@ -1291,9 +1294,6 @@ static bool chk_domain_generation(unsigned int domid, uint64_t gen)
 {
 	struct domain *d;
 
-	if (!xc_handle && domid == dom0_domid)
-		return true;
-
 	d = find_domain_struct(domid);
 
 	return d && d->generation <= gen;
diff --git a/tools/xenstored/domain.h b/tools/xenstored/domain.h
index 7625dca8cd..224c4c23e2 100644
--- a/tools/xenstored/domain.h
+++ b/tools/xenstored/domain.h
@@ -82,6 +82,7 @@ int do_get_domain_path(const void *ctx, struct connection *conn,
 int do_reset_watches(const void *ctx, struct connection *conn,
 		     struct buffered_data *in);
 
+void domain_early_init(void);
 void domain_init(int evtfd);
 void dom0_init(void);
 void domain_deinit(void);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:52:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:52:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675814.1051363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwai-0006fs-T4; Mon, 05 Feb 2024 10:52:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675814.1051363; Mon, 05 Feb 2024 10:52:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwai-0006fh-Ov; Mon, 05 Feb 2024 10:52:44 +0000
Received: by outflank-mailman (input) for mailman id 675814;
 Mon, 05 Feb 2024 10:52:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwah-00064Q-Qk
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:52:43 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id af8e7d03-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:52:42 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id ACCA222288;
 Mon,  5 Feb 2024 10:52:41 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 72995132DD;
 Mon,  5 Feb 2024 10:52:41 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id Lxx6Gvm9wGX2NgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af8e7d03-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130361; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CNBMmRI5jm2OY3r5fUBNPy2uDNZ81cgAaFW7uO2/RZQ=;
	b=rd2j+e3DNUEu/4Q9X+OKH3cW4mxygqpC2Q4CSidj4iIM6NEpol3IKANp8g8I2Vkv//VyIi
	kJzIkXPdk4Wu5Pse18J6awFQUOfbrZ1ZqINl9rPR8O0YgA0RsNJPBa2mUpCIIw/ZR3B2Hx
	gOp/VaS1Jxatyn3uVijFCGVU6N7sHXA=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130361; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CNBMmRI5jm2OY3r5fUBNPy2uDNZ81cgAaFW7uO2/RZQ=;
	b=rd2j+e3DNUEu/4Q9X+OKH3cW4mxygqpC2Q4CSidj4iIM6NEpol3IKANp8g8I2Vkv//VyIi
	kJzIkXPdk4Wu5Pse18J6awFQUOfbrZ1ZqINl9rPR8O0YgA0RsNJPBa2mUpCIIw/ZR3B2Hx
	gOp/VaS1Jxatyn3uVijFCGVU6N7sHXA=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v4 28/32] tools/xenstored: map stubdom interface
Date: Mon,  5 Feb 2024 11:49:57 +0100
Message-Id: <20240205105001.24171-29-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-2.10 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: -2.10

When running as stubdom, map the stubdom's Xenstore ring page in order
to support using the 9pfs frontend.

Use the same pattern as in dom0_init() when running as daemon in dom0
(introduce the own domain, then send an event to the client side to
signal Xenstore is ready to communicate).

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
 tools/xenstored/core.c   |  2 ++
 tools/xenstored/domain.c | 27 ++++++++++++++++++++++++++-
 tools/xenstored/domain.h |  1 +
 3 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 3bf2ec7734..48fc787ac1 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -2767,6 +2767,8 @@ int main(int argc, char *argv[])
 		lu_read_state();
 #endif
 
+	stubdom_init();
+
 	check_store();
 
 	/* Get ready to listen to the tools. */
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index 621b415088..1a7d5e9756 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -37,6 +37,10 @@
 #include <xenctrl.h>
 #include <xen/grant_table.h>
 
+#ifdef __MINIOS__
+#include <mini-os/xenbus.h>
+#endif
+
 static xc_interface **xc_handle;
 xengnttab_handle **xgt_handle;
 static evtchn_port_t virq_port;
@@ -500,6 +504,11 @@ static void *map_interface(domid_t domid)
 	if (domid == xenbus_master_domid())
 		return xenbus_map();
 
+#ifdef __MINIOS__
+	if (domid == stub_domid)
+		return xenstore_buf;
+#endif
+
 	return xengnttab_map_grant_ref(*xgt_handle, domid,
 				       GNTTAB_RESERVED_XENSTORE,
 				       PROT_READ|PROT_WRITE);
@@ -509,7 +518,7 @@ static void unmap_interface(domid_t domid, void *interface)
 {
 	if (domid == xenbus_master_domid())
 		unmap_xenbus(interface);
-	else
+	else if (domid != stub_domid)
 		xengnttab_unmap(*xgt_handle, interface, 1);
 }
 
@@ -1214,6 +1223,22 @@ void dom0_init(void)
 	xenevtchn_notify(xce_handle, dom0->port);
 }
 
+void stubdom_init(void)
+{
+#ifdef __MINIOS__
+	struct domain *stubdom;
+
+	if (stub_domid < 0)
+		return;
+
+	stubdom = introduce_domain(NULL, stub_domid, xenbus_evtchn, false);
+	if (!stubdom)
+		barf_perror("Failed to initialize stubdom");
+
+	xenevtchn_notify(xce_handle, stubdom->port);
+#endif
+}
+
 static unsigned int domhash_fn(const void *k)
 {
 	return *(const unsigned int *)k;
diff --git a/tools/xenstored/domain.h b/tools/xenstored/domain.h
index 224c4c23e2..844ac11510 100644
--- a/tools/xenstored/domain.h
+++ b/tools/xenstored/domain.h
@@ -85,6 +85,7 @@ int do_reset_watches(const void *ctx, struct connection *conn,
 void domain_early_init(void);
 void domain_init(int evtfd);
 void dom0_init(void);
+void stubdom_init(void);
 void domain_deinit(void);
 void ignore_connection(struct connection *conn, unsigned int err);
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:52:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:52:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675815.1051372 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwap-000769-32; Mon, 05 Feb 2024 10:52:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675815.1051372; Mon, 05 Feb 2024 10:52:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwap-000762-0N; Mon, 05 Feb 2024 10:52:51 +0000
Received: by outflank-mailman (input) for mailman id 675815;
 Mon, 05 Feb 2024 10:52:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwan-00064Q-8e
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:52:49 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b2dd3724-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:52:47 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 55CBF22100;
 Mon,  5 Feb 2024 10:52:47 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1BBC413A3A;
 Mon,  5 Feb 2024 10:52:47 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id oxxXBf+9wGUANwAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2dd3724-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v4 29/32] tools/xenstored: mount 9pfs device in stubdom
Date: Mon,  5 Feb 2024 11:49:58 +0100
Message-Id: <20240205105001.24171-30-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 55CBF22100
X-Spam-Flag: NO

Mount the 9pfs device in stubdom enabling it to use files.

This has to happen in a worker thread in order to allow the main thread
handling the required Xenstore accesses in parallel.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- add logging in case of errors (Julien Grall)
---
 tools/xenstored/core.h   |  6 +++++
 tools/xenstored/domain.c |  2 ++
 tools/xenstored/minios.c | 54 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+)

diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index f6af086f01..fe0ee90581 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -36,6 +36,8 @@
 #include "list.h"
 #include "hashtable.h"
 
+#define XENSTORE_LIB_DIR	XEN_LIB_DIR "/xenstore"
+
 #ifndef O_CLOEXEC
 #define O_CLOEXEC 0
 /* O_CLOEXEC support is needed for Live Update in the daemon case. */
@@ -399,6 +401,10 @@ void handle_special_fds(void);
 int get_socket_fd(void);
 void set_socket_fd(int fd);
 
+#ifdef __MINIOS__
+void mount_9pfs(void);
+#endif
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index 1a7d5e9756..64c8fd0cc3 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -1236,6 +1236,8 @@ void stubdom_init(void)
 		barf_perror("Failed to initialize stubdom");
 
 	xenevtchn_notify(xce_handle, stubdom->port);
+
+	mount_9pfs();
 #endif
 }
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 22ac8defbd..562a9b4972 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -17,10 +17,20 @@
 */
 #include <sys/types.h>
 #include <sys/mman.h>
+#include <syslog.h>
+#include "talloc.h"
 #include "core.h"
 #include "utils.h"
 #include <xen/grant_table.h>
 #include <mini-os/lib.h>
+#include <mini-os/9pfront.h>
+#include <mini-os/sched.h>
+#include <mini-os/xenbus.h>
+#include <mini-os/xmalloc.h>
+
+#define P9_STATE_PATH	"device/9pfs/0/state"
+
+static void *p9_device;
 
 void finish_daemonize(void)
 {
@@ -74,3 +84,47 @@ int get_socket_fd(void)
 void set_socket_fd(int fd)
 {
 }
+
+static void mount_thread(void *p)
+{
+	xenbus_event_queue events = NULL;
+	char *err;
+	char *dummy;
+
+	err = xenbus_watch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs", &events);
+	if (err) {
+		log("error \"%s\" when setting watch on \"%s\"\n", err,
+		    P9_STATE_PATH);
+		free(err);
+		return;
+	}
+
+	for (;;) {
+		xenbus_wait_for_watch(&events);
+
+		/*
+		 * We only care for existence of the state node.
+		 * State changes are handled in init_9pfront().
+		 */
+		err = xenbus_read(XBT_NIL, P9_STATE_PATH, &dummy);
+		if (!err)
+			break;
+		free(err);
+	}
+
+	free(dummy);
+
+	err = xenbus_unwatch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs");
+	if (err) {
+		log("error \"%s\" when unwatching \"%s\", leaking watch\n",
+		    err, P9_STATE_PATH);
+		free(err);
+	}
+
+	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+}
+
+void mount_9pfs(void)
+{
+	create_thread("mount-9pfs", mount_thread, NULL);
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:53:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:53:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675819.1051383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwb1-0007qY-Cb; Mon, 05 Feb 2024 10:53:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675819.1051383; Mon, 05 Feb 2024 10:53:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwb1-0007qP-84; Mon, 05 Feb 2024 10:53:03 +0000
Received: by outflank-mailman (input) for mailman id 675819;
 Mon, 05 Feb 2024 10:53:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwaz-00064Q-63
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:53:01 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b997754a-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:52:58 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 9BEC022100;
 Mon,  5 Feb 2024 10:52:58 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5EEF9132DD;
 Mon,  5 Feb 2024 10:52:58 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id RM2vFQq+wGUQNwAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b997754a-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Julien Grall <julien@xen.org>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v4 31/32] tools/xenstored: support complete log capabilities in stubdom
Date: Mon,  5 Feb 2024 11:50:00 +0100
Message-Id: <20240205105001.24171-32-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 9BEC022100
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

With 9pfs being fully available in Xenstore-stubdom now, there is no
reason to not fully support all logging capabilities in stubdom.

Open the logfile on stubdom only after the 9pfs file system has been
mounted.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- remove now stale comment in sysconfig.xencommons.in (Julien Grall)
---
 .../Linux/init.d/sysconfig.xencommons.in      |  1 -
 tools/hotplug/Linux/launch-xenstore.in        |  1 +
 tools/xenstored/control.c                     | 30 +++++++++----------
 tools/xenstored/minios.c                      |  3 ++
 4 files changed, 19 insertions(+), 16 deletions(-)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index 433e4849af..1bdd830d8a 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -58,7 +58,6 @@ XENSTORED_ARGS=
 ## Default: Not defined, tracing off
 #
 # Log xenstored messages
-# Only evaluated if XENSTORETYPE is "daemon".
 #XENSTORED_TRACE=[yes|on|1]
 
 ## Type: integer
diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in
index e854ca1eb8..da4eeca7c5 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -98,6 +98,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 	[ -z "$XENSTORE_DOMAIN_SIZE" ] && XENSTORE_DOMAIN_SIZE=8
 	XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory $XENSTORE_DOMAIN_SIZE"
 	[ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem $XENSTORE_MAX_DOMAIN_SIZE"
+	[ -z "$XENSTORED_TRACE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS -T xenstored-trace.log"
 
 	echo -n Starting $XENSTORE_DOMAIN_KERNEL...
 	${LIBEXEC_BIN}/init-xenstore-domain $XENSTORE_DOMAIN_ARGS || exit 1
diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index b2f64d674f..dae23a5ac0 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -201,19 +201,6 @@ static int do_control_quota_s(const void *ctx, struct connection *conn,
 	return EINVAL;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_logfile(const void *ctx, struct connection *conn,
 			      const char **vec, int num)
 {
@@ -222,13 +209,26 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 
 	close_log();
 	talloc_free(tracefile);
-	tracefile = talloc_strdup(NULL, vec[0]);
+	tracefile = absolute_filename(NULL, vec[0]);
 	reopen_log();
 
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
 
+#ifdef __MINIOS__
+static int do_control_memreport(const void *ctx, struct connection *conn,
+				const char **vec, int num)
+{
+	if (num)
+		return EINVAL;
+
+	talloc_report_full(NULL, stdout);
+
+	send_ack(conn, XS_CONTROL);
+	return 0;
+}
+#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
@@ -309,10 +309,10 @@ static struct cmd_s cmds[] = {
 		"[-c <cmdline>] [-F] [-t <timeout>] <file>\n"
 		"    Default timeout is 60 seconds.", 5 },
 #endif
+	{ "logfile", do_control_logfile, "<file>" },
 #ifdef __MINIOS__
 	{ "memreport", do_control_memreport, "" },
 #else
-	{ "logfile", do_control_logfile, "<file>" },
 	{ "memreport", do_control_memreport, "[<file>]" },
 #endif
 	{ "print", do_control_print, "<string>" },
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index e70386f8c7..a229954cf4 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -122,6 +122,9 @@ static void mount_thread(void *p)
 	}
 
 	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+
+	/* Start logging if selected. */
+	reopen_log();
 }
 
 void mount_9pfs(void)
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:53:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:53:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675820.1051393 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwb4-00088S-Mk; Mon, 05 Feb 2024 10:53:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675820.1051393; Mon, 05 Feb 2024 10:53:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwb4-00088L-JX; Mon, 05 Feb 2024 10:53:06 +0000
Received: by outflank-mailman (input) for mailman id 675820;
 Mon, 05 Feb 2024 10:53:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwb4-00064Q-5p
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:53:06 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bcf3692b-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:53:04 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 4327822277;
 Mon,  5 Feb 2024 10:53:04 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0E8EE132DD;
 Mon,  5 Feb 2024 10:53:04 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id EVQjAhC+wGUVNwAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:53:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bcf3692b-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v4 32/32] tools/xenstored: have a single do_control_memreport()
Date: Mon,  5 Feb 2024 11:50:01 +0100
Message-Id: <20240205105001.24171-33-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 4327822277
X-Spam-Flag: NO

With 9pfs now available in Xenstore-stubdom, there is no reason to
have distinct do_control_memreport() variants for the daemon and the
stubdom implementations.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 tools/xenstored/control.c | 27 +++++++--------------------
 1 file changed, 7 insertions(+), 20 deletions(-)

diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index dae23a5ac0..9561289179 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -216,23 +216,11 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 	return 0;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
 	FILE *fp;
+	const char *filename;
 	int fd;
 
 	if (num > 1)
@@ -255,8 +243,12 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 			if (!fp)
 				close(fd);
 		}
-	} else
-		fp = fopen(vec[0], "a");
+	} else {
+		filename = absolute_filename(ctx, vec[0]);
+		if (!filename)
+			return ENOMEM;
+		fp = fopen(filename, "a");
+	}
 
 	if (!fp)
 		return EBADF;
@@ -267,7 +259,6 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
-#endif
 
 static int do_control_print(const void *ctx, struct connection *conn,
 			    const char **vec, int num)
@@ -310,11 +301,7 @@ static struct cmd_s cmds[] = {
 		"    Default timeout is 60 seconds.", 5 },
 #endif
 	{ "logfile", do_control_logfile, "<file>" },
-#ifdef __MINIOS__
-	{ "memreport", do_control_memreport, "" },
-#else
 	{ "memreport", do_control_memreport, "[<file>]" },
-#endif
 	{ "print", do_control_print, "<string>" },
 	{ "quota", do_control_quota,
 		"[set <name> <val>|<domid>|max [-r]]" },
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:55:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:55:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675828.1051402 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwdE-0000wH-2l; Mon, 05 Feb 2024 10:55:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675828.1051402; Mon, 05 Feb 2024 10:55:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwdE-0000wA-07; Mon, 05 Feb 2024 10:55:20 +0000
Received: by outflank-mailman (input) for mailman id 675828;
 Mon, 05 Feb 2024 10:55:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rWwdD-0000w4-D7
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:55:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWwdC-0002yX-OO; Mon, 05 Feb 2024 10:55:18 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rWwdC-0006Ws-JF; Mon, 05 Feb 2024 10:55:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=nXZrI8kOeWkz/2wG9JHrRBvKDwoqnzPqw4h63Hpu3zg=; b=OwaiXa21PiQHC8tRCbm8mYJ5qc
	GEgQndWAeoYoLisV3GxqC+8ZCzC2pOceF3VyCQOV3miuLLAlIJwh4so7wLhab/M4wIRAuUaaAqbbo
	KUxVNpIvanVG3Uw7jJ3tz5csMfef4Uy6avFvkzCxI6Ai9nq41zt1f8PxTytpLncEnuXY=;
Message-ID: <689635a7-6d34-44fe-b00b-31fdc03f6969@xen.org>
Date: Mon, 5 Feb 2024 10:55:16 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 00/32] tools: enable xenstore-stubdom to use 9pfs
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Nick Rosbrook
 <rosbrookn@gmail.com>, Samuel Thibault <samuel.thibault@ens-lyon.org>
References: <20240205105001.24171-1-jgross@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Juergen,

On 05/02/2024 10:49, Juergen Gross wrote:
> This series is adding 9pfs support to Xenstore-stubdom, enabling it
> to do logging to a dom0 directory.
> 
> This is a prerequisite for the final goal to add live update support
> to Xenstore-stubdom, as it enables the stubdom to store its state in
> a dom0 file.
> 
> The 9pfs backend is a new daemon written from scratch. Using a
> dedicated 9pfs daemon has several advantages:
> 
> - it is using much less resources than a full blown qemu process
> - it can serve multiple guests (the idea is to use it for other
>    infrastructure domains, like qemu-stubdom or driver domains, too)
> - it is designed to support several security enhancements, like
>    limiting the number of files for a guest, or limiting the allocated
>    file system space
> - it doesn't support file links (neither hard nor soft links) or
>    referencing parent directories via "..", minimizing the risk that
>    a guest can "escape" from its home directory
> 
> Note that for now the daemon only contains the minimal needed
> functionality to do logging from Xenstore-stubdom. I didn't want to
> add all the 9pfs commands and security add-ons in the beginning, in
> order to avoid needless efforts in case the idea of the daemon is
> being rejected.
> 
> Changes in V4:
> - patch 2 of V3 was applied
> - added support of reading directories
> - addressed review comments
> 
> Changes in V3:
> - new patches 1, 23-25
> - addressed review comments
> 
> Changes in V2:
> - support of multiple rings per device
> - xenlogd->xen-9pfsd rename
> - addressed review comments
> - fixed some bugs
> 
> Juergen Gross (32):
>    tools: add access macros for unaligned data
>    tools: add a new xen logging daemon
>    tools/xen-9pfsd: connect to frontend
>    tools/xen-9pfsd: add transport layer
>    tools/xen-9pfsd: add 9pfs response generation support
>    tools/xen-9pfsd: add 9pfs version request support
>    tools/xen-9pfsd: add 9pfs attach request support
>    tools/xen-9pfsd: add 9pfs walk request support
>    tools/xen-9pfsd: add 9pfs open request support
>    tools/xen-9pfsd: add 9pfs clunk request support
>    tools/xen-9pfsd: add 9pfs create request support
>    tools/xen-9pfsd: add 9pfs stat request support
>    tools/xen-9pfsd: add 9pfs write request support
>    tools/xen-9pfsd: add 9pfs read request support
>    tools/libs/light: add backend type for 9pfs PV devices
>    tools/xl: support new 9pfs backend xen_9pfsd
>    tools/helpers: allocate xenstore event channel for xenstore stubdom
>    tools/xenstored: rename xenbus_evtchn()
>    stubdom: extend xenstore stubdom configs
>    tools: add 9pfs device to xenstore-stubdom
>    tools/xenstored: add early_init() function
>    tools/xenstored: move systemd handling to posix.c
>    tools/xenstored: move all log-pipe handling into posix.c
>    tools/xenstored: move all socket handling into posix.c
>    tools/xenstored: get own domid in stubdom case
>    tools/xenstored: rework ring page (un)map functions
>    tools/xenstored: split domain_init()
>    tools/xenstored: map stubdom interface
>    tools/xenstored: mount 9pfs device in stubdom
>    tools/xenstored: add helpers for filename handling
>    tools/xenstored: support complete log capabilities in stubdom
>    tools/xenstored: have a single do_control_memreport()

I haven't checked what's the state of the 9PFS patches. Can part of the 
xenstored changes be committed without the 9PFS changes?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:57:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:57:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675831.1051412 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfE-0001Vv-Ec; Mon, 05 Feb 2024 10:57:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675831.1051412; Mon, 05 Feb 2024 10:57:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfE-0001Vo-Bg; Mon, 05 Feb 2024 10:57:24 +0000
Received: by outflank-mailman (input) for mailman id 675831;
 Mon, 05 Feb 2024 10:57:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZL-0000qL-2S
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:19 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7d259515-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:51:17 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 3AB8D2228A;
 Mon,  5 Feb 2024 10:51:17 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0AF08132DD;
 Mon,  5 Feb 2024 10:51:17 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id x5wxAaW9wGVeNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d259515-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v4 13/32] tools/xen-9pfsd: add 9pfs write request support
Date: Mon,  5 Feb 2024 11:49:42 +0100
Message-Id: <20240205105001.24171-14-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 3AB8D2228A
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add the write request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/xen-9pfsd/io.c | 54 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index 031cff9f0e..b763e3d8d9 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -33,6 +33,7 @@
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
+#define P9_CMD_WRITE      118
 #define P9_CMD_CLUNK      120
 #define P9_CMD_STAT       124
 
@@ -1246,6 +1247,55 @@ static void p9_stat(struct ring *ring, struct p9_header *hdr)
     free_fid(device, fidp);
 }
 
+static void p9_write(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint64_t off;
+    unsigned int len;
+    uint32_t written;
+    void *buf;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "ULD", &fid, &off, &len, ring->buffer);
+    if ( ret != 3 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || !fidp->opened || fidp->isdir )
+    {
+        p9_error(ring, hdr->tag, EBADF);
+        goto out;
+    }
+
+    buf = ring->buffer;
+
+    while ( len != 0 )
+    {
+        ret = pwrite(fidp->fd, buf, len, off);
+        if ( ret < 0 )
+            break;
+        len -= ret;
+        buf += ret;
+        off += ret;
+    }
+
+    written = buf - ring->buffer;
+    if ( written == 0 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "U", &written);
+
+ out:
+    free_fid(device, fidp);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1321,6 +1371,10 @@ void *io_thread(void *arg)
                 p9_create(ring, &hdr);
                 break;
 
+            case P9_CMD_WRITE:
+                p9_write(ring, &hdr);
+                break;
+
             case P9_CMD_CLUNK:
                 p9_clunk(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:57:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:57:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675832.1051416 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfE-0001Y9-MM; Mon, 05 Feb 2024 10:57:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675832.1051416; Mon, 05 Feb 2024 10:57:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfE-0001XS-Hu; Mon, 05 Feb 2024 10:57:24 +0000
Received: by outflank-mailman (input) for mailman id 675832;
 Mon, 05 Feb 2024 10:57:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwaE-0002wU-HF
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:52:14 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9ea7bc5c-c414-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:52:13 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 736251F8BA;
 Mon,  5 Feb 2024 10:52:13 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 41B7E132DD;
 Mon,  5 Feb 2024 10:52:13 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id EQSNDt29wGXQNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ea7bc5c-c414-11ee-8a45-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4 23/32] tools/xenstored: move all log-pipe handling into posix.c
Date: Mon,  5 Feb 2024 11:49:52 +0100
Message-Id: <20240205105001.24171-24-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 736251F8BA
X-Spam-Flag: NO

All of the log-pipe handling is needed only when running as daemon.

Move it into posix.c. This requires to have a service function in the
main event loop for handling the related requests and one for setting
the fds[] array. Use a generic name for those functions, as socket
handling can be added to them later, too.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V3:
- new patch
V4:
- rename fds to poll_fds (Julien Grall)
---
 tools/xenstored/core.c   | 69 +++++++++++-----------------------------
 tools/xenstored/core.h   | 11 ++++---
 tools/xenstored/minios.c | 14 ++++----
 tools/xenstored/posix.c  | 47 ++++++++++++++++++++++++++-
 4 files changed, 79 insertions(+), 62 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 7b5e1d0c0f..86ab330eaf 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -56,7 +56,7 @@
 
 extern xenevtchn_handle *xce_handle; /* in domain.c */
 static int xce_pollfd_idx = -1;
-static struct pollfd *fds;
+struct pollfd *poll_fds;
 static unsigned int current_array_size;
 static unsigned int nr_fds;
 static unsigned int delayed_requests;
@@ -69,8 +69,6 @@ char **orig_argv;
 LIST_HEAD(connections);
 int tracefd = -1;
 bool keep_orphans = false;
-int reopen_log_pipe[2];
-static int reopen_log_pipe0_pollfd_idx = -1;
 char *tracefile = NULL;
 static struct hashtable *nodes;
 unsigned int trace_flags = TRACE_OBJ | TRACE_IO;
@@ -145,18 +143,6 @@ void trace_destroy(const void *data, const char *type)
 		trace("obj: DESTROY %s %p\n", type, data);
 }
 
-/**
- * Signal handler for SIGHUP, which requests that the trace log is reopened
- * (in the main loop).  A single byte is written to reopen_log_pipe, to awaken
- * the poll() in the main loop.
- */
-static void trigger_reopen_log(int signal __attribute__((unused)))
-{
-	char c = 'A';
-	int dummy;
-	dummy = write(reopen_log_pipe[1], &c, 1);
-}
-
 void close_log(void)
 {
 	if (tracefd >= 0)
@@ -467,7 +453,7 @@ static bool conn_can_write(struct connection *conn)
 }
 
 /* This function returns index inside the array if succeed, -1 if fail */
-static int set_fd(int fd, short events)
+int set_fd(int fd, short events)
 {
 	int ret;
 	if (current_array_size < nr_fds + 1) {
@@ -479,18 +465,18 @@ static int set_fd(int fd, short events)
 		 */
 		newsize = ROUNDUP(nr_fds + 1, 8);
 
-		new_fds = realloc(fds, sizeof(struct pollfd)*newsize);
+		new_fds = realloc(poll_fds, sizeof(struct pollfd)*newsize);
 		if (!new_fds)
 			goto fail;
-		fds = new_fds;
+		poll_fds = new_fds;
 
-		memset(&fds[0] + current_array_size, 0,
+		memset(&poll_fds[0] + current_array_size, 0,
 		       sizeof(struct pollfd ) * (newsize-current_array_size));
 		current_array_size = newsize;
 	}
 
-	fds[nr_fds].fd = fd;
-	fds[nr_fds].events = events;
+	poll_fds[nr_fds].fd = fd;
+	poll_fds[nr_fds].events = events;
 	ret = nr_fds;
 	nr_fds++;
 
@@ -505,18 +491,16 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout)
 	struct connection *conn;
 	uint64_t msecs;
 
-	if (fds)
-		memset(fds, 0, sizeof(struct pollfd) * current_array_size);
+	if (poll_fds)
+		memset(poll_fds, 0, sizeof(struct pollfd) * current_array_size);
 	nr_fds = 0;
 
 	/* In case of delayed requests pause for max 1 second. */
 	*ptimeout = delayed_requests ? 1000 : -1;
 
+	set_special_fds();
 	if (sock != -1)
 		*p_sock_pollfd_idx = set_fd(sock, POLLIN|POLLPRI);
-	if (reopen_log_pipe[0] != -1)
-		reopen_log_pipe0_pollfd_idx =
-			set_fd(reopen_log_pipe[0], POLLIN|POLLPRI);
 
 	if (xce_handle != NULL)
 		xce_pollfd_idx = set_fd(xenevtchn_fd(xce_handle),
@@ -2324,12 +2308,12 @@ static bool socket_can_process(struct connection *conn, int mask)
 	if (conn->pollfd_idx == -1)
 		return false;
 
-	if (fds[conn->pollfd_idx].revents & ~(POLLIN | POLLOUT)) {
+	if (poll_fds[conn->pollfd_idx].revents & ~(POLLIN | POLLOUT)) {
 		talloc_free(conn);
 		return false;
 	}
 
-	return (fds[conn->pollfd_idx].revents & mask);
+	return (poll_fds[conn->pollfd_idx].revents & mask);
 }
 
 static bool socket_can_write(struct connection *conn)
@@ -2897,8 +2881,6 @@ int main(int argc, char *argv[])
 
 	talloc_enable_null_tracking();
 
-	init_pipe(reopen_log_pipe);
-
 	/* Listen to hypervisor. */
 	if (!live_update) {
 		domain_init(-1);
@@ -2913,7 +2895,6 @@ int main(int argc, char *argv[])
 		xprintf = trace;
 #endif
 
-	signal(SIGHUP, trigger_reopen_log);
 	if (tracefile)
 		tracefile = talloc_strdup(NULL, tracefile);
 
@@ -2934,43 +2915,29 @@ int main(int argc, char *argv[])
 	for (;;) {
 		struct connection *conn, *next;
 
-		if (poll(fds, nr_fds, timeout) < 0) {
+		if (poll(poll_fds, nr_fds, timeout) < 0) {
 			if (errno == EINTR)
 				continue;
 			barf_perror("Poll failed");
 		}
 
-		if (reopen_log_pipe0_pollfd_idx != -1) {
-			if (fds[reopen_log_pipe0_pollfd_idx].revents
-			    & ~POLLIN) {
-				close(reopen_log_pipe[0]);
-				close(reopen_log_pipe[1]);
-				init_pipe(reopen_log_pipe);
-			} else if (fds[reopen_log_pipe0_pollfd_idx].revents
-				   & POLLIN) {
-				char c;
-				if (read(reopen_log_pipe[0], &c, 1) != 1)
-					barf_perror("read failed");
-				reopen_log();
-			}
-			reopen_log_pipe0_pollfd_idx = -1;
-		}
+		handle_special_fds();
 
 		if (sock_pollfd_idx != -1) {
-			if (fds[sock_pollfd_idx].revents & ~POLLIN) {
+			if (poll_fds[sock_pollfd_idx].revents & ~POLLIN) {
 				barf_perror("sock poll failed");
 				break;
-			} else if (fds[sock_pollfd_idx].revents & POLLIN) {
+			} else if (poll_fds[sock_pollfd_idx].revents & POLLIN) {
 				accept_connection(sock);
 				sock_pollfd_idx = -1;
 			}
 		}
 
 		if (xce_pollfd_idx != -1) {
-			if (fds[xce_pollfd_idx].revents & ~POLLIN) {
+			if (poll_fds[xce_pollfd_idx].revents & ~POLLIN) {
 				barf_perror("xce_handle poll failed");
 				break;
-			} else if (fds[xce_pollfd_idx].revents & POLLIN) {
+			} else if (poll_fds[xce_pollfd_idx].revents & POLLIN) {
 				handle_event();
 				xce_pollfd_idx = -1;
 			}
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index 0de2d8a26e..44c4d0f8b8 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -25,6 +25,7 @@
 #include <sys/types.h>
 #include <dirent.h>
 #include <fcntl.h>
+#include <poll.h>
 #include <stdbool.h>
 #include <stdint.h>
 #include <time.h>
@@ -361,6 +362,8 @@ extern int dom0_event;
 extern int priv_domid;
 extern bool keep_orphans;
 
+extern struct pollfd *poll_fds;
+
 extern unsigned int timeout_watch_event_msec;
 
 /* Get internal time in milliseconds. */
@@ -387,15 +390,15 @@ evtchn_port_t get_xenbus_evtchn(void);
 void early_init(bool live_update, bool dofork, const char *pidfile);
 void late_init(bool live_update);
 
+int set_fd(int fd, short events);
+void set_special_fds(void);
+void handle_special_fds(void);
+
 void init_sockets(void);
-extern int reopen_log_pipe[2];
 
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
-/* Open a pipe for signal handling */
-void init_pipe(int reopen_log_pipe[2]);
-
 #ifndef NO_SOCKETS
 extern const struct interface_funcs socket_funcs;
 #endif
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 45bb0440be..3208169187 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -24,12 +24,6 @@ void finish_daemonize(void)
 {
 }
 
-void init_pipe(int reopen_log_pipe[2])
-{
-	reopen_log_pipe[0] = -1;
-	reopen_log_pipe[1] = -1;
-}
-
 evtchn_port_t get_xenbus_evtchn(void)
 {
 	return dom0_event;
@@ -53,3 +47,11 @@ void early_init(bool live_update, bool dofork, const char *pidfile)
 void late_init(bool live_update)
 {
 }
+
+void set_special_fds(void)
+{
+}
+
+void handle_special_fds(void)
+{
+}
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 97c8dcaba3..1ea8475293 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -33,6 +33,9 @@
 #include "core.h"
 #include "osdep.h"
 
+static int reopen_log_pipe0_pollfd_idx = -1;
+static int reopen_log_pipe[2];
+
 static void write_pidfile(const char *pidfile)
 {
 	char buf[100];
@@ -82,6 +85,19 @@ static void daemonize(void)
 	umask(0);
 }
 
+/*
+ * Signal handler for SIGHUP, which requests that the trace log is reopened
+ * (in the main loop).  A single byte is written to reopen_log_pipe, to awaken
+ * the poll() in the main loop.
+ */
+static void trigger_reopen_log(int signal __attribute__((unused)))
+{
+	char c = 'A';
+	int dummy;
+
+	dummy = write(reopen_log_pipe[1], &c, 1);
+}
+
 void finish_daemonize(void)
 {
 	int devnull = open("/dev/null", O_RDWR);
@@ -93,7 +109,7 @@ void finish_daemonize(void)
 	close(devnull);
 }
 
-void init_pipe(int reopen_log_pipe[2])
+static void init_pipe(void)
 {
 	int flags;
 	unsigned int i;
@@ -183,9 +199,38 @@ void early_init(bool live_update, bool dofork, const char *pidfile)
 
 	/* Don't kill us with SIGPIPE. */
 	signal(SIGPIPE, SIG_IGN);
+	signal(SIGHUP, trigger_reopen_log);
 
 	if (!live_update)
 		init_sockets();
+
+	init_pipe();
+}
+
+void set_special_fds(void)
+{
+	if (reopen_log_pipe[0] != -1)
+		reopen_log_pipe0_pollfd_idx =
+			set_fd(reopen_log_pipe[0], POLLIN|POLLPRI);
+}
+
+void handle_special_fds(void)
+{
+	if (reopen_log_pipe0_pollfd_idx != -1) {
+		if (poll_fds[reopen_log_pipe0_pollfd_idx].revents & ~POLLIN) {
+			close(reopen_log_pipe[0]);
+			close(reopen_log_pipe[1]);
+			init_pipe();
+		} else if (poll_fds[reopen_log_pipe0_pollfd_idx].revents &
+			   POLLIN) {
+			char c;
+
+			if (read(reopen_log_pipe[0], &c, 1) != 1)
+				barf_perror("read failed");
+			reopen_log();
+		}
+		reopen_log_pipe0_pollfd_idx = -1;
+	}
 }
 
 void late_init(bool live_update)
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:57:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:57:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675833.1051433 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfK-00027u-T8; Mon, 05 Feb 2024 10:57:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675833.1051433; Mon, 05 Feb 2024 10:57:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfK-00027j-PJ; Mon, 05 Feb 2024 10:57:30 +0000
Received: by outflank-mailman (input) for mailman id 675833;
 Mon, 05 Feb 2024 10:57:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwaP-0002wU-Fm
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:52:25 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a554a333-c414-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:52:25 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id A66501F8BA;
 Mon,  5 Feb 2024 10:52:24 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6982913707;
 Mon,  5 Feb 2024 10:52:24 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id yqo1GOi9wGXdNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a554a333-c414-11ee-8a45-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v4 25/32] tools/xenstored: get own domid in stubdom case
Date: Mon,  5 Feb 2024 11:49:54 +0100
Message-Id: <20240205105001.24171-26-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: A66501F8BA
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Obtain own domid when running as stubdom.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V2:
- replacement of V1 patch (Andrew Cooper)
V3:
- rename called function to get_domid() (Julien Grall)
- change stub_domid type to domid_t (Julien Grall)
---
 tools/xenstored/core.c   | 1 +
 tools/xenstored/core.h   | 1 +
 tools/xenstored/minios.c | 4 ++++
 3 files changed, 6 insertions(+)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index c045a6e5db..9eb3cdf446 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -2557,6 +2557,7 @@ static struct option options[] = {
 int dom0_domid = 0;
 int dom0_event = 0;
 int priv_domid = 0;
+domid_t stub_domid = DOMID_INVALID;
 
 static unsigned int get_optval_uint(const char *arg)
 {
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index 6180ec8b5c..f6af086f01 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -361,6 +361,7 @@ do {						\
 extern int dom0_domid;
 extern int dom0_event;
 extern int priv_domid;
+extern domid_t stub_domid;
 extern bool keep_orphans;
 
 extern struct pollfd *poll_fds;
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 6d0c11423c..22ac8defbd 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -20,6 +20,7 @@
 #include "core.h"
 #include "utils.h"
 #include <xen/grant_table.h>
+#include <mini-os/lib.h>
 
 void finish_daemonize(void)
 {
@@ -48,6 +49,9 @@ void unmap_xenbus(void *interface)
 
 void early_init(bool live_update, bool dofork, const char *pidfile)
 {
+	stub_domid = get_domid();
+	if (stub_domid == DOMID_INVALID)
+		barf("could not get own domid");
 }
 
 void late_init(bool live_update)
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:57:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:57:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675835.1051443 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfP-0002bd-9F; Mon, 05 Feb 2024 10:57:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675835.1051443; Mon, 05 Feb 2024 10:57:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfP-0002bU-5l; Mon, 05 Feb 2024 10:57:35 +0000
Received: by outflank-mailman (input) for mailman id 675835;
 Mon, 05 Feb 2024 10:57:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwa3-0002wU-3F
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:52:03 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 97ed90f9-c414-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:52:02 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 2E0BD1F8BF;
 Mon,  5 Feb 2024 10:52:02 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id E5098132DD;
 Mon,  5 Feb 2024 10:52:01 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id S2JXNtG9wGW/NgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97ed90f9-c414-11ee-8a45-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v4 21/32] tools/xenstored: add early_init() function
Date: Mon,  5 Feb 2024 11:49:50 +0100
Message-Id: <20240205105001.24171-22-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 2E0BD1F8BF
X-Spam-Flag: NO

Some xenstored initialization needs to be done in the daemon case only,
so split it out into a new early_init() function being a stub in the
stubdom case.

Remove the call of talloc_enable_leak_report_full(), as it serves no
real purpose: the daemon only ever exits due to a crash, in which case
a log of talloc()ed memory hardly has any value.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V2:
- rename function
- move patch earlier in the series
V3:
- add more init code for the daemon to early_init()
- move mkdir(XENSTORE_LIB_DIR) into a later patch (Julien Grall)
- remove stale stubs in minios.c
---
 tools/xenstored/core.c   | 30 +++---------------------------
 tools/xenstored/core.h   |  7 +++----
 tools/xenstored/minios.c | 11 +++--------
 tools/xenstored/posix.c  | 31 +++++++++++++++++++++++++++++--
 4 files changed, 38 insertions(+), 41 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index d4c76d71dc..eba7744fde 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -79,7 +79,7 @@ char **orig_argv;
 LIST_HEAD(connections);
 int tracefd = -1;
 bool keep_orphans = false;
-static int reopen_log_pipe[2];
+int reopen_log_pipe[2];
 static int reopen_log_pipe0_pollfd_idx = -1;
 char *tracefile = NULL;
 static struct hashtable *nodes;
@@ -2612,7 +2612,7 @@ static void destroy_fds(void)
 		close(sock);
 }
 
-static void init_sockets(void)
+void init_sockets(void)
 {
 	struct sockaddr_un addr;
 	const char *soc_str = xenstore_daemon_path();
@@ -2903,34 +2903,10 @@ int main(int argc, char *argv[])
 	if (optind != argc)
 		barf("%s: No arguments desired", argv[0]);
 
-	reopen_log();
-
-	/* Make sure xenstored directory exists. */
-	/* Errors ignored here, will be reported when we open files */
-	mkdir(xenstore_daemon_rundir(), 0755);
-
-	if (dofork) {
-		openlog("xenstored", 0, LOG_DAEMON);
-		if (!live_update)
-			daemonize();
-	}
-	if (pidfile)
-		write_pidfile(pidfile);
-
-	/* Talloc leak reports go to stderr, which is closed if we fork. */
-	if (!dofork)
-		talloc_enable_leak_report_full();
-
-	/* Don't kill us with SIGPIPE. */
-	signal(SIGPIPE, SIG_IGN);
+	early_init(live_update, dofork, pidfile);
 
 	talloc_enable_null_tracking();
 
-#ifndef NO_SOCKETS
-	if (!live_update)
-		init_sockets();
-#endif
-
 	init_pipe(reopen_log_pipe);
 
 	/* Listen to hypervisor. */
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index 480b0f5f7b..72173f1684 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -384,12 +384,11 @@ static inline bool domain_is_unprivileged(const struct connection *conn)
 
 /* Return the event channel used by xenbus. */
 evtchn_port_t get_xenbus_evtchn(void);
+void early_init(bool live_update, bool dofork, const char *pidfile);
 
-/* Write out the pidfile */
-void write_pidfile(const char *pidfile);
+void init_sockets(void);
+extern int reopen_log_pipe[2];
 
-/* Fork but do not close terminal FDs */
-void daemonize(void);
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 0779efbf91..4f48f63083 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -20,14 +20,6 @@
 #include "core.h"
 #include <xen/grant_table.h>
 
-void write_pidfile(const char *pidfile)
-{
-}
-
-void daemonize(void)
-{
-}
-
 void finish_daemonize(void)
 {
 }
@@ -54,3 +46,6 @@ void unmap_xenbus(void *interface)
 	xengnttab_unmap(*xgt_handle, interface, 1);
 }
 
+void early_init(bool live_update, bool dofork, const char *pidfile)
+{
+}
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 7e03dd982d..9463ef5c8d 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -20,14 +20,17 @@
 #include <sys/stat.h>
 #include <unistd.h>
 #include <fcntl.h>
+#include <signal.h>
 #include <stdlib.h>
+#include <syslog.h>
 #include <sys/mman.h>
+#include <xen-tools/xenstore-common.h>
 
 #include "utils.h"
 #include "core.h"
 #include "osdep.h"
 
-void write_pidfile(const char *pidfile)
+static void write_pidfile(const char *pidfile)
 {
 	char buf[100];
 	int len;
@@ -49,7 +52,7 @@ void write_pidfile(const char *pidfile)
 }
 
 /* Stevens. */
-void daemonize(void)
+static void daemonize(void)
 {
 	pid_t pid;
 
@@ -157,3 +160,27 @@ void *xenbus_map(void)
 
 	return addr;
 }
+
+void early_init(bool live_update, bool dofork, const char *pidfile)
+{
+	reopen_log();
+
+	/* Make sure xenstored directory exists. */
+	/* Errors ignored here, will be reported when we open files */
+	mkdir(xenstore_daemon_rundir(), 0755);
+
+	if (dofork) {
+		openlog("xenstored", 0, LOG_DAEMON);
+		if (!live_update)
+			daemonize();
+	}
+
+	if (pidfile)
+		write_pidfile(pidfile);
+
+	/* Don't kill us with SIGPIPE. */
+	signal(SIGPIPE, SIG_IGN);
+
+	if (!live_update)
+		init_sockets();
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:57:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:57:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675837.1051453 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfT-00034x-H5; Mon, 05 Feb 2024 10:57:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675837.1051453; Mon, 05 Feb 2024 10:57:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfT-00034m-DV; Mon, 05 Feb 2024 10:57:39 +0000
Received: by outflank-mailman (input) for mailman id 675837;
 Mon, 05 Feb 2024 10:57:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwYn-0000qL-Pc
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:50:45 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 692a4a00-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:50:44 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id AC6BA22100;
 Mon,  5 Feb 2024 10:50:43 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8224E132DD;
 Mon,  5 Feb 2024 10:50:43 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id IPddHoO9wGUqNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:50:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 692a4a00-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4 07/32] tools/xen-9pfsd: add 9pfs attach request support
Date: Mon,  5 Feb 2024 11:49:36 +0100
Message-Id: <20240205105001.24171-8-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: AC6BA22100
X-Spam-Flag: NO

Add the attach request of the 9pfs protocol. This introduces the "fid"
scheme of the 9pfs protocol.

As this will be needed later, use a dedicated memory allocation
function in alloc_fid() and prepare a fid reference count.

For filling the qid data take the approach from the qemu 9pfs backend
implementation.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- make fill_qid() parameter stbuf const (Jason Andryuk)
- free fids after disconnecting guest (Jason Andryuk)
V3:
- only store relative path in fid (Jason Andryuk)
V4:
- store a path directly usable by *at() functions in fid (Jason Andryuk)
---
 tools/xen-9pfsd/io.c        | 164 ++++++++++++++++++++++++++++++++++++
 tools/xen-9pfsd/xen-9pfsd.c |   6 ++
 tools/xen-9pfsd/xen-9pfsd.h |  14 +++
 3 files changed, 184 insertions(+)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index 839dd1112c..4f6acefc87 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -11,11 +11,14 @@
  * before looking for the next request.
  */
 
+#include <assert.h>
 #include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <sys/types.h>
+#include <sys/stat.h>
 #include <xenctrl.h>           /* For cpu barriers. */
 #include <xen-tools/common-macros.h>
 
@@ -23,6 +26,7 @@
 
 /* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
 #define P9_CMD_VERSION    100
+#define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
 
 #define P9_MIN_MSIZE      2048
@@ -461,6 +465,126 @@ static int fill_data(struct ring *ring, const char *fmt, ...)
     return pars;
 }
 
+static struct p9_fid *find_fid(device *device, unsigned int fid)
+{
+    struct p9_fid *fidp;
+
+    XEN_TAILQ_FOREACH(fidp, &device->fids, list)
+    {
+        if ( fidp->fid == fid )
+            return fidp;
+    }
+
+    return NULL;
+}
+
+static struct p9_fid *alloc_fid_mem(device *device, unsigned int fid,
+                                    const char *path)
+{
+    struct p9_fid *fidp;
+    size_t pathlen;
+
+    pathlen = strlen(path);
+    fidp = calloc(sizeof(*fidp) + pathlen + 1, 1);
+    if ( !fidp )
+        return NULL;
+
+    fidp->fid = fid;
+    strncpy(fidp->path, path, pathlen);
+
+    return fidp;
+}
+
+static struct p9_fid *alloc_fid(device *device, unsigned int fid,
+                                const char *path)
+{
+    struct p9_fid *fidp = NULL;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    if ( find_fid(device, fid) )
+    {
+        errno = EBADFD;
+        goto out;
+    }
+
+    if ( device->n_fids >= device->max_open_files )
+    {
+        errno = EMFILE;
+        goto out;
+    }
+
+    fidp = alloc_fid_mem(device, fid, path);
+    if ( !fidp )
+        goto out;
+
+    fidp->ref = 1;
+    XEN_TAILQ_INSERT_HEAD(&device->fids, fidp, list);
+    device->n_fids++;
+
+ out:
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    return fidp;
+}
+
+static void free_fid(device *device, struct p9_fid *fidp)
+{
+    if ( !fidp )
+        return;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    fidp->ref--;
+    if ( !fidp->ref )
+    {
+        device->n_fids--;
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        free(fidp);
+    }
+
+    pthread_mutex_unlock(&device->fid_mutex);
+}
+
+void free_fids(device *device)
+{
+    struct p9_fid *fidp;
+
+    while ( (fidp = XEN_TAILQ_FIRST(&device->fids)) != NULL )
+    {
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        free(fidp);
+    }
+}
+
+static const char *relpath_from_path(const char *path)
+{
+    if (!strcmp(path, "/"))
+        return ".";
+
+    return (path[0] == '/') ? path + 1 : path;
+}
+
+static int fill_qid(device *device, const char *path, struct p9_qid *qid,
+                    const struct stat *stbuf)
+{
+    struct stat st;
+
+    if ( !stbuf )
+    {
+        if ( fstatat(device->root_fd, path, &st, 0) )
+            return errno;
+
+        stbuf = &st;
+    }
+
+    qid->type = S_ISDIR(stbuf->st_mode) ? QID_TYPE_DIR : 0;
+    qid->version = stbuf->st_mtime ^ (stbuf->st_size << 8);
+    qid->path = stbuf->st_ino;
+
+    return 0;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -502,6 +626,42 @@ static void p9_version(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "US", &ring->max_size, version);
 }
 
+static void p9_attach(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint32_t dummy_u32;
+    unsigned int dummy_uint;
+    struct p9_qid qid;
+    int ret;
+
+    ret = fill_data(ring, "UUSSU", &fid, &dummy_u32, &dummy_uint, &dummy_uint,
+                    &dummy_u32);
+    if ( ret != 5 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    device->root_fid = alloc_fid(device, fid, relpath_from_path("/"));
+    if ( !device->root_fid )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    ret = fill_qid(device, device->root_fid->path, &qid, NULL);
+    if ( ret )
+    {
+        free_fid(device, device->root_fid);
+        device->root_fid = NULL;
+        p9_error(ring, hdr->tag, ret);
+        return;
+    }
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "Q", &qid);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -561,6 +721,10 @@ void *io_thread(void *arg)
                 p9_version(ring, &hdr);
                 break;
 
+            case P9_CMD_ATTACH:
+                p9_attach(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/xen-9pfsd/xen-9pfsd.c b/tools/xen-9pfsd/xen-9pfsd.c
index 73b6c3a30e..b83b896bcc 100644
--- a/tools/xen-9pfsd/xen-9pfsd.c
+++ b/tools/xen-9pfsd/xen-9pfsd.c
@@ -304,6 +304,9 @@ static device *new_device(unsigned int domid, unsigned int devid)
         return NULL;
     }
 
+    pthread_mutex_init(&device->fid_mutex, NULL);
+    XEN_TAILQ_INIT(&device->fids);
+
     val = read_backend_node(device, "security_model");
     if ( !val || strcmp(val, "none") )
     {
@@ -399,6 +402,8 @@ static void disconnect_guest(device *device)
     }
 
     device->num_rings = 0;
+
+    free_fids(device);
 }
 
 static void close_device(device *device, enum xenbus_state state)
@@ -482,6 +487,7 @@ static void remove_device(device *device)
     XEN_TAILQ_REMOVE(&devs, device, list);
 
     disconnect_guest(device);
+    pthread_mutex_destroy(&device->fid_mutex);
     free_device(device);
 }
 
diff --git a/tools/xen-9pfsd/xen-9pfsd.h b/tools/xen-9pfsd/xen-9pfsd.h
index 0cde0d2bb8..6c755411a7 100644
--- a/tools/xen-9pfsd/xen-9pfsd.h
+++ b/tools/xen-9pfsd/xen-9pfsd.h
@@ -21,6 +21,13 @@ struct p9_header {
     uint16_t tag;
 } __attribute__((packed));
 
+struct p9_fid {
+    XEN_TAILQ_ENTRY(struct p9_fid) list;
+    unsigned int fid;
+    unsigned int ref;
+    char path[];
+};
+
 typedef struct device device;
 
 struct ring {
@@ -71,10 +78,17 @@ struct device {
     unsigned int num_rings;
     struct ring *ring[MAX_RINGS];
     int root_fd;
+
+    /* File system handling. */
+    pthread_mutex_t fid_mutex;
+    XEN_TAILQ_HEAD(fidhead, struct p9_fid) fids;
+    struct p9_fid *root_fid;
+    unsigned int n_fids;
 };
 
 extern xenevtchn_handle *xe;
 
 void *io_thread(void *arg);
+void free_fids(device *device);
 
 #endif /* XEN_9PFSD_H */
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:57:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:57:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675838.1051463 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfU-0003Nz-QF; Mon, 05 Feb 2024 10:57:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675838.1051463; Mon, 05 Feb 2024 10:57:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfU-0003N5-Mg; Mon, 05 Feb 2024 10:57:40 +0000
Received: by outflank-mailman (input) for mailman id 675838;
 Mon, 05 Feb 2024 10:57:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZF-0000qL-LD
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:13 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 79ce377d-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:51:11 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 9CE161F8C5;
 Mon,  5 Feb 2024 10:51:11 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 71686132DD;
 Mon,  5 Feb 2024 10:51:11 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id Jy0lGp+9wGVXNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 79ce377d-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4 12/32] tools/xen-9pfsd: add 9pfs stat request support
Date: Mon,  5 Feb 2024 11:49:41 +0100
Message-Id: <20240205105001.24171-13-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 9CE161F8C5
X-Spam-Flag: NO

Add the stat request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V3:
- use fstatat() (Jason Andryuk)
V4:
- add "s" format to fill_buffer() as a preparation for reading dirs
---
 tools/xen-9pfsd/io.c | 102 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index 583eea6059..031cff9f0e 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -34,6 +34,7 @@
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
 #define P9_CMD_CLUNK      120
+#define P9_CMD_STAT       124
 
 /* P9 protocol open flags. */
 #define P9_OREAD            0   /* read */
@@ -60,6 +61,25 @@ struct p9_qid {
     uint64_t path;
 };
 
+struct p9_stat {
+    uint16_t size;
+    uint16_t type;
+    uint32_t dev;
+    struct p9_qid qid;
+    uint32_t mode;
+    uint32_t atime;
+    uint32_t mtime;
+    uint64_t length;
+    const char *name;
+    const char *uid;
+    const char *gid;
+    const char *muid;
+    const char *extension;
+    uint32_t n_uid;
+    uint32_t n_gid;
+    uint32_t n_muid;
+};
+
 /*
  * Note that the ring names "in" and "out" are from the frontend's
  * perspective, so the "in" ring will be used for responses to the frontend,
@@ -166,6 +186,7 @@ static void fmt_err(const char *fmt)
  * S: String (2 byte length + <length> characters)
  *    The length is obtained via strlen() of the parameter, being a pointer
  *    to the first character of the string
+ * s: stat (struct p9_stat)
  * U: 4 byte unsigned integer
  *    The parameter is a pointer to a uint32_t value
  */
@@ -176,6 +197,8 @@ static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
     const void *par;
     const char *str_val;
     const struct p9_qid *qid;
+    const struct p9_stat *stat;
+    uint16_t tlen;
     unsigned int len;
     unsigned int array_sz = 0;
     unsigned int elem_sz = 0;
@@ -259,6 +282,18 @@ static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
             *data += len;
             break;
 
+        case 's':
+            stat = par;
+            elem_sz = sizeof(*stat);
+            tlen = stat->size + sizeof(stat->size);
+            fill_buffer_at(data, "uuuUQUUULSSSSSUUU", &tlen, &stat->size,
+                           &stat->type, &stat->dev, &stat->qid, &stat->mode,
+                           &stat->atime, &stat->mtime, &stat->length,
+                           stat->name, stat->uid, stat->gid, stat->muid,
+                           stat->extension, &stat->n_uid, &stat->n_gid,
+                           &stat->n_muid);
+            break;
+
         case 'U':
             put_unaligned(*(const uint32_t *)par, (uint32_t *)*data);
             elem_sz = sizeof(uint32_t);
@@ -1148,6 +1183,69 @@ static void p9_clunk(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "");
 }
 
+static void fill_p9_stat(device *device, struct p9_stat *p9s, struct stat *st,
+                         const char *name)
+{
+    memset(p9s, 0, sizeof(*p9s));
+    fill_qid(device, NULL, &p9s->qid, st);
+    p9s->mode = st->st_mode & 0777;
+    if ( S_ISDIR(st->st_mode) )
+        p9s->mode |= P9_CREATE_PERM_DIR;
+    p9s->atime = st->st_atime;
+    p9s->mtime = st->st_mtime;
+    p9s->length = st->st_size;
+    p9s->name = name;
+    p9s->uid = "";
+    p9s->gid = "";
+    p9s->muid = "";
+    p9s->extension = "";
+    p9s->n_uid = 0;
+    p9s->n_gid = 0;
+    p9s->n_muid = 0;
+
+    /*
+     * Size of individual fields without the size field, including 5 2-byte
+     * string length fields.
+     */
+    p9s->size = 71 + strlen(p9s->name);
+}
+
+static void p9_stat(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    struct p9_fid *fidp;
+    struct p9_stat p9s;
+    struct stat st;
+    int ret;
+
+    ret = fill_data(ring, "U", &fid);
+    if ( ret != 1 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+    fill_p9_stat(device, &p9s, &st, strrchr(fidp->path, '/') + 1);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "s", &p9s);
+
+ out:
+    free_fid(device, fidp);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1227,6 +1325,10 @@ void *io_thread(void *arg)
                 p9_clunk(ring, &hdr);
                 break;
 
+            case P9_CMD_STAT:
+                p9_stat(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:57:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:57:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675842.1051473 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfZ-00041b-2R; Mon, 05 Feb 2024 10:57:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675842.1051473; Mon, 05 Feb 2024 10:57:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfY-00041Q-VM; Mon, 05 Feb 2024 10:57:44 +0000
Received: by outflank-mailman (input) for mailman id 675842;
 Mon, 05 Feb 2024 10:57:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZW-0000qL-Vy
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:31 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 83f64304-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:51:29 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 722122228B;
 Mon,  5 Feb 2024 10:51:28 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 30D55132DD;
 Mon,  5 Feb 2024 10:51:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 00Z8CrC9wGWONgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83f64304-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130288; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4t9CaY2HdSuK4w688pTUcmMnQS10iOMn7pZg/fdTzSY=;
	b=FpXGNn3J0p85LBUD59mX6ixHhRtOEUaPKA5cRgtiNCkAioACVKEwKq0rd2cN25Lq+wLZsO
	WKaRiGRU+S4yv4QuNwa8F27Js0QNNWsHC5+vt+nj/iw6ttr8PHphL8gLbD4NoF5Nc929bX
	pndAAos6MwBLMzqVZCD7wDRL5vWSVFQ=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130288; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4t9CaY2HdSuK4w688pTUcmMnQS10iOMn7pZg/fdTzSY=;
	b=FpXGNn3J0p85LBUD59mX6ixHhRtOEUaPKA5cRgtiNCkAioACVKEwKq0rd2cN25Lq+wLZsO
	WKaRiGRU+S4yv4QuNwa8F27Js0QNNWsHC5+vt+nj/iw6ttr8PHphL8gLbD4NoF5Nc929bX
	pndAAos6MwBLMzqVZCD7wDRL5vWSVFQ=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	George Dunlap <george.dunlap@cloud.com>
Subject: [PATCH v4 15/32] tools/libs/light: add backend type for 9pfs PV devices
Date: Mon,  5 Feb 2024 11:49:44 +0100
Message-Id: <20240205105001.24171-16-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email,cloud.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,citrix.com,gmail.com,xen.org,cloud.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Make the backend type of 9pfs PV devices configurable. The default is
"qemu" with the related Xenstore backend-side directory being "9pfs".

Add another type "xen_9pfsd" with the related Xenstore backend-side
directory "xen_9pfs".

As additional security features it is possible to specify:
- "max-space" for limiting the maximum space consumed on the filesystem
  in MBs
- "max-files" for limiting the maximum number of files in the
  filesystem
- "max-open-files" for limiting the maximum number of concurrent open
  files

For convenience "auto-delete" is available to let the backend delete the
oldest file of the guest in case otherwise "max-space" or "max-files"
would be violated.

The xen-9pfsd daemon will be started by libxenlight automatically when
the first "xen_9pfs" device is being created.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: George Dunlap <george.dunlap@cloud.com> # Golang bits
---
V3:
- regenerate go bindings
V4:
- rename libxl_device_p9_dm_needed() to libxl__device_p9_dm_needed()
  (Anthony Perard)
- reorder span related functions (Anthony Perard)
- add comment for xen9pfsd_spawn() return values (Anthony Perard)
- add LIBXL_HAVE_XEN_9PFS to libxl.h (Anthony Perard)
- use a copy of 'p9' in xen9pfsd_spawn() (Anthony Perard)
---
 tools/golang/xenlight/helpers.gen.go      |  16 +++
 tools/golang/xenlight/types.gen.go        |  13 ++
 tools/include/libxl.h                     |   7 +
 tools/libs/light/libxl_9pfs.c             | 159 +++++++++++++++++++++-
 tools/libs/light/libxl_create.c           |   4 +-
 tools/libs/light/libxl_dm.c               |   2 +-
 tools/libs/light/libxl_types.idl          |  11 ++
 tools/libs/light/libxl_types_internal.idl |   1 +
 8 files changed, 206 insertions(+), 7 deletions(-)

diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 35e209ff1b..8f44397a4e 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1280,6 +1280,9 @@ return fmt.Errorf("converting field Rdm: %v", err)
 }
 x.RdmMemBoundaryMemkb = uint64(tmp.rdm_mem_boundary_memkb)
 x.McaCaps = uint64(tmp.mca_caps)
+if err := x.Pirq.fromC(&tmp.pirq);err != nil {
+return fmt.Errorf("converting field Pirq: %v", err)
+}
 return nil
 }
 
@@ -1613,6 +1616,9 @@ return fmt.Errorf("converting field Rdm: %v", err)
 }
 hvm.rdm_mem_boundary_memkb = C.uint64_t(tmp.RdmMemBoundaryMemkb)
 hvm.mca_caps = C.uint64_t(tmp.McaCaps)
+if err := tmp.Pirq.toC(&hvm.pirq); err != nil {
+return fmt.Errorf("converting field Pirq: %v", err)
+}
 hvmBytes := C.GoBytes(unsafe.Pointer(&hvm),C.sizeof_libxl_domain_build_info_type_union_hvm)
 copy(xc.u[:],hvmBytes)
 case DomainTypePv:
@@ -2434,6 +2440,11 @@ x.Tag = C.GoString(xc.tag)
 x.Path = C.GoString(xc.path)
 x.SecurityModel = C.GoString(xc.security_model)
 x.Devid = Devid(xc.devid)
+x.Type = P9Type(xc._type)
+x.MaxSpace = int(xc.max_space)
+x.MaxFiles = int(xc.max_files)
+x.MaxOpenFiles = int(xc.max_open_files)
+x.AutoDelete = bool(xc.auto_delete)
 
  return nil}
 
@@ -2452,6 +2463,11 @@ xc.path = C.CString(x.Path)}
 if x.SecurityModel != "" {
 xc.security_model = C.CString(x.SecurityModel)}
 xc.devid = C.libxl_devid(x.Devid)
+xc._type = C.libxl_p9_type(x.Type)
+xc.max_space = C.int(x.MaxSpace)
+xc.max_files = C.int(x.MaxFiles)
+xc.max_open_files = C.int(x.MaxOpenFiles)
+xc.auto_delete = C.bool(x.AutoDelete)
 
  return nil
  }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 7907aa8999..d31722407a 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -122,6 +122,13 @@ NicTypeVifIoemu NicType = 1
 NicTypeVif NicType = 2
 )
 
+type P9Type int
+const(
+P9TypeUnknown P9Type = 0
+P9TypeQemu P9Type = 1
+P9TypeXen9Pfsd P9Type = 2
+)
+
 type ActionOnShutdown int
 const(
 ActionOnShutdownDestroy ActionOnShutdown = 1
@@ -651,6 +658,7 @@ SerialList StringList
 Rdm RdmReserve
 RdmMemBoundaryMemkb uint64
 McaCaps uint64
+Pirq Defbool
 }
 
 func (x DomainBuildInfoTypeUnionHvm) isDomainBuildInfoTypeUnion(){}
@@ -888,6 +896,11 @@ Tag string
 Path string
 SecurityModel string
 Devid Devid
+Type P9Type
+MaxSpace int
+MaxFiles int
+MaxOpenFiles int
+AutoDelete bool
 }
 
 type DevicePvcallsif struct {
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index f1652b1664..a554f2ccd6 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -615,6 +615,13 @@
  */
 #define LIBXL_HAVE_HVM_PIRQ 1
 
+/*
+ * LIBXL_HAVE_XEN_9PFS indicates the presence of the xen-9pfsd related
+ * fields in libxl_device_p9: type, max_space, max_files, max_open_files and
+ * auto_delete.
+ */
+#define LIBXL_HAVE_XEN_9PFS 1
+
 /*
  * libxl memory management
  *
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 5ab0d3aa21..68fa9947ad 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -33,20 +33,173 @@ static int libxl__set_xenstore_p9(libxl__gc *gc, uint32_t domid,
 
     flexarray_append_pair(front, "tag", p9->tag);
 
+    if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD) {
+        flexarray_append_pair(back, "max-space",
+                              GCSPRINTF("%u", p9->max_space));
+        flexarray_append_pair(back, "max-files",
+                              GCSPRINTF("%u", p9->max_files));
+        flexarray_append_pair(back, "max-open-files",
+                              GCSPRINTF("%u", p9->max_open_files));
+        flexarray_append_pair(back, "auto-delete",
+                              p9->auto_delete ? "1" : "0");
+    }
+
+    return 0;
+}
+
+static int libxl__device_from_p9(libxl__gc *gc, uint32_t domid,
+                                 libxl_device_p9 *type, libxl__device *device)
+{
+    device->backend_devid   = type->devid;
+    device->backend_domid   = type->backend_domid;
+    device->backend_kind    = type->type == LIBXL_P9_TYPE_QEMU
+                              ? LIBXL__DEVICE_KIND_9PFS
+                              : LIBXL__DEVICE_KIND_XEN_9PFS;
+    device->devid           = type->devid;
+    device->domid           = domid;
+    device->kind            = LIBXL__DEVICE_KIND_9PFS;
+
     return 0;
 }
 
-#define libxl__add_p9s NULL
+static int libxl__device_p9_dm_needed(void *e, unsigned domid)
+{
+    libxl_device_p9 *elem = e;
+
+    return elem->type == LIBXL_P9_TYPE_QEMU && elem->backend_domid == domid;
+}
+
+typedef struct libxl__aop9_state libxl__aop9_state;
+
+struct libxl__aop9_state {
+    libxl__spawn_state spawn;
+    libxl__ao_device *aodev;
+    libxl_device_p9 p9;
+    uint32_t domid;
+    void (*callback)(libxl__egc *, libxl__aop9_state *, int);
+};
+
+static void xen9pfsd_confirm(libxl__egc *egc, libxl__spawn_state *spawn,
+                             const char *xsdata);
+static void xen9pfsd_failed(libxl__egc *egc, libxl__spawn_state *spawn, int rc);
+static void xen9pfsd_detached(libxl__egc *egc, libxl__spawn_state *spawn);
+static void xen9pfsd_spawn_outcome(libxl__egc *egc, libxl__aop9_state *aop9,
+                                   int rc);
+
+/*
+ * Spawn the xen-9pfsd daemon if needed.
+ * returns:
+ * < 0 if error
+ * 0 if no daemon needs to be spawned
+ * 1 if daemon was spawned
+ */
+static int xen9pfsd_spawn(libxl__egc *egc, uint32_t domid, libxl_device_p9 *p9,
+                         libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    struct libxl__aop9_state *aop9;
+    int rc;
+    char *args[] = { "xen-9pfsd", NULL };
+    char *path = GCSPRINTF("/local/domain/%u/libxl/xen-9pfs",
+                           p9->backend_domid);
+
+    if (p9->type != LIBXL_P9_TYPE_XEN_9PFSD ||
+        libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/state", path)))
+        return 0;
+
+    GCNEW(aop9);
+    aop9->aodev = aodev;
+    libxl_device_p9_copy(CTX, &aop9->p9, p9);
+    aop9->domid = domid;
+    aop9->callback = xen9pfsd_spawn_outcome;
+
+    aop9->spawn.ao = aodev->ao;
+    aop9->spawn.what = "xen-9pfs daemon";
+    aop9->spawn.xspath = GCSPRINTF("%s/state", path);
+    aop9->spawn.timeout_ms = LIBXL_DEVICE_MODEL_START_TIMEOUT * 1000;
+    aop9->spawn.pidpath = GCSPRINTF("%s/pid", path);
+    aop9->spawn.midproc_cb = libxl__spawn_record_pid;
+    aop9->spawn.confirm_cb = xen9pfsd_confirm;
+    aop9->spawn.failure_cb = xen9pfsd_failed;
+    aop9->spawn.detached_cb = xen9pfsd_detached;
+    rc = libxl__spawn_spawn(egc, &aop9->spawn);
+    if (rc < 0)
+        return rc;
+    if (!rc) {
+        setsid();
+        libxl__exec(gc, -1, -1, -1, LIBEXEC_BIN "/xen-9pfsd", args, NULL);
+    }
+
+    return 1;
+}
+
+static void xen9pfsd_confirm(libxl__egc *egc, libxl__spawn_state *spawn,
+                             const char *xsdata)
+{
+    STATE_AO_GC(spawn->ao);
+
+    if (!xsdata)
+        return;
+
+    if (strcmp(xsdata, "running"))
+        return;
+
+    libxl__spawn_initiate_detach(gc, spawn);
+}
+
+static void xen9pfsd_failed(libxl__egc *egc, libxl__spawn_state *spawn, int rc)
+{
+    libxl__aop9_state *aop9 = CONTAINER_OF(spawn, *aop9, spawn);
+
+    xen9pfsd_spawn_outcome(egc, aop9, rc);
+}
+
+static void xen9pfsd_detached(libxl__egc *egc, libxl__spawn_state *spawn)
+{
+    libxl__aop9_state *aop9 = CONTAINER_OF(spawn, *aop9, spawn);
+
+    xen9pfsd_spawn_outcome(egc, aop9, 0);
+}
+
+static void xen9pfsd_spawn_outcome(libxl__egc *egc, libxl__aop9_state *aop9,
+                                   int rc)
+{
+    aop9->aodev->rc = rc;
+    if (rc)
+        aop9->aodev->callback(egc, aop9->aodev);
+    else
+        libxl__device_add_async(egc, aop9->domid, &libxl__p9_devtype,
+                                &aop9->p9, aop9->aodev);
+}
+
+static void libxl__device_p9_add(libxl__egc *egc, uint32_t domid,
+                                 libxl_device_p9 *p9,
+                                 libxl__ao_device *aodev)
+{
+    int rc;
+
+    rc = xen9pfsd_spawn(egc, domid, p9, aodev);
+    if (rc == 1)
+        return;
+
+    if (rc == 0)
+        libxl__device_add_async(egc, domid, &libxl__p9_devtype, p9, aodev);
+
+    aodev->rc = rc;
+    if (rc)
+        aodev->callback(egc, aodev);
+}
+
 #define libxl_device_p9_list NULL
 #define libxl_device_p9_compare NULL
 
 static LIBXL_DEFINE_UPDATE_DEVID(p9)
-static LIBXL_DEFINE_DEVICE_FROM_TYPE(p9)
+static LIBXL_DEFINE_DEVICES_ADD(p9)
 
 LIBXL_DEFINE_DEVICE_REMOVE(p9)
 
 DEFINE_DEVICE_TYPE_STRUCT(p9, 9PFS, p9s,
-    .skip_attach = 1,
     .set_xenstore_config = (device_set_xenstore_config_fn_t)
                            libxl__set_xenstore_p9,
+    .dm_needed = libxl__device_p9_dm_needed,
 );
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 0008fac607..5546335973 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -1761,9 +1761,6 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
         libxl__device_console_dispose(&console);
     }
 
-    for (i = 0; i < d_config->num_p9s; i++)
-        libxl__device_add(gc, domid, &libxl__p9_devtype, &d_config->p9s[i]);
-
     for (i = 0; i < d_config->num_pvcallsifs; i++)
         libxl__device_add(gc, domid, &libxl__pvcallsif_devtype,
                           &d_config->pvcallsifs[i]);
@@ -1900,6 +1897,7 @@ const libxl__device_type *device_type_tbl[] = {
     &libxl__vdispl_devtype,
     &libxl__vsnd_devtype,
     &libxl__virtio_devtype,
+    &libxl__p9_devtype,
     NULL
 };
 
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 14b593110f..2aaaeb5aa7 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -3761,7 +3761,7 @@ int libxl__need_xenpv_qemu(libxl__gc *gc, libxl_domain_config *d_config)
         goto out;
     }
 
-    if (d_config->num_vfbs > 0 || d_config->num_p9s > 0) {
+    if (d_config->num_vfbs > 0) {
         ret = 1;
         goto out;
     }
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 899ad30969..682092a058 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -150,6 +150,12 @@ libxl_nic_type = Enumeration("nic_type", [
     (2, "VIF"),
     ])
 
+libxl_p9_type = Enumeration("p9_type", [
+    (0, "unknown"),
+    (1, "qemu"),
+    (2, "xen_9pfsd"),
+    ])
+
 libxl_action_on_shutdown = Enumeration("action_on_shutdown", [
     (1, "DESTROY"),
 
@@ -943,6 +949,11 @@ libxl_device_p9 = Struct("device_p9", [
     ("path",             string),
     ("security_model",   string),
     ("devid",            libxl_devid),
+    ("type",             libxl_p9_type),
+    ("max_space",        integer),
+    ("max_files",        integer),
+    ("max_open_files",   integer),
+    ("auto_delete",      bool),
 ])
 
 libxl_device_pvcallsif = Struct("device_pvcallsif", [
diff --git a/tools/libs/light/libxl_types_internal.idl b/tools/libs/light/libxl_types_internal.idl
index e24288f1a5..39da71cef5 100644
--- a/tools/libs/light/libxl_types_internal.idl
+++ b/tools/libs/light/libxl_types_internal.idl
@@ -34,6 +34,7 @@ libxl__device_kind = Enumeration("device_kind", [
     (16, "VINPUT"),
     (17, "VIRTIO_DISK"),
     (18, "VIRTIO"),
+    (19, "XEN_9PFS"),
     ])
 
 libxl__console_backend = Enumeration("console_backend", [
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:57:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:57:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675844.1051483 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfa-0004M1-Lh; Mon, 05 Feb 2024 10:57:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675844.1051483; Mon, 05 Feb 2024 10:57:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfa-0004LN-GO; Mon, 05 Feb 2024 10:57:46 +0000
Received: by outflank-mailman (input) for mailman id 675844;
 Mon, 05 Feb 2024 10:57:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZm-0002wU-6T
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:46 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8de5d924-c414-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:51:45 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 558361F8BA;
 Mon,  5 Feb 2024 10:51:45 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 19C2A132DD;
 Mon,  5 Feb 2024 10:51:45 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id pOnnBMG9wGWnNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8de5d924-c414-11ee-8a45-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Julien Grall <jgrall@amazon.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v4 18/32] tools/xenstored: rename xenbus_evtchn()
Date: Mon,  5 Feb 2024 11:49:47 +0100
Message-Id: <20240205105001.24171-19-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 558361F8BA
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Rename the xenbus_evtchn() function to get_xenbus_evtchn() in order to
avoid two externally visible symbols with the same name when Xenstore-
stubdom is being built with a Mini-OS with CONFIG_XENBUS set.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 tools/xenstored/core.h   | 2 +-
 tools/xenstored/domain.c | 2 +-
 tools/xenstored/minios.c | 2 +-
 tools/xenstored/posix.c  | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index ad87199042..480b0f5f7b 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -383,7 +383,7 @@ static inline bool domain_is_unprivileged(const struct connection *conn)
 }
 
 /* Return the event channel used by xenbus. */
-evtchn_port_t xenbus_evtchn(void);
+evtchn_port_t get_xenbus_evtchn(void);
 
 /* Write out the pidfile */
 void write_pidfile(const char *pidfile);
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index 409b53acf9..6ef136e01f 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -1208,7 +1208,7 @@ void dom0_init(void)
 	evtchn_port_t port;
 	struct domain *dom0;
 
-	port = xenbus_evtchn();
+	port = get_xenbus_evtchn();
 	if (port == -1)
 		barf_perror("Failed to initialize dom0 port");
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index b5c3a205e6..0779efbf91 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -38,7 +38,7 @@ void init_pipe(int reopen_log_pipe[2])
 	reopen_log_pipe[1] = -1;
 }
 
-evtchn_port_t xenbus_evtchn(void)
+evtchn_port_t get_xenbus_evtchn(void)
 {
 	return dom0_event;
 }
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 6ac45fdb45..7e03dd982d 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -111,7 +111,7 @@ void unmap_xenbus(void *interface)
 	munmap(interface, getpagesize());
 }
 
-evtchn_port_t xenbus_evtchn(void)
+evtchn_port_t get_xenbus_evtchn(void)
 {
 	int fd;
 	int rc;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:57:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:57:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675847.1051493 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfi-0005GH-S6; Mon, 05 Feb 2024 10:57:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675847.1051493; Mon, 05 Feb 2024 10:57:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfi-0005Fk-On; Mon, 05 Feb 2024 10:57:54 +0000
Received: by outflank-mailman (input) for mailman id 675847;
 Mon, 05 Feb 2024 10:57:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZP-0002wU-ON
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:23 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 80824b0b-c414-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:51:23 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C6FFC1F8C0;
 Mon,  5 Feb 2024 10:51:22 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 99820132DD;
 Mon,  5 Feb 2024 10:51:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id PxkaJKq9wGVjNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 80824b0b-c414-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130282; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zG6E8AnOVGoVetT7WwKYTZr00RdCVDPWEd53rIfPGH8=;
	b=UU62A3+MXUu2WBMoDgVQoEf4iO6lCkjABHyh+SpKYyuMWTpGm2Nkb9suU+HVxnAu3kQd/K
	e87se/bad9W6phvLymD3akQwZk+ppyxLQMMQDUFNtp2H5FwWrrbvaJORaDXXwUrP4s+LQ/
	hxc1ooYCFK2lbZEtZgQne0g7JxRgEjM=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130282; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zG6E8AnOVGoVetT7WwKYTZr00RdCVDPWEd53rIfPGH8=;
	b=UU62A3+MXUu2WBMoDgVQoEf4iO6lCkjABHyh+SpKYyuMWTpGm2Nkb9suU+HVxnAu3kQd/K
	e87se/bad9W6phvLymD3akQwZk+ppyxLQMMQDUFNtp2H5FwWrrbvaJORaDXXwUrP4s+LQ/
	hxc1ooYCFK2lbZEtZgQne0g7JxRgEjM=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v4 14/32] tools/xen-9pfsd: add 9pfs read request support
Date: Mon,  5 Feb 2024 11:49:43 +0100
Message-Id: <20240205105001.24171-15-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [0.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: 0.90

Add the read request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- make error check more readable (Jason Andryuk)
V4:
- add directory read support
---
 tools/xen-9pfsd/io.c | 90 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 90 insertions(+)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index b763e3d8d9..cfbf3bef59 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -33,6 +33,7 @@
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
+#define P9_CMD_READ       116
 #define P9_CMD_WRITE      118
 #define P9_CMD_CLUNK      120
 #define P9_CMD_STAT       124
@@ -1247,6 +1248,91 @@ static void p9_stat(struct ring *ring, struct p9_header *hdr)
     free_fid(device, fidp);
 }
 
+static void p9_read(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint64_t off;
+    unsigned int len;
+    uint32_t count;
+    void *buf;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "ULU", &fid, &off, &count);
+    if ( ret != 3 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || !fidp->opened )
+    {
+        errno = EBADF;
+        goto err;
+    }
+
+    len = count;
+    buf = ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
+
+    if ( fidp->isdir )
+    {
+        struct dirent *dirent;
+        struct stat st;
+        struct p9_stat p9s;
+
+        while ( len != 0 )
+        {
+            errno = 0;
+            dirent = readdir(fidp->data);
+            if ( !dirent )
+            {
+                if ( errno )
+                    goto err;
+                break;
+            }
+            if ( fstatat(fidp->fd, dirent->d_name, &st, 0) < 0 )
+                goto err;
+            fill_p9_stat(device, &p9s, &st, dirent->d_name);
+            if ( p9s.size + sizeof(p9s.size) > len )
+            {
+                seekdir(fidp->data, dirent->d_off);
+                break;
+            }
+            fill_buffer_at(&buf, "s", &p9s);
+            len -= p9s.size + sizeof(p9s.size);
+        }
+    }
+    else
+    {
+        while ( len != 0 )
+        {
+            ret = pread(fidp->fd, buf, len, off);
+            if ( ret <= 0 )
+                break;
+            len -= ret;
+            buf += ret;
+            off += ret;
+        }
+        if ( ret < 0 && len == count )
+            goto err;
+    }
+
+    buf = ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
+    len = count - len;
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "D", &len, buf);
+
+ out:
+    free_fid(device, fidp);
+
+    return;
+
+ err:
+    p9_error(ring, hdr->tag, errno);
+    goto out;
+}
+
 static void p9_write(struct ring *ring, struct p9_header *hdr)
 {
     device *device = ring->device;
@@ -1371,6 +1457,10 @@ void *io_thread(void *arg)
                 p9_create(ring, &hdr);
                 break;
 
+            case P9_CMD_READ:
+                p9_read(ring, &hdr);
+                break;
+
             case P9_CMD_WRITE:
                 p9_write(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:57:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:57:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675849.1051503 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfm-0005mH-46; Mon, 05 Feb 2024 10:57:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675849.1051503; Mon, 05 Feb 2024 10:57:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfm-0005m8-0U; Mon, 05 Feb 2024 10:57:58 +0000
Received: by outflank-mailman (input) for mailman id 675849;
 Mon, 05 Feb 2024 10:57:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZ4-0000qL-Gw
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:02 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 73357ec5-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:51:00 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 6190A1F8BB;
 Mon,  5 Feb 2024 10:51:00 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 38566132DD;
 Mon,  5 Feb 2024 10:51:00 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 2E1cDJS9wGVFNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73357ec5-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130260; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pkY/yeFHjo5eL439i+/3VdOn4jDMqN4bEBUDTVhVOHY=;
	b=ADQmRxO77rZcn29jNH9MAIPkKXIBysVSKul248LPHHNK38xBqqAC0Y/wCs273iIaXJ5PR/
	1ybt1oSdufVIt2+RsQMDVmr152hMaYaf4tPBmrj2Kq8mRCzgUguHI729EedFN9Bcj65tU7
	JF3joxZEHPVL4+6Vohcmm55XdCHjrJ0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130260; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pkY/yeFHjo5eL439i+/3VdOn4jDMqN4bEBUDTVhVOHY=;
	b=ADQmRxO77rZcn29jNH9MAIPkKXIBysVSKul248LPHHNK38xBqqAC0Y/wCs273iIaXJ5PR/
	1ybt1oSdufVIt2+RsQMDVmr152hMaYaf4tPBmrj2Kq8mRCzgUguHI729EedFN9Bcj65tU7
	JF3joxZEHPVL4+6Vohcmm55XdCHjrJ0=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4 10/32] tools/xen-9pfsd: add 9pfs clunk request support
Date: Mon,  5 Feb 2024 11:49:39 +0100
Message-Id: <20240205105001.24171-11-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -0.30
X-Spamd-Result: default: False [-0.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 TO_DN_SOME(0.00)[];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[12.82%]
X-Spam-Flag: NO

Add the clunk request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V3:
- use unlinkat() (Jason Andryuk)
---
 tools/xen-9pfsd/io.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index 2e2275e8ab..74efd21c10 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -32,6 +32,7 @@
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
+#define P9_CMD_CLUNK      120
 
 /* P9 protocol open flags. */
 #define P9_OREAD            0   /* read */
@@ -962,6 +963,44 @@ static void p9_open(struct ring *ring, struct p9_header *hdr)
     p9_error(ring, hdr->tag, errno);
 }
 
+static void p9_clunk(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "U", &fid);
+    if ( ret != 1 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( fidp->opened )
+    {
+        fidp->opened = false;
+        free_fid(device, fidp);
+        close(fidp->fd);
+        if ( fidp->mode & P9_OREMOVE )
+            unlinkat(device->root_fd, fidp->path,
+                     fidp->isdir ? AT_REMOVEDIR : 0);
+    }
+
+    /* 2 calls of free_fid(): one for our reference, and one to free it. */
+    free_fid(device, fidp);
+    free_fid(device, fidp);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "");
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1033,6 +1072,10 @@ void *io_thread(void *arg)
                 p9_open(ring, &hdr);
                 break;
 
+            case P9_CMD_CLUNK:
+                p9_clunk(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:58:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:58:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675851.1051513 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfp-0006CN-Bz; Mon, 05 Feb 2024 10:58:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675851.1051513; Mon, 05 Feb 2024 10:58:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfp-0006C1-7g; Mon, 05 Feb 2024 10:58:01 +0000
Received: by outflank-mailman (input) for mailman id 675851;
 Mon, 05 Feb 2024 10:58:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZs-0000qL-Nd
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:52 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 913c5fb1-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:51:51 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id E7BD61F8BB;
 Mon,  5 Feb 2024 10:51:50 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B22E3132DD;
 Mon,  5 Feb 2024 10:51:50 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id Mk79Kca9wGWvNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 913c5fb1-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v4 19/32] stubdom: extend xenstore stubdom configs
Date: Mon,  5 Feb 2024 11:49:48 +0100
Message-Id: <20240205105001.24171-20-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: E7BD61F8BB
X-Spam-Flag: NO

Extend the config files of the Xenstore stubdoms to include XENBUS
and 9PFRONT items in order to support file based logging.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 stubdom/xenstore-minios.cfg    | 2 +-
 stubdom/xenstorepvh-minios.cfg | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/stubdom/xenstore-minios.cfg b/stubdom/xenstore-minios.cfg
index a41704bb6b..239da519b9 100644
--- a/stubdom/xenstore-minios.cfg
+++ b/stubdom/xenstore-minios.cfg
@@ -3,7 +3,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
diff --git a/stubdom/xenstorepvh-minios.cfg b/stubdom/xenstorepvh-minios.cfg
index 6af51f5753..752b90d7d3 100644
--- a/stubdom/xenstorepvh-minios.cfg
+++ b/stubdom/xenstorepvh-minios.cfg
@@ -4,7 +4,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:58:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:58:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675857.1051523 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfs-0006gk-N1; Mon, 05 Feb 2024 10:58:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675857.1051523; Mon, 05 Feb 2024 10:58:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfs-0006gW-JG; Mon, 05 Feb 2024 10:58:04 +0000
Received: by outflank-mailman (input) for mailman id 675857;
 Mon, 05 Feb 2024 10:58:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZh-0000qL-GV
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:41 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8a8666d6-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:51:39 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id A9D531F8BD;
 Mon,  5 Feb 2024 10:51:39 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 73C17132DD;
 Mon,  5 Feb 2024 10:51:39 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id QSfEGru9wGWeNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a8666d6-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v4 17/32] tools/helpers: allocate xenstore event channel for xenstore stubdom
Date: Mon,  5 Feb 2024 11:49:46 +0100
Message-Id: <20240205105001.24171-18-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: A9D531F8BD
X-Spam-Flag: NO

In order to prepare support of PV frontends in xenstore-stubdom, add
allocation of a Xenstore event channel to init-xenstore-domain.c.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 tools/helpers/init-xenstore-domain.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/helpers/init-xenstore-domain.c b/tools/helpers/init-xenstore-domain.c
index b2d5df8ba5..140ed610ae 100644
--- a/tools/helpers/init-xenstore-domain.c
+++ b/tools/helpers/init-xenstore-domain.c
@@ -248,6 +248,13 @@ static int build(xc_interface *xch)
     dom->cmdline = xc_dom_strdup(dom, cmdline);
     dom->xenstore_domid = domid;
     dom->console_evtchn = console_evtchn;
+    rv = xc_evtchn_alloc_unbound(xch, domid, domid);
+    if ( rv < 0 )
+    {
+        fprintf(stderr, "xc_evtchn_alloc_unbound failed\n");
+        goto err;
+    }
+    dom->xenstore_evtchn = rv;
 
     rv = xc_dom_mem_init(dom, memory);
     if ( rv )
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:58:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:58:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675860.1051538 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfw-0007DE-DP; Mon, 05 Feb 2024 10:58:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675860.1051538; Mon, 05 Feb 2024 10:58:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfw-0007Bq-6V; Mon, 05 Feb 2024 10:58:08 +0000
Received: by outflank-mailman (input) for mailman id 675860;
 Mon, 05 Feb 2024 10:58:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZA-0000qL-5V
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:08 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7686bdb6-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:51:06 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 097FF22288;
 Mon,  5 Feb 2024 10:51:06 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id CEB86132DD;
 Mon,  5 Feb 2024 10:51:05 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id ZqK1MJm9wGVMNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7686bdb6-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130266; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=TiXUk4IRdN7gv++KjZZd3JZjo0LQdKQrTPv9GcMVW4c=;
	b=FQVch8aIWj1QLZXmLTqR9kRhfv7MfWttqidcAhomHJWNdoZql0vPUgSPp01jnkoeHovpLq
	8+/qdAPGYfOWY9xKWIHcyrhlUrj1ny4kfM9tNaYY5nKYDxFqojh7eEC09VuKXIFUNMYOWd
	2IhC1N50egyMa7Q/6MBUP62W47vIesM=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130266; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=TiXUk4IRdN7gv++KjZZd3JZjo0LQdKQrTPv9GcMVW4c=;
	b=FQVch8aIWj1QLZXmLTqR9kRhfv7MfWttqidcAhomHJWNdoZql0vPUgSPp01jnkoeHovpLq
	8+/qdAPGYfOWY9xKWIHcyrhlUrj1ny4kfM9tNaYY5nKYDxFqojh7eEC09VuKXIFUNMYOWd
	2IhC1N50egyMa7Q/6MBUP62W47vIesM=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4 11/32] tools/xen-9pfsd: add 9pfs create request support
Date: Mon,  5 Feb 2024 11:49:40 +0100
Message-Id: <20240205105001.24171-12-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [0.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 TO_DN_SOME(0.00)[];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: 0.90

Add the create request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- set permissions correctly (Jason Andryuk)
V3:
- use opendirat() etc. (Jason Andryuk)
- rework error handling a little bit
---
 tools/xen-9pfsd/io.c | 151 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 151 insertions(+)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index 74efd21c10..583eea6059 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -32,6 +32,7 @@
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
+#define P9_CMD_CREATE     114
 #define P9_CMD_CLUNK      120
 
 /* P9 protocol open flags. */
@@ -42,6 +43,12 @@
 #define P9_OTRUNC        0x10   /* or'ed in, truncate file first */
 #define P9_OREMOVE       0x40   /* or'ed in, remove file after clunk */
 
+/* P9 protocol create permission masks. */
+#define P9_CREATE_PERM_DIR        0x80000000
+#define P9_CREATE_PERM_NOTSUPP    0x03b00000   /* link, symlink, ... */
+#define P9_CREATE_PERM_DIR_MASK   0777
+#define P9_CREATE_PERM_FILE_MASK  0666
+
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
 #define P9_WALK_MAXELEM   16
@@ -963,6 +970,146 @@ static void p9_open(struct ring *ring, struct p9_header *hdr)
     p9_error(ring, hdr->tag, errno);
 }
 
+static void p9_create(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    unsigned int name_off;
+    uint32_t perm;
+    uint8_t mode;
+    unsigned int ext_off;
+    struct p9_fid *fidp;
+    struct p9_fid *new_fidp;
+    char *path;
+    struct stat st;
+    struct p9_qid qid;
+    uint32_t iounit;
+    int flags;
+    int ret;
+
+    ret = fill_data(ring, "USUbS", &fid, &name_off, &perm, &mode, &ext_off);
+    if ( ret != 5 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    if ( !name_ok(ring->str + name_off) )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( perm & P9_CREATE_PERM_NOTSUPP )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || fidp->opened )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    path = malloc(strlen(fidp->path) + strlen(ring->str + name_off) + 2);
+    if ( !path )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, ENOMEM);
+        return;
+    }
+    sprintf(path, "%s/%s", fidp->path, ring->str + name_off);
+    new_fidp = alloc_fid_mem(device, fid, path);
+    free(path);
+    if ( !new_fidp )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, ENOMEM);
+        return;
+    }
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    new_fidp->ref = fidp->ref;
+
+    if ( perm & P9_CREATE_PERM_DIR )
+    {
+        perm &= P9_CREATE_PERM_DIR_MASK & st.st_mode;
+        if ( mode != P9_OREAD )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        if ( mkdirat(device->root_fd, new_fidp->path, perm) < 0 )
+            goto err;
+
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+        free(fidp);
+        fidp = new_fidp;
+        new_fidp = NULL;
+
+        fidp->fd = openat(device->root_fd, fidp->path, O_RDONLY);
+        if ( fidp->fd < 0 )
+            goto err;
+        fidp->data = fdopendir(fidp->fd);
+        if ( !fidp->data )
+            goto err;
+    }
+    else
+    {
+        flags = open_flags_from_mode(mode);
+        if ( flags < 0 )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        perm &= P9_CREATE_PERM_FILE_MASK & st.st_mode;
+
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+        free(fidp);
+        fidp = new_fidp;
+        new_fidp = NULL;
+
+        fidp->fd = openat(device->root_fd, fidp->path, flags | O_CREAT | O_EXCL,
+                          perm);
+        if ( fidp->fd < 0 )
+            goto err;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+        goto err;
+
+    fill_qid(device, fidp->path, &qid, &st);
+    iounit = get_iounit(ring, &st);
+    fidp->opened = true;
+    fidp->mode = mode;
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "QU", &qid, &iounit);
+
+    return;
+
+ err:
+    p9_error(ring, hdr->tag, errno);
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    free(new_fidp);
+    free_fid(device, fidp);
+}
+
 static void p9_clunk(struct ring *ring, struct p9_header *hdr)
 {
     device *device = ring->device;
@@ -1072,6 +1219,10 @@ void *io_thread(void *arg)
                 p9_open(ring, &hdr);
                 break;
 
+            case P9_CMD_CREATE:
+                p9_create(ring, &hdr);
+                break;
+
             case P9_CMD_CLUNK:
                 p9_clunk(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:58:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:58:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675859.1051533 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfv-000797-VQ; Mon, 05 Feb 2024 10:58:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675859.1051533; Mon, 05 Feb 2024 10:58:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwfv-00078o-RN; Mon, 05 Feb 2024 10:58:07 +0000
Received: by outflank-mailman (input) for mailman id 675859;
 Mon, 05 Feb 2024 10:58:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwaK-0002wU-St
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:52:21 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a207ee68-c414-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:52:19 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 091E31F8BA;
 Mon,  5 Feb 2024 10:52:19 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id CCE5E132DD;
 Mon,  5 Feb 2024 10:52:18 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id ITeRMOK9wGXZNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a207ee68-c414-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130339; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CIIvWmhSlUU/eQ16VgaFDL39qVtBS1MR2oAB75tbNM8=;
	b=XlSAier3wLqUUZy224T2xQV+Q0qEsF4bUOoL/VFBIzfH1H94CJBFKUD3bdUvkjscxg9ZTT
	EqLeTnEkTyUUngiXFXdhHAtcja1exG3UROR7oo74dTytMwD5GCPNhoRjct5A34EAJL6/Rf
	w9REjz+tK+5OVlHZyH34MMIs8yg6Vjs=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130339; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CIIvWmhSlUU/eQ16VgaFDL39qVtBS1MR2oAB75tbNM8=;
	b=XlSAier3wLqUUZy224T2xQV+Q0qEsF4bUOoL/VFBIzfH1H94CJBFKUD3bdUvkjscxg9ZTT
	EqLeTnEkTyUUngiXFXdhHAtcja1exG3UROR7oo74dTytMwD5GCPNhoRjct5A34EAJL6/Rf
	w9REjz+tK+5OVlHZyH34MMIs8yg6Vjs=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4 24/32] tools/xenstored: move all socket handling into posix.c
Date: Mon,  5 Feb 2024 11:49:53 +0100
Message-Id: <20240205105001.24171-25-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-0.997];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

All of the socket handling is needed only when running as daemon.

Move it into posix.c, allowing to remove the NO_SOCKETS macro.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V3:
- new patch
V4:
- make sock local to posix.c (Julien Grall)
---
 tools/xenstored/Makefile.common |   4 -
 tools/xenstored/core.c          | 160 +------------------------------
 tools/xenstored/core.h          |   7 +-
 tools/xenstored/domain.c        |   9 +-
 tools/xenstored/minios.c        |  15 +++
 tools/xenstored/posix.c         | 164 ++++++++++++++++++++++++++++++++
 6 files changed, 188 insertions(+), 171 deletions(-)

diff --git a/tools/xenstored/Makefile.common b/tools/xenstored/Makefile.common
index 189ab81b8d..ef63ef650c 100644
--- a/tools/xenstored/Makefile.common
+++ b/tools/xenstored/Makefile.common
@@ -16,10 +16,6 @@ CFLAGS += $(CFLAGS_libxenctrl)
 CFLAGS += $(CFLAGS_libxenguest)
 CFLAGS += $(CFLAGS_libxentoolcore)
 
-ifdef CONFIG_STUBDOM
-CFLAGS += -DNO_SOCKETS=1
-endif
-
 $(XENSTORED_OBJS-y): CFLAGS += $(CFLAGS_libxengnttab)
 
 xenstored.a: $(XENSTORED_OBJS-y)
diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 86ab330eaf..c045a6e5db 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -20,10 +20,6 @@
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <poll.h>
-#ifndef NO_SOCKETS
-#include <sys/socket.h>
-#include <sys/un.h>
-#endif
 #include <sys/time.h>
 #include <time.h>
 #include <unistd.h>
@@ -61,8 +57,6 @@ static unsigned int current_array_size;
 static unsigned int nr_fds;
 static unsigned int delayed_requests;
 
-static int sock = -1;
-
 int orig_argc;
 char **orig_argv;
 
@@ -486,7 +480,7 @@ fail:
 	return -1;
 }
 
-static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout)
+static void initialize_fds(int *ptimeout)
 {
 	struct connection *conn;
 	uint64_t msecs;
@@ -499,8 +493,6 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout)
 	*ptimeout = delayed_requests ? 1000 : -1;
 
 	set_special_fds();
-	if (sock != -1)
-		*p_sock_pollfd_idx = set_fd(sock, POLLIN|POLLPRI);
 
 	if (xce_handle != NULL)
 		xce_pollfd_idx = set_fd(xenevtchn_fd(xce_handle),
@@ -2260,97 +2252,6 @@ struct connection *get_connection_by_id(unsigned int conn_id)
 	return NULL;
 }
 
-#ifdef NO_SOCKETS
-static void accept_connection(int sock)
-{
-}
-#else
-static int writefd(struct connection *conn, const void *data, unsigned int len)
-{
-	int rc;
-
-	while ((rc = write(conn->fd, data, len)) < 0) {
-		if (errno == EAGAIN) {
-			rc = 0;
-			break;
-		}
-		if (errno != EINTR)
-			break;
-	}
-
-	return rc;
-}
-
-static int readfd(struct connection *conn, void *data, unsigned int len)
-{
-	int rc;
-
-	while ((rc = read(conn->fd, data, len)) < 0) {
-		if (errno == EAGAIN) {
-			rc = 0;
-			break;
-		}
-		if (errno != EINTR)
-			break;
-	}
-
-	/* Reading zero length means we're done with this connection. */
-	if ((rc == 0) && (len != 0)) {
-		errno = EBADF;
-		rc = -1;
-	}
-
-	return rc;
-}
-
-static bool socket_can_process(struct connection *conn, int mask)
-{
-	if (conn->pollfd_idx == -1)
-		return false;
-
-	if (poll_fds[conn->pollfd_idx].revents & ~(POLLIN | POLLOUT)) {
-		talloc_free(conn);
-		return false;
-	}
-
-	return (poll_fds[conn->pollfd_idx].revents & mask);
-}
-
-static bool socket_can_write(struct connection *conn)
-{
-	return socket_can_process(conn, POLLOUT);
-}
-
-static bool socket_can_read(struct connection *conn)
-{
-	return socket_can_process(conn, POLLIN);
-}
-
-const struct interface_funcs socket_funcs = {
-	.write = writefd,
-	.read = readfd,
-	.can_write = socket_can_write,
-	.can_read = socket_can_read,
-};
-
-static void accept_connection(int sock)
-{
-	int fd;
-	struct connection *conn;
-
-	fd = accept(sock, NULL, NULL);
-	if (fd < 0)
-		return;
-
-	conn = new_connection(&socket_funcs);
-	if (conn) {
-		conn->fd = fd;
-		conn->id = dom0_domid;
-	} else
-		close(fd);
-}
-#endif
-
 /* We create initial nodes manually. */
 static void manual_node(const char *name, const char *child)
 {
@@ -2579,46 +2480,6 @@ void corrupt(struct connection *conn, const char *fmt, ...)
 	errno = saved_errno;
 }
 
-#ifndef NO_SOCKETS
-static void destroy_fds(void)
-{
-	if (sock >= 0)
-		close(sock);
-}
-
-void init_sockets(void)
-{
-	struct sockaddr_un addr;
-	const char *soc_str = xenstore_daemon_path();
-
-	if (!soc_str)
-		barf_perror("Failed to obtain xs domain socket");
-
-	/* Create sockets for them to listen to. */
-	atexit(destroy_fds);
-	sock = socket(PF_UNIX, SOCK_STREAM, 0);
-	if (sock < 0)
-		barf_perror("Could not create socket");
-
-	/* FIXME: Be more sophisticated, don't mug running daemon. */
-	unlink(soc_str);
-
-	addr.sun_family = AF_UNIX;
-
-	if(strlen(soc_str) >= sizeof(addr.sun_path))
-		barf_perror("socket string '%s' too long", soc_str);
-	strcpy(addr.sun_path, soc_str);
-	if (bind(sock, (struct sockaddr *)&addr, sizeof(addr)) != 0)
-		barf_perror("Could not bind socket to %s", soc_str);
-
-	if (chmod(soc_str, 0600) != 0)
-		barf_perror("Could not chmod sockets");
-
-	if (listen(sock, 1) != 0)
-		barf_perror("Could not listen on sockets");
-}
-#endif
-
 static void usage(void)
 {
 	fprintf(stderr,
@@ -2796,7 +2657,6 @@ int set_trace_switch(const char *arg)
 int main(int argc, char *argv[])
 {
 	int opt;
-	int sock_pollfd_idx = -1;
 	bool dofork = true;
 	bool live_update = false;
 	const char *pidfile = NULL;
@@ -2907,7 +2767,7 @@ int main(int argc, char *argv[])
 	check_store();
 
 	/* Get ready to listen to the tools. */
-	initialize_fds(&sock_pollfd_idx, &timeout);
+	initialize_fds(&timeout);
 
 	late_init(live_update);
 
@@ -2923,16 +2783,6 @@ int main(int argc, char *argv[])
 
 		handle_special_fds();
 
-		if (sock_pollfd_idx != -1) {
-			if (poll_fds[sock_pollfd_idx].revents & ~POLLIN) {
-				barf_perror("sock poll failed");
-				break;
-			} else if (poll_fds[sock_pollfd_idx].revents & POLLIN) {
-				accept_connection(sock);
-				sock_pollfd_idx = -1;
-			}
-		}
-
 		if (xce_pollfd_idx != -1) {
 			if (poll_fds[xce_pollfd_idx].revents & ~POLLIN) {
 				barf_perror("xce_handle poll failed");
@@ -2986,7 +2836,7 @@ int main(int argc, char *argv[])
 			}
 		}
 
-		initialize_fds(&sock_pollfd_idx, &timeout);
+		initialize_fds(&timeout);
 	}
 }
 
@@ -2999,7 +2849,7 @@ const char *dump_state_global(FILE *fp)
 	head.length = sizeof(glb);
 	if (fwrite(&head, sizeof(head), 1, fp) != 1)
 		return "Dump global state error";
-	glb.socket_fd = sock;
+	glb.socket_fd = get_socket_fd();
 	glb.evtchn_fd = xenevtchn_fd(xce_handle);
 	if (fwrite(&glb, sizeof(glb), 1, fp) != 1)
 		return "Dump global state error";
@@ -3235,7 +3085,7 @@ void read_state_global(const void *ctx, const void *state)
 {
 	const struct xs_state_global *glb = state;
 
-	sock = glb->socket_fd;
+	set_socket_fd(glb->socket_fd);
 
 	domain_init(glb->evtchn_fd);
 }
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index 44c4d0f8b8..6180ec8b5c 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -301,6 +301,7 @@ int rm_node(struct connection *conn, const void *ctx, const char *name);
 
 void setup_structure(bool live_update);
 struct connection *new_connection(const struct interface_funcs *funcs);
+struct connection *add_socket_connection(int fd);
 struct connection *get_connection_by_id(unsigned int conn_id);
 void check_store(void);
 void corrupt(struct connection *conn, const char *fmt, ...);
@@ -394,14 +395,12 @@ int set_fd(int fd, short events);
 void set_special_fds(void);
 void handle_special_fds(void);
 
-void init_sockets(void);
+int get_socket_fd(void);
+void set_socket_fd(int fd);
 
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
-#ifndef NO_SOCKETS
-extern const struct interface_funcs socket_funcs;
-#endif
 extern xengnttab_handle **xgt_handle;
 
 int remember_string(struct hashtable *hash, const char *str);
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index 6ef136e01f..040df98b8c 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -1739,14 +1739,7 @@ void read_state_connection(const void *ctx, const void *state)
 	struct domain *domain, *tdomain;
 
 	if (sc->conn_type == XS_STATE_CONN_TYPE_SOCKET) {
-#ifdef NO_SOCKETS
-		barf("socket based connection without sockets");
-#else
-		conn = new_connection(&socket_funcs);
-		if (!conn)
-			barf("error restoring connection");
-		conn->fd = sc->spec.socket_fd;
-#endif
+		conn = add_socket_connection(sc->spec.socket_fd);
 	} else {
 		domain = introduce_domain(ctx, sc->spec.ring.domid,
 					  sc->spec.ring.evtchn, true);
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 3208169187..6d0c11423c 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -18,12 +18,18 @@
 #include <sys/types.h>
 #include <sys/mman.h>
 #include "core.h"
+#include "utils.h"
 #include <xen/grant_table.h>
 
 void finish_daemonize(void)
 {
 }
 
+struct connection *add_socket_connection(int fd)
+{
+	barf("socket based connection without sockets");
+}
+
 evtchn_port_t get_xenbus_evtchn(void)
 {
 	return dom0_event;
@@ -55,3 +61,12 @@ void set_special_fds(void)
 void handle_special_fds(void)
 {
 }
+
+int get_socket_fd(void)
+{
+	return -1;
+}
+
+void set_socket_fd(int fd)
+{
+}
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 1ea8475293..496329dfd1 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -24,6 +24,8 @@
 #include <stdlib.h>
 #include <syslog.h>
 #include <sys/mman.h>
+#include <sys/socket.h>
+#include <sys/un.h>
 #if defined(HAVE_SYSTEMD)
 #include <systemd/sd-daemon.h>
 #endif
@@ -32,10 +34,14 @@
 #include "utils.h"
 #include "core.h"
 #include "osdep.h"
+#include "talloc.h"
 
 static int reopen_log_pipe0_pollfd_idx = -1;
 static int reopen_log_pipe[2];
 
+static int sock_pollfd_idx = -1;
+static int sock = -1;
+
 static void write_pidfile(const char *pidfile)
 {
 	char buf[100];
@@ -180,6 +186,142 @@ void *xenbus_map(void)
 	return addr;
 }
 
+static int writefd(struct connection *conn, const void *data, unsigned int len)
+{
+	int rc;
+
+	while ((rc = write(conn->fd, data, len)) < 0) {
+		if (errno == EAGAIN) {
+			rc = 0;
+			break;
+		}
+		if (errno != EINTR)
+			break;
+	}
+
+	return rc;
+}
+
+static int readfd(struct connection *conn, void *data, unsigned int len)
+{
+	int rc;
+
+	while ((rc = read(conn->fd, data, len)) < 0) {
+		if (errno == EAGAIN) {
+			rc = 0;
+			break;
+		}
+		if (errno != EINTR)
+			break;
+	}
+
+	/* Reading zero length means we're done with this connection. */
+	if ((rc == 0) && (len != 0)) {
+		errno = EBADF;
+		rc = -1;
+	}
+
+	return rc;
+}
+
+static bool socket_can_process(struct connection *conn, int mask)
+{
+	if (conn->pollfd_idx == -1)
+		return false;
+
+	if (poll_fds[conn->pollfd_idx].revents & ~(POLLIN | POLLOUT)) {
+		talloc_free(conn);
+		return false;
+	}
+
+	return (poll_fds[conn->pollfd_idx].revents & mask);
+}
+
+static bool socket_can_write(struct connection *conn)
+{
+	return socket_can_process(conn, POLLOUT);
+}
+
+static bool socket_can_read(struct connection *conn)
+{
+	return socket_can_process(conn, POLLIN);
+}
+
+static const struct interface_funcs socket_funcs = {
+	.write = writefd,
+	.read = readfd,
+	.can_write = socket_can_write,
+	.can_read = socket_can_read,
+};
+
+static void accept_connection(int sock)
+{
+	int fd;
+	struct connection *conn;
+
+	fd = accept(sock, NULL, NULL);
+	if (fd < 0)
+		return;
+
+	conn = new_connection(&socket_funcs);
+	if (conn) {
+		conn->fd = fd;
+		conn->id = dom0_domid;
+	} else
+		close(fd);
+}
+
+static void destroy_fds(void)
+{
+	if (sock >= 0)
+		close(sock);
+}
+
+static void init_sockets(void)
+{
+	struct sockaddr_un addr;
+	const char *soc_str = xenstore_daemon_path();
+
+	if (!soc_str)
+		barf_perror("Failed to obtain xs domain socket");
+
+	/* Create sockets for them to listen to. */
+	atexit(destroy_fds);
+	sock = socket(PF_UNIX, SOCK_STREAM, 0);
+	if (sock < 0)
+		barf_perror("Could not create socket");
+
+	/* FIXME: Be more sophisticated, don't mug running daemon. */
+	unlink(soc_str);
+
+	addr.sun_family = AF_UNIX;
+
+	if (strlen(soc_str) >= sizeof(addr.sun_path))
+		barf_perror("socket string '%s' too long", soc_str);
+	strcpy(addr.sun_path, soc_str);
+	if (bind(sock, (struct sockaddr *)&addr, sizeof(addr)) != 0)
+		barf_perror("Could not bind socket to %s", soc_str);
+
+	if (chmod(soc_str, 0600) != 0)
+		barf_perror("Could not chmod sockets");
+
+	if (listen(sock, 1) != 0)
+		barf_perror("Could not listen on sockets");
+}
+
+
+struct connection *add_socket_connection(int fd)
+{
+	struct connection *conn;
+
+	conn = new_connection(&socket_funcs);
+	if (!conn)
+		barf("error restoring connection");
+	conn->fd = fd;
+
+	return conn;
+}
+
 void early_init(bool live_update, bool dofork, const char *pidfile)
 {
 	reopen_log();
@@ -212,6 +354,9 @@ void set_special_fds(void)
 	if (reopen_log_pipe[0] != -1)
 		reopen_log_pipe0_pollfd_idx =
 			set_fd(reopen_log_pipe[0], POLLIN|POLLPRI);
+
+	if (sock != -1)
+		sock_pollfd_idx = set_fd(sock, POLLIN|POLLPRI);
 }
 
 void handle_special_fds(void)
@@ -231,6 +376,15 @@ void handle_special_fds(void)
 		}
 		reopen_log_pipe0_pollfd_idx = -1;
 	}
+
+	if (sock_pollfd_idx != -1) {
+		if (poll_fds[sock_pollfd_idx].revents & ~POLLIN) {
+			barf_perror("sock poll failed");
+		} else if (poll_fds[sock_pollfd_idx].revents & POLLIN) {
+			accept_connection(sock);
+			sock_pollfd_idx = -1;
+		}
+	}
 }
 
 void late_init(bool live_update)
@@ -242,3 +396,13 @@ void late_init(bool live_update)
 	}
 #endif
 }
+
+int get_socket_fd(void)
+{
+	return sock;
+}
+
+void set_socket_fd(int fd)
+{
+	sock = fd;
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:58:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:58:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675882.1051553 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwgW-00013O-RJ; Mon, 05 Feb 2024 10:58:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675882.1051553; Mon, 05 Feb 2024 10:58:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwgW-00013H-Nz; Mon, 05 Feb 2024 10:58:44 +0000
Received: by outflank-mailman (input) for mailman id 675882;
 Mon, 05 Feb 2024 10:58:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwYz-0000qL-8U
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:50:57 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6fdc9fc0-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:50:55 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id CE8E01F8BA;
 Mon,  5 Feb 2024 10:50:54 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A4FAC132DD;
 Mon,  5 Feb 2024 10:50:54 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id y93GJo69wGVANgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:50:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fdc9fc0-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130254; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=FTPfVWkUlQhndtMJjvkYVvwFLg3Kri0CAO7AyEi1mRk=;
	b=nA93Rx9YrZr+hop2VC7Ain11fxephLZyrBCuGFLkClL6chC4gvPYH6mTMK1PmSUHfrSRi9
	sbz39q7p1lPl91m3o5YHxBxMrUxL7UUx1++H45cSjvKNndTg22FMEHnMfDCvc74uhqD6ma
	TYYXWXKjt8cFw4AHI4qYp555QtfjJhg=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130254; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=FTPfVWkUlQhndtMJjvkYVvwFLg3Kri0CAO7AyEi1mRk=;
	b=nA93Rx9YrZr+hop2VC7Ain11fxephLZyrBCuGFLkClL6chC4gvPYH6mTMK1PmSUHfrSRi9
	sbz39q7p1lPl91m3o5YHxBxMrUxL7UUx1++H45cSjvKNndTg22FMEHnMfDCvc74uhqD6ma
	TYYXWXKjt8cFw4AHI4qYp555QtfjJhg=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4 09/32] tools/xen-9pfsd: add 9pfs open request support
Date: Mon,  5 Feb 2024 11:49:38 +0100
Message-Id: <20240205105001.24171-10-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 TO_DN_SOME(0.00)[];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add the open request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- don't allow to open symbolic link
V3:
- use openat() (Jason Andryuk)
- use common error handling in p9_open()
---
 tools/xen-9pfsd/io.c        | 137 ++++++++++++++++++++++++++++++++++++
 tools/xen-9pfsd/xen-9pfsd.h |   4 ++
 2 files changed, 141 insertions(+)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index 5b827706b7..2e2275e8ab 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -19,6 +19,8 @@
 #include <syslog.h>
 #include <sys/types.h>
 #include <sys/stat.h>
+#include <dirent.h>
+#include <fcntl.h>
 #include <xenctrl.h>           /* For cpu barriers. */
 #include <xen-tools/common-macros.h>
 
@@ -29,6 +31,15 @@
 #define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
+#define P9_CMD_OPEN       112
+
+/* P9 protocol open flags. */
+#define P9_OREAD            0   /* read */
+#define P9_OWRITE           1   /* write */
+#define P9_ORDWR            2   /* read and write */
+#define P9_OMODEMASK     0x03
+#define P9_OTRUNC        0x10   /* or'ed in, truncate file first */
+#define P9_OREMOVE       0x40   /* or'ed in, remove file after clunk */
 
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
@@ -829,6 +840,128 @@ static void p9_walk(struct ring *ring, struct p9_header *hdr)
     free(names);
 }
 
+static int open_flags_from_mode(uint8_t mode)
+{
+    int flags;
+
+    switch ( mode & P9_OMODEMASK )
+    {
+    case P9_OREAD:
+        flags = O_RDONLY;
+        break;
+
+    case P9_OWRITE:
+        flags = O_WRONLY;
+        break;
+
+    case P9_ORDWR:
+        flags = O_RDWR;
+        break;
+
+    default:
+        errno = EINVAL;
+        return -1;
+    }
+
+    if ( mode & P9_OTRUNC )
+        flags |= O_TRUNC;
+
+    return flags;
+}
+
+static unsigned int get_iounit(struct ring *ring, struct stat *st)
+{
+    return (ring->max_size - st->st_blksize) & ~(st->st_blksize - 1);
+}
+
+static void p9_open(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint8_t mode;
+    struct p9_fid *fidp;
+    struct stat st;
+    struct p9_qid qid;
+    uint32_t iounit;
+    int flags;
+    int ret;
+
+    ret = fill_data(ring, "Ub", &fid, &mode);
+    if ( ret != 2 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+    if ( mode & ~(P9_OMODEMASK | P9_OTRUNC | P9_OREMOVE) )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+    if ( fidp->opened )
+    {
+        errno = EINVAL;
+        goto err;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        errno = ENOENT;
+        goto err;
+    }
+
+    if ( S_ISLNK(st.st_mode) )
+    {
+        errno = EMLINK;
+        goto err;
+    }
+
+    fidp->isdir = S_ISDIR(st.st_mode);
+    fidp->mode = mode;
+    if ( fidp->isdir )
+    {
+        if ( mode != P9_OREAD )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        fidp->fd = openat(device->root_fd, fidp->path, O_RDONLY);
+        if ( fidp->fd < 0 )
+            goto err;
+        fidp->data = fdopendir(fidp->fd);
+        if ( !fidp->data )
+            goto err;
+    }
+    else
+    {
+        flags = open_flags_from_mode(mode);
+        if ( flags < 0 )
+            goto err;
+
+        fidp->fd = openat(device->root_fd, fidp->path, flags);
+        if ( fidp->fd < 0 )
+            goto err;
+    }
+
+    fill_qid(device, fidp->path, &qid, &st);
+    iounit = get_iounit(ring, &st);
+    fidp->opened = true;
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "QU", &qid, &iounit);
+
+    return;
+
+ err:
+    free_fid(device, fidp);
+    p9_error(ring, hdr->tag, errno);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -896,6 +1029,10 @@ void *io_thread(void *arg)
                 p9_walk(ring, &hdr);
                 break;
 
+            case P9_CMD_OPEN:
+                p9_open(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/xen-9pfsd/xen-9pfsd.h b/tools/xen-9pfsd/xen-9pfsd.h
index f01fffb0bb..757be2da4b 100644
--- a/tools/xen-9pfsd/xen-9pfsd.h
+++ b/tools/xen-9pfsd/xen-9pfsd.h
@@ -25,7 +25,11 @@ struct p9_fid {
     XEN_TAILQ_ENTRY(struct p9_fid) list;
     unsigned int fid;
     unsigned int ref;
+    int fd;
+    uint8_t mode;
     bool opened;
+    bool isdir;
+    void *data;    /* File type specific. */
     char path[];
 };
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:58:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:58:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675883.1051563 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwgY-0001K4-33; Mon, 05 Feb 2024 10:58:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675883.1051563; Mon, 05 Feb 2024 10:58:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwgX-0001Jx-WA; Mon, 05 Feb 2024 10:58:45 +0000
Received: by outflank-mailman (input) for mailman id 675883;
 Mon, 05 Feb 2024 10:58:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZy-0000qL-Ft
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:58 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 94923e2f-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:51:56 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 801021F8BA;
 Mon,  5 Feb 2024 10:51:56 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4E3C9132DD;
 Mon,  5 Feb 2024 10:51:56 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id +NmVEcy9wGW4NgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 94923e2f-c414-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4 20/32] tools: add 9pfs device to xenstore-stubdom
Date: Mon,  5 Feb 2024 11:49:49 +0100
Message-Id: <20240205105001.24171-21-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 801021F8BA
X-Spam-Flag: NO

Add a 9pfs device to Xenstore stubdom in order to allow it to do e.g.
logging into a dom0 file.

Use the following parameters for the new device:

- tag = "Xen"
- type = "xen_9pfsd"
- path = "/var/lib/xen/xenstore"
- security-model = "none"

For now don't limit allowed file space or number of files.

Add a new libxl function for adding it similar to the function for
adding the console device.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- add security_model parameter to new libxl function (Jason Andryuk)
V4:
- rename function to libxl_device_9pfs_add() (Anthony Perard)
- use a libxl_device_p9 pointer as parameter (Anthony Perard)
---
 tools/helpers/init-xenstore-domain.c |  7 +++++++
 tools/include/libxl.h                | 15 +++++++++++++++
 tools/libs/light/libxl_9pfs.c        | 16 ++++++++++++++++
 3 files changed, 38 insertions(+)

diff --git a/tools/helpers/init-xenstore-domain.c b/tools/helpers/init-xenstore-domain.c
index 140ed610ae..1683438c5c 100644
--- a/tools/helpers/init-xenstore-domain.c
+++ b/tools/helpers/init-xenstore-domain.c
@@ -433,6 +433,12 @@ int main(int argc, char** argv)
     int rv, fd;
     char *maxmem_str = NULL;
     libxl_ctx *ctx;
+    libxl_device_p9 p9 = { .backend_domid = 0,
+                           .tag = "Xen",
+                           .path = XEN_LIB_DIR"/xenstore",
+                           .security_model = "none",
+                           .type = LIBXL_P9_TYPE_XEN_9PFSD,
+    };
 
     while ( (opt = getopt_long(argc, argv, "v", options, NULL)) != -1 )
     {
@@ -543,6 +549,7 @@ int main(int argc, char** argv)
     }
     libxl_console_add_xenstore(ctx, domid, 0, console_evtchn, console_gfn,
                                NULL);
+    libxl_device_9pfs_add(ctx, domid, &p9, NULL);
     libxl_ctx_free(ctx);
 
     fd = creat(XEN_RUN_DIR "/xenstored.pid", 0666);
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index a554f2ccd6..e6ab5ddb94 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -583,6 +583,13 @@
  * libxl_console_add_xenstore() in libxl.
  */
 #define LIBXL_HAVE_CONSOLE_ADD_XENSTORE 1
+
+/*
+ * LIBXL_HAVE_P9_ADD_XENSTORE indicates presence of the function
+ * libxl_device_9pfs_add() in libxl.
+ */
+#define LIBXL_HAVE_P9_ADD_XENSTORE 1
+
 /*
  * libxl ABI compatibility
  *
@@ -2074,6 +2081,14 @@ int libxl_console_add_xenstore(libxl_ctx *ctx, uint32_t domid, uint32_t backend,
                                const libxl_asyncop_how *ao_how)
                                LIBXL_EXTERNAL_CALLERS_ONLY;
 
+/* libxl_device_9pfs_add writes the Xenstore entries for a domain's
+ * primary 9pfs device based on domid, and device parameters.
+ * If needed it will start the backend daemon.
+ */
+int libxl_device_9pfs_add(libxl_ctx *ctx, uint32_t domid, libxl_device_p9 *p9,
+                          const libxl_asyncop_how *ao_how)
+                          LIBXL_EXTERNAL_CALLERS_ONLY;
+
 /* May be called with info_r == NULL to check for domain's existence.
  * Returns ERROR_DOMAIN_NOTFOUND if domain does not exist (used to return
  * ERROR_INVAL for this scenario). */
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 7593686a90..8f5d475322 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -208,6 +208,22 @@ static void libxl__device_p9_add(libxl__egc *egc, uint32_t domid,
         aodev->callback(egc, aodev);
 }
 
+int libxl_device_9pfs_add(libxl_ctx *ctx, uint32_t domid, libxl_device_p9 *p9,
+                          const libxl_asyncop_how *ao_how)
+{
+    AO_CREATE(ctx, domid, ao_how);
+    libxl__ao_device *aodev;
+
+    GCNEW(aodev);
+    libxl__prepare_ao_device(ao, aodev);
+    aodev->action = LIBXL__DEVICE_ACTION_ADD;
+    aodev->callback = device_addrm_aocomplete;
+
+    libxl__device_p9_add(egc, domid, p9, aodev);
+
+    return AO_INPROGRESS;
+}
+
 #define libxl_device_p9_list NULL
 #define libxl_device_p9_compare NULL
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:59:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:59:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675893.1051573 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwgt-0002Ju-Aa; Mon, 05 Feb 2024 10:59:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675893.1051573; Mon, 05 Feb 2024 10:59:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwgt-0002Jl-7u; Mon, 05 Feb 2024 10:59:07 +0000
Received: by outflank-mailman (input) for mailman id 675893;
 Mon, 05 Feb 2024 10:59:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwar-0002wU-Sm
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:52:53 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b6364be0-c414-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:52:53 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 0023422100;
 Mon,  5 Feb 2024 10:52:52 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B63F1132DD;
 Mon,  5 Feb 2024 10:52:52 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 9ar+KgS+wGUGNwAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b6364be0-c414-11ee-8a45-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v4 30/32] tools/xenstored: add helpers for filename handling
Date: Mon,  5 Feb 2024 11:49:59 +0100
Message-Id: <20240205105001.24171-31-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 0023422100
X-Spam-Flag: NO

Add some helpers for handling filenames which might need different
implementations between stubdom and daemon environments:

- expansion of relative filenames (those are not really defined today,
  just expand them to be relative to /var/lib/xen/xenstore)
- expansion of xenstore_daemon_rundir() (used e.g. for saving the state
  file in case of live update - needs to be unchanged in the daemon
  case, but should result in /var/lib/xen/xenstore for stubdom)

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- make absolute_filename() return a pointer to const (Julien Grall)
---
 tools/xenstored/core.c      | 15 +++++++++++++--
 tools/xenstored/core.h      |  5 ++++-
 tools/xenstored/lu_daemon.c |  4 ++--
 tools/xenstored/minios.c    |  5 +++++
 tools/xenstored/posix.c     |  8 +++++++-
 5 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 48fc787ac1..bada1ad9a2 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -63,7 +63,7 @@ char **orig_argv;
 LIST_HEAD(connections);
 int tracefd = -1;
 bool keep_orphans = false;
-char *tracefile = NULL;
+const char *tracefile = NULL;
 static struct hashtable *nodes;
 unsigned int trace_flags = TRACE_OBJ | TRACE_IO;
 
@@ -137,6 +137,17 @@ void trace_destroy(const void *data, const char *type)
 		trace("obj: DESTROY %s %p\n", type, data);
 }
 
+/*
+ * Return an absolute filename.
+ * In case of a relative filename given as input, prepend XENSTORE_LIB_DIR.
+ */
+const char *absolute_filename(const void *ctx, const char *filename)
+{
+	if (filename[0] != '/')
+		return talloc_asprintf(ctx, XENSTORE_LIB_DIR "/%s", filename);
+	return talloc_strdup(ctx, filename);
+}
+
 void close_log(void)
 {
 	if (tracefd >= 0)
@@ -2759,7 +2770,7 @@ int main(int argc, char *argv[])
 #endif
 
 	if (tracefile)
-		tracefile = talloc_strdup(NULL, tracefile);
+		tracefile = absolute_filename(NULL, tracefile);
 
 #ifndef NO_LIVE_UPDATE
 	/* Read state in case of live update. */
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index fe0ee90581..e58779e88c 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -341,7 +341,7 @@ void close_log(void);
 extern int orig_argc;
 extern char **orig_argv;
 
-extern char *tracefile;
+extern const char *tracefile;
 extern int tracefd;
 
 /* Trace flag values must be kept in sync with trace_switches[] contents. */
@@ -405,6 +405,9 @@ void set_socket_fd(int fd);
 void mount_9pfs(void);
 #endif
 
+const char *xenstore_rundir(void);
+const char *absolute_filename(const void *ctx, const char *filename);
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/lu_daemon.c b/tools/xenstored/lu_daemon.c
index 71bcabadd3..6351111ab0 100644
--- a/tools/xenstored/lu_daemon.c
+++ b/tools/xenstored/lu_daemon.c
@@ -24,7 +24,7 @@ void lu_get_dump_state(struct lu_dump_state *state)
 	state->size = 0;
 
 	state->filename = talloc_asprintf(NULL, "%s/state_dump",
-					  xenstore_daemon_rundir());
+					  xenstore_rundir());
 	if (!state->filename)
 		barf("Allocation failure");
 
@@ -65,7 +65,7 @@ FILE *lu_dump_open(const void *ctx)
 	int fd;
 
 	filename = talloc_asprintf(ctx, "%s/state_dump",
-				   xenstore_daemon_rundir());
+				   xenstore_rundir());
 	if (!filename)
 		return NULL;
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 562a9b4972..e70386f8c7 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -128,3 +128,8 @@ void mount_9pfs(void)
 {
 	create_thread("mount-9pfs", mount_thread, NULL);
 }
+
+const char *xenstore_rundir(void)
+{
+	return XENSTORE_LIB_DIR;
+}
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 496329dfd1..d88c82d972 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -326,9 +326,10 @@ void early_init(bool live_update, bool dofork, const char *pidfile)
 {
 	reopen_log();
 
-	/* Make sure xenstored directory exists. */
+	/* Make sure xenstored directories exist. */
 	/* Errors ignored here, will be reported when we open files */
 	mkdir(xenstore_daemon_rundir(), 0755);
+	mkdir(XENSTORE_LIB_DIR, 0755);
 
 	if (dofork) {
 		openlog("xenstored", 0, LOG_DAEMON);
@@ -406,3 +407,8 @@ void set_socket_fd(int fd)
 {
 	sock = fd;
 }
+
+const char *xenstore_rundir(void)
+{
+	return xenstore_daemon_rundir();
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:59:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:59:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675900.1051583 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwhO-0003Jx-PE; Mon, 05 Feb 2024 10:59:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675900.1051583; Mon, 05 Feb 2024 10:59:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwhO-0003Jo-L5; Mon, 05 Feb 2024 10:59:38 +0000
Received: by outflank-mailman (input) for mailman id 675900;
 Mon, 05 Feb 2024 10:59:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwa8-0002wU-PT
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:52:08 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9b5b870e-c414-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:52:08 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id CFAA222100;
 Mon,  5 Feb 2024 10:52:07 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 92C56132DD;
 Mon,  5 Feb 2024 10:52:07 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 3wJdIte9wGXHNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:52:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b5b870e-c414-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130327; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Zf1h+BIq0Z8Nwwp8lu1GEpj5zMTW9hh2PPOlxbQgdSw=;
	b=OF/utg6Saod0ZXXQVz6YP8a8kQYLJUaOHsJvdE98qsRyg1rJK7pZsSihiaAanOsrHPkhXQ
	bAOhTj/u4IjiplMJze44HbhqLo/70CHki0K8JQyCGG6lKYHZL8KPi/7lraL2RGGm7YpA1V
	gP5kqTV+ARpE8qHQAPd0MJNPT6FUbao=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130327; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Zf1h+BIq0Z8Nwwp8lu1GEpj5zMTW9hh2PPOlxbQgdSw=;
	b=OF/utg6Saod0ZXXQVz6YP8a8kQYLJUaOHsJvdE98qsRyg1rJK7pZsSihiaAanOsrHPkhXQ
	bAOhTj/u4IjiplMJze44HbhqLo/70CHki0K8JQyCGG6lKYHZL8KPi/7lraL2RGGm7YpA1V
	gP5kqTV+ARpE8qHQAPd0MJNPT6FUbao=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v4 22/32] tools/xenstored: move systemd handling to posix.c
Date: Mon,  5 Feb 2024 11:49:51 +0100
Message-Id: <20240205105001.24171-23-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Move systemd handling to a new late_init() function in posix.c.

This prepares a future removal of the NO_SOCKETS macro.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- new patch
---
 tools/xenstored/core.c   | 17 +----------------
 tools/xenstored/core.h   |  1 +
 tools/xenstored/minios.c |  4 ++++
 tools/xenstored/posix.c  | 13 +++++++++++++
 4 files changed, 19 insertions(+), 16 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index eba7744fde..7b5e1d0c0f 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -54,16 +54,6 @@
 #include "control.h"
 #include "lu.h"
 
-#ifndef NO_SOCKETS
-#if defined(HAVE_SYSTEMD)
-#define XEN_SYSTEMD_ENABLED 1
-#endif
-#endif
-
-#if defined(XEN_SYSTEMD_ENABLED)
-#include <systemd/sd-daemon.h>
-#endif
-
 extern xenevtchn_handle *xce_handle; /* in domain.c */
 static int xce_pollfd_idx = -1;
 static struct pollfd *fds;
@@ -2938,12 +2928,7 @@ int main(int argc, char *argv[])
 	/* Get ready to listen to the tools. */
 	initialize_fds(&sock_pollfd_idx, &timeout);
 
-#if defined(XEN_SYSTEMD_ENABLED)
-	if (!live_update) {
-		sd_notify(1, "READY=1");
-		fprintf(stderr, SD_NOTICE "xenstored is ready\n");
-	}
-#endif
+	late_init(live_update);
 
 	/* Main loop. */
 	for (;;) {
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index 72173f1684..0de2d8a26e 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -385,6 +385,7 @@ static inline bool domain_is_unprivileged(const struct connection *conn)
 /* Return the event channel used by xenbus. */
 evtchn_port_t get_xenbus_evtchn(void);
 void early_init(bool live_update, bool dofork, const char *pidfile);
+void late_init(bool live_update);
 
 void init_sockets(void);
 extern int reopen_log_pipe[2];
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 4f48f63083..45bb0440be 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -49,3 +49,7 @@ void unmap_xenbus(void *interface)
 void early_init(bool live_update, bool dofork, const char *pidfile)
 {
 }
+
+void late_init(bool live_update)
+{
+}
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 9463ef5c8d..97c8dcaba3 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -24,6 +24,9 @@
 #include <stdlib.h>
 #include <syslog.h>
 #include <sys/mman.h>
+#if defined(HAVE_SYSTEMD)
+#include <systemd/sd-daemon.h>
+#endif
 #include <xen-tools/xenstore-common.h>
 
 #include "utils.h"
@@ -184,3 +187,13 @@ void early_init(bool live_update, bool dofork, const char *pidfile)
 	if (!live_update)
 		init_sockets();
 }
+
+void late_init(bool live_update)
+{
+#if defined(HAVE_SYSTEMD)
+	if (!live_update) {
+		sd_notify(1, "READY=1");
+		fprintf(stderr, SD_NOTICE "xenstored is ready\n");
+	}
+#endif
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:59:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:59:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675901.1051592 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwhP-0003Zw-Vi; Mon, 05 Feb 2024 10:59:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675901.1051592; Mon, 05 Feb 2024 10:59:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwhP-0003Zn-T3; Mon, 05 Feb 2024 10:59:39 +0000
Received: by outflank-mailman (input) for mailman id 675901;
 Mon, 05 Feb 2024 10:59:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwYt-0000qL-Bg
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:50:51 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6c8bcce9-c414-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 11:50:49 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 4433B22105;
 Mon,  5 Feb 2024 10:50:49 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 19F8D132DD;
 Mon,  5 Feb 2024 10:50:49 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id /rjnBIm9wGUzNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:50:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c8bcce9-c414-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130249; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Bf1Cys4GWKYwJ14lNNx92x1baFWNjqmyMtleMunbvkA=;
	b=EIG+DjNxmW0fpUfivt6bfCqvYTaj+xVjYV3cD3UGsQIv11Qluwj4NoAkKOaFYmQTaqH2Oh
	M9ARoC8mKaLvkfbMW63sRqbfdJNF0LhqZXIOr/fCZBIF94xrXq0QKiYa1TrIjWGCnVrTe/
	uSY7vMWRHF3HBFJsIfB+gyWkU9eA1fM=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130249; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Bf1Cys4GWKYwJ14lNNx92x1baFWNjqmyMtleMunbvkA=;
	b=EIG+DjNxmW0fpUfivt6bfCqvYTaj+xVjYV3cD3UGsQIv11Qluwj4NoAkKOaFYmQTaqH2Oh
	M9ARoC8mKaLvkfbMW63sRqbfdJNF0LhqZXIOr/fCZBIF94xrXq0QKiYa1TrIjWGCnVrTe/
	uSY7vMWRHF3HBFJsIfB+gyWkU9eA1fM=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v4 08/32] tools/xen-9pfsd: add 9pfs walk request support
Date: Mon,  5 Feb 2024 11:49:37 +0100
Message-Id: <20240205105001.24171-9-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 TO_DN_SOME(0.00)[];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add the walk request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- don't allow walking across symbolic links
---
 tools/xen-9pfsd/io.c        | 171 ++++++++++++++++++++++++++++++++++++
 tools/xen-9pfsd/xen-9pfsd.h |   1 +
 2 files changed, 172 insertions(+)

diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
index 4f6acefc87..5b827706b7 100644
--- a/tools/xen-9pfsd/io.c
+++ b/tools/xen-9pfsd/io.c
@@ -28,9 +28,11 @@
 #define P9_CMD_VERSION    100
 #define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
+#define P9_CMD_WALK       110
 
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
+#define P9_WALK_MAXELEM   16
 
 struct p9_qid {
     uint8_t type;
@@ -478,6 +480,21 @@ static struct p9_fid *find_fid(device *device, unsigned int fid)
     return NULL;
 }
 
+static struct p9_fid *get_fid_ref(device *device, unsigned int fid)
+{
+    struct p9_fid *fidp;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    fidp = find_fid(device, fid);
+    if ( fidp )
+        fidp->ref++;
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    return fidp;
+}
+
 static struct p9_fid *alloc_fid_mem(device *device, unsigned int fid,
                                     const char *path)
 {
@@ -578,6 +595,10 @@ static int fill_qid(device *device, const char *path, struct p9_qid *qid,
         stbuf = &st;
     }
 
+    /* Don't allow symbolic links. */
+    if ( S_ISLNK(stbuf->st_mode) )
+        return EMLINK;
+
     qid->type = S_ISDIR(stbuf->st_mode) ? QID_TYPE_DIR : 0;
     qid->version = stbuf->st_mtime ^ (stbuf->st_size << 8);
     qid->path = stbuf->st_ino;
@@ -585,6 +606,20 @@ static int fill_qid(device *device, const char *path, struct p9_qid *qid,
     return 0;
 }
 
+static bool name_ok(const char *str)
+{
+    if ( !*str )
+        return false;
+
+    if ( strchr(str, '/' ) )
+        return false;
+
+    if ( !strcmp(str, "..") || !strcmp(str, ".") )
+        return false;
+
+    return true;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -662,6 +697,138 @@ static void p9_attach(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "Q", &qid);
 }
 
+static void p9_walk(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint32_t newfid;
+    struct p9_fid *fidp = NULL;
+    struct p9_qid *qids = NULL;
+    unsigned int n_names = 0;
+    unsigned int *names = NULL;
+    unsigned int walked = 0;
+    unsigned int i;
+    char *path = NULL;
+    unsigned int path_len;
+    int ret;
+
+    ret = fill_data(ring, "UUaS", &fid, &newfid, &n_names, &names);
+    if ( n_names > P9_WALK_MAXELEM )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        goto out;
+    }
+    if ( ret != 3 + n_names )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        goto out;
+    }
+    if ( fidp->opened )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        goto out;
+    }
+
+    path_len = strlen(fidp->path) + 1;
+    for ( i = 0; i < n_names; i++ )
+    {
+        if ( !name_ok(ring->str + names[i]) )
+        {
+            p9_error(ring, hdr->tag, ENOENT);
+            goto out;
+        }
+        path_len += strlen(ring->str + names[i]) + 1;
+    }
+    path = calloc(path_len + 1, 1);
+    if ( !path )
+    {
+        p9_error(ring, hdr->tag, ENOMEM);
+        goto out;
+    }
+    strcpy(path, fidp->path);
+
+    if ( n_names )
+    {
+        qids = calloc(n_names, sizeof(*qids));
+        if ( !qids )
+        {
+            p9_error(ring, hdr->tag, ENOMEM);
+            goto out;
+        }
+        for ( i = 0; i < n_names; i++ )
+        {
+            strcat(path, "/");
+            strcat(path, ring->str + names[i]);
+            ret = fill_qid(device, path, qids + i, NULL);
+            if ( ret )
+            {
+                if ( !walked )
+                {
+                    p9_error(ring, hdr->tag, errno);
+                    goto out;
+                }
+                break;
+            }
+            walked++;
+        }
+    }
+
+    if ( walked == n_names )
+    {
+        bool ok = false;
+
+        if ( fid == newfid )
+        {
+            struct p9_fid *new_fidp;
+
+            pthread_mutex_lock(&device->fid_mutex);
+
+            if ( fidp->ref != 2 )
+            {
+                errno = EBUSY;
+            }
+            else
+            {
+                new_fidp = alloc_fid_mem(device, fid, path);
+                if ( new_fidp )
+                {
+                    new_fidp->ref = 2;
+                    XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+                    XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+                    free(fidp);
+                    fidp = new_fidp;
+                    ok = true;
+                }
+            }
+
+            pthread_mutex_unlock(&device->fid_mutex);
+        }
+        else
+            ok = alloc_fid(device, newfid, path);
+
+        if ( !ok )
+        {
+            p9_error(ring, hdr->tag, errno);
+            goto out;
+        }
+    }
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "aQ", &walked, qids);
+
+ out:
+    free_fid(device, fidp);
+    free(qids);
+    free(path);
+    free(names);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -725,6 +892,10 @@ void *io_thread(void *arg)
                 p9_attach(ring, &hdr);
                 break;
 
+            case P9_CMD_WALK:
+                p9_walk(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/xen-9pfsd/xen-9pfsd.h b/tools/xen-9pfsd/xen-9pfsd.h
index 6c755411a7..f01fffb0bb 100644
--- a/tools/xen-9pfsd/xen-9pfsd.h
+++ b/tools/xen-9pfsd/xen-9pfsd.h
@@ -25,6 +25,7 @@ struct p9_fid {
     XEN_TAILQ_ENTRY(struct p9_fid) list;
     unsigned int fid;
     unsigned int ref;
+    bool opened;
     char path[];
 };
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 10:59:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 10:59:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675902.1051598 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwhQ-0003ca-AF; Mon, 05 Feb 2024 10:59:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675902.1051598; Mon, 05 Feb 2024 10:59:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwhQ-0003bx-4L; Mon, 05 Feb 2024 10:59:40 +0000
Received: by outflank-mailman (input) for mailman id 675902;
 Mon, 05 Feb 2024 10:59:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwZb-0002wU-3o
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 10:51:35 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 87365d1d-c414-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 11:51:34 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 0C76E1F8BA;
 Mon,  5 Feb 2024 10:51:34 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D28CE132DD;
 Mon,  5 Feb 2024 10:51:33 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id qvb3MbW9wGWVNgAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 10:51:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87365d1d-c414-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130294; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=h0jxnSPLZTXt0UgW8EvhU376ps4C4d/D3zwFe+T6tRE=;
	b=EnUOwUQ/3F5sKaosP/ma2J4roQfFx+oWZZzhZeBFhGT6sKJXu3PO70y2VtPVMHZEfasPrK
	xrBOXPAvMS5UNMIh+sVxO16ohUDPO+OQM3nWqKAbW8vre94mjojrgyiWk/LSFq6/mi+xkD
	+PFHUU162jed1J0ctHrvSoNASg0VKPc=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707130294; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=h0jxnSPLZTXt0UgW8EvhU376ps4C4d/D3zwFe+T6tRE=;
	b=EnUOwUQ/3F5sKaosP/ma2J4roQfFx+oWZZzhZeBFhGT6sKJXu3PO70y2VtPVMHZEfasPrK
	xrBOXPAvMS5UNMIh+sVxO16ohUDPO+OQM3nWqKAbW8vre94mjojrgyiWk/LSFq6/mi+xkD
	+PFHUU162jed1J0ctHrvSoNASg0VKPc=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v4 16/32] tools/xl: support new 9pfs backend xen_9pfsd
Date: Mon,  5 Feb 2024 11:49:45 +0100
Message-Id: <20240205105001.24171-17-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240205105001.24171-1-jgross@suse.com>
References: <20240205105001.24171-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add support for the new 9pfs backend "xen_9pfsd". For this backend type
the tag defaults to "Xen" and the host side path to
"/var/log/xen/guests/<dom-name>".

Do most of the default settings in libxl. Unfortunately the default
path can't easily be set in libxl, as the domain name isn't available
in the related 9pfs specific function.

Settings the defaults in libxl requires to move the sanity checking
of 9pfs parameters from xl to libxl, too.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- test max_files and max_open_files, too (Jason Andryuk)
V4:
- fix man page to use the "xen_9pfsd" type due to idl limitation
  (Jason Andryuk)
- set (most of) the defaults in libxl (Anthony Perard)
---
 docs/man/xl.cfg.5.pod.in      | 36 +++++++++++++++++++++++++++++++++--
 tools/libs/light/libxl_9pfs.c | 18 ++++++++++++++++++
 tools/xl/xl_parse.c           | 23 +++++++++++++++++++---
 3 files changed, 72 insertions(+), 5 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index ea8d41727d..039e057318 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -772,10 +772,16 @@ settings, from the following list:
 
 =over 4
 
+=item B<type=TYPE>
+
+The backendtype for the PV device. Supported values are B<qemu> and
+B<xen_9pfsd>.  The default is B<qemu>.
+
 =item B<tag=STRING>
 
 9pfs tag to identify the filesystem share. The tag is needed on the
-guest side to mount it.
+guest side to mount it. For the backendtype of B<xen_9pfsd> the tag defaults to
+"Xen".
 
 =item B<security_model="none">
 
@@ -785,12 +791,38 @@ squash or remap).
 
 =item B<path=STRING>
 
-Filesystem path on the backend to export.
+Filesystem path on the backend to export. For the backendtype of B<xen_9pfsd>
+the path defaults to "@XEN_LOG_DIR@/guests/<guest-name>".
 
 =item B<backend=domain-id>
 
 Specify the backend domain name or id, defaults to dom0.
 
+=item B<max-files=NUMBER>
+
+Specify the maximum number of files below B<path>. A value of 0 (which
+is the default) doesn't limit the number of files. Only valid for
+B<type=xen_9pfsd>.
+
+=item B<max-open-files=NUMBER>
+
+Specify the maximum number of concurrently opened files below B<path>.
+Multiple opens of the same file are counted individually. Only valid for
+B<type=xen_9pfsd>, which has a default of B<max-open-files=5>.
+
+=item B<max-space=NUMBER>
+
+Specify the maximum used disk space in MiB below B<path>. A value of 0 (which
+is the default) doesn't limit the usable disk space. Only valid for
+B<type=xen_9pfsd>.
+
+=item B<auto-delete=BOOLEAN>
+
+When set the backend will delete the oldest file which is currently not
+opened by the guest in case the disk space limit set via B<max-space> or the
+file limit set via B<max-files> is being reached. Only valid for
+B<type=xen_9pfsd>.
+
 =back
 
 =item B<pvcalls=[ "backend=domain-id", ... ]>
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 68fa9947ad..7593686a90 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -20,6 +20,24 @@
 static int libxl__device_p9_setdefault(libxl__gc *gc, uint32_t domid,
                                        libxl_device_p9 *p9, bool hotplug)
 {
+    if (p9->type == LIBXL_P9_TYPE_UNKNOWN) {
+        p9->type = LIBXL_P9_TYPE_QEMU;
+    }
+    if (p9->type == LIBXL_P9_TYPE_QEMU &&
+        (p9->max_files || p9->max_open_files || p9->max_space ||
+         p9->auto_delete)) {
+        LOGD(ERROR, domid, "Illegal 9pfs parameter combination");
+        return ERROR_INVAL;
+    }
+    if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD && !p9->tag) {
+        p9->tag = libxl__strdup(NOGC, "Xen");
+    }
+
+    if (!p9->path || !p9->security_model || !p9->tag) {
+        LOGD(ERROR, domid, "9pfs spec missing required field!");
+        return ERROR_INVAL;
+    }
+
     return libxl__resolve_domid(gc, p9->backend_domname, &p9->backend_domid);
 }
 
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 9b358f11b8..80ffe85f5e 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2233,6 +2233,20 @@ void parse_config_data(const char *config_source,
                     replace_string(&p9->tag, value);
                 } else if (!strcmp(key, "backend")) {
                     replace_string(&p9->backend_domname, value);
+                } else if (!strcmp(key, "type")) {
+                    if (libxl_p9_type_from_string(value, &p9->type)) {
+                        fprintf(stderr, "failed to parse 9pfs type: %s\n",
+                                value);
+                        exit(1);
+                    }
+                } else if (!strcmp(key, "max-files")) {
+                    p9->max_files = parse_ulong(value);
+                } else if (!strcmp(key, "max-open-files")) {
+                    p9->max_open_files = parse_ulong(value);
+                } else if (!strcmp(key, "max-space")) {
+                    p9->max_space = parse_ulong(value);
+                } else if (!strcmp(key, "auto-delete")) {
+                    p9->auto_delete = strtoul(value, NULL, 0);
                 } else {
                     fprintf(stderr, "Unknown 9pfs parameter '%s'\n", key);
                     exit(1);
@@ -2243,9 +2257,12 @@ void parse_config_data(const char *config_source,
 
             libxl_string_list_dispose(&pairs);
 
-            if (!p9->path || !p9->security_model || !p9->tag) {
-                fprintf(stderr, "9pfs spec missing required field!\n");
-                exit(1);
+            if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD && !p9->path) {
+                char *path;
+
+                xasprintf(&path, XEN_LOG_DIR "/guests/%s", c_info->name);
+                replace_string(&p9->path, path);
+                free(path);
             }
         }
     }
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 11:02:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 11:02:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675930.1051613 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwk4-00061e-Mt; Mon, 05 Feb 2024 11:02:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675930.1051613; Mon, 05 Feb 2024 11:02:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwk4-00061V-JI; Mon, 05 Feb 2024 11:02:24 +0000
Received: by outflank-mailman (input) for mailman id 675930;
 Mon, 05 Feb 2024 11:02:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWwk3-00061F-0Z
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 11:02:23 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0930ec22-c416-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 12:02:21 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-40fd280421aso14375455e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 03:02:21 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n8-20020a05600c500800b0040fd162944fsm6859582wmr.35.2024.02.05.03.02.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 03:02:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0930ec22-c416-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707130941; x=1707735741; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=JMhBhF+jLHqSxlmz3PoFc0DY3oDXSGsNldebH1x70jc=;
        b=ei0mN1u2KtEkwRRtxOLTKB6Ds3wJE0AFcMoyj6hIHUCevHQL4xDVsYQSoZw51zLCkP
         WLE4gt5zmFYviAJUu2Zid6VoIHK/8I4O03xgw1c04lc+tut61MMxThVom84PnSlVkf76
         6M/MMcZ46ttXC0NVH96Z2npbeF1b7M9iuIjBxdoX7YgDu+8oIv8RDjRD7merDoNrYwNj
         G9wkxhMX2xYc0NxSilDlJ1k20IH/3KEbNvtAj6YZ/ccMLcCgbKoPtqPK+9pgTVPv1U30
         XwGy4Yk4s68EZVC7Yx6M+eYeslgvIG4z8ppVmeXAh4BcIEPOGOfFwtiy37bedKIbWI5T
         UZIA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707130941; x=1707735741;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JMhBhF+jLHqSxlmz3PoFc0DY3oDXSGsNldebH1x70jc=;
        b=cLFRgxCTVm0vuVw1PlgBS1mRdtOca1TcQLKI7z6BnOBM7Jwbuvqyv/QlxfMicwBLs1
         l7G49dQqUAk3PMgxXh1LBE68l1YSmMYWYbl/k2VNQBILwnzTtxSXWja9laWcJ9V9SiiQ
         fCiu9rzn0GWRjwW/NRa5tkBsqt86L97J/AqKJqmYacCsfTrmQz4Tn4gU8dY7UsGr4Jhk
         z/iGTPG2IkGDC+8w3ygII98L8wRmGbQBVsdIewkQgz1hJS6S/I6Bizrg3gSJdnwKkuMH
         XxHkjMbWH9QBH2kFs+00IO/GZhXQz+j2pPrLsE+nX3MlmwHb/ShiZ1MO4TTruIUFZ3Q+
         paBw==
X-Gm-Message-State: AOJu0YxR5udJqNU7z8P7Bfe6ir1580tS6L69MVaVU9h26SuBiSAWr+me
	4m/RCYXpewBL9g1e3Glml5eU5IZ7JER6dqpHN3PsOnK3wh+PXsjVTqC6ksh4Cskq8GHxe9YM/Nc
	=
X-Google-Smtp-Source: AGHT+IHoZ2j2fykQIUqgEUwSeqD2KadUvquUBLsnbiO59SBJLvMWkN6sm6rrXOn0LXcqOeZLcS/9eQ==
X-Received: by 2002:a05:600c:4f44:b0:40e:d30d:3fe2 with SMTP id m4-20020a05600c4f4400b0040ed30d3fe2mr4784821wmq.5.1707130933375;
        Mon, 05 Feb 2024 03:02:13 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUCwKy49F54VvtTcCjc4HlA6iTbflap4YpOlO614g9fbYRLWvDndOS1UOUGQqmyWMB/JH2TZjRf50/6r7ncU/iK5U4HWrjvZbZlFwTSo+X6h9Eu9df9wPP7YlE0jqLSYurjemKjG5S60anv7/eKlHG/v0tN6mCV5Rd2114F
Message-ID: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
Date: Mon, 5 Feb 2024 12:02:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/2] gnttab: hypervisor side XSA-448 follow-up
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Let's try to better deal with effectively no-op requests.

1: fully ignore zero-size copy requests
2: almost fully ignore zero-size flush requests

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 11:03:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 11:03:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675934.1051622 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwl9-0006eg-3P; Mon, 05 Feb 2024 11:03:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675934.1051622; Mon, 05 Feb 2024 11:03:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwl9-0006eZ-0X; Mon, 05 Feb 2024 11:03:31 +0000
Received: by outflank-mailman (input) for mailman id 675934;
 Mon, 05 Feb 2024 11:03:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWwl7-0006eH-GH
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 11:03:29 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3063a019-c416-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 12:03:27 +0100 (CET)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2cf4d2175b2so47748451fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 03:03:27 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n8-20020a05600c500800b0040fd162944fsm6859582wmr.35.2024.02.05.03.03.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 03:03:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3063a019-c416-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707131007; x=1707735807; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=qQnuT178TRlW0ncotS0NtNaqASxlSbwyqI0Oiy5iAaw=;
        b=RT7X4dPqXiOzCDIg1dsTI2F/db9kpvikuFcNR+4poG4f+7MB58XIzzDA234H1lnQHq
         kOzvrQ+d+SdxxU3JQnHEDcGmwfkLfp3lCKPkSBdkbmkMOuJdx97na4WosON5JeND+9PC
         GOlwQHvJ7XMZ2C45zDFWRLWBS3O26Doc2kUkjLdkrmUH5/QO/eycNJCBr0BLOE8TlUvJ
         3OMZAk5CY21WcncZ8GXG2haVpgSVLMEUnE4i72P0mVJaEOazGpyEbeYe7FYVVL2rwvX7
         77kKqGIY5hUvoRk2xcX2rphf5RWSBsPrYkED3PDYR/aLTtfOQkMZA1Ict6B6+e0cItwK
         CGNw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707131007; x=1707735807;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=qQnuT178TRlW0ncotS0NtNaqASxlSbwyqI0Oiy5iAaw=;
        b=d3iRGH1PXdb0l/B8xMAMYX/4HUrfa1Z9D49XLD9khn1IrGn3zLO1is3GftXFgQSnAj
         +XKL7Oqlw5IV5R4qkpEcNBpUJzPcqkhVVVw1yo/cOma2jJTPz8b4qwXpDqom1gLlPfUB
         spYKgfNA0Cg6AOGTp3nZp3qKWheyfHFfdNv6qHzBsnEl2mFtLfGj5ta2z2uaDjBceHmm
         IhfroP0hI7n1Wp5CMWa/gabhBB9c5XSNPmz4dG/AIWIABa/83atKY0MibqfKRYFS/s4Z
         Rrbd6SUFXoM5T7QGEGPMbHkjW52GENaWJAZR41TIhVStZfyGU+Wu/H7ipLc31KPCIZcs
         H1wA==
X-Gm-Message-State: AOJu0YwFFuNmhtM37tFZUzdm8ETNKN3kq3iRQXkCJhwMYZqfdA32/BEw
	23cdh7STVQUYmMfDib3Gwkj8dAH6InnV5LcRoDrsad7np7A07HM2Nn2FN8zv82DMtMvdqB1tJ1A
	=
X-Google-Smtp-Source: AGHT+IEX+Xb1nJ4GIZ5yrhUdDAX/Ds+MeoIxYHNvhJWpN+FsxmZuvqOAjgil9esX3O9SdAD0jO6WNg==
X-Received: by 2002:a05:6512:783:b0:511:546c:1f14 with SMTP id x3-20020a056512078300b00511546c1f14mr770910lfr.63.1707131007135;
        Mon, 05 Feb 2024 03:03:27 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXDMB1RGsEUkeHRLVavpaWfRbrLd8Toqzzn8e1+D4q+0FGcU8N0tC2gBCw0as+r8UIdv/fHpDkySDW+L1hO5Zi+aZMtumWZSx0fI7w0VFoc+FxnyPvDQfofbLCBZAb+gyO7W7EQcwImPnFZcyBuhiF1o1cMHHZug7OL7451
Message-ID: <a73f9ba8-8914-4c41-be55-a57c93c571f0@suse.com>
Date: Mon, 5 Feb 2024 12:03:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 1/2] gnttab: fully ignore zero-size copy requests
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Along the line with observations in the context of XSA-448, no field in
struct gnttab_copy_ptr is relevant when no data is to be copied, much
like e.g. the pointers passed to memcpy() are irrelevant (and would
never be "validated") when the passed length is zero.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3061,6 +3061,9 @@ static int gnttab_copy_one(const struct
 {
     int rc;
 
+    if ( unlikely(!op->len) )
+        return GNTST_okay;
+
     if ( !src->domain || op->source.domid != src->ptr.domid ||
          !dest->domain || op->dest.domid != dest->ptr.domid )
     {



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 11:03:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 11:03:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675936.1051633 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwlZ-00077n-Ah; Mon, 05 Feb 2024 11:03:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675936.1051633; Mon, 05 Feb 2024 11:03:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwlZ-00077g-7l; Mon, 05 Feb 2024 11:03:57 +0000
Received: by outflank-mailman (input) for mailman id 675936;
 Mon, 05 Feb 2024 11:03:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWwlY-00075h-2M
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 11:03:56 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 40fd76d9-c416-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 12:03:55 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-40fc22f372cso32611295e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 03:03:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n8-20020a05600c500800b0040fd162944fsm6859582wmr.35.2024.02.05.03.03.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 03:03:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 40fd76d9-c416-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707131035; x=1707735835; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=bsP4/6DK9onXjg/QyVKo8/M3vS3DN9Ae4cLk5QBN9lY=;
        b=aZ4SB3pdjH8Fc9j/lAJ5KVzZhR1HBEaXz9XuEe84V0PdsEaRpjIk7QfgFs1+sKVbqL
         H7ZCz2D1oyMwKATjAc4vRO0Hqcicy8EwzmtPwv5x9ts5zpc9OHF2WK5jXV/GInx+PB9b
         MePgPmf0E5iyCCKiTYwWokksjrA3j9fxFIQXHBwmj0i7tyuVAebRxptkKMBo73jrdIpH
         pGkZc4uw+F4dA4incZdgDo9/OsJx9ny2Aajjx8AUNOLKzX1z0kZEKoT+rjK/R/gZo9fF
         Ff/iSzDljYKeYdK/08x83/iaA3DZ4eSpgVzvYVJjhpcdQEvb9A6e5XKWenHlE2QjhKl5
         FzQg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707131035; x=1707735835;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=bsP4/6DK9onXjg/QyVKo8/M3vS3DN9Ae4cLk5QBN9lY=;
        b=bLen14K6jqrN2JCIaRTfJEGzB6kdaz4/3JxhCZDkv892kvzqK4w6ubeoo5nP4yzRys
         HvJ1fwoxZG2yoYJ0IskJtwj1CSfJmD/Ohs5zgJiTelS85LyeCIwMNYcXRs1A5Kw6rdDO
         mss2bVKG0EozQKbFLBc7JA2UhXbwvsth+aFvAVNTXCL5lZYAWXVETx19WmtGLRoxukA3
         GpImTQSVhYDtMI9qQMfIfiJacGFa6nMAdiYOnatDXfTAKzcfektEX3dGvs8l6yYgjad3
         laJ5QTwcraXIFbapWdjPWGpIwuiVc7I5mft48yyHfU6V1mGdpQzxrjkD/VaFeYSJPCMH
         WzKg==
X-Gm-Message-State: AOJu0Yw+aeVWBfIH9hFUf1j9k3K85lTmBofYTKr0B6GYHoc7YoBD9DgG
	WE6HHAPcxuIiQxsiYTnf0JsTwa2JoCU+qnEh4PB9eKw3tBeT845W6d0qeUGHXQJSc7WLYGQRx3k
	=
X-Google-Smtp-Source: AGHT+IE55s07IQQaRh1RDB8AefWOYLW2L5s2DFBXn797p0KxXpT3Q95nwt514Q7W32OlUbh5wYOrfw==
X-Received: by 2002:a05:600c:4ec7:b0:40e:5ed3:ccf4 with SMTP id g7-20020a05600c4ec700b0040e5ed3ccf4mr4517267wmq.16.1707131034678;
        Mon, 05 Feb 2024 03:03:54 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVE5WJ+iiDOkVCWZTykBJ5YA1suaUNHO5lzMjBbiGOK1Dw/i8PjCk1srnuFfF9vxsRkUq67b52U47WSVaDa92JAJStd0ma541aw80NxF8z2Ujw2WwtFaEgUyQpLKUs4nUm/vBxF9uo0f4XWciRDT4yREzaoZ0zxFd+PL0tL
Message-ID: <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com>
Date: Mon, 5 Feb 2024 12:03:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 2/2] almost fully ignore zero-size flush requests
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Along the line with observations in the context of XSA-448, besides
"op" no field is relevant when the range to be flushed is empty, much
like e.g. the pointers passed to memcpy() are irrelevant (and would
never be "validated") when the passed length is zero. Split the existing
condition validating "op", "offset", and "length", leaving only the "op"
part ahead of the check for length being zero (or no flushing to be
performed).

In the course of splitting also simplify the moved part of the condition
from 3 to 2 conditionals, potentially (depending on the architecture)
requiring one less (conditional) branch.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3528,15 +3528,16 @@ static int _cache_flush(const gnttab_cac
     void *v;
     int ret;
 
-    if ( (cflush->offset >= PAGE_SIZE) ||
-         (cflush->length > PAGE_SIZE) ||
-         (cflush->offset + cflush->length > PAGE_SIZE) ||
-         (cflush->op & ~(GNTTAB_CACHE_INVAL | GNTTAB_CACHE_CLEAN)) )
+    if ( cflush->op & ~(GNTTAB_CACHE_INVAL | GNTTAB_CACHE_CLEAN) )
         return -EINVAL;
 
     if ( cflush->length == 0 || cflush->op == 0 )
         return !*cur_ref ? 0 : -EILSEQ;
 
+    if ( (cflush->offset | cflush->length) > PAGE_SIZE ||
+         cflush->offset + cflush->length > PAGE_SIZE )
+        return -EINVAL;
+
     /* currently unimplemented */
     if ( cflush->op & GNTTAB_CACHE_SOURCE_GREF )
         return -EOPNOTSUPP;



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 11:09:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 11:09:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675943.1051643 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwqQ-0008Lx-UH; Mon, 05 Feb 2024 11:08:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675943.1051643; Mon, 05 Feb 2024 11:08:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwqQ-0008Lq-Qs; Mon, 05 Feb 2024 11:08:58 +0000
Received: by outflank-mailman (input) for mailman id 675943;
 Mon, 05 Feb 2024 11:08:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rWwqP-0008JF-PP
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 11:08:57 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f2f31bb7-c416-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 12:08:54 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 814C12212D;
 Mon,  5 Feb 2024 11:08:53 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 442F8132DD;
 Mon,  5 Feb 2024 11:08:53 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 0IHfDsXBwGWUOwAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 11:08:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2f31bb7-c416-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707131333; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=OTAX6IEAs+BtLQ3mCkBm+QT1FK6ZCTnoPrPnxLn3ttk=;
	b=ekxObCXlEApDTZ0EY4rTOaL4U2cQ37Fzp0mjVDsYy/WtrKni/xCsp4JDIL5u7uldij9p3I
	iJzXwJcZyKSjnABiU0/aEiXEoeLLFYfQHPMdP4JutLJ423PeOoarpqllzlk7S0v47lEgcd
	REwkwhk+klBYZ8SqARZ749ajHhSFPj8=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707131333; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=OTAX6IEAs+BtLQ3mCkBm+QT1FK6ZCTnoPrPnxLn3ttk=;
	b=ekxObCXlEApDTZ0EY4rTOaL4U2cQ37Fzp0mjVDsYy/WtrKni/xCsp4JDIL5u7uldij9p3I
	iJzXwJcZyKSjnABiU0/aEiXEoeLLFYfQHPMdP4JutLJ423PeOoarpqllzlk7S0v47lEgcd
	REwkwhk+klBYZ8SqARZ749ajHhSFPj8=
Message-ID: <d4740502-795a-4e22-afe2-abd0746da843@suse.com>
Date: Mon, 5 Feb 2024 12:08:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 00/32] tools: enable xenstore-stubdom to use 9pfs
Content-Language: en-US
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Nick Rosbrook
 <rosbrookn@gmail.com>, Samuel Thibault <samuel.thibault@ens-lyon.org>
References: <20240205105001.24171-1-jgross@suse.com>
 <689635a7-6d34-44fe-b00b-31fdc03f6969@xen.org>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <689635a7-6d34-44fe-b00b-31fdc03f6969@xen.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.58
X-Spamd-Result: default: False [-3.58 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 BAYES_HAM(-3.00)[100.00%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.71)[subject];
	 FREEMAIL_CC(0.00)[xen.org,citrix.com,gmail.com,ens-lyon.org];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Flag: NO

On 05.02.24 11:55, Julien Grall wrote:
> Hi Juergen,
> 
> On 05/02/2024 10:49, Juergen Gross wrote:
>> This series is adding 9pfs support to Xenstore-stubdom, enabling it
>> to do logging to a dom0 directory.
>>
>> This is a prerequisite for the final goal to add live update support
>> to Xenstore-stubdom, as it enables the stubdom to store its state in
>> a dom0 file.
>>
>> The 9pfs backend is a new daemon written from scratch. Using a
>> dedicated 9pfs daemon has several advantages:
>>
>> - it is using much less resources than a full blown qemu process
>> - it can serve multiple guests (the idea is to use it for other
>>    infrastructure domains, like qemu-stubdom or driver domains, too)
>> - it is designed to support several security enhancements, like
>>    limiting the number of files for a guest, or limiting the allocated
>>    file system space
>> - it doesn't support file links (neither hard nor soft links) or
>>    referencing parent directories via "..", minimizing the risk that
>>    a guest can "escape" from its home directory
>>
>> Note that for now the daemon only contains the minimal needed
>> functionality to do logging from Xenstore-stubdom. I didn't want to
>> add all the 9pfs commands and security add-ons in the beginning, in
>> order to avoid needless efforts in case the idea of the daemon is
>> being rejected.
>>
>> Changes in V4:
>> - patch 2 of V3 was applied
>> - added support of reading directories
>> - addressed review comments
>>
>> Changes in V3:
>> - new patches 1, 23-25
>> - addressed review comments
>>
>> Changes in V2:
>> - support of multiple rings per device
>> - xenlogd->xen-9pfsd rename
>> - addressed review comments
>> - fixed some bugs
>>
>> Juergen Gross (32):
>>    tools: add access macros for unaligned data
>>    tools: add a new xen logging daemon
>>    tools/xen-9pfsd: connect to frontend
>>    tools/xen-9pfsd: add transport layer
>>    tools/xen-9pfsd: add 9pfs response generation support
>>    tools/xen-9pfsd: add 9pfs version request support
>>    tools/xen-9pfsd: add 9pfs attach request support
>>    tools/xen-9pfsd: add 9pfs walk request support
>>    tools/xen-9pfsd: add 9pfs open request support
>>    tools/xen-9pfsd: add 9pfs clunk request support
>>    tools/xen-9pfsd: add 9pfs create request support
>>    tools/xen-9pfsd: add 9pfs stat request support
>>    tools/xen-9pfsd: add 9pfs write request support
>>    tools/xen-9pfsd: add 9pfs read request support
>>    tools/libs/light: add backend type for 9pfs PV devices
>>    tools/xl: support new 9pfs backend xen_9pfsd
>>    tools/helpers: allocate xenstore event channel for xenstore stubdom
>>    tools/xenstored: rename xenbus_evtchn()
>>    stubdom: extend xenstore stubdom configs
>>    tools: add 9pfs device to xenstore-stubdom
>>    tools/xenstored: add early_init() function
>>    tools/xenstored: move systemd handling to posix.c
>>    tools/xenstored: move all log-pipe handling into posix.c
>>    tools/xenstored: move all socket handling into posix.c
>>    tools/xenstored: get own domid in stubdom case
>>    tools/xenstored: rework ring page (un)map functions
>>    tools/xenstored: split domain_init()
>>    tools/xenstored: map stubdom interface
>>    tools/xenstored: mount 9pfs device in stubdom
>>    tools/xenstored: add helpers for filename handling
>>    tools/xenstored: support complete log capabilities in stubdom
>>    tools/xenstored: have a single do_control_memreport()
> 
> I haven't checked what's the state of the 9PFS patches. Can part of the 
> xenstored changes be committed without the 9PFS changes?

The following patches can go in without the 9pfs daemon:

tools/helpers: allocate xenstore event channel for xenstore stubdom
tools/xenstored: rename xenbus_evtchn()
stubdom: extend xenstore stubdom configs
tools/xenstored: add early_init() function
tools/xenstored: move systemd handling to posix.c
tools/xenstored: move all log-pipe handling into posix.c
tools/xenstored: move all socket handling into posix.c
tools/xenstored: get own domid in stubdom case
tools/xenstored: rework ring page (un)map functions
tools/xenstored: split domain_init()
tools/xenstored: map stubdom interface


Juergen


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 11:11:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 11:11:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675950.1051653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwsy-00028A-AV; Mon, 05 Feb 2024 11:11:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675950.1051653; Mon, 05 Feb 2024 11:11:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWwsy-00027v-7L; Mon, 05 Feb 2024 11:11:36 +0000
Received: by outflank-mailman (input) for mailman id 675950;
 Mon, 05 Feb 2024 11:11:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i4VU=JO=amazon.co.uk=prvs=758ae075e=eliasely@srs-se1.protection.inumbo.net>)
 id 1rWwsx-00026U-9u
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 11:11:35 +0000
Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com
 [99.78.197.217]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 50ac4bd8-c417-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 12:11:32 +0100 (CET)
Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214])
 by smtp-border-fw-80006.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2024 11:11:28 +0000
Received: from EX19MTAEUC002.ant.amazon.com [10.0.43.254:51294]
 by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.45.85:2525]
 with esmtp (Farcaster)
 id 7e6d3638-080d-404b-985a-14bdd4d4a538; Mon, 5 Feb 2024 11:11:26 +0000 (UTC)
Received: from EX19D018EUA002.ant.amazon.com (10.252.50.146) by
 EX19MTAEUC002.ant.amazon.com (10.252.51.181) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1118.40; Mon, 5 Feb 2024 11:11:26 +0000
Received: from [192.168.8.102] (10.106.83.11) by EX19D018EUA002.ant.amazon.com
 (10.252.50.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Mon, 5 Feb
 2024 11:11:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 50ac4bd8-c417-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209;
  t=1707131493; x=1738667493;
  h=message-id:date:mime-version:to:cc:references:from:
   in-reply-to:content-transfer-encoding:subject;
  bh=07uNVDwPNMFmJ8ZLTBjGgp4AYgWfP3D1106qVC1N05g=;
  b=OfmVb+Owk8Oj/vseUpW8SmRzj0XDs4zHjvAAeViVLDtVJLF4e29i9gBC
   59glSS5uWWtK1KbZJoV5aE4IaXPThIX/ryXkA7tnbvgrS29wPPoCzdSGv
   EZWLvtL53cahiDhI+aj9k3r/IyobvXDLwkRdHZ8U47JdXAeVXqxWHEDLa
   A=;
X-IronPort-AV: E=Sophos;i="6.05,245,1701129600"; 
   d="scan'208";a="271091896"
Subject: Re: [PATCH v2 (resend) 00/27] Remove the directmap
X-Farcaster-Flow-ID: 7e6d3638-080d-404b-985a-14bdd4d4a538
Message-ID: <217a6edd-96f8-4589-b500-2dc31a1b0bee@amazon.com>
Date: Mon, 5 Feb 2024 11:11:17 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
CC: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, "Bertrand
 Marquis" <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Lukasz Hawrylko
	<lukasz@hawrylko.pl>, "Daniel P. Smith" <dpsmith@apertussolutions.com>,
	=?UTF-8?Q?Mateusz_M=C3=B3wka?= <mateusz.mowka@intel.com>,
	<xen-devel@lists.xenproject.org>
References: <20240116192611.41112-1-eliasely@amazon.com>
 <04d113bf-71ab-4eba-a95f-72318c888d1f@suse.com>
From: Elias El Yandouzi <eliasely@amazon.com>
In-Reply-To: <04d113bf-71ab-4eba-a95f-72318c888d1f@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.106.83.11]
X-ClientProxiedBy: EX19D046UWB003.ant.amazon.com (10.13.139.174) To
 EX19D018EUA002.ant.amazon.com (10.252.50.146)

Hi Jan,

On 29/01/2024 08:28, Jan Beulich wrote:

> On 16.01.2024 20:25, Elias El Yandouzi wrote:
>> Julien Grall (8):
>>    xen/vmap: Check the page has been mapped in vm_init_type()
>>    xen/vmap: Introduce vmap_size() and use it
>>    xen/arm: fixmap: Rename the fixmap slots to follow the x86 convention
> 
> Btw, if there was clear indication that e.g. this patch (fully ack-ed
> afaict) can go in ahead of earlier patches, I probably would have put
> it in already. Considering it sits half way through the series, I don't
> want to blindly chance it, though.
> 

I just forgot to strip off those ack tags. The patch got approved quite 
a while ago and I thought it would be better to submit it again.

That being said, moving it earlier in the series would still work.

Elias


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 11:19:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 11:19:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675957.1051663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWx0Q-0003rJ-2E; Mon, 05 Feb 2024 11:19:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675957.1051663; Mon, 05 Feb 2024 11:19:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWx0P-0003rC-Uh; Mon, 05 Feb 2024 11:19:17 +0000
Received: by outflank-mailman (input) for mailman id 675957;
 Mon, 05 Feb 2024 11:19:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KzPa=JO=linaro.org=alex.bennee@srs-se1.protection.inumbo.net>)
 id 1rWx0P-0003qF-0y
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 11:19:17 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6589e0fa-c418-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 12:19:15 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-40fc549ab9bso26334535e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 03:19:15 -0800 (PST)
Received: from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id
 t9-20020a05600c198900b0040fbad272f6sm8302365wmq.46.2024.02.05.03.19.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 03:19:14 -0800 (PST)
Received: from draig (localhost [IPv6:::1])
 by draig.lan (Postfix) with ESMTP id 60D695F888;
 Mon,  5 Feb 2024 11:19:14 +0000 (GMT)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6589e0fa-c418-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1707131955; x=1707736755; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:user-agent
         :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=gHPLepVUxVSyG653+rgbNocVyEKTkP/abvuOnlRA7oU=;
        b=ppj5VYJBklSRzUkeohOwJaJMFHgdOjs29K1I+jeF8y1a/fSvIJD5auZE1NgAoqp5FL
         kyxnIProKBBJ5oE9eGWIgflbvDhI+MaACXfAUxrvdyr8HyzPdHzjUoGaFs4112bYhvfr
         KvoJGJeGEcO1CCh7aLdGjNmqrpmwIEB0YgQnR0O8RYT6sHpoR3NCybs6I8y7YSLw2aHd
         MiKCiBQsKNdEvw4CDXIOM+CraNMVDU9Q8Uia2klxsIhPT/cYzRwwinK1nL5AL0RYTEfd
         QltTfGSaLIFcKzJPRo73bsfrZRp7I5fnfT/Ny50tD19GpWs1Ml4mIPPS2iktr7sT/4gc
         TU3w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707131955; x=1707736755;
        h=content-transfer-encoding:mime-version:message-id:date:user-agent
         :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from
         :to:cc:subject:date:message-id:reply-to;
        bh=gHPLepVUxVSyG653+rgbNocVyEKTkP/abvuOnlRA7oU=;
        b=gAOWNz5CPnh5KHB2I2nu9t1Mj/Jka+gWW8hypvkvcJArQzWEIxLOBRB6CSpZDUX/8Z
         u4Xh17xN+GDb3bKUYKX/AsbB7yG6TMvVSRwjZN1faFB5bSnTSsBN7Xiu/f6S5fpzVmku
         geWang3eM/3v03zykUxK8+5TYl1m2fdxhXTZZIwng6eTKiZK9+HvW+2lffhEDTWIEg6x
         e6SZCHV38rEh1cGKFKXlYWJtxgV+EEcpPbMhw6AHxgov92IxLDl6Jpc2akTJN8bYtFj+
         /DQidlvqSb1oADVkn+xTo6W/5I/iloiNVwVEPIwXw8ZwmfBb1XeJKFQD6k/EWrXVAKzM
         665A==
X-Gm-Message-State: AOJu0Yy0FPlNIztwKGS9kPDoZnCtAmS14aAJ10vqnuyFKFTXeROw4ToZ
	PbuunIGZOeCzrsIsZy8ymRw1jBw0m01mvfWwt0mJMFxBgcHEV0cC5uwPI1l6ApA=
X-Google-Smtp-Source: AGHT+IEAlXKZT/TgeewMv8dB2GCLBQpedlyp5xgWJymg+H/r//ldB7MV2bz/YpJjY8Bc8QHLvOwMCg==
X-Received: by 2002:a1c:4c04:0:b0:40e:e6ce:71ff with SMTP id z4-20020a1c4c04000000b0040ee6ce71ffmr4478002wmf.41.1707131955229;
        Mon, 05 Feb 2024 03:19:15 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUHhuDGptbhLvabzYfqKG2vMvSVkG6wFj5G6YtJkOQyRx2TINnOqgLJKwmrMOrTYh0sVB/kyo9dFQOnM1iEdd7nCIjYsXXGVn16rEQnXpQVld24eaH+74BgB2WHuoOtycD4dyLVFFtEbAmnX+8hTy5vPRR/iL0vu2tuML+jDw6ulA==
From: =?utf-8?Q?Alex_Benn=C3=A9e?= <alex.bennee@linaro.org>
To: Julien Grall <julien@xen.org>
Cc: xen-devel@lists.xenproject.org,  sstabellini@kernel.org,
  bertrand.marquis@arm.com,  Manos Pitsidianakis
 <manos.pitsidianakis@linaro.org>
Subject: Re: [RFC PATCH] xen/arm: improve handling of load/store instruction
 decoding
In-Reply-To: <e82e5a15-88e8-4d24-b872-548b608cebbc@xen.org> (Julien Grall's
	message of "Mon, 5 Feb 2024 10:42:10 +0000")
References: <20240131175043.1488886-1-alex.bennee@linaro.org>
	<e82e5a15-88e8-4d24-b872-548b608cebbc@xen.org>
User-Agent: mu4e 1.11.27; emacs 29.1
Date: Mon, 05 Feb 2024 11:19:14 +0000
Message-ID: <87r0hrf9al.fsf@draig.linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Julien Grall <julien@xen.org> writes:

> Hi Alex,
>
> On 31/01/2024 17:50, Alex Benn=C3=A9e wrote:
>> While debugging VirtIO on Arm we ran into a warning due to memory
>> being memcpy'd across MMIO space. While the bug was in the mappings
>> the warning was a little confusing:
>>    (XEN) d47v2 Rn should not be equal to Rt except for r31
>>    (XEN) d47v2 unhandled Arm instruction 0x3d800000
>>    (XEN) d47v2 Unable to decode instruction
>> The Rn =3D=3D Rt warning is only applicable to single register
>> load/stores
>> so add some verification steps before to weed out unexpected accesses.
>> I updated the Arm ARM reference to the online instruction decoding
>> table which will hopefully be more stable than the Arm ARM section
>> numbers.
>
> I am not sure if the links to the Arm websites are stable. But from
> past, experience, URL tends to disappear after a while. This is why we
> went with the section + the Arm spec.

Depends if you can get the older spec. The section numbers unfortunately
change between versions. We have the same problem in our TCG aarch64
backend unfortunately when we named the encoders after section numbers.
>
> This also has the advantage that we can check any differences between
> version. So my preference is to stick the Arm ARM reference. Bertrand,
> Michal, Stefano, any opinions?
>
> Anyway, the idea looks fine to me. I left mostly some style comments.
>
>> Signed-off-by: Alex Benn=C3=A9e <alex.bennee@linaro.org>
>> Cc: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
>> ---
>>   xen/arch/arm/decode.c | 20 ++++++++++++++++++++
>>   xen/arch/arm/decode.h | 38 +++++++++++++++++++++++++++++++++++---
>>   2 files changed, 55 insertions(+), 3 deletions(-)
>> diff --git a/xen/arch/arm/decode.c b/xen/arch/arm/decode.c
>> index 2537dbebc1..824025c24c 100644
>> --- a/xen/arch/arm/decode.c
>> +++ b/xen/arch/arm/decode.c
>> @@ -87,6 +87,26 @@ static int decode_arm64(register_t pc, mmio_info_t *i=
nfo)
>>           return 1;
>>       }
>>   +    /*
>> +     * Check this is a load/store of some sort
>> +     */
>
> Coding style: This is a single line comment, so the preferred format is:
>
> /* .... */
>
>> +    if ( (opcode.top_level.op1 & 0b0101) !=3D 0b0100 )
>
> NIT: We are trying to avoid opcoding value in Xen. Can you add some  defi=
ne?
>
>> +    {
>> +        gprintk(XENLOG_ERR, "Not a load/store instruction op1=3D%d",
>
> Does the value need to be signed?
>
>> +                opcode.top_level.op1);
>> +        goto bad_loadstore;
>> +    }
>> +
>> +    /*
>> +     * We are only expecting single register load/stores
>> +     */
>
> Same here.
>
>> +    if ( (opcode.ld_st.op0 & 0b0011) !=3D 0b0011 )
>> +    {
>> +        gprintk(XENLOG_ERR, "Not single register load/store op0=3D%d",
>
> Same remark as above.
>
>> +                opcode.ld_st.op0);
>> +        goto bad_loadstore;
>> +    }
>> +
>>       /*
>>        * Refer Arm v8 ARM DDI 0487G.b, Page - C6-1107
>>        * "Shared decode for all encodings" (under ldr immediate)
>> diff --git a/xen/arch/arm/decode.h b/xen/arch/arm/decode.h
>> index 13db8ac968..b1580178eb 100644
>> --- a/xen/arch/arm/decode.h
>> +++ b/xen/arch/arm/decode.h
>> @@ -24,9 +24,27 @@
>>   #include <asm/processor.h>
>>     /*
>> - * Refer to the ARMv8 ARM (DDI 0487G.b), Section C4.1.4 Loads and Stores
>> - * Page 318 specifies the following bit pattern for
>> - * "load/store register (immediate post-indexed)".
>> + * From:
>> + * https://developer.arm.com/documentation/ddi0602/2023-12/Index-by-Enc=
oding
>> + *
>> + * Top level encoding:
>> + *
>> + *   31  30  29 28  25 24                                             0
>> + * ___________________________________________________________________
>> + * |op0 | x  x |  op1 |                                               |
>> + * |____|______|______|_______________________________________________|
>> + *
>> + * op0 =3D 0 is reserved
>> + * op1 =3D x1x0 for Loads and Stores
>> + *
>> + * Loads and Stores
>> + *
>> + *  31    28 27   26   25  24             9 8                        0
>> + * ___________________________________________________________________
>> + * |  op0   | 1 | op1 | 0 |       op2      |                          |
>> + * |________|___|_____|___|________________|__________________________|
>> + *
>> + * Load/store register (immediate post-indexed)
>>    *
>>    * 31 30 29  27 26 25  23   21 20              11   9         4       0
>>    * ___________________________________________________________________
>> @@ -35,6 +53,20 @@
>>    */
>>   union instr {
>>       uint32_t value;
>> +    struct {
>> +        unsigned int ign2:25;
>> +        unsigned int op1:4;     /* instruction class */
>> +        unsigned int ign1:2;
>> +        unsigned int op0:1;     /* value =3D 1b */
>> +    } top_level;
>> +    struct {
>> +        unsigned int ign1:9;
>> +        unsigned int op2:15;
>> +        unsigned int fixed1:1; /* value =3D 0b */
>> +        unsigned int op1:1;
>> +        unsigned int fixed2:1; /* value =3D 1b */
>> +        unsigned int op0:4;
>> +    } ld_st;
>>       struct {
>>           unsigned int rt:5;     /* Rt register */
>>           unsigned int rn:5;     /* Rn register */
>
> Cheers,

Will fix up the comments and repost.

--=20
Alex Benn=C3=A9e
Virtualisation Tech Lead @ Linaro


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 11:28:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 11:28:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675963.1051673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWx8k-0006pQ-Vw; Mon, 05 Feb 2024 11:27:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675963.1051673; Mon, 05 Feb 2024 11:27:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWx8k-0006pJ-Rw; Mon, 05 Feb 2024 11:27:54 +0000
Received: by outflank-mailman (input) for mailman id 675963;
 Mon, 05 Feb 2024 11:27:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWx8i-0006nm-Vz
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 11:27:52 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9888831c-c419-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 12:27:50 +0100 (CET)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-5112d5ab492so5690977e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 03:27:50 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f19-20020a05600c4e9300b0040d8ff79fd8sm8332123wmq.7.2024.02.05.03.27.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 03:27:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9888831c-c419-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707132470; x=1707737270; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=fYnEBcBtbki/bFVmQXYCbZtWXujkOTMfuQztPIp1uOo=;
        b=U1Qrzn4BjvHfVUjDDTvTtBxcmXYpwhuLbyK1gtPsKUgNPDk9Rp+KOhza8IzgRLcsZL
         TincstJpyZgYl8mfPxqE3C15ASlSa+c1Qr9/rlAA+Q7nK9q5PchYlr+3H2KDOqxEL766
         rUhYR6kIITSfuvISAwq6kkY/T0MOS8fGJwlDBF8jAt8szg1bW5msLVEpdCzjQiWS+6c0
         SVhniXeLG6COJ2Z2abwMBpaBSpNd5F/m/oAvKxM52nVZTLmOI3U8lzDVpH5bH/TLEDCh
         EISwr1Fk2DPHIzaDma/LioVWsoU9gLSlu2witJNgu/qU0qJxnc9SrK3gvYyeKAuifh+I
         YV+g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707132470; x=1707737270;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=fYnEBcBtbki/bFVmQXYCbZtWXujkOTMfuQztPIp1uOo=;
        b=FEQFeU7n9xFtJGHH6GGbq1mSzhuPxqjoz11VYewdi31nX1D6CZLExhR8FH9NtgLyLW
         7FhptusJFXVTVYb0kVtl8aVbZu/YLw56w+Re1rP5NZbnzK9mrpCh+5x6GK+BCyZfJQp8
         xflrQzJtotxbDM07re9Qzs04TB4XqzIZGoBounzkTmaVzolE5/B28PcggPN/UFi98mPu
         2vwUIy5yWwrafr2ZwnnkmjkHF3u2WSaef9jKghPRIrthOMHOhEZ6TIFVQgjKwGtnIBMZ
         HJayaUCFKcbn7QCOgx9WlwBOoUA9j5uWJKzPMdN+R9sa1xEuSd4vHKwz2V/thwycyXcl
         ONgw==
X-Gm-Message-State: AOJu0YyFJc5dYCVXn9aewue556RXf2hNJCcLCOmySqlPyR8FuImUp8/x
	k2bZ/hCQYzQ0KpfZh1IqrMSlEwCOddaUZYOkEJqMwj6yZjkR8V4b0LcIk7B+Ng==
X-Google-Smtp-Source: AGHT+IGDh3Y5Umv2OydBzg3FZ9m+QhkDcRk9f6GQj95jiqTPykKXhx1GVmTSHUq8ClL489xx2rcvVw==
X-Received: by 2002:a05:6512:1046:b0:510:1b51:e431 with SMTP id c6-20020a056512104600b005101b51e431mr10380360lfb.4.1707132470363;
        Mon, 05 Feb 2024 03:27:50 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVmI96vZoZDRlIMWt3+VKIN49BpCfWxW0ScQKNF3AO0X5+ekCZB4cNvEjwryzc4qjnYy+MZMKzHCWDHoMwbfMaR3d4wLxNZZ2WxxAxlsnBLcQzAahmvrJgCObBdRg+76EMhS1mAU7zMa2pzD6QKGJyPK6LchfAMLgY1B9UaLEj5nEEFNVcLZP7EjcJponO6TreZCL5+LJH+I3XHSOmoy0ySCbt8ww33hEbjUWUzChhlLuNtgQ6lNdM=
Message-ID: <d566e379-b339-44e9-8dd5-5f042e917409@suse.com>
Date: Mon, 5 Feb 2024 12:27:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/5] Constify some parameters
Content-Language: en-US
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Dario Faggioli
 <dfaggioli@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240205104504.14830-1-frediano.ziglio@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240205104504.14830-1-frediano.ziglio@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 11:45, Frediano Ziglio wrote:
> Make clean they are not changed in the functions.
> 
> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>

For this and the other 4 patches - I didn't see a cover letter; was there
none? And then please follow patch submission guidelines: Patches are to
be sent To: the list, with individuals on Cc:.

Thanks, Jan



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 11:31:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 11:31:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675966.1051683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWxBw-0000jD-DZ; Mon, 05 Feb 2024 11:31:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675966.1051683; Mon, 05 Feb 2024 11:31:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWxBw-0000j6-AX; Mon, 05 Feb 2024 11:31:12 +0000
Received: by outflank-mailman (input) for mailman id 675966;
 Mon, 05 Feb 2024 11:31:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWxBv-0000iw-UG; Mon, 05 Feb 2024 11:31:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWxBv-0003jr-Q3; Mon, 05 Feb 2024 11:31:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWxBv-0003Uu-BW; Mon, 05 Feb 2024 11:31:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWxBv-0007tQ-B8; Mon, 05 Feb 2024 11:31:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=tuoMZf8d677uMzb/J6NJmRP5HHVTPJRHGczjmx9lhgg=; b=7H89bHcaG7Vp+IEk2tSICCpQOI
	tv6RGns36+2FVZz++E/F/kPJED3L03mm4Msc0s89fmUP2RoCoByi8JtCi4/uuBxQBmrKsz95yp02S
	FtUrL4SdLZbVhbHVKdRoupJtQ9GJDnqq5/ZyloHW1BTLXrpqqaQ/tffH99zlokY77Bak=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184589-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184589: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
X-Osstest-Versions-That:
    xen=3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 05 Feb 2024 11:31:11 +0000

flight 184589 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184589/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184586
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184586
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184586
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184586
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184586
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184586
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184586
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184586
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184586
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184586
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184586
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184586
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
baseline version:
 xen                  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc

Last test of basis   184589  2024-02-05 01:54:06 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 11:31:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 11:31:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675970.1051692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWxCN-0001Fd-MG; Mon, 05 Feb 2024 11:31:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675970.1051692; Mon, 05 Feb 2024 11:31:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWxCN-0001FW-JH; Mon, 05 Feb 2024 11:31:39 +0000
Received: by outflank-mailman (input) for mailman id 675970;
 Mon, 05 Feb 2024 11:31:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWxCM-0001Do-AP
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 11:31:38 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1eb0a0c7-c41a-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 12:31:36 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-40fc654a56dso25647515e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 03:31:36 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 iv15-20020a05600c548f00b0040e9d507424sm8330753wmb.5.2024.02.05.03.31.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 03:31:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1eb0a0c7-c41a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707132695; x=1707737495; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=dLfvIavVKbCmqidYG6Sb7t9toMcY9qBVM8LAb2HEdAA=;
        b=aR1B1xsPba0/0gNl6XTQ1dUKy8QRYApBMJUNLbZ938TUDEKwK+Vf5PCRcNI1TLpiXH
         5cQSwJWrf9DTgI+SBvh2LFphgU9uGGNsyQXay7YpCezwYCwCAUgMi+hVe/tjvfGqHBkw
         l9mnvOxTFMsc+9hccQDu6X6H2zoA4qg4zIyPg3dPeQuY1nCYhibAZM2OxLbFC3mX5W8f
         5fboSvKWOx0Ts03mMNlZn66dn3nmXADJ1SawsgpxYfvzXBVj/kx6+CCsuuH8yuga6+mX
         GxL5mpXuxNMdZ8PT48FfclLR4CN3ATKBsPWXTlEOp4sx1nBqevrexY89P0hPebkE/Mop
         1QxA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707132695; x=1707737495;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=dLfvIavVKbCmqidYG6Sb7t9toMcY9qBVM8LAb2HEdAA=;
        b=WXDDARTfEY4n3p/pkZ/bnGqGWumAMJBas8bsglhxENQUKanCKUQUcO/dAiFI+nkfWK
         Lej1vFu7SHqzj903lXaP85U6JMh2O79CpFDxO+WVsXh1G3tfGWH8moiKif4TySGcdrks
         EiyDgFYpGgETsYLciMzuSTaLdHcA4ZxFoxF73bzSQ/3zdld8PjQHnW046pC9SR8gJ/MF
         5uD9zVNxwQOs+uJCYkIXcslfYDw5uNVcb8IRInGH2YOct5LbayQiz+yM/RttirFF1zKJ
         ao+/ddhuEyMQzCW9XQiQ0jtPaz2AOfwsHeqb8/22IcA7MuOudfIf8oCkBEc+3uALZNEU
         EKGw==
X-Gm-Message-State: AOJu0YzdagxQLXZmbQ50qwZQiF1rvd6jCPMhAIc7sTt5WbXndwfLxC2G
	Yb2rCw9tgUgho/DBNXoQzhDsCwUkCweRlau6F6t7ELgUmYWVEPXKugVlSnKESg==
X-Google-Smtp-Source: AGHT+IGxsgavsEHtBJNUYoGCN2eGWvGZeQury7INmJUgsEBB+Wl/9H8j6kzNWivajGwdlTDvTOAiSg==
X-Received: by 2002:a05:600c:3484:b0:40f:24e:d476 with SMTP id a4-20020a05600c348400b0040f024ed476mr4130803wmq.29.1707132695418;
        Mon, 05 Feb 2024 03:31:35 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXcFz13M0Y+hCHCypfNP5fZAVrxfe5MgrMuJmvWHyJHeyB+7ku53z5QhcM1i9APIHJOHMSc993JZe6CSZYra0UylGXuBNbb/bHNsTKKKr8BRPpYxdu5Kd7aIbIxMq3FmTOfuSQYIpRsHJmWi3fWIFJ1XiA95snK1LmJZUotDKuydI00eAHWTUeK3eafG9vcm0STsLqaqJGi/RZCQyyd3zCu6rtIsnYeBPqjLSA+iONkOZw1sneNfH8=
Message-ID: <e9961509-3453-42aa-a826-d3bea34f3a94@suse.com>
Date: Mon, 5 Feb 2024 12:31:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/5] Constify some parameters
From: Jan Beulich <jbeulich@suse.com>
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Dario Faggioli
 <dfaggioli@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240205104504.14830-1-frediano.ziglio@cloud.com>
 <d566e379-b339-44e9-8dd5-5f042e917409@suse.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d566e379-b339-44e9-8dd5-5f042e917409@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 12:27, Jan Beulich wrote:
> On 05.02.2024 11:45, Frediano Ziglio wrote:
>> Make clean they are not changed in the functions.
>>
>> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
> 
> For this and the other 4 patches - I didn't see a cover letter; was there
> none?

Even worse - while there is an In-Reply-To, they look to not have been sent
as a thread.

Jan

> And then please follow patch submission guidelines: Patches are to
> be sent To: the list, with individuals on Cc:.
> 
> Thanks, Jan
> 



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 11:59:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 11:59:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675980.1051704 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWxdE-0007cY-1P; Mon, 05 Feb 2024 11:59:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675980.1051704; Mon, 05 Feb 2024 11:59:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWxdD-0007cR-T9; Mon, 05 Feb 2024 11:59:23 +0000
Received: by outflank-mailman (input) for mailman id 675980;
 Mon, 05 Feb 2024 11:59:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GYrN=JO=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rWxdC-0007cK-MR
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 11:59:22 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ff1ce4ad-c41d-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 12:59:21 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-55fe4534e9bso3837359a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 03:59:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff1ce4ad-c41d-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1707134360; x=1707739160; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=whbs/pgXCP/L+jODEpOkjDwEFEYoQjVHY2lqzQ/c6g0=;
        b=JMn/lq1yAUHMA/DK7gz0GkFIEqdTZljcgkVtbl+fQQpjUybLAj9ltoFvWqknc3v3sh
         fbG5HNBHEV8hNgxsDucNZS0NyMDeDqMJP46Jtm9vNYK9r2xQTmXjh91q1COYpBaLG6fA
         //9re0VynjnZa4A+hxNZQAezGy0XiMOT6Qy/Mw6+GmeyFtx6urimhnT/4Fn+FU+VhbfZ
         oWtH0zyq1TvJdSNuvZl5F7gmb8517+AT4V8NN8orNUnxyWuQa244sMf57nrMSIRtnYy3
         k9szRJL3VbT4+nG92tMK/F4Fq0nbcywMOb5vJoMua0Fp9niZUySPwg0HAv/jboDUZ5tL
         PZDQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707134360; x=1707739160;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=whbs/pgXCP/L+jODEpOkjDwEFEYoQjVHY2lqzQ/c6g0=;
        b=gPW0jlD08rtTGWrU/4o8Ju/Z9fddbkmUmZBf60iERUblMuKVmEURsvvEnCRGlUihuc
         4sxH6TorZ594Mgn8ZtFSC2U9HCm+bF0ZAFX7P9HXVuu0LfK4yBfwb0g3Fs/MBnAXx9aF
         xtiVP5zuASVn0Tj+Buie1+Xm8GJQcalumoijXi/5hgwtEn5VtjsY4IFo9WDmRNA5o57v
         JZoEBpaOlaeQucAOIAkjoc+u+qY2jMQielgL496y+Q78ea3YbqBYetkUo0wLPplpYxka
         2MZQ+t2ATcDu+h53SZCwclOffRpVnhoHsLhTvNpAT+7bMaY0+qMkUouk/9GJugJzb+Q0
         UtKQ==
X-Gm-Message-State: AOJu0Yxluadnpj/7toxMplrUosUn9QAXbI0qlu9JoO3bjjJqcKrK/X8C
	nhq3mcqIjvs1BvWAqnUyJQlmDROFjaQOTlEk7JnUD+MHJ5B8EbHpWgg4b92Mnx2IcD53yi9mIGU
	DBLRvbsIgcUIprDWexWR7Q0aJQnxyTARAYcXnpQ==
X-Google-Smtp-Source: AGHT+IEUJkjdkJpq385u7vLcAxfs0pXhw1DySgWrt+NjPsIIgTFvaydTS+OvChDXX6tQ+rER0y+M9btDG3zd4t17s8I=
X-Received: by 2002:a05:6402:1d01:b0:560:7543:877 with SMTP id
 dg1-20020a0564021d0100b0056075430877mr1206408edb.3.1707134360273; Mon, 05 Feb
 2024 03:59:20 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-11-carlo.nonato@minervasys.tech> <1739dbb4-5d7e-4d84-8d4b-bfb13c4041d8@suse.com>
In-Reply-To: <1739dbb4-5d7e-4d84-8d4b-bfb13c4041d8@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Mon, 5 Feb 2024 12:59:09 +0100
Message-ID: <CAG+AhRUJysNCrUXLiu8tvh2d7oL2hNAtkM3A2-Hy73f=+PxkTA@mail.gmail.com>
Subject: Re: [PATCH v6 10/15] xen: add cache coloring allocator for domains
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Marco Solieri <marco.solieri@minervasys.tech>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Thu, Feb 1, 2024 at 4:53=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 29.01.2024 18:18, Carlo Nonato wrote:
> > @@ -157,7 +158,11 @@
> >  #define PGC_static 0
> >  #endif
> >
> > -#define PGC_preserved (PGC_extra | PGC_static)
> > +#ifndef PGC_colored
> > +#define PGC_colored 0
> > +#endif
> > +
> > +#define PGC_preserved (PGC_extra | PGC_static | PGC_colored)
> >
> >  #ifndef PGT_TYPE_INFO_INITIALIZER
> >  #define PGT_TYPE_INFO_INITIALIZER 0
> > @@ -1504,6 +1509,7 @@ static void free_heap_pages(
> >              if ( !mfn_valid(page_to_mfn(predecessor)) ||
> >                   !page_state_is(predecessor, free) ||
> >                   (predecessor->count_info & PGC_static) ||
> > +                 (predecessor->count_info & PGC_colored) ||
>
> How about a 2nd #define (e.g. PGC_no_buddy_merge) to use here and ...
>
> >                   (PFN_ORDER(predecessor) !=3D order) ||
> >                   (page_to_nid(predecessor) !=3D node) )
> >                  break;
> > @@ -1528,6 +1534,7 @@ static void free_heap_pages(
> >              if ( !mfn_valid(page_to_mfn(successor)) ||
> >                   !page_state_is(successor, free) ||
> >                   (successor->count_info & PGC_static) ||
> > +                 (successor->count_info & PGC_colored) ||
>
> ... here? That'll then also avoid me commenting that I don't see why
> these two PGC_* checks aren't folded.

Yes for me it's ok (I even suggested that in v5). Do you want this change t=
o
be merged with the previous patch? Or should they all belong to this one?

> > @@ -1943,6 +1950,161 @@ static unsigned long avail_heap_pages(
> >      return free_pages;
> >  }
> >
> > +/*************************
> > + * COLORED SIDE-ALLOCATOR
> > + *
> > + * Pages are grouped by LLC color in lists which are globally referred=
 to as the
> > + * color heap. Lists are populated in end_boot_allocator().
> > + * After initialization there will be N lists where N is the number of
> > + * available colors on the platform.
> > + */
> > +static struct page_list_head *__ro_after_init _color_heap;
> > +static unsigned long *__ro_after_init free_colored_pages;
> > +
> > +/* Memory required for buddy allocator to work with colored one */
> > +#ifdef CONFIG_LLC_COLORING
> > +static unsigned long __initdata buddy_alloc_size =3D
> > +    MB(CONFIG_BUDDY_ALLOCATOR_SIZE);
> > +size_param("buddy-alloc-size", buddy_alloc_size);
> > +
> > +#define domain_num_llc_colors(d) (d)->num_llc_colors
> > +#define domain_llc_color(d, i)   (d)->llc_colors[(i)]
>
> Nit: Unnecessary parentheses inside the square brackets.
>
> > +#else
> > +static unsigned long __initdata buddy_alloc_size;
> > +
> > +#define domain_num_llc_colors(d) 0
> > +#define domain_llc_color(d, i)   0
> > +#endif
> > +
> > +#define color_heap(color) (&_color_heap[color])
>
> Wouldn't this better live next to _color_heap()'s definition?
>
> > +static void free_color_heap_page(struct page_info *pg, bool need_scrub=
)
> > +{
> > +    unsigned int color =3D page_to_llc_color(pg);
> > +    struct page_list_head *head =3D color_heap(color);
> > +
> > +    spin_lock(&heap_lock);
> > +
> > +    mark_page_free(pg, page_to_mfn(pg));
> > +
> > +    if ( need_scrub )
> > +    {
> > +        pg->count_info |=3D PGC_need_scrub;
> > +        poison_one_page(pg);
> > +    }
> > +
> > +    pg->count_info |=3D PGC_colored;
>
> The page transiently losing PGC_colored is not an issue then (presumably
> because of holding the heap lock)? Did you consider having mark_page_free=
()
> also use PGC_preserved?

I did something similar to what it's done in unprepare_staticmem_pages():
first mark_page_free() is called and then PGC_static is added to count_info=
.

> > +    free_colored_pages[color]++;
> > +    page_list_add(pg, head);
> > +
> > +    spin_unlock(&heap_lock);
> > +}
> > +
> > +static struct page_info *alloc_color_heap_page(unsigned int memflags,
> > +                                               const struct domain *d)
> > +{
> > +    struct page_info *pg =3D NULL;
> > +    unsigned int i, color =3D 0;
> > +    unsigned long max =3D 0;
> > +    bool need_tlbflush =3D false;
> > +    uint32_t tlbflush_timestamp =3D 0;
> > +    bool scrub =3D !(memflags & MEMF_no_scrub);
> > +
> > +    spin_lock(&heap_lock);
> > +
> > +    for ( i =3D 0; i < domain_num_llc_colors(d); i++ )
> > +    {
> > +        unsigned long free =3D free_colored_pages[domain_llc_color(d, =
i)];
> > +
> > +        if ( free > max )
> > +        {
> > +            color =3D domain_llc_color(d, i);
> > +            pg =3D page_list_first(color_heap(color));
> > +            max =3D free;
> > +        }
> > +    }
>
> The apporach is likely fine at least initially, but: By going from where
> the most free pages are, you're not necessarily evenly distributing a
> domain's pages over the colors it may use, unless the domain uses its
> set of colors exclusively.

We don't find it problematic since the exclusive set of colors are to be
preferred (as per the documentation).

> > +    if ( !pg )
> > +    {
> > +        spin_unlock(&heap_lock);
> > +        return NULL;
> > +    }
> > +
> > +    pg->count_info =3D PGC_state_inuse | PGC_colored |
> > +                     (pg->count_info & PGC_need_scrub);
>
> Isn't PGC_colored already set on the page?

Yes but here I need to make sure that only PGC_state_inuse | PGC_colored ar=
e
used so an assignment seems legit.

> Together with ...
>
> > +    free_colored_pages[color]--;
> > +    page_list_del(pg, color_heap(color));
> > +
> > +    if ( !(memflags & MEMF_no_tlbflush) )
> > +        accumulate_tlbflush(&need_tlbflush, pg, &tlbflush_timestamp);
> > +
> > +    init_free_page_fields(pg);
> > +
> > +    spin_unlock(&heap_lock);
> > +
> > +    if ( test_and_clear_bit(_PGC_need_scrub, &pg->count_info) && scrub=
 )
>
> ... this, can't the above be simplified?

I tried to replicate what happens in alloc_heap_pages() where:

>  /* Preserve PGC_need_scrub so we can check it after lock is dropped. */
>  pg[i].count_info =3D PGC_state_inuse | (pg[i].count_info & PGC_need_scru=
b);

and then after the unlock the bit is tested.

> > +        scrub_one_page(pg);
> > +    else if ( scrub )
> > +        check_one_page(pg);
> > +
> > +    if ( need_tlbflush )
> > +        filtered_flush_tlb_mask(tlbflush_timestamp);
> > +
> > +    flush_page_to_ram(mfn_x(page_to_mfn(pg)),
> > +                      !(memflags & MEMF_no_icache_flush));
> > +
> > +    return pg;
> > +}
> > +
> > +static void __init init_color_heap_pages(struct page_info *pg,
> > +                                         unsigned long nr_pages)
> > +{
> > +    unsigned int i;
> > +    bool need_scrub =3D opt_bootscrub =3D=3D BOOTSCRUB_IDLE;
> > +
> > +    if ( buddy_alloc_size )
> > +    {
> > +        unsigned long buddy_pages =3D min(PFN_DOWN(buddy_alloc_size), =
nr_pages);
> > +
> > +        init_heap_pages(pg, buddy_pages);
> > +        nr_pages -=3D buddy_pages;
> > +        buddy_alloc_size -=3D buddy_pages << PAGE_SHIFT;
> > +        pg +=3D buddy_pages;
> > +    }
> > +
> > +    if ( !_color_heap )
> > +    {
> > +        unsigned int max_nr_colors =3D get_max_nr_llc_colors();
> > +
> > +        _color_heap =3D xmalloc_array(struct page_list_head, max_nr_co=
lors);
> > +        free_colored_pages =3D xzalloc_array(unsigned long, max_nr_col=
ors);
> > +        if ( !_color_heap || !free_colored_pages )
> > +            panic("Can't allocate colored heap. Buddy reserved size is=
 too low");
> > +
> > +        for ( i =3D 0; i < max_nr_colors; i++ )
> > +            INIT_PAGE_LIST_HEAD(color_heap(i));
> > +    }
> > +
> > +    if ( nr_pages )
> > +        printk(XENLOG_DEBUG
> > +               "Init color heap with %lu pages, start MFN: %"PRI_mfn"\=
n",
> > +               nr_pages, mfn_x(page_to_mfn(pg)));
>
> This message can be issued more than once. Is that really desirable /
> necessary?

No I can drop it.

> > @@ -2458,7 +2626,14 @@ struct page_info *alloc_domheap_pages(
> >      if ( memflags & MEMF_no_owner )
> >          memflags |=3D MEMF_no_refcount;
> >
> > -    if ( !dma_bitsize )
> > +    /* Only domains are supported for coloring */
> > +    if ( d && llc_coloring_enabled )
> > +    {
> > +        /* Colored allocation must be done on 0 order */
> > +        if ( order || (pg =3D alloc_color_heap_page(memflags, d)) =3D=
=3D NULL )
> > +            return NULL;
> > +    }
>
> I think I had asked before: What about MEMF_node() or MEMF_bits()
> having been used by the caller?

MEMF_node() is used for NUMA, right? I think that for the moment, since cac=
he
coloring is supported only on arm64 and NUMA is not yet supported, it's saf=
e
to ignore it.

I'm not sure I understood what MEMF_bits() is for. It restricts the allocat=
or
to return pages in some range, right?

Thanks.

> Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 12:18:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 12:18:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.675991.1051712 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWxv3-0003Ub-Gx; Mon, 05 Feb 2024 12:17:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 675991.1051712; Mon, 05 Feb 2024 12:17:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWxv3-0003UU-Da; Mon, 05 Feb 2024 12:17:49 +0000
Received: by outflank-mailman (input) for mailman id 675991;
 Mon, 05 Feb 2024 12:17:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWxv2-0003UO-GM
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 12:17:48 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 91973205-c420-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 13:17:45 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-3394b892691so1697885f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 04:17:45 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g10-20020a5d554a000000b0033ad47d7b86sm7890967wrw.27.2024.02.05.04.17.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 04:17:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 91973205-c420-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707135465; x=1707740265; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=pnLQU156qglclpTYG17iu3Nhhrp07dcXEqyoenXfh04=;
        b=emlClxIIH8Mp4+VA2S80FQyIJflDYbHJ+njOtGI+LWXFYnjo/5c34NBaxYdDETiNvo
         ZyykLisWjUxRTVBAEy08nvKgHOu5BpLLI9GhzHwq3kejESfcOtQUHyVgYaMkfYiadj4E
         6eQjrLdMKLnSW5C7OvKKKDBvIStqh68oxHFjF6pgeTNThAF1aQc/A1VXtzUxJQGS0e5j
         a+5eMVaKEasMmyoutKhXg0fWlBwWh1l8dsNEZEWXJhi15MOpv4jkg+hYh/oEleRxjlMO
         ti4rsGBgXyqS3SagkADhRRVi04qLDkR6cP5Llg18LCgvZfAlRC6/s506nzH2I1SEp8XD
         gLAg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707135465; x=1707740265;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=pnLQU156qglclpTYG17iu3Nhhrp07dcXEqyoenXfh04=;
        b=CfTgI9ptzZVKsBnwJBsKRLk6R9wNhMTjN+0Y5kmdd7Bnni2iXs89VfIila3Yli/OMP
         BCHOJLzPx4TeXNiaEYXMLH9yNo3XkMxkvOCZIsyY0sxyC61zqvDnsEIn1oPwZ4GcfRha
         zCquRHaeAjl22IAi29Yd4gt/kEzsnyUyUzGTCnBCaOzyWYwF0rCEJ1PpSVnj313E2JaJ
         6gVHuLRm4kJHEnJhzwBsWCRZfgG0Xz3wH4XikJBFDU5R/DQsWFRT+C/M2tU/Hhu+oKki
         MUBGjuRw3kXh7Szb32XVDnMgiseTNz8VkVd3/IPW+1MRjWtQzSPCmzlRFUffKmfxu6jo
         QXiQ==
X-Gm-Message-State: AOJu0YxkX2E1C55p+FCDflfdlq8Al0eIUtHckj83KDINH6hA6fhNV3E1
	f9ymQ9i9tgfavOBdrFz600FgcwF/zVjWgEm3gM7WPHTxitYQvQMP1NbVc55NgA==
X-Google-Smtp-Source: AGHT+IEmvNlfdIOqYIu2HoIMGa9WQj3AuoOf4P9j0A5ZOKhcvi96EsdNbGzOibVuIccV2Km+wAVdsQ==
X-Received: by 2002:a5d:5f4f:0:b0:33b:17f1:c70e with SMTP id cm15-20020a5d5f4f000000b0033b17f1c70emr5251349wrb.24.1707135465123;
        Mon, 05 Feb 2024 04:17:45 -0800 (PST)
Message-ID: <b3cd10f2-c3b4-49cc-8bac-28f525dd574b@suse.com>
Date: Mon, 5 Feb 2024 13:17:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Ping: Ping: [PATCH] Argo: drop meaningless mfn_valid() check
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Christopher Clark <christopher.w.clark@gmail.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <580c6c48-9dd5-4296-8696-2b40beac2bc3@suse.com>
 <5d76295f-0438-4c5d-8b2e-73a1abcc96e6@suse.com>
 <CACMJ4GbvMhDdgOp=kUeL_+_j2yS_NVU69P8Xh9QYDLUwJ31Eow@mail.gmail.com>
 <CACMJ4GbNJgsg7tts=9UQPUr9ofsXajuHz7wYO1r5LnevWEL4_g@mail.gmail.com>
 <5aea32e3-033e-4299-9088-279d23801ba6@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <5aea32e3-033e-4299-9088-279d23801ba6@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 04.01.2024 09:12, Jan Beulich wrote:
> On 23.12.2023 22:35, Christopher Clark wrote:
>> On Sat, Dec 23, 2023 at 12:47 PM Christopher Clark
>> <christopher.w.clark@gmail.com> wrote:
>>>
>>> On Sun, Dec 17, 2023 at 11:55 PM Jan Beulich <jbeulich@suse.com> wrote:
>>>>
>>>> Christopher,
>>>>
>>>> On 27.11.2023 14:55, Jan Beulich wrote:
>>>>> Holding a valid struct page_info * in hands already means the referenced
>>>>> MFN is valid; there's no need to check that again. Convert the checking
>>>>> logic to a switch(), to help keeping the extra (and questionable) x86-
>>>>> only check in somewhat tidy shape.
>>>>>
>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>
>>> Reviewed-by: Christopher Clark <christopher.w.clark@gmail.com>
>>
>> I'd like to hold off on this just yet, sorry -- the change does look
>> ok as far as a transform being applied the prior logic and the
>> necessity of the check, but with it applied, it's not obvious that it
>> handles all the page types as best that it could there, so I'd like to
>> look at this (and the previously submitted patch again) please.
> 
> I'm puzzled: This patch merely removes a pointless check. Whatever is
> lacking with it in place will have been lacking before. Also can you
> please give a rough estimate towards when you'll be getting back on
> this, or ideally on both patches?

Another month later: I'll give it this week, and without hearing back
I'll commit what there is some time next week, with the R-b you provided.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 12:28:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 12:28:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676000.1051722 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWy5E-0005ns-FI; Mon, 05 Feb 2024 12:28:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676000.1051722; Mon, 05 Feb 2024 12:28:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWy5E-0005nl-CZ; Mon, 05 Feb 2024 12:28:20 +0000
Received: by outflank-mailman (input) for mailman id 676000;
 Mon, 05 Feb 2024 12:28:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWy5C-0005mM-Ee; Mon, 05 Feb 2024 12:28:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWy5C-0004jx-CY; Mon, 05 Feb 2024 12:28:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rWy5B-0004p6-Ug; Mon, 05 Feb 2024 12:28:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rWy5B-00088X-U9; Mon, 05 Feb 2024 12:28:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=FQrx9wA+j1DoIA2deDbYqxAiCdkvAWoB7+trQSzkGnQ=; b=kyDxJlRtP0VlWWa/7Jsci76zci
	tSV/fE3N7Ai6IPR5C0oJFbQH529qJ4/ZQunm/neH6K0CnVrqcik/2RAe08RZQuxuCRNStIjKhZRy1
	Hjkzgi4nYJ//AayUTQVDGEmscyGg1+/7ZjIkzuzRGbC6K6bW3drJ7pKldbLFpNeqa0Bs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184590-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184590: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=660f8a75013c947fbe5358a640032a1f9f1eece5
X-Osstest-Versions-That:
    xen=3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 05 Feb 2024 12:28:17 +0000

flight 184590 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184590/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  660f8a75013c947fbe5358a640032a1f9f1eece5
baseline version:
 xen                  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc

Last test of basis   184560  2024-02-01 23:00:27 Z    3 days
Testing same since   184590  2024-02-05 10:02:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Paul Durrant <paul@xen.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   3f819af8a7..660f8a7501  660f8a75013c947fbe5358a640032a1f9f1eece5 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 12:38:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 12:38:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676006.1051732 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyF6-0008Hn-Cm; Mon, 05 Feb 2024 12:38:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676006.1051732; Mon, 05 Feb 2024 12:38:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyF6-0008Hg-A5; Mon, 05 Feb 2024 12:38:32 +0000
Received: by outflank-mailman (input) for mailman id 676006;
 Mon, 05 Feb 2024 12:38:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWyF4-0008GM-Hp
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 12:38:30 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 76cf4bf9-c423-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 13:38:29 +0100 (CET)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2d09faadba5so18844191fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 04:38:29 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 eo12-20020a056122388c00b004b725f7854esm1068290vkb.8.2024.02.05.04.38.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 04:38:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 76cf4bf9-c423-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707136709; x=1707741509; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3trfr1/jd87pTRlJejfRRX3tevYM6vuwk7UV2QsR7MI=;
        b=HdOZO+Vi7ZS6xW1qdmD8E8I9PdyleqGoSjqRxpX+Z3kF17ajqK26q6PTqPUr6ePwG1
         pTrg045gE2lVAiM5Tt6A+dvQMhpwbvKD9le5dkNIbfm1KcZIgGU4SmNvyp7AD1HYgmMp
         3DaD47S6fHQYEJCNzb/SuufThXDiaqTSQaDCqAeOXKIt5FiUcB8Ec2kMl9lQH23nj6dn
         z5Th0lFOcuhqe0BUJZBu1sN0laA3q8NTNJJpsXTXwlfpCrZaQZ/4LaNG5uxD6RVvdeCd
         Kh/vsAWkzsJ3sQBlvzPcid9V1zIvvbZU+sMCjbB8mYOCbiwwR+u5nnXETKpm4isX7oA0
         OV2Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707136709; x=1707741509;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3trfr1/jd87pTRlJejfRRX3tevYM6vuwk7UV2QsR7MI=;
        b=JudMBrVCod+5ALfUl61nn0OePizK5xQF5yEARmVrNOzAWeh5qnOjCnxsuhWvtx2Sdx
         eD3cUG0L5d1rOTxriU7kz0j/KzaayD/ey3+qhYekfavrJKT4RxSSSuN1f0lqQu7xgnlD
         3lr9wmPvnWaD2f5jdS0V15RbZ7ZDmukfamQiqJxO/akNAETzEHXToXS/8kPnk5JcBVWh
         qFkYTn5MPN9ZziSqRISgrwzzwzilbggjMmb7qijhX6DB+REVgudWpw2XCXF58SUWDH16
         haz1haOKBSAxaJ1haoKH74QVs1bwKIv0+ONq3a/c69QDb6k7iIfm4qGr2NtVfsfS3S0k
         t1Ng==
X-Gm-Message-State: AOJu0YwsWiclEgKhG9IPJLZYr0FA64/kcWtP+EytVHKorHjRSmZ4LCS6
	Js5URfIhlh1gld7+GgEhZzgPKdk+Z3w718ewQqPW6ou+Bi4eoT7p4F2oS057Yw==
X-Google-Smtp-Source: AGHT+IFTon0ZzyaEZE9dOphyfso7CpiT/BnWMJ5IfT8T3zeYmlCw0H5+yPGw+oe1hWdr0l9wVC4ADA==
X-Received: by 2002:a2e:a4a4:0:b0:2d0:7fea:2920 with SMTP id g4-20020a2ea4a4000000b002d07fea2920mr5529873ljm.37.1707136708726;
        Mon, 05 Feb 2024 04:38:28 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVg08DLOELWkH0AzCWtG/rxVM1Y/cmnhlsHcrWOZ3fITOnbhkZTIR5ZkTM8MiXxhMB/sf1+xwdpEZflaWJlzgbCzb24XCUGeVVj3RIAQJk2basfFN9q0lX8YORD6zt017w6fm30+Jt9pyXQsC0iFeBJjj6bu7c4grFV09ZMAA9x3yUasNbvJPzjW3/g5hmJrzNyb/LQz9G2MeSFw1yiGGuNz4EFclcLi7K/jVjaofl1kminf2P1V0tiBNavvwExcAWHq8pN1sv9E2hV0SqTkE7I/xdW5y8krmbIUYccj8Lq/XaEYfkkdUOJGhSwDdsVDkKvIdKeBefFK0Q0bxIOSwxQUTR2wLi9SYbH31q819Ojs22cGxWuE1RZKHFAhnz3LFmIaw==
Message-ID: <0ef140da-77d3-411f-8f19-eee0aaf1fa9d@suse.com>
Date: Mon, 5 Feb 2024 13:38:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 10/15] xen: add cache coloring allocator for domains
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-11-carlo.nonato@minervasys.tech>
 <1739dbb4-5d7e-4d84-8d4b-bfb13c4041d8@suse.com>
 <CAG+AhRUJysNCrUXLiu8tvh2d7oL2hNAtkM3A2-Hy73f=+PxkTA@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRUJysNCrUXLiu8tvh2d7oL2hNAtkM3A2-Hy73f=+PxkTA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 05.02.2024 12:59, Carlo Nonato wrote:
> On Thu, Feb 1, 2024 at 4:53 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>> @@ -157,7 +158,11 @@
>>>  #define PGC_static 0
>>>  #endif
>>>
>>> -#define PGC_preserved (PGC_extra | PGC_static)
>>> +#ifndef PGC_colored
>>> +#define PGC_colored 0
>>> +#endif
>>> +
>>> +#define PGC_preserved (PGC_extra | PGC_static | PGC_colored)
>>>
>>>  #ifndef PGT_TYPE_INFO_INITIALIZER
>>>  #define PGT_TYPE_INFO_INITIALIZER 0
>>> @@ -1504,6 +1509,7 @@ static void free_heap_pages(
>>>              if ( !mfn_valid(page_to_mfn(predecessor)) ||
>>>                   !page_state_is(predecessor, free) ||
>>>                   (predecessor->count_info & PGC_static) ||
>>> +                 (predecessor->count_info & PGC_colored) ||
>>
>> How about a 2nd #define (e.g. PGC_no_buddy_merge) to use here and ...
>>
>>>                   (PFN_ORDER(predecessor) != order) ||
>>>                   (page_to_nid(predecessor) != node) )
>>>                  break;
>>> @@ -1528,6 +1534,7 @@ static void free_heap_pages(
>>>              if ( !mfn_valid(page_to_mfn(successor)) ||
>>>                   !page_state_is(successor, free) ||
>>>                   (successor->count_info & PGC_static) ||
>>> +                 (successor->count_info & PGC_colored) ||
>>
>> ... here? That'll then also avoid me commenting that I don't see why
>> these two PGC_* checks aren't folded.
> 
> Yes for me it's ok (I even suggested that in v5). Do you want this change to
> be merged with the previous patch? Or should they all belong to this one?

Or make yet another small prereq patch?

>>> +static void free_color_heap_page(struct page_info *pg, bool need_scrub)
>>> +{
>>> +    unsigned int color = page_to_llc_color(pg);
>>> +    struct page_list_head *head = color_heap(color);
>>> +
>>> +    spin_lock(&heap_lock);
>>> +
>>> +    mark_page_free(pg, page_to_mfn(pg));
>>> +
>>> +    if ( need_scrub )
>>> +    {
>>> +        pg->count_info |= PGC_need_scrub;
>>> +        poison_one_page(pg);
>>> +    }
>>> +
>>> +    pg->count_info |= PGC_colored;
>>
>> The page transiently losing PGC_colored is not an issue then (presumably
>> because of holding the heap lock)? Did you consider having mark_page_free()
>> also use PGC_preserved?
> 
> I did something similar to what it's done in unprepare_staticmem_pages():
> first mark_page_free() is called and then PGC_static is added to count_info.

I had guessed this would have been served as reference, yet by saying what
you did you don't really answer my question. (Really I'm not entirely sure
the similar staticmem behavior is entirely correct.)

>>> +    if ( !pg )
>>> +    {
>>> +        spin_unlock(&heap_lock);
>>> +        return NULL;
>>> +    }
>>> +
>>> +    pg->count_info = PGC_state_inuse | PGC_colored |
>>> +                     (pg->count_info & PGC_need_scrub);
>>
>> Isn't PGC_colored already set on the page?
> 
> Yes but here I need to make sure that only PGC_state_inuse | PGC_colored are
> used so an assignment seems legit.

Well, yes, you won't be able to avoid an assignment, but couldn't you
preserve PGC_colored just like PGC_need_scrub is preserved? Or else
at least assert the flag is set, to avoid giving the impression that
right here pages can suddenly become "colored" ones? Then again them
becoming so _may_ be needed during initialization.

>> Together with ...
>>
>>> +    free_colored_pages[color]--;
>>> +    page_list_del(pg, color_heap(color));
>>> +
>>> +    if ( !(memflags & MEMF_no_tlbflush) )
>>> +        accumulate_tlbflush(&need_tlbflush, pg, &tlbflush_timestamp);
>>> +
>>> +    init_free_page_fields(pg);
>>> +
>>> +    spin_unlock(&heap_lock);
>>> +
>>> +    if ( test_and_clear_bit(_PGC_need_scrub, &pg->count_info) && scrub )
>>
>> ... this, can't the above be simplified?
> 
> I tried to replicate what happens in alloc_heap_pages() where:
> 
>>  /* Preserve PGC_need_scrub so we can check it after lock is dropped. */
>>  pg[i].count_info = PGC_state_inuse | (pg[i].count_info & PGC_need_scrub);
> 
> and then after the unlock the bit is tested.

Again I was indeed assuming that you used existing code as reference.
Yet again my question was whether that's actually what is needed here
(which then may or may not mean that it could be done differently
also there).

>>> @@ -2458,7 +2626,14 @@ struct page_info *alloc_domheap_pages(
>>>      if ( memflags & MEMF_no_owner )
>>>          memflags |= MEMF_no_refcount;
>>>
>>> -    if ( !dma_bitsize )
>>> +    /* Only domains are supported for coloring */
>>> +    if ( d && llc_coloring_enabled )
>>> +    {
>>> +        /* Colored allocation must be done on 0 order */
>>> +        if ( order || (pg = alloc_color_heap_page(memflags, d)) == NULL )
>>> +            return NULL;
>>> +    }
>>
>> I think I had asked before: What about MEMF_node() or MEMF_bits()
>> having been used by the caller?
> 
> MEMF_node() is used for NUMA, right? I think that for the moment, since cache
> coloring is supported only on arm64 and NUMA is not yet supported, it's safe
> to ignore it.

NUMA or not, I'm wary of silent ignoring of inputs. I'd rather request
that, just like with staticmem, precautions are then taken to avoid
coloring and NUMA to be used together. Recall that much like your work,
work to support NUMA is also in progress (unless it was canceled, but
not so announced).

> I'm not sure I understood what MEMF_bits() is for. It restricts the allocator
> to return pages in some range, right?

Correct.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 12:49:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 12:49:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676010.1051743 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyPX-00020c-Be; Mon, 05 Feb 2024 12:49:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676010.1051743; Mon, 05 Feb 2024 12:49:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyPX-00020V-8Q; Mon, 05 Feb 2024 12:49:19 +0000
Received: by outflank-mailman (input) for mailman id 676010;
 Mon, 05 Feb 2024 12:49:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rWyPV-00020P-CS
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 12:49:17 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f79eae0f-c424-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 13:49:14 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a2f79e79f0cso589791166b.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 04:49:14 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 lj8-20020a170907188800b00a37409166fasm3154725ejc.132.2024.02.05.04.49.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 04:49:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f79eae0f-c424-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707137354; x=1707742154; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=9tU6iOL9zEo+jHOFaAz1pgjQhaSWOvDWsIx7kDUWXzU=;
        b=Ux1BV8WsHiolyafynxGmUZoI8df+L9+JLoVfRLX5XTggGxFXNk/v1VM+C5FeKZf90X
         M1JNVFyf2B5UBzT2WmLkXafM/bHD2mmZZXBp1AO4k5E4RabQ7x6VtDFHnnPs9ACNmXAA
         u/oH30ygZ84qVd8MPwtHzSJIIdLZAol71en3caB93bXnTC4dAqzFjFkNkiKVa5Gcq2YZ
         gIJZxa2tLFMFOBtsA1UBc6zsAfLat128vVZIXoJisamMdnCTGJNx3NgGdDVXlDZ0tmQi
         ISIGsu1dNG8v574Dc5coLTGvEoF88McjnCMnaFwPpqRDsS5WaA05Td3aPlVUwyu1XX8w
         59JQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707137354; x=1707742154;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9tU6iOL9zEo+jHOFaAz1pgjQhaSWOvDWsIx7kDUWXzU=;
        b=cGX5I0yG/WWbTjTB/mWQ/rPkzwQb79PeyThcwvrrc08rmsmH/fnhbPFb5i0YEu/lr7
         a0cze1z3HJ4CPED5eAY1SwjAcX8t/0o5qxWwjVXlmYrygWIjxIoubJSBPMOVn668G+dP
         zUFzaK6UwZ4N2y/MoI6Vks8EKLRiJm2FIPMvXCMI/rtzB6lFAJ9mOuu/HokuDy+S4h+j
         BejkWWdXotAtarX/jZHyxDcTNZT3fG8dJ/xxfK2IVrF2smCorldxBNb6gbAx9jQXwJUR
         zTNpXfrmgMqWCzD+jaTU1JABiM3uVysTeLnkFMBionxOMWImaQor+Diy4vLuxGR/RLnN
         HMeQ==
X-Gm-Message-State: AOJu0Yw8I3sOWVkKnWmUhah+ZZOJH/Itj1adwP+sni0bxZnO7SXSDN96
	uZ6ILXk3qYuPJ8cny4w68lOpOKXipurgID8S+9YIO41IQEAIif1b
X-Google-Smtp-Source: AGHT+IGdz3cQFDGFDCnsBVPDgwxcgxV5p1pSQUy9X+5c5TJBrAZOCwZxwkuiAMv/u2h8HHXd8odlcw==
X-Received: by 2002:a17:907:7784:b0:a37:b1df:30ad with SMTP id ky4-20020a170907778400b00a37b1df30admr1828897ejc.46.1707137354080;
        Mon, 05 Feb 2024 04:49:14 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXP7zPhXyqaLXwRudUBhdm2w2FXV04zyYu4Idv1lFmot2guLY7n2ct6aWciGOwEY36n/bFzWD4t9+1tSjYGOxPPjwGvFdSwt1dnfAUwCWYlB3nYqZkWVty8Tq6AN1KD+Abaflb5TmQ8c7ngvYCP9RLD8f24vtW7X2HiYHhPz5p6a3+WgAcPLbbmLP3ASJ4gw+m8Tkdtw2Nn6l8V9cFRcMELw8tIVnchogJEc/aPvDF1S4FscrOcS0Fe5oY7QIguo2jE06FxGh9M+orBsCJZBBfhEyDf8NgJxPLzOtjYTYQrbT3InHfqd8U=
Message-ID: <211e7d7fb7d52d0bc7dbb12d5619a39f98d5b21c.camel@gmail.com>
Subject: Re: [PATCH v3 31/34] xen/riscv: add minimal stuff to mm.h to build
 full Xen
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Mon, 05 Feb 2024 13:49:12 +0100
In-Reply-To: <44a9a92c-ac4c-43d0-bc1e-685818e4331d@suse.com>
References: <cover.1703255175.git.oleksii.kurochko@gmail.com>
	 <4411f6af38586074b347cd6005f19f9c670faa74.1703255175.git.oleksii.kurochko@gmail.com>
	 <d347c4d9-e93b-4937-8e33-e5fbbdcd6bfb@suse.com>
	 <f3858360fba14a2c5b794bdd0400cd7ab8e66f73.camel@gmail.com>
	 <44a9a92c-ac4c-43d0-bc1e-685818e4331d@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-05 at 08:46 +0100, Jan Beulich wrote:
> On 02.02.2024 18:30, Oleksii wrote:
> > On Tue, 2024-01-23 at 14:03 +0100, Jan Beulich wrote:
> > > On 22.12.2023 16:13, Oleksii Kurochko wrote:
> > > > +/* Convert between Xen-heap virtual addresses and machine
> > > > frame
> > > > numbers. */
> > > > +#define __virt_to_mfn(va) (virt_to_maddr(va) >> PAGE_SHIFT)
> > > > +#define __mfn_to_virt(mfn) maddr_to_virt((paddr_t)(mfn) <<
> > > > PAGE_SHIFT)
> > >=20
> > > These would imo better use maddr_to_mfn() and mfn_to_maddr(),
> > > rather
> > > than
> > > kind of open-coding them. The former could also use PFN_DOWN() as
> > > an
> > > alternative.
> > We can't to as __virt_to_mfn() when is used it is usually wrapped
> > by
> > _mfn() which expect to have unsigned long as an argument.
>=20
> #define __virt_to_mfn(va)=C2=A0 mfn_x(maddr_to_mfn(virt_to_maddr(va))
> #define __mfn_to_virt(mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn)))
>=20
> ?
I had an issue with the compilation when I tried to define them in
similar way.

I'll double check again.

>=20
> > > > +/* Convert between Xen-heap virtual addresses and page-info
> > > > structures. */
> > > > +static inline struct page_info *virt_to_page(const void *v)
> > > > +{
> > > > +=C2=A0=C2=A0=C2=A0 BUG();
> > > > +=C2=A0=C2=A0=C2=A0 return NULL;
> > > > +}
> > > > +
> > > > +/*
> > > > + * We define non-underscored wrappers for above conversion
> > > > functions.
> > > > + * These are overriden in various source files while
> > > > underscored
> > > > version
> > > > + * remain intact.
> > > > + */
> > > > +#define virt_to_mfn(va)=C2=A0=C2=A0=C2=A0=C2=A0 __virt_to_mfn(va)
> > > > +#define mfn_to_virt(mfn)=C2=A0=C2=A0=C2=A0 __mfn_to_virt(mfn)
> > >=20
> > > Is this really still needed? Would be pretty nice if in a new
> > > port we
> > > could get to start cleanly right away (i.e. by not needing per-
> > > file
> > > overrides, but using type-safe expansions here right away).
> > We still need __virt_to_mfn and __mfn_to_virt as common code use
> > them:
> > =C2=A0* xen/common/xenoprof.c:24:#define virt_to_mfn(va)
> > mfn(__virt_to_mfn(va))
>=20
> Are you meaning to enable this code on RISC-V.
Yes, that is what I meant.

~ Oleksii

>=20
> > =C2=A0* xen/include/xen/domain_page.h:59:#define
> > domain_page_map_to_mfn(ptr)
> > _mfn(__virt_to_mfn((unsigned long)(ptr)))
>=20
> Hmm, yes, we should finally get this sorted. But yeah, not something
> I want
> to ask you to do.
>=20
> Jan




From xen-devel-bounces@lists.xenproject.org Mon Feb 05 12:51:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 12:51:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676014.1051753 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyRf-0003Rg-QH; Mon, 05 Feb 2024 12:51:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676014.1051753; Mon, 05 Feb 2024 12:51:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyRf-0003RZ-N3; Mon, 05 Feb 2024 12:51:31 +0000
Received: by outflank-mailman (input) for mailman id 676014;
 Mon, 05 Feb 2024 12:51:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=VsQK=JO=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rWyRe-0003RN-MW
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 12:51:30 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4608bb93-c425-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 13:51:27 +0100 (CET)
Received: from BLAPR03CA0128.namprd03.prod.outlook.com (2603:10b6:208:32e::13)
 by DM4PR12MB5134.namprd12.prod.outlook.com (2603:10b6:5:391::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.14; Mon, 5 Feb
 2024 12:51:23 +0000
Received: from BL02EPF0001A0FC.namprd03.prod.outlook.com
 (2603:10b6:208:32e:cafe::a5) by BLAPR03CA0128.outlook.office365.com
 (2603:10b6:208:32e::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.34 via Frontend
 Transport; Mon, 5 Feb 2024 12:51:23 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL02EPF0001A0FC.mail.protection.outlook.com (10.167.242.103) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Mon, 5 Feb 2024 12:51:22 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 5 Feb
 2024 06:50:34 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Mon, 5 Feb 2024 06:50:32 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4608bb93-c425-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=F/qoyHRDqIlGwgdkElfjVGEniNgEHmgVlQt2gd9k4gI1wGKefgjxdJuSYvcMNu60U+JTqhDHsrdkTmuR/9zwPU7eRLvDSjqnZNP/kqJFxPpT+RPZ8l82UYAfGLfKXaBNEqYzlps0q766azoee9m4JN0WTwzRTAMyUTODwWmHDiSKiS8ckbDwXfZhH+QVG+j3KcgTfdwVz5VieFo3IYr11PBu7J1E9SrJ9NBUWkY1vRP2N0n8ptsfIpaMUsoDbR68y1ubFqtNRY4Qg3tDy/VK1kQyAzRuQPvcSE/I31tV/bR9bBCCzwGZTOrGx8/Mf2vR0d8A8Kp+clmpzRbZPOzGOw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=oZI5TNxZT3y6TKxyWrg3hJtD8+hQ67fWO8ncRaQqWhY=;
 b=C216T5hRHzk04wH1Qo6t5sV5b1VWC6LUHuxWD2053JA9T4DAIo6PyZw8OHnbL9ohNexxavqJnVs0hu1rLkUf0+sEJ3RwZhS4pPimkwRT6iQCeQ6tWZ/1flvI3unmCvt1ZKJ/mHTEf1Lk97lSYejmasT2VsQ9E8cjYK42ripKM8inJ2fR2dcCh+qqPIWnr4JMB94b6nGJzgFWeOb4tNyB4iXhufAvDahkuEYYlWxJQI3egTF5zj0dOzNo6zcMfAa7qX/YuBhRxd8iwPBvu2xbHbUoOhlaZIsHF77QDrihjoehs3Mz81NIvqEERxIeZRO6dNYYjOcT5Q5+qgfGmJmOkA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=oZI5TNxZT3y6TKxyWrg3hJtD8+hQ67fWO8ncRaQqWhY=;
 b=Ji0Nrw/mN+zG7UCrnJJSqxczqwrSRIiaVP8+XJIxCl4BZtuG5ZqxZVBeUHtTtY6mUxkR/p9sV486XCG3j5tJE6oOdmaU0JskHizmdqTfCd7cgYeYLAsBkX6GXocX6EuiDv09xQulizZqKpVbJlGpzwXM7IfFuqJp76XQX/lMN8k=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <d6f57871-c3fc-4a51-bec8-f9a12a048186@amd.com>
Date: Mon, 5 Feb 2024 13:50:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH] xen/arm: improve handling of load/store instruction
 decoding
Content-Language: en-US
To: Julien Grall <julien@xen.org>, =?UTF-8?Q?Alex_Benn=C3=A9e?=
	<alex.bennee@linaro.org>, <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <bertrand.marquis@arm.com>, Manos Pitsidianakis
	<manos.pitsidianakis@linaro.org>
References: <20240131175043.1488886-1-alex.bennee@linaro.org>
 <e82e5a15-88e8-4d24-b872-548b608cebbc@xen.org>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <e82e5a15-88e8-4d24-b872-548b608cebbc@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FC:EE_|DM4PR12MB5134:EE_
X-MS-Office365-Filtering-Correlation-Id: 371e650a-017e-4e7e-6188-08dc26492811
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	oF1hBRAZ40n+Ci/v7tsgXxkdTKkKHGvbS1vNCObvArcWAlV93SP1v/uITB/l76vrHztfkjQ53vYJnFpk1S1Y89M4yY8VzojXzZu6GUBHSxLP4CGec1ct7r3aBRPEIhn9FCdFH5vZZb6XlTp3T6yE7ST7hODS2Xwipi5F6hVQMCyM9UNZ8reR6993Rolr58vLh86KtEQC4N+VsKSPhOZK+8iWnGqqB0QmGxgOH/VMqWFcxe/nv5mbOCmak1bUvlnYaU9RT/wOfBa/B2zzM3JGBk7SYNoQp1FuGOBIlmR45h1Fk5TwYc7Jkn4GugJLrRIs9aoV+eUO4Qjv8v2zL5BWpC4bcs59e6ApNcqf9pgFREtX5EXtNAzftTCLTxHY3muk1NcU6tvrXSXrIkE1KdxVvNuqmIf+lrTiP3kmHUMGJosF6OMX0DzMRx7H+Kfsb7XieNbnC4prwZuLNbZICoYiL7Ekl6I6/fYM6gdsvv/6elTArZstZa9wFKC5KaSa4u+fP+sodTYfsGXB/f1iyrKm25cE4wUJygZRxUWvZWJFnnQjNiJrMqdnVh12PKNtGUbh80Tl4q5W2Y7VFx0jAZQIP27k9y9dR9MZLn3+B4OnJt98alfj89DjF7RQHD02IYoqTd5BqMoO0GtpEAMyFMhQaXPmEdRrZAhZcf1+b+w4LG/LRAnQhlbvLMtdrHIzMR2mArF3EIMvAwJYCz+10IbEQ9IhtWuYs9LY9zEQUEMioL+7JcLsHmdQzluij6SNjxheZrvHsYYuscJDCVtTPOtiW4wKQg26GD/zxprg9uetRcU=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(136003)(346002)(39860400002)(396003)(230922051799003)(230273577357003)(82310400011)(64100799003)(186009)(451199024)(1800799012)(40470700004)(46966006)(36840700001)(40460700003)(40480700001)(31686004)(41300700001)(478600001)(31696002)(86362001)(53546011)(36756003)(426003)(336012)(66574015)(2616005)(81166007)(82740400003)(47076005)(83380400001)(26005)(356005)(2906002)(5660300002)(316002)(70206006)(54906003)(44832011)(16576012)(36860700001)(8936002)(4326008)(8676002)(70586007)(110136005)(43740500002)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Feb 2024 12:51:22.5994
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 371e650a-017e-4e7e-6188-08dc26492811
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL02EPF0001A0FC.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5134



On 05/02/2024 11:42, Julien Grall wrote:
> 
> 
> Hi Alex,
> 
> On 31/01/2024 17:50, Alex Bennée wrote:
>> While debugging VirtIO on Arm we ran into a warning due to memory
>> being memcpy'd across MMIO space. While the bug was in the mappings
>> the warning was a little confusing:
>>
>>    (XEN) d47v2 Rn should not be equal to Rt except for r31
>>    (XEN) d47v2 unhandled Arm instruction 0x3d800000
>>    (XEN) d47v2 Unable to decode instruction
>>
>> The Rn == Rt warning is only applicable to single register load/stores
>> so add some verification steps before to weed out unexpected accesses.
>>
>> I updated the Arm ARM reference to the online instruction decoding
>> table which will hopefully be more stable than the Arm ARM section
>> numbers.
NIT: commit msg should be written in imperative mood.

> 
> I am not sure if the links to the Arm websites are stable. But from
> past, experience, URL tends to disappear after a while. This is why we
> went with the section + the Arm spec.
> 
> This also has the advantage that we can check any differences between
> version. So my preference is to stick the Arm ARM reference. Bertrand,
> Michal, Stefano, any opinions?
I would prefer if we refer to Arm ARM instead of links to Arm websites.

Also, looking at Arm ARM J.a C4.1.88 (Loads and Stores) the encoding is a bit
different compared to that website and includes op3 and op4 (op2 is 24:23).

~Michal


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 12:59:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 12:59:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676018.1051762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyZ9-0004VF-Fk; Mon, 05 Feb 2024 12:59:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676018.1051762; Mon, 05 Feb 2024 12:59:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyZ9-0004V8-Ct; Mon, 05 Feb 2024 12:59:15 +0000
Received: by outflank-mailman (input) for mailman id 676018;
 Mon, 05 Feb 2024 12:59:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rWyZ8-0004V2-0Q
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 12:59:14 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5c2183a8-c426-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 13:59:13 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-55a90a0a1a1so5471982a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 04:59:12 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 i23-20020a056402055700b0055fba4996d9sm3885294edx.71.2024.02.05.04.59.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 04:59:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5c2183a8-c426-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707137952; x=1707742752; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=aekdhF8+mBnuPMS0XztHvPgSeHTu1C66eam1fhIlJ2A=;
        b=l55ROmaGy/kvQJZwzIoak4ZCVflHriz9+RDvlXUgMDwlG/8eWTN87f8jSxHx4FbvUG
         uL5EWELSPIHHOTP8whdc8WJjO9NOe59jnNRxLEAl4GhR3tHqBPV+C1BlCbJWHhOR+x0x
         Ky3Ue7LlyhdpWSdQcbtr+7WrR5Wea5vi+BakcE6/OitlewKQgfT/ToDi04wCqCrev67T
         XAlH57HVCCemP9F63W7mozwqqIBhmLaif2i6QvNCckS2H8t7NqU7ar1NiQO98ZTD6T3s
         ARyzKicJ3gWjx6XWqlaDoD1ENo0WUQ6te4GcMAN05u2cnAac7kd2ovQze+P5vHI+Q9VM
         NDJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707137952; x=1707742752;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=aekdhF8+mBnuPMS0XztHvPgSeHTu1C66eam1fhIlJ2A=;
        b=ZIYl1X2MKSMmv+QfEfvw5wFK1Uwob24jPYZbgwCRj84RLamIndMB4OT6yJebUmnIvw
         KTjn15FoySJTdW6ep8HGtUidbJUL1OCy+OI1mSuA+LCtLZXGLcnFnq56JE39iObAThxT
         hZfxT9rFcBGjk9ijxAIgR+HUffJk5YYIc0VXgzfixN/j7brC2obRRD/DleqQzw8I4hXq
         esqignhTUXjGC4tuMHdGEPJMEQ1RZYGU6WaNsnagQSPQSovS0CqJqYzPEglJ5WUKw6Vr
         7W3wdH1YPgGQUgpHsMHanXJVa2+U8x3RrcLpK849dIQ7boptgbh1GoKF1me6nazTxOlb
         okkA==
X-Gm-Message-State: AOJu0YwuwBvIKn+g64SIhE466SDBB2Y1gpMin/xYELVXE4F/bcTGXOFh
	Pcp382kkpNtmuLgVTx0z+2LFA4jdp8VKKODohvdUdZMD9PPPilY9
X-Google-Smtp-Source: AGHT+IHHyvUujXCiv2azAf+10uecScQ75UBjG9WbKaCQVDxIjJFZ6EquwEjMLBTLfNqq1PBOQ6CA/Q==
X-Received: by 2002:aa7:c0d1:0:b0:55f:4602:bf7c with SMTP id j17-20020aa7c0d1000000b0055f4602bf7cmr5332115edp.28.1707137952302;
        Mon, 05 Feb 2024 04:59:12 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVTGg9rydxaYacFBKj4gsvGjy6FTtbeNsKL9yo5bxODu3jDv6jgRN059K/celqb1ZaiTBGOmOwwUtPWcAwOZPla1qSDMXqdofRBvgRFP2WGMomTOxT3giJ8KqNNz9Dw1AwQv4LODrWFdE7akYsqGd0OPhiGezhYgKqjYZM0OtEM5YitEBvD+VWMvqC4gBRMEeT6b0uabR/1q8mwkCk=
Message-ID: <1da6b3acbc07490323c007188e2f34500149b089.camel@gmail.com>
Subject: Re: [PATCH v7 5/7] xen/asm-generic: introduce generic device.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Date: Mon, 05 Feb 2024 13:59:10 +0100
In-Reply-To: <392e3c39-e984-4e4c-8ca9-f4a2f2ea86cf@suse.com>
References: <cover.1706281994.git.oleksii.kurochko@gmail.com>
	 <621ff5bd992ea8e6202ec03fa52c0e09aacd8f83.1706281994.git.oleksii.kurochko@gmail.com>
	 <392e3c39-e984-4e4c-8ca9-f4a2f2ea86cf@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Wed, 2024-01-31 at 15:54 +0100, Jan Beulich wrote:
> On 26.01.2024 16:42, Oleksii Kurochko wrote:
> > Arm, PPC and RISC-V use the same device.h thereby device.h
> > was moved to asm-generic.
>=20
> It's not "move" anymore with the splitting off of the Arm and PPC
> parts. For reasons mentioned before, I'm not exactly happy with
> it not being a move anymore, but I expect you were asked to split.
>=20
> > Arm's device.h was taken as a base with
> > the following changes:
> > =C2=A0- #ifdef PCI related things.
>=20
> Well, not really, with ...
>=20
> > =C2=A0- #ifdef ACPI related things.
> > =C2=A0- Rename #ifdef guards.
> > =C2=A0- Add SPDX tag.
> > =C2=A0- #ifdef CONFIG_HAS_DEVICE_TREE related things.
> > =C2=A0- #ifdef-ing iommu related things with CONFIG_HAS_PASSTHROUGH.
> >=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > Changes in V7:
> > =C2=A0- keeping DEVICE_PCI_HOSTBRIDGE available for every build based o=
n
> > the reply:
> > =C2=A0=C2=A0=C2=A0
> > https://lore.kernel.org/xen-devel/926a5c12-7f02-42ec-92a8-1c82d060c710@=
xen.org/
>=20
> ... this. Specifically ...
>=20
> > --- /dev/null
> > +++ b/xen/include/asm-generic/device.h
> > @@ -0,0 +1,162 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +#ifndef __ASM_GENERIC_DEVICE_H__
> > +#define __ASM_GENERIC_DEVICE_H__
> > +
> > +#include <xen/stdbool.h>
> > +
> > +/*
> > + * DEV_TYPE_MAX is currently not in use, but it was added because
> > the enum may
> > + * be empty when !HAS_DEVICE_TREE and !HAS_PCI, which could lead
> > to
> > + * a compilation error.
> > + */
> > +enum device_type
> > +{
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +=C2=A0=C2=A0=C2=A0 DEV_DT,
> > +#endif
> > +
> > +#ifdef CONFIG_HAS_PCI
> > +=C2=A0=C2=A0=C2=A0 DEV_PCI,
> > +#endif
>=20
> ... this is now inconsistent with ...
>=20
> > +=C2=A0=C2=A0=C2=A0 DEV_TYPE_MAX,
> > +};
> > +
> > +enum device_class
> > +{
> > +=C2=A0=C2=A0=C2=A0 DEVICE_SERIAL,
> > +=C2=A0=C2=A0=C2=A0 DEVICE_IOMMU,
> > +=C2=A0=C2=A0=C2=A0 DEVICE_INTERRUPT_CONTROLLER,
> > +=C2=A0=C2=A0=C2=A0 DEVICE_PCI_HOSTBRIDGE,
>=20
> ... this. Either we want PCI-related #ifdef-ary, or we don't. There
> shouldn't be a mix (unless there's a good reason).
Agreed, I overlooked the fact that we now have inconsistency.

Then it is needed to update remove CONFIG_HAS_PCI in device_type enum
too.

>=20
> Also the use of blank lines inside the earlier enum would better be
> consistent.
>=20
> > +=C2=A0=C2=A0=C2=A0 /* Use for error */
> > +=C2=A0=C2=A0=C2=A0 DEVICE_UNKNOWN,
> > +};
> > +
> > +struct dev_archdata {
> > +#ifdef CONFIG_HAS_PASSTHROUGH
> > +=C2=A0=C2=A0=C2=A0 void *iommu;=C2=A0=C2=A0=C2=A0 /* IOMMU private dat=
a */
> > +#endif
> > +};
> > +
> > +/* struct device - The basic device structure */
> > +struct device
> > +{
> > +=C2=A0=C2=A0=C2=A0 enum device_type type;
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +=C2=A0=C2=A0=C2=A0 struct dt_device_node *of_node; /* Used by drivers =
imported
> > from Linux */
> > +#endif
> > +=C2=A0=C2=A0=C2=A0 struct dev_archdata archdata;
> > +#ifdef CONFIG_HAS_PASSTHROUGH
> > +=C2=A0=C2=A0=C2=A0 struct iommu_fwspec *iommu_fwspec; /* per-device IO=
MMU
> > instance data */
> > +#endif
> > +};
> > +
> > +typedef struct device device_t;
> > +
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +
> > +#include <xen/device_tree.h>
> > +
> > +#define dev_is_dt(dev)=C2=A0 ((dev)->type =3D=3D DEV_DT)
> > +
> > +/**
> > + *=C2=A0 device_init - Initialize a device
> > + *=C2=A0 @dev: device to initialize
> > + *=C2=A0 @class: class of the device (serial, network...)
> > + *=C2=A0 @data: specific data for initializing the device
> > + *
> > + *=C2=A0 Return 0 on success.
> > + */
> > +int device_init(struct dt_device_node *dev, enum device_class
> > class,
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 const void *data);
> > +
> > +/**
> > + * device_get_type - Get the type of the device
> > + * @dev: device to match
> > + *
> > + * Return the device type on success or DEVICE_ANY on failure
> > + */
> > +enum device_class device_get_class(const struct dt_device_node
> > *dev);
> > +
> > +#define DT_DEVICE_START(_name, _namestr,
> > _class)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
>=20
> Would be really nice if in the course of generalization these leading
> underscores would also disappear. Yes, that'll require changing two
> of the names more than just to drop the underscores, to account for
> ...
>=20
> > +static const struct device_desc __dev_desc_##_name
> > __used=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +__section(".dev.info") =3D
> > {=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 .name =3D
> > _namestr,=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 .class =3D
> > _class,=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
>=20
> ... these field names.
>=20
> Also there's a strack backslash on the last line above.
Sure, I'll drop the leading underscores. Thanks.

~ Oleksii
>=20
> Both comments similarly apply to the ACPI stuff further down.
>=20
> > +#define
> > DT_DEVICE_END=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +};
> > +
> > +#else /* !CONFIG_HAS_DEVICE_TREE */
> > +#define dev_is_dt(dev) ((void)(dev), false)
> > +#endif /* CONFIG_HAS_DEVICE_TREE */
> > +
> > +#ifdef CONFIG_HAS_PCI
> > +#define dev_is_pci(dev) ((dev)->type =3D=3D DEV_PCI)
> > +#else
> > +#define dev_is_pci(dev) ((void)(dev), false)
> > +#endif
> > +
> > +struct device_desc {
> > +=C2=A0=C2=A0=C2=A0 /* Device name */
> > +=C2=A0=C2=A0=C2=A0 const char *name;
> > +=C2=A0=C2=A0=C2=A0 /* Device class */
> > +=C2=A0=C2=A0=C2=A0 enum device_class class;
> > +
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +
> > +=C2=A0=C2=A0=C2=A0 /* List of devices supported by this driver */
> > +=C2=A0=C2=A0=C2=A0 const struct dt_device_match *dt_match;
> > +=C2=A0=C2=A0=C2=A0 /*
> > +=C2=A0=C2=A0=C2=A0=C2=A0 * Device initialization.
> > +=C2=A0=C2=A0=C2=A0=C2=A0 *
> > +=C2=A0=C2=A0=C2=A0=C2=A0 * -EAGAIN is used to indicate that device pro=
bing is
> > deferred.
> > +=C2=A0=C2=A0=C2=A0=C2=A0 */
> > +=C2=A0=C2=A0=C2=A0 int (*init)(struct dt_device_node *dev, const void =
*data);
> > +
> > +#endif
> > +};
> > +
> > +#ifdef CONFIG_ACPI
> > +
> > +struct acpi_device_desc {
> > +=C2=A0=C2=A0=C2=A0 /* Device name */
> > +=C2=A0=C2=A0=C2=A0 const char *name;
> > +=C2=A0=C2=A0=C2=A0 /* Device class */
> > +=C2=A0=C2=A0=C2=A0 enum device_class class;
>=20
> I understand it's this way on Arm right now, and I'm also not going
> to insist that you do anything about it right here, but it's still
> odd that struct device_desc doesn't simply have a union to cover for
> both DT and ACPI.
>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:03:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:03:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676023.1051772 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyd1-0006Ao-V7; Mon, 05 Feb 2024 13:03:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676023.1051772; Mon, 05 Feb 2024 13:03:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyd1-0006Ah-SD; Mon, 05 Feb 2024 13:03:15 +0000
Received: by outflank-mailman (input) for mailman id 676023;
 Mon, 05 Feb 2024 13:03:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rWyd0-0006Ab-8M
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:03:14 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ead80a6b-c426-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 14:03:12 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2cf3ed3b917so53760481fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:03:12 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 s11-20020a05651c048b00b002d0a98330b3sm463428ljc.108.2024.02.05.05.03.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 05:03:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ead80a6b-c426-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707138192; x=1707742992; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=DjuTW+h/eS6V5ntAPTSQrDbv/oSKHjDglZUV3FxC1jM=;
        b=DPtyMCKFCul2mDVzM1cZ8pQH+vTov+Ut5VEZPlsTwKWzM0kDDu5ldLvIeYhSEzk6Gn
         6yKgG2kw44eweqCi56unQl6D9O11F5qgBzsDAkerjwfiAKNn9zpJBbE1pkcdH+2ch0+g
         2UdtrzlTLBFh225pNXp+m+jL8jn/mNm2WzuHRJRRVeS5N3A2wO6JZ9TSDbY2RUavjac2
         ABbA4qnCXO/YawS0D7eKeRUyw4fntRKb2dhNojrJGayxw4NdQGp4r1J81sRZbZHGj6rB
         hOXX2jK/DV/Yu96luRt50Q5jfFL37xX1ZjNCdI15UjB5XG21QUyD8Gx3TpZfAVmjNBhU
         dkbg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707138192; x=1707742992;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=DjuTW+h/eS6V5ntAPTSQrDbv/oSKHjDglZUV3FxC1jM=;
        b=WlYmY9B4u3DlSsuCeRzl1oX1M3jGvNQBttxttEUN2CY7DzwlLT92ksPTycvB3Vl2Jg
         HGAj+yn1/mhoH8RugfwVwFqr0QP8aIU7qw3zVc5RgoFUt+ba7YiqKhU9OGoua2mR8jWK
         m+Kw5OzZ2LwVvDz19N5B3uaGazbaTLcdDO4eTWwbjOM3XAn1Tilo21oNt7RDDrwcFBdc
         /Za6Ok+QvyOMwgonkpx5jwldxxmtd/kG6+G3xSJWdhL4lW1Ivs6adiTLImjlzz0z2bXN
         +IP5oHys9c4Y2o8vWZeNS8vlBkW6dINSSjRs/qTTGGiGLfe+DIEg2Ym+PimZs8z/Znqn
         SMGg==
X-Gm-Message-State: AOJu0Yw0W4U1vapE9dwJCJrn8DoPcGDHVgG56eYGawku1WfkiOOMiqwO
	of2DpYLAMUMpKZjIa2KkZqvcLNVasZEVNr1kMARwxPcB62J1z2wv
X-Google-Smtp-Source: AGHT+IFRnFKuM/mN9Me7PDjUhz8T6c505MdFEUD0vq0fOyotGu4evIe57yl2pXJIdCuSa8kpX87aOQ==
X-Received: by 2002:a2e:9e10:0:b0:2cd:4883:6e25 with SMTP id e16-20020a2e9e10000000b002cd48836e25mr9898144ljk.50.1707138191557;
        Mon, 05 Feb 2024 05:03:11 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVWS/vbF0cVXfaf5vHhyYPt3XLsT4i5rPvZljThiZjDFUg5R/PdBtINOjB36uMU5DYsJTYXzTvRCAGs69pq/bm4gl0DlgbYTRSc5hRV2cXKZxII6e6oftGXyhejJHMkrB+ALQmzGbGaPXzsTvRAO/+bJMs/r2JzKmjfZsS+wylOxYc59tNWkcyHrlkvuNvf++eBgsyGcELLeNgwdFUoGz3l5NqtMmzLUNLrxQ==
Message-ID: <d26713350e9e8f584450acf4f102436a41217028.camel@gmail.com>
Subject: Re: [PATCH v7 6/7] xen/arm: switch Arm to use asm-generic/device.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>,  Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, xen-devel@lists.xenproject.org
Date: Mon, 05 Feb 2024 14:03:10 +0100
In-Reply-To: <fdbfe86f-3b7c-4e35-90dc-ac64ee94fcd1@suse.com>
References: <cover.1706281994.git.oleksii.kurochko@gmail.com>
	 <d5d2b0515516f0554a0532ff4d4fbd9c704e0a1b.1706281994.git.oleksii.kurochko@gmail.com>
	 <fdbfe86f-3b7c-4e35-90dc-ac64ee94fcd1@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Wed, 2024-01-31 at 15:56 +0100, Jan Beulich wrote:
> On 26.01.2024 16:42, Oleksii Kurochko wrote:
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> I'm not an Arm maintainer, but if I was I wouldn't let you get away
> with
> an empty description here. Specifically at least ...
>=20
> > --- a/xen/arch/arm/device.c
> > +++ b/xen/arch/arm/device.c
> > @@ -16,7 +16,10 @@
> > =C2=A0#include <xen/lib.h>
> > =C2=A0
> > =C2=A0extern const struct device_desc _sdevice[], _edevice[];
> > +
> > +#ifdef CONFIG_ACPI
> > =C2=A0extern const struct acpi_device_desc _asdevice[], _aedevice[];
> > +#endif
> > =C2=A0
> > =C2=A0int __init device_init(struct dt_device_node *dev, enum
> > device_class class,
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 const=
 void *data)
> > @@ -45,6 +48,7 @@ int __init device_init(struct dt_device_node
> > *dev, enum device_class class,
> > =C2=A0=C2=A0=C2=A0=C2=A0 return -EBADF;
> > =C2=A0}
> > =C2=A0
> > +#ifdef CONFIG_ACPI
> > =C2=A0int __init acpi_device_init(enum device_class class, const void
> > *data, int class_type)
> > =C2=A0{
> > =C2=A0=C2=A0=C2=A0=C2=A0 const struct acpi_device_desc *desc;
> > @@ -61,6 +65,7 @@ int __init acpi_device_init(enum device_class
> > class, const void *data, int class
> > =C2=A0
> > =C2=A0=C2=A0=C2=A0=C2=A0 return -EBADF;
> > =C2=A0}
> > +#endif
>=20
> ... this new #ifdef-ary would want justifying, imo.
It was added because all ACPI-related things are under #ifdef
CONFIG_ACPI. Therefore, if CONFIG_ACPI is disabled, it will result in a
compilation error.

Perhaps, you are right; it would be better to include this information
in the commit description.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:25:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:25:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676031.1051782 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyyD-0001L3-Q5; Mon, 05 Feb 2024 13:25:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676031.1051782; Mon, 05 Feb 2024 13:25:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWyyD-0001Kw-NN; Mon, 05 Feb 2024 13:25:09 +0000
Received: by outflank-mailman (input) for mailman id 676031;
 Mon, 05 Feb 2024 13:25:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWyyC-0001Kq-R2
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:25:08 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fa2a9a7b-c429-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:25:06 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-40fdd11475aso5040365e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:25:06 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 q14-20020a05600c46ce00b0040ebf603a89sm8663304wmo.11.2024.02.05.05.25.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:25:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa2a9a7b-c429-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707139506; x=1707744306; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=BbMP18wcQMyz4HKulVAs9H/YyQXXuulGkgiEx5VK0Lk=;
        b=dt8bSGiHYWFITQ9BtFJhDd3ndguwCU7n/oRE1+a/Y3jLN1d4afEqzNO+6mJoB+RX9g
         EDZ65QLuFujdVfvgtY2fiDZs3FPMPdJU5wIpTm0b/fWpop7jyQPYOUBLfm/exanN3JqI
         1Yi3XBawUyYasb0thtiqQuKxzOD/ZnmNVn1REHZkCOpeqK90pHWQyGJMfYArmxLW1+71
         CsZfvEGiJrkgUI6sEYpJr9IUO0exUF5e7To+KVjBatOTpXER7i19uw9ANv4pppvQPpHg
         ILYf4EcycgPFUQAo/nYNFKkfkATLMj6WPrklDZYLbkbkstmOSxAdW+GED0yijQW8h+A+
         BmNg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707139506; x=1707744306;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=BbMP18wcQMyz4HKulVAs9H/YyQXXuulGkgiEx5VK0Lk=;
        b=atXjiZqXhEN7wZ/3ASpLc0lvop/Qg0LVPgx/uR9GpWBZgrVJ4EAZd4FqVqiIc1MDDO
         K3sVhmmKgoQF9q8mc/ZpekT031FH0WyRnZsydut7F4cK2XAsuPn7uuyK1a9dwDC6mGdq
         9l6KgPd5LX1ej7s8vf1jS45YzHqs3VnD7z/JHcWYR59EHF/r2e8oET/6+2CIgjY5DwwU
         OJC4kc+4v8CjWgtGs3pFVClXG+dF+WQmf0Mf+5SzAr7jJwa7GXxMsumYkwkIfoBeAGBR
         o6XwKbZWDWL3S294fkhnYG8sH3ivEwYgG0JDHi45rS16783SDjhRpYc7OJlhUC09l/o4
         zfbw==
X-Gm-Message-State: AOJu0Yw182FCKZvg+zW3i7N0aI/Aqes2gt9Y11yLPkIlDdMPWte//Qwl
	v4dYE/oCwG/Fo9HXlhQTIB55lCEQNir7WkV/STLjlN/4yOdx0vTOctIyrk2EWpmhJYIArnO7tos
	=
X-Google-Smtp-Source: AGHT+IE6wIvlKDUdtbdGxzMztCl7pYvvP+4xpzP8xnOD95HgtZjgDqMWTRvv+j9sBG/xcRoOXDLydg==
X-Received: by 2002:a05:600c:2159:b0:40e:f632:723d with SMTP id v25-20020a05600c215900b0040ef632723dmr4805255wml.16.1707139506100;
        Mon, 05 Feb 2024 05:25:06 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCV3AxxSVuzE8FTDwQJn+OA8JEtMQ25Q79HN6i7gf5+Tj0CDd+LCef6tgn4FAw5mgj3zIHvRMH1VhL9lLV62RyO6d2BCBzyyEwNG2jzOcymTxzI7O87rMex6sisZSKKGxBmu1ErXUmFO+KjR1eupFWgecQ4/m/bBey+nejdx
Message-ID: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Date: Mon, 5 Feb 2024 14:25:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v3 0/8] limit passing around of cpu_user_regs
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Unlike (synchronous) exception handlers, interrupt handlers don't normally
have a need to know the outer context's register state. Similarly, the vast
majority of key handlers has no need for such.

1: serial: fake IRQ-regs context in poll handlers
2: keyhandler: drop regs parameter from handle_keyregs()
3: serial: drop serial_rx_fn's regs parameter
4: PV-shim: drop pv_console_rx()'s regs parameter
5: serial: drop serial_[rt]x_interrupt()'s regs parameter
6: IRQ: drop regs parameter from handler functions
7: x86/APIC: drop regs parameter from direct vector handler functions
8: consolidate do_bug_frame() / bug_fn_t

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:27:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:27:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676035.1051793 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz0a-0001xw-9e; Mon, 05 Feb 2024 13:27:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676035.1051793; Mon, 05 Feb 2024 13:27:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz0a-0001xp-6U; Mon, 05 Feb 2024 13:27:36 +0000
Received: by outflank-mailman (input) for mailman id 676035;
 Mon, 05 Feb 2024 13:27:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWz0Y-0001wW-CR
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:27:34 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 517f0cfd-c42a-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:27:33 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-40efcb37373so40352735e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:27:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020a5d67ca000000b0033b3cf1ff09sm2278382wrw.29.2024.02.05.05.27.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:27:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 517f0cfd-c42a-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707139652; x=1707744452; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=ozjB5J6HgFQRbYi1xrcNJWW3dzIQpHOtBsFN5IPmgeU=;
        b=N75r1lEm7ZV9bxdme4uvOMyOHqMFOzF7yv2Tgdrszlyi/i5foX1lCjC/f3Au3suE81
         0092pC5hJqJm500JFPWSOb9aO3Q9BUVSArWmXKyzrgkmJIpt+BwLUoSpEHOZ1p2tAMH/
         jfTTRAb7HdnPG7qT8NuDsynUgR34iNKY7TbzlgLTPy9bX1DwxAl+S4KU9WUK/Oi0908f
         1rl6cone2cyWgvLIrSw/Rq+0MHAQySZmVu6F7jjGl+aTYLTgOG4Yz2zWjbcusiB/GBtB
         Lp5ywGM+AKPR/HDbr3Sbaa0mHxYyQP7Q8zhuyAV+o5gvksHTdBKydCk/62f9c2gSrzOw
         JObQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707139652; x=1707744452;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=ozjB5J6HgFQRbYi1xrcNJWW3dzIQpHOtBsFN5IPmgeU=;
        b=omVhkbYJRBt5CiVuwoX0P8eUKOxQ4aAhKkAQYP4PI+qvXtABuL5QTnP1neXeJliYqd
         HrzDwjUEkTzRT+3gHAUZZXoodOkRmimzQRfgcb7Ip460OriioHmwCk28UqfP6ZrNDPv5
         a6kgAueE5D1s/oumIAeyIULIDMUbV5kyTxgi2AbRSs4z/OuUamJLwrDlWusR3dUrj+SG
         RBBQ63rVmxtukxlnoXyWHLZMp3/W9oGIpeGXiTdOKHpc2mfls8qPH6tvgFZCKmzmEO6n
         L71TUdO5UwXUGGrIy8DN0hC3R6TgbTNIly8QKuvQpXfKLq5m9DpTzoIA5L9AXyqiod44
         VAOw==
X-Gm-Message-State: AOJu0YwKKh3z7hi9Pxcyjew9aJwx//e6qctGQD2IJfCJSfDQlP6Eoqd6
	IRSJc6EuzcZni2qNCzugWt5JoCEN7x95VCjmxp/5EswpWMxnPZvuHZjomBhliUrNtb+Q0Qp3kAM
	=
X-Google-Smtp-Source: AGHT+IGc3yD9tlPmWCc3rLBakfLjqIRUvsu9Jtp9Vlhp7n5P7rswL2RvRIzw6bwThJ4Qnw9t60xh9w==
X-Received: by 2002:adf:fccf:0:b0:33b:1bf4:de87 with SMTP id f15-20020adffccf000000b0033b1bf4de87mr6880273wrs.24.1707139652555;
        Mon, 05 Feb 2024 05:27:32 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWN/NZWe1fmN3Ql8B3HxMKepciFXQzWxSSKE0lF+fPsQqQGpFYQvehBVCvgG1cYu8iiDVvdRU6jC2HNAOMkMIL0I9j93NgMDarT8SBQIGFODpqRguKxjpMuEkhw1E/RAv1wT9iSG/tNGPj+T5Rs7xTtpEj7fQvh07ft0TkoLLiebm4kxcDKtVX41UpMYTR77p4xrBi8Wp5gW//wYsXO
Message-ID: <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
Date: Mon, 5 Feb 2024 14:27:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Marek Marczykowski <marmarek@invisiblethingslab.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

In preparation of dropping the register parameters from
serial_[rt]x_interrupt() and in turn from IRQ handler functions,
register state needs making available another way for the few key
handlers which need it. Fake IRQ-like state.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsistent with
other console poll functions we have, and it's unclear whether that's
actually generally correct.

Andrew suggested to move set_irq_regs() to BUGFRAME_run_fn handling;
it's not clear to me whether that would be (a) correct from an abstract
pov (that's exception, not interrupt context after all) and (b) really
beneficial.
---
v2: New.

--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -1253,6 +1253,7 @@ static void cf_check _ehci_dbgp_poll(str
     unsigned long flags;
     unsigned int timeout = MICROSECS(DBGP_CHECK_INTERVAL);
     bool empty = false;
+    struct cpu_user_regs *old_regs;
 
     if ( !dbgp->ehci_debug )
         return;
@@ -1268,12 +1269,17 @@ static void cf_check _ehci_dbgp_poll(str
         spin_unlock_irqrestore(&port->tx_lock, flags);
     }
 
+    /* Mimic interrupt context. */
+    old_regs = set_irq_regs(regs);
+
     if ( dbgp->in.chunk )
         serial_rx_interrupt(port, regs);
 
     if ( empty )
         serial_tx_interrupt(port, regs);
 
+    set_irq_regs(old_regs);
+
     if ( spin_trylock_irqsave(&port->tx_lock, flags) )
     {
         if ( dbgp->state == dbgp_idle && !dbgp->in.chunk &&
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -211,10 +211,14 @@ static void cf_check __ns16550_poll(stru
 {
     struct serial_port *port = this_cpu(poll_port);
     struct ns16550 *uart = port->uart;
+    struct cpu_user_regs *old_regs;
 
     if ( uart->intr_works )
         return; /* Interrupts work - no more polling */
 
+    /* Mimic interrupt context. */
+    old_regs = set_irq_regs(regs);
+
     while ( ns_read_reg(uart, UART_LSR) & UART_LSR_DR )
     {
         if ( ns16550_ioport_invalid(uart) )
@@ -227,6 +231,7 @@ static void cf_check __ns16550_poll(stru
         serial_tx_interrupt(port, regs);
 
 out:
+    set_irq_regs(old_regs);
     set_timer(&uart->timer, NOW() + MILLISECS(uart->timeout_ms));
 }
 
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -1164,6 +1164,7 @@ static void cf_check dbc_uart_poll(void
     struct dbc_uart *uart = port->uart;
     struct dbc *dbc = &uart->dbc;
     unsigned long flags = 0;
+    struct cpu_user_regs *old_regs;
 
     if ( spin_trylock_irqsave(&port->tx_lock, flags) )
     {
@@ -1175,10 +1176,15 @@ static void cf_check dbc_uart_poll(void
         spin_unlock_irqrestore(&port->tx_lock, flags);
     }
 
+    /* Mimic interrupt context. */
+    old_regs = set_irq_regs(guest_cpu_user_regs());
+
     while ( dbc_work_ring_size(&dbc->dbc_iwork) )
         serial_rx_interrupt(port, guest_cpu_user_regs());
 
     serial_tx_interrupt(port, guest_cpu_user_regs());
+
+    set_irq_regs(old_regs);
     set_timer(&uart->timer, NOW() + MICROSECS(DBC_POLL_INTERVAL));
 }
 



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:28:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:28:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676037.1051803 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz1D-0002l4-Hb; Mon, 05 Feb 2024 13:28:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676037.1051803; Mon, 05 Feb 2024 13:28:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz1D-0002kx-Ed; Mon, 05 Feb 2024 13:28:15 +0000
Received: by outflank-mailman (input) for mailman id 676037;
 Mon, 05 Feb 2024 13:28:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWz1C-0001wW-Dd
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:28:14 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 696dd4f7-c42a-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:28:13 +0100 (CET)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-3394ca0c874so3065680f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:28:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020a5d67ca000000b0033b3cf1ff09sm2278382wrw.29.2024.02.05.05.28.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:28:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 696dd4f7-c42a-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707139693; x=1707744493; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=ydAWDx9h9IVXFzTidsD6Z1TLZe7e5dY/H1Vr9S9PwiE=;
        b=M85Rma8KDEK2bz8CMRFn/wGvt6wJ7J2+5qMwGf5DD1VXe/thRwDw6T2ExVmEWsV29y
         /CePWXA8xEBj77p67SOY9AvGefJh+qJgMwQSn+nVHRX0d8jkURFQmRrmcV7G2d2t5eDB
         LQ8Hym1Xk3EcgfIyMje12GvhpZdivZ7v9O4dIneSoZqk7sXN7pxAJiCbTF76vA9THLnd
         qyHnBVKwRrNjaxtk8XZkLjwf6HRJyAfd7iNz0zAFxHQrPP/oHoHF6H89vFVwOAh0MAPK
         5PJ8o3u6rLjX6fY1b5xuA3v3BiUp5gXtn4sYaHbugQKaPBIU+k8zx2G7SiuVlU7OrOhz
         Z7Tg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707139693; x=1707744493;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=ydAWDx9h9IVXFzTidsD6Z1TLZe7e5dY/H1Vr9S9PwiE=;
        b=Bwo09h8G6bpHZN6lIOTeMqFblWjgd2qXsLl57Ute1Cxo7oPmQIuUbo9+27/0VE8ERe
         IToFXXVuFETD4AoksIU650I+iJp6W0K1NDPGlFVc+ywtOrlDSW3+n9oEhL2W4kKq9JyB
         ZFQzdaRStD2FN8qHAaw+uOCqAF8lXXlKA8yc9mUKpTM0Bx4uhJvK5lW9HTYL8CD8bCJX
         h8g5h2BjZI+eV4PrNAXRZcgx54dA7qOGIgVIGo3XDsSX7lE2aVXvZiRGoYXBUEbWVPiU
         RCHfrWzsWoUl6Muy/3p7NO3eDdhgT6amfFRcuQUTc+j/k7Vc9VZ2PRub+4T0tT6IFEG6
         wUrw==
X-Gm-Message-State: AOJu0Yze5PhHjapHwMAEQQmHHXrzfXqmqJaw2+NgQ3P0kexxCTU9x+/p
	Aj+/1qKpq6o48Prqhms3uLKP/76qQi/Aav3m/ADaWRcb6syTgoWL0hwC5vO0CwzLo4VLLgejzmw
	=
X-Google-Smtp-Source: AGHT+IG9O74TbbFhs8wDKqrVpx4p5dH0NkSqdkZDU59Z6j8Sq5nvwWXjpfokpf2TTmtpmiWYKrMfrw==
X-Received: by 2002:a05:6000:156b:b0:33b:4164:5fbe with SMTP id 11-20020a056000156b00b0033b41645fbemr1659186wrz.20.1707139692820;
        Mon, 05 Feb 2024 05:28:12 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUZWkTBBEG2t7OzrySSCvjbFwDbr1QH0r1fZRnhkzOu/ApKX7CMgkT6xIEayL91ZcWyMJYO0X5Os1eviVGmpjpr5+IawEty+KY0EZ26glJRyS8BVr/KZEWyN89RNnMyrn1Uu2uYhYM/y/N9rwwjN5jMnAd+jKwlLvM/sWJq
Message-ID: <5258d8a9-a7ef-4342-9b5c-fc0078139bb2@suse.com>
Date: Mon, 5 Feb 2024 14:28:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 2/8] keyhandler: drop regs parameter from handle_keyregs()
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

In preparation for further removal of regs parameters, drop it here. In
the two places where it's actually needed, retrieve IRQ context if
available, or else guest context.

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
As an alternative to the new boolean parameter, I wonder if we couldn't
special-case the idle vCPU case: It's only there where we would not have
proper context retrievable via guest_cpu_user_regs().
---
v3: Re-base.
v2: New.

--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -73,12 +73,12 @@ static struct keyhandler {
 
 static void cf_check keypress_action(void *unused)
 {
-    handle_keypress(keypress_key, NULL);
+    handle_keypress(keypress_key, true);
 }
 
 static DECLARE_TASKLET(keypress_tasklet, keypress_action, NULL);
 
-void handle_keypress(unsigned char key, struct cpu_user_regs *regs)
+void handle_keypress(unsigned char key, bool need_context)
 {
     struct keyhandler *h;
 
@@ -88,7 +88,7 @@ void handle_keypress(unsigned char key,
     if ( !in_irq() || h->irq_callback )
     {
         console_start_log_everything();
-        h->irq_callback ? h->irq_fn(key, regs) : h->fn(key);
+        h->irq_callback ? h->irq_fn(key, need_context) : h->fn(key);
         console_end_log_everything();
     }
     else
@@ -169,7 +169,7 @@ void cf_check dump_execstate(struct cpu_
 }
 
 static void cf_check dump_registers(
-    unsigned char key, struct cpu_user_regs *regs)
+    unsigned char key, bool need_context)
 {
     unsigned int cpu;
 
@@ -182,8 +182,8 @@ static void cf_check dump_registers(
     cpumask_copy(&dump_execstate_mask, &cpu_online_map);
 
     /* Get local execution state out immediately, in case we get stuck. */
-    if ( regs )
-        dump_execstate(regs);
+    if ( !need_context )
+        dump_execstate(get_irq_regs() ?: guest_cpu_user_regs());
     else
         run_in_exception_handler(dump_execstate);
 
@@ -245,8 +245,7 @@ static void cf_check dump_hwdom_register
     }
 }
 
-static void cf_check reboot_machine(
-    unsigned char key, struct cpu_user_regs *regs)
+static void cf_check reboot_machine(unsigned char key, bool unused)
 {
     printk("'%c' pressed -> rebooting machine\n", key);
     machine_restart(0);
@@ -474,8 +473,7 @@ static void cf_check run_all_nonirq_keyh
 static DECLARE_TASKLET(run_all_keyhandlers_tasklet,
                        run_all_nonirq_keyhandlers, NULL);
 
-static void cf_check run_all_keyhandlers(
-    unsigned char key, struct cpu_user_regs *regs)
+static void cf_check run_all_keyhandlers(unsigned char key, bool need_context)
 {
     struct keyhandler *h;
     unsigned int k;
@@ -491,7 +489,7 @@ static void cf_check run_all_keyhandlers
         if ( !h->irq_fn || !h->diagnostic || !h->irq_callback )
             continue;
         printk("[%c: %s]\n", k, h->desc);
-        h->irq_fn(k, regs);
+        h->irq_fn(k, need_context);
     }
 
     watchdog_enable();
@@ -500,8 +498,7 @@ static void cf_check run_all_keyhandlers
     tasklet_schedule(&run_all_keyhandlers_tasklet);
 }
 
-static void cf_check do_toggle_alt_key(
-    unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_toggle_alt_key(unsigned char key, bool unused)
 {
     alt_key_handling = !alt_key_handling;
     printk("'%c' pressed -> using %s key handling\n", key,
@@ -566,7 +563,7 @@ void keyhandler_crash_action(enum crash_
         if ( *action == '+' )
             mdelay(10);
         else
-            handle_keypress(*action, NULL);
+            handle_keypress(*action, true);
         action++;
     }
 }
--- a/xen/common/sysctl.c
+++ b/xen/common/sysctl.c
@@ -134,7 +134,7 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xe
         {
             if ( copy_from_guest_offset(&c, op->u.debug_keys.keys, i, 1) )
                 goto out;
-            handle_keypress(c, guest_cpu_user_regs());
+            handle_keypress(c, false);
         }
         ret = 0;
         copyback = 0;
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -279,7 +279,7 @@ static int *__read_mostly upper_thresh_a
 static int *__read_mostly lower_thresh_adj = &xenlog_lower_thresh;
 static const char *__read_mostly thresh_adj = "standard";
 
-static void cf_check do_toggle_guest(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_toggle_guest(unsigned char key, bool unused)
 {
     if ( upper_thresh_adj == &xenlog_upper_thresh )
     {
@@ -306,13 +306,13 @@ static void do_adj_thresh(unsigned char
            loglvl_str(*upper_thresh_adj));
 }
 
-static void cf_check do_inc_thresh(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_inc_thresh(unsigned char key, bool unused)
 {
     ++*lower_thresh_adj;
     do_adj_thresh(key);
 }
 
-static void cf_check do_dec_thresh(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_dec_thresh(unsigned char key, bool unused)
 {
     if ( *lower_thresh_adj )
         --*lower_thresh_adj;
@@ -531,7 +531,7 @@ static void __serial_rx(char c, struct c
     switch ( console_rx )
     {
     case 0:
-        return handle_keypress(c, regs);
+        return handle_keypress(c, false);
 
     case 1:
         /*
--- a/xen/include/xen/keyhandler.h
+++ b/xen/include/xen/keyhandler.h
@@ -24,9 +24,8 @@ typedef void (keyhandler_fn_t)(unsigned
  *
  * Called in hardirq context with interrupts disabled.
  */
-struct cpu_user_regs;
 typedef void (irq_keyhandler_fn_t)(unsigned char key,
-                                   struct cpu_user_regs *regs);
+                                   bool need_context);
 
 /* Initialize keytable with default handlers. */
 void initialize_keytable(void);
@@ -46,7 +45,7 @@ void register_irq_keyhandler(unsigned ch
                              bool diagnostic);
 
 /* Inject a keypress into the key-handling subsystem. */
-extern void handle_keypress(unsigned char key, struct cpu_user_regs *regs);
+extern void handle_keypress(unsigned char key, bool need_context);
 
 enum crash_reason {
     CRASHREASON_PANIC,



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:28:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:28:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676040.1051813 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz1j-0003Mm-P5; Mon, 05 Feb 2024 13:28:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676040.1051813; Mon, 05 Feb 2024 13:28:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz1j-0003Mf-MS; Mon, 05 Feb 2024 13:28:47 +0000
Received: by outflank-mailman (input) for mailman id 676040;
 Mon, 05 Feb 2024 13:28:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWz1i-0003FY-EJ
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:28:46 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7c23d31a-c42a-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 14:28:44 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-40ef3f351d2so30817615e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:28:44 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020a5d67ca000000b0033b3cf1ff09sm2278382wrw.29.2024.02.05.05.28.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:28:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c23d31a-c42a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707139724; x=1707744524; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=psVWVy2hduK7Z6lfch3dygRM4LRvywrY0uUoZLjxYv8=;
        b=XxsJcSpVeLPdoygIXnZKEQxv3TpI3O4yXTZcxsG7/1DkqBBZmyHUbfnJF0goCAcxyM
         1/opdVEh/awrbMjplAAIwgLEfyiuOOcdxqhYn+u7dQcfXpp4WkDWYo7mu/YI0RwdyaVj
         ANgZCdqhXv4pfRUvjHVpIc0jYugKnXhhOXiqNzv0nI8yBg8Ks3M+7eq4riBG8NpkhtQs
         BruiQ8KKIthtlpVR1olclttMdPpBpyPiAl45l9H4Dj2bCRlFe+Rxvlv9wK5by+WKfp/h
         BLgRgKfZkOLGstdxdMmO4Ox5TJrL2NSSwTOXxp8CprQC8gb3g2l8H2EpCITfstB9fdn5
         BRjA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707139724; x=1707744524;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=psVWVy2hduK7Z6lfch3dygRM4LRvywrY0uUoZLjxYv8=;
        b=HZ5ujlHHoob2g6BZlc50WsNM9ta/YpOw95JPYOPCqMAqYjyi7kbJPFLjXPa/4titoi
         O+Pa86BwUsV70BQJSY6xe4YBu6BFl1sBTcxCyF5IQYDFgC268z8blWk5pVlm856gM+QX
         ue7GmPTMqXcy9XzOlAciYDAmvJMo7zLtERAsSbtneBm77Ha7haCrQAoIs7S27sFAtYBM
         ntYvAjjK7MjLQwr+KDnaIbLdWW/JG1vNOPmMe38OATAW73/MzClQZuwc1Vm5BwvHbmlJ
         8VBUYCmZIy5eCjQmPfEq6EhFkhm1i7uz3MuySNuc8KuR0Tgav1RHAowPfKalFych/ctd
         q8rg==
X-Gm-Message-State: AOJu0YzFpNZgjfQKHogK1A6StPMerRSLYG8z2FFhriDJGONbCMWiy9jk
	QR01nWFtDh1UHX//tC+DQpzu0RaSRS1my6TMB9jeNrADz+o4x96gwGJ1hvqwmQ97iaSKYUMH9bw
	=
X-Google-Smtp-Source: AGHT+IGadlMeSymDT4u27ECo2thETBDBX07yTDsj5dYtgL1QRTtoKZr2/2EiQhy21ukKeSs0azWEgQ==
X-Received: by 2002:adf:f54b:0:b0:33b:1ab8:3736 with SMTP id j11-20020adff54b000000b0033b1ab83736mr3609865wrp.1.1707139724189;
        Mon, 05 Feb 2024 05:28:44 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWrOAgZ6PvaxGE80/qVy3edix+XQ0UT7axQGqqF0eVHZGh63882Z1eDXnGQhaaag+ud2jBLdzd5hcfwRzDCimBcCGIICuJyylauEK4CzgTWpkSHsnnw7kBv3emdxgMgt3Y/8kY2PbZd1ll47wCEwmyGPgafERC3dxUyBvq9
Message-ID: <df7f2df7-d45c-4c96-a84a-5ca816a5b733@suse.com>
Date: Mon, 5 Feb 2024 14:28:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 3/8] serial: drop serial_rx_fn's regs parameter
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

It's simply not needed anymore.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: Re-base over earlier (new/split) patches.

--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -526,7 +526,7 @@ static void switch_serial_input(void)
     printk("\n");
 }
 
-static void __serial_rx(char c, struct cpu_user_regs *regs)
+static void __serial_rx(char c)
 {
     switch ( console_rx )
     {
@@ -578,7 +578,7 @@ static void __serial_rx(char c, struct c
 #endif
 }
 
-static void cf_check serial_rx(char c, struct cpu_user_regs *regs)
+static void cf_check serial_rx(char c)
 {
     static int switch_code_count = 0;
 
@@ -594,10 +594,10 @@ static void cf_check serial_rx(char c, s
     }
 
     for ( ; switch_code_count != 0; switch_code_count-- )
-        __serial_rx(switch_code, regs);
+        __serial_rx(switch_code);
 
     /* Finally process the just-received character. */
-    __serial_rx(c, regs);
+    __serial_rx(c);
 }
 
 static void cf_check notify_dom0_con_ring(void *unused)
--- a/xen/drivers/char/serial.c
+++ b/xen/drivers/char/serial.c
@@ -68,7 +68,7 @@ void serial_rx_interrupt(struct serial_p
     spin_unlock_irqrestore(&port->rx_lock, flags);
 
     if ( fn != NULL )
-        (*fn)(c & 0x7f, regs);
+        fn(c & 0x7f);
 }
 
 void serial_tx_interrupt(struct serial_port *port, struct cpu_user_regs *regs)
--- a/xen/drivers/char/xen_pv_console.c
+++ b/xen/drivers/char/xen_pv_console.c
@@ -118,7 +118,7 @@ size_t pv_console_rx(struct cpu_user_reg
     {
         c = cons_ring->in[MASK_XENCONS_IDX(cons++, cons_ring->in)];
         if ( cons_rx_handler )
-            cons_rx_handler(c, regs);
+            cons_rx_handler(c);
         recv++;
     }
 
--- a/xen/include/xen/serial.h
+++ b/xen/include/xen/serial.h
@@ -15,7 +15,7 @@
 struct cpu_user_regs;
 
 /* Register a character-receive hook on the specified COM port. */
-typedef void (*serial_rx_fn)(char c, struct cpu_user_regs *regs);
+typedef void (*serial_rx_fn)(char c);
 void serial_set_rx_handler(int handle, serial_rx_fn fn);
 
 /* Number of characters we buffer for a polling receiver. */



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:29:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:29:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676043.1051823 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz2R-0003tu-2y; Mon, 05 Feb 2024 13:29:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676043.1051823; Mon, 05 Feb 2024 13:29:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz2Q-0003tn-VK; Mon, 05 Feb 2024 13:29:30 +0000
Received: by outflank-mailman (input) for mailman id 676043;
 Mon, 05 Feb 2024 13:29:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWz2Q-0003FY-2H
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:29:30 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 96289b97-c42a-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 14:29:28 +0100 (CET)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-3392b12dd21so2444406f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:29:28 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020a5d67ca000000b0033b3cf1ff09sm2278382wrw.29.2024.02.05.05.29.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:29:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 96289b97-c42a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707139768; x=1707744568; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=YO3g36PvHebDUpK9/W9a6xU16xqzRXgrwigdPTnIuIA=;
        b=g2MbNanWmMBW1mXFwpf9WhmxUvOHR+Wxk305SNGPET7KdNPIXavA6/U0yaK+ixxMyI
         BO2lVLsZHc6Jr8/hOt2FNVxvzqZg8mO4rtRTm12lfARNqdoC0WGUQiq4t0d8xOaPydnY
         7zJbyM9ka2IgghQUeMuEveXsQQa+rkD2vIAITf3ONh0Bm/UAg31ueWdSKEg4gbNPMDnv
         OjDb4DPnjAdfNJI81zTN0thvU+A7YMiYQGk5wcqdOezfYl3a+R6qSL/ttmngxwPCQjSA
         E+4mRFYUpAYbRRLlT1jfpBppL30paHojevvLfnRF9sH5y2DQ8rOZtFkpwc4dtUwauGuw
         N9iw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707139768; x=1707744568;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=YO3g36PvHebDUpK9/W9a6xU16xqzRXgrwigdPTnIuIA=;
        b=ZLWEOlnod+3fdGQWAHtbAwQnmJvZR7ugaYl1I7QuqsIlOO1YHe5qHwfvBPjfFB0MmM
         zKPzrAipg6vT/PQHuWAJOxQk7mZGCR+JziaGJ42itoAY6nasmdkBsOT8qrUTcC8MKnsd
         czn0BKNmpC4QyR185tria61M0Qv57rHt8btejYLFu7Mx1LZH7I6YRgK6zFtPL8poDYat
         cSgL0XTV83F4DhPgiyg5F7fx6uMIX4luCgXNAzyqklbrBqYZ01WLlo7s8ZlpDNk/PMGB
         +nTc4kysZS0rQy9X7rzx+fs4j1Qc4ReN5bZQQwe93hn7nzedJr6q38wUtBDhuX7gYinZ
         rekw==
X-Gm-Message-State: AOJu0Yx8OdFXzF7EcOA8JPtXt5JZtQT7ivS82nV4M7sEe0EFhdSTsTT5
	zYiyb5ySJ56eBJxG6okmAhVz8M+XKFZHx7Z7voIOCYw8oy3FjOftESnmZHqQPXRt17LJ5PuYfic
	=
X-Google-Smtp-Source: AGHT+IEh7yjmBHiIGhoxsC8ZOitxgl5YrK3VW6GD8r1s9VL9siV/a5Vfv15NM47eHWMWt7W38eHMcw==
X-Received: by 2002:adf:b184:0:b0:33a:e8be:51bd with SMTP id q4-20020adfb184000000b0033ae8be51bdmr5584809wra.51.1707139767744;
        Mon, 05 Feb 2024 05:29:27 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVm84qLJXYYsjwVfmXrUWyqek0PAuFuHurAmJQnD6G03zBzM1DV9CdKp+QdglcvAAEdXvl4KpbSm/pQgJqRT850IUT/n0tWwcVRmYkA+NjtHs9yoiVVh10RUl84tHg32oQTMEHBHyq82cimKpPvIjaG/vR3PYlvKGR5AGbzGUNlknWZyi74xvgGB+LoECbto2lL+A==
Message-ID: <5f76c01f-4cf0-4508-af94-81516b745d37@suse.com>
Date: Mon, 5 Feb 2024 14:29:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 4/8] PV-shim: drop pv_console_rx()'s regs parameter
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

It's not needed anymore. This is in preparation of dropping the register
parameters from IRQ handler functions.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

--- a/xen/arch/x86/guest/xen/xen.c
+++ b/xen/arch/x86/guest/xen/xen.c
@@ -181,7 +181,7 @@ static void cf_check xen_evtchn_upcall(s
             port += l1 * BITS_PER_LONG;
 
             if ( pv_console && port == pv_console_evtchn() )
-                pv_console_rx(regs);
+                pv_console_rx();
             else if ( pv_shim )
                 pv_shim_inject_evtchn(port);
         }
--- a/xen/drivers/char/xen_pv_console.c
+++ b/xen/drivers/char/xen_pv_console.c
@@ -94,7 +94,7 @@ evtchn_port_t pv_console_evtchn(void)
     return cons_evtchn;
 }
 
-size_t pv_console_rx(struct cpu_user_regs *regs)
+size_t pv_console_rx(void)
 {
     char c;
     XENCONS_RING_IDX cons, prod;
--- a/xen/include/xen/pv_console.h
+++ b/xen/include/xen/pv_console.h
@@ -9,7 +9,7 @@ void pv_console_init(void);
 void pv_console_set_rx_handler(serial_rx_fn fn);
 void pv_console_init_postirq(void);
 void pv_console_puts(const char *buf, size_t nr);
-size_t pv_console_rx(struct cpu_user_regs *regs);
+size_t pv_console_rx(void);
 evtchn_port_t pv_console_evtchn(void);
 
 #else
@@ -18,7 +18,7 @@ static inline void pv_console_init(void)
 static inline void pv_console_set_rx_handler(serial_rx_fn fn) { }
 static inline void pv_console_init_postirq(void) { }
 static inline void pv_console_puts(const char *buf, size_t nr) { }
-static inline size_t pv_console_rx(struct cpu_user_regs *regs) { return 0; }
+static inline size_t pv_console_rx(void) { return 0; }
 
 #endif /* !CONFIG_XEN_GUEST */
 #endif /* __XEN_PV_CONSOLE_H__ */



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:30:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:30:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676047.1051833 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz3C-0005Mc-Eo; Mon, 05 Feb 2024 13:30:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676047.1051833; Mon, 05 Feb 2024 13:30:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz3C-0005MV-Ah; Mon, 05 Feb 2024 13:30:18 +0000
Received: by outflank-mailman (input) for mailman id 676047;
 Mon, 05 Feb 2024 13:30:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWz3B-0004HJ-E0
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:30:17 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b2dab967-c42a-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:30:16 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-40fd280421aso15725275e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:30:16 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020a5d67ca000000b0033b3cf1ff09sm2278382wrw.29.2024.02.05.05.30.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:30:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2dab967-c42a-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707139816; x=1707744616; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=0DlbwChHh3lVOiwYmNPTJJLzXFhauBoipOMIM8LYm04=;
        b=fFKq0LqqmrSiqo+Cli0FN1OV82rY8UgBbtiETk0CUTgAR/A/2y6iK5ZcekWPfswbLg
         TWUTiH0KaUCD+r1tDOzYrQmco+Qu8VB1GDlpxVUnZeiw3xvEHLHrdP10py7ppP5vDpQ3
         uK6aZ0gqb9sxgq1ondzBcl6HgaJtVcJvxbS3QV92GH07uuVvOVVAJBIWPiJCgMyVRXRP
         4vHgND8Yk62tig7OJY66FUSe/b3ZUcaftSKnlICYyS2MaYGgZpz7P4cp6VLL1WbI9tsh
         d9Daqyg2jp++JjXbfpmdHtuGj5nwnQzutg/ndlqK4p5FhK7WobdPWRX3R4lJzZ81UG92
         Xdjw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707139816; x=1707744616;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=0DlbwChHh3lVOiwYmNPTJJLzXFhauBoipOMIM8LYm04=;
        b=CUPfr58Q1dDsEWfJVXWK6sKSsHvuQDcQ5jg13Sc9ssFoIK3t4CtqoUPEIqAtluvofu
         euNx/4Q3eJ4dECEPHYJgh7LVF4NX+Y1wjyg1I53nYh3NHJCkJ2n5lprWIUgGisMtYYHv
         JTxu2/Pb4wnz9SFN9TQMlcBTcfzSRthg7I0P5ZlUwHpgBRP7V26oADTP5V8xEcAiHDpr
         CwsxAKQQwI4s24qxUb2orQ0fzoaLmkP8kn8BLIz9+hrhTsV46WmHcF6T5Pm8L6zHzX1J
         p7Syy/Oulvz1n0ViXOu2tfw+WnenZeOAK9vkMP4z1jOrG1opKeN75u/ByuNditDAAW1b
         ZNoA==
X-Gm-Message-State: AOJu0YwSz86IRUvLhE495Y8GAKGKU/vpuTPkVQzRKshOH1kIqA15ljHX
	tz6M2wZMWjRdFnNXG+W96bxn1o2emEAWCpK13wf2BsOPpPV2cOHuMBZbUbsARs+vcLxahpEV4uM
	=
X-Google-Smtp-Source: AGHT+IEWfBJhc0XvEQRQk7IjE0kBMObIrmCcQzT6dWwQH+7AdZByvK3ccdT0AUCQHrl2vijDWLcVcw==
X-Received: by 2002:adf:a39b:0:b0:33a:fc6e:c728 with SMTP id l27-20020adfa39b000000b0033afc6ec728mr6175580wrb.32.1707139815886;
        Mon, 05 Feb 2024 05:30:15 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWFJN9zGAhJw44sNlKGBPB2AVw9uieWHp0F7BpfVyryjKkSP6jkROxDikQOcnmwgCoRAYGpUHUMeYKj/wSXFgipt/nLruwrbXeUR63nhh1KYMp/LB2aNBwERpjto3WYIHaZ7F9PAk9ND0OGeTUOvUQsNPJi+NesYbMNX/Kl/bLgG+R9ln1AMx9XIiG2JDWAGn+lKnaWz0wYtDbx6GrpC8Hg0MzCufcerGTzILpk/1RS7Bk8cHT2B5r7vR7GkLPZAU6KqO3d+nzhUA==
Message-ID: <93aec041-70f6-4385-9a52-8808e0cea931@suse.com>
Date: Mon, 5 Feb 2024 14:30:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 5/8] serial: drop serial_[rt]x_interrupt()'s regs parameter
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

They're simply not needed anymore.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: Setting of IRQ regs split off to an earlier patch.

--- a/xen/drivers/char/cadence-uart.c
+++ b/xen/drivers/char/cadence-uart.c
@@ -51,7 +51,7 @@ static void cuart_interrupt(int irq, voi
         /* ACK.  */
         if ( status & UART_SR_INTR_RTRIG )
         {
-            serial_rx_interrupt(port, regs);
+            serial_rx_interrupt(port);
             cuart_write(uart, R_UART_CISR, UART_SR_INTR_RTRIG);
         }
     } while ( status & UART_SR_INTR_RTRIG );
--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -1273,10 +1273,10 @@ static void cf_check _ehci_dbgp_poll(str
     old_regs = set_irq_regs(regs);
 
     if ( dbgp->in.chunk )
-        serial_rx_interrupt(port, regs);
+        serial_rx_interrupt(port);
 
     if ( empty )
-        serial_tx_interrupt(port, regs);
+        serial_tx_interrupt(port);
 
     set_irq_regs(old_regs);
 
--- a/xen/drivers/char/exynos4210-uart.c
+++ b/xen/drivers/char/exynos4210-uart.c
@@ -81,7 +81,7 @@ static void exynos4210_uart_interrupt(in
         if ( status & (UINTM_RXD | UINTM_ERROR) )
         {
             /* uart->regs[UINTM] |= RXD|ERROR; */
-            serial_rx_interrupt(port, regs);
+            serial_rx_interrupt(port);
             /* uart->regs[UINTM] &= ~(RXD|ERROR); */
             exynos4210_write(uart, UINTP, UINTM_RXD | UINTM_ERROR);
         }
@@ -89,7 +89,7 @@ static void exynos4210_uart_interrupt(in
         if ( status & (UINTM_TXD | UINTM_MODEM) )
         {
             /* uart->regs[UINTM] |= TXD|MODEM; */
-            serial_tx_interrupt(port, regs);
+            serial_tx_interrupt(port);
             /* uart->regs[UINTM] &= ~(TXD|MODEM); */
             exynos4210_write(uart, UINTP, UINTM_TXD | UINTM_MODEM);
         }
--- a/xen/drivers/char/imx-lpuart.c
+++ b/xen/drivers/char/imx-lpuart.c
@@ -48,10 +48,10 @@ static void imx_lpuart_interrupt(int irq
     rxcnt = imx_lpuart_read(uart, UARTWATER) >> UARTWATER_RXCNT_OFF;
 
     if ( (sts & UARTSTAT_RDRF) || (rxcnt > 0) )
-	    serial_rx_interrupt(port, regs);
+	    serial_rx_interrupt(port);
 
     if ( sts & UARTSTAT_TDRE )
-	    serial_tx_interrupt(port, regs);
+	    serial_tx_interrupt(port);
 
     imx_lpuart_write(uart, UARTSTAT, sts);
 }
--- a/xen/drivers/char/meson-uart.c
+++ b/xen/drivers/char/meson-uart.c
@@ -69,10 +69,10 @@ static void meson_uart_interrupt(int irq
     uint32_t st = readl(uart->regs + AML_UART_STATUS_REG);
 
     if ( !(st & AML_UART_RX_FIFO_EMPTY) )
-        serial_rx_interrupt(port, regs);
+        serial_rx_interrupt(port);
 
     if ( !(st & AML_UART_TX_FIFO_FULL) )
-        serial_tx_interrupt(port, regs);
+        serial_tx_interrupt(port);
 }
 
 static void __init meson_uart_init_preirq(struct serial_port *port)
--- a/xen/drivers/char/mvebu-uart.c
+++ b/xen/drivers/char/mvebu-uart.c
@@ -76,10 +76,10 @@ static void mvebu3700_uart_interrupt(int
 
     if ( st & (STATUS_RX_RDY | STATUS_OVR_ERR | STATUS_FRM_ERR |
                STATUS_BRK_DET) )
-        serial_rx_interrupt(port, regs);
+        serial_rx_interrupt(port);
 
     if ( st & STATUS_TX_RDY )
-        serial_tx_interrupt(port, regs);
+        serial_tx_interrupt(port);
 }
 
 static void __init mvebu3700_uart_init_preirq(struct serial_port *port)
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -188,9 +188,9 @@ static void cf_check ns16550_interrupt(
         u8 lsr = ns_read_reg(uart, UART_LSR);
 
         if ( (lsr & uart->lsr_mask) == uart->lsr_mask )
-            serial_tx_interrupt(port, regs);
+            serial_tx_interrupt(port);
         if ( lsr & UART_LSR_DR )
-            serial_rx_interrupt(port, regs);
+            serial_rx_interrupt(port);
 
         /* A "busy-detect" condition is observed on Allwinner/sunxi UART
          * after LCR is written during setup. It needs to be cleared at
@@ -224,11 +224,11 @@ static void cf_check __ns16550_poll(stru
         if ( ns16550_ioport_invalid(uart) )
             goto out;
 
-        serial_rx_interrupt(port, regs);
+        serial_rx_interrupt(port);
     }
 
     if ( ( ns_read_reg(uart, UART_LSR) & uart->lsr_mask ) == uart->lsr_mask )
-        serial_tx_interrupt(port, regs);
+        serial_tx_interrupt(port);
 
 out:
     set_irq_regs(old_regs);
--- a/xen/drivers/char/omap-uart.c
+++ b/xen/drivers/char/omap-uart.c
@@ -70,9 +70,9 @@ static void omap_uart_interrupt(int irq,
     {
         lsr = omap_read(uart, UART_LSR) & 0xff;
 	if ( lsr & UART_LSR_THRE )
-            serial_tx_interrupt(port, regs);
+            serial_tx_interrupt(port);
 	if ( lsr & UART_LSR_DR )
-            serial_rx_interrupt(port, regs);
+            serial_rx_interrupt(port);
 
         if ( port->txbufc == port->txbufp ) {
             reg = omap_read(uart, UART_IER);
--- a/xen/drivers/char/pl011.c
+++ b/xen/drivers/char/pl011.c
@@ -95,7 +95,7 @@ static void pl011_interrupt(int irq, voi
             pl011_write(uart, ICR, status & ~(TXI|RTI|RXI));
 
             if ( status & (RTI|RXI) )
-                serial_rx_interrupt(port, regs);
+                serial_rx_interrupt(port);
 
             /* TODO
                 if ( status & (DSRMI|DCDMI|CTSMI|RIMI) )
@@ -103,7 +103,7 @@ static void pl011_interrupt(int irq, voi
             */
 
             if ( status & (TXI) )
-                serial_tx_interrupt(port, regs);
+                serial_tx_interrupt(port);
 
             status = pl011_intr_status(uart);
         } while (status != 0);
--- a/xen/drivers/char/scif-uart.c
+++ b/xen/drivers/char/scif-uart.c
@@ -119,11 +119,11 @@ static void scif_uart_interrupt(int irq,
     {
         /* TX Interrupt */
         if ( status & SCFSR_TDFE )
-            serial_tx_interrupt(port, regs);
+            serial_tx_interrupt(port);
 
         /* RX Interrupt */
         if ( status & (SCFSR_RDF | SCFSR_DR) )
-            serial_rx_interrupt(port, regs);
+            serial_rx_interrupt(port);
 
         /* Error Interrupt */
         if ( status & params->error_mask )
--- a/xen/drivers/char/serial.c
+++ b/xen/drivers/char/serial.c
@@ -45,7 +45,7 @@ static inline void serial_stop_tx(struct
         port->driver->stop_tx(port);
 }
 
-void serial_rx_interrupt(struct serial_port *port, struct cpu_user_regs *regs)
+void serial_rx_interrupt(struct serial_port *port)
 {
     char c;
     serial_rx_fn fn = NULL;
@@ -71,7 +71,7 @@ void serial_rx_interrupt(struct serial_p
         fn(c & 0x7f);
 }
 
-void serial_tx_interrupt(struct serial_port *port, struct cpu_user_regs *regs)
+void serial_tx_interrupt(struct serial_port *port)
 {
     int i, n;
     unsigned long flags;
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -1180,9 +1180,9 @@ static void cf_check dbc_uart_poll(void
     old_regs = set_irq_regs(guest_cpu_user_regs());
 
     while ( dbc_work_ring_size(&dbc->dbc_iwork) )
-        serial_rx_interrupt(port, guest_cpu_user_regs());
+        serial_rx_interrupt(port);
 
-    serial_tx_interrupt(port, guest_cpu_user_regs());
+    serial_tx_interrupt(port);
 
     set_irq_regs(old_regs);
     set_timer(&uart->timer, NOW() + MICROSECS(DBC_POLL_INTERVAL));
--- a/xen/include/xen/serial.h
+++ b/xen/include/xen/serial.h
@@ -12,8 +12,6 @@
 #include <xen/init.h>
 #include <xen/spinlock.h>
 
-struct cpu_user_regs;
-
 /* Register a character-receive hook on the specified COM port. */
 typedef void (*serial_rx_fn)(char c);
 void serial_set_rx_handler(int handle, serial_rx_fn fn);
@@ -155,8 +153,8 @@ void serial_register_uart(int idx, struc
 /* Place the serial port into asynchronous transmit mode. */
 void serial_async_transmit(struct serial_port *port);
 /* Process work in interrupt context. */
-void serial_rx_interrupt(struct serial_port *port, struct cpu_user_regs *regs);
-void serial_tx_interrupt(struct serial_port *port, struct cpu_user_regs *regs);
+void serial_rx_interrupt(struct serial_port *port);
+void serial_tx_interrupt(struct serial_port *port);
 
 /*
  * Initialisers for individual uart drivers.



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:31:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:31:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676050.1051843 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz3w-0005tU-NG; Mon, 05 Feb 2024 13:31:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676050.1051843; Mon, 05 Feb 2024 13:31:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz3w-0005tN-KX; Mon, 05 Feb 2024 13:31:04 +0000
Received: by outflank-mailman (input) for mailman id 676050;
 Mon, 05 Feb 2024 13:31:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWz3v-0004HJ-Ac
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:31:03 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ce35a050-c42a-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:31:02 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-40fdcae97e7so5909665e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:31:02 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020a5d67ca000000b0033b3cf1ff09sm2278382wrw.29.2024.02.05.05.31.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:31:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce35a050-c42a-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707139862; x=1707744662; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=vY7h+x/uY/tx4SThVUK3+EyH+28fbreju2/f4B+vk7s=;
        b=OjcsHD+hW5fNoTNferm4LkddzLGFXrfQPHgAwaXPDqOJBGv7QnLlo9fmQmzBA+gv4x
         beCVuywxYgitic1ATn38iknkcoP/SMbXkjJ/YLhZgjX6ImZdFmddfcNnlE88ES/BHP/S
         ZRL6yMSZE2BIHwCKlYDj6I+bv5kfRyR2cgvIBBCdtVYLiICZDuok7il7jD0cEBL8Z/c2
         EVnGmba9m7ET5Bl5XBmDLOSO0Vsw8LDonIXXbKcz/miWrWCIInj9N6e/9yYcpKUtvaXA
         9x4ug4vkaCq7RmpAAh+5MEIMxQ9YlgVa7ec82HGSaMJLGeqeG+Z/I8lFdn0VHanH2G78
         w08Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707139862; x=1707744662;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=vY7h+x/uY/tx4SThVUK3+EyH+28fbreju2/f4B+vk7s=;
        b=as8ZTcE9FnDUAcWFtlez62oknRnTc5XiydVBpvUnwQ5XOqEMhyw1RZKWHPrN6eF14X
         jklmcHNp2b5tsX9PUCyZu+HIgxFMSeqk89hTUXqF+qW4mfFghWh5WFkppvhcI6DmlevE
         8FJNqI13uCPsBsTxPeVemTn3guQRRa377B7xo19Nps86EgMilObdFMtKSzNwCDZraXIB
         TEY2E718tWRu+CLfZdR2oZAyxEhD7bNy0aDbRnY08uzLNz1bFs9UrD3IQNcMiT9CQ47z
         2SN7zQWUm2Sok67uoGl1GbznqZQffRLStLmAy+kspc9nn8+791Ws/RFX08r/4YMVB2+R
         DrMQ==
X-Gm-Message-State: AOJu0Yy+hn4aqWjVxPHx04/ytq4kXpHehRUpki7UGdqy6wS0rOxAwpJx
	jRXFShbbqt3KrW33lQuOILDMJLEPwJiDsa8viB6tclKJ82yku8UjDzagcyZfh/EQ+hOaaflR/Og
	=
X-Google-Smtp-Source: AGHT+IH6MbR/g8j+hw5ZnhR8vKIqLjsBF5B3Dwfy9d0iWHbVDtujDNlWP0J9CAu9n0+/I02GtX5C1Q==
X-Received: by 2002:a5d:55c4:0:b0:337:8f98:8ab4 with SMTP id i4-20020a5d55c4000000b003378f988ab4mr5650559wrw.37.1707139861828;
        Mon, 05 Feb 2024 05:31:01 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWcXc548klzDRD7WHkkYPkGFws+xgjAypm5h/s1MxCi+j3IwmwiM53r3gm+lJiP0Z0W/wdUYq7HVuL8Fg8uvRbXvZHjzc2f042CvpuYRloOrIbS8qyI69DM5pM2rNP8JA88aO9srDB3pj0wa26ci7w12uAL0R+Hs68ymUrhSPu/PNo9SADSUFMqop1VtuHPEz0pCrAET51452KTp9ak4RHgE3zklPBZ64JKwKwLmDQ/WbaDdbpaHOCBBbpyVhy224cDcEEHpXK6Yw==
Message-ID: <d7eb50e2-4bd6-4c2b-a98e-a750ede211e4@suse.com>
Date: Mon, 5 Feb 2024 14:31:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 6/8] IRQ: drop regs parameter from handler functions
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

It's simply not needed anymore. Note how Linux made this change many
years ago already, in 2.6.19 (late 2006, see [1]).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Julien Grall <jgrall@amazon.com>

[1] https://git.kernel.org/torvalds/c/7d12e780e003f93433d49ce78cfedf4b4c52adc5
---
v2: Arm build fixes.

--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -397,7 +397,7 @@ void gic_interrupt(struct cpu_user_regs
     } while (1);
 }
 
-static void maintenance_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
+static void maintenance_interrupt(int irq, void *dev_id)
 {
     /*
      * This is a dummy interrupt handler.
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -182,8 +182,7 @@ void irq_set_affinity(struct irq_desc *d
 }
 
 int request_irq(unsigned int irq, unsigned int irqflags,
-                void (*handler)(int irq, void *dev_id,
-                                struct cpu_user_regs *regs),
+                void (*handler)(int irq, void *dev_id),
                 const char *devname, void *dev_id)
 {
     struct irqaction *action;
@@ -276,7 +275,7 @@ void do_IRQ(struct cpu_user_regs *regs,
 
     do
     {
-        action->handler(irq, action->dev_id, regs);
+        action->handler(irq, action->dev_id);
         action = action->next;
     } while ( action );
 
--- a/xen/arch/arm/time.c
+++ b/xen/arch/arm/time.c
@@ -241,7 +241,7 @@ int reprogram_timer(s_time_t timeout)
 }
 
 /* Handle the firing timer */
-static void htimer_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
+static void htimer_interrupt(int irq, void *dev_id)
 {
     if ( unlikely(!(READ_SYSREG(CNTHP_CTL_EL2) & CNTx_CTL_PENDING)) )
         return;
@@ -255,7 +255,7 @@ static void htimer_interrupt(int irq, vo
     WRITE_SYSREG(0, CNTHP_CTL_EL2);
 }
 
-static void vtimer_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
+static void vtimer_interrupt(int irq, void *dev_id)
 {
     /*
      * Edge-triggered interrupts can be used for the virtual timer. Even
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -962,7 +962,7 @@ static int __init cf_check irq_ratelimit
 __initcall(irq_ratelimit_init);
 
 int __init request_irq(unsigned int irq, unsigned int irqflags,
-        void (*handler)(int irq, void *dev_id, struct cpu_user_regs *regs),
+        void (*handler)(int irq, void *dev_id),
         const char * devname, void *dev_id)
 {
     struct irqaction * action;
@@ -2009,7 +2009,7 @@ void do_IRQ(struct cpu_user_regs *regs)
         spin_unlock_irq(&desc->lock);
 
         tsc_in = tb_init_done ? get_cycles() : 0;
-        action->handler(irq, action->dev_id, regs);
+        action->handler(irq, action->dev_id);
         TRACE_3D(TRC_HW_IRQ_HANDLED, irq, tsc_in, get_cycles());
 
         spin_lock_irq(&desc->lock);
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -237,8 +237,7 @@ again:
     }
 }
 
-static void cf_check hpet_interrupt_handler(
-    int irq, void *data, struct cpu_user_regs *regs)
+static void cf_check hpet_interrupt_handler(int irq, void *data)
 {
     struct hpet_event_channel *ch = data;
 
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -198,8 +198,7 @@ static void smp_send_timer_broadcast_ipi
     }
 }
 
-static void cf_check timer_interrupt(
-    int irq, void *dev_id, struct cpu_user_regs *regs)
+static void cf_check timer_interrupt(int irq, void *dev_id)
 {
     ASSERT(local_irq_is_enabled());
 
--- a/xen/common/irq.c
+++ b/xen/common/irq.c
@@ -29,7 +29,7 @@ int init_one_irq_desc(struct irq_desc *d
     return err;
 }
 
-void cf_check no_action(int cpl, void *dev_id, struct cpu_user_regs *regs)
+void cf_check no_action(int cpl, void *dev_id)
 {
 }
 
--- a/xen/drivers/char/cadence-uart.c
+++ b/xen/drivers/char/cadence-uart.c
@@ -40,7 +40,7 @@ static struct cuart {
 #define cuart_read(uart, off)           readl((uart)->regs + (off))
 #define cuart_write(uart, off,val)      writel((val), (uart)->regs + (off))
 
-static void cuart_interrupt(int irq, void *data, struct cpu_user_regs *regs)
+static void cuart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct cuart *uart = port->uart;
--- a/xen/drivers/char/exynos4210-uart.c
+++ b/xen/drivers/char/exynos4210-uart.c
@@ -45,7 +45,7 @@ static struct exynos4210_uart {
 #define exynos4210_read(uart, off)          readl((uart)->regs + off)
 #define exynos4210_write(uart, off, val)    writel(val, (uart->regs) + off)
 
-static void exynos4210_uart_interrupt(int irq, void *data, struct cpu_user_regs *regs)
+static void exynos4210_uart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct exynos4210_uart *uart = port->uart;
--- a/xen/drivers/char/imx-lpuart.c
+++ b/xen/drivers/char/imx-lpuart.c
@@ -37,8 +37,7 @@ static struct imx_lpuart {
     struct vuart_info vuart;
 } imx8_com;
 
-static void imx_lpuart_interrupt(int irq, void *data,
-                                 struct cpu_user_regs *regs)
+static void imx_lpuart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct imx_lpuart *uart = port->uart;
--- a/xen/drivers/char/meson-uart.c
+++ b/xen/drivers/char/meson-uart.c
@@ -61,8 +61,7 @@ static struct meson_uart {
     struct vuart_info vuart;
 } meson_com;
 
-static void meson_uart_interrupt(int irq, void *data,
-                                 struct cpu_user_regs *regs)
+static void meson_uart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct meson_uart *uart = port->uart;
--- a/xen/drivers/char/mvebu-uart.c
+++ b/xen/drivers/char/mvebu-uart.c
@@ -67,8 +67,7 @@ static struct mvebu3700_uart {
 #define mvebu3700_read(uart, off)           readl((uart)->regs + (off))
 #define mvebu3700_write(uart, off, val)     writel(val, (uart)->regs + (off))
 
-static void mvebu3700_uart_interrupt(int irq, void *data,
-                                     struct cpu_user_regs *regs)
+static void mvebu3700_uart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct mvebu3700_uart *uart = port->uart;
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -175,8 +175,7 @@ static void handle_dw_usr_busy_quirk(str
     }
 }
 
-static void cf_check ns16550_interrupt(
-    int irq, void *dev_id, struct cpu_user_regs *regs)
+static void cf_check ns16550_interrupt(int irq, void *dev_id)
 {
     struct serial_port *port = dev_id;
     struct ns16550 *uart = port->uart;
--- a/xen/drivers/char/omap-uart.c
+++ b/xen/drivers/char/omap-uart.c
@@ -59,7 +59,7 @@ static struct omap_uart {
     struct vuart_info vuart;
 } omap_com = {0};
 
-static void omap_uart_interrupt(int irq, void *data, struct cpu_user_regs *regs)
+static void omap_uart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct omap_uart *uart = port->uart;
--- a/xen/drivers/char/pl011.c
+++ b/xen/drivers/char/pl011.c
@@ -82,7 +82,7 @@ static unsigned int pl011_intr_status(st
     return (pl011_read(uart, RIS) & pl011_read(uart, IMSC));
 }
 
-static void pl011_interrupt(int irq, void *data, struct cpu_user_regs *regs)
+static void pl011_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct pl011 *uart = port->uart;
--- a/xen/drivers/char/scif-uart.c
+++ b/xen/drivers/char/scif-uart.c
@@ -102,7 +102,7 @@ static const struct port_params port_par
     },
 };
 
-static void scif_uart_interrupt(int irq, void *data, struct cpu_user_regs *regs)
+static void scif_uart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct scif_uart *uart = port->uart;
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -718,8 +718,7 @@ static void cf_check do_amd_iommu_irq(vo
 
 static DECLARE_SOFTIRQ_TASKLET(amd_iommu_irq_tasklet, do_amd_iommu_irq, NULL);
 
-static void cf_check iommu_interrupt_handler(
-    int irq, void *dev_id, struct cpu_user_regs *regs)
+static void cf_check iommu_interrupt_handler(int irq, void *dev_id)
 {
     unsigned long flags;
     struct amd_iommu *iommu = dev_id;
--- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c
+++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
@@ -642,7 +642,7 @@ static void ipmmu_domain_irq(struct ipmm
                         domain->d, status, iova);
 }
 
-static void ipmmu_irq(int irq, void *dev, struct cpu_user_regs *regs)
+static void ipmmu_irq(int irq, void *dev)
 {
     struct ipmmu_vmsa_device *mmu = dev;
     unsigned int i;
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -1097,16 +1097,14 @@ static irqreturn_t arm_smmu_global_fault
 }
 
 /* Xen: Interrupt handlers wrapper */
-static void arm_smmu_context_fault_xen(int irq, void *dev,
-				       struct cpu_user_regs *regs)
+static void arm_smmu_context_fault_xen(int irq, void *dev)
 {
 	arm_smmu_context_fault(irq, dev);
 }
 
 #define arm_smmu_context_fault arm_smmu_context_fault_xen
 
-static void arm_smmu_global_fault_xen(int irq, void *dev,
-				      struct cpu_user_regs *regs)
+static void arm_smmu_global_fault_xen(int irq, void *dev)
 {
 	arm_smmu_global_fault(irq, dev);
 }
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -880,8 +880,7 @@ static void arm_smmu_priq_tasklet(void *
 
 static int arm_smmu_device_disable(struct arm_smmu_device *smmu);
 
-static void arm_smmu_gerror_handler(int irq, void *dev,
-				struct cpu_user_regs *regs)
+static void arm_smmu_gerror_handler(int irq, void *dev)
 {
 	u32 gerror, gerrorn, active;
 	struct arm_smmu_device *smmu = dev;
@@ -926,12 +925,11 @@ static void arm_smmu_gerror_handler(int
 	writel(gerror, smmu->base + ARM_SMMU_GERRORN);
 }
 
-static void arm_smmu_combined_irq_handler(int irq, void *dev,
-				struct cpu_user_regs *regs)
+static void arm_smmu_combined_irq_handler(int irq, void *dev)
 {
 	struct arm_smmu_device *smmu = dev;
 
-	arm_smmu_gerror_handler(irq, dev, regs);
+	arm_smmu_gerror_handler(irq, dev);
 
 	tasklet_schedule(&(smmu->combined_irq_tasklet));
 }
@@ -945,16 +943,14 @@ static void arm_smmu_combined_irq_taskle
 		arm_smmu_priq_tasklet(dev);
 }
 
-static void arm_smmu_evtq_irq_tasklet(int irq, void *dev,
-				struct cpu_user_regs *regs)
+static void arm_smmu_evtq_irq_tasklet(int irq, void *dev)
 {
 	struct arm_smmu_device *smmu = dev;
 
 	tasklet_schedule(&(smmu->evtq_irq_tasklet));
 }
 
-static void arm_smmu_priq_irq_tasklet(int irq, void *dev,
-				struct cpu_user_regs *regs)
+static void arm_smmu_priq_irq_tasklet(int irq, void *dev)
 {
 	struct arm_smmu_device *smmu = dev;
 
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1117,8 +1117,7 @@ static void cf_check do_iommu_page_fault
         __do_iommu_page_fault(drhd->iommu);
 }
 
-static void cf_check iommu_page_fault(
-    int irq, void *dev_id, struct cpu_user_regs *regs)
+static void cf_check iommu_page_fault(int irq, void *dev_id)
 {
     /*
      * Just flag the tasklet as runnable. This is fine, according to VT-d
--- a/xen/include/xen/irq.h
+++ b/xen/include/xen/irq.h
@@ -18,7 +18,7 @@
     ASSERT(!in_irq() && (local_irq_is_enabled() || num_online_cpus() <= 1))
 
 struct irqaction {
-    void (*handler)(int irq, void *dev_id, struct cpu_user_regs *regs);
+    void (*handler)(int irq, void *dev_id);
     const char *name;
     void *dev_id;
     bool free_on_release;
@@ -119,12 +119,11 @@ extern int setup_irq(unsigned int irq, u
                      struct irqaction *new);
 extern void release_irq(unsigned int irq, const void *dev_id);
 extern int request_irq(unsigned int irq, unsigned int irqflags,
-               void (*handler)(int irq, void *dev_id,
-                     struct cpu_user_regs *regs),
+               void (*handler)(int irq, void *dev_id),
                const char *devname, void *dev_id);
 
 extern hw_irq_controller no_irq_type;
-void cf_check no_action(int cpl, void *dev_id, struct cpu_user_regs *regs);
+void cf_check no_action(int cpl, void *dev_id);
 unsigned int cf_check irq_startup_none(struct irq_desc *desc);
 void cf_check irq_actor_none(struct irq_desc *desc);
 #define irq_shutdown_none irq_actor_none



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:31:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:31:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676060.1051853 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz4g-0006Sc-51; Mon, 05 Feb 2024 13:31:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676060.1051853; Mon, 05 Feb 2024 13:31:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz4g-0006SV-1T; Mon, 05 Feb 2024 13:31:50 +0000
Received: by outflank-mailman (input) for mailman id 676060;
 Mon, 05 Feb 2024 13:31:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWz4e-0004HJ-O2
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:31:48 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e984baf9-c42a-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:31:48 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-5100cb238bcso7942285e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:31:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020a5d67ca000000b0033b3cf1ff09sm2278382wrw.29.2024.02.05.05.31.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:31:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e984baf9-c42a-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707139907; x=1707744707; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=gEqRJt64VcumCTtnR8WkrhpV/gUr3OF3qVsMsouyC7A=;
        b=VawSKcJgQXK21l1YXTA5hk0LH8xnlRqfvMRRoNjFPkc37gYrkRUACWLaweYpNhSh16
         DctfDcsLwpu4360lWSYwwSLgjxXxYoMMJbvMK6Aydx1GeavKjBjota3gGeRbRMurDFTa
         Kv7kUcK6m4lfiP0TDx3nRusxd9RmvII1sKf/135+EJAnhcN1i8fldSakpPpXPsXQ4aQm
         MTtl0AwPfh1XQzHXyJK50KDXLW12y6wuLIa4cUZqiG7D/GPq1jvOEYiu1Z9H3430RL0F
         9V1TMvD9AJf6sh3GAO+EGhkfsyNnyCOX/cwlnG7hqwkypPSgNFTvfFwfweV2QgwItlR/
         6/pw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707139907; x=1707744707;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=gEqRJt64VcumCTtnR8WkrhpV/gUr3OF3qVsMsouyC7A=;
        b=otNwWGLSvS+lmX9dB4UFbS6B+Xcyg67x82K2yrtTszyNj7T9KDmWnKi9eGPpntUoQA
         RrsG9HpWO7Px4Yj/E0JTjI1WYJeVWoThpHGLzDa67o/kd7mHzQeNQ85KHk8A+Lw+z0Nk
         YqzAAL0gdQ/Xio+caSvz83ZajfHOb391+/9Y5bijw2Glrzhyopj2Xye4A6VdWEEya36L
         m3T39kLz7yJmWoeiIHD6WJmXt1yuANcloUSHUZSu4DiPAQ87GJ6wxWLWbG1IWi2A/Gl5
         vrjLy/8TslSxXA1WzrStLHpltEZG5KmzXTEDo8+mqeSgorOYcLMKx99NjBHO51/2UIbe
         T2uA==
X-Gm-Message-State: AOJu0YyJl1Gjn2kKTza8fTyOrHxFU6lHBlXUp9nfXVpWfwiJoREMD4Jc
	Urm94eB1AGRvmGV8ySAAvpCwbsQnZ/AdhyyY1x81Rt6/cE9bRKR3G+i1wDAq1DKr2KXjmuvsIQM
	=
X-Google-Smtp-Source: AGHT+IFrFVP3TfIbgIMT87bMnfEeRRPkJ7aKkrdXuEKYf5lnr2tL3R7tzhfm5/tp2tedPXGNUgc8ag==
X-Received: by 2002:a05:6512:1289:b0:511:5385:595d with SMTP id u9-20020a056512128900b005115385595dmr1473621lfs.41.1707139907575;
        Mon, 05 Feb 2024 05:31:47 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWdxlNgwzzOZXU8TNg2tyMXtWd0Pidw4BilmsICRarkIGrec4jvn4gCavIxalaAY7DXQkQqhAIJtNvRpnlcahl7akcDyRxq1caEIGnk1CbKimOnm4IHCvN0jzO/oobMf26WvepPfI6hCPQC5GgK6GUcZXVV2YEttFh24XHg+CzZrcjbWrmMZCNhqBjk0kpn12ocIPpGg39JNw==
Message-ID: <a815fcd8-1790-40c9-9fed-d5986f20d2a1@suse.com>
Date: Mon, 5 Feb 2024 14:31:47 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 7/8] x86/APIC: drop regs parameter from direct vector
 handler functions
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

The only place it was needed is in the spurious handler, and there we
can use get_irq_regs() instead.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -1322,7 +1322,7 @@ int reprogram_timer(s_time_t timeout)
     return apic_tmict || !timeout;
 }
 
-static void cf_check apic_timer_interrupt(struct cpu_user_regs *regs)
+static void cf_check apic_timer_interrupt(void)
 {
     ack_APIC_irq();
     perfc_incr(apic_timer);
@@ -1341,7 +1341,7 @@ void smp_send_state_dump(unsigned int cp
 /*
  * Spurious interrupts should _never_ happen with our APIC/SMP architecture.
  */
-static void cf_check spurious_interrupt(struct cpu_user_regs *regs)
+static void cf_check spurious_interrupt(void)
 {
     /*
      * Check if this is a vectored interrupt (most likely, as this is probably
@@ -1355,7 +1355,7 @@ static void cf_check spurious_interrupt(
         is_spurious = !nmi_check_continuation();
         if (this_cpu(state_dump_pending)) {
             this_cpu(state_dump_pending) = false;
-            dump_execstate(regs);
+            dump_execstate(get_irq_regs());
             is_spurious = false;
         }
 
@@ -1372,7 +1372,7 @@ static void cf_check spurious_interrupt(
  * This interrupt should never happen with our APIC/SMP architecture
  */
 
-static void cf_check error_interrupt(struct cpu_user_regs *regs)
+static void cf_check error_interrupt(void)
 {
     static const char *const esr_fields[] = {
         ", Send CS error",
@@ -1407,7 +1407,7 @@ static void cf_check error_interrupt(str
  * This interrupt handles performance counters interrupt
  */
 
-static void cf_check pmu_interrupt(struct cpu_user_regs *regs)
+static void cf_check pmu_interrupt(void)
 {
     ack_APIC_irq();
     vpmu_do_interrupt();
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -58,7 +58,7 @@ bool __read_mostly lmce_support;
 #define MCE_RING                0x1
 static DEFINE_PER_CPU(int, last_state);
 
-static void cf_check intel_thermal_interrupt(struct cpu_user_regs *regs)
+static void cf_check intel_thermal_interrupt(void)
 {
     uint64_t msr_content;
     unsigned int cpu = smp_processor_id();
@@ -642,7 +642,7 @@ static void cpu_mcheck_disable(void)
         clear_cmci();
 }
 
-static void cf_check cmci_interrupt(struct cpu_user_regs *regs)
+static void cf_check cmci_interrupt(void)
 {
     mctelem_cookie_t mctc;
     struct mca_summary bs;
--- a/xen/arch/x86/guest/xen/xen.c
+++ b/xen/arch/x86/guest/xen/xen.c
@@ -158,7 +158,7 @@ static void __init init_memmap(void)
     }
 }
 
-static void cf_check xen_evtchn_upcall(struct cpu_user_regs *regs)
+static void cf_check xen_evtchn_upcall(void)
 {
     struct vcpu_info *vcpu_info = this_cpu(vcpu_info);
     unsigned long pending;
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -2776,7 +2776,7 @@ static struct hvm_function_table __initd
 };
 
 /* Handle VT-d posted-interrupt when VCPU is blocked. */
-static void cf_check pi_wakeup_interrupt(struct cpu_user_regs *regs)
+static void cf_check pi_wakeup_interrupt(void)
 {
     struct vmx_vcpu *vmx, *tmp;
     spinlock_t *lock = &per_cpu(vmx_pi_blocking, smp_processor_id()).lock;
@@ -2808,7 +2808,7 @@ static void cf_check pi_wakeup_interrupt
 }
 
 /* Handle VT-d posted-interrupt when VCPU is running. */
-static void cf_check pi_notification_interrupt(struct cpu_user_regs *regs)
+static void cf_check pi_notification_interrupt(void)
 {
     ack_APIC_irq();
     this_cpu(irq_count)++;
--- a/xen/arch/x86/include/asm/irq.h
+++ b/xen/arch/x86/include/asm/irq.h
@@ -72,17 +72,15 @@ extern int opt_irq_vector_map;
 
 #define platform_legacy_irq(irq)	((irq) < 16)
 
-void cf_check event_check_interrupt(struct cpu_user_regs *regs);
-void cf_check invalidate_interrupt(struct cpu_user_regs *regs);
-void cf_check call_function_interrupt(struct cpu_user_regs *regs);
-void cf_check irq_move_cleanup_interrupt(struct cpu_user_regs *regs);
+void cf_check event_check_interrupt(void);
+void cf_check invalidate_interrupt(void);
+void cf_check call_function_interrupt(void);
+void cf_check irq_move_cleanup_interrupt(void);
 
 uint8_t alloc_hipriority_vector(void);
 
-void set_direct_apic_vector(
-    uint8_t vector, void (*handler)(struct cpu_user_regs *regs));
-void alloc_direct_apic_vector(
-    uint8_t *vector, void (*handler)(struct cpu_user_regs *regs));
+void set_direct_apic_vector(uint8_t vector, void (*handler)(void));
+void alloc_direct_apic_vector(uint8_t *vector, void (*handler)(void));
 
 void do_IRQ(struct cpu_user_regs *regs);
 
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -743,7 +743,7 @@ void move_native_irq(struct irq_desc *de
     desc->handler->enable(desc);
 }
 
-void cf_check irq_move_cleanup_interrupt(struct cpu_user_regs *regs)
+void cf_check irq_move_cleanup_interrupt(void)
 {
     unsigned vector, me;
 
@@ -913,16 +913,14 @@ uint8_t alloc_hipriority_vector(void)
     return next++;
 }
 
-static void (*direct_apic_vector[X86_NR_VECTORS])(struct cpu_user_regs *regs);
-void set_direct_apic_vector(
-    uint8_t vector, void (*handler)(struct cpu_user_regs *regs))
+static void (*direct_apic_vector[X86_NR_VECTORS])(void);
+void set_direct_apic_vector(uint8_t vector, void (*handler)(void))
 {
     BUG_ON(direct_apic_vector[vector] != NULL);
     direct_apic_vector[vector] = handler;
 }
 
-void alloc_direct_apic_vector(
-    uint8_t *vector, void (*handler)(struct cpu_user_regs *regs))
+void alloc_direct_apic_vector(uint8_t *vector, void (*handler)(void))
 {
     static DEFINE_SPINLOCK(lock);
 
@@ -1907,7 +1905,7 @@ void do_IRQ(struct cpu_user_regs *regs)
     if ( irq < 0 )
     {
         if ( direct_apic_vector[vector] )
-            direct_apic_vector[vector](regs);
+            direct_apic_vector[vector]();
         else
         {
             const char *kind = ", LAPIC";
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -245,7 +245,7 @@ static cpumask_t flush_cpumask;
 static const void *flush_va;
 static unsigned int flush_flags;
 
-void cf_check invalidate_interrupt(struct cpu_user_regs *regs)
+void cf_check invalidate_interrupt(void)
 {
     unsigned int flags = flush_flags;
     ack_APIC_irq();
@@ -387,14 +387,14 @@ void smp_send_nmi_allbutself(void)
     send_IPI_mask(&cpu_online_map, APIC_DM_NMI);
 }
 
-void cf_check event_check_interrupt(struct cpu_user_regs *regs)
+void cf_check event_check_interrupt(void)
 {
     ack_APIC_irq();
     perfc_incr(ipis);
     this_cpu(irq_count)++;
 }
 
-void cf_check call_function_interrupt(struct cpu_user_regs *regs)
+void cf_check call_function_interrupt(void)
 {
     ack_APIC_irq();
     perfc_incr(ipis);



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:32:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:32:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676063.1051863 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz5V-0006zk-EM; Mon, 05 Feb 2024 13:32:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676063.1051863; Mon, 05 Feb 2024 13:32:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWz5V-0006zd-B0; Mon, 05 Feb 2024 13:32:41 +0000
Received: by outflank-mailman (input) for mailman id 676063;
 Mon, 05 Feb 2024 13:32:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWz5T-0006zC-KT
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:32:39 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 06b4388b-c42b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 14:32:37 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-40fcb4ef7bcso24017385e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:32:37 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020a5d67ca000000b0033b3cf1ff09sm2278382wrw.29.2024.02.05.05.32.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:32:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06b4388b-c42b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707139956; x=1707744756; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=l1ReB3jZD1WlYuF9U7WzsUOq1WPtCay0Ptk0kyAo6EU=;
        b=OsdNTd6VMeUh4ga+XuwOrHQkxh7A6GWbkpi5eCDqNmXBkK834Jk91+PHum1oZG0uRY
         IbeMiLbL5Qcsp0gAJN2LvqZX9zOZb3D7Ex9VTfhtK91vPnvcB3e6bGEzOIo92tFh40HQ
         zY456ruFJqnqln0ALSAajjDy4OgmqWiB/QcBCFTZwxkrVwjiMN1CYanWby17eoWp6+rm
         WOt82awziojisi29NohcHVLvrug6WuVEYYY7qIsgqLDOQbNtCZ6pMMMbvgY7gVeAwfyr
         lMXhzKY6lwYYKLYIxOhtJ+aX4853V52XNuHTx2S4oP827WCjoKNAkVdg6BFD2lzTGult
         WSfQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707139956; x=1707744756;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=l1ReB3jZD1WlYuF9U7WzsUOq1WPtCay0Ptk0kyAo6EU=;
        b=kObTvxeaDCx9jVG2wR7dMXqgkc/SOPy93t5UEIkcCw4JVKLaHkvgIr3phM5ZkHdPoz
         CvMnEeLe7n8E4vc36YSQjFgGhgyJtGhP1z1mexpKMjlDefspBLD8NyoJZ53vxZnVRvez
         79QGkTmUuLi3mqt51Xi+BCl5e+YKubCOU3/wzIYrhTVRmtzpwncU2NGGDt2rEVilr9gR
         BpzYiVqWvTTIJ7dWUl/Y0mz9Nrc7xzfVdwY5wIxIlBEpiVLn8nTB4M7xPC/3/D33+ETj
         ZrhvecvoNs/Z19hHBa4NtvCl0EfvrXEYHEdNQ9wsedRXJ/qR8PihAlKP10ML25KWn8YI
         tIRA==
X-Gm-Message-State: AOJu0YxJqQwAQN+KpEOjBjZ6RBXbvelnNZ/lZ8+XkVL0PJCd2jM5fK1+
	pP55waFQkGLwK7rs6WO8bAMLBQGp2INcyA+u+dtjPlP0EdZ8xaWUfITLREbr9W2qhXEPg/H5jU4
	=
X-Google-Smtp-Source: AGHT+IF5ujFRkwOP55BlIw1cbRp7a5E9Ub1z36GcCR65nDdw4WLhkgsaaFcxuA6JhcSbFMJ2OgtpNg==
X-Received: by 2002:adf:fc07:0:b0:33b:381f:a89d with SMTP id i7-20020adffc07000000b0033b381fa89dmr2590949wrr.67.1707139956605;
        Mon, 05 Feb 2024 05:32:36 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUzG0P9zKzwFHwq5DPm4P9WDTXo3Zio+7hklMnfnsR5ZTOxFq+l7Aqqzb3rFGSatuJAUaPnHDX7I0n1fg8v7aPhvjDIsY42b2FhjXJRSW9pGCldwZTGy8+jl8BqgNMT7rDsN0WOdDG48EnSZ75tczgGbRCqN7de3njxXpq9UMLcU+Wzsi5Uzldk91rLha61HW8jwbDJWtGQN7avnT+dRarUUie1fDPZhpHozEAHlpjVA4shmyUOXSRZ5INT2kXwEdxTEUA/togY1DNLDDEMeFkiy2B1COzJJtqHOBYWylo=
Message-ID: <f20d2cef-4435-4b9a-9ad8-61ceddb6bcba@suse.com>
Date: Mon, 5 Feb 2024 14:32:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 8/8] consolidate do_bug_frame() / bug_fn_t
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

The type not being used in do_bug_frame() is suspicious. Apparently
that's solely because the type uses a pointer-to-const parameter,
when so far run_in_exception_handler() wanted functions taking pointer-
to-non-const. Expand use of const, in turn requiring common code's
do_bug_frame() as well as [gs]et_irq_regs() to also gain const. This
then brings the former function also closer to the common one, with
Arm's use of vaddr_t remaining as a difference.

While there also replace the bogus use of hard tabs in [gs]et_irq_regs()
(I clearly didn't mean to put it in like this).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
This is an alternative proposal to
https://lists.xen.org/archives/html/xen-devel/2023-12/msg01385.html.
---
v3: Retain / extend use of const. Make part of series.
v2: [skipped]

--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -220,7 +220,7 @@ void do_IRQ(struct cpu_user_regs *regs,
 {
     struct irq_desc *desc = irq_to_desc(irq);
     struct irqaction *action;
-    struct cpu_user_regs *old_regs = set_irq_regs(regs);
+    const struct cpu_user_regs *old_regs = set_irq_regs(regs);
 
     perfc_incr(irqs);
 
--- a/xen/arch/x86/include/asm/processor.h
+++ b/xen/arch/x86/include/asm/processor.h
@@ -409,8 +409,7 @@ static always_inline void rep_nop(void)
 void show_code(const struct cpu_user_regs *regs);
 void show_stack_overflow(unsigned int cpu, const struct cpu_user_regs *regs);
 void show_registers(const struct cpu_user_regs *regs);
-#define dump_execution_state() \
-    run_in_exception_handler(show_execution_state_nonconst)
+#define dump_execution_state() run_in_exception_handler(show_execution_state)
 void show_page_walk(unsigned long addr);
 void noreturn fatal_trap(const struct cpu_user_regs *regs, bool show_remote);
 
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -1896,7 +1896,7 @@ void do_IRQ(struct cpu_user_regs *regs)
     struct irq_desc  *desc;
     unsigned int      vector = (uint8_t)regs->entry_vector;
     int               irq = this_cpu(vector_irq)[vector];
-    struct cpu_user_regs *old_regs = set_irq_regs(regs);
+    const struct cpu_user_regs *old_regs = set_irq_regs(regs);
 
     perfc_incr(irqs);
     this_cpu(irq_count)++;
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -643,7 +643,7 @@ void show_stack_overflow(unsigned int cp
     printk("\n");
 }
 
-void show_execution_state(const struct cpu_user_regs *regs)
+void cf_check show_execution_state(const struct cpu_user_regs *regs)
 {
     /* Prevent interleaving of output. */
     unsigned long flags = console_lock_recursive_irqsave();
@@ -655,11 +655,6 @@ void show_execution_state(const struct c
     console_unlock_recursive_irqrestore(flags);
 }
 
-void cf_check show_execution_state_nonconst(struct cpu_user_regs *regs)
-{
-    show_execution_state(regs);
-}
-
 void vcpu_show_execution_state(struct vcpu *v)
 {
     unsigned long flags = 0;
--- a/xen/common/bug.c
+++ b/xen/common/bug.c
@@ -10,7 +10,7 @@
  * Returns a negative value in case of an error otherwise
  * BUGFRAME_{run_fn, warn, bug, assert}
  */
-int do_bug_frame(struct cpu_user_regs *regs, unsigned long pc)
+int do_bug_frame(const struct cpu_user_regs *regs, unsigned long pc)
 {
     const struct bug_frame *bug = NULL;
     const struct virtual_region *region;
@@ -44,14 +44,10 @@ int do_bug_frame(struct cpu_user_regs *r
 
     if ( id == BUGFRAME_run_fn )
     {
-        void (*fn)(struct cpu_user_regs *) = bug_ptr(bug);
+        bug_fn_t *fn = bug_ptr(bug);
 
         fn(regs);
 
-        /* Re-enforce consistent types, because of the casts involved. */
-        if ( false )
-            run_in_exception_handler(fn);
-
         return id;
     }
 
--- a/xen/common/irq.c
+++ b/xen/common/irq.c
@@ -1,7 +1,7 @@
 #include <xen/irq.h>
 #include <xen/errno.h>
 
-DEFINE_PER_CPU(struct cpu_user_regs *, irq_regs);
+DEFINE_PER_CPU(const struct cpu_user_regs *, irq_regs);
 
 int init_one_irq_desc(struct irq_desc *desc)
 {
--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -135,7 +135,7 @@ static void cf_check show_handlers(unsig
 
 static cpumask_t dump_execstate_mask;
 
-void cf_check dump_execstate(struct cpu_user_regs *regs)
+void cf_check dump_execstate(const struct cpu_user_regs *regs)
 {
     unsigned int cpu = smp_processor_id();
 
--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -1246,14 +1246,14 @@ static int cf_check ehci_dbgp_getc(struc
 /* Safe: ehci_dbgp_poll() runs as timer handler, so not reentrant. */
 static struct serial_port *poll_port;
 
-static void cf_check _ehci_dbgp_poll(struct cpu_user_regs *regs)
+static void cf_check _ehci_dbgp_poll(const struct cpu_user_regs *regs)
 {
     struct serial_port *port = poll_port;
     struct ehci_dbgp *dbgp = port->uart;
     unsigned long flags;
     unsigned int timeout = MICROSECS(DBGP_CHECK_INTERVAL);
     bool empty = false;
-    struct cpu_user_regs *old_regs;
+    const struct cpu_user_regs *old_regs;
 
     if ( !dbgp->ehci_debug )
         return;
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -206,11 +206,11 @@ static void cf_check ns16550_interrupt(i
 /* Safe: ns16550_poll() runs as softirq so not reentrant on a given CPU. */
 static DEFINE_PER_CPU(struct serial_port *, poll_port);
 
-static void cf_check __ns16550_poll(struct cpu_user_regs *regs)
+static void cf_check __ns16550_poll(const struct cpu_user_regs *regs)
 {
     struct serial_port *port = this_cpu(poll_port);
     struct ns16550 *uart = port->uart;
-    struct cpu_user_regs *old_regs;
+    const struct cpu_user_regs *old_regs;
 
     if ( uart->intr_works )
         return; /* Interrupts work - no more polling */
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -1164,7 +1164,7 @@ static void cf_check dbc_uart_poll(void
     struct dbc_uart *uart = port->uart;
     struct dbc *dbc = &uart->dbc;
     unsigned long flags = 0;
-    struct cpu_user_regs *old_regs;
+    const struct cpu_user_regs *old_regs;
 
     if ( spin_trylock_irqsave(&port->tx_lock, flags) )
     {
--- a/xen/include/xen/bug.h
+++ b/xen/include/xen/bug.h
@@ -101,8 +101,7 @@ typedef void bug_fn_t(const struct cpu_u
 
 #ifndef run_in_exception_handler
 
-static void always_inline run_in_exception_handler(
-    void (*fn)(struct cpu_user_regs *regs))
+static void always_inline run_in_exception_handler(bug_fn_t *fn)
 {
     BUG_FRAME(BUGFRAME_run_fn, 0, fn, 0, NULL);
 }
@@ -133,7 +132,7 @@ static void always_inline run_in_excepti
  * Returns a negative value in case of an error otherwise
  * BUGFRAME_{run_fn, warn, bug, assert}
  */
-int do_bug_frame(struct cpu_user_regs *regs, unsigned long pc);
+int do_bug_frame(const struct cpu_user_regs *regs, unsigned long pc);
 
 #endif /* CONFIG_GENERIC_BUG_FRAME */
 
--- a/xen/include/xen/irq.h
+++ b/xen/include/xen/irq.h
@@ -134,21 +134,22 @@ void cf_check irq_actor_none(struct irq_
  * Per-cpu interrupted context register state - the inner-most interrupt frame
  * on the stack.
  */
-DECLARE_PER_CPU(struct cpu_user_regs *, irq_regs);
+DECLARE_PER_CPU(const struct cpu_user_regs *, irq_regs);
 
-static inline struct cpu_user_regs *get_irq_regs(void)
+static inline const struct cpu_user_regs *get_irq_regs(void)
 {
-	return this_cpu(irq_regs);
+    return this_cpu(irq_regs);
 }
 
-static inline struct cpu_user_regs *set_irq_regs(struct cpu_user_regs *new_regs)
+static inline const struct cpu_user_regs *set_irq_regs(
+    const struct cpu_user_regs *new_regs)
 {
-	struct cpu_user_regs *old_regs, **pp_regs = &this_cpu(irq_regs);
+    const struct cpu_user_regs *old_regs, **pp_regs = &this_cpu(irq_regs);
 
-	old_regs = *pp_regs;
-	*pp_regs = new_regs;
+    old_regs = *pp_regs;
+    *pp_regs = new_regs;
 
-	return old_regs;
+    return old_regs;
 }
 
 struct domain;
--- a/xen/include/xen/kernel.h
+++ b/xen/include/xen/kernel.h
@@ -110,8 +110,7 @@ extern const unsigned int xen_config_dat
 struct cpu_user_regs;
 struct vcpu;
 
-void show_execution_state(const struct cpu_user_regs *regs);
-void cf_check show_execution_state_nonconst(struct cpu_user_regs *regs);
+void cf_check show_execution_state(const struct cpu_user_regs *regs);
 void vcpu_show_execution_state(struct vcpu *v);
 
 #endif /* _LINUX_KERNEL_H */
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -173,7 +173,7 @@ extern char *print_tainted(char *str);
 extern void add_taint(unsigned int taint);
 
 struct cpu_user_regs;
-void cf_check dump_execstate(struct cpu_user_regs *regs);
+void cf_check dump_execstate(const struct cpu_user_regs *regs);
 
 void init_constructors(void);
 



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:37:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:37:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676066.1051873 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzAU-0007lQ-0D; Mon, 05 Feb 2024 13:37:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676066.1051873; Mon, 05 Feb 2024 13:37:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzAT-0007lJ-Tg; Mon, 05 Feb 2024 13:37:49 +0000
Received: by outflank-mailman (input) for mailman id 676066;
 Mon, 05 Feb 2024 13:37:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzAS-0007l7-2D
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:37:48 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bf1f29c4-c42b-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:37:46 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-40fb63c40c0so39433665e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:37:46 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f9-20020a05600c4e8900b0040f035bebfcsm8625476wmq.12.2024.02.05.05.37.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:37:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf1f29c4-c42b-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707140266; x=1707745066; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=07aP1qEUKfU9p7fDjH02hl2Wgx9V9VqMoOdijrGIkAk=;
        b=eCxBGvqHEdz2eWnhgkBVOolFsi7fJewod/mwqWstWnTt4rajUQVxkdtSeK47vlY6c6
         f4UPZyDxpl4dwenBJk4pIxlm115VZ1FhD+ywWmrs7KFrio3YXn5OS0m+RHCRSi3pp51S
         6HvII86pSxU8s38KMc2VWJx5CBRJpp6Bv6D9tgXL/K3RCcA0NZ1KFUw1aWRAbdahJoGY
         JHXQL2yurKCZFi7OVemu2NuZrSr9ytnSWNmJbV2GNdwCgwqpSk26pPM8UHWKjSmnLrYp
         KHv8U/r/1HZmgykPKMyOW3Vps1TQlYIzOtsnLLMcEueC+3O9WllfNAEccsX81JMpRR81
         WbAg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707140266; x=1707745066;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=07aP1qEUKfU9p7fDjH02hl2Wgx9V9VqMoOdijrGIkAk=;
        b=PanFpET/74xqyY/rN6QPZL13t0OFxVG7XwWV8BKZMM+xzg9HoDHbhrfyMcCW5+7ivd
         qwE8rHvjEXVwAwm3lcK5+xnF1mUHLl6mQYzAWuPc28wCfdB+jXA1OdCg2GwuwC+iVm9a
         mVbf2Y70/s8borkLKS2NadrUGgy1GXxH2qHS1DpGyjnI743Dq38PM62KEQHtSWdUVkJe
         rgbZh+Xvz4hcyjGVj/TxG6oVOCzpMyGBxvajuJpSAQxP9xLogJYxRKsEwH8UZh4nN+D/
         w3uRfxf/Lw+bf30NWeze7+x0yWbeuHomVxdksXx6IENL/gRfCxN0FJ6RQ3L4FXPuO0N9
         J9Tw==
X-Gm-Message-State: AOJu0YzYgVrUSI4+9YGVOGyXLsh2UXkupDx3IY0W/8MXXyiWn1iiCHq0
	l25Cggz6UH+bY6r8BFIcTkbaUbkCCcussATx0swkltbe7uHOeQpjoVv68jQaD6x16JD6JU5E1D0
	=
X-Google-Smtp-Source: AGHT+IEJB1Xik0xxmpKBk6ml66jNYoIJ8zz8EiPv6nzfeKDtgWnW0Adn8C9VdjUfu3QlSSkGpwAJDQ==
X-Received: by 2002:a05:600c:1c88:b0:40d:8bc2:6059 with SMTP id k8-20020a05600c1c8800b0040d8bc26059mr4904521wms.36.1707140265918;
        Mon, 05 Feb 2024 05:37:45 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWXs1K47j0vODaBHD2YixDtZniESQOohEYalDit9pxWRSDDqj8ycL1T7kPNOmhzogkrBpP0oEX8tg7tgGOkLSG/2BPmK8yWePs54H+ecyZBeeW+fJQPbnrGnkZkubcx8W0OpAVp10G1dO9br1O9qWrZNhzSs1od9vTfTU+8/b0=
Message-ID: <bc782b14-d897-4a94-b71d-97c4abeb85df@suse.com>
Date: Mon, 5 Feb 2024 14:37:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v4.5 3/8] VMX: tertiary execution control infrastructure
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <9dd23064-c79e-4a50-9c71-c0e73b189944@suse.com>
 <aa553449-888f-4e52-85b7-0bc0b7f010b4@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <aa553449-888f-4e52-85b7-0bc0b7f010b4@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

This is a prereq to enabling e.g. the MSRLIST feature.

Note that the PROCBASED_CTLS3 MSR is different from other VMX feature
reporting MSRs, in that all 64 bits report allowed 1-settings.

vVMX code is left alone, though, for the time being.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v4.5: Bump zero-padding width in vmcs_dump_vcpu(). Add
      TERTIARY_EXEC_VIRT_SPEC_CTRL. Constify
      vmx_update_tertiary_exec_control()'s parameter. Re-base.
v2: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -164,6 +164,7 @@ static int cf_check parse_ept_param_runt
 u32 vmx_pin_based_exec_control __read_mostly;
 u32 vmx_cpu_based_exec_control __read_mostly;
 u32 vmx_secondary_exec_control __read_mostly;
+uint64_t vmx_tertiary_exec_control __read_mostly;
 u32 vmx_vmexit_control __read_mostly;
 u32 vmx_vmentry_control __read_mostly;
 u64 vmx_ept_vpid_cap __read_mostly;
@@ -228,10 +229,32 @@ static u32 adjust_vmx_controls(
     return ctl;
 }
 
-static bool cap_check(const char *name, u32 expected, u32 saw)
+static uint64_t adjust_vmx_controls2(
+    const char *name, uint64_t ctl_min, uint64_t ctl_opt, unsigned int msr,
+    bool *mismatch)
+{
+    uint64_t vmx_msr, ctl = ctl_min | ctl_opt;
+
+    rdmsrl(msr, vmx_msr);
+
+    ctl &= vmx_msr; /* bit == 0 ==> must be zero */
+
+    /* Ensure minimum (required) set of control bits are supported. */
+    if ( ctl_min & ~ctl )
+    {
+        *mismatch = true;
+        printk("VMX: CPU%u has insufficient %s (%#lx; requires %#lx)\n",
+               smp_processor_id(), name, ctl, ctl_min);
+    }
+
+    return ctl;
+}
+
+static bool cap_check(
+    const char *name, unsigned long expected, unsigned long saw)
 {
     if ( saw != expected )
-        printk("VMX %s: saw %#x expected %#x\n", name, saw, expected);
+        printk("VMX %s: saw %#lx expected %#lx\n", name, saw, expected);
     return saw != expected;
 }
 
@@ -241,6 +264,7 @@ static int vmx_init_vmcs_config(bool bsp
     u32 _vmx_pin_based_exec_control;
     u32 _vmx_cpu_based_exec_control;
     u32 _vmx_secondary_exec_control = 0;
+    uint64_t _vmx_tertiary_exec_control = 0;
     u64 _vmx_ept_vpid_cap = 0;
     u64 _vmx_misc_cap = 0;
     u32 _vmx_vmexit_control;
@@ -274,7 +298,8 @@ static int vmx_init_vmcs_config(bool bsp
     opt = (CPU_BASED_ACTIVATE_MSR_BITMAP |
            CPU_BASED_TPR_SHADOW |
            CPU_BASED_MONITOR_TRAP_FLAG |
-           CPU_BASED_ACTIVATE_SECONDARY_CONTROLS);
+           CPU_BASED_ACTIVATE_SECONDARY_CONTROLS |
+           CPU_BASED_ACTIVATE_TERTIARY_CONTROLS);
     _vmx_cpu_based_exec_control = adjust_vmx_controls(
         "CPU-Based Exec Control", min, opt,
         MSR_IA32_VMX_PROCBASED_CTLS, &mismatch);
@@ -338,6 +363,15 @@ static int vmx_init_vmcs_config(bool bsp
             MSR_IA32_VMX_PROCBASED_CTLS2, &mismatch);
     }
 
+    if ( _vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS )
+    {
+        uint64_t opt = 0;
+
+        _vmx_tertiary_exec_control = adjust_vmx_controls2(
+            "Tertiary Exec Control", 0, opt,
+            MSR_IA32_VMX_PROCBASED_CTLS3, &mismatch);
+    }
+
     /* The IA32_VMX_EPT_VPID_CAP MSR exists only when EPT or VPID available */
     if ( _vmx_secondary_exec_control & (SECONDARY_EXEC_ENABLE_EPT |
                                         SECONDARY_EXEC_ENABLE_VPID) )
@@ -468,6 +502,7 @@ static int vmx_init_vmcs_config(bool bsp
         vmx_pin_based_exec_control = _vmx_pin_based_exec_control;
         vmx_cpu_based_exec_control = _vmx_cpu_based_exec_control;
         vmx_secondary_exec_control = _vmx_secondary_exec_control;
+        vmx_tertiary_exec_control  = _vmx_tertiary_exec_control;
         vmx_ept_vpid_cap           = _vmx_ept_vpid_cap;
         vmx_vmexit_control         = _vmx_vmexit_control;
         vmx_vmentry_control        = _vmx_vmentry_control;
@@ -503,6 +538,9 @@ static int vmx_init_vmcs_config(bool bsp
             "Secondary Exec Control",
             vmx_secondary_exec_control, _vmx_secondary_exec_control);
         mismatch |= cap_check(
+            "Tertiary Exec Control",
+            vmx_tertiary_exec_control, _vmx_tertiary_exec_control);
+        mismatch |= cap_check(
             "VMExit Control",
             vmx_vmexit_control, _vmx_vmexit_control);
         mismatch |= cap_check(
@@ -1080,6 +1118,7 @@ static int construct_vmcs(struct vcpu *v
         v->arch.hvm.vmx.exec_control |= CPU_BASED_RDTSC_EXITING;
 
     v->arch.hvm.vmx.secondary_exec_control = vmx_secondary_exec_control;
+    v->arch.hvm.vmx.tertiary_exec_control  = vmx_tertiary_exec_control;
 
     /*
      * Disable features which we don't want active by default:
@@ -1134,6 +1173,10 @@ static int construct_vmcs(struct vcpu *v
         __vmwrite(SECONDARY_VM_EXEC_CONTROL,
                   v->arch.hvm.vmx.secondary_exec_control);
 
+    if ( cpu_has_vmx_tertiary_exec_control )
+        __vmwrite(TERTIARY_VM_EXEC_CONTROL,
+                  v->arch.hvm.vmx.tertiary_exec_control);
+
     /* MSR access bitmap. */
     if ( cpu_has_vmx_msr_bitmap )
     {
@@ -2068,10 +2111,12 @@ void vmcs_dump_vcpu(struct vcpu *v)
                vmr(HOST_PERF_GLOBAL_CTRL));
 
     printk("*** Control State ***\n");
-    printk("PinBased=%08x CPUBased=%08x SecondaryExec=%08x\n",
+    printk("PinBased=%08x CPUBased=%08x\n",
            vmr32(PIN_BASED_VM_EXEC_CONTROL),
-           vmr32(CPU_BASED_VM_EXEC_CONTROL),
-           vmr32(SECONDARY_VM_EXEC_CONTROL));
+           vmr32(CPU_BASED_VM_EXEC_CONTROL));
+    printk("SecondaryExec=%08x TertiaryExec=%016lx\n",
+           vmr32(SECONDARY_VM_EXEC_CONTROL),
+           vmr(TERTIARY_VM_EXEC_CONTROL));
     printk("EntryControls=%08x ExitControls=%08x\n", vmentry_ctl, vmexit_ctl);
     printk("ExceptionBitmap=%08x PFECmask=%08x PFECmatch=%08x\n",
            vmr32(EXCEPTION_BITMAP),
@@ -2174,6 +2219,7 @@ int __init vmx_vmcs_init(void)
         vmx_pin_based_exec_control = 0;
         vmx_cpu_based_exec_control = 0;
         vmx_secondary_exec_control = 0;
+        vmx_tertiary_exec_control  = 0;
         vmx_vmexit_control         = 0;
         vmx_vmentry_control        = 0;
         vmx_ept_vpid_cap           = 0;
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -114,6 +114,7 @@ struct vmx_vcpu {
     /* Cache of cpu execution control. */
     u32                  exec_control;
     u32                  secondary_exec_control;
+    uint64_t             tertiary_exec_control;
     u32                  exception_bitmap;
 
     uint64_t             shadow_gs;
@@ -196,6 +197,7 @@ void vmx_vmcs_reload(struct vcpu *v);
 #define CPU_BASED_RDTSC_EXITING               0x00001000U
 #define CPU_BASED_CR3_LOAD_EXITING            0x00008000U
 #define CPU_BASED_CR3_STORE_EXITING           0x00010000U
+#define CPU_BASED_ACTIVATE_TERTIARY_CONTROLS  0x00020000U
 #define CPU_BASED_CR8_LOAD_EXITING            0x00080000U
 #define CPU_BASED_CR8_STORE_EXITING           0x00100000U
 #define CPU_BASED_TPR_SHADOW                  0x00200000U
@@ -260,6 +262,14 @@ extern u32 vmx_vmentry_control;
 #define SECONDARY_EXEC_NOTIFY_VM_EXITING        0x80000000U
 extern u32 vmx_secondary_exec_control;
 
+#define TERTIARY_EXEC_LOADIWKEY_EXITING         BIT(0, UL)
+#define TERTIARY_EXEC_ENABLE_HLAT               BIT(1, UL)
+#define TERTIARY_EXEC_EPT_PAGING_WRITE          BIT(2, UL)
+#define TERTIARY_EXEC_GUEST_PAGING_VERIFY       BIT(3, UL)
+#define TERTIARY_EXEC_IPI_VIRT                  BIT(4, UL)
+#define TERTIARY_EXEC_VIRT_SPEC_CTRL            BIT(7, UL)
+extern uint64_t vmx_tertiary_exec_control;
+
 #define VMX_EPT_EXEC_ONLY_SUPPORTED                         0x00000001
 #define VMX_EPT_WALK_LENGTH_4_SUPPORTED                     0x00000040
 #define VMX_EPT_MEMORY_TYPE_UC                              0x00000100
@@ -296,6 +306,8 @@ extern u64 vmx_ept_vpid_cap;
     (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_MSR_BITMAP)
 #define cpu_has_vmx_secondary_exec_control \
     (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS)
+#define cpu_has_vmx_tertiary_exec_control \
+    (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS)
 #define cpu_has_vmx_ept \
     (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_EPT)
 #define cpu_has_vmx_dt_exiting \
@@ -423,6 +435,7 @@ enum vmcs_field {
     VIRT_EXCEPTION_INFO             = 0x0000202a,
     XSS_EXIT_BITMAP                 = 0x0000202c,
     TSC_MULTIPLIER                  = 0x00002032,
+    TERTIARY_VM_EXEC_CONTROL        = 0x00002034,
     GUEST_PHYSICAL_ADDRESS          = 0x00002400,
     VMCS_LINK_POINTER               = 0x00002800,
     GUEST_IA32_DEBUGCTL             = 0x00002802,
--- a/xen/arch/x86/include/asm/msr-index.h
+++ b/xen/arch/x86/include/asm/msr-index.h
@@ -347,6 +347,7 @@
 #define MSR_IA32_VMX_TRUE_EXIT_CTLS             0x48f
 #define MSR_IA32_VMX_TRUE_ENTRY_CTLS            0x490
 #define MSR_IA32_VMX_VMFUNC                     0x491
+#define MSR_IA32_VMX_PROCBASED_CTLS3            0x492
 
 /* K7/K8 MSRs. Not complete. See the architecture manual for a more
    complete list. */
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -760,6 +760,12 @@ void vmx_update_secondary_exec_control(s
                   v->arch.hvm.vmx.secondary_exec_control);
 }
 
+void vmx_update_tertiary_exec_control(const struct vcpu *v)
+{
+    __vmwrite(TERTIARY_VM_EXEC_CONTROL,
+              v->arch.hvm.vmx.tertiary_exec_control);
+}
+
 void vmx_update_exception_bitmap(struct vcpu *v)
 {
     u32 bitmap = unlikely(v->arch.hvm.vmx.vmx_realmode)
--- a/xen/arch/x86/include/asm/hvm/vmx/vmx.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmx.h
@@ -81,6 +81,7 @@ void vmx_realmode(struct cpu_user_regs *
 void vmx_update_exception_bitmap(struct vcpu *v);
 void vmx_update_cpu_exec_control(struct vcpu *v);
 void vmx_update_secondary_exec_control(struct vcpu *v);
+void vmx_update_tertiary_exec_control(const struct vcpu *v);
 
 #define POSTED_INTR_ON  0
 #define POSTED_INTR_SN  1



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:37:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:37:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676067.1051883 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzAV-0007zr-CL; Mon, 05 Feb 2024 13:37:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676067.1051883; Mon, 05 Feb 2024 13:37:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzAV-0007zk-7q; Mon, 05 Feb 2024 13:37:51 +0000
Received: by outflank-mailman (input) for mailman id 676067;
 Mon, 05 Feb 2024 13:37:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W3Sh=JO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rWzAT-0007lD-9T
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:37:49 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bf48ad15-c42b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 14:37:46 +0100 (CET)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-5605c7b1f32so1260760a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:37:46 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 t6-20020aa7d706000000b005605ebacec5sm1316929edq.43.2024.02.05.05.37.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:37:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf48ad15-c42b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707140266; x=1707745066; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=FnfkxvKA/F7T4oAMOqBwsz1+RAZM28v3kElo3Ihlm40=;
        b=N7oOqWurJC3ly7Kt9R9NDxGOf6FXOCE4o7KYqY4vicYQSll7dpEljrcrJW3uEQnHT6
         JAIupPWJrPRcOVw6AN2NM5l0bnnQXhNjczwmu3Ph4owtebWI/2WF5LK0bOx9PicOW2T0
         COeThe+QJyNvHg3uI16fPQiUtJpqcxMt3o2n0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707140266; x=1707745066;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=FnfkxvKA/F7T4oAMOqBwsz1+RAZM28v3kElo3Ihlm40=;
        b=sP6pUMj/YgUbA5BXjy+AyTIojshG9RetaRGIPWTEPl8a4TEJHiKkwTOSot2GpPHKVM
         Iri3eOWplDsMLJ4BBRXO1cIqwMJDMVI2wYrznJceB2FpiSjDjLz+J/9G1hEQo+0c/aSz
         c2kMv67D8lznId2ZSQpPZ18Xhc631RS+wmUoBHDiG+saIeQEruO9OU7UrsGgdjq57wOL
         74rMfp4ApaMQQmrb2z9z3JuvEazdyQUdiPUDHWuPpyDGtSDfWAdxAjELpsJ9TibgyzPX
         PvvSPw3N+6QDd4eA7ETn3ym7f6W9BL7OHnGiwewGS6P+f6hsx29IiNLzfPMw1l4nPj9M
         9cYw==
X-Gm-Message-State: AOJu0YzutLs+fUjvjenexGgcY5waPWbMZHmCYPEAxd3n4WOMIGVh7Wh/
	lUXLY+6iEHqQsb/8/HD8rBhKmiaUUb1pFAodsmVSf1T8Kpk9o40Kojv4ufExG1Y=
X-Google-Smtp-Source: AGHT+IEdjklQk1J1uFuhCunR9Omu2Q8a/zfcPD3G9knP9utjOu0NtazleXZNdCLZ6wfO+MHl9xMGNA==
X-Received: by 2002:a05:6402:2c3:b0:560:e6d:4ce9 with SMTP id b3-20020a05640202c300b005600e6d4ce9mr4074250edx.41.1707140266348;
        Mon, 05 Feb 2024 05:37:46 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWFkthbLqPvaJSaA9iBjEobrjoS8HfMQoyannNe3oMrwtpazljfnUWFz07v4/RZKuYqkvIAiZDSW895cSaqs07vtUKC1zlVZqgs/jX3jozoCKajdSSL/4wUjRM+iO69g1X7Gfi0sxCIphNApPG/OF/7sJcWy/w6Dj17tiS31wOiYxNs
Message-ID: <35a04808-9ff3-474d-a394-169fd787d468@citrix.com>
Date: Mon, 5 Feb 2024 13:37:45 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 3/8] serial: drop serial_rx_fn's regs parameter
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <df7f2df7-d45c-4c96-a84a-5ca816a5b733@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <df7f2df7-d45c-4c96-a84a-5ca816a5b733@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05/02/2024 1:28 pm, Jan Beulich wrote:
> It's simply not needed anymore.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:39:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:39:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676073.1051893 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzBs-0000dU-Lk; Mon, 05 Feb 2024 13:39:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676073.1051893; Mon, 05 Feb 2024 13:39:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzBs-0000dN-II; Mon, 05 Feb 2024 13:39:16 +0000
Received: by outflank-mailman (input) for mailman id 676073;
 Mon, 05 Feb 2024 13:39:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W3Sh=JO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rWzBr-0000bx-Oa
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:39:15 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f32ff78f-c42b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 14:39:13 +0100 (CET)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-55ff4dbe6a8so3505902a12.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:39:13 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 t6-20020aa7d706000000b005605ebacec5sm1316929edq.43.2024.02.05.05.39.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:39:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f32ff78f-c42b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707140353; x=1707745153; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=XLD+1dJ8ux83BRuEoUgHNkrO0SO+RLAYOKnM2f3QAyc=;
        b=bt/poVZvtEqycEH+ADhkPgM2/r9zO/UkeLtuoX0gHZpRSBEP9b5gtTc3pxEfhmVU6a
         J9sGIjHT1R5PP1M8t5qvt8vGdfiSYlot4tPd+zKAEz6PL9bNpRZ4AR9hl4zRaTausfyn
         DMA1IqFFX5ffEhqxK/QGCu4QayibVoI0L80CQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707140353; x=1707745153;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=XLD+1dJ8ux83BRuEoUgHNkrO0SO+RLAYOKnM2f3QAyc=;
        b=ZBQbI+HYjdGA2wU807aNJbOLsgMO7t8DtCMXJMxtlqkDTzcEO/grH7WrbpWRV6IbmP
         lOVIvbFgy0hSvyajVWY/NNDtGhbz2b6mnoVACjxQv7ZzUhSJCHuX3Ys5+wKIzbhL2xw+
         n31YAZKy82bp5pDz3m4Txj57FUICtYPhlvv+/NmjvORk50AyrWJG4xB4bqzDg7dBGVRr
         l3Kktg651c9Y2GInU52w8NfehOMeIX2OT0iTBFsiKGP2q4w7+SfYyrRg3iA4/DhaOJqD
         UL2fGMnhCelAcTNxo38jvpD+mdSCCqm/Q8RR3/mVgpYM1YPIM0Du22OzX65w2e6YT/EO
         Xhsw==
X-Gm-Message-State: AOJu0YyLIatOHYpvKqoiRk0PyVSuGVMTyunMbdEwctWCSeUNuiNHuBkb
	ChDRnFZlwdlP/6wqKgqL9Kd9Ti6TC+L35oST/hGxcRvNjUQdUV7myX+2kalmeGQ=
X-Google-Smtp-Source: AGHT+IFFl+BgrLXhvNem+b23szpWzo0Dmp46SOwDZqtr0uhPp6HTKgDrhlKQDccve/DaSSBPFRWbwg==
X-Received: by 2002:aa7:d58b:0:b0:55f:dbb8:9163 with SMTP id r11-20020aa7d58b000000b0055fdbb89163mr5091899edq.37.1707140353466;
        Mon, 05 Feb 2024 05:39:13 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVnQzaiqmLcyao6a1BcwhYLexToqXEXHds1VEy4BVNNBD1RFGJZpBcMZaeSu54Wt3M10NUIbjCxKAO8y8qyJyjF5V33g6XVDGoZeAttIQG/hSRD7AxGrHicH3e9p3fqv1ADZMdIy68DC1neBoIneIPKq74s/rQTIr4WwcPQZRTuYVXZ/A7uGbu0CEbki1MzqvfG2PPepp99NGgJEKDIF4J1WApN7fKE33NbsUO7BKl/azd0Gbao8VjyY81jKSNmWmiSRGTyrZKKCY0QpxLkjw==
Message-ID: <89c8e8bd-8a76-42e6-baee-0a03d021cdca@citrix.com>
Date: Mon, 5 Feb 2024 13:39:12 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 5/8] serial: drop serial_[rt]x_interrupt()'s regs
 parameter
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <93aec041-70f6-4385-9a52-8808e0cea931@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <93aec041-70f6-4385-9a52-8808e0cea931@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05/02/2024 1:30 pm, Jan Beulich wrote:
> They're simply not needed anymore.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:51:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:51:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676083.1051902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzNv-0004Mi-OT; Mon, 05 Feb 2024 13:51:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676083.1051902; Mon, 05 Feb 2024 13:51:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzNv-0004Mb-Lf; Mon, 05 Feb 2024 13:51:43 +0000
Received: by outflank-mailman (input) for mailman id 676083;
 Mon, 05 Feb 2024 13:51:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W3Sh=JO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rWzNu-0004MV-BL
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:51:42 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b08ca260-c42d-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:51:41 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-40fdc500db5so5781195e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:51:41 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 fj3-20020a05600c0c8300b0040fdd361c5fsm1902192wmb.47.2024.02.05.05.51.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:51:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b08ca260-c42d-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707141101; x=1707745901; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=oa4bBqi9S19tUVsoSzmhSdurXd96E8njIM7dJUC1EpY=;
        b=RIh2AFyw0pWtWGXQL1pefOMBDy5hGa/ZoA9uSN+RwcNk536kpn4GD9BwAS9uQtccxu
         m/QeVnDOXcoZiufclpcMadSkaknBG70euhW6VKJljbTwfK58rvAc5uwQhGhfz++2hI3e
         SvhAItqIFnMDyYuvSzfryeI3j2ASUR3VVvOyY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141101; x=1707745901;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=oa4bBqi9S19tUVsoSzmhSdurXd96E8njIM7dJUC1EpY=;
        b=mKH0fPqjCZZxRic7J1bGXmfrbGSgg0KD6rNNn3evH+gRyXqfKeV+PDzSTocB4flPXL
         LbpIVKZW+1kBrxoagqq+NP1xYFrdBhhHlT400KYpy0WbbZK1zF13aZK6p1Rm7DPzZXKV
         LcGpGxIlAjsacK4P1YcxD6AefoAIvxyAXVxaY6e/TY8fX8x1kBdksmBlH5d92ULEPwz1
         oW1ElLLEoVBrlF+MbbRf1Z/LWYk09JUDKZc4zJ48izdXtDPFpxKjg7qoR6/Yc4xIDLtL
         mIp3j9gxvwL8ck9w3/ES1p9FpPCla1/FKt8IDQ0VtsirGXPL8FGcmSFJ9uutXXxmDIUs
         DqbQ==
X-Gm-Message-State: AOJu0YwrT4hfR9zjbbI6xMbDi6h4QOzjMbppts54d4gBQclRTC0FoIon
	B6pOyUogMZyszRVMjkxaYwiFqbxTsjrwbnUzbtQqlUDArpabDm3OotMUgxf4QWE=
X-Google-Smtp-Source: AGHT+IEIKbafgRlyI/ns248dlUKYhMt3avcs4W0vGOtzIFr7TL2F+9E8U4S4k9TqzPpuvJTtUIxssQ==
X-Received: by 2002:a5d:6da9:0:b0:33b:3b86:b881 with SMTP id u9-20020a5d6da9000000b0033b3b86b881mr3160495wrs.38.1707141100725;
        Mon, 05 Feb 2024 05:51:40 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUf6zHtOb7u5CxHNTdniPQ2/u/Fq0M8tVKk6y9+bv3O6ycNFaoe3kAF3ln5LFJm5+t2A/v8thc3qH0N0dLQfXgyQIuMnVKejPjUek8Ut5GiIbB63utlgAoENvfY4OTOWJEtt+s2D0Q19fCf65T5Ymxn4dDXjKOOePRcvGGkNeDnLf8n87s3RDxsgRLN2C4I98h5thRr11b5aYdq+7ga933tF5IDlfUBS4fX0A2dgXk/74BKDr/Qr5R3hfbjshGxNsNx4Jl9XQs7vxDUxEgATMLLctzpo+8HyHWsXwLOEuU+SLdsiXs=
Message-ID: <f97088c6-ec2e-4c60-b91d-ec1a30d077df@citrix.com>
Date: Mon, 5 Feb 2024 13:51:40 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 8/8] consolidate do_bug_frame() / bug_fn_t
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <f20d2cef-4435-4b9a-9ad8-61ceddb6bcba@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <f20d2cef-4435-4b9a-9ad8-61ceddb6bcba@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 05/02/2024 1:32 pm, Jan Beulich wrote:
> The type not being used in do_bug_frame() is suspicious. Apparently
> that's solely because the type uses a pointer-to-const parameter,
> when so far run_in_exception_handler() wanted functions taking pointer-
> to-non-const. Expand use of const, in turn requiring common code's
> do_bug_frame() as well as [gs]et_irq_regs() to also gain const. This
> then brings the former function also closer to the common one, with
> Arm's use of vaddr_t remaining as a difference.
>
> While there also replace the bogus use of hard tabs in [gs]et_irq_regs()
> (I clearly didn't mean to put it in like this).

I meant to query that at the time and clearly forgot to.

> Signed-off-by: Jan Beulich <jbeulich@suse.com>

I'm still confident we can get rid of the fake frame in the serial
drivers, but this is an improvement nonetheless.

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

I'll rebase my work over this.  It's going to collide horribly.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:53:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:53:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676086.1051914 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzPW-00055M-41; Mon, 05 Feb 2024 13:53:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676086.1051914; Mon, 05 Feb 2024 13:53:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzPV-00055F-Vc; Mon, 05 Feb 2024 13:53:21 +0000
Received: by outflank-mailman (input) for mailman id 676086;
 Mon, 05 Feb 2024 13:53:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzPV-000559-4p
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:53:21 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id eb68434a-c42d-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:53:19 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-40fdc653665so6486715e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:53:20 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i11-20020a05600c354b00b0040fc76ed923sm8733163wmq.6.2024.02.05.05.53.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:53:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb68434a-c42d-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707141199; x=1707745999; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=DIKjvlVFi5CkKNLwlSC6jlcSYTnjZwuK2qb5cT77qAw=;
        b=BM7u4dQr9bLjPafSMCpRGr9BT1E3tQsnAggTT/6Jd+JT2rA25Ls4XtBQlTycW56D55
         6pYmZivgYbarPPaf5NdU+Wz9dRyYDCW7JcHvagWWSmXCXm1+93DN+VrsQNEjaConZ8FC
         XHEx+Ss/2q8FxA3PTyMD6W/oegoXWb06+tn83Q6MxC92SAiAetU0IFGLKp76878Er741
         3WdmXJFcit2m0dU7k0aoFuK6jXVnqN3y+FZ7s8Ghr5HZd7MwrTY9P4StkuQJP6d03xcV
         ZS3skGUptbD8Ypxb2Ro6oCf5ajXZhg5KN7nBYMMGVpUOTHvZ0zl1Zmjo7kDGXIj3qWI0
         wkIA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141199; x=1707745999;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=DIKjvlVFi5CkKNLwlSC6jlcSYTnjZwuK2qb5cT77qAw=;
        b=DAjN0f/8683kOPbCCW2J12Kfb34d+SOUUg7Elk7MER46+qGc23WQpT3uhwMMbdGPoJ
         v6SVh5Mq0qQWwf4mEGY8ERVoXPHtfZ1+dbogBsbenpPu03s4JTjYm5MpE5kPF8I8/s/R
         aPNeAGE/KVnhsWJQjAQtzczYE3zs/8Udsql26yCp88JOc7qoxE20q+lE5a3dBn2zlMlo
         P8dtVL2BZbfye6f5Njuei8fwWosc4Dnw7dl/GTAAUtM6Uy/n3GHNTxg8XRkkMTaqm5Yv
         +Hn/VADTITDkMJPdrpeFreuTWpZ4j+pLOQnUmR0eARDR5ZNEz/fuEJ+UF2lNxIgPSEC9
         Vp/g==
X-Gm-Message-State: AOJu0YywcAL2XkIPW74pd0iVfhvc8xDRLW8hUHVUmGnbgZxTEy2/ShvZ
	ZJlSCSMBGMDY1SeIoUmRHZbON88VuM6kwXRe+Iu7EBdIJU4n71Zxr3SW4NCiiJ/3pWAzO6+taK8
	=
X-Google-Smtp-Source: AGHT+IHs2ujKhyGseOly8on/SYHeEp19k590Bb3bNtNsSuvLOS0B5sSukZEsVkDwE7MjYnZZnafLzw==
X-Received: by 2002:adf:e50f:0:b0:33b:1ac9:1545 with SMTP id j15-20020adfe50f000000b0033b1ac91545mr7656047wrm.17.1707141199504;
        Mon, 05 Feb 2024 05:53:19 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWBIQGtkyXi2j0tqiTFXHiVLevIAVSsJfACJXnoa707jZzkY24vls+WEmiZr5qbDyWWeV1bB6MREOXsp/bF1S0wBcvoQUMqwEUerEvOskpyB4yt
Message-ID: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Date: Mon, 5 Feb 2024 14:53:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/7] VT-d: SATC handling and ATS tidying
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On a Sapphire Rapids system I noticed a line in the serial log
indicating a DMAR entry type we're not recognizing. While arranging
for it to be both recognized and respected, I noticed some further
potential for cleaning up.

1: VT-d: parse ACPI "SoC Integrated Address Translation Cache Reporting Structure"s
2: IOMMU: rename and re-type ats_enabled
3: VT-d: respect ACPI SATC's ATC_REQUIRED flag
4: VT-d: replace find_ats_dev_drhd()
5: VT-d: move ats_device() to the sole file it's used from
6: VT-d: move dev_invalidate_iotlb() to the sole file it's used from
7: VT-d: move {,un}map_vtd_domain_page()

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:55:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:55:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676090.1051922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzRS-0005zJ-FW; Mon, 05 Feb 2024 13:55:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676090.1051922; Mon, 05 Feb 2024 13:55:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzRS-0005zC-D0; Mon, 05 Feb 2024 13:55:22 +0000
Received: by outflank-mailman (input) for mailman id 676090;
 Mon, 05 Feb 2024 13:55:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzRQ-0005z2-IB
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:55:20 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 328309ed-c42e-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:55:19 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-40fdf2b69b2so431675e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:55:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s10-20020a05600c044a00b0040efc268713sm8570104wmb.26.2024.02.05.05.55.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:55:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 328309ed-c42e-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707141319; x=1707746119; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=xWCA/5RL2ARwAkMhUTOsGwspLyYaB9W2mN7GzM8E/nc=;
        b=VJAEkUsHmFp2LnhIGGi3MaJmXgKRWUeK98uD3HkkxOQkq2MWx8OGnPh7/tJt5nthNa
         EzZyJ2+WTXiIPMZorzEJOcN3iDgEqXmtb1wDrzsf4z3VKaGqIwVsYDCopffHV1H6J1YX
         6ShcJ6RylYSVTzlZRq83Fs4buDGu6Ln9ni7K+HrUDzuFQKhaaF3MaCSIWUtO23aIGVOU
         gRJvwuU/ekwQUAw9zlcW+IKQ7NQ1bk95uxcjw4gUYLpCwB69oYESJWH8MjVume3p5CEu
         T1lC9v1Yrv43UXPMhmzhgemtT2XEhoJUhntO3QR+bg5Qo9OpaGUHIToWLTydtJ7QHGL2
         FBfg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141319; x=1707746119;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=xWCA/5RL2ARwAkMhUTOsGwspLyYaB9W2mN7GzM8E/nc=;
        b=H4giu3hRoqONnTbzSI1q5G3lDQVuEPCi3Yze4cw0ljz13CSoDfiaYAtnCIrxs+oeYW
         vx3q9HKz8qWVUeVIfEUFO8F7bBmD0vu6CVP7H+2EI/MrI4nyPUWUCxFJpVMIsqbLhIrQ
         /2fXc8B8W4FGEM3QMql3sa4Uf//+e0/fMml3/YktdzWkz17pOTrZ/jzFfcYldCxZMZmh
         4liMdnFgGbzeq37pvwnHg7C3hZ9DB0DeycNfI6DSjiLszBpoCag/8hjofOT3tPDn2Uuu
         /bPKWzHsuEN+V23KhA1oTl/UU5t+MCpkHgdGZDzoBJlOh+Q8QCyxwGYlV4TXX9OCE2Ba
         KbRw==
X-Gm-Message-State: AOJu0YwIcZ14sMMg2YgXiddho3snEKpJOTCXB/8E9C8WZlab9d19ISos
	sJJDQ0sDZc07oCMw4bG49MwFZPpWG5fDOMtmVwdkI72+nl6wOe+UoU0Jez6DE/czrU1JmCcbUMs
	=
X-Google-Smtp-Source: AGHT+IE3Y6Nnj8BJo5oq8ZdSRhY8/K2PsXc/x8+2sF7Kx4VHPHvV8o+WE+HTnA9gj7y71d+JW8fk2A==
X-Received: by 2002:a05:600c:19cb:b0:40e:cca6:d82f with SMTP id u11-20020a05600c19cb00b0040ecca6d82fmr4310954wmq.16.1707141318842;
        Mon, 05 Feb 2024 05:55:18 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCV0KkxzR5NiOqJ00N1ujGxrmv45Diry5WnCzIm0nWt87sZTJAC3qQhvKya64f9AKgujYygQZbdLBMFC6KdviRsZgBZDAkIrvrHxs7RBBdCus6Yj
Message-ID: <b5a58dee-9a4c-4833-be59-b52c62f7137d@suse.com>
Date: Mon, 5 Feb 2024 14:55:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 1/7] VT-d: parse ACPI "SoC Integrated Address Translation
 Cache Reporting Structure"s
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

This is a prereq to us, in particular, respecting the "ATC required"
flag.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Should we check scope entries for appropriate types? (If so, then also
for e.g. ATSR.)

--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -47,6 +47,7 @@ LIST_HEAD_READ_MOSTLY(acpi_drhd_units);
 LIST_HEAD_READ_MOSTLY(acpi_rmrr_units);
 static LIST_HEAD_READ_MOSTLY(acpi_atsr_units);
 static LIST_HEAD_READ_MOSTLY(acpi_rhsa_units);
+static LIST_HEAD_READ_MOSTLY(acpi_satc_units);
 
 static struct acpi_table_header *__read_mostly dmar_table;
 static int __read_mostly dmar_flags;
@@ -764,6 +765,95 @@ acpi_parse_one_rhsa(struct acpi_dmar_hea
     return ret;
 }
 
+static int __init register_one_satc(struct acpi_satc_unit *satcu)
+{
+    bool ignore = false;
+    unsigned int i = 0;
+    int ret = 0;
+
+    /* Skip checking if segment is not accessible yet. */
+    if ( !pci_known_segment(satcu->segment) )
+        i = UINT_MAX;
+
+    for ( ; i < satcu->scope.devices_cnt; i++ )
+    {
+        uint8_t b = PCI_BUS(satcu->scope.devices[i]);
+        uint8_t d = PCI_SLOT(satcu->scope.devices[i]);
+        uint8_t f = PCI_FUNC(satcu->scope.devices[i]);
+
+        if ( pci_device_detect(satcu->segment, b, d, f) == 0 )
+        {
+            dprintk(XENLOG_WARNING VTDPREFIX,
+                    " Non-existent device (%pp) is reported in SATC scope!\n",
+                    &PCI_SBDF(satcu->segment, b, d, f));
+            ignore = true;
+        }
+        else
+        {
+            ignore = false;
+            break;
+        }
+    }
+
+    if ( ignore )
+    {
+        dprintk(XENLOG_WARNING VTDPREFIX,
+                " Ignore SATC for seg %04x as no device under its scope is PCI discoverable!\n",
+                satcu->segment);
+        scope_devices_free(&satcu->scope);
+        xfree(satcu);
+        return 1;
+    }
+
+    if ( iommu_verbose )
+        printk(VTDPREFIX " ATC required: %d\n", satcu->atc_required);
+
+    list_add(&satcu->list, &acpi_satc_units);
+
+    return ret;
+}
+
+static int __init
+acpi_parse_one_satc(const struct acpi_dmar_header *header)
+{
+    const struct acpi_dmar_satc *satc =
+        container_of(header, const struct acpi_dmar_satc, header);
+    struct acpi_satc_unit *satcu;
+    const void *dev_scope_start, *dev_scope_end;
+    int ret;
+
+    if ( (ret = acpi_dmar_check_length(header, sizeof(*satc))) != 0 )
+        return ret;
+
+    satcu = xzalloc(struct acpi_satc_unit);
+    if ( !satcu )
+        return -ENOMEM;
+
+    satcu->segment = satc->segment;
+    satcu->atc_required = satc->flags & 1;
+
+    dev_scope_start = (const void *)(satc + 1);
+    dev_scope_end   = (const void *)satc + header->length;
+    ret = acpi_parse_dev_scope(dev_scope_start, dev_scope_end,
+                               &satcu->scope, SATC_TYPE, satc->segment);
+
+    if ( !ret && satcu->scope.devices_cnt )
+    {
+        ret = register_one_satc(satcu);
+        /*
+         * register_one_satc() returns greater than 0 when a specified
+         * PCIe device cannot be detected. To prevent VT-d from being
+         * disabled in such cases, reset the return value to 0 here.
+         */
+        if ( ret > 0 )
+            ret = 0;
+    }
+    else
+        xfree(satcu);
+
+    return ret;
+}
+
 static int __init cf_check acpi_parse_dmar(struct acpi_table_header *table)
 {
     struct acpi_table_dmar *dmar;
@@ -817,6 +907,11 @@ static int __init cf_check acpi_parse_dm
                 printk(VTDPREFIX "found ACPI_DMAR_RHSA:\n");
             ret = acpi_parse_one_rhsa(entry_header);
             break;
+        case ACPI_DMAR_TYPE_SATC:
+            if ( iommu_verbose )
+                printk(VTDPREFIX "found ACPI_DMAR_SATC:\n");
+            ret = acpi_parse_one_satc(entry_header);
+            break;
         default:
             dprintk(XENLOG_WARNING VTDPREFIX,
                     "Ignore unknown DMAR structure type (%#x)\n",
--- a/xen/drivers/passthrough/vtd/dmar.h
+++ b/xen/drivers/passthrough/vtd/dmar.h
@@ -91,6 +91,13 @@ struct acpi_rhsa_unit {
     u32    proximity_domain;
 };
 
+struct acpi_satc_unit {
+    struct dmar_scope scope;
+    struct list_head list;
+    uint16_t segment;
+    bool atc_required:1;
+};
+
 #define for_each_drhd_unit(drhd) \
     list_for_each_entry(drhd, &acpi_drhd_units, list)
 
@@ -106,6 +113,7 @@ struct acpi_atsr_unit *acpi_find_matched
 #define DMAR_TYPE 1
 #define RMRR_TYPE 2
 #define ATSR_TYPE 3
+#define SATC_TYPE 4
 
 #define DMAR_OPERATION_TIMEOUT MILLISECS(1000)
 
--- a/xen/include/acpi/actbl2.h
+++ b/xen/include/acpi/actbl2.h
@@ -345,7 +345,8 @@ enum acpi_dmar_type {
 	ACPI_DMAR_TYPE_RESERVED_MEMORY = 1,
 	ACPI_DMAR_TYPE_ATSR = 2,
 	ACPI_DMAR_HARDWARE_AFFINITY = 3,
-	ACPI_DMAR_TYPE_RESERVED = 4	/* 4 and greater are reserved */
+	ACPI_DMAR_TYPE_SATC = 5,
+	ACPI_DMAR_TYPE_RESERVED = 7	/* 7 and greater are reserved */
 };
 
 /* DMAR Device Scope structure */
@@ -427,6 +428,15 @@ struct acpi_dmar_rhsa {
 	u32 proximity_domain;
 };
 
+/* 5: SOC Integrated Address Translation Cache Reporting Structure */
+
+struct acpi_dmar_satc {
+	struct acpi_dmar_header header;
+	uint8_t flags;
+	uint8_t reserved;
+	uint16_t segment;
+};
+
 /*******************************************************************************
  *
  * HPET - High Precision Event Timer table



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:55:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:55:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676092.1051932 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzRr-0006Tk-Nq; Mon, 05 Feb 2024 13:55:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676092.1051932; Mon, 05 Feb 2024 13:55:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzRr-0006Td-L9; Mon, 05 Feb 2024 13:55:47 +0000
Received: by outflank-mailman (input) for mailman id 676092;
 Mon, 05 Feb 2024 13:55:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzRq-0006TN-A8
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:55:46 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 414c2133-c42e-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 14:55:44 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-40fdc63f4feso6434175e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:55:44 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s10-20020a05600c044a00b0040efc268713sm8570104wmb.26.2024.02.05.05.55.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:55:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 414c2133-c42e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707141343; x=1707746143; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=fiaRUGgpUjwp5AtuY8peqqWBSdls0Fsu1GdGBsfM0J0=;
        b=eVJT46VGTA5zY0j+N+7lUTSfJEJF5N+ILVOdLtfG798NGdusdZoAy8oyMSKpX6mX3V
         /U4gXWgR5y6DMljAdaJ6Az1ok8sIyIyPnc3MakSYaZkGbUHybmKFxXcFdfGTJb1KmfyD
         DUn6t/QYzGt55h8d1fDxFdBwyV+AZ94ZKwgjQ/7htqvvgCwNOHde3SMUzflJD69BUbHd
         rLB50gEXvr317eUGb6LXLPubG4aoTsE/2lcrdhqOF2eTta6SUG+27PAs1qbHgLGvIeoz
         pANfG3QrmVgPCVAIPNR88ydgTMfG+/w9ayV+LCiEMZLV7vgEYYw4pikNYdCME8Xm2660
         L5Tg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141343; x=1707746143;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=fiaRUGgpUjwp5AtuY8peqqWBSdls0Fsu1GdGBsfM0J0=;
        b=qKvG+U+a6fYqLioPudPRDTq5D8d5XyInf6fFHhkWcTpofYKoE12TpRxJ9ammanz+Z8
         Si7h8LRjF5LzZBh3EgfMTeYzpFCw6YwWzJNMFWwRQ5W1K1T5WPvE8oEJ41Y7JlIqDLHF
         cyoOTZMTYpewrNVPqTH+wnPCIOgafBwg0PL54HZmu19cv8dT08WsmmM26qMZz0kLPTF0
         ElfeMsO2dk2VIqv9fH3gifIL+Shk5uRn2aiwnueQ/dITtzH4Jj3e43FM2pYlsEQM98Hc
         XzxAE89yR1vZGQT7FXiI6wKzZn7Kj3jA53mSTnlv2vrIsZJ2IBP88/zCTNLm7eFwlBeS
         i2fQ==
X-Gm-Message-State: AOJu0YxFiEiaf9fIiljPkqntHCVQ5dXnn3qX2cC7a5SQVIrOnDTaq+A0
	IOMJbtHDO2PF94RlhZUkJghA/mJfxUIW8EPY36/iOSdJWKUWjyVK109sjXk1ZKFNxifBluxlMFo
	=
X-Google-Smtp-Source: AGHT+IEORSTwgdGzj2v6A6YK92N93MbcF4KbXz8hjo3fgBIdt/R1+qwm6YYcycVcpUYAr3uRdJSWWw==
X-Received: by 2002:a05:600c:468a:b0:40f:ddfb:8c63 with SMTP id p10-20020a05600c468a00b0040fddfb8c63mr925331wmo.14.1707141343544;
        Mon, 05 Feb 2024 05:55:43 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWNBHAB6Fq63DmwP5Jnjt2xj3vsjpWCfO13NCh26JGNny+nQbAl/7wbKI2lKWrAo7ls4iR5fWVA/Aushpl52rZqsqqvB7mshRL8KOonoXrYWPQwDDOu17Oy8/aLXiFzSA6m7dtUM5sdbO5rLsRk
Message-ID: <467d24e1-8ed3-4dda-a334-70ff747bf94b@suse.com>
Date: Mon, 5 Feb 2024 14:55:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 2/7] IOMMU: rename and re-type ats_enabled
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Make the variable a tristate, with (as done elsewhere) a negative value
meaning "default". Since all use sites need looking at, also rename it
to match our usual "opt_*" pattern. While touching it, also move it to
.data.ro_after_init.

The only place it retains boolean nature is pci_ats_device(), for now.

In AMD code re-order conditionals to have the config space accesses
after (cheaper) flag checks.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
In domain_context_mapping_one() I'm a little puzzled that translation
type is selected based on only IOMMU and global properties, i.e. not
taking the device itself into account.

--- a/xen/drivers/passthrough/amd/iommu_cmd.c
+++ b/xen/drivers/passthrough/amd/iommu_cmd.c
@@ -282,7 +282,7 @@ void amd_iommu_flush_iotlb(u8 devfn, con
     struct amd_iommu *iommu;
     unsigned int req_id, queueid, maxpend;
 
-    if ( !ats_enabled )
+    if ( opt_ats <= 0 )
         return;
 
     if ( !pci_ats_enabled(pdev->seg, pdev->bus, pdev->devfn) )
@@ -340,7 +340,7 @@ static void _amd_iommu_flush_pages(struc
         flush_command_buffer(iommu, 0);
     }
 
-    if ( ats_enabled )
+    if ( opt_ats > 0 )
     {
         amd_iommu_flush_all_iotlbs(d, daddr, order);
 
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -185,10 +185,11 @@ static int __must_check amd_iommu_setup_
         dte->ex = ivrs_dev->dte_allow_exclusion;
         dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
 
-        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
+        if ( opt_ats > 0 &&
              !ivrs_dev->block_ats &&
-             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
-            dte->i = ats_enabled;
+             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
+             pci_ats_device(iommu->seg, bus, pdev->devfn) )
+            dte->i = true;
 
         spin_unlock_irqrestore(&iommu->lock, flags);
 
@@ -248,10 +249,11 @@ static int __must_check amd_iommu_setup_
         ASSERT(dte->sys_mgt == MASK_EXTR(ivrs_dev->device_flags,
                                          ACPI_IVHD_SYSTEM_MGMT));
 
-        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
+        if ( opt_ats > 0 &&
              !ivrs_dev->block_ats &&
-             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
-            ASSERT(dte->i == ats_enabled);
+             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
+             pci_ats_device(iommu->seg, bus, pdev->devfn) )
+            ASSERT(dte->i);
 
         spin_unlock_irqrestore(&iommu->lock, flags);
 
@@ -268,9 +270,10 @@ static int __must_check amd_iommu_setup_
 
     ASSERT(pcidevs_locked());
 
-    if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
+    if ( opt_ats > 0 &&
          !ivrs_dev->block_ats &&
          iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
+         pci_ats_device(iommu->seg, bus, pdev->devfn) &&
          !pci_ats_enabled(iommu->seg, bus, pdev->devfn) )
     {
         if ( devfn == pdev->devfn )
--- a/xen/drivers/passthrough/ats.c
+++ b/xen/drivers/passthrough/ats.c
@@ -18,8 +18,8 @@
 #include <xen/pci_regs.h>
 #include "ats.h"
 
-bool __read_mostly ats_enabled;
-boolean_param("ats", ats_enabled);
+int8_t __ro_after_init opt_ats = -1;
+boolean_param("ats", opt_ats);
 
 int enable_ats_device(struct pci_dev *pdev, struct list_head *ats_list)
 {
--- a/xen/drivers/passthrough/ats.h
+++ b/xen/drivers/passthrough/ats.h
@@ -22,7 +22,7 @@
 #define ATS_QUEUE_DEPTH_MASK     0x1f
 #define ATS_ENABLE               (1<<15)
 
-extern bool ats_enabled;
+extern int8_t opt_ats;
 
 int enable_ats_device(struct pci_dev *pdev, struct list_head *ats_list);
 void disable_ats_device(struct pci_dev *pdev);
@@ -43,7 +43,7 @@ static inline int pci_ats_enabled(int se
 
 static inline int pci_ats_device(int seg, int bus, int devfn)
 {
-    if ( !ats_enabled )
+    if ( !opt_ats )
         return 0;
 
     return pci_find_ext_capability(PCI_SBDF(seg, bus, devfn),
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1543,7 +1543,7 @@ int domain_context_mapping_one(
         }
 
         context_set_address_root(lctxt, root);
-        if ( ats_enabled && ecap_dev_iotlb(iommu->ecap) )
+        if ( opt_ats > 0 && ecap_dev_iotlb(iommu->ecap) )
             context_set_translation_type(lctxt, CONTEXT_TT_DEV_IOTLB);
         else
             context_set_translation_type(lctxt, CONTEXT_TT_MULTI_LEVEL);
--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ b/xen/drivers/passthrough/vtd/x86/ats.c
@@ -46,7 +46,7 @@ int ats_device(const struct pci_dev *pde
     struct acpi_drhd_unit *ats_drhd;
     int pos;
 
-    if ( !ats_enabled || !iommu_qinval )
+    if ( opt_ats <= 0 || !iommu_qinval )
         return 0;
 
     if ( !ecap_queued_inval(drhd->iommu->ecap) ||



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:56:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:56:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676094.1051943 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzSL-0006yr-23; Mon, 05 Feb 2024 13:56:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676094.1051943; Mon, 05 Feb 2024 13:56:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzSK-0006yk-UX; Mon, 05 Feb 2024 13:56:16 +0000
Received: by outflank-mailman (input) for mailman id 676094;
 Mon, 05 Feb 2024 13:56:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzSK-0005z2-EW
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:56:16 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 53ff4ee0-c42e-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:56:15 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-40fdc63f4feso6439345e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:56:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s10-20020a05600c044a00b0040efc268713sm8570104wmb.26.2024.02.05.05.56.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:56:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 53ff4ee0-c42e-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707141375; x=1707746175; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=f3H4Ur0/gJhZeUF8ayz9NPkqLMEexvZyofs8rqJKkAY=;
        b=bnk6Ne5TkNLQlBI/EGNGIGMrB5XCMSXlaXx22raY7a4IrIFUVjhLevhHDdQycULtPk
         IrSRqXifhqn+NPcoeRWCzIMLXiV92mu5XmW44eeAPv4EbMKDQ1939kvRzIzR5NwPOsC/
         L4p58qwUOHRzCLbf8ltHvvUfC6PMRavRVGFsWz6O23LtjarWSlZ2Vx5NRWKbY7zVea0V
         RvUMbD6Oj4z9lzpIISwB4RM+dbKwYgBQdZh9RDa3YdyWpK2NtZHcQqIAtriMVQL1AcRA
         j3sc4h7pURDYTtoPiPSnsDMP67zGZb11bgvPnexBhMWU5kA9BpJHy6G7RrG01wjP0Gtk
         cmOA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141375; x=1707746175;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=f3H4Ur0/gJhZeUF8ayz9NPkqLMEexvZyofs8rqJKkAY=;
        b=lMyYoMg/mEs4zKlpgb/ZzjN2+7tNhH6gpNoK++2MdrUuIs2EKM+7nchyIBfbfDlOk8
         kllICMOzINyzCwCoCmnxF3q8Q/t3HjYQanOE7H0Z4jn5NOisVMHg7ZVYZ2bItKHV1qRT
         02mhP9zbNZ1pSL7JkE+Bx5ZqUHiIPB8ggkRLBnpzkqjV0Yakbt6uH11ei5Tt7PSX/qfQ
         FPK1v6uYoHGKxfj5At8uL4fg1Sg4zFz1jPo5qLZpGgG6gnZVSSRQy0YOnqRXqo40PlSA
         w8XYl1UhFuwJgnnnSgCJq6JWOECNMNHUHZxLS+s9/5qfhH0787NAcmpls8A3wNpYZhzM
         ajjg==
X-Gm-Message-State: AOJu0YwMniZxxgNRlaWLZUQsTvCHHdCSK1s2eo73a7xd9N0rK2PHOS6O
	7VkYKlY5rxgAGbkrNofPqXCeY6yJfUwIRvP6PoIItPi5bYkkFXI2iEGF9NQwYwfu80q904B93Wo
	=
X-Google-Smtp-Source: AGHT+IHKs0cihTxNqeV8nuhiv1fHgALzE+NMfvV0hATXeqzhpcrCBAjxfZRCmqp2eFQgxjF23ijOOg==
X-Received: by 2002:a05:600c:3583:b0:40e:bfbf:f368 with SMTP id p3-20020a05600c358300b0040ebfbff368mr5255342wmq.2.1707141375004;
        Mon, 05 Feb 2024 05:56:15 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUdLl+7bK14Slxd41prmGWLqlG/OLqz6/Vf8MXC/p7KCgrx7cXUy6+eZgWOqDWZdfbp3o7Qsc8XLekPoLOzpEICNsAlr8BUdGVvywCVVBMC9Ynf
Message-ID: <72496fb7-8835-4d45-bd92-7e7d612861c7@suse.com>
Date: Mon, 5 Feb 2024 14:56:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 3/7] VT-d: respect ACPI SATC's ATC_REQUIRED flag
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

When the flag is set, permit Dom0 to control the device (no worse than
what we had before and in line with other "best effort" behavior we use
when it comes to Dom0), but suppress passing through to DomU-s unless
ATS can actually be enabled for such devices.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Is ats_device() using acpi_find_matched_atsr_unit() unconditionally
actually correct? Shouldn't that check be skipped for root complex
integrated devices?

--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -225,7 +225,10 @@ exceptions (watchdog NMIs and unexpected
 > Default: `false`
 
 Permits Xen to set up and use PCI Address Translation Services.  This is a
-performance optimisation for PCI Passthrough.
+performance optimisation for PCI Passthrough.  Note that firmware may indicate
+that certain devices need to have ATS enabled for proper operation. For such
+devices ATS will be enabled by default, unless the option is used in its
+negative form.
 
 **WARNING: Xen cannot currently safely use ATS because of its synchronous wait
 loops for Queued Invalidation completions.**
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -254,6 +254,24 @@ struct acpi_atsr_unit *acpi_find_matched
     return all_ports;
 }
 
+const struct acpi_satc_unit *acpi_find_matched_satc_unit(
+    const struct pci_dev *pdev)
+{
+    const struct acpi_satc_unit *satc;
+
+    list_for_each_entry ( satc, &acpi_satc_units, list )
+    {
+        if ( satc->segment != pdev->seg )
+            continue;
+
+        for ( unsigned int i = 0; i < satc->scope.devices_cnt; ++i )
+            if ( satc->scope.devices[i] == pdev->sbdf.bdf )
+                return satc;
+    }
+
+    return NULL;
+}
+
 struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd)
 {
     struct acpi_rhsa_unit *rhsa;
--- a/xen/drivers/passthrough/vtd/dmar.h
+++ b/xen/drivers/passthrough/vtd/dmar.h
@@ -109,6 +109,8 @@ struct acpi_satc_unit {
 
 struct acpi_drhd_unit *acpi_find_matched_drhd_unit(const struct pci_dev *);
 struct acpi_atsr_unit *acpi_find_matched_atsr_unit(const struct pci_dev *);
+const struct acpi_satc_unit *acpi_find_matched_satc_unit(
+    const struct pci_dev *pdev);
 
 #define DMAR_TYPE 1
 #define RMRR_TYPE 2
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -2364,6 +2364,25 @@ static int cf_check intel_iommu_add_devi
     if ( ret )
         dprintk(XENLOG_ERR VTDPREFIX, "%pd: context mapping failed\n",
                 pdev->domain);
+    else if ( !pdev->broken )
+    {
+        const struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
+        const struct acpi_satc_unit *satc = acpi_find_matched_satc_unit(pdev);
+
+        /*
+         * Prevent the device from getting assigned to an unprivileged domain
+         * when firmware indicates ATS is required, but ATS could not be enabled
+         * (e.g. because of being suppressed via command line option).
+         */
+        if ( satc && satc->atc_required &&
+             (!drhd || ats_device(pdev, drhd) <= 0 ||
+              !pci_ats_enabled(pdev->seg, pdev->bus, pdev->devfn)) )
+        {
+            printk(XENLOG_WARNING "ATS: %pp is not eligible for pass-through\n",
+                   &pdev->sbdf);
+            pdev->broken = true;
+        }
+    }
 
     return ret;
 }
@@ -2375,12 +2394,27 @@ static int cf_check intel_iommu_enable_d
 
     pci_vtd_quirk(pdev);
 
-    if ( ret <= 0 )
-        return ret;
+    if ( ret <= 0 ||
+         (ret = enable_ats_device(pdev, &drhd->iommu->ats_devices)) < 0 )
+    {
+        const struct acpi_satc_unit *satc = acpi_find_matched_satc_unit(pdev);
+
+        /*
+         * Prevent the device from getting assigned to an unprivileged domain
+         * when firmware indicates ATS is required, but ATS use was disabled
+         * via command line option.
+         */
+        if ( satc && satc->atc_required && !pdev->broken )
+        {
+            printk(XENLOG_WARNING "ATS: %pp is not eligible for pass-through\n",
+                   &pdev->sbdf);
+            pdev->broken = true;
+        }
 
-    ret = enable_ats_device(pdev, &drhd->iommu->ats_devices);
+        return ret;
+    }
 
-    return ret >= 0 ? 0 : ret;
+    return 0;
 }
 
 static int cf_check intel_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ b/xen/drivers/passthrough/vtd/x86/ats.c
@@ -44,9 +44,10 @@ struct acpi_drhd_unit *find_ats_dev_drhd
 int ats_device(const struct pci_dev *pdev, const struct acpi_drhd_unit *drhd)
 {
     struct acpi_drhd_unit *ats_drhd;
+    const struct acpi_satc_unit *satc;
     int pos;
 
-    if ( opt_ats <= 0 || !iommu_qinval )
+    if ( !opt_ats || !iommu_qinval )
         return 0;
 
     if ( !ecap_queued_inval(drhd->iommu->ecap) ||
@@ -56,6 +57,10 @@ int ats_device(const struct pci_dev *pde
     if ( !acpi_find_matched_atsr_unit(pdev) )
         return 0;
 
+    satc = acpi_find_matched_satc_unit(pdev);
+    if ( opt_ats < 0 && (!satc || !satc->atc_required) )
+        return 0;
+
     ats_drhd = find_ats_dev_drhd(drhd->iommu);
     pos = pci_find_ext_capability(pdev->sbdf, PCI_EXT_CAP_ID_ATS);
 



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:56:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:56:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676099.1051953 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzSh-0007VJ-D7; Mon, 05 Feb 2024 13:56:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676099.1051953; Mon, 05 Feb 2024 13:56:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzSh-0007VC-9X; Mon, 05 Feb 2024 13:56:39 +0000
Received: by outflank-mailman (input) for mailman id 676099;
 Mon, 05 Feb 2024 13:56:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzSf-0005z2-Nh
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:56:37 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 60d851ad-c42e-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:56:36 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-40fc52c2ae4so28919015e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:56:37 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s10-20020a05600c044a00b0040efc268713sm8570104wmb.26.2024.02.05.05.56.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:56:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60d851ad-c42e-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707141396; x=1707746196; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=K+PEfDQdWqrjmqD3hRobqZrRVXYu7/7wPkddLoGAyb8=;
        b=fcJP2FSG31i8dW7Rnoa/DlRHcEoJN08gElPzTROw2hJrtMySjPN5hbs468JHaLPqPD
         g981uNy4BNpxVhJIF18YypccCGnh7JUCoADBSpPe8zW3Sorf1RFWdi1/AUAga+iWXZeT
         riWmlTtqy6Jb5rKBq8IVe2KqOgqA4oSoE15uRd5OUT+713b5OWeb0dxFGxTcSGoLKKR3
         NgNwI8BXqrJa/s/PN/Sg7rZMrkpXZss25PHaE73R7Jgq4iYdhezN0WhQZpWzPq31zpc8
         BVvwWKTn8rXsbZcRVLg6xFbcxROD4BcYgWHeVV5czJ2bBR0clVbQDXiv61fpXE6zpEwD
         4B1Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141396; x=1707746196;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=K+PEfDQdWqrjmqD3hRobqZrRVXYu7/7wPkddLoGAyb8=;
        b=A67wGLCaff8jTd0PAQIJYdqdnyGq1EmhxlrNF4nn7jwNJABoYa02ubrVHOIbtf3Fyq
         IrvE2JZPPFnDwAglkss4ENhCDDPbF8FVWT86Ykap0ws7CVbjBZqOZ3XVqd5+c4OccYfW
         6+QyQ9fPbMYoYT25V2zYtyo0yIvXPF/6Bc+sTpXFWha/WbaEa8YjR3P2BK70kD3WAsLL
         /nro8W7GMbWxLhrzfjx05re6KPQYqunCfgnP3gEJwXAmaawPFXRl5SK4uCFoUAxAYH8K
         QxPfKMAV6gCGK24gLM9Ze05LpnXKknkRTLdsfz7hD6mpdFoDbaf1t4/1HNcZoPPSV1ic
         v3SQ==
X-Gm-Message-State: AOJu0YwbxhhS4x8S9QyM7EloQ7pp+p6n6Kzqf4ORgg6a8KiMlt5vtpHt
	JctAUtn/5jJqtf1zifcsFRG3cAG/RXS2026s2kXdIuFc3ZQcVd5KArcgRbzC2TQf26culYoeIOw
	=
X-Google-Smtp-Source: AGHT+IH0gtD1EaC9rv3MSpVyGG/VHdKnT9uqSW0cm30+gNZ5Bnx2NDrq09wKTc9jpodHulyT0soWnQ==
X-Received: by 2002:a05:600c:3c83:b0:40f:b166:7688 with SMTP id bg3-20020a05600c3c8300b0040fb1667688mr4324676wmb.15.1707141396500;
        Mon, 05 Feb 2024 05:56:36 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXdFQiJue1hYUdiMogxydIS3naOKWQvRVsy3kUPh2cbI2Ok3y7t3TmVvt3gB8HU3M91kLRaeOHlHsUK9hhJe8cT8ICVYOIEeTyAG/FED4ILJFLi
Message-ID: <a75bef32-8bbb-4471-b26b-981b06173cd5@suse.com>
Date: Mon, 5 Feb 2024 14:56:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 4/7] VT-d: replace find_ats_dev_drhd()
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

All callers only care about boolean outcome. For this there's no point
in allocating a duplicate of the respective DRHD structure; a simple
boolean suffices (which eventually may wantg to become a count, such
that the "any ATS devices assigned state" can also clear again). With
that boolean, remove respective parameters from internal helper
functions right away, as those have access to the flag through another
parameter.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -65,8 +65,6 @@ struct acpi_drhd_unit *ioapic_to_drhd(un
 struct acpi_drhd_unit *hpet_to_drhd(unsigned int hpet_id);
 struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd);
 
-struct acpi_drhd_unit *find_ats_dev_drhd(struct vtd_iommu *iommu);
-
 int ats_device(const struct pci_dev *, const struct acpi_drhd_unit *);
 
 int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -624,8 +624,7 @@ int cf_check vtd_flush_iotlb_reg(
 }
 
 static int __must_check iommu_flush_iotlb_global(struct vtd_iommu *iommu,
-                                                 bool flush_non_present_entry,
-                                                 bool flush_dev_iotlb)
+                                                 bool flush_non_present_entry)
 {
     int status;
 
@@ -633,7 +632,7 @@ static int __must_check iommu_flush_iotl
     vtd_ops_preamble_quirk(iommu);
 
     status = iommu->flush.iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH,
-                                flush_non_present_entry, flush_dev_iotlb);
+                                flush_non_present_entry, iommu->flush_dev_iotlb);
 
     /* undo platform specific errata workarounds */
     vtd_ops_postamble_quirk(iommu);
@@ -642,8 +641,7 @@ static int __must_check iommu_flush_iotl
 }
 
 static int __must_check iommu_flush_iotlb_dsi(struct vtd_iommu *iommu, u16 did,
-                                              bool flush_non_present_entry,
-                                              bool flush_dev_iotlb)
+                                              bool flush_non_present_entry)
 {
     int status;
 
@@ -651,7 +649,7 @@ static int __must_check iommu_flush_iotl
     vtd_ops_preamble_quirk(iommu);
 
     status = iommu->flush.iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH,
-                                flush_non_present_entry, flush_dev_iotlb);
+                                flush_non_present_entry, iommu->flush_dev_iotlb);
 
     /* undo platform specific errata workarounds */
     vtd_ops_postamble_quirk(iommu);
@@ -661,26 +659,23 @@ static int __must_check iommu_flush_iotl
 
 static int __must_check iommu_flush_iotlb_psi(struct vtd_iommu *iommu, u16 did,
                                               u64 addr, unsigned int order,
-                                              bool flush_non_present_entry,
-                                              bool flush_dev_iotlb)
+                                              bool flush_non_present_entry)
 {
     int status;
 
     /* Fallback to domain selective flush if no PSI support */
     if ( !cap_pgsel_inv(iommu->cap) )
-        return iommu_flush_iotlb_dsi(iommu, did, flush_non_present_entry,
-                                     flush_dev_iotlb);
+        return iommu_flush_iotlb_dsi(iommu, did, flush_non_present_entry);
 
     /* Fallback to domain selective flush if size is too big */
     if ( order > cap_max_amask_val(iommu->cap) )
-        return iommu_flush_iotlb_dsi(iommu, did, flush_non_present_entry,
-                                     flush_dev_iotlb);
+        return iommu_flush_iotlb_dsi(iommu, did, flush_non_present_entry);
 
     /* apply platform specific errata workarounds */
     vtd_ops_preamble_quirk(iommu);
 
     status = iommu->flush.iotlb(iommu, did, addr, order, DMA_TLB_PSI_FLUSH,
-                                flush_non_present_entry, flush_dev_iotlb);
+                                flush_non_present_entry, iommu->flush_dev_iotlb);
 
     /* undo platform specific errata workarounds */
     vtd_ops_postamble_quirk(iommu);
@@ -692,7 +687,6 @@ static int __must_check iommu_flush_all(
 {
     struct acpi_drhd_unit *drhd;
     struct vtd_iommu *iommu;
-    bool flush_dev_iotlb;
     int rc = 0;
 
     flush_local(FLUSH_CACHE);
@@ -703,8 +697,7 @@ static int __must_check iommu_flush_all(
 
         iommu = drhd->iommu;
         context_rc = iommu_flush_context_global(iommu, 0);
-        flush_dev_iotlb = !!find_ats_dev_drhd(iommu);
-        iotlb_rc = iommu_flush_iotlb_global(iommu, 0, flush_dev_iotlb);
+        iotlb_rc = iommu_flush_iotlb_global(iommu, 0);
 
         /*
          * The current logic for returns:
@@ -734,7 +727,6 @@ static int __must_check cf_check iommu_f
     struct domain_iommu *hd = dom_iommu(d);
     struct acpi_drhd_unit *drhd;
     struct vtd_iommu *iommu;
-    bool flush_dev_iotlb;
     int iommu_domid;
     int ret = 0;
 
@@ -762,21 +754,18 @@ static int __must_check cf_check iommu_f
         if ( !test_bit(iommu->index, hd->arch.vtd.iommu_bitmap) )
             continue;
 
-        flush_dev_iotlb = !!find_ats_dev_drhd(iommu);
         iommu_domid = get_iommu_did(d->domain_id, iommu, !d->is_dying);
         if ( iommu_domid == -1 )
             continue;
 
         if ( !page_count || (page_count & (page_count - 1)) ||
              dfn_eq(dfn, INVALID_DFN) || !IS_ALIGNED(dfn_x(dfn), page_count) )
-            rc = iommu_flush_iotlb_dsi(iommu, iommu_domid,
-                                       0, flush_dev_iotlb);
+            rc = iommu_flush_iotlb_dsi(iommu, iommu_domid, 0);
         else
             rc = iommu_flush_iotlb_psi(iommu, iommu_domid,
                                        dfn_to_daddr(dfn),
                                        get_order_from_pages(page_count),
-                                       !(flush_flags & IOMMU_FLUSHF_modified),
-                                       flush_dev_iotlb);
+                                       !(flush_flags & IOMMU_FLUSHF_modified));
 
         if ( rc > 0 )
             iommu_flush_write_buffer(iommu);
@@ -1488,7 +1477,6 @@ int domain_context_mapping_one(
     uint16_t seg = iommu->drhd->segment, prev_did = 0;
     struct domain *prev_dom = NULL;
     int rc, ret;
-    bool flush_dev_iotlb;
 
     if ( QUARANTINE_SKIP(domain, pgd_maddr) )
         return 0;
@@ -1637,8 +1625,7 @@ int domain_context_mapping_one(
 
     rc = iommu_flush_context_device(iommu, prev_did, PCI_BDF(bus, devfn),
                                     DMA_CCMD_MASK_NOBIT, !prev_dom);
-    flush_dev_iotlb = !!find_ats_dev_drhd(iommu);
-    ret = iommu_flush_iotlb_dsi(iommu, prev_did, !prev_dom, flush_dev_iotlb);
+    ret = iommu_flush_iotlb_dsi(iommu, prev_did, !prev_dom);
 
     /*
      * The current logic for returns:
@@ -1882,7 +1869,6 @@ int domain_context_unmap_one(
     struct context_entry *context, *context_entries;
     u64 maddr;
     int iommu_domid, rc, ret;
-    bool flush_dev_iotlb;
 
     ASSERT(pcidevs_locked());
     spin_lock(&iommu->lock);
@@ -1908,8 +1894,7 @@ int domain_context_unmap_one(
                                     PCI_BDF(bus, devfn),
                                     DMA_CCMD_MASK_NOBIT, 0);
 
-    flush_dev_iotlb = !!find_ats_dev_drhd(iommu);
-    ret = iommu_flush_iotlb_dsi(iommu, iommu_domid, 0, flush_dev_iotlb);
+    ret = iommu_flush_iotlb_dsi(iommu, iommu_domid, 0);
 
     /*
      * The current logic for returns:
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -484,6 +484,7 @@ struct vtd_iommu {
     spinlock_t register_lock; /* protect iommu register handling */
     u64 root_maddr; /* root entry machine address */
     nodeid_t node;
+    bool flush_dev_iotlb;
     struct msi_desc msi;
     struct acpi_drhd_unit *drhd;
 
--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ b/xen/drivers/passthrough/vtd/x86/ats.c
@@ -28,22 +28,8 @@
 #include "../extern.h"
 #include "../../ats.h"
 
-static LIST_HEAD(ats_dev_drhd_units);
-
-struct acpi_drhd_unit *find_ats_dev_drhd(struct vtd_iommu *iommu)
-{
-    struct acpi_drhd_unit *drhd;
-    list_for_each_entry ( drhd, &ats_dev_drhd_units, list )
-    {
-        if ( drhd->iommu == iommu )
-            return drhd;
-    }
-    return NULL;
-}
-
 int ats_device(const struct pci_dev *pdev, const struct acpi_drhd_unit *drhd)
 {
-    struct acpi_drhd_unit *ats_drhd;
     const struct acpi_satc_unit *satc;
     int pos;
 
@@ -61,17 +47,10 @@ int ats_device(const struct pci_dev *pde
     if ( opt_ats < 0 && (!satc || !satc->atc_required) )
         return 0;
 
-    ats_drhd = find_ats_dev_drhd(drhd->iommu);
     pos = pci_find_ext_capability(pdev->sbdf, PCI_EXT_CAP_ID_ATS);
+    if ( pos )
+        drhd->iommu->flush_dev_iotlb = true;
 
-    if ( pos && (ats_drhd == NULL) )
-    {
-        ats_drhd = xmalloc(struct acpi_drhd_unit);
-        if ( !ats_drhd )
-            return -ENOMEM;
-        *ats_drhd = *drhd;
-        list_add_tail(&ats_drhd->list, &ats_dev_drhd_units);
-    }
     return pos;
 }
 



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:57:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:57:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676101.1051963 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzT1-0007zl-Kv; Mon, 05 Feb 2024 13:56:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676101.1051963; Mon, 05 Feb 2024 13:56:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzT1-0007ze-I0; Mon, 05 Feb 2024 13:56:59 +0000
Received: by outflank-mailman (input) for mailman id 676101;
 Mon, 05 Feb 2024 13:56:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzT0-0007Qa-8F
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:56:58 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6c801d7b-c42e-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 14:56:56 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-40fb3b5893eso36544255e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:56:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s10-20020a05600c044a00b0040efc268713sm8570104wmb.26.2024.02.05.05.56.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:56:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c801d7b-c42e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707141416; x=1707746216; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=xeo9DHqLhyAKD6WkqcE7ZsPzOC6r3GO9FyNnjBbVjWg=;
        b=U0imfmc5NLC1819iyKXKi/sCwqF0YaJNGnuIbbCeE+srnjJuV5lv6Pekbiu4YIWowJ
         Z11HT6JwI9KMkM2M57r60g6z7+Uc+PVEQGu1KdPj2s8EAhCYxR5siK5jdCvt/3L/xBIy
         2bOknoJLQ4bIjtBhVnAyFKIq9+bv6ybiwCQuwyt0rRsNXkpdHNgl08wD4hWJyyGHFnWC
         8oPSuafUKrBM6jaNng9iVjZf9U5ht3EmbyqelIHZSYrQpSdqWrU50WurKFB9a2A3qJbi
         WKO5lYShbETp23lMHyVvCmEHGcMHla7CLVyx5q/ttmCLmSkFRQWJIpqHL0DoMPJSnF1T
         vS+w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141416; x=1707746216;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=xeo9DHqLhyAKD6WkqcE7ZsPzOC6r3GO9FyNnjBbVjWg=;
        b=kVNvKgQhmJsYrHqKi5TZdnUeUowYN2eQGrFCFE/HQrsjlhyYMFhFqGznLLUOzcRv+/
         8ROtuSpNsl+O0O/9QNFoJvGrWzhPh/i5nYUFiDL5Wdk3CTiXGsHwrX+owb0+Ki3ashBr
         XFD5/9tc5YxkEZPLO+lo0ON94XNO7cK7Nn5VATxssHDaKIHBAinVCXoJ+HAgZ2jq5N6Q
         /QhuGqnTnWINftVUNIxL6/ViO3kYqAB+tRGwxsBn59MfmiAdS0Ep2/IclaKVbp04t7GD
         vriVboUBnwA6CnHUmxvDoqnMrrQRgR4wsNk8Yyrr+Tc1GEWbBhCdMjp3V6F7LL2++GXa
         x2zA==
X-Gm-Message-State: AOJu0YwMUmBZb2YaPKh1SLIqeLIbTY+lZtILPEUhkoQwrCLjlZv82SaN
	Dng7fouzKED0DllACdTM8uxKMftXp8/hV9ff2E5HuxoblFC+wWy6eWLB62O47gwBB6E1aEZ/0MQ
	=
X-Google-Smtp-Source: AGHT+IFh4g3nSQAwzF4QQsuBLBhkW4JIZ4T2NTiGOlKNdj81gjIVBNg1B7egKsB3lrK01voNNtbn3A==
X-Received: by 2002:a05:600c:548f:b0:40f:ba0b:16b3 with SMTP id iv15-20020a05600c548f00b0040fba0b16b3mr5246317wmb.2.1707141415954;
        Mon, 05 Feb 2024 05:56:55 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVaYke9c25HWtuApe/uvtGvV7vnZoOn1fWG9SS7v5zvQJe4TjNQdcTy+WCDdN2AZwlDQI3m5PG7dztrGpLLB8lzp+OFQ6BnqY1plG2UMeiRw9NT
Message-ID: <c6d803ba-9478-4ab0-9ab6-a9af94808682@suse.com>
Date: Mon, 5 Feb 2024 14:56:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 5/7] VT-d: move ats_device() to the sole file it's used from
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

..., thus allowing it to become static, and thus reducing scope overlap
between it and pci_ats_device(). There's nothing x86-specific about this
function anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -65,8 +65,6 @@ struct acpi_drhd_unit *ioapic_to_drhd(un
 struct acpi_drhd_unit *hpet_to_drhd(unsigned int hpet_id);
 struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd);
 
-int ats_device(const struct pci_dev *, const struct acpi_drhd_unit *);
-
 int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
                          u64 addr, unsigned int size_order, u64 type);
 
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1455,6 +1455,33 @@ static void __hwdom_init cf_check intel_
     }
 }
 
+static int ats_device(const struct pci_dev *pdev,
+                      const struct acpi_drhd_unit *drhd)
+{
+    const struct acpi_satc_unit *satc;
+    int pos;
+
+    if ( !opt_ats || !iommu_qinval )
+        return 0;
+
+    if ( !ecap_queued_inval(drhd->iommu->ecap) ||
+         !ecap_dev_iotlb(drhd->iommu->ecap) )
+        return 0;
+
+    if ( !acpi_find_matched_atsr_unit(pdev) )
+        return 0;
+
+    satc = acpi_find_matched_satc_unit(pdev);
+    if ( opt_ats < 0 && (!satc || !satc->atc_required) )
+        return 0;
+
+    pos = pci_find_ext_capability(pdev->sbdf, PCI_EXT_CAP_ID_ATS);
+    if ( pos )
+        drhd->iommu->flush_dev_iotlb = true;
+
+    return pos;
+}
+
 /*
  * This function returns
  * - a negative errno value upon error,
--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ b/xen/drivers/passthrough/vtd/x86/ats.c
@@ -28,32 +28,6 @@
 #include "../extern.h"
 #include "../../ats.h"
 
-int ats_device(const struct pci_dev *pdev, const struct acpi_drhd_unit *drhd)
-{
-    const struct acpi_satc_unit *satc;
-    int pos;
-
-    if ( !opt_ats || !iommu_qinval )
-        return 0;
-
-    if ( !ecap_queued_inval(drhd->iommu->ecap) ||
-         !ecap_dev_iotlb(drhd->iommu->ecap) )
-        return 0;
-
-    if ( !acpi_find_matched_atsr_unit(pdev) )
-        return 0;
-
-    satc = acpi_find_matched_satc_unit(pdev);
-    if ( opt_ats < 0 && (!satc || !satc->atc_required) )
-        return 0;
-
-    pos = pci_find_ext_capability(pdev->sbdf, PCI_EXT_CAP_ID_ATS);
-    if ( pos )
-        drhd->iommu->flush_dev_iotlb = true;
-
-    return pos;
-}
-
 static bool device_in_domain(const struct vtd_iommu *iommu,
                              const struct pci_dev *pdev, uint16_t did)
 {



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:57:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:57:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676102.1051972 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzTJ-00006H-Sn; Mon, 05 Feb 2024 13:57:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676102.1051972; Mon, 05 Feb 2024 13:57:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzTJ-00006A-QD; Mon, 05 Feb 2024 13:57:17 +0000
Received: by outflank-mailman (input) for mailman id 676102;
 Mon, 05 Feb 2024 13:57:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzTH-0007Qa-Ti
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:57:15 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 76e9b37f-c42e-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 14:57:13 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-40fdf2b69b2so446095e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:57:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s10-20020a05600c044a00b0040efc268713sm8570104wmb.26.2024.02.05.05.57.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:57:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 76e9b37f-c42e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707141433; x=1707746233; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=DHe5MlAMnMR5U49EdLt7+W2yJjwbBdMvr2wcdgX2lS0=;
        b=fyV/cFrHngTlCj+uR2CxjhBlYSRV/IEM41P20GaIY5wW1EWG9B5tAg4auH+0YG2n87
         sT+s2rTvArlnL0LPte5gK0P7s0o0pSu/7bNZTxm1UEH1tdMzF0s7RYxGztr9nJ1i61GH
         lKvXctNW/3StCwSWrHiRMKANxS1sbJ+L6mqApy9Achih9n/JlsPckcaxh7IooHOr3glm
         mgeniiqNdQcGumZoWTxh4L8jKPAm2fsvgQ91LCzLPNGeothzZfcKA8gOTz7iQ8+Cbg2o
         XAEo1Z2Gpfq9W3/VK2vR/5Wq/FyNAXrikCuscOlmcRUrvOIfEEFDqoAmq5Sy778Lumx6
         EW/Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141433; x=1707746233;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=DHe5MlAMnMR5U49EdLt7+W2yJjwbBdMvr2wcdgX2lS0=;
        b=IFAIXc9cyZ5x2K8oVAaRngunMtyrDaeiLPRnaKMqjvaZ8VK9xZCyLs2sCTs559JI7l
         ot8oNWhb11nlriQyzbWlo66MTwv97l+jSqKXFUFrIb1AZVOMpDXJ2OE0vTBjyPvWht6m
         wFu1Er5nujtq0tzsgFLVIbIl6SjRcfgpVZM/uVhhovC/Nwu3eiA1A9wqvRI2+1dokPak
         pCnozKEUiqLDT4PkJr9F/WzuKzrk2LX7J+A4F1v5j55bFs9Tp4qcC07d7xSO04xQl1GP
         A323n+sCNbJsW5O3+ivkeA0KxlORnxq+O/o5gOzw5hgNKhAuUDUwTFyAEhbvPrCDUjn7
         TC1A==
X-Gm-Message-State: AOJu0Yx7E2AdnMK7eF+Pirnwl0/EaMSYNDSQu60pj3wg1hbW+XvwLsn8
	vvgy44dwi7KGAL4923rskji1vFVY8hcj+Lp2MHCSndiXJ4BPWdUg9ZUS/0q+ZRa2B/f2+frpIrE
	=
X-Google-Smtp-Source: AGHT+IEiEcXe2SY92ZwEiLXKnHjQdViDe4WS5m9kWAg3gy+3vDjt0e9/mdD9dVZaDQxIMCIYa3lxgw==
X-Received: by 2002:a05:600c:35cc:b0:40f:cb0d:4de6 with SMTP id r12-20020a05600c35cc00b0040fcb0d4de6mr4305258wmq.5.1707141433377;
        Mon, 05 Feb 2024 05:57:13 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUWUpE7oHc1i8YvDQQ4BMXPfmqOMeZNLennM73+mNklHfyTKYKaeVh+FZWhQBu/doKHY9RsZDtY+30D0ODfyE6tQXPLUUCYocw1xiYc03pJy0KX
Message-ID: <0d14f148-32e0-4dac-b5e8-3230adfc5f9d@suse.com>
Date: Mon, 5 Feb 2024 14:57:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 6/7] VT-d: move dev_invalidate_iotlb() to the sole file it's
 used from
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

..., thus allowing it and qinval_device_iotlb_sync() to become static.
There's nothing x86-specific about the function anyway. While moving,
adjust types to better match ./CODING_STYLE (albeit use of fixed-width
types for parameters is retained to limit the effective change).

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -65,12 +65,6 @@ struct acpi_drhd_unit *ioapic_to_drhd(un
 struct acpi_drhd_unit *hpet_to_drhd(unsigned int hpet_id);
 struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd);
 
-int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
-                         u64 addr, unsigned int size_order, u64 type);
-
-int __must_check qinval_device_iotlb_sync(struct vtd_iommu *iommu,
-                                          struct pci_dev *pdev,
-                                          u16 did, u16 size, u64 addr);
 
 uint64_t alloc_pgtable_maddr(unsigned long npages, nodeid_t node);
 void free_pgtable_maddr(u64 maddr);
--- a/xen/drivers/passthrough/vtd/qinval.c
+++ b/xen/drivers/passthrough/vtd/qinval.c
@@ -251,8 +251,9 @@ static int __must_check dev_invalidate_s
     return rc;
 }
 
-int qinval_device_iotlb_sync(struct vtd_iommu *iommu, struct pci_dev *pdev,
-                             u16 did, u16 size, u64 addr)
+static int qinval_device_iotlb_sync(struct vtd_iommu *iommu,
+                                    struct pci_dev *pdev, uint16_t did,
+                                    uint16_t size, paddr_t addr)
 {
     unsigned long flags;
     unsigned int index;
@@ -282,6 +283,101 @@ int qinval_device_iotlb_sync(struct vtd_
     return dev_invalidate_sync(iommu, pdev, did);
 }
 
+static bool device_in_domain(const struct vtd_iommu *iommu,
+                             const struct pci_dev *pdev, uint16_t did)
+{
+    struct root_entry *root_entry;
+    struct context_entry *ctxt_entry = NULL;
+    unsigned int tt;
+    bool found = false;
+
+    if ( unlikely(!iommu->root_maddr) )
+    {
+        ASSERT_UNREACHABLE();
+        return false;
+    }
+
+    root_entry = map_vtd_domain_page(iommu->root_maddr);
+    if ( !root_present(root_entry[pdev->bus]) )
+        goto out;
+
+    ctxt_entry = map_vtd_domain_page(root_entry[pdev->bus].val);
+    if ( context_domain_id(ctxt_entry[pdev->devfn]) != did )
+        goto out;
+
+    tt = context_translation_type(ctxt_entry[pdev->devfn]);
+    if ( tt != CONTEXT_TT_DEV_IOTLB )
+        goto out;
+
+    found = true;
+ out:
+    if ( root_entry )
+        unmap_vtd_domain_page(root_entry);
+
+    if ( ctxt_entry )
+        unmap_vtd_domain_page(ctxt_entry);
+
+    return found;
+}
+
+static int dev_invalidate_iotlb(struct vtd_iommu *iommu, uint16_t did,
+                                paddr_t addr, unsigned int size_order,
+                                uint64_t type)
+{
+    struct pci_dev *pdev, *temp;
+    int ret = 0;
+
+    if ( !ecap_dev_iotlb(iommu->ecap) )
+        return ret;
+
+    list_for_each_entry_safe( pdev, temp, &iommu->ats_devices, ats.list )
+    {
+        bool sbit;
+        int rc = 0;
+
+        switch ( type )
+        {
+        case DMA_TLB_DSI_FLUSH:
+            if ( !device_in_domain(iommu, pdev, did) )
+                break;
+            /* fall through if DSI condition met */
+        case DMA_TLB_GLOBAL_FLUSH:
+            /* invalidate all translations: sbit=1,bit_63=0,bit[62:12]=1 */
+            sbit = 1;
+            addr = (~0UL << PAGE_SHIFT_4K) & 0x7FFFFFFFFFFFFFFF;
+            rc = qinval_device_iotlb_sync(iommu, pdev, did, sbit, addr);
+            break;
+        case DMA_TLB_PSI_FLUSH:
+            if ( !device_in_domain(iommu, pdev, did) )
+                break;
+
+            /* if size <= 4K, set sbit = 0, else set sbit = 1 */
+            sbit = size_order ? 1 : 0;
+
+            /* clear lower bits */
+            addr &= ~0UL << PAGE_SHIFT_4K;
+
+            /* if sbit == 1, zero out size_order bit and set lower bits to 1 */
+            if ( sbit )
+            {
+                addr &= ~((u64)PAGE_SIZE_4K << (size_order - 1));
+                addr |= (((u64)1 << (size_order - 1)) - 1) << PAGE_SHIFT_4K;
+            }
+
+            rc = qinval_device_iotlb_sync(iommu, pdev, did, sbit, addr);
+            break;
+        default:
+            dprintk(XENLOG_WARNING VTDPREFIX, "invalid vt-d flush type\n");
+            return -EOPNOTSUPP;
+        }
+
+        if ( !ret )
+            ret = rc;
+    }
+
+    return ret;
+}
+
 static int __must_check queue_invalidate_iec_sync(struct vtd_iommu *iommu,
                                                   u8 granu, u8 im, u16 iidx)
 {
--- a/xen/drivers/passthrough/vtd/x86/Makefile
+++ b/xen/drivers/passthrough/vtd/x86/Makefile
@@ -1,2 +1 @@
-obj-y += ats.o
 obj-y += vtd.o
--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ /dev/null
@@ -1,123 +0,0 @@
-/*
- * Copyright (c) 2006, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
- * more details.
- *
- * You should have received a copy of the GNU General Public License along with
- * this program; If not, see <http://www.gnu.org/licenses/>.
- *
- * Author: Allen Kay <allen.m.kay@intel.com>
- */
-
-#include <xen/sched.h>
-#include <xen/iommu.h>
-#include <xen/time.h>
-#include <xen/pci.h>
-#include <xen/pci_regs.h>
-#include <asm/msi.h>
-#include "../iommu.h"
-#include "../dmar.h"
-#include "../vtd.h"
-#include "../extern.h"
-#include "../../ats.h"
-
-static bool device_in_domain(const struct vtd_iommu *iommu,
-                             const struct pci_dev *pdev, uint16_t did)
-{
-    struct root_entry *root_entry;
-    struct context_entry *ctxt_entry = NULL;
-    unsigned int tt;
-    bool found = false;
-
-    if ( unlikely(!iommu->root_maddr) )
-    {
-        ASSERT_UNREACHABLE();
-        return false;
-    }
-
-    root_entry = map_vtd_domain_page(iommu->root_maddr);
-    if ( !root_present(root_entry[pdev->bus]) )
-        goto out;
-
-    ctxt_entry = map_vtd_domain_page(root_entry[pdev->bus].val);
-    if ( context_domain_id(ctxt_entry[pdev->devfn]) != did )
-        goto out;
-
-    tt = context_translation_type(ctxt_entry[pdev->devfn]);
-    if ( tt != CONTEXT_TT_DEV_IOTLB )
-        goto out;
-
-    found = true;
-out:
-    if ( root_entry )
-        unmap_vtd_domain_page(root_entry);
-
-    if ( ctxt_entry )
-        unmap_vtd_domain_page(ctxt_entry);
-
-    return found;
-}
-
-int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
-    u64 addr, unsigned int size_order, u64 type)
-{
-    struct pci_dev *pdev, *temp;
-    int ret = 0;
-
-    if ( !ecap_dev_iotlb(iommu->ecap) )
-        return ret;
-
-    list_for_each_entry_safe( pdev, temp, &iommu->ats_devices, ats.list )
-    {
-        bool sbit;
-        int rc = 0;
-
-        switch ( type )
-        {
-        case DMA_TLB_DSI_FLUSH:
-            if ( !device_in_domain(iommu, pdev, did) )
-                break;
-            /* fall through if DSI condition met */
-        case DMA_TLB_GLOBAL_FLUSH:
-            /* invalidate all translations: sbit=1,bit_63=0,bit[62:12]=1 */
-            sbit = 1;
-            addr = (~0UL << PAGE_SHIFT_4K) & 0x7FFFFFFFFFFFFFFF;
-            rc = qinval_device_iotlb_sync(iommu, pdev, did, sbit, addr);
-            break;
-        case DMA_TLB_PSI_FLUSH:
-            if ( !device_in_domain(iommu, pdev, did) )
-                break;
-
-            /* if size <= 4K, set sbit = 0, else set sbit = 1 */
-            sbit = size_order ? 1 : 0;
-
-            /* clear lower bits */
-            addr &= ~0UL << PAGE_SHIFT_4K;
-
-            /* if sbit == 1, zero out size_order bit and set lower bits to 1 */
-            if ( sbit )
-            {
-                addr &= ~((u64)PAGE_SIZE_4K << (size_order - 1));
-                addr |= (((u64)1 << (size_order - 1)) - 1) << PAGE_SHIFT_4K;
-            }
-
-            rc = qinval_device_iotlb_sync(iommu, pdev, did, sbit, addr);
-            break;
-        default:
-            dprintk(XENLOG_WARNING VTDPREFIX, "invalid vt-d flush type\n");
-            return -EOPNOTSUPP;
-        }
-
-        if ( !ret )
-            ret = rc;
-    }
-
-    return ret;
-}



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 13:57:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 13:57:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676106.1051983 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzTZ-0000cp-B4; Mon, 05 Feb 2024 13:57:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676106.1051983; Mon, 05 Feb 2024 13:57:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzTZ-0000ce-7Q; Mon, 05 Feb 2024 13:57:33 +0000
Received: by outflank-mailman (input) for mailman id 676106;
 Mon, 05 Feb 2024 13:57:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzTY-0000cP-QM
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 13:57:32 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8181d953-c42e-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 14:57:31 +0100 (CET)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-5113a7e662aso2866125e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 05:57:31 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s10-20020a05600c044a00b0040efc268713sm8570104wmb.26.2024.02.05.05.57.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 05:57:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8181d953-c42e-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707141451; x=1707746251; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=uXBzhqSfaSDHiztTRpLA+4XGmypB9k5L3zoFSjZ2FGw=;
        b=WTepeszGzejkvWAtf3tKou+/LYsiSLa3v0GInR0N9Ge14AbFbv1iKA96qNa+Tcr4V+
         mQmwsx+tCnrW/Us4kvssg0sX7FQsaIUjZi+vGllMXlIPcSkygXRtd+TOUFcqyXpnNmnD
         +exqSn/Fh8EBuWIGWOGffBvOOmJqbtId9c/ZPjlpwfxCW1upYpduoHDJbCZ4Y/NNTQ7N
         JMKMZWXzQGrmdpdbIp6Pj6NR6q0EOMuC3bqVIfSk0sBFwDUeAHayI5GkwaFys1hWbgsH
         QDg8rdvECZef0UfZPuuILljnToTcsXRKzs3J6E/1I4nDT5PBHKzAGyNDtxc0YSrQf8XZ
         KwcQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141451; x=1707746251;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=uXBzhqSfaSDHiztTRpLA+4XGmypB9k5L3zoFSjZ2FGw=;
        b=PUWzhIlaOVQ+BldPS4bnPCn909hUMX9RmrCL29JuVPPCSnjdYbp8S805zkGkguJOPJ
         LfGb+bhQl3KEX/MLapdeLXgBVqxU4JbAB1fQruHjbi8wg9RQZDEvZr0jzIMQfROkQe5J
         VS/RjvoPSEpRKdh22GgU8goVrQxS/lFFmSKs7uH0C4Wkqf3ypEtWsUETJ+WhxjusOmuq
         6/JrmCUGZ+2rafesMbIKH6GlSoISglK/9XTygIIoaFSCKnkRzIrTLaNQeS9nAYbVURgW
         E1Epk/nEWwuaM6z60z3XeW9iVFuhYfUwXBhs9VDiW/CqZb0sn77gZKIABR4WfIW+ccre
         25Ng==
X-Gm-Message-State: AOJu0YzvfYGkEAxj0gXmlUU3zU0tlYDQ+PV6CUDsyfFJE+CTTboHVXz6
	ntk+v18v+EQzsInfeQ3nSdoLGOBnwWwmJSCNKkSVAr16AmB/7+IPTjNZaw7MEomDTKAGvQVTrtA
	=
X-Google-Smtp-Source: AGHT+IHLs7xqGMOV9FUSIW0Qe9gIZaLvyw2Qc/RKngXz/bIVhXNReXe7jtU8m9R58AapmeNettF78A==
X-Received: by 2002:a05:6512:3986:b0:511:38e6:6b24 with SMTP id j6-20020a056512398600b0051138e66b24mr7361998lfu.3.1707141451332;
        Mon, 05 Feb 2024 05:57:31 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWsYbpE8CY1tALojkxNPjsAGiREs6ASV67DA7aeFNc+U3dc/3OoOVQeWMP45ydfdCroMNsPU33ctkZIT9Qa+hpjxc1GTgdD+0O7YsaxyXIv/R+N
Message-ID: <22e7036c-cf49-4160-bd26-fbba6b67ff5d@suse.com>
Date: Mon, 5 Feb 2024 14:57:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 7/7] VT-d: move {,un}map_vtd_domain_page()
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <25506838-b818-4686-8c16-3a198338af44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

..., thus allowing them to become static. There's nothing x86-specific
about these functions anyway.

Since only the "iommu_inclusive_mapping" parameter declaration would be
left in the file, move that as well. There's nothing VT-d specific about
it (anymore?): "dom0-iommu=map-inclusive" is similarly generic, and
documentation also doesn't say anything.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/drivers/passthrough/vtd/Makefile
+++ b/xen/drivers/passthrough/vtd/Makefile
@@ -1,5 +1,3 @@
-obj-$(CONFIG_X86) += x86/
-
 obj-y += iommu.o
 obj-y += dmar.o
 obj-y += utils.o
--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -21,6 +21,7 @@
 #define _VTD_EXTERN_H_
 
 #include "dmar.h"
+#include <xen/domain_page.h>
 #include <xen/keyhandler.h>
 
 #define VTDPREFIX "[VT-D]"
@@ -68,8 +69,6 @@ struct acpi_rhsa_unit *drhd_to_rhsa(cons
 
 uint64_t alloc_pgtable_maddr(unsigned long npages, nodeid_t node);
 void free_pgtable_maddr(u64 maddr);
-void *map_vtd_domain_page(u64 maddr);
-void unmap_vtd_domain_page(const void *va);
 int domain_context_mapping_one(struct domain *domain, struct vtd_iommu *iommu,
                                uint8_t bus, uint8_t devfn,
                                const struct pci_dev *pdev, domid_t domid,
@@ -79,6 +78,16 @@ int domain_context_unmap_one(struct doma
 int cf_check intel_iommu_get_reserved_device_memory(
     iommu_grdm_t *func, void *ctxt);
 
+static inline void *map_vtd_domain_page(paddr_t maddr)
+{
+    return map_domain_page(_mfn(paddr_to_pfn(maddr)));
+}
+
+static inline void unmap_vtd_domain_page(const void *va)
+{
+    unmap_domain_page(va);
+}
+
 unsigned int cf_check io_apic_read_remap_rte(
     unsigned int apic, unsigned int reg);
 void cf_check io_apic_write_remap_rte(
--- a/xen/drivers/passthrough/vtd/x86/Makefile
+++ /dev/null
@@ -1 +0,0 @@
-obj-y += vtd.o
--- a/xen/drivers/passthrough/vtd/x86/vtd.c
+++ /dev/null
@@ -1,48 +0,0 @@
-/*
- * Copyright (c) 2008, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
- * more details.
- *
- * You should have received a copy of the GNU General Public License along with
- * this program; If not, see <http://www.gnu.org/licenses/>.
- *
- * Copyright (C) Allen Kay <allen.m.kay@intel.com>
- * Copyright (C) Weidong Han <weidong.han@intel.com>
- */
-
-#include <xen/param.h>
-#include <xen/sched.h>
-#include <xen/softirq.h>
-#include <xen/domain_page.h>
-#include <asm/paging.h>
-#include <xen/iommu.h>
-#include <xen/irq.h>
-#include <xen/numa.h>
-#include <asm/fixmap.h>
-#include "../iommu.h"
-#include "../dmar.h"
-#include "../vtd.h"
-#include "../extern.h"
-
-/*
- * iommu_inclusive_mapping: when set, all memory below 4GB is included in dom0
- * 1:1 iommu mappings except xen and unusable regions.
- */
-boolean_param("iommu_inclusive_mapping", iommu_hwdom_inclusive);
-
-void *map_vtd_domain_page(u64 maddr)
-{
-    return map_domain_page(_mfn(paddr_to_pfn(maddr)));
-}
-
-void unmap_vtd_domain_page(const void *va)
-{
-    unmap_domain_page(va);
-}
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -19,6 +19,7 @@
 #include <xen/paging.h>
 #include <xen/guest_access.h>
 #include <xen/event.h>
+#include <xen/param.h>
 #include <xen/softirq.h>
 #include <xen/vm_event.h>
 #include <xsm/xsm.h>
@@ -36,6 +37,12 @@ bool __initdata iommu_superpages = true;
 
 enum iommu_intremap __read_mostly iommu_intremap = iommu_intremap_full;
 
+/*
+ * iommu_inclusive_mapping: When set, all memory below 4GB is included in dom0
+ * 1:1 iommu mappings except xen and unusable regions.
+ */
+boolean_param("iommu_inclusive_mapping", iommu_hwdom_inclusive);
+
 #ifdef CONFIG_PV
 /* Possible unfiltered LAPIC/MSI messages from untrusted sources? */
 bool __read_mostly untrusted_msi;



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:02:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:02:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676116.1051993 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzYA-0002yr-Sw; Mon, 05 Feb 2024 14:02:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676116.1051993; Mon, 05 Feb 2024 14:02:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzYA-0002yk-QM; Mon, 05 Feb 2024 14:02:18 +0000
Received: by outflank-mailman (input) for mailman id 676116;
 Mon, 05 Feb 2024 14:02:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzY9-0002ye-Sd
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:02:17 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2b429e2a-c42f-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 15:02:16 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33b402116e5so411569f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 06:02:16 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 t10-20020adff04a000000b003392206c808sm8079238wro.105.2024.02.05.06.02.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 06:02:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b429e2a-c42f-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707141736; x=1707746536; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=YdbLGgIFQ8R+s1/q4QmsyMW3KYfCZDydBE9Pbqdi9y8=;
        b=bHeVE96iEAvxIMHNnM2T6CREHTl5talzetbNyZbvMyOek+y47e96PjYkpi+sOuxsAY
         WWBRb+fzTmw+azkJfchkVSw+c737kyyejS1yAvtSf8ylHaBUj9dY5SRbn68xwnEA0j4N
         U0EjADXOv/1MkRGHlhW7Vl6nT4iy8OqEVVMK/+s7Ycn7lya2DIyqlZmw3DfzjjYuNc/a
         WHVAzXIAdRR/TSOccWUC9Y/Z2jvdzENmRmp6DzLbGeE9YDleyU+V0C+wfAqX0Qbz7MdK
         MjvPzV5B6ieyYi5s2M5f3q++pamrfOKwTATiEWlaKSeIUdJ39FNKuSVZxf+8gsWy4R/5
         sqNg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141736; x=1707746536;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=YdbLGgIFQ8R+s1/q4QmsyMW3KYfCZDydBE9Pbqdi9y8=;
        b=L2SF6yjmvskD3eF96lM4yIR9vogfLhCtCRNUuySAit4uJrpfYEhun0gFyF0nCJ4MNz
         hL6csroQ+xHL1xTLGhg1yGyRxnMUZhkKl2BdHN4B1EtV5KM/YmAO1z5zvj8JEcTesNbq
         9LWC72GMRv4+lgMGL5491gYthhYuz1T77ElZwXPIcGsS0wZdw958/bAZEuMI4Vt2JIby
         I9KZHRq6f0EH6IaqvEUbPBB8XzoS8rcEvEuKiDaNsfBpJXkhV96kVbkNMjPzE7ArwEw8
         r30zzyLMYFI6eGm1SF71xEXbZaIPFHMl1ZqGK+fXNjxqk2JxylVmlipYjgFPnC34gJZl
         q05g==
X-Gm-Message-State: AOJu0YymT3Xv66NVMcn/7H/2PmfbPcxy2PXkhvJBxV382MX3u9nD9Ikk
	YDpj+9jsQ08+WScdvAUckKTGMaWaHn+3WUH48AfSBxGPTeXVAvK5fXEFI6bHVw==
X-Google-Smtp-Source: AGHT+IFBGL7x0Lq/IGB0XCGLTTtLnpd/GbVhHxBgeyF0c4E5jkBKxx/llTTndYurambVlr8t+w59dA==
X-Received: by 2002:a5d:4809:0:b0:33b:1131:ebfd with SMTP id l9-20020a5d4809000000b0033b1131ebfdmr8781342wrq.49.1707141736150;
        Mon, 05 Feb 2024 06:02:16 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXP0fPeHZzcNDfu4yKGWVDpuOV+waIgQWKtnTH5eMbjKwfLC/hCNVZ652vqrXfLG6QvH1fYovHWMNx5SJEUW1BCRXkVSRwoXy/Xn/Ut1FQesTCosTUi3v7QkpPQPyKeqsnKy2IfnCkC8GVbHjlzmHIYWTX2xqSnwL22vYERM2QA1uk5X0J8iGwoEeprZGORgPSYkaatPxoIW3iIt0Mzn9iqzCs7oOXBkVzdDLCnha9/8Ya54IUg+0jebjySPNKLljcN9YLO0F3eYa6IkbTZwSGmx88BKWpBKTJM8FPQXjuO2gGpzr4=
Message-ID: <6f4dc7e2-c54d-4dce-8cd3-7fd45391eb0d@suse.com>
Date: Mon, 5 Feb 2024 15:02:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 8/8] consolidate do_bug_frame() / bug_fn_t
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <f20d2cef-4435-4b9a-9ad8-61ceddb6bcba@suse.com>
 <f97088c6-ec2e-4c60-b91d-ec1a30d077df@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f97088c6-ec2e-4c60-b91d-ec1a30d077df@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 05.02.2024 14:51, Andrew Cooper wrote:
> On 05/02/2024 1:32 pm, Jan Beulich wrote:
>> The type not being used in do_bug_frame() is suspicious. Apparently
>> that's solely because the type uses a pointer-to-const parameter,
>> when so far run_in_exception_handler() wanted functions taking pointer-
>> to-non-const. Expand use of const, in turn requiring common code's
>> do_bug_frame() as well as [gs]et_irq_regs() to also gain const. This
>> then brings the former function also closer to the common one, with
>> Arm's use of vaddr_t remaining as a difference.
>>
>> While there also replace the bogus use of hard tabs in [gs]et_irq_regs()
>> (I clearly didn't mean to put it in like this).
> 
> I meant to query that at the time and clearly forgot to.
> 
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> I'm still confident we can get rid of the fake frame in the serial
> drivers, but this is an improvement nonetheless.
> 
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Thanks.

> I'll rebase my work over this.  It's going to collide horribly.

For the patch here they're affected only because I stuck the patch at
the end of the series. I think it ought to be possible to move it to
the front, and then it could be left to be determined whether my
introducing of set_irq_regs() in the poll handlers could actually be
avoided by whatever work you have pending / in progress.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:05:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:05:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676125.1052003 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzbB-0003cz-A8; Mon, 05 Feb 2024 14:05:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676125.1052003; Mon, 05 Feb 2024 14:05:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzbB-0003cs-75; Mon, 05 Feb 2024 14:05:25 +0000
Received: by outflank-mailman (input) for mailman id 676125;
 Mon, 05 Feb 2024 14:05:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzb9-0003cm-Hq
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:05:23 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 99562ea2-c42f-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 15:05:22 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-40fc52c2ae4so29014335e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 06:05:21 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p4-20020a5d4e04000000b0033b3ceda5dbsm2367341wrt.44.2024.02.05.06.05.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 06:05:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99562ea2-c42f-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707141921; x=1707746721; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ngGvdTM+gCWTGJaPodlHCVbpAEcmrptnfv2LPQTxfYc=;
        b=d/YXxRTGgWOwCsc6s4yRt5xziPLNxzXegu6VA9HJJk3E4ZL3Ga6VS+Uw8GAnBtp6IC
         5KsrnMsh9mfYN7/iVgucjCuBRX3ZBY8cdDCk8IhN+zsVJBOXgqAIPeebQvooQiOUlXZZ
         ICMOHWEJzqraQMG0hZAvSwpGMDwXsvsUNLLHjVZaUCJoeAPazYU1qWsx9PLP3QuCjUXh
         HDV+pibjUw7JDgkkf/5N6oY19NUgdE28WgEnE4HdiB6UT6VMtTSuYEpOSY1QnGz0j+K3
         sJgRI/gRFQszoEQL1b3D98EhmcRAWgBRL7laac5oPQ/c2mzyr6A7jiqE5CLLVHsm1q07
         A6jg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707141921; x=1707746721;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ngGvdTM+gCWTGJaPodlHCVbpAEcmrptnfv2LPQTxfYc=;
        b=Fp02stL7yvNQZaO1RNwc+7KmZlQ0aDtQ6R4KEqOJ3D4DWGVa+VrCERo4JoaEw5CTCF
         W6PO7OoL4L7KwNyFGbai+kkKUjRxoVrWazeMtDug0qFqOP6zf68hIN/EAiEzEZZh0VTM
         zK5/NpYMzLCBw/lf5B1bFeu9bxb4U89MZnRs/SZk2tjVUcUYVNzshECOUQbcFw0Y9AcA
         3sAB3DKdVBd4Kl3iRhPtVOsiGDA6IM94vLujyA6C+hkeczWwGh3ymAxJBJD+lGvfBqMK
         s5G2rLNGoV5EJak+0YWma5Hn7gqZsz/8ITosh4Sm8PkbNKr1Xi+wGRZ6mKTnYpU7TS1s
         VOmA==
X-Gm-Message-State: AOJu0Yyu8DdZfbIUhhBWGpCyf9PpOi8NKWp0qZfjP0NlV66GZnqR6cKH
	AjCIDj6eegxSGxb5IfJLV1nZzYoj8eN6O3pJDJvGZTZ4tjjDzfyxM2ccKzE4hg==
X-Google-Smtp-Source: AGHT+IHfJYlcae+Bt1kKIZMeJZjsKkMR12d10EsaZDT9hPt3lzAJT0OZ8Q1p/g/PPu05A/y5kAuatA==
X-Received: by 2002:a05:600c:4ed2:b0:40f:dd0e:1ed3 with SMTP id g18-20020a05600c4ed200b0040fdd0e1ed3mr1351733wmq.28.1707141920811;
        Mon, 05 Feb 2024 06:05:20 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXDiYRclsbruPeGzd9FReUxd0ATFLcLR4CvNfwa9bJc/7RtEdVIjIhEROgv4iwTguayoRFL52DRloG4D7iZJTO7gqRHp23nNy3Lx0mviijZBFtM/biAW/xnKY089NWG+jnqYIFT+oq46GIZAOnxW8D2t8gUtcn3/+0RqRLrpEvaRXLDBEAJAhyPEbtw1sDvHMuOeNqHojIlVviVBYl582lnxc9pHBlD7lXdvshz17N9rP4QrYteK0eLz1f25P8yRQl0lQmJFcYpfX9iVEcewMDICUXwFG7eJ6Zf8LQIdNvZjE5D5xtffuc=
Message-ID: <8b71122a-8446-4619-992e-ec2475e602d0@suse.com>
Date: Mon, 5 Feb 2024 15:05:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 31/34] xen/riscv: add minimal stuff to mm.h to build
 full Xen
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1703255175.git.oleksii.kurochko@gmail.com>
 <4411f6af38586074b347cd6005f19f9c670faa74.1703255175.git.oleksii.kurochko@gmail.com>
 <d347c4d9-e93b-4937-8e33-e5fbbdcd6bfb@suse.com>
 <f3858360fba14a2c5b794bdd0400cd7ab8e66f73.camel@gmail.com>
 <44a9a92c-ac4c-43d0-bc1e-685818e4331d@suse.com>
 <211e7d7fb7d52d0bc7dbb12d5619a39f98d5b21c.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <211e7d7fb7d52d0bc7dbb12d5619a39f98d5b21c.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 05.02.2024 13:49, Oleksii wrote:
> On Mon, 2024-02-05 at 08:46 +0100, Jan Beulich wrote:
>> On 02.02.2024 18:30, Oleksii wrote:
>>> On Tue, 2024-01-23 at 14:03 +0100, Jan Beulich wrote:
>>>> On 22.12.2023 16:13, Oleksii Kurochko wrote:
>>>>> +/* Convert between Xen-heap virtual addresses and page-info
>>>>> structures. */
>>>>> +static inline struct page_info *virt_to_page(const void *v)
>>>>> +{
>>>>> +    BUG();
>>>>> +    return NULL;
>>>>> +}
>>>>> +
>>>>> +/*
>>>>> + * We define non-underscored wrappers for above conversion
>>>>> functions.
>>>>> + * These are overriden in various source files while
>>>>> underscored
>>>>> version
>>>>> + * remain intact.
>>>>> + */
>>>>> +#define virt_to_mfn(va)     __virt_to_mfn(va)
>>>>> +#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
>>>>
>>>> Is this really still needed? Would be pretty nice if in a new
>>>> port we
>>>> could get to start cleanly right away (i.e. by not needing per-
>>>> file
>>>> overrides, but using type-safe expansions here right away).
>>> We still need __virt_to_mfn and __mfn_to_virt as common code use
>>> them:
>>>  * xen/common/xenoprof.c:24:#define virt_to_mfn(va)
>>> mfn(__virt_to_mfn(va))
>>
>> Are you meaning to enable this code on RISC-V.
> Yes, that is what I meant.

And why would you do that? Even Arm doesn't use it, and I'd expect no
newer port would care about this very old interface.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:10:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:10:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676128.1052013 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzgC-0005RO-SC; Mon, 05 Feb 2024 14:10:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676128.1052013; Mon, 05 Feb 2024 14:10:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzgC-0005RH-Oq; Mon, 05 Feb 2024 14:10:36 +0000
Received: by outflank-mailman (input) for mailman id 676128;
 Mon, 05 Feb 2024 14:10:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GYrN=JO=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rWzgB-0005RB-7L
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:10:35 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 52c48451-c430-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 15:10:32 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-55ff5f6a610so3227807a12.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 06:10:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 52c48451-c430-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1707142232; x=1707747032; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=isqSLNsyO1Z/m6NrOhdX+SC2ehMdHYvA5evFUfcg2nc=;
        b=DOWinvSsVWTNshJBqxHfeJGBjeCT+x/gpA4+2uw678bUqfdNY+Pajiq/wuQmc0/7Wl
         +GcvsQghdTSaoFbgx5LI4UGvwACMuOL9gJsKsY2I9UiMXWQx17l2ndw4REidjnekWjb9
         tXGc+qDIhu/splydL5/FECFMan/SOhmn1SQKlUZYv3FN6TWRoCY8b0KiXi0SpKNFyzWD
         PFbUS+A7YpeyUjTu18OJBwcHQUssMu2yR+PH5nA5GZHBVmoFgYQYynUGnKXTWRLlygwH
         1KCyv3/zT1ZqtIoEquWhLfnqQThTTba9DDnXWwvEaNeE+/Hh96O4FD/FEjlJqbIT5jss
         t3lw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707142232; x=1707747032;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=isqSLNsyO1Z/m6NrOhdX+SC2ehMdHYvA5evFUfcg2nc=;
        b=qh4sbmn+fla/3nTzi/qj+sCdtEImOqJ/p02y/t7akiwc8623gD4/fZHHZEejOALak9
         Eo3S4xUh9S2bLFIcC1C4XAenGQ92sx6bVeeNUj4GsquI/h3oUjH75Lq3BJtBIuR5ZEvy
         +SZW71EWj1hY2Y9Gxu0xHGtguxQ/y4DWEQu+eLgFy/tf1YP7p2zMNSjr+tFOpyRcqc68
         yYmqxv5e7V60ErG3u94f8C0/+KIwhTztZUh7hQh0q3Q1VT+CarvD0eNFjDN8cNgBhbt5
         x1v4R8hK/zWXn7tJTLcGWLGFsqwOB3FL4t57JmzQDr2NqcckJ5MQxhPWt1TkMH8G0gzV
         4mzg==
X-Gm-Message-State: AOJu0YziLsZG6Rkfp066sudg8NjcaYDw6ztB6EVS9AQtbwFl17Pc5rhG
	k68eGsEPnVCW+osdpMFzZvOwcpbRHsvKDyn8xXQaXmE2U3DphVAL1SxXZKfJck7y3o7aARklOnn
	K4PnT9opi4p4x/45Xpeb7fO3Kp50HdiFNbv6/KFee7GClhY+t+CIvUg==
X-Google-Smtp-Source: AGHT+IGmrKVliBNlWXLrOIXZoAUW/ldABDPwlMcAPTe6FDs55Rt3TbRi3JGhkbXPAtgwohauXxtMn6hS+iNCVs/XHf4=
X-Received: by 2002:aa7:cf18:0:b0:55f:6b8:b304 with SMTP id
 a24-20020aa7cf18000000b0055f06b8b304mr5457510edy.8.1707142231692; Mon, 05 Feb
 2024 06:10:31 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-11-carlo.nonato@minervasys.tech> <1739dbb4-5d7e-4d84-8d4b-bfb13c4041d8@suse.com>
 <CAG+AhRUJysNCrUXLiu8tvh2d7oL2hNAtkM3A2-Hy73f=+PxkTA@mail.gmail.com> <0ef140da-77d3-411f-8f19-eee0aaf1fa9d@suse.com>
In-Reply-To: <0ef140da-77d3-411f-8f19-eee0aaf1fa9d@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Mon, 5 Feb 2024 15:10:20 +0100
Message-ID: <CAG+AhRVTpcEHpWVY1qUNt3Uc-cDPqV2ZRe8rwfvY4YZUPUP7Ew@mail.gmail.com>
Subject: Re: [PATCH v6 10/15] xen: add cache coloring allocator for domains
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Marco Solieri <marco.solieri@minervasys.tech>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Mon, Feb 5, 2024 at 1:38=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 05.02.2024 12:59, Carlo Nonato wrote:
> > On Thu, Feb 1, 2024 at 4:53=E2=80=AFPM Jan Beulich <jbeulich@suse.com> =
wrote:
> >> On 29.01.2024 18:18, Carlo Nonato wrote:
> >>> @@ -157,7 +158,11 @@
> >>>  #define PGC_static 0
> >>>  #endif
> >>>
> >>> -#define PGC_preserved (PGC_extra | PGC_static)
> >>> +#ifndef PGC_colored
> >>> +#define PGC_colored 0
> >>> +#endif
> >>> +
> >>> +#define PGC_preserved (PGC_extra | PGC_static | PGC_colored)
> >>>
> >>>  #ifndef PGT_TYPE_INFO_INITIALIZER
> >>>  #define PGT_TYPE_INFO_INITIALIZER 0
> >>> @@ -1504,6 +1509,7 @@ static void free_heap_pages(
> >>>              if ( !mfn_valid(page_to_mfn(predecessor)) ||
> >>>                   !page_state_is(predecessor, free) ||
> >>>                   (predecessor->count_info & PGC_static) ||
> >>> +                 (predecessor->count_info & PGC_colored) ||
> >>
> >> How about a 2nd #define (e.g. PGC_no_buddy_merge) to use here and ...
> >>
> >>>                   (PFN_ORDER(predecessor) !=3D order) ||
> >>>                   (page_to_nid(predecessor) !=3D node) )
> >>>                  break;
> >>> @@ -1528,6 +1534,7 @@ static void free_heap_pages(
> >>>              if ( !mfn_valid(page_to_mfn(successor)) ||
> >>>                   !page_state_is(successor, free) ||
> >>>                   (successor->count_info & PGC_static) ||
> >>> +                 (successor->count_info & PGC_colored) ||
> >>
> >> ... here? That'll then also avoid me commenting that I don't see why
> >> these two PGC_* checks aren't folded.
> >
> > Yes for me it's ok (I even suggested that in v5). Do you want this chan=
ge to
> > be merged with the previous patch? Or should they all belong to this on=
e?
>
> Or make yet another small prereq patch?

Ok.

> >>> +static void free_color_heap_page(struct page_info *pg, bool need_scr=
ub)
> >>> +{
> >>> +    unsigned int color =3D page_to_llc_color(pg);
> >>> +    struct page_list_head *head =3D color_heap(color);
> >>> +
> >>> +    spin_lock(&heap_lock);
> >>> +
> >>> +    mark_page_free(pg, page_to_mfn(pg));
> >>> +
> >>> +    if ( need_scrub )
> >>> +    {
> >>> +        pg->count_info |=3D PGC_need_scrub;
> >>> +        poison_one_page(pg);
> >>> +    }
> >>> +
> >>> +    pg->count_info |=3D PGC_colored;
> >>
> >> The page transiently losing PGC_colored is not an issue then (presumab=
ly
> >> because of holding the heap lock)? Did you consider having mark_page_f=
ree()
> >> also use PGC_preserved?
> >
> > I did something similar to what it's done in unprepare_staticmem_pages(=
):
> > first mark_page_free() is called and then PGC_static is added to count_=
info.
>
> I had guessed this would have been served as reference, yet by saying wha=
t
> you did you don't really answer my question. (Really I'm not entirely sur=
e
> the similar staticmem behavior is entirely correct.)

I think holding the heap lock allows writing pg->count_info. Anyway, yes, I
can use PGC_preserved in mark_page_free(), but then pages must be correctly
initialized with PGC_colored elsewhere.

> >>> +    if ( !pg )
> >>> +    {
> >>> +        spin_unlock(&heap_lock);
> >>> +        return NULL;
> >>> +    }
> >>> +
> >>> +    pg->count_info =3D PGC_state_inuse | PGC_colored |
> >>> +                     (pg->count_info & PGC_need_scrub);
> >>
> >> Isn't PGC_colored already set on the page?
> >
> > Yes but here I need to make sure that only PGC_state_inuse | PGC_colore=
d are
> > used so an assignment seems legit.
>
> Well, yes, you won't be able to avoid an assignment, but couldn't you
> preserve PGC_colored just like PGC_need_scrub is preserved? Or else
> at least assert the flag is set, to avoid giving the impression that
> right here pages can suddenly become "colored" ones? Then again them
> becoming so _may_ be needed during initialization.

Yeah, now I understood.

>
> >> Together with ...
> >>
> >>> +    free_colored_pages[color]--;
> >>> +    page_list_del(pg, color_heap(color));
> >>> +
> >>> +    if ( !(memflags & MEMF_no_tlbflush) )
> >>> +        accumulate_tlbflush(&need_tlbflush, pg, &tlbflush_timestamp)=
;
> >>> +
> >>> +    init_free_page_fields(pg);
> >>> +
> >>> +    spin_unlock(&heap_lock);
> >>> +
> >>> +    if ( test_and_clear_bit(_PGC_need_scrub, &pg->count_info) && scr=
ub )
> >>
> >> ... this, can't the above be simplified?
> >
> > I tried to replicate what happens in alloc_heap_pages() where:
> >
> >>  /* Preserve PGC_need_scrub so we can check it after lock is dropped. =
*/
> >>  pg[i].count_info =3D PGC_state_inuse | (pg[i].count_info & PGC_need_s=
crub);
> >
> > and then after the unlock the bit is tested.
>
> Again I was indeed assuming that you used existing code as reference.
> Yet again my question was whether that's actually what is needed here
> (which then may or may not mean that it could be done differently
> also there).

Ok. Probably isn't needed since we are referring to a single page. So I can
first test for the bit, save the result in a variable and remove the
preservation, then later use the variable instead of test_and_clear_bit().

> >>> @@ -2458,7 +2626,14 @@ struct page_info *alloc_domheap_pages(
> >>>      if ( memflags & MEMF_no_owner )
> >>>          memflags |=3D MEMF_no_refcount;
> >>>
> >>> -    if ( !dma_bitsize )
> >>> +    /* Only domains are supported for coloring */
> >>> +    if ( d && llc_coloring_enabled )
> >>> +    {
> >>> +        /* Colored allocation must be done on 0 order */
> >>> +        if ( order || (pg =3D alloc_color_heap_page(memflags, d)) =
=3D=3D NULL )
> >>> +            return NULL;
> >>> +    }
> >>
> >> I think I had asked before: What about MEMF_node() or MEMF_bits()
> >> having been used by the caller?
> >
> > MEMF_node() is used for NUMA, right? I think that for the moment, since=
 cache
> > coloring is supported only on arm64 and NUMA is not yet supported, it's=
 safe
> > to ignore it.
>
> NUMA or not, I'm wary of silent ignoring of inputs. I'd rather request
> that, just like with staticmem, precautions are then taken to avoid
> coloring and NUMA to be used together. Recall that much like your work,
> work to support NUMA is also in progress (unless it was canceled, but
> not so announced).

Ok.

> > I'm not sure I understood what MEMF_bits() is for. It restricts the all=
ocator
> > to return pages in some range, right?
>
> Correct.

So I need to think about this, since I see no easy way of supporting such a
thing with the current colored allocator implementation.
Do you have any suggestions?

Thanks.

> Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:14:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:14:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676135.1052023 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzkE-0006Mn-Ft; Mon, 05 Feb 2024 14:14:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676135.1052023; Mon, 05 Feb 2024 14:14:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzkE-0006Mg-Cl; Mon, 05 Feb 2024 14:14:46 +0000
Received: by outflank-mailman (input) for mailman id 676135;
 Mon, 05 Feb 2024 14:14:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W3Sh=JO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rWzkE-0006Ma-2p
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:14:46 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e8c4872b-c430-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 15:14:43 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a37878ac4f4so160937666b.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 06:14:43 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 tk3-20020a170907c28300b00a36d0dae491sm4386283ejc.202.2024.02.05.06.14.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 06:14:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e8c4872b-c430-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707142483; x=1707747283; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=41QmEK7DxB5U6RS33FRHcI/eQ7monSTUA7T7HhrUaQM=;
        b=apBMmfmZc4YTmkzT4YVFc7WJuza9uTNZGpRpS9i6v/mLmSTnb8YJCFXOVCWMuB1dnJ
         bQeXF9CRm+5b5L66aQfUFZ23NxpExXFWhgbXEHUe79GiCoeBGsP0/ZKhwKvHeOHj58h3
         QZH0oom/gzSfFsbkLmBehDBcw88X1L+yAck+w=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707142483; x=1707747283;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=41QmEK7DxB5U6RS33FRHcI/eQ7monSTUA7T7HhrUaQM=;
        b=fac0clUxeqDJEGqkNcfntGDbMlL4az8UQwQrmg37Rrji6sUzgWDMd93NNnwxcyAREC
         NFA7VmSbzAcFCAApNv5/qVQ8u3qQXaHRgMO/pY3/yqbxccXJRK4Wg7j8vDfEW1xb3jLT
         5PNrsPrrKOrSfZzk/T89Wche9TRX2iyPeWsTUT1rkBqPhenixcs4Rkv+ERS6rIl7oDWA
         yF1gRkOD0U7mLxfPyHB60mQXnz1cTIOuoC0XSRin+vdDEVaFAhGQlhKIRbe4T8j6bmlQ
         tn4xFw6vScSssAli3EoWUbDgwIftqg6b3xk1D3ntPSjljaoQewZ0GrHPzb/vyHTvQeB/
         UMZA==
X-Gm-Message-State: AOJu0YxT/QgRoO4S0Bo89WTw7/GDiubUvgHDLeB/9g2r7PSukvZ5Oon2
	9MAi7eJQzqa0H3XRAdDkiIh+/GU6c1tMmZC+3R5HxLNs3QjuktQNzRW4aTmktq7LtgzyJkIAXuG
	s
X-Google-Smtp-Source: AGHT+IGz7CuvzQlHrzbkmmxFpkz2VFiR2Dj/lSIvS8xZZn9YBXerEWe58k9CxXWv2QkFw1jGOJIAgQ==
X-Received: by 2002:a17:906:b56:b0:a37:8227:debb with SMTP id v22-20020a1709060b5600b00a378227debbmr3459007ejg.30.1707142483148;
        Mon, 05 Feb 2024 06:14:43 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCW8ItI73Mh/4o2xJVmIaqrJb3ccRHjjLexE04qMZwKg0YDtPE4z2esDIkEAeJBWzEvIbYy1PFBkOLmoIAIkTq/iFK2artPkBp84vMmRCxPUwesmesZdKtARnJBESUgb3KsAd/U=
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/cpu: Fix mixed tabs/spaces
Date: Mon,  5 Feb 2024 14:14:41 +0000
Message-Id: <20240205141441.1880188-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Fixes: 660f8a75013c ("x86/CPU: convert vendor hook invocations to altcall")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 xen/arch/x86/cpu/common.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index 26eed2ade1e3..28d7f34c4dbe 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -336,8 +336,8 @@ void __init early_cpu_init(bool verbose)
 
 	c->x86_vendor = x86_cpuid_lookup_vendor(ebx, ecx, edx);
 	switch (c->x86_vendor) {
-	case X86_VENDOR_INTEL:	  actual_cpu = intel_cpu_dev;    break;
-	case X86_VENDOR_AMD:	  actual_cpu = amd_cpu_dev;      break;
+	case X86_VENDOR_INTEL:    actual_cpu = intel_cpu_dev;    break;
+	case X86_VENDOR_AMD:      actual_cpu = amd_cpu_dev;      break;
 	case X86_VENDOR_CENTAUR:  actual_cpu = centaur_cpu_dev;  break;
 	case X86_VENDOR_SHANGHAI: actual_cpu = shanghai_cpu_dev; break;
 	case X86_VENDOR_HYGON:    actual_cpu = hygon_cpu_dev;    break;

base-commit: 9eee78988ca081eeed0a69fc9773e2e4268d1ed7
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:30:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:30:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676140.1052033 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzzD-0003lF-Oj; Mon, 05 Feb 2024 14:30:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676140.1052033; Mon, 05 Feb 2024 14:30:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rWzzD-0003l8-M2; Mon, 05 Feb 2024 14:30:15 +0000
Received: by outflank-mailman (input) for mailman id 676140;
 Mon, 05 Feb 2024 14:30:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rWzzD-0003ky-0Y
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:30:15 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 11cadc41-c433-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 15:30:11 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-40fd55c7f93so13886085e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 06:30:11 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z4-20020a05600c0a0400b0040fcaec04b8sm32498wmp.22.2024.02.05.06.30.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 06:30:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11cadc41-c433-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707143411; x=1707748211; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wKHQD7J0KrjI4QYeHCYKNTmsHSzza/9OcMlhsFtXRr0=;
        b=LG9OV58lYRFKA4T2BAS1QpWFEK5uyXjq7ftB1uqXUymhuTb/uOZ+zIhpv2a8RgXXal
         2hIZWK/yl7/K5ojCnx6CWfVfMIuJ7c34LIBraQjaLlKNoB3ahmMzDeHeNWAicJMQf2ns
         7zfHou98irNLHZQsTiAqM2jlrCeaqfrBnlRJg18w0RMukLQVknk8dYZaGyEAEn3EO8Gy
         B+TecxUz2kURiTM8uJA+GemUFETowJnsqiPMsPh7U/WabZH57xu+KYhUHZjXRXW4Cqa4
         lh83Wd/fGpwcFil4/4zelpq18n/VRpT7CvaPPVuhi5I1qjNw8sEJUuYlD6DkBHuXUBf7
         aEYA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707143411; x=1707748211;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wKHQD7J0KrjI4QYeHCYKNTmsHSzza/9OcMlhsFtXRr0=;
        b=jqWdpd4Zu2YFDreVOwPd7wXlswhw0yfMR5Qfc3XvUlH3Mx5EtEw+I3wSyTK3xxII3r
         /nB/FnyxNCkWUxiEbnnmwCuEkcjuG9PlhBCnbyyatY6KSFvzfviwwjTHdU+Z4VEE8oeW
         JRUCOK/TPSQFY5oBbnlEWgE7z2mKHLY8LGL/Pa+47xfeROVvJhLpwh2y/fUAsnK93e6/
         AUSRVLwA+4MC4sOtIXP6zO8gdbMBNBDGX59MtgGNLfH4zZF22tJmWmNYi+L3aWC6l1eC
         QIbwZc49tAck6IkWh2MClodHQv+Fh2GXQKnuOz9UL1kgaZtr+ej0AhLFtUSLTgiOUNU1
         o+Uw==
X-Gm-Message-State: AOJu0YxHf8pxr1cd873CBgQwlaLDv8rwAu/kHZyAGdfuLLSdtPZsRHTm
	iJl2kwybMLgx/bSBj08+mo2dTfptSApTX9AOpHeRWdndxurHC4DMr2B3AJKQbA==
X-Google-Smtp-Source: AGHT+IH5jAhmzYd8DjRNLQOdzebHVnH6igLRPb8eZcOOfUC9cm6wQBScAQ5yXoVvXk0A2yp8DaILDw==
X-Received: by 2002:a5d:5f46:0:b0:33b:2856:5188 with SMTP id cm6-20020a5d5f46000000b0033b28565188mr8502022wrb.44.1707143411118;
        Mon, 05 Feb 2024 06:30:11 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWXVlDLlODcFbYoddE/1EVCOlGIw7wGf4OAdTkoONe9rjPg83HO/6iH7vDCiPtg874Nq+S5y9m7rhvLWDglrUruNDn6TvZmAIL35Jny2NTx9DJiMXZce4pzyMCYj/QQj7Zf3fOQCx8rpClZolDjM3S9VTrxWXKnxlMxTAd8oWpBMXBtuIyYlzmakd5qK2HSIFfS4F1pADB0Jayt08tU6XIkKJlQRqhGrq3Tgdep2DwjozYg/GhYBzNLMsJBRejQrvFTpU7BdOg+0e3e0cNcgwaZYeamCGDTaVLsxumT/taPTrUJGI85M3I5pWR/YUNBO/Di0Z62CqpAIwctYAd1jFajZSwB79U10P2A9/ejnr4iTCY3YXmyZXefUp4TrXil8sfVeA==
Message-ID: <805b91bd-e341-4dd4-aa69-867c86bc4eb1@suse.com>
Date: Mon, 5 Feb 2024 15:30:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 10/15] xen: add cache coloring allocator for domains
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-11-carlo.nonato@minervasys.tech>
 <1739dbb4-5d7e-4d84-8d4b-bfb13c4041d8@suse.com>
 <CAG+AhRUJysNCrUXLiu8tvh2d7oL2hNAtkM3A2-Hy73f=+PxkTA@mail.gmail.com>
 <0ef140da-77d3-411f-8f19-eee0aaf1fa9d@suse.com>
 <CAG+AhRVTpcEHpWVY1qUNt3Uc-cDPqV2ZRe8rwfvY4YZUPUP7Ew@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRVTpcEHpWVY1qUNt3Uc-cDPqV2ZRe8rwfvY4YZUPUP7Ew@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 05.02.2024 15:10, Carlo Nonato wrote:
> On Mon, Feb 5, 2024 at 1:38 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 05.02.2024 12:59, Carlo Nonato wrote:
>>> On Thu, Feb 1, 2024 at 4:53 PM Jan Beulich <jbeulich@suse.com> wrote:
>>>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>>>> @@ -2458,7 +2626,14 @@ struct page_info *alloc_domheap_pages(
>>>>>      if ( memflags & MEMF_no_owner )
>>>>>          memflags |= MEMF_no_refcount;
>>>>>
>>>>> -    if ( !dma_bitsize )
>>>>> +    /* Only domains are supported for coloring */
>>>>> +    if ( d && llc_coloring_enabled )
>>>>> +    {
>>>>> +        /* Colored allocation must be done on 0 order */
>>>>> +        if ( order || (pg = alloc_color_heap_page(memflags, d)) == NULL )
>>>>> +            return NULL;
>>>>> +    }
>>>>
>>>> I think I had asked before: What about MEMF_node() or MEMF_bits()
>>>> having been used by the caller?
>>>
>>> MEMF_node() is used for NUMA, right? I think that for the moment, since cache
>>> coloring is supported only on arm64 and NUMA is not yet supported, it's safe
>>> to ignore it.
>>
>> NUMA or not, I'm wary of silent ignoring of inputs. I'd rather request
>> that, just like with staticmem, precautions are then taken to avoid
>> coloring and NUMA to be used together. Recall that much like your work,
>> work to support NUMA is also in progress (unless it was canceled, but
>> not so announced).
> 
> Ok.
> 
>>> I'm not sure I understood what MEMF_bits() is for. It restricts the allocator
>>> to return pages in some range, right?
>>
>> Correct.
> 
> So I need to think about this, since I see no easy way of supporting such a
> thing with the current colored allocator implementation.
> Do you have any suggestions?

Well, at least properly fail requests you can't fulfill.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:33:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:33:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676144.1052042 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX02H-0004gQ-5d; Mon, 05 Feb 2024 14:33:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676144.1052042; Mon, 05 Feb 2024 14:33:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX02H-0004gJ-2n; Mon, 05 Feb 2024 14:33:25 +0000
Received: by outflank-mailman (input) for mailman id 676144;
 Mon, 05 Feb 2024 14:33:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rX02G-0004gD-DA
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:33:24 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX02F-0007Tc-Mr; Mon, 05 Feb 2024 14:33:23 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX02F-0008AZ-GI; Mon, 05 Feb 2024 14:33:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=cfHHG6y2PPIcHDovtO25KneISISlD2FDvZ7cmXCylRM=; b=2R+ea/fBeNC8Wbjp2TZ5mhyjcJ
	00bxsBW90531HFCNPvUegh9zSgbZenjuqowAi5tpGbdL8+B5G3QkpSMlViuKKeN4TEdpy1iZXPYL6
	YHNQ0CFXU9afYx/rdBM9jILdOEeGlcFJ39XXjp1GFv8NZ9xhn4dMscZjJBgqDWa29vE0=;
Message-ID: <c3bde61e-58f1-40fe-afa0-23510b6c5720@xen.org>
Date: Mon, 5 Feb 2024 14:33:22 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 23/32] tools/xenstored: move all log-pipe handling into
 posix.c
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240205105001.24171-1-jgross@suse.com>
 <20240205105001.24171-24-jgross@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240205105001.24171-24-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Juergen,

On 05/02/2024 10:49, Juergen Gross wrote:
> All of the log-pipe handling is needed only when running as daemon.
> 
> Move it into posix.c. This requires to have a service function in the
> main event loop for handling the related requests and one for setting
> the fds[] array. Use a generic name for those functions, as socket
> handling can be added to them later, too.

I would mention the rename in the commit message. With that:

> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:33:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:33:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676146.1052052 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX02o-00059J-DW; Mon, 05 Feb 2024 14:33:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676146.1052052; Mon, 05 Feb 2024 14:33:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX02o-00059C-Ap; Mon, 05 Feb 2024 14:33:58 +0000
Received: by outflank-mailman (input) for mailman id 676146;
 Mon, 05 Feb 2024 14:33:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rX02n-00058n-4k
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:33:57 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 97221ef5-c433-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 15:33:55 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-40fcb4ef7bcso24670545e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 06:33:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 u7-20020a05600c19c700b0040fdf2832desm45036wmq.12.2024.02.05.06.33.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 06:33:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97221ef5-c433-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707143635; x=1707748435; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2dX9ddEHESpBoftdrS/QI0RH29LBWiehjXMlgyAI5HM=;
        b=XZ4AT4jjjeEoxdzkIQo2RNULIN0YEvqRZgRIYRRSSGuUNXBeqiZAQsG5pUwR7WthT1
         aVpuIsLZkTrqGtLrd8wwxNak718N5ViHlqVebE5o3ys07tugDLUBEpoCJid4bXzEfN0H
         cp1uRmJ9y+FK4AyW/0DqunJDi8P5kVr+gGWqn00eGVs1L64qmzraLQW+h6o2h1itCFKe
         4kE50NM0QwFInVCL+aq6cyqa6NpMJdfiFjR8qwk2q5Oq6KVk1FzuijL8c49ysksEFPei
         XdnY4Inm2BwXmSimXCccoz3A02KNLVBiqk1S40EayRoad2R1sTNjkQnUkd40h4xWjasg
         Q3OQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707143635; x=1707748435;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2dX9ddEHESpBoftdrS/QI0RH29LBWiehjXMlgyAI5HM=;
        b=m88DcQhBBUDmc3eb6taBFDrhoEmGWESWKTsMITlqscnBkD1nkdYYRazPwloel/GuFS
         vJlM9nCq7xmpqjeynWCyuUBwejtHRmBaD7jeDcip3Ye37EWpIiqoC/ZJOND3ms0Vl3YI
         iT3BJN0i4EqA9DmC/sqQv13hOWwbuuzaakco+rAgX4tJKt6dX997kQtDqd4W+P27Xd8R
         eNO6+Yipw4BHvBGJx4vDZdWigltkokYiS/0l0gEEkn6/Wn7+07FJU4hFXR93ySTD7inu
         L6mNuqOzA4TRmD1itDiDCW9tRTySyUl3WOlXTjMq/XZtbthQ2VZBd8FnTcv3oj2fi7O1
         fRFA==
X-Gm-Message-State: AOJu0YzGOifE+rUfleZjGAR2JPTKP4zwAXZfE2wtoZcBDFN/P3wY/ioB
	PTuwISnJDx1i32MxFWiWfepRKtiZZNVXoMVVgaI4eiab3vWph1xj7PkMwtsTtA==
X-Google-Smtp-Source: AGHT+IF+ClaJg8wn5gO1z2LJirSxIydyHYsBLPMQ14HStRGq8HyvamZDMD90NX/0M4MVpRc9fEGw2A==
X-Received: by 2002:a05:600c:35c7:b0:40f:bd81:e738 with SMTP id r7-20020a05600c35c700b0040fbd81e738mr4561032wmq.29.1707143634911;
        Mon, 05 Feb 2024 06:33:54 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCU4oPKQY8qPfoUlFJzCVqNly3VUDRbd8wKWqKUuBfeEM6lIkkDUj8lC8nZXQ30Jd1f3IaM52dxe0mIPjiYaYjmpCXRWsrJfT9ymTgXqtyF1x00HYMbW0UDF5vQ=
Message-ID: <cb716326-4a1c-44ce-8d00-ec46cb112242@suse.com>
Date: Mon, 5 Feb 2024 15:33:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpu: Fix mixed tabs/spaces
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240205141441.1880188-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240205141441.1880188-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 15:14, Andrew Cooper wrote:
> Fixes: 660f8a75013c ("x86/CPU: convert vendor hook invocations to altcall")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Without the (or at least this very) Fixes: tag
Acked-by: Jan Beulich <jbeulich@suse.com>
The patch you mention didn't touch the padding at all. (It also didn't
correct the prior inconsistency, but that's mainly because I didn't
even notice.)

Jan

> --- a/xen/arch/x86/cpu/common.c
> +++ b/xen/arch/x86/cpu/common.c
> @@ -336,8 +336,8 @@ void __init early_cpu_init(bool verbose)
>  
>  	c->x86_vendor = x86_cpuid_lookup_vendor(ebx, ecx, edx);
>  	switch (c->x86_vendor) {
> -	case X86_VENDOR_INTEL:	  actual_cpu = intel_cpu_dev;    break;
> -	case X86_VENDOR_AMD:	  actual_cpu = amd_cpu_dev;      break;
> +	case X86_VENDOR_INTEL:    actual_cpu = intel_cpu_dev;    break;
> +	case X86_VENDOR_AMD:      actual_cpu = amd_cpu_dev;      break;
>  	case X86_VENDOR_CENTAUR:  actual_cpu = centaur_cpu_dev;  break;
>  	case X86_VENDOR_SHANGHAI: actual_cpu = shanghai_cpu_dev; break;
>  	case X86_VENDOR_HYGON:    actual_cpu = hygon_cpu_dev;    break;
> 
> base-commit: 9eee78988ca081eeed0a69fc9773e2e4268d1ed7



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:37:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:37:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676150.1052063 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX05f-0006AH-Qh; Mon, 05 Feb 2024 14:36:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676150.1052063; Mon, 05 Feb 2024 14:36:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX05f-0006AA-Nn; Mon, 05 Feb 2024 14:36:55 +0000
Received: by outflank-mailman (input) for mailman id 676150;
 Mon, 05 Feb 2024 14:36:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=b4fC=JO=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rX05e-00069x-KV
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:36:54 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 01413d4d-c434-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 15:36:53 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 133CB21FCE;
 Mon,  5 Feb 2024 14:36:53 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DFE65132DD;
 Mon,  5 Feb 2024 14:36:52 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id HzPINITywGUScQAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 05 Feb 2024 14:36:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01413d4d-c434-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707143813; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=nwoD9Zy7keDnO3fVr14IzGz2VJiJA8OCau4j8AlUMVE=;
	b=Ity8oxRJeiIR9cSX244Q288KEbMeW1AsJLiKOW/4opzpgLd7fE+gUflasBoGnVua3gKt0c
	8TBEz27yLDR45Ll05XYD579JuKDaVzvIRzb4hStSe58qQtCxz3OysdExbJsgTxa65o5pWF
	j5qfto1uj1dIiIxMDxjsm7QRnsGn94g=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707143813; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=nwoD9Zy7keDnO3fVr14IzGz2VJiJA8OCau4j8AlUMVE=;
	b=Ity8oxRJeiIR9cSX244Q288KEbMeW1AsJLiKOW/4opzpgLd7fE+gUflasBoGnVua3gKt0c
	8TBEz27yLDR45Ll05XYD579JuKDaVzvIRzb4hStSe58qQtCxz3OysdExbJsgTxa65o5pWF
	j5qfto1uj1dIiIxMDxjsm7QRnsGn94g=
Message-ID: <51b8f1f0-418d-42ba-bc69-29eb6a6960e2@suse.com>
Date: Mon, 5 Feb 2024 15:36:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 23/32] tools/xenstored: move all log-pipe handling into
 posix.c
Content-Language: en-US
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240205105001.24171-1-jgross@suse.com>
 <20240205105001.24171-24-jgross@suse.com>
 <c3bde61e-58f1-40fe-afa0-23510b6c5720@xen.org>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <c3bde61e-58f1-40fe-afa0-23510b6c5720@xen.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=Ity8oxRJ
X-Spamd-Result: default: False [-1.51 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BAYES_HAM(-1.83)[94.03%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.63)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 133CB21FCE
X-Spam-Level: 
X-Spam-Score: -1.51
X-Spam-Flag: NO

On 05.02.24 15:33, Julien Grall wrote:
> Hi Juergen,
> 
> On 05/02/2024 10:49, Juergen Gross wrote:
>> All of the log-pipe handling is needed only when running as daemon.
>>
>> Move it into posix.c. This requires to have a service function in the
>> main event loop for handling the related requests and one for setting
>> the fds[] array. Use a generic name for those functions, as socket
>> handling can be added to them later, too.
> 
> I would mention the rename in the commit message. With that:

I have modified the commit message to:

tools/xenstored: move all log-pipe handling into posix.c

All of the log-pipe handling is needed only when running as daemon.

Move it into posix.c. This requires to have a service function in the
main event loop for handling the related requests and one for setting
the fds[] array, which is renamed to poll_fds to have a more specific
name. Use a generic name for those functions, as socket handling can
be added to them later, too.

> 
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> 
> Reviewed-by: Julien Grall <jgrall@amazon.com>

Thanks,


Juergen



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:39:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:39:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676153.1052074 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX08c-0007aS-9e; Mon, 05 Feb 2024 14:39:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676153.1052074; Mon, 05 Feb 2024 14:39:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX08c-0007aL-4X; Mon, 05 Feb 2024 14:39:58 +0000
Received: by outflank-mailman (input) for mailman id 676153;
 Mon, 05 Feb 2024 14:39:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rX08a-0007aF-E6
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:39:56 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX08Z-0007ce-Nt; Mon, 05 Feb 2024 14:39:55 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX08Z-0008Vn-Gq; Mon, 05 Feb 2024 14:39:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=3v2B+1bf2u8wUgFhFX2ioDgMbxwNW2MJi8f5Ed7apnQ=; b=XbqBgPSWBCZGGMd94Dq+2V7KGI
	JqFtKs+cvIf1a7TCL+l4looSP2uhdt9pmzdH3Wy3k8lKKdaDQHGmhdEx5MDFDkOmGg6KI8uBg76eN
	aQLxtLdJ8f1qI2IDf9uVW3QjXrYQuiIVl/EaFnXvQzl7AkwKljpUV4UaJBzu8EosjEA4=;
Message-ID: <cee3522d-6f33-4157-8e96-e179820c9392@xen.org>
Date: Mon, 5 Feb 2024 14:39:54 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 24/32] tools/xenstored: move all socket handling into
 posix.c
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240205105001.24171-1-jgross@suse.com>
 <20240205105001.24171-25-jgross@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240205105001.24171-25-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Juergen,

On 05/02/2024 10:49, Juergen Gross wrote:
> All of the socket handling is needed only when running as daemon.
> 
> Move it into posix.c, allowing to remove the NO_SOCKETS macro.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:40:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:40:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676154.1052083 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX08l-0000L0-Hy; Mon, 05 Feb 2024 14:40:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676154.1052083; Mon, 05 Feb 2024 14:40:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX08l-0000Kq-EH; Mon, 05 Feb 2024 14:40:07 +0000
Received: by outflank-mailman (input) for mailman id 676154;
 Mon, 05 Feb 2024 14:40:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jdDv=JO=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1rX08k-00007n-5v
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:40:06 +0000
Received: from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com
 [2607:f8b0:4864:20::22f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 722d3d30-c434-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 15:40:03 +0100 (CET)
Received: by mail-oi1-x22f.google.com with SMTP id
 5614622812f47-3bba0ac2e88so3383059b6e.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 06:40:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 722d3d30-c434-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1707144002; x=1707748802; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8GtgUTwFFp/baIi5Y1KiC1kr4cfbXr3t2oWJQI/zDrk=;
        b=SbwVMhRJYyiitIdM3sHZvwDtZ/Fd9ly3qI4WTm8dpQEnFrUAOFTzDjzbkaGjZBi9pI
         WgTemaO/cgeaW9VlQ695sVy8RMkeLeehr8vEHKfAnNuCyWjreBluCKOh/5NtAPuWv/lm
         UK8aosv9oh5kW9LsfMVaGCptU/dYpFQJHbIb7jgBHI4NLXXBFimPJKVbOGCvRd+m43a1
         fKhiVEk6ZIH1f2VR9YcMvB4EemKv1sHtcmYVYqLHkRRVJCK5CHBmTQlisGRp8I52gXi7
         FHE7bKJs71TZyauiFBAKCF1d6eqlQxmomICWjbp3d7x3GCZMWwzetzS5acULgHdgL84c
         kCeQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707144002; x=1707748802;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8GtgUTwFFp/baIi5Y1KiC1kr4cfbXr3t2oWJQI/zDrk=;
        b=t58eHwTG99LWrqD3GF9wT8nVLrHcJ8WEgol77RwzBDyg4ZShMNcm4hJ3M68KCpCuTS
         lvQc80qfj2ffQywFypIGR4oI+Mmd9XrzhYBP7p3wASJZsIRYdrqD8qTVbmyCSD639Pnl
         qV9QxIhutW7Horp4Y2EibpOpGSUFYBvku0GHCBGOZAioOc38XR4VJYbc1CRKQJWzU1fu
         Ju0q1QwysR63zb77v+HPjFppJ2aZJcWzNZRDVwH/KFx4/Snf8KuI2yNnSXF9tmO9XzyP
         EizMB3PeU9Zudu6pIhW4xhX5qZ+uw42f0FmuU1webKXOGGQXE0V9poTb6z1jQxmepdVL
         yQHQ==
X-Gm-Message-State: AOJu0YydczPVl9o3w4tkQVqlQ+gAA8a+OX9L/KYmx2JICygHyEYQ61xB
	48aWmWMXGO6dxHs1zcIY/znfeavV2S3lGHm7gRGqO2YkANhK2dbMkyuNzEMd4BeQb1vkPqTB+1r
	Hy1ClPgcJBD6lV9iJFG+rPQvPGHn44+e3CGeCog==
X-Google-Smtp-Source: AGHT+IEmlw7ujFJoWkML+FSgXeBLHZoqiiUph5XzJ67laEdrb2tMjoL5cXvDOQRL5SLbxVGADf2VvPC+GDtFel6khso=
X-Received: by 2002:a05:6870:7e12:b0:219:8075:25d4 with SMTP id
 wx18-20020a0568707e1200b00219807525d4mr1392773oab.13.1707144002141; Mon, 05
 Feb 2024 06:40:02 -0800 (PST)
MIME-Version: 1.0
References: <20240117110618.2183977-1-jens.wiklander@linaro.org> <6980D75D-544A-4A3B-906B-DC27D68A260B@arm.com>
In-Reply-To: <6980D75D-544A-4A3B-906B-DC27D68A260B@arm.com>
From: Jens Wiklander <jens.wiklander@linaro.org>
Date: Mon, 5 Feb 2024 15:39:51 +0100
Message-ID: <CAHUa44GS7hX=7msPgzMestM+Dpre0rjuFs9gOq-WY+c=waxi8g@mail.gmail.com>
Subject: Re: [XEN PATCH v3] xen/arm: ffa: reclaim shared memory on guest destroy
To: Bertrand Marquis <Bertrand.Marquis@arm.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, 
	"patches@linaro.org" <patches@linaro.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
	Michal Orzel <michal.orzel@amd.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Bertrand,

On Thu, Feb 1, 2024 at 2:57=E2=80=AFPM Bertrand Marquis
<Bertrand.Marquis@arm.com> wrote:
>
> Hi Jens,
>
> > On 17 Jan 2024, at 12:06, Jens Wiklander <jens.wiklander@linaro.org> wr=
ote:
> >
> > When an FF-A enabled guest is destroyed it may leave behind memory
> > shared with SPs. This memory must be reclaimed before it's reused or an
> > SP may make changes to memory used by a new unrelated guest. So when th=
e
> > domain is teared down add FF-A requests to reclaim all remaining shared
> > memory.
> >
> > SPs in the secure world are notified using VM_DESTROYED that a guest ha=
s
> > been destroyed. An SP is supposed to relinquish all shared memory to al=
low
> > reclaiming the memory. The relinquish operation may need to be delayed =
if
> > the shared memory is for instance part of a DMA operation.
> >
> > The domain reference counter is increased when the first FF-A shared
> > memory is registered and the counter is decreased again when the last
> > shared memory is reclaimed. If FF-A shared memory registrations remain
> > at the end of of ffa_domain_teardown() a timer is set to try to reclaim
> > the shared memory every second until the memory is reclaimed.
> >
> > A few minor style fixes with a removed empty line here and an added new
> > line there.
> >
> > Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
> > ---
> >
> > v3:
> > - Mentioning in the commit message that there are some style fixes
> > - Addressing review comments
> > - Refactor the ffa_domain_teardown() path to let
> >  ffa_domain_teardown_continue() do most of the work.
> >
> > v2:
> > - Update commit message to match the new implementation
> > - Using a per domain bitfield to keep track of which SPs has been notif=
ied
> >  with VM_DESTROYED
> > - Holding a domain reference counter to keep the domain as a zombie dom=
ain
> >  while there still is shared memory registrations remaining to be recla=
imed
> > - Using a timer to retry reclaiming remaining shared memory registratio=
ns
> > ---
> > xen/arch/arm/tee/ffa.c | 253 +++++++++++++++++++++++++++++++++--------
> > 1 file changed, 204 insertions(+), 49 deletions(-)
> >
> > diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
> > index 0793c1c7585d..80ebbf4f01c6 100644
> > --- a/xen/arch/arm/tee/ffa.c
> > +++ b/xen/arch/arm/tee/ffa.c
> > @@ -54,6 +54,7 @@
> > #include <xen/mm.h>
> > #include <xen/sched.h>
> > #include <xen/sizes.h>
> > +#include <xen/timer.h>
> > #include <xen/types.h>
> >
> > #include <asm/event.h>
> > @@ -144,6 +145,12 @@
> >  */
> > #define FFA_MAX_SHM_COUNT               32
> >
> > +/*
> > + * The time we wait until trying to tear down a domain again if it was
> > + * blocked initially.
> > + */
> > +#define FFA_CTX_TEARDOWN_DELAY          SECONDS(1)
> > +
> > /* FF-A-1.1-REL0 section 10.9.2 Memory region handle, page 167 */
> > #define FFA_HANDLE_HYP_FLAG             BIT(63, ULL)
> > #define FFA_HANDLE_INVALID              0xffffffffffffffffULL
> > @@ -384,11 +391,6 @@ struct ffa_ctx {
> >     unsigned int page_count;
> >     /* FF-A version used by the guest */
> >     uint32_t guest_vers;
> > -    /*
> > -     * Number of SPs that we have sent a VM created signal to, used in
> > -     * ffa_domain_teardown() to know which SPs need to be signalled.
> > -     */
> > -    uint16_t create_signal_count;
> >     bool rx_is_free;
> >     /* Used shared memory objects, struct ffa_shm_mem */
> >     struct list_head shm_list;
> > @@ -402,6 +404,15 @@ struct ffa_ctx {
> >     spinlock_t tx_lock;
> >     spinlock_t rx_lock;
> >     spinlock_t lock;
> > +    /* Used if domain can't be torn down immediately */
> > +    struct domain *teardown_d;
> > +    struct list_head teardown_list;
> > +    s_time_t teardown_expire;
> > +    /*
> > +     * Used for ffa_domain_teardown() to keep track of which SPs shoul=
d be
> > +     * notified that this guest is being destroyed.
> > +     */
> > +    unsigned long vm_destroy_bitmap[];
> > };
> >
> > struct ffa_shm_mem {
> > @@ -436,6 +447,12 @@ static void *ffa_tx __read_mostly;
> > static DEFINE_SPINLOCK(ffa_rx_buffer_lock);
> > static DEFINE_SPINLOCK(ffa_tx_buffer_lock);
> >
> > +
> > +/* Used to track domains that could not be torn down immediately. */
> > +static struct timer ffa_teardown_timer;
> > +static struct list_head ffa_teardown_head;
> > +static DEFINE_SPINLOCK(ffa_teardown_lock);
> > +
> > static bool ffa_get_version(uint32_t *vers)
> > {
> >     const struct arm_smccc_1_2_regs arg =3D {
> > @@ -853,7 +870,6 @@ static int32_t handle_partition_info_get(uint32_t w=
1, uint32_t w2, uint32_t w3,
> >             goto out_rx_release;
> >         }
> >
> > -
> >         memcpy(ctx->rx, ffa_rx, sz);
> >     }
> >     ctx->rx_is_free =3D false;
> > @@ -992,53 +1008,75 @@ static void put_shm_pages(struct ffa_shm_mem *sh=
m)
> >     }
> > }
> >
> > -static bool inc_ctx_shm_count(struct ffa_ctx *ctx)
> > +static bool inc_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
> > {
> >     bool ret =3D true;
> >
> >     spin_lock(&ctx->lock);
> > +
> > +    /*
> > +     * If this is the first shm added, increase the domain reference
> > +     * counter as we need to keep domain around a bit longer to reclai=
m the
> > +     * shared memory in the teardown path.
> > +     */
> > +    if ( !ctx->shm_count )
> > +        get_knownalive_domain(d);
> > +
> >     if (ctx->shm_count >=3D FFA_MAX_SHM_COUNT)
> >         ret =3D false;
> >     else
> >         ctx->shm_count++;
> > +
> >     spin_unlock(&ctx->lock);
> >
> >     return ret;
> > }
> >
> > -static void dec_ctx_shm_count(struct ffa_ctx *ctx)
> > +static void dec_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
> > {
> >     spin_lock(&ctx->lock);
> > +
> >     ASSERT(ctx->shm_count > 0);
> >     ctx->shm_count--;
> > +
> > +    /*
> > +     * If this was the last shm removed, let go of the domain referenc=
e we
> > +     * took in inc_ctx_shm_count() above.
> > +     */
> > +    if ( !ctx->shm_count )
> > +        put_domain(d);
> > +
> >     spin_unlock(&ctx->lock);
> > }
> >
> > -static struct ffa_shm_mem *alloc_ffa_shm_mem(struct ffa_ctx *ctx,
> > +static struct ffa_shm_mem *alloc_ffa_shm_mem(struct domain *d,
> >                                              unsigned int page_count)
> > {
> > +    struct ffa_ctx *ctx =3D d->arch.tee;
> >     struct ffa_shm_mem *shm;
> >
> >     if ( page_count >=3D FFA_MAX_SHM_PAGE_COUNT )
> >         return NULL;
> > -    if ( !inc_ctx_shm_count(ctx) )
> > +    if ( !inc_ctx_shm_count(d, ctx) )
> >         return NULL;
> >
> >     shm =3D xzalloc_flex_struct(struct ffa_shm_mem, pages, page_count);
> >     if ( shm )
> >         shm->page_count =3D page_count;
> >     else
> > -        dec_ctx_shm_count(ctx);
> > +        dec_ctx_shm_count(d, ctx);
> >
> >     return shm;
> > }
> >
> > -static void free_ffa_shm_mem(struct ffa_ctx *ctx, struct ffa_shm_mem *=
shm)
> > +static void free_ffa_shm_mem(struct domain *d, struct ffa_shm_mem *shm=
)
> > {
> > +    struct ffa_ctx *ctx =3D d->arch.tee;
> > +
> >     if ( !shm )
> >         return;
> >
> > -    dec_ctx_shm_count(ctx);
> > +    dec_ctx_shm_count(d, ctx);
> >     put_shm_pages(shm);
> >     xfree(shm);
> > }
> > @@ -1306,7 +1344,7 @@ static void handle_mem_share(struct cpu_user_regs=
 *regs)
> >         goto out_unlock;
> >     }
> >
> > -    shm =3D alloc_ffa_shm_mem(ctx, page_count);
> > +    shm =3D alloc_ffa_shm_mem(d, page_count);
> >     if ( !shm )
> >     {
> >         ret =3D FFA_RET_NO_MEMORY;
> > @@ -1350,7 +1388,7 @@ static void handle_mem_share(struct cpu_user_regs=
 *regs)
> >
> > out:
> >     if ( ret )
> > -        free_ffa_shm_mem(ctx, shm);
> > +        free_ffa_shm_mem(d, shm);
> > out_unlock:
> >     spin_unlock(&ctx->tx_lock);
> >
> > @@ -1401,7 +1439,7 @@ static int handle_mem_reclaim(uint64_t handle, ui=
nt32_t flags)
> >     }
> >     else
> >     {
> > -        free_ffa_shm_mem(ctx, shm);
> > +        free_ffa_shm_mem(d, shm);
> >     }
> >
> >     return ret;
> > @@ -1486,6 +1524,41 @@ static bool ffa_handle_call(struct cpu_user_regs=
 *regs)
> >     }
> > }
> >
> > +static bool is_in_subscr_list(const uint16_t *subscr, uint16_t start,
> > +                              uint16_t end, uint16_t sp_id)
> > +{
> > +    unsigned int n;
> > +
> > +    for ( n =3D start; n < end; n++ )
> > +    {
> > +        if ( subscr[n] =3D=3D sp_id )
> > +            return true;
> > +    }
> > +
> > +    return false;
> > +}
> > +
> > +static void vm_destroy_bitmap_init(struct ffa_ctx *ctx,
> > +                                   unsigned int create_signal_count)
> > +{
> > +    unsigned int n;
> > +
> > +    for ( n =3D 0; n < subscr_vm_destroyed_count; n++ )
> > +    {
> > +        /*
> > +         * Skip SPs subscribed to the VM created event that never was
> > +         * notified of the VM creation due to an error during
> > +         * ffa_domain_init().
> > +         */
> > +        if ( is_in_subscr_list(subscr_vm_created, create_signal_count,
> > +                               subscr_vm_created_count,
> > +                               subscr_vm_destroyed[n]) )
> > +            continue;
> > +
> > +        set_bit(n, ctx->vm_destroy_bitmap);
> > +    }
> > +}
> > +
> > static int ffa_domain_init(struct domain *d)
> > {
> >     struct ffa_ctx *ctx;
> > @@ -1501,11 +1574,14 @@ static int ffa_domain_init(struct domain *d)
> >     if ( d->domain_id >=3D UINT16_MAX)
> >         return -ERANGE;
> >
> > -    ctx =3D xzalloc(struct ffa_ctx);
> > +    ctx =3D xzalloc_flex_struct(struct ffa_ctx, vm_destroy_bitmap,
> > +                              BITS_TO_LONGS(subscr_vm_destroyed_count)=
);
> >     if ( !ctx )
> >         return -ENOMEM;
> >
> >     d->arch.tee =3D ctx;
> > +    ctx->teardown_d =3D d;
> > +    INIT_LIST_HEAD(&ctx->shm_list);
> >
> >     for ( n =3D 0; n < subscr_vm_created_count; n++ )
> >     {
> > @@ -1515,65 +1591,141 @@ static int ffa_domain_init(struct domain *d)
> >         {
> >             printk(XENLOG_ERR "ffa: Failed to report creation of vm_id =
%u to  %u: res %d\n",
> >                    get_vm_id(d), subscr_vm_created[n], res);
> > -            ctx->create_signal_count =3D n;
> > -            return -EIO;
> > +            break;
> >         }
> >     }
> > -    ctx->create_signal_count =3D subscr_vm_created_count;
> > -
> > -    INIT_LIST_HEAD(&ctx->shm_list);
> > +    vm_destroy_bitmap_init(ctx, n);
> > +    if ( n !=3D subscr_vm_created_count )
> > +        return -EIO;
> >
> >     return 0;
> > }
> >
> > -static bool is_in_subscr_list(const uint16_t *subscr, uint16_t start,
> > -                              uint16_t end, uint16_t sp_id)
> > +static void send_vm_destroyed(struct domain *d)
> > {
> > +    struct ffa_ctx *ctx =3D d->arch.tee;
> >     unsigned int n;
> > +    int32_t res;
> >
> > -    for ( n =3D start; n < end; n++ )
> > +    for ( n =3D 0; n < subscr_vm_destroyed_count; n++ )
> >     {
> > -        if ( subscr[n] =3D=3D sp_id )
> > -            return true;
> > -    }
> > +        if ( !test_bit(n, ctx->vm_destroy_bitmap) )
> > +            continue;
> >
> > -    return false;
> > +        res =3D ffa_direct_req_send_vm(subscr_vm_destroyed[n], get_vm_=
id(d),
> > +                                     FFA_MSG_SEND_VM_DESTROYED);
> > +
>
> I think here we should dinstinguish cases depending on the error received=
:
> - retry: we should keep the bit set and retry later
> - other: unset the bit as there is not much we can do but log it.

Makes sense.

>
> The interrupted case can happen, we handle that already in the direct mes=
sage
> request but we will get back here if that does not work after some retrie=
s so
> putting it in the other category is ok i think.

In the case of FFA_MSG_SEND_VM_DESTROYED is think it makes more sense
to also retry FFA_RET_INTERRUPTED later. To me it seems that there
will be an even better chance for an eventual non-secure interrupt to
be handled if wait a little before retrying. So, if you don't mind,
I'll skip clearing the bit below for both FFA_RET_INTERRUPTED and
FFA_RET_RETRY.

>
> > +        if ( res )
> > +        {
> > +            printk(XENLOG_ERR "%pd: ffa: Failed to report destruction =
of vm_id %u to %u: res %d\n",
> > +                   d, get_vm_id(d), subscr_vm_destroyed[n], res);
> > +        }
> > +        else
> > +        {
> > +            clear_bit(n, ctx->vm_destroy_bitmap);
> > +        }
> > +    }
> > }
> >
> > -/* This function is supposed to undo what ffa_domain_init() has done *=
/
> > -static int ffa_domain_teardown(struct domain *d)
> > +static void reclaim_shms(struct domain *d)
> > {
> >     struct ffa_ctx *ctx =3D d->arch.tee;
> > -    unsigned int n;
> > +    struct ffa_shm_mem *shm, *tmp;
> >     int32_t res;
> >
> > -    if ( !ctx )
> > -        return 0;
> > +    list_for_each_entry_safe(shm, tmp, &ctx->shm_list, list)
> > +    {
> > +        register_t handle_hi;
> > +        register_t handle_lo;
> >
> > -    for ( n =3D 0; n < subscr_vm_destroyed_count; n++ )
> > +        uint64_to_regpair(&handle_hi, &handle_lo, shm->handle);
> > +        res =3D ffa_mem_reclaim(handle_lo, handle_hi, 0);
>
> Same here as for the VM_DESTROYED, there are some error code that we shou=
ld
> not reiterate on as they will also fail next time:
> - invalid parameters: stop and log an error, this will always fail
> - no memory: for now I would say do the same as invalid parameter with a =
comment
>  maybe because we could retry and memory coud be available next time
> - abort: stop and log an error, this is a major fault on the other side
> - denied: retry.

I agree, I'll propose something like that in the v4.

>
> > +        if ( res )
> > +        {
> > +            printk(XENLOG_G_INFO "%pd: ffa: Failed to reclaim handle %=
#lx : %d\n",
> > +                   d, shm->handle, res);
> > +        }
> > +        else
> > +        {
> > +            printk(XENLOG_G_DEBUG "%pd: ffa: Reclaimed handle %#lx\n",
> > +                   d, shm->handle);
> > +            list_del(&shm->list);
> > +            free_ffa_shm_mem(d, shm);
> > +        }
> > +    }
> > +}
> > +
> > +static void ffa_domain_teardown_continue(struct ffa_ctx *ctx, bool fir=
st_time)
> > +{
> > +    struct ffa_ctx *next_ctx =3D NULL;
> > +
> > +    send_vm_destroyed(ctx->teardown_d);
> > +    reclaim_shms(ctx->teardown_d);
> > +
> > +    if ( ctx->shm_count )
> > +    {
>
> If i do understand right you only retry if there are remaining SHM to rec=
laim but what
> if you have destroy messages that were not sent successfully, shouldn't w=
e reiterate
> in that case to send the destroy message again ?

I agree, I'll fix.

Cheers,
Jens

>
> > +        printk(XENLOG_G_INFO "%pd: ffa: Remaining unclaimed handles, r=
etrying\n", ctx->teardown_d);
> > +
> > +        ctx->teardown_expire =3D NOW() + FFA_CTX_TEARDOWN_DELAY;
> > +
> > +        spin_lock(&ffa_teardown_lock);
> > +        list_add_tail(&ctx->teardown_list, &ffa_teardown_head);
> > +        /* Need to set a new timer for the next ctx in line */
> > +        next_ctx =3D list_first_entry(&ffa_teardown_head, struct ffa_c=
tx,
> > +                                    teardown_list);
> > +        spin_unlock(&ffa_teardown_lock);
> > +    }
> > +    else
> >     {
> >         /*
> > -         * Skip SPs subscribed to the VM created event that never was
> > -         * notified of the VM creation due to an error during
> > -         * ffa_domain_init().
> > +         * domain_destroy() might have been called (via put_domain() i=
n
> > +         * reclaim_shms()), so we can't touch the domain structure any=
more.
> >          */
> > -        if ( is_in_subscr_list(subscr_vm_created, ctx->create_signal_c=
ount,
> > -                               subscr_vm_created_count,
> > -                               subscr_vm_destroyed[n]) )
> > -            continue;
> > +        xfree(ctx);
> >
> > -        res =3D ffa_direct_req_send_vm(subscr_vm_destroyed[n], get_vm_=
id(d),
> > -                                     FFA_MSG_SEND_VM_DESTROYED);
> > -
> > -        if ( res )
> > -            printk(XENLOG_ERR "ffa: Failed to report destruction of vm=
_id %u to  %u: res %d\n",
> > -                   get_vm_id(d), subscr_vm_destroyed[n], res);
> > +        /* Only check if there has been a change to the teardown queue=
 */
> > +        if ( !first_time )
> > +        {
> > +            spin_lock(&ffa_teardown_lock);
> > +            next_ctx =3D list_first_entry_or_null(&ffa_teardown_head,
> > +                                                struct ffa_ctx, teardo=
wn_list);
> > +            spin_unlock(&ffa_teardown_lock);
> > +        }
> >     }
> >
> > +    if ( next_ctx )
> > +        set_timer(&ffa_teardown_timer, next_ctx->teardown_expire);
> > +}
> > +
> > +static void ffa_teardown_timer_callback(void *arg)
> > +{
> > +    struct ffa_ctx *ctx;
> > +
> > +    spin_lock(&ffa_teardown_lock);
> > +    ctx =3D list_first_entry_or_null(&ffa_teardown_head, struct ffa_ct=
x,
> > +                                   teardown_list);
> > +    if ( ctx )
> > +        list_del(&ctx->teardown_list);
> > +    spin_unlock(&ffa_teardown_lock);
> > +
> > +    if ( ctx )
> > +        ffa_domain_teardown_continue(ctx, false /* !first_time */);
> > +    else
> > +        printk(XENLOG_G_ERR "%s: teardown list is empty\n", __func__);
> > +}
> > +
> > +/* This function is supposed to undo what ffa_domain_init() has done *=
/
> > +static int ffa_domain_teardown(struct domain *d)
> > +{
> > +    struct ffa_ctx *ctx =3D d->arch.tee;
> > +
> > +    if ( !ctx )
> > +        return 0;
> > +
> >     if ( ctx->rx )
> >         rxtx_unmap(ctx);
> >
> > -    XFREE(d->arch.tee);
> > +    ffa_domain_teardown_continue(ctx, true /* first_time */);
> >
> >     return 0;
> > }
> > @@ -1739,6 +1891,9 @@ static bool ffa_probe(void)
> >     if ( !init_sps() )
> >         goto err_free_ffa_tx;
> >
> > +    INIT_LIST_HEAD(&ffa_teardown_head);
> > +    init_timer(&ffa_teardown_timer, ffa_teardown_timer_callback, NULL,=
 0);
> > +
> >     return true;
> >
> > err_free_ffa_tx:
> > --
> > 2.34.1
> >
>
> Regards
> Bertrand
>
>


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:40:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:40:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676157.1052093 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX099-00014q-Qb; Mon, 05 Feb 2024 14:40:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676157.1052093; Mon, 05 Feb 2024 14:40:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX099-00014j-NP; Mon, 05 Feb 2024 14:40:31 +0000
Received: by outflank-mailman (input) for mailman id 676157;
 Mon, 05 Feb 2024 14:40:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX098-00007n-BM
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:40:30 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8152a711-c434-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 15:40:28 +0100 (CET)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2d09faadba5so20792801fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 06:40:28 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 l8-20020a2e9088000000b002d0932543cdsm940125ljg.88.2024.02.05.06.40.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 06:40:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8152a711-c434-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707144028; x=1707748828; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=XGQD37hKVqUpnFueHaxsgJwTWNtOEI0LC1x5sFXSaTU=;
        b=eCCiPXUCcVdSJBUm3KiYgUxr8KE70fNby5Q1oiN+wRifUg1EzUP2/XSJN76hKNffxu
         XjxHIlLg1SQTFdRFRYNnT1VSLPghT1Vzi9fekxlkp2euMCYLlJl+WNSvl5t//hV569S/
         5eClg042FA7Tcq4bulFPNrTkWlCSa2IXxH+7x6nmYiCo3eE2YnR+2WcgIEn9hCjCcUsQ
         OgVy9XUn0Xf2856dIfr6ir/GDqY2dWIb4RmPctKIUqPXh4yOjVf79SUeBaOzPxUEB8qf
         wLMGuZ9MfymUqRp50DGOkEPTbw3Iezm9IehR5KoTmRXDLqNulJQUEkFwqXY7mT4DopRq
         6LqQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707144028; x=1707748828;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=XGQD37hKVqUpnFueHaxsgJwTWNtOEI0LC1x5sFXSaTU=;
        b=MDkf7pmicPg1Aj6j24PewAck29/a9/2k4MyjdX15sA2sFPLRs+8yzwsNURShaUQxbR
         SQiMbyqKo/SbgNG3EcwB8VC+Yz6ls/XmiIn92wOEn2X7S7JbYhjnONwnLIfabCpyGipn
         xyJJZs/o6k8fOI6U5aovBD/T6n0ItzzkJDf+2oIVYUagsqVSClPbOInLcBuwAawclVt7
         W38XVaPxQdX5zw7kJpPXyU5dqEMdPZV6qfcjbdPBR5VzNv3sIkGjlmM0ozlhovrFX9uT
         YDFxyE0K+HQmV2uv0CAu6Ogqasl5lMgYNKuQSORKflyzusRS5lGM0B3DDCO6V/rvJD4E
         Vgsg==
X-Gm-Message-State: AOJu0YxgSh6WaLaQRsAQvGtfirOi84CAyM+LOYA/JoCDwPpv/VVzBC95
	16jtvA/QOr3/DIres5F7tfFnqg3yHp8g1SpgUM/S/zyVdQoE9Qz5
X-Google-Smtp-Source: AGHT+IEwornWo29oo9AHaoTgL7XfVBWkxNUiNYnSQnortDLO1rVn3k7WNNnwUJXQqP7a8T0cNWc6SA==
X-Received: by 2002:a2e:9bd8:0:b0:2d0:b244:5f09 with SMTP id w24-20020a2e9bd8000000b002d0b2445f09mr1002003ljj.51.1707144027533;
        Mon, 05 Feb 2024 06:40:27 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVTJUlUusCDwKe3Dp8gTrJ031Q4G8JRtHdCfZsCcBsxTMgu5r7MI4+44LRbnFmcLPxK4HUmddJuWRKGXmrshRV/lpsy0uaU+b8DWvsTJOAL/mkN/kqbSHh5upNRx/GDDAJ9EeshUy2Kb5VioXMrB5/GkBy3XYs1rOW86USxfCRAtX4cuk5WeIZMhl4387Wtw/zePXcjkbMXIsAE6EF2H7TyYb792Zasxfeto64rRF2mBcFyfyfSh1EL3H/kAR4BaogwRlIaIh15RlsHHLOn1Ta/4RqTcuoU76jTnvmuex2234N4mNSetnc=
Message-ID: <9f3cd4f8940d46b7698de0cb8038a252058e21f6.camel@gmail.com>
Subject: Re: [PATCH v3 31/34] xen/riscv: add minimal stuff to mm.h to build
 full Xen
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Mon, 05 Feb 2024 15:40:26 +0100
In-Reply-To: <8b71122a-8446-4619-992e-ec2475e602d0@suse.com>
References: <cover.1703255175.git.oleksii.kurochko@gmail.com>
	 <4411f6af38586074b347cd6005f19f9c670faa74.1703255175.git.oleksii.kurochko@gmail.com>
	 <d347c4d9-e93b-4937-8e33-e5fbbdcd6bfb@suse.com>
	 <f3858360fba14a2c5b794bdd0400cd7ab8e66f73.camel@gmail.com>
	 <44a9a92c-ac4c-43d0-bc1e-685818e4331d@suse.com>
	 <211e7d7fb7d52d0bc7dbb12d5619a39f98d5b21c.camel@gmail.com>
	 <8b71122a-8446-4619-992e-ec2475e602d0@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-05 at 15:05 +0100, Jan Beulich wrote:
> On 05.02.2024 13:49, Oleksii wrote:
> > On Mon, 2024-02-05 at 08:46 +0100, Jan Beulich wrote:
> > > On 02.02.2024 18:30, Oleksii wrote:
> > > > On Tue, 2024-01-23 at 14:03 +0100, Jan Beulich wrote:
> > > > > On 22.12.2023 16:13, Oleksii Kurochko wrote:
> > > > > > +/* Convert between Xen-heap virtual addresses and page-
> > > > > > info
> > > > > > structures. */
> > > > > > +static inline struct page_info *virt_to_page(const void
> > > > > > *v)
> > > > > > +{
> > > > > > +=C2=A0=C2=A0=C2=A0 BUG();
> > > > > > +=C2=A0=C2=A0=C2=A0 return NULL;
> > > > > > +}
> > > > > > +
> > > > > > +/*
> > > > > > + * We define non-underscored wrappers for above conversion
> > > > > > functions.
> > > > > > + * These are overriden in various source files while
> > > > > > underscored
> > > > > > version
> > > > > > + * remain intact.
> > > > > > + */
> > > > > > +#define virt_to_mfn(va)=C2=A0=C2=A0=C2=A0=C2=A0 __virt_to_mfn(=
va)
> > > > > > +#define mfn_to_virt(mfn)=C2=A0=C2=A0=C2=A0 __mfn_to_virt(mfn)
> > > > >=20
> > > > > Is this really still needed? Would be pretty nice if in a new
> > > > > port we
> > > > > could get to start cleanly right away (i.e. by not needing
> > > > > per-
> > > > > file
> > > > > overrides, but using type-safe expansions here right away).
> > > > We still need __virt_to_mfn and __mfn_to_virt as common code
> > > > use
> > > > them:
> > > > =C2=A0* xen/common/xenoprof.c:24:#define virt_to_mfn(va)
> > > > mfn(__virt_to_mfn(va))
> > >=20
> > > Are you meaning to enable this code on RISC-V.
> > Yes, that is what I meant.
>=20
> And why would you do that? Even Arm doesn't use it, and I'd expect no
> newer port would care about this very old interface.
Oh, sorry, I read your question wrongly. I am not going to enable that
config.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 14:41:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 14:41:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676162.1052102 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX09l-0001je-2a; Mon, 05 Feb 2024 14:41:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676162.1052102; Mon, 05 Feb 2024 14:41:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX09k-0001jV-W5; Mon, 05 Feb 2024 14:41:08 +0000
Received: by outflank-mailman (input) for mailman id 676162;
 Mon, 05 Feb 2024 14:41:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rX09j-0001i1-JZ
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 14:41:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX09j-0007eV-20; Mon, 05 Feb 2024 14:41:07 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX09i-0008Vn-Ta; Mon, 05 Feb 2024 14:41:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=z5IRWa2bwMWZXQZNpLSVFvglGaXNH6tk5BGioab6tM0=; b=tLrFU3SdoJLNc+520L+HBCr3LK
	nQM7YKgBpD4MRQ5bYaL8Fr6igEuoLYyKKhnJXNHMtpaNhCID1LWxm/rLY8E4awBFclB99fyCC+BKR
	SMSb2gPrajgndcxLm+aKoT6I3GYodTxfK8jE4veAa2G07mLAQL2s9r002EN89oybGtVs=;
Message-ID: <484f6649-4720-4108-ad2d-8b729e7352fd@xen.org>
Date: Mon, 5 Feb 2024 14:41:06 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 23/32] tools/xenstored: move all log-pipe handling into
 posix.c
Content-Language: en-GB
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240205105001.24171-1-jgross@suse.com>
 <20240205105001.24171-24-jgross@suse.com>
 <c3bde61e-58f1-40fe-afa0-23510b6c5720@xen.org>
 <51b8f1f0-418d-42ba-bc69-29eb6a6960e2@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <51b8f1f0-418d-42ba-bc69-29eb6a6960e2@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Juergen,

On 05/02/2024 14:36, Jürgen Groß wrote:
> On 05.02.24 15:33, Julien Grall wrote:
>> Hi Juergen,
>>
>> On 05/02/2024 10:49, Juergen Gross wrote:
>>> All of the log-pipe handling is needed only when running as daemon.
>>>
>>> Move it into posix.c. This requires to have a service function in the
>>> main event loop for handling the related requests and one for setting
>>> the fds[] array. Use a generic name for those functions, as socket
>>> handling can be added to them later, too.
>>
>> I would mention the rename in the commit message. With that:
> 
> I have modified the commit message to:
> 
> tools/xenstored: move all log-pipe handling into posix.c
> 
> All of the log-pipe handling is needed only when running as daemon.
> 
> Move it into posix.c. This requires to have a service function in the
> main event loop for handling the related requests and one for setting
> the fds[] array, which is renamed to poll_fds to have a more specific
> name. Use a generic name for those functions, as socket handling can
> be added to them later, too.

AFAICT, the rest of the patches have been reviewed by either Jason or I. 
So I will update the commit message while committing.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:11:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:11:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676175.1052112 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0cl-0000ii-GV; Mon, 05 Feb 2024 15:11:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676175.1052112; Mon, 05 Feb 2024 15:11:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0cl-0000ib-Dp; Mon, 05 Feb 2024 15:11:07 +0000
Received: by outflank-mailman (input) for mailman id 676175;
 Mon, 05 Feb 2024 15:11:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+Hmw=JO=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rX0ck-0000iV-2u
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:11:06 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c5d10498-c438-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:11:01 +0100 (CET)
Received: by mail-wr1-x42c.google.com with SMTP id
 ffacd0b85a97d-33b29b5ea86so1550440f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:11:01 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 j17-20020a05600c1c1100b0040fbdd6f69bsm8975640wms.33.2024.02.05.07.11.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:11:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5d10498-c438-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707145861; x=1707750661; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=9rYXsl2v6e1LdNb7inNZK9lMEFl6jRhiXD/5FjT+twc=;
        b=pzS9EYaqXPKTaSfHNqJe3gYM2FgMQsBWNG0E6UzXzgrwFoyqRUOv9lJJbg1A2t28tP
         VdGo/RHoiCWM0xTYjaegXgXZKzGTgGEenfamMTHGhB3IJxgqXthed610TfpLWbtW/Lcn
         oNpWu4u3jCj2VrNYoDPPqJYMugjnmAxRYm7DM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707145861; x=1707750661;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9rYXsl2v6e1LdNb7inNZK9lMEFl6jRhiXD/5FjT+twc=;
        b=U/lr4rLZuKTAqeHaeqpc0A0R5TkYxYvD8H8fo6zzue/SBAy5Gu1KCx/WrDY5/Pdp4/
         l9UxvXRkMhn63BebaGyuFQLDg+xIct2TnP4VOvqoADNy0ou7RENqB3E189LBFqOlh6go
         GCdN3kbIvBxn/bqMWGt6js4GBavTpENmJuGlg/nXZeCKmwmDa/FqTOfZVUWo8/33WVX9
         wZphTrPVGLsuyRH41T/iPdjRTg7GKB4CYbafX/3Cq7i9HhAsy67y0FG8LHmq7PobuOXf
         5Vw6t3RYtC8GTiqcSp0SiO7yaQbjt2go9/pm71PiitzcxGS1iq4KEM2pVbCpgIrXQC20
         NMbQ==
X-Gm-Message-State: AOJu0Yzin5xDHGDP1NsheC0n70kCAI1WKnmi826PBm0eSk9thGFr/c2R
	8icSQ51RquPzQihL7Dmk68GsHKvQ4gaP1s/GtsiM+8X/REOhhECmQy0DMy2VNIk=
X-Google-Smtp-Source: AGHT+IGxjpGAeWr26eoG7MKtxebnMikpGshyIMfh7v1Oh6sv5tvIdl96unujFnfOg5/A7vjB0BkAfg==
X-Received: by 2002:adf:f044:0:b0:33b:2262:ca61 with SMTP id t4-20020adff044000000b0033b2262ca61mr5997366wro.70.1707145860620;
        Mon, 05 Feb 2024 07:11:00 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVy61Gxs3FiqTm0ZmMJt8xgVOrn6Whl0g6mMG4YO9xX/Yldbr7K5uVy37k/nVJ1VnkfXarNnTPcY4IXJ4w7hxNQB+ridQm0xv0X51qLxYfUGzB1M1SM5CLntuCR6Yc2RvZNhj+Gbrskz/0K4KZJygv3ZoFk1csJt2+XJjnG0f6y0fIuw2hVj/J/6ZHbAIx0sasKwR0Qk3gdSZzQjuGvY61+0O+MHGzA2hNcTFdnj6B19vjL
Date: Mon, 5 Feb 2024 15:10:59 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Juergen Gross <jgross@suse.com>,
	Viresh Kumar <viresh.kumar@linaro.org>
Subject: Re: [PATCH] libxl: Add "grant_usage" parameter for virtio disk
 devices
Message-ID: <ace5b3c8-38e7-4661-9401-ac9ac77a5e9f@perard>
References: <20240202104903.1112772-1-olekstysh@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240202104903.1112772-1-olekstysh@gmail.com>

On Fri, Feb 02, 2024 at 12:49:03PM +0200, Oleksandr Tyshchenko wrote:
> diff --git a/tools/libs/util/libxlu_disk_l.l b/tools/libs/util/libxlu_disk_l.l
> index 6d53c093a3..f37dd443bd 100644
> --- a/tools/libs/util/libxlu_disk_l.l
> +++ b/tools/libs/util/libxlu_disk_l.l
> @@ -220,6 +220,9 @@ hidden-disk=[^,]*,?	{ STRIP(','); SAVESTRING("hidden-disk", hidden_disk, FROMEQU
>  trusted,?		{ libxl_defbool_set(&DPC->disk->trusted, true); }
>  untrusted,?		{ libxl_defbool_set(&DPC->disk->trusted, false); }
>  
> +grant_usage=1,?		{ libxl_defbool_set(&DPC->disk->grant_usage, true); }
> +grant_usage=0,?		{ libxl_defbool_set(&DPC->disk->grant_usage, false); }

For other boolean type for the disk, we have "trusted/untrusted",
"discard/no-discard", "direct-io-save/", but you are adding
"grant_usage=1/grant_usage=0". Is that fine? But I guess having the new
option spelled "grant_usage" might be better, so it match the other
virtio devices and the implementation. But maybe
"use-grant/no-use-grant" might be ok?

In any case, the implementation need to match the documentation, and
vice versa. See below.

> diff --git a/docs/man/xl-disk-configuration.5.pod.in b/docs/man/xl-disk-configuration.5.pod.in
> index bc945cc517..3c035456d5 100644
> --- a/docs/man/xl-disk-configuration.5.pod.in
> +++ b/docs/man/xl-disk-configuration.5.pod.in
> @@ -404,6 +404,31 @@ Virtio frontend driver (virtio-blk) to be used. Please note, the virtual
> +=item B<grant_usage=BOOLEAN>
>
> +=over 4
> +
> +=item Description
> +
> +Specifies the usage of Xen grants for accessing guest memory. Only applicable
> +to specification "virtio".
> +
> +=item Supported values
> +
> +If this option is B<true>, the Xen grants are always enabled.
> +If this option is B<false>, the Xen grants are always disabled.

Unfortunately, this is wrong, the implementation in the patch only
support two values: 1 / 0, nothing else, and trying to write "true" or
"false" would lead to an error. (Well actually it's "grant_usage=1" or
"grant_usage=0", there's nothing that cut that string at the '='.)

Also, do we really need the extra verbal description of each value here?
Is simply having the following would be enough?

    =item Supported values

    1, 0

The description in "Description" section would hopefully be enough.

> +=item Mandatory
> +
> +No
> +
> +=item Default value
> +
> +If this option is missing, then the default grant setting will be used,
> +i.e. enable grants if backend-domid != 0.
> +
> +=back
> +
> diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
> index ea3623dd6f..f39f427091 100644
> --- a/tools/libs/light/libxl_disk.c
> +++ b/tools/libs/light/libxl_disk.c
> @@ -181,6 +181,9 @@ static int libxl__device_disk_setdefault(libxl__gc *gc, uint32_t domid,
>              return ERROR_INVAL;
>          }
>          disk->transport = LIBXL_DISK_TRANSPORT_MMIO;
> +
> +        libxl_defbool_setdefault(&disk->grant_usage,
> +                                 disk->backend_domid != LIBXL_TOOLSTACK_DOMID);
>      }
>  
>      if (hotplug && disk->specification == LIBXL_DISK_SPECIFICATION_VIRTIO) {
> @@ -429,6 +432,8 @@ static void device_disk_add(libxl__egc *egc, uint32_t domid,
>              flexarray_append(back, libxl__device_disk_string_of_transport(disk->transport));
>              flexarray_append_pair(back, "base", GCSPRINTF("%"PRIu64, disk->base));
>              flexarray_append_pair(back, "irq", GCSPRINTF("%u", disk->irq));
> +            flexarray_append_pair(back, "grant_usage",
> +                                  libxl_defbool_val(disk->grant_usage) ? "1" : "0");
>          }
>  
>          flexarray_append(front, "backend-id");
> @@ -623,6 +628,14 @@ static int libxl__disk_from_xenstore(libxl__gc *gc, const char *libxl_path,
>              goto cleanup;
>          }
>          disk->irq = strtoul(tmp, NULL, 10);
> +
> +        tmp = libxl__xs_read(gc, XBT_NULL,
> +                             GCSPRINTF("%s/grant_usage", libxl_path));
> +        if (!tmp) {
> +            LOG(ERROR, "Missing xenstore node %s/grant_usage", libxl_path);
> +            goto cleanup;

I wonder if it's such a good idea to make this node mandatory. Could we
just apply the default value if the path is missing? I don't think the
value is going to be used anyway because I don't think from_xenstore() is
used during guest creation, and it looks like "grant_usage" is only
useful during guest creation. Also, the "grant_usage" node isn't
mandatory in "libxl_virtio.c", so no need to do something different
for disk.

> +        }
> +        libxl_defbool_set(&disk->grant_usage, strtoul(tmp, NULL, 0));
>      }
>  
>      disk->vdev = xs_read(ctx->xsh, XBT_NULL,

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:14:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:14:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676178.1052123 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0fq-0001p7-V9; Mon, 05 Feb 2024 15:14:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676178.1052123; Mon, 05 Feb 2024 15:14:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0fq-0001p0-Rq; Mon, 05 Feb 2024 15:14:18 +0000
Received: by outflank-mailman (input) for mailman id 676178;
 Mon, 05 Feb 2024 15:14:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W3Sh=JO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rX0fq-0001ou-A4
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:14:18 +0000
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com
 [2a00:1450:4864:20::22f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3a03512b-c439-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:14:16 +0100 (CET)
Received: by mail-lj1-x22f.google.com with SMTP id
 38308e7fff4ca-2cf5917f049so52635011fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:14:16 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 q5-20020a5085c5000000b00557aa8d72c9sm3869905edh.25.2024.02.05.07.14.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:14:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a03512b-c439-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707146055; x=1707750855; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=pZMRWCjeqgfIORlG9fhev9dwrUTk11miJ02JZjWQfQA=;
        b=HsYp2/tGgtr1rxagc3Ojwi8kzQF2mZV6MZxODaWSQC1pF62OsqsqtX8Wd5C3+z6+8U
         pEegMpeURTKLG96qprLP7nrf3sK+cKSHNi3akiTUhgL45v8Ob6Oc0hdWXIobYIep+7Ck
         ZPBk3jgafEHIBUd7/5zeb/A2XNbURbmn/krb0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707146055; x=1707750855;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=pZMRWCjeqgfIORlG9fhev9dwrUTk11miJ02JZjWQfQA=;
        b=Sn35CZ2KUTvaXW2pktrmiwB9c0a6Hy3xB1hFsV/VWw068iAqYl9wNm5RHl/XIZN7U2
         kO07+F8qa8s0QdkDy4QwYVKHasZp9Q7/G+/y4cFLVL6gDKr9XqIxVBKU7mRtAGSgFweK
         a62iDFExr9PpOxia2sAqVBTN5es8k45EGNowfZoNyRcgJnMlKV/JYs85cX/JeOym89i0
         XQcGQTlRXUOnbqQELmq0cX95dXrTnOh/TPzsxEz1BJe2+E3owmNEjDUi011SOk/QWJ3X
         608UoTzkft+C/3mo+j9UO9rEpvpSV9K/79BCiF+gjldSiVUSd6d3Ux3+F+aGi9jSCmW2
         E0Uw==
X-Gm-Message-State: AOJu0Yx+HJkKYyqxyHdkkCrz+WF2RS+Q3Lo/Br3+b2uNy3OpUgmXU2la
	3HJtJ/4ep8T8yOgtersbjllf53M9yfIQJdFxwkUfyJgCFAEksZ8bYdyCihdZkbLINcFmHqOj+Os
	F
X-Google-Smtp-Source: AGHT+IGw6S7F2JTGw+GcAA4LPr5r/0NU8ApHDuQvU6XSTnGEAG0ioEI1i5SVIa7kZ0KnoPSY5M88iQ==
X-Received: by 2002:a05:6512:3e13:b0:511:1fb0:1f71 with SMTP id i19-20020a0565123e1300b005111fb01f71mr12597388lfv.51.1707146055356;
        Mon, 05 Feb 2024 07:14:15 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUIMQCz7/oDivSd0bF2lgC58tsYT9ImErvDVVULeAdU9ZorLPtruSn15aMek96uQYsv3iMnPSHKPaPxvk6HIHqZlEeDDUtcCd+ar62iXhlTwBIHCIyIHSicAfdjMSPjt79OVDtQXcBE81zWXZ6yEnrdbF40OTAGbhj8Vl4VPktJGcPuYoAtXkyBPZgDRubSoA==
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH] x86/bitmap: Even more signed-ness fixes
Date: Mon,  5 Feb 2024 15:14:13 +0000
Message-Id: <20240205151413.1919983-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Further to commit 558e84b7ffec ("xen/bitmap: Consistently use unsigned bits
values"), it turns out that GCC's range analysis isn't good enough to notice
that 'bits / BITS_PER_LONG' is always within the unsigned range of k.

As a consequence, it re-sign-extends 'k' when calculating the pointer holding
'bitmap[k]' on each loop iteration.  Switch all internal variables to unsigned
int, which allows for better-still code generation.

Also clean up the prototypes for __bitmap_{set,clear} and the *_region()
helpers, along with the internal variables.

Bloat-o-meter for a random x86 build reports:
  add/remove: 0/0 grow/shrink: 0/10 up/down: 0/-277 (-277)

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>

Full bloat-o-meter details:

  add/remove: 0/0 grow/shrink: 0/10 up/down: 0/-277 (-277)
  Function                                     old     new   delta
  __bitmap_equal                               105     102      -3
  bitmap_find_free_region                      135     129      -6
  __bitmap_subset                              121     115      -6
  __bitmap_intersects                          105      99      -6
  __bitmap_full                                 91      85      -6
  __bitmap_empty                                90      84      -6
  bitmap_release_region                         68      45     -23
  bitmap_allocate_region                       104      72     -32
  __bitmap_clear                               136      43     -93
  __bitmap_set                                 136      40     -96
  Total: Before=4063985, After=4063708, chg -0.01%

Other functions have changes, but no net change.
---
 xen/common/bitmap.c      | 76 ++++++++++++++++++++++------------------
 xen/include/xen/bitmap.h | 15 ++++----
 2 files changed, 50 insertions(+), 41 deletions(-)

diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c
index 9d9ff09f3dde..18e23e2f0e18 100644
--- a/xen/common/bitmap.c
+++ b/xen/common/bitmap.c
@@ -57,7 +57,8 @@ static void clamp_last_byte(uint8_t *bp, unsigned int nbits)
 
 int __bitmap_empty(const unsigned long *bitmap, unsigned int bits)
 {
-	int k, lim = bits/BITS_PER_LONG;
+	unsigned int k, lim = bits / BITS_PER_LONG;
+
 	for (k = 0; k < lim; ++k)
 		if (bitmap[k])
 			return 0;
@@ -72,7 +73,8 @@ EXPORT_SYMBOL(__bitmap_empty);
 
 int __bitmap_full(const unsigned long *bitmap, unsigned int bits)
 {
-	int k, lim = bits/BITS_PER_LONG;
+	unsigned int k, lim = bits / BITS_PER_LONG;
+
 	for (k = 0; k < lim; ++k)
 		if (~bitmap[k])
 			return 0;
@@ -88,7 +90,8 @@ EXPORT_SYMBOL(__bitmap_full);
 int __bitmap_equal(const unsigned long *bitmap1,
                    const unsigned long *bitmap2, unsigned int bits)
 {
-	int k, lim = bits/BITS_PER_LONG;
+	unsigned int k, lim = bits / BITS_PER_LONG;
+
 	for (k = 0; k < lim; ++k)
 		if (bitmap1[k] != bitmap2[k])
 			return 0;
@@ -103,7 +106,8 @@ EXPORT_SYMBOL(__bitmap_equal);
 
 void __bitmap_complement(unsigned long *dst, const unsigned long *src, unsigned int bits)
 {
-	int k, lim = bits/BITS_PER_LONG;
+	unsigned int k, lim = bits / BITS_PER_LONG;
+
 	for (k = 0; k < lim; ++k)
 		dst[k] = ~src[k];
 
@@ -115,8 +119,7 @@ EXPORT_SYMBOL(__bitmap_complement);
 void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
                   const unsigned long *bitmap2, unsigned int bits)
 {
-	int k;
-	int nr = BITS_TO_LONGS(bits);
+	unsigned int k, nr = BITS_TO_LONGS(bits);
 
 	for (k = 0; k < nr; k++)
 		dst[k] = bitmap1[k] & bitmap2[k];
@@ -126,8 +129,7 @@ EXPORT_SYMBOL(__bitmap_and);
 void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
                  const unsigned long *bitmap2, unsigned int bits)
 {
-	int k;
-	int nr = BITS_TO_LONGS(bits);
+	unsigned int k, nr = BITS_TO_LONGS(bits);
 
 	for (k = 0; k < nr; k++)
 		dst[k] = bitmap1[k] | bitmap2[k];
@@ -137,8 +139,7 @@ EXPORT_SYMBOL(__bitmap_or);
 void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
                   const unsigned long *bitmap2, unsigned int bits)
 {
-	int k;
-	int nr = BITS_TO_LONGS(bits);
+	unsigned int k, nr = BITS_TO_LONGS(bits);
 
 	for (k = 0; k < nr; k++)
 		dst[k] = bitmap1[k] ^ bitmap2[k];
@@ -148,8 +149,7 @@ EXPORT_SYMBOL(__bitmap_xor);
 void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
                      const unsigned long *bitmap2, unsigned int bits)
 {
-	int k;
-	int nr = BITS_TO_LONGS(bits);
+	unsigned int k, nr = BITS_TO_LONGS(bits);
 
 	for (k = 0; k < nr; k++)
 		dst[k] = bitmap1[k] & ~bitmap2[k];
@@ -159,7 +159,8 @@ EXPORT_SYMBOL(__bitmap_andnot);
 int __bitmap_intersects(const unsigned long *bitmap1,
                         const unsigned long *bitmap2, unsigned int bits)
 {
-	int k, lim = bits/BITS_PER_LONG;
+	unsigned int k, lim = bits / BITS_PER_LONG;
+
 	for (k = 0; k < lim; ++k)
 		if (bitmap1[k] & bitmap2[k])
 			return 1;
@@ -174,7 +175,8 @@ EXPORT_SYMBOL(__bitmap_intersects);
 int __bitmap_subset(const unsigned long *bitmap1,
                     const unsigned long *bitmap2, unsigned int bits)
 {
-	int k, lim = bits/BITS_PER_LONG;
+	unsigned int k, lim = bits / BITS_PER_LONG;
+
 	for (k = 0; k < lim; ++k)
 		if (bitmap1[k] & ~bitmap2[k])
 			return 0;
@@ -200,11 +202,11 @@ unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int bits)
 }
 EXPORT_SYMBOL(__bitmap_weight);
 
-void __bitmap_set(unsigned long *map, unsigned int start, int len)
+void __bitmap_set(unsigned long *bitmap, unsigned int start, unsigned int len)
 {
-	unsigned long *p = map + BIT_WORD(start);
+	unsigned long *p = bitmap + BIT_WORD(start);
 	const unsigned int size = start + len;
-	int bits_to_set = BITS_PER_LONG - (start % BITS_PER_LONG);
+	unsigned int bits_to_set = BITS_PER_LONG - (start % BITS_PER_LONG);
 	unsigned long mask_to_set = BITMAP_FIRST_WORD_MASK(start);
 
 	while (len - bits_to_set >= 0) {
@@ -220,11 +222,11 @@ void __bitmap_set(unsigned long *map, unsigned int start, int len)
 	}
 }
 
-void __bitmap_clear(unsigned long *map, unsigned int start, int len)
+void __bitmap_clear(unsigned long *bitmap, unsigned int start, unsigned int len)
 {
-	unsigned long *p = map + BIT_WORD(start);
+	unsigned long *p = bitmap + BIT_WORD(start);
 	const unsigned int size = start + len;
-	int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG);
+	unsigned int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG);
 	unsigned long mask_to_clear = BITMAP_FIRST_WORD_MASK(start);
 
 	while (len - bits_to_clear >= 0) {
@@ -255,11 +257,11 @@ void __bitmap_clear(unsigned long *map, unsigned int start, int len)
  *
  * Returns either beginning of region or negative error
  */
-int bitmap_find_free_region(unsigned long *bitmap, int bits, int order)
+int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, unsigned int order)
 {
 	unsigned long mask;
-	int pages = 1 << order;
-	int i;
+	unsigned int pages = 1 << order;
+	unsigned int i;
 
 	if(pages > BITS_PER_LONG)
 		return -EINVAL;
@@ -270,8 +272,9 @@ int bitmap_find_free_region(unsigned long *bitmap, int bits, int order)
 
 	/* run up the bitmap pages bits at a time */
 	for (i = 0; i < bits; i += pages) {
-		int index = i/BITS_PER_LONG;
-		int offset = i - (index * BITS_PER_LONG);
+		unsigned int index = i / BITS_PER_LONG;
+		unsigned int offset = i - (index * BITS_PER_LONG);
+
 		if((bitmap[index] & (mask << offset)) == 0) {
 			/* set region in bimap */
 			bitmap[index] |= (mask << offset);
@@ -291,23 +294,26 @@ EXPORT_SYMBOL(bitmap_find_free_region);
  * This is the complement to __bitmap_find_free_region and releases
  * the found region (by clearing it in the bitmap).
  */
-void bitmap_release_region(unsigned long *bitmap, int pos, int order)
+void bitmap_release_region(unsigned long *bitmap, unsigned int pos,
+			   unsigned int order)
 {
-	int pages = 1 << order;
+	unsigned int pages = 1 << order;
 	unsigned long mask = (1ul << (pages - 1));
-	int index = pos/BITS_PER_LONG;
-	int offset = pos - (index * BITS_PER_LONG);
+	unsigned int index = pos / BITS_PER_LONG;
+	unsigned int offset = pos - (index * BITS_PER_LONG);
+
 	mask += mask - 1;
 	bitmap[index] &= ~(mask << offset);
 }
 EXPORT_SYMBOL(bitmap_release_region);
 
-int bitmap_allocate_region(unsigned long *bitmap, int pos, int order)
+int bitmap_allocate_region(unsigned long *bitmap, unsigned int pos,
+			   unsigned int order)
 {
-	int pages = 1 << order;
+	unsigned int pages = 1 << order;
 	unsigned long mask = (1ul << (pages - 1));
-	int index = pos/BITS_PER_LONG;
-	int offset = pos - (index * BITS_PER_LONG);
+	unsigned int index = pos/BITS_PER_LONG;
+	unsigned int offset = pos - (index * BITS_PER_LONG);
 
 	/* We don't do regions of pages > BITS_PER_LONG.  The
 	 * algorithm would be a simple look for multiple zeros in the
@@ -328,7 +334,7 @@ static void bitmap_long_to_byte(uint8_t *bp, const unsigned long *lp,
 				unsigned int nbits)
 {
 	unsigned long l;
-	int i, j, b;
+	unsigned int i, j, b;
 
 	for (i = 0, b = 0; nbits > 0; i++, b += sizeof(l)) {
 		l = lp[i];
@@ -345,7 +351,7 @@ static void bitmap_byte_to_long(unsigned long *lp, const uint8_t *bp,
 				unsigned int nbits)
 {
 	unsigned long l;
-	int i, j, b;
+	unsigned int i, j, b;
 
 	for (i = 0, b = 0; nbits > 0; i++, b += sizeof(l)) {
 		l = 0;
diff --git a/xen/include/xen/bitmap.h b/xen/include/xen/bitmap.h
index b9f980e91930..4e71b4350f59 100644
--- a/xen/include/xen/bitmap.h
+++ b/xen/include/xen/bitmap.h
@@ -85,12 +85,15 @@ int __bitmap_intersects(const unsigned long *bitmap1,
 int __bitmap_subset(const unsigned long *bitmap1,
                     const unsigned long *bitmap2, unsigned int bits);
 unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int bits);
-extern void __bitmap_set(unsigned long *map, unsigned int start, int len);
-extern void __bitmap_clear(unsigned long *map, unsigned int start, int len);
-
-extern int bitmap_find_free_region(unsigned long *bitmap, int bits, int order);
-extern void bitmap_release_region(unsigned long *bitmap, int pos, int order);
-extern int bitmap_allocate_region(unsigned long *bitmap, int pos, int order);
+void __bitmap_set(unsigned long *bitmap, unsigned int start, unsigned int len);
+void __bitmap_clear(unsigned long *bitmap, unsigned int start, unsigned int len);
+
+int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits,
+                            unsigned int order);
+void bitmap_release_region(unsigned long *bitmap, unsigned int pos,
+                           unsigned int order);
+int bitmap_allocate_region(unsigned long *bitmap, unsigned int pos,
+                           unsigned int order);
 
 #define BITMAP_LAST_WORD_MASK(nbits)					\
 (									\
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:14:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:14:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676180.1052133 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0gV-0002Gz-7I; Mon, 05 Feb 2024 15:14:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676180.1052133; Mon, 05 Feb 2024 15:14:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0gV-0002Gs-3A; Mon, 05 Feb 2024 15:14:59 +0000
Received: by outflank-mailman (input) for mailman id 676180;
 Mon, 05 Feb 2024 15:14:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W3Sh=JO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rX0gU-0002Ch-3v
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:14:58 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 52b1d90b-c439-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 16:14:57 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-5112a04c7acso7255996e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:14:57 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 h13-20020a17090634cd00b00a36ed37683fsm4354132ejb.215.2024.02.05.07.14.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:14:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 52b1d90b-c439-11ee-8a45-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707146097; x=1707750897; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=4DhM0qNlcS2wh71SjY+jNcszwrNeSrXjoBdpVlOY/lM=;
        b=adfFfhqKKh9IcUxlztugRAQeiH3dp3yvJKu01WV4b2oSmGfYNheqsE67M3eGOpXnNG
         wQB0Q1unjeTv7NREuf7JuMQaPTvbYczoJdIyTvETu4RPm9owYkiepP67BbP2u8XRjl3W
         Isw5Q6P8SrLk/1NSIru3RTsNHp68eIb6HTu8M=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707146097; x=1707750897;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=4DhM0qNlcS2wh71SjY+jNcszwrNeSrXjoBdpVlOY/lM=;
        b=IZhC7uDrOgJLRgVEqoTiOkPPHKuygkjRHcdQifW+gQ/QFvV+4xy/m8DXXfAF9xQBm7
         3bYfJWcG5X2UQmepwzNQDumfo5PFl17bVc2z7XZ5teyRIrVxoRcFe4IQbEAKzrUm47dT
         97EGrvjbrARago0wPm94KW0pwDrrPlIcFo2bxxJ6Pd1NdIaQ4Qejfli6dyYuVUAEgl2w
         /ZCU9HjNwaHAS2O14bfvpHofZ9Z7gZC6mHck5Ndh/LHuUQItfyuYZJ6rtahbnmIl9/QV
         KGa4HemAiVQzZnw5U5PlJBNA9SN2m1mpq4IpUDoUiPVt3CYIM9SIVht/iQ9daKmoz+QP
         OdXg==
X-Gm-Message-State: AOJu0YxYFQ3TTdDU+ZHCp6JYdhwCX5MvWDIN69gz2LAs1LDtS72QuXBm
	A6ObEZ+iKJHg2fNOqCEM1TLPF539geSvq0ULYCxjEU14/YYFaFrEslF3VMs1eGvEW85y/VE06Eq
	Q
X-Google-Smtp-Source: AGHT+IE7rhcgFNDX8G2SpT8u8ssM8srOraMr1JiDs90deVAqXRAdHt80NUPYF9oQUx64GSa8eDifBg==
X-Received: by 2002:a05:6512:3c8c:b0:511:558f:f398 with SMTP id h12-20020a0565123c8c00b00511558ff398mr884770lfv.27.1707146096628;
        Mon, 05 Feb 2024 07:14:56 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUZ1jxG58mYd+swy1KdY+em1qGc1rN4hdHLkhRo74BeSRzoUzx5U+ekCEQYRyy3RBpmJma1Bhpp0GCatrEqgYe1amomCkaB7pDselenOAXOYE4RMVLyGCBbLT+tIU4gwWJDsJ7oIO8RPfBoX5QxJdgP0r/XmmXSZQDypNTwOjjx6NV7QoKQrd3c0y3bS4VYyw==
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH] x86/bitmap: Compile with -Wsign-conversion
Date: Mon,  5 Feb 2024 15:14:54 +0000
Message-Id: <20240205151454.1920291-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Use pragmas to able the warning in this file only.  All supported versions of
Clang understand this, while older GCCs simply ignore it.

bitmap_find_free_region() is the only function which isn't sign-convert
clean.  This highlights a latent bug in that it can't return successfully for
a bitmap larger than 2G.

Add an extra check, and explicit cast to silence the warning.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>

Slightly RFC.  This is our first use of pragmas like this.
---
 xen/common/bitmap.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c
index 18e23e2f0e18..b14f8a3b3030 100644
--- a/xen/common/bitmap.c
+++ b/xen/common/bitmap.c
@@ -14,6 +14,9 @@
 #include <xen/lib.h>
 #include <asm/byteorder.h>
 
+#pragma GCC diagnostic warning "-Wsign-conversion"
+#pragma clang diagnostic warning "-Wsign-conversion"
+
 /*
  * bitmaps provide an array of bits, implemented using an an
  * array of unsigned longs.  The number of valid bits in a
@@ -263,7 +266,7 @@ int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, unsigned i
 	unsigned int pages = 1 << order;
 	unsigned int i;
 
-	if(pages > BITS_PER_LONG)
+	if (pages > BITS_PER_LONG || bits >= INT_MAX)
 		return -EINVAL;
 
 	/* make a mask of the order */
@@ -278,7 +281,7 @@ int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, unsigned i
 		if((bitmap[index] & (mask << offset)) == 0) {
 			/* set region in bimap */
 			bitmap[index] |= (mask << offset);
-			return i;
+			return (int)i;
 		}
 	}
 	return -ENOMEM;
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:23:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:23:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676187.1052143 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0oU-0004p6-WA; Mon, 05 Feb 2024 15:23:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676187.1052143; Mon, 05 Feb 2024 15:23:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0oU-0004oz-TR; Mon, 05 Feb 2024 15:23:14 +0000
Received: by outflank-mailman (input) for mailman id 676187;
 Mon, 05 Feb 2024 15:23:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vGJX=JO=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1rX0oT-0004ot-4z
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:23:13 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2611::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7926a945-c43a-11ee-8a45-1f161083a0e0;
 Mon, 05 Feb 2024 16:23:11 +0100 (CET)
Received: from AM9P250CA0017.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:21c::22)
 by AS8PR08MB6373.eurprd08.prod.outlook.com (2603:10a6:20b:33a::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.34; Mon, 5 Feb
 2024 15:23:07 +0000
Received: from AM3PEPF0000A799.eurprd04.prod.outlook.com
 (2603:10a6:20b:21c:cafe::fd) by AM9P250CA0017.outlook.office365.com
 (2603:10a6:20b:21c::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.34 via Frontend
 Transport; Mon, 5 Feb 2024 15:23:07 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM3PEPF0000A799.mail.protection.outlook.com (10.167.16.104) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7249.19 via Frontend Transport; Mon, 5 Feb 2024 15:23:06 +0000
Received: ("Tessian outbound 31df1b57f90c:v228");
 Mon, 05 Feb 2024 15:23:06 +0000
Received: from 8b9f1b5d3fdc.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 2D5E3101-83EE-4305-9307-4E855F47A3F4.1; 
 Mon, 05 Feb 2024 15:22:59 +0000
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8b9f1b5d3fdc.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 05 Feb 2024 15:22:59 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by PR3PR08MB5836.eurprd08.prod.outlook.com (2603:10a6:102:8f::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.34; Mon, 5 Feb
 2024 15:22:57 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::2257:5fcb:b449:fff0]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::2257:5fcb:b449:fff0%7]) with mapi id 15.20.7249.027; Mon, 5 Feb 2024
 15:22:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7926a945-c43a-11ee-8a45-1f161083a0e0
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=IzmbJT/857NLqZRFDxNSyToU4onC3MQZOfDMbhMz9OovFGeTSfvs1TZtzcF5UPw5MrnjVE7yNu37w1xGYpwSjTjxPwfNDd0LfbiR4dwQr4MJvwMYjEwcjrVa5aGAktHvwfT9/l2lJXgIiYD2kIRimMRS05a9Nh2dRheOm0junQ4fpdZCmoQ7gdSLxPN9qwXinbocFO1G5I55+2vWeVSfFaY6UM+MoIwz6jbC2KYSkJqH3GsAKk3peGjrQm5zhbyaLzaTO/YAkkXCBBcLsCeSiDkwT5L/3ad23X6VrlkPpxfweoZM52nZ7nUcBPxCPpmw+dFa+1uYuUBaVg+KfONiaQ==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=SRY6zly75qyZO3IoYXgvNPG6LAcRfIxsTT5sGoA2obE=;
 b=LiLb42WTlBuagCwuFGOZuOBfKCtw1frYVp843TYxdhTQFEA3yXxoWivhtE3Y4skeKvzZS3r+rzcG07JR9UTgKgSiJof0T+Sj9wC6QacxBnlcPlZzcXyOgl4tv8z/2d2MTa6iYZqsnlEvSscrQnduy6EPFRIWiwl3XHEG5NdqwOjidHCbU2DSbyNIRxK9DXjPcZdf0V6da++90IFGdNQ0xE11l8D++zQVzRQvfefzjTtZCT6jenoSOn5LqSkxB2IAg6wcrtCf/M+T5uAn78bXK8/kqtU81YdFPLalZgOWjTiBVKmJbW3NNkalcytopP8X13nG+aRLD0h0hr5zrLOKIQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0
 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=SRY6zly75qyZO3IoYXgvNPG6LAcRfIxsTT5sGoA2obE=;
 b=IYUZxdtPsbgfQIQO8dKatPV4PW3DVlcIXDcMr7+Kh676obbmDhpHOrHgdJtV1wp0rDReg2F7oOjPfMwlGMTf0sT911IcZ0P7+dcdI93cer5fHWrFonhBoFItx4Q+Nubc2KfcmUgM6jwMll10LdUO1nWqVeYp/9VqZqriEfmF81w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 628d59e15ef772d7
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Y5mBjdptPFuBU5/awCjvxatSfiVj82ZuCk5dJBO9Bwvp57RwbtFBfQ7LbFEfSbz4IS3DkeljIagy3E5pwyy5AhNZS7OtyS0Hk7bAxfbwnrAdawOQxR0m8v67d5Dx1a5hDupoloxDSRdkJN3xJ6PdKtDT5L/p6q1nn/y4QWU27SlAURP3REJPUtT2PS2Cgqc7hi0uWarVlP8CBFAOR6mfikZ6PdHmFl/Cy2ni6SbmKS79IBPkOZEoOdysLnGRxWa+V6hHFCXHQ2LRaW1Ke9tl1lhbtpXz5eUCXbctjW37gqv3As5ePDf02HODsN2Do1gxvDnQBijWcoa34Pqk5YdNOw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=SRY6zly75qyZO3IoYXgvNPG6LAcRfIxsTT5sGoA2obE=;
 b=EGvGpoqeKonxlkmw6huy/8VJQmxpZHsQIwTNJRsyxuPOKssu5Qyrp/oNxYJhCvGS3tsn/1UM6VX2YRd7BBykXti8gtKM9yGuUeJL58g9rve5T+zvVEyqgh4XkGq+oh4tkUVZm5eFHuPMXsllL3cgLVRlQOFmmaYodQcCpws1+6MkTVxp2hPlYGZ4HY7n4jQQWmTSx2vSlziz3VOZp2KEnt3OkPgGVmecoX1IpJK9dHg4KyxN1MTJFKlagwuke8gDAVbG7XuqmLJaHEF4ys0ab8jV3jCXvQJUxxL+M5Y40/qhDO+hGlAfC1jDRYxC5XqrFvt7a9EZ2z+FAD7KHmMXNA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=SRY6zly75qyZO3IoYXgvNPG6LAcRfIxsTT5sGoA2obE=;
 b=IYUZxdtPsbgfQIQO8dKatPV4PW3DVlcIXDcMr7+Kh676obbmDhpHOrHgdJtV1wp0rDReg2F7oOjPfMwlGMTf0sT911IcZ0P7+dcdI93cer5fHWrFonhBoFItx4Q+Nubc2KfcmUgM6jwMll10LdUO1nWqVeYp/9VqZqriEfmF81w=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Jens Wiklander <jens.wiklander@linaro.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, "patches@linaro.org"
	<patches@linaro.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Michal
 Orzel <michal.orzel@amd.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [XEN PATCH v3] xen/arm: ffa: reclaim shared memory on guest
 destroy
Thread-Topic: [XEN PATCH v3] xen/arm: ffa: reclaim shared memory on guest
 destroy
Thread-Index: AQHaSTVKNVT0ywRTlE22Kr4Xf5r90LD1mryAgAZVM4CAAAv9AA==
Date: Mon, 5 Feb 2024 15:22:57 +0000
Message-ID: <141E52D2-CBC3-4562-B41C-905F2CA78C85@arm.com>
References: <20240117110618.2183977-1-jens.wiklander@linaro.org>
 <6980D75D-544A-4A3B-906B-DC27D68A260B@arm.com>
 <CAHUa44GS7hX=7msPgzMestM+Dpre0rjuFs9gOq-WY+c=waxi8g@mail.gmail.com>
In-Reply-To:
 <CAHUa44GS7hX=7msPgzMestM+Dpre0rjuFs9gOq-WY+c=waxi8g@mail.gmail.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.200.91.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|PR3PR08MB5836:EE_|AM3PEPF0000A799:EE_|AS8PR08MB6373:EE_
X-MS-Office365-Filtering-Correlation-Id: 5efeede4-6519-4317-7272-08dc265e5ab5
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 KKhsRU89ymIWl1obtOD7uK3N5Mto3dGEKZe17n5Tae6IAPBdczcxDrchYQnCAlyVneiqa5z7Yrrcm8utPTt3yhVJEjvAfEJj1itssxFSSv0SnOMCZlO90bid6RT2KCCNEdGjZmwnf172QWl72scQ8NTUyTe/CCP4iSJ/12NE1NIMicovI9n3JY12h8HoNXbwujVmqfDxS2r2jtDeS0vmCcZd1WTE5UG3H1TYNfZZoBMHfpzwNDtnHBfvChU2LO1qfKLJZxPceSgGELdO2eSMIcWguibapu2rhqsVEDyETef7fBPEFPVkXH6H58/zaxvZ4sYAmDEEWOLRSJHfffQ203ndFhRvg6B4gPEu2sIFaydwSR/KrHIP7j0RKCZuEVPaZRCkqfeSpUpzgvMdIzle+qkp/NSNYsvid8eEoXMt9sp3epTi6OeEIiz9tb2YuaRHe3y7CzbpFlwGBhxD2adbE2y9xII9JmxXcEWSlhRUYUxKYVr9x0dqT2hRvOIjhuqSkQx4RSTwUD1r86bt2uxhjpELdY/Q39QtawANrcGxt5bSi5LBZ2b3tPoF3+pCUp8qK8EIH4GiDoiSYOpw7nPAPt1+gp1J1W5OMdvh++HW+TMOirHef8Gl3YmRWn/S6rKICJ78IqZ9uD+BBppJvir1dQ==
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(366004)(396003)(346002)(376002)(39860400002)(230922051799003)(186009)(1800799012)(64100799003)(451199024)(478600001)(86362001)(26005)(6506007)(6512007)(53546011)(122000001)(6486002)(38100700002)(71200400001)(2616005)(83380400001)(33656002)(316002)(64756008)(54906003)(66476007)(66446008)(66556008)(76116006)(66946007)(4326008)(41300700001)(5660300002)(30864003)(2906002)(36756003)(6916009)(8936002)(8676002)(38070700009)(45980500001);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <41E6FD16E774F2488AD498A72E0675AF@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR08MB5836
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM3PEPF0000A799.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	37bfcb5b-d7a8-4091-ee7b-08dc265e550a
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	mCKH7wBWaJgSSLFWQnTEAnX/bbj69eWFz7N5hXGepUXVOoWHGIXBfFKhlQiXM1wiGFZUE/AhXJxELn+aw0tJFVoWmkSPAgGDdb3GZhjsTiUz4y4BhJXEMVOToDWS5QrU6fX1KWrGvZFa+2ZNyZleLrVW4EOl/pBmciG+2mkO5PZp0p68wAVRPfe1VDo0GqBvdZ4uGhhPhkQ05T2KOE6Gidw3q6nNWELHw8oj3zrYxUIm+DMxd7maIPIOzC1mY1VFhgmPaxuCJRfq7VSNAPDYg9yAGkiYDPwOr3Sg4HgnnqNLuKdUJvcle7I1BPoLCq3vJST/eqxSBidvxpeuaDrn11SZnSXwSdOxOnUoqDBuph12XTE3zT/YXw/4peT51SsAH035OivH+/3eYkEtxf6PJQUDIvBVxCkozZWES1wSuOC2H7k/lbvtaCLQ7v1vKJxKL2jFmyLVeCxTEZo7Up0P8MO4gZYXh2fcW1Mk47WYgGsl3ebBtUcnEPqmd6Ow4lXS8j3GB4UBKwRp6ShMUNpMEwicWC3Ap4zIbWB00gfI0nD7cAixN3lZdTrjhYyc+CpIYInvzmv0QZ/ihDqcrGpoAyYQiJ5CVRCdor6RtDGxZMb/xROyVhrPAFyy8R/glnUWmnFx3GG+OHJYi6NJ+++CmiAnGAI+2Nk4+8bLLYeaMgzWKY73B4wxQPhmXAbwZc05gSqr1VdE755+dlAUbqIPthjPcNEPNOp56NHWLktodUrFJ9HUPsDcQID7ZHI6NUy2
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(136003)(376002)(39860400002)(346002)(230922051799003)(1800799012)(64100799003)(82310400011)(451199024)(186009)(36840700001)(40470700004)(46966006)(41300700001)(6506007)(6862004)(8676002)(8936002)(36756003)(6486002)(4326008)(47076005)(70206006)(54906003)(2906002)(70586007)(30864003)(316002)(36860700001)(5660300002)(86362001)(356005)(81166007)(6512007)(82740400003)(53546011)(2616005)(478600001)(83380400001)(107886003)(33656002)(26005)(336012)(40480700001)(40460700003);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Feb 2024 15:23:06.9764
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5efeede4-6519-4317-7272-08dc265e5ab5
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM3PEPF0000A799.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6373

SGkgSmVucywNCg0KPiBPbiA1IEZlYiAyMDI0LCBhdCAxNDozOSwgSmVucyBXaWtsYW5kZXIgPGpl
bnMud2lrbGFuZGVyQGxpbmFyby5vcmc+IHdyb3RlOg0KPiANCj4gSGkgQmVydHJhbmQsDQo+IA0K
PiBPbiBUaHUsIEZlYiAxLCAyMDI0IGF0IDI6NTfigK9QTSBCZXJ0cmFuZCBNYXJxdWlzDQo+IDxC
ZXJ0cmFuZC5NYXJxdWlzQGFybS5jb20+IHdyb3RlOg0KPj4gDQo+PiBIaSBKZW5zLA0KPj4gDQo+
Pj4gT24gMTcgSmFuIDIwMjQsIGF0IDEyOjA2LCBKZW5zIFdpa2xhbmRlciA8amVucy53aWtsYW5k
ZXJAbGluYXJvLm9yZz4gd3JvdGU6DQo+Pj4gDQo+Pj4gV2hlbiBhbiBGRi1BIGVuYWJsZWQgZ3Vl
c3QgaXMgZGVzdHJveWVkIGl0IG1heSBsZWF2ZSBiZWhpbmQgbWVtb3J5DQo+Pj4gc2hhcmVkIHdp
dGggU1BzLiBUaGlzIG1lbW9yeSBtdXN0IGJlIHJlY2xhaW1lZCBiZWZvcmUgaXQncyByZXVzZWQg
b3IgYW4NCj4+PiBTUCBtYXkgbWFrZSBjaGFuZ2VzIHRvIG1lbW9yeSB1c2VkIGJ5IGEgbmV3IHVu
cmVsYXRlZCBndWVzdC4gU28gd2hlbiB0aGUNCj4+PiBkb21haW4gaXMgdGVhcmVkIGRvd24gYWRk
IEZGLUEgcmVxdWVzdHMgdG8gcmVjbGFpbSBhbGwgcmVtYWluaW5nIHNoYXJlZA0KPj4+IG1lbW9y
eS4NCj4+PiANCj4+PiBTUHMgaW4gdGhlIHNlY3VyZSB3b3JsZCBhcmUgbm90aWZpZWQgdXNpbmcg
Vk1fREVTVFJPWUVEIHRoYXQgYSBndWVzdCBoYXMNCj4+PiBiZWVuIGRlc3Ryb3llZC4gQW4gU1Ag
aXMgc3VwcG9zZWQgdG8gcmVsaW5xdWlzaCBhbGwgc2hhcmVkIG1lbW9yeSB0byBhbGxvdw0KPj4+
IHJlY2xhaW1pbmcgdGhlIG1lbW9yeS4gVGhlIHJlbGlucXVpc2ggb3BlcmF0aW9uIG1heSBuZWVk
IHRvIGJlIGRlbGF5ZWQgaWYNCj4+PiB0aGUgc2hhcmVkIG1lbW9yeSBpcyBmb3IgaW5zdGFuY2Ug
cGFydCBvZiBhIERNQSBvcGVyYXRpb24uDQo+Pj4gDQo+Pj4gVGhlIGRvbWFpbiByZWZlcmVuY2Ug
Y291bnRlciBpcyBpbmNyZWFzZWQgd2hlbiB0aGUgZmlyc3QgRkYtQSBzaGFyZWQNCj4+PiBtZW1v
cnkgaXMgcmVnaXN0ZXJlZCBhbmQgdGhlIGNvdW50ZXIgaXMgZGVjcmVhc2VkIGFnYWluIHdoZW4g
dGhlIGxhc3QNCj4+PiBzaGFyZWQgbWVtb3J5IGlzIHJlY2xhaW1lZC4gSWYgRkYtQSBzaGFyZWQg
bWVtb3J5IHJlZ2lzdHJhdGlvbnMgcmVtYWluDQo+Pj4gYXQgdGhlIGVuZCBvZiBvZiBmZmFfZG9t
YWluX3RlYXJkb3duKCkgYSB0aW1lciBpcyBzZXQgdG8gdHJ5IHRvIHJlY2xhaW0NCj4+PiB0aGUg
c2hhcmVkIG1lbW9yeSBldmVyeSBzZWNvbmQgdW50aWwgdGhlIG1lbW9yeSBpcyByZWNsYWltZWQu
DQo+Pj4gDQo+Pj4gQSBmZXcgbWlub3Igc3R5bGUgZml4ZXMgd2l0aCBhIHJlbW92ZWQgZW1wdHkg
bGluZSBoZXJlIGFuZCBhbiBhZGRlZCBuZXcNCj4+PiBsaW5lIHRoZXJlLg0KPj4+IA0KPj4+IFNp
Z25lZC1vZmYtYnk6IEplbnMgV2lrbGFuZGVyIDxqZW5zLndpa2xhbmRlckBsaW5hcm8ub3JnPg0K
Pj4+IC0tLQ0KPj4+IA0KPj4+IHYzOg0KPj4+IC0gTWVudGlvbmluZyBpbiB0aGUgY29tbWl0IG1l
c3NhZ2UgdGhhdCB0aGVyZSBhcmUgc29tZSBzdHlsZSBmaXhlcw0KPj4+IC0gQWRkcmVzc2luZyBy
ZXZpZXcgY29tbWVudHMNCj4+PiAtIFJlZmFjdG9yIHRoZSBmZmFfZG9tYWluX3RlYXJkb3duKCkg
cGF0aCB0byBsZXQNCj4+PiBmZmFfZG9tYWluX3RlYXJkb3duX2NvbnRpbnVlKCkgZG8gbW9zdCBv
ZiB0aGUgd29yay4NCj4+PiANCj4+PiB2MjoNCj4+PiAtIFVwZGF0ZSBjb21taXQgbWVzc2FnZSB0
byBtYXRjaCB0aGUgbmV3IGltcGxlbWVudGF0aW9uDQo+Pj4gLSBVc2luZyBhIHBlciBkb21haW4g
Yml0ZmllbGQgdG8ga2VlcCB0cmFjayBvZiB3aGljaCBTUHMgaGFzIGJlZW4gbm90aWZpZWQNCj4+
PiB3aXRoIFZNX0RFU1RST1lFRA0KPj4+IC0gSG9sZGluZyBhIGRvbWFpbiByZWZlcmVuY2UgY291
bnRlciB0byBrZWVwIHRoZSBkb21haW4gYXMgYSB6b21iaWUgZG9tYWluDQo+Pj4gd2hpbGUgdGhl
cmUgc3RpbGwgaXMgc2hhcmVkIG1lbW9yeSByZWdpc3RyYXRpb25zIHJlbWFpbmluZyB0byBiZSBy
ZWNsYWltZWQNCj4+PiAtIFVzaW5nIGEgdGltZXIgdG8gcmV0cnkgcmVjbGFpbWluZyByZW1haW5p
bmcgc2hhcmVkIG1lbW9yeSByZWdpc3RyYXRpb25zDQo+Pj4gLS0tDQo+Pj4geGVuL2FyY2gvYXJt
L3RlZS9mZmEuYyB8IDI1MyArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKystLS0tLS0t
LQ0KPj4+IDEgZmlsZSBjaGFuZ2VkLCAyMDQgaW5zZXJ0aW9ucygrKSwgNDkgZGVsZXRpb25zKC0p
DQo+Pj4gDQo+Pj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS90ZWUvZmZhLmMgYi94ZW4vYXJj
aC9hcm0vdGVlL2ZmYS5jDQo+Pj4gaW5kZXggMDc5M2MxYzc1ODVkLi44MGViYmY0ZjAxYzYgMTAw
NjQ0DQo+Pj4gLS0tIGEveGVuL2FyY2gvYXJtL3RlZS9mZmEuYw0KPj4+ICsrKyBiL3hlbi9hcmNo
L2FybS90ZWUvZmZhLmMNCj4+PiBAQCAtNTQsNiArNTQsNyBAQA0KPj4+ICNpbmNsdWRlIDx4ZW4v
bW0uaD4NCj4+PiAjaW5jbHVkZSA8eGVuL3NjaGVkLmg+DQo+Pj4gI2luY2x1ZGUgPHhlbi9zaXpl
cy5oPg0KPj4+ICsjaW5jbHVkZSA8eGVuL3RpbWVyLmg+DQo+Pj4gI2luY2x1ZGUgPHhlbi90eXBl
cy5oPg0KPj4+IA0KPj4+ICNpbmNsdWRlIDxhc20vZXZlbnQuaD4NCj4+PiBAQCAtMTQ0LDYgKzE0
NSwxMiBAQA0KPj4+ICovDQo+Pj4gI2RlZmluZSBGRkFfTUFYX1NITV9DT1VOVCAgICAgICAgICAg
ICAgIDMyDQo+Pj4gDQo+Pj4gKy8qDQo+Pj4gKyAqIFRoZSB0aW1lIHdlIHdhaXQgdW50aWwgdHJ5
aW5nIHRvIHRlYXIgZG93biBhIGRvbWFpbiBhZ2FpbiBpZiBpdCB3YXMNCj4+PiArICogYmxvY2tl
ZCBpbml0aWFsbHkuDQo+Pj4gKyAqLw0KPj4+ICsjZGVmaW5lIEZGQV9DVFhfVEVBUkRPV05fREVM
QVkgICAgICAgICAgU0VDT05EUygxKQ0KPj4+ICsNCj4+PiAvKiBGRi1BLTEuMS1SRUwwIHNlY3Rp
b24gMTAuOS4yIE1lbW9yeSByZWdpb24gaGFuZGxlLCBwYWdlIDE2NyAqLw0KPj4+ICNkZWZpbmUg
RkZBX0hBTkRMRV9IWVBfRkxBRyAgICAgICAgICAgICBCSVQoNjMsIFVMTCkNCj4+PiAjZGVmaW5l
IEZGQV9IQU5ETEVfSU5WQUxJRCAgICAgICAgICAgICAgMHhmZmZmZmZmZmZmZmZmZmZmVUxMDQo+
Pj4gQEAgLTM4NCwxMSArMzkxLDYgQEAgc3RydWN0IGZmYV9jdHggew0KPj4+ICAgIHVuc2lnbmVk
IGludCBwYWdlX2NvdW50Ow0KPj4+ICAgIC8qIEZGLUEgdmVyc2lvbiB1c2VkIGJ5IHRoZSBndWVz
dCAqLw0KPj4+ICAgIHVpbnQzMl90IGd1ZXN0X3ZlcnM7DQo+Pj4gLSAgICAvKg0KPj4+IC0gICAg
ICogTnVtYmVyIG9mIFNQcyB0aGF0IHdlIGhhdmUgc2VudCBhIFZNIGNyZWF0ZWQgc2lnbmFsIHRv
LCB1c2VkIGluDQo+Pj4gLSAgICAgKiBmZmFfZG9tYWluX3RlYXJkb3duKCkgdG8ga25vdyB3aGlj
aCBTUHMgbmVlZCB0byBiZSBzaWduYWxsZWQuDQo+Pj4gLSAgICAgKi8NCj4+PiAtICAgIHVpbnQx
Nl90IGNyZWF0ZV9zaWduYWxfY291bnQ7DQo+Pj4gICAgYm9vbCByeF9pc19mcmVlOw0KPj4+ICAg
IC8qIFVzZWQgc2hhcmVkIG1lbW9yeSBvYmplY3RzLCBzdHJ1Y3QgZmZhX3NobV9tZW0gKi8NCj4+
PiAgICBzdHJ1Y3QgbGlzdF9oZWFkIHNobV9saXN0Ow0KPj4+IEBAIC00MDIsNiArNDA0LDE1IEBA
IHN0cnVjdCBmZmFfY3R4IHsNCj4+PiAgICBzcGlubG9ja190IHR4X2xvY2s7DQo+Pj4gICAgc3Bp
bmxvY2tfdCByeF9sb2NrOw0KPj4+ICAgIHNwaW5sb2NrX3QgbG9jazsNCj4+PiArICAgIC8qIFVz
ZWQgaWYgZG9tYWluIGNhbid0IGJlIHRvcm4gZG93biBpbW1lZGlhdGVseSAqLw0KPj4+ICsgICAg
c3RydWN0IGRvbWFpbiAqdGVhcmRvd25fZDsNCj4+PiArICAgIHN0cnVjdCBsaXN0X2hlYWQgdGVh
cmRvd25fbGlzdDsNCj4+PiArICAgIHNfdGltZV90IHRlYXJkb3duX2V4cGlyZTsNCj4+PiArICAg
IC8qDQo+Pj4gKyAgICAgKiBVc2VkIGZvciBmZmFfZG9tYWluX3RlYXJkb3duKCkgdG8ga2VlcCB0
cmFjayBvZiB3aGljaCBTUHMgc2hvdWxkIGJlDQo+Pj4gKyAgICAgKiBub3RpZmllZCB0aGF0IHRo
aXMgZ3Vlc3QgaXMgYmVpbmcgZGVzdHJveWVkLg0KPj4+ICsgICAgICovDQo+Pj4gKyAgICB1bnNp
Z25lZCBsb25nIHZtX2Rlc3Ryb3lfYml0bWFwW107DQo+Pj4gfTsNCj4+PiANCj4+PiBzdHJ1Y3Qg
ZmZhX3NobV9tZW0gew0KPj4+IEBAIC00MzYsNiArNDQ3LDEyIEBAIHN0YXRpYyB2b2lkICpmZmFf
dHggX19yZWFkX21vc3RseTsNCj4+PiBzdGF0aWMgREVGSU5FX1NQSU5MT0NLKGZmYV9yeF9idWZm
ZXJfbG9jayk7DQo+Pj4gc3RhdGljIERFRklORV9TUElOTE9DSyhmZmFfdHhfYnVmZmVyX2xvY2sp
Ow0KPj4+IA0KPj4+ICsNCj4+PiArLyogVXNlZCB0byB0cmFjayBkb21haW5zIHRoYXQgY291bGQg
bm90IGJlIHRvcm4gZG93biBpbW1lZGlhdGVseS4gKi8NCj4+PiArc3RhdGljIHN0cnVjdCB0aW1l
ciBmZmFfdGVhcmRvd25fdGltZXI7DQo+Pj4gK3N0YXRpYyBzdHJ1Y3QgbGlzdF9oZWFkIGZmYV90
ZWFyZG93bl9oZWFkOw0KPj4+ICtzdGF0aWMgREVGSU5FX1NQSU5MT0NLKGZmYV90ZWFyZG93bl9s
b2NrKTsNCj4+PiArDQo+Pj4gc3RhdGljIGJvb2wgZmZhX2dldF92ZXJzaW9uKHVpbnQzMl90ICp2
ZXJzKQ0KPj4+IHsNCj4+PiAgICBjb25zdCBzdHJ1Y3QgYXJtX3NtY2NjXzFfMl9yZWdzIGFyZyA9
IHsNCj4+PiBAQCAtODUzLDcgKzg3MCw2IEBAIHN0YXRpYyBpbnQzMl90IGhhbmRsZV9wYXJ0aXRp
b25faW5mb19nZXQodWludDMyX3QgdzEsIHVpbnQzMl90IHcyLCB1aW50MzJfdCB3MywNCj4+PiAg
ICAgICAgICAgIGdvdG8gb3V0X3J4X3JlbGVhc2U7DQo+Pj4gICAgICAgIH0NCj4+PiANCj4+PiAt
DQo+Pj4gICAgICAgIG1lbWNweShjdHgtPnJ4LCBmZmFfcngsIHN6KTsNCj4+PiAgICB9DQo+Pj4g
ICAgY3R4LT5yeF9pc19mcmVlID0gZmFsc2U7DQo+Pj4gQEAgLTk5Miw1MyArMTAwOCw3NSBAQCBz
dGF0aWMgdm9pZCBwdXRfc2htX3BhZ2VzKHN0cnVjdCBmZmFfc2htX21lbSAqc2htKQ0KPj4+ICAg
IH0NCj4+PiB9DQo+Pj4gDQo+Pj4gLXN0YXRpYyBib29sIGluY19jdHhfc2htX2NvdW50KHN0cnVj
dCBmZmFfY3R4ICpjdHgpDQo+Pj4gK3N0YXRpYyBib29sIGluY19jdHhfc2htX2NvdW50KHN0cnVj
dCBkb21haW4gKmQsIHN0cnVjdCBmZmFfY3R4ICpjdHgpDQo+Pj4gew0KPj4+ICAgIGJvb2wgcmV0
ID0gdHJ1ZTsNCj4+PiANCj4+PiAgICBzcGluX2xvY2soJmN0eC0+bG9jayk7DQo+Pj4gKw0KPj4+
ICsgICAgLyoNCj4+PiArICAgICAqIElmIHRoaXMgaXMgdGhlIGZpcnN0IHNobSBhZGRlZCwgaW5j
cmVhc2UgdGhlIGRvbWFpbiByZWZlcmVuY2UNCj4+PiArICAgICAqIGNvdW50ZXIgYXMgd2UgbmVl
ZCB0byBrZWVwIGRvbWFpbiBhcm91bmQgYSBiaXQgbG9uZ2VyIHRvIHJlY2xhaW0gdGhlDQo+Pj4g
KyAgICAgKiBzaGFyZWQgbWVtb3J5IGluIHRoZSB0ZWFyZG93biBwYXRoLg0KPj4+ICsgICAgICov
DQo+Pj4gKyAgICBpZiAoICFjdHgtPnNobV9jb3VudCApDQo+Pj4gKyAgICAgICAgZ2V0X2tub3du
YWxpdmVfZG9tYWluKGQpOw0KPj4+ICsNCj4+PiAgICBpZiAoY3R4LT5zaG1fY291bnQgPj0gRkZB
X01BWF9TSE1fQ09VTlQpDQo+Pj4gICAgICAgIHJldCA9IGZhbHNlOw0KPj4+ICAgIGVsc2UNCj4+
PiAgICAgICAgY3R4LT5zaG1fY291bnQrKzsNCj4+PiArDQo+Pj4gICAgc3Bpbl91bmxvY2soJmN0
eC0+bG9jayk7DQo+Pj4gDQo+Pj4gICAgcmV0dXJuIHJldDsNCj4+PiB9DQo+Pj4gDQo+Pj4gLXN0
YXRpYyB2b2lkIGRlY19jdHhfc2htX2NvdW50KHN0cnVjdCBmZmFfY3R4ICpjdHgpDQo+Pj4gK3N0
YXRpYyB2b2lkIGRlY19jdHhfc2htX2NvdW50KHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBmZmFf
Y3R4ICpjdHgpDQo+Pj4gew0KPj4+ICAgIHNwaW5fbG9jaygmY3R4LT5sb2NrKTsNCj4+PiArDQo+
Pj4gICAgQVNTRVJUKGN0eC0+c2htX2NvdW50ID4gMCk7DQo+Pj4gICAgY3R4LT5zaG1fY291bnQt
LTsNCj4+PiArDQo+Pj4gKyAgICAvKg0KPj4+ICsgICAgICogSWYgdGhpcyB3YXMgdGhlIGxhc3Qg
c2htIHJlbW92ZWQsIGxldCBnbyBvZiB0aGUgZG9tYWluIHJlZmVyZW5jZSB3ZQ0KPj4+ICsgICAg
ICogdG9vayBpbiBpbmNfY3R4X3NobV9jb3VudCgpIGFib3ZlLg0KPj4+ICsgICAgICovDQo+Pj4g
KyAgICBpZiAoICFjdHgtPnNobV9jb3VudCApDQo+Pj4gKyAgICAgICAgcHV0X2RvbWFpbihkKTsN
Cj4+PiArDQo+Pj4gICAgc3Bpbl91bmxvY2soJmN0eC0+bG9jayk7DQo+Pj4gfQ0KPj4+IA0KPj4+
IC1zdGF0aWMgc3RydWN0IGZmYV9zaG1fbWVtICphbGxvY19mZmFfc2htX21lbShzdHJ1Y3QgZmZh
X2N0eCAqY3R4LA0KPj4+ICtzdGF0aWMgc3RydWN0IGZmYV9zaG1fbWVtICphbGxvY19mZmFfc2ht
X21lbShzdHJ1Y3QgZG9tYWluICpkLA0KPj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IHBhZ2VfY291bnQpDQo+Pj4gew0KPj4+ICsgICAg
c3RydWN0IGZmYV9jdHggKmN0eCA9IGQtPmFyY2gudGVlOw0KPj4+ICAgIHN0cnVjdCBmZmFfc2ht
X21lbSAqc2htOw0KPj4+IA0KPj4+ICAgIGlmICggcGFnZV9jb3VudCA+PSBGRkFfTUFYX1NITV9Q
QUdFX0NPVU5UICkNCj4+PiAgICAgICAgcmV0dXJuIE5VTEw7DQo+Pj4gLSAgICBpZiAoICFpbmNf
Y3R4X3NobV9jb3VudChjdHgpICkNCj4+PiArICAgIGlmICggIWluY19jdHhfc2htX2NvdW50KGQs
IGN0eCkgKQ0KPj4+ICAgICAgICByZXR1cm4gTlVMTDsNCj4+PiANCj4+PiAgICBzaG0gPSB4emFs
bG9jX2ZsZXhfc3RydWN0KHN0cnVjdCBmZmFfc2htX21lbSwgcGFnZXMsIHBhZ2VfY291bnQpOw0K
Pj4+ICAgIGlmICggc2htICkNCj4+PiAgICAgICAgc2htLT5wYWdlX2NvdW50ID0gcGFnZV9jb3Vu
dDsNCj4+PiAgICBlbHNlDQo+Pj4gLSAgICAgICAgZGVjX2N0eF9zaG1fY291bnQoY3R4KTsNCj4+
PiArICAgICAgICBkZWNfY3R4X3NobV9jb3VudChkLCBjdHgpOw0KPj4+IA0KPj4+ICAgIHJldHVy
biBzaG07DQo+Pj4gfQ0KPj4+IA0KPj4+IC1zdGF0aWMgdm9pZCBmcmVlX2ZmYV9zaG1fbWVtKHN0
cnVjdCBmZmFfY3R4ICpjdHgsIHN0cnVjdCBmZmFfc2htX21lbSAqc2htKQ0KPj4+ICtzdGF0aWMg
dm9pZCBmcmVlX2ZmYV9zaG1fbWVtKHN0cnVjdCBkb21haW4gKmQsIHN0cnVjdCBmZmFfc2htX21l
bSAqc2htKQ0KPj4+IHsNCj4+PiArICAgIHN0cnVjdCBmZmFfY3R4ICpjdHggPSBkLT5hcmNoLnRl
ZTsNCj4+PiArDQo+Pj4gICAgaWYgKCAhc2htICkNCj4+PiAgICAgICAgcmV0dXJuOw0KPj4+IA0K
Pj4+IC0gICAgZGVjX2N0eF9zaG1fY291bnQoY3R4KTsNCj4+PiArICAgIGRlY19jdHhfc2htX2Nv
dW50KGQsIGN0eCk7DQo+Pj4gICAgcHV0X3NobV9wYWdlcyhzaG0pOw0KPj4+ICAgIHhmcmVlKHNo
bSk7DQo+Pj4gfQ0KPj4+IEBAIC0xMzA2LDcgKzEzNDQsNyBAQCBzdGF0aWMgdm9pZCBoYW5kbGVf
bWVtX3NoYXJlKHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzKQ0KPj4+ICAgICAgICBnb3RvIG91
dF91bmxvY2s7DQo+Pj4gICAgfQ0KPj4+IA0KPj4+IC0gICAgc2htID0gYWxsb2NfZmZhX3NobV9t
ZW0oY3R4LCBwYWdlX2NvdW50KTsNCj4+PiArICAgIHNobSA9IGFsbG9jX2ZmYV9zaG1fbWVtKGQs
IHBhZ2VfY291bnQpOw0KPj4+ICAgIGlmICggIXNobSApDQo+Pj4gICAgew0KPj4+ICAgICAgICBy
ZXQgPSBGRkFfUkVUX05PX01FTU9SWTsNCj4+PiBAQCAtMTM1MCw3ICsxMzg4LDcgQEAgc3RhdGlj
IHZvaWQgaGFuZGxlX21lbV9zaGFyZShzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncykNCj4+PiAN
Cj4+PiBvdXQ6DQo+Pj4gICAgaWYgKCByZXQgKQ0KPj4+IC0gICAgICAgIGZyZWVfZmZhX3NobV9t
ZW0oY3R4LCBzaG0pOw0KPj4+ICsgICAgICAgIGZyZWVfZmZhX3NobV9tZW0oZCwgc2htKTsNCj4+
PiBvdXRfdW5sb2NrOg0KPj4+ICAgIHNwaW5fdW5sb2NrKCZjdHgtPnR4X2xvY2spOw0KPj4+IA0K
Pj4+IEBAIC0xNDAxLDcgKzE0MzksNyBAQCBzdGF0aWMgaW50IGhhbmRsZV9tZW1fcmVjbGFpbSh1
aW50NjRfdCBoYW5kbGUsIHVpbnQzMl90IGZsYWdzKQ0KPj4+ICAgIH0NCj4+PiAgICBlbHNlDQo+
Pj4gICAgew0KPj4+IC0gICAgICAgIGZyZWVfZmZhX3NobV9tZW0oY3R4LCBzaG0pOw0KPj4+ICsg
ICAgICAgIGZyZWVfZmZhX3NobV9tZW0oZCwgc2htKTsNCj4+PiAgICB9DQo+Pj4gDQo+Pj4gICAg
cmV0dXJuIHJldDsNCj4+PiBAQCAtMTQ4Niw2ICsxNTI0LDQxIEBAIHN0YXRpYyBib29sIGZmYV9o
YW5kbGVfY2FsbChzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncykNCj4+PiAgICB9DQo+Pj4gfQ0K
Pj4+IA0KPj4+ICtzdGF0aWMgYm9vbCBpc19pbl9zdWJzY3JfbGlzdChjb25zdCB1aW50MTZfdCAq
c3Vic2NyLCB1aW50MTZfdCBzdGFydCwNCj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdWludDE2X3QgZW5kLCB1aW50MTZfdCBzcF9pZCkNCj4+PiArew0KPj4+ICsgICAgdW5zaWdu
ZWQgaW50IG47DQo+Pj4gKw0KPj4+ICsgICAgZm9yICggbiA9IHN0YXJ0OyBuIDwgZW5kOyBuKysg
KQ0KPj4+ICsgICAgew0KPj4+ICsgICAgICAgIGlmICggc3Vic2NyW25dID09IHNwX2lkICkNCj4+
PiArICAgICAgICAgICAgcmV0dXJuIHRydWU7DQo+Pj4gKyAgICB9DQo+Pj4gKw0KPj4+ICsgICAg
cmV0dXJuIGZhbHNlOw0KPj4+ICt9DQo+Pj4gKw0KPj4+ICtzdGF0aWMgdm9pZCB2bV9kZXN0cm95
X2JpdG1hcF9pbml0KHN0cnVjdCBmZmFfY3R4ICpjdHgsDQo+Pj4gKyAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgdW5zaWduZWQgaW50IGNyZWF0ZV9zaWduYWxfY291bnQpDQo+Pj4g
K3sNCj4+PiArICAgIHVuc2lnbmVkIGludCBuOw0KPj4+ICsNCj4+PiArICAgIGZvciAoIG4gPSAw
OyBuIDwgc3Vic2NyX3ZtX2Rlc3Ryb3llZF9jb3VudDsgbisrICkNCj4+PiArICAgIHsNCj4+PiAr
ICAgICAgICAvKg0KPj4+ICsgICAgICAgICAqIFNraXAgU1BzIHN1YnNjcmliZWQgdG8gdGhlIFZN
IGNyZWF0ZWQgZXZlbnQgdGhhdCBuZXZlciB3YXMNCj4+PiArICAgICAgICAgKiBub3RpZmllZCBv
ZiB0aGUgVk0gY3JlYXRpb24gZHVlIHRvIGFuIGVycm9yIGR1cmluZw0KPj4+ICsgICAgICAgICAq
IGZmYV9kb21haW5faW5pdCgpLg0KPj4+ICsgICAgICAgICAqLw0KPj4+ICsgICAgICAgIGlmICgg
aXNfaW5fc3Vic2NyX2xpc3Qoc3Vic2NyX3ZtX2NyZWF0ZWQsIGNyZWF0ZV9zaWduYWxfY291bnQs
DQo+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdWJzY3Jfdm1fY3JlYXRlZF9j
b3VudCwNCj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN1YnNjcl92bV9kZXN0
cm95ZWRbbl0pICkNCj4+PiArICAgICAgICAgICAgY29udGludWU7DQo+Pj4gKw0KPj4+ICsgICAg
ICAgIHNldF9iaXQobiwgY3R4LT52bV9kZXN0cm95X2JpdG1hcCk7DQo+Pj4gKyAgICB9DQo+Pj4g
K30NCj4+PiArDQo+Pj4gc3RhdGljIGludCBmZmFfZG9tYWluX2luaXQoc3RydWN0IGRvbWFpbiAq
ZCkNCj4+PiB7DQo+Pj4gICAgc3RydWN0IGZmYV9jdHggKmN0eDsNCj4+PiBAQCAtMTUwMSwxMSAr
MTU3NCwxNCBAQCBzdGF0aWMgaW50IGZmYV9kb21haW5faW5pdChzdHJ1Y3QgZG9tYWluICpkKQ0K
Pj4+ICAgIGlmICggZC0+ZG9tYWluX2lkID49IFVJTlQxNl9NQVgpDQo+Pj4gICAgICAgIHJldHVy
biAtRVJBTkdFOw0KPj4+IA0KPj4+IC0gICAgY3R4ID0geHphbGxvYyhzdHJ1Y3QgZmZhX2N0eCk7
DQo+Pj4gKyAgICBjdHggPSB4emFsbG9jX2ZsZXhfc3RydWN0KHN0cnVjdCBmZmFfY3R4LCB2bV9k
ZXN0cm95X2JpdG1hcCwNCj4+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgQklUU19U
T19MT05HUyhzdWJzY3Jfdm1fZGVzdHJveWVkX2NvdW50KSk7DQo+Pj4gICAgaWYgKCAhY3R4ICkN
Cj4+PiAgICAgICAgcmV0dXJuIC1FTk9NRU07DQo+Pj4gDQo+Pj4gICAgZC0+YXJjaC50ZWUgPSBj
dHg7DQo+Pj4gKyAgICBjdHgtPnRlYXJkb3duX2QgPSBkOw0KPj4+ICsgICAgSU5JVF9MSVNUX0hF
QUQoJmN0eC0+c2htX2xpc3QpOw0KPj4+IA0KPj4+ICAgIGZvciAoIG4gPSAwOyBuIDwgc3Vic2Ny
X3ZtX2NyZWF0ZWRfY291bnQ7IG4rKyApDQo+Pj4gICAgew0KPj4+IEBAIC0xNTE1LDY1ICsxNTkx
LDE0MSBAQCBzdGF0aWMgaW50IGZmYV9kb21haW5faW5pdChzdHJ1Y3QgZG9tYWluICpkKQ0KPj4+
ICAgICAgICB7DQo+Pj4gICAgICAgICAgICBwcmludGsoWEVOTE9HX0VSUiAiZmZhOiBGYWlsZWQg
dG8gcmVwb3J0IGNyZWF0aW9uIG9mIHZtX2lkICV1IHRvICAldTogcmVzICVkXG4iLA0KPj4+ICAg
ICAgICAgICAgICAgICAgIGdldF92bV9pZChkKSwgc3Vic2NyX3ZtX2NyZWF0ZWRbbl0sIHJlcyk7
DQo+Pj4gLSAgICAgICAgICAgIGN0eC0+Y3JlYXRlX3NpZ25hbF9jb3VudCA9IG47DQo+Pj4gLSAg
ICAgICAgICAgIHJldHVybiAtRUlPOw0KPj4+ICsgICAgICAgICAgICBicmVhazsNCj4+PiAgICAg
ICAgfQ0KPj4+ICAgIH0NCj4+PiAtICAgIGN0eC0+Y3JlYXRlX3NpZ25hbF9jb3VudCA9IHN1YnNj
cl92bV9jcmVhdGVkX2NvdW50Ow0KPj4+IC0NCj4+PiAtICAgIElOSVRfTElTVF9IRUFEKCZjdHgt
PnNobV9saXN0KTsNCj4+PiArICAgIHZtX2Rlc3Ryb3lfYml0bWFwX2luaXQoY3R4LCBuKTsNCj4+
PiArICAgIGlmICggbiAhPSBzdWJzY3Jfdm1fY3JlYXRlZF9jb3VudCApDQo+Pj4gKyAgICAgICAg
cmV0dXJuIC1FSU87DQo+Pj4gDQo+Pj4gICAgcmV0dXJuIDA7DQo+Pj4gfQ0KPj4+IA0KPj4+IC1z
dGF0aWMgYm9vbCBpc19pbl9zdWJzY3JfbGlzdChjb25zdCB1aW50MTZfdCAqc3Vic2NyLCB1aW50
MTZfdCBzdGFydCwNCj4+PiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdWludDE2X3Qg
ZW5kLCB1aW50MTZfdCBzcF9pZCkNCj4+PiArc3RhdGljIHZvaWQgc2VuZF92bV9kZXN0cm95ZWQo
c3RydWN0IGRvbWFpbiAqZCkNCj4+PiB7DQo+Pj4gKyAgICBzdHJ1Y3QgZmZhX2N0eCAqY3R4ID0g
ZC0+YXJjaC50ZWU7DQo+Pj4gICAgdW5zaWduZWQgaW50IG47DQo+Pj4gKyAgICBpbnQzMl90IHJl
czsNCj4+PiANCj4+PiAtICAgIGZvciAoIG4gPSBzdGFydDsgbiA8IGVuZDsgbisrICkNCj4+PiAr
ICAgIGZvciAoIG4gPSAwOyBuIDwgc3Vic2NyX3ZtX2Rlc3Ryb3llZF9jb3VudDsgbisrICkNCj4+
PiAgICB7DQo+Pj4gLSAgICAgICAgaWYgKCBzdWJzY3Jbbl0gPT0gc3BfaWQgKQ0KPj4+IC0gICAg
ICAgICAgICByZXR1cm4gdHJ1ZTsNCj4+PiAtICAgIH0NCj4+PiArICAgICAgICBpZiAoICF0ZXN0
X2JpdChuLCBjdHgtPnZtX2Rlc3Ryb3lfYml0bWFwKSApDQo+Pj4gKyAgICAgICAgICAgIGNvbnRp
bnVlOw0KPj4+IA0KPj4+IC0gICAgcmV0dXJuIGZhbHNlOw0KPj4+ICsgICAgICAgIHJlcyA9IGZm
YV9kaXJlY3RfcmVxX3NlbmRfdm0oc3Vic2NyX3ZtX2Rlc3Ryb3llZFtuXSwgZ2V0X3ZtX2lkKGQp
LA0KPj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgRkZBX01TR19TRU5E
X1ZNX0RFU1RST1lFRCk7DQo+Pj4gKw0KPj4gDQo+PiBJIHRoaW5rIGhlcmUgd2Ugc2hvdWxkIGRp
bnN0aW5ndWlzaCBjYXNlcyBkZXBlbmRpbmcgb24gdGhlIGVycm9yIHJlY2VpdmVkOg0KPj4gLSBy
ZXRyeTogd2Ugc2hvdWxkIGtlZXAgdGhlIGJpdCBzZXQgYW5kIHJldHJ5IGxhdGVyDQo+PiAtIG90
aGVyOiB1bnNldCB0aGUgYml0IGFzIHRoZXJlIGlzIG5vdCBtdWNoIHdlIGNhbiBkbyBidXQgbG9n
IGl0Lg0KPiANCj4gTWFrZXMgc2Vuc2UuDQo+IA0KPj4gDQo+PiBUaGUgaW50ZXJydXB0ZWQgY2Fz
ZSBjYW4gaGFwcGVuLCB3ZSBoYW5kbGUgdGhhdCBhbHJlYWR5IGluIHRoZSBkaXJlY3QgbWVzc2Fn
ZQ0KPj4gcmVxdWVzdCBidXQgd2Ugd2lsbCBnZXQgYmFjayBoZXJlIGlmIHRoYXQgZG9lcyBub3Qg
d29yayBhZnRlciBzb21lIHJldHJpZXMgc28NCj4+IHB1dHRpbmcgaXQgaW4gdGhlIG90aGVyIGNh
dGVnb3J5IGlzIG9rIGkgdGhpbmsuDQo+IA0KPiBJbiB0aGUgY2FzZSBvZiBGRkFfTVNHX1NFTkRf
Vk1fREVTVFJPWUVEIGlzIHRoaW5rIGl0IG1ha2VzIG1vcmUgc2Vuc2UNCj4gdG8gYWxzbyByZXRy
eSBGRkFfUkVUX0lOVEVSUlVQVEVEIGxhdGVyLiBUbyBtZSBpdCBzZWVtcyB0aGF0IHRoZXJlDQo+
IHdpbGwgYmUgYW4gZXZlbiBiZXR0ZXIgY2hhbmNlIGZvciBhbiBldmVudHVhbCBub24tc2VjdXJl
IGludGVycnVwdCB0bw0KPiBiZSBoYW5kbGVkIGlmIHdhaXQgYSBsaXR0bGUgYmVmb3JlIHJldHJ5
aW5nLiBTbywgaWYgeW91IGRvbid0IG1pbmQsDQo+IEknbGwgc2tpcCBjbGVhcmluZyB0aGUgYml0
IGJlbG93IGZvciBib3RoIEZGQV9SRVRfSU5URVJSVVBURUQgYW5kDQo+IEZGQV9SRVRfUkVUUlku
DQoNCkkgYWdyZWUuDQoNCkNoZWVycw0KQmVydHJhbmQNCg0KPiANCj4+IA0KPj4+ICsgICAgICAg
IGlmICggcmVzICkNCj4+PiArICAgICAgICB7DQo+Pj4gKyAgICAgICAgICAgIHByaW50ayhYRU5M
T0dfRVJSICIlcGQ6IGZmYTogRmFpbGVkIHRvIHJlcG9ydCBkZXN0cnVjdGlvbiBvZiB2bV9pZCAl
dSB0byAldTogcmVzICVkXG4iLA0KPj4+ICsgICAgICAgICAgICAgICAgICAgZCwgZ2V0X3ZtX2lk
KGQpLCBzdWJzY3Jfdm1fZGVzdHJveWVkW25dLCByZXMpOw0KPj4+ICsgICAgICAgIH0NCj4+PiAr
ICAgICAgICBlbHNlDQo+Pj4gKyAgICAgICAgew0KPj4+ICsgICAgICAgICAgICBjbGVhcl9iaXQo
biwgY3R4LT52bV9kZXN0cm95X2JpdG1hcCk7DQo+Pj4gKyAgICAgICAgfQ0KPj4+ICsgICAgfQ0K
Pj4+IH0NCj4+PiANCj4+PiAtLyogVGhpcyBmdW5jdGlvbiBpcyBzdXBwb3NlZCB0byB1bmRvIHdo
YXQgZmZhX2RvbWFpbl9pbml0KCkgaGFzIGRvbmUgKi8NCj4+PiAtc3RhdGljIGludCBmZmFfZG9t
YWluX3RlYXJkb3duKHN0cnVjdCBkb21haW4gKmQpDQo+Pj4gK3N0YXRpYyB2b2lkIHJlY2xhaW1f
c2htcyhzdHJ1Y3QgZG9tYWluICpkKQ0KPj4+IHsNCj4+PiAgICBzdHJ1Y3QgZmZhX2N0eCAqY3R4
ID0gZC0+YXJjaC50ZWU7DQo+Pj4gLSAgICB1bnNpZ25lZCBpbnQgbjsNCj4+PiArICAgIHN0cnVj
dCBmZmFfc2htX21lbSAqc2htLCAqdG1wOw0KPj4+ICAgIGludDMyX3QgcmVzOw0KPj4+IA0KPj4+
IC0gICAgaWYgKCAhY3R4ICkNCj4+PiAtICAgICAgICByZXR1cm4gMDsNCj4+PiArICAgIGxpc3Rf
Zm9yX2VhY2hfZW50cnlfc2FmZShzaG0sIHRtcCwgJmN0eC0+c2htX2xpc3QsIGxpc3QpDQo+Pj4g
KyAgICB7DQo+Pj4gKyAgICAgICAgcmVnaXN0ZXJfdCBoYW5kbGVfaGk7DQo+Pj4gKyAgICAgICAg
cmVnaXN0ZXJfdCBoYW5kbGVfbG87DQo+Pj4gDQo+Pj4gLSAgICBmb3IgKCBuID0gMDsgbiA8IHN1
YnNjcl92bV9kZXN0cm95ZWRfY291bnQ7IG4rKyApDQo+Pj4gKyAgICAgICAgdWludDY0X3RvX3Jl
Z3BhaXIoJmhhbmRsZV9oaSwgJmhhbmRsZV9sbywgc2htLT5oYW5kbGUpOw0KPj4+ICsgICAgICAg
IHJlcyA9IGZmYV9tZW1fcmVjbGFpbShoYW5kbGVfbG8sIGhhbmRsZV9oaSwgMCk7DQo+PiANCj4+
IFNhbWUgaGVyZSBhcyBmb3IgdGhlIFZNX0RFU1RST1lFRCwgdGhlcmUgYXJlIHNvbWUgZXJyb3Ig
Y29kZSB0aGF0IHdlIHNob3VsZA0KPj4gbm90IHJlaXRlcmF0ZSBvbiBhcyB0aGV5IHdpbGwgYWxz
byBmYWlsIG5leHQgdGltZToNCj4+IC0gaW52YWxpZCBwYXJhbWV0ZXJzOiBzdG9wIGFuZCBsb2cg
YW4gZXJyb3IsIHRoaXMgd2lsbCBhbHdheXMgZmFpbA0KPj4gLSBubyBtZW1vcnk6IGZvciBub3cg
SSB3b3VsZCBzYXkgZG8gdGhlIHNhbWUgYXMgaW52YWxpZCBwYXJhbWV0ZXIgd2l0aCBhIGNvbW1l
bnQNCj4+IG1heWJlIGJlY2F1c2Ugd2UgY291bGQgcmV0cnkgYW5kIG1lbW9yeSBjb3VkIGJlIGF2
YWlsYWJsZSBuZXh0IHRpbWUNCj4+IC0gYWJvcnQ6IHN0b3AgYW5kIGxvZyBhbiBlcnJvciwgdGhp
cyBpcyBhIG1ham9yIGZhdWx0IG9uIHRoZSBvdGhlciBzaWRlDQo+PiAtIGRlbmllZDogcmV0cnku
DQo+IA0KPiBJIGFncmVlLCBJJ2xsIHByb3Bvc2Ugc29tZXRoaW5nIGxpa2UgdGhhdCBpbiB0aGUg
djQuDQo+IA0KPj4gDQo+Pj4gKyAgICAgICAgaWYgKCByZXMgKQ0KPj4+ICsgICAgICAgIHsNCj4+
PiArICAgICAgICAgICAgcHJpbnRrKFhFTkxPR19HX0lORk8gIiVwZDogZmZhOiBGYWlsZWQgdG8g
cmVjbGFpbSBoYW5kbGUgJSNseCA6ICVkXG4iLA0KPj4+ICsgICAgICAgICAgICAgICAgICAgZCwg
c2htLT5oYW5kbGUsIHJlcyk7DQo+Pj4gKyAgICAgICAgfQ0KPj4+ICsgICAgICAgIGVsc2UNCj4+
PiArICAgICAgICB7DQo+Pj4gKyAgICAgICAgICAgIHByaW50ayhYRU5MT0dfR19ERUJVRyAiJXBk
OiBmZmE6IFJlY2xhaW1lZCBoYW5kbGUgJSNseFxuIiwNCj4+PiArICAgICAgICAgICAgICAgICAg
IGQsIHNobS0+aGFuZGxlKTsNCj4+PiArICAgICAgICAgICAgbGlzdF9kZWwoJnNobS0+bGlzdCk7
DQo+Pj4gKyAgICAgICAgICAgIGZyZWVfZmZhX3NobV9tZW0oZCwgc2htKTsNCj4+PiArICAgICAg
ICB9DQo+Pj4gKyAgICB9DQo+Pj4gK30NCj4+PiArDQo+Pj4gK3N0YXRpYyB2b2lkIGZmYV9kb21h
aW5fdGVhcmRvd25fY29udGludWUoc3RydWN0IGZmYV9jdHggKmN0eCwgYm9vbCBmaXJzdF90aW1l
KQ0KPj4+ICt7DQo+Pj4gKyAgICBzdHJ1Y3QgZmZhX2N0eCAqbmV4dF9jdHggPSBOVUxMOw0KPj4+
ICsNCj4+PiArICAgIHNlbmRfdm1fZGVzdHJveWVkKGN0eC0+dGVhcmRvd25fZCk7DQo+Pj4gKyAg
ICByZWNsYWltX3NobXMoY3R4LT50ZWFyZG93bl9kKTsNCj4+PiArDQo+Pj4gKyAgICBpZiAoIGN0
eC0+c2htX2NvdW50ICkNCj4+PiArICAgIHsNCj4+IA0KPj4gSWYgaSBkbyB1bmRlcnN0YW5kIHJp
Z2h0IHlvdSBvbmx5IHJldHJ5IGlmIHRoZXJlIGFyZSByZW1haW5pbmcgU0hNIHRvIHJlY2xhaW0g
YnV0IHdoYXQNCj4+IGlmIHlvdSBoYXZlIGRlc3Ryb3kgbWVzc2FnZXMgdGhhdCB3ZXJlIG5vdCBz
ZW50IHN1Y2Nlc3NmdWxseSwgc2hvdWxkbid0IHdlIHJlaXRlcmF0ZQ0KPj4gaW4gdGhhdCBjYXNl
IHRvIHNlbmQgdGhlIGRlc3Ryb3kgbWVzc2FnZSBhZ2FpbiA/DQo+IA0KPiBJIGFncmVlLCBJJ2xs
IGZpeC4NCj4gDQo+IENoZWVycywNCj4gSmVucw0KPiANCj4+IA0KPj4+ICsgICAgICAgIHByaW50
ayhYRU5MT0dfR19JTkZPICIlcGQ6IGZmYTogUmVtYWluaW5nIHVuY2xhaW1lZCBoYW5kbGVzLCBy
ZXRyeWluZ1xuIiwgY3R4LT50ZWFyZG93bl9kKTsNCj4+PiArDQo+Pj4gKyAgICAgICAgY3R4LT50
ZWFyZG93bl9leHBpcmUgPSBOT1coKSArIEZGQV9DVFhfVEVBUkRPV05fREVMQVk7DQo+Pj4gKw0K
Pj4+ICsgICAgICAgIHNwaW5fbG9jaygmZmZhX3RlYXJkb3duX2xvY2spOw0KPj4+ICsgICAgICAg
IGxpc3RfYWRkX3RhaWwoJmN0eC0+dGVhcmRvd25fbGlzdCwgJmZmYV90ZWFyZG93bl9oZWFkKTsN
Cj4+PiArICAgICAgICAvKiBOZWVkIHRvIHNldCBhIG5ldyB0aW1lciBmb3IgdGhlIG5leHQgY3R4
IGluIGxpbmUgKi8NCj4+PiArICAgICAgICBuZXh0X2N0eCA9IGxpc3RfZmlyc3RfZW50cnkoJmZm
YV90ZWFyZG93bl9oZWFkLCBzdHJ1Y3QgZmZhX2N0eCwNCj4+PiArICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgdGVhcmRvd25fbGlzdCk7DQo+Pj4gKyAgICAgICAgc3Bpbl91bmxv
Y2soJmZmYV90ZWFyZG93bl9sb2NrKTsNCj4+PiArICAgIH0NCj4+PiArICAgIGVsc2UNCj4+PiAg
ICB7DQo+Pj4gICAgICAgIC8qDQo+Pj4gLSAgICAgICAgICogU2tpcCBTUHMgc3Vic2NyaWJlZCB0
byB0aGUgVk0gY3JlYXRlZCBldmVudCB0aGF0IG5ldmVyIHdhcw0KPj4+IC0gICAgICAgICAqIG5v
dGlmaWVkIG9mIHRoZSBWTSBjcmVhdGlvbiBkdWUgdG8gYW4gZXJyb3IgZHVyaW5nDQo+Pj4gLSAg
ICAgICAgICogZmZhX2RvbWFpbl9pbml0KCkuDQo+Pj4gKyAgICAgICAgICogZG9tYWluX2Rlc3Ry
b3koKSBtaWdodCBoYXZlIGJlZW4gY2FsbGVkICh2aWEgcHV0X2RvbWFpbigpIGluDQo+Pj4gKyAg
ICAgICAgICogcmVjbGFpbV9zaG1zKCkpLCBzbyB3ZSBjYW4ndCB0b3VjaCB0aGUgZG9tYWluIHN0
cnVjdHVyZSBhbnltb3JlLg0KPj4+ICAgICAgICAgKi8NCj4+PiAtICAgICAgICBpZiAoIGlzX2lu
X3N1YnNjcl9saXN0KHN1YnNjcl92bV9jcmVhdGVkLCBjdHgtPmNyZWF0ZV9zaWduYWxfY291bnQs
DQo+Pj4gLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdWJzY3Jfdm1fY3JlYXRlZF9j
b3VudCwNCj4+PiAtICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN1YnNjcl92bV9kZXN0
cm95ZWRbbl0pICkNCj4+PiAtICAgICAgICAgICAgY29udGludWU7DQo+Pj4gKyAgICAgICAgeGZy
ZWUoY3R4KTsNCj4+PiANCj4+PiAtICAgICAgICByZXMgPSBmZmFfZGlyZWN0X3JlcV9zZW5kX3Zt
KHN1YnNjcl92bV9kZXN0cm95ZWRbbl0sIGdldF92bV9pZChkKSwNCj4+PiAtICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgIEZGQV9NU0dfU0VORF9WTV9ERVNUUk9ZRUQpOw0KPj4+
IC0NCj4+PiAtICAgICAgICBpZiAoIHJlcyApDQo+Pj4gLSAgICAgICAgICAgIHByaW50ayhYRU5M
T0dfRVJSICJmZmE6IEZhaWxlZCB0byByZXBvcnQgZGVzdHJ1Y3Rpb24gb2Ygdm1faWQgJXUgdG8g
ICV1OiByZXMgJWRcbiIsDQo+Pj4gLSAgICAgICAgICAgICAgICAgICBnZXRfdm1faWQoZCksIHN1
YnNjcl92bV9kZXN0cm95ZWRbbl0sIHJlcyk7DQo+Pj4gKyAgICAgICAgLyogT25seSBjaGVjayBp
ZiB0aGVyZSBoYXMgYmVlbiBhIGNoYW5nZSB0byB0aGUgdGVhcmRvd24gcXVldWUgKi8NCj4+PiAr
ICAgICAgICBpZiAoICFmaXJzdF90aW1lICkNCj4+PiArICAgICAgICB7DQo+Pj4gKyAgICAgICAg
ICAgIHNwaW5fbG9jaygmZmZhX3RlYXJkb3duX2xvY2spOw0KPj4+ICsgICAgICAgICAgICBuZXh0
X2N0eCA9IGxpc3RfZmlyc3RfZW50cnlfb3JfbnVsbCgmZmZhX3RlYXJkb3duX2hlYWQsDQo+Pj4g
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVjdCBm
ZmFfY3R4LCB0ZWFyZG93bl9saXN0KTsNCj4+PiArICAgICAgICAgICAgc3Bpbl91bmxvY2soJmZm
YV90ZWFyZG93bl9sb2NrKTsNCj4+PiArICAgICAgICB9DQo+Pj4gICAgfQ0KPj4+IA0KPj4+ICsg
ICAgaWYgKCBuZXh0X2N0eCApDQo+Pj4gKyAgICAgICAgc2V0X3RpbWVyKCZmZmFfdGVhcmRvd25f
dGltZXIsIG5leHRfY3R4LT50ZWFyZG93bl9leHBpcmUpOw0KPj4+ICt9DQo+Pj4gKw0KPj4+ICtz
dGF0aWMgdm9pZCBmZmFfdGVhcmRvd25fdGltZXJfY2FsbGJhY2sodm9pZCAqYXJnKQ0KPj4+ICt7
DQo+Pj4gKyAgICBzdHJ1Y3QgZmZhX2N0eCAqY3R4Ow0KPj4+ICsNCj4+PiArICAgIHNwaW5fbG9j
aygmZmZhX3RlYXJkb3duX2xvY2spOw0KPj4+ICsgICAgY3R4ID0gbGlzdF9maXJzdF9lbnRyeV9v
cl9udWxsKCZmZmFfdGVhcmRvd25faGVhZCwgc3RydWN0IGZmYV9jdHgsDQo+Pj4gKyAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgdGVhcmRvd25fbGlzdCk7DQo+Pj4gKyAgICBpZiAo
IGN0eCApDQo+Pj4gKyAgICAgICAgbGlzdF9kZWwoJmN0eC0+dGVhcmRvd25fbGlzdCk7DQo+Pj4g
KyAgICBzcGluX3VubG9jaygmZmZhX3RlYXJkb3duX2xvY2spOw0KPj4+ICsNCj4+PiArICAgIGlm
ICggY3R4ICkNCj4+PiArICAgICAgICBmZmFfZG9tYWluX3RlYXJkb3duX2NvbnRpbnVlKGN0eCwg
ZmFsc2UgLyogIWZpcnN0X3RpbWUgKi8pOw0KPj4+ICsgICAgZWxzZQ0KPj4+ICsgICAgICAgIHBy
aW50ayhYRU5MT0dfR19FUlIgIiVzOiB0ZWFyZG93biBsaXN0IGlzIGVtcHR5XG4iLCBfX2Z1bmNf
Xyk7DQo+Pj4gK30NCj4+PiArDQo+Pj4gKy8qIFRoaXMgZnVuY3Rpb24gaXMgc3VwcG9zZWQgdG8g
dW5kbyB3aGF0IGZmYV9kb21haW5faW5pdCgpIGhhcyBkb25lICovDQo+Pj4gK3N0YXRpYyBpbnQg
ZmZhX2RvbWFpbl90ZWFyZG93bihzdHJ1Y3QgZG9tYWluICpkKQ0KPj4+ICt7DQo+Pj4gKyAgICBz
dHJ1Y3QgZmZhX2N0eCAqY3R4ID0gZC0+YXJjaC50ZWU7DQo+Pj4gKw0KPj4+ICsgICAgaWYgKCAh
Y3R4ICkNCj4+PiArICAgICAgICByZXR1cm4gMDsNCj4+PiArDQo+Pj4gICAgaWYgKCBjdHgtPnJ4
ICkNCj4+PiAgICAgICAgcnh0eF91bm1hcChjdHgpOw0KPj4+IA0KPj4+IC0gICAgWEZSRUUoZC0+
YXJjaC50ZWUpOw0KPj4+ICsgICAgZmZhX2RvbWFpbl90ZWFyZG93bl9jb250aW51ZShjdHgsIHRy
dWUgLyogZmlyc3RfdGltZSAqLyk7DQo+Pj4gDQo+Pj4gICAgcmV0dXJuIDA7DQo+Pj4gfQ0KPj4+
IEBAIC0xNzM5LDYgKzE4OTEsOSBAQCBzdGF0aWMgYm9vbCBmZmFfcHJvYmUodm9pZCkNCj4+PiAg
ICBpZiAoICFpbml0X3NwcygpICkNCj4+PiAgICAgICAgZ290byBlcnJfZnJlZV9mZmFfdHg7DQo+
Pj4gDQo+Pj4gKyAgICBJTklUX0xJU1RfSEVBRCgmZmZhX3RlYXJkb3duX2hlYWQpOw0KPj4+ICsg
ICAgaW5pdF90aW1lcigmZmZhX3RlYXJkb3duX3RpbWVyLCBmZmFfdGVhcmRvd25fdGltZXJfY2Fs
bGJhY2ssIE5VTEwsIDApOw0KPj4+ICsNCj4+PiAgICByZXR1cm4gdHJ1ZTsNCj4+PiANCj4+PiBl
cnJfZnJlZV9mZmFfdHg6DQo+Pj4gLS0NCj4+PiAyLjM0LjENCj4+PiANCj4+IA0KPj4gUmVnYXJk
cw0KPj4gQmVydHJhbmQNCg0KDQo=


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676202.1052223 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xo-0000a4-BP; Mon, 05 Feb 2024 15:32:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676202.1052223; Mon, 05 Feb 2024 15:32:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xo-0000ZX-6s; Mon, 05 Feb 2024 15:32:52 +0000
Received: by outflank-mailman (input) for mailman id 676202;
 Mon, 05 Feb 2024 15:32:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xm-0007Hv-Uc
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:50 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d21fed8e-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:32:50 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-510faac8c57so2310580e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:50 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d21fed8e-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147169; x=1707751969; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jujKuKcHk/bGVpGrSTxRhAACNFbzeped1VAo2Fwe7V4=;
        b=JTTcyGWrG/Sm/ASFdQvhndLrJIeda4VYzRZAUjwVo5RbMQHtqSyugPG1rw0kVwb7HM
         6r+PwUshbF/HldBKkGMYQB5dA7fsIfo+5yXhCqQu4Pbt7OiU0J7RkhJyZ8VwOLVUFIee
         XAA47g1Jq3niyjjJAUE+ep5rFHN7DtRSNWZNxeqiVaXNa+q5K7xJYdPImFCEHxjb7TlV
         Lla+2WxMJzBl2vC5R1zxbrEcfLczTzR/jghRUMLJejq8+U6kmXejnlOBETH2BCg98koP
         evel2hWIWZ/EkVF5kOqoSkBmptfhSuYmpmJZ4HTVDqHEY4Ha6rPfh1G5bauCDSgYElWF
         RjKg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147169; x=1707751969;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=jujKuKcHk/bGVpGrSTxRhAACNFbzeped1VAo2Fwe7V4=;
        b=sJsFq9foavMfgjx37jd8souItCRpzxi4HxTuoQn9ruQyEXK61Q1uUW8Trj3TsEKYLZ
         DmPKwcQ4iu9bImadfyZi4AMYminhSf4/4z1oIIVnU3Dhhgj+t+XxSHSQaLmSh96xxLO0
         N7QL0ODvDeSrfhbmNAcnMeQ1CX+hkQK4TMDcK+Z2OfdYU/rKv/v9fuG3zkuxi/p8HAHr
         wjF/Ifs6YiP+CieshxkPP6iBVPMRyQGNl1bOlBU8cESBJFfeQR17C/PQ524DUbD3uU8B
         dJXkVOqgwwj5mLiAkJVU8/Zva8VNtaxOcRSgHlA+KHAfXj4ETkHFKP1WBF1aZdEqFU7x
         m5Vg==
X-Gm-Message-State: AOJu0YyDWG6PgeMXyecBjIW5Q589NlY78EMU2xPLAFCNWBA93C3nKoY1
	pQPTicnnKvu4xVIVP6FFSh5Y6VKRZ4BxphzbbgA+W5lOYKs48K5VflDBS9/P
X-Google-Smtp-Source: AGHT+IF5joLMpboqZ7YvXzw1AujwcY43pMBJ4ztKpRXJKsjYJR7EOe8dGcLYV3ZESg0gbGKowODmvQ==
X-Received: by 2002:ac2:5b06:0:b0:511:49e5:f36a with SMTP id v6-20020ac25b06000000b0051149e5f36amr1500843lfn.10.1707147168866;
        Mon, 05 Feb 2024 07:32:48 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXXzy8tRKrdIG0OTSJdxQf+eTm1rFKunQ+zIHvY16CdSqXwClYmzS/0rh8e6Y6f8QaBBmLLiptZb5m9cr5Y7czaEmFFcRYtS/mLSsOspwvli8Nc7aLQg/YfOVEt71HKBQKuKblX9k8oecsHHRY649cdO/NWaoFWDN1kto7N0zUBlfEKcTf3HkAn98hp6URmwdjlsAolTg5sCs2Il9ot55wK+4RhWGl9jgBIvD38qKL0hU4mxXKUFKsh28WFdVKWMGiSRn3C3hfGxZ2+nEpY4fWQe+Q+bUCdfUrXBucLxvQ/b9nQyK5a2wq3jzVYZkZOqvWmuAP/
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 08/30] xen/riscv: introduce setup.h
Date: Mon,  5 Feb 2024 16:32:15 +0100
Message-ID: <9556f01b1c95c8b22dae4cfec7cca9a1040e5395.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V4:
  - Nothing changed. Only rebase
---
Changes in V3:
 - add SPDX
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/setup.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/setup.h

diff --git a/xen/arch/riscv/include/asm/setup.h b/xen/arch/riscv/include/asm/setup.h
new file mode 100644
index 0000000000..7613a5dbd0
--- /dev/null
+++ b/xen/arch/riscv/include/asm/setup.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef __ASM_RISCV_SETUP_H__
+#define __ASM_RISCV_SETUP_H__
+
+#define max_init_domid (0)
+
+#endif /* __ASM_RISCV_SETUP_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676197.1052169 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xi-0007a0-Mt; Mon, 05 Feb 2024 15:32:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676197.1052169; Mon, 05 Feb 2024 15:32:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xi-0007Zg-FN; Mon, 05 Feb 2024 15:32:46 +0000
Received: by outflank-mailman (input) for mailman id 676197;
 Mon, 05 Feb 2024 15:32:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xg-0007I6-NG
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:44 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cd64af5d-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:32:42 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-50eac018059so5500705e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:42 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd64af5d-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147161; x=1707751961; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=XkKBTIpds/dxSWE/+St+qbfVnfucg4jDXBYkSl6B8WQ=;
        b=GXnLvwbuTQpegs5+P0zjBMhOcLQORpb3yr4Mph+Dc1/bTqDWFKmuCzs1c5B9r9IRWU
         ZcRtMFf/WdbrXM9VeB5VCYgjU/UtxGtstsPAgQT43e1rINimBsptn4f+AVwmpeKp4hCC
         kEY4K3QRJqymy1JYXL5MNff3K3WQnICU7b23VYM5Cr9/doQqS5vb1+KiIRQmbgU/qCUo
         I9TH+R+xZ0qyhJNY4qEXgoqF7NcPee1YQvJZHl1/QZEfpDAk6UwMTAZ9quJn63Mj9sZJ
         N5DU4LJKIzm+hZhFg4Ni+xQsQsi2rEYli2Ki5BNXoVzr4tmG2MNHcK4xxvmrr2LY46/5
         ehfw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147161; x=1707751961;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=XkKBTIpds/dxSWE/+St+qbfVnfucg4jDXBYkSl6B8WQ=;
        b=jSn7pjRHvR8zMwaAMVAAKJWfD5zsl8w2xCnwoquhGGUfpfFA5GZ/mHvuFBTB19aXeI
         //3xuoVHWTEsmEtAann/A00i+UOP/y3mLlsnKwghghiaKCrMYKUVpoEHNNfZthiulrAZ
         21YVyXFTp2JHGqXNmB65LEUVGF5F4FEflrvlpOke2ltpOrFASkwHvR8VGHuR++MwaWT1
         SXxobm6wG2dkRz0y1eRc+rav/L4Kz6ZQ7jPN902xYkhf5c9XkQEkeBVAUaxDL4S2pSOn
         UXIomOwM/t4T9Uso5cSjv5X7G4sni609p0qK8cqdDfuebBmQCgU1jlPvyvU36yPdA1+l
         MbvA==
X-Gm-Message-State: AOJu0Yy03UYu8WvbAR21DqhXSQU70aSaxLZe2dvmw1xL6i0wI5km+XUe
	JTDJJ6zNU5eFamNHSBJxopGqEbQswtcB+cTar7iXzzJ81YeEpxitKKFaF8Em
X-Google-Smtp-Source: AGHT+IEF8E/nDMiQpL9fiHzdUvzAl/ml77VloKsI50UdF2sGA8VSWi2E7/ekkWZ8fwWv95YeR3AUhQ==
X-Received: by 2002:ac2:46f7:0:b0:511:454e:6030 with SMTP id q23-20020ac246f7000000b00511454e6030mr4123262lfo.24.1707147160832;
        Mon, 05 Feb 2024 07:32:40 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCX9xbflx0XvRv31KwZ+AL5hyPj2S2QvGmuphBjFrxFJbiUK89oq/5QUcJld8GvY1mH/15a1YjNUGwHCXe4oqOaUjItiqaJqZOAj5vqzwGZxVp8RZuPBXGfe5/5DUYXnu/xC1M8SLN9l/ULjsr5uGm7GqR5N3KsYh4NO7DwlR/TQ8ho0hwR/0y+kN29BF3f4VLwv5XDGm5Otik7ieghpuVU6xh8s3E7uHW5SFbOJZ0dwMkKk7HK4haAhgdDugAK6956odUOmFoR0GeUW9UWjD1RRJRcObM1SMRAHW1ngJ1Hei8PBgIywnEw+eqOS6fUPj0Q2wTbAqs9iC1e6VePLn/c/ANMSzhg3Btm3R/+8HGZqYDFCh8MFMEcb/7okQAlQ87djw6/7M6sTenTnIpeL6bpG/yCXpsc0RXAwgo7S64Hc5KtQGdyDybWbejg5351fBjA2LO5T5Q17/d/kNPyyftOJPJKpvQXGNxs3uaEghZ5t9Fku+iNn0FcFM2FQq3t2ryEBXMPAxfxtVenP1W807JmT//WuXJMboxmNDODHDuTRpnv+tU9S+0gvOQCaJjtILhfNPcYY9VhFVFVbm3xDwFsJbF1KC5MifwX8x8F1G1mo6B5pS78LMgclLF0PDaWjG+DZxvE=
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>,
	Paul Durrant <paul@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH v4 00/30]  Enable build of full Xen for RISC-V
Date: Mon,  5 Feb 2024 16:32:07 +0100
Message-ID: <cover.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch series performs all of the additions necessary to drop the
build overrides for RISCV and enable the full Xen build. Except in cases
where compatibile implementations already exist (e.g. atomic.h and
bitops.h), the newly added definitions are simple.

The patch series is based on the following patch series:
-	[PATCH v6 0/9] Introduce generic headers   [1]
- [PATCH] move __read_mostly to xen/cache.h  [2]
- [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(), ASSERT_UNREACHABLE() to xen/bug.h [3]
- [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE() [4]
- [PATCH] xen/lib: introduce generic find next bit operations [5]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.

[1] https://lore.kernel.org/xen-devel/cover.1703072575.git.oleksii.kurochko@gmail.com/
[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/
[3] https://lore.kernel.org/xen-devel/4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com/
[4] https://lore.kernel.org/xen-devel/42fc6ae8d3eb802429d29c774502ff232340dc84.1706259490.git.federico.serafini@bugseng.com/
[5] https://lore.kernel.org/xen-devel/52730e6314210ba4164a9934a720c4fda201447b.1706266854.git.oleksii.kurochko@gmail.com/

---
Changes in V4:
 - Update the cover letter message: new patch series dependencies.
 - Some patches were merged to staging, so they were dropped in this patch series:
     [PATCH v3 09/34] xen/riscv: introduce system.h
     [PATCH v3 18/34] xen/riscv: introduce domain.h
     [PATCH v3 19/34] xen/riscv: introduce guest_access.h
 - Was sent out of this patch series:
     [PATCH v3 16/34] xen/lib: introduce generic find next bit operations
 - [PATCH v3 17/34] xen/riscv: add compilation of generic find-next-bit.c was
   droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
 - All other changes are specific to a specific patch.
---
Changes in V3:
 - Update the cover letter message
 - The following patches were dropped as they were merged to staging:
    [PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h
    [PATCH v2 04/39] xen/riscv: add public arch-riscv.h
    [PATCH v2 05/39] xen/riscv: introduce spinlock.h
    [PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S
    [PATCH v2 34/39] xen: add RISCV support for pmu.h
    [PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
 - Instead of the following patches were introduced new:
    [PATCH v2 10/39] xen/riscv: introduce asm/iommu.h
    [PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
 - remove "asm/"  for commit messages which start with "xen/riscv:"
 - code style updates.
 - add emulation of {cmp}xchg_* for 1 and 2 bytes types.
 - code style fixes.
 - add SPDX and footer for the newly added headers.
 - introduce generic find-next-bit.c.
 - some other mionor changes. ( details please find in a patch )
---
Changes in V2:
  - Drop the following patches as they are the part of [2]:
      [PATCH v1 06/57] xen/riscv: introduce paging.h
      [PATCH v1 08/57] xen/riscv: introduce asm/device.h
      [PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h
      [PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h
      [PATCH v1 13/57] xen/riscv: introduce asm/iocap.h
      [PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h
      [PATCH v1 18/57] xen/riscv: introduce asm/random.h
      [PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h
      [PATCH v1 24/57] xen/riscv: introduce asm/percpu.h
      [PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h
      [PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h
      [PATCH v1 38/57] xen/riscv: introduce asm/monitor.h
      [PATCH v1 39/57] xen/riscv: introduce asm/numa.h
      [PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
  - xen/lib.h in most of the cases were changed to xen/bug.h as
    mostly functionilty of bug.h is used.
  - align arch-riscv.h with Arm's version of it.
  - change the Author of commit with introduction of asm/atomic.h.
  - update some definition from spinlock.h.
  - code style changes.
---

Bobby Eshleman (1):
  xen/riscv: introduce atomic.h

Oleksii Kurochko (29):
  xen/riscv: disable unnecessary configs
  xen/riscv: use some asm-generic headers
  xen: add support in public/hvm/save.h for PPC and RISC-V
  xen/riscv: introduce cpufeature.h
  xen/riscv: introduce guest_atomics.h
  xen: avoid generation of empty asm/iommu.h
  xen/asm-generic: introdure nospec.h
  xen/riscv: introduce setup.h
  xen/riscv: introduce bitops.h
  xen/riscv: introduce flushtlb.h
  xen/riscv: introduce smp.h
  xen/riscv: introduce cmpxchg.h
  xen/riscv: introduce io.h
  xen/riscv: introduce irq.h
  xen/riscv: introduce p2m.h
  xen/riscv: introduce regs.h
  xen/riscv: introduce time.h
  xen/riscv: introduce event.h
  xen/riscv: introduce monitor.h
  xen/riscv: add definition of __read_mostly
  xen/riscv: define an address of frame table
  xen/riscv: add required things to current.h
  xen/riscv: add minimal stuff to page.h to build full Xen
  xen/riscv: add minimal stuff to processor.h to build full Xen
  xen/riscv: add minimal stuff to mm.h to build full Xen
  xen/riscv: introduce vm_event_*() functions
  xen/rirscv: add minimal amount of stubs to build full Xen
  xen/riscv: enable full Xen build
  xen/README: add compiler and binutils versions for RISC-V64

 README                                        |   3 +
 .../gitlab-ci/riscv-fixed-randconfig.yaml     |  27 ++
 docs/misc/riscv/booting.txt                   |   8 +
 xen/arch/arm/include/asm/Makefile             |   1 +
 xen/arch/ppc/include/asm/Makefile             |   1 +
 xen/arch/ppc/include/asm/nospec.h             |  15 -
 xen/arch/riscv/Kconfig                        |   7 +
 xen/arch/riscv/Makefile                       |  18 +-
 xen/arch/riscv/arch.mk                        |   5 +-
 xen/arch/riscv/configs/tiny64_defconfig       |  17 +
 xen/arch/riscv/early_printk.c                 | 168 -------
 xen/arch/riscv/include/asm/Makefile           |  13 +
 xen/arch/riscv/include/asm/atomic.h           | 395 +++++++++++++++++
 xen/arch/riscv/include/asm/bitops.h           | 164 +++++++
 xen/arch/riscv/include/asm/cache.h            |   2 +
 xen/arch/riscv/include/asm/cmpxchg.h          | 237 ++++++++++
 xen/arch/riscv/include/asm/config.h           | 107 +++--
 xen/arch/riscv/include/asm/cpufeature.h       |  23 +
 xen/arch/riscv/include/asm/current.h          |  19 +
 xen/arch/riscv/include/asm/event.h            |  40 ++
 xen/arch/riscv/include/asm/fence.h            |   8 +
 xen/arch/riscv/include/asm/flushtlb.h         |  34 ++
 xen/arch/riscv/include/asm/guest_atomics.h    |  44 ++
 xen/arch/riscv/include/asm/io.h               | 142 ++++++
 xen/arch/riscv/include/asm/irq.h              |  37 ++
 xen/arch/riscv/include/asm/mm.h               | 246 +++++++++++
 xen/arch/riscv/include/asm/monitor.h          |  26 ++
 xen/arch/riscv/include/asm/p2m.h              | 102 +++++
 xen/arch/riscv/include/asm/page.h             |  19 +
 xen/arch/riscv/include/asm/processor.h        |  23 +
 xen/arch/riscv/include/asm/regs.h             |  29 ++
 xen/arch/riscv/include/asm/setup.h            |  17 +
 xen/arch/riscv/include/asm/smp.h              |  26 ++
 xen/arch/riscv/include/asm/time.h             |  29 ++
 xen/arch/riscv/mm.c                           |  52 ++-
 xen/arch/riscv/setup.c                        |  10 +-
 xen/arch/riscv/stubs.c                        | 415 ++++++++++++++++++
 xen/arch/riscv/traps.c                        |  25 ++
 xen/arch/riscv/vm_event.c                     |  19 +
 xen/include/asm-generic/bitops/__ffs.h        |  47 ++
 xen/include/asm-generic/bitops/bitops-bits.h  |  21 +
 xen/include/asm-generic/bitops/ffs.h          |   9 +
 xen/include/asm-generic/bitops/ffsl.h         |  16 +
 xen/include/asm-generic/bitops/ffz.h          |  18 +
 .../asm-generic/bitops/find-first-set-bit.h   |  17 +
 xen/include/asm-generic/bitops/fls.h          |  18 +
 xen/include/asm-generic/bitops/flsl.h         |  10 +
 .../asm-generic/bitops/generic-non-atomic.h   |  89 ++++
 xen/include/asm-generic/bitops/hweight.h      |  13 +
 xen/include/asm-generic/bitops/test-bit.h     |  18 +
 .../asm => include/asm-generic}/nospec.h      |   6 +-
 xen/include/public/hvm/save.h                 |   4 +-
 xen/include/xen/iommu.h                       |   4 +
 53 files changed, 2640 insertions(+), 223 deletions(-)
 create mode 100644 docs/misc/riscv/booting.txt
 delete mode 100644 xen/arch/ppc/include/asm/nospec.h
 create mode 100644 xen/arch/riscv/include/asm/Makefile
 create mode 100644 xen/arch/riscv/include/asm/atomic.h
 create mode 100644 xen/arch/riscv/include/asm/bitops.h
 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h
 create mode 100644 xen/arch/riscv/include/asm/cpufeature.h
 create mode 100644 xen/arch/riscv/include/asm/event.h
 create mode 100644 xen/arch/riscv/include/asm/fence.h
 create mode 100644 xen/arch/riscv/include/asm/flushtlb.h
 create mode 100644 xen/arch/riscv/include/asm/guest_atomics.h
 create mode 100644 xen/arch/riscv/include/asm/io.h
 create mode 100644 xen/arch/riscv/include/asm/irq.h
 create mode 100644 xen/arch/riscv/include/asm/monitor.h
 create mode 100644 xen/arch/riscv/include/asm/p2m.h
 create mode 100644 xen/arch/riscv/include/asm/regs.h
 create mode 100644 xen/arch/riscv/include/asm/setup.h
 create mode 100644 xen/arch/riscv/include/asm/smp.h
 create mode 100644 xen/arch/riscv/include/asm/time.h
 create mode 100644 xen/arch/riscv/stubs.c
 create mode 100644 xen/arch/riscv/vm_event.c
 create mode 100644 xen/include/asm-generic/bitops/__ffs.h
 create mode 100644 xen/include/asm-generic/bitops/bitops-bits.h
 create mode 100644 xen/include/asm-generic/bitops/ffs.h
 create mode 100644 xen/include/asm-generic/bitops/ffsl.h
 create mode 100644 xen/include/asm-generic/bitops/ffz.h
 create mode 100644 xen/include/asm-generic/bitops/find-first-set-bit.h
 create mode 100644 xen/include/asm-generic/bitops/fls.h
 create mode 100644 xen/include/asm-generic/bitops/flsl.h
 create mode 100644 xen/include/asm-generic/bitops/generic-non-atomic.h
 create mode 100644 xen/include/asm-generic/bitops/hweight.h
 create mode 100644 xen/include/asm-generic/bitops/test-bit.h
 rename xen/{arch/arm/include/asm => include/asm-generic}/nospec.h (79%)

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676199.1052192 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xl-0008Fv-B8; Mon, 05 Feb 2024 15:32:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676199.1052192; Mon, 05 Feb 2024 15:32:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xl-0008Fh-85; Mon, 05 Feb 2024 15:32:49 +0000
Received: by outflank-mailman (input) for mailman id 676199;
 Mon, 05 Feb 2024 15:32:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xj-0007I6-Np
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:47 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cf9f5f23-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:32:46 +0100 (CET)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2cf206e4d56so62538941fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:46 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf9f5f23-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147165; x=1707751965; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=E7nIInFHf/RafTmU6CQuEwerWXKJeKZlR4hvjiIhjCs=;
        b=XV9FPg150ydHA0kA5D1DSQKXu9oUyZdrsbhsB/dlnYOmqT8YhcChFi99+QPbcnGGOu
         PB8/D9KAU3/h4oyQ53BlV8QoWibvuAUm84mJX8ET+oHdw6GMntaPQaR10ji/cFUySrWv
         Hn6HsTDGjs8fkDdGtfeZj3Oq0UuTiyfw4TPn2wnNaymvXq2MW9KlSlnJ0vr8Iv2vMW2I
         Y9tuR3iAI5lZ1CNmpPm2loLkYCMsC0RzMGnBhq6tOOK/8XQwS46NNOIXEj2w7mVoyJeT
         inkiGy9mf7RwMxp88Tw424XI3Vu1HjRnK5MQSJs3xMligSaYvXIqbwAAh8qfqJvQ6gUg
         yYRg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147165; x=1707751965;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=E7nIInFHf/RafTmU6CQuEwerWXKJeKZlR4hvjiIhjCs=;
        b=s0A3q2b+0BjyzSwbZcXbG5HAXiCZVUkn9kW6y7VFo8iaoPVuB0jbdHxJVzz13zV9Xs
         AnrvfK6TtSfMEcGCr5aBzSLKDVWxUOn750fkwlZsZpqnQiTQDS1bhl2iQeIeiO3luFNm
         7VyVOncrALcxuIg2lbvUrfNBSdAK8lIHpvYrq3qkbYXS6b3i6PEAvCCmvbjKfwXHNPhF
         oRzuYyTTuAuV+9bYqQYjOUWdcz1V4YAuWiobzNndZMDIC/FPxhTmxNSR3SAw3V5/CW+D
         NdHadOIOm1N4obc9xJTCmdI0k2OqlMmbpzuKdV3NlH9AYhHbQcsdsfB2+E6mMX0CYKin
         twKw==
X-Gm-Message-State: AOJu0YwF8D6tLwYBUjCaUp3QSL5o886H5Iyxmgn4HpNEteZfMgdJYJ16
	gWtbV4REqVHjl1TH5z7jL+KdI7EeX4R+chxvlTKFqBbPfjamEI1PoMpkJtLU
X-Google-Smtp-Source: AGHT+IFd+bY/f3tOidrDAoSnrjgkYzg/PzJxIDWWsz5vk2I/6CC/zK1pHIKy39mg2sVxdskAAqt7Xw==
X-Received: by 2002:a05:6512:3494:b0:511:530b:6026 with SMTP id v20-20020a056512349400b00511530b6026mr1337078lfr.27.1707147165433;
        Mon, 05 Feb 2024 07:32:45 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUaO8apSIii5fy7hiIr4GJtFrmfvIiIQzlpxzA8WWC28Z3zeVp5ovRDKCeTVzYjqILkDChdbbAi/S+5QbpEe2y26hM2gESmOtJSpzYIiku5NKD4O8+6RHxTbWivbeEu3URtQILagduFe3/Iw2rjQqbvYxtc479XKTjvpLIS+ZP48L8TSqwR9XHBJetx63NWjE/747l1kt/g8JCajWGwAzqmO5Fi/WTDJXbioGDC5C0Jg+HaLAjcZUxOrspHGFtjnzbPFFaDUGdPnum2jFGDoO6FEO82u88csdg7N0r7tUs062blmx27gPZju7moQw5L/Dh7pTsJ
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
Date: Mon,  5 Feb 2024 16:32:11 +0100
Message-ID: <4d58fd5071272acb145cdfd286a90b7bf1c0682b.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V4:
 - Nothing changed. Only rebase
---
Changes in V3:
 - add SPDX and footer
 - update declaration of cpu_nr_siblings() to return unsigned int instead of int.
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/cpufeature.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/cpufeature.h

diff --git a/xen/arch/riscv/include/asm/cpufeature.h b/xen/arch/riscv/include/asm/cpufeature.h
new file mode 100644
index 0000000000..c08b7d67ad
--- /dev/null
+++ b/xen/arch/riscv/include/asm/cpufeature.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_CPUFEATURE_H
+#define __ASM_RISCV_CPUFEATURE_H
+
+#ifndef __ASSEMBLY__
+
+static inline unsigned int cpu_nr_siblings(unsigned int cpu)
+{
+    return 1;
+}
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_RISCV_CPUFEATURE_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676203.1052228 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xo-0000fF-TB; Mon, 05 Feb 2024 15:32:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676203.1052228; Mon, 05 Feb 2024 15:32:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xo-0000e6-In; Mon, 05 Feb 2024 15:32:52 +0000
Received: by outflank-mailman (input) for mailman id 676203;
 Mon, 05 Feb 2024 15:32:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xn-0007I6-6m
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:51 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d1b5319b-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:32:49 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-51124d86022so7009589e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:49 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1b5319b-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147168; x=1707751968; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=s6/hgd3PF/7Am9C/zD33wVCQ9jkwbx6B85pbrUoqBYs=;
        b=Lx6pSLZ+Eqz0S/DmldPFDmV31SHh+ri0fggp4u1dSYOIq7z/f70fcuq4iL19GDMZje
         T0Ikyq++3kezZ6sr69/+ZPcgnBt5tRA0MVMOIgREtgYPMebEmS5quTt38RYpUmdy60UB
         d9Cb7aUDotVQ9mwrvi0U8hOYHEwuDg2eSYNosfsLPsIFnkJtwROkKV7ALrs4Q5beHveC
         zK6eA3Niirm/MJ7EHNxmtbuC6VMtOqeLxtqoQyHlEozCuKEHhyevYbYEnk64e1P5mSO4
         tFxeoZ/lSWrhfWFliUWF1GsKrdguJhWr8+S18xaFfAY71Sn+78gSF7ciWWmSahJ0Q09B
         fvPQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147168; x=1707751968;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=s6/hgd3PF/7Am9C/zD33wVCQ9jkwbx6B85pbrUoqBYs=;
        b=b0+ge5ycF75eQM2nAw3cjbIK52E/RLt/1u2cIyYbaojg73nMrPFlGMl4FBelY9S+vh
         cVDSrh0LbvRfRDVosJa+mm/MMJ9hwool36fY7cB7LZ8d3FeslMes//KsMENzaTFs2jg5
         B+0e1EY7XDhDJc3eQ+lW0CfH2/OOKm5rdJZcPPbSgidbzBTOmf+rQ4sm+d9tvYO/DtnM
         hBMujfEzZkNY4TXNmjgH0jwfiNbnqr7giAY7d0bR2vRynz//y8BAcak+9DHX8FExDcvS
         fbHwuh6QOtMD1oVZHiMmsS7awO1wMr6PEY7iuFbXYWYSA16MXJKzHqR2NtM6iLOC7aOo
         7BHQ==
X-Gm-Message-State: AOJu0YyH5JvK0GFKk6RO9cLuYpvN3g4dwOcq+4terPR7uIw8W4+hW6Fk
	iP3VCCh8le9qx9CZgCDOXn2NO9KQmPDnW7QZqJFQtZsEr9HvlxlczB1g/C6g
X-Google-Smtp-Source: AGHT+IGZlSaylwpgBTUq0NGkad9qCB2LeEIVcHnT9RLHaZQCbEdKHOXdIIfDxm07LHWW1ovQS5Y0mA==
X-Received: by 2002:ac2:558b:0:b0:511:4268:3a54 with SMTP id v11-20020ac2558b000000b0051142683a54mr4092626lfg.29.1707147168101;
        Mon, 05 Feb 2024 07:32:48 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXnvkWL4RaH/9B7oHf5PoxCB+1ZLi0vazkBkGHpCGWfbcELZnACjzfTr7yn7wJ5A7e3tRWl5KYe52ShsYFzlTY5k/Z18aebVItqBxW/iuzKrXX/6+fWokRe48VU9yQE3Fv+l1J91ujrTi9YqKHsDQujLWbpHwz/5l5x5nzw6yO1QMozfoWlROu948Nug46nyGhpNzXtsmLZkn2cSaj4plSbqFgRH+XzfTYWIJgCrymJDDNKqgicsO/Z4fmlRrFNIofhnvjxTE8CIlqaRdMje4GjveOzb95BLCjq2mmVYo726VbNcZmeQXOlMy8ahn3QpqohlDMoryj2VPfhqpmivG5D3qVkwtg7XcRy0TuvrJMh7jr8AApcTSChhKXoTH+Dz27kc0f/n1olOKW8GjLCe2K+5saAq+zgN7I9USsw22QXP4b1S167t8F3Z9DhY3HNNp1AV7oguXaHoKkiGPfOkL4EyNs=
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v4 07/30] xen/asm-generic: introdure nospec.h
Date: Mon,  5 Feb 2024 16:32:14 +0100
Message-ID: <5889d7a5fa81722472f95cc1448af0be8f359a7d.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The <asm/nospec.h> header is similar between Arm, PPC, and RISC-V,
so it has been moved to asm-generic.

Arm's nospec.h was taken as a base with updated guards:
 _ASM_ARM_NOSPEC_H -> _ASM_GENERIC_NOSPEC_H

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V4:
 - Rebase the patch. It was conflics in asm/include/Makefile because it doesn't contain
   numa.h in it because of the patch: [PATCH v2] NUMA: no need for asm/numa.h when !NUMA
 - Properly move/rename the Arm's nospec.h with only guards update in the header from
   _ASM_ARM_NOSPEC_H to _ASM_GENERIC_NOSPEC_H.
 - Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V3:
 - new patch.
---
 xen/arch/arm/include/asm/Makefile                 |  1 +
 xen/arch/ppc/include/asm/Makefile                 |  1 +
 xen/arch/ppc/include/asm/nospec.h                 | 15 ---------------
 xen/arch/riscv/include/asm/Makefile               |  1 +
 .../include/asm => include/asm-generic}/nospec.h  |  6 +++---
 5 files changed, 6 insertions(+), 18 deletions(-)
 delete mode 100644 xen/arch/ppc/include/asm/nospec.h
 rename xen/{arch/arm/include/asm => include/asm-generic}/nospec.h (79%)

diff --git a/xen/arch/arm/include/asm/Makefile b/xen/arch/arm/include/asm/Makefile
index 4a4036c951..41f73bf968 100644
--- a/xen/arch/arm/include/asm/Makefile
+++ b/xen/arch/arm/include/asm/Makefile
@@ -3,6 +3,7 @@ generic-y += altp2m.h
 generic-y += device.h
 generic-y += hardirq.h
 generic-y += iocap.h
+generic-y += nospec.h
 generic-y += paging.h
 generic-y += percpu.h
 generic-y += random.h
diff --git a/xen/arch/ppc/include/asm/Makefile b/xen/arch/ppc/include/asm/Makefile
index ced02e26ed..2e8623bb10 100644
--- a/xen/arch/ppc/include/asm/Makefile
+++ b/xen/arch/ppc/include/asm/Makefile
@@ -5,6 +5,7 @@ generic-y += div64.h
 generic-y += hardirq.h
 generic-y += hypercall.h
 generic-y += iocap.h
+generic-y += nospec.h
 generic-y += paging.h
 generic-y += percpu.h
 generic-y += random.h
diff --git a/xen/arch/ppc/include/asm/nospec.h b/xen/arch/ppc/include/asm/nospec.h
deleted file mode 100644
index b97322e48d..0000000000
--- a/xen/arch/ppc/include/asm/nospec.h
+++ /dev/null
@@ -1,15 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/* From arch/arm/include/asm/nospec.h. */
-#ifndef __ASM_PPC_NOSPEC_H__
-#define __ASM_PPC_NOSPEC_H__
-
-static inline bool evaluate_nospec(bool condition)
-{
-    return condition;
-}
-
-static inline void block_speculation(void)
-{
-}
-
-#endif /* __ASM_PPC_NOSPEC_H__ */
diff --git a/xen/arch/riscv/include/asm/Makefile b/xen/arch/riscv/include/asm/Makefile
index ced02e26ed..2e8623bb10 100644
--- a/xen/arch/riscv/include/asm/Makefile
+++ b/xen/arch/riscv/include/asm/Makefile
@@ -5,6 +5,7 @@ generic-y += div64.h
 generic-y += hardirq.h
 generic-y += hypercall.h
 generic-y += iocap.h
+generic-y += nospec.h
 generic-y += paging.h
 generic-y += percpu.h
 generic-y += random.h
diff --git a/xen/arch/arm/include/asm/nospec.h b/xen/include/asm-generic/nospec.h
similarity index 79%
rename from xen/arch/arm/include/asm/nospec.h
rename to xen/include/asm-generic/nospec.h
index 51c7aea4f4..65fd745db2 100644
--- a/xen/arch/arm/include/asm/nospec.h
+++ b/xen/include/asm-generic/nospec.h
@@ -1,8 +1,8 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 /* Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. */
 
-#ifndef _ASM_ARM_NOSPEC_H
-#define _ASM_ARM_NOSPEC_H
+#ifndef _ASM_GENERIC_NOSPEC_H
+#define _ASM_GENERIC_NOSPEC_H
 
 static inline bool evaluate_nospec(bool condition)
 {
@@ -13,7 +13,7 @@ static inline void block_speculation(void)
 {
 }
 
-#endif /* _ASM_ARM_NOSPEC_H */
+#endif /* _ASM_GENERIC_NOSPEC_H */
 
 /*
  * Local variables:
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676196.1052162 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xi-0007WZ-99; Mon, 05 Feb 2024 15:32:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676196.1052162; Mon, 05 Feb 2024 15:32:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xi-0007WS-6I; Mon, 05 Feb 2024 15:32:46 +0000
Received: by outflank-mailman (input) for mailman id 676196;
 Mon, 05 Feb 2024 15:32:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xg-0007Hv-1q
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:44 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cdf39f89-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:32:43 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-5114fa38434so1247127e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:43 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cdf39f89-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147163; x=1707751963; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7qHLrKRIvGr/pj6MSn1px01rEsYpCg0fb7RhU3fagZE=;
        b=VwYgLVao75azo8Ad0pvpJIN0O94E27TZy/e8p1lulAzitQca1YF5COBtSFEoAbekzi
         n5raDAoih5eimQizJYD6tGsGHPLtbqWbhKt50a5DkiHxksBXVndupx6aatoRHB3Mdtb5
         INDq9Xw6MC9PGN/6ve6inFRzez3vCW+0appx/AI97QSXBys/vN5rfkv1BGUTWRWONRtp
         yEcE7N1aaHOnTgkhuNxTIr4RpGIxTzPYHPE/G3OJgE5edEJsceD+u/WRBVOVTr5H4j7K
         4PDcAM9+D6jrT7rJG9HQQ7ISNhixd7w+LGlLRCLTih8jna6TaTzxZb151fRonGMlBYEX
         wj/A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147163; x=1707751963;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=7qHLrKRIvGr/pj6MSn1px01rEsYpCg0fb7RhU3fagZE=;
        b=PwvPSjLd70kE7kPbfn674rrRmuzhkFVuupx8QiBmyqStbiCs+CAhF3tYNj7VLqwUz7
         iTmoJ9mm40wD0hjeX39gTx/NTGL/Mxu3M1yk1b5+1u4XuSiSxFIe4MpDDfQ5kr4Kawim
         9er0dL1HCvG031YLKD9SGVXUy/hsBwuQ9kM/ppqzuW5SNsEwY89xaFg8gAi9MaJlHOMW
         EhixiPBNuhuaUg0pnzfKzmMjzbTQxZ9/1yrGjiqkCoQ6gtrxSNFpFObDGXHjj4DJb4YA
         mfFDYMnUwm88Lfghxph5ub3NwBpUBM1sVMUUI3ZyirObHWortOgieoa4HmNaRAbAKbcY
         VS+w==
X-Gm-Message-State: AOJu0YyuBhUsRwF19l1VldMGfTkpbc2DRmK9gahYOKZ0p+SC9mwfFXtS
	8JPA6eJQaEWutf1SgVjjDPsyi3gkDr6ySVTorZBIOifpRP5HizS027l4uBj/
X-Google-Smtp-Source: AGHT+IHIA8wkaD2m5YuCwoeg2lrGgd+V9iJbriZOzQfA4jKlxnqVtqIuodYTcjXqoKuOXiokAircvw==
X-Received: by 2002:ac2:46c5:0:b0:511:4ab5:a5dc with SMTP id p5-20020ac246c5000000b005114ab5a5dcmr2980586lfo.44.1707147162678;
        Mon, 05 Feb 2024 07:32:42 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUuqtsQu29gubfjte2Td+r0JNm/Gnl3mOFEdno4vV4QCgS0EPmwW9S5olZ+IFUhzMyaDGnSFjyJZTXd+NaXFoY3dQrX+mFukZGdQcrS1r6xSQ/QXz4JYOi0pj+CreU+9VMPvn9aoH5Zu6bG5J7yZizNK53yr0EOZk9WnCqKdnQD4Ex6pNZXMEaaWbNNq6IojdYyFH+RBiA5tqElR+i0sh0/4HydrebivnSu7Q9Xs7FvFSbyfnjfhNYdfVxOMKji5bTmDnrqP6TkHXcUeqhyP9mYJ1Auk18QnroQSRHYBbSxj7d6Zt91iGFqbkbdARg4snrpWjeM
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 02/30] xen/riscv: use some asm-generic headers
Date: Mon,  5 Feb 2024 16:32:09 +0100
Message-ID: <a721f0c092306b589fae5f44bdaafcd94c60ed14.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Some headers are the same as asm-generic verions of them
so use them instead of arch-specific headers.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
 As [PATCH v6 0/9] Introduce generic headers
 (https://lore.kernel.org/xen-devel/cover.1703072575.git.oleksii.kurochko@gmail.com/)
 is not stable, the list in asm/Makefile can be changed, but the changes will
 be easy.
---
Changes in V4:
- removed numa.h from asm/include/Makefile because of the patch: [PATCH v2] NUMA: no need for asm/numa.h when !NUMA
- updated the commit message
---
Changes in V3:
 - remove monitor.h from the RISC-V asm/Makefile list.
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V2:
 - New commit introduced in V2.
---
 xen/arch/riscv/include/asm/Makefile | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/Makefile

diff --git a/xen/arch/riscv/include/asm/Makefile b/xen/arch/riscv/include/asm/Makefile
new file mode 100644
index 0000000000..ced02e26ed
--- /dev/null
+++ b/xen/arch/riscv/include/asm/Makefile
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+generic-y += altp2m.h
+generic-y += device.h
+generic-y += div64.h
+generic-y += hardirq.h
+generic-y += hypercall.h
+generic-y += iocap.h
+generic-y += paging.h
+generic-y += percpu.h
+generic-y += random.h
+generic-y += softirq.h
+generic-y += vm_event.h
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676201.1052213 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xn-0000Ir-0J; Mon, 05 Feb 2024 15:32:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676201.1052213; Mon, 05 Feb 2024 15:32:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xm-0000Ih-Rg; Mon, 05 Feb 2024 15:32:50 +0000
Received: by outflank-mailman (input) for mailman id 676201;
 Mon, 05 Feb 2024 15:32:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xl-0007I6-Oa
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:49 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d0af2684-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:32:47 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-51124db6cf0so3952831e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:47 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0af2684-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147167; x=1707751967; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=EBotcF0+SThVpP3W9n7XOyM76ldShQpdZhp7LK29d70=;
        b=bTD2yDLrXJ6LzZXaF0oSRYvepLB07HVyKTqZjqvG8asR7M4nlzZRPwOzZZYMQfuqVx
         MIYh+nWLHzCYi5dKrYB7jG5JcohulixFBi3cnAqxdcVkR2JP6p/oXBxi06XnowVVfYHP
         v6nuJMOAvoYJPs3c7SKLac9swSaoPMaZ95fmvvXe/NiobAO00mU/G7wAGnKgwjhbEGfP
         Afc/5gLKMWL9iVmwhM7eO0dCM6/Nl9AxoWG7nzsZDUgK42cya2zL04rsMdndOvBnFSNS
         bdWRNX3h93QQWXuAJGt/z5Hbci/MqCcYswWIClYjLcCCkvXbrOMdqHznTRQcRopLsLfc
         CW9w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147167; x=1707751967;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=EBotcF0+SThVpP3W9n7XOyM76ldShQpdZhp7LK29d70=;
        b=Jf2iGiL6juPFl2QIUvagxmAhcnbM47ziJqiND7TwK5qsWnhw5HZP3/jxwdJ7rNuF26
         GDQc1gxAjeCfRHdPOZgE8dYcbeuc1rQK6IZ/KnmYAiatt00vcSwh8P29WUOSK6/mMEQ1
         JINZZ75u9AsbJBD7gvQEnS7FiZk8hH1HxnGQ1lVCMmD8ysQI9854F0Xub7RaY5cYCWcj
         i4cTgTQh9Tgk3htByy9l+ZspPRkBCrx+ZneQK/z60MT4TLDaS1AUEqyWI791ZpbM97uZ
         Yb9R4Zi0JOWwPi6eJXQ8tWUqWvStGee4ZDMZwisBy4IQb1sfIFQ8l1dOLs0kAs5qQrDV
         sYMQ==
X-Gm-Message-State: AOJu0Ywf2KHvJfmCRBVFl+y4Hmj3k5FUU1xuHIb7CFGCs7ysvYjeNo1g
	vGq07zZpU1Vtfc39fCD30YX1tLtre/6MDFvcconp/dLY/xR3/S9e9o4CCWsw
X-Google-Smtp-Source: AGHT+IGlApynLcTSmBkHRExhVCI8u5HUGtusmYhRVxMrylZdRgq9ep6QS77uqsxZ/UfDXjAB9p32hw==
X-Received: by 2002:a19:4f56:0:b0:511:56fb:b2a8 with SMTP id a22-20020a194f56000000b0051156fbb2a8mr122013lfk.8.1707147167141;
        Mon, 05 Feb 2024 07:32:47 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXnEv4K2PpNuztEqD+bWPCE7ysgM6vo9uLx9ZkjlfOwMNyBtdDQpC4sybGDlJU5awDLLOC6X/EGUrjNzWzLh0Uldqk5N2/moeqwv2MoHDKwb45hlAdvzxdQBKVvdkezRrdzbrDEXg==
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
Date: Mon,  5 Feb 2024 16:32:13 +0100
Message-ID: <a94aba19d32bbe8d2fa78d6d8931f9f0ca51160a.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

asm/iommu.h shouldn't be included when CONFIG_HAS_PASSTHROUGH
isn't enabled.
As <asm/iommu.h> is ifdef-ed by CONFIG_HAS_PASSTHROUGH it should
be also ifdef-ed field "struct arch_iommu arch" in struct domain_iommu
as definition of arch_iommu is located in <asm/iommu.h>.

These amount of changes are enough to avoid generation of empty
asm/iommu.h for now.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - Update the commit message.
---
Changes in V3:
 - new patch.
---
 xen/include/xen/iommu.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index a21f25df9f..7aa6a77209 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -337,7 +337,9 @@ extern int iommu_add_extra_reserved_device_memory(unsigned long start,
 extern int iommu_get_extra_reserved_device_memory(iommu_grdm_t *func,
                                                   void *ctxt);
 
+#ifdef CONFIG_HAS_PASSTHROUGH
 #include <asm/iommu.h>
+#endif
 
 #ifndef iommu_call
 # define iommu_call(ops, fn, args...) ((ops)->fn(args))
@@ -345,7 +347,9 @@ extern int iommu_get_extra_reserved_device_memory(iommu_grdm_t *func,
 #endif
 
 struct domain_iommu {
+#ifdef CONFIG_HAS_PASSTHROUGH
     struct arch_iommu arch;
+#endif
 
     /* iommu_ops */
     const struct iommu_ops *platform_ops;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676198.1052174 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xi-0007fM-TM; Mon, 05 Feb 2024 15:32:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676198.1052174; Mon, 05 Feb 2024 15:32:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xi-0007e8-Ns; Mon, 05 Feb 2024 15:32:46 +0000
Received: by outflank-mailman (input) for mailman id 676198;
 Mon, 05 Feb 2024 15:32:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xh-0007Hv-NS
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:45 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cf13b1bf-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:32:45 +0100 (CET)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-51124d86022so7009478e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:45 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf13b1bf-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147164; x=1707751964; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Iky23uGR3BS6IkxuYzRpFhihZCpnB/wevBTC5pwq7kA=;
        b=O8Sc/2420TVqmHcErK7mxEFulRKLw+KUufdjoVf7+Klsa7nJRT3kBWPhKnr6eLWyyI
         er9P6akeT2WPCeHvVRTrMzTuJDXRDCdjSca/ZWJzCkXuhWvrJNAz+qbmxlxEE1gISHNy
         RyrmtXdbBr+USdjaX9wm3yBD8HCotCUIZPcBP5FaNpkm5jPI3eLPm95WEewoCXJ9B+TT
         JGtWhV2vjgmA97WXL2q7ND5je0Z0KtRMZCu3ywJl9rAMZ1WRT0mwEw0LuuV6he+o2sMz
         zjfiJ9mhr1Th4OanzZWPbcoA6vnJFICBb07MTfUrz5n0HWwKtcyY5M0tbCGKzEVbF97+
         dwYg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147164; x=1707751964;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Iky23uGR3BS6IkxuYzRpFhihZCpnB/wevBTC5pwq7kA=;
        b=BhPsfVUOufGKJAeH8C3lXqG13LWL0SMUmsKrZvwbTspmYbJ6BrxY8oZSGazHwaSfy/
         pHdQa29M+OyRXpTRjrlEzFEm2wx3y09Pyp+a4gkSXtf5bxN399bMqLCQi4KyDrX/kbqe
         mulkWhgicHEJr6zIdPh1W1nBYkYr0rzgkjrU+xa3anKg1jcxtvet3SZMn85+JZHmh3lH
         TNUqgyNE2UGE4uC6qh0Rb4nWqM+RQS0520jc80CzkfGQ2LHAgSOc60akZwxCEHQgSMf8
         yAmgx39WpmzWz7R5PC/oobvROd48CSpx3s8gHhTSb/DOPqM5CKGnZ0ouwbDgao+0mAhU
         uexA==
X-Gm-Message-State: AOJu0YyNKUhzzgX9SPBOFBIaavTUK1eXkMBcdHv8l0Pk4MUCzI5XgK3t
	DGL2MX0dvoWeh/cTh1d86QtmWRjkJD/6b1c9ic/XrbGL7QyqatNWJcBrhLKs
X-Google-Smtp-Source: AGHT+IFtASMyTmydnnfzuUhde2FJ4NxmjxoapaSKIh+dcYVoPZ6lxtpErVslZIDRXWdTgdn1F7bmFQ==
X-Received: by 2002:ac2:5969:0:b0:511:4caf:5fe1 with SMTP id h9-20020ac25969000000b005114caf5fe1mr2478560lfp.37.1707147163897;
        Mon, 05 Feb 2024 07:32:43 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCX8EdkfnzPl4qrey/3AsNj1w0K59JWx765bJQm1AJiSw/nckbZIPn+PdTX0AAN77ZPpLQSyoLMTZMbNkeprio06FBzeew7sjLlY8dl6zc3/eZi6cvql9IrjissCHtgM1SGixmiakbGBUEDQf0B0t3Laj4BYWNM8ceugiGICJOjwNtNdqMsu1Ucaidn50E9C8fhPV1gZy9Om6MuUqORbzEVbRIopy0XCBKIV2w==
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
Date: Mon,  5 Feb 2024 16:32:10 +0100
Message-ID: <c929f784466e1ba57c19663e88f22c598d961709.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

No specific header is needed to include in public/hvm/save.h for
PPC and RISC-V for now.

Code related to PPC was changed based on the comment:
https://lore.kernel.org/xen-devel/c2f3280e-2208-496b-a0b5-fda1a2076b3a@raptorengineering.com/

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
- Updated the commit message
---
Changes in V3:
 - update the commit message.
 - For PPC and RISC-V nothing to include in public/hvm/save.h, so just comment was
   added.
---
Changes in V2:
 - remove copyright an the top of hvm/save.h as the header write now is a newly
   introduced empty header.
---
 xen/include/public/hvm/save.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/include/public/hvm/save.h b/xen/include/public/hvm/save.h
index 5561495b27..72e16ab5bc 100644
--- a/xen/include/public/hvm/save.h
+++ b/xen/include/public/hvm/save.h
@@ -89,8 +89,8 @@ DECLARE_HVM_SAVE_TYPE(END, 0, struct hvm_save_end);
 #include "../arch-x86/hvm/save.h"
 #elif defined(__arm__) || defined(__aarch64__)
 #include "../arch-arm/hvm/save.h"
-#elif defined(__powerpc64__)
-#include "../arch-ppc.h"
+#elif defined(__powerpc64__) || defined(__riscv)
+/* no specific header to include */
 #else
 #error "unsupported architecture"
 #endif
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676195.1052153 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xh-0007IJ-23; Mon, 05 Feb 2024 15:32:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676195.1052153; Mon, 05 Feb 2024 15:32:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xg-0007IB-VQ; Mon, 05 Feb 2024 15:32:44 +0000
Received: by outflank-mailman (input) for mailman id 676195;
 Mon, 05 Feb 2024 15:32:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xf-0007Hv-Cn
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:43 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cd757792-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:32:42 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-5115744dfe5so5542e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:42 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd757792-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147161; x=1707751961; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=QPaE7lsiNetODMb4yDxZSXp3uI/1BFAvrBNl/vZuK+E=;
        b=bKqRZn5jdJN6+wt/o+/lL9ABu/9YZy27WMS/LlSgHnuGLpYtAb+4LXisR1YH8HCqFm
         3iHEKjEhk3L922sC8EXf0UcHwFHf6y07i4KMxUSz3ge7BOvv7GmLbWpvWCBwZOAA9DgW
         8BTic73wpqNu2w+Tyfd6J/Mi4Wk6F4B904d7pEPP1GidqzXyv3bZGe/JCd/jVO9jvuTP
         9PSy3jMfMgB+FGhFIn+rCrkUIPsxzy6npB2iFjleBFfncHv2xSUid1VqGCMMLeVzBJXe
         xugkHkJgeC6jezInvU0lgQ1JPOh/ExoVBgYV1S6RyN2px/YXnzc92jwkb9nlxxMyQDz9
         d29g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147161; x=1707751961;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=QPaE7lsiNetODMb4yDxZSXp3uI/1BFAvrBNl/vZuK+E=;
        b=N7TQRdE4KK4TUzKGbp13ZB18D4U/mwFYW0Cuz3hU8vaKdAq0J1uZNeGAFfikY9FGiP
         w+vROJLeC99X5AFmCe/94Kn1kigl6bn5shYAXg04+oGV00sGZroysV9e2qo/1uWWliJh
         YnLBUv5B4NDX6g+FfCHMsAQTIuRY89zSNvFqnWf2EoWS9Jk8bkkYOtkEMn1sCgMqY2si
         rUgoH7cpSW/rLG0mg6WPtGI2AlM+Mm5n0Ie69DbscHhJI4zPipxx13x+VTmV2nE+nL8P
         5l0i27PQRb1t5/y5nIKJw7sqxOEMenumtHCr8U4N0kd2adkqOKW4J781UOH2+iTlea8t
         WT1w==
X-Gm-Message-State: AOJu0YweoOJUqjm3oG4HijURvzUtehpchwicuaXMPsxQ4gh7F//WUf4J
	zk8WJj3+4Z5kTs1kvGF2j/RrPvRP8iErSlFN52n8LphSh09HKbfw8d93pTh6
X-Google-Smtp-Source: AGHT+IHj7ev+GqSUfsoELsOVw/8nhp+7AMv61Oet/v+xMIOQqrZa4qZ6u5PeQKgFzTMP5CzOqPrZJg==
X-Received: by 2002:a05:6512:31ca:b0:50e:a789:dd3b with SMTP id j10-20020a05651231ca00b0050ea789dd3bmr7231537lfe.1.1707147161488;
        Mon, 05 Feb 2024 07:32:41 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVFa5DpX6sTlNLEgC5k0Lz+lvWZArIUVNAnFr7qgtcW4FKjWwsmyqFiHuoWMgkZQKQ9J3KupSVv0hL+HxfDn3obLPaUicS1IvBaiTsUf+Z/Gaz5dsL9rbP6HxAtvRUcLtlOn7Tka/BJpOXnFMmzS7S6snhK++tJVvjM482qCQO4LattImaKbC0nxKKMQkD2m/lNa4jaFhoI1CgijKO88ksdYEL1
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v4 01/30] xen/riscv: disable unnecessary configs
Date: Mon,  5 Feb 2024 16:32:08 +0100
Message-ID: <3a6ac88525e645e57c0dd44324fadc7ce499369e.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch disables unnecessary configs for two cases:
1. By utilizing EXTRA_FIXED_RANDCONFIG and risc-fixed-randconfig.yaml
   file for randconfig builds (GitLab CI jobs).
2. By using tiny64_defconfig for non-randconfig builds.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - Nothing changed. Only rebase
---
Changes in V3:
 - Remove EXTRA_FIXED_RANDCONFIG for non-randconfig jobs.
   For non-randconfig jobs, it is sufficient to disable configs by using the defconfig.
 - Remove double blank lines in build.yaml file before archlinux-current-gcc-riscv64-debug
---
Changes in V2:
 - update the commit message.
 - remove xen/arch/riscv/Kconfig changes.
---
 .../gitlab-ci/riscv-fixed-randconfig.yaml     | 27 +++++++++++++++++++
 xen/arch/riscv/configs/tiny64_defconfig       | 17 ++++++++++++
 2 files changed, 44 insertions(+)

diff --git a/automation/gitlab-ci/riscv-fixed-randconfig.yaml b/automation/gitlab-ci/riscv-fixed-randconfig.yaml
index f1282b40c9..344f39c2d8 100644
--- a/automation/gitlab-ci/riscv-fixed-randconfig.yaml
+++ b/automation/gitlab-ci/riscv-fixed-randconfig.yaml
@@ -5,3 +5,30 @@
       CONFIG_EXPERT=y
       CONFIG_GRANT_TABLE=n
       CONFIG_MEM_ACCESS=n
+      CONFIG_COVERAGE=n
+      CONFIG_SCHED_CREDIT=n
+      CONFIG_SCHED_CREDIT2=n
+      CONFIG_SCHED_RTDS=n
+      CONFIG_SCHED_NULL=n
+      CONFIG_SCHED_ARINC653=n
+      CONFIG_TRACEBUFFER=n
+      CONFIG_HYPFS=n
+      CONFIG_SPECULATIVE_HARDEN_ARRAY=n
+      CONFIG_ARGO=n
+      CONFIG_HYPFS_CONFIG=n
+      CONFIG_CORE_PARKING=n
+      CONFIG_DEBUG_TRACE=n
+      CONFIG_IOREQ_SERVER=n
+      CONFIG_CRASH_DEBUG=n
+      CONFIG_KEXEC=n
+      CONFIG_LIVEPATCH=n
+      CONFIG_NUMA=n
+      CONFIG_PERF_COUNTERS=n
+      CONFIG_HAS_PMAP=n
+      CONFIG_TRACEBUFFER=n
+      CONFIG_XENOPROF=n
+      CONFIG_COMPAT=n
+      CONFIG_COVERAGE=n
+      CONFIG_UBSAN=n
+      CONFIG_NEEDS_LIBELF=n
+      CONFIG_XSM=n
diff --git a/xen/arch/riscv/configs/tiny64_defconfig b/xen/arch/riscv/configs/tiny64_defconfig
index 09defe236b..35915255e6 100644
--- a/xen/arch/riscv/configs/tiny64_defconfig
+++ b/xen/arch/riscv/configs/tiny64_defconfig
@@ -7,6 +7,23 @@
 # CONFIG_GRANT_TABLE is not set
 # CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
 # CONFIG_MEM_ACCESS is not set
+# CONFIG_ARGO is not set
+# CONFIG_HYPFS_CONFIG is not set
+# CONFIG_CORE_PARKING is not set
+# CONFIG_DEBUG_TRACE is not set
+# CONFIG_IOREQ_SERVER is not set
+# CONFIG_CRASH_DEBUG is not setz
+# CONFIG_KEXEC is not set
+# CONFIG_LIVEPATCH is not set
+# CONFIG_NUMA is not set
+# CONFIG_PERF_COUNTERS is not set
+# CONFIG_HAS_PMAP is not set
+# CONFIG_TRACEBUFFER is not set
+# CONFIG_XENOPROF is not set
+# CONFIG_COMPAT is not set
+# CONFIG_COVERAGE is not set
+# CONFIG_UBSAN is not set
+# CONFIG_NEEDS_LIBELF is not set
 
 CONFIG_RISCV_64=y
 CONFIG_DEBUG=y
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676200.1052199 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xl-0008Jd-Ms; Mon, 05 Feb 2024 15:32:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676200.1052199; Mon, 05 Feb 2024 15:32:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xl-0008Il-Gm; Mon, 05 Feb 2024 15:32:49 +0000
Received: by outflank-mailman (input) for mailman id 676200;
 Mon, 05 Feb 2024 15:32:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xk-0007I6-OC
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:48 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d028054a-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:32:47 +0100 (CET)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-51121637524so6822970e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:47 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d028054a-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147166; x=1707751966; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=r6k+h8FV3d3vqiaEMlAmNrvziqvskuI2jnLZD/dQBlM=;
        b=BgVd3K+6wg0nuW7AQWKED/JY0hLQaTZzM4vXEdBNcyhSKrrMGj5hE+7FcO7sZV8eh8
         HHM0dLIx8DXOpAYL8LIybxN5VDhGh52B1NpCguA5ZngX4STh3n7zq7dS2e4wELc07Wva
         QvTBT9LFdsK/CAI+DnXilpu9bh8kUSeJrTx4epcVwTf+sfPmwItegTHJ+wCLxq2Ep3v1
         cTSfdvZ7i1GYcvYnNFxeE2NYMGb/6HGA2AFGUgf9zP4O1rcUIkpxpXfC8ttmcyIAw6nY
         2gO4DLBoWYG9ar/O4XOOw+dztC/aTIn93rSlTAu3c9aaFnTX3/ANCXF9lhIeztkbEXub
         77Xg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147166; x=1707751966;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=r6k+h8FV3d3vqiaEMlAmNrvziqvskuI2jnLZD/dQBlM=;
        b=NONUKb2QKp5v0bht11wHzRBX+8CjKY82uD0hhhl12pc+mj7iumUEWLyu24b1vXc93C
         kcgcBwqmxgSYg0LREdjU8pK7OJhwLAAtw1S4aj5A4AkciMP+tJB8h9eIvgY0UmIUvylX
         rf6ys/pT2xSFvN/MVrvydNIXkAXNp4vMiRY7EGwNhFV8/WzI44bLqwVqktxkwuSHoPzR
         0+gxcOkLDm3FsRElyh0/FtoZhzxck4aSxQQ0I2jXuX38DSRh/hTXw9dm7GPizBBX+ycw
         cjsrTG7dlHTGE8/94Koeh6VR6KTmzVRVHCsIkGcJtOPEaOndhQ1JSw0k6FmUCsP4ciLC
         kJNA==
X-Gm-Message-State: AOJu0Yxu0AbUQvZ+cPuQt5gj+n7fnaHXsQgNR89O9uhiRE+VtTpQjJ7r
	MVQ61mIDKPIm+Xuq4yPbHN0WhzdZiZUNWIvwJTBbOJ0FQNf/NN7PnOCrZAFE
X-Google-Smtp-Source: AGHT+IH2pEQL4vZHR4MIEJz68fgv+P3tICF/uAeNOq4jVyksCsAdQPpNvtJllexRR5adepWtKjsiXg==
X-Received: by 2002:ac2:4141:0:b0:511:56cf:5547 with SMTP id c1-20020ac24141000000b0051156cf5547mr341802lfi.1.1707147166303;
        Mon, 05 Feb 2024 07:32:46 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUDAoZxLjfy7HkxTNBi3WUn1ERJyszXw4tprspv4xeX6CniIEbx62uxIUGO2bOy3eu8pVs9iyJ86O+M5hmcFSa3Dk1vJLbHMRtg/aAHExQOCR1vm999z27iGd104XrjDG/+76NqhFh11WTetvT877pp3trNF5xmkeINlC1wqnyELm4+QQdizAZiBzPsg5jMbevc53ljIAKeJ7Q3BpN0CPX8kbcRJnnB5FjZvzpz0R1NP1UalY5MgCy5ykM+Oin/gHEE+67qFCo8cj40bVDh2W39CisiC/WiYg0MoWY+OWJaLhxd0LkYTa/6KZzbWHy4DROOxq1w
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
Date: Mon,  5 Feb 2024 16:32:12 +0100
Message-ID: <542c61229e1056c3fdec3a5c086136e3b5909b07.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - Drop in guest_testop() and guest_bitop() casts of function arguments.
 - Change "commit message" to "commit title" in "Changes in V3" to be more precise about
   what was changed.
 - use BUG_ON("unimplemented") instead of ASSERT_UNREACHABLE
---
Changes in V3:
 - update the commit title
 - drop TODO commit.
 - add ASSERT_UNREACHABLE for stubs guest functions.
 - Add SPDX & footer
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/guest_atomics.h | 44 ++++++++++++++++++++++
 1 file changed, 44 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/guest_atomics.h

diff --git a/xen/arch/riscv/include/asm/guest_atomics.h b/xen/arch/riscv/include/asm/guest_atomics.h
new file mode 100644
index 0000000000..de54914454
--- /dev/null
+++ b/xen/arch/riscv/include/asm/guest_atomics.h
@@ -0,0 +1,44 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_GUEST_ATOMICS_H
+#define __ASM_RISCV_GUEST_ATOMICS_H
+
+#include <xen/bug.h>
+
+#define guest_testop(name)                                                  \
+static inline int guest_##name(struct domain *d, int nr, volatile void *p)  \
+{                                                                           \
+    BUG_ON("unimplemented");                                                \
+                                                                            \
+    return 0;                                                               \
+}
+
+#define guest_bitop(name)                                                   \
+static inline void guest_##name(struct domain *d, int nr, volatile void *p) \
+{                                                                           \
+    BUG_ON("unimplemented");                                                \
+}
+
+guest_bitop(set_bit)
+guest_bitop(clear_bit)
+guest_bitop(change_bit)
+
+#undef guest_bitop
+
+guest_testop(test_and_set_bit)
+guest_testop(test_and_clear_bit)
+guest_testop(test_and_change_bit)
+
+#undef guest_testop
+
+#define guest_test_bit(d, nr, p) ((void)(d), test_bit(nr, p))
+
+#endif /* __ASM_RISCV_GUEST_ATOMICS_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676204.1052243 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xr-0001E8-Dx; Mon, 05 Feb 2024 15:32:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676204.1052243; Mon, 05 Feb 2024 15:32:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xr-0001Dm-8X; Mon, 05 Feb 2024 15:32:55 +0000
Received: by outflank-mailman (input) for mailman id 676204;
 Mon, 05 Feb 2024 15:32:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xp-0007I6-Up
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:53 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d356a980-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:32:52 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-511490772f6so1897755e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:52 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d356a980-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147171; x=1707751971; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iwio54x9hUhe11rwf/yjaXDL3sN8O/ZlCRLr2jTGTZY=;
        b=YiKPcRS4xFOtfvkWcvJEIARKHy4yISd78K9cDcOMFvuaKHlaVQ5XbUqwiCcrzA1P2E
         s2vCaAqGMfKbO+E8ZiNGlC/gS4HEU4+ZH3fDsf9YSmBrrvcIdagogyV5Q429V6CnrKI0
         5c588vcuWyvqRrYoKjUsv6MnNqASLA0mph9qc77MZsiftkc+1Kc1vgU3CLL3Fb6sYcPH
         PU10YNv9qwbJbTaJm6GoGoQbIOG2WD/iwf7z1+UVZJkTl05w3ccrAemTb+rr12EUeXwD
         dJ7j9MZt4mlzRB8Bx02sF3q6qmP8Z5KMO8gBryv6Xkr6ixv5cNyWe9ioENjiDZeJHhNY
         ligw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147171; x=1707751971;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=iwio54x9hUhe11rwf/yjaXDL3sN8O/ZlCRLr2jTGTZY=;
        b=T3J2nD3LVZrsw71BPe9WwMCfmyj6ysIMPXT4M1mMGhDUA32Dih0Mq7XmHcbLcbRl9f
         ZJlwG2SUxvJ4aKXcmJtAF4a2w4uWb10YO/HLifVlGWu3IjJ9D6MEVy3Tt25DDP8Thj8Y
         VHQvuvQunKpKZX4Dd12UNjtOOZvqmojY1H4Zbp2cM3FXvOrN7T8/n+jlqoW83d3nuEM0
         D94we+Yoh3rhi7eoh3gFANGDwr3BGM9BemOVKEmBLWZDdEw2/qF1DT7REZsggl41oOnr
         IWyZ7GKgS2Vm+LioToRvV2kpjCYq8KUyfrG9TjkZdRo2OcXU6uddDBrCtEvovE7P1jPf
         WGaQ==
X-Gm-Message-State: AOJu0YxvY1DdcnhOphsgyolMiszYnsFbajNMa/kAGgWgPxJ/4FiJnZau
	tp72z/aPU/+V1XA65VQ6iLj20/Nqb43MnN9eHfT3J47hdwBa7InqNleyGofa
X-Google-Smtp-Source: AGHT+IFZGdwH2lHHVG2t/xS6L0Xo/4euF9vnU21rET/44ZrxCDcHHerFqoPvtRdlUYOOp0GTM1M7Xw==
X-Received: by 2002:ac2:5b85:0:b0:511:480a:adfd with SMTP id o5-20020ac25b85000000b00511480aadfdmr4253507lfn.51.1707147171368;
        Mon, 05 Feb 2024 07:32:51 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXMLrvdR3TOf3sG6UVBkrDQgQbFc3dXFK7MkHm3juaKt6SaYUZGlZVEm/ETsK+A1U2k4DPZt6nl9cRxNQgs76fc4GRvf6u3uhLTL4RXneFi89If90F8xCQ/72+zFpjgXeVbo1w8TOA20aQthv1YbS0YD7lcdcup3en6Owyu0E/3m8bqn/vSwA0mpgb+auJpU6a0Tf5AHy3bpPhsZDpNz8puqaf8sYHP1ud6srqliQFZYu/0tzQO+WYFXHlLRnXRiCqTMqj1ZgE2+yL6/RT6OFjxNud/2pJYQ01+gvNAnGJU1IkpkjYTt35m4iGaTJuTHQLwN015
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
Date: Mon,  5 Feb 2024 16:32:17 +0100
Message-ID: <0aa76252c83b60952d410b00645eda0b6e0b3be1.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V4:
 - s/BUG/BUG_ON(...)
---
Changes in V3:
 - add SPDX & footer
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/flushtlb.h | 34 +++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/flushtlb.h

diff --git a/xen/arch/riscv/include/asm/flushtlb.h b/xen/arch/riscv/include/asm/flushtlb.h
new file mode 100644
index 0000000000..7ce32bea0b
--- /dev/null
+++ b/xen/arch/riscv/include/asm/flushtlb.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_FLUSHTLB_H__
+#define __ASM_RISCV_FLUSHTLB_H__
+
+#include <xen/bug.h>
+#include <xen/cpumask.h>
+
+/*
+ * Filter the given set of CPUs, removing those that definitely flushed their
+ * TLB since @page_timestamp.
+ */
+/* XXX lazy implementation just doesn't clear anything.... */
+static inline void tlbflush_filter(cpumask_t *mask, uint32_t page_timestamp) {}
+
+#define tlbflush_current_time() (0)
+
+static inline void page_set_tlbflush_timestamp(struct page_info *page)
+{
+    BUG_ON("unimplemented");
+}
+
+/* Flush specified CPUs' TLBs */
+void arch_flush_tlb_mask(const cpumask_t *mask);
+
+#endif /* __ASM_RISCV_FLUSHTLB_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676205.1052252 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xs-0001UB-Qf; Mon, 05 Feb 2024 15:32:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676205.1052252; Mon, 05 Feb 2024 15:32:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xs-0001TJ-I6; Mon, 05 Feb 2024 15:32:56 +0000
Received: by outflank-mailman (input) for mailman id 676205;
 Mon, 05 Feb 2024 15:32:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xq-0007I6-V0
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:55 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d336bf53-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:32:52 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-5111c7d40deso7533965e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:52 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d336bf53-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147171; x=1707751971; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=x10WJ61xCAu2SaWC9hWesUNvYqTdqZDZmiUpwgQkjrI=;
        b=IcYWkWQlRBeefnIFAhvyF+KU2qDZIESYZdRgRMi8Jd6RiAxGakI6K4ambeif9b7WAQ
         VaPMn0KyJPZY5U13e6k7cFj2ldA5k0Rn7dyqyZDpQxJ1EZAmQaMBJIrHyeevbU40X+XS
         gLI84KirE3W5b9/+W8PwWRI1Huc3WhhNGlwWXDhM4YlNbkxBPRkfVxzvW+SPowzjjc1G
         1xKU8iqUUrh7JNiko5wvvw1AHUKykaL+Mquiy3LcHpnhYUo+CVBdKeRGuJNOaBfCl+mN
         CS1Dixywbw/O1WZFitDXYFbujdEVSsPesOW0TAGq7b65DqT4ATqSQgSiwOqjzk9Owrs3
         Z8pQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147171; x=1707751971;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=x10WJ61xCAu2SaWC9hWesUNvYqTdqZDZmiUpwgQkjrI=;
        b=iB5VVAaikaxsv6LmMzDeatkEKDaepuz/AqpuC/mImX5AZAZRwv3Co9yp6EYUSsk8OC
         d5IWpgqo0pxuO5/yKct/bsLwJrwZKhdYP38v1yGdN5YXmQpmDZz3uH8YDu5YlqPsN1xk
         XaXjU2V7K67s/x8wcesR37ftL3wE0Mls9CtumtSDCTyHx40qyYaoXdIV/kQe+LNe0DWF
         4mOM8Ve1FmhPAbiup5FK87FTOMF63+qH6oMZoMFKBj4Yrcs+dj8+Dwb8qlNej2NnV7QF
         Sf4PvLCFV8rcQEurVgqy/CfKSonni1+cemRX4MSePTqMfx8TD+jIv7JzSLjXMpde476B
         bXgg==
X-Gm-Message-State: AOJu0Yw/LfeDdRo0BPkFgc9rrsnGvYP35zAzNUg0MLr/qD9CSatNgqDN
	RB9QtrZyRsj21d+7tm7o9iWatwGsrSrgAzAE0cHypkb+pgr5aC7vU0rdxU6G
X-Google-Smtp-Source: AGHT+IHauYIzyY/SreFYiH+TO2/dV5e2WfFB9n/UzR1q4AS1BvCvidzDxNuLmu9yk9dFL7oF1+9FvQ==
X-Received: by 2002:a05:6512:67:b0:511:56c0:695 with SMTP id i7-20020a056512006700b0051156c00695mr396831lfo.51.1707147170525;
        Mon, 05 Feb 2024 07:32:50 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWuf/VbPOVxho42QWsXirVtyn+9P1tq1GL9spedb6hvSRzDSJpKHJiubL/qp1bQtilTIicwPkoCli3DPy2F7bd0lGCWRrDAB3zveAxc5DDB/bbD6y+cylM/KC34M18OoTmkLDjzxr81/dlCYPO0zvo+EEUdKtkX6N073HNBEQNBZhAp9XWWo+Z7RSTVWU8mC1fynoDzINmSH1NRsrccbFdZGMWiEzE4x17/uXGQucEi9xESWIgYFwSPwHA26pBd5XDPiF5zLqvYX4AI4ECX//1YLykP+DkYBojdiE0GsocOuX5HT1grVujqdmlhLSoKPJIevP+F
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 09/30] xen/riscv: introduce bitops.h
Date: Mon,  5 Feb 2024 16:32:16 +0100
Message-ID: <f76bd85f4b64a47c59c0b306ce425036819fa380.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Taken from Linux-6.4.0-rc1

Xen's bitops.h consists of several Linux's headers:
* linux/arch/include/asm/bitops.h:
  * The following function were removed as they aren't used in Xen:
      	* test_and_set_bit_lock
      	* clear_bit_unlock
      	* __clear_bit_unlock
   * The following functions were renamed in the way how they are
     used by common code:
     	* __test_and_set_bit
     	* __test_and_clear_bit
   * The declaration and implementation of the following functios
     were updated to make Xen build happy:
	* clear_bit
	* set_bit
	* __test_and_clear_bit
	* __test_and_set_bit
* linux/arch/include/linux/bits.h ( taken only definitions for BIT_MASK,
  BIT_WORD, BITS_PER_BYTE )
* linux/include/asm-generic/bitops/generic-non-atomic.h with the
  following changes:
   * Only functions that can be reused in Xen were left;
     others were removed.
   * it was updated the message inside #ifndef ... #endif.
   * __always_inline -> always_inline to be align with definition in
     xen/compiler.h.
   * update function prototypes from
     generic___test_and_*(unsigned long nr nr, volatile unsigned long *addr)
     to
     generic___test_and_*(unsigned long nr, volatile void *addr) to be
     consistent with other related macros/defines.
   * convert identations from tabs to spaces.
   * inside generic__test_and_* use 'bitops_uint_t' instead of 'unsigned long'
     to be generic.

Additionaly, the following bit ops are introduced:
* __ffs
* ffsl
* fls
* flsl
* ffs
* ffz
* find_first_bit_set
* hweight64
* test_bit

Some of the introduced bit operations are included in asm-generic,
as they exhibit similarity across multiple architectures.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
  - updated the commit message: dropped the message about what was taken from linux/include/asm-generic/bitops/find.h
    as related changes now are located in xen/bitops.h. Also these changes were removed from riscv/bitops.h
  - switch tabs to spaces.
  - update return type of __ffs function, format __ffs according to Xen code style. Move the function to
    respective asm-generic header.
  - format ffsl() according to Xen code style, update the type of num: int -> unsigned to be align with
    return type of the function. Move the function to respective asm-generic header.
  - add new line for the files:
      asm-generic/bitops-bits.h
      asm-generic/ffz.h
      asm-generic/find-first-bit-set.h
      asm-generic/fls.h
      asm-generic/flsl.h
      asm-generic/test-bit.h
  - rename asm-generic/find-first-bit-set.h to asm-generic/find-first-set-bit.h to be aligned with the function
    name implemented inside.
  - introduce generic___test_and*() operation for non-atomic bitops.
  - rename current __test_and_*() -> test_and_*() as their implementation are atomic aware.
  - define __test_and_*() to generic___test_and_*().
  - introduce test_and_change_bit().
  - update asm-generic/bitops/bitops-bits.h to give possoibility to change BITOP_*() macros by architecture.
    Also, it was introduced bitops_uint_t type to make generic___test_and_*() generic.
  - "include asm-generic/bitops/bitops-bits.h" to files which use its definitions.
  - add comment why generic ffz is defined as __ffs().
  - update the commit message.
  - swtich ffsl() to generic_ffsl().
---
Changes in V3:
 - update the commit message
 - Introduce the following asm-generic bitops headers:
	create mode 100644 xen/arch/riscv/include/asm/bitops.h
	create mode 100644 xen/include/asm-generic/bitops/bitops-bits.h
	create mode 100644 xen/include/asm-generic/bitops/ffs.h
	create mode 100644 xen/include/asm-generic/bitops/ffz.h
	create mode 100644 xen/include/asm-generic/bitops/find-first-bit-set.h
	create mode 100644 xen/include/asm-generic/bitops/fls.h
	create mode 100644 xen/include/asm-generic/bitops/flsl.h
	create mode 100644 xen/include/asm-generic/bitops/hweight.h
	create mode 100644 xen/include/asm-generic/bitops/test-bit.h
 - switch some bitops functions to asm-generic's versions.
 - re-sync some macros with Linux kernel version mentioned in the commit message.
 - Xen code style fixes.
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/bitops.h           | 164 ++++++++++++++++++
 xen/arch/riscv/include/asm/config.h           |   2 +
 xen/include/asm-generic/bitops/__ffs.h        |  47 +++++
 xen/include/asm-generic/bitops/bitops-bits.h  |  21 +++
 xen/include/asm-generic/bitops/ffs.h          |   9 +
 xen/include/asm-generic/bitops/ffsl.h         |  16 ++
 xen/include/asm-generic/bitops/ffz.h          |  18 ++
 .../asm-generic/bitops/find-first-set-bit.h   |  17 ++
 xen/include/asm-generic/bitops/fls.h          |  18 ++
 xen/include/asm-generic/bitops/flsl.h         |  10 ++
 .../asm-generic/bitops/generic-non-atomic.h   |  89 ++++++++++
 xen/include/asm-generic/bitops/hweight.h      |  13 ++
 xen/include/asm-generic/bitops/test-bit.h     |  18 ++
 13 files changed, 442 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h
 create mode 100644 xen/include/asm-generic/bitops/__ffs.h
 create mode 100644 xen/include/asm-generic/bitops/bitops-bits.h
 create mode 100644 xen/include/asm-generic/bitops/ffs.h
 create mode 100644 xen/include/asm-generic/bitops/ffsl.h
 create mode 100644 xen/include/asm-generic/bitops/ffz.h
 create mode 100644 xen/include/asm-generic/bitops/find-first-set-bit.h
 create mode 100644 xen/include/asm-generic/bitops/fls.h
 create mode 100644 xen/include/asm-generic/bitops/flsl.h
 create mode 100644 xen/include/asm-generic/bitops/generic-non-atomic.h
 create mode 100644 xen/include/asm-generic/bitops/hweight.h
 create mode 100644 xen/include/asm-generic/bitops/test-bit.h

diff --git a/xen/arch/riscv/include/asm/bitops.h b/xen/arch/riscv/include/asm/bitops.h
new file mode 100644
index 0000000000..1225298d35
--- /dev/null
+++ b/xen/arch/riscv/include/asm/bitops.h
@@ -0,0 +1,164 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#include <asm-generic/bitops/bitops-bits.h>
+
+/* Based on linux/arch/include/linux/bits.h */
+
+#define BIT_MASK(nr)        (1UL << ((nr) % BITS_PER_LONG))
+#define BIT_WORD(nr)        ((nr) / BITS_PER_LONG)
+
+#define __set_bit(n,p)      set_bit(n,p)
+#define __clear_bit(n,p)    clear_bit(n,p)
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+#if ( BITS_PER_LONG == 64 )
+#define __AMO(op)   "amo" #op ".d"
+#elif ( BITS_PER_LONG == 32 )
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITS_PER_LONG"
+#endif
+
+#define __test_and_op_bit_ord(op, mod, nr, addr, ord)   \
+({                                                      \
+    unsigned long __res, __mask;                        \
+    __mask = BIT_MASK(nr);                              \
+    __asm__ __volatile__ (                              \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (__res), "+A" (addr[BIT_WORD(nr)])       \
+        : "r" (mod(__mask))                             \
+        : "memory");                                    \
+    ((__res & __mask) != 0);                            \
+})
+
+#define __op_bit_ord(op, mod, nr, addr, ord)    \
+    __asm__ __volatile__ (                      \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BIT_WORD(nr)])             \
+        : "r" (mod(BIT_MASK(nr)))               \
+        : "memory");
+
+#define __test_and_op_bit(op, mod, nr, addr)    \
+    __test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define __op_bit(op, mod, nr, addr) \
+    __op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define __NOP(x)    (x)
+#define __NOT(x)    (~(x))
+
+/**
+ * __test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ *
+ * This operation may be reordered on other architectures than x86.
+ */
+static inline int test_and_set_bit(int nr, volatile void *p)
+{
+    volatile uint32_t *addr = p;
+
+    return __test_and_op_bit(or, __NOP, nr, addr);
+}
+
+/**
+ * __test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ *
+ * This operation can be reordered on other architectures other than x86.
+ */
+static inline int test_and_clear_bit(int nr, volatile void *p)
+{
+    volatile uint32_t *addr = p;
+
+    return __test_and_op_bit(and, __NOT, nr, addr);
+}
+
+/**
+ * set_bit - Atomically set a bit in memory
+ * @nr: the bit to set
+ * @addr: the address to start counting from
+ *
+ * Note: there are no guarantees that this function will not be reordered
+ * on non x86 architectures, so if you are writing portable code,
+ * make sure not to rely on its reordering guarantees.
+ *
+ * Note that @nr may be almost arbitrarily large; this function is not
+ * restricted to acting on a single-word quantity.
+ */
+static inline void set_bit(int nr, volatile void *p)
+{
+    volatile uint32_t *addr = p;
+
+    __op_bit(or, __NOP, nr, addr);
+}
+
+/**
+ * clear_bit - Clears a bit in memory
+ * @nr: Bit to clear
+ * @addr: Address to start counting from
+ *
+ * Note: there are no guarantees that this function will not be reordered
+ * on non x86 architectures, so if you are writing portable code,
+ * make sure not to rely on its reordering guarantees.
+ */
+static inline void clear_bit(int nr, volatile void *p)
+{
+    volatile uint32_t *addr = p;
+
+    __op_bit(and, __NOT, nr, addr);
+}
+
+/**
+ * test_and_change_bit - Change a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is atomic and cannot be reordered.
+ * It also implies a memory barrier.
+ */
+static inline int test_and_change_bit(int nr, volatile unsigned long *addr)
+{
+	return __test_and_op_bit(xor, __NOP, nr, addr);
+}
+
+#undef __test_and_op_bit
+#undef __op_bit
+#undef __NOP
+#undef __NOT
+#undef __AMO
+
+#include <asm-generic/bitops/generic-non-atomic.h>
+
+#define __test_and_set_bit generic___test_and_set_bit
+#define __test_and_clear_bit generic___test_and_clear_bit
+#define __test_and_change_bit generic___test_and_change_bit
+
+#include <asm-generic/bitops/fls.h>
+#include <asm-generic/bitops/flsl.h>
+#include <asm-generic/bitops/__ffs.h>
+#include <asm-generic/bitops/ffs.h>
+#include <asm-generic/bitops/ffsl.h>
+#include <asm-generic/bitops/ffz.h>
+#include <asm-generic/bitops/find-first-set-bit.h>
+#include <asm-generic/bitops/hweight.h>
+#include <asm-generic/bitops/test-bit.h>
+
+#endif /* _ASM_RISCV_BITOPS_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/riscv/include/asm/config.h b/xen/arch/riscv/include/asm/config.h
index a80cdd4f85..56387ac159 100644
--- a/xen/arch/riscv/include/asm/config.h
+++ b/xen/arch/riscv/include/asm/config.h
@@ -50,6 +50,8 @@
 # error "Unsupported RISCV variant"
 #endif
 
+#define BITS_PER_BYTE 8
+
 #define BYTES_PER_LONG (1 << LONG_BYTEORDER)
 #define BITS_PER_LONG  (BYTES_PER_LONG << 3)
 #define POINTER_ALIGN  BYTES_PER_LONG
diff --git a/xen/include/asm-generic/bitops/__ffs.h b/xen/include/asm-generic/bitops/__ffs.h
new file mode 100644
index 0000000000..fecb4484d9
--- /dev/null
+++ b/xen/include/asm-generic/bitops/__ffs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS___FFS_H_
+#define _ASM_GENERIC_BITOPS___FFS_H_
+
+/**
+ * ffs - find first bit in word.
+ * @word: The word to search
+ *
+ * Returns 0 if no bit exists, otherwise returns 1-indexed bit location.
+ */
+static inline unsigned int __ffs(unsigned long word)
+{
+    unsigned int num = 0;
+
+#if BITS_PER_LONG == 64
+    if ( (word & 0xffffffff) == 0 )
+    {
+        num += 32;
+        word >>= 32;
+    }
+#endif
+    if ( (word & 0xffff) == 0 )
+    {
+        num += 16;
+        word >>= 16;
+    }
+    if ( (word & 0xff) == 0 )
+    {
+        num += 8;
+        word >>= 8;
+    }
+    if ( (word & 0xf) == 0 )
+    {
+        num += 4;
+        word >>= 4;
+    }
+    if ( (word & 0x3) == 0 )
+    {
+        num += 2;
+        word >>= 2;
+    }
+    if ( (word & 0x1) == 0 )
+        num += 1;
+    return num;
+}
+
+#endif /* _ASM_GENERIC_BITOPS___FFS_H_ */
diff --git a/xen/include/asm-generic/bitops/bitops-bits.h b/xen/include/asm-generic/bitops/bitops-bits.h
new file mode 100644
index 0000000000..4ece2affd6
--- /dev/null
+++ b/xen/include/asm-generic/bitops/bitops-bits.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_BITS_H_
+#define _ASM_GENERIC_BITOPS_BITS_H_
+
+#ifndef BITOP_BITS_PER_WORD
+#define BITOP_BITS_PER_WORD     32
+#endif
+
+#ifndef BITOP_MASK
+#define BITOP_MASK(nr)          (1U << ((nr) % BITOP_BITS_PER_WORD))
+#endif
+
+#ifndef BITOP_WORD
+#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
+#endif
+
+#ifndef BITOP_TYPE
+typedef uint32_t bitops_uint_t;
+#endif
+
+#endif /* _ASM_GENERIC_BITOPS_BITS_H_ */
diff --git a/xen/include/asm-generic/bitops/ffs.h b/xen/include/asm-generic/bitops/ffs.h
new file mode 100644
index 0000000000..3f75fded14
--- /dev/null
+++ b/xen/include/asm-generic/bitops/ffs.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FFS_H_
+#define _ASM_GENERIC_BITOPS_FFS_H_
+
+#include <xen/macros.h>
+
+#define ffs(x) ({ unsigned int t_ = (x); fls(ISOLATE_LSB(t_)); })
+
+#endif /* _ASM_GENERIC_BITOPS_FFS_H_ */
diff --git a/xen/include/asm-generic/bitops/ffsl.h b/xen/include/asm-generic/bitops/ffsl.h
new file mode 100644
index 0000000000..d0996808f5
--- /dev/null
+++ b/xen/include/asm-generic/bitops/ffsl.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FFSL_H_
+#define _ASM_GENERIC_BITOPS_FFSL_H_
+
+/**
+ * ffsl - find first bit in long.
+ * @word: The word to search
+ *
+ * Returns 0 if no bit exists, otherwise returns 1-indexed bit location.
+ */
+static inline unsigned int ffsl(unsigned long word)
+{
+    return generic_ffsl(word);
+}
+
+#endif /* _ASM_GENERIC_BITOPS_FFSL_H_ */
diff --git a/xen/include/asm-generic/bitops/ffz.h b/xen/include/asm-generic/bitops/ffz.h
new file mode 100644
index 0000000000..5932fe6695
--- /dev/null
+++ b/xen/include/asm-generic/bitops/ffz.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FFZ_H_
+#define _ASM_GENERIC_BITOPS_FFZ_H_
+
+/*
+ * ffz - find first zero in word.
+ * @word: The word to search
+ *
+ * Undefined if no zero exists, so code should check against ~0UL first.
+ *
+ * ffz() is defined as __ffs() and not as ffs() as it is defined in such
+ * a way in Linux kernel (6.4.0 ) from where this header was taken, so this
+ * header is supposed to be aligned with Linux kernel version.
+ * Also, most architectures are defined in the same way in Xen.
+ */
+#define ffz(x)  __ffs(~(x))
+
+#endif /* _ASM_GENERIC_BITOPS_FFZ_H_ */
diff --git a/xen/include/asm-generic/bitops/find-first-set-bit.h b/xen/include/asm-generic/bitops/find-first-set-bit.h
new file mode 100644
index 0000000000..7d28b8a89b
--- /dev/null
+++ b/xen/include/asm-generic/bitops/find-first-set-bit.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FIND_FIRST_SET_BIT_H_
+#define _ASM_GENERIC_BITOPS_FIND_FIRST_SET_BIT_H_
+
+/**
+ * find_first_set_bit - find the first set bit in @word
+ * @word: the word to search
+ *
+ * Returns the bit-number of the first set bit (first bit being 0).
+ * The input must *not* be zero.
+ */
+static inline unsigned int find_first_set_bit(unsigned long word)
+{
+        return ffsl(word) - 1;
+}
+
+#endif /* _ASM_GENERIC_BITOPS_FIND_FIRST_SET_BIT_H_ */
diff --git a/xen/include/asm-generic/bitops/fls.h b/xen/include/asm-generic/bitops/fls.h
new file mode 100644
index 0000000000..369a4c790c
--- /dev/null
+++ b/xen/include/asm-generic/bitops/fls.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FLS_H_
+#define _ASM_GENERIC_BITOPS_FLS_H_
+
+/**
+ * fls - find last (most-significant) bit set
+ * @x: the word to search
+ *
+ * This is defined the same way as ffs.
+ * Note fls(0) = 0, fls(1) = 1, fls(0x80000000) = 32.
+ */
+
+static inline int fls(unsigned int x)
+{
+    return generic_fls(x);
+}
+
+#endif /* _ASM_GENERIC_BITOPS_FLS_H_ */
diff --git a/xen/include/asm-generic/bitops/flsl.h b/xen/include/asm-generic/bitops/flsl.h
new file mode 100644
index 0000000000..d0a2e9c729
--- /dev/null
+++ b/xen/include/asm-generic/bitops/flsl.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FLSL_H_
+#define _ASM_GENERIC_BITOPS_FLSL_H_
+
+static inline int flsl(unsigned long x)
+{
+    return generic_flsl(x);
+}
+
+#endif /* _ASM_GENERIC_BITOPS_FLSL_H_ */
diff --git a/xen/include/asm-generic/bitops/generic-non-atomic.h b/xen/include/asm-generic/bitops/generic-non-atomic.h
new file mode 100644
index 0000000000..07efca245e
--- /dev/null
+++ b/xen/include/asm-generic/bitops/generic-non-atomic.h
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * The file is based on Linux ( 6.4.0 ) header:
+ *   include/asm-generic/bitops/generic-non-atomic.h
+ * 
+ * Only functions that can be reused in Xen were left; others were removed.
+ * 
+ * Also, the following changes were done:
+ *  - it was updated the message inside #ifndef ... #endif.
+ *  - __always_inline -> always_inline to be align with definition in
+ *    xen/compiler.h.
+ *  - update function prototypes from
+ *    generic___test_and_*(unsigned long nr nr, volatile unsigned long *addr) to
+ *    generic___test_and_*(unsigned long nr, volatile void *addr) to be
+ *    consistent with other related macros/defines.
+ *  - convert identations from tabs to spaces.
+ *  - inside generic__test_and_* use 'bitops_uint_t' instead of 'unsigned long'
+ *    to be generic.
+ */
+
+#ifndef __ASM_GENERIC_BITOPS_GENERIC_NON_ATOMIC_H
+#define __ASM_GENERIC_BITOPS_GENERIC_NON_ATOMIC_H
+
+#include <xen/compiler.h>
+
+#include <asm-generic/bitops/bitops-bits.h>
+
+#ifndef _LINUX_BITOPS_H
+#error only <xen/bitops.h> can be included directly
+#endif
+
+/*
+ * Generic definitions for bit operations, should not be used in regular code
+ * directly.
+ */
+
+/**
+ * generic___test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic___test_and_set_bit(unsigned long nr, volatile void *addr)
+{
+    bitops_uint_t mask = BIT_MASK(nr);
+    bitops_uint_t *p = ((bitops_uint_t *)addr) + BIT_WORD(nr);
+    bitops_uint_t old = *p;
+
+    *p = old | mask;
+    return (old & mask) != 0;
+}
+
+/**
+ * generic___test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic___test_and_clear_bit(bitops_uint_t nr, volatile void *addr)
+{
+    bitops_uint_t mask = BIT_MASK(nr);
+    bitops_uint_t *p = ((bitops_uint_t *)addr) + BIT_WORD(nr);
+    bitops_uint_t old = *p;
+
+    *p = old & ~mask;
+    return (old & mask) != 0;
+}
+
+/* WARNING: non atomic and it can be reordered! */
+static always_inline bool
+generic___test_and_change_bit(unsigned long nr, volatile void *addr)
+{
+    bitops_uint_t mask = BIT_MASK(nr);
+    bitops_uint_t *p = ((bitops_uint_t *)addr) + BIT_WORD(nr);
+    bitops_uint_t old = *p;
+
+    *p = old ^ mask;
+    return (old & mask) != 0;
+}
+
+#endif /* __ASM_GENERIC_BITOPS_GENERIC_NON_ATOMIC_H */
diff --git a/xen/include/asm-generic/bitops/hweight.h b/xen/include/asm-generic/bitops/hweight.h
new file mode 100644
index 0000000000..0d7577054e
--- /dev/null
+++ b/xen/include/asm-generic/bitops/hweight.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_HWEIGHT_H_
+#define _ASM_GENERIC_BITOPS_HWEIGHT_H_
+
+/*
+ * hweightN - returns the hamming weight of a N-bit word
+ * @x: the word to weigh
+ *
+ * The Hamming Weight of a number is the total number of bits set in it.
+ */
+#define hweight64(x) generic_hweight64(x)
+
+#endif /* _ASM_GENERIC_BITOPS_HWEIGHT_H_ */
diff --git a/xen/include/asm-generic/bitops/test-bit.h b/xen/include/asm-generic/bitops/test-bit.h
new file mode 100644
index 0000000000..6fb414d808
--- /dev/null
+++ b/xen/include/asm-generic/bitops/test-bit.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_TESTBIT_H_
+#define _ASM_GENERIC_BITOPS_TESTBIT_H_
+
+#include <asm-generic/bitops/bitops-bits.h>
+
+/**
+ * test_bit - Determine whether a bit is set
+ * @nr: bit number to test
+ * @addr: Address to start counting from
+ */
+static inline int test_bit(int nr, const volatile void *addr)
+{
+    const volatile bitops_uint_t *p = addr;
+    return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD - 1)));
+}
+
+#endif /* _ASM_GENERIC_BITOPS_TESTBIT_H_ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676206.1052262 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xu-0001rG-Fo; Mon, 05 Feb 2024 15:32:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676206.1052262; Mon, 05 Feb 2024 15:32:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xu-0001pe-9z; Mon, 05 Feb 2024 15:32:58 +0000
Received: by outflank-mailman (input) for mailman id 676206;
 Mon, 05 Feb 2024 15:32:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xr-0007Hv-Hn
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:55 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d4d97775-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:32:54 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-511413e52d4so2567629e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:54 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d4d97775-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147174; x=1707751974; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=yd/D5KaXlx6vAlR/DP81B7tIQ/qT+Y2h20i78E/WLDg=;
        b=QCZdrNgrS5sBYrAY+mi6VZtZ8hObK27cLVEjSNf+xBTqFV7b38xGMoQ8OJvsS65Ihh
         fVGHQSduci/l9vftwCcfjq0lCZnvnKtvt7EKK1fGt3UeEZWik3+MZjBCrl32dkLZ98pU
         RG/LBatFPqD7luoQZxWFfp+NPrfrjv3v56sn+gMCUH210wOhJOzKGCRreVLOU3TdSiCI
         oSKEOnAOFaYVWZzqtSjCgVR71j9v0yXENhNYY68MpeGj5yZzqXOzAJlczx8lTzRv2GsW
         OEzOXqIAuwfVG0ENf2I6R/1+D+QP2EcBhNC/7iHgPsGq6KBiEIr2YbioRbxOtBOoOATv
         CCrA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147174; x=1707751974;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=yd/D5KaXlx6vAlR/DP81B7tIQ/qT+Y2h20i78E/WLDg=;
        b=txPOsLnQSERRWGFdSIUz4Zg/LOnDQJRBug/RRcjoWMit+ufqv42Y4quX04ARMarSNn
         5lsbT2wKoJ+SAAUOtCIZPR3MiSsUnCSn8TNfLMKoqkqD90QBoG8Ugytlt3gjTyB507XY
         woYiNictvWl6UTIIV09HxX1VsBfkkZ+mKkRB8xt7cxlB1QKEkTgXdMZJwh5nEh7yLAss
         mvICa7KNRrJ5CPH5sIuQtEInkDWAaxUUu/chXEKPnFLLM4kDcmTMlCwU9iX7Jh1t+T1I
         0sfUdCgRV7E08ybZUjdVtC18xLmUYzcN4Yqp5bxUpMq8KTf+h7fo3LVGXJxDel9HpSa2
         mlkw==
X-Gm-Message-State: AOJu0YwishMIX0092W269kl1DmVCsvkUGfIkV2jbt29FMqsectsAuTKI
	3kGGrDPndMAXzJ5niA98CIdPLJL8llAJTtFGJHS0XGJXHyOKWrzJQwkgz7eH
X-Google-Smtp-Source: AGHT+IHvXEiNZHWqya7iE9QdE2B0AV78daWPxqCRsZsXBuoNhkLis/5I/7wpWYR/PjNzgQ183HXw5g==
X-Received: by 2002:a05:6512:38a2:b0:50f:18f7:855e with SMTP id o2-20020a05651238a200b0050f18f7855emr7997617lft.39.1707147174216;
        Mon, 05 Feb 2024 07:32:54 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUrm07S+ZfxtX5MVJHn3PsHlgEtGQmHiXSobCqh/I+VR62nJIKm1bE6ufQD8xujoZpT0TK/rwboGJGfRVmk9xdeUu62LTmRbn2mr3m97AjMDPSVR7ZiOsxo1cQ6zpLpgXJkmUzKvI74qwbMiH6mnGbtTCgqupwjGFccSALSaYx+kly9rDXvDVVcxEm7gIaTWPADpmt8PupnMUFm8X6U0KqYj2gvT1q+qZyIa+id81n7fj0j324Pk44DNDUS1aYK5pvQTyJXTQoZpZrSmqNuek+UwsS7w5iOIJDhnn06Qhv2vFAI12E+sBdfmzJl3U+OGHhmqfL7
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Date: Mon,  5 Feb 2024 16:32:19 +0100
Message-ID: <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The header was taken from Linux kernl 6.4.0-rc1.

Addionally, were updated:
* add emulation of {cmp}xchg for 1/2 byte types
* replace tabs with spaces
* replace __* varialbed with *__
* introduce generic version of xchg_* and cmpxchg_*.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - Code style fixes.
 - enforce in __xchg_*() has the same type for new and *ptr, also "\n"
   was removed at the end of asm instruction.
 - dependency from https://lore.kernel.org/xen-devel/cover.1706259490.git.federico.serafini@bugseng.com/
 - switch from ASSERT_UNREACHABLE to STATIC_ASSERT_UNREACHABLE().
 - drop xchg32(ptr, x) and xchg64(ptr, x) as they aren't used.
 - drop cmpxcg{32,64}_{local} as they aren't used.
 - introduce generic version of xchg_* and cmpxchg_*.
 - update the commit message.
---
Changes in V3:
 - update the commit message
 - add emulation of {cmp}xchg_... for 1 and 2 bytes types
---
Changes in V2:
 - update the comment at the top of the header.
 - change xen/lib.h to xen/bug.h.
 - sort inclusion of headers properly.
---
 xen/arch/riscv/include/asm/cmpxchg.h | 237 +++++++++++++++++++++++++++
 1 file changed, 237 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h

diff --git a/xen/arch/riscv/include/asm/cmpxchg.h b/xen/arch/riscv/include/asm/cmpxchg.h
new file mode 100644
index 0000000000..b751a50cbf
--- /dev/null
+++ b/xen/arch/riscv/include/asm/cmpxchg.h
@@ -0,0 +1,237 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2014 Regents of the University of California */
+
+#ifndef _ASM_RISCV_CMPXCHG_H
+#define _ASM_RISCV_CMPXCHG_H
+
+#include <xen/compiler.h>
+#include <xen/lib.h>
+
+#include <asm/fence.h>
+#include <asm/io.h>
+#include <asm/system.h>
+
+#define ALIGN_DOWN(addr, size)  ((addr) & (~((size) - 1)))
+
+#define __amoswap_generic(ptr, new, ret, sfx, release_barrier, acquire_barrier) \
+({ \
+    asm volatile( \
+        release_barrier \
+        " amoswap" sfx " %0, %2, %1\n" \
+        acquire_barrier \
+        : "=r" (ret), "+A" (*ptr) \
+        : "r" (new) \
+        : "memory" ); \
+})
+
+#define emulate_xchg_1_2(ptr, new, ret, release_barrier, acquire_barrier) \
+({ \
+    uint32_t *ptr_32b_aligned = (uint32_t *)ALIGN_DOWN((unsigned long)ptr, 4); \
+    uint8_t mask_l = ((unsigned long)(ptr) & (0x8 - sizeof(*ptr))) * BITS_PER_BYTE; \
+    uint8_t mask_size = sizeof(*ptr) * BITS_PER_BYTE; \
+    uint8_t mask_h = mask_l + mask_size - 1; \
+    unsigned long mask = GENMASK(mask_h, mask_l); \
+    unsigned long new_ = (unsigned long)(new) << mask_l; \
+    unsigned long ret_; \
+    unsigned long rc; \
+    \
+    asm volatile( \
+        release_barrier \
+        "0: lr.d %0, %2\n" \
+        "   and  %1, %0, %z4\n" \
+        "   or   %1, %1, %z3\n" \
+        "   sc.d %1, %1, %2\n" \
+        "   bnez %1, 0b\n" \
+        acquire_barrier \
+        : "=&r" (ret_), "=&r" (rc), "+A" (*ptr_32b_aligned) \
+        : "rJ" (new_), "rJ" (~mask) \
+        : "memory"); \
+    \
+    ret = (__typeof__(*(ptr)))((ret_ & mask) >> mask_l); \
+})
+
+#define __xchg_generic(ptr, new, size, sfx, release_barrier, acquire_barrier) \
+({ \
+    __typeof__(ptr) ptr__ = (ptr); \
+    __typeof__(*(ptr)) new__ = (new); \
+    __typeof__(*(ptr)) ret__; \
+    switch (size) \
+    { \
+    case 1: \
+    case 2: \
+        emulate_xchg_1_2(ptr__, new__, ret__, release_barrier, acquire_barrier); \
+        break; \
+    case 4: \
+        __amoswap_generic(ptr__, new__, ret__,\
+                          ".w" sfx,  release_barrier, acquire_barrier); \
+        break; \
+    case 8: \
+        __amoswap_generic(ptr__, new__, ret__,\
+                          ".d" sfx,  release_barrier, acquire_barrier); \
+        break; \
+    default: \
+        STATIC_ASSERT_UNREACHABLE(); \
+    } \
+    ret__; \
+})
+
+#define xchg_relaxed(ptr, x) \
+({ \
+    __typeof__(*(ptr)) x_ = (x); \
+    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)), "", "", ""); \
+})
+
+#define xchg_acquire(ptr, x) \
+({ \
+    __typeof__(*(ptr)) x_ = (x); \
+    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)), \
+                                       "", "", RISCV_ACQUIRE_BARRIER); \
+})
+
+#define xchg_release(ptr, x) \
+({ \
+    __typeof__(*(ptr)) x_ = (x); \
+    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)),\
+                                       "", RISCV_RELEASE_BARRIER, ""); \
+})
+
+#define xchg(ptr,x) \
+({ \
+    __typeof__(*(ptr)) ret__; \
+    ret__ = (__typeof__(*(ptr))) \
+            __xchg_generic(ptr, (unsigned long)(x), sizeof(*(ptr)), \
+                           ".aqrl", "", ""); \
+    ret__; \
+})
+
+#define __generic_cmpxchg(ptr, old, new, ret, lr_sfx, sc_sfx, release_barrier, acquire_barrier)	\
+ ({ \
+    register unsigned int rc; \
+    asm volatile( \
+        release_barrier \
+        "0: lr" lr_sfx " %0, %2\n" \
+        "   bne  %0, %z3, 1f\n" \
+        "   sc" sc_sfx " %1, %z4, %2\n" \
+        "   bnez %1, 0b\n" \
+        acquire_barrier \
+        "1:\n" \
+        : "=&r" (ret), "=&r" (rc), "+A" (*ptr) \
+        : "rJ" (old), "rJ" (new) \
+        : "memory"); \
+ })
+
+#define emulate_cmpxchg_1_2(ptr, old, new, ret, sc_sfx, release_barrier, acquire_barrier) \
+({ \
+    uint32_t *ptr_32b_aligned = (uint32_t *)ALIGN_DOWN((unsigned long)ptr, 4); \
+    uint8_t mask_l = ((unsigned long)(ptr) & (0x8 - sizeof(*ptr))) * BITS_PER_BYTE; \
+    uint8_t mask_size = sizeof(*ptr) * BITS_PER_BYTE; \
+    uint8_t mask_h = mask_l + mask_size - 1; \
+    unsigned long mask = GENMASK(mask_h, mask_l); \
+    unsigned long old_ = (unsigned long)(old) << mask_l; \
+    unsigned long new_ = (unsigned long)(new) << mask_l; \
+    unsigned long ret_; \
+    unsigned long rc; \
+    \
+    __asm__ __volatile__ ( \
+        release_barrier \
+        "0: lr.d %0, %2\n" \
+        "   and  %1, %0, %z5\n" \
+        "   bne  %1, %z3, 1f\n" \
+        "   and  %1, %0, %z6\n" \
+        "   or   %1, %1, %z4\n" \
+        "   sc.d" sc_sfx " %1, %1, %2\n" \
+        "   bnez %1, 0b\n" \
+        acquire_barrier \
+        "1:\n" \
+        : "=&r" (ret_), "=&r" (rc), "+A" (*ptr_32b_aligned) \
+        : "rJ" (old_), "rJ" (new_), \
+          "rJ" (mask), "rJ" (~mask) \
+        : "memory"); \
+    \
+    ret = (__typeof__(*(ptr)))((ret_ & mask) >> mask_l); \
+})
+
+/*
+ * Atomic compare and exchange.  Compare OLD with MEM, if identical,
+ * store NEW in MEM.  Return the initial value in MEM.  Success is
+ * indicated by comparing RETURN with OLD.
+ */
+#define __cmpxchg_generic(ptr, old, new, size, sc_sfx, release_barrier, acquire_barrier) \
+({ \
+    __typeof__(ptr) ptr__ = (ptr); \
+    __typeof__(*(ptr)) old__ = (__typeof__(*(ptr)))(old); \
+    __typeof__(*(ptr)) new__ = (__typeof__(*(ptr)))(new); \
+    __typeof__(*(ptr)) ret__; \
+    switch (size) \
+    { \
+    case 1: \
+    case 2: \
+        emulate_cmpxchg_1_2(ptr, old, new, ret__,\
+                            sc_sfx, release_barrier, acquire_barrier); \
+        break; \
+    case 4: \
+        __generic_cmpxchg(ptr__, old__, new__, ret__, \
+                          ".w", ".w"sc_sfx, release_barrier, acquire_barrier); \
+        break; \
+    case 8: \
+        __generic_cmpxchg(ptr__, old__, new__, ret__, \
+                          ".d", ".d"sc_sfx, release_barrier, acquire_barrier); \
+        break; \
+    default: \
+        STATIC_ASSERT_UNREACHABLE(); \
+    } \
+    ret__; \
+})
+
+#define cmpxchg_relaxed(ptr, o, n) \
+({ \
+    __typeof__(*(ptr)) o_ = (o); \
+    __typeof__(*(ptr)) n_ = (n); \
+    (__typeof__(*(ptr)))__cmpxchg_generic(ptr, \
+                    o_, n_, sizeof(*(ptr)), "", "", ""); \
+})
+
+#define cmpxchg_acquire(ptr, o, n) \
+({ \
+    __typeof__(*(ptr)) o_ = (o); \
+    __typeof__(*(ptr)) n_ = (n); \
+    (__typeof__(*(ptr)))__cmpxchg_generic(ptr, o_, n_, sizeof(*(ptr)), \
+                                          "", "", RISCV_ACQUIRE_BARRIER); \
+})
+
+#define cmpxchg_release(ptr, o, n) \
+({ \
+    __typeof__(*(ptr)) o_ = (o); \
+    __typeof__(*(ptr)) n_ = (n); \
+    (__typeof__(*(ptr)))__cmpxchg_release(ptr, o_, n_, sizeof(*(ptr)), \
+                                          "", RISCV_RELEASE_BARRIER, ""); \
+})
+
+#define cmpxchg(ptr, o, n) \
+({ \
+    __typeof__(*(ptr)) ret__; \
+    ret__ = (__typeof__(*(ptr))) \
+            __cmpxchg_generic(ptr, (unsigned long)(o), (unsigned long)(n), \
+                              sizeof(*(ptr)), ".rl", "", " fence rw, rw\n"); \
+    ret__; \
+})
+
+#define __cmpxchg(ptr, o, n, s) \
+({ \
+    __typeof__(*(ptr)) ret__; \
+    ret__ = (__typeof__(*(ptr))) \
+            __cmpxchg_generic(ptr, (unsigned long)(o), (unsigned long)(n), \
+                              s, ".rl", "", " fence rw, rw\n"); \
+    ret__; \
+})
+
+#endif /* _ASM_RISCV_CMPXCHG_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:32:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:32:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676207.1052269 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xv-0001yC-7m; Mon, 05 Feb 2024 15:32:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676207.1052269; Mon, 05 Feb 2024 15:32:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xu-0001wZ-TE; Mon, 05 Feb 2024 15:32:58 +0000
Received: by outflank-mailman (input) for mailman id 676207;
 Mon, 05 Feb 2024 15:32:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xr-0007I6-V7
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:55 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d440b2ba-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:32:53 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-51147d0abd1so1854784e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:53 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d440b2ba-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147173; x=1707751973; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GPYTkHhSwzrOZRp3wpcoqlNmpQs+KVtpWBOtoJhORF4=;
        b=N6fQ3arj9uzaWfCGCZg9oxvOC+ecricw435eFs+3HwhzgFXcBAbXGd8qWURFqt/WHH
         AyZOVFesv/DX5ipZluk4Pol7jJdRHrK2uD31YdYuF2eCK7OteaJmnBV6LXYO06wdXudo
         QASlFY1ObNbSqsth3IpaEwyW4p/OBh6DV77xU9yY9p/FrciLr19orUjzypMvqG8UmLUn
         DeP+bzdB0xhpB+U9X6LDaaQrgyOwb4ItPIPblkeLW9PZwrLU4Foem/dnEBVx4JPPYUp1
         GtjZAFJhgDV6lCh6VeVHrX9IoXqiN6WWzBVLX+VjsQwwdOmUteY1BDyId3o3Ea/TrUAO
         1+CA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147173; x=1707751973;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GPYTkHhSwzrOZRp3wpcoqlNmpQs+KVtpWBOtoJhORF4=;
        b=oyV+d+CmVPIR1YmDAPbsdswPapOBtJR58O2KPoMA3cjAWssC3MzseeWDGvWsbd4e1t
         4GOWowixQXftNY3kp0B+xVxWy4HobZ2VYzOKPs9YszcwaHuu6JD2dElX6w0OIYrn9Ufe
         5RUseyuTt9XCrrnDzFbH/hndLb/6uP/IBfioHCBnRPk8GQMPJbf57EwNxcVOYQ8Byhab
         Y0J061FOUA6nbHJorgTyxC57026lY9GXXWNnZCqmKNvEexFH4YhAoXONHbxuc2Tei7OI
         17NJdbADlz1qTGrBJ0ao8tJFJ342qwHxhj0+HDmSCHSFH5Z7rftEU4K+zx3zzwvnK93J
         4EjA==
X-Gm-Message-State: AOJu0YzGaKjmP//3Ql009M2OG6GDBYbwzSZ+EjHXsyUVT+BkOm/wrzhb
	nKOXykoRCYRqLeVP6J+nHkQpgdvemn+AVQAoqCU27PA/TG7OEdbJNpumr1BR
X-Google-Smtp-Source: AGHT+IGFYaDPh3Bv0AIiC3KgkB0bgdwRX0YdJQs0WkC9q94Zv4C6VfrE0rJG5h6jmqfIamIONInRog==
X-Received: by 2002:ac2:5f5c:0:b0:511:4cd7:7181 with SMTP id 28-20020ac25f5c000000b005114cd77181mr2387902lfz.62.1707147172499;
        Mon, 05 Feb 2024 07:32:52 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWcT4n+QS3hhrZlKic82jv8g/SG5vZmY9sJlP9nnVMuAUN3iu1r0Orngmm5IcQslrCiFY9o1cG3mXeYojZQfkIhSksNcZgI+tjn9iHyvMgSsYKX2lo98E/breUMlpZaMEpjiRgdUjMnH9tyDtsu/VLkf0jgMrOegLN9zUXa4D2zTWarEY0OqpPORTm/XYObE/tlj78Im+3iR3JIMHR3UmPRpQEV+9VJVs5ggw5g2ISqoy0fujuGFye9RMpHuXclW2ZD4PQRdN6cg+VIjFhjBtkwel9zRauTfTeTpH1RBIzAA/QXGE9/rllhQOnkYoTP8hDh0wNx
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 11/30] xen/riscv: introduce smp.h
Date: Mon,  5 Feb 2024 16:32:18 +0100
Message-ID: <4d0a8aba3ec17fa428f8d18f0008f863481b1f67.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - update the V3 changes ( remove that cpu_is_offline() was droped and instead of message
   used subject )
 - drop cpu_is_offline() as it was moved to xen/smp.h.
---
Changes in V3:
 - add SPDX.
 - drop unnessary #ifdef.
 - fix "No new line"
 - update the commit subject
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/smp.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/smp.h

diff --git a/xen/arch/riscv/include/asm/smp.h b/xen/arch/riscv/include/asm/smp.h
new file mode 100644
index 0000000000..b1ea91b1eb
--- /dev/null
+++ b/xen/arch/riscv/include/asm/smp.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_SMP_H
+#define __ASM_RISCV_SMP_H
+
+#include <xen/cpumask.h>
+#include <xen/percpu.h>
+
+DECLARE_PER_CPU(cpumask_var_t, cpu_sibling_mask);
+DECLARE_PER_CPU(cpumask_var_t, cpu_core_mask);
+
+/*
+ * Do we, for platform reasons, need to actually keep CPUs online when we
+ * would otherwise prefer them to be off?
+ */
+#define park_offline_cpus false
+
+#endif
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:33:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:33:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676208.1052275 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xw-00028M-4F; Mon, 05 Feb 2024 15:33:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676208.1052275; Mon, 05 Feb 2024 15:32:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xv-00025T-GQ; Mon, 05 Feb 2024 15:32:59 +0000
Received: by outflank-mailman (input) for mailman id 676208;
 Mon, 05 Feb 2024 15:32:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xs-0007Hv-PD
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:56 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d5a2f590-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:32:56 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-5114cd44fdbso2336153e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:56 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5a2f590-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147175; x=1707751975; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=B4eMIfLBmdHylf/GKrqfYgIxqeR1GAXR1e29mkuHIvw=;
        b=cQ+NA4PHsJf5bqkwI78v1ycIS5JBE7gHtCZpEErSuB7pKrl1e7xklco5xsnkNvoiQX
         HZP0NDi1Q2PVT8kBEBlzZSmlDVrXPkiWE1hwmkvpUmaj5ylcbvV9RGQ9BcuK979goBNv
         p5PVBY4XOSU8fgnjx0uFQnBGdWpsoFrRfmuBuyYeuuPoAwwsb6aJZ58iGm/TC6z8c3OY
         6zOI85JA7AzVBERHQyEPniZpGiFCtEOdw2Cnzco4Tl+T56iFK1X616hkVhyKCWN9p8d/
         6tkyV0Jjlu6PNXKz5UXqkQINr8KGoSv6sblb8Gkjjqu6lQ7GwYZcvjf4N9T3uthzDFPu
         hutg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147175; x=1707751975;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=B4eMIfLBmdHylf/GKrqfYgIxqeR1GAXR1e29mkuHIvw=;
        b=c4uOBxiGPI7RWLIfqlU2l8AnZi8d5HCf9O6da4y5Mo+2wS9bp3QkSp7XGyuBZAsRMy
         yYV8lOdWJnJNxuqGTHswtwkbpiQ3qKrLsqfB1wmAmcs5eeYo/lzUJ/+oZX9iV62sceL2
         FZceXeUv9rg0ITJj7LaAoBB3CqSz4QgAKPoffHSkb9RZAxcjtNEkOAlU8iopoK4vOjV0
         RslzG+8OiPUM9FxJY+aT7qeJ9GVvze13B5y5t5L1DMjyErtXYWRzxKqosBERYNUhZMD/
         DwLPD9ZGwCktqGVPAJYujqwL/c4BdelPakzzXVjQh7jopN7AisswSA2m8kBUDvzWHeaX
         NjQA==
X-Gm-Message-State: AOJu0Yx+mvGIqryRZVuwSotww7l/KA5hK3AAK+J3zKJsjjXlDNwkPJFL
	/CLTZGvwSH1Gb8XtOKVVvkqYgQkxEGTTFYxeE7dpF+WljBUvc/HGcyMD1Hda
X-Google-Smtp-Source: AGHT+IHKPIhVNBwsZykURNlJGS0BKoSDbw+LWo4E8hC3aDGWQ/scYCbvIN6O3HIhuXb0gQjbpOIhgw==
X-Received: by 2002:ac2:54ac:0:b0:511:4db9:d6c7 with SMTP id w12-20020ac254ac000000b005114db9d6c7mr2300376lfk.67.1707147175099;
        Mon, 05 Feb 2024 07:32:55 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUGBoM+qWiIPUT4iMVMfMFxU84N5INAS2iE0GVSc9JZRRztDCVbMucN6Dc8Zv1lBz5vA1x1rWvtHSN9lJLyvudlTkGQtTnkSvYX1G+USD+8kqyyfFC8ag4l2moRAJ5YTe2EnwlBZystJA0krJJ1G05rhN7ewmAEBq/yXRZ+pvPuI813ILZzGrym7EBeuhYBdAgp5kvZoVIy4lF9LXLF3xITlLkrBmVfvrXg6viw5SvN5dUINKrY1svWO8fROCKg0IMNUNbXk/rk4xDsdgCE/jrASYJ1BwXw865RvG6cvMyOZoaAg39VjOXKmKR1pr6A+vLdukP7
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 13/30] xen/riscv: introduce io.h
Date: Mon,  5 Feb 2024 16:32:20 +0100
Message-ID: <64afc021f680e55d486f8429c02513bc9bf4b9ea.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The header taken form Linux 6.4.0-rc1 and is based on
arch/riscv/include/asm/mmio.h.

Addionally, to the header was added definions of ioremap_*().

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - delete inner parentheses in macros.
 - s/u<N>/uint<N>.
---
Changes in V3:
 - re-sync with linux kernel
 - update the commit message
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/io.h | 142 ++++++++++++++++++++++++++++++++
 1 file changed, 142 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/io.h

diff --git a/xen/arch/riscv/include/asm/io.h b/xen/arch/riscv/include/asm/io.h
new file mode 100644
index 0000000000..1e61a40522
--- /dev/null
+++ b/xen/arch/riscv/include/asm/io.h
@@ -0,0 +1,142 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * {read,write}{b,w,l,q} based on arch/arm64/include/asm/io.h
+ *   which was based on arch/arm/include/io.h
+ *
+ * Copyright (C) 1996-2000 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ * Copyright (C) 2014 Regents of the University of California
+ */
+
+
+#ifndef _ASM_RISCV_IO_H
+#define _ASM_RISCV_IO_H
+
+#include <asm/byteorder.h>
+
+/*
+ * The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't
+ * change the properties of memory regions.  This should be fixed by the
+ * upcoming platform spec.
+ */
+#define ioremap_nocache(addr, size) ioremap(addr, size)
+#define ioremap_wc(addr, size) ioremap(addr, size)
+#define ioremap_wt(addr, size) ioremap(addr, size)
+
+/* Generic IO read/write.  These perform native-endian accesses. */
+#define __raw_writeb __raw_writeb
+static inline void __raw_writeb(uint8_t val, volatile void __iomem *addr)
+{
+	asm volatile("sb %0, 0(%1)" : : "r" (val), "r" (addr));
+}
+
+#define __raw_writew __raw_writew
+static inline void __raw_writew(uint16_t val, volatile void __iomem *addr)
+{
+	asm volatile("sh %0, 0(%1)" : : "r" (val), "r" (addr));
+}
+
+#define __raw_writel __raw_writel
+static inline void __raw_writel(uint32_t val, volatile void __iomem *addr)
+{
+	asm volatile("sw %0, 0(%1)" : : "r" (val), "r" (addr));
+}
+
+#ifdef CONFIG_64BIT
+#define __raw_writeq __raw_writeq
+static inline void __raw_writeq(u64 val, volatile void __iomem *addr)
+{
+	asm volatile("sd %0, 0(%1)" : : "r" (val), "r" (addr));
+}
+#endif
+
+#define __raw_readb __raw_readb
+static inline uint8_t __raw_readb(const volatile void __iomem *addr)
+{
+	uint8_t val;
+
+	asm volatile("lb %0, 0(%1)" : "=r" (val) : "r" (addr));
+	return val;
+}
+
+#define __raw_readw __raw_readw
+static inline uint16_t __raw_readw(const volatile void __iomem *addr)
+{
+	uint16_t val;
+
+	asm volatile("lh %0, 0(%1)" : "=r" (val) : "r" (addr));
+	return val;
+}
+
+#define __raw_readl __raw_readl
+static inline uint32_t __raw_readl(const volatile void __iomem *addr)
+{
+	uint32_t val;
+
+	asm volatile("lw %0, 0(%1)" : "=r" (val) : "r" (addr));
+	return val;
+}
+
+#ifdef CONFIG_64BIT
+#define __raw_readq __raw_readq
+static inline u64 __raw_readq(const volatile void __iomem *addr)
+{
+	u64 val;
+
+	asm volatile("ld %0, 0(%1)" : "=r" (val) : "r" (addr));
+	return val;
+}
+#endif
+
+/*
+ * Unordered I/O memory access primitives.  These are even more relaxed than
+ * the relaxed versions, as they don't even order accesses between successive
+ * operations to the I/O regions.
+ */
+#define readb_cpu(c)		({ uint8_t  __r = __raw_readb(c); __r; })
+#define readw_cpu(c)		({ uint16_t __r = le16_to_cpu((__force __le16)__raw_readw(c)); __r; })
+#define readl_cpu(c)		({ uint32_t __r = le32_to_cpu((__force __le32)__raw_readl(c)); __r; })
+
+#define writeb_cpu(v,c)		((void)__raw_writeb(v,c))
+#define writew_cpu(v,c)		((void)__raw_writew((__force uint16_t)cpu_to_le16(v),c))
+#define writel_cpu(v,c)		((void)__raw_writel((__force uint32_t)cpu_to_le32(v),c))
+
+#ifdef CONFIG_64BIT
+#define readq_cpu(c)		({ u64 __r = le64_to_cpu((__force __le64)__raw_readq(c)); __r; })
+#define writeq_cpu(v,c)		((void)__raw_writeq((__force u64)cpu_to_le64(v),c))
+#endif
+
+/*
+ * I/O memory access primitives. Reads are ordered relative to any
+ * following Normal memory access. Writes are ordered relative to any prior
+ * Normal memory access.  The memory barriers here are necessary as RISC-V
+ * doesn't define any ordering between the memory space and the I/O space.
+ */
+#define __io_br()	do {} while (0)
+#define __io_ar(v)	__asm__ __volatile__ ("fence i,r" : : : "memory");
+#define __io_bw()	__asm__ __volatile__ ("fence w,o" : : : "memory");
+#define __io_aw()	do { } while (0)
+
+#define readb(c)	({ uint8_t  __v; __io_br(); __v = readb_cpu(c); __io_ar(__v); __v; })
+#define readw(c)	({ uint16_t __v; __io_br(); __v = readw_cpu(c); __io_ar(__v); __v; })
+#define readl(c)	({ uint32_t __v; __io_br(); __v = readl_cpu(c); __io_ar(__v); __v; })
+
+#define writeb(v,c)	({ __io_bw(); writeb_cpu(v,c); __io_aw(); })
+#define writew(v,c)	({ __io_bw(); writew_cpu(v,c); __io_aw(); })
+#define writel(v,c)	({ __io_bw(); writel_cpu(v,c); __io_aw(); })
+
+#ifdef CONFIG_64BIT
+#define readq(c)	({ u64 __v; __io_br(); __v = readq_cpu(c); __io_ar(__v); __v; })
+#define writeq(v,c)	({ __io_bw(); writeq_cpu((v),(c)); __io_aw(); })
+#endif
+
+#endif /* _ASM_RISCV_IO_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:33:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:33:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676209.1052281 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xw-0002Fw-QZ; Mon, 05 Feb 2024 15:33:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676209.1052281; Mon, 05 Feb 2024 15:33:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xw-0002D5-5x; Mon, 05 Feb 2024 15:33:00 +0000
Received: by outflank-mailman (input) for mailman id 676209;
 Mon, 05 Feb 2024 15:32:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xu-0007Hv-BE
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:32:58 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d6a1f12a-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:32:57 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-51025cafb51so6831349e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:57 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d6a1f12a-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147177; x=1707751977; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=MJSf6y28Unp6Qbv5KujwB4TmNmYEQsh1IjcgrrZViJk=;
        b=jrASYNRO7Q+s+/OkfnR6uh+wx7RIaznYfzU4++ca5lmpflUvEeOcJmRObDpRRXIxiv
         wr4qe+kq5p6vw6nvL25IPFcXYGluI0cz6FAevgs1SvmX4fOEqB7O2uLuIol3oOPqhCik
         /p//tYeatoq5uN4tpGRuH3lQpA6/mTPAIDGXun104Yy6Bf3IfQgAKY01h54ew1B7kTSQ
         CDVraHJu0ifHZi0YXKYsouTHXI1bcpLws2nSofdwownLYntZiOiEXX7MtNpHvyx3l+uM
         EkjkSWeYUPx7tdKSxlHpaeP5AGimCfyKcTTxq2yatswLGL1HmOCBGDVhVmjESgHD80Eu
         fEoA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147177; x=1707751977;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=MJSf6y28Unp6Qbv5KujwB4TmNmYEQsh1IjcgrrZViJk=;
        b=DKFs+215CmvYs1Wub0faSPUwLd01+eoJlWioJty6bsNxA4Jm32x50BJDY6K4eU+RRc
         qoZEYg0RrZcQEeQbdZFbRDINYr540OZUP+Q4onsSC+F4ySV0+vVRxiGn4tWIvTE9uxd2
         mUslFlINZNv4O9NuS41qP5m+UnYOMBuPjCluorW65gr62UFqv4xNkRkH02fEZbxcNV/7
         iXSq/Eq1qdIBljuj1Dy+aOAeC0ODlSGB2d0fMi2EL5Viv+KfN9mVTcu3IHnntPD33Hbc
         1+QlrCuVsCYqQaHYSSCmZkSylNf3TSQaw/a38BCet8+EvSSPHM6tNFYunwdTA+IK2zmI
         jbhw==
X-Gm-Message-State: AOJu0YyinkluULKXoL2IjaeEiSPHWBcABW1gxwOOUZ1K6a1SWBplcmwR
	lCiDKtzBw6agzhmfw11hs6OSOBe7tUKQRX0EDCLDHd6kDmnZA7/Uhkfc26Uo
X-Google-Smtp-Source: AGHT+IGvEaoKvKsO92WZqjGKP+iQ9RbfnIrHTW8AyWlaAQCFTGYXBWJCe3c8TCEn6bVzOOcCGWGbMA==
X-Received: by 2002:ac2:4142:0:b0:511:51f6:4760 with SMTP id c2-20020ac24142000000b0051151f64760mr1507160lfi.4.1707147177293;
        Mon, 05 Feb 2024 07:32:57 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUrD1xm0E1CoDYjps100oEy4yRSr9LyRJECv0o9KokL1HGC2ay2oh3XADiLtn0rfbLlJ+lzzXHMjWkR/X+NbZgu+h21QNpiULjfsNii1rRr/9sGWOqzZzd1WYImFNCUtotsCbZ8cltRCzGtnZkmDxxxIGNjy4VqIO76FoBbEaNOr8xWER1QOMA/i1l1FtOWDKotnypmPFDy5iqUc6jZLkNW/0f3VSQBCWzDda9Jdt2vBUxAmgJ6B6toZQOab1aAIdCpLkxAQUvLOdW8LprygTkkUcp2z1Y7pK/ao3CNIwgTt0Uw5VEHieUdLuBJ84n+Qe4YRKa9
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 15/30] xen/riscv: introduce irq.h
Date: Mon,  5 Feb 2024 16:32:22 +0100
Message-ID: <6f0bfcda83095b35cb7639538a4705dbb5aa9d58.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V4:
 - Change BUG to BUG_ON("unimplemented").
 - Add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V3:
 - add SPDX
 - remove all that was wraped to HAS_DEVICETREE_... as for RISC-V it is going to be
   always selected.
 - update the commit message
---
Changes in V2:
 - add ifdef CONFIG_HAS_DEVICE_TREE for things that shouldn't be
   in case !CONFIG_HAS_DEVICE_TREE
 - use proper includes.
---
 xen/arch/riscv/include/asm/irq.h | 37 ++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/irq.h

diff --git a/xen/arch/riscv/include/asm/irq.h b/xen/arch/riscv/include/asm/irq.h
new file mode 100644
index 0000000000..0dfd4d6e8a
--- /dev/null
+++ b/xen/arch/riscv/include/asm/irq.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_IRQ_H__
+#define __ASM_RISCV_IRQ_H__
+
+#include <xen/bug.h>
+
+/* TODO */
+#define nr_irqs 0U
+#define nr_static_irqs 0
+#define arch_hwdom_irqs(domid) 0U
+
+#define domain_pirq_to_irq(d, pirq) (pirq)
+
+#define arch_evtchn_bind_pirq(d, pirq) ((void)((d) + (pirq)))
+
+struct arch_pirq {
+};
+
+struct arch_irq_desc {
+    unsigned int type;
+};
+
+static inline void arch_move_irqs(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+#endif /* __ASM_RISCV_IRQ_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:33:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:33:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676210.1052294 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xz-00031v-SH; Mon, 05 Feb 2024 15:33:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676210.1052294; Mon, 05 Feb 2024 15:33:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0xz-0002y0-FJ; Mon, 05 Feb 2024 15:33:03 +0000
Received: by outflank-mailman (input) for mailman id 676210;
 Mon, 05 Feb 2024 15:33:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xw-0007I6-GY
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:00 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d729c54d-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:32:58 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-5111e5e4e2bso7067735e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:58 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d729c54d-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147178; x=1707751978; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=xxDcUc6rWW4h3ga9tFvFXTGNQ6qcEb8Oz/2ADhIp7pw=;
        b=Pm7M3/YEuwEjbXhiGhmvB56UpTQ6qE3iZd4QHrfkUVXeEXZhnczjO439G5pDN4Huki
         gnm5HxjCIVNrJXUHg2GBMiaX3qWPO3zHj+RrzU3aAhjmhpF0OC3Gov5Qi4M4OtELJrud
         AVtIPIRBAEZTQ+NSuQGCXZDgxEXGumQrNnTQs7pldDFBnC364J+HUxGEbDl2OJsJjIpv
         nh79e+U9q6zthO8KuJqSqmYc7wMPX+9rS2Vsq58PkhOiwmrAxRRUroV/o9OS0ozckecC
         Y7updL9f/clFzdDnojqwi3fQ/DW8jf+AlLLvgdY8YQLHqpEy+OGCAD0bkC+vfERrwXQC
         rvdg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147178; x=1707751978;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=xxDcUc6rWW4h3ga9tFvFXTGNQ6qcEb8Oz/2ADhIp7pw=;
        b=fTF+Ob825z1RmxmpDcJklvuKeQ81zSRtzFuJ690mCHRlv3fGoZJ2hiG4jpzzIjo3Jb
         /tdE49v9O/JILFe61hQHzKjJEfPfRDop0QluIrfu7SdyItOnGYTHKDDL93JG+cTeqDSB
         1eUR5xCbIHlHo7jCgGnjJ8rfvjWPj2ysU8EoVqQFs5ot9Idz9guwkgPA7vqEiRO/nWEM
         G/gxB3vg3J5p7nOvlH4wg61KsURqzqPfhNlIql1uSokFJZRw78Xs/BmGL9l+lQJrxwsb
         RCOFX7wV8j+MRFwYj1ufOgPlEN22HydI6iiicCHF6XetdFq9fQmOOn6l8YkjOB/rtdrn
         FtSw==
X-Gm-Message-State: AOJu0Yzfj0r/B/byngtBsjTtmGpIR6WatGUkAZ0idg50fhwOiz+VYnZc
	7xOEet5tLhnMGp5/vxFXS7CpNuUD/pp1Rj1ocJd8R9MIQnGSLMDK86AM9QJx
X-Google-Smtp-Source: AGHT+IE94854AnAZmL/mD77mY6EfUqRNri0QqHNQ8nF3T+G1eJJ13BWu6j9fVDPkW8li9ZiPEU1QiQ==
X-Received: by 2002:a05:6512:480b:b0:511:479c:1d8e with SMTP id eo11-20020a056512480b00b00511479c1d8emr4082037lfb.29.1707147178118;
        Mon, 05 Feb 2024 07:32:58 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXAsRSSR4SGgCBMKW0WY8FQ8mSKDehSpoPXPn5SivkPmY7UrlC9cbswWo95Oe1OXbiqdVmK6OaPRREyFfWUrM3a7VxNma5svTNZ/6izY+mETrWNvdLyYWSzcwY984T+8RUdJuqCuExxqDBK7wh4cbG5T/6AIoHi57ZGfU/anHI44KChIgHYE5SJkZyd7nrry/XAeTaMcSkSotiTphN020BJpMGGAI1T/Tdc88O8JoO9TQfNa9kefCZ+5GdxDcNWfE5d3Wf3ZYUQa51TeCeiNq7mOQ6/xX+1GGkxCvWTfmFOWcErNDJkqguijjfmAZG3y+M4y+kF
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 16/30] xen/riscv: introduce p2m.h
Date: Mon,  5 Feb 2024 16:32:23 +0100
Message-ID: <c2a2e872f8c32d81c3d3f428f0273819dd7df081.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - update the comment above p2m_type_t. RISC-V has only 2 free for use bits in PTE, not 4 as Arm.
 - update the comment after p2m_ram_rw: s/guest/domain/ as this also applies for dom0.
 - return INVALID_MFN in gfn_to_mfn() instead of mfn(0).
 - drop PPC changes.
---
Changes in V3:
 - add SPDX
 - drop unneeded for now p2m types.
 - return false in all functions implemented with BUG() inside.
 - update the commit message
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/p2m.h | 102 +++++++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/p2m.h

diff --git a/xen/arch/riscv/include/asm/p2m.h b/xen/arch/riscv/include/asm/p2m.h
new file mode 100644
index 0000000000..8ad020974f
--- /dev/null
+++ b/xen/arch/riscv/include/asm/p2m.h
@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_P2M_H__
+#define __ASM_RISCV_P2M_H__
+
+#include <asm/page-bits.h>
+
+#define paddr_bits PADDR_BITS
+
+/*
+ * List of possible type for each page in the p2m entry.
+ * The number of available bit per page in the pte for this purpose is 2 bits.
+ * So it's possible to only have 4 fields. If we run out of value in the
+ * future, it's possible to use higher value for pseudo-type and don't store
+ * them in the p2m entry.
+ */
+typedef enum {
+    p2m_invalid = 0,    /* Nothing mapped here */
+    p2m_ram_rw,         /* Normal read/write domain RAM */
+} p2m_type_t;
+
+#include <xen/p2m-common.h>
+
+static inline int get_page_and_type(struct page_info *page,
+                                    struct domain *domain,
+                                    unsigned long type)
+{
+    BUG_ON("unimplemented");
+    return -EINVAL;
+}
+
+/* Look up a GFN and take a reference count on the backing page. */
+typedef unsigned int p2m_query_t;
+#define P2M_ALLOC    (1u<<0)   /* Populate PoD and paged-out entries */
+#define P2M_UNSHARE  (1u<<1)   /* Break CoW sharing */
+
+static inline struct page_info *get_page_from_gfn(
+    struct domain *d, unsigned long gfn, p2m_type_t *t, p2m_query_t q)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+static inline void memory_type_changed(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+
+static inline int guest_physmap_mark_populate_on_demand(struct domain *d, unsigned long gfn,
+                                                        unsigned int order)
+{
+    return -EOPNOTSUPP;
+}
+
+static inline int guest_physmap_add_entry(struct domain *d,
+                            gfn_t gfn,
+                            mfn_t mfn,
+                            unsigned long page_order,
+                            p2m_type_t t)
+{
+    BUG_ON("unimplemented");
+    return -EINVAL;
+}
+
+/* Untyped version for RAM only, for compatibility */
+static inline int __must_check
+guest_physmap_add_page(struct domain *d, gfn_t gfn, mfn_t mfn,
+                       unsigned int page_order)
+{
+    return guest_physmap_add_entry(d, gfn, mfn, page_order, p2m_ram_rw);
+}
+
+static inline mfn_t gfn_to_mfn(struct domain *d, gfn_t gfn)
+{
+    BUG_ON("unimplemented");
+    return INVALID_MFN;
+}
+
+static inline bool arch_acquire_resource_check(struct domain *d)
+{
+    /*
+     * The reference counting of foreign entries in set_foreign_p2m_entry()
+     * is supported on RISCV.
+     */
+    return true;
+}
+
+static inline void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
+{
+    /* Not supported on RISCV. */
+}
+
+#endif /* __ASM_RISCV_P2M_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:33:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:33:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676211.1052303 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0y1-0003Dg-93; Mon, 05 Feb 2024 15:33:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676211.1052303; Mon, 05 Feb 2024 15:33:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0y0-0003BP-HJ; Mon, 05 Feb 2024 15:33:04 +0000
Received: by outflank-mailman (input) for mailman id 676211;
 Mon, 05 Feb 2024 15:33:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xx-0007I6-GU
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:01 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d7da89d9-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:32:59 +0100 (CET)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2cf4d2175b2so51938751fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:32:59 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7da89d9-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147179; x=1707751979; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7y/mb9zRfl82LoI+/wV6Uo5A2HP42Gm2KKQIS8V0N2w=;
        b=IUNwMfYUD14wOJYgArrkYudTVxLlKZqZtsk5BTzy9nQDy7UnJJ8mT4ngv/K+FWl689
         Ja5x3vRhUW6/5nMtTkKluuc8Jz4YsKUkL6rSXQddSW7IZITtnMFJDLNbBPkxphfr6AMQ
         jbVeaWvgwWD7lEEBLEJ3Keq4KjQNp5xS2HeBYzNfjARZAxwrbt1tpjughcN4qtJIwqKA
         M9C76CNfAmq0d9xtTF/IswzIU+82CXkRJoUxLbII+jUwy8ygU5tM5ObLyOprWXAIzHBw
         B+y7gbp0TDKhuqzLkrE3o5OFbM3Wy4nBb9w8beqr0Xgl5S0luc2suIHPGioXAY5UNhWc
         Pxlw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147179; x=1707751979;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=7y/mb9zRfl82LoI+/wV6Uo5A2HP42Gm2KKQIS8V0N2w=;
        b=cdvWprqGp8xcHIvN4+9bd217MUnP2tyO0sfgkTbN/MKxa7qSDNmmg7VXgXlRlSI9Dp
         aJEYdiH/uyHc/KMte7UkfrTSecZ4d4PmqV7enj2VlDVTaAEErOe1k4Ccw/BM9eZOGu2A
         T+kcuWj9weH9DR3mGwZLcKQ5Akq6MkYzuG8PAMvmKkvW6mHX8BzEKpA8Qzi4tNFoZNSw
         P96p18VYL1+FA2g2WpWF9d+4YM4+4HEzd0pvphvY+yJ8yoH6ur/cOJ2gCy4o7NhPfJEJ
         1XgkF5uVEM2f5DH6utVNzMiCPNER1sp7i81RdV8OIyUOw48WY/OKVhzen6ZyQcjlLOq8
         lnJw==
X-Gm-Message-State: AOJu0YzsH62+VD+c7noF9Mi78y35LlPE/kS6WD4rRjfHN3tGhU1IiMwT
	2UgrV2ru8Z2BY43R6sPSq8UUgYM/2s0Q2HbksQbF5x8CQg0Xtw69ln7wJd+v
X-Google-Smtp-Source: AGHT+IESW/KeXOwEVS0hk9Mu825OKbeA8BW1rAcOgqkk5KyjZhQ2QtdkURrz2cCoHf+RsnV3HWEzbA==
X-Received: by 2002:ac2:58dc:0:b0:511:4f2a:3ff8 with SMTP id u28-20020ac258dc000000b005114f2a3ff8mr2264718lfo.67.1707147179026;
        Mon, 05 Feb 2024 07:32:59 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXm9QxYDlF3e9hFbfLvTxkGIdysntha7dD/yORxMYb8xmcjeXqW6Bg+te1SGI1adMJDNZxVxaJFF4n89MU6gvyoiGayzDJCtg+o6vAZsJGe+MMt9vt+Toqf2qWonl1mq3s+8l8jLRmshWw5JxA+/HEGVj3F/KuSDtMEu5zbhPPlSlIDK+Xk+mGR2GBksLI2MBUbWDURfpHqnWscY5w2bVXcArIYvUEe41UaTTcuy2eb/4Y2Hrx78hbDU0COsQNQIqUZ+ieI7rmYuo48X0een5D4OpuYPVUorwpdg0LRU3bpAD03O+8KMVQwP38Dpun8LAGjWhal
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 17/30] xen/riscv: introduce regs.h
Date: Mon,  5 Feb 2024 16:32:24 +0100
Message-ID: <13cfdfa01b303d2597124f09c0ec0734859b697d.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
------
Changes in V4:
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
 - s/BUG()/BUG_ON("unimplemented")
---
Changes in V3:
 - update the commit message
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
 - remove "include <asm/current.h>" and use a forward declaration instead.
---
Changes in V2:
 - change xen/lib.h to xen/bug.h
 - remove unnecessary empty line
---
xen/arch/riscv/include/asm/regs.h | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/regs.h

diff --git a/xen/arch/riscv/include/asm/regs.h b/xen/arch/riscv/include/asm/regs.h
new file mode 100644
index 0000000000..c70ea2aa0c
--- /dev/null
+++ b/xen/arch/riscv/include/asm/regs.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ARM_RISCV_REGS_H__
+#define __ARM_RISCV_REGS_H__
+
+#ifndef __ASSEMBLY__
+
+#include <xen/bug.h>
+
+#define hyp_mode(r)     (0)
+
+struct cpu_user_regs;
+
+static inline bool guest_mode(const struct cpu_user_regs *r)
+{
+    BUG_ON("unimplemented");
+}
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ARM_RISCV_REGS_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:33:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:33:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676212.1052312 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0y2-0003Ps-JC; Mon, 05 Feb 2024 15:33:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676212.1052312; Mon, 05 Feb 2024 15:33:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0y1-0003N8-GC; Mon, 05 Feb 2024 15:33:05 +0000
Received: by outflank-mailman (input) for mailman id 676212;
 Mon, 05 Feb 2024 15:33:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xy-0007Hv-2v
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:02 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d8db995f-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:33:01 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-5114cd44f6aso1310796e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:01 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8db995f-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147181; x=1707751981; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=kwBu84D0n1DoTgY2J9dL0hCB/ukCq+oQwz679ugGOF4=;
        b=OFFxYpOT/SCyDL3P4UH7yKxo6jAD4zIDgfyQmZhiMldsCDOSXRwOyrRoILJ4Qu3f7E
         8ddYejoBv0KJqgCMnUTkP000y04jlVZdxJfCwFI9ZuoTW34S1SeHKtHci6Fl3VDyXg7o
         bQ8Z1SuaXbVGLpCRstLzshOr8fRMBwEBJQtbYBm/HxEYqfvRzrFpv2T3Y9CH/+oH/jmb
         vVX74E+Rq1Z2pWPtlyY0ZIjrD3rCtUPM2J7GLyOKhH9pRtZpLU2NhTp7CIHy4PIqdHjp
         8/BetZJC4JpZjmrKR7+y4lQYcq+bSC2aVBzB27n75Z58qWcOJnx3oWjDqohEOTdwJq65
         toBA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147181; x=1707751981;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=kwBu84D0n1DoTgY2J9dL0hCB/ukCq+oQwz679ugGOF4=;
        b=O3zDQUjGbKaS6Tlj1T2Sopxs0SP5xoa/4xjqWsCCI6p9qnQdzI0dnrC0DP/praq6Yz
         65tedYkiTCjlsq5gk/Ldhd0pOIwZtentGbofJCO3CT59kIzX/tUBdEEHxLHmftgeja2+
         X/lNuf2bcBv0Jdn/lOka1mkRzoUmLZ7GXf72OqjixGUqonG8zxEoF8ehOXfendoObbXw
         fQedE3a4X6AuinGHuY7960JBQzpiMTxhjKuD4lJ5/4UsFpQ2pgtBbDTp9tQx31tikF3i
         aVP2oeRzxCGgO6FsnmAYdcqjPtvv2lf+acDibYi/MPWHVinTE3X+DEho/RdjXd7jOT5N
         cFqQ==
X-Gm-Message-State: AOJu0YxT6khiSJnHiqB5VDVVEoshXOjlD7wHVLAp9NieH/ItBIMrE8d1
	EXGFvrmfpshvqG5Hoo7BK9fbzyDcvql4rP9ZhXxKooIu7b/M9tRddUA2s3Wd
X-Google-Smtp-Source: AGHT+IHFs7tlpIVryWCEx1JXjwZyMAuN58CNQvo4IL2SB1CB+7ciG6j8sE5ZI3WEmiEJI+a19G/ChQ==
X-Received: by 2002:a05:6512:480a:b0:511:52b4:8b93 with SMTP id eo10-20020a056512480a00b0051152b48b93mr1399173lfb.29.1707147180898;
        Mon, 05 Feb 2024 07:33:00 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWtUKb5cQm7vsiITfH1eqCM5692QeSptFfBnvihdgDBloWsy0FZFfbMu6VWt6mVQwB/ujLAGwBpm8WDazQDnfAfnk4lWXNygCw5d5LzWBNfX1HVHgtjg8FAwUCmcKtcDI1bh9emA4QJcqbb5rNaMMxH1/4ADlWrKgSM5iZlTWTtYidxoeMHPjv+zdBmXtQFnKXe5TQn+t5MqI6Tj/cYJRM4/UqvcxtTDC8xE0sy1DIASGuGmBEhhLUaq9wfpnp5m7ac5VbuWr1sgBiQs81Ey9f08rkW66UkHEUWv5rqieRBPdsKZkoQnMw5Hx6knmwU1iUAJUlW
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 19/30] xen/riscv: introduce event.h
Date: Mon,  5 Feb 2024 16:32:26 +0100
Message-ID: <a57aed8d2d8424c63b1f27d007b2d5cfe35460cc.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - s/BUG()/BUG_ON("unimplemented")
 - s/xen\/bug.h/xen\/lib.h as BUG_ON is defined in xen/lib.h.
---
Changes in V3:
 - add SPDX
 - add BUG() inside stubs.
 - update the commit message
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/event.h | 40 ++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/event.h

diff --git a/xen/arch/riscv/include/asm/event.h b/xen/arch/riscv/include/asm/event.h
new file mode 100644
index 0000000000..b6a76c0f5d
--- /dev/null
+++ b/xen/arch/riscv/include/asm/event.h
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_EVENT_H__
+#define __ASM_RISCV_EVENT_H__
+
+#include <xen/lib.h>
+
+void vcpu_mark_events_pending(struct vcpu *v);
+
+static inline int vcpu_event_delivery_is_enabled(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline int local_events_need_delivery(void)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline void local_event_delivery_enable(void)
+{
+    BUG_ON("unimplemented");
+}
+
+/* No arch specific virq definition now. Default to global. */
+static inline bool arch_virq_is_global(unsigned int virq)
+{
+    return true;
+}
+
+#endif
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:33:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:33:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676213.1052317 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0y3-0003oz-LM; Mon, 05 Feb 2024 15:33:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676213.1052317; Mon, 05 Feb 2024 15:33:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0y2-0003kK-TB; Mon, 05 Feb 2024 15:33:06 +0000
Received: by outflank-mailman (input) for mailman id 676213;
 Mon, 05 Feb 2024 15:33:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xz-0007Hv-3M
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:03 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d94cedc5-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:33:02 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-5114fa38434so1247558e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:02 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d94cedc5-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147182; x=1707751982; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=0gARUjv581cUW1g1ZCe6+dI3h9g3+PYO7UcATFZv3k0=;
        b=g22vWrY8+fK2ayS8ZcnJ+I8FXx9GbFTAsuqgW7pHhNR3jjusQZSTwuuLhGbLvhqVVk
         BecYFd4JPXe634r33P02NvFLEA3q/3acujM5FHzEat+NIde1I7L2316k5AjSh121mOFG
         N5UJUSqppO1uRoZLalb9JsidOMvE6QYNuKMpph9vvAGPZxwfKB+C6MDpNYUAruDdIMoV
         LVY9TSESl8xZ6eS7Y4QqfjTz/Id3eptsXm722C+k4Usmop8eZsef1mGbzeQAhBfY8Bqe
         Fb47rl4Rr7abKexipprAGGC73FLK/2Tc8JdworLOtsJ4G7RqxDm7Oe9CEnsz0sggxo16
         +dfg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147182; x=1707751982;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=0gARUjv581cUW1g1ZCe6+dI3h9g3+PYO7UcATFZv3k0=;
        b=Rg/hTMx9olmzxDx7/d03+muXi2+Av/9deLg1QgWpuIm5R2qe5mL7CRWsjDqiMqZrug
         eHSrYj5ITcv682oui52/3bw5zbL3JgArIkTveY8j+UHBeWfT1HaqVsHjGO/dpnph/37G
         yTFW9VReOcJNvJbkU5xxoV0ZWD/nNq/88WR57ors+7eWzddRUSaoGTJd4NaFZ0wFPwEP
         YRx8TumyLLftUZQTJ+8fI6lEO+xb9uUjCpJw9Bh5GMdAhb+wyRGUOr08AXlawplauKoS
         /ebdHiMqEqMFENzKjpbtgrqrv5zOICqlO6DikTl7Zbvm2SC2z2LhGKe05FLvzNdkHVRp
         3r6w==
X-Gm-Message-State: AOJu0YwG1mQHfNQBck9fWspna26LdmlKofruibGejCS2gUOTnBJPgpf3
	/yzNqCUyHdWO57JmpaVlFwbZsOT2YWTITVkQrT67kzRaRnAsIT5BIr7RQis2
X-Google-Smtp-Source: AGHT+IFKTLFETIdqzqjZRLukSGR3tjSLZkxLYn00YvDSQ8zxBRdVKq45GMT3uiVnqoV8z0pF1UOhFA==
X-Received: by 2002:a05:6512:2c0a:b0:511:4758:c99e with SMTP id dx10-20020a0565122c0a00b005114758c99emr4788620lfb.25.1707147181786;
        Mon, 05 Feb 2024 07:33:01 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXbhFD80mHCSO8eudoZxHaF6IND71BLdMe16lXGKwf8DfEF7jmIIt2zP8H0uJwmhBjRP44zwK/ByhIxkMwptkJuvGzkZW5vYiesQZYQKBL0QYa6idXKfQDYeet7YMLIE1WFYtTJ4MhPNQwmOMe4PFEFNOwS1jtXaB8SOW1Df363y/CuJtQntPczNwXwf/nLbLSP3+k77xHP3FZ9YORNTm/9oUbzTre6SwvEaTG0diBZvukPAWSAWdBMTH2e+JJZ6XeyEw==
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v4 20/30] xen/riscv: introduce monitor.h
Date: Mon,  5 Feb 2024 16:32:27 +0100
Message-ID: <846a0c47e1808b01d5650a72d3366c3c0f2764c1.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Taking into account conversion in [PATCH v6 0/9] Introduce generic headers
 (https://lore.kernel.org/xen-devel/cover.1703072575.git.oleksii.kurochko@gmail.com/)
 this patch can be changed
---
Changes in V4:
 - Nothing changed. Only rebase.
---
Changes in V3:
 - new patch.
---
 xen/arch/riscv/include/asm/monitor.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/monitor.h

diff --git a/xen/arch/riscv/include/asm/monitor.h b/xen/arch/riscv/include/asm/monitor.h
new file mode 100644
index 0000000000..f4fe2c0690
--- /dev/null
+++ b/xen/arch/riscv/include/asm/monitor.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_MONITOR_H__
+#define __ASM_RISCV_MONITOR_H__
+
+#include <xen/bug.h>
+
+#include <asm-generic/monitor.h>
+
+struct domain;
+
+static inline uint32_t arch_monitor_get_capabilities(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+#endif /* __ASM_RISCV_MONITOR_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:33:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:33:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676214.1052323 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0y5-00047i-7P; Mon, 05 Feb 2024 15:33:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676214.1052323; Mon, 05 Feb 2024 15:33:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX0y4-00040L-6l; Mon, 05 Feb 2024 15:33:08 +0000
Received: by outflank-mailman (input) for mailman id 676214;
 Mon, 05 Feb 2024 15:33:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0y0-0007Hv-3e
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:04 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d99e9ed1-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:33:02 +0100 (CET)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-511570b2f49so135907e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:02 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d99e9ed1-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147182; x=1707751982; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2DAJ7IaJfHQvUHlz0u6Dc8IIe22blASYjWQPSCDm2uk=;
        b=KbmjpN73Af4yHM7urjg9RVCIeenEEE4rlz0iVgAMIbdMvcpwpJSukT/KHByIsaKQYW
         lf2qCAyyw0K8lnozw7H4bQmwU/3fPpO5JBGW623OWNvHe4j3RCehMblwi+q7LlZE6Jdm
         noAxlbTSAASpoCtTagJQZT7nrDz/8OVF4eqD3B4UHYruplJgB1EepdgKlLOIoi0xCMsj
         b1I689aP9hs9XWgZD+ZaRyQ9dTrR0yJ3hYpb5CXosJlE5F32cHWg4YNNs4MtaNy1LNGx
         j6uadJ9lo0SiBjthH1+qxFECi5Vj9nDJjw+ERWInoESDoXStZcK5Hmhmrcpl2c/8UR18
         T4Dw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147182; x=1707751982;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=2DAJ7IaJfHQvUHlz0u6Dc8IIe22blASYjWQPSCDm2uk=;
        b=kjW4zKzQIXey7VF6B0cwUkcn/X3NoH0eujJJf+MY8M82kWl7NmjItavIBdHBoPg++K
         IY9Zx4fUo7uugRYAKXUYURbjA3kkmj2CYb2MPdGifM6Hxc91Za2KMO/DZ16001Jil5TQ
         SwoVsRNhBkfi0HUW35mmHSqgTtj9bvDvpWoVEw568vp3E1zACvuZBYnze9ml8C/eiklX
         CwviOTDKpugXRNZTXsstBw4rCCXQPJy0F0AulbiJ42qwgo49xmh3iuUutWDKXXcviCqD
         0iyoxuWLy0VFS7NwtDkwsaCxAAwfUo4R8BEeahBGN3VToZgd56BIJiK8fnabHQdv6ymc
         0lyQ==
X-Gm-Message-State: AOJu0Yzxn3bnKFaOs0YyJk7gm9K6SZV9/87rvd/dYhuyy2Xd0NXDAMFo
	rn/E4w+ibMfoofdDHBWnle9O5+Cyq9x6jeZONE3iNglqxv5DQSyCR4PmJ7du
X-Google-Smtp-Source: AGHT+IFtbI9+CWD1Auxa9moJtSAklOkWsFFpBqwrVQEwf1r4zBMEyVFFRDmDo5gEKvONDfOdYqMwxg==
X-Received: by 2002:a05:6512:2812:b0:511:463c:32c1 with SMTP id cf18-20020a056512281200b00511463c32c1mr2648458lfb.19.1707147176431;
        Mon, 05 Feb 2024 07:32:56 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXjRqVStwX6pbmwqKFDtpxYXWQ2xqzbjeMcJjPMa8k5/uxF5sgYQ/3Nae4aUgMYqxDzYs87+I+3QjPQZ0dPS3MWDTuJSioRFJodZWLLfVT9fN5+euwV5j+LTMZQLdW1d+/1iBHjyVf9bG/1BxO8VtO1Zi++2TAz33rK0TmqIBhx/9V0EgNftyVnyHoyT+oh+fG+NQEPl6MkcX5dwUZyGod4ucetAsajapedUjFQcJpMMXyArBeKVmO66rp6UFno95+B55wrF/5hcDbAgsil5rrpGdiNOnzGaz4/FTBSh9CzhBXjj8AC3oiN6jk9mEbhNbxnEObsdgvl
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Bobby Eshleman <bobbyeshleman@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Oleksii Kurochko <oleksii.kurochko@gmail.com>
Subject: [PATCH v4 14/30] xen/riscv: introduce atomic.h
Date: Mon,  5 Feb 2024 16:32:21 +0100
Message-ID: <6554f2479e19ed3eae6de842ac1568c31d236461.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Bobby Eshleman <bobbyeshleman@gmail.com>

Additionally, this patch introduces macros in fence.h,
which are utilized in atomic.h.

atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n)
were updated to use __*xchg_generic().

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - do changes related to the updates of [PATCH v3 13/34] xen/riscv: introduce cmpxchg.h
 - drop casts in read_atomic_size(), write_atomic(), add_sized()
 - tabs -> spaces
 - drop #ifdef CONFIG_SMP ... #endif in fence.ha as it is simpler to handle NR_CPUS=1
   the same as NR_CPUS>1 with accepting less than ideal performance.
---
Changes in V3:
  - update the commit message
  - add SPDX for fence.h
  - code style fixes
  - Remove /* TODO: ... */ for add_sized macros. It looks correct to me.
  - re-order the patch
  - merge to this patch fence.h
---
Changes in V2:
 - Change an author of commit. I got this header from Bobby's old repo.
---
 xen/arch/riscv/include/asm/atomic.h | 395 ++++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/fence.h  |   8 +
 2 files changed, 403 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/atomic.h
 create mode 100644 xen/arch/riscv/include/asm/fence.h

diff --git a/xen/arch/riscv/include/asm/atomic.h b/xen/arch/riscv/include/asm/atomic.h
new file mode 100644
index 0000000000..267d3c0803
--- /dev/null
+++ b/xen/arch/riscv/include/asm/atomic.h
@@ -0,0 +1,395 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Taken and modified from Linux.
+ *
+ * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated to use
+ * __*xchg_generic()
+ * 
+ * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
+ * Copyright (C) 2012 Regents of the University of California
+ * Copyright (C) 2017 SiFive
+ * Copyright (C) 2021 Vates SAS
+ */
+
+#ifndef _ASM_RISCV_ATOMIC_H
+#define _ASM_RISCV_ATOMIC_H
+
+#include <xen/atomic.h>
+#include <asm/cmpxchg.h>
+#include <asm/fence.h>
+#include <asm/io.h>
+#include <asm/system.h>
+
+void __bad_atomic_size(void);
+
+static always_inline void read_atomic_size(const volatile void *p,
+                                           void *res,
+                                           unsigned int size)
+{
+    switch ( size )
+    {
+    case 1: *(uint8_t *)res = readb(p); break;
+    case 2: *(uint16_t *)res = readw(p); break;
+    case 4: *(uint32_t *)res = readl(p); break;
+    case 8: *(uint32_t *)res  = readq(p); break;
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define read_atomic(p) ({                               \
+    union { typeof(*p) val; char c[0]; } x_;            \
+    read_atomic_size(p, x_.c, sizeof(*p));              \
+    x_.val;                                             \
+})
+
+#define write_atomic(p, x)                              \
+({                                                      \
+    typeof(*p) x__ = (x);                               \
+    switch ( sizeof(*p) )                               \
+    {                                                   \
+    case 1: writeb((uint8_t)x__,  p); break;            \
+    case 2: writew((uint16_t)x__, p); break;            \
+    case 4: writel((uint32_t)x__, p); break;            \
+    case 8: writeq((uint64_t)x__, p); break;            \
+    default: __bad_atomic_size(); break;                \
+    }                                                   \
+    x__;                                                \
+})
+
+#define add_sized(p, x)                                 \
+({                                                      \
+    typeof(*(p)) x__ = (x);                             \
+    switch ( sizeof(*(p)) )                             \
+    {                                                   \
+    case 1: writeb(read_atomic(p) + x__, p); break;     \
+    case 2: writew(read_atomic(p) + x__, p); break;     \
+    case 4: writel(read_atomic(p) + x__, p); break;     \
+    default: __bad_atomic_size(); break;                \
+    }                                                   \
+})
+
+/*
+ *  __unqual_scalar_typeof(x) - Declare an unqualified scalar type, leaving
+ *               non-scalar types unchanged.
+ *
+ * Prefer C11 _Generic for better compile-times and simpler code. Note: 'char'
+ * is not type-compatible with 'signed char', and we define a separate case.
+ */
+#define __scalar_type_to_expr_cases(type)               \
+    unsigned type:  (unsigned type)0,                   \
+    signed type:    (signed type)0
+
+#define __unqual_scalar_typeof(x) typeof(               \
+    _Generic((x),                                       \
+        char:  (char)0,                                 \
+        __scalar_type_to_expr_cases(char),              \
+        __scalar_type_to_expr_cases(short),             \
+        __scalar_type_to_expr_cases(int),               \
+        __scalar_type_to_expr_cases(long),              \
+        __scalar_type_to_expr_cases(long long),         \
+        default: (x)))
+
+#define READ_ONCE(x)  (*(const volatile __unqual_scalar_typeof(x) *)&(x))
+#define WRITE_ONCE(x, val)                                      \
+    do {                                                        \
+        *(volatile typeof(x) *)&(x) = (val);                    \
+    } while (0)
+
+#define __atomic_acquire_fence() \
+    __asm__ __volatile__( RISCV_ACQUIRE_BARRIER "" ::: "memory" )
+
+#define __atomic_release_fence() \
+    __asm__ __volatile__( RISCV_RELEASE_BARRIER "" ::: "memory" );
+
+static inline int atomic_read(const atomic_t *v)
+{
+    return READ_ONCE(v->counter);
+}
+
+static inline int _atomic_read(atomic_t v)
+{
+    return v.counter;
+}
+
+static inline void atomic_set(atomic_t *v, int i)
+{
+    WRITE_ONCE(v->counter, i);
+}
+
+static inline void _atomic_set(atomic_t *v, int i)
+{
+    v->counter = i;
+}
+
+static inline int atomic_sub_and_test(int i, atomic_t *v)
+{
+    return atomic_sub_return(i, v) == 0;
+}
+
+static inline void atomic_inc(atomic_t *v)
+{
+    atomic_add(1, v);
+}
+
+static inline int atomic_inc_return(atomic_t *v)
+{
+    return atomic_add_return(1, v);
+}
+
+static inline void atomic_dec(atomic_t *v)
+{
+    atomic_sub(1, v);
+}
+
+static inline int atomic_dec_return(atomic_t *v)
+{
+    return atomic_sub_return(1, v);
+}
+
+static inline int atomic_dec_and_test(atomic_t *v)
+{
+    return atomic_sub_return(1, v) == 0;
+}
+
+static inline int atomic_add_negative(int i, atomic_t *v)
+{
+    return atomic_add_return(i, v) < 0;
+}
+
+static inline int atomic_inc_and_test(atomic_t *v)
+{
+    return atomic_add_return(1, v) == 0;
+}
+
+/*
+ * First, the atomic ops that have no ordering constraints and therefor don't
+ * have the AQ or RL bits set.  These don't return anything, so there's only
+ * one version to worry about.
+ */
+#define ATOMIC_OP(op, asm_op, I, asm_type, c_type, prefix)  \
+static inline                                               \
+void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \
+{                                                           \
+    __asm__ __volatile__ (                                  \
+        "   amo" #asm_op "." #asm_type " zero, %1, %0"      \
+        : "+A" (v->counter)                                 \
+        : "r" (I)                                           \
+        : "memory" );                                       \
+}                                                           \
+
+#define ATOMIC_OPS(op, asm_op, I)                           \
+        ATOMIC_OP (op, asm_op, I, w, int,   )
+
+ATOMIC_OPS(add, add,  i)
+ATOMIC_OPS(sub, add, -i)
+ATOMIC_OPS(and, and,  i)
+ATOMIC_OPS( or,  or,  i)
+ATOMIC_OPS(xor, xor,  i)
+
+#undef ATOMIC_OP
+#undef ATOMIC_OPS
+
+/*
+ * Atomic ops that have ordered, relaxed, acquire, and release variants.
+ * There's two flavors of these: the arithmatic ops have both fetch and return
+ * versions, while the logical ops only have fetch versions.
+ */
+#define ATOMIC_FETCH_OP(op, asm_op, I, asm_type, c_type, prefix)    \
+static inline                                                       \
+c_type atomic##prefix##_fetch_##op##_relaxed(c_type i,              \
+                         atomic##prefix##_t *v)                     \
+{                                                                   \
+    register c_type ret;                                            \
+    __asm__ __volatile__ (                                          \
+        "   amo" #asm_op "." #asm_type " %1, %2, %0"                \
+        : "+A" (v->counter), "=r" (ret)                             \
+        : "r" (I)                                                   \
+        : "memory" );                                               \
+    return ret;                                                     \
+}                                                                   \
+static inline                                                       \
+c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t *v) \
+{                                                                   \
+    register c_type ret;                                            \
+    __asm__ __volatile__ (                                          \
+        "   amo" #asm_op "." #asm_type ".aqrl  %1, %2, %0"          \
+        : "+A" (v->counter), "=r" (ret)                             \
+        : "r" (I)                                                   \
+        : "memory" );                                               \
+    return ret;                                                     \
+}
+
+#define ATOMIC_OP_RETURN(op, asm_op, c_op, I, asm_type, c_type, prefix) \
+static inline                                                           \
+c_type atomic##prefix##_##op##_return_relaxed(c_type i,                 \
+                          atomic##prefix##_t *v)                        \
+{                                                                       \
+        return atomic##prefix##_fetch_##op##_relaxed(i, v) c_op I;      \
+}                                                                       \
+static inline                                                           \
+c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t *v)  \
+{                                                                       \
+        return atomic##prefix##_fetch_##op(i, v) c_op I;                \
+}
+
+#define ATOMIC_OPS(op, asm_op, c_op, I)                                 \
+        ATOMIC_FETCH_OP( op, asm_op,       I, w, int,   )               \
+        ATOMIC_OP_RETURN(op, asm_op, c_op, I, w, int,   )
+
+ATOMIC_OPS(add, add, +,  i)
+ATOMIC_OPS(sub, add, +, -i)
+
+#define atomic_add_return_relaxed   atomic_add_return_relaxed
+#define atomic_sub_return_relaxed   atomic_sub_return_relaxed
+#define atomic_add_return   atomic_add_return
+#define atomic_sub_return   atomic_sub_return
+
+#define atomic_fetch_add_relaxed    atomic_fetch_add_relaxed
+#define atomic_fetch_sub_relaxed    atomic_fetch_sub_relaxed
+#define atomic_fetch_add    atomic_fetch_add
+#define atomic_fetch_sub    atomic_fetch_sub
+
+#undef ATOMIC_OPS
+
+#define ATOMIC_OPS(op, asm_op, I) \
+        ATOMIC_FETCH_OP(op, asm_op, I, w, int,   )
+
+ATOMIC_OPS(and, and, i)
+ATOMIC_OPS( or,  or, i)
+ATOMIC_OPS(xor, xor, i)
+
+#define atomic_fetch_and_relaxed    atomic_fetch_and_relaxed
+#define atomic_fetch_or_relaxed	    atomic_fetch_or_relaxed
+#define atomic_fetch_xor_relaxed    atomic_fetch_xor_relaxed
+#define atomic_fetch_and    atomic_fetch_and
+#define atomic_fetch_or     atomic_fetch_or
+#define atomic_fetch_xor    atomic_fetch_xor
+
+#undef ATOMIC_OPS
+
+#undef ATOMIC_FETCH_OP
+#undef ATOMIC_OP_RETURN
+
+/* This is required to provide a full barrier on success. */
+static inline int atomic_add_unless(atomic_t *v, int a, int u)
+{
+       int prev, rc;
+
+    __asm__ __volatile__ (
+        "0: lr.w     %[p],  %[c]\n"
+        "   beq      %[p],  %[u], 1f\n"
+        "   add      %[rc], %[p], %[a]\n"
+        "   sc.w.rl  %[rc], %[rc], %[c]\n"
+        "   bnez     %[rc], 0b\n"
+        "   fence    rw, rw\n"
+        "1:\n"
+        : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
+        : [a]"r" (a), [u]"r" (u)
+        : "memory");
+    return prev;
+}
+#define atomic_fetch_add_unless atomic_fetch_add_unless
+
+/*
+ * atomic_{cmp,}xchg is required to have exactly the same ordering semantics as
+ * {cmp,}xchg and the operations that return, so they need a full barrier.
+ */
+#define ATOMIC_OP(c_t, prefix, size)                            \
+static inline                                                   \
+c_t atomic##prefix##_xchg_relaxed(atomic##prefix##_t *v, c_t n) \
+{                                                               \
+    return __xchg_generic(&(v->counter), n, size, "", "", "");  \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_xchg_acquire(atomic##prefix##_t *v, c_t n) \
+{                                                               \
+    return __xchg_generic(&(v->counter), n, size,               \
+                          "", "", RISCV_ACQUIRE_BARRIER);       \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_xchg_release(atomic##prefix##_t *v, c_t n) \
+{                                                               \
+    return __xchg_generic(&(v->counter), n, size,               \
+                          "", RISCV_RELEASE_BARRIER, "");       \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n)         \
+{                                                               \
+    return __xchg_generic(&(v->counter), n, size,               \
+                          ".aqrl", "", "");                     \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg_relaxed(atomic##prefix##_t *v,     \
+                     c_t o, c_t n)                              \
+{                                                               \
+    return __cmpxchg_generic(&(v->counter), o, n, size,         \
+                             "", "", "");                       \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg_acquire(atomic##prefix##_t *v,     \
+                     c_t o, c_t n)                              \
+{                                                               \
+    return __cmpxchg_generic(&(v->counter), o, n, size,         \
+                             "", "", RISCV_ACQUIRE_BARRIER);    \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg_release(atomic##prefix##_t *v,     \
+                     c_t o, c_t n)                              \
+{	                                                            \
+    return __cmpxchg_generic(&(v->counter), o, n, size,         \
+                             "", RISCV_RELEASE_BARRIER, "");    \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n) \
+{                                                               \
+    return __cmpxchg_generic(&(v->counter), o, n, size,         \
+                             ".rl", "", " fence rw, rw\n");     \
+}
+
+#define ATOMIC_OPS() \
+    ATOMIC_OP(int,   , 4)
+
+ATOMIC_OPS()
+
+#define atomic_xchg_relaxed atomic_xchg_relaxed
+#define atomic_xchg_acquire atomic_xchg_acquire
+#define atomic_xchg_release atomic_xchg_release
+#define atomic_xchg atomic_xchg
+#define atomic_cmpxchg_relaxed atomic_cmpxchg_relaxed
+#define atomic_cmpxchg_acquire atomic_cmpxchg_acquire
+#define atomic_cmpxchg_release atomic_cmpxchg_release
+#define atomic_cmpxchg atomic_cmpxchg
+
+#undef ATOMIC_OPS
+#undef ATOMIC_OP
+
+static inline int atomic_sub_if_positive(atomic_t *v, int offset)
+{
+       int prev, rc;
+
+    __asm__ __volatile__ (
+        "0: lr.w     %[p],  %[c]\n"
+        "   sub      %[rc], %[p], %[o]\n"
+        "   bltz     %[rc], 1f\n"
+        "   sc.w.rl  %[rc], %[rc], %[c]\n"
+        "   bnez     %[rc], 0b\n"
+        "   fence    rw, rw\n"
+        "1:\n"
+        : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
+        : [o]"r" (offset)
+        : "memory" );
+    return prev - offset;
+}
+
+#define atomic_dec_if_positive(v)	atomic_sub_if_positive(v, 1)
+
+#endif /* _ASM_RISCV_ATOMIC_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/riscv/include/asm/fence.h b/xen/arch/riscv/include/asm/fence.h
new file mode 100644
index 0000000000..ff3f23dbd7
--- /dev/null
+++ b/xen/arch/riscv/include/asm/fence.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef _ASM_RISCV_FENCE_H
+#define _ASM_RISCV_FENCE_H
+
+#define RISCV_ACQUIRE_BARRIER   "\tfence r , rw\n"
+#define RISCV_RELEASE_BARRIER   "\tfence rw,  w\n"
+
+#endif	/* _ASM_RISCV_FENCE_H */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:36:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:36:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676254.1052368 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX114-0001rf-By; Mon, 05 Feb 2024 15:36:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676254.1052368; Mon, 05 Feb 2024 15:36:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX114-0001rY-96; Mon, 05 Feb 2024 15:36:14 +0000
Received: by outflank-mailman (input) for mailman id 676254;
 Mon, 05 Feb 2024 15:36:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GD/d=JO=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rX112-0001lB-SS
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:36:12 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 49b6f029-c43c-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:36:11 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id B3FCF4EE0737;
 Mon,  5 Feb 2024 16:36:10 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 49b6f029-c43c-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Mon, 05 Feb 2024 16:36:10 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Simone Ballarin <simone.ballarin@bugseng.com>, Stefano Stabellini
 <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, Maria Celeste
 Cesario <maria.celeste.cesario@bugseng.com>, Jan Beulich
 <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Kevin Tian <kevin.tian@intel.com>
Subject: Re: [PATCH v2 1/2] x86/IOMMU: address violations of MISRA C:2012 Rule
 14.4
In-Reply-To: <746a33fff1386b2e76657b5f7cfb31f3b117a1fe.1702310368.git.maria.celeste.cesario@bugseng.com>
References: <cover.1702310368.git.maria.celeste.cesario@bugseng.com>
 <746a33fff1386b2e76657b5f7cfb31f3b117a1fe.1702310368.git.maria.celeste.cesario@bugseng.com>
Message-ID: <891a1ab0b772910999f49878ae679634@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2023-12-13 17:10, Simone Ballarin wrote:
> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
> 
> The xen sources contain violations of MISRA C:2012 Rule 14.4 whose
> headline states:
> "The controlling expression of an if statement and the controlling
> expression of an iteration-statement shall have essentially Boolean 
> type".
> 
> Add comparisons to avoid using enum constants as controlling 
> expressions
> to comply with Rule 14.4.
> No functional change.
> 
> Signed-off-by: Maria Celeste Cesario  
> <maria.celeste.cesario@bugseng.com>
> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
> ---
> Changes in v2
> - rename prefix from AMD/IOMMU to x86/IOMMU
> - move changes on msi.c and hpet.c in this patch.
> ---
>  xen/arch/x86/hpet.c                      | 6 +++---
>  xen/arch/x86/msi.c                       | 4 ++--
>  xen/drivers/passthrough/amd/iommu_init.c | 4 ++--
>  xen/drivers/passthrough/vtd/iommu.c      | 4 ++--
>  xen/drivers/passthrough/vtd/quirks.c     | 2 +-
>  5 files changed, 10 insertions(+), 10 deletions(-)
> 

+Stefano

Hi all,

this patch seems not to have been committed into staging, unlike the 
other patch from this series. Since these are the only remaining 
violations for Rule 14.4, then I think these changes could be reviewed.

This should apply cleanly on staging, but I can send a rebased version 
if there are problems.

Additionally, despite having patches addressing R14.4 already in 
staging, the rule itself is not in docs/misra/rules.rst due to concerns 
on the patch that aimed to add it [1], so it's probably best to send a 
new version of that as well, if Stefano agrees.

[1] 
https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2308301840520.6458@ubuntu-linux-20-04-desktop/


-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:37:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:37:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676260.1052380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX12a-0002dQ-Nv; Mon, 05 Feb 2024 15:37:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676260.1052380; Mon, 05 Feb 2024 15:37:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX12a-0002dJ-JZ; Mon, 05 Feb 2024 15:37:48 +0000
Received: by outflank-mailman (input) for mailman id 676260;
 Mon, 05 Feb 2024 15:37:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0y1-0007Hv-3r
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:05 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d9e34def-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:33:03 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-5112bd13a4fso4071568e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:03 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d9e34def-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147183; x=1707751983; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nSkC4h9RHu0ZPi8yVkfyYrb9TlPun8CaqbFwSR6X+nY=;
        b=lgAOSssgK0izZvo/eByOYxCZOcyQuLl1LphlcFgDfrw4bTM0CMdT+7oaC8UE0pqGMZ
         o6DoafnVO+RZsMiDEiq4bWPyGNGENGVrVu+1AIwfmRmlX0Kz3Wb6cym2z2sZdQIxM+8D
         lZggEUd3deN3Nc68HFWj2E9A0ljMhOu67U0XkogkDk/zmMXOXfeeElMG9YZzufBLHIX+
         zqUn4B6M2z1yKPQBr3SHlMuThthRLG8mLjjcf5KlkjBAzWFwnUlHdheGtQKwJKF9di5b
         +NrHefe3dpHsopNA8xgc86q8axLgXHel3zeXB7TIfSLbDdU75b0SrE4cEd8oQi9pPdei
         NWGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147183; x=1707751983;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=nSkC4h9RHu0ZPi8yVkfyYrb9TlPun8CaqbFwSR6X+nY=;
        b=OHtEHmKiDBhJQyBoDiTg2aykeOmnjtmO1wJE9ZPaUtVuoYC5O4IiDghheR4itjBq2M
         2gvn2rQ2I6GTkpdox2FgeEbMnudhsatWamhk8fb2QsWpiBnAzSU2XBb5j2rU6m8905Tq
         p7uJTl1N4DdZkAUzivMNgZOH9t10V5QMUiptVvELLDngoKrI82ClCpmCf/ujH/4bO24k
         OmXeVI7WUgJDvND9QMmNPEnrFMkmVR1iuwmCVCuysovSKhPs3unAIJswNOlhAfWFEfYl
         +kJ8LJ74zDgWvPbtTc97c+ep5zuJwWSbgYVw9V4542MGfK5HfQuWNMytrwPLdsN99uvq
         RK7Q==
X-Gm-Message-State: AOJu0Ywz/VL81NFVqYzOkItZKlgP1Z8im6UkeaKy2jL5/40NOJBL51Zy
	zCwIDHSJX2ezPTtGhRIzheaRnAbXeehb+Qs7BS0uvfirUCqm0TZppYE2RrCZ
X-Google-Smtp-Source: AGHT+IGTlPwspuKMw53Na5DdBtJmFPAAq5A3ih9VNf0oCvaDXAg4cGvieTC4z72KOBq4nEL9xYpnWw==
X-Received: by 2002:a19:4f57:0:b0:511:4646:54f3 with SMTP id a23-20020a194f57000000b00511464654f3mr14586lfk.10.1707147182714;
        Mon, 05 Feb 2024 07:33:02 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVGXvLe4XJsrxfdoixZ0NqlGgka//FC5/RH2AS+rnUaVAp3xJu/mcmHp12yRfb30V2wK5zQgc9x03U4B4wHAUQfJrYvTTGmFUrhsHMWJN7LP2wJGa3ZsD9Uc2WJ7tRri/tJXigsrxgosYLy+fSoMDCwQXBuuCSyh4EyDim9bZ4DtIWfxwXzfRQlVW9TSfdfHK07mcmqdIVNtUkTz7GLtliRrzbh7WUkclVrm0plT50H1FanZgtI20Pep8E6Gi/vJvYTjMQbr+e63Eeu9bskPRfnFW2u1FyIke69s7aw/kArt41+sFXcurvgvorUa7hEqy8AFf5+
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 21/30] xen/riscv: add definition of __read_mostly
Date: Mon,  5 Feb 2024 16:32:28 +0100
Message-ID: <5aff6b7d13b17e4ce5807f589c0be20f2e9e0bc0.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The definition of __read_mostly should be removed in:
https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

The patch introduces it in arch-specific header to not
block enabling of full Xen build for RISC-V.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.

[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/
---
Changes in V4:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/cache.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/riscv/include/asm/cache.h b/xen/arch/riscv/include/asm/cache.h
index 69573eb051..94bd94db53 100644
--- a/xen/arch/riscv/include/asm/cache.h
+++ b/xen/arch/riscv/include/asm/cache.h
@@ -3,4 +3,6 @@
 #ifndef _ASM_RISCV_CACHE_H
 #define _ASM_RISCV_CACHE_H
 
+#define __read_mostly __section(".data.read_mostly")
+
 #endif /* _ASM_RISCV_CACHE_H */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:37:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:37:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676261.1052381 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX12a-0002g5-Tk; Mon, 05 Feb 2024 15:37:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676261.1052381; Mon, 05 Feb 2024 15:37:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX12a-0002fR-QF; Mon, 05 Feb 2024 15:37:48 +0000
Received: by outflank-mailman (input) for mailman id 676261;
 Mon, 05 Feb 2024 15:37:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0y5-0007I6-SU
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:09 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dc634284-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:33:07 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51032e62171so7141623e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:07 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc634284-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147187; x=1707751987; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Y7brBIk5MysXO16VnJHkZ93Jfj50NOxcYHO448pB7ag=;
        b=fe7mh46Xyrq6i5QD/11j0xHs6PptfKuUnHsugcVOR24T8My0RU6nizwVVfpUmoSeuB
         BtlOBnH0vwhS7UGJ3+L5txy7u+qSJk3Ml5VCiZhYOJMDqB9S+A273YFRdkipHwRqmF8Y
         yeDU9USZ7PkhFHK7qE3NN/Z+X9f7g+CZ7nNUg+8PGyL+80HMAgCmuqrBp+gct02QK8PU
         82K5eKj2PmhRdBAp0TQIWVXqutytQiLXvAO9HoXLiPs+hezQaS9j4C8g9Oxs8lqlNYIs
         GLcUQgTNQ5zYADs2limQ4nc2DOCedsh/dXt1Tl9wIeXgqkbYODOF2ZjYzzaZRUsHM155
         nAnA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147187; x=1707751987;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Y7brBIk5MysXO16VnJHkZ93Jfj50NOxcYHO448pB7ag=;
        b=aJcnTHL5sWYe/8UecIKtGFMG8mxX3/x0NcWx7VS9BZi4lCfxG4rWtaSzfj8U36ie9y
         UzYyVT+73H5wGcSUxG+3XXRhxWR4tC+YN1e/KCscHN6i5kVOJrBGi8gRR9MpxGYQwFwz
         B1DNiiK6N+zolfoPvwTDDRkAu/xO+p7LJVLqfidYsrCyKUb2e9QgEbvhs1jOqvnTpO1U
         PHYs5pTbWk5V7rQOyUa4WCl2VfF2T54XT7Hy3rUBbj6LukxZ8yYUTN0DOkIHOvT3jsUn
         aom12p80kNlW6J5akE8WXP+Lg1NuFBJGdZgnA6wAAORIUSQy4ZBmq103ESMGobB/Du6S
         lrKg==
X-Gm-Message-State: AOJu0Yz7njnfImFwMreJ1YfbpeAf1AmGZhWww3OKc/RpChQHu7Oz+f4x
	sp/y/mB3uL7bKseEpk8UhPEQX5o4YnCZThiRPBBXCrbrBPE2eMfAY4PhlE+Y
X-Google-Smtp-Source: AGHT+IGrGY9dXD4pPSfu5iE7VQQMzy81yEw7tzqQsibUYMqVTWoJSX/R/n24juQ1uTbrQLCHkrNi3g==
X-Received: by 2002:a05:6512:3086:b0:511:454e:6032 with SMTP id z6-20020a056512308600b00511454e6032mr4547401lfd.63.1707147186804;
        Mon, 05 Feb 2024 07:33:06 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUT+G1Wcy19fjqdydwn2cz2uc6BFNnClMwDyYcDUt+dwrwULDfpu8oNQtWL8YSKKi/bKFdRjSwkuwwTZhKqIZaAj4pDC2SBZV9QaTgxjIoftws95qEgJRbT5lUE03IuwsGmtpcz2OmfMTONKVvAFGAebyQxIb1JBflv9bp8sFFAqwo+zxVPdF++r6lqnP8gZDIp2tPHo1OlDLVPqATrhfUHJiZWNDtiMheV/UIQJ3fSIQTWubpWHBVP192SZGVd3EbuClbVhRcSQnRCeKHqwO4nbNu+I4nVJQcV0Ag52861SIc4S+9w2FE5k1/0qnbdtmAbvVZX
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v4 25/30] xen/riscv: add minimal stuff to processor.h to build full Xen
Date: Mon,  5 Feb 2024 16:32:32 +0100
Message-ID: <9efbc232f64b6192cf83f865b8987846fe082720.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - Change message -> subject in "Changes in V3"
 - Documentation about system requirement was added. In the future, it can be checked if the extension is supported
   by system __riscv_isa_extension_available() ( https://gitlab.com/xen-project/people/olkur/xen/-/commit/737998e89ed305eb92059300c374dfa53d2143fa )
 - update cpu_relax() function to check if __riscv_zihintpause is supported by a toolchain
 - add conditional _zihintpause to -march if it is supported by a toolchain
Changes in V3:
 - update the commit subject
 - rename get_processor_id to smp_processor_id
 - code style fixes
 - update the cpu_relax instruction: use pause instruction instead of div %0, %0, zero
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 docs/misc/riscv/booting.txt            |  8 ++++++++
 xen/arch/riscv/Kconfig                 |  7 +++++++
 xen/arch/riscv/arch.mk                 |  1 +
 xen/arch/riscv/include/asm/processor.h | 23 +++++++++++++++++++++++
 4 files changed, 39 insertions(+)
 create mode 100644 docs/misc/riscv/booting.txt

diff --git a/docs/misc/riscv/booting.txt b/docs/misc/riscv/booting.txt
new file mode 100644
index 0000000000..38fad74956
--- /dev/null
+++ b/docs/misc/riscv/booting.txt
@@ -0,0 +1,8 @@
+System requirements
+===================
+
+The following extensions are expected to be supported by a system on which
+Xen is run:
+- Zihintpause:
+  On a system that doesn't have this extension, cpu_relax() should be
+  implemented properly. Otherwise, an illegal instruction exception will arise.
diff --git a/xen/arch/riscv/Kconfig b/xen/arch/riscv/Kconfig
index f382b36f6c..383ce06771 100644
--- a/xen/arch/riscv/Kconfig
+++ b/xen/arch/riscv/Kconfig
@@ -45,6 +45,13 @@ config RISCV_ISA_C
 
 	  If unsure, say Y.
 
+config TOOLCHAIN_HAS_ZIHINTPAUSE
+	bool
+	default y
+	depends on !64BIT || $(cc-option,-mabi=lp64 -march=rv64ima_zihintpause)
+	depends on !32BIT || $(cc-option,-mabi=ilp32 -march=rv32ima_zihintpause)
+	depends on LLD_VERSION >= 150000 || LD_VERSION >= 23600
+
 endmenu
 
 source "common/Kconfig"
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index 8403f96b6f..a4b53adaf7 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -7,6 +7,7 @@ CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64
 
 riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
 riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
+riscv-march-$(CONFIG_TOOLCHAIN_HAS_ZIHINTPAUSE) := $(riscv-march-y)_zihintpause
 
 # Note that -mcmodel=medany is used so that Xen can be mapped
 # into the upper half _or_ the lower half of the address space.
diff --git a/xen/arch/riscv/include/asm/processor.h b/xen/arch/riscv/include/asm/processor.h
index 6db681d805..289dc35ea0 100644
--- a/xen/arch/riscv/include/asm/processor.h
+++ b/xen/arch/riscv/include/asm/processor.h
@@ -12,6 +12,9 @@
 
 #ifndef __ASSEMBLY__
 
+/* TODO: need to be implemeted */
+#define smp_processor_id() 0
+
 /* On stack VCPU state */
 struct cpu_user_regs
 {
@@ -53,6 +56,26 @@ struct cpu_user_regs
     unsigned long pregs;
 };
 
+/* TODO: need to implement */
+#define cpu_to_core(cpu)   (0)
+#define cpu_to_socket(cpu) (0)
+
+static inline void cpu_relax(void)
+{
+#ifdef __riscv_zihintpause
+    /*
+     * Reduce instruction retirement.
+     * This assumes the PC changes.
+     */
+    __asm__ __volatile__ ("pause");
+#else
+    /* Encoding of the pause instruction */
+    __asm__ __volatile__ (".insn 0x100000F");
+#endif
+
+    barrier();
+}
+
 static inline void wfi(void)
 {
     __asm__ __volatile__ ("wfi");
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:39:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:39:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676273.1052400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX13q-00047N-AU; Mon, 05 Feb 2024 15:39:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676273.1052400; Mon, 05 Feb 2024 15:39:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX13q-00047G-5v; Mon, 05 Feb 2024 15:39:06 +0000
Received: by outflank-mailman (input) for mailman id 676273;
 Mon, 05 Feb 2024 15:39:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0xy-0007I6-Gk
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:02 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d8531ee9-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:33:00 +0100 (CET)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-511531f03f6so810453e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:00 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.32.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:32:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8531ee9-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147180; x=1707751980; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Fh1fXKNPLCGaG/VnLCvzTyh2EW511i6o4O6lrTHgeBw=;
        b=UpuoIWF6MvaSPzfv6soZwe0sbtgkS9Yip/6za+EPpxY88tZSbeutaM1tOOPtXMjo98
         lay7woK7VDAApoAxqlQRLbPhuxlOKZUYbIQmNDiJ5bW6ZbUQJsbkuMv5nUdiFT+NNLbP
         gBU6xqQdIrog9AxJXDiPg0R5s+5bKXuXCDpLqaNuXuEsuuvpC2E8knGfZL/Sio7TzK2J
         Z1l3HccB7SpwDutagRoKVTPiOMSmUOM3eYFJgYwZ/fa7nM9bkErXQ5bTM0dbpYHg6emK
         mi0YTrwp+rqO8LHnxc1mR1w4YamccPHB7xfektTywCzzxn+3lq0+z7t7t4Cbb+qOe1Zx
         +lQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147180; x=1707751980;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Fh1fXKNPLCGaG/VnLCvzTyh2EW511i6o4O6lrTHgeBw=;
        b=N2sBglsHC71O8wKAiP3C7xcd8ruKpH5ThLz5zolJ69Y1nxGannhdVtAUUBM2s5VEJV
         YQNRryrcDdvDXUBLyL/KwxdBZKF7dSTOxb3Acoh/yXabXpX6TRy1DP1l39krYAzImEST
         0JTbAiRni9/20sdQ3yaPHPVZ8oh2ZvmtKMnyMYClnEb0XOPjbKic53VTf1ixuyvok/oS
         RQoq3LpqdgUJmdiJdNXp99ENzEY4ffFO7k1eHlFOfzu8ItdZmESC467HVQlU8q6S7IE3
         44vSJ8p9iHejd7xeZf6DBZ557txgzUeey8NHf9PTlrwGv1z+cbhBUp6Q6BHWWt6RiQSc
         vVGg==
X-Gm-Message-State: AOJu0YxLQZ5cY5V+MvVAvWmA6Ejd463vmbeWWcoH6fVVzgFb9zfLIjCa
	GBLvH5/gyipUOEwEdYjEkbqvFSo07BKkCR6Nd4M+Y+wACBH2mU/tmjU/YGY5
X-Google-Smtp-Source: AGHT+IHUe+H4IHHtzSGa/KZg/3OWOYeEcxwYLKDEgMAfnmSQCcWtVTsDgjs4MKtetv1K08sdHB0AcA==
X-Received: by 2002:a05:6512:3151:b0:511:4ee4:48d1 with SMTP id s17-20020a056512315100b005114ee448d1mr2311431lfi.32.1707147180083;
        Mon, 05 Feb 2024 07:33:00 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVv0V2Sn1riFTcccMtaw4DrDISuAWzeSjJbFZLS60Bt1zzm5rHHNZZ5719o0GuJ3Zloh8JRFR0HWBsQljxCnJIG4dy4pcTfVNMkTQlozDoJRy+gAQtiuH0Yyu7MVjp1B+qAAk609pFnX4azd5lmdlumMjsLfqM3KFxVu41h95BXd3MpBtNtCWdCiL1Bpv2LylLvLH2rcDmnBclr9a0YkFLuukm6VwYXwke3MkV+2VMm2Q/gTtx+6goiKCCWRmdVMWQ2dnUbDxYKNzFloaLeXv8/PM+fSDzFGJ3wqHLedvq9nwNTljLZyK+OXWtWIvV/2JWgbGzr
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 18/30] xen/riscv: introduce time.h
Date: Mon,  5 Feb 2024 16:32:25 +0100
Message-ID: <f0a4566c185469fa9f5de41b5bbbc7234cfa9912.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V4:
 - s/BUG()/BUG_ON("unimplemented")
---
Changes in V3:
 - Acked-by: Jan Beulich <jbeulich@suse.com>
 - add SPDX
 - Add new line
---
Changes in V2:
 - change xen/lib.h to xen/bug.h
 - remove inclusion of <asm/processor.h> as it's not needed.
---
 xen/arch/riscv/include/asm/time.h | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/time.h

diff --git a/xen/arch/riscv/include/asm/time.h b/xen/arch/riscv/include/asm/time.h
new file mode 100644
index 0000000000..2e359fa046
--- /dev/null
+++ b/xen/arch/riscv/include/asm/time.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_TIME_H__
+#define __ASM_RISCV_TIME_H__
+
+#include <xen/bug.h>
+#include <asm/csr.h>
+
+struct vcpu;
+
+/* TODO: implement */
+static inline void force_update_vcpu_system_time(struct vcpu *v) { BUG_ON("unimplemented"); }
+
+typedef unsigned long cycles_t;
+
+static inline cycles_t get_cycles(void)
+{
+	return csr_read(CSR_TIME);
+}
+
+#endif /* __ASM_RISCV_TIME_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:39:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:39:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676274.1052409 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX13r-0004Ll-GO; Mon, 05 Feb 2024 15:39:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676274.1052409; Mon, 05 Feb 2024 15:39:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX13r-0004LY-Dj; Mon, 05 Feb 2024 15:39:07 +0000
Received: by outflank-mailman (input) for mailman id 676274;
 Mon, 05 Feb 2024 15:39:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0y5-0007Hv-D1
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:09 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dcf60bb0-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:33:08 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-511570b2f49so136041e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:08 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dcf60bb0-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147188; x=1707751988; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9TiqYRhQ4dcRbS7+3AgrdpkDQ6DcscMTVBQpmiSzNsk=;
        b=llAvG+9wbsdpolhktv/1m8aP0/DJcT2g62HV1P31daQbTsvNQrdJ4gRsOADnw0XDe6
         S3KWibAsW9lneDvsg0My4RUTv12148THgMlBHrjSGpSoKx/kjp8JM1tF5IZLOJuzPGy6
         67BMAhjlF2JHxCsV13a3UyQQ7yp0FZAAz1J1w82Gpgfqg5eQeujQq1ZSTKaOGYbxSgCf
         ZZ2pV0YDiuN+tQjxq8TResuOqTRSCoGQWrNlUmAqDM/LHTuouUPnoPr3KWnpFW4LvWA2
         KDAn2EX5YTQmSu05LgtO7Y9oeXntmarTfyDVjprboMR8JEM75Y9zfFZWWdYm73G6Q1YF
         9j5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147188; x=1707751988;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=9TiqYRhQ4dcRbS7+3AgrdpkDQ6DcscMTVBQpmiSzNsk=;
        b=nMBxpY5ISnUZ219iiijlhefmtsY/84Vm6hIz4TGj6NWzMAlzViNe9gfKmqkeGQZ20m
         IeHSnRC1ZKqMertcLSaQOtIFoidmlZDCBRB0ELJ6sbMuykvWBVgfZI1iO9/tI6g5CI3S
         AfC8sbRKDVqiF5vErEIHh6U/eG3ghbx+1siT1BjmD5NZ0Zf4YwZVDzmBOEVfNXT+5CAp
         xAWq4r89X4nkej91/68an8GlSX15Um4PzIp5Z/dtBJAXx2foahKUeNKJAUY5V0Acuud3
         gFms2sLAaXx0/c8tnSNLsUsnulln2gItmykqdcndp8unbFWJPqW27JeB3NQGJc0iACM6
         9lXQ==
X-Gm-Message-State: AOJu0YxTgYwhuiY5xqg5xW8T4DAiknmMHRiYN1bSAxvNpy/Zu2z9d5tp
	m2Ru4CqmEZ8DsxKV1It+HYATXcHcEf71EYRU3BR0LuQWEW5lKcytiJO76rZe
X-Google-Smtp-Source: AGHT+IGIC9dZAHKKEFYYA2ddyPhitcB+XaJ84TU2oHRdoylepvqUSn6XWSwKYnSEo/3aCiVUx3xiyg==
X-Received: by 2002:ac2:4c07:0:b0:511:48fe:1b07 with SMTP id t7-20020ac24c07000000b0051148fe1b07mr12441lfq.5.1707147187749;
        Mon, 05 Feb 2024 07:33:07 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXqqlMbaSt5dGe6wWB/Mq9M7T+par/wtcW9fLFZQD0jxwZqidUCJuT7gLsc0f9m5J5Z0qt6rISb/JSAYl79PL/E4plmr7tdNgMVghLgHOXCYjfvDZ0bdqIx95KTjKCM+RAwLlhFo6A+wMdgkkOr9903rCCkn+jtVV1XbIjhy7f1WnAvKXn7+YJ5KXvJlyzFWETuCl8er+K94UNkv7F/SimG+ceohExy49hPvZqnjGBC16yPsKA0c/AtgMp81TG2Bm/0KBZwr2OlfR49KakGMAxSkT+e9iG5N2xCvKREBtmkgIHjxHUyt9KPsUnmAtF+3YfvHxxO
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 26/30] xen/riscv: add minimal stuff to mm.h to build full Xen
Date: Mon,  5 Feb 2024 16:32:33 +0100
Message-ID: <369fe9e80d11c5e499810a6819fc1c282ce60628.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - update an argument name of PFN_ORDERN macros.
 - drop pad at the end of 'struct page_info'.
 - Change message -> subject in "Changes in V3"
 - delete duplicated macros from riscv/mm.h
 - fix identation in struct page_info
 - align comment for PGC_ macros
 - update definitions of domain_set_alloc_bitsize() and domain_clamp_alloc_bitsize()
 - drop unnessary comments.
 - s/BUG/BUG_ON("...")
 - define __virt_to_maddr, __maddr_to_virt as stubs
 - add inclusion of xen/mm-frame.h for mfn_x and others
 - include "xen/mm.h" instead of "asm/mm.h" to fix compilation issues:
	 In file included from arch/riscv/setup.c:7:
	./arch/riscv/include/asm/mm.h:60:28: error: field 'list' has incomplete type
	   60 |     struct page_list_entry list;
	      |                            ^~~~
	./arch/riscv/include/asm/mm.h:81:43: error: 'MAX_ORDER' undeclared here (not in a function)
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
	      |                                           ^~~~~~~~~
	./arch/riscv/include/asm/mm.h:81:31: error: bit-field 'first_dirty' width not an integer constant
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
 - Define __virt_to_mfn() and __mfn_to_virt() using maddr_to_mfn() and mfn_to_maddr().
---
Changes in V3:
 - update the commit title
 - introduce DIRECTMAP_VIRT_START.
 - drop changes related pfn_to_paddr() and paddr_to_pfn as they were remvoe in
   [PATCH v2 32/39] xen/riscv: add minimal stuff to asm/page.h to build full Xen
 - code style fixes.
 - drop get_page_nr  and put_page_nr as they don't need for time being
 - drop CONFIG_STATIC_MEMORY related things
 - code style fixes
---
Changes in V2:
 - define stub for arch_get_dma_bitsize(void)
---
 xen/arch/riscv/include/asm/mm.h | 246 ++++++++++++++++++++++++++++++++
 xen/arch/riscv/mm.c             |   2 +-
 xen/arch/riscv/setup.c          |   2 +-
 3 files changed, 248 insertions(+), 2 deletions(-)

diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
index 07c7a0abba..0254babcc1 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -3,11 +3,252 @@
 #ifndef _ASM_RISCV_MM_H
 #define _ASM_RISCV_MM_H
 
+#include <public/xen.h>
+#include <xen/bug.h>
+#include <xen/mm-frame.h>
+#include <xen/pdx.h>
+#include <xen/types.h>
+
 #include <asm/page-bits.h>
 
 #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
 #define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
 
+#define paddr_to_pdx(pa)    mfn_to_pdx(maddr_to_mfn(pa))
+#define gfn_to_gaddr(gfn)   pfn_to_paddr(gfn_x(gfn))
+#define gaddr_to_gfn(ga)    _gfn(paddr_to_pfn(ga))
+#define mfn_to_maddr(mfn)   pfn_to_paddr(mfn_x(mfn))
+#define maddr_to_mfn(ma)    _mfn(paddr_to_pfn(ma))
+#define vmap_to_mfn(va)     maddr_to_mfn(virt_to_maddr((vaddr_t)va))
+#define vmap_to_page(va)    mfn_to_page(vmap_to_mfn(va))
+
+static inline unsigned long __virt_to_maddr(unsigned long va)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline void *__maddr_to_virt(unsigned long ma)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+#define virt_to_maddr(va) __virt_to_maddr((unsigned long)(va))
+#define maddr_to_virt(pa) __maddr_to_virt((unsigned long)(pa))
+
+/* Convert between Xen-heap virtual addresses and machine frame numbers. */
+#define __virt_to_mfn(va)  mfn_x(maddr_to_mfn(virt_to_maddr(va)))
+#define __mfn_to_virt(mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn)))
+
+/* Convert between Xen-heap virtual addresses and page-info structures. */
+static inline struct page_info *virt_to_page(const void *v)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+/*
+ * We define non-underscored wrappers for above conversion functions.
+ * These are overriden in various source files while underscored version
+ * remain intact.
+ */
+#define virt_to_mfn(va)     __virt_to_mfn(va)
+#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+
+struct page_info
+{
+    /* Each frame can be threaded onto a doubly-linked list. */
+    struct page_list_entry list;
+
+    /* Reference count and various PGC_xxx flags and fields. */
+    unsigned long count_info;
+
+    /* Context-dependent fields follow... */
+    union {
+        /* Page is in use: ((count_info & PGC_count_mask) != 0). */
+        struct {
+            /* Type reference count and various PGT_xxx flags and fields. */
+            unsigned long type_info;
+        } inuse;
+        /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
+        union {
+            struct {
+                /*
+                 * Index of the first *possibly* unscrubbed page in the buddy.
+                 * One more bit than maximum possible order to accommodate
+                 * INVALID_DIRTY_IDX.
+                 */
+#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1)
+                unsigned long first_dirty:MAX_ORDER + 1;
+
+                /* Do TLBs need flushing for safety before next page use? */
+                bool need_tlbflush:1;
+
+#define BUDDY_NOT_SCRUBBING    0
+#define BUDDY_SCRUBBING        1
+#define BUDDY_SCRUB_ABORT      2
+                unsigned long scrub_state:2;
+            };
+
+                unsigned long val;
+        } free;
+    } u;
+
+    union {
+        /* Page is in use, but not as a shadow. */
+        struct {
+            /* Owner of this page (zero if page is anonymous). */
+            struct domain *domain;
+        } inuse;
+
+        /* Page is on a free list. */
+        struct {
+            /* Order-size of the free chunk this page is the head of. */
+            unsigned int order;
+        } free;
+    } v;
+
+    union {
+        /*
+         * Timestamp from 'TLB clock', used to avoid extra safety flushes.
+         * Only valid for: a) free pages, and b) pages with zero type count
+         */
+        uint32_t tlbflush_timestamp;
+    };
+};
+
+#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
+
+/* PDX of the first page in the frame table. */
+extern unsigned long frametable_base_pdx;
+
+/* Convert between machine frame numbers and page-info structures. */
+#define mfn_to_page(mfn)                                            \
+    (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
+#define page_to_mfn(pg)                                             \
+    pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)
+
+static inline void *page_to_virt(const struct page_info *pg)
+{
+    return mfn_to_virt(mfn_x(page_to_mfn(pg)));
+}
+
+/*
+ * Common code requires get_page_type and put_page_type.
+ * We don't care about typecounts so we just do the minimum to make it
+ * happy.
+ */
+static inline int get_page_type(struct page_info *page, unsigned long type)
+{
+    return 1;
+}
+
+static inline void put_page_type(struct page_info *page)
+{
+}
+
+static inline void put_page_and_type(struct page_info *page)
+{
+    put_page_type(page);
+    put_page(page);
+}
+
+/*
+ * RISC-V does not have an M2P, but common code expects a handful of
+ * M2P-related defines and functions. Provide dummy versions of these.
+ */
+#define INVALID_M2P_ENTRY        (~0UL)
+#define SHARED_M2P_ENTRY         (~0UL - 1UL)
+#define SHARED_M2P(_e)           ((_e) == SHARED_M2P_ENTRY)
+
+#define set_gpfn_from_mfn(mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0)
+#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
+
+#define PDX_GROUP_SHIFT (16 + 5)
+
+static inline unsigned long domain_get_maximum_gpfn(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+/*
+ * On RISCV, all the RAM is currently direct mapped in Xen.
+ * Hence return always true.
+ */
+static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
+{
+    return true;
+}
+
+#define PG_shift(idx)   (BITS_PER_LONG - (idx))
+#define PG_mask(x, idx) (x ## UL << PG_shift(idx))
+
+#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
+#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
+#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
+
+ /* Count of uses of this frame as its current type. */
+#define PGT_count_width   PG_shift(2)
+#define PGT_count_mask    ((1UL << PGT_count_width) - 1)
+
+/*
+ * Page needs to be scrubbed. Since this bit can only be set on a page that is
+ * free (i.e. in PGC_state_free) we can reuse PGC_allocated bit.
+ */
+#define _PGC_need_scrub   _PGC_allocated
+#define PGC_need_scrub    PGC_allocated
+
+/* Cleared when the owning guest 'frees' this page. */
+#define _PGC_allocated    PG_shift(1)
+#define PGC_allocated     PG_mask(1, 1)
+/* Page is Xen heap? */
+#define _PGC_xen_heap     PG_shift(2)
+#define PGC_xen_heap      PG_mask(1, 2)
+/* Page is broken? */
+#define _PGC_broken       PG_shift(7)
+#define PGC_broken        PG_mask(1, 7)
+/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */
+#define PGC_state         PG_mask(3, 9)
+#define PGC_state_inuse   PG_mask(0, 9)
+#define PGC_state_offlining PG_mask(1, 9)
+#define PGC_state_offlined PG_mask(2, 9)
+#define PGC_state_free    PG_mask(3, 9)
+#define page_state_is(pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st)
+
+/* Count of references to this frame. */
+#define PGC_count_width   PG_shift(9)
+#define PGC_count_mask    ((1UL << PGC_count_width) - 1)
+
+#define _PGC_extra        PG_shift(10)
+#define PGC_extra         PG_mask(1, 10)
+
+#define is_xen_heap_page(page) ((page)->count_info & PGC_xen_heap)
+#define is_xen_heap_mfn(mfn) \
+    (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn)))
+
+#define is_xen_fixed_mfn(mfn)                                   \
+    ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) &&   \
+     (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1)))
+
+#define page_get_owner(_p)    (_p)->v.inuse.domain
+#define page_set_owner(_p,_d) ((_p)->v.inuse.domain = (_d))
+
+/* TODO: implement */
+#define mfn_valid(mfn) ({ (void)(mfn); 0; })
+
+#define domain_set_alloc_bitsize(d) ((void)(d))
+#define domain_clamp_alloc_bitsize(d, b) ((void)(d), (b))
+
+#define PFN_ORDER(pfn) ((pfn)->v.free.order)
+
 extern unsigned char cpu0_boot_stack[];
 
 void setup_initial_pagetables(void);
@@ -20,4 +261,9 @@ unsigned long calc_phys_offset(void);
 
 void turn_on_mmu(unsigned long ra);
 
+static inline unsigned int arch_get_dma_bitsize(void)
+{
+    return 32; /* TODO */
+}
+
 #endif /* _ASM_RISCV_MM_H */
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index 053f043a3d..fe3a43be20 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -5,12 +5,12 @@
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/macros.h>
+#include <xen/mm.h>
 #include <xen/pfn.h>
 
 #include <asm/early_printk.h>
 #include <asm/csr.h>
 #include <asm/current.h>
-#include <asm/mm.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 6593f601c1..98a94c4c48 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -2,9 +2,9 @@
 
 #include <xen/compile.h>
 #include <xen/init.h>
+#include <xen/mm.h>
 
 #include <asm/early_printk.h>
-#include <asm/mm.h>
 
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:41:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:41:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676289.1052419 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX15s-0006lH-3B; Mon, 05 Feb 2024 15:41:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676289.1052419; Mon, 05 Feb 2024 15:41:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX15r-0006lA-VW; Mon, 05 Feb 2024 15:41:11 +0000
Received: by outflank-mailman (input) for mailman id 676289;
 Mon, 05 Feb 2024 15:41:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0y2-0007I6-S0
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:06 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id db09ce0c-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:33:05 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-51032058f17so5125718e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:05 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: db09ce0c-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147184; x=1707751984; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=1EpDP5Niihr1G1U8oddvxlYPXrLXztpUK4y9JjJucU0=;
        b=AN8Ihp99dtBw5I6JNYW9tD1VeVnWJhpSqUdhQusmlUkYQMXzRPizWRsdOzVWCfkFPY
         sVbR7kkOYc4KcDoH4o4EPF9HwLIansoShymgj5nZvpG1zVApQnWj4khyH7dUbmSYwjPb
         j4VYEfPTezmLtKBNC1HJluyUoE/507+riVMUNSwwaoEmE4uBE77gJ+RkNfwTclTBTGXf
         GRcRKVezLDBxVuImBadl25HCJFckJuMnOw+vsmlKMEFnpJL/onR56IQ+gfTeesONC+zK
         VEkB4Q0FAgviuLuqwmCO+Wd6sYISoE+kZWxpH6MoG629KV4n/j9EoAqo74faaEzfz9+a
         oBBQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147184; x=1707751984;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=1EpDP5Niihr1G1U8oddvxlYPXrLXztpUK4y9JjJucU0=;
        b=A37pG9JcXNa8pSVWzHx0l+L7GmtvgAqYbhTC4RMH1vdIRNipyOooslq+uSAe261spc
         mJKQ1W4xGOKczMJlOqBdB3px9xuFfMJN5uzYZCXIp/mHgpAmGwQVwdF52dX5SL11Hojq
         /pTyktBn8KUmo1jJbQ2rQuNJhl46mVvidWIqakSMkUZeZ8ynMdIXpzJh/8uaLPltI2oS
         LzvS6HVTQfsLrnN3ZNXmxEPhGp0TOarv20V8vLmTKVetP6ESuzF4dd3v4ELBwCiN/zfP
         2/Lccjx160WAAFbk99H0QY1t2zWzr+DGI1VfG/BxOp6J+l1TiKqymz2yZiPWsp4RUBOu
         bVRQ==
X-Gm-Message-State: AOJu0YwzOP4X9DWWEKXAM8//wJvdlMvsWqOXN6i6ix8Rkes1+BDPxd4p
	4mdlbVb8RsllOqElZbA+IvZxDz+VDA80ijFvBWkhH5HJdtuRZMUHQ27EP3sf
X-Google-Smtp-Source: AGHT+IERUkJXhuMp6A3TCMQGfl0J9cJVgHeyC6Y/rwpGu/DjlIBSpgILDv+ywuxhQjJOp3Cp0Ta7HA==
X-Received: by 2002:ac2:4851:0:b0:511:530c:d0eb with SMTP id 17-20020ac24851000000b00511530cd0ebmr1483769lfy.46.1707147184598;
        Mon, 05 Feb 2024 07:33:04 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWKXyZj/KBMIs89OsXKMtwVDES+1i99zwaSRsx7gSuulzLmPt4qUpMFIoHLFRwPQ98NF+gS/TW5nF67SeRbE1DrwZLh+Pw2XEpWG91ddhub8ZOaf76QcvtzX6Urd7N7015H1+DPLqzCByJmuugNYz5/RUgcRB/zZF2Hcg1/FPzBizWrBn6+imbYIYR/lQ7mo/OWB+LUT8cUAscQBJbC9MMpBS9zMG4be2AqIF057WKoq8R+Tg3ofXQlyzDrsIJxbxtUP4qOl46izRKjw2r/U4LsFXxjSNumXYjZtJQUsmupi6+yqBC2e8WFH2RIrNXPty4qM3Ay
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 23/30] xen/riscv: add required things to current.h
Date: Mon,  5 Feb 2024 16:32:30 +0100
Message-ID: <acf0cfbd19dcc05a37ad768d4902a7a6274d64f1.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add minimal requied things to be able to build full Xen.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V4:
 - BUG() was changed to BUG_ON("unimplemented");
 - Change "xen/bug.h" to "xen/lib.h" as BUG_ON is defined in xen/lib.h.
 - Add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V3:
 - add SPDX
 - drop a forward declaration of struct vcpu;
 - update guest_cpu_user_regs() macros
 - replace get_processor_id with smp_processor_id
 - update the commit message
 - code style fixes
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/current.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/xen/arch/riscv/include/asm/current.h b/xen/arch/riscv/include/asm/current.h
index d84f15dc50..aedb6dc732 100644
--- a/xen/arch/riscv/include/asm/current.h
+++ b/xen/arch/riscv/include/asm/current.h
@@ -3,6 +3,21 @@
 #ifndef __ASM_CURRENT_H
 #define __ASM_CURRENT_H
 
+#include <xen/lib.h>
+#include <xen/percpu.h>
+#include <asm/processor.h>
+
+#ifndef __ASSEMBLY__
+
+/* Which VCPU is "current" on this PCPU. */
+DECLARE_PER_CPU(struct vcpu *, curr_vcpu);
+
+#define current            this_cpu(curr_vcpu)
+#define set_current(vcpu)  do { current = (vcpu); } while (0)
+#define get_cpu_current(cpu)  per_cpu(curr_vcpu, cpu)
+
+#define guest_cpu_user_regs() ({ BUG_ON("unimplemented"); NULL; })
+
 #define switch_stack_and_jump(stack, fn) do {               \
     asm volatile (                                          \
             "mv sp, %0\n"                                   \
@@ -10,4 +25,8 @@
     unreachable();                                          \
 } while ( false )
 
+#define get_per_cpu_offset() __per_cpu_offset[smp_processor_id()]
+
+#endif /* __ASSEMBLY__ */
+
 #endif /* __ASM_CURRENT_H */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:42:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:42:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676312.1052429 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX17H-0007ai-Bl; Mon, 05 Feb 2024 15:42:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676312.1052429; Mon, 05 Feb 2024 15:42:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX17H-0007ab-96; Mon, 05 Feb 2024 15:42:39 +0000
Received: by outflank-mailman (input) for mailman id 676312;
 Mon, 05 Feb 2024 15:42:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0y2-0007Hv-45
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:06 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id da8776ec-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:33:04 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-5111e5e4e2bso7067914e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:04 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da8776ec-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147184; x=1707751984; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=cOrP2+p0UdKM7OlT4vC28W6y4WuqTbyr5CWNeFF+F4o=;
        b=YGJW7di8pScrI6ltCOLJMsfXysoG9482jYPNjyBzQC6DGqQdzfw4cfTdxBnzPH+ZUx
         dMIgqEKPMUR9AlX9xN5RzFFthTympLX8KfZvHabgJ+FjDzEkB0xQG/VksCz/WOZnayac
         PxDLO/KkHfCamz/WMeUTuwv3YGtoJ/CwnUv92XH81CgmmJvrOKExMIxRkQUAVnIbKAKs
         6W8hVmjp+UeOxjRT+HP+G0hrFyG/CNELAThHb2rFsGNC/3M6jE+wH1Rdh20Fni0GwsVj
         GtUoynlKL1ZGDbqzW0CD+GI3DE9VjYjdybi7XqBd+ZCcDUyFGSxy6DN7w6uo231OztK/
         q5FQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147184; x=1707751984;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=cOrP2+p0UdKM7OlT4vC28W6y4WuqTbyr5CWNeFF+F4o=;
        b=A3Wya7rhy4/70rPJA4n9/94QctAY3jwmlJpYGDUpZ4l6JrJiZdx4VyYXjb2JJb8QNW
         CEMuNQTg5kMpbcGWZjMUlPy+wehRO93VmhkQlcfgmQFVnApZA1BTtOILc8VEopQ16KPW
         YpQkbqHf9bYHbfmKTMkLr/YSMmac+fWjaC6O59eDFjdUsQo2+PUKpS9xwfGdeLHjPu3d
         DwDMV7vinx7AoG32eeu9+Xr3BIzZgmIAXX5WjfNDfMpmRXdrl1T7+edd4IaQ3hiidYAT
         uEMPemiVFZvFdH0MCMDAMFE3qChr6zdMXjeEtgWct7dsGE1mauoSkj43/5spWQC1oT/g
         tmiQ==
X-Gm-Message-State: AOJu0YyWW7ZiL6Qm5llLcdSpHHy1dYg4cHgVYAiSxPugA/nCsSfhDOAE
	5jKM/C5Wx3H1fFkDecCe6tqlAQ1WOn8q7oX9S3wmNn0yz1Dt+n7GIPJny6FX
X-Google-Smtp-Source: AGHT+IGbDRXclnbreTILTKBBtioAUKxt0a+2i7VfOI26G3E8gmpVxDgb54h4P+cwwIgyl12KSrt5vA==
X-Received: by 2002:a05:6512:448:b0:511:354a:ca45 with SMTP id y8-20020a056512044800b00511354aca45mr6044525lfk.31.1707147183768;
        Mon, 05 Feb 2024 07:33:03 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVO+jxkUlEzqUVVxisF9a+E+MCNu0KdKQsRbW2xTq2iHqcLiOgYUCCypbcXZs8CUMxL/Z2zmvOddU5IeJjljyit5tMdTo351eD1LXUEBl3y2BOlEZ5FRZ3S+BQSfqIqxIDvBiIo80poTfqJMoISnVILUevEIWYV58IcEXPtMvxANWlNxsi9uwfceWKAIJ8a7FqIZVyhlEfeM2/n32FfyhkVW7P8+Sc7SiYhr0ZEMhuA1K5AVcJOeLEWaptYyY1k7kILgEody9wv2SArxun47TMbHDt8FIpW5s0ROgb9dLYnC4VNv9M+mQ2pKnAsIpqMVLBhFhq4
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 22/30] xen/riscv: define an address of frame table
Date: Mon,  5 Feb 2024 16:32:29 +0100
Message-ID: <796853f19b883c474998ba97c1ee66cf224b1bde.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Also, the patch adds some helpful macros that assist in avoiding
the redefinition of memory layout for each MMU mode.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
 - move "#define VPN_BITS (9)" inside CONFIG_RISCV_64 as for SV32 it should be defined differently.
 - drop SLOTN_ENTRY_SIZE and introduce DIRECTMAP_SIZE. It is not needed for now, but will be needed in the
   future.
 - update memory layout table and some related macros.
---
Changes in V3:
 - drop OFFSET_BITS, and use PAGE_SHIFT instead.
 - code style fixes.
 - add comment how macros are useful.
 - move all memory related layout definitions close to comment with memory layout description.
 - make memory layout description generic for any MMU mode.
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/config.h | 105 ++++++++++++++++++++--------
 1 file changed, 77 insertions(+), 28 deletions(-)

diff --git a/xen/arch/riscv/include/asm/config.h b/xen/arch/riscv/include/asm/config.h
index 56387ac159..479da15782 100644
--- a/xen/arch/riscv/include/asm/config.h
+++ b/xen/arch/riscv/include/asm/config.h
@@ -6,6 +6,16 @@
 #include <xen/const.h>
 #include <xen/page-size.h>
 
+#include <asm/riscv_encoding.h>
+
+#ifdef CONFIG_RISCV_64
+#define CONFIG_PAGING_LEVELS 3
+#define RV_STAGE1_MODE SATP_MODE_SV39
+#else
+#define CONFIG_PAGING_LEVELS 2
+#define RV_STAGE1_MODE SATP_MODE_SV32
+#endif
+
 /*
  * RISC-V64 Layout:
  *
@@ -23,25 +33,78 @@
  * It means that:
  *   top VA bits are simply ignored for the purpose of translating to PA.
  *
+ * Amount of slots for Frametable were calculated base on
+ * sizeof(struct page_info) = 48. If the 'struct page_info' is changed,
+ * the table below must be updated.
+ *
  * ============================================================================
- *    Start addr    |   End addr        |  Size  | Slot       |area description
- * ============================================================================
- * FFFFFFFFC0800000 |  FFFFFFFFFFFFFFFF |1016 MB | L2 511     | Unused
- * FFFFFFFFC0600000 |  FFFFFFFFC0800000 |  2 MB  | L2 511     | Fixmap
- * FFFFFFFFC0200000 |  FFFFFFFFC0600000 |  4 MB  | L2 511     | FDT
- * FFFFFFFFC0000000 |  FFFFFFFFC0200000 |  2 MB  | L2 511     | Xen
- *                 ...                  |  1 GB  | L2 510     | Unused
- * 0000003200000000 |  0000007F80000000 | 309 GB | L2 200-509 | Direct map
- *                 ...                  |  1 GB  | L2 199     | Unused
- * 0000003100000000 |  00000031C0000000 |  3 GB  | L2 196-198 | Frametable
- *                 ...                  |  1 GB  | L2 195     | Unused
- * 0000003080000000 |  00000030C0000000 |  1 GB  | L2 194     | VMAP
- *                 ...                  | 194 GB | L2 0 - 193 | Unused
+ * Start addr          | End addr         | Slot       | area description
  * ============================================================================
- *
+ *                   .....                 L2 511          Unused
+ *  0xffffffffc0600000  0xffffffffc0800000 L2 511          Fixmap
+ *  0xffffffffc0200000  0xffffffffc0600000 L2 511          FDT
+ *  0xffffffffc0000000  0xffffffffc0200000 L2 511          Xen
+ *                   .....                 L2 510          Unused
+ *  0x3200000000        0x7f40000000       L2 200-509      Direct map
+ *                   .....                 L2 199          Unused
+ *  0x30c0000000        0x31c0000000       L2 195-198      Frametable
+ *                   .....                 L2 194          Unused
+ *  0x3040000000        0x3080000000       L2 193          VMAP
+ *                   .....                 L2 0-192        Unused
+#elif RV_STAGE1_MODE == SATP_MODE_SV48
+ * Memory layout is the same as for SV39 in terms of slots, so only start and
+ * end addresses should be shifted by 9
 #endif
  */
 
+#define HYP_PT_ROOT_LEVEL (CONFIG_PAGING_LEVELS - 1)
+
+#ifdef CONFIG_RISCV_64
+
+#define VPN_BITS (9)
+
+#define SLOTN_ENTRY_BITS        (HYP_PT_ROOT_LEVEL * VPN_BITS + PAGE_SHIFT)
+#define SLOTN(slot)             (_AT(vaddr_t, slot) << SLOTN_ENTRY_BITS)
+
+#if RV_STAGE1_MODE == SATP_MODE_SV39
+#define XEN_VIRT_START 0xFFFFFFFFC0000000
+#elif RV_STAGE1_MODE == SATP_MODE_SV48
+#define XEN_VIRT_START 0xFFFFFF8000000000
+#else
+#error "unsupported RV_STAGE1_MODE"
+#endif
+
+#define DIRECTMAP_SLOT_END      509
+#define DIRECTMAP_SLOT_START    200
+#define DIRECTMAP_VIRT_START    SLOTN(DIRECTMAP_SLOT_START)
+#define DIRECTMAP_SIZE          (SLOTN(DIRECTMAP_SLOT_END) - SLOTN(DIRECTMAP_SLOT_START))
+
+#define FRAMETABLE_SCALE_FACTOR  (PAGE_SIZE/sizeof(struct page_info))
+#define FRAMETABLE_SIZE_IN_SLOTS (((DIRECTMAP_SIZE / SLOTN(1)) / FRAMETABLE_SCALE_FACTOR) + 1)
+
+/*
+ * We have to skip Unused slot between DIRECTMAP and FRAMETABLE (look at mem.
+ * layout), so -1 is needed
+ */
+#define FRAMETABLE_SLOT_START   (DIRECTMAP_SLOT_START - FRAMETABLE_SIZE_IN_SLOTS - 1)
+#define FRAMETABLE_SIZE         (FRAMETABLE_SIZE_IN_SLOTS * SLOTN(1))
+#define FRAMETABLE_VIRT_START   SLOTN(FRAMETABLE_SLOT_START)
+#define FRAMETABLE_NR           (FRAMETABLE_SIZE / sizeof(*frame_table))
+#define FRAMETABLE_VIRT_END     (FRAMETABLE_VIRT_START + FRAMETABLE_SIZE - 1)
+
+/*
+ * We have to skip Unused slot between Frametable and VMAP (look at mem.
+ * layout), so an additional -1 is needed */
+#define VMAP_SLOT_START         (FRAMETABLE_SLOT_START - 1 - 1)
+#define VMAP_VIRT_START         SLOTN(VMAP_SLOT_START)
+#define VMAP_VIRT_SIZE          GB(1)
+
+#else
+#error "RV32 isn't supported"
+#endif
+
+#define HYPERVISOR_VIRT_START XEN_VIRT_START
+
 #if defined(CONFIG_RISCV_64)
 # define LONG_BYTEORDER 3
 # define ELFSIZE 64
@@ -75,24 +138,10 @@
 #define CODE_FILL /* empty */
 #endif
 
-#ifdef CONFIG_RISCV_64
-#define XEN_VIRT_START 0xFFFFFFFFC0000000 /* (_AC(-1, UL) + 1 - GB(1)) */
-#else
-#error "RV32 isn't supported"
-#endif
-
 #define SMP_CACHE_BYTES (1 << 6)
 
 #define STACK_SIZE PAGE_SIZE
 
-#ifdef CONFIG_RISCV_64
-#define CONFIG_PAGING_LEVELS 3
-#define RV_STAGE1_MODE SATP_MODE_SV39
-#else
-#define CONFIG_PAGING_LEVELS 2
-#define RV_STAGE1_MODE SATP_MODE_SV32
-#endif
-
 #define IDENT_AREA_SIZE 64
 
 #endif /* __RISCV_CONFIG_H__ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:46:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:46:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676323.1052438 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1AV-0000JP-QJ; Mon, 05 Feb 2024 15:45:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676323.1052438; Mon, 05 Feb 2024 15:45:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1AV-0000JI-NH; Mon, 05 Feb 2024 15:45:59 +0000
Received: by outflank-mailman (input) for mailman id 676323;
 Mon, 05 Feb 2024 15:45:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0y6-0007Hv-E1
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:10 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ddca2410-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:33:09 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-511490772f6so1898306e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:09 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ddca2410-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147189; x=1707751989; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=pr1JIkqZUw59vhziDxsEOtOcbHipI28A4qI7Th4nh3I=;
        b=ebOoL/yHt91EqBggIqPED2pcz6NPC6I0H2eMbiGMkGaUIqr5AMu9rLat+EB3uo+2T/
         iSPXO3rufyhTjkhrRvlqucCG46SE2G3qnhkO3XrBnxq7447EEc60uchglMy5RueB8ZuZ
         z0cf9f642hCplgR3qu3AfTXe4dVqQFRGOEOHGRjVksal9PBLj7rKffxtng1iyjmjJtFx
         RBFb4hni+aMHjmn2fSgnsrV8yalX5FsvyUZ/J7ArBiqn+C7jqlPwS3H9KKAE9JDawv3b
         hftKusZwBjS510dtIZPjUXxwqGufXhRhLVq65BBOA7iKMdwWaunVPjT/SW6jN4hPupwf
         Sm8w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147189; x=1707751989;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=pr1JIkqZUw59vhziDxsEOtOcbHipI28A4qI7Th4nh3I=;
        b=js0yVxnwGi6FhWSgmaGGcoB2GTsR4eji1HR0it0Vw1DYuhKNGjYHtcTmKd176KgDbm
         Ii135pykExjETatPTI/KJ2Up6Iv9Ey+NA3RYFNJ9uEiZDpOgSvV2t4iwDhGdzjWmAbAP
         fRM6HByD99SR2vfvs386vUjOjecuxUuwpj+m/y0Z7MFVrBbxPsdFyh/jqi8snZJtQGiA
         xwFeaBGrCR3uJFz3vgpoZuihRoDw4Uf/BOCRglRqpH+aEx7UXDWdukX4EvG4xwvDEs7k
         Bgm1AdwuOXSzTlt9cPdou9miRMeyBsKblMJN+VdCl1nAVf/YgJesB+Y53Aq1muw9nx8Y
         uDGw==
X-Gm-Message-State: AOJu0Yw4WdHDZ/tRYEAgv1zUnFivBe5E6BcdFEuHTP8x4SF+n6td2Ai6
	EpohZIU56BloZTyAfeqHUP/JOFY8F79trS7mnqH1uvB/DYQDhCV6bWXReVAJ
X-Google-Smtp-Source: AGHT+IGwJaRnIl/URwdxyGrPY2aME7vWG9iYCEASGJw/ZYqBRvRAZwQ5rWk5qWkpdt+and2V7g4MqQ==
X-Received: by 2002:ac2:4a66:0:b0:511:448c:3225 with SMTP id q6-20020ac24a66000000b00511448c3225mr3916512lfp.68.1707147188770;
        Mon, 05 Feb 2024 07:33:08 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWGLfC38zmbTLqFoizgpOLmX9xgciuDWE1OExVLIcIFtkeRRWddEA80x4w1bcD2gLdeJ4coDuHNAxUzMQpVfp0YswBKs65ruHtXYXnqXTNH/JkCyvaXg9bbLNVxfQzWa2sJol/PaTJU1auIvrbbR2FIcjJUSEHKSGs7PtCDKrDW8ylwJN34zANmAxBBVcCmgLRifT/hU+1V7b4pVHH/2qI0T3pXtMoMAVkDM+tu5qhqRSxmckrpOX/pQ0ZhCAuVQl+ZbA==
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH v4 27/30] xen/riscv: introduce vm_event_*() functions
Date: Mon,  5 Feb 2024 16:32:34 +0100
Message-ID: <acc19ced590be534606d0923d929d3c15f887530.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
  - New patch.
---
 xen/arch/riscv/Makefile   |  1 +
 xen/arch/riscv/vm_event.c | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+)
 create mode 100644 xen/arch/riscv/vm_event.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 2fefe14e7c..1ed1a8369b 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
 obj-y += traps.o
+obj-y += vm_event.o
 
 $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
diff --git a/xen/arch/riscv/vm_event.c b/xen/arch/riscv/vm_event.c
new file mode 100644
index 0000000000..bb1fc73bc1
--- /dev/null
+++ b/xen/arch/riscv/vm_event.c
@@ -0,0 +1,19 @@
+#include <xen/bug.h>
+
+struct vm_event_st;
+struct vcpu;
+
+void vm_event_fill_regs(struct vm_event_st *req)
+{
+    BUG_ON("unimplemented");
+}
+
+void vm_event_set_registers(struct vcpu *v, struct vm_event_st *rsp)
+{
+    BUG_ON("unimplemented");
+}
+
+void vm_event_monitor_next_interrupt(struct vcpu *v)
+{
+    /* Not supported on RISCV. */
+}
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:47:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:47:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676329.1052449 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1BV-00014V-3L; Mon, 05 Feb 2024 15:47:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676329.1052449; Mon, 05 Feb 2024 15:47:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1BV-00014O-0T; Mon, 05 Feb 2024 15:47:01 +0000
Received: by outflank-mailman (input) for mailman id 676329;
 Mon, 05 Feb 2024 15:46:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0yA-0007I6-7Y
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:14 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id df504564-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:33:12 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-511234430a4so7878640e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:12 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df504564-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147192; x=1707751992; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=KLI6lq0SpNz5bYmO78wLTxQzJ8VUm+LfbX77KEtwseM=;
        b=HxrMOziOIG+6vcHhtWubY3qojyZZNml1R8svnnRFQgb/sKWu6RvIAZFYWG6qFBMB0K
         +atOewezyNbR8dLbiftU02MOkVcdXVHtMJkGKDFVcjd+9EJ8UCcuOvg5Wt6ppsNyfxsc
         UbBDmodW/cZRfa4SIVkDdUq99aYZiXM4yFZfq/T6ZFBus2ICPE1PTaMqTr/nzSSAW8qc
         /NoqZ8WzK1ZOXJJRtDvUy8CntuIBAS24H629ZX7QiK9aSWPSTDus1UIinYLAE4W7bVa/
         TqP8gY7ajTNcDbM9X46HNITDwwP1gj8VUuKXFDTLAvbRjd/EZqJX9S+UKvpt8NaJ7IMD
         fR9g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147192; x=1707751992;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=KLI6lq0SpNz5bYmO78wLTxQzJ8VUm+LfbX77KEtwseM=;
        b=k2ePvIpZ18uDLUGG7X7VHCwkIWljQ5MWxSyBTqFnHttMaH5JMFib+PLouLvEwbtgqo
         qG1ajF3cOR//ZTyP15Je2FYA8Ryv6TiZ5/70NBSU7zNXzkceItGEcbi9seUEsalmkQl1
         r4nNFtdu/JqkyQhysJrO5otdLJY9nNa1eUISoOQuL4YBSuvtjM8z41QasmD4BwyNykrL
         vvqvb+hgghGyEph0pNS9aAKQ7Fu4NK8sgTdw3S6+X+7eySMd6U50LSjBi87u4ymwaTtO
         slmzncNqm3cbgaqWbDTszHgb94bct4sesB12H+NjMXON0f10Sa6Aiql7tHyr+biNaQ35
         JYkg==
X-Gm-Message-State: AOJu0YyvaVosvlX1kOgnjMA4l6wN9lJ/nzOGUjANF4zi5dFAnVFSTcFN
	JtWlL/rvhFtiCJb0Yz7dL/4+7Bcwj4nMES7C7ZDSim4s8z+OAa49MxV46bms
X-Google-Smtp-Source: AGHT+IFe+rCHqFGFufnLMYqzBmwcYvZ3as2LZcsD4SBHiavybweYKVNV0Gd4IQd37KKs7FtMGKI18w==
X-Received: by 2002:a05:6512:31d6:b0:511:4e8c:7d02 with SMTP id j22-20020a05651231d600b005114e8c7d02mr3100104lfe.48.1707147191667;
        Mon, 05 Feb 2024 07:33:11 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCV44fS1p5IRnq3YdTpz528fwm22m9il8nSvpX86/u1ORmJapFDJa3Ei6vxJ3G8nyoElin9ketatL7ORP/k1bUl85YvNV6AHRxFVMBRpxBYiXDgB11mEmoTXFozZ/o7SQQQvAMN2bzeV7yEE9qjkhjgNEesw8HiOiBcLTnXJEDkLYEzXMKQyFhnIpPYGsKU3uZtjPe2rJ4C4MR1nJ9lxsr5v58zxRt+ZlVDTFp9BsVe5YmhGAgjEglgZ7cWvzPGPZEbkryf+BfTXEXUo2TkUksUPSSfUbz+jfSl4PtwrkJyzU+huOX/FO1nTVk27NiZadrkUHoAI
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 29/30] xen/riscv: enable full Xen build
Date: Mon,  5 Feb 2024 16:32:36 +0100
Message-ID: <153f9404c8afdfa4b3f91cdfabe1ffaf6e5c851b.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V4:
 - drop stubs for irq_actor_none() and irq_actor_none() as common/irq.c is compiled now.
 - drop defintion of max_page in stubs.c as common/page_alloc.c is compiled now.
 - drop printk() related changes in riscv/early_printk.c as common version will be used.
---
Changes in V3:
 - Reviewed-by: Jan Beulich <jbeulich@suse.com>
 - unrealted change dropped in tiny64_defconfig
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/Makefile       |  16 +++-
 xen/arch/riscv/arch.mk        |   4 -
 xen/arch/riscv/early_printk.c | 167 ----------------------------------
 xen/arch/riscv/stubs.c        |  23 -----
 4 files changed, 15 insertions(+), 195 deletions(-)

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 60afbc0ad9..81b77b13d6 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -12,10 +12,24 @@ $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
 
 $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
-	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(objtree)/common/symbols-dummy.o -o $(dot-target).0
+	$(NM) -pa --format=sysv $(dot-target).0 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).0.S
+	$(MAKE) $(build)=$(@D) $(dot-target).0.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(dot-target).0.o -o $(dot-target).1
+	$(NM) -pa --format=sysv $(dot-target).1 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).1.S
+	$(MAKE) $(build)=$(@D) $(dot-target).1.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
+	    $(dot-target).1.o -o $@
 	$(NM) -pa --format=sysv $@ \
 		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		> $@.map
+	rm -f $(@D)/.$(@F).[0-9]*
 
 $(obj)/xen.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed_dep,cpp_lds_S)
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index a4b53adaf7..4363776f34 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -14,7 +14,3 @@ riscv-march-$(CONFIG_TOOLCHAIN_HAS_ZIHINTPAUSE) := $(riscv-march-y)_zihintpause
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += -march=$(riscv-march-y) -mstrict-align -mcmodel=medany
-
-# TODO: Drop override when more of the build is working
-override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
-override ALL_LIBS-y =
diff --git a/xen/arch/riscv/early_printk.c b/xen/arch/riscv/early_printk.c
index 6d0911659d..610c814f54 100644
--- a/xen/arch/riscv/early_printk.c
+++ b/xen/arch/riscv/early_printk.c
@@ -40,170 +40,3 @@ void early_printk(const char *str)
         str++;
     }
 }
-
-/*
- * The following #if 1 ... #endif should be removed after printk
- * and related stuff are ready.
- */
-#if 1
-
-#include <xen/stdarg.h>
-#include <xen/string.h>
-
-/**
- * strlen - Find the length of a string
- * @s: The string to be sized
- */
-size_t (strlen)(const char * s)
-{
-    const char *sc;
-
-    for (sc = s; *sc != '\0'; ++sc)
-        /* nothing */;
-    return sc - s;
-}
-
-/**
- * memcpy - Copy one area of memory to another
- * @dest: Where to copy to
- * @src: Where to copy from
- * @count: The size of the area.
- *
- * You should not use this function to access IO space, use memcpy_toio()
- * or memcpy_fromio() instead.
- */
-void *(memcpy)(void *dest, const void *src, size_t count)
-{
-    char *tmp = (char *) dest, *s = (char *) src;
-
-    while (count--)
-        *tmp++ = *s++;
-
-    return dest;
-}
-
-int vsnprintf(char* str, size_t size, const char* format, va_list args)
-{
-    size_t i = 0; /* Current position in the output string */
-    size_t written = 0; /* Total number of characters written */
-    char* dest = str;
-
-    while ( format[i] != '\0' && written < size - 1 )
-    {
-        if ( format[i] == '%' )
-        {
-            i++;
-
-            if ( format[i] == '\0' )
-                break;
-
-            if ( format[i] == '%' )
-            {
-                if ( written < size - 1 )
-                {
-                    dest[written] = '%';
-                    written++;
-                }
-                i++;
-                continue;
-            }
-
-            /*
-             * Handle format specifiers.
-             * For simplicity, only %s and %d are implemented here.
-             */
-
-            if ( format[i] == 's' )
-            {
-                char* arg = va_arg(args, char*);
-                size_t arglen = strlen(arg);
-
-                size_t remaining = size - written - 1;
-
-                if ( arglen > remaining )
-                    arglen = remaining;
-
-                memcpy(dest + written, arg, arglen);
-
-                written += arglen;
-                i++;
-            }
-            else if ( format[i] == 'd' )
-            {
-                int arg = va_arg(args, int);
-
-                /* Convert the integer to string representation */
-                char numstr[32]; /* Assumes a maximum of 32 digits */
-                int numlen = 0;
-                int num = arg;
-                size_t remaining;
-
-                if ( arg < 0 )
-                {
-                    if ( written < size - 1 )
-                    {
-                        dest[written] = '-';
-                        written++;
-                    }
-
-                    num = -arg;
-                }
-
-                do
-                {
-                    numstr[numlen] = '0' + num % 10;
-                    num = num / 10;
-                    numlen++;
-                } while ( num > 0 );
-
-                /* Reverse the string */
-                for (int j = 0; j < numlen / 2; j++)
-                {
-                    char tmp = numstr[j];
-                    numstr[j] = numstr[numlen - 1 - j];
-                    numstr[numlen - 1 - j] = tmp;
-                }
-
-                remaining = size - written - 1;
-
-                if ( numlen > remaining )
-                    numlen = remaining;
-
-                memcpy(dest + written, numstr, numlen);
-
-                written += numlen;
-                i++;
-            }
-        }
-        else
-        {
-            if ( written < size - 1 )
-            {
-                dest[written] = format[i];
-                written++;
-            }
-            i++;
-        }
-    }
-
-    if ( size > 0 )
-        dest[written] = '\0';
-
-    return written;
-}
-
-void printk(const char *format, ...)
-{
-    static char buf[1024];
-
-    va_list args;
-    va_start(args, format);
-
-    (void)vsnprintf(buf, sizeof(buf), format, args);
-
-    early_printk(buf);
-
-    va_end(args);
-}
-
-#endif
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
index 529f1dbe52..bda35fc347 100644
--- a/xen/arch/riscv/stubs.c
+++ b/xen/arch/riscv/stubs.c
@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
 
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
 
-/*
- * max_page is defined in page_alloc.c which isn't complied for now.
- * definition of max_page will be remove as soon as page_alloc is built.
- */
-unsigned long __read_mostly max_page;
-
 /* time.c */
 
 unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
@@ -419,20 +413,3 @@ void __cpu_die(unsigned int cpu)
 {
     BUG_ON("unimplemented");
 }
-
-/*
- * The following functions are defined in common/irq.c, which will be built in
- * the next commit, so these changes will be removed there.
- */
-
-void cf_check irq_actor_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-}
-
-unsigned int cf_check irq_startup_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-
-    return 0;
-}
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:47:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:47:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676334.1052459 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1CJ-0001vt-FQ; Mon, 05 Feb 2024 15:47:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676334.1052459; Mon, 05 Feb 2024 15:47:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1CJ-0001vm-CG; Mon, 05 Feb 2024 15:47:51 +0000
Received: by outflank-mailman (input) for mailman id 676334;
 Mon, 05 Feb 2024 15:47:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=8/S5=JO=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rX1CI-0001Vi-3x
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:47:50 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e93dd040-c43d-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:47:48 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-51109060d6aso6438945e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:47:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f6-20020a5d4dc6000000b0033b40a3f92asm1462046wru.25.2024.02.05.07.47.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 07:47:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e93dd040-c43d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707148067; x=1707752867; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1rVO9e5D92rCqHAjFHdqkTCys0FSsMEaNoKUX7kZn2k=;
        b=LwLmniRqIKnWnOVa9NutNwBXSBuhgQrVRElrUs6HR32tGcgs1sU14pbNpmgY0o+uUq
         7hnJmpIzbbKsJN7Nyczqwc2cAXsgWUwsNXgtmpHtUXn9OaHXmnynRokbS/VUK4xM9OuE
         rMKKhMYX2selPTQ7es8dmtWDLqkM26ijHsiutCRVtmc0RD93qCte9EDDF/DWnwrxDj+M
         4+czffAwXOPhOz8yfU0eThvbeTWb2pKp/q3r3lFpoTSC1Ailqc0POgjgY0P05277zUjC
         9ufMURQ1mZ/UyqqAiElAQL98/0B/icedN9koJBROKvRCR6J7NvDP78RWXwiDYlw0AsKh
         DbXg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707148067; x=1707752867;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1rVO9e5D92rCqHAjFHdqkTCys0FSsMEaNoKUX7kZn2k=;
        b=O4jjsWB1BEvSppG+PvZKmsKkRQHmrS/LRXfs4RApepNAfRklLwVydEq6Ofw+7lHZIy
         NkjkEe+Cy3Nemsv9yT9LRb3yKGHEwzS0nN2DOajtcvQdFzrlpyXVh2kU1cZxG949M02g
         /5FkgMsTBTOyUjz7KzhhUK0fKlqRIBa7AI2Fc95zPbuzjFtQpxOlAWd7et+JfIDgbK86
         qzMJ/egSITCXrOjna1cEA2XopveBABtMSkeqr2NAOhWp4RXrcyZzseuCuxMJOBOeYOPm
         NH/GzbeaJoZV4t4RCZsJP9ZC2kGDYvgJYch1H7lXTPhKqrGy2QCCaAQdiIxLZqgPKMeG
         1+lA==
X-Gm-Message-State: AOJu0Yy3V6htlfTnc15FaZOx1+3RGqsgoM+VjgQN28UwmKbRSOmEHxsj
	7pA726exXEZ3XlcnXQeDUHol7xa/N8gQbyChR+wUk2SO8KgkHI7JXGWPmA8tRg==
X-Google-Smtp-Source: AGHT+IG5VSNUfcRlkbw8HtS1/77aKseWakb/Vnp4BYvtYMCA7GYXGnB50RKYetfposQ5u3WCcscLEg==
X-Received: by 2002:a05:6512:247:b0:511:4dac:f1ab with SMTP id b7-20020a056512024700b005114dacf1abmr7746lfo.40.1707148067606;
        Mon, 05 Feb 2024 07:47:47 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCW/dmIqM9uVgxqG0FLJ93uD9oMbRyzHRyIhab5OSj8RfgH3RlySxMTfLooy0Vt1VHdLT10kIerNHIul+j5jTCu5JL1pCbTylBdpRtnx8TNwcXSfaBinWS3F1l/Cy0v8xJsY6dg0VeZfvaJ8R83xdOvpQMOsnAIOhTkT0aAg773nSYVkhpTN41nxcRP2X8lZeJ6QD5DJ3bJNqL1N/5Xy7BbrXakGl9Tz8H0OOhsYhA0MpvhV8iQiQoSvExDgVFn73FTUCZeKwFmR5fc+3nsFFf+F7VB4eFEIKh9r0O180kFIyVA1BNMYeuLiggIfAFS2oXa+
Message-ID: <2a21b871-8762-43ee-b9f3-bf34869d008f@suse.com>
Date: Mon, 5 Feb 2024 16:47:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/2] x86/IOMMU: address violations of MISRA C:2012 Rule
 14.4
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Kevin Tian <kevin.tian@intel.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1702310368.git.maria.celeste.cesario@bugseng.com>
 <746a33fff1386b2e76657b5f7cfb31f3b117a1fe.1702310368.git.maria.celeste.cesario@bugseng.com>
 <891a1ab0b772910999f49878ae679634@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <891a1ab0b772910999f49878ae679634@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:36, Nicola Vetrini wrote:
> On 2023-12-13 17:10, Simone Ballarin wrote:
>> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
>>
>> The xen sources contain violations of MISRA C:2012 Rule 14.4 whose
>> headline states:
>> "The controlling expression of an if statement and the controlling
>> expression of an iteration-statement shall have essentially Boolean 
>> type".
>>
>> Add comparisons to avoid using enum constants as controlling 
>> expressions
>> to comply with Rule 14.4.
>> No functional change.
>>
>> Signed-off-by: Maria Celeste Cesario  
>> <maria.celeste.cesario@bugseng.com>
>> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
>> ---
>> Changes in v2
>> - rename prefix from AMD/IOMMU to x86/IOMMU
>> - move changes on msi.c and hpet.c in this patch.
>> ---
>>  xen/arch/x86/hpet.c                      | 6 +++---
>>  xen/arch/x86/msi.c                       | 4 ++--
>>  xen/drivers/passthrough/amd/iommu_init.c | 4 ++--
>>  xen/drivers/passthrough/vtd/iommu.c      | 4 ++--
>>  xen/drivers/passthrough/vtd/quirks.c     | 2 +-
>>  5 files changed, 10 insertions(+), 10 deletions(-)
>>
> 
> +Stefano
> 
> Hi all,
> 
> this patch seems not to have been committed into staging, unlike the 
> other patch from this series. Since these are the only remaining 
> violations for Rule 14.4, then I think these changes could be reviewed.

It's no surprise the change isn't committed yet, when it hasn't had any
of the necessary tags. As far as I'm concerned, I seem to recall
indicating clearly that I'm not happy with this change, and hence acks
would need to come from elsewhere.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:48:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:48:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676336.1052469 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1Cf-0002T5-Nf; Mon, 05 Feb 2024 15:48:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676336.1052469; Mon, 05 Feb 2024 15:48:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1Cf-0002Sy-KZ; Mon, 05 Feb 2024 15:48:13 +0000
Received: by outflank-mailman (input) for mailman id 676336;
 Mon, 05 Feb 2024 15:48:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0y4-0007I6-SS
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:08 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dbd8ab44-c43b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 16:33:06 +0100 (CET)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-51124d43943so7188941e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:06 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dbd8ab44-c43b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147185; x=1707751985; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=qcO8xwP8eE9Ny9t0TSMCXWd0AmNb0Mm6hrsaI38kCRE=;
        b=ksnoSg4Z9AEiHTvUoDuGqCzxP+ktIU3NoCj7666sOMkhpWjvgVhI994avG86kGJY/x
         DbRbESEfAjSmUMNwnivWyA58XIGRLfTR3Myd9JFxq0q5pfw4CwQNc4CBBDYDqHBAGv43
         j9DqwXX48MxeGbCpzS4rpybZBevdT1K/bFd3Rfa9m9daZpw1w6ZFhzZ+3u5ljk57JID+
         i5B0V5TYjscicsk4WPHf1SQnXb25jMsBBsKCAa/DK9tWCYy3A1Ova7FnjxPtsvX5u3Oy
         u86kIWgVLbnRHknWJwFZx6sSivXbPOmOuKlXyzZ6euiVpWVIf9NXiCggW5O0BKO5p4we
         o3iw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147185; x=1707751985;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=qcO8xwP8eE9Ny9t0TSMCXWd0AmNb0Mm6hrsaI38kCRE=;
        b=b+dTMvX0JPBPVKuDC/NnruVnCQDVqOlVLC+eKuJ00A4qckigTXuxToKpyq/oS985g4
         idzXGWlfSgIRs3PQ98DqgHVuGuWkUoWMO1AI51VpuWgPhMUhQFjN7YSNZ1C5nBp4Xaic
         AJcxkZ0rNFYWOJSSKL29/+w9UBKb3QOmRKw+qCXhLgH4lR9pGwmivSrcNvcnnvCmc0s8
         me8wYwzRmnBFd5dQvnU/a8VwmYqgcj673iHEoPxKyazEoUZRQtewuFlKPSBQfeChs+a+
         v07bUtMQvQn1OpIobG5FYV6IOBkhAfkXI54tqgCWFY/DNaKvaJP78mBzh6Ig1dvOaJC1
         zmNg==
X-Gm-Message-State: AOJu0YwMuCUyOAxSpaAwqU9BIZcjV6a7KAEDPWZayLbDsZku13sa6IaA
	tJUqIg906LsD3MBSWHH4xg2divlXZYdb+6OQ9tQo4QPX3dwvEEIsEcRIbV+6
X-Google-Smtp-Source: AGHT+IFIzC8AdcGSBzbEPiks7ytlRuyAsj/j9OWBPq6/l5/zEMKKM6cwDyYthtTUCsqaXf4ND+HjDg==
X-Received: by 2002:a05:6512:3b08:b0:510:293e:83b with SMTP id f8-20020a0565123b0800b00510293e083bmr7694639lfv.18.1707147185488;
        Mon, 05 Feb 2024 07:33:05 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWJ8cDpdCo+idSwySOUCyUOifCs758PnOfDsZoIkm1E4U8dA9TJ9d38AiaxHlLDlEf4RT6oDxbcKWDTbsZPWax5nO0VmtTyEUvHs2GPyAkdLHStPZSU+gDN7eFtbWY7cqubJ1TOmjYQxWt2tcuYCYbylEJLrWpwj5nXZABy8TpSUnciTONjFFLgKbdhit+o9F3/CV+iOoVHFBN1MWH+ySKLmRE32Hchd+Z9bcjYgZnsqKVL1RCFE+IgjXdl5CCPQ4rBHHw/wB0KzwLyRkAXneh3jhw6dDbg/SRVXWPf8Hn4H5nORnjKrDPdhogm8sbccHGFwtqT
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 24/30] xen/riscv: add minimal stuff to page.h to build full Xen
Date: Mon,  5 Feb 2024 16:32:31 +0100
Message-ID: <63338d9a50979ef0e819f92719eb4e11154f6cff.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V4:
---
 - Change message -> subject in "Changes in V3"
 - s/BUG/BUG_ON("...")
 - Do proper rebase ( pfn_to_paddr() and paddr_to_pfn() aren't removed ).
---
Changes in V3:
 - update the commit subject
 - add implemetation of PAGE_HYPERVISOR macros
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
 - drop definition of pfn_to_addr, and paddr_to_pfn in <asm/mm.h>
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/page.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/xen/arch/riscv/include/asm/page.h b/xen/arch/riscv/include/asm/page.h
index 95074e29b3..c831e16417 100644
--- a/xen/arch/riscv/include/asm/page.h
+++ b/xen/arch/riscv/include/asm/page.h
@@ -6,6 +6,7 @@
 #ifndef __ASSEMBLY__
 
 #include <xen/const.h>
+#include <xen/bug.h>
 #include <xen/types.h>
 
 #include <asm/mm.h>
@@ -32,6 +33,10 @@
 #define PTE_LEAF_DEFAULT            (PTE_VALID | PTE_READABLE | PTE_WRITABLE)
 #define PTE_TABLE                   (PTE_VALID)
 
+#define PAGE_HYPERVISOR_RW          (PTE_VALID | PTE_READABLE | PTE_WRITABLE)
+
+#define PAGE_HYPERVISOR             PAGE_HYPERVISOR_RW
+
 /* Calculate the offsets into the pagetables for a given VA */
 #define pt_linear_offset(lvl, va)   ((va) >> XEN_PT_LEVEL_SHIFT(lvl))
 
@@ -62,6 +67,20 @@ static inline bool pte_is_valid(pte_t p)
     return p.pte & PTE_VALID;
 }
 
+static inline void invalidate_icache(void)
+{
+    BUG_ON("unimplemented");
+}
+
+#define clear_page(page) memset((void *)(page), 0, PAGE_SIZE)
+#define copy_page(dp, sp) memcpy(dp, sp, PAGE_SIZE)
+
+/* TODO: Flush the dcache for an entire page. */
+static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache)
+{
+    BUG_ON("unimplemented");
+}
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_RISCV_PAGE_H */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:49:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:49:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676341.1052479 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1Di-0003fe-0Y; Mon, 05 Feb 2024 15:49:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676341.1052479; Mon, 05 Feb 2024 15:49:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1Dh-0003fX-Tw; Mon, 05 Feb 2024 15:49:17 +0000
Received: by outflank-mailman (input) for mailman id 676341;
 Mon, 05 Feb 2024 15:49:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0yA-0007Hv-5F
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:14 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e00f2f61-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:33:13 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-5112a04c7acso7293837e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:13 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e00f2f61-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147192; x=1707751992; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=mHfOArS8uJvdUvmhNcbyjrkgkF05plzKh9QEvVv6hbw=;
        b=bLoNxZCoOZz11OEsjc8LJGnXFcSvDYbnpuvISuu5kV9G0XNkROTmOrbuHslvbCwB7/
         8yVGHvX0G+dJo08hoBkFIG0o1DKW1BPelVWjxsD3uXOYDomSGtgx+xp7wqHleKW54VXr
         X5r4rs72J/UtLVnM798rieJxK8DvLFqxq4kngMI4Cx2A7Rl6rK2EXodEB9Gj7xtme7YI
         E8vVeSD5yM2a9fs7KqvZ6Pr8W7og01EKKvWcanRqWt85LmV3XKXw7N/G64JCzErxfqob
         kXJ0NLJvUqcXIFj8GJcJVefjXJIZehML3pg5+sb7X5e6UAnODOEx604l9oOA3PPVcKNU
         VBmw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147192; x=1707751992;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=mHfOArS8uJvdUvmhNcbyjrkgkF05plzKh9QEvVv6hbw=;
        b=OC26O00Xke4vbFjyJf4o1Us3ZZyaXZ59+y4CuGfQSYEj7ZULrQUxe5yVKAR2aXg3aY
         JpuvAmrPs5i7XyfFKaM3+rldBmXD/+D6thhrMcksj/rCO55Hojt4DLmzdW3pJ02dY33p
         OrbGLZYUCUfosYQKYsAo9qm26OBO55iQtGhI65gb9k4qyO5dHnxSIa2zqeSxah87KN6I
         qCaynFosp/WQoOn/4PpcXavE6EJ6ChnYKhBByCpn6X10KGEt+IdfCXU7g8YJH5ripIXk
         pqghPCg2R9sqNI0eCE9oyCz3Vg3fkVMyza9TqGRP4VbH0i3wiQh6i6pHuq2lO4upvUUS
         MQmQ==
X-Gm-Message-State: AOJu0YzqTIU167HfXgwYy+CUW4MFRSjvvrAEc2QwGKijGrwNtH4BjJBG
	NBNcWxNNkl64+as2AC/yYFJCMuVnnp/Yh1Ue4jJy0CqUy1yTtZMiFBgVBsQl
X-Google-Smtp-Source: AGHT+IFBvdM1BK0z6wUkWhrRiZ3w2wNb0SO1kX6dmGbvkfVIsOwXcHO3jk/3iF64lOET80/d5b1U7A==
X-Received: by 2002:a05:6512:3d1d:b0:511:47ed:d744 with SMTP id d29-20020a0565123d1d00b0051147edd744mr4893576lfv.12.1707147192622;
        Mon, 05 Feb 2024 07:33:12 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXFoXbLJwXhoXwmg9HQ+lqPXk8ylmx6jbLjT+zM89gRYK/F0wv/tsMO4LoKFT5RIGEPAjMnZtD9g8xotGFo6TZrZ7UmGcGLygewAGSL5QM9fTZN1n8ffiiEhq1ZH2XVuRwtsb9xjdRLQTjItvvEuW6JbI7lAfj4eugYX4XgbSMr/lBK6App7anZC0zPg96h0B7Xd5vLLt39uhriz3C3sqzNSUnrzA4uWJYcKw==
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 30/30] xen/README: add compiler and binutils versions for RISC-V64
Date: Mon,  5 Feb 2024 16:32:37 +0100
Message-ID: <d62dff38ee661f3fb713554d544c966fa889fd83.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Changes in V4:
  - Update version of GCC (12.2) and GNU Binutils (2.39) to the version
    which are in Xen's contrainter for RISC-V
---
 Changes in V3:
  - new patch
---
 README | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/README b/README
index c8a108449e..9a898125e1 100644
--- a/README
+++ b/README
@@ -48,6 +48,9 @@ provided by your OS distributor:
       - For ARM 64-bit:
         - GCC 5.1 or later
         - GNU Binutils 2.24 or later
+      - For RISC-V 64-bit:
+        - GCC 12.2 or later
+        - GNU Binutils 2.39 or later
     * POSIX compatible awk
     * Development install of zlib (e.g., zlib-dev)
     * Development install of Python 2.7 or later (e.g., python-dev)
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:49:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:49:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676342.1052489 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1Dr-00043K-8I; Mon, 05 Feb 2024 15:49:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676342.1052489; Mon, 05 Feb 2024 15:49:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1Dr-00043D-5E; Mon, 05 Feb 2024 15:49:27 +0000
Received: by outflank-mailman (input) for mailman id 676342;
 Mon, 05 Feb 2024 15:49:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jdDv=JO=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1rX1Dq-0003z0-GY
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:49:26 +0000
Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com
 [2607:f8b0:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 22b2eb8d-c43e-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:49:25 +0100 (CET)
Received: by mail-pg1-x52c.google.com with SMTP id
 41be03b00d2f7-5cdf76cde78so4083109a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:49:25 -0800 (PST)
Received: from localhost.localdomain (h-217-31-164-171.A175.priv.bahnhof.se.
 [217.31.164.171]) by smtp.gmail.com with ESMTPSA id
 h35-20020a635763000000b005d651d4236dsm69184pgm.86.2024.02.05.07.49.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:49:22 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22b2eb8d-c43e-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1707148163; x=1707752963; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=x0jQSs+xMp/AYCnSyfvm8mzv7aEdRMeM6lVxU/SiBzI=;
        b=IahC1+KwCBk/mFo4se9lDvIgPpLceWsderCRcZ/D1YlOJLtgV2oqvSzYeQvtfccYpv
         cZUGr+wiLQ59bI0xYQH4A/19Q+hEHrHXsIKQ7bDk1svao9nt3GGqCbTw/5RocIChuVTz
         +VE66xp0ZzMHK1fr9bfSqXHBIIJWQS5GWEZfIZKdY0r3YSWoO9YkzPhfth8gGLn+t7of
         JL67rXSyVVvp/eQLGsYPn/wTmnChq39ZHfi00mVvsiQ3MqMRwCdff1iaakMS20IDEcvO
         sKGEajVIm2AcoY35Yyzidc9Iz+D5ToHkrjFZRYyN/rSxhJeJXPIgSCIKO9+gLwjhfZoo
         0raQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707148163; x=1707752963;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=x0jQSs+xMp/AYCnSyfvm8mzv7aEdRMeM6lVxU/SiBzI=;
        b=kMKHjv+Zspou+qcNR9nNIaCccjGZ3xqXL7dMoF9C93QZKEcEiRUgvq7yBkJRV+tC9Q
         2rDT9rmjUUJRGHpSkSSMuqBk5xn9AKRaoGg119NckLioTxwXzWfCxO1fc9s1xXzDi61A
         nVUvfGltaCW5ddnTLbJd97sjK0IRAdnaKps+NHyv7i3K+WmSWQXYg6LobJZ5bXY8tMcR
         Lcq9nDk8jAptObBSjL+SosOSUySCBKJH0iRg9m8p1l5rCzKfD4LSxA//MLrdgQakf6A9
         5b41PI3KlbHHfUeJAy/yV514TbbDtzxgX4HCqGz8z75F+tjW5MHf0x10V4toQJOycpzx
         XQng==
X-Gm-Message-State: AOJu0YzgB8OKXle0UZ73CZ9i2K+jZ4ksq0xOAkFFl5emkO+AXJKfO5Xb
	9rmtyyvVBrVliLGa7+ho1B4b5EronFjVJh9M4YuO+0AmWb32NTZaIX1AO+fCPT35vJwnesgdFre
	0
X-Google-Smtp-Source: AGHT+IFfabSLrxkpkxLEuQudPIXI8NorlRh9x2u1XpsVWHNL2uys0t746eB8FRDVgN/awy2Okclwhw==
X-Received: by 2002:a05:6a20:9c97:b0:19c:9c9a:1b09 with SMTP id mj23-20020a056a209c9700b0019c9c9a1b09mr9678959pzb.10.1707148163378;
        Mon, 05 Feb 2024 07:49:23 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUesF0V/mXcz0KWIfH7HXtsKW+/Ha9lyYjSGxdk5SrsxFRpurqYhs/fldEk19gyiIJdu3+DzXK2lfIYK02C3T/uqaDsKxG2ZGhIsjD9AiAwd2qFLSsZNOpbtjT10o0dxv9hfKV2pPyoswIDP7y9dEexhxcgQFBOb3vN06cVc8hzBXGyIN6R81yEEXrBdfft9k9pde+neEIBYB0eKJFu5+wrtAmX0d1CbvBuIrX4nLYmyWjRViWrqyGpD1LAMhwReIi5HIPHfGUumeB2TUHTmi2TwI0/GqLC6gE=
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [XEN PATCH v4] xen/arm: ffa: reclaim shared memory on guest destroy
Date: Mon,  5 Feb 2024 16:49:09 +0100
Message-Id: <20240205154909.2392231-1-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

When an FF-A enabled guest is destroyed it may leave behind memory
shared with SPs. This memory must be reclaimed before it's reused or an
SP may make changes to memory used by a new unrelated guest. So when the
domain is teared down add FF-A requests to reclaim all remaining shared
memory.

SPs in the secure world are notified using VM_DESTROYED that a guest has
been destroyed. An SP is supposed to relinquish all shared memory to allow
reclaiming the memory. The relinquish operation may need to be delayed if
the shared memory is for instance part of a DMA operation.

The domain reference counter is increased when the first FF-A shared
memory is registered and the counter is decreased again when the last
shared memory is reclaimed. If FF-A shared memory registrations remain
at the end of of ffa_domain_teardown() a timer is set to try to reclaim
the shared memory every second until the memory is reclaimed.

A few minor style fixes with a removed empty line here and an added new
line there.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
---

v4:
- Retry FFA_MSG_SEND_VM_DESTROYED on the returned errors
  FFA_RET_INTERRUPTED and FFA_RET_RETRY even after all shared memory
  handles has been reclaimed.
- Retry ffa_mem_reclaim() only on temporary returned errors, permanent
  errors like FFA_RET_INVALID_PARAMETERS is dealt with in recovery mode
  instead since the SPMC isn't expected to use that error under normal
  circumstances.

v3:
- Mentioning in the commit message that there are some style fixes
- Addressing review comments
- Refactor the ffa_domain_teardown() path to let
  ffa_domain_teardown_continue() do most of the work.

v2:
- Update commit message to match the new implementation
- Using a per domain bitfield to keep track of which SPs has been notified
  with VM_DESTROYED
- Holding a domain reference counter to keep the domain as a zombie domain
  while there still is shared memory registrations remaining to be reclaimed
- Using a timer to retry reclaiming remaining shared memory registrations
---
 xen/arch/arm/tee/ffa.c | 285 ++++++++++++++++++++++++++++++++++-------
 1 file changed, 236 insertions(+), 49 deletions(-)

diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
index 0793c1c7585d..bbb6b819ee2b 100644
--- a/xen/arch/arm/tee/ffa.c
+++ b/xen/arch/arm/tee/ffa.c
@@ -54,6 +54,7 @@
 #include <xen/mm.h>
 #include <xen/sched.h>
 #include <xen/sizes.h>
+#include <xen/timer.h>
 #include <xen/types.h>
 
 #include <asm/event.h>
@@ -144,6 +145,12 @@
  */
 #define FFA_MAX_SHM_COUNT               32
 
+/*
+ * The time we wait until trying to tear down a domain again if it was
+ * blocked initially.
+ */
+#define FFA_CTX_TEARDOWN_DELAY          SECONDS(1)
+
 /* FF-A-1.1-REL0 section 10.9.2 Memory region handle, page 167 */
 #define FFA_HANDLE_HYP_FLAG             BIT(63, ULL)
 #define FFA_HANDLE_INVALID              0xffffffffffffffffULL
@@ -384,11 +391,6 @@ struct ffa_ctx {
     unsigned int page_count;
     /* FF-A version used by the guest */
     uint32_t guest_vers;
-    /*
-     * Number of SPs that we have sent a VM created signal to, used in
-     * ffa_domain_teardown() to know which SPs need to be signalled.
-     */
-    uint16_t create_signal_count;
     bool rx_is_free;
     /* Used shared memory objects, struct ffa_shm_mem */
     struct list_head shm_list;
@@ -402,6 +404,15 @@ struct ffa_ctx {
     spinlock_t tx_lock;
     spinlock_t rx_lock;
     spinlock_t lock;
+    /* Used if domain can't be torn down immediately */
+    struct domain *teardown_d;
+    struct list_head teardown_list;
+    s_time_t teardown_expire;
+    /*
+     * Used for ffa_domain_teardown() to keep track of which SPs should be
+     * notified that this guest is being destroyed.
+     */
+    unsigned long vm_destroy_bitmap[];
 };
 
 struct ffa_shm_mem {
@@ -436,6 +447,12 @@ static void *ffa_tx __read_mostly;
 static DEFINE_SPINLOCK(ffa_rx_buffer_lock);
 static DEFINE_SPINLOCK(ffa_tx_buffer_lock);
 
+
+/* Used to track domains that could not be torn down immediately. */
+static struct timer ffa_teardown_timer;
+static struct list_head ffa_teardown_head;
+static DEFINE_SPINLOCK(ffa_teardown_lock);
+
 static bool ffa_get_version(uint32_t *vers)
 {
     const struct arm_smccc_1_2_regs arg = {
@@ -853,7 +870,6 @@ static int32_t handle_partition_info_get(uint32_t w1, uint32_t w2, uint32_t w3,
             goto out_rx_release;
         }
 
-
         memcpy(ctx->rx, ffa_rx, sz);
     }
     ctx->rx_is_free = false;
@@ -992,53 +1008,75 @@ static void put_shm_pages(struct ffa_shm_mem *shm)
     }
 }
 
-static bool inc_ctx_shm_count(struct ffa_ctx *ctx)
+static bool inc_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
 {
     bool ret = true;
 
     spin_lock(&ctx->lock);
+
+    /*
+     * If this is the first shm added, increase the domain reference
+     * counter as we need to keep domain around a bit longer to reclaim the
+     * shared memory in the teardown path.
+     */
+    if ( !ctx->shm_count )
+        get_knownalive_domain(d);
+
     if (ctx->shm_count >= FFA_MAX_SHM_COUNT)
         ret = false;
     else
         ctx->shm_count++;
+
     spin_unlock(&ctx->lock);
 
     return ret;
 }
 
-static void dec_ctx_shm_count(struct ffa_ctx *ctx)
+static void dec_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
 {
     spin_lock(&ctx->lock);
+
     ASSERT(ctx->shm_count > 0);
     ctx->shm_count--;
+
+    /*
+     * If this was the last shm removed, let go of the domain reference we
+     * took in inc_ctx_shm_count() above.
+     */
+    if ( !ctx->shm_count )
+        put_domain(d);
+
     spin_unlock(&ctx->lock);
 }
 
-static struct ffa_shm_mem *alloc_ffa_shm_mem(struct ffa_ctx *ctx,
+static struct ffa_shm_mem *alloc_ffa_shm_mem(struct domain *d,
                                              unsigned int page_count)
 {
+    struct ffa_ctx *ctx = d->arch.tee;
     struct ffa_shm_mem *shm;
 
     if ( page_count >= FFA_MAX_SHM_PAGE_COUNT )
         return NULL;
-    if ( !inc_ctx_shm_count(ctx) )
+    if ( !inc_ctx_shm_count(d, ctx) )
         return NULL;
 
     shm = xzalloc_flex_struct(struct ffa_shm_mem, pages, page_count);
     if ( shm )
         shm->page_count = page_count;
     else
-        dec_ctx_shm_count(ctx);
+        dec_ctx_shm_count(d, ctx);
 
     return shm;
 }
 
-static void free_ffa_shm_mem(struct ffa_ctx *ctx, struct ffa_shm_mem *shm)
+static void free_ffa_shm_mem(struct domain *d, struct ffa_shm_mem *shm)
 {
+    struct ffa_ctx *ctx = d->arch.tee;
+
     if ( !shm )
         return;
 
-    dec_ctx_shm_count(ctx);
+    dec_ctx_shm_count(d, ctx);
     put_shm_pages(shm);
     xfree(shm);
 }
@@ -1306,7 +1344,7 @@ static void handle_mem_share(struct cpu_user_regs *regs)
         goto out_unlock;
     }
 
-    shm = alloc_ffa_shm_mem(ctx, page_count);
+    shm = alloc_ffa_shm_mem(d, page_count);
     if ( !shm )
     {
         ret = FFA_RET_NO_MEMORY;
@@ -1350,7 +1388,7 @@ static void handle_mem_share(struct cpu_user_regs *regs)
 
 out:
     if ( ret )
-        free_ffa_shm_mem(ctx, shm);
+        free_ffa_shm_mem(d, shm);
 out_unlock:
     spin_unlock(&ctx->tx_lock);
 
@@ -1401,7 +1439,7 @@ static int handle_mem_reclaim(uint64_t handle, uint32_t flags)
     }
     else
     {
-        free_ffa_shm_mem(ctx, shm);
+        free_ffa_shm_mem(d, shm);
     }
 
     return ret;
@@ -1486,6 +1524,41 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
     }
 }
 
+static bool is_in_subscr_list(const uint16_t *subscr, uint16_t start,
+                              uint16_t end, uint16_t sp_id)
+{
+    unsigned int n;
+
+    for ( n = start; n < end; n++ )
+    {
+        if ( subscr[n] == sp_id )
+            return true;
+    }
+
+    return false;
+}
+
+static void vm_destroy_bitmap_init(struct ffa_ctx *ctx,
+                                   unsigned int create_signal_count)
+{
+    unsigned int n;
+
+    for ( n = 0; n < subscr_vm_destroyed_count; n++ )
+    {
+        /*
+         * Skip SPs subscribed to the VM created event that never was
+         * notified of the VM creation due to an error during
+         * ffa_domain_init().
+         */
+        if ( is_in_subscr_list(subscr_vm_created, create_signal_count,
+                               subscr_vm_created_count,
+                               subscr_vm_destroyed[n]) )
+            continue;
+
+        set_bit(n, ctx->vm_destroy_bitmap);
+    }
+}
+
 static int ffa_domain_init(struct domain *d)
 {
     struct ffa_ctx *ctx;
@@ -1501,11 +1574,14 @@ static int ffa_domain_init(struct domain *d)
     if ( d->domain_id >= UINT16_MAX)
         return -ERANGE;
 
-    ctx = xzalloc(struct ffa_ctx);
+    ctx = xzalloc_flex_struct(struct ffa_ctx, vm_destroy_bitmap,
+                              BITS_TO_LONGS(subscr_vm_destroyed_count));
     if ( !ctx )
         return -ENOMEM;
 
     d->arch.tee = ctx;
+    ctx->teardown_d = d;
+    INIT_LIST_HEAD(&ctx->shm_list);
 
     for ( n = 0; n < subscr_vm_created_count; n++ )
     {
@@ -1515,65 +1591,173 @@ static int ffa_domain_init(struct domain *d)
         {
             printk(XENLOG_ERR "ffa: Failed to report creation of vm_id %u to  %u: res %d\n",
                    get_vm_id(d), subscr_vm_created[n], res);
-            ctx->create_signal_count = n;
-            return -EIO;
+            break;
         }
     }
-    ctx->create_signal_count = subscr_vm_created_count;
-
-    INIT_LIST_HEAD(&ctx->shm_list);
+    vm_destroy_bitmap_init(ctx, n);
+    if ( n != subscr_vm_created_count )
+        return -EIO;
 
     return 0;
 }
 
-static bool is_in_subscr_list(const uint16_t *subscr, uint16_t start,
-                              uint16_t end, uint16_t sp_id)
+static void send_vm_destroyed(struct domain *d)
 {
+    struct ffa_ctx *ctx = d->arch.tee;
     unsigned int n;
+    int32_t res;
 
-    for ( n = start; n < end; n++ )
+    for ( n = 0; n < subscr_vm_destroyed_count; n++ )
     {
-        if ( subscr[n] == sp_id )
-            return true;
-    }
+        if ( !test_bit(n, ctx->vm_destroy_bitmap) )
+            continue;
 
-    return false;
+        res = ffa_direct_req_send_vm(subscr_vm_destroyed[n], get_vm_id(d),
+                                     FFA_MSG_SEND_VM_DESTROYED);
+
+        if ( res )
+        {
+            printk(XENLOG_ERR "%pd: ffa: Failed to report destruction of vm_id %u to %u: res %d\n",
+                   d, get_vm_id(d), subscr_vm_destroyed[n], res);
+        }
+
+        /*
+         * For these two error codes the hypervisor is expected to resend
+         * the destruction message. For the rest it is expected that the
+         * error is permanent and that is doesn't help to resend the
+         * destruction message.
+         */
+        if ( res != FFA_RET_INTERRUPTED && res != FFA_RET_RETRY )
+            clear_bit(n, ctx->vm_destroy_bitmap);
+    }
 }
 
-/* This function is supposed to undo what ffa_domain_init() has done */
-static int ffa_domain_teardown(struct domain *d)
+static void reclaim_shms(struct domain *d)
 {
     struct ffa_ctx *ctx = d->arch.tee;
-    unsigned int n;
+    struct ffa_shm_mem *shm, *tmp;
     int32_t res;
 
-    if ( !ctx )
-        return 0;
+    list_for_each_entry_safe(shm, tmp, &ctx->shm_list, list)
+    {
+        register_t handle_hi;
+        register_t handle_lo;
+
+        uint64_to_regpair(&handle_hi, &handle_lo, shm->handle);
+        res = ffa_mem_reclaim(handle_lo, handle_hi, 0);
+        switch ( res ) {
+        case FFA_RET_OK:
+            printk(XENLOG_G_DEBUG "%pd: ffa: Reclaimed handle %#lx\n",
+                   d, shm->handle);
+            list_del(&shm->list);
+            free_ffa_shm_mem(d, shm);
+            break;
+        case FFA_RET_DENIED:
+            /*
+             * A temporary error that may get resolved a bit later, it's
+             * worth retrying.
+             */
+            printk(XENLOG_G_INFO "%pd: ffa: Failed to reclaim handle %#lx : %d\n",
+                   d, shm->handle, res);
+            break; /* We will retry later */
+        default:
+            /*
+             * The rest of the error codes are not expected and are assumed
+             * to be of a permanent nature. It not in our control to handle
+             * the error properly so the object in this case is to try to
+             * minimize the damage.
+             *
+             * FFA_RET_NO_MEMORY might be a temporary error as it it could
+             * succeed if retried later, but treat it as permanent for now.
+             */
+            printk(XENLOG_G_INFO "%pd: ffa: Permanent failure to reclaim handle %#lx : %d\n",
+                   d, shm->handle, res);
 
-    for ( n = 0; n < subscr_vm_destroyed_count; n++ )
+            /*
+             * Remove the shm from the list and free it, but don't drop
+             * references. This results in having the shared physical pages
+             * permanently allocate and also keeps the domain as a zombie
+             * domain.
+             */
+            list_del(&shm->list);
+            xfree(shm);
+            break;
+        }
+    }
+}
+
+static void ffa_domain_teardown_continue(struct ffa_ctx *ctx, bool first_time)
+{
+    struct ffa_ctx *next_ctx = NULL;
+
+    send_vm_destroyed(ctx->teardown_d);
+    reclaim_shms(ctx->teardown_d);
+
+    if ( ctx->shm_count ||
+         !bitmap_empty(ctx->vm_destroy_bitmap, subscr_vm_destroyed_count) )
+    {
+        printk(XENLOG_G_INFO "%pd: ffa: Remaining cleanup, retrying\n", ctx->teardown_d);
+
+        ctx->teardown_expire = NOW() + FFA_CTX_TEARDOWN_DELAY;
+
+        spin_lock(&ffa_teardown_lock);
+        list_add_tail(&ctx->teardown_list, &ffa_teardown_head);
+        /* Need to set a new timer for the next ctx in line */
+        next_ctx = list_first_entry(&ffa_teardown_head, struct ffa_ctx,
+                                    teardown_list);
+        spin_unlock(&ffa_teardown_lock);
+    }
+    else
     {
         /*
-         * Skip SPs subscribed to the VM created event that never was
-         * notified of the VM creation due to an error during
-         * ffa_domain_init().
+         * domain_destroy() might have been called (via put_domain() in
+         * reclaim_shms()), so we can't touch the domain structure anymore.
          */
-        if ( is_in_subscr_list(subscr_vm_created, ctx->create_signal_count,
-                               subscr_vm_created_count,
-                               subscr_vm_destroyed[n]) )
-            continue;
+        xfree(ctx);
 
-        res = ffa_direct_req_send_vm(subscr_vm_destroyed[n], get_vm_id(d),
-                                     FFA_MSG_SEND_VM_DESTROYED);
-
-        if ( res )
-            printk(XENLOG_ERR "ffa: Failed to report destruction of vm_id %u to  %u: res %d\n",
-                   get_vm_id(d), subscr_vm_destroyed[n], res);
+        /* Only check if there has been a change to the teardown queue */
+        if ( !first_time )
+        {
+            spin_lock(&ffa_teardown_lock);
+            next_ctx = list_first_entry_or_null(&ffa_teardown_head,
+                                                struct ffa_ctx, teardown_list);
+            spin_unlock(&ffa_teardown_lock);
+        }
     }
 
+    if ( next_ctx )
+        set_timer(&ffa_teardown_timer, next_ctx->teardown_expire);
+}
+
+static void ffa_teardown_timer_callback(void *arg)
+{
+    struct ffa_ctx *ctx;
+
+    spin_lock(&ffa_teardown_lock);
+    ctx = list_first_entry_or_null(&ffa_teardown_head, struct ffa_ctx,
+                                   teardown_list);
+    if ( ctx )
+        list_del(&ctx->teardown_list);
+    spin_unlock(&ffa_teardown_lock);
+
+    if ( ctx )
+        ffa_domain_teardown_continue(ctx, false /* !first_time */);
+    else
+        printk(XENLOG_G_ERR "%s: teardown list is empty\n", __func__);
+}
+
+/* This function is supposed to undo what ffa_domain_init() has done */
+static int ffa_domain_teardown(struct domain *d)
+{
+    struct ffa_ctx *ctx = d->arch.tee;
+
+    if ( !ctx )
+        return 0;
+
     if ( ctx->rx )
         rxtx_unmap(ctx);
 
-    XFREE(d->arch.tee);
+    ffa_domain_teardown_continue(ctx, true /* first_time */);
 
     return 0;
 }
@@ -1739,6 +1923,9 @@ static bool ffa_probe(void)
     if ( !init_sps() )
         goto err_free_ffa_tx;
 
+    INIT_LIST_HEAD(&ffa_teardown_head);
+    init_timer(&ffa_teardown_timer, ffa_teardown_timer_callback, NULL, 0);
+
     return true;
 
 err_free_ffa_tx:
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:49:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:49:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676344.1052499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1E0-0004Zg-KD; Mon, 05 Feb 2024 15:49:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676344.1052499; Mon, 05 Feb 2024 15:49:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1E0-0004ZZ-GJ; Mon, 05 Feb 2024 15:49:36 +0000
Received: by outflank-mailman (input) for mailman id 676344;
 Mon, 05 Feb 2024 15:49:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX0y8-0007Hv-5g
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:33:12 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dea7b323-c43b-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 16:33:11 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-51124d43943so7189082e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 07:33:11 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 s1-20020a056512314100b005114d401157sm474445lfi.2.2024.02.05.07.33.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 07:33:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dea7b323-c43b-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707147190; x=1707751990; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=QjJUMRgdV4G9iSHV/64jl4oYvH1idC9s9JFsJgFEqL0=;
        b=JxIjw7e8lPM5xi4QASfbjaV1m4FqDrCb7lcgcb/ffyzek10Ex1qdqJSCm0nVVb78e0
         Qv50ATAmWSEKHSL5L963zPe+COSNjZHBbKuiZ0UcUk/O+/SW46c7WJ3TMTMp6lgZ4l6N
         54C800q6ct1+SU14+iCo1OM93RBA7LxPiwy3DAu1yIM4HM3oW0w3Ruu/qURKnW2SCX9c
         DgfEnEcCyzXaENqj6Ba+AvJMpQMH8F9o9wQ6BStyvvKZwsq3GwVAylwOQE/Y+/+LGj8j
         tNtxFdyoGMMQ6yDNXvNcAoU2dADi9RDkEG4+GfV0cdDGLVOpV5cON9uhnKcPpYkisGG2
         NZRQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707147190; x=1707751990;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=QjJUMRgdV4G9iSHV/64jl4oYvH1idC9s9JFsJgFEqL0=;
        b=xQ98++RQUeYJPBd5uPj/y3XRqmq0IqD46tUXiYjTF1FyT6++P4aFgcvw3T1gTzFDNh
         qvPxBIUd+RrQISY4z+9pEZ9N+bzatX1XXM9JdeUbO7Yql6JDHpeQNjX7jEtxnEFUPmxw
         L98YmyPTfat8/yOBh1sjvIGzJkDjHq6MOXnBpx2BOAYWcQ3OTk5on1m6YII58LQeHOwI
         YImnieP3cSZFqagPjwESSfJ7Ngi89Ea0L1nAKEmhTZZCvKXrmk8EstHPTWwklQqI+85v
         KkGImdHqSFHXW6lGwp+OLT6TKFA2vo7WFZrR5DD0N4vtgWOdIdAHHjg1w9DU6smBVKuW
         54mA==
X-Gm-Message-State: AOJu0YxibeHjkEdzGRHuIGd8qT37c5lr2tLfr4YwEoLhQEItzUrDiy9Y
	aTCOKwVGFthKjV3t8HNgtL4PvztE+pUqWPJ+ITzJBtUuL+oRrmm+hNZue4hG
X-Google-Smtp-Source: AGHT+IFevWNlgkqOrGWxG9VGuyiieMGXHUouY/Wd94hUuX5G4mAyz47JB3L3hubnQp0m9HK2lyuoTA==
X-Received: by 2002:a05:6512:3b08:b0:510:293e:83b with SMTP id f8-20020a0565123b0800b00510293e083bmr7694847lfv.18.1707147190146;
        Mon, 05 Feb 2024 07:33:10 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUAcQagK9RBOsqVV/TIf9F+rGkw6KO6C24HfrTDOxG7Jheo7AdEIbupCYkgG5sBKPMCgDgnwB0LVxgXIFm3/1P76wiJRtN7ZYG9YDM12beV3+2BqDDurY8/NehRNW+55aK8MBnc1SOxOUMO4nyW/JefDekncGZvte06G5uz0tjiH3YipEOWa+EE7V3OS3k5tXHQViwqt7VHHKmYNCHkA02n4vMnbDT+BIjpDzsA3WOMYmX1oKksURM1uaPez5owCHqmX8hGcCrgK2v19d08kGU/TGTI4AW2mUX7GY6Loc/FNmXowNRU+6sQy8EE+IgmOfit2ga5
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4 28/30] xen/rirscv: add minimal amount of stubs to build full Xen
Date: Mon,  5 Feb 2024 16:32:35 +0100
Message-ID: <e25d55d1cb95c1b2b8ad0b4eae9794b5b6f84075.1707146506.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707146506.git.oleksii.kurochko@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V4:
  - added new stubs which are necessary for compilation after rebase: __cpu_up(), __cpu_disable(), __cpu_die()
    from smpboot.c
  - back changes related to printk() in early_printk() as they should be removed in the next patch to avoid
    compilation error.
  - update definition of cpu_khz: __read_mostly -> __ro_after_init.
  - drop vm_event_reset_vmtrace(). It is defibed in asm-generic/vm_event.h.
  - move vm_event_*() functions from stubs.c to riscv/vm_event.c.
  - s/BUG/BUG_ON("unimplemented") in stubs.c
  - back irq_actor_none() and irq_actor_none() as common/irq.c isn't compiled at this moment,
    so this function are needed to avoid compilation error.
  - defined max_page to avoid compilation error, it will be removed as soon as common/page_alloc.c will
    be compiled.
---
Changes in V3:
 - code style fixes.
 - update attribute for frametable_base_pdx  and frametable_virt_end to __ro_after_init.
   insteaf of read_mostly.
 - use BUG() instead of assert_failed/WARN for newly introduced stubs.
 - drop "#include <public/vm_event.h>" in stubs.c and use forward declaration instead.
 - drop ack_node() and end_node() as they aren't used now.
---
Changes in V2:
 - define udelay stub
 - remove 'select HAS_PDX' from RISC-V Kconfig because of
   https://lore.kernel.org/xen-devel/20231006144405.1078260-1-andrew.cooper3@citrix.com/
---
 xen/arch/riscv/Makefile       |   1 +
 xen/arch/riscv/early_printk.c |   1 -
 xen/arch/riscv/mm.c           |  50 ++++
 xen/arch/riscv/setup.c        |   8 +
 xen/arch/riscv/stubs.c        | 438 ++++++++++++++++++++++++++++++++++
 xen/arch/riscv/traps.c        |  25 ++
 6 files changed, 522 insertions(+), 1 deletion(-)
 create mode 100644 xen/arch/riscv/stubs.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 1ed1a8369b..60afbc0ad9 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -4,6 +4,7 @@ obj-y += mm.o
 obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
+obj-y += stubs.o
 obj-y += traps.o
 obj-y += vm_event.o
 
diff --git a/xen/arch/riscv/early_printk.c b/xen/arch/riscv/early_printk.c
index 60742a042d..6d0911659d 100644
--- a/xen/arch/riscv/early_printk.c
+++ b/xen/arch/riscv/early_printk.c
@@ -207,4 +207,3 @@ void printk(const char *format, ...)
 }
 
 #endif
-
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index fe3a43be20..2c3fb7d72e 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/cache.h>
 #include <xen/compiler.h>
 #include <xen/init.h>
@@ -14,6 +15,9 @@
 #include <asm/page.h>
 #include <asm/processor.h>
 
+unsigned long __ro_after_init frametable_base_pdx;
+unsigned long __ro_after_init frametable_virt_end;
+
 struct mmu_desc {
     unsigned int num_levels;
     unsigned int pgtbl_count;
@@ -294,3 +298,49 @@ unsigned long __init calc_phys_offset(void)
     phys_offset = load_start - XEN_VIRT_START;
     return phys_offset;
 }
+
+void put_page(struct page_info *page)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long get_upper_mfn_bound(void)
+{
+    /* No memory hotplug yet, so current memory limit is the final one. */
+    return max_page - 1;
+}
+
+void arch_dump_shared_mem_info(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int populate_pt_range(unsigned long virt, unsigned long nr_mfns)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int xenmem_add_to_physmap_one(struct domain *d, unsigned int space,
+                              union add_to_physmap_extra extra,
+                              unsigned long idx, gfn_t gfn)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
+
+int destroy_xen_mappings(unsigned long s, unsigned long e)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int map_pages_to_xen(unsigned long virt,
+                     mfn_t mfn,
+                     unsigned long nr_mfns,
+                     unsigned int flags)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 98a94c4c48..8bb5bdb2ae 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -1,11 +1,19 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/compile.h>
 #include <xen/init.h>
 #include <xen/mm.h>
 
+#include <public/version.h>
+
 #include <asm/early_printk.h>
 
+void arch_get_xen_caps(xen_capabilities_info_t *info)
+{
+    BUG_ON("unimplemented");
+}
+
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
     __aligned(STACK_SIZE);
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
new file mode 100644
index 0000000000..529f1dbe52
--- /dev/null
+++ b/xen/arch/riscv/stubs.c
@@ -0,0 +1,438 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#include <xen/cpumask.h>
+#include <xen/domain.h>
+#include <xen/irq.h>
+#include <xen/nodemask.h>
+#include <xen/time.h>
+#include <public/domctl.h>
+
+#include <asm/current.h>
+
+/* smpboot.c */
+
+cpumask_t cpu_online_map;
+cpumask_t cpu_present_map;
+cpumask_t cpu_possible_map;
+
+/* ID of the PCPU we're running on */
+DEFINE_PER_CPU(unsigned int, cpu_id);
+/* XXX these seem awfully x86ish... */
+/* representing HT siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
+/* representing HT and core siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
+
+nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
+
+/*
+ * max_page is defined in page_alloc.c which isn't complied for now.
+ * definition of max_page will be remove as soon as page_alloc is built.
+ */
+unsigned long __read_mostly max_page;
+
+/* time.c */
+
+unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
+
+s_time_t get_s_time(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int reprogram_timer(s_time_t timeout)
+{
+    BUG_ON("unimplemented");
+}
+
+void send_timer_event(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void domain_set_time_offset(struct domain *d, int64_t time_offset_seconds)
+{
+    BUG_ON("unimplemented");
+}
+
+/* shutdown.c */
+
+void machine_restart(unsigned int delay_millisecs)
+{
+    BUG_ON("unimplemented");
+}
+
+void machine_halt(void)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domctl.c */
+
+long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
+                    XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_domain_info(const struct domain *d,
+                          struct xen_domctl_getdomaininfo *info)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+/* monitor.c */
+
+int arch_monitor_domctl_event(struct domain *d,
+                              struct xen_domctl_monitor_op *mop)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smp.c */
+
+void arch_flush_tlb_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_event_check_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_call_function_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+/* irq.c */
+
+struct pirq *alloc_pirq_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int pirq_guest_bind(struct vcpu *v, struct pirq *pirq, int will_share)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_guest_unbind(struct domain *d, struct pirq *pirq)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_set_affinity(struct domain *d, int pirq, const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+hw_irq_controller no_irq_type = {
+    .typename = "none",
+    .startup = irq_startup_none,
+    .shutdown = irq_shutdown_none,
+    .enable = irq_enable_none,
+    .disable = irq_disable_none,
+};
+
+int arch_init_one_irq_desc(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_state_dump(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domain.c */
+
+DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
+unsigned long __per_cpu_offset[NR_CPUS];
+
+void context_switch(struct vcpu *prev, struct vcpu *next)
+{
+    BUG_ON("unimplemented");
+}
+
+void continue_running(struct vcpu *same)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_local_execstate(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_vcpu_execstate(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void startup_cpu_idle_loop(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_domain_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void dump_pageframe_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_vcpu_struct(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_create(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_vcpu_destroy(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_switch_to_aarch64_mode(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_create(struct domain *d,
+                       struct xen_domctl_createdomain *config,
+                       unsigned int flags)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_teardown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_destroy(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_shutdown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_pause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_unpause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_soft_reset(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_creation_finished(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_set_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_initialise_vcpu(struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_reset(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int domain_relinquish_resources(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_domain_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_vcpu_info(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_mark_events_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_update_evtchn_irq(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_block_unless_event_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_kick(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+struct domain *alloc_domain_struct(void)
+{
+    BUG_ON("unimplemented");
+}
+
+struct vcpu *alloc_vcpu_struct(const struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long
+hypercall_create_continuation(unsigned int op, const char *format, ...)
+{
+    BUG_ON("unimplemented");
+}
+
+int __init parse_arch_dom0_param(const char *s, const char *e)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guestcopy.c */
+
+unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long raw_copy_from_guest(void *to, const void __user *from,
+                                  unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* sysctl.c */
+
+long arch_do_sysctl(struct xen_sysctl *sysctl,
+                    XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
+{
+    BUG_ON("unimplemented");
+}
+
+/* p2m.c */
+
+int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
+{
+    BUG_ON("unimplemented");
+}
+
+int unmap_mmio_regions(struct domain *d,
+                       gfn_t start_gfn,
+                       unsigned long nr,
+                       mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int map_mmio_regions(struct domain *d,
+                     gfn_t start_gfn,
+                     unsigned long nr,
+                     mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+                          unsigned long gfn, mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+/* Return the size of the pool, in bytes. */
+int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
+{
+    BUG_ON("unimplemented");
+}
+
+/* delay.c */
+
+void udelay(unsigned long usecs)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guest_access.h */ 
+
+static inline unsigned long raw_clear_guest(void *to, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smpboot.c */
+
+int __cpu_up(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_disable(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_die(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/*
+ * The following functions are defined in common/irq.c, which will be built in
+ * the next commit, so these changes will be removed there.
+ */
+
+void cf_check irq_actor_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned int cf_check irq_startup_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c
index ccd3593f5a..ca56df75d8 100644
--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -4,6 +4,10 @@
  *
  * RISC-V Trap handlers
  */
+
+#include <xen/lib.h>
+#include <xen/sched.h>
+
 #include <asm/processor.h>
 #include <asm/traps.h>
 
@@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs)
 {
     die();
 }
+
+void vcpu_show_execution_state(struct vcpu *v)
+{
+    assert_failed("need to be implented");
+}
+
+void show_execution_state(const struct cpu_user_regs *regs)
+{
+    printk("implement show_execution_state(regs)\n");
+}
+
+void arch_hypercall_tasklet_result(struct vcpu *v, long res)
+{
+    assert_failed("need to be implented");
+}
+
+enum mc_disposition arch_do_multicall_call(struct mc_state *state)
+{
+    assert_failed("need to be implented");
+    return mc_continue;
+}
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 15:55:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 15:55:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676363.1052508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1Jn-0007s6-7g; Mon, 05 Feb 2024 15:55:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676363.1052508; Mon, 05 Feb 2024 15:55:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1Jn-0007rz-4h; Mon, 05 Feb 2024 15:55:35 +0000
Received: by outflank-mailman (input) for mailman id 676363;
 Mon, 05 Feb 2024 15:55:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rX1Jl-0007rr-Mb
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 15:55:33 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX1Jl-0000uB-Hq; Mon, 05 Feb 2024 15:55:33 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX1Jl-0003Vw-Cd; Mon, 05 Feb 2024 15:55:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=QKshiG3nxZwm8q/Fqxj+/eHcM0mPQ48QLY97q67MNpE=; b=UuyveCB53Oxut1PbRgARUpp2tU
	ED5RZMD8XJ/rPv/n7n64eSF/CGcpkRbjTGxrNdXcELaSDNOw9loU6g0AfXUGYSw8drBulBemqe6wB
	fH3ziszeeULfnlLY50n00UaQHbzkkneiJ7SaLp1nkcFkkt2lYKd2kznmQ6NfCKthaof4=;
Message-ID: <072da633-de82-4d93-b467-8372973406fe@xen.org>
Date: Mon, 5 Feb 2024 15:55:31 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/bitmap: Compile with -Wsign-conversion
Content-Language: en-GB
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: George Dunlap <George.Dunlap@citrix.com>, Jan Beulich
 <JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <20240205151454.1920291-1-andrew.cooper3@citrix.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240205151454.1920291-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Andrew,

Title: You seem to change common code. So s/x86//

On 05/02/2024 15:14, Andrew Cooper wrote:
> Use pragmas to able the warning in this file only.  All supported versions of
> Clang understand this, while older GCCs simply ignore it.
> 
> bitmap_find_free_region() is the only function which isn't sign-convert
> clean.  This highlights a latent bug in that it can't return successfully for
> a bitmap larger than 2G.
> 
> Add an extra check, and explicit cast to silence the warning.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Wei Liu <wl@xen.org>
> CC: Julien Grall <julien@xen.org>
> 
> Slightly RFC.  This is our first use of pragmas like this.

The only other approach I can think of is specifying the CFLAGS per file 
like Linux did. I don't know if our build system supports that though.

AFAICT, the only advantage would be to avoid duplicating the pragmas. So 
this is not a strong preference.

> ---
>   xen/common/bitmap.c | 7 +++++--
>   1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/common/bitmap.c b/xen/common/bitmap.c
> index 18e23e2f0e18..b14f8a3b3030 100644
> --- a/xen/common/bitmap.c
> +++ b/xen/common/bitmap.c
> @@ -14,6 +14,9 @@
>   #include <xen/lib.h>
>   #include <asm/byteorder.h>
>   
> +#pragma GCC diagnostic warning "-Wsign-conversion"
> +#pragma clang diagnostic warning "-Wsign-conversion"
> +

OOI, any reason why wasn't added at the right at the top of the file?

>   /*
>    * bitmaps provide an array of bits, implemented using an an
>    * array of unsigned longs.  The number of valid bits in a
> @@ -263,7 +266,7 @@ int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, unsigned i
>   	unsigned int pages = 1 << order;
>   	unsigned int i;
>   
> -	if(pages > BITS_PER_LONG)
> +	if (pages > BITS_PER_LONG || bits >= INT_MAX)
>   		return -EINVAL;
>   
>   	/* make a mask of the order */
> @@ -278,7 +281,7 @@ int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, unsigned i
>   		if((bitmap[index] & (mask << offset)) == 0) {
>   			/* set region in bimap */
>   			bitmap[index] |= (mask << offset);
> -			return i;
> +			return (int)i;

It took me a while to realize that this patch should be reviewed after 
"x86/bitmap: Even more signed-ness fixes".

Before hand, 'i' is a signed int and we would return -ENOMEM if 'bits' 
is negative. So I wonder whether the change in common/bitmap.c should 
belong to the other patch?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 16:03:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 16:03:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676371.1052519 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1Qu-0003Eb-Tv; Mon, 05 Feb 2024 16:02:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676371.1052519; Mon, 05 Feb 2024 16:02:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1Qu-0003EU-RB; Mon, 05 Feb 2024 16:02:56 +0000
Received: by outflank-mailman (input) for mailman id 676371;
 Mon, 05 Feb 2024 16:02:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rX1Qt-0003EO-7S
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 16:02:55 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX1Qs-0001dd-TD; Mon, 05 Feb 2024 16:02:54 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX1Qs-0003ua-Nm; Mon, 05 Feb 2024 16:02:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=b/0BApgWBiSv/AJ200Ww5kZHybmDRS0Q2pWanwftxhk=; b=yNejvuUVWBwjHh59xbOcVbIuF7
	PY8PwV93SJq1HTMtSiwmtaEiLavAo9RLrNg50T9YXn67ngJxZJhq8w0ALZ6usaSETXiiyqxkm9C3Z
	zY6AgNznbMCoeBZunAhZXWUTgQAW8A3xSVj6Oi0/8y6ochVRO7VS/3YWX2qJ1DMKTsuU=;
Message-ID: <6997ec6f-15a0-4c88-9a96-c7f4f450b2dd@xen.org>
Date: Mon, 5 Feb 2024 16:02:53 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/bitmap: Even more signed-ness fixes
Content-Language: en-GB
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: George Dunlap <George.Dunlap@citrix.com>, Jan Beulich
 <JBeulich@suse.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <20240205151413.1919983-1-andrew.cooper3@citrix.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240205151413.1919983-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Andrew,

title: s/x86// as this is common code.

On 05/02/2024 15:14, Andrew Cooper wrote:
> Further to commit 558e84b7ffec ("xen/bitmap: Consistently use unsigned bits
> values"), it turns out that GCC's range analysis isn't good enough to notice
> that 'bits / BITS_PER_LONG' is always within the unsigned range of k.

I would suggest to add the compiler version. This would be helpful if we 
need to revisit this decision in the future. Althougth, I don't expect 
to be the case here (switching the bitmap functions to unsigned is a 
good move and IIRC was discussed during the MISRA meeting).

> 
> As a consequence, it re-sign-extends 'k' when calculating the pointer holding
> 'bitmap[k]' on each loop iteration.  Switch all internal variables to unsigned
> int, which allows for better-still code generation.
> 
> Also clean up the prototypes for __bitmap_{set,clear} and the *_region()
> helpers, along with the internal variables.
> 
> Bloat-o-meter for a random x86 build reports:
>    add/remove: 0/0 grow/shrink: 0/10 up/down: 0/-277 (-277)
> 
> No functional change.

I am not sure about this...

> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

[...]

> -int bitmap_find_free_region(unsigned long *bitmap, int bits, int order)
> +int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, unsigned int order)
>   {
>   	unsigned long mask;
> -	int pages = 1 << order;
> -	int i;
> +	unsigned int pages = 1 << order;
> +	unsigned int i;

... I think your other patch is fixing a latent bug you introduced here. 
Before hand, if bits was "negative", we would return -ENOMEM. Now if we 
pass 2GB or higher, we would go through the loop.

So I would fold the hunk from common/bitmap.c here.

>   
>   	if(pages > BITS_PER_LONG)
>   		return -EINVAL;

[...]

> -int bitmap_allocate_region(unsigned long *bitmap, int pos, int order)
> +int bitmap_allocate_region(unsigned long *bitmap, unsigned int pos,
> +			   unsigned int order)
>   {
> -	int pages = 1 << order;
> +	unsigned int pages = 1 << order;
>   	unsigned long mask = (1ul << (pages - 1));
> -	int index = pos/BITS_PER_LONG;
> -	int offset = pos - (index * BITS_PER_LONG);
> +	unsigned int index = pos/BITS_PER_LONG; 
NIT: While you modify the line, can you add a space before after / as 
you did above?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 16:18:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 16:18:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676376.1052528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1fR-00077e-5D; Mon, 05 Feb 2024 16:17:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676376.1052528; Mon, 05 Feb 2024 16:17:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX1fR-00077X-2i; Mon, 05 Feb 2024 16:17:57 +0000
Received: by outflank-mailman (input) for mailman id 676376;
 Mon, 05 Feb 2024 16:17:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W3Sh=JO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rX1fP-00077R-UE
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 16:17:55 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1cfa0b6d-c442-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 17:17:53 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-40fd2f7ef55so9959165e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 08:17:53 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 iv15-20020a05600c548f00b0040efd192a95sm9143064wmb.1.2024.02.05.08.17.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 08:17:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1cfa0b6d-c442-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707149872; x=1707754672; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SFHnnRYvvg5yc0tc4E6hsQCgxM3uca7VhuRJZIGmQk0=;
        b=kbFBIiOddvN/tYo4hTO9s5Pp0MM/n08Z47xF4+7nsOO8acWSkUdBxfurPyNNDXJdvP
         kGgIrTJNvk4F04amWtjFGkcLD9IXZR32Eq2ZNyRP4VdH2RT7gzcVUpU7YjNvwEe9YC8Q
         aECyY1ctlxeCkjABwztcGeggs9ydL9T/aHL64=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707149872; x=1707754672;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SFHnnRYvvg5yc0tc4E6hsQCgxM3uca7VhuRJZIGmQk0=;
        b=xC3iwww5FXV7P3qtNNfJeLVtsM1br+m89PX8niEzBznSM7zqrkSl5gBXcCQQfRMH51
         6zvdIzKpaxuJDPuK1kXL3bhlRudscG+uhYfepUz54cNio34CHyFD+v917pdmdWcS2z06
         2AS5mJrtggjHOyEyfEaYyBqFwDTAyRVzFOlljsl1pxyN1u0L3gH46z7ia9Ebu9QMkKdn
         XejckiDYNxbUYZtDPX5n6T3+/TPeIVAS+eCAW24/VoburLLkqwllulSHM1QZx8lKR7q7
         AgY15ogxgi56bqIMdfh3zExIKQ5Jg9rTo6LmwFe9jB8oGzCzDqLVXm4unUWnb13/wqiU
         TJiQ==
X-Gm-Message-State: AOJu0YxX7xsgB2LNpfdaihuzcyO/wBavBRfzH/e6JDkfbgTWN8kzkJnc
	iMx8Z5SAN5/4MxbgUOI67T1YDQe65FFbMpBXg/ZTPuVWny/CnzsX2T7jwbsidiQ=
X-Google-Smtp-Source: AGHT+IEFE3O30I7b2utP5Ec0cHpJvsnTFE4pou9vjqlemwSzGuXijm8OW4ScQywqPWtHmsrzz9J5Fg==
X-Received: by 2002:a05:600c:4f04:b0:40f:d242:a2d2 with SMTP id l4-20020a05600c4f0400b0040fd242a2d2mr97981wmq.12.1707149872349;
        Mon, 05 Feb 2024 08:17:52 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCX75qYoglkKH4/8z/AgN3JxK2+6XW+3JX/nTTNiP1859+bH4mqcPgA3U9Nro0yCVkvzteJReDG/Ea7765bRL3ezv4LOJ1QuKpCBL/1fSnEpp1QGwXRsLEUB9qs=
Message-ID: <11c0cf19-fe89-4ea8-acd2-8f165969b1dc@citrix.com>
Date: Mon, 5 Feb 2024 16:17:51 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpu: Fix mixed tabs/spaces
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240205141441.1880188-1-andrew.cooper3@citrix.com>
 <cb716326-4a1c-44ce-8d00-ec46cb112242@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <cb716326-4a1c-44ce-8d00-ec46cb112242@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 05/02/2024 2:33 pm, Jan Beulich wrote:
> On 05.02.2024 15:14, Andrew Cooper wrote:
>> Fixes: 660f8a75013c ("x86/CPU: convert vendor hook invocations to altcall")
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Without the (or at least this very) Fixes: tag
> Acked-by: Jan Beulich <jbeulich@suse.com>
> The patch you mention didn't touch the padding at all. (It also didn't
> correct the prior inconsistency, but that's mainly because I didn't
> even notice.)

Oh, right.  I'll drop the fixes tag.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 16:52:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 16:52:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676382.1052539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2Cr-0006sK-UZ; Mon, 05 Feb 2024 16:52:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676382.1052539; Mon, 05 Feb 2024 16:52:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2Cr-0006sD-RD; Mon, 05 Feb 2024 16:52:29 +0000
Received: by outflank-mailman (input) for mailman id 676382;
 Mon, 05 Feb 2024 16:52:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Q1ha=JO=epam.com=prvs=8765dbd22b=oleksandr_tyshchenko@srs-se1.protection.inumbo.net>)
 id 1rX2Cq-0006rm-Ly
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 16:52:28 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f025c0b2-c446-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 17:52:26 +0100 (CET)
Received: from pps.filterd (m0174676.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id
 415F2mM8026052; Mon, 5 Feb 2024 16:52:21 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2050.outbound.protection.outlook.com [104.47.0.50])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3w2v44ssvf-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 05 Feb 2024 16:52:20 +0000 (GMT)
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com (2603:10a6:10:ed::15)
 by PAXPR03MB7869.eurprd03.prod.outlook.com (2603:10a6:102:210::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.34; Mon, 5 Feb
 2024 16:52:16 +0000
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167]) by DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167%4]) with mapi id 15.20.7249.032; Mon, 5 Feb 2024
 16:52:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f025c0b2-c446-11ee-8a46-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HZJ47ThzbMwGwcntNa7tFv2XcqQgiyASQ8/jfJxTRcbprxGOIG3cBnUQdE8peA6Qz2nENpzXroodqe97GMMXAVTrzq8WLivkScRkGNoI71x6YuZm0+NNR1zSctfly2Zoamv8iIqRLg4nx1dKQ7GhMmKUaHaY9oEqFgh2UqxTamok7mQecOpF0/lqrB7zM8ZuqoiAfBja9uPxVsqUPFw6gKii9buTxGv6v/Pb5MynY/fAqesqO9NgELImm1f0P1h9Tb9NLYhTqdNl3KsFKs7PCpaLg1dP/ETh53jF/3aJmvEFR6gR6w+3Deu8KNa1H+NJytiLrJsO8Dv4FMgVnAI3qg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=9hH4fDEl3SduQIsesqnjYXAGmusAaoKxOQK7fjzydhE=;
 b=ny452bJCAZZer+jwQcnuIZVXgf7c3YtaEwvdd1oAttfpXfNZJAass2TuIRYfb+Phq3cJz+ZHf33DyGaZ3+6/0pdFsXww6GZgcmfWgCMwq5+mbq0LCxJTGGpSv1r5a+vrwByJ5Qxai6y6EatmkyXkJAmGrU6voFGukAwRnpZ6YCVrSQOweNv6OZ058EMo57/6RLtWXBgKLpx8QN3fgPTDRDMKXsV4wr63Cyus7aKZORUQH1FLDmGTPmgd3I6r52qNsR6ubaV2sMPcsvDnljR8vQTI/vmyKdTepdOy6CAXuew9+x2TP/TFMiNgNfyGGN+mIY2H1wPOmYmGALcMWmXHxQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=9hH4fDEl3SduQIsesqnjYXAGmusAaoKxOQK7fjzydhE=;
 b=na/LV8pogA6trEsLxAH5ZVgQhMfQ9y0CZLMrH8+PjNdXSH0ngGmQ8Bcu3PTAfI3KzXkykerqOpMGv8sSkUJW3+9/J9OZBOTOjbe7cC93t6oePBxc/R78ijBDykErmEop86p9i8QNhClb+RDRv8s4gy+Yoy4QMQrIjk9NpOQEK4VwOCWLXMj5W3Jo7N+uj48dxxmp3oJg3KmtTsBLrc6qZlbquF4JnBCnjizh0dR3qDKflRr0IOKKYV8noSXakl7yw/RG/W+4GXN30hEd/t2/D2ixB5wRZT0y/yFPx8O0iz/YxhB7lRr3BMLy37ySDoOBe819pys/jq1wSdzHWfd4wg==
From: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
To: Anthony PERARD <anthony.perard@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Wei Liu
	<wl@xen.org>, Oleksandr Tyshchenko <olekstysh@gmail.com>,
        George Dunlap
	<george.dunlap@citrix.com>,
        Nick Rosbrook <rosbrookn@gmail.com>, Juergen
 Gross <jgross@suse.com>,
        Viresh Kumar <viresh.kumar@linaro.org>
Subject: Re: [PATCH] libxl: Add "grant_usage" parameter for virtio disk
 devices
Thread-Topic: [PATCH] libxl: Add "grant_usage" parameter for virtio disk
 devices
Thread-Index: AQHaVcWXJV0m8uwkbEKeSgAhd5N6SLD734GAgAAcRoA=
Date: Mon, 5 Feb 2024 16:52:16 +0000
Message-ID: <78bc7890-cfcb-4de4-a676-a3c2bbbb6d9f@epam.com>
References: <20240202104903.1112772-1-olekstysh@gmail.com>
 <ace5b3c8-38e7-4661-9401-ac9ac77a5e9f@perard>
In-Reply-To: <ace5b3c8-38e7-4661-9401-ac9ac77a5e9f@perard>
Accept-Language: en-US, ru-RU
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DB8PR03MB6108:EE_|PAXPR03MB7869:EE_
x-ms-office365-filtering-correlation-id: bc5096ca-edd4-4a4b-6e30-08dc266acf52
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 P9vWiNHEcycHqOaM5Z8ON4FvJcAXl72xIVtXaB/7bzvQZR9PSQuk6sZHn4hY42mEtzPMlW3L1Y68k6zB3oFVYBX9hMD1E6TSzVp3xbnfacexYzlwiuB4JMBNpIv1l6Qh5nfE/MwbvTTEMArBdPNiyclKEb0oLadZ0gTgaRm4rggFBLyhFyIdA6VVUPy6wrYfWqMVMatPVtwSDrS5uwV03A1afsckFgvFVfZweumWhap7bAq+4rCi7Eo2coF95nbmnm68zdR8ywfjVmhi52lGaJs9E4bxZEgqccKbYZYjcTPywzvHr2cZTq/nqCyCZy5PC/Nk7kkKAc1xS9Innzsl8TegkCJaWhYGEpEKxS/6/hWPSQ31ezig4d+4qy5IUN3ii/IIM/24TEzoitV8bDzbQWaMtNa5fyia9nPY/TPbZrtL8ito2s0BF1GUdYGts85p8UgqoNgqMN1kG0HVB6W1hFM7nWDvIK3tNmeXxxjFWt36HaPOK0+bq9TIOSuHyvm8fVTIw7iS2IW/QLVBFxJTfjOSCMQAvVSUBq1RYTLpgXcSF0qNjHX0pegk5pYXKPSnZDIlhnVu4aOioXZLidAL7xJuOy3cXh5Hp1R0ggvTvOezx2T9lin9LQcLXGrh+2hPhfhmV6ziPHPK9Un1xpVEQg==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR03MB6108.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(396003)(136003)(39860400002)(376002)(346002)(366004)(230922051799003)(1800799012)(186009)(451199024)(64100799003)(41300700001)(38070700009)(6916009)(54906003)(66556008)(64756008)(66446008)(66476007)(66946007)(76116006)(36756003)(316002)(2906002)(5660300002)(8676002)(8936002)(4326008)(83380400001)(31686004)(38100700002)(122000001)(71200400001)(6506007)(6512007)(53546011)(478600001)(6486002)(2616005)(26005)(86362001)(31696002)(45980500001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?dDJZWlcvVEU4cEo5QnhTblRzZi9YU2drejZJVVUySlJMV2EzRVMyZmdlY2s0?=
 =?utf-8?B?SGxWRXdHZ05ESzRLODFDQXR2N282NUJNQnJ6Vkg1K0FweTJ2b1hxTVFUQTg2?=
 =?utf-8?B?cEovM2dhQzBwUFJrZDAzL2F6ZUE1OERmMnFRUGEvQW03MCt3TVlQbjhwYzB5?=
 =?utf-8?B?eThpckRwY3lmSnVEaVJXNDN4SEprU1p3bndiV0kvemkraFk2LzdlSS83Wlo0?=
 =?utf-8?B?ZVdNaDZ4REZyNXdMK2tObHRQbkwraXBWS3lsSjM2ZTNVd2JtUnRvalBNWTBF?=
 =?utf-8?B?bDZJK0VkSEhBYjFCUnI0dFc4SUpWVmZzVHZmMVhoN2ZFMWtreXkwazhPQUVC?=
 =?utf-8?B?cExiYVpJTDV5KzhsaHZFcmZ0TUhSaHBQMWJLWjhvdVNyNHlES3JQZXVYLy83?=
 =?utf-8?B?djUwalZlczBZMklzYkNEbFBCZ2lteGhoOGVvR2cycnZubW9xckNsTXlnODUy?=
 =?utf-8?B?aXFteHJURnhmbVNEQ0Rtc09NSVBGVlVSVzBBM1prUVhPcWsvQnFpZ0FNWHdU?=
 =?utf-8?B?Y2FnMEFWcnJZblFNQ013TzFpQkRsbHhNRlBHMnVISkQ3SUZhZjBnL3FWM3Vp?=
 =?utf-8?B?N2tQaW9GS3VuQ1ozSWFmcWxhUld3clFUR09GWlZTYlFyRlZkcEsyUDZiRDh1?=
 =?utf-8?B?dG5JeiswT2ZZZE93QlBGQzI0TTNpbWY4ZXFoUUM1MzRTVE1HaFEzeVZDdTBB?=
 =?utf-8?B?ZVR2MGlNUFRaNTRnR3pnVmtGOW85a21tMDlNdzl4ak5zL1VyTFNIdStrRm1F?=
 =?utf-8?B?U2JRczNndUZxcE5MRE9OTnAzWjZRb2laSlhYY3Z4dy9zRVp2ellKM09MRE5R?=
 =?utf-8?B?NUFEcUhOV2RTcW9BVjNva2I5QVlmckhubnN4WHVxK3ZjWkRHd2ltdE1DUTRr?=
 =?utf-8?B?VHFVY0RmdlRucjZLWGNwWFVFNEJRcFNrR3BKa2VxRUNRa1IzZkZaMGxzVG05?=
 =?utf-8?B?MU5DMWtSVWVSMmJDc0VpN0toMUdkaTZEYkhvWTZ6REZEakg3cGNBcUcra09i?=
 =?utf-8?B?YU53TzdpWCtKcHhYSkx2OTBvTVZXRnloTCtiZ25xdGVxTGNQKzhzVjFsU0Jk?=
 =?utf-8?B?MHc5YU44UUp4d09kZCtYbzV1L3hYcUloR1JISU56L1V0MkpVeHZ4bFlDR01x?=
 =?utf-8?B?Mlc1Y0I2YU04WWloME94QkwvNFlVY2lEVmtsUjZXM1N2bUpqTWFEWmFUMHZC?=
 =?utf-8?B?aThzV1dpancrYVc3Mk9MRmpyRFFMYkUyUFQ4aUJtVDU4VUVQN3RUWEpyMFMy?=
 =?utf-8?B?a1lDRFNieWprKzlQaVNYTmd0cTE1S2FWK0Y0djJwODEycmF3b2dwbnAzd2Jj?=
 =?utf-8?B?azQ2S1N6OUU4ak1LM2Y3ZXJIYTBqVndzTXgrb1JUbXZGekdRRVg4Vjc1OVho?=
 =?utf-8?B?Rk9NZjdkUHJ0MHlEN1hweHI1WUdUQWRkcVF5SVZwQVZkbTFoRG02OEpSSEQv?=
 =?utf-8?B?Q2Yxb1g3R2VkSnplT0dQUmR3WVdGMFpnRHVIeG4yMll1Zlc2bVQwYUJEQVdR?=
 =?utf-8?B?VlNibE1Gb2lLaTUwd3Q4WVZsYWpBaVhNdkJ4RHRjK011a0diUWt0TWdxb01k?=
 =?utf-8?B?YlBOVnhPVWRoejY0NDVpa3lpb0N6NDYwWXNVaXo4UjZjcU52bUM3Q2VNT0pQ?=
 =?utf-8?B?cTVWaDl6S09xUXpQYnM1NTJUZWtNa2lhNWhOVWJLYnA2aERwV09ScFk5TmZW?=
 =?utf-8?B?SmY4Vi9QblJJNnZ0WkNmMUdoaWJZOFRQV2FPNkdGWU9mS1o3VmVNSjM3ZXBU?=
 =?utf-8?B?Nmh0S1RYWm1oaHg1ZDc3NjBkM09FSDVuMVNrY2hsc1VQVmk1cDVoVHNYVlZl?=
 =?utf-8?B?M0pRMGxucVlrOUlKUFIzS0NQcXQvMWNGNkVQNUhHV0IxSDdSNkJ2VUJPNjUv?=
 =?utf-8?B?ZXc2VjVNTHRqRTZneDk4SUdnWjQvVSttOURhMjU1Q3hjU1FmVytscXBUNWxR?=
 =?utf-8?B?cTFpdEpBT05VSWxQQnB4YnNLdVFHQ1lBaTBmTi9WUC9CNXUzbXJwc0lwNVk4?=
 =?utf-8?B?U3JHakxPTGQ0dUFQNWprdm4yL3FoOXJuVFhtQXlCWmxKR3J1ZTQ5YitBRjg2?=
 =?utf-8?B?R1dVay94RmxQSlRlNmhmYlIwV1pENnl3WWtVeUhGeDc3NHJrREl2ZDlLRGN6?=
 =?utf-8?B?RGxYQkxnSzVzR1kxckV6cjhBK0pSL3hPbFlpYk5vaFJyT1ZaTFVlVy9xclY5?=
 =?utf-8?Q?FPoTJ7giIUKeJBbnMONXC6E=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <B1F6A9E794E0F7409EFA6FB155E95365@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB8PR03MB6108.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bc5096ca-edd4-4a4b-6e30-08dc266acf52
X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Feb 2024 16:52:16.5862
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: iLiescpjSUCMZcF7Pa53V622f/WKqISGN+OTkcLrBocLKkGbwnV+0QK6piNOBBFakJ9KdXG77iqhaU0WNhpY/zjbHBzP5piSooIgn8Hj4fM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR03MB7869
X-Proofpoint-GUID: AuFMKlunBdAuBreQXdJACUFPx8l_qjOn
X-Proofpoint-ORIG-GUID: AuFMKlunBdAuBreQXdJACUFPx8l_qjOn
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-05_10,2024-01-31_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 suspectscore=0
 mlxscore=0 malwarescore=0 phishscore=0 adultscore=0 mlxlogscore=999
 bulkscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2401310000 definitions=main-2402050127

DQoNCk9uIDA1LjAyLjI0IDE3OjEwLCBBbnRob255IFBFUkFSRCB3cm90ZToNCg0KSGVsbG8gQW50
aG9ueQ0KDQoNCj4gT24gRnJpLCBGZWIgMDIsIDIwMjQgYXQgMTI6NDk6MDNQTSArMDIwMCwgT2xl
a3NhbmRyIFR5c2hjaGVua28gd3JvdGU6DQo+PiBkaWZmIC0tZ2l0IGEvdG9vbHMvbGlicy91dGls
L2xpYnhsdV9kaXNrX2wubCBiL3Rvb2xzL2xpYnMvdXRpbC9saWJ4bHVfZGlza19sLmwNCj4+IGlu
ZGV4IDZkNTNjMDkzYTMuLmYzN2RkNDQzYmQgMTAwNjQ0DQo+PiAtLS0gYS90b29scy9saWJzL3V0
aWwvbGlieGx1X2Rpc2tfbC5sDQo+PiArKysgYi90b29scy9saWJzL3V0aWwvbGlieGx1X2Rpc2tf
bC5sDQo+PiBAQCAtMjIwLDYgKzIyMCw5IEBAIGhpZGRlbi1kaXNrPVteLF0qLD8JeyBTVFJJUCgn
LCcpOyBTQVZFU1RSSU5HKCJoaWRkZW4tZGlzayIsIGhpZGRlbl9kaXNrLCBGUk9NRVFVDQo+PiAg
IHRydXN0ZWQsPwkJeyBsaWJ4bF9kZWZib29sX3NldCgmRFBDLT5kaXNrLT50cnVzdGVkLCB0cnVl
KTsgfQ0KPj4gICB1bnRydXN0ZWQsPwkJeyBsaWJ4bF9kZWZib29sX3NldCgmRFBDLT5kaXNrLT50
cnVzdGVkLCBmYWxzZSk7IH0NCj4+ICAgDQo+PiArZ3JhbnRfdXNhZ2U9MSw/CQl7IGxpYnhsX2Rl
ZmJvb2xfc2V0KCZEUEMtPmRpc2stPmdyYW50X3VzYWdlLCB0cnVlKTsgfQ0KPj4gK2dyYW50X3Vz
YWdlPTAsPwkJeyBsaWJ4bF9kZWZib29sX3NldCgmRFBDLT5kaXNrLT5ncmFudF91c2FnZSwgZmFs
c2UpOyB9DQo+IA0KPiBGb3Igb3RoZXIgYm9vbGVhbiB0eXBlIGZvciB0aGUgZGlzaywgd2UgaGF2
ZSAidHJ1c3RlZC91bnRydXN0ZWQiLA0KPiAiZGlzY2FyZC9uby1kaXNjYXJkIiwgImRpcmVjdC1p
by1zYXZlLyIsIGJ1dCB5b3UgYXJlIGFkZGluZw0KPiAiZ3JhbnRfdXNhZ2U9MS9ncmFudF91c2Fn
ZT0wIi4gSXMgdGhhdCBmaW5lPyBCdXQgSSBndWVzcyBoYXZpbmcgdGhlIG5ldw0KPiBvcHRpb24g
c3BlbGxlZCAiZ3JhbnRfdXNhZ2UiIG1pZ2h0IGJlIGJldHRlciwgc28gaXQgbWF0Y2ggdGhlIG90
aGVyDQo+IHZpcnRpbyBkZXZpY2VzIGFuZCB0aGUgaW1wbGVtZW50YXRpb24uIA0KDQoNClllcywg
SSBub3RpY2VkIHRoYXQgaG93IGJvb2xlYW5zIGFyZSBkZXNjcmliZWQgZm9yIHRoZSBkaXNrLiBJ
IGRlY2lkZWQgDQp0byB1c2UgdGhlIHNhbWUgcmVwcmVzZW50YXRpb24gb2YgdGhpcyBvcHRpb24g
YXMgaXQgd2FzIGFscmVhZHkgdXNlZCBmb3IgDQp2aXJ0aW89Wy4uLl0uIEJ1dCBJIHdvdWxkIGJl
IG9rIHdpdGggb3RoZXIgdmFyaWFudHMgLi4uDQoNCg0KQnV0IG1heWJlDQo+ICJ1c2UtZ3JhbnQv
bm8tdXNlLWdyYW50IiBtaWdodCBiZSBvaz8NCg0KICAgLi4uIGxpa2UgdGhhdCwgYnV0IHByZWZl
cmFibHkgd2l0aCBsZWF2aW5nIGxpYnhsX2RldmljZV9kaXNrJ3MgZmllbGQgDQpuYW1lZCAiZ3Jh
bnRfdXNhZ2UiIChpZiBubyBvYmplY3Rpb24pLg0KDQoNCj4gDQo+IEluIGFueSBjYXNlLCB0aGUg
aW1wbGVtZW50YXRpb24gbmVlZCB0byBtYXRjaCB0aGUgZG9jdW1lbnRhdGlvbiwgYW5kDQo+IHZp
Y2UgdmVyc2EuIFNlZSBiZWxvdy4NCg0KDQpTdXJlLg0KDQoNCj4gDQo+PiBkaWZmIC0tZ2l0IGEv
ZG9jcy9tYW4veGwtZGlzay1jb25maWd1cmF0aW9uLjUucG9kLmluIGIvZG9jcy9tYW4veGwtZGlz
ay1jb25maWd1cmF0aW9uLjUucG9kLmluDQo+PiBpbmRleCBiYzk0NWNjNTE3Li4zYzAzNTQ1NmQ1
IDEwMDY0NA0KPj4gLS0tIGEvZG9jcy9tYW4veGwtZGlzay1jb25maWd1cmF0aW9uLjUucG9kLmlu
DQo+PiArKysgYi9kb2NzL21hbi94bC1kaXNrLWNvbmZpZ3VyYXRpb24uNS5wb2QuaW4NCj4+IEBA
IC00MDQsNiArNDA0LDMxIEBAIFZpcnRpbyBmcm9udGVuZCBkcml2ZXIgKHZpcnRpby1ibGspIHRv
IGJlIHVzZWQuIFBsZWFzZSBub3RlLCB0aGUgdmlydHVhbA0KPj4gKz1pdGVtIEI8Z3JhbnRfdXNh
Z2U9Qk9PTEVBTj4NCj4+DQo+PiArPW92ZXIgNA0KPj4gKw0KPj4gKz1pdGVtIERlc2NyaXB0aW9u
DQo+PiArDQo+PiArU3BlY2lmaWVzIHRoZSB1c2FnZSBvZiBYZW4gZ3JhbnRzIGZvciBhY2Nlc3Np
bmcgZ3Vlc3QgbWVtb3J5LiBPbmx5IGFwcGxpY2FibGUNCj4+ICt0byBzcGVjaWZpY2F0aW9uICJ2
aXJ0aW8iLg0KPj4gKw0KPj4gKz1pdGVtIFN1cHBvcnRlZCB2YWx1ZXMNCj4+ICsNCj4+ICtJZiB0
aGlzIG9wdGlvbiBpcyBCPHRydWU+LCB0aGUgWGVuIGdyYW50cyBhcmUgYWx3YXlzIGVuYWJsZWQu
DQo+PiArSWYgdGhpcyBvcHRpb24gaXMgQjxmYWxzZT4sIHRoZSBYZW4gZ3JhbnRzIGFyZSBhbHdh
eXMgZGlzYWJsZWQuDQo+IA0KPiBVbmZvcnR1bmF0ZWx5LCB0aGlzIGlzIHdyb25nLCB0aGUgaW1w
bGVtZW50YXRpb24gaW4gdGhlIHBhdGNoIG9ubHkNCj4gc3VwcG9ydCB0d28gdmFsdWVzOiAxIC8g
MCwgbm90aGluZyBlbHNlLCBhbmQgdHJ5aW5nIHRvIHdyaXRlICJ0cnVlIiBvcg0KPiAiZmFsc2Ui
IHdvdWxkIGxlYWQgdG8gYW4gZXJyb3IuIChXZWxsIGFjdHVhbGx5IGl0J3MgImdyYW50X3VzYWdl
PTEiIG9yDQo+ICJncmFudF91c2FnZT0wIiwgdGhlcmUncyBub3RoaW5nIHRoYXQgY3V0IHRoYXQg
c3RyaW5nIGF0IHRoZSAnPScuKQ0KDQoNCllvdSBhcmUgcmlnaHQsIG9ubHkgMSAvIDAgY2FuIGJl
IHNldCB1bmxpa2UgZm9yIHZpcnRpbz1bLi4uXSB3aGljaCBzZWVtcyANCmhhcHB5IHdpdGggZmFs
c2UvdHJ1ZS4NCg0KDQo+IA0KPiBBbHNvLCBkbyB3ZSByZWFsbHkgbmVlZCB0aGUgZXh0cmEgdmVy
YmFsIGRlc2NyaXB0aW9uIG9mIGVhY2ggdmFsdWUgaGVyZT8NCj4gSXMgc2ltcGx5IGhhdmluZyB0
aGUgZm9sbG93aW5nIHdvdWxkIGJlIGVub3VnaD8NCj4gDQo+ICAgICAgPWl0ZW0gU3VwcG9ydGVk
IHZhbHVlcw0KPiANCj4gICAgICAxLCAwDQo+IA0KPiBUaGUgZGVzY3JpcHRpb24gaW4gIkRlc2Ny
aXB0aW9uIiBzZWN0aW9uIHdvdWxkIGhvcGVmdWxseSBiZSBlbm91Z2guDQoNCg0KSSB0aGluaywg
dGhpcyBtYWtlcyBzZW5zZS4NCg0KU28sIHNoYWxsIEkgbGVhdmUgImdyYW50X3VzYWdlPTEvZ3Jh
bnRfdXNhZ2U9MCIgb3IgdXNlIHByb3Bvc2VkIG9wdGlvbiANCiJ1c2UtZ3JhbnQvbm8tdXNlLWdy
YW50Ij8NCg0KDQo+IA0KPj4gKz1pdGVtIE1hbmRhdG9yeQ0KPj4gKw0KPj4gK05vDQo+PiArDQo+
PiArPWl0ZW0gRGVmYXVsdCB2YWx1ZQ0KPj4gKw0KPj4gK0lmIHRoaXMgb3B0aW9uIGlzIG1pc3Np
bmcsIHRoZW4gdGhlIGRlZmF1bHQgZ3JhbnQgc2V0dGluZyB3aWxsIGJlIHVzZWQsDQo+PiAraS5l
LiBlbmFibGUgZ3JhbnRzIGlmIGJhY2tlbmQtZG9taWQgIT0gMC4NCj4+ICsNCj4+ICs9YmFjaw0K
Pj4gKw0KPj4gZGlmZiAtLWdpdCBhL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfZGlzay5jIGIvdG9v
bHMvbGlicy9saWdodC9saWJ4bF9kaXNrLmMNCj4+IGluZGV4IGVhMzYyM2RkNmYuLmYzOWY0Mjcw
OTEgMTAwNjQ0DQo+PiAtLS0gYS90b29scy9saWJzL2xpZ2h0L2xpYnhsX2Rpc2suYw0KPj4gKysr
IGIvdG9vbHMvbGlicy9saWdodC9saWJ4bF9kaXNrLmMNCj4+IEBAIC0xODEsNiArMTgxLDkgQEAg
c3RhdGljIGludCBsaWJ4bF9fZGV2aWNlX2Rpc2tfc2V0ZGVmYXVsdChsaWJ4bF9fZ2MgKmdjLCB1
aW50MzJfdCBkb21pZCwNCj4+ICAgICAgICAgICAgICAgcmV0dXJuIEVSUk9SX0lOVkFMOw0KPj4g
ICAgICAgICAgIH0NCj4+ICAgICAgICAgICBkaXNrLT50cmFuc3BvcnQgPSBMSUJYTF9ESVNLX1RS
QU5TUE9SVF9NTUlPOw0KPj4gKw0KPj4gKyAgICAgICAgbGlieGxfZGVmYm9vbF9zZXRkZWZhdWx0
KCZkaXNrLT5ncmFudF91c2FnZSwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBkaXNrLT5iYWNrZW5kX2RvbWlkICE9IExJQlhMX1RPT0xTVEFDS19ET01JRCk7DQo+PiAgICAg
ICB9DQo+PiAgIA0KPj4gICAgICAgaWYgKGhvdHBsdWcgJiYgZGlzay0+c3BlY2lmaWNhdGlvbiA9
PSBMSUJYTF9ESVNLX1NQRUNJRklDQVRJT05fVklSVElPKSB7DQo+PiBAQCAtNDI5LDYgKzQzMiw4
IEBAIHN0YXRpYyB2b2lkIGRldmljZV9kaXNrX2FkZChsaWJ4bF9fZWdjICplZ2MsIHVpbnQzMl90
IGRvbWlkLA0KPj4gICAgICAgICAgICAgICBmbGV4YXJyYXlfYXBwZW5kKGJhY2ssIGxpYnhsX19k
ZXZpY2VfZGlza19zdHJpbmdfb2ZfdHJhbnNwb3J0KGRpc2stPnRyYW5zcG9ydCkpOw0KPj4gICAg
ICAgICAgICAgICBmbGV4YXJyYXlfYXBwZW5kX3BhaXIoYmFjaywgImJhc2UiLCBHQ1NQUklOVEYo
IiUiUFJJdTY0LCBkaXNrLT5iYXNlKSk7DQo+PiAgICAgICAgICAgICAgIGZsZXhhcnJheV9hcHBl
bmRfcGFpcihiYWNrLCAiaXJxIiwgR0NTUFJJTlRGKCIldSIsIGRpc2stPmlycSkpOw0KPj4gKyAg
ICAgICAgICAgIGZsZXhhcnJheV9hcHBlbmRfcGFpcihiYWNrLCAiZ3JhbnRfdXNhZ2UiLA0KPj4g
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsaWJ4bF9kZWZib29sX3ZhbChkaXNr
LT5ncmFudF91c2FnZSkgPyAiMSIgOiAiMCIpOw0KPj4gICAgICAgICAgIH0NCj4+ICAgDQo+PiAg
ICAgICAgICAgZmxleGFycmF5X2FwcGVuZChmcm9udCwgImJhY2tlbmQtaWQiKTsNCj4+IEBAIC02
MjMsNiArNjI4LDE0IEBAIHN0YXRpYyBpbnQgbGlieGxfX2Rpc2tfZnJvbV94ZW5zdG9yZShsaWJ4
bF9fZ2MgKmdjLCBjb25zdCBjaGFyICpsaWJ4bF9wYXRoLA0KPj4gICAgICAgICAgICAgICBnb3Rv
IGNsZWFudXA7DQo+PiAgICAgICAgICAgfQ0KPj4gICAgICAgICAgIGRpc2stPmlycSA9IHN0cnRv
dWwodG1wLCBOVUxMLCAxMCk7DQo+PiArDQo+PiArICAgICAgICB0bXAgPSBsaWJ4bF9feHNfcmVh
ZChnYywgWEJUX05VTEwsDQo+PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICBHQ1NQUklO
VEYoIiVzL2dyYW50X3VzYWdlIiwgbGlieGxfcGF0aCkpOw0KPj4gKyAgICAgICAgaWYgKCF0bXAp
IHsNCj4+ICsgICAgICAgICAgICBMT0coRVJST1IsICJNaXNzaW5nIHhlbnN0b3JlIG5vZGUgJXMv
Z3JhbnRfdXNhZ2UiLCBsaWJ4bF9wYXRoKTsNCj4+ICsgICAgICAgICAgICBnb3RvIGNsZWFudXA7
DQo+IA0KPiBJIHdvbmRlciBpZiBpdCdzIHN1Y2ggYSBnb29kIGlkZWEgdG8gbWFrZSB0aGlzIG5v
ZGUgbWFuZGF0b3J5LiBDb3VsZCB3ZQ0KPiBqdXN0IGFwcGx5IHRoZSBkZWZhdWx0IHZhbHVlIGlm
IHRoZSBwYXRoIGlzIG1pc3Npbmc/IEkgZG9uJ3QgdGhpbmsgdGhlDQo+IHZhbHVlIGlzIGdvaW5n
IHRvIGJlIHVzZWQgYW55d2F5IGJlY2F1c2UgSSBkb24ndCB0aGluayBmcm9tX3hlbnN0b3JlKCkg
aXMNCj4gdXNlZCBkdXJpbmcgZ3Vlc3QgY3JlYXRpb24sIGFuZCBpdCBsb29rcyBsaWtlICJncmFu
dF91c2FnZSIgaXMgb25seQ0KPiB1c2VmdWwgZHVyaW5nIGd1ZXN0IGNyZWF0aW9uLiBBbHNvLCB0
aGUgImdyYW50X3VzYWdlIiBub2RlIGlzbid0DQo+IG1hbmRhdG9yeSBpbiAibGlieGxfdmlydGlv
LmMiLCBzbyBubyBuZWVkIHRvIGRvIHNvbWV0aGluZyBkaWZmZXJlbnQNCj4gZm9yIGRpc2suDQoN
CkkgYWdyZWUgd2l0aCB5b3VyIGFuYWx5c2lzLCBubyBuZWVkIHRvIHJhaXNlIGFuIGVycm9yIGlm
IG1pc3NpbmcsIGxldCdzIA0KYXBwbHkgYSBkZWZhdWx0IHZhbHVlIHdoaWNoIGlzIHRoZSByZXN1
bHQgb2YgImRpc2stPmJhY2tlbmRfZG9taWQgIT0gDQpMSUJYTF9UT09MU1RBQ0tfRE9NSUQiLg0K
DQoNCj4gDQo+PiArICAgICAgICB9DQo+PiArICAgICAgICBsaWJ4bF9kZWZib29sX3NldCgmZGlz
ay0+Z3JhbnRfdXNhZ2UsIHN0cnRvdWwodG1wLCBOVUxMLCAwKSk7DQo+PiAgICAgICB9DQo+PiAg
IA0KPj4gICAgICAgZGlzay0+dmRldiA9IHhzX3JlYWQoY3R4LT54c2gsIFhCVF9OVUxMLA0KPiAN
Cj4gVGhhbmtzLA0KPiA=


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 16:53:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 16:53:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676392.1052549 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2E5-00083x-8q; Mon, 05 Feb 2024 16:53:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676392.1052549; Mon, 05 Feb 2024 16:53:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2E5-00083q-5G; Mon, 05 Feb 2024 16:53:45 +0000
Received: by outflank-mailman (input) for mailman id 676392;
 Mon, 05 Feb 2024 16:53:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+Hmw=JO=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rX2E4-00082P-7H
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 16:53:44 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1eb7b2c2-c447-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 17:53:43 +0100 (CET)
Received: by mail-wr1-x42c.google.com with SMTP id
 ffacd0b85a97d-3394ca0c874so3242352f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 08:53:43 -0800 (PST)
Received: from localhost.localdomain
 (cpc92320-cmbg19-2-0-cust35.5-4.cable.virginm.net. [82.13.64.36])
 by smtp.gmail.com with ESMTPSA id
 n8-20020a5d4c48000000b0033afe816977sm15271wrt.66.2024.02.05.08.53.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 08:53:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1eb7b2c2-c447-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707152022; x=1707756822; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=BdVtkLJg5fedcHtSRpvJksczWE/cqssw9PeeRwuokvo=;
        b=pjFUCueDGCNbJxP6RHwaBpzAPpNNDty/D6ipcwQs59qGER3a605YcXFdeJDriM6dM5
         kBAc+0R1AixenG+MpSKfA+k4jUK9yGhL/6f3ok4SU1FFp6mqLnAi9W2j+ogcHufiIe3+
         +aj/Dyy4DS/1+PGjV8+fLlgL1tEtJCU0nrfh8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707152022; x=1707756822;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=BdVtkLJg5fedcHtSRpvJksczWE/cqssw9PeeRwuokvo=;
        b=g2ke5kqrRQ1bBMKegc5UWLMdZBxvBnJO2dSRV9lECO1ZBvX24tlpKvqJixYl3AuPTy
         eDFYnFkX5dobzF2hslTE9LOgn1AvV2meeAwvjQXTSVAn1CDji8K+sniAPNX3jv3nyWTp
         EZcd4uPvnWcyD8RSBTuyyDbOl6wwAO2jIQ+zASH3lNw0cNvngLT5vNwF7e1KQ14+Y9cy
         WmozHnDwkl/Ddo4lbd4HgdKjMjd5jJaJ3PiEM2jYqPZg2ASXNKoQbYfyLkG1S7H7e7tl
         KJs3aSUykxL3mSFEooaCM6mDVLbKnEkq+w4FVC44I6ykfEQefW64/AQiRP1Fxy6pVDRu
         YGgg==
X-Gm-Message-State: AOJu0YwX3NUGHzTyHIPYqdamb/9SpmQXXKsclcZrkamNEhq/6c0Dh78H
	1hdoWXsqnp69Gy0dDfj2uKRdAQShwvFbWiq9Lki8HX4gIijVccsMrQlwVz2Cn/4tGHm2q5hyqZL
	o
X-Google-Smtp-Source: AGHT+IFM6k4OHVQk9zMrt5OiI+LAFFO4fPK7BpwB8rVwjwoLnolQq8TvkokNspvGRmZdfwykHDV19A==
X-Received: by 2002:a05:6000:1ca:b0:33a:fa97:7ca6 with SMTP id t10-20020a05600001ca00b0033afa977ca6mr60368wrx.10.1707152022011;
        Mon, 05 Feb 2024 08:53:42 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCU5iyW4Bb/yGL3EYlRlOIOZgSwWumXNfhjk4MaSt8SfjNMBOdRLWK7yJgIyEbFMZDkmf+CAbBimb9SVnnbGcf+R1TnHOOy/+5sYvnMyEos1+/mcM6k06H30y+rgU/nhrzD2RPcJqlNzpaQ=
From: Anthony PERARD <anthony.perard@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony.perard@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH] golang: Regen binding
Date: Mon,  5 Feb 2024 16:53:31 +0000
Message-Id: <20240205165331.18627-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Fixes: 024e7131be5c ("tools: don't expose XENFEAT_hvm_pirqs by default")
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/golang/xenlight/helpers.gen.go | 6 ++++++
 tools/golang/xenlight/types.gen.go   | 1 +
 2 files changed, 7 insertions(+)

diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 35e209ff1b..0f8e23773c 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1280,6 +1280,9 @@ return fmt.Errorf("converting field Rdm: %v", err)
 }
 x.RdmMemBoundaryMemkb = uint64(tmp.rdm_mem_boundary_memkb)
 x.McaCaps = uint64(tmp.mca_caps)
+if err := x.Pirq.fromC(&tmp.pirq);err != nil {
+return fmt.Errorf("converting field Pirq: %v", err)
+}
 return nil
 }
 
@@ -1613,6 +1616,9 @@ return fmt.Errorf("converting field Rdm: %v", err)
 }
 hvm.rdm_mem_boundary_memkb = C.uint64_t(tmp.RdmMemBoundaryMemkb)
 hvm.mca_caps = C.uint64_t(tmp.McaCaps)
+if err := tmp.Pirq.toC(&hvm.pirq); err != nil {
+return fmt.Errorf("converting field Pirq: %v", err)
+}
 hvmBytes := C.GoBytes(unsafe.Pointer(&hvm),C.sizeof_libxl_domain_build_info_type_union_hvm)
 copy(xc.u[:],hvmBytes)
 case DomainTypePv:
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 7907aa8999..9c8b7b81f6 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -651,6 +651,7 @@ SerialList StringList
 Rdm RdmReserve
 RdmMemBoundaryMemkb uint64
 McaCaps uint64
+Pirq Defbool
 }
 
 func (x DomainBuildInfoTypeUnionHvm) isDomainBuildInfoTypeUnion(){}
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Mon Feb 05 16:54:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 16:54:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676402.1052559 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2Ez-0000O8-II; Mon, 05 Feb 2024 16:54:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676402.1052559; Mon, 05 Feb 2024 16:54:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2Ez-0000O1-Er; Mon, 05 Feb 2024 16:54:41 +0000
Received: by outflank-mailman (input) for mailman id 676402;
 Mon, 05 Feb 2024 16:54:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W3Sh=JO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rX2Ex-0000Np-IM
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 16:54:39 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3f19a4e6-c447-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 17:54:37 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-5112a04c7acso7448402e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 08:54:37 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 d8-20020a5d4f88000000b0033ae50e2c6asm16490wru.83.2024.02.05.08.54.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 08:54:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f19a4e6-c447-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707152077; x=1707756877; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=BKfBRmbXv56Vn3QaN3CNW3o0CXzD4Qv3mgebMPPj7Oo=;
        b=ae0unsS+SiDATPUK5iAPMcHvKDi9pd+9dBi8N/+L4yy+fjC6DSoz+JGw6Eq9yyp7wz
         odWutN7CldY9sIABBNZosd3ZIAk1cNyNQgRky2YG8zXYg3X6swFV8FY17r/lCuSks+9h
         J4+QKt1ZhdLdRdjbXickovIo16AkjTJuUMfUU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707152077; x=1707756877;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=BKfBRmbXv56Vn3QaN3CNW3o0CXzD4Qv3mgebMPPj7Oo=;
        b=cB5Zl5ikwHzaovRiH/tSGb5c32yuqKnH4RD+H7mFq5TnLm9GRbjsKP6NmmorLF0hDm
         T+wrg+zEnRv48ub1MnPy2a+y8Ufpqgnwnd8FcRQiu1D59PGrtaq+eZ5JNudEpoZIbdOZ
         lquN0irXI/yXeMp4pDhi4l5ZKTks+J//r3JmfwRlnxLlvqUOYIiUM4ozNcZTI3+2oRQJ
         vxK9FXZm9Wa9l5i5AW+9LQQ7fXBUZASd/W4oaKPJYRbov9QYhrUPksmLAvGY6uScfLib
         bQBuGptRHPq17JWZKTiF5M2agGFSbSfddPb/nQgzLKhukXjWZrEJV+EUgp6ASbAr/dry
         swMA==
X-Gm-Message-State: AOJu0YyOM3BPZgj0HRBywXWayZsBCAKvZmwJGJuz6xJyITvkLRghklGj
	DMKaKUptaHlZFXpBEzRP65g8LpgoeCNoybde9XJMxsOMyK4Qc1sOUc70agiA8EU=
X-Google-Smtp-Source: AGHT+IGHieaEAPxp1ksOtGx0ekAWhDqlQJ2A0Tmv1KTVCerwzFX2qm5L0dDDbCJsChFSz+a0KR00ng==
X-Received: by 2002:ac2:5968:0:b0:511:4aa7:b5c2 with SMTP id h8-20020ac25968000000b005114aa7b5c2mr121413lfp.26.1707152076993;
        Mon, 05 Feb 2024 08:54:36 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUBB7FLeRZAwP9jaPEhOKVxuy0nbPf4WI951CpeKKFRXSZewC0WpQSZc/MczCBPeF+A2MXvRsLY/0ZkiNLZVcDdV1HO4Ch7CdupdOJwsMs40mPw9mPMEmRmQLJ/yzble1bzdzkIHJw5+2dM1+48abGZxO+tE00jRqmZxDYmNFrtbaojVq9c5sJEP+tGhDeoOpGDCb5FoYY2BlFVYlAKmDRFaJr/iwUPx7iMFApw3/+UN6Abw7f2u1eyntTcz8YABDh9SA==
Message-ID: <3fc56b77-9b8c-402f-a338-5f415e5ab603@citrix.com>
Date: Mon, 5 Feb 2024 16:54:36 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4] xen/arm: ffa: reclaim shared memory on guest
 destroy
Content-Language: en-GB
To: Jens Wiklander <jens.wiklander@linaro.org>, xen-devel@lists.xenproject.org
Cc: patches@linaro.org, Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
References: <20240205154909.2392231-1-jens.wiklander@linaro.org>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240205154909.2392231-1-jens.wiklander@linaro.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 05/02/2024 3:49 pm, Jens Wiklander wrote:
> diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
> index 0793c1c7585d..bbb6b819ee2b 100644
> --- a/xen/arch/arm/tee/ffa.c
> +++ b/xen/arch/arm/tee/ffa.c
> @@ -992,53 +1008,75 @@ static void put_shm_pages(struct ffa_shm_mem *shm)
>      }
>  }
>  
> -static bool inc_ctx_shm_count(struct ffa_ctx *ctx)
> +static bool inc_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
>  {
>      bool ret = true;
>  
>      spin_lock(&ctx->lock);
> +
> +    /*
> +     * If this is the first shm added, increase the domain reference
> +     * counter as we need to keep domain around a bit longer to reclaim the
> +     * shared memory in the teardown path.
> +     */
> +    if ( !ctx->shm_count )
> +        get_knownalive_domain(d);
> +
>      if (ctx->shm_count >= FFA_MAX_SHM_COUNT)
>          ret = false;
>      else
>          ctx->shm_count++;
> +
>      spin_unlock(&ctx->lock);

This is subtle.  It reads as if there is a reference leak.  There really
will be one if FFA_MAX_SHM_COUNT happens to be 0.

You could add a BUILD_BUG_ON(), but IMO it would be far clearer to
follow if you moved the get_knownalive_domain() into the else clause.

>  
>      return ret;
>  }
>  
> -static void dec_ctx_shm_count(struct ffa_ctx *ctx)
> +static void dec_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
>  {
>      spin_lock(&ctx->lock);
> +
>      ASSERT(ctx->shm_count > 0);
>      ctx->shm_count--;
> +
> +    /*
> +     * If this was the last shm removed, let go of the domain reference we
> +     * took in inc_ctx_shm_count() above.
> +     */
> +    if ( !ctx->shm_count )
> +        put_domain(d);
> +
>      spin_unlock(&ctx->lock);

You want a local bool called drop_ref, set within the lock, and move the
put_domain() down here.  put_domain() is potentially a large operation.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 17:18:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 17:18:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676449.1052568 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2bn-0008Tg-E0; Mon, 05 Feb 2024 17:18:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676449.1052568; Mon, 05 Feb 2024 17:18:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2bn-0008TZ-BX; Mon, 05 Feb 2024 17:18:15 +0000
Received: by outflank-mailman (input) for mailman id 676449;
 Mon, 05 Feb 2024 17:18:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rX2bm-0008TT-Ov
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 17:18:14 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX2bl-00033V-Nz; Mon, 05 Feb 2024 17:18:13 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX2bl-0004Yg-Ep; Mon, 05 Feb 2024 17:18:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=dtFtpwA9k+Tnq6W1cMFNl2l9PfphsVGGnDsjmQ4CZ+w=; b=QBRMHebTJVKTjLIdSE+xiptcP5
	aVtqNpO23BZNl7LryCBIjieB+z1lKnfrEfWEZoUFmZgeYYNr7UHwRQZDnZJ09KL3dEZl51S3iG3LK
	3jcoF+pjpjG0XnYAYjlJEd/Qv6CkCZ013oqQ7ATGd9FX9uz3kzP7dBZxlue9J6Q3CQZM=;
Message-ID: <c635e693-fe07-458c-9d58-273e2779f5b9@xen.org>
Date: Mon, 5 Feb 2024 17:18:11 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 00/32] tools: enable xenstore-stubdom to use 9pfs
Content-Language: en-GB
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Nick Rosbrook
 <rosbrookn@gmail.com>, Samuel Thibault <samuel.thibault@ens-lyon.org>
References: <20240205105001.24171-1-jgross@suse.com>
 <689635a7-6d34-44fe-b00b-31fdc03f6969@xen.org>
 <d4740502-795a-4e22-afe2-abd0746da843@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <d4740502-795a-4e22-afe2-abd0746da843@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Juergen,

On 05/02/2024 11:08, Jürgen Groß wrote:
> On 05.02.24 11:55, Julien Grall wrote:
>> Hi Juergen,
>>
>> On 05/02/2024 10:49, Juergen Gross wrote:
>>> This series is adding 9pfs support to Xenstore-stubdom, enabling it
>>> to do logging to a dom0 directory.
>>>
>>> This is a prerequisite for the final goal to add live update support
>>> to Xenstore-stubdom, as it enables the stubdom to store its state in
>>> a dom0 file.
>>>
>>> The 9pfs backend is a new daemon written from scratch. Using a
>>> dedicated 9pfs daemon has several advantages:
>>>
>>> - it is using much less resources than a full blown qemu process
>>> - it can serve multiple guests (the idea is to use it for other
>>>    infrastructure domains, like qemu-stubdom or driver domains, too)
>>> - it is designed to support several security enhancements, like
>>>    limiting the number of files for a guest, or limiting the allocated
>>>    file system space
>>> - it doesn't support file links (neither hard nor soft links) or
>>>    referencing parent directories via "..", minimizing the risk that
>>>    a guest can "escape" from its home directory
>>>
>>> Note that for now the daemon only contains the minimal needed
>>> functionality to do logging from Xenstore-stubdom. I didn't want to
>>> add all the 9pfs commands and security add-ons in the beginning, in
>>> order to avoid needless efforts in case the idea of the daemon is
>>> being rejected.
>>>
>>> Changes in V4:
>>> - patch 2 of V3 was applied
>>> - added support of reading directories
>>> - addressed review comments
>>>
>>> Changes in V3:
>>> - new patches 1, 23-25
>>> - addressed review comments
>>>
>>> Changes in V2:
>>> - support of multiple rings per device
>>> - xenlogd->xen-9pfsd rename
>>> - addressed review comments
>>> - fixed some bugs
>>>
>>> Juergen Gross (32):
>>>    tools: add access macros for unaligned data
>>>    tools: add a new xen logging daemon
>>>    tools/xen-9pfsd: connect to frontend
>>>    tools/xen-9pfsd: add transport layer
>>>    tools/xen-9pfsd: add 9pfs response generation support
>>>    tools/xen-9pfsd: add 9pfs version request support
>>>    tools/xen-9pfsd: add 9pfs attach request support
>>>    tools/xen-9pfsd: add 9pfs walk request support
>>>    tools/xen-9pfsd: add 9pfs open request support
>>>    tools/xen-9pfsd: add 9pfs clunk request support
>>>    tools/xen-9pfsd: add 9pfs create request support
>>>    tools/xen-9pfsd: add 9pfs stat request support
>>>    tools/xen-9pfsd: add 9pfs write request support
>>>    tools/xen-9pfsd: add 9pfs read request support
>>>    tools/libs/light: add backend type for 9pfs PV devices
>>>    tools/xl: support new 9pfs backend xen_9pfsd
>>>    tools/helpers: allocate xenstore event channel for xenstore stubdom
>>>    tools/xenstored: rename xenbus_evtchn()
>>>    stubdom: extend xenstore stubdom configs
>>>    tools: add 9pfs device to xenstore-stubdom
>>>    tools/xenstored: add early_init() function
>>>    tools/xenstored: move systemd handling to posix.c
>>>    tools/xenstored: move all log-pipe handling into posix.c
>>>    tools/xenstored: move all socket handling into posix.c
>>>    tools/xenstored: get own domid in stubdom case
>>>    tools/xenstored: rework ring page (un)map functions
>>>    tools/xenstored: split domain_init()
>>>    tools/xenstored: map stubdom interface
>>>    tools/xenstored: mount 9pfs device in stubdom
>>>    tools/xenstored: add helpers for filename handling
>>>    tools/xenstored: support complete log capabilities in stubdom
>>>    tools/xenstored: have a single do_control_memreport()
>>
>> I haven't checked what's the state of the 9PFS patches. Can part of 
>> the xenstored changes be committed without the 9PFS changes?
> 
> The following patches can go in without the 9pfs daemon:

It looks like the gitalb CI is not happy with the following patches [1]:

In function ‘free_stat’,
     inlined from ‘write_9pfs’ at 9pfront.c:935:9:
9pfront.c:120:14: error: ‘stat.name’ may be used uninitialized 
[-Werror=maybe-uninitialized]
   120 |     free(stat->name);
       |          ~~~~^~~~~~
9pfront.c: In function ‘write_9pfs’:
9pfront.c:929:20: note: ‘stat’ declared here
   929 |     struct p9_stat stat;
       |                    ^~~~

I think...

> 
> tools/helpers: allocate xenstore event channel for xenstore stubdom
> tools/xenstored: rename xenbus_evtchn()
> stubdom: extend xenstore stubdom configs

.. this is related to this patch. Can you have a look?

I have just pushed a new branch without this patch. Let see if the CI 
[2] will pass this time.

> tools/xenstored: add early_init() function
> tools/xenstored: move systemd handling to posix.c
> tools/xenstored: move all log-pipe handling into posix.c
> tools/xenstored: move all socket handling into posix.c
> tools/xenstored: get own domid in stubdom case
> tools/xenstored: rework ring page (un)map functions
> tools/xenstored: split domain_init()
> tools/xenstored: map stubdom interface

Cheers,

[1] https://gitlab.com/xen-project/people/julieng/xen/-/pipelines/1165147815
[2] https://gitlab.com/xen-project/people/julieng/xen/-/pipelines/1165166977

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 17:27:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 17:27:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676453.1052579 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2kc-0002qA-9B; Mon, 05 Feb 2024 17:27:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676453.1052579; Mon, 05 Feb 2024 17:27:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2kc-0002q3-6I; Mon, 05 Feb 2024 17:27:22 +0000
Received: by outflank-mailman (input) for mailman id 676453;
 Mon, 05 Feb 2024 17:27:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+Hmw=JO=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rX2kb-0002px-L4
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 17:27:21 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d030ca10-c44b-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 18:27:19 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a358ec50b7cso602417866b.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 09:27:19 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ti12-20020a170907c20c00b00a36fa497a65sm50901ejc.110.2024.02.05.09.27.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 09:27:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d030ca10-c44b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707154038; x=1707758838; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=awdyICZKqpWCHNwcWSP6CybyXKbC+Vyt3NLnV4CSWMQ=;
        b=lZ4Omjd/la8O5RByLi/XyKnlhN+51GFdW7YsRXJb9xGyxBu/qI34wLHn6rhdn8zhyV
         te3riLasY2hIQ2Qpv9hWc3KwqFv7+8irzFaPVRHUWiVxK+J405JxZk5VXo9DtUdzfx06
         0Ws3We3uBzvLmSOgiV+tsRVs808TuFBNSo4F8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707154038; x=1707758838;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=awdyICZKqpWCHNwcWSP6CybyXKbC+Vyt3NLnV4CSWMQ=;
        b=KI63/yOLgQSv9PDprpfygL8UsPJP9l3sU6fzp8AcbYZt5t66QT8Fte5KXeAHUmjXqI
         FqhAJU85/7e/ksk+EdpgXOgzC62iLnofI7Fd7wX3JwsYH3J1CLmJTUwi6aExE4eaxyfQ
         oIJ6F4xtArTeb7c49dG0vzaGV09/x+Y2K9boqx/zfw0sR/aRoNvALRI8XWaUyD9xQPyq
         zfkx4MJh4eB13/EXf9scK80//X2bUnbSKILXxRoPH1Zg4IlGf97oHengtxiOvpoIRXb9
         KfG2h3qja6eUM1aRAoBKDfllV2CrUVO00Bq2tMHfnYVTK66uLkt/Bs8VtK1BAnfTnb/E
         7Jww==
X-Gm-Message-State: AOJu0YwjTZmx6qszfcUtPcZUVpayarX/I6pe12nn0H4aaQKvZtT0A0KU
	txQHemhWJTWRjL1wMTMHlGM16O7uhEYwg7FTEMSNHZmDnI5s+b/qbwABBr1+n+8Kjci00QlI+D4
	P
X-Google-Smtp-Source: AGHT+IGmRA8eleUDlMIOzy/6BiQJrCEg5AazEKptofHP2EWYZYzrkni/pfSf4js5hALIFOGkwNwEww==
X-Received: by 2002:a17:907:1708:b0:a37:8424:3d0 with SMTP id le8-20020a170907170800b00a37842403d0mr343465ejc.9.1707154038624;
        Mon, 05 Feb 2024 09:27:18 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXF8xgmBSCcl2NU/7Xjg6Qw3pzBW81agg6w77+Qzj8VdOdAhst5UAxXUMo9iVqe9+HSn5I8TWk+5LaQ
Date: Mon, 5 Feb 2024 17:27:17 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Cyril =?iso-8859-1?Q?R=E9bert?= <slack@rabbit.lu>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] tools/xentop: add option to display dom0 first
Message-ID: <a8194dec-70f0-4f90-898d-f15152a5018b@perard>
References: <87846acd5b31991e38561c9765eb97730c79d0f3.1706723494.git.slack@rabbit.lu>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <87846acd5b31991e38561c9765eb97730c79d0f3.1706723494.git.slack@rabbit.lu>

On Wed, Jan 31, 2024 at 06:51:34PM +0100, Cyril Rbert wrote:
> Add a command line option to xentop to be able to display dom0 first, on top of the list.
> This is unconditional, so sorting domains with the S option will also ignore dom0.
> 
> Signed-off-by: Cyril Rbert (zithro) <slack@rabbit.lu>

Hi Cyril,

Your patch looks like a good idea, but xentop segv without '-z' now, when
there are guest running.

Revelant part of a backtrace:
#0  xenstat_domain_name (domain=0x121) at xenstat.c:344
344		return domain->name;
#6  0x00006344dd283651 in top () at xentop.c:1209
        i = 2
        num_domains = 2
        sort_start = 1
        sort_count = <optimized out>
        dom0_index = <optimized out>
1209		qsort((domains+sort_start), (num_domains-sort_count), sizeof(xenstat_domain *),
1210		      (int(*)(const void *, const void *))compare_domains);


Also, could you update the man page? Here "docs/man/xentop.1.pod""

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 17:31:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 17:31:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676459.1052590 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2oo-0005a8-QG; Mon, 05 Feb 2024 17:31:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676459.1052590; Mon, 05 Feb 2024 17:31:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2oo-0005a1-Lp; Mon, 05 Feb 2024 17:31:42 +0000
Received: by outflank-mailman (input) for mailman id 676459;
 Mon, 05 Feb 2024 17:31:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX2on-0005Zb-Ow; Mon, 05 Feb 2024 17:31:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX2on-0003KR-MI; Mon, 05 Feb 2024 17:31:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX2on-0002tC-6w; Mon, 05 Feb 2024 17:31:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rX2on-0004sl-6S; Mon, 05 Feb 2024 17:31:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hehxSH3KMk/Um7oVEAkHDBn3ivw/bExmB5KEfIK8rPU=; b=PQlGKjoxA9DmdwvhMbIuv9DvSH
	KnyMkWly7gmPTw0bkWDZxwoPTgQ2OlLdA0o/getw5wbeNEtoVjrKYf2Zsev5ZNfJ1OC6U0S/fdzQX
	T2xLRyIBt3jg5CmU3ghw/+eCz0JIyCVozJ/Yj3dd3eiHDNYxnJDOu1Vsnf26h8pKTwCs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184592-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184592: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9eee78988ca081eeed0a69fc9773e2e4268d1ed7
X-Osstest-Versions-That:
    xen=660f8a75013c947fbe5358a640032a1f9f1eece5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 05 Feb 2024 17:31:41 +0000

flight 184592 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184592/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9eee78988ca081eeed0a69fc9773e2e4268d1ed7
baseline version:
 xen                  660f8a75013c947fbe5358a640032a1f9f1eece5

Last test of basis   184590  2024-02-05 10:02:05 Z    0 days
Testing same since   184592  2024-02-05 13:02:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   660f8a7501..9eee78988c  9eee78988ca081eeed0a69fc9773e2e4268d1ed7 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 17:37:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 17:37:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676467.1052599 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2uo-0006ap-Bz; Mon, 05 Feb 2024 17:37:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676467.1052599; Mon, 05 Feb 2024 17:37:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX2uo-0006ai-98; Mon, 05 Feb 2024 17:37:54 +0000
Received: by outflank-mailman (input) for mailman id 676467;
 Mon, 05 Feb 2024 17:37:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+Hmw=JO=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rX2un-0006ac-SM
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 17:37:53 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 493ca140-c44d-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 18:37:51 +0100 (CET)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-5608f7fd83cso603052a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 09:37:51 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 o19-20020a170906359300b00a376d1a18adsm62496ejb.98.2024.02.05.09.37.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 09:37:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 493ca140-c44d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707154671; x=1707759471; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=kFsiMNkbE9GmL1Qe11snaulIwa7azZb2uW2FDnu9gZ8=;
        b=QuY8JwUzIGJ7S8SCetAyMIAQHLHb29qNXem6ZGFMEOboJqqi5C8rdYCWGxcQzdlcwq
         2Mi01yG/Ta835Pn45JLAKNefhJgEw/8WO6ZNSd0XP/6m9qwuc5RNuwTVMtuOB/qoAres
         DzGHCWiRW0UbuoQlRt0PM3jG+tX6WIP/0jW0s=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707154671; x=1707759471;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kFsiMNkbE9GmL1Qe11snaulIwa7azZb2uW2FDnu9gZ8=;
        b=Ph/wwR4C1+XsoTeZAadt3U+yo07JWN30sMLLNhDqJEyeQr6Nak9IcHkMOzEjw2137h
         mCbj9p9C18eZEDDVZxFsR6/14xjcfoLwokZ2JY495uBijtjRm6QBCTpHvJzeNZWG2bS2
         MUqzV7Uwfg6K5lcBwfijZD2y/bUA5SnuO6SqrDsbtmpB7r8G/HOz+f9tNsyqdFMzcLfo
         6KEQnc8mUh64o0vSfLhp7mRlUqVUJGgVeBrmdkUnFwxhn2hP8bQzW5JatW2UVnUyMK2x
         91NSA8uAXkDEw4nBD8ypk5iQCAC9knPQYRkjIW+r83dgiHwKyN+obXfgWQGAUqjUAdKU
         huAg==
X-Gm-Message-State: AOJu0YxTMUZO5ik9V+Yn7XOWaBaHbAr9xwRhJ0VDNgQXGXNrwxPvPQas
	P/6tfkTYeOGNemIt2eeMh8Wu7x/vHYBHLI0M5F7N+gqsB79PivdNSBl9nsENXlw=
X-Google-Smtp-Source: AGHT+IFQkATHx3wU3GO6x6I7cYW+bTHTWRzQNg5JAhZQ+qf7hfiGYVnvg+NVpT1sGDWVHQaEjIbmMQ==
X-Received: by 2002:a17:906:b2d1:b0:a37:249d:2799 with SMTP id cf17-20020a170906b2d100b00a37249d2799mr84762ejb.62.1707154671168;
        Mon, 05 Feb 2024 09:37:51 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWr3vZw/xu3JImqFdXsrh5AwoaA55HU9lgkMTJAi02T9ntBeN4YLSEJJ5rQ6cze1ouJc5hShOTXYslN
Date: Mon, 5 Feb 2024 17:37:50 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: zithro / Cyril =?iso-8859-1?Q?R=E9bert?= <slack@rabbit.lu>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] tools/xentop: fix sorting bug for some columns
Message-ID: <e8e56c7f-4181-4ad0-8eab-8369dccf9f28@perard>
References: <1bb93dbf9d09091fa36a3966ad8ffbccdb742166.1707041980.git.slack@rabbit.lu>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <1bb93dbf9d09091fa36a3966ad8ffbccdb742166.1707041980.git.slack@rabbit.lu>

On Sun, Feb 04, 2024 at 11:19:40AM +0100, zithro / Cyril Rbert wrote:
> From: Cyril Rbert <slack@rabbit.lu>
> 
> Sort doesn't work on columns VBD_OO, VBD_RD, VBD_WR and VBD_RSECT.
> Fix by adjusting variables names in compare functions.
> Bug fix only. No functional change.
> 
> Signed-off-by: Cyril Rbert (zithro) <slack@rabbit.lu>

Looks like this wants:
Fixes: 91c3e3dc91d6 ("tools/xentop: Display '-' when stats are not available.")

Otherwise, patch looks fine:
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 18:02:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 18:02:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676473.1052609 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX3IF-00059M-7Q; Mon, 05 Feb 2024 18:02:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676473.1052609; Mon, 05 Feb 2024 18:02:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX3IF-00059F-4i; Mon, 05 Feb 2024 18:02:07 +0000
Received: by outflank-mailman (input) for mailman id 676473;
 Mon, 05 Feb 2024 18:02:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=W3Sh=JO=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rX3IE-000599-71
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 18:02:06 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ab3d377e-c450-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 19:02:04 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-40f02b8d176so41439265e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 10:02:04 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 l16-20020a5d5610000000b0033b3ca3a255sm153245wrv.19.2024.02.05.10.02.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 05 Feb 2024 10:02:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab3d377e-c450-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707156124; x=1707760924; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=JT0xhifb6Bu8lOtIVAFqbBuPV+JGQAed1+MCbSuzUDw=;
        b=b1IC2KVV3nEOOWp1AF73nWVMSbv+VnR6GwGVTra54+iiv4uhFJWZ9vdxS+4zARpnIW
         ZC+s6wGZizURPM043Vflpqsp6uvjhls7GQk6zwCVRpTGNSh7V0etVueQ70lMTDvCDMLj
         qF4a0MRvQ5+nbDd9hWtqwHNpj1rfLha9Gk6QI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707156124; x=1707760924;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JT0xhifb6Bu8lOtIVAFqbBuPV+JGQAed1+MCbSuzUDw=;
        b=QlP14G9XpHSoFPEk2MBf2ZwKeHqNfvP0c1aTSE3Ur18ullO+MppYDUVdv3Wt/hBynN
         QBKwsgkEmmNjDjYDBQabKCHOVyvAbuzUl0foXT+BZYWV6RiLS+r+v20/3TbVVgmOtrzK
         N4wNu6EmwaPGnUdlXNtEbUSxyS4kudRKxUI8IVAG5fRoKq7r4TCPFvHNXoGPXpyeER5l
         jt6OVDOaQtWJ45qvwQPjZmXd+E91fqoDgwff0TBflQAtHqbBITl7zHfYp8YUvygBeX3n
         cztApAXFn7Eqq6StVoUsc7deB9foq96m21pMHuza9Ds0T37nLUiZL0Ua3qgyJVd7Aau9
         BaoQ==
X-Gm-Message-State: AOJu0Yy1rq4DOI0rhfa40RMiVri+ZLJ9/PYmUNSKIAzDO8Hv7CBZweJA
	A2tMh/EcCs+b/MofH6wsu73FMWY7CgN1iTg6HPgJ0pOcNsWfaeKs2yP5C7B1B2s=
X-Google-Smtp-Source: AGHT+IFJIoUb6GxISdeBrVOcE3YOOS0kiKAeGweSNj9mS8qaSOWuAmVFkXw9nv9C+TApSzXocVNb4A==
X-Received: by 2002:a5d:4b8b:0:b0:33b:1588:2250 with SMTP id b11-20020a5d4b8b000000b0033b15882250mr203618wrt.8.1707156123931;
        Mon, 05 Feb 2024 10:02:03 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVoD7l6baA8cvFKfdwmGrCVXMtbh/DS6Yk5dNoOu/PcHiITV0JHP6wj29Yeda/wLE0l4plMFES1Os0ev0rYpQt8t2W9j5rOl1c9xf0=
Message-ID: <8aab5e27-8b87-43bc-9d53-5af7df37118c@citrix.com>
Date: Mon, 5 Feb 2024 18:02:03 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/xentop: fix sorting bug for some columns
Content-Language: en-GB
To: Anthony PERARD <anthony.perard@citrix.com>,
 =?UTF-8?Q?zithro_/_Cyril_R=C3=A9bert?= <slack@rabbit.lu>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <1bb93dbf9d09091fa36a3966ad8ffbccdb742166.1707041980.git.slack@rabbit.lu>
 <e8e56c7f-4181-4ad0-8eab-8369dccf9f28@perard>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <e8e56c7f-4181-4ad0-8eab-8369dccf9f28@perard>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 05/02/2024 5:37 pm, Anthony PERARD wrote:
> On Sun, Feb 04, 2024 at 11:19:40AM +0100, zithro / Cyril Rébert wrote:
>> From: Cyril Rébert <slack@rabbit.lu>
>>
>> Sort doesn't work on columns VBD_OO, VBD_RD, VBD_WR and VBD_RSECT.
>> Fix by adjusting variables names in compare functions.
>> Bug fix only. No functional change.
>>
>> Signed-off-by: Cyril Rébert (zithro) <slack@rabbit.lu>
> Looks like this wants:
> Fixes: 91c3e3dc91d6 ("tools/xentop: Display '-' when stats are not available.")
>
> Otherwise, patch looks fine:
> Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Committed, thanks.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 19:24:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 19:24:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676481.1052619 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX4Zh-0000N6-8m; Mon, 05 Feb 2024 19:24:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676481.1052619; Mon, 05 Feb 2024 19:24:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX4Zh-0000Mz-69; Mon, 05 Feb 2024 19:24:13 +0000
Received: by outflank-mailman (input) for mailman id 676481;
 Mon, 05 Feb 2024 19:24:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TPgQ=JO=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rX4Zf-0000Mt-3b
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 19:24:11 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 216a6e30-c45c-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 20:24:07 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-51025cafb51so7199209e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 11:24:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 216a6e30-c45c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707161046; x=1707765846; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=cwJBPcNJmmrerSUs52ZYrXNaXNd9wYCH1lR6Sjnrv0Q=;
        b=IvKojqNxHOxHpbTb5FbbdO9uErxhczthKhd5xSI6CsTgkL51A/a6pznnQei1Y7nLH5
         j9ST7y7FvcYuRclhaaLA/IgjZmJwh4CuL1lgLD5vuFitR6QI0X8pLle8AALKwo8TtJXq
         hkZ90iOXP2JnX6fB+G1oi1R8Z7st6MNKCEHAhSVJV4I1GedYwUD+sgi46sxa0wEQFD0H
         PyegYo1L383rpDOE/6vO0haE3YbNT3RR2tGjh6PI3d0ANL6v0nwZAoH5hK/nqH9DTnIj
         wsOx52q7ktWfoCZ0aZfQ3VOfgsBwuB2gHvH3gHzRpSRJWyDnEl9DFlz+uBCMAOoEKF47
         8EAw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707161046; x=1707765846;
        h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cwJBPcNJmmrerSUs52ZYrXNaXNd9wYCH1lR6Sjnrv0Q=;
        b=NyLXxkM1mrRlQnN1VhJjiGEOMIB9YzxQOlR1bu/+iLJqCzqEX06L3OPlmXUycmk8dI
         i2NiZL5MIYqyJ1BEVifCbdrYkhoYWBotkJTqbxtbJk00L5mG1KxPo0zaWl+Z8qm/+pxf
         2NYa20fyzs4Ggs8OSdQMOoOiKGbmjxsnh2/SVRhPvlcEuCPro2cPmC3xuuaz7BnAgAG3
         IhFk4o8cdOtwtzcDNIYnaFJ952/0bMfliAkyCxh+XLZk0t3BYRPkuV5uFo1andcn/4cn
         OKb5ucnVn86c54CdrEP4nlW/J6uOwFuljN27iRpnFfG17+Ndc/GH2ue+FWCQKLJUjaBy
         l6CA==
X-Gm-Message-State: AOJu0YyjsPfzKE6UbB6iQDbyU63UaUU9+604Pr4xz3U/rddbkiRfsxmX
	gppxX0Tkf/kc271bXXIMMbaAeKDi/UCy2GyJKN21cFKEQPrl0TTBAtBEJTWdvbGzUdddPRwVBE1
	31OvmmJ/xx3udMFuLNmzGzwbC0WOQp3Bnb34=
X-Google-Smtp-Source: AGHT+IHdBHHkk5HS55p2dNgPu7P+oVVVa9HqzAuMQX6DSnLjFQ5Hh6vx04QiUHFyZKy+Dv+YlMRnfLMAMkCVm+JKpUA=
X-Received: by 2002:ac2:530a:0:b0:511:5936:c874 with SMTP id
 c10-20020ac2530a000000b005115936c874mr266319lfh.6.1707161046149; Mon, 05 Feb
 2024 11:24:06 -0800 (PST)
MIME-Version: 1.0
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Date: Mon, 5 Feb 2024 20:23:30 +0100
Message-ID: <CAMacjJzio2EDwudCUHoL6psKYztkHZOWwLNDknATvpN6Y0im=Q@mail.gmail.com>
Subject: Xen 4.19 release status tracking list
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: committers@xenproject.org, community.manager@xenproject.org, 
	Kelly Choi <kelly.choi@cloud.com>
Content-Type: text/plain; charset="UTF-8"

Hello everyone,

I would like to share with you a list for status tracking based on Xen ML:

Arm:
  * [PATCH v5 00/13] Arm cache coloring [
https://lore.kernel.org/xen-devel/20240102095138.17933-1-carlo.nonato@minervasys.tech/
]
  * [PATCH v1 repost 0/4] xen/arm64: Rework the MMU-off code (idmap)
so it is self-contained [
https://lore.kernel.org/xen-devel/20240116143709.86584-1-julien@xen.org/
]
  * [PATCH v13 00/14] PCI devices passthrough on Arm, part 3 [
https://lore.kernel.org/xen-devel/20240202213321.1920347-1-stewart.hildebrand@amd.com/
]

PPC:
  *  [PATCH v2 0/7] Early Boot Allocation on Power [
https://lore.kernel.org/xen-devel/cover.1702607884.git.sanastasio@raptorengineering.com/
]

RISC-V:
  *  [PATCH v4 00/30]  Enable build of full Xen for RISC-V [
https://lore.kernel.org/xen-devel/cover.1707146506.git.oleksii.kurochko@gmail.com/
]

x86:
  * [PATCH 0/4] iommu/x86: fixes/improvements for unity range checks [
https://lore.kernel.org/xen-devel/20240201170159.66330-1-roger.pau@citrix.com/
]
  * [PATCH 0/8] x86: support AVX10.1 [
https://lore.kernel.org/xen-devel/298db76f-d0ee-4d47-931f-1baa1a7546cf@suse.com/
]
  * [PATCH v13 00/35] x86: enable FRED for x86-64 [
https://lore.kernel.org/xen-devel/20231205105030.8698-1-xin3.li@intel.com/
]
  * [PATCH v4 0/8] x86emul: misc additions
https://lore.kernel.org/xen-devel/9dd23064-c79e-4a50-9c71-c0e73b189944@suse.com/
  * [PATCH 0/7] VT-d: SATC handling and ATS tidying [
https://lore.kernel.org/xen-devel/25506838-b818-4686-8c16-3a198338af44@suse.com/
]
  * [XEN PATCH 0/9] x86: parallelize AP bring-up during boot [
https://lore.kernel.org/xen-devel/cover.1699982111.git.krystian.hebel@3mdeb.com/
]
  * [PATCH v2 00/12] x86: memcpy() / memset() (non-)ERMS flavors plus
fallout [ https://lore.kernel.org/xen-devel/8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com/
]

common:
  * annotate entry points with type and size" series [
https://lore.kernel.org/xen-devel/5e74a501-2824-42af-804b-47b515aa9bcf@suse.com/
]
  * [PATCH v2 0/8] limit passing around of cpu_user_regs [
https://lore.kernel.org/xen-devel/33db57a2-b569-4fa8-b0b0-36ea92528d7b@suse.com/
]
  * [PATCH v2 (resend) 00/27] Remove the directmap [
https://lore.kernel.org/xen-devel/20240116192611.41112-1-eliasely@amazon.com/
]
  * [PATCH v4 00/32] tools: enable xenstore-stubdom to use 9pfs [
https://lore.kernel.org/xen-devel/20240205105001.24171-1-jgross@suse.com/
]
  * [PATCH v7 0/7] Introduce generic headers [
https://lore.kernel.org/xen-devel/cover.1706281994.git.oleksii.kurochko@gmail.com/
]
  * [PATCH] move __read_mostly to xen/cache.h
[https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/]
  * [PATCH v2] NUMA: no need for asm/numa.h when !NUMA [
https://lore.kernel.org/xen-devel/94cffa48-5e31-4557-827d-c6dec6031d05@suse.com/
]
  * [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE() [
https://lore.kernel.org/xen-devel/42fc6ae8d3eb802429d29c774502ff232340dc84.1706259490.git.federico.serafini@bugseng.com/
]
  * [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(),
ASSERT_UNREACHABLE() to xen/bug.h [
https://lore.kernel.org/xen-devel/4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com/
]
  * [PATCH] xen/lib: introduce generic find next bit operations [
https://lore.kernel.org/xen-devel/52730e6314210ba4164a9934a720c4fda201447b.1706266854.git.oleksii.kurochko@gmail.com/
]

Please reply with items you would like to see in 4.19 so that people
know what is happening and prioritize accordingly.
You're welcome to provide a description and use cases of the feature
you're working on.

Have a nice week!

Best regards,
 Oleksii


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 20:14:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 20:14:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676489.1052630 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX5ME-0006nQ-1B; Mon, 05 Feb 2024 20:14:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676489.1052630; Mon, 05 Feb 2024 20:14:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX5MD-0006nJ-SP; Mon, 05 Feb 2024 20:14:21 +0000
Received: by outflank-mailman (input) for mailman id 676489;
 Mon, 05 Feb 2024 20:14:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DQRq=JO=gmail.com=christopher.w.clark@srs-se1.protection.inumbo.net>)
 id 1rX5MD-0006nD-14
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 20:14:21 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 25309d30-c463-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 21:14:20 +0100 (CET)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2d051fb89fbso730481fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 12:14:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25309d30-c463-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707164059; x=1707768859; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jXe+1bcPtK0g/zBwZbjckLDPRV+tVJBOzR3z17zKq/4=;
        b=FcS93UDNnM3v0jd4N7amR4riF+swIuSB72zaLXbz+Nax2MUWD3vsb1R/rJwHpGNM9Q
         njE3l955Bvrxd7L0SOsz2shbTgV7/zdoEfJhrX7RMc9nVsyFVSIMfHqZEnp/kqelVoxb
         K6eJwYowYTEqIVT0YupPhbyRCmh+dZKhff18P+q6SpfQraQ/PGASvKdRkhuNL+tSmpPv
         5FRyIgKjITQBBcYJ22T3GTV9MDORwOCL5APzUWazoF2cR3KZhxBtrOFj2h7VesTQsQar
         pyASZIQ4MU9sdhwmKcZO9ZVfpjIoge+Rd5GmwTP0mFTu7yWuKYsIo2L0xacjcT12l4vX
         LfXg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707164059; x=1707768859;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=jXe+1bcPtK0g/zBwZbjckLDPRV+tVJBOzR3z17zKq/4=;
        b=b/At2ZAoYlq9vlbS30VWqqZZnvRu2EbTxYQ4bhpWte8a0JUkSRaY/284nbXjLnMAvH
         AVOFJIuOrRbgjW9mo8BAXsEfMknnjVpw/hPCYplgD8rRk0e0KkwqpsEJXSRRL2GeXAGa
         c5bQw9kw3YNfkCSLCkX7HTI/3Mr/+FsqKGHlPI7ioe8hC4UhRkToFPE8eQRze63NF5rA
         0AxCQrnocivj1hhKDkAKtFG9Y0jHGuHx+v1hvAopiKwaeehkXnF3qhw9Q/mXbaERh2RN
         Kbkkh8APj9z4h/yow+I6KGFIbb/cte70LLHjESkOlo/dqPkCSCnr7bz2E8hZQ6kV0z5E
         8kMw==
X-Gm-Message-State: AOJu0YzLh6Ovowp2NgVLte6cVrXNXja6KfxoHepz4r0fVM+DYb+/A/+e
	1Fp6H41h9aaKWlnmyqdc3hKzIsIq6hz9F+zwAB9QWpEaV/F0+AaKOzX+IE1fdaEnbIOs2dH4FI9
	E4Cck7JDHLIxTv/hwEtBqBNLePnI=
X-Google-Smtp-Source: AGHT+IF0H2DhJ01UdGjSBeD5Jqhd5PHSR8v5f6yAwdttrE+uzR07rx+ZH2NTUFZup7LIvMsD2Gbtgw/Qj7SadYs5SW8=
X-Received: by 2002:a2e:b0f0:0:b0:2d0:b018:c0f0 with SMTP id
 h16-20020a2eb0f0000000b002d0b018c0f0mr535774ljl.33.1707164059333; Mon, 05 Feb
 2024 12:14:19 -0800 (PST)
MIME-Version: 1.0
References: <580c6c48-9dd5-4296-8696-2b40beac2bc3@suse.com>
 <5d76295f-0438-4c5d-8b2e-73a1abcc96e6@suse.com> <CACMJ4GbvMhDdgOp=kUeL_+_j2yS_NVU69P8Xh9QYDLUwJ31Eow@mail.gmail.com>
 <CACMJ4GbNJgsg7tts=9UQPUr9ofsXajuHz7wYO1r5LnevWEL4_g@mail.gmail.com>
 <5aea32e3-033e-4299-9088-279d23801ba6@suse.com> <b3cd10f2-c3b4-49cc-8bac-28f525dd574b@suse.com>
In-Reply-To: <b3cd10f2-c3b4-49cc-8bac-28f525dd574b@suse.com>
From: Christopher Clark <christopher.w.clark@gmail.com>
Date: Mon, 5 Feb 2024 12:14:07 -0800
Message-ID: <CACMJ4GZGb+Yh74pAdhp9ise6U_iM0AFhxtcfXXYF6xYWcySLJw@mail.gmail.com>
Subject: Re: Ping: Ping: [PATCH] Argo: drop meaningless mfn_valid() check
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 5, 2024 at 4:17=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 04.01.2024 09:12, Jan Beulich wrote:
> > On 23.12.2023 22:35, Christopher Clark wrote:
> >> On Sat, Dec 23, 2023 at 12:47=E2=80=AFPM Christopher Clark
> >> <christopher.w.clark@gmail.com> wrote:
> >>>
> >>> On Sun, Dec 17, 2023 at 11:55=E2=80=AFPM Jan Beulich <jbeulich@suse.c=
om> wrote:
> >>>>
> >>>> Christopher,
> >>>>
> >>>> On 27.11.2023 14:55, Jan Beulich wrote:
> >>>>> Holding a valid struct page_info * in hands already means the refer=
enced
> >>>>> MFN is valid; there's no need to check that again. Convert the chec=
king
> >>>>> logic to a switch(), to help keeping the extra (and questionable) x=
86-
> >>>>> only check in somewhat tidy shape.
> >>>>>
> >>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >>>
> >>> Reviewed-by: Christopher Clark <christopher.w.clark@gmail.com>
> >>
> >> I'd like to hold off on this just yet, sorry -- the change does look
> >> ok as far as a transform being applied the prior logic and the
> >> necessity of the check, but with it applied, it's not obvious that it
> >> handles all the page types as best that it could there, so I'd like to
> >> look at this (and the previously submitted patch again) please.
> >
> > I'm puzzled: This patch merely removes a pointless check. Whatever is
> > lacking with it in place will have been lacking before. Also can you
> > please give a rough estimate towards when you'll be getting back on
> > this, or ideally on both patches?
>
> Another month later: I'll give it this week, and without hearing back
> I'll commit what there is some time next week, with the R-b you provided.

Sorry for the slow communication. I'm not unaware of this but it has
been difficult to provide an update on when I will be able to give it
the appropriate attention.
I understand that you'd appreciate a response before next week, so
I'll aim for that.

thanks,

Christopher


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 20:30:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 20:30:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676494.1052639 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX5ba-0001DV-8g; Mon, 05 Feb 2024 20:30:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676494.1052639; Mon, 05 Feb 2024 20:30:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX5ba-0001DO-65; Mon, 05 Feb 2024 20:30:14 +0000
Received: by outflank-mailman (input) for mailman id 676494;
 Mon, 05 Feb 2024 20:30:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX5bY-0001DE-PS; Mon, 05 Feb 2024 20:30:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX5bY-0006bu-Np; Mon, 05 Feb 2024 20:30:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX5bY-0007X6-8p; Mon, 05 Feb 2024 20:30:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rX5bY-0004yV-8K; Mon, 05 Feb 2024 20:30:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ZzXS1obeqS6rGP34hUnN841jPwyG+lYbhXxeVP9805k=; b=Kx+zqevz2Djvx3J0V8KREdkzuF
	13LxlMIyIImo+P4pTN4CNWl5xZeNUp7Z7CTb4DUCJz/Mr7ubwecGToKLI8yckVwCvdZNjRJxaWXW4
	NHRm8VSWPjcvK8l9PfzJ3ULzTc8i54dsE5CKVwtlRHc0HeTYwRM7a4wIK1967m0iV4SM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184593-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184593: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=29f17d837421f13c0e0010802de1b2d51d2ded4a
X-Osstest-Versions-That:
    xen=9eee78988ca081eeed0a69fc9773e2e4268d1ed7
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 05 Feb 2024 20:30:12 +0000

flight 184593 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184593/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  29f17d837421f13c0e0010802de1b2d51d2ded4a
baseline version:
 xen                  9eee78988ca081eeed0a69fc9773e2e4268d1ed7

Last test of basis   184592  2024-02-05 13:02:03 Z    0 days
Testing same since   184593  2024-02-05 18:00:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Cyril Rébert (zithro) <slack@rabbit.lu>
  Cyril Rébert <slack@rabbit.lu>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   9eee78988c..29f17d8374  29f17d837421f13c0e0010802de1b2d51d2ded4a -> smoke


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 20:41:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 20:41:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676501.1052649 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX5md-0002tD-9c; Mon, 05 Feb 2024 20:41:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676501.1052649; Mon, 05 Feb 2024 20:41:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX5md-0002t6-6W; Mon, 05 Feb 2024 20:41:39 +0000
Received: by outflank-mailman (input) for mailman id 676501;
 Mon, 05 Feb 2024 20:41:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=b4S5=JO=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rX5mb-0002t0-HF
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 20:41:37 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f32911c7-c466-11ee-98f5-efadbce2ee36;
 Mon, 05 Feb 2024 21:41:34 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id A7B35611B9;
 Mon,  5 Feb 2024 20:41:33 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 49BFCC433F1;
 Mon,  5 Feb 2024 20:41:32 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f32911c7-c466-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707165693;
	bh=B71qTvwZU7PEXKS13VOXgNZIWUDc+/QogFY8w2OUaME=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JdAT40cq3KyKBDIllMCyGskml8HaHhgV/rbbMqnl/Y0e/rtIGsrzd6pDHgtHOh/U9
	 AZ84dDVJDab+lVZFaEZgKlE7qA4fD1GJXaS1/BeoPV9XpkgWOn6zj20KB9nz9epS5E
	 d71aqd4WJBzxJs2MOM5AwyrJByKF9AgnLfHkemLG9ydd0T/a+tK4zspO1fPkEBqTbm
	 sLShot1cfvVPh5dzhsj1ihAf21xwqZRXn5iR4BKB0e5pfHRyqxG5ODB0DpQ9DGiOXa
	 Y/bPdPiZnpoiH2lqOlKp8bbH+4mjvDBXNiLhIgDt62zidQ3wUQzdcb+SyZ2P/TOU4m
	 /2+I0eUKqtPJg==
Date: Mon, 5 Feb 2024 12:41:30 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Frediano Ziglio <frediano.ziglio@cloud.com>
cc: Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH 2/5] Remove useless assignment
In-Reply-To: <20240205104637.14868-1-frediano.ziglio@cloud.com>
Message-ID: <alpine.DEB.2.22.394.2402051241240.1925432@ubuntu-linux-20-04-desktop>
References: <20240205104637.14868-1-frediano.ziglio@cloud.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 5 Feb 2024, Frediano Ziglio wrote:
> The variable is assigned later, that value is never used.
> 
> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
>  xen/common/event_channel.c | 2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
> index a7a004a084..15aec5dcbb 100644
> --- a/xen/common/event_channel.c
> +++ b/xen/common/event_channel.c
> @@ -324,8 +324,6 @@ int evtchn_alloc_unbound(evtchn_alloc_unbound_t *alloc, evtchn_port_t port)
>          goto out;
>      }
>  
> -    rc = 0;
> -
>      chn = evtchn_from_port(d, port);
>  
>      rc = xsm_evtchn_unbound(XSM_TARGET, d, chn, alloc->remote_dom);
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 21:13:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 21:13:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676506.1052658 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX6Hc-0006ut-I8; Mon, 05 Feb 2024 21:13:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676506.1052658; Mon, 05 Feb 2024 21:13:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX6Hc-0006um-FI; Mon, 05 Feb 2024 21:13:40 +0000
Received: by outflank-mailman (input) for mailman id 676506;
 Mon, 05 Feb 2024 21:13:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rX6Ha-0006ug-MD
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 21:13:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX6HZ-0007KX-RG; Mon, 05 Feb 2024 21:13:37 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX6HZ-0007xj-KW; Mon, 05 Feb 2024 21:13:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	References:Cc:To:From:Subject:MIME-Version:Date:Message-ID;
	bh=P+HzqDk5bXex/unV08zPnvaewpAc0p8Ya3Cp51axNaw=; b=5E4GrSUIa9NqR0pfVsbd9Ad6sp
	+rxOKm7QfLqxpiz258a+oVIxNceU2+bS6yMSKDo+8N5dyapK1q1cm9+xG31bj7YGbmPLmuVxYEPzu
	77/lbBk7NVzslrRkq3Dg85gsGLgM+W6eRm5X7br0TWt9CzB99cbduNj3DvBJNL5bI/KA=;
Message-ID: <ad801dea-7a7a-4307-927b-8a4e9edb2134@xen.org>
Date: Mon, 5 Feb 2024 21:13:35 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 00/32] tools: enable xenstore-stubdom to use 9pfs
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Nick Rosbrook
 <rosbrookn@gmail.com>, Samuel Thibault <samuel.thibault@ens-lyon.org>
References: <20240205105001.24171-1-jgross@suse.com>
 <689635a7-6d34-44fe-b00b-31fdc03f6969@xen.org>
 <d4740502-795a-4e22-afe2-abd0746da843@suse.com>
 <c635e693-fe07-458c-9d58-273e2779f5b9@xen.org>
In-Reply-To: <c635e693-fe07-458c-9d58-273e2779f5b9@xen.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit



On 05/02/2024 17:18, Julien Grall wrote:
> Hi Juergen,
> 
> On 05/02/2024 11:08, Jürgen Groß wrote:
>> On 05.02.24 11:55, Julien Grall wrote:
>>> Hi Juergen,
>>>
>>> On 05/02/2024 10:49, Juergen Gross wrote:
>>>> This series is adding 9pfs support to Xenstore-stubdom, enabling it
>>>> to do logging to a dom0 directory.
>>>>
>>>> This is a prerequisite for the final goal to add live update support
>>>> to Xenstore-stubdom, as it enables the stubdom to store its state in
>>>> a dom0 file.
>>>>
>>>> The 9pfs backend is a new daemon written from scratch. Using a
>>>> dedicated 9pfs daemon has several advantages:
>>>>
>>>> - it is using much less resources than a full blown qemu process
>>>> - it can serve multiple guests (the idea is to use it for other
>>>>    infrastructure domains, like qemu-stubdom or driver domains, too)
>>>> - it is designed to support several security enhancements, like
>>>>    limiting the number of files for a guest, or limiting the allocated
>>>>    file system space
>>>> - it doesn't support file links (neither hard nor soft links) or
>>>>    referencing parent directories via "..", minimizing the risk that
>>>>    a guest can "escape" from its home directory
>>>>
>>>> Note that for now the daemon only contains the minimal needed
>>>> functionality to do logging from Xenstore-stubdom. I didn't want to
>>>> add all the 9pfs commands and security add-ons in the beginning, in
>>>> order to avoid needless efforts in case the idea of the daemon is
>>>> being rejected.
>>>>
>>>> Changes in V4:
>>>> - patch 2 of V3 was applied
>>>> - added support of reading directories
>>>> - addressed review comments
>>>>
>>>> Changes in V3:
>>>> - new patches 1, 23-25
>>>> - addressed review comments
>>>>
>>>> Changes in V2:
>>>> - support of multiple rings per device
>>>> - xenlogd->xen-9pfsd rename
>>>> - addressed review comments
>>>> - fixed some bugs
>>>>
>>>> Juergen Gross (32):
>>>>    tools: add access macros for unaligned data
>>>>    tools: add a new xen logging daemon
>>>>    tools/xen-9pfsd: connect to frontend
>>>>    tools/xen-9pfsd: add transport layer
>>>>    tools/xen-9pfsd: add 9pfs response generation support
>>>>    tools/xen-9pfsd: add 9pfs version request support
>>>>    tools/xen-9pfsd: add 9pfs attach request support
>>>>    tools/xen-9pfsd: add 9pfs walk request support
>>>>    tools/xen-9pfsd: add 9pfs open request support
>>>>    tools/xen-9pfsd: add 9pfs clunk request support
>>>>    tools/xen-9pfsd: add 9pfs create request support
>>>>    tools/xen-9pfsd: add 9pfs stat request support
>>>>    tools/xen-9pfsd: add 9pfs write request support
>>>>    tools/xen-9pfsd: add 9pfs read request support
>>>>    tools/libs/light: add backend type for 9pfs PV devices
>>>>    tools/xl: support new 9pfs backend xen_9pfsd
>>>>    tools/helpers: allocate xenstore event channel for xenstore stubdom
>>>>    tools/xenstored: rename xenbus_evtchn()
>>>>    stubdom: extend xenstore stubdom configs
>>>>    tools: add 9pfs device to xenstore-stubdom
>>>>    tools/xenstored: add early_init() function
>>>>    tools/xenstored: move systemd handling to posix.c
>>>>    tools/xenstored: move all log-pipe handling into posix.c
>>>>    tools/xenstored: move all socket handling into posix.c
>>>>    tools/xenstored: get own domid in stubdom case
>>>>    tools/xenstored: rework ring page (un)map functions
>>>>    tools/xenstored: split domain_init()
>>>>    tools/xenstored: map stubdom interface
>>>>    tools/xenstored: mount 9pfs device in stubdom
>>>>    tools/xenstored: add helpers for filename handling
>>>>    tools/xenstored: support complete log capabilities in stubdom
>>>>    tools/xenstored: have a single do_control_memreport()
>>>
>>> I haven't checked what's the state of the 9PFS patches. Can part of 
>>> the xenstored changes be committed without the 9PFS changes?
>>
>> The following patches can go in without the 9pfs daemon:
> 
> It looks like the gitalb CI is not happy with the following patches [1]:
> 
> In function ‘free_stat’,
>      inlined from ‘write_9pfs’ at 9pfront.c:935:9:
> 9pfront.c:120:14: error: ‘stat.name’ may be used uninitialized 
> [-Werror=maybe-uninitialized]
>    120 |     free(stat->name);
>        |          ~~~~^~~~~~
> 9pfront.c: In function ‘write_9pfs’:
> 9pfront.c:929:20: note: ‘stat’ declared here
>    929 |     struct p9_stat stat;
>        |                    ^~~~
> 
> I think...
> 
>>
>> tools/helpers: allocate xenstore event channel for xenstore stubdom
>> tools/xenstored: rename xenbus_evtchn()
>> stubdom: extend xenstore stubdom configs
> 
> .. this is related to this patch. Can you have a look?
> 
> I have just pushed a new branch without this patch. Let see if the CI 
> [2] will pass this time.

It passed. So I committed all the patches you mentionned but "stubdom: 
extend xenstore stubdom configs".

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 21:21:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 21:21:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676511.1052669 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX6PA-00006U-9Q; Mon, 05 Feb 2024 21:21:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676511.1052669; Mon, 05 Feb 2024 21:21:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX6PA-00006N-6q; Mon, 05 Feb 2024 21:21:28 +0000
Received: by outflank-mailman (input) for mailman id 676511;
 Mon, 05 Feb 2024 21:21:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rX6P8-00006H-Di
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 21:21:26 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX6P5-0007RS-Tz; Mon, 05 Feb 2024 21:21:23 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rX6P5-00088N-NL; Mon, 05 Feb 2024 21:21:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=GSC24i2dQ31Wyy1eONMEhN79Ac5qxXifBbooRVtuCaw=; b=nub6J2VdbpUsGUDthSi9tAtn+w
	xqzBQxBLUgFg1wKm7fje8YtD77o+PZ9oxVD567U0C/qUSBZa7ykZQ6M7mjfhhX5cHBSU/45Q9z4gg
	DCiuHRkHw61yI0IlvlxfksV9Qrsz2+Pb0An6A0IGhjbDbF/iXl5f0g5v/sck1M7aSENc=;
Message-ID: <cbdf382c-a688-45f5-b834-9e3ab70ffc53@xen.org>
Date: Mon, 5 Feb 2024 21:21:22 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/5] Remove useless assignment
Content-Language: en-GB
To: Stefano Stabellini <sstabellini@kernel.org>,
 Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240205104637.14868-1-frediano.ziglio@cloud.com>
 <alpine.DEB.2.22.394.2402051241240.1925432@ubuntu-linux-20-04-desktop>
From: Julien Grall <julien@xen.org>
In-Reply-To: <alpine.DEB.2.22.394.2402051241240.1925432@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

The tag says '2/5' but I don't see a thread. Is the series meant to 
contain more patches?

Also, the title is not very specific about where the assignment is 
removed. I have committed with the following title:

xen/evtchn: Remove useful assignment in evtchn_alloc_unbound

On 05/02/2024 20:41, Stefano Stabellini wrote:
> On Mon, 5 Feb 2024, Frediano Ziglio wrote:
>> The variable is assigned later, that value is never used.

I also add 'rc' just for clarity.

>>
>> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> 
>> ---
>>   xen/common/event_channel.c | 2 --
>>   1 file changed, 2 deletions(-)
>>
>> diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
>> index a7a004a084..15aec5dcbb 100644
>> --- a/xen/common/event_channel.c
>> +++ b/xen/common/event_channel.c
>> @@ -324,8 +324,6 @@ int evtchn_alloc_unbound(evtchn_alloc_unbound_t *alloc, evtchn_port_t port)
>>           goto out;
>>       }
>>   
>> -    rc = 0;
>> -

This looks like a missing clean-up from e5ba5165cae6 ("xen/evtchn: Purge 
ERROR_EXIT{,_DOM}()").

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 21:59:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 21:59:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676525.1052691 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX6zX-0004Qm-D8; Mon, 05 Feb 2024 21:59:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676525.1052691; Mon, 05 Feb 2024 21:59:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX6zX-0004Qf-91; Mon, 05 Feb 2024 21:59:03 +0000
Received: by outflank-mailman (input) for mailman id 676525;
 Mon, 05 Feb 2024 21:59:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX6zW-0004QV-9h; Mon, 05 Feb 2024 21:59:02 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX6zW-00083M-7H; Mon, 05 Feb 2024 21:59:02 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX6zV-0001RP-P0; Mon, 05 Feb 2024 21:59:01 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rX6zV-0007Ll-OZ; Mon, 05 Feb 2024 21:59:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=S4R5B0UA4XD6KIM2IgTQyzBhdZ+u9RVEMyJ2zvLZviM=; b=ERIynFUTStsHFlQBbLmbGWGGX7
	IQgAKiA1+iWncJcgM9o9CYL0lZI+nxEHXmp0VsaIkuOr3DJxRV/qlgz8ZQIEseDR4V9/XqpSI1biQ
	MOvB70A/j8uyHRtXfu48OwEQbS3H7gjlvDr2aplo5woVDDS3lpW/iLVTmpqXXhLBN4i4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184591-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184591: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-ovmf-amd64:guest-start/debianhvm.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt-raw:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=660f8a75013c947fbe5358a640032a1f9f1eece5
X-Osstest-Versions-That:
    xen=3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 05 Feb 2024 21:59:01 +0000

flight 184591 xen-unstable real [real]
flight 184594 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184591/
http://logs.test-lab.xenproject.org/osstest/logs/184594/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-ovmf-amd64 20 guest-start/debianhvm.repeat fail pass in 184594-retest
 test-armhf-armhf-libvirt-raw 17 guest-start/debian.repeat fail pass in 184594-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184589
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184589
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184589
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184589
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184589
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184589
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184589
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184589
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184589
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184589
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184589
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184589
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  660f8a75013c947fbe5358a640032a1f9f1eece5
baseline version:
 xen                  3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc

Last test of basis   184589  2024-02-05 01:54:06 Z    0 days
Testing same since   184591  2024-02-05 12:38:54 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Paul Durrant <paul@xen.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         fail    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 fail    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   3f819af8a7..660f8a7501  660f8a75013c947fbe5358a640032a1f9f1eece5 -> master


From xen-devel-bounces@lists.xenproject.org Mon Feb 05 22:55:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 05 Feb 2024 22:55:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676563.1052716 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX7sC-00040Z-QF; Mon, 05 Feb 2024 22:55:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676563.1052716; Mon, 05 Feb 2024 22:55:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX7sC-00040S-Nh; Mon, 05 Feb 2024 22:55:32 +0000
Received: by outflank-mailman (input) for mailman id 676563;
 Mon, 05 Feb 2024 22:55:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=b4S5=JO=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rX7sB-00040M-RP
 for xen-devel@lists.xenproject.org; Mon, 05 Feb 2024 22:55:31 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a89edfac-c479-11ee-8a46-1f161083a0e0;
 Mon, 05 Feb 2024 23:55:30 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id A9DA960C7E;
 Mon,  5 Feb 2024 22:55:28 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3A51C433F1;
 Mon,  5 Feb 2024 22:55:25 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a89edfac-c479-11ee-8a46-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707173728;
	bh=05cWxMhdq+n612fGlfifUOSf4CVxJkXMiIQ8goeMVxg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=jwH4W4OVn0h2PrtXiB+6WfoLwpqkEltLq0ekgw5mdhLdVq3uQPeIfXsqSegxaIU/8
	 tXbZ0xs74D3pW+ca4uCrKGilSi9oKmIBLZB6zb76fOzH3JZhGEUkabh9ghiS4ain3k
	 +MJ6MAhfiQu859nqXrJyUI+YXP4bUI6Cnn4lAvvnz/vyr9WdI/LGW1DfVZn3xoy8+6
	 pKvJ0AK4/GlQ9rhP0vLa8tFC4WbE8riLKTGM9zwgs7u0O8iEb9U0wQ8xHK31D9uCfh
	 S4YoOfZ66XaMfUhWii4/R7T0cFNcUbHHFbqeKar/FHSvKPoM7trqiSFAC7TKfLD6a8
	 dgfjDtEZDWO4w==
Date: Mon, 5 Feb 2024 14:55:24 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Elias El Yandouzi <eliasely@amazon.com>, julien@xen.org, 
    pdurrant@amazon.com, dwmw@amazon.com, Wei Liu <wei.liu2@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    David Woodhouse <dwmw2@amazon.com>, Hongyan Xia <hongyxia@amazon.com>, 
    Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH v2 (resend) 02/27] x86/setup: Move vm_init() before acpi
 calls
In-Reply-To: <6fc809af-9662-47fd-b91e-533fd75d0473@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402051455110.1925432@ubuntu-linux-20-04-desktop>
References: <20240116192611.41112-1-eliasely@amazon.com> <20240116192611.41112-3-eliasely@amazon.com> <6fc809af-9662-47fd-b91e-533fd75d0473@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 25 Jan 2024, Jan Beulich wrote:
> On 16.01.2024 20:25, Elias El Yandouzi wrote:
> > From: Wei Liu <wei.liu2@citrix.com>
> > 
> > After the direct map removal, pages from the boot allocator are not
> > going to be mapped in the direct map. Although we have map_domain_page,
> > they are ephemeral and are less helpful for mappings that are more than a
> > page, so we want a mechanism to globally map a range of pages, which is
> > what vmap is for. Therefore, we bring vm_init into early boot stage.
> > 
> > To allow vmap to be initialised and used in early boot, we need to
> > modify vmap to receive pages from the boot allocator during early boot
> > stage.
> > 
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > Signed-off-by: David Woodhouse <dwmw2@amazon.com>
> > Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> > Signed-off-by: Julien Grall <jgrall@amazon.com>
> > Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> albeit ...
> 
> > --- a/xen/arch/arm/setup.c
> > +++ b/xen/arch/arm/setup.c
> > @@ -748,6 +748,8 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset,
> >  
> >      setup_mm();
> >  
> > +    vm_init();
> > +
> >      /* Parse the ACPI tables for possible boot-time configuration */
> >      acpi_boot_table_init();
> >  
> > @@ -759,8 +761,6 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset,
> >       */
> >      system_state = SYS_STATE_boot;
> >  
> > -    vm_init();
> > -
> >      if ( acpi_disabled )
> >      {
> >          printk("Booting using Device Tree\n");
> 
> ... with this change the title claiming x86 isn't quite right. Hopefully
> Arm folks will spot the need for an ack there nevertheless.

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 00:38:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 00:38:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676568.1052726 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX9TS-0008JX-58; Tue, 06 Feb 2024 00:38:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676568.1052726; Tue, 06 Feb 2024 00:38:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rX9TS-0008JQ-2Z; Tue, 06 Feb 2024 00:38:06 +0000
Received: by outflank-mailman (input) for mailman id 676568;
 Tue, 06 Feb 2024 00:38:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX9TQ-0008JG-5T; Tue, 06 Feb 2024 00:38:04 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX9TQ-0002xE-30; Tue, 06 Feb 2024 00:38:04 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rX9TP-0002H2-HX; Tue, 06 Feb 2024 00:38:03 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rX9TP-00007p-Gz; Tue, 06 Feb 2024 00:38:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7SCX7wgYi4dDbBHxY9LNou0Ixn2SQcvgwydMzwFpFng=; b=T7U6lKNusuX+Z+gIUXARl7V50U
	P5x0VxwF2Vo7jK5B3FRNncc/5OL8JY6gsZV2gG1DwjKUs+n6ShuQ5KtxkYSaCBOmZ40aaDwURkonr
	BODllq69/A38cFH4dwYnWuuNOTNME4WpT5+IwhQnpZtFgQvUOoDf5oTvN3AQAEUgRU7Y=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184596-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184596: regressions - FAIL
X-Osstest-Failures:
    xen-unstable-smoke:build-arm64-xsm:xen-build:fail:regression
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=3ce2e1f606cef3aa16b856396b31068c447b1262
X-Osstest-Versions-That:
    xen=29f17d837421f13c0e0010802de1b2d51d2ded4a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 06 Feb 2024 00:38:03 +0000

flight 184596 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184596/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm               6 xen-build                fail REGR. vs. 184593

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  3ce2e1f606cef3aa16b856396b31068c447b1262
baseline version:
 xen                  29f17d837421f13c0e0010802de1b2d51d2ded4a

Last test of basis   184593  2024-02-05 18:00:25 Z    0 days
Testing same since   184596  2024-02-05 21:00:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Juergen Gross <jgross@suse.com>

jobs:
 build-arm64-xsm                                              fail    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 3ce2e1f606cef3aa16b856396b31068c447b1262
Author: Juergen Gross <jgross@suse.com>
Date:   Mon Feb 5 11:49:57 2024 +0100

    tools/xenstored: map stubdom interface
    
    When running as stubdom, map the stubdom's Xenstore ring page in order
    to support using the 9pfs frontend.
    
    Use the same pattern as in dom0_init() when running as daemon in dom0
    (introduce the own domain, then send an event to the client side to
    signal Xenstore is ready to communicate).
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit 097388d520eb104ebd93b486d126f319536a54e4
Author: Juergen Gross <jgross@suse.com>
Date:   Mon Feb 5 11:49:56 2024 +0100

    tools/xenstored: split domain_init()
    
    Today domain_init() is called either just before calling dom0_init()
    in case no live update is being performed, or it is called after
    reading the global state from read_state_global(), as the event
    channel fd is needed.
    
    Split up domain_init() into a preparation part which can be called
    unconditionally, and in a part setting up the event channel handle.
    
    Note that there is no chance that chk_domain_generation() can be
    called now before xc_handle has been setup, so there is no need for
    the related special case anymore.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit 5edc819ed50d4d0bc0de422eed72301b69309862
Author: Juergen Gross <jgross@suse.com>
Date:   Mon Feb 5 11:49:55 2024 +0100

    tools/xenstored: rework ring page (un)map functions
    
    When [un]mapping the ring page of a Xenstore client, different actions
    are required for "normal" guests and dom0. Today this distinction is
    made at call site.
    
    Move this distinction into [un]map_interface() instead, avoiding code
    duplication and preparing special handling for [un]mapping the stub
    domain's ring page.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit a3e37f7024b0fb25c648315b2edf3583aa140b32
Author: Juergen Gross <jgross@suse.com>
Date:   Mon Feb 5 11:49:54 2024 +0100

    tools/xenstored: get own domid in stubdom case
    
    Obtain own domid when running as stubdom.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit e199a3068097708a9e712bf5d86f9fb51e6615e8
Author: Juergen Gross <jgross@suse.com>
Date:   Mon Feb 5 11:49:53 2024 +0100

    tools/xenstored: move all socket handling into posix.c
    
    All of the socket handling is needed only when running as daemon.
    
    Move it into posix.c, allowing to remove the NO_SOCKETS macro.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit aae54458a1b5e947dbc4fb44beca7ec94fe80001
Author: Juergen Gross <jgross@suse.com>
Date:   Mon Feb 5 11:49:52 2024 +0100

    tools/xenstored: move all log-pipe handling into posix.c
    
    All of the log-pipe handling is needed only when running as daemon.
    
    Move it into posix.c. This requires to have a service function in the
    main event loop for handling the related requests and one for setting
    the fds[] array, which is renamed to poll_fds to have a more specific
    name. Use a generic name for those functions, as socket handling can
    be added to them later, too.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit 511cc30cf8bbf083f2788cf4eee91edec0b85ac0
Author: Juergen Gross <jgross@suse.com>
Date:   Mon Feb 5 11:49:51 2024 +0100

    tools/xenstored: move systemd handling to posix.c
    
    Move systemd handling to a new late_init() function in posix.c.
    
    This prepares a future removal of the NO_SOCKETS macro.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit e767a98f040d8274b0f1dc991b9eefbc13b39e2d
Author: Juergen Gross <jgross@suse.com>
Date:   Mon Feb 5 11:49:50 2024 +0100

    tools/xenstored: add early_init() function
    
    Some xenstored initialization needs to be done in the daemon case only,
    so split it out into a new early_init() function being a stub in the
    stubdom case.
    
    Remove the call of talloc_enable_leak_report_full(), as it serves no
    real purpose: the daemon only ever exits due to a crash, in which case
    a log of talloc()ed memory hardly has any value.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit 1c2c7f790147ca003270b32da41550c04c7d15d2
Author: Juergen Gross <jgross@suse.com>
Date:   Mon Feb 5 11:49:47 2024 +0100

    tools/xenstored: rename xenbus_evtchn()
    
    Rename the xenbus_evtchn() function to get_xenbus_evtchn() in order to
    avoid two externally visible symbols with the same name when Xenstore-
    stubdom is being built with a Mini-OS with CONFIG_XENBUS set.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

commit 9c076988fe5f9d55a0ecb846549bdbdfddcf3841
Author: Juergen Gross <jgross@suse.com>
Date:   Mon Feb 5 11:49:46 2024 +0100

    tools/helpers: allocate xenstore event channel for xenstore stubdom
    
    In order to prepare support of PV frontends in xenstore-stubdom, add
    allocation of a Xenstore event channel to init-xenstore-domain.c.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 01:21:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 01:21:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676578.1052750 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8v-0007aF-MK; Tue, 06 Feb 2024 01:20:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676578.1052750; Tue, 06 Feb 2024 01:20:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8v-0007a8-JS; Tue, 06 Feb 2024 01:20:57 +0000
Received: by outflank-mailman (input) for mailman id 676578;
 Tue, 06 Feb 2024 01:20:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+4KK=JP=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXA8u-0007Lc-E7
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 01:20:56 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f9f6cc8f-c48d-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 02:20:55 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51109060d6aso7221716e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 17:20:55 -0800 (PST)
Received: from georged-x-u.eng.citrite.net ([185.25.67.249])
 by smtp.gmail.com with ESMTPSA id
 cu9-20020a170906ba8900b00a3726a5e5fdsm486803ejd.95.2024.02.05.17.20.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 17:20:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9f6cc8f-c48d-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707182455; x=1707787255; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=UyL8iRnNOrzO0tXt0n8exVji9pmykaMWjAoTvDaMaqU=;
        b=OWHkK360S0r4Ul1/tP7Q4uxYpaaG9VAlVUaP7/TbyXbh786wlEY2kaTyTRq4BsPlVU
         QmuJPmEcLKgmJVXq1hOTeN2s/sxU/Wk1F96HWzhXJZrIjm448KnLnPhRJgb9zho1d1GX
         N0u4XkO5N8l6QkysRwRaA2cXU/dXrfZAdnXAk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707182455; x=1707787255;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=UyL8iRnNOrzO0tXt0n8exVji9pmykaMWjAoTvDaMaqU=;
        b=Xaxyo93gf30AOsLQf9jh2woG/VDoH5BSkNm01b+omIk5ng8JQdljjs544KWzTmSpgo
         71YyNEMQKQE8A2QMp53F2AQur1RvTg8RUIsQI1URDevmNrlb0lBp6Ff2TL14fpesRjVk
         tpGLKvsv8Aj3Cmdo4EmMZagjVKaQgg+wTRkI0yHBoalFpyIjuKm8qNdUOtwsG9NuIDIz
         xLsKSRLF6nO76qA75w2FY7SnBOCg8UX23HoB6aJDbqp7W0EmTpZqzoKCQgcpwIdvpPDi
         3iujbUwccQJeeaZBol1arpx1kDJDkj1lAqS41O38BpRYfNJ2QtgtFcA+7+yHRL1/bMR2
         fMyA==
X-Gm-Message-State: AOJu0YxLJIIThWEEQsw+mUDHOIrPkANdC1nBU10+v/vHd+yqmoL8x5Vi
	UXYU9IyZjNA/rZvcU62iy0D71XApR6e0BAEBGX4KUEk2Gkfmi962OOtxXdmdT4l29/3vWdsX0J+
	NS00=
X-Google-Smtp-Source: AGHT+IFXWt1zdRs1Qg+qzCQpO7rNI1yCQJ/5UArCsgophMlXMPxm+ZJT0/q9e/bXuOnjUq/kUe4MRw==
X-Received: by 2002:ac2:533c:0:b0:511:525a:a527 with SMTP id f28-20020ac2533c000000b00511525aa527mr491189lfh.49.1707182455217;
        Mon, 05 Feb 2024 17:20:55 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXjUY728lp6AlPZ93eaAYRGO8gG2OWdp8QzNUYPwV1CfEh1Q0r0oPr6krNNWob5guieNOn3eVe3Gu+9KBHYlER6WPdwbRZqLCz7rCHjkEzZgfoCBomlAuH92z+4Ie1ROU1ie4HOtf/2mMMwopr+C9/ZXX5N0Frs0hDw8/lTyYlrrTLMQJO3HDR4N9Djn7JyScRfDWPxQqgMjrjKaakGd8+fDpW6sg4yRFnj321K
From: George Dunlap <george.dunlap@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>
Subject: [PATCH 3/6] xen/hvm: Move other hvm_function_table booleans into the caps bitfield
Date: Tue,  6 Feb 2024 01:20:48 +0000
Message-Id: <20240206012051.3564035-4-george.dunlap@cloud.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20240206012051.3564035-1-george.dunlap@cloud.com>
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Moving them all together has several advantages:
 * Collects them all in one part of the struct
 * The `caps` field means that we can drop the "_supported" suffix, as it's
   clear what is meant.

Signed-off-by: George Dunlap <george.dunlap@cloud.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: "Roger Pau Monné" <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Jun Nakajima <jun.nakajima@intel.com>
CC: Kevin Tian <kevin.tian@intel.com>
---
 xen/arch/x86/hvm/hvm.c             |  6 +++---
 xen/arch/x86/hvm/svm/svm.c         |  2 +-
 xen/arch/x86/hvm/vlapic.c          |  4 ++--
 xen/arch/x86/hvm/vmx/vmcs.c        |  2 +-
 xen/arch/x86/hvm/vmx/vmx.c         |  8 ++++----
 xen/arch/x86/include/asm/hvm/hvm.h | 29 ++++++++++++++---------------
 6 files changed, 25 insertions(+), 26 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index ae9d4c4756..aa2f2d054a 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -136,7 +136,7 @@ static struct notifier_block cpu_nfb = {
 
 static bool __init hap_supported(struct hvm_function_table *fns)
 {
-    if ( !fns->hap_supported )
+    if ( !fns->caps.hap )
     {
         printk("HVM: Hardware Assisted Paging (HAP) not detected\n");
         return false;
@@ -144,7 +144,7 @@ static bool __init hap_supported(struct hvm_function_table *fns)
 
     if ( !opt_hap_enabled )
     {
-        fns->hap_supported = 0;
+        fns->caps.hap = 0;
         printk("HVM: Hardware Assisted Paging (HAP) detected but disabled\n");
         return false;
     }
@@ -190,7 +190,7 @@ static int __init cf_check hvm_enable(void)
     }
 
     if ( !opt_altp2m_enabled )
-        hvm_funcs.altp2m_supported = 0;
+        hvm_funcs.caps.altp2m = 0;
 
     if ( opt_hvm_fep )
         warning_add(warning_hvm_fep);
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 40bc1ffbc6..b551eac807 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -2580,7 +2580,7 @@ const struct hvm_function_table * __init start_svm(void)
     if ( !printed )
         printk(" - none\n");
 
-    svm_function_table.hap_supported = cpu_has_svm_npt;
+    svm_function_table.caps.hap = cpu_has_svm_npt;
     svm_function_table.caps.hap_superpage_2mb = true;
     svm_function_table.caps.hap_superpage_1gb = cpu_has_page1gb;
 
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index 71a4b954b0..dcbcf4a1fe 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -1326,7 +1326,7 @@ int vlapic_has_pending_irq(struct vcpu *v)
     if ( irr == -1 )
         return -1;
 
-    if ( hvm_funcs.virtual_intr_delivery_enabled &&
+    if ( hvm_funcs.caps.virtual_intr_delivery &&
          !nestedhvm_vcpu_in_guestmode(v) )
         return irr;
 
@@ -1361,7 +1361,7 @@ int vlapic_ack_pending_irq(struct vcpu *v, int vector, bool force_ack)
     int isr;
 
     if ( !force_ack &&
-         hvm_funcs.virtual_intr_delivery_enabled )
+         hvm_funcs.caps.virtual_intr_delivery )
         return 1;
 
     /* If there's no chance of using APIC assist then bail now. */
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 53f9d81aa9..aff69d5320 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -112,7 +112,7 @@ static int cf_check parse_ept_param_runtime(const char *s)
     struct domain *d;
     int val;
 
-    if ( !cpu_has_vmx_ept || !hvm_funcs.hap_supported ||
+    if ( !cpu_has_vmx_ept || !hvm_funcs.caps.hap ||
          !(hvm_funcs.caps.hap_superpage_2mb ||
            hvm_funcs.caps.hap_superpage_1gb) )
     {
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 9cfc0140b4..4bcf436d2c 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -2963,7 +2963,7 @@ const struct hvm_function_table * __init start_vmx(void)
         return NULL;
     }
 
-    vmx_function_table.singlestep_supported = cpu_has_monitor_trap_flag;
+    vmx_function_table.caps.singlestep = cpu_has_monitor_trap_flag;
 
     if ( cpu_has_vmx_dt_exiting )
         vmx_function_table.set_descriptor_access_exiting =
@@ -2986,8 +2986,8 @@ const struct hvm_function_table * __init start_vmx(void)
                 printk("VMX: Disabling executable EPT superpages due to CVE-2018-12207\n");
         }
 
-        vmx_function_table.hap_supported = 1;
-        vmx_function_table.altp2m_supported = 1;
+        vmx_function_table.caps.hap = 1;
+        vmx_function_table.caps.altp2m = 1;
 
         vmx_function_table.caps.hap_superpage_2mb = cpu_has_vmx_ept_2mb;
         vmx_function_table.caps.hap_superpage_1gb = cpu_has_vmx_ept_1gb;
@@ -3000,7 +3000,7 @@ const struct hvm_function_table * __init start_vmx(void)
         vmx_function_table.update_eoi_exit_bitmap = vmx_update_eoi_exit_bitmap;
         vmx_function_table.process_isr = vmx_process_isr;
         vmx_function_table.handle_eoi = vmx_handle_eoi;
-        vmx_function_table.virtual_intr_delivery_enabled = true;
+        vmx_function_table.caps.virtual_intr_delivery = true;
     }
 
     if ( cpu_has_vmx_posted_intr_processing )
diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
index f50476f50f..bbd83a8275 100644
--- a/xen/arch/x86/include/asm/hvm/hvm.h
+++ b/xen/arch/x86/include/asm/hvm/hvm.h
@@ -86,20 +86,19 @@ struct hvm_vcpu_nonreg_state {
 struct hvm_function_table {
     const char *name;
 
-    /* Support Hardware-Assisted Paging? */
-    bool hap_supported;
-
-    /* Necessary hardware support for alternate p2m's? */
-    bool altp2m_supported;
-    bool singlestep_supported;
-
-    /* Hardware virtual interrupt delivery enable? */
-    bool virtual_intr_delivery_enabled;
-
     struct {
         /* Indicate HAP capabilities. */
-        bool hap_superpage_1gb:1,
-            hap_superpage_2mb:1;
+        bool hap:1,
+             hap_superpage_1gb:1,
+             hap_superpage_2mb:1,
+
+            /* Altp2m capabilities */
+            altp2m:1,
+            singlestep:1,
+            
+            /* Hardware virtual interrupt delivery enable? */
+            virtual_intr_delivery;
+
     } caps;
 
     /*
@@ -642,18 +641,18 @@ static inline void hvm_enable_msr_interception(struct domain *d, uint32_t msr)
 
 static inline bool hvm_is_singlestep_supported(void)
 {
-    return hvm_funcs.singlestep_supported;
+    return hvm_funcs.caps.singlestep;
 }
 
 static inline bool hvm_hap_supported(void)
 {
-    return hvm_funcs.hap_supported;
+    return hvm_funcs.caps.hap;
 }
 
 /* returns true if hardware supports alternate p2m's */
 static inline bool hvm_altp2m_supported(void)
 {
-    return hvm_funcs.altp2m_supported;
+    return hvm_funcs.caps.altp2m;
 }
 
 /* updates the current hardware p2m */
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 01:21:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 01:21:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676579.1052756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8w-0007dl-4b; Tue, 06 Feb 2024 01:20:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676579.1052756; Tue, 06 Feb 2024 01:20:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8v-0007dS-ST; Tue, 06 Feb 2024 01:20:57 +0000
Received: by outflank-mailman (input) for mailman id 676579;
 Tue, 06 Feb 2024 01:20:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+4KK=JP=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXA8v-0007Zy-6V
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 01:20:57 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f96eec07-c48d-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 02:20:55 +0100 (CET)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a271a28aeb4so714699466b.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 17:20:55 -0800 (PST)
Received: from georged-x-u.eng.citrite.net ([185.25.67.249])
 by smtp.gmail.com with ESMTPSA id
 cu9-20020a170906ba8900b00a3726a5e5fdsm486803ejd.95.2024.02.05.17.20.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 17:20:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f96eec07-c48d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707182453; x=1707787253; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Z/yrXJJsJhqpr0r2UtYZz4VSbTISscV6gheiVUuHW1M=;
        b=O3hzLTzQltj0zjGpOT5vQMX96rLR3LiJlBi++aF/6sjc7DYBSFjKBKDPTopZ5HRvrw
         JFP80InIu+spPZqtxfnx6bGXPq+XnkOuutWnPqDxyYbmWQUdZMcX6S7kHISQ7qvPhG/W
         Oq2vPAEBBhI8/6rihjec6GIqVZht7nHz64h18=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707182453; x=1707787253;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Z/yrXJJsJhqpr0r2UtYZz4VSbTISscV6gheiVUuHW1M=;
        b=E9GWTo1n5dllHFX/LTI2EaF9yQ3WvySuPCv7eTc8FE4KqXe+pRcAb9KLzz+tKtKNzK
         qBZ6A1kvo4wxhqIME95/gBys27PtY+Yem7cD4KHCzmsD/1E9rSRbZd33r+BxASbiAhqv
         UWzz+Md4RwxeyI+dlJI76pPGpNwxZu3pcn+olHya2ltVDx196dvuClMMaBg7XtH+wcvO
         vxTFhsEVWbx5Fz1T1P9WCYUyZB680O5EUYnHTOlStt4DtCk0P46uHeAoKz4LVkVUBxHv
         Wtvg/WC0ceN3ITxjK2uLcJaU0FjRIe8PCVbSOYAptHGGGZrOAszkm447ZXEXaVNGxXdq
         rFBw==
X-Gm-Message-State: AOJu0YzTKy/LJpqfTDsGZQOaa+jcmFA8Ynn1MXgsTARO1c0RF0UYHk48
	tcP706Gq2PYaZxdxiuVwTXqLC9I4+NFRXEOlxrWcCoe8V+PhH94e+mlw9EKDqy2eqLlKwke3UcP
	twho=
X-Google-Smtp-Source: AGHT+IFz7wN1YZXcim1jMkDN2nlXE79nJzM5CRYWjwIr9wKdmhWJhE07gZMct6I6hchSFwuOHrq0Jg==
X-Received: by 2002:a17:906:f149:b0:a35:42d0:5d61 with SMTP id gw9-20020a170906f14900b00a3542d05d61mr742736ejb.58.1707182453601;
        Mon, 05 Feb 2024 17:20:53 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUlrbboXAtOCOdIpCONVHZAzqIuLKcoSWYRquXj2dK9z5g87ypseXi3osVX17hipHn9hXCYvSjhOaBcru5PZs50K4ShNJ0O+YxgOktx8rPzXUOBWsHWlB/xkTgkZojHWu0RPuZIBFOs+vMnIXILW30W/fldC8dfXlZzQ0hd7fM=
From: George Dunlap <george.dunlap@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 2/6] svm: Improve type of cpu_has_svm_feature
Date: Tue,  6 Feb 2024 01:20:47 +0000
Message-Id: <20240206012051.3564035-3-george.dunlap@cloud.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20240206012051.3564035-1-george.dunlap@cloud.com>
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The "effective type" of the cpu_has_svm_feature macro is effectively
an unsigned log with one bit set (or not); at least one place someone
felt compelled to do a !! to make sure that they got a boolean out of
it.

Ideally the whole of this would be folded into the cpufeature.h
infrastructure.  But for now, duplicate the more type-safe static
inlines in that file, and remove the !!.

Signed-off-by: George Dunlap <george.dunlap@cloud.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: "Roger Pau Monné" <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 xen/arch/x86/hvm/svm/svm.c             | 2 +-
 xen/arch/x86/include/asm/hvm/svm/svm.h | 5 ++++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 5741287355..40bc1ffbc6 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -2580,7 +2580,7 @@ const struct hvm_function_table * __init start_svm(void)
     if ( !printed )
         printk(" - none\n");
 
-    svm_function_table.hap_supported = !!cpu_has_svm_npt;
+    svm_function_table.hap_supported = cpu_has_svm_npt;
     svm_function_table.caps.hap_superpage_2mb = true;
     svm_function_table.caps.hap_superpage_1gb = cpu_has_page1gb;
 
diff --git a/xen/arch/x86/include/asm/hvm/svm/svm.h b/xen/arch/x86/include/asm/hvm/svm/svm.h
index 687d35be40..9e9a148845 100644
--- a/xen/arch/x86/include/asm/hvm/svm/svm.h
+++ b/xen/arch/x86/include/asm/hvm/svm/svm.h
@@ -38,7 +38,10 @@ extern u32 svm_feature_flags;
 #define SVM_FEATURE_SSS           19 /* NPT Supervisor Shadow Stacks */
 #define SVM_FEATURE_SPEC_CTRL     20 /* MSR_SPEC_CTRL virtualisation */
 
-#define cpu_has_svm_feature(f) (svm_feature_flags & (1u << (f)))
+static inline bool cpu_has_svm_feature(unsigned int feat)
+{
+    return svm_feature_flags & (1u << (feat));
+}
 #define cpu_has_svm_npt       cpu_has_svm_feature(SVM_FEATURE_NPT)
 #define cpu_has_svm_lbrv      cpu_has_svm_feature(SVM_FEATURE_LBRV)
 #define cpu_has_svm_svml      cpu_has_svm_feature(SVM_FEATURE_SVML)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 01:21:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 01:21:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676581.1052777 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8x-0008Bh-Ri; Tue, 06 Feb 2024 01:20:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676581.1052777; Tue, 06 Feb 2024 01:20:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8x-0008Ae-N6; Tue, 06 Feb 2024 01:20:59 +0000
Received: by outflank-mailman (input) for mailman id 676581;
 Tue, 06 Feb 2024 01:20:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+4KK=JP=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXA8x-0007Lc-70
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 01:20:59 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fb987c98-c48d-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 02:20:58 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a26f73732c5so724843966b.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 17:20:58 -0800 (PST)
Received: from georged-x-u.eng.citrite.net ([185.25.67.249])
 by smtp.gmail.com with ESMTPSA id
 cu9-20020a170906ba8900b00a3726a5e5fdsm486803ejd.95.2024.02.05.17.20.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 17:20:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb987c98-c48d-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707182457; x=1707787257; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GdeAbZaD683kMJ6A0ydqo1sa61D2UtSXLjxCScOD/n8=;
        b=IDsQqPYuYB86ldYJXs5m1Xro5/xh67OKgg3LzqNjFKhNpfpo5bqwCY77cei6PsJz1x
         3v0pX9w4Wl7Vib0JumZocoDkegrPtx7E7DBLrwprHFOFQcoAqY7ofdTr+NWjrwANyke6
         EpjArEG+dWZyQAOUyBcP8W7+jThdB18o5uhqU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707182457; x=1707787257;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GdeAbZaD683kMJ6A0ydqo1sa61D2UtSXLjxCScOD/n8=;
        b=Q5Jhl0TGblFQgLZi12quN6QfYaJrh37f9f2VYqIK3jMdyiL79Jz9Qd+QRrX6VJyYyu
         EoD+NDvKGCcDSoLH9IyB4R7RLITq2dp9fta8WK6MC2xa4c1Rwd2wQbi5TLV07j6bbvSb
         J8ZuhqzfL24/8ODWHLl0PNn9gM+uQlkAqa8nIB7+RZHOcryl/dI1PaYduYzT1MJlbr/i
         TuIzv/g42WF1aW0c7weOgwJDtKsQWD88Bs0nMfkhR/0Cov8Eu/pN7MneCd2rpdUM7/4y
         dLqpnkLBezFYuk4OcdRql7YS4Qn2gqt6lc6iZ3X5tHNJoN58TlYwM7O5Zom5bLbVlQ90
         4izA==
X-Gm-Message-State: AOJu0YzKGUXbYLc7zzguyha3LFFqXBvMF5T4zY9HLk/XuBgFiWtC+W6E
	cAI7jgPOYlKlQWAs14qV5qL42vYqvBdYExp8QKnUPJwT6PVlkflWCGvixGura9cDzjyalgU0IYF
	xejk=
X-Google-Smtp-Source: AGHT+IGzy0Am05FrIfvfRq++RKAyzFHRmhuYgfQ+0JosP6aof8bSAW7/VRp8UboTAEikVUvAz8njIw==
X-Received: by 2002:a17:906:882:b0:a37:1dee:a16f with SMTP id n2-20020a170906088200b00a371deea16fmr757939eje.71.1707182457128;
        Mon, 05 Feb 2024 17:20:57 -0800 (PST)
From: George Dunlap <george.dunlap@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@cloud.com>
Subject: [PATCH 5/6] nestedsvm: Remove bogus debug message from nestedsvm_check_intercepts
Date: Tue,  6 Feb 2024 01:20:50 +0000
Message-Id: <20240206012051.3564035-6-george.dunlap@cloud.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20240206012051.3564035-1-george.dunlap@cloud.com>
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Changeset ef3e8db8068 ("x86/hvm: Corrections and improvements to
unhandled vmexit logging") introduced a printk to the default path of
the switch statement in nestedsvm_check_intercepts(), complaining of
an unknown exit reason.

Unfortunately, the "core" switch statement which is meant to handle
all vmexit reasons is in nsvm_vmcb_guest_intercepts_exitcode(); the
switch statement in nestedsvm_check_intercepts() is only meant to
superimpose on top of that some special-casing for how to interaction
between L1 and L0 vmexits.

Remove the printk, and add a comment to prevent future confusion.

Signed-off-by: George Dunlap <george.dunlap@cloud.com>
---
 xen/arch/x86/hvm/svm/nestedsvm.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/nestedsvm.c b/xen/arch/x86/hvm/svm/nestedsvm.c
index d02a59f184..a5319ab729 100644
--- a/xen/arch/x86/hvm/svm/nestedsvm.c
+++ b/xen/arch/x86/hvm/svm/nestedsvm.c
@@ -1292,6 +1292,10 @@ nestedsvm_check_intercepts(struct vcpu *v, struct cpu_user_regs *regs,
     ASSERT(vcpu_nestedhvm(v).nv_vmexit_pending == 0);
     is_intercepted = nsvm_vmcb_guest_intercepts_exitcode(v, regs, exitcode);
 
+    /* 
+     * Handle specific interactions between things the guest and host
+     * may both want to intercept
+     */
     switch ( exitcode )
     {
     case VMEXIT_INVALID:
@@ -1347,8 +1351,6 @@ nestedsvm_check_intercepts(struct vcpu *v, struct cpu_user_regs *regs,
         /* Always let the guest handle VMMCALL/VMCALL */
         return NESTEDHVM_VMEXIT_INJECT;
     default:
-        gprintk(XENLOG_ERR, "Unexpected nested vmexit: reason %#"PRIx64"\n",
-                exitcode);
         break;
     }
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 01:21:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 01:21:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676577.1052740 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8t-0007Lq-Fz; Tue, 06 Feb 2024 01:20:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676577.1052740; Tue, 06 Feb 2024 01:20:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8t-0007Lj-C0; Tue, 06 Feb 2024 01:20:55 +0000
Received: by outflank-mailman (input) for mailman id 676577;
 Tue, 06 Feb 2024 01:20:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+4KK=JP=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXA8s-0007Lc-4c
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 01:20:54 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f83e2c02-c48d-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 02:20:53 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-511344235c6so6207961e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 17:20:52 -0800 (PST)
Received: from georged-x-u.eng.citrite.net ([185.25.67.249])
 by smtp.gmail.com with ESMTPSA id
 cu9-20020a170906ba8900b00a3726a5e5fdsm486803ejd.95.2024.02.05.17.20.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 17:20:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f83e2c02-c48d-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707182452; x=1707787252; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=PFj8HgVee9CL5hZuq7phfC1KZf2NA8rTrVNFEifSRSA=;
        b=MeJl3dQjKCWjNWPLOwEG56vON5yxO35mrixTbR40Ri5eTtBRWNraaV9ixqyeIbq9xO
         SePK0pBnD4HVIu1ueTln9nEt4ZGzQQ6X+dIGR0e/mBggmotb1/hn5z8gN9hibNjiGM/T
         h9WiYISu3csH6AunmlSkwEsUPVh1fItnjrPNw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707182452; x=1707787252;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=PFj8HgVee9CL5hZuq7phfC1KZf2NA8rTrVNFEifSRSA=;
        b=kJIig/95vbBNSw9gnot71zG4nI+1qvG9/E73Zu3O300KGZJkcg1g30Av8WEGeoapoC
         m2nQ7csdrNG1nhblcZN09ehyX3VQHlZGyF8qcID25hOqYzlBRV08zyPuhYgRyR7/lOdQ
         YHVafeiwkK56LfIEgEB98qtvjPO6bGNo1ccXSYcfPKgXlAUsS6udcQsqJ8kH51xxYvHT
         tjclvCMrb2pPij2NaAGaLfMXLIALqxW6B5AtBq+tJCAn8O5ihlZgD2+RJIcAu7Hp6JDQ
         eReue0zWkU0MguyJxb2gC7qG4R4ooBXO4JFb7jRf73eqfndpGUHy5r9Q0egYI+pt68cr
         hJ5g==
X-Gm-Message-State: AOJu0YxnFxXd5ugOrDiCHHOpp4Uo1gUQg9XX5g5OBZwEcV03+4+VaJo0
	0W8E0flujNbYsWhtta45RAIz7dFdpnTtXfgc4mCCbfq0avCzX0huy1P2Vk3Solwp2PThiBVr7ZS
	D6/U=
X-Google-Smtp-Source: AGHT+IEIXw3PeeC2SheiJ2UFfimdTWI3ueOlX7KSPtKagS+F+N+GgcQi4W9jueJ7L7tA2EfS08nmDg==
X-Received: by 2002:a05:6512:318e:b0:511:5ca2:350d with SMTP id i14-20020a056512318e00b005115ca2350dmr356805lfe.47.1707182451818;
        Mon, 05 Feb 2024 17:20:51 -0800 (PST)
From: George Dunlap <george.dunlap@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@cloud.com>
Subject: [PATCH 0/6] AMD Nested Virt Preparation
Date: Tue,  6 Feb 2024 01:20:45 +0000
Message-Id: <20240206012051.3564035-1-george.dunlap@cloud.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This series lays the groundwork for revamp of the AMD nested virt
functionality.  The first five patches are clean-ups or reorganizations
of existing code.  The final patch is the first major step towards making
the feature supportable: allowing Xen to refuse nested virt support if certain
hardware features are not present.

George Dunlap (6):
  xen/hvm: Convert hap_capabilities into a bitfield
  svm: Improve type of cpu_has_svm_feature
  xen/hvm: Move other hvm_function_table booleans into the caps bitfield
  nestedsvm: Disable TscRateMSR
  nestedsvm: Remove bogus debug message from nestedsvm_check_intercepts
  svm/nestedvm: Introduce nested capabilities bit

 docs/designs/nested-svm-cpu-features.md      | 110 +++++++++++++++++++
 xen/arch/x86/cpu-policy.c                    |   3 +-
 xen/arch/x86/domain.c                        |   6 +
 xen/arch/x86/hvm/hvm.c                       |  14 +--
 xen/arch/x86/hvm/svm/nestedhvm.h             |   1 +
 xen/arch/x86/hvm/svm/nestedsvm.c             |  22 +++-
 xen/arch/x86/hvm/svm/svm.c                   |  65 +----------
 xen/arch/x86/hvm/vlapic.c                    |   4 +-
 xen/arch/x86/hvm/vmx/vmcs.c                  |   6 +-
 xen/arch/x86/hvm/vmx/vmx.c                   |  19 ++--
 xen/arch/x86/include/asm/hvm/hvm.h           |  47 ++++----
 xen/arch/x86/include/asm/hvm/svm/nestedsvm.h |   5 -
 xen/arch/x86/include/asm/hvm/svm/svm.h       |   5 +-
 13 files changed, 191 insertions(+), 116 deletions(-)
 create mode 100644 docs/designs/nested-svm-cpu-features.md

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 01:21:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 01:21:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676580.1052771 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8x-00083h-Af; Tue, 06 Feb 2024 01:20:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676580.1052771; Tue, 06 Feb 2024 01:20:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8x-00083U-5H; Tue, 06 Feb 2024 01:20:59 +0000
Received: by outflank-mailman (input) for mailman id 676580;
 Tue, 06 Feb 2024 01:20:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+4KK=JP=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXA8v-0007Zy-R6
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 01:20:57 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f9161d00-c48d-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 02:20:55 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a364b5c5c19so45634966b.1
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 17:20:54 -0800 (PST)
Received: from georged-x-u.eng.citrite.net ([185.25.67.249])
 by smtp.gmail.com with ESMTPSA id
 cu9-20020a170906ba8900b00a3726a5e5fdsm486803ejd.95.2024.02.05.17.20.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 17:20:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9161d00-c48d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707182453; x=1707787253; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Amao/RbYbGVlKRe+4sVFHaBWBTnz/mv5zt9jnAf2gO8=;
        b=LKNiowik0XnHDD0CqYaXV0eLUrXBTSvg1XkU7YuEpWLAXFJ80hcDktP0gShj+pV3AT
         WtG38gll1uEeNGdM1K9IrwQSK+5o5lsVueOpRKIgnujkfJ2hI8SRqLgjDKvPwo6sxclk
         Rm8eANHeT3XU82laFliNWDepOKq4A+A7PFoxM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707182453; x=1707787253;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Amao/RbYbGVlKRe+4sVFHaBWBTnz/mv5zt9jnAf2gO8=;
        b=D/G1T5MTKT6Qlfs/kirS2onFYTvY/z2fnYKItZSUL8ABOrRpFj6icsXc2NA+xN3Jwh
         Qr3c4VUnjpEeNTWuQ7wB4Jm6k9SKUQ9Hv2sb3R50rHFil1XmkzR5SG7UZZV8S4YoYSfA
         LJrdrr6yfHwQ6+A8xevivITZvVL9saDInx9sS7R1hF5SKbRFCfAEnPTUW4KfysKksO4Z
         OInKfIzWpqIwsa+E3qn/XSbPTXPqg66UeCmKIEn5DpBgqHPwLb7PBhLjatOfJDVxZUdT
         TbG7kAKV3/d9rsgbeok2B3tSRwN75ExTZ1ie3nmzthAzFc0tTUqqAP7XIh1o1zPVzJpm
         CvSg==
X-Gm-Message-State: AOJu0YztnstcGKhQtCmXakjc4lJyiT7kqMsLD+zSeRiLZk2MvAmePNDk
	yY2CzY1RkdviUcoX+YXImYPvW6yJnpW7Lj+Af1gCXT7Vd1A/F+a1iAiZ8r4WJ5Dh1x3Bwckvhx3
	x8mA=
X-Google-Smtp-Source: AGHT+IHafK91jBtYVmLvGZnuwJyOkbsecn2SYHRWDD7tDRE43HnCVB/X8GZ1kkmM7X/wQATl2fHBYA==
X-Received: by 2002:a17:906:4095:b0:a37:2696:b3e with SMTP id u21-20020a170906409500b00a3726960b3emr862442ejj.59.1707182453136;
        Mon, 05 Feb 2024 17:20:53 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUgi6wyoYuGerkTH4fh6jBCjoCFbYJKmhCh45bbre+82qjLmXHDSP4F7NpPugLeTkxNPBKzt8DFEM0/JzDXgKYdYw4Ln4X0kDpJr7xaCv1MEf82JS4Y+Sv1h9R3+GcXonCWSGgasb8BlSk+LBzaNMumFX5tq/AtmA7c0yvwg4YXKBmsEzpHXs9MSC9VSYcWxBvKjT1vfOnDmPw06ItZTcoZuti96CDA7bzu+Brv
From: George Dunlap <george.dunlap@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>
Subject: [PATCH 1/6] xen/hvm: Convert hap_capabilities into a bitfield
Date: Tue,  6 Feb 2024 01:20:46 +0000
Message-Id: <20240206012051.3564035-2-george.dunlap@cloud.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20240206012051.3564035-1-george.dunlap@cloud.com>
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

hvm_function_table is an internal structure; rather than manually
|-ing and &-ing bits, just make it a boolean bitfield and let the
compiler do all the work.  This makes everything easier to read, and
presumably allows the compiler more flexibility in producing efficient
code.

No functional change intended.

Signed-off-by: George Dunlap <george.dunlap@cloud.com>
---
Questions:

* Should hap_superpage_2m really be set unconditionally, or should we
  condition it on cpu_has_svm_npt?

* Do we really need to "!!cpu_has_svm_npt"?  If so, wouldn't it be
  better to put the "!!"  in the #define, rather than requiring the
  user to know that it's needed?

CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: "Roger Pau Monné" <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Jun Nakajima <jun.nakajima@intel.com>
CC: Kevin Tian <kevin.tian@intel.com>
---
 xen/arch/x86/hvm/hvm.c             |  8 ++++----
 xen/arch/x86/hvm/svm/svm.c         |  4 ++--
 xen/arch/x86/hvm/vmx/vmcs.c        |  4 ++--
 xen/arch/x86/hvm/vmx/vmx.c         |  8 ++------
 xen/arch/x86/include/asm/hvm/hvm.h | 19 +++++++------------
 5 files changed, 17 insertions(+), 26 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index e8deeb0222..ae9d4c4756 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -174,17 +174,17 @@ static int __init cf_check hvm_enable(void)
     {
         printk("HVM: Hardware Assisted Paging (HAP) detected\n");
         printk("HVM: HAP page sizes: 4kB");
-        if ( fns->hap_capabilities & HVM_HAP_SUPERPAGE_2MB )
+        if ( fns->caps.hap_superpage_2mb )
         {
             printk(", 2MB%s", opt_hap_2mb ? "" : " [disabled]");
             if ( !opt_hap_2mb )
-                hvm_funcs.hap_capabilities &= ~HVM_HAP_SUPERPAGE_2MB;
+                hvm_funcs.caps.hap_superpage_2mb = false;
         }
-        if ( fns->hap_capabilities & HVM_HAP_SUPERPAGE_1GB )
+        if ( fns->caps.hap_superpage_1gb )
         {
             printk(", 1GB%s", opt_hap_1gb ? "" : " [disabled]");
             if ( !opt_hap_1gb )
-                hvm_funcs.hap_capabilities &= ~HVM_HAP_SUPERPAGE_1GB;
+                hvm_funcs.caps.hap_superpage_1gb = false;
         }
         printk("\n");
     }
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 65f437e958..5741287355 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -2581,8 +2581,8 @@ const struct hvm_function_table * __init start_svm(void)
         printk(" - none\n");
 
     svm_function_table.hap_supported = !!cpu_has_svm_npt;
-    svm_function_table.hap_capabilities = HVM_HAP_SUPERPAGE_2MB |
-        (cpu_has_page1gb ? HVM_HAP_SUPERPAGE_1GB : 0);
+    svm_function_table.caps.hap_superpage_2mb = true;
+    svm_function_table.caps.hap_superpage_1gb = cpu_has_page1gb;
 
     return &svm_function_table;
 }
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 4fe1213855..53f9d81aa9 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -113,8 +113,8 @@ static int cf_check parse_ept_param_runtime(const char *s)
     int val;
 
     if ( !cpu_has_vmx_ept || !hvm_funcs.hap_supported ||
-         !(hvm_funcs.hap_capabilities &
-           (HVM_HAP_SUPERPAGE_2MB | HVM_HAP_SUPERPAGE_1GB)) )
+         !(hvm_funcs.caps.hap_superpage_2mb ||
+           hvm_funcs.caps.hap_superpage_1gb) )
     {
         printk("VMX: EPT not available, or not in use - ignoring\n");
         return 0;
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 1500dca603..9cfc0140b4 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -2989,12 +2989,8 @@ const struct hvm_function_table * __init start_vmx(void)
         vmx_function_table.hap_supported = 1;
         vmx_function_table.altp2m_supported = 1;
 
-        vmx_function_table.hap_capabilities = 0;
-
-        if ( cpu_has_vmx_ept_2mb )
-            vmx_function_table.hap_capabilities |= HVM_HAP_SUPERPAGE_2MB;
-        if ( cpu_has_vmx_ept_1gb )
-            vmx_function_table.hap_capabilities |= HVM_HAP_SUPERPAGE_1GB;
+        vmx_function_table.caps.hap_superpage_2mb = cpu_has_vmx_ept_2mb;
+        vmx_function_table.caps.hap_superpage_1gb = cpu_has_vmx_ept_1gb;
 
         setup_ept_dump();
     }
diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
index 985c1c14c6..f50476f50f 100644
--- a/xen/arch/x86/include/asm/hvm/hvm.h
+++ b/xen/arch/x86/include/asm/hvm/hvm.h
@@ -61,14 +61,6 @@ enum hvm_intblk {
 #define HVM_INTR_SHADOW_SMI    0x00000004
 #define HVM_INTR_SHADOW_NMI    0x00000008
 
-/*
- * HAP super page capabilities:
- * bit0: if 2MB super page is allowed?
- * bit1: if 1GB super page is allowed?
- */
-#define HVM_HAP_SUPERPAGE_2MB   0x00000001
-#define HVM_HAP_SUPERPAGE_1GB   0x00000002
-
 #define HVM_EVENT_VECTOR_UNSET    (-1)
 #define HVM_EVENT_VECTOR_UPDATING (-2)
 
@@ -104,8 +96,11 @@ struct hvm_function_table {
     /* Hardware virtual interrupt delivery enable? */
     bool virtual_intr_delivery_enabled;
 
-    /* Indicate HAP capabilities. */
-    unsigned int hap_capabilities;
+    struct {
+        /* Indicate HAP capabilities. */
+        bool hap_superpage_1gb:1,
+            hap_superpage_2mb:1;
+    } caps;
 
     /*
      * Initialise/destroy HVM domain/vcpu resources
@@ -402,8 +397,8 @@ int hvm_get_param(struct domain *d, uint32_t index, uint64_t *value);
     (hvm_paging_enabled(v) && ((v)->arch.hvm.guest_cr[4] & X86_CR4_PKS))
 
 /* Can we use superpages in the HAP p2m table? */
-#define hap_has_1gb (!!(hvm_funcs.hap_capabilities & HVM_HAP_SUPERPAGE_1GB))
-#define hap_has_2mb (!!(hvm_funcs.hap_capabilities & HVM_HAP_SUPERPAGE_2MB))
+#define hap_has_1gb hvm_funcs.caps.hap_superpage_1gb
+#define hap_has_2mb hvm_funcs.caps.hap_superpage_2mb
 
 #define hvm_long_mode_active(v) (!!((v)->arch.hvm.guest_efer & EFER_LMA))
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 01:21:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 01:21:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676582.1052790 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8z-00006V-4G; Tue, 06 Feb 2024 01:21:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676582.1052790; Tue, 06 Feb 2024 01:21:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA8y-00006L-UF; Tue, 06 Feb 2024 01:21:00 +0000
Received: by outflank-mailman (input) for mailman id 676582;
 Tue, 06 Feb 2024 01:20:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+4KK=JP=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXA8x-0007Zy-PD
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 01:20:59 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fb2ca541-c48d-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 02:20:58 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a36126ee41eso631826366b.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 17:20:58 -0800 (PST)
Received: from georged-x-u.eng.citrite.net ([185.25.67.249])
 by smtp.gmail.com with ESMTPSA id
 cu9-20020a170906ba8900b00a3726a5e5fdsm486803ejd.95.2024.02.05.17.20.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 17:20:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb2ca541-c48d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707182456; x=1707787256; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=L70F5M+Yt0degs+BEb21BR1J/47RzHBiPjSDLgFIrFk=;
        b=j1EYUQ+fHKnTmqxI0VGdNQ9+dFBX/mF06iiq9gWHcVDBC5n+b5QaeSk8SmHxFSrqFE
         kBd+ISy/Kafa9rP272XnDVTSsQ4/FJZeCK1PFkqgL/feoFT6sA1djl+SK/c9bYvc6SIN
         89Vao6Z+tTAR+CGjNPmtOEwtFRY6oeFc05lUU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707182456; x=1707787256;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=L70F5M+Yt0degs+BEb21BR1J/47RzHBiPjSDLgFIrFk=;
        b=h37pmgz5+BpVPJQ2u17vd2A6jMT4VAQMw7wcefuN1LgbF5rG4s8YuJ19yWkxETy/DZ
         mgK+gxkzYKVCCEI6/ia30MkQU0JdH7kFBCfvST4E+Gyaf6QA16WaoRPJKp7rvNQGS1Um
         sxWFLdyTZF0C1xvajiD3835OFVHiZt+uYtgZfFuV49A41TJH06rx7uU18NKGit5XoO+M
         hfKaLqP1ednf+9FuAHeLSlBf6hI84n7Akb1kB5JyUVdbKq2NqKNreyUG0U9FMQUyXF99
         jdwbHO46bE0RBY+ADMkCtCxM11EfTzjPKDA61wNrITDOPDHGOALYB0Fr6j9XWXh7S16G
         ooiw==
X-Gm-Message-State: AOJu0YwEh+WfO5aO8EyW7YilnjAzX6WMZLziO9yZUExwfLksvrvl/46S
	mD0lx7ZeUWTjpp3po++a83v+vfpVEIQioCi96z+A3cgpsUVMKnJQvxIReEHNlzT91sIYH36qY1s
	Z2dE=
X-Google-Smtp-Source: AGHT+IEXW5qmvPYuFsiInq8Uq3VpZtdGvVqLvFwDeeB+2lRmMIo5xrqr5HiC4wRrxRfOjM99F7cohw==
X-Received: by 2002:a17:906:2898:b0:a35:fdf9:e7e4 with SMTP id o24-20020a170906289800b00a35fdf9e7e4mr383456ejd.20.1707182456690;
        Mon, 05 Feb 2024 17:20:56 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUdFVxDXqUTy+KWuGDk9yEfcrUsYbrUs0c2vEvbCgp4OIQ4VwxWTe0XpXtjdTOM7vPlMl4shxt7HWuNdwmidnyMU7/6r2m9xA7zHup0o0TUvBeGNE0RNNNS+6A18/SRccuDndw6os5gq7hMMgpaLvPcTCJs7wREdDrF1KmzBaY=
From: George Dunlap <george.dunlap@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 4/6] nestedsvm: Disable TscRateMSR
Date: Tue,  6 Feb 2024 01:20:49 +0000
Message-Id: <20240206012051.3564035-5-george.dunlap@cloud.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20240206012051.3564035-1-george.dunlap@cloud.com>
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The primary purpose of TSC scaling, from our perspective, is to
maintain the fiction of an "invariant TSC" across migrates between
platforms with different clock speeds.

On AMD, the TscRateMSR CPUID bit is unconditionally enabled in the
"host cpuid", even if the hardware doesn't actually support it.
According to c/s fd14a1943c4 ("nestedsvm: Support TSC Rate MSR"),
testing showed that emulating TSC scaling in an L1 was more expensive
than emulating TSC scaling on an L0 (due to extra sets of vmexit /
vmenter).

However, the current implementation seems to be broken.

First of all, the final L2 scaling ratio should be a composition of
the L0 scaling ratio and the L1 scaling ratio; there's no indication
this is being done anywhere.

Secondly, it's not clear that the L1 tsc scaling ratio actually
affects the L0 tsc scaling ratio.  The stored value (ns_tscratio) is
used to affect the tsc *offset*, but doesn't seem to actually be
factored into d->hvm.tsc_scaling_ratio.  (Which shouldn't be
per-domain anyway, but per-vcpu.)  Having the *offset* scaled
according to the nested scaling without the actual RDTSC itself also
being scaled has got to produce inconsistent results.

For now, just disable the functionality entirely until we can
implement it properly:

- Don't set TSCRATEMSR in the host CPUID policy

- Remove MSR_AMD64_TSC_RATIO emulation handling, so that the guest
  guests a #GP if it tries to access them (as it should when
  TSCRATEMSR is clear)

- Remove ns_tscratio from struct nestedhvm, and all code that touches
  it

Unfortunately this means ripping out the scaling calculation stuff as
well, since it's only used in the nested case; it's there in the git
tree if we need it for reference when we re-introduce it.

Signed-off-by: George Dunlap <george.dunlap@cloud.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: "Roger Pau Monné" <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 xen/arch/x86/cpu-policy.c                    |  3 +-
 xen/arch/x86/hvm/svm/nestedsvm.c             |  2 -
 xen/arch/x86/hvm/svm/svm.c                   | 57 --------------------
 xen/arch/x86/include/asm/hvm/svm/nestedsvm.h |  5 --
 4 files changed, 1 insertion(+), 66 deletions(-)

diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index 10079c26ae..d71abbc44a 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -407,8 +407,7 @@ static void __init calculate_host_policy(void)
                                (1u << SVM_FEATURE_PAUSEFILTER) |
                                (1u << SVM_FEATURE_DECODEASSISTS));
         /* Enable features which are always emulated. */
-        p->extd.raw[0xa].d |= ((1u << SVM_FEATURE_VMCBCLEAN) |
-                               (1u << SVM_FEATURE_TSCRATEMSR));
+        p->extd.raw[0xa].d |= (1u << SVM_FEATURE_VMCBCLEAN);
     }
 
     /* 0x000000ce  MSR_INTEL_PLATFORM_INFO */
diff --git a/xen/arch/x86/hvm/svm/nestedsvm.c b/xen/arch/x86/hvm/svm/nestedsvm.c
index ee9602f5c8..d02a59f184 100644
--- a/xen/arch/x86/hvm/svm/nestedsvm.c
+++ b/xen/arch/x86/hvm/svm/nestedsvm.c
@@ -146,8 +146,6 @@ int cf_check nsvm_vcpu_reset(struct vcpu *v)
     svm->ns_msr_hsavepa = INVALID_PADDR;
     svm->ns_ovvmcb_pa = INVALID_PADDR;
 
-    svm->ns_tscratio = DEFAULT_TSC_RATIO;
-
     svm->ns_cr_intercepts = 0;
     svm->ns_dr_intercepts = 0;
     svm->ns_exception_intercepts = 0;
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index b551eac807..34b9f603bc 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -777,43 +777,6 @@ static int cf_check svm_get_guest_pat(struct vcpu *v, u64 *gpat)
     return 1;
 }
 
-static uint64_t scale_tsc(uint64_t host_tsc, uint64_t ratio)
-{
-    uint64_t mult, frac, scaled_host_tsc;
-
-    if ( ratio == DEFAULT_TSC_RATIO )
-        return host_tsc;
-
-    /*
-     * Suppose the most significant 32 bits of host_tsc and ratio are
-     * tsc_h and mult, and the least 32 bits of them are tsc_l and frac,
-     * then
-     *     host_tsc * ratio * 2^-32
-     *     = host_tsc * (mult * 2^32 + frac) * 2^-32
-     *     = host_tsc * mult + (tsc_h * 2^32 + tsc_l) * frac * 2^-32
-     *     = host_tsc * mult + tsc_h * frac + ((tsc_l * frac) >> 32)
-     *
-     * Multiplications in the last two terms are between 32-bit integers,
-     * so both of them can fit in 64-bit integers.
-     *
-     * Because mult is usually less than 10 in practice, it's very rare
-     * that host_tsc * mult can overflow a 64-bit integer.
-     */
-    mult = ratio >> 32;
-    frac = ratio & ((1ULL << 32) - 1);
-    scaled_host_tsc  = host_tsc * mult;
-    scaled_host_tsc += (host_tsc >> 32) * frac;
-    scaled_host_tsc += ((host_tsc & ((1ULL << 32) - 1)) * frac) >> 32;
-
-    return scaled_host_tsc;
-}
-
-static uint64_t svm_get_tsc_offset(uint64_t host_tsc, uint64_t guest_tsc,
-    uint64_t ratio)
-{
-    return guest_tsc - scale_tsc(host_tsc, ratio);
-}
-
 static void cf_check svm_set_tsc_offset(struct vcpu *v, u64 offset, u64 at_tsc)
 {
     struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
@@ -832,18 +795,8 @@ static void cf_check svm_set_tsc_offset(struct vcpu *v, u64 offset, u64 at_tsc)
 
     if ( nestedhvm_vcpu_in_guestmode(v) )
     {
-        struct nestedsvm *svm = &vcpu_nestedsvm(v);
-
         n2_tsc_offset = vmcb_get_tsc_offset(n2vmcb) -
                         vmcb_get_tsc_offset(n1vmcb);
-        if ( svm->ns_tscratio != DEFAULT_TSC_RATIO )
-        {
-            uint64_t guest_tsc = hvm_get_guest_tsc_fixed(v, at_tsc);
-
-            n2_tsc_offset = svm_get_tsc_offset(guest_tsc,
-                                               guest_tsc + n2_tsc_offset,
-                                               svm->ns_tscratio);
-        }
         vmcb_set_tsc_offset(n1vmcb, offset);
     }
 
@@ -1921,10 +1874,6 @@ static int cf_check svm_msr_read_intercept(
         *msr_content = nsvm->ns_msr_hsavepa;
         break;
 
-    case MSR_AMD64_TSC_RATIO:
-        *msr_content = nsvm->ns_tscratio;
-        break;
-
     case MSR_AMD_OSVW_ID_LENGTH:
     case MSR_AMD_OSVW_STATUS:
         if ( !d->arch.cpuid->extd.osvw )
@@ -2103,12 +2052,6 @@ static int cf_check svm_msr_write_intercept(
             goto gpf;
         break;
 
-    case MSR_AMD64_TSC_RATIO:
-        if ( msr_content & TSC_RATIO_RSVD_BITS )
-            goto gpf;
-        nsvm->ns_tscratio = msr_content;
-        break;
-
     case MSR_IA32_MCx_MISC(4): /* Threshold register */
     case MSR_F10_MC4_MISC1 ... MSR_F10_MC4_MISC3:
         /*
diff --git a/xen/arch/x86/include/asm/hvm/svm/nestedsvm.h b/xen/arch/x86/include/asm/hvm/svm/nestedsvm.h
index 406fc082b1..45d658ad01 100644
--- a/xen/arch/x86/include/asm/hvm/svm/nestedsvm.h
+++ b/xen/arch/x86/include/asm/hvm/svm/nestedsvm.h
@@ -18,11 +18,6 @@ struct nestedsvm {
      */
     uint64_t ns_ovvmcb_pa;
 
-    /* virtual tscratio holding the value l1 guest writes to the
-     * MSR_AMD64_TSC_RATIO MSR.
-     */
-    uint64_t ns_tscratio;
-
     /* Cached real intercepts of the l2 guest */
     uint32_t ns_cr_intercepts;
     uint32_t ns_dr_intercepts;
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 01:21:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 01:21:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676583.1052799 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA90-0000PO-Bd; Tue, 06 Feb 2024 01:21:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676583.1052799; Tue, 06 Feb 2024 01:21:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXA90-0000OY-7j; Tue, 06 Feb 2024 01:21:02 +0000
Received: by outflank-mailman (input) for mailman id 676583;
 Tue, 06 Feb 2024 01:21:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+4KK=JP=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXA8y-0007Lc-Ka
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 01:21:00 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fc567744-c48d-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 02:20:59 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a2f22bfb4e6so673260466b.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 17:20:59 -0800 (PST)
Received: from georged-x-u.eng.citrite.net ([185.25.67.249])
 by smtp.gmail.com with ESMTPSA id
 cu9-20020a170906ba8900b00a3726a5e5fdsm486803ejd.95.2024.02.05.17.20.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 05 Feb 2024 17:20:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc567744-c48d-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707182459; x=1707787259; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+pM0fbVdhQKXAGnV8MgqEAxKe6Vv2X0dLKmpVPSvjpw=;
        b=LwaqsOU10jyaNlqLRkN4Hli1zqcwp68kwJSYHNhDLxak210u9Bhw8t62tghU6kABmj
         PvtzeKSBZ26zkuAJHDknjoINcjXTz97Jy9EQkeZZGPT984afg+gEVoMgaJTqoboDBUrn
         HzODJJEPPY01vBeR+7ik6FWJWkDDKegi8Fvx8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707182459; x=1707787259;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=+pM0fbVdhQKXAGnV8MgqEAxKe6Vv2X0dLKmpVPSvjpw=;
        b=cmICTA/tLXP2Gu2Kww0lEkyxucncIt9IE0DMbHzvhXZDEJj6yyzlifYpQ1ugMzLqnk
         ej8ezi9T7BF/tr8XB6dzplLkO+YC+pcsBRHz2pjBjBedwxg9zoG3OM5CKAnsrqapywC9
         25mNMqwRJ3pcgMTr9Jq2IY6Tg+ueDpPGoDqKsLJUciqYL2KjesAyytbVnlF4my1A84i/
         26P/AnMOFUYxrvzWzo8D1LkwMfNG+MiYOZy1ZATLiWisr10veJpcSTmeR7j9rn4jjwTm
         ifbYJD6zevI9nUO6xM3g3XHgAFghRpwZLpQXajpPDid4CY+x9QenvKrQItqLKT3py58x
         hyXw==
X-Gm-Message-State: AOJu0YxxzallZ5vn6wSBLCG12R9hl2Iu8cfsrlg/6qcgv+Yb31bzadGK
	6I6kVpWFz7SsMRgYre/nh0nKWrPpYeekqQJtzmPFuLr8fFaPuTVvmpxoWcmvM+vD70KBTnOJEhe
	Er+E=
X-Google-Smtp-Source: AGHT+IH2cOu+Wp9hhWPN0ENYOLxHKZWM4fH3SAD3uRu30XVaQsueubP9ferpWl940F1nAwW/E2ZzYA==
X-Received: by 2002:a17:906:6711:b0:a35:d634:ed71 with SMTP id a17-20020a170906671100b00a35d634ed71mr408368ejp.23.1707182458745;
        Mon, 05 Feb 2024 17:20:58 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCX9XwTTAZaIAdCAi/HjSjsAx57kB++2feCoFTQuGBsw2R4RJY5t9gw2L1ZdwW2gEP4miMASqd6CRLVRSFKsOoGP6085iRQ3LHhEur5aFe4DcuS+ax9cw7Y+46MokRR+cLLdJ4y6VV7ZcgwUvqrJD5dyQQ9mj9wBM0ya/ANHYm4654S5qlBP/tstx65hgovQsLmeSgcqE7m64iTtmi93XejmBETN0HFbbzRGjb+4KeEPdMAME+HR4Y9ymg0lZbI+KapGR/EnW34R/YJlTwgHGn7E3M5de/j6N/aCmbK1CNIF8Ct3tOYjk0EoMETHXy4H
From: George Dunlap <george.dunlap@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@cloud.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>
Subject: [PATCH 6/6] svm/nestedvm: Introduce nested capabilities bit
Date: Tue,  6 Feb 2024 01:20:51 +0000
Message-Id: <20240206012051.3564035-7-george.dunlap@cloud.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20240206012051.3564035-1-george.dunlap@cloud.com>
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

In order to make implementation and testing tractable, we will require
specific host functionality.  Add a nested_virt bit to hvm_funcs.caps,
and return an error if a domain is created with nested virt and this
bit isn't set.

For VMX, start with always enabling it if HAP is present; this
shouldn't change current behvior.

For SVM, require some basic functionality, adding a document
explaining the rationale.

NB that only SVM CPUID bits 0-7 have been considered.  Bits 10-16 may
be considered in a follow-up patch.

Signed-off-by: George Dunlap <george.dunlap@cloud.com>
---
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: George Dunlap <george.dunlap@citrix.com>
CC: Jan Beulich <jbeulich@suse.com>
CC: Julien Grall <julien@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: "Roger Pau Monné" <roger.pau@citrix.com>
CC: Jun Nakajima <jun.nakajima@intel.com>
CC: Kevin Tian <kevin.tian@intel.com>
---
 docs/designs/nested-svm-cpu-features.md | 110 ++++++++++++++++++++++++
 xen/arch/x86/domain.c                   |   6 ++
 xen/arch/x86/hvm/svm/nestedhvm.h        |   1 +
 xen/arch/x86/hvm/svm/nestedsvm.c        |  14 +++
 xen/arch/x86/hvm/svm/svm.c              |   2 +
 xen/arch/x86/hvm/vmx/vmx.c              |   3 +
 xen/arch/x86/include/asm/hvm/hvm.h      |  11 ++-
 7 files changed, 146 insertions(+), 1 deletion(-)

diff --git a/docs/designs/nested-svm-cpu-features.md b/docs/designs/nested-svm-cpu-features.md
new file mode 100644
index 0000000000..7ffb8daefd
--- /dev/null
+++ b/docs/designs/nested-svm-cpu-features.md
@@ -0,0 +1,110 @@
+# Nested SVM (AMD) CPUID requirements
+
+The first step in making nested SVM production-ready is to make sure
+that all features are implemented and well-tested.  To make this
+tractable, we will initially be limiting the "supported" range of
+nested virt to a specific subset of host and guest features.  This
+document describes the criteria for deciding on features, and the
+rationale behind each feature.
+
+For AMD, all virtualization-related features can be found in CPUID
+leaf 8000000A:edx
+
+# Criteria
+
+- Processor support: At a minimum we want to support processors from
+  the last 5 years.  All things being equal, older processors are
+  better.  Bits 0:7 were available in the very earliest processors;
+  and even through bit 15 we should be pretty good support-wise.
+
+- Faithfulness to hardware: We need the behavior of the "virtual cpu"
+  from the L1 hypervisor's perspective to be as close as possible to
+  the original hardware.  In particular, the behavior of the hardware
+  on error paths 1) is not easy to understand or test, 2) can be the
+  source of surprising vulnerabiliies.  (See XSA-7 for an example of a
+  case where subtle error-handling differences can open up a privilege
+  escalation.)  We should avoid emulating any bit of the hardware with
+  complex error paths if we can at all help it.
+
+- Cost of implementation: We want to minimize the cost of
+  implementation (where this includes bringing an existing sub-par
+  implementation up to speed).  All things being equal, we'll favor a
+  configuration which does not require any new implementation.
+
+- Performance: All things being equal, we'd prefer to choose a set of
+  L0 / L1 CPUID bits that are faster than slower.
+
+
+# Bits
+
+- 0 `NP` *Nested Paging*: Required both for L0 and L1.
+
+  Based primarily on faithfulness and performance, as well as
+  potential cost of implementation.  Available on earliest hardware,
+  so no compatibility issues.
+
+- 1 `LbrVirt` *LBR / debugging virtualization*: Require for L0 and L1.
+
+  For L0 this is required for performance: There's no way to tell the
+  guests not to use the LBR-related registers; and if the guest does,
+  then you have to save and restore all LBR-related registers on
+  context switch, which is prohibitive.  Furthermore, the additional
+  emulation risks a security-relevant difference to come up.
+
+  Providing it to L1 when we have it in L0 is basically free, and
+  already implemented.
+
+  Just require it and provide it.
+
+- 2 `SVML` *SVM Lock*: Not required for L0, not provided to L1
+
+  Seems to be aboult enabling an operating system to prevent "blue
+  pill" attacks against itself.
+
+  Xen doesn't use it, nor provide it; so it would need to be
+  implementend.  The best way to protect a guest OS is to leave nested
+  virt disabled in the tools.
+
+- 3 `NRIPS` NRIP Save: Require for both L0 and L1
+
+  If NRIPS is not present, the software interrupt injection
+  functionality can't be used; and Xen has to emulate it.  That's
+  another source of potential security issues.  If hardware supports
+  it, then providing it to guest is basically free.
+
+- 4 `TscRateMsr`: Not required by L0, not provided to L1
+
+  The main putative use for this would be trying to maintain an
+  invariant TSC across cores with different clock speeds, or after a
+  migrate.  Unlike others, this doesn't have an error path to worry
+  about compatibility-wise; and according to tests done when nestedSVM
+  was first implemented, it's actually faster to emliate TscRateMSR in
+  the L0 hypervisor than for L1 to attempt to emulate it itself.
+
+  However, using this properly in L0 will take some implementation
+  effort; and composing it properly with L1 will take even more
+  effort.  Just leave it off for now.
+
+ - 5 `VmcbClean`: VMCB Clean Bits: Not required by L0, provide to L1
+
+  This is a pure optimization, both on the side of the L0 and L1.  The
+  implementaiton for L1 is entirely Xen-side, so can be provided even
+  on hardware that doesn't provide it.  And it's purely an
+  optimization, so could be "implemented" by ignoring the bits
+  entirely.
+
+  As such, we don't need to require it for L0; and as it's already
+  implemented, no reason not to provide it to L1.  Before this feature
+  was available those bits were marked SBZ ("should be zero"); setting
+  them was already advertised to cause unpredictable behavior.
+
+- 6 `FlushByAsid`: Require for L0, provide to L1
+
+  This is cheap and easy to use for L0 and to provide to the L1;
+  there's no reson not to just pass it through.
+
+- 7 `DecodeAssists`: Require for L0, provide to L1
+
+  Using it in L0 reduces the chance that we'll make some sort of error
+  in the decode path.  And if hardware supports it, it's easy enough
+  to provide to the L1.
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index bda853e3c9..a25f498265 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -673,6 +673,12 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
          */
         config->flags |= XEN_DOMCTL_CDF_oos_off;
 
+    if ( nested_virt && !hvm_nested_virt_supported() )
+    {
+        dprintk(XENLOG_INFO, "Nested virt requested but not available\n");
+        return -EINVAL;        
+    }
+
     if ( nested_virt && !hap )
     {
         dprintk(XENLOG_INFO, "Nested virt not supported without HAP\n");
diff --git a/xen/arch/x86/hvm/svm/nestedhvm.h b/xen/arch/x86/hvm/svm/nestedhvm.h
index 43245e13de..31cf2af8e4 100644
--- a/xen/arch/x86/hvm/svm/nestedhvm.h
+++ b/xen/arch/x86/hvm/svm/nestedhvm.h
@@ -35,6 +35,7 @@ enum nestedhvm_vmexits
 nestedsvm_check_intercepts(struct vcpu *v, struct cpu_user_regs *regs,
     uint64_t exitcode);
 void svm_nested_features_on_efer_update(struct vcpu *v);
+void __init start_nested_svm(struct hvm_function_table *svm_function_table);
 
 /* Interface methods */
 void cf_check nsvm_vcpu_destroy(struct vcpu *v);
diff --git a/xen/arch/x86/hvm/svm/nestedsvm.c b/xen/arch/x86/hvm/svm/nestedsvm.c
index a5319ab729..92b063daa5 100644
--- a/xen/arch/x86/hvm/svm/nestedsvm.c
+++ b/xen/arch/x86/hvm/svm/nestedsvm.c
@@ -1666,3 +1666,17 @@ void svm_nested_features_on_efer_update(struct vcpu *v)
         }
     }
 }
+
+void __init start_nested_svm(struct hvm_function_table *svm_function_table)
+{
+    /* 
+     * Required host functionality to support nested virt.  See
+     * docs/designs/nested-svm-cpu-features.md for rationale.
+     */
+    svm_function_table->caps.nested_virt =
+        cpu_has_svm_nrips &&
+        cpu_has_svm_lbrv &&
+        cpu_has_svm_nrips &&
+        cpu_has_svm_flushbyasid &&
+        cpu_has_svm_decode;
+}
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 34b9f603bc..5c2e171777 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -2527,6 +2527,8 @@ const struct hvm_function_table * __init start_svm(void)
     svm_function_table.caps.hap_superpage_2mb = true;
     svm_function_table.caps.hap_superpage_1gb = cpu_has_page1gb;
 
+    start_nested_svm(&svm_function_table);
+
     return &svm_function_table;
 }
 
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 4bcf436d2c..6b5ad4a509 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -3021,6 +3021,9 @@ const struct hvm_function_table * __init start_vmx(void)
     if ( cpu_has_vmx_tsc_scaling )
         vmx_function_table.tsc_scaling.ratio_frac_bits = 48;
 
+    /* TODO: Require hardware support before enabling nested virt */
+    vmx_function_table.caps.nested_virt = vmx_function_table.caps.hap;
+
     model_specific_lbr = get_model_specific_lbr();
     lbr_tsx_fixup_check();
     ler_to_fixup_check();
diff --git a/xen/arch/x86/include/asm/hvm/hvm.h b/xen/arch/x86/include/asm/hvm/hvm.h
index bbd83a8275..8a3df0eca7 100644
--- a/xen/arch/x86/include/asm/hvm/hvm.h
+++ b/xen/arch/x86/include/asm/hvm/hvm.h
@@ -97,7 +97,10 @@ struct hvm_function_table {
             singlestep:1,
             
             /* Hardware virtual interrupt delivery enable? */
-            virtual_intr_delivery;
+            virtual_intr_delivery,
+
+            /* Nested virt capabilities */
+            nested_virt:1;
 
     } caps;
 
@@ -655,6 +658,12 @@ static inline bool hvm_altp2m_supported(void)
     return hvm_funcs.caps.altp2m;
 }
 
+/* Returns true if we have the minimum hardware requirements for nested virt */
+static inline bool hvm_nested_virt_supported(void)
+{
+    return hvm_funcs.caps.nested_virt;
+}
+
 /* updates the current hardware p2m */
 static inline void altp2m_vcpu_update_p2m(struct vcpu *v)
 {
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 01:34:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 01:34:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676614.1052810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXALu-0004HQ-Pp; Tue, 06 Feb 2024 01:34:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676614.1052810; Tue, 06 Feb 2024 01:34:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXALu-0004HI-LZ; Tue, 06 Feb 2024 01:34:22 +0000
Received: by outflank-mailman (input) for mailman id 676614;
 Tue, 06 Feb 2024 01:34:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+4KK=JP=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXALt-0004HC-Rq
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 01:34:21 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d9b8a42e-c48f-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 02:34:20 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-51025cafb51so7559237e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 17:34:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d9b8a42e-c48f-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707183260; x=1707788060; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=PbrH1sImvlrG3C1Riag1f9VpsJwpPYrDcJrwv9y4gzQ=;
        b=iZXwR3/mvjBQF9SEAJC7VdDijrwXq+bBz2W/M6ORx6paEyy4juyzw5vo6ySBxgsadP
         AOW+ShZ73gQkKbRta+8oEszcsij+KCgE1HELjoTQBl1dfPFR+dMrpBVcY9u2X6Oh+yhi
         HBK464FLJcvdAtXBn1U6wQ9oASi/IoYTCDQLc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707183260; x=1707788060;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=PbrH1sImvlrG3C1Riag1f9VpsJwpPYrDcJrwv9y4gzQ=;
        b=iu5OcBk7bo547bJD4aGMIat9aSEq8rAUOVeagVnW9VEkE+n4dC02zKUyOqmldUkqkk
         sZfwO7f9TeiUGehHN19NLNnArJw9oYq5wB8yhxZ56mILHHf13HZGqB6CYREDBUeVuxDr
         UffM8m8ATulIqeUhvwVS2NthrFafank8NZBA/aCPLTX1R7ystTXbU8/ded/Nn5gOUOsH
         YdMukp9aZowEe5MPX9WCYNZnPOMMafJ8tzoJbncTSveLDt22FyojP0EghAgWKKnS3UFJ
         qen/6YPs9C6QCMrSLekrRllc21rKjP8DIRalQwSKOkG0lfPEWLBPNenl/V/gB8z3skFB
         CQkg==
X-Gm-Message-State: AOJu0Yxo7fUMl99YKGqVnOQw43QtCgXw5V8VkKkhHnZrMjYnWbDJLTiq
	WRrRroW7cxBMrkOHWytehJgt92HOxcXi+sFPEHKZdYmWMQUgoIE8xx75mBp7zooJqxFqQxglNPG
	ocaUZYXgrrRGSSuq5ier6g8fWfrasS3qvGXcWHENwuUTPyLd81mU=
X-Google-Smtp-Source: AGHT+IGbTCWD3B5BM7Y+OMzCe6vKbM1c5kR82VgYSKsd8P6ifi4O5HJO91SIPukLts1RDL3d4L/9DRi9A5a4X0q0r08=
X-Received: by 2002:ac2:494c:0:b0:511:4c52:8fc8 with SMTP id
 o12-20020ac2494c000000b005114c528fc8mr750063lfi.8.1707183260010; Mon, 05 Feb
 2024 17:34:20 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com> <20240206012051.3564035-7-george.dunlap@cloud.com>
In-Reply-To: <20240206012051.3564035-7-george.dunlap@cloud.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Tue, 6 Feb 2024 01:34:08 +0000
Message-ID: <CA+zSX=YtseSSJrKeCrpU_tfv-Lvyod2eb-z7LBS6Dd8Qv17cLg@mail.gmail.com>
Subject: Re: [PATCH 6/6] svm/nestedvm: Introduce nested capabilities bit
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 6, 2024 at 9:20=E2=80=AFAM George Dunlap <george.dunlap@cloud.c=
om> wrote:

> @@ -655,6 +658,12 @@ static inline bool hvm_altp2m_supported(void)
>      return hvm_funcs.caps.altp2m;
>  }
>
> +/* Returns true if we have the minimum hardware requirements for nested =
virt */
> +static inline bool hvm_nested_virt_supported(void)
> +{
> +    return hvm_funcs.caps.nested_virt;
> +}

NB this is missing the !CONFIG_HVM version of this function; I'll
include it in v2.

 -George


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 04:33:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 04:33:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676623.1052829 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXD9V-0000n7-T1; Tue, 06 Feb 2024 04:33:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676623.1052829; Tue, 06 Feb 2024 04:33:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXD9V-0000n0-Q3; Tue, 06 Feb 2024 04:33:45 +0000
Received: by outflank-mailman (input) for mailman id 676623;
 Tue, 06 Feb 2024 04:33:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXD9V-0000mq-9f; Tue, 06 Feb 2024 04:33:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXD9V-00017W-7C; Tue, 06 Feb 2024 04:33:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXD9U-0006BU-OB; Tue, 06 Feb 2024 04:33:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXD9U-0002y1-Nh; Tue, 06 Feb 2024 04:33:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/yGEurvzX5djgblIANn82GP84igP9nAPEAd5wD9au/Q=; b=hjNUs9cTTnt2Z9Zkz6C5BevaNH
	p9sRl5pyXZhwiSfxKYBtcrDfnTVb8WSExKgdbXN/AHspT9qbYpLJhRBGl1bI/b0L3yL3HZpoMCT4g
	tflCYhD6kCDQuL2ambkiJrMBtL3t6ShP/XmVwRvr39CFch3sAqdcW7YX4m9m/0Trxuuo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184599-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184599: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=1cb7949b37b3d8892d8a3cea3ad1e2448352a34d
X-Osstest-Versions-That:
    xen=29f17d837421f13c0e0010802de1b2d51d2ded4a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 06 Feb 2024 04:33:44 +0000

flight 184599 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184599/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  1cb7949b37b3d8892d8a3cea3ad1e2448352a34d
baseline version:
 xen                  29f17d837421f13c0e0010802de1b2d51d2ded4a

Last test of basis   184593  2024-02-05 18:00:25 Z    0 days
Failing since        184596  2024-02-05 21:00:25 Z    0 days    2 attempts
Testing same since   184599  2024-02-06 01:00:34 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Frediano Ziglio <frediano.ziglio@cloud.com>
  Juergen Gross <jgross@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   29f17d8374..1cb7949b37  1cb7949b37b3d8892d8a3cea3ad1e2448352a34d -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 04:48:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 04:48:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676630.1052839 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXDO0-0002SO-7K; Tue, 06 Feb 2024 04:48:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676630.1052839; Tue, 06 Feb 2024 04:48:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXDO0-0002SH-3E; Tue, 06 Feb 2024 04:48:44 +0000
Received: by outflank-mailman (input) for mailman id 676630;
 Tue, 06 Feb 2024 04:48:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXDNz-0002S5-0A; Tue, 06 Feb 2024 04:48:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXDNy-0001XJ-TH; Tue, 06 Feb 2024 04:48:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXDNy-0006em-JS; Tue, 06 Feb 2024 04:48:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXDNy-0007OM-Ip; Tue, 06 Feb 2024 04:48:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=5fAF8OG/r4ibNfyXf02XoeCTkkHvU4crPKIsMKkdGtQ=; b=qN/yV+XkG9ygpNAGHu3c9or0cR
	51oksal6a/0haFAAzgzEuGTsoXWmxYxWHk5EnbSXuu5oAGGaDkMx5KqcNWjeKbBHXKv+D1FVcGfxZ
	/sehHY55aN7nNWsDPYrmDnaKWt5IoJjtsC1li7HxLJj9LW4hjBIYYkPfSyUayiPs6yiY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184595-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-6.1 test] 184595: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-6.1:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=f1bb70486c9c11d7e2d55240d4557f9fc575fbac
X-Osstest-Versions-That:
    linux=e5c3b988b827c76f52d0f62343e863b9133a0cd2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 06 Feb 2024 04:48:42 +0000

flight 184595 linux-6.1 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184595/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184570
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184570
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184570
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184570
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184570
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184570
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184570
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184570
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 linux                f1bb70486c9c11d7e2d55240d4557f9fc575fbac
baseline version:
 linux                e5c3b988b827c76f52d0f62343e863b9133a0cd2

Last test of basis   184570  2024-02-03 02:27:03 Z    3 days
Testing same since   184595  2024-02-05 20:44:35 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Guilherme G. Piccoli" <gpiccoli@igalia.com>
  Abel Vesa <abel.vesa@linaro.org>
  Abhinav Kumar <quic_abhinavk@quicinc.com>
  Adam Ford <aford173@gmail.com>  #imx8mp-beacon
  Adrian Reber <areber@redhat.com>
  Ahmed Zaki <ahmed.zaki@intel.com>
  Al Viro <viro@zeniv.linux.org.uk>
  Alan Maguire <alan.maguire@oracle.com>
  Aleksandr Loktionov <aleksandr.loktionov@intel.com>
  Alex Deucher <alexander.deucher@amd.com>
  Alex Lyakas <alex.lyakas@zadara.com>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Potapenko <glider@google.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexandre Belloni <alexandre.belloni@bootlin.com>
  Alexei Starovoitov <ast@kernel.org>
  Alvin Lee <alvin.lee2@amd.com>
  Amit Pundir <amit.pundir@linaro.org>
  Andrei Vagin <avagin@gmail.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrii Nakryiko <andrii@kernel.org>
  Andrii Staikov <andrii.staikov@intel.com>
  Andrzej Hajda <andrzej.hajda@intel.com>
  Anna Schumaker <Anna.Schumaker@Netapp.com>
  Anton Ivanov <anton.ivanov@cambridgegreys.com>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Arnd Bergmann <arnd@arndb.de>
  Aurabindo Pillai <aurabindo.pillai@amd.com>
  Bailey Forrest <bcf@google.com>
  Baochen Qiang <quic_bqiang@quicinc.com>
  Baokun Li <libaokun1@huawei.com>
  Benjamin Berg <benjamin.berg@intel.com>
  Benjamin Berg <benjamin@sipsolutions.net>
  Benjamin Poirier <bpoirier@nvidia.com>
  Bharat Bhushan <bbhushan2@marvell.com>
  Bjorn Andersson <andersson@kernel.org>
  Bjorn Helgaas <bhelgaas@google.com>
  bo liu <bo.liu@senarytech.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Breno Leitao <leitao@debian.org>
  Catalin Marinas <catalin.marinas@arm.com>
  Celeste Liu <CoelacanthusHex@gmail.com>
  Changye Wu <wuchangye@huawei.com>
  Chanwoo Choi <cw00.choi@samsung.com>
  Chao Yu <chao@kernel.org>
  ching Huang <ching2048@areca.com.tw>
  Chris Riches <chris.riches@nutanix.com>
  Christian Brauner <brauner@kernel.org>
  Christian König <christian.koenig@amd.com>
  Christian Marangi <ansuelsmth@gmail.com>
  Christoph Hellwig <hch@lst.de>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  clancy shang <clancy.shang@quectel.com>
  Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
  Daniel Borkmann <daniel@iogearbox.net>
  Daniel Stodden <dns@arista.com>
  Daniel Vacek <neelx@redhat.com>
  Daniel Vetter <daniel@ffwll.ch>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Dave Hansen <dave.hansen@linux.intel.com>
  Dave Kleikamp <dave.kleikamp@oracle.com>
  David Howells <dhowells@redhat.com>
  David S. Miller <davem@davemloft.net>
  Derek Barbosa <debarbos@redhat.com>
  Dmitry Antipov <dmantipov@yandex.ru>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Dominique Martinet <asmadeus@codewreck.org>
  Douglas Anderson <dianders@chromium.org>
  Edward Adam Davis <eadavis@qq.com>
  Eric Dumazet <edumazet@google.com>
  Fabio Estevam <festevam@denx.de>
  Fei Shao <fshao@chromium.org>
  Felix Kuehling <Felix.Kuehling@amd.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Frédéric Danis <frederic.danis@collabora.com>
  Gabriel Krisman Bertazi <krisman@suse.de>
  Gao Xiang <hsiangkao@linux.alibaba.com>
  Ghanshyam Agrawal <ghanshyam1898@gmail.com>
  Greg KH <gregkh@linuxfoundation.org>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guenter Roeck <linux@roeck-us.net>
  Guilherme G. Piccoli <gpiccoli@igalia.com>
  Guo Ren <guoren@kernel.org>
  Guoqing Jiang <guoqing.jiang@linux.dev>
  H. Nikolaus Schaller <hns@goldelico.com>
  Halil Pasic <pasic@linux.ibm.com>
  Hamza Mahfooz <hamza.mahfooz@amd.com>
  Hannes Reinecke <hare@suse.de>
  Hans de Goede <hdegoede@redhat.com>
  Hans Verkuil <hverkuil-cisco@xs4all.nl>
  Harald Freudenberger <freude@linux.ibm.com>
  Hardik Gajjar <hgajjar@de.adit-jv.com>
  HariBabu Gattem <haribabu.gattem@xilinx.com>
  Harshit Shah <harshitshah.opendev@gmail.com>
  Heiko Carstens <hca@linux.ibm.com>
  Heiko Stuebner <heiko@sntech.de>
  Heiner Kallweit <hkallweit1@gmail.com>
  Helge Deller <deller@gmx.de>
  Helge Deller <deller@kernel.org>
  Herb Wei <weihao.bj@ieisystem.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Horatiu Vultur <horatiu.vultur@microchip.com>
  Hou Tao <houtao1@huawei.com>
  Hsin-Yi Wang <hsinyi@chromium.org>
  Huacai Chen <chenhuacai@loongson.cn>
  Huang Shijie <shijie@os.amperecomputing.com>
  Ian Rogers <irogers@google.com>
  Ido Schimmel <idosch@nvidia.com>
  Igor Russkikh <irusskikh@marvell.com>
  Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
  Ilya Dryomov <idryomov@gmail.com>
  Ingo Molnar <mingo@kernel.org>
  Inki Dae <inki.dae@samsung.com>
  Jack Wang <jinpu.wang@ionos.com>
  Jaegeuk Kim <jaegeuk@kernel.org>
  Jakub Kicinski <kuba@kernel.org>
  James Clark <james.clark@arm.com>
  Jassi Brar <jaswinder.singh@linaro.org>
  Jay Buddhabhatti <jay.buddhabhatti@amd.com>
  Jay Vosburgh <jay.vosburgh@canonical.com>
  Jedrzej Jagielski <jedrzej.jagielski@intel.com>
  Jeff Johnson <quic_jjohnson@quicinc.com>
  Jens Axboe <axboe@kernel.dk>
  Jesse Brandeburg <jesse.brandeburg@intel.com>
  Jiri Kosina <jkosina@suse.com>
  Jo Van Bulck <jo.vanbulck@cs.kuleuven.be>
  Johan Hovold <johan+linaro@kernel.org>
  Johan Jonker <jbx6244@gmail.com>
  Johannes Berg <johannes.berg@intel.com>
  Jon Hunter <jonathanh@nvidia.com>
  Jonathan Gray <jsg@jsg.id.au>
  Jose Ignacio Tornos Martinez <jtornosm@redhat.com>
  Josip Pavic <josip.pavic@amd.com>
  Juergen Gross <jgross@suse.com>
  Jun'ichi Nomura <junichi.nomura@nec.com>
  Justin Tee <justin.tee@broadcom.com>
  Jérôme Pouiller <jerome.pouiller@silabs.com>
  Kalle Valo <kvalo@kernel.org>
  Kalle Valo <quic_kvalo@quicinc.com>
  Kees Cook <keescook@chromium.org>
  kernelci.org bot <bot@kernelci.org>
  Kieran Bingham <kieran.bingham@ideasonboard.com>
  Konrad Dybcio <konrad.dybcio@linaro.org>
  Kuan-Wei Chiu <visitorckw@gmail.com>
  Kunwu Chan <chentao@kylinos.cn>
  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
  Lee Jones <lee@kernel.org>
  Leo Yan <leo.yan@linaro.org>
  Leon Romanovsky <leon@kernel.org>
  Lin Ma <linma@zju.edu.cn>
  Linus Lüssing <linus.luessing@c0d3.blue>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
  Ma Wupeng <mawupeng1@huawei.com>
  Manas Ghandat <ghandatmanas@gmail.com>
  Mao Jinlong <quic_jinlmao@quicinc.com>
  Marc Dionne <marc.dionne@auristor.com>
  Marco Elver <elver@google.com>
  Marek Szyprowski <m.szyprowski@samsung.com>
  Mark Brown <broonie@kernel.org>
  Mark Rutland <mark.rutland@arm.com>
  Martin K. Petersen <martin.petersen@oracle.com>
  Mateusz Jończyk <mat.jonczyk@o2.pl>
  Matthias May <matthias.may@westermo.com>
  Mauro Carvalho Chehab <mchehab@kernel.org>
  Max Kellermann <max.kellermann@ionos.com>
  Maxime Ripard <mripard@kernel.org>
  Meenakshikumar Somasundaram <meenakshikumar.somasundaram@amd.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael Tretter <m.tretter@pengutronix.de>
  Michal Simek <michal.simek@amd.com>
  Michal Vokáč <michal.vokac@ysoft.com>
  Ming Lei <ming.lei@redhat.com>
  Ming Qian <ming.qian@nxp.com>
  Mingyi Zhang <zhangmingyi5@huawei.com>
  Minsuk Kang <linuxlovemin@yonsei.ac.kr>
  Miri Korenblit <miriam.rachel.korenblit@intel.com>
  Mukesh Ojha <quic_mojha@quicinc.com>
  MyungJoo Ham <myungjoo.ham@samsung.com>
  Nathan Chancellor <nathan@kernel.org>
  Naveen N Rao <naveen@kernel.org>
  Neil Armstrong <neil.armstrong@linaro.org>
  Nicolas Dichtel <nicolas.dichtel@6wind.com>
  Nikolay Aleksandrov <razor@blackwall.org>
  Oleg Nesterov <oleg@redhat.com>
  Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
  Oliver F. Brown <oliver.brown@oss.nxp.com>
  Osama Muhammad <osmtendev@gmail.com>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Palmer Dabbelt <palmer@rivosinc.com>
  Paolo Abeni <pabeni@redhat.com>
  Paul Moore <paul@paul-moore.com>
  Pavel Machek (CIP) <pavel@denx.de>
  Peter Robinson <pbrobinson@gmail.com>
  Peter Zijlstra (Intel) <peterz@infradead.org>
  Peter Zijlstra <peterz@infradead.org>
  Petr Machata <petrm@nvidia.com>
  Philip Yang <Philip.Yang@amd.com>
  Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
  Prarit Bhargava <prarit@redhat.com>
  Praveen Kaligineedi <pkaligineedi@google.com>
  Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
  Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Rafal Romanowski <rafal.romanowski@intel.com>
  Ranjani Vaidyanathan <ranjani.vaidyanathan@nxp.com>
  Richard Weinberger <richard@nod.at>
  Rob Clark <robdclark@chromium.org>
  Robert Foss <rfoss@kernel.org>
  Romain Naour <romain.naour@skf.com>
  Ron Economos <re@w6rz.net>
  Rui Zhang <zr.zhang@vivo.com>
  Ryan Schaefer <ryanschf@amazon.com>
  Sakari Ailus <sakari.ailus@linux.intel.com>
  Salvatore Bonaccorso <carnil@debian.org>
  Sasha Levin <sashal@kernel.org>
  Sathya Prakash Veerichetty <safhya.prakash@broadcom.com>
  SeongJae Park <sj@kernel.org>
  Shannon Nelson <shannon.nelson@amd.com>
  Shawn Guo <shawnguo@kernel.org>
  Shiji Yang <yangshiji66@outlook.com>
  Shuai Xue <xueshuai@linux.alibaba.com>
  Song Liu <song@kernel.org>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Stanislaw Gruszka <stf_xl@wp.pl>
  Stanley.Yang <Stanley.Yang@amd.com>
  Stephen Boyd <sboyd@kernel.org>
  Stephen Rothwell <sfr@canb.auug.org.au>
  Su Hui <suhui@nfschina.com>
  Suman Ghosh <sumang@marvell.com>
  Sumit Saxena <sumit.saxena@broadcom.com>
  Sunitha Mekala <sunithax.d.mekala@intel.com> (A Contingent worker at Intel)
  Takashi Iwai <tiwai@suse.de>
  Tanmay Shah <tanmay.shah@xilinx.com>
  Tejun Heo <tj@kernel.org>
  Theodore Ts'o <tytso@mit.edu>
  Thomas Bourgoin <thomas.bourgoin@foss.st.com>
  Thomas Gleixner <tglx@linutronix.de>
  Thomas Richter <tmricht@linux.ibm.com>
  Tobias Waldekranz <tobias@waldekranz.com>
  Toke Høiland-Jørgensen <toke@toke.dk>
  Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
  Tony Krowiak <akrowiak@linux.ibm.com>
  Tony Lindgren <tony@atomide.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Venkata Prasad Potturu <venkataprasad.potturu@amd.com>
  Venky Shankar <vshankar@redhat.com>
  Ville Syrjälä <ville.syrjala@linux.intel.com>
  Viresh Kumar <viresh.kumar@linaro.org>
  Vladimir Oltean <vladimir.oltean@nxp.com>
  Wang, Beyond <Wang.Beyond@amd.com>
  Wayne Lin <wayne.lin@amd.com>
  Weichen Chen <weichen.chen@mediatek.com>
  Wen Gu <guwen@linux.alibaba.com>
  Wenchao Hao <haowenchao2@huawei.com>
  Werner Fischer <devlists@wefi.net>
  Will Deacon <will@kernel.org>
  Wim Van Sebroeck <wim@linux-watchdog.org>
  Xiang Chen <chenxiang66@hisilicon.com>
  Xianwei Zhao <xianwei.zhao@amlogic.com>
  Xiaowu.ding <xiaowu.ding@jaguarmicro.com>
  Xin Ji <xji@analogixsemi.com>
  Xin Liu <liuxin350@huawei.com>
  Xing Tong Wu <xingtong.wu@siemens.com>
  Xiubo Li <xiubli@redhat.com>
  Yafang Shao <laoar.shao@gmail.com>
  Yann Sionneau <ysionneau@kalrayinc.com>
  Ye Bin <yebin10@huawei.com>
  Yihang Li <liyihang9@huawei.com>
  Yonghong Song <yonghong.song@linux.dev>
  Youquan Song <youquan.song@intel.com>
  Yu-Che Cheng <giver@chromium.org>
  Yuluo Qiu <qyl27@outlook.com>
  Yuntao Wang <ytcoode@gmail.com>
  Zenm Chen <zenmchen@gmail.com>
  Zhengchao Shao <shaozhengchao@huawei.com>
  Zhipeng Lu <alexious@zju.edu.cn>
  Zhiquan Li <zhiquan1.li@intel.com>
  Zhu Yanjun <yanjun.zhu@linux.dev>
  Zijun Hu <quic_zijuhu@quicinc.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   e5c3b988b827..f1bb70486c9c  f1bb70486c9c11d7e2d55240d4557f9fc575fbac -> tested/linux-6.1


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 05:39:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 05:39:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676638.1052848 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXEAt-0008Nx-TI; Tue, 06 Feb 2024 05:39:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676638.1052848; Tue, 06 Feb 2024 05:39:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXEAt-0008Nq-Qh; Tue, 06 Feb 2024 05:39:15 +0000
Received: by outflank-mailman (input) for mailman id 676638;
 Tue, 06 Feb 2024 05:39:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+4KK=JP=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXEAr-0008Nk-PT
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 05:39:13 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0e977a9c-c4b2-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 06:39:12 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-5112d5ab492so7158142e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 21:39:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e977a9c-c4b2-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707197952; x=1707802752; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=dG+A5vlwccgptvGfzo09IcHsRnQBynlBnr+SMFU7cg8=;
        b=GL2TCVqoIBQfahhjNUcDQJiB/Mxj7EmwKDo8RRsup9rHFeQoMwTommuBbCSKQb9wbo
         DZh5ga9AvgwzQbe/7tG4ev7TUGdNl/zNrzB3dj4jVs2xgsAf8ZeAl+f034WL9/kxIwXC
         uvAlCz+4uRRfFR4aXKlyzLtgov0DNwvwOOZuo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707197952; x=1707802752;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=dG+A5vlwccgptvGfzo09IcHsRnQBynlBnr+SMFU7cg8=;
        b=n0bl2sb+SH4RL2Tj09KejosgLV+qCHwwR/GH+27MBz6OHJU/tCPsy+OTvhx+8Y8kzg
         Fa7pJLz5hb6XLTOxK8TOplcVhG8KjpZfJJAeoYdQIGuimCoDN0KH7qjWv1IYr1FmNfmI
         SfvgdVUr1f/M9BS/7nwfLb7gLfZ5L0CwYQdu8sd0MaGw8hFTnYw4awfFUxyX7x8+uyDv
         Kl0fTFfbNKyJOza9GJrVCFdJ6asGfK0QRs1AGw4CVfjWA+3soEi6eDurFtzJ3IDxvFkb
         Sm0zd/gGeb3UxiVn1PknqYustTjsSlpCh+oWdgoyecQQ266mCZ+CHlSgWPRVlcMedRLy
         i2DA==
X-Gm-Message-State: AOJu0YyF8RfUFDuyh3NbNdWzvNnHdHKkAy1a8gfP/eZcqld+fSFUh24Q
	uOp+HX3mKorare7PqYuTBAWgVzKZ0S5YTyfAyDKmpytx99bvVZloDviiYEnMG1i7ZsVF2Fx+JMG
	qywS2RptzCA00jrg2LeP4FHY6hai7YNvvRslItA==
X-Google-Smtp-Source: AGHT+IFHEPYAuM8wiaNKvgnebRdFQxjfVyEBnXibwDd6ClvmKilVRvgTuoL3SccDz8j376/cV5YFzH+8IqVVPIaM2O8=
X-Received: by 2002:a2e:9917:0:b0:2d0:ab64:a7f4 with SMTP id
 v23-20020a2e9917000000b002d0ab64a7f4mr1061220lji.6.1707197951812; Mon, 05 Feb
 2024 21:39:11 -0800 (PST)
MIME-Version: 1.0
References: <20240205165331.18627-1-anthony.perard@citrix.com>
In-Reply-To: <20240205165331.18627-1-anthony.perard@citrix.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Tue, 6 Feb 2024 05:39:00 +0000
Message-ID: <CA+zSX=bG6ffjScLA4-uFszJMyjtsPvrNAokZUA5vvN-GMEk8Gw@mail.gmail.com>
Subject: Re: [XEN PATCH] golang: Regen binding
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Nick Rosbrook <rosbrookn@gmail.com>, 
	Wei Liu <wl@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 6, 2024 at 12:53=E2=80=AFAM Anthony PERARD
<anthony.perard@citrix.com> wrote:
>
> Fixes: 024e7131be5c ("tools: don't expose XENFEAT_hvm_pirqs by default")
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: George Dunlap <george.dunlap@cloud.com>


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 06:17:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 06:17:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676647.1052863 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXEm1-0004kp-Qd; Tue, 06 Feb 2024 06:17:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676647.1052863; Tue, 06 Feb 2024 06:17:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXEm1-0004kX-Lo; Tue, 06 Feb 2024 06:17:37 +0000
Received: by outflank-mailman (input) for mailman id 676647;
 Tue, 06 Feb 2024 06:17:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=z75a=JP=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXEm0-0004kB-98
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 06:17:36 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 68b6be2e-c4b7-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 07:17:32 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 4972E21FC2;
 Tue,  6 Feb 2024 06:17:28 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1015713786;
 Tue,  6 Feb 2024 06:17:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id E+1kAvjOwWWmPQAAD6G6ig
 (envelope-from <jgross@suse.com>); Tue, 06 Feb 2024 06:17:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68b6be2e-c4b7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707200250; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=15V5eFj2jLAbKEF5egEw/9Nn2ao4TpiAAM6VXmsstgs=;
	b=MDKwijcSTZSjPiSNoI6oMeSvn1EOmCiDJVsnqHVG31lmMGrHjdBKZ/cBL4kju1aOzzQTbz
	TDRKzOkKQnkF9RlSs56DYGSudNprjl46yXO6bL+LKnWkeZXjcHUrxFvN5q6WgfgzD9FFtm
	QwtcshgFXo3TFyMJSHWFGorAgTjj0b4=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707200248; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=15V5eFj2jLAbKEF5egEw/9Nn2ao4TpiAAM6VXmsstgs=;
	b=reB0C4nyfUk2/9p7m4JG9ILrEUd3B7b64VcDhi6c1Yk+cLQdR05DFDZSak0mk4jJESXmf7
	cSEIM/RSI/Jfskc36Nb1NXojIcmOFBocwp0y/WdhdBb12xUCS9FH7YfeqRznZ+9IEvgrpd
	aAa6wIF3DKH294LdUPuz7+HM7Qc6MWs=
From: Juergen Gross <jgross@suse.com>
To: minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org
Cc: samuel.thibault@ens-lyon.org,
	wl@xen.org,
	julien@xen.org,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH] Mini-OS: fix 9pfs frontend error path
Date: Tue,  6 Feb 2024 07:17:21 +0100
Message-Id: <20240206061721.27297-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=reB0C4ny
X-Spamd-Result: default: False [4.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[6];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[xen.org:email,suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[22.69%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 4.69
X-Rspamd-Queue-Id: 4972E21FC2
X-Spam-Level: ****
X-Spam-Flag: NO
X-Spamd-Bar: ++++

The early error exit in p9_stat() returns without zeroing the p9_stat
buffer, resulting in free() being called with an uninitialized pointer.

Fix that by doing the zeroing first.

Reported-by: Julien Grall <julien@xen.org>
Fixes: 2d1dfccd3aa3 ("Mini-OS: add read and write support to 9pfsfront")
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 9pfront.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/9pfront.c b/9pfront.c
index 315089bc..33eaadce 100644
--- a/9pfront.c
+++ b/9pfront.c
@@ -716,10 +716,11 @@ static int p9_stat(struct dev_9pfs *dev, uint32_t fid, struct p9_stat *stat)
     uint16_t total;
     int ret;
 
+    memset(stat, 0, sizeof(*stat));
+
     if ( !req )
         return EAGAIN;
 
-    memset(stat, 0, sizeof(*stat));
     req->cmd = P9_CMD_STAT;
     send_9p(dev, req, "U", fid);
     rcv_9p(dev, req, "uuuUQUUULSSSSSUUU", &total, &stat->size, &stat->type,
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 06:50:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 06:50:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676661.1052872 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXFHI-0000FV-BE; Tue, 06 Feb 2024 06:49:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676661.1052872; Tue, 06 Feb 2024 06:49:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXFHI-0000FO-8T; Tue, 06 Feb 2024 06:49:56 +0000
Received: by outflank-mailman (input) for mailman id 676661;
 Tue, 06 Feb 2024 06:49:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rluy=JP=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1rXFHH-0000FI-DC
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 06:49:55 +0000
Received: from mail-oa1-x2e.google.com (mail-oa1-x2e.google.com
 [2001:4860:4864:20::2e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ede8bc22-c4bb-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 07:49:53 +0100 (CET)
Received: by mail-oa1-x2e.google.com with SMTP id
 586e51a60fabf-2143a96d185so2559181fac.3
 for <xen-devel@lists.xenproject.org>; Mon, 05 Feb 2024 22:49:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ede8bc22-c4bb-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1707202192; x=1707806992; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YOvz1DCTpUYpJz6Ibk0AJZpg+ObusQM8DdGve4UWe+Q=;
        b=Ua3132LkHhBx3QYIzVFLOPq/+N/6OxAeU4w9MFzmfAYKlqda6zm4Ykslh9eeMNFPhO
         1JtMg4ma8MfhkMh0ANZgMnyK/NuJH+WlRP/erQDsdJ4sPjoyX12rtMam3cmMU49Y0hya
         pa3JGvIIrqy/UhRzp4NXelOgl9QKrXBE+vKmF1E59jX4fYlbGDWBwtZYDnV+XSCTESbV
         mm+necLjbB+lE1cZJIrbT2iwoBNLGx7wsxyTcFtBNjsJMOqHAuUN9G9c1Cyyz6c+PfW1
         tXkKvtVm8/IKW4BmjIimrbEGS9u7FYgrM7RakPKmBAzAkDAqtmCoftK2xyR3mN42rvaT
         qSTw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707202192; x=1707806992;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=YOvz1DCTpUYpJz6Ibk0AJZpg+ObusQM8DdGve4UWe+Q=;
        b=nRUUqjhIT2rdmc5haYrAgMbpfSBfVjil9xEjckqB1wSI71KcDbzfK9V8NNFihkgZ+Z
         mITe4JXv84vycC230w7dtZm0gFFoWYOqEWSNn/ynDHitpXHDhC/hY0WGJLjgUxSGgm5c
         gAUL63edtoMQMFaVgzw/sGZixUXQAN8oE7vasxYmfl369oSsRCh9t4zsDA1igMR5hJqV
         dtjLrPU5R10cZ1adQM5oKGN2BOLKTuWTDKZ7P6WOFh4xZLzPIauOb6e50k9RSgsIo1D7
         /07q6RLEyP4KaNoezNGBdtS4K7pxx3IGAF2o9ctbIsGgCbYltYE77tLpyOOwnJB4e8Zj
         zJlA==
X-Gm-Message-State: AOJu0YzWSM8/jcOXIKTucK92IJQ/zNY37ExPTUM+oymdS5f0dxcA45JT
	c8uPZY6ntRRU+3rYbb7lt6+K0JgUcoSybhlDffnNm6CyhtI/yGXYGlWT4C93iTcj31w7C/T61Lm
	vk1CBsNNV5XNmvn0iSh2ij0ob+JcII2j9TUD2Xw==
X-Google-Smtp-Source: AGHT+IFhA+k/0YdehkkgmfWfFUJBLTeL4MQ+0VSgoDstsz0fnx5E6uziXZoRs6LVndL1aYjjIBC4nu0TEecRiwvDrME=
X-Received: by 2002:a05:6870:9689:b0:219:6eeb:c724 with SMTP id
 o9-20020a056870968900b002196eebc724mr2175629oaq.20.1707202191896; Mon, 05 Feb
 2024 22:49:51 -0800 (PST)
MIME-Version: 1.0
References: <20240205154909.2392231-1-jens.wiklander@linaro.org> <3fc56b77-9b8c-402f-a338-5f415e5ab603@citrix.com>
In-Reply-To: <3fc56b77-9b8c-402f-a338-5f415e5ab603@citrix.com>
From: Jens Wiklander <jens.wiklander@linaro.org>
Date: Tue, 6 Feb 2024 07:49:40 +0100
Message-ID: <CAHUa44GK6XKLcjB0NkQS3uqr9Fp+Z0CrOKWJDMivfJLju_FsEg@mail.gmail.com>
Subject: Re: [XEN PATCH v4] xen/arm: ffa: reclaim shared memory on guest destroy
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, patches@linaro.org, 
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>, Stefano Stabellini <sstabellini@kernel.org>, 
	Julien Grall <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, 
	Michal Orzel <michal.orzel@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 5, 2024 at 5:54=E2=80=AFPM Andrew Cooper <andrew.cooper3@citrix=
.com> wrote:
>
> On 05/02/2024 3:49 pm, Jens Wiklander wrote:
> > diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
> > index 0793c1c7585d..bbb6b819ee2b 100644
> > --- a/xen/arch/arm/tee/ffa.c
> > +++ b/xen/arch/arm/tee/ffa.c
> > @@ -992,53 +1008,75 @@ static void put_shm_pages(struct ffa_shm_mem *sh=
m)
> >      }
> >  }
> >
> > -static bool inc_ctx_shm_count(struct ffa_ctx *ctx)
> > +static bool inc_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
> >  {
> >      bool ret =3D true;
> >
> >      spin_lock(&ctx->lock);
> > +
> > +    /*
> > +     * If this is the first shm added, increase the domain reference
> > +     * counter as we need to keep domain around a bit longer to reclai=
m the
> > +     * shared memory in the teardown path.
> > +     */
> > +    if ( !ctx->shm_count )
> > +        get_knownalive_domain(d);
> > +
> >      if (ctx->shm_count >=3D FFA_MAX_SHM_COUNT)
> >          ret =3D false;
> >      else
> >          ctx->shm_count++;
> > +
> >      spin_unlock(&ctx->lock);
>
> This is subtle.  It reads as if there is a reference leak.  There really
> will be one if FFA_MAX_SHM_COUNT happens to be 0.
>
> You could add a BUILD_BUG_ON(), but IMO it would be far clearer to
> follow if you moved the get_knownalive_domain() into the else clause.

I see your point, I'll fix it in the next version.

>
> >
> >      return ret;
> >  }
> >
> > -static void dec_ctx_shm_count(struct ffa_ctx *ctx)
> > +static void dec_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
> >  {
> >      spin_lock(&ctx->lock);
> > +
> >      ASSERT(ctx->shm_count > 0);
> >      ctx->shm_count--;
> > +
> > +    /*
> > +     * If this was the last shm removed, let go of the domain referenc=
e we
> > +     * took in inc_ctx_shm_count() above.
> > +     */
> > +    if ( !ctx->shm_count )
> > +        put_domain(d);
> > +
> >      spin_unlock(&ctx->lock);
>
> You want a local bool called drop_ref, set within the lock, and move the
> put_domain() down here.  put_domain() is potentially a large operation.

OK, I'll fix it in the next version.

Thanks,
Jens


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 09:16:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 09:16:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676680.1052883 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHYS-0008OO-3q; Tue, 06 Feb 2024 09:15:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676680.1052883; Tue, 06 Feb 2024 09:15:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHYS-0008OH-1G; Tue, 06 Feb 2024 09:15:48 +0000
Received: by outflank-mailman (input) for mailman id 676680;
 Tue, 06 Feb 2024 09:15:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXHYQ-0008OB-Hz
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 09:15:46 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4e3981c2-c4d0-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 10:15:44 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-40fd2f7ef55so1786745e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 01:15:44 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s13-20020a05600c45cd00b0040e4733aecbsm1315832wmo.15.2024.02.06.01.15.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 01:15:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e3981c2-c4d0-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707210943; x=1707815743; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Hcgqavd5ViKDWn4ZTVmJrmTvxwn61LKHCoTm2YoE7hA=;
        b=fCfQnFGL58CZIHgjY5Uw6DwGDO2ethUIVrmW4SMv0gQjJ4J8uxnhMcP1coIsX4NnS5
         GqkXQBS6IRFMmxfz4+tjtlifnjozjVlZ3gXRWtVpLe9o3jug3YZ5aBCx6KBfijLQjQ1Y
         VOv4hrGE8QXJSpWudmyOS9mkNM8WF8pGoptqOq36TCKOaav9mSyUu8RfAXB/m4PXjZEs
         pjtdD/C3wb9io9sqf28dmaDwV/q/8wqNGVt7mJ/0Yj+iSi6ZHwIQcDkHmEQbxDBCy3IP
         MuAuFafn7/VmUGvmURPElygRXQ6L7vtD+FjApiyloREOVRO40UTizmVXjW8neYHCNZSM
         6Gwg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707210943; x=1707815743;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Hcgqavd5ViKDWn4ZTVmJrmTvxwn61LKHCoTm2YoE7hA=;
        b=qwtzEKVzrimJzg7fZTLJYO5yceypGrar7otIuTkrUYjImpoYWERPc0lTmXrDZ2IUi4
         7q2ver8MDWRw0D1qWYluGlOxY3jZYBv0Oed6iEXtC1Epo42OdmMIvysD7HPtbPzTR+oS
         uLkGvvSDxCGP+xeurlaM0vRzZJt8jS2H1zce1ejgYDcZ8sAhZWBWMiUHBc5fudV4g5d2
         oLLZAkII8WKn+RJbaBWIt5fDMAgGPNRUU10WgoK9s99yZEENUlXwDRgHAcUd7GDGvQjy
         q48XjK4k3n3sz1kcmt1GowVtbUJ5Sb6jxtODmZD5hfCn5yujpdhVzuADlZJ7maiegXt8
         FRSA==
X-Gm-Message-State: AOJu0YyGOj5A95AjHEC/mdhdPdMFS+xB1BDTDLxU/hUxvk9jQDQiBpHH
	pT077ohEPvOMiZHeBQUVdZv8j2rpNS4aB63KOVhTlyrCI1M07aE0WSDRXOiLIQ==
X-Google-Smtp-Source: AGHT+IGixe5/V/9tAra8sNR0z5Fhi8t7iskxkyEhgHeYdBCuKeSqZA310qZgVXel9JR/uG8R03ZqCQ==
X-Received: by 2002:a05:600c:474f:b0:40f:c234:2006 with SMTP id w15-20020a05600c474f00b0040fc2342006mr1652814wmo.8.1707210943592;
        Tue, 06 Feb 2024 01:15:43 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXZ39n2kynwBnXkyRX+VvxOY8DfbiGNZ6bvj34kJIzCycUOI/ghZYC+ANDPL71QeNrIx+MDMb8f2N8x1yHfHKCK0OW2nXK0gXyon8JbKSIDBMlP711ldzy2KL5cBuE2yXv0J0BZKaZD+pDqUWmn2Iv1eaDeLUEAEz+hfaoYwZwBqqxXt1sIK6O8PWi0qduNxjqPyJpbMxfrf55J6s37bh0=
Message-ID: <dfb48bce-378d-478d-b65e-ab8e27aedcf3@suse.com>
Date: Tue, 6 Feb 2024 10:15:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/5] Remove useless assignment
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Julien Grall <julien@xen.org>,
 Frediano Ziglio <frediano.ziglio@cloud.com>
References: <20240205104637.14868-1-frediano.ziglio@cloud.com>
 <alpine.DEB.2.22.394.2402051241240.1925432@ubuntu-linux-20-04-desktop>
 <cbdf382c-a688-45f5-b834-9e3ab70ffc53@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <cbdf382c-a688-45f5-b834-9e3ab70ffc53@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 22:21, Julien Grall wrote:
> The tag says '2/5' but I don't see a thread. Is the series meant to 
> contain more patches?
> 
> Also, the title is not very specific about where the assignment is 
> removed. I have committed with the following title:
> 
> xen/evtchn: Remove useful assignment in evtchn_alloc_unbound

Luckily in what you committed it is actually "useless" ;-)

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 09:17:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 09:17:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676682.1052893 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHZl-0000Ri-DH; Tue, 06 Feb 2024 09:17:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676682.1052893; Tue, 06 Feb 2024 09:17:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHZl-0000Rb-AE; Tue, 06 Feb 2024 09:17:09 +0000
Received: by outflank-mailman (input) for mailman id 676682;
 Tue, 06 Feb 2024 09:17:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXHZk-0000RT-VM
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 09:17:08 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8014f883-c4d0-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 10:17:07 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-40fdf17fa89so1884255e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 01:17:07 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s13-20020a05600c45cd00b0040e4733aecbsm1315832wmo.15.2024.02.06.01.17.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 01:17:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8014f883-c4d0-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707211027; x=1707815827; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=arBGPFrOLwqCftlxAZTfXEpNoXrQlI3w1e8E8Eg7n1w=;
        b=AHMkt1Tum5vSProNBRaH5p7S7cZNts2se5XgXRj9b5CySQREVjUBhIsHkht+VqZvtC
         ngzv9R2bCvSnXlQZSPv88pnB33Y7j4f/Jlkqt7hJg7+7tvZKLa5Ro+7g6KCclrwV7DD3
         rXaVMtW/jLEa2vfb8Bm5Dip/1sywaGtN1J73BFhU724QHmKhyzTUxuY762PRc9BUDR+J
         NkPBEq9npa+7XZbCBAotoxN9fcDlxY3wUu/0McOXLQqwECYPx7mjC5nTPTCp1Xx/JLvy
         Pic21MbL6K0VgWrg+sLtDllrfx48snk30hPai7jAYceQudDy1zoiX17TYX1QlM6QUlWD
         9y+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707211027; x=1707815827;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=arBGPFrOLwqCftlxAZTfXEpNoXrQlI3w1e8E8Eg7n1w=;
        b=NRDb1P5pe7tciTenH6UjzA65tpTGf/29/0aaxQu7cXb4Tb1pT2UOuV4gy3qZh0KQz2
         9egwMlzmcpb5AgJTM6lH4d9nW9YsnxGDJZ9mZ0bGYlIjNmb+yShWcFuY/b8VjimOr4vG
         nROmEiVdekkiT6wySjcg/074T8tfubJww5dSz1SNwTf8lklRXWEVCJuUFCIXCUfyqlp/
         1zSvnNd8XP4nJabKd3/7zs3xpiuP4SmIbtYx3UlJ4ufU3CcLQIEGxVmeeBnGElj+0B8W
         VrBzlOs7jCYXaep6BZzfzIAylfyNfRuNA7Bnxt8UZ/ix1x0Lx/u0rZ0clFIQAdCqnMMx
         9pTQ==
X-Gm-Message-State: AOJu0YwT+Y2OnLRcR+KQDN46qZDBPNSecigYHWN8HwqLR8tYEjU5x/hm
	senYJA7cj4MZaGquWK8bOFAVjuS4p1ojSoKI34HLOM/4wNAbQKMs0JXxEh7bAA==
X-Google-Smtp-Source: AGHT+IFEpZh0/jizEzpIcYwCMsFRDVOBRr3/+nvor086leDd6s4pqGIVz31Xcc8UEDv8QOWdLEvmHA==
X-Received: by 2002:a05:600c:474f:b0:40f:c234:2006 with SMTP id w15-20020a05600c474f00b0040fc2342006mr1657194wmo.8.1707211027304;
        Tue, 06 Feb 2024 01:17:07 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVZRKutFXgwgIQVZPbZI4lhx7CqzhIP76aQ9f3DTPUjtDpZmqaOcmAUxBkcivl1MdLMC7K+DL1YXNdY4My4v8XVcjYre9Po/5JQwL5ZIerIyqpP0lF9DsEzCJN5J4sC67MC4cCIMPuWmL2UEzepWDwBNS0Q2bsWYpi7iSPUKzPp8v+vpZVoFT9JvS3zlUscufgMFOgZkBAqjFvzv68qzuY=
Message-ID: <6f8b3145-3e7a-4968-b68b-efa56c99fe55@suse.com>
Date: Tue, 6 Feb 2024 10:17:06 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/5] Remove useless assignment
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Frediano Ziglio <frediano.ziglio@cloud.com>
References: <20240205104637.14868-1-frediano.ziglio@cloud.com>
 <alpine.DEB.2.22.394.2402051241240.1925432@ubuntu-linux-20-04-desktop>
 <cbdf382c-a688-45f5-b834-9e3ab70ffc53@xen.org>
 <dfb48bce-378d-478d-b65e-ab8e27aedcf3@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <dfb48bce-378d-478d-b65e-ab8e27aedcf3@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 10:15, Jan Beulich wrote:
> On 05.02.2024 22:21, Julien Grall wrote:
>> The tag says '2/5' but I don't see a thread. Is the series meant to 
>> contain more patches?
>>
>> Also, the title is not very specific about where the assignment is 
>> removed. I have committed with the following title:
>>
>> xen/evtchn: Remove useful assignment in evtchn_alloc_unbound
> 
> Luckily in what you committed it is actually "useless" ;-)

Oh, I'm sorry - I see I got To: and Cc: mixed up in the earlier reply.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 09:24:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 09:24:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676689.1052902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHgt-00023p-4j; Tue, 06 Feb 2024 09:24:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676689.1052902; Tue, 06 Feb 2024 09:24:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHgt-00023i-1d; Tue, 06 Feb 2024 09:24:31 +0000
Received: by outflank-mailman (input) for mailman id 676689;
 Tue, 06 Feb 2024 09:24:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXHgs-00023c-67
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 09:24:30 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 865991fa-c4d1-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 10:24:27 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-40fd55c7f93so5363375e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 01:24:27 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bi24-20020a05600c3d9800b0040fed19162bsm357907wmb.28.2024.02.06.01.24.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 01:24:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 865991fa-c4d1-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707211467; x=1707816267; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=laPJ5Nt1+Y0hCXDgUPaCxT0oID96i9h2VojfeyHBMFE=;
        b=Ob/AwHpdBCVhfIlCGhpUQ8HhEF5tqigvYHmGlLQn5bnKi87pqnqYMphdigsHoPzmz/
         ZCf7I2w1umJsK+vKw+dZlByHq8FM01PXmwT5uA3aycsyl4lr8O6adWFurw1VW5X2skDH
         KO7tarmDhuPxR+ih5WPp2mGEZcNcvxDVg4INQ4Rn0O5GlXBAYXJgW+Zo1Jla9xtDFJSc
         TQObElQyEYGjtaQvSWk/nld+rOmCQOvSdx2xyYBXHdxa8h3MvocbxMx1/RxE+IZcY6Oy
         eF5QbTwuRwGxgu6deWSSXiQcoyq1hTgL2ZyT1qJUChEeXmsaIPoDIzRuctTD6revifIv
         uVAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707211467; x=1707816267;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=laPJ5Nt1+Y0hCXDgUPaCxT0oID96i9h2VojfeyHBMFE=;
        b=wejt9jBQ5hVFTiiaHPOYuenOdc1H3W16y0/KgriHUOPEgu7dOioiiVqMEkxQgYHH27
         WQpOZwICGklN8iC08vPjRB31X8ZTa+RF/juEWwt3vpkXeC/Y/VZciNrnOpAz0KF/gRi9
         HrmoUpjmH2AvpANulaFFEA5T5zWWL494pc2vVjop2YPg8P/2Fk4KxV6nu50BJMjFSKPC
         QRjcvR9AcWlSQ4TA5LW8t2zZDJMlK13EjtqXFd3U5NUiX/Se0u+WtRpFZnG4Wpt0FQ2s
         mLqZCpyEW9AvV6MimEvJp6fvnJaW+fiF/ehBb3F7a+hhm9wTyZ2uxUI3yTsGUaSH+3N2
         X1ug==
X-Gm-Message-State: AOJu0YyB1Yqm3IkPhIRl/tfDp9ynSwzHkFDo77E1XtX1DPUf1/TDOpt8
	KA+5dUrjxPQLiaRLae75DBXXMOgcrWD9y3B27sGnMFXe/Cat9D9ZphYJMDDnpA==
X-Google-Smtp-Source: AGHT+IG0d1F9cWnsObh6/1ZrVDtHJPPRl3PXXA4/mO8sltBu3Ec5SfmIKdFiNm6Wljdb9NTdKp2hCQ==
X-Received: by 2002:a05:600c:3ba2:b0:40f:86:fea with SMTP id n34-20020a05600c3ba200b0040f00860feamr1795403wms.13.1707211467199;
        Tue, 06 Feb 2024 01:24:27 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXgAzG+UOupj0iGMqmRS2VtP3PNJmovj4ruwkTX/8RqMwr5RkGHfrXJ43bTvwZ3Woh1WcKVwYvSVkRX3Qwe4IaiRdTavDNHw6rmVV+yRov2QmP3bQZZIekQLAzN/2RaigjNp5wMfMXipaWXSBCWJULKt9mMtwsgEkQLGMPSemv0u4Q2gi9XoA==
Message-ID: <7799755f-fad2-4f73-a2c1-de97bcce95fa@suse.com>
Date: Tue, 6 Feb 2024 10:24:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen 4.19 release status tracking list
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: committers@xenproject.org, community.manager@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <CAMacjJzio2EDwudCUHoL6psKYztkHZOWwLNDknATvpN6Y0im=Q@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAMacjJzio2EDwudCUHoL6psKYztkHZOWwLNDknATvpN6Y0im=Q@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 20:23, Oleksii Kurochko wrote:
> Hello everyone,
> 
> I would like to share with you a list for status tracking based on Xen ML:
> 
> Arm:
>   * [PATCH v5 00/13] Arm cache coloring [
> https://lore.kernel.org/xen-devel/20240102095138.17933-1-carlo.nonato@minervasys.tech/
> ]
>   * [PATCH v1 repost 0/4] xen/arm64: Rework the MMU-off code (idmap)
> so it is self-contained [
> https://lore.kernel.org/xen-devel/20240116143709.86584-1-julien@xen.org/
> ]
>   * [PATCH v13 00/14] PCI devices passthrough on Arm, part 3 [
> https://lore.kernel.org/xen-devel/20240202213321.1920347-1-stewart.hildebrand@amd.com/
> ]
> 
> PPC:
>   *  [PATCH v2 0/7] Early Boot Allocation on Power [
> https://lore.kernel.org/xen-devel/cover.1702607884.git.sanastasio@raptorengineering.com/
> ]
> 
> RISC-V:
>   *  [PATCH v4 00/30]  Enable build of full Xen for RISC-V [
> https://lore.kernel.org/xen-devel/cover.1707146506.git.oleksii.kurochko@gmail.com/
> ]
> 
> x86:
>   * [PATCH 0/4] iommu/x86: fixes/improvements for unity range checks [
> https://lore.kernel.org/xen-devel/20240201170159.66330-1-roger.pau@citrix.com/
> ]
>   * [PATCH 0/8] x86: support AVX10.1 [
> https://lore.kernel.org/xen-devel/298db76f-d0ee-4d47-931f-1baa1a7546cf@suse.com/
> ]
>   * [PATCH v13 00/35] x86: enable FRED for x86-64 [
> https://lore.kernel.org/xen-devel/20231205105030.8698-1-xin3.li@intel.com/
> ]

This is a kernel series. I don't expect FRED support is in scope for
4.19.

>   * [PATCH v4 0/8] x86emul: misc additions
> https://lore.kernel.org/xen-devel/9dd23064-c79e-4a50-9c71-c0e73b189944@suse.com/
>   * [PATCH 0/7] VT-d: SATC handling and ATS tidying [
> https://lore.kernel.org/xen-devel/25506838-b818-4686-8c16-3a198338af44@suse.com/
> ]
>   * [XEN PATCH 0/9] x86: parallelize AP bring-up during boot [
> https://lore.kernel.org/xen-devel/cover.1699982111.git.krystian.hebel@3mdeb.com/
> ]
>   * [PATCH v2 00/12] x86: memcpy() / memset() (non-)ERMS flavors plus
> fallout [ https://lore.kernel.org/xen-devel/8f56a8f4-0482-932f-96a9-c791bebb4610@suse.com/
> ]
> 
> common:
>   * annotate entry points with type and size" series [
> https://lore.kernel.org/xen-devel/5e74a501-2824-42af-804b-47b515aa9bcf@suse.com/
> ]

The bulk of this has gone in, but there'll want to be follow-ups.

>   * [PATCH v2 0/8] limit passing around of cpu_user_regs [
> https://lore.kernel.org/xen-devel/33db57a2-b569-4fa8-b0b0-36ea92528d7b@suse.com/
> ]
>   * [PATCH v2 (resend) 00/27] Remove the directmap [
> https://lore.kernel.org/xen-devel/20240116192611.41112-1-eliasely@amazon.com/
> ]
>   * [PATCH v4 00/32] tools: enable xenstore-stubdom to use 9pfs [
> https://lore.kernel.org/xen-devel/20240205105001.24171-1-jgross@suse.com/
> ]
>   * [PATCH v7 0/7] Introduce generic headers [
> https://lore.kernel.org/xen-devel/cover.1706281994.git.oleksii.kurochko@gmail.com/
> ]
>   * [PATCH] move __read_mostly to xen/cache.h
> [https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/]
>   * [PATCH v2] NUMA: no need for asm/numa.h when !NUMA [
> https://lore.kernel.org/xen-devel/94cffa48-5e31-4557-827d-c6dec6031d05@suse.com/
> ]

This went in already.

>   * [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE() [
> https://lore.kernel.org/xen-devel/42fc6ae8d3eb802429d29c774502ff232340dc84.1706259490.git.federico.serafini@bugseng.com/
> ]
>   * [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(),
> ASSERT_UNREACHABLE() to xen/bug.h [
> https://lore.kernel.org/xen-devel/4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com/
> ]
>   * [PATCH] xen/lib: introduce generic find next bit operations [
> https://lore.kernel.org/xen-devel/52730e6314210ba4164a9934a720c4fda201447b.1706266854.git.oleksii.kurochko@gmail.com/
> ]
> 
> Please reply with items you would like to see in 4.19 so that people
> know what is happening and prioritize accordingly.

Beyond (on x86) AVX10 I'd like to see whether we can also make some
progress towards supporting APX. That'll be quite a bit of work, though,
including quite a bit of reviews that are likely to end up on the
backburner (as has been commonly happening for, especially, more
involved emulator changes).

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 09:28:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 09:28:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676696.1052912 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHkm-0002vS-Ni; Tue, 06 Feb 2024 09:28:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676696.1052912; Tue, 06 Feb 2024 09:28:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHkm-0002vL-Kr; Tue, 06 Feb 2024 09:28:32 +0000
Received: by outflank-mailman (input) for mailman id 676696;
 Tue, 06 Feb 2024 09:28:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=x4wk=JP=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXHkl-0002vF-3O
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 09:28:31 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1611637c-c4d2-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 10:28:28 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a380c98e824so78472066b.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 01:28:28 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 cx6-20020a170907168600b00a381da7b320sm623769ejd.177.2024.02.06.01.28.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 06 Feb 2024 01:28:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1611637c-c4d2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707211708; x=1707816508; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=rJvSmlDSKQ+A0UGCJt6qFwMR0lxaOLbqyGG2NiGrajQ=;
        b=A8l0HFeK9fPo1xWtlcW6am7l0wvVxx1Thb+dq1VxYg69qWGI3L/MR8pfwa0JirIy2p
         yTmmlUE+ZXunFWIzvdrpwpTp9WI1MhYIHabCNe78qdRf5fpZL1VzVW3l0aEWhFqc0yoB
         5yHS9mAKSqHEwZKmE4M8iLEScc7sLsY11Oxvg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707211708; x=1707816508;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rJvSmlDSKQ+A0UGCJt6qFwMR0lxaOLbqyGG2NiGrajQ=;
        b=Ow3RskDEaNJl8vNNwzxBwUf15upcs3JZcx4dQz652jgOdLo4XBsk365rrlLvqnDdjB
         c0bp/aggWsSDLGvmdSsQ+xcqh+BEm1I57UsLzUU/kCx5ZsZ6oZVjpVd33wUa+ge6pYUC
         kZdxSLdPMg4sQqk6wdoJQFuYVmu2fEy2HWIF5eP71TW2D4c1OH/9VdviSK5FGRh/dIcZ
         eMfFb2fQZQtqpfiXouagSvncNYv8yhjUm4sylhjOG4hXPDsga7fWisP966SMOeWE90JY
         dA1n8Yjx+Yy47OfCZZU4xr5dUYT17uP1mBr9oizLxka43Z7Ay2S6ewqhqVS3zc4uNlYg
         TpDQ==
X-Gm-Message-State: AOJu0YyGV6LBQbIotkFrj0UyUw0XuYrn6Z9S+2FRPIry6z7GpVvvqg97
	GghnXkDdWP/JzQ1n6LrVx5UVdxr3XleiMaUj+C4iZDtNo2zsyRAo+BcQ8Q+AC3Q=
X-Google-Smtp-Source: AGHT+IG9xsl73RafWKOZHSFzGvdyUvUcw60yoFKkmDrKvlNji1NANozWPHJ9ZG3B+kXThzFP1ulU9g==
X-Received: by 2002:a17:906:512:b0:a37:ad51:86dc with SMTP id j18-20020a170906051200b00a37ad5186dcmr1327806eja.9.1707211708400;
        Tue, 06 Feb 2024 01:28:28 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXtyZ2u60eFlJ82YbG7g/CVSzuGf8YwjumT/CR7vTQioOA6+C0Pyaoo+cXuj6tDdFg9KlohKT6r01U5k9piHhx7Mm7/s3a7/9aQvkjITm3UMtx0ACn8RCmdsKZaMVLWbKFdS0gmBtZglfyIFEeOuL3Z1KKuTfvZnNu5AlqMS4FQSrYZgA==
Date: Tue, 6 Feb 2024 10:28:27 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	Kevin Tian <kevin.tian@intel.com>,
	Jun Nakajima <jun.nakajima@intel.com>
Subject: Re: [PATCH v4.5 3/8] VMX: tertiary execution control infrastructure
Message-ID: <ZcH7uwpxwYUq9yR0@macbook>
References: <9dd23064-c79e-4a50-9c71-c0e73b189944@suse.com>
 <aa553449-888f-4e52-85b7-0bc0b7f010b4@suse.com>
 <bc782b14-d897-4a94-b71d-97c4abeb85df@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <bc782b14-d897-4a94-b71d-97c4abeb85df@suse.com>

On Mon, Feb 05, 2024 at 02:37:44PM +0100, Jan Beulich wrote:
> This is a prereq to enabling e.g. the MSRLIST feature.
> 
> Note that the PROCBASED_CTLS3 MSR is different from other VMX feature
> reporting MSRs, in that all 64 bits report allowed 1-settings.
> 
> vVMX code is left alone, though, for the time being.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

> ---
> v4.5: Bump zero-padding width in vmcs_dump_vcpu(). Add
>       TERTIARY_EXEC_VIRT_SPEC_CTRL. Constify
>       vmx_update_tertiary_exec_control()'s parameter. Re-base.
> v2: New.
> 
> --- a/xen/arch/x86/hvm/vmx/vmcs.c
> +++ b/xen/arch/x86/hvm/vmx/vmcs.c
> @@ -164,6 +164,7 @@ static int cf_check parse_ept_param_runt
>  u32 vmx_pin_based_exec_control __read_mostly;
>  u32 vmx_cpu_based_exec_control __read_mostly;
>  u32 vmx_secondary_exec_control __read_mostly;
> +uint64_t vmx_tertiary_exec_control __read_mostly;
>  u32 vmx_vmexit_control __read_mostly;
>  u32 vmx_vmentry_control __read_mostly;
>  u64 vmx_ept_vpid_cap __read_mostly;
> @@ -228,10 +229,32 @@ static u32 adjust_vmx_controls(
>      return ctl;
>  }
>  
> -static bool cap_check(const char *name, u32 expected, u32 saw)
> +static uint64_t adjust_vmx_controls2(
> +    const char *name, uint64_t ctl_min, uint64_t ctl_opt, unsigned int msr,
> +    bool *mismatch)
> +{
> +    uint64_t vmx_msr, ctl = ctl_min | ctl_opt;
> +
> +    rdmsrl(msr, vmx_msr);
> +
> +    ctl &= vmx_msr; /* bit == 0 ==> must be zero */
> +
> +    /* Ensure minimum (required) set of control bits are supported. */
> +    if ( ctl_min & ~ctl )
> +    {
> +        *mismatch = true;
> +        printk("VMX: CPU%u has insufficient %s (%#lx; requires %#lx)\n",
> +               smp_processor_id(), name, ctl, ctl_min);
> +    }
> +
> +    return ctl;
> +}
> +
> +static bool cap_check(
> +    const char *name, unsigned long expected, unsigned long saw)
>  {
>      if ( saw != expected )
> -        printk("VMX %s: saw %#x expected %#x\n", name, saw, expected);
> +        printk("VMX %s: saw %#lx expected %#lx\n", name, saw, expected);
>      return saw != expected;
>  }
>  
> @@ -241,6 +264,7 @@ static int vmx_init_vmcs_config(bool bsp
>      u32 _vmx_pin_based_exec_control;
>      u32 _vmx_cpu_based_exec_control;
>      u32 _vmx_secondary_exec_control = 0;
> +    uint64_t _vmx_tertiary_exec_control = 0;
>      u64 _vmx_ept_vpid_cap = 0;
>      u64 _vmx_misc_cap = 0;
>      u32 _vmx_vmexit_control;
> @@ -274,7 +298,8 @@ static int vmx_init_vmcs_config(bool bsp
>      opt = (CPU_BASED_ACTIVATE_MSR_BITMAP |
>             CPU_BASED_TPR_SHADOW |
>             CPU_BASED_MONITOR_TRAP_FLAG |
> -           CPU_BASED_ACTIVATE_SECONDARY_CONTROLS);
> +           CPU_BASED_ACTIVATE_SECONDARY_CONTROLS |
> +           CPU_BASED_ACTIVATE_TERTIARY_CONTROLS);
>      _vmx_cpu_based_exec_control = adjust_vmx_controls(
>          "CPU-Based Exec Control", min, opt,
>          MSR_IA32_VMX_PROCBASED_CTLS, &mismatch);
> @@ -338,6 +363,15 @@ static int vmx_init_vmcs_config(bool bsp
>              MSR_IA32_VMX_PROCBASED_CTLS2, &mismatch);
>      }
>  
> +    if ( _vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS )
> +    {
> +        uint64_t opt = 0;
> +
> +        _vmx_tertiary_exec_control = adjust_vmx_controls2(
> +            "Tertiary Exec Control", 0, opt,
> +            MSR_IA32_VMX_PROCBASED_CTLS3, &mismatch);
> +    }
> +
>      /* The IA32_VMX_EPT_VPID_CAP MSR exists only when EPT or VPID available */
>      if ( _vmx_secondary_exec_control & (SECONDARY_EXEC_ENABLE_EPT |
>                                          SECONDARY_EXEC_ENABLE_VPID) )
> @@ -468,6 +502,7 @@ static int vmx_init_vmcs_config(bool bsp
>          vmx_pin_based_exec_control = _vmx_pin_based_exec_control;
>          vmx_cpu_based_exec_control = _vmx_cpu_based_exec_control;
>          vmx_secondary_exec_control = _vmx_secondary_exec_control;
> +        vmx_tertiary_exec_control  = _vmx_tertiary_exec_control;
>          vmx_ept_vpid_cap           = _vmx_ept_vpid_cap;
>          vmx_vmexit_control         = _vmx_vmexit_control;
>          vmx_vmentry_control        = _vmx_vmentry_control;
> @@ -503,6 +538,9 @@ static int vmx_init_vmcs_config(bool bsp
>              "Secondary Exec Control",
>              vmx_secondary_exec_control, _vmx_secondary_exec_control);
>          mismatch |= cap_check(
> +            "Tertiary Exec Control",
> +            vmx_tertiary_exec_control, _vmx_tertiary_exec_control);
> +        mismatch |= cap_check(
>              "VMExit Control",
>              vmx_vmexit_control, _vmx_vmexit_control);
>          mismatch |= cap_check(
> @@ -1080,6 +1118,7 @@ static int construct_vmcs(struct vcpu *v
>          v->arch.hvm.vmx.exec_control |= CPU_BASED_RDTSC_EXITING;
>  
>      v->arch.hvm.vmx.secondary_exec_control = vmx_secondary_exec_control;
> +    v->arch.hvm.vmx.tertiary_exec_control  = vmx_tertiary_exec_control;
>  
>      /*
>       * Disable features which we don't want active by default:
> @@ -1134,6 +1173,10 @@ static int construct_vmcs(struct vcpu *v
>          __vmwrite(SECONDARY_VM_EXEC_CONTROL,
>                    v->arch.hvm.vmx.secondary_exec_control);
>  
> +    if ( cpu_has_vmx_tertiary_exec_control )
> +        __vmwrite(TERTIARY_VM_EXEC_CONTROL,
> +                  v->arch.hvm.vmx.tertiary_exec_control);
> +
>      /* MSR access bitmap. */
>      if ( cpu_has_vmx_msr_bitmap )
>      {
> @@ -2068,10 +2111,12 @@ void vmcs_dump_vcpu(struct vcpu *v)
>                 vmr(HOST_PERF_GLOBAL_CTRL));
>  
>      printk("*** Control State ***\n");
> -    printk("PinBased=%08x CPUBased=%08x SecondaryExec=%08x\n",
> +    printk("PinBased=%08x CPUBased=%08x\n",
>             vmr32(PIN_BASED_VM_EXEC_CONTROL),
> -           vmr32(CPU_BASED_VM_EXEC_CONTROL),
> -           vmr32(SECONDARY_VM_EXEC_CONTROL));
> +           vmr32(CPU_BASED_VM_EXEC_CONTROL));
> +    printk("SecondaryExec=%08x TertiaryExec=%016lx\n",

I thought you wanted to split the print into two 32bit halves here?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 09:34:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 09:34:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676700.1052922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHqR-0004NM-9i; Tue, 06 Feb 2024 09:34:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676700.1052922; Tue, 06 Feb 2024 09:34:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHqR-0004NF-74; Tue, 06 Feb 2024 09:34:23 +0000
Received: by outflank-mailman (input) for mailman id 676700;
 Tue, 06 Feb 2024 09:34:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXHqQ-0004N9-6i
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 09:34:22 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e71c25c8-c4d2-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 10:34:20 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d090c83d45so40151481fa.3
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 01:34:20 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n1-20020a05640206c100b0055fd90b0556sm836293edy.90.2024.02.06.01.34.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 01:34:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e71c25c8-c4d2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707212059; x=1707816859; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=d9nNKtEgZz65B54Rk3a0q+Gvw5E3vaE1jvwO7dd9KVQ=;
        b=fgIbipHyq/mD2Q7q2EoA3KzNUcP4Z/JSbJ183VJEiO1Y7DVVXJE2kLT5SMjnHmGlIe
         YoH9nRqfjJkThQw7P06ifC32lVLOrZK0mXWE+ZP2/5Yyu7PVFKQjyNiRtTWq+sBZ7Pzt
         NawNve/y9GGyW+U71Bdsw2ETiw1EBMS4pFlPxinnxJckhpHHqTfE73D9r6Fm7vWMFsEB
         b360qc/1t/Q5l3rgzGe7owe1m60nQCtOBORHBCJ6qPHFnBQeJnXlBqPyJXiwTxt10IR1
         XqSH8aZ41nCuJauJKTnRhKvOZb6KY1kTUcyYmw1mY4ZIVTfYMmzWBIiZQSGHlb8gHdav
         DZqw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707212059; x=1707816859;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=d9nNKtEgZz65B54Rk3a0q+Gvw5E3vaE1jvwO7dd9KVQ=;
        b=G0k3Ye6Yc3sS0CuXBGjVpTbu5J+ZFd8tCfyZjXDptzBhQM9YVAPrsB7POE6I8Cbk4x
         o4F6oGteQd/d3Twy5pUZl7+eCQtWPEJ4L/W3No7MhedhtwU0pKfZyKxuU8u5bMWuEAjh
         pnPFl3+y6+aGCUs3GO8fniyzCMkiKd6HxVr0LdOR9K8Qs7/x3Y2Vx+uKLOhgWZ9ShUFa
         Ie8ON1j0svu7j0FhcIBhsxFwwjCYwXUIb1wducmtBDXSEhk62f50OUiyfBxJjesKNLC6
         uSjF6PkIhEA3JNqn+2fuQTfj4dc5/+wjoqegN6M+kA6MSCIemRlOY6cUHxGlrTmSIamf
         lB+w==
X-Gm-Message-State: AOJu0Yw1zW8yFyu2sSCbwgnvY8FB4G9/SsyTwUVDzGKrNZbZkm3Fm2U6
	PRH/BNwXcAV9juNb8EF8Jy8rVsHxQfaojLVTnFO8rDzBcG0HgVs5iTUo91OUOg==
X-Google-Smtp-Source: AGHT+IH7jvvefEXIsXF/CAVB+vVMCHIJrhUenBSP+iKIREr0wPKr0mOzWcNAKL/R7oPBRVqxn6tsqA==
X-Received: by 2002:a2e:95c9:0:b0:2d0:9efa:666f with SMTP id y9-20020a2e95c9000000b002d09efa666fmr1536096ljh.21.1707212059020;
        Tue, 06 Feb 2024 01:34:19 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXBVf3WloR43BkmJiX3X/UfAh5+rShGkDL/YoOJPiRKM6Je7XXY740I5amSKOwRSsgknjjtjOlLXVRl8oC50HVndpQY7mT1wacwErA+6LrxCBsv5gQFgzUP4JUPc1Mo0mzTWI8ROrotQT23jsdzae0cDRIMMhkoQnhssrWXUfts63dkNyNRiZ6XJMwC174=
Message-ID: <1256d6a8-5744-4790-b12c-8a36b9d144ed@suse.com>
Date: Tue, 6 Feb 2024 10:34:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/bitmap: Compile with -Wsign-conversion
To: Julien Grall <julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Xen-devel <xen-devel@lists.xenproject.org>, Wei Liu <wl@xen.org>
References: <20240205151454.1920291-1-andrew.cooper3@citrix.com>
 <072da633-de82-4d93-b467-8372973406fe@xen.org>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <072da633-de82-4d93-b467-8372973406fe@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:55, Julien Grall wrote:
> On 05/02/2024 15:14, Andrew Cooper wrote:
>> Use pragmas to able the warning in this file only.  All supported versions of
>> Clang understand this, while older GCCs simply ignore it.
>>
>> bitmap_find_free_region() is the only function which isn't sign-convert
>> clean.  This highlights a latent bug in that it can't return successfully for
>> a bitmap larger than 2G.
>>
>> Add an extra check, and explicit cast to silence the warning.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>> CC: George Dunlap <George.Dunlap@citrix.com>
>> CC: Jan Beulich <JBeulich@suse.com>
>> CC: Stefano Stabellini <sstabellini@kernel.org>
>> CC: Wei Liu <wl@xen.org>
>> CC: Julien Grall <julien@xen.org>
>>
>> Slightly RFC.  This is our first use of pragmas like this.
> 
> The only other approach I can think of is specifying the CFLAGS per file 
> like Linux did. I don't know if our build system supports that though.

It does, see e.g.

# Allows usercopy.c to include itself
$(obj)/usercopy.o: CFLAGS-y += -iquote .

in arch/x86/Makefile.

> AFAICT, the only advantage would be to avoid duplicating the pragmas. So 
> this is not a strong preference.

My other concern there are old gcc versions we still support. I haven't
checked (yet) when support for these pragma-s was introduced; I only
know they haven't been there forever. However, ...

>> --- a/xen/common/bitmap.c
>> +++ b/xen/common/bitmap.c
>> @@ -14,6 +14,9 @@
>>   #include <xen/lib.h>
>>   #include <asm/byteorder.h>
>>   
>> +#pragma GCC diagnostic warning "-Wsign-conversion"
>> +#pragma clang diagnostic warning "-Wsign-conversion"
>> +
> 
> OOI, any reason why wasn't added at the right at the top of the file?

... this may be relevant: Inline functions may have an issue with being
processed with the warning enabled. Otoh it may also be a problem if
the warning isn't enabled for them.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 09:37:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 09:37:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676706.1052933 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHtm-00052E-Nu; Tue, 06 Feb 2024 09:37:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676706.1052933; Tue, 06 Feb 2024 09:37:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXHtm-000527-LB; Tue, 06 Feb 2024 09:37:50 +0000
Received: by outflank-mailman (input) for mailman id 676706;
 Tue, 06 Feb 2024 09:37:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXHtk-000520-LZ
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 09:37:48 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6276fda5-c4d3-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 10:37:46 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-5114cd44fdbso4043375e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 01:37:46 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d6-20020a5d5386000000b0033af5086c2dsm1626212wrv.58.2024.02.06.01.37.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 01:37:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6276fda5-c4d3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707212266; x=1707817066; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9un3fFUe0EdRZG6+nIG7Cnyyr83cTu3bB29nOdSiALU=;
        b=SU6jI8dSsPWi7eabpP3kgWuYxEWiG8JEWDBmcT4yyqeyZePyUX2+Zr9rOwOojdXBIy
         VkgOy+URlaMPVqQ11wx/z5JsiukzVJrSn+nTA/qk4gK0oJlgF6MwAFXwJX9ZVRE+77RH
         aG2L4nhwF+lQx7GeCSpgaZH7hZHQBgi4fB7aHahjKhTQaUCdSv2xCtq9o9rkbkYDggFn
         /XoHcfQOuLJe6NETL3ApuGJiLZL4uO9ZHzsWlITpCp0EFAUVRwJIJOHt3Me1GVg4gvZO
         YumBcLbSBjJ7AVWx3/myge/yW1Xkr0DOyfw3VouERnGb5lzqXIaUUkak0V1vPsO+6/6O
         MkSQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707212266; x=1707817066;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9un3fFUe0EdRZG6+nIG7Cnyyr83cTu3bB29nOdSiALU=;
        b=TAvp4oOOVr0H/QohA93DZMpXFCM/Mtw4zzY4WfFrXjaFaw9qo/Xk1Jv+iXydKU4rXw
         A0mXdjX8ccfqC24/s8kNCSKOGJXKRHs6wcjVyGuBB0PK6ABI1Psot+Gkzw0mAjku93uN
         0uRTiB3ZimW2hKXDz3WZaA79w6WvAL1s+dLl3rik73SxxnnzpLy+o7GA7HK3BpX6YWJS
         Wuk1Owbawr7mg3jyjIch/4zmzSKSyUl1JO8oYmmxBBCgdRKOgvkN9dm7UN0SnGUbuafI
         CtNZ4fKWzjDhJrtIvrpqGfwYj1DK+Sw+55XGpkOv97JmjR07Vbl5iAPrwOc7RkyUPYdh
         7Bzw==
X-Gm-Message-State: AOJu0Ywr8P0xX4aWx0I79s6fxC9h+6uzCgeykYbFluebJlX6euYQT39S
	Ckla4/i2QvWCwWKxDJbFSqC/ZzgpB+KsYD9a0OXPBtn7Ww95Ew4twQIbLEaFBw==
X-Google-Smtp-Source: AGHT+IFKyYlASpDPaTxHBmtUYrRmHkjLup9BqfUxS6DDzuFLiCgufNhQs+hKF2eKd45YHRJcMfwPqg==
X-Received: by 2002:a05:6512:2028:b0:511:4967:9f0a with SMTP id s8-20020a056512202800b0051149679f0amr1358976lfs.57.1707212266076;
        Tue, 06 Feb 2024 01:37:46 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWSRN+t5t0jQKvBqHzFJ6ZeOpmgadrvi8r3qClWw45C/ctZ0WDwoLpqxvIRZIBcVClDhNVw7pNVq4OXfpj1mBRtgoZnhEqzNbwIp7/kPsoppNliNsDdIq2Ft54Y1H/UBAkbsPSo4A6byPuMrcV1qICeJFePdaFskSi4OksymAMfQPUDew==
Message-ID: <3125dcdd-573a-4bad-a0a2-9c501d94cc82@suse.com>
Date: Tue, 6 Feb 2024 10:37:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4.5 3/8] VMX: tertiary execution control infrastructure
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <9dd23064-c79e-4a50-9c71-c0e73b189944@suse.com>
 <aa553449-888f-4e52-85b7-0bc0b7f010b4@suse.com>
 <bc782b14-d897-4a94-b71d-97c4abeb85df@suse.com> <ZcH7uwpxwYUq9yR0@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcH7uwpxwYUq9yR0@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.02.2024 10:28, Roger Pau Monné wrote:
> On Mon, Feb 05, 2024 at 02:37:44PM +0100, Jan Beulich wrote:
>> This is a prereq to enabling e.g. the MSRLIST feature.
>>
>> Note that the PROCBASED_CTLS3 MSR is different from other VMX feature
>> reporting MSRs, in that all 64 bits report allowed 1-settings.
>>
>> vVMX code is left alone, though, for the time being.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks.

>> @@ -2068,10 +2111,12 @@ void vmcs_dump_vcpu(struct vcpu *v)
>>                 vmr(HOST_PERF_GLOBAL_CTRL));
>>  
>>      printk("*** Control State ***\n");
>> -    printk("PinBased=%08x CPUBased=%08x SecondaryExec=%08x\n",
>> +    printk("PinBased=%08x CPUBased=%08x\n",
>>             vmr32(PIN_BASED_VM_EXEC_CONTROL),
>> -           vmr32(CPU_BASED_VM_EXEC_CONTROL),
>> -           vmr32(SECONDARY_VM_EXEC_CONTROL));
>> +           vmr32(CPU_BASED_VM_EXEC_CONTROL));
>> +    printk("SecondaryExec=%08x TertiaryExec=%016lx\n",
> 
> I thought you wanted to split the print into two 32bit halves here?

Indeed I wanted to, but when about to do so I noticed several other
uses of %016lx in the same function. I value consistency higher here;
if we want to introduce such splitting, I think we want to do it
uniformly. And then we may even want to think about possible ways of
indicating the desire for such splitting right in the format string.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 09:49:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 09:49:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676715.1052944 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXI4i-00077J-Ry; Tue, 06 Feb 2024 09:49:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676715.1052944; Tue, 06 Feb 2024 09:49:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXI4i-00077C-Nm; Tue, 06 Feb 2024 09:49:08 +0000
Received: by outflank-mailman (input) for mailman id 676715;
 Tue, 06 Feb 2024 09:49:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXI4h-000776-V3
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 09:49:07 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f7d8c98f-c4d4-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 10:49:06 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-33b29b5ea96so197957f8f.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 01:49:06 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 2-20020a5d47a2000000b0033b3cf1ff09sm1658565wrb.29.2024.02.06.01.49.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 01:49:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7d8c98f-c4d4-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707212946; x=1707817746; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2xxWdwj1rfmffidHvI1N7kDJOByOykfylcPogO39CiQ=;
        b=U2FzDbQ/95IhQmwQLrTFdgURnMkqPiaGESmKa1leBWNcKAU1ZCxxKpWBpM5ot+/sVi
         jsiVvFE1jgRI64ixMeXM3i2LciX+Xz0nodFCqQUPZP2kCVI9qYy2x7ZwSUPXAdK4Rf+O
         ZlX1/urGxqAoX/DUlpXq9UKLid34HwVTz3q4G21esM5WYynAomnYWX0bx6l0+7dYqk1c
         0jaqFoSrYkT8eqg9OF6R/b7qhY4sV4YcD5Qql9+Q1n/YvDDdHCYKB4wcylxYRs+w1tHO
         0aRs2f/BpYLIc2ivvYAt+cxsCioYSXYRJ5zUVjGmdzZm2MikbhqdnCSyEGyTFs0iTqBX
         EzjA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707212946; x=1707817746;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2xxWdwj1rfmffidHvI1N7kDJOByOykfylcPogO39CiQ=;
        b=CxOLDSwk95fQRQuCyh3GCCzyHZG76+lDh079UOlZ9QjJz0uxs4gBQVaFu7Vrb5o47K
         IUrtVDEMWwKjjIgnHs3st4I9BZwpzr/pIsUA5yKzRbOJSqAYG2wMwN2kdXwYXz+ZIzTO
         /CXGetJEjJiXG87Ghfh2pSRhvqJZAhIBX8KBo9bZPg8f0an/WdKNmukflN2+Ob3ruDiG
         KZUHLka51yxJ58zmIZ3y66g6plvL9bwkJdIqxPUMbFwu7VZcSuxZMM53yg2bzoFNL5yd
         GyOked4p7H2LdC8p02+GT/Is6avHKbeFQ9MwP6x5b4Yk8HcTPirMCxCwO8sks+SbUxSe
         KM3w==
X-Gm-Message-State: AOJu0YxReoe8I34K9aVFKNz2Ia5fGiYqjW1mWAUgd7KrvkPy/VQmHtN5
	bQrJQqQwtRBrrHMBmr8LzXsLNvDNbwZbmp4H3zMWSLgI3g1PUA2fzNCPjtUk+A==
X-Google-Smtp-Source: AGHT+IHsKJ9HFaHLZ6wMweln86ZCP5UdGmnUNhdFAthi6Hu2fgU4OVOOlgkX4A4f4/5cSEULKQrROQ==
X-Received: by 2002:a5d:4ecf:0:b0:33a:f090:f758 with SMTP id s15-20020a5d4ecf000000b0033af090f758mr987689wrv.27.1707212946083;
        Tue, 06 Feb 2024 01:49:06 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUd8MHWiweO30fiKWaBQhFE3weq2XpwCbgM3zxgTOQ9uW6/uiDtpJIzVM/mh55YIhKJ5W7QjTu4Vtf+y6w0cw1t2e93BFYdPbto9pR+hw8sz/0bnh77K2UuhSKnb+jWuabU+o0JM3DZgkcbQ8LoYB8+1ilUHRd+lx6tuV9YouBPhUOCIHV8mqIBROBOZJs=
Message-ID: <0ede3663-7611-4684-b223-42305a9d0b96@suse.com>
Date: Tue, 6 Feb 2024 10:49:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/bitmap: Even more signed-ness fixes
Content-Language: en-US
To: Julien Grall <julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Xen-devel <xen-devel@lists.xenproject.org>
References: <20240205151413.1919983-1-andrew.cooper3@citrix.com>
 <6997ec6f-15a0-4c88-9a96-c7f4f450b2dd@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <6997ec6f-15a0-4c88-9a96-c7f4f450b2dd@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 17:02, Julien Grall wrote:
> On 05/02/2024 15:14, Andrew Cooper wrote:
>> -int bitmap_find_free_region(unsigned long *bitmap, int bits, int order)
>> +int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, unsigned int order)
>>   {
>>   	unsigned long mask;
>> -	int pages = 1 << order;
>> -	int i;
>> +	unsigned int pages = 1 << order;
>> +	unsigned int i;
> 
> ... I think your other patch is fixing a latent bug you introduced here. 
> Before hand, if bits was "negative", we would return -ENOMEM. Now if we 
> pass 2GB or higher, we would go through the loop.
> 
> So I would fold the hunk from common/bitmap.c here.
> 
>>   
>>   	if(pages > BITS_PER_LONG)
>>   		return -EINVAL;
> 
> [...]
> 
>> -int bitmap_allocate_region(unsigned long *bitmap, int pos, int order)
>> +int bitmap_allocate_region(unsigned long *bitmap, unsigned int pos,
>> +			   unsigned int order)
>>   {
>> -	int pages = 1 << order;
>> +	unsigned int pages = 1 << order;
>>   	unsigned long mask = (1ul << (pages - 1));
>> -	int index = pos/BITS_PER_LONG;
>> -	int offset = pos - (index * BITS_PER_LONG);
>> +	unsigned int index = pos/BITS_PER_LONG; 
> NIT: While you modify the line, can you add a space before after / as 
> you did above?

Instead of any of this - how about we finally purge this dead code? All
of bitmap_*_region() were dead in 3.2 (and perhaps even before), and they
are still dead.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 09:54:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 09:54:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676720.1052952 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXI9e-0000FQ-Gf; Tue, 06 Feb 2024 09:54:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676720.1052952; Tue, 06 Feb 2024 09:54:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXI9e-0000FJ-Dj; Tue, 06 Feb 2024 09:54:14 +0000
Received: by outflank-mailman (input) for mailman id 676720;
 Tue, 06 Feb 2024 09:54:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXI9d-0000FD-Jn
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 09:54:13 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ad46879d-c4d5-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 10:54:11 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-511413e52d4so3529044e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 01:54:11 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 o25-20020adfa119000000b0033b437f3216sm1675126wro.74.2024.02.06.01.54.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 01:54:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad46879d-c4d5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707213251; x=1707818051; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8UWJnYtEMGQtCa1T6Gl0yX15YIuBgiUtiUSuo4amgWM=;
        b=aeU4ap3d/X8u6dm88l4FERjBtwZ1GpUetlhTy/3ujw9rmzI5lg8MYahlcQWOsJQJdB
         /GdDH9d6OtiHNJ8Sso80bjNdBo8nwxuIaMvLYxuXCFvNbZLxp/a1ud5ggeQXC8t44gvY
         SvGEcu6rUBFfwdjQp/oT7pK0qN9b9XMMGj024ZCVXSuqLiYSvOi4esSwDuGF4i0wTc6P
         1U/V1AxhjIH2RgZYt3QLXfGVpaSZyXoMP2K/TsLOP70052K2z6VXVRrmARtiWxV0CsHQ
         LjZtQhJOt1F9dxWTFRIT7gO+7wdhBItvLzV1aVT38tiMCCzL8PtqW2y7yCEd6NbGtF1V
         peRg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707213251; x=1707818051;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8UWJnYtEMGQtCa1T6Gl0yX15YIuBgiUtiUSuo4amgWM=;
        b=seYmjhlCgsCt7sVD4Zamb3zpeYQ9lNgAJ3S7AgUSIVBFM42qCFD383ls59MhX5ckLN
         UMGD7+C8QuX4O/Il6zjK4PIm9O7ZTIMawPVRxyA3virPBhs09wIdcPPI4sAaAdLEqBx6
         jn4TPpKD3n4ha8MSNjxd7v8yRn0vnvE8wJWq4T7dBZq80bHD1dsF62VhasGsOMafdZ0I
         8siP+DlsfwQjyrmt/WNVttV4MP/5DUp0X53cpJSMHjokBJZr6PbPonpB9vWzPc9MrngZ
         wkiVeiQKa/4GI4dTf0CV7Wh5E68VcF/3YxSia3o62ouVG6StxTMLjTP/5HB+ngipVZ19
         QB2Q==
X-Gm-Message-State: AOJu0YyuVE+HAOamSDLzhU8U0LzlcAJCRWkaJfJeKjg1nR8zwohR8Sxl
	46zlY1+MY0FL+63xxzP01EEB1VVwVEfXClNlfAe5QLq3Cew9XzrXY7PMC9cMMQ==
X-Google-Smtp-Source: AGHT+IEweeEDvPP57kGdadllL3gb67m1LRTXuuTNp0wlPByZm2bYQc9oflIZa8A5tULSckFM4pGlBw==
X-Received: by 2002:a05:6512:33cc:b0:511:5724:dcd1 with SMTP id d12-20020a05651233cc00b005115724dcd1mr1878893lfg.11.1707213250969;
        Tue, 06 Feb 2024 01:54:10 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUhtjaWV1kZrLqpS98bkVIsJXM+al18LT1Jww3h6bDsRGey7+mGXvqM+1jEaODEtKZ7UpWX7/mBMl7k70+0TfryYcuM1DVswCo30zd/+2ouDoHjMKLpJaNc1vucdFmBNFe9Ov7Eba+Yd6ziIti4bva8YQQMV/5M5V7R0aOf7ei3Blzc
Message-ID: <90090302-ec5b-427e-8f64-aa57ce4aa19d@suse.com>
Date: Tue, 6 Feb 2024 10:54:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/bitmap: Compile with -Wsign-conversion
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240205151454.1920291-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240205151454.1920291-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:14, Andrew Cooper wrote:
> @@ -263,7 +266,7 @@ int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, unsigned i
>  	unsigned int pages = 1 << order;

If we mean to keep this function (see my reply to the other patch),
shouldn't we also check "order" along with

>  	unsigned int i;
>  
> -	if(pages > BITS_PER_LONG)
> +	if (pages > BITS_PER_LONG || bits >= INT_MAX)
>  		return -EINVAL;

... these checks?

> @@ -278,7 +281,7 @@ int bitmap_find_free_region(unsigned long *bitmap, unsigned int bits, unsigned i
>  		if((bitmap[index] & (mask << offset)) == 0) {
>  			/* set region in bimap */
>  			bitmap[index] |= (mask << offset);
> -			return i;
> +			return (int)i;

I agree with Julien that it looks like this change actually belongs in
the other patch.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 09:57:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 09:57:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676725.1052963 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXICQ-0000nk-TF; Tue, 06 Feb 2024 09:57:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676725.1052963; Tue, 06 Feb 2024 09:57:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXICQ-0000nd-QZ; Tue, 06 Feb 2024 09:57:06 +0000
Received: by outflank-mailman (input) for mailman id 676725;
 Tue, 06 Feb 2024 09:57:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXICP-0000mt-Os
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 09:57:05 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 14161715-c4d6-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 10:57:03 +0100 (CET)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-337d05b8942so4123185f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 01:57:04 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i9-20020adfe489000000b0033b45bdb2a1sm1414894wrm.4.2024.02.06.01.57.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 01:57:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 14161715-c4d6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707213423; x=1707818223; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=JuNrHofBsjzXnwInvms5nbhtrurzzvQIMsoyTqKEL3s=;
        b=LJgFrex/qMMVYFbmBgiBP11EYNHGqzamUQvYggzk5D9BEVz5BbLcsfulbnL3tnI6/v
         k3JhZObXynOGzjuUS2k/PH/MbH15bRWfC/GXaoepYOpvt2Y5P0Vhe8Qxv/Fn8Ojm1ZP/
         iPb0HBFfXrL8UZUZq006C/LpUyCNUFytqiC+1GuqRE5IjQJSgG7uIWr+WNssbhaa6+9t
         t+DSTRP7XqrKTh/1O3LVl87KXI2xJEGl0G4pvetuAgKkT8qLkriukocwyp9vZ2hFLNN2
         kLmLdOlTvtHfTu905PsarecG0fYtkjDFkC3BnK/pjpgJ3IMxdhzM7JRI8oo1OWqJGqda
         +zjg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707213423; x=1707818223;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JuNrHofBsjzXnwInvms5nbhtrurzzvQIMsoyTqKEL3s=;
        b=wlH1U8dY8BBCJVTx8ztSLVBkMQhbDiMMGh6ah6HRsDJtrpuL0+Fu2VHegL1gfA3R/G
         Mav00KWnyk5uEdq3tHQfor88Iyltom6FRdGVVGTmEYU58p1Ix94Xmd3gPjgnE8VXEBfW
         PlTixJsGJL+07DrwXlNgUtjLR7dc1XYRzyvqoEWUZkhxI9W/xDwdUQtm7GT6K8BRkC07
         50+wgs4feHcRvEJKX9OfZaK5FsCIyQwfsTJ/GuJyv+gy8EbJpZhNZlnwQcuCAzPbIwew
         RAl2Njsw+S++uJbIAqH27TYIfR7QDTy88HMQrjLLRp6X31jEFFl9BuQkFRrzrBFM50eU
         m2iw==
X-Gm-Message-State: AOJu0Ywj5DxNGd0I3VAHyTeMA9xA+8fSHLf42n+jR721e3I/oBuZyuj4
	knxA+iRKSIPS3sY2aZZHZAuavdm/KWST6LFuCmdhM34FN9wpjy+oqBjEf2rCOA==
X-Google-Smtp-Source: AGHT+IFGFOknQLwiTJwHI3pjFWjOgNX5xBqOSSWFBhusWxv21BDKMed9ETxdTRNxoQaspFkCJX6S/w==
X-Received: by 2002:adf:cd08:0:b0:33b:2fac:6a44 with SMTP id w8-20020adfcd08000000b0033b2fac6a44mr941929wrm.34.1707213423579;
        Tue, 06 Feb 2024 01:57:03 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXYNn3TtvTz+MhHHut/Loqb38D7QNw/gyUp0W8DqiFe8VRot61YSm2LMKclJoLQ2Ugw9WFFu4we3Ukzm1m95EMesz6E6kHFNb/3nSYykBlg/IF65gaLuJGz5R7FJmoZ6hztlgf3cLFJfyjFyOoMVftc8Kl321UP+Q08zJAqayEMBdRVIBG/eQTasjYwK4W6MfoNrR2I4tZXPY1f88bLnmnzpGpGzN15RZ7TOrHaD4urcq81XbK9IsY=
Message-ID: <1179f656-5205-4d62-80fc-3338f3e9bedb@suse.com>
Date: Tue, 6 Feb 2024 10:57:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/5] Constify some parameters
Content-Language: en-US
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Dario Faggioli
 <dfaggioli@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>
References: <20240205104504.14830-1-frediano.ziglio@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240205104504.14830-1-frediano.ziglio@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 11:45, Frediano Ziglio wrote:
> Make clean they are not changed in the functions.

s/clean/clear/ ?

> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Tue Feb 06 10:00:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 10:00:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676735.1052989 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIG3-00030U-J5; Tue, 06 Feb 2024 10:00:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676735.1052989; Tue, 06 Feb 2024 10:00:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIG3-00030N-Fd; Tue, 06 Feb 2024 10:00:51 +0000
Received: by outflank-mailman (input) for mailman id 676735;
 Tue, 06 Feb 2024 10:00:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXIG2-00030H-4v
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 10:00:50 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9957abba-c4d6-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 11:00:47 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33929364bdaso3322045f8f.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 02:00:47 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i6-20020a5d5586000000b0033b376b9219sm1669211wrv.92.2024.02.06.02.00.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 02:00:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9957abba-c4d6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707213647; x=1707818447; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=CnoR/Y6jLIkW24zcJAjTd/HCAh644Diy4laGIgxK7Yo=;
        b=UnS9m+e4jNuwxpi6zGrfvFwERPrcdf5wHKAAOGDazc5sfl4GfXpoAqIULmilKOLudL
         TB1D6VqXdEPVprijdEQxUBgiPYFy6tHWbxgeMalRCwPZdNwA2i+ifevL5zqDbMQ/JufN
         OHnTuw/h7GI5z9VZFX5gzfZkW8I/PZa/Ha+KrF4rtcL2ZQKw1AEuFCTaODQLeCOYvwsx
         FtajmdbIMNf7SQHKDRYBaMJKSbiysCWXwupNJKkWIKjHKGj+S9pHEfIgRiP8PDtibKDC
         SUnjA3U68s4fxBJ1HuTYGORSC1nxJb8WcenIJDHOySbRjVWK4JPV1WR4+1Gx8Ss9F96z
         RG8g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707213647; x=1707818447;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=CnoR/Y6jLIkW24zcJAjTd/HCAh644Diy4laGIgxK7Yo=;
        b=VdR6TCJY3Tgd7PRgC0PGxVobmdFTQsgNrty7mI0B8ZVLr6qfCtJYZYflnANOyfQVB4
         11AQ8nd3+5ZHAp2E0QomYR9n2ETFLbbeb2d0ZGbt2wWd2YjOuBx7TEO4KCUzEa6kKKmF
         4UVlo8HaLgS8VojMaXVXUbxarL5uI2khzahzr9xsrNAtR8HhF8Ga+TJYe0dgOWuOZZw8
         PDtS/hfcXg3aW7FCTrkP8L/M95oZ7mom24Z8BtBu6JMxFTOK6OV7q81YqFdfkpRirXmn
         TpU5JgnuxKba6TJBnH3qUAP0RSmdrPRgg2LzLweM9nB6PV+G1EgGSM/T8djchTXgfTd+
         0q+Q==
X-Gm-Message-State: AOJu0YzRP7a/vGLO8bk/9lVfsjoPvfAZ/3+fvlpUrr2jTBGqiRTTjRfd
	RUt/bAYV/692lLpdUGq4dNbGlLnT3G8snGHhHOWjzi1FU1g8OiEK8xVLBTaVFg==
X-Google-Smtp-Source: AGHT+IGAyaTsV3w+n8j+ZBTN5O3TUkWWZfuulzEZLCMVXUydfbhEMfxn/FI7QmKYjZib8jr7K781ZA==
X-Received: by 2002:a5d:47a2:0:b0:33b:4967:4d2 with SMTP id 2-20020a5d47a2000000b0033b496704d2mr621910wrb.41.1707213647103;
        Tue, 06 Feb 2024 02:00:47 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVqOC7/imqtbA88h+nNBQu287CLUd2YtHrQmSZipNSpe1dIi7MGfoeIv1Csk5wEV2pCKi9auKzXIHZQF9b8LEMvzdAY5Nz7QsInGCGSRLaFdoK00y4Oy/kiWwsjbHC7rYQH6D3zalY6UXF/06ahdenzw6yztklnR3Tq5NuQIlzi71sqvQ11
Message-ID: <a7c27052-1feb-4536-9fcf-716f4ae5114f@suse.com>
Date: Tue, 6 Feb 2024 11:00:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/5] Use more specific constant
Content-Language: en-US
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu <wl@xen.org>
References: <20240205104740.14905-1-frediano.ziglio@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240205104740.14905-1-frediano.ziglio@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 11:47, Frediano Ziglio wrote:
> __HYPERVISOR_arch_1 and __HYPERVISOR_paging_domctl_cont for x86
> have the same value but this function is handling
> "paging_domctl_cont" hypercall so using the latter mnemonic in
> the code is more clear.
> 
> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

I can't reconstruct why it was done like this. arch_do_domctl()
uses the "more specific" constant in any case.

Jan



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 10:02:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 10:02:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676738.1052998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIHE-0003Y6-RW; Tue, 06 Feb 2024 10:02:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676738.1052998; Tue, 06 Feb 2024 10:02:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIHE-0003Xz-Oa; Tue, 06 Feb 2024 10:02:04 +0000
Received: by outflank-mailman (input) for mailman id 676738;
 Tue, 06 Feb 2024 10:02:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXIHD-0003Xn-Us; Tue, 06 Feb 2024 10:02:03 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXIHD-00083M-SZ; Tue, 06 Feb 2024 10:02:03 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXIHD-0006j7-DR; Tue, 06 Feb 2024 10:02:03 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXIHD-0000cq-Cu; Tue, 06 Feb 2024 10:02:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dYrOErGeBi5jV4w1GMLlSyu8aD4R/4vCH+8MnZBz8ag=; b=ZyZcDaFEb5Te/7kGBM8UDaLlSt
	Xi26ZMwTczjBYM3/SvqmblP5Wemhw23Ktsqmt0ODlcL7MHriFJhbQ/JeKSUALOPYID9Lh6g2AT/jy
	obShOxE6HkVbChNQylL5m6iDj0wZv8DegkcI0/q4n/cNZOboP7geUmXNkwpTmckScV8o=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184597-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184597: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-i386-qemut-rhel6hvm-amd:guest-start/redhat.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-qemuu-freebsd12-amd64:guest-start/freebsd.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=29f17d837421f13c0e0010802de1b2d51d2ded4a
X-Osstest-Versions-That:
    xen=660f8a75013c947fbe5358a640032a1f9f1eece5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 06 Feb 2024 10:02:03 +0000

flight 184597 xen-unstable real [real]
flight 184602 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184597/
http://logs.test-lab.xenproject.org/osstest/logs/184602/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-qemut-rhel6hvm-amd 14 guest-start/redhat.repeat fail pass in 184602-retest
 test-amd64-amd64-qemuu-freebsd12-amd64 21 guest-start/freebsd.repeat fail pass in 184602-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184591
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184591
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184591
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184591
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184591
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184591
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184591
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184591
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184591
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184591
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184591
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184591
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  29f17d837421f13c0e0010802de1b2d51d2ded4a
baseline version:
 xen                  660f8a75013c947fbe5358a640032a1f9f1eece5

Last test of basis   184591  2024-02-05 12:38:54 Z    0 days
Testing same since   184597  2024-02-05 22:09:17 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Cyril Rébert (zithro) <slack@rabbit.lu>
  Cyril Rébert <slack@rabbit.lu>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   660f8a7501..29f17d8374  29f17d837421f13c0e0010802de1b2d51d2ded4a -> master


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 10:11:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 10:11:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676754.1053008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIQ5-0005lr-UK; Tue, 06 Feb 2024 10:11:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676754.1053008; Tue, 06 Feb 2024 10:11:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIQ5-0005lk-RK; Tue, 06 Feb 2024 10:11:13 +0000
Received: by outflank-mailman (input) for mailman id 676754;
 Tue, 06 Feb 2024 10:11:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXIQ4-0005le-6f
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 10:11:12 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0c4bbfe8-c4d8-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 11:11:09 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-40ff0799f56so347855e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 02:11:10 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w9-20020a05600c474900b0040e9f7308f4sm1496867wmo.10.2024.02.06.02.11.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 02:11:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c4bbfe8-c4d8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707214269; x=1707819069; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=g+MyVLWTRba1VFaavpxn1vfOdY+KSP36/07oNymEwEQ=;
        b=a5ZPKeDETkkCDOhNx6/B1JOTw/W5bwidDIqUuvaEWMlr0GhQhQtvzniuIesg0OlErn
         41StvnqK5RJOkcipAYp99kvRLRr9QWXHHAADI9+lF2AIXRcxy/TLm7LlEZSAxvqpx1zw
         6ClXXh3JD8y0CjZVMhzMxBjC6zQuYiLVHu04oAbZaWpmT2WcSUMHZH1CC15jSKNgyCPs
         Mah295BgHhxTe/k3pPUQujLPyTjboCeSvAaOeKXytPIgvyosbc8hy/E7w7EIsjYd3G33
         rwbD5Yi8BD0O02pXQllk288OVdSiflRZYWS5L0U8C7UOpk0WXjoeoH1ZpM6kb+kV1sIN
         7HIw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707214269; x=1707819069;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=g+MyVLWTRba1VFaavpxn1vfOdY+KSP36/07oNymEwEQ=;
        b=QyJa0qiI98AFuakvSZLw4/JH7Ulzjg/VR7ggTaCrJgcR8ou4R334E5MTCWF586YHGG
         55252eK/stfa4Au/5GuTb0OGObDmdrqsYpY5pGCPty5a9mB/0bvsBnlk48G6YYQQRyjq
         bPgLsyxJiSxlaw/uBXejzFHMf2cECH68euvxmQ7SHTmWpnhpFQ76qt0hQM0x5ASIVKsi
         h3Xf3RJPDalelFVkfDz0aMHEbivA6M+4pC71iqbWtfg8r1KGmLXJXsgQUsEog6HLx1wW
         LQiE+0hpyzbk9CpHPWziSQtx2SAdlvBEAmcrfg2fd2b8m/cOkmtTzIjuL/zb2nZgrWue
         rq5g==
X-Gm-Message-State: AOJu0YyhHme1byUShxmWYEy3j2NJDU+9dSEcmiVFck06hEH63f1YEJ+2
	JOfwvcOS+Ehe6tLFGJcMAjfY8MwO6WriVGtmi1XTOdX2RlyMoHgdCFUX+tloXg==
X-Google-Smtp-Source: AGHT+IGfpk+VAtBQ421FXKv9E0220UHtOWxMasmTrSLIcrpUc7RwmaLNeBmf0Wve7/TvZwtB9coEOQ==
X-Received: by 2002:a05:600c:45c9:b0:40d:92c8:2cf5 with SMTP id s9-20020a05600c45c900b0040d92c82cf5mr1484628wmo.20.1707214269507;
        Tue, 06 Feb 2024 02:11:09 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCW7Woc/GDtOnaaHCHGWuWQWfEYWALM3k3qvqMrnW+sngpHs/wpiFLX/rc6y7n909FQ16vrjFk8VUIAGTfRB+uJIV4LdZphMR/qpORusJtD+bLg7Sy6F4IDTyOaBdchy+KBvCDBAEed8OqgazA==
Message-ID: <1c07f2f0-9007-48e3-9914-a283d6497009@suse.com>
Date: Tue, 6 Feb 2024 11:11:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/5] Reduce assembly code size of exception entry points
Content-Language: en-US
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>
References: <20240205104840.14942-1-frediano.ziglio@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240205104840.14942-1-frediano.ziglio@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 11:48, Frediano Ziglio wrote:
> We just pushed a 8-bytes zero and exception constants are
> small so we can just write a single byte saving 3 bytes for
> instruction.
> With ENDBR64 this reduces the size of the entry point from 32 to 16
> bytes (due to alignment).

Oh, good - when recently touching these entry points, I was thinking
of doing exactly this. Just didn't get to it, yet. However
- since using different size stores can raise performance concerns,
  I'd like to suggest that you mention the fact that we actually
  already have cases of this, in autogen_stubs,
- I'd further like to ask that this conversion then be done consistently
  everywhere, perhaps even including the storing of TRAP_syscall. I find
  it particularly puzzling that you ...

> --- a/xen/arch/x86/x86_64/entry.S
> +++ b/xen/arch/x86/x86_64/entry.S
> @@ -898,28 +898,28 @@ END(handle_exception)
>  FUNC(entry_DE)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_DE, 4(%rsp)
> +        movb  $X86_EXC_DE, 4(%rsp)
>          jmp   handle_exception
>  END(entry_DE)
>  
>  FUNC(entry_MF)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_MF, 4(%rsp)
> +        movb  $X86_EXC_MF, 4(%rsp)
>          jmp   handle_exception
>  END(entry_MF)
>  
>  FUNC(entry_XM)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_XM, 4(%rsp)
> +        movb  $X86_EXC_XM, 4(%rsp)
>          jmp   handle_exception
>  END(entry_XM)
>  
>  FUNC(entry_NM)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_NM, 4(%rsp)
> +        movb  $X86_EXC_NM, 4(%rsp)
>          jmp   handle_exception
>  END(entry_NM)
>  
> @@ -933,28 +933,28 @@ END(entry_DB)

... skip entry_DB here. And even ...

>  FUNC(entry_BP)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_BP, 4(%rsp)
> +        movb  $X86_EXC_BP, 4(%rsp)
>          jmp   handle_exception
>  END(entry_BP)
>  
>  FUNC(entry_OF)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_OF, 4(%rsp)
> +        movb  $X86_EXC_OF, 4(%rsp)
>          jmp   handle_exception
>  END(entry_OF)
>  
>  FUNC(entry_BR)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_BR, 4(%rsp)
> +        movb  $X86_EXC_BR, 4(%rsp)
>          jmp   handle_exception
>  END(entry_BR)
>  
>  FUNC(entry_UD)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_UD, 4(%rsp)
> +        movb  $X86_EXC_UD, 4(%rsp)
>          jmp   handle_exception
>  END(entry_UD)

... entry points below here, where an error code is pushed, will
have zero in the respective field, so these can be converted too.
And entry_NMI isn't any different either. Whereas entry_MC even
exactly matches the pattern you have been following, afaict.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 10:13:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 10:13:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676758.1053018 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIRy-0006ID-9e; Tue, 06 Feb 2024 10:13:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676758.1053018; Tue, 06 Feb 2024 10:13:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIRy-0006I6-75; Tue, 06 Feb 2024 10:13:10 +0000
Received: by outflank-mailman (input) for mailman id 676758;
 Tue, 06 Feb 2024 10:13:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXIRx-0006I0-Ba
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 10:13:09 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 523c2879-c4d8-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 11:13:06 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-40ff13d2628so259235e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 02:13:07 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v15-20020a05600c444f00b0040fdb17e66csm1519236wmn.4.2024.02.06.02.13.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 02:13:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 523c2879-c4d8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707214387; x=1707819187; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lFQ2fZ7ptp0gQ3lQ8/JqAFd2cGFa29Kt/kIlBhW3Oyk=;
        b=CzmiVk9dIVepnAS0DcgQZ76E9JPs7XyvXesmQq5UksOvbJlFcXf7R2l025JjW9sR5i
         x8XDn4tN5ir7Mg+Z8G+fZjAk1PBcU8/taM0UQ9cpHfb7gPbaxxxxvnXkYeh/UjvHCzSI
         1uobyiu7CrBkUWEX29n7jdt4wvYo/csEa4y/EBHO9BIWulwct+5aF+y11/VltFZpjnzW
         KNflHyIDnUFKdOa7Z02sdYqncMSwHQJSsAZo5XV2NOd+7uD0DIZBAPp1hKevehBzrvJK
         ULNMlVc2h/KT0Q9eTpknE8vCYPTToIC5u4QCZlD5orc4M7CSVX938EV/xx9kHdgXx9Uj
         4eYg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707214387; x=1707819187;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=lFQ2fZ7ptp0gQ3lQ8/JqAFd2cGFa29Kt/kIlBhW3Oyk=;
        b=WMK+47efdQMbIQouqUCoK8OLJD9pQP1zAU3dtNXngxVDBpck4Cxp/izCl9KukCBZFt
         8SSbG0p6foe4uPfKoV3s8s3Rqq4Fn7glKac6AKxmAuFH7k+39sadT9kJN/2IJhB2JwzX
         O0hkjvtddA8ud5btJK8a7aN15FHCltS3POqevI1YUbzo73ND+Z1UwpnnhSI/AURhVC2N
         qy0S7DIGq/clIHoddUUwl4lJZzV/zOlyDyqtXUxfZLRM/ShWTEPa899/OBT0MwhN5M2W
         Fb7b8lHTjc2n/KovS0qfpJA8aYfIppgnQk7DYcmy7cWyDHPuzTEIeDA/yQNBIhZY9Hy5
         5RnQ==
X-Gm-Message-State: AOJu0YzZjggOhFBhWtsovaEgEfj7qSfATTJbtHXkJwJrb3Dz/nSyOayx
	Wi17cIUN9Pw5WkhgdMQKPWWzrem65H7ldjUdDOauKPYf9qC5tGvLl0jXcVfU2w==
X-Google-Smtp-Source: AGHT+IGFA4d1LwA5BvGuFvtIE9j9bcX0iLYJG/Yc3Vsxb0jTk5VQA/Yp8s3rGvtuU1SwD9wI2dvdFA==
X-Received: by 2002:a05:600c:4f42:b0:40f:e40:b53d with SMTP id m2-20020a05600c4f4200b0040f0e40b53dmr1241079wmq.1.1707214386845;
        Tue, 06 Feb 2024 02:13:06 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXTk3k9DuB4qzAjWmHHU24kUl5q5jXaK/snrN9CwD4bLpe2fBSllYQAkN0zCHsCRnIYA8yk3VpnOCLKHOIAVeJSuM0e2WAjVpEOAdCvDaTxCjvbuXnFjsP4N4aK9vkZaMy0addCHs/mc7NeZQ==
Message-ID: <2a30ce41-7802-4e22-bccb-101ec5b0cace@suse.com>
Date: Tue, 6 Feb 2024 11:13:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/5] Reduce assembly instruction size
Content-Language: en-US
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>
References: <20240205104919.14983-1-frediano.ziglio@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240205104919.14983-1-frediano.ziglio@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 11:49, Frediano Ziglio wrote:
> Use 32 bit versions in all cases, not only for registers till 8th.
> This reduces the encoding from (example with r14):
> 
>      49 c7 c6 ff 7f 00 00    mov    $0x7fff,%r14
> 
> to
> 
>      41 be ff 7f 00 00       mov    $0x7fff,%r14d
> 
> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 10:17:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 10:17:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676765.1053029 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIVa-0006su-QL; Tue, 06 Feb 2024 10:16:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676765.1053029; Tue, 06 Feb 2024 10:16:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIVa-0006sn-Mw; Tue, 06 Feb 2024 10:16:54 +0000
Received: by outflank-mailman (input) for mailman id 676765;
 Tue, 06 Feb 2024 10:16:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXIVZ-0006sh-Mm
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 10:16:53 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d8bed5c5-c4d8-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 11:16:52 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-33b1b21c020so3156595f8f.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 02:16:52 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f17-20020adfdb51000000b00337d6f0013esm1691714wrj.107.2024.02.06.02.16.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 02:16:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8bed5c5-c4d8-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707214612; x=1707819412; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=/fdb57JZvf0dvGSH2X/1n7eUey51PwWD70FB2aYbEw0=;
        b=ExfoqCD0nkJX5ktDXNeDFv1FV2Ioi5FOL5TmC6DxFVDGrzYIm+1y89A6MaTqTXy/iS
         LGYvdbB0FbMdW0PN0LZKOKGrE43UlumyvREvFaguBU4s83ShWtYZ/e+CPl+AX6n8SXdT
         DSOIkjFcYpxlcZtZzuo81Jtb2/g2TrL+2ALkCxzOrBEKmkKLjNI//G437FOxSLWd8z75
         J3b5Hc+6kfrzelDv2hLwWCEF1YLcZ/2OevyzCQcTthBaWcufJvp7vt5chMaLZudpAGYi
         mJJaiGMWrVy057KyHhUjzQd10uhOBI1sZlFTqBlw700E85TFWLRSW6U1LZI+FKyv1flD
         rqVA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707214612; x=1707819412;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/fdb57JZvf0dvGSH2X/1n7eUey51PwWD70FB2aYbEw0=;
        b=arZlGyre2HbfIOZfbq9n0aNqd2xJDpTNnCnSZFlt8MmJmJATbLT4r2aZKabHfguBsm
         D0BJKd4Y2vNRldZd6KHb1dRn/A3r/qnnzeoJrhHSRK65kQD98zmCEhSgdQNgaEFop8Xi
         r9mmxAPb3GThDk4qmFTUJUdneM++MFiF7NQhVqcwetm2kyeKT4jbmzVd/gZhT1HbbHNL
         j0XymzLkHrsaX77WfxcNVUSfzdzka9Ejw0MGlPb+CcbJQgTHD6d2JcupjCCFbfMKIwsk
         Vw7Ogx+9CJ4PABf83yYK9RsYYEltmp2tBsEVs/4u24qwR0ZFjNnJJzkfql8+jLfx45YA
         8srg==
X-Gm-Message-State: AOJu0YwA1EAittp2zgC/+QIL+A3IjTohDmJ30fvblbWKIg80BVZ3Ncyu
	QygIYgbX4r53H5gmwgmEErbOGP0IFGs5dxkEu3l/gQTpAdsrLMnzI0UXNfC1+8nD/PutaBwzauE
	=
X-Google-Smtp-Source: AGHT+IHONHAsNChAsu7enF6j9CqMMhBRNLh70v3O/Foqo0VjZHYjV4JHpTlH1ObDCMbPmvaZAbV1Sg==
X-Received: by 2002:a05:6000:1372:b0:33b:44d0:63ea with SMTP id q18-20020a056000137200b0033b44d063eamr916777wrz.59.1707214611941;
        Tue, 06 Feb 2024 02:16:51 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUA6+G5HgP+JHQaDNGBSemww5tbmw1Bs5CtrGI6SfKPSlim/RibMGZcp4bHF9sCPMiG7qlefsD4MTIOm4dLh6HIKvLP+N3bbLzSRwBeh7QDPZpFOKHyE3jLp2m/4Qb/
Message-ID: <e531ab31-6137-4fbb-9b05-b972ca102c4b@suse.com>
Date: Tue, 6 Feb 2024 11:16:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/4] amd-vi: fix IVMD memory type checks
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, oxjo@proton.me,
 xen-devel@lists.xenproject.org
References: <20240201170159.66330-1-roger.pau@citrix.com>
 <20240201170159.66330-2-roger.pau@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240201170159.66330-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 01.02.2024 18:01, Roger Pau Monne wrote:
> The current code that parses the IVMD blocks is relaxed with regard to the
> restriction that such unity regions should always fall into memory ranges
> marked as reserved in the memory map.
> 
> However the type checks for the IVMD addresses are inverted, and as a result
> IVMD ranges falling into RAM areas are accepted.  Note that having such ranges
> in the first place is a firmware bug, as IVMD should always fall into reserved
> ranges.
> 
> Fixes: ed6c77ebf0c1 ('AMD/IOMMU: check / convert IVMD ranges for being / to be reserved')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Tue Feb 06 10:22:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 10:22:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676752.1053046 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIbD-0000cP-Nm; Tue, 06 Feb 2024 10:22:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676752.1053046; Tue, 06 Feb 2024 10:22:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIbD-0000bv-HJ; Tue, 06 Feb 2024 10:22:43 +0000
Received: by outflank-mailman (input) for mailman id 676752;
 Tue, 06 Feb 2024 10:09:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HdcV=JP=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1rXIO6-0004eO-L4
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 10:09:10 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c3ae577f-c4d7-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 11:09:07 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-5101cd91017so7129273e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 02:09:08 -0800 (PST)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 n29-20020a05600c3b9d00b0040ef2e7041esm1512947wms.6.2024.02.06.02.09.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 06 Feb 2024 02:09:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3ae577f-c4d7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707214147; x=1707818947; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=jPnnm05qAiSkgDvvHl4Q0+hRe1/bOWUsax4BKvjLB0s=;
        b=BxMO9jVrq315VdEvQ/tKd+ZWeI88EY5F/vegssX+sjUFi8uv9wbb9mZN70NHnLnidz
         eRItLHRo1NSVR4/XNxm4J+6R7WJsx77KA576p1dPqYnmfI+6nBrvzeF50hGvE7exWIcA
         pyj9SL2K4OsEf9qj/Qx07ZWmytPGiItmoduMN83sUKI3Bn7LhrphFC0vPAnXakJDYiq1
         MhLq8qoaVYa9hBE4O1tjsW7kPvnwO8V3/dHVp0dHv4HY33yA2VBSMINxo+iMdZbNJCuR
         dTsBu6BhKoBgsqguUWQuFmRxtJbDuzFKwC/jJrLJr0BT6VvZPrYuSfUNU0FQcEcYb29m
         JSLA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707214147; x=1707818947;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=jPnnm05qAiSkgDvvHl4Q0+hRe1/bOWUsax4BKvjLB0s=;
        b=SIoavMjYGCNs+mTCa/aYuNwqZgI+qmSETlz7QlEAWJLuw+nQ78u3Cr3R3FF6eg+ds8
         mT7yYcV3HUx8+bJskZ4HAYBgrBkruWwy0clsZEVf/6o3ldnNmJTC9wIaxY4lB6OKOZol
         S7VEiHkS+5vAe3GNYR0UDxaajY48tF4SddrcUsHcQOzN3JF8LU4uJDTJgJpHCmJa78O6
         yTjbeCWw6QhqkBj8WN9shEsPcGGN1J/ZOr10Tn12dawqncsqExnPcqkn4blHBOjQjfEn
         ySn4NV+nkpxOLdi3ioOrZOCSeu2rDgATppaP5pE5Z90HTJBYiNQvjnkPQSzGVqBpr/LV
         Ryog==
X-Gm-Message-State: AOJu0Yxgqkf7VpSL7RlNFTqjQr83lRou5BcJVpj8rif8lH0ZhYx2XTCn
	yDz3fsNeOaE0WdRN9iLdGrSBJafHvZsyf/MI0LtemFOj9YmcbnnDsGZ6d56s0xY=
X-Google-Smtp-Source: AGHT+IE2sPU/iuvgTVBZei/G/7BCTdd+ElefS5WgRHtbX/6p8Qcv0r9DPbfZZQFRwbiKXcSNkmSw6A==
X-Received: by 2002:ac2:5208:0:b0:511:4f67:7d47 with SMTP id a8-20020ac25208000000b005114f677d47mr1263615lfl.59.1707214146915;
        Tue, 06 Feb 2024 02:09:06 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVn8uwEicXsvw5E6fCK400wiDcCTVuwu6qFZsR7d2YlAyspCZB9sLFLhPt9cyE9WjMQnuj4Cvwk0MDNjOC0iqhsgUv2zKUx/9V1EShBTYLyZ25lnqnDGAPEibOnlPujFWCgos+LeIslSEVG/U582wXee2ESHgUL9BfWPYZBKTCnDjlLnlHDVuwVexHJw9kjYEvd3G9/D2JnuTxZHpdylx3jyfwv2PA4s0EPp8NNmcKa8Wl5CvjqVTqGrQsqU71ax9taXoBrKrXDVg2+MsJnbJpdT5AxGMa8J0uhaeXAO5RZU/Rf0eAj0BIe3Oxklq2rL29y3HVV31b7p46+I0sOTk2efFPFn62ipyJfqzKmBW7wjijQ7slaW4hilP2xXS3O/IWsOuH8I2X0C1RM3Q==
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH] x86: Add configuration options for max_altp2m and max_nestedp2m limits
Date: Tue,  6 Feb 2024 10:08:59 +0000
Message-Id: <9143475dddf8d0141fb3864ed90153591554a425.1707213019.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

This commit introduces the ability to configure the maximum number of altp2m
and nestedp2m tables through boot-time parameters.  Previously, the limits were
hardcoded to a maximum of 10 for both.  This change allows for greater
flexibility in environments that require more or fewer tables, enhancing Xen's
adaptability to different workloads and scenarios.

Adjustments include:
- Adding boot parameters `max_altp2m` and `max_nestedp2m` to allow setting
  these limits at boot time.
- Modifying various parts of the code to use these configurable limits instead
  of the previous hardcoded values.
- Ensuring that if the configured values exceed the maximum supported EPTP
  entries, they are adjusted down with a warning logged.
- Adjusting the initial allocation of pages in `hap_enable` from 256 to 2048
  pages when `old_pages == 0`.

  This change anticipates scenarios where `max_altp2m` and `max_nestedp2m`
  are set to their maximum supported values (i.e., 512), ensuring sufficient
  memory is allocated upfront to accommodate all altp2m and nestedp2m tables
  without initialization failure.

This enhancement is particularly relevant for users leveraging Xen's features
for virtual machine introspection or nested virtualization support.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 .../paging-mempool/test-paging-mempool.c      |  2 +-
 xen/arch/x86/hvm/hvm.c                        | 22 +++++++-
 xen/arch/x86/hvm/vmx/vmx.c                    |  2 +-
 xen/arch/x86/include/asm/domain.h             |  8 +--
 xen/arch/x86/include/asm/p2m.h                |  4 +-
 xen/arch/x86/mm/altp2m.c                      | 16 +++++-
 xen/arch/x86/mm/hap/hap.c                     | 14 ++---
 xen/arch/x86/mm/mem_access.c                  | 14 ++---
 xen/arch/x86/mm/mem_sharing.c                 |  6 +-
 xen/arch/x86/mm/nested.c                      | 16 +++++-
 xen/arch/x86/mm/p2m-ept.c                     |  6 +-
 xen/arch/x86/mm/p2m.c                         | 56 +++++++++----------
 12 files changed, 105 insertions(+), 61 deletions(-)

diff --git a/tools/tests/paging-mempool/test-paging-mempool.c b/tools/tests/paging-mempool/test-paging-mempool.c
index 1ebc13455a..b94d4a4fe1 100644
--- a/tools/tests/paging-mempool/test-paging-mempool.c
+++ b/tools/tests/paging-mempool/test-paging-mempool.c
@@ -35,7 +35,7 @@ static struct xen_domctl_createdomain create = {
 
 static uint64_t default_mempool_size_bytes =
 #if defined(__x86_64__) || defined(__i386__)
-    256 << 12; /* Only x86 HAP for now.  x86 Shadow needs more work. */
+    2048 << 12; /* Only x86 HAP for now.  x86 Shadow needs more work. */
 #elif defined (__arm__) || defined(__aarch64__)
     16 << 12;
 #endif
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 482eebbabf..cb5e190083 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -105,6 +105,12 @@ static const char __initconst warning_hvm_fep[] =
 static bool_t __initdata opt_altp2m_enabled = 0;
 boolean_param("altp2m", opt_altp2m_enabled);
 
+unsigned long __read_mostly max_altp2m = 10;
+integer_param("max_altp2m", max_altp2m);
+
+unsigned long __read_mostly max_nestedp2m = 10;
+integer_param("max_nestedp2m", max_nestedp2m);
+
 static int cf_check cpu_callback(
     struct notifier_block *nfb, unsigned long action, void *hcpu)
 {
@@ -191,6 +197,20 @@ static int __init cf_check hvm_enable(void)
 
     if ( !opt_altp2m_enabled )
         hvm_funcs.altp2m_supported = 0;
+    else
+    {
+        if ( max_altp2m > MAX_EPTP )
+        {
+            printk(XENLOG_WARNING "HVM: max_altp2m must be <= %lu, adjusting\n", MAX_EPTP);
+            max_altp2m = MAX_EPTP;
+        }
+
+        if ( max_nestedp2m > MAX_EPTP )
+        {
+            printk(XENLOG_WARNING "HVM: max_nestedp2m must be <= %lu, adjusting\n", MAX_EPTP);
+            max_nestedp2m = MAX_EPTP;
+        }
+    }
 
     if ( opt_hvm_fep )
         warning_add(warning_hvm_fep);
@@ -5207,7 +5227,7 @@ void hvm_fast_singlestep(struct vcpu *v, uint16_t p2midx)
     if ( !hvm_is_singlestep_supported() )
         return;
 
-    if ( p2midx >= MAX_ALTP2M )
+    if ( p2midx >= max_altp2m )
         return;
 
     v->arch.hvm.single_step = true;
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 1edc7f1e91..71a9269f1d 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -4857,7 +4857,7 @@ bool vmx_vmenter_helper(const struct cpu_user_regs *regs)
         {
             unsigned int i;
 
-            for ( i = 0; i < MAX_ALTP2M; ++i )
+            for ( i = 0; i < max_altp2m; ++i )
             {
                 if ( currd->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
                     continue;
diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h
index 619e667938..429f0d6668 100644
--- a/xen/arch/x86/include/asm/domain.h
+++ b/xen/arch/x86/include/asm/domain.h
@@ -259,9 +259,9 @@ struct paging_vcpu {
     struct shadow_vcpu shadow;
 };
 
-#define MAX_NESTEDP2M 10
+extern unsigned long max_altp2m;
+extern unsigned long max_nestedp2m;
 
-#define MAX_ALTP2M      10 /* arbitrary */
 #define INVALID_ALTP2M  0xffff
 #define MAX_EPTP        (PAGE_SIZE / sizeof(uint64_t))
 struct p2m_domain;
@@ -349,12 +349,12 @@ struct arch_domain
 
 #ifdef CONFIG_HVM
     /* nestedhvm: translate l2 guest physical to host physical */
-    struct p2m_domain *nested_p2m[MAX_NESTEDP2M];
+    struct p2m_domain **nested_p2m;
     mm_lock_t nested_p2m_lock;
 
     /* altp2m: allow multiple copies of host p2m */
     bool_t altp2m_active;
-    struct p2m_domain *altp2m_p2m[MAX_ALTP2M];
+    struct p2m_domain **altp2m_p2m;
     mm_lock_t altp2m_list_lock;
     uint64_t *altp2m_eptp;
     uint64_t *altp2m_visible_eptp;
diff --git a/xen/arch/x86/include/asm/p2m.h b/xen/arch/x86/include/asm/p2m.h
index 40545f5fa8..0810293f7f 100644
--- a/xen/arch/x86/include/asm/p2m.h
+++ b/xen/arch/x86/include/asm/p2m.h
@@ -862,7 +862,7 @@ static inline struct p2m_domain *p2m_get_altp2m(struct vcpu *v)
     if ( index == INVALID_ALTP2M )
         return NULL;
 
-    BUG_ON(index >= MAX_ALTP2M);
+    BUG_ON(index >= max_altp2m);
 
     return v->domain->arch.altp2m_p2m[index];
 }
@@ -872,7 +872,7 @@ static inline bool p2m_set_altp2m(struct vcpu *v, unsigned int idx)
 {
     struct p2m_domain *orig;
 
-    BUG_ON(idx >= MAX_ALTP2M);
+    BUG_ON(idx >= max_altp2m);
 
     if ( idx == vcpu_altp2m(v).p2midx )
         return false;
diff --git a/xen/arch/x86/mm/altp2m.c b/xen/arch/x86/mm/altp2m.c
index a04297b646..468ccb0b58 100644
--- a/xen/arch/x86/mm/altp2m.c
+++ b/xen/arch/x86/mm/altp2m.c
@@ -120,7 +120,13 @@ int p2m_init_altp2m(struct domain *d)
     struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
 
     mm_lock_init(&d->arch.altp2m_list_lock);
-    for ( i = 0; i < MAX_ALTP2M; i++ )
+
+    if ( (d->arch.altp2m_p2m = xzalloc_array(struct p2m_domain *, max_altp2m)) == NULL )
+    {
+        return -ENOMEM;
+    }
+
+    for ( i = 0; i < max_altp2m; i++ )
     {
         d->arch.altp2m_p2m[i] = p2m = p2m_init_one(d);
         if ( p2m == NULL )
@@ -141,7 +147,10 @@ void p2m_teardown_altp2m(struct domain *d)
     unsigned int i;
     struct p2m_domain *p2m;
 
-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    if ( !d->arch.altp2m_p2m )
+        return;
+
+    for ( i = 0; i < max_altp2m; i++ )
     {
         if ( !d->arch.altp2m_p2m[i] )
             continue;
@@ -149,6 +158,9 @@ void p2m_teardown_altp2m(struct domain *d)
         d->arch.altp2m_p2m[i] = NULL;
         p2m_free_one(p2m);
     }
+
+    xfree(d->arch.altp2m_p2m);
+    d->arch.altp2m_p2m = NULL;
 }
 
 /*
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index 9f964c1d87..812a0f37d8 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -468,7 +468,7 @@ int hap_enable(struct domain *d, u32 mode)
     if ( old_pages == 0 )
     {
         paging_lock(d);
-        rv = hap_set_allocation(d, 256, NULL);
+        rv = hap_set_allocation(d, 2048, NULL);
         if ( rv != 0 )
         {
             hap_set_allocation(d, 0, NULL);
@@ -487,7 +487,7 @@ int hap_enable(struct domain *d, u32 mode)
     if ( rv != 0 )
         goto out;
 
-    for ( i = 0; i < MAX_NESTEDP2M; i++ )
+    for ( i = 0; i < max_nestedp2m; i++ )
     {
         rv = p2m_alloc_table(d->arch.nested_p2m[i]);
         if ( rv != 0 )
@@ -515,7 +515,7 @@ int hap_enable(struct domain *d, u32 mode)
             d->arch.altp2m_visible_eptp[i] = mfn_x(INVALID_MFN);
         }
 
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < max_altp2m; i++ )
         {
             rv = p2m_alloc_table(d->arch.altp2m_p2m[i]);
             if ( rv != 0 )
@@ -538,11 +538,11 @@ void hap_final_teardown(struct domain *d)
     unsigned int i;
 
     if ( hvm_altp2m_supported() )
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < max_altp2m; i++ )
             p2m_teardown(d->arch.altp2m_p2m[i], true, NULL);
 
     /* Destroy nestedp2m's first */
-    for (i = 0; i < MAX_NESTEDP2M; i++) {
+    for (i = 0; i < max_nestedp2m; i++) {
         p2m_teardown(d->arch.nested_p2m[i], true, NULL);
     }
 }
@@ -590,7 +590,7 @@ void hap_teardown(struct domain *d, bool *preempted)
         FREE_XENHEAP_PAGE(d->arch.altp2m_eptp);
         FREE_XENHEAP_PAGE(d->arch.altp2m_visible_eptp);
 
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < max_altp2m; i++ )
         {
             p2m_teardown(d->arch.altp2m_p2m[i], false, preempted);
             if ( preempted && *preempted )
@@ -599,7 +599,7 @@ void hap_teardown(struct domain *d, bool *preempted)
     }
 
     /* Destroy nestedp2m's after altp2m. */
-    for ( i = 0; i < MAX_NESTEDP2M; i++ )
+    for ( i = 0; i < max_nestedp2m; i++ )
     {
         p2m_teardown(d->arch.nested_p2m[i], false, preempted);
         if ( preempted && *preempted )
diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
index 3449e0ee85..85f38e0ad4 100644
--- a/xen/arch/x86/mm/mem_access.c
+++ b/xen/arch/x86/mm/mem_access.c
@@ -347,12 +347,12 @@ long p2m_set_mem_access(struct domain *d, gfn_t gfn, uint32_t nr,
     /* altp2m view 0 is treated as the hostp2m */
     if ( altp2m_idx )
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+        if ( altp2m_idx >= min(max_altp2m, MAX_EPTP) ||
              d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
              mfn_x(INVALID_MFN) )
             return -EINVAL;
 
-        ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        ap2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, max_altp2m)];
     }
 
     if ( !xenmem_access_to_p2m_access(p2m, access, &a) )
@@ -403,12 +403,12 @@ long p2m_set_mem_access_multi(struct domain *d,
     /* altp2m view 0 is treated as the hostp2m */
     if ( altp2m_idx )
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+        if ( altp2m_idx >= min(max_altp2m, MAX_EPTP) ||
              d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
              mfn_x(INVALID_MFN) )
             return -EINVAL;
 
-        ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        ap2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, max_altp2m)];
     }
 
     p2m_lock(p2m);
@@ -466,12 +466,12 @@ int p2m_get_mem_access(struct domain *d, gfn_t gfn, xenmem_access_t *access,
     }
     else if ( altp2m_idx ) /* altp2m view 0 is treated as the hostp2m */
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+        if ( altp2m_idx >= min(max_altp2m, MAX_EPTP) ||
              d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
              mfn_x(INVALID_MFN) )
             return -EINVAL;
 
-        p2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        p2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, max_altp2m)];
     }
 
     return _p2m_get_mem_access(p2m, gfn, access);
@@ -486,7 +486,7 @@ void arch_p2m_set_access_required(struct domain *d, bool access_required)
     if ( altp2m_active(d) )
     {
         unsigned int i;
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < max_altp2m; i++ )
         {
             struct p2m_domain *p2m = d->arch.altp2m_p2m[i];
 
diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
index 94b6b782ef..bb4ce3c8cc 100644
--- a/xen/arch/x86/mm/mem_sharing.c
+++ b/xen/arch/x86/mm/mem_sharing.c
@@ -918,7 +918,7 @@ static int nominate_page(struct domain *d, gfn_t gfn,
 
         altp2m_list_lock(d);
 
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < max_altp2m; i++ )
         {
             ap2m = d->arch.altp2m_p2m[i];
             if ( !ap2m )
@@ -1280,7 +1280,7 @@ int __mem_sharing_unshare_page(struct domain *d,
     {
         unsigned int i;
 
-        for ( i = 0; i < MAX_NESTEDP2M; i++ )
+        for ( i = 0; i < max_nestedp2m; i++ )
             p2m_lock(d->arch.nested_p2m[i]);
     }
 
@@ -1389,7 +1389,7 @@ int __mem_sharing_unshare_page(struct domain *d,
     {
         unsigned int i;
 
-        for ( i = 0; i < MAX_NESTEDP2M; i++ )
+        for ( i = 0; i < max_nestedp2m; i++ )
             p2m_unlock(d->arch.nested_p2m[i]);
     }
 
diff --git a/xen/arch/x86/mm/nested.c b/xen/arch/x86/mm/nested.c
index 03741ffae4..a765bb27d4 100644
--- a/xen/arch/x86/mm/nested.c
+++ b/xen/arch/x86/mm/nested.c
@@ -28,7 +28,13 @@ int p2m_init_nestedp2m(struct domain *d)
     struct p2m_domain *p2m;
 
     mm_lock_init(&d->arch.nested_p2m_lock);
-    for ( i = 0; i < MAX_NESTEDP2M; i++ )
+
+    if ( (d->arch.nested_p2m = xzalloc_array(struct p2m_domain *, max_nestedp2m)) == NULL )
+    {
+        return -ENOMEM;
+    }
+
+    for ( i = 0; i < max_nestedp2m; i++ )
     {
         d->arch.nested_p2m[i] = p2m = p2m_init_one(d);
         if ( p2m == NULL )
@@ -50,7 +56,10 @@ void p2m_teardown_nestedp2m(struct domain *d)
     unsigned int i;
     struct p2m_domain *p2m;
 
-    for ( i = 0; i < MAX_NESTEDP2M; i++ )
+    if ( !d->arch.nested_p2m )
+        return;
+
+    for ( i = 0; i < max_nestedp2m; i++ )
     {
         if ( !d->arch.nested_p2m[i] )
             continue;
@@ -59,4 +68,7 @@ void p2m_teardown_nestedp2m(struct domain *d)
         p2m_free_one(p2m);
         d->arch.nested_p2m[i] = NULL;
     }
+
+    xfree(d->arch.nested_p2m);
+    d->arch.nested_p2m = NULL;
 }
diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index 85c4e8e54f..6d9267552e 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -1292,7 +1292,7 @@ static void ept_set_ad_sync(struct domain *d, bool value)
     {
         unsigned int i;
 
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < max_altp2m; i++ )
         {
             struct p2m_domain *p2m;
 
@@ -1499,7 +1499,7 @@ void setup_ept_dump(void)
 
 void p2m_init_altp2m_ept(struct domain *d, unsigned int i)
 {
-    struct p2m_domain *p2m = array_access_nospec(d->arch.altp2m_p2m, i);
+    struct p2m_domain *p2m = d->arch.altp2m_p2m[array_index_nospec(i, max_altp2m)];
     struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
     struct ept_data *ept;
 
@@ -1518,7 +1518,7 @@ unsigned int p2m_find_altp2m_by_eptp(struct domain *d, uint64_t eptp)
 
     altp2m_list_lock(d);
 
-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < max_altp2m; i++ )
     {
         if ( d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
             continue;
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 4251144704..265584d243 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -104,7 +104,7 @@ void p2m_change_entry_type_global(struct domain *d,
     {
         unsigned int i;
 
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < max_altp2m; i++ )
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
                 struct p2m_domain *altp2m = d->arch.altp2m_p2m[i];
@@ -137,7 +137,7 @@ void p2m_memory_type_changed(struct domain *d)
     {
         unsigned int i;
 
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < max_altp2m; i++ )
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
                 struct p2m_domain *altp2m = d->arch.altp2m_p2m[i];
@@ -906,7 +906,7 @@ void p2m_change_type_range(struct domain *d,
     {
         unsigned int i;
 
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < max_altp2m; i++ )
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
                 struct p2m_domain *altp2m = d->arch.altp2m_p2m[i];
@@ -977,7 +977,7 @@ int p2m_finish_type_change(struct domain *d,
     {
         unsigned int i;
 
-        for ( i = 0; i < MAX_ALTP2M; i++ )
+        for ( i = 0; i < max_altp2m; i++ )
             if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             {
                 struct p2m_domain *altp2m = d->arch.altp2m_p2m[i];
@@ -1390,7 +1390,7 @@ p2m_flush_nestedp2m(struct domain *d)
 {
     unsigned int i;
 
-    for ( i = 0; i < MAX_NESTEDP2M; i++ )
+    for ( i = 0; i < max_nestedp2m; i++ )
     {
         struct p2m_domain *p2m = d->arch.nested_p2m[i];
 
@@ -1410,7 +1410,7 @@ void np2m_flush_base(struct vcpu *v, unsigned long np2m_base)
     np2m_base &= ~(0xfffULL);
 
     nestedp2m_lock(d);
-    for ( i = 0; i < MAX_NESTEDP2M; i++ )
+    for ( i = 0; i < max_nestedp2m; i++ )
     {
         p2m = d->arch.nested_p2m[i];
         p2m_lock(p2m);
@@ -1484,7 +1484,7 @@ p2m_get_nestedp2m_locked(struct vcpu *v)
     }
 
     /* Share a np2m if possible */
-    for ( i = 0; i < MAX_NESTEDP2M; i++ )
+    for ( i = 0; i < max_nestedp2m; i++ )
     {
         p2m = d->arch.nested_p2m[i];
         p2m_lock(p2m);
@@ -1770,7 +1770,7 @@ bool_t p2m_switch_vcpu_altp2m_by_id(struct vcpu *v, unsigned int idx)
     struct domain *d = v->domain;
     bool_t rc = 0;
 
-    if ( idx >= MAX_ALTP2M )
+    if ( idx >= max_altp2m )
         return rc;
 
     altp2m_list_lock(d);
@@ -1876,8 +1876,8 @@ static void p2m_reset_altp2m(struct domain *d, unsigned int idx,
 {
     struct p2m_domain *p2m;
 
-    ASSERT(idx < MAX_ALTP2M);
-    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+    ASSERT(idx < max_altp2m);
+    p2m = d->arch.altp2m_p2m[array_index_nospec(idx, max_altp2m)];
 
     p2m_lock(p2m);
 
@@ -1902,7 +1902,7 @@ void p2m_flush_altp2m(struct domain *d)
 
     altp2m_list_lock(d);
 
-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < max_altp2m; i++ )
     {
         p2m_reset_altp2m(d, i, ALTP2M_DEACTIVATE);
         d->arch.altp2m_eptp[i] = mfn_x(INVALID_MFN);
@@ -1918,9 +1918,9 @@ static int p2m_activate_altp2m(struct domain *d, unsigned int idx,
     struct p2m_domain *hostp2m, *p2m;
     int rc;
 
-    ASSERT(idx < MAX_ALTP2M);
+    ASSERT(idx < max_altp2m);
 
-    p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+    p2m = d->arch.altp2m_p2m[array_index_nospec(idx, max_altp2m)];
     hostp2m = p2m_get_hostp2m(d);
 
     p2m_lock(p2m);
@@ -1958,7 +1958,7 @@ int p2m_init_altp2m_by_id(struct domain *d, unsigned int idx)
     int rc = -EINVAL;
     struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
 
-    if ( idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
+    if ( idx >= min(max_altp2m, MAX_EPTP) )
         return rc;
 
     altp2m_list_lock(d);
@@ -1985,7 +1985,7 @@ int p2m_init_next_altp2m(struct domain *d, uint16_t *idx,
 
     altp2m_list_lock(d);
 
-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < max_altp2m; i++ )
     {
         if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
             continue;
@@ -2007,7 +2007,7 @@ int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
     struct p2m_domain *p2m;
     int rc = -EBUSY;
 
-    if ( !idx || idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
+    if ( !idx || idx >= min(max_altp2m, MAX_EPTP) )
         return rc;
 
     rc = domain_pause_except_self(d);
@@ -2020,7 +2020,7 @@ int p2m_destroy_altp2m_by_id(struct domain *d, unsigned int idx)
     if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] !=
          mfn_x(INVALID_MFN) )
     {
-        p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+        p2m = d->arch.altp2m_p2m[array_index_nospec(idx, max_altp2m)];
 
         if ( !_atomic_read(p2m->active_vcpus) )
         {
@@ -2045,7 +2045,7 @@ int p2m_switch_domain_altp2m_by_id(struct domain *d, unsigned int idx)
     struct vcpu *v;
     int rc = -EINVAL;
 
-    if ( idx >= MAX_ALTP2M )
+    if ( idx >= max_altp2m )
         return rc;
 
     rc = domain_pause_except_self(d);
@@ -2080,13 +2080,13 @@ int p2m_change_altp2m_gfn(struct domain *d, unsigned int idx,
     mfn_t mfn;
     int rc = -EINVAL;
 
-    if ( idx >=  min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+    if ( idx >=  min(max_altp2m, MAX_EPTP) ||
          d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
          mfn_x(INVALID_MFN) )
         return rc;
 
     hp2m = p2m_get_hostp2m(d);
-    ap2m = array_access_nospec(d->arch.altp2m_p2m, idx);
+    ap2m = d->arch.altp2m_p2m[array_index_nospec(idx, max_altp2m)];
 
     p2m_lock(hp2m);
     p2m_lock(ap2m);
@@ -2142,7 +2142,7 @@ int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
 
     altp2m_list_lock(d);
 
-    for ( i = 0; i < MAX_ALTP2M; i++ )
+    for ( i = 0; i < max_altp2m; i++ )
     {
         p2m_type_t t;
         p2m_access_t a;
@@ -2165,7 +2165,7 @@ int p2m_altp2m_propagate_change(struct domain *d, gfn_t gfn,
             else
             {
                 /* At least 2 altp2m's impacted, so reset everything */
-                for ( i = 0; i < MAX_ALTP2M; i++ )
+                for ( i = 0; i < max_altp2m; i++ )
                 {
                     if ( i == last_reset_idx ||
                          d->arch.altp2m_eptp[i] == mfn_x(INVALID_MFN) )
@@ -2565,12 +2565,12 @@ int p2m_set_suppress_ve_multi(struct domain *d,
 
     if ( sve->view > 0 )
     {
-        if ( sve->view >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+        if ( sve->view >= min(max_altp2m, MAX_EPTP) ||
              d->arch.altp2m_eptp[array_index_nospec(sve->view, MAX_EPTP)] ==
              mfn_x(INVALID_MFN) )
             return -EINVAL;
 
-        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, sve->view);
+        p2m = ap2m = d->arch.altp2m_p2m[array_index_nospec(sve->view, max_altp2m)];
     }
 
     p2m_lock(host_p2m);
@@ -2633,12 +2633,12 @@ int p2m_get_suppress_ve(struct domain *d, gfn_t gfn, bool *suppress_ve,
 
     if ( altp2m_idx > 0 )
     {
-        if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+        if ( altp2m_idx >= min(max_altp2m, MAX_EPTP) ||
              d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
              mfn_x(INVALID_MFN) )
             return -EINVAL;
 
-        p2m = ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
+        p2m = ap2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx, max_altp2m)];
     }
     else
         p2m = host_p2m;
@@ -2669,9 +2669,9 @@ int p2m_set_altp2m_view_visibility(struct domain *d, unsigned int altp2m_idx,
 
     /*
      * Eptp index is correlated with altp2m index and should not exceed
-     * min(MAX_ALTP2M, MAX_EPTP).
+     * min(max_altp2m, MAX_EPTP).
      */
-    if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+    if ( altp2m_idx >= min(max_altp2m, MAX_EPTP) ||
          d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
          mfn_x(INVALID_MFN) )
         rc = -EINVAL;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 10:22:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 10:22:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676750.1053039 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIbD-0000aD-DS; Tue, 06 Feb 2024 10:22:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676750.1053039; Tue, 06 Feb 2024 10:22:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIbD-0000a6-AS; Tue, 06 Feb 2024 10:22:43 +0000
Received: by outflank-mailman (input) for mailman id 676750;
 Tue, 06 Feb 2024 10:08:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HdcV=JP=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1rXINT-0004Uc-Qm
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 10:08:31 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id adac36df-c4d7-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 11:08:30 +0100 (CET)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-33b4437e132so592318f8f.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 02:08:30 -0800 (PST)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 z9-20020a056000110900b0033b43aaac8fsm1706914wrw.7.2024.02.06.02.08.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 06 Feb 2024 02:08:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: adac36df-c4d7-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707214110; x=1707818910; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=mzv8a2Pw/K0/iyLlK8ysT6UaHzFq2GFrZoFzc1+mp2Q=;
        b=dL+rVXKnv5FlCpjDYeCTlu2UeelQ1pitIID7SxQ/y5J1SNqMjMOE16KmRGrpj7MusE
         uZT/JMrlDHNnu6DecNZx0Kul4tw+indFZ3vJnLP591BMiW1zKuqlJbLKOJjVqrmwgvV3
         X5vWR7fJH4q4fm1h1Q3ftESOMR+lS74V8QRJIYKgUSY/RO6xP/zeuNEDwQiATVSBNpqR
         MqKgCbFgdZ8pfOEY5rWgbvlAs+ymaptXy00qE4+f/wN8Bpui1wbe448/VsQKRL+T74ZC
         nh7ul5e/CN15U1MBnveBnfQdxcG6KVN73otfSK83xoikabi6HQyxPEEA3QKL+nAi+BEI
         znVA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707214110; x=1707818910;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=mzv8a2Pw/K0/iyLlK8ysT6UaHzFq2GFrZoFzc1+mp2Q=;
        b=ISOu1P6EsSDwHREyaMeeboKn63OL1NBT1oIS6Zz9JZY5P4kj0I6o85G/58pZ4aeyqu
         TT2/mAdBgB3sJ1nQEOrJU5KEk15YlM5F96iZgNrs4XPqmX89fEOHpPhks/ezmon0tLCc
         s9m7cSUUjzC8YXwY6iyYsX7Z2pOld96yitZeg1gkFCqsZIfhfL+nOF/QbIS+nwM25wIk
         +7RJpGSzutcOUrEVFriwC8vT2Bgz+BNNjq0H56fAix6RRrDV6nGPvoIJ3BwPZp7W4IMh
         aSAamjXKqlxdNcuLZo+2/cKEQovSnLzHS3ENhVrs7jkwdgEfpmGssBxPH2rEij7vLkLB
         8a0Q==
X-Gm-Message-State: AOJu0Yw9mi5KFUZGd6EJXrGdsfH31uuPZRH7+ey3af/+C2no32ppxUZJ
	B2b28j3G18cw0WcqYdg3Uf23cCHAIuCbgIvFawRZE9fBf4jgnDE5M2TyDyt44fo=
X-Google-Smtp-Source: AGHT+IFMq+xArz6wMDh9WSL/7949AS5ugVC/LOZX62KrJl2879zr2oaRkdjKsN094FJ6zdu+NnqSQw==
X-Received: by 2002:adf:e9c7:0:b0:33b:4672:ff32 with SMTP id l7-20020adfe9c7000000b0033b4672ff32mr869165wrn.45.1707214109778;
        Tue, 06 Feb 2024 02:08:29 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCU1Mgb/+Zbs2qnv4ZZxnnXcOrvERHQxmBxgyo6HOq0ENU1BDrm5X6S01jPavOqeDl8MMtdlqA1AEXj5OmjZIRX0a8bNPIToHfgxIrfs5pAVjQC1CoKIhWWhxKV1AKkWBY1maDmAdbK2MgOb5BxYFps7R0dJfk9Dy1mRfyjNjPAWRlJcfmLKY4YMXyx70dKjo4OtqV/aNCkWQgNI3qxOlZn0SJMy51faFa2jM2fXaRvmlH3AuzxYODzeK+BS+CXsYtJ8IPQMiiMR/NU4i9wvm2EWplMv9xwYsTI/Rx877/CuJOC9zfwUra2Rb2/XDK+T9s8S6ctURKQ6K6NViTKR
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor ap2m->default_access
Date: Tue,  6 Feb 2024 10:08:22 +0000
Message-Id: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

This patch addresses a behavior discrepancy in the handling of altp2m views,
where upon the creation and subsequent EPT violation, the page access
permissions were incorrectly inherited from the hostp2m instead of respecting
the altp2m default_access.

Previously, when a new altp2m view was established with restrictive
default_access permissions and activated via xc_altp2m_switch_to_view(),
it failed to trigger an event on the first access violation.  This behavior
diverged from the intended mechanism, where the altp2m's default_access
should dictate the initial permissions, ensuring proper event triggering on
access violations.

The correction involves modifying the handling mechanism to respect the
altp2m view's default_access upon its activation, eliminating the need for
setting memory access permissions for the entire altp2m range (e.g. within
xen-access.c).  This change not only aligns the behavior with the expected
access control logic but also results in a significant performance improvement
by reducing the overhead associated with setting memory access permissions
across the altp2m range.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 tools/misc/xen-access.c | 14 --------------
 xen/arch/x86/mm/p2m.c   |  3 +++
 2 files changed, 3 insertions(+), 14 deletions(-)

diff --git a/tools/misc/xen-access.c b/tools/misc/xen-access.c
index 4097eebe6f..6cf3b6a42c 100644
--- a/tools/misc/xen-access.c
+++ b/tools/misc/xen-access.c
@@ -517,9 +517,6 @@ int main(int argc, char *argv[])
     /* With altp2m we just create a new, restricted view of the memory */
     if ( memaccess && altp2m )
     {
-        xen_pfn_t gfn = 0;
-        unsigned long perm_set = 0;
-
         if( altp2m_write_no_gpt )
         {
             rc = xc_monitor_inguest_pagefault(xch, domain_id, 1);
@@ -551,17 +548,6 @@ int main(int argc, char *argv[])
         }
 
         DPRINTF("altp2m view created with id %u\n", altp2m_view_id);
-        DPRINTF("Setting altp2m mem_access permissions.. ");
-
-        for(; gfn < xenaccess->max_gpfn; ++gfn)
-        {
-            rc = xc_altp2m_set_mem_access( xch, domain_id, altp2m_view_id, gfn,
-                                           default_access);
-            if ( !rc )
-                perm_set++;
-        }
-
-        DPRINTF("done! Permissions set on %lu pages.\n", perm_set);
 
         rc = xc_altp2m_switch_to_view( xch, domain_id, altp2m_view_id );
         if ( rc < 0 )
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 0983bd71d9..4251144704 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1849,6 +1849,9 @@ bool p2m_altp2m_get_or_propagate(struct p2m_domain *ap2m, unsigned long gfn_l,
     amfn = _mfn(mfn_x(*mfn) & mask);
     gfn = _gfn(gfn_l & mask);
 
+    /* Override the altp2m entry with its default access. */
+    *p2ma = ap2m->default_access;
+
     rc = p2m_set_entry(ap2m, gfn, amfn, cur_order, *p2mt, *p2ma);
     p2m_unlock(ap2m);
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 10:27:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 10:27:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676780.1053059 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIg9-0001wz-AL; Tue, 06 Feb 2024 10:27:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676780.1053059; Tue, 06 Feb 2024 10:27:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXIg9-0001ws-7L; Tue, 06 Feb 2024 10:27:49 +0000
Received: by outflank-mailman (input) for mailman id 676780;
 Tue, 06 Feb 2024 10:27:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gRx7=JP=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rXIg7-0001vY-6o
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 10:27:47 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5dc2209f-c4da-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 11:27:45 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-511612e0c57so145492e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 02:27:45 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 k19-20020a05600c1c9300b0040ee51f1025sm1560842wms.43.2024.02.06.02.27.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 06 Feb 2024 02:27:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5dc2209f-c4da-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707215264; x=1707820064; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=gIJji4S2DBBQRmFrU6n0aaD7nQmednIs6SXBmScTcv0=;
        b=bNYVJTSkrJeLqyJfstkdU2qirhExiwiRXA2ATNYKLnzF0FIDu1kT8GwE7ZefJRNliZ
         eq941Rk8CJbo6wqMb/AIU4/2GKgr3Mhuf3hJ0BnDQ5qcnA9H9bdPf+ahcLfpO17Aqos7
         V1RIU+ZGlSu5RonnytPXVNPgZxB+GcEIOvd2Y=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707215264; x=1707820064;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=gIJji4S2DBBQRmFrU6n0aaD7nQmednIs6SXBmScTcv0=;
        b=bNHjZQI61GDccvUGYVf/crhErvEo74C53r5O0kqmryAUHDmgfi4dXt5svhZzXg1Zyu
         bfPmll9bY67R7tiCfYeuBTvsZuLEdGELq7oOAWSYZVTe1UBV7n8OPUnBYZWLD8uTGBB+
         fiSMPuE1O4Hbewu+1wWErOFy0mSpD3TuijTGlWN5E1/yzk1GCn0SG2eCx6N5tPAoWCVh
         K7gP3uA+vXv32IWcOGfu0n9roKBFvTLhaCQwjDW+DoKVz4HIOd577wyYgI3fcaolczim
         NGJXCmnxQCaXu/ENB+Q0tWw7hbal+gOOXTmKRZUPp+qiaSxLSCCor2DwM/WzQ9v5m/jL
         +qvQ==
X-Gm-Message-State: AOJu0YwD2j9D9Zjth8fUqIHpCIEKUg6cp8jrRq8/st1rVv/D/b92Caoe
	eUWna9k7lKxn0XvDelszWkO7WHgNoMtHx9UKgqX2M3eVGW75l7tCJHhXY250uJA=
X-Google-Smtp-Source: AGHT+IHwdeQVjTbM1YbFPncQovpbpjZOh7TQfQc0fQ55UWz60u0ml8AbmNV/0j4V/RmZciofC2SLlA==
X-Received: by 2002:a19:6750:0:b0:511:4e8f:e21f with SMTP id e16-20020a196750000000b005114e8fe21fmr1117265lfj.6.1707215264537;
        Tue, 06 Feb 2024 02:27:44 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUokOxB4uYlUEO3NTIvG48esRUrubYCuHT1IHwaJwaMa+DwoyWzEBQ03qSbwiHb2SkZK7m4lU2Zrw4BZo5apC0UEcwMWNT7ZGyRUx3tF2/4IQ2sZSqhOoo09cG4X7dUZuv3KByDuDFzByfDkO6NVwncDvD9awq5UdBGbEAwrv4hu4P+7AySsIXdiiOHD0rOl45zNu0RTlqXy4KqT9shH60S+A6WqPhbOk0=
Date: Tue, 6 Feb 2024 10:27:43 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Wei Liu <wl@xen.org>, Oleksandr Tyshchenko <olekstysh@gmail.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Juergen Gross <jgross@suse.com>,
	Viresh Kumar <viresh.kumar@linaro.org>
Subject: Re: [PATCH] libxl: Add "grant_usage" parameter for virtio disk
 devices
Message-ID: <d7291530-9e32-43dc-8b82-34d3962d2551@perard>
References: <20240202104903.1112772-1-olekstysh@gmail.com>
 <ace5b3c8-38e7-4661-9401-ac9ac77a5e9f@perard>
 <78bc7890-cfcb-4de4-a676-a3c2bbbb6d9f@epam.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <78bc7890-cfcb-4de4-a676-a3c2bbbb6d9f@epam.com>

On Mon, Feb 05, 2024 at 04:52:16PM +0000, Oleksandr Tyshchenko wrote:
> On 05.02.24 17:10, Anthony PERARD wrote:
> > On Fri, Feb 02, 2024 at 12:49:03PM +0200, Oleksandr Tyshchenko wrote:
> >> +grant_usage=1,?		{ libxl_defbool_set(&DPC->disk->grant_usage, true); }
> >> +grant_usage=0,?		{ libxl_defbool_set(&DPC->disk->grant_usage, false); }
> > 
> > For other boolean type for the disk, we have "trusted/untrusted",
> > "discard/no-discard", "direct-io-save/", but you are adding
> > "grant_usage=1/grant_usage=0". Is that fine? But I guess having the new
> > option spelled "grant_usage" might be better, so it match the other
> > virtio devices and the implementation. 
> 
> 
> Yes, I noticed that how booleans are described for the disk. I decided 
> to use the same representation of this option as it was already used for 
> virtio=[...]. But I would be ok with other variants ...
> 
> 
> But maybe
> > "use-grant/no-use-grant" might be ok?
> 
>    ... like that, but preferably with leaving libxl_device_disk's field 
> named "grant_usage" (if no objection).
> 
> >> diff --git a/docs/man/xl-disk-configuration.5.pod.in b/docs/man/xl-disk-configuration.5.pod.in
> >> index bc945cc517..3c035456d5 100644
> >> --- a/docs/man/xl-disk-configuration.5.pod.in
> >> +++ b/docs/man/xl-disk-configuration.5.pod.in
> >> @@ -404,6 +404,31 @@ Virtio frontend driver (virtio-blk) to be used. Please note, the virtual
> >> +=item B<grant_usage=BOOLEAN>
> >>
> >> +=over 4
> >> +
> >> +=item Description
> >> +
> >> +Specifies the usage of Xen grants for accessing guest memory. Only applicable
> >> +to specification "virtio".
> >> +
> >> +=item Supported values
> >> +
> >> +If this option is B<true>, the Xen grants are always enabled.
> >> +If this option is B<false>, the Xen grants are always disabled.
> > 
> > Unfortunately, this is wrong, the implementation in the patch only
> > support two values: 1 / 0, nothing else, and trying to write "true" or
> > "false" would lead to an error. (Well actually it's "grant_usage=1" or
> > "grant_usage=0", there's nothing that cut that string at the '='.)
> 
> 
> You are right, only 1 / 0 can be set unlike for virtio=[...] which seems 
> happy with false/true.
> 
> 
> > 
> > Also, do we really need the extra verbal description of each value here?
> > Is simply having the following would be enough?
> > 
> >      =item Supported values
> > 
> >      1, 0
> > 
> > The description in "Description" section would hopefully be enough.
> 
> 
> I think, this makes sense.
> 
> So, shall I leave "grant_usage=1/grant_usage=0" or use proposed option 
> "use-grant/no-use-grant"?

Let's go with "grant_usage=*", at least this will be consistent with the
option for "virtio".

Cheers,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 10:55:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 10:55:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676787.1053069 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJ7C-0007CM-DA; Tue, 06 Feb 2024 10:55:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676787.1053069; Tue, 06 Feb 2024 10:55:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJ7C-0007CF-AZ; Tue, 06 Feb 2024 10:55:46 +0000
Received: by outflank-mailman (input) for mailman id 676787;
 Tue, 06 Feb 2024 10:55:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+UH4=JP=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXJ7B-0007C9-2W
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 10:55:45 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 459117fe-c4de-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 11:55:43 +0100 (CET)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a370e63835cso476695766b.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 02:55:43 -0800 (PST)
Received: from [10.80.67.148] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 p27-20020a170906229b00b00a35a9745910sm994976eja.137.2024.02.06.02.55.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 02:55:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 459117fe-c4de-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707216942; x=1707821742; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=q7EGBs9t96C7x6Ctji4e+haWvBsMICLofs43/FPqB+k=;
        b=CG2qAfKyRboRsIFs3fowpV7ZV8WEBLdAnDZ3T/B7LXJ3c43DwAWuK8JL54Wpwx/9tE
         9ETlgf4uaVKhwjioMC2QuLxZtBJU5JHtnxbKE67Zq+2fHEqaKGxjB4MuIBzFg3F6wSnx
         3MvBXogib3fOFlNE7RhLjnXn1+gnLGQhOG0BU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707216942; x=1707821742;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=q7EGBs9t96C7x6Ctji4e+haWvBsMICLofs43/FPqB+k=;
        b=tkTFfZ6C8J3y6HvHWsn6YbHRY5Rl3IoWcliJBI8XaaMeVZfPmJN46fQ8x4lNPtwZGh
         QVNXfwo+db5UpbZkzGHslO0/lMEcbAb6+qiNjhTOJWjclkwifjQkQAHDqypEkmZxDgyS
         Dh09FyGqyiEVMMFsVqZET8k1QfDhxzAza3WBiJXUgrobrH0UNdai7Ww7HL6TpHzYVgio
         KldvrjWk/Msh3LjUvsA2+opTLhrUyQZ3/76cm2LONyUjtHaLOOaeGDx8FIEzDzTDt3CB
         RKDB5aiEEU72LXMWsj47DXox9CxhfoQpfR+xLZM9IIubQTOU4DEuC9dQbExFpPYKdnOG
         8XUg==
X-Gm-Message-State: AOJu0YyFjmypmwhAXOaqaQsxhP6aULF+ha+NxDlkPY5SoFe+gsQIzez9
	X7BrK1Aug4E7pLErFHpl7nlQIUQfA2tBr/k9FMm+tjP5MkOro8WYxlJajp9363k=
X-Google-Smtp-Source: AGHT+IGSvqqcsqfU2MPY9aUWvcirIzL2odXl28JYitWhL5z2xTHVwFC/tXyYO8yRNKow1ULFV5+bRg==
X-Received: by 2002:a17:906:3e4e:b0:a26:c9e6:d5e0 with SMTP id t14-20020a1709063e4e00b00a26c9e6d5e0mr1518283eji.74.1707216941973;
        Tue, 06 Feb 2024 02:55:41 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXUTnqkMA/zXs7PZMVpAMNXUX+4Qnev6qFK0gUNA8wUI1QE95z7gmWNFk6V9xuLhKMqP8Rw6izHv0Qol71k9O4me3Aj755gm1ShYhpDtPs5yRgjlRrQYgXX5CdtKKl/jN3OTps8mzY3k+8B78R1VebnWzC2tz7t+UkXeiBf9+irl5u6FEwE0Logk4qeNq+VoLjlbNVf+cJhDGf9FrimX4AiLj/6ptTyP2mwOkfKw3TZGeBHEtIE6OzTfDZ5b4qcfvRdCvqlkcpsm7BkszRB1IWx2aoXP3CCDJGxMVJsP6yExm7NpCkG54bHdKKJohUT3zqoPgYtMQA4Q/YKTPDQDFLfW3ytY80iC0c5PzOkTj3xdFJ4UErTN0ITLs8nKxTv4y45Adie
Message-ID: <c0312e70-c11c-4b15-b6c9-457004b988d1@citrix.com>
Date: Tue, 6 Feb 2024 10:55:40 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86: Add configuration options for max_altp2m and
 max_nestedp2m limits
Content-Language: en-GB
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Jun Nakajima <jun.nakajima@intel.com>,
 Kevin Tian <kevin.tian@intel.com>, George Dunlap <george.dunlap@citrix.com>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>
References: <9143475dddf8d0141fb3864ed90153591554a425.1707213019.git.w1benny@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <9143475dddf8d0141fb3864ed90153591554a425.1707213019.git.w1benny@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06/02/2024 10:08 am, Petr Beneš wrote:
> From: Petr Beneš <w1benny@gmail.com>
>
> This commit introduces the ability to configure the maximum number of altp2m
> and nestedp2m tables through boot-time parameters.  Previously, the limits were
> hardcoded to a maximum of 10 for both.  This change allows for greater
> flexibility in environments that require more or fewer tables, enhancing Xen's
> adaptability to different workloads and scenarios.
>
> Adjustments include:
> - Adding boot parameters `max_altp2m` and `max_nestedp2m` to allow setting
>   these limits at boot time.
> - Modifying various parts of the code to use these configurable limits instead
>   of the previous hardcoded values.
> - Ensuring that if the configured values exceed the maximum supported EPTP
>   entries, they are adjusted down with a warning logged.
> - Adjusting the initial allocation of pages in `hap_enable` from 256 to 2048
>   pages when `old_pages == 0`.
>
>   This change anticipates scenarios where `max_altp2m` and `max_nestedp2m`
>   are set to their maximum supported values (i.e., 512), ensuring sufficient
>   memory is allocated upfront to accommodate all altp2m and nestedp2m tables
>   without initialization failure.
>
> This enhancement is particularly relevant for users leveraging Xen's features
> for virtual machine introspection or nested virtualization support.
>
> Signed-off-by: Petr Beneš <w1benny@gmail.com>

Thankyou for the patch.  These fields should definitely be configurable.

I realise you're copying existing examples, but they happen to be bad
examples that we're trying to remove.  It is erroneous to believe that
there's one system-wide configuration that applies to all domains, and
this has been the source of too many bugs.

Instead, they should be parameters to the domain create hypercall, so
they can be chosen on a per-VM basis.

e.g. for some introspection tasks you only want 1 view + 1 alt per vCPU,
for usually only a single vCPU.

On the other hand, for nested-virt, you really need a minimum of two per
vCPU, and probably 4, and there's no architectural upper bound.


See the series adding vmtrace= support, which is the last time I plumbed
a number like this into the domain create hypercall.

You will want to split the patch into a series.  For each of option,
you'll want one patch plumbing the new parameter into the hypercall
(along with a suitable toolstack default, and sanity checking), and a
separate patch to rearrange Xen to use the new dynamic size.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 11:07:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 11:07:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676798.1053078 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJIq-00010r-E8; Tue, 06 Feb 2024 11:07:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676798.1053078; Tue, 06 Feb 2024 11:07:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJIq-00010k-BU; Tue, 06 Feb 2024 11:07:48 +0000
Received: by outflank-mailman (input) for mailman id 676798;
 Tue, 06 Feb 2024 11:07:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXJIp-00010e-K2
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 11:07:47 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f50168db-c4df-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 12:07:46 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d051fb89fbso10021631fa.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 03:07:46 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f17-20020a05600c4e9100b0040ec66021a7sm1638724wmq.1.2024.02.06.03.07.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 03:07:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f50168db-c4df-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707217666; x=1707822466; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=iUG6/+tZCYrHwnRon06RIRkeDgg4m+Y80C/Gx+1VKU0=;
        b=Bx1pRlBJsMnpXH5H7HwgQTIEgnQVxMd8/WIRC83NQqC9X2GxaIUFT5g6eejSIJm/+V
         kUM25ZK0y75pwIUpP0Re3GONx1NsogbFsw1QpEy4k5tKRPWWEOLLKDDMUQbYw4YngVEH
         KOC0vVij5GLuf/H+DHLyAtA9DTgYW0ajVj8CbzVKryTlu2KGmQ+qzAKMzg3FFaCe+9Po
         rmb7TDCLdVKUiOuCFRl7r7McL1drwESHTLUfJLlP44QT+3LzwX/g+qNPzFPWhVoBeGn3
         mDgiJRCQCxXW6EjmXmk039+UypW3d5RjVL5TtDqn1V3ByJxz19HGEWsDWtCX6FI7iwff
         p2jQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707217666; x=1707822466;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=iUG6/+tZCYrHwnRon06RIRkeDgg4m+Y80C/Gx+1VKU0=;
        b=tsU+cl52M808Mw/A8ecnfXnHKZMS2GJ5A/vTJhYYADmPD+tVcdwJvA94PVsohVFgm5
         0fRvgD7VYhY/s4bokCvkTDOG4I2KP38wEaM9s0FiYMe8bTqYahuhMeO5brYnkiy5S8UJ
         Et1939YCFmZ8f9rEJ3Jj9TEdCokMN8BEmiL8U/+qR9/vgdvZotb55Gc5quK5ImLi72rf
         MDPbQ78LXugFGRulDYiD4u52gGrsqmhrrTXCUC2D2AGHbAe+tIBNrIvPd879Rz53x+Pd
         cUGegG1n41pIJrJvd8yNjTVYIS5Q/8z71gVMcHZ4wLJU8m1ru1GNDtNVp5utEiGbLsQN
         qQ7Q==
X-Gm-Message-State: AOJu0YwaGXgtZINcKjpag2OBU5j6gSGK4VaXDlT7lxKSHLoOpMeMycVo
	Nf3kJgxo/1DNh1VL5868YS6eIopX1d6+iZROF/7SHGOnoRgA1FyoAHQiYipylw==
X-Google-Smtp-Source: AGHT+IGgVS+NWNpXS5ZX77lU+K7ee0WtEzreozvtX+2uX4dl1Slz0a+vsGrtoIoa/nMc8x7y9oHZcg==
X-Received: by 2002:a19:ca52:0:b0:511:34fd:7de4 with SMTP id h18-20020a19ca52000000b0051134fd7de4mr1612576lfj.18.1707217665820;
        Tue, 06 Feb 2024 03:07:45 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVIjpbIKnhRlb6SP53yBM9WC6d7K5xdi97vyboiSb4woEtfwNtlSyCqtM56HzDscYXn627LSJqk0oLGrfRVbG4Dt2L63sf5O9Q+2zL4Y7gDX+DgsdgW+PkBS1jeKa3HRNG59BrqjVufEL9CGWTyqYDwLNyV804bLFj53Yogxf1fD47p1a69z7U4No7LJDOj4elG9gWH7IsLjBTMIS+uO744APLGZrIUBhCpqUQniDZz47L1cxl+AxgbpK8+dreo/sH1b3YiCGPLZcM3CixuZyU2DQxU5dQOD8B2QCKwY5l6CQP3SgNbA/hRImwSPgL1K3m7vXW0iZ0poTuQ+3FG0RI0uFJTTKQWnM/M6Jfm1WuvnEiFMZwz5v4gvSzI20PrJYM7/azvyIvNX9AyhC4=
Message-ID: <8169b6f0-b6d6-4d1a-bac4-a068c6f15306@suse.com>
Date: Tue, 6 Feb 2024 12:07:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86: Add configuration options for max_altp2m and
 max_nestedp2m limits
Content-Language: en-US
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>, xen-devel@lists.xenproject.org
References: <9143475dddf8d0141fb3864ed90153591554a425.1707213019.git.w1benny@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <9143475dddf8d0141fb3864ed90153591554a425.1707213019.git.w1benny@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.02.2024 11:08, Petr Beneš wrote:
> From: Petr Beneš <w1benny@gmail.com>
> 
> This commit introduces the ability to configure the maximum number of altp2m
> and nestedp2m tables through boot-time parameters.  Previously, the limits were
> hardcoded to a maximum of 10 for both.  This change allows for greater
> flexibility in environments that require more or fewer tables, enhancing Xen's
> adaptability to different workloads and scenarios.
> 
> Adjustments include:
> - Adding boot parameters `max_altp2m` and `max_nestedp2m` to allow setting
>   these limits at boot time.
> - Modifying various parts of the code to use these configurable limits instead
>   of the previous hardcoded values.
> - Ensuring that if the configured values exceed the maximum supported EPTP
>   entries, they are adjusted down with a warning logged.
> - Adjusting the initial allocation of pages in `hap_enable` from 256 to 2048
>   pages when `old_pages == 0`.

This change ought to come separately, with its own justification. Assuming
it can be justified at all, considering it also affects the non-nested,
non-alternative case.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 11:17:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 11:17:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676807.1053089 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJSJ-00032O-Bf; Tue, 06 Feb 2024 11:17:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676807.1053089; Tue, 06 Feb 2024 11:17:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJSJ-00032H-8G; Tue, 06 Feb 2024 11:17:35 +0000
Received: by outflank-mailman (input) for mailman id 676807;
 Tue, 06 Feb 2024 11:17:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXJSH-00032B-H0
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 11:17:33 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 51c23eee-c4e1-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 12:17:31 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-40ff2838851so490985e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 03:17:31 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l6-20020a5d5266000000b0033b04f7585dsm1858668wrc.3.2024.02.06.03.17.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 03:17:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51c23eee-c4e1-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707218251; x=1707823051; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jTlDZCF5STgE7bqWsxYU6ACHRM/5epAF+J6uP5ItGAM=;
        b=VWV5xwnVT0AUGlqINTDnBXoETeRORyV3w03tJAchIao3yGlawrOfs0XZ2KkwKtuUKd
         I+omcQy27tU/880700M/6sb2jZFQGP3iZHTKyqLXutNmW5bGmjL0xmsbpCIDQVVb2U/Y
         6Z9R70kkefYo+a7BGC0KX5pql1F9AFYQFZfuUxRQ01swHM1mvzjzHvUbRs8ksiLe7NKR
         3SWwSH0o9WibIsU98TFrPKxkha+PovuM6FHg/CbpRTncr9cNKzcqFt55k/Da/jHMuLTn
         dVetLaM826oC2AIBfq1w25orRysO6wNNB7TRxaLQKoMcrJ8Z27keQJ7v/qsoS71a5VZq
         keng==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707218251; x=1707823051;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jTlDZCF5STgE7bqWsxYU6ACHRM/5epAF+J6uP5ItGAM=;
        b=gWzf27G2S6W/es8wSXuIiH/L/ChOj87a915dDsTYfz7dCnAOn/L+HrJcAIVesT1tfC
         +w5cmsSlDV9KBGxnuqeByToZjbmMkVUhKyUDCMtxAPijrI+XCTm/cs/kpnseulT7pXx6
         3l/dloAAYAj7p/0zkecTwyCc0aAq1RdhNPFlxpapKu7Je9PQ+UCO7KUflka78LuWhV1Y
         jwsI7uY2S3G8PZ+j6rohZvm1xnOlpSkM5xSd+/liqayX4fLClGmqp/GcW7mgA/OV9kfe
         lXLiOWgRtflQ5nbNwTl3MSEKwueGBYAnWRlH6Ot8An0Nzf1BU7/PALfVQzT0IRvb7Ykz
         Nc5A==
X-Gm-Message-State: AOJu0YwXkI10b8qz2CtlDpr54d7/1fogHmQLql4Eubno6CtWYEiULhio
	X5ofCjPqAfhe5Pnmm3MwmLfIrSmN5g6msnbxJLwNfNPA3+nHC9Z6rM23QAEVyQ==
X-Google-Smtp-Source: AGHT+IEOcEzkFCGNqeeONaQK3q2Lc4xS5yI1gSmBmhUmfaoBhl1Vg7v1CVt46dKiGzIJ3dEvyOd0aA==
X-Received: by 2002:a05:600c:3147:b0:40f:af68:3ab with SMTP id h7-20020a05600c314700b0040faf6803abmr1425313wmo.5.1707218250866;
        Tue, 06 Feb 2024 03:17:30 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVbob1Gbdym7yT8cB9HR0nks/P8VgAsH0cGnGgwaw/E5DU6q+s173LJANGedFTZufirTGHck5bozyayq9R/aJUY6lQZzdKCdCcM4hpEPeaAN0cQ1iWildsN9Xluw0Hquoxq2MXGxgatqg==
Message-ID: <232baaae-5fe0-4a94-8469-d1dac520c1af@suse.com>
Date: Tue, 6 Feb 2024 12:17:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/4] iommu/x86: introduce a generic IVMD/RMRR range
 validity helper
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
References: <20240201170159.66330-1-roger.pau@citrix.com>
 <20240201170159.66330-3-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240201170159.66330-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.02.2024 18:01, Roger Pau Monne wrote:
> IVMD and RMRR ranges are functionally equivalent, and as so could use the same
> validity checker.

May I suggest s/equivalent/similar/?

> Move the IVMD to x86 common IOMMU code and adjust the function to take a pair
> of [start, end) mfn parameters.

[start,end) ranges generally come with the problem of not allowing to
represent the full address space. While that isn't specifically a problem
here, seeing that both VT-d and V-i present inclusive ranges, how about
making the common function match that?

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 11:18:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 11:18:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676811.1053098 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJTb-0003yp-OL; Tue, 06 Feb 2024 11:18:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676811.1053098; Tue, 06 Feb 2024 11:18:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJTb-0003yi-Lb; Tue, 06 Feb 2024 11:18:55 +0000
Received: by outflank-mailman (input) for mailman id 676811;
 Tue, 06 Feb 2024 11:18:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+UH4=JP=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXJTZ-0003xK-Oz
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 11:18:53 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 82476d33-c4e1-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 12:18:53 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-55fbbfbc0f5so853560a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 03:18:53 -0800 (PST)
Received: from [10.80.67.148] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 g23-20020aa7d1d7000000b0055f0f643ebcsm918583edp.96.2024.02.06.03.18.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 03:18:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82476d33-c4e1-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707218332; x=1707823132; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xs+Sg++ZA2uBkTdsuj9RsfrUwvJUfG9iNfmchaGUroA=;
        b=SonfRVi5SaGJy6cSFtxoMlUjHoa9TQmN9iHIzJOG5e15MabVu0wZ8bHeDjqfC0QXHm
         rx4GtG8l42ChZ9oE9z3FCX9FvbEDBbte6/9P2QJtThMiLnyI21Ltk7jff7G+ZCApHAh1
         9z8LdQooPAYPhkcCw1621Mv6SVUd+V9WuxqqM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707218332; x=1707823132;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xs+Sg++ZA2uBkTdsuj9RsfrUwvJUfG9iNfmchaGUroA=;
        b=cckxrfnbJIENTxj2Inm3F/usFU+1cQI3P6vSRPGSJ/4mALIRRUBY5KYnWNGL/SBkAp
         ryV5oaweSAOt7b6jGDTZJJXT7SC54XrzV7zI6jusM+V0D3ywumTFW2WzRkHKkuuhbUmP
         TvSrZp97NNekX7g7hlmXIe7UT3iW09Zag7rtM6Kikf99aWn2QcJDZBfqcyRdOVJ3UEVm
         yA6e3Cswo8TUaKupmNdY/vEmalaYUYStT1PkkzP8ga9MF0uR4bWfCSsJ8qF/glp4gw3o
         qglhlY1LDFoxI05u0SWHCOGbA/eOEeGxHQEXoctqCmpTLITXJyGFzZ1xzNKT+ZzHIBAu
         wtZw==
X-Gm-Message-State: AOJu0Yyf9pr5/zZILIRES9luVMwVv8IexTzdL7WdpAAVPmBoC3Do/cJO
	NcumzqNSB2xs0oqvVnfEvIadhjYY/ZY/KSzcBtGr8HivYwC/a+OeVnEztl4bEY1ZVrRUTEutGsP
	x
X-Google-Smtp-Source: AGHT+IFC33qOIq8bgLv/ZsPxxasn+ldEZOFZP75E4KalB2C3JpC5GiC8GQnk3YLBCl14cBaQZh3OvA==
X-Received: by 2002:a50:cd98:0:b0:55f:cb24:4242 with SMTP id p24-20020a50cd98000000b0055fcb244242mr8867440edi.4.1707218332381;
        Tue, 06 Feb 2024 03:18:52 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCU8zOGPLCRLTvDhENuflEjYvoGgbKmvZzPaNqDX7yoG8gKnl/lTOQHADMn7lxFYCKWLvZcQ4orMt6ZruD3cGvu5Utg3V6Jx7GNCc1hvD1HiWvR1xcfWdGArYDoh8NewBW1a14J3H4oX7kSxX00lLY2rdGv/WxhwAbss3YZxAfWz900+ytG1q9CdIiJ0bCmL3tzZTO209BGbgvvN7CKaHvxGxww=
Message-ID: <f7a16167-d876-4055-8ff8-c3dc8155030d@citrix.com>
Date: Tue, 6 Feb 2024 11:18:51 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen 4.19 release status tracking list
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: committers@xenproject.org, community.manager@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <CAMacjJzio2EDwudCUHoL6psKYztkHZOWwLNDknATvpN6Y0im=Q@mail.gmail.com>
 <7799755f-fad2-4f73-a2c1-de97bcce95fa@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <7799755f-fad2-4f73-a2c1-de97bcce95fa@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06/02/2024 9:24 am, Jan Beulich wrote:
> On 05.02.2024 20:23, Oleksii Kurochko wrote:
>>   * [PATCH v13 00/35] x86: enable FRED for x86-64 [
>> https://lore.kernel.org/xen-devel/20231205105030.8698-1-xin3.li@intel.com/
>> ]
> This is a kernel series. I don't expect FRED support is in scope for
> 4.19.

Correct - that's not a Xen series.

Xen FRED support is waiting on Intel to provide a version of SIMICS with
FRED and working shadow stack support.  I have no ETA.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 11:28:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 11:28:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676818.1053109 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJca-0005m0-KK; Tue, 06 Feb 2024 11:28:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676818.1053109; Tue, 06 Feb 2024 11:28:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJca-0005lt-GL; Tue, 06 Feb 2024 11:28:12 +0000
Received: by outflank-mailman (input) for mailman id 676818;
 Tue, 06 Feb 2024 11:28:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXJcZ-0005ln-RM
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 11:28:11 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cdf0dd83-c4e2-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 12:28:09 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33b4ac95664so15439f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 03:28:09 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 o28-20020adfa11c000000b0033b3ceda5dbsm1886233wro.44.2024.02.06.03.28.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 03:28:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cdf0dd83-c4e2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707218889; x=1707823689; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IgFVud1VtQo/0riAc4PR+zOQ4U+BxG/Cyu9S5J2Memg=;
        b=AXjgD8woZ4FTUQ0o9L+SuRLgQvB+3/4LElHYISIUi4dTxPNx15J2q9pho9ONOGLXKH
         89F4kR/3eQ4e4/2JN8245dU6GwJCAxx04r5c7ggOyXQNaHA6K0QztYbBQVxIUzWcPmtm
         v3nVdbjCcOvWPUgvsDx/J6Wgtaj9ytO+x6ZB0kXd7QIJ1lA6CzHg5tRFP53dcKdbN8+c
         kFu9Daz4v8itb998Y0OzlMJxChbgqjxGjos8RHkX4Ypikgi2YcinLsQTTaSZJt4mkamY
         upzHu533u8c+QyCwigwAg06ZiRwXR6X30SlU2QvnRxqIIGpPm503SMn6VCVclxstfTTZ
         KqEg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707218889; x=1707823689;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IgFVud1VtQo/0riAc4PR+zOQ4U+BxG/Cyu9S5J2Memg=;
        b=aUX4QLxNU1aFohlL7VwvXSuEyCUpk86DLJhrFRHKk/bl25GSSDN5bfqMRwGceGzGcI
         4TPvdzdVm9scE1jNb+QsO9aki6FYR+EEa3ju/TuJrmDsZZtB5dYYzU3YlGQbNUxJr7Ya
         87+wQJC1gR19/+62U1S75c2tNHR8C79lEG0hQJQLN5mLOItxOydWKVpkExLkt62gOtvH
         LRFSfy3B+PET2lqneh8B+tzs7ounBeQ+DmwSwLzbDeFid4qmm8/RMdL5Oxs3TRSSRvjC
         OuDIHK6pa00/WDdUmp6Aloc+wVv5RIN8PogSzytXHES4eiCRR8ANfMypHMCIflEIXVTk
         rW4g==
X-Gm-Message-State: AOJu0YyPGtMRjkPp04FMtJq84nVtO7z5p5qkobSaM1m3IjdpPbkLD9bL
	9amKgA72XmiHNKTxqBKZlKQNXVOQroGx54M8stucWIKV9ZtXAz2mwGKRj5UxNg==
X-Google-Smtp-Source: AGHT+IHNxLlaJEcrnfnaO9vXdYmsdm1sp5O2cxxriPLcKsijA3O1PFqHMr9wYozRIFJN9VXd8RXd5A==
X-Received: by 2002:a5d:4565:0:b0:33b:1786:f911 with SMTP id a5-20020a5d4565000000b0033b1786f911mr921685wrc.54.1707218888732;
        Tue, 06 Feb 2024 03:28:08 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWi10q8FyXP+VaIxn+OMzRAs1Ee8IpwpMidNQ7P5cLDzL5QR/W3uxr8n5DR7JDQeaJhH3DSmbGOHEjxTUMeLrmuV7LGsGov+Q2jSJp3HzY=
Message-ID: <90f8e183-5e78-49e8-ae2f-9898c1d7686b@suse.com>
Date: Tue, 6 Feb 2024 12:28:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/4] iommu/vt-d: switch to common RMRR checker
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240201170159.66330-1-roger.pau@citrix.com>
 <20240201170159.66330-4-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240201170159.66330-4-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.02.2024 18:01, Roger Pau Monne wrote:
> Use the newly introduced generic unity map checker.
> 
> Also drop the message recommending the usage of iommu_inclusive_mapping: the
> ranges would end up being mapped anyway even if some of the checks above
> failed, regardless of whether iommu_inclusive_mapping is set.

I'm afraid I don't understand this: When not in an appropriate E820
region, you now even fail IOMMU initialization. Shouldn't such
failure only occur when inclusive mappings weren't requested? At
which point referring to that option is still relevant?

Further to this failing - in patch 2 shouldn't the respective log
messages then be XENLOG_ERR, matching the earlier use of
AMD_IOMMU_ERROR()?

> --- a/xen/drivers/passthrough/vtd/dmar.c
> +++ b/xen/drivers/passthrough/vtd/dmar.c
> @@ -642,17 +642,9 @@ acpi_parse_one_rmrr(struct acpi_dmar_header *header)
>             return -EEXIST;
>         }
>  
> -    /* This check is here simply to detect when RMRR values are
> -     * not properly represented in the system memory map and
> -     * inform the user
> -     */
> -    if ( !e820_all_mapped(base_addr, end_addr + 1, E820_RESERVED) &&
> -         !e820_all_mapped(base_addr, end_addr + 1, E820_NVS) &&
> -         !e820_all_mapped(base_addr, end_addr + 1, E820_ACPI) )
> -        printk(XENLOG_WARNING VTDPREFIX
> -               " RMRR [%"PRIx64",%"PRIx64"] not in reserved memory;"
> -               " need \"iommu_inclusive_mapping=1\"?\n",
> -                base_addr, end_addr);
> +    if ( !iommu_unity_region_ok(maddr_to_mfn(base_addr),
> +                                maddr_to_mfn(end_addr + PAGE_SIZE)) )
> +        return -EIO;

Hmm, noticing only here, but applicable also to the earlier patch: The
"RMRR" (and there "IVMD") is lost, which removes some relevant context
information from the log messages. Can you add a const char* parameter
to the new helper, please?

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 11:45:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 11:45:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676826.1053119 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJtK-0000bG-1G; Tue, 06 Feb 2024 11:45:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676826.1053119; Tue, 06 Feb 2024 11:45:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJtJ-0000b9-Td; Tue, 06 Feb 2024 11:45:29 +0000
Received: by outflank-mailman (input) for mailman id 676826;
 Tue, 06 Feb 2024 11:45:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gRx7=JP=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rXJtI-0000b3-RL
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 11:45:28 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 380f825f-c4e5-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 12:45:26 +0100 (CET)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-33934567777so559587f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 03:45:26 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 y8-20020adff148000000b0033b40a3f92asm1922876wro.25.2024.02.06.03.45.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 06 Feb 2024 03:45:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 380f825f-c4e5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707219926; x=1707824726; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Tka5qtyLrYE27vOZvMawkfiBkGIW5D3nt1DAHLzndpU=;
        b=Vi46zJC/LO3B3Orj1M8bKwqpQTspwdX/MCrKmH3QlZV5oZeDgVOvJtJfDA2XQq5hZv
         VB8GEd3ybbrOoFXwx16gLT7WFizNSKiCFBZ7H2whhg3vZouMv06yvS8nSVEZmvE5Ekbl
         ZCD32TWGkn8PHbOz9dRU7a0U7pmRUTWsIwySM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707219926; x=1707824726;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Tka5qtyLrYE27vOZvMawkfiBkGIW5D3nt1DAHLzndpU=;
        b=T9UpbcCeus1BlGmf793kfRxlFo3Zdhp/2bq7hYVqn+ufk2L7ZzVhSZ4wNXQBC4rG2d
         K9Kj0SZaEFGP7mJ0x86Zq2g5JiyBeA9XqVJB0RDIIBvhmmH4Ei78Kg/IUbBtTr71lcHG
         /7ndG3RQQ0vBRKoAgVShNFunhAHpIZ7KFpIvhaOqZzyBx76WfFUVAGOCgZtzXmLZhDdU
         E4RKSCkEhMl/839k4ZVcH7T8f8wSzV0SZjau6sFz54zWlg8j2+gkk71PxFeUBtvS0Vbx
         XUOcm+yW5D73kDYDRz3UiE0kWpcld4SMFyxZFJkuKwX7E46qhqiMAQxdeiXFMKRMnI87
         diWw==
X-Gm-Message-State: AOJu0YzrGJ5v+fQwAzbzO/stgJp/8j8th8xVHXa1O+yQJCaIXgzJ+6KJ
	5R2xmtPljAfVzZCNmBNNfAt4wnP++zLI40zZwxtQ8oZcSZzjILDwQy1KlLLNDqo=
X-Google-Smtp-Source: AGHT+IFKvVsfnNXDXHhgHwWadUcH1NTA60yZQO+vuApHBTFW3qvVaiNBBIo8N57FuQo+ftuit0uCDQ==
X-Received: by 2002:a5d:4c8f:0:b0:33b:4649:9943 with SMTP id z15-20020a5d4c8f000000b0033b46499943mr1248257wrs.52.1707219926226;
        Tue, 06 Feb 2024 03:45:26 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCViLwz0+EKonMwihsmyiI1usf71NwAnC05iY5C+w3hyTs3np5UcY0JleRRLVc8hzo/hHcWhMtK4/NaL
Date: Tue, 6 Feb 2024 11:45:25 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
Subject: Re: [PATCH 1/4] block-common: Fix same_vm for no targets
Message-ID: <cd9e0194-5814-4735-bca4-df2577102c19@perard>
References: <20240201183024.145424-1-jandryuk@gmail.com>
 <20240201183024.145424-2-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240201183024.145424-2-jandryuk@gmail.com>

On Thu, Feb 01, 2024 at 01:30:21PM -0500, Jason Andryuk wrote:
> same_vm is broken when the two main domains do not have targets.  otvm
> and targetvm are both missing, which means they get set to -1 and then
> converted to empty strings:
> 
> ++10697+ local targetvm=-1
> ++10697+ local otvm=-1
> ++10697+ otvm=
> ++10697+ othervm=/vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4
> ++10697+ targetvm=
> ++10697+ local frontend_uuid=/vm/844dea4e-44f8-4e3e-8145-325132a31ca5
> 
> The final comparison returns true since the two empty strings match:
> 
> ++10697+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o '' = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = '' -o '' = '' ']'
> 
> Replace -1 with distinct strings indicating the lack of a value and
> remove the collescing to empty stings.  The strings themselves will no
> longer match, and that is correct.
> 
> ++12364+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o 'No target' = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = 'No other target' -o 'No target' = 'No other target' ']'
> 
> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 11:46:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 11:46:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676830.1053129 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJuk-00016z-9n; Tue, 06 Feb 2024 11:46:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676830.1053129; Tue, 06 Feb 2024 11:46:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJuk-00016s-6j; Tue, 06 Feb 2024 11:46:58 +0000
Received: by outflank-mailman (input) for mailman id 676830;
 Tue, 06 Feb 2024 11:46:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JoWx=JP=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rXJui-000164-Su
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 11:46:56 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6c90067a-c4e5-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 12:46:54 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-55a90a0a1a1so6950090a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 03:46:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c90067a-c4e5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1707220014; x=1707824814; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=p8/oMxMR+vfjnyX3RtwcY6FGZAgbqksWkgFNdCxGN/Y=;
        b=EQ5WZs7JcdFtBZNvzSmBX3/hRBHC5C5LACW2/LhhBs60kqkT0pOsfvTzvFewiRk82I
         7wcGoT1vTHISg9DvgLoAptggzDwSJOmStroas1ECsUS4ZJH1Avn2VvI+gyH2yCheouMK
         zyKqD1n09LWklp6KfsvQP+Ms9mWm4+d9B11N78pM57W0q9g38UACGTMKhlctLQbdQRkQ
         QNo8liJpYoGnyhoKArVB0gm9FlvxvcUCKd0i5p6cQd3R7WRCRj87uJo04KknBLW/jfG+
         8dKYuRHgdpbQMsSvZEtFQ1fWsaTQFQLQBBUX6X4n2C056wMLAeMwywk2IcIhnKaCI8xz
         YPqA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707220014; x=1707824814;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=p8/oMxMR+vfjnyX3RtwcY6FGZAgbqksWkgFNdCxGN/Y=;
        b=aegREF0jhLUCT4TrVXv6Oc2FM8SexKXyMB+Q0kb7QaJfoEgtLaox0tavPiQNKJrgRu
         iWlOwbLAPtZqepxg7Vi76cBN2K9jx9xpLDzbjtsj6uWTcfsFWYSlAVgOI2B7K9lYpbIS
         7NyJW5WAR+GieKTcCL0WvJB7bECvWjM7vieuLkspCfFZueIjSsPUNjtOXmLXlS1H24ep
         XJN5ppl4nWpX75CVTTw8T8E/z55WX5aeYO2Xq9Z3Z9eKK8+K7Su59E9qOntU9GmeJTeB
         rLEZ0BZGs0M8O6nHHHI3IsMYFhr7Sqw/FvySkcAN5nWdlMHEt96lJjFQ3b1j8u2IvVIg
         7qUQ==
X-Gm-Message-State: AOJu0Yy77FyB//GWJAAraFkpyPy24V97PtlwBB7iob+NifFyDWi5LBDF
	jyGevRe4tOdP5jzqFLVMEkjhq87hhKz2DXR2cn+310zN7v8GylgVrcy+NpyNkDqPOXaCHMNgZF3
	Y0kZffhm2g2nmNuPsGwhUblC/11i+eJDMwlnVOw==
X-Google-Smtp-Source: AGHT+IG6CgiBWyW3Ari4+zP/yA0q67PInJczVKi7aOcOSsudE+rjN7rZMBSBjfm339oApfWkXUMJqgcEExwDjHTI9qQ=
X-Received: by 2002:aa7:d643:0:b0:55f:fba9:de8c with SMTP id
 v3-20020aa7d643000000b0055ffba9de8cmr1418413edr.21.1707220014002; Tue, 06 Feb
 2024 03:46:54 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-6-carlo.nonato@minervasys.tech> <49827753-14f1-42cc-8791-27c5400e6e50@suse.com>
In-Reply-To: <49827753-14f1-42cc-8791-27c5400e6e50@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Tue, 6 Feb 2024 12:46:43 +0100
Message-ID: <CAG+AhRX82h40t53LB2h91A6N0fCVafyaCGQn7Pwmn6GZ+RM+mQ@mail.gmail.com>
Subject: Re: [PATCH v6 05/15] xen: extend domctl interface for cache coloring
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Thu, Feb 1, 2024 at 2:51=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 29.01.2024 18:18, Carlo Nonato wrote:
> > @@ -858,6 +859,16 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t=
) u_domctl)
> >                  __HYPERVISOR_domctl, "h", u_domctl);
> >          break;
> >
> > +    case XEN_DOMCTL_set_llc_colors:
> > +        if ( !llc_coloring_enabled )
> > +            break;
>
> With "ret" still being 0, this amounts to "successfully ignored". Ought
> to be -EOPNOTSUPP, I guess.
>
> > +        ret =3D domain_set_llc_colors_domctl(d, &op->u.set_llc_colors)=
;
> > +        if ( ret =3D=3D -EEXIST )
> > +            printk(XENLOG_ERR
> > +                   "Can't set LLC colors on an already created domain\=
n");
>
> If at all a dprintk(). But personally I think even that's too much - we
> don't do so elsewhere, I don't think.
>
> > --- a/xen/common/llc-coloring.c
> > +++ b/xen/common/llc-coloring.c
> > @@ -4,6 +4,7 @@
> >   *
> >   * Copyright (C) 2022 Xilinx Inc.
> >   */
> > +#include <xen/guest_access.h>
> >  #include <xen/keyhandler.h>
> >  #include <xen/llc-coloring.h>
> >  #include <xen/param.h>
> > @@ -229,6 +230,30 @@ int __init dom0_set_llc_colors(struct domain *d)
> >      return domain_check_colors(d);
> >  }
> >
> > +int domain_set_llc_colors_domctl(struct domain *d,
> > +                                 const struct xen_domctl_set_llc_color=
s *config)
>
> What purpose has the "domctl" in the function name?
>
> > +{
> > +    unsigned int *colors;
> > +
> > +    if ( d->num_llc_colors )
> > +        return -EEXIST;
> > +
> > +    if ( !config->num_llc_colors )
> > +        return domain_set_default_colors(d);
> > +
> > +    colors =3D alloc_colors(config->num_llc_colors);
> > +    if ( !colors )
> > +        return -ENOMEM;
>
> Hmm, I see here you call the function without first having bounds checked
> the input. But in case of too big a value, -ENOMEM is inappropriate, so
> such a check wants adding up front anyway.
>
> > +    if ( copy_from_guest(colors, config->llc_colors, config->num_llc_c=
olors) )
> > +        return -EFAULT;
>
> There again wants to be a check that the pointed to values are the same,
> or at least of the same size. Else you'd need to do element-wise copy.

Sorry to bring this back again, but I've just noticed copy_from_guest()
already checks for type compatibility. For what regards the size I don't th=
ink
I understood what to check. colors is defined to be of the same size of
config->llc_colors.

Thanks.

> > +    d->llc_colors =3D colors;
> > +    d->num_llc_colors =3D config->num_llc_colors;
> > +
> > +    return domain_check_colors(d);
>
> And if this fails, you leave the domain with the bad settings? Shouldn't
> you check and only then store pointer and count?
>
> > --- a/xen/include/public/domctl.h
> > +++ b/xen/include/public/domctl.h
> > @@ -1190,6 +1190,13 @@ struct xen_domctl_vmtrace_op {
> >  typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
> >  DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
> >
> > +struct xen_domctl_set_llc_colors {
> > +    /* IN LLC coloring parameters */
> > +    uint32_t num_llc_colors;
> > +    uint32_t padding;
>
> I see you've added padding, but: You don't check it to be zero. Plus
> the overwhelming majority of padding fields is named "pad".
>
> Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 11:49:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 11:49:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676835.1053139 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJwv-0002B8-LB; Tue, 06 Feb 2024 11:49:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676835.1053139; Tue, 06 Feb 2024 11:49:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJwv-0002B1-I5; Tue, 06 Feb 2024 11:49:13 +0000
Received: by outflank-mailman (input) for mailman id 676835;
 Tue, 06 Feb 2024 11:49:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXJwu-0002Av-7h
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 11:49:12 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bd133a91-c4e5-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 12:49:09 +0100 (CET)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-33b466bc363so360090f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 03:49:10 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w17-20020a5d6811000000b0033afcc899c1sm1915771wru.13.2024.02.06.03.49.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 03:49:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd133a91-c4e5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707220149; x=1707824949; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/eIw2pgQ2mmhwfkUOy8UoWwbTRbjlat4j7eQPYE7Jeo=;
        b=gNRPyy+IzwNTcnuZqwPtiYbpg+KdK1ezKhqjkafUGrzsU0TpB0KfGICdMbLsqdHaiC
         /pC10TguFwHRTYidglf8eKWKH6VvbEV41qBvy+lNMMWeo4n0WgpnoOEXVghduYFuclGl
         BINfsaNzUrHcpItrwRNDbpnhYwCphtAatWCGvs9Ydj97jANAabf/0FoWTbzJ/FByYQiL
         vnu3Ef3IVi38Vh5VQepadScBzVTKhTyH5pwXPUOHnBBuht4DxYcg1tzKUaXvnjnSFBvR
         OO/Vk/5/xyYzdtqwWfGgb8XmiPtScWrDb1Erqtbv71YsZFPZWIN56jHku1WPzjX8Me9O
         oaFg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707220149; x=1707824949;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/eIw2pgQ2mmhwfkUOy8UoWwbTRbjlat4j7eQPYE7Jeo=;
        b=NB1rTtK+DulPArLjRDXgBJ2UiShfldtfe1C5Qw1CBvumCkKaKdWTGJZ2EXU2D3a5xZ
         3VnSaVkNphHWO//1QPc76g3VsHGuiBaK+/snj/LrLM6yEPm7IVWpZUC2JsZUBV0jHh7a
         Z0YbtllT5q/JFzqf0TfLOvhmDXJL8JdnDcm2o78lIkqNWRFE54W17G0ESvmkcnMQUeir
         cYqmRDOQBShkU2jv1qy85gihkrBoOGzL4VBd753az9TBnEt+zm8fxmaJcRnU5+OS2+br
         014/vfZPOpS82MhM/fJ8FQwkVU1c3eqKvSeheBGH/dt8utRp/xheM0dVrSHhpi2ymDpZ
         VfXA==
X-Gm-Message-State: AOJu0Yzx7/zf9o2KXGBqlMVSaSCVD4S+9pAHE6SQPqAHNNiIyqEF69ud
	MXRi5jB3bcU+h0YqetDOWTd0svsz5JRKHdeG3Ei5ykdXTtWs5jNXCiwJk6FIeA==
X-Google-Smtp-Source: AGHT+IFRD2gWEvd0gMa6ZYAJ6KqPQy/HtW5u9R+oB+TjXj+dWBgctvilTvaHWmeSv7OjXhhFjhKbUQ==
X-Received: by 2002:a5d:4f0e:0:b0:33b:2633:b527 with SMTP id c14-20020a5d4f0e000000b0033b2633b527mr1046884wru.20.1707220149418;
        Tue, 06 Feb 2024 03:49:09 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWNik0RgHRVebarSf9EAdgtO4ko4pv9kSpgVkEYJlJhn8FjzBq4bvDXC3Zf7rhtShNk+jttx316BhLWon+k5qKjOsnitg0tAHTM4fe5wXU=
Message-ID: <5db22ee1-40b3-4df8-88b8-446a2e224d22@suse.com>
Date: Tue, 6 Feb 2024 12:49:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/4] iommu/x86: make unity range checking more strict
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
References: <20240201170159.66330-1-roger.pau@citrix.com>
 <20240201170159.66330-5-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240201170159.66330-5-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 01.02.2024 18:01, Roger Pau Monne wrote:
> Currently when a unity range overlaps with memory being used as RAM by the
> hypervisor the result would be that the IOMMU gets disabled.  However that's
> not enough, as even with the IOMMU disabled the device will still access the
> affected RAM areas.

Hmm, no, I think this is going too far. Not the least because it is
s/will/may/. But also because if we really wanted such behavior, we
ought to also parse the respective ACPI tables when the "iommu=off".

> Note that IVMD or RMRR ranges being placed over RAM is a firmware bug.

As written this is wrong: They're typically in RAM, just that the E820
type for that range should not be RAM_TYPE_CONVENTIONAL.

> Doing so also allows to simplify the code and use a switch over the reported
> memory type(s).

I'm afraid this isn't right either: page_get_ram_type() can set
multiple bits in its output.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 11:51:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 11:51:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676840.1053148 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJzH-0003dc-2y; Tue, 06 Feb 2024 11:51:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676840.1053148; Tue, 06 Feb 2024 11:51:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXJzH-0003dV-0L; Tue, 06 Feb 2024 11:51:39 +0000
Received: by outflank-mailman (input) for mailman id 676840;
 Tue, 06 Feb 2024 11:51:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXJzF-0003dM-Vr
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 11:51:37 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 140842cb-c4e6-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 12:51:35 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-33b466bc363so361375f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 03:51:35 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e1-20020a5d65c1000000b0033b3c1200f0sm1928689wrw.10.2024.02.06.03.51.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 03:51:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 140842cb-c4e6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707220295; x=1707825095; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=mT96k+bHjcM/gATgja6hrTTsgje/0DnO63CWpafRJbA=;
        b=QUlxS7cHgSCZ8lnteXlmLN1WnmHYU8Hs5dlhXOeGAARUgEj3djXSK5rsAivNOWGgb9
         FM4HpguyQIT+Pw0Iu08s9QR/ItpUXImOhwp/+WoTSzlH/SBeyKjp7RnnMjC67Y+0beM4
         1bEBaz5GiOALv1FNhwIy1TWTFdgQ87RfIVHheooDxLIO5WZmz6LTPbg9wzVtjhBuoqVJ
         RoCeRwkJ2dYYt+H4ScD5HC4YGfEX/Q9AfASBkCP2UC1S902RZVRSZbTZzebQ2HkkXgNP
         CgZXtU+SsKSMfZKTgRyhL2qX8j9031whrclZPPYaaTsu5pkr1WAsy9N1Y94UCfNByyKJ
         6IJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707220295; x=1707825095;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=mT96k+bHjcM/gATgja6hrTTsgje/0DnO63CWpafRJbA=;
        b=VrMPEKOExt40uwSSjdUyoJfhGEU5kWE6YI4TbU1IOH+Kyxvu88bOmQznIlbM3crcKU
         3oi2bkuhXwhFq7Dhklga9e9WLXWaKlI9rW6PATGp473aPQlwAZZ6wMVv2PoxV5JRiXk3
         ffJ8nwc1XdD4Alj1hVBU85O7Di0BVAmgnrSg2RMHYeNU4qdj5ZvdGFyKLWZOsWT5JIiJ
         GnsM9Dvzt8qov/YhKpNbcC4zr9EAVk3pxdTVOlYElZHFC4302mwyMHdnUXBS0ke2DQZ9
         So0wDRQxQXC0JeVZcgb+TKZy8OsrijbyCuhZo6hMptwcz48/RNFJJ9HK12e7ETCB3FRJ
         8U/g==
X-Gm-Message-State: AOJu0Yx6/4Gn7P2HG/4BkbLdXoHCHelxFDBFAEHUCDZHjADZz2atsQkW
	kCEuGedmJYp4kzr5xFylIfYAL/InNFzVLogwyBgy3j6IGRL+hvPLSodZKZaj3w==
X-Google-Smtp-Source: AGHT+IEthtXhD9ibDWKoo4KMGOoDkFKUukxDvbKKpklgDxWZbBrql0KgdTfxcNxgwt6ymbvWOZaZ6w==
X-Received: by 2002:adf:ce90:0:b0:33b:44b4:da60 with SMTP id r16-20020adfce90000000b0033b44b4da60mr1158557wrn.41.1707220295303;
        Tue, 06 Feb 2024 03:51:35 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUCB7if+BCSHCrbaEOEeIUfmKXCECRozR23DUnNiXL0R+E6NNdboC/n9+dRIR4Dvv+ap8/Vl9mBtFNYylkFdJ1MjpecVzdRZhA2uPjYkyhzSaPoDlB8A23p3xa0MlP7xVp0I7dcjVkeNwD38h5Zm4lxBsCZO8l+JjNbWxJl9FhtHwRRQbVBRXV9Xi2r7keFb7THfUXdIwspfc3HeRpZo5Bldl0sY19bT2YwChcUA/YCag+10vdP/gqrJUz9do52uGokt9wINBUsRifm7CEL2CZ/9JoObXd2
Message-ID: <baec35bd-fb96-41de-b3de-2158b53ad08a@suse.com>
Date: Tue, 6 Feb 2024 12:51:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 05/15] xen: extend domctl interface for cache coloring
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Marco Solieri <marco.solieri@minervasys.tech>,
 xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-6-carlo.nonato@minervasys.tech>
 <49827753-14f1-42cc-8791-27c5400e6e50@suse.com>
 <CAG+AhRX82h40t53LB2h91A6N0fCVafyaCGQn7Pwmn6GZ+RM+mQ@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRX82h40t53LB2h91A6N0fCVafyaCGQn7Pwmn6GZ+RM+mQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06.02.2024 12:46, Carlo Nonato wrote:
> On Thu, Feb 1, 2024 at 2:51 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>> @@ -229,6 +230,30 @@ int __init dom0_set_llc_colors(struct domain *d)
>>>      return domain_check_colors(d);
>>>  }
>>>
>>> +int domain_set_llc_colors_domctl(struct domain *d,
>>> +                                 const struct xen_domctl_set_llc_colors *config)
>>
>> What purpose has the "domctl" in the function name?
>>
>>> +{
>>> +    unsigned int *colors;
>>> +
>>> +    if ( d->num_llc_colors )
>>> +        return -EEXIST;
>>> +
>>> +    if ( !config->num_llc_colors )
>>> +        return domain_set_default_colors(d);
>>> +
>>> +    colors = alloc_colors(config->num_llc_colors);
>>> +    if ( !colors )
>>> +        return -ENOMEM;
>>
>> Hmm, I see here you call the function without first having bounds checked
>> the input. But in case of too big a value, -ENOMEM is inappropriate, so
>> such a check wants adding up front anyway.
>>
>>> +    if ( copy_from_guest(colors, config->llc_colors, config->num_llc_colors) )
>>> +        return -EFAULT;
>>
>> There again wants to be a check that the pointed to values are the same,
>> or at least of the same size. Else you'd need to do element-wise copy.
> 
> Sorry to bring this back again, but I've just noticed copy_from_guest()
> already checks for type compatibility. For what regards the size I don't think
> I understood what to check. colors is defined to be of the same size of
> config->llc_colors.

Oh, you're right. But the other case was a memcpy(), wasn't it?

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 11:57:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 11:57:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676848.1053158 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXK5H-0004OV-Jr; Tue, 06 Feb 2024 11:57:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676848.1053158; Tue, 06 Feb 2024 11:57:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXK5H-0004OO-H2; Tue, 06 Feb 2024 11:57:51 +0000
Received: by outflank-mailman (input) for mailman id 676848;
 Tue, 06 Feb 2024 11:57:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXK5F-0004OE-W5; Tue, 06 Feb 2024 11:57:50 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXK5F-0001cY-SV; Tue, 06 Feb 2024 11:57:49 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXK5F-0004Iu-En; Tue, 06 Feb 2024 11:57:49 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXK5F-000682-EL; Tue, 06 Feb 2024 11:57:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=EfsWaFnNOwQqPKi19KQOat83X4jntJfvzutvd911vvw=; b=Tax1GryZnXZAGl11p10tbeosKQ
	yxNASnbbt9qId4kyh0iBdEYVBMvk55MLgW0CIW3qQ5Jpes5hNMaap8PBXP7CDV4O2a96iqyV3donM
	UUpWg6sg+3MiqjEZJXf6fK0zSv2PfVsLBrnSPWZ4q3BybM+Y/eWb0WSmMBSpRVKiaG5A=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184603-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184603: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=ae59b8ba4166384cbfa32a921aac289bcff2aef9
X-Osstest-Versions-That:
    ovmf=056b4bf74bf5cfa2c0c9c5e44a7c5f3f461d17c0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 06 Feb 2024 11:57:49 +0000

flight 184603 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184603/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 ae59b8ba4166384cbfa32a921aac289bcff2aef9
baseline version:
 ovmf                 056b4bf74bf5cfa2c0c9c5e44a7c5f3f461d17c0

Last test of basis   184587  2024-02-04 13:41:21 Z    1 days
Testing same since   184603  2024-02-06 09:15:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dun Tan <dun.tan@intel.com>
  Zhou Jianfeng <jianfeng.zhou@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   056b4bf74b..ae59b8ba41  ae59b8ba4166384cbfa32a921aac289bcff2aef9 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 12:01:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 12:01:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676858.1053169 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXK8S-0006Y1-Cr; Tue, 06 Feb 2024 12:01:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676858.1053169; Tue, 06 Feb 2024 12:01:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXK8S-0006Xu-9u; Tue, 06 Feb 2024 12:01:08 +0000
Received: by outflank-mailman (input) for mailman id 676858;
 Tue, 06 Feb 2024 12:01:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JoWx=JP=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rXK8R-0006Xo-Nm
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 12:01:07 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 67da6379-c4e7-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 13:01:06 +0100 (CET)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-56025fcaebaso3369221a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 04:01:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67da6379-c4e7-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1707220865; x=1707825665; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=1pHEquF62EFhSDTVVJ/5iuMaY7D/+BBlsEZFu8/1icw=;
        b=3dUjN5+z4Dd1DgWs5QKjiL/8ZnDYPUxqrs2MdO4W5dhp3GNw6mrtzJDk3UKTwL8y2n
         AKvFMYImwUN0aaSB1RT3PZJcqLvxiSvSgTA5pYlPAo0qHhyQk7UpDNscEh28B4CF+V5W
         +zgkBaIPYUVDNckgsqvGjaIw83nc1POx2Rgy7MB2k/89bF0bhIxMVIyX6FBted6b8vXv
         0CAHEwM1dOGYjB2RxS+B7erz+OCyFUIKkksXx1IN2fHM0ln+cEKX6J7BI/khuD2Vpxxb
         z8jYEeeq7TUt4xJ3IXx1J+ZcGaFxMgggocRCpmajZ0o+hwpLibht8UDIDk7OS5uAfLLr
         fFAA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707220865; x=1707825665;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=1pHEquF62EFhSDTVVJ/5iuMaY7D/+BBlsEZFu8/1icw=;
        b=LNR2iecyet7EgLkWCSI624JYSW1y/A0CInXL9mss82LiWKy3zIH8OCdU/m4HIUals3
         KL5jziCaKFwcv6Lnzukas2Gh/JhH0P+PnaD1rwjVamI4zBmhB6WMphLFjNquc0NnisQH
         jJuVpJEc9QuVXugaeO7G7v5mvfuu7sbQosLcW4r8u9W77sBORmVN6q44ScE44hckwG/D
         sB/csQ5mfDwkJ/mYbswkouqVTRXvwYEJkitclNkxVksr9lRenyUiMXgqwN5eLTdnhVoG
         u+vFVnwEmRtN3QhtwCmjYoe1d3Vzdb+qSh/3NJkU4eD5x/+SCW+ZrGopTMGoyvIwcYEj
         zZSg==
X-Gm-Message-State: AOJu0YxZJG1ljw5AYF3zCIJKKmSR6xMI7U10/yzo7GWX4FDy9ufzmzRK
	EOv7Eaipch8EC+F6THrNeE+5HbPc7lLDRC9qJRxj+N3skwSA1n6JQYF7186JbJ+FnB8+OM3pZpg
	Be9n/f2cIY7ZX8JOHn2aSPMqlYbbaE9Z/OGXMtOi2qzz+tYJtjz7dfQ==
X-Google-Smtp-Source: AGHT+IFoPtx73AYtcrcJ+xwtDoyPIevOlSV3JZNIHMLTbaB6Am0yKpHJ2X/GcHhqGl5Tc69EHOi0qEfBMQvRpbei4v0=
X-Received: by 2002:a05:6402:1203:b0:560:64f4:cbd5 with SMTP id
 c3-20020a056402120300b0056064f4cbd5mr1521208edw.21.1707220865041; Tue, 06 Feb
 2024 04:01:05 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-6-carlo.nonato@minervasys.tech> <49827753-14f1-42cc-8791-27c5400e6e50@suse.com>
 <CAG+AhRX82h40t53LB2h91A6N0fCVafyaCGQn7Pwmn6GZ+RM+mQ@mail.gmail.com> <baec35bd-fb96-41de-b3de-2158b53ad08a@suse.com>
In-Reply-To: <baec35bd-fb96-41de-b3de-2158b53ad08a@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Tue, 6 Feb 2024 13:00:54 +0100
Message-ID: <CAG+AhRVFTBnAbmRQXN6FNdRd3_=P1FxNxmvL7-0xy0n5m37B0A@mail.gmail.com>
Subject: Re: [PATCH v6 05/15] xen: extend domctl interface for cache coloring
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 6, 2024 at 12:51=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 06.02.2024 12:46, Carlo Nonato wrote:
> > On Thu, Feb 1, 2024 at 2:51=E2=80=AFPM Jan Beulich <jbeulich@suse.com> =
wrote:
> >> On 29.01.2024 18:18, Carlo Nonato wrote:
> >>> @@ -229,6 +230,30 @@ int __init dom0_set_llc_colors(struct domain *d)
> >>>      return domain_check_colors(d);
> >>>  }
> >>>
> >>> +int domain_set_llc_colors_domctl(struct domain *d,
> >>> +                                 const struct xen_domctl_set_llc_col=
ors *config)
> >>
> >> What purpose has the "domctl" in the function name?
> >>
> >>> +{
> >>> +    unsigned int *colors;
> >>> +
> >>> +    if ( d->num_llc_colors )
> >>> +        return -EEXIST;
> >>> +
> >>> +    if ( !config->num_llc_colors )
> >>> +        return domain_set_default_colors(d);
> >>> +
> >>> +    colors =3D alloc_colors(config->num_llc_colors);
> >>> +    if ( !colors )
> >>> +        return -ENOMEM;
> >>
> >> Hmm, I see here you call the function without first having bounds chec=
ked
> >> the input. But in case of too big a value, -ENOMEM is inappropriate, s=
o
> >> such a check wants adding up front anyway.
> >>
> >>> +    if ( copy_from_guest(colors, config->llc_colors, config->num_llc=
_colors) )
> >>> +        return -EFAULT;
> >>
> >> There again wants to be a check that the pointed to values are the sam=
e,
> >> or at least of the same size. Else you'd need to do element-wise copy.
> >
> > Sorry to bring this back again, but I've just noticed copy_from_guest()
> > already checks for type compatibility. For what regards the size I don'=
t think
> > I understood what to check. colors is defined to be of the same size of
> > config->llc_colors.
>
> Oh, you're right. But the other case was a memcpy(), wasn't it?

Yes, in that case your suggestion was needed.

Thanks again.

> Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 12:04:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 12:04:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676866.1053179 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKBZ-0007Dd-R5; Tue, 06 Feb 2024 12:04:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676866.1053179; Tue, 06 Feb 2024 12:04:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKBZ-0007DW-Nt; Tue, 06 Feb 2024 12:04:21 +0000
Received: by outflank-mailman (input) for mailman id 676866;
 Tue, 06 Feb 2024 12:04:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXKBY-0007DQ-J0
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 12:04:20 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id db59c915-c4e7-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 13:04:19 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-40ff13d2628so1256205e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 04:04:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 y6-20020a056000108600b0033b48190e5esm1142771wrw.67.2024.02.06.04.04.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 04:04:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: db59c915-c4e7-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707221059; x=1707825859; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PDFsRplueq1dCGsrwap+oi9L81OYSkGPgI5OtHRQPt0=;
        b=CZQIhkxYXOTszJeui2CmaibiyS+SfEU6tv8Kj3LKWaU7d0/zIfKnxGgzcICfOwLEvl
         EN1VZb+Oxe+K2lQeqWbpRg8AFyOy+Lx8I+yesmlsh/8Bk6LC0xl8P8DPx4C0GGwBsKkw
         Gx1hLaAo/FlGxa0DGx5Rses0T8epfbhcp1BCjEveR9gX2ZBq1V/6P4kOGTEqOBOM93il
         bQ7k+GtGkF28pRQ5xmNS0bTTpHfD1gyapQKsyaKbKl2SJsj0LpcZDv3BpFwyg0BYv+eC
         t8PfdIbOQISn4P2shO52v4Vde1im46YNzNfKICXVBkpMAWE/1TICGDl4liXI2aD/+Z6v
         iObA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707221059; x=1707825859;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PDFsRplueq1dCGsrwap+oi9L81OYSkGPgI5OtHRQPt0=;
        b=wBiaz+DKk481CqwY03mQsE/tcwrN8elvjnYo8geuth5DThALo9YZYjJd/4itrNAvC6
         +L2/T1EOyFIs5iSYu920yIrqC+6uGTYiytT8oSW6s1ODm9juSzp8stc9CfvyTPBGtKVL
         MF2vzrxMpugSwLxxgx1Y3SEjAhz02Ey/gBwUpk3OCrzU0KtdaGWJdHiHlVbHJsEBGlg5
         i6/9ifhIl82dK0BISZa+xagPOafue0pCrgXqM6kEMSbVbTJfunlYZ2BM9P2G+G2NdWkg
         6DAyWfGDX1XZx/e+6uRnv6VhXk1TP+ZWcp9YcGbb4Sm2GdzhUlu9SOc8B/JkemxDheWs
         vsFg==
X-Gm-Message-State: AOJu0Yw6jp4ZR9WlmNPgk3BSaht3M6GJ4CWrebDfIFNv+0qVKqEiDx0Q
	jHBXuIeccUugBo/lfguoR1xXzIbdd/iNkVKPRDJuMvAEZH5L1Tgdw+CEL6dx5g==
X-Google-Smtp-Source: AGHT+IHZeUbwUVREyRUvntKsEr26KtuNTB2oDglTFws6gyR+jv0KW8pkgoRE6ChZyZAKmwIlyvLqag==
X-Received: by 2002:a05:600c:4e0d:b0:40f:f1ab:f71d with SMTP id b13-20020a05600c4e0d00b0040ff1abf71dmr273992wmq.9.1707221058801;
        Tue, 06 Feb 2024 04:04:18 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCU0IdNNHHTkOQXjoukpKfTL3geEvGtI6Z8DJMdl5z0Wh2e80IYthpG8rkQyUUhFKmwCzN8ieZ5QjTADUgXi5HsjhCpKNYT20pz59THQPbYhGmawwpX0QLJW7nLxJbulzPx4STq8iv1Ao+TtSfeIAf7eWkM2B90cuRNJsT7KH3v4KIR5qWHvNW7g2ZR8gAsOg24VOcwjqAAgwMfGRuG6tTVYNUIMR5/4zdoXnMPmRBrG4cdC0dMXGu414YgUAQ4+hxca5LfUHak8dLB6oRGRQPCtfNgDutNxleiiD+fQwxKyk2OHDJi8d0qLXrRWno3FBdCaKdowXFoYQj6SFkXXYOUK3+7qhS9JXTm9uP5VCzY/afyBdyCuaGIUovzFXMCfo4//qxhKPLzPQ00bwfPAXpIh+bA9mQrBCt7/iYGOn3OZLoy5J708y0bRFSwXjvOw2Jw=
Message-ID: <09869431-ed00-4102-93fd-84c697a5c950@suse.com>
Date: Tue, 6 Feb 2024 13:04:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and
 logging effects
Content-Language: en-US
To: Simone Ballarin <simone.ballarin@bugseng.com>
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 16:16, Simone Ballarin wrote:
> Rule 13.1: Initializer lists shall not contain persistent side effects
> 
> Effects caused by debug/logging macros and functions (like ASSERT, __bad_atomic_size,
> LOG, etc ...) that crash execution or produce logs are not dangerous in initializer
> lists. The evaluation order in abnormal conditions is not relevant. Evaluation order
> of logging effects is always safe.

I thought I said so before: When talking of just logging, evaluation order
may very well have a impact on correctness. Therefore we shouldn't mix
debugging and logging.

> Function hvm_get_guest_tsc_fixed (indirectly) performs different side effects.
> For example it calls hvm_get_guest_time_fixed that contains an ASSERT and calls
> to spin_lock and spin_unlock.
> 
> These side effects are not dangerous: they can be executed regardless of the
> initializer list evaluation order
> 
> This patch deviates violations using SAF commits caused by debug/logging macros and
> functions.

DYM "comments"?

> --- a/xen/arch/arm/device.c
> +++ b/xen/arch/arm/device.c
> @@ -331,6 +331,7 @@ int handle_device(struct domain *d, struct dt_device_node *dev, p2m_type_t p2mt,
>          .p2mt = p2mt,
>          .skip_mapping = !own_device ||
>                          (is_pci_passthrough_enabled() &&
> +                        /* SAF-3-safe effects for debugging/logging reasons are safe */
>                          (device_get_class(dev) == DEVICE_PCI_HOSTBRIDGE)),

Taking this just as example: I think the comment is too long. Just
saying (leaving aside my comment higher up) "debugging/logging"
would imo be sufficient.

> --- a/xen/arch/arm/guestcopy.c
> +++ b/xen/arch/arm/guestcopy.c
> @@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf, uint64_t addr, unsigned int len,
>  unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
>  {
>      return copy_guest((void *)from, (vaddr_t)to, len,
> -                      GVA_INFO(current), COPY_to_guest | COPY_linear);
> +                      /* SAF-4-safe No persistent side effects */
> +                      GVA_INFO(current),

I _still_ think this leaves ambiguity. The more that you need to look
up GVA_INFO() to recognize what this is about.

> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -800,6 +800,7 @@ static int cf_check hvm_save_cpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
>  {
>      struct segment_register seg;
>      struct hvm_hw_cpu ctxt = {
> +        /* SAF-3-safe effects for debugging/logging reasons are safe */
>          .tsc = hvm_get_guest_tsc_fixed(v, v->domain->arch.hvm.sync_tsc),

A prereq for this imo is that the function take const struct vcpu *.
But I'm not sure that'll suffice. The function can change at any time,
rendering the comment here stale perhaps without anyone noticing.

> --- a/xen/common/sched/core.c
> +++ b/xen/common/sched/core.c
> @@ -1521,6 +1521,7 @@ long vcpu_yield(void)
>  
>      SCHED_STAT_CRANK(vcpu_yield);
>  
> +    /* SAF-4-safe No persistent side effects */
>      TRACE_2D(TRC_SCHED_YIELD, current->domain->domain_id, current->vcpu_id);
>      raise_softirq(SCHEDULE_SOFTIRQ);
>      return 0;
> @@ -1899,6 +1900,7 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>          if ( copy_from_guest(&sched_shutdown, arg, 1) )
>              break;
>  
> +        /* SAF-4-safe No persistent side effects */
>          TRACE_3D(TRC_SCHED_SHUTDOWN,
>                   current->domain->domain_id, current->vcpu_id,
>                   sched_shutdown.reason);
> @@ -1916,6 +1918,7 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>          if ( copy_from_guest(&sched_shutdown, arg, 1) )
>              break;
>  
> +        /* SAF-4-safe No persistent side effects */
>          TRACE_3D(TRC_SCHED_SHUTDOWN_CODE,
>                   d->domain_id, current->vcpu_id, sched_shutdown.reason);
>  

For all of these iirc the suggestion was to latch current into a local
variable (named "curr" by convention).

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 12:06:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 12:06:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676870.1053189 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKDV-0007jy-6I; Tue, 06 Feb 2024 12:06:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676870.1053189; Tue, 06 Feb 2024 12:06:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKDV-0007jr-2q; Tue, 06 Feb 2024 12:06:21 +0000
Received: by outflank-mailman (input) for mailman id 676870;
 Tue, 06 Feb 2024 12:06:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXKDU-0007ji-8o
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 12:06:20 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 223b6c45-c4e8-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 13:06:18 +0100 (CET)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-33b1d7f736bso3124846f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 04:06:18 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w17-20020a5d6811000000b0033afcc899c1sm1949748wru.13.2024.02.06.04.06.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 04:06:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 223b6c45-c4e8-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707221177; x=1707825977; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=QO7nEsK9WdqbE5nV/bhuqZUjZzpr1P3VwDIWoHgSZCQ=;
        b=Kn1dNM82bq+hsNRFbX1G1ZQ7RIP12yWJyQO+5YEoStBpmehIKYARsWylpRBa41ykWU
         ddKjzJRrkdGVmbH6Syz8mN8/Q3zWFL4iIEdYqR6yXU2CiNFO/mmzZcU1JZvdUowO5HY6
         dlYyypkD+Yv8MBisAeNLFCDUp6Py31FjiAJkA1uAwzsS7WywtFq1eSyzM4oZ6kamlFDx
         tuQLw3YWliF82lauZtHwn3GnZHcBCYOKr/3H0CcJb5ySv0rTAX+buz4gWsUhEYmTPFE1
         MXhu1L0P+XJUcAFmvynVLQTD6EPS7gxEeMARARceryQ2J2QlRvgU4KHuzIbLZStLIc49
         tBwg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707221177; x=1707825977;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=QO7nEsK9WdqbE5nV/bhuqZUjZzpr1P3VwDIWoHgSZCQ=;
        b=WsoPaDtsU8scjf2uf7atCSdXtjFSAHiOfeqtT2213GvtwrgcplFB0JvvAc3iMdjr/s
         r2O93QgZgnOPAOb5q/sd2iyHgIx2JuzRfZ0U7QK/JaWypk4cpzvqU4JTMXYmYOisBxse
         6+qPuCmWsQ/PRp7JRqgc0kFjJReoLqrGjXZHCXemYTHJap6WiHg9NDlgpzaCmJwf1zMM
         5n9DpJZEq9qS6fpebk0jm1GLlKAzVdQGdikS9uic/LUO/rPZt1qPz48CJh18Orx8ZRNK
         jn3hzZnNFiO7Znhatw496mtpv/UPENg+1m9wTJsc3MEuLLiEbIDpiBWWAnJrKOfSKjpF
         BLHg==
X-Gm-Message-State: AOJu0YywpP8NtA5N+VfZoogDQC2btcXj65eGRei7pZeO7jkiDlsJkvFQ
	Dv1c2Ls1ceFY63UW2i6y99gBLDjdkDAwX2M2dc7F5QpRnOYXVcCRYaxoeZUkIaituoLWhHKS/J8
	=
X-Google-Smtp-Source: AGHT+IG67+2F1w5AhrZaA4fpFAV+ZWoU2Vs6fDG+S1kuGcd0rGh/P7cZDMYFO0apRdPhV+z0oP7wNA==
X-Received: by 2002:a5d:4a85:0:b0:337:67e:a1ef with SMTP id o5-20020a5d4a85000000b00337067ea1efmr963429wrq.7.1707221177717;
        Tue, 06 Feb 2024 04:06:17 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCX/IYOHB6h//hf8a8dFuYOpxN2aQdMNxe6Svu9tx0jXgqXhGElNRm1QQPlhVc9TAtxTiA+fw7S+6hDdDi4ezsfDR21JNUirAZzWQh6aK1Sg50YQN+cJGiOprlbMY3sx2i1+kujDR5yD
Message-ID: <c4c1d2b3-591e-403f-879b-bbb897f7ff25@suse.com>
Date: Tue, 6 Feb 2024 13:06:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Manuel Andreas <manuel.andreas@tum.de>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86/HVM: tidy state on hvmemul_map_linear_addr()'s error path
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

While in the vast majority of cases failure of the function will not
be followed by re-invocation with the same emulation context, a few
very specific insns - involving multiple independent writes, e.g. ENTER
and PUSHA - exist where this can happen. Since failure of the function
only signals to the caller that it ought to try an MMIO write instead,
such failure also cannot be assumed to result in wholesale failure of
emulation of the current insn. Instead we have to maintain internal
state such that another invocation of the function with the same
emulation context remains possible. To achieve that we need to reset MFN
slots after putting page references on the error path.

Note that all of this affects debugging code only, in causing an
assertion to trigger (higher up in the function). There's otherwise no
misbehavior - such a "leftover" slot would simply be overwritten by new
contents in a release build.

Also extend the related unmap() assertion, to further check for MFN 0.

Fixes: 8cbd4fb0b7ea ("x86/hvm: implement hvmemul_write() using real mappings")
Reported.by: Manuel Andreas <manuel.andreas@tum.de>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
While probably I could be convinced to omit the #ifndef, I'm really
considering to extend the one in hvmemul_unmap_linear_addr(), to
eliminate the zapping from release builds: Leaving MFN 0 in place is not
much better than leaving a (presently) guest-owned one there. And we
can't really put/leave INVALID_MFN there, as that would conflict with
other debug checking.

--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -696,7 +696,12 @@ static void *hvmemul_map_linear_addr(
  out:
     /* Drop all held references. */
     while ( mfn-- > hvmemul_ctxt->mfn )
+    {
         put_page(mfn_to_page(*mfn));
+#ifndef NDEBUG /* Clean slot for a subsequent map()'s error checking. */
+        *mfn = _mfn(0);
+#endif
+    }
 
     return err;
 }
@@ -718,7 +723,7 @@ static void hvmemul_unmap_linear_addr(
 
     for ( i = 0; i < nr_frames; i++ )
     {
-        ASSERT(mfn_valid(*mfn));
+        ASSERT(mfn_x(*mfn) && mfn_valid(*mfn));
         paging_mark_dirty(currd, *mfn);
         put_page(mfn_to_page(*mfn));
 


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 12:31:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 12:31:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676882.1053199 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKbC-0004dz-4f; Tue, 06 Feb 2024 12:30:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676882.1053199; Tue, 06 Feb 2024 12:30:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKbC-0004ds-1u; Tue, 06 Feb 2024 12:30:50 +0000
Received: by outflank-mailman (input) for mailman id 676882;
 Tue, 06 Feb 2024 12:30:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nZJ8=JP=epam.com=prvs=87667faf2e=oleksandr_tyshchenko@srs-se1.protection.inumbo.net>)
 id 1rXKbB-0004dm-1A
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 12:30:49 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8cf5d1d9-c4eb-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 13:30:46 +0100 (CET)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id
 416Bo7GN003642; Tue, 6 Feb 2024 12:30:42 GMT
Received: from eur01-he1-obe.outbound.protection.outlook.com
 (mail-he1eur01lp2051.outbound.protection.outlook.com [104.47.0.51])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3w355a28jy-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Tue, 06 Feb 2024 12:30:42 +0000 (GMT)
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com (2603:10a6:10:ed::15)
 by PAWPR03MB9155.eurprd03.prod.outlook.com (2603:10a6:102:340::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.34; Tue, 6 Feb
 2024 12:30:38 +0000
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167]) by DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167%4]) with mapi id 15.20.7249.035; Tue, 6 Feb 2024
 12:30:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8cf5d1d9-c4eb-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MVmDq/FgSdIlQfS7hCz9xbOFKGHzYD5ecsBFcnS/vEIHhiFhQ+TxxcIsSVBYH4F+jzstCAQxadQIDHmFr+Z0AhudJESOkzHmgbLPgQsM/EXZCQDVqvHt2SJKUOry52uvs+IcXSPMR3Nxv101r0LilzX//am919R1ZRFCasB96EjAjLiu59pz90Kv8OCVG6XQnl2hnQxT721sJ6wPUbWgpG7TopctBusssQI6MBAFhqa5F8n84zozKvYslDLl2INKL1b5LZh9tGZk/SEsolXwP+9d+jdlehvuz2cE4Y/tjopWNW8fXLMn66CmEbnDI5w8YRSUJSCbRIbPVFSeSgMOCQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=kjTwo6Y8AhEnUu4RAdJb86EaseaQxPTBS9Cm49fcLlo=;
 b=XJKAnmV0DDZ/U+GCwpxAVQzmhb/c4oUi/fnd0Ekb95Ly3/xQWcfork9j9Y/iWZFUfMSnHuL4winqelTAZcDBb9IoM30KOEIzOn9nswZSh//d6/RiSsEpnTvAJ3ts/J6aPTSxoOcg2c2o/42h0uBwEDDcwO197e8gQnAa4tKfd5RDzhTCiBTS/Xntz3rDUi8pVScom8wBRusFXeS48o/30DeXbgsi/1MMJUgxFtG1jEynSgb4UHBtbYn10R1EhGMxzUtB9ai/nUO59cDVGkLyISIKDEzYFqcjcJb+jxZbXeUJgmHYhxFsWzm3LGbU6+5ZfAynJZuHCReHYnZPSA4ENA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kjTwo6Y8AhEnUu4RAdJb86EaseaQxPTBS9Cm49fcLlo=;
 b=CrZicSFu2diSlpCHibFWOGngxwlA40TTfr1Uxeqo99Yo9/QW2RMn2eSD7QmUU8OGvoWeSrTRLFiaZCK4xJRANUz8LSlKRWW9b9zQ0QTt1RKX4rmPNKazC5L999refcIUcQ6zSzfg3L2E8vbKTsBUKDjeYe+Mu3rb66dlK+flcu/YbaoYu0eLXO3lcCHYbjsIweBLtGTGk0RVjOEoKPmRniCNpYGObU50FlvuTqNJPvfwDknyNNbITbMSnoWMlcsEVTkdiJmjS9f7om0xsq+4aAakexfn3AqMTKIIkQT8RJVmIIo+4Gxj5aBcYZQhanIB+t8lcBIOtqfjaKjOf40kbA==
From: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
To: Anthony PERARD <anthony.perard@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Wei Liu
	<wl@xen.org>, Oleksandr Tyshchenko <olekstysh@gmail.com>,
        George Dunlap
	<george.dunlap@citrix.com>,
        Nick Rosbrook <rosbrookn@gmail.com>, Juergen
 Gross <jgross@suse.com>,
        Viresh Kumar <viresh.kumar@linaro.org>
Subject: Re: [PATCH] libxl: Add "grant_usage" parameter for virtio disk
 devices
Thread-Topic: [PATCH] libxl: Add "grant_usage" parameter for virtio disk
 devices
Thread-Index: AQHaVcWXJV0m8uwkbEKeSgAhd5N6SLD734GAgAAcRoCAASbqgIAAIlMA
Date: Tue, 6 Feb 2024 12:30:38 +0000
Message-ID: <ce3dedb7-2e66-477d-bafa-485b1261a504@epam.com>
References: <20240202104903.1112772-1-olekstysh@gmail.com>
 <ace5b3c8-38e7-4661-9401-ac9ac77a5e9f@perard>
 <78bc7890-cfcb-4de4-a676-a3c2bbbb6d9f@epam.com>
 <d7291530-9e32-43dc-8b82-34d3962d2551@perard>
In-Reply-To: <d7291530-9e32-43dc-8b82-34d3962d2551@perard>
Accept-Language: en-US, ru-RU
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DB8PR03MB6108:EE_|PAWPR03MB9155:EE_
x-ms-office365-filtering-correlation-id: 80c750cb-3aa8-4c79-00cf-08dc270f6cc0
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 TpTfjKkZAVCcfUtxwJts0oLEkzI1zIzABvu8rM5mVGP2Dg+7miV7nunC5UU/WGIZUyZEq9JLIfo0OxsuoD3FOColrGszZb9zJyRnH0N/wu3fqc52toSbtq0viF6pzS7veJJeehyWUxk9tqpz59z+e9EZHQRWnNnCTXskTdrQ9SYLNcznt3AgIODjtD5PXMIi2OjFajeaQ4C4UuHlmqymgW/cpVL9HrP8BNNVpkkxHn+0g+WEzqSrgeNXlvbB2Bx8OnWbkIa98hVMW+uFKz5Tra/8hPFe0Y/o7xj/3jkDa+ilZK0f6ubaRpDUh7YJBrAdZMGBP2WtePah3elooA3qTZtLHIGHFdpaHm4lWfHvspbyko3bsZdscsAbvnS0DguRSWmw0DmMtrzLOMDx/if3J/VTjltZjLVQRxxDDW6AnjslJdFHc6GUZ7/lPpq3yCJ1a/4pK5VpgF+y/9PmmcB4Kp6BqNmJPHnB105goZM9EQIeegRxbgU9OmzBUyK+o3DI3rIkfP9YKqsjRxGju4qBfqpzOilbzq7yjscVbCWs8yylL5GzCY6z6lk5N510jgEFCPNtiU9uIMA+8alGMQ4gNsa8x6UwQ+yRLrLiBno63eqDIx8zXR4XyMijCR79esb18NZ+lblVLR4rm4/9ByMXFw==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR03MB6108.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(366004)(346002)(396003)(136003)(376002)(230922051799003)(1800799012)(64100799003)(451199024)(186009)(53546011)(38070700009)(71200400001)(478600001)(6486002)(6506007)(86362001)(122000001)(6916009)(38100700002)(31696002)(66446008)(66556008)(41300700001)(6512007)(5660300002)(31686004)(2906002)(66946007)(66476007)(2616005)(316002)(64756008)(36756003)(54906003)(76116006)(8936002)(4326008)(8676002)(83380400001)(26005)(45980500001);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?NmdCTFlPNFNmN2szdWdtdXFOdXNJRW5BTk1HOW5OZmRqelhCY3hSVkNDRExJ?=
 =?utf-8?B?Q3RHL0owaGxKM3F2d0RBVXNXMXZ2cmlVSUltcm5mV3R5QTNYS0xsZXl1Z0g3?=
 =?utf-8?B?SEp6b0k4RWlwVUNaUGRaRVZlbGVvaTZvWFZldzIzMXdzMXYyd2pZNnlMQTNJ?=
 =?utf-8?B?VHNZQmIvWEUwemx4dVJ2NXg0SGYxR0t3YnppZVdkQW9NcHhudnlaN28vZ3N5?=
 =?utf-8?B?czFMcU45ZDQwRWNMRmpyelJFcElpUlJTQU9EZ3RrOFl0OE5HSXZPeU1aWWpU?=
 =?utf-8?B?Wjg1R25qdExwNlBUVTVQWjFub0lHaTVoTnpLbGFmWGhhdDQyaEZMelNZbkw1?=
 =?utf-8?B?T1BMS3grS1VPWVVIcm93Yk9iM1pNS0Zvem52dXFYU2g0RmdoTnhEL1lFOHdS?=
 =?utf-8?B?S0p6OExMV1ZVa1Q4L04zVzdERTF3OVR2Sk9HU2FmK0pSaXlYVUJFQVJZSnFP?=
 =?utf-8?B?aTV0d0xDVVc5SWlEL0oxdmtlQ3RXV1hpZmthc1BDSE1mVzZmVVc5VUJ0aWs4?=
 =?utf-8?B?dFlrK2daZFRrMGI0VTNWa2FneG5YdWZJUWZWT1lGTWlrQXpGWEp0Y1J3OWhs?=
 =?utf-8?B?UWhrRzVrS2lyays1VVh6VFFKNlJWR2k3c3FwNDdrWXRML1lJK210MlBva0tm?=
 =?utf-8?B?SHhRNEZSZURMTC83dG1ybGc3VkIwb2FkY2h3cElqVEdNblo4bzJDRTN1SEpX?=
 =?utf-8?B?bVFMU0JWcTBSZjZFSXA4NlZhMDh4a3JsU3dXL2RtMFRVbGJxMnhseGp0S1pW?=
 =?utf-8?B?SGxzc1dkMjd5dEYySnhjYVZkbEFVL1h0WGxlV0tVWU1YU0NrQ0s3elVqSU9i?=
 =?utf-8?B?SEk2VlpqYkpEZGxLUW1mb3U2b3V0OFRuRnd3dVJDQzN4Y083WlRCOHptZmtN?=
 =?utf-8?B?eGRlSEVpMlAycVlEa2xrcTFsZStFOXArR2VBUHFsa0Yzdm9hQ0QzeWtKZURV?=
 =?utf-8?B?YVNNb0J5b0cyL3lyakhmcUUrRUV4THF5Rm04WDgvRjFZRUNlUG96MlNmSGlN?=
 =?utf-8?B?UkEzVkhORW04VEFPWWxSV2dyTlJ2WmpKTWtkR25VeGVrdWREOEFvbXBqTVVC?=
 =?utf-8?B?TXBvVTlja2ZwRnpwMUx5SldIT1NnWUI4SHU5UkNjRE5TV29DRnFtVVhOQU9M?=
 =?utf-8?B?U2FJUGtMVnNyNXNsUmxMOGUrRURhUzVib00rSm9pYlNBVVloVU55Q2NIQlQw?=
 =?utf-8?B?WFBxNEVNeWVMbEs4aXVjZzJyZDl2c1ByVFY1elJUUVRhTGUwQ3ZuRW14WFZ5?=
 =?utf-8?B?WHNwUGFBb2hTVDFwOFVYMkJBbVJFemRobmdIcnFRYWJFcE9MNlJ6T0dxVHVx?=
 =?utf-8?B?aE9PeUFwMUVLbUlUZHZwTWl1QmE2MWZHUlVRbVhGWXBETWlabEpvZnNZWG1R?=
 =?utf-8?B?TzFEN3lEWUpXaURwUmszZjA0Tng3MzEzY09jd0NPZHRhVFpvYkRHM2ZoNFFa?=
 =?utf-8?B?V1AyNGhSMkd6K1ptb2NJNTh2WTZXaVpKenAxbW9sVVhjTm9JSEd5QWdXRWRy?=
 =?utf-8?B?Q1lZNGVDaVdOZTlEUWl5b2NWcVduOVBTWWFlM1hrREZySnZvME8yQ0pEWDdH?=
 =?utf-8?B?eDNwd2NRRjZWQWw2Z2orY2dNelg2SVZIbHhWV0VqRFVZQkFOMUx2RDdoV3B5?=
 =?utf-8?B?dmRSbGNBR3MzQjNNTmxseFczNFVkZlkrWlNMV0prNkNvanVtdEhwMlNPQ0NU?=
 =?utf-8?B?c0c0S1NnV3YzaWk1a2JTNFh4aFc1K3ptN1d6TW9WbWFlWks1UmdBZTRxQ1k2?=
 =?utf-8?B?ejhOMGVwTDk1VEFtTjhLT241N0t3VCtqdlFaRkxTMmUwL2RIRVNEOEVZOWJO?=
 =?utf-8?B?NDEybERvNk04ODZUZ3BoVFdvWU81VkkreGFRL0hzSkxKUUJSTWxQcGgrVFox?=
 =?utf-8?B?dmRybkpKbW9LUU1JaEdvT1dKdkJFTzJvVEtnS0hCQ2w1cmFyVEV0K2ZzejZY?=
 =?utf-8?B?M1BEU2E5cjRrY2dKWUU2Z1RLZzhXTEZEY3JYTmxJQ0hpUnF0blJQdTVRZitG?=
 =?utf-8?B?OGFCdzV2bk9jdUVCUnI1cUsvSGluc2lwV20wS2Z2QW5RelJBemRPTjNxeWNt?=
 =?utf-8?B?QWFUUjVtWDFxZmRoMThpRGdBU2EwcjNxSUdzaHBEQVhrbmllMThEQ3paOGps?=
 =?utf-8?B?TTkrZWFkYWtoUzNQVmZJL0tVbFRvU2pOL1lNd3RBaGRNaWRBanJSRkxiU0hS?=
 =?utf-8?Q?vuhYrCN7AOEelbNt6futuN4=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <4ABE65077FDF7E4DA83413FDDA06EDC4@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB8PR03MB6108.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 80c750cb-3aa8-4c79-00cf-08dc270f6cc0
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Feb 2024 12:30:38.2004
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ju4bLkBgNbl1+fGUAeeCPmqaHOHNHxVlzWAV1gZSgz6w757IpWYwI2vyO31ca4nIYI1DlrPGFRNtJxFLftZbM+vnC4O3L0sS7HGgcQFMlo0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAWPR03MB9155
X-Proofpoint-ORIG-GUID: -3rZAFBQrGk1hHSXl7ixMfE6VqIs4y1G
X-Proofpoint-GUID: -3rZAFBQrGk1hHSXl7ixMfE6VqIs4y1G
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-06_06,2024-01-31_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0
 impostorscore=0 phishscore=0 clxscore=1015 priorityscore=1501
 malwarescore=0 mlxlogscore=999 adultscore=0 suspectscore=0 bulkscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2401310000 definitions=main-2402060088

DQoNCk9uIDA2LjAyLjI0IDEyOjI3LCBBbnRob255IFBFUkFSRCB3cm90ZToNCg0KDQpIZWxsbyBB
bnRob255DQoNCltzbmlwXQ0KDQo+Pj4+IGRpZmYgLS1naXQgYS9kb2NzL21hbi94bC1kaXNrLWNv
bmZpZ3VyYXRpb24uNS5wb2QuaW4gYi9kb2NzL21hbi94bC1kaXNrLWNvbmZpZ3VyYXRpb24uNS5w
b2QuaW4NCj4+Pj4gaW5kZXggYmM5NDVjYzUxNy4uM2MwMzU0NTZkNSAxMDA2NDQNCj4+Pj4gLS0t
IGEvZG9jcy9tYW4veGwtZGlzay1jb25maWd1cmF0aW9uLjUucG9kLmluDQo+Pj4+ICsrKyBiL2Rv
Y3MvbWFuL3hsLWRpc2stY29uZmlndXJhdGlvbi41LnBvZC5pbg0KPj4+PiBAQCAtNDA0LDYgKzQw
NCwzMSBAQCBWaXJ0aW8gZnJvbnRlbmQgZHJpdmVyICh2aXJ0aW8tYmxrKSB0byBiZSB1c2VkLiBQ
bGVhc2Ugbm90ZSwgdGhlIHZpcnR1YWwNCj4+Pj4gKz1pdGVtIEI8Z3JhbnRfdXNhZ2U9Qk9PTEVB
Tj4NCj4+Pj4NCj4+Pj4gKz1vdmVyIDQNCj4+Pj4gKw0KPj4+PiArPWl0ZW0gRGVzY3JpcHRpb24N
Cj4+Pj4gKw0KPj4+PiArU3BlY2lmaWVzIHRoZSB1c2FnZSBvZiBYZW4gZ3JhbnRzIGZvciBhY2Nl
c3NpbmcgZ3Vlc3QgbWVtb3J5LiBPbmx5IGFwcGxpY2FibGUNCj4+Pj4gK3RvIHNwZWNpZmljYXRp
b24gInZpcnRpbyIuDQo+Pj4+ICsNCj4+Pj4gKz1pdGVtIFN1cHBvcnRlZCB2YWx1ZXMNCj4+Pj4g
Kw0KPj4+PiArSWYgdGhpcyBvcHRpb24gaXMgQjx0cnVlPiwgdGhlIFhlbiBncmFudHMgYXJlIGFs
d2F5cyBlbmFibGVkLg0KPj4+PiArSWYgdGhpcyBvcHRpb24gaXMgQjxmYWxzZT4sIHRoZSBYZW4g
Z3JhbnRzIGFyZSBhbHdheXMgZGlzYWJsZWQuDQo+Pj4NCj4+PiBVbmZvcnR1bmF0ZWx5LCB0aGlz
IGlzIHdyb25nLCB0aGUgaW1wbGVtZW50YXRpb24gaW4gdGhlIHBhdGNoIG9ubHkNCj4+PiBzdXBw
b3J0IHR3byB2YWx1ZXM6IDEgLyAwLCBub3RoaW5nIGVsc2UsIGFuZCB0cnlpbmcgdG8gd3JpdGUg
InRydWUiIG9yDQo+Pj4gImZhbHNlIiB3b3VsZCBsZWFkIHRvIGFuIGVycm9yLiAoV2VsbCBhY3R1
YWxseSBpdCdzICJncmFudF91c2FnZT0xIiBvcg0KPj4+ICJncmFudF91c2FnZT0wIiwgdGhlcmUn
cyBub3RoaW5nIHRoYXQgY3V0IHRoYXQgc3RyaW5nIGF0IHRoZSAnPScuKQ0KPj4NCj4+DQo+PiBZ
b3UgYXJlIHJpZ2h0LCBvbmx5IDEgLyAwIGNhbiBiZSBzZXQgdW5saWtlIGZvciB2aXJ0aW89Wy4u
Ll0gd2hpY2ggc2VlbXMNCj4+IGhhcHB5IHdpdGggZmFsc2UvdHJ1ZS4NCj4+DQo+Pg0KPj4+DQo+
Pj4gQWxzbywgZG8gd2UgcmVhbGx5IG5lZWQgdGhlIGV4dHJhIHZlcmJhbCBkZXNjcmlwdGlvbiBv
ZiBlYWNoIHZhbHVlIGhlcmU/DQo+Pj4gSXMgc2ltcGx5IGhhdmluZyB0aGUgZm9sbG93aW5nIHdv
dWxkIGJlIGVub3VnaD8NCj4+Pg0KPj4+ICAgICAgID1pdGVtIFN1cHBvcnRlZCB2YWx1ZXMNCj4+
Pg0KPj4+ICAgICAgIDEsIDANCj4+Pg0KPj4+IFRoZSBkZXNjcmlwdGlvbiBpbiAiRGVzY3JpcHRp
b24iIHNlY3Rpb24gd291bGQgaG9wZWZ1bGx5IGJlIGVub3VnaC4NCj4+DQo+Pg0KPj4gSSB0aGlu
aywgdGhpcyBtYWtlcyBzZW5zZS4NCj4+DQo+PiBTbywgc2hhbGwgSSBsZWF2ZSAiZ3JhbnRfdXNh
Z2U9MS9ncmFudF91c2FnZT0wIiBvciB1c2UgcHJvcG9zZWQgb3B0aW9uDQo+PiAidXNlLWdyYW50
L25vLXVzZS1ncmFudCI/DQo+IA0KPiBMZXQncyBnbyB3aXRoICJncmFudF91c2FnZT0qIiwgYXQg
bGVhc3QgdGhpcyB3aWxsIGJlIGNvbnNpc3RlbnQgd2l0aCB0aGUNCj4gb3B0aW9uIGZvciAidmly
dGlvIi4NCg0KDQp0aGFua3MgZm9yIHRoZSBjb25maXJtYXRpb24sIHdpbGwgZG8NCg0KDQo+IA0K
PiBDaGVlcnMsDQo+IA==


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 12:38:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 12:38:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676889.1053209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKiT-0005ab-Vd; Tue, 06 Feb 2024 12:38:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676889.1053209; Tue, 06 Feb 2024 12:38:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKiT-0005aU-Ss; Tue, 06 Feb 2024 12:38:21 +0000
Received: by outflank-mailman (input) for mailman id 676889;
 Tue, 06 Feb 2024 12:38:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3A9u=JP=gmail.com=olekstysh@srs-se1.protection.inumbo.net>)
 id 1rXKiS-0005ZV-Fu
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 12:38:20 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9b6f7579-c4ec-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 13:38:19 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a293f2280c7so730991166b.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 04:38:19 -0800 (PST)
Received: from EPUAKYIW03DD.. ([91.123.150.198])
 by smtp.gmail.com with ESMTPSA id
 cw9-20020a170907160900b00a31805b4165sm1113495ejd.11.2024.02.06.04.38.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 06 Feb 2024 04:38:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b6f7579-c4ec-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707223098; x=1707827898; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=BWugegBYO7vklVvyXEYXtd27rTRIL053PsXMtxyCO3w=;
        b=GLLsP552F5YeDFoafXIfPcyOJDVduQ7gLx5uOBlP+6rqW/bkER6WbyofArPziN+U1H
         Ox+9teb5S3q2vMDcikGMAgt/hOXLccbPDX70lpU4s3z9QOBtLw+fLpHBcgfSDr4zcVEQ
         zTcg9utVnXH+CUpsb5EjU60K4Ovd5KaWncwgSikfF5gkQ8L9E6+N0xK7opn73gnZUa8O
         eEp6BhGTze38qx7BifLRr8wSuV19U75GfZcOFYyRfio9JMiFywH47gjwKi9oG5+UtQpn
         +U1j2hRB0kVWrlDFR3ifcbN/Smf4l+zRC61ZUCdalNqeD+YY8aSg/quHKrMgrrh9BUjv
         8IRg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707223098; x=1707827898;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=BWugegBYO7vklVvyXEYXtd27rTRIL053PsXMtxyCO3w=;
        b=tKpLbi69FCsM6hH8kG+wVWvjkbpjNztIROyyYWB7Um02xSf2wLaUuxlFD603mLOWEt
         xVpxcLUCUmIFHZ2Tx2Yvze8tISfw9qDb/NP9hO9ZZGWuRzoP1bAoWIRoTw7od4xFg/RM
         JjfOnPyrtoJXWJxfgukTt9rkRld+EkU6S+jO0ICRyJCFPx5tK3rB3e0JsInV4kdunSI6
         6T4/S+LU2XxCfxUBAaLzi0gP9lELFZCMT6reTuTpIU7Np7q/f5o86Z2gu6a5EpJIoP+K
         7gPNcDm0pCKxXpNVPwkSf5ZO0jrT6xbpHZwXKLN5nQAdhm1KZ1055JPi5yYE3qYrluPY
         RsQg==
X-Gm-Message-State: AOJu0Yyj/LigbAmLby1YQBDjBXr0pYOWm+GPaWIFmIiyAyGBBNPPDdb6
	npkN/3uSNv7rU2fcuR5It3IvRRjtmzkCAIDn/ZyTmUS+rVltFm8tEaHtV3v8
X-Google-Smtp-Source: AGHT+IHJitrq8+qdsPlsncfVI4fWtYH1dtuN6vBNvqEF/1lNW69vEcx2f7uthyA5Wyq8XUxWMwF3GQ==
X-Received: by 2002:a17:906:3e56:b0:a37:f60d:1cf3 with SMTP id t22-20020a1709063e5600b00a37f60d1cf3mr1412206eji.7.1707223098301;
        Tue, 06 Feb 2024 04:38:18 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCW4UPLLt2HFoC5B9r1C7Jpq6gix6M2xaHmIxK48D+w6cGxeCMSkDzt0T1vsz0lpeQnJ1TjTm95VuRENoXHPRhuUwhTuzgJqxKKnIqov3sismgF5qyQO1SphAJDMlglm0MIvwLGYatwt37z+9KAI8KXmmR0F91IZnMEUXchwWurGrNC2xH6UrtYuZicA9FJ0999iFHBp3UBrcyIOHutb40K0II+L/N7x9qGyoyBtfZk=
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Juergen Gross <jgross@suse.com>,
	Viresh Kumar <viresh.kumar@linaro.org>
Subject: [PATCH V2] libxl: Add "grant_usage" parameter for virtio disk devices
Date: Tue,  6 Feb 2024 14:38:14 +0200
Message-Id: <20240206123814.2308837-1-olekstysh@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Allow administrators to control whether Xen grant mappings for
the virtio disk devices should be used. By default (when new
parameter is not specified), the existing behavior is retained
(we enable grants if backend-domid != 0).

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
In addition to "libxl: arm: Add grant_usage parameter for virtio devices"
https://github.com/xen-project/xen/commit/c14254065ff4826e34f714e1790eab5217368c38

I wonder, whether I had to also include autogenerated changes to:
 - tools/libs/util/libxlu_disk_l.c
 - tools/libs/util/libxlu_disk_l.h

 V2:
  - clarify documentation to match the implementation
  - apply a default value if "grant_usage" is missing in the Xenstore
    in libxl__disk_from_xenstore()
---
 docs/man/xl-disk-configuration.5.pod.in | 24 ++++++++++++++++++++++++
 tools/golang/xenlight/helpers.gen.go    |  6 ++++++
 tools/golang/xenlight/types.gen.go      |  1 +
 tools/include/libxl.h                   |  7 +++++++
 tools/libs/light/libxl_arm.c            |  4 ++--
 tools/libs/light/libxl_disk.c           | 14 ++++++++++++++
 tools/libs/light/libxl_types.idl        |  1 +
 tools/libs/util/libxlu_disk_l.l         |  3 +++
 8 files changed, 58 insertions(+), 2 deletions(-)

diff --git a/docs/man/xl-disk-configuration.5.pod.in b/docs/man/xl-disk-configuration.5.pod.in
index bc945cc517..55e78005cf 100644
--- a/docs/man/xl-disk-configuration.5.pod.in
+++ b/docs/man/xl-disk-configuration.5.pod.in
@@ -404,6 +404,30 @@ Virtio frontend driver (virtio-blk) to be used. Please note, the virtual
 device (vdev) is not passed to the guest in that case, but it still must be
 specified for the internal purposes.
 
+=item B<grant_usage=BOOLEAN>
+
+=over 4
+
+=item Description
+
+Specifies the usage of Xen grants for accessing guest memory. Only applicable
+to specification "virtio".
+
+=item Supported values
+
+1, 0
+
+=item Mandatory
+
+No
+
+=item Default value
+
+If this option is missing, then the default grant setting will be used,
+i.e. "grant_usage=1" if backend-domid != 0 or "grant_usage=0" otherwise.
+
+=back
+
 =back
 
 =head1 COLO Parameters
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 35e209ff1b..768ab0f566 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1879,6 +1879,9 @@ x.ActiveDisk = C.GoString(xc.active_disk)
 x.HiddenDisk = C.GoString(xc.hidden_disk)
 if err := x.Trusted.fromC(&xc.trusted);err != nil {
 return fmt.Errorf("converting field Trusted: %v", err)
+}
+if err := x.GrantUsage.fromC(&xc.grant_usage);err != nil {
+return fmt.Errorf("converting field GrantUsage: %v", err)
 }
 
  return nil}
@@ -1927,6 +1930,9 @@ if x.HiddenDisk != "" {
 xc.hidden_disk = C.CString(x.HiddenDisk)}
 if err := x.Trusted.toC(&xc.trusted); err != nil {
 return fmt.Errorf("converting field Trusted: %v", err)
+}
+if err := x.GrantUsage.toC(&xc.grant_usage); err != nil {
+return fmt.Errorf("converting field GrantUsage: %v", err)
 }
 
  return nil
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 7907aa8999..0b712d2aa4 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -740,6 +740,7 @@ ColoExport string
 ActiveDisk string
 HiddenDisk string
 Trusted Defbool
+GrantUsage Defbool
 }
 
 type DeviceNic struct {
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index f1652b1664..2b69e08466 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -578,6 +578,13 @@
  */
 #define LIBXL_HAVE_DEVICE_DISK_SPECIFICATION 1
 
+/*
+ * LIBXL_HAVE_DISK_GRANT_USAGE indicates that the libxl_device_disk
+ * has 'grant_usage' field to specify the usage of Xen grants for
+ * the specification 'virtio'.
+ */
+#define LIBXL_HAVE_DISK_GRANT_USAGE 1
+
 /*
  * LIBXL_HAVE_CONSOLE_ADD_XENSTORE indicates presence of the function
  * libxl_console_add_xenstore() in libxl.
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 1539191774..1cb89fa584 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -1372,12 +1372,12 @@ next_resize:
             libxl_device_disk *disk = &d_config->disks[i];
 
             if (disk->specification == LIBXL_DISK_SPECIFICATION_VIRTIO) {
-                if (disk->backend_domid != LIBXL_TOOLSTACK_DOMID)
+                if (libxl_defbool_val(disk->grant_usage))
                     iommu_needed = true;
 
                 FDT( make_virtio_mmio_node(gc, fdt, disk->base, disk->irq,
                                            disk->backend_domid,
-                                           disk->backend_domid != LIBXL_TOOLSTACK_DOMID) );
+                                           libxl_defbool_val(disk->grant_usage)) );
             }
         }
 
diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
index ea3623dd6f..ed02b655a3 100644
--- a/tools/libs/light/libxl_disk.c
+++ b/tools/libs/light/libxl_disk.c
@@ -181,6 +181,9 @@ static int libxl__device_disk_setdefault(libxl__gc *gc, uint32_t domid,
             return ERROR_INVAL;
         }
         disk->transport = LIBXL_DISK_TRANSPORT_MMIO;
+
+        libxl_defbool_setdefault(&disk->grant_usage,
+                                 disk->backend_domid != LIBXL_TOOLSTACK_DOMID);
     }
 
     if (hotplug && disk->specification == LIBXL_DISK_SPECIFICATION_VIRTIO) {
@@ -429,6 +432,8 @@ static void device_disk_add(libxl__egc *egc, uint32_t domid,
             flexarray_append(back, libxl__device_disk_string_of_transport(disk->transport));
             flexarray_append_pair(back, "base", GCSPRINTF("%"PRIu64, disk->base));
             flexarray_append_pair(back, "irq", GCSPRINTF("%u", disk->irq));
+            flexarray_append_pair(back, "grant_usage",
+                                  libxl_defbool_val(disk->grant_usage) ? "1" : "0");
         }
 
         flexarray_append(front, "backend-id");
@@ -623,6 +628,15 @@ static int libxl__disk_from_xenstore(libxl__gc *gc, const char *libxl_path,
             goto cleanup;
         }
         disk->irq = strtoul(tmp, NULL, 10);
+
+        tmp = libxl__xs_read(gc, XBT_NULL,
+                             GCSPRINTF("%s/grant_usage", libxl_path));
+        if (!tmp) {
+            LOG(DEBUG, "Missing xenstore node %s/grant_usage, using default value", libxl_path);
+            libxl_defbool_set(&disk->grant_usage,
+                              disk->backend_domid != LIBXL_TOOLSTACK_DOMID);
+        } else
+            libxl_defbool_set(&disk->grant_usage, strtoul(tmp, NULL, 0));
     }
 
     disk->vdev = xs_read(ctx->xsh, XBT_NULL,
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 899ad30969..6d76f25528 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -803,6 +803,7 @@ libxl_device_disk = Struct("device_disk", [
     ("active_disk", string),
     ("hidden_disk", string),
     ("trusted", libxl_defbool),
+    ("grant_usage", libxl_defbool),
     ])
 
 libxl_device_nic = Struct("device_nic", [
diff --git a/tools/libs/util/libxlu_disk_l.l b/tools/libs/util/libxlu_disk_l.l
index 6d53c093a3..f37dd443bd 100644
--- a/tools/libs/util/libxlu_disk_l.l
+++ b/tools/libs/util/libxlu_disk_l.l
@@ -220,6 +220,9 @@ hidden-disk=[^,]*,?	{ STRIP(','); SAVESTRING("hidden-disk", hidden_disk, FROMEQU
 trusted,?		{ libxl_defbool_set(&DPC->disk->trusted, true); }
 untrusted,?		{ libxl_defbool_set(&DPC->disk->trusted, false); }
 
+grant_usage=1,?		{ libxl_defbool_set(&DPC->disk->grant_usage, true); }
+grant_usage=0,?		{ libxl_defbool_set(&DPC->disk->grant_usage, false); }
+
  /* the target magic parameter, eats the rest of the string */
 
 target=.*	{ STRIP(','); SAVESTRING("target", pdev_path, FROMEQUALS); }
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 12:44:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 12:44:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676893.1053219 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKnp-0007Jp-JA; Tue, 06 Feb 2024 12:43:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676893.1053219; Tue, 06 Feb 2024 12:43:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKnp-0007Ji-FJ; Tue, 06 Feb 2024 12:43:53 +0000
Received: by outflank-mailman (input) for mailman id 676893;
 Tue, 06 Feb 2024 12:43:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=z75a=JP=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXKno-0007Jc-HS
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 12:43:52 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 60845f2e-c4ed-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 13:43:50 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id B6F4C1FB7B;
 Tue,  6 Feb 2024 12:43:49 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8C932139D8;
 Tue,  6 Feb 2024 12:43:49 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id w/7XIIUpwmVgLwAAD6G6ig
 (envelope-from <jgross@suse.com>); Tue, 06 Feb 2024 12:43:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60845f2e-c4ed-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707223429; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=pLWMim2B/YQSmAwCWn+D8u0DBo5Nz/tbQLySCJq5lh4=;
	b=hVIN8I8uFlEDikCOeI0K2mFEXTK31guGfdCKPeZ62Mg9iZWmLfnRQ5r27q2Ujf1gO6eEml
	jqIK/nJtCPCQnZzdvN1UHwdVI0x9i2QZTcNKhRaCyniNJdxF192mX/kLGYTy5Lnt7fybA3
	Tzd5G81kmw3fq+7EHFvT97G9w148fwo=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707223429; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=pLWMim2B/YQSmAwCWn+D8u0DBo5Nz/tbQLySCJq5lh4=;
	b=hVIN8I8uFlEDikCOeI0K2mFEXTK31guGfdCKPeZ62Mg9iZWmLfnRQ5r27q2Ujf1gO6eEml
	jqIK/nJtCPCQnZzdvN1UHwdVI0x9i2QZTcNKhRaCyniNJdxF192mX/kLGYTy5Lnt7fybA3
	Tzd5G81kmw3fq+7EHFvT97G9w148fwo=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH] tools/libs/light: don't allow to stop Xenstore stubdom
Date: Tue,  6 Feb 2024 13:43:45 +0100
Message-Id: <20240206124345.19433-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [4.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 TO_DN_SOME(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[34.66%]
X-Spam-Level: ****
X-Spam-Score: 4.90
X-Spam-Flag: NO

A Xenstore stubdom should never be stoppable.

Reject attempts to do so.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 tools/libs/light/libxl_domain.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c
index 5ee1544d9c..21d1b0babf 100644
--- a/tools/libs/light/libxl_domain.c
+++ b/tools/libs/light/libxl_domain.c
@@ -1167,13 +1167,19 @@ void libxl__destroy_domid(libxl__egc *egc, libxl__destroy_domid_state *dis)
 {
     STATE_AO_GC(dis->ao);
     uint32_t domid = dis->domid;
+    libxl_dominfo info;
     int rc;
 
     libxl__ev_child_init(&dis->destroyer);
 
-    rc = libxl_domain_info(CTX, NULL, domid);
+    rc = libxl_domain_info(CTX, &info, domid);
     switch(rc) {
     case 0:
+        if (info.never_stop) {
+            LOGD(ERROR, domid, "Non-stoppable domain");
+            rc = ERROR_INVAL;
+            goto out;
+        }
         break;
     case ERROR_DOMAIN_NOTFOUND:
         LOGD(ERROR, domid, "Non-existant domain");
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 12:44:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 12:44:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676880.1053228 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKoa-0007nc-Qs; Tue, 06 Feb 2024 12:44:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676880.1053228; Tue, 06 Feb 2024 12:44:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXKoa-0007nV-OD; Tue, 06 Feb 2024 12:44:40 +0000
Received: by outflank-mailman (input) for mailman id 676880;
 Tue, 06 Feb 2024 12:15:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bmrB=JP=outlook.it=giuseppe.de.rosa@srs-se1.protection.inumbo.net>)
 id 1rXKLw-0001Ye-5c
 for xen-devel@lists.xen.org; Tue, 06 Feb 2024 12:15:04 +0000
Received: from EUR03-AM7-obe.outbound.protection.outlook.com
 (mail-am7eur03olkn20801.outbound.protection.outlook.com
 [2a01:111:f403:2e0e::801])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5a4ce124-c4e9-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 13:15:02 +0100 (CET)
Received: from DB9P192MB1434.EURP192.PROD.OUTLOOK.COM (2603:10a6:10:296::21)
 by DB9P192MB1705.EURP192.PROD.OUTLOOK.COM (2603:10a6:10:390::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.34; Tue, 6 Feb
 2024 12:14:59 +0000
Received: from DB9P192MB1434.EURP192.PROD.OUTLOOK.COM
 ([fe80::3954:cef6:c539:697a]) by DB9P192MB1434.EURP192.PROD.OUTLOOK.COM
 ([fe80::3954:cef6:c539:697a%6]) with mapi id 15.20.7249.035; Tue, 6 Feb 2024
 12:14:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a4ce124-c4e9-11ee-8a47-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hLPhaatdKBVW7VGtj+r2XroO3mY9Y9i6GU8IPVoh5kkTFblOrSmgEXzbjZ7tZgmJmRrIMqywDVwWCcpEkC0shcSohYX9WlH+07rPRl8MG1xOhM8azHjveBmaJjimRG/Ag4qGnKjVvCOPYF8ILA+9piUY115LeNKYoFphjS0tBnkuqMDm1sJucfCkcL8Kc8o2K8U8cRjznsrheakegABH/lf4XfgSTthEwaY/wSsRDb1/7QgRG2a/bD/MNSRpjObjpzsFdaXzyflgI4ZlK6aAy3+rLglTsX/cSfgxnco8vu0VTwx8ykcmAv01QRVvGfcjH6DmXnvsB63aABpzhufO0Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=bfJIk17PpbBfHpBm6+Fpzi6fN/Ybt/PSCAmmnyNCP78=;
 b=SO+93Z0NqClN/OltVtZJIySGLt4Pr4j+d0V9YHqxRDRXuAwQ/ffCLmngqSonCDoNJrmYHQxEG5alPJTpyQO8jogrpjZWEpit+nETOUQ/7W1Sjh3aCCVfcsSoM/ZaJ7455TOYa5SnYN5i8SgTLYxwVTYIOn9rtD/6VgZJSV4RXr45+LQ1q+NxFUt61joiFnb+TPGzwL5XtJcARroAVmIf/yyDSrDLU72v9TU1KiH5+vQXO8lMPQO1B7CS1GviWN5fJpJMYCAbOmpdDP+v+dwTVRfx3pYsC94HvgQ9f6E8Cd4CICeJQNz9+KULiJXgqIuvo7CEX9oiyVEPg5jxH6v5Pg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none;
 dkim=none; arc=none
From: Giuseppe De Rosa <giuseppe.de.rosa@outlook.it>
To: "xen-devel@lists.xen.org" <xen-devel@lists.xen.org>
Subject: [BUG] Possible FATAL PAGE FAULT in domain_crash function on Xen
 4.14.6
Thread-Topic: [BUG] Possible FATAL PAGE FAULT in domain_crash function on Xen
 4.14.6
Thread-Index: AQHaWPFxTFnf2PP9dU2mKqDyY7lMyQ==
Date: Tue, 6 Feb 2024 12:14:59 +0000
Message-ID:
 <DB9P192MB1434AD2FD1069E39FB480068A6462@DB9P192MB1434.EURP192.PROD.OUTLOOK.COM>
Accept-Language: it-IT, en-US
Content-Language: it-IT
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
x-tmn: [V/mlFo0ZEgixTlu3mNp4LKO0UiuMDr42p2szM09yAXs=]
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DB9P192MB1434:EE_|DB9P192MB1705:EE_
x-ms-office365-filtering-correlation-id: d48f151e-8de6-4a29-ca8a-08dc270d3d2e
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 jvJJFa68BFx8WzPB5lKytTFkFz7qfpgeNKbZpOPasKY1wZfEi6oXJ+xcKDvLOwSOXGHUtkGfHD6zPCiq3eyhZaR0QhK7OqWLmYSH9bnXyCoJhpR3VmrQXrga1KwLCmVLBNK29QUYxncLn24c+rJudZ4hXTN2Kkh70NYe968xN4ZVVaCpL78HKwsQkDC45SwnLgnPgXQYn6W7ZSX7pfE+7c3R5FTK8HtQX0djBnNvCsSLQBH+nQmok94QmfxPOM9mXcikrs5vCD0uLvnl+RGEZkndz5RB3mK6E7wXItinBxXOoOYHB2NSKClSnN6myyIDmRtWbgzn8jxkHw+kaWi3+IdNBTJQbdoO4qnDEmOyQ5iJULrolRFhWqpzQMpmUpUkOaCKh2u8IrZq52fedxazdaFiUd2w1YOu35H0XGqlmljNNbbWzC0NQSmvPkT564pFNvwUix78R/6QApJZLYNu86Z5udVngVTdGQntoIGvM9iCvHTJ2ls7dfy0JTTaqXo8BgqwYD8n1GYwupGuxwkqk6LaBLYow+xPV/MPK6hZsSSp6XpRMaWJQI5qrqFDaNFY
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?iso-8859-1?Q?NfhndCs4G5IL7tbRWdWDO42a6ZSgG9SAmxmvl9lrLBQPj29bLBW2/p4lY2?=
 =?iso-8859-1?Q?RwC2h4fZHWyQ0FhNZOAb6kVwf4v6UNtyW2gsyb/t2IZOICGQ4jlob7Cu9h?=
 =?iso-8859-1?Q?lJwIORLjQs4XaycQWVKkemt/+dwdM696gJjobd4eVfo9tQfHq7CvGqZMYc?=
 =?iso-8859-1?Q?8VfKRjG03mpM888ImOjUo65wxK7et0i8MLJU8/jAXhzl+rY4esiGEaqTUj?=
 =?iso-8859-1?Q?xj6sOQaFIxteEfoZUZtY7WwQ1Rg9pHZM607BuXb+EnbyRrUtN6Mau36ys0?=
 =?iso-8859-1?Q?G5dOEAk4EN6m0UdhbuFCXXcHcNyecQwhzHs1lZMqGlBo9YdmM7bE/ek5yA?=
 =?iso-8859-1?Q?s3Q/vCMe62hl4VISORtXTCp6RVQGpapvzFsQQeWH2+vNkbaDsgwHxvLxqc?=
 =?iso-8859-1?Q?m1EgRb5R83Pe7poir8v94IPfDV3FIWhFU44/3vyll+/SbtUnnJ6KZbRaxX?=
 =?iso-8859-1?Q?1WTDMxFR2xefjhPjqAwvFKRrs7OnVVP7HoJZ9UdFZGqaVk7A59SzXz+1o8?=
 =?iso-8859-1?Q?MWXk8ZY4x2eGhfBnlrhvmuxm5r+Nq5yMl2aXu1qK78S8KSND/qDo/PJHqd?=
 =?iso-8859-1?Q?HoEAduwFb7ISrAXrf1GtFAYlCAZzZq6w2a9Ir2ceDDTckUKyEBvtkOPSVP?=
 =?iso-8859-1?Q?XDJz6fDBXmtIyUYL1ohjz5r0XeLP3lvFYGEM/QVVMvKs0hzvi5llT7lAfR?=
 =?iso-8859-1?Q?B2qfYvSWueGd/evcUKpIZm4SbK7d++KkW7Z+eO03t1iOw1Gatxw5ufE8gd?=
 =?iso-8859-1?Q?/GNXpPXzg9ae6sAwjU59YCDvN/pwZe1vsVyKdLOEFaJOuY9sPoKy2X4x4n?=
 =?iso-8859-1?Q?HPAumaErlZ1LyYZPH/jFByAqTLYvCXqqDND+aTgATiAcqMrW3To2LbtGrb?=
 =?iso-8859-1?Q?oyG1Ej3WIrKd+U9VTM+66twtiT4DfeI1EbT6IxfEBBsH6c23bTB6OAsLmk?=
 =?iso-8859-1?Q?7gN8Sex5G/xAPO0vHf6G7hE5XL26jl/WvpunGxCi9cmmn34lbUOP0u2TXP?=
 =?iso-8859-1?Q?uxvFzHZxXYnOpGvSvWWKalMb2NfTCyrYeGPMocYBp4mXw6lXS+4mAJEnEY?=
 =?iso-8859-1?Q?9OYGrgKt+J77MHwVbHMo61ZcnPZ84UZILHltNB4ZFI/ohf4lEIIw5gsnXm?=
 =?iso-8859-1?Q?UFM792/XHFBWpuYgLoxibrpirxFJ0lpvGIEFm+J1bTQy8LE4ZNg0UxfFRH?=
 =?iso-8859-1?Q?Ic/sszdaKAZMaMPeLI8bcl71xpmMwLd2ZOwfj9GhGzQ9cQ/fBPLZWXUuem?=
 =?iso-8859-1?Q?nZ40jTvo69Tp0s8cQvs/nQE9MqKLn6XOS/VbgRNL0=3D?=
Content-Type: multipart/alternative;
	boundary="_000_DB9P192MB1434AD2FD1069E39FB480068A6462DB9P192MB1434EURP_"
MIME-Version: 1.0
X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-fb43a.templateTenant
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB9P192MB1434.EURP192.PROD.OUTLOOK.COM
X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-CrossTenant-Network-Message-Id: d48f151e-8de6-4a29-ca8a-08dc270d3d2e
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Feb 2024 12:14:59.4144
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9P192MB1705

--_000_DB9P192MB1434AD2FD1069E39FB480068A6462DB9P192MB1434EURP_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Bug detailed description:

----------------

While booting a Linux Debian 7 "Wheezy" VM, Xen crashes with a FATAL PAGE F=
AULT.



Environment :

----------------

HW: Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz (2 CPU, Xen in nested virtual=
ization upon QEMU/KVM), 4GB RAM

Xen: Xen 4.14.6 (xen-hypervisor-4.14-amd64 package)

Dom0: Linux 5.10.0-26-amd64 (Debian 11 "Bullseye"), 1 CPU, 1024GB RAM



Reproduce steps:

----------------

1.      Install debian 11 and xen package

2.      Boot a clean debian 7 image in hardware-assisted virtualization

3.      Single bit-flip bit number 2 of VMCS field "VM_EXIT_QUALIFICATION" =
(field number 6400). In my case, value changed from [100049] -> [10004d]

4.      Leave the debian7 guest executing after the bit flip.



Current result:

----------------

Xen crash



Console error log:

----------------

(XEN) d1v0 Unexpected PIO status 1, port 0x10 read 0x00000000ffff

(XEN) domain_crash called from io.c:166
(XEN) Domain 1 (vcpu#0) crashed on cpu#1:
(XEN) ----[ Xen-4.14.6  x86_64  debug=3Dn   Not tainted ]----
(XEN) CPU:    1
(XEN) RIP:    0010:[<ffffffff8100712e>]
(XEN) RFLAGS: 0000000000000046   CONTEXT: hvm guest (d1v0)
(XEN) rax: 0000000000000000   rbx: 0000000040000000   rcx: 0000000000000001
(XEN) rdx: 0000000000000000   rsi: ffffffff81666a80   rdi: ffffffff81617830
(XEN) rbp: 0000000000000020   rsp: ffffffff81601e78   r8:  0000000000000200
(XEN) r9:  ffffffff8168f2a0   r10: 0000000000000007   r11: 0000000000000007
(XEN) r12: ffffffff81601f58   r13: ffffffffffffffff   r14: 000000000008c800
(XEN) r15: 0000000000001000   cr0: 0000000080050033   cr4: 00000000001000a0
(XEN) cr3: 0000000001605000   cr2: 0000000000000000
(XEN) fsb: 0000000000000000   gsb: ffffffff81696000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: 0010
(XEN) d1v0 Bad rIP ffffffff8100712e for mode 0
(XEN) domain_crash called from vmx.c:4413
(XEN) ----[ Xen-4.14.6  x86_64  debug=3Dn   Not tainted ]----
(XEN) CPU:    1
(XEN) RIP:    e008:[<ffff82d040206fa9>] __domain_crash+0x9/0x80
(XEN) RFLAGS: 0000000000010296   CONTEXT: hypervisor (d1v0)
(XEN) rax: ffff830139c0506c   rbx: ffff8301308a0000   rcx: 0000000000000000
(XEN) rdx: ffff830136ddffff   rsi: 000000000000000a   rdi: 0000000000000000
(XEN) rbp: 0000000000000000   rsp: ffff830136ddfee0   r8:  0000000000000001
(XEN) r9:  0000000000004000   r10: 0000000000000001   r11: ffff82d040372d40
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000001526e0
(XEN) cr3: 0000000136da6000   cr2: 0000000000000208
(XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen code around <ffff82d040206fa9> (__domain_crash+0x9/0x80):
(XEN)  0f 1e fa 55 48 89 fd 53 <80> bf 08 02 00 00 00 75 2d 48 89 e3 0f b7 =
37 48
(XEN) Xen stack trace from rsp=3Dffff830136ddfee0:
(XEN)    ffff8301308a0000 0000000000000000 ffff82d0402a1798 000000000000100=
0
(XEN)    000000000008c800 ffffffffffffffff ffffffff81601f58 000000000000002=
0
(XEN)    0000000040000000 0000000000000007 0000000000000007 ffffffff8168f2a=
0
(XEN)    0000000000000200 0000000000000000 0000000000000001 000000000000000=
0
(XEN)    ffffffff81666a80 ffffffff81617830 000000fa00000000 ffffffff8100712=
e
(XEN)    0000000000000000 0000000000000046 ffffffff81601e78 000000000000000=
0
(XEN)    0000000000000000 0000000000000000 0000000000000000 000000000000000=
0
(XEN)    0000e01000000001 ffff8301308a0000 00000030f9686000 00000000001526e=
0
(XEN)    0000000000000000 0000000000000000 0006020200000000 000000000000000=
0
(XEN) Xen call trace:
(XEN)    [<ffff82d040206fa9>] R __domain_crash+0x9/0x80
(XEN)    [<ffff82d0402a1798>] S vmx_asm_vmexit_handler+0xf8/0x210
(XEN)
(XEN) Pagetable walk from 0000000000000208:
(XEN)  L4[0x000] =3D 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 1:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=3D0000]
(XEN) Faulting linear address: 0000000000000208
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...



HVM guest config file:

--------------------------------

name =3D "debian7"

builder=3D "hvm"
memory =3D 1024
vcpus =3D 1
cpus=3D ["1"]
disk=3D["qcow2:/home/test/debian7.qcow2,xvda,w"]
vnc =3D 1
vnclisten =3D '127.0.0.1'
vncdisplay =3D 1
boot =3D "d"
serial =3D "pty"

Discussion
--------------------------------

Hello, i am conducting robustness tests on Xen 4.14. I bit flipped the fiel=
d VM_EXIT_QUALIFICATION during the exit handling of an IO_INSTRUCTION. Afte=
r a VM Read on that field, Xen crashes with this error log. This is my (pos=
sible) explanation:


Xen correctly detects an error in the field, resulting in a domain crash (U=
nexpected PIO status 1) called from this point in the code:

xen/arch/x86/hvm/io.c
```
gprintk(XENLOG_ERR, "Unexpected PIO status %d, port %#x %s 0x%0*x\n",
                rc, port, dir =3D=3D IOREQ_WRITE ? "write" : "read",
                size * 2, data & ((1u << (size * 8)) - 1));
domain_crash(curr->domain);
return false;
```

In the `handle_pio` function, in the handling of the `IO_INSTRUCTION` exit =
reason. However, the code continues and ends here for an issue in the proce=
ssor mode:

/xen/arch/x86/hvm/vmx/vmx.c
```
mode =3D vmx_guest_x86_mode(v);
    if ( mode =3D=3D 8 ? !is_canonical_address(regs->rip)
                   : regs->rip !=3D regs->eip )
    {
        gprintk(XENLOG_WARNING, "Bad rIP %lx for mode %u\n", regs->rip, mod=
e);

        if ( vmx_get_cpl() )
        {
            __vmread(VM_ENTRY_INTR_INFO, &intr_info);
            if ( !(intr_info & INTR_INFO_VALID_MASK) )
                hvm_inject_hw_exception(TRAP_gp_fault, 0);
            /* Need to fix rIP nevertheless. */
            if ( mode =3D=3D 8 )
                regs->rip =3D (long)(regs->rip << (64 - VADDR_BITS)) >>
                            (64 - VADDR_BITS);
            else
                regs->rip =3D regs->eip;
        }
        else
            domain_crash(v->domain);
    }
```

However, the domain pointer has already been deallocated due to the previou=
s domain crash, resulting in a page fault that leads to Xen crashing.

I would like to report this crash and ask for your opinion. Based on previo=
us research, I do not believe it has been reported before. I am unsure if i=
t could potentially be a security issue, hence I am posting it here. Let me=
 know if I should provide other results. Thank you for your kind response i=
n advance.



Best Regards,

Giuseppe De Rosa.




--_000_DB9P192MB1434AD2FD1069E39FB480068A6462DB9P192MB1434EURP_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
1">
<style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo=
ttom:0;} </style>
</head>
<body dir=3D"ltr">
<table style=3D"text-align: left; background-color: rgb(233, 234, 234); wid=
th: 100%; color: rgb(51, 51, 51);">
<tbody>
<tr>
<td style=3D"text-align: left;">
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">Bug detailed description:</spa=
n></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">----------------</span></p>
<p class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"font=
-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">While=
 booting a Linux Debian 7 &quot;Wheezy&quot; VM, Xen crashes with a FATAL P=
AGE FAULT.&nbsp;</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">&nbsp;</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">Environment :</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">----------------</span></p>
<p class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"font=
-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">HW: I=
ntel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz (2 CPU, Xen in nested virtualizati=
on upon QEMU/KVM), 4GB RAM</span></p>
<p class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"font=
-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">Xen: =
Xen 4.14.6 (xen-hypervisor-4.14-amd64 package)</span></p>
<p class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"font=
-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">Dom0:=
 Linux 5.10.0-26-amd64 (Debian 11 &quot;Bullseye&quot;), 1 CPU, 1024GB RAM<=
/span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">&nbsp;</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">Reproduce steps:</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">----------------</span></p>
<p class=3D"elementToProof" style=3D"text-align: left; text-indent: -0.25in=
; margin-left: 0.5in;">
<span style=3D"font-family: Arial, sans-serif; font-size: 12px; color: rgb(=
51, 51, 51);">1.</span><span style=3D"font-family: &quot;Times New Roman&qu=
ot;; font-size: 7pt; color: rgb(51, 51, 51); line-height: normal;">&nbsp; &=
nbsp; &nbsp;
</span><span style=3D"font-family: Arial, sans-serif; font-size: 12px; colo=
r: rgb(51, 51, 51); background-color: rgb(233, 234, 234); line-height: norm=
al;">Install debian 11 and xen package</span></p>
<p class=3D"elementToProof" style=3D"text-align: left; text-indent: -0.25in=
; margin-left: 0.5in;">
<span style=3D"font-family: Arial, sans-serif; font-size: 12px; color: rgb(=
51, 51, 51);">2.</span><span style=3D"font-family: &quot;Times New Roman&qu=
ot;; font-size: 7pt; color: rgb(51, 51, 51); line-height: normal;">&nbsp; &=
nbsp; &nbsp;
</span><span style=3D"font-family: Arial, sans-serif; font-size: 12px; colo=
r: rgb(51, 51, 51); line-height: normal;">Boot a clean debian 7 image in ha=
rdware-assisted virtualization</span></p>
<p class=3D"elementToProof" style=3D"text-align: left; text-indent: -0.25in=
; margin-left: 0.5in;">
<span style=3D"font-family: Arial, sans-serif; font-size: 12px; color: rgb(=
51, 51, 51);">3.</span><span style=3D"font-family: &quot;Times New Roman&qu=
ot;; font-size: 7pt; color: rgb(51, 51, 51); line-height: normal;">&nbsp; &=
nbsp; &nbsp;
</span><span style=3D"font-family: Arial, sans-serif; font-size: 12px; colo=
r: rgb(51, 51, 51); line-height: normal;">Single bit-flip bit number 2 of V=
MCS field &quot;VM_EXIT_QUALIFICATION&quot; (field number 6400). In my case=
, value changed from [100049] -&gt; [10004d]</span></p>
<p class=3D"elementToProof" style=3D"text-align: left; text-indent: -0.25in=
; margin-left: 0.5in;">
<span style=3D"font-family: Arial, sans-serif; font-size: 12px; color: rgb(=
51, 51, 51);">4.</span><span style=3D"font-family: &quot;Times New Roman&qu=
ot;; font-size: 7pt; color: rgb(51, 51, 51); line-height: normal;">&nbsp; &=
nbsp; &nbsp;
</span><span style=3D"font-family: Arial, sans-serif; font-size: 12px; colo=
r: rgb(51, 51, 51); line-height: normal;">Leave the debian7 guest executing=
 after the bit flip</span><span style=3D"font-family: Arial, sans-serif; fo=
nt-size: 12px; color: rgb(51, 51, 51);">.</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">&nbsp;</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">Current result:</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">----------------</span></p>
<p class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"font=
-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">Xen c=
rash</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">&nbsp;</span></p>
<p class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"font=
-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">Conso=
le error log:</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">----------------</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">(XEN) d1v0 Unexpected PIO stat=
us 1, port 0x10 read 0x00000000ffff</span></p>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) domain_crash called fr=
om io.c:166</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) Domain 1 (vcpu#0) cras=
hed on cpu#1:</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) ----[ Xen-4.14.6 &nbsp=
;x86_64 &nbsp;debug=3Dn &nbsp; Not tainted ]----</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) CPU: &nbsp; &nbsp;1</s=
pan></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) RIP: &nbsp; &nbsp;0010=
:[&lt;ffffffff8100712e&gt;]</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) RFLAGS: 00000000000000=
46 &nbsp; CONTEXT: hvm guest (d1v0)</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) rax: 0000000000000000 =
&nbsp; rbx: 0000000040000000 &nbsp; rcx: 0000000000000001</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) rdx: 0000000000000000 =
&nbsp; rsi: ffffffff81666a80 &nbsp; rdi: ffffffff81617830</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) rbp: 0000000000000020 =
&nbsp; rsp: ffffffff81601e78 &nbsp; r8: &nbsp;0000000000000200</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) r9: &nbsp;ffffffff8168=
f2a0 &nbsp; r10: 0000000000000007 &nbsp; r11: 0000000000000007</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) r12: ffffffff81601f58 =
&nbsp; r13: ffffffffffffffff &nbsp; r14: 000000000008c800</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) r15: 0000000000001000 =
&nbsp; cr0: 0000000080050033 &nbsp; cr4: 00000000001000a0</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) cr3: 0000000001605000 =
&nbsp; cr2: 0000000000000000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) fsb: 0000000000000000 =
&nbsp; gsb: ffffffff81696000 &nbsp; gss: 0000000000000000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) ds: 0000 &nbsp; es: 00=
00 &nbsp; fs: 0000 &nbsp; gs: 0000 &nbsp; ss: 0000 &nbsp; cs: 0010</span></=
div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) d1v0 Bad rIP ffffffff8=
100712e for mode 0</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) domain_crash called fr=
om vmx.c:4413</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) ----[ Xen-4.14.6 &nbsp=
;x86_64 &nbsp;debug=3Dn &nbsp; Not tainted ]----</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) CPU: &nbsp; &nbsp;1</s=
pan></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) RIP: &nbsp; &nbsp;e008=
:[&lt;ffff82d040206fa9&gt;] __domain_crash+0x9/0x80</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) RFLAGS: 00000000000102=
96 &nbsp; CONTEXT: hypervisor (d1v0)</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) rax: ffff830139c0506c =
&nbsp; rbx: ffff8301308a0000 &nbsp; rcx: 0000000000000000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) rdx: ffff830136ddffff =
&nbsp; rsi: 000000000000000a &nbsp; rdi: 0000000000000000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) rbp: 0000000000000000 =
&nbsp; rsp: ffff830136ddfee0 &nbsp; r8: &nbsp;0000000000000001</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) r9: &nbsp;000000000000=
4000 &nbsp; r10: 0000000000000001 &nbsp; r11: ffff82d040372d40</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) r12: 0000000000000000 =
&nbsp; r13: 0000000000000000 &nbsp; r14: 0000000000000000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) r15: 0000000000000000 =
&nbsp; cr0: 0000000080050033 &nbsp; cr4: 00000000001526e0</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) cr3: 0000000136da6000 =
&nbsp; cr2: 0000000000000208</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) fsb: 0000000000000000 =
&nbsp; gsb: 0000000000000000 &nbsp; gss: 0000000000000000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) ds: 0000 &nbsp; es: 00=
00 &nbsp; fs: 0000 &nbsp; gs: 0000 &nbsp; ss: 0000 &nbsp; cs: e008</span></=
div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) Xen code around &lt;ff=
ff82d040206fa9&gt; (__domain_crash+0x9/0x80):</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp;0f 1e fa 55 48 8=
9 fd 53 &lt;80&gt; bf 08 02 00 00 00 75 2d 48 89 e3 0f b7 37 48</span></div=
>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) Xen stack trace from r=
sp=3Dffff830136ddfee0:</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;ffff83013=
08a0000 0000000000000000 ffff82d0402a1798 0000000000001000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;000000000=
008c800 ffffffffffffffff ffffffff81601f58 0000000000000020</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;000000004=
0000000 0000000000000007 0000000000000007 ffffffff8168f2a0</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;000000000=
0000200 0000000000000000 0000000000000001 0000000000000000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;ffffffff8=
1666a80 ffffffff81617830 000000fa00000000 ffffffff8100712e</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;000000000=
0000000 0000000000000046 ffffffff81601e78 0000000000000000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;000000000=
0000000 0000000000000000 0000000000000000 0000000000000000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;0000e0100=
0000001 ffff8301308a0000 00000030f9686000 00000000001526e0</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;000000000=
0000000 0000000000000000 0006020200000000 0000000000000000</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) Xen call trace:</span>=
</div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;[&lt;ffff=
82d040206fa9&gt;] R __domain_crash+0x9/0x80</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp; &nbsp;[&lt;ffff=
82d0402a1798&gt;] S vmx_asm_vmexit_handler+0xf8/0x210</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN)</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) Pagetable walk from 00=
00000000000208:</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) &nbsp;L4[0x000] =3D 00=
00000000000000 ffffffffffffffff</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN)</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) **********************=
******************</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) Panic on CPU 1:</span>=
</div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) FATAL PAGE FAULT</span=
></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) [error_code=3D0000]</s=
pan></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) Faulting linear addres=
s: 0000000000000208</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) **********************=
******************</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN)</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">(XEN) Reboot in five seconds=
...</span></div>
<p class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"font=
-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp=
;</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">HVM guest config file:</span><=
/p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">------------------------------=
--</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">name =3D &quot;debian7&quot;</=
span></p>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">builder=3D &quot;hvm&quot;</=
span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">memory =3D 1024</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">vcpus =3D 1</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">cpus=3D [&quot;1&quot;]</spa=
n></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">disk=3D[&quot;qcow2:/home/te=
st/debian7.qcow2,xvda,w&quot;]</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">vnc =3D 1</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">vnclisten =3D '127.0.0.1'</s=
pan></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">vncdisplay =3D 1</span></div=
>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">boo=
t =3D &quot;d&quot;</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">ser=
ial =3D &quot;pty&quot;</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);"><br=
>
</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">Dis=
cussion</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51); back=
ground-color: rgb(233, 234, 234);">--------------------------------</span><=
/div>
<p class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"font=
-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">Hello=
, i am conducting robustness tests on Xen 4.14. I bit flipped the field VM_=
EXIT_QUALIFICATION during the exit handling
 of an IO_INSTRUCTION. After a VM Read on that field, Xen crashes with this=
 error log. This is my (possible) explanation:<br>
<br>
</span></p>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">Xen correctly detects an err=
or in the field, resulting in a domain crash (Unexpected PIO status 1) call=
ed from this point in the code:</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);"><br=
>
xen/arch/x86/hvm/io.c</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51); back=
ground-color: rgb(233, 234, 234);">```</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">gprintk(XENLOG_ERR, &quot;Un=
expected PIO status %d, port %#x %s 0x%0*x\n&quot;,</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; rc, port, dir =3D=3D IOREQ_WRITE ? &quot;write&=
quot; : &quot;read&quot;,</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; size * 2, data &amp; ((1u &lt;&lt; (size * 8)) =
- 1));</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">domain_crash(curr-&gt;domain=
);</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">ret=
urn false;</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51); back=
ground-color: rgb(233, 234, 234);">```</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);"><br>
</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">In the `handle_pio` function=
, in the handling of the `IO_INSTRUCTION` exit reason. However, the code co=
ntinues and ends here for an issue in
 the processor mode:</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);"><br>
</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">/xe=
n/arch/x86/hvm/vmx/vmx.c</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51); back=
ground-color: rgb(233, 234, 234);">```</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">mode =3D vmx_guest_x86_mode(=
v);</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; if ( mode =3D=
=3D 8 ? !is_canonical_address(regs-&gt;rip)</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;: regs-&gt;rip !=3D regs-&gt;eip )=
</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; {</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
gprintk(XENLOG_WARNING, &quot;Bad rIP %lx for mode %u\n&quot;, regs-&gt;rip=
, mode);</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);"><br>
</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
if ( vmx_get_cpl() )</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
{</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; __vmread(VM_ENTRY_INTR_INFO, &amp;intr_info);</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; if ( !(intr_info &amp; INTR_INFO_VALID_MASK) )</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; hvm_inject_hw_exception(TRAP_gp_fault, 0);</spa=
n></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; /* Need to fix rIP nevertheless. */</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; if ( mode =3D=3D 8 )</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; regs-&gt;rip =3D (long)(regs-&gt;rip &lt;&lt; (=
64 - VADDR_BITS)) &gt;&gt;</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; (64 -=
 VADDR_BITS);</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; else</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; regs-&gt;rip =3D regs-&gt;eip;</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
}</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
else</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; domain_crash(v-&gt;domain);</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp; &nbsp; }</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">```</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);"><br>
</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);">However, the domain pointer =
has already been deallocated due to the previous domain crash, resulting in=
 a page fault that leads to Xen crashing.</span></div>
<div style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-se=
rif; font-size: 12px; color: rgb(51, 51, 51);"><br>
</span></div>
<div class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">I w=
ould like to report this crash and ask for your opinion. Based on previous =
research, I do not believe it has been
 reported before. I am unsure if it could potentially be a security issue, =
hence I am posting it here. Let me know if I should provide other results. =
Thank you for your kind response in advance.</span></div>
<p style=3D"text-align: left;" class=3D"elementToProof"><span style=3D"font=
-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">&nbsp=
;</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">Best Regards,</span></p>
<p class=3D"elementToProof" style=3D"text-align: left;"><span style=3D"font=
-family: Arial, sans-serif; font-size: 12px; color: rgb(51, 51, 51);">Giuse=
ppe De Rosa.</span></p>
<p style=3D"text-align: left;"><span style=3D"font-family: Arial, sans-seri=
f; font-size: 12px; color: rgb(51, 51, 51);">&nbsp;</span></p>
</td>
</tr>
</tbody>
</table>
<div class=3D"elementToProof" style=3D"font-family: Calibri, Helvetica, san=
s-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
</body>
</html>

--_000_DB9P192MB1434AD2FD1069E39FB480068A6462DB9P192MB1434EURP_--


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 13:08:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 13:08:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676905.1053239 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLBh-0003Th-RG; Tue, 06 Feb 2024 13:08:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676905.1053239; Tue, 06 Feb 2024 13:08:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLBh-0003Ta-OH; Tue, 06 Feb 2024 13:08:33 +0000
Received: by outflank-mailman (input) for mailman id 676905;
 Tue, 06 Feb 2024 13:08:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+UH4=JP=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXLBg-0003SG-FQ
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 13:08:32 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d3088b51-c4f0-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 14:08:31 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a354408e6bfso95762166b.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 05:08:30 -0800 (PST)
Received: from [10.80.67.148] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 mj17-20020a170906af9100b00a37624d003fsm1116768ejb.121.2024.02.06.05.08.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 05:08:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d3088b51-c4f0-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707224910; x=1707829710; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pPUjJfvGfFIgin1mthrOyZ/H9kzL2CwZtV4akxtGA/c=;
        b=j7Tl168QRHMl/a6u8LRR7+Ys6Nssbn81CmQKh1EPnU8Sm7e2xWCkBS7iztJfawvtnG
         Gh4gGXV9SCFQqiPpGhEf5lS9gmMKQwVvPp6MYEkYKdXSfqJv+FsRqgMK/epq3kzaFIqd
         KWfQ814OuAfOtN95FjgsFldTN8RQwDq63X5Z4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707224910; x=1707829710;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pPUjJfvGfFIgin1mthrOyZ/H9kzL2CwZtV4akxtGA/c=;
        b=WSezzLY9ayS+CnK2yqaLesRkKZgW13iCB0Nu4x9aC4H/sD4iVqMzqSwxYO7GGU9lY8
         Y34+1YmO+37vOMjdMtKKjxc5Btcz/1oan021l+yX7IzMcliMaK94qYrkt0ubcrKFT+t8
         bTvMDPmIySkbN9cGGkhR9k7skasZ47w7hKA5DqLPuUnjvJzfP3dijq/2PgZkq2YdEs87
         bGaVtZSNnIeuuts07aH9WIzhrbSugEDQJEMuLKgUb8V0gSN3Zs50cBia4gh5GMhklHHf
         JdLvB1F9iPhvwKDqf5B1mqJP/pFPXgbTw6LK1JgCY2RrwGvDJ0UNxSG45YRfi6pEBXlX
         JpMw==
X-Gm-Message-State: AOJu0Yw9SRrTuHxKm1Faet+A52xRv3hC8Rfzt9KCtlRBQoXZfYngUn54
	NgpXlnUTSqHH1Eb3TaAFBpqbmE7cblcOiRjcMXm+ASCq4H3Qb7xy/3XAjCsCvhg=
X-Google-Smtp-Source: AGHT+IFT7X840P0xvonXmToZnsZqaBgCt/QbNUpG/852PEnpG/RZozuPUzDhOWCxU2LcVZQca6zM2g==
X-Received: by 2002:a17:906:3095:b0:a38:158b:bbbc with SMTP id 21-20020a170906309500b00a38158bbbbcmr1614684ejv.38.1707224910174;
        Tue, 06 Feb 2024 05:08:30 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWPmPkta+pSO0SkANVWsFLOdp3LzjZkSBWZJKswfFTp5rG9PcqswDyDL32forJS8HDlq2G9BSCnRIIjhctYT7bS0EF1hY1xaKPxyMvzB1lBCcM31NnUzGohSnHA6TvsRXu93dx0wlTd3GE=
Message-ID: <d420a9c2-3fa0-40cc-adf7-ffd88fcffbfe@citrix.com>
Date: Tue, 6 Feb 2024 13:08:29 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/libs/light: don't allow to stop Xenstore stubdom
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240206124345.19433-1-jgross@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240206124345.19433-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06/02/2024 12:43 pm, Juergen Gross wrote:
> A Xenstore stubdom should never be stoppable.
>
> Reject attempts to do so.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>

I don't think this is a clever idea.  `xl destroy` is also the "please
clean up my system when it's in a very dead state" command, and that
also includes a dead xenstored stubdom.

If you're looking for some protection, then maybe a `--force` flag to
override, but there must be some way of getting this to run.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 13:10:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 13:10:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676908.1053248 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLDW-0004yu-6H; Tue, 06 Feb 2024 13:10:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676908.1053248; Tue, 06 Feb 2024 13:10:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLDW-0004yn-3D; Tue, 06 Feb 2024 13:10:26 +0000
Received: by outflank-mailman (input) for mailman id 676908;
 Tue, 06 Feb 2024 13:10:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXLDU-0004yh-Q0
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 13:10:24 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 15986c18-c4f1-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 14:10:22 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-51117bfd452so9396383e87.3
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 05:10:22 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f3-20020a05621400c300b0068c67e305edsm1009470qvs.49.2024.02.06.05.10.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 05:10:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15986c18-c4f1-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707225022; x=1707829822; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Sbu5zSZw+7rAbIx1HfYcvSHGwrUqEhEUfT+cea6T//M=;
        b=U9wzEGdHOLZ6fze3gLyl+BDvF7uSCLzd1XsJgP2+ZwlG0TXIU4ICMqEqPFn/eGW2Qo
         BEFFrbrLkeZTUkG6RZluopVQNuCpgm9IQMoT11Es9vBHVLrkbC2KcqvoPGRT2rgBFVoC
         ETCfQzYdrHbyPP4IMMseZR9jBr7hZDU4H5TksRWKIXLlCer4O9ewL2hal3T0Ky7YrYII
         pHjj5VUE/yk3b6acxSN8NbxJCIi6WmmeEnZ9kVTZB9CgwFm2ngAjdvQUtJiL7dWDrU8P
         PtZ1eAaC6d7n5oYqvb81pBz64bRI1BWpbrcgG/EVJOnpcuYFJcKN72cyhNl/zwK7vsmb
         dedA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707225022; x=1707829822;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Sbu5zSZw+7rAbIx1HfYcvSHGwrUqEhEUfT+cea6T//M=;
        b=Dkm0bj/fWlPzVxuTs5G/9cXldXqzy/BjezKOu7sCoYzYLYMHveWO4cLyOrfP0XzzDI
         QibJaKJGjVhL+AMY5c5z3bPOoem4SNECgNwOP+FyXgXNUnbgHc+oziIOqkXluazQH78S
         I/iMwoqwyQSouYauAl1gueC4aN9l39NUPe83I8K/b443TGBTF6IElaWMMz4vxEPA0DkW
         ibogpMBuPIqcinE2Qd6UjqKrYu4N7vIfuVLEAewVuKSmQmfGbhfdBrVNFnFMv7KqVI7x
         ZvCnA8p3xW+7MiCg79cwGOVaLEfRPD3w6LPiSJ76ATCbLb17AWkecfW5HEXjW4Urm35o
         e/nA==
X-Gm-Message-State: AOJu0YyuZmpVtpCNgQvVivQ86lbwTSNlR1ud2+gclYywZYDS8+qrhQX8
	7qhZSyLqZwl3P47U5xw1Uo6ZCzd1P9CWzaLptwjajetpCFlaFncBayAjtW9Z9Q==
X-Google-Smtp-Source: AGHT+IGX/rwwY+EG7JQeSwLT2bTjHMAizpUmw51UJ/nnZSwfFjzjLodVx/JYFa/z7KFd1wzmaBV5WQ==
X-Received: by 2002:ac2:5a1a:0:b0:511:4307:f6a4 with SMTP id q26-20020ac25a1a000000b005114307f6a4mr1604777lfn.60.1707225021965;
        Tue, 06 Feb 2024 05:10:21 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUXkiX9vw/MX7PahvaCwNgEqnVwwfSLBtzhTgBDFrKGQjBd3rgOjISp3BePnTd5IKxQjF2wLqQZE5qv490af66h+DVJmodA6mHJvTGyGOsPWlA5kzpfaxCjTe305iV+zaYpUyFfSfKCrTEt8OLSfLEYLgfEzetLY1bcrQwhFIvz2Gq1lVlzguk3CnueF5Rc1WyCmQkbCRg6QetB+vca51dSP1SZxpR6hdBndubMhmmO/zqvSAzqTTm8mOCLOYB4kLdPw2Ymp5+xs/j1FDCuanz81QRmT+RsLT1KHQ==
Message-ID: <2c897009-8cc0-4ea4-90b1-a6af1512b58e@suse.com>
Date: Tue, 6 Feb 2024 14:10:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 2/4] xen/ns16550: address violations of MISRA
 C:2012 Rule 13.1
To: Simone Ballarin <simone.ballarin@bugseng.com>
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <08f1d4a7475c557da016482025067f54e2e56b99.1706886631.git.simone.ballarin@bugseng.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <08f1d4a7475c557da016482025067f54e2e56b99.1706886631.git.simone.ballarin@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 16:16, Simone Ballarin wrote:
> Rule 13.1: Initializer lists shall not contain persistent side effects
> 
> The assignment operation in:
> 
> .irq = rc = uart->irq,
> 
> is a persistent side effect in a struct initializer list.
> 
> This patch assigns rc separately outside the structure.
> 
> No functional change.
> 
> Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
> Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Nevertheless I think this would have benefited from limiting rc's
scope.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 13:14:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 13:14:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676911.1053259 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLGu-0005dh-Kz; Tue, 06 Feb 2024 13:13:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676911.1053259; Tue, 06 Feb 2024 13:13:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLGu-0005da-Hd; Tue, 06 Feb 2024 13:13:56 +0000
Received: by outflank-mailman (input) for mailman id 676911;
 Tue, 06 Feb 2024 13:13:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXLGs-0005dD-Co
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 13:13:54 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 92274308-c4f1-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 14:13:51 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-40f02b8d176so48632555e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 05:13:52 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 fc19-20020a05600c525300b0040fcaec04b8sm1996152wmb.22.2024.02.06.05.13.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 05:13:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92274308-c4f1-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707225231; x=1707830031; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=w/RTkzwejvO8GT5sxjE1uL/S79JQp+vP3t+IpcqvOSI=;
        b=Ee0Td7hBxJrBVk2VlNHu2BNxBhHTitSiKBhWUoO0tyI+ZdgqxDG1E0MM67RjXfn20l
         k1DkqkU2xJBG9fPV2agJV+X1ueQXyOaK09WunF7ZUWX6pLuwlt+Qrv25eJ6Q7vBCmjcR
         iW0zvcdXH6v7YVXHPeLMxbEmnTSDB3zmqnjuDPh59/dy0MOENzYnYq/N3hRBChJ/9HVs
         Yh1moiTIE9QcXUNt46RzE0Q0QoJgn0UzHLzlDR5FXcnXl/sENJzjUCEUjok35arqKv1z
         7BO+54+PCjB2eIysbxVcq33O7jLaxuTmg0CR0/T4Stnrgk8mjBT8Nn6ptZ3hkyzVFHoD
         wwdw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707225231; x=1707830031;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=w/RTkzwejvO8GT5sxjE1uL/S79JQp+vP3t+IpcqvOSI=;
        b=BXbu46f+/grZfVRQwI4BNAExmdNvZPChdoGwUmWpIg+3lz1mWvdnOCYTExQxBHg0Fr
         Q30ErUbzKq2JvsBQWG8GRkQ4nePNO/HgnrdxpS0bL35mO3SOUF1xXs1q+nkATErBIygj
         pInT/ijnmLtn6IyB7lwJQtGlZFM6IfnT6xAjmJjOBJJ/vIcdWklMtyTF25i3MgMrH/F1
         YtLeo6ZNgRaf3xRtwJIi1ut587teLXDQTvDOMPSLYI66DSJHsg0Q33biIPr4dXFr3W7E
         opkkiG2fOwHtPJYCtivGyp9p9/ypcT1pM3nyGvxvYS+vpLy+iorvvHr2Lx8SH4JNokkz
         16cQ==
X-Gm-Message-State: AOJu0Yw/I3GhV+l/rfZUK42dCMt9rbqyiL/2/IU9qcdAYLNM9QPxmT7z
	dsJSFloPP04p/7uD8k7HOFZZd1lYv5/re6bxk0hj2HePj/1J2gzPVNCPbb4BAw==
X-Google-Smtp-Source: AGHT+IENXEYRmfoz3vzYurbgEB9BslGMtuJvh8RijiU02++tGp9rzba9fbp+S95ZlW8JktcIFDuhRQ==
X-Received: by 2002:a05:600c:4f53:b0:40e:f693:94f6 with SMTP id m19-20020a05600c4f5300b0040ef69394f6mr1962721wmq.35.1707225231676;
        Tue, 06 Feb 2024 05:13:51 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVgbweWNOu1465ZembDIY5p07eJyABHlEzCe31bKXNZR2ClswGWLF9LCuOqpQTDwHzERb88dIrQMATBlHi6BhbttRnRzxJPsxJtyIddzWzbgrIOKX/bpfHS5Dy/PEkVwJLZPiKzPTvkOfotxXBGDESGrZwz9jFR2gbRPOCAW2a16ovUsf8SfvEGYVcVIHqnJD3gDY23JzHqoKTWiRGmuvJYd3ci
Message-ID: <b23ffbf2-e1b1-42f5-b0ea-9f0889a5a7af@suse.com>
Date: Tue, 6 Feb 2024 14:13:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 3/4] xen/x86: address violations of MISRA C:2012
 Rule 13.1
Content-Language: en-US
To: Simone Ballarin <simone.ballarin@bugseng.com>
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <16bb514ac0a5fe0d6e9a2c95279a8200ff4495c6.1706886631.git.simone.ballarin@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <16bb514ac0a5fe0d6e9a2c95279a8200ff4495c6.1706886631.git.simone.ballarin@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 16:16, Simone Ballarin wrote:
> Rule 13.1: Initializer lists shall not contain persistent side effects
> 
> This patch moves expressions with side-effects into new variables before
> the initializer lists.
> 
> No functional changes.
> 
> Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>

To be honest, I don't like this. It's more code for no gain. Really its
hampering clarity imo. I'm willing to be convinced otherwise, but for
now this gets a nack from me.

As an aside, ...

> --- a/xen/arch/x86/io_apic.c
> +++ b/xen/arch/x86/io_apic.c
> @@ -2559,9 +2559,12 @@ integer_param("max_gsi_irqs", max_gsi_irqs);
>  
>  static __init bool bad_ioapic_register(unsigned int idx)
>  {
> -    union IO_APIC_reg_00 reg_00 = { .raw = io_apic_read(idx, 0) };
> -    union IO_APIC_reg_01 reg_01 = { .raw = io_apic_read(idx, 1) };
> -    union IO_APIC_reg_02 reg_02 = { .raw = io_apic_read(idx, 2) };
> +    uint32_t reg_00_raw = io_apic_read(idx, 0);
> +    uint32_t reg_01_raw = io_apic_read(idx, 1);
> +    uint32_t reg_02_raw = io_apic_read(idx, 2);

... while you properly use uint32_t here, ...

> +    union IO_APIC_reg_00 reg_00 = { .raw = reg_00_raw };
> +    union IO_APIC_reg_01 reg_01 = { .raw = reg_01_raw };
> +    union IO_APIC_reg_02 reg_02 = { .raw = reg_02_raw };
>  
>      if ( reg_00.raw == -1 && reg_01.raw == -1 && reg_02.raw == -1 )
>      {
> --- a/xen/arch/x86/mpparse.c
> +++ b/xen/arch/x86/mpparse.c
> @@ -798,11 +798,12 @@ void __init mp_register_lapic_address (
>  
>  int mp_register_lapic(u32 id, bool enabled, bool hotplug)
>  {
> +	u32 apic = apic_read(APIC_LVR);

... why the being-phased-out u32 here?

Jan

>  	struct mpc_config_processor processor = {
>  		.mpc_type = MP_PROCESSOR,
>  		/* Note: We don't fill in fields not consumed anywhere. */
>  		.mpc_apicid = id,
> -		.mpc_apicver = GET_APIC_VERSION(apic_read(APIC_LVR)),
> +		.mpc_apicver = GET_APIC_VERSION(apic),
>  		.mpc_cpuflag = (enabled ? CPU_ENABLED : 0) |
>  			       (id == boot_cpu_physical_apicid ?
>  				CPU_BOOTPROCESSOR : 0),
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -885,13 +885,14 @@ static struct domain *__init create_dom0(const module_t *image,
>  {
>      static char __initdata cmdline[MAX_GUEST_CMDLINE];
>  
> +    unsigned int max_vcpus = dom0_max_vcpus();
>      struct xen_domctl_createdomain dom0_cfg = {
>          .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
>          .max_evtchn_port = -1,
>          .max_grant_frames = -1,
>          .max_maptrack_frames = -1,
>          .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version),
> -        .max_vcpus = dom0_max_vcpus(),
> +        .max_vcpus = max_vcpus,
>          .arch = {
>              .misc_flags = opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0,
>          },



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 13:20:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 13:20:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676920.1053268 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLMl-0006lz-85; Tue, 06 Feb 2024 13:19:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676920.1053268; Tue, 06 Feb 2024 13:19:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLMl-0006ls-57; Tue, 06 Feb 2024 13:19:59 +0000
Received: by outflank-mailman (input) for mailman id 676920;
 Tue, 06 Feb 2024 13:19:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=60ZD=JP=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1rXLMk-0006lm-Ne
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 13:19:58 +0000
Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com
 [136.143.188.12]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6b18bdb0-c4f2-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 14:19:57 +0100 (CET)
Received: from mail-yw1-f179.google.com (mail-yw1-f179.google.com
 [209.85.128.179]) by mx.zohomail.com
 with SMTPS id 1707225592988261.1018244172735;
 Tue, 6 Feb 2024 05:19:52 -0800 (PST)
Received: by mail-yw1-f179.google.com with SMTP id
 00721157ae682-6047cfd1f5cso5137627b3.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 05:19:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b18bdb0-c4f2-11ee-8a47-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; t=1707225594; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=cL02Ku0IQtKMqV/NTJie9lYsLUetZX4DOhwQnft6FXOD7upXALl7MzJbO4h0OX+R6GnkfO2W9iGE03xMyfxlcFFYQ3HVAAOFR/sFTuWuKH3hSYZ49Ds7AH2flXtuMOk5B0zTyGWExAoKsB035IS0ip3y/g4r/df1hu0N9EauY/U=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1707225594; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=AcLOC/B+zC/6qFNeSnAd6vo5Xv4mxXGkXuFcuaV4UJw=; 
	b=EuO1DaqpziDI1J5oFH9eQS0H5svK+ZpPe6O6AJwupGm+2AKAyRBcuiViXvS5DxaOXop/GlI5UUqe9et09DWW78xapJ5c0hIk567My+yNIAId8mYSYDXIr1Oi3VfbBKAfoSumDa7o8cVs7j8a2dnnjzW66XQ+sy0thRGTkHYtHyo=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1707225594;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=AcLOC/B+zC/6qFNeSnAd6vo5Xv4mxXGkXuFcuaV4UJw=;
	b=N8LCTPGbSgB2LJRJqMbBIEdgM1mgFxt3Lvw5jMmqEvJl36c+KfVribOowDky4Ag9
	48N62H4m7nSp0KjhxzQzee70ftfHaaO8qHXNKnwZoVt7GZSB/U9g1qYxNIYj8XUYQZG
	oYqeHMlH9P8WdcX7lFJeySGABHBTnqclHAJavFug=
X-Gm-Message-State: AOJu0YzTBnDNZNG7rha8BujhAnfkCfg1HcZ2mdLG+j7Q4At3BJI1+sHA
	5NDztyOB96Zk2PRQkFg2hZaU/SwrdFjJ6K0OcT+ZcrTfxT1Na6Oh+Q29e8E19/3litlYhwRIQTD
	E4HL2sBL7nSeF5SMeEJHmwRxV4L8=
X-Google-Smtp-Source: AGHT+IGoYFQfrjNcmQfvWdJFAmAU4/LKUO44kppH4PWiT0XTvp3pvHQNBlTd7uxRAN6c9gNhjGnzF+uvEp7mXkK4m48=
X-Received: by 2002:a81:b3c7:0:b0:603:bc00:b469 with SMTP id
 r190-20020a81b3c7000000b00603bc00b469mr1874263ywh.11.1707225591949; Tue, 06
 Feb 2024 05:19:51 -0800 (PST)
MIME-Version: 1.0
References: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
In-Reply-To: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Tue, 6 Feb 2024 08:19:16 -0500
X-Gmail-Original-Message-ID: <CABfawh=V14gV14McU_5mhb+714QCjXA1+G3y5WQmSMeFJVYKBQ@mail.gmail.com>
Message-ID: <CABfawh=V14gV14McU_5mhb+714QCjXA1+G3y5WQmSMeFJVYKBQ@mail.gmail.com>
Subject: Re: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor ap2m->default_access
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: xen-devel@lists.xenproject.org, Alexandru Isaila <aisaila@bitdefender.com>, 
	Petre Pircalabu <ppircalabu@bitdefender.com>, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 6, 2024 at 5:08=E2=80=AFAM Petr Bene=C5=A1 <w1benny@gmail.com> =
wrote:
>
> From: Petr Bene=C5=A1 <w1benny@gmail.com>
>
> This patch addresses a behavior discrepancy in the handling of altp2m vie=
ws,
> where upon the creation and subsequent EPT violation, the page access
> permissions were incorrectly inherited from the hostp2m instead of respec=
ting
> the altp2m default_access.
>
> Previously, when a new altp2m view was established with restrictive
> default_access permissions and activated via xc_altp2m_switch_to_view(),
> it failed to trigger an event on the first access violation.  This behavi=
or
> diverged from the intended mechanism, where the altp2m's default_access
> should dictate the initial permissions, ensuring proper event triggering =
on
> access violations.
>
> The correction involves modifying the handling mechanism to respect the
> altp2m view's default_access upon its activation, eliminating the need fo=
r
> setting memory access permissions for the entire altp2m range (e.g. withi=
n
> xen-access.c).  This change not only aligns the behavior with the expecte=
d
> access control logic but also results in a significant performance improv=
ement
> by reducing the overhead associated with setting memory access permission=
s
> across the altp2m range.
>
> Signed-off-by: Petr Bene=C5=A1 <w1benny@gmail.com>

Acked-by: Tamas K Lengyel <tamas@tklengyel.com>


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 13:22:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 13:22:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676925.1053278 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLPH-0008AE-Jx; Tue, 06 Feb 2024 13:22:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676925.1053278; Tue, 06 Feb 2024 13:22:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLPH-0008A7-HK; Tue, 06 Feb 2024 13:22:35 +0000
Received: by outflank-mailman (input) for mailman id 676925;
 Tue, 06 Feb 2024 13:22:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXLPG-0008A1-1g
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 13:22:34 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c7ce4b5f-c4f2-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 14:22:31 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-40ff2838851so1689085e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 05:22:31 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e5-20020adfef05000000b0033b4a77b2c7sm438068wro.82.2024.02.06.05.22.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 05:22:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7ce4b5f-c4f2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707225751; x=1707830551; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=neL3aSBf46/Z9wmi5MvCpK5OMpLGZ272JdpQZUqJSPo=;
        b=M23GvP8ZF/WaBuu+mPB+HNR79r3bAusu8Xt5GUgmh3yMdyD0RM77wn9YqXIOIy+v4A
         U8XEqTVITQw6Ugk0xw6p7OCCz8qFe7j8N37gzMhK29ILgCnYctDlx5hf8nM0F8ljBnBY
         QM1W5njLo32LDCvq1PhGLBL+y2Gk0U1port1Ksk3r/Zp9jl29+isqg+l/0vkGeh2atKU
         5DVXvEg3b4MrYgmvtV7Hm49ncOJRtJkIzj+mlovwzvyhzPyLU309LBzYHshMbSkMQcGh
         Cw9lDuWGrpSFcXHOtllS30wuXxSjXmoJOhPcyrz1LzEh4t60+mKym+SzxmYqfudwiS7d
         K55A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707225751; x=1707830551;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=neL3aSBf46/Z9wmi5MvCpK5OMpLGZ272JdpQZUqJSPo=;
        b=rzR4X1KfaNJAVEOSc/o6BtlwcrZ8tFF6QuAvwU60QCj+AoIBmnm7MyBX0owi0UO0EW
         iUY8zB8ZYqcOTkokpdpcXzGtCp94hB7gyZBgzqhJmkXP1kal81vgeqIk8zHEvqUtvuup
         hpSmkusYJTdtjVGq/Ap3/CWBj4Bf8+9ma+Y0ysd+m1yAYTPwblM41Ynz+M1TOXHoxENQ
         f26tvbdsDSAIMyMJEcqqyzEXTvbOrikuByAuAnvFXqEFYdOMKHAuviKtiexY5SvGqHDO
         XHxxxDMLSTUZafpGDgQdZYco6QZBVAGP5F0vEcv5n8RGE0GUnVFIxT2PBc0OuLtcJPpe
         i7HQ==
X-Gm-Message-State: AOJu0Ywl/Tyafn6RakQzMu24bIU/OYlQ0HcP2NQkuAPlBb7/npoAQYz1
	iP1M5IxcBeNQ4JwwYHzQX4GKIc8b1kcplDEaYmA7QAkpqgR7vtoxbJdq2COoMA==
X-Google-Smtp-Source: AGHT+IFgghMx5fJeiYPe7TiLRKzwCNWZNPDuoDTlFzWQ57aX+Zf6kdC5yo8KZm9WnugTk7oX+nwpaQ==
X-Received: by 2002:a05:600c:1d89:b0:40f:ecd4:7ef0 with SMTP id p9-20020a05600c1d8900b0040fecd47ef0mr852699wms.24.1707225751273;
        Tue, 06 Feb 2024 05:22:31 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUPUns1S8Hbyixg0G0muhZX2BQ6zfZFjqjf9jCZPQMMFanszw3w0SIvU8sP5j4fdUsUzyU4qu+3BZkS1sWYxuLQmAJJTOrdMp0vTIF97fZkcl9Pzdbxd3kDMRnRj388jWQAZzpC9r6wzxfo+drxaLrrXntwrCKtJIwsd7ox2RjQw+dz7As/o9j/ULVHDUfQ7kfoAkT+X/RmRcsX7kwgpBquUZSIldLCRdHsZ8n6AMcpK/VI+nHPrco=
Message-ID: <2c0d22be-0b31-4e62-a8f8-2dc82147e51a@suse.com>
Date: Tue, 6 Feb 2024 14:22:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <42fc6ae8d3eb802429d29c774502ff232340dc84.1706259490.git.federico.serafini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <42fc6ae8d3eb802429d29c774502ff232340dc84.1706259490.git.federico.serafini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.01.2024 11:05, Federico Serafini wrote:> --- a/xen/include/xen/compiler.h
> +++ b/xen/include/xen/compiler.h
> @@ -64,6 +64,13 @@
>  # define fallthrough        do {} while (0)  /* fallthrough */
>  #endif
>  
> +/*
> + * Add the following macro to check that a program point is considered
> + * unreachable by the static analysis performed by the compiler.
> + */
> +#define STATIC_ASSERT_UNREACHABLE() \
> +    asm ( ".error \"static assertion failed: unreachable\"" )

In the comment s/Add/Use/? The macro is there after all when this gets
committed. Overall maybe

"Use this macro at program points considered unreachable, to be checked
 by the compiler's static analysis."

?

Also while asm()s without operands are implicitly volatile, I think it
would be a good idea to make that explicit nevertheless.

I'd be happy to adjust while committing, so long as you agree. If you
agree, and provided diagnostics resulting from this are useful (an
example would have been nice in the description):
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 13:25:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 13:25:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676931.1053289 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLSG-0000Nh-5y; Tue, 06 Feb 2024 13:25:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676931.1053289; Tue, 06 Feb 2024 13:25:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLSG-0000Na-1t; Tue, 06 Feb 2024 13:25:40 +0000
Received: by outflank-mailman (input) for mailman id 676931;
 Tue, 06 Feb 2024 13:25:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXLSE-0000NU-9t
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 13:25:38 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 35b6e8dc-c4f3-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 14:25:35 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-33b409fc4aeso1261184f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 05:25:36 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n8-20020a5d4c48000000b0033afe816977sm2086647wrt.66.2024.02.06.05.25.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 05:25:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35b6e8dc-c4f3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707225935; x=1707830735; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HwxfR3P3szSE5y1QYgAVF9WnC4T0JlErkuO1LusfE44=;
        b=QZnCONj+AXpAwWa+T9Fl/QwoMeKp/EPNsSbjKkjhq6A9dwUreiDY568CEvPnJQm4qe
         sdeMyDGGYP7w9i/tjO8fQFzbp58vA2X39q9YgpCjgW3KcqP+GTfY8AiKmCC2Zjq+oimh
         HexHBJ5kIY5o3WGlPte8rjk5FQ/Sf8hJVWw41kkodpN1rzmA+7+h2v7MTO5twlLIHiqF
         Zgr0c4hITx8rpyK8FXe+jldTDpdCo/V8Q8H7yH3dyCwjJ7InQmY7Wj2QZvrcaCX9OY0G
         TZfLfDuU7C9+4nRX8WyVzLllLlLzZsp1FSCWY14/MfUJH+/1iU15HFJdvMe8L3Zo89Aw
         aJ8g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707225935; x=1707830735;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HwxfR3P3szSE5y1QYgAVF9WnC4T0JlErkuO1LusfE44=;
        b=guUO/yclSI0+li0cf1s/Mmr0VAGfbsEtzJoiRi6fpUC0SSv79mqPnstFniskPztkS7
         rDcf2/85arbYpkMp2FU3Jdf1e4s8HjnU6u6gi07F9NOQebmxexu6PiwWnN+Wf/RRthiD
         gHoS5w6BrcWZ6XqW/uqsYWYF4jbTyTNvpAxc8E3gmeO0cdG8nzuIhiG7qiCZK8RvxxMx
         OIi3rX/y6nZvYZmNEeUKpsmfQ5BouX2gIhPfoOKgEp98rpciQ56AYbQYwifVpKLpAHCG
         qvVQroP3eOW5gjPtotbeBxRVul8fY+bKr2EnNqWbpXPyhSwQnKePcjPv8uW0E3Xi80od
         Hcrw==
X-Gm-Message-State: AOJu0Yy8EyWVtPLV8vy1gGYGxRT4VhZw1JXN7JCWhi1knAr5SJomhJsl
	BE0CmHplf9C0s7H9DAU4CQxI2L/+ybSs/RHCRsovTi/hUDVV9vKX2uZCtGvoxQ==
X-Google-Smtp-Source: AGHT+IFvkXG3gKt0NtUfEQjtYZl+LTSpno0oEWlTA9mv/VuH5fDr9sqg2jL6QMt5nVrwVpNnMtGuew==
X-Received: by 2002:a5d:56d2:0:b0:33a:ff67:74a with SMTP id m18-20020a5d56d2000000b0033aff67074amr1166247wrw.63.1707225935672;
        Tue, 06 Feb 2024 05:25:35 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUdV8hCV5AQH6Xz8TCSjYEJcOMQoXagGCepqQ2r1Tc50478FOOD58reSB5CUFDOpIeZAmWDrHXJFX/X3VyUCtQI0+gX0zJeV0rmtLSvy4S8Z/yftoFo+awBqdQvROKkJDVncWvWP8NYoldKNMRhFqf+MnyLse7cEMUqr4ROeBt6upNMsLwbL1PEHVVU
Message-ID: <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
Date: Tue, 6 Feb 2024 14:25:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.01.2024 11:05, Federico Serafini wrote:
> @@ -208,7 +205,7 @@ do {                                                                       \
>      case 8:                                                                \
>          put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>          break;                                                             \
> -    default: __put_user_bad();                                             \
> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>      }                                                                      \
>      clac();                                                                \
>  } while ( false )
> @@ -227,7 +224,7 @@ do {                                                                       \
>      case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>      case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>      case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
> -    default: __get_user_bad();                                             \
> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>      }                                                                      \
>      clac();                                                                \
>  } while ( false )

Related to my remark on patch 1 - how is one to know the macro this was
invoked from, when seeing the resulting diagnostic?

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 13:26:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 13:26:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676935.1053299 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLTW-0000sx-Em; Tue, 06 Feb 2024 13:26:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676935.1053299; Tue, 06 Feb 2024 13:26:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLTW-0000sq-Bo; Tue, 06 Feb 2024 13:26:58 +0000
Received: by outflank-mailman (input) for mailman id 676935;
 Tue, 06 Feb 2024 13:26:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=gccX=JP=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXLTV-0000s7-2w
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 13:26:57 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 65cb7542-c4f3-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 14:26:56 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-40fd2f7ef55so3422545e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 05:26:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n8-20020a5d4c48000000b0033afe816977sm2086647wrt.66.2024.02.06.05.26.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 05:26:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65cb7542-c4f3-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707226015; x=1707830815; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=yIaW+g9kYbMb8fPMAantRbdmlPa6spMR0FsE3eqJ6GI=;
        b=CNKi3V6cGQa/o3/zrWA0ppwmYK+tjlQYGMBKF2Yt+uzofZ6i2g/TGQ9YRYqxYf4OJH
         tWnNr2GCKrMuHS+xbdvArxMjHzhCipGbnUP75WY1PXJZmZ1RoeV5jW7bdoa5uj/JYeRm
         oYPDw6MnLTmp3y5gny3Ge7L40wS/ANp/dOiS9DUTrk5KTuLBb4vh/Q+TBbm5fxU8xo8S
         Yclt9hOl2pE/ATv/0B2HVsf5z3HJ6azCVr2bzJfXPecrF2vEk65op3uPIfopu0jDJbY1
         OefwJ7Uogfiq+4PpfCDb/BOUwF9sO8IRBH2WTFKt4Y866RZlyEl1amHHC5JRJR9olOsF
         HkOA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707226015; x=1707830815;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=yIaW+g9kYbMb8fPMAantRbdmlPa6spMR0FsE3eqJ6GI=;
        b=bEDE/jiGRk549z2egfhX/kwAjUGTmr3PYhRvXgRJ3XQOGC1vx8o7Ur0ryYchKfbjpR
         q3u+LCSZdys80bV9uXMGHfdKRj6//RqMBIopICEhAFBbJ0c9bnJYuvAxy3Uq7JFSnDMu
         943HZN1OWSaqdsE4MM54bhGu608Y74dSBSEdScVgixavlDFX7yOzR+CimRUbY4kTpfjK
         gqcGv3/UUSbkTN+rw7OxQg0GNzeEs6VPBdcWk0y2j7v06fZ+GNemsrM6V0ALzJZ8OgKi
         1DznIkQ7RrMZm588OJOTGP/oRc7loYNmb3x+ApWiiTeiNOL9KgEcU2fPobMiTXweAhRe
         F0XQ==
X-Gm-Message-State: AOJu0Yz2cEQQUtBV74+m65UOICUEvFrup8IpSZHLnnraUc/zdQ8WuL/F
	HzehWPApjfibQ4sRunZ8Iqaqgl32e+VMrVDp3HQRvfkjSrlJRV2G0MbMJNY860lGxoRR1OZTcBQ
	=
X-Google-Smtp-Source: AGHT+IGOJvrLASMT0nOEeLB913YWT8yr11YTe5KtmmlhVfrsVCE/NMIlkQqQDbZbzN3Jd5s5Qc5erg==
X-Received: by 2002:a5d:6887:0:b0:337:68df:5426 with SMTP id h7-20020a5d6887000000b0033768df5426mr1426575wru.34.1707226015512;
        Tue, 06 Feb 2024 05:26:55 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWL1xAFMWeaeanMcqlqs142ipj6CftNhUyvelzdtteLKAGCQ+Z1xOxz87htfqDC6FSXIz/8qUon3pFqakRsRwtpMaD4j0di9jQRSWAF/HwOGisHFuIepZ2MfRnAuNyhzOUGiIPuKREjyzpuE8L4AZM9wuDTTwxOu0aoIRgAg+VdU8MiZV1QpGlBwd1Z9oU5zxMGB16l1fND941kH7OHvf2Ss8uHSDBKI20c7f34xJ77oQoUucPX+qA=
Message-ID: <0e673e08-6fa8-4e1e-bfde-a2764b121d61@suse.com>
Date: Tue, 6 Feb 2024 14:26:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <42fc6ae8d3eb802429d29c774502ff232340dc84.1706259490.git.federico.serafini@bugseng.com>
 <2c0d22be-0b31-4e62-a8f8-2dc82147e51a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <2c0d22be-0b31-4e62-a8f8-2dc82147e51a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 14:22, Jan Beulich wrote:
> On 26.01.2024 11:05, Federico Serafini wrote:> --- a/xen/include/xen/compiler.h
>> +++ b/xen/include/xen/compiler.h
>> @@ -64,6 +64,13 @@
>>  # define fallthrough        do {} while (0)  /* fallthrough */
>>  #endif
>>  
>> +/*
>> + * Add the following macro to check that a program point is considered
>> + * unreachable by the static analysis performed by the compiler.
>> + */
>> +#define STATIC_ASSERT_UNREACHABLE() \
>> +    asm ( ".error \"static assertion failed: unreachable\"" )
> 
> In the comment s/Add/Use/? The macro is there after all when this gets
> committed. Overall maybe
> 
> "Use this macro at program points considered unreachable, to be checked
>  by the compiler's static analysis."
> 
> ?
> 
> Also while asm()s without operands are implicitly volatile, I think it
> would be a good idea to make that explicit nevertheless.
> 
> I'd be happy to adjust while committing, so long as you agree. If you
> agree, and provided diagnostics resulting from this are useful (an
> example would have been nice in the description):
> Acked-by: Jan Beulich <jbeulich@suse.com>

Actually, having seen patch 2, I need to withdraw this right away.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 13:43:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 13:43:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676944.1053308 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLjh-0004kR-RJ; Tue, 06 Feb 2024 13:43:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676944.1053308; Tue, 06 Feb 2024 13:43:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLjh-0004kK-Om; Tue, 06 Feb 2024 13:43:41 +0000
Received: by outflank-mailman (input) for mailman id 676944;
 Tue, 06 Feb 2024 13:43:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=z75a=JP=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXLjg-0004iW-VP
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 13:43:40 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bc49236a-c4f5-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 14:43:40 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 4639222030;
 Tue,  6 Feb 2024 13:43:39 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1E07B132DD;
 Tue,  6 Feb 2024 13:43:39 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id xXpxBYs3wmVGQgAAD6G6ig
 (envelope-from <jgross@suse.com>); Tue, 06 Feb 2024 13:43:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc49236a-c4f5-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707227019; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/ktfHFMT+dUDRU7YcjswPtaU36Yaut2wQu0bdPBH468=;
	b=F5uHvTaQfC+hnSpuorNCwbz7+uMIAYuYMuHHpla1YuzZ5dOfldmANwP9PqMvfQJAdlon7n
	KRAKS1Sk9XG8KhtHAJs//hnPdpPssVqTlv2pmB7qasFnCFKhbkWIUSDl2Os6MV2WBVOyky
	WyczExRKQJHJ+TE672fyUYHcIHqLJXY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707227019; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/ktfHFMT+dUDRU7YcjswPtaU36Yaut2wQu0bdPBH468=;
	b=F5uHvTaQfC+hnSpuorNCwbz7+uMIAYuYMuHHpla1YuzZ5dOfldmANwP9PqMvfQJAdlon7n
	KRAKS1Sk9XG8KhtHAJs//hnPdpPssVqTlv2pmB7qasFnCFKhbkWIUSDl2Os6MV2WBVOyky
	WyczExRKQJHJ+TE672fyUYHcIHqLJXY=
Message-ID: <244f3950-b7e8-4563-8972-bd230021433a@suse.com>
Date: Tue, 6 Feb 2024 14:43:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/libs/light: don't allow to stop Xenstore stubdom
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <20240206124345.19433-1-jgross@suse.com>
 <d420a9c2-3fa0-40cc-adf7-ffd88fcffbfe@citrix.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <d420a9c2-3fa0-40cc-adf7-ffd88fcffbfe@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=F5uHvTaQ
X-Spamd-Result: default: False [-0.11 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 BAYES_HAM(-0.58)[81.36%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.77)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: -0.11
X-Rspamd-Queue-Id: 4639222030
X-Spam-Level: 
X-Spam-Flag: NO
X-Spamd-Bar: /

On 06.02.24 14:08, Andrew Cooper wrote:
> On 06/02/2024 12:43 pm, Juergen Gross wrote:
>> A Xenstore stubdom should never be stoppable.
>>
>> Reject attempts to do so.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> 
> I don't think this is a clever idea.  `xl destroy` is also the "please
> clean up my system when it's in a very dead state" command, and that
> also includes a dead xenstored stubdom.

I don't think xl destroy for a dead Xenstore stubdom will ever work.
xl destroy tries to read (and delete) Xenstore entries, after all.

I think you'd need a program using libxenctrl without all the xl/libxl
actions for achieving this goal. And this would work with my current
patch, too.

> If you're looking for some protection, then maybe a `--force` flag to
> override, but there must be some way of getting this to run.

A system without Xenstore is probably quite useless anyway. At least today
there is no way a new Xenstore would be able to connect to existing domains.

OTOH I'm inclined to add more hooks, e.g. for "xl pause" and "xl migrate".

And I do think that libxl is the right level for that, as I don't want users
to be able to kill/pause/migrate Xenstore stubdom via libvirt either.


Juergen


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 13:59:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 13:59:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676952.1053319 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLz7-0007Py-5r; Tue, 06 Feb 2024 13:59:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676952.1053319; Tue, 06 Feb 2024 13:59:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXLz7-0007Pr-2g; Tue, 06 Feb 2024 13:59:37 +0000
Received: by outflank-mailman (input) for mailman id 676952;
 Tue, 06 Feb 2024 13:59:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXLz5-0007Ph-SW; Tue, 06 Feb 2024 13:59:35 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXLz5-00044K-PK; Tue, 06 Feb 2024 13:59:35 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXLz5-0003B0-FZ; Tue, 06 Feb 2024 13:59:35 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXLz5-00077i-F0; Tue, 06 Feb 2024 13:59:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=KLAV87Rn9yZM7fgGLfvboF7pG4XjSgIPK2yYQ3S9xiE=; b=re4gseFzaFdhsQ5PHlrxrRLThF
	GbPh+2kl2NoH2dkJptlMR7O5ApM48RTV4qiw+OC+UCtSlv0uy7IBDQFHZNF2BPzSO3M4EHgTmWEGE
	Cchh6VQ07WgRkFdP87Uqm10EpAM8YlVoifxCfNBz+gsoDvmA2bs/8mtPAKRy6HuxAcZU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184605-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184605: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=57a923bfd920307c5bfa533c95dba16a13ea0225
X-Osstest-Versions-That:
    xen=1cb7949b37b3d8892d8a3cea3ad1e2448352a34d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 06 Feb 2024 13:59:35 +0000

flight 184605 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184605/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  57a923bfd920307c5bfa533c95dba16a13ea0225
baseline version:
 xen                  1cb7949b37b3d8892d8a3cea3ad1e2448352a34d

Last test of basis   184599  2024-02-06 01:00:34 Z    0 days
Testing same since   184605  2024-02-06 11:03:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  David Woodhouse <dwmw2@amazon.com>
  Elias El Yandouzi <eliasely@amazon.com>
  Frediano Ziglio <frediano.ziglio@cloud.com>
  George Dunlap <george.dunlap@cloud.com>
  Hongyan Xia <hongyxia@amazon.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  oxjo <oxjo@proton.me>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Wei Liu <wei.liu2@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   1cb7949b37..57a923bfd9  57a923bfd920307c5bfa533c95dba16a13ea0225 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 14:11:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 14:11:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676961.1053329 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXMAK-0001lY-6W; Tue, 06 Feb 2024 14:11:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676961.1053329; Tue, 06 Feb 2024 14:11:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXMAK-0001lR-2r; Tue, 06 Feb 2024 14:11:12 +0000
Received: by outflank-mailman (input) for mailman id 676961;
 Tue, 06 Feb 2024 14:11:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+UH4=JP=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXMAJ-0001lL-8b
 for xen-devel@lists.xen.org; Tue, 06 Feb 2024 14:11:11 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 929d3896-c4f9-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 15:11:08 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-5112d5ab492so7870439e87.0
 for <xen-devel@lists.xen.org>; Tue, 06 Feb 2024 06:11:08 -0800 (PST)
Received: from [10.80.67.148] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 h21-20020a170906111500b00a372330e834sm1191468eja.102.2024.02.06.06.11.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 06:11:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 929d3896-c4f9-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707228667; x=1707833467; darn=lists.xen.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=VJiZybQedixqXPT0Vc0ESISL14VktV+tIavcWkYU7D0=;
        b=ascs5k/SoYFDShJtQ3xXG171jy/DxlwVwq8nZWg3Hp2LoLmACn7HJQ1TYsGA9dDpu3
         PeL9nY+hwwF2iZ5nPbVETHOW8NghWoQ+8nO4Hgkom7IIAb9cx7p9iNGE9VMpTzNQZ/Si
         OCXVsAQy1HiBzHD7lwRrP2NXuRSFzIu3ApDik=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707228667; x=1707833467;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=VJiZybQedixqXPT0Vc0ESISL14VktV+tIavcWkYU7D0=;
        b=tplZCn+2yeaRqBuVqH0xv+B7ExRaMmEAqr4iGd3DZzqxDCscEwp6ZSRQF/jPTeJPUE
         9W25zn902X+x3TI5m2Z/WmyY+hwwrRZq5gTtilunI3KYwHBWLcaDEnFXSM2DQDlnkTYs
         +EC0Y50+719fMBwk0gbzwLZAodfm8XdD9Mux54nsNpzxbs4wUbGgoalZWmEDklalMpxL
         HfMKgWIwJurCym2Jmnbv8vh/yGm1UEzmeBuiLRB4dYQDgukuYVDXAoGg3Ml2mDG2i0IR
         1X0NH3ohN6qDXA9+N8u0IEjJtR5ICJtKYMpXStv7+S605q5iAt7lMUo8ipwAL1yFi6GS
         QsqA==
X-Gm-Message-State: AOJu0Ywn2yFmjRk3/NL5fGcNFCKXlAfozibvnVWOThMivMzFIdPS5PC1
	Nv2xArrQWpzIB8cYqQC/OFs+SBXLTkW90GK43oiEwdENUzJTiJTiCc3BsyoT4RI=
X-Google-Smtp-Source: AGHT+IHK/53MGzQSVJi5PfK3MxK+WFR6s1o2EDVIlNbabm3AYvW2eoKqD5jsPkyPfUYWgS1ERvFBrQ==
X-Received: by 2002:a19:641c:0:b0:511:add:cfe9 with SMTP id y28-20020a19641c000000b005110addcfe9mr1750375lfb.26.1707228667478;
        Tue, 06 Feb 2024 06:11:07 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCUvLAdHhJCqsTMn2UfIFWPH1dxxAsJgkTCh1loRe4b1KUEakqHLQbG6fcOyqGODu/okONeaqgNjARtsqQtQzWOK3ZRw4vKwJg==
Message-ID: <5cb25d4f-c3be-4c45-a434-421e1bf54be3@citrix.com>
Date: Tue, 6 Feb 2024 14:11:06 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [BUG] Possible FATAL PAGE FAULT in domain_crash function on Xen
 4.14.6
Content-Language: en-GB
To: Giuseppe De Rosa <giuseppe.de.rosa@outlook.it>,
 "xen-devel@lists.xen.org" <xen-devel@lists.xen.org>
References: <DB9P192MB1434AD2FD1069E39FB480068A6462@DB9P192MB1434.EURP192.PROD.OUTLOOK.COM>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <DB9P192MB1434AD2FD1069E39FB480068A6462@DB9P192MB1434.EURP192.PROD.OUTLOOK.COM>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06/02/2024 12:14 pm, Giuseppe De Rosa wrote:
> Bug detailed description:
> 
> ----------------
> 
> While booting a Linux Debian 7 "Wheezy" VM, Xen crashes with a FATAL
> PAGE FAULT. 
> 
>  
> 
> Environment :
> 
> ----------------
> 
> HW: Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz (2 CPU, Xen in nested
> virtualization upon QEMU/KVM), 4GB RAM
> 
> Xen: Xen 4.14.6 (xen-hypervisor-4.14-amd64 package)
> 
> Dom0: Linux 5.10.0-26-amd64 (Debian 11 "Bullseye"), 1 CPU, 1024GB RAM
> 
>  
> 
> Reproduce steps:
> 
> ----------------
> 
> 1.      Install debian 11 and xen package
> 
> 2.      Boot a clean debian 7 image in hardware-assisted virtualization
> 
> 3.      Single bit-flip bit number 2 of VMCS field
> "VM_EXIT_QUALIFICATION" (field number 6400). In my case, value changed
> from [100049] -> [10004d]
> 
> 4.      Leave the debian7 guest executing after the bit flip.
> 
>  
> 
> Current result:
> 
> ----------------
> 
> Xen crash
> 
>  
> 
> Console error log:
> 
> ----------------
> 
> (XEN) d1v0 Unexpected PIO status 1, port 0x10 read 0x00000000ffff
> 
> (XEN) domain_crash called from io.c:166
> (XEN) Domain 1 (vcpu#0) crashed on cpu#1:
> (XEN) ----[ Xen-4.14.6  x86_64  debug=n   Not tainted ]----
> (XEN) CPU:    1
> (XEN) RIP:    0010:[<ffffffff8100712e>]
> (XEN) RFLAGS: 0000000000000046   CONTEXT: hvm guest (d1v0)
> (XEN) rax: 0000000000000000   rbx: 0000000040000000   rcx: 0000000000000001
> (XEN) rdx: 0000000000000000   rsi: ffffffff81666a80   rdi: ffffffff81617830
> (XEN) rbp: 0000000000000020   rsp: ffffffff81601e78   r8:  0000000000000200
> (XEN) r9:  ffffffff8168f2a0   r10: 0000000000000007   r11: 0000000000000007
> (XEN) r12: ffffffff81601f58   r13: ffffffffffffffff   r14: 000000000008c800
> (XEN) r15: 0000000000001000   cr0: 0000000080050033   cr4: 00000000001000a0
> (XEN) cr3: 0000000001605000   cr2: 0000000000000000
> (XEN) fsb: 0000000000000000   gsb: ffffffff81696000   gss: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: 0010
> (XEN) d1v0 Bad rIP ffffffff8100712e for mode 0
> (XEN) domain_crash called from vmx.c:4413
> (XEN) ----[ Xen-4.14.6  x86_64  debug=n   Not tainted ]----
> (XEN) CPU:    1
> (XEN) RIP:    e008:[<ffff82d040206fa9>] __domain_crash+0x9/0x80
> (XEN) RFLAGS: 0000000000010296   CONTEXT: hypervisor (d1v0)
> (XEN) rax: ffff830139c0506c   rbx: ffff8301308a0000   rcx: 0000000000000000
> (XEN) rdx: ffff830136ddffff   rsi: 000000000000000a   rdi: 0000000000000000
> (XEN) rbp: 0000000000000000   rsp: ffff830136ddfee0   r8:  0000000000000001
> (XEN) r9:  0000000000004000   r10: 0000000000000001   r11: ffff82d040372d40
> (XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
> (XEN) r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000001526e0
> (XEN) cr3: 0000000136da6000   cr2: 0000000000000208
> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> (XEN) Xen code around <ffff82d040206fa9> (__domain_crash+0x9/0x80):
> (XEN)  0f 1e fa 55 48 89 fd 53 <80> bf 08 02 00 00 00 75 2d 48 89 e3 0f
> b7 37 48
> (XEN) Xen stack trace from rsp=ffff830136ddfee0:
> (XEN)    ffff8301308a0000 0000000000000000 ffff82d0402a1798 0000000000001000
> (XEN)    000000000008c800 ffffffffffffffff ffffffff81601f58 0000000000000020
> (XEN)    0000000040000000 0000000000000007 0000000000000007 ffffffff8168f2a0
> (XEN)    0000000000000200 0000000000000000 0000000000000001 0000000000000000
> (XEN)    ffffffff81666a80 ffffffff81617830 000000fa00000000 ffffffff8100712e
> (XEN)    0000000000000000 0000000000000046 ffffffff81601e78 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN)    0000e01000000001 ffff8301308a0000 00000030f9686000 00000000001526e0
> (XEN)    0000000000000000 0000000000000000 0006020200000000 0000000000000000
> (XEN) Xen call trace:
> (XEN)    [<ffff82d040206fa9>] R __domain_crash+0x9/0x80
> (XEN)    [<ffff82d0402a1798>] S vmx_asm_vmexit_handler+0xf8/0x210
> (XEN)
> (XEN) Pagetable walk from 0000000000000208:
> (XEN)  L4[0x000] = 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 1:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0000]
> (XEN) Faulting linear address: 0000000000000208
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
> 
>  
> 
> HVM guest config file:
> 
> --------------------------------
> 
> name = "debian7"
> 
> builder= "hvm"
> memory = 1024
> vcpus = 1
> cpus= ["1"]
> disk=["qcow2:/home/test/debian7.qcow2,xvda,w"]
> vnc = 1
> vnclisten = '127.0.0.1'
> vncdisplay = 1
> boot = "d"
> serial = "pty"
> 
> Discussion
> --------------------------------
> 
> Hello, i am conducting robustness tests on Xen 4.14. I bit flipped the
> field VM_EXIT_QUALIFICATION during the exit handling of an
> IO_INSTRUCTION. After a VM Read on that field, Xen crashes with this
> error log. This is my (possible) explanation:
> 
> Xen correctly detects an error in the field, resulting in a domain crash
> (Unexpected PIO status 1) called from this point in the code:
> 
> xen/arch/x86/hvm/io.c
> ```
> gprintk(XENLOG_ERR, "Unexpected PIO status %d, port %#x %s 0x%0*x\n",
>                 rc, port, dir == IOREQ_WRITE ? "write" : "read",
>                 size * 2, data & ((1u << (size * 8)) - 1));
> domain_crash(curr->domain);
> return false;
> ```
> 
> In the `handle_pio` function, in the handling of the `IO_INSTRUCTION`
> exit reason. However, the code continues and ends here for an issue in
> the processor mode:
> 
> /xen/arch/x86/hvm/vmx/vmx.c
> ```
> mode = vmx_guest_x86_mode(v);
>     if ( mode == 8 ? !is_canonical_address(regs->rip)
>                    : regs->rip != regs->eip )
>     {
>         gprintk(XENLOG_WARNING, "Bad rIP %lx for mode %u\n", regs->rip,
> mode);
> 
>         if ( vmx_get_cpl() )
>         {
>             __vmread(VM_ENTRY_INTR_INFO, &intr_info);
>             if ( !(intr_info & INTR_INFO_VALID_MASK) )
>                 hvm_inject_hw_exception(TRAP_gp_fault, 0);
>             /* Need to fix rIP nevertheless. */
>             if ( mode == 8 )
>                 regs->rip = (long)(regs->rip << (64 - VADDR_BITS)) >>
>                             (64 - VADDR_BITS);
>             else
>                 regs->rip = regs->eip;
>         }
>         else
>             domain_crash(v->domain);
>     }
> ```
> 
> However, the domain pointer has already been deallocated due to the
> previous domain crash, resulting in a page fault that leads to Xen crashing.
> 
> I would like to report this crash and ask for your opinion. Based on
> previous research, I do not believe it has been reported before. I am
> unsure if it could potentially be a security issue, hence I am posting
> it here. Let me know if I should provide other results. Thank you for
> your kind response in advance.


Answering somewhat out of order.

You've posted this publicly so the cat is out of the bag regardless.  If
you have concerns about security, please email security@xen.org as your
first point of contact.

That said, nested virt is not security supported.  It's still an
experimental feature, so bugs like this are fine to come straight to the
public mailing list.

The "Bad rIP" logic is buggy and has since been deleted.  Xen 4.14 is a
very old version of Xen, and is outside of general bugfix support.

It is quite likely that this bug still exists, but please use an
up-to-date version of Xen.  Fixes need developing against master and are
unlikely to be backported in this case, given its experimental status.

Also, you should be using a debug build of Xen generally for work like
this.  (I have no idea if it would alter your observations.)


I'm not sure what I think about bitflipping the exit qualification.  In
other places, that will definitely cause more severe crashes, and Xen
won't be getting in the game of auditing the VMX implementation against
the VMX spec.


For this crash you've got, there should be no way for the domain pointer
to have been freed in the sequence you've described.  While the vCPU is
still scheduled, the structures will remain.

Looking at the second backtrace, It looks suspiciously like a NULL
pointer was passed into __domain_crash() from vmx_asm_vmexit_handler()
but there's no such call call, nor a tailcall out of the handler, even
taking a peek at the 4.14 code.

If you have local changes, I'd look at those first.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 14:30:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 14:30:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676968.1053339 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXMSY-0004VU-SB; Tue, 06 Feb 2024 14:30:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676968.1053339; Tue, 06 Feb 2024 14:30:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXMSY-0004VN-OZ; Tue, 06 Feb 2024 14:30:02 +0000
Received: by outflank-mailman (input) for mailman id 676968;
 Tue, 06 Feb 2024 14:30:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=x4wk=JP=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXMSX-0004KN-HI
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 14:30:01 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 346ce9b7-c4fc-11ee-8a47-1f161083a0e0;
 Tue, 06 Feb 2024 15:30:00 +0100 (CET)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a37721e42feso351487866b.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 06:29:58 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 c25-20020a170906171900b00a3667f53691sm1220020eje.86.2024.02.06.06.29.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 06 Feb 2024 06:29:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 346ce9b7-c4fc-11ee-8a47-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707229798; x=1707834598; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=fsZlrZvi9YFerP5MveeH4iiwb2SzDqfcOIVBQPs9lQE=;
        b=E9EcdVMbLKePRG6MkrzIosdqA3gGakEZJYpOQ4t8xUfU9qzBCSuNrpQBSGSLDEJuP4
         XcwAb0KP/AQRJkyDNZpzj8AJ2zLa7IgSeQQ2m93bCZ/5Ile8HDFeWbPw6O+hK+aWl5RU
         tsoeXWNak/B2utBWvv8tqRAxHiausoo6pXfko=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707229798; x=1707834598;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=fsZlrZvi9YFerP5MveeH4iiwb2SzDqfcOIVBQPs9lQE=;
        b=T5akLn+sJ+5vjd6iIAamp+jdI4CJZUwrf3aXSDJkFIYta6TQn4/MJvg/oeMFWOE+L0
         M3qCY1cc90SibGrBRoP4JTsDEonpupuY/618aSEFezf1K8Xxxk2AYA6HhJgkSdS1VBVq
         CvXyDlTc6xhGA5Q9oHkUekevGnMDgmyVS7jR8XyCikcZr6JGjHm/X0EZVVESS9xjbwjS
         CN8mr9zZD8NVdy3md5cR1fa4zmGt4xMP/o5GdY0nHEzZ2V/h2plpQU8UU3cVlSCvawKc
         ER/bCU5EtleT/HqiMs5lX8Vcdy0kK9WRIKmDgJVQ42u/POOngm5vVq49XHsQDYBa2oeH
         66lA==
X-Gm-Message-State: AOJu0Yz5F5IJp3u/UtM3aRxBSsUNxTYaRKlMWfbioWYJvK92EnJEpDdT
	jMjvh9BewdT/Lqs5+PdWCfOApFEiVkfv4Bmsi7Ttmlt9ICyq8vfH2F7tcKQ29XXgqUr0cseeyMw
	y
X-Google-Smtp-Source: AGHT+IG5Y7nMGxysjOYLAICTvl7WITL49d/nFugczn+f8opqv7J+ZMw0W7qU8ePnbxcYdSjIxMfirw==
X-Received: by 2002:a17:907:1048:b0:a36:d1e1:71ec with SMTP id oy8-20020a170907104800b00a36d1e171ecmr1939767ejb.46.1707229797924;
        Tue, 06 Feb 2024 06:29:57 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCVez7Ui5noFWY0PuCXOpYFV42XauGh1uqkWVG7LhA6WT77G4YD8THNooDlndHrKqL6GYiLhVtp6NuIT16dzRXfidRn/yp0xCf5RzOuVaeLozqnaqzAcQEk1L+AGIkaimSG6DJxitPrHPrmULG9U+gwJPJLx40Plbtab21ASWudfdwtnCWQXcKS3RxyEJbA+X0ubPvfl4XI=
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/vmx: add support for virtualize SPEC_CTRL
Date: Tue,  6 Feb 2024 15:25:07 +0100
Message-ID: <20240206142507.81985-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The feature is defined in the tertiary exec control, and is available starting
from Sapphire Rapids and Alder Lake CPUs.

When enabled, two extra VMCS fields are used: SPEC_CTRL mask and shadow.  Bits
set in mask are not allowed to be toggled by the guest (either set or clear)
and the value in the shadow field is the value the guest expects to be in the
SPEC_CTRL register.

By using it the hypervisor can force the value of SPEC_CTRL bits behind the
guest back without having to trap all accesses to SPEC_CTRL.  It also allows
getting rid of SPEC_CTRL in the guest MSR load list, since the value in the
shadow field will be loaded by the hardware on vmentry.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Applies on top of "VMX: tertiary execution control infrastructure"
---
 xen/arch/x86/hvm/vmx/vmcs.c             | 10 ++++++-
 xen/arch/x86/hvm/vmx/vmx.c              | 40 ++++++++++++++++++++-----
 xen/arch/x86/include/asm/hvm/vmx/vmcs.h |  5 ++++
 xen/arch/x86/include/asm/msr.h          |  9 ++++--
 4 files changed, 54 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 9e016634ab5c..adcbd014123a 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -202,6 +202,7 @@ static void __init vmx_display_features(void)
     P(cpu_has_vmx_tsc_scaling, "TSC Scaling");
     P(cpu_has_vmx_bus_lock_detection, "Bus Lock Detection");
     P(cpu_has_vmx_notify_vm_exiting, "Notify VM Exit");
+    P(cpu_has_vmx_virt_spec_ctrl, "Virtualize SPEC_CTRL");
 #undef P
 
     if ( !printed )
@@ -365,7 +366,7 @@ static int vmx_init_vmcs_config(bool bsp)
 
     if ( _vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS )
     {
-        uint64_t opt = 0;
+        uint64_t opt = TERTIARY_EXEC_VIRT_SPEC_CTRL;
 
         _vmx_tertiary_exec_control = adjust_vmx_controls2(
             "Tertiary Exec Control", 0, opt,
@@ -1378,6 +1379,10 @@ static int construct_vmcs(struct vcpu *v)
         rc = vmx_add_msr(v, MSR_PRED_CMD, PRED_CMD_IBPB,
                          VMX_MSR_HOST);
 
+    /* Set any bits we don't allow toggling in the mask field. */
+    if ( cpu_has_vmx_virt_spec_ctrl && v->arch.msrs->spec_ctrl.raw )
+        __vmwrite(SPEC_CTRL_MASK, v->arch.msrs->spec_ctrl.raw);
+
  out:
     vmx_vmcs_exit(v);
 
@@ -2086,6 +2091,9 @@ void vmcs_dump_vcpu(struct vcpu *v)
     if ( v->arch.hvm.vmx.secondary_exec_control &
          SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY )
         printk("InterruptStatus = %04x\n", vmr16(GUEST_INTR_STATUS));
+    if ( cpu_has_vmx_virt_spec_ctrl )
+        printk("SPEC_CTRL mask = %#016lx  shadow = %#016lx\n",
+               vmr(SPEC_CTRL_MASK), vmr(SPEC_CTRL_SHADOW));
 
     printk("*** Host State ***\n");
     printk("RIP = 0x%016lx (%ps)  RSP = 0x%016lx\n",
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 48376cc32751..3911e4ecb0d6 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -823,18 +823,28 @@ static void cf_check vmx_cpuid_policy_changed(struct vcpu *v)
     {
         vmx_clear_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
 
-        rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
-        if ( rc )
-            goto out;
+        if ( !cpu_has_vmx_virt_spec_ctrl )
+        {
+            rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
+            if ( rc )
+                goto out;
+        }
     }
     else
     {
         vmx_set_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
 
-        rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
-        if ( rc && rc != -ESRCH )
-            goto out;
-        rc = 0; /* Tolerate -ESRCH */
+        /*
+         * NB: there's no need to clear the virtualize SPEC_CTRL control, as
+         * the MSR intercept takes precedence.
+         */
+        if ( !cpu_has_vmx_virt_spec_ctrl )
+        {
+            rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
+            if ( rc && rc != -ESRCH )
+                goto out;
+            rc = 0; /* Tolerate -ESRCH */
+        }
     }
 
     /* MSR_PRED_CMD is safe to pass through if the guest knows about it. */
@@ -2629,6 +2639,9 @@ static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
     switch ( reg )
     {
     case MSR_SPEC_CTRL:
+        if ( cpu_has_vmx_virt_spec_ctrl )
+            /* Requires remote VMCS loaded - fetched below. */
+            break;
         rc = vmx_read_guest_msr(v, reg, &val);
         if ( rc )
         {
@@ -2652,6 +2665,11 @@ static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
     vmx_vmcs_enter(v);
     switch ( reg )
     {
+    case MSR_SPEC_CTRL:
+        ASSERT(cpu_has_vmx_virt_spec_ctrl);
+        __vmread(SPEC_CTRL_SHADOW, &val);
+        break;
+
     case MSR_IA32_BNDCFGS:
         __vmread(GUEST_BNDCFGS, &val);
         break;
@@ -2678,6 +2696,9 @@ static void cf_check vmx_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
     switch ( reg )
     {
     case MSR_SPEC_CTRL:
+        if ( cpu_has_vmx_virt_spec_ctrl )
+            /* Requires remote VMCS loaded - fetched below. */
+            break;
         rc = vmx_write_guest_msr(v, reg, val);
         if ( rc )
         {
@@ -2698,6 +2719,11 @@ static void cf_check vmx_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
     vmx_vmcs_enter(v);
     switch ( reg )
     {
+    case MSR_SPEC_CTRL:
+        ASSERT(cpu_has_vmx_virt_spec_ctrl);
+        __vmwrite(SPEC_CTRL_SHADOW, val);
+        break;
+
     case MSR_IA32_BNDCFGS:
         __vmwrite(GUEST_BNDCFGS, val);
         break;
diff --git a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
index a7dd2eeffcad..58140af69153 100644
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -270,6 +270,9 @@ extern u32 vmx_secondary_exec_control;
 #define TERTIARY_EXEC_VIRT_SPEC_CTRL            BIT(7, UL)
 extern uint64_t vmx_tertiary_exec_control;
 
+#define cpu_has_vmx_virt_spec_ctrl \
+     (vmx_tertiary_exec_control & TERTIARY_EXEC_VIRT_SPEC_CTRL)
+
 #define VMX_EPT_EXEC_ONLY_SUPPORTED                         0x00000001
 #define VMX_EPT_WALK_LENGTH_4_SUPPORTED                     0x00000040
 #define VMX_EPT_MEMORY_TYPE_UC                              0x00000100
@@ -436,6 +439,8 @@ enum vmcs_field {
     XSS_EXIT_BITMAP                 = 0x0000202c,
     TSC_MULTIPLIER                  = 0x00002032,
     TERTIARY_VM_EXEC_CONTROL        = 0x00002034,
+    SPEC_CTRL_MASK                  = 0x0000204a,
+    SPEC_CTRL_SHADOW                = 0x0000204c,
     GUEST_PHYSICAL_ADDRESS          = 0x00002400,
     VMCS_LINK_POINTER               = 0x00002800,
     GUEST_IA32_DEBUGCTL             = 0x00002802,
diff --git a/xen/arch/x86/include/asm/msr.h b/xen/arch/x86/include/asm/msr.h
index 1d8ea9f26faa..eed7b36cd992 100644
--- a/xen/arch/x86/include/asm/msr.h
+++ b/xen/arch/x86/include/asm/msr.h
@@ -302,8 +302,13 @@ struct vcpu_msrs
      * For PV guests, this holds the guest kernel value.  It is accessed on
      * every entry/exit path.
      *
-     * For VT-x guests, the guest value is held in the MSR guest load/save
-     * list.
+     * For VT-x guests, the guest value is held in the MSR guest load/save list
+     * if there's no support for virtualized SPEC_CTRL. If virtualized
+     * SPEC_CTRL is enabled the value here signals which bits in SPEC_CTRL the
+     * guest is not able to modify.  Note that the value for those bits used in
+     * Xen context is also used in the guest context.  Setting a bit here
+     * doesn't force such bit to set in the guest context unless also set in
+     * Xen selection of SPEC_CTRL.
      *
      * For SVM, the guest value lives in the VMCB, and hardware saves/restores
      * the host value automatically.  However, guests run with the OR of the
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 14:38:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 14:38:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676972.1053349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXMaQ-000643-NH; Tue, 06 Feb 2024 14:38:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676972.1053349; Tue, 06 Feb 2024 14:38:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXMaQ-00063w-JC; Tue, 06 Feb 2024 14:38:10 +0000
Received: by outflank-mailman (input) for mailman id 676972;
 Tue, 06 Feb 2024 14:38:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=d8l/=JP=rabbit.lu=slack@srs-se1.protection.inumbo.net>)
 id 1rXMaO-00063q-Vx
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 14:38:09 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 57495528-c4fd-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 15:38:06 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-33929364bdaso3498044f8f.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 06:38:06 -0800 (PST)
Received: from [192.168.2.1] (82-64-138-184.subs.proxad.net. [82.64.138.184])
 by smtp.googlemail.com with ESMTPSA id
 e5-20020adfef05000000b0033b4a77b2c7sm600795wro.82.2024.02.06.06.38.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 06:38:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57495528-c4fd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=rabbit-lu.20230601.gappssmtp.com; s=20230601; t=1707230286; x=1707835086; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:references:cc:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9JjcHkICbDf4yb/GLYRyN14BpoDErXvG9PwCggk40Xg=;
        b=mLNinxHRuCyt41v0avM7h32VXLzgEHdhF5ghLdeO1Ph1Juq41SzbLw9IePLBrxN+nv
         SlsRHLk3LlE9iNYP4+zluhS5EXTgHs0TKmpjEshnS0DVM7nqJxRfpndYn76PsxlZlVyu
         ZJGN5yZQS8kWZmx2C3EE1D8LTdcYWaguhgOEbcno1ocbGVGO2tr6wIA5U6YvzB2VTfff
         c1mBQm+e2YcWTM46vGqNhRXXXWpQr/BsCx482x21FDfIKu8VsvTzgexw58fgYHaT2hjE
         nsPbfeqCcXZzazFzWBMW6dTVUku1oa4Q6tLDWySsDAtDjIhj7HqhLulfpAk1gWaXyUtb
         HiKA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707230286; x=1707835086;
        h=content-transfer-encoding:in-reply-to:from:references:cc:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9JjcHkICbDf4yb/GLYRyN14BpoDErXvG9PwCggk40Xg=;
        b=DI44ZUhYYPTXpJ2L4VkKy0xnABjo8cr07lIOiILLn4gcB2/lRo912jOIKV/NjRGX3v
         qULuxSxiE/340DpF703R264shHjFXGG+zUsTAz27JTNkUBdhI96FeMds+vxEdzedJjBg
         NQb0ADKuhWSCN4i/rF3VgKqlWfAzPc92DZ9AeE8BTidQ/wegYu1H30otmsVvPorWFXuQ
         VyQNqbd79ZJydNOLd0pOWY3OKD1W4rWyFpcTQnDk0lMQFOp1w0bfj8rb3oy6gaDdu+sf
         thw550941A12kdyQMF5fKJBQlCp19cDSFZ6MRS8YAZm7FJKQvlLxx+Caylf02pfulre7
         i5IA==
X-Gm-Message-State: AOJu0YyNPPF30x1enIX/8y2v7nbMFMKJNVR2S+5do5f3BYllEvAFfAYQ
	aTwU004N4yAkR9cwR0MmsyBcMjA73L2dIJLq5xG0U3fYnPpNDQZ07hOkZJ3/WN6pGajIJyiDi2M
	ADVc=
X-Google-Smtp-Source: AGHT+IH8dsuGnjeAzuII9kpk2nYFeY/M61PYOV3TObHXoNvBxgt1m2KgotWgZLWf5fDl5vwRpFtbgw==
X-Received: by 2002:adf:fe46:0:b0:33a:ea2c:5b36 with SMTP id m6-20020adffe46000000b0033aea2c5b36mr1183811wrs.0.1707230286046;
        Tue, 06 Feb 2024 06:38:06 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWsH4N2j4g9SHA/eNhrOCEhXUNGAVRYJjq5UKJoWPTUzvCn41AAeBx6R6Ro1Y6W1E7V2PjojPm+LXQt
Message-ID: <1f2fab6d-6eaf-4e32-a124-cf1042c31312@rabbit.lu>
Date: Tue, 6 Feb 2024 15:38:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/xentop: add option to display dom0 first
Content-Language: en-US
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <87846acd5b31991e38561c9765eb97730c79d0f3.1706723494.git.slack@rabbit.lu>
 <a8194dec-70f0-4f90-898d-f15152a5018b@perard>
From: zithro <slack@rabbit.lu>
In-Reply-To: <a8194dec-70f0-4f90-898d-f15152a5018b@perard>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 05 Feb 2024 18:27, Anthony PERARD wrote:
> On Wed, Jan 31, 2024 at 06:51:34PM +0100, Cyril Rébert wrote:
>> Add a command line option to xentop to be able to display dom0 first, on top of the list.
>> This is unconditional, so sorting domains with the S option will also ignore dom0.
>>
>> Signed-off-by: Cyril Rébert (zithro) <slack@rabbit.lu>
> 
> Hi Cyril,

Hi Anthony,

> Your patch looks like a good idea

Thanks, I also have a "display dom id column" in the pipes, almost ready 
to send, but I found kind of a bug doing this (field_id/fields offsets 
off by one).
Are there "most wanted functionnalities" concerning xentop, while I'm at 
it ? There's a TODO in xentop's folder.
Things I'd like to add, if possible, is domain management 
(pause/unpause, maybe shutdown/destroy), columns hiding and domains hiding.

> but xentop segv without '-z' now, when there are guest running.

I failed at the strict minimum here, should have tested ... Sorry for 
wasting your time !
Bug spotted ("sort_start" incorrectly initialized), will post a v2.

> Revelant part of a backtrace:
> #0  xenstat_domain_name (domain=0x121) at xenstat.c:344
> 344		return domain->name;
> #6  0x00006344dd283651 in top () at xentop.c:1209
>          i = 2
>          num_domains = 2
>          sort_start = 1
>          sort_count = <optimized out>
>          dom0_index = <optimized out>
> 1209		qsort((domains+sort_start), (num_domains-sort_count), sizeof(xenstat_domain *),
> 1210		      (int(*)(const void *, const void *))compare_domains);

What soft did you use to get that output ? A debugger ?
(It's a real question, I'm a noob).

> Also, could you update the man page? Here "docs/man/xentop.1.pod""

Will do with v2 !

> 
> Thanks,
> 

-- 
++
zithro / Cyril



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 15:17:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 15:17:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676982.1053359 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNBt-0003Mx-K1; Tue, 06 Feb 2024 15:16:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676982.1053359; Tue, 06 Feb 2024 15:16:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNBt-0003Mq-Gp; Tue, 06 Feb 2024 15:16:53 +0000
Received: by outflank-mailman (input) for mailman id 676982;
 Tue, 06 Feb 2024 15:16:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=d8l/=JP=rabbit.lu=slack@srs-se1.protection.inumbo.net>)
 id 1rXNBt-0003Mk-5t
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 15:16:53 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c001537d-c502-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 16:16:49 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-33b1a51743fso4194516f8f.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 07:16:49 -0800 (PST)
Received: from localhost.localdomain (82-64-138-184.subs.proxad.net.
 [82.64.138.184]) by smtp.googlemail.com with ESMTPSA id
 r13-20020a05600c458d00b0040fe5994d0csm1931007wmo.0.2024.02.06.07.16.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 06 Feb 2024 07:16:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c001537d-c502-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=rabbit-lu.20230601.gappssmtp.com; s=20230601; t=1707232609; x=1707837409; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=nt3pBAxgAVXBBQ7nYfBf27vXYcq8ELUyng70/ndviP0=;
        b=MwF4JPWs88huWm+0kdmpSiOyBRXIGLZNQWj7mrhIsbmEJy6iF8vZNaAy1O2tRGKbw8
         rC+hHdyDad/0TuRR7x4GevISidyUAmsXEPR96PelreiOpHqfITFuT1VPvYNq+KZt1yhp
         CafRefUmXKIdT7Y4ozTauDqNzLNeS7NNAGd976OQ3eoH2ayx+6eqZDusu8OoZrXl7e/l
         HIJRCjObeKhR65cx/wQDx3AgxDiIrPsl9SnbH1Tx9Db0O1KookW/ZLSjGSq/CWf07aw+
         B0rW4ZxLDSUayKh7D1wHCGaqgma5m6uoqxQNW7fyZhVKA670JbHkzzogaWxMh6jnZuuV
         0fHA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707232609; x=1707837409;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=nt3pBAxgAVXBBQ7nYfBf27vXYcq8ELUyng70/ndviP0=;
        b=GyGbPn/eExiGPJE6WNTpm03xV67W6jJPKTW1EQazUQByrNcePntXzZ/BSRWqkj8ToR
         ZoEZSSSoX1lc+FSq6BclCC5okNakeXQixRMjP1kTEqr+qIPjlRH3zG4KQOXw1lanR0Io
         riDfRv+Kdcz9TiDIeunXq4sBvOQQunK6eklAnNESoinqhESYA1FAkJD2/PkHt33zBTDA
         tzd8THGdaS6YYnFVmVADU/L37FkMpTPAif73Xl4Bo5wF8LyXXnjAIok10lqAjxQNDSih
         u+S18o9FcC2hfqWubSWWdXXZQLC7/nQ2oitEmf4ZX8muO4DM72HMIYGaTpl5oFT6knZD
         U78g==
X-Gm-Message-State: AOJu0YyWVoaP2xUkuHZsNFkyRtScn+0RlmTXUZKlBfdafMwZxIhcSCOZ
	zuqMl3KPiM3UZThSFEuSluOPQP8AaRBwWoaVWMruI58k8ApWfW/4tMgl5KiXfIjdrMYpCnHIZPY
	gQ1g=
X-Google-Smtp-Source: AGHT+IFNo9k+1iOcZnBCKBwduPLLIMWgCsta3NPB65LSqL99J59FrXholPdx7B1B+lB+TLYX8imb9Q==
X-Received: by 2002:a5d:5f8d:0:b0:33b:28c0:7c98 with SMTP id dr13-20020a5d5f8d000000b0033b28c07c98mr1617011wrb.61.1707232609219;
        Tue, 06 Feb 2024 07:16:49 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCXOz4P2x4qja9CRke+qbQ3doz3a6hQjUCoHZIa/ktjRfGa61FIdkP+FfKZLLwT5QzqGC8c9dB8dyLUNVFYytEPdt8mVJt6L7LGEVE1DCOEva7pLf4Sb
From: =?UTF-8?q?zithro=20/=20Cyril=20R=C3=A9bert?= <slack@rabbit.lu>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?zithro=20/=20Cyril=20R=C3=A9bert?= <slack@rabbit.lu>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v2] tools/xentop: add option to display dom0 first
Date: Tue,  6 Feb 2024 16:15:45 +0100
Message-Id: <efdca3e0c02576c624403d273c7168c9303fcc52.1707232545.git.slack@rabbit.lu>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add a command line option to xentop to be able to display dom0 first, on top of the list.
This is unconditional, so sorting domains with the S option will also ignore dom0.

Signed-off-by: Cyril Rébert (zithro) <slack@rabbit.lu>

---
Changes in v2:
- bug fix
- add documentation
---
 docs/man/xentop.1.pod | 6 +++++-
 tools/xentop/xentop.c | 5 +++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/docs/man/xentop.1.pod b/docs/man/xentop.1.pod
index 126f43d2f3..593a484ce7 100644
--- a/docs/man/xentop.1.pod
+++ b/docs/man/xentop.1.pod
@@ -5,7 +5,7 @@ xentop - displays real-time information about a Xen system and domains
 =head1 SYNOPSIS
 
 B<xentop> [B<-h>] [B<-V>] [B<-d>SECONDS] [B<-n>] [B<-r>] [B<-v>] [B<-f>]
-[B<-b>] [B<-i>ITERATIONS]
+[B<-b>] [B<-i>ITERATIONS] [B<-z>]
 
 =head1 DESCRIPTION
 
@@ -57,6 +57,10 @@ output data in batch mode (to stdout)
 
 maximum number of iterations xentop should produce before ending
 
+=item B<-z>, B<--dom0-first>
+
+display dom0 first, ignoring interactive sorting
+
 =back
 
 =head1 INTERACTIVE COMMANDS
diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
index 9068c53fd2..5462e6a426 100644
--- a/tools/xentop/xentop.c
+++ b/tools/xentop/xentop.c
@@ -211,7 +211,7 @@ int show_networks = 0;
 int show_vbds = 0;
 int repeat_header = 0;
 int show_full_name = 0;
-int dom0_first = -1;
+int dom0_first = 0;
 #define PROMPT_VAL_LEN 80
 const char *prompt = NULL;
 char prompt_val[PROMPT_VAL_LEN];
@@ -1164,8 +1164,9 @@ void do_vbd(xenstat_domain *domain)
 static void top(void)
 {
 	xenstat_domain **domains;
-	unsigned int i, num_domains, sort_start, sort_count = 0;
+	unsigned int i, num_domains = 0;
 	int dom0_index = -1;
+	int sort_start = 0, sort_count = 0;
 
 	/* Now get the node information */
 	if (prev_node != NULL)
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 15:20:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 15:20:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676985.1053368 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNFR-00054n-2D; Tue, 06 Feb 2024 15:20:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676985.1053368; Tue, 06 Feb 2024 15:20:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNFQ-00054g-Vr; Tue, 06 Feb 2024 15:20:32 +0000
Received: by outflank-mailman (input) for mailman id 676985;
 Tue, 06 Feb 2024 15:20:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5Zit=JP=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rXNFQ-00054V-DR
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 15:20:32 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4276dcba-c503-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 16:20:29 +0100 (CET)
Received: from SA9PR13CA0106.namprd13.prod.outlook.com (2603:10b6:806:24::21)
 by CY8PR12MB8066.namprd12.prod.outlook.com (2603:10b6:930:70::5) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.14; Tue, 6 Feb
 2024 15:20:23 +0000
Received: from SN1PEPF00026367.namprd02.prod.outlook.com
 (2603:10b6:806:24:cafe::c8) by SA9PR13CA0106.outlook.office365.com
 (2603:10b6:806:24::21) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.17 via Frontend
 Transport; Tue, 6 Feb 2024 15:20:22 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF00026367.mail.protection.outlook.com (10.167.241.132) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Tue, 6 Feb 2024 15:20:21 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Tue, 6 Feb
 2024 09:20:21 -0600
Received: from XIR-MICHALO-L1.xilinx.com (10.180.168.240) by
 SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.34
 via Frontend Transport; Tue, 6 Feb 2024 09:20:20 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4276dcba-c503-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=N1qFRwkpm9JeAQMbJNmbPmx3lfTcrsDBWJYFQ1Yde+BXp0Ji5PD0FJzqS1GK/R5SZj1GA02jcqCzRfFjhX5i97Xunf/6oI8lQpzOvMNlynta8P0e4bAnkmUUG8yH4jJW2ZmXJFNyVIRjfh/M+nG53lOU3fpQKbhHFnKh1Rmm+j/Kx4qFmaoVxCwK+IvQUhCwZNTFdVAqPrTbDcPZeAK69xK4/XnNuvVErUJIl172JxVJR0SenxSg1rTVJuguQmhbcLNDx0dfIIdtfOPJ6Yl/yPSx7seG5s0qsvxt5+X046nyPXLEcgh2EywvaRzo5P1u4vfjthL8uzLYCtz2S92JFw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ObWFSn0auNly0ZikMZjFpNo0QtEDh16Yhyn3UiPb+uY=;
 b=DZpj3i9e4tVZtnLZRQDYhKlIh2XEbD30wdF5bxHqVqPvfd+S+8KpcJZ/yvGMhLmvOpNIMxQWkADP2dSqMKP3cVZ3I4IgN3oNdK9VqbIHCX+QCNHzwlD96LCMVr9M156kbm+w+OiBJAm/8qV+wEs7wOlh1df4XAJeVHV1hNjlbyJwKjveOE1nGU7Kag3wq9JpD6SX+K1m/W7uGu7lyTHI21I4x3iWnfktuxkM0WW7zdRR478G9yYyFt75xtt9r79zYv/DEoWzWp0I+CQF88OkNpiNx7hFAEdQZfPw0+LO//iTXo5aAQchj35VTFloS5QgGlFo4G8U+8s8z4uFjTIq5Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ObWFSn0auNly0ZikMZjFpNo0QtEDh16Yhyn3UiPb+uY=;
 b=BdHE+XenGgVGgNlxUoUl/UjMmgJInwj1jP+FI1J+DAijO9w8UzPRmGXCcVw4EO6HyScIHLqz2xL0F25AEKpwr2vUKkQ8kDBm72fe0x80r8uKfuu6zylaUV/RAUlSBVsFkOkLBvORkWff+NiWMBciH3Yg1xlrM9m79Ne6kApC6Bc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Michal Orzel <michal.orzel@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Michal Orzel <michal.orzel@amd.com>, Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>, <andrew.cooper3@citrix.com>
Subject: [PATCH] automation: Switch yocto-qemux86-64 job to run on x86
Date: Tue, 6 Feb 2024 16:20:12 +0100
Message-ID: <20240206152012.51107-1-michal.orzel@amd.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF00026367:EE_|CY8PR12MB8066:EE_
X-MS-Office365-Filtering-Correlation-Id: dfcdc488-6195-4e2d-1b5b-08dc272722b3
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	mO4qtd2SVCLa5146Cb2XDLL0IlyVaFqZcVZVhIWdDn7+NA1aWoRb6CJ2Y1p7GmrB74MwTvQFmClz5y5A30mJF6fwz78Q+fh8sO1m2mPD5IjjCXUhHdZdCiXCQj+Pgy03pYg4RHkmsmzQ+6gWnDNPdO44OQEDkjmQ7e9lKP+Ah7SaD9g5qrJ3vIWVkBQPtMMakNv5+RE5U4ZnP2W38VYEhkPWhWtmxkmJmmHyk/J8BhThb0oVP9GDO3/T+S6Zp4XnNNO5mbanbDYNcATVZdq9PJDebPK6pALr4LXy3k0AnIBYcagD/fA6Fdeu5nys5Je1RWfQ8BPgwr3OyLb5hWRrUesOCo2a4pROuL6oKrg74EPeeGQvoilBAEBPHo6gVmFYi2MK6au3Pg6xYPsNDCE4Py9v8+GPKUcocpilEbxxjygNoQA4XgBfVFW8ybQu1a65EmK4BoQPnjWhjtR3tPHeN6Oym3uJTdEKDeTuHgt8eaEQk5e7SaOYpVOctQFFXpXul+6ItRd7fUKC/I8aCpsGugxCkRHuEVqYJqI1Fd7tsHiuHUCuZpZtaO15gS59maWnTGLalHZx+j7H+G4IhMhizXJ6ZuxhKBYonN/0XyQR/ah8oFgjWQg9OBZo+PDf1YYYDaA3wAHV7wbG12LbTUBnln/QyH44r0gH9rdO6pMoxuXj+kl/Dnvr7nhQXUEEaTX5tolf7z+xucQXmW3LvTNf7aJcfXUltBIy0tNDwGKenpI/t/eeXrRZdhO/PPPx2S3+qP/v9UhzEBPGv7Cq5O9/3g==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(376002)(396003)(346002)(136003)(230922051799003)(1800799012)(451199024)(82310400011)(186009)(64100799003)(46966006)(36840700001)(40470700004)(36756003)(44832011)(8676002)(4326008)(8936002)(5660300002)(41300700001)(2906002)(966005)(36860700001)(86362001)(1076003)(2616005)(26005)(40480700001)(40460700003)(6666004)(81166007)(478600001)(83380400001)(47076005)(356005)(70586007)(316002)(6916009)(82740400003)(70206006)(426003)(336012)(54906003)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2024 15:20:21.8513
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: dfcdc488-6195-4e2d-1b5b-08dc272722b3
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF00026367.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8066

At the moment, all Yocto jobs run on Arm64 runners. To address CI
capacity issues, move yocto-qemux86-64 job to x86. Reflect the change in
the makefile generating Yocto docker files and fix CONTAINER name
definition that incorrectly expects YOCTO_HOST variable to be set for x86
container as well, which does not have a platform name appended.

Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
I built and pushed a new container to registry.
Pipeline:
https://gitlab.com/xen-project/people/morzel/xen-orzelmichal/-/pipelines/1166227598
---
 automation/build/yocto/yocto.inc |  6 ++++--
 automation/gitlab-ci/build.yaml  | 10 +++-------
 2 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/automation/build/yocto/yocto.inc b/automation/build/yocto/yocto.inc
index e4328ec7f378..2f3b1a5b2a34 100644
--- a/automation/build/yocto/yocto.inc
+++ b/automation/build/yocto/yocto.inc
@@ -18,11 +18,13 @@ YOCTO_TARGETS = qemuarm64 qemuarm qemux86-64
 YOCTO_ARCHS = amd64 arm64v8
 
 # Architecture we want to use in gitlab CI (depends on runners arch).
-CI_ARCH = arm64v8
+CI_ARCH-qemuarm64 = arm64v8
+CI_ARCH-qemuarm = arm64v8
+CI_ARCH-qemux86-64 = amd64
 
 define GEN_DOCKER
 # Make all is generating architecture we use in the CI.
-ifeq ($(CI_ARCH),$(3))
+ifeq ($(CI_ARCH-$(2)),$(3))
 CONTAINERS += yocto/$(1)-$(2)$(4)
 else
 CONTAINERS_EXTRA += yocto/$(1)-$(2)$(4)
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 32af30ccedc9..6d2cb18b8883 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -213,7 +213,7 @@
     - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs --xen-dir=`pwd` ${YOCTO_BOARD} ${YOCTO_OUTPUT}
   variables:
     YOCTO_VERSION: kirkstone
-    CONTAINER: yocto:${YOCTO_VERSION}-${YOCTO_BOARD}-${YOCTO_HOST}
+    CONTAINER: yocto:${YOCTO_VERSION}-${YOCTO_BOARD}${YOCTO_HOST}
   artifacts:
     paths:
       - 'logs/*'
@@ -224,16 +224,12 @@
 .yocto-test-arm64:
   extends: .yocto-test
   variables:
-    YOCTO_HOST: arm64v8
+    YOCTO_HOST: -arm64v8
   tags:
     - arm64
 
-# This is not used by any test job as we only run Yocto on arm based machines.
-# Keep it here so that someone having x86 hardware can easily add jobs.
 .yocto-test-x86-64:
   extends: .yocto-test
-  variables:
-    YOCTO_HOST: amd64
   tags:
     - x86_64
 
@@ -576,7 +572,7 @@ yocto-qemuarm:
     YOCTO_OUTPUT: --copy-output
 
 yocto-qemux86-64:
-  extends: .yocto-test-arm64
+  extends: .yocto-test-x86-64
   variables:
     YOCTO_BOARD: qemux86-64
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 15:26:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 15:26:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.676993.1053383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNKt-0005lL-Rc; Tue, 06 Feb 2024 15:26:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 676993.1053383; Tue, 06 Feb 2024 15:26:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNKt-0005lE-OS; Tue, 06 Feb 2024 15:26:11 +0000
Received: by outflank-mailman (input) for mailman id 676993;
 Tue, 06 Feb 2024 15:26:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=onkB=JP=ens-lyon.org=samuel.thibault@bounce.ens-lyon.org>)
 id 1rXNKs-0005jM-BY
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 15:26:10 +0000
Received: from sonata.ens-lyon.org (domu-toccata.ens-lyon.fr [140.77.166.138])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0a3e8872-c504-11ee-8a48-1f161083a0e0;
 Tue, 06 Feb 2024 16:26:04 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by sonata.ens-lyon.org (Postfix) with ESMTP id 8C8BFA0121;
 Tue,  6 Feb 2024 16:26:03 +0100 (CET)
Received: from sonata.ens-lyon.org ([127.0.0.1])
 by localhost (sonata.ens-lyon.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id uaO-G1aid_Ox; Tue,  6 Feb 2024 16:26:03 +0100 (CET)
Received: from begin (nat-inria-interne-52-gw-01-bso.bordeaux.inria.fr
 [194.199.1.52])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by sonata.ens-lyon.org (Postfix) with ESMTPSA id 36986A00FA;
 Tue,  6 Feb 2024 16:26:03 +0100 (CET)
Received: from samy by begin with local (Exim 4.97)
 (envelope-from <samuel.thibault@ens-lyon.org>)
 id 1rXNKk-0000000Ge82-3XMf; Tue, 06 Feb 2024 16:26:02 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a3e8872-c504-11ee-8a48-1f161083a0e0
Date: Tue, 6 Feb 2024 16:26:02 +0100
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org, julien@xen.org
Subject: Re: [PATCH] Mini-OS: fix 9pfs frontend error path
Message-ID: <20240206152602.5mbztpfam7x2ykcw@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org, wl@xen.org, julien@xen.org
References: <20240206061721.27297-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240206061721.27297-1-jgross@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Juergen Gross, le mar. 06 févr. 2024 07:17:21 +0100, a ecrit:
> The early error exit in p9_stat() returns without zeroing the p9_stat
> buffer, resulting in free() being called with an uninitialized pointer.
> 
> Fix that by doing the zeroing first.

This is not coherent with the usual conventions: when a function fails,
it is supposed not to have done anything, and thus the caller shouldn't
have to clean anything.

I.e. i'd rather see the free_stat() call be put after the check for
an error returned by p9_stat.

> Reported-by: Julien Grall <julien@xen.org>
> Fixes: 2d1dfccd3aa3 ("Mini-OS: add read and write support to 9pfsfront")
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  9pfront.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/9pfront.c b/9pfront.c
> index 315089bc..33eaadce 100644
> --- a/9pfront.c
> +++ b/9pfront.c
> @@ -716,10 +716,11 @@ static int p9_stat(struct dev_9pfs *dev, uint32_t fid, struct p9_stat *stat)
>      uint16_t total;
>      int ret;
>  
> +    memset(stat, 0, sizeof(*stat));
> +
>      if ( !req )
>          return EAGAIN;
>  
> -    memset(stat, 0, sizeof(*stat));
>      req->cmd = P9_CMD_STAT;
>      send_9p(dev, req, "U", fid);
>      rcv_9p(dev, req, "uuuUQUUULSSSSSUUU", &total, &stat->size, &stat->type,
> -- 
> 2.35.3


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 15:37:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 15:37:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677005.1053397 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNVk-0007gu-RG; Tue, 06 Feb 2024 15:37:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677005.1053397; Tue, 06 Feb 2024 15:37:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNVk-0007gn-OE; Tue, 06 Feb 2024 15:37:24 +0000
Received: by outflank-mailman (input) for mailman id 677005;
 Tue, 06 Feb 2024 15:37:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=z75a=JP=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXNVi-0007ev-VN
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 15:37:22 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9c9d8f69-c505-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 16:37:19 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 818A51FB70;
 Tue,  6 Feb 2024 15:37:18 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4B76B132DD;
 Tue,  6 Feb 2024 15:37:18 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id cPTyEC5SwmUAZAAAD6G6ig
 (envelope-from <jgross@suse.com>); Tue, 06 Feb 2024 15:37:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9c9d8f69-c505-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707233838; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1SKFxj/bg0P9ufOg8Y/28ozfDvOWazBqn9Gcw17kpOM=;
	b=Ayn0njpcnlgZL3jp3DO/89+vDsVsuT8hFGJstI4YnXeFgy4SopFLWQ8/BqNotqCrHeVSm0
	a5g1gcsWYmnrK2chpCJE4eJtDM44mgcCKIeZFdlERlfHQJuMMqgIcqX7QUgR1PmVwbOAVj
	UXJaExYxjGvk/WMd6546jScTcZ9Yn7o=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707233838; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1SKFxj/bg0P9ufOg8Y/28ozfDvOWazBqn9Gcw17kpOM=;
	b=Ayn0njpcnlgZL3jp3DO/89+vDsVsuT8hFGJstI4YnXeFgy4SopFLWQ8/BqNotqCrHeVSm0
	a5g1gcsWYmnrK2chpCJE4eJtDM44mgcCKIeZFdlERlfHQJuMMqgIcqX7QUgR1PmVwbOAVj
	UXJaExYxjGvk/WMd6546jScTcZ9Yn7o=
Message-ID: <6ff940a5-5918-4da4-9a1e-ebe6fe9570a0@suse.com>
Date: Tue, 6 Feb 2024 16:37:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: fix 9pfs frontend error path
Content-Language: en-US
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
 wl@xen.org, julien@xen.org
References: <20240206061721.27297-1-jgross@suse.com>
 <20240206152602.5mbztpfam7x2ykcw@begin>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240206152602.5mbztpfam7x2ykcw@begin>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [0.48 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-0.34)[76.15%];
	 MIME_GOOD(-0.10)[text/plain];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.91)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: 0.48

On 06.02.24 16:26, Samuel Thibault wrote:
> Juergen Gross, le mar. 06 févr. 2024 07:17:21 +0100, a ecrit:
>> The early error exit in p9_stat() returns without zeroing the p9_stat
>> buffer, resulting in free() being called with an uninitialized pointer.
>>
>> Fix that by doing the zeroing first.
> 
> This is not coherent with the usual conventions: when a function fails,
> it is supposed not to have done anything, and thus the caller shouldn't
> have to clean anything.
> 
> I.e. i'd rather see the free_stat() call be put after the check for
> an error returned by p9_stat.

I can do that, but this would require two calls of free_stat() (one in
p9_stat() in an error case reported via req->result, and one in the
caller of p9_stat() in case of no error).


Juergen


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 15:44:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 15:44:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677016.1053407 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNcK-00015W-Hu; Tue, 06 Feb 2024 15:44:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677016.1053407; Tue, 06 Feb 2024 15:44:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNcK-00015P-ED; Tue, 06 Feb 2024 15:44:12 +0000
Received: by outflank-mailman (input) for mailman id 677016;
 Tue, 06 Feb 2024 15:44:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=T6vd=JP=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rXNcI-00015J-Hc
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 15:44:10 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8feef67d-c506-11ee-8a48-1f161083a0e0;
 Tue, 06 Feb 2024 16:44:08 +0100 (CET)
Received: from SA1PR05CA0006.namprd05.prod.outlook.com (2603:10b6:806:2d2::10)
 by MW5PR12MB5622.namprd12.prod.outlook.com (2603:10b6:303:198::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.14; Tue, 6 Feb
 2024 15:44:03 +0000
Received: from SN1PEPF0002529F.namprd05.prod.outlook.com
 (2603:10b6:806:2d2:cafe::4a) by SA1PR05CA0006.outlook.office365.com
 (2603:10b6:806:2d2::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.17 via Frontend
 Transport; Tue, 6 Feb 2024 15:44:03 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SN1PEPF0002529F.mail.protection.outlook.com (10.167.242.6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Tue, 6 Feb 2024 15:44:02 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Tue, 6 Feb
 2024 09:44:01 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Tue, 6 Feb
 2024 07:44:01 -0800
Received: from [172.23.114.18] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Tue, 6 Feb 2024 09:43:59 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8feef67d-c506-11ee-8a48-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IwHKLgJBvmpC0YzBpPTi+hwV828OIdVdPvAIkm6CN8tl/caH9cQrdhbIFpeb2ngsVDvR4rsMYUeobuPcyNSHjEM6GgtbtPbKMwPoBY+jou2zuX2DXqt+OgOnGyLMhPkK/hAdCIQlWxgzRsCISuEgcAjb2kHf7dEJZkXG3bHUzjwyoz/KfRD8Pg1ATsk6dRSqmWh9U0w2aIecdaClE+P8D5Tg3T1sf2gMXsy+g3Wr0qevQXrYrGc1Xr8wjPBJZMjgGzf7UyeLz9aNJBjLLceggJ3dVlgsiv5chUFYPCeQ/gPNKMTCAMCTlCxbOS3AmJE0wKkXNBqivJsfdUTA9GbwgA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=uW+dyA4uXT1vZWzti66UCaFqrNJtNdR9zsDv9KTJ3eY=;
 b=BhNCUUVKoS+Yyw/FtzcoLnYMEaY6dU6PelaFx5U0TZ6bjCbNakzKxfaOFpkxBKQIp2Rocdb611a5pHs4uDLcJd/jUlrr3zu4/ta7put/gIYWZ08U+cSzntftFNveMZusI3Q5M8XwTI48iuTmLnItUgyKdz+aBmSfaKDni6uPAwTbemvAilq8hohKeaX5fN2TQk6mAkGhmGeeBl/2FxdMXxkEjtlwHuD9khQhXUqk9SsIHncZVKI53u4YnLynHBOS9ji3UDIBpK36ETA81vDwUWTUt0L9sGA9GWysv3wQwYUnEBLJipFH+M/QSzVoqZBEXkgcDlorHyCrc4aAInBg0g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uW+dyA4uXT1vZWzti66UCaFqrNJtNdR9zsDv9KTJ3eY=;
 b=iETKXN4DhY5ELucE175SoAcrjFE2FilfN24V6sUi199Tfd8E2Jr+cAjeb/zEEeDnJw33QIbSirJ/ELKkKZsXjY6pjZLeurgVwSxY04hCGxCb80ohFfzm2nER0d0ErauxUSiTnijW5GytufWNM9r5Y5uVbaCx1nyIqXnCX+XDQiM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <b9d8ddcd-cdf1-414f-9990-7c2fd3d88f4b@amd.com>
Date: Tue, 6 Feb 2024 10:43:54 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Jan Beulich
	<jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Jun Nakajima
	<jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>, Paul Durrant
	<paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <20240202213321.1920347-2-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF0002529F:EE_|MW5PR12MB5622:EE_
X-MS-Office365-Filtering-Correlation-Id: 9f966c06-e0b3-4cc9-9a7e-08dc272a7177
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	7MMACuPhJEhnZ1V6gCoC/C0miSB8ujupGsoVg87urkkujswFbxI2lNDzAQkbANry/utQEKaKtEZ0pYTLbo4hgUXyE2kxA8qrEkqGM8kt7enI5ZlYnG0+nJhCA/Tn9HYkzfpRwlUCOyjlUnWhb0Iqcw3WuCSH45+OKzwC8zrSOX0B6x0xXiF/njkYu9l05nVD9qYOoOALNAv6eI65HenRrBmx8boJRHU6docQY3ZFc0C5cyUhBV5N0Pqjb/K9OglHV/MdQOBgLh+7VABuOI0v259pCiuLobFMk+tPDc9t/7Lgfags2ZKeED7mA5ak5OrH85pqg/EIG2OlakzHgSmOODsNn5/8Cc62UPQ5zoxAwx41XVOEZqNyCLGN5oQeVAFf9TAk6VoOAgDnSoaYxqWkomQ1Vn8B5ro/ZtIDqF8Gf057+lwER7LnYO4Qz12y2moQdbMyPCff3j4XqcRvGM8hQjoWBgx3Y4TFCVYJS4+KYYky5IA+f+1h5AvhfSyBnMseG7VOhsTP624PqS0mXKYAegy80S9slBOUs9luEtcPhwAj06/ZT9E77I8iSbwCiGD1dqvWCdHLPIldpYErzVTTgtzToH7rlBj4CnPXC9NLGrcLv+j5MvP+98JHPA6d+kPCDNIM3BxAIk7AzH9QNJDEcJ8xNO0j/U5zgMaoY/zxhzGOxISFj1sEYxqhSIptxXPTR/8sND/afk34YiESePY0Z99WewdERtCYhpquGyrik3Vmdl3T/Hws+WNvPqW0cyZs+6g8jqhugHhDkwyeUTERmwIK6Ct1OutkKPECjY0fipk=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(346002)(136003)(376002)(396003)(230922051799003)(64100799003)(451199024)(186009)(82310400011)(1800799012)(36840700001)(46966006)(40470700004)(6666004)(26005)(2616005)(31696002)(83380400001)(70586007)(70206006)(356005)(41300700001)(82740400003)(336012)(36860700001)(966005)(426003)(8676002)(8936002)(316002)(2906002)(16576012)(6916009)(81166007)(54906003)(4326008)(44832011)(5660300002)(7416002)(86362001)(47076005)(53546011)(478600001)(36756003)(40460700003)(40480700001)(31686004)(36900700001)(43740500002);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2024 15:44:02.5073
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9f966c06-e0b3-4cc9-9a7e-08dc272a7177
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF0002529F.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR12MB5622

On 2/2/24 16:33, Stewart Hildebrand wrote:
> ---
> Changes in v13:
>  - hold off adding Roger's R-b tag even though it was provided on v12.2
>  - use a wrapper construct to ease readability of odd-looking ASSERTs
>  - new placement of ASSERT in __pci_enable_msix(), __pci_enable_msi(),
>    and pci_enable_msi(). Rearrange/add pdev NULL check.
>  - expand commit description with details about using either
>    pcidevs_lock() or d->pci_lock
> 
> Changes in v12.2:
>  - drop Roger's R-b
>  - drop both locks on error paths in vpci_msix_arch_print()
>  - add another ASSERT in vpci_msix_arch_print(), to enforce the
>    expectation both locks are held before calling vpci_msix_arch_print()
>  - move pdev_done label in vpci_dump_msi()
>  - update comments in vpci_dump_msi() to say locks (plural)

Here's a patchew link to show just the diff-of-diff from v12.2 (where
Roger had given a R-b) to v13.

https://patchew.org/Xen/20240115194309.45683-1-stewart.hildebrand@amd.com/diff/20240202213321.1920347-2-stewart.hildebrand@amd.com/


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 16:05:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 16:05:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677027.1053417 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNwU-0004uo-4h; Tue, 06 Feb 2024 16:05:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677027.1053417; Tue, 06 Feb 2024 16:05:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXNwU-0004uh-24; Tue, 06 Feb 2024 16:05:02 +0000
Received: by outflank-mailman (input) for mailman id 677027;
 Tue, 06 Feb 2024 16:05:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HR0g=JP=gmail.com=euidzero@srs-se1.protection.inumbo.net>)
 id 1rXNwS-0004ub-Qj
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 16:05:00 +0000
Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com
 [2607:f8b0:4864:20::102a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 79567307-c509-11ee-8a48-1f161083a0e0;
 Tue, 06 Feb 2024 17:04:58 +0100 (CET)
Received: by mail-pj1-x102a.google.com with SMTP id
 98e67ed59e1d1-296a02b7104so1780613a91.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 08:04:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 79567307-c509-11ee-8a48-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707235497; x=1707840297; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=BDD4xO+tU5cqx3drot12zKF0OHarPvqjDEeUhqYVy3k=;
        b=DQEmJM0hOaWqEIocYsZMHym5ImVOAZWr8/J2iPa23F3IZyIs9F6gOYKI0+8KyUrYVx
         VR8Yjwzp7b0TFchkERF/QFmFRJmk8+l66wwLaDZ1R6FbZ03coejmrttnRdn8n8aKzkwA
         CMRl332uIGI4kXUODSBSzAZkJxjzGZiSUJkJEqUGQgK6MJiiC4OuxZW+xQhszP6kybDe
         ONDp/T1MqODL2hJ2JXizbCDLPyYRzk4sXf9A5ewoJksSryou7FyV+RRhe22mQt75xWHH
         UjVjDNeTUReFExLAKhHD/0/k4XuItHRcCpwyYw15soArKcsgforxhNgOYtyGRHHSue68
         aong==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707235497; x=1707840297;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=BDD4xO+tU5cqx3drot12zKF0OHarPvqjDEeUhqYVy3k=;
        b=eT6edCAmXH8Awq0plh5Dr6oYPnqV/vEFUtXQTP+jQ0n8VntbiWUwuTkY0BC2AHsCgA
         SamEMBK9yZnYwP8WFgT9jwGmtDD+yx4Wyzdzk//+HjS5Cqx+QVZp4VkDd652JedG3zpp
         Fp6jspHY9F/XkenY/dpHXgeEaLTeiHwWDUlwQ4kAWov08sexvTYj8TcV3SBEwuN6QLPl
         zMh7hLRdNxn38AJzs2SVZFSuiKOJNbmwA12WM5tKFgRvh2Y2iw9/tGFzYxK0aJ9wnAQu
         WTYixasnUdF5hl7DEknEDq8DNEKXyQIzQ4qYqwFnKIDAsBqYDhDzfmbrJwx0CGEj7veI
         jg/w==
X-Gm-Message-State: AOJu0YyBeVh1h2i4Nujms7s+ZzZWf2fph/qqmUnacCfJGyCAbNUg1oZ7
	0cJjGfVA2Dzso7+L2hDbETTCcpniPEwqzwohkCnFzjyJYO9aBe1QNcrVz57AkMrhIPDIspQG98p
	qmNA4ob2aK+rfG2fCf98q4RrQfSMpaRAAwBFTwg==
X-Google-Smtp-Source: AGHT+IFZAAk48D1jEvUq5gFRdS9N76/AJ2mA/VAKw7vxAkd1JX9MoN1D6ko+oOa60+ThMyKujom2BqS1wp81F+G4sZE=
X-Received: by 2002:a17:90a:e643:b0:296:344d:5a0b with SMTP id
 ep3-20020a17090ae64300b00296344d5a0bmr2759314pjb.15.1707235497020; Tue, 06
 Feb 2024 08:04:57 -0800 (PST)
MIME-Version: 1.0
References: <CAKm-Umas=5=JzooRzPHxUDigKpFK3Ze7cQcG8eR5kWgn-d_7fQ@mail.gmail.com>
 <CAKm-UmaUZ=rj-F3RvMoR7H3OvBHm1iy4PTRZAjyi=MQUVTuYVg@mail.gmail.com>
 <CAKm-Umat77MTh6=GEuH+bf1WRX+_B4-N5hY4xwADALfhj4QCPA@mail.gmail.com>
 <6ba9568a-ff8e-48ad-907f-74303e04d3a2@suse.com> <CAKm-UmZVoUhidEV15Xk+J_XN5a2GsBqMbXL9wqdFdoAFf_u5AA@mail.gmail.com>
 <bcbceba7-42ac-45be-80ab-e3dfe7741ec6@suse.com> <2eafe45b-fd97-472c-a173-459d6ff0b957@suse.com>
 <CAKm-Umbs2D7NHFE+NW2xKOu7-AZhFpH4uzE5QTuQusnGA_eNWA@mail.gmail.com>
 <f539144c-885a-461c-a506-bdb73e626a65@suse.com> <CAKm-UmY126AfdGhWcZ3s9vwN9+ksVRRFEhOu0ZFMoDvxrqOibw@mail.gmail.com>
 <CAKm-UmYt3iV8zOhSmtqMGhi_8T93c_nCbnefs4m3UC+3UABqeQ@mail.gmail.com>
 <CAKm-UmY-KXEAtBagikW4Jvp=SFXtmEg8P62pHfSo3Hr2s-0_-A@mail.gmail.com>
 <CAKm-UmYbMCfXc1Ny0=qfB+UaLSXV9oEHZiSgS=mwKMwmOFGVrQ@mail.gmail.com>
 <77c1c05d-a0a0-4292-9257-9b7fbebee0e3@suse.com> <2859ad22-8714-4628-8142-fc77fc2072c3@amd.com>
 <CAKm-UmZpyGkWXugYTJqU+qqVDyCFEKghtd=NTr2wK5EMCeL9Ww@mail.gmail.com>
 <214585d5-689d-4ba6-bd48-359428a7ed8f@amd.com> <CAKm-UmZoKwre8-G793VqRNFCmzAti1o-0Kp3ZyV_Z5cc0YNiKw@mail.gmail.com>
 <CAKm-Umb=kGFqc5je9E3bbfQ0bcbZeY_Ntv5JDmO-vXj3N0MvPg@mail.gmail.com>
In-Reply-To: <CAKm-Umb=kGFqc5je9E3bbfQ0bcbZeY_Ntv5JDmO-vXj3N0MvPg@mail.gmail.com>
From: =?UTF-8?Q?S=C3=A9bastien_Chaumat?= <euidzero@gmail.com>
Date: Tue, 6 Feb 2024 17:04:45 +0100
Message-ID: <CAKm-UmZ113q-a8wEE5yo0OPPM3JpNqJzKaU1eNiCzT2YkGU0pQ@mail.gmail.com>
Subject: Re: [BUG]i2c_hid_acpi broken with 4.17.2 on Framework Laptop 13 AMD
To: xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Mario Limonciello <mario.limonciello@amd.com>, 
	Juergen Gross <jgross@suse.com>, 
	=?UTF-8?Q?Marek_Marczykowski=2DG=C3=B3recki?= <marmarek@invisiblethingslab.com>
Content-Type: multipart/alternative; boundary="0000000000005921250610b8bdd3"

--0000000000005921250610b8bdd3
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Le jeu. 1 f=C3=A9vr. 2024 =C3=A0 13:30, S=C3=A9bastien Chaumat <euidzero@gm=
ail.com> a
=C3=A9crit :

> I spotted the following warning for IRQ7 (along with IRQ6 and 10)
>>
>> [    0.686073] fedora kernel: __irq_set_trigger: genirq: No set_type
>> function for IRQ 7 (IR-IO-APIC)
>>
>> This comes from kernel/irq/manage.c
>>
>>
>> int __irq_set_trigger(struct irq_desc *desc, unsigned long flags)
>> {
>> struct irq_chip *chip =3D desc->irq_data.chip;
>> int ret, unmask =3D 0;
>>
>> if (!chip || !chip->irq_set_type) {
>> /*
>> * IRQF_TRIGGER_* but the PIC does not support multiple
>> * flow-types?
>> */
>> pr_debug("No set_type function for IRQ %d (%s)\n",
>> irq_desc_get_irq(desc),
>> chip ? (chip->name ? : "unknown") : "unknown");
>> return 0;
>> }
>> Could this have a role in the IRQ misconfiguration by xen ?
>>
>
>  Things are getting even weirder :
>
> xen 4.18.1-pre (vanilla, no patching of pci_xen_initial_domain()), kernel
> 6.8.0-rc2) :
>
> checking /sys/kernel/irq/7 :
>
> actions: pinctrl_amd
> chip_name: xen-pirq
> hwirq:
> name: ioapic-edge
> per_cpu_count: 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0
> type: edge
> wakeup: disabled
>
> now checking  xen with xl debug-key i
>
> (XEN)    IRQ:   7 vec:51 IO-APIC-level   status=3D030 aff:{13}/{13}
> in-flight=3D0 d0:  7(---)
> (XEN)     IRQ  7 Vec 81:
> (XEN)       Apic 0x00, Pin  7: vec=3D51 delivery=3DFixed dest=3DP status=
=3D1
> polarity=3D1 irr=3D1 trig=3DL mask=3D0 dest_id:0000510d
>
> So even after dom0 is set up, the kernel considers  IRQ7 is of type edge
> while xen has it registered as IO-APIC-level.
>
> Reasoning backward  (using a  kernel without the pinctrl_amd driver to
ensure xen only is at stake) :
 checking the diff in IOAPIC  between bare metal and xen  (IRQ7 is on pin07
on APIC )

using kernel argument : apic=3Ddebug

bare metal :
[    0.715330] fedora kernel: ... APIC VERSION: 81050010
...
[    0.715433] fedora kernel:  pin07, disabled, edge , high, V(00), IRR(0),
S(0), physical, D(0000), M(0)

xen :
[    2.249582] fedora kernel: ... APIC VERSION: 00000014
...
[    2.249730] fedora kernel:  pin07, disabled, level, low , V(60), IRR(0),
S(0), physical, D(0000), M(0)

So the APIC table is not the same.

As strange as it looks the  (IOAPIC 0) pin07 is correctly described by the
APIC in xen but yet differently than in baremetal.
But the APIC message comes long after the
[    1.833145] fedora kernel: xen: registering gsi 7 triggering 0 polarity =
1

so I wonder if the APIC pin07 info had any influence.

--0000000000005921250610b8bdd3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div>Le=C2=A0jeu. 1 f=C3=A9vr. 2024 =
=C3=A0=C2=A013:30, S=C3=A9bastien Chaumat &lt;<a href=3D"mailto:euidzero@gm=
ail.com" target=3D"_blank">euidzero@gmail.com</a>&gt; a =C3=A9crit=C2=A0:<d=
iv class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:=
0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">=
<div dir=3D"ltr"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quot=
e" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204)=
;padding-left:1ex"><div dir=3D"ltr"><div>I spotted the following warning fo=
r IRQ7 (along with IRQ6 and 10)<br></div><div><br></div><div><div>[ =C2=A0 =
=C2=A00.686073] fedora kernel: __irq_set_trigger: genirq: No set_type funct=
ion for IRQ 7 (IR-IO-APIC)</div></div><div><br></div><div>This comes from k=
ernel/irq/manage.c</div><div><br></div><div><br></div>int __irq_set_trigger=
(struct irq_desc *desc, unsigned long flags)<br>{<br>struct irq_chip *chip =
=3D desc-&gt;irq_data.chip;<br>int ret, unmask =3D 0;<br><br>if (!chip || !=
chip-&gt;irq_set_type) {<br>/*<br>* IRQF_TRIGGER_* but the PIC does not sup=
port multiple<br>* flow-types?<br>*/<br>pr_debug(&quot;No set_type function=
 for IRQ %d (%s)\n&quot;,<br>irq_desc_get_irq(desc),<br>chip ? (chip-&gt;na=
me ? : &quot;unknown&quot;) : &quot;unknown&quot;);<br>return 0;<br>}<div>C=
ould this have a role in the IRQ misconfiguration by xen ?</div><div><div><=
/div></div></div></blockquote><div><br></div><div>=C2=A0Things are getting =
even weirder : <br></div><div><br></div><div>xen 4.18.1-pre (vanilla, no pa=
tching of  pci_xen_initial_domain()), kernel 6.8.0-rc2) :<br><div><br></div=
><div>checking /sys/kernel/irq/7 :</div><div><br></div><div>actions: pinctr=
l_amd<br>chip_name: xen-pirq<br>hwirq: <br>name: ioapic-edge<br>per_cpu_cou=
nt: 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0<br>type: edge<br>wakeup: disabled</div>=
<div><br></div><div>now checking=C2=A0 xen with xl debug-key i</div><div><b=
r></div><div>(XEN) =C2=A0 =C2=A0IRQ: =C2=A0 7 vec:51 IO-APIC-level =C2=A0 s=
tatus=3D030 aff:{13}/{13} in-flight=3D0 d0: =C2=A07(---)</div><div><div>(XE=
N) =C2=A0 =C2=A0 IRQ =C2=A07 Vec 81:<br>(XEN) =C2=A0 =C2=A0 =C2=A0 Apic 0x0=
0, Pin =C2=A07: vec=3D51 delivery=3DFixed dest=3DP status=3D1 polarity=3D1 =
irr=3D1 trig=3DL mask=3D0 dest_id:0000510d</div><div><br></div><div>So even=
 after dom0 is set up, the kernel considers=C2=A0 IRQ7 is of type edge whil=
e xen has it registered as IO-APIC-level.</div><div><br></div></div></div><=
/div></div></blockquote><div>Reasoning backward=C2=A0 (using a=C2=A0 kernel=
 without the pinctrl_amd driver to ensure xen only is at stake) :</div><div=
>=C2=A0checking the diff in IOAPIC=C2=A0 between bare metal and xen=C2=A0 (=
IRQ7 is on pin07 on APIC ) <br></div><div><br></div><div>using kernel argum=
ent : apic=3Ddebug</div><div><br></div><div>bare metal :</div><div>[ =C2=A0=
 =C2=A00.715330] fedora kernel: ... APIC VERSION: 81050010</div><div>...</d=
iv><div>[ =C2=A0 =C2=A00.715433] fedora kernel: =C2=A0pin07, disabled, edge=
 , high, V(00), IRR(0), S(0), physical, D(0000), M(0)</div><div><br></div><=
div>xen :<br></div><div>[ =C2=A0 =C2=A02.249582] fedora kernel: ... APIC VE=
RSION: 00000014</div><div>...</div><div>[ =C2=A0 =C2=A02.249730] fedora ker=
nel: =C2=A0pin07, disabled, level, low , V(60), IRR(0), S(0), physical, D(0=
000), M(0)</div><div><br></div><div>So the APIC table is not the same.</div=
><div><br></div><div>As strange as it looks the=C2=A0 (IOAPIC 0) pin07 is c=
orrectly described by the APIC in xen but yet differently than in baremetal=
.</div><div>But the APIC message comes long after the <br></div><div>[ =C2=
=A0 =C2=A01.833145] fedora kernel: xen: registering gsi 7 triggering 0 pola=
rity 1</div><div><br></div><div>so I wonder if the APIC pin07 info had any =
influence.</div><div><br></div><div><br></div><div><br></div><div><br></div=
><div><br></div></div></div>

--0000000000005921250610b8bdd3--


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 16:18:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 16:18:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677035.1053427 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXO8t-0006y9-BL; Tue, 06 Feb 2024 16:17:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677035.1053427; Tue, 06 Feb 2024 16:17:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXO8t-0006y2-8G; Tue, 06 Feb 2024 16:17:51 +0000
Received: by outflank-mailman (input) for mailman id 677035;
 Tue, 06 Feb 2024 16:17:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bmrB=JP=outlook.it=giuseppe.de.rosa@srs-se1.protection.inumbo.net>)
 id 1rXO8r-0006vo-Nd
 for xen-devel@lists.xen.org; Tue, 06 Feb 2024 16:17:49 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05olkn20800.outbound.protection.outlook.com
 [2a01:111:f403:2e13::800])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 426c56a4-c50b-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 17:17:45 +0100 (CET)
Received: from DB9P192MB1434.EURP192.PROD.OUTLOOK.COM (2603:10a6:10:296::21)
 by DU0P192MB1951.EURP192.PROD.OUTLOOK.COM (2603:10a6:10:411::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.36; Tue, 6 Feb
 2024 16:17:42 +0000
Received: from DB9P192MB1434.EURP192.PROD.OUTLOOK.COM
 ([fe80::3954:cef6:c539:697a]) by DB9P192MB1434.EURP192.PROD.OUTLOOK.COM
 ([fe80::3954:cef6:c539:697a%6]) with mapi id 15.20.7249.035; Tue, 6 Feb 2024
 16:17:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 426c56a4-c50b-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hCUtJHrn8hzxkeTzsYNGuOLAlwOpQBIyyDdUFka9i8yFh41GSvI03yUm1AQr1Ik11Dnm22D3sEPT/Q23d3X37RX6QZxf3T8RjY1bV39Rd/u/xHYUFPS4iUOFu6DhEYJ9kqqCa0+rGTm7cCjHwkCTWNWraY6Vx9u5VM8WwV99CkGRyQ017nWC19Cn6bxvssh7DtpDoV+GV68dInJnXrj9HHrQDPPaSpZgPNvSMMCTj4P4n0mGPEMFtASc94cif2AjT/3r30kPMW9jwN4PhjLW0gOK5nG0exAVuPw9ymmtbqZJktDVEtT2VdHeZ3g5y+D6CPKOgDEGL/5k7npYs6iGMw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=7kycROZqAaTkyG+kYa3gPHkmWsxhIihlVtcW94aOH6c=;
 b=B4wyJzsN5a4gBCJqdujznaiZhOEIHUqJJcPGrHZGdLk5oTeZnqywZNk3X/mpYNOaedcHqagGAa/XzlxrJSWuUFya41sgtkFJkqMPEtD1tMNYWsdpZziYYsTGjqKLmihjieJafuAh6aGOkZYeqNHrhAE17W5n/Xupol3doGZlEV/f081O4NefQxa6q8AitZQ5adj1gB6C2NlvuUAOvqId1AgVJ/zV2xIk6vCKoSSulERZ+FQradEgSt5icY77nVugIuCjLPs/UqcXcc9czmXvjQEr9TJcX06jEIvwlJXY76zb2iNOM/vwcFb2Ghs/m6bc/0FZ6p3haxhL/V8wsJpnaw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none;
 dkim=none; arc=none
From: Giuseppe De Rosa <giuseppe.de.rosa@outlook.it>
To: Andrew Cooper <andrew.cooper3@citrix.com>, "xen-devel@lists.xen.org"
	<xen-devel@lists.xen.org>
Subject: R: [BUG] Possible FATAL PAGE FAULT in domain_crash function on Xen
 4.14.6
Thread-Topic: [BUG] Possible FATAL PAGE FAULT in domain_crash function on Xen
 4.14.6
Thread-Index: AQHaWPFxTFnf2PP9dU2mKqDyY7lMybD9WsMAgAAheyA=
Date: Tue, 6 Feb 2024 16:17:42 +0000
Message-ID:
 <DB9P192MB143408981DB724A477C0ADA6A6462@DB9P192MB1434.EURP192.PROD.OUTLOOK.COM>
References:
 <DB9P192MB1434AD2FD1069E39FB480068A6462@DB9P192MB1434.EURP192.PROD.OUTLOOK.COM>
 <5cb25d4f-c3be-4c45-a434-421e1bf54be3@citrix.com>
In-Reply-To: <5cb25d4f-c3be-4c45-a434-421e1bf54be3@citrix.com>
Accept-Language: it-IT, en-US
Content-Language: it-IT
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
x-tmn: [RltGXoKOyqgU5NHietxuq3XOYJ6cYoPKdcTFnw77gm0=]
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DB9P192MB1434:EE_|DU0P192MB1951:EE_
x-ms-office365-filtering-correlation-id: 1a7e45af-fe60-4161-3fba-08dc272f2576
x-ms-exchange-slblob-mailprops:
 WaIXnCbdHrOEviK8+uzePdFfNisOgVEmkoxbfNPwsUIZpudLBB2q+9cq/tPX4qPvM9FK6TzTI9rXOBABhonhKmXq1H3F2xXUyeji9R/Jt2YNKYtQopjSEoamd26t5/SYcWiwenN6naua5uiNTod1ZDoMgjcnY01FOL9H3+wA+jtsZbEwuJqJOfKmJOT6QXpFcUhVIvnsQ0eW+MHnJUCBnQWpTynyoRgUJ6gr6kNQ8bTm9kSL272yHV5bmeC41HI9SgQOa34jMmvwghi8BVcYh1b8e6YEQ4x0XYdCovJTGuOo8BgpseJjUCBP62190Kf1ZCtENnJXDyDPYiQwNdJ6SV6z0XYNhbsPhbjr+z2OWiU8hLWxaJ1rvOAbVfotWL4m8lDUIuy/2GWtnVrL30k9/0Cc2G6ORqqzh1AIWrzZ0pcCjJLY508ketau8gOldP3dykQpsLNHLtKQW/zXkNISv2Vmy8sxHT5rKrAS5sEmEG41ZRmJOcPOJ3UB9vvxikmBBD0iyeryTI+6vaSkNlvHQFkAFmJtIACmf1dUCdkiwi5b62FbsCpkbGARl+wQp2i5kCpETiBcNogZIK2HD7n9aJwJcCWI7eZZ9SvzcNIwqq+hfMbCm9GSDCHdb8dmmBb28hH+B2rvi0cFwnFCh2WvWAbFsBr8YpY+s85Cb05f5Lam0C6eqNsnMVjJNGGuP7+3TdT/Ux283crDnhuy8/PopAnl/ENwGvK8Oh4PHnJrw6yWQ0YTh+21tTLqB44xFQYpf7n9/sd2BbU=
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 9sVgsrjV/J6jqibLehPI/Fqk7mRDY3Zf4YCmEZI86Ij3YlS5mbh7xsPWkfrk6MwQyH/rMGygyEK+OljHSoZO67eh4o7jKS91eqdQTNYnAYjKXGcopxFAlDzpdD6hrmfjYiFDpo/iux2/4selZVMIKKADCgZDggQk/CibyugU2RIRG0Zh7uUFx1iuEPpi2saazpqYIqzfRJd/9saW/rS4crMpgcJuEDC5xkZVHogZHFSNdK72LrLRfBEL/WkAV1jUOs5ctTFcchEyDhnZv+A/9mICu+2H0B63n4rCe8aXQbHp+zELijy4fKyWNc9ywgueel6LT4MGCBeP64LlbnvtK12Dg7A2y3sh9YmHnAr5HmXx9ujpJPr5BFvaSgL2x9IIMWZsORqR1efYqCf8zwpa8gnjGhQfGigWjI8kd81ZbB2yO2HSUaQeR8z27OT8pDNRMZu2LGzd5X3BjPn8nKPg0VmGFhBadfnkqtr+hVRtofmt/C/x7y0BXhRlCAfqOzniE7TZXp0Bizm+5Zh6AkxPu91QiNW7nxXYxSrQftDEZNBAQ3N0/mpYfe7EU2SX52dc
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?iso-8859-1?Q?+Amfm7Lm5R98xgsH+GNdYsDQGrHcRa9w55HsaP/aMJvMQSdQb0fNo770cJ?=
 =?iso-8859-1?Q?hhJs9EYb4ZZFW6PM6nQNPSiw9wEPX/gCvHoomUh0XQ7NEznKph9040yoVr?=
 =?iso-8859-1?Q?xTF4Fhdfkf834IxnbOWPGpkyiUUcBPf2VSrtgKxQkGPggukxAUqR/bc+ak?=
 =?iso-8859-1?Q?X/0TTcrxyk+Lfl7Z5273/reuYau7b1m7yv5C4Z+xXlvf3m48YC2n+6OdCA?=
 =?iso-8859-1?Q?SlyOxmgPNu1mru3LmdKKDU708UaLElHJSqkrz7G2PKNjBoyYN8pAWJbnoO?=
 =?iso-8859-1?Q?6SLFSfMRsFIZP/uIUPM4OEkXkvHkF6mFZw1h9+oLzBJL4WEASQuZtGkIKx?=
 =?iso-8859-1?Q?Q78QvbVuLgqJd2FsiF4fwRKc8oXgnV7LDg9QwoNJK9juQWBBaCnZW3I/qH?=
 =?iso-8859-1?Q?w7F2M3fsLZ2fS9sO6m5NVxnayrHVkayoMtir1l3xA+RV8r0hXDRInW6v4C?=
 =?iso-8859-1?Q?vKIByHgfTvexSjYm+J6DFiX/biL3x3pvHVRQGYyU5Kq2Zr6dRHpgXpKhhY?=
 =?iso-8859-1?Q?hn9JvxEiVt/HYmgLFNAq0FE8Bizxrn5TeyTtpXf/P+EwMq1V4J3wwBPcNb?=
 =?iso-8859-1?Q?rNDmE9fse7WGCtnzaZ5yBVgpaXs58T30uh7kn1LP/KF3m74eqcEjA16qq2?=
 =?iso-8859-1?Q?26oj3KjRbNlBO9+978KgVNjixk+MIIScxpJc/FS51jDFkeHnyrcFGrdZ9F?=
 =?iso-8859-1?Q?4GHO9csb4K/yInnV7t82ROfH9noWREMfT6xuP81bs0tA2sd1g54U+HL2Of?=
 =?iso-8859-1?Q?WsAxJO6rNqW3eZ8eR7441Dg078BuWyU3slb8CuxZwyOD5MhGQe29uqPXWo?=
 =?iso-8859-1?Q?yZ0YQ5LMgaFs9+Dji4layDIV8TCT51Bwyj+vU0Rif4Uicw76TBlGmjQYsM?=
 =?iso-8859-1?Q?qOp7vKju+HAEtaWhasrrSd6lb0UkoOQsPVrgrPNx1NfCzKfmW19dMfW8sb?=
 =?iso-8859-1?Q?sOIHKgREwSm6GFWN5e2dxKGXwRbdTFOhsyHVEBD31mbYSC+paoKzjrYiIV?=
 =?iso-8859-1?Q?PQI9WRhRTooQ39OSGrL6ka1OrJtwdANPJ3K/FCZu/intvcXfxpiPc0GVt6?=
 =?iso-8859-1?Q?dK3g458OYIT2L+33uIPYfM1DLvU05xT4w7p2/8O5+xQDpfhBGwVwPgpccE?=
 =?iso-8859-1?Q?9yyMJLOUQWM4i1WNNeH1QsaU6dwQcJr41k5T+6FZHab0WBiwCn4NFhOxSh?=
 =?iso-8859-1?Q?u4wVygyx4ZCHNO56AU08mmXsesQ35iaEirIphoPvKaoXPGXjWKsJk62soV?=
 =?iso-8859-1?Q?1kBs14rgta7L4XzI1/CNGPJrT0wBFBXw0URlB1DDA=3D?=
Content-Type: multipart/alternative;
	boundary="_000_DB9P192MB143408981DB724A477C0ADA6A6462DB9P192MB1434EURP_"
MIME-Version: 1.0
X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-fb43a.templateTenant
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB9P192MB1434.EURP192.PROD.OUTLOOK.COM
X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a7e45af-fe60-4161-3fba-08dc272f2576
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Feb 2024 16:17:42.4753
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0P192MB1951

--_000_DB9P192MB143408981DB724A477C0ADA6A6462DB9P192MB1434EURP_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hello Andrew,

Thank you for the quick response. I have no local modifications to Xen, pre=
cisely in order to test it as cleanly as possible.
As soon as possible, I'll try to use a more recent version. Regarding the s=
tack trace, I believe the series of function calls starts from:

vmx_asm_vmexit_handler -> vmx_vmexit_handler -> domain_crash -> __domain_cr=
ash

Thank you for addressing the concerns about security, next time I'll follow=
 your advice.

Best regards,
Giuseppe De Rosa.
________________________________
Da: Andrew Cooper <andrew.cooper3@citrix.com>
Inviato: marted=EC 6 febbraio 2024 15:11
A: Giuseppe De Rosa <giuseppe.de.rosa@outlook.it>; xen-devel@lists.xen.org =
<xen-devel@lists.xen.org>
Oggetto: Re: [BUG] Possible FATAL PAGE FAULT in domain_crash function on Xe=
n 4.14.6

On 06/02/2024 12:14 pm, Giuseppe De Rosa wrote:
> Bug detailed description:
>
> ----------------
>
> While booting a Linux Debian 7 "Wheezy" VM, Xen crashes with a FATAL
> PAGE FAULT.
>
>
>
> Environment :
>
> ----------------
>
> HW: Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz (2 CPU, Xen in nested
> virtualization upon QEMU/KVM), 4GB RAM
>
> Xen: Xen 4.14.6 (xen-hypervisor-4.14-amd64 package)
>
> Dom0: Linux 5.10.0-26-amd64 (Debian 11 "Bullseye"), 1 CPU, 1024GB RAM
>
>
>
> Reproduce steps:
>
> ----------------
>
> 1.      Install debian 11 and xen package
>
> 2.      Boot a clean debian 7 image in hardware-assisted virtualization
>
> 3.      Single bit-flip bit number 2 of VMCS field
> "VM_EXIT_QUALIFICATION" (field number 6400). In my case, value changed
> from [100049] -> [10004d]
>
> 4.      Leave the debian7 guest executing after the bit flip.
>
>
>
> Current result:
>
> ----------------
>
> Xen crash
>
>
>
> Console error log:
>
> ----------------
>
> (XEN) d1v0 Unexpected PIO status 1, port 0x10 read 0x00000000ffff
>
> (XEN) domain_crash called from io.c:166
> (XEN) Domain 1 (vcpu#0) crashed on cpu#1:
> (XEN) ----[ Xen-4.14.6  x86_64  debug=3Dn   Not tainted ]----
> (XEN) CPU:    1
> (XEN) RIP:    0010:[<ffffffff8100712e>]
> (XEN) RFLAGS: 0000000000000046   CONTEXT: hvm guest (d1v0)
> (XEN) rax: 0000000000000000   rbx: 0000000040000000   rcx: 00000000000000=
01
> (XEN) rdx: 0000000000000000   rsi: ffffffff81666a80   rdi: ffffffff816178=
30
> (XEN) rbp: 0000000000000020   rsp: ffffffff81601e78   r8:  00000000000002=
00
> (XEN) r9:  ffffffff8168f2a0   r10: 0000000000000007   r11: 00000000000000=
07
> (XEN) r12: ffffffff81601f58   r13: ffffffffffffffff   r14: 000000000008c8=
00
> (XEN) r15: 0000000000001000   cr0: 0000000080050033   cr4: 00000000001000=
a0
> (XEN) cr3: 0000000001605000   cr2: 0000000000000000
> (XEN) fsb: 0000000000000000   gsb: ffffffff81696000   gss: 00000000000000=
00
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: 0010
> (XEN) d1v0 Bad rIP ffffffff8100712e for mode 0
> (XEN) domain_crash called from vmx.c:4413
> (XEN) ----[ Xen-4.14.6  x86_64  debug=3Dn   Not tainted ]----
> (XEN) CPU:    1
> (XEN) RIP:    e008:[<ffff82d040206fa9>] __domain_crash+0x9/0x80
> (XEN) RFLAGS: 0000000000010296   CONTEXT: hypervisor (d1v0)
> (XEN) rax: ffff830139c0506c   rbx: ffff8301308a0000   rcx: 00000000000000=
00
> (XEN) rdx: ffff830136ddffff   rsi: 000000000000000a   rdi: 00000000000000=
00
> (XEN) rbp: 0000000000000000   rsp: ffff830136ddfee0   r8:  00000000000000=
01
> (XEN) r9:  0000000000004000   r10: 0000000000000001   r11: ffff82d040372d=
40
> (XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 00000000000000=
00
> (XEN) r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000001526=
e0
> (XEN) cr3: 0000000136da6000   cr2: 0000000000000208
> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 00000000000000=
00
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> (XEN) Xen code around <ffff82d040206fa9> (__domain_crash+0x9/0x80):
> (XEN)  0f 1e fa 55 48 89 fd 53 <80> bf 08 02 00 00 00 75 2d 48 89 e3 0f
> b7 37 48
> (XEN) Xen stack trace from rsp=3Dffff830136ddfee0:
> (XEN)    ffff8301308a0000 0000000000000000 ffff82d0402a1798 0000000000001=
000
> (XEN)    000000000008c800 ffffffffffffffff ffffffff81601f58 0000000000000=
020
> (XEN)    0000000040000000 0000000000000007 0000000000000007 ffffffff8168f=
2a0
> (XEN)    0000000000000200 0000000000000000 0000000000000001 0000000000000=
000
> (XEN)    ffffffff81666a80 ffffffff81617830 000000fa00000000 ffffffff81007=
12e
> (XEN)    0000000000000000 0000000000000046 ffffffff81601e78 0000000000000=
000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000=
000
> (XEN)    0000e01000000001 ffff8301308a0000 00000030f9686000 0000000000152=
6e0
> (XEN)    0000000000000000 0000000000000000 0006020200000000 0000000000000=
000
> (XEN) Xen call trace:
> (XEN)    [<ffff82d040206fa9>] R __domain_crash+0x9/0x80
> (XEN)    [<ffff82d0402a1798>] S vmx_asm_vmexit_handler+0xf8/0x210
> (XEN)
> (XEN) Pagetable walk from 0000000000000208:
> (XEN)  L4[0x000] =3D 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 1:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=3D0000]
> (XEN) Faulting linear address: 0000000000000208
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
>
>
>
> HVM guest config file:
>
> --------------------------------
>
> name =3D "debian7"
>
> builder=3D "hvm"
> memory =3D 1024
> vcpus =3D 1
> cpus=3D ["1"]
> disk=3D["qcow2:/home/test/debian7.qcow2,xvda,w"]
> vnc =3D 1
> vnclisten =3D '127.0.0.1'
> vncdisplay =3D 1
> boot =3D "d"
> serial =3D "pty"
>
> Discussion
> --------------------------------
>
> Hello, i am conducting robustness tests on Xen 4.14. I bit flipped the
> field VM_EXIT_QUALIFICATION during the exit handling of an
> IO_INSTRUCTION. After a VM Read on that field, Xen crashes with this
> error log. This is my (possible) explanation:
>
> Xen correctly detects an error in the field, resulting in a domain crash
> (Unexpected PIO status 1) called from this point in the code:
>
> xen/arch/x86/hvm/io.c
> ```
> gprintk(XENLOG_ERR, "Unexpected PIO status %d, port %#x %s 0x%0*x\n",
>                 rc, port, dir =3D=3D IOREQ_WRITE ? "write" : "read",
>                 size * 2, data & ((1u << (size * 8)) - 1));
> domain_crash(curr->domain);
> return false;
> ```
>
> In the `handle_pio` function, in the handling of the `IO_INSTRUCTION`
> exit reason. However, the code continues and ends here for an issue in
> the processor mode:
>
> /xen/arch/x86/hvm/vmx/vmx.c
> ```
> mode =3D vmx_guest_x86_mode(v);
>     if ( mode =3D=3D 8 ? !is_canonical_address(regs->rip)
>                    : regs->rip !=3D regs->eip )
>     {
>         gprintk(XENLOG_WARNING, "Bad rIP %lx for mode %u\n", regs->rip,
> mode);
>
>         if ( vmx_get_cpl() )
>         {
>             __vmread(VM_ENTRY_INTR_INFO, &intr_info);
>             if ( !(intr_info & INTR_INFO_VALID_MASK) )
>                 hvm_inject_hw_exception(TRAP_gp_fault, 0);
>             /* Need to fix rIP nevertheless. */
>             if ( mode =3D=3D 8 )
>                 regs->rip =3D (long)(regs->rip << (64 - VADDR_BITS)) >>
>                             (64 - VADDR_BITS);
>             else
>                 regs->rip =3D regs->eip;
>         }
>         else
>             domain_crash(v->domain);
>     }
> ```
>
> However, the domain pointer has already been deallocated due to the
> previous domain crash, resulting in a page fault that leads to Xen crashi=
ng.
>
> I would like to report this crash and ask for your opinion. Based on
> previous research, I do not believe it has been reported before. I am
> unsure if it could potentially be a security issue, hence I am posting
> it here. Let me know if I should provide other results. Thank you for
> your kind response in advance.


Answering somewhat out of order.

You've posted this publicly so the cat is out of the bag regardless.  If
you have concerns about security, please email security@xen.org as your
first point of contact.

That said, nested virt is not security supported.  It's still an
experimental feature, so bugs like this are fine to come straight to the
public mailing list.

The "Bad rIP" logic is buggy and has since been deleted.  Xen 4.14 is a
very old version of Xen, and is outside of general bugfix support.

It is quite likely that this bug still exists, but please use an
up-to-date version of Xen.  Fixes need developing against master and are
unlikely to be backported in this case, given its experimental status.

Also, you should be using a debug build of Xen generally for work like
this.  (I have no idea if it would alter your observations.)


I'm not sure what I think about bitflipping the exit qualification.  In
other places, that will definitely cause more severe crashes, and Xen
won't be getting in the game of auditing the VMX implementation against
the VMX spec.


For this crash you've got, there should be no way for the domain pointer
to have been freed in the sequence you've described.  While the vCPU is
still scheduled, the structures will remain.

Looking at the second backtrace, It looks suspiciously like a NULL
pointer was passed into __domain_crash() from vmx_asm_vmexit_handler()
but there's no such call call, nor a tailcall out of the handler, even
taking a peek at the 4.14 code.

If you have local changes, I'd look at those first.

~Andrew

--_000_DB9P192MB143408981DB724A477C0ADA6A6462DB9P192MB1434EURP_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
1">
<style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo=
ttom:0;} </style>
</head>
<body dir=3D"ltr">
<div class=3D"elementToProof"><span style=3D"font-family: Calibri, Helvetic=
a, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Hello Andrew,</span><=
/div>
<div><span style=3D"font-family: Calibri, Helvetica, sans-serif; font-size:=
 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div><span style=3D"font-family: Calibri, Helvetica, sans-serif; font-size:=
 12pt; color: rgb(0, 0, 0);">Thank you for the quick response. I have no lo=
cal modifications to Xen, precisely in order to test it as cleanly as possi=
ble.</span></div>
<div><span style=3D"font-family: Calibri, Helvetica, sans-serif; font-size:=
 12pt; color: rgb(0, 0, 0);">As soon as possible, I'll try to use a more re=
cent version. Regarding the stack trace, I believe the series of function c=
alls starts from:</span></div>
<div><span style=3D"font-family: Calibri, Helvetica, sans-serif; font-size:=
 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div><span style=3D"font-family: Calibri, Helvetica, sans-serif; font-size:=
 12pt; color: rgb(0, 0, 0);">vmx_asm_vmexit_handler -&gt; vmx_vmexit_handle=
r -&gt; domain_crash -&gt; __domain_crash</span></div>
<div><span style=3D"font-family: Calibri, Helvetica, sans-serif; font-size:=
 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div><span style=3D"font-family: Calibri, Helvetica, sans-serif; font-size:=
 12pt; color: rgb(0, 0, 0);">Thank you for addressing the concerns about se=
curity, next time I'll follow your advice.</span></div>
<div><span style=3D"font-family: Calibri, Helvetica, sans-serif; font-size:=
 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div><span style=3D"font-family: Calibri, Helvetica, sans-serif; font-size:=
 12pt; color: rgb(0, 0, 0);">Best regards,</span></div>
<div><span style=3D"font-family: Calibri, Helvetica, sans-serif; font-size:=
 12pt; color: rgb(0, 0, 0);">Giuseppe De Rosa.</span></div>
<div id=3D"appendonsend"></div>
<hr style=3D"display:inline-block;width:98%" tabindex=3D"-1">
<div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st=
yle=3D"font-size:11pt" color=3D"#000000"><b>Da:</b> Andrew Cooper &lt;andre=
w.cooper3@citrix.com&gt;<br>
<b>Inviato:</b> marted=EC 6 febbraio 2024 15:11<br>
<b>A:</b> Giuseppe De Rosa &lt;giuseppe.de.rosa@outlook.it&gt;; xen-devel@l=
ists.xen.org &lt;xen-devel@lists.xen.org&gt;<br>
<b>Oggetto:</b> Re: [BUG] Possible FATAL PAGE FAULT in domain_crash functio=
n on Xen 4.14.6</font>
<div>&nbsp;</div>
</div>
<div class=3D"BodyFragment"><font size=3D"2"><span style=3D"font-size:11pt;=
">
<div class=3D"PlainText">On 06/02/2024 12:14 pm, Giuseppe De Rosa wrote:<br=
>
&gt; Bug detailed description:<br>
&gt; <br>
&gt; ----------------<br>
&gt; <br>
&gt; While booting a Linux Debian 7 &quot;Wheezy&quot; VM, Xen crashes with=
 a FATAL<br>
&gt; PAGE FAULT.&nbsp;<br>
&gt; <br>
&gt; &nbsp;<br>
&gt; <br>
&gt; Environment :<br>
&gt; <br>
&gt; ----------------<br>
&gt; <br>
&gt; HW: Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz (2 CPU, Xen in nested<br=
>
&gt; virtualization upon QEMU/KVM), 4GB RAM<br>
&gt; <br>
&gt; Xen: Xen 4.14.6 (xen-hypervisor-4.14-amd64 package)<br>
&gt; <br>
&gt; Dom0: Linux 5.10.0-26-amd64 (Debian 11 &quot;Bullseye&quot;), 1 CPU, 1=
024GB RAM<br>
&gt; <br>
&gt; &nbsp;<br>
&gt; <br>
&gt; Reproduce steps:<br>
&gt; <br>
&gt; ----------------<br>
&gt; <br>
&gt; 1.&nbsp; &nbsp; &nbsp; Install debian 11 and xen package<br>
&gt; <br>
&gt; 2.&nbsp; &nbsp; &nbsp; Boot a clean debian 7 image in hardware-assiste=
d virtualization<br>
&gt; <br>
&gt; 3.&nbsp; &nbsp; &nbsp; Single bit-flip bit number 2 of VMCS field<br>
&gt; &quot;VM_EXIT_QUALIFICATION&quot; (field number 6400). In my case, val=
ue changed<br>
&gt; from [100049] -&gt; [10004d]<br>
&gt; <br>
&gt; 4.&nbsp; &nbsp; &nbsp; Leave the debian7 guest executing after the bit=
 flip.<br>
&gt; <br>
&gt; &nbsp;<br>
&gt; <br>
&gt; Current result:<br>
&gt; <br>
&gt; ----------------<br>
&gt; <br>
&gt; Xen crash<br>
&gt; <br>
&gt; &nbsp;<br>
&gt; <br>
&gt; Console error log:<br>
&gt; <br>
&gt; ----------------<br>
&gt; <br>
&gt; (XEN) d1v0 Unexpected PIO status 1, port 0x10 read 0x00000000ffff<br>
&gt; <br>
&gt; (XEN) domain_crash called from io.c:166<br>
&gt; (XEN) Domain 1 (vcpu#0) crashed on cpu#1:<br>
&gt; (XEN) ----[ Xen-4.14.6 &nbsp;x86_64 &nbsp;debug=3Dn &nbsp; Not tainted=
 ]----<br>
&gt; (XEN) CPU: &nbsp; &nbsp;1<br>
&gt; (XEN) RIP: &nbsp; &nbsp;0010:[&lt;ffffffff8100712e&gt;]<br>
&gt; (XEN) RFLAGS: 0000000000000046 &nbsp; CONTEXT: hvm guest (d1v0)<br>
&gt; (XEN) rax: 0000000000000000 &nbsp; rbx: 0000000040000000 &nbsp; rcx: 0=
000000000000001<br>
&gt; (XEN) rdx: 0000000000000000 &nbsp; rsi: ffffffff81666a80 &nbsp; rdi: f=
fffffff81617830<br>
&gt; (XEN) rbp: 0000000000000020 &nbsp; rsp: ffffffff81601e78 &nbsp; r8: &n=
bsp;0000000000000200<br>
&gt; (XEN) r9: &nbsp;ffffffff8168f2a0 &nbsp; r10: 0000000000000007 &nbsp; r=
11: 0000000000000007<br>
&gt; (XEN) r12: ffffffff81601f58 &nbsp; r13: ffffffffffffffff &nbsp; r14: 0=
00000000008c800<br>
&gt; (XEN) r15: 0000000000001000 &nbsp; cr0: 0000000080050033 &nbsp; cr4: 0=
0000000001000a0<br>
&gt; (XEN) cr3: 0000000001605000 &nbsp; cr2: 0000000000000000<br>
&gt; (XEN) fsb: 0000000000000000 &nbsp; gsb: ffffffff81696000 &nbsp; gss: 0=
000000000000000<br>
&gt; (XEN) ds: 0000 &nbsp; es: 0000 &nbsp; fs: 0000 &nbsp; gs: 0000 &nbsp; =
ss: 0000 &nbsp; cs: 0010<br>
&gt; (XEN) d1v0 Bad rIP ffffffff8100712e for mode 0<br>
&gt; (XEN) domain_crash called from vmx.c:4413<br>
&gt; (XEN) ----[ Xen-4.14.6 &nbsp;x86_64 &nbsp;debug=3Dn &nbsp; Not tainted=
 ]----<br>
&gt; (XEN) CPU: &nbsp; &nbsp;1<br>
&gt; (XEN) RIP: &nbsp; &nbsp;e008:[&lt;ffff82d040206fa9&gt;] __domain_crash=
+0x9/0x80<br>
&gt; (XEN) RFLAGS: 0000000000010296 &nbsp; CONTEXT: hypervisor (d1v0)<br>
&gt; (XEN) rax: ffff830139c0506c &nbsp; rbx: ffff8301308a0000 &nbsp; rcx: 0=
000000000000000<br>
&gt; (XEN) rdx: ffff830136ddffff &nbsp; rsi: 000000000000000a &nbsp; rdi: 0=
000000000000000<br>
&gt; (XEN) rbp: 0000000000000000 &nbsp; rsp: ffff830136ddfee0 &nbsp; r8: &n=
bsp;0000000000000001<br>
&gt; (XEN) r9: &nbsp;0000000000004000 &nbsp; r10: 0000000000000001 &nbsp; r=
11: ffff82d040372d40<br>
&gt; (XEN) r12: 0000000000000000 &nbsp; r13: 0000000000000000 &nbsp; r14: 0=
000000000000000<br>
&gt; (XEN) r15: 0000000000000000 &nbsp; cr0: 0000000080050033 &nbsp; cr4: 0=
0000000001526e0<br>
&gt; (XEN) cr3: 0000000136da6000 &nbsp; cr2: 0000000000000208<br>
&gt; (XEN) fsb: 0000000000000000 &nbsp; gsb: 0000000000000000 &nbsp; gss: 0=
000000000000000<br>
&gt; (XEN) ds: 0000 &nbsp; es: 0000 &nbsp; fs: 0000 &nbsp; gs: 0000 &nbsp; =
ss: 0000 &nbsp; cs: e008<br>
&gt; (XEN) Xen code around &lt;ffff82d040206fa9&gt; (__domain_crash+0x9/0x8=
0):<br>
&gt; (XEN) &nbsp;0f 1e fa 55 48 89 fd 53 &lt;80&gt; bf 08 02 00 00 00 75 2d=
 48 89 e3 0f<br>
&gt; b7 37 48<br>
&gt; (XEN) Xen stack trace from rsp=3Dffff830136ddfee0:<br>
&gt; (XEN) &nbsp; &nbsp;ffff8301308a0000 0000000000000000 ffff82d0402a1798 =
0000000000001000<br>
&gt; (XEN) &nbsp; &nbsp;000000000008c800 ffffffffffffffff ffffffff81601f58 =
0000000000000020<br>
&gt; (XEN) &nbsp; &nbsp;0000000040000000 0000000000000007 0000000000000007 =
ffffffff8168f2a0<br>
&gt; (XEN) &nbsp; &nbsp;0000000000000200 0000000000000000 0000000000000001 =
0000000000000000<br>
&gt; (XEN) &nbsp; &nbsp;ffffffff81666a80 ffffffff81617830 000000fa00000000 =
ffffffff8100712e<br>
&gt; (XEN) &nbsp; &nbsp;0000000000000000 0000000000000046 ffffffff81601e78 =
0000000000000000<br>
&gt; (XEN) &nbsp; &nbsp;0000000000000000 0000000000000000 0000000000000000 =
0000000000000000<br>
&gt; (XEN) &nbsp; &nbsp;0000e01000000001 ffff8301308a0000 00000030f9686000 =
00000000001526e0<br>
&gt; (XEN) &nbsp; &nbsp;0000000000000000 0000000000000000 0006020200000000 =
0000000000000000<br>
&gt; (XEN) Xen call trace:<br>
&gt; (XEN) &nbsp; &nbsp;[&lt;ffff82d040206fa9&gt;] R __domain_crash+0x9/0x8=
0<br>
&gt; (XEN) &nbsp; &nbsp;[&lt;ffff82d0402a1798&gt;] S vmx_asm_vmexit_handler=
+0xf8/0x210<br>
&gt; (XEN)<br>
&gt; (XEN) Pagetable walk from 0000000000000208:<br>
&gt; (XEN) &nbsp;L4[0x000] =3D 0000000000000000 ffffffffffffffff<br>
&gt; (XEN)<br>
&gt; (XEN) ****************************************<br>
&gt; (XEN) Panic on CPU 1:<br>
&gt; (XEN) FATAL PAGE FAULT<br>
&gt; (XEN) [error_code=3D0000]<br>
&gt; (XEN) Faulting linear address: 0000000000000208<br>
&gt; (XEN) ****************************************<br>
&gt; (XEN)<br>
&gt; (XEN) Reboot in five seconds...<br>
&gt; <br>
&gt; &nbsp;<br>
&gt; <br>
&gt; HVM guest config file:<br>
&gt; <br>
&gt; --------------------------------<br>
&gt; <br>
&gt; name =3D &quot;debian7&quot;<br>
&gt; <br>
&gt; builder=3D &quot;hvm&quot;<br>
&gt; memory =3D 1024<br>
&gt; vcpus =3D 1<br>
&gt; cpus=3D [&quot;1&quot;]<br>
&gt; disk=3D[&quot;qcow2:/home/test/debian7.qcow2,xvda,w&quot;]<br>
&gt; vnc =3D 1<br>
&gt; vnclisten =3D '127.0.0.1'<br>
&gt; vncdisplay =3D 1<br>
&gt; boot =3D &quot;d&quot;<br>
&gt; serial =3D &quot;pty&quot;<br>
&gt; <br>
&gt; Discussion<br>
&gt; --------------------------------<br>
&gt; <br>
&gt; Hello, i am conducting robustness tests on Xen 4.14. I bit flipped the=
<br>
&gt; field VM_EXIT_QUALIFICATION during the exit handling of an<br>
&gt; IO_INSTRUCTION. After a VM Read on that field, Xen crashes with this<b=
r>
&gt; error log. This is my (possible) explanation:<br>
&gt; <br>
&gt; Xen correctly detects an error in the field, resulting in a domain cra=
sh<br>
&gt; (Unexpected PIO status 1) called from this point in the code:<br>
&gt; <br>
&gt; xen/arch/x86/hvm/io.c<br>
&gt; ```<br>
&gt; gprintk(XENLOG_ERR, &quot;Unexpected PIO status %d, port %#x %s 0x%0*x=
\n&quot;,<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; rc, port, dir =
=3D=3D IOREQ_WRITE ? &quot;write&quot; : &quot;read&quot;,<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; size * 2, data=
 &amp; ((1u &lt;&lt; (size * 8)) - 1));<br>
&gt; domain_crash(curr-&gt;domain);<br>
&gt; return false;<br>
&gt; ```<br>
&gt; <br>
&gt; In the `handle_pio` function, in the handling of the `IO_INSTRUCTION`<=
br>
&gt; exit reason. However, the code continues and ends here for an issue in=
<br>
&gt; the processor mode:<br>
&gt; <br>
&gt; /xen/arch/x86/hvm/vmx/vmx.c<br>
&gt; ```<br>
&gt; mode =3D vmx_guest_x86_mode(v);<br>
&gt; &nbsp; &nbsp; if ( mode =3D=3D 8 ? !is_canonical_address(regs-&gt;rip)=
<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:=
 regs-&gt;rip !=3D regs-&gt;eip )<br>
&gt; &nbsp; &nbsp; {<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; gprintk(XENLOG_WARNING, &quot;Bad rIP %lx =
for mode %u\n&quot;, regs-&gt;rip,<br>
&gt; mode);<br>
&gt; <br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; if ( vmx_get_cpl() )<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; {<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; __vmread(VM_ENTRY_INTR_INFO,=
 &amp;intr_info);<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if ( !(intr_info &amp; INTR_=
INFO_VALID_MASK) )<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; hvm_inject_hw_=
exception(TRAP_gp_fault, 0);<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; /* Need to fix rIP neverthel=
ess. */<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if ( mode =3D=3D 8 )<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; regs-&gt;rip =
=3D (long)(regs-&gt;rip &lt;&lt; (64 - VADDR_BITS)) &gt;&gt;<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; (64 - VADDR_BITS);<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; else<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; regs-&gt;rip =
=3D regs-&gt;eip;<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; }<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; else<br>
&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; domain_crash(v-&gt;domain);<=
br>
&gt; &nbsp; &nbsp; }<br>
&gt; ```<br>
&gt; <br>
&gt; However, the domain pointer has already been deallocated due to the<br=
>
&gt; previous domain crash, resulting in a page fault that leads to Xen cra=
shing.<br>
&gt; <br>
&gt; I would like to report this crash and ask for your opinion. Based on<b=
r>
&gt; previous research, I do not believe it has been reported before. I am<=
br>
&gt; unsure if it could potentially be a security issue, hence I am posting=
<br>
&gt; it here. Let me know if I should provide other results. Thank you for<=
br>
&gt; your kind response in advance.<br>
<br>
<br>
Answering somewhat out of order.<br>
<br>
You've posted this publicly so the cat is out of the bag regardless.&nbsp; =
If<br>
you have concerns about security, please email security@xen.org as your<br>
first point of contact.<br>
<br>
That said, nested virt is not security supported.&nbsp; It's still an<br>
experimental feature, so bugs like this are fine to come straight to the<br=
>
public mailing list.<br>
<br>
The &quot;Bad rIP&quot; logic is buggy and has since been deleted.&nbsp; Xe=
n 4.14 is a<br>
very old version of Xen, and is outside of general bugfix support.<br>
<br>
It is quite likely that this bug still exists, but please use an<br>
up-to-date version of Xen.&nbsp; Fixes need developing against master and a=
re<br>
unlikely to be backported in this case, given its experimental status.<br>
<br>
Also, you should be using a debug build of Xen generally for work like<br>
this.&nbsp; (I have no idea if it would alter your observations.)<br>
<br>
<br>
I'm not sure what I think about bitflipping the exit qualification.&nbsp; I=
n<br>
other places, that will definitely cause more severe crashes, and Xen<br>
won't be getting in the game of auditing the VMX implementation against<br>
the VMX spec.<br>
<br>
<br>
For this crash you've got, there should be no way for the domain pointer<br=
>
to have been freed in the sequence you've described.&nbsp; While the vCPU i=
s<br>
still scheduled, the structures will remain.<br>
<br>
Looking at the second backtrace, It looks suspiciously like a NULL<br>
pointer was passed into __domain_crash() from vmx_asm_vmexit_handler()<br>
but there's no such call call, nor a tailcall out of the handler, even<br>
taking a peek at the 4.14 code.<br>
<br>
If you have local changes, I'd look at those first.<br>
<br>
~Andrew<br>
</div>
</span></font></div>
</body>
</html>

--_000_DB9P192MB143408981DB724A477C0ADA6A6462DB9P192MB1434EURP_--


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 17:03:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 17:03:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677045.1053436 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXOr8-00053f-Qv; Tue, 06 Feb 2024 17:03:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677045.1053436; Tue, 06 Feb 2024 17:03:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXOr8-00053Y-OQ; Tue, 06 Feb 2024 17:03:34 +0000
Received: by outflank-mailman (input) for mailman id 677045;
 Tue, 06 Feb 2024 17:03:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=m3u1=JP=chromium.org=keescook@srs-se1.protection.inumbo.net>)
 id 1rXOr7-00053O-8S
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 17:03:33 +0000
Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com
 [2607:f8b0:4864:20::42c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a6c05e2e-c511-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 18:03:30 +0100 (CET)
Received: by mail-pf1-x42c.google.com with SMTP id
 d2e1a72fcca58-6de3141f041so4113855b3a.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 09:03:30 -0800 (PST)
Received: from www.outflux.net ([198.0.35.241])
 by smtp.gmail.com with ESMTPSA id
 h4-20020aa786c4000000b006dbda1b19f7sm2168838pfo.159.2024.02.06.09.03.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 06 Feb 2024 09:03:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a6c05e2e-c511-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=chromium.org; s=google; t=1707239009; x=1707843809; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=umeDhyeqcFKdvC6xxd3vUkMjNRJ2Irf8wU+CDgjfryk=;
        b=PCGX82r/Mw+WkO9TYmyLG8mmozOBgfB3YIXndMVZz10J0woGLP93/fqtNx+WTC5qCh
         U40+AXm7oHphHD8XypW4VeviAHLngItOcaTV7UwtBLFZPcFojG81VoqWr5ClMvu3rN3+
         d0OzzczcbXYVYAanGukr8J0CriMVpPrUY70PM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707239009; x=1707843809;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=umeDhyeqcFKdvC6xxd3vUkMjNRJ2Irf8wU+CDgjfryk=;
        b=Y7ziGN6ZaceSI5da3/RzJc5yLaOL2qL+Z2WCWxeRrOoiY7iEHTmMxE+XoMfgIFV5pN
         Ft1H3ROuY5SzgUWtUrCr6vKDpwYFyMM6z9OjXj+GW6yy0tTmYvizTxTCTXd1Oou6hqBC
         7GPkcw/S568rG/TJThX2Xuc/ddt0uiP74TuvuGnwZHGWQxYC6oRiLTsstOsgaF/zIlus
         xE4HMcqLkDoLLKiIDeYJAWOUXvsbgjoGn8SV46ACzc9C0NCf9cCYgRRLnv2G98w1ell+
         i+5Uf5ZEiMHg0diHoCAqpg9MdMifaB88/TXrnIBx31nabaO8IiOWLkdstx+txZAJN5l8
         Q8KA==
X-Gm-Message-State: AOJu0Ywh8oZDFYKgkugP7jQYnY3IuIvlU637COA4TW+PAyrBkrjfPew0
	nZuep6jLpITnMnGF4uCe0zNtmjPRZji8XEJj78JLHCqR0ywDFX9Qe0pIphuruQ==
X-Google-Smtp-Source: AGHT+IGvoigLJEbHKpYBrrZJJVxKU9n0m5DvgYvJ8ZsIdLNHqnrzt3qcY0VxotJUZVvRPos9blAcsQ==
X-Received: by 2002:a05:6a00:842:b0:6e0:28eb:275e with SMTP id q2-20020a056a00084200b006e028eb275emr102249pfk.22.1707239007816;
        Tue, 06 Feb 2024 09:03:27 -0800 (PST)
X-Forwarded-Encrypted: i=0; AJvYcCWfB/tr7pyFK2HPXESsJ1h9TpKXYrK9eOqf81YqGqlYqEBLTPjHVqjnFSch/E1RwQgtmUg6t/v1lbDkytUEOHoI+aPHG9TJb8Fjr1LUujkShO2c0Xt8LdUmwvicI/JBXQ2643PJqQGtsn/SOX8WNNhexsAZBeu/5DERV85KCPtfxGgG65Nj+Q0Q0nNft5VzbKWpP6fOg0dV1CRjqPX+zVrjeWj5NsaSfE6hrXbQav7R/+4KuONSyqslgF6VURAuJ8bjkvxUDAH/4PFKePXGPHpiSI2lGiW0evYlY2s=
From: Kees Cook <keescook@chromium.org>
To: Juergen Gross <jgross@suse.com>
Cc: Kees Cook <keescook@chromium.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	"Gustavo A . R . Silva" <gustavoars@kernel.org>,
	xen-devel@lists.xenproject.org,
	linux-kernel@vger.kernel.org,
	linux-hardening@vger.kernel.org
Subject: [PATCH] xen/gntalloc: Replace UAPI 1-element array
Date: Tue,  6 Feb 2024 09:03:24 -0800
Message-Id: <20240206170320.work.437-kees@kernel.org>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
X-Developer-Signature: v=1; a=openpgp-sha256; l=1659; i=keescook@chromium.org;
 h=from:subject:message-id; bh=U8HuNwUdlEEPLHdv3QRnH+uJMA60aHmIoxCx5CgE2PE=;
 b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBlwmZc1Jfxj5fa/YwMvoyvAPkFWN+gRJLJEPn7F
 n308vxItD6JAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCZcJmXAAKCRCJcvTf3G3A
 Jm0GD/oDXr7gvAa43q/4w9dV1f3MZINBzTm2QNk9mHvC7Lp7XSOmcgrCG2Fi8o4c5s5TyWIybl8
 slT25ToB1KNPSc6IPGYoFcQTXwRn39TzQq9T6n+5z5alG6Bz6ZcAk34BQcIm6vfpD3/N11fHF5F
 pdXab4HtiJd2JS9uhggiaHQbN59FGqFQjmpFiouIQ/v5r3awL2QgSC+DznPmZAbee6vD1Rz7G3F
 Ko9yltkr64NVBYCrLDQEylWIDjzGcaMn7kb9npaAYzOCcDcDcQgyfA41idDc+YZlQhxJhuTBe+h
 kHCag8yYsbha7fMXGE+7jHPzjKzE92858eEHiqDJUS+n7u5FMMzarLFrfzeX2LRKJT3oKVKLNhA
 pZ8OYm0q7YJihVOtzaWRo2aNTZ1TpHpDtp7xQo0UFmVh8lcE8p41nIm3bKoC/PbT6ygufKRWEgT
 F1cqwtPM+91yYdVb5HqaHdJ1hIahqo84hQrS6uoeJb2fDi7VWnvopFiUKfVEjWhqUqDo4DCatRo
 9wGXoVt2ljvrVqDQoE56vPuRpeLfSDuLDp/koDL82+UGsSCzmRyXeFrADIlakadryKPFCIqd6xf
 eyoiFXJ0khVBIixLTkPKRY4vvEmldd97HWkoEuj0POSSL7w/8LzGaHlA9x8qqoGsFQwQ6Uvf4U+
 Vekxg2a ltTDiLAQ==
X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026
Content-Transfer-Encoding: 8bit

Without changing the structure size (since it is UAPI), add a proper
flexible array member, and reference it in the kernel so that it will
not be trip the array-bounds sanitizer[1].

Link: https://github.com/KSPP/linux/issues/113 [1]
Cc: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Cc: Gustavo A. R. Silva <gustavoars@kernel.org>
Cc: xen-devel@lists.xenproject.org
Signed-off-by: Kees Cook <keescook@chromium.org>
---
 drivers/xen/gntalloc.c      | 2 +-
 include/uapi/xen/gntalloc.h | 5 ++++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/gntalloc.c b/drivers/xen/gntalloc.c
index 26ffb8755ffb..f93f73ecefee 100644
--- a/drivers/xen/gntalloc.c
+++ b/drivers/xen/gntalloc.c
@@ -317,7 +317,7 @@ static long gntalloc_ioctl_alloc(struct gntalloc_file_private_data *priv,
 		rc = -EFAULT;
 		goto out_free;
 	}
-	if (copy_to_user(arg->gref_ids, gref_ids,
+	if (copy_to_user(arg->gref_ids_flex, gref_ids,
 			sizeof(gref_ids[0]) * op.count)) {
 		rc = -EFAULT;
 		goto out_free;
diff --git a/include/uapi/xen/gntalloc.h b/include/uapi/xen/gntalloc.h
index 48d2790ef928..3109282672f3 100644
--- a/include/uapi/xen/gntalloc.h
+++ b/include/uapi/xen/gntalloc.h
@@ -31,7 +31,10 @@ struct ioctl_gntalloc_alloc_gref {
 	__u64 index;
 	/* The grant references of the newly created grant, one per page */
 	/* Variable size, depending on count */
-	__u32 gref_ids[1];
+	union {
+		__u32 gref_ids[1];
+		__DECLARE_FLEX_ARRAY(__u32, gref_ids_flex);
+	};
 };
 
 #define GNTALLOC_FLAG_WRITABLE 1
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 17:06:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 17:06:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677049.1053447 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXOu9-0005bB-8A; Tue, 06 Feb 2024 17:06:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677049.1053447; Tue, 06 Feb 2024 17:06:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXOu9-0005b4-55; Tue, 06 Feb 2024 17:06:41 +0000
Received: by outflank-mailman (input) for mailman id 677049;
 Tue, 06 Feb 2024 17:06:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tN57=JP=embeddedor.com=gustavo@srs-se1.protection.inumbo.net>)
 id 1rXOu7-0005aw-Rq
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 17:06:40 +0000
Received: from omta040.useast.a.cloudfilter.net
 (omta040.useast.a.cloudfilter.net [44.202.169.39])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 16094190-c512-11ee-8a48-1f161083a0e0;
 Tue, 06 Feb 2024 18:06:37 +0100 (CET)
Received: from eig-obgw-6010a.ext.cloudfilter.net ([10.0.30.248])
 by cmsmtp with ESMTPS
 id X5gxr4gHzTHHuXOu4r1C0R; Tue, 06 Feb 2024 17:06:36 +0000
Received: from gator4166.hostgator.com ([108.167.133.22]) by cmsmtp with ESMTPS
 id XOu2ryq8ziDDpXOu3r5RmY; Tue, 06 Feb 2024 17:06:35 +0000
Received: from 187-162-21-192.static.axtel.net ([187.162.21.192]:34842
 helo=[192.168.15.10])
 by gator4166.hostgator.com with esmtpsa (TLS1.2) tls
 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96.2)
 (envelope-from <gustavo@embeddedor.com>) id 1rXOu2-003dF2-1M;
 Tue, 06 Feb 2024 11:06:34 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16094190-c512-11ee-8a48-1f161083a0e0
X-Authority-Analysis: v=2.4 cv=dIkFm/Zb c=1 sm=1 tr=0 ts=65c2671b
 a=1YbLdUo/zbTtOZ3uB5T3HA==:117 a=WzbPXH4gqzPVN0x6HrNMNA==:17
 a=IkcTkHD0fZMA:10 a=k7vzHIieQBIA:10 a=wYkD_t78qR0A:10 a=NEAV23lmAAAA:8
 a=iox4zFpeAAAA:8 a=VwQbUJbxAAAA:8 a=DBfv99YmAAAA:8 a=cWRNjhkoAAAA:8
 a=cm27Pg_UAAAA:8 a=IUHhT28BEWNEk4qSCmMA:9 a=QEXdDO2ut3YA:10
 a=WzC6qhA0u3u7Ye7llzcV:22 a=AjGcO6oz07-iQ99wixmX:22 a=c73wXdw0ADZYY2z2LwuN:22
 a=sVa6W5Aao32NNC1mekxh:22 a=xmb-EsYY8bH0VWELuYED:22
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=embeddedor.com; s=default; h=Content-Transfer-Encoding:Content-Type:
	In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date:Message-ID:Sender
	:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:
	Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:
	List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
	bh=Xom8zApxC86I/dIUvjWT77cxYWKLUOyYSTZXqNSiH60=; b=anFh+OynbpLIpeaK4DkLUiuKxJ
	YP0kUfSLPy0oZqhN8WGkhXzTW8gTKmyudSMVqvPb6hb8+ryTPcLq9GDTzBPQ5DZ01/nvt2aySxi0s
	FRkatydQWF9N2CqV/xky/GkmFDvEjMT821hPBQQCtNqaF6iKGq4EVUXiMnb9aeqbCrPRmnQHnes7a
	QlRIomTosqjQwLShSekZdWLQnvJkWvKtHffRg8V4AMuuh2SSfRSD066TtvkdmY8teGJopfegd3pWI
	nZ4szOJYIQ/+mUHUhY7YPRT7E8d0qw0iQKaYXrwV3W1AOtMnk9Y5hNjU8Kb9Y9vSnU2jxdXHi466d
	ggnsl+Zg==;
Message-ID: <33b313d4-ee53-4cd5-842e-147d60f462d6@embeddedor.com>
Date: Tue, 6 Feb 2024 11:06:29 -0600
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/gntalloc: Replace UAPI 1-element array
To: Kees Cook <keescook@chromium.org>, Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 "Gustavo A . R . Silva" <gustavoars@kernel.org>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org,
 linux-hardening@vger.kernel.org
References: <20240206170320.work.437-kees@kernel.org>
Content-Language: en-US
From: "Gustavo A. R. Silva" <gustavo@embeddedor.com>
In-Reply-To: <20240206170320.work.437-kees@kernel.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - gator4166.hostgator.com
X-AntiAbuse: Original Domain - lists.xenproject.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - embeddedor.com
X-BWhitelist: no
X-Source-IP: 187.162.21.192
X-Source-L: No
X-Exim-ID: 1rXOu2-003dF2-1M
X-Source: 
X-Source-Args: 
X-Source-Dir: 
X-Source-Sender: 187-162-21-192.static.axtel.net ([192.168.15.10]) [187.162.21.192]:34842
X-Source-Auth: gustavo@embeddedor.com
X-Email-Count: 3
X-Org: HG=hgshared;ORG=hostgator;
X-Source-Cap: Z3V6aWRpbmU7Z3V6aWRpbmU7Z2F0b3I0MTY2Lmhvc3RnYXRvci5jb20=
X-Local-Domain: yes
X-CMAE-Envelope: MS4xfOM5DpdON3qFgfxUu1DobqtAksbT1w7udGcYHS+AaSxYQ54GflvE+GpN+Sscv6PMFIvkTv9owA/LIRxN01JimhU0m4+idmyNkkAfulXIRixzIKAx9KUD
 wnWZp+auIXBDZdjrMmiYYLgERTOBVVxzoov4mNBU58QTzHsFkBSKcRtud6cXP9PFKPx6fayjwbIpPw5mB3GRfThvtMAyqikX9BbAVZ6FHNU4LtOedKcLYRtK



On 2/6/24 11:03, Kees Cook wrote:
> Without changing the structure size (since it is UAPI), add a proper
> flexible array member, and reference it in the kernel so that it will
> not be trip the array-bounds sanitizer[1].
> 
> Link: https://github.com/KSPP/linux/issues/113 [1]
> Cc: Juergen Gross <jgross@suse.com>
> Cc: Stefano Stabellini <sstabellini@kernel.org>
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> Cc: Gustavo A. R. Silva <gustavoars@kernel.org>
> Cc: xen-devel@lists.xenproject.org
> Signed-off-by: Kees Cook <keescook@chromium.org>

Reviewed-by: Gustavo A. R. Silva <gustavoars@kernel.org>

Thanks!
--
Gustavo

> ---
>   drivers/xen/gntalloc.c      | 2 +-
>   include/uapi/xen/gntalloc.h | 5 ++++-
>   2 files changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/xen/gntalloc.c b/drivers/xen/gntalloc.c
> index 26ffb8755ffb..f93f73ecefee 100644
> --- a/drivers/xen/gntalloc.c
> +++ b/drivers/xen/gntalloc.c
> @@ -317,7 +317,7 @@ static long gntalloc_ioctl_alloc(struct gntalloc_file_private_data *priv,
>   		rc = -EFAULT;
>   		goto out_free;
>   	}
> -	if (copy_to_user(arg->gref_ids, gref_ids,
> +	if (copy_to_user(arg->gref_ids_flex, gref_ids,
>   			sizeof(gref_ids[0]) * op.count)) {
>   		rc = -EFAULT;
>   		goto out_free;
> diff --git a/include/uapi/xen/gntalloc.h b/include/uapi/xen/gntalloc.h
> index 48d2790ef928..3109282672f3 100644
> --- a/include/uapi/xen/gntalloc.h
> +++ b/include/uapi/xen/gntalloc.h
> @@ -31,7 +31,10 @@ struct ioctl_gntalloc_alloc_gref {
>   	__u64 index;
>   	/* The grant references of the newly created grant, one per page */
>   	/* Variable size, depending on count */
> -	__u32 gref_ids[1];
> +	union {
> +		__u32 gref_ids[1];
> +		__DECLARE_FLEX_ARRAY(__u32, gref_ids_flex);
> +	};
>   };
>   
>   #define GNTALLOC_FLAG_WRITABLE 1


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 17:54:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 17:54:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677061.1053461 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXPdm-0003PK-Q2; Tue, 06 Feb 2024 17:53:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677061.1053461; Tue, 06 Feb 2024 17:53:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXPdm-0003PD-N1; Tue, 06 Feb 2024 17:53:50 +0000
Received: by outflank-mailman (input) for mailman id 677061;
 Tue, 06 Feb 2024 17:53:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=onkB=JP=ens-lyon.org=samuel.thibault@bounce.ens-lyon.org>)
 id 1rXPdl-0003NK-TM
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 17:53:49 +0000
Received: from sonata.ens-lyon.org (sonata.ens-lyon.org [140.77.166.138])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ac3b3d85-c518-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 18:53:45 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by sonata.ens-lyon.org (Postfix) with ESMTP id 3DDE2A0121;
 Tue,  6 Feb 2024 18:53:45 +0100 (CET)
Received: from sonata.ens-lyon.org ([127.0.0.1])
 by localhost (sonata.ens-lyon.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id FoEOQOKMLb2i; Tue,  6 Feb 2024 18:53:45 +0100 (CET)
Received: from begin.home (aamiens-653-1-111-57.w83-192.abo.wanadoo.fr
 [83.192.234.57])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by sonata.ens-lyon.org (Postfix) with ESMTPSA id E19D6A0112;
 Tue,  6 Feb 2024 18:53:44 +0100 (CET)
Received: from samy by begin.home with local (Exim 4.97)
 (envelope-from <samuel.thibault@ens-lyon.org>)
 id 1rXPdg-0000000HQU3-1y01; Tue, 06 Feb 2024 18:53:44 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac3b3d85-c518-11ee-98f5-efadbce2ee36
Date: Tue, 6 Feb 2024 18:53:44 +0100
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org, julien@xen.org
Subject: Re: [PATCH] Mini-OS: fix 9pfs frontend error path
Message-ID: <20240206175344.ljhcmkn2bso4hrvf@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	=?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
	minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org, julien@xen.org
References: <20240206061721.27297-1-jgross@suse.com>
 <20240206152602.5mbztpfam7x2ykcw@begin>
 <6ff940a5-5918-4da4-9a1e-ebe6fe9570a0@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <6ff940a5-5918-4da4-9a1e-ebe6fe9570a0@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Jürgen Groß, le mar. 06 févr. 2024 16:37:17 +0100, a ecrit:
> On 06.02.24 16:26, Samuel Thibault wrote:
> > Juergen Gross, le mar. 06 févr. 2024 07:17:21 +0100, a ecrit:
> > > The early error exit in p9_stat() returns without zeroing the p9_stat
> > > buffer, resulting in free() being called with an uninitialized pointer.
> > > 
> > > Fix that by doing the zeroing first.
> > 
> > This is not coherent with the usual conventions: when a function fails,
> > it is supposed not to have done anything, and thus the caller shouldn't
> > have to clean anything.
> > 
> > I.e. i'd rather see the free_stat() call be put after the check for
> > an error returned by p9_stat.
> 
> I can do that, but this would require two calls of free_stat() (one in
> p9_stat() in an error case reported via req->result, and one in the
> caller of p9_stat() in case of no error).

Indeed, but that still looks more coherent with usual conventions.

Samuel


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 18:49:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 18:49:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677070.1053470 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXQVm-0000ma-Nd; Tue, 06 Feb 2024 18:49:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677070.1053470; Tue, 06 Feb 2024 18:49:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXQVm-0000mT-Kr; Tue, 06 Feb 2024 18:49:38 +0000
Received: by outflank-mailman (input) for mailman id 677070;
 Tue, 06 Feb 2024 18:49:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rXQVl-0000mN-9E
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 18:49:37 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rXQVk-0001WW-TP; Tue, 06 Feb 2024 18:49:36 +0000
Received: from [15.248.2.233] (helo=[10.24.67.33])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rXQVk-00042C-MB; Tue, 06 Feb 2024 18:49:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=bDmWwyYrnhxFUlZ/CYWFLNkGcwAtNAoaw6XzzPgFMAc=; b=INcdQKM7TU1qDOL1EoMQpl0tb+
	2JrOs+o2gqd9DrgMNsb/3XPYwej5VNZMxD9VUy3JL71VAZTm3e58Hr/9IoCbxkXFJ4yucGJb2sDQ7
	YPEguck/o3DlWfaFp39O6L6eORlAaJkq4Zr6aGyBwmCKSTp10x4vzIapqoRlK3G2noRw=;
Message-ID: <c7841d11-d072-4787-9d76-ee94123605bb@xen.org>
Date: Tue, 6 Feb 2024 18:49:34 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v4 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
Content-Language: en-GB
To: Ayan Kumar Halder <ayan.kumar.halder@amd.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, michal.orzel@amd.com,
 luca.fancellu@arm.com
References: <20240131121049.225044-1-ayan.kumar.halder@amd.com>
 <20240131121049.225044-2-ayan.kumar.halder@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240131121049.225044-2-ayan.kumar.halder@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Ayan,

On 31/01/2024 12:10, Ayan Kumar Halder wrote:
> There can be situations when the registers cannot be emulated to their full
> functionality. This can be due to the complexity involved. In such cases, one
> can emulate those registers as RAZ/WI for example. We call them as partial
> emulation.
> 
> Some registers are non-optional and as such there is nothing preventing an OS
> from accessing them.
> Instead of injecting undefined exception (thus crashing a guest), one may want
> to prefer a partial emulation to let the guest running (in some cases accepting
> the fact that it might result in unwanted behavior).
> 
> A suitable example of this (as seen in subsequent patches) is emulation of
> DBGDTRTX_EL0 (on Arm64) and DBGDTRTXINT(on Arm32). These non-optional
> registers can be emulated as RAZ/WI and they can be enclosed within
> CONFIG_PARTIAL_EMULATION.
> 
> Further, "partial-emulation" command line option allows us to
> enable/disable partial emulation at run time. While CONFIG_PARTIAL_EMULATION
> enables support for partial emulation at compile time (i.e. adds code for
> partial emulation), this option may be enabled or disabled by Yocto or other
> build systems. However if the build system turns this option on, users
> can use scripts like Imagebuilder to generate uboot-script which will append
> "partial-emulation=false" to xen command line to turn off the partial
> emulation. Thus, it helps to avoid rebuilding xen.
> 
> By default, "CONFIG_PARTIAL_EMULATION=y" and "partial-emulation=false".
> This is done so that Xen supports partial emulation. However, customers are
> fully aware when they enable partial emulation. It's important to note that
> enabling such support might result in unwanted/non-spec compliant behavior.

Can you remind me why this is built by default? In particular...

> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> ---
> Changes from v1 :-
> 1. New patch introduced in v2.
> 
> v2 :-
> 1. Reordered the patches so that the config and command line option is
> introduced in the first patch.
> 
> v3 :-
> 1. Defined a macro 'partial_emulation' to reduce if-defs.
> 2. Fixed style issues.
> 
>   docs/misc/xen-command-line.pandoc | 11 +++++++++++
>   xen/arch/arm/Kconfig              |  9 +++++++++
>   xen/arch/arm/include/asm/traps.h  |  6 ++++++
>   xen/arch/arm/traps.c              |  9 +++++++++
>   4 files changed, 35 insertions(+)
> 
> diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
> index 8e65f8bd18..22c0d7c9f6 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1949,6 +1949,17 @@ This option is ignored in **pv-shim** mode.
>   
>   > Default: `on`
>   
> +### partial-emulation (arm)
> +> `= <boolean>`
> +
> +> Default: `false`
> +
> +Flag to enable or disable partial emulation of system/coprocessor registers.
> +Only effective if CONFIG_PARTIAL_EMULATION is enabled.
> +
> +**WARNING: Enabling this option might result in unwanted/non-spec compliant
> +behavior.**

... leads me to think that the default config should have it off. Still 
letting the integrator optionally opt-in.

It also wants some explanation about the security support statement. Is 
the goal to support any security issue that may arise from someone 
adding 'partial-emulation=true'?

> +
>   ### pci
>       = List of [ serr=<bool>, perr=<bool> ]
>   
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index 50e9bfae1a..8d8f668e7f 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -225,6 +225,15 @@ config STATIC_EVTCHN
>   	  This option enables establishing static event channel communication
>   	  between domains on a dom0less system (domU-domU as well as domU-dom0).
>   
> +config PARTIAL_EMULATION
> +	bool "Enable partial emulation of system/coprocessor registers"
> +	default y
> +	help
> +	  This option enables partial emulation of registers to prevent guests
> +	  crashing when accessing registers which are not optional but have not been

I think we need to list somewhere (possibly in the command line 
documentation) which registers are partially implemented. This will help 
the admin to quickly figure out whether this option makes sense for them.

> +	  emulated to its complete functionality. Enabling this might result in
> +	  unwanted/non-spec compliant behavior.

The description leads me to think if this is selected, then Xen will do 
the partial emulation. However, this is not matching the code.

Selecting this option doesn't result to unwanted/non-spec compliant 
behavior. What trigger the unwanted behavior if the command line option.

So I would suggest to reword the Kconfig to:

"Partial emulation support"

"Some of the required registers are not properly emulated by Xen. This 
option will allow the admin to select at runtime (via the command line 
'partial-emulation' whether the registers are unimplemented (i.e. a 
fault will be injected on access) or they will be partially emulated.

Partial emulation could be useful if a guest is trying to access any of 
the registers (see XXX for the list).

> +
>   endmenu
>   
>   menu "ARM errata workaround via the alternative framework"
> diff --git a/xen/arch/arm/include/asm/traps.h b/xen/arch/arm/include/asm/traps.h
> index 883dae368e..9a60dbf70e 100644
> --- a/xen/arch/arm/include/asm/traps.h
> +++ b/xen/arch/arm/include/asm/traps.h
> @@ -10,6 +10,12 @@
>   # include <asm/arm64/traps.h>
>   #endif
>   
> +#ifdef CONFIG_PARTIAL_EMULATION
> +extern bool partial_emulation;
> +#else
> +#define partial_emulation false
> +#endif
> +
>   /*
>    * GUEST_BUG_ON is intended for checking that the guest state has not been
>    * corrupted in hardware and/or that the hardware behaves as we
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 9c10e8f78c..d1c7a6c516 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -42,6 +42,15 @@
>   #include <asm/vgic.h>
>   #include <asm/vtimer.h>
>   
> +/*
> + * partial_emulation: If true, partial emulation for system/coprocessor
> + * registers will be enabled.
> + */
> +#ifdef CONFIG_PARTIAL_EMULATION
> +bool __ro_after_init partial_emulation = false;
> +boolean_param("partial-emulation", partial_emulation);
> +#endif

I think we should use warning_add() to print a message indicating the 
admin has enabled a configuration that is potentially unsafe for the guest.

> +
>   /* The base of the stack must always be double-word aligned, which means
>    * that both the kernel half of struct cpu_user_regs (which is pushed in
>    * entry.S) and struct cpu_info (which lives at the bottom of a Xen

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 18:56:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 18:56:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677075.1053481 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXQcW-0002IN-EG; Tue, 06 Feb 2024 18:56:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677075.1053481; Tue, 06 Feb 2024 18:56:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXQcW-0002IG-AE; Tue, 06 Feb 2024 18:56:36 +0000
Received: by outflank-mailman (input) for mailman id 677075;
 Tue, 06 Feb 2024 18:56:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXQcU-0002I4-RI; Tue, 06 Feb 2024 18:56:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXQcU-0001dA-ON; Tue, 06 Feb 2024 18:56:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXQcU-0004Oj-CU; Tue, 06 Feb 2024 18:56:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXQcU-0001fu-C2; Tue, 06 Feb 2024 18:56:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2HeRCWpS7BWJpAkKgIeHkmshAtfXXLbmDxYnp4KYJKs=; b=Al1Y7Wyy0K7mYy2+8T2rxPdaqF
	9lc6Zl6FKjmVMhXrnmZHc31KjifssQ22Ym4d6I/oEHCniHc3WBH6oWzUKy5w0YIP64eICKMieCcwU
	d0dKDPT2bJrIZRTLsTTrMj8WlYHGHQBpHW7f6MUgT/sh0gcxIxhuPBI1KyvrFsYA/71w=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184601-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184601: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-arm64-arm64-xl-credit1:debian-fixup:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=99bd3cb0d12e85d5114425353552121ec8f93adc
X-Osstest-Versions-That:
    linux=54be6c6c5ae8e0d93a6c4641cb7528eb0b6ba478
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 06 Feb 2024 18:56:34 +0000

flight 184601 linux-linus real [real]
flight 184608 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184601/
http://logs.test-lab.xenproject.org/osstest/logs/184608/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl-credit1  13 debian-fixup        fail pass in 184608-retest

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 184588

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit1 15 migrate-support-check fail in 184608 never pass
 test-arm64-arm64-xl-credit1 16 saverestore-support-check fail in 184608 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184588
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184588
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184588
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184588
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184588
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184588
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184588
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184588
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                99bd3cb0d12e85d5114425353552121ec8f93adc
baseline version:
 linux                54be6c6c5ae8e0d93a6c4641cb7528eb0b6ba478

Last test of basis   184588  2024-02-04 19:44:13 Z    1 days
Testing same since   184601  2024-02-06 08:13:56 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Guoyu Ou <benogy@gmail.com>
  Helge Deller <deller@gmx.de>
  Helge Deller <deller@kernel.org>
  Kent Overstreet <kent.overstreet@linux.dev>
  Linus Torvalds <torvalds@linux-foundation.org>
  Mathias Krause <minipli@grsecurity.net>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   54be6c6c5ae8..99bd3cb0d12e  99bd3cb0d12e85d5114425353552121ec8f93adc -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 19:05:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 19:05:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677083.1053491 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXQkq-00044e-DI; Tue, 06 Feb 2024 19:05:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677083.1053491; Tue, 06 Feb 2024 19:05:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXQkq-00044X-9W; Tue, 06 Feb 2024 19:05:12 +0000
Received: by outflank-mailman (input) for mailman id 677083;
 Tue, 06 Feb 2024 19:05:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rXQkp-00044R-8r
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 19:05:11 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rXQko-0001oy-QS; Tue, 06 Feb 2024 19:05:10 +0000
Received: from [15.248.2.233] (helo=[10.24.67.33])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rXQko-000577-Jy; Tue, 06 Feb 2024 19:05:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=9/LqaCaGVK2AihcOmcNwgx4zFOFq/CqvfKVNCDwmvV4=; b=wWx9tn1XYqO1f78tWvd1u4lYRG
	M9DowVasE7ZNg1HttM0haZ7kRob44d4wySM73dosRjKYG0lrCiyBzkvXnSVIW6/RPUEsHaeZQuykI
	A61Q1HSnC0ikQh3H2uWym1IQXg4skN3CI6x+bgGPd03KVzAhBdf6eGd/SSWJ66jGEMRk=;
Message-ID: <0c4e4a40-7327-4377-bfdd-82e69bc838ae@xen.org>
Date: Tue, 6 Feb 2024 19:05:08 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v4 2/3] xen/arm: arm64: Add emulation of Debug Data Transfer
 Registers
Content-Language: en-GB
To: Ayan Kumar Halder <ayan.kumar.halder@amd.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, michal.orzel@amd.com,
 luca.fancellu@arm.com
References: <20240131121049.225044-1-ayan.kumar.halder@amd.com>
 <20240131121049.225044-3-ayan.kumar.halder@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240131121049.225044-3-ayan.kumar.halder@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Ayan,

On 31/01/2024 12:10, Ayan Kumar Halder wrote:
> From: Michal Orzel <michal.orzel@amd.com>
> 
> Currently, if user enables HVC_DCC config option in Linux, it invokes access
> to debug data transfer registers (i.e. DBGDTRTX_EL0 on arm64, DBGDTRTXINT on
> arm32). As these registers are not emulated, Xen injects an undefined
> exception to the guest and Linux crashes.
> 
> To prevent this crash, introduce a partial emulation of DBGDTR[TR]X_EL0
> (these registers share the same encoding) as RAZ/WI and MDCCSR_EL0 as TXfull.
> 
> Refer ARM DDI 0487J.a ID042523, D19.3.8, DBGDTRTX_EL0
> "If TXfull is set to 1, set DTRRX and DTRTX to UNKNOWN".
> 
> Thus, any OS is expected to read MDCCSR_EL0 and check for TXfull before
> using DBGDTRTX_EL0. Linux does it via hvc_dcc_init() ---> hvc_dcc_check(),
> and returns -ENODEV in case TXfull bit is still set after writing a test
> character. This way we prevent the guest from making use of HVC DCC as a
> console.
> 
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> ---
> Changes from
> 
> v1 :- 1. DBGDTR_EL0 does not emulate RXfull. This is to avoid giving the OS any
> indication that the RX buffer is full and is waiting to be read.
> 
> 2. In Arm32, DBGOSLSR is emulated. Also DBGDTRTXINT is emulated at EL0 only.
> 
> 3. Fixed the commit message and inline code comments.
> 
> v2 :- 1. Split the patch into two (separate patches for arm64 and arm32).
> 2. Removed the "fail" label.
> 3. Fixed the commit message.
> 
> v3 :- 1. "HSR_SYSREG_MDCCSR_EL0" emulation differs based on whether
> partial_emulation_enabled is true or not.
> 
> 2. If partial_emulation_enabled is false, then access to HSR_SYSREG_DBGDTR_EL0,
> HSR_SYSREG_DBGDTRTX_EL0 would lead to undefined exception.
> 
>   xen/arch/arm/arm64/vsysreg.c         | 28 ++++++++++++++++++++++++----
>   xen/arch/arm/include/asm/arm64/hsr.h |  3 +++
>   2 files changed, 27 insertions(+), 4 deletions(-)
> 
> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
> index b5d54c569b..94f0a6c384 100644
> --- a/xen/arch/arm/arm64/vsysreg.c
> +++ b/xen/arch/arm/arm64/vsysreg.c
> @@ -159,9 +159,6 @@ void do_sysreg(struct cpu_user_regs *regs,
>        *
>        * Unhandled:
>        *    MDCCINT_EL1
> -     *    DBGDTR_EL0
> -     *    DBGDTRRX_EL0
> -     *    DBGDTRTX_EL0
>        *    OSDTRRX_EL1
>        *    OSDTRTX_EL1
>        *    OSECCR_EL1
> @@ -173,10 +170,32 @@ void do_sysreg(struct cpu_user_regs *regs,
>           return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
>       case HSR_SYSREG_MDCCSR_EL0:
>           /*
> +         * Xen doesn't expose a real (or emulated) Debug Communications Channel
> +         * (DCC) to a domain. Yet the Arm ARM implies this is not an optional
> +         * feature. So some domains may start to probe it. For instance, the
> +         * HVC_DCC driver in Linux (since f377775dc083 and at least up to v6.7),
> +         * will try to write some characters and check if the transmit buffer
> +         * has emptied.
> +         *
> +         * By setting TX status bit (only if partial emulation is enabled) to
> +         * indicate the transmit buffer is full, we would hint the OS that the
> +         * DCC is probably not working.
> +         *
> +         * Bit 29: TX full
> +         *
>            * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We emulate that
>            * register as RAZ/WI above. So RO at both EL0 and EL1.

The sentence "we emulate that register as ..." seems to be stale?

>            */
> -        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 0);
> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 0,
> +                                  partial_emulation ? (1U << 29) : 0);
> +
> +    case HSR_SYSREG_DBGDTR_EL0:
> +    /* DBGDTR[TR]X_EL0 share the same encoding */
> +    case HSR_SYSREG_DBGDTRTX_EL0:
> +        if ( !partial_emulation )
> +            goto fail;
> +        return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 0);

AFAICT, all the emulation helpers have an explanation why we are using 
them. But here this is not the case. Can you add one?

> +
>       HSR_SYSREG_DBG_CASES(DBGBVR):
>       HSR_SYSREG_DBG_CASES(DBGBCR):
>       HSR_SYSREG_DBG_CASES(DBGWVR):
> @@ -394,6 +413,7 @@ void do_sysreg(struct cpu_user_regs *regs,
>        * And all other unknown registers.
>        */
>       default:
> + fail:

AFAICT, this would violate MISRA 15.3 [1]. We didn't seem to have yet 
(?) accepted the rule, but I don't see we would not given I feel this is 
similar to what Rule 16.2 is trying to prevent and we accepted it.

I think case, I move all the code within default outside. And then call 
"goto fail" from the default label.

>           {
>               const struct hsr_sysreg sysreg = hsr.sysreg;
>   
> diff --git a/xen/arch/arm/include/asm/arm64/hsr.h b/xen/arch/arm/include/asm/arm64/hsr.h
> index e691d41c17..1495ccddea 100644
> --- a/xen/arch/arm/include/asm/arm64/hsr.h
> +++ b/xen/arch/arm/include/asm/arm64/hsr.h
> @@ -47,6 +47,9 @@
>   #define HSR_SYSREG_OSDLR_EL1      HSR_SYSREG(2,0,c1,c3,4)
>   #define HSR_SYSREG_DBGPRCR_EL1    HSR_SYSREG(2,0,c1,c4,4)
>   #define HSR_SYSREG_MDCCSR_EL0     HSR_SYSREG(2,3,c0,c1,0)
> +#define HSR_SYSREG_DBGDTR_EL0     HSR_SYSREG(2,3,c0,c4,0)
> +#define HSR_SYSREG_DBGDTRTX_EL0   HSR_SYSREG(2,3,c0,c5,0)
> +#define HSR_SYSREG_DBGDTRRX_EL0   HSR_SYSREG(2,3,c0,c5,0)
>   
>   #define HSR_SYSREG_DBGBVRn_EL1(n) HSR_SYSREG(2,0,c0,c##n,4)
>   #define HSR_SYSREG_DBGBCRn_EL1(n) HSR_SYSREG(2,0,c0,c##n,5)

Cheers,

[1] 
https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_15_03.c
[2] 
https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_16_02.c

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 19:11:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 19:11:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677089.1053505 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXQr2-0005jF-5L; Tue, 06 Feb 2024 19:11:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677089.1053505; Tue, 06 Feb 2024 19:11:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXQr2-0005ir-1P; Tue, 06 Feb 2024 19:11:36 +0000
Received: by outflank-mailman (input) for mailman id 677089;
 Tue, 06 Feb 2024 19:11:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=z75a=JP=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXQr0-0005iR-R9
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 19:11:34 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 890635d0-c523-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 20:11:31 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 6629C1F38E;
 Tue,  6 Feb 2024 19:11:30 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 22C49139D8;
 Tue,  6 Feb 2024 19:11:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id W+45O2CEwmXSHQAAD6G6ig
 (envelope-from <jgross@suse.com>); Tue, 06 Feb 2024 19:11:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 890635d0-c523-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707246690; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=v3tdZNE3CtH/sw9mKXkbSutXzAO/0g8n6ykGpI2lzis=;
	b=EZbfd8XCXsYGmQoQxJeGsrmeM4xhPDzP40waWElzCWaAkcw+lNPc4zYMQmVVQAFWE5r70t
	IpFlAejVodVeLVOkdgytNKGHpoO0jFABQZq1FU6iChOzh6Z2Q+o40g6jsNIR8J4JE4tok6
	q6+qVl/Flny/RuutjA5IjLKXgapYWDE=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707246690; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=v3tdZNE3CtH/sw9mKXkbSutXzAO/0g8n6ykGpI2lzis=;
	b=EZbfd8XCXsYGmQoQxJeGsrmeM4xhPDzP40waWElzCWaAkcw+lNPc4zYMQmVVQAFWE5r70t
	IpFlAejVodVeLVOkdgytNKGHpoO0jFABQZq1FU6iChOzh6Z2Q+o40g6jsNIR8J4JE4tok6
	q6+qVl/Flny/RuutjA5IjLKXgapYWDE=
From: Juergen Gross <jgross@suse.com>
To: minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org
Cc: samuel.thibault@ens-lyon.org,
	wl@xen.org,
	julien@xen.org,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2] Mini-OS: fix 9pfs frontend error path
Date: Tue,  6 Feb 2024 20:11:25 +0100
Message-Id: <20240206191125.13065-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [4.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[6];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Level: ****
X-Spam-Score: 4.90
X-Spam-Flag: NO

The early error exit in p9_stat() returns without zeroing the p9_stat
buffer, resulting in free() being called with an uninitialized pointer.

Fix that by calling free_stat() in p9_stat() in case of returning an
error and potentially having allocated strings.

Reported-by: Julien Grall <julien@xen.org>
Fixes: 2d1dfccd3aa3 ("Mini-OS: add read and write support to 9pfsfront")
Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- call free_stat() in p9_stat() in case of returning an error
  (Samuel Thibault)
---
 9pfront.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/9pfront.c b/9pfront.c
index 315089bc..042879a7 100644
--- a/9pfront.c
+++ b/9pfront.c
@@ -728,6 +728,8 @@ static int p9_stat(struct dev_9pfs *dev, uint32_t fid, struct p9_stat *stat)
            &stat->extension, &stat->n_uid, &stat->n_gid, &stat->n_muid);
 
     ret = req->result;
+    if ( ret )
+        free_stat(&stat);
 
     put_free_req(dev, req);
 
@@ -932,13 +934,13 @@ static int write_9pfs(struct file *file, const void *buf, size_t nbytes)
     if ( f9pfs->append )
     {
         ret = p9_stat(f9pfs->dev, f9pfs->fid, &stat);
-        free_stat(&stat);
         if ( ret )
         {
             errno = EIO;
             return -1;
         }
         file->offset = stat.length;
+        free_stat(&stat);
     }
 
     ret = p9_write(f9pfs->dev, f9pfs->fid, file->offset, buf, nbytes);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Tue Feb 06 19:15:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 19:15:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677099.1053519 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXQua-0006O5-Lh; Tue, 06 Feb 2024 19:15:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677099.1053519; Tue, 06 Feb 2024 19:15:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXQua-0006NB-HZ; Tue, 06 Feb 2024 19:15:16 +0000
Received: by outflank-mailman (input) for mailman id 677099;
 Tue, 06 Feb 2024 19:15:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=onkB=JP=ens-lyon.org=samuel.thibault@bounce.ens-lyon.org>)
 id 1rXQuZ-0006Mg-8k
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 19:15:15 +0000
Received: from sonata.ens-lyon.org (domu-toccata.ens-lyon.fr [140.77.166.138])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0c5d9e84-c524-11ee-98f5-efadbce2ee36;
 Tue, 06 Feb 2024 20:15:11 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by sonata.ens-lyon.org (Postfix) with ESMTP id ED6E6A0112;
 Tue,  6 Feb 2024 20:15:10 +0100 (CET)
Received: from sonata.ens-lyon.org ([127.0.0.1])
 by localhost (sonata.ens-lyon.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id dvuld9gsIx-b; Tue,  6 Feb 2024 20:15:10 +0100 (CET)
Received: from begin (aamiens-653-1-111-57.w83-192.abo.wanadoo.fr
 [83.192.234.57])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by sonata.ens-lyon.org (Postfix) with ESMTPSA id 9300AA00DD;
 Tue,  6 Feb 2024 20:15:10 +0100 (CET)
Received: from samy by begin with local (Exim 4.97)
 (envelope-from <samuel.thibault@ens-lyon.org>)
 id 1rXQuU-0000000HSQe-0cV0; Tue, 06 Feb 2024 20:15:10 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c5d9e84-c524-11ee-98f5-efadbce2ee36
Date: Tue, 6 Feb 2024 20:15:10 +0100
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org, julien@xen.org
Subject: Re: [PATCH v2] Mini-OS: fix 9pfs frontend error path
Message-ID: <20240206191510.zzlla3r2w7df3hvq@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org, wl@xen.org, julien@xen.org
References: <20240206191125.13065-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240206191125.13065-1-jgross@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Juergen Gross, le mar. 06 févr. 2024 20:11:25 +0100, a ecrit:
> The early error exit in p9_stat() returns without zeroing the p9_stat
> buffer, resulting in free() being called with an uninitialized pointer.
> 
> Fix that by calling free_stat() in p9_stat() in case of returning an
> error and potentially having allocated strings.
> 
> Reported-by: Julien Grall <julien@xen.org>
> Fixes: 2d1dfccd3aa3 ("Mini-OS: add read and write support to 9pfsfront")
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Thanks!

> ---
> V2:
> - call free_stat() in p9_stat() in case of returning an error
>   (Samuel Thibault)
> ---
>  9pfront.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/9pfront.c b/9pfront.c
> index 315089bc..042879a7 100644
> --- a/9pfront.c
> +++ b/9pfront.c
> @@ -728,6 +728,8 @@ static int p9_stat(struct dev_9pfs *dev, uint32_t fid, struct p9_stat *stat)
>             &stat->extension, &stat->n_uid, &stat->n_gid, &stat->n_muid);
>  
>      ret = req->result;
> +    if ( ret )
> +        free_stat(&stat);
>  
>      put_free_req(dev, req);
>  
> @@ -932,13 +934,13 @@ static int write_9pfs(struct file *file, const void *buf, size_t nbytes)
>      if ( f9pfs->append )
>      {
>          ret = p9_stat(f9pfs->dev, f9pfs->fid, &stat);
> -        free_stat(&stat);
>          if ( ret )
>          {
>              errno = EIO;
>              return -1;
>          }
>          file->offset = stat.length;
> +        free_stat(&stat);
>      }
>  
>      ret = p9_write(f9pfs->dev, f9pfs->fid, file->offset, buf, nbytes);
> -- 
> 2.35.3


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 20:23:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 20:23:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677111.1053529 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXRyO-0006vH-EN; Tue, 06 Feb 2024 20:23:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677111.1053529; Tue, 06 Feb 2024 20:23:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXRyO-0006vA-9Y; Tue, 06 Feb 2024 20:23:16 +0000
Received: by outflank-mailman (input) for mailman id 677111;
 Tue, 06 Feb 2024 20:23:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXRyN-0006v0-Hi; Tue, 06 Feb 2024 20:23:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXRyN-0003aD-F0; Tue, 06 Feb 2024 20:23:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXRyN-00074c-6m; Tue, 06 Feb 2024 20:23:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXRyN-0004rJ-6J; Tue, 06 Feb 2024 20:23:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=juKP3ItFQfYhkjjih6KhUdUmDfJTHSedkTwIPYFz42w=; b=ulzyk+aNP97ioTnIx/b2TfFpD7
	NIEH9WEC7LJV17mv83QiUKv+hxeQjDVCay6QrFOvEW7+OkzehGgZ6X+BvG5MbcOIV9B+dGys21qjF
	7+rPmD7ldrJ8XI/FgA19HIIaTOHZmikiSfz/IoVYXO3xazkQVc/pd2JTHA5JmlMrcIKs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184607-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184607: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=62b43ec8960372abed59c1e1d596c2b324340a07
X-Osstest-Versions-That:
    ovmf=ae59b8ba4166384cbfa32a921aac289bcff2aef9
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 06 Feb 2024 20:23:15 +0000

flight 184607 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184607/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 62b43ec8960372abed59c1e1d596c2b324340a07
baseline version:
 ovmf                 ae59b8ba4166384cbfa32a921aac289bcff2aef9

Last test of basis   184603  2024-02-06 09:15:38 Z    0 days
Testing same since   184607  2024-02-06 16:14:43 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Chao Li <lichao@loongson.cn>
  Gerd Hoffmann <kraxel@redhat.com>
  Michael D Kinney <michael.d.kinney@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   ae59b8ba41..62b43ec896  62b43ec8960372abed59c1e1d596c2b324340a07 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 21:30:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 21:30:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677122.1053539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXT0r-0005UN-AL; Tue, 06 Feb 2024 21:29:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677122.1053539; Tue, 06 Feb 2024 21:29:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXT0r-0005UG-6H; Tue, 06 Feb 2024 21:29:53 +0000
Received: by outflank-mailman (input) for mailman id 677122;
 Tue, 06 Feb 2024 21:29:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7seG=JP=gmail.com=marietto2008@srs-se1.protection.inumbo.net>)
 id 1rXT0o-0005U5-RS
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 21:29:51 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d9fdc795-c536-11ee-8a49-1f161083a0e0;
 Tue, 06 Feb 2024 22:29:49 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a3832ef7726so120894366b.0; 
 Tue, 06 Feb 2024 13:29:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d9fdc795-c536-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707254986; x=1707859786; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Wc0TQ/QSpTIAaD8olFlSw7O8iNWhc2W7oRAIo0Ds/2I=;
        b=lUcMuszMwBsUXXPs3sziplzxi3neW2ebc2QayNCZGKxZPh/ugJN+16ycUzV9rVn9Tl
         4UZxDaXDa5/NTvhTzfcSIpmCb0ci0vUT3n4McviE0dYNngRWjRNeH/7YpraA22dAz8uu
         1Vi5Qs221glXyDCpBZrPXduvgOjx7+YkwuOasG/kdEskqPOpjpDSn9bplQbJ2rBGSior
         AVPyDCarQSf+zuQJfufgPqDzzLpX25cBhobb7EUZBMHzC4tr41sImqEgrN9eQI3V750O
         BZRpMk2Uw/HOVfgoMGr/EpWFeqJmSMhgMTxN7ai3QKOlHeECGQIqfTqaJ6+vk7gw0AWX
         l5XA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707254986; x=1707859786;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Wc0TQ/QSpTIAaD8olFlSw7O8iNWhc2W7oRAIo0Ds/2I=;
        b=cwZdLqb/DDcHFHOK8EtoB8AE+rbIRCVoi60f4Mwe6uPwB2wh/t2/HAkmh/hirl9NTh
         ud4IHtdH+PMfaSPYBjNT1tIbrLOKLiPwOd6UQPLKN7safYUOmOq3DPexsDFXwmSZickd
         0k1AQZ5KlvnB6JGDoZnFjL8wyuPUKcmr/ryClzUWUrGhZV5URwy4St3rNy5Dmhwk9oK8
         pCkwbonZgsnEw4UFULzjQpnkFXkN2cRUXVkdA6/Qn2nUSq5QwZNyjgAWCKRx5JqwnBUi
         vhsHht+ah2h/LZdrgkDfn3WCOpGm4nBEWDeR/VketEID4EALDFuKzBvuYg/050JqZC7P
         vYBg==
X-Forwarded-Encrypted: i=1; AJvYcCWo8HdMsufG7UUtj+s74EbLdyrSkOucH3Zzw6SeRmW1G22oNO6asYihGqjUZnNHIoZuXvhrBWnCPQljERRav3YO/KIwxyZ7Lm7dYeomYMA=
X-Gm-Message-State: AOJu0Yz3tE5cAAwXJD9ns+hEZ1W/Rqj0KYt6lTKVLO1Kl3c0Jq5gl3m7
	iTUNa7xQAL0Ytx5gA817aJNDzNVeUMnpz45LJrSD3729G4ON+F4RPG4Bsv2DcBdEDsX7nKm4Dp4
	Qyco0tIjDnKstpT2R+6OXMN1Gy1kWm5GMD7M=
X-Google-Smtp-Source: AGHT+IFj8zfLZ6FsHTKqdjZWNA1HhDSHaH7qSPSv2y4fkrCnb5W3k/waUY5QtzPoDUyiD31h2YoIY/Fdi2iGFPE/hQI=
X-Received: by 2002:a17:906:8a78:b0:a38:5e8d:20d7 with SMTP id
 hy24-20020a1709068a7800b00a385e8d20d7mr322874ejc.17.1707254985651; Tue, 06
 Feb 2024 13:29:45 -0800 (PST)
MIME-Version: 1.0
From: Mario Marietto <marietto2008@gmail.com>
Date: Tue, 6 Feb 2024 22:29:08 +0100
Message-ID: <CA+1FSij2EMk8ktjFeybLWUKWnACro3Afgcju3fqG8udP01og9g@mail.gmail.com>
Subject: KVM run failed Function not implemented.
To: xen-devel <xen-devel@lists.xenproject.org>, freebsd-xen@freebsd.org, 
	Warner Losh <imp@bsdimp.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, 
	Stanislav Silnicki <stanislav.silnicki@mailgate.us>, Artem_Mygaiev@epam.com, 
	Michal Orzel <michal.orzel@amd.com>, xen-users@lists.xenproject.org, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <royger@freebsd.org>, 
	Elliott Mitchell <ehem+freebsd@m5p.com>, mhorne@freebsd.org, 
	freebsd-hackers <freebsd-hackers@freebsd.org>, freebsd-arm <freebsd-arm@freebsd.org>
Content-Type: multipart/related; boundary="000000000000f639c50610bd461b"

--000000000000f639c50610bd461b
Content-Type: multipart/alternative; boundary="000000000000f639c30610bd461a"

--000000000000f639c30610bd461a
Content-Type: text/plain; charset="UTF-8"

Hello.

Probably FreeBSD didn't boot as domU under xen because I was using the
kernel recompiled from the @Elliott Mitchell <ehem+freebsd@m5p.com>
code,WITHOUT its world,that's of FreeNSD 15.0-CURRENT. I mean,I was still
using the world of FreeBSD 13.2.  So,what I did was to even upgrade the
world of FreeBSD 13.2 with the world of the Elliott Mitchell code. This is
what I did (directly on the ARM Chromebook,booting FreeBSD 13.2 as vm with
qemu / kvm :

root@freebsd:/usr/freebsd-src # make KERNCONF=GENERIC TARGET=arm
TARGET_ARCH=armv7 -DWITHOUT_CLEAN -DNO_KERNELCLEAN buildworld
buildkernel

root@freebsd:/usr/freebsd-src # etcupdate -p
Failed to build new tree.

root@freebsd:/usr/freebsd-src # make installworld installkernel
no errors.
root@freebsd:/usr/freebsd-src # etcupdate -B
Failed to build new tree.

root@freebsd:/usr/freebsd-src # etcupdate resolve
no errors

root@freebsd:/usr/freebsd-src # nano /var/db/etcupdate/log

>>> update command: rerun= tarball= preworld=
>>> Building tree at /var/db/etcupdate/etcupdate-qcKPX9R with make  -DNO_FILEMON
make: don't know how to make distrib-dirs. Stop
make: stopped in /usr/src


Despite those errors,I've rebooted the FreeBSD virtual machine,which boots
from this script :

#!/bin/bash
# Start QEMU VM in the background

stty intr ^p

rm -r trace*
ip link add name br1 type bridge
ip link set br1 up
ip a add 192.168.20.1/24 dev br1
iptables -t nat -A POSTROUTING -o enx8cae4cd6c871 -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward

UEFICODE1=/usr/share/AAVMF/AAVMF32_CODE.fd
UEFIVARS2=/usr/share/AAVMF/AAVMF32_VARS.fd

UEFICODE3=/usr/local/share/qemu/edk2-arm-code.fd
UEFIVARS4=/usr/local/share/qemu/edk2-arm-vars.fd
UEFIVARS5=/var/local/lib/libvirt/qemu/nvram/freebsd13.1_VARS.fd

DISK=/Dati/Files/kvm/FreeBSD-15.0-CURRENT-armv7.img

qemu-system-arm \
    -enable-kvm -serial stdio \
    -m 1024 -M virt -cpu cortex-a15 \
    -drive if=pflash,format=raw,unit=0,file=$UEFICODE1 \
    -drive if=pflash,format=raw,unit=1,file=$UEFIVARS2 \
    -drive file=$DISK,media=disk,format=raw \
    -drive file=/dev/sda,media=disk,format=raw \
    -net nic,model=virtio,macaddr=52:54:00:00:00:01 -net bridge,br=br1 \
    -device virtio-gpu-pci \
    -usb -device nec-usb-xhci \
    -device usb-kbd -device usb-mouse \
    -device vmware-svga,id=video0,vgamem_mb=16 \

This is the huge error that happened and that prevents me from using
FreeBSD 15.0 :


[image: Senza titolo.jpeg]


If you can't see any attached image because the Mailing Lists removes the
attached pictures,you can see which error I got clicking on the picture on
the link below :

https://ibb.co/k3qdyhs


-- 
Mario.

--000000000000f639c30610bd461a
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><table cellpadding=3D"0" class=3D"gmail-ajC"><tbody><=
tr class=3D"gmail-ajv"><td colspan=3D"2" tabindex=3D"0" class=3D"gmail-gL">=
Hello.<br><br>Probably FreeBSD didn&#39;t boot as domU under xen because I =
was using the kernel recompiled from the=C2=A0<a class=3D"gmail_plusreply" =
id=3D"plusReplyChip-0" href=3D"mailto:ehem+freebsd@m5p.com" tabindex=3D"-1"=
>@Elliott Mitchell</a> code,WITHOUT its world,that&#39;s of FreeNSD 15.0-CU=
RRENT. I mean,I was still using the world of FreeBSD 13.2.=C2=A0 So,what I =
did was to even upgrade the world of FreeBSD 13.2 with the world of the Ell=
iott Mitchell code. This is what I did (directly on the ARM Chromebook,boot=
ing FreeBSD 13.2 as vm with qemu / kvm :<br><br><pre class=3D"gmail-bbCodeC=
ode" dir=3D"ltr"><code>root@freebsd:/usr/freebsd-src # make KERNCONF=3DGENE=
RIC TARGET=3Darm TARGET_ARCH=3Darmv7 -DWITHOUT_CLEAN -DNO_KERNELCLEAN build=
world buildkernel

root@freebsd:/usr/freebsd-src # etcupdate -p
Failed to build new tree.

root@freebsd:/usr/freebsd-src # make installworld installkernel
no errors.
</code><code></code><code>
root@freebsd:/usr/freebsd-src # etcupdate -B
Failed to build new tree.

root@freebsd:/usr/freebsd-src # etcupdate resolve
no errors

root@freebsd:/usr/freebsd-src # nano /var/db/etcupdate/log

&gt;&gt;&gt; update command: rerun=3D tarball=3D preworld=3D
&gt;&gt;&gt; Building tree at /var/db/etcupdate/etcupdate-qcKPX9R with make=
  -DNO_FILEMON
make: don&#39;t know how to make distrib-dirs. Stop
make: stopped in /usr/src</code></pre><br>Despite those errors,I&#39;ve reb=
ooted the FreeBSD virtual machine,which boots from this script :<br><br>#!/=
bin/bash<br># Start QEMU VM in the background<br><br>stty intr ^p<br><br>rm=
 -r trace*<br>ip link add name br1 type bridge<br>ip link set br1 up<br>ip =
a add <a href=3D"http://192.168.20.1/24">192.168.20.1/24</a> dev br1<br>ipt=
ables -t nat -A POSTROUTING -o enx8cae4cd6c871 -j MASQUERADE<br>echo 1 &gt;=
 /proc/sys/net/ipv4/ip_forward<br><br>UEFICODE1=3D/usr/share/AAVMF/AAVMF32_=
CODE.fd<br>UEFIVARS2=3D/usr/share/AAVMF/AAVMF32_VARS.fd<br><br>UEFICODE3=3D=
/usr/local/share/qemu/edk2-arm-code.fd<br>UEFIVARS4=3D/usr/local/share/qemu=
/edk2-arm-vars.fd<br>UEFIVARS5=3D/var/local/lib/libvirt/qemu/nvram/freebsd1=
3.1_VARS.fd<br><br>DISK=3D/Dati/Files/kvm/FreeBSD-15.0-CURRENT-armv7.img<br=
><br>qemu-system-arm \<br>=C2=A0 =C2=A0 -enable-kvm -serial stdio \<br>=C2=
=A0 =C2=A0 -m 1024 -M virt -cpu cortex-a15 \<br>=C2=A0 =C2=A0 -drive if=3Dp=
flash,format=3Draw,unit=3D0,file=3D$UEFICODE1 \<br>=C2=A0 =C2=A0 -drive if=
=3Dpflash,format=3Draw,unit=3D1,file=3D$UEFIVARS2 \<br>=C2=A0 =C2=A0 -drive=
 file=3D$DISK,media=3Ddisk,format=3Draw \<br>=C2=A0 =C2=A0 -drive file=3D/d=
ev/sda,media=3Ddisk,format=3Draw \<br>=C2=A0 =C2=A0 -net nic,model=3Dvirtio=
,macaddr=3D52:54:00:00:00:01 -net bridge,br=3Dbr1 \<br>=C2=A0 =C2=A0 -devic=
e virtio-gpu-pci \<br>=C2=A0 =C2=A0 -usb -device nec-usb-xhci \<br>=C2=A0 =
=C2=A0 -device usb-kbd -device usb-mouse \<br>=C2=A0 =C2=A0 -device vmware-=
svga,id=3Dvideo0,vgamem_mb=3D16 \<br><br>This is the huge error that happen=
ed and that prevents me from using FreeBSD 15.0 :<br>=C2=A0<br><br><img src=
=3D"cid:ii_lsauz5jc1" alt=3D"Senza titolo.jpeg" width=3D"275" height=3D"488=
"><br><br><br>If you can&#39;t see any attached image because the Mailing L=
ists removes the attached pictures,you can see which error I got clicking o=
n the picture on the link below : <br><br><a href=3D"https://ibb.co/k3qdyhs=
">https://ibb.co/k3qdyhs</a><br></td></tr><tr class=3D"gmail-ajv"><td colsp=
an=3D"2" tabindex=3D"0" class=3D"gmail-gG"><br></td><td colspan=3D"2" tabin=
dex=3D"0" class=3D"gmail-gL"><br></td></tr></tbody></table></div><div><span=
 class=3D"gmail_signature_prefix">-- </span><br><div dir=3D"ltr" class=3D"g=
mail_signature" data-smartmail=3D"gmail_signature">Mario.<br></div></div></=
div>

--000000000000f639c30610bd461a--
--000000000000f639c50610bd461b
Content-Type: image/jpeg; name="Senza titolo.jpeg"
Content-Disposition: inline; filename="Senza titolo.jpeg"
Content-Transfer-Encoding: base64
Content-ID: <ii_lsauz5jc1>
X-Attachment-Id: ii_lsauz5jc1

/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAYGBgYHBgcICAcKCwoLCg8ODAwODxYQERAREBYiFRkV
FRkVIh4kHhweJB42KiYmKjY+NDI0PkxERExfWl98fKcBBgYGBgcGBwgIBwoLCgsKDw4MDA4PFhAR
EBEQFiIVGRUVGRUiHiQeHB4kHjYqJiYqNj40MjQ+TERETF9aX3x8p//CABEICAAEgAMBIgACEQED
EQH/xAAyAAACAwEBAQAAAAAAAAAAAAAAAQIDBAUGBwEBAQEBAQEAAAAAAAAAAAAAAAECBAMF/9oA
DAMBAAIQAxAAAALiJrn95QkkGBFORGcJrKE4w2mtc4TGCIuuymhImpqNORKUSti0bQNADHDAJCcE
ZRG0yMkK00KUJWRaYk0OE41OqytK4tbzOcJzUmjNcoyglFjAlcZRJJoU4yE0RJpkQBjATQgVCAk4
uJAEYTjUgCNlc0UJwJwnAlCcKcoyIX0XQnKuUhOuycWU6pwHmvz7zO2q2JyjLOgHDTSyQQpRmRak
JSiMAjKAWOLCudZFM1BMBpkJwkKUZDTUEoyquUZEkEQlGVNMISjIYEoBZBSjQAMAAUTAG04Iygsm
CIEsk0KUJ2QlFiTVCcSddlaVJreJyjOakJ51JxkMThgSg0SjJClGQIImIEANxkIASapAI2mSQ5Yx
kqkmkhZCYoWVk67KyyEo0pwmQthKWymyuJRHZFqNOEq6ln0Z9ZnbVdEmnnUgUs4tEkIJwlEZwkCc
RtFQSKtE4K512JqRFpg4yK7ITIgySaglF1CUZDGFcoyGBEJRdSBQxMjGcKAKaYIaiYmNpwRlBZNJ
JRaqSaIzhMhKMiKasIyiTrsrqpSjrE51zmptGa2mpKMoYErTRJNClGQRYSAhAiTAE0JNUgaMTJCI
UZRqaAjOuxVCcEnCyslGUKVlVoDri2EokZxjTi1YQaqVF9FkrqrZZNPNkmokmlaYKUZxCSkRTVA0
VyJAxChOFilGRCUZCnXaVTjIQImnEJQmQlFkgZBxZIUiDQSQA0xQsrACxiaiAk04YmOMoxOLCIRq
xARnCZEASCiM4JOuyuqk1rEpRnNSGTTacDCJDjLKMokgCM4yIjBjUJiJAAmCTVRYWNxcMASETBEb
KrArnXVtdkBppK7qbSUJwUdbscWhNFKE41Oi6lmdtVsswM6knGJppWJg04TTEhji40SjMScERGdQ
BpCUZ1GyqyISiyUZRWYmRlCZBuJNxZBoJOMiKlEkAMTFCcKBMHGQgCTTgYxJocoTiKlGglEUoyIt
MSlGnCcElCyuqkG8SnCebMRNOUXLNCiYOUjKBYmEZQkCYSAhJok0xAIlJVEaoaYMIUWqmmiFldhG
FkCyudZJBZCYiyLUsYSjrI4lNOIxBOi+hJ3U3SsZnUlKMTjKKgNFKEgaaxacOEo05JkYTrsGpEJR
aRnCdQnCQmpBGcJZAEZ1zEmiYgiwG4yFGUCYAwQ4ShSaAYxAEmmEouBNA1IUWiaaIyhYRBihONOE
4pKuyFUJreJShKW0TzokgmglmCllGUIm0yE4TEESZFEiMiYiQQUCAjJUmmMThIKaaI2V2Ci4lkJw
EIsm04gOvSUA1kBAkwGEqb6Cd1N2bICakmQ01KNNE0DcZEWpKoyiTFIhXOIMLIAWKSkQnGUsWmSh
OBMGVyUiMZwJsCDTJCYV2Vk2gaaEpKoAA06AIk4sbThxlEGpEBxJgEJRkDQEZwpxlElCddlKa3iU
oTLCLzuYnDAlmhROE4RMi6U4OIjKJKUIaWQNEmgBBGSCSkCYtZKNkhhXZXYRjOssrsrsjKudScSF
BrUSauWmCYKAEqbqidtVubOMoyzTUsgJVKLAAUk0jKMlUJxJgEBNBMqCasJRlEZIUjNI42RUkpRX
JKpQsrJgyAIlKLHVbUTaaiBBNVEAGnYgBtOVtOAEDjIUZIkmEJRnUQCUJRBSikoONUprfm5wks2n
nTlFytxlEkEs4SiOQEZJwkwkACYrARNMiACYKcZDTSxjKNkwCuyuYoThZOuUKhOMrJRSRpOkmINM
Q0o1KCqyup2QszZxZKxqVtMUoSgABpkZxkRjKJNNEXGQMRWmrGANNDjKJMQOUJSpBUq7KybjIUJw
JNMITgSBjTQkyoxkgaBpgxSBpw00JqRBSiTAIThOogDjOA4yijjONZ016eblGUspRcsnFyuUXLIC
JRlGWYOIyjIITiTBkVKJIGqYJFSQk0SlGSgIUWrJCCucLBQlCx1zjoMEUZRuREqTFDQA01HGUpVZ
XZZbVbmsFLMFK2mJpw00MAUoyIpoalEUk6E4kYyjY1KBOMok67IEmEsZwmKMokq5wsk05SMo1ICH
CcByixgCTLIpoABqQsZxkDThphFxkKLjVggjOEkFZWsoTgSrsihGSrOpR9POTTlbBZOEocouWQEs
4yjmzaZCcJwk0TGEYyiTaYAKkJGgHJA0BFONSQWRnCQq7K6QFjHCwixCQ1SaQTBMapjWNdkEsnGe
a01LNNK5RcJpw00MAUkyLiyUXEUh0oyiRjsNZyR2pM8diXNDajMaiMj1BlWpGaOtGZ6QzQ1ozvQL
RHVEzyuCkuRSXBnWiNUu1FbmiLYJjgTCEkyVVtY2BCcJhFolCcQGqE4lMZw3iQMYwTCVyjIcoTzq
UJQi1pkZxlKQnEckxRnEbTgGhKSpJgxSENEE42SUghKMqjXZXYOMrEhIpEgGKkwi2gABpirnEsnC
ea1JSyTYNSlhJSgUkA0JpkGpURlEbTCMoG11PXlYVouKnVjqZY6nJYVNbCtk3WE3ALHUFpUyx1CW
ulrY6mWlYWqsLSATIBMiEiIMTGiQm2RUmQVrKS9FC0hnNDMkdoYF0XXMOmHMOmzlHWRyDsM42vcL
W7HJSrwzmhmU1hjNbMS3hgXQDnvoC846EjmLphy11WchdeRxId8rgHfDzh6NnmK/VM8pH1rrx8fZ
NPGR9ql8We0ieMPZB449dA8oepDy0vTB5aPqInnZeiJfPr0CjhPuJeJPrEcZ9VHNj1ILgW/HEGC1
yTRxnFRjIxsgQNb35ZFrEympGd6HWZ6FLQXMpLkVO0KnYJWTFiSBDBMQxA3ALFALCuMXFIXOhVoM
waXlRreNGwxqNrwFbzCG54CN757rc8DNzwEbzCG+XPZveAOgYGbngDoLAzoPnM6L5zOi+cHRfMkd
M5odM5gdSXKZ1JclHXfIDsnHDsnGZ2Z8IO6+EzuHEDuHEZ2ZcVp2TkNescmR1Hy2nUOaHTOcHROc
zoPns6BgdbjGzWZmXlYWOtlijIGOgYFdqOYuo5ePHthw4d9nn16EPOQ9MHkAEAYmANMAYADExgAD
EwAGAAxMYmMAaGDAAYJsTAAByiEkmAIYmCYJTCKmEVNlRcFBezMtTMi2hgj0Wc6PTZyjquuSdVnJ
OsHJXYDjrshxTtuOEd1nBO8HCXfVvAPQs86vRuPNx9Mzy56iVeVh6xnkT1xHkX6xJ5U9VE8xL0kT
z53kcN9pLx114py30onPe1GN6QzPQih2BW5oipxEyJIrgaJZEbnzg6cuSHXOQHXnxg7UuIztnFas
BBpgMBpgAMAABgADAGAMTAGmDQMAGANMGA0MAAYhgxMQ0wBoYAAADFJAxMGANBITGhg0waYAAMAA
YANSEADAabATG0waYAA0xiAaYAwAG0wcXTEwABpgmhgxMEAFGmDTRidSIhJxCRFghkI2BUrWuaGx
HkgMmADAYAAAMBoG0DBDaYMAAGADAAYADEEhMGmAANMABiYDBAwABgCYDQMAYANMACQAAEkAwAYg
kmABIQAwGgYMBMGgkAMAGgYpUBIQ1ACJADAFKLBpgBQxI2hWAgAAA2mAFMTGIGADQAAgBoDyTDNA
AYAAMAAdAOATBgAMAYmANMAYpJgAMQNgMTAENgAAMABiYADATAaBgDQMTCUWNAMTGANoGAMENgAM
GgYAMBSXMrqICQnDBidSLmmNxlXjqarNTQpZTr+h8P7qVMIGmJSUMChoGAMAYmACNDEwGJg0UwAE
wAAAAAAPJNPNAYAADEwGIqQiGDAAGmAAMBgDEwaBgA0EgAaQwYNMGgaaGDAAGmAMTAABoYpIGADj
ITGCYNxYwABjTAaYNAxMYBHg+gdV2IJCImJnJr7RQ1KE1KvCX1m44zqH7rwvt8rAJTh92o4ffqtJ
AAJgxoADEwAAAYIYAwAaKYIaAABiBoDybTzQGAAMABiYAxUwIGANMBMYAwAYhgAwGmhtMYIYmDQM
GJoJCYAA0waBpoG0AwABgA0DAGDAAYAxMAdAEMUgAAGAOkMBpjABoGKQNBweX7R2eIPbleP9dJSs
CHGQIGAmAAMEGgbQMAAQ2gYmNADAAKAAAAEMA8k+czoPnNeg+cHROcR0jnB0Tnh0TnCdI5rXpLnN
Oic8Oic5nROcHRfODovnM6K57N757N5hDe8LNpiDa8Ml2GRmsyM1mRml5w0POGkzkXuksvKWtjql
EyITIskCptMAYmMTAGANMGAAxMQwYNMARIAHFkhMAAlFjEwaBiY00NpEhAwaAmACjTABBoG0DQDA
AAYmAAxAAAARkIYI+duw2rcwrJhByCJISJIWLYIYIAGgZESREJOCLCsLSoLSoLStFxUi50BeZxdL
yhqMyNays1GVGt42azIGwxNNqxC7XhDcYWbTEzaYhNxia7DGG0xo3LEzc8DNzwEdB84Oi+azpHNZ
0nzA6hy2dSXJDrPko7D47OycYO0+Iztvhh3XwXHefn2egfng9FLzgekfmw9JLzLPTnmA9RLyzPUS
8q19VLyYetfkRPXvyBHr348PYHkGevPIh648kz1h5R16s8sz078vI9I/OOPRHnmegOCHdOHI4Yyk
MENWJsEMVDEBgkwTGRYKACGImMiNkRukxiGqAaAEsVIIkgiTCCmFbmECYQJsqLGUlwUlwUlwUFzM
5oRQ7woWgM5oVZzSGc0i5TSJmNLMq1BlNSMy1MyPUGVawyGsXItgmM2C5FsZhe1GM2kYTcrMRuS4
1tDEbQxvb608Ie3rXxp7tp4Q90jw57eJ4o9oHjD2SPHHsEeRPWqPKL1cTy56crzK9Kjzh6JR549B
E4L7YcRdpHHOsjlrppeeBmDQSTs3Ki51SpqEmUADQkbi5RxCSQNxBgU0CA0MbIliIDZFSFgTUkXJ
AmLFtokwQ2RG1ixkWAiSESBJgm0AMQwRIIgxDBEgi2CJIiSiAwiSCLYJTCJNERyWBOIhgmMQxENK
JgmMfpPN98xVlM9tfovK+nvhMrCwrCwqC0qC0pC9UhcUhcUBeUBoVAaDOGgzhoM4aDOGgzB4xoxp
gWTuhZ6Tq8m28xRuEpWgKC4KVeFDuZQr2Zy91nL3Gc0IpLmZzQFDuClXhQXhndwULQVQXMoLyKC8
KC8KC5lBeFBeikuFpLhKVdIzu1rSXJKXaLUWoqdjSkuRWWMpdgVlgVOwKlaitWhUWhWrRaXYFZaF
JayksZU/RYo5R0pHLO3Wck6Gk4zvtMcetzBEkIGNxCbrC10stlQF7zhpMrNUsYbHiRvMIdA57Og+
ajpvmo6j5TTqnKivXOMHZfFRW08VtSq+3Z6KvJP1is8merDyh6pHlj1KPLnpw8w/TB5h+lDzR6QP
Nr0gebPRh5w9Ejzx6BHAO+jgndRwzuBwzuROKu2jjHZRx12Y2ch9UOSdUXlnTDmHTRzTpI5y6KOe
bgxGsMi1hlWoMpqRmNKM5ezO7kUl0SsmJAkiLGIaUTATBDBDBDCJIIthsWQi+eUOg+cjRowB0qsI
bs9EiI1SJAk2kRhFyFiMiJNCUioqYJSCJIIuSIkgiTSJSCBZFa3IKy1GNp4rcZr2fRed9EgY51pd
DLjPeMTAEM595pEiRkuLSm4AAMukZkuLQAAAAAAWeg3ldohghoAAKrhDBDrJgxDqLCiwmOIwYhgh
ggABiGCBiGCGESQRJBFTCtWhUrgqVwULQGdaQzLUGVawyLYGM2BjW0MRtDEbQwLoCc9dEOcdEOad
IOYuoHKOqHKOqjlrqhyV1w5B1w467IcU7QcRdwOGu6HAO+Hnj0IedXow8A1JSUZV2vQ+d9FHOvq3
GG2FhDFtpF1KbgTRy9uLaZp13FKaKuvzOmMAwXX+fOobMJWq+gU57YGjXwu8MAXN6XOLaqNJNZ+o
YYCLb7ucX0pF2jFaRWfQb5RkEJo4/Ry3EDPIt1RxlurBtLwDPC3nE+lxNRbbniXozm6/mdIYAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAmAAAAAAAAAJh8+knRKLO16PzXpYYgYgaAGgYh
BMCMhRMITAYgdcwcJBVDQEHIKb0DEDptCrPtCi8Ch3AZ9IZjSGaOsOdfqCMgAAyLYHNt2M51mwMU
tbAAhRqDBdpDm6dAZOd3MRV00wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAD59JKpCK0W50mt45JreRGx4mbHia7HiSbnhR0DnyN75wdF80Ok+YHTfLDqPlB1XyQ6xyGdd8
cOy+KHbfDZ3HwRe+cFJ6A8+HoH58PQHnw9A/Pi+hfnQ9E/OB6N+cD0b82j0z8yHpjzJHpzzAenPM
B6d+XLPUnliX1R5UPVPygesfkg9aeSD1p5IT1r8gL69+QR7B+PR7F+ND2R40PZPxgezPGB7N+Mie
1PFB7U8Uz2h4oPavxLPani0e1PFh7Q8WHtDxgezPGM9keOR7I8az2J45nsDyAevPIM9ceRD1x5EP
XHkWetPJM9YeTkeqPKkeqPKyPUHmA9OeYDjNGqNOrASOTrS2EoExghulFTiUJVo51zIyjaVtiqNl
ZZCVaTjOtXIiMkiLlFE5QE0DEKxCMUlk4SIikImki2lVldhBJk4NiUgRIIk4kSSE42ESQQJoiNkS
LG5RAUyKUhFkSI4jYDcZESQRlCZCUJgpIEwcCUJAMjIaQAAAA4gwABDAUaBiEYgGhWIGIRiBiCSA
BA0BnEXTcWWJpJRBJqISlWyaiiyKCyCCcUDsqEk4C2RiJZBBZGIspQRY62TrTS6lpZwQOytDsrRK
UALaQm6xL41ltqqcOyljsqBzrC+pIsdYWKuRIgh20ssdUyzO0mnPZUrEE7KWDjYQnCRG2oLa3AnF
BOUESspAsrIVtUiRAqaiDlGMMABMBAwABDEDEDAATViEaESExAA0WsTkBgmmCFQDATjMBrQxFgEk
o2iVE0RcbiociMo2lJbIobZEcysYItrETCAhGIViABgTiRc0RJhXOFxSADQDQMQjHNYImQGyJKIE
4A7YEABi1mZFhUwE3EbJlQAwsKwkRAAkhElAWVgpqotMAIAQxMQAADEwAAABANMAAAAAGAAhoYmg
YAADBEkIJRYJgmgYBmA3oGiYGZcq2koOI512EZqA7awtrtQQhMjKuY4OKWRcVtFEg1aVBYVtMU3E
k4FTrmQ67UCheZycALahq2oCaLaZyKrarQSVTpuoGBJdTbAQmpOEyDVpXaQFOuYppFYAADtpuGRZ
VamVyahR00VECAAYgaAAAAAAAAAAAABiBgAAAAAAAAA0AwABiGhiYmACBggaDOB6aBMmBjI5sqBk
WTIDQFtRNRBiYi1FYAADEDQA0DTQAAAAMaQAAxANA0A0ANAxAAwTQ0ANMAAAAEMTATAAAAAAAYIc
oBIQCZDAAABMBMTAAAAAEMABAwStxaMQNNA0A0wAAABMAAaagAmmgAAA0AAGcD10pIJgYzZCdcO6
MCddkAjOC2V2VpoHUOMoFsJ1kAATQxAwQxAxAxMBAxMAAAAAAAAAAAAAAAAAABoGIGIGmCaBgAJg
AAAAAJg0QDQwAABoAAYgYgYACABiBqgAARiBgAAAMAATBDAAUAABBpqAAAACA0AC5wPag0SaPPLQ
QxA5QCyCC2AgcQJRCSQDQNAAMQ0AAAINCtAAADQxAxAxAxAxAADBDEDEwAAAAAAGgAAGgaYJoGgG
JgCGBABTAgAAAAAAGhiAAAaAAABkWADAAAAAYAKAIAhgKDQxA0CMAAFAEAAAM4zo0gIk0/PNylnz
JuQsFYEFdnN+KdRJ2wKhoAAAAAdhAnG2JTOc6zl4Uxvqi2rRmFdVcFF9qZVPQuQ20lBOIjXWZzTY
Y9Ga4qFMlVdSM35ipaIlJpDNpzdAwKV5nJ1hbVMSvqIrSGZiG47TGO0pWmBUacpOO3IIujCnGRSS
vM2iuRAlMqhdSW16soTaK2mINRlatKnZIodtJJasqhJFiYVlkyqN0CKsmlDurIl0FgWVgAFjaUkr
jPOF5Q1aVFsTKg6KNBIDylqrMywra2W5prbnQllaC+WdDuoBzgE62i+lBfSBprqC23KVoWcNedKN
OdMuzsLbsgk9WNGhQgsRheUtNcclq1XVaTLfCBFSDTWVlpUzRKuJTfRIV1F5PNOsuhKs34lIvhBE
AA0Z7SvdikWVuJrxMLqJxi2NYXlNhDXkVSjKcTjXE0Z7Ki+mUV1mSQgQ7KppDZjms1ALaJJLqnEt
hELipk50ourQs7swTdTLo1hZW4owC50Ba6gV1TWNtYlirZmYdNQwYHjAVuVdriQWiBUrqhrTSQCR
GyUSo0VlZdUOOvIBcyi2voGBPcZKr6AAAAABoAAQAVpoAAaBiBiYhoAYmgkkwAAAAAaYCYmgYAAA
AAEDENNADAAAAaYCFAEYmAmAMQ0DBQAAEAAGgYADVAwTSMBQGIaAEjYAAoAACABQmuswIAPFboxG
F1JqIEJllcajfnrY9GFmrJLUQKrKsx6M8XU3Uppz6JrkvovKNOXQZmgGIaaBpiYgaYAhgxCY0AAA
AAANMEAxANAxMAAaBoBiBiYJoYnA0AAAANAxMAAAGgAGIAYA0A0ADAAUAAGIBGAACjTBMRMAacoB
QmAmIgY0ADBMS52HZBoADnAasMogaAAAAAekyoYhoGgAYgBoYKcykAAAAAGIAZNlYAACYhiAARhY
tZdSA5FbegzJgAA0A0DL6iIAAA51jExAAwEAMCAAGmJgAMRbUMBQmkimAK4qHcUE4iYKDQDBNMBS
EDQC4oLIABA7a1RfVUS1JWTmtICNMAsrBMWgDtypJiQc9v04NHmhVZEsVlKTp1023U3VxDXz9Bm2
Y9dZ69FEIAAAADVl1Jm6HO2rGq+NPNXsCFsTP0OXtTHbVslqnl3pVm3VrKjRErLMZolPCS1S1Jgz
7sK68urMaKNOQ6NJlicNFFSvVpmDYUZejzTrcvTkNULaygEar83SK8s4y006c1jQzpc7VjNUN2cr
jHeczoc/UZ782mWFkJjovyJdHqcdd+PZWVWx0mHRm0mTZk1pXRpzDBG/DpqW2u+kasRVoy3JTbVZ
ELaraacCyjXkjbjtpW5SLISALIRWptIaM2hXVfAln05UpE+2IASlHwOyo8q9WQNEKg05kzbnqQ7a
kPZiC6pBqyMNeQB6MrC2kR7MTXXRWG3PUGsyKjfgYaMzgvzhpqrE6GWkXTnTL87DTdgDTXUzo85B
pzoL6okaLcQaZ43WiWUNuIDTmA1rKC2ZAt1YGdXNjRqzICSDXmiGyGZmhUCvTlIlEAvoZqpqEsnS
GhUC32ZAdtQGnKy2oAaE0UAumuoLCtivpAacFlYkyKJxAaTLIIJODCcCgTCcCJEQcWioZ35QMjGc
Oe6tXL1+dzac+0xzLQxaIxUuljJXQiZi3TWE2QiePoYSK6eEsmMxFmizG9TV0aMR0aawmrgx645C
JtwwwAAAENxYNAAA0DEwE4BFMTAAGiBoBiGIpggaBiYAABA0DEwAAAYmAAAANDE1AEGgYCsQjEDT
FAAARgKNEjABpgAAAmmMTENDBUxMBOEMqoH34TAjCcOfRrzx8q9GXQWxzxNNF2U00RI2GXRVPS5U
yevDWdzkKqN2WtmuWXQZevyiw6fOqXt8Wyk34LqTfLPIy7cW0jiZAIGIBoGIGAAAAAAAA0ANAxAx
EMABOgAAAaAaYAAAAEDQMAAAAGJg0ADVNNAAAAYKAIADAACBpgAAFMCAChpgBAADFTBDQwAAAACp
tfQ8waIwsr5tgzyqGREZSJwE5a0xCksRsiNQJoaBBpgDVG1mEENSiMAE7zONAtllmFKSoagCQgAB
iUpEDpc0BbDINkRgJghoZdIzkogTiIABdE57cQcoDEwNtBSOyKydYyewwF9RFyuM735ihzFrdulM
S1i5GtJnJbDEt+AAEZ0MRAvZndqisN9uAd6UOy0qq15iLsQ4a8kWFd5Vdm1GXTROoLRnAV5Gno4A
eiqKAPp+bGohXbTza20ajx1NqBF12CpmjU4TSOjNElZjulXP2YgNmMDTQRDYY92Ho1h6nJ6kPHdC
rYKwm4SOX2uL0jAR3lVnO6SGPfz17NN3Oi6ufNNenNA16M2gu4HR55uw9LnLqo1YU3RumRTyl1+X
QOucCVckdHLbzTbbjujlKzRWPvcHsGDSmW8jsccYB0sG2g6GjnqMfZ5O85uyi+sO/PXHTzZbDN05
ZDdjuRh3YdZdi1QWzRg1kaoTJ4N2MADZk1Um6FcSnq88KNOe8z31RLbslqW5tWc6vHskTqCDVRIz
6ab6rlRoChxHozTLkUls4VEE19LyYBGm6nm27Kjw1ZGKJxQXUiLo1g76Ak6wnAIsrAnAAmoUX0AT
gGihBopiEraCHbSWWKBKWVgrqXVrpIsrAkRCxQDTVWG3PUE4gSnUixRCREJwAtjALIxCZAG4g2gZ
FkogDTAAABicAIbQMTBoAAYmoAMTAAaBGIG0DQDE1BMaCQYqbiDABoGAAANMAQwAAAABr6XkDEhR
fn5vTfdTl8NJ6dxw9xIsw1bzG98Dnd7naDkXVds4u6YTw5+kYn0awzb8Jkvu2HOot3xnxacwCYCB
ghoBoAaCSQDQMQNAMAAAAAAGgYIYACBpgAAAAAAA0QxOhogAGAAANMAAAVgA0DEwaAaaAmAEDTtT
AAJBpgAAFAANMAAaYAA0DQA0DEwTRIa+n4sBIZtObm9N2jkvn3aUoezEHXw5hOxVzUbur5wXVVUi
erEzrY8qOxVzA69PPDVu4rNzwI6/LioAAAAAAAAAAAAAAAAAAAAGgsraGJgJgAAAADEDAgEUwQxA
wBiBgQNAxMABiYNCjTAAGgYmAAMUjAUApiYAA0SMABOgAABiYNAxMAAAATAAAEsi39PwBtKcuvJz
e11uXpc++aGwxGqkrOpzSN9+5OHbXvXnB0TnGzOVrr5DG0DV2kwF2kwG3OVHRzmY0kZrjQYRdM5p
rRlOryxX9AOONAbNJynp2HKN0CqHbwGB2WkqlpMBd0Dku2kADVm2wMpLVGM6cTnrbqOQbtRx7deO
pU3UgADCATGIGJgAMTAAGgYmoAMTgaYmKmBDEDEWMCGIpgABABTQAwBoViEbQDQMQAAxAxMsJL6n
MmMrx7cXN6vpY+1zenB6WDokcPRzl1VlRfJXnnurz+8ec7fM1rbnszJ0Of1eVGQA2qCM/pvM9cuq
qpMnSsgBeHE6ePsr53tcjoHM9B5/oJtz28kfSnUcoA6SrUt8s+qyLVAtF3PHpvzRh3VbljHm9dMO
G2qgQS38+ay6NmSNPI33JLmy6Joy4hdXM0Z0QFMTACAAYAADEDAGJgAACsQjExgAADQMCACmJwAD
AtAIEywcQkIGADQMAAAAAAAAYF6T+pygmRwb+fzerCfN6xVlxlAEwEACciIAb80jXXkrENAAAAAB
0OeAdLDFaLylTgNDENDABMAQMW4xAAJg0DJIQkSQAAo7aRiYHStjkkWjF115J0ImA7XHInf4KA2R
EwZ0jmlgtZYysmyt9bmlZaFR0tRxC2BCVuo576mAqOhzwOniKbNdhzQAct5kp6ETC9V5zjXmInSw
pWb6Bq8MJfcYepzuscvRnmss++oyPdiRPpYDRVaGN67TMM+pxoYseb0ubze0uryevzevK9B57tLn
x9OKHMp7BUaKjmeh871o52d761Zapjh6HyZ0behmjj9HBqMvR4vaKsfQCfN35iEJi4u1xuucjfz+
wZ465SS41W6uhhhrlssyaDkV7c1lXY4/pIxUW2kuXX0jlAq69vPIh2cGxcWjFtFi28k9H5/q8g62
PbkNJKRz+rw/RnAql3jHz5h2vN9/gnV5PW5Ceg5vR5y4xBPtcPtnH9N53rmnjuZu4Hb451OR2uad
WrlI6VT2nK7/AJ/oHNsu6Zz9maBqzW80to6/OOmsN5zWg6nMt3iKbyedZCV1+VNKKFvooSdB5rTJ
3uPURV2xZc56S5ZqUlHbhNFCZueML9GBF4j6vG2msOb1OVzeuszHN7Hb4ijs8/MHZx4WdW7iIfW4
4atPNQ9GYNlvODp38UizRjQdHnB0cdQd/k5g6s+OGqulD7XEZ18GcO/xKw7WPCLqniDTo5whryBu
zVB2+PAAAfX5CH1uSLollDTmA6OCJAAW3ZAvpQaNXNZrqoE0UIqyAEpVOBoHKDJERZIAAGgG0DQD
aBiBiYxAwQxMGgbRA0UxCMAYgYgYgbQDTAAAAABpgAABqGfV4kxrHldfkc3s9mX03L7ea63NoO7w
u5wgQAIRiFYgYgAAEDEADgEEkAIAEwAAAAAAAAAYgAaBiBiBiBgAANBKADEDEINMTQMEMRTAACGJ
iaBtAxNRoRgKNMAAAQAGIGArAACGJgADToARiBiYCYAAJjEwAAAGgYIkJggNjUvq8SGIuP2eNze/
f0+b28vvZineXciUQQhiAAAAAATBAQAAAAAAAIAAABiABgpRBpgmCAAAYAAADEBKAA0DEDBDEwAA
EMBACgAYnA0KxAwEGhWJjEDEDABoGADQMAAYmENNA0DaAAGJ2AANA0AAwamRJ1gMAmEDoowB0Dnk
rS4H9XgG1KcTucPw95C18vvlFfFB3+CIARdAgCGOIxMSaGgAAGnAutA5pqygmyIWFYdI5rvzgdLK
VR7nIKR7DEPUZDUjMdDAqOngNOfWRga7Jx7quwcUfTJcv1XlBAIHpeYvOW/KVm/acOW/ccA6/NKj
0HIMx38hywDbRstOSaukcRdfkDFaOv1Hnig78TgvrXnC0L0J5d39A5HQndE+L0azC/Rcg2VV6zFq
5XUJc/czlW9DQcqi/PTsqZ2OX0uXGqru5DnO7ZXJ63F6ic7bz+sY9ma1bcWrlmqn0PCNtHQpTl78
OwzdLkdAwdXL0Vwsf1PnAAcLu8Pw9593gdzk6eH2+JtOp57fzwTDt2YaYy9rn9Uza5RlhwujnsxG
rLSHrjGG0xM6Zrx5tC9PLXpiUoIy3wrLL40GXF0Mddng9jjx6vm7KSXR40jN1vOdg5Xc852whSzt
YbOQK3VpPOdznTMPZ4vpjz5v5x0+T1OWAB3OH0Li/l6dxDM6F3UwypHTswnRpniNst/Fjm2R6NR7
3l2R6uO408Hoc8biztcfdUej4+eyI9vz9ZX2+NeUdLBTXS08ScdvllJ6Lj45m/XyKg6vJddO3kRj
q6eEjpc5OmIO5y87O1jwh0zmCS6vJRbWCvbhDqZsjTVCgLNGQNWUB6MwOUCOk0fV+ewA4fd4Xh73
drzxy9O2mhxplkmb1kRX2eIFvW4Yb+p53TL0OMkm7GkG7CBsxg9GVhZWG/LSLdfjENmMGkGzMRGr
Kiwgh20gTgDEDcQYhWIGBAgBiDfhDuU8kGmhkQkIGIGIGIN+EAEIxMGhRoG4sYgbixghiBuINoGI
G0iQgYmMQMQjAGIViCQgYhAAYmDQMAGgYANAxMBEdQkvq/PTYkeL3eB4dCerpcvTwuq+VHc4/e87
HdwapFOuG5fL9/mTDmd/EYYet4R08K0xwjqdSvLdDL2Sni9/GZKvV8OOvzqthx47+seX9f47uHE7
fE9QcvN2+GdnF2uCVT9BWeSQrBkiDRA0AANxBiapoGAAAADQAADQMQMTE0IxCsQMBGIUAGAMQMTA
AYmAANAxAxBIQMENxYwSNoVoYNAxCMQAANBJAMAaAYgkRYwAATrDPqfPAB+f7/A8Pd+y8tVydU9l
Osr5Xf48R7tHLXt0cvcV6uF0Dpedvzp1MFIvo3x6Tf1/N0m/q+fqPXeQupN+StHp6uDE7erzcS3r
8RE+1wker4vPI9Ni4weiq4SGgpppACUAAAAAaAAGmCYAAAAmAAAAAAAAAAMTJEW4gDA1wM5dAiSt
KA2mIeoyO2kkkDEwAGJgADQDAAQ2gYEApCaBiKYCMQMESEDABoCUWDRHYGfU+cAWHn/Q+f8AHoiu
rzOTqidjkQgBHR54iUykLik1ZQDXGQ9hgXz0ut3I8ZLq9U8ken8xYgmQOxjXGa95xS3rnDNsznvQ
FmL0HAg3X9s8eaOyedNvRMfN9d5ELqdJfz/X8o4p2c5zjVlDv8H0xmw6A2cbrcc7fL28YgmgmelP
MdLTuPN1dHnBOGw7fPhqjkHWvrznreH1jgdjjWHd8/v6ccjN6fy1eq811PPHY1YOjELIY1d1vJN+
qpj4Hd4IDSAAAUxAwAAGBABQ0QwQ2gYAAUNAwEAAAGJgADTBxcdsZ9T5qaaHnvRed8ejtcjq4+Tr
6bnxJer5/wBb5Y9P5b0HDPR0PkxPscr0Bm8n0+ZRqy6jftm4852vP+gOT6bzHog8n63yIaM99dmz
diinldLjmy3n+jK61yjscL0XnD0HnPWcU6NdeyKLeL2KxbuRvL/Oek80GjPYduyOIs6fmfQnmQB+
h871zBv08c2c6VR3OXvoOeAdPq+Y6J0qORXG3BtxU7aQ9Tm4kI71/myun2PJB1tXngn3OCHouBAO
7xIM7z4Djq9DzQaO75sXoa+IJ2OQg7PGAAAAGJgAAFMTBoGJwAA0wEwaBgABTEIxAxA2gYmAEd0D
6nzQBl+d9D57y6Tu+eOTr23cwjt8eCO5m5guyioOl2vJ9CPQ+N01EHdnTpYK2FtQPTlCysYWUyXt
8RAAImhWIGJg51gIhs7xwHdRQG+MB6TnnMOrEnz59Y8+S9GeafSuOOvR8UznpOOZF6Gs4S63SPLl
tQAACGAAmDiDEDEwAAAABiCSTBoAAYnAAA0DQNoGgBp0AA0DAACGCJIAaKYnDEUxAxNAAAAaYxB3
gf0vlpjF570Pn/PotuXrOPs8/wAPT3V81H23l4yP2Hnjn2+kxnn7+j6E8Uex8Sep8t6bzQgEl1N+
2XyHS07DjVdnzler8z2vOx17d3ljoYetsOVz9mOjdh65u4Po+NHoOBv8+AizqJwl6dN2cs4Hp/MH
Wz9fzZ7Lh7hXVLAQuneElIz8r0HDTTHt+aXtq4ink9G85/L2bLOOrJFL6/HA9B58AABDAAAABiBg
hgABDEDAoAAHAIGAMEDQMRTABoGADQMAGgYmgAMTUE0YA0mAIYAwDvjPpfKGmp530XnfL309bz9n
J29To8DNL1uT0+Qej81qxnp+JCg7vQ8tI6HPcD1Xlb8xUAncv85YujsebD0fnUHrvNZg9D54B+z8
9OOQNWPt8QX0/FxEdmrlgCLJ+i82S7dPJDp86IdLnIOocsLerxQn0OYGhUBvz0B1ceYLihjvzgTr
BiCSTAATQMTAQDQDQMAABgAAAAMAAACAAGgYmAA0AwKBMYgYASiokAAFDQjELJAjAATGIGAAI9Ew
+j8tMEfnPRed8+ila9XJ28o9DwCBdSoXSM6v0nPHrjJbbnNeL1PliICMv7S+eL95yTv8kzP0fOjm
ruROKHYrjAINal1czv8AHig9XkPPPv8AOMPreH1DzfR50q7XC9D5+O5wuvSc0AAEYhQBABQAAAAG
IAGiAViAABoAGIaAAGmAANME0MAAAAGJwAAAAFNNDAACBhQAAAAQ3F0ADAAagaKYCAAAwAAQekY/
ofLQNDzno/O46LfY+P6/H3cPv0bI1eX28k9b5vr+Yr1fP18COr6Dg9A2eE7GCu75n2fljIAm3t5d
cubVjmef6/H651OJ3OObIO482HYrjDEOpy7Jd2Tv+dO7PmM6vLh0THv4/VOA+lE6Xm+7wzpG7zwk
AAIAKAAAA0AAAAAgDVDAABNAAAwABDAAAAAYAANkQtKjVmAAQwAABiGxDQAA30I5wKmIGIGADQSE
AAMTBogAQadoJgAemQ/ofIGmp5z0fnPP3rtyrm7uhlpUdnkwDVlAvpQWlRF0IBfVAAGJ66VqTA7P
GD0eLlEIRYAxDFQwtqATEMPXR5E2ZqrcmQOryy/P7DyERL7zES6Zhz+l82K2vvnMz9rmGRbbTnHZ
0HnXZ6A83Pb3zyD7XMNHP73EIr02M4x6VnlycQH7E8aenoOAep45HB7PzJhBk+lh9CvnLOh3o8hP
vXHnKPSebr0XmvRcU1ZvW4o5r6Fp5nsb6Dg9aMzDtlhrfxb+hE+R2sRp0ZpHnkOxAAANAMTAAGmA
IkghiBiKYmNAenYd/wAdMFPOek85joyz1er5e7xEvVZ5eFR3uLUp+r8xGLodXAcfd0uicTk6MydD
nbcQMDt0Xejl8pPm+mOVp2xVYdkU84pRs29zkXS5un571RkeqsfOwdw84gs6PpvE+nl4Hd4fqjP5
/wBN5Y9F5b13ljv+c9XwD02rl5Ylnq6lR8x6jy4d7g9o6fFoznZ1X1QYLcdbMlXaPMeg58THHvcw
6/I7nCNu6rGvRx050rzW1D9L5noHXXHoOipcg9L569GMYXeo8vqNXV8xGXvbfJ+gNXnKIHsPOYiz
1XL5Kl9Bd5kO9p8yJt2cZV2ednD0vCoD0nHxkd44LItqgABSEwAAAAAgAoaBgAAMQA0epB9/xgQS
836TzmOnT2PJ9Xl73v5FUtVTjXq/K6M8ekh57QdPd5WRFMs7nDCEDXrT4wdyrkB3jgkdmPKjQDTR
ZiRLu8Br6Pm85ktmFAMRbcQr6vJca1lK6WKkjfTmdSuotKZw1mRTuM5soKTscmIvX1Dz4OkNAzux
wj0HGKD0/AM56HOcZSVIu9JHljq7Tzp6XnHMPWcEwHrcRwD0mg8nb0OqeWv6HTPMHo4rxMvp/Pxn
YWdfL3iXzfR25Tk9rDYYae4WcNd/Cc+XoudLu5WrWQ0+b75xuyM869fdTDHZyF6PM7uYfn/Qcojn
9FFPOghiYAUAAADRDBHq0zv+Mhujzno/OY6cq2+j5e/x69HXLwD1/nTEexznlldpTAX615p1sJnP
R8cjZ6FR5TrzDka+tccrD3+Advi77Tl95XHlq93fPJHd4dLqcztm/j9CETnZA4XpfK+hPNSXsTyB
6ms5PO9xwCvJ3rly2x5xDpHMTo2wlFnnezwq7Obo4Ts8t8ot7D8wWVBR6bzXrIhl38swdbn9E4Ho
1yy3p8ncUUw6YMktcruYYe4sSXaL6ZY5btpn4tndM5h3hF4SNfZ8zVbi09nm57l18hdk5qzdZJZ8
nPPQS87bRd1+NL1bM+BJbMOc7mbmdM02+e2HY5FTO9y8Wde9Xzsyeso4FZEAaAYimIhiYAA4tPVs
O74qY7X5z0XncdM+5w+zy/Rk6nLx+oYU16bYLzu15K6zV6TxHq40+T6cV10aMxpolaU8ZejKZcnS
lnG7vINmuXHLJ4rDo6/Nd8wcvo86l2+LqPQcvna47b8+zH6TzWiqOlzA6PQ88Hoefmoj2MPKQO90
PIs1eh8mHat4UT1HFwlen4+AOphpZs7HmmXZxEuny2b6syj0HFpDv18QCLKOhzwuKZGjNKJqyX1R
OMrDNJABcUl+cbVxSvQcIgDA7kl4Jo7yeZejuHm4+k86RbBE4A5hA7es8ybO9HljqdJfMP0OA5y9
V55Gen86c86bOU0DQDEDAATAEMTT1jJd3xYgw876LzuenPblOf6O7LWpelzkzRXASA0OUblhC+mE
dLCVmy45pLtnDL9BgO5xSK9NxzCdjYebJeoPKqcLGHUjmHrObLxV6a88lLp9k8ktSrMbOgcun0/P
jBD08DzG/pI870r9Zwp9xnO5XoYBx928xnL9GcDv18gz097g2P0XnfVSrgd7Ib+RDqmaPYxHmJdb
uni9OvsnnuvyewX8lzXbj3wifF9JmPOdqfnrJei4voTlx27I5+bdkXfwutz05TjKvUFkpcPM2Umz
iduk0Zsek6XM15zv8fVxzfutRh52zQR5U+sUlV5Hhei5R3fLei4Kdjz3d4J7CnJFeA0IxAxMAAAA
AAK9dKMuz4UWOl570Xns9cfSee73L9LLTtctvmp9FMfZgHm7uh2a8f6rldCW7k7GUT4voCyLyRVD
Zya7Mk4zxjbXP6vK6qPJrynM73n+yefrsr1Dscjqy9TPbTLoo02ScXrcLo1R2uZzD0+Lh6zF6DTw
jpTnSPkaOsZ8/M75Ex1lmmfnTd058U0vZMp5+XtlXAtrsPVeU3r3ORisjsWczCekn5q0jp5hWn0v
kZx1NXDqPXeaoieo5vKD0d3lg6HX8wG+fMDvQ4oek4+MPS8PMhNB1KcTNPV4SWzThEAQxB0c1Aa4
5wNGe1arKrSp2VI0WFZ1OYI6eVc5o2ScpNUASACgAMQxFMQewafZ8FOMh+d9F53PW+35qzn+l33x
ccvoI4OYeq6PhLic8ast9V5CUvruBgD1/H4zPSX+UDR6TybOnfxA9N5yIem5GFHR28EDdgYRZYr6
JFt2QlunmY9GW4oGkAdKSIDa1wnR3HAN1hzTq8wibbzlnp+Uc19HacA7XOMyaQkvQnnn6rgy417L
zBkfotZ5A0X1gun6SPMQ9TmOXh9JlMuX0KOLd1kec7MsxzevfYcpddrn4nU1SWcK/pnmhqu9riS8
H0vA6Z57t8h2dLkegynLj7Hgmc7/AAF7N1OeIa8e45vVo3nmTtBXxs5Xbo3ZIzdXAzdRdzjHRuw2
AhGIVgQAAAAI9i0+34SGIee9DwJ1c56/Uc30/Gr0meXhHsfOmF+xgePLrqyWS9RHmaPX8kz4O9bH
D09mB5fuqBz7+wzNwe3Ms5lnTXm9F4Dl+kpwE+N6ry9kejz+lXoOdux4S1UY6z9XD0DLze4zi4fY
8erOfLnnqHWo8/1+J6Q5noPI+nOFq6AW8iu4hs5XdLMluchvzYy7g+n8xR6bzPoy/jdLnxfj00HW
x6dZz+t5NnS1+d9ESjZySjt3eeOhosxGqinaLkU+gEuL0DWYMRt0Pz51ttPJMwB6WzhZl0dLnVEq
CKej4ioOlkoR67kcxHf2eSJbvReXDd1/NB6LJyA9HycLr0fO5rPV4eEo3dbzaSUQEBTEABA0DQUx
Eexal2fAQFHB7/n89Z3OH3eb6iK3LxOjPOd6gI5Pa8lbqbej5n1MZpafPFfe0cc6eGzSU8zL6Qg+
XIsts86dXo18o7OXn9AT4PZN/ltOaw6PPnXoXwJZvft8vMO3562unyq0lt+N163icxy+jl5pkuxx
QPQeeadzHzw9HzuYHpOXhS7Ohww07+OFtIWLTncO2uKtSSRsrvM6kCC0qPSeeWDAT6G84CGJvvHA
XV5YgkKPrvPxjXpM5wgBnZ6h5F9bpnlZ7+oeeq7vGK10aDK+jqOPr60Y87uv7B5rRRYSz+jzrTyd
szbl5XcMlmtHnOzg9UeNhqyIAAAAgYFJpgBHsWpdnwUAPgd/gTry25Dw+l0cVal6nMQbK6QiAg0E
kgBgjZrl5JLunANsznno+IUL1HNOS5+sPHmqiystrEPcYj2OOXzU+p0zyupdU4cPTcs5p2OemeXo
sBRf1Yy+e7kcJk2dHoHD5HXqrmS9TwE7WSroS8jo8buHD9L5v2BwuH6Hz9Lt8XtJ0+X2OJL0ebRp
OZ7fi7Ty3SzelPOdB8w73m+laXcnt807HNXMNnSnnOV6jynbOX1PMezF53ocI9l5bs+aPaed6Xmy
LQdrr8TYZs+7mmh25jt8bs+cj1PJ40a9P5rvqLeHb3CmfA7irkbNaU8vs8JfQYtPnzr4dHJOj6Hz
2hNXQ8boOnw76AAoE4AQxMABDR7JqXZ8ETjEuD3uE6qfS+d7fN9Sjm9vzB6nyXqPMnpL/M+ijzOj
f3K8hp2ehjzXN9p5c6PL7Qsq7MEC182uqKUc/PfRZPfyO8c/s+U7ByvRec7Zz+Tqy6hsx6z02F3Z
vlvTeZ9LXm/Q8fshKvz8ew4/Fvrfy/U8s6FNeqOR2PPegIafN9E3008kso9BybI+gyc2XF6Lzuqy
XX830l6vmLqUXY4867UuFKL9/FDrdjyAu/ChDTmK6+LKR6jkc4XpY6Um/seYa36+YB0OeHf4Ak3Y
kGrMhQAmVkXVxCREJwAAAAGgAAAQxEMCkAAAAAAAAmiGgAaGJiYHsnGXX8FDQ+F3eFOqHf8AMX+H
1O15npcw9R5nXkPY8equM12Asv8AU+PlL6LLyaz12bzQej0+UC/0nlA6xykel4uQPT8nnonswFh1
eUHQ5zBSiF5SDm7jLKUyhb8IgQwYFuyOaxWNDUTBEkiJyWs6fYjyhq7R5o9H5ykNIJgmCoYIaQAE
MEDVEojQAAAEJpiABoHFipiQaIAaoYJSQDQAANAAJgAAJoaAAABDAEMhMKAIAAAPYyT6/giBXw+5
w51c16/R+P0vIHpKZeC/V8OMB7jMePLbrMkrfRLwsnYhFOX0Ljha+tWeZ6sfR1xcOroxR5/rV1q5
Hf4xi3dPdHK242T4Hp/M2RGaJ9KqNXa8t3c3ndjjXlXKlv1OcvT+ZFJSOz04as3xa1ZtZXo/O9OX
ZDo+alt6nU8ydvDv4Jn9Z5roHPjv7JzfPdjj2ICxNoABAAAIbEAJgomgTZEGIAAIAAAENAAAAJoG
gAcCBQAAABiABNAAAAAAANNAAAMQEAAJleycZ9XwEmpXxO3xXXHp83t8/wBN4uVE7nE6dkdOlOXl
drzN+pf3fI+jieDdwivtX842c/VpOJ1vO9quD6jy3WEul5w62QoPRcujpRXz57q2+W1ZLEwrfTmI
9TVw3Lp7nlInoPPBYmCAOu3zM5L0eaxDs8Vnd4QL1+QBs18hmjTzkPbhDVjYIHYlJCGCGQgAAAAQ
AJgmCoAAEAFTThDQAANCGCGCGCBkWAJgmEoAAIAATBDATBMBAAME0xDBDBDD2Moy6vgJMlON2ePO
vLdZ0PH6fnep0eEnV4tYbY4ypwBCysW2tBKIAAA0NAjAAAAABkWAAxDBMAAEMBMAABgiSEMEMEME
MEMEMItgkwRJCGgUiIjBDCI2RYxJlAEqARDBDFQ0AwSYIYRJIRIIjBDBKRCUkIYCYqGCAAAQwQyB
MEMsQNUMhAAAAM9hJS6fgxGg5HX5E6+t2+L2+b6i8X7Txtzzoyj6RDBDBDUAwQwBgJioZYmAA4Q0
AMEwTGRYANkRghiAMiSBDBMBDBDBDBDZEYIYJSCJIWJIIkgiNkVNESSRNixGCU0RJBAkESYQJhAk
ECQRJMgSlVZY4qLgpV7szmkMxrDIa5JiN7MEeizmnUmnIXYZxjtNOKdsThneLOCeijHAj6SVnmj0
k18w/STZ8w/Ryt80/Rys80elmlzsnfnUvQZ9IcP0HAnV0u3xO149x4z2fjbOcpR9IhgmMiNkWMQ2
RJAhghgiQkWwRIIkgi2CUmQcgiSBEgiTCBZJKlezOaQzmoTKbJGE6LOadSSck7Ejivus4K9BM86v
SyPMr1STy79S08q/Vh5WXqGnln6kPLP1TTyr9S08tP0wnmn6SJ56ffScJ9ws4suu2eSdZJzZbwwG
13OV6Umd3Jmt2qypXCVqTSLGkJCSSSWRCZBoIyIDJVrOM4q6yyko2RBqVQsigarqTlXbKM4Wscaa
ZZ0UPn2mAcDvcN77Ot4946/WeUHZjjus1Oa+nKTlPrWJxV3ZHAfoJnnT0szy56mSeWfqpHlH6sTy
z9RYnlH6qR5R+rE8zL0jjzk+8zhy7aOPLrCc06SMM9gZXqiZ5XpKZXIrchExwpJDiywY5qINE2gI
uwAAYRYImAACBpEkhElUSSZQxIsABDUYlhUFjrhVxSLaoyRMYk4ibJohcFcbktanKykvlN54aUUR
0i0q8WhaiXAdIXkz6qXlvphz49GMvPW9W5TSloLkUympYySGQS71Ja4USQRmli5xUlEJOJK2mIBR
oGJiABiAGIZCaYDGhMEMExDTFQyBJEnCSggYhZJghigOWJJINNqk0CQJpqEbpGdbAyLY1xPYRkjt
DFLWLkeoXMtQZHqJcxpDNHVkpNFkpVBMqS6JZUbjAG5Ykb1gVdBc+B0jlxOquSk68eQjsLjRXtLi
xTtriRO6uFE70eEl7q4STvLhRO9HhJe7HiI7hw4ncXER248ZHYjyw6cMAbDIzRGkLYxYhuPUuJvh
kDmkADTVExYkiWDm1rchYkpS0rQGda2uV6UudaRc8rmtBpIzPQLQXktDuCkuFqLQqlMItgAKCBgA
AAAAAKg0EVEzHqpuKJqExlFw1TMsRAsREbrZYKJMgExMABed9FxrMUdj3nCugzmrphyjrM452Q4q
7cjhHdI4Uu6HDfbZw322cN9xHFfZDjPss4x2GvHfXI5B2A5D6zOQ+tE5b6Ic57gwm0Ma2MxvXMwm
5GJ7EZFtDFLbEyPUjMakdAxI2mJVuWFG9c+J0lzInUOSjrrjxOycUOycRV2ziI7Zww7a4gdxcMO1
fwOll14unNm8krNcJ1TVpAJlUyRQF5UFpmvJFUyFmTUTEDAAAABcnr5hc7FPKv0XnfT6OuUBXQkR
JRIykwhJhCcCFwytWBCNoCYDTDPfQgJ0NRJqsJpzKi0KS1ECaEmgTiScKy+MAkQCbpiaVmiaY5IG
18+B0zlROscaB3X54PSHmYnpl5dHqI+XiepXlYnq15SB6xeTR6teTieuj5KJ66PlJR6eHnpHYOw/
TPFO1I4b7jXhz7Mjiy7Djjz6sjkvrBy5dNnMfTDnPouOdLezny2oxy1NcmiU4ng3qOfbrLHTcpqi
2QVWgUu0IKwKrRlM5hRawZEJEEWFaLSlF5nY6LEPTjgb48+s6px6zuPgQPQnnIHpV5iB6o8pA9av
IxPYLx0T2R4qB7c8LE92vCRPdVeKmnro+VZ6aHn2dyHIkdKGJmqNUiUJtaY6pxgh1WceHdkefj6a
R5RewZ42HtbI8K/cB4mXsg8fP1YeWfponnp95S8R9hHMluZkNSMc74isrRfPIja+dGOxLiQPRPzi
PSPzIemj5pno4+dD2Sb9cxJIQ0MSJEUTdQWFaLXQi8zovljgb1zInVfGrrvHnqz0h5hR6o8nWevP
HRPZLxcD2x4WJ7teDS+7h4Zp7WvyMj1NfnHHehxpL1IYWa4UyJ1zklNW2Zzq+uziQ9DOXzJ6eJ5q
fow8/LuBxpdWJzpdBRhnqiUu1EZEScqYmqzAjpWcdHafDUd44Ad1cMO5Dih2TjB11yResuSzpnMD
pR54b1hDbHIGlZwvjWiZAiSCgFDEwAAAAQwQwAQxMAEDaAEwBA0A0DQAJnYh5mXtn0EeGzsR5Uo3
wyM0RrkRhdIyx3BzodeUcSPoBeBPuhxH2UnJn0iOdPajJPQLS7EDikulmF1ywqOkcwOouWHUhz0d
E54dBYEb1iDYYw0mYL1Si9VBMiEkkNoGgGIG4gxAwQwQ3FgIJRAYACcAgYgAAEDcWoAMQDQNAMEM
TAENNDBDTIAAcWAANADQAACGAAAAAIGAAMiDBNFhie5sWNmoySNJkZoKEaI0stVaLCBE1EGJgAMR
TEwE4AKYgYgbQNBEkgYmDRTAgEiQACYNAxA0ANAxAxAxMABiAAGACaGAAIYiGAAIYgBhEYAANAwS
jTAAEwBMABoYIBicIABoaAYgaGJiAAGmAAgAaBghoAAAAzAbgAAACYxMBMaTBoGIGIGBABQ0waBo
BiIYgYANAxA3FgADQAA0AxAxAwAaBiBoYCYAiQgYECaGJ0ApWAAhGANAAAAKmhGgGJgMEAoJgAgJ
jEKNAwAQDaAABAMQMTArqNJXZAAAAAACGANAAAAAJmUa3BoGgBpgJjQAADQAEMTGhEkAxMBMABoB
gAAMABA2gZFjEAADAAAEwABMBoGIGAAAOMoAFAAEwAAAAEGhQBGkEkCoYIObZuniqOoRctawTs6L
zaSmFNZ0I1RC7ForZl1cmWyenMm/PZUV6cW0wdTkdYw59NVa80iLNOXQc3Rl7ByOrVZLMi4YACAa
YAhiYAgGhgGRo3JCIAAAGIVoBgIAhiYADEDAABQEkgBMQxMGgYgYANMQANAxCsBAEMAAAaBiBiag
EDTAQSSY0AAA0IAKAAADQNAjEKxAuZ1M1knisrcnVGHRRsqndzelGAcqUa2WOdR0MG6Mc7WZDp0R
mY+hzeocrq8rprmU6krtdFaNWTZHJ6GDqHL349heBmgANMTQDQAA0AAhgGUDUaAGgGmoAAAAI0AA
DAUEwAQaBiagAAIAwAUAGIGgQExiBiYAKADBDQA0IxMAIYmoAACDQoAAANANCMAAaoaAAAAExZtX
Ns24p11qbrirbj3nL6fN3maQymOmBbm2UGzmdOlaqZyHokzlara0ydSu2XPRtCPO6oYdwGGrpM5v
QkINErEDBDBA0DQAANDAizK0bjEQwFAEaAGAAKAAAjQKNAxMQ0khIkk1GhGAAmNDUEDEIMAABxYx
NRMQEDE1YgYhBogaFY0NADQMTATATABABQAABggABphCaKJ2NBNKMBiAAQaYJipoGAAIYEAAAAAA
0JgAAAAmhiBgAAAIYAAhxYZgNQBiBqCEYCtADTEDE0wEwAAAAAGAAAANCMTUEDAAAABiEYmoANAA
OEDAABNABRNDaBiYAAANAAwTTEAMQMAAAAGIBpwgVMAYgBA2gaYJpwAhgA4sYmCAYmAAAAIAYAIA
BgCaAGAgAaAGIaMwGoMSgAADTBNMBMGgYgTAYgGAAAAAAwAEwAAAAAaBiYADQA0DE4BMGgYgaaGJ
gAJpgDAEMATAAAABoGmAmAJiABpgJwAAAAFJgAEAFDQMQMCAEMAAAcWMAEAAwABNAAMQAANAAACG
0wAMpGWgAJiGAAANMABxGAAAAAg0DE1BMAAAGJgAAAAQAAADRTBDAkAFABgAAAAxMBAOMgAAAYgY
IYmAACYpIBoACGgoAGIhoAadAEAAhgNMBA0wTAABpghkIkhEmRJhAmFZYFbmEFYFZNkFcFKvZmel
Lnd6KnYJWWC88DcGACBoBgACRtMTBRANoGgACAaBoGJiYAAMRTQDQ4AAAAESEA0wAAAABiBgAAAA
DAEwAhMKAcJkiDmECwKy0Ki4KS5lJcFJdKMz0BnNBVDuiQU3EHMIEkoSSJgMKywzTLnSjQZkaTMy
8pmswmVlsrKHfOMsN8jAbpnNOpI5C7czgL0UzzL9PI8zL0cDgvuhxn1Q5cuhE8uI2YADQNAxCSTC
M0laABuItOgCAAAYhgCYCBgADAJEG3EVaVUXBS7mUO9RSaAojpZlWyRiexGV6QoLgqdkSEpMiSUA
ipOMC2NcDUZQ0maReqmWOuQBIRORUWsoWpmNdGS8uPZacQ70zz53pHBs7IceXWRzH00c83hinqRm
laFcpRLZUBqeMN6wI6EcBHQOaV0TnBvOcHQXPDoLno6CwI6C54bzAjfHCG1YUb1hRuWINscSMha6
qLmUO9mc0MzGkMxrDLHa4xPYzG9QZZXhS7WtRYERiCGMQNwRaUovM6NFdUTYs4aChlyrZYVsk4sZ
CZGNszMtkjBHpyOW+pE550IGN65mM2RMz0hmdyE2iwrDRLIGxYw1vGGwws2rGjaY0bFiDdHGG0wM
2PEjdHGGwxBtjja6zIGtY2a1kDXHKJrMiNayo1mRGsxlbI5UazIjWsqNTyI1rKGpZg0xoRoKImhU
MtKkWlQWKAkyIrEwUmaTIRrjmRqeYNLylajKGpZpFyqZJNkC0KVoZQWhUrgpV4VObK3YEJiJSgi5
0o0FAXSzhoVAaFQF5QjS8rNBmZojQGgzo0rKGqOUL55Q0mYNBmDSs6NKzhoWcNBnC9UBoVAXmdl6
oC8zhoVKNBmZeqGWuhF5Si4qC1VhYVhMgyREGAMbIKwKyxlRaykvRSXyMxqDKa2mN7GuJ7WYnsZj
ewMb2IyPWRlekTO7wpdiqJJA4BY6kUOpW2lZFhWqtKmWOkLlUFpSJc6BbilF6qC5VxLikLikLilF
5SFyqC10haUheUBeqAvdAWqoLlUJcUotKhbSoLVAJkAmQYxMAEExUxkSQRJsrLBKy1rSXNKC+UZz
SVmelxlNTXKaRMz1BmNQuY0iZnoZnleLSWiVOwK5MACJOsqbqRcqQ0GZGoxo2rCG4wFb1z0dE5od
JcxL0zlpOocoOmuYHROaHQWENkcjNKzhfGsJpAKQWECpkAmQZIgomRKsjEJEQkRCQkSEiQhWIQaI
AdCkhEiIkwgWMrLWUlwUu1lDvCkvCh3hQXhSXhS7QqdgVuaESCLlETQNwkScIljqZaUoslQGgzI1
GVGsxhtMAbjAG454dA5yOiueG9YWbVhib1hDYYytazEaFQVcVBYoMaACQQVjiouCk0hmNIZnoZme
kMz0hnNSMz1Izl7KS1lLuCosZWTcQJhFySxJRMzm9SpzZW5hAsCstIrcyq3NrW5pIkyIkgiSBDKT
ThiBuIScAmVhY6kXFKLyhGgzI0vKjWZImwxqtpiRsMaNqxhsjlDVHMGkzheqWWKATIg0MSlIrVqK
y1lJezOtJGZ6QzGoMxpZmNQZXpDOaGZy9lBcFLvCktCstCssCtzIg5xESBDAUgipBFtqhiRJCpSE
BioYIYRbBMAABjiJJEW5ECaIqaKFSblxSi8oRoM4aDOGhZw0rOGhUI0KgS8oFuVYWEAkRYAApMgr
ArLWlJc1zmgM5pZmNIZnoZmehRQaAodwUu5FTtCp2MqLGVuYQJoiTCLbIqaExiTZFlksEwTAaYRY
0TGJMUAAGJSBMAQxEkIYAwAZFiAbIknEHJCJBFTCJMIOQRJITYIkEScQCREkECwIqbK3ayk0WGN7
5nOXWsPHu43KXcFJcistZS7ArLSKy0KywK3NkCaItgiQRbkQJxESQmwQwQwFJCYAABbXCGCGADEp
RGEiLAAapMAaBME2CU0IaAGJjEMAaAYIYRbcRGCbiSJISkEWmJtANkSQIkyBNlMrwoeqRjN845p1
pHIXZvPPP0NVcR+mmeXfqZR5SXq5Hk5+pkeXs9Ejgz7BXKs3RKSVZdZiDfPlRjty8/XXo4+cgenr
8zA9RHytZkbNSIwTGJiBoGJqhiIbIsFBkNAIYJjENAME0wTBNggZEmhDABkRiAAmNUDESCJIiLaA
ZUXIiJIESZBXSKDTIyrdIwPpM5Z1pnGO3ZHBffZ56XomecfpLDzEvSs85L0COFZ12cizpIwz0xKZ
TQSpDXPDA6b5UTtLhM7r8/WekXmoV6ePmUelhwCO/HgRO/Dg5q9PX5lV6M85E9DHzyT0EOEl7ceQ
jprmlnQjhDXHMGhUBtt5+uW5EIqzVS1JNAxBcSISYAwQMQ2RGlGAAxMBDBDcRbKRJrFWEVuckqLm
Z3pkZHrkYXukc86MzlrrSOQ+vM4x2mcR9qRxX20cafVI5c+gLhlsDLO1pWSiStzBqliRvfOR1Dkw
rtQ48TtnCid2PDgd9+eR3o8FV3lwEd+PBid5cKB348Ow60cjjUs0DVVzlp0FgSbo42aVmtJjCMb4
lCvZmLqQAoALdnO0RoHHNwqEtyQhGAAAhi2yhfFFWtVmAAaAAJRI3zya5efT0+bZKVdliBrpjteL
jNwYXuZhN7MD3IxPYzHLUzK9CKZWIg2xuATdSL55SNTyKtr56OiufE6ZyUdY5UzpLnC71z4ydJci
GnZOLE7a4sU7RxEvbhxyuvDmJOkuczfHCG1YytSzON1nO1ReQJZZTPpNIsTTQAUDaY10aTK7WULZ
jABAaUaC3Xz90SybMkueUZayNCgAbcVhBas5F2SLnMIYduQQCAA3EXoPPpzcFW7FYxOwBiYhghyi
LbWgIyCMlIRIItgtmGUdPHsjLyZOGpaVzOmuasumcxW9N8oOmcxHUOZqjSUOLlVTWivIVrWVJqWY
rRGkLSoLFOoaQMTBNA0I9GcXa67M1Zb8tgBQAgIGJrLTbgNTy6jPXOADSAAADaS9ADLHXZXo2hAA
AA2Y7BQ1RWjTFlmO/OgAAWFYNVqzWRsqujLzmjWWJjQDQF5SE1EUTBDAAQEDE1l0OV0It53Sojny
S0tdQWlQWqppYVi2KDJCYKUioulGY0yMktmkbHlRi6arknTVcsCgBAAAAYl1yU8oZbaaQ1QAgAAM
smC7JRojLBqgCwAACGmja1LOqKL8+o0ywTBDAAjTPGGuOYUAQAAAAKcZRl6EoTzclGnNqSEIwQxA
xAxAyLGJEiISIhJwFk4Wm9SWbz6ujisg4uySUlQ2kVYFbta0GkMxrcYzbI576DOdLdIy7VKGlFY5
9EUpdqqDaJOtF8syNTy3xa5ERboLK8ZprWYNBnC+NTJomVDRnE9QAAAAIco3LeBGWtqgBACgAAJR
oScAUAACwAACGKS7RmWWm6nRiEaQMQMQMTGkK3EGIRiFYMRbeWSZm04ulUmI0FZ1qdZHqcZFtkYH
vZzzohge8MD2oyPUGWeiZRoZmoaCNNOmpZEm5YkdCOFG5Y2a45Q1RzhoVDrZbz9eVzaljmWXUsQU
xA0AxAzXmIgAMBojRs5e6Leb08pgtpnpYJg0DALbM9xSW0pmAoaBoIYmq1Ua4cJwMiaoAQaABqhg
iQRJsrLGVFrKS9mc0yM2m5wJktWbdGzA9wYntDE9oYpa4mZ3oqcgBIk60XlETU8iNsuejonPvNIP
NAKVNVNmlZyrikLVWyQkNSgAAwAvocdEya5TLr5hFp6jEDEwANK0OOcac1ACDQDTXoTy6s3Jj6fM
sm4SpgA0DBHQwW1AAjEwaFcoEdSNN0cuvoc+rnVZTExiBiYARnNgZDWqzPQGe2wHKoi0pROsRJwj
VrpRodGmESIFJUlnrNayFazKjUZQ0rOGhUBa6QtjALbsrN5m1ZqaqQzRNABABQAZp1HMVtQAINA3
Frds5vRzXl1ZUzOL0YCAAacwvSWa0MuysyAI0AMRLoc7fLPDuyy5JQlqMBAAaA0vK115AAAAEGhZ
dLl7ot53RzGKdcrJiBgKAAAjCRFoVi0FA0kujy9mbfy+rmXn20z1LAAAGCJbMOyK4yyDgimgQAAN
BnN+AAAAAAJRF2zovgz6MAgLAAAACRtq0C85AACAFEkRLdh3SvLpyxQ0ajEDEwBGjdzdCyxNAAgA
DQr6HP3ROjRTGCUZaACAAwAAExDEwAAAevJdLrrsjLzJI1JAkYmAANSWL1oyAIwAAHbS16cG8uSa
culgmgANAPZiDZo5m9cAA9/P1mU0Z0coC9NU35vMp6POstlVZTEDEDnWEqJRQAAQMRTaBxYAIYIY
iJJo1XRlLDFqygBYAAAEoi6zIAAACAASjIltxbZTJryFIFAFgAAEAA0ANA0AwB68mqXRCcY5ri9G
AgAAA5Q6S446aCkAGixicEoi9FEs3BVqyakgEAAGh6czXoVWVFVOzGAFjEQ3Fmy/Buzaud1ubUJ1
zpiEYgYgejOhiCRFGmqAsgRPpcfoRdzunnjnSiaTddgNImVoE0g0UADAE04BFMTBMEwGE5dgEuai
2mxidAAAIBdLSa60oAANRlL6AaZZtxbJTHsxFYFgBQBAApoV4sXS5yIAAAaCWjPau1Bm8wlHUYFg
AAEGnNYr3UVFKAALACBprsuya8quf0+bQ4uxiBiYAKNADSAMQwQwXS5myXTl1Ql5hOGo5QZMgE3W
EysSwrCwrFsUGMGQ217YnCSlx5ujTZiNRWV6nGR6mY01YAUADFYTa0xgJ1hKNtWU7ccQE6LqdMt6
cZccGWCZSYIABOGqJOuwxAU9mINWUIGmW68uqUxbMRACwAABQA1PIHT50QALAAGiJTqmvQAzcFV9
GoxCMCgAaCAAAAAUaEnAFt34N0ph21GElGxiCSQSIgxA02IbIFjKi2ZR0VbmgIozbq7MRsRkesMr
1BmelmV6WZ3ocZ5TotslRE3S56Tec8N0MgaY5yrylFyqCCCwAAALK9JZXKqKgQTgW3VIQacGzHtW
UJ0xmQWAFAAADnWRbWhQCwAAHACNGrNolMW3EVgUxCACgAAAAAAxCNxAshsW4azc+To5rM5JUhsi
TZWWBWXMoL2ZzSzKawy2aZwTTlIyiV06Y1mdysrclCZMTsctTsZS7YkTNCzY8QbjCG2OQNMc7q0q
CxQCREJacjOkZtOaV2ZDOQlqMBGIGAoJoCFYgYhIAUAAAoAgAAAAABK0wltzaYM99JmAsAFAAAAA
AAAABAbIynolcwUoviYVqjZnL2ZzSGY1MympxlNQZnoZmeh1mnc4VtYWFUVujVFLFUi10RrUZQ1l
OmIOTlg5Ko101ppMxWhUBcqgsUAYgYgYgaAv1862NonLDn7MVjaKYmAAAIADQDE1AAEx7cN0bK7F
HLLK9SZGQAACpgQKbIErFoaEiDEMESawJtKyxlRa1pLxKDRJc11lkDAUZIor1KsxoCh3BUXMod5F
LtFqLIIEUWSoK0yxo2mKMbliRsWRVqWdGkzBsni0RcMlRKBHPBWSUXTQAAACAAAAAAgdlbXfLNpg
xbMJACgAAEAAAUAsAAAAAGnG6zLqzqvn9LBZEToAQaYAAEyBsmuAlFAAAAlEOhPJrzc2PpYKhKLp
jVACABLdz9UuboV2nOWrKaCtJc6JLaNxFsIkkIohWkyiaXkF1GUNKzhcqhLYwFkRBiC7RisNQPNE
ZrFUFNAACAAAKAAAgBQADTjXbn0TRl05CoCwAQABoViEaChoGBAmAAt2vDugxbMxnadACAACBitU
q7GIyAUDQxEMAs28/dEsuqpcTRYwEGhWgQaCzbXNcBryAIRgAAS387VLfm0wl57cdSQnQIkAYAlm
ogAH/8QAAv/aAAwDAQACAAMAAAAhkZjbsDqesccwJnuOMiETfrmq9ViQfSsujNCKCkvoFxVtyey8
clrr95DsvE/NPC7oKaTh/r9mTSjupPp57pRR+Jg9FHdHuzGSJNUsyr5OJVG5QceM4pqMH5SQF1j/
ABJXvu0mlm+cnBJpJqAMDA9y6vbxUQeR3dwwsHd/Qxp0QQaCakMzg/K/sFOEOplAvO3z1Wzs97LJ
6hoa5lrb4EGsjYJptYviADNXLuXWWHaIm2XW2Vz5SQMKD0P1sPf5tCINPp/2gpT9X7x1WI8xCdnO
C/gJzCFtiiCJ+xnO/wARxyK38rbwhxKpr7AguHV74BfQhMOIuoxAxD58uMcGiJ0H3H5VXqIbYJe/
KNTm1Q7jC16sHUFbyypSZhYwxSbK5C0hhpRpphPFL4qVmS6ygbbg9xdjgaG364ac/wAYG6x0us5u
JtZ4oI+OVx66XQQ+kqUdl0we8+M9EQmi+GgG1qJMBgNbdvgrw5dG+De9hI2TWqU3HuFwbRViG8lo
yIP9Frm7871DYOsVKcQKBh0WpYiaY8AnLrtKiLqIhHadw5B7SpQ9Ni9oMdSDLErJ2JrxP7dhDzfn
jb/rtPYeAesuTDAaGaY6iG2jAsgDSAKJNkv+6TTpxgNDPKGII6v42xd/Xq6ZINPtXFtBHXh9NU1R
iied+wWOmC5LXyw6RKoMFUgD6ke0F9EWA/vxnRUf6nLmRBCml/g5D6wYTT0sXL/DODx5A5k1DTvu
Xa+GyEugglMpZ9biPLj1GO+RixuWpCYsQP3AIIEfjolJn7REmOVbaiRcqrn/AOLfTzjgIC7ODNLD
JfKHHAntvOkKXeNNj36tvx37K1q5/wCssCrKrpCSH5RXCRBygXPJ96PNR/ycKD3pv8m7AbCVxwyQ
iTAbqAwiz+xCZyjb4LZKbgRgbgJpCTBDijxBotLqtecuv2lTDIJgQT5fvMMru/8AbPfBwhZ1INRd
l5ZVtlZHKeTTbzD7jbDhpN99L3/UwWe4UO/zzfefVtD1pBRh9pN9fnjWGjzLFF9d4Q5hAIAQphYQ
lIkQ0BcItthEIfKou3f/AC/9036wo+x6ungiCNKFGJfKu+R560+SNCwogUcrTwAKTAIEADGAAAFS
IeVY2DBDLPEumlJo3hvrkihlgjiGIjlirqoBZBdfQ8HDMAIfXY35fNuOnpmONK/7eDKGZJDIP6Av
lJDurmtlp1xokglhoEHkr4mPzrAKNrBEFCNWQwxQOMMdfQggqvjHkljPNw2pMJADHKLFMlKgoGGL
oqslrlvkwwuhhJDm7WuPJk5IHEHIcSQdTVw4RHPDXakx9uohMsqvnlpvolpNOrmLnnqpnpMqhrts
ngnuFmhmiMAHNGPEwdNAOCIIRTQRbSQ8sWVbQbaANntnym9139y3aQcfSCPDMNBpSOtjlIPNLJDJ
HJKINIHCHLHDPFDAGaXKAbQTUdWwz8+99/IPLCUMAGqr858w99566ohGBqECXPPOHQANHOfzz29i
pJkBWQa/9r52VQeJDQx04z/Qac95+01pffPQR6aDOy+uKvHakYIX6jjyspgytqn7dSNSXfSbWXHc
ENDvZGOMdEADKBNHl+WU9uv8mr7n/G/fHaj359DdUUaUN5zi322orNMnPiDEv9d4w1Qdbc4yrOPI
ROHKJv7cfc+UcffXTAAUfBhWRokzjq2wKoisk047+KNsr+ww7h27h6pq126ZY1ffWLCOOERfZbcR
vhpmoiuoBlEONOVCrOwYp3tcOgoi0+ptovrpnjkaeGICEvCHHPsqAoKU6xFDJMCNJBRdUbWbEGRd
KmnPHOCNERYAtP8Ar/yAggQwwAAAQgAAABzjDTzjDDjTDDjDDzDDiACAABBACBBBDABAADPMMMPM
c/8A/wD3/wC++9/tlXBhCBQzTggCwwxAwCxTBBhRxiBSRAiACBgACQAAAAAAAAAAAAAAAAAAAAAA
AAABAAAAABCmdcwxyw13xywwwywywxzyyxCDCAACAjygAAAyygAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAACdWXnn3q9xgIaeD2V20AmzU0XXCzyTSh45b15bIZ484YT3WlWUzjywTDXHHH33XnX3nnHP
vf8A3ogF5eZebt/CC3k6+nCjCD+OmzqHbeSO2yug6y4AQe0MJhdF5wc+bbD7Trzz++JriGLPPX7/
ADahEZaFklBH/g4xJln6vJFErr4Orgvksghzkn0nLFvCJMKLJqlGCo23w1y30/l1y/7B87/eSvZY
ERefWsLhADiBTBPBKNKOrLD1j2kqos8XfrhvBhgDMgmLKhivvs40z6wwz4w2vx31x523gv8Af4GE
T0xrZ5dJKTizbw7vLJhTQRohz8NnFVUZYCR+HUtzbM9+rbppav08cPP9/wDfDz3L6uFn1MDZBg1P
OK+04sQkUgk5l4soIYsE4We6wRlIow44819AeDTL7jSynjdEpdZsxjfff4SrXffvJgBJjrODWM4A
wG3Mo4MIgAwgAAAQww0kINRsAYAwwsu2P/j3NNRN74wxNdF5RRz+89oUa5zyLgvMMdfvMsMkNJZ8
1se0wMMMcI4EMIQ8o1dokQscIEy1T7Df9Z98d5R1xBNZAjX00MdTK7uCAuLlMzi2qEQ40TbQMeeW
LG1oM+PKWmPDx1FsqSkcIcVFX3Pjf/Nrr/v7jE44yiTlMI0u/nvz/u/D5CP8W8A0siCk8SKSX73R
zevXwK3jZoJI2Ww4QNxHbTJT3H0cEPaQ3tdBV0E8MVZvzy3/AKso6E20/hR9tHfAmpsggnKfFPJB
CdebObMXaCLBQRV51+9x4zvyyWdYANfdYQEFQHNDYQIBfVV6dY/vlwyrDsYkEDJlKJqBOKHKENYU
CPSVLHDCVX4+/wC8sNNvP8V13FTTh0HyhGWoBB3lXGTwT4g576oJLa5jZpu75bT4IACQz1F3GVB2
2wBDCUH2HwP+uOd8I9FnXGBzCgzFHl+6R3z1UlRWPgmV8A6LBIDQBfsCzD2otbZ3L+rXkdkDnH0w
Qz5RhX1V2rKtrKijX0HCxRX830dpm0QERjlrfU45qpZYpLIBh9KsKThr7/5mETDgDKtmF1ETggUy
CCEBLLK+OrSCzy3AziS48dO8MvEkMNYpreQCXZJtDbtQr2REj6C4KKyizyggowkE6+cASwjDa5vP
8sH3c520A01BT3S/esNfMuYUku1eMCBBQyIRTif67EnRCwBIyyQwxQDDxy0V7MnDyiCAV+5Pc8kX
imH2DH0F+Of0P2lcsVVgAHXffYB5jDUzjLZusLI5YJ7bFjqoZBM1VnB3AwlXDSywlHs8dO8sjniZ
gDElG9BV3Gmv+/0EnWd0Kalmv7+sor6rLC6CYDjhmhaHkYtbuX0QmkEPhT5PAjedkvuuum7LbjTS
QAAiWlks+2UBm3VWEnzQyxxRx5qJiTyxjoFKuyoBIryzJbHBCH13TE32nHhscdsP9jBDBE0FFjx+
U32TH2lQBDw585Thq6rdRSzlXxa7o4JZq4y554OEBKUUAzEDzX26i75P+vvxCwgSkn+iBev9E2lG
kERSQhTwFVLSS8e2RiyxGhwoKIIIJCBHTyrzxQklDX00IwTwQ3OYvcuCwCyjScLQhScNnH022lUg
jT3sN5cbwg0zERihzjziIMNudbbGlB2UXSTAUcd//RTzJoMvUvO0hRqKTY6181HflXWz0S031xzH
Lilvl3imQyooccvaEgi/ezirD1EfrskwS4t+5f0jJ9P/AN1JR8d5R9lT1LzEeRNNVtNJ1N00pbdu
1oow8gIggERBR7uXb3FZNMhpBhNZ0s86LOC2zXLPEQAkQ4E8go8wQscY8tRdJ5A0ljtvPCeeUN0E
m0c5H75tHygIjhq2luz3FTyiSq6t4KWKtUCOQEcsQ40Npx0lEwIcpAVzj8I1VZpzUImgT6+ccUcA
jmqeP7v/AOEFMEQdRfZYFGAutniv2ce769sNlJHJHOANOLIpPdSbTdeUccKUNlaYvnrPvbDBMDPO
jgluot4EYeMZbTTUcUvjj6/xzUf6/wCdq0ShBH33TDYr6hE1lmHlnHGwQhzjHMkSI45yABDqLooA
zz03hWpDz3HHmL74ZYorbOH0u44G4gQxaxSwDDJ/8b5nW1vg0XMhn3XyZxwqoKXgW3RzzfM9v/mX
kXHn3jDZbPORPZFGYY6r6T5Ryi2SnD8dNcZNcRjrLjH2Q5wUHkzsfvdmFLdttVjZ6dj54LqRpxk/
Zk1dtlHQrVESEGmeoSzjyBd/tjsccPRcZxzyx2XzxnUEWW8doIzoQzB3A+eedsO4lG32XXkXW1AS
I6boJIxEM6aJ9uqSrwAUyyRyQhgT3kRzw3FmFGTGFAaTWohsePQ7ziLTECZZpopvGI2m+/8APquC
iWSyiCKvOOTO+SOa4cAkIcAEoFgQgFF99cAU1JAB5MPjJ/am2juIkRA0okEUCGScV+2uOu+T/tgg
AAIWO6myyHzBIRtgIQIQkgkky6+8xN9FBBoD+84GCLrh9dvc6zt7UY0Ig6Yu6eqdyO01dlUllDDf
RdRTJl7meqiDzwYwxDwm2iiAEBx1JxprdOejSCDnb6SpaFG4tbTnj5j5FKCNFqMBPp8Z/wBx24XX
QRTf3xkw497issGQCoqmphtCTeZCPsLVLK4bAFNC24882sAaDGKPlrJmS1f783TWecJAWbDGMMMH
GJfOvtrrmqFJINlkrGjCDVdODewiIcOnlfPAB863N4nkDPXycTTweECLNTAtiUJIEJJEMDBDPjgE
c39x7jPLEDOICVWKeSVNQVSF+2mwvuLAI0BLFOHtcDsjSrx740lgtMOETTKMLCHLLANAEEIIc952
/wB6B03W4LCUx3Fk0ACnM/MvSRDpYL9DDC6oT2hPeaJRmH797vLzzwznwzyzzTjjyhRD30sMNFXW
qBDTajBTazGHEGzopHB8dl+l2RdU1t2mTG7M3Ubodd5P4J5b76/jQxb7KDR33gRHnkXU1nUU2cwx
zCwTRirPjjxkhMYf33WuMfuhLRI/lTQyyazSg/cvYaNYWoCf10HIF0VHoQaJABZ5KLxrq5VTxxji
ARYbxhCeJ5U4Dy2UveMUvBneou273BEVnlM15ZdOmU0PGBG502nXIm7JanY5PV1V8/UhiE0DLSxT
AzTVHH06TrTZGPp00/SvWzzwEwrE7gxhgTa/jSZNu9jdv0310GWCLWKKolVJII584ZJcepRzyYZf
BwFo7zs4LS+Osc9t8LTmmPevVRzF9M0GeSznlEEXRTkarY5Jhl5pd92oPdc/Nb+PcMw4oIuugLVz
NBXlhiTAl9/sMLKuGhhSOfsbJDAhDwxa/pzRjxo/Pc+/H/jG1Gn2gX8pavcedcYJ7avrIu2b4PZB
sI3w4RTBzVP9ONbDdusevPe+O2J2kWYJfzIZnnsp9lJr6IZLbqI5JvfSc4qILIBfllnxMkCpzgJR
JjW29mTju0P/ABhtNFVXPH5s/HHgG2PjDzvtJ1uvsdFVRNkIJRhULY/y2mMEVLTNlFp1pZ1B1uSE
UOxpm+qPRnA08Sq8swfFmC6oGHL1ddF4/fGqO55FI2Ga+wCCOoNMSCyWk1JrO7RIfRHbjPfUAsyA
bB9N4tpflRVbd90s7Kp+2xNt9RVFlRykwLddVqC+iQAWHiwM+ucObAn58mWEDVtZtVfBNb7xXntf
NwICLT9AE0JXejvTL0u0MQUd/HSBlJP1lHrDie6CiH+COg4y5Y4SGOArPH5iq2yKoNw5sAAoLlmf
ygFmSH8zJjtKijBBttZVNlmlplvJj3KDCOG3TbgamKC28OBaZKHh3/o+Q6r3nTN9xsMKso4jPjZO
X4jARn8DwTwV40QlN0g+GuaCzBw1/B5w0Ioue2KmqiWyAE1RWldR62u2Vd6+MBZRGaGveclUPL9n
Vuy1Bx40MdrrtBQl/wCokjlokKAFABDDMoKjosiCheLlM5Gsx8V6dPk54JSt+XupO+/z3kv4z4w7
z+SQFOOPPjzZ2SVbe/pniilvLPLFLCOpmCV3Dvaxtd4afQLCLj3YYDYHrDzXM2/MKMENmUdeyyw/
36vjQKhz580wy6UYipjnslqpsKOMMLeqjJSiA8hPLJNQSXS7SMYTXnHDJGHmJFDZXdfE3suqkTj7
zjIEqsOPDM1/3iDCPPkkeMvjms4ZcOzCAN1/wzvbSpjnirsvuy8w5/2/w9z8/spilrwjggvkk6vu
iAtjTMCRWJOMgyR0XD9NG2Gg7e7NKCLGcffEYVU5+4tigE3/AIfMXueXxUUFAmS4vJ67jk24h6b2
mWffp9OeabjvSZ9Paz0T0hKniItPZLwjRDh2v4m0iyTyE3uvX/8AzpIABNBLykhn6OOXBsRJReCj
JIack+xgQG8cdCT6V1FFcGJ8kI0k8jbgYQhM7brz1JQfvK38UVIDDggAiCe6j+GOyEUopd0zTF03
DmQkwUkEs84u5WrF9fxVa3LGIgRxCjyBhxAMAAAAIIMsQIUYIsAEIA6YglxBJPjLBV9xYq+OKrJh
VJxt10kVGgYQc226skEoMjbyoc44skkgMAAAwbOkg4Ms4IMccIfQ0Wigc4WUOeieCM2GA4MkRMGB
gFHTJNooeOeIVrTtoZCnEIs8MYwoIMIIIaKW6im2Gc4omWrjTrf1AuCGiwf7zBQieCc8ssmO6eKO
asERDjjvrjb1F7ltb/g9xPuOSXdR+iIwcXnFAM5hnLTTj6YsUA8wUuQmSSS2mi6iOi4GquOa+xrf
L5lDcgVvvgY5Y2KOSiWSOi2OxVgQ1tswUpddhrPDz/M0Ekc0IgUASyKyu2W+CyiOUAoEoJYIkY02
P7zMQ+0MMoWe6oymS6CKS+O84dN9B1Rlf7HPzD/tEqUwcEYMKy2eO6muLD6mWS0pCVR4Jd41tdjo
IgQGSSmGKiu2ef8AxojWSdBV1ydoj4184r32Ycss/RJMNJLGKGBhoV8zlnggmdbTaWUeNf213+68
30twTZCgk4YScccDfea0giksk5ETVspS8uyWmkBD3WZaFoW4jQR6hiRdBfQ88xng5+3zwwhw5+ZT
TQcJzoZQebTWMqjnmsp9tvJHRCkzT2opngte729t/C84VsYaYAQNuZ25/oks3/SCsYyEPeUECEWX
fSPBHJeYyls5ovrr08MPCCMbKj+7lt7MhHerhsrjTjBYYnPGCRT0v34xgoeLiNTXLQSWFDaYSYET
eBDb/slomGOs8uPBPKBK1sBOYUPIFdrvq9vttv4RipMJiov+nwmIuKKANPOAELaHEMPFSFGOugu8
vojuRHMKPDAPPBItNBHJEPvjrunusQjvZfwIMVdurgwvvEKGABMDIIOLCGLCPDCEALKmtpliAjut
cMMMBEELKKCiglNMnLLclsojsXeAVdjPIeYeaWhpMOIAGAAMbSGLICAGJgvrJJ0tgktiDmqMBJEA
CJJMlPD2yAv+wzlrhyf5+y18x1/129ODSPtDHABeYHCGPtviqshHGggtjqumtjguFcWL4e64xyRQ
BT42cX4/s/x0tz/1nuS2z3918xw393tNDDVdNOoAwwsjnnPCLlNJJhh2lEGFdX2RcXXWaeVXZdDX
SWdXQcbb9eTQwRWSXeOQWYj31mvKEDBEojr7EGy3k/8AHI4vMKItvZ7Z6ofNd8EFGH0XH9s/9tv4
JppcPcM1HHW20E3RDSz/AO+M/eux9oo0wwNf7oyE0NwgVQsY4QQsaumLVbLb7y5z/wAikrhst/8A
Kf4LbrsYpT3zBizTLdP3EHkTLAgVA2QRwL6QwBDDwhyDoMW45AQigFTzyDUAltWvo+cJPGJ/u3MU
Xt+sPF3lS2mm+JqZZffGFk/ln0zQVAGckU/f9Oe/c+95GVX1WG3mUETXzSEinmHFAY6zLyAhwR75
++va1XV13GlvsIamiQDM3iSJ7sjr+8pqdvMeMdySDhByAjQRhhQwBa7Z5OllHWFBEkPftd++JfZ8
ACSyRwKLI85yxTABBER3zQSjxTjmDzDHGdIefOv2F3kHGFlZa5vvH5wxAAnTyhwRjvesZIIYpfci
XHigT1GyAQyx8dfYa+/GVWQnVGEWtdutdsEgpDBTAwiQBGnWL+NVq8Jrg8yWkt9v2ya7bujhQAsx
Cgqn1W33j3HRI774ZK7qICRghE2hzGOYzLa0FD0H3+fcfsWMO/q+c9ZcgCNccyJ96KkGNvK7v46N
MOPO7jqTk/VRa8+823lCgU0wProvu2OMHFnWogjQTSk3CcvuOMf9jQ07gcEnAPgCAC+Q+Hx4qJOW
3GWV0H28F3Et3CLDY+5zHkO0vsUwSsLJDgR/apEau77zTABgV0xi9fu0UACSzCj9xgDOAhTrs3Ke
w23Gwpa8X/HLM8ryzaf/AHkFIvPDHoUh3r6v0B6ffV8UyDPMN18EIZE8gBD6ML5z+z3/ANG69v8A
vOZb7339caVFHENMJfPNVK7iec99Puc9OAmDYtgk0QAoOWX1Rc1W1DM8dQAwk28kFX+AlTuUXrPL
rFHchPBN/wD/AO3rt/soICw0Kmqmgpp5bpsL95ummhJ5hcBHLOFML4aVCJFBBfaV2ujUM7+stusn
rlMVhJiZZvux/Hjr1fiAwDMKBPAM0MvDigHrHz/LGrDLBCNSbGlezGBCFt0BIKx1hBA7wTvJLHLL
HIOrQAXvIGO53lkxg05FPN357Qqs/wB/jKLc630ccL71mij38ojDxQqdBMNPcMMfwVVe8tsB7/2d
3u88segqNOYeOr/8MOctPOKSocuM6j8MNwvRPt81RCxUjReRjygMM81HEUV9P0sNMNYL7/8ArV5W
99//AH73x0gzu292oPrW/HLN/wDdRQpE8k9Go9/+Xsd5anOijARe30Gs0lGHJwQgAEYsMNfs21PX
/wCkx/8A71TqgYd5kkoc8s//AO/va7fvP+epzoKJ/wD+6lhEtQr7X99ZHH5HNw06QhBfpBFjrB9D
jyyyShRjcqc/xxbzz6jSE8SSaS+SO486thH/AIjNU6JpmgzBHHfffYx/PfzOn/ffT/vddz5Wcg9s
vuti1/crwnshADSNE2tgqf8AEv3ofPeviATShqpjs8Is64/HHDHHHHLItIMIJIIII8OtSxgIwhs8
+jukwkKTX/6h80zwy4454Gle5rLf98oJr+9ssNWNs68s8o8s4MjrM7xw00eDNOAQdcTYzbLLppII
MMMM/CkNb4PuIHHHHc597+tMcLPP++sPbAHWrDaERu56Akzxx967rH5lx7//AO+3997wo+//ALj3
LQ01qX+owzzkvNDkmwJt7n7/AAw8MON8Mg/sZ77/xAAC/9oADAMBAAIAAwAAABBJf3t9XXAahlQe
IyRxtkQWQYEzdfI2xAlsmAxTCg0rqMz3+xK51msSQWP0W9acFn0svCfWwh/BECGIcF0JYyUDRtZD
eP3AWr4WYiJ8dcKisSYeHsNd62r+9zLdtn8kDDa83DWV8Tkxg8+Wzb0KMSH48TTQzKn/AOyHdzGf
uOcQ0rl5GHFDpkNf/wAXKPYwAu9U0rWK9tm/UwOrMJH3ftqXZI6gLks6/s9OfbVSDKNVtQiJHIjr
KA9yBbp6hq7dO5qONJoA9Masrp+3KfW7SloqGjc36uy1U3fx4qyQob6VdXAHUPDWLgcQwectftaM
A4/a6V12tsX88Ztixfks3NW0jQaSi3s77lMZxl+1J0pu03eLIJcktLlR7IBxPkgpcPcvB28DmwLv
p6fDODstMP42aLSG2HOfWjA5CcXWMS/GdL33ce2RqXIYuim0tW/eQs55NnCrJFaJSV0pgh2hTofl
vsc9KisQh8sGJPYzORB94ltHGYFahuIr1BF03YQ6F29pBKeE1oOxAibXibq2T0KAunM0uTQ3glB+
6yLR4q/1vNKZSO8yHyVu1MKJOT7W4/17z/52TqQvwUwe8VwGEtmkwDZ+yjABkyixp/CoVf303Q3p
fFhAAyujeFV1NxxfnM75FHi18pDtC0QaGbDUPerPujxAWn0i1XvZAF4Xs6K0qOZNeoas+3NyPDxW
bqRsbTE6j02TcwPmRbsYdADpK3NQcHMAM5/4JJC8XL8gSlVVBZPWzFMANTIUI1QY4ZXJIQqMIGFf
OKpQGwCbTJ9mSbwYCCGLJZdyGPawx++onbnhvhOBT7B4mm49zp/TSp8TAoRI+hxUnW+qOJOvyOxH
Aqq6pMl9clb/AAT0X0JTNd2T5S4ECBDSjzHYqGEYCYmCKjE0DwWBxtuOlrQy5K9Mas8ry37TvZK7
Ih4eLBqHyXxJKylX+I7xQlYOo55dJAhFlKFMNNAkVEF1EEdvd+21F208suTi46hyztOF2hSIO3u+
ZqaN98AzFnjmzLVtMN+OMnic1uXhxUUTW1cca+Oagy9NilNFx0zAyzh6J/abK8Ms+BwwB4rMJiMh
ghChvrbpeEJbFaAaoLMudv8APFrvrOpayKknz7jL/p9Vn1IltBh/jOHjf1AHLjGqqAdQzv8AwkW8
544vn3GPmzecSWOk9xEOh99wl4982I8Zd43eaVe4znF185zaRVR+4q0zLyKKK9KCHNi28StP2zzi
k3/3+88R20av4BgR3/078/x1/e2WE54y4z++3+3+gv8AvP1XtLyv78Y2iZDRTEFf9/PkcwN/c5cp
q4NvOT25Jr20Yx1zM90XP1mW0XSP0H+s8uv9PXtcfOGhGX3jbOHiBByhTHmdPeddacyBgX3pZU9N
a84VNtATUMeMFUiDVnSQ70y+vaBAAHMeWzlzk2XwDGwyiECySRFmQjH2NN9sXorPfP8Agoy2HOPP
4VFJpIQJlRGWkX7wiSreucMVA8FZkmcs4Rx3VpwQ4c4Ngw3fbiQlHDXXyJJs3jHXOqMhF9/ZLE6h
qpUNENb8dS4ENAJh5qx/Vy01Is5/FNFVoxXFweJ0hN9173nnF1jaI4qerSJcdj8D6Z0BSoUiwRdT
/LSjjEtp/wCiRRkD5S08du+DBIkkLHPWMlALRSQUgeGiigPjmrvgAQ/ot4krLXcQVqusW5yu7uio
lHGz0m18kKWcg9jbZ2d44fz86ntw5Y8t+/lnww61QZeV9U6JDLTXe5WRzDchyYSQjN8y0b2x2gma
9ohFOAinviNCtEk3fZ+KqvJ06Z6zyy44OA00PvgGABIJBeRoM63vsIBCBBCBCADAAAAAGMMNPOEo
ggilMMMPOMOMAkggssskogksogsosccYccRfTTXf/wC98NOLV72UsBT6bvwD9+vNYAz9kdLRdiAT
GeWoCyGEGQAAAAAAAAAAAAAAAAAAAAAAAAABAAAAABAcHg4444+8xzw45776wxx5zzCCDAACDp5o
ACC+2gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACHxJ883jWfuCkCs83npZbIe8fHwC3XVeMYrmev
KADGoWeo7y5YMvf/AFJE0800889ww051/wAtupPRJFt3I+RzuogX7Ves7CHUL5p34OGSaafck5AO
63mUuNJDAJAOWwCCUdNIBeKA1ZPN40VcXdIUFCG1RdusZPbavJJ+KgJkAP1Hujon6+zG46zo2xHk
wXXFAvqfShHPo5rjq2g5CN/C0KlaGtWtzcwdsRhOqiKixiti/o71EpgSmt7+pfO7odDskZVwQw3K
V5y//wAgC6brPOOcssbvMVCoOecUZXpKM7NbiLmEi9cd2VmRbqYrbOMHTWm5HhwGTZMRPVYV8qdt
d7yfnVNWEECHPd8HXWfP3Zg20Nm7NvCgAQ6GM9st5pDqc+JYqYrtUy0nzhnmf3nGRA3cJusnyUGI
m+YfVae4CIsEqCvOBFNecOXh3g9V/plUkT1wT9tdeQz3Ug+dZqpURCnl3kVVg6JKixAf+I7Xrr88
PuIdvwDrfqKQ8UbRedxnBAwPlgnbqZAsYH/4Iw0032U9ucHlhD01+1hjQdiMJbU9sKNI9O89fP8A
+ZVmmK/M2VhbxagS7RphZ3sHbIHIA3rp5QCZi1sJVtI0wuKrA0mQCeScTLWfz57Zp0ltlqGXNMpr
+majCHyad5CZiFY/NND/ANYX9DG3oh54MmbcZLxUl/G5nkJ7848iLuy3U31Ukx2TDiW4644ACgJ5
1MUIGfYF30CYARqB8wjqFEKnHAh7xZDywpICqlTMLSb+YPIs+/we3Ts2ew88hCXZe+PAQMBMYQFO
y/ktlnM4+CJ+MJ0OnhpGqoOpcXGDP8DVQbGYQSTSHKg5kqmv7DixU01cwDefc1KEMerHIXWVWIqi
vgR2RcGuqMqNKEDIGtTDs5jn7JYaFYAIVZRUIQKPcjv41rTDX61YdscKijCcfynBbHSQaKIArCm4
W8OEvmqtf200gZb2LfhccSzQdG1nrCVaSr2WiiLbrr2gh3B1Uck25XwdS5IZVATNHQNrWMV6kuU4
kMguY5v/AEC6oOegz9M0HWYQQFU3XkQORnkAayi1H7oHnz2AzhhKMNdk+vHkfbcg9J61i6Ook/zf
LVEzy0f6RxYgGGKgoTzgNLY0XzklL8499jJe2iM9wG1izEgcOtOM8fcSh7Ayv8z13Tf3v0tQY/yH
yLxhtP77H2oIcVQi8KgXG3gUCIOZHxoFgdcXQ0kU89tVdW0de0R0lQicNMEasylqgJYPPK+JYYJ7
lApowORaqbMg1rDSlSaMijYeOUEmyUoSpBEkWuz3ndi//d1CzIrSoF0TBi3H6rZoJ2daSoj3kAE5
Z3YjFpa98fNpSzkw8GqpXdnOFHG0k5p7La5yGfxL9mXWWfTfTys3P65r34Z6SyQHSYHPYlUMVUO+
F/7/AP6Sw9EjSA8ku/pZFBo/bENNhVQdukUgFkAoBRFx+ik7KtIsQzBsgAEXfKumaGNKld70e3tK
IhN49BYAudP7Pj5FaGAwh5BDIg/rqk850Y8hIoI0uCI8qK5pmnCyruyte+Cyzx3Tm729/vXIYNAQ
gD9UE50SLQNxGAxRdIH2IAAvzZIUZ5Eox9YBhDwY8L7v746QkYf3DSMNDOt1/i7QIsdtplyyGKJg
NnuOTadEXY9rL0WCRzNlPxVl8NthF8JJX5ea02nl2kcuiAlqHm2rVlvz6J3UqGLJ90gDXsS0JPez
THrDv8ddFVtnZf8A8OiLl8YcNeciz41Js15x3sgqttKbPDA2TGdctXG3nJZRWeKGIm5BYu5+8evs
hokCor2NKZLpIAGx80wVBe4aqN03m+WKwWn5fIcE3jUarHFF9SO280T/AFUsdy4uWlXPobRM+9o3
SM6iz3CEvea8lPW3gzkACv0mjFCUUxMYMV07fXUA3QwtlDBjC10IwTD37ai8AOVW88OaZhss/ubo
UVjBLR3G0x4AGFjHiFkigB90Zy94tgjwjNEFnlkh3jUHz92wTXVfM5Ov/P23cu+WidkG2FV3QBL7
tH023xD6hmxVUVxEW5KNxQGP85WtePHctP47tanL/wDLJxeuiKGe2OH5RPiMzAaKW4hBNBZ7euDy
9J1EzmDw7wAUSz4p4oR6HAQV056RZE9HPxJj0808vHMAfDpeCttxdKi5filMxQJmiSefC3BUef1n
ne0Cp5hwmFZUmGVi/GbFjnMATj37K2be4HW21AabjGWJg8CxvIKfHr5LH+hSiS2x2XYkJ91fiO/7
RxNDembcuTxZ9PHymRtd0HHbrzLHvpBko+9VlZBLeObbxPrMq5DrEflRlld7JJBc9xB/COQpcdh+
SjzBobpIoyP7iGxLMi8Qk+VH1nr7Di33V5MN9NwTwq1oUFt51p5FVMLXsFkZFJN9+TVZOchxEZfH
db8ocD6tb8FWskYwTTHUJCKSyqVcwyKvz6mg6KwubAYGR48dxgABIw11Lq+MV/WgAEpGGBYMHqlX
9hXBTCDgmtx5tvr/AEeE/tcJceYB1KQDP20ZdwcU1BlHtjjdYdMwW/vk8KNQVScODiCD9dEgJgNM
ip6ix6fIHAC66Y2R1fc/8lbEwe9DH8dWAkOIo4i/+Fuy/wD1C3tYu+7tgUWyFHfTNR1zcKH/AOWI
BNJZ9zm6t1SlpXmZLQRRL5pODHS6LjiW9dBJQxf33/jXphFZ6+KgaFMdpdpcbnYRgfuuBFYDcUZO
f5W1eUF6X1T8ACMXYskpfD5xXSyCxl5XPBgbmWGv95lFhksJxCJcdtAQdYpc18+JJbWyJdFF99/Q
DP13uKwg5J9DhHe+7DDv9st8xZ1p9BRPLvrSXx8FEu7UtcZBdw5F6VdepGEA+LAujHNHP0LMiEfx
FqSzDqjcmbzCDaOKiCWGhBJVl0ZLPf8AVDF4YWciBUc7FXcdHeNyEStsJeeOTWxXq5zcUtOmnX5G
AjworokBGQox/QUayzMMwYAUQLEc6teLlbThGTYCJlzGMAqJkb32+Vea85mDeyc6uZaBgIPcNtle
bESkcxE8y++/34VZNkjNBDCMsAtsPCZeSdfHGjpSGMPuM5wf3sG618vwONdYaUXm5bQvk9abUbwX
SnaxBYQjf8v/AFbBbHI7eWkHfOQV/wAAdboVh4R0Yf3h1GZPhbP6SMGlD9MZWj4Xsvht9spLS5w9
8o0lyo6LvzxpEe3miiNhKye2r24OReTBhw6m80sbk8tB+KfFNJmAFq0VltIKVbYQp7RDv5M7NbDP
Tow1WHem2Iq12jrBqbT/AKx9pae2pay3yz2myhNQiH7vlNt2ZPsALKZ0enfvai+003+TNC3l0IBX
WypuBAVtHk+daChbG1mn987mryaQU1c7FjIBfSDJei5R9ZUtMVtngCooOOPGZNFGXi10w2ZFKlli
Z7m2lSgopBDPHtWg07vnycd33OyeMhip5ZONgqMyOXYnBmM79MJ5X6/wnaRs6JNCGYrz64PFP32t
QcP4yx58tISdeD1Qwnp2Sd4DwyFtvWk3nqndJ6+SmHACKmK3tTgLJf8AIaNtJeKpzC8M5VDPNNLe
21EgT1XJDbrJk12JKscQLlpUjLYAjKwHUWLqZXAtX1rmxkaNN/mWHQDEcyGFk1H0d01FxViABoZX
YbJLzhaeswEMMEeGYxLiS8XLOd3NN3ft7eTZqQ/h6k6aH+ZaqFehV3XGISFpgmxTy/8A/qoi6bDv
affkw+jz/FtSVKF9OmGWgcywFK80WAji0q7e6rBCbK3Bes6RemHfT42yoJZRAQbNLUD5DjYJC2uZ
pu5LmCP+8OoNKhT5DtGio6FJA2lV2oBNJtn4kg3+BsntKnOuhCIvI62Bhggyuo09AyTsOhsNBpBv
PHrneknkOSg7Ua6GlBWWLthCThFi2gv98O/395k1/g8LPT8lLAWY/val7CS2XeUvrZzfRlNb1XTb
peQYwmPlIZi40CoK3N/QmREJpuJti0AFbOW+jT/l7n+saO8PsDZ3VFN7eam2lV/YRhRQ1HTdTexl
81uRJ3nVVzHO60AkSEUua41RIjSlU6c5V5CskphZlAL8P2+VMsjl/wDTqj09fAYdcu40QSEV+dy5
ETGyoNN0848bmUPrvLhswZ4Z2ZdMCBBXoxv9wBn3+N3rFd9weQnWQAgrlKy6QAisBVfX6ERcdRss
0wNTE1uz6JmPFw/jZw58+y129vCGIFgtl77o7h/v2U0vTwuRRhnvDVNsChQhTP8AO21BJYzquLIB
e9fIz0jUjBK1D+cUcXFeUhWLucJVgPXan/cIKg+RmZkPEEtaOBjPsEWlRYtpXWUJyqlC0h546vR3
LsbZ4FxkCgPYmmvf3aL6rNnOTJWeLf8Aws0oFKbUteYNA3We6rKmRjP4ircVg8DPGd/9pTfyGfOp
nGE4yeyK7sCi2NYSxDyBCy8raUw+2EE2TPKFlPK5ZQyw4A00scQKpr97FF74v2WlfkDAyRwSzzAM
AAAAAAIoUIUcIsMIIAsprG88Q9JBnnvR8eVphY6O40ckMZxQD1dHnhb+VZhXe1e38xww1UloAAAA
Uipp9FhQ4wsscIfqisYOjX9SIxxog71p/bLD2Pl2TSaTnp0IqoFnwipjyqD8S/8AjPKMnihihiuK
mOLmPphkmoIL5436a1GumBC9+KKHzuEE63BPovgIWdZ2z0JIKMojqMC9wwQXkWH4rIBUVVpO1aeq
Btw1IV3STddMlirkngbwQjquu9836zgSxfZUWUYDu+LpIgpiY58hGfO0wVYDZZavgZhr2yosxcaI
XdMu54wSunhkpMBFMPiqu445Z7nVPO8+44CLTxyx6zk3sSS8NHJOnrtAzswvx7+dUz4jJMXRUTz1
z879UytImOHGcW5ceU291etR6wfa6sc4v3miVMLOscbXSzrl/b2os07jAbbmGPdQ09dvn0USUK0Y
03GiXzplpQWYsBcb2rQFZQWeYt63fGbbaPmjgnk/3o+Hz3yR/ol8ijDVINGxnsrqs4p3yONlOqno
36UsVdm6a161gi6JVbio07T607mp13YXQUPQdT29eSt3SE08ccXSHunmERj/AJ7rpN5wpYU3/g5l
8rioy7dAPpn6uY/5s3fNs8IAQ84uIQ41ooMMRwT/AOLH6UQwNtXS3Te4lYzDuDC2G6bPqN/vkAeo
hCGKMG7h3mSNzzTekLiRRlFMDWBDzHaljxQMHzMtQx9cI5C2eY8i+oBcG2H5lCbAqKjLQoRsIWL0
lgM5Q2xF/DNJJDoJ0W86+SPcEQMQxU1cAwpQ4EKaKLCG+hGSeyS+o4AjZL95JltIuv3PIcNOZ9ye
43PyKlXYUq4mAsO26y+oUV1AsQlg0wkM8SOqPX3My9CCiWOO1IIfpzjXp4CxJED3ddpCS/66Ff8A
yt1jkDlIjLiDES+gdXARFDBZWtk/VM6ghj9yqOO5ygj8CIaDmURkjVxqpBXcVEAin72gMFPIVVrl
4tc7lk53YHZJNtrmtrmKMtp7zKDLGTEX6lDWvFvpHImu9C65QfkSNUDIdq75CEW04vAX8kmnSeSy
ZLSeCY2OsMYWWc47TsaYZyXGd6XeCbxYdUbVpropm+ng1y58/hjkLtJd6UZaxicFe7Rg6zlwadcW
2ZXCnvXMeOKryQFOdTKDUYd20knMB73g22x+TZbf6OsLJG9n1yVBCrzvxhW670ZHFtPc7LFbPbx0
ufG7fx8izXGcSWSaSSaSdWkNoPOWnMMQS93V2iuzjyWQXBUBh7ua4IAg9xTh7bT3O8g3yN61fY9u
sjknno1FT7DBnpvt+gd6z+Ja5Z/FxUCHmAYaiyfnubHqKISLyhhEeZdZpjYEbhX+MQdBCS9XdryW
cGJTXpx6vo+tMixo2Vl32i0PrgtxVnExYSYYe5TFvrUgokjbWf1dWdGfdfpTHXw9scwnp4jt+pqh
iVbVedXp3hjitMSX/wD+2jYJI7BmzSMYb8VXwmw0rncGmDepK4RdOM2t97JOa6raRNYd1nUQgbO6
RaharVFNaSF73EQ0IiKqafxTBGGWE6Ipvv0O8DRvZppbHDzqf1RC7K7kM7orCIvRhvHV+v8AL/xv
eYKdBVW+hFARMD+mmdstFHAHwga3PoFjbLQNlNj+B7zXyWIMdMZ9o3KS3FqqFRK6Xg/TShQSjJL9
mSvsgQq2HxFLN8B1gPG/Wx7qrov8j2H3O8VEEsrjxd4oVRnMrAKBw8p0p6S46xAUBiueXbObvMIX
jl1Y32y82wlamlknW+4Fy5djDwt5/HmW2Q/6XAFUGYjVJtR5v1wWurY7UsXm81rbIWXSCjpAoxJX
lWMtexgY+f2NW3zEmodZnDOva+C3cVrDb0OQ166YmdeGIfoGtwo3PrOefTUDHLe/TDbvH6anNdEK
WLLTRjyy4VlKcPninbjrY4qnHZz9y3DT6FQyHHLUHWhBfpWAdQwTjsLLPpUZVTnhDBZvpdeg0p9a
TVMwzcuOLwbErWh+1b9vj2W34xhDyRanZxGP8ypx3izAjfDLzAiLIE1cjCZvCK/b/ShjDUVIdx1O
ozk4ZJdc1M9WnrmSIYwxwlb4rYeNDIVkwsMQu/JL9FUlNA9C1j+xnJZcDLJjmCWdg2Y4OHbhljSp
CyiGGfPCS+S++6a6acCBJobtvN/dq6yMMfveNETgzgSNmFLZe3gDDKJ7LX1VacHTV6xYJws89NAQ
3P2eTKWSG2ntUP5zVl1r7qYChTn1OanpHxXBHrC3JLCERr83/XplbjHNWMWaTojzT9gTCHnTftWM
ho3WKGDd1DPNBxFFPFlSfrzqMCDDpc0+h1DffueA7JsHjf0L+pPwahX/ANwxXNTx15qpyyX/AF2R
SsV6L/8A3t9BrNJRhQg3IABS7DfrV9uPJD2qXyetMAVhULMFaR7sdszf7w8v7z90FzRoMtNsBCmE
7tOLHdtdlvJvZraZQhBbfxdT3vHDTOGscRhjHRzXRxX3/wBrwYOBXFDLHBc/aqt/skU+I2LIfaDY
fVfbfYx8Da5RW8dcVzseeyVb3j+votuhy6JfdKCc9dNUdPruGb85Cp/WiFmY2ioVAD518MfHDffb
ARSUQim7dQkossph0ZuFuREsSjisV7VN25AUTpYvItr6DAHLymZCli81ysrgEfS89V9fLzxwhRSB
duwcULbGMDh6qLRqr9LgR92+EMAUo170kXYQt1+sRTQVasVv77y+Hf8AOWly7TEHTeD9xY+a/e5a
KFCAQpflNIMMIIsEH03oJP8AuLRlDTQbJCDDHft80e0t8Kr+f/NrPDztqogv72m2/8QANhEAAgEC
BAUBBwQCAgMBAQAAAAERAhIQEyExAyAwQVFxBRQyQFBhgQQiM0JSYCORU7HRQ3L/2gAIAQIBAT8A
7ixeLEiMGMpFsPBj5VisKd0LYrWDHyND5UsGIWKxZVtgsEcPucQqHyMjCC0op15GPkSHyITHg8Y5
VjTuhbFYxjWnLBHIsHgsVgsKtilCEilHD2OIPkeMEaDRQuR9OnFj50IRTuhFY8H0ULkSxQhFSg3E
hISEik4g9+R4onCjF9SnkfOhYU7o7FQxj6SwZHKhFa0KRUipIEinY4gx4vFYJYJ4MfJ2HzU8j6Cw
p3RGhUhjQx9FCHyMQlg2JalKIIIKdjiD51hSMXVQuR8iXIsKd0IqQxjHyQQNYLorcgZQhIjBlJxB
j5lhST0FyIYhcj5FyLCndC2KioeDwjGcVi+ajcaI1KaRYsRXuMfMsFzSTgsFsMQ8FyPGOZC3Fsip
DGMawnFvmYjtiila4KkS5GIr3GPmWxHRgQ0IYh4IQ8H0UU7lOyKhoaGhj5ViuRIjClCwpXKxFe4x
4PlWCxjCBrBDQsFihYvkXIhFO6Fsh4NDQ0MgeKHgsUIggoRAlg3yoq3wqHzJYItILS0tLS0SGi0a
Eh0lpBBBA0WlpaJEEECWCE9EN4tDRaOkdLEmOlkMeCTIGhIS0IEilYIb5lsVb4PB86F8hJPTkXEa
Ly8vHUTywQi2ktXghEIheC1eBJEIhCJJJwlcnYaLB8Nj4TMmoyKh8OpdiGRihJEIhEIhEIhEItRa
i1FqLUWotRai0tLS0sHQyxljLH4LKvBZV4LavDLKi2ohkMhkMhkMhkMhkM1JZqampLJJLmSyWXFx
cXFxcXFxeXIuRci5FyJRKIo8FtPgso8GXQzKoE/nJJJfkuflkvySSSSSTh+DTwjTwjTwjTwRT4Ip
8EU+CKfBFJbSRSW0ltJZSWLyWLyZa8mWvJlfcy/uZbMsymZbMuoy6iyotqIqIqIqI+vr/SF9Hj5N
IVCgsRxFD/0WnsLZYcXfBD+WXSgj6ChcWDORXVc/lMtmWzLZlssZYyxljMuosqLGWVFlRYy1+CGW
stZay1kMh+CCGQy1kMhkEYRyQR9ElEolEolEo0NDQ0NDQ0NCKSEWotpLUW0jppLaSykspLEWItRZ
SZdJlUmVSZVJlIykZdJlIyUZKMlGSjJRkoyTJMlD4BkMyTJZksyWZLMlmSzKqMqoyqjKqMuosqLG
WPwWVFlRbV4LWWshkMh4QT0JfOsWakvGeSSXhOE8k8skk4y8JJJxkklcsIhEEEIhEIhFqLUWItQh
FHCTplmTQZNJlUmVSZSMmkyUZKMlGUjKRkoy4LSwfDkXCMkyjKZlMymZY+GWMVDLGWMsZYyxlrLW
WuS1lrLWWstY0xJlrIZDIZDIZDIZDIZAqXe3A18gkJHD+D5OCCCMYIwggggggggggjCC0gggaw0w
0xgggtLS0gtLS1FpaWlpaWioLCCCiuEZpmF5eZheZhmGYXozEZheXovRei9F6MxGZSZiL0XovRfT
5LqfJei9F6L0XIuRci5EolEoklEokklEolDwgggXNJPUgj5uenJJJJJJJJJJJJJJJJcy5lzLmSSX
MuZcy5lzL2XsuZey9l7IIH/o8D5II/0GCCCxFiLBUFhYixFiLF4LF4LEWIsQuGjLRloy0ZaMpGUj
KRloy0ZaMtGWZaMpGWZZlmUZX3Mr7mUZZlmWZX3MsyzLMssZlljMssLGWFhYWFjLCxlhYWFjLGWM
sZYyxljLGWMsYkQRgiMXyQPkjnggjCCMIIIIIEhojoRzwQRhBBGEYwRyQRguWSScZJJ5JJG+ZMno
pjwf0NUjpGsIwQ+pBBGM8z5Z+ipEFW/XXSnB9R/PwQV8sdKSeSeafpkEHF3xf1qejBBxl+7/AEaC
Djr94+Z81pHLGKwS15Y5ER8g/lIw/U/HyPoIknmfJI30FzST8i+jPJaWn6r4+hA+nPQnnn6HaQfr
FHE+pP5SC0/X6cbBfLR9FfRSIPaK/wCbBddrmXLIuhStF8wxdL2l/KRjPSknqx0VovoMEHtNRxl6
c8ciI5I6T+mQQe1f5l6dNEkkkkk4x9KfQtLT2uo469MFikR1YI5II545Y6MckcsfI+2P5qfT5FrC
cZ1O/L5x8nc1hC6T5Vg/lfbP8tPp1ZJJJ5G/kp+gQQe2l/y0enMx8nnB/TV1Fh7c/k4fpzNJ80DX
0hcjF1Iw9uL/AJOF6YRyPGVthPR8YTjOpJ35Zx0+iQe3V+/hejG9sPJH7h/DsVOCNjyJ4dxzoRox
7r0Z2Q+x3Q9SNzxpgt/zh5O/4PB3O+LI2EecF9BYlh7e34P56D5515JJ6s4ySJzi/lniiVy+3lrw
fyMq3RMJndE6SVb064f1YnI2+yO45lEtSKZP7fgY90S9cPApwfYnQ74R+4cyOYFsU7LDsxkxgu4t
2KMGPsaQxYQeRkwjtgjXDuduRYPpe3tuD+cGtSEQQhqcHhBGMEazg1JCNuZYuH0o6M/LLH278PC/
JU4QxTr/AKL7d+DhFVNWsM1t+4lC5n9OfTfI8PbnwcIeCcj6MrXmkTlEk4SSvJ3GxsZS5WEqSSSV
EklxOE68iZJI3yTj3JxRODGeMZU8k4e2/wCLh+pX29UVfCylRSsIV/4KvBOi9RfE1hI2PY7C/rA4
h+Tvr4P6M0EIe7lo8D7eo936FO61JVx3G9GSU7LDueRdvQScM7C3FMLBqXyIj7IjGMI5I6rJx9t/
xcP1KqUztGH7vsNJjSHSJJGnLoP5CF9GjH2z/DR6lVSTN14Fu1JP7oTnyOpy9dew220pE/Iqk0Kd
HO46oGxP9qE5QuzGyvYqF3WxTutILn9U9s/w0+pXQ6nHYqvt21FMfDBDbUqIHS9dtR0vRoSggten
hDUsakpUIgt212IGpUDRBBH3+XlfKvoe2P4KfUuTbXgmeRvBl22g6mnsOpzECc4XIlDaRKwkdWjJ
RKE5bGJ6InfCo7PUUeRbvBsnQQzwSVEJi0QmLuPc8nc8C7/LSSe13/wL1P8A9KvRG9NUPuUxqogX
xVETd6k/CUKEVbEzSkhrWn1KviUso74J6UoXxMejkjRi7snVjmGoGtZ1PEIW7GJOEh74NGsMg1nB
ojSBDEiBpMjSOSCMI+XWHtb+BepVRS90WqIgtSLKfA0iqkpVs4QsIWKULnlLBlyJUmiY2SoJJJJ+
jIk9qa8D8lfEdNb9C62nUVcuCmtvsOty4Ww69F9xVSU7P1eDr1Lnp9xNuUUzG4m2TEi2KnoJuYKe
/rh/b8C+EWqEKUjuzun9j/6d2TLk7Hk2WEyvoc4e0v4Cqia2+0DobpS8PQSfdJFNMUwKncscLyiH
BSmsHTrKLdi2GxKBKBqWsGpEoI1waTIWMIhckL6QsPaP8DKq0qmhVJrCSUSsKXvODqS7lyLkXJsu
ROKeklyJ3HtOFOyO8deOpHycsk9ofwM4ibraXgqf7VBSofYoSiSFNZrFBRENyUNa+uDbkhxSVbMp
ep30nF6oWxa7UaqRp2xhSmkRLn6cnh+u/hqHSpktQqaVssIRGEIZKweqFTztxyJ6slDaFqh6IU9y
S5DaQ3pOCLkXLBskTE28HuhsliYu4mSyTwdmdhSeH1p/9kn61zwajicWKoRmftbE6u4qqmXVNuIH
W4TXcSfcU9xjcVv0G2oG3qJud++D+KPsPR0jb/cbVLCpaofZFOxRsL4mRvLO+pR8JUpTFSdkiHFR
s047D1pJF3IeigjR4NaojcScyJOcHLGhr/2ISIIIIEsIIXWbw/V/xVHEobqlQOluiIWFKhQWuXDL
NvtgtJwtUlukFqiBUrCFMjSZajTGEyEQvBCw0xTTJU4tpCaYmmSsG4JUSSKoTLhvUnYnTBEsl4To
SSeD8kk4ToxE9D/6Tofqv4mcTiRXEGYokVcuGi/7aFyL0hVJ6l6ezL3v2HU5hF7aQm9UJuYmRt3R
qOqClypK9KWJ6qHJLiZKp0G2i/SRVSy5yb3M3anwdl6iWrQvi2gS/c9TVt64V9vU7VHek3Y052f/
AGd1Pg7J/cetxuzskeUbM2g3T9Rb4QanZKCHBGmMa4RhBqR0HMfnD9V/HV6HGTzdPBH7fzqRNX4N
6aV3kdDb2RUmoFom1qKXUiHDpgaacwWtJCmW4EtU4gh3T9h0uZiSlQitSmWwy17dhqSJHSogSfdi
oemo6B0lqiBUipIUyOlYNEIVKRZT4wjopp4ShtIbJnknCcZNOg348nk/U/BV6FSpunuQhKlELBwS
iUXIdSW7JQqkxVUl9I6ksKnCL/sXodTmEiR1KJFU5hovfjQdb1gdW0F7j7zBc1MiblT3G2u/cctu
GU6pMrcITidZFKtc7icuXI6miXoOYNm0vAoTUIT7watj8E/tRrDkUzrg+/qdxbHY1Fh5w7IXcQt3
0a3o/Vk6M/UN2NfY438urjQuqt37idtUdoLm6qWOtp7obl0oel8Gzpg/qz+34Oy9R71eh/if1qKv
gEV/CxTKk7NDjyJN0rYdLtZvUoOyRqp0Iaj0LXH5ktbkSbjTY/dd8I005SKVCSK1KLdX4FS9J7Fr
1XYtXga2IbWoqS3WWyz7jRCLVEEEJYQiEQsYXQ06Fb0q9SDj/C/Q4nDniT2gscR/0Lht7mStC1Do
kVGjkXD8sykOhMspiBUJCopRbS3MEJiQ0QiEQsIIJQ6qUXKJE0yVMCrTLlME6xhU4UiqlxAqpY6v
sSipxBdoxVPuXPSdmXPXwNtuETMFzgmJE3pg3q9RzGCegmJs8se54J0Ed8PPQqfxerNdTj/C/Q4v
EtriGZiiRcTtBmfbQuRVXEQZjhz2FW5Uj4lWr7Drcwi9x95guamWXVaNvcdVWrnYltpfY4dUor2Y
ntB5Y3rqKpqkVbhsV27Z/We8lWxUodKRc4Yk7vwOblJqrnPcianPgom0r+Fim7U/t+B6XItemw1E
fYiVUbs10UbDT1UbmqcwWtJFrhkCT0walsjQgShYQyMIIEQQPoVdxvf8HGc0v0OPObp4P6/nU3r0
8C7PvOw6G3sipNWkOq5oSlohw1A005h7Fjj8yKluXAqanCa2LatV5LHKgophFSlFmw6Hr4LX2YqU
lBai1ruWItThjpTLKYgVCQqaUx0puYLU91g8dMU08JUxg3GKck6xGDeuxOkk6SJ4ShsnCdHhrO5I
+hX/AG9R9/wcbar0K6aXVJbSRStkRTMxjGFymJHUluXKJkVSYq6Wx1Q4gTwbFXMSi79yUFTfZFNa
alirT2Zemy979h1VTCZmOEKrSZkVTfYl3JSOZRW9UUS6UV/CyndQRFf4K3I6miZp3FKpn7FLqnuJ
fueo3U24keyKpSQk9ZKUJK54NO7TwLY/qpFvpsJGw9GyNjZCEdxogXPV/b1J1f4OL8D9DjfyxLSg
odTpevozh7P1Y1FSjeSqupOFA+JVCKeI3Ml7lazJ2qF8X4Oy/wD6F8f4HNykdKvRVuk9ilJLQq2K
U1bOxVvT6lduzZTS3SWOHqUvZQxpw6YKqftJY0l9i1tVfcV3ektqbUoqpmPUdLmUUKEVKU0KjVFu
sj4acipggt0gVEd2RrI6Ey1DUoVKRHzFX9vUThs4uzOLw7uLPaCilpQyml0p+rFTxJb0LKXuirhp
7aC4bhyxcPVNsylI+GmWUxAqEhcOlMaUyNJ7lKSWhA0iUaCggaJUwRhpjVXHZl20KSl3LpR146r/
ALep5n7Feqq9DjcVU1xDM5RIuK32FxZjTQuRVWkXuJYq6pU9zMqltbIrr0Lml31fcpqdzTclNVVz
krreqRP7VLexwm3Tg6qW4lFdKbhLVkK5LtAopTKalVsMqf7m/AtjiTboLVNKRN1NfZFKT1e8lcR8
RdFKOHEaPoPmgjlgj5Gr+3qNy3+Cvao40vjaeDLilyKZiZKZimdpKqKm5SRVS0qZEpVUbC/c6UiG
pphlVLtRXS40KE3VMFrvb+w6KtYW5bWkoRw6XSnOFq8IsrluTLbjXUppaWrnGxJCUJIqpVShioSL
EiymZgsp8IdKfYSS6EEEc8EYxhBHJHNHJU9zc4nw1ehVxOGq23uJ0cSUU8OmnZFlMRCggdJai0gg
gjCCMY6EEfIwQQQQQQRjBBBBBBBBBBBBBBGFb3JOI5pqOJ8dXqfpvjfp0I5oII5YIIwggggggggg
gggtLS0tZay0sLGWMsfgy6jLq8GVUZNfgXAqZ7vWZFfg93qPd2e7M93+6Pdn5R7sxfpfue6ryL9I
vJ7rSe56xJ7g/IvZ0/2F7MX+RVxKWPityXtt+jOJ8dR+l+N+hBGEEEEEEEEEEEEEEEEEEEEEGh+3
yTT5RNPklFyLqSV4JXgn7Mn7E/Yl+DXwa4KcE8EiCCBQQaeBaoceBeg4HMiF6eDZ6GqPUip+gqZX
qUpaQL794KTRMnBb/gr/AEqqqbuOFwFw3M4aGnkmnyXUl1JdSXU/cvX3L0XrwZn2Re/sZj+xmMvq
FW/JfV5Ln5HU/LJwlGmDq1JJEySSSSSSS4TJRKJXkup8l1Pkvo8mdw1/Y944X+SPeeF/kj3rg/5H
vfC/yPe+Cu575w+0nvnD8M99pmYY/wBbT/gz33b9jH+sn+jPfKv8B/q6n/Q984n+KPfeL/ij3vje
Ee+8f7Hvn6nzSe9/qf8AJHvf6r/yHvX6r/ysf6r9V/5qsJJxcEkk4ThJJJJJJJKLl5L15L15Myny
ZlPkzafJnUmdSZ1Jnoz14M8z/sZ7M+oz6zPrM6szq/Jm1+TMr8mZX5Kaqn/ZltX+TLKvLLH5Zlsy
0Za8Fi8Fi8FiLUWotRBCIIII6mZT5HxaPJnUeTOo8mfQZ9Jn0mevDM77Mzn/AImdX/iZtfgv4g+J
WjMrL6vJfV5L6vJdV5Ln5JZPQvV0dXhbiaJRci5FyLkXIuRci9F6LkXl5cXovLy8vLy8vLy8uLi5
ksykZJkmUjKRlIykZaMtFiLUW0lqOMlAlLHw0qZnrU0VLiz2gfUTglksmo1NTU1IZqQyGQQRhGOh
oaEomnyXU+S+nyZlHkzaPJnUGdQXovRmUmYjMRmIzTNM0zjNM1mayqu4pcOR8WU1HRghkMhkEMhl
rLS0gtIIWGhoftJp8l9Hkvp8mbR5M6gzqTPpM+kz6T3heD3heD3nwjPq8HvFRn1GfWZ9ZnVmbX5M
yvyX1+S6ryXVeS5+S5+eb9x+4iotqLWWssZYWFhYWFhYiwsRbT5LafJFI7ETQXUF9BmUGZQZtJmI
zUZ32M5+DOfgzajNqMyovqL6i6olkvya+TU1880fMviUGdQZ1JnUmejPRnofHPeH4PeH4M6ozazN
rMysvqJq8kvyS/Jr55Y+owQRzR/qE8z5H83BGEEFpDwtY1H1dUyOhEaliIFSi3UdKgSGJEDQtiNS
NcK/q9DGRrhVuLbB7CNzvhUJ6YPfCrb6DPy1GHcbGy4vLhVIlDq1L0VVSJl5cXFVU/V0y5kkkkkk
80k/6rH+jST9RnkkkkkkkuLiS4kkuLiSSSRk/PSSSSSSSiSUSSXEkkklzE8ZZLJZrhDLWW1eGWVe
C1lr+xaWryR9yCEaGhGEdOeSSSSUXFxcXFxcXFxcXFxcy5ksl8sPwW1eGWssflf9lv3Rb90Wr/JE
L/IinyyKfLIp+5+3wyV4JXgleET9kT6f9EsufkmryyXyRjBGEEEEEFpaXodaLi4uLi4uLmXMlmpr
jqQy2otfgtIZH3IXlELyQvJC+5+0/b4NPB+ET9kS/CJZNXk18muEEYQQQQQQQQQQy0tIIIILS0tL
S0tIIIIRHJJJBBBaQWlpCLUQvBC8YRhqakMgggggggtIIFSWlpaQWlpaOktLS0tLUWkEEEEEELHT
CUSSSSSSSSSSThqampqQyGQQQQQQQQQQQQQQQQQQQQQQQQQR0ZJxnCSSSSSeXU1Emy1lrLSxlpYW
FhYixFi8liLUQiCFyQQRjBHy8YRhDIZDIZaWlrLS0tLS0tRai1EIhGhoaGhOMkkklxcSSSSSXEkk
8sk9KCCCC0ggtIILS0ghGhoaGmOhJJJciS4uJJLi4uLi4uLiSSSSSSSSeScJJwnCSOSOSCOTTGSS
SSSRsuJJJJLiSS4uLi4kkknCScJJJJ5pxknCSSSSSSSSSSScZJJJRJcSSSSSSSSSSSSSSSSSSSSS
Ti3jJPNOEkkklxJKJJLi4uLiSSeTXkggjCCGKhkkk88kkk8skkk4SSSSSSSSSiS4kuLi4uJJJZJJ
PLAqWWstZbV4LKvBY/A6WQKhsymZZllhZSWIsRai0qpEhUojGScJJJJJJLiS4uLy4uLi4uLi4lks
lkvoWvwWVeDLq8GXX4MuoymZTMgyB8FQWwKkp4SMuksp8Fq8ELwRgiCCqkgSS5GQLkaGoZRVOMkk
kjbJZrywyGQy1+BUVGVUPhtFpYxcJmSZJkoykZSMqkyqSyksp8FXCSIKKCMIHzMrpKN+R4wRi0PR
lFUrB9GpCcMTlYZRkmUjJHw0i0VMi4aLEWU+C1DSIQkiFi0NFCEulUQU7dJlS0FoxdOtFDh4TySi
5eS5eS5FyKq8KamhN4xgx7lK5XguRj3KNuWOZj3KHoum2iSiqUSSSiUXIuRcOtIzV4L5HLIYpP3G
pD8kMaeFKllpBBaiBLlqwp6bGU7LoSSSOoZTXBei9FyLi4u+xcyWSx1VDqbEKkggtIIIIK6SCinQ
gggjCCBoqpOHpV06ngsJJJJRKJRJcSVNvBNlxey9lzNTU1IZBBaVUDEU0lpaiMYwa0KqYOGlPMsW
hqGUPTotDUMpcrlkuE8NSCCCGQOSRCRBBBBai1Fo6SpQJSxIggjBYsrRRuhcjFiyso0q5YFyVoo0
fSrRRVD5ahISwjCORoZSuiyso35Y5aynddKpE6i6LNhOV0WpREFLlcjRHRYyldKso3F0a9hbi6LG
UOVzLFoqWpw3yxy1opcPrPbCnpV7lG/KxclW2C25Y5alqcPkjlZWtClwxPnkkraawpqLh1FwuVcl
WwinpVblO/Sqwo2XSq3KXAnySSiS5DqRVVIhNlzLmSxtlzLmKS1lklgqSCCOWOWoQulVuU79Kp4U
MklEkouRci4uHVhAsNRyNkibYqSwsLC0gtIHSOSlSxLkjq1MW/SbwQmSXFxeXFzJY5IZaxUshkED
wQqS0hEIggggaKqSChadKqkTgTnoySSSSSSN8kvF1E4akFpaKkgggawQkQQRzNDRT0mVIo51ixqG
Ucr5ZwnkggggggY2ISI6DKtyhdOop35p5ainfmkT5K0JwLn/AP/EAEMRAAMAAQIDBgIGCAMHBAMA
AAABERACIAMSMAQFITFAUQYTFBVBUnGRMjM0QkNTcoEiUGEWIyREYnPBYJKhsVSC4f/aAAgBAwEB
PwAaEQWXiky8whBYWHlkH5D8dRo8hb68rZd7wsahIvgcQ4aEhYYtlKNjJ0l0EPCLmlNT8GfaaBFw
uhd6Hii2NlNZw91LtuGXZSEJi5uxbKXLGh+Zw8LbeldjxpxSlKamcMW2E2MTOYt2MW94W6ZedXgm
Ws0i6KH02IbGy4Zq8zSacrNy9l6jyxdFmryYvM0iZejS9F4gzUxMpTV5nDFsu17Vh9VYfQfkz7TS
xYTLspcN9VsbKJ4ZoELoMm+bUPcsPoPyY/NiNL8FhbLhblubEyj1DwsPzNHkIWX016hs1P8AxM0m
liZdy2PC2PDeHqLsgzSIRencLF3vbOk/Jj/SZpNImJiF0kMexjHsWGaBb7seZl5ubh4Wbh4uWMfk
x/pM0iZRC6LKIb2NlNTxCEEiD8zSJ5W1ZfH0nzUfO0nz9B8/QfO0nztJ83T7nztAuLpp83R7nzNH
ufM0+583R7j16fcq9zmXuc+n3OZe4tS9zmXucy9x6tPuVCaOZHMij8mPzEIQmJlVE8UubmjZc6sN
+IttGITE8LLwttxOnWV48Sv3Lq9yv3Kyv3ObV7nM/c5tXufM1nPqKxMrOc52c7Hqfuc+r3Pma/c+
Zq9z5uv3Pm6z5us+drPna/c+dr9z5us+drPm6j52o+c/Y+cz5hzHzEfNPmI+YjnQuIj5mmi4ugXG
0e587R7nztH3hcbR7nztH3hcTQ/JlKLCLlYpSlxS7PDNh57LufoYTMzCbIQmzxPErQuJr+8z5vE9
2fO4nufP4nuLtHEy+pS7E8UuylLm4pSlLupSlKUuKUpUXNLsUPA8B7IJdJdC9CdelLtpS9OlHilK
UuKUeyl23/KV/wCh2xMvrri/5K8r0i9Cui/UQnpUylxUUpSlKUpSlxSlWylKXF33ZcUpfXUpcIux
5omUpS4pSlKMotRzFGzmKJlOY5jmOYpTmOYpSlKUpzFKX0UGptfTZRFKUoxdG4XoFs5NfLzcrnvi
lKUpSlKXNwn/AKjIPC1DZSiZSlKUpcobEylzeki5pSl2UuxMbKNmvtSXd/DWl6W/KTyKUpSlKUpS
lKUpcvzGXbS9CjZd9zSlzSlKXZdl2vFE+strSIcpDlOVHKcpyo5TkOQ5TlOU5TkOQ5DlOUjIyM5S
EZGRkOUjIzlIRkYkQhGRkzN8wzx6UITa/wDNYQhCEIQhCEIQhCZf/oaLDxS4pc3dc3pXbf8AJbhs
pSlKVnMczKys5ilZR6jnYtRzHOcxzM5mcxzFKU5ilOY5zmOc5znOY5jmOY5jmOYpSnOcxzHMUpTm
OYpSlKUpSlKUpc6tzysLci7UIpc3rUpS4pd12UpdlLsgx9S5gkQhOt7+hXoni4fn0WLrLauusTrr
ooZqzd+nzW+D3Ie2+he+D9Ax4e+dBD6U6dLvhOg9q6TH0mhE3UvorvvqVhmoosXevWwnonuuHiwu
NWETovcxE2TZB+GaPrwnpIMbxBYexbG99yxYeU/Eex5mx9BdKl3wa2PKL6W5nq4TMJ0LlO9Cl9Ov
S3Cw80o9s2ITi9K9y2r062PauksPC6FF5bmLEHhbHvX6Q+nCDRN12p72xPCxemn1b07sXTe2dOjE
4U0+WJlZQ1h4S9AvS3dS9anD8Vm4Rc3Yh+gvXmbidSdClOC/DovM3tzfMPoXqzZOvCDxwHhD2Tpp
GpC3PD8heRcTZNiLuvpqNwpwF4YuHunQT2XoQpcXc/Q0fSuNb8WJnZf0Xvmx+nYvQP0et+LOY7L+
i/x20T2IvobuYuldz60GiYY3Wy+Z2J3Rq/HZMJDW2bITL2TMJlIaxCbl62+L/Avmdgd06vxQtjNJ
VH0U0VTb9nRW2EzOjerd8NLqx3f+hq/FCw/QwnQhBiGswmIQhN8INEJmExCbJh7NDaQmzu51ahC8
h7oTCXQXTe59NdFsXU0+Qju39/8AsIo8XZSlLilKXKZdyKXbS9RlL6SmnyRzHdXjp4gl6Ol9ZNqX
Xaon4I5juh+HF/FeshMXpzoTqrDxCYWuzHc8nF/tvm6EJshCEIQhMweIQhCEJsmYTqQmUsvYkmxe
EQ2dyN3jf2FtRB7EJ9dbFueE9i6d20uaXYtUY/M+07j/AE+N+CE9qL6ClKXoXa+g/T6mU7k/T434
IhMQmYQhBomJhZnob6Z9KGrzKdxO8Tjfgi7KXFLijZS5RSl2Te8QmZmdebUh7IQmEa2+ZnMdw/rO
N+CxHmEITYxEIQhCEITCGib0TZETovL6T6COJays7g/Wcb8Fs+xDwxPFwxIfRYuimXZSlKUpRPF9
PxH4vHw/+t434ITZSlKUeLi9G74QhCE619Rr4jbdx3A/97xl9sRpVRDlIREIMRCE2rDwuo/8kZPF
/ghHw/8AtHG/pQmUo2cxcPCZSlLiiZcp5pSlKUpS7b6V5a6L82M+H/1/F/oQl4ExCMmHiEIQmUQg
0LqX/JGJpFrPh5/8RxP6BeWWXywx4WPt2opSrrLovqQnThManXUU+Hn/AMXxP+2JlZXml2URS4hB
l3Qg0JDRCEEiYg1hEJsmydB9VN4+HP2vif0GnSQiIQ5diREQeELDwhYexiLsWaXFLh4WylzcXrs0
2nOz4d/atf8AQLUUo2cyObbSjeUyvEFi7PHE6SITEIQmHiZm6dOiaQpT4cd7ZrX/AEGnT4HIcpCM
Wk5TlIQhCEJjSTKIQnRXWeKXdSlKXpLUn4iPhl/8br/7bNMi2VC23FKUYhMuKJlOY5ilGylLhPFL
s8TxzcvEzCZmITpM4KXKvwHfD2Phj9r4n9ArEVlZ44WIyEZCEOUgxEIQaEiEIQiIiIhBmkmIQgtq
y/L0D3Q4HEcX4I+Y/A+FI+2cT/tmnyIiEILpM07G0J7KUpUeBUNicKVFRSiZSlws3CfoFt4XgtP4
I5vI+FP2viP/AKBavA5ilKJlKcwmVnMysrwxFYmzx2RkZHmMg8JUg0QaIJEIQhCbEts2Ta8TbokX
4H+FrSfCP7VxP6TToOUhykIQhCEIiEIakLKGiHh0WafMey7qXZS5pSlKXa9+htz/AFSNN/wfiz4Q
/auM/wDpNEmyid2UqKioqOZGp0TFqKUbOZnMcxTmaOdnMVlZXnx2LaszKJiEFshNj3NnC+z8EJKI
+Ev2jif0M0txFZWV58cJMjIcrIQmEjlIQek5RaSIiOVD0kRCGlGoXnmEEQmVm5Qz7cXfSlHv4T8V
+CE5oTPhF3tnF/oZoX+FDSGIiIREXQZpGITHqOZYpSoepDaKii1DdE4cxTmHqOY5jmKUpSlLil3X
qw4as/pRq03QvdM+EfDtXF/pNOqaRvFRWLUcyOc5ilZzM5mVlY8XLz44rIyMgkco1OlS9a5u27eF
ZpX+gmp4Hwol9I1/0f8Ak0aXEcpDlOViRCEQkQaGhLGtYgj7Bjwl4D8zRlixr6l9Vwm7p/pRpUS8
Pc+FHe06/wCh/wD2KcqLmrZSiY2VCaKjVqTwtRzI5kPUilOYbFqaOZnOzmZzsrG7vpemui+l2dPU
1+CE+X82j4TX/FcT+gScQ7pHqZzsotTOZlZWUpcUuKUo+pc3Nyil2UpS9ClLi7KXPZv0U/tiOWpQ
+E012rV/R/5xxPIvSpRZpcNlKUpSlxSl3URSlKUpzL3KVHNp9znXuPiafc+boX7yH2ngr9/SfS+B
/M0j7Z2defF0fmfTuy/ztH5ofeXYl/H0/mfW/d9n0nh/mau9uw6X48fQPvvu1OPjpM+u+7l/HQ/i
Du6/rH+T+w/2g7BKtbH8SdgTk1i+IexPy5hfEnZWr8vUL4j7I5NOofxJwV/B1QfxLw/s4L/M4PZe
Npi1KeBwuxLStNbp3FwtPD4jaXnEfYji5RRPFKUuylOZFOdFRUc+n3HxdH3kfN4f3kfSOF99H0jh
/eQ+2cL7x9M4PufTOB94fbuD7i7bwfc19v4aPrThex9b8L7o+9tH2Ife6+4Pvd/dH3xq+6Pvnjey
H3vx54QXe3ab4w4ne/aX5OH1r2z+Z/8ACOJ3h2x+K47Qu9O2+H+8Nfenbm/1upfgau9O2/8A5Gs1
dv7Xr8+Nr/s4Ltvaub9fxP8A3M19p7Q4/man/cXH4tfNq1L/APZi4nF1amufU1L4s1vkbT0x04XM
pq5ravFmuvm8WrBPiKtNNTS3+cYmnw215r/wzVpcTb5m9LX5OjtcfvF+Kp56FV+H90KampJVV+It
PL/8U0zyenxUf5eB8trW/sU/+mNvTU/wX9jk/wAOp6fb/wDo054ezcNOi6lp/GM5ayHdGqa2/Zo+
l8H7yNfauBq/fX5j7TwP5iH2zsy8+Kj6f2b7OImPt/Z/d/kz6f2f7Gx95cH7FqH3lo+6z6y0S8rH
3n7aD6z1fyl+Z9Ycb7ukfb+N7o+ncU+m8Z/aPtHFa/SPpHF+8au0caeGti4uv3NXF1/eYtWv3Y9e
r3Hr1e5dRdS+1jbbNHDT0N5bPEjZyi0jRyMehj4dXkatDRysejXFEzk4n3Wz5PGf8HX+R9E7S7/u
OJ+R9C7Z9nZuL/7WfVfbtXl2fWfU3eD8H2XWLuDvO1cGC+H+9E/1F/ujR8N94pXk03x89Q/hnvLU
3fl/mafhjt3m9XC/MXwr296rq4nC/Omn4V4/JyauNw1+FNPwpxtPgu1aJ+AvhRR3tSr/ANDR8Lad
Hh9KUdv+H3NPwx2dLx7Rq/I0fDnZtP8AH1v+yP8AZvsVvzNYvhru9fvcX80f7Pd2yNcT8z6i7ut5
Nf5n1N3b9vBv4tn1T3fU/kL82Luzu9Jpdn0i7t7Av+W0C7B2Nf8ALcP8h6Vachpumj16yumnVqvm
xeZ9orsjI4adOq+QtLHpfscmr2FofscmufosXB4jX6LPo3Fb/QYuy8afoM+h8f7jF2Hjv9wfd/aP
uGnuzjfahd28UfdXGb89MF3Vqnmj6o1t35wu6l/MPqrS/wCIfVPC+8z6r4Huxd19n/1Pqvsn3WLu
3sq/cF2Dsq/hoXYeyr+Gj6J2f+Wj6J2b+To/IfA4C/haPyOXhfytH5E0fc0/kX/RHMznZWVlZX7l
eLthBdNdh7W/Lg6jT3d2v+TqPqztv8lmnurtr8+HBdz9qvlpF3N2j72kXc3H+/oNPc/FTr4uk+p1
/NF3PoT8eM/yF3Rwft4moXdfZ/vahd39mX2MXd/ZvZi7D2ZfuH0Ts/3EfReB/LQuBwl+4j5XD+4j
k0fdRyr2IujOhNmvyIyPZHiEIRkIyEIQhCEIQhCExDmY9RzFKyspzFKUuNA3BcR809W0RERERERE
RbniYjIyMjOVnKzlZys5GcjPls+WLQRkZGcrOVnKzlZyHKcpynKcglDVppp4aTvq4QhynIchynKc
pynKcpyo5SERykIQhMQm2baVZpS4vUmYTZCdGL1aykQWGiEIQ5SEJtnrL6yZnrp/kD9TOksTC9DS
lzSlLunThOsh7J0l0JvpcrNyyieFveF0p0plbJunVe9bmsLC6KFueyE6Uxd9xcLpva8TEJsmUvSv
qJYexi67QkTEIT0y9AvQL1s6S6V9OhvqQWH6OD6SH1Z6F9BkzCb4TE6SVy8Ifpp04TEIJE6E2XCG
8XN2XNLsu5YhMzZOt4YqPDorozYtj6Ks2Qn+dPNGXK3PdfV0pS4uKURSlKXFKUpSie6ieaXbR4pd
tLmlKXNLm9ClL0aUuKUpcUuxFKUpdzRSiLtpcUubtT20eaUpSl2UuKUpdt6E3whCEITesPNLtpcP
N6yIQmYTEIQhCEIQRNsIQm5Y1eRc0pS4pdzJ0JshCEIQhBLEINExCYW6ExCbJ0V0JiEIQhCbITe+
vCb4TcupNlRMTM3QSIQSITEw8QmITEJshCdRbKXCzUU5ilHqKUomMpzFpMwRSYmILM2vM6lKiopS
nMjnRSjZzHMUTZSlGylExlKUomXasNbYJEJ0KUpVllKc6OYpzHOznOc5xaimrUNlKUpduljxdzyh
Ma2rZMJ4eKVHMjnFqpzDZzIes52czKVlZSlFqZRjZcLL2LD6aw9l2wSEMmJtmUsQm1CH1EzV01h9
NENSJmExCHKQXQg805il2o+wfTWH00sQ5TlZCE2JCSJi4pcUomLFL0Ex7mTYswhN8EsMhEREzdiZ
RYbKUuyieGMu9YfQpRMe1IhN6PDDzc0pRMQx9BCHtWxCGPopj3TZpGt1Lm7Ll7FtQh7niiYh9JYe
yE2Ig10Vh4Txemh7bsQh9JD2zDysPpIfSoxi6iw+kjV0ULD2LFyhGpdLSQazCZhCYmETpIvgPooQ
x9FYfRQhk3QhDThkzCZbRzFLil6dL0tI+hMJYaJiEIcrIQS2eA4LExSnMcxS5TEMb3PqPE3TCQh7
YQmIeRUVFxSizRsTLmlNLxq6SeHunUeyEIRDhRMpcUostlKXemN9JCYx7ZtSHpJsWxCRBImEXcmQ
5i9JDfTXkPbKcpCZQx7ps0ko/Df/AP/EAFkQAAEDAgEHCAUIBwYDBgYABwEAAgMEERIFEBMhMUFR
FBUiMlJhcZEgMFOBoSMzNEJicpKxJDVAVHOCwUNQYGOi0QZE4SVkcIOT8BY2dICywkXxRtJVhOL/
2gAIAQEAAT8CCdmKCPpFD0h+2nOduY+i5FDMMxW5DZmCchmGcZih6u+cZyh6W/N9bM5N2Zt+YhBH
0XJvUzBNQRQRzlBFD1A9B2YoI5ihnGYoZt+c+gf2Q5yhmKGbfmKGbfmcjnGY5hmCdsQ2eiMxQ9Ae
tPpFb82/MU3MUM29H0XIdXMEEEUPQKajm3+rKCcgihmGY5zn3+pH7Ec5QzjMduYoZt+ZyOcegEU1
FDZ6BQ/aN+Yobc2/M5NzFM2IrfmOcq6+pnagimo5yhn3+kVv9EIoIoZhnGYoeifVn1p9AZih6BQ9
ByOcZygnJqKHoFDMf2MekNuY7cztibmKZmKB9Er6uduxBFNRzb0UP2AoIoZhnGcZjtQz7/RH7EfQ
GYoegUM2/M5FDMM5QRTdiKHojMUP2EoelvzHbmKZnbtzHOc7l9TOEEUEducoLf6Z9S1FNzDMUMxQ
zFDPv9A/s4zFDb6Bz78zvRC35hmbsRQ9EZih+y78/wBbMduYpp159/oXtmOYo9XOEEUEduYIoLf6
Bzn0T6DUUPRGYoZih6ZQ/ZhmPon0XI+gFvzDM3Yih6BQzFD9hKGffmKG3Mducbc7lf1Dur6DUUEd
uYIoLf6Bzn0T6AzDNvzFN9FyGc+gUP2IoI+s35nI+mMzNiKb6BQzFD1R9MoZ9+Yobczs+9DMc1/T
d1cwzBFBb8wRQzjMc59A5zm35hmO1BFDPvzOQzn0Ch+xjOPROYoeg5FD0hmYim5t/olD9l35im7c
zs5CbnPpXzP2egEUFvzBFD0TnO30DnObfm35j6goZzmGYofsR9AZjnOYoZt+Yooek3M3MPQKGYof
su/MUNuZ3obM59Q/Zn3Zigt+YI5t/qB6Rzb82/MUPTKGc+iPSPpj0TnKGY5zmKHoOR9AZihmbmbm
35ihnH7LvzjbmdnCOY+pfs9AI5t+YI+gPUH0Cgjtzb8zkPTKGc+gfTPrChnGffmOY+i5HMMwzhFM
zDNvzjMVf9iHolN25neifU71JszDMPRCOYIoeoPoFBHMczkM2/OdqCKGcoZz+xlD1BQRznM5H0Bm
3IIpmZvplDNb9gGffmKG3MfRJ9K2cbVJszD0d+YI5gih60I5jmKGzMUMxQRQzlD9mKGY+mPQOZyP
oD0Wopubfm3ZjmH7CUM+9WRW/MfXDapMwz78wzBHMEUPQPqCgjmKCKGYoZjmKGcofsI9IZ9/olD0
DmKPqQih6gfsJQznbmK35jmHrBtUmYZ9+YI5t+YIoegc59IIoJyGZuYoZih6LkPVn9kKGbfmOYo+
oKaih6Izj0j6wZztz78x9cE/M3PvzBFBb8wR9E5z6ZQRQzBXRTcxQ9E+sPqz6e/MUPQOY+qCKb6I
zFD1J9QUM5zFb0cwzH1Y2qTMMwzhHNvzBH0TnPplNRQzb84zFDNvzlD0Ch6R9cfQ35im+gUE5H0i
gihmHqB6k+oKGffmP7CNqftzNz78wRzb/SHon0D6Dcwzb84zHOdvoD0D+yn0im+gUNicj6V84RQz
hHOUPRHrR6J9AI5x6kbVJmGffmajnKHpFDMfQPoDMM2/OMxQzH1BQ9I+uPoHNuQzhOQRR9MZgjsQ
zb84zH0h+xFH9gG1Sbczc+/M1HNvRQ9AZjnPqBtzb8x2+kMxQ9Mofslroi3pDMczkNiKPpjOUM29
DMMx/YBmPpHOEfWjanoJqKC35mo5t6Oa2cZxmPo6ILRBaJq0IWiC0LVogtAFoQtCFoFoVoO9cn71
oO9cn71ybvXJzxXJ3cQuTO4hcmdxC5O7iFyd/cuTP7lyeRaCTgtBJwWhk4LQydlaGTsrQydlaCXs
FaGXsFaKTsFaN/ZKwP7JWB3ZKseBVjw9IGycdfqignI+rCuFcLUtSGcfsV7K9/VW9WNqftQQRzb8
zUcw2o5t+Y5yhmP93WHBYRwCwt4BYGdkLAzsBaOPsNRii9m1aGH2TVoIfZhcng9mFyan9muSwdhG
jpux8VySDgfNcip+9cih+0uQQ9py5tgP13rmuH2j1zZH7Ry5sZ7V3kubWe2Pkubf87/SubP874Lm
x3th5LmuT2zfJc2ze0YubZj9diiomM67cR8UGgbIwi1nsh5BYGexB9wWji9gPJaKn9gPJaCm9gPJ
cnpvY/muT0nsvzXJqT2f5o01H2T5lclo+/zXJKPifNcjpfaHzXIqb2p81yGn9sfguQQ+1K5uj9sV
zYPbHyXNn+d8FzW723+lc2Se1b5I5Ml9oxc2z9ti5tqeLEcmVXFnmubar7J965tq+wPNc3VfY+K5
BWex+IXIqv2BRo6v2DlyOr9g9clqfYP8kaef2L/JGCX2T/wlaKTsO8lY9k+S1K4WJvFXHH027U/a
gm+i3OEfTPqLq6xK6urq6xK6urrEsSxLEsSxLEsaxrEsSxLGsaxrGtIFpFpFjCxhYwtIFpAsYWML
GFiCxBYgsQWIK4VwrhXV/wC47BWCsOAWFvZCwM7IWCPsBaOPsBaNnZC0bOC0bO/zK0be/wAytG3i
78RWDvPmsP2nLAe2VgPbKwu7ZWF/aXT4/BfK9oeStJxC+U+yvleAV5OAXT7I810uwFb/ACh8FZu+
AfBaOHfTDyC0NPvpW/hCMFJ+6t/CuTUP7t/pXIqA/wDL/ArkGT/Yn4rm3J/sz5lc3ZP4H8SOS8nb
3u/EuaaD2zvxLmmj3TP8wuZqc/27vguZo/3h3wXMjf3g+SORT+8fBcyu9v8ABczze2b5Lmio9oxc
z1XajXNNZ9jzXNVZwb5rmut7A80cnVo/svintcw2c0g9/oD09MtMtMtMtMtMFpgtMFpgtMFpmrTN
WmC0oWmatK3itK3itK3itI3itKFpG8VpG8VpG8VjbxWkbxWMcVjHFYgsQ4rEFdXV1cK6urq6urq6
ur5rq6urq6xFYjxWI8VjdxWM8VjdxWkd2itI7iVpX8VpX9paV/FaV/FaZ/FaZ/Fad607+5cof3Ll
B4LlJ4BcpdwC5SeC5V9lcp+yuUDguUt7JXKh2VyocFyhneuUs71ymPvXKI+9cojXKI+K5RFxWnh7
S08PaWmh7a00PbC00fbCEsZ+sFpGdtvmsbe0Fjb2gg4bnBXHFauObf6Gv1A9Zq4Kw4BYG9kLBH2A
sEfYCws4LC1YG9/msA4u8ysP2neaw/bcrHtlWPaKkhbK0tebgrmmm7b1zRD7V6ORme3PkuZeFR8F
zK7248kciy+2b5Lmab2rFzPU9ti5pqvseaOSqzst81hbwWBvALRs7IWjZ2QtEzshaKPshaKPshaG
LshaKPsBaGPshaCPsrQRdlcnj4Lk0fZXJ4uyuTRcFyWLguSRd65LH3rkkfFy5LHxcuSR8XLkjO05
ciZ23LkTe25cjb7QrkX+YVyP/N+C5E72vwXI3+1+C5FJ7ULkc3bC5HP22rklRxC5LVd3muS1Xd5r
k1X2fitBWdj4rRVfsz5rR1ns3LBV+ycrVXsnL9KH9m7yWKo7DvJY5+w7yWlk4HyWnfwPkuUO4LlB
XKVylcp8Fyody5S1cpauUtXKWrlLVyhq5Q1adq07Vp2rTtWnYtM1aZq0jOK0jVpG8VjbxWkbxWNv
FY28VjbxWNvFYxxWMcVfvVxxWLvV1dXV811fNdXz6s2LvWM96xu7RWkd2ytK/tlaaX2hWlk7ZXKJ
e2uUzdpcpm7S5TNxXK5FyuRctl4Bctk7IXLndlcvPYXLv8tcu+whXjsLlzOyVy2PgVy2PgUKyHvQ
q4e0uVQdtcpg9ouUwdsLlEPtAtNF2x5rSx9tvmtIztBYhxCur5tfq7q/qh/cVyrnirlXOf3BauAV
hwCIb2R5LCzsN8lhZ2G+SwM7DfJYI/Zs8loofZN8loofZM8loKf2LFyal9g1ckpPYhcjpPYhchpP
Y/FcgpPZfErm+k7LvNc203GQe9c2U/tJPgua4fayLmuP2z/Jc1t9sfJc1j258lzV/wB4Pkuav+8f
Bc1H94+C5qd+8DyXNT/3geS5rk9u3yXNUvt2+S5qn9uxc11Xto1zbWduNc21nbj81zfW9qPzXN9d
xj/Eub6/7H4lyCv+x+JchyjwH4lyLKPYH4lyTKXsviFybKPsD5haDKHsHLRV/sHrBXfu7/JWrP3e
TyWKs9hJ+FY6n2D/AMKM0g2xu8lyk9n4Lla5WOC5Y1crYuVsXKmLlTVyli5Qxadi07OK0zOK0rOK
0rOK0re0tK3isbeKxjisY4rH3rErq6urq6urrUtSuroPPaPmtK/tu81ppPaO81p5fau81yiX2rly
mX2hXK5u2uWT9v4LllR2h5Lls/2Vy+b7C5wl7LFy9/Yb/hsZrlXPFXdxVzxV3cViKuVdauy3yWFn
YZ5LRQ+yZ5LQQewj/CuTUv7tH5LkdH+7R+S5BQ/u7VzdQfu48yubcn+w+JXNdB7I/iK5poOy78S5
no/8z8S5mpO3L5rmWl9tIjkWDdO9cys/eXeS5lH7yfwrmX/vX+lcyP8A3v8A0rmWX96HkuZqj94Z
5Lmeq9vGuaK320XxXNVd7SLzXNeUO1H5rmzKP2PxLm7KfZaf5lyHKfsfiEaTKQ/sCtDXj/l5PJYK
793k/Cv0sf2En4ViqfYv/CtJOP7J34Vp5Ow7yWnd2T5LlP2VyocFykLlLVylq5SxcoYtO3/wDurl
XKur5tXBWb2QjHETrjb5LQwexZ5LQU3sI/wrk1L+7x/hXI6P92j8kaGhP/Lt/btf/wBglv8ADg9L
cgLrCVYrCVY/+BY1KesmqHlxeQNzRuWN/bd5prpfav8ANGWdp+ef5qir5ceGQ4mbO8X3q3+AT/fl
w0Ek2AFyhlSEn5t4b2lqPpTTQwAGR9r7FFPDM28b8XocfBDYE3U4FSPYG9Ea0UNSO30Nf+OZIxJF
Iy9sTbJtFXNdg0ertbkxoYxrNuEAXQ9HKtPLJopGMJsLGyoIZWTOkLS1oZbXv9AK23uK1I5xsb4D
0MszyiWKMamYbrJVRIZTGTduG/gh/j+59EbVLqll++780Hbiha+1HagoddPD9wehNBDOzDKy9lDS
08DSImWvv3/+A1fk2XSulhbiDtZHAo0lSNsD/JaCf2T/AMJWhm9i/wDCVBRVEzrCMjiSgA1rWjY0
AD0La/8AwIueKueKBPFEk+tv/cd/7w3f3jf+43X1W4+hjKxuWN3FY3LG5Y3LG7isbuK0juK0r+K0
juK0r+K0snaWlfxWmfxWmk4rTScVpn8Vp5OK08i08i5Q9coeuUvXKZFymRcpf3LlT+5cqfwC5W7g
uVO4LlR4LlR7K5V3Llf2Vyv7K5X3LlY4LlTeyuVt4LljOyuVs4LlbO9crj71yuLvXK4lymLiuVQ8
VymHiuUw9pcph7S5RF2lp4u0tPD20Jou2FpYu2FpIu2EJI+2FjZ22rGztjzV29oeauO0Fq7QVxxW
riPQ1KysrFYSrKxz2zW/wDiKxOWN3FY3cVjcsbuKxuWNyxlY3LG5aRy0jlpHLSOWkctIVpHLSFaU
8AtKeAWlPBaXuWl7lpPsrS9y0n2VpPsrSjsrSjsrS9y0o4LSN4FaUcCtK3gVpR3rSt71pGd60jO9
Y2cVpI+Kxs4rGzisbOKxs7SxM7QWJnaCxM7QV29oK7e0FdvELVxC1cQvJeWeyt3KysrHgrFa10u9
Xd3q54ldLiVifxKxv4lY5O0VpJL9YrSye0d5rTS+0d5rTze0cuUTe0K5TP7QrlM/tCuVVHtCuV1H
tCuWVHbXLKntrltR21y6o7S5dUcR5Ll9RxC5wqO5c4T9y5xn4NXOM3Bq5xl4Bc5SdgLnJ/YC5yd2
Aucz7Nc5/wCUuc/8pc6C/wA0udGeyK5zj9mVznF2Cuc4eyVzlT9krnKn7LlzjTcHLnKm+0ucKXiV
y+l7RXL6XtFcupO0uW0ntFy2k9ouWUvtFyul9qFyqm9qFp6f2rVpofat81pYfat81pIvaN81iZ22
+axM7YV29sLV2gveM1lZWVirFWKsVYqx9Xoe9aI8VoStEeK0R4rRHitEVojxWhctE5aJy0TlonLR
OWictE5aNy0b+C0b1o38FgdwWjfwWjfwWjfwWB3BYH8FhdwWF3BYXcFY8FhPBYSrHh6FlZa/2e54
lXPFYncSsTu0Vjf2isb+0Vjf2itI/tlaR/aK0knaK0knbWlk7S0snaWlfxWlk4rTSd3ktNJ3LTP7
lpn9y0z+DVp39y0zuAWndwatO7shac9kLTnshaf7AWn+wFpvsLTfYWmHY+K047K0zOyVpmdkrTM7
Llp2dkrTM4OWmj+0tNH3rSx960sfErSx8VpGdpaRnaWOPtrHH21jj7YWNnaCxM7QWJnaCxM7QV29
oK47QWriF5K3grKywqxVjm6XEq7uJWOTtO81pJe27zWlm9o7zWnn9q/zXKaj2r/NcqqfbPXK6r2z
ly2q9s5curPalcvrPaLnGq7a5yq+2uc6riPJc61X2VztU/ZXO1R2WLnafsMXO0vs2rneX2Tf2+3q
bDgrBWHBYRwWEcFhbwWFvBYG8Fgb2QsDeCwM4LRt4LRt4LRs4LRs4LRMWibwWiatE1aFnetE1aId
60I71oRxWhHFaEcVoRxWhHFaEcVoe9aHvWh71oTxWhPFaE8VoTxWhdxC0JWhK0Lu5aJy0TlonLRP
WictE5aJy0T1o38Fo3cFo3cFgfwWB/BYHcFgdwWA8FgdwWE8FgdwWE8FhdwWE8FY8FZWVj6ACsOK
1ZtSsrKywrCrKysrHirHiulxXS4rp9oq7+0Vd/aKu/tFXf2ljk7SxycVjk4rSSLSSdy0knctLJwC
0j+AWlPZC0p7IWlPZWlPZWl+wtL9n/BNv7ksqenpzBEdCzqjcpeQMOHQsc7gArUO00mr7oTaSheL
thjI7lyCj9gxc30P7u1c3UPsAubaD2AXNdB7Bc1UHsfiVzTQey+K5ooPZnzRyRQezPmuZ6HsO81z
NQ8H+a5movt+a5lo+L/NcyUnaf5rmSm7b1zJT+0euY4favXMUPtnrmJnt3eS5ib7c+S5i/7x8FzF
/wB4+C5id+8DyXMUvt2+S5jm9sxcxz+1YuZantsXMtXxYuZqz7C5nreDPNc0VvYb5+jbUtXFauK9
4Vu8L3r3hW7wrd6t3hYfBWWErCVYrCVhdwWErC7gsDuCwu4Kx4Kx4Kx4KxVjwVjwVjnt/gsTmPJb
Xjbox/sqUEGUk7VNKA+7XOVBUOif3ONreKxLEsSxLEsSxLEsSxLEsSxLEsSxLEsSxLEFiWJYliWJ
YliV1dXV/R3DxWGysFbuCcN6srKwVlYcFhHBYQsI4LAFgCwhYQrBYQsIWFYVhVvHNr4la+0V0u0U
S/tFYn9srpdoq7uK18V0uK6XFdLiulxHkul3eS6Xcrv7vJdL7Pkru4N8lc8GrpcGq57LVr7IVz2Q
v5Vr7K/l+Kuex8Vf7HxWrsfFauwfNXHYPmrjsnzWJvYcrt7Ll0ey5Xb2Xq7eDldv2l0ftK7e/wAl
dnf5LocT5K7eJ8l0e0ujx+C6PaXR7S6HbC6PaC6PaC6PaC6PbatXaC1doLV2gveF7wveFbwVvBWP
crFYSsJWF3BBrkQ7gsLuCseCseCseCseC18FY+jbO0OdkkAC5wj81FJhPc4JwsSoQTJHYfWCIN9m
9YSsJ4LCeCseCseCseCsVYqxWta/W3V1cq5V1cq59EC/mt7eOqyfA11M6ScCN+7XtzbW+9YVhKwr
CsKwrCVhVlZWWFYVhKsrKywqysrLChmsrehb0LZ7K2a3qLZ7ept6FlZW9CysrKysrKysrLCFhHBY
W8FgCwhYG8FhCssIVgrKyt4q3jmwuw3124qx4la+K19ooteNTrjuWvtFa+JWJ/bKu7isT+0Vjk3P
K0k3tn+a0tR7d/mtPU+3f5rlFT7d/muU1Xt3rlNV+8PXK6q3z71yyr9u5ctrPblctrfbFcvrParl
9b7Vc4VntFzjWe0+C5yrO0EMpVvaC5zq+LVzpV/ZXOlXwYudajsMXO0/s2LnWo9mxc6z+xYudZPZ
MXOzvYtXOzvYBc7f5C53H7v8Vzuz93K53j9gVztF7F3osGpG4LbbQmnHINJIde0qWKjEbnMncSN1
kNn7FZW9C2aysrKwz2CwqysFYKywhYVhCwhYVhCsFhCwrAsKwLCsCwhYFgWBYFgWFYFgWBYVgWBY
VhWBYSsKwrCsKwFYSsJWFYFhKwFYVhWFYVhVlZYSsKoKp7vkixtmR3B8Expr5jJIMLWM14VU0rGN
ikYXYHGxxbQqqiigZMdML6sDd65tg0mA1Gtw6A3oZNGB4FQDM3duUgPNTDi2SrmyISYX1A1joDeU
yAafRSOw4b3d4KppWxRxyxyYmONllQfpY/hjNZWVlb9iA9Rb1cQuEGLD3KywqyssKsrKywqyssKs
sKwqysrKytmsrKysrKysrK2aytnt+w2/uWic1sxLiANG4KhlbGHsc4txtAuNxVVi0YaanSnWdtwq
4tdUSkEG7Qp3N5dTEEWszX71TOaK6oOIWIfr96//AIZh/wA7+iqyDXUxFtjPzTCzl9TrbiscBPFV
cj30QEkgc8Ta/JZQsahh/wAkJ0rTTNh0IFrdLerZrZtXoWVs1lZWCsrKwVgtSsrKywqywhW1Kywq
yssKwqywrCsHo5KHy3uKsOCwjgFhb2QtGzsN8lo4+w3yWhi9m3yWgh9kzyWgg9kzyXJ4PZM8lyan
9izyXJaf2LPJclpvYt8lyWm9k1ckpvZNXJKb2QXI6b2QXI6b2QXIqX2a5FS+z+JXIab2fxK5DTdj
4lcgpuyfMrkFNwPmub6f7XmuboOL/Nc3QcXLm2DtPXNsXbf8FzZD23rm2Ltv+C5ti9o9c2Re0f8A
Bc2M9q5c2N9qfJc2D23wXNf+d/pXNf8An/6VzWfbfBc1u9sPJc1ye2HkubJfat8lzZN7Ri5tm7bF
zZP22Lm2ftMXNs/FnxXNtRxj+K5tqOMfmubqj7Hmub6ng3zXIKnsjzXIar2fxXIqr2XxC5FVex+I
XI6r2J8wuSVXsD8FySp9ifguS1PsHLktT7By5NU+xcuTVHsXrk8/sX+S0E/sX+S0E/sX+S0E3sX/
AIStDL7J/wCErRv9m/yK0buw/wAisDuy7yVu4+S9xXuK1K4VwrhXbxV28VdvFXbxXR4q7eK1LVxC
1cV0eK1cQtXFavR1Z7KysrKytb07Z7ft4WSvnvd/guysOCwjgFhHALCOAWFvZCwM7IWjj7DfJaKP
2bfJaGL2bfJaCH2TPJaGH2TPJaCD2LPJcnp/Ys8lyWm9gzyXJab2LPJckpvYM8lySl9gzyXI6X2D
PJcjpfYtXIqX2LVyKk9i1chpPYhchpPYhcgpPZLkNJ7JcgpPZfmuQUnslyCk9n8SuQUvs/iVyCk7
HxK5upOwfNc3UnZPmubqXgfNc20v2vNc2U32/Nc2032/Nc2U3F/mubKfi/zXNdP2nrmuDtv+C5rh
9o9c1w+0f8FzVD7R/wAFzVF7V65rj9q5c1R+1cuame2cuam+3d5Lmoe2Pkuah7Y+S5q/zvguaf8A
P+C5qf7b4Lmp3tvgua5Pahc1y+0aua5rfONXNk/aYubKjixc1VPFq5qqvsrmuq+x5rm2q4M81zbV
cG+foBZK+e881dK+KDE068QQilsP0h3kE5sgisJdfaKmcWQSOB1hhVM9z4I3O2kK/ovmnNZoWOAG
C97IMqQdczT/AC5jsKo53TwY3WvdRaXD8pa/cmTNe+Rg+pt9B85ZURxkanjUe/NFU6WaZgbqZquo
3PdixMw9LV6mSZ7XECB7u8JtcHFwEEpI2iyik0jb4HN7j6iGZkzMTNnoySsibie6w9B80UfXe1vi
Vyqm9uz8SZLHJ1Htd4HM5zWi7iAOJ/wvkr57zzZT+jfzhNfU2HyLdnaVeXcifcWOpTfQ5P4R/JUz
8FCx3Bl1Tuie0vmY57ieySPcqUltQ5rGPERGq42H0JHFuUrht/ktibLIXWMDh36lWSOxwwNdbHtP
cn0jQx+Bzw7CbHEVkzXS+8rJxdJSnET1iqaBpmqW4n9E8UNQz17CYMbesw4gjO3k2m+zdZPjLKcE
7X9JUb3u0+I3tM4IzOmnkbptGxm/eShOYahjNPpGv8wUyV4rZInO1Ft2qpq5WTkMPRZbGq2d0VPi
YdZIsm3wi+23oUWqprfvqoqHCRkMVtI7juCfJVwAPkLHMvrwiyqqgxRMe22tw8lG+rkwvsxrDuPW
spKqVtVoWsBu3UpKmqp8JmEZaTbo3WkrnNxsjjtuBvdU04nibIN6optNDiwgdLchWSukkjZDdzXH
fqshVSNlZHNEG4+rY3UrpGt6EeM8L2TquaKxmp8LL7Q66yn9Dd4hDYPDO5jHdZoPimMZznI3AMOC
9rJ2GFhLIr9zQjWlmuSnexvaKyjro5P5fzTawYW4YZXi20BQzMmbiYVy+PXgileOLW3Cik0jb4XN
7nC2ep+jT/w3Ki+iQ/dWVfmI/wCIESBrOxc4UvaP4Sqgg0sxB/s3fkqH6HD4I5QpA1rtLtUM8UwJ
Y6/+CMlfP+45spNLqawF+kE3YPBZQBNI+3cpR+hvH+V/RUzMVExh3ssqaQUzNDLqtsPFRzxSEhjr
+g97WZSxOdYaLahU05/tWqtYWywT7Qw9LwT6uDA60gJtsG1ZM+i/zLJX0X+YqB7Y6urD3BtyCLoa
85AIsUcQAo/84fhQFtSodtV/HcoI4hVTxysBJN234IU9MCCImX8FW/Jy089ths7wKpodPDUucPni
fJRSGY0kNvmzd38vo0v0ut+8FURtNezS9R7dXiubaP2fxKyo0No2AbA4IbAv/wCK/wDkrKo/Rm9z
whsCyV9GP3yslfR3D/MKofpFb/EVd85Rn/NCqKiRsscMQGN2u52KvFRyV2kcy1xsCrjfJ7j3N/NN
6rfD0Lf9qf8Akf1VTU6EsaG4nvPRCrjUmkfjYwDVvU36pH8Jiprcnht2Aqfo5QqmDZYH3qOOoowQ
xmkjvqH1goJ452Ym+8cM8/zMv3CqSGd1NEW1JaLbLBZQjmZC0vnLxi2WssquIpfFwusNfgwYKfDa
1taiglhoZo5CD0XWtwssn/Q4fBZJaBS4t5OtSjR5Rgc3+0BDvd/gcLJXz/n6yywt4DNgaPqhAAbE
ABsFk6ON/WY0+IQ1ehoo9JjwDFxzNY1t7DadakhiltjaCmUlOxwc2MAhSRskbheLhMY1jQ1osAm0
8TZDIG9I7T6LYWNkkeNr9qlhjmbheFyL/vE/4lPTsnjwOvZBGBvKdPc3w2sp4GzRlhOampxTsLQb
3ddU9OIGuaHXu4lQ04ikmfi+cN/BT0+m0XStgeHKemMj2PZJge3fa6kopZmkSVJPCzbKWn0lLocV
tQF/BNGFrRwHoGB3LBNcW0eFVNO+R0UjHAPYd+xTQVk7HMe+No7r6ynU7zRaG4xYAPJMirYWNYwx
uAH1tSpad0ekfI673nWr5Q7EPmVS05i0rnHpSOubbBnkBLHgb2lUcb4qaNj9ousoQSTU+FgucSqI
BPAYz/7KbJXsaGmmDyPrYwLr5WSndjZhcQeje6oY3x0sbHixF1QuqWQHBEJBi2XsQo46iWqbNKzA
GDotvf8AwRko/pA/wPQwSQxOa+3XJ1f4JjcW7CtO/tu81yiTtnzXKJO2fNcpl7bvNcpl9o7zXKZf
aO81ymX2rvNcpm9q7zXKpvau81yqf2r/ADXK5/av81yuf2r/ADXLJ/auXLZ/auXLqj2rly+o9qVy
+p9qfgucKn2v5LnCp9r8Aucar2nwC5xqfafALnGp7fwC5xqe38FzlU9oeS5zqOI8lznU8W+S5zqe
LfJc51P2PJc6VH2PJc51H2PJc6T8GeS51m4R/Fc6y9li51k7DFzq/wBm3zXOrvZDzXOp9kPNc7f5
P+pc7f5P+pc7f5PxXOw9h8VzsPY/Fc6t9ifNc6s9kfNc6s9kVztH7Jy51j9k74LnWH2b/gudYfZy
fBc6QdiT4LnSDsPXOcHZeuc6fg/yXOdN9vyXOVN9ryXOVLxd5LnGk7R8iucaTtnyK5xpO2fIrnGk
7fwK5wpPafArnCk9p8CucKT2nwK5wpParl9J7VcupPahcupPbBctpPbNXLaT2zVy2l9s1cspfbs8
1yul9uzzXKqb28f4lyqm9vH+ILlNN7eP8QXKqb28f4guVU3t4/xBcqpvbx/iC5TT+2j/ABBcpp/b
M/EFp4PbM81p4Pas81povaN81pYvaN81pI+23zWkj7bfNaSPtt81pGdsLGztBY29oLG3tBY29oLE
OIWJvaCxDiFccVccVccVdX/uwbD6G4+HoXV81/Svnvmur5r+jdXV1dXV1dXV1dYldXV1dXV1dXWJ
YliWIrErlYliWJYliWJYliWJYliV1iWJYliWJXWJYliWJYldXWJYldalccFq4LUtSuFq4LUtWbUt
XBauC1cF0eC1LUtS1cM11dYlfvKxHiViPErGe0fNaR3ad5oSv7bvNaR/bd5rSye0d5rTSe0f+IrT
S+1f5laeb20n4itPN7eT8S5RN7Z/4lymf20n4lymf20n4lymo9vJ5rlVT7d/muU1Pt5PNcqqfbv8
1yuo9u/zXK6n27/Ncrqvbv8ANcsqv3h/muWVXt3+a5bVfvD1y2r9u5cuq/bOXLqv25XL6v2xXOFZ
7YrnGt9r8FzjWe1+C5xrPaDyXOVZ7QeWYDO3Yc7uCb9f7qFkRrIRFluQBK1ZmYCw3vqRwfVv782w
ZtQ2oiyt0boa9iurNwA4tqNtxuralcW26+Gb88xFrd/oH1J1NHemfW8FY2zC52Z7ZsDrXstebd6Q
1n9q8UfQtqJvm2bUdSC3XCGsrDuG3Pf0r+jf+4G9U5mcczNj/uZpOsVL1/cPyTeq/wAE3qSeA/PN
J1z7vyTepJ7s79tuCG0eKeI9I7pO2oluADWdav8AJW+2oz025j82PHMD0H+IzN+t9w5nfUPcn/2f
DRhN1h4+zdNOvxQHTsdxRNzf1DdbgE83P5KM9J38NyabG6k26t4Ump+FX6B+zb4odW/fZMOu3FHe
nOdo2dI7SvehrKcbu7k362q9hsTcdx0H24WUgwvspT8o7DuJRHQaVDcSeKL5DtcmddviE7U92E7z
mddhsNoW1tx71/ZfzIdJwBPgmEX7kdSfqNuGdjcWLuCxX2/DNixMdcN1W2ZsbuDfJPPRaCBfuQwh
pcfBMOM21X3WXuurt9k/zC91lwHBrfyX9n/Mm6xh37kNWsjMNbH+Lcz+u7u/2X9n4OX1R4pm0juK
bqx/cKBsR4p4s8j+7W9U5r9G3fma61+8IHWibklYuPmsWqyDiFfXdXTHWut6abHOXB23bxRI3BE9
C32k11nDMCMOHvzDqu92YGxXQ77Jzrnu3J1rRjfowsVmm205j1A7edXlmYGE9I6s7wwYcLr6teZo
YWvudYGrM02Bd7hmj6x+45AKQ3Jt4J5xnF8FqDD32+CbrYW370wa7nYFfWtsY7idWZuoOd7h78zD
1hfrCytJitcojpe9SfOyfeK/sf51G4B7bqQFrnX4pnWb4hO6z/vHNJrcX7itjDfeR8EBeM/eUTem
Hbm60NiGstf5+70BfaNyaceK/DarpvUk8G/nmFgMXlma46Nwba+ooPfe4A8lq3hfJezP4ytW4W+K
cNh4gfALZHr3n8k1uJPOku4eWYfNv9yKc03xcVqDB3klbWeDkzUXHg0puskcW2zPN3u/u0bD+2k9
T7tsw8U517cP2Mejfo4d2a99aubWzX1W3el/ZH7wWI7FdXLWW4n0ASEXE6tg4ZtKQCMLLHbqRdda
S7QC1ptsuFf7LR4BXsbhY3W2+PoBx18DuRJO1X1EcSg6xRV9WYOPu4Im51oFYjs3Zsbv+v8Adw2H
M4dX7o9EbfQeLH9sAv8AtAF1ax1rac262bYc1tV/8EjYc0mrR8cAXWa7iFHrewd4TtRNs0brOaLC
xKO0+KYAb32DaipvnCndAt1C9t6c5pjBDGjXYoAaydgQIJANrcU5pa4hGzQ3VckI2Q6zdVxdSObj
f8lv4obQnjBIRtCe2waePqArNJtrvx3Jws4t4IM6NydScG/V2d6ay4eRusmgOsB1swawg2cbhpNr
epAJWjI2mx4ZsHFwHDvRaW7QrHDi3a/gi3Vfb4Zi75IN+3mjj0jHOu3VxKc3D9Zp8Df0BE/Q6sOs
6zdFuE2Ucb3WLbbUVY7VhNr5g1xbdP6kfoP6sXgb5gxxBNtmdzHN2hW2qxtfNgkDSS0245mddvin
dd33j/fLdhzSDVEf8sKPU15+zbzUXzkf3gidvjmb12eKd13/AHioxfSDi3+qsVN84sQDBibiGuyc
b6gLDcoz0ZNQOxY/8tnknOLnEnesbrcfFPDLBzeNrIbQpfnX+Kb1gpvnX+KP0cff9QwYnWvuug+L
AbsJ271N8/L97+if1Yf4YzM+bm9yG0W4hTfOv8VHtd9x/wCXqYTbSHg3VnwNx2e+x81IBhis7Fqd
r96udA0f5rvyUA6duLT6DLaObVuCFuHoWGgI3aT+mZo+Uj++1Ha7xKYbRTe5RnC9pHEfFWGnw7tJ
ZPcXSE+SkIMcPgfh6Wjc46yARsubKe9472vgGsLFhjZh2uJv7lusmamTfdTOi66kbaR44OTRqm/h
/wBf78bsOYkq6BsQc7ZLfUanPv8AUb4q9li37+KLrm5ROC7MNxfetRY8iMDVtQJBWkPdfjmxatYu
i4nMX4usNaG0J7vlZL6+mU52K2aaUyuBI3ZmyFrJG261swNiCNylfpHl1rJjyw3COH6vDyT3YnOd
xTX9HC5tx+ScWWsB71G4tZMfs/1QcxrsVt+ocETfWVHoQdch6p3cUWsAFn392aUxE9BuEZg6LRPu
3p7jmbhxtxDVfWnYcTsOy+pMfgdfzVoTsdYcDtR7k/CXl4dqJ1jenEYIwN2L80fmmj/Md+SpyBK2
/oRloxYthCc0t4eh/ZEf5g/LMyN5LSLWxDepY3B8h1Yb8UPmpR4fmt7fvBP1TvP2yU5rdrTq39yf
hwRAHZj9Hh4hSsBeXC1iesn2ww23MshgdHhJDSDqPijGWjpG3DvUWG01/ZFMYMV3EYARc8U52J7n
cSmRSfKdA/NpzHt6zbf32M9jb09eYPI4HxVz6lxxOc7ib+uv0XjiP2S/7SD0Zv4R/P8Av9uaNmJw
1ar609smvUQOGbCbXtqzNa47Gko7T45i5zohf2n9M5zy9f3BQgGVoOwrb/j5uaH55niiTruShtTW
TE4h/wBFMLSuCBcCLOI1p/zsv8R2b+y/8wflmcyQNZhYdYviCwuMchcwiwvcqMC7iR1WkrGbjFsV
TbTG3AKH51nj/j8ZovnWeOaE9P3Gyw7b+9T20ptwCDXG3RNrqRrtLN0T13ZsJ0PVPzo/LN03NY5p
2NAI8ERJhJubKLXjbfrNICwO6tlU/O2+y1Rgl7bbb/4/Ho3zB726g4oyPItjKaSDcFaaXtuRJO0r
gnEuNzmxG1r6le62f4/Gz1Qa52wJwLTY+ttqv/jM+iM1OwPftGpOjaBfSsPvQaSHEbkWdFrtxRYR
bv2LDqaeITmFuvaOIWq6MTjAAHMALjfWnMwm2r3JrHHcnRubt+HqpOi2Nm+1z7035io/k/NFjrXt
q45mxude27O5j29YWzaCTACGO2pzXN2gjNhGhL+JsFB9Ii8VvPii0jarXNlszWJ9MvMkEl2tu17N
g45y1oijP1nflmcMLrHu+Pp9COFkhbic46uCJudlvRiAMsYOwuR1OcO/0yCGtJ339MiwaeN/h6DG
Y76wLC5JRjY4Exu2C5B2+g1kGjjc8vu6+zuT9HfoYrW3pjC91gjowLXcTx3ZhgbFpDrJJAG5bSgC
42Cla1r7NNxYa/H0Nlr7xf03dE27vQAAaHO37AnNFsQNxe3hnb1v5Hfkm6wFb1Y+bd970C13ZPqR
mpfnh4HNGehP9xD5iQfaav7Fnc4p7vkqdu7CfzVOMTnM+qWlbQEQOTN/iO/JbE0h8WjxYTe/cnMk
jbb6p2cD6mFt5G32DW7wCc8vcXHeoXBsdUbX6A/NMmfG/Fff0hxUjA2pMY2Yx8VUPOkdG3U1uoWT
3aSFzndZhHkUw4IXPt0i/CPJRynGA7pNc4A3T24XvbfYbLSSCkjAe4dMhFzj1nE+KGsgcSp7BzWD
YxtlTfSIvFCMNMkj+qHkDvT3l7rlYZWRxGOI9Jty5YJHwzGSOxbrxW2qm+kR960jmn5M2H5oHXfv
VU7BUv0eo7SfFPs+FstrHGWnvUTfknyaPEcWEKHSTSYJI92p1rWW8+Kb8xUeMf55mNLntbxKmcDI
bbG9Ee5YnN6Q3Kpqpm1Dg3DbVu7lTC87Ad91G5jmytcNgc5p8F16e+9hA9xRwcljdbpYngJny+Jr
gMeHokdyDjqITqqQU8JwR3cTfVwTnFxJNvcjaKOJ7WAl17uOxPcHW6Njvzxm0sX3whHilmLjZjXm
5UjwTiw2AGoI2hwdG8hF9ewBODHxF7RYt6w9B9R+j0/yUWvFtHBFxcSSGjw1LoRsjcRdzm3F9isy
WGSUNsWbeCiwG7H/AFth4FMjLMb5W6mbB2ijdPmPJ6c6KK5Ltyc4uNzb3akJIQNcDj/MnOjI6MLh
34kD8hN3yMCgNp4/G3mr4JTq2POpGDE8GPqO134KRzS84RYbAvkiyEOeQQHbu9SRFgDrgtOwhRg6
Ce3EA+CEYffRuu4bj/ReC0zuTMOjiviP1USSbm3uFlbDH3vH+lSbWfwmfln17lNNL8lhwi8Tb9EI
k6yVI7QuLBtG0kXuntGjjkAtd1im9JmDfrLf9kRgZb67h5BdIEOBsQp5ptJqfYYRuGbSf5UXxRdi
+q0fdT9kX3EPmZ+7CfimFtyHbD8FgdDrdt+r/uoANKwdxHwQfa9h0eypQGu1bMOIIiSPC1rHbBdw
G1SNOja8twnHZNvezRr+KLZNHIXsOq1nKMAv17ACT7kJX9wFtnoHoBgbtLbkrUYX6teMJt76hrWC
QtfjB1N1OK17ippJbt+UcOiNmZuld1Rq4KVuEjVbVsQmI3M8vRGaF+CQOT2WFwbj4pnVm72IfNye
LV/Zfz/0TizQQA7ddj70LQtccXTIsBm/sG/xXflmZHjGpwvwXUhe0v6Rt0eFjmwR8nMmI4r2w5o2
tdI1rnWBO1StDJHNBvbfm0beTCTHrvbDm6kPe91vcM0IaWVAJtdn9U2Nzj9ntblK/FO947Qt7lKw
yOMjNeK2reE9ujgLb9J5HkFH043RXF8WJqjhdjBfqDTc+5SOxSPdxKY3SU5a3rNfit3Iix1qCzXG
Q7GC/vWtQ/PxfeTp/l5mya2F5v3J7DG6x18DxQ0skMTmON2twuaCiKjAXOLsPiqb6RD95DYiqn6T
L7vyQ+iH+KoTIYHsjNnh+K3EIPrTfpP1Zo9dPVfyfmgQVD0WyS9kWb4lDYndV3gqu+mxbnAWPuUJ
+Wi++FDtl/hyKP5ifxYhGXUrCNz3qluJdJbosDrlN6o8E76ND99+Zr5Y26jYd6mAMbJNHgu61s0k
UkdsbbX2Jnzkf3x+adM18s0LmhrdIbHvUrHNLmHaqnW5so6rmj4INIhnc7VcNaPP0HD9FhPBxvmM
swpocGzAAdV7EIzVDoyCeidupMYXuDQpHtqRZm2Pq/aGZ30WE8HvvmbJGB9HYe83RezCQIGN7xdR
txtkZv6w/lUDSZmatTTc91kTiLjxJKD2xR6B31hd57N0Wlri125cDZYSynOIdaQWTOlHKzfqcBxs
oAdOzV1XXd3K99aP0Zn8U/lme+lc4utL4arKp0HQLA65YLdw9CXqQO3aO1/DNLJUH5Rr3YHAe5Sa
Y2xl23VdMH9oeq34lP8AlbyfX+v/ALrcptrHbixtkCtKexF+AJzi7c33Cyw442EfUuD4IttC+/18
NvdrTGY3d31lj0+6zh1PDgofno/vIts7CNfBS7bA7GBqdpJOmHHcCL7CE4PFsRPmv7Ho9rpJg6Et
3fV4qIgO17CLeaMT8VvQd02MI2gWKtaGTXruNSb82/jq8lHbp3P1StycLhhGvor/AHUrSXnhuT7f
J679BCR42H0R6b+pB3A/n+xX3ZgSNhzlzjtOYGxuESXG5N1c2tnJJN1ckAbhsQJB1J8j39Z10CQ4
EbRnJLnEnasRw4d175jPKW2xIGxBXLam3WHkpJXyHpWVzgw7r3ztnkYCL3HegbOaRuN00kXtvBHm
g4hrm8f6LWKWMg/2p+IT5nvGEnVwzYjgDdwJPnmbNI0W1Ed+tPkc/rG+aWV8pbiPVFgt4PBElznE
7Sbpz3ODQT1RZMkczZs4bk+R7+sfAbh6DJXM2bN43Jxu4lMkezqusnzSSAAmw4DYmvc1rgPrbU1x
aQ4bQnG7ieJTZHN2HxG4pxub6s4JGsGxTppHixPj3ppLXBw3K5N779qc9zsOLcEyV7BYbO/Wi5zj
cklfBOle4Wvq/PxzXOHDuvfOXE2vuFvQa9zdhzNc5puCnEudiJuVc2tuCBINxmD3NvhO30NmtEkm
5W5BXN796BI2HMDY3W03O1A2N8+sCwPoDV6N83d6Y9EQyFt7ei1rnEADaiLEjgSPQETy3Fh1eiGm
xdbUN/otie7w4lEWJCAJNgnwuYL6v27EcOHde/8AhYZibKRuGRwULcUrQnyOe8vvvNu5T69FJve3
X4haO8eMbj0go2BwcSbNaNuZt4RHr6Tz5C6k+dm/iO/NblI3DI9vAqBmKaMHitPI5+MOPcqga2PG
x7box/JiQG439yZHiY917ADaha4vsRkxwVAAs0YbD35o4XSAkDYpIMLcQeHC6GG/SdbvUsLDTwDl
DA23miBcgOB7wqJjS/FpW4sJ1J8bWtxadjzvt/jZsjWD5pju8qombpT8gw7NaprCaPxVrajuU3zd
L4OPmoMWlbb3+CqrNwBnze496p49JJbhr8UKWpfKHPFtd9qqInskkLhqdI63mmyRMbrgDzxuqiSL
TS/IA99+5Uf0hibqaFP81SfwyqX53D9Vw6Q7lV2aWRt6lrjvzRfM1X3W/nmlNqemA2HET555Po9L
90/nmpPn/wCR/wCSHVH+NyTmL4Hlz3NIO8cVJJjde1hawHctIGw2b13bTwCie0XY/qEeRQJCa9wc
04jqKmfjleb36RtmJJJJTXOa4EbQiaZ5xnEOLVLJjfe1gBYDuWJrILN67+t3BRua6MxPOr6h4HNE
QI6gcWi2ZkkRi0c17A9EjcnxwMYTjxHdbM8g09OL6xivmpSBUC5+q78kOqP/ALBtpA709sMT8Dru
dvPD1UkcERwPcS7u3esAJIA3p7Sx7mHcpI8Gj+1GHeoItncwtDCfrNuEI7wPkv1XWt6yNjXk3laz
xU8RhkwE31A38c7muY4tcLFNa5zg0C5KfTGNpxSNv2d/qY4HvGO4aztFPbhcW3v6MTDJK1g+srhX
HH9ha1zjZoJKlhkjPSHv3egyOSTqt9/BPZgNsTTq3G/oGJwiEp6pdYZmwPcLmzb7MW/wTmuYbObY
+hYjb6ZFtR9BsYwhz3hoOzvT2FhGyx2EegAxjGOd0i7YEesVo2AdN9iRqH+6e3C6x7kyMEY3Gzb+
aey3SaeiUxmInXYWuSsDH3wHZfUd9uHokDQtNvr29VTva2RuKMO6Y9yqpYmzyAU7Se1dQBrpomuG
ouRGt3iU9reTQvtrc5wUcbNHJI4mzE4072mzSw7ra00FzmjeSv0aI4SMZ3ngpog0Me112P2KB0Ye
NJHi1jequSn5RMOTkm+3EhwTmU0LnRvxPdvPBSBgd0HXHqaQdJ0p2Ri/vWskk71LGNHSSONm6EBP
gi0RlhfcDa3eFDE6V+FvvPBaKlJwCXp9r6qex0b3NdtCFOxjA6d9r7GjapYMLNIx2Jl/LxTNGZGt
kvhJ2hVbKJtQQZJAdWwalq3KJmkljZxPwVQ/FMbdVvRb7kz6FVffYoYQ8Oe44WDaUBcgDXc6lyaJ
vRlmAf3KWF0T8LuFweKZA58WNvbwnu1bU6OERm0l3jaFBAZjIBtDLhMgiccGn+U3D6vmnNLHOa7a
NqbTWaHSvwA7OKlp3RDFcFh6pCd1Sqz52L+CzNSNBmu7qsBcfcnOL3ved5WTmwmdh0vT6XRspo6d
uK1Tidi2WTY7xzPv820HzKGtzRxNlKzRyPZ2SuTBrQZpQy41DaVNHon4bg8LKKF0gcdQaNrin03Q
0kT8bRt4puC/SdYcbXU8VNyWmvVWb0rdHanYcRwuxDjsTYBga+WQRhw6PFTRaN3WDmnYc9H9Lg8T
+So5tDUv+Ta677eGtZTlBkMAja0Rm9x4JtP0Q6V4Y07OKlgdGA64LTscM+q4veyngpGwUt53C7Tb
o7UcGI4CS3cSLJsLNGJJZMIOwbypYcAxhwczFa4/qooscbyD02/V7lFFjJJNmNF3FXG+6mjo2RU3
ysmtuqwTsGvDe3ftXJwQDymDzTog0X08Tu5pTCY6WZ41F0jW3VLd7zCeq9p1d6gwOLA9xAO/vWhl
0uit07qXA17ww3A396lZJeOCPc258SnMLHYXCxUbYiTpJMI4bytFE+Nz4XdUXLXbU04XA6jYove+
klLjf5YI7CpmPmc2WNhe0i3gQp9WjZe7mMs4/wBM9wLXbibv12VWads9tAXdEfXsvdtK+QidgeC8
/W3WUjA3AQbtcOiVha6LE3rN6/8AusIbCLjpv6vcOKiNpW4mBzSQFUyMFRKBTM27SSm2uMV7b7L9
F/z/APSnlmvBisB9ZTddg7MbPyWrkh+zL+aja1/Q+ttaePcmtwNc97dexrTxTSRbZ71JL8jTnQxX
17ltOwbd2xTfPy+Kd8zD4uHknfNU/wDOfio9cdR9y/kouvht1+igzRY5Lh2HULd/HOTqRIiswAYr
XcT3p2E0oLdXyusJjsP1QTuR6cchcyzmjamktcHA+m3rM+8FV/SZfFQfPw/xGp4IfIPtH81ICKKl
+85RyujJtrBGscUBDPewwSBp1fV1KkcOUQE7MSkmYJJAaVl8SlqBJGyMRhoab6kNrfEKr+lz/eza
eN9tOy5t1xtU8Qjc2xu1wu31L/k6Zke+TpHw3Zqq+ioeGiVF1qkbtAbqkvo6vjoNS1Yfcq3rwE+x
ZdV/0s/cbZQfMVw3aP4onUFXfS5PBv5ZoOhDNN/IzxzUoY6mqg91hdirdbYnR/Mbu496orGsp79r
+iccnlzr6a+9VMsUggEd+gCNaa4jJ8tt84v5ZqNxa6oI28nctgBHcsoW5Y/vDFW8h5Q7SGUOtu2K
SSl5IYoi/wCcB6Sd1Sqvr0//ANO1blrjojxmdb+UZqH6XD7/AMk7rv8AvlQ/M1v8MfmmfOR/fCrP
plR97+iL6apwaUlklrX3KWB0LgCQbi4I2FN5NyCASl4u93VUMtDC57mul1sItZDYFN9Dov8AzPzz
B9PKyNk2JrmjCHDh3qeDQ4DiDmu2OGel1VUH30WMaJxG8GTTXk8FlT6ZUeA/JV/0t3c1tvJR/Qqo
bgWEeN/QqPo1D9xyOwqq5INC14l1RNthsnS0wpnxRtk6TgekonPbKwx9a+pZQDMI0VtHpDjt2s03
0eiP2XZhFS2F6rXv6BUjKdo+TnxHhhshropPsztd5hUf0uM8A78k3qpuM0obf9IMZtxwr6upVzy+
Rt+qWAjvTiXUbS7WWzWB7rJjGCLSvvbHhDeKp3xHTWp8PyJ14rq91H9DqR9ppR2KdzoX6JjsLWa/
E8VNZ8TJrWcX4Xd+c9Uqq+eB4xNUZGkjO4PCqJcM0gdTxbd42qSfShgswYNmFUvRk0l+iwa/9lU9
J2mHVf8AA8Fw8VVD9Kl77H4IWuL7Fig9gf8A1CnYT1WYdXG6lsdFIN7bHxatlI6/15Bb3KJmN9tg
2k8Ap3ioGkA1s2ju45if0eH7zkdhUwJkLxra+xCkBbHC07bucfesJdA221jj5OTQRFM47xhCiv8A
K4eto9Sha/Stwjfr8EbYnW2YjbMRcEKVt7SDYWi/iFgPJHX247ptxAS3bj1ngFHpCybE8kYNVz6j
eD3qZ+kle/ithB3gp1QH63RjGN+5PnfJG1j9zrqOQMuHMxA+aM7WgiKPDcHWdZzNqgR8tGHd42qa
QPdqYGgLh4qeTSTPfbbm09MdZgOLu2KaZ0pBIAsLCyDiDcKrqNPIHYbdHNBU6KCaMtvj2ZmuwvY6
2xwKq5hPNjAt0Qha4vs3qeXSzPdu3eGZ1S10EETmXDRt3hGaNkbo4gekNbztUUjonhzf/wCaElIO
nojfsblNKZZC8ps0T42snaejqDxtspZo9CYom2adp4o7FpaWfC6W7XgWNvrWzTSMdHBG3Y0XPic0
b2inqWb3Ybe4qnm0eJrtcbusP6q+CS8buq7olaSjmJdI0sdqvbepTCXfJNIHemvbyWSO+syg5oHt
ZpsX1oXNHiV9VVj2yzlw2YGhCaCZjRO04wLYwp+SiwhxE31uTGRPvjn0fuupxRS6LDVWLGYeqnbS
L79qqZGvlGDqNbhbmpnhlRE47AU83e8je8n4qmlbE92JuJrm4XBXpInY2dPX0RwVU8SVEr27Cf6L
BRED5RzOIOtTysfo2xtIYwG11DOwRmKVmKMm47lJyBsZwFz3HZ3ZnyMNLTs+s0uv78zRRFgu9zHW
177qeWJzWRxNIY1xOveU4dE2VXJTvezQswgDXuURwzRO4PBUOgfVyukmLGlznA+9ZSNHIdNFLie4
gFvctLDK1omBDmi2McFJLHo9FE2zL3ce16DJoXQthmuMPVeE7DiOHYmTxOYyOdl8Oxw2hSupsIET
DftFRStja9w+cOpvd3qnlEd2u1xu6wWq5tsvqTJYjE2KVp6N8Lht1p+HGcOzdnhmMRPRxNIs5pT5
omtc2CO2IWLjw4BRFgkZjF2jcjJIZNLfp3U72yPxBtr9bxTJm4AyVuJrdh3hSy6SzQwNY03AH9VH
KA1zHsxMOvvumVOEkNZZhFu8+K2JsmGKVluvh+GbThzQJY8dth2H3qWUyW1AAbAPQbN0Q17A+wGH
cczalwbZzQ8W3qSQyOuQBwARfeNjNgHxTXlrXttcOzaZ2ANc1rrbCd3oMley9th2jcnuc913H/os
ZwOZuJ1priwgjNfohvAk+eZkj2bD7lrO0koOIOo2T3Oe65OZ0j3Cxd6DXuZ1Stdtu1Nc5puDZXN7
3/8ABY5onUuD5Vhvfcohk+SQMDHa77062J9tlzbwQhjY1jpXbRfApNGXfJggd6ihx4iThY0XLkYY
Zb6B5xD6p35rZoINJjJdha3aUYYJgdA7pN+qd63ehyW1I+UnpC2ru9CCDS4yXWY3aU6njc174H4g
3a07fRmptFSxyX6RfYj3KFsTn2lcQOK5Pk/96eqynZA9rWvxAtuo6YGPSyPws+JU0AbHpY342XA8
PFRRPlfgZtXJGO6McwMnD/Zdx2jb/igAn0HZ4vk6eaTeeg1RtvJG3i4BVDGcpkMr7a7ABSxGPCbg
td1XBXtk93fMAqcltTCR27eamb+kTNHtD+agoqj5a8f9k4J8E0YGOMhbKF/fMAfJUuqqg+8pOhPJ
YDoyG3mpINLhmgbqebOb2XKq0bAyFuvDtd3qmiD3FzuowXctKZYa932W2ViSBxVPRz2qLsteOwun
wyR2xsIR1ZObb60xuqD6YwcQ4HyTDopOriwOOrwUlMXPYYdccmw8O5VeixNjjHU2niqZg1yv6jPi
VI8y0Lnk6zUf0zUsWlnY3dtPgqmTSVD37tg8AomGqpRE3rwm44EFSx8lpSx/XlI+Corl84b1jAcK
DX36IOIHyWUWhtbJbgD/AIsdszazs27lV2DmQjZELe9Ndhex3BwKq2OFQ920PN2lSNLKGIO2mW4H
cohpKWaMdZrg8BUsb3VEfRNmm5Ujg+WR4+s4lQzOBkvIfmnAa96L3uAu5x96hGkpZ4x1mvDwFRxl
1THqNmm57lI4OllfxeVA8UUd5Nsm0cBxU8Jik4tdra7ionUbqVsb3uab3dbeoIqDQ1IbK7CW9JSY
WyHRuuBsKgqJPlrzO+bNte9GSRwAdI53imjHQvaNscuL3FUDTypj7GzQSSnG7nni4nzUD2UkLYpX
G8v+lSxPhe5h2/mrUMlPC3TlthrHehTUnI3jlHQ0t796dbEQDvUN4aSWbfJ0GLcpjgoqZrNQffGV
T3lgqI362sZdp4FNJGFwNiNhWlqp7Na3WdRcFXPElXKRu6Pl/ix2zMx5je143FFxLnOO0m+aKpmj
1XuNwO5SSPkN3uumucx2JpseKkq5ZBa9hvtv9Br3sdiY6xUlbM9ttTb7bb004S022FSyvlkL3LTO
MIiOsA3HdmZK5jZW7nttn1WUUskTsbHWKkrZXswgBoO2yjfo3tda9lLIZXuc7enzOfHGx21mx2+3
DMJSIHxW1F2LNJMXxwx7oxmhqZImlmp0fZKlrJHswNaGM4BDUWngU6veG4YmCMd3+LT1fV6/7p1f
sp2/ser0g1xxWHVbc+Ho/mjcGxBB7/XhrnGzQSVLBLF1m6uPoRQySuwsHjmjhkfrA6PaOxSxPjsT
1T9bcmMe82aLp1NIASCHW22zNppHNB1NvsB3oggkEWI9CJuOVjTvum7PTdszWNsVjbirE6gsLsWG
xurHDi+rxRa4AEg2OzM5j2WxMIHfma1zr4Wk222Tf1ZN/GGYsc3rMcBxIQ17Frw4rG3FEEWuDY7D
6nXcWGu+pVwYJWMA1sYA5Yf+zGfxynAjaCPHNhcdYaSM+zbqPfmfTysjhdgccYO5awdYIzSRMZRw
uI+Ukdf+VZPAM8gI/sHJvVC2bQgCdjSfDPhcdYaT7kBfYgHHY0nwGbuR1GxBCje6N7Xt2hVUzp6K
nkcBfSuGrMdhVXGyLQxgdMNu/wB6sdttXFWN7b1x7tq16tR17O9EEGxBHigCTYAk8At9iCD35oWu
FRT4mEfKDaFWC1ZU/fTWPf1WOPgMwY8mwY46r7NyAcQXBpIG053dUrKf0v8A8sItdgD8PRJtdC5N
gLk7k+OSM2ewt8fQjoah8Mr9EbjDg71JTzxAGSMtQBcQALkqSGWL5xhao43yOwsFzmkY+Jxa8WKi
yfUPglfg16sClp54baRlr7NaDXnYxx8AtHL7KT8JVLC2aoYx3V1k+AQrWOIY6BuhJsBwU8WhqHxb
gdXgVLE+F+Fy0bxE2UjouOpROgbidKL26rOPisom9Qw22xAqkF+Vf/TlCiqcGLB/L9bNHSzSDFqa
DsLt/gpI5Izhe2xTHaClbI35yR9geACxGpglD7aSNuIO7s91yWHkYcahg+V63u2J7WNdZsofq2hR
QOk6Vw1g2uKkp3MbjDg9naChi0uPX0gLgcVDEZX4Rq1XJ4LVfUdV9qkhpRTQOM7tZdrwp2C/QcSL
bSLIUxcAdPAL8XJ8GBt9NCe5puVGSylqJB1iWsBVKXOk0RPRkBuocBIEhIHFGF+n0NuliUgY17ms
NwN6pZn6WBgsBfzVrEjvKDS42AJ7grFlNKHasThZu/VmgY+OZr3gsA46rqPpPj73j81UdOeUnjby
U+tlPJvcLH+XPFG12NzzZrNvf3Km0Lp47DAdepWAuOB3LHS3A0Zt2t6lZgkc30OCd1Tmb+rp/wCM
FF8/D/ECZbnb+d35KkfeeojI6DsfR8FC4y0tUHbA3E3uVPhippKktBOLC1R1smMaWzmE2ItsU8Gj
qXQt2Yhb3qqndDJoILNDAL96dKJclzHCA7SDFZUMJcyaUMxObqaFEMo4xpIQWE6wVoxDlVrBs0gt
71TSXrKiEgaN2Lo+CgcZqeqY7Y1mJvd6mhZ8sZT1Ymlyc9z3ve7a43UL2MyU5xbcibUqep5RJyed
rS1+w9lQwYqsQO7ZB9ylr3tkc2BrWtaSLcVVNY+GGpY3Dj6Lm94TS2kpIZMAdJLsJ3KGXlpMEwbi
LTgfbXdDG03Bs4KpraltNQubJYuZrUkkkj8T3XKij0srI+0VXSB9SQ3qx9Ae5ZM+lH+E5U8TKeFt
ROOOjYpZHzSF7us4p8dVTYY6eDcLv4qqidJR8okh0crXWPeCsmNY6Woa/qmE3Ta+UdQNa3hxVB9N
p/vqqqnRVEjaezA13Daq4Mcylna22lb0h3hU8csVHHLFDjkkvr7ITY6mpimbUw2IYS19td0NgTv1
XF/9Qc1DC2apaHdVvSPuUspmmlkP1nG3gh+q5f47VQfTqf3/AJKhw8rqg4XGGT81DUYsnzY2gmIt
wHhiVQ8yUMMzusJHNuny8jigZE0Y3sxuf4oScsp52yNGkjbja4b1k/ABUTvbfRN1DvUNdK+oj0wa
5pcPd4KoZpMpSM7U1lNFlBj8FNGGxttbvWUoncnhlewNkNw6ynndDJQ4NXyTMXetKW5UdCLCMmxb
4i6tYuHAkZndUqppeUVLXHVGIAXKoqNOQAMMbeo1Mfyahjez5yZx6XABUc7qhxppjiDwbHeChffn
gll5vremeiW21olx2ucfEqkhmFE6SItEj32ueAVJBUguZUSsdEW69d1E98Tw9p1grQ01ucLasN8H
205znuc9+suNyoXHm6tAOoYc1Ly/RnQF+HEi3LGu5kWT/n5B/kPX1Asp/TP/AC2qlY2tZoJNsesO
7uCqpNJMejhazosb3I7FXG7qY8adqyY7DPKf8kpssjXCXGcW1VzG8vLe1g+Kro6c1DsVVhtsZY6l
USwGkijbLjc123uUn0OkPB7wqLrVH/07k3qt8M5/Vrf/AKj+mZ/JuSUgkc8aj1RdNlo44ahjDIS9
ltYTC4OaW9a+pVYLac4GgOeRprd+Z/0Om/iPzCOnsMVSAeGEp7acN6E2I/dITBipKgdl7XKi+lw+
/wDJaul4lMLuTYNWmLDg44c1L9Jg++ndeT77vzTXFpuDYqN5m0jZNdmEg7xZblHPIHsGLEHOF2nW
pMMMzsOxjlUDDUScD0h71Jqgphv6TvPPGL01QB3O8lS66iK3FNYH1WA7DK5aaTYyNltwLblVn0l/
gPRd1TmpmGakqYm9fEHAKmpX6ZheMIa4Xum/rc/xT+SpPpkn/mKj+Zqh/kqm+WpZae4x3D2d6io5
5JAC3CAekVVzB1W+RuwFvwVZA98unjGJkgGxGAxZLnxbS8GyoXYo56cPwudrYe9CHKV7Xkv95QY+
WxYycQksbql/Wb/GRUGyq/glfVU9HJBDHKXAtfmpaV9S9zGOAsL605pa5zTtBtmqKKSnjje5zSHc
M0nyNAxn1pjc+GaCJ0uSpQ3rCa6yfC91Ux5Fms1kqOoa3KGn+rpD5FVFJM2Z2Flw5xLSO9VTNDQw
QO6+PGRwWF1VQwCPW+HUW9yyfBI2YTyNLWMBOtE3c48SSpI3yUFG5gJwXDrJzXNNnCxVF8lFUVR3
DC3xObJZtXM+45R1QrGmCpsLn5N3AqWOSF5Y8dJpVUayQtnppHljm7AdhU/OAj+WMmA8Ssm/PT/w
HIdUKi+l0/3wqj6TUfxXKc/9n0Hi5RmeahiFPIQ+LU5oO0IjKuB7iZcIvfWhusLoxSc0joO+kXVi
DrBHim/I5Oe/605wj7ualjM9HVQs6+NrgFk+klFVG+QYA3jxKpfp9V4Sqn+g1v8A5f5p2vJbf/qT
+SljdV09PLFrcxmB7d+pQxPpoKiaXo448DG79ayfYtqKdxtpWdHxCgoKjTx424Q1wuVVP0WU5H9m
QFVNPUzSmWnlJjfwcqqCoiA0pve9uldV2yhd/lMUn66H3x/+Kk+em/iO/Nck/QBVaTf1ffZHYVU1
k8LqPA7VoGnDxVTCx7eVQdQ9YdkqICpoxBjAkjfdt94Kpqd1I/lE5DQ0Gzd5KGeitJHVQYg0yAYb
9ynppoMOkG0qB0U1M+mkeGnFiYTsT8lSMaXPkiAG9U8Jnexg1bz3BGvaJsAH6MBgt3cVUwaCbCDd
pF2nuVHo3sqad78OkDcJ7wp6eSDDjtrvsTXvbskcPAoySn+1f+Iqmm0E8cltmo+BXI6e+l5S3Q4r
/wDRVM+nmfJbV9Udylk5NGyCJ/SJxSPH5KpcyaIVAsH7Ht49+apcHClsdkABVBKyKoc9/V0bkyGA
9PTARcD11Uzaed8uzZh9yLqWrcHvfopLa+BU7adjgIpS/idyhki0boJr4S64cNxUjqeGORkLy97x
Yu3AehDJCYnQzXDScQcNxTxG13QfiHFRTxaLQztJZe7SPqqYUQZ8kZHO8lA+ONxkdrLR0B3qGd8c
pe43xfOd6OHE7DfDfVdRyxaLRSsOG+IOG0FPwY+hfD354pTG64F7ixB3hGoia12ghwkixJ/oo8DX
NxNu0bk6R75NIT0vyUsgkfiw2Nul4qN+CRjuDrpzsT3niSVE9jQQ6Fr/AM06foFkbAxpGviUx+B1
8LXdxXKbfNwtaeO3Myoc1oDmtfbq33J73vdiebnO1zmOxNNijVSXuLN167b1iOLFvxXT53uG4EjW
4DWnPc43PADy9DcndXMCWm4NjxT5pJLY3krSyaTHi6XFB7muxA69fxTXFt7G1xY5nVEzm4TI4jNH
NLH1Hkdy0j7PGI9LaihVVAZh0ptZBxDg6+u+1CR4k0gPS4pr3tJLXWuLHMZJHMYwvJa3ZmjkfE/G
x1it98z5ZHNY1zyWt2Zi90ut7tgtmxvwYA42vdOqqhzcLpTbNHUzxNsyQ24IuJ2uJTXOY7E1xB4h
S1E0vXkJzRTzQkmN5F1JI+R+N5uVpHaPR36OLFbMyR8bg5jrEZpJZJbF7rkCyjmliN45C1S1E01t
I8lMkfG4uYbEtt7jma4tc1wNiDcIuLnFx2k3KMjyxjCei0mw8U17mOu1xB7lLV1MowvlJHBRyPic
HMdYrnCu/eD8FJLJK7E913cUXvc1jXOuG9XuzNc5jg5riCN6dPM9zXOkJLdiEj2vLg8hxvc+Ka5z
Q4A6nbVjdgwX1XvZMe5jsTHFp7k975Dd7y49+Z80sgAfI422JznOJc43J3qOSSM9B7m+Cc5zzic4
k8Si4m1zsFgsby/HiOLjmu7DhxHDfZuzFzja5vYWCxGxFzY7Rmc5zus4nx1+jcnaSfT1ehq/vIZn
dU59m30bEbvR1+pDHFjngam7T/jp/VdmomM+WmkF2xt2d5UdWJ3timibgdqHcpI3RyPYfqmyijMk
gZxzUMTHOkfJ1I23ITK3TPEc0TdG7Vbgp4TDNJGdx1eClhkicGvG66EMronSBvRbtzUMbYmhrh8p
Iw+4IbFDEZZGs4o7xwWT4mPlkc/qsZcple2Z7YpYW6J2rwU8WhnkiO46vBSxSROwvFtV02CV0TpQ
3oN2qmF6GvAHBcjqRFpNGSPjmhp5pvm2X79ympp4LaRlr70ASbAEngubqzDi0Xx1/wCM7J/Vdmi/
V9Z95iZ85F99qyj9Om/lVHHk/TtLZyXW2KaOiDXGKe7uFlS/Qq/7oQvdtuIWVPpr/uNVKBWM5NIe
kzXG/wDoq6QC1LH1Gbe8qkhBc6V46EfxKoZnTZQc872FbiqKPJ+niIqCX9nvVTDQfKmOoOO56Kov
o2UP4a3N9yyrbljvuNVG3lsXJn7Wa2P7uCrpA0ilj1Mj295WTX4KeudwaCoauoZKx+kJ6Qv3rKMT
YqyUM2GzvNV08lPoYIThAjBTp53twvkLm8Csm4WMq57a426ka2rccWnddEkkk7Sf8Xn0H9U5qKSM
aaGQ2bK21+BUVBonskmkbgab+Knl0sr5OJUExhlbJa9s1FMxkj2SdSRmEngosnaN7ZHzNwNN78VU
zaaokkGwnUnu5DT4Gn5Z+sngpf0uDTt+dZqkHEcVTV8lPGWNaCCb61RZSfNOGOjaNSqqg1EmIsDb
atShlMUzJAL2KccTnO4uJVDOyKZwk+bkZgco8mYHte+ZmiBvdVk4mqpHjZsCceQU+Fp+Xk2ngFN+
lwcob86wWlHEcVk3RmOtD3WaWDWoaB+NrnPZomm+O+2yrpWz1Uj27Ng9yLGZQjhc2VrZ2swuad9l
UUElPHjfI3b1VQTxMfLFL1JW2J4IZIlOts7C3intwPc24Nja4/a7D/Ch3+g/quz3NrX1ejc2tc24
ZiSVcjZm7/RubWubcM1zxWviVdXNrXNuGcknaSfHMCRsJ/xhuPoP6rvDNDHpX4MQHiqijlpw0vLS
HbLeoip3ysmc23yYuVuWgk0Gm+rit+ySQPjjhkOyQXHqJad0TKd5OqVt/D1MEJmnjixWxHapGYJJ
GdlxHlnpaeCc4X1GB2KwFtqqYdBPJFe+Hf6dVSvpnta8g3bfV6uGPSTRx3tida6ni0M0kV74Tt/w
IDYo6yn9V3hnm/VdJ985oaGaVmMkMbxKqKOWnsXa2n6wUUT5XhjBrXNMtjgmYXdlOa5ri1wsRtCg
oJpW4yQxvEqkpZYY67FrBj1EbCog04A5+EHfwU1E6GmMralr2X1gZnZNlbhJcMBbfHwU1E+OLStk
bJHvLdyjjfK8MYLkpuS2vdg5WwP3tVXSimeGaUPO+27PTxaadkV7Yt6joHG+lkEYvYEqeHQyFmK/
eqfJ8k0ekc4Rs4lVNDNTtD7h8faChhknkEce1OyccWGKZr372qppzT1BiLr7NfiqumNPMY731A3T
aYupJqjF1HWsoWRvkDZJdGO0qyk5O2EifSMd1UdS5slxMu8BhYHF3iqmmZCGOZOJA5UtJNUuOCwA
2uKfky7SaedsuHaN6a0uLWgaybWTsk4Whr6lolP1dyylGY4MnscNYa4Z4KQzwzSB1tGfgn5PYGxy
icGA7Xb1VU/JntGLExzbtco8mt0WkqZtEDsG9VNCYQJGSB8ROpwVLSPqXnpBrGi7nFU1FAKqF1PU
h+F3SBToxLlOSImwdO4KVmjlkj7LiFLTtZS0swJvLe6g+kQfxG/msqD/ALQn9yghfPK2Jm0/BCjy
cehyz5Tj9VTwyQSujftHxz7lX1AqJo3AHVEB71Qwsnqo4n3sb/BSDC+QD6riPJVUDImUpbf5SLEV
TUURh5RUyYI76u9VLcmaEmnedJq1G6o6XlBeXPwxsF3FNhyXOdFE9zHfVd2lS0uOu5NLuDrqOkop
opQ0ua6HrO4hTxU7qRlRA0tAfhcFE/Ryxv7LwVUSaWeWTtOv/gaTqP8ABDNL+qaT+IgLuaOLgsrO
dygR/VawWCoLyQ1kLtbdHcdxVNTSTnDHuFyVHTUrJm/p9ng8FlYWrZe9oWVSRyWIHo6O6yY93J62
O+rBceSb1Qmfqab+MndUrKkh0FEzdgusl9Wtbu0SyXM2KojLtjm4fNMyZNHVCR0rLB9731qtlZLV
zPZsOfJ306m+8q97n1s9z9ay3KvZQ/JMmnLLM1CyZJk6KmqYmVBdjbvCyX0YK546wj1eSiOF8Tgd
eILLH0/+Vqyx9MHfGFT/AKorvvDNWfq7JydsKyq46Ghbu0d81MKYZIGlLg10hxEKmkyRTzCRkklx
3KhwPysCOrjeWqpJfUTl23GVlAl1Hk1x24TnpfoGUf5Uf1Q3/wCp/oqu3JMlk7La1li5riDsDBhV
HryflBp6osR4qmvzPWkbcetZPOGtp8PaWzLX/wDsKt1VlT99VP6qyf4uUfzkX32rK36wl8GrIttP
PcXOi2LlWSf3N6r6ptVM2RrC3oW1+jkv9YQe/wDJVPRnqB/mOWUQRBk+/slW3OTcnuHVG1YhxVJ0
8m1rG9bFe3coNc8Ibt0gsgRz+bdn+ioP/wCJD/Lco/1LP/FH9P8AApzv6jvDNBTuncQ1zBq+sU+j
vQQwieLE119qmjdDJhxNJGu4X6PXhhdLo5mts7vRfT0UEsbH45HjWsnVEURkjkNmyNw4uC5LTUrh
JJPjF7iyypIyWqLmm4LAg6nroYmPk0c0Ytc7wqZ1HTsqIRLd7mHWqePSuazG1mraUylY2gkpzUxY
nOvtVVBoThxtfcbWrKEjJIqItcDaPWsmSsa+pDnAYotSyex5nidhuGa3eCmoKOWV8nLRrPEKqpXU
zmguDmuF2uGeje1lZTucbAOVWQaucg7X5mT0lXFFFUuLJGag8bwqmnoIouhUF7929UNUKeU4heN4
s8LQ5MhLZROXi4wtWWvpv/lhOfQ17GGWXRSsbYo1FEKGqpoju1X+sqeKOVxD6gR6t6mioZKWnh5c
waPep2Bj3sDw8bnDeq+aOSKiwuuWx2Oajq2RRyQzNLoX8NoKMORmtLtO53AAqKR0crHt2tdcJ5yV
UHTlzmOPWYsoVcFRBTaLVgcRh4Z6aaNlNWscdb2DCtKzm7Q36YnxW7lUTMko6OIHpMviCbVUdQxj
awEPaLCQb1VVdOKc09KyzCek7iqKrFOXhzMUcmp4TKnJdLI18LHPP/4qea9ZJNH7TE1SVeS5/lZY
nCTeOKq61lTTQMDMJY/Z3Km5ANc7pQ8O3bFVT5KqJDIXS4rW2KnnkglZKzaFyvJL+nLTEPO0bVO9
kkpdHFgbub6Mcjo3se3a03CdlKkf0zR/K/C6rK01bYLts5l7qkr307TG5gfEdxVTXUr4iyGkDSd5
AUFRLTvxx22WIO9HKkbelFSNEh+sVTVBhqWznpHXfvuoKnROqTgvpWuHhdNqCyjkp8HXcDf/AAIb
eg/qO8CgtSsOGY59XD0bDgtQz0NW2B0jZAdHI2xtuQpMm3uawFvDeq2pim0LYmkMjFhf1P8AuspT
xz1DZIzcaMD0NXD+/vcfQAc7qtLvAXW8g7fTfHJGbSMLTbf6HBTwvglMbiDqB1d+faQOPp09NJOT
bU0bXFT00kFibOYdjh6D8nxMNnVrGngpGtY8ta/EB9bio6eMsxSVLGX2Dep6d0Qa4OD43bHD0mwQ
tja+d5GPqtG1TQ6PC5rsTHXwn+npv6rvBDZmjY6R7Wt2lSMdG97HbWqqptA6MYr4m4vUFrgGkjUd
mdtPK6CSYdVipYdPURxXti3qRmjkkZtwuI9ZQ09DUYWPkkEp3blMzRzSR9lxGamgdUTtibv3p7Cx
7mk7Db1s9PGyjopW9Z98XpsYZHtYNrjZPbgkewnW1xHlmuOOfVmqYTTzGMncPjm1ZqKkhdE6oqHY
YgbDvU9JTSQPnpHE4OszNdHUm80QtAfilcRr7lXUkcOikideOTZ3Kl5vawvqMTnYuonQUFVDK+ma
Y3xtvh4pjXPLGt2uNk+ipI6KpAIfLGNbuGZrXONmtJPAJ8UsfXjc3xGdrXEgNFzwT8mzingc2Jxe
b4hwT45InYZGFp4FRxSym0cZce5SRSRG0jC096EMro3SNYS1u0qON8rgxjcTjuQjeZNHbpXtZPyV
Ucmhwx/KXOPWpYZYX4JG2KEM51iCQ/ylOhmaLuieBxIVCyMR1FS9t9ENQ71DU84OME8bdY6BG4qO
GSSbQi2K5GvuUjHRvcxw6QNrKenkp3NbJa5bfUmV8sUTGQtY3VrO8qsOno6epLQH48JtvUMMsz8E
bblS5MqY2F4LH22huZmSahzbue2PuKmhdBIY3EHvCoo23dO/qRa/E8FlNxe+medroAc8bHyPaxgu
SU6gZ1BVMMvZWVb8t/8ALaoqek0YdLWAE7mqppdBgcHh8b+qVk1sDqmPHixX6I3KUUIMgBnxXPC1
0wNL2hz8IvtQgyc86NlQ/GdhOy6fG6OV0bjsdYo0lL+/t/CpooYw3BPpL/ZsptWTqYduQk+5UnSi
q4d2ixDxCpdC92GX6w1O4FQ0p5Q9s2psYu88USwvJAwsxah3KpotPUyPbNHrGoKSOSJ+CRtiooJJ
icAuR32VQDBSsgdfGXYjwVLA2Zz8Rsxgu5cqpMNhRDZt3prS57WNGsmwQoKnksoLNeMEDwU0MsVx
I2xsq356EcIW2QF8nS/YmB81TljwaeTUHHou4FYHUkT3PA0ruiwd3HPbM7qu8ENmbJjAzTVLhqjG
rxReXEvO0m6raXSuge+QMYItZVRQBsWmgk0jBt4hQQPqJBGzb+SOT6TqcrAl4brqeCSnlMcm381T
5PDohLNLgYfNS5PbojLTS6Vg2jeo9EXDSE4eIVVDk4UtJjkeGW6BG9VDaZr7QPc5tt+au/RqKnpR
1ndJ6yZ9Pg96kp5Zq+ojjH9qdfBVsMEMjWRvLiB0/FU1C18WmnlwR7u9PoIJIy+jmxYRrYdqgY2S
aFh2OcAnUFLTyP5RMbX6IG23FVTKdj/kJMTLI5Op2wwVD5cLCwYhxKdR0s0EktI912dZjlTU7qmV
sbN+/gFybJLfkjOcfa3XVXSvpZtG43Frg8QsnfT6b7yrR+mVP8Q5sn/IUlTVnbbCxD6t+IuspU8d
PVYGbMIKjpo35MqJ/rsksCqKjNXKRiwsaOkV/wBgjoaz9rWmsxy6OLXd1mkp0OSqT5Ke8j9/cq2k
ZCyOaF+KF+zuKMNBCyjllZfSM2d/FGkoIKoQPBdpdh7N1NEYZpIyeq62eduPJmTg0XNzZTQU1HTY
ZWh9RINnZVBSNqHSGQ2jjb0ihLkmd2i5OWXOp+xVUDqed8Tjs2Huz5Jma2ojjMTSSdTt4VVlBgmn
ZySI2c4XVJTmonji47SnTUkJMLaEuYLgmyyjSRwPjdHcMlFwDuQiZXUY0bQJ4RrA3hTMZRUOiIBn
m29wVHUup5L4GuuRtWUcoS09SWNjjPRBuQsYM2ORtwX3cFyrI/7ifJVU1JIGaCn0evWqnVkeiA2F
6yRrmnZudCVQ1PJpQ8i7SLOHchk2JlSaguHJgMY/2VTUOqJ3y8dg7k1jpHBjBdztiym5jGUtKDcx
t6Sp2RyStY+TAD9ZclFPR1HJTpnPGs9y3BUH0LKX3P6Lcnzc3UlO2JjdJILlxVRX1NRGGSYbXvsz
tJaQQbFVVRPzfQvEjgTt1p7nOu5ziTxKNJVMp4I6Z7WdG7zxKnhm5unFS9rnN1tKoark8nS1xv1P
CnjZk2KR8Zu+U2Z9kIfFVTnc10TsR2o71GMtmNmAi1tV1VsytoH6Z3Q32UGvJVbbthZO+nQeJ/JT
3FXOQbESmyhEdUxlY9h0kYNwPrEJ8rppHSu2uVPA+olEbPeeAVdPG7RwRfNxb+JVE2Lm+oc+QsDn
WLgqV2TaaUSCrce6xVLo35SBHUMpIVY5z6qcuJ1PIHgmtucLRrKq6Sp0cdPDESxutx4uVfTVBjpn
CM9CGzu7Pkm3LP5DZdonbcrKpPKYzvMIUlPSU2Fs+N8hF7N3Kq0RyZBow4N0uq6yf9Ng8T+Sm1Tz
/wARypafTy4b2aASSmy0WkaGUl9YscRusofTpvcr5L7EymNJYaFrwb68XBOs/JsTvZvN/eqD/mn/
AFRCRdRNLsDANZ2Kf5WE0jZLzMaL/a7ldXYq6+hosXXwm6umYnUNXpTcC2C/FUPS5RFvkjNlo5R0
TG+/hmbJaCobiN3BtvNbVM100NPMxpJa3A/3JzXRUDg8WdJILDwVNFpH3d82zW4qSTlzSQLSsvhH
Fvou6rvBDZmrf0ekhpRtPSetyysSY6Nu7Asj6zVM+qWa1kfqVZG0R6luvvWVNdJQvPXLVljZRt+r
gWRfpjhuLDdPsC8DcSsofQMneH9M2TYNNVtv1WdIqtn09TI/dew9yyb9Pp/EpwZ+lxUz/lyblEHp
X27/ABU0lGyho9NEXtw6rKHKGTqdxMdNICRZU30uA/5oWVieXy+AW4rKJ/QMnDu/osja5qn+CshE
aWbVr0WpctycNXIdfiq+tbVujLY8OAELJ/0+m+8q/wCnVP3lYmzRtJsFlQtiZTUbfqNufFcPELLX
0xh/ygqYHmSrPF/+yyZc0mUWjrYf6K4WT3NbXUxJ1YlX9CsqMXbJ9yOrIfSHWm6Kyj9DyZ90quP/
AGrSn+GspfT6n739E2GV0RkbG4tG05oKiKmybRSvjxWJA7llKn6Qqo3Y4pN/ArJ3ylJXwt65Fwo2
ve9jGjWXBZZcDW24RgHPQG1dTffVaxza2ouDreSskvDK6O+8EKprsqQyuZhvY6jg2qqqqqcs04tb
Z0bLJjRAx9dJqaBZvespMEzW10WtrhZ/2UDrb94LLMchq2uDCQYxrCaSyRpw62uvYrnp/wC6xKrr
31TWtMTG4TfUomuq8k6Jmt8L9iyfFJTCoqZWloEdhfeqeF88jIm7XLT0mLm76mHDi+0p4ZKeR0bx
rHxVPSS0cGNkOOoeNXBqmpKyIOlmjO3pO8VFRVU0eOOK7diyZTTUr5J5m4IwwonEXHiSVk76PlEf
5SbraPBPhOUKOmdERpIxhLVNk+rgj0kjWgePoMgdV5MgZERjicbhT08sL9HJa9k6NmUoISyRonYL
EHeqjJklNFjkkj8OKyfAxznzy/Nxa/EqKsFa6Snn1B/zZ4KRjopXxu2tNlDG2syfFA2QNkjdfWqi
nNO/A5wJtfUtJJ7R/msbzte4+9UNRFGZY5fm5W2J4KOCloXad1SH26jQowZ5wHOAxuu4lT5QLKhg
p/modVu0q5sGJssLxaTWW7wVBHSCjDG1bGOeLvNxfwU1FRRxOLa0OIGoXCo6pkIkjlF4n7e5Opsl
taXcsce4WumSYHtez6rri6dzXVPMj3uifvCn0DZv0dzi0fWPFcurf3hyq64ywwMbK7qfKZ4ZnQyM
e3a1OnycTpdC/SbcO66rqllU+N7QRZlj4p9VRzhrqiJ2kAt0d6mrBLTaHR4bPu22yyY9zHteNrTc
KWqo5buNMdIfK6p53wPD26zaxHFGsibd0FMGPO867eCqZtPO6S1r7s8FQ+BxLbEEdJp2FT1r5W4A
xsbN4G9QTmBxc1gLraidyY57Hh4d0r3upZdJIZMIaTw4oZTnt83GT2k+R8j8b3XKjrqiNmAYHN+0
FNUzz9d2rsjYgSNh1oZRqwLYwe8jWrnXc3OeOWSI3Y6yfI+R+J7rlY34CzF0TuTXOY4OabEb/Rf1
XeCGxUZhFTGZTZo1qrn09TI/v1eGapqqGojp43E6m9fge9cppqSB7Kd2N79rlS1DqaUPbr1WcOIQ
5mcdJdw36NV1Vyp7bCzG9UKGrp5IWQVbTZvVeNy5ZRUsbhSgue4dbNDVUctKynq7jB1XBVAphJ8g
67bKnq4oaGdovpn6s1JKIqqGR3VB1qec8tknid9e7SqyemqAyVowzfXbuKpa6NkWgqY8cV9XcpZc
jCJwihLnEatyido5Ynn6rgVXzxz1b5GdU2zVNTHLSUcYvij6yybVR0073PvZzLKCaSCRsjDrCFdk
uTpzUtn79V1UyxSyl0UIjbwVJNk2LRPc2bSN28LqqnyVOZHgTaR35qjmjhqY5JGkhvDippTNNLIf
rOzMynA6JjKqnxluxwUuUxJTzwaHC0joWVNUy00okjOveOKGVKE63UPT8AnvxSveGhtzcAbkzKrC
0cpphI4bHKrrpKqwwhsbeq0KoqNNT00WGxiB18VUVmmqYpsFsAbq+6qqfT1EktrYtypsovgpJoMF
8V7Hhdbk6qLqOGnw9R97qlrTTtkYWY43jq96hmkheJIzhcEctSYejTsD7dZPe973PebuJ15wSLEa
iNi54nMeF0THG1sWaLLNaxuE4H952qpqZal+OQ691tymq5Zooo3WDWDYFDVTQxysYRheNYOaHKlZ
FGGAtIGzELpznOc5ztpNznhmlhfjjcQVUVlTUW0kmrgNQUNRNA4ujdYkWXvU9RNOQZH3sLLnWv8A
bfAKauq5mFkkt28LBQ1E8F9FKW3U1TUT/Oyud3ZmvkaHBryMQsbZmue03a4tPEKSWaS2OV7vE+g1
zmm7XFvgbIuJNySTxOtXI2ajxRJPWcT4m67rm2fb6Orh/do9A7HeCbs/8D9x8E3Zmo6OSqc4NNrb
09pY57XbWmxUWRy9jXcpZrHBHIb7G1Q3y/Yb/wCOePgm7AmtLiGgaydSp8NNNTUjTr1ukPuVT9Jq
P4hXvPmob0mTHy/Xl6v/AIFf7JuwLJFPiEs4tiGpoKpcnVTK0TyPYdt1lGjlhldI4iz3m1lDCZpo
4h9YrK8o0zIW9WJv/gW3YmSSM6r3N8CqStfFUxuklkLd+tVU2lnkdiJbi6N1k6pgp5XvkBvh6Kc4
uc5x2uN/7iH+BACdg/Y2RySOwsaXFSRyROwyMLT6Ecb5XtYwXJT2Oje9j9rdqZkyqexrgY7EcVLk
2ohic9zo7Dvzckp2RsM1Rhc4XAHenWxOtsvqVPTvqJMDeF7+k3Z/c10WPbYuaRfZff6yDJ9ZOzHH
HdvipY3xPcx4s4bczQ5zg1oJJ3IggkHaDrVPTS1DyyMaw2+dmRq1zMXQb3HanNLS4HcbZ9Z9NrS8
hrQSTsA9TRQtqqmOJ5IHd3KZuCeVg+q8jy9N8MjIopXWwvvb3enJAWQQyk/OX1eHoUVI2fSPkdaK
Prd60FBVseKQFkjRfXvQvwN+GYgjaCNV0yTJMTQDC+Q8Sq2ngEMVTBcMeerwVJAyVztJIGMaLuO/
3LKccDW0joWWDgVkj6Wf4ZTaeocC5kLnNudYzRwyykiNhcVLTTw20sRbfPHG6V2Bm1T5LmDYMDWX
wdO53qSN0Tyx1rjgbqChqZ24mtAbxdqupqeaB+GRtuHArk8nJ9OLFgNjxCgppJ8eCwDRrJ2KGPSm
2NrdW0qroW3h/SImARAa9/eiz5TAHNdrsCNhXNdd7Nv4lNSVNOGukAFzbag91PkxrmGz5JNvcoHv
qqeohkdiLW42HeqWBtQcOkwuw9HvKgp5JajQ2sR1u5PDGyPa1+IA7VB+jwibY+QhrPDiq/VXT+P9
FiPaPmn/ACWTwD1pn/6QotDd2lD7W+qq3kQdT4myu+Sbax3LjbYo/wBHFPH9eVzS/wAOHpDZmjFG
aWXE+031c1PFpp4479YqdmR6d+hexxdvOtOtc22XzwwumlZG3aVI3RyPY7a02zXHraGmgdS1VROO
i0WaskD9Phvwcqph5VUBjDbSHYMwBJsASeATmub1mub4i2YQzluIQvLeOHNQUDqiYB7HCPCdadSV
TASaeSw32zZKphPWNuOiwYiqt7JJ53MADcRssq/Rcm/c/oE2CZ8bpWxksbtKGuwAuSn5PrWMxugd
ZblyKqxsZoTd4uFUUVTTWMrLA990ykqXxskbHdrnYRZTZMroY8botXcb5o8k172YxGB3E60+N8by
x7S1w3LJr3trYAHEAu1hZU/WFR4j8s2SGYeUVTtkTDbxTnl7nPO1zifNUeTahzRLDWNaS3YP6qxB
IO26yfRiZzHmdjbPHRO9ZRpDLNi5cIrDq3R3676zrz5Ko4MMkmnDnOjtbs3VRR0cUJcytD3DY1U1
Fpqepmx20W7jqUMelliZe2NwHmnUYblDkmM2xAYvFPyVTQOPKKvCCejx96rqDk2jc1+ON2xyyPFS
aVrtITKL9FVEWSW6W00pfr1d6gY2SVjHyBgO1ybQZIkOjZUnHu1qSndFU6B/bAv3FVNDk2lk+Wkf
Z3VYFW0MccTainfihPwz5K+nwe/8lyF9TlCqGxgkJc5VfJdNanZZg1Xv1lybJ9HGzlYL5Hi+Hgqq
kpnU3KqS+EHpNzwuax4c5geB9VVVXBHSUr+StcHbGncqmVssmNsTIxbY1Nhioo4y6mM0rxc6tQVR
Tw1FLJUMpzDIza3iFk50EgfSytb0+q/fdQUgoxNPUAHBqYO0mTObLpcLb3JtbUqmukZQ00rY2Xcq
ioknfjeG7NyZU5Nwt/QCTZVFRRuhe2OhLD2rbFDqyNWff/2WSjauh7wU6V9NlCd7N0p1dy5FT1Ez
Ktrhoj0pB3hVlUaqcv8AqjUzwVLT00uLS1OjO4LKbHinhEYbydp1EG+vvVgVXfQsnH7JWSD+nD7p
U2UqplTLgdZjHkBtuCyuxrZo5Wj5yO5VVNJR09LFAcOJtyd6lqamVuGSYuHD0Mpa6bJ/3P6IAXA4
lV0UBe1rqzRgNHRVS+m5CYhU6VwfdqoZnRz4bYmv1OaspN5JCyniBDHuu4/0X1T4LKf/ACZ/yc3J
G/8A+Tb+JVMDI2tIqxLr2XupelkmnPYlsVkz52b+C5MuA223cqnSckkLQ3TuYNJbbZUsImmjjvqK
qKCrlqMQwYG9UX3BZVpZBM+fo4TbxTIzI9rBtcbLKLwanAOrE3CFuKrrYaM8YVQsY+pZjOoa1yHF
U6V1Wy+MG3hu9IbFG4Nka4txAHWOKcyhqMnTTR0wYW5slz04lhjNONJ21X1dIyokjdRtc7tehkad
mnbEYW4jez96qq+Bk8zORRmzjrQw3F26r7Am5Sya0BrKK47wFlOlgYyGog1Nk3KnjpOamTTRg4CS
qaelr3mnkpmtJHRcFNE6KWSM/Vdb1GvYNp2LKdqajpqNu213LJP6xh8HKsymaWpkjhhZt6R4lZUj
ifTwVsbcOPrBQ4Mn5OjnwAyycVS1vODzT1MTNbThIWT6IPyg6KTWIr377KXLU7JXNiZHo2mwFllS
KMxQVcbcIk6w71kOomdOYS7oiPUFJX1jsbDMbaxmg/Q8kSS/XmNgvqqWjdWQZOb9UM6R9yyhWW/Q
4BgjZqd3rIcPQqJmgYx0WXUUOVBK1z6lhbfWLLK0bI62XDsIBVfVyQUlJotT3sHS7gFJUzytDZJX
OF7qmndBkIvb1rkD3lZIqJjW4HPJDwb3VBTsdlZ7bdGNzyB4FTQPklc7nLDr2A7FlrRllKRI18g1
EjeqHVW0v8QLKv6xn935Zqz9FybBTfWk6T82Qvprv4RUotJJ98qD5+E/5jVlv9Yf+WPQyJtq/wCG
m7Fkz6FlIfZ/oqP6VTfxGqq6OXYz9tiyxryjLfcGqfXkOk+//usjfrBn3XKf6RP/ABHKgo+VzFpd
ZrRdyhlyUKqNsVO6+MWddZV1ZUH8iy6CKyMnYY9Si/UNTft6sz45I7Y2Ftxqusl/T6fxP5KR1PUc
qo2Esfe/3isLoZQ2QWLHC6y2DymOT6ro9RVB0cmVz3dU3t5IZ6sF2SKJ3A5q2qqmwQy01sBGvVdS
1+UXRvDurbX0FQ0zqidrRqDdbiqgw5Sjljjd04jq71xB3KVj5cj0mBpcWu12T2PbqcwjxTMr1TGh
ojh1dymypUzROjc2KzuAWTbSwVVKdrxdqydRTw1GkmZhawHWjeoqXlg1ySalymnonR0lrtPzp8VW
0hpZSNrHa2Fc2VxAOiBHiFK3kmS5IZSMcjtTUFV/qygPeslfTo/BynydLLUyOhLSx7tt9iypLG+W
ONjriNmElYWZQp4A2QNmj1WKqsnyU0eN8zPD0MMVdSwM0wY+LbdTR6KQsEgdbeEX0te2LSSiKVo1
33qpgggAwVQkdwAVHJDTxyTlzTJazG71TVLHsfT1Tui7WHn6pWy4vfvWloqmCFs8jo3xiw71KIxI
4RvxN3HPS1MbGvimZiift7k+ppIoXx0ocTILF5VI+COUPmvZuwd6bVyio5R9a+sd3BTPiMznw4mt
OvhYrSSe0f5lVlSJ5sQxWwga1STtgmEjm4rA28VrNydpOvNNPpWU7cPzbbZmdF7XW2G/pBBUf6nr
PE5snfT6f7yyp+sJ/d6GSP1hD4OVdqragfbVBAyesjjfs2lVOUnU0xgggbZmrWFljGaCmLxZ2MXU
cbpMhEDaLnyKyRG91fG7CbNBusoOBrqm3aVNRVFUHmJos1G4JBGsIayBxVVQ1FKGGUCzuGakoJ6t
r3R4ejxRBBIO0GxWSKfS1YcerGMSrKjlFVLJuvZvgslfrGD+b8llSmnFbIRGSH2wlZQboclUkDuv
fYi012SYRFrfERdqyVRTx1OnlZga1p2qhq425VleT0ZC4XU+SqwTvDI8TS7UVlXDBR0lLe7hrKyK
8Nr9f1mEKfJtbG6U6Logk3UERmmjiH13WWWZgZ2QN6sLbe9blU1U1Pk+gdG612j8lURMyhByqAfK
t+cYskzxWmppTZso1HvXMFVfVIy3HWqiLQSPjuDbeFlX6Jk0/Z/pmb+oH/xP6rJJ/wC0Iff+SFRy
XKksu4SvxeBUuSoayR08FQ3C73qso2UkjWiUPNterYsn5Ome6nqMbMIcDt1rKWTZZZ5Z2yMtbZ4K
hjZLWQteQG3ub9yylUCorZHDqt6I92bJlIymkE7qqI3j4qto2R6WVtVE/pdUHXrTDaSP74WWJY5K
3FG8OGjGsZtfArXwPksnVbaWcl46Dm4SqqLJLYnuhnJf9VqydWMppHiQXjkFnIS5GpXiWG73X2cF
VVUT8qMnabsBZr8FlGaOesfJGbtICkqonZKhpxfG1+tU1QaedkoF7blPUZIfjfoH6R35qgqzSS48
NwRZwRyhk+I46ekOkv8AW3Kuq21NUJWtIFm7VlKshinZHPBjZguOKrsocoY2GKPRwjcmOLXNI2g3
WUsocrMVo8IaqafQTxy4b4TsU1QZKp87eiS647lW1wqxEdDhe3a6+1U2VZIYtFJEJWDYq3KMtS0M
whkfZGcKlylUUrMDcLm8DuVTVyVLw54aLcFS11RS30ZFuydiqcqVVQzA7C1u/DvUVXNFDLGywx7X
b1BI+GQPjNnBSSvkkfI613HXZU9bU04IifqO461PUTVD8crrm1uGcHWDwU1XUztwyTEjgmSSRuDm
OLT3Ikkkk3J2p0krw1rnuLRsHBMqaljcLJ5GjgCnEvdic4k8Tmu6waXGw2C+pAkG4WzZmsFYehqO
5D9tgdC2UGZhczgEzKeTY4nRNppMB2hVT6N+Hk8Tmcbqnm0NRFLa+E7FVzioqXygEYtxUFFUztxR
suFzRX+zHmjkmuAJwDzTHujex7NThrC56xDp0jS63WutM9sokabOBuFz3Jb6OzHbrXU+UJqiBsUg
Gp18SgmkhyIJGGxD/wCqflupczC2NrD2ldUdfNSY8ABDuKc4uc5x2k3zVWUJ6trGyYejwzUlbPSY
9HbpcUTcknaTdQ1c8DJWMIs8a80UskUjZGGzm7EzKdcwuIl2m+vWpZpZn45HlxUU0sLscby0qavr
Jm4XzG3dqzMyhWxtwNqDh80XOc4uJJJ35nV9Y5mE1DyE172OD2OIcNhCLnOcXONyTrOZ0kpa1jpH
Frdg4Jsj2XwPc2+2xstS09QBbTyW4YjmxE2u4m2wX2Zt1tduHp7c9hwVhwz0dfJSBwbGx1zvXPlT
7CH4qbLFTLG+MxRAOFtnrCq6u5ZIx+jw2bb/AAIFx8fSbLMzqSvb4Gy5TVfvMv4iuUVG+ok/EfSF
XKKQ01hgvfv9Rf0bj1GjkAuY3AcbZtFKW4hE/Dxt+w3/AMNO3+PoUVFTSRaaecBl7Ydifk2kmic+
iluW7WlQsbJIxrn4Ad6FPkNvQdKS7jcqvojSSN14mO6pWT6GKZj55nWjZtTaPJVWHMpnFrx4rRv0
mjt0sWH3p1LkqiaxlT0nkKvoWQYJYTihfs7s0kMsWDSMtiFwg0uIaNpNgnRZMyexjJ49I9w16rqu
o4GxR1NP80/dwWB+HFgdh7VtSDXOIa0Ek7AEWuaSHAgjcVk2k08+J/zUetyy2yNs8JY0AFm7NPTS
waLHbptuExhkkYwfWcAp6qlybhgZT4jhu5ZSp4dFFWQCzJNoXJp9Bp9H8nxUMEs8mCJtynMc15Y/
okOse5UHIWYoILPOC7nr/dZOhgippK2Zt7amBVGWKidj48LA1wtZUDGPrYGP2YlUZVqYazQMibhD
rYbbVlqGOOrBZqxsuQqTJ9RVa2Cze0VVZLqqZuM2czeW7v8AE4XHxVHNFFO0ysDmHUb7lLklxqm6
L5l+u/BZVdTaRkMTB8ntdmyS3k9LPUSamnYtypsjyTMjfpmYTw2rLMwL4oQwgMG9ZHdpGT0rmHA7
eNygomZPjlnF5XC9rKgOkyjCXb3lyysb18vdYLrZAF/qu/qqXJctXFjD2jXZVuTH1DKYCQAxtsoo
HQ5ThifukCywScoP7mhM15Ak7n/1WTatsTjBLrhk1eBUVHDkzT1Dzit82nOmqZ77ZHuVa9lFSx0c
fWd11l75ym+4qOjpaiMulqgw4tiraSjlZTaSqDcLOibjWqZjGZViY14e0SanLK/6xl8Grb/w/wCD
v/2WTKsQy6KTXFLqI71KyHJNNIY9ckhs1Hbcm5O1ZA+lTfwv6o7T4lQyQxZGgfKzEAerxN1DUU+V
McElPgOG7SqaCV9W2FjrPDz0vuqXKNDFVtY/W9osZLbFlaGWOoxvkxh46Lk+OAUdNGanQtw7ja6g
dQQMkYa8Pa4Ws4+g43P+Iwjtd4lNa5zg1o1nUFAWUsUFK+XpuaqmCSnmdG/btB4hZOodMdLLqhb8
VlGqfUkMja7RN2C21NDndVpJ7kwVINmCUHgLrKoPN1NpfnbhUmIZGlMHzmu9tqyOKwVWyTRkHFfY
pHx0+Vi5uprZfzWVqSZ1VpI4y8PA1hVTTTZFZC/ruOz33WsbCVX1cc0NGGPN2M6Shk0UscnZeCsq
0c08zJ4GYw9m5VDOSZHbA/rvOz4rJlFymbG/5pm3vXKqfKWnpdh/sz4Klk5DXfLMPRuDZOypkp7i
XUpJO8tCyvPSsia18OJz2dDVsW5VtYypjpWBhGjbYqKTRSRyD6jgVXUDq57Kmme2zmi6yiWUtBBR
h13fWWSqZjQ+sn1MZ1VDWMymJaaYWJ1xFSxSQyPjeNbSsgfS5f4X9U7rO+8VQPhqaN1DI/C692FU
1GzJmOonmBOGwCoawRV2nfscTi7rqXI7KiV8zKoYHm/FZXqIJNBDEcQiFiVDNS19JHTzvwSM2FT5
Ko4InOdWa7atn986v7u3o9Z3iVR1QpZtIYserV3KeeSeYyu27u5VVeaqKNr4hib9dU2WXQQMi0Ad
hG265/f+6j8SFVIypM8fRde9ty5+n9gy/iqmqmqX4pHXts7lTVlRSuJidt2g7CpMt1j22AYzvC4q
nypWU7MDXBzd2Lcp6maofjldf+noU9fV07cMcurgdamnmmfjkfiKbV1DIXQtk6B2hNc5jmuabEbC
pZpZn45HXdmlnmmw6SQuw7PQjmmjvo5Xt8DZE67kkk7ysb8ODG7D2b6lc31GyLiTckk8SgSNhIz7
cwtnsP7owP7DvJFjxtY4eI9CnoamqvomahvKqaWamcGygAkX1FMgnkF44XuHcE5jmOwuaQeBTWOc
4Na0knYFNS1ENjJE5o45m5PrXR6QQHD8c0FHU1F9FHccdinglgdhlZhP92H0Andd/wB4/sN8+F3Z
d5LfbXdaOX2b/JbNR9dS0MlTHM9rwNHu456Ci5ZK9mPDZl0Rb0KShZUU1VKXOBjGryzwMEk8LDsc
8ArKNKylqjGwnDhB1+nS5MonUcU887m4u+wU1HkdsTyyrOIN1dJZNpo6qpEcl7YSdSdS5CY4h07r
jvVY2lbN+jG7MPxVHS5Iqjha2TEG3Nyq3m/UKZrwb9K/oBrj9R3lmoaA1L5MQcGtYTfvToZ2tu6F
4HG2djXPc1rdrjYKWTJ+TsMHJ9IbXcVlSkij0VRD81Lu4FOp52xCV0ZDDscoaeeckRMxWGvNkiBk
ETcYGkm1/wAoTjoqp7hboTEjzX/xBN+7s81lOpecmRaQAPlI1KCmlqH4I23PwU2Sa2FmMsBA22Kp
YDUyhgcBfiuR1DMnxwRShjwdbrqtinimDZZhI62290cs1eoMwMAGyyykdNk6lne20hcFk60FFVVV
ukNQWTq6Wre+Cos4PabalQUwOUdE7WGOd8FU5TqmV0ln9Bj7YVlqJrZ45Gj5xuvxWUZX0lPSQxHB
q12Tn8ryQ90mt8TtqP8AdZ9F3Xf94rJVJDUzSNlFwGJ7cL5G8HEKWkgGSopw3pmyOpWPA5qGeSkd
ITTOcHN4LUsLrYsJtxsjDOBcwyAccJzR09RKLxwvcO4J1FVtaXOppAPDPQ/Tab+Iq3KjaWfRGDEL
KqpaaqpTV0osR1mqkyrJRwaPQhw2g3VVlF8NHBOIwcdtS5a+ryjRvcwNs8BZQyrNS1Giaxh1X1p7
mZRyZLO6MNey+vwzsYXyMYNrjYJuRKwvc04AB9bcVV0M9I4CQajscNipaOSqMgYQMLb61DkOqewO
MjWX3FSxuikfGdZabalDkKoezFJI2Pu2qrop6RwD9YOxwT6BwoWVQkuDtFtipac1NQyIG196qYRB
UPi0l8J2puSaWSGR8NYXdHWgqDJnKGGaWTBEsnUMUE75IZxIwst33UcEk9RombS4+5c2ZMZ0JKzp
79Y2qto5KOXC7W09Vyp8mUwp2z1cxYHbBsVPTwR0daYJccbmH8luGak+lU38VqyhFk1tRpaskkjo
t8FX0EDYG1VK75M7QmUkE2SjLGy0sZ6XesmUwqatrSLsGtyrNDyqYQtswOtbwzx0slXkWmjZa976
/epci1cMbpCYrNF9RWRf1g37hU+Rax88r2mOznk7VU08lNNo5LXtfUsg/TH/AMJSdd/3jnY4Me1x
aHWOw71HlPKLrWoDg8Cst07I6iN7BbSDWFkasnlxxOIsxmpT5TrJ43RueMJ22GfJ9uXU38RZX/WM
3g38lPryBT/fWS6ljgaKbqP6qq8OTKPk8R+Ulvd3csm0gnn6Xzcet6pKvlOVifqBhDFU6qmo/iOV
HBp6mKPidfgFlmfS1mAbIhb3rJzGjJd9Nosbtb1Tmngfc5Sxi2xzlUaPlM2jPQxnDZS68gRH7X9c
1FV08DSJaUSG+1V7W1tHyuN7uh9Qqm6eQ6lo3FyyPrr4vB35KjcBlufvLwq5uGrqh9srLWqKi+4V
lvpCjk3Fqov1TXHd/wBMxBFrg69n93b0/wCdk+8VkH6TN9xTfPTffcpv1DF7lkiNnJZZmsD5Qdim
yxLrjmovPVmyRWz1OlZLY4WhO67/ALxWTnsZkgve0ENxGxVDlSeerbG8NwPvq4Kugjiyi6L6hc34
rKtTU0phZTtwttuCylUTjJsLr4HvIDh456H6dS/xAsufT/8Aywsg64qsbk/XGso/qmh/l/JUf02l
/ihVxySJ/wBJHTt3qYMmydIygLcA6zd+el+lU/8AFasvSv5RHHiIbguoSZ8hz4zfBexPcv8Ah/6T
N9xVM0ktRK5zz1ysmAPyhBi16yfgsryvfXSAk2bYBMvPkGTHrwXsfBZIcJ4KmjdvF2rI8Wg5VUSC
2AYfJOe6R7nu2uNysg9SsHhmrdWRKUN2HBdZA+kzDdo/6rI1ucKjwP5qWsyQ2SQOpHYsRvqWVK+G
rZC2ONwwHesufN0VurrWR7mlruH/AEXDNS/Saf8Ait/NZf1VcR3aP+qg1ZBnLuJssjTiOq0b+rKL
e9QwjJ8OUJbb7M/p6GOVmQoTG5wIfu96dUVLmkOnkIO0ErIv6wZ9xyramqFXUAVEgAebDEnOc43e
4uPE61kK/LTq/siqlpbNNq/tHZ8ksifXRiTgbeKqnZYfVlsYc1l7C3Bf8Q7aX+ZZBI5VIL7Y1Nk6
sjxuMJwi+vPC/BKx4+q4HyVdQGveyopntsW61lQsgo6aiBu4a3LJsTKeB9dNsHUCZI3K1M+N9hOz
W1UWUH0OljMIN3a9aocq8pqWxaANuDvWVat085jwACN5F+KyQGww1NW76osFiLiXHaTdZNqqc076
OoNgeqU7I1O3pPrhh8AjYF1jcX1IzRcxYNI3Hi2X19bM2hyU8BwrrdxIVXVUkNGaSmdjv1nLJtbH
TufHKPkpNvcmS5JocUkUukedg2qOaRswnv08WJOmyNUvE0pLX72rKFZyuYOtZrRZoVNX0rqcU9Y0
lreq5VtdTcn5NSt6F9ZW5ZRyjT1NPFHGw3BBOrZ/dw2qT52T75WQngVbxxYqukqG1Mw0Tjd5IsOK
rm8nyRDC7ragqd1bEdJTtk9wuFSSyVzXR1VLu61lIwMke0G4DiAsgtcJKi4PVClY8PfdpHSKpv1D
P/MsmfT6bx/oss/T5futUvLY6aBtK3H0dbisoc4uLXVTCButsz0X02l/iBZTybVVNTjjw2w21p5i
yXQGNrryv/NX6NllD9T0X8v5Kk+mU38VqylkueqqdIxzAMNtahijyVSzGWQFzs9N9Jp/4jVlGihr
JmjT4JGt+Cq309Dk80jH4nu2rIL2tqpbkD5NSfOSffKildDKyRu1punx5OyidPp9G61nC4Cr6qmj
pOR0zsXacqSo5PUxS7gdfgsqVVM2kdHBI0mV93WN0wNdIxrnYWk63cFQ820YktWtdjtvVZBQxMBg
qsbr9VUVfTcn5JVDobnKCuyTRvtDezus/XqTKp0NWZ4u2feCjV5FnOkmiIedu3+irpaR72imiwsA
81TZSpjTiCsjxBvVO1MytQRB8UULhGW7bb1uzRvwSMd2XA+SkyxTTvImpcUf1eKrspGoY2KOPRxD
co8ZkZg62IW8Vl6c4YYP5nehR5XfSwCLQhwB4o5ekII5MzzVLUPpZmytAJF9veppXTSvkNruN9Wa
LLNXGxrRHDqFtiqso1FUwMkbGBe+oZwSCCDYhOyrXvYWGbVbhrUtRNKIw99wzqpri1wIJBGwqTKF
bKwsfOS07R6Ec00fzcr2eBsiSSSSSTvKxuw4cTsPC+pAkG4JBzXPH0dXD1V/7ut6A2qT52T7xUcj
43tew2cNhTMvvsMcALuIKq6yaqeC/YNgVHlSakaWBoc2+xTZcqHttGxsfxQcWva7UbOvrQy5M3q0
8QVTlOWoiLHQxDvCZXTspX04w4HeailfFIyRnWbsUMnLcoxGot0jbhsVXNlOGXR08HyYHR1XRNQ7
J9Ry5rRq1DO1zmuDmmxB1Fc75R9v/pCfI+Rxc9xJO/MZpnRtjMjiwbGq5G9curf3qTzUkssnzkjn
eJvngNp4f4jfzWXXfpkeF39nuV/VX9bRZRhpoQ00oc8HrKeeWoldLIdZ9TbVfP7/APCEnzsn3j6x
mVMoMbYT6u8XU9VUVHzspco4pZXYY2Fx7lLS1MLbyQuaO/1Iz6t3qtE8xukDTgG13D0ACSANZOxR
5AlLAXzhruFlUU8tPKY5Br/PPQU1PUPeJp9HYalzVkv9+/1BZSoo6SRjGOJuL61Q5M08enmk0cX5
quhyfGxvJqgvdfWFS00lVMI2e88AnUWR4OhLVOx71tsGjwXIMmUrWCrlu8jv/oso0HJXMcx14n9U
rJ9BDJE+pqT8k3dxT6Kgq4ZHUWp7N3FUzIHzsbM/AzeU1+Q3yCAQ7dQfZV9LyWpfHu2t8FFBRUVJ
HNUx43yblV01NPScspWYbdZqyVRsqJHul+bjF7KKbJda80/Jgy/VdaynhME0kR2tNv8ABXH0Zvnp
fvFUMTJauFjx0Sdar6UR1xhhaddrBVlJR0dK1rhind35i1w1lrh7s2B+INwOud1k6GZnXie3xFkI
3luPC7DxtqXJKrDi5PJbjbNFT1EwvHE5w7lJDLEbSRuYe9RyywPxxus5ZRe6TI8D3bTg9BjHyODW
NLncAjkWvwYsDNmy+tFjg7AW2de1iuZa/SYLN2XxblU5MqqZuN+Et4jcqShkqyQ17W24qvyVpnxY
JGMsy3iqmDQTOjxtdbeEzI9S9lO5rm2kH4VVZHdBTuk07XYdosoMnmejlnZJ0mfUsmMMj42Da4gL
KFE2jlawS47i/hnoaWKokdpJxG0D3rmihluIKy77cQVowyfRzdGzrOKjgycMmyASEwl3SdvVcMmh
reSucXX158jsx18V91ysqTOdXTm56BsPcssfK0VHP9Y7feFDSU1ZQXhZhnj296pqOCCifU1bA6/U
aV7lkamEtYHW6LBdZQn5TWSuvqvhb4KsojLDBDpAyJjekVWZKNPFpY36SPeVkOI6Cqe3rHohc0U7
7htaDMd1wdapIXMylDFINbZNay1ryhJ90Kf5TIMBO1pH+yk1f8PRW32v5rIRtWkcY1O21ROOEjlk
uldPUtP1IzclZWnE9Y9zeq3ohZe2UQ7isl68n5Qb3H8lkn9WV1tuv8lk8kVVL99qywP+0ZfBv5f4
At6+f56X75WTPp9P4rk2Geeptidh6I9yqJpJpXyP2/ksm0jYqTlWi0kh6oUUlTO7BUUYDD71WwCm
q3xjZcW96yhUspY45RGHSHU26qcqVFTFo3sZbuWSXNZkx7nC4aXG3gqPK1VLWRtdhwPNrcFlClZz
oImahIW/FTRVbAyOkwMYBvU1PNLQTNqcBeLlpatxVZ+oqfwZ6GQIm4J5t98I7kKecSB5ynv2bllz
Rcpgewi9tdvFZenkY2CNriA691kxzpcn1sbziABtfwTCQWa94X/EHz0H3CtxVVK9mQ4MJtdrBmyJ
UaKr0Z6sg+Kosn6LKsvYj1j3qun5RVyybr2HgM+TsmsqGPmmfaJqoOaRVgU2PHr17llPVlCo+9/R
Qa8gVP3ijnyPIGZQi7wQsqswV1QOJuPesqfJ5Moozt1fkshxSGpdKDZjB0u9ZZaaiCKoifijbtH9
c0H6FkeSX68uz37FsCy1jmpqWVlzHv8Aesntc3JNZj6pDreSyG/FBVQA9Ii496oKWpFfD8m4YX9I
2VdMI8s6TcxzLrLFLK6pE0bC9r27tarm8nyPTwO6xI1fFUl6rI0kDeuzYPisk0j4Hvnl6IDN4so4
31lWWs+u8m/AKpgqIafktHCdnTep6WensJY8N9injdlLJ1O+GxkZtCZEcn5LqNNqfJcW8VkSZgfL
A820jdXiqPI88NU18hbo2G4Kr52z1s0jdl7D3LJWT4azS6Rx6OwBSMwSyMvfC4i/goI2yTxMc6wc
6xKyvQwUj4tFfpA6vDNBRUbskvmJ6dib943f31f129T/AD0v3ismfrCn8VWVz6bKu06PCMQWWKQN
IqY+o/b48VRSSSZKboHfKMFvJcpy6T1X/gVRJO+Uma+O4usv/M0vjmoP1LUfzqgP6bS/fCyrJosq
xSdkNKr4KmpEU1JMbEbA6ymo8pxwvfK92EfbUNNPUXEUZdbaqiknfkmKAM6YDdXgp4JoH4JW2Ns+
RqyKJ0kEpsyTYn/8PvLyWztw7tSq6YUs+iEgfsuVl/rU2vcVkYjktdr3f0Q2BVMFNlRkMragNsFV
wNgmfGJA8AbVVzROyJTtDxfo6s1yCCNo1hVVUG5NNRaz5IwPP0MmV9PHDJTVHUdfWo6zI9G8GC7i
dRPAKumZNWTSMN2k6lk2vghikp6hvybtar35LMbRStOLFrOvO1xaQQbEG4Kblqke1pqKXFIN9gVX
Vr6yXG7UB1Wo5SDaBtNDGWH67lQZRNIHsczHG76qcRiJDdV9irso8rZCwRYGszUOVJ6UYLB7OB3K
uypPVNwWDGcBvUU0kLw+N1nBPy9WltsMYPFF2IknWTtVPlasgjwNLSN2LWqipmqJMcrrn4BQVEsE
mOJ1iqjKdZUMwPf0eA1KColp5NJEbFc85R9q38IVRWVFTh0z8WHZqsoaieAkxSFqmnnnN5ZC7M6q
qnNwuqJCOGLM2SRl8D3NvwNs7nOcbucT4/3j7/2U+jUfPy/eKgmdBNHKBctKq6k1MxlLQLjYo8pT
MpXU2Frmd+5U9TNTOxRPt/VHL1ZbqR/FPe57i5zrkqprqipaxsmHo7LZoqyoigfC13QdtTXuY5rm
mxB1FTVEs8mOV1zayhqqiD5qUt7lNWVU4wyzFw4KKongvopS2+2y5xr/AN6epJZpnYpHlx7/AEBJ
IBYSOt457DNYZrDNQRZNe1xqpsLg7Z3LKleypcyOL5pmzv8A8Wnet/oVHz8v3j66GCWeTBE25U+T
6ynZjljs2/H1l/UQ5Nc6lfUSy6Jlujq2rd6PxVBzfWxyAUbGlo16s2T6qKF2CSmbJicNZ3LLsUcd
RDgYG3Zuzcipqqj0tI3DIzrs/wAQ1Hz8v3iqKNktXCx4u0u1rKNIGV2hgZ1gLBVlPSUVG2NzA+Z2
9W3b06GZgu6J4HEjNoJw5rdC+7tgttUlJUxNxPhe0eCbTzPjdI2MljdpXNldhxcndbNT0NVUi8UW
ridSqKKppraVlu/cmucx2JriCN6qXufkFrnG5s3X7/Qp6eWolEcY1r/4fqLfPsv4KaF8Mjo5BZwU
OSJ56aKZjx0zs4BTZD0UT38qF2i9rKkppaqURx7dpPAJuRKbHg5b0uGpZRooKQtDZy529qpMksfC
JqmbRtds/wDZVZk6jigMkNXituuNeZ+ToBkttUJDjt7vD0KODlFVDFxdr8AsuTkztgb1IwNXeoGh
1RA07DI0FVdPkmhdifFix7GKspaKWh5XTNwW2hUlNA/I9TIYwXgusd+pZOa19dA1zQRfYVlONkdd
M1rQBq1DwX/D22q/lzM67PvBf8Q/PU/3DmyLM6Oua3dILFZUhEVdMBsPS8/246/8BVPz8v3isnfT
qb765MBUST7X4bNVTLLLPI6XrX2cFkmlwUZqBGHSu6qhflB0mGeCPRnaspwNgrHtZs1EKvq+S0sU
gaC82DVPlOsnjMchbhPcshuwUMzjueSqbK1Y+tiu/oPfbCsrQNOU4m7NIG381PT1GBkdPK2JoHBa
B5pZY6mVslxttbNJ/wDLzPAfn6FFT1FRLaA4SNruCjoNDURl+UumHDon/wDmsvj9Ki/h/wBUyR8f
/D+JpscP9VqWRZ2xVlnf2gwjxQybHDXmofUsAxYrb1lKdlRWPezq2sqXKFHLTMpqxvV1Byq8kRNh
dUUsuNg2jbmvuv6GRyG5Riv9ofBZVBbXz332KpvpNP8AxGr/AIg+kw/w1S/qGp8XLJ3SyPVtH2/y
WSGPfXw2HV1lZY/WM38v5L/h7r1PgERYnxKYCXN1HrBf8QNcXUxAOw5skRl+UIrfVuSssSB9fJb6
oA/xBU/SJfvFZO+nU/31X1j6XKjHfVwDEFlila5rayLYetb81k55myXo4n2kYLK2Xz7T4KpNRpXC
cuLxqN1lz6HS+P8ATNkj9WVXi78lRfS6X+I1ZbfgyhTv7LQfiqylGUoopqeUXU2RaiKF8j5mahs1
qmpJ6okRDYEaGY5JFNqx2/qqukmpXhsltY2jPkOoijklje7DjthKGTaWll5TNVXAddZeex9RBhcD
8mtLH/8AD+DGMWy3vzZFjlNY14b0WdY+KqKTJM0r3yVWsnthZRyfyTA5j8Ub9igZkSSBmkeWPt0t
anrqCmpHwUnSx3zGro+aRBh+V8N/H0I3Fj2vb1mm6r425Qp2VUGt7R0m70x+B7HW6rgfJZSrm1ks
bmsLcLba1BlARUM1MY747678VQ5Qlo3kgYmna1SZfk1aOAN169d1UVZmq+UYBtHR8EMvFvVpGj3q
syo6qi0egY3XtVJlOWljwNijOu9yuf6r2UXxT343ueQBc31KnlpsnUZlEjXzyDciXOJLjck3P93R
RSSvwRsLncApqOpgbikhLRfb6sAkgAa1xHo8hZzbyrGcWK1t239v3qp+fl+8VDK6GVkjbXad6q6t
9VNpHtANralBlKohp3QDCWHjuuoZ5YH4onlpXPlfb6nknvc9xc5xJO9TVlTOxjJH3DdmaOpqImOY
yQhrtoQJBBabEbFLNNM7FI8uPeo5pYjeORzPAqSonl1SSvd4lNkfGbse5p7jZcqqv3iX8RTnvebu
cXHvN/UZKroacSxzdSTeuT5AacRnv3XWU69lTo44m2jZ6mKaaG+jkLb7beqIItcHXm5JVWvyaS33
fSnpaiDDpY8OLZniikldhjjc49y5vrv3aTyUkUkT8MjC08D6iOOSQ2YwuO3Vm3KalgZQwTtnu9x1
t9C3o6uOfVmfQltBHU3N3Pth9Bkb5DhY0uPAKrmqeRwQSUpYGHrkJkUkjsLGlx4BGlqQzGYH4eNl
LTVETQ6SJzQeKFHVEREQn5TqqehqqduKWIgcVG98bg9jrEb1WPfLkON7zc3F/PNBkqsnZia0Afa1
KqpJ6V1pW+B3JmRqx4aRo7Ft9qbkSsdi6gsfNNhldLogzp3thXME1vnWX4I08kc4heLOuB5p2RcD
naSpaxm5yrcnOpmNkbIJIz9YLJFNTmSOXT9MfUVXR5O0k73VlnXJw6lTQSTyNjZtK5rycDgNYdJ4
jaqqlkpZdG/XwPFDJlHDCx1XOWuduCqYWRZGmEb8bL4mn3/3BVfPy/e/Y6KjdWTaIPw9G91NEYZp
IyblrrfsjnQZMpYMMGMv2qtqeUzl+iwatibG93VY4+ARaWmxBB70yOSS4jY5x7hdZRqZ5o6dklMY
8O8jathuslVk9VFUaUg4QFuUMEs78MTC4qbJ9ZAzHJDYeaAJIAF77FzRXhmLRi2219ay/spD3HP/
AMP/AEqX+F/VVGWK5k8rA5tmvI2J0k9ZUsxuGJ5DbqspHUk+ic4O1XuubzzfyvSfy277KmyO+opo
5my9Y7LbAjkNrmnRVYe4bkGPx4A3pXtbvTciRMaOUVQa7hq/qq+lhpnsEc2kuPJZIpqMBz45cbyy
zu5VMGR2xPMdQ8vGwd6puT6ZvKMWj32VfQ0UdEyogxa3D4pjHPe1rdrjYJ1FkqjDW1Li558f6Kvy
fFHE2ppnXidu4KloKE5PiqJh3vPFQQ5IrsccURY4DUU9pa9zTtabeWet5DRYJeTNLnagLKZlNX5P
kqGQiN7L7O5ZKpYNBJWTC4ZsHgocoUla8QSUoAdsUjXZOryAMWHW2/AqTKMjcmMqgxtzuVbXTVhY
Xhow8M8E0sEgkidZwWUJHTZHhkdtJaSsi/T2/ccqSokkqq2J+sMPRCEr6vJNUZDcglPnfBkWJ8Z6
WFousk1MlTpoJzjBbvRbgc9vZcQn6/8Ah4eP/wCypmh9RCw7HPAKrIo3vaDW6Kw6t7KudTc2ujNU
2Rw2G+tVzy2kycQd7PyVW8jLFGO5UjG871h32TuZxKXOqH48V9+1ZSqqaoqqV0JvY6z71l8nlELd
2C6HSyBL3H+qySbV0Pv/ACWUm2r6j7yyDblUv8NSVmTGPe00ZxXN9Sra6Oqnpi2MtwO13WX76eA7
sBUNzkKp99sxFv23eFV/Py/eOZsUr9bI3OHcEQQbEEFNjkfiwsc622yjo6qUYmQPI42T2uYSHNII
3FcjqrxjQO6fVUuT62JhfJCQ0JjHveGMaS47kckZQAvofiERa99SGTawxxPEdxJsUmRa2ONzzg1C
+1G25c2VPI+U9HDa9t9vQyD9OP8ACKqaeWoynPHGNZk8lX0MVI5jRUYnfWFtiyjk4Ugiex5cx/FZ
MoBWSPDnENaNypMlQVAqvlXdB9mlHI9C9jZY53aMdc33BZRoIIqeOop3ksJsb58nwxzVsMbxdpup
aXI9FIdNrxdVu2wWUHULnt5I2zba1likp4IaYxRhpJsVkamhmZV6SNrrWtdZDp4ZxVCSMO2DWuW5
MZMIBSAtvhxWCytRsp6roamube3DPQ176PHhYHX4rKFfLSwwPaxpx8VUzvqZXSPtc8FDlOt0TBBQ
dEDbY2WU26bJzZ3xYJG23e5U9XNSuLojtGtZdN6Omce1/TNkDZV/yoKgp3sybHoC1sj9ZcQoIatp
dp52yNI2WWS4IxlWoG6O+HzVVlKrbXSYZThZJbDu1LL+tlI7xz5A+mSfwv6qesyOyWRr6e7g436C
0kD8pwvgZhZpGatiy6P05n8JM/8Al13v/wDyUDy3/h9xG3CfzWQzavA+wVSgc+vH23rK5c7KEuPY
LWVgv+Hz8vOPsJw6bvvHNPryBB4j81Q25ZTfxWrLYIryTswCyp9WQp8WzXZf/wBO+7/9lkP6cPuF
Vn0up/iuz5e6lGfFZL/VlaPvfksmfpGSaiBvX16lk2kqeWQ3icA03JIWWXtflB9vqtATWOnyEGxi
7gdnvUsE8VtJE5t9l88MMszsMbC4p9LO7IzIdH8oLavesnRSwZUiZIzCcJVFqyrXtWT/ANV1w+9+
SihFTkeOPEBiGo991RUnNzZZp5G7Nyp4ZKydwZa7ru1oUEvNZpSW4v8AqqiCWhnYCQXdYW7lIyky
qyOQS4Xgawq7J9PSwYtOHPvqCrZozk2gs8XBbv7lW1EPOlE/SNtbWbqSvbBlaWVvSYbA2T6fI1Q4
zcpw4tZGKyq+SMnaKdxLG2ue9ZXqIKiWAxPxdA3UNXA3JVTA5/TJNgqebQSxyD6pupp8iVDtLIXY
yNmtQTugmEse4/BGsyPOcc0Jx79R/oq2anllGgiwMAt4qLKlO+FsdXDjw7CpcqwOpp6dkBa3DZn7
ZbPbYqv5+T7xVBTCpqmRnq7SpXVcR0dNStwDvWU6bTUWmfHglaF/w/8A81/Kp8tTsleyFjAxpsFP
M+eR0j9rlW1TqWhY9o6XRAUuU62aN0b5AWu26lkZrYqKepIu7X5BQ5XreUMLn3aXa2rL8DWyRzD6
4IPuWndT5EZI3bohZOratwIdUSEHbrzcvquTcmxdD4+hkL6ef4ZRi0TKp1PZ0xJP/ROL3OeXk4id
d1B+nZHcw9eP+mxU55Fkd8v137PfsWQ/odX4/wBFSn/sGf8AmU//AMvw+I/PPkn9Y0/v/JZa/WD/
ALrUdiy5rpKV27F/RZCb8lUuF7G21f8AD3Wqvcr/ACw/i/1X/EPzsH3Tn3LLf0Ki8R+Spg11TA13
VMgusoPyhGWR0kXRttAVcJhkd2mN36r+aZTzy3EcZd4LKdNPNQ0zWRkuBbce5OikbLoyw472wrI1
LUQafSx4cVlPQ1UDcUkdm3VIY6/JvJsdpGLmCp+tMy3vWSahlNW9MjC4Fl1UZFMlS+UTNDHOxFZe
wmClLTcYtXlnyE4NrXXO2IquI5ZU29oUx5Y9juyQfJTtyblARzGpwG3EBVFTQjJk9PDKOh0Rr2qG
pgGRJYjIMevo71kqaOGsY+R2FtiLqapwZQknhN/lSQn1WSK3C+c4H8FlE5PvGKTvxHX/AFWTqzkl
RjLbtIsVVS5DdHK5jSZDstfaqN1IJb1QcW4dVuKdlHJPJxBgkLBusql9M6bFTNc1vfxTMswPY0Vd
NjcN9gfzVflM1LRFGzBEN3FNyiBk11JozfX0veqGr5LUCXDi1EWU8mlmkktbE4m2eryhJVMiY5jR
gVNXy08U0bQ0iTioJ5ad4fE6xT8uVr2FvQb3hcTfWVS11TS30TtR3HYqqtqKvDpSOjsAGeComgcX
RPLTZc6ZQ/eCnVNQ6QSmV2MbHITziR0gldjdtN0x72hwD3AO2i+1XH/w9t1j/wDuT3yPtieXeJV+
Cue0fNas2rhmt/de8Kr+kS/eKyPK2OuZi+sMKygcqsn+QxFh4BVL8raE6cvDF/w9/wA1/KpfnJPv
lFZa/V1P95ubIs0clPNSOdYm9vAqDIU7ahpe9uAOusuVTJpmRsN8G3xVR+oGfcbn5vpeaeUY+na9
7/D0MjSMjrgXuA6B2p1e6mynPIx2Jhf0hxWVeRPw1EErLu6zVkisZT1BEjrMeNfissVcEghggcCx
us22dyyXW08EFU2R9idip62nZkmeAu+UN7BPrYHZHZT3OkB2e/PRzinqopXA2addllCpZU1TpWA2
IG1blSZZEcAhnhxhuwrn+ziG01mW1DvWT8oGidKdFix96x9LFvxXVfXurHMJjDcIttv6FTXz1MUU
bw2zOGZmWK9keDGD3ka0+tq3xOjdLdrjcqnrammDhFJa+3Vdc75R9v8A6Qnzzvm0zpDjv1lzplD9
5d8FLV1UwwyTOcOCa5zSHNJB4hPq6p4s6okI+9mMspFtI63C6udWv+4LK3dm0MuHFo3242KCZTzS
/NxOd4KSOSN2F7C096Yx7yGNBJJ2KakqoW4pIXAcfQbkatc0GzNfeqmhqab5xmriNihgkmkDIxcq
XI1ZFGX9B1tw2qKJ8r2tjbcncuYaq3zkaqIJYJTHILH0QCdSGR4GNbyiqwvO7Uq2jfSS4XawR0XK
KlyM6JhdVG9tetDJuS5IpXRyOOEbbqkpn1MrY2b9p4BOp8i050cshc/ft/ojtNtl9SA7lWMbzTRP
AF7gKmOSI4WmUGR5+rwVRS0dRRvqKVuAs2hUVKaqZrBs2k9ylqck07jE2lEltRdqRtidYar6kyjq
ntDmQPI42T2OY4tc0hw3H9jCrPpEv3s0OWq2Jgb0X97tqq8o1NUAJCABuCpq2opS/REdLaiSSSdp
zS1dTNGyOSS7W7M17awjW1ZFjUSW8c2OTBgxuw9m+r+5WgkgAEngFJTzxi74ntHEjNHQ1crcbIHF
vFFjgbEWKgppqh+GJmIqooaqnAMsdgd+1RQSzPDI24ijkWvAvgb5pwLXFpBBG5UVG+rn0YNt7jwC
nyXRRRSHlnSaNmr0qTJMU1Kyd9Rhup8ikRF8E2ktuUEL55WRs2uXMtF82ao6XxH5Kpp300z4n7R8
VFk7J8UDJKmo64uLallDJsUMLaineXRFWzRRmSRkY2uNgn0+R6PBDMMTyNqynQClkaWfNv2dyoaW
kio+WVLcXAKamoq6lfLSswPZu2LJVI2qqLP6rRcoVeTZpzS8nFj0Q6yq6QwVZhG8jD71PLTZLZFG
IQ9x2rKcEMtLHWwi1+sFkyOOno5ax7bnXh8FBUx5VZLDLHhIFwsm0olrtG8fN3xe5S5YMdYYNG3R
NdhKyvStgqbsFhIL271VzuybS08cLRiI1nwVRavyXpy35SP+iyQGxUdTVbTrt7lkzKE1VM6GexDm
6lUw6Gomi7LtXhnyjLKzJ1I5j3N6uzwVO99Rkio02uwdYlZFGClqp9/+wWSayflbA55cJNt1Wg0+
UZtGcNnYm271BT5QnlY/5TWb4ysuyxvnia3WWg3UOT6J8THGuAuNmpSZPyc1jiK7XbuWTqPlktr2
aBdxT5shxuMfJ8VtVwLql0Lsps0TSIzJqBWWCXZQkB3NFlXnS5Jo5TtDgM1R+iZHZH9ebb71kFnT
ndvDQAhkyicSx1X8uduveuSPbVimf1sQHuKq68UMjKeGFtg3WspOjfkmF7G2BeDZUVPFPIWvmEep
TwmnydJHTfKA9d11kEDSzfcCZzLj0JBL72x96q6bktS+K+raD3KJ2XJGNLLBu7YFllrzSUr5RaQO
s71/+39V/wC/jmC4Ks+kSfe9Pbu9VT0lRUkiKO9tp3KooKqmF5Y7DiNaAJIAFyUzIdc5od0B3E61
JFJE8se2zhuVFk+ervg1NH1iqvI9TTR6S4e3fbcqemlqZNHFtX/w+796F/BVVLLTS6OT3HiqWnM8
8UQNsR2qaHIVM7Ryt6Q8VJhxvwdW/R8PRyRTQ1FUWStuAy6MWQtMYsNn3tvWUaHkcwAN2OFxdUtH
R01IKmrF8WwKoo6KqpHVFIMJbu8FS0zqqdkTTa+/uU9JRx5NqWxNaSwG7t98wa4/Vd5IsePqO8k1
jnuDWgkncEckV4bfQ/FEWUWRq2RmLCG+JVTRVFKQJWbdhGxZKYyno5qxzLnXbwCoMoOrnyQVDG62
6lT0Y5z5OdjXnyCyhlWogqjHFYNZa/estMa+Knqm/W2+/Wg91BkZj47Y321/eWTqh9dDUQTnFq2+
Kpqqeke8xkXthN1k6fKUtWy75Cz619iyyWGvfh7Iv4rJcNDG52gnL3FutV8OSmtmLZSZuF96oKJ1
ZKWXs0DpFPpMhRnA+XX95TiITSCJ12X6Jz4C/IDQBc//APSyJFNC2d8gLWd6yGGurZ3fZNvNTSP5
bLLfWJj+ay+0aSnfxaQmjWPIKs/RcjRwO67tVvisnyUDWPFRAXuvq1X1LT5K/cHf+mqLRPyw0xx4
GXNm+5ZX/WM/g38lX9PItK4/ZWUf1NSW+x+SyD16kfZCyEAJKwKBxFTF/GH5rKgHOlEfD81l36bF
/DUVnZBlB3X/ADW3/h024f8A7LIh/Tf5CsnW55rf5vzVZqran+KVlzq0d+9Zf/5U+Kyd+qaz+f8A
JZP6WRakDb0lkj6fB4FZVtzjUe78swTp4IsnUr5o8bbN/JZXc/kUehtoHWxW4LI5D6Sri3/7hZMa
TWUw4O/JZX15Qm8G/kocrVkTGsD2kN4hZYijNPT1IZhe7b7wqefI7II9LA4vt0vFPrMiljgKV2sc
FkB40k7N5AIUkb4nvY8EEEqkl0dTC/g8XWWoXisEgaS17B5hV40WSKWJ3WLhqVFBp6mKPdfX4BZa
mElZgB1Rt+KyHMG1D2H67dXuUVBVtrWgxO1SXxLKMrW5XiffqYLrK1JPJUiWJheHM3KrhfFkWJrh
rDgSm0FXNG2VkWJpWTaeWjjqJKjoNtsWR6hsNSMR6LxZczVAqdVtHjviWU54nZTYb9FmEO81lGlr
Kl7DBJ8nbispR4MkxMDw7A8XI/YeCGat+fk+8snxskrYGPbcE7FUzZLpZNHJTN2dgLKdBTin5VT6
hvG6xWSKKKQPqJeq06h4KKroK95pzBqtqVRSGCs0G3pC3vUstNkmGNrIrl3xVZHDX0HKmNs9ov5b
lHDLL83E93gFyGr/AHWXyUsEkRtJG5viqTJlTVDE2zWcXb1VZJqqdmM4XN34dygp5KiVsce0r/4e
lt9Ib5Krp300mB41/mpnOyfkqIR6nmwv3lZLqH1sFRDUHF/sVkWnvXPxf2QPmq3KNU2ukwykNY+2
HdqWXmAx084Hd5qcupMjRCM2Jtr+8shzPk08L3FwtfWiZKaokEby0scRcKjoK99TE8te3pAlzll6
Zj6iNjTcsab+9ZKlyfigY2H5e3WVfVZNjnkbLS438bKhpeV1Wj2N2nwTn5NgcYuQl+HViwXWV6GK
ARzQ6mv3Z8hfTXfwin5Kqpa578OFhkvdZenY+WONpvgBv71lvVSUo+1/RZB6UdW3dqXVOo6xvCov
1JV/zoG2Ejcm1uUyB+gfFVNTlXQS3owBhNzdZAhGimm+tfCFFT1zZGufWAje2yy2xgrRh2vaNnFR
0uWXsaX1eDuWVGO5rGkOJzS3Wodf/D77dl35rI36wj+65MI/+IX+H9FlUWyhUe78llH9S038iyh0
si05H2FkD56f7gUhvM947ZPxUOXi97GPgsCbXBWWKRkFQ0s2PF7LIH0qb+H/AFVZ9Lqf4rvzWQfm
am3Wut5vt35xtVJO6nyNpWgHDf8ANPkdlLJrjGbPHWb/AEWQ5QytLT9dlveFVUsvL5Yw3ryavesv
uGOmZwaVk2jEEfLJxu6AVc+rqJXSvheANmo6gqDKbqNr2iIOxG+1H/iCfdAzzTa5xygyqeANYvZZ
VyfPLU6WFuIPaLrK1oaClpr9LV8FC3l2SNC09Nn9Fk6nfQw1M0/R1fksjVLYqvpnVKPimZHmFfj1
aISYllecPr+gfmgPPaqmnGVIYJoXtBG26rsNJkxtLcF52/7rJM8UkM1HIbYr4feqWiZk0STSyjq6
lTVpireUEdZxxeBUmTqapn5S2foHWVlesZUVDWsN2xi1+9Qvpco0bIZpMEkarJqejoTSQvxPdtWT
K5tM9zJPm37e5Rc00JfMyfEbahe6lldNLJI7a43z1NZBJkunhDvlARceCyflCFkMlNU9Q7D/AEVH
WckqMbOkzWLcWrnPJkWKWKE6Q9yZUu5VyiRuPpXLVzhkfbyI3+6FX5QfWPGrCxuxueKaSF7ZIzZw
Qy+bDFSgnjdVVQ6pndKWgX1WCp8tVEMYYWh9thKq6yaqkxybtgGwKkrJKWQvY1pOG2tOe573vdtc
blAkEEGxGwrnqvw4cTfG2tXuSXG5O0qHKVbCzAyXo9+tPrKmRjmPmJDnXKhrKmBuGOYgcFPVVE9t
JK52YVlWGYBUSW8czaidrcLZpAOF9SucOHEbXva/7AN3/vfn3hVvz8n3lkv9YU6ylkyeqqcbS0Nw
ga1lN8UFC2lBu6wHuVMf+w3H7DlkzVlCn8VXgc9Uv8qy/wDPwfcKyQP+zJx3v/JZHdXhsgp2MLb6
8SvlnsU/xWU21rqiAVOHpahhVRC3Rxxio0IHDVdQOp4GvDq4PB7ThqVLTTT1Tm05thJ6fAJuT2wz
MdJlI4wRqK/4iAxU58VlrXQ05HaH5L/h8HSVJ7gsjuHOFYON/wA1XD9Lqh/mFZb6NBTN33H5LKXy
mSIHD7CyAP0ic/Ysqoh1TUHjI5RZarW4B0Hbtiy7EzBBMG2cTYrJX6xg96yz+sJPutWQSOVyDjH/
AFVXlerp53xmBmrYdesKtyjU1UbGvja1t7jvz5C+nH+GVFlJ7MpSxSu+TLy0dyytRcnmJaOhJs8e
CqmPrsmQPiF3NsbLJ0LqKjqZZhhJ3eGbJ/6lrP5/yXBc75Q9t/pCflSve0tM+o9wWRKiINlpXm2P
qp2QqrGbTtw990YmUeUY2ufiDHtJKr6Gasex8c4DLKvETMkSMY/Fgs333WSKiHRy0kpsH7PeoKKm
yaXzyT31akKt4rOVb8eK39FLBk2vLZ9Ph1a9f5rK9bHMY4ITdjN6ydV0z6Z1HUkAfVJRqKDJ0Egp
5Mb3991k/kekcKrqluo96Y3IMDhIJASNmu6yhW8snxAWa3U1ZJqoaWpc6U2BZZVD2yVE727HSEhU
Va+klxgXB1EJ2UcjPON9OS7f0VVSxSVEj4o8DTsGdmUGjJr6TRm537lQ1rqOXEBcEdIKonD6ozxj
Ab31cUMvy4RenaXcbqeaSeV0khuShl+cNA0DNnFS5dqHse3Qs1i3oQ5TrIGYGS9HdcXspZpZnl8j
8RUM8sLsUTy0qesqqgWllLhwzctrMODlD8PjmZLLHfBI5vgbJz3ON3OJPE5i5x2kn35tX7Vq9AC6
c1zes0jxGano6ioJETL22nYFVUNRS4dKBr2WP7DfYv8A38c43Kt+ff4rJf6wp/FZQr5qWvZr+Twi
7VlimD421cWvteHFZIcJ6KemJ16/9SybkupiqxJKLBmzWsoVQOUtK3WI3D4LKNI6vjglgIOr4FPA
ydkl0bndN4PmVBV1FPcRSlt0cqV/7y74J080j2vfK5xGy6kbT5XpmWkwvHmFJkOKGN75Kkah2VkK
ojjkkY82L7WTsm00ExqZqi4Dr2WXpGPNPhcDqKoqqlqqPktQ7CW772Tp6LJtM9sD8b3d91SVLqao
ZLt4hOdkWd4qXSNxbbE/0WU68VcwwDoM2d6ydlGBsBpaodDcU/KVBTQOZRDpO+HmqCengmJnjxsL
bcUK7IjDiZBrH2FX176yQG2FreqFRzinqopSLhu1V1U2qqXStaQLAa+5RyPika9hs4bE3L+oaSmB
dxBVflB9YWfJhgbsz0lU+lm0rWg6rWKmkMskkjtrnXU2UaienbDJhIG/fqVNW1FLfRO27jsVVX1V
VYSv1cBqzMqqiOF8LJLMftHo8qqbW5RL+I5tJIBYPdbxzndmsE0ekFbMf2CycxzdrSPEZuTzluIQ
yW44TmZG6Q4WNJPAKWkqYmh0kTmjj6EMEk8gZGLkqpppaaTRyDXa+eGCSaQMYLkrmCpDfnWX4J7H
Mc5rxZwNiMwVJklk1K2Z1Rgv3KfIpEZfBPpLblTwPnlZGzaVzNSX0fKvleGpPgMNToZtVnC57uK5
JkQf8z/rVZQ0AoHzwl3cb5rZtG4i+B3lmZFI/qMc7wF0WPYbPaW+IsslUkT9LPMOhHuURyflISRN
gEbgLtKpqVz6sU7u2Q73Kauo6Wbk3JgWtsCVlWkZTTNMfUeLhNdFk2hifosUkqqhFX0LqpjMMsfW
VDXspWv+QDnnquVFWnKDpIJ4mWw3Qp71ToIz9cgX7lBR1MVBJE17RIXanXVfT1MLoxNPpL3trvZU
eTJ6kEizW9oqryVUU0ekuHt323ev2L/38UNWYbWqu+ff4qCZ8ErJGWuOKqquWqkxvA2W1KLKNVFT
mAFuHvHFRyyRPD2PLXDepMq10rcJlsO4WzQ1dTACIpXNHBSzTTOxSPLj3+gCRraSD3Jz3u6z3HxP
7XbuPl6JpKoNxcnktxtmhpaiovooi4DepoJoXYZWFpTI3yPDGtu47An5LrWMLjDq7jfNBkmsnjD2
gNB2YlPTTU78EjbHd3qlo5qp+GMbNpOwKoyNVQx47teBtsg0uIDdZOxMyBKWgunAPCyqaeWnlMcg
1/mqPI7pohNJKGNOxVmSHU8Wljlxt3qlpn1MzY2e88AuZKPWwVJ0niPyU0L4ZXxv2tOfJ1FRS0sk
szD0Sd/AJtDkmqDm07rPHj/VCmfykU/1seFOhyNSFsUjQXW2nWVlOhFLK3B82/Z3LJ1DBonVVT1B
sCq6zJ0sD2Q0tnHY7CBmwngVZQUdTP8ANRE96moKuBuKSKw43WSYo4qaWse25F7e5UtYzKQkgmiA
1XCyfRh2UTFJr0ZdfvsqjLEkVW6MMGjY6xWWqdjJY5mDVINajtk/J7HtZeaRVdbXTtwT6m3vbDZQ
U8tRIGRjWuYDvqRfwU0OhmkiJvhO1ZLibS0+nf1pCA3wKy6P02P+F/VUOTWTR6aZ+CNS5Jp5GOdS
T4iN17rIsrI6s49WJth4qCirWZQMjpDo7k3vtWVJWS10rmHVqHlmChbiyE8Dbr/NZDimi0znghne
si4HV05HA281LK4ZQlkvrE5/NZfb8tTv4sITGF72tG0myy04RxU1K07NZ9yoZcmtitPAXPvttdcp
yT+5OP8AIqKCKqryMNo8ROHuG5Pkyg15bDSs0Y2a1lumZGYpmMw49Th3qjrMoaBrIKZpDd9lWNfP
kx754rSM1qi/U1Xb7f5LJJtW0/v/ACUQDcvye/8AJZQ+n1V+0sra8n0J36vyWWvo9Cd3/RZJtySv
B2Yf6KnMYfGZBdm9U09A/FDSnRPdvspYnwTPjcek07VSuc7JFbrOq6uSLk7lIyk5DStmlLGYRs3m
yjqMlQRyMFSSHC1jr9eNg/8Ae9cM43Ku+kSfe9G4zEWRBAFwRf0IYZpiRFG53Gylp54baSJzb7L5
m5Kry2+g+IT2lri0tsRtCpaGoqj8m3UNpOxVOS6qmZjeAW8W7la5AAvfYo8hVTmAl7Gngqinkp5S
yQWP5qhydNWE4ThYNrlU5DmijL2SB9totZNa57mtaLlxsAmZBjwjTVBDzwVbRSUkuB2sHqlZOyZy
kOkkdhiHxUuR6d8RdST4iO+6DSSABrJtZUuQ4tFeo1vO4HYjv8c+SYo5a1rZGBwwnUVLUZJinMLq
YXBtfALLK1Ayncx8Wpj93BUUFNSUfLJxicdiw0mVKeQxx4JG7Fkht8oRfzKqy2IJ3xaC+E7bqrnN
TUPmLbX3IUlW7WKaX8K5BW/u0nksiUzJalz3DVGNXiVHlp76wRlg0bn4e9ZVpxFW2YNUtvNV9Q7J
0FPFAAP+ie7l+SXSPHTj/MLIrWshqakjWP6a1k/KdTJWNbI67X31cFU0jOdxEB0XuBWWKyeKoZFE
8sAZfUq88pyTDOR0hY3+CpncmyI+VnWNz5myyLVSGd8T3lwc2+tUcAGWHs3RlxCylPJy+U4j8mbN
7rLLYx01LP8A++kEZJCxrC92AbG7lTXiyJMZN4db3r/h5oxVDt4DQoJXcubJfXpf6rL7LVUb+LPy
z5JBdk2paNpLvyWS8nz08pmms0BqpXtly2ZB1S51vJZW/WE/u/JZT6eSKZ+/ofktFT82U4mdhjDW
k96NDk6rieaR1nt8f6rJM1RFJLo6fSahfuXK8ofuH+pZSfUS1TNNFozYAeaylUvoIIIoNV9/gpq+
snZgkluL7LKHXkCW3B35rIv09n3Sn1PJcsTSfVxWd71LT5KlkNQajbrLbrLhaaelt7vJVlYKWlgk
DA5xsG+SimGVKSdsjAHM2L/h4D9IO/UFI98kj3PJLiVk6l5RUtZ9Xa5VcNZLVQ4I/ko3DeFlymme
4Ttb0WM1rSUzMl07pY8bMLdSjyrk6G5jpnDwAWS4o6qumkc3UOkG+K56eapzDGNFiw23rLFNHT1Q
wCwe29swWTZXR5KleNrC5aV2U8nPwOwybwFkWQMrcJ1EsLVV00nOMrA09OS496y+flaZnBpKyLBp
KrGRqjHxVdPp6yaTdew8AqLKj6SNzBEHXN73X/xBUewYsn1f/aWlks3SX8NarWZXbUO0L3lh1hVo
rwGcqLtd7C6quV8lpxRbLblgnbkqoE7rvwOJWRJmFs1M7641KgyXNT1OOUjAy9jxRrBzmakdXSfD
YqzJhrJuUQyMwvAustyx/IUzDfRjWoMGUcnsgx2liUjG5NyfO17wZJdQsqChbVB402FzbalT5NZR
SieWobZirJxPVzSjYTq9yye9nNte0uGw/khsCpKqmqaUUtUcOHquUtFkqKNzuVlxtqAIKGz13/v4
obvd+aH1f/e9BdlV30h/3lk9jJK2BrxcE7FUy5LppNHJTtva/UCylQU3JxU09hxtsIWR6GOXHPLr
a06h4KGtoK2Q0+i8Ljaq2m5NUPj3bvBZKouU1F3fNs1lZdtySIj2gTIJpb6OJzvALm+u/dnqWnnh
tpInNvsupX815PiDGjGdXvVLNznRysmaMQ1LItOH1bnOHzQ+Kqcr1DK1waRo2PtZZfhaNFONp1FS
uNBkiPR6nG2vvKyRVSVInhndj/2KyVTjnSRvssVllKtqBXSYJXAMOoBZbAfRwT216viE+9NkNuA2
JaNf3lkGV+mliLiRguqCFvPM4tqZjIWVnvdXza+rqCyp8pkqnlO3oHzCm+SyCzDva34rIDrVUrNx
jv5KrBirp8BsRKSFkWeaeSpdK65s1P6z/vHPkT9YN+45VWSJqisfJjaGEjxWXKiN2CBh6lyVlL9U
Uv8AJ+SyAfl5x9gKnnjpcpSPcOiHvCGUMkzSa4OsesWLK9EylkBj6rwdXCyhOWDEzowdUbbp4yyW
O1wbO9f8Pf8AND7qZfTM46T+qy1YVVEe/wDqsv8AWpv5lkvVkqq4Xf8Aksk9LJdU0bel+SyeCaym
t21WPAy5T/yrLjf0wHjGFs/4f17x/VRjSZAcBuB+BWRBeuuNzCqaQc+zd+Jqyoxza6fV1jcLK/Ry
fSR7+j8Asm0HKH6ST5pnxWVa/TyCKP5pnxKyA+00zOIB8kyneK9sWA303wusvvBqYmcGfnmCyO61
DUkbQT+SyfVcvglp5z0rbe5U96PKMbZPqPsfA71lWgqJKovjYXB4GxZYIioqamvr1fBMHL8kNiYe
nHbV4LJdJLSCeWcBgsoKuenfI+J1se3UueMo+2+AUtTNO8PlkxFuxVETMq00T4njG3d4qoyTPTwP
lfIzo7lkeeIxy0kjrY+r71SUDcnufPLK2wbYKWTT1D5NmN/ldR5FhicJZqgFo1nVZZUrGVU4wdRg
sO9U09LWUYpqiTC5mwnuTpKLJ1LI2KQPkf3rJtbySa7uo4Wd/un02RZHmTlAF9wcppxBVONHL0Rs
KblWuxDFObX16llbKAlLGQTXYW9KyoMpRxRcnqG4o/yRqMhxC7IsR8D/AFVFVupZtI3Z9Zvcucck
YtLoHY9uzeq2rfVzmQi2qzR3Z6fKAho5oMFy++vxVFWOpJcY1i1iOKqKjS1LpmM0esHVx4puX5g3
pQNLuN1NNJPK6SQ9Iqmr5aaKWNjG9P62/wBGPKtfG3C2bzF1PUz1Dg6V+K2xRV1XC3DHMQOCNXVH
SfLu6fW70CRayfWVcjcL53lvC+Zs87BZkr2jgCt6BINxqKJJNyb5tu39kH/vzQ2j3fmh9VDd7kN3
uVf8/J98rJn6wp/FZSybPVVIc3CG4ba1lJ8dNk8Uodd1gFQ6siPI7L1k36dTfeWWoy+tiazrOaqp
7cnUDadh6btv+6yvYZNp/Fv5LJLq/wCV5O1hbvxcV/2zwgWVBWaWA1JZt1YVl/5qn++v+Hv+a8Wr
IhHKK3xVYP0mo/ilZd1UcA+1/RZV6WTIT3sWQB+kTn7CyY8c7Vnfi/NZSbatqfvLKnRyVTM39AfB
VPymQmEbmt+CyC08qldwjVHK3nyo+0XALKzC2um77ELKnyeSIIzt6A+CP6RkEYdrWj/SsgRk1Esm
4Msqt+OrqHf5hX/D3Xqv5VJ15PvnPkb9YR/dcp6+Smys8OcdFqBHC4WWqPC7lDOo/reKjby/JDY2
nps/MLJ1K+hjqJp7DV8AqaB9ZO5oIBN3a1FkOqxtxuYG312WXZ45Jo4mm+AG/vQrqwAAVMlh3o1l
WdtTL+JZJq201T0+o8WKGSYG1XKNL0L4sKypVNqqq7D0WiwKa+mypTMa+TDIzaqyempKLkkL8Tnb
VkuubSyOEnzb9vcUxmSaR7qgTi+uwxA+SqKl09S+fYb9H3LlmTK2NnKrNe1ZTr4pWMp6f5tu/wAF
k3KDaXGyUXjf8Fzjk2ljfyRl3nxTZZGyiXF08WK/ehlegkDXTwHGPs3VfWurJsVrNGpoVPlejjpW
ROhfqbYiwsjligA1Up/CFHO+KYSx6iCuf9X0bpeKllkmkdI89J2emyhLTwzRtY0h6hmkhex7D0mq
rqn1cuOQDZbUocrVsTAwOBA2XCmmlmkMkjruKhnlgfjifhKnrquobhllu3gNXoMkfGbse5vgbJ80
0nXle7xN8xc47XE+/wBG3rrKyt6myII2gjxzGnnAvon244c0cMkhsxhce7WpYJobaSNzb7L5443y
PDWNJJRyRlAC+hHmERYm4tbaoMlVkzcQYAPtalU0s9M/DK23A7iqallqZMEY12uqjJNXTxmQ4XAb
bLV64f8AvzQ3e780Pq+5D/ZDcq/59/3lkz9YU/iso18tLXxa+hhGJqyvTCWJtVFr1a/BZHkZNRzU
99evycsnZLqIqsPlb0WXsp6+GPKxkIxMa3Dq4p2Wcmk3MDifuhVlXFBTMkdHiB2BMqqiEv0TyzEb
kBc4V37y9STSym8kjneKZosqUDGY7SNtfuIXyOSaR4x4pHbO8rJ1UKWqa93VcLOT6DJ8s3KdNqJx
WvqWV6xtVKGMPQZ8SqCspZqTklUQLbCeCNRk/J1O8U78b3cDdU9Q+Cdkw1kHX33TqnI07mzSEY+B
WU6/lcjcGqNmzvWTcpMgjME4vEd6lypRQQubRs6Tu6yZI9j2vaekDe6GW6R4aZqc4x3Aqurn1kuI
izR1WqgyjJRlwtiYfqqpy458RZDFgvvzUNe+jdIQzFiCJuSTvN88E8kErZY+sFNNJPI6STrFGuqj
T8nL/k7W2KGeaB2KJ5aVPWVVQAJZS4cNia57XBzXEOG8I1ta7UamTz9Vq4ep1+mG9/pYSNoI9COG
WW+CNzrcBdSRSRECRjm34iyDXOIDQSTuT8nVrGFxgNlZRZMrZWB7YtR4mymgkhfgkaQVS0c9S60b
dm0nYqrJdVTsxuALd+FRxvkc1rBdx2Bcw1WG+kjvwT43sc5jxZw2hUeSJZWCVzwwHZqVbk6WksSc
TD9ZUVA+rksDZo6xUuRRgJp6jGRuUcbpHNY0dJxsuZqJuFktQdIe8BVlI+km0btfZPHPkijpp2TO
lbfCeKZS5GqrshNne/8AqpoHwzuhd1gbeao8kUzG2nAfIRe3BEWLh3lZNoYXt087ho27v91luGOK
qjwMDQY9yssDuB8kQoKCqnF44iRx2KehqqcAyx2HG6oBHR5PfWFl3HYoZmZVgmilYA8dVZFgbJVH
GPmx8U3LUhqsLmDRYrLKtKKer6A6LxcBTyHJ1LFFTx/KO2naqqqqZ3NE51t2arKlo5qqTCz3u4Lm
L/vTfJU1TLSyl0ZF9YVDWV8tVH8o5wLtfCyyyGcvfh7IxeKdlCteGjTEYR9XUq0ufkWF0vXuLFUB
0OSquYdbXZZHqZOU6N7y4PFtaqY9FUzx7mvPrh/t+abtHu/ND6qG5D6vuVf8/J95QyuhlZI3a0qr
q5KqXG8AaralDlKqhpzC3CW69u66ilkheHxuIcFLlevkYWl4HgM8lTUSsax8pLW7B6AuNYJCOvbr
zWHr7ehb0Ld2ZsE7hibE8jiGlW9C2aloqiqvo2ah9Y7FVUFTS65G6uI2KjyfNVhxY5gDeK5hqfbR
/FVtFJSPa1zgbi+pUeTJ6rpDoM7R3qvyYaSNr9MHXdbZbNQQsmrIo3jom91V0YGUuTxWaDa11zPQ
s6MlUcXiAq+gNI8a8THbCqCgfVvOvCxu0qpp8jxxShs3ygBt0t6yJGx9U/G0H5PeqmPDU1ADdQkN
vQyHBDJpy+NrrWtcKGfJdU/Q8mDSfshV9MKWpfGNY2hZPpKanDOkHSvbf3KsFquoA9o5ZKiihppK
142Xt7lT1UWVGywyRWIC5PKZnwtYXOaSNXcub6392epaaeC2kiLeF06Xm7J0OBoL3ceJQk5yybLj
b02fmFkKNvy05+qLKhyrUS1YbJbA86hwVTSMGVhD9R7gbfmsr1s8M0UULsPQuso2qcmQ1NukLf8A
VRuNLkTSM6ztd/FZHqJJ9PDM4vGG+tZGhDa+UH6gI+KfWzjKLn6Q2Etrdyy3COVU7+3qPuWXCWtp
om6m6/goSZ8iSg7WX+GtZOODJNS8ben+SyJIW1bW367TdQRgZdkHeXeayo484Tm+y1llrp0dHN4f
EZgsgdWpHgsm5NqIagSy9ENv8VlGZk1bK9vV1DyWQ5Hvqpcbifkt6mFpph/mO/NRPILderGLhf8A
EA+Wpj9krJU1THJJoYNJca+5cqyt+4jzVeamWsbp48DiAAO66yrVSUbYIYOjq+AUtZUzgNklJAN1
Lr/4fb3W/NZD+ln+GVT1baTKVQ49QvcD5owZJEhn0+LpXwArLZ/SqTw/qsp13JDEWRtL3DaeCqC2
vyY+csAkj/osmuMeS6qRnW1/ktusleCiy6Rga+AW2EgrLFKyCdj2bJd3gsm0PKZbu+bb1j/RZUrR
USiOP5qPZ3lUFpclVkW8Yisk662DxWUj+n1P3vXD/wB+aG1vu/NDYPchu9ybtCyh8+/73oFauOa1
9iwnh6EME0xtEwu8FNS1EFtJE5t8zMkV7m4tEPeU+N7HFr22cNoVLRT1RIibqG1x2KoyTWQRl5wu
A24VYm1t+xRZBqHMu+RrTw2qqpJqaTBIPA8VQZOlqybHCwbXf7KfIJbGXRTYiNxTWOeWho1k2ATc
i0rGN5RUkPPeAFX0Bo5Brux2wrJuTuVYpJDaIfFHJmT6iN3JZOkO+6bDI6YRW6Zdht3o0uSKMMZP
YvO8rKmT2U2GWL5t/wAM+TGMfXwNc0Ea9R8FVZQpKabRPg3bQAsr0kAjjqYRbEdfffesnRQ0tG+t
kbc7vBQVMGVI5Ynx4SNn+6oKPSV+ik1hl799lU5YNPUmJsIwMNiss08Y0VSwWx7UzJ1bI0OZAbHZ
sXNOUPYf6gtG/SaPD072t3qPIDy0F8+E8ALqvouRvazSYsQ4WVbI+hoKeOHUTquqGV1dQVEcpu4a
rqOWaMHRyubxsVkp1TPVdKaQsYLnpLKM3KK59tgIYFWwVJhipqbULdJ2zUFVZPqaazpdY4g3WTKB
k+OWX5tnxVLzU+obyfU9vxWWNWUHfdaqfJVVU4JTazteInWstyjDBBr1a7qhhcclNaw4S/WT4lOy
JGWOMFRicsii1a8Ea8B/NT5Uhpah0Ig1A9I+Ky1TRs0c8Y1P25/+H/8Amf5VSZKNPNppZG2bcrKV
Syoq3uZ1QLLIhPLf/LKr/p1T99D/AOX3eB//ACWRb8uH3CrT881HJyMffsWHLB/tIQsrMrgyM1Ej
HNvqtxWWNdFRnw/JZC+bqfELIeuCqb3qi+lU436UKtcBlqk8AsuAitYeMadqyBr/APfSTvlMgatw
/IrII/SZf4f9Vk1452q+/F+aqYnCtmb/AJur3rLrrOpPeVl0Ym00o2a/iqK7Mj1LjvxWWSflMmVM
W/pfELI0TjWM1dQG60wbl4uvqxYfgsrwPFe84dTwLLLXQoqSHfcf6RnyAelUeAVBXaeSalqDfFfC
qymdTVDojs+r4LIR/THfwlV6qupH+a5Bf8Qf8mfvKCqnpyTE/Dfajlav9v8AAKSonmka+SQucNiq
YWZVgikheA9u0KpyVNSwulfIzUslyxTU8tHK62LqqmpY8ltlnmlB1WCbeabWbaR/5qLJVNSPE09S
CG7tir6vlVU546oFmpktHlCmjZNJo5I991WTUtNQmlhkxuftIWTK9lMXsl1xP+CNHkXbyvVwxBQc
iFY7HfQ3cAUIsgsOPlF+691lKu5ZMCBZjB0VFW5JFI2EyWFtY1/0Uz8g6J+jHSwm1r7VQVrqSXFa
7SLOC51ydCHvp4DpCOFk9znuc5207fXDd7vzQ+r7vzQ+qhfV7kNyyh8+/wAVQRskrYGPFwTsVS/J
dM8MkgZe1+ospUFOaflMFh4bCFkehjmL55eq06h3qKryfVyOpxF8NqraXk9S6IbLi3gVI+HJdNGG
RXuqjR12THTFuEgEj3JkM0vUic7wF1yGs/dpPJS088NtJEW32XT3c15OjwNGM28yqGfnKlmZMBfY
VkanxVpxi+jB81V5WqY6x4ZbRsda3FZdiZghnHh7kSaLI7TH1iBr73LI1XLMZopXYtVxdZPp287P
ZuixELKtVPy54bIQGWssrfK5Ogm39E+YWunyFdu0s/8AyWQpHCpfHfU5l/JQQjnx43AucssEvr5A
T1QLKsOlyLE87QGH+iHyeQdW9v5lZEdaut2mFRRjn+T3n4LLGvKEvg1VHTyFGfssRzZJ/WMHv/JV
mSnVVVpNIA2wHesszxYI6Vn1NvuU/wCoY/BqyJ9NP8MoVLaXLEzz1C4gqoydRzyGo5TZrtZWWcHI
IMOzE23kqOLKhporVDA3CLC19SNNlIj6aPwLI8Q5xkDjcsB81lOaWSslBcbNNgETxWWulS0r91/z
CyDqZVHvC4qj/QslST/WfrH9FEbPjJ7QWWamoiEIjfhDr3Kp5H1GSanSuxWxa/isl9LJMwG3prJN
+XQW3Xussi9fqG1gUVbVwtwMmcANgWU+nkyJ8os/oqnxzZELIusLj4rIbJRUSHCQ3AsnkHLFVbZ0
/wA1lGmnNbLaNxxnUbLLZwUlLEdt/wAhn/4f69T/ACrJ9cRVSwTPJa5xAuspUnJajUOg7W3/AGWR
vp7fuOWU/p9R97+iyTJHNTTUbztvb3qhoOQGSaaRvVUlQ51TJO1xaXOJFlyyq/eJfxJ8j39d7neJ
uqV8FdQimkfZ7Nnu3qV0GTaJ8bJMUj1kysbSz9LqO1FMhyZBMakTjjbEFV1RqKl8w/k8AuU5Or4m
GoeGPb32WVK+F8bKaDWxtrnwWTcoxwh8Mw+ScucMm0kT+S63u8VDUPhmbM3rA38Vznkt7mzPidpR
3Ktq3VU5kIsNjR3KjyqxkIhqI8bBs3qvyoKiMQwx4I96o6ySklxt1je3in5d6LtFTBrjvurm9767
3uosvTNZZ8QeeN7KrqpaqXSSe4cM9JWzUjnOjtrFtaxuD8YPSxYr96qa2eqw6Ujo7NSgqJoJMcbr
Gye9z3ue43Ljc5pZ5pgzSSF2HZ6DZHs6ji3w1Jz3v673HxOY3O0k+nb9mG1vu/NDd7vzTd3uQ3Jv
1fcsofPP8Vkz6fT+Kynk+oqqhpZbDgtdZQdHS5OFNiu4gALJ/RyM8/Zesm6q6mt2ll76TH9xMtyJ
nLsHvWU2SuobU2HR21hvDuWSXVvyvJ2tLdV8S/7Y/wAhZWFaHwcoLLX1YVl76ND99f8AD+2q/lWR
yOW1vj/VVt+VVP8AEKy1qydAO9v5LKHSyREe5iyEP0uU/wCWqF455qe/Esqi1dUe78llDoZGgadt
mBO+VyCLbmD4FZDaTWOduaxRTDn1/eS1ZYYW1rzbrAWVYNDkWKM7SGj+qp71GQ3Mb1gCPJZFhcao
vwkBrfzTKhoy4X31F+D+iyvSTmrxsjLg8DYq/wDRslQw36XRGfJX6wp/f+Sr6ySmymDiOANF2rK1
M2WNtXFrFulbhxVBhrMmPpcXSbq/2VBRGh008729X4J7jPUEj+0k1e9Q5Bc14M0rcI4LK9Y2eVsc
ZuyPf3oTzAWEzwOGJaab2sn4iqGp5LUsk3bHeClpcm1h02mtfgf91lCCkgexkEmLUcWu6pKukqKQ
U1UQMOwnUqiro6OkdBSuxOdwN0212g7Li/gsq1sMrIYYHXYNZzU2VKZ9O2KsZfDvte6rspwug5PS
ts07Ta2pZPr3Ubzqux20I5Zo2a4aY4vABT5QEleypYzU22ornyn28k1+5VtfLVuFxZg2NVHWy0r8
TNYO1qmy7M9hayMMvvvdUlS+ll0rQDqtrTcuVoHVjPiqiolqJTJI7X8Bnp6qenJMTrX2ok3vfXe6
mqqifDpZC7DsTJHxuDmOLXDeE5znuLnOJJ2koGxuNqfNNILPle4d59G2bV6ev07enbMGOOwEoxSA
a2O8vSYxz3BrRcnYFJk2tjZjdCbDNT0FVUNvHHq4nUqmlmpnhkoAJHG6pcm1VSMTG2bxKq8m1VKM
TwC3iFBBJPIGRi5KfkKrawkPY48FFEZJmxE4SXW17lzBxqh5KtyVyaDS6fFr4Khya6pBeXYGD6yq
skOjiMsUwlA62bJuTBVh73uIaNQtxUjNHLIzsuI8vXDd7vzTdo935pv1UP8AZDa1ZQ+fd4rJn0+n
8VlCvlpa2IX+Tw9ILLFM2WNtVHuGvwWRntko5ae+y/k5ZPyZUR1bXSNs1nxWUahjsqMP1Yy26yrR
zVYh0ViAqeM0GT5NM4b/AP8AkoampgB0Upbfauca/wDeXqWeab5yVzrbLqIxZToGxl9nttfuIQEO
SaV934nu+JVBV8mqhI7YdTvepKCgqJRU6Tbr26isr1zaiRrIzdjN/ErJ1dTyUxpKkgWGoneFyjJ+
ToX6B+N7t17qGokinbMOsDfxujV5GqcMs1g/v2rKWUBVvaGX0bdnesnZSFM0xStvGfgpMr0cMbhS
xdI91gsT2uDsXSve/em5cgLG6amJeOFrKtr31kgNrMb1QqKvko3ktGJp2tVRl2V7C2KLAe0TdblF
lyrZGGlrHW3lVNVNUyY5D4AbBnjkfFI17DZw2KaaWeQySuu5CqqRCYRKdHwTHvY7ExxaeIUtRUSi
0kznDgczp53Ns6aRw4F3o2HD0benb0begNqLXcCrIMLjYAk8AnMLSQ4EHvzR008gJjic4Dbb0I4n
yODWi5JsFzFq6VUAfBPbhe9t72da+egpDVTYL2FruVTR5KjZJ+kHG0HVi3oZgFkyip3QyVE4uG31
eCr5cnSMaKaIh19e7VnAQ5NQUUTzDjL/AOqrZ46ifEyHR2FrLIlOy8srm626gq/Kks9O5hpS0E9Y
58ita+sOJoPyadX0IqXQPpR1sN7BZVo2Us7cHUeNngslxUsLIZC4Okl2dyyqLZRn935LIZwxVbuF
k/L1Q5pGiYFHFJIQ1jS48Auba/8Ad3KakqILGWItBVHk6eq1ssG9oqbIk8UT5NMw4RdZCjb8vMfq
iyoMqVElW0SnoP1W4KppA3KZgbsc8W96ygawFtPRsIaxust/JVXKRIOUF+K31kytqmwCFkuFoO7a
qSR0mSakzPLutrPgv+HwMU534AqKacV7TclxfZ3gstx4K8nttBTGuc5rRe5KyyWxR0tI36ouVYvy
ANH7/NZCbLjlFjoy3X4psDjOYG7cZamPbFVU1HGdTGnH5KuGGtqR/mH1w3e780Nrfd+ab9X3Ibvc
h9X3LKHzzvFQSmGZko2tKrKx9XLjcwN1WsoMqVENOYQGkd+66imlheHxvIKkyxXvbhxNb3gZoa+s
hbhjmIHDap6mef52Qu9AEg3BI8EXE7ST4/tJY9oBcxwB2XGZtFWOF+TSeScxzHFrmkEbj6VJQT1d
8Fg0fWKqclVNPHjOFw323KKGSWRsbBclDIDsP0jX91TQSQyGOQWcqHJslV0r4GDeq2gpYIcTKnE6
+zVm1cUxt3NGK1ztXIsjwWbNLd3ef9llLJ8UMTZ4D0CRq8VQ0T6uXDsaOsVlSjpYaSJ8LB1wL8VD
FpZY4+06ylOS6ItidACbdm6ytRwxaKaEWa/dnyH9Md/DRyuBVGB0Ox+G91limZDUNLBYPGzvWS+R
wxxOxYpZTbwWWP1hJ91qYxz3BrRrJ1Kkjjp4xTg9IMu73qOJ8kojaNZdqXMtdwZ5qoybU00ekkDc
PcVkqkgOgn5QMd+oq+jo5Zi6aqwG2y4UEDqibRR7zt7uK5ryazoPqTj+8FXUTqSXCTdruqVk1uTw
48neS7DrVfzTgnDb6a547Vk2gbUF75T8mzb3psWRag6JjdZ2HWnRNpK3BMMbGnzCppqR1FI+OK0Y
xXaq2ppJwzQU+Cx1nYqGkhgpWzyQmV79gtdaCCtjex1KYiNhw2VJVGillvEH7vJVlcYKOKcRg4iN
XiFPWPmqRPhAOqw8FkyufViXEwDDbYqvKlRVRGJ7WAX3d2fIf03+Qp2ScdY+Z0nRLr2WWKtk9Q1r
Nkd9feVk36bT/fWWh/2i/wC61ZOyi2kEodGXhypqujrjoH0wbcak2CeHKehgdZ4ccJPCy0OWf3qP
yWVYa9sbHTzte3FqtxVQ91PkWHRG18OvxWJ+wvdr71kLpQ1TPD8lRg8pg/iD81WPDcuU3gFlbKFT
TztjiIHRvdV7uVZJhqHDpg/9FSUclTMGDZ9Y8Asq1UbIxQwbB11S1UlLMJGe8cQo8pxyytENIGyu
PWKy64GuaOEayLBjq8W5guq6fT1kz76r2b4BUdfPSXwWIP1SqCtqKp5e5oZExp2bCVkh0BqaqZzg
Dfo371T0lM2qE3Kw+Q33jXdZZgp2SmRs15HP6TOHrhu935pu73fmh9X3Lh7k3aPcsofOnxzW9ELV
6mlo56p1o27NpOxT5Iq4WF/RcBtwq2xRZCmcwF8oYeFrqropaV+F+/qniqHJ8lW42OFg2uUuQbMJ
jnu7gVBA6aZkQNiTvXNOT2apKk4vvALKOTuS4XNddhPkmMxyMZfrOAVRNSZObHEIMSq521E7pGx4
RwTWud1QT4C6dG9u2NzfEZ6ADltP99VmUhSzaPQ4tV73WVYYJaVtXGLHV7wVkyKkJklncOhsaVlp
wfRQPG94+IVNLoZ2SYcVtyOXKkydGJmG+zesusZggkt0r2UOTKyaMPawYTsuVzLXcGeadDI2Yw26
eK1hxUeQ2hgM89j3LKFFFSCMslx4tylcaXI0ejNi4N1/eWRp3zNnhlcXeKyMwNrZ29gEfFTS1Ar3
kOdjEuoLL7B8g/frCkpZuRw08JA1Wc5VWSJqePGHB4G1ZPouVT2d1Gi5UlbkumeYtBe3BoVbLBNO
XQx4WW2WsqPJclUwSCRoZf3rK3yNFFTta7DcdLwUcsseLA8tuLFVv6npPFn5KNzmSxuZrcHCylpq
eo0T52YX8L/BZcMuOJpZaIbD358h/TD9wrmqI1Tp3PJ6V7LK1SKioGHqx6veqD6dT/fWWR+nn7gW
R6cAPq3jUL4Fk508lRUySsc3Fa1xuQhlZWRt6UZdJqJC5FVb8oP8llOleymLnVrn2PVJ2rJ+qupv
vrLg/TWfwgsg25RL9xTVeSmSyB9OcWI4ujvWVMoQ1bYmxtd0XXuVkL6VJ/DVV9Lqf4rlkY/oMthc
43avcufLf8r8VW1Tqmo0jmYbC1lkv9WVX8/5Juz3Js8oyXDJAA4hguPBDKuVJOrC38JTnFxe521x
JKyjryPTH7mbIB6VSPup2p7x9o58jH9Pj8HKetdS5YkuegbYh7llmkDXCpj6r+t/uqE2rab+IFl3
6aP4YVBQcsElpQ1zdypMm8ifp5phZoVXU6erlmbcXPR9yM83tX/iKxOO1xPiVQ1VNNScjqHW4FOy
ZQRtLjWjUO5ZNrOSTh56pFnJrcjwy8pE444bqqqnVFU+bZr6PuXLMm1sbOVHBI1ZRr4Hwx01N823
esm12T4KXC9+F5viRkyBwv5phgE9yCYcezfhQyjkunBdTQHH4KSV8sjpHnpOOtUmUWUtNOwMdpH7
DuzUNXFTvcZINICFV5XdLEYYYtE07c1PUGnmZKG3wnYqmc1FQ+Utti3euG0e780N3u/NN+r7k3d7
k36vuWUPnT4qhjZLVwse24JU7ckU7gySKMG3ZWUsn04g5RT6hvG6yoKHlcxF7NbrcsoQUzMmv0bG
9G1j70U3KeTqWJjYI78bf9VVshrsn8oDbODbj3KOOSXqRud4C65FV/u0v4VJTzRW0kTm34qkydUV
Qu2zW9oqfItTGwuDmvtuCggdNKyNm1yGQNWup/0qton0jgHG4OxyaeR5GDmdYtv73LI9XNK+SKV+
PVcXVHTtGWHstqY5xWVqqYVj2tkcAwC1llE6fJcU2/olQfIZDLxqJaT5rIkhZV4Nz2/kspDR183i
HKnyTVT4JDYNdrxE3Ky1KBHDTC+438Fk6ujhEcRgxOL+t4qvyjyRzBosWIcUb1dZwMrx7lUVcGTG
RxRxXPBVuVX1UOi0OHXfbfPQfTab+Iq3JnKpw/SWGG2xZWnjihZRx7rX8AjsKyp+q6XxZ+Waircl
xRxXjwyAazh3rLMOJsVQJLt2W8eCyfT1rqSMsrMLTsFr2XI6vflB3kqKMDK+Evx2LulxKyw5z61z
SdTQLBWVb08jQHhgKyEPlpj9hcpfDWyTR9t3vCbldkjgGUvyjtQJWX3j5Bm/WVlOaXkMT4nEB1rk
cFkd7301RpCS3vWQHDFUN8CFUQyRzSNcDfEUWuBAwm53ICsiJaNMzuF1JpOZX8o62Df8M1VryHB/
IqAtFbT4u0sqUdZNUhzGlzbatexZTuzJcbZT8pdvnnyJ9O/kKdVmlytMfql3S96y1SgEVTOq7rf7
qi+mU/8AECy39OH8MKlytNTQNibG023lZPym+pke2QMbZt1X1clTL07WYSBbN7lSSNjqoHuNgHi6
yxPFLVtMbw4aO2rxVLVOpphI33juTqnIlR8pL1t97/0WUZqN74xTN1NGsrJlXHTVJfJfCWW1Kokb
JUTPbeznkhUVa+kkxN1g9ZqOWKDrclOLwCqZ+UVEkuHDfcqTKPJ6eeLRXx779yGpUeUZqS+HpNP1
VJ/xBMWkMgDTxJuuKkr5pKRlM4Nwt379WalrZ6VzjGRrG9E3JPHPHJJG8PY6zhsKkkfI9z3uxOO0
p087oxGZXYB9VAkJz3vdic8uPE60CQdRsU5739Z7j4n0dX7eN3u/NDd7vzTfqe5DZ5IfV9yyh877
1kz6fT+KynQVFTUtMbejgsTdV+ClyYKcu1kWCa97CS15abW1KT9Q/wAg/NFQQvnkbHGNZ+CyhJHR
0LaVh6RFvdxWSX1jdKIGNcNV7rFlU/2cA95WVhWEwCfBYnVhT6cCBkTZXRgbwodFBivVF333XQif
UV8jaZ31ycY3BHJsTHB01c7F3lZf+Yi++qvpZFYR2GLIg/THnhGqZ457n7wR5LKwtXz+A/JVXyeR
I2naWtCj+VyER/ln4LIzS6tDuyw3WUzjr57a9g8go8oVkTWsZPqGwWuspfKZKa+QdPoH3qn+kQfx
GrL3ztP90qjeGVlO47A9ZYpJ5JWSxsLxhsQFJSVEbMb4XNbxOei+mU38QLKtRJBXwvaTqZs96yjC
2spWVMW0C/iFuWU/1TS/+X+SpIWz1DInOwg31rmD/vP+lZWnhZDFSxm5aRfusvPNBKYJo5G/VOxS
OyVXBsj5Q0+OEqvZk+NrBTPu7Fr131LJ9fA2E01T1Nx/on19BTQOZR2LncNyi0Ylj0mtmLpeC5wy
TB0oI7u7hb81PUPnmdK/ad3AKjyoIYtDLHjZuVVlZroDDTw4Adp2fkoZpIJGyRmxCGX9WumN/FVV
a+oqGzYcJaBYbdiGX5La6cX+8qzKE9Xqd0WdkZnVs7qVtMcOAb9+rMzK1cxuESA+Iup6iaodildc
52PfG7ExxaeIRc5zi5ziSdpKMshYGF7i0bG31ZiSTckk9/8AgUbfL80No935pv1P5UPq+5N+r7ll
D533rJv0+n8VXZQkpayIf2Zb0gssU2Njalmvj4cUVJ+oB9wfmhdxAA1k2CpKM0dMcLMcx2qoyZlG
V75X4SfFQ1dTT4tFJhvtXOmUP3j4BS1NRNYySl1ti+QypRgY7O/IrmGNoJkqNQ7rLItRFFUva82x
jonwUuTKfTOqJZja+Kyy1NG+CnwuBu6/wWTa2ndTmlqCBwJ3habJ2T4nmJ4e47r3JUc8jJxOOviu
nVeSKoNkmw4h2tqyllBtU5rI/m2fErJuURS3jkF4z8E/KtDCx3JYumfs2VLVGCpEz249Zxe9c9UP
sH/hCrsoPq7C2Fg2N/3TX4XseB1XA+Srq81b2HR4cIPxzQZXq4WBnReN2JVeVKiqj0bgwNvuztc5
rmuabEG4Us80z8cr8RtZNqKhkZjbM4MO7MZJHNa10jiBsF9WbG/tu8z/AH4RY/3ONo935pu73fmm
/U9yG73Jv1f5VlH50+Kp5jBPHKBfCdirazlcuPBh6NrKDKk0NMYMAcNdicxr5zSCm6OHjvTXFrmu
G0G4XPVf2meSdlavc0jSjXwHoBzm62kg9ydJI7rSPPic+r9lZDK8XbE8+AXI6r93l/CnxPYbPYWn
gVFTzTEiONzlJk6sjaXOhNh4elDC6aVkbdriq7JzqTRnHiDu6ytnsrd+a2YNJNgE6KRouY3j3FZO
FCDI+p3dUHYVDPkyrdoWwWuOzZVcIp6mSLaGn4LnOgp2tZBDiHl+ayrDE+mjqmCx1eR/vQ/tA2j+
X80N3u/NN+p7k3d7kz6v8qyj867x9YGkkAC5O4I0NW1heYHho35mZMrXtDhDqPE2T4nxuLXts4bl
TUs1S/DG3xO4KbJNXCwvOFwHDNk7JwqsT3m0YVZS5LjgcY5bv3AOuqKikq3GxwtG1ykyEMB0c13c
CnDCSCNY2qhyRpWY57gHY0KojbHPLGNjXEDNk4A10Hispxs5yY1xwtIbchY8iU/ydmOPG2JZUoqc
QtqIAANWobNfpUmVZaeBsQiabbDdUFZUVOJz2MbGPzVQ/l9eMGwnCPAb1WVTKCJkMDRit8E/K9a5
j2HB0ha9s1s4WRYcVS5/Yb+aqrVtDPh2sebfy+hk6lp2UZqZI9Ideq19nBTV2THxvaKbpW1dEBZF
MRfJE5gJLbqWncKw0+/HYeCy0WMjp4GjZr8tSiEOTqJsrmXe78zuU2WnyRSM5PbE217qFgdJGwus
CbEqkoKanJcx2N/eVWiblculFnEqmp31EzY2+/uCyxKxkMVIzda/gP8ABg2+X5pu73fmh9T3Jv1f
cmbv5VlD50+KoYWTVcUb+qdqkpsjwHDI1gPeSq/JkGg09PuF7brLJdBFK1082toOobveuU5F2YI/
wKq5O6tGhA0ZLfBVVVDRtZePUdWpSx0uUKQytbrsbHeCE1rnmwBJ7ta5JUH+wk/CU+GSPrxub4iy
oY4qGh5Q8dNw/wDYU2V6qVjmYGAOFlkqnEtY3ENTBiVblWaGqcxgGFm3vWWmMfTw1AH/APIqD9Dy
RpG9Ytv7ysk1s00j4pXYtV0+ngGUZIpHYI8RVNFSMpXtikvHrubqtZkxsY5ObyYuKo/kcjveNpa9
3vWR3ubWBt9T2m/uWVWYa+Xvs5ZOrp6qrs+1mxbAq0fplR/EKyTTRCJ9VLuvbutvVNlKnqpms0ZB
2tustfTP/LCo8kNqImSul6J+qAssY2UsUTGfJ3Fz4I54sl1sjWuEYsRfWVU0M9MGmQDpcCoIXzSs
jZtJWUZm01O2kiOsjX4f9Vku3LoPf+Sy19MH8MZsn0baqR4dJhDQnZFpQCTUO+GcLJbWQ0LpJDYP
Nye5Ujsni8dO8a9ZF7qpi0M8sfA6vDPRZQmpBa2KMnZ/sm8kynE7oFrhv3qnlMM0cg+q7WnU4fWw
VA2YD/0WUZdNVzHcDhHuVXG6sydE6LXsdZciq7E6B4AG9QUc1Qxzom3AVBk6rjqo3ujwAbdayu4S
VzWs1kMDffdQUrqSmIiaHSnaVPkusa2WZ7mHe7X/AIMG0e7803d7vzTfq/ypu73Jn1f5VlH5z3rJ
f0+D3rKtLUTVIMcRcMAWHkmSSyQ68BHmskVcLWOppTa51X33U+RL3MMvg0rRPhnbG8WcHt/NZRo5
KpsYY4DCd6dosm0JZiu43t3krJMtQwythh0g1X12Wnyl+6N/EsrmrcItMxrRfVZVMsFPTsfI24bb
CO9U9RT5RZIwxWtuKyO3BW1LOAt5FZQBFXU/eWUNWSIR3M/JVHSyI37jVkYfpp/hlZSIdXT+5ZL/
AFXN/OhsCpunkRwHs3rJDSa5h7LTdZWfir5O4ALIn0x/8NV302p++qAaTJEjG7bPCyXE81sRwno7
Vldrn18bWi5MYsuS1zdQhm9yqMbMjkTHp4f6o7c7cp17Whom1D7IU1VUT4dLJitsWR6fDBp7dJ+z
wUmSKmaR75J23ceCqaeSgnhOME9YauCLaTKcQOKzh5hT5Ighhe9051NW1WGfcVXVkDqGGCF99l/A
Kmn0FRFLuB1+CylUQVE4fFfq69Vs9FlSKngET4SbE2IU+WsUZbBFhJ+sc0OWJo6cRaIEgWDr5qWu
npeoQW9k7FJlirkYW4YxdU1XPS30TrX3FPytXOFsYHgEC7FixG973XOdf7f4BPrqyRpa6d1ju/uK
391f9PzTdo935pv1P5UN38qb9X3LKHX96yZ9Pg96yhlKallwMY09G+tVNZPUkGQ/yjYm008keNsT
nN7lk5uUWVDB8pg+tiWVy3ltNx1X81lieWFsBjcR0lMxmUqEPb1to8eCgrKilxiOwvtBC54r+238
KnqqipA0r722JmDKOT2txdMW/EFSUrcnRyySyDWqOsEVdpnbHk396qMmQ1Uwn0mo2vbessVbHlkE
evCeksmVUMlOaWbVw7whyHJsb3Nfdx77kp8jpHOc7aTcrJ9VTxZOma+QB3S1IbAsm5RFNdkg+TPw
Rypk6FrjTs6Z4Nsnvc9xe49JxuVQVTKWoMjmkjBbUqibTVEslrBzlRV0lI91hiYdrU/LrzbBBbXv
Kmr5ZatlQGhpbsC5+n9gzzVVWT1RBkIsNjRs9JlTUsbhbO9o4ArlNV+8S/iKL3uN3OLj3m+bxuf8
RDaPd+abtHu/NN+p7k3d7kz6vuWUPnFSzCCojlw3sVX1Qqp8YaQMNtasqPKklLHo9HjbfVuUmXZi
OhCAeJN0+WR8mkc67r3uqqvmqsAkw9HgqatqKbEInCx3FOe57nOdtJuc7JJGG7Huae5Pmmk+clc7
xObE61sTreObarDPu/xt/wBPzQ2j3fmm/U9yG73Jv1f5VX/OKiye+qa5weGgGy5iPtx+Fcxj2/8A
pVRFopZI73wnb/4DjaPd+abu935pv1Pcm/V9yb9X3Kv+cHj/AEWRPmZfv56/6XP9/wDw7b9g1eot
6W22aysrLCeCwu4FYHcFgd2StE/gtG7gtE/ghu935obfL80Pq+5D6v8AKm7v5fyVf857/wCiyJ8z
L9/PX/S5/vI/4IsrejbPZWz2Vs1lb0MJ4FYT2StG/sO8lopOw7yWhl9m5aCXsFcnl7K5NL2fiuTS
8PiuSS9y5LJ3LkknELkknEIUMnH4LkEvf+Fcgk+3+Bcgfwk/Aubn8H+S5sf3rmx+rV8QubHcP9QX
Np4cPrcVzb/XfwXNovbo7bbSubR9jzK5tANjg223rkEdr3bsuhk2PtDb2UKGPtcPq8VyGLtcfq8F
yKLido3cVySG17u2OO7cuRw3t0+sAuTQ2J6XVvtXJqfFax69tvcjTw4bhn1OKFPT4up9Y7zuC0MP
s/qj4rRRA9Qdb8lo4/Zt2LAzX0G+S3+X5obfL80GmzfchGeO8fBBgCyj88fvLIvzEv389f8ASp/v
/tFvR1f3HZYTwK0b+wfJaKX2bvJcnl7BXJ5uwuTS9n4rk0vd5rksvd5rkr+IXI39pq5E/tBcgk/9
tK5vl+1+Fc3P+3+Fc2u+35Lmt/B3wXNZ4O8wubPsO/EFzWOHxXNjeDfNc29zPiubmdlnxQydHvw+
S5vi+z+FciZxH4FyJvab+BCjhsuSxfaXJYvtea5NFxctBFwWhh7C0EXYC0MPY+KwRWPRCwR4rYG9
a2zuutHH2G+SwM7DfJC2rVub8VitsvsJXS70Cbbd39US8H3uTydX3gnHo+LXfFai4D7Y+AX/APaP
9RRJxgntu+CGweEYW0e6RWIdbvYPLWr3afuH/UVqx/8Am/k1btXYaPxFE67/AGnnyVtg/ht/qr6j
915W+32mfBO6n8h/1FfW/wDM/IIWLO/Rj/Ur2d/O4+QstwHdGFtu775/oh9Ud7Au/ueVbX+EeSJ1
HwJX+4zCO21Bupbs24rKHz7vvLIvzMv389f9Kn++f2u2aysrKysrehZYT2ShG/slaGXsFaCb2ZXJ
5vZrk8vZ+K5LL9nzXJX8Wrkru21Cjedjx8UKCTv/AAlc3yfa/Cubnfb8kMmu4P8AgubXdl/mEMmH
sH8S5rH/ALK5sH2fMrm1vBnxXNzODFzdH9j8K5BHxb+Bchj4/Bchi7R+C5MztPXJY+2/zWgi+15r
QwcPiVoYOx8VoYfZhaKL2bfJaOPsN8lhZ2G+SFhsAWIq54q5V/Q35t+a6vntm1o5/wDpmG1HqnwK
/wB1q+A+K1XHjf8Aoju/lWvAT4refvfkEdfkt/mhqt4tR6v8p+JVwSO535K277LR8VfX+NG+L3s+
GtX6P8jj+Iodf/zfyardD+Rv+oq+u/2nnyFk3ZhPCMf1R6t/svKLddvtMH4RdX6H8h/1Ff2n/mfk
1aw1v3G/6iiRt73ny1K2q33AsXW8Hnz1L6/8wHkEb4NXs/zK1Yj9/wD/ABCHU/kHxX1yftk+QW7+
VvxQ/q5Dq+4L/qh/std83HN2llL54/fWSJomRShz2jpb1yum9uzzRrqMf8wzzVXI2Spmc03Bdq9O
ysrejZYTwKwHslaGTsFaCX2ZXJ5vZlcmm7K5LLwHmhSSfZXJT2ghSOP1whk+Tv8AJc3ScHfhQyc7
g/yXNj+DvgubHdk+YQyX3fFc1jg3zQyYOyz4oZNZwYhk+P7HkuQM4j8K5FH2vguRxcXfBclj4uXJ
o+LlyeHe34lcnh7C0EPswtFD7NvktHF7NvksLB9QeS/6enfMPR3eq3Z7K3qN+ay93plHxVxm/wB0
2+oHg34oH+vwR2j/AN7E7Y4b8JW0/wA39Ff/APX4p3+5Q/IgLYf5f6p2/wC8fgtvw/3Xu3PKt0v5
m/AKxsLdj/8AIoXxe8/AJrHcPqsHxRa4Ndq7ZWHX72DyWkbY9IXwO39op00GO+ljtpL9YbmrSwYB
8tH1Gjrd+tCohvfSt2vO1CSE26YPVGq52Jxab2D79L6jt/uV9fVktcf2bt3uWvCehJ1SOo7euncn
Qzda/V9ytLs0E2wDYFo5jr5NJtJ+r/uhDUWtyd31d7d3vWhqvY9ra4b1yepB+bZu+vw9yNNVcI9n
b4+5cnqf8obPrHcuTVPbh+K5LLvli/Cf91yV37wz8H/VclP7z5MWz0coMe+Z2EfWXJ5DuXJpOyFy
aX7PmuSP4hckd2ghQScfgubn/a8kMnO+35Lm132lzWeB8whkv7H+pDJfd8VzYzg34rm1v2Fzez7P
kuQM4j8K5CztfBcjj4lcki+0uSxfaXJ4ez8VyeDsIRRa+hvWjj7DfJaOPsN8lhbwCv8AsP8AvmJz
8fSCd1feF/19AmwOa3q9fo6vQHHNbMS3tALSR+2b5haeD20fmtPD7VnmtKzj5a1p28JPwFacezl/
9MrS/wCXL/6ZWM+xlP8AKvlT/wAvJ/p/3VpD/YP82/7rBJuhd5hFtRugH4lgrPZR/i/6LRVXYi/E
f9loKnsxfiP+y0NQd8Y8ytBUe1i/Af8Adcnn9rH+A/7rksvtm/g4e9ckk/eP9IQpXb6p/k3/AGXJ
f+8Sf6f9loB7eT4f7LksftJPxLkkHal/9Ry5LT/5n43LktL2XfjcuS0ns/iUKejH/Ls94utDSfu0
X4QtFTbqeP8ACFaL2TfJdAfUb5LH3BaUrTFaUrSnitL3rS/aWm+0tL3rSjitL3rShaUcVpgtMOK0
wWmCMyKCKO/Noojc4Rc71oot8bfJYI+w3yWFvZGa/wCwDMfD+4Jb4Q4bv9k0vd9XVr+BVjmxDiFi
b2gscfbb5rSRdsLSR9paRnf5FYx2X/hKxfZf+Fa+w5Xk9i74L5X2LvML5X2PxWGbsM/F/wBFhm7L
PM/7LBL9laOftM8itFUe1Z+D/qtDN7Vv4VoX+1+C0B9s/wCH+y0H+c/4f7Lk49rJ5rkse98n4yuS
Qfb/ABuXJoPtfictBB2Fyak9hH5Lk9L7CP8ACFoqf2TPJBsQ2MaPcsSxLGsaxrGsax96x960netI
OK0i0gWkC0oWlC0oWmC0q0y0y0y0yM60y0y0y0xWm71plpVpStKtKtMVpVpVpu9aZvaHmuUs7bfN
cqi9oFyqL2gXKoh/aBcsh7S5ZD2lyyPifJctZ3rlo4FctHArln2SuOfj6PFC+YftNkL2HoauIRcz
thaSLthCRnaWId/kVf7L/JYv8t/krv8AZO+C+U9i7zCs/sfFYZey3zWGbgzzWCb7HxWjm7bfw/8A
VaOX2g/CtE7t/BaA+1d8FoD7Z/wWg/zJPNaFnF34itDH3+ZWgi7K5PB7NvkuTweyZ5IRxjYweX7J
pVpFpFpVpVpVpVpVpVpCtItIVjKxlaRaXvWmHbC5TH7RvmjVRe0b5rlcPtmrllP7ULlsHtPzXLYe
38CuXw8XeS5fHfY/yXL4+y/yXL29h65f/luXOH+UfNcvd7L4rlsnsviuWS+zHmuWTdhq5XUdlnxX
Kajg3yXKaju8lpqjj8FpartnyCx1PbKxVPtXL9J9o/zVp+2/zWCXe93mtE/iVoHLk5XJvsrk54Lk
5XJ1yYrk65OVyZcnXJ1yZcm7lyb1O5b8+/0LZiRxQc3tjzWkj7Y81pYu2FiaVfud5FYvsP8AJYv8
p/kun7I/BWf2PisMnZ+Kwy8G+awTfYWjk7Y8lo39v4LRO7fwWiPtHfBaL7blohxd5rQs7/MrQx8F
oYuwFoYvZt8lgZ2QrDh+33V1f1Tqt1yNH9bijWS+yHmuWTezYuV1HZZ5LlVV9nyXKKriPJaWr7fw
Wkq/aHyWKq9oV+k+0erT+0f5rRS9t3mVoX8Xea5O7vXJVyXuXJO5ck7lyRclXJVyVckXJe5clC5K
FyYLkw4LkvcuTdy5MuTjguT9y5P3Lk/cuTjgtA3gtAFoO5aBaDuXJ1oAtCtChCtB3LQLQLQLQLQo
woQrQrQrQrRBaFXHB3kr/Zf5Lpdhy6fsz8FaX2fxWGbsDzWCbst81o5vsrRycWrRP7Y8lon+0Hkt
E72vwWh/zHLQjtuWgZxd5rQR9/mVoIeHxWii7AWji7DfJYWdkLo8FdXV1dXWJA3/AG6pnEMeI8bD
xza1UT6F0J3OdYomwzX1gJ51s1/WzSBxcyziNp8UFILuYb7HJ+77wzDrv92YdZ/3l/an7qdtb45r
9K3cgdbkT0m+k6HpP1fWK5P3LQLQDguTrQDgtCtD3LQrQrQhaLuWhWhWgWhWhWh7loVoVoVololo
VoVololowtEtF3LRdy0XctF3LRdy0XctCsLBtc3zXyXtGeYWKH20f4gtJTfvEX4gtLTe3j81pqft
+QK0tPxf+By08PCX/wBNy0rPZzf+mVpB7Cf8Cuf3ab/T/uun+6yfib/uvlv3X/WF8v8Au7f/AFF+
kewj/wDU/wCiwz9iD8Z/2QbN/kfEq0nbg/Cf91aX20X/AKf/AFVpP3hn/pq0n71/oCs7fVP8mrV+
8Tf6f9liWJYliWJYljWMLGtItItItItItKtItItItItItItIsaLysaxlaRQOu52vNcLSM7SD2nYc
xkYHNaTrOz1V811N8zJ9wpvVb4eqyrfBFwxfFU0oliYb3O9PygWve0jY4qaofO8X9ydcRMvxbfzz
SfU+8pb/ACf38x67fetlx70/qj7zfzUnV94/PMOu8+GYdaT739Fb9IB+xmucZG6wsrfLNNvqlDru
TsWkj1atd/SMesrRrRrAtGtGrN4hfJ9pvmscO+RnmtJB7VnmtJB7QLTQdr4FaaL7X4StKzsyfgK0
g7En4Vj/AMmT4LGfYSfD/dY5P3d3mFim/d/9QWKf2A/Gvl/ZR/jP+y+X7EX4j/srVPGH4q1V24fw
n/dYKn20f4P+qwVH7w38H/VYJd9R/pCwO/eX+Tf9lgP7zL/p/wBloW+3m/EtCz2034ytDF25fxla
GD7f43LRU3ZP4itHS+xYtFS/u8fksFN7CP8ACrQexj/CFeMfUb5LS8LLTHitOeKMzuK0x7S0x7S0
/etN3rTd60y0oWlWlvvWlWl7itKtKtOOKNSztt81yqP2jfNaRaRaRaRaRaUdoLTs7YXKIu21crg9
oEa2Dtrl0HaXL4ftLnCPsvRr26/k3Ll/+UfNcvPsviuXSeyC5ZN7MLldR2WrlNV3eS01Tx+Cx1R+
ur1J/tXK05/tH+a0cvbd5rQP4lcnWT4sEkn3cznB92o6jZRdceGZ3z0f3Xf0T3Btu82TjhaTwCe/
5Fzx2LhNuWgnbbYru5QG/V0d/ipcQMWE7X2PgpdUbyNoF1KeiPvN/NVP9j/FanAFpCid8gxxP1Bd
A3F0y5o334PUfzbPAeqqoNNEW317QopZKabYftNVfYzB42PCo2h1RGHbLogEEFdIDiozj1kWI3cE
9pdaxtY3WtW1hOja4tJ+qURfNbO24c/vPrCy/wDav81o2+0k/EtGztP/ABlYI/tfiKMUB2sWgpfY
t8loqb2LPJYIRsib5LEBuWkWkWkWkWkWl71pFpe9aXvWl71pVpZMR6uHVbitKtKtL3FaTuWk7lpP
BGccR5rlLBte3zRrYfbM/EFy+n9vH5o19L7di5xpfbtXOdL7cfFc60vtD5Fc7U/ad5Lnan/zD7lz
vD2JPgud4vZPXO7PZPRyuPYu80crH2P+pc6v9iPNc6y+yYudKjsMXOdTwYucan7HkucKri3yXLan
t/BcsqvalcpqvauRqKj2z/Naab2r/NYpO09Wee0tFIfqOXJ5fZu8lyOo9g/yXIas/wBg5fpB+u5Y
Z+27zWil7R81oH965OVyVclXJe5clXJlyZclC5MOC5MFycLk44Lk/ctB3LQLQLQjgtCtCtCtEtF3
LRbVEzCTmd13fe/orhQ9YZnfPReDlOzGGar4Xgro9ym+Zk+6c1jyi/8Al/1UgJ0dtz1ICY3gb2lP
aSweLfgVKwuwW3PBzNj+QEZ7GFC+9MhDItHfj8U0WaBwHq5IIpeuwFChphboXtx1p1LTuc1xjFxs
tqz2F7+lcK44rG3isbeKxs4rSs4rSs4rSsWmatM1aYcFpvsrTHsrTO7K0x7kaj7TUamMf2jPNGtp
x/bx+a5wpf3iPzXONL+8MXOlH7cfFc70ftT5FHLFJ23fhXPNN/meS55g7Mq56h9lIuemboX+a56/
7v8A6lzy/wDdx+Jc8zewZ5o5YqPZMXO9V2I1zvWcI/JHKlb2mj3LnKu9t8AuX13tz5BcurP3hyNZ
VfvD/NcqqT/zD/NaWoP9rJ5rFIfrv8yrPPb+K0T+w/yXJ5d0D/Jckn/d3+S5FU/u7/JDJ9X+7uQy
dW+wK5rrfZfFc1VnZb5oZJrN+BDI1V2mLmWp7bEMiye1auZf85czRe0K5ni9qfJDJMOK2N3km5Jp
vavXM1J2noZIo/t+a5pouw7zQyVRexJXNlLb6OubaYf8sFzfT2+itQoIB/y7FySP2DEIG6/k2BGP
Vq0YXQ7cauz2rFpIR/bsWmg/eGo1FN7daHuWiWhWiC0IWiWi7lololo1o1o1o1o1o1gWBaNaNYFg
CwKzVePtBXj7QV2cViZ3+SaQeOaw4LV+w3CuOKxN4rG3itI3itI1aVi0zVp28FpDwWlPBaZ3ZWnd
fcjPb6zVyto/tWea5dF+8R+aOUaf95Z5o5UpR/zAXO1J7Y+RXO9J23+S54pf8zyXPFP7ORHLMW6G
RHLI1fo581zw793/ANS54k9g3zXPFR7Ji52quzGudKz/AC/Jc51vbb5LnGt9t8AuX1vtyjWVJ/5h
/muUTH+3k/EtLIf7V/mrvP1nnzWF/Zf8VoZPZP8AJclm9g7yXIqn2DlyCr9gUMnVW+JDJtYfqN81
zVWcG+a5pqvsea5oqe2xc0S3tpmLmd/7wPJczf8AePguZ2fvB8lzPD7dyGSaX2ryuaaXi9c00f2z
71zXSD+zPmubKTfAfMoZOo/YIZOpr/MDwsm0MQ/5dnkhSRD+wZ5IUzfZs8gm08bHOOrXxssDR2Fs
+s1F0XtGrSRe2ajLTW1zC65RT+2Rq6X2h8lyuk7bvJctpd5cjXUn21y+lH1XLnGn9m5c4w+xPmhl
CMf2J81zmPYBc5f5DVzm/wBixc5y+zYucZ+DVzjVcW+S5wqu0ucKrtrl1Uf7QrldT7Vy5RN7QrlE
3tHLSydsrG7tFYndoq54+lZvELodoLodoK7OKxM7/IrE3g7yVx2Sr/YKufZn4K7+x8V0+yPNdPg1
fKfZVpe03yVn9v4Kx7ZVj7Q/BYftuWBvF3mVhZ3+awx8Fhi7IVo+yFccFiWNY1jWNvFY28VpGcVp
GrStWmC0w4LTfZWlPBabwXKO9qNW32jPNGtiG2ePzTso0/7yzzRynSj/AJkI5WpPbHyXO9J23+S5
4pf8zyXPMHspFzzGP7B3mue/+7/Fc8v/AHcea55n9kxc71XCPyXOlZxZ5LnOt9oPJc41vt/gEa2r
P/MOXKqrfUP81ppz/bSeaxSH60nxWF5+q9aF/sneS5NL7B3khST+xK5FUeyXIKrsDzXIKngEMnVH
Fq5tm9oxc2Sb5Grmz/PHkubG+1XNkftShk6DtPXN1N9vzXN9L2Xea5BSD6hPvQoqf2K5JT+wC5LA
P7Bq0MY/sWeS0X2GrC2+xi6I3tWJvbajK24+UC0sftQtPD7Vcop/aLlVN7Ry5XT965ZT77rltP2H
Ll8Ps3LnCL2PxRyg32K5w/ygucXezaucJew1c4T8G+S5wqfsrl9T2guXVXtFyyq9qVyqoP8AaFae
f2rkZpT/AGhWOTtlYnHeVc8T6Fv7gu3gFiCxrGsaxrGtItJ3rSLSd60i0i0i0ixlaRGYcQuUxD+0
b5o1tMP7ePzXONJ+8sXOlF7cLnai9t8Ecs0fbd5I5bpeEnkjlyH2MnwRy4z2DvNc+f8Ad/8AUjlu
X2DfNHLNTuiYud6vsxo5WrfsD3LnOu9r8EcoVntyuWVf7y9cpqD/AG8nmtLKf7R/mruO9ywE/VK0
LvZlcnfujXJaj2RXJKjsLkdRwC5FN9lcgl7TUMnv9oFyDjIuQN9oUKBm95QoIO09chp/tLkdPbqn
zQpaf2S5NT+xatDD7JvktGzsNQjHZavJYh2gsbfaLSM9oFpYvaLTwj66NTD2iuUxd65XDwK5XFua
VyxnYXLB7NctO5gXLX9gLlsnZajWTdy5ZPxXKp+2uUzdsrTzdsrSydsrSP7ZWJ3aKueKueP9/HK9
H7X4Lnij7TvJc9Uv+Z5LnuDsSLnyP2T0ctt9gfNc9v8AYDzXPc/sWo5aquwxc8VfCPyXOtd2m+SO
Uq723wXL6794cuV1f7w/zRqKnfPJ5oySn+1f5q7u07zVvFYD2Von9grQS+zKFPN7Mrk0/YXJJuAX
IZ+5Chm3lq5A/thch4yLkLfaFChi7blyODi5cjh70KWDsLk8HswtBF7ILRM9m3yWFuzCELAblq4h
Yh2gtIzthaSPtLSxdpaaNcoi71ymNcqb2Vyr7C5UeyFyp3ZC5S/uXKZFyiXiuUS8VppO0jI/tFY3
cSrniVc8f76v+2aCXsrk8vBcllXJJO5cjfxC5E7tLkX21yL7a5G3tLkUfaK5HF3rksK5NB2VyeLs
rRM2YVomdgLCB9ULyXvWIcVib2lpG9paVnFaVnFaeNadi5Q3guUDso1P2Vyg8Fyhy08i08nFaaTi
tK/tLG/isTuKueP+K8beKxs4rSMWlatM1aZq044LT9y05WnK054LTPWletK/itI/isbuKxHir/4J
P/3l3zYmdpXB2HM6VjdpXKY+9MlY/Yf8XucBtK00fFNe12w59NGN608XemuDhcZnzhptZcqHYKjd
jF1LJo7atq5S7shQymTFcDNLPY2b5rSSdopk727dYQIIBCqHOaG2O9RPdpG3Kl6jljdq6RzVPXHh
mp3Yo/BT/OnwVL9dONmuPcr31lcnNtuvMx2JjDxH+LTqBKc4uNytC/DiXgmHExpR2HMRbyVOeiR3
5p/nD4K2q6pjqcFUnWzwRFg08VS9d33c8OHR6vejBJc2Cia5rACqnqDxUfzjPFS/Nv8ADPU9Zvgm
NxNf3C6p3dO3FVPzg+6qXa9S/Nv8MwXJ4k0ACw/xa7W0+GaOVtgCuTs700BosFIbMd4ZqgWLPBU5
+U92af5z3JmuKXzVOflPEKoPyngFKLMi8FT/ADnuzTRYSXDYmuLTcFMqAetqOap+b96Z12eKk6j/
AAW5DYPBVP1FS9d3gj0H+BVT1mnuVL1neCf1H+C3KQOdG3Dmp/mx4/4ukhvraiOKZK5ngmkEXCn+
aPihtCqR0Ae9MNntPfmn6/uVPrLx3JhwvHiptcj1U9WJQfOjNjZ2gnQMcLtKcLGxVO+7SOCn+ZKb
1m+Kf1XeC3JnUZ4Kp2M8VT/Oe5VLdbSi67W9ypuufuo7CtyjN2N8E/ru8VTfNfzf4u5T9hdGVtyE
4WcRwVNscqjqt8VHrkZ4qUXjdmabgFVHXHgqbrO8FKLSFDW4eKquoPvKL51njmeLPcO9QStawtJt
rUrg59wqa/TUgvG4ZjUdHZrVkBZoCqeo3xUHzrVK3FGRmp/nPdmlhLTcbEC4cU2N7tgTW4Wgf4uI
LTYpkxaLWRNzdQNws8VUnW1QfOhW3Kx4KEnRi6na4vFhuUDHBxuNymjLiCAo4n4xcKVhe23emQPD
mnVqOaWIP171yeTuQpnbyE1oaLDM+Brtexcl+2mwsZ45i0O2hBrR9UeiT/jFzGu2hcnZxKbDGN2b
UtXD+79h/wDs7t/4Sn+6L/8A2nauKuOKu3isTVfuK/lK19gqz/ZlYX9lYJOAWCX7P+LLeo9xXuVj
wKwu7Kwv7Kwu4LA7uWF3csDuIWjPaC0f21o/tLA3tFYG8SsLFhYsDOCszshdHshauAWJYlcrEVf1
A/w3buK9xWF3BYH9lYH8Fo3dy0buIWiPaC0f2lg+0sDeKwt4lYWKzOC6PZXR7KuOyFq4BX7ldXKx
FYirnPf0dfo3HFY2D6wWli7YWmj7S08fFB99jXH3K8nsZPwoNn/d5fwrRVf7rIuT137sfNCirz/Y
gfzLm+v4R+abk+tN7uiC5tqP3iPy/wAFe4qx7JWF3ZWB/ZWjfwWjf3LRu7loj2gtF9paMdpaNvaK
wM4lYI+9YWcFZnZVmdkLo9kK44BXWIrGViRcViV/VEqN5LncFj17QtLH2wtNF2wuUQ9pcph4rlEe
4O8lpf8AKk8lilP/AC8vkg2qI1Ur/ehBXO2U3xXI8o+wHmuRZQ7Mf4lzflDjF5rm2u3ywrmybfVN
/Cuanb6v/Suam76t/kuaYt9RKua6T2kp96GTKDacZ965vyf7In3rkVAP+XCFNRD/AJePyWjpd0DP
JN0bf7JvknSjsjyWnRnPFaYrTHitKtKVpVpVj9cLAf3L71cK7eKxBX7itfZKs7sFYZOwVo5OytHJ
3LRP7lojxC0Zv1gtF9taJvaK0TOJWjj71hj7Kwx9lWZ2QtXAK6xFYirlXz39TiHELE3tBaSPthaa
Lthcoi7a5TF2lyhnB3ktPfZG/wAlikOynl8kG1J/5WRaGu/dT5rk+UPYDzXJMo9hg965DlDtRBc3
1/tYlzdVfvTfJc1z3+l/Bc0nfWO8lzTFvqZVzTTDbJIfeua6P/M81zZQdgn+YrkGTx/Y/EoUtF+7
sWhpB/y8fkg2IbImeSxNB1AeS0xWlctKeKLr71jPFaU8UZNa0i0mtaQLShaVaZaULSBaVaVaVaZa
ZaVaUrSrSLSrSLSrSLSLSfsQ/YvcVZ3ZKwu7JWB/YK0cnZWietE/uWhf2gtAe2tD9taIdpaJnErR
R960cXA+awRdlYIuwFZnYCuOAWJYisRV1fPdEoegSgVib2gjIztBGaLthaeLtrlEXFcpj3Bx9y0/
CJ59yxynZTvX6Sf+WcsNZ+7fFCKu9gPNcnrv8tclrPaRoUVV7dq5BPvqvghk8211TkMnM31Ei5sp
/ayIZMpN5f5rm6i7J80KGhA+a+K5LRj/AJdq5PSfu7PJCOnGyCPyXR19EeS0q0y0hWNaRY1pFpFp
FpFpFpVpO9aVaTvWlCMiMi0i0xQlWkWkWkK0q0juKEixrSd6xrGtIVjWkWNY1jWNaRY1jWkWkWkW
NY1jWkWNaRY/2HXwQDuBWjf2StE/gtC/gtC/uWhPELRd60TeK0TO0VgYsLeCws4LCzsqzOyuh2Qt
XBXWJYii4pjrq5V1dX9Tib2gtIztt81pou2Fp4u0uUw8VymPvWn/AMt/ktJJ7B6vU7qZytWewWjr
fZtQp609hcmrPasC5JUfvHwQonHbUFchbvnkQoYfaSJtFTfaPvXI6XdH8UKamH9mFoKb2TfJBkQ/
s2+SGAbGhXWNY1jWNY1jWNaQLGtIsa0i0i0nELSLGtKtKtItKtIsfetIsa0i0i0ixrSLGsaL9SD1
jWNY1jWNY1pFjWkWMLSLGtItIsaxrGsaxrGsZWNY1jWNYljWNY1jWJYliWJYliVytE/gtC9aF3ct
CeK0XeFoR2loR2lom8VomcStFGtHHwWGPsqzOyF0eyFfuV1iKur5rq/p3VxxV29oLGztBaWPthaa
LtBcoi7S5TF3+SM7ODvJCS2yN60knsXLFP7Er9J9krVfswsFX9haGq7bFoKn2oXJpP3j4Lkh9u5c
ib7Z65HF2nLkdPwd5rklL2PiuT0/swtFAP7JvkrR+zb5Lo9keSxrGeK0h4rGUZdyBtvWNY1jWNaR
Y1iWkWkWkWNY1j71pCsaxrGsaxrH3rEsaxrEsSxrGsaxrEsa0ixLGsaxrGsSxrGsaxrGsSxLGsax
rGsaxrGsaxrEsSxrEsSxLEsSurq/q7KysVZWWErAVhKurq6v6d1iHFY2doLSx9oLTR9paeLtLTsX
KB2SuUf5ZWmfujK0k26JYqjsBfpP2Vhqe01aOf2gWik9qtA72pXJx23LksfacuTRcD5rk8PZQgg7
AWjh9k1YYvZtXQ7LfJXbwCusR4rErrFqWJYliWJYliWJYliV1dYliWJYldF4QcTv1K/BYliWJYlj
WJYliV1dYljWNY1iWJYliWJXWJYliWJYljWJYliWNYliWJYliWJYliWJYliWJYldXV1dX9XZWVlZ
YVhWErAsCwLAsC0S0a0S0S0S0S0S0S0S0QWjCDFgCwBaY9krSv7C0knYWKXsK83AL5biFabtLDJ7
QrRu9oVovtlaFvFaGNaKPgsDOysLOyrN4IFXV1fNdXzXV1dByxK6usSusSxK6usSxLEsSxK6xLEs
SxrGsa0ixLGsaxrEsaxlYljWMrEsSxLEsSxLEsSxLEsSxLEsSxLEsSxLEsSurq6urq5V1dX9Tr9C
ywrCVhKwFYFgWBYFo1o1o1o1o1o1olo1o1owtGtGFowsAWELCFhCsFZalqzalcK4WILEtIFjC0i0
i0i0q0ue+a+a+a6urq6urq6usSxK6JV1dYliWJYliWJYliWJYliV1dXV1dYliWJYliWIrEsSxLEr
q6xLEsSurq6urq6urq6ur57+tsrKxVlhWFWWFYFgWBYFgWBaNaNaNaNaNaNYFgWBYAsAWELCsIVg
rDNqWpalqVwrrEsQWJY1jWNY1jWkWkWkWlC0oWlC0wWmHFadvFaccVygLlC5QtOtOVpnLSuWlctI
5Y3cVdXV1iV1dXV1dXV1dXV1dXV1dXV1dXV1dXV1iV1dXV1dXV1dXWJXV1dXV1dXzXV/W2VlZWVl
ZYVhKwFYVhWFYFgWBYFgWBYFgCwLAsKwrCsKwrCsKtmt6GpXV1dYldXCxLGsaxrSLSLGtItJ3rSr
SrSrTBaZaZaZacLlC0y05WmK0pWlctK5aQrGVcq5V/TsrLCVgKwFYCsC0ea+a6ur5rq6v6F1f0L+
jf0b+qtnsrKysrLCsKwrCsKwLAsKwrAsCwrCsKwrCrKysrKyt6er0Lq6xLEsSxLGsaxrSLSLSBaR
aVaTvWlHFaYLTLTLTLTLTLTLTlaYrSlaVy0jlpHLG5YisRV1f07KysrFWKwlYFgWBYFgWjWBaNYF
gWBYVhWFWVlZEBWzX/a7KyssKwrCsKwrCsKwrCsKwrCrKysrZ7epCurq6urrEsSxLEsaxrEFjWNY
1pAtIFplpQtMFplplpVplpVpStI5aQrGVjcsRWIrEVfV6mxVirFYSsJWArAVgWBYFgWBYFhCDFgC
wLCsKwrCsKwqysrKwVv2eysrKysrKysrKysrKywqywqywqysrBWzWVvVXV1dXV1iCusSxrEsaxrG
sa0i0ixhaRaVaVaVaRaVaRaQrSFYysbliKxFYirn1FirFWKwlYSsJWArAsCwLCsCwrCsCwLAsCwr
CsKwLCsKwqysrKysrLCrKytnOe3q7etIVlZWVlZW/YLq+a6urq4V1dYliWJY1jWNY1pFpAtIFpVp
FpFpFpVpFpCsZWMrEVcq5Wv1FlYqxWErCVhKwLAsCwLRrAsCwrAsKwrCsKwrCsKsrKysrK2e3oD1
ev8AuSysrZ7q+e6urq6urrEsSxLEsSxLGsYWNY1jWNY1jKxrGViKxFYirlXzX9KysrFYSsJWErAV
gKwLAtGsAWBYAsAWFYVhCsrKysrKysrK2aysreqH7PbMPXWz29K2a2a3o2Vlb0LLGsaxrGsSxLEV
iKxFYirq/qrFWKwlYSsJWBYCsCwLAsCwBYAsKwhYQsKsrKysrZ7ZrK3qbelbNv8AQv6DHFtwN/qB
/c9lb0LelZYVZWWHuWjctE7ghDJ2ShSy9krkFSfqFDJ1QfqoZLnXNMttoVirKywlYSsKwrCsKwrA
sCwrAsCwrCsKsFZWVkArKywrCf7/ALejb1VkPVWVs1jwWA8FhPBaN/BaGTguTTdgoUU5+oVzfUdg
rmyoP1VzXMhkqTtBDJevW8LmqP2i5shBHSJQybTjim0NOHbE2ip2nqalyel7DUIoR9Rq6A2Nasbe
IRmatMztJ08fFcqi3Fcrj4q2eysrKysrfsR9C3oj0w1ttfpX/Y7fslvQt6NlZWVlZYVhutG7gtBJ
wWgl7KFNKfqrkc3BCgqD9RDJ05+qhkubuXNb9hchky/1wuam9tc1xC3TUNJC+aZv1WjUUMnwLkNN
2SuR0/ZWhph/ZhaOHV0QrRbmt8libbgFpGLTs4rlEa5SxcpajVNRq28Fy49yNcuW9+pGuNly08Ua
xOrnLlZ4o1Tu0uVu4o1X2kaodpcrHFcrHf8A37b1Vs1vSsrKyssKwFaN3BaCTslCml7JXJZuwVyO
bsrkE/ZQydOdyGTJkMlv4hc1/aC5sHtFzbHfrrm6G/WK5vp/tJtDT21sRo6Yf2a5LTdhaCm7ARbB
cNLGpzYeDViZutZaRoA6S0rFpmhcoajUN9y5SzcU6p1bkKtoFrLlYHgjXNOxGs1hctRrTxXLD3o1
ZtZctfuK5W7eUavvXLPtI1g7SNaO2uWNG8o1o71yzuK5YeC5W7guVSLlMvcuUS8Uya+30JJgNQOt
Xv8A31b0bK2ayssJWErAVoncEIH8FyeTslcll7KFHNwQoZDuQyfJdc3u7Sbk+/1lze3trkEYHWQo
IuJXIoLb0KOn1aiVyaDsowQdlYIhcBoVmW6rVibwCDwsY3FGX7S0zPrXWmatO1cpb3LlIRqtWpcr
tvXK+8LlfeuWd9lyz7S5WuVrlh7kauQ6lyl3FGqPaRqj2kaz7a5a3tLlre0uVs7S0l9d1pHcVpTx
Wl706WwujUu3LlEi08naWml7RWkk7RWJ3H111G+4tmd1TmB/u+ysVZWKwFYCtG7gtE87kIHn6pXJ
pOwVyWTsoUsh2IUTyuQu4rkXeuQjiuRDiuSN1rksS5PDwQgg7K0cQ+qFaO/zbVhZq1BdHuWIBaTv
WlbfatM1aZvBacLTMXKAuUtXKhxXKlylcq71ysrlTlykrlBXKHLlBWncjM5aZ3FaY8U6fDtK5W3i
VyxveuWDcCuV/ZXKjwRqn8FymTuXKJe5aeTinTS9taR/aKxP7RWvj6cby0q91ZbAU95ch6lmxYYs
DtztybGCxzi4C3qQbJpuL5pmYXXzD9mtmsrKxVisJWFYSsBWjctE5aJy0LuC5O7guSvQpXrkj+IX
JDxXJFyQcVyVo3rkrFyWNcnj4LQR8FoouCwx8FZnAK7L7Art7ljF1jA96xjisbVjC0oWnWmC06M4
XKWrlC5SuULlC5QVpytOVpitP3o1A7QWnHtAuUsH1kalvbXKmcSuVM71yocCjV/ZXKj2Vyp/ALlM
nctPLxWnl7S00vbKxP7RWJ3Epk7ht1hYr5rpz8I1pzsR9NsUjxdrbrk03ZXJ5OLfNaE2s58fmhTi
9tM1CJgcQ+SxvwToYo3Wkee5TxaJw13B2enE7dmdraRxWz1THRgdIFRO0d3OZdp3qZzHPYWtspsQ
FjG0d4z29KN1jmewOCLSEPRsrKywlYCsBWArRlaIrRFaE8FoXLQlcnK5OuTrQLQBaALQtWhYtExa
NnBYGcFhbwVmqzVcK4WILEFjCxhaQLShaQLS6ta0y0y03etN3rTrThacrTIyrTLTW3rTjiuUjijU
tG9cqbxXKmcVypveuVN4Fcr+yuVu7K5S9coemzhyxK6xLEnzOvq1LE7irnj6mxO4rA/sFW9QySx7
s8/X93qKQ30kfELXvzg2N+CqhiwScQpflKeOTeNRXztLfez09iabtzTts/x9WXOIAJ1BGUmMMO5X
PHMUDu9OJ2JuadtteYHPgasAWELC1WC1LUrhXCurhXCxBYljWMLGtIsa0gWkWkWkWlWlRlWkRkWl
71pu9acdpcob2lylnFcpbxTZwd6xLGsaxIvARqeAXKJO5aaRaWTitLJ2lpH9pYndo+uBsmSYs8z/
AKo9TTBhjk6IJCixyMxARtHgnVMgJALfJcpn7aFRLcdNVg+UB4j1LDdozVA1g+oifgka5VMdpNQ6
2tYX9k+S0Uvs3LQTH6hWB/JnNeNmxQMOieH2wkalTdC93tsdydYHV6cDtozStuxFD19jwKseCsVh
PEKw7QTvFRvwuzEAqRmB2a6EjljWNY1jWk71pO9aUcVpRxWlHFY1jWNY1iRlC04XKO5co7lyg8Fy
h3Bady0z1ppOK0r+K0j+Kxu7RWJ3Eq54+oY/NfNIbn1LWucbALkwb15AFoac7J1yN99otxUrMEjm
8PUt64zy9f1NEflHDiES4XZfVfNa6MAbC4k9JVHSgif6mA9HNMLsPqeUTW66083tCtLL7RyxvP1j
6trrOGeVuF2a+a3eFb7QVh2l0O0uhxKuzvV2dkrE3sK/csZ7ljcsb+KxHir+i0FxsENgzPYCnQkb
FgdwWB3BaR/FY3cVidxWIq5V84TX39CR+71ehkw4resZszSGzPVRfJ05c3aib6ymkA623XK5OyE9
xe4uO31O/PN1/UwvwSB25F1KXE2csVN7N3mtJANkPxWmj/d2p85czDgAHolvRv6EPXzFbz6mxz+9
e9auK1LUrhXHBX7liWMkZoblgRUseIKx9XY8FhdwWjfwWifwWhetA5cnPFCn71ydCnamsa0ah6Ba
rBdHiPVXTTqzE2COs+p0MvZXJ5eyoeUDURqUtOx+zUU9hY6x9UzqjNN1R6qCo0eoi7UWUrtYfhWC
mbtkupZsQDWtsPVb0M0+0fsvvR8U02cM8zbP8c91dXVyrlX9aGngmwOO1NFtWd7AVoO9aBcnWgCE
AWhbwWhahEFowtGOCwBYVhVlqRLRvWlZxRmbwWm7lp/srTuWnetLJxWN/aKxO4+tA1ZpT6rlEvFC
aZ2rEmYma5HqWq3NV7+qG7NPu/aGbBmn3etur+pjjc492d7A5OjcM1irHgsD+CwP4LRv4LROWict
CVoVoO9aBaBq0LVom8FoxwWiC0YWBWQKuOIWNnFaRnaWkZxWlYtMxadvBaf7K054LTOWmetK/isb
+KErxvUct9voSzCPVbWsZe659Q0YnAKVmA2xX4+saLnO/b6sEg3Cc9ztp9Xvzz7R+0MFmjNK3E31
lirHgsLuC0b+C0T1onrQlaArQd6EHehA1AAbPQsFhVgsIWpXFtyuziFjZxWkj4rSxrTM71phwWn7
lpzwQm17E2QOQznUpJdzT71f9gjmt0TnqPnSh6iJwbI0lVLbPxbj6yPMUdv7O3rDPNt/ZcJ4LRHc
o4cOs+g6IFaDvWhWgWgC0LVoWrRNWjbwWjbwWALCFZWVlqCu3iFjZxWkZxWNh3+k+bXZq0snFY3c
Vc8Vf1scgOo7c9STcNQ9TSDpk8AnG7nHv9MqB+JvhmqY7tvwzNPqGTWGFwu3gniK3RPuPq4xqzPP
R/Z8LuCjjtrzvZiRjcFhPBYXcFgdwWjctE5aJy0TloVoVoVoVoQtE1aJq0beC0Y4LAFhCtmt6l0z
RsWn+ytP3LTO7lpnrTScVpH9paR/FY3cVf02SIa888pvhHv9ToAGBz5MN0YOgXMdiA9R3pjsQvmq
m7HeqptUUrvUxOwP7s8rMLrZh+xNaShqzPFx6qxWErA5YCsBWjK0RWiK0S0S0S0QQiarelZWCtnt
muAsbOK0jOK0rVpW96044LTDsrTfZWmdwWmctK5aR3FY3cViPFXVyo5Tv9Am21Sy4tQ2eoLHD6pz
OY9nWFvUQP1lueTru8fUx/MfK9XctRjLYLd6fBhhxnbw9RTu2jNI3E0jMPUCZwjMY3+p3KF9224Z
qhmJt94zNPqrFYSsJWArAVgK0ZQiVvQc260QWiCwBYAsAWALAsIWFYVbNcLE3isbOK0jFpGLStWm
bwWmHBabuWnPBady0rlpX8VpHcVjdxQe7imScfRkeBsOtX/YYn2dbPUE6h6imw6TWhp9Mb3w/BMj
aJXvt0RsT4XyHE5zRfYCnxuY6xHpt1PGeoHTvx9SycYNG8Xag6mBuGvRqsVw6MFvqAcLxnnbhf4/
sbHYXjMQpWYXZgfUaONjA5+tWCwrUtSuFcLE1Y28VpGrStWlC0q0q0q0pWkctI5Y3ISOQcD6L33/
AGKF+7PK/CNXqYIsYLjsCJj0IOHUpdHfoeo3obM1T9X1Mb5XODcZU0gbhY3cdakayfC4SAalVPac
LQb29Mphu0ZqgbD+xnYoXYmDNUNu2+YepssLuB9AU8hF7akRY2zwPuLHbmmZiGZp9RA9zujuzOdZ
F5V1f197JjsQzyv3ep0bWMBwYk4sLfm7H1G5R9YZ593qYdMLuYsRdDct9yqI2Na1w1E7vUFM6ozT
jofse5QHoZni7T+yQGzrZnC4RFiR6mmkaLg2RqpQTbCmSOfA9zuB9B4c6n6W70GOwvBQRUzbOzD0
2PLDcLYnuxH1EDQXIBoxn1UOzOTcn1DbXF07EbaNwsnEtiOPX6mLrjPPt9SyR7OqVyt2C1tfFOcX
bT6gqLqNzTdQ/skB6RGdws4+paC42C5PEy2kk1oxQkXZL5+ovYgoHNUN3+rbqpD6EM78VnG4Kmic
15s3UjqzwPuLcM0zMTcw9K+aV3qQ4hYj6qMas0nV9S219exYYNz7KSYFmEeo3KLrjPP1v2MqHqDN
J1HeqDHnY0rQTdgpzHs6wt6hhs8ZrKYdP1NL86FLHHiLpHINpHbC4FTRGM93qKd3RtmlbduYep0/
yWjt6LaiUfWTnFxuc7HYXq90Qpm2dmBV1fNZ3BYXcFhPcnG5/YhmaNWaY9H9j3KLrZ5ut+x7lD1c
zth9VTfNFYaw8VKJR17+pbrAzVA1X9S04SCnFlQ0dKzghSuvrc2yqXtIDQb29REbSZ5BZx/YNas7
slFrhtGaB922zSsxD0MZWN3FYncVc/srduebd+yRdbPN1vVQw4/BVMUbGtw+o3Kn35ijtPqYtPY4
EZJb9cqG8kLsez1B2KA9DNILtPq7nj6m6YcTQc07dV/QsVhKsrd6s3tLo8V0e9XZwWJvZWP7IWPu
C0jljdxVzxz0o25ipI0WkejYrC79ki62eXrepZCXa1oI+2pYTH4eg2KOw1lTRCO1j6EPWzzdf1MU
Ree5SSgdBmxVAJib6jcoOtnk659THK+M9ErlZ7AUlRI/1NOduY7E4WJ9XY5rK3etXFauK6Pejh3Z
qbqnM4XCc3D6/WrFYSmwE7UAGiwzlFqwrAOCwrCsKw+pELijA/0ALrQE7wnNLTY+hDmKf1j6kPds
umQA6y5VLjYN3ZxtTGFh2hVJBIzlQ7c8vXPqWVGFtsK5V/lhPnGjuLXT3FxufUQ9fPL1/wBiwutf
NEemM8w6Xo3V1dYir+qAJNgo2YRne1GNYSsJ4LCVo3LRlaNaNaJaMLAFoxwWAcFhWFYUGoZrjisb
eKL2cVjYsbVpAtIFpFpFjPqGOwlGQlQk2N1J1jnYHX1INw696kcXO15yoRqzFHb6gI0xtcG6ZFJi
CqXCwHoXPow788nWP7JF1xnn6w/ZIm3dnkbiCLSPVWKwuWFy0ZWjK0a0aESYxrd3oEKysrLCsKsr
KyssKwqysnahdaUrSFaRyxFYir/sLYy7YhGGKR7vQa8tRmefRj6ozPPR9UJHjetK/j6qDPJ1z+xX
7s0XXCGaZtx+xWKbGSmNAGcotRjC0YWjC0YWBvBYBwWELCsKwrCsKwqysF0VdvFaRvFaRvFadi04
4LT9yM3ctKVpXLSO4rG7isR4q5QceKZJx9CR+FqLnHaf2dk+EbEajuTpMQ9U3Zml2fsQRzQ55Osf
2MC6jZhzlPj4LCVYrCVgKwOWjKwLAtGtGtGFowsAWELCsKwoD0tSuzisTeK0jVpGrShCRpzW9Bxa
3aUZCVc8f2GOTcVfNUbv7jZ1s8v7IBdMbYZ5GX9TYqxWErCVgKwLRrRrRoRBNaB6JCwqysrLCrLC
rKysrZtS0jVpQtKOC03ctM5aZy0r+Kxu4rEeKufUMksgQRnkdhbdEk/skbrjNILtP9xxtzyDV+wW
KsVhKEZTGW9EtWjWjC0awBYFhWFYVhVlZWVlZWWpXCxN4rSM4rSsWmatOOC0y0pWlK0jljcsR4rE
UyTj6JIG1PfiPd+xsfhOefqfssJs7MVK2zj6prcRsnsLDYpgu4BTRYLcP2PCU2ND0HRrAsBWBYFo
1o1o1gWBYAsAWELCrBWVlZDPdXVwrhYmrG1aQLStWlCDmn0nuwoyOWN3FYjxV/XteQmm4zyuu79l
jN25pOqf2ZjrjNM3V6ppsVKNLGHDMPlYbHaiLZrKysVhKwlYFgWjWBaNaNYFgWBYAsAVkPUalqWr
PZWzlYwtIFpe5aVaUrSFaRyxu4rEeKuVc+mE2T0HPwhOcSf2SEnPJ1z+yBQndnkFneqjZjNrrkzf
aIUre2niziPUQu3ZinixzD1EEobqKdT3PRKiZorkuVSG4rg7c1grKysrKyt6V1cLEFjC0gWlC0oW
lWlK0hWkKxlYirlXKumPQN/Re6/7JG64zzdb9lhPSzy9b1bontFyPVMNnZ5h6tt3Gykdoo8I9S02
IQNxmlb6vGRvWInfn0i0hWkKEiHonUEXlXKv+wtdYoZ5H3/ZYc8vW/ZY+tnn3eqjw4ukjo3M7lIY
LdEeqYbjNIOifV0zB1lJEXuviXJ2e0R2+ohduzEJwsfXf//EACsQAAIBAwMDAwUBAQEBAAAAAAAB
ERAhMUFRYSBxkTCBobHB0fDx4UBQYP/aAAgBAQABPyE6BUWUqmwlJoIJaiyKjshqtUVEMeDWiF0q
iqxUYqk+kKmtTWMiKskN0EWMTAxGZOjRl1DEKr6E1OiyhIdXlUbFS7JTClA8GKhYh7CzRq66I8mF
ORgLaiTUaxiZoYeRDFkQ8UaroVZRCUNDMOk1GgjE1FRqKI6Ghrr1rNchV4VZUeHQOjSojahDXRoY
jsh7VJiVNOsPrMdV0FkgY8oVCpeNMTAkyMaTQeRXGYoajAWtiMWBZMDAyQkOh0QxidxUZfRh0B4N
ZNCJoVSENizWB56E0fTHUqlVjR4MuoHRp0Gg4jUeKcBLC2rKiVMVXRPofShDGLNGPCtqXjTCtDBQ
3AzQqYjsNVFgwrYUaGpgYjyaVMVGNcQmrdxjGnGh4MqMw6R5ETQs9DF/xMqvBjVlR5rwGaqmdCya
GIkLNeUGJiY01ph0C6V0tUVJFRmVdRDwZaYamIkKNYbptDGSO6Ggy9FioxMDIWBZGBgPKNDWjq8D
mRZEx0eRZMKcaHgyo6n0DDrMw6X/AMB46DKjdxU4UY0qZDZiKTVVb6GFWtHj0BqKjq6JUXQyq8oQ
2LKmCl6NQhj3obGJhic0MTEuYEXEY0YmJgosswo0UWaPoISEqOmo8UY0PBl6CZdF0zTIQqKjH1qq
o8kRSasKjzUqMaUMWaLAw2HTuMMDEg1o6sVXI6Kro+maMRJNTFQ9g6LgQzfRjVCZowNI2NEY0pYw
GUKZMwMTTRZqRFCQlSRrCNR4owrWaMxJoai3okY8+hqj62LpeSJRFIS6lmjrVNasBUWKNBoYGgtx
gY01E+gw6a6m6z0sqR0DX0xUdFMZYKlsaEMY80ZFoLFXEwGhlRjRoKtVIXQSHkbsZDxUxAlUGvQJ
EGQvRng19U9qGhEjFmjoxUdXARqJCNUPBiPDoYGAzUS6DDoMVH6BegEMZqZqsxWurkmkjIvSVCr4
Dx6HE1H0CpkKjGojIeBB9FajPoKnATpmaCo3RggkdHXWiqqOpmRFNa1U0HnpWtEajY+g6mA6mhUx
6L630KrFXRU99NNXDyhDzQyROR1Jw5MdFoLFTCnSrgLIxUyEqyrnSDIi1SEmQjQWazcRj0mRIph1
WNUdWqLqIkZl1CqM0dOWaLBqPBhTWOlmouoVR0Q+nBNFViyaU0VWiaJplCTYmxNDpNH06yhBdA0M
KVg1DFR5o+gx5oqF6Fa0Y8iZj0ugnR26j6s9JDMBUZD3qm9Sqs16JVwNTAxQzNjodCyYVNVT6c0k
miyO9SYy7pEELoYfRI6KssoWKMhUOqsCHU811FTCirwMFHUS1HmjNRdXWKLBcJ3IIq6F1ST0sa5d
BULBgKjyLpCo1HinA1Dx0SHipoEG6Ji9GEQQQOjAzXqCaImhsbrJIqaOgLFDELI68Woxr1EM1FWk
R0GaEaRYoxoxFEj0DlFMBZ6HR9SFVVY0Ys1eRdAWXTIWKyEiDAeR9AN2GpqKxoKl7oV6IS6X1QRR
mZBA6EoMckQn0TpPSwUkZokWB5EIPFGRp0vUQzUVHmrGqixSsGNbMwpl0h04UQzXqd10aVT9FLNd
aOhmvTraipaajdhE0YVSG7CuFkY0oiOhUfVqJEVZVZsMBB0p3E6PrmuAxQsmRoIbspkPBgZGnRoN
akriGPNY6CsUYGFNy7oV0q2MBU0Neti6J6UjqYqGa0wpZrRjQccQhV0EMxMDEk1M0y6WXWYnPUhE
1ZdBNqEWPTMkbox016psGplRDoydCHgPBh0Cy61kVSIo6mBhQ8GRFGNFsTGKuFSHg1F6hVRAxUas
LNGPIjHqGgtCqjWidGBgPajyIZCj6U0wEupVY0ZdEhkoNjirI6KsEVyRpNTI0FRoZVGHioqJX6Kq
VWIwMRLUMyYheYUwr1FRLCoqMWBdGtCRgRFEKiqRBg1o+ha1HmmFFGIVExZGIeDEwMKaiNOiyEND
NRDIIEvRiEKkUZVuYugMSpjFVsvRVNAsmZpRqaV1Q6iGLNSVSq3RO5gJYS1LFQ+gaqROoiaZegKi
qqqjosEpNatVcKPNWBjQhCosmHQXW8iGKjMiaM16bnSKKs9A8VYCdxuehjYiB9eClqZGghjSuqGI
Yh1Ni6yDUeBhu1LFlTToUTQWBD6WYuh0YuhmvS6tIs1eRVGaiLTAPWqIRYhDowr1owFVlV9R9Cqx
9DWuYY6mqlIH1mQsiGgsUPFRGphQ3cQxdQXR0NRmbNKVW8UIwqyFiq6MzAn0Sos1XVWa5CGk6zzV
wFt0UKrDqEsIYQ8dAiZGPqOkkiG6MYqvK6RIdkMYlSPSSRJGLpCowoeRDxVkIkYVXmjEZCJEqFCo
kJjMh9LrRF/Q3WPQdEWCRRmoh4qyQjClgKKiq6GKjGjoPFCElCFR0WOhjRFEOh0Iv0nmph0S6opF
bNGRkMYUOohIILA7hKw6GPPQKupwjfocbL4uc7hzsbXlnOzmZGckdxoOFO5lMePQL89E2BtCRNl5
OENOjyOlk+se/GgH/k0SP8R/IP4g1/iOV4Iezq4ESxDlyQtz3GroQxShMeaYiuKFrAjUimohmA0Z
Ypcie4ndQhsy6IpHRFGhE0Y0agkEDzRjVFZE0VT69eqEsCDwYmS6AsDYmhUyMKlRqs9NiaL0FW1J
Eq4pBBLp70Vb7mNKQjgEv4D+USK/hQ3qPjLf2hv8BP8Agp/9pnF8s/W2bp5U82aeZHh+Q2/lHon3
R+8j9dH8ZScVrD2/v7i/140fkEGGeRtR9Y/GkRGQvZUKbxjc8YkL/QH8yE6bl8DmCFgCysPdS234
Q26fsP6Ynx8Y5reNF77Qn+/5LmH67j0vMND8x+6z+1QrNYn8GcPwL+TB/wA0/jBhQu4v9PERLn3h
Ho+4bQ5jgD2xK3LbkWIIEIFKXEGhCVxqxgPKNDJjCwa0YqZCoyKM1JiDYmIzFWsz6Sl6aMxoJBpW
lXWelynOc5znIXMi30c9bkWJSpNJW5JoSIhkUvSKp9E9E0waEKkEScBwH84/gFm/iMX0TN9E0w07
OzaEG5jWjEiTS+cfsZeX2Rt2fY/nIh/yiJ4/ruRol/W5/FZ+1xuU/Ky1l9yYZf1wOX5f8Ery5jeX
sM8k3AAb/DLd/mP5iT0KUel7hy6UlMajwaM8hG/mB6PjGiNJVkN63nH7bLuf3Y6Wu5n9og+1GN22
SQIgauIQNCIIGrnYdp2nYdtYXV+Dzi3OkBDlptigcYWwOMcA5kcQ5kR3RFnccx31F0Dj0EyW53k9
zkZFqOccg5xz6JL0XxSq+lw0i2/EWulM3wW3Jf2Yfkc0/sCOhrbKuyhwnbHO4FuvApc/BM/wFNH3
HErSaV+5/GH9UX+oMZHuajREtopqUiIpc8ENaHuRqJypSHjAqTJexpSSbk1Q77DjYja8Ev4j+Ofy
UfyaXv8Ake55KZJf9DJaeQ/ZQv4xyH8lbsfvoen8Q3+ONNQsLJt5xZ/MPcfdj0aBvE6uIghUirdD
pNFIrh2aNyfJzvJ+tnP5n9g/sH9cnCRax+kja8aHhHiG+jlP0DiDY8LNv5z9pkX5zmOHQttH7Tm+
B/OCKj/yI/wjWQ0OTbyPcR/nH9QQxPwpSWxKsO0do7ZeOQ7Y6XwE+5zHIznOSk5i+Jyh8UWxOCcA
2EOARegW0IbCO5ZqdxImjsJoSEobWwmhPc7nk5fI/un9U0PlP7JfTvI5AgpBW1OyT7C/oiSK2ZwP
J+kn6Sal6Aj7YsZh7GobwL+Rn7kz9Zmx5Rf54/nRPx5kSfmFZlUXOw9j2JFWazRHciPQdDF0MXSk
L0XIiJIXRexL3p7sTe5zErD3KUs8Fv8AA/Uj+Ehr8YX+AP5A/nD+dJPwzmvaTfiocn+43/nZK8PI
ctmLifeOx4iH9EfsIcVvELP24jUdC676RkpSvS80/pT9Fn6DP7bP74/tD+0IdaGfwx/GEAoAH/NI
fxUX/UkH5hDnyjDL3nO8izT4LY7B2BcXk50/cTf7Q5TvHOr+i8IWzOOcc4BDYdxHetwORDYhsTsJ
EBd4njyhK/IP6YX+yP7Qlh+9IWy90P4Y5vA4/B/kX9ITc+3JHqyIdFTAupelFNenTqj0FciqounS
mejGhqaVt0WRsRuRYWaTQ2+TkqJzjmChdzmJPbwhylvwCbPin8ONuX+wk/BJMguG2kTbKSjZ+cSv
SP44el8J+E0bfh082V0XAFvKdjyDY+cjwz3H9wbT/eNe8NX2Ah0F+Q0x5RIPzhrIn3jT+cPOX7jd
80h/LJat4ZyfDEaDl+RfoybUWm3666YrFEPpXTrRlvQv1ya1VcVVFRdO1YyTRMity9ZpNrdFyK3F
YhZ6ZIzREirrSCaQJDVY6pJJc5ZLck1FuE96G+Edk/ikqZ9g/wDBk34A3/gD1fBP4uLqVV6S6l6K
3TPp26GKt/Rjr1o+lK1NCBVVJpN1RUkRqXrelzSu9JrPVFJpqRVdKfSiRvqtSfTXVFYI6V0qq9N1
itqx0rpQ+m4jeiroTBBrS2gqKrEJlqp1bvSehKM+lE0XW6L/AJUdK6UKq/41I8nv026F1x0RXQVF
0a0Yi9V6G5wKkuCTNLNCJIzRiJ2dFxQ46LQpaJpes1WTWkGhMein/wAcEf8ASvQVNaY63MEVVYpj
plFmjSmtFVEdG4jWvFJpGaKnFMGvTc7kELnpd+xN87mQk9hTwvLE4nsTLY88VWaNM27rR7CSbT0I
JYsU3kmKNwRekumosegrE/8AEqKiH6C679GjofUuhOsehFF06EkiRHThVvSKSXrmt4FXUcrzM2SN
WyZgaahqU+Kqmptn5lvwOS0M6NeRUsK8HqxaTpkpVGBm7zkZtKcNNfU+RTTYgXAagRtVOq6F0x1L
okmirIhVVFVdS6I611SSLod6Kq6VXT0busvYTsSak0XTPXoOiNzKmJdxyVXhcveYEEm6FUnuJJaC
GSp1M1yPugsRIQqI5M03V8kLoJmBvQvfsNLlefpGooLDsXd4tLwPaTET2DSk46l2ounT/hnpXXrW
fTj0J6LUYkL0F1RVMYixgkRvRVgi7F0R1JyjBlsRrWSKTas8EDs4G73FnorBsS2w1iQZC6ezGk7v
+B1VoFrPDQjDrZUKjEqaeqvVz/xr1Ii6MkVj0lRekjJGvWum5HTNNaRVVv0KmouhUflz3fAYA++t
kkU3o4STPprkVglK5vAlvTKElWKL059N09ulPPrLomqpqMmiqjampIsdDos0kmmpFEadDG6ovXNE
Kkm5nq1ZgQtbUmtujNJFgZkVzBdvJLxTc4h5lgbZgkVNSMUvTYXQXVeq9VVlbkoTRJKpK3MdC9Jo
6e9Z6JohR0KjYrQMRIxVXVNF0TRsIZFNBEUXQi1FRdEUZ7ksXPRN0OmsCw+pYpiCXtTFPJgSqyTO
pfSrNKZ6Z6pJ9PWIsJJpZychyDkZyHIzmHPFuDk03LOYb/RBqJeHMjmR2zsnZFwC1oI9AkaURw2c
cWzL2BIqhrM3KfzDkC3Rw0J/yCZqLfeCTVSr+CmRltiTRUdyzTFJ+QuR4UOR9hGr+Uxdz3N5PJP+
49xM7oTa6E2hPYuRBwkxJ0Yk5wQ72ZIWSew0y8bntS8UvVYIdFySaU0MUUwXZBMuiFRksu1c0EMU
y+pmlZJ9O9ZpHRyHNUDkouQ5qfMc6O2c6Ox4Owdg40cSONVAOCS2nZP2mjH+iP8ARyvIt55Ibzkd
FMHCJNKBy+Ap8jkeC2f3qlnWHTRFn9Q/pE/7z9zLb+R7+RHbyR2JEhLYbvR0HAyF3oLc86JP+giL
+kf2DM+emIyf1qK/9GslCX6UftQ/5jgeBnP4Av4z9qP5R/KL5cjX6ieBN6PJGfyLJ/UOX7oiEKh5
IdEFMEv+D+SL/ML32C4VTDYUlRJv5TW+cT/wqWX+GJfwBf7p/VP2MSX+glOGifBPgnQ4jgOBnAcB
D2IexD2IexHVIcQW6Os05DjHIjtHaJNqmh4R7dDxVm5NDyTlnJOccs5JzjlHAzmEPkhkE9hsjs9S
xYhUgtSKy93ROd5P6Ql/nP6dA/vn9SrFzjln7kLl+xwPByB+hH6kcPiQfgOHxN/w0Zanyn67P7bJ
/wC7LvyHa8kf6p3L8n9k/vo/vI/VVE4B+tHN4n8Ae78C/mP1I/cmLZfNATKxH/TFF9w0/kMvuHv5
EGFPMHFEtjSud0Seklv5WQ/lZ/RH9cf2Qlgh/MFp+QS3Py0ZFc3whJ/yJWjwJZF/GcPifrMWv83R
8x0wR0xViRBFI6IMUggiiEXyMtSSFDepBwDiRxDhHD8HBOOPYUTji2pwzh1ke0IXGzvn6GPTk5xy
i/kPdHJOQc02HpSExxTgnELmFK5UdkW4GvYvaHYHxHZocAtjoqVrOl54t2co5RzjkHKpd1jmE9mS
2oQyCOBjcQcAjcQuBJiO6Erxcicqe9juyzuZyDleSN3yRvn6Gf1Kk1/Oqa/lOdeD9SP2I/g0/fp9
wTg+Tn8j9ZrHU6QXpHox0R0xI/RiskEURHTHQyCCCOiCCKpHdWI64MdDrDIERWKRVqmpCPakcCSG
4wZjGkL+1BdtdyzAW78AgubVEfxiaj/7bHtvLoB/0j+lGvyT+5P6E7SmbBbfnR+0j9ZH6CP5ytbO
ml/6Aj1PZn6jOb8j/wBtnN5EOoH/AJnp1G0iNgjYGl/sR/2IWwgP0M7Xk/YyW/kie68nY8nY80lt
/NJwnOOWXMxb45xziDQ5ngk1UIZA4VY7E0So6JXI6ImrXTBBHTAiGYHEiUItIqaUj0kMiugxV0rp
S1YIrozFOj9w6w5TPbLJ6D4wLqctfhlkcG1zTEiRIkSJEiQIESJAgRIkaESBAgQIECJEiR6Xf9WC
EOY+STREC2WXFEdjiI7F/A4RxD2BwHAcRwHEcVKPJDd+TufkXPyIe78kf6z9LFd90QcfMQyspJOS
NuZ/mfrCOx4R+xCd4Tv8BDSulK38Yn/Gz91k/wB5+uxt6p5J2PJL2jcSgTuGaLL+eP6KP7aJ9rwf
sI/bQtnxRl+BuFyFMCSk/aGPD7xO34Z2/DP1JkS/Mfrk/ez9jJf85+tiX+0j+5H9CP6E9/IXPyRu
PyFxeUdryPg8kw2AwOSc45QtwX8iNw4GQ9mQ9mQ9hJkqI0zydindiP3MItLjsJ+Z4+zG2CziqXOO
cc44mcbOBkbGRyIZBcuXL7lyWSy5JLJbk6hOk+mgKNBdR4P3JEgyr17NiLIasJaiRxEtiWxKjgJb
E5JkyVHAS2J02RQqeAkEDFpBBEkyGQRTA1QxEDEENDUkEEEUQRRBBikEEEECCEQIPXQkKCKILREC
OxDZENiGxDYhscConJaw4TiRwDiIYpaimBJrKthju1P0kmtfIh8kKZ7haJP7xOMyIN6A7kdusztC
TXPhkStZuuEiGg90IFFVOcoH9Y/okFknejIWL/I5ov5ULb+EQv8AAfyh78JDrewpMR7D/wAYV+SY
v7bpKvlZ+8Jv5yWvzItQWt5D+uP/AHun5hImiBp8oxePfvCFOWtPc+oQiKpISIpAlSCCCLEEUQEk
tCFCIIDS2IRRA4CFsQjiIbEdjiOIicBA4KCToqiJCJDcSkDJRA7yG5DenuqTa6HIy9yCidEnQ+lS
mtB0TCWxPYlsT2GxM7DgNrFkUzuLFcg3fFJnsJ8F4kG5DFyVY9yO1IuVT7l/Tm1f9D+EbCjGSwlF
gBDNX7m5AC2Ts0z3B/yyEdg6VhBDityGQQyC1IGaIvNIZIncTsOWhqwpGlmqDGPoZoKjbO4i83Ia
B8ScuGd9Z9PU5PbokyZOmZFMyZOiRIlRBKSRBFEDRFIkgggikSQR0opFIIpHQudiCCLkEGj7Ugis
DV56IuLFIEiJfvV/EXjjVEFvrN+zETT7KBGHO5a/Cac6DTrRDRzENIy9kZZlDeidAaMyU72vEbM2
llEguvila61hsy/IxDd0dFyBFiBHBGwghbEKMCSSGxCIENjiIDPEcRGwaOmIkocOSIiQQ51mgzGM
03a0WBCEtZWGHGo38w/iBsX2h/Hn8Cfyp/Gn8AfxB/BH8o/hVsG//Q4Pln6fyURy6C39pWkf3T+k
j9b8D9Bfg/e/A/YX4oT+Yj+KOYYFLZ+u5cw/fc/U/k/R/k/pT+Uz9pn6LP32f3/wP634H9R+Br1f
3fgiCz/E8SJm5AfrH3P6P5GX93k/cRH/AID/AJZ/Coz+zP7MdJHN+lNQ588nF/OI/gxx/gxvczlO
U5TgnBOCcUbTScEneiZlO0dii7AluLbkclt6KCIWgklRd5phEQRtZZEuEFx0hlyCVmQQJjSkOSKJ
Dnoh7GNDLNC4kQRS5rR56FVUfWf+pH/LBC2RDZHEP4x/GP4x/IG7PjP5Q/ij+OP4k/nRv/HP5cbP
wz+HP4s/kKI/nn8epD+yzg+WcTyz9THG8uvFN2sv602l+8/tjiNtDSmp7j/30ft/gf3V+D9v8KG/
QQ/8lH8tG0r2R/IH89XP4MeKVqoHpeEf+SxwIY/A/wC8f3mP/RZ+ls/U1cNVckfQosrQJHWZV+th
wgbRF+AifKQ5qZLZBZfTaEWaQoQk4/ujw7ZCIhGZWGw1m+ME3ssk3foX5mbkKOhF/dZpyMrplb+i
nCLUQ0EMKRtdxkm9iGH6Ekm5RdRjpionie5NXCUpiwWJ+ES1kzA/pRqVWWQhNNJp9TsJzgbSy/8A
4yBH0KIWsNBkP3YkGoFM600Qzw9vYyM7QC2gSnBSkuCeiRJ4KH+iQjTk7Pklyd7OiDOzADTkaEpx
0ombkbBLSh03O4kCvZV08JnbIkyxFrvsZDZt7imbCJ0SIZnYSNeVoNmltlPG00J7ii9uBxiOKFq7
mIbHzkyLBPfoWD3/AFJNF2cN5i0yhMZTrcmzLI3KRqp2TvMLQWXuIe74HD8mqmJ1HAA6lvyiKI1L
lD59yUYEb2miBxdl4wpZa3wSi8x+QXzKDtftAvsv+eivgHSSvhUEYRelNpRk2QEqQ4Gl3gWz/UTH
49Pa0G7RDTynsyRwM4MLi30XCqz+pYcTfr7iFsSRS29C9Grmx5Fvmm5NBwsgkTizm3BZxnD0fz/8
JDESLKqDK22BKT4YWqS/yGOFeVgiYw5vczDu7Nlb0JCBKXZ9EDUdS7kAlM9JEDTg0SvLUXfPkMln
2NLlSpdRR4lDJE0008OrkiU1DROKcKB8nIhEihJWoNy6ceZa9pE4MrOEjaae4EQZZdlkN/c36Lp/
Y7ENLBNMWaSJesrCaNdoZ8AhQ/WR7Zun8o+AYu32hI2yfoIv1asXt1eRhUnYkuxPm7hOX1Z3i/gO
m/j0WK95DCfkseWX13i7aufpeBF0P2TQV2V+uRMac0OO5JOfhmW5qkr/AFgd1FiOi/JD3G5bt7Fh
uEp2ZMRupjjYirF0m4fMfUtRey7Epojm8bf/AAl6sYu30dcEdEHlI/hESJqUl7wIIRK+hZ0JsrDx
NisQMRIkkklp0N6Wtab0ZOtXuTFlm4kwUAdxvmrVuxA8UJCP63jXpVm7WexfIWVunuho8wbGY6ST
tmwkJLYaTwugc2SbTlcCUJdhl+RN8jnGe8iTM4YBXtaTMDlkmNI+BS80oRPncnuNsi+swKe3QxZm
ZazMlksLUT3IUFTUWg3nQUVhT9JGSfyTLi2g7ST0cLhEc31NnERTWn4CrlaQXuhZ0X15y5MSycTH
1LLraUPZC3nSPcriiVr8NTFzJjRZ14H4+whOJXyW2yRrnl2/+EVUWHh/T/4e4saEps//AIaxNEDb
Exf6gX+wP74/pT+9P78/qxKx5Z/dCKOlKL/fP7J+kj9VH8AcrwOd0BoE4ngfyPQ48+YV/U/Jx+b8
n99+T+MP12frM/uqb/e/g5L+ux2/12IU5W6P8yfyh/fR/SVFbv6u5+p+R/O/IX+Kvyfz1+ae4KJ/
0vRdEVyeixxn6kzg+Gfymfuulf0z+ufwB/Jn8x6HEDngDBn8mfwp/Hn8kfyh/KH8of1Uf2z+uf1T
+uf1j+wf1jiHEOAQ3RDdErdEk/8AkJkpCSTkSl+0vkliZJKiapZIhsTEkkiR2EiaJ5GySehS6LMk
TJE9yWSRMlvTM76JEqEjmoluSqSokTZMkTJTVkTJkq7YnS+Ko7ERjCJ2I4RO14J2LwTtXg4CY4HY
jsRbZHaMp2jsCeyPidp2kgmlMSuxA7n5LMN+WT/0Z/QZ/aZ/dH94f6Wf35F+XRiGhYL/AHB/dH9I
f1gv9iP/AEZ/a0vd6Wqwvf1CD89C/dRB+BH8JH6UOA/YfwJB+DRjr8qi0LGk0RNpP1dDO2sXzoNW
wZBU/wCkO4lOFLLoSmRq7RIXRophTkaz4UEl2trNjJEMxkJ1RKS7wJGcouOwUzVLg9C55QpM5w18
i0ShDx4Dkdd0T5GtdJiSLN7USLUkmiZNEiFhdJ9jDumRpgwIyKmyyiTUPgaTKijhK96Pt0YG4+lM
npkdZLQyR5JFkfQkoTlDVtxWotasjXaGlHelkUsqwjy1EmbpJZbGsiVIkC1ZJuRNM0npaQXomlia
JJJ0pPRPpXJpJkkkkmifSIKX0x30HM3ZL331RuPEmYU+DL7PYfF/UgAY3jD93miuPETlJA+faKS8
CFBBDZqY2To83eC3cjyz5ekPN90ktlqmruTeR7MV39Fh212nyxeAbwEKLxYy25D2Gu3X6lRWgy2i
V2x2Edifsog9VpIqekaXcWa9JbW0IPfbxsLXjkEKS6ZIRSnsQPHhfYbbyz73EaEiQjCy7IWZqSfl
eCGZkrSWJaeGiCY19WK0rmH3IiHhDGjUqnEI/XbjuVqFTEifNx22xkeSE7MmknfUtLe2fYneJqAj
dkmmXGounBGHoh96oSW4kJr4YtCDUldBsEWdIFEq5v8AqSIFHeEexoF+pipbmt+QlK9jZUilO6dN
m5HP6GxkS72+tEWw5PsJF7pJcklzf7STC9hrHBfAs0bmWL2yPMtX+C9zVni40m2YvKRIsQj8Hlry
OiGT0z0MVMda9aXRVRNS7XcoxY7HyQSib40HvMtyXqE+4c4FCt7jndaoS4J42Gzbbd2JvKU1DQ4V
jlQIle1u9JIaDWoWKwZe5Z67vgU89x59xqbNyexq9RoVq3D2mj5kOTlcm5ARgo7B7NZ8jZBbZPhU
f3Lt3TSKuUHgtRBIlLgIaUKEN6b1fU1JHVx2hb7oY2tphvRCdMSSexaOYjdYaTTLd+yuGmUV0k27
iXiMl8xsOTPf7kONmRqvBI++x7n+KRuwhlzI4153GK54u9qQctHt9qHFhKElBXQj9fuN+nqM94P+
CTxziri4neqX5QyRkk+xLxNzRLt1X7kt3ZNLtwRoE2me9ReC53GQiRZDMwu/PsNtu7vNzFQbE1Nh
SAtapZLIUi1Ux9DMh8BtL+oZwoe8TE7KZYCGiYWr2HuhEN5S43p803J9iWyUHPtBKKPtlhXUt72a
Lh/02FkrSeYlLiMuBbC7eCSaPSkj6Z6ZNPTfSuldKdM+hdDpJJJNZrPRNJpIp2aK3s3RE3gkYCYY
9CaTSaS1dUY7xxgY0OVkbbbbdyXEaUTJqBvhRu0ndmoWmYJJ3+SJaxwNy23q2370Ta1J5J4I+mxs
SlZGDka2DzRwia4RvlbiB+wWRN5EpbBDWTML2UDg/wC4QNX9gCZDIe6LhKE5avvRUThlktsMj20w
oQnUzBPsMkRkTTLRk8CNp+gNkabDVhjYruwlSODN9dTpFZoutdc0XS6bkdGg3WOpUyuQdzOmENhi
MCWpdcUaf/ExkuFJJPVPo6TcdZqxklljaBIYk0Lc3JeS0p+KJtDWUN5e+Ruk3equt0fqzSfUfpti
fq5CJY5MEpkWp7oRbor5MtmG1MZHNx8jwHKl3LF7Mh8zwiewabaLG/uW8P4EtXXGsLjnY5m1psXg
3BWSbGfcc3QVsrhh2RJDw3lbDxMsCUQqNmUzL+RiYKB7kA4T59BZcFstLri11loJoRS89iwbP5Bm
+J9mXJd2PYdp9y/ahgsvRepJS5gakK8ui5TFwdQ5PI4mo0DVA0kKXdE7kks66l7RSNlWaGupZZFZ
3FPbZxFjPE+zkZXJJaUhIb7sTE0wmp9y130YFWCDd3+H0P8AHOU6K60ZMykTjvBEzKUSaZLBGsbP
BFxpxOqI8UDa/wBKL/p3IJoyfRmsGpFI9CKl600BFm7S9wT9jUQ33PqI+NP0G4yDmz4M0YuSTLZC
vHnJMRAiEJcoWNMQ8NTMO4nyvBMgTXEhE0k0yjsXCJO43Sk+Uj5Mbyli41k7utvZKmtIrbJOWFNn
wiAuIuskSz+rBXxfJrRHb2+sdz5WPJFLW4/TbxYXt1WpiV2iRuzEpoxFoEjhRewtUy0nxJmXTCqs
Fyn6oiYTowBWbFJJWUCpnlQeTC/aSYKLxGZ5B7NBp2y9x+Xi0MJDXiPuqohTMENuEKS3fQ1SdzuJ
3Ze5KZcWQiXwDybN4+RolWya3TZyeEqb6/C0/Bii6H/zT6iXQ/Rh1dQ1CbstBs+FtoNWaNOsNX2n
ZtXEztt5RXEzJp3Gz+TU+49TdII8BDP3GNBNJvsJDVmjWjshtvLuRcMS2gnScQsWpFyaGrFvcHPF
IW3IsoSWKIqKI2ooktTaKPzppLLEcQvYgInZqHqmMkOUtpTw3BL2Xv4gSkQU7aPsLgnyErSdqjw+
5em3fPaGoxpJbyI07tccUEyWnQPDEF2y65ogmWztGo6XSKHBjETewdqFDSbpkyXpAyu/bJHGpLtX
A+li15lIyuLyezQazJK9/YWFV4jQ+p2egwScpWjkRJJuryUWni40WGIarlpxwx1K2bwF+7uPEw4V
5vI23f490dJ9kfv0pw22d4Y75nuy5PoUNOZTIh8bdjciYIu7uwfZ/uIag32IItXE7g0clkm4u5TI
OecxMfb0Hn/hRHpSTSaJk1mrpPQqqmLrkRbraTKowbhmNQ3SptMxp6DG7o+XrKPA/lM1J6Z656Wy
XrSX9BDd/cUlboXeBMk0VvQd/XToqP0l6MifS3TUyYpAlnR+W+ANt2vc7EQQ33Kl27AiV9Q5ShNp
yiastIp4lRJtwhIbTpEQxUnX7wJXljE4TEekk3I//Gj1HVP/ALJJJqvV19e7sYOlsLd7vAzcru0u
B7I3M8+G/oFrlGHHcwiTDg/ab0/d3jZJ3gZG+gnJVpXfBcmULvAmMukraJcHALfgaGcBKEN+hJPT
PW36a/49fUj17dD9FLo16WX9Jerqaq3Oe7Is4d5zBKJYtLDM5pVnwOHTILpciG4UxpJfRCe7yxWv
mNNaeVA02eVIh6i97g7kXhJvXI5m0HqIkyOVY8CO4g9Vs2vHVP8A5k1npfoR/wAU9E36Z6JJ/wCW
R0t0Y0lpynVs4l4wSJIBcMbGanlSSJJ8H9Ieyxvdk3bbAlNlrFG1HaGBtCdEl4E2yaynKf8A3T/6
76LT/wBT6l1ZvSeRMzVFr6apKULLL9PQgn1J6dCf/HXQ/wDggj1V6UEi/Ri6NWkbh6jWkTRSZhCW
X2GKbOpsTGzSSywxutPJPvAsuXelOwrCbS7kpeLG6iyJabRulDFQ2YnT3ZF2Nb3L0pOkoN3Wt8CC
p398BFrZaTB3QlMjFSyxNJHRMcaRkaw82dEl5X/NsjA9ftIaRyHcI04m4mgKWNNoedhS8CwE3S+z
7U1Lis09mPpW5RWqvtkbXZYdkaFOG8JIF26JghGGCTRG5MOHBUiKqKlBNCpBo68PrtrK17VmquiV
NSKiCBzKtoAkiQ1JoRaroQmatjhuRUVb5bn2FTvN6JLVixEHjb53oqFyXtZqxuRwlOi0EjMxhvHW
6E9Dsm03aY6Iql07wfmuhffT28uNZISWNybZkOiy1v6xPZIsImsUjoSbwZeqVfFI3olGxOKQ4mK6
9X9XsLGCL9mRmtPNBlWyXaJGc9vzJjyzXKwxORwQ2FGL2CjAb2iX3lwyQJmuXoiNY5vcMyTDipg9
8GXJtPgIaOhXaLEGk1IsY3GKndpHKOTPRXMUZwQuzaxPcpw2IimJ3iB3LPdpEa7KEjS8Pc7sZHUX
Yncam4SS2WyG/I3IqWp0RJKulaDhlMi1vDQ/dCZTqbSeqTaSBdV/yRq2jcvQWgJifdJBIKiROkzY
aVnLqRKOUh5TJ+1fkqV4xTSjU+1pJTVMrNSmQTU1zCL8TS90y0c1u4sTkZK9vrCIz+sN9jAlJKQn
tZvXzcQGAdEXKz2Qhk6XI9kC91wGG+C9HN/rJ9VCdhDElkcC35IyQUPG+WI3LH17UPVDo50GTJzv
XUpaHEVFgN8ZQ1PcuhJWvDgeJQorV0DOiNM9klWifGBZSI3M1Z6DMlbYWBHGGrWQ2ztrpQOUDyPa
GxLrWfshIzFq+YhMmi2pyrHWexAimORLV9zMfSJ5kTabDWo8xFlnhZOSNagVE/cS7Nrp2lSOFWFe
23A33LewG0+HkuPyYkV9a02hp2tJbWU1cXY1Ilt7QIRYVHiBDLukVzmUL3tncX6tUn+8slo2J3+p
Oq/KS1lyiYvm+up+1/kcD6S+41mdP1GjtLvYJI8ab3aMc2KcpFsW3nZse4wB8TS3N/acCY6RHOyb
Z38EqbGZQ1Npm46tIdWejsXois2zd4gSvnEtBsNIq1yKu0hJDV30MNBJI7D2JNHHYuiXMidjJNia
eg3GpNdqe7E2Now+5uMR8BFC7zDiiJRqEIZaWfZiHLyuL/QNanQXeUPZ/a5Ktm87B9dDpuCVhlJH
HF0k3dlkSoNZPaBIhNx12NbsO5yJontKXgZGWBFPYQ3zxypyl+SmVr7JXopuib84F8wpOQ5RnwkA
SSNMkXEslc0k8Nkw6z1k7Q0a7IH9BydsZCnzVstBkCNPZlof4UkKSl5ee5m8BmCRZosmieUIT2G5
cnGARDsza5nO7gt7FH3CxM+d9MR+b8DVcETMUMiic8laxu93PkuMNpfSEFnJcer4oSERO1uIJFe9
6wi7b5ZD9PB+t3Eey7bhwK0zTXsYLeyLxmy9ll9j7SItIzbdplDcI8vxkyRMpiC03vuJXsXmdRI6
8hs0iGIchPdW526LQVk22k0Giia54Np/uykzWay9lqyE2T6l96LOkW4TeiGpcQ2SYw1Li4z+RSfO
OZ6BgNlI33LkCqyz3uOK/MOC/IvSG3nlF0pEPD3GnwWd5srsWPz+QEOsTJTGxLUuPc2/NMLSpUqb
9h9nO2SQWiIU43lZJIroOLalsGef8Gjk6UY4DonU5CqGly/hQzSFQSSLbRDV2F4KN2WBFnZ3HoYg
YXR2PoJ3w/KpTGTEOx1aa4zFlu2yQ2RpogWKl2GY7oCprMnnyskVYawIGgimE5amRDbXToZ4LA29
qbPDk3LrvYIWZ2hyPLvhxWGgF9hK1CTbrxIspVmct+wWUhSqG6H6ZFzYTuuyRrJ3O5YS9ia20ib5
EaVjaOnGrJMmoyhTx3xPUVvRn4pNoJNXONqZQXYd6QCa0sUchkNak4TbsuJWbuuxoJtNNZVxgZ3b
ljcz2W0iU2hrUZpugPXhkpmKPDlol9lAp877HJLWG0904YrO8cWY5RlOUQMlC+IgPBsoJk03jziq
q7RXR2HZktO6GrNdt2zFgw6kNvoml2BFg60sn3o2t4B3oXZwoUbBldg7LRdiE1DGDOIm0hlk17OT
MEG9zTtB6wJI1G2+XsayJR9g6HblLxLdizKU6JQh/cSZa0fciAHEGjRYm1haDPYwDXcZCbxOZVPc
RJwSdEoVWsjQayKtvst7tyFCWlSOQaXk3LjTaT4JSJ7Es7E55fLxOxLV0226sybOaFH5KW09vzEV
v9ouxNXb1tMr5Iu+W35Jk0/gZGtxjYru5KXLJ0RleRfYi0K6CbZI4awMbJbyyXJTZ5RZjKwak5uk
VtMSaujFhiUysMbchO4ZEs9ybt75omwwlshKKMZtKzTVM1miZYdNLnG3U+o0aWmr7DUNrVO5FZ2z
wRaa7DurdD4jOqaMrPYnu6ELW1C7BD3jT7YFtEt4SEVnG1pv/wAU+pZz23OOmf8Akmunqa1X/FHV
H/HqpAyLQ6QKd4bv7CgOn7EL+oS7Fdrbh2Foy6MB/YKYROlZI5fups7MbHWPBpacn7KRwlP4LYTb
MPs9RynA45+SevkzYeQjuUxkanFYtFm1k9CbWkHGhMqQr1YHjnLl4mxRQVjAx/RCRvlWyRxIwjXp
PVPrT0Kt/wDy4/8AIdEixOaLMvJzBcYnai7nBcX6diStEOSZhf5mUG/Zsu+qRULKalGuuR6dCTXD
QU1sYHEPCIPrF9xaFG3JkRhvTrIPeb3XlCNF6IwP1bRKOTS+s9h8M2asPv0PTXa7qY3pe8WCPvJH
Gsf+Qxf9q9V/8yJ6proLFWHLcsTaaacNNNew1WZhdvwND7gkE+yDXxC7kJvorDRfE77mNyvPJcOj
dPZseGPQlsYvDJQyTkzvPgi1gO2QU47j4BcJXNv6XNBt3dX2Oi0q/Z8i9y4fdRd0wm16PVCUHkRS
eOl+oqT/ANjov+Vf8S9CfQgjriOqCxPoTSf/AHX/AMS9GP8AmVLerAqxVKAy0XkhWtEFlISY+uML
dx5GPWfT1Z6ltS2he49RS6TgxLb+Z9BjQ1Drg09qNgw/d0rrVEas6x7UwDWi2pGAYld7jEWAtxyx
aUq7KPrnKXjRb2LGIRdYc36bKzlDfCkTtdYE7RI/RVbx0IW1SELC9F+yeh5eamHo7hy3duxKMwI8
i6tee1Flk6JoKeaKYZrVsKEWnzRDrZJsc7yJw+4hGBr3IS7YnBtJMlNI1pImq5vwoTy2OmxKFIlT
XMbCeoY+KYaVhp7EQKYctshKKZa5T2ZFoIksJGUZk1OkQLdNn7TSerSiRTWG3cIKiJZ1oThEky1L
CV4YslcB9mNQ5YWrZDLVyO4kOEl3Y8Xz7wZrUue60E7CtOyLikAGm1fsI20mW4XdikEFYsGFMjy7
P0UJapv2EN5GaWQ79xuNhzVvgRVc7DcaiPuZMUVGVDRirPyIpSr4SslPRUEENqTSngO53RLidh6T
bfyYjahHFo7atmfJfxvleyGJTaBFqZkWFcl3ZaVeAwnA1uxeYS4PC29nMWZoL63ExkT6FwRBDWk7
CkuvOrwFSfY+6PgszN3/ABRHR4LLyLvftoh2oKodjcTylwuq/c4KFvYF43l3Jmc2JE2Bmi8g2RPu
zTbkkT2edi+S0/D2JEu3YENhL6ZznbgjQ5OHVwsxkepvYgoWZU7+6r+63ETvZp+IWT7CyEwdeb+2
wzTfDme/NZWSkrtKYGMqaZScnNxbVDyBk9C9rzRsNnfgdh7chDqap57kKmzNnsu4oE7GsK5NULHD
bc3uRNneNox8CEFJ7ntNGGxlMMpmMsdjN9rIplCWZA0+o52ZLbuK2dvPXUPrHC9Od5jt9ydhyQGh
eAbgey+clgbhDwyRMXtSi9HAqQGKvtGXHcYkzPhWW22FIPS3HPBtBdu2gS7jTiKcMrO3clXn9S2U
xRlGFutEG3ax+lkTNAibiL5GCMYE5ayRo+Y+0loXxF2qZ9h+KgveQpWSyTwrN03mo6a2J3ewzZJ2
tElMfBayydpQ9LjbkaJvSQkvQZuV4j4Mn/Lcife+1ESMaE7sOlIzSSSzcYsnkp5h2TtFhUQmb0Ui
tuZbs+CISWN3ILqHOWY9jAMnbEkJpr36N6/qNz4z6Dx+5cc1RBpgWXeZLToJ+EFyNc0l2yNgz8oi
vbTd+SJzdI/Rbn7XA1Kdzemp2/Azv/6xyXPZrv1aE3rvGI/abiS9kZmV+zI8YbcX8kvuW8Uvc3/Y
yJ20Ou0X+YfcZaEohbEcDc37jbbQqB30+ZEclZ2WE3R37DEbgr2DA5WJKfY7YbeBYDxK9xMJFWTD
glGvBHwWZhzgOWemRp+SjAWcP1S9R/0O4vgfQSDDznGrFBHSYLURjuT1GDjmW4mlPZGRzS11Tsno
4DfYMba0omnqPL4LzY7urC/CLUiw9Eu0XN+MmDJJZJt4vlD+JjPcHI2Go5M52sLNMRZiSbzoKKzH
700IvgPhjdmWhrQTiS3NZfmxGrvHaA1pJMPsE1jQJ5T47jzWyDSlYsch+UC9mUoGBzIyeuJux9fE
xyW7ELkonQu2nnMkmayILSisd3XkbGksMlmvxC/fEUeyL8iJczabJR4eSx9lCzIza8GnznARpuKS
zMRTh8Efgg/VMX5W0d1MSWzL6RLGtcVEiFcVk0Fr3FXyNVKDsjUleUi1ZhFp4z4RZrDS3iGhF0Qj
VLAs23KpbkyKVaS+Zt6EiTUc+YIS6YL60n8sE05kUDnUlCy0REMjYsWS4iBFf5mDQZu82skbCNOj
SitsEfgsxE8e0CbROEGnyi64pRhZuhzibcjShQ9COjwnuQxSD1DYiEkrRghzFRCoaNxLjiFGX3L3
bI/AlXCat7CbTTUSnrdGCibsyy6IwSFLImmvYSqsQopC3Jd7GBqmVo3gsxoDuEvYe0LihNyJIKeo
Wxi9jILYybpNQ8I9GW5ez/oK1zZLZbFlYaM2F0YKfJSWXA4hXnXKA3+DOCaM9DGEsnfrv6BotnK/
gc8PluXaWjdas8miyK4SijzB9CBDt4G4y0zlIWuw20tyFGDSYaFyfWQqtZzOUhIQmrOzcXkI8O2X
R/8AFy+9hGGIuzkNJ7RDYc3t1LCu8b/ZNeCJdPiEr/cRqldRy8jvyGDL7iwpG9UKS7unDUsvRdM3
sSXBeot4GPUQP9bGxXNi3WHXZMj0JGXOA7M6OEFBa+Sv2YTRMOMVnLL7Vkvcdu8idINQPS2okZxX
PWNjmspl8CXHFxK89wxUgtffXI4XG4zBFTM+O2Qar5s1s20IGhtDeAmBmNamGsD3YuJ7cdiW517O
wiNQcWtoL7o6z15C2jDhobVMguesuLLEYIyH1NpyWdhJGspGDlqXc3uPCU/eiMiI/KNMEaNrJq7a
uwrIZEo3UckchhCIGwqZbcat2RgQpT0a1VJ4AkkLptYWFWLPORuxLq3hLROBJQeVy47GatKL4aeh
q7ajfbJKNlt113G2NsHq3LEZuZaoupRhaL2FZym091Ylsi0xaY3Zakj5tqnI3Pe1mShMTu7S5l9T
x0PqeOtenI/Rdc+lJPoyTSSOldEk006m5J65tWSf++fVQlikUwpMSZstBcq9RtFJHpMzhguZ4iZa
+w0yG4abitLip1wJ7Rt8XYTTJNNrRT7KliKM7YgrZP1KJsdLCGrPUhcwzRulomb/AEO2QMrekURO
Vs2EyEVxrDRn8pDHGSZQp3M1+ASEGzNWYQES++FyxylLl8PhiGm0QyEejXSvSnpnrmk0f/anSOnX
/mQxX9WYjQiuA6NgL8vKuzYP5YVvot9aSWp7fHHZ8k0gNt57LC1mmkJKPUqTGWMla7TyJ2X0wg1p
tXktmBTxaRcF+wGhaW1/M0NU7H2E38WGyEj1LaEkPrb7tCu+dxnLY3ajtXOkxAuCJN9U0E597vdW
IVuxrfH7+mhZmH/IyTV9gOanwBa6Zk8gSGusLCzEc1t750INITuWkUiZf7tf8C9d109Of+e3/NqL
pn0Vp7dLFpJaycsbU0uXyfw/GQTeqVNNCI2pVyoDb/3MJZHg0Td2iBoFkvuzM/lG7BBcY2Y3vmK0
lDGY7xMWC1YcRc9OoBOd/a4xktOnU5GnkhflJCLmNc4mlv7/AOATyFum4/bmg/iGi2BKXSPcLImT
Z69TAG12PVEioyJ3bjKr2cNkWBEkCTQ90NgzvY1ZeC2UD7RbJ1a0Gz3v0tEx86hNGjFH2eChveTG
1DoZ/wCqf+Getf8AJFF/xKj9BIV10izVMykZhLTe9EMUXgyuwkMbThCw2vCElo7BPmYqpOL1VCRo
ARxOHixnVeFsNJVLOvZxRB0rrxVNORcCVD2a5HKZkdclt8swxlUt8bLYS6nY+UUhPhy2aGM8koq5
e9I0NS3vPakoZ7InA4eHGhIfc/h6U/8Asx6L/wCR+joKub04ZYtvXUik9M9CG7Ek1kknpbSy/UbS
yxNlZ/8AAuiYEag013NP+Kd3Rg052Frzrse7pV2kk22ElLYnAASH66s6NlJA+0OV71Rc5hS2iNWu
RC4KJ6FhCicCXCW+2vY0QUPLQmPE2JLV0G58hPoXhWhwpGlXxVYrqqvPGNkicSJbdkXmqWjYeIfw
yKesiLOkwZsNIVJJs4mBZfqvsDsKp3EEvIraEl8DTapi20iBkMgrPs/RSdczQTkjfuFat3GrbClX
vJB3zEImiQaUy0m0aCUtJKW8JDUoV7CQJNuEic6caUoh2GoDTZqGPUwIG4RC3jWr7GQxps0xNZTU
MdtO0phJJu1qXu4ds2wJCBbpmhjpI23hD2L1eRwIacpE23hK7JSA2ah/JCRLaVKEE8zCMKljODVY
PzpE1XbostI4iptsJXkSbsT+hG40lytxfAlzNkhnAkiX8DTTIDRIfySiHyeUU35IJzJpxOZGatbZ
GRghBNlqYWFKlLu+iWrz98RkLBpnYRNzCRSxAauEV0Yg0pByd2IgjspH1mLJENdlKkw40xwqNLrU
xwxTsWdXvdi171Fh/QQSp5H9DeIctEBq01RHKaMvlqPgHmOYTXZjWQmFddxxsMWjqrFn5bDo4rcv
chKsv6oysGJu+g+2RZWauIKRuOYeq3HwuSwp1MCXVtBp1JicpOjgZCkrW0EQfdA1HFxM2fIw/bcx
7Hgb3RLVR4R6wLzISb0rcmaR4o6bwIEpaSbftwNLb7JyKIokpQMdN70g9GNiE3Fh9o0d7xMiYllp
ozsQLaNx5Cwlq+C05xaXDLIItPrEk/E3H8FxbT+yanA1OmhkxO1sbIVJRbwmG55XtYRnHJdvhWaW
FZlnoJc5FkpTsZ64g3WfJeCntdaPm8Oz4NhK3uhI0btRIWeI3/d1FwYvCHFqZaFxL86PPWw0WGW8
Jsg6rzepsKkli70RzsYRdiA8pwJ3VyH9JeJGYtds0lwRGtd2KKeJ3Et43ZOI7lsbegpoZaO8WRkS
rccCg1ovEkZKdRXbgl9MN+DZklGm1xceSFhbQm7xOg8C7Nk7CDISa7rdWJwRThXgep2kpxgev2fb
URJCoRD24niTDce7ILIChYWyL0sMhl8R9gHAomK59nIgtSwmKsh7P7kS3WTjLrIkq7IMbhK5bWzs
GIKjoNB5ZwJL9vo0wtHti8FK+gjH2+U+d9YRFq5J8BjrfGLUgv6vsMqBy+qNKX2GIryhotxLNC9K
QtFCRGtjjV9jGeDTHDcSb2Ww3Hzkfw4hkQ2ZRWeQNP8AJR0+CSXbfpXeXCGym4Tpy+SPah2yngFy
1uVgKijJWftSJsKQbrrdh7PeR/UaKBKiNJSMYCSNbj6q7pPBc430Ie5Ke4KtiZFd2RdPL5uTLiNG
JY8rFDm6RchMtC7i+gLeyna7SeSd8QYhkORQQ7lN8MW5qOTnPBGShO7XzNltksdINo9xrVuTamrd
y5Wl0+Mr4D0Lk81YN6mhSkVI39pFt3N2eDa2CsTbNk+aNvgV5UDZotvuCPs+zpF/BMXbl9wg3f4S
53FiIiLQNHa/QtUBBb4HAmhLr6ciU0lxKJq0rIJm27a9rnMWTdXCyfC4a1W3E0/aEv1CNMrP2bZe
zKPnJk7yv9s/fgkk0XtSIgiydmBmfKbdJIF8YsDBOq6NiwMvoZNEMdlMuTe6hOh4g1u32XSLJFvI
s0PwXPJb4klS6CQZNJW9iErkJTKmjOP3Qydtu6LJXILhx3KaNF+Zoo0exW8DtLs6xuVf2syJ9uME
00moS/ug0gTb18sMmFLtTxBYv74xb134hrc2GWYTnWCTSu5HAM4BJUcZB5lcONULRvbw5YguMzSt
NeKDfwoZYC/W7kz+Ex+D5JhiUHF3SI3BjPCSn6DMNhoReIHAONkh/Iuz/dSZFYi8abkPkKvYSQlv
f6o0p4pWoz12iC7w3AZedi64BWCfXu1ti0v0+CILrSYokx/mU6oF8pCWiN1Ay4Wgko4Yys36KOnq
z3akEyzTxJRLy7v3qr+57YSMCbS9hZFM+jsZrneTcYlepdE5E5ZocgtcrymYgAeYYkwXQm6KU4E0
K9mr6Fkb+4ZIKw9hk7rw28SHWQ0JmUmJPMlJ0DFJXObDQjkeeYfI6HVfgbuEOcXZBCiufYWI4F4+
HjPaReiK5slfvxz04xcF1W7RFyIRD5KR3EtuC8okclN3mbQTbGR3qfegSZzK1uuB3ZqvlJo7KSMc
zcUideJJh2z3LrrA/wAVnKa8IIplJwe5FgZih2LQwozuyXGOZx5E8RFr9wfYTZkSXyuZ49jVy5nM
iRp3gY7Vja1bcIrMg1vg+HuN7zh2VrRtzY2dSAYU8/gDCff7USotYWBdeRhMjJmywaiunfI6vrW+
QmnYJqi0rdYYrJDCktsS8e86DQBjnJcbG9AiDRktUjdBdt+4agZkOqZCiDHWG9KJjUbUrZjcszlt
yzQR0GFegm0MlPcZvZD4Ep0T21vuIzhfYtEEMv3vAzlxu3ImJPAQ8VthfFNeGtV4GqSpJvGBtW+s
Gx0c0nQ+5pA/tVm9iZi1jHgdtC6YXhDwLBwShQQYiZyh8stM5Yxq2m0FzeJoPE2U94H1l4a5P5An
vijsHoEYTZTHeZCI/UtoO/OGax/9ISXeDUvt07iduSwvrRZoUDcDeBhnKYfm9eZE9TWS5ee8hols
NS7rXvtYbctty222+WNy2mSnlvS8WBJoloLdNSWcGz1TtwN03aw2s81VG1PdHJlQbJttcs/EU1k9
iE06I2KihYLTJNJ6ZJ6FSfRnpkkn11kl8l4p1RzCDU4npcCWU7j6Emdk3aSZ6HKSbsnh9DdLcltT
/wALov8AxY/8K9J61WOqfSaz3IEh4RZ2hYRZyifIJSDUWXeIMybe5bi1Wc/hSK6Rd+b3jF2I6/VH
Gb566CBo2iPRrgnHuu/BoTbmrWyrSTgIM4c48SWPcNP2FBJxp6k1uJVz1HvW+5YEuecGzT2LhQhv
wPnhtY7jWcJWHsK43hvg8jG9gjlCW8MIpbE2K2JiHwIeqh6r1n/4kdM9C9FeivXX/Hie7+g8vsbU
KawidvQP225urZeBafNDqB/lTs49yE2vteRZ5wlcg3iQe2kJNRJP6AnlqzW3A5SIvLNnaMghaRid
WmcHvj9T6hO6mJJtvVNbhPKEWMIv3yVeEbFLQiDbWXn8Yy5EGbJyEwSzfKFCDwdh38yG+nT1Zqif
+OSeuf8Ajn/y7k+7Gr+aM+OLA89j9Bq1pK09GIPZ+7LAkyXbdyc21fyWdHvqRoV4+Bt9E7W0IdDG
NJGKSTRn05FlZu5W6F9gnyXEYGmhllotHdyZKRwTqWLzu5SMi5rsohVoZQSrRsMur6s9nJHEU0CD
TWp24+lKUYK8jBbvYtIc+jjh1uOZzPQP/qWRIlv2/wCWTWj6J9BVn/tX/B9Vjefel2fGNKTyO9Cb
ldOUcmJWMjBNtu0XuJhyalQ+aJtNI2mtUTRUXymOPAxsyz0u5LJhEqHDiUJlabPPJFGjkt4pN09t
SF90N0StJrZNx/2T/wA8dK9OP+vT14rbBuNfcSToplhEKj7w2hDPoQzzSOrc7NsK6Mwam/4/4pGN
IXt3r0FG22qWz0VKw1kxaRjxzfd5RW9Xpd7jUaRZEzfqx5GfaP6SDRquKRzzHWxPSqTVdcwJ+sqz
/wBa6GvuNW8l5ge5AnnCwiE8maDcEsFUrXF0LCwEjBvYbIDdjJljjYY1pNHYe32wmQeRE3wHwmqk
w2x2TfBBLV7zdDkthNMHkKK7C3Mkt7DerVSs3KNRS0uag1YY+SOjVGxCdb2JAc4/OJol1PZblpu4
2jXTCBjbfvBiAXb3gnwqdylTsW+a+C7FhsgmLzpC6Q8B0WWGhK9hz+krpLttoJKS6cmKUO2CPg8e
0VmNkynrqEoR4HEFtPJZCkMjb+oj4YYx3FCtgkRfCE0pjdD8TEaEyTd93gc92N4UUEg7X0kPm88J
uWLjWauEt8mibomjTbJZE7ydzybLNDdw4wEy6xe43dg7iNJYrqcwVDM8iR+nc+6kmG9yUTdGG6Vm
hp8N+LNiX4DufcslPxLLbwnIuOmelf8ALn/kXrt5H+aPU+RMFT5n7im2EH7sQXbW1g7I1F8BBy6G
HEIwKe2Ui4qyZH7RL7zGJRq6PcQx6Cewn2PgFyob6Rdr048m7yGzFx//AEOS5k9Q+yr+pwOHBAuE
PJE6J+Nogn5kkmWcD4QvIPaiW77mc/1PcSf1sUyeGfHZx3+6EJJWRvJ6GZ18F0XLNocmqt9SRyD1
6QxvrbSt9kOjtRbCcM2khJrguyUPoErufbkAZNBlDaxpFj7MsJY/N+RHc/aSxy/ZiL9LBDsg3XEt
wpk/3UXwF86aySPAVz5+4Y3O34QmyN3QXYhIuSRN7Bqz7Vc71V7n9dyeKJv/AMCf+C3/ABwZeSDc
+bFhDqqpYDNo9u0cHgCNlDcKsU8R1RaXc2iQ893sFm5i3T3gQXu9FsmDAVxLpsYwmLbzJLCWbKXi
MTDPhkotSAho7FrUktWIDNrh3F+DNe0b/wB2Fc6ZXsLkNMaawOGoEHp+rDH9vmKaT3gar4mZjcvI
ES1M8l3ebNiCdwn2kYxlO21uBX7U00ySlW+nIkXOBIh/gXbFLs4WF79Epupo1WzGZsp32KpLwfKR
WGZGN2j/AFgCgwynBgutthZzA3NJM/Ig/b8kmDnLTsewlztlWwYkWyn8kTQ44qNh1KeNJbkJ3XxW
mpB2eqeUXrtNoUGLkW4OlhUK+wK477VPkzwHTc7DwuS8y7GUxNs7QTmlLAI9ye2SELwSWqbX3ggf
pIkgocshxHpT/wCZPrPIif3km1bNZDlocMUaKBE8ihIdxJMJRw1dItEQhw1g44lgSRbMUZoyLPkI
eM3yJtn6gvSwrU0rZQ40PwfN7YZ0rGwOsk0l9bdiSehNokn/AJ1/wuVEo7qOicTZaO+gcpQ0mU7N
C6l9wSS2dGWm7SHtRTMKkmiTbZJJb9dmrO0RLbgHNXZMZdfGSlfIgRpGAnJCpqc+5DXsNP68jG1T
SeYov8InoW4iF+OdXNJgjHb79HyhgomOXwkIRvQ4cjsSsiOttVJbbeKrUUl+RDR71nCkuDKRxMdE
eg3l8ss0F2m749Yo3GHc9iQgY2+Sw4q0JW6XHQqB5adq6x0YrSnuRXSGjcSiabySmWh2jd4RIHaZ
PZItydyFfUan7mResys97kyQSGcrOdmJEKnZq7C94u3P3ECJxJVZrcc8E2CCmLRPckUd3uypxOYl
kBDN2cVWNXYTLHQb4zQs0+eAeltlDgu4ddKF6DLWAnA/z3u6iayPCgraESEcxKdvYXJKeGgbFdll
IXXW54e4Wg7e8sJw6qQpOQ4m58GkV1phckcXW3dWrFsNs3ROUG2S3ZrEELryaSvApbauseTkk9By
QF/mxmvB7utnCyQu12ZpTt3ES7LJjZUTjuP0KD9xKNG7sYrUf6J7wE433OpjwW0pGZ8CME1s43EM
+SXUzdlbjf3WBZzNlbaoi1m0pmld20ScdydWLwGIfE25soGiDpW49hSrQlyn1GwcbZwnqSlSusxw
SLE6jXvmB6hwjJaB38asLilNHaDkO6ofgWxGesauRvYaiokjHv8AUYauL5pipmoKJ0uYfJDaRSQ/
Yb086CiLrt7N2LucRqQImsunom6E5sa6l5PGWaWnXzlFovbjLcRNTC6l7JZeBFaAkP3aF145NCbm
XmRjsI5mtrd2GXQfpxI1LIypFiqWkgc6ct3T2HO2Rry3sQAbOXHBpQJfyBvvR17gGVB0GHzn0IKI
qz0xJdZ38x3IbtPF7joJ5g0sZ9i177DS5Nys4Fq2bfLTZs54rqcuwy9as87A8AiE1wLn/wDZRMMb
pbqrLAMnwNAUL684jcXIpSWk3uSoPMbFX+9CI3eDV29yVEZ7SWRru28pjk5lyk2LDjEoLeLzc0FY
Fs5El7ITuKGsQbKJwiLAJp2Tz3UYObA/knu6oghnt+sHPhdQcRQmjaTBNdEIhTKSdbwXeeCZlwc6
lcEhQ4OC5f6uJy4JakXJH9vFZF2q2sjDWUX45TplMIaw3yS7CqHyDSLPI+kFnTsDfUu+ioBGmnL1
TrIzqinfOTVbbnLyxxSq4JglamqlW0RYsk5/BKz+zF4oLLS4+ySIWRGS3r8cGpZ+YGuDwvZSKwoO
S0lpYgpyd0Tn2MtJVXdoZyXRojHruVJd3YiNvBUxGlLuoJ0sZNLeiunmSQv9EVXA1lI+4zJYtPL1
Gj9a4SF+0l7upbMI01834iFSVo278D/zUNJVNJFnAsHmPDY23bm5MuRRpOR06XTYyELzaMzpQ83X
Mclu65HnC5r2uMjqDhu9E+xVETE2nqmn2aaFMdULeLjVLd7ZH80w3a0nsPyt0iW2uD209xMFhv17
iWL/ALJt2+5obnzv0LFMEcd5uR5HNA97CSWDAs/rUQmz3a7fJxyG91BHmLjwSSJVcGJr2Y9OXo74
z/DxH6rYiq05NnOEKpZRm7PcNZlTuwbI9DHkfbXI7seXtD4HbtDdg/pCwN/uNRMXe5ePYlOb/kbI
1gnMn7PAr/boTfqEMxKqeNX2d8lu4+qJRrSoa29vkJCSbShComD3ahCOFhtSd2BSLkBBxH+SztPo
GzDsKwrofAeG7puQMlzyvscVO3UDW0RHuRVn60rMOC8itSh7aMbbwveWQj4DOAoZk2u6NtbZfMlU
WI3IyH6sQe9SJvLGIklo3GjNvzGZUrWJCbpzQ8u9FrY5Jzhascl3j+fBtKl7N0SWQ2xEkacmNZGb
yeS0LsCUtCUqhJFw2X72fw6Eu94VhzSrjfckkTGN80seoDq7OckxjH6BszCibnsHsa6fARq3e92G
0psM0MaWhqV/zTZw01H3jQcVs3Y5WOYhIiq9Q21wPg77ElN2ReKxTG4k2qivwjM1EzKFjfRu+B5l
XDLGAks4HTidxW3Hc3cSb31IpJhXNGxtfATaFi4l1yVk9+B+uN1/nG1GB2lNiEzjPBEJGHpwKqas
In67tR3CBqQxhGMXMZkDRPrKcVWBAhE34f5sliSt51GhI89seuv55LlcRSUqWFx2afqKUAwpI8QR
uI5VhGN02JLprKYinE2NmMbbb1bJpPe78eGODnNoJS7J8vexwPGTQWh+BU+UMBO4UnpK0GaDZ2jf
sJL4IX3kJ31+yL2hcQ7nBpVy6FeNc7sZ7gy3cCc28mlBR5QTtc5z7jD7jDR2yKygn90bsStJiOcP
tF7hrWfcdNyA2Xuxckaeb2YwdtMJUYQ8Mt5uJLg2FVqbyRPnv3T0Y3yXCJkVQQk15g1/1C4KOkSn
EISTCtuGctj9tB3TRFr6Sicx3IBF3HebmAAhmE2ZCKbohkVpEo+wVythKUtcyOP3LmYaC7gxFad2
W0il7zK7hQIAuo9DYVkQxcZr+c/kXmvjvG5HwehGz4J5vd0y4lJqXGbUhNtmh3dXLZ3L2vdOZ5MD
HCfsIO7ITBDFla2m92MSX0vKU6o+hu52RD2pa9yqVwXh91qL0mX/AGCQVLOJsS905nWdyVVhWMKV
pT96JgYvocsIlh+TABusvCol5wOpbUjfurDLK1YUy6JUrd30HJ2iXyJiXewcMbyz3Z9RLh3DKm3i
r3CUT0JI5SFmK2DBYttSSSSXSaz6M9DYmT0T6Kpo9hadi9H8ww9MuskjdZ6Jq3fpmskkk1npn1dS
ZJ6WqTSeuOiSfRn/AMqeldci/HQ15RhotHqspXYyNGDgs/pv1JIH2lg71n1bOnSf+SSfQmsi9Kf+
Zf8Ajqs9KvngSMM2HjvHwiX0sk5ZDVPkGF/R1FKWNcMSPKZF+EzCX/NPXPov/gRP/Yv/AEkvtTWk
WfdTTAdW2Pdl3PvBubocgvJLozIQ+2o1pYjn/jn/AM+fRmq9Of8Aln/oX2XR+DCS18zakmzNwkxs
zjNGwnwOa7DJTA/+5b39CfXjHVp0w9ZDlf8ARA01b/g1oqR6DCGN7JSW0frK9UpsjiJKLUB1/wA6
NMg0KqSdCJyLTJSl5v7DpzLDT9KWGrg3bcQoeUdNulqSTSvZsITxbYVUlPsYep1fXMDfWlNG0hXH
6CyOFqJImRRE+Yohx0L0IH5GparBKXNkzTChZFLYlCGEmzQjG3HlxZH1HMYF97+uxAVL2jdVSZCT
etuOttBrCWxr4z0R0OhBO+VgnFlc8o6tCFo1N/cJp4ZZZcEp4ZKmJQ2kpbgtEzYsfmjsQTTwLUyT
zKzHOqC3REv6iPZaSClyTaSwvnAmolcSPS9JtS100Le8230eDIzQeAVs09DeMjrvfXRieUJWRXFN
VlIVshaE3DXxVVQm8wm4x3IQmVDJ3LfYGwOXQ7DMfyz2GJ74Ir8yJ+02fgQeuNEV1wPkg2WWS+of
yApTRoVkr809eRdVCcGh7eXPbsCRI9noWR4Jsk6iYFeMZ7SuF9mYs8O4lIIbv3Rkk2CS+5wjKK3c
cXI1JwnoiK6zu2nYKfoLSkYEvo60rSXbeGew1XCzOyI00sgTlyO1XUJNi49mUlval6GyCVOSU9Ua
egiUjmEX4FNSTvdhdxpZtq/FOBIEv4IO+Yml5NJFPbSzRJPzg4xL0uIJczOlFLm1444Qy+YCUGWb
t+q0I2YYCV2xi7Fh/CE02YrqfYrFU3YTSBpeeLnIwJNc3iLR9BaRHiF4GbWDGdfMsz9hsot0rRzG
bQPyN5/dcMNhJRk+6OXKaZGB4xY+5nle8596wpIOBRqws5LnWSVHkN5xNh5KkQkiDwDUWbjaiQeg
iho7lvlflj/T3NCUvrXyyd+xasyStyLWLmTkdXNdvtQ5k421JK7epDlupJvYyfiNbcp6Vy0ybUfd
PxItGOCC/MQ90lyfk01AkJApmvcVmwn1tv6t1I0yHsLLQmGGIasraEDB2QkI0tibSPJNWeAeQs2M
PUWhfg+uTzZQ1hFfpPJhFZv1qK84uRDXuMbDNJF0mXtbHwWHc1ImABECCt5hC2GF5ZiUhXMoqm0m
04cMvMu/2BxzTT9yH5ZTgUiSb64FJHR18yIbZN6WoB5y/YSwpS5J7/t9xz3sd8p1QkRqe5JJbdwZ
kzSRJbiQOcL2JEssr4QjKRiWnHBtLnsMX7T3NNc+4T4DmSFPaBFrSnfhtFs5eAXsS5sRWDN3YnWk
ey2Jxx900OSeDe4TR2WjLQ8u0LRVRgQbRQhsYIbxd+BpWuN0W2RvvBzBAOu5sSvhHVtPAyFpqpOw
5uW6e/NGsdSTNIlkJyxmRcnb/p+02LPbJmpkUpsruDLr+oaYkzvJlJouex05IQuWf1BIT/RSTu1G
g28/cLte2rgULsNoPbKwiIZcGEU3aUBVxl6VuhK0e0LEWJLbAtVShq3auTaSo4XAje4E9zBKeDNH
uraodNPBgQm6rOPFZMqM4tGRpQKM7Fv2/MUsTOfqIge31Ro/cT0z5RhAv/cvL4P0YvKfRgZqNZJe
C9PbQ61b/ZmZ5TUZ/R5+dy5d/wAmiFG7M8y+SOw4V9JBEob9DUN57me5kb9YU7M1bUWlY2dkluJQ
qITpSr3yXiWhmzJ7Ab/DwayzGluFtUhqZMv7wpTUabNNPMow5hJLWUMZW1KhFiM4SFf+RW4qHLP1
HeyXwYTZt5NQx79c17jhst7pR34R2EjHmCZhJpzAws7JkN+9oN0dcbd0lr1YxJHr9zlbk/G8RTl1
iZRA6bCdsCXgonpSOiE8VvImoDxIXumZViex62GM3K+F3Q4uIYfJdAXLCfI60mJCFsLsiR4Jhl9z
aVkVe4ihrilM8jd4z0GwVDAIstSKNQOS3bkMq2AtxLHZYbcumJTa73ITGSpLfgPo1MPcyotOyG/V
sfpcE1/UbC567syK5RvGhv4tHPYZYGwlhOBJMsg9wThDRo4gcmFvhDrjLLiXsIwCY01s0I0GWSXd
mDBE2m+zo8xE0XZYh6GEbNCNWp30Haq9gHEaS2ArNvQ2pn1SKn592tRm7QeW3V3Ynng51EM0Dv2E
IyaIb6jhyvKsSaCp7NWYB57iHl2JZaQt4FpREb8GpaR7IHJ4eQlN/O4mXfs20b2xi7ZWScEi1YSV
LLUZDg8lGAjSRXYZZOVzd7DOECZygcJMFMumRQt9xVKk4mUW+3ZNb7MsYxbZvLM4TfZEf4mRBu2t
ZQOqOydORmtaliP2vAabn3Z63J8b3D3DbyBMRhDBLukaX1Jvyc7k8ifKjw8vcb+54jg1h1KQvkIr
ZG6UxFN1mpRiMtnc/BAVDSeHJBGyXd5bL3FlhraBUrzveA/EF/kCojkTcNeZEZV33+9WaY+Jk2X8
IIyuhIpkae8OW5J5zCXlI1JWm8ZFjcXZ8MzMpBCItOgQ9hTQRBChFYJENpTL93nBfQW3OGGKmZZ6
tiSK33bsEJA4sENjTy6W6NzxfLWh2QkMaa1Qolcu1hWQ2NSi7MdDgujLFCshpMSik0kmkk+oyL01
pqLXuyPDnLLixj3b5J3C+8Yo53wFSyHsD3FHGYP4ge2JKdw+JsyD2r7/AImU/hE03io/4HyT7/7U
QbX1q6BiNw5K0NaoyAtu7uS0007q6ezFSor+T3dLtxySSp3HP0sM+WLvI0ldaWEoRIrEsTmbxLBd
h9bO+F2OWtrXutSXttRT7xRdSpiyh5HA2u2TVOWmnKa0GJ4qGnBzOUCHiZZmW6ZUC7ldg6ZikSpL
mByFG17UBuNJUSwxmuykuc5HLlbwTjgnUXjtYhLAkiF+stkjFOAJGEolBfP9M5VOScIIafqKaS4J
FWSRMmj3mdMk+lNZ9KazSSeqemaJ+lNF0LpYiDAzvjAvqVkVJ285nT4Jwm3JC5folvtc0XnofRYS
njo5OmG5hNxmLxSUPl/TpDspFORsvAmngfoz03VU1i1JpNJ/4p9SSayT1N9Nqp9E3/45ERYiBESi
6ky7g2lkhkly5NRbkcszZMk/83dTAzWtWs9hMqL/AM2PVtuG8EiLmr3eRNThXdBMTPPP2FD/ACWZ
bDagu5Wd3Qq2Uu47F9Dn+nQcm9ZtbJnViuGjQ9zeIsGxxVoaIaFc0+8aE8vdkSOybNOE+jPN5EZp
IjB7wm7ZLAzli19BWtedrGogGJ7dh0OdIZiVyIi9myhvdfiYL7j3LeFKHGGy5EEaKwejRLoL7coc
kpCJTW/oSN0nomsf8F/+6eifXl+pGSBKmZqvIxc/U3ai74vp2EkPC0vgcGG99yROzN4G7s7a0Cyc
00idCWjl3ii4zRQmwhuto75Hz9KvEmVwC5ps7ibtlOuCIx5y1FK9GjP60HXZKdQTJ7nKNu49QdqX
7sP/AJibiJd7Akj4XpW9xIoYMBtct72F5sM/1MEXXwh69hJNqXlDbt/Bm2YYlnq2YIJ+pqXumLkB
8hSyruhPjsRuFS84kZL4+NaD30V8D/pYsPVpE2tjSiyRHC/4p9Jx/wCc36Couqel6kG1Mj9duSKH
wbtjK8+Hs3t9jVPfUBL69c7RNdimv3GkbCRtifpuhCSxy5zi5EiWfvAlyqwbuIEkuMLUY96mcFtR
MEyQnw4LD4FuosjkcfZkfrBad98kWgi62jLQbzYVff3Ic5vNUuWqHRmvJElJJQDUweRhBfLjTAzI
oXsNw0UmKKyJx4uWulnvuI0OvtCIlEf5P1eB+s3EG/uUXuIS/ciOiO3hlxFjqHEuPwYMe370Fqua
Gp+wnKXqJdaQ4/5Ey3TPprevPoP15ojeSKaCwP025ERguj2D34d4PRg5Zz5MQYYZoKL5j6l2hb86
QkbM4QUUS3IL1hyv5He68u/uRmTBJF4hotOzosbbaC7SNbXwrsh0hPwfceCbTRozH/JTCVl2Hi5h
sInoiaSIUQvKcGtswGS2ckZuXdiBGxk7NOGh2bGWS2N5ebpxSBt5Nvu5omwssDhnAqv1oI/4odU1
En9OKZUbsVUiaKeZ0Ih9wXLHPAmtDok8qhjNBWFLELG6lYgwlU6WdiNIaad08i1noyzh8kvnwPs6
TWOuaT6s9Ek+hJIiTBmtcqQfRNJFR9WCAr4FLwm+yk2PMGmpEmyLiZ+aNNoGns1FH6cEDrZBOVNO
dOxOsEhqZhteCazGzKsPK5tSaO2XLGzEBqU0jog7Mz+0M2WGsXfwMMXOIuibcIan+CIPjOWMT6DB
rKLSa0dCaUxbphXwRRdEWRCNMu6VUlWvcZNi3aSbv3ECcI9xcb9vgH23m0R5NJdhp30i1BqSzQHi
2A9FHvMStXRZJq9R6O7GXcYmkQzs3PAdpnQyWpdRqpIra4vHVLJEYSxiMyRKrma+DQ7KELKbknXS
eML7SIk1/sDN8p43ZDNF3r/bmVUT0T6c/wDEurDyb1WUfvNxn2QV41EIMKJ4Y58nSc5kSRc+oopC
P85ZQO5woTbhFjc3CkidpHvflBRA3SbArAMuqLS1KaD7b1ynuZyfXrv7oWLodgyP1WsxKkViihX1
I6yMpFrFdPeJrVTSLWHNiWMv1zPsCIT5mHe7vQx9YKuR891nwKLI8OpC4VwsxC95JMLrGecztSLb
VCcf3A9FLFHeGIoRY5nE4b86E+um6aSnQhf8QnLI6b0wUhOmJO4haoVHgWeZN002T6FP125EitXL
LsGUnzMTqSKnmngiG2e8hVWIgmNFqIfqqcWzF4wCTT2pOJ7hLa+w2WtbkqGfPfVFi4wr5qgYKbxc
GNvbOMubbBa61Gj5LtuLhWwkFq5vYSOIpGifViTJW1wb0GRmM0QStNM8W0Y5HMLdhYz/AGY7S5PI
Z/qlsje8jtGLxA/T5k1WjAzuWEPY7cQpJYQ97JqKn5IoM74VoJqyR9TBk4KQY0zP5LW8qTwhYTI+
kljGUwNCSoJJaMrgkmsxgl+hN+tf8r1oqJWdy39a58f9T9NuXfvyIs7Y120EIpTo34F07OLyTisR
wJ+hqMOryidh9ZpuwThJM42R30j2NBufYAB56JDcutfKIojav45juvt9jQhyT+oylT9puQagsJxL
bL5VtUWaPi/qYB0V4iGJPkl7oWrrCZeuV6zfY17Obn/R6XJ3eRnW/vH73ZiwM/8AODI7Su5HIVF6
o7C1cG7z3E0PLFWJ3P8AJEb228jTspkbCmz2RsRN5qBq1HEjSh5tv6Lxd3y6ImPRd4cSF5uuMDIx
bWJDYOX2rb5Emyh/URITJODzVTyaTEeq4Fj/ACMeUiR+vQSeut+zEc02c1gvRjok0M8h8zDv2EMI
nFH9Fy2Id9fcv3saN+5UIatA25MSpYHhjqBxiur7XHfe0z7krDufF9BZbXF9yFRLSiJWjG8I0QSl
cfzleCg91Kqv9yRUiH3YFMcInkhLJX3LeSyfOfK7ECf7yMxPRpftYuvTyfN+LmqBTZNEFBYXoTWf
+iRUsKrsxrJBqYj9JuM672/ZiuyIyJoYkM4vlnEaba3uRT0s0PrqSniG6TI9W6hrL3ea5Phl/JGu
FlKtSV9yFQdj/o670n+TF7bPLXsiSvaGWWb+kiMuFyGjVda8IWKNH6VyG7CRlxVf69lsuVlkvkdO
Nys/uaV/fgVcOZYt3Et1J9jOTi4d+zILWzUXbIpMrYbiIc8SmhhnV8NyTa77xYvT6c1mEPMnCnad
qaa/YBnCLbdpPgc/w1BrVtlzcmyli5sHdJGRFtizMsPOOwimc7HkLSrv4R1RV8NNwyZ+a3izYpGA
WekssENWMD5nOfqLQje/mruzEprKZepoNpUG91IR6DZ7Shw0xQOMCU+CyxVC0meU9Pgy1QZLY2o0
RypoexyxBOH8GpA00ye6cVkkjYpCo70miZLJE0n/AKJ9Gemel5LuTdmhJg7n7bck72QQUlbRDc1f
jwhxHylaDuTGYsCYqGDjczm6wKTjcwOTMWplWTkaykyZXRgHQyVjhZELTUQGtcoaKjlCRo0aG1Fz
wWamcsM7ITiaZNOU1vQbVNmsYlLCG27Ij05q5uSTmW33JcynFZ4VJJpNEkskkb69B7VDU0LYWrLC
S0VIsRRKkHA2SaHSJLTZH6k+lI6OqpJP/Mn1z67y/ejIFn3P229Ij0duGIT7TaYirFpheEWn/MbB
MeThP0UHMuBNMlaoiVkehFE0E1JWGY6o/lQm7Y711vR8kKdVNsJuqSRqC3clPkVtSk7332tErCdc
PkP6wck2O7P8wX5VtF6+yEpyDbjlI3rS2bPYI8Dc4I4LX7G5qdXP6yRaTfiDPr8x3EKnjM9xZfSI
zE3smZ6t6dl9zB8G3PkQyAdpKNoN6lt+eh+hPqTWf+SehdU1kmsj61R/cOmBZRYjYpUkJMaHdHWy
b91LD/BiNJj1bJUa7DrTy+wplTu9PkYbK4c13c4wzdNJHNfWlBVeGEkSTjqWVsPal7Njq2GMIliN
wyNMkuS1DYrgta0x81b59wgxQmZNFeaJqg74RnNMbjDkRuDmwJKzHPEPtsz3JhiUoh1LSazlpN2k
SD5ckonwK8VjtOSb/TcSuDT9E3KiiE6cvjOTkOBwISb3hljK2c/9HbRf1ckJhpFuvfQehl4oJ17m
awIyukaeNDvHnhCihkuIMWlWBKQ8LXgfZ5ezYsmqFu7P1FqcfUEf4R7bCG+SlO3jJ2T7KOKfWOZo
m11tebEUOrg6Q6w//Nkb60+lM8I7lh9cJJwMQxK6P1G5879CGmV2vuGs3uI4aE1Zvb0sX7Tu2MNd
/aFLa4iyx5WM5mUi7vysGzEaIDTBpeQj2839BFmvAlYfxU664yzD2KSddlhP2kmebZU41EkNEbTA
/nqQyXim/wBiWdr8g1BMVzOWtme78ko3C9uBEtWftx8lP2UHSXNWu8ZIjb5NgvcoLbt9RMZVmD82
M279oLub8TZ3w/0k1BWBKyXDb3WgJwcXGkwXjwtI4JK8RbU8HQTLfDGWOT+XBuL81xyRL2/fUFh5
ufsOIqdu1DuWpn37mcirfdCpnncXvI3knMdiWQUpY0hjGTVCbpCIViPsLBI5OZmMF4R0ZwkCB0LV
yNRSmNLukRPN2EknoTHc91MnF5YIphSP/in/AKo9RQM+hJiaRTQfqtxvL+gnNug8D1IeXp9G+4TU
Vus76Lijl4YxRo01GD5P6EnxaYkPf94zYtg2tzXuhTdhpFDsRJanZCZm+pIgVkHicQyVeuZSYmZE
Ro5wx02pBRU0XdzHhq0h+BgNcmhLcFvIwTXWvaidOGybNEX/AD22O1NC1a5vfo4FOai/mIpzSiMI
bYb3LOdBEtpGHuOjfyZDRogKPBT5IKRo2hlQKJM7x3Ip38xmFO7U8KcHENZmXgkhfS3R5Ei6ze9z
GHMR/Zjer5YpyOZ5LbPLbEZd6Fhw6y+gjk19n3RFutSbu4qowtXU2Y9uJ02UwQnsNK7zGH7CW9rE
6GR6cuWaj9miQ5peKoDDuzf1q+rT/ha/5pGkoy9ND9HAeKzdd6OWDkoeoz2oUHOBnHSaTzRktT6r
K7huEL3gP/NS29yy60kEiZylhF75HF1zRo0YlNKIsibj3lLr5FtCucEvgdP5MDZ+UiByy02q0oIb
OxPBa5zckzTgElsa0Ue0sE4TFojynXfrlbkzjqnrn/tn/udHGnooms0VJriG1KqzPr6VrXmJS+pC
bYSkefUlFX0r1IySKwrqYFEpwk2IQRYNaYcOKU+RegmYRqMdmoyzJz04Lf8ANP8A5E1TLehPp7jT
mq0P2W5aOkm5ZqsXdnuVfp79iTaRN7Ect/KR2uN5Q2b4HJGWHXchOgmx2k5UjWVtlbD5IXUh8l4K
PCX8hGwmRwM3vY26C8Te98Jbn06kakkr8DpWy4+4KlOS/qQCWSMbg1JLbuHv1mWF7Gi9FNLywp+2
q/4UmciNqbpfQ2k+sGWRmTTcKzlYcNk47cpKBqf8reMFglC2Btp3olYMRpkQhh8DfuyS/RuZNDUE
PauhbUQp/wBxmzcL29eP/AknrT6oz2rAtBqdln5kRtC+5tFTbZoImVWr4wK12bHgX3DwtpL3dJ6S
rmu9sXi8I8UiLKBa0Yrv3VAcjRnq9i+uERQUEWpO6QLriOWIMi1d4M8BSbu4S8NT3iNdz3L/AL6s
FsC87j3GntK941IGlmi2T4FAkZk0bOmF6JxNujfEgJ/VR2aLHbfWPlvqXfrYF1xpDRbk3ax8Ch2L
w+1yQEzYXI1m2UMdKfSoPzeVf9ypFUR/0yL/AIp9Cayr+/1Hn2dOxtWXKPdFcDLuK12hdvy3Kacr
ySI+0iQMlJk+BGtIrP2rjFqv48qx4Q5bxs4Ic2a67yL+ZG24g0t7W2oW/krATRazyhaC/YFUajQu
LDnUn9zN5vJJW83BA1kuWAVvOXlD5xkbK5fbojd4U6uTRIbxfTk1Vkt3pr7Cf7Ev4exkVZO7AlrC
MOs0MSCheXOMbDelbncW2BnZF2+V3nxjf5Hl58tTHNZeYkbyOZA5kkJTsWmCk2P4PCtGbt/9qI9L
ROHfHQ6JzCU8CE22qUdK6boagbOEkpbGmmxQ04a2arBnI/AKbHryT1Lpf75Naa+wsKeTe05LAsQb
DgIknMZhsFLiDD7oboz3ln55bZOK3yVMZ/LUb8YTZZTRCflCewdWp5tSL4Ls1omuSiXfQTlcwcc3
zWCwklgttRADWh00uL9K3hDd1gzroST1tni8SZPfolCadZLAEJUqJFss6Ia3de6fRBHBcZBNyf0q
2QFLSzCoS1L5g6opZ+jgk2VNhHg5Sq1rx2pDMkiOidgto5I3O4jkbnSDpuNbyu9ZqasEsgbQkwph
CxDMwSxLU7tlxqNITTUeirLuXJ3Fya+BhLGQlSr28CLkyvw2uLLqeG7D1kSnLXTBfrlkpbw2J3nN
3EK79oZ7dfl2SYWpMsv3Y/YMbX1GcRucW7F/LaTB7CVud6LdnCYZH92KatDEYcyR9mSwxeDEdJF6
E+lNZpnTT7mvsTRZPnf+OIqLTKcCIzJ2tY6II4Lj6IpHJEakrfoUuyyIXVzdxd6i0KxRf7wI5X7t
ZxRBIfyN0OV06Hg3aeClGkilCOGnK9hIQmUKMpiwOCVrTuSxbqaj4GoGaEV22OcqS6F2C9BWQXoe
1kzZXhCXgnJIqijImvFCf+wKQLK2A4kjHrSvsxpXHp9+BPKtEEFM0e3dYdhnPLKtYp0TvZ8CewMn
LePFhCbeV3CEpV7zFV6kJcsU5XajOKhOa7E8F1vl3zky7l+9lEtx1NXrclbU+4yMw4Mxcm+RmbO5
9hYr3CElNJPTmBJNlC5VhID2fDH/AIzaYwBJ7Csize842hkOCA2XMmO/RB/NQZ7CgTj4MbIiWyq+
7JaF1uVtfghGEkBb75LJ4I9zU1JtyUuxIdqttcMjzbzQ5knr549lPdaeRg+7ypMhnHyO/wBFTXPk
SkMgsShJ7NC7yXNifoGYGvD/AOOSem+H+3Nfb7mlEqikHIjWhoQGjUMx7MwTCOAvhlPIiGjVCvuM
jUHKZhUohI8+E5jWRGyynlFv9rXdTuTZ1MSuoNZ5vUR+03RLdYvom7Jhn2vc1TKJh+xetfeUsiTt
fhbly7myXYLurw31rnRksYUkY5vN0TTZtbV55LQWBrYmpEWCYlMx0qhMZwJO8nfNgtb7tJ0rZzR7
B5Ya60WkjuRqMEkaG4gxu0F/5STg82dhW1OBGpTGZ41+acf9ZPyIgVU8yKaEuEXeGjfDZQYTaYdb
4/mh0fy/pICaHg7mjjkPoPFPqxfCDLYRJgDWcn7R6E4zcJdlBFhHeiP5Ik7fUIF7ITIx/ozMSPO5
yz8xA22/sLOAVj9i9JsPP6NkXfrSbiflXRLLe0KxKXL9x9XEs2KWN6CJrbqMwthfW+gl2tuXaBuU
Ul+2JlWYXbLBOl6uHvqXak7Rku8OumoTBkP5BcOKd9IGJHaqMm6yyysIreqQWK5FqtJzaDg0Av7M
i6STeZC28OwuRGM93mBiVLXDfgaFCEwXHY1/KE9djFpc+4DT7RmGP9LSfnkeZ1j9xYQ236rfqNfT
7lkdvuL8/USlsTSFO/G20J8P5aJeyLXZZxf2ki4B+O8y2Ir5lMYE/mylaSjYtAo/VKVFl91Aohi+
YfIXqNEtLd7Cvo2YumlbYxhtNVSV71md3gPSGp5SQb5S473hz5q4/gHyhZLNCrq79psZC3dD5C0l
1/U0mYhQtv3tavIaWBw4j2tnfd7h7C4sruWkT5hORxDxOhi2Q7ksfzII4Lz1GZnIowLwvbDJcfdL
lnDanOTlWGOR11lSW1UUasJ7obcyadxCwq6X3SLRIjEzs5NJgLSb5lCLe2p6mJ8yZuSIWDPRoklQ
obSn8Fy7jZAmXOY8pPUTDN2FuHTBJ9SCWN5X5LRHTjQmxpVKsgZlH3Q7aDqTLJWvIk6JOSjJyNam
JpoyPysrUR/C91rQRidRw1sxcFNpOfYltsZslt5bYxi6ipkY69AVY1MG1qvcbPxoZ76xdCEHBtLM
OYJo7j4g7nCKxM48k05Zp7qxb++Fd3GlJCC0zFyC0EWq113ggiKypz/xT0fvyLP7uKLe/wBaKhDd
5o+WNE11onBeHXDlJeYMidfu5GsN+joankFFTc0Z3ndwLgqrktxd+7cZoLO+3J41QuCtMtAkNGIw
gu3L3d8xuWnPs8JMH0fLkhmz1YzaByJNscyWp1weks1dcOQyuoF+ajdajBLiowt0yGOnPDGjJbIg
lKK+pmST8tlya6MSL44XdouTcWLMrTQ1feVZZBDvKQu9xmJQETXYS/gC3LMyxIpm8MhkkkymxsML
92x4GxI2aWE3iliEcUhNy0jGKTR0it6Z6YpFbboh0ysik4GycNl3UUxFjsvJciaseYycK1SGWDeF
1Zyk9iqFfUkk3GPvg/K/4W74OeSnDGmOgm3L8kKF8NbohnsQQQiJtvCRteWwvkkffFakwdVpFQ/A
uPunRsfpoCSKO2dPIIsm5dtoMeJEXJHGlfUyvPhePYfAk8sXuMCx4QaTMRl7l5idmmynA3Bcahm7
IiH6+vt9xKI7fcWn7qKHBkhMG6kmVLCH9Ax5UxwiIHDJSZQhtvl0eBxkLSiZkxprDVmSe44KG5Gj
4k+Bal4iXG2nQ0mRBC6Em8KsEdKTciVYIroX6m9hhEtiVx80QL7exhPkYm0Waeg6RjOiXuzSbxGk
+BxNu2ncvg2wrxssIbKGmIEIkEvpO3vNcGVMEdDxy8VCvGottqblxGol+9acLuXexNUQwu6mtG42
VKJEL9iIYEzeJ1JkLcw6v3DUDB51+g29vX6tiykedSo7bnDvJT7ob9D+/BeYjgYb4H7NqR4xEXxn
d3Ex1jFzb4IO14FvqK8yivqi6UmZN8RNu8YVcJSySWM4yjgPBiddryIFQ/BJZfuz4NjHr6FFhjU4
4NvGTOHxqUpi9EppeYuJykaeS6gXsISQftblIf6pdXOg2fLbaDXAtq0dpS4yXyEdkTpZUV5kycLt
IGkOFfAinmXwJkk1mfcZbKXkgQxbXYjNnqado9QenruXJ2XeWZkyKym4v5FmrB5PmyJ2npfIrKxp
nlJ7ieHcFzxZSqwkQkLZLlDpcb656En9A54Cn97mQh81S25BZFps6JPBn2UkXahytGQOsdDyXaE9
xVmutD3gaAYhJZbZAydZgffzGG7Qzsc7IYIdp35C2hOEttxC3FHBbGi8SjCbmo1WxLJOrEpzfwI8
Vqe7wPcisqrVUtXkfZTH23JwGjcraMnn4z3STxbcUxkylp1JrcUBgltognC8xTq5NDDO7MJpaVu3
MQ9rg7Y4wlbGyTTTVmnkW71hROCEgtyb3E3E/wCIRhT3BDK356db5E9WCcJLdkEuxICFLp3ahepk
uEBCBNCnA9ZliiJaPazDaU29rAv1W45PYTS6tngY7Hm8d+X7SiyJveglcNJpKFbjIoKtL7ysgycJ
7QIjX265KpJtuwry3obHq4LzMwnyEtLwBszYFENGbkI2oF9sFqMG5h50rFT9Go8sf2zmFN1eVf6d
TQR+eeQB3vJkQaPc/dO9f4PZwdbgV/TMDz+nYxFLBlnmUQnD7ifxYBqx7CzbvoCCaLYayBa9Mskd
xHaCkWt4VCLpNzuyYA8x/Iq59GCZiUmIfYc7C6OASDHhn9Ray6XyDTMXuHfR8oWGBs2amZkw8fUi
RplruiwO07uS+pMQ1o4FGctt2E9Fd3o5sasahpMl5BPyTFfOOKkWwFdyb9OnoptEY/dTjj7kyhYb
cCeYIAXxsOxetQcpiuIiJ4aMtRC05urG5sUYk02kpjaBw10c1id9J4cNWxnXeN9TEtYczOCZK4+8
R31qVGJI76TF2DdEsvqSL6W5oluLIn7x+USK0YTcjZ+TFsnQVqKuEHugfloxktxf6O7skof5XWGd
TG8c6TnNmKUV8kSJjMlaITqkH6C1hLhxctllTDew1W7j02CmZRBLyLinQ9s3UDEfuN0NejWdrE8C
+je46B9th8GLPL6kycsmWZDJv9uBmoMmu6Ii0l/ePGEQxGbeiSJWq2rXAxs4a9gT9t7/AHgs1Zmv
eCz6A6Hk8HvEfcjeUMN/qDhz7YFc/wCknG2/IQXRciWu222ND2qSXt5HgafLO7eiIicgxM6UvOrX
3FqOVe5Io6ttfa5c27r7YHZvgy1OvuITdC2jR67twaPylLTnlwjAjY+D1Qj4t0/bcsRt2+IvPBbg
PCggT3yjYRLn3gnV5JPgwuAVmidO6IZtWPeWN1Ivkh06lAOFu0grS9le5NyTlbYSMhXebaruIBzU
62eWxVuFNu3INGNCj32gzxP/ABRExOy7sHM2namJDHIckqzkt7jef5tt2IQLN3udKQn3t63ZNNOi
X9mYhfD/ACQCUuAhXfyUONhCarIMMarsRzd/3mNVMSjKe5DtMEcrnW2ZbYp2zCRQ8kVjna7L9XnY
IUUJyk8E2gRkEYjaKxxZu6GuOoIner6J6J6Hdr91LG3ASsv3UX4EOBzPlP6F3OGVzGv1w1ZEsrm7
70vKO8fvJ/oZJnsEhgFKeSfQ9x41p3dybHHmrWhvVkwdjxRxdJCCJc+Wdljs0btokPPURh/TsvT/
AAna9T+IH7FL8hRXlLySrBbJLE0X49kdft0LP3LC+ssi+kLnuC73OSdy71/Zboyzwdeg7Ok+ITOc
5eFDRFVky8BPWgpakraoUJHO4IcKsxALJylQ0O5ItKUCgAFxqTtLjuoFJkuuFw1d9n4tGlM5KBwR
P4xiJpmkYJuNjERLtpiVaEXXblR87oVSxrL+Y0J/v7HBcT6kzfKlc+rMo10TeZPhKnbLTcu8oaWR
qViEDStxhEiJkc0T1ORvteliSUXRpsbhe1vaR8clpbIxqtwhap8pbyNqElhKkiraMPADyzdX9jQ9
crXuIC4c6F8DcnFaICsTCfzNIfndlks24PlWZlzB/qp3J00ooWnRkv0Q9h9M0a2EnVw1pFiRrQk2
+DyxsRCNxCDgLBN+RjoXqptX7JaV2QV0LCbJyfnxvJ+hG7vyI0+Cs1K3Q/EJZpOUHLJtwVc6xoO3
OwLlREI1SByVmvkZt8AfQdOWYhyapxay3VNEZc3YK3TbvDjQbVuEcX/ubMQV1kk1xLDTYrLh2NO7
jQW9WW3kzCarA7biVSJqerc1XOzNnoyZbqQ0uQuCCWse8G/pKA/37S2NjsOC2agdOSCyE8+yxjlK
9j8mcxwmXekC+0z4jEwKc8aN7FuScbw/ezleCRJHTh70viFaWtmN46NktiluW223lu4oG9idI0iX
EzE6lnkaUkK1OBGR6JEWIW1IqbURqsWEnkcuqLkTr1KbSQ282L6kCkA7QQcNDJYxDXElqRSTSGta
e5AQorHKh0SNDevyy4Tw0MZ1mho6JLItyWiFeNRN7W/GBCF/xyx8De92NPgJ3vA31izl627mNSCS
yo7kSJojOUzQlMiee7YnNtFeeS50rLGJZHHAUk34LGjUF5CCmIHafqJsvthNbEI0KZ8/AujilstB
jrLzbZmzJbxqXI4ZPAnWuQ1sFn2tf+QZmj7nYeYfnSVJ9NXK2F9xLHsLDfsQZnBjcy4U5WSIDrjD
Cw/UdNZGlsNIUD7j6szN5R2Udy+TmH+jt0N+SmhkH7yMakhd+4klggSjFPYnovS/UqKNWRJjJZuz
onat5BF3x0TKmM3Ygh6InmTLEvkldSlJ6+BxvEC33Zgb4IUGvtohtHP8NE4H9aBh20wTlIQI2YTd
sfa/PBAJS62TdCtKzrlDrE9t1yNShu+waja7N4ZFZREfPNLDzi6uspNCM2Js/oD7O/27jk1wPJA3
JvyWw9Ea9hxqyM5FiF+CLkn4mZQ009nYTtoVmxL5M7ylYd2L8exVt7n7+Ke6LU5xNJYRBvi1d7kG
p6HK1NqC21an4NgxbAnOt4S3ZYtxesXia4YnKt/y2L9uDTt+6Tfkz3G3RPgUTKXPywdk8VbQItPM
WrS9TlqYRXJTO1HuRErX95m1sj7D9JCuICsk7stvk9m1F1p7MBJj2gWM3idi4lBtL3sZQiJhvKNj
EX5ye21OLS4DmP6pk/2MCSv9TfzXyJuRPIf7U1FZgydvd+TarVrN5JKc2i272SYpAN7nIjMMLeas
k/r6b7BY+nVPoLoaITb9am3t9RaX2MvYZ3IgtGVR2ycor6l1LNuESonob1V7A/TzIIxu9BGV4atK
mNhWzMpjClcCQ3+cxv3CdgM0ItWx96LObdzbyuyboYKclvfZEVd6p7EZ9OQxbbQjHyJOo91fkkpd
alWbL7Dfmemqviwz67QzOwvcqaBFIabMvFZNh8ikSPyAF7oPvZsLMM2xMThIh52JJp6Y0YiSsJvC
JVXgwmGF5TEIWJsnhyLDdszLk4EqziPgWxksuGcMfeHN4m35kOF5yv1wSvEtLsgto2syyhDHL8PJ
tSwcyZUF3yDLDvnIIO8U5uiO1r/p9SPbT5ARrzhPyD+5uaw07vK9rUSp9wCE9POdx2S/KiF8yhS5
ak6qFE/UdDMu7BplK4nZ9Ql8JfsHcJD5IZTsWYsJ8N4itllEzZ5EB7I1ZML8zEdnYWBWgUzDtI3Y
Fsgr3dEW/tkhhEttOAg+EjSRntkWj0aG1o2TU4JbEP7QfaRbdfqxMPJLaivPJaEbJTcZPtq7S+5e
ozcvnqJ4/wCwTxFJOmQkFNik/IleI/xe5E2xHKcnDKXRxe8M3eT3YP2R1Gx20D/0WKlcqOMGNBZm
LcFr/ethEhuSXDEDak+ccxbAxeL89bQy25bsI4+JcyxPscuF+EhIzcTU8WJAmI5KPgi6tDLQbXKb
2E72y7UiSE88DcZLR2BsduCrYvuX9XkRv2JQn1lhdh+ruPsDZC0FfwLYdwkrfWw7HHjTFA8qTZ6g
4IJ2J2J1M1JU7BKGxrujl+RYv5V7E8ZFjwx/beb0H8sdKx02ScfFdq922KmyYc4aJ/e5zyXPIY5j
JFXLzbGCLX5LLXgtewJMK31kIh18moVsmp88Jh/9sOk5xy7zAnmnyYGpPHhhSPprd8hYLL73yxi3
bO4dUpFaall7vtc/d79BJQHPYR4t9peEi2A3bds+RqTGtJIrfvJZe7LFjxbg+WQPct1XYFi1P9C9
kwHc9L/AW/x9oyZYP6oyVjsPwe0t+45af6pjtrg+5Zl8zzmzE76Cc+tHdCHbL3mUXvrsO3kOu960
EOUMJLPTBZyUeOSSVWdv3UyGtsM0ZDW81qbfga3ty0wDsnyjVjW0zXCwQxWrnr+ZoJBmbLUuw7Qc
3tkk0E+RwqjXQS96jR5JkdSmE3Mr88MNF+UhsO4rzoaFJa8KWH7yJJ5mO56i32cI4NEqlYhG2+Y0
Rnt2BoNZetFIbgSoy9/YkNlYMGDCnBjZOcPN2kD7bLhATkJtK3NkkWteUQBq4GqdnWFlCBXXNwYu
rWgmvAIKR1tT7Ek34Vsh5xvTC0CtavcQ3GliL6huNk0Ql4Pb/wCcdpLsbGClDQ1hrQcX9lshNJQX
nzkpEuTbbby2SAe4nDJEm5cmvI+53o0nlUTj/gyXZD9DkfY+pgT37QkAfOfQavEu2J5F7BXlm6U/
1LHLj8Cv5RXyWpXDd8hIl2UJTSTRAltPyJvJdlk+c+hKAP8Abv5YxL/WS5swFv4k+DhsF8jHtRci
9Z+RW8/wF/2/QPaeDyxS782TLT/eUI+HTsJwbaJpcrkH3neWxS1ppw9pPjD9zvV/3tBsfnFoF0Qf
ZOja+41F21Hh+DsJlyK8becYPQ5JZ84h+gT3IhII3KnkXciRbPRmktxp7j5N5+1bLV/8u99GOaS9
yc2eW4NJhlsajq2DWROxGRAr4LAkDZZOV4a0EyXU3pehIYNHr6y7D4wOE8tiA2fMSeH43syaIf8A
LZjXhOY2r7WR0k40tL6MeeNuTOeZL/CojWzC9LQRNMD+zERa1FGDL3hM0fC6i7E8Grs+6N9QgSPx
0OjA9XfQKIQbPQ1TF6S7Zs1Ry8tv3pmq21FVKjkdUmyUj21w+tO7K72G69kNfUgYX11yBKTg+ih8
u2jNcI1khpfsImIMZrHZoU3jiQQJp4LhcMuG7JLkhj/O5SHsdutE9WCfCDKQfS+owSCZ8qnfN/Qh
h3O4JhmnkW67zEEJFcuEpyxpa57g97xv7xYytWdUPCdsEf3ybHStOYJ494OFybkDloW2ROjIf31J
mSA0M2SytcLLKhiE9F9xY3BSXnngz8Ij9wVvolkTtukLXRyO9uXdYiExOjRy6a0uWOWpdeQ6rVsP
ZsVf0ebJqX5csPdDYL5O07cGLCm8dQ3F0PzJje96v4V3KlQx/wAnDMKFYj5sbkLSTlNK18iY5c2J
PgwpkZDFll9vQ8E2iXG02LQNJ6CSVGtijv03pAbeOh0kWCPci2RK9Iwh+wCVBFXia2m+gbVcJpeR
5UoSZZHn1e6ceCRtMaD6hn2revYe7bYkGZaZNMEra9wvfIhjF2GCXpWuTfJEniqovsx1+qrsReJ3
ePsY1BrkSCkrC8RMeGwd9lRXG23TqVo4HjSG0OYvvIo1roEOVczw4FJNFeGOc7WnlobJfkXTON+x
sO9w1OHKezGNYOJWfJMF3Cgdz6Hr8IUz+WWnKIV3K4fKBquJGLxiSC+2o9iTSuYPczLrIb5EuyzY
QcU8pgqCbN0xNQxGO+YtvbF51oFq9y6u85nP4HWHbbsrGnJ5pvkWgVjXabesrs4tAfqcj6SG+gzX
kfOkfd4k8Ml3ra2D8rAmksNSSi17iCu8uJksbsZJWFHRmSTdOBm0s23bl04iWZyiEqKsvoguXExF
UxK3Rm5BJal3UU5GeCHUiZBOFazIEAnOydIbqRJ3/Y/kDKL4Ebg/QI0FWFoT6rBMTCIL5GWUqSwQ
fvwfP8fZn31r2Qxi8lM2OfSnUsDKUSslNVkk7GzUFutuJR8okISk17Mny/TPs4Er1la9yT2/YrZ3
LtOFefdEoFhJIX+aTSb0LMdbw32EtiIqpsdxE68aE+rGvSig9glShL2JiLrUseDnBn3sH50K9x4J
dVBnZ2Yia674G+traLiCONPmFtslC9iLiaFT8Rg3ensMDHumRo8JJcpI3LS6cIQicZn6oZkPPK+8
SMlbJTn2DlZsHvGJJKfBtOGcSARyebdB3pBW9j5RYOaw4D1GObsi2XeSQRV2FbJheQu49PuIWC7N
6bhRbtmOW7xAtWil5UtpNB/dndDALb+BGQby2RjAVWUQNtzibk8Md0vcDHr0cDUgTaO6JKJYk/ZC
nHx+Ouet3HsoCz2vqHzfrEDc6o+XpFk5FBBF2ggTNCN9lI1puSje1fYY3dZhgsGcG8fAljmwpKpN
WULHvHLgjhwBBKetDlCcRZbQm7YgOpabO53HE4QY0miEOxabifA0u1uQyJZML2qS9vO8zsxKt3Yt
ll9j2YOP3kUuNyaDVLSRv4J63IjRuKJGkCclWDW67HYLiFKKSKyglCcky/i+RY9KdUTo7CI1MuvL
BHKxmH7CpEmjrKlMiSqWbSfNMp8DvD54G/jbcZS2tPAYWbu4lsuRrJ2syVHeymBTtY20N6Ea+8K7
OVDmwj3FSk7iYifMj/NMYlz2IZB9tRK2RcvCcOZPYUMJrnAVRWrZRhQL9yeDc+CLhaOI+B0cIprl
CUFar1vY7QKzHE5Tos0CbbEj7yYCgr3jUtLbwr4FjvzTAbyUUEjJ9qFI/tM/wSahUOws3E/I47x9
kt79tCuF/gY/Xs+Wz9RqZ7kLbZi2bQoDvj7GA9/rbHa78UCH2XhPZKET8JX8IN06XiGXK746CysS
/ERppT6CD7GgrJaOX6j7pyM9Ys0crhtzY+X1P0VkbTk+D9g5+yxYnPkbw5QhFuti+hlm0c6MfUqF
lN5M7w89qJgsTC91vCmXMNKGSL1l7T3Ee0lci91O5r/4E4nVwLK1PI2fsA6B27R6MW5YvK57QSFv
FLLnUsXvGlc6yLxcz90Y7VwyGdxgb3L5bn1s04LIGNu31MkfZ+RP37C0N93CdxooV2I5pSTayAgC
YzsQ1Yv5dOJVKGwh+klvr+Npe7Z5zVR98uaAn4OuyNYyQtiBfvNssjmDnDE5I3lA+BJOOjI4qvM3
uylENy+/AZXz1zjDQtGuIfCEIVpU40kw7OQk5jGd3EjmI5Fq7NXdDwEJ8bSZte4EJvdrlbRti4nL
tcV/ZbD3Givgk/yneHBlUz47phRoEn8yTfaEJGl5/Jav9pPEGenqN8kBSagb/B2Q8W7NSDhb9ghl
H+6JMhv9FkMd2bhZdywNEEei/QRKbbdtl4r75YbHHvyOVWl4J6mVE3CH6ftoSB/AHpTG7Ne4naQV
OCS1lf3Y8iRwrNpfQeezXK1kbOYFEuS1M07CNAdJDWOA3o/vMOYi+49i1tDwsLO2+giovwfxGwjU
E/6Ox1DQ0mNn5BD2bE0PJa3eXwQTomJby9fYfnh9nk1YlJCU7E5CUrksJqlGn+kGjujjYkZOS5iY
nM8DQ/bPk2MUnzDMllUc7osungjV73Cs2aZcpOSieIRBeRRtDHAZp9hymecmqdxLVpG+DELwlExJ
j3yXYVtuA5SMns+EMmRdLE2ZLqVnLpEfLE9B0RcGW7Nvgj/dxnYb5Yc0hbEmVc7UgXI49KfS+J9Y
/R3UX2TIP1eD536GEqOz5Lcir7jQOZRmERW8v7zCpQp+j3PkAoSqWad3wS+i+RgPdu5L4cfjTG4d
ZIRL/WS7slivwmj9qo+WSPai9mXbrJe4llT4RuliluWpfcUfNvhx/wBoe5WgboQQW6UO5eRtWHdr
UZJ5L2DlgRJurD4o/wAN2PIxv27hO23Os08mOIgO4QV9UsRoMLPAgmnO0nYyDY1/s25wYZRkpoZI
bckZFeJyIX3sKnug5RPBdmBydjIuSxD3ESZt7CdWxJzER8xqYv6uoOMIybAWtYfc7LOjgkfLffdF
h5ndu01wOO2W/BculH1irnycvF4Y76pvCcomy1SMSdy3MtSTFoWhsqr/ALrElMl33ZHrLcyK17YI
wVwZDM9ZhLYxDGkw1Zo4/o1qk0s44MpokmEqRJEUdF0ENbslN2citGozasFllogkRdj2HmyUjY00
uXUwC1CNR31Khpx7IVx8m76D7SSeQKMA5o9jT7CDra+Uu59FauWKzbTMvyIOLH3MXoG31SCjmI96
9uYfRkrrxIoZ3qXAXzm/90ei+nI36XI+wzBdxn9j9ng+d+gm2I/fiRp06HXeOVcRf7tp7iV0q27O
4xnmG03Cci71+E7XWAtNDFdfI3Z+IdJwLLMHvOOajsPOUStHpKC99z9m6TCTB0l7hc7GdwIoMLVv
c1RbfiIrt8xoMGllvkLZIsTh4FhpLdq3HzZ+FLl9hkv98xKFo/njhSU3skce+e7qzcku+VySPbgO
xawcnU1kyPPGyOKz5zLCaDkliLIW1vKxw9JEZc4ZDEad8li9vgct+BBpasjfeAlCtREkopfpkQQR
SRFVdiUkKX4GNLQxUfLsiWI6jKSH8ljJ90lIsQRHmRyy1bC/eTVyuSNSGQYn8pbcEX2IgdmhKFuR
xQwPM262Jsu9EAl7FtGItL9hVpjS4UtpOpb0SrL+DSRpeRBxymbX0quEKaJU3kUkNTWxFeGmtjFg
qUb/AL3Fc2Y44t4TFXlKLyac8oli/liBhEJvEkrA84J2REgNZXwiZsj9WJ6trSOrAuiHvZmJsQek
2CKnMcphDMkosPMVaIVPkWgLKf8ARuzuIQfdVYnQIJe5+Ec7DSLmnajM2oWBps55L6oeTH6Fz6sm
XYS+wvrUf1vvIynJz/b4F6pyknqWTCNL2GXfKTzQKfNRO65Gq6col3m85EPS1oh5G6cfhOyXsuib
N3aH8DKWm7T9aNJ6KsLRLqRBrRYd/cs+iDGRK1MGaChiT7SOEJKQPAzRGsNEVSdHUeQwzsT+T3/q
NFm0XJpbjD6kCA8NboRtppre3wWeNVf9g0WZQRFkWuKepaJHeX6CVDMZxuTGhMN/ZXIqBdheEETO
Gqe0l/qkwf3bFSDNTGJlDzSDs6/qi0rsP2Et+W0xAs87obbshf0tCVxGTliTUaLV6iQk+kmBMys2
4JpPlJZ7BzsWWjxIXuLsMs4QHAYnYuLH6E+5ufPbdhVRw4N/kgCdtItVbdftJndRrfVhiq6SzCuL
rLIVLixAViGdJ4sO6bz8m6gnHpnOj25NVy3i4Qd9QMDRi/uL4B85uohreTcosPV7F5taPcDx+jBB
ylAXPH3G+5BHF47CwroRSlfJ+z/BYdAiRDF/qqNPeyZNgkupXIbg/lAVVF4fA96H5F2W/CZIA4t/
ccyIS17nL6F4XRuNgXmsoladbCUPGn7uSCf7/hENzOHu9eGdye+B9QJefOdYfIrjBJZCG3+I9Y3w
PL9ch/37ixf+5Gnu/Sfs8EHdSCVOaJOo1UiCBEEDgu+CQbXSkpJOCUtuEt2PfHTYFOFrBHCyab7I
sNWhC+BDJ8ntgt95fak9tHXywq1PlWXzJN2ie7LObSVmm8DS77M+gvHehUgdS0mtDMJUYIEpFm32
FyIrv1XIgmzq90S/VXu+ouRGtaXExMfxQZ11aGaHvIpqgmqeVCF7siBpXRaOxoIYZm5iIDEqc2dm
hrspfYyNUXuWRhd0vYeVVq1tF/LGzSVUNLcVNNWjrshcNG5y8skOCqxd2Q+gmT1tLpVy+BstseUZ
n6SL0Zl3exGFAv1ZHU2TFqHl0/TcDnjNuJxGkO60iKbwPcxewaURKXyyM6L4rNB89GZEvKP4AQLc
M+IGlf0Z3WMeRrQsiLCUe8TA37dyKR+kkbYVnOwvwktp/IzIOm/PNy5XItaJM9reyQ8DiO+BlCfe
hnZz+mBZPbDC8Aj5oiFe/wBIYDbXgeRpvShhP8qSPK+uyIwhDLK3FNkzwkKgJdYcWDGXg4tq3H9R
6S0uso0vuiP3ZaSlBb/3A0T91hd+wpMzTtWBKIjD0ntwIyvJvGY0GFx3C58og3bkUWTPILKnhXL6
k2qQxFIVmIWsKyE6kKWHDXcXhy6y/BCFCmk0ToTLLapiPW+AWf75GRXfvuST3fiJ+/YVev6fYuwk
ELkiGjJthzKOTNWnohkxbGr2hk/fYdHtsSGu7FxuH6rUho061mh4F/sBa2koRI9K8l7Gq1CHJBjN
rhDYZuAlL7u7jomlZfIPNZbwaMMKV9COEsaLxJOys+WzG1Kty0DJNyZ7i23GQtJR2QKAXqCbHW0T
ksNuo1vR6B50K0hNG1MHFt2y2y2z2q/BDHduZZEh3wy2mhm/vM0ZDnrG7pHMZ1mjmLozOHkG77OX
3j/YFm13eUkUmLB6PtDdukXyKC84nhjwPVaQZfYyqycc7+LCFInuE3nS5u612hwUwzLyNmiCGUNy
LRvXhKR2Yz8jIn6FxDkhG/fQQnK2wkRPv3AdGHsLq8aiEWxbu8B4N/TbsUE3cSauS08JH+jX9VeT
3LbCQQ5tkJT1WbTeQpeYtuMOUpJ1YisQf37EJz3LmAWJvMmV51ZwxsRjM1X5Jdy9+fuWVo6jgiaW
vQkImBYQ+SDUt9+CBBApeTlufkteNh2CTBOzhIxxlm2+WOj1eKD+WSDS+CGdrDBNOGnKY4bqsvkl
jtxOGRFpxI/qTXOSNiJIU4JG5NCaS/8AnzEin9cjL99TF+9T9vYzfrB8j9BUyw6m40wVebVsbikj
aJRjDAlq/E3ZKN/LhvRtxDF9YAx47dy+5BXdJqThdxLZXsw7C3CsbC3iJbtNNC+oq999DjRo56b7
pn+D8BnMahP7mFcWEyvlCkK3mLHMSwDqStwA0HrwGLP3rn7ndDs4QnibGuuCHmR2V0e5RDwNYRF3
ZusX6+77PsO77QfIGNifAiXaPZKUF1JtEw2TZC58i0rzclqhGkrb2XJrvNWmO9f8Q9R3kiRybY8L
cHfOoi4+0p/kOL10bBPOvRDhbNPQxSq5Kx7IfGiTWzI15+Fgj77wiZP3WeCGhQ5Ws93TD4cNSaRc
CxeCi6pW0S7Kq6rVZkO4/E0sS2ZPtDQ9iRubGW0ujh5Rjqluk1noTgbF/wAD6YfoRSKr14w7F+1y
P0d2fefUzT9ZM/6wfs8E5ZUcHMSaYyRmV4o8k3lkcscikNSXtL0QiN2/bsjtKUJqUN2onzjeZ4Xg
VnEpN79sT5YpxCRthoOQrFOZsbjvtm7BD0SaYhM+4tOMyue2g1ybvmcjBERiSTwWo2SU5Dc2u3/B
8Cq+5IISbuRrqYGzJKdR8h04NCt5DfqqWZkO6aLTErsl4GaGzanLjuXdL7BjZonZ2RKF7E6rZV2u
S4SMhS9mnZUbVFvbprPTqfHoJSQphNMgsnEqjXTH/Suq0ei+mKw6T6yUs79Q8u36x9D62Y/vqbG3
0Cft2Egb26JHT4kiyktw9GcJ0bFQwrME5gYjCzOVclWEuSpJFnoeWKy0P4F8IdmtDhqGWeZfe4km
EZElpWCC9XR80i+SCKq9IkuctaJ8eYWPrMEMjxOYVl7sQ/KltNo8MhdEGlKVOnIrZONTgY4dDsFq
UkxAYb2SlkARu4hman/INx9VZkSvhAkDaXyuYWe7j+hP3r+RfUpHkXotUj/kdHSCKIjo4erHqFz/
AEyMv31MP31PofQz9XiiTIZcxlnuKHqQQ6ubYCJbFAwltp7DVhsOCVB9Q47mMME6FcsCTMwnul7k
Ctu/FrNv8Eb85kcfTp9kINatWQkmPYSZCPKaGojy+h+5O7djMIgVrUqeew7anNiRxmWZ+6GvVdYZ
akiLjXQna5jLI3trzeX3D8Io/wAw0VoTwtzLDtsEGpIJEUaCTcWO5Zkn1X+6Ho6JXGRWcCmiwied
/O5AakbS0yjXA9+wJ0fbSBBgSnd3LtiNqRkxJE7/AGK3LTMs0W8EhvpGkaQIh7/kHp63HAPqmMD3
6YpYYk3ZDWUyPUj1ZH6MekvUwFr/ALuM/wB8sw/XU/V2Yv6dhU8tcN4RPalMAkMFgctwGuSy4WWI
35H8j2uFjdcd722CVWggdLYI3Asjy8H9AlpQ2v1ieXkuyoPm+EJ1mNZzDL1DsOdCzX16uLgqVfjk
a3az4S8bxt+ILp+JVs3SOd59Eq9zGyNWOTSeYeBp5g5uJIrWXgcWoxGFuVd0sWNM7uELVynuQih+
92RCeUeSeCKPhIzqkujIVkidWaVzB9mDLcj9XPYoPaoK+/8AWY0OeKXaJc9xEDC3KuqJIvs5Y9iJ
7OJb6j+75dURyHhaHrItPaKEb7GXrwMMvspO/wBcl5wVSkl3kKGhTxwNtIwI2Ow1gkEOW+guNJLn
gZ+6jhT+EP5d3Sn6EykyKadLqlSCOiaRzSP+1z/0zy/XM/V3Zh+upY/0wz9nY+V9BGUFPuT+vQaR
Zic9JMkY+bG3keIkX2E6iG+42A8/VAnSyWlfuS6fdS7VV2bmh4AnCxh3E8mpPbuYmLe/KHkstIuC
0+NodLt9QcS0aeEYPP0iqR6ohpZHiRaNR4QQBf50nz/1pjLwt/IjECucYtBHXYnuxNQEco+DT2Vf
M4VoTTFe4IXAN7JNpCS+CVgan2oXQmO5i8ZpUMsCnQWnjZc3ZqUkpNxxTW9jY24X23TMmj7rMs1o
p3cariJMGt9R7M2hbskLBcUGNOlhNlieDLmikNqZIIhMkSh6u/fEu4Tedznk+8VDsU+K3M9MdKH6
0dEFj9WB9LUeiqPpnrWV2H4P1j9Hdn1/uP29mJ+3B8z6BRmyzkWeaxYQv7iJSSMCaU4j3HEKIFbl
Z4eYWoq+1dlch/cTNXQxyx24kkhLMolxuDL5pdrELYwDX8U5LQkQXKA/xJOyxOiG1KCalhguk2pp
wLGY79xkoec7laD+AYzjZlXbC3L0LnuyW0wzljyVbfyRC8glwc8JIvujZCTdsKGGlktyWNbvJaY2
TqQVrsBuVyuZOOb5Jc2bXZwO8Nu42+iSSHHpISJ/5Y9eOqB9L646o/5A/p8jKz9tli7/ALz6H0s+
p9hrmV0pcqCHfJHKh1hkO6SkhJ4JhIUm6F6CKUQzInkDdEqRxUvM5dbK6y8EfY8XKJECtieDA0ku
kJGEqMlJiemf+S9Y/wCeCKx1tEdMEU0II9GOmB0X/HHoLKP93mfv7st/fc+2+Gfa+hm1yKRav6O4
v2vyauSrIH1z6D9GPSSpBHpR1QRSOmCCCOuCCCCOuKR1PoikEUZFY6Iq+uOmOuOpf6e8S7fu7Pof
eY8PtYoH7/FV8waH66joXVFI9SCCKwa1gggjoikUggikEdMdKGuiCKQQRSCCCCKIIIpwNECIIIII
IIIg6wQRSJJIapBFG32ENUYUJuo/lHPftTOSc6my7xw/vM/f3Z+z3Pn+4I4B+tx12dI6VSCKMjpg
aqlSKtCrFGKkVjoa6IpBBBAkNVggggggikEEDXQQNEUQRUaIHQhjajBBHDP5h/FYv9gf3hteA/gH
72jjeBx/Al/0R5gJr/KL+2M4+Qn/ABYhyvIL/eCZ9yJxSm10hNTSwn3jmaiorpjQDVJu1kxahi7o
UEzi32bbEq76pEi+KJuzJMYWXtU7jOlWf92d/wBr7y6tpwDnaOdyCSB8ktVOwrNeGuTeFoNW6vak
XAt3Z5bE3S03BcIWtz8vIjdN14EaUYU21bL8LLejQizv8D+vvMR1+LZmtLMCey2j2Fw6P9D9vivy
g6xWCKQR0wJdMEFnRBAggisEEUgggggggggikVSGiCCLUsQiIpBPZ+D+MJv5VISnzfgszDwOLxED
x4jdSJv+xJ+JiZZNv3BO085vBh/CLV8oWrT+WZY8R7j97CVyCVYnkXldP3yW5h4ailv4C+5aHPsk
i3b5L8GRT3w/A7z0NyCFJy9WWMPLJbOziG19yPCSomWdhqsEJ2d8MuHz8IQfjilW7sxBPeqlllqH
hbkNrCcs5Rk143MZxZMzohpInm53uSZZPkK5iZcWbuTsAb7LBBL10OsyPJzfMGaWS/tL7081IJqF
Wr5Ey6dfh4EkYZX3BPifyYNPC8JNVasB2idITcQfKRTGOS+6vu5G/cGs7l+onZf1HBJNe30opu/v
Epb/AHhF/q/LIKh+zIiS5/0hJhwrwNT8BbY1/RH7/FcAGPoiaQKkUggjoggjogiiOoGRFIJbM/mD
X42Xs/sInHd5QtdF7KBDlXufuszo7JhrfjAL+8KKeR1TfAVviNjRjdCDbv8AyQZX7ODF9UL/ACWL
9h/civ8AAEvT4ln7P4Eeou6/BsQ9g/7BvTKPRrq0NI1AvvCJ/bkX4BAsOyGhMyXGhO1EjSfxqf6X
3QWl9kP7i2uBSvwYeR3tA7w+CGX3Idr3hak2yl5G0JTUSyZsvoPXkZvwWNfRE7M3B302fEksrreL
utmbwEiU6o/YUNl943F/WvkL70w83E22hb0LJI0faCTXn6QNAgKzd4mdw2SOcN17WEZVo1A5u9fs
IRPjAkyszEUp6ZJVhMi9yHlt/nuBiKhzxyXGneUIHsMfbfFDFw++OxmseSgRdp9/C9wzJ6RfgFzp
bfu8k6CxKPfB5yM+c/hCSiavGbxy+D7PmTYNWfvwb2IcXblndfQfHb4aNBp/DMoLNWRuMgggh9Ig
ghEUpuPEfxBQ/bEPeBf0oX+AN35G6kWtqFZiWM4be8uCaf4Yv9JKRN13n/Sc/wBCFYlHZNi2kvfT
CVfdopc/Z8EOPOtPYSnM/Eoky5K/j5ZbsMS+2NK32icsn2HYS4JY3OpeVfDQtpHKY7OYfwe2v2MF
2VXp3P8AB5e5/o9H2PwQr+5AkrWIQ8mqQ0L0+wy92KM/U09jUnvoX+CTOTdOg7pw9TJDVhf5LpKM
WLXJyNWYhGrrdFiaghXMW5NPYu2v1geCG98BLWn+kCrSu19wszp+pFLLOzi1LWy+cELt/pY1pTev
YSaZyVj/AGwHMe6eYCbl/wBBqCbxsBC1brbqCQuLT/AJqG32Vw5JTCvoiSRaazRZGlDlZFsV3q9k
QfUjVesYZcg1FDpL0hEJxLTcPCErOwe2XERhidlUNRkx6au/Ym+X0qgJyhrbavqyW7nULlQICxhO
IlGsX+1Ui0RKYTsXLIG7vaj6BYIbPdkkxrzZgtRpbw9ZP03bPuHfU3+TUevdlxCHnwa5fInJ/VVI
31lqWb0xP3HOepRpiP8AJjc+ATOJiu/KFHjxYl2sxgfw4laPEWs/tCFu+RyeQnavIg0eWX+Kwuz+
QNzxBIwrsjgS9SfrRXj2Fy9qJff6m/Yn70i89voIWESa+T/TL7fYRggStqvqQSr+5N+D+xFyPsOY
Y/zTQfsckZEU/wAJBDwyb+5AifFIgiy7IZgkf2Hv3NVbUjRwaeR51z9hfgh+0M2PccKIOy5GuCGG
qWhFpi9Dds6bkrnaoa3f3S7l2j0X+0WVK13fQOD6TfsT/q/Bd7YzY8D9TBEM8A/ffuj3VLn8p92F
/BPH1InznXtlKUWL+65DjX0zMfgpx3SLZp4x7AbPxK61OivuIH6TcX+PD6H7x9zn/Vyc5+m5b1+/
9zvh+h0iSXYgl2WnpJVjzoIrAW3guZWTnPrCfxH8Q2mjc3cQ8BrZe7DmY9wPkpWRRNWTacEobhkK
43GTEvUcr7DTXiFhwj2JJCb3HyZL/Qu2thIWa/4Jm1JFgwHhoRPMhQSa+Tej+/2ptVKPL+o8Psb+
9dfdEYrjwxLPc19zPwJDR/tP8IkggiYavqHrTSLu5g+xwDsrs/rIwfVRZv4B/fOKd32d9qXQf5hP
+WBow/3/ADJb9/5P2an+owWuc4l92zZX7n3oRu/t7i1m9lG4ZuM258D8G+/00P1fmOL93Il7u7b+
p/IkWAIvxT43ERBUwNXQ2HaGlAhuPeY9w5Ny9kcZwU9h925gTGT4HZ7GedSVr7G89yW7I3jLG5a9
iWSW7HuHu/hLcaLT5HFfBsNX4g/9Ef8Avj/OS0XvfDpx6HwHDRvid/1wSSNfAefI3Eud6XGnA3GW
oYnScC0Ec0dvgwTSCCSzGiLjRdyP8kRoQQxQgzKvCIdiGQN/7hLPlQgdDLtJspya/wC0baAkxr7P
uJzxfruSnt+X+D+w/Bsw92OH9zG5+ruS5+MbdPt/g/R/A5u/Wwt5/TczWu/bI2Zl7s5rvI/jTCH2
CSWEv+N0e9ZO7Yxd9zP3+xP6EvqS31HaG6ocLpyGmqL3GvPkIc+Cf58P/VP22cfwGvUHMe80RGQ5
7wNf91Q2zH7+xtr7/wCT+yJH99jbYHH+4l1Jx4+JqBfz0f3USzuSX80bXc+4/uMky35ZLpInG0gt
ov0LJLYk0E17I7TtO2gl2Lz7mZo0bi1nky370Y0Lh/UhSqJDFFoS2IewnlV7kNXcaBqz4VJ0F/Zn
fdi7gzmP1e5w8qfcXD9hLp/XY/vhcDyS6P65OGZf8ib7H+CP+pz3vo7keWf6Fz+KEvC/ZHGI/wC6
F+CExN4kSTE+knC2ZewSx5R+4x6aPcaCXn/IzLf/AIkn4A/545/mJ/zRvyNP+aTZbe7JbSKeTD+D
9IOwke8hsxI9HS4jgN5UiQRKSMGIRGdxAL5BxC1hUOwhv1IbHAR2O3QyQiQ7CGxCMIs0O0g01LWI
I1XvENiL8xknjzSdHiEm/V9T9CGx+/sKZ2p/ehJ/tnA+zOF/XNM/uU5/EvsL/TG5y3++nub5CX+C
T+EI8eNChhCBEiRpjuXH/dqeaI3YdhXkxaH4eotjZctc1wxiQoTB+HSwUWFs0Y0qS4K1ad7DQm/R
uj2KP2uETY4/URuH9RTHvZ+BzVs19CBG89S/2dyAjsXsCG2w7gt0QlEdiJBod47DsGjvejsIDsWR
cUiC0REj+ogQwZ50ZHctROhGlxpu+Pn10P3gkljTcbFh7RNFdbhF8sEbAf4GZdxd9T9ERY/T4Nh3
7bCDuU8PqJmPg/UY299I73uf9X+C7R7uOO90alJ7/iL+L3hl+Fhx3ZPycv2P8TV9iP5ggQIUxIkD
mI7jREakL3NVPfqd9Mx8zuG/IynA2oQU3QlVI8OjRlwcAZxJTKU4RmBP0JIb0ghr6f8AEvdw9KVO
5+AjsRJd42Imh4GZirKbwr4SiNuO0SUXp+jp8AJp/aCx+9Yamz/FC+I8U/e4Dnya+SBS5f6Ej7E5
90I03RpC2e7E4JuS5xVEkZbZDY7DtOyhqz5Bw/EGnC9p/Hkf2bnN/bg2W/77H6j9jZd719T9CT70
eeh76/uc0v8AX/wYieKPcDlwezfgemMlyns9Ey/Eq/JAKbs/E+gbM/E+gzbfb++5+v8AOPeu7fuW
nLM5Un8aRK3jEWkCxV4wuolWCZ0N5NB+AyK4+Q4uwz9xI3Xe8aIOzI5s6jVrOg1YR5GR8Bl6ieRD
PgD1vCGJjYkaLryP/fGjPlQ1Z8xq/ObANU3524Hdv4D0WexhvjICe/Dbjyjbo+TYT7MbNSbdI3le
w/ygecSXPkE2fMxPlky3SjJLiHdsRPRpw8+x+xxSJpLPKE7sfu4/B3beKR1LDJOxjvzIy5DvooEP
EibVDaPsQP8A3iK2kTaJ6lZom1UM+xFJhqw+oIrcNo/RbDF6GD2TyhmbRFmbCM/PXsjV+LdlItCz
FYW7rEK+TQLLwnCcHchyZsTaTIhaw90/FIrcSil1pwhp+0EKZ9Oc3DwkS+hvM932P1X7ndd2/cww
+96SS8eOLCPtEtZVU7kd5Og+aGiEA1EN2K5qxLL7pH9w4b0Hc1H8yIJZ8hBmeA/aIaIP/GiUbO5H
n5BoiNDRh7sTS3GcvawafyPT1tPyH/MH/sIZdvBG8P8ArsWZXnj/ANZj0/lP57Hxg2lb/wDI/hIn
pib3kEn5mSrJ/Itc+zJ8O9wvuUhYHjP6RzIk1fcc/kzekTC4khDY7BLqqmy21M1vgRYBDDZCTYvY
EdlghsQIbGsNIs0YpuMqjrRzCQbLSn7LglOQV2JcJKUtvwLCLSNpKQ+iwN9oZneQXuhSrKd4GcjT
2VIuXPECKkmm2Rzka1vnJx2Lx6arzo/3SEzRSarAvwV0C+/TK3ORHGOGccaq1mrc7pxM2jLypeAP
bEWULuaBe0yXijSyf3x3p18CHWLSu7MOHLdjFqeyHo/ANX2Ub6N7h/742vIP6semjyPYB9jlO2Jt
UuEP1VVzNpJt88x5jQ2jBM/MJ8n3iMykZ/wGvyIl2+wTybI9xn87F/XHbv2BbygokBR2fazLH4YF
kfnJU4DRlYSn5TEtmvrYmkpnrkX40MZ+ISVbwoQs8ASoM+c2LM+RHBe9JxOZuEifMOyRAdh2UuAX
EzQhsdh2mLsKnrI7ETWJCBE4iA1ZaIPzo/tjL+hjCL3JqnGI2KsKU6QvTlHIjjHFOHRcp3jujuQ2
vwNz7hWO0sJ7F8kkIWzyapXuhtXXfZsPVRjQIZBNfcHI7pPc/eNG4v8AkGqHuhlwtOTpJdMjZV7x
6Cfdm0n2Zzi26XZR38AbFfxo+xYeU/3G9X3jyvdCbUCd+eJmGe8T/wARNj4ybd8oeYPacOZNVK2n
3LZpGpwzeGlZ7w4M+ND2fCRo/KiyTTP94kIA+reciE7cWYSUvvZ7T8HRTZBIFfumguxuF8DShj7o
hXV7kztbE0+WOH4EftB5FII8v2En9iNA0/pjjjKEtmdwo37V5J6E4MBxT4ReceIeKBcB27bE+S9B
dhvs4u5Gzf8ArDY7+Vk6+8Nz+6cj3Y293W2xbSgfoZP+yn4jf2p/BP0wbfl/I/pH7/8AJ+psjcnl
n9L+Tgew/nI/Z/Ahr5KPJGvkOCfwyL8aFpqRIDsZGbnDOHQ853zhY94KWqlbr/Q1Wl7oVyr2GGGd
ImBY/huKz9QPePeNGJB/yDSaujVj2G7zQ3or8myoccWzQ9o2ag/xEbLhbg949Re4ervZi333CYFn
8ZssJmkFFg9xt3X7kwL8CYmm4Vv/AAQavsOZBCf14nPnMf1mTpNvQvxMWlPYRj7aIoQeBLROzNxX
uLotvWRqu/MNWn2N54IHbsJGe5SNUf8AsH9Ye57hsKKUGxn/AJIpsY4k7Ik/EPBL2Gyhpuz5DLeU
bvyjzHH9gltRIuw4EhC2RB7UbpI2SJ0Rd1kknodbE1lipD+I4CA16O780O81yHzGHmNlpSeYhnzD
zF+0ynijTP1Ro/3Gv8GFcN7OaBw0AGU0vHG2jMaBXdxrB8sn0vZjGvYLYsF2QZX40N2fIPJK85/v
N0e7N0i0vAKW74E7BJv+0f2jd+QobqNb4xcj2Nb4KNofkOLyEqANkFN9gicRhLx4EQPtIThaNC79
4gV08kewOfAbn4Fjc9hpVpDgNo+qOSWsBguJHFC8AziPsNuVORTm6FbokufIN8fUN9jmGctjuW2I
VZo5J6JH6GtJJpNJZNJIpp056H1T0adOKbm+zDVrl7APTf4Hos90aIP96PQ87Nh+Rvwr3Ddplvb2
Qbv8BvyL8oRpdnvG27d7i7RhM1eBOf2CTHgGcAnhPygX9Z9kmLUC38Bv+AYJKy4Wun9xPd7mMkn8
ItPwDaPshLAyRiQ5CbKkE5HOyDcPYbHpmDoJgbCQ5UM0hu5mWEv5j+8cgl7ukntRusjF0SSTWfQl
Vkkn/jt1ZD/4p6WxUvKp0XiRzhRkocTkUm/wLXdiFshK0ci3hG7BvSRIIbLuQYXxkLZCcWG0pdwG
nNV0G5xM23NUMCToHso5q/S6iZ5jHyBt7smkk0T6E6yTV0msk9MirNZpNJ/6n1x0Ik063029R1dG
wcxymymcDLlG9g2UPZDc8I5xtpfJG/WT3ZLJ9KeqemeqSfTn1J6ZNemTPoTE/UkVLkdEC6V6E+ku
ieh3Lkk9KfVr1zSSfUkv0M0Jon0ST0SJjF6iquqelelPTrWf+OSReo6Kj656ZovSfSqqs+vP/Or0
noVdOpehPXPpT0tdD9afVktr0TXXoi48elr1z6c0XTMdMk0jpXWuhvpXXPVPUlbouLqmxr0SKsOs
N0SNVmiezFjT7U+0dxzWRdRc2diSOti/4Io7i6Vmsf8AOup+ncQmT6Mj/wCJsTtRHiP3ow23tSRv
Ofic3gIMsUk93Y/qFlqLj0ytjaCGIRGDUxPbIbybeEzGgyQpSGVNacjNPTahEDbU3NafCLA+Isdg
NZOUcEMxszyMnmksmBNpyrNHuKdyfTdXJNbE9C6J6p6F0p9U+kmOk0X/AI89T8Ah7LY8SlrAm1fJ
YJxy1ctZwx6klcoztuj4Qy8oPcyZYuTG8ookSItILgaDZtczsCXy8EwudMx2Fu8iwcgi2F+RY7Aw
eEfLmhgnGw2Ntp53E/ChEzWaT6k9N6L0l0YJ6GT6CJ6U/wDukVL+rNVaayNB59DSi+C63N/IszQh
TfI0PCPggZb0ZuwRobQIoBI/AdxsPHuEjFCX+CzEy3ryaU+jHjtRPPNxc/dDW7jKJ6lJ44S7dH4Y
SUc6Nd1m+bDTlpzKGfvOt9OvTIs0kQ/+eCPTsQQaUnodJ6n/AODIySU4peUMThILdnYxPwsf3EQk
q5RPsKfDTTgWF7BUA7GgN7qBIVtb4Gj3aNFn5xmST3WBkRdZHMXx7F/dQ0M4GHyp+FFvcj68Utyo
ZFdUaHNcvdUIi7CDj3n1Hpumf+KfSuT0ST6bb60PoXpTXXqfQvSfU99kNdPkJJwJ+Bu9oGt7ND2A
sDiRXiV7Dwcmqa7Nh4d0ZvVRLotAsktLUSQlyH/DCXcS5FrIbxx9RLdTmo4E1kjitRSXj3vob5JS
qZ7a6MqW2v2ETSaLYewzuxSXT0H/AMVupli1UPqn1Z9KOufSVZ9DTpUTkdupiGrNboeVYLHmtCab
LGsbWQiHg+CyDQvxLwOKRqJQ8WPgOV60E1O0MSbAk5IKzBjOvIo0g43GxI0OXAhrhKjuF3Akm/xH
EqXudFUTF2SH2pCWiqxYJJfo6kk9bnQV7+vHQh1VHWeiaz6tyeu1Z9Sf+PBIrpRcbnu6Q2UhJMJS
/wD4FvVvwekqRSP+GfTXoT6yd/Q0/wCFD9Cav0Jq/TVF0tEOiH1TWRqV/wCFYt0rpXU/Vm5L61/w
z6Uk+k6KkMv6U+jPqMQ6z/wT0Sz/AMiG46p9J4ET68Cz14rPTNZ631z6E1sY9J9So+helNExu9GL
onpnq06FRf8ASVF6E+nPQkt0ukFvTXUhi6UqT6M1ZJqOsVajNF6EdT6XSOmYM1mj/wCSaLqm/Rbr
VJM9Cv0QiPSVEM0J9BdNum5es0nqdVTJCNeh/wDLNEO9Gyehz6E+lHrIkVW+tdUwTbpnqXUqMXoR
6sVikESQ623J2h0W5H4FL8Bf/Om/uIl/2f1qxRFXWemOrfqcVVW+uaY9GeifTYsC6po+uOufRv0v
DFRiIJDVIPctGRRuRw6EcvBtlJhqRf0jEtwTEv8AKJfwQ30iEO95NOGJJaASNIlwRI5jkLiWSSIi
urrl0kWOpQQn1LNJ/wC19OfUXpxRqkIstRQxwLs/BL/ASZc+k4/k3flQLlw7RFf4I3mI7ujRqEnE
TwNkJNDYPtpcxyE2om08kvckSIkY1oFJqW3JW6FroPM8x/YOOP8AgZG8M7LE2+4YwSRW8AtLxhXK
75CXTMAheWyXQfDPoT9Gws9M/wDDPorqXXPShIgjptuSlrS7/AfzRb4kHD8i/sN50fKcfB+1G54K
b+1i33noTCL8Q9qRFQlQJyZJJJJJJBDkaauxavKHu3EFCIp5j0uNP5BqCDSc1+xJki5W8kWAz3nx
KoJmRd1Qtm9pcxiXRGwvJhEWqhfiQWoPCIBfqSO33DijNeRsYsqVivtJVk9kE43dhJX+1Ce4X3oH
O8jRrYluODtAku59JUgZcIuxvrXozV9D6H6CYmhyI4hLqJdPAyG/CfzRFfrZxeRyrRp4WuxHd4N0
P6JweQv7jgmwUr+Ie2BHyE2pLckmiRsTdbl9huNUf1h/65F+WkMXK8G28QyY1495g5OVAn49xDj+
6k/1xWv7oWsvyLWFhNCpfif7EwF14aLnFJVzFp9ZI8PAsTwRO4vtIjRw4Qg1Q3JxlmOQ+WHAtLAu
Ias7hJC/kcFzkY4PJldeDkM9ENxppQtw55g5B8ia1LNRuMdw2LseixVZNWI3ZUXXBDI6LEqkPTwH
8IlLbC5+ZwryT7fcRO6vg/eC1E37EElzedIL+02Ch/CQ4V7EuDnOQluS3JbJqLCSS4xMReYGnPkE
/wAwhQUet9kSWGSYRIub7CboRz/4DOPdQ1rfcbFdAzp+xreEy3jjMBkblvuhXLCWf7gsxYXuXI8G
wM9otJfKQlDwJod2jeY5RLIYHuE2rGl7wPmhqr6ktyBbIm8iWMpOYSWVBbkSFZjNRunkv1ErUluZ
g+LseuanvHoM76OBE7qZ7kdzvIDHcT3pd/FEdxtuLnRi67F6Si249hLhjW54NFQBQZn6mJ2Ye4or
lZUQXMOWzCjuUZp0iRrdJDZE9hbgkncfK7ycxIkSG5JpJDL7Utuhq/INNAEeke58GzPshScJgkx4
BTfIJdC9y1OTk1JfcmhtLj+EzuvsE/x1SjmfMasH+4zNU2KDGENoZElwOyLdKISNeRcjvINSQTHc
sJUnceoa7mWScHuKNLZLchtTg7mOROg3R3ksSTiJI3uWK7qx3MoakT3GZUrkOOpOaGxMMOcX6Cxd
keNy4iRZdHeXa0t+kJkyZIkSqWXReRa7HCJVAufQO95IBxBLwT2Ao6CZyk9yUu5LJ0X3JHBcl0hu
cAet5CP8xAbWT/ioTSsZrXXsJ+DkK9WVe5vJ9ztBM0ze9tGO8bvtIyNsIn98Wq4YLvIS8U6kjCfY
JpeF9h7fag3AwksuyZORYhSm4tZ3DRF2kCiZwkjkZWG7LNR67hBODuEJ3uSWB6JE4zS+R3inqOMH
eN21cmWncQVawykwFWQMjARvyPlR3HdRIluS6W768hi+ieCWOXS5cuS6pSNhNQmTFSEXfQ3O8kTJ
HSSGrQ06Ro/JQmvSP+QenPg2W+xwaNwk78pZx+4uySvEN2Qc92o+833FqeUWu0fqYgvqUFKj6Als
uyD0PAWqmyY2Ra1xShkVRHcaC4kRlDMnJIdF7RFLUy7IJFZCUmJ3neWESb1Gb6bRsJhsIyoluORP
pZ8iEF4zvoVb3kzuoxJbkqkhOSomiRLJZLJZNFJcghskSJE9iRPmknJ108Y6VVPsIkDgEjQRWHJa
kJBxDaDtSTK0jcDmPOFtkWDxhISsOGQJpRQokdpJWwpCSRsjQ6BI6XS76YRJ3jpLcOYkoiIdpbWS
j5Us45HTt1GzgUCT1oT6B31XOnIdOVPcT6IxMnS3EwylRIkkcimOi40IZBDEZ1JUEmT2pmJiZ2Cp
qilOISbEAqkKsUIEhJlE0HNQa1NpTbQ03JRNE0TRNVL6VwK2mzI+slKhmb6rR3oYwyRDYTUMd5fR
Ki0v6NMmTJEyRJNqJpIqX6YZFEqUiRInT2Etjsrp6FwF0JESkKgghQgpQiBFBuVCJChAgIQ3IDUd
2hEjuPcG3UaNRo1OIMFnfQdCexwDrBzDZ0I+s3FdHmjvrOm6egdlR+kk7b0Jk0SJZJNJL1vWOgTJ
k+hEztrIYCPZRClAgRIkCBEiQIISTIEIsWJECBEgRpQIECBEaHeOetCG5HcjuMIkSNBy06IN9B7h
ynKc5yEtyawQQyWxMl06B50TTPQpJb1G6TUmk0SJkk0SSMnoSSS6XpFEEVOwmTJkyVMxdJlcLohI
h4ECAlIUJIgSRYbQoJEolEqjMgQIDWiFaFBLuNNyNGi1GpHca7kN6e2g6U945jmJtTlGEiRL3JfT
IkS6KpUzroJSBAUiNeBEjTAQhFEkkk0kkmkkv0YIGiCCGQxUZ1Zi6DVSloiJSBAhEIgQiEYqqShq
IUwIEKI1Fc6VtEC2PeIbnN0jkN6HMc4985jlOSgklLc6DbpJPRD2OA4ji6jIndVHtL2Bb0HsEciU
iiA0IENqUsEDaHRCrYgggiiiCKa0lTBFYII6RT0yJ9f0asCNdAhVElS1bUkkgQqRoRIEKYUwO8gh
qzmFyIERrTIn0B8hznKSanMS9yX0XLnAcBxUOGsSJsnTCntICTY7CBGvDp1LTAiQqYVZdEVQQLFF
0RemXTBBpSCGMSLvVBCCBJdVqa1ksWESaiWhEj0GG5AhuQ3Gm9J7xCmNMtiW3QjznMchO4vuS+iG
Q9iWxxdNVIkSIvBAiQrRI7VIEKFHSvCpBFEKkCVjoaVIHeOwuFYohqjER0Z6WhIhiNRrpisZIEQS
1EMiUTUTRNESNfuogMRIbjXcgPcGm9eddOhznOcpzE9yWSL1hkMlsS2OI4ji6JE53ioktajsQEEa
0aWFIQgQJK0EIiRJQyKWIQ4GevTrR0yJxIlcgQyCBdEEJ2GpCGapFcOkEEEPYggSoxVFIgSqgSDV
KRCpFZXUixA+gJEifSAS3JEvcl7kk9EMh7UOLoOlUItdCFUjsQUoiiRqWEWItRFSCCCCDFbjFIIE
qE40Ms4gdZVgRMBoasQJXHFIglp1xYVIiuvRBBpSB0gSkgSIpAsEEUSIErVTIpAwlZkCkhkOhy1E
JiaciTQ5Q04+Nkn7RpPGbS7mzgVw+1NCROoS6nQIkCBFEBDi64CZkGUQRWKwadWvUiJIHRMdIMU1
IEmRWCCCCKOtyOiKQOiRcgSHAlahjqxCIrFI6CEEEDRBBBAxBMQ0LeHKOYTayY27P9jAGm6YtQFb
nLNUktrdiizfsYUO2Bllxol2oGVwcMM70YBHsR9oDS8BJzciuJWMjVaV7CdlFRAl1BbEEEGpr0Sy
KR0ITawxm9aQNMwFbpR1MzyNyOhMmSMgRoR0NFyetCKQQJVjojogaEQJm8mBzAkOhIggTEjWSE+x
LYT4IT7oSYKdgJ5TbCc8/BqgdjVvyFNAnuOxg84IFj4HMtnvCJWOJBvqLRN18i/0BaDiW+qWIeyj
cOwieIwYGuLxYvwHPyXYeVew89mqfYBqmdehbykTv4CAuxuGwS8je5bD6D1Aj+xKS2oQa0OryQRW
1YNKw56X0Mdc0RFumJrArVmirFII64LBYo4IRFYEjCaKiCCGQxQomT2J7M4RMwzNC52GPxC/zDkS
JyzGkfUcLyM5NPeQT3hm8ISrsuDHBSS2z0vBvM+RBK+RA8fJE1Mfcy08Si1V7dClatN19BzhJ4Ob
kayauky/eYkaX1DhULknjKOTRTwY6bctRTcSMtjI1vDYwkjYiWNUe5FKiEj8o2bt5HxkvGjaLwDW
jDeLByfJwR8BIEjSWKtJ0JG4YocgNriX0voSo2KkVj0o6H05RAkjQ1pCGiKoa6mqZoghEEEiRFE6
CGTQloeQHbuG3aZJ7cnDFi6kNYSfrReJwuDNl8F02wu0kckLcYieFXIn2PloSWtew0zY2hCGw51B
G5BEL2M7yWhmlNtCNZHZc7BpTv2LTUyKOyR7yk3PiWPQjQk7ph6bwS3GyRGStyNWdG5/QEtHkRH5
GrGhX+w0k1SPUpUyHgfdiOVeBt1IYTZkuSTOW6209JMKU2UNknDt6Yib0jqfqwRVLpi1EPghsaIZ
IVTcI16CfqFiOSWg4P44nvIsiSbGb6I5lAn5ink4hCS7pwJesS+oTyuDUiXiwRr3NjOwWIkWfUWd
zYn4eR3rE9bjfkbLJkNpSGqbqkcXZmKU8jeZPZHeux3Xcxyx5jfuOe1lsP61SjaI4buGJyAbf0Rb
xvCHGG7YKXNDFm0UBuyz3J3CWQqS68CLBAbTT0Oy7ekSJblgbeVtGaTULf0XNJFBosDDpcRWBpUR
gt0RVDQqQRRDEzJ9FCatp2jJNBO0pE2g4QtQL2FCerlEJ2R7JDASWYDFpI72tizLEtBwF2MHViOZ
Fovq4kajVdi8sDS9xNZNiRyi+BhvNhsTrI9wadWN15uc18iD+4P/AHBUBotT2HsGPBMePmWYh5oT
BtJ4P3IbA/tH9oYQRQJzkkbCJ/AY1v2XW4MJucbyiPKO4S3QfIdS5eErkIXhFxzG2SyiEpk5bruz
fYhicEgu0PQTF0T0MCYfDgiyotlBc5N1CZBIDxQlEonqelN0cVGRy09BoNKtSSJsnVOI42T6CboT
6US2hbdCVETNcQqMgOAsoX7KcQStEQrQWyiLVUoyVDKA4jmQnyERLoMsR3HmPETbkbGRtKvpSRaT
fU0bQvcHMb2Yab/I2YTyPYQtpFqw6skOFx6Dw4Jtb9zkUhEUUE8krckWE72JPwkk2mofoMUmFiju
K4ehBbxjuaZynDuRRikeUioO0xVoADm6l3x1Q1k0NZRFMZfqF6LcrwIRgoaz1Q2KJRRGN63dlkyh
ln3pZJGzgEMdBS4GtooVCFCKlHcjWZU8B3neO+j2DSYjcmJ6ocAeshcLBZy+ZIk9RtuZ1jvy0HMq
XkDblj+gX3rEelIkVZrSSB+Ri9Bgu81Nz3SQP5ktUH/Oi7makgXdvpk9bJStiEYFx6DJbDv7l7c0
gm/OE78AlCVo7vvFHV6uLDwOHK4jKwnqRYb3VJ7dYEUJobon0ZQkS/iOcdrycD7lyDZLEh63pqJp
pMWQ1I1i00omQjhwNDvO4abigGuwSHgcckwZFpNjYzF5ucTMcq24xqwh7g6Hve0Rlcgl79SJFDYn
JIbE2MeikSmZ3GamBd5RZ3nCEDs3oIeO5XPRMm3VGNQzYPaiZoSFz7LKSG1W8r0NGY21RwLrSUU7
cIb8g2/kHkecu8tkEIjqlE35FiaMeos7qiEdif8AoftZuqd3wT/lOA/2C1h5I7D9SOb4OacoluTS
aJiLix2KKWopyMcusHMOU5iRLozTGpfNEyRqE+ljLfTkWVbpiMv0rAKSbbGPI2yTI4MxQvYywnf0
EKy9yaL8Regl0m9UF/K3gei0JhSe425OXJn15t0IakkqoyKc0SU0OUh7idZr7EPY4CGQ90ewtQd7
J5nCxkjsJbIcJNqPJMwI+MrA1Nprpnph7HOOfQp4XGJmtBuCO5LvJEFKkDQhrA+M7T0prBMpBOMd
yP0U7WI7QrM3ELJF7eiyzsDTPmelcTjQZy5aCTba+iSEpRPn0UaC5Kk5+BehPS+m8USWqEJtIlvN
WO91EdhPgmcxzMkSSLonph7MZ1F9sQhIKEq6PcZd7O6nsKZbQnohbFEgSECGxAWXRUWgYijMcaHw
eBlbg5Jr6SNRaQkZj0UJaSLeDSIiSMc+UbXP0FRYXZVX/Lg1JTcUWz0msk0klEkk0SJZJJNJqp2s
y6rbjV2sQ9jgOccmtieXjlJ70obs7hxiJxjhVCMi2iIyL/WGjKkGih7x3B7kewWbKds5aLm0LeVZ
LMkGw9Aag1EquPh6kCKj3r01JkMey5+khYGioz9r1JRJJJJJJJJLJMsYnxR/KnYdvSvscDOcJ+qk
TNEdoW6cpIJC+U2IYSxBqWGybDXYU9BxgcZT5EbJo00nc8F3Aei1HijLmMMXECRHJiW8bUNnl0no
nFIl2L46kNbwsTlUs7CGF0RXGhJZl76fpyLLkTGM3/KmqfMFTG6z/wAMPYTcCVpJoI1DrFjS6fcR
3ry2SLQWwoEg4BqQghuh7yGYtmXcRYCEX6Y2RjUbC5cznEnluj9JSnPRoW0ES/TWHPHlutDU75DR
Y+aNARIiej3AQYk9PY9OKmB/+e5AgSzRkC5oByqwcImm9SluT3FypXKcQtgg0CBxIg0RHYSjAuhA
yCCyu2kNNVJttHoodnwcA5xzxt1jdrUbfUlQm2xoSmTRLBjIn0InEwtI1q+Rp6DlNJlEDRY9pkid
vQ9sv6ejCbsk2MpyiTgJwxp/4nosEIdCNNdUMhkM4DhFsdHvKcguYjEVPJFp0wiIigQJECB5TOCM
PIPTHJOYMuIdse6bjHMJdQ23YsksQtehNOqEloRgn5dcNiKXB2EpHiV11oYnMaUWRGlMLrknJxfm
Ji5KseWXYakl6DM6Yp2GLp+iicjJ+i1IdOyEWwCLkejJxHAcZwnCJ5fErIkSt0J00Eg4zgOEW0JN
hbBCRINCEQ6o4VDzVnkdBhwKpG8c43cQ8jPQliT6Bs8v0IcTHoIalnaqi0hdbw8bdx1UrNmdBCFJ
ZyTXuQ0P7bdSdxp3IsUi4gvQ7S/dCU2ae8HbqJHEuFCrNUPZ0ZsZJTa4TE/Rn0J5o8kqLEhNrRiZ
IuuwpZSpQIDgIIxbgzPcIB6JEfElscaGXMcwsxTboeBrQiX68mhdlV8OTJ65E9iiF3NuNRIG7jWe
tMmFfJgJMAhdXYt5maaZuRNUqDTEW9D695OuBkq2P+P4DsKnKl6ZM9GOKfi6FaBNhtk02USZRs4G
Tu+lIPQcYvkIQtx4nuSJJJrD9FPgXWsDa+/oxrINibbnpFpR7+isS2tRX1vWPoVAIzGl/FFXc+jP
oyMY+zGR/ii/4UyV3ECGIe80E/QnemzYgPiHWJuIWESK4kwaf29GzA0pCyS8Kzo8qs9GoQbSyxsv
oPU4SF0KHl+g6jcXJl6HAJIiCILNb0NOgYe3o32EvmEymb0bT2qLNJf8KFdgWHBAORdaEhF2NCyb
RDShZYHbrXAGTpPcZG1HvRegj3RP5qnDkhcNOTO1uoEbQ1DoxE7IeBTN1ijwySaSTRJpeixsxw59
BDLWjQw/QdV94bXLeBmut/UDB2F/xYVlyuPSSyx2Ez8Y8u+g95jUuiPu9F0ptUxZ9h4RZeYF33bD
6kaMkl0GSao3otmsPfokQwngcXS3SRaH5EiJokLtv0hL2YtJzfcT9U7skH/xtSBFxT5H/GxjNKfT
/wCNiYQniGL0Hcad02brzGy+fqVHoxp3FJU2UXW1LlMWUdIzHLdJPMZPrkjr2dGlBbVJJrNUPJJJ
JJJ2MT8Aiuql55VJFDTTh1SVY55zTkJ/49BZXuLFHw/43gx0Zl9Jjy7Jkeyu7v6J8SaFjvC9CB+O
pKTL3GUJN12PPoEicUi1F6PI6J6JFdImiT2MVTNeDhXknujsjQ/xM8qXJ8kGggbba9jlg5JzieaM
vogp316YucBxf8bwJK0Y+HoqroRou4hnMtr0Oscz2HFwmqvHQ5fR2nuHKRZljmSH6B7lX5IvQkHs
E+XsSRZLZehNi4lEliBUknquQ9jgIfB3I7BAd6ltH5oz9ykwh7R0STSSSeiS5Gw4hP0Ht0CwiEJj
EkvDXagSbI7KJeglLF02Qgm1cair3hCtFIPeSrEu2LAw8+jRNAiUXwixHdVJRSJzTiGXLSvXAW8X
pRCVqeCRdJWwa8z61gapbhf8FxTLIXNIqGhHLfolbHZRI5CW5JPoRoItUYhpkWDjouOoT3O8hvT4
xUQSijUIkShz4ViMmfDQZmOP0FSNSM7MXc9hk3Kril924YHQqMimI13oJLFUEX0LjaLno5H0P/os
wIZ8D1JJJJJJq5L2oywDmGuqaXIexPoziOKj2xPud4p5MRUyQdPtqWkBKRoS13Tw5diXU5Bv1OWi
XR9WfQdWF4aljKiIXRjmaoNy6IYkA2XHpW1ON+XG2/RyYyP+OJIoSai9rK/4YZwsy1iFKqCmbA5q
3FQFtVVQl0J3CXoI6M0bj3VBge3WD3JJkcwmwt2CIGKm10Lyxk9c+utaiS6S2R6CFkWFo0f8WDd2
Qm1NbM/8uc0IjS81wEu4adDiOEVWJ7iK3jURrZOM4iGwgkIKTRkob1tEWim5B7RxstWBQ6IIIFja
Ow7mYoJZJJPqJwXUzsKVFd1E/Vn/AIotRJQSsiR7L/ka0IRHXMWRqH13OA4jgOOnMlud530WidEU
HcRpVKFEejQOFHhpJbDiVWN2WHvjkZLfW7LBOE6y3gNJb/5IPeke8UTJ/wC6aPy6Mms9aGQzgOIT
9B4QtIoxbd0NKCSsRgqj6f2ZLDkRxqBo1O+PRY4o4qPINmtFzmgE00QJURy9jGKP+IhkGgnKpd7v
+SSybi6Al6D4zG8kuHCbLsnLf8UMTDeEhUY0SYG9KRM7qIEdiOxxC2DiIHEREVQsSQID3kPfOYe5
TewcLNdIRFIIEJc0TGBtyxLf1kMOCAdXNWi/5YSiSr/lTaaa0EKaGiSVF6D1s1QqjpmAiRFJE6XB
SkSqqhEiJdqS2C0TodIRajuFyIIqQJDJZGse1QlRZ8wzPfHMchLJ6WhmgzKEqK5CZP8A5IkSSP8A
5oz0MkfS0EFrIM7hChzD9CKlZTkn/SbZiGtMbCYlcsGfCnEQ2IiUyIUQQQQWIHOc9PlqT4jqS985
zmOQluNm4jDocJNtjW4/45pMkj7tP+WKhixQvRSbT0URDE7DMD9JNrAiEskDkY3PoSIiGPBMpii9
CRYTIeUzJo2qG+MnfogzBw5BtuS/XQ8LlRjGjT/kY+atP/Ke2qZC9G2lY5CJLp0j9CcEC+CRDJ59
LME+IkNSuEhl1oZDotMmVRep/8QAKxABAAICAQQCAgIDAQEBAQEAAQARITFBEFFhcYGRobEgwdHw
8eEwQFBg/9oACAEBAAE/EMCZejKJGajIVlzVlXjooEDSZblUOkiRdTsqAYuWMCYGEtDUV3LTZEhi
iisuyBDDXRdRY6UDDbB0bcvUtIpml3BpAqKGyOCZqQEI83GnCOEGIUow3ZKlUV0l1zjgM/EOMxMQ
GaOsTFE4QpmktQQ3Oev5RYQjKdS0iAqFGZq4ujiFT0A6GA4iFxK/UqwhiY1ZFiMbEFedGJGSFiio
6yjFuglQjcKrvTNvvoCqgu/SIEN1BVUDN6HVUFtUxdGjHiZRToL0RlxRUxVqcGVqZBlcRKmd9AVU
QCEcWKjEweYYEUDeA+0KGZfmNVQal0BXVokSFB1axAhVIQqWZfTV1aR4RSWiuNEMM1mPENxQi7i8
WJv0adGIgW7JRJfSXWjymqDEQWViOIJdwlamdQOmNwQjC6mYtZitm0IaiSCRmWOqyJjk5ms0l4uZ
SYyxUQy8IjbVzM5iUZiygzKFh06NNzMWIP1HbMIxnqDM1fUKxuE2XMnNsU1YAEVwUHSGblCLEaNS
uVLilVMuhqdOsraKnDorUbUFf8O6UQykKqYC3QLI33ARhLYswixqssYdAlJbcFuD0qBmo7YemkV1
mnrDNENdNai1fQGkVYlKg5JbGGvEWIlmVBOEdP1LpqcLCsrcqJ0g3K8w5mYzaD0Kiy43ojDcCKKL
oFwSzfQGoqJovaBuLNHLDmxURqNmZywJm4oIcR1FiixEvJ4egaS9Zz6w0jCmBiv1lAzMS4YNX0rc
Gm4rJUo7HQtx60FxkVCG0xILUWZ5je66aQagKjOqWoaSkcFdFom0xJlidKuEySpx0OlF7l0nuBi5
zC1xDUZanuGiL/DlRG3eha4zDUDUdzRJchYrjWFYZAQjA1mBMwYrK7RXOSiKkohKHQxlw105jG+h
tjZNxojO1M95W5zCOLEuCnoXE3EilQ5TL0QghbK3qUgMMBLi49ozBeZZSO4Al730lifgdFgSpFLx
FOAjaxVGXUEMrqBbwIylgkxUZCsnMnSyMRYS0fUTcXSXJVOVMSaSsS9MF9EkZQFEScyUVAiWNQNy
lBjqXZ0trUzGam+nEzUuOyGjoDcU4ZuMsg4OlIeyZmlsPTid4HdxNJHUIVNEDCAtSaRBC1iJzllQ
1TIYqCWDLBm2akuEJDUV5mIMQKziNxZUHMYJmMUbXoY6lolnopFlsXfowSsiGTiZpNoteoL3AF9Y
HdAFkeek0qIKFCvaIjQg/GPoDCFfZAxFmguNYqhOJt6XqagsOILuXsA6Y2xcwJGcvmZzpM4pfEuU
qIqHXtIcZgVMTE3zNQFU9OapeWqKkTqEFr0WweqHTSH8BwigQWODcyXom3WGHueIIaHpohkyhK1c
pFFSrWaIiYsTSIpBJ6wwaQJjKcwYkqigkOgLMpcWZQgDBMG4e4LBliXNZv0Ii2JwQ4YcvUuuhLE5
EqCPPRdESxJtL4Kb59FiuVZQfCG4dXMJq3CsQ0YCF46C9ugw0M3dBUyjAammUzBGkJkzc4uKbIqZ
inHEFO5iVqMgHMQkAZsiVZSUijg1KGUGJtNI4Y9YwzFwlFwcyzPThHZB6WdMQwXR2VGLpempZj6a
2QSLETEYmMNEW46it9FkzQlSToNKXZuZo6FeZbCxWMyScw41LVA1EjBBxUWPvoNXKGLeYdFqYltS
4uhiPKGsNTAZd6oy552cw2RpHh6BSFQLmLi+jZdxIwD0qoNFNkIcOoBL1FLsiIEb5qzTE28RhF9E
mJvGKAJpGIXFbXRSAgM1tFlM3PeGOiYRg6EGKhUESbixAmUxlahuEOo20i2dCDjoOIxLhdyouYZY
VioWQbhuXNZdAjBZNmAqZXMINS4ypiVKsolKQbJQuVOiDGCGibaaoLgmIjpHmbZk4qJZ0wmMzXQr
HUzNI6gnRQxK6ZXOISsTX+Qu4qmaExJLtUqbfZi4uXcciNZZN0RGbAsrEVGoVKmUXQI+XabTcgxM
RuZVLnQkxDlYiuhdxpKS6jtlaIiy5SCyBRFcrILbc06NyxHlHQ4jLhVXFXt0dIKUNSpyroF1K9Be
IKD5imUdzWZiuZN9Kt6KVLVVRCyzoE5hSyzCG4IQH+CqqJNGC1LEuYgzNpkQRFMWBQgUHfpfErcI
GLBLoRkqYxTRiAfc0gqRzCoHMKoy4NzbBdQ6wzDfTEdTMLDMSumL/Bix1GVklzO5RLU395xHU3JS
9yW5l2MCKXIQuJWLmZFShg3IqPM0RNJw99Ld49ClTCFbmkOYVUHQEyg4l4ikUBUFx0MoKvHI9AQS
1GaCS76IuOmWFHiVjbpxNyIgzNg6G1/BmEAXMzcAmhKhiJu+lUmISpqWtjaJeppL36jZKm0RRBep
z9y1RuDoxHuIjCYQIEbSOT0LZMcjL/eaQWRoQlQYuLPzMxWTCKPQbKmVdFxb6JcU6FxfwXhY2S8X
Q4ahsjFrL7xwJZKIWMlHaKIsxdMCjoqZZHKRB7kOOgIQcQZPc16XHRZGOkW1RFdJqaYMvFthxGGp
m2D0h3EJMWnEAJZUmsbCHCGopSsKuk8wCVHkqGk4IGI6RZi3Ubg4uEYdAEaQI4hFPQ0SoRrG3Mo1
GXEFDMiUTSGGAwTSPGJbMtTEKdML6wikUqKxUymXl4omiphBtRuegBpKQVUFpK6TlhKVuJm7hioV
RGmMdXLlxHcR30DNOhRZjKpLqC1ZgqaszAlVBYMUlsNzBSZJgiKjUI+paJWumSe4cStQ3GNE1zSB
RHQQnWawV0N2UDrTJxF3EIzCVUdXhawVF6dTBSQbmMioTumMzmAnEGIrWFmsXQlLmXQlgUbhEZGe
ipeSNuJSKiF9UIEKQFzUJsnKLCKRSpshFEjkRYhguCEVthmc3mWQZz6Bx0aS9UTUG5tiWIQj5voa
zBLRsHNzZ66LcQoOMsWiIxQWjpTGN9pmo3Ubd9AUFGhCjcdrcrhAMrGoGBi3E3rtNylYmsNzWUrM
YGIl5lDLGcRlSy19B+2LJhsm5AwpsTSMvHiBmVEK9LZKJ0xGXQtwI6nbAOrM3FQdEsQZU1TIiswz
MZIroDxDiLMdkqyDrCpQUwgeol2Q46DGrnBC5a5ZgQguMNRmMegt+neYmLBFFYE06AlI0RhHcyek
RWMBNEM0nO5UpsmZVuoqRW03ikvZMNkFmEEgZgSiMHRjm5e4kAh0OSHcFLo21BUmiYmVFH0iK1fR
U6F30Cy1QHmEVV9Bmgx9wOIqhpM4uAjsm+bpaOx0my4KOh11dXoDUVJfeVcroxYisJsTV0WTXQW0
JiGKz0qh6dY8pUIgI8RQXXQLG4dkHEMyqjqJDZEqD0A6CJLYahdS+mtJ0FMUgR/qXBqa3Hg6efTI
+YXRApKrQadRqwhVll3CQtRsTAo6LjqKFGMInykxGYwhHNoLag304nMT0Ubi76XslFzfQWgQh0rn
MNwAFPS056tKlpiGGNqOXo3fQlFQlTmJB9sUA0goxqkIBLpFVFnHlNULX3GFRFjpY7IGKiqBV6ru
S66ZSiXL3BjEeBULcwENqXqWMBCxUOhCqmpL1iBt9zVSsS4S2UeZUYoHnmBiYOrLjzBGGpdzBqZ5
uXBgylyspeJqd02SrRIkjpDUcWrM1hlwmC6l4eoVSy5pHFcGiDbEjWPExtFTjwmido82VKANQGCK
0xMEGWBzHcdSqSCVHRLESosvrZYqmUZlubpWJqOghDBFYytPUeWPmG2WBEcPRXO/4BbDVTQw2RL2
wNGWUhpDct0TWDGZqpeWlSoEuwpFSKtQsWiLpHDMY1iww2wRgK4gxNpeo7EwcZQMpYqEsSsNTK0Y
+UTZrMV9yrqZJsmpniByRPSQnSVbMmEiL0auAhKuOCHUJXTWMGpe5feWbQSBBmUr04KDEbzDYlEI
EtYoIuOgMELKms0Ki4qVHQ9ZhJUKmILMJxDQvSrmG8x0yn8RW+m0KqC4qY6i5eu8w2ypVdG62HLp
eW5omhNYAZlqEoQQGoqdOxmXiZEcMKXpaQRxEg1QwdAmM1qOkxZlRTC5oTEdA4nGHBFalymprDZL
NTAgzToyjqWuGUJhLmlQYjqWCUFxUSOSZ1gqNsxeawlymtQMS0WypQYrjaIRxNsRgSyXCsZh0rXQ
H8LeKiNMwur1rhDEERg8kHGLRMoCyVCw89FoMapzM0msdRj0AwuOo46LhXBS++rNwTJcBUIpXQQ+
UpUCXlrhgjNJrMV0WiOkmJKVKv4x0MFxGFrBnoXMsvQoINQYu4JfeMpNx1JCktcZLgDb56DiJZzM
SaXMmHHSy3WEB3E6gEIB0qmZU7JS0ZjIix0bpUGGOiXiuipmEj0gSWprpXRiYqIrpdxuV0VdLliR
m5cWYGb6bdBCxlWkGBmKy8S8yge4lhLBDcJxcwimrjxcrXEraCiOWYIURXFqYuPiaTyjx8StRcRW
Y5pN52msoSUl4mfSdXoaqKzApegcS4o6dmXHcOnRVFVoYhtg1FoiZcQ2wMFGJUp46A89QQdjwQtE
oQuFQiRRq83YtTSbLNYcQ5iZrK2yyXPRRHcGHXaZtBlHLJdC/fRYKXHaXgzSPXQ2sF9BNJKWkOld
HEhVS6lPRaSrN5z0NQMFwnE5hU0zWbS5qbT26Jo6PFjsJ69Jwx1huzxA2hyRPiDcCawqpluKmORL
JfQMRvR4ubNTabek6AMAxIqJlWUqoy9RcQZlRgNzenoTaZjLFyyouIkqYmkYEX0F2zJFmpyRlKJh
KojuUw1AlyFiQwAxtEdQFiOKWzNJpFoyhNEmpBNZ6yiCmU6FG4LpldFmXcakWSwXmO02fc2jhHeF
dDC/cMNrpWiaYuh0zdg4gXCGlGwWJx0R6EqW9BmkRgModKCRrUdS8nTSYD3DUt6F6ipdmprLshre
Kqh1UzJHtOyGIax1qqK1TKiuJRJaPKEmaxFAlusCwzFDm4US0LC4sRgHQJG+8slQRKrAINwytYjm
RslwkL105McyxqUUIujvSsAEIfcSXNsAOgZgwgo44hlN5hSaQah1c5e4aqppAt9IYm2ZkzjWV0NS
WqItQgSiI0zgmiXEmjce7vK2jCIGogWtwNRKOoXhyDhm0OZlKYw3hXQUnGaVEq6lxai3Eal2FRMk
KOOmK3KzACbgSsQNIM5IJWisyhGGhg5iwI21Mbk6GVyG/hBCLD2lA3DcCoMynv07onUdIRzImjot
yBohuDlJQhi6ZduWxvpnMjqZiEroMwIRTcMM0msVMFsKDiEdx7RhMrQqDMRYqJnoahGmUxKxAbiE
IIGKqSmVTDoMopZsFQuYbr3NcxrywghILCGoOi8a9BmHBGOo1CcbJQoos9ylOSU9xW7+0saIAVnk
xMRlTZfTsxf6h/7CL3SJR+tS60fqLEzXIufC6ICw/wBSwrf2m3PqBv8AJFravEW8flMefyJWogLD
PLJnqWrkzI/THa+9E/8APlG/tymv7UC/yp3S+JYYpw9p8P1KHLGcRrGqjNu7QoyYpdUgLXzBieLL
jjmAw2VgPHToYUdcRqgNwGHOWCBcUgIzCc6jHiNqhjKQjCe2ohtceZXKizFJVXJiA3ZMDqZOoBhm
xjpmTAAolpVxxGMVXQtKJW4wWMxxWDM8F4iO0NTdG5ncqgMDC1xtbi7gBAuCoTcSFXK6EZiZuDMO
blGc0o5RUs8QbTYiYma/hAcOjeo7YsepsdAQKiU0xS4CFUzDCphIWY5UTERNxJRuXG7yQmnUs8QK
miYTG6haBtqY7RC5Vpth4wo6lHM53EXamlCClpYnmBgiUytywUCNJoPiXHD8S238coUsekAfFtDB
i1crZU8m4VhbzXogqFrXdRdx+WckCvQF6mOUmSEQ3xK5pI8KAUxmEEO87sfQhD9CBwfISXYp9SAX
8TLtg1pEH3+5/cZWm+16jQPl/wA01U4xPbBlE/LAJd5AeoJVqbWVmcNjp5JTljvBQrO3f9bFBjea
BLMowFq/aKwBrbRRw0PzOUFdqI1F9ymi2QpowLwxQA8Mi4z8lNPRl9Pv8dL5ZXOLe/8AGLqBe3Gs
32EBbY8hjWU3zH/byo186lVPjS0FN6giZq7s0Px3+SCw/b/nBNfXecon0QKqX5s5o578QpB8h/UB
oz0Oa3cIlSp7JXK3+ScL7iY7rHw1A5WVKbsjBKi4RcCPCtxxhQ1EviLwgqc/bK0jqVjG8I7wGW7Q
ZjXM2R0BGCmGEOhJySnoGqJyziiz2hd6mOm+kgv0Am4UiIKN+YDHQ4bgiU7RkQsZIlKmKkbjTl1H
FXmPaj2MSxVwF3FeM8GeHOyLhtpAsBEF8Xedj8p5rlby4gHBJeE7iWJcCzLqECLoYDhOF9BQ1Cyr
HYHJAFzOaICr5ig5wVAC2Bi5Z2AmK7LLKgiF4XcvN779KWSubxG4HIUpwkAHLBwWDnURdVRL2SYK
S5nHWvVxXCleNR5dPZM62PFYZB6EVKmmbdCUGErwTTbT/UGXdL3/AJIsUFRsC/zFgBeP8jAK77r9
oAKL+9f8QyQg98vxOYLXVH+oWWA8V/hmyX93Si2N5cvwIFt9yqikw75/zSgJa5rGvplcspWIA00q
yz+mU3xN/wDUnV9bP3GG9w/sSyw63T3FFIXqwiUv8Tt2rvB1vXEyiKPUVSkvdI8hc2QoWMdMQbjy
aQ6+JFp7zuRGg13YONvUkVU9tNX7ZzX+4c2n5m0NL9ZQ6BPiX9KIc/qEE/Em7W9Sjh5qx+0PNRdR
ikWiyXj2QPRmSqsXeBLc6wtsRN9so6irYw54DcxQWe6asMeAYVrOASWXK8Caylc8cG5JL8hMOTM0
tgkL3vcdMMpIHcOzL/vE7wlRwLizVbmGoGuUjS1YHeGOGK7wB5YSC8y6nbOwpPOgq2nlQS4UV/JL
84Pcr/ragZNaBf8A2RrD9kAAtU8iNbn9ErM/GCH/ADE50PsInb9YjsXshLE77KBcnxTKaP3wfgS9
UNnKQ5ZwJv7obEqK1ZQy69ko0vxUtN0ehjUB+CGG2OWiDpSKly91R2PntKQq1lUAKfktAGw+MJeY
u8iEMJmhtFze4It7GEkMi5HEC27PjOKXwg1eCpMrwjMM4SVZhIFqCUNwGRYO7M44l7XECpb7h3io
5CwcyzY4MwiZSS1C1svwV4qJABoWVtXwxuwDK3GGphauIWpmCYiDMoaRTIYJ6gXfis2nik/4mWbn
GUA8gIV4BFuBjhES6fR/phxn6L+4VKD3+4sz2L0P8QUR/wBXqA4vhf1HIEpL/kqjxn8zpG+HErIi
QPkTwPmnIL9wAdfTINeqMy8ki0voxv8A6oU1k9TJanxFuH0RUP1Q5JCXMPFTkq+I3svxGn+qCkQJ
2hLtixtMBeS4/wDczxXpQT/PmG7L4lIqJUo+tE3HsRDPi9D/ABHU8dowvQvhHCYvUO70WAFle5cK
J7MAX8VoXU/eI96VzLMKzvUNWF+ZRYVUKncWviG0DejAzI1ohtbaeHlbR8Mcgvqt1G9fx2i4J8/T
8x+o7sCs91JVT80ELJDRVX3UbCn3hirL/aGBoj4Ah3R8xQ39oFz949r7wWhX2SnCPuDZunLcyaSN
xZn3KuVQDgh5URPDK3WMFQ0jV+kHzNIWkXGM4ol2pKcyNX5pduSdCK8zT+yHG/mA19k319kOAwOS
EY4CCosd4O+Z4kFPGpRGyIoj2LiuBiYqBG78MsVePvSox8SJVovTl/8AlzDZzteXFs43DQblIt3u
ogRdfEEK5epSwnsuB6S1lCNQXaUK/D/lDi/gUhqtX3UVq4+NquaMpV3+jUACh9kyZ+CIbZAgSfuV
9vdNB5mM4rZJhA01/viCFnzU/ZLP2YZaPmFb+A/yys/13uGUJ5pUZCfYTJqzEXDG4HclpuBvSnZV
QYGpmnEqiYbhq4KTeYCqxYpcus9KuJnEuUy0cQpIICCC7miBAJjBR0dal0Sr6NiBFNSsEtSWHMHc
MRLqDgyt2xGVTJqPicxt1cK2zLLytlIXmZxX2QjtfuYBviZhUUAvEzVcUvIvoirdn0IdyezMqv6E
o5fcha+1IalR8/4pfV+uWEGeVFoHeCAu+o3JftQXZ+OszN5YVbx+Y1H1cKig5qNgB8klhr0JPw7i
x+yLa+X/AARUwH3n46Mbw11xBLYdPbOb6CGQPlmlLLPCsuSrEY1+1DCZvcKLfwSjr7ZaClZoa9c3
ds1PtfFDcPjhrsH6nyfqTbQnqLQzfUnWT1IP9a4mb+gsoczItM5TT807zXO82rou5x5h9hAUv42i
S2h2T7ued/MTdMM5kdiH6IOZc6ySgyhh3oNCHxG2osyS3FZgxTGa++LSp077hn3QtlTdINoTvslX
aQxwI8RLMUlTmxfMOMxYhVkgcI9LAaEdoYKUXkY6X8kO3S7G8VyvtjwKBXDezC631EPf+JA23v8A
qijlPj/OU47yhKRvEPcslRDoHQIzjoUizLMBm5cRYLOB/DN6lPLLXUshxLbhDGGY3Cczk6AI3hKl
Yo6EGOIcyjpmClVucHSo3K+oFwRQGzoF4JQlZrUvYTMNm5q4EKwuF95WU0EyyouHAglCFGEbUIGj
7lRZfcjZAFCiBQTgeZbWUoIGXEKykW+adxwMu4kWr1TLDkQAoYS8293BCre7Zwfkg1ma93MEDXfM
56xsXf3KtYu7Jp3u6/4Yrlf3PFheYu3ELGP4Mvszdb+EJep+Ka+j9SpaP1JXp9MOKj9SY2M9uWmv
7eNFfKlbovZg7R9y04t8zWqd5cSu37GJ73sMVYnyEDWvimkfISWx8MWtelD/ABiWz+CPdb1eUM/L
PNTkAXyzsKK6zbEMC9jHeKPeadpcHthS1XsSJcv5qB8It2IexDj/ALJYgGUyoVOSUyoAEbuEdRzH
U0S2+lw23LZRywMQMxyjHTBqBUCuiLnoGXc1CrgukHc0WVS2/MKqXL4lY89SiJTxDMCKd4Z5lYYL
0Vpcq3EBIcKxqrmLLZTyhLtzEg1Dm4o1TjUrdzRiJZC5spnNIDHmHYvccKYrKHEYEEtilPvcp6lW
2Roq2BZcoNT40xeQG7g39QDkpjA5GJXjiFUHnMsMHeVoMoNlpC22pWOLlLuF/mc1WI743qWFQso5
YmxblMxZ9QRqVzNeFQ4I+XMssLY2TUC805lFvEA30AK0WxqXmqiFwmm7ljTBYbZWP3Ttualyd1hF
Z2j8TynxG21fRBtt3xgCiHL/ALI7be5vMmK+X6MBtj9DFKIIcy1cQEg3LibyS49GIOIuIDTCJtnE
JiGzcWVjcvUPzGiK+l5JxLsZgZ2S2VO0DhM3DPTeIZUyiHMFIU5smBvpboZghN7qUIbLicyqiJTv
KmZa2zFkAlQ3B4CBlhipuCDDAxnMEqA/jMoK9QqXvhgGUsBsiMazEdRN4mVogJXaIvvNaqFFCUAi
balLtykIe7D6YWKa8yqgMjHUsJMOVqAvGAbYttC2S30rUumajZzMhXBDExhZsrzCmzBzHanjNQc+
ajfNQF+UukrV3C1XUxxChlOM1uUS5Z3qANrywD4m0WcJULgpz1EzDU5u4L0PicbltM7XMSMZJ2zL
mzdw5zEm0qE3eoblTLFh3hC+IZi0TJqV0vtAU1mAE03N8xMkve4wlBKYBDGCekeJdQt7ztErU+ei
tZQYq4j2xAwTU3AzuLU2k0xFbIqJYEqFJbdSqIGV04g0y8hMmGpzqFVfThDmpmm2BceIu80qoFZe
0vEKzTOe8q2m+hci5csrcYOKYJikSGEBhKNw2p7ivvAW5UxvErM0XtiWkgwXo7w1aMEygceZiqz5
qBMcbgo08kQGkqXgc3MqmFmzUL0cxgc2cG8sbMFxtmaoxCqmM+yFR7+YdxxAHdm5RQRJmuZZL8Qs
uYCam5yhSS0wzGYQhLJijzLpl3zLx0o6eJqXLsguYYEmGFZUxQQOlrmoYelldCjqZqNleYWkxbLu
5cxBzC06CWy5WrlQq8s5gs5gKwBYIpOUNz4mm5Zu4tsahUoxc0zSUSi5hZz4hT6i53ubZgC7los7
S1dr4hqDKQ4yNwczNTdhqWEZbwud61NtGoWuKhSr3KYoBcNrmVlhgxG2GSkYA2y7gS2sNTtCBljp
lCKgAO1wXQmrwwKLacRVLtqvUGquUdEwXMypgCrnNEXBpdyy1CDGHAAaWE6Nyl2rKtKupXDe57gJ
4IkaIaGycVeZWlH3BukrNGWFI5XroDZdTbBwwqoEOZWYdyBkjDUFYS6cTcFiY1GWmo3tgS4bjElY
vmGoJhAscQCaToFalTLCVBqBTKblbuBdwxtg3/EGZitxM5lsHMUHUYViUWPVZF2pA6DXSpXNy2Ko
CGCTMcdSLRBVL1G1BxKtvjoYld2xYB0nipdE7xc1KNe5W5TmiBqYZhfx0B3kh8qgydo5H4lVkluN
S28JQWTdi5fOZlSAkyIYstY27ERxiPkTvCGEqXiqhWYiB5ZhwUQp0lIuxl9Atjlpct7jhbKrKd/8
s+AcVkMAA0TchfXABZ3Dnwqo1DNJmMNlsvAZriAB3OYW0JV9sVBl4PxULh2BmarxEkSCttTXdicj
Pj5h+amXEKolNuo1TawFqUFso6JUGljpmauG5fMvEuHMCXO0zBj2gl9MCK3DOOilk8ziXBqVklTN
RXCOpRZADpZe4CumFhl6LMsIGIExx0W0SmFkC410Zvcamm5ZRMdFUwToBc0W9KuJYsMCVUpplITW
YU13gUMPymlS4VeahtwolTNEYmnmK3V7YBQOmXfgllMuM2+oeVSw7Q5JFCowZnd5gNvU2hWjcuGK
BG1n3MYzzuIKbhdED0K1ggcLyPhQipUAGlZEgXTE2vMCCPEof3KcmFKL0K/BZlo80S7PcoxunNx5
qCv1OJICvZKWvMzqzJ3Jrrx8SqVnaFXo8cgGYJLwfYmVq5ZhRUtb2gWrZr1LyPx5hFpq43WHiLjM
tBTiUPe43WiC1F7nO+JtCsbnKDmbXF8alk4myaGZk4hphLm4gjzBo2xeij0sysy0IbX1F7TM2l9D
d5I5Jk1C+hEYwEuoEYKLiYuVUsZUpq4vG5dupcwxzHdywzyZXd9CLFgXEei5ouIdS8UvQqpbAsrt
G6G5tTXES7qFVuUVOZfdlXWYlZGNhjiFFwTFZl9xUFhbJHNqgvEKx+aixlWrg3Uqie34lEAWauI4
zNKXM1UL4hiVKq265itZ2ZyzHNrijuPWhMRzxdqlVI2VNN4VzhHmC0BzCsAiwGyJVcA27KRIZuxu
lUMwb8XA8xwFEZQq6uPZcF6RAluxLaq6Lh1lKR7ITqilOcJqLEuDpfMb3EFs9Vss51pbuPfi4PLL
IY3lYEFbsmbeBNDRFZFzXW8PuYjsSo7hVwm5tlwWXLG4hmKSuhFRbqopmEMEGeoVVwhvoF4TJt6d
gQ30MEuhg4zORgsIaehBd3LnqCyooDBxLQgHeCS7Qt5lld1cw3c5X0x2ZxAJjMCqlOZgEtAloEG3
PU0lU1WIwNSlzDf5nKpijUyud8EQSsTVhTxAKxDm5lLCClNS2twYuNEL3dko0RLW4A7hBUlA6lCo
728xREyCBmeZW74IVm4tGlogLtNagvEvpULsmaIJ4QBtoSxCXN5zBdJoLZWWoCr6IUvhHwpWxvTD
Sm5M3LsvCSwPq+p/mfrMWbjRbCuNwd7jcofJGHQB7Q9sdUYgEb5i9ipb05gBqpeagjC3VdoS22Xq
ZlSujyy6uAVuBqB9R4hxDnzMTSxXpb2mxhiLLxcaiqubmuh0LGVKxPUZRKpnmUQ5zMkpq66Bl3gh
dtyrIFDOOgQpSbaIAtJAKyVFamAmCkiNzGAlZcwqMvmF3NJBGX1aSlnfJLh0ViUXctGt9LUVApkg
Bwlrmpu4VTK2neZKl4qINDE35jYFKzeoje4CviAjqC3SzMRlM0XCmDWmKhQAdoWYnOJTBtol81Eb
w/EbtNQ3U1D3MCDZDOECwDNsVcEpJaAiPm4vl0aXNsBt1oyfqJuT9/4Ij/ufiafHd7hVxzvqo1Ku
2Lj3KDxLHaHY5nA4UZjBfKNlsL2MwiBzLdVLEyQUTlgyze56JtJguiECcQLxADmtRc7gIzISvMWX
L7xbJdE4JlLzMjiDMxWIUsvM7iV0OnHQXt0t4JayqrM3Qx7RZg59zTgmYWOgTKaGDA7Ql10LRVyl
glNDi4US20AkVxU53B4ntULWYQd4RhcqicIAmIqnC4bgYxDjtEyzdxnOpgVmXBfeZ7QXZYE7KqIL
zcDm8XH6TALLMKmDQbgqVHHN4lYJ3m4VgSmuLlmlty56Qurg5MQdKmG44qqrNynDcdO4y/EKLjmA
vRSRYxDayC5eoJbSVOCa7QqA0DELpFwxAzCK1fPmCWxemGFZg5RlF2xrlENM7uMycpBrCwUFzGI+
GLVeehq7lPMxTNIFBXS6g5iwRx9RUqu2NwO0zD3MQUj0tud+i4uLNTFwywPYjfshOZWF26zL1kgj
gRaQFSxh7lwohcBDLA3pn+I1zG4e4CtQbvEGsELxLaxcTvKHMSuWOkiUy6lCFK0ygQ2xezmWsLl7
I0lAmADuM5q+h4i1qYgQZpOMJ0O6hLqoTVX0IwzhHCKFEMq54NR7GW8BKBTuVGxjlnkS+6GjMWXJ
UXGKmRZ3gAzczohclUHuUpgVmNmolbZA1Bpp3KNXE4Lw7iqN3d3BxEJRzcvRO1RPMQQlgSlGN4CW
YDbAlfPE4yzGVQaB1L5qCg7XDWy8Yl4tIbK0VCgeRhi07rNxviINVZAbuFYOrmbGFC5jUK0rDHqb
jwZa9G7gmje2XKfmZQuFxXU4g5mIKTNy7l1LBisl8nJKL0eoWxWWAFXIayQ+5N7bnZlWwqxAcilq
GG7iQnVYhUNsELvLqy6dPmeJTauu0XxV6IniBgLUrxYlpm1QmWrWIlpM5Un+REWtbxUWdUNlqNX9
0VuG9OtHjF5vK+HzDsLrxO+1F8qH3cLszpYryzL8xe19ZQx94K3PFMTz+oNgk8S0sf1Al39R2MvU
2CRUJdmBGlnZD2uZK+a4EBV3uLMUnNwNdXskaAXyiYrLkMMLvjBuDvBlZg2YSBtQWFKC9QLdePMd
XgglgEbSLI93BgkNXUA2qlimL7GYI/mxGUHK8w7FbhVdVmpnyEwEUpVEDY/MA2pAZgLktKcWQctK
RHN3coGIF0UsHi/cDZu7agnGP1UtlhWL1GsrHxC70mecTcXQB3iIaI0LrEpHUVl86JTYWsyZ3Bny
wVzzrolp8kCSpSAUeNsKtNMaIs0HMxFNEDvD3KgUJdziCqoIAbbl4PMGxtgoLmOagG5x0uDmDcrc
PD0eUuyXDAm7zG25VJCO493AihyjSme+yLy1bwPvhulfklZQPqdz6IAYfepyP0QZuvWA1o+sv2V9
Q5fomT+meX+IDL/XY0c/i4GZeLkXqx9pftPlg1XbzDflp4hDUFCmB/8AeJjz95DU/eQ5t3cgTBtl
pUQ7/wBIUZgpMPxH/wBpLsF9Sjb+0LlrXpgx+phd/mII/wCSW3+2HB98zRctvwhh/RJaGvpwTr4h
AvJ9WHl9Ur3+0e1+ZiYz4neNDFS+ocgQBibH9syzVXygu8XllWL/AJzMovmH9naBNnzqAIfcgLgP
ShYrbexA9fK2Nv8AfCO32mGOvlb6Ia0S+7vsSznV6oQdWPgkMdfKZy/jErRt9mADPxpDkPpiTZ5b
nj+qkBtOurgC3CtbaUi7+IszU8sA2Uyol+qSqc3pMQq82T/15F5q9pczIKh3qbeJhFGqk7f7jhyD
ax3ZLmZ5ro4hXdtgx3DapCBekOYDDATtlNW+ArX4UQFNK8/EwbYYWtYItfTjZs+lOG19hC3VeZ5l
6YPr8ic9H3EKEgHRLMH1TJV9TyJ31N2U15TFVpcKpiN6ZTEzcbZcFarEoINYJoLEjSeFB8U6ehzw
7kIf+oax9pxYuJ6qdiSm2n3Fg1muHiP3Ez/OHM5ZmFjtCGHPJ/XEzPoqf82J5fpnfmP/AIYiW/VA
S8HqeD6S90/UXZY/Ut0oklMTuTNSztA3KYdGHiY7y/U4gCJyCPZIEo5SUJdk9JTvMF7p8zLf52V5
PuYVZ+9O396DUfZn/QTtfdDjL5Q2xf8AUhKE19JP9Mn+gQJkewiduP1n/HTuP1zguV2FT1hzD3Kh
XpnLfw0R/wDKerb/AHW6CRcWXzlLb+zNc+8r8Q4flkDbej0GxFfJAyX8LA92HwQ/8UjTx+MHdHub
2GhgfnKf9lL/AP2gmUHuQjYeoeP6mJNebc8/OH8kA1Iu14LsQWh4SstWLfSUF/oyjYfYmED7EFWy
dmtkaLlOahUv3tgVBd3lhsq9BMKjkHr1QBbihAUhoJEY+RMBVd7kRX7FiAemaQqZivxECcuQ3NvQ
IJVnxnP/ALjuF7cLK/NiDR9EA2ozTOT7mBKMCo3KvZEpK5nfNbjdUwPEQKlchMM9E8TKIxQTulEP
CPaN9L3coE3sm8zJzJYqpVXeaKzcyGBpUS+JaOMVhKJkIon6CN/9RB9/RHFl+JotdcI3NfXLN/XO
GUwUp8RczjnDmRFYLXj9szf5sob3K2X9zDze4dn7xU4sUMPvL80RVKKCUr+pwYnPHBRABb+qOr6E
R3/UsNPqW1j9TsfMSrf0xwuKYRNsh2JNOFxWw+YvYv5iu33mfH5zhj+Y9v7TGWF33j2X3OUWnvNn
9oX1JmyvggCaJ23DnlS4PM/50SJU/wBQw5i+Im1+CCm0oa+hKEv6IZf1TNtDNhlzpnkS8m8FxNrI
cTEYSnbDCFWROr+yFcEeEhhaInt+Zif3QCmb3LmivzBtfcyoA81KHD96dj7WWLPyIUIfegePyRgG
tj70yjFq9Eou39EsVR5qktTX6Tgv6YAa+Zim6KkCkQ8jET8O4X2P4zMTt9s57vSIcxPFptn0Oqly
mVCUVDVXBbKshBkyxxHslMQGGDo76Vro2IDmURLxUsjEgMQqXuCUKFXxKagBUxKKZxCNxmlw18xx
G13KQFBdVGx++hrcRhVREuNdCQQmGsQFbzfSq4ShdQIDcNyl1KkUalJVyl1LmK3SJGq+4ZDLCoXx
C+ZW4hKrMUyqQS9QDtAv7gCKxthAVMUzpqU012gVZi2B2Sm9wvSRum5WGbLWUVWYBw5Y4sQAUpuI
NYGN26aj0HCLhVVrbLYcpivL9pUa97Zbh5RLNNsmhm6V3PgKi+5v/sQ2OEVWCdVm8EHRR3jD1Npx
Kb7D0OMOierR9y7Xyyr/AIIaKPqh4fkk5K+cRLz+ZR19pf8AOTwvlk6PsTf/ABZT19JMin+KJ6eX
U9KcH8MmqhcOlWlFolMzjP5lkLPUIF/ZAFZfmBf8kV4fWFv8KWf8CU0zr8/DSUcLPYwM19EbjV+y
IcB9CINunkl+pooZdiKaPnplO362ID+qWf4IEc5QLZZ3QJR9EqDDM7Y1Uu4BmJY3cAzccCxN4MsL
TEAyyditTFnVZpEBmKVLmEglRNSoS479WpbWISL7IgYZIbjXaJTc0g3MQmZkfEECC2paWKL6ElVk
RL5hFUxtiAL0C4FTvZKZkuBmRRNbjD3zAraGjEDU0hUdrBTabwsGiGdOiIhVwJnGBCAaYZIxhhR2
d4rAyqilraoFPuLtY5JKBvKwCAJsDikrKu0Funnpk6u08nQwdax/xKGP7ekdHe7Dp/l4nnj355uv
4ZUWXkgRVpvzCO9NFVW2qqBOU4qYX9EGiotoeIl4JlyJzUh2cU0IsfkqW8YNwkeQynjHtIm6RYTu
hbiTiE+WcrX5Zgz96WMn2ot3+RHi+9NdH85leH5youpMKlQD6puICwO4wS8/lq4Dzx5y12Jl2+kv
n27ktUIQnNxLu8XCPF9OC5fuR+U9xbd/a/zH/wBn/mNVX/695gr5q/zlRj7Zsr906nw1EKqvjOXa
szygGKVnP/Eqrde1P/E/3X+I3tfS/wAQFyGe3/Epf9z6masHcn2S8xMmekz/AKn+cV39f+cZYn7/
AM0U0FWcGv3AcLHj/uOE4oUtR7cLBR5XFGCxtqXgbhlzDupZ/wC/aAslB/rqDYJjdf4zdNR/rUDL
SG7VLfz7ltgNhiWrT9GIuno7YnGXPbD6/bBvdvtiJ/RlEbZgesyKvaQ7D9UG6V+EyC6eyAurzI2w
1OTHl1Nd/VEuxvU5X0QsV9Uvewdo7MV1qXBX1Skv8U2ZfU7F+o7KaJZeo1apMS32Gc0tYvMWH2OR
GHlYfdOI093pwGrFnK03cD9MU58QtsW5iO12hmPxRuL2dp+Tn0XaUuZ/m54tJRwy3ZlnmX+EfJl/
ZL7mPcdTZlgx7eZqpIEtjXQ80uzDp1BzxDrfiIfyPqW7azqXZwK5jEj0xgbebLH5ovCP1EuaJzwU
9k7MCWVL1NlKF89k48TxMS4YtmmL4IUDFbsxsqmL1LAeI42xVl01xM7EG9ZYq9RVGpaLkrcdkYlW
pjGWgZ2Qo1WYr+IUZkxBiWWwVVUbVHESlEJyLQje6JemZRcTVwTww6bhiVi6iXJAvEPCJvoVD4mT
CwPCzke8GtLz7iFgGSULiohtriIqcyTdonLS/U3H4JVofURNPqW8Zowiu5a1DHNEehg1YSqx3ahy
NPDmPGy4Jsa9srAmnCWjUBVEo5NPOYJbl5ZoZ2c3lS6PNp538saPK8254GUdvzBbate8TJQdzHVQ
jlAeUYIQO4piTyHKxyGsbLQPRMKFjELBXMWtxcNdgSB0XanCt+zCw78L7L1Bg4SpDu4QQyDa3gJi
OkXQqW/xBHM5T7gXb+5BhV3dxrD9eWIb+TLyPxMIs/lIKX0k3bcrswFPwUw2H0xiL+xgSzV7IZ+b
w5vPsxKqZdr4pZZJTuxmjlvRj2hqGoWOpnpzeJKtP2JZBxKcutnBLcIaraqyEqa843M1JoqoniWL
gG4iBdxLuB2mxjnK3qazOkKP3AXSSjFLKJTeUHwZoib1K3RNOkOBsgeQnfBjBa6QtyRTcFrwphe3
GPyOmvdHER+EH0Tu5YjW4eaiuI0F3E7xBIpV3qcGKm+CPF3RBrUzX7lYAq6le/EqsXmU8NRzRTUV
q9tMzLaZjrmN6dlw4jUv2Ny1X4mNN8zFdkVtFy+I5PmFS1xL3gumVEriWSg1ceOG9CJ5sxDtcQjq
HK4m4kTaMsB1wJY53MWpak76hZcLbBpIzMEcZ6IHmBSWa1Bmaq3d0o4JfldxmFMU78jcVliqwR9A
cOKQ2hYxbrI01S6bjkiQoFxUbxe4uF+d56qHZ/5gwZq81GlCphuorNSwbiqNyoMOd7nJuJbjU0S9
6ehW8kaJTMiu5mwBUoEuNh5iOqgEsOFEQRjO4MCEMu1LhsXKDacBAV1iViynBU13BWiGWcxu5tDM
ayo1iS+deeCU4Aljx4ibYHiJN8oCyKUyTyjuHl0vlL94q+Z3EdKm6lraeYExqwRC2msyynSz5Iqv
lgYS6jmMfOBFVOAmXUXWoOnETGOp3YqiH9ol6hhriVVvuW4PEryQQuOY2lZ60PQaVLXT4lU/DARK
Nsp9YiPzDB9TOfbFfqVZL3BSv3HbUDeNLK/UBIRoTVI71C30TAPupb9TY9sppvh+5TZ8xg/aVg9R
GZsWydsqyntAbLeY61xKRRy7sfEs5mz0RAbgD8RP2SxUkIWqbhmyw1Cy/EPP1nMK7CBhoSg3XZCT
78DoFgiF/GrsplreBFNLRiiCPNlGjFoZ1ggYHV7GkYDLn92audpsul0uzq7Yh8sRhUE4icDGLSPj
KKKSlINSwaI2DWFaoRe6CC4pK5HgDBstmIQVE8JEuImUxZAamEoYltA+YTMq7hK7SIpDk3K3uD5n
ATEvKWLqADcKqXKYQThK4GguXf4yK7+om6/pTLb8kKQ09ugKrvoMKxTIYFlvpV3/AA/RipX4Zw/t
my++O0Xr/PC+0/R+oV5Hr/PPaXvN+4s3IwBMUukep7Hql/8AH/4na/1vEcP7YP8A60njkds5dxPj
/Cdj5Yrleu4dv5RMbUm+/jjQM5cnjP5y0lfKrirXp/8ASbW2TxfQY0lfdLgr7D/XR610Sv8Ako3s
egP7ilsi7u80mfLniLUFgcJ/ax/1P99KwW4a4VPi8vqBr7pm/wDOUEV8iz9wS/rh1MKbn4+i6kv4
5l1ETqx3/pUrcHX+1THkt5/8ooUR5/wxsOk1Yf1A/OSGfCNy4IaNMQN9MErzCJ8Zjyfi5dSnt2TJ
nj5nH+WCIYfMLT98aLxVhuW2o+Zg3/Mcs7e8ccaHmDmflnaeYmgYGrrXeHFKBM3dl3BtHMLMAMSx
ACCbKEQxL9mLXBl1WRuJhCu0T7JNVCkIHtAHq5k3PUUStXEpXiLuB+yMCRLgL7Rw5S3DmUXMCwlM
V4gVNTgoiZuoWoVHbaU5sU6qJQWJQ3LBi+O80yUxuqmPqVaC6jUO0TIw9zCKzZiZhio5ky3rM/8A
4VEolSpRKJRKJRKJRKlSpUqV0VKlSvLK8yvMrzK8ymUymUymZlMzMzMplM9Cf8if86f8cn/AT/nJ
/wA5P+CiFrfJim09wX30r9Rtk4Q+eKyt/ea24+26P2v0CZTf0en8twsj/QY7fzQ9By1m0f7eZ4f1
MJv/AHfMr0v9fM2nr/8AWYKILi+AiR5fRdi0S3Fp0rvWPePZi+j9D/ERqU6z749v3xnP9/8ASXiD
5/wgef8Ab8ROvUxL7g7Pmnneg+c9yf8Ayke1/X/MWBYa5EqMWwfd9RabJ5cv4+aFa9MROGXzCd0/
+3aN/wBL8RO23/t2iOz/AGeInZZNfP6jTPQlWzpxN8cKDh3uIywF80UjlK75yAED9iL3V1KVLUqt
iqgPbBHTfVijbwOUhvLA17IIpdWqX4JVdJFDEbbTZLkxuCZf5pTeP4UD22tOFlwswitm+kCFV1Vs
Hr/4qcrVWvq2XwRCHsLiVFeiw80L/GzooCsyiHdFfg/xSGBtFzoYgEE0lnVNYbBXr3BND3jxWOs3
q+iiyVAFtZWGgRLE0j/JAVQAtYAFCOklagW1n/8At0SiUSiUSiVKJUqVK6VK8zEIKHRUtWRLglCN
zdxUPLEsD3VcDWJWdwlnZh4KxfF17QEFVFv0IUb8T1dQO34naKt2g4jadRTPalGtOuzgUShLZMl7
FUN4JK9jtzwQ0xO1/uLNs2ww1ABbbjuvVdhO26g8Kx3dGPtiUxKX+MNX7GvhBCKYAGQOcaJKlD2b
QpxLMRszgHkJbSYTamyl1TFYSr5vuVhcLPemddXZFSM1+yoHWYZC7KH00IHiGzK713KlCqfNhlM2
Ytl/o1JdF8oqIs8NRsG8W6G8Napi2M7NheEpIqf8P7rgK7Qc93TlqKqaqZ2sIP3g2EhyquUVBHLX
VgSxfLfWZeYdSQpNU19XLXtGGbHUb7sAitF6KJdRjlL02sUa2wreuBbvkyTUssQRlH2HcjZgtqqb
puL6g1oPjrUqv/Nl7hokoTt+qHtqQoBtVg7DhSvyylVHstvYlsiS9V1+qzQYaP7AWQSxraQfJR//
AK9f/UAylwVUvdMbui+8CWqM2lBRJVQSbY6C3BhcazwzbJmDBdIXI8tfsl83SaEtAAfIdWC6SCUX
aILUoi0uliGWAzXV1YAZhAWuh/BKDlV06g1N1cGuFg2I9ROLI0iUkbNjd+CXoYBoCCo7j4syxpIP
OKR2MlSilWrdMSMzrkOLfi6PjBz1eIaHy/agcudIooVRYbff+eBGlu9AiECNBI5PeR+EBoV64kTV
DxlDs/8AiLcc/WkWcu8fiJlTPwrI6iwtz13suIXO0ub7pCf5+1Ejcu+66sR/uSkB7opS0lsEFpYY
m+Allbbuv1eG4BsV5DHWAfNWq35hD0zlxdrBJenhAq3KHXEl2Ve3GoZr0WKuCSs8Wo1aaS9Cmrqx
oViqvHCiG8xmg9gIr/3s4sApuR1CCoLi6mqH/wDgyS6JzcG9RX5ZD+VHYQBoD+DtL2T/AICICIIl
Iw6caBEioKpQUW8wxXG6QX8QAYUkQ8WQawUAoA/gyaAHBCt9FVaOFWcsPLTfMekmleUqoblgpTKs
1Bh0T4JcTWhywXF1/EX5HaxjRRMRu2jXCThlmLW26ZvdMdWjLl0CU6BhVLE+9Fi7lIvQIcHSpuwE
GAWoqmpUtHosoRevuXhbJr18qGMIF2iqmKsUZvJtlclNxuY3V81HQ4lVdKvqw7XrN3bx/fk3duUj
HnY6i6dal/iy1uIXCZ6K2u0vYlvItas1fwRRQJFiT1TH1r75S1Fd76g4PvKiDcFSUFsVXgdDAs3D
x3bbxGXyFYQGqRXTNDSiBNEoEcXlLaZjHsZoIJpUOc3atdI//wCELMaublRh+QJOZmZmZmZmZmZ6
XLZbLf43Lly5cuXLlkuXLlyyWS5ZLJZLJZLJcs6XLJZ/8qOlHSv/AIVKlZgMlvQYqlf/AOEMkKAE
AOpdJBvtsRpPVQ9f7fzNJGBwwAe/Gmou+g4/1Z43Wp/tnDYxgr+zLH9yF39mAfFIDc4LZ6v+Z2Pr
/wAZXr/c8QqTL/txDLXyf44Wf7fxP93+OBjd/P8A5zPZ6plmZFGn9yf3EJuV+n9zm+K3+ZyCYFtv
U/8A6EQO/qP7g+pTn+yQdt6CFFv3v8Q5vvyVP2VyCQc59zwR5XqrO8HiBzzib9z45jl/n/yn/koH
/jdE6nEeT/3fqwnu/wDmhyl7ic3zQOi2B2X3/gnm/wC3iD6f+3iDf5f4oX1Xr78llIfEH/e6UNv8
ekXSMPOS30JoXU4bXxdBQdqGNro/omoRl7fUAAE08v8AhYPr6M/4Wf8AAwUv8Sf8BB9fUn/XJ/3y
Da+2f9af97pFO5KdyX5Jfkn1MzPiZmZmZmZmZmZmZmZmZmZmZmZntM9pntM9pntM9pntM9pntM9p
ntLl+JfiXC7bnEoyw5SIYQaKQOWqfQ/ucVwR6AMjLPMZosFu4yuxXi5tCVqvpvKHcWSyk2S/eWSX
I8ssz2Zc5ill496Eh2ss+mYNwbmWx7kLrtNiYvaZotjysK8QOiPEy1Zi5WWpjuXQCs8TPuPqQS7l
BB+CLiq6BjSRJsZey9o4skd7DcRxGXC/EvxHvThgkzbauNDRAXgnZS1vMpdm4K22PiX4po7SnKZi
o7XYn/OJ5DHaKv8ASiLiv0hu5uyNdV54IZXa4n+4INa/GRzP1kShRcQDNZ+S4l8w+IBtTMszXUvl
szLO8KdL1cOXI2SBcK1yso5BbLpS4qnTkKA3U7Na13IIQ0SpbL7gDkJnQLsrf/rGy8d/65irj8f5
5vo/9O8LlOvOY/MA0X+3eKxFrEfntlGT0xelLvSBmFXYgBtfFovh67Q/zUhN1eIeOMLfsg4YzcX2
I46+8hXTT5FLWPkSG89GOdofM0d+1I/z8HYiFVLCi6LfRNbIRZ7izxDaDKxHHAV7u4R+c36k67Qq
WxEZtL+ICS0mjgljBK2wQfCw1mrjgIDUWVrp9SpGyYjIQWtxbHtj70fSnorLAoOxu4Fq5iqwCYou
W3Swl+Zpr3VxztQk1VAXKGmqGPs+VaVaqwQFgDsEPzKr0r8H/EELYNbX7gPaZtcqOcAQZWqlIfEU
CG7hwa3UEk0i/mckR2q6v7lUy2JkRVrMSiH6TVxHUqjkD6NDFrLR/qosyWET2YqkZSi2rhe7uDLs
QZYLnam3NpiEVDkrcGpZRa8TOKgaF5QVOboqUrQKxxdy4HamFW5WXeCWc95si8PeIRV/LFoPiGWV
L5l4ubjBwJNA9RTY5v8AEAtTR8xZiLK2JNjUvEOzB/cBVrlC15DtM/yaTmEBboWWqATVX5WVJlqn
llIUbAXuOGF/Ii60WigGYQFqPPycTMNWVF4OFBsPTzPbyzAub15hTqfMbZd9ybNwwqALXmoqvz0A
NRUbEA0Rq0nEEQzuNRcQi/LqMsuZIrFwX0WZGNpeggpKhFILal7jhuKTc5iaIjCyM05uWMuiwb9s
LkHllCG/1S0oeGMDKn7JuIvxWjqgqVt4s+ahA7LyvxF803hqNSlP50Rjo10A4G4WkA97YWRRl9LF
qjcrU9TftVIUdsSmh9wmpmmj8lTCw5iFa/1IJGEcN9VIDFcEujz6TGJQSu82rmX4c7fCYy80oPds
hOGC/CiAMCX+GW3YBjwZs74r6W6i5g56XBii3uUr0Qe5iBieguIhEixO+SAOWr+dTt56E3UtAVKU
2vJsl7TFXW1QfDBfi6C8FWxnaiXlLwVcJu0v0ajxjQtSwwupTWeMpX2wOlWyGl2h60IpdqYLsgFE
zHbtqtuu0Eq1N+aYr6WWObWY04IvOM3HYtIEedtU/wBAzMQuwrwiGFDFFi8lMSkW2tfLKKypwtZa
FylFfhl+Uz4bxtYlmawjVWawQTv38yYT5hXVkDyYhKNF93ta/mul2TmFTGlp/qBXsdIfnvLqmLGL
AVe27Upy3FBg2aRC5y7pCNr3i6wrXMGgtDNdv6qX2Co5O7FsuF+S0BfSxPewg2quaEvsjznAj2sj
YXLV6r/qJsBA3Vb3QUSlUwtN+iCER4lV1rs2ptG77RcUpHRWCAl1Lt4sEFNcBO6Cv3Aa59JgqC1w
98j8wO4oPK5XDYqPWU2x4ifuJ14mC3L5OIHmf5mKmICicYcy58zhRNoXfolmxFsg4qClsUly8fEV
Z6GZeY9SXXMu7nzAuUTTh6Fk2g2bhhitir1ipbMzgloiqa84ogmLhrtl9rDAslFxbHmZElqUMrWu
ADUxlRTXIulijSFJpLuoPGtdPUI8UVZQN9ivNwkvQZSo0zU6fhcu/wA9BBsJzjR27x60UpXd8cQX
/wABKRpqAsYJu41xXY7YbioNlLyaYKzkx9n7lisHFeETuOyPHd0tX1IdZQid07+WX9monOAMFFdK
7NqDzcckIY6CKZFvkQwxjf1e1kHgVb+ILL9VrCrLVTb5jD2PerSQqDZYp3BK/QI6ZvUTjl2hFzeF
eyrgMaefAoYxZVcOZLa7RY28p5IvzbAbi7VMYFsuKZjaRl7XDPEstPlctWeXCxiQ51DKsiOnDBxm
z7bv2l5U1ca3MMcApA1FLt19xMOoiW7tKe5PtgpmGk31Uvtg+q5gWuNS7/tYwi6r/tyzdSwWSA8D
RftFG4bv229ZlOAq1e8GIr3n1VWpiGRfpcplgeIzPWUzUVcvtywy6V1bZKGvcQTtCaNMMBLIrmLp
pRvWsA6p8wWZaz58+kMGxWXdgbIEO+Y0PaZ9usMfRMnmMWPeRL8akaRN5X7SBN9+vEJgWi3DWlX+
ZYjMPHOYr2vXQvj58y+PEbR5vpYKZyiVX3cvRR9kWdWB6JFlWbXpF/ZKWKqXlNQ+5YU2juhPyRSv
dD21LiEtD9JS/mWZdRKp3m5w0w6g/ucfExKdswVb8dRzcbdXNQ9xoWVcJcupbeYMTW4QGmJzKqUc
w3Fczjq3YQwRWjPRJrylznoLuILxLV6FXFcyECR0lAIcosvLNtQTcszFQbmYP6n92DncvLgLhSWf
fEf3LlTQrVf6gMVDRd+2XbBu+mYwY3ZXcg6l01XPQL+4Kk4uXuC7Kc5l3lmRTkqUyZyy4lriA1DS
YSOwU5Vuc+5XXFy5TSbsguA7c8ZlgVxcbqdjE1bd9q6q4RaGwUOyzEXSKHyTkJ5Eq2XxKWtRC9Zi
+bZuXaXIWbzG+wuoOffeYMtgERILNOjoU8rB9oOYMQsQ3kwOycw5UlXlPYbhgQNN0ljChgACqZLi
go0aIRQwaRvK4DRTblVCZhKS4ZUko48a5lvLxUy+5w54guOlS0qrSKpuweC7oh7wT58SNmzSex4Y
nJ7kDNKc/DZA1TaC7rqrNqq4WSqAvAcR9wXiyFQwZON94hczkfqIbvsooHhh6rAQ8mEf7lq3FAv1
LX4tiwlmujuXmJXxM3uL5xLNQl4gtxvFZtuVrO430GDedQM3DLa9wwmA5ixviEUS2XCzBYrFnEx0
EfbM2eLgKp+YWUlWdOLl7lFl5W+oswWhEfow2qAH7BjUCXDEu7lCylxqm4RTnIkpxKhH+F5qa63m
MykufPRVdQivL9CUuMjBl56Nz+0Fg1eZyXO1dNS4TNEUwDUTc0TMwl3LxVwB7wHzGyAdQ3uUB7Zj
4KfULllkHm1fvo4VJh7RFdVSq7WJIoWJx0JbLtlL0LzLZ/khYxSDfz1ucxl3xzBr10R8x30XE0vT
n5lsuK4l46YDrFhnHRi1XS5phkuMvTNzJmSDU4belst9y47Ki1LgcTJL8xJcYEqJAjWJjEcTPykx
aiG7sGwq4WtRpgDeI7ARi9UxUZC8YUeCZFY3cQBb4YYlf/P5mRSsbyCiBWtFXudwKw2UDzQhWpVx
Xw1K6JldohUVAi+S2iWgXQitWiGHy0gzbiqF61EBWA5BeGDYGLNYXZUGPCzyI7qGtV0ie2BMtlYZ
PM+FX4DSMz3mZqXOWd+gkmhovtCkrbV4H5lXmT9ylYqF7puGG0hKFjnEXhvEeW34qYuUglRZxblY
aYOITNFIWfcqn4IldcZh28xlMasgATsglcITTg1Dpt2l27ED2gq9+iIF/cqn7MpxQRhXSrypkzbE
F6kQIu3ZGLc9gaV3qX0sBVU5qDjaOAb3cZG7d08+SX8KhMveBmSbye7pguYVdm9TDgRUAi/RGvpm
F9h8sxElhryh0Wlg4wS5yhqAC6F9EXmcHaWq1Zc2PKB7dTL5BBlkSVvxcuZj0uFgrHmGsbaMZSpb
dMQtBoI8i1LNoMABVAqhBVxqtS5xEZUKZZc3eZwwnY8Tm7lwvPTiVmvMzKxBlymLpg5z2mDTmUq5
geKJduuYsaikJWLhgiNQhL1E6nNR9xUBV6lJuPQFLL4iR4hKYohr4I3L6GY8IsVB2fyFBByrRNQP
+6g3GlH+6Qzna4xypQjmOCsPEs67Av0VFGoRNYrH5cQ64Gzbk2vLKl5IYB9HwwVC9quiblxoV4mZ
pyjC7jIECyJkLnNQp/pmF5V0fcITdH9x3lEnQbLF9yMq78RKRSNuxOlklVKzqCgGpeAKkXW1gBsw
fUaqV+J/MKC6KeVcMQtrH+BwlohvYCQhmCo95mU4Qv4hSmd3ogZjVyoHmclOLsuFgsPNbiCe2EGb
ULCua0kHYgVEu3h7XCHkZ8UH5iVjInGS/wBkoH3LF1HIytQWit1JOAVwV0IglJASkNcpuKYQ8FRA
woLkowIjZ+zHONRWXVqWeZTGJ9l2FDBCd71FIIK2lgBaAgoKB9tKwlMwcyhDaBAKtUTT9AAaxYPE
rdsKCaLs9Qt2lWrw2IGgtkW28wxigQ9wl/Fyu9ylkAslEOAno1GMcrRVFTNW1MoneKAgFXFizFOO
01ZDod9Dfz/Blz/E5ffS2c31xUo6YqURYVqJz0ZiFsVmZbKYEpdTuRSYmA6C7g4iczFyvErzLl3A
OgxdKMs6RA4UcMEqiy6YDN0xnxDTm7VfluKl+piliWtpf7PexMeEBsTYxqyXKqitYIU96gqrplsF
1XEKkd+uQ1tETUCJLy7Y1wfGpatiyvdZcmpTmPqMKtMBVniDyx0xQCsPdbjvXo/uGN6JmnsF9iL0
kMSgb1O5FCdk5N2s75gqECLvZqZtZT3gB01wxbSFDsAzLbexpBSMqkcUVLh7q8x8aA+wP6TKIBug
7jdIpcKVwBAICp8iF+IPVcYDiJ2xcnITtVuVv1XwFdz3CBS7Dwm4MA0pUNXYpoFN3iblzcRTpIMK
V7jUWeuYSZsA8HEMkvW6PSQAYJZ2/VYlyyZv1uLlpAA6u3jhIe2lTuVPlF0FFnYMYM9gU6tT8CGV
lcRXqKC20UET2gRPDzFmMECNTpfA5zBOKSkAlaiiLKK2dhl/jIzuhqXMZh/gMVnhy4isbqi7eaPe
vEttaKqzV/uXcubY6j92I7gKTA+ZUllQ3nEeeuOgE3cz9+/YeWuVj+90C35QQZ3zXF5rISCsriqL
z2ymELO2guI8NWpC4gZm0fpSWvZjRLtjoqP9yyLBiqLl2VBr76LmNpKjxcI7+ZglXfucS8My/bLo
+JRvPMBccXMdKzHV3zBlTcVJTlj0EL0G+5iO7rEB029AtS2oXUQ30TFRq4T3iymFibZl0RrX25mB
lwneXAvUAymoSwsFK1DxcATLi4D4JzuZcBLlQnJL6GEAsBoYMQYrmXBxOJxL6Pid6luJbCTFoPhL
n4i5+3oHFwVOhAZl3NsvorLzfRdQSXCphmj3FXLdy3pqO+l1BjTTMWYvUsp9MGUGp7cxVXp/M1Fx
C7+YuKmIUXLisb7RGzldo6kG6tWlr3iKFOruvMKD5Igtt7uFoD3fuN3DDK2+uj0xEKHQly8ypeWK
VzqFQ6E/xLbYH76LptLly5dTjU4gvQi4Z36EOZdzWZYxu5YIl1FJnfmZlhOCVmFkZWJVHMJpKPKd
K9W9OAXHjtoEKPKVOYQxxOWXo4c2M9CmVTeSD0pHDFPz0LNExAQKvETGE2ckMyx4GqYfgMn3FWQC
XV0XHBhN4NdMMomJfRJlOmEaAV/grrepctiQbhiXGc/MvBUIwXEaqc2dKuVTOHpUOt5iYnMevFSi
yP6S+h/cLhdznPQJX7Zpl3FjorVShPcu/qEqU9BenzMze4D2l4/hfjiLcqPqH99M9NTb0tuGEFTK
US38Tl6OcHHRW5i2ViVu5WalRmZZLOjLqUriNgEMPQltx6EpuZuEMsvYxxFv3Uq7lqgKQOG1slDt
mU1r7RxQYsFXqChHdlRe1iH/ADqcUuLwhYOG3kIr7NB95Us3kU+iDhd6CPD2qU3TUlO1e4T3jDoA
xStgcK3WAPe6elKidp/M1MAazOx3mMMsuLcNy5uXLXqOIEsrME6EpUWDMJ0+eelGY30rcDrqVM9V
kWKw305+Yw1L/hz058XCVmUc7qG0ZmWQYwYv7mAuYqHQjU7S8dSDib+5xKMYlyumjozCZuDPV3Kn
+etxEuBDMysTCBAP4LUGVCrmKmDpUzfV4mOhYTtNsGEzHaxgXSw8tQ46K59o/UbONf5oTUMiUBfW
4OVvJ3xlTvL3rf7lYxCFZS4twTYati3ATSLnAWxvgZN2wXUwCho2fgylvMmnOnZlks0q6bezAlaQ
1ekjn5lBTB3uHzC4ck5b0PR3bAC2/kFXR30Isp0qPLDE46E5hcJcGivUXruM7fx1N107TctuHSqZ
d8QehqXM18QvrxLjBWJG1zFojNy4a+Jm06M3cdPqYuOrJ/mGSV+yGEo6U2/EvqVNvzBD6631JjFT
buUJc+SZqUqXltnF+JfQbGMF12xMk2j0WzRMteoznoE7y3d9GWsW2UjLg9MEZxM9C1xPENEewhOL
xGK0niYYCVk0xMzV0w8shoLYQTvwfUOj/CFQ8UIjRuDAq6VUf/fzD8wUtggqi7RwxaIKtxCriOWn
epmJaD0KI66GA5EmPPK+1uXLOhrqPV63mMvpjoQ5luZd1MS5cGriXp6IhmZJZLajolwbT1HqSnnq
e+lxJUd75hqO+jNHTRuXnLODpxCXg6i3NfUs/MsI135nMSFVDM56V0OmCCr/AAEGulZeaZWOgv4n
f30XE5+Zh+puEGVa9CHMucTmExL67x0PQl4ZiriN9BdRxYypdBFufrRqMrDDzGXglk7ld+gNifX3
FYAaRd7lwneJLl46buEAnZHBLQK7wF6uN2whmd+g9KUvquUE6OdTK6ls79F6VdzMzOJUwStUJfS5
uokP0jhMzluczfMLjgtx1vEz0efc4PU3DUetMpCCn4/hmDHiEUrpTUDbXMvPTiHXujj+AKOu6lzc
SBmc77dEs6HR3BJHgHMqOLl1tJbrxC+lXAeldOCWr+J0ovpSkEE7EL4auVNsYWr1GByK7uB2hCK2
0N1QR3LofIsQQEzQ0jhe8zIHSsvDMEN6wBe78kprSqU25HMNRq7VE1XtOIlhuZNQM/BXDWgiwor8
BG8gMNn9kRjcz1qUQlBldVDisi3lB9R+YQWdqjhdTKqJU3vQ3UpyII02aToNA/xquJuN7iUHdLfC
2HXssGmpUZVUF8NjAhrGK+cWsbA+mAFogFWOmXAILQWxySCrRSQYArWgtjmM5oufPNVKyCw3TNwn
Z6laUxIFWCX4lfYC5VSmOj1AtjFUNaDcTy6lFXgX6jo9bh7jX8xfGbJcGoy21FLquYI8ygzjEXkb
rQKllqDOrKIoVFOkzU5nM87AmGVGAWeEJY6YDicHqOPeOnO53Rg843+5zHBcym78HQWMuFgz3aX+
5mGR4RhQzA7KWHfGISeOi2+OVYVWalF/c1BRqc2YVsXSKVZMtEU/l2vTZ8EDqYaRR+EYuF/bg8nk
lTlRur0uZjWd4A5XwRX4X6+QiP5ZWIqa7wiQPtyD+JUWiBc/vLvEoLzplOYwPgXGf5iNnxB0UBdD
suBAFaUqtn6uYhVzdY5h2A5Qq8iDEQbzMtS+WZLBWtEyWVUG2ucy5hKmdMXeBTcA2wq5zFKCwGJY
d7DeLxFiFhigXNu7GD5LjY0lJsl6CQ5Oll0xKUorosW7I6xEacwKyNKWYe0tfsEMSrcuFu691LsD
6EhFPtLySBBs/ZwOMCUDMOY1zDFgw0BAOCFfqGA6qyGgBbNtuypeI6mcy4PEvq7oDo0Zlyq9M+jQ
ShmUHRj+BYPByo3oOJeDUPsFH5i8miOAtV22spKXRKxhfkYe/RYFaqZrYHpzPirnKdaqsGXnx1Fk
BYICNJf5lQaRd10QlJVKbtrfuLZ4X6SJUebNqh4JZsKQ0eoZxHUexDXldz17qF2zPQkUBhlIXtIt
QNGeuTDyukUoLVfEr50+mIX5lymZ0C72RdrvLCLHMAOfKypjx8H7kVfqI2RGthy/BK1xYrwZfLKl
hqgStIyp33QpUu4UWyfKly8OYIPkEgYEHlWDR5DLJC7Jw+WE122MRaReUgSsUGhRvuQudsbVdYLi
RWrqi8BM5kCkl48Eh56817A4li4Nwgb/ABJJ+xNCTyMLBWjZmvuUOQWUyIo3Godq0lr13KXiEtMq
RsssxHm6OeqcY9Ja10DgtgCO7n6KcrUQaQkro3KpmGOz+DLFTQfQgPHMsUVt4YK3QdopjiX0ixfM
SDoCu14tgOrSlChwMPUZo+XaYUvZpN8HM9G8qk8AtLKhmsuoA4eUN+GSFgrFzAngi5VkAGYECC6e
zMjlgjLZQNcl3MBbLKYX8BZwnJN7G0E9xbbCXGcFh7BN4qzh9v6PE1Epr+Oi7PqWD7RINncdkGop
WYjq9EaEtWgLXQGoJQBJY5eAIlJDVRA5GaahFbHuA7V9+IlaDIhbIXTuDQH2BmZ9Ysq7aMFXBAXU
9qnmr5ioXhKLA+7Ma9TNXtXFsfp1fL0T84GB9XCkNUwnd6do3PemW4l8XyFs4QVu6iZdiHk4+Kj0
sUUbF0gJYaxKHBTRgBVki0hlRWOTc5ahpM2C6TARlDre6UqiggAy9wa7B33D4gpCx4kwGAI3M6t8
oANtkOBNu0rJihValjXr0CjKyuVLyq2q8sqrmgQqriGI3eZGB00xlL7bmPD3jSNQN3DTUxLylQ/2
k8joYuf3KbPMAMpb5IhEIKoLsMoOCM58KShflmNqdDRjF2Ji6gYDUUslAHODcpDgTNcA8veLRmFo
3YtrUvawxgxYtdBreSLLGVUkq7jW4rQwqrw6L4Ntwoqvu6zLqXpwBXl/ZZx9QKbp2EsCxAJWkGFG
a1XZeAP1C94expY7Yiyo7IpF+YBgIO7Jesr6H4mx3vuCSICjumZHTEhQ7hvutCkhLLIXpQxIT5r8
1ljpiFZ/bUsqDwyz+TIcuBfS6E2p9AD4LNktnWbrhVMM/wB6plwe/wBmIV2GZv3/ACQqQGPzGDbw
7WBCPHWJZUinJaJeVWxhiBqFhessFTvKOi8k46fQxGsoAswAzIXb2e0JjPHf4BEDq/1EUrdP4HM2
WR251ExkhfFlY2RqDE9phWJfzeoLO9DAHmpfcYFhmyhKtaj4TPkGhziKuwX8IGx2/iUz8QrIo3wB
gQFlfcbz2mY3r5X3RQSw6e5e8RBzdlxpWd3FT3AUCN2MtE2UFHhQg7Z9Y8r6mJIEF0URqJYSlrey
t9Fl2YT2TJHNqLyw2nmuKbdpQCaKudfyGH7wEgBAFKiKy3hcQvZ0Sv6uA6UngceBltQvEGvSXPLD
JtkMZStGl1uRI6hIg0o+LlOCP9KyYY9A/NodgCne2YjQVoAFAVghFDZRaAvGWMcsY89BW9GFpYLK
o38q5YUT7qUvs3FtucKwqyEy8vkx/O5WwwJCcAjmjlS74/laX2KiWsANJY1M3C7oGGlaA2vPKFpx
kxYvhuIGlVvydCZDbJsVteXRHTIQVndfmiyG2Vhp84guqqQLs4YeKium5/JiV8SSqN0vTMZdFtPC
wHIZ06IBhzXqMMuRZaFuZ7rl2OUdA0aQWO7WpQMDaqbLMMwEvDLVDXO+xDkg9dkA6FpLvlqXSo2Q
S7gmNu8iwCg9oKTR4ZDvZLMNhQ8DmK8ti04gsM3U2IoW5WnJiiNWzvCiYuWtc4elFuClkaHuLLqN
5VpFXPQrcZoipglyo3uKM23BN1BQ8bqXLUCqLBerly1uDV/DcXflWJimXay8qmXVu3mFcw+72i6n
EVBpBpI9TSrLiB3WbcLpcGFbRAPZGxjmlSO1dse62CaGTUYY6wTKptC0fRKUh9skAABHNzFkP0Q/
BBArcfVcLTWEsAPkhpZrOTXdMzCCKzJY3Ar9lmBiLWFE6wuAU5mUucPqEUVaYcaq+pWsM6aW4v0Z
63FRld472IB5FamklNkujdeabZoqKoVxWeScytiVMZeiDm8PM8DgiloSmLE+IHf3K1MnauhK0sKl
Fu2BsTGsuAvjiJlFZFVKtilzxDijs6OIafUKQShiTSkzRxwiuxMuydukqhDM6Z3I2KQfVP0CPZiE
22ggKBdgUC9iFka6VYKybiptpdIo4Ccblw6MXaoKVicRXgtXvIe3SCy4g/nJtO1hYyqtlXd5ggm0
MZ3tqMrbos04W0S1XI09F9kmEa1grqq0QBUu1KSXmXNnjU6DccDrauCLQYPFgYAUAUfatmF5Am17
HEVGOTR4NEc4o4ZsYHWqSuQUjAAAUEFMsFDuclwKJZMfmPetqNkYJSUe0rGsXYa1cTVVUKNjAMM7
Od95SNeThPMulNYjJ0zThIxQxlNsOKjoYZZStpU83DW44DOUQYAoKJXRmGUSzdTOXlMy3IpNMW0m
1du+l+IWUoz4lt1m3liaK+pmyojbcoITYzHh9TSylCNELphKqBR49LmOuIgDsThlDRO3ojTEPuoI
jGDHZbQbgTDB5TkM/W4K30EMDBF4XPELo60TJhbj5st6tjZI5VqJNfWQr4jR1gm5vuTlhFhHodHb
0qCt5mnpaEvv0FCEbZm4syyklblJOJn8wuVxX6cozHS5boS4XL3Mx18TklMZx8S2Hvj+BDpaDiVm
Y+nS2cwz0vronHTlOOgxGXzKZbLxCujM3CqiQh2g1Lv6gPQiszNypU4mHSoy7Id/4uZ2lSobMxtv
vmO3oy59IpctlBiaEufyDKf275GSQZMRjyUBCCUOSaRAJw/d8sNHjETOTX5dDusKu3i46EF04QX3
GnWv35dDiZlaRbshgDrrvcav4mcxL2q2UlEwOooFqsHs4Ru3HowLOtsWPEwjoHKpHIR+NZMFu32x
l0ze4EQ5frt4UzxdtJQfQtFtuYVb6R4mWNKL2sDlQ2N3z0Y5hMSyWbIRiWdMEdwnE2vR30tl3LX0
uLhOr1Kg2e+jOI4hUdQ6Hbz1NdLuNS2pc/wdO88Qlty5m4zUNE3NnTj+FwIR5hsmz46AxGD14YS4
dGtTj5lHHaEzCckf66bibg+IZIYxL89GOUmLp6Y79Gv4VtqEqAq/DExBdvDECQap5qNiQbpNtWu5
aprwdixhusj6oIu6jvbGo86haEAwFy+xpr0BAdSZkqfom73fsKqe2X8ZigFxJcbAJgpgq8hziwoi
q/Q27YSghFIoL2pATyI+zEwXsRPFKW0RhdP6WKf7KrsU27kufkv0oFhbxFHnrNV0LmcKw7lDQ1Kt
mxPGcHgRYPKgpKlGQ8fz7wq+lbhFip05rxGbnEcxMwutTkl3BIwmOjUJsqcdLgTzcplwePMUWZ3n
EGPMJ2l3XSr5mJx0phH+5fHQYlfyxKrptOhF6XRCb6EIFdOYPQdBjshublUQjKhbOPiE+ZWYnaBE
F6IvEFzcWrlGOWU9Ksh2uAqpVXCJa8TCHMchEa0DK9sRrjQOFWTLiECt7Huj44I+tQhyvho4gIac
ZG0rQiAwpVMNSstZU9GUyf5Fq4BDYcMQsu1fQQQ4beSNovkcXOB9T7tvzM+Bp53X3DmIMb+PUcLe
IAuuXlZY3hvghNaTr5I+IGqlGXXaFHxcWsU2Ztcmbmac2tF0BCiKQjOYOJcuiK3EsOjK6b6OmDip
XMvLEEWFGNI5/ipfzOxMA+oShp/gdBuEuXLDLLFpjxU/z1KSeL56BolPxHosLq5czj+Qx6EW4Suj
icJeut9K6Dk99M2Q/hzMfwZYXLm2GPqXXVczef4Ecvb066Zh/CS2Bb8M4cdK6K5QodOKjfSyVz0V
UrMYtYuZb0dPqNMCcTPecTC3zAiZl9CM467Oj0roZjVRcQl9L30/zLLhUSmBCJLhglnS+ly/4Yrf
U3MSyYlEpGHTN9DrdsYTjoY6eWW/n+ACY6aemMwVSqemTpdTnc46X4ldK6D0pQ9yqI1cLuGfrpac
1KVVKo6VDmXdzMtgZ6GvialXDaZBMX0Ek0MY6wpPtUSiEGnEXXmMBeiull7jsJ2iTLkQHtUTOJ46
l2hGA1gxbATrcYIstZxLuVelDyqiUDBo2DBsNy3F0qV5mrlXKeutxcrBh856WwYQB7SkcDbg3jm+
llRjz0MGNEufExBHFgSngR6uG1AXLqXgqo4cLTsaRyZIe1LQTLa6NQUwllwuWXDouFHfomxNTas9
BpV9CXk6MMKOgWQgFBaOLRqLgJli5pUdwlhLtiJSjm66sFdaAaadeZrq7EVhuhat7EFadTZu4aYc
YgNZOZriUkd3qq8+IhjBJDbVXAzKWMUKtRT7VJFVdI4uzUAaDykWQC833S0OIRxOIaWEAews6U78
wTN9FQcBBtp4XoSxxNINrlcVi+Wwx2eYT3CtGVhOTkjVvUDLEEcOF4FW5FwpucF3R7hKNVl4HYj7
KlyE2KeJb291ql0wqDKOTb98SwCSofLDE4qOjF3TEwgjsuVncxKm6T4ISsuUaqVxeUmE1KCOmaxG
rSxxE9JVCxCz1HMpYhU8SoUHPMIMo4eFCVr7hcNUJzAUirAPWYxMJSqw1aGKIG6YWjgwGjlruEXU
ZPSM24wC2HPpmomcutZnED6VgXFlBLG/tnSwOGPwgQKHkYSsxq5cuX0Ja4dvmkr2LWa8uYzD4e6u
vKpi8qrvatm0YKNDtRzt4Pw4ZfnwJ50nhmOK/Wt49JwV7zFYDvCwsAlitZviBvtYgrUANI3RS2ws
40vwgt+iANwrSw3D3P8A0JgI39o8JjuwtZyMpcblv9FssWJdxII23lTS4lwZSmPvJg87MGYxedxK
oVGTLQWVLApHHvpsYqv9a+2HsUpWPjsYNgNVXFf/AFZJy+iX9rXLoriZEbey34COCv8AulsYgpRj
TG2bg8N0O6w5yA5BV6uXeU8VLYMzvk1k1U1FAQf8AGH+s26LcdzLb1VsFOYq7VrK+gim2OQTN7IC
bMGUQ7MtJURVGreAyunDUn4uhP8AWcwBHEn0xWD32KDbTwZhroE0+TJsLLMLsOOhLIAWFbqO9TCB
hvZo4lQHZTCuzM4TTDY1TJg7kC412FxFDhHnfTJOYYVdRoHkwlf80D1GDBNJd2+kWMJe7BPZhAgB
EFs90S/06Wa+yNdY/RBUMoFypsrF2mDCYBulg8C4ma9l5B+tZgTvBtI0ri5SFY6nVfVolD2L1gd0
YSNay7bj06QjGa9cU0SmI49QB3R3jNVHgCUQJzloQ1047YiBoiMWHmw3U851LhL17M5V4aYj2AXE
GhQUOOIW7yqC5cZs6qslXVgt0alW0XCR2uyXqrpS+9i7Spq36ROUFCsClRmjUsgnAxNhKhAT6Qi2
ry8CVUwt5eU4SH5LrK2mwS+AhiF388iDTZ3MkFLEuMK1naC/Iiq13BIwNK9s40get7TzQLaJveql
cAAh0rpa7so9FweDe90hS2q+42YYV026ZWepw3DGj6gmJXeAuVmB9hF6IZpcu4ChvCYjdYcMO24r
7XpyjUanyBWkqBwRpKJyUwLqVKuvmPLzFW4pKf6ek/0vbMv7H3SMqVK+1peoV/pKLQQZvjDHdsfR
EVUpN8qEfGyUbeIoZcFaaT+462f+eJo8H7MryjyNJFpYGik4eKy5MNuLasjJA+iW2ig4CfuWRwXT
XesdKeC2DvuYTki0mV8re8VAMEAtFRPFahZCaOLvEKtrSjeLxhSTiEbC7t3Fdx2QLXsscR2hwU4M
RjE81+Mxi5y330Bvq8aaa9C5U4KH0Qp1L94kVQ3E0C0kUrX7AhGB21cobK1TTQqMZXkqlgGhH+gQ
2KIPJCfkM1ta9BTLAVEHglQSrJ46opmhLLvvw2RdMMN3HldFqnffSJpx3nAF2ftQ0ng/TJe0q6/i
Hl3p+sMJnKCFvLxrdRvcsmDdQAVZoNt5xfDoaNKNQ4RamZdiUuRI+OFTbrkKBCwvY2wrYMzQB9R+
W/AsFkCrO7b+3Mr3RfszHf7SlWo1vYzxVNT+0MIZE8xcVuRAyv8A5Sw+MRKAfNjOi1TLrD4pZSvD
A9yAWqUV4xxR5IrMH6eRirLUvLMsCGopfhGalzA4mfEZbCxCnsUKuNIrFqLFyhqtxWE1VKnInECl
WIBHa7pHcrIvLJwFERB9WSlmsrNxuJzFCQJaaLiKvP1Wf1HxKag34QVHblTxGyKQGr1j0q4atvaL
Pn9MwE79GBmSiNKVrWVx2XVGgrqlZbkS8M17WWCg1ikyq1XEvTr7Zcu13QPQQUQlNsHFkEpxu/8A
BGrotLt3ChEe7iaVyVZQoErBUKtdm9l9o+O1pYXBP306tr6TcKgbLHshvI4lChHmzUHAdoyFXJQS
ZMrgfQZYH/ys0/AmeOuNqSnlUKH0RNJw7SprZU66Sg6cIX/abL/qLpDPZBJm4tMX+wjWJbiKUm21
lsRWEMhlmZZLiZSo8RxN91wQce8qoQlbsHLYwIqceRS3ySsgd6BKk5opKr0QIZRujTfiZDRXDZWk
lDNDEZWbuPtOA0HmbghDf3a43YstxQIJCAgLKdJyQSZigAvSYTP2l3QMIQb41aumBtJea7ZmU2ku
0i0vCXQpLaLbqPNffdMtxGgpcG0OQlO3drccIlCnDKKDW0tuQFUlKsXK1F4zYp8TMdRQ2v3ILmVO
x9Q08HpXpkYmfayQH3aIWBSaKkZvHSvwvEIijSqtqmJSCv43eZlHt7Wv2SM0FVUposWIGvOrNDUG
4SWje9LmZQzjdlHDTLoIV1uikQwZK18RRy1ppsvjKLeqvLGypTXB1TCcAALWeBlmiz0T4aKYQ5pK
gIrpyPtjXDKMh4roDCbfeqEpLPFa27LhKxzikbLu5fwXarSJRVVXJXlBCi7StrmL379rmiBbVoIt
0tzYZzEY4MaWktprbDUuKAA5o0iU9z2lLnakLVlhUuU9TCtUdo8aPoLVhgxc7fTLeye4XypUgkUC
sNlRsrYRNpdn49CK1TLYIGAeBhuHaxEwR8XTySXoiGCnRQ3Hs6tatDVDieoHrkjYXvv23rxIC2ir
cFxfmpf09F27bI1SzmGkVm4LbXRAv96u+9kX6Pl9qEuaac0YG+Ll46o+u0XwlPcUtWJS0gsNB2Zo
uKgiwRKUKu5ig51YMqKxwOo2UORUpRLCHYGq3wEUorGJSAya8cUtIsaw9Dze/lHgqKlilsXYxDMR
opqgUReECgNAuoGajoJXSS2UYYooH7YMSV7Pr0ylK+D0qtJAwjki0RhfuKNMENBKuUfmUI2qdh5r
vFJkfhQ8SjkXiaFWf0lbd1GWwRyMW0wvQKqKwE2rI7QyMurezAvYOBm2uTk+WUbKpKvmnuQY6zHj
zAxcsBCNIovskG5RyJ4BlhgAAGg6WjNH2D8TZcDd2kX6Rsi7Ix3h2QUaNBBaKJcrpu6VmUrUwxN1
EZuO5u4ME0sOhcpqNxd9CRei2dFvrbB1LVLWQuZly2txuoS0quJa1EDU4fcdVG4qoNE830XFRtGC
y8bnMATR1cu2Z6eaneDTGmCk0jGpeq8RVuZiZmWYrOYqsZiW1hhpZg1BkVWH6QIMucBBpKiiPQ3B
menMvhLgrCktC2O5iag2T5jcuieegDHHS8weJbcuZvxfQmqOnuEs63HTP8nRqbNzPRth099C9X0c
kLkYu3ibWOCczExjTCCGoVL4aSXrTXVB0yworrLtKVxcA0/PLFYOVFu4IIeGRUXQHlg0QFtCLVAN
U8wMijLxsYlMKpNWX6RCZ3pBtIXS4SYqo4VzBalx99X5V9CkJREaAoGULqGoRp84FsUGwuMDZ8R1
fGJwhVsAtQbW4TliRvrq3wtW6JI72Uh9EOBTKLqLs8yvSV47Q4g+AqZUHY4jQ88RwzyGKsMlU7Ax
nMIx8MYkGX0EqXLSeWUxGJDZGA5lxcyuj0ch6XrzBnL1LxjvBnL7mEnJBx0IwsKly5RCpmZjL6XD
oQnMDE5rohmeaha4wlxhFlQqY7xjd9AeldDqYnHQnEqmDdRrMpEERGycV4i5lwKJZ0XpeIkBXEdV
MBeIaeaiUwIKvLfUFuanvyV2F8MGqLueESqoOFc1XgY85C8hVAGjY56q5iEVXlbzIdT37AgTi1SE
8VUVpWWqIfTvnVmo+i/FCMKtFQrogE7QGiMA3NPpSgrNbamZulEd5VJBEV6rrRHfD7xXRLk+vhd7
I/U/CMXBVAoHIafqAt1WDykwiimxisDmprbmvVmGia7dh0wBbeuASwMn6MCAaYN+2eybYJRTp8aE
QUMGHCPBi3c3zm+cq7Ig0pDUOOprSG2e1Q9h7pnwhqVncvpcznodLs6V0dzQly4uWNwl29eejL6c
ESzUtm7eYahNMt7QbdRnglwqKX0H8KuOOlxYNQmZi5crpcWXB5ZTCmVTeYPMzEydCD0dziDia10u
HSyd4PShWaiCrgDUzKlQI03LhKjxM1OXpWKurqBrw/SUbei+zCo1HtqFrgI/93HsBcJTTdcBMo5a
AcFAxhmRaEyF1W/vEpBW7BQO+455VdNWkqrl42RpBhgCiL+YSy7SDaod2d4inlY/OxvdEyVHrWDm
mL9mdl7kEUSObaOVkBEC8LZExClVbvYbmKGwfBpGEQAIPTwuZewIlqVXKkT+oNZalR2s0iRtT20V
0kO21TcyDxMpGbzG5vK2UhlqWH1nKaoaGXEAO7nU8LahCDo7cG02XNrk510Tnz4CBrXBKe7M1xl5
RS9aLzEBqXZL4/iFRjvo3Z0HtOfmGo6mjpUWVDobizCDL6EYdKw28M1T0d/wriFkpDc1cEDokpgi
HaX056k1KmB6C30dR6c9RMb/AIOOjCHQoYdLLnHxKmqOmui8y3OulZly+tz6sRn0Qqoe+xDmcFfu
EagrWi6lxIL5VwWYS6zlOZr9YltWXRN92RTW6fEbniighgVGmZCYlVHadj4jIVimKcKzLcoErVrp
iJawNE4IASq7X5F3RIhAJ3YG7JdYvE3E1j2MbrZwWjgQgTUm+zKrpdOxolk6MsnCfBEaq8o7gmoJ
U7uUSJZXhQHrh7XHSxA62Wjh0C/8spzt3hWrwxpbyuS3tVFQouug4uWx2TirmZbUGoYfHTfV6d+h
UNR10qFx3BxGZmZnp2nPSpuJGzHmEuXj46hbG5mVCEYUU9tSuvEvE43CXNyuSYCa6bldAY1Ez1MS
syyCvPQIHT1C6hG6nPQczmcTiBfErZ0OoDBLYmSOXpTSwJcNwoEsHiyJn6lOJu32r7hSEz0NxY5h
Kl9MoPKsGj2x2ksi3ChzO0VmXdx6LcKQbuKpXTJSolGn30MWL/ALS+lzvKHBcKQYTeozmBBg9OOg
ggGoyRaa63KZm49WMu7jiZtGxOo7xM+ZwdEBVKDKyyE3Qpp0yilblYlV1tvptjGEXE4iwHv1C0jm
rTDDF9LzMWVqiIGVltikImhpysTUFTnpdS6ugLI8BOZ9LS+HrUL35qVuF9EIXcM3Ar7roTGI5NAB
tmra0RZAJQracI0zswehhbWHA+Xi43QzVPqXNpOJtSXDMDBti5oZvYCiFdzcuFKvkSYlmeFihpUi
psj8iDeGc7hFtqX5D2dsoitJbZqFXUAYBumBP1C6OyjAXtio38QwIvRC0Ik0wvAoWhe8ZyMBlVlO
AUt13viV5dNFG/ZcQCdLRa4gA+6CFU02rX2WNm4aBVs/zVC1fwZZhFa7aI2uSswvFoJQmgWvqoRR
V5texZtqgJ7zDGypr+GCXLuVrP7laICkUVByQfQtu1Rh771Ppe5SxCAUOHykEbCJERtZWvoI3n+j
+mEkK6AtY7URemjTuR4Yl3iHwxULsMXfDs0BYRZdxyxsjHGywexllQBLDvRKKLAoiUnhGVSyBaEg
7taIfXf84BNVcAtYyUlPsKtQW+moBwrVUrsEHnY76oSE3PpSQK1cppFzqPwkO0gU8kJ8ZTWHzs3F
6uKva0VYXgIgEIXBx2ewIO9Vun+WNBsu5V3pmHEXaq9Rt7oz6oQq2rqYdP0kPCOETGQQvrY0vukH
tFIHCVu4OMtQTR6sSmSJ+maitVsnMpf4IRwFLPuFVTKTfzd8Rp2U0j6I3W3Qtq7QpJi6jQi1RFay
AM5BR+WuHlPTKisDWrGDkMFieyP3i2EK2MvZTeqN3dV7gw0to1oWlgZVkM1r7u0GUWotoXyYKsyr
NvtYZTkXv/FHpl5sLSGwDlste2Otg7NuRM3FdDhg0CvRfzP2R5WBLmropM5bWoVhCyjAeszJywJp
I4bsUgcUmEZW7wqtoLozIYtqjB9IR65MseGqnYcUIHdgelC3WLe3mYS97TKAIAz8DUuIPYCgAauh
ZaVVbLbwLVCKDYX5Iv2EpZohgdrIeoYHbf8A2IXTtarQjfBSFMGmUcqLYE5ND7NJODNpNhrBUqVT
jSc1RmvPoqwb0yjF+VDFuHENTRW3Z5+NSw37Ya9tOI0pI7V1xbeAELuqcyBkCRyW0gLUu6h2r0QA
DbSo3j28FTd+1i1RjnedUrJYAPHDmWxvcq2ejjUJPN0XqPZDfG+B78ubiina2mNDWlwxG1qnkS6m
RfNyKyMHFO/9QKf6FM1Fdo1YeIW7eP3OEGxv+nAPjfrGotpnuqE7UcLVhhstXOVZBGWf2FapFXM0
gMLlmFuI06+rleA7IaXljJptrIxSEFOqrstUYViUJk7zahXOYdrtQUYyi04JjFp9XnMGkvH8NX0x
DDPedRlQZqwjdHXwl7qZ7TYx1mOtLLQ3EDDS3391AQ/Ea2Kwdm+FO37VEumJsE3DjqtBnajQQxxW
KX0uG1yRtapQw8Cbfbmao5X0StoItVipCr+YrDVUWpqV/gZgsaCXNrfvFqYdvDNlLEsNa0dllIft
BS0ChgC4por2TCASDFo/d2qgzehhduqkNHjjWmL8MouH2P8A25y08w1RfbpTTFLcgcBr6pn3Gv0i
2Zf3SwOV7ZGztGlESYhJgTuXD62up4RlCTlZSQOlvyjaUhCXxzeBUdtpuuSq5jqyMyNtZV6S8ZSG
iUSqgm1IEl0alzAiODDwwQjfnRr33sAwy+VzUAKrNwpvuaH1UBRy5hru5YK3fYqjiogq0lMAx5Is
ulkAFxgwvoTDH5TLGshPT960pgKRHFRauEjREmdcNpsaufPA8BKyMY6ruJuKF0pWfcBxONaVLuF1
ngRVYZG7wiptJi+qgPIb/wAwEaoO3qpg1YTQq7VvlmR7x0CRKIjR0pMKzNra3avEWf44roQMgK1N
TSDHc7fWyLkCNjXsob7/AD1RLZv+iNQ2VP8AdtqN5SPMbRNnVvMbz41nRqMjM7x0eb0kZG4Kaa4D
hCJpJ7LZF2C5X3ayPyHQJG3OYV99mYMVw1Y9Clpoma8IrMAfPXDxX6XENA8hyE2MtHg+0QqxVfiU
OSHCip9RmqQpS2DyHzMGAWFqY+gNDqQu0gGsla3RSsBq4PCHkR0NIftrxxUAYP2N7zzQmm1tZxRu
N2qp8NU0MZrCeFrMkAAtoFBcpmuz+o5fH6l2s8Ecl2IpquZglx4alFJXe+pTTAJR5fzOFIjLnC2G
pSpUMrhSsuCA213jeGCaw3UvG23NN0QsQGBtCaLXTHz74y0Kbx/NK23EHVDR8ErgrR/SwhtFjRsr
RRsuXcvU3pWl1RUr0wvq2sS7Fgjw1DtZg3ceQFaiO6sxjEQY6agtX3Tx2RZFQeBDzyufLmlz/Bpi
I/AWN0SAcbpLlQylE4qvJGhQGPDJCfIbJWVUQNVNOpV1nTli5UUW1ebjscySLTYXYfhlTrfBdnhY
b8nWgbJneXCh2XGZvMyDW6/VDlktLDULefEa06bjRcNSDzKG6Jd/GSb8WmSU1flRdTCK7u1PNUj4
M8zZeYAxUd5NzVQjLVLapJK73p/U/wCidT+j/bh0iWSyCHvXNxh0UiadKUYVXwV+8ca66HNMvDxr
uB/OQWFU4dQX2hxB5PvQYPgfpR1V4YdvaFpP8ERdnK82EYFSlV3Mk+VGomchdrQUVBWuV9iuXBO8
SZAtN9QBk1MbtF3MgJYi7aaMPPWT6B04obTDz88csZiqFarmL1xOwwNjiBw5zJcdL+kS0BJpB+5W
Nm92WCNOa0CqL1g/TcMwEM2hHMczYLnG3nSDFtHe5ZhAQKTqCwgDz0/JicaNwBtMi7gw04V4OaVb
Vy3bCHy36AJSsf00QvkW+IJEUAXrQ4ywly6hPCskmU3Hnvdb3aplzuA028TDVAV7Zkh1paeEsteI
iEqlLu5o7nEo7BZNPzIZDmrkS1WMp1QAANNuFjw6DR1g1lVSMuFXO2KrBuMWA7AzxPJY3pp4EwFt
qlcpLi5ouYtSkGLUVcZoGqc1DxVbzVKx5V0mrIRxSkITsLqDl6rOV7UKgKrJqslXdxpRGGhOL5IO
BDYAA0AQsg0wY9wDzsbiGUr2Ur2g3HMZy1onaOV5+WKQJu9wouNYJnCDfwfqFlllD1LU40Ez3/TQ
jxFFrVU2aqUB16xXYj/+qrl7l94OXnxF4YUWqSwaSuySh2WhqXZqg8SlaSM/RgdyquTi2XKMyiar
iVC+K6173DzzXbA5ti41U7vf7gX9QFZqIxo1xHLNna6qNQiEam9gxEhJHlYt0cktu4O4yItmSoI4
XeXXEXcHs6mrEqjKuGUYTRWUywQalVUfNXKgeLXRF7leUvRcYvxYp9PclIFFc48lLl6NxHKEJk82
xcwWOAt6ETy8m2xR6HeIiDg0mY/B+vMWmQcDVvNbh68Git33WIwfrSr24CNzRym9hTCMcYI/YUdm
QnkMpC2sYmlgp7QA0TITTZdbgXkcUfDBcBu6CHApKYuFRkTllANNFFRZh2aTgne2PQxU0yxvW+rO
xohTgA72SKNk1RdlxoGiuyZVIFSaqMGowG0HYiX8NiOR7kX7w4NhxVTL33caKlIcBUXUY0cNikHw
OFWah2BAAxJu4pmIfLBlUtWLN7FwXXD5jkStxXbtL4Q8EAcEAofRTmoYSsAoaUcjLarCat2jnMIs
xdVxkRpPSS4mu7fjXMQtBUqwbNgiH3E5tG2CmHZiABVYgAqKpVqsxeQ5hcSk103Am1nhjUIoKJSF
C4QAKhvfUJLEZRMJqlS4a6aQacRtLZuD0qF10MwVEUhVbi6ZdTabUdF8dLqGx8xbmNE5j0IUkYxs
Ec7/AFL0iDV4gBpjBLviCaEuuIVVK0JcuWMzxHnMIZGiEv1c+kqYmyVqsWBoOWGw0xeimW9QjbRs
KG+0IkRhQypp63HpmpWITOf5PUZmJmZkSujRCXA6PRhcIyyXN9DpiF5majiW1KZYTMtOIy+JeOlY
YOOvtBqLjoLUVlrB3L5l5uHMcT5lwjhYdTovTUv+BHqS1IPRYl4mZcs6rG4fm+ndBMsvotEzNM1L
Hpm5h6mLmqm09Qk7hH4Jhff/AMmQfP8AUBUkv4EO08EvANnq5UV88MGSjv2Qexr5Ed+lR4JHaqrg
LjAljDUvlILbcAYyVGOWZbKifE27lTE4NWn2OZYW3LemqcwAyqzdsHOwJLyKAvb9qdMTU8yyFBuX
WsLFANQ+LytYXAZRvTK5iHOxLirRyR21aBaxfFAU7Cud4ACNjGpPuvt1fFy+JiAeuybtIkjwEBPv
mB9kosQGkKRNj1vPTj+A3FbZuWmoJUVMOuOnMyYeh0teowZUYUsJeCWy5imGV6ZlYh/FgYnE0YB0
vMWEDRmDE6d9OelfzHR6XBiiwnErqdCDdRu/noTno09OahVdLnjx0xZK6MINl9C7hFmlzX6kSt5f
qamyR7PE7jiIcu1FmHJRXrKB4IVC6uTV4HdiJTkuBvJLYOTBQFKA3gME9WWiPwqYzpqiaJRsM3TF
ZJYWQyrcRtHgQnKmkTeWUPbjbRLf7Y+c6Q33N/gGCD+iu9JQnn+9Sx8NVyFMGExHFktRUQe9MXzk
NZYRIc044IXALrtxeCzwWCicjEftfXS0sy2aoF5qop17vllvRtmbjbhKrmDFl9K6XUW89OJcFHoo
BmLl7hfUYsGWRg9L6XRAOlZ6GDJOOtTv0YmplNzn+Fku4B156ZKl9NdFs4lQSYTqMxDpcvpzvrz0
YwlS6Zd9Oem/uCfys6GH4l1BxuDGKxbvfolDeKgcjxdRYxGY8QmnqB+EL1apZst/m5i0WmT1OKEH
qytWCpQK0Fq9riBvFMnSiHQligHFleFGew4hqJorbE1H8WKtCehnrsh68+8gaya9ID/ktCaZW3kB
V0YVFHjjcUmmRi+yABbVqxohvYGcgcrBTRVwihWFxbIuuLkr1xzbfBgasgAgO1Ubhbd0V2vmGYv+
NWgsOHQu0KinNhrr5OlpFoi3BYsJc7dGXZGZYQ6D1J5g5ZUtlXJoly77O/XMbl4jqHHTNvW4t5rp
R0voOJcIWoFWKmDjrSCdDUJzOd9SWS+i5iU3BuV1KJzCLLtl5jOOhnq1LJfQ1CMNdXMNkEemYdUz
8x6upfE4JtTU0vt/wglnt/UaGiH78EpUaREGIBNVoFdhir1uvTFA0rVp+ItEQR2MQmgClUcWxj0g
KU7NdHdlYiJ8kta22t2sXMXQT3vbfWkQ3KFbDJEBoLg9xongNbIQhAUBoHZlwu3d+/0lFQQBRWBh
PSS2PGn9rMdriRTxWXgZ46o9M1M9K5hf8Bl+eoy4ZgxnF9HqpoWd5m+jDdTUIxerqCQZvoSty2up
ARIdRcovpZNSyWkthLvpgGToV04g1LvoC9DodBl/wOlkvpb0u4dFwIVXS2DcxcxUx09y5uUlBOIo
1s0iQhi+Y0NI6m47pT4IpA62zRq456SU4LgMytlRuMNTMYGlK0Wi4+oqHksmD8gS7+o7jTAlsb6E
t6MzGV1vcvE0iaWtEafSRunD0JWYlwJqLSewwuuCU0DT9zI7jK6cQYPQkSe7QWkAYuHZFutsv5zn
iI3mNQDSB0M3DU7MUpnYPvEKvRL0F1AjCcwSpjEcwcxF1LRiQrwXaDeomNgvXW5madXHRzvL/hca
tL66Sx1Mw63CVXQU11tL79PTpdVMRfMqXTOP4LDtLmAOgruGNdeJdS4dGc6iqLB3cYlPuKrHeGQ6
LvUPEHepd4vxlztuICa2dpnl3SX8yww0VmktLFcDMbLcV5QBytw42vnBdKrthsl4whrYRBndYRK1
RTuw2xWJ0hir2lzsQLSywejBOoMD2uf8keyPsAd14IeXlZJZczsYpfATV10bcsEaBg7YqpHqKUqV
T/YdCZGcoKFb5aYhhJhxrqkNnYV0G1Lirenvktmu8IOJD7QKVZIkIi5jdfzDTJsAGlFc0ux5ZeHu
hZeQLuqWA5SLzLepJbdTCAbfD4O8GaA9PiMN85zrqCZLPOhSq1b8mXb0OHxXIWoKwS4D8iMVWnxn
0wscjQavNpV08RV2rD8IWou1NjzwujTKIUN2WrDcEZ7Dq2LJA0rtajDhp/wnu6fZhE5K6DtymdpP
IwXpnDs66ypDsudpI2wiIoEq5dhGSRYG2kCXuabdl61HnuFuLQsXJAvMHOwQtrdEuYAtaeKk6Ycn
RnxXu+veRsQ6PCe66mnXgQ1eaU8+xlSiFV0KvpQS5YQ3FblsUvoMuWS+hHcWHTljUshs6OOlwO6u
lEqMHEu4NQltz5g9GDbNQh1sEtg9LZbFh0sHrfQZcMKM19wKL2tXyRAX2zC3aFA8/wCJ3HEsWZt2
OcHfkYIYmVwGosurLMKy4D0qCi6rs5jm10p4XS3MNSzBwVVGuCOJTBsFsqBA2Kw/dfWYtic2ldoJ
+UJC3V+A3UCcqq91aZiINK0sJ/pfV0CxTpQ8y/eLnp+IDUWQ4Sr7Gos3BSLusJh+uZF1EF0flKTE
xOhabrDl8v5lztr+mFBWm+prCJs1M6c0F8lX2zTuChFFZuIlBZuh16VFL9GzA1nEZCN8FbGDxrXd
lgAh2gNhqW3z0I23fEsyfhWhI7XlorxW7NBtUutsTOorHTT2zlrayXXJ0kVPmeFf0dwVakL7i7rI
w/GE7rLa0SxbJkEv0byl2eZKzZCFUfCM9HHRfVwHPIT5UApA1ntx5KLZ6jloBp0OhBaR9iYDgM21
m71VQbQYE+SprPf/AAsux5Z9wz0CPvoMaZupkhGGHcuUV0Ll2zx056HS8RrDLJcJfTmcQ7Jsualw
nMs1Uw1B67ejKei8JZLJcJzOP4OZxLJisS2cdLzAKlyw+fuKvh+xOT5ZVuGxmV/MXfrmSapReKJU
JUJvdwb/AHo7z3EmHeoatLIxUavaitkJuEEvhIh8VRlmiVIPuRYsGUlmutYaMARiI6dXOJW8uqDS
Mtwgr5NjLbbErSBTEwmf1VeIw/63X0mVLKsLRVINETqWkG/QRLYx5iLQy5RhJxwsIRRfM4uvT0CU
/wA8XZoEbTdwm9gYfHfPtlYwsAnKg06rK33fmD+1br3oVITnTW0wL61SoFzHl9bYCmXUa+VWSIVm
1avxqX50TxoeDuYZntQtPPUd+jvWPI7uDKUVqlz8S2hlCxzmhj6E7JVoBFV6OzEAlTUx7TXm3knF
YkyEROU22xuLW5WtGkY3h61FBzFobivMlGZYaegpMgw5OcrWUhUW14iy67ENMe5geBy1y9Oet1Ik
6tafDG9bra/AYPHGuu1lSyc34bxy70Sr3newZLfHRdy1oNqqyT3tvojLLHqFU62akIQW+PW8fEeY
66EqEZYS+J2HQnE5gupaQqriwmehLuVHFZ6moR10z1cypzHqbl/wa6N94HTPS4Cc9e8ZcvodHcAm
dERjYpf5qI5d/wARZXz+5ZmByf6ErsOIEVL7nB+GUDSOwVAKFzAgVABEseJa0LKaNy60Q0kHklZn
cqoKiEfEKa/CTACPYlBaB9TZmp7v6wyDwu3U++DdLAEsqkvLFywQAACMw0lUUag0xvEiIo4QsHje
HVFVGNUu0tGX3n/Ii4qEWWOgcxyZmXMR6FXFywcniWUYIKluILL5ltwDDLxaQVINrdxOpz0tSC30
QhuWVO8MxQd9Lxcp6XW5xEQSMubqezrz/CswhfUDaXV9vV9NmYJzHttCA90YVLtHWeR6GXeIB0sA
u/QbmMC6rVi8dSA2oZR0W1KzTqRHDMqdodNrLVaCF9k3Y8J02yowL1MScCqdLh1Dou7uHklSpdRV
1i4Oa5HXF8zX8UAqwuylhDuYVu7KqWJroFOK9BEtISu5VbcWl1MV8TaP4fye40iaMpF5HgEuI3YK
yH+1Mi54f3Mgwxd+Yw/3MR6I1UVLJwuhAgLq8Sn8GUpLjhUqmV0b79OIu4x6KpqO76URULue6vUD
fa0Us5RULmV2auMtdyrmBcC5n+F+ZXkiXUJXsCmmCUUfcFh6HsMrLN5jPNT3ZqyLOZSEtsZ6KECj
pVyQw56CUrAaBtJFiwaUuOII0JfaJXWo7IWrUftD4FVWX0TWh7Y6O0pgFDMWAJdC7iwAqAG1YLp/
pWcCKwFJO9KjwGBH8BZJq53ZQciNFC2XgXHfEbBZVagXQj+Y7K3CQjhoTmb3TChKcbbrcJr4s+gS
06Kri+9h3K1tuo3efEBuYotg5ohSqUiV+euDDYbg+nw+NJ1JiEFXJSzJa0Ovay3UUhbHyTHhqFVI
dIFaZMlthAqtsXzkIPatWTjHIkeaubgfB2uEmT3wM9qStIg9tLICc2yCId3O8cMT7jzfA9w8QTTT
3V1VNbYufsoDgtqihlnVI8FNWQYRdarC1gVAKtBTNrVV3j11X8txDMPVsIetQ5hTmVCBXOrsYPSs
6mse3wQssS00ey7i1ByRMuFr18bl3KY8efYamiQkFue4Lk7oqQAeDwHqNXxOqPFILwZRQSa+m4vQ
t6b+4k0+UIVtpVgDtYPyQK1vHnSEP1I1wbSnZgwlVrV1eAaJEVzG78REEugsGFhPMxFaRSQaV9AL
xaCPU3M+vQoPPi4aIWXOoHlAUHdi1peWNyG02BaEoInd2R0FogRo1WHapK+AIB2N+GAyZcurEgjQ
mP7jmQuyHZvJFjF1jK/EoduoAoTYd93ZmPBXOy2zDr7XFdtWc6SoYXiDUaHcNLZiUwZEMDu6hYev
EPTFdobVE4QhdoCIsZFAxRvFjEUVpJbWiU0NsA7rgI6GFS3rcNa6p/fLGLKbm1Kk87rI7oBEsjVf
KibmzlsOYzku1QPiWc2EUuOEHgzmhbUzcI6Fxcb+XWNZdxQ8N+0O4KBrYtAIgGOEifD/AG5Qq03Q
WqjTL6oUd1bRIP8ANumwyuvIlVU1UwzxugtRBsR0dkUe3AbKgGVehgLHxjPeBI62idpveBYfI+fY
sViGrpbDTAtqcPpj2VO8rcG8BVzGX7aRV7Vcyen/AA8qXVKHRurpi4EAdTeXF2zx+PeBV3Iju4Mu
OVJUooNnM+ILhkHpAzNBq6r1QXAiXINGpnqDR4En7dM8gJkHz4NkVpriTXAvoN5dCvatRCIMWMtu
7PMO4AzFnEaFfauMYyg390+CKvdoB1qVP5ag0mhuNQoVqdQqyNVTNMJl8F+oqA95MfRm/LBYKWMb
l8r6G5eTu/SOLJnbf87xiCWRfVoWwpZFmjS462ZwtSi4rfmmGMT/AHLoLmUAw4WvJfeLq9r3d4Vq
4cyoS96s048XF1f3hYWtj6XnM7P0Mk6k7UokBIwM4El/t729h8EMLOS651Eutm7SWg+hUlPCkJKw
tGio7DzRxhsY2hlZVAS06Fd0pD6eDhdxiBkyIgJLKhT7j+1ktKbCmk5r7wh1255XKkJrnmkpm5RD
eeoSF4PuVsf6OJX0uscZL2Qlym9yb1iYbqsqmfvUydRUCuAs5uGxM/sJCBbcMCJaffqqXjMtrKsK
DOwOFjY+pcOqew2xAw17TqVFNVkwLL9JDILv3sbg5leaTcQJKEruu1EtOc3Y/WESXr87pvjNCKNl
L6zCcYhs0skeJdbd5ZLVQGsDMv3hhK2aZ16yxva6uELd4ufcN34eH5gJ/rvNF7p9p9SQUCFt0HdZ
gdXK7j8WOLfH3oJWe1l1bZKjaLfkxKpASnasrBsoeflxU4DENXUZW4JxBj+qkCWKxautxowSjp4x
rey62zVFX+xnEva+l7tmhkmqL1Gy2bgip29V7yR51U3mvc1iGxOQRRjdf0llINpk9qqeYnxqFe4K
acRcY5EuuiACj7gVNStVWtRV/ry4b/fpA42YDlaI0FnDlS6430AgFdVvvEervlBgWukSbRk+UARr
ctxqvwPgY+fgW0RDG2WjqabbWD7w0fL9yX8wXRW34iLTSS3tUqU7wdLHnt4D2gLWw8vBirJmqImz
Ngvfts2S7bnlP6pJmhzXlruDxL17dhE+NBASq691uqi5he5LhcPXtKp6gDKgRewHMdPf01mJZGBE
dwNf2EK20bFVVUDR5czpUKk7Vy3D8ti3B7vUYnwls5Qe1Zh0+oFWaoLWcsZidnV/DGuXRrb3oZ3e
QubvIQi1eVUpJbaX0nS3yf1Hu3uwdw6ortRR0EE4jaYK9q2EhXHEIxvk8QZbHgn6K0DeFXoBZB6s
0gs9TrAnI1pmmOJ8tH0yDcynUtSmBgENAYHYLhYq2KfYsftVwvDQwubuvg5dmKbZ4Gub6IqWSPxS
dv8ADKi2+BA/4XzhmlBLOkni+1RQjmcylWExht23rVSyoAqlHYUhhpM5yRxKTPvKa7xIrdPHaU5d
BodbZuDU4oqODpfuV/Vzw9sxcYrCqKLchGlVCvU22TEyYV0il2sf+gWasg2aTTi8oXxPMzDLqbAx
jWJpiJtDVgTUvi4w9jCbAb+GMLQqSo1YDEc43hLY0JsKeN8lfcoeCKUEjMwLIx5og22peKOADRCK
wrVXYtCxOuMCFCDbHOKYbuBmIioLH2idButRavRaEcgimqVfewY/qOHQHAGCVSaaq8jueXCE4TF9
j9w0ef7Q209mKtfX5gvvV/iZ+MgF0kFrQokrWu2YSvcbT2RaW8rVVAjQhStAQe4rVPs9zkZUDNl/
tXShilanewjJawQsPWKnvJeC2tq+XmHQ73mHRmRC5MuIIuqsiMfsKm3I2LQUXUQtXB2UbIHyAT7G
cdXSyFzSBQs57k66icyi5hCo07uqKw7kNgowBiuGDPdhxM0wsyXp2fAkeW5IFs16VQw0xgYvh2gx
Iung8UUS9bmoUChph1Ua8aiUQuncbmnrjuk8H4gBl1YJbJYvjk1XdtUzUOvldMBJL6l2W8mUVeBm
xG1pswVUUYnle5gcY6wUrm7O2mGE3HpgEr4Nl0cCoarwpiQV+xXKXFzAqx4bZ71UdQ8/u8HYIrFh
rY67VYkWK+fRXZiAgNIM2MMDQLo+TOOaZdM7Q9DAVOKqiRVKIFI2Uyiq3GFLiQXaxIXFcBRaug4J
wMuCTVWrCcahKOcMdnkG4LpZhZQ2uda7svLvcYKE9DA3bUCAgUFtFQPQFb0ukWQatwp3oJKQFaQ/
CEbvUEU9Sq27QaKIbO1DaviU5lt8nuDCGYIRu2lKBPNoifKFdBxcdp2KL5IXIdUj1ZlrrGxV3zox
pOcInFJ2gbbVVVW1XllKUyFCNMCgAvcGpZPeZSkZgpBOzKiqB4hSUC+YNEoq7lZoBgppghu4wOXO
WXLy8Q6Lb3LgzbLqL5gtxehjcrMySw6GDZG4KOxi9My/3LxqDGWjYueYLy/1c/CP3DJuNMqOhc72
wxL6ZiDcYYVAlFxcS57y9keWwW7Jap5Jffo0wS5fJEyuLsuNpCWmL6Lvrw1LXFRcy24PUWpeXo46
BS2OYMvMXosvBLx0IzGUMkRL1NMHMsvpZddDmGFRhBl2RqCwSzUYOIwaRmmX0sJdty5bNOmP4DBH
pUI7gzTmWS+pXSyXx0IPVely2DKl9FhLjKl46rh0xFiMJeevzGCFZCKN9whlDpYXez6n481ArYt4
GFOORBbqyG2sGm7ki57ZS3arFH2RKlwcWcS4t3BjmW1OY9LWZuCtTLWZ6F3KbWOmiEuKS2oS3rb0
viXGEthHo4Qj/EWU6AxuVFTEtYP8GGnpdBLuYx6DiXBetzbNMuD0F6ZlsvqHrm5m5asuXUs7wbhF
uHQ10JXQg9bqX1EJcWC9+vM+eo4l7m0u3oAuDz1YSmA1lL6e1RMsrMA+7+mbXcRoBD8wSOuBuwko
lsUAmgB9MWxbVnSmCHaKsZVTyy9x1Ki3LZeJb0Z6isx0Kiy2XEVVS2X1tOhGGNV0qO5cWE0Zf8Ll
wmDp76XXS2c9DBDpqKsMS+l0dXEH+BEDUojucQ1NFdC4LNylM7RgwTqddzFzTCHRzLxHiG+jLqbC
WczMzz0XE46a6ty6ly+lvaG+g56d4w0TcXodlt1AYPRE0dYiNkB7PmwxubtDY3oY3w2vyO7iiyCd
wkbG8iRmaAvBzD70R8bxYxerK1G+ltzP8c30Oq5eJfRly5iEYdV67/lbN9Su/RhG5fQ31upbBDDF
/kwJMTmZ/gpRDoS/5BzL6EXoJl3OYwl9HoSpUK6aWWkvoOI7lzbrbCMcfwuEz1ZeZXQhiK10BgcN
1cAwPqefUcw0t6X6l80oWKZGLV1CCMH3SrqBB9wrtFEbjHFSz86c5SxjFvpcWWkFEuV1xGMuDj+D
0UCTfTPR10x1EpgSj2iJ0HcLjuX0voJLlMpihRNdKf4XLahlz/AzMSv43jpx0sgEYhY4GVLhdxZc
IdW4uIbmYTUcYMuUd9GnS5SxKhdS5YzF91tKjbRBWQToEKeZbLZmWQ63PUpYgNOty4ERZFmIeLj0
eWiC34IVRvs2JzvpzMwRZqMFqvBMaEsl1eGHfgWI9IOYYnGPuKUU8yw8L3JDYgu6y8yj5S9PAdTN
XUCw8IINXySqxCXGOX6iyV3ljojCHtjhpxFUt1Uo7yhL6JcCmWyyFzGqCxcQh0JYfBFDmZVdznMk
s146kehLuERbRH43eIVFZzC8tsp2XbcVXMqJ/AF3E6LDcNVvbmxhwdSopezovoKa9AIw5JClFIzB
aw4NUM3YgUR2JhIBYSqABdrgJaoCwaNzaVZbTTHXSwNpoLQ2YceSYmoXfSrJRLJaLeCKKIikCUiN
IypWiYGWQxNsqWxbg7WMsCPMr9sii66OM8RzCY7zkrQcxzD9t37saYGoPpuKFgL2tQtIIckrKJdC
5lCQ7uCCsM4MryHVNpoX3FG10wFZEIzMNlSBHdhKqPywqNWUkC0lErZNA58SstgS25oLI0hvakXT
MHyq1apqnuFADOFjsRSfeCWkMlPtfpIfgVa5CvgbGnww4VXwfay3pRkO6tPU7HoA/KKgwAW+3DlC
VU+0nfck3PVuBrK9a0N4i5avAFUuro1MiO/B4sGGWtV5MfISrW8HtjYgn3qq7zHhvc0R0Zm6uNYO
fUOjGy7Y+FuF7aJBiumuVhepQ3HbmcOIim9jblkwHn7whhhhpgo4q5fQtB3eEzfK7M3zcaOVsPi9
5e3qH28AvLHYoqOWN0uBrgfiHKrtKK1biKuEv/bUxPthEy69lE0HDM0RY13edFrBYnlqjA5nM7yx
O03K0NzlXmpdg/DhVVly7iFCPmFgpHyMrRSyDsN4GJcLNTUuLLlxtocx6zJ2dTcBxDcZSMwYQhC9
jhGk5GXerSG+IfDmj4BGrNVFR60VQ5EIWLJm89s6AQU9VFA5ehobIi1wkwl1KUYupk3v/MLJI0hg
icoo/wAAINOC1Sh5RZoIBuEaCVUnErxB60PC4kgKGD3DRMgxWRO6EupLbDDm4LDLhaovdWvk7jKu
ZzdSQJ5+nV/LQ0hbMZvvNRfZ62hxwzBOrdoi3BnVXIxdRrVX/B51g3nMBsq6vPLpnNbqXm5pxG6y
lIAjbYF5eApTawW0UyWhQRZWACTA9SNv4wRZ/jrewajsdr1U1C7794fgIoGtuihdR++0CSsSgUcF
VwH2MdFeyce2I+V9O6LQ31Vvu/c4FbV2ntRKXvtuXVIdNMlp6RDuaqbrFl2zJaBNFMytuE+TwlE0
UKFOaVPjrfRSwRCRizM3qghuA0AT3wrUXaOEoKbgtCu6EZftr+c1NJLBOQQKucJSsBobFs93EJ01
66EWOjzIzlKF3VptwaHd1UBdJBBwnsQ79whCZAmlu5sW5z5WWpQYjIsXPv0Mo7LwZuRvtLBOmOee
MzVOKAXd7WYz1RWz0SzR0RcGQw6jbFO/zbDfwk5zCAyK+k8w/wA8KUrBtKZLiYcLIsFUt34gKBQO
j4gFpK/GgEyVC2Atq2NJH4rU4d/g9qhYS2/TrRmhizgnqW4cKg3/ABeZFIRccnuK0nLATzS7rqd8
8axtwRnyy/BD40Bnerhz/WOnux2whhinawUNDcAFJwadwAr2r9Q3qAX9RS4UV8TkGv2RkaRao3FO
Tg0QwgrdSwGUMWxKVD75gZYXFyXRwSyZzFu6iEBZafr99gV0r5Q2o6iadTOcv3dsKyiODhAA9C0s
F8/c2KetaGugC3c6PX3FVAE5SiUxqD5G79w680tI4BqIbhcDV9KFuKvfvjbXqoDJSLXtJUHwwvPR
pWOdzhov2gCyFltzijB2feRZIAeyLkbJulhi+FOAezi7hkKrp03gxlmWvh5xcgt6M8hROMa9ylDN
SJaSKTH2GbR5uriz12CtFwVZaYJXcR6UBYy8ViOgdFJp0JdkVtVPUVGDrd/sRgull/1et3GaaI0x
Vf8AZPwc/pyrOP1YS1mllQl9mavnli1knnf9535b9ceXr9xtgDPYZpxj9wiryFD2t+/CwLGLRdAT
AnV0zKost5n64Cag8S1gDjR9gtJl2Sn0LlXmnJBd/Ayou5pxwbZ4SY25SbK5fADnkKQC9tQuYhGo
g+qr6ExuTSWVdZghdLXpLGYeTZ3w7lc3maX8PdnLB6yuIAgOyglIkaLNvEJHaex2rF0wfQ/az1N/
lvg/FqJVBa7cGPKbb4Spb6X7w5d2Ea1DqDCqe83khUWiVtlad3qAVEWc2vsYR5pUTczKusC+Uylt
FDMmkeMYDGkXgW/t0ICw0In3DuvuswJaTFA0+bJVLqXPGQ1HZoQ7pVbeRZUpXdv40mHxVZD7WEeI
h1uge4Vxmi6YXqBzdVUtmeCePMCwPRHDjh42sLRG9HtkDY7Hw69COL2o3gPjUuRKOQ0q6heZ9uWz
9zauhd+2iAqjUbAocw8+iYtsEpHmHfb8bL1qOA5I4AgGrXcWqrj9S8Y+viDTfglDWJtXqnHxHn7Q
XFnzfpInwRfmn2p+R+uNr6NVFXkldYscVu4IzVqcdyZe2CKOaR4+h6Q4z4h8L0eAb6bg7AnMasCK
CSmeBdI/5L7RSPpgp2V6tKCGD24SOYVBjDRTiuowNodqpJU1fv06xctW2cYyUecH5x3t10BTD1Qv
vMoUFFGE1jZbLyZUtwmqZtHK4j8e8zvMVA0ecDQ7WU+9ksHpnU3M2dJJxv8AAQkqoHDB+ajmjfFW
aY0OfzJtGPalIvL7Xsk1SAp9ioz/AFdxNjASvLSPlH9w6zbioudGfNHuPm6F2QvfbtSVodxK2AsY
6ACY54ZPjaLhcamCQrCsZt+kUqxoKcJAFVheh4JpAv8AJB6DZ0RCoAe4u/u2m7Jd15S0K+ooGhZe
Nl+qY04jQ1ApxxNZKIxnA6QbRWrFyuat3QV2Qd1odqvQIfa6nFVdt1jJFf8ACtdwkbVJ/QqKowMh
Ds4cjfadoRaiS+hzxYCXHg7RSozJJ8ksd7oTYEKg0ut5wVkmAHJ2PAEGVISGcNLRfmga7nfL1Fgh
iLZZ6m7tCdym0K83GTepDZsVTcSoCwB8EnEaN1g8dUtD3qtixdlixnkcyeq1BSOtKL3DMMXpha4T
EcNDDaRIKBxhFTvQuWKzwsX7jrAZd3Fld8TaVVQlyU7c9BFmfbXvL0VShQV5RolgFqREvDklhJXC
p/SUDFREyDhlDBHpeIJATEasuAoDsBUQLBzecwLIB6Kil3pWCpG5c4DLsuHcdF+jpfmcwlNxxm4d
XcVkaUUYGb3j+oFr0fqFUeoVYe0FV7F+YkQFXW4ZZS1oHL3NT1cCKPFFsstgkaVCNSwc3BYipe72
6LPS67+6AMAVf2acMYBDSA/cAndpB7eSD4/+daYZ16hp3ww196nd3Smb9svb9MZtW1VV2rD7+y0p
Vgi4Wj91tChZwWwyMWySKpjroVihl9Q0KSLZTLlbWby8hsMnCAgkFaY7KRGD1LhUrdX0S5ZFKwvB
4IT5d1jtojvL4h44bI6qj1C4bRVDsJWMjAttLBQBIhSjSM5J4aEZ0qqpEqPIRnFbWXbVbiHibvhR
e9keIodhjzBJkUYJ6uBoO25W+oouxVqWVUqERgMacuizeiirEaT1U2bOW1cq91YqWlttT+ktKndr
3gNWeBSNrYuNKYgALWsQDQAilQdz9BEbA+osAYCBqCY2CKa+mHLcVoHtcYZcowliJuLNS5LPMQgl
3Go1sRPY3GAQZjel6FlqbmSWs3LdS76DXRZFnZDraMufMxLgst0IRmYOY5T3B6KVLxCFglSy+ocS
iyKXFQZcu+gzKy8wMw4RxE3Bply2COt8QuW0yswAGP8AaiRT4/HQOAy0QtYySw8/8ktJbLT7XqLT
CCFkEHsUdiJdAbsZByVldmOtUxUr4diRvL0cbolv4IOUCFi1+uuCJLhBEsbIDV8DMM5g40i6Gju1
og3kjgtjAmqzW/KRDJAmLnXZ173UoFYuK2XF66zLYLzPMVFgstgLMG5pSEXqtZT2l9FytVFjDfS4
PTiXL6EZdzMPMJRdnVctg9B620L1UqEEEFCEYS2XccQZXaBBmehuLSqgy66N+IQY/wADNweg9C+l
sIddc9B6FHGf7qNj6v5qKID1EsBaFhMms5K9zIIDywUKJS6vJAx8q2RVmbj0KszaVsU0Q2dR7Re3
h5cbgHnIbMVtpAGHbQr4kH59u6hU44per8TRxG9y8kGswhWY1JOgU+8tI0gVkeDWFQ4gtOuItP2k
3ZrUW2VCt+AiHQKEHcYurKW6WRKa1Y5ziFuCXK0LPHZiq0c8XqDuVlDfLY2swWwHqYZHarFU1btU
Tq7rYAcqxW84C7jStcEFtXeO6XpVtVBXpYei4GIzdCAjBai1mkwQJC82OmzBNXquZF6OTwFQOeXa
r2GKo4a6IKsqyXFxLg2Rq+mIeMvr+DNxlpuZqZahHC9RzKwN9FhmcxgnS4ai2YS8y6bJdma/gfwG
MNy5alTPQnFXCYuWsKj0MJFzBohYlt5gku356FvpcHqpLJcuGZfT10LlgXLVlsFuXL6XRCrixrRM
l4fUQ38/qUSholSrpEZdM4/fAyxUdXDcfe2po3sMWqK5WqukAzRRlj3mzvs+45rI2+TMorO/Imxd
9J6mpS5tiw70Fqqs65Uspw8EzN1PCysIP1z/ABrCH+hmtRrs3KlwRxm7cOxmkvxcS4a2f4syue8O
vN9ML/UFyVr3xL0l270ehAyrHZ6eYSVi99OWVxveBWK3C0Uvg3zN+CPU15Wz4zkCy3OFiwJjTJ4+
JXUDNOQrHfff1RDJn+5EqBGavqhnOmqGLECA1cfKyahI6C+LTF+PcbxSO7WFN1u33Cmso6UjUaA0
Y8g0y8xAGGIqgOnEvHS+t/wP4DBljEy5lsOmWWzLCXUVSrvmYqXiWnW/PQ6mut+IOD+DLYNy+lsJ
dTG5fUZeYNkzLYT5mbqX1vEW4M5+YFzF30cdCKkVrcVdKN30uX+oNTzCjLGVBp6DPdafiUv1f6mB
HiUd5ruCv9rKIJJHtFBCpSLtTS94jNjdwGcGDV02YChXqhRg0qJt+3FW8BM5gyO+lIOmd5bjIb7Y
wLxp8qYc8w6GFKVCpvTmwtmoGwReQtsRtlm0s+aiVjKrquCZb3opkiChKrk0wlIOpdKYWbKbsZKb
3Ye5xC3dC0dJ2/JulA5hXsr52QzoabmPHHbKEkHWgDkWSA9cqtRr3CWh3lWS3lG5OBkTJNxRO2ho
eGM97kQFxX43DVG394ThPNPmi9qUee8rN3L6GtCk0ndlsug2mIDF6JqxYA1QpXaKhWJTXTU29DfS
ivP8Ufq5YM5gn8LC3iaqJzmXXUuHQeldQnP8CcSjZcz6HWkG4Rg9CWvQ2X0LbE46EKh7nPS7JqX2
juF7hLzDCXNy4VOYMGLorCcSpdNN7gkUvGunN9B4jqHRmkmZA27v6iM5y3DNjEuCoeYaDt+5Erco
N6eYdga8arB6h9b5GZqnEagqowvIkeL5ZKF1HU7ZKt4dS7BVSAngCUGqTYUdu9ifjlrTsynaq1YK
FNNsdhEnjZyzxGLceiNW0D+oRm81eB7aCMy+iLy21sXMFd7UtJGiW9WAqCKMoIeD/K1VGYCRKK2W
PwRP7WwrysL+Xv51d8YXWWWQciamR4qDvKwiQiWnTxZM1luUeCV2P1ZT7ZbKKTwAqAKQYAO71OIE
2l//ABOoXqXDUExOlPab64/jdy8BKWKDlJssZRiXIB0wd/QW1MCSnUHsszwv9QGzIAW3RaSt9Ypx
azX2Fi9B0hQ0x/qz84zcJ8AvwyoopJH0TRWGvZh3uAllvDtC8G3FipgAUExSMfotDX9h2g9t1z67
DHTLdOJbc1LqGutykKlO0slmWbgQgYIsGGYYl1La6BhCFGV6XUETosMs0RFSlNdGHMobrp/U7CUN
XOPua8QJ/uZmHQeq4haiExCUqWsM7elp0UFrUEtuoKXkd4bT2x+kGqx/v2gJaqmhX4S4Ao/+/ad4
loX0/wDxMb6Uy0YbW15dXCIG4TIcVTna0iv7EqrtUVGBcWaShxRYDyCVFDC4qaqxGxUt2lyrlME3
RKsg9CKOR3lLaFkzPW2zooh64aXWBpb3vSeoYVfGXyC6jhABnB10yaz7WPSwLueY7mfYQCrKEtEG
+MCxuPjaVd8rUx0tEA3lqNNcl95MhxIUaDWoWiggWvUwLYaAe4pEDS7vhIapZ+SFk3ySS5rSu/ML
C5lMtCyupq9Hl4pfC5FWF3UccKNVEWlYr4VbCB9ZqRHckEvMLrBV7qXILHpq4S9Z66DHPVFg1B6P
wLL6tDyqk039ynvwWmd9und6Et+yxvSZX1BRZxCJiaS0WZUAgBuUZZL6kxWoFtTiDO0wIMrzBrEu
5cvrcHreet1O/Qt66DR0DMOXy/ES0+X9RKgWnzDZ+YmEDjMx1Wsv5VzamCZ/tvq2Vm1dBlg4E1cW
2+5WqgexNjvs1AnuFZdC2EM0ats9qlXD7AtMr7qUVd4jyfec+8BRdriEwgyoS5YRmtetmPFIaYjh
CmSnEkn23POUI5uo6W01crhCFme+Y4t/Lgk2UsRi5hswC0LDoqUrt4nDp1ir2l5a4b8SobvrWZtP
Csk4MpktWjDG3u7f28MYZo1XcrbpltU1Vde1h2Be1od2d4vH2BpqgkQwVZqEJPC4s5vEhDqdVUXO
bK9eBV+IlIA60QcKkfRcPkiHpG7YK7vTfwptfDoqTtGEGfyuwyt1QdRdVLgNrQCsts9MJM+bo9Ed
AnnGRgmUStWyShokk0bde8u37xNyF7tLxF064MntCKi4jAJopexxWq6Xx6rqGC2ahvfGFVLyefBC
uuU2rMWWCwEcQx0C48UbNfhhv3r/AFy41R3rzacxS0WLmK5qw9hoFY4oVTJ7jiKtT7ygjXftJBHa
hONrg7XdwqVhYB6/XzzuOzYJdW7qnETdN/uiKrDLgM0lVTOeSVxjpARdxweklKdBn1BBf5oUzhv3
1mS6imPjMM/PH2JeL4vsGGbWVVwRFTkBNWnfQrpUErVMUtW4MvEJdQYPRWUzucwely4q30GMz0vX
QU5i7zCCXLJUJucyxjBqXBdEq7YhXywFEAKxEaeINDtNnL87bd94R/aZPXUSnEQmt6D/ANou5YKE
4bs+oUD1eiyoDj9+GIzBY24D8oNNEzsU3N5ZAjvYu24S2IbsWLZGFAolx0kR9oXR/wB9bhSzBfiK
28wdJ2/bK7tiiPjNQHNBZtphRBpMTEqf7uMaRvfQhc5/ICkPtz28RSMGoAItlbl8nItwfk1wAq1j
uJTztqK667XVp+orNy3g5TlK9qO4BEcflO2Gqs+phnMU8uF3p38Kgj+rFyT9V5jVty9s1gAwKKWF
Poa6vFLvvKe7dECp4/qXipY4iH6QYhdC6sUzRTdZHWGpOu2Dpmsb3p8y1FyrTuuejqA7tuXZVooI
WeSPrSok+9FCFzYAtIcXBnnUKVdSWjwaINPTMsJoE8+C4BO0YHarzb1toX8SMYWF5Rg7+E2Fu9yx
sqDGumCouzMKfu6tsKLtY3/3YCOK2i6p50IbTrNS2043T30bjwFlqx9EHQeW2ajTm3G8WG2Laq4M
lRh3d+iMSkjbwPc3tgtbuh1ZKvoQPRtqDbastC6OaIXRI0UK+bqyg3b5hzAe5gyfCMYc6Y2ZX6Gz
gRYa4ZUMrpz7tW4T2uvaO5IiSxatLcSKB/Vva0Et0fTvXEaV4GVrkvW2XBa6GEGPRvowcVL6XNwh
UXxCuhL6cdClzFr+Z9OirLmrYxNaZY04WDHxG8g5n5cJ/q7Sm/X82+XdqEYWRah7eDJDQLjw8DTN
Pb33fHCOQCK2ASs9aWiDMLFhampqdo1d/wDYpjxmSte5c1RWEQ4DOhb/ABdRtLqLNMRQ0zbIcwvu
nT4Rc41cY78798VcQ/JNMFHmgGbUarhQLL1GM9a/Xh1yfmuS51ojzuvKkavCtJSzRQNiXjrFAHTW
17I/tycO6caxr7MrldrF0OH8xKt2d0Iwb0WLt3BOM4fUlwY7Re7sQjruQqLu8GTvPKOhIWCqhRoY
fCAy+fcMwrYWG7QXfA8mpSAxwtqkWCpTjo1uC4dgUjZzFUubcAm87j0vSoKzHe5RTcoHuKngUEVr
0GLCj/tuY/jGUmQfwrAp1FvqtizguOs4SpSjk8aHqCYUuhf2noQQNF6BpEgkC9KHuM++QFaDZuDm
FTg5GUXKYF7KImC1S5Yx2h29HvUKWa2GeVibVos3cTUK0w08jtcZs22qqtqsCC2kL7ILnLltXKse
ctOy2mu5GC1iWM5D6IOEjaNQWA2CFBRFENLcUMRuuoo5ei8S1xbY7gyyVfS4dLi30NwTpdQZcu4a
h1pOLhSty+meJfHRhD3l30vEBRGmDBe8orLC5rAZj9iat0pEuD8MyqaefpORguyps0KGRHpWI1QX
undMyO+7xq3MW1V14MsVYArF1BGfdFRuxgLLLOg7oy5M9HBSJCsMVW0SvzhxmgjjNrqOMfNwEs7E
ZtQqYqxlR/GGyIxtZkIzsRsK9VAvIQRA2IkV/wAuY3jpvguW3NtwrgirwAibmltbvFSdW1Sr8sAC
EaRqFEdzQFPZLoCgOx0U4Ag5nqTLcVDvSzBaiLC2otsqU9mZNkLY1anPaKjf9trZssqKeGNRlEFg
6Nb1BTiXFupVy13cbq3eLyrEnqF3bywbj0DNS3qst621NEuGEVlxZfW5cGobdA10CkuWil9GRr+G
elsIy+rcziXTFHiWwINQczBzL6JTp6OJYnS4JILWpeehlZUMYhOeuYQeifni7ZkMcGjmXSeEenn9
6bxeWKVMtqrjXJHUu5mX1yKFEETCJyMK6tD9tYo+MYzHwBcVoOZVuvRV9CJiXDproOKi7lhTdQHS
nIxFNJjjo5j0roLszOIWf6L1QwpnpzHThl3gAS0Pr++ghFcVbdSLpQxKFTuD83tOZuWKaAmXnfEa
Wtk2w+rgjdnRMlxaT56JX2WXdXUqMJD7X6tGTDyrLVEwVvaT1ivUlu2vd4ns8XQ5Gku/sKtOwqX0
pNBsLBXUu6I4kdRL3ECte2ZqNNfFZwkx1jFQ5mjQmbzjc9tS/VrLn35aG0riNka6DPaXLOlxcQnz
0XPmHSz+Fy5bLdCCJYzE5Ie5cxLlkIrB6WRWXiDLuFckGFGmKVLCWQEsuWcS5cuHQX1Fj4YRTcLa
qVXUqXmaRq5UcNMAHULuq5ZSGUeHJCAcL8yzvdhtSeInQHLaptqNzyRgvl9Yqc7XRlYzNqFD5SUN
VyxY5Y1N3FWwCyNCI+xAo1GoOzCriNobiqpV5MRdQ2vlwGQWn8DLgci21sZ5pMMpAjmEZvKaiHfH
EbChgWlcCJtmUXGOhFj+F1uE+W2HyhHeHagovFTNN6GmjEZQlVq81XDMi9UlhdxvEybfTbBSA8hQ
3PjTdrqZXZ6YuJbAH6W9dTc6pHywZaYndNWkWTN8p7p7cKOPnMfzGAbzBqiN9mVd660Y4bIO6ljM
jj9qH9TQTN+EHmWrY9bqI6afjiYgQbPFtWS88xKyIGmFQgVQbu6sMMpxV1QN5DEwgQpbPq9cyxiO
wfPoTKx3NJqX314W9XfgwpyiPhy/7znuxgezZ1fbJgbtviNmdXPbxUDXy5esv6jFBPqj7eoNqeyX
GhQx05j0H+Fxv+Dq/wCNw6HS4rBl9R6DLX+Fy5xLejcxLhkRGEH+Ysucy6UoilUHUDKmK4ziXDZM
zHaPFFIwZWu8Rn2zIYh7yaruLMf97xTQo5QZj2plfdUqiqvqdomS0XSFR+sQFFljMu99bHEIKC76
VQH9GloinUqLsjaCITWjK+dfAXU1345bfAKZViFju5RX5Js+CFxz0WwKl9pHBhYjlEBOodtcZx0V
rVU0iWlU1RMmLZiSO/20peG5jwzra1JEDhmxiNlUpLFdWKRh0/6XbFRK6/jecoq/f10N1EYt1Ya7
S8BL7YKomCCE2j7EbS/48yKqvUWDLG0E+zKIvzCS2UFo8TXm7xbS1p2s+pangZhzk3iKyWhmwvKN
wkRt1pG8FsonWpyRb7J7h8xQG1xlsuqeDAq87ek3ohp95UircjXBz+Gh5szgJfLSXXGtyyzu1Jy4
yqv0eG8thEKCLsp2wgNFIzJ2QARyulQxSX4cp218o4rxXjZFNlpzjUHNaLjd9dqIED6auFTdcXGV
hxu6kgKKUFiz0TpzK69+mY3XWq1LvoRi2Ef4YjBhuc9GEZxNMFYs2dDUJWFhfRWdzR1JUJcvq/Nq
NC8wsWoMqXvpUWqgm42vlM23rMqRBlPylw13pKLP9XMj4NUN4IR9R2wWekg21vkrsdxMsfkrIJVV
Qy5RRMH7R5MznURxpsTPA7jJtJrZAjsIi7WG3AKKLPLKwOK5TA1aBFsR7JFgrglZvDLLRsd/ZVx+
YGM3SP41B308u6Cb1geVcTotAUvhFI9cEIC+GFievKS3lbY+U0LTkCQpX2VRcAALajCvaEkhNVCt
MjXZD3td4x3ad1YE0ygGujVILCCT16Ix1coVmJifgIosYympvSuzOXViNh7vlhFTpDnvyuUMu2nT
98JLNMvbaL+sRq4V9uAgri3NF609oHBSKrIhqru5K5DkZQeBejyCxJOReparMREy6PYRIkXTRYHt
I2yquR9uYidc/V8Fvd0S7BGK16ELvIEgsN4MIt9X/YrGGNur8eoUEvC3WRGkYTNqSiTBqXMcNI7N
i4Lba7tXKsag4qUJQ7WmXA7Xu6vF+olVdvWyWQ5lh/JfRlv8ToZZQpY+pUDEcw6HS6Zu+lw6cQyS
25qXLWut2i4FUJM41FP4GupuEWKsMUt5hQx1lgXl7yuly4WzXD+4KVXzLFmcwVHeK0Xf+eUUo3QF
Ulkv/o4aNS4Fv7wcSuIDYRwJVfbsOSo/mL/29aWqtlpkH2iitUZuxGZoOYkxeUA4AJW0u6H02hWI
hFg7gI4aqBYpNt9ImC+JfEMIiiQ0XlBP0uVwb35m0SkKtVojkC2KdoplMFoEQaG4Mqs416l8eDwK
6s6NwlwumtTWpcLe+hGovkjcpoSe4DZcHoS4sBGrhn+LFxf4MSWx3ctOt1Ltj1oZzHfRh0vod6iw
6HXTjpxXQqam4S+ly5bBgjF6m4IMzBU6EzKqmydkXUymes30Ou430spm0yrxFxLiFjE9NmMUnBK2
V8ypDZmXlKjRMzTSSuHtfTAHpQbhK6OZnqjLYR5jrAaObRK9mHV30ji49L6Uwl565jYO+4X26KMu
XKwy6InOQ7RKxknMSNFLLmBmOwIRbYTtKjc750uknAcCnwMSAhNb2qKMrSqsypIsxuH154bm8HdL
InS1jkNQV4mmX03HodFlrAw3LIvQqahT0vXTiEX0IyiugzfU6LMdRx/C4fx4glVHJFSHR6CwW+mo
stLVzOYEbLUuZhqEemyoAJiLLnJLzLqCzbcG+JuVffv9wVjO/wBxKphdqyy8LZZ8Mxmbg6FXNoM3
qInjBpFcy/AlvaapVXwE74aLfakqwdzjzDqrDWDvDW+A1iUAorYoD0C43q7lSgjYQpRsY0TcI/p2
iNt6UeqkvmBZIjMJltWU79pt6A9SyqG0qIWE7tMx8PnsRLFGxiilE0LkzgUTkAuvUG2VqoKuMG7s
zLYCsO91rA7RW708wUKOp7Z4m8CxAFmG6ppHC9CWkFR9SS3GjVXCA9yYtkovrHfOpAWTxlBoxhVc
i3oCUxRUgBqTDdZZB2FoIxh7tPYvzEt2b8YekR+4t8m6EuUM4lwwjvpvqwmZcdzf8Llw3Gs2yypd
PU/hzFaC8Cj+HPQvqRZvqTCHWof/AAcwjQQIbidugy+OJUcS3qNMvBCVBmsA8w1111Ah0eo6D3fu
ao5p/ZLpRlqfcAer/cACv+0VetMI0zxXIe1HgS8iqqg+CD7bjYqIsDVAq59xW4tk3EzUiQDZmKK/
0a033rr0bCR6Kuh5TiAV5VbFDpqv4kB+A24FXyMsZN43MvF+KN68Sui3zA3fVxLwV3Ld8WDqZvfE
O+Bk7J0BYplWVlNKGoXVaQzageXNWmZ0UuuNigbBd4hkG5LklaHWyMdFQCQ5sXUe4RGECphuVntg
+dj7yL/adZfwT47t+2HNXenvNIXjrQIx8v6cCeabJhRZUYl0FRCg5lVHutZtpjUUC+2GXvsbmVXU
rnpUf4sJfWpmEz1qug7xv+FRTLUadRcafyCVEf4HXP8AGuiVCe5TiWWaal9LCWZivQSpx0IEtqHe
LDxBYaFdCXLroNx6FHo5BqwQhscfsJldb/zEqyKdO5Bl4/slM/ZLw/ONtw/S5+T1K4BqqvphDldv
c2430AbA6HrtFfjnT/pri/tUvcwEDxcO/aJGUH16vFy7fRSgZaQAm8NQWybstTaDH/BStDeQP8pe
3G6VYFa3ltNvT7TjxVGkpbwXgTDWygqgd+zjW1ezYwdAI7Jl0w4rr0pN0E0BHxbQdcs1oq7Yah4i
RcSwOrdzlfxlNvvOVFovS5GEzMQhVW4rn8wQkq03V0VBTX0X3YqLE9fJwBSly39W03XqTsg1qaN4
wRs6K+2HV90XybrZXQhWtZ/xHsQ5xxtqLVerAb/iSo/yejC+m+gJFuV/AQPSipU0R4lwviZqAigF
SRBrs9Ag1cVmGUxAMBs5FuJQozBb6YdCVdwFalSqZS6IFbjZCgKWIeAI/NdBSikSBKlhuoGCJUYU
KqpNwp2/hzvoQ3GWwbSY6XCARZb36DhhVzlFL6UXLgrk5fiKutwrAnf9w3IN6Nn7hr/SzCgpvLPm
oyrQ31XtADKS5bBBF3Q2w9tSSsRaqruW6iNtL0+s3LCrUnLDX+4dWKd6jIFqpCxIIG1u02om9aBj
7BClu2uL6jy3AoTssJEX9wAUpWIO19FxDsgci/m5YoqBe6X3l5iLr7t6uAl+5M2T+CYHn25q9Uoi
rbIqESpmU9ARim/S1fMzbaVbVyq8suX0rUUvtNEjM30SFssItilHcvcCoCpoC1exKUktxajnRKYH
QrKJqLi/bbAezKZRBwO9XZVtS7XzyqARgmnTHqnjpZ0Sq4ikuZfG5laY2wXc1TlrAGbkm6yZ0maO
D4ldKeCYBo+Zlo9IULVEoNVzqU5Bmokuw6d6t8SpoJV0Eam4EMphT2DmiN1CtEjeliIcrKojQ3VJ
MlEFTeuCs1LLtaONwqgikJ83alrmrtOYt56+UcF4MbiUvMKgDlf3/DBAvGpWiqMx4GXzY6lN9bYs
226CJaKtl2HibOyyoNbI+CbDL9lSC7xBV3WmxH8b12u5DMnW2tuAc5qxgbgmCBrpCTcWLsOqWDrA
geAuag1fIRisvmMsXMi52ls4OlruVBjZhKlQJxL6GGHeYM30bS+KmqtNPnKcfP5XEEC1ZkKSwlTp
tdM9uh0emZfRifwI1sRIxQVUMaKVHcIkzcphLXMPU53EeUnGVZKXTCKKInsxOzZ0YEM70BdSuvr5
aaikFC1RV33oQzAti/sI8KC1v1Ah1QKZbSwWZNFKaU4bCHQ2KsY+YS32Y/knLZXSO6aCMy1z8+XY
AIgdhoAhQchqPKqJfWqyiVuJMT7/AKGDq2qFqVTlAq1oxFsIcRSc4432TPnLt/mGUV7mSyX+W1RW
9oK20wnnNri9MVr/AOVUxqTBu4IEpiPN/cVgEYhsnC6tqKyiNp40UjwvdaJy4H9PBDGTLXXpZnOQ
bOyRnKso0NUtT4EhbLdCbpZZK/XbSm2DkDUpsDwLEgRKOPK20h5Fq8b2CwJAObOEOWplC6l5HTOf
30jzHAMA7B2EAqtwVKDvc7fZUWVhsmzAol7tjtnDyNLrbS2grS2x9xPbfiTdYlwoMuctvMQyw2ij
WpQ/lfQln19fNcK8U+LW46n6ISW4kIp8sVCbhVVDcb1jBVXqBkJVb+3w7ex+nHGMU7acwbvnLB6T
CRaxUT6QQ1h72X9BcCnkSuNNXEDmGJfR9/yGZYeo7ly+iy0Oq1zLzEopx+YKabq0XYxnJZ7mPPEq
TU05UFW0TOzK/MwTdrBwexhCMv385qEXPVAfSzD1+8HkibQuGicvvDp+6Y+paLbHoQFsZaqaHpDh
GEnQMsDYnginPV+JFQbJXMa1pipp75i+lOZWboLfvzc2/VWlfowcXUkuuzMCas1Ray2J1VLwLmm4
s8xNmmGCGJkl4CJiVLd2d7a+Gd8ZJGAqRprFVKzFsarnFl/EFKogQkUUtJpdpN3zhAxn44CQtLDL
Ak+q9qysUe/VsZUH8QCoygjryI0i5gTvhVd3K8wf9lY4yna9fbREYb8b9JFbeY7soUOMH8mXbAVc
9AWSaQnWVaknG931sVqw3PmPwxH0eV+8Ehe7OQ0oIvs/HK0eJ233f7pGToTQAFSonNV1lRS4pXyf
ppsMqn25S3a4ye9+kTTiw471EwpEK+6KHhN/SO3LMImsKvvPPh+cn4kc6F+VLdpBBVawpBBRiduY
9hGO7Xx5VUT2HBElRAdN14HLcf4ALSyWM5zyU2judFTlzN/VmgfFOPv2uVGNqKOGpUBtGkzNWd5n
qVlZcSRuJLBzdVDU+wljvSxip7Qi4YupHY3vaRWFNKyN1EPa0dZbs7iUXbKhdRlTblRM4Bnjvaay
/prU3FRuaTYqmbTRVeL38xAT8V/YVMZG/VNnaR5fUt2irslQ9A5U53IaHsRQtWZjvoGJRz0rUZbG
EHPQWAvQ6MouB0uXMK3wD21gl5VX1J/D+yehH9zEu5KHDcLH63TtmlACmJd0JLM3zQ6GljZNIxcj
AzVtHUrWsE0KKAjyvnVbmYBsARUD5+ENykRg9j1D9mIxvELjrtqawlCAqoZwBLThzidtyCCEUdea
ewUzFMyVqjREwM7F7uNLju6uBBg+/NcPk++Oelw7s/3UeWEdez98W91VP60ZlNtfE16UAVZdUoeV
LJPASKQjXL7hZMF8QO8/mmv/ADlpqIyvcypMpPEJ85/uhqLOiWqEbmw6UoxJTppnrCc+W6rq5e7R
6srJdSsBlmxvMyy+xIhVUDVLnq1KOVRGotxktQti45ffQ2DDZVs0ftjWMtTHHdwJZi/pZuuV4UCS
9XXsjCddHO0onGykImcrPHagVp/ftXslaMlandUQOuhKkBpFKjtY580R3KqFdt4I/TOHevbB0exk
VbzZoec88qhW7AatoJiCIOuKeMrtwih1NJ4Y0DurmImA2kEd8+rqpTR4817aqPEOXO3XyRrWaw/z
Zi6SHrQtWU2s/v4FlMebkWiizcupgoqhPIBn9SjDZqGKfYqpe3ylX8MI09FPTNwbAWu+QRav8Vit
AcqKtV9k1lfcCE1bs7ZXtWmlgywD3qUGRbC1KFAPUFaAlW0TBiUXjHQGpbcVxKcXcqBll7NQu1Fq
mtdAuFAArKTE+blXCgu5Yl9F6N3OOlt9Lgy3ppBZhiYgN25sgrlqlisF5/tFFjDeTtGNUl5Udnns
iyJkrytiudv4jT5pVV/o+go8kqmRAbWvdQUiEprsm7CNfPfZP/W+PQwjAm5rfFRBDZ0VQMv0huXh
zHfdkFfMFteXcagdHlOtstdRtBCG9l8i+PUsP37ddIwdNf41LeF6PZVoNXgBzcIWNTFi9xJRwx3q
lACtkPntQygDAz7S2tJAdCxzGw8a1XKiVNg6SUVAug5lszQ2iNiQxYxOsa6XrZdXwQuR7bQKvGKN
I0ocZQjrQR2C9SPtzdCjTRF4/axekiEaVSWGADqqix6xxfbKqhbJRhA9h1KYeEszUAAB0pEHyQLW
As4ju0L7xTshcyb6aSq0S3pBgQiMYwJahqUjmGErswIF8M+54pvFl9rhVhYjBRz23LaLsn7Sq9QW
0ZKtXvQqoS4puajkupL1caBResS1q+++WtXBVxgkJvVpMRhXBD2xNzULRAfO8XCcndGrqv4i8oG3
tEZBjzMblwVTVOUJ3ZTa2iVTpjdVA9IyxVifEbbIAWqtSrE9iglc9Iyq9nyQoV9spyVDN4zxZaYh
HOuzmcZUNxD6YEt4t7pm5fEqLA8hZHA5mmglrO7rP4Il/f8ArBRVj9hUezS2ZEG5B9ov8lnex8CH
tWywEZvmuIy2UxEYOZeIah7mow6XUXGI4IcTsPzE8toe7u/lB9D9xUZXELE1PxVPCbHuRoj6S/ZS
nFW5HzaxMJIPH0xnLottLWJZK+yrhkUWm441Go9aUHwkYvFG/JAGosstox9WqPQNuxvNy710qJBK
SAFBMA0XNkNbnuK5GJlKlMElYxFHUrvMy+BolmbMRGBGNI0UKX2lUyilXCFbp6V0IuFaYV2AjCTR
auizeCEYm6geZC/8Q6UYRI889uwfmF6AVbHyynn9ajutBFuKs3YXG9UBwjD3OoLgXitLJ44MSpzU
tK6BaQUOrVfGXKvu64FWx3uCElNXpC1p2gBNoFScOIwaFo1yCKbna0C6CyzCQHdjqWIlpUsNSPTn
rgVtglXyRKqraSAb8qV8ucTDu0pXop7cAJea4kuigQLVGndE16w4OjVCyd6cpOKE71doVFLW55VP
YPP2oV0fpow9wFXrlpeLocu/+kqSphC0+LUM8uQt1qhu1Yd5G2PF6ea1r6SVq09u1tQCj+qVW5IW
1EdqzCVLJeGOu9tdRRnYslTPGFE+uo1WMGwNUSLzuCMykCaaXdkcPtXa4ze2cDXdFhvTFdSBW124
aIN63ysSRTBI1ILRfDxpd49ovOdUuAtrRURslnBASBza7xdisNSVfZB/sQ5QwYfRCvqDGmw08CCW
CmZbpLRBKEo2eikki8AfcKzwQ/abXUcF7LbDHdKz+5TLQ8yrvgzLDTCmRllPQbI2S4VGtj0LrpSw
jbyIYUpij9IlCY/4qWs8AXA273T+Y6q5nhLueuJqyXkUCNwCrEpjyNmxqfUVZDYCJ9wubCCnpXRT
cqiHecn2Dc7KmocqB+FpB/BPvAAEQrbaaEdb/QTuMG96iPqDbKAJu4D4MezqyBoKhihpokB/1b6Q
eFirKe1Lb1lvMLttMk9xay21yjLQqdxmVnojqkG0JxA/xu2/UJvkazzkIe4HupOcDndnqMuDi2F3
v5Ai5VMArlrJaoywixuFZ+ohKdgh7UjuXo1mJG4uyVpvKKgnCd5RPEb4U5iQV/sQF53fgarstj7J
hK5VifCZhrtJVgyX5e72xKLVa3zkUs2KXUKnRFLLddyZ/apAklVzdqaPhl1X8BLKs7a9ljFAKK9o
dDyyrY9XVGPQWK3c1b0yLdK6rhcdVzO9qYMKviSXIrL0bhFoPhEY1ftlgAJrdNc7SiSd3Rc1uiHq
/Fa6Ze7KohlYCeCs2iej6cpIV8WvL33s0Yn9AYRxuS82WfhL/R/eIELgfAmgvD8UlBeIxTyfakCG
6fkhF5zP1mi7XHvE65+DD4sdpvL6lpe771i3F6UqWWY+fd6mkJllVZfVRE7ow8T74xPIBuEA2BL5
ILeVNqHDCAOK1RKsGIYfmmE27SlqS/LLL0UeJCKsTzhlmfr94X6YEkS4kHy3D7OJaHw/dBqwWvG9
mAvWzt4vWI2u7ZLJXwt1Uuv7IxjWkXPFtJWTKrks0YWVTTLedtRfut6HteAgK4EiPLUPxc1WNTN7
00U/dAuRbJnZL6LWwgXcDJsxBYy5z0pnEt1BMdoAKc6gNcjh5cyBwH7hAo2JNJq0H7lNbZN2H6t0
JgEpYusCp0daQofd45UXEUCj3I+0yj6GOrcc1MUGbz9+UwVe5AtIWNSWXUBWC3V+5RQwYSIu+WPR
IQF43bHcQ9peqaHtSk9DK771u9UMyhyVLqlK5RFgWERDbWdox9xL1taQuRc3eWYmaarR0kzKVLnE
hghSFKgD62bUuU9BHV6RbY1nfFa4jY3vMzUPSkc+bgwA4618ec2MNFr0MWCYNRZmPnhHbnttblCl
/wCPjpwkAyvx5LODXJh3vu4DjCR2Crf6Yj2V1lgmJiJx2SDi4BLqy1cHc08lXMonOtXmEIney6rS
GEK0lCeuF7vw7tgNit/EXbmlieqsl8kqd5+PFc2iV+8Rqd78T7WAvuR5Jc63Ea0Z14Htviu3C8j9
Zp5AvpTEaDdcpiQdJBxDlm4E55qIZagFZ00o1jsblnBbV4m3Bhe2BmFhB0N4YIRRPsqizzLGjaVO
Y01jU3K1AobtMSqn9mrG9mYzDdzK/gHqixR9at4hJuM82r+5CJ2ouLRLpk36NrBdtX11daw9eBrY
shn5y7Q2dXSTHaNjIMyjvKXIFBLlm8xLyht2NbUNRcmkrAHDQ1txS5iMCDWfYJeR0Y1w+Xxj24ZV
jsyqb6niujFXd9Ow6+BG4nJcT8Xo6hjcHO0jeLphIbwvAgNz5r91Y6VLGExp6IHiO1E75WVda6TX
LwRGLL4E5DkYfmSrj9zTO61Awpd02PBVKjNbPLGrCragt2VC+LaattqHVa1QtBJRX6n7laIvlllO
jD61jFww1KD+iZbL0oXdCS6O86D6Ip9FQiZrKK7JRry5ebWbFXQxe6I7cbki7w4uMOi6g8S6mR0J
qEKMXmWw4dpVX85YbYV+GSr00+7wBVDqKGjNRAYvLHf+nnAj6wki1TuJlZ1bivi0cv8AUrW6f4S6
/BGDu2f2JZ7wwGPFQIvy40BRB2qCbym7qCwFZPBGTKYeFUhADqr4SoiRRGiG8tVHiNzy9Nqv252j
JnmPuwhHEtq9uJ1t9kkObdfUXFy+fLeYfKg95X0+JKY2NecZkp7fvgPLnXyD6+hKOa19E1CYsh/2
Zbvo68sQeq30Ms0uGXwW4vZSQu3mRI1dzRSLKW3L8xnoZhUWKTsmRhgVoNzsH+d3sLCHLG2NT81Z
bee+taHFv1B9vdlbbMtWkuNnYtVtu0NKsBvR19R4AW1hEDatebP0gnP3j50Uleu8qyok4ubIrdW1
Jfb1VxSheolw6qqoGfcKFO4qyU7unsm2Cc3yXWnhH9V8SlvkQOuXQjm+nbLgJiRFSi0MCx2jBGO5
Oa4DPY1HtvGkirzLZ37eYWAKpkBPjKFA0XYjWVIsmXX7wQYa/oA6WZ3bAJTrmDTRotjsmbGH4AOw
0Et8BalB2DhlNjypf3SsAAj4TZESPrd2uWtTttYqkVvjHembKDG3tZeVq1iNIzi81iftlrnUNocu
hr5ExVFBEHIGGIcCYmTQt6kFB2IAzUNzW7JXiXQaxAly7IMvVQWqvEAYEail00Pa5RQxtXbnxKxh
s7xFN26gFSnEGuOgKvOiAiM6A2+yEMe6sbwLZZDWGN2KQvo7ijmU9FemmXLiyybjEHMh+ZPJH8CY
EK1u/m4x7AXAS8AS+adX+jnKxavEiRjbgL2Y2oOuVq+plS+usZULppJpq6QosVNSgugD5DpqLfqz
EJ4LLNfGUZ1Y0loEphr0BDEMWF0UG4f3k9JvKhyYpbdhKPJVrrTAz3e+V5LX1NU2ULoIqA3E534C
fVsm1nVp7zTc3EKm4bLrJmfmyoSrsXsuuOVeSEWXZaxlAdJbbnc+dJ07UVubUsCraj3eXJ6TkZaV
KABegMZG1Rvs3bHCyGa7rAYhwCGqLqDiabarpasJ2Zz3PepYwdl1e8XE8WF9dcClly0yuiulweGe
NNUheaS0KXys7TuJPQMsb13CryrvFCgJLNo1az8sGrloXe7xMlAOu5RS5tmQQHoiTYWOMZlG7guM
QLjIWaubjClmCAOGDcamdzZhnJqUoZKDEqPQupuZoKtAFqwkXGkF9UJ5tQZT10Sne6g1fDT4YMX9
PY9Eran0zoaNQRF/XuABtXBHdYihIUy2LOfQ4DlOAiqQ7Ui9UW0CEwBHzKuIU71RFUVrLQLpzYyw
rKCb7AR7w91exfnLaV1ESXfAVsJdbG+nX1N8FN9iKaRdhUEqCFYzFGfITROnNlVFwAwoH2QAmloI
dyA8hYFtEyc0R8HKnmLskzcc+BuIKL6u1fkuF3hld7Ah9vH7TH1D3Ke63yjIBVVLcHdmxmmzKVEG
PxbJake2urCvIIO1hBTKtqUctQKi7AifE2JEy4pctPMuVHoalxlzirgbOQB7kdcNH7UrcFlU9tQ4
DuibxuhDXKLoMSx1sKRlb/GaUga1q7fCNHYl2dhyQbk0yeBVkYG1or4S1QWMFCWDsNEWXCxMwdq1
f+KWSnbMb9sGlxaluncv7ljQLiEpBIAaUeOlFihSAOJblMXLXoJ5Y5gcCImzpVRIxVAHbXmbXixL
CNbAgoAY43L8v0n9RoSI7CUnsZUYFZRm2o6IQMLcBa3cMbGiyO1sgPBKrTuNiNC0Ntl6YbI09QIm
ADu2/eqBjBbZBvvnKdFu0WaGoOhtnHOaYN85zOgEFilopXa7lPb2jadJAt2ABxKlnnAeylsgl7bN
bCiSo3luuaXUGnhDQ4HhiZg3Nsz172sZ0SrFkKq9yLkMwZFOfSoV5lr9zhbrNu2huWeTO8TpiVXj
H+1xZpSvYywM18MllJwUVZ8Md59M/DUQgx6qloumDaTPsGfaLFc628XVl6gGA2BahCl1ctUVhDbN
jKKxlDEwnNcZ6MEZMbmyMK+qtC3AqnYhJwY8yF2RXfDVokMcAvwo501H2Ld3Aj/z2zeCgkf8KOFT
NFtRqTTg5C7VSwRoxOtSBbS8GttTG97TKKaPXRAos2aKghhwtn5xeKO/LRK+BSOSisOXUhlsvyFM
VDeiNhQEkFdyYolofhoeqKWNEpWy77fsLgw+DRZbcZc+/oCMpb/K4ddq+QgguX4JhZ3X6Wssfx+x
1BaCSj3fGh/PReNS6AFLbUcM5/LtUdAu83Ln7EUTTAMgWX5ZndRyFi2CE/dp7SYIJroX4XiD/Jd9
D+AzKHI9MiErAifebDWVvXNqVyaNqVsdRZFzDTL7ld69xFBNeJgVUHLADBWBdYRaCKioELb0eLlS
iVBsvdFge7HfdrDFOyS1jKgALVeAmbro+orF6BQUDhInbs6/tdOZRJn7tXCFIJknAAlqZ2e+NIs0
MvfFWO2yg/AwFbYTpNuTHwGFtgWjHF9SUcCIyquHsItzyc2HLoS1beGMvxsOLgFabWETJfx0pGCV
9ttKowJjON+FDljEguN8D3gZl4sOTfrivHeWOBM21ZdneVL2o8qQL2VpvBEmkOGRuHQ6wrSUJqFq
wCVinCRJlXzW2ixlVnGoccyxcUAv5TL7nF1735gfPjffkJaoKAqwwdgn4mUwJNlbIqpiHz9051+M
7nZxsfBWs6/T/TPiVl7RHFJNKw+Pw351o1fBDbR29KajsG9XWA2t3ZdVpBrt699PaeVLbRgeuaK+
4Mk2719Ma/0A+BAp2VfxsMWCGwte1d1iNyf+UpUirBbbE2N7gN4Rzek2g5GdnHW+0xPaUi+uy0Jv
MPxQMOE4OVrtatxA8InKmsYeq7fsnxcqXnLD39em4siHLgmarMbKaqW3B3Jnh1/0mFkuENfr7LkX
e6CrWrpTTKyv3hIp3V/i8MW29dLouUMTvp1fTUtIjaaU/WtVOWbh9Wmn4MOZhzg1IFJUzKbkXYtM
e2pdjbtdape4sLxQDplKTLKqcOGWrB2OnlgX5Lz3KKq9fA7WEAxNhRXVXM9Gw5cYPie2k14YIXI1
wGLc/kEFE9n7Xgq6DLRptBGsS06UnMuH1U7iR/frwOl5srRCXlscyovIvjaYxnUuDTPgCJi+At0M
xpwEQ5GHY9oXeKmjoMuX0vPRmpzK6FTDNYYobsFIYoXt9KbiIrKFG2de5CIWhYyWte6E5IPMQteE
LK6xrB2QGX76cyouq+Gd8YJjVc5ylwMvqu1xyztJmdpam/dBaDR4QxCq8ByVnQEYhJCdRFzCwOE0
tQg74jS1dNYHQ2c8bnAHG0vfFT6FHmViWyjXtNZIvlajaAo8/nWSxe+DDauKWVEd0IHUg3wWREFS
j3ithH5VHnVBbysb3KC+tFeYxCHNo3DujCvGeiEDOP2ehBl8ES6mbaCUcpR1RLhtb+sDvP4vKX6G
toUx36/Xnhfp4ztYR7KWpVpW6SoXpUbQFc53+pCQNPlS7AZvUk+PYnIT0PppLZYkHuO74X6NsVdU
K90iU/xUsb01e88yFf7ZMSutV+6i5f4pkMuAiYIxOOq/7MvMyGwZfxOYCPEVgL+iY/Xzm8pA1zgf
pVCLl73NlES/xXOEMF3RraJK5cpDfBaKYuDvIuvNNiWDv0y8iSlLxg+BE3GuOmVdQyrVCEWcIdg/
UxiUZA0dAQ7j2pisT0KUVbzS4HPRZQou8rSoywxl/wDgoJlSGTW7zvcvwNkxM1CZcwYwyqDGZQ8c
CURp7q7ITpVOcbVTVewrkzBq/wCLbnHxL1G+9MzuGGXG7h7FEf75DeqrlmwdxHz/AHWAsxCN9Iab
ghJu1VlX3lHRA83ASNlqMsq85qj5MWguW6A14CChg3StiQHANy5bBKrIeJgs3cw9xV/IKFd0RU4V
pg38DULxxJTAKqMqZbKUaRgtUrEe1QADBN7kZSfBFF5Ki1XlWCgzTg+SHgXFsvzL2FKVY0y9Mmas
v76ciS9UR1EHKFyozjoS5fTUu4tdLjAbP+4lsZzNjTtb7UOTOx+5UvBs9sAUK/6+8DKXJe1lG905
GtJOMNnuxE/+FWK6L55uEAetHut6HNvxwUsOtToPCpr/AG5ObTW9KF3cKsaIwE4mQDsnqaE2D8z2
/wD45qrE/DZWhq72yR9j+E7pEHojMIN6PMN/7L4uOcSLfzO/MPJjMU1W/IpmNdnv3MRO9grrigMW
/F35aKPmkUiPZOoXb9uHaGpUXeNUCuW3iqUfFK2Gh6xSdd3jCH3CsOhoaWMvy6Oa2qJrWytZBhNj
NFNSIhJtAhgVglJKvTfHNt4Yx1aAO156lBohaYwUXorIWqkYXB2Kg6m3Q1dABL4kGWRoPZmeynx5
9SHrNP8A7ZN+6tVIKnvqkoZxS9wxTUNFJ7he1t/Lsv3G2C1UomKY2htLc+ZiqlLp46E3giWY0TdF
bVVkUZ0+OrmfHqYDt4CMSVUtSLao3ncXgtkTkDYw5NzU1d5tYNPqotEK4F6A4HBD1HVSkOyYYcSD
lU8w5elkGO1KR9xWl2y7O9MMAUTSOoPLNlpelYoiJYyhxlZT+5gKCOBZAVdQLIm6Y+oHWcz5ldom
qY3BhfRTCWlSoCk7g+5YypGiDFx8ah0uDDMRBUcAtYCMOi19UhlpqBXkshB3uoGiShHFrUHmpn4W
1eG6lIyo0BaGvy9iY6AuyYmagNQcIzbXerYR4tp+AyucEvw1qEbdA7mNIRJu5a6W8ENfxIvXTG2X
08ygAmOaN2n7RGT/AKu5zWA+zcobi4AaO9fqKv8AZzhp26RTSDpiu+/UsIi0OatwVZkfEyK30DKw
nJFLzR5UcG+uxXS7g1ysu7ihGqi+H4BDjCDX4Ii+d8iKt5EJXK0daVOEFjfaBe5G1sIx5tme3sdP
hSMEteym0PRl6qHELQs0JcSi5ppmrOk14y7AsGF9ylDA3n+cMGMcF8bNEJCLKlvPubbrw58+OLOl
lpe5Ax+wjFwW3plbqQUlMGoL+abWNWgemhdg6ENFAGYCkSdpvrBoAgFgUEZ1ih0pFNCDslhlTIQ3
vkJcd3t1hccpMpDJ9TV+W1cq9CAaIJFN76l34TQBVE2w91DaAJ44iGwPRMxILzXSmPTXTG6YJVXN
g4ig1ZmJj4jombAl0qDh5jRwfmBupHtLQHLREFYXaFXQFr6CFGcvLt9xRKgKxr5StZ7gItWVAdoG
6kW0eCF/WmOHpMELJWiC2onqBMsvgHdaSGpQtj4K0yr71572wjBKANqWICll3Fcj3Akut6po6eqj
TcVViUpNNUujQh95teX4uXcpV4ST9Lq6L21squOqUf5SVUFoLsUqkr6mzJ6xGvEJocWTRZ7uPZ9i
hdsFjBHpCWLDHFc6B+8KGVsstBwtotfUE/Hf4YhMdKCP0y6awhZ4Yoa3UdvUIsp9L1ErQzvnq6Ob
lT3cdw/ZZW9Yp+K80zSZzjL8OLiWxkTmkqB40OdHLqrTCV9fKV1cf7D1ymr0zKv2/wAgOa7lj+Jn
d/uHjLO24ocn1u6gYoJgfbND1voR6DNn8Ktx/SceX3kks8p+WB4a/VyiW8F8T6xy4+K97CMZnps6
ArzFLUSxbBK47F9A7DkgJzq7yFT7uIoZUf8AxqKKNbl56C5wVC0k9kXtu3fcy2OWaitEUFAnkgAo
JbObWY2MzxAx5JuI1dSmK0d4lMEbOW2I1FWx8Q3EVkJQZhSDBI0BjSFDoS33ESGUwsYfkIopmyxE
pElVKYGo0DiCvUZYNWS+103ALHmXXZjrUtWVLwCHN8MrWni8oGm7i4naBtXISOyjbzS3tnpFxuQr
JoWR5Xfeb1RxrmvxFD21sbBTc7HdcEtY6C3WyjRgqFV/ICRrCwvMYNSpWLO9RMwCIyV6gqRurges
TL3sR5xqc7bCDdWxoWHiKrFQNF3lpvjvxn3cDrne048SQUvpqbVccX95IGfXsFgXQku/m+hsWMre
ziFeRZN8/iKtovNvci7KjSbESv8AaNBGpIj8aql9uu8T1wporjVWwVvrVnXTVmm7iDCp+fGKzW8M
Sav8mbmSZu4Q7s6h0Sd5kkWzsuMlPiIMBmdhlfeYFuZol38/ritnbb1Jubt08PmBFAWC+0M4oAB6
UDRsq3ALSe9n9JB02Jq+BjaVfJg6f6g1MxZSA0upgohMepBY70R29R9MmPdoSfzcbTkRnVsi3YJR
uNn1WMYG9NNYpXb8eCnpGUeV5OEqgs52yqqwABCGya/jIZda/Q4UQMSXtIbkNRSzQKhp5gvG0guC
+iFuO2gPoIvXcYNEy31Ki9LYAA7xPMorRkkXLnB8LjK85R85iLlf4J9hYXeI2QAebgVllk9A5WWV
ytcuB+zfpFGLYCi+4ymUvEaQ0gHfs2fJgScaOKTi4tyu0AC1XQVLeAhp1INDpJq5GyrtcCK1lA96
QjvDSFugEuwaL14IiGyp9piZMBqyvYSy2ilHozzbFotVQEsV8eVwyLfxI3NZ/jetvngQ5w+1R4DG
fRnuVN+CV3fdu+a1Ikw7vayy/ZlUp0GcYaCxEaZSo1rUk6H70nqQnZPAvUPKlXA8owKH7wV4J/dS
IE2efJUmPudVDh18T1dJY0oV2n1AmoM1NLThB/sWSPksOo5FCjWS2MQXoFCvmmDyqYd7yPKMLt3d
UdrRi6piMvxzUJTedWoKBc5ge4ocrjBU4+bdqJartBJV/NEFRu61daUXKnbJjwuiu3lx6WOndriq
/c1yHY7ftlpnafEMrqSnlNfVRx6pGmyVrVQ0I8jzB9ZibRe1goJmU4357EUFDIN0DFW8n2DM+zs+
JKQ0M5UuVo3eRK6vRb9qPtJtVdzBM93fuVv9TOPlr+3H4MDHiR+l5xmP7WY5sSvmVyrdPMi7Yp9i
Bu0/Nyv3u6d8qh2Nv+p3mH65Sch6/JmhVCmu6GKWhYCrVLiGbeUSzKvhdtDiwijXih6VI91/OnVZ
vytoRxu3n2pRZ3+nLrcRnYuSn3M3ufrkXBMJB4iyqnmOePaCgsApMOEW65zQ05376gjTfrrc8hi8
K1BVm69szKq7zbcS4Nwl+ZuEjA3lNcwad68tFKbkC6XYqHasIHAD2alTsyy55shoX/MRFNGcMm6P
YTS+Dmn3UVShMPa2K4t0a5TEuU+7h3EVAiFmJtXzFWpUWErNxTrcJmXLJglLXP0XmY1z+yKrrdvp
x2Vcn2zCFgdv6M05ux9Ipol2qC0weKKLyqFcUvubDhlOP8KsrKvLP3WMYwlGTxoF8S/Qso45ma6t
m3qJDiwXnHepivbWS+KNlQiEo2rqlGG1eV6wbCJw0C+DXBamUVtp3CV383Er3E++22C7YEcmRdV+
d8bJMTiwuVbB6eZCE3GvvNcuixvNDVp5ZS+SWyyVzYi8oKFev+4Qf4S/M4tA0H4PKRwz5l5B+cHF
NOmHr/fCFflNJV8+GC8os9H48pP5s+suls8Lu3DG+Sbryx5BvX5rJHbTYvgYNrkdHBNUPe6XI2jT
oEUVhSIoO1sA9v8AAip9/veNggfALcGrqrTePLUhefjcdcS2VrROV5C2sQOSMFcwqKbT8C8L42Ba
5i1+RSfDIx8dpKTaRREvfLJPmNrWuBZqVVr0+DL5zUwEagS/xWx5Zgj5l8HgsFLay8IpuX9l3Xqe
dogP3X7gI8yttLjyow88jQCi21iK9La8WTFtXmdlyjkjtZqHqIWaUwkW39FLjVgugSxcHy3NTj08
+DesbCv2whcg0PRLt3S7cbkSrV61PcfTMXmgOVMMaoCi+GewDavXC9okcC1ZEF7kXbmvMFHjtS3u
0moiaGohjO7tEXAcV77vQjk3L+Tby73CPvRveyolAr6TyVAGMaithasZugwwUshGfVCu3lcNkO2g
vCjC9Uzhe7Uw4gq6CrjQOgleIOEiqOd2Xj2wMKVuv3rFLw02LwxwVDS9PuNI2SzNI4VEAAJ2S4AF
ACWkU4mBUKXDdHRrEZo1L61LFfwJzGEqM/HJNU1xIlceRrzL4Xy/3A13v+kza9V+KQW/b9uAsDQ+
m6yunmdQbWQKyXkv1ur6SCq2f2t6llPq46vXpOQcWqWPhio+fsk0VxBsz0PahuBEyuWoro/eW8Fy
O8/0Dhb8/li2lz+lBCdrEOcfIxPCf+TjG2fphCB1XHd5xtTmB2xMR9dPI4jBnHkITTO34rdULOmT
3a3nt0puDhb72lx1G3asdZMBW+F6ng1FeR6vJmoFhjiVJ5H3JS4N/MkZ2ZlyepEc8fRHwzFWYLLD
XDEYmHaPkPTovzI47a13u3gJovcL+gZYjjy39xhLFj2zH25F4/iwNMS/KTV1kwYVgcMdrqad96uH
SePXPUXKqjmiOmgSgVqalosJEGfELyRb+alpcprjVRKqpqaQFvOu8nDwxKmze5h4zm5vGa9w7vli
ejAbX+NlXklWUC2/20GN2R3imK4K6wyyZ4OzsmJjoG+pCQa3Lp6ouMow8w1ndKAKUrQWy6BGMLi6
/tRAVlY+lbUo7iQhZRX1szR1JkF8JTR0QG1LbZdXUxF0RJZgVLChxEXydLm8ougx3jQBTsxG9wKF
7asS/uIzFQdXGr6Vkth3NCSyHI0zE4Lto90Mu4kqmH4lap3yRwyoSpX8UFqYauVtcd1GEAjYxUxt
AYTDTVlrVxApTSs+x+KX+bCXXp8DlOAhlJvG8Aw8gKDkcBHmP1A0v69d6wjLLtCN2Zc3tLBOS8Cl
NJFBnVQZZskRFKLaOouIlkSXmO5dS4pcJcuVGrQvzhKfEzosd78xuPks+bgHABX9jK4f9VlDL/1y
/E6aaWopAQvmNqS/3DzRuDezLE0lQHjduGeeCp840yx0AboxFkBKZ00e9BlD9Cka9Ick7SqOu+VW
PKh/mzrE0Ah1M63C/hYzLVdKtK4H1y4cJqNVQaLl70g3F26ptCtBC8ht0tLLxalmvhlfrpYuvGK6
5Gab2cqA3c+I95qLAeWVwT8mW45vsldPC229onSSL0t/BFLA0gY8SbCtbHm+/uVK3DIPNRE8lLh2
QZtg4muZZGmqbGKsEICaACG01CYJaKWEs8TjC4LeWLd+9Swq3CKGkRsSoyW7R+lzAAUVVHVq2EQV
9JAIAHYKnsuCoIcM2KlNQjmcS3oUEXdS11L1uUrMDeJfvLYFx9qnsL+pcHByl/URa25Xp0g3oJUU
e8K8lkByJY/LKA1dShLBplHylrjOdANpA3I6Bq/mSo1aJQXVOkIlgvliS1UWRwnU661NoVTJFQLa
3RuLxUEugLUJbSbbAQXde0btCviFwL2cmbXHavtuDRiRgXd2wx8iU6DahDiKgCmVZTUS9WSoBFfa
QubVxzz6684I+Z62UbSv3hYAfvGLvT7fSydd1KktiwKLWIB+yMd/3dysLDLk1fsOAmNi3lY0+LHw
iSr2ljSnQ8bHGcEwzszbWWSh7Sb6uiJ/43ObqvELUUU6BNbCZA9mE2GHuiN6WRYVVVSnigrKulWF
Ov8Arq4DF0IqQm3rF0Fbnv8AtTJLg+Wvtcvoaisqumf4HMblmHNev25Vk0fcskLiw/mGgd09VFHM
/oYrp0P1Uh1+3YCqSDa1qtbK2sBO5ZdgEPk5BsLgckr7xTTmwuslTlbgFDpB6VnZ8nK7lBLuXCcK
ds1e4R994/uLRuyBUyoCgrxGkpLO0AbF6KlrEviVVS1lY1BY+pYjZ5RgFKAbtiCBSPqJXSszZigt
UXVxbJf0wyWd6uaKoBOzKnVvIAxU2713YsaFrGyVXmrd1kjTZN8SntFGoHciFjm8eNUcbjnY90j0
JtHlXgJhvzVTf3NbtihlGk5GW/1UWTtj5iPbW/EAuNRNSTZcTi7vIynDtEdVqh3fMHmjLKjSZXUn
Yx+2N+tjKufCpNLLHAsvBHmVBEJjXaYEFNQ3Qa+Flm8VXOaGxDy4dxWrJUx66L5xOIUeZWK72F2p
USJM9rA/NJYn7gFt2sPt3zL1hpuR8EIzZqTDRskpqUuoeItFUaMX2i4LsNXfxUafcdahsewlkZNa
q8TWpNh33Co1HMreFaLwBKiDoYpOXLd9FjKV4Ws6Leeatl141zGqxkXwPf3BBsNB7FpaXXJBC6Rd
LEvCH2QbULUu07vUWstl2WgLt1YwklOQ9J4ZSPZQkklleE07au8EDTGp9qW3vZY9GDOCqtLKPjM9
pKhbdhbCxqFtj6iKS6Exp0RETb8orxq5ozA8lx81+g0BlBoeYJ6hjfthk3DCmqVEHdeTdtzDeUFq
VQlOOdGt49woP3yvfdVtl9599eAL2rwvBqHmvGoqCWzOpdmXOuuSK2uxGcqvlKWWL9HlgaBh4t1J
Fb6YepVzGbJXmczUtlRlXKUV/wBUFtzX5yZ44VXgh2ncf0lhDL+xMD3VT1Xo08ohdGZ2REFGah7C
Z1NeHDAYkp0WglmK7QjBk5jJUy0MhzSEupDCLU0BDBTbdQoVu6YP0zN96WzwYi1rjTxcrc4rA9rO
8w+0KP0ox32teSDMtawNgQsGWKwOhjJtYfp6eHvYUvlSg90YaCnk9faQGZMc26I2aYeTVoN4P1FJ
TVpJu9eo5XHcbiFaUgmP1tj4JVl8kP6uX9FYB9913hro0hCFer3KC37Z9oxZJdf4osKSVW0Xdzce
Osrb4RllrNc1b/xbRElbtLYVA7VBpkZ/fYmKpwlG1A2yyfXtziAUHxbtDBXp3mnswvMXOwbthsQe
ebTKgaAmuVX3YeriwRDMve+4CbhkWlxQ7zBZQQFd2iyAh9lUF+yJHXn/AHwqDespd2DLb8MqZzhj
omUo/v8AFArWWsEEZKVgArRWFE2vK13Vcs/TEDQHhKSyqA2PsMFRFWfMlxa/AnZbeKGljxFDvGqJ
eV7f6yqOJ1NM2qvgJWj10Uj9QLluEqLtDO3/AJIs5mlfUr3T1tRQWyfFP1kRuCBQuqhPF3v3hUGr
jLeQT/m/PooNzUl9jJ7QKCSFQnN1EfNmQ7/4c7Emz6zcPc1tDcKVWJ8CBUSqwcPZf5Iy+NYH1aY4
gr2704QRgYMpamKXv2C2S0wUw+PG9wiKKOZ7Yrig/JBuGJtK2zUXFpF1v8FSPfv1Ww2cRzyMG4u6
3eH5VP8AGd6B4IxG4YiKzcTxll9m5GdOJhySrkMYrVg0UTxWfzTSyBOGU0AVuXmXc1CVGPSrYIRV
ZUWG8eH8DlzzkWKjyX8yw8n852fQfShCd7fYinK8ZBMXngBthlJpKb5Waqz1XESpL4hOCpNLurLr
fdHQefuZIMsi9ZzbBuoHBDq4qUJ8cM5JHWpCDlovpHBNwMyLPVgfpILoAtWV2ntnf2RIorsHDBC8
4rYiybHPIXBhDb2qkIpL93asozWd7Ug91Xzvfy41Rn2jBtbSPSrpVuq3bzdQKk9oKoAeVlVcx1Sd
OtW+mmYMsAyBoYRHFo6YmFo670SsCKoJlSO2dKzPLs51LBlb9LzLsPySRaHNYZQTUyZyiy9h9VBZ
RAwFBQHS2GlP0ESjt02lvHPPBoLjAwGxh6m98EPJdzfLePMlfmR2oJ6+umK1F1G47JLJhPpLc4Jo
eVyJDRR3NkqdrXfcQeGAT602FCD3Zn3FvTS8Qda+nHE5xVQcfolZRXnQokGQ3qid8rEttDc2juAD
zKpOJ3hDFiurw6jht4lU8k+8azA00CDxvpzW3E7CTek53KXdr2rEbb9EOGrOuRZVLzdClS75H38S
mn/5RxQhwD4oqKb97w6IDcULyNwdATAVaR4EU2outz3S+CmOwaL8wVLxwJqQ4wFwn9V0w2o/m0GU
lgmSl5VWwIjRK5LB8QL1M6eGxmUnVa0taJQ6K6y6FllcxU0GHbRRqLc60p4RRYwGkRsRj4oBZoOL
g1qQdsvZFluRNPsxh6OFASV1uPECamQ0giIKwy3AjF80dBVzbL46KQY29LZkm2VEphLj/DFRI9Ms
nH75uJsfUkgcXd8xr+YIEHNvpQvpP6Sh3/3OJ70hAbwNHfs7jeHO3F2Mv7H+xNFGazLeTy8csqnR
+Ru0z3VYcyHF+6o0gaeIou2SkKxsdXB2PKltOUylJNmU0m5uKiN6bHeP92E0SCeLsD7TEpS/fJJj
YeZWIQSqE96TvX947Mv+HwJgVBRVF5AYevh9I4MkSv8A6KUlxKuP6cvDB/i26G5hyAC6aVot4yiF
XqJVlc9A+mzPaeMK3Igni9AtG/earLbH2VHbf7WieUyKu7ALnHd5klXc1bKwPNiAgLWBAu6Llhu/
lS4fb2q7EK73USPHBVWalTvsFQuNFebRF2uaLUSpqyTU0qy7BlhTkAftJU4NAGRaMajqql+SYRWd
odvkKzZAS+V4mCBgC16wbnOzFUpogQJG0YfU4HzRaDSu5j3DqLVCuUBi252QwH6Z/bKD/TR6dqL5
ixVbhcjYiiNWA6tY8cyxUapNRV7rI4clSuztq1aywxO4IBYMEpiwURJauYOYomIIXzFzcWmLuXLI
zIsllSkY9Kcxt3KsmutkxKeldF/CmHQvHS4PSrN5lokezUz1GWuhFzK/hUffS5UKIBbo/dl30/nJ
o71f5QtXZX3QMjt9QhOfk/rMff8A2jPFBuy2oNwk1WxWhGHgiv8A08Z5RUClBL5q9yuuJK5TLDUc
UZChjfXsnFvqFS+dUT1CNhrKa1NwkZYJRrlVYB2NuEWW75dAl4XC2ODc32ljV++7q3QmsxImaF1g
0JK/TeDKbPfiFi8TK0rBV+iWP/KbHeOVM7HY+e5cUu63cXD8W4n2o02nEBUNi0KyoWAyZ1RYBDTu
PivFkM4uELoGaOy1RshiLFDHvSZghwKEW6AExDoZhvXa4txwqpUA/DiUWRBrYjYjSS2X4Kr9zJdO
22NQai1VSipLDBRDRmDTZvoUuSMFpiX2ZalXNXLa6VHgqIiCFI9OTpbpidqS4hGxhMxOipWajdxI
4lSujZL/AICRywaj0S9vQLjqZmoiJRcvrTCa1mG/Zcsz0tjcz0qEEHFRiZmohzOY71Ei6YRyqovR
30BmRm2ZIuHpUqBGA5Wppw5l5gS3mYcRlCbP3Sy0a1nu9egJt+D+JvfI0faBeb/qYlkzWBVJcGG7
Wurdy7vNm+OkMYCpR89FGGQFJvi6ekQPqS8yKRtMowddT+X6vaIlWd/VyzAgkaAYHCR+ZeJi8hEA
iYgsAOcdCosh5QjTMxXKSUK0gF4RzHFdI15v+AgbQo6ZCGM/nZIEtUW9TLOt5p4jgxg6vZQR6l3q
DmoIalTNyuSWY1QEaycleiC3fRi4ip6li3ERRSemUbFJhtoihStljw+ozD6UnwRAG7KHtSIYiL7K
s1zjegx/W7yhEAeUyvumYoo14PaLUO/d1K49zJ8xmWZuVUZkAIHbcEU7RzCMJjoCwAe8SXldKixl
TEqV1tqljEJcyRt3CVKXuUVDVHNQOhpt6DccLddKg0ygnBIRWtzPROjEqEdSmVZGkQIkOme8Rq5V
xHoy9yap+cX3/nIUnJU9LC+v9Zvj3o9MMQ5KP0gMQTJK9JRoSirRUKYEA5sY9iInErFxvnU4zwEr
sxwEHLa5TbiVfktJPSiVNrdPJ3IJN3L+Qw6sW0g20CIW4n/WenLc8CJ0rhN5Nyoj4XgJedlBJVHt
sFIxw778xuNarKXa5YZK0HlqA3NEFmHBQHQCiRY8GNJevmWarWcHgYUNwLlT4jhvRVpXSQLgtNxd
k1RB1965S4Fnu2byu+sZvI6o6slsAdJimiWMQBZn/Sg71/zdhHI4EKiQqXEoOitu1WJ5uZK7aEMk
ZeaxgShA4Mr2vqGm5+rYZ23hVbQcIB9b3VTV1U8twdt5TOV5fA9iUUfufjt5KIqzKNOOVS0TTnV6
LBljOIjBcTeYJCqSWbW3ptlSm+iiUOYI+bHUQdg6Ka6b6VKYkQJxKuJkiO8oqMCUXEt3K4jFlRwT
bNTLiZK9DEZcq2UqxKMyuZY6FyypxOZvpl6UatqcSpmVKYSyWVuX0rEsqlIftgssmfxF/hoPsP3O
oOCsktre3xiKWYNUdqCSNurUEvR15lyKLTiq3BbsS6th9wWKTBkUFDtGVY0sw0DxxwCKQ0UVl96M
Gof79oMJtsSp2qXD4fXueoquCmRCHPgS1NkTHKTbFRY/epIxVGXddvUJIwV1SKdid8VDmMYcmveD
UTW/qj1zyll+NOzIQfNIZuwqy2oj7sZMSoGEysKIsTRG9nFG7WjpHiom83Dc3cWlQapTCJO1NGMs
3PEStwnm1eqBTUvFRdOCVBNIasLb9TE9w98l+YVsAV7FoRHVCSHEwvpO3C9jDaRFXj54fPS1MT7k
a146/hsNRSzbhQ/tkfpidLk3VLVugD7qZAZhvYlrmNXm9MCY/HMp7vBEb4TWCwzQz5WwzUORgQPV
rkTZeLjULfhsqsCMOGA6VUU6LSr/ACsyTqsum3cVIMNJG3SsrhjEuA9pzBT1TqYCWlUy1tW4qXcq
umpXQzNdEIEaxUcE4iNSpuN9KzKj05hH1LqZb6hEa6X3Ikw6VKhcXqp12S8JLm5REQvq7gRrEZsL
/tTAfEiyvIfmFgnmp9dI4Y07H6w1/r5w/dQdXaUUGKHv0ZZfjqjqiG3i7X1rHNXBFWUz3ESJr76F
YojfkPAQVYcCFxRycOtwLx7fhufcrebUXJWJrSPXN1Ba0KQroUHxTCnf6AJMl6yeSHu4t9SUpqx8
iUDUr7MhOwlAVkIcwvyLUJVbf2uDH7L/AMkMS/2rBQecySwxtdwYd16BtCHS95ZqiaaAb+IN0mu9
22PsIhwlZjkG8QX0d6NQSO6h5Srwht5edaxNVSCOik7NVqVa7uZ//OF7s4t+lU1cRDIoo5GmmOt+
ZWcEpqI8y4LriSEQagmCqobUIXGWnrLG4hxAgD9VnY9iSJ7Sti8wmiOSsyxprwoYTSCu8t5e7Hlw
t2Nnkqkjr0lur2tnfRrd7wenFKB8FuGgldtq2yzzFsfSMYG+tUfIlIB0umo3AJKhtlEbMrpmd5Xm
bygelRnMeqSmEXo4Wsrp4gnBFIJSRs3MxxAiauPQjd6iEIgwygr+AWhFtkfTfR67YHRIEzO7KaOh
XS89FPR6NdDg7n+cF+OFp7SprwKDYDhb9UuB7n1hTNv9+EIbD3VIZkz3Ir2A84s3QZlu/qnSe055
2/VhF9tKjMqQnjw53WMepFyIsvH6Er4moqXmiVv5d2WrHZhRMVqHDbaxM771yMEjGpd3ubqtpJKG
Ms7frLYe5FfFpeuCUog1GFNWXH0Jhva6Th0oYYl2nnZ7a5GEx3DwuJezxLFx82gAhQwG5T200Lj4
FFaymkgoKqPMGwoltfTpRdi83MmBK6uFpa35YrvodCO+NRYLiC/Bq2FVAHFxVZXuy/Ea0Q1jfuXe
WYtiQUFSEQTZLsZrpx0uW8EVeZTcES2uC43MnXicxlVLlQoWAyoAX0xmUStyniOGOYwcpJVxOtXA
ojuVGz10q4hKuMEp6VnphOMOgVLgmJU5lwLlY6NQlPUtuaScMJiHTm5dyqiUSmBX8KlESUu8fmmJ
AHYvqTsrc1g2gH/PMU95Vyl4r+s5z9nRsIOM+ptSt4g7FioGbl26dyaxchs7hZjt4iUeRYUKttYT
Faq4ckqR0JVpbLgpLeLRa07NQknanA+CWVSWVVM8Psn6XCgAUQpcDVkVUT3qWhuK6s1AhQrKPSql
sWtTKDUHcOYEa1Dcur/g9KvoYenMPUCItwu5VkylVKmulNSmZ6p0CBEiRIGJVSo9EvopEqYjduAk
xeolHSu0Kssh0JKSglXKpiTKGeWJbAyIOLCVGJ0qulREYxolNxiNwRB4lISxLYr2iQJVdK6MrEph
07wIG+iSzKZdS031YXVRq/Kr8pY90HJzUQpj3xiLuUfNDWlih+sGhejpauN1/X+UF3edH0m7FeLY
A6N9+iV0ZaXcITMzMwLdQJeZ4gVHoR3K6HnCISuhSQMzbE6WKsBGUSyWKiMpmSFy4hEKOlEVKsZX
EPMS258RrWJtqERmVGAJmJ0UjRqIlNQgRMxIj3hCprmOksypSQLLjqJ1UXHDErEDDKzCyXLa6pG2
pTqBoqKlN5mTMpUzrqYmIwUQBywTiVKyzirlbldGpx0pIqulPVViRUqugI5gSlBnA/KJV4yxUUNV
+57ZRYp2TNnN2/CCvGP1lRhwbrgdSk6J0SVjfQl9Ag02RNRvmCxc23UrMqC4mLnEqUSoWlQgPHSi
VjModQOmpxqJiUzKekM5UrxKLKCN8So05gKh4fxZalOmUibl3mYRUqNpfpiUSswHJKywESNRIMwi
VKlNys2jeq6FQwjXEsxVkeyM1yMt0MgIGIOkQtRxgSW/gTlMoLUqwWvQ1L3NZ6SpmnIIBVNympac
IkOCcEo6C6rMtmhYI0rwAwxbPaN2BdxRWv1QwT32XuFuD9krhqxX1Bob5irXv+vDR32Zd14H5wua
ZP6Jl6/69GY+5+iLcrMqUIAw8tzXQLzKCCVNJhAgzKspCVKxCI7yhDpXRqMAFwZyEE9yhm0SCyBl
MdpWFlTCV4lKQvoVESUrGAWKDEq73GjqDZQyhgLKGCqWWW5i4ONJnENV0Vh39AjuWJDqtpiMvDwh
VnBAmGDwy8LJyMQFGZQqsuXSwv1L6yeoUF+i0HoEZL1ALsfHBVv5TnKn7pMoctWH7Y2eO/8AdhfV
OYYGG4wWiynv3cEBS1kx/UVZ+JzlT0tQ4CAjaiW7qllQtPQzVl4WBaM6gbVTY3o8eA7jym2M8Rce
G1YXpVDZG8Hao89RtW+dtNnEckdUrsOzBamBgr6cdfVXHOKGXuUvtnvIGGpjBNqNyFbajlU9zFZX
jVDcbITh8FDU4ldirZe5/wAZdlEwF+MNzN0Fxri+wFxMKt0zTMeRLbBY5TUVuloNg3CxEL7cWQMg
Kk3dJhSqp/WlRQrHXGSPM7q+oNQ0s8CH9wRSVx3A4/MtmYMncoS4NP0MNf6NejM37x0jVSpZgVio
xUtCKmBmVfSs5YRFQrWVEnqVC/S2JKLZZ2nPQON3ccmoIKZhAlTW+g7ohWCEWYQ53LdBKITaVmE0
RK4ivEQu+klOm5ReUieBB3ZEbJELlLCGX96DLT+nMKL6hcgWNcoa13qV5p72iCYoaq+X5lpxeZrh
pZXbbtbLDDRrJ/UGUX9rf0SuTeIDaLO9cDr9+DIMnuI5w9DBVaeSiC08O45CjGn/ABIWl1Hm+Vqm
PTLYL2tzC4ZaGce0hQGpq/7i6o1tL/plWIE0aEUCF55f4YPZG7aPXAeJDGx3gAiU3JX9R3aFWcEv
oKCDfL8y0qGVujVAzNFbO2JLeIGlVwwD1XnM2oqAxle/XEYmCy8Xw94VA1oBVgwLo1Y4WxisGorX
RlgUKLGbzo/UYqs4NgsKlCEq2bqOTKaFviI7Eq53w/2Sm5NHzmjsoEeP7uF6UKuFWsiKIfksm9W9
FXI4eT/xQ5zQXXxdLEKWzzELWMzxGvFwSGdX8iwxC5X4+eSynrH+TCUpaHXqWYR+RERuaR6pQtZQ
+oswjYpfPVCFC8YrgLky4Bs+GT6wD3FlYclLy6GMfVF6LmGLSD7rnkKp9YoYABr5xYKKBYrGedEY
d00qDUTfFWj7l+C/XozCSOYCBcMMp2So1E6BmMEkrtHKZsQEDGpVOoCvTUJfoFxgXMpLYxLy/RZl
oOAdjUb3UaMeZStQbI3oWC0C3rOZKVfQ9Zxb+4qN9M7WiYCBYW8kdw/RHtOXPAhaWV3zwPUpPJn9
ZKenkKNy/k5aks2zezqDbylFH7WVbZ8rCrjNbwvNheKv3OqfUjPyNR6gOma8zPZ+ZlhVP+3MZ2EZ
fB8zYaqwcQpU+D37mu3QPGHHAgOaDYUY+ICi6pXuzEi1tm3k+Y4BciLW/kylMFQ7tbm2rRajb3Bx
D3knpGVo9VysBcUARqFulafNTJsX3Iu0dwvDbpuZO+a2x/n17G4ZpuCt0SqvFEQMme1nceA8CItE
FWPqoQLYQlvdgFPns4ahCXBYXRxmWRd35hdc2LshqoHQYB2rEAbjd1asy20NLWPGYoULvjTuF+bK
lq7QWosa3eC42i2BVLHtQO+0DLJQ1d8Q2R2du9yQAhV8LJ/ULU1avI7RRkrutrMqRLGF7RRmCATX
akQYQhP3kBLq164QC49kxar2x3K2C/ISGMqXzSv7mYAcw5tQhLyq3dggjcFo8lE2iOxBef7xBI87
XnLHsYJfARU0PegqJSznTcnIMGAnqwwqrWuVFYAfXij2lt5sSIVFIVeeVoQV9XnKmqkN8hCjVk+k
TFur+2kEtZcqp0ULsbzGEIwTfFMRQo0Jncoi2qfHKiCs2fjlbZaNTg94Ug+wP2iAWS9mpVp2dzzM
W1fZEyFO+YfO2lbMamA7p+MqIsA4s7Of/RDCkqo6w2Q3Fl89JNTkgJHDVS11LijVwu7OkkGo8MwT
IPCiuDx5RNhz3icnylQSlT3aqKMfeCa+lkktCQ375Yo7sACFlP4YgYv4FEg+ygm6P7/bYMYPwyor
YDgmAa/0PBDufto4ixyfskiAJ9xYXmHO3mvPzLS0yoKZxAKhc4B4i/XtzH6M0XIx6CGKHQPk4ZlW
2m+/8wOnyldsVq5HLTEJQ2GEohAxWKDUXihWWKxdyi9q7+2J8scUrnWNQFZawb7MxCzRzLSzjljW
C2yYhG1bNExC2ETI3uouPub/AOmop/b5Y7DuRdLrfLtBfYuKsSm6ceYhgVAltLzDHhIkj2H8wX9n
6gIzVEA04zHl75qX62YuLTDun5jur4IK0vZkPERRfL8ZWJdh5efE8mwnjM1cJWtneOKvdqDegW0Y
g2EKlFKXD63EoKuB0cMIwpvjvO6621xUEIDpDgioO8OLiKYNjW+xOTy+uZfnSwSoyFe8QmQAtt8R
BVyBfhTLHFHIYaLDR9Xgs5HscqKJmZjThtxoYIcCYdCII8AnN7IQihFrxnjxc2wpYPFyYFYQNAbS
Q2JWytptlzcbLbyZVytCb0WkIaNGp1aHGSj8AILzJcXENVAatWMsoNUPRslNvo9DSr9CYp5U7rZF
oSzCn/SgFKNBteDbAHcG8Eu4vO4wLUYKp9wsMblotGw0ZJTvUtGvTslg2dx137RQVAyG+pchGIsg
zHOJAqGSjw1M9qw5UuruRoUZNvohjVDgMv8AETDgtiswcandmzt6t1N2ubbraJQKIjvz+0GKC3HG
VyZkbFLNLO2DjRoqChV2aHsgLWSIV/pqIib2OIqGXaHzUAv6NYZC67iWwfsoVAB8ZROm88sL1N4G
P2FV0Q7yPYng71OtXq8gSkuz3AumXzALGnhXAqr7fcNqZ1YVNcpMhbtieg9pNl5qOBLfvQLL6l55
/H/E11j8WON80BgFxnMF2CsLm+MXECvsaVbO5V4eXgSWClRmOpYg1T/WIrj3EAN4tMQyNQHBj48y
wyZk0FCssxVtwm89/wBQKqbt9Qao5smNDjH7nDM3e5NuOIDa1wHmLfBP1LJyGjzSIRzZnt3laKT0
gEO4H0g6dsc9iUaTmN6cNfcyXBZNIt64r1AAF7bd1K0LMh+MqcnKyd2OF0c/qHGK1HD/AK5jWhS4
q8RsRHbZ4hddHHHaGzv/AGjj8yuSrmG+N5IilezMYXW4mzPIb7zkWuHHdjp5AY1Aychx4iHwx+cx
LW9R3BZB+U0LG6hFzf7Vx1YLWL5C5QsG0N3Ksu2zKU7T8QrAsRArqCkNm82c8kG1HHIQQdxNnd+2
KkpeIFKLRTU/wmJJXkqmrVUYpcxr9NNppjxMOifEtRtabBf82S9ELbu+zNwdG0F3gWpXcVWFLo6m
hn4UCUFOX+pwyjtuObZBGtZ3c7vCIJEYwSlauKrALAlLr+H/AHRtKH7vZalrdLEa2/aw9kXZ/uoK
gL1XEilHuxf6iB3pVW3+uOO/pP1Ie33ufUOcbT9y0LVBq/3Iw4t8C35yn0ajn7ZyHLIWnU9wUSsL
AaZfAuTRUTTtaL84lrDFXjtEQE8ZsNxa9Mco2OSXyihdqteXmaNVZtYuueF33ZaN8xipemPcEUu1
ihLUQg45pk78TDbe46waoIF688zahn3xUZoKpc9yoPbPnEOkawV4iYEaVVjO6KWTVbll2bNW2I6b
54oUgTZQQhRx2le7r35lXMMARdJo8wvSoATPG3tgWiYGPNzJdl0F15lMq4Jm2l0SxD4jlaf9Ma34
iWWOT9xrjuS/cuXAOUP2wJhqv7jaDmsPzDqCjYPe4kVwXH+0EpFFDW45UsiDTkhtM6f6llZ7qBpf
EMB7P3EGM9zzaVodW1KbtjKqKW+2/qAFYo/8hk0OP0QoH1OA8f3A/uUW8wAXyp+ipoU7p+JinkcA
AqvR3ZsK4YCW53b6nIU4f6nIDGHT9Z3OXzcUD85uKpVm+RA/LDdRm9q8UV7S4K2O0DQR5Woml7/x
SstoLlnJmkryRJf147b/AFb+iG4cJ/5cL5P9/wAQ1a75P9kMKMYxX9kjRpthc+ZLGLe+SDIpe/6d
wts1Gh+lKrW9N/SK3Tlv/mJ/oN+sv4PSf3Fv6X+blQ1Ox/YwSkNaDCd/hfqBqV6L9CK3SvCP7EAb
tvmMNv7G/cg5GNf5kxdtd8zHWOF4A/U17W7/AKQlaiImDZB0mjbuJzkie5EZyReATyTBA3Z5jx86
jX6wb5mYDdwcxMcxpTCF5Sai3x7m1ttirCisSl5NVe4nNBiHBhaCPJXMDwq6Sx4pjBYXf6CAvDQ+
5lvlW4s6sDHiWFJhPuObt2fgltRCxdwpclthaVZj7JQvtzLgnceYQKeHrcSb2vGGoGhBosW7uWo8
ejLadzUs1dkjUq42Wxq1vknt8bFLrVF8zZV9JFFpy+wRKVY9kLPg4uO2zb+oq2vOPiNTvaiqjyZe
qlRAUbMaR1i5XY1nfuV6+9fcO19mPvcUZaNm+blTBkq7+YApfH7jYuLSN9sxo7Xj9wF4dv3C0Xui
YKtupZr1GZB6B5ln2ICNFWuz9zg38/Ms1mCwf7updnz+5mteP7mCUfc2cb4iB8ggb+BKWfKcqOf6
jc33/qegKw+pcmZd7jTg5rZ5leznt5igbr7JlWdXsXLusWyQNN8xq00QwejwQceC0n9RNZKr4mcp
/H+0nqPZ3e5crBe3m9KCiyex/QxB0aKvg9TpnlVpcO/8CCNvhG17jPfw6ftxpA8aD93HRxuASTnX
5P6kF34RJ+hAbwKz/TACzKV1Wt6iwp9kl7a7wftAtT/wbBNMPR/+Jly3RmoilKl7ryPVReDKGgci
F49v1hU3moe9eVKnyp+IIs2ofllsLw7YDNS6v3HIom2eUr3UI8PuGvI2Y4AdXcaD25YDyOoToeAs
FHiQb1/ilg/Kf4pTr0pL5Au7njJk0K/cQcfCO/mH0mXZxxKNo2JtP5lk5Xc/qHt+OUjjhz3Zbdpe
Fr5i9uVfL+2NOzp4/wA2LQFjcC05C4/wQ1fWf6JY26XUHIJhA3clQHAnmaplGSm9V2keYjYp8kro
+lKVv0JfZO4MsNalWxcwhMwEBjNnbLmDCmroQDZNHHTl9RrsMM3fkgI6cO5jne4BDlaadVcaPeKD
Cz8y4nz/AHFamrr9CNo+QJZcdunFX8y7Gqx93GqLSrOfMFmHaZy1oyHmFjkFGUlAapyCZkioEMCq
gKYZxz5hSW6aXnwRBWB5H7I1z8Rf3UMMr3qH2wXb+TzLjfF4MDgHlL+o7A55/wAMsOr5x/ULf3Be
X5a2+1jatnqTuH6CAil/Kf6IcvtB+khmx/o8w529v+2Bcvp/aGyVe/GFAtH8EAaP/rR/LHW/4WdL
OgiBFVPhq4njcDmolUX2l9b/AIuyA6yye4nBLIf5YOagbeXFVVeF/uWYD1/ZC7o/Uo3+rHG9tf44
3e90D9EaZbU16dGqZYKbvmhaLsm/8suDgO/PJ22rFww+oGYrBfBh6QQAmvUarhem7DBt5zhllUa3
XhA7KjUBaBH6mH/LEJT32JUY/Cdm7lbQOrnZpA5cmypjrwlK+ItTNniszLBRYgbwN7lSxejRe42v
c4QSrrkNRCvF481KGjn+i4rvtgUSuX7IK5tdSxs8y7Z2YhwLEaKLF8QSBaq1UqY+qFjkTTuIOi4E
E5omDkeWKwjgh2yZY4AluK/rSwtu+QijkHqP5SaL5ERma3Wrmnf3K7hjvflP6lJlDd/6kHAcy8P5
7kUurHZv7i3f1f8AM849RFYtfFX0QJW++K/0QYkO8NN3rv8A5poVq7r+2LCK+f7IvOD0P0UGk+gJ
V0Ep5okqbb5Q8Dr/APGoCrB/+KxcF935jaEptsiaRO0HHxSaEyvziAzLXiPlGeQzUSRT8hdUqYKC
iUKgeDMExpJjFBgQK7Wc7jDdn8E4Z6tb7IwhT7yBm5VPhRAWS76tcZgusb8gf3FPqp6QxLDB/GGv
4ckq7e0GuFsEcU1MeBrUGABC1M8QI2YYFVdns3qCMi6JWaNjM/O+DeIk2dcQx5qXUEuu4C78JTKl
K4YEcXua221uWbHMDf7kR7gn2rNTQGMnEMFnDDZYQAFigjgxqHl7I+YU+5Q23m5tQGkJYEwV5hV3
28MHWMlcTC7siB2UtkLFkumQQ1tnYf3Ce8F8UrLSDJwTece+KUGnUdG/qUq98f3tE+V6b6nK3eXH
9scJ43lFo4/uEqlJ6/SxMep/xFF3KfG/QwKUlox/1IX0PN2f1DW1o7/qIu/bn7EV2UaP/NleR43/
AGnL75n+ljvBjgH5uUZeX/3nHWvjgD8R5ieaeYlO5M+yPeneZ4kz1SKcs351E01GHVGH+pg2Nf4l
41FQaRCuNkvS2YWOqzcWBubTy2GXgrcyd6Y5c0UxzF1vPiPFAQG66G2Q7s/6sMBVXFqUYTV6xvLJ
cuX0uXGUKtiLSCXGwrRLeRAnvH5D9f8A8WGl5svYEmR7uwmbmtBb4SEY6hQ6LP2ysK7t5LGEKd7/
AIBuMxTlfGSVnc/1/iPxrYeHf5jd1Y/MtXBT0CzEbsED4W/uM1uwg+So+8WALoiKUWXdWFYMb3u4
REFKj5LEiXdAuEaKI0RTllsadXfE0bVV3x1RnPyUye5P0xdPi8d4UShuOGQ1FsXewRU2R5DUH+2j
2/vY51td/wDSPdcVhf1A79Q8iOG/35gfHgv8JBORP9m5zY+4xyP/AKWHDhH3G3Q+o/TgvIPclf6W
/CaAP+/ccBfLLkL+R+57h+iftTkn+1uHYH1/Y5W/R/qZmFej+glrKegj+/8A+uAuHnmW1Tr+d9qJ
1KC8f3cfnQnmNZ1MZRmMCyneA2nf10BqGOlkSrZzqs3dEOmSyiL5FnAvOYEvK3DOo61cZtiaoajk
mEICl2P2wGT0c6wQwHFqFBeFXMMZawG4dKoZlNZmuI921zD4HdFoJyI3Ui2APM8uLirKBVsRFa+4
gslfcEKtl7MXK1/qh6m5bPaJWL/HhNMb1cqYq8HAtj1JK5Tice5Qt1eUmo5quzsTiexiw5HLmUPW
YXH5JIWlPqcO7uyN5+GCbSjOCk/tpqNm3uFbM3dtMvJzm2I/Xbe4QBOP4hdEEmgUiqtcPz0DuX6M
wpZD/dYj27k70uIbUtPFUsWyJsQySvqruioEl+hl4gV+cYY14BXc3GG5L7CWTS1OwrUFMpGPt75K
qwQN3E2WMp6FkaGCI7TvJasS3P8A8KNppwDVcRsvukDPaMcJXMtzR7lBLGA0kJAoGFWfeGMF/Nt2
uvycyiMSWh5GzZAvlfQwORt+Ya+Uxr4e5OAn2FZKKpgChB2HEAUHQUS4kNH+k4jNVfRBlSulEolH
U1rDYIeo/uQP6Snaf68SMDeL/qQxe3y39orVrbyGFFWaorW6HUJ0ONFQNHq47a5jno+YqrpqO0Ss
zkVvEXnVEue5xeE2BOI1mSVAwrUS8MUHhpVbDbYqY3zxnHeOa05gsfIlwIXbnuYsTY7BMtpLsaJt
8bmBAHz2sHGIBi8XeRonFMDY1CVWrhVEHJI8HAAYwMaT3Fpyu7mlOq7OTqZtM01Zx22bvMKL7X+E
K0C3S04gPl5tDxWn/cUbB9f54saPQ2WTT6EaLJ4JqlFXiDs69lEs0o85S2xfee0t5vlx5dBwOw3e
8mAA1hIcFDskffhXWIFvsIsc99lG4vvRGc1eUi9AUVwQVxdhT2eoPWcTQWi1KIJZRVQuEFS7DSmv
Fwxq2f0uAsBAhTXwSrd4c0LVUiyGAlGC3EKteSUcfidvizQZYag7r/4IAwUHSjPCp+UoAoyfqWa1
RvxrpTVwzeRHd0sP1cqVaE1iLeykDy5+FFL2qeFoajNb6fQWGpZZ3UIH2b+lPwTV22eW4ljLU2F0
83QyKKJxfmoZ3wc2T+0ydeR70r/5kiq0jJ8kaFHyeShhDTjapWuPUBA3KeSfx8SJbL5iW/vJ/wBP
ogmRBEC91w8z2JsO/jtHXU8mNr8tR5B3z2j3mxvxceFcb8xrsA5udlb3E5WypJYXjcxzB3M3V032
xKr4LqaWdutVFS7N4nq6/MOON7MWli+uY1n3WWVr2MAWli7XySuyL3NeqfLNU+uca49Nx+vMiaNf
SZew4sKP+Ii/mBA5TmlBQixcxl94k3zm6UX2BtiffaCS/htCtWiNEfsjDHzDBJZ94kWWXVVidjm7
US5MWz9Klfpe9lCVTnQpBwHEHLs0whreJ7ZdXz5n7ewWC7xaVg9xYzP05pjAhpGKLTVyZAWVSGph
B+GHqi1RLRNDa2OcTLVKsIDOdNXAmPaXtu4j0DtCTFLOUy/NClnEhIkjU0wkGsn6gCgQO2aDz+PE
pusRbTn9QLWHw3E3jv8AxDlgs1Ap2TKxDswC8TJfRi4ftCO8PuRxp6Mo5D6L/U7I/T/1Fmr/AKmQ
6ZFzra10oA0dFqBZdPJfTNolHXH80NpP+lENh8zLX54hDw5h3+o8SviIcWhMigoxRgf0uNpfP7Sw
hBfL3SkocquRA55mcNU7L+4cqMG/OhwuN/zy3PghlAebQ4e30oXFNHU9Ma+yU6PpnLdhsxuKWj7P
ao1FIAuC9/PScvKbeOW8vd8W0R4lzHixMs9zErv8MY8/VTMl7ibhHiHf3NyPuj/VSgWE8ryvrnkL
9kytR6wT9hrLAI1gJ0X5kixuKNX5sI387AS6qppQHu0oJf4zmPrDoTi/USLwN5kgD7KS5fYYsm0F
sL18dsSDFCCX3i4AyIFQu4DhLYrYXgDCD6HAHsioUNbeccte5ChEVu9mI2Ntaj9QjVklKgFKVY3i
qqF7WVhNXkgDy7qSufCiCj56N6nwVl2KeTFN81OmZN2dpnH5jE0a2sXLG3SXERvlmJBeAI64ItVu
OmegqXVgdWjZ3IWgor2VtWc3N5ktz/csQ/algdFbS52+xiCn7pXKu+W4Cl3LZTCEoVE9l8kxr9KV
4/GiOg+hf1PKek/qC6/1e5BqR/r+5fh/992P9iCLtH4D9DL/AM9f1PiHzf0R4Phf2E8D6hOf639r
O58X+Gd76f6Q5lXwP0EtW2+L9ME2+3/bLrVO95n/ACsp2fFIeAfETjAROxCxYq5/0YkLoniRMu9e
onW96xMNkg3x1+Zk190cVVHuymsOeYuL1N3b9xVfdTNsTkjqAt89AWLFviyitmXVRuW+rF0pWGov
YWUyNbdo80uu75m7cVvKjR+5cbSvCqszH2nEqZ9ae1M8BLf6in6ITsd0479oiTak80rPm3sRk96Q
dj/KKFV+SQsSt8Eq19gR9n49iCrrKuGn8ty5Q/TSm94IYqqXLXX1J/JAonIshvnEwVbdMLlq6jOC
Ci3KxogG0YoUh5LNZ6kKsAzaSHAindYcFXsARMUW4T1A4VqotYHFJjTbBxZZuuB5MwstTMNFhW+x
FrDD6uOKKfEVZPoTDpU0o+I7uIWmFEsYOOGr+8rDjdhGxr0mUmh+EQNL0CKKwb1n6xGcgrvlELB5
cEz9mbPjVsow+hhtDT3KG7EvdV8wFtGY65UziKRYYqaKmEtgIfLFzQRwESc1ESBFxUU8S0LcS5zC
XaCN0Zg0y8oZYlsb4hpZd3FqaB8AgZgHojMM9CXriLxklFKRwtETmBBwHH1cpt4XDrlw6lSM4X9T
VKJOnTHC1YHmaMHdJLj1TFlCZhl0J7b+hy0tPvxxk4F8zwJTbPyJqR3cLr89DLfFolhm29ydSatK
8SGevmpzf4If4q4VTs98Y+8WN2/dSO1zdLGOV949ie1QbdDviUhh2uRaWfsw2r4Q5PsbnC54THG6
l8Tiwr4pFgfDCWWR9JhPnutzIn7UGAunKrD4N7iqHlViRm8SsEvDczJcbpQRzEOLwiZRilalu6wB
TfmrhBz9RlMTiHT6UIC7EPpXlSotQ9y6zvwq3UAUeWFAu4TBARhGlcEYKUnnBO2BG3PmILVC8DFR
tPmJnZ8plmPuovdv5il/sjdlDFgJeIVErWqjbdEu5V+egDcWSEozmmWTjcMrei7IYZzHpmHM9pcU
RCGSCQhS4wF02CWmOdxag3Uo53M3LRmlTMNMPfQssuDFWL0d+mEajpCuhuKZmQF8xWzMHR7EZS/C
UO2L8JVfoJ1/fHLStfICPB8sr1UfLyzj4zhSf6l3mp9ERVpX4jxr8QJZ5T2pMk9iRRQrlZW+0JLM
/aYBQ/zhCm9uCJSvK8VEKw9pHq9gxJrM3NL6K3LuDXzCwDrxLm4XEVW30ERrsTZuFfKtyJl+0Gld
5tjDUVwp+oUrRVwEbJD6JyVNcpBGu/JKNaHeWh+FHcV6hmuQzHwEtlTctjhAPmIFFUBgku3Eb2q/
UXVE+IHy9xuLFFp33EloNwrfJHGodri+0+Yre5bJAOyaRlkqwlsLYY+IMTXRrrKqLcuYFTloltQt
ZRLhWbgkSYMvoFZSRygXTPUsgtSxh7lt6l+JzctXosuY3MS5uOVzLAS4pM/EviV5gMIi76FHMacx
qoEcczaQa0bnmZY2X0Kix6XCKhdfPRrDAgiWxqFYgrQo5Rq+YPs4dkjFcEZCQdj+CGu/gg6fVSxa
kRyb+5nlvaf+sgFWBCKbWtZiQr7IV9U8ICXU9BGtkz2SylHG4jY9dx+7O9ThMYjVqUXLeMR7TKA9
1Y3U+7GGBDqJca/1MlgPglpTTPAR4alyxmKAv3xXKXbB8LOT90FdszuyUHNTuJYcQC2oJubOlMFW
5c3m0zbAzNEimLlqgstIiUjWumMS0mJmJY8MvMYuaSWheMxyJzBDcbW5ll53DTFZUddHcrb2lhLj
bSQpqdpccus1C6YNtRFZS30NHS66AtZqaILcIRharGJQp2I30MMXeoS/EzzNOGFTsP4CkGLZfiIX
BCUVDc5xMszU0ksgrU1M9NBkQbdbhTh3Ed2jyuV6uiAS9RLvOVMEIUUMSZsjzGRbRI/mdWcty2+5
llX3Bbtn3G7awZqbyrNcsGal3Be8bq5cuyC0MvrbCCl430Gzo0xMYgty8xx0tq4m+l0S767I+5ZL
xM94I9wY1OIMWoNR3GWkvJTLyTCZZaymXRGFUX05leYVKV0U7uWsvD0tIheIFSgxKHSWyumanEIt
Szl6DMDGkxOUV7jBgsZmiXxDzFqQAdFdS05myANdNcS8zETtETmWdFsC76XZKjHXR6LqXE2ysfcu
XDapcMFHMuC7bjrUe04I1UzLl9C+hemt9MJ0Djq3B8zM0S8QlsHEHc1uXGkIubzBVxFcvCECsDmL
L6AqHQMLqWC4TfQulnVbl1FUly9daLFzqLNsvS2ZeZRLshZel8RCBbuWGouels3GM0RLbi5HtC0s
myoLLIrAW+pkSCr5m+jB8RRqpUKrq3ZBZbFaxK8sOc3CuJm9xdQlwZZBZdcyyiYnaaGEu5b36cRT
vFKnKZYpcpz1x2l5jF30pTp2Wx73KQDmXF8wqVHeoblEx3jth3g10HfQ4ddBPmLUpLz0tvpcvO5m
DVTzLK6Zg0VHdnQy8dLmIwZmpdRbl43mZ3fQTqJMxb6YrUsjsyw6vUXMTLnBHSDuZFlsL6Y7l5W4
sYE1Kcy6l1lxmAjKuUQEWdpjoO4LKlY99AWA3GYOejUfFLuEI0wS0ICtX0q2OXo3KJU5OgsW25ym
ai46HuXdw6JVY695WI0YOkKVqcal4hhLzGLYhiWQpIQWEt5ZY55lwbIyxqDRVtdBmoWi5z/BzOyE
4GDFgwly5xNQ8ZdstXXRGo8DLI7uCldSKlXKU1Lm4NEG4rt7TiUPS42X0VRCmYhKNRpqWwRtl4id
MJhuKLjoRet1FZuX0d4NXFxcuyFssTZjiP8AAlqJNS7Z2LnplG3M3FXKQLjyMtW41O8z230ouyIJ
iaalI2b7IkN4SMYWylMPwulV1FCN4DwLSktnegg9/Om0o8R5D05LmrneW10PVeZYypuBFnEArMMz
JKneOmtyl6KSmKLBiXUW4agazFgR9Rvo0zZUWpe5xNTiHQc9LxOJzKhLydOJrpfQVMT1Lh0UzNQh
cWeZdJFnaO8RUxU3KLqEXUs6DVTFPUZh0GoyojUMRaegEWZuNN9CKQtZ3iZmK+0FD3IMwYPmBplv
hRcPMFMXvKUWw6VpTCY0Ysh8k72U4nkiBUBaSo583bCElFYRe0UwMBmqCZb+XbHFEoOUJe5T9rsR
2167Q2qzez4ZkaGyZmVGvUrawRfEQi22e4CiUOUaFWo3c1+9yyLZ5X04iJ1aw9TfPGseyZ+kt8rA
26ZJ4urgBlVjyJMOli/iYwYuWRvJHMqBMPQ3BDoJzLnHQNw4nhKS7lYlkuVLphEqeuJazvUFYcym
CrMW4VTcsuKLLJdcS/EoKZiXSTVzcHqWDGGelkHLLEgGJg9CkuKosVhsi0sWwJZUHMKZq4pB6eLg
y+hQRThlxlo3cu6ZumYj0FhLjqGpvpeZfRhmOUvct64VguaRiAtvfgg50ps5LmKDJkRxFqDZp3iG
eGiMpUv/AECW+cHplxWjG/Yh+wsuv5AETuxHyxgcV9NTRPL6YmwrScRDa924XbWqqtvzGkd2tBZK
GyzYN7Zd2R/IwlnaX7jK3MHRnUUsvYZV3wfTMZYw+GW5dM9HEufLZ8LGPmMOOS0WCzWai22UpNl5
iC7iTKyijVTZLEuek79GAV0K7y7nBjFQC2agkF2m0gVrpcDujPaY7xy4el8QmLgy5cWGd5aEHMVU
yIXmXBgvpxnoPS4alsUteItM4ghm4o6lhNyzkg9LYMGDeH+HMvvBJeZZOZwS5cvqsx2mLCV00xCQ
JLqDFsuuirzU46cSzcJVypcMEhccQz7SolaMIMILUgjekHZQm5gVbbEPoL7xKUktAYibuP8AYz4i
LwgLS0l8Q0elPkzCEcL+4qU2PvMRfJTBEL8MWmq2s3eGH3eCMdqHNMoN71Fr26PG34GL9UeW/wDF
DfgTOJv25Iw77vS7izthpHR5/wDaMRz+uZz8QcaUJdZQTKhG3SRWzA/UDvMTmGmJOKqbThuC30Go
Zi930vEGo9k7mojCFE2XLGLLI7uEahqO8S5qKahLhXS8QYsqFXUSYzG0IJzMXNkMQh0sgwEYu+n0
hWcunEyhL6KXiHdFhfTmXNIPEvMW4sJfUFY6YNdSMahCXR1uYg9LablQIlZGFwW66HQsUxVYyL37
kdMRsZWWzZ4DxFWsJWXMrzEUDkC+kiX9AB+cRBHEpMhoRfmOvzjxdRXni9Eepxf0oR/LD9nQFHvC
Il6DYytj2s9QkpwfFQKa0/5ngT90yU2p/iDTPEVrAvOH8R1XvEC6b2Goae0eS7JWPdfhIHuiTKj2
RRzA17IZuXNYa/uAyzG5jo4gt3M1vovWly4rylPecPW9zJjquAuduiSyoRVY6qX0DbK61mXOIwud
gzD3BaYS+Oi0EZzHoXMUdGrhqXURLa3B4iuaIYnMZaZM1HqqUzmZnO52hnI9LroJiDgQ56OoS8QT
tOZfbqbqWJPfWoTU4Q8MptZkag1cdTAaXY/USUleYTXwPKiJbVtHeBS+Zsu716JanP4GFy/3Fcta
tkXs3G73EpX7fmGPBD5IEuSV82y6hr9hjt+n2VHWeriUgn1LKuQ0rCJBedw4sbYCmyLfO4Udtw7p
mC0Ohx5If95Xgu2IpKqAEVCrNSvrw+SZ1tD9wGK9zsm6QWXMjJQxUuZuKzD15sWTFe5CghJ677s7
Sy5UYOpRcJSDUsjS9LrMuW2xlxinMKlzNal3U56VFZg301CptBi1xHHSt1MIdaYM7IZhvoQl5rpx
EupYEuEy1OKl1FxBi3Pcttg5nJG3QzHFHS4LFQlMBldAnPSubmXS4AS5dyljNMrFQolly4JbLguo
EBJfeDIsh6jLnM5IMzSAo0h9x/Ar7O8bkptOyIoz2yiUk1zRMr6b+YL8KfiWK0iPzA0rxRyiKR5O
S4SHxwvNwtu1T3uJPoqUmc/5U4jWvAW0FSr/AARauIJdsFCqrsTsxvpH3uoWlJsspPySEvdjE7L7
Rhe6SysrxnMT9p9eieRlIBuwYyCF0guUDdQFQHomButkuUO+Y2pdxQC5cbYkVhLINQrvNQ5ThOcy
8MJVTtK30jDSXBmpdXZBNzHfpeiWTlD+DuHReZmZnU9AxVBXc9ykuX56cIsuoW6WR1BroysQ6rAK
O84QwqVLol3Lo6YCEYnzBxaqXc8McS1ZcuuZd6jHJBCAvcxubYMalnQ1CYrcfcqW6uC7lr00xeYZ
voVAAC8PJL3UeGM0R6W43OYuygxVSeipcVcsu/iDQjFKqHieVnOZVadxWFYqepdlTXS5ZLu5zO05
hgYNxCjpQyptlSo95mX5qmZF4meiErWYGSJFLS+tsHLKzFsFWyktuDLzLm4b6F4izCFW9GBlGYG4
7lQmtLKejcOleYkGjcYL0KMSXcdFQ6EOY4c7hklxPbo26JTUdRcamWANsODGOsdSXDpXVm42FREj
dwcwepFraHQG5qXCXGMWW3D3LJU+Y3LDOXTnpnrgiJxD3PaVDcsJYy+mm5kzmXQxQ10rZCcDfTEA
xLzK3BljVMWDE3mLOITckSlsUF61UxuXnqsBLselIVEHdTSMu4bZuVAqWzM5gCS6hDTLirfW7jZM
w6F5ahbGZo6C8TOegxgrZGxhlpLglx6KuFly2oxS7GHQxcKZajNxWoqzOep2I31ZcKjsGXEL5j0r
HRviWlzMPfSostxLi1DU4jCoPTmAZUXREqYg89DqmqqEcdBoWWMHELgJUsumZcxPBC2mLMK7wXQt
dpfVu5ZB6t4g4u5aEcwZcqyV5uUYmiWSZqNgdLb6ViZJmIhKhggzEEo6gVzMy0NidCDEhioahuU/
MpGOpkYKyo6hTEZdHQpwxKHMDHRYPRyYgokaZ6gziG9y2pnvLJbUFJbk6CGZcWX0NxyenEMHW6i2
Q6EMIrpO0UsW5dRcSllMGpY6g9XDnphiVLLBW3S+morRKGXwl94zScxuWgMtWIIPeCDW42YS89MS
iWjE9BElhxxM9SoUo9+huG+mO50s6FegLiZp05lzMrojdSmoXBctuqmbltSxYy3aVCkMnRO0wdxy
zM5iQ0wuFX0u4QvMCiaVQtIkZjuL26IX0GczncvzDUdOgXHDCVfEKjLNS7mlsWbdKYnQlzJPcsup
dUSpU3iHTEZdM3XSyKS42wsx0JtiuYjCcTUGY6FucMuCMvcOioalmyO44XxKTiaY0jaV0DiaILIZ
OgBCrScS3tCZl/c9xhGq3K0kucsOiy6xSlLiL1ToGObnl1Xodo7gx1dRbLwHTCMieCyzLxUplXmG
ibYzwj3gZc1qbq5cHEUpqOZ7czHQmIMsvaAu2U9otXDUNQjluJM3UBhQxFGNZQg1K0kCpsdLehiW
U09DnodKurZYNdC4LmVggiEYDLX02q+oCblHQyPTVyo8qlzt0R3C8V0y6NMS4sKOx046SHaLWpUa
qiahOGXc5lxNS6mYMRTKYHbERswguSG5UZeJvD0WYW104lstj0GbuFS4Hf0MCiUuoIWTSYTcOgzb
0XEysTRfaWVcQGY1lcRsDXS5yt4mWpcKirdsNEVuDDcLmM3LwkuL0rMxVy2UwSyIrqV5YeXMe8sM
xENxKmTHbUuZuLSpm6gMzFuXAXmOIIZvUxmoENzMIUcxhEdKh0CF1np6gzTuE3KZyjFW2FRIgS74
l/wWmZGo9OZ6m5zTPCE3GW94JKZUGVKxHU0R6YZeagLm5XQwRO6mHoe5iNRm+i8QiYuVXM0bhEBq
WhRKeZm5dxdksHR2wTxMmKlcQdqhdKS8w9xNkzNkmYbVgg3AF0mO6Donyhm0npQFZ9tEOOj0THoH
sQoQC+9CBOx8stCIWYQbghibhS52jvUzHCSxuAVMW4gFQKsvENkvMKw5sl76XBSVfEYrxKYEQxC6
lsNTEdwaI7HS4UuGGUPQgd5xUph5RruTTTHcYuGiYmmM3CowYbYCocGLMSOWGLl6iwhZowWqIXHf
Sl6Uu5eKhLnM5ZVkMPMblgIy8M2ROh5jcqMyv4AKXAUloG49kwaikorZAtAkQLuNWM7sUQRe9ELd
fVLP67SrPl4gu4a82DyztxiFOAe2DwnFVVwG/gUq4fEGW/jCVG/6hyL5j/2oXWb5Y80UrlWLf1QV
V9MB0XxNlF6IX3R9RZjgtLu7lZL05iqi5JwpnubYIKz00MW4N9DJmLqmXcMN0dyqSc0S4b6EYY1J
rHRpYTneJfDcaFSFrjbM3hg4g94reDoxZiCVU9w/hzLuBKd3CUdAJUVgrM/yWEGHOL6BLgUR6MKj
llTG/cq+lNQt6JNyllPLAuDgVyQBlYuyCFqE8ScBsi539MF6+9Ocb0otps9MxYm4G33D/wAhOyMK
YpnZfwMK7b0TlfWKBvx2IkbR+Id5+2ZJT5gV0+ZXa0qAoC+I/wDlTtH8EXsH1AsjFcNohHMhYizF
6hF2y9S3JChkiUaIovBgfOZkyYmViJ4+5gl/LgFhPSLF/hQ0zSIJhkDv97ELAfumCXb8pVZfmst6
XkZm4iyTHPmyagpAbVQBntUgXrVNL76EIz5lIKzFQsTREq4+IRgEjHJhhLg05jq5dlS6qL3lMtww
5lLMMblsKRLqVKnxL8dFhO6OXDNEvEVlKQcMVGunL1+Y2OhuYjqd1dQLTM5hjbMMxWyX247QlnAv
wwMF3pyzSebzbJgML5Qg5T8pg/6xWlP03FXPxjAspjLaPQgY3vQIFkT5JkuAT90Zbo+2Z7LIbfpl
3H1Iux/ECOpbzE82yxdzRVlpuD7xcrUt3hh8dFtQ0iE0UQ5GI7wVwAj9sKPjVkWxBW+BMzrZb0LH
PXrOFz4FsFcs7klH4ib5ZW8H7YQHDiQrPzp677sf5Kw/SK4nU17pyvVt7CGFV3CSK+AYGa+4JnLR
3Y/mCO1Gjyb6WARTTVhlXjjqbu4NBDAkMAKIOEAuigURY6LxxBUMAheGoj7gAeAJuosW5C4mNywS
XCZnEJiGJZfEeZSoHUAUXQwkzHUplLqG4VO3VTFCzpUxzOMdDKYIb6KgQMblouuvdCrJbkqZSJUM
kqM7qSzaI3SeQvklJq/q2Nwn1K1Q77x4Zl3XtaIW8PYhsnGcm9rAD+tYXQKWtoLh+KRgtvfBHHmr
0nIv7c4FvaZR/wC8AcH4iQr6EQzVXYIrK0xcy2c5S/lL6tizmCeWLibuKjaTKaWBZoib2g7fZEUs
PhB2o+zEltfrGZb0PQsy4T04a+w1H6PNgDB+JOV7p/efXSnDWyktr4RRpfsHHfhVf1EBX9OCP4SU
H4lT9w9mebCUK34gI2NbkmeUgHAHy+VVPShzmVOXuVBK6o6hVbsCtPiIAOXQf4hgroxxM2YmPc4Q
u7YsdwBksXuXMMWsxrLtHe7YlFXGjn3VHLbOJhEnqA9lIEtxVC2gShTmyAKBCJlLuZCspuWtqhI5
aLuLgAVCRVzIzFyXtRfEQNtTjuBxejMbIxQsMtLbgwVBhdziA1EcE56EFEVNEErJUrjGJp6gtbxB
FgAuP0W1c5qB1LhcuEd6gVUteormoouAhTmV5I0NLCiZIwEe8vaV8o4KX04Euv4nPBXkj2II6/cX
sPuU5A/bMi/BvAqDX1vA0gLbgMAhfs2XLc2yFtvagLsfKZoyd3Vw0PfBU+GMCCgegIyrbcvbtOKC
5mX7w2zEuiUIcy3aMKHO4LOGXrBnuRRp+zEszcl+mVYRvdMyq9xgj4RFwuPCjxXbubgj5ajapnuV
WBvaqveYtdbTl3BhetG4iHvDlt6XdYhdG6FQreAuSFI47yQv3Zlla08uFHOFc2X2goobNtiPt73I
AgQowDh1g1OMHcAlUGWURzqnkg5lKzbLd2jUPF1vmAooO8AYRUSyoJibtM6loKZa8xNmV6IjaFTC
AWmAIMVvm4LWtmFKRITyLgugl8RLhnCNs4jDYFwYMmnFx8B1MwVaiRXcvVimBCrm8UXmLuzGoFqO
eIaCqxpKtB7uqyy0XuV+WDWkjfdO3QHclxXS6Sg6IuVhAXUNmqQuiBlQiRu9S7ll6ZU5zATvoeBl
pVkLNUhCrRxLofEbWvwpbH6kFWkd6jcAfKTHkfSCKvZGO9ggeGYLWegjV2PUoc/cKD2LjaK43cHd
D7gDSylef1CrAnqJmC+IDoE05UFaie6Kp5EdysMkOaUJhDMyKNwyYlHlL7sWlpe2bIfZn+MkS38L
cd74hZSxbF1hftGk9LgBbbatZN4vmQ1fcMadcVarMhfbwyXjDsLmNJbeSWW7bZAppXqo8LTyJTvL
eWSi6LIR+bykRSpTU7lINmcKIXMJV/WkmYoGe1RYt1EkuF3KTygiF792QJiCxJ2I4zJzBFEHCxuE
uNwXangiWg3V1LmUP9OmFRub8MLi5igRZKHDpuyIJkyuKXzHRMvXlBhRTAEEzFPnKCw3ARkOZii5
FlwF4i+6EMhmkaiq4QtuLapFrNXGm4Vu1g99TIqO7xURd3AVVb3Fm4BdMoApnMoYQZaqohC2oxYy
gGHCJGVmGs33ig0LO9Y9nyiFtMGrIrTBxXuYpbFq8QubY3sGlyyiKcR3SkHD2w/sMbFD3MhEHZSw
yfggmF13CVVcBNwB7/cHIPzBSQmA+oMn6EHT6JfqjzUVc3hVGUtMV2wuys9othvKRU2S9uJVwwTl
gBVJXd6+5Dj+DjWKz0ibP2QwtoOsq8KZWleoTC7MRfbHzILn/EX2fKwpID3MW+yTL7C44W5bW2WO
P8sXQhewpL7/ALxLyh3VhG1v9zVfcrgb+VBEFUcqsKVP2G0rj00sG74QaAAPQLjULXGrhZkxqLq7
y3G1aM+PcTaMpY0ES7h0YYu/MSwNwC2guZAbhmAWysDLFHeIFQVuVtteJUsqgGaqWpWI7lxRqUG2
O3dHMLgxZeQqFg0YbnFU3LBdyaJoktpUGEaI6UwcsPEIAdNxW9nUMjDcUAHiV5XPZCpWfMTa3ErV
R0DBmSUvbFamPMpLEzxEEWwGViqXCsFpiBTBczLMFiWMmI/KiBQxdTRiMVuNNxIKWhwYbkzRcZZf
kxfeNtHEbqIkyy7KuKDEu24lsziuU9EWpYlmPahbMku6a6ThNEXpi1AsAblqIbbYCsItUbHUa7YX
4Xtmy+0gFtXpEOD5m2T1ALt6UW5v0mftchrfsklTbPLUoQSuWF8PehKioNA/UqZMXolKWLNRTbXs
hWLkp2X1hX7GG7f5TLRm7pZeuDFYIYMR4wZjtHAlo+RSLAKxLUtHJBtS0sCEQHkhaMi0LKjyiyG4
bu1hRFu0trDOVi4sfZCUqJtxfEcARFuuIU7xKEtxKd4RFuWAIjwG4mVrAc1MOSIrMaK3MKhhmFTA
lVnNyrTLFWqJeL6d7C+WWfiIGmciWQi15ibVsXxuCoMjhcSBVY60wDvFylOWY8zHUcrLKIuLeJld
M8wwmSo57lDUrUYEEmUNTLCBJdeYuFSrqKYJgKy5imZHLGL3npG6VbYr1LUghoy0rERcDCNsFYzw
kOwyyLcbcEvKHDFC8IY6YN1C7mdox7pCBuivcrFLqFNTgpOCXLKBMuid0JYWoKCOOtItYfLFqibt
/BllrHiCbCWmTMmmHuX5Ve6wJlvtheRfDM1meCB5cSvQ/EqgB8EGBQTiO4FYkI81G9crgZXRUMhc
QbxqVVNrYPEW23EwVwquYM0wwqK5cEpdRPJN7jSkXglza+IfLRGJVVEi1HOkM2xNYigudtxpOVuE
hDXzLPE2kAm7nvUaG9pgrfSI7N+YizCxUfXOSrcPlEcsTxac4VGUJY0pB5XPYsYTli8xTpDAHN2y
17i3Nyw7uLRnrl4jmczPuYmZrtYU5ilwQ4l2JRckL4mWUytMwVYZ3JY1LOIjxBMwSVHQXswzWphT
iBOKenbpJwbJdYUTcGhieCG2CpayTtRazDAItmWyN0A5zKMkozia1EEShqGXBPORq3MkVO4fXQSb
YDN1E2qnSpwmYRSBtxwyxPee9wETVjcKE1VYMDJK1B7jeDfuGwzSA5WbJHGa3E7wYWJUQBW4FYYV
MxSXM6xEGQmdXKpiYm9QmxK4mIXBx84PvHHDBG2UMPasBLtmURkcLcQbmFKxGe6PMwpzPNFMMmYi
GhBuLixHYxsaeh2IdyARSKYVEViXZeJd5idxWDDcBisp6A6gr1K+IbEgxqDFwk80yg2DTF40zBUd
GRste4gEwpYk0Q9IZrDo6+IVZIBeJSDrwTFRRmeqhXTDjndRAjBETo4zJPMTlY1TC5Cd8XPNF4sg
m0ecY9tAlplbq9C5ynq47wPmNf6RFV6I2NMzYi9lxv3Y3whsjfcp0xWMsqNiD7wHLAIlIq9wHeLI
BiQqbESwrqZ4hzIW3SJGi4XdxF3BW5lEssSndWYHJWHnN9wFoY01cKlvMK6lu8uKX0CihgsU9Bbv
GruPYwyi76bdF5dzA+Za2XitxtC5iZhcynxM1Ly6LLdo9kLNPXBJR0iBiuLai/BMsSiF4DpUhTmG
yGkIwJkXsxfiZswpuNGpSwThodoiF4hSHOyEkY0lpK+enftjVdyvmVYuOu5VzHkmkMqXhcDhiAuO
8Y0rWMhBbl/PS5lidm4FqrEbpjRVdDWWRfbOxRiVhbbnmzLd4p5YdMuJ2UzxTyoPxDQkidiG5Z54
dyxzu+gvtjliCMdGBGC0FcsJjTCkwuCjfEuodNYyxyxmu+i8DzF2uCjcCpbFe8s1ct0tmWCCFsGs
UcTCClquoRasQX3G5agrqVQKXeOmnborALVMMWBg5gcEBKaJWtTIQL9oUw7MuahdKSwkTAYmAtBE
MrNw7nT808srcqWJhuPelU7iNe0G8ShyRFVcpI9+pbxDLIdtCIiphBfMmZ0YPOiLcxZqPHF6J54m
5WMw8M3MwtHcse+y3lnkSyW9KlPZgnDC/U8cuqF+oJBjct5g+bJd2QPBA5I81DZKDBA41CNMVAw7
cOKFkZDVd1meCYM2lpblhNpbUXUte+i83LGsyxlrG5mXmK10NTMYt2ngnpBVqFep4InaXhkshFu/
Uya0MMdxzMUupStTZqU7QIwTsp4ukdKAckZkAIupZKXLL2TyEIu+oRmYtxsGNOICHKyveIN9I3U5
qIhN7jbsiAzpzBZBPCmaXNTcQNtymqY1WhllWRPE7ZMzCt5TYtFJ5rTuqeSKUR+DH8xWe0PLo2sL
rTMW0E0p5UGjxw6R5oM5miZe6pirgveAGCpCWCwzVL7EaMAnZ4naQ21BuC8bhTMFxWlqD4QhQlrE
lRZqAPEIBSwMyuy42KqWXoIqCyAu+iXkYiGiIWMKOJTKb69u0W5hN4Ja6Nw1PFAswym9w8ZW5vMm
VswMRNauHLuAOCUvMcmYRZB8SkFKIHAlAzFWkvsl+OimrjbEuYG5ThnkiXMe9CroMd5iDnEKthEX
gRRdkS3GtxmG4QxcaNxBWeoU6R2yOempiroy7cu0p5MO+j3WWy2Zldkp6GeRBtKeVBeIdFic9M50
B5WBe8JFZRLg7RLXKGbjAdkzQ8ZS9T2QoMBcCO4iIqUBxM+odiZ2iUc3EEuWhW7gJW4bZuTEKiXx
DKO1E+cSU2ReKnYERiTcS2lMQiBWJRBxVE5mZmUBhRG2yGOoLiWMMshILiVgen0hAdNICUXCVbKI
VmZTCy6HaUZYUZlWNogQQ3EruPSoNyrmEgRZThjQuyPK4W5hXTHUx400WKDL0jJuJZTe5fzATRls
PYi7mZMMVl55W5S42l9yOhWAypuDZqeZPLlmo8Ut4nhJizDvzzwCJizAPMIG6oE2FRHYhVo6CvaB
JysIggqKdoCtEzOIFvHQePTQdRDFSjtMzBAaMTdYlCXBTipR2lSsZnkJpfMw0CynuG2hLZWWpbdw
jmSWjUokOcyhklGVnm4B2gKxAJWZTG3cUlrieaCuJV6SjoDdylnxBQuOpjAwsjcCR2rct2lN1FKu
IEDMI23KUSSvQdhKRN0St6RMe9A7xt3TGT/CQ5lnAnhqAe8aaz0B8keK4r0rrqMFR3oemXmlvMvu
ZfTwM8TBzCnnQR2hTfQjjqeuJ1mIrMzcOdxiHDKObncIdiZMx4yA4h2umlOJg1AXolMCkwzFcTWp
URJT1KEa8TJcuLBQJQEy6gSUYgWpZhhGpaQFcQG6BmYQXUpINy1QMRtYJ+glyuV5lLqAWWl6gqWS
yoCkFeYMEgLA4GBXnRFOHE2x7JntAuU11HO5Wr6K1iFtWqGWrmFlReZhkgQnEIM4lsqIT6gGYbR7
pZFIEHbmKWMXEplF1G8bEK9JUd22GOpbWJSqzKxlTpPR/NO+zzy/anlnPHnS3eK7zLMzLDsMu5Sq
Dwy8EK+IWwlttmYXMVskNTKuI2ah20qahRdRiEYLgacTPpJgvZieCVqWZh0lCMZkJME+5RUTUB2Q
L0k7Q2hRqWnmOzcxHLMxjB+oKxL8ZmbMxwCsKWN/c3gVQ25lHOdyzymJFYYolWTFcRzYTUMzdxJq
VcpgrS+YDWZzdAzbc2obmSy4bqDtuUgRXFwgUXdxzjmXeZaCoIw1uDA5mFquJaHKNVgCZcHQXEVA
I4wRjUlJAsslV1FS+YLeI6KmPXxDvgASKGrlhYKeMx1BHnEKAYsFoGlVXPqixTK4B3NgUhAvxwjT
IXlIsWvInvEO3DswfiHSRwTBDmVZTuwjeaYBxMGqhxiEheCVagO6lO0DTcssIi4xf0LRmVLQIyGO
MxLngXEKgXFkMaTc2xrt0GYynGZs9dCa6LoAzMHJm9dCEQKSZKku4UW5Ra4Ym8mLvoKbJrVxFtqC
voVANwBN8TSF1RKIFowSoNMRyFyqWWYK0SlVlLEGGZyhqOtVbL7QBcHYlsFRqholQL1KagLYDc2+
JWVjoSuI5Jt0cVlIZSp6B8kv2jABCNSyxkSPEuVrPog9JB4U4giiQwBt8gpFr5DNOVmFwhdwnPsA
TUUewmNIaRSWWoigG0ILUL9VUmxe6JNFlX3+NwUI4M7i1LMzZrxUuOqPISV5QFaLlCatspKEum0b
zSEAA+bqNUoeGIEJjBYzgQSwylKgGD3xDpAUuNGHQCcwTib0biJxLyk2QSqjBN+CystMC2+mhA2t
zULFFyo6xAUkSmIiM3iWEEHUoWzUSO8AIFNNXKsAlHQknE2upRJirHYHEvXzfSiFMvVwyBAdRRCl
N5lKtPW60TbEG+g4QCciYHQqpWZTvF7SrZTqFISsUd4FM8JQM3biqagyC9zEMO5LCeJR7YrkmQsb
VkbILKDtAs7v9QLa+os2Nb4IIWRgdXY9Sr2puCam+F2zEL3WdTWRe3rzqNvB9ZYg11Oeo1FuzowT
Xx1GHRalErti+CUSpuMVIjHzQC1A4pcEVXjQtXGkBWWikgZYFYIuphPiipDDrteo4K18lEAS1LRE
XYhlBMXFDluUAEncnMRuVIMrCUqu4agrtpYHNwQAN7G47mFNgnIa7Ti8UF1ADb1CNCVysti3exGB
WHhjst8RYgPqUjkmgV0L7SzSAqWZq4KsMI1KItuoYpMxzVRgUuoXc5hbN4Jm5vmGGoGWamTpeQs1
zcNyhe7b6UqQg5uZlWAXAzTEwRl5lVcUlM10KVAO8zmNSy1UOl1081BamYpaxMkBFQQUQnMBdbmB
mBEWitzJEzmZtEucS7iJcRF0Fun3UK3KFhV3qeeeyN18RFcE5Umoe0/dvMSgHTeGMca6qmosUacr
UYm9O8Cal5eNQxro1S3cqU1TPTpgVhsudHAZmhgu6OSDBG2QD01CA57dlS2KEtKge7WJLwjRGD2D
VI1U8dCUgHxBFZopCtfhCwAdC/1AmsOVBECwSWXk2TFGNFMFDYHN4dCQ26r7hNeNqgEnDaBhbJGg
rYxuM5VpGUIhKAmUtauyYvam2I3dYmPNK2riDDITbt5DDMwAmtzZv7y5mX7YRw/VLLVpbywlY6Ow
HtI1Xau5a8SS4JeQPiU2Q7JDY1LQ3i4u4ZSyGG8GiKWVXbAJa4ZdFSoIxFwMy9kCVTEB4lK1E6Kx
AIwpipgCpayrxKxUJjvCn3AFyzeJWahkI1cs0TIQK4jV5JrREyENi4AZoEpYHiATIRwJcz3gtgX1
ovcoGZtIrCYpW+Y4XChUC6L1DtdDM1MeTcLWZe8Zk1DRTAdijUpMG0XG8alJB21aSkYW+GNVaYeL
E0Uw5SUJLUsCRo94ZxVm7iwuBaMDLhp0CD8QlWESTBde0AuK6t6JjL1bgKdS7GJkiKJg/uODT0MX
CgqqilHHeEqpXD1gIRSM00ykUqUAqtERQE0A4COcN8ZcQJxFNxBweHdgAm9mdQ2DFMEyQGyiWCxz
QaNxGLhFxPcVyuai05FtuY6EDww0qoErMbwdSmGSlMILw4o49Io0oo4jJhEcpdE4GWCq+iVB4qmi
mxlF1Dn9zEDinki9KRhXO7axvuarpnNBGa1Svv8AzFm37YtWh8rLuJKyRmpUugJfR6XBjuEKGoxJ
3u5OCHkAqitp5lTXS+mYoa6FRzBccARzU2uXKqNEIRFECwajUAJZnEthvUbgMu4Wx1CmYSlyraiJ
sgI2cxw1sjuBcRWYxRXnpGLU7Mjk0IXBDBmE2J8Qvasaix+oI0P3UqLdwnF3qV2b6jYLrBTdw/tI
JAEEowU6JsgtRhjQy7aKl1qwMSv1C0ZWU15LG85h/wBVtwYo3RlLJuN05JmJLUzIH+bgC5ddAawE
aoDsWVZYFW0RS8wlaNsRaMqLxmFyoKJYZUY6qiCNjOSNQJKojhK0dYtimx6jIhE7kuzKIi+GNaDl
dt7uWtJjHKWZVhc5mWGjguL4spWK4CHQK0mTMrMhILdouZWhYvvHgENY1FcOtMyCfaEzs88wf2pi
Vh+VivL7uAUfERIrzCDiIxYEVfzXP3LJftflinLF26lG66F1bMpDILfZxBAjh1KOIMRQuKttWxCs
lkJ89KjR0vEw7e0sBS0/RC5mGxXaGddKjOemehsPOId9xEpqc4fjYNMVNzCUdCjmI4lUQMphFk3C
xSUwIhiFTiEqolwNVKKgplgiq1PAzMVG+6jdgmkqFhc7OWYSNdhEmmqHO/TK6nXeDrSnJCGNQpjm
r92O0CB7JUt3KIsa3tUOwWJmVXeMQOn9wVk+5yomXNwrvDxA7W+o1BgcJRRq7WREPIXQEuaKPgRx
illezF9hZehWt1K2KUQVKi7bhaNKbq5qEebm2F8x1Z9jEbAwNsDVpa5l1GiDWwx4oOIKEtFl7mnJ
MZWGObgFgvmkCLdfFJTFfOZ3P6WBa9oh2pKqjtiw8w1AXtsQ18lsS2n1juDCvb6ELpkttflAm80X
Ys+yMTgLIMdyruMXsXHJiR4vwBD+Rwy12QUtL2Usgk9glim5yE4E1YTNSYN2vTFxNhh6zPkGlfE4
mIx6CwqpyQ12zCOUtEQVKplTMoNMTMKm2oqvVBecYFQUekSlAY+Vli0a5+vJKqOi5YtiUJ0OgTE3
YOLin8D2YJmVFUGCczgw4SZGIPxFsJCyD5VBakCadzBWuAnKANWYGUds0U33hEZWYW7PMaV2wBaj
MRa3AForEC63UWtFm4ODBgZmYyTEGreY0MHxuVKojQBUVqBmDgOYkoa+YMzVVOdwi2ghBS1qOyyM
sbUMAEJHew0uobWxfFgzdvYRdQ5h7YTBbm0UrqPe5ktz3uFj8e7l1SWzDTJDP1sb6/BErRPMJ4wi
CCPMJMHOY/3CbD25rL3uON3gr0i5o87LFlvvKf8AVlHOfc8UpD1FlZKDUJVco8zgA7JhlH24qqGx
KSZP5DMm6a9RDFk7YiHzB1CzMOjBjllwKGWcNVcJX0WLZEd2Ub2yvcifZAH0H4EyMZyLvTphzrT+
JUG+lM0wlwogwtLyWTOey56RKehcJrGX0plTUu+m+2FojLZHtkAJA4Lx0U3CitMplEzOMzic2b4l
ZdV1FWHFQ2oTQlnErowDWZUFsqYu5Co6xUoVoiTjCNUhYwSxdSi9XCvIQnU6CQlJfWJuBIBLlnYq
VrJGrMAGiKrmCyuVzcaVRCOSCcs5RIhYS6q8R3f7gCqMSsXPE4S+ou4ni8S3aSUuGInKKctgi0n7
iWqq5yuPBQ9T/gxffrFtzwCfmK7+xBXL7nz0p1URWCxYS3EviMA2Ny3WBG7lmV+wLLzxBiEp6XGH
Q/NXmmoqnilK6ihIorjFdh4KFU4bO5CvYa+4bdCpZ0ejoE2MC8Ux4o/grwaqFVLxHcOlwpUBPIwm
rlFF0uyDKHe0L2vukcYnzRHZIrg6hi2kcWTa6l17LqIJYQPiEE6qxJD7YwMkeqDlgjG+nsh0vdsd
zfQqpQSwm5Z3mBqG5BZxvuQ5y9wVqvkCF5fvGIWe4tP1EqP1KlwaWJMGkjIQTJKoN8qPSFRveCd9
KVJzhhdIKhZn7J/ZKBP90G4HxcUdxrwwgsReTdwcDXxPLMQUgLhnzMT/AG6Qs0Ni8gGLKi3aiSTs
fsiXd8xXbfM8+WzZuUduqRlLmEDlEMUykuUtX/Ax6MIzBu0BHdZzAtYcKAlJnHG69RuzQt6V/Jbu
XyNkQgp6qwUgwIrCV0SYOgKFn6YfcgHS6lVDlpWglGlCIxksle7q/wAn5I1BOgMzCVEWVjq5/ue4
KtoCe7gxp6uf4OSDMAUZGoJSfom0+7LpV84qZftWU+pZxPboRlkvzDJTcrHgD6YLA08xK2DAGiQL
OGCTD9pgtH3DT+lQowfhYg38U+XfQP3GvP2BKzCPmb6UPe8eIfzAgoHxlLYR8Am6a+/vi21CFIwx
6So51FCPRuljl5EeRhIUxeZHKLEsm8X5zyM80tvbBaLdwhaFUtlcvMCcvMve5eWWy/5BaSxpT91K
bqs9K6V/E3FDY0xOQEaqW2iVc7Rlxl/wMSyylBokZVYYAnoZQUeqikzKp0axXSul9Noy32QSaI3i
IJeT0EtYks5f4FzFh2pIw3UowWwjXuISFJ3Y3KBcxU/TAX4l3iB04Tef4vMqobmCjqKg4H9SsE80
jOaQiBBvmXCFqML4U8iCw9s+5xP2xEnR2YPN64Jf/ngPL+iU83tmDX3YFNLJf6l8g7ACKx3VSyB4
M+7dGwCbIS5csjFwZcpeGC6Ut19Mu1ETlANIrgTfEhVnP1E5PqGC7w4Sie2Q5gVM4PJmZAoVMCIN
+4h0ZfS/4G5ZFUxWray25zakUbH+Vy4C6gQlCYiaR/ZGQiVd5Ibir2afcdlJyrHqv8CBHTBT5oxf
MVDJHpj+IzzSK4uefWSCNB3D3UPkW0LMJf8AC3qi4598Ko7FyobQ3XorFwcxZzLhGFCXLlzcUJl0
RGUy8S4hfclgpfcYpV71UcTgXAxfiJWoh1RgjFUx0Ji19I17IpzmD+6eZj5PRSmKS2X0DG5cz5h/
4kTKoYwU8/LB+sCo33WJcuRAuYfCg3uQrzlQbuBpby62twPGUTc8WTRp9Sm8GonIENxDtzPmDtEv
/u9sRaE+jpQmw79xQ/ynaOMbE+oRaVfU2fwMt3T7jbZyyi76PS+meoYij3CQ7ECFQSDVxYR/kkzD
BUCjBBqqwBDOq0uCa9ml7Y0rb/KupLY2xgHR6nm+h/8AiEuosA6XL/i3CuUsvWInE+fpBl9QSWBu
U7z2lneU56FY9O5PNHvS5zFvRc9pcGUmhBggdK6cmmbsDknBYghhQ4/ogmoXYtBWibLBiC7EEciD
DJc3gbloHADSFYZEMuVugGaoJeCvUDzDxBBBMckfLBtZvSOp9q47kVsv0Im1b+JQCE8RfaEt/sY2
KMxaMW4fUWlTfxM0SfeYFDBveiAoY4lIQLMqAPmnUZcvgNB2JzLl2w3Lb6A3N/eBBKaboV0I8S4M
egV1IamcS6Rl9ywet/8AwUAGklvLH+QyzptDY+SB9BDUoSwMM9FJcuXCLLl9CkIr0Uh0lSpu425u
AgBa6IodguN3qYtyhApHrcuXL6PSu5BNN0DSSwW0gDX3QbiFbJPDhQWoqs2Hj7hmCymo2LgHpHCZ
FFaw1CAyL4lPBA4sz5gUEHcQ/uKP17qm2vxmerdHEFyvkAgsKiOM/LMMA0+eCaj8OGGhUXEiBQAy
rgI5s1KMW9i5lkfmXPZly29y1YoI9sMAKsQWikjBZTcrpzL3AaWNEEtKhi+8Kh5goxCDMQOlIHQC
scoqnZp6l3MH8X+GEvOJCIIPqK18/wAL/hfQYsGXnpbBiy5cuWMkXMF9ulwxwVUD63S0ntLly+i9
LilfyzKZnswXS+pd2MdvQu8Uxq/s9iFhEsY2oy8Wi8kUXUFylD6gBuw5Vh2mYVyOxeObl2DOEIcZ
li6IVqCNP9ktW/dDDf3Es5Oo2MkeLDAEJplE2MHOpVYqpc2tYmy/zF8v3S9lfLC+7EvMohLj0v8A
gxhqOFRT76DZDGovYiRxCq68yrYS+joTP7Y8i1RIS9TjoS0scmSVIiqjASUg/wB5CWGXRhFvoGXB
jP3Bwepl/wBli/CS7/lcMxxG5WLzE7RGVlxFy9LWX0u+lxYfwuX1JfU3BiEqXDrR1qBR0jTgmRjW
LncmSAmlLtKd6dAEBcCZgshz3gmlMuVAcspt2YBss2qxYbJbGjAMyuFfRHKUhvCod3Pkl54iM0mR
CnBEbI0ow7ygBmnH3YnX3zhr8wJwy7n0Rv0eopv74NSvzK9f2itrFe7Nw6HfWYKKXt4hhcJ2QJaf
sHL2gsy5eJdn8Su48Xe5c0zEsBLv+NzMSIiXcIZ2Q5jIjUCLC9MwUivUTtAIbqPiBUhL6H8LVeWo
zCOKxFpg3EvgLZGBJSvpUwz0y5eeoS5cvpfQai3LVWCUrxMo5EUpOhnrmCcdK7hh3kE1KOgObEzb
g3CHNDVbILtgW1iIGrMzXwFS6lRCoZio7QgLxKIh4Oi2XEQDGTBkmUcWY958EwskpwoQK6+RjWUL
8swIReEPxGv+Mstgs3B3PsiP8kA0+6UjeR2Q0RsZRMdonqO2HWuHbuLlzEGLLgigW4fN7rVErWVa
DlYgJCwepGHQjZHZtF9CKTd7ZYyY6EuXjphGWqoXPwYvgGMC73Dp7Yzv1uHRYZEmFfdB7h98w9kR
QlI0xcQu+tkHpxB4DLRnMOhCB0XpeEvNWCFS4LjeT2RTcS8QYvMuLiGuukr5wXlPJh3HWIJS5WLv
vAwKCFJLCIMrrhPnoD78Oe8JeAJZyJyBK6AVKbBMx9oizBL0SptEQ/8AMS5iWrlN0kT5RF4lrwI0
0Rt5CUMvzRqXKgnVT3hQERIkqoWhUr8bg1Fu2LLmP4ESIhKnTKE3H+JMWLmWxRumZl9s7Yh/jx0L
iZZfULJ5wWNyphkPyKJe4TIfUmrEyB5Ogsagy5ULlh6lraWhKXsX5IpZiX1uHQi8MtRvyELIgpCJ
TVxp8g0XdHS4RdnRUywlJTBsR23EolRFVwJRLC+lxh/G0IQZeOmKgxN64QIOMyvUUg5CUN9KK/he
ZzuFmprs+DRc8UMuOkrIPdSpdko2mBokJPfMVCDkgcKPGJ2IK7R7KX85lTZDSi6yR9SumKXRFSoR
bmLLWMvq6nv+J1C5Rr63TN03Gxhdq3sbNrbLj/C4IbltaHkWXdCmChcHeIsDUu/4kzEsNwJm/cGK
mT+Xo0l9dyoKIpEdk74Ut4i7LA93YmbST0kvwtWPTnqwXQ7QDuRNJVnKoQ/jcehMdLl+IfwC25h7
TbhLNXEYbbgxYgwc9CXL6VHsJ6IBqoLW1QcdBohjA2AXEo1tNNlyhFvOdkrWdLNIaGwZXRFXTnpq
DNQabln0xU4EF5iB+BEGmorynkl+8V3ZaWwF1GzT/B/gbiYqEHLrelKIuX/EmLJckoW7qVJ3g3Ua
tr+D15e5arzBoEKSAhDpf8nRIKBkjgtLqcwTeZfjqR6YgzKhonp7PEeOQSEP5iLpqXcodkuuma/i
yUNvcvNLhcpkQwx9WEZcJp6GoMahrpfSyyX20DHua53GHpKm5pSplw630J3oEozGEFBN40sUgowR
fRAy6o5lqvjTZZGOIVcsGNzLDBcOy5UzLtRnIPMOLcN3KdMvQ+4ASP1wEcIo0Ea/mwEUKTFQbAYx
o1f8TobjzFT9xlN7C4yvL0P5eiVwltXvG1GEIWD1N9ToyL98So3FES4fxuIKHLBd5DMfdYfxIMs0
+JgB0N4iPTT+Nwlf/Go2khFc2ItC3EsTJUZvhTf+N9FFKKUVEbZHcS1ysSqYvLEqBX8BjH7shGZA
MvEFpdChHZ0vosuX0WajscA+0slxCDSQxw1WcuAVtFlMSqDYlMEYLslxrxV4mUBB3IyKOyKELEcs
QegMpEsVQtVH/wCDRYikXp2fyHobYcymbFb3Kiz0vrbGERxdikImjOEAs2b760fwuP7w3BLJjGZr
8f8A2D+F5mr1G/dLYBA5TUOqy/4VRvcXBrPmgyIJZf8AIYNMs/DR+Y/gJWE72XCGv4s1K7cDwy2R
incdv7F6wcV0fmXBlwb6KIJSzTOswJi7iz4mRp4mapcr+F4hroXQMX31Aly+ksMCgS4vvYYMpLl0
OIWixIQZckxFxTpBLNN6IvK/hiJY5+rARFP/AMLNGZSTwUISmt7/AIM9SHUf/gtIV+eZjiZ+ghCU
/wAK/wDnUzJHQdmW5l4XtTaLH8Q6sXXh7qULT8UIsDwcjq/xYp7sg91Eqo3LKG/4Ll9VdwknDYEB
BMtp3hikWGT+Ay+hBBaYZeIxlQuMWyDUA10DF6FslvRhGk94jv1qWm+Jvh8MZJPmObuPyjUSyFzh
kjoKToLK0g+CL8XrEU398e9iuVY/yOj0uP8APULOOFzhINH3f5X/ACIy/wCNKz8ZiWTEiovBDovU
6hLS2soQ1YWb/nbEjC3iKuUTMQHZRS+hGCR6ZN98qYxC8oh1gwCAEdl/nkPEEy5SdH8F0f8A8Le8
UKVO1wiyzouLLmEQI2QqOzMrGIFfLot6AhTGCtfgE7o/M1LOUir5FGm2eqi3wv3REtfNBQpvayhs
D2LFNnpGUCA9AIqN/ZLt/fFO0xWWOGpxFblqCoQ1cply7gLhgmnBGrx/+99dEWpxBioZdm0S/wCe
2M0D5gKJhjttSVmVCrzFZODhNp9Z2dVjCNPENYjk1LFvYhGH8SZQ0ZYpm0ncqOQURo9QV/Mq0qvb
JdMaSCvYiP4H8MOndZGLr71SWO9u0S+puXCXCqGKL05lwgPMZThgw6V9BlzMrsYK0KIR3n7S9/3y
kO5Nf+CDcsUTCt7tFiiXBhOMw+GSPVrrb3li+iyv4C5bLei+xg2lBdOPVeXClABAThlkZWpIbpOc
J4EAYL4lmLPEYP8AB6XFAC2Wx8kEgpuoiR6MKnazii6gR0OhvpGBYQal06vWupK0Pid+4Eh8eWzq
OoFu4MMQtuGZoeMOmiIt4mBL3ELXDoS/4LMFZgpR3GyAF5YiSKylBngK6MP4LKUV7jDUXEqXouXL
gy5cNdL/AIXCUyuyLrjcoIrc9uallxyppuOoZ4ERwJaA66NXal+8Uy+ly5cvoAQrG51yxBgqHibj
K5QatDvoPInaHfIRh5cOdQKruGskE+B9SlojDUqtsIYIVRUb9hAf3IiZE2qR7hYhjrHANETidkIr
Yj0JX8eU9sDgIrY8pRDSx6Vobi+SrRN/0IM9HBbYyo2Wez/4AItW7gk7Ety62cwziFWPWrAXzLXa
9LzHYTeDEub3nqTmEuX1GKtR6vW2LKWiElauHJ36Ll9CPSypZUuX0Yy8VLy/eLdW9CvpWwIkxUFF
s0xCjpcuLLhFz0Ze4UNCQ0l1SeEg5axypO+HizMoQ95cGXTc5UOVrzCVYOV5EH5JwknimfEHxGQa
qJzAfmOxAR7ZOC85TLtuLZ5OinHVmiGQR/kQtlIN6/2j0Ke2+tjLBLjsu3qyYfRBHol2y5cv+BLh
YMHEJYkFVY9Lj0vov82XiXL63/8AGyWRS+9y8sSgaYKuMUM/6ep/F63Lly+hbonZGDak6OCU9rvL
omHoX9ly+1AjtPdA9jOIIYsJhYwPYIRT0ckBzKhqIdod56ZaVZe1HBv0ThwQuPzinSUsGWlWRXS6
M5xO05Zhyl+yMegR0EvgRXbNLCxLZbDPW5l0uWy5fQaf5MGmElu8dyRGS5f8L6qUSQZGo3tYfwv+
d1L6X0uGhQQFWs6GsUzDN7hety+r/wDC5fWnk1lcgkbhMIk4GOJaaIv04jiHbId0mPMKu4S5MHyx
HecyKNSFWMC4wF1NVhaQdTLEcSzmNLmcAdBPZxEvKd2ZwyQLfKEMN9AYCUgMCFnJlXIdiLbSxfbL
d5pBRbi1CXB/myKYR7IgaxYS4xB6XHUejDoy5cuUgx/hf8xl9Tca26fLZR6oioz7pf8A8l/+FwuE
BHfeaDpfDw2RaEqP8MymVbTCzaWwdMjkxDvw54O6C7tOYrK+idDeYxUBllqKj2ScKILMISpC7Upc
bQJxKxnaxPmVtCYxPFJ4AmbZFHhG4m4we4zncZbC4suXmXzC6ygEBlcyrY1Tbr2iNysGX/M/lrov
S0BGkYa1MMzLL4LQuKpRlELH8GXOZdy+r0XpfW//AKZEq9QaFsxAqGZ8XW5Vf/OmU9p4GeBnkQ7y
aiEMzRC4BroRgW4jyEy4A66HwSvAQ3YTjBC3EreoNYICtSvRbsQGVJzWQFv80S3JI0nKWIaETThX
/li14yztHgjlM76gGnHu6YMjHM6CTEJ+TLIYHB1uX1v+Vsetx5hBbMoE5JcLZ2gYdRqW9L63Lgy/
430CSbNCK5lExMLcI+tdLgy4eAPFwychd8JMEIFjtLTf5v8AK5fWpknEzSyNHQW7hokBWagXDPJ0
yRbfR7kH1BzxhxAJ7EeKVQIYIFZIIYlAzLhEdEo5OkKWTE8V6cibnCVw5pVlSMxO5QyvTkDLHXgI
vamCRHlUVeY9L/hf8big/LFM6roxBOWj/wCB0WXLl/yGLZcHvalQ+8kSlPMP/kfxv+G9ZXOfwz0F
UDY30WYSpbLl9SO6ZYyzv4hsalXjgVsTTHdbHp4YLxPFDvoO66JdLS5zO5ADL3B1yOxDbgHCUELj
cYOzEeWeCJbUDK2WeEScDpK8QoQEZ23A7VTnrnElpqGnUoOIt3Mt3iucXuS5S3pFuei3rccEhY3w
SwehURO+BEb3fW//AIX/APBcEtNRKMdkAU94fwelwZf8bly/4IqZ5cdNN4cwYPUhcuXGa+x2yvGf
inlCUCSA9+g/xFjJsZbIqYgEildFLxL/AJUG677XGh22QQUcalHVlp3dE+iU8SnK4dKo4IzWmHRs
7SO2yphtMrLjHvM7sImC5YsXiGOum8zcRdKeTF+UW2oG3aBA7PMKvoMkolIAEd04al9L/jf/AMhi
sMwHaGEBUR3UDEf4D/8AapxLA7kroe3hIPQ9CP8AAFZVyXnqS/4vMXvzTGbzWKtPQ6kuj+DtpINk
qhjcdXESv8Ll9BlM8MoB2Q3GH6TUUYfxGFGVXxzMwx5Yp29FGKngngmgOIDTmCS42kLzJeAjjTiL
conal3Mt7y3vLZb/APSELwxVXzErUWFi+9CX/wDj1mrEA6Mv8jPS2XHpfSv4HViHE4ufcf8AwU9C
0YMFtdy/BHJD0pVdB1JcuG40UcMW03CGyd4IKUGEvokOl9GDUooRELaI3QMSiG6ZUvrSPTaCa3Wo
tkJJ7QytDBis6XHodHrXT//Z
--000000000000f639c50610bd461b--


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 22:28:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 22:28:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677163.1053565 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXTv7-0005lR-HD; Tue, 06 Feb 2024 22:28:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677163.1053565; Tue, 06 Feb 2024 22:28:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXTv7-0005lK-Dd; Tue, 06 Feb 2024 22:28:01 +0000
Received: by outflank-mailman (input) for mailman id 677163;
 Tue, 06 Feb 2024 22:27:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXTv5-0005ku-Hz; Tue, 06 Feb 2024 22:27:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXTv5-0006E9-4X; Tue, 06 Feb 2024 22:27:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXTv4-0002R2-QZ; Tue, 06 Feb 2024 22:27:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXTv4-0000bp-QB; Tue, 06 Feb 2024 22:27:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=h+fFBobaNqhpYBCEiAERX2l6oNHBrK69TJkuM5o9GbY=; b=WVA0BxldEoxTR7fHVF0zkw01cK
	CsfrHfLkZx/4X5ogYEEQMPML02jiqmBf+8Bd7WlwNevbor/8zhvWm7erHX/yiRkY4rDLbtZP9P0PI
	1j2dyAdL6oUXRLdo5/kY/iyclE2He7dAjiVFIVEeDbg8P3b/IaYV9eyWLOwIfje1Noh8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184609-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184609: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=1d0b95f6457d225c5108302a9da74b4ed7aa5a38
X-Osstest-Versions-That:
    ovmf=62b43ec8960372abed59c1e1d596c2b324340a07
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 06 Feb 2024 22:27:58 +0000

flight 184609 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184609/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 1d0b95f6457d225c5108302a9da74b4ed7aa5a38
baseline version:
 ovmf                 62b43ec8960372abed59c1e1d596c2b324340a07

Last test of basis   184607  2024-02-06 16:14:43 Z    0 days
Testing same since   184609  2024-02-06 20:43:02 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Doug Flick <dougflick@microsoft.com>
  Doug Flick [MSFT] <doug.edk2@gmail.com>
  Doug Flick via groups.io <dougflick=microsoft.com@groups.io>
  Zhi Jin <zhi.jin@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   62b43ec896..1d0b95f645  1d0b95f6457d225c5108302a9da74b4ed7aa5a38 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Feb 06 23:24:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 06 Feb 2024 23:24:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677170.1053575 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXUnR-0004rx-L0; Tue, 06 Feb 2024 23:24:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677170.1053575; Tue, 06 Feb 2024 23:24:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXUnR-0004rq-HW; Tue, 06 Feb 2024 23:24:09 +0000
Received: by outflank-mailman (input) for mailman id 677170;
 Tue, 06 Feb 2024 23:24:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HdcV=JP=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1rXUnP-0004rk-Ta
 for xen-devel@lists.xenproject.org; Tue, 06 Feb 2024 23:24:07 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d2592b93-c546-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 00:24:06 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-40fe79f1aaaso9578405e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 15:24:06 -0800 (PST)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 p16-20020a05600c469000b0040fdd8f5e18sm141232wmo.34.2024.02.06.15.24.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 06 Feb 2024 15:24:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2592b93-c546-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707261845; x=1707866645; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=d6GrdyKlPKPAQWNWWj73dIKliIETF37U+jzX0SwkFgA=;
        b=cfE2orHZER5I88vhMC9O8M1q2dNk4FCqnjLl66AjUpyrBC9tNgdZiBPlUO8m/YNx+E
         //iQaDmvx5M+oyzG8CYxtCV/8FHiu4/hBBzWFsxLT+haDCwojRvJUDRMHqZP/3f/m0jg
         aNq3ySgqPNeXo/ZHyXsUYa5jXBDhepvd47nnOTJwyscp9xoax93idZjUbu2v2hzN4PpG
         bNW+H3uaHU9mKWJFadd55B8gJg65/EWlaOAP2oLRihMBqNT+Y57gVsThd22k63E7IiBr
         AF4kkIPJe5jHuzdBKQ3aRvjkToFU2gD1boS6aBFF/+Qe7wlupmH6NQK9dKRjBClDErVl
         Yq3w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707261845; x=1707866645;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=d6GrdyKlPKPAQWNWWj73dIKliIETF37U+jzX0SwkFgA=;
        b=lL0/yHzo18nXy81U1XInKOyntsAYfIx13gvsYG2ECOGSgprvRrlRxIUl1sJbH+lv8a
         fbIquNH0DapusT1C8MZ1HFa5Fpz+O+epn/AkVGqOwlVBxkE2Si7v7glmQPc7/mzI4rwT
         jrGwi4W3hbvMFc5ZTpzDrqAIOxEGO8btrHLz61Svq3WYbfG5u+PEUQKzEGZbOMFXD5Vf
         PzDj7eRh2AUR0m9IgdkQftViLia6d1toH/V9B8diHcSO79OfT4EGD1QSR85uIAhA3Vj+
         ssgsWFcPHkTJdO32yETUaflundiokAl7NGA8fCCAUWoKpTzEhv3HAWD8C8hLG6zr2G+J
         liEA==
X-Gm-Message-State: AOJu0Yz0yYUJ895uCIXGiuMb8b7if6O7X6h/eH1m+WdPSvXRXrU0t2VV
	PqFiZYYt+csS7tPvsf+o1ShbglK9uT3R0axoOE/hn7QXDQ7Lo9yiKw6CGSJweFg=
X-Google-Smtp-Source: AGHT+IHeNVevjJhKLYZ1yKwczXrLnghYa+VOCn61kzuIVv7hmeEabAytsZM90X2UbXQD+1eZ+2Dkpw==
X-Received: by 2002:a05:600c:1e17:b0:40e:f9d4:2b03 with SMTP id ay23-20020a05600c1e1700b0040ef9d42b03mr3086733wmb.12.1707261844645;
        Tue, 06 Feb 2024 15:24:04 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWvNQw6yFq4HAIAm+SxKYNcD4i9fE1JlWnLfdZOy6GSqX/mgz/K865dczsrvWxVUpx7XTzkO36eZpelSFZmvOutnhULn7j+1/qJW/IVCgTEXFx/hweO7G9cUzXMbLlw7l3JWjM9p+UJqys3q6s5fNuRKRFzqg6yZZj9
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	=?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH] libxl: Fix comment for LIBXL_HAVE_VMTRACE_BUF_KB
Date: Tue,  6 Feb 2024 23:23:45 +0000
Message-Id: <2bb71b2ba88e6eb6177c27dd65f2af608a634ac2.1707261567.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

It's located in libxl_domain_build_info, not libxl_domain_create_info.
---
 tools/include/libxl.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 907aa0a330..14f69823e0 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -519,7 +519,7 @@
 #define LIBXL_HAVE_PHYSINFO_CAP_VMTRACE 1
 
 /*
- * LIBXL_HAVE_VMTRACE_BUF_KB indicates that libxl_domain_create_info has a
+ * LIBXL_HAVE_VMTRACE_BUF_KB indicates that libxl_domain_build_info has a
  * vmtrace_buf_kb parameter, which allows to enable pre-allocation of
  * processor tracing buffers of given size.
  */
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 00:42:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 00:42:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677179.1053585 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXW12-0006pA-L2; Wed, 07 Feb 2024 00:42:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677179.1053585; Wed, 07 Feb 2024 00:42:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXW12-0006p3-Hi; Wed, 07 Feb 2024 00:42:16 +0000
Received: by outflank-mailman (input) for mailman id 677179;
 Wed, 07 Feb 2024 00:42:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXW11-0006ot-5u; Wed, 07 Feb 2024 00:42:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXW10-0001Fa-Ud; Wed, 07 Feb 2024 00:42:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXW10-0005Qp-BQ; Wed, 07 Feb 2024 00:42:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXW10-0003us-AU; Wed, 07 Feb 2024 00:42:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HL98ydzYLtuu42I4os8cBftLVU+kfRNUKcIACwKszFc=; b=zoVJA0wRrs5dI2nzWPabjVVkgS
	RMqICJmkpm6TiL9rPGd+C6TpGzoa2tHhxUNHUWK97qEGhvSf5wmziNoq23wyHVa6ZuON0gpkclHIQ
	4f8AI4vRj0hhxYnBhPe9SWgid4vCqLD2IHoMiIzRiLNZb5UvUXdOCey2/znWhSCAef0Q=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184604-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184604: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-examine:reboot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=1cb7949b37b3d8892d8a3cea3ad1e2448352a34d
X-Osstest-Versions-That:
    xen=29f17d837421f13c0e0010802de1b2d51d2ded4a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 07 Feb 2024 00:42:14 +0000

flight 184604 xen-unstable real [real]
flight 184610 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184604/
http://logs.test-lab.xenproject.org/osstest/logs/184610/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-examine      8 reboot              fail pass in 184610-retest
 test-armhf-armhf-xl-credit1 18 guest-start/debian.repeat fail pass in 184610-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184597
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184597
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184597
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184597
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184597
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184597
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184597
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184597
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184597
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184597
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184597
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184597
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  1cb7949b37b3d8892d8a3cea3ad1e2448352a34d
baseline version:
 xen                  29f17d837421f13c0e0010802de1b2d51d2ded4a

Last test of basis   184597  2024-02-05 22:09:17 Z    1 days
Testing same since   184604  2024-02-06 10:06:23 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Frediano Ziglio <frediano.ziglio@cloud.com>
  Juergen Gross <jgross@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   29f17d8374..1cb7949b37  1cb7949b37b3d8892d8a3cea3ad1e2448352a34d -> master


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 00:47:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 00:47:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677197.1053639 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXW5n-0007xL-Rv; Wed, 07 Feb 2024 00:47:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677197.1053639; Wed, 07 Feb 2024 00:47:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXW5n-0007xE-Oj; Wed, 07 Feb 2024 00:47:11 +0000
Received: by outflank-mailman (input) for mailman id 677197;
 Wed, 07 Feb 2024 00:47:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aVye=JQ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rXW5m-0007x8-9p
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 00:47:10 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6b693504-c552-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 01:47:08 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 48962CE1774;
 Wed,  7 Feb 2024 00:47:01 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 47E45C433F1;
 Wed,  7 Feb 2024 00:46:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b693504-c552-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707266820;
	bh=/R57MmC0z3d2YbVpLZQ+paJCglOOu9H5nf/rYdJyaBU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=NeCHMnsgGf8Nyf6VUi3Qb01HkToHjYnps+HyP9VJsV3pM3Xyj8jyE1wyx1rJNhLj0
	 VD3Msop8d3W+WiGDXWarpXJO9L09c9BkdJaj/aYvfJhUu2hXp83P15qBmBY+oDaUmU
	 ABz6q3/sZVN9bQ/5wk+eVh/ekJbexaJLzDQItqcjg14a3dzhu+BE1rZodBNhdWFBlJ
	 eSXyLKHDCOVcR/6z3Y2dECYo1TgghAQEM4AWdJrF4MjbErZ68Ds+smBrsxJ49YCSvE
	 UIwWYaUI3bweBNMc4/l5u7kQXboXau3yDgGUsLj6pv31ps1+5+nSQtLkXNg7S6AlYS
	 qJ4sDgtxRD+wQ==
Date: Tue, 6 Feb 2024 16:46:57 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Michal Orzel <michal.orzel@amd.com>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, andrew.cooper3@citrix.com
Subject: Re: [PATCH] automation: Switch yocto-qemux86-64 job to run on x86
In-Reply-To: <20240206152012.51107-1-michal.orzel@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402061646480.1925432@ubuntu-linux-20-04-desktop>
References: <20240206152012.51107-1-michal.orzel@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 6 Feb 2024, Michal Orzel wrote:
> At the moment, all Yocto jobs run on Arm64 runners. To address CI
> capacity issues, move yocto-qemux86-64 job to x86. Reflect the change in
> the makefile generating Yocto docker files and fix CONTAINER name
> definition that incorrectly expects YOCTO_HOST variable to be set for x86
> container as well, which does not have a platform name appended.
> 
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>


Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



> ---
> I built and pushed a new container to registry.
> Pipeline:
> https://gitlab.com/xen-project/people/morzel/xen-orzelmichal/-/pipelines/1166227598
> ---
>  automation/build/yocto/yocto.inc |  6 ++++--
>  automation/gitlab-ci/build.yaml  | 10 +++-------
>  2 files changed, 7 insertions(+), 9 deletions(-)
> 
> diff --git a/automation/build/yocto/yocto.inc b/automation/build/yocto/yocto.inc
> index e4328ec7f378..2f3b1a5b2a34 100644
> --- a/automation/build/yocto/yocto.inc
> +++ b/automation/build/yocto/yocto.inc
> @@ -18,11 +18,13 @@ YOCTO_TARGETS = qemuarm64 qemuarm qemux86-64
>  YOCTO_ARCHS = amd64 arm64v8
>  
>  # Architecture we want to use in gitlab CI (depends on runners arch).
> -CI_ARCH = arm64v8
> +CI_ARCH-qemuarm64 = arm64v8
> +CI_ARCH-qemuarm = arm64v8
> +CI_ARCH-qemux86-64 = amd64
>  
>  define GEN_DOCKER
>  # Make all is generating architecture we use in the CI.
> -ifeq ($(CI_ARCH),$(3))
> +ifeq ($(CI_ARCH-$(2)),$(3))
>  CONTAINERS += yocto/$(1)-$(2)$(4)
>  else
>  CONTAINERS_EXTRA += yocto/$(1)-$(2)$(4)
> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> index 32af30ccedc9..6d2cb18b8883 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -213,7 +213,7 @@
>      - ./automation/build/yocto/build-yocto.sh -v --log-dir=./logs --xen-dir=`pwd` ${YOCTO_BOARD} ${YOCTO_OUTPUT}
>    variables:
>      YOCTO_VERSION: kirkstone
> -    CONTAINER: yocto:${YOCTO_VERSION}-${YOCTO_BOARD}-${YOCTO_HOST}
> +    CONTAINER: yocto:${YOCTO_VERSION}-${YOCTO_BOARD}${YOCTO_HOST}
>    artifacts:
>      paths:
>        - 'logs/*'
> @@ -224,16 +224,12 @@
>  .yocto-test-arm64:
>    extends: .yocto-test
>    variables:
> -    YOCTO_HOST: arm64v8
> +    YOCTO_HOST: -arm64v8
>    tags:
>      - arm64
>  
> -# This is not used by any test job as we only run Yocto on arm based machines.
> -# Keep it here so that someone having x86 hardware can easily add jobs.
>  .yocto-test-x86-64:
>    extends: .yocto-test
> -  variables:
> -    YOCTO_HOST: amd64
>    tags:
>      - x86_64
>  
> @@ -576,7 +572,7 @@ yocto-qemuarm:
>      YOCTO_OUTPUT: --copy-output
>  
>  yocto-qemux86-64:
> -  extends: .yocto-test-arm64
> +  extends: .yocto-test-x86-64
>    variables:
>      YOCTO_BOARD: qemux86-64
>  
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 00:52:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 00:52:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677203.1053649 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWAZ-00018H-Dj; Wed, 07 Feb 2024 00:52:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677203.1053649; Wed, 07 Feb 2024 00:52:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWAZ-00018A-A9; Wed, 07 Feb 2024 00:52:07 +0000
Received: by outflank-mailman (input) for mailman id 677203;
 Wed, 07 Feb 2024 00:52:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aVye=JQ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rXWAY-00017Z-CZ
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 00:52:06 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1b0379f4-c553-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 01:52:04 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 38683CE17D4;
 Wed,  7 Feb 2024 00:52:01 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 823B8C433A6;
 Wed,  7 Feb 2024 00:51:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b0379f4-c553-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707267120;
	bh=OZ9lrDX/2MuUzHRp0v6dDGpVXgFTkxmaBQKsLaph3Ec=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Q4PRnr4UI+j0s/ZiXSFGiRimA4AbjvomnSgT06IU78zVLjRrEwOz810nScYUI+UJT
	 1LvMX56WJ1FKqUB2cOIorxILICktIuC97/BYnyqA0GpirhkftSIb7XE0oj5+aazjnB
	 wWW+CVB6XDN1DNajTkMJGWvlgANs0mfIi6sSuakX10CInewM3J2hr8h7swQR15/LMt
	 g5Cx9br4ZJqpnroEuy2xEZ8T4zTtEkyp5wOK27XpTQ8nss4eG4A02bjBA3wXxPFezT
	 AU60rBkx6Me+mNc1wpxleiMmuaXoTiQUrw8tuXk68gIMoVt+UbvX0fC6Rho64OM6Ym
	 KAFLg3nyh65BQ==
Date: Tue, 6 Feb 2024 16:51:58 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Simone Ballarin <simone.ballarin@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>, 
    Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, Kevin Tian <kevin.tian@intel.com>
Subject: Re: [PATCH v2 1/2] x86/IOMMU: address violations of MISRA C:2012
 Rule 14.4
In-Reply-To: <746a33fff1386b2e76657b5f7cfb31f3b117a1fe.1702310368.git.maria.celeste.cesario@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402061651140.1925432@ubuntu-linux-20-04-desktop>
References: <cover.1702310368.git.maria.celeste.cesario@bugseng.com> <746a33fff1386b2e76657b5f7cfb31f3b117a1fe.1702310368.git.maria.celeste.cesario@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 13 Dec 2023, Simone Ballarin wrote:
> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
> 
> The xen sources contain violations of MISRA C:2012 Rule 14.4 whose
> headline states:
> "The controlling expression of an if statement and the controlling
> expression of an iteration-statement shall have essentially Boolean type".
> 
> Add comparisons to avoid using enum constants as controlling expressions
> to comply with Rule 14.4.
> No functional change.
> 
> Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

However it needs an ack from an x86 maintainer


> ---
> Changes in v2
> - rename prefix from AMD/IOMMU to x86/IOMMU
> - move changes on msi.c and hpet.c in this patch.
> ---
>  xen/arch/x86/hpet.c                      | 6 +++---
>  xen/arch/x86/msi.c                       | 4 ++--
>  xen/drivers/passthrough/amd/iommu_init.c | 4 ++--
>  xen/drivers/passthrough/vtd/iommu.c      | 4 ++--
>  xen/drivers/passthrough/vtd/quirks.c     | 2 +-
>  5 files changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
> index 7be26c6a9b..d1ddc8ddf6 100644
> --- a/xen/arch/x86/hpet.c
> +++ b/xen/arch/x86/hpet.c
> @@ -279,7 +279,7 @@ static int hpet_msi_write(struct hpet_event_channel *ch, struct msi_msg *msg)
>  {
>      ch->msi.msg = *msg;
>  
> -    if ( iommu_intremap )
> +    if ( iommu_intremap != iommu_intremap_off )
>      {
>          int rc = iommu_update_ire_from_msi(&ch->msi, msg);
>  
> @@ -353,7 +353,7 @@ static int __init hpet_setup_msi_irq(struct hpet_event_channel *ch)
>      u32 cfg = hpet_read32(HPET_Tn_CFG(ch->idx));
>      irq_desc_t *desc = irq_to_desc(ch->msi.irq);
>  
> -    if ( iommu_intremap )
> +    if ( iommu_intremap != iommu_intremap_off )
>      {
>          ch->msi.hpet_id = hpet_blockid;
>          ret = iommu_setup_hpet_msi(&ch->msi);
> @@ -372,7 +372,7 @@ static int __init hpet_setup_msi_irq(struct hpet_event_channel *ch)
>          ret = __hpet_setup_msi_irq(desc);
>      if ( ret < 0 )
>      {
> -        if ( iommu_intremap )
> +        if ( iommu_intremap != iommu_intremap_off )
>              iommu_update_ire_from_msi(&ch->msi, NULL);
>          return ret;
>      }
> diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
> index 7f8e794254..72dce2e4ab 100644
> --- a/xen/arch/x86/msi.c
> +++ b/xen/arch/x86/msi.c
> @@ -189,7 +189,7 @@ static int write_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
>  {
>      entry->msg = *msg;
>  
> -    if ( iommu_intremap )
> +    if ( iommu_intremap != iommu_intremap_off )
>      {
>          int rc;
>  
> @@ -555,7 +555,7 @@ int msi_free_irq(struct msi_desc *entry)
>              destroy_irq(entry[nr].irq);
>  
>          /* Free the unused IRTE if intr remap enabled */
> -        if ( iommu_intremap )
> +        if ( iommu_intremap != iommu_intremap_off )
>              iommu_update_ire_from_msi(entry + nr, NULL);
>      }
>  
> diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
> index 5515cb70fd..e02a09a9a7 100644
> --- a/xen/drivers/passthrough/amd/iommu_init.c
> +++ b/xen/drivers/passthrough/amd/iommu_init.c
> @@ -1480,7 +1480,7 @@ int __init amd_iommu_init(bool xt)
>              goto error_out;
>      }
>  
> -    if ( iommu_intremap )
> +    if ( iommu_intremap != iommu_intremap_off )
>          register_keyhandler('V', &amd_iommu_dump_intremap_tables,
>                              "dump IOMMU intremap tables", 0);
>  
> @@ -1498,7 +1498,7 @@ int __init amd_iommu_init_late(void)
>  
>      /* Further initialize the device table(s). */
>      pci_init = true;
> -    if ( iommu_intremap )
> +    if ( iommu_intremap != iommu_intremap_off )
>          rc = iterate_ivrs_mappings(amd_iommu_setup_device_table);
>  
>      for_each_amd_iommu ( iommu )
> diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
> index e13b7d99db..bd6d69a6f5 100644
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -2543,7 +2543,7 @@ static int __must_check init_vtd_hw(bool resume)
>      /*
>       * Enable interrupt remapping
>       */  
> -    if ( iommu_intremap )
> +    if ( iommu_intremap != iommu_intremap_off )
>      {
>          int apic;
>          for ( apic = 0; apic < nr_ioapics; apic++ )
> @@ -2559,7 +2559,7 @@ static int __must_check init_vtd_hw(bool resume)
>              }
>          }
>      }
> -    if ( iommu_intremap )
> +    if ( iommu_intremap != iommu_intremap_off )
>      {
>          for_each_drhd_unit ( drhd )
>          {
> diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c
> index 5a56565ea8..950dcd56ef 100644
> --- a/xen/drivers/passthrough/vtd/quirks.c
> +++ b/xen/drivers/passthrough/vtd/quirks.c
> @@ -392,7 +392,7 @@ void __init platform_quirks_init(void)
>      map_igd_reg();
>  
>      /* Tylersburg interrupt remap quirk */
> -    if ( iommu_intremap )
> +    if ( iommu_intremap != iommu_intremap_off )
>          tylersburg_intremap_quirk();
>  }
>  
> -- 
> 2.40.0
> 
> 


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 00:53:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 00:53:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677205.1053659 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWBR-0001d1-MR; Wed, 07 Feb 2024 00:53:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677205.1053659; Wed, 07 Feb 2024 00:53:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWBR-0001cq-JH; Wed, 07 Feb 2024 00:53:01 +0000
Received: by outflank-mailman (input) for mailman id 677205;
 Wed, 07 Feb 2024 00:53:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aVye=JQ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rXWBQ-0001cg-LC
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 00:53:00 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3ba4dcba-c553-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 01:52:58 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 46360CE17A3;
 Wed,  7 Feb 2024 00:52:52 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id E9964C433F1;
 Wed,  7 Feb 2024 00:52:49 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3ba4dcba-c553-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707267171;
	bh=xAfUPySfxt0Pz2hB+12JwdNp3voxUh0xmb4J46MURxw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=OeilyJfle8VsXGQwFkgi5vpEhENounSHbJ2XNqZqQuwhP+Pz76MTkT885zL4je7iG
	 w8/AmpbvWrOkJ2ZQVZ2gxwHo6nIuMuTTlnS5wvA4GgChA+6mR9S4CoJWyENWxtMFBQ
	 0FleC91qGsEilcU6FzscPyNKPVMopkLQKVZLZioyJC5UpYsSzfN61Aj5YJ73DrHJDo
	 OfBak4vrPUkAzDd7hl+ADClN/89IQTVfN38LwazWWnWCaYJBZOQh9Iv+SQPjs2C4bx
	 K26rInZHZb7Ak6hYFmNNt7oqiyqynrjtYqIa6px78Yz93twnk4bxEZvFBiOCniK0tZ
	 p7VA6UoNRG1Gw==
Date: Tue, 6 Feb 2024 16:52:48 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Nicola Vetrini <nicola.vetrini@bugseng.com>, 
    xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, Kevin Tian <kevin.tian@intel.com>, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH v2 1/2] x86/IOMMU: address violations of MISRA C:2012
 Rule 14.4
In-Reply-To: <2a21b871-8762-43ee-b9f3-bf34869d008f@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402061652170.1925432@ubuntu-linux-20-04-desktop>
References: <cover.1702310368.git.maria.celeste.cesario@bugseng.com> <746a33fff1386b2e76657b5f7cfb31f3b117a1fe.1702310368.git.maria.celeste.cesario@bugseng.com> <891a1ab0b772910999f49878ae679634@bugseng.com> <2a21b871-8762-43ee-b9f3-bf34869d008f@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 5 Feb 2024, Jan Beulich wrote:
> On 05.02.2024 16:36, Nicola Vetrini wrote:
> > On 2023-12-13 17:10, Simone Ballarin wrote:
> >> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
> >>
> >> The xen sources contain violations of MISRA C:2012 Rule 14.4 whose
> >> headline states:
> >> "The controlling expression of an if statement and the controlling
> >> expression of an iteration-statement shall have essentially Boolean 
> >> type".
> >>
> >> Add comparisons to avoid using enum constants as controlling 
> >> expressions
> >> to comply with Rule 14.4.
> >> No functional change.
> >>
> >> Signed-off-by: Maria Celeste Cesario  
> >> <maria.celeste.cesario@bugseng.com>
> >> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
> >> ---
> >> Changes in v2
> >> - rename prefix from AMD/IOMMU to x86/IOMMU
> >> - move changes on msi.c and hpet.c in this patch.
> >> ---
> >>  xen/arch/x86/hpet.c                      | 6 +++---
> >>  xen/arch/x86/msi.c                       | 4 ++--
> >>  xen/drivers/passthrough/amd/iommu_init.c | 4 ++--
> >>  xen/drivers/passthrough/vtd/iommu.c      | 4 ++--
> >>  xen/drivers/passthrough/vtd/quirks.c     | 2 +-
> >>  5 files changed, 10 insertions(+), 10 deletions(-)
> >>
> > 
> > +Stefano
> > 
> > Hi all,
> > 
> > this patch seems not to have been committed into staging, unlike the 
> > other patch from this series. Since these are the only remaining 
> > violations for Rule 14.4, then I think these changes could be reviewed.
> 
> It's no surprise the change isn't committed yet, when it hasn't had any
> of the necessary tags. As far as I'm concerned, I seem to recall
> indicating clearly that I'm not happy with this change, and hence acks
> would need to come from elsewhere.

Thanks Jan for not blocking it. I gave my reviewed-by, let's see if one
of the other x86 maintainers is OK to ack it.


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 00:56:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 00:56:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677211.1053668 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWF9-0002GJ-4P; Wed, 07 Feb 2024 00:56:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677211.1053668; Wed, 07 Feb 2024 00:56:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWF9-0002GC-1u; Wed, 07 Feb 2024 00:56:51 +0000
Received: by outflank-mailman (input) for mailman id 677211;
 Wed, 07 Feb 2024 00:56:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aVye=JQ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rXWF7-0002G4-47
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 00:56:49 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c3885c07-c553-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 01:56:46 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 30040CE0AD1;
 Wed,  7 Feb 2024 00:56:44 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 40CA5C433F1;
 Wed,  7 Feb 2024 00:56:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3885c07-c553-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707267403;
	bh=2qY4drNQqGRmpUWpSieyaKhSzukR+nwgPbenc7u9fnc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=G21ArrC73I926WzFlER8xrNjumJtFLJznI6lz8G4XlaRTxp69biEoQj8s91T37E4A
	 nG31w9+TSEtrxiT7jVX93iFeZsQuKHvvorH3s85K4qolnkOiE55HczRsp42/0BG0ju
	 oDOkM1CIQGWx1lXP5KDJjDuP1+QaedMxSpUBPm2i7fTSTXLqTl9RzZu1Ja0YFBiqqR
	 Xo/eqO3o91XTqt7NJS59bE1EckDe4LT3rf7tDp/lUxvvf7ugp+y6kXKLfa9rG5qg0K
	 p3iRoPgweja2c+I52vi4sfvOBpFoEX5XHQbaoXrOPuPPCpaZLdlmV10KeHHTi000yy
	 tTHIpvVHUaFpw==
Date: Tue, 6 Feb 2024 16:56:40 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Simone Ballarin <simone.ballarin@bugseng.com>, consulting@bugseng.com, 
    sstabellini@kernel.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH v4 3/4] xen/x86: address violations of MISRA C:2012
 Rule 13.1
In-Reply-To: <b23ffbf2-e1b1-42f5-b0ea-9f0889a5a7af@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402061656080.1925432@ubuntu-linux-20-04-desktop>
References: <cover.1706886631.git.simone.ballarin@bugseng.com> <16bb514ac0a5fe0d6e9a2c95279a8200ff4495c6.1706886631.git.simone.ballarin@bugseng.com> <b23ffbf2-e1b1-42f5-b0ea-9f0889a5a7af@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 6 Feb 2024, Jan Beulich wrote:
> On 02.02.2024 16:16, Simone Ballarin wrote:
> > Rule 13.1: Initializer lists shall not contain persistent side effects
> > 
> > This patch moves expressions with side-effects into new variables before
> > the initializer lists.
> > 
> > No functional changes.
> > 
> > Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
> 
> To be honest, I don't like this. It's more code for no gain. Really its
> hampering clarity imo. I'm willing to be convinced otherwise, but for
> now this gets a nack from me.

Which part of the patch you don't like? The changes to
bad_ioapic_register?



> As an aside, ...
> 
> > --- a/xen/arch/x86/io_apic.c
> > +++ b/xen/arch/x86/io_apic.c
> > @@ -2559,9 +2559,12 @@ integer_param("max_gsi_irqs", max_gsi_irqs);
> >  
> >  static __init bool bad_ioapic_register(unsigned int idx)
> >  {
> > -    union IO_APIC_reg_00 reg_00 = { .raw = io_apic_read(idx, 0) };
> > -    union IO_APIC_reg_01 reg_01 = { .raw = io_apic_read(idx, 1) };
> > -    union IO_APIC_reg_02 reg_02 = { .raw = io_apic_read(idx, 2) };
> > +    uint32_t reg_00_raw = io_apic_read(idx, 0);
> > +    uint32_t reg_01_raw = io_apic_read(idx, 1);
> > +    uint32_t reg_02_raw = io_apic_read(idx, 2);
> 
> ... while you properly use uint32_t here, ...
> 
> > +    union IO_APIC_reg_00 reg_00 = { .raw = reg_00_raw };
> > +    union IO_APIC_reg_01 reg_01 = { .raw = reg_01_raw };
> > +    union IO_APIC_reg_02 reg_02 = { .raw = reg_02_raw };
> >  
> >      if ( reg_00.raw == -1 && reg_01.raw == -1 && reg_02.raw == -1 )
> >      {
> > --- a/xen/arch/x86/mpparse.c
> > +++ b/xen/arch/x86/mpparse.c
> > @@ -798,11 +798,12 @@ void __init mp_register_lapic_address (
> >  
> >  int mp_register_lapic(u32 id, bool enabled, bool hotplug)
> >  {
> > +	u32 apic = apic_read(APIC_LVR);
> 
> ... why the being-phased-out u32 here?
> 
> Jan
> 
> >  	struct mpc_config_processor processor = {
> >  		.mpc_type = MP_PROCESSOR,
> >  		/* Note: We don't fill in fields not consumed anywhere. */
> >  		.mpc_apicid = id,
> > -		.mpc_apicver = GET_APIC_VERSION(apic_read(APIC_LVR)),
> > +		.mpc_apicver = GET_APIC_VERSION(apic),
> >  		.mpc_cpuflag = (enabled ? CPU_ENABLED : 0) |
> >  			       (id == boot_cpu_physical_apicid ?
> >  				CPU_BOOTPROCESSOR : 0),
> > --- a/xen/arch/x86/setup.c
> > +++ b/xen/arch/x86/setup.c
> > @@ -885,13 +885,14 @@ static struct domain *__init create_dom0(const module_t *image,
> >  {
> >      static char __initdata cmdline[MAX_GUEST_CMDLINE];
> >  
> > +    unsigned int max_vcpus = dom0_max_vcpus();
> >      struct xen_domctl_createdomain dom0_cfg = {
> >          .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
> >          .max_evtchn_port = -1,
> >          .max_grant_frames = -1,
> >          .max_maptrack_frames = -1,
> >          .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version),
> > -        .max_vcpus = dom0_max_vcpus(),
> > +        .max_vcpus = max_vcpus,
> >          .arch = {
> >              .misc_flags = opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0,
> >          },
> 


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 00:59:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 00:59:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677214.1053679 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWHJ-0003MH-GW; Wed, 07 Feb 2024 00:59:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677214.1053679; Wed, 07 Feb 2024 00:59:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWHJ-0003MA-Di; Wed, 07 Feb 2024 00:59:05 +0000
Received: by outflank-mailman (input) for mailman id 677214;
 Wed, 07 Feb 2024 00:59:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aVye=JQ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rXWHI-0003M2-L0
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 00:59:04 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 13b09431-c554-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 01:59:01 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 027FFCE17CF;
 Wed,  7 Feb 2024 00:58:56 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9458BC433C7;
 Wed,  7 Feb 2024 00:58:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13b09431-c554-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707267535;
	bh=b+aQSryH0cgUoxvEaW9f1UBUJHVdWVNf1/UTc/09FM8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=mQMu2/Il8mmuT+Z0PffMNBwQa6kzHSQLoH7fdMX/NQkxQCW+ffPcb8aLr4sDSjEZd
	 ZzIBs49RjMmL41xZbU0vRPS2mtNE6iC8lwOc6tCMZP7HyP/zraVkX0KIAyHdaEjPPc
	 ePDrjLw1W/5GBMP/dPnsm83SyG40khxs3M4C62KMAEV3V3ZMvPL6+czoMRoH8+SWbB
	 wknIdne2aOGNY1h8jjIDYbqCv2z9FB6bR1JIAS3rBMi1EoyTFGbe932Emnc7h8C0ew
	 haUFcmeSvWf71A/V3YUij87goR2c0CxkjH9B52IeaSrECYzK8PAiymcCVKj9f+vV9x
	 z+UBEcNiTBaPA==
Date: Tue, 6 Feb 2024 16:58:52 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Simone Ballarin <simone.ballarin@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    sstabellini@kernel.org, 
    Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH v4 4/4] eclair: move function and macro properties
 outside ECLAIR
In-Reply-To: <387b160ae93c221c4bc2426605b96b432b26224e.1706886631.git.simone.ballarin@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402061658400.1925432@ubuntu-linux-20-04-desktop>
References: <cover.1706886631.git.simone.ballarin@bugseng.com> <387b160ae93c221c4bc2426605b96b432b26224e.1706886631.git.simone.ballarin@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 2 Feb 2024, Simone Ballarin wrote:
> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
> 
> Function and macro properties contained in ECLAIR/call_properties.ecl are of
> general interest: this patch moves these annotations in a generaric JSON file
> in docs. In this way, they can be exploited for other purposes (i.e. documentation,
> other tools).
> 
> Add rst file containing explanation on how to update function_macro_properties.json.
> Add script to convert the JSON file in ECL configurations.
> Remove ECLAIR/call_properties.ecl: the file is now automatically generated from
> the JSON file.
> 
> Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>

This is much better, thank you!

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

> ---
> Changes in v4:
> - add missing script for converting the JSON file in ECL configurations;
> - improve commit message;
> - remove call_properties.ecs.
> ---
>  .../eclair_analysis/ECLAIR/analysis.ecl       |   1 +
>  .../ECLAIR/call_properties.ecl                | 128 ---
>  automation/eclair_analysis/prepare.sh         |   2 +
>  automation/eclair_analysis/propertyparser.py  |  37 +
>  docs/function_macro_properties.json           | 841 ++++++++++++++++++
>  docs/function_macro_properties.rst            |  58 ++
>  6 files changed, 939 insertions(+), 128 deletions(-)
>  delete mode 100644 automation/eclair_analysis/ECLAIR/call_properties.ecl
>  create mode 100644 automation/eclair_analysis/propertyparser.py
>  create mode 100644 docs/function_macro_properties.json
>  create mode 100644 docs/function_macro_properties.rst
> 
> diff --git a/automation/eclair_analysis/ECLAIR/analysis.ecl b/automation/eclair_analysis/ECLAIR/analysis.ecl
> index a604582da3..684c5b0b39 100644
> --- a/automation/eclair_analysis/ECLAIR/analysis.ecl
> +++ b/automation/eclair_analysis/ECLAIR/analysis.ecl
> @@ -30,6 +30,7 @@ if(not(scheduled_analysis),
>  -eval_file=deviations.ecl
>  -eval_file=call_properties.ecl
>  -eval_file=tagging.ecl
> +-eval_file=properties.ecl
>  -eval_file=concat(set,".ecl")
>  
>  -doc="Hide reports in external code."
> diff --git a/automation/eclair_analysis/ECLAIR/call_properties.ecl b/automation/eclair_analysis/ECLAIR/call_properties.ecl
> deleted file mode 100644
> index c2b2a6182e..0000000000
> --- a/automation/eclair_analysis/ECLAIR/call_properties.ecl
> +++ /dev/null
> @@ -1,128 +0,0 @@
> -
> --call_properties+={"name(printk)", {"pointee_write(1..=never)", "taken()"}}
> --call_properties+={"name(debugtrace_printk)", {"pointee_write(1..=never)", "taken()"}}
> --call_properties+={"name(panic)", {"pointee_write(1..=never)", "taken()"}}
> --call_properties+={"macro(^domain_crash$)", {"pointee_write(2..=never)", "taken()"}}
> --call_properties+={"macro(^(g?d|mm_)?printk$)", {"pointee_write(2..=never)", "taken()"}}
> --call_properties+={"macro(^guest_bug_on_failed$)", {"pointee_write(1=never)", "taken()"}}
> --call_properties+={"macro(^spin_lock_init_prof$)", {"pointee_write(2=never)", "taken()"}}
> --call_properties+={"macro(^sched_test_func$)", {"pointee_write(1..=never)", "taken()"}}
> --call_properties+={"macro(^dev_(info|warn)$)", {"pointee_write(1..=never)", "taken()"}}
> --call_properties+={"macro(^PAGING_DEBUG$)", {"pointee_write(1..=never)", "taken()"}}
> --call_properties+={"macro(^ACPI_(WARNING|ERROR|INFO)$)", {"pointee_write(1..=never)", "taken()"}}
> --call_properties+={"name(fdt_get_property_by_offset_)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(read_atomic_size)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(device_tree_get_reg)", {"pointee_write(4..=always)", "pointee_read(4..=never)", "taken()"}}
> --call_properties+={"name(dt_get_range)", {"pointee_write(3..=always)", "pointee_read(3..=never)", "taken()"}}
> --call_properties+={"name(parse_static_mem_prop)", {"pointee_write(2..=always)", "pointee_read(2..=never)", "taken()"}}
> --call_properties+={"name(get_ttbr_and_gran_64bit)", {"pointee_write(1..2=always)", "pointee_read(1..2=never)", "taken()"}}
> --call_properties+={"name(hvm_emulate_init_once)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(__vmread)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(hvm_pci_decode_addr)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(vpci_mmcfg_decode_addr)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(x86emul_decode)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(unmap_grant_ref)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(arm_smmu_cmdq_build_cmd)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(pci_size_mem_bar)", {"pointee_write(4=always)", "pointee_read(4=never)", "taken()"}}
> --call_properties+={"name(_hvm_read_entry)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(hvm_map_guest_frame_rw)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(guest_cpuid)", {"pointee_write(4=always)", "pointee_read(4=never)", "taken()"}}
> --call_properties+={"name(epte_get_entry_emt)", {"pointee_write(5=always)", "pointee_read(5=never)", "taken()"}}
> --call_properties+={"name(mcheck_mca_logout)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(set_field_in_reg_u32)", {"pointee_write(5=always)", "pointee_read(5=never)", "taken()"}}
> --call_properties+={"name(alloc_affinity_masks)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(xasprintf)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(find_non_smt)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(call_rcu)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(getdomaininfo)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"^MAPPING_(INSERT|SEARCH)\\(.*$", {"pointee_write(2..=always)", "pointee_read(2..=never)", "taken()"}}
> --call_properties+={"name(FormatDec)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(FormatHex)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(p2m_get_ioreq_server)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(elf_memset_unchecked)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(set_iommu_pte_present)", {"pointee_write(7=always)", "pointee_read(7=never)", "taken()"}}
> --call_properties+={"name(clear_iommu_pte_present)", {"pointee_write(4=always)", "pointee_read(4=never)", "taken()"}}
> --call_properties+={"name(vcpu_runstate_get)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(va_start)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(sgi_target_init)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(get_hw_residencies)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(x86_cpu_policy_to_featureset)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"^simple_strtou?ll?\\(.*$", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(msi_compose_msg)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(print_tainted)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(get_hvm_registers)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(x86_insn_modrm)", {"pointee_write(2..3=always)", "pointee_read(2..3=never)", "taken()"}}
> --call_properties+={"name(cpuid_count_leaf)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(rcu_lock_remote_domain_by_id)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(cpuid_count)", {"pointee_write(3..=always)", "pointee_read(3..=never)", "taken()"}}
> --call_properties+={"name(efi_boot_mem_unused)", {"pointee_write(1..=always)", "pointee_read(1..=never)", "taken()"}}
> --call_properties+={"name(collect_time_info)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(setup_xstate_comp)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"name(map_domain_gfn)", {"pointee_read(5=never)", "taken()"}}
> --call_properties+={"name(fdt_getprop)", {"pointee_read(4=never)", "taken()"}}
> --call_properties+={"name(fdt_get_name)", {"pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(fdt_get_property)", {"pointee_read(4=never)", "taken()"}}
> --call_properties+={"name(pci_get_host_bridge_segment)", {"pointee_read(2=never)", "taken()"}}
> --call_properties+={"name(dt_get_property)", {"pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(dt_property_read_u32)", {"pointee_read(3=never)", "taken()"}}
> --call_properties+={"name(dt_device_get_paddr)", {"pointee_read(3..4=never)", "taken()"}}
> --call_properties+={"name(get_evtchn_dt_property)", {"pointee_write(2..3=maybe)", "pointee_read(2..3=never)", "taken()"}}
> --call_properties+={"name(setup_chosen_node)", {"pointee_write(2..3=maybe)", "pointee_read(2..3=never)", "taken()"}}
> --call_properties+={"name(queue_remove_raw)", {"pointee_read(2=never)", "taken()"}}
> --call_properties+={"macro(^memset$)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"macro(^va_start$)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"macro(^memcmp$)", {"pointee_write(1..2=never)", "taken()"}}
> --call_properties+={"macro(^memcpy$)", {"pointee_write(1=always&&2..=never)", "pointee_read(1=never&&2..=always)", "taken()"}}
> --call_properties+={"name(get_cpu_info)",{pure}}
> --call_properties+={"name(pdx_to_pfn)",{pure}}
> --call_properties+={"name(is_pci_passthrough_enabled)",{const}}
> --call_properties+={"name(get_cycles)", {"noeffect"}}
> --call_properties+={"name(msi_gflags)",{const}}
> --call_properties+={"name(hvm_save_size)",{pure}}
> --call_properties+={"name(cpu_has)",{pure}}
> --call_properties+={"name(boot_cpu_has)",{pure}}
> --call_properties+={"name(get_cpu_info)",{pure}}
> --call_properties+={"name(put_pte_flags)",{const}}
> --call_properties+={"name(is_pv_vcpu)",{pure}}
> -
> --doc_begin="Property inferred as a consequence of the semantics of device_tree_get_reg"
> --call_properties+={"name(acquire_static_memory_bank)", {"pointee_write(4..=always)", "pointee_read(4..=never)", "taken()"}}
> --doc_end
> -
> --doc_begin="Property inferred as a consequence of the semantics of dt_set_cell"
> --call_properties+={"name(set_interrupt)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --doc_end
> -
> --doc_begin="Property inferred as a consequence of the semantics of __p2m_get_mem_access"
> --call_properties+={"name(p2m_get_mem_access)", {"pointee_write(3=always)", "pointee_read(3=never)", "taken()"}}
> --doc_end
> -
> --doc_begin="This function has alternative definitions with props {write=always,read=never} and {write=never,read=never}"
> --call_properties+={"name(alloc_cpumask_var)", {"pointee_write(1=maybe)", "pointee_read(1=never)", "taken()"}}
> --doc_end
> -
> --doc_begin="Property inferred as a consequence of the semantics of alloc_cpumask_var"
> --call_properties+={"name(xenctl_bitmap_to_cpumask)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --doc_end
> -
> --doc_begin="The call to bitmap_and causes the pointee of dstp to be always written"
> --call_properties+={"^cpumask_(and|andnot|clear|copy|complement).*$", {"pointee_write(1=always)", "pointee_read(1=never)" "taken()"}}
> --call_properties+={"^bitmap_(andnot|complement|fill).*$", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --doc_end
> -
> --doc_begin="The .*copy_(to|from).* helpers all have a memcpy-like expectation that the destination is a copy of the source.
> -Furthermore, their uses do initialize the involved variables as needed by futher uses in the caller."
> --call_properties+={"macro(^(__)?(raw_)?copy_from_(paddr|guest|compat)(_offset)?$)", {"pointee_write(1=always)", "pointee_read(1=never)", "taken()"}}
> --call_properties+={"macro(^(__)?copy_to_(guest|compat)(_offset)?$)", {"pointee_write(2=always)", "pointee_read(2=never)", "taken()"}}
> --doc_end
> -
> --doc_begin="Functions generated by build_atomic_read cannot be considered pure
> -since the input pointer is volatile, but they do not produce any persistent side
> -effect."
> --call_properties+={"^read_u(8|16|32|64|int)_atomic.*$", {noeffect}}
> --doc_end
> -
> --doc_begin="Functions generated by TYPE_SAFE are const."
> --call_properties+={"^(mfn|gfn|pfn)_x\\(.*$",{const}}
> --call_properties+={"^_(mfn|gfn|pfn)\\(.*$",{const}}
> --doc_end
> diff --git a/automation/eclair_analysis/prepare.sh b/automation/eclair_analysis/prepare.sh
> index fe9d16e48e..47b2a2f32a 100755
> --- a/automation/eclair_analysis/prepare.sh
> +++ b/automation/eclair_analysis/prepare.sh
> @@ -43,4 +43,6 @@ fi
>      make -f "${script_dir}/Makefile.prepare" prepare
>      # Translate the /* SAF-n-safe */ comments into ECLAIR CBTs
>      scripts/xen-analysis.py --run-eclair --no-build --no-clean
> +    # Translate function-properties.json into ECLAIR properties
> +    python3 ${script_dir}/propertyparser.py
>  )
> diff --git a/automation/eclair_analysis/propertyparser.py b/automation/eclair_analysis/propertyparser.py
> new file mode 100644
> index 0000000000..0d02f505a6
> --- /dev/null
> +++ b/automation/eclair_analysis/propertyparser.py
> @@ -0,0 +1,37 @@
> +import json
> +import os
> +
> +script_dir = os.path.dirname(__file__)
> +properties_path = os.path.join(script_dir, "../../docs/function_macro_properties.json")
> +output_path   = os.path.join(script_dir, "ECLAIR/call_properties.ecl")
> +
> +with open(properties_path) as fp:
> +    properties = json.load(fp)['content']
> +
> +ecl = open(output_path, 'w')
> +
> +for record in properties:
> +
> +    string = "-call_properties+={\""
> +    if record['type'] == "function":
> +        string += f"{record['value']}\", {{".replace("\\", "\\\\")
> +    else:
> +        string += f"{record['type']}({record['value']})\", {{".replace("\\", "\\\\")
> +
> +    i=0
> +    for prop in record['properties'].items():
> +        if prop[0] == 'attribute':
> +            string += prop[1]
> +            i+=1
> +        else:
> +            string += f"\"{prop[0]}({prop[1]})\""
> +            i+=1
> +
> +        if i<len(record['properties']):
> +            string += ", "
> +        else:
> +            string +="}}\n"
> +
> +    ecl.write(string)
> +
> +ecl.close()
> diff --git a/docs/function_macro_properties.json b/docs/function_macro_properties.json
> new file mode 100644
> index 0000000000..74058297b5
> --- /dev/null
> +++ b/docs/function_macro_properties.json
> @@ -0,0 +1,841 @@
> +{
> +   "version": "1.0",
> +   "content": [
> +      {
> +         "type": "function",
> +         "value": "^printk.*$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^debugtrace_printk.*$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^panic.*$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^domain_crash$",
> +         "properties":{
> +            "pointee_write": "2..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^(g?d|mm_)?printk$",
> +         "properties":{
> +            "pointee_write": "2..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^guest_bug_on_failed$",
> +         "properties":{
> +            "pointee_write": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^spin_lock_init_prof$",
> +         "properties":{
> +            "pointee_write": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^sched_test_func$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^dev_(info|warn)$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^PAGING_DEBUG$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^ACPI_(WARNING|ERROR|INFO)$",
> +         "properties":{
> +            "pointee_write": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^fdt_get_property_by_offset_.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^read_atomic_size.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^device_tree_get_reg.*$",
> +         "properties":{
> +            "pointee_write": "4..=always",
> +            "pointee_read": "4..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^dt_get_range.*$",
> +         "properties":{
> +            "pointee_write": "3..=always",
> +            "pointee_read": "3..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^parse_static_mem_prop.*$",
> +         "properties":{
> +            "pointee_write": "2..=always",
> +            "pointee_read": "2..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_ttbr_and_gran_64bit.*$",
> +         "properties":{
> +            "pointee_write": "1..2=always",
> +            "pointee_read": "1..2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^hvm_emulate_init_once.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^__vmread.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^hvm_pci_decode_addr.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^vpci_mmcfg_decode_addr.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^x86emul_decode.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^unmap_grant_ref.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^arm_smmu_cmdq_build_cmd.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^pci_size_mem_bar.*$",
> +         "properties":{
> +            "pointee_write": "4=always",
> +            "pointee_read": "4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^_hvm_read_entry.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^hvm_map_guest_frame_rw.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^guest_cpuid.*$",
> +         "properties":{
> +            "pointee_write": "4=always",
> +            "pointee_read": "4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^epte_get_entry_emt.*$",
> +         "properties":{
> +            "pointee_write": "5=always",
> +            "pointee_read": "5=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^mcheck_mca_logout.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^set_field_in_reg_u32.*$",
> +         "properties":{
> +            "pointee_write": "5=always",
> +            "pointee_read": "5=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^alloc_affinity_masks.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^xasprintf.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^find_non_smt.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^call_rcu.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^getdomaininfo.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^MAPPING_(INSERT|SEARCH)\\(.*$",
> +         "properties":{
> +            "pointee_write": "2..=always",
> +            "pointee_read": "2..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^FormatDec.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^FormatHex.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^p2m_get_ioreq_server.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^elf_memset_unchecked.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^set_iommu_pte_present.*$",
> +         "properties":{
> +            "pointee_write": "7=always",
> +            "pointee_read": "7=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^clear_iommu_pte_present.*$",
> +         "properties":{
> +            "pointee_write": "4=always",
> +            "pointee_read": "4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^vcpu_runstate_get.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^va_start.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^sgi_target_init.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_hw_residencies.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^x86_cpu_policy_to_featureset.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^simple_strtou?ll?\\(.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^msi_compose_msg.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^print_tainted.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_hvm_registers.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^x86_insn_modrm.*$",
> +         "properties":{
> +            "pointee_write": "2..3=always",
> +            "pointee_read": "2..3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^cpuid_count_leaf.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^rcu_lock_remote_domain_by_id.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^cpuid_count.*$",
> +         "properties":{
> +            "pointee_write": "3..=always",
> +            "pointee_read": "3..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^.*efi_boot_mem_unused.*$",
> +         "properties":{
> +            "pointee_write": "1..=always",
> +            "pointee_read": "1..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^collect_time_info.*$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^setup_xstate_comp.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^map_domain_gfn.*$",
> +         "properties":{
> +            "pointee_read": "5=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^fdt_getprop.*$",
> +         "properties":{
> +            "pointee_read": "4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^fdt_get_name.*$",
> +         "properties":{
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^fdt_get_property.*$",
> +         "properties":{
> +            "pointee_read": "4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^pci_get_host_bridge_segment.*$",
> +         "properties":{
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^dt_get_property.*$",
> +         "properties":{
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^dt_property_read_u32.*$",
> +         "properties":{
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^dt_device_get_paddr.*$",
> +         "properties":{
> +            "pointee_read": "3..4=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_evtchn_dt_property.*$",
> +         "properties":{
> +            "pointee_write": "2..3=maybe",
> +            "pointee_read": "2..3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^setup_chosen_node.*$",
> +         "properties":{
> +            "pointee_write": "2..3=maybe",
> +            "pointee_read": "2..3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^queue_remove_raw.*$",
> +         "properties":{
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^memset$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^va_start$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^memcmp$",
> +         "properties":{
> +            "pointee_write": "1..2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "macro",
> +         "value": "^memcpy$",
> +         "properties":{
> +            "pointee_write": "1=always&&2..=never",
> +            "pointee_read": "1=never&&2..=always",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_cpu_info.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^pdx_to_pfn.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^is_pci_passthrough_enabled.*$",
> +         "properties":{
> +            "attribute": "const"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_cycles.*$",
> +         "properties":{
> +            "attribute": "noeffect"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^msi_gflags.*$",
> +         "properties":{
> +            "attribute": "const"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^hvm_save_size.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^cpu_has.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^boot_cpu_has.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^get_cpu_info.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^put_pte_flags.*$",
> +         "properties":{
> +            "attribute": "const"
> +         }
> +      },
> +      {
> +         "type": "function",
> +         "value": "^is_pv_cpu.*$",
> +         "properties":{
> +            "attribute": "pure"
> +         }
> +      },
> +      {
> +         "description": "Property inferred as a consequence of the semantics of device_tree_get_reg",
> +         "type": "function",
> +         "value": "^acquire_static_memory_bank.*$",
> +         "properties":{
> +            "pointee_write": "4..=always",
> +            "pointee_read": "4..=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "Property inferred as a consequence of the semantics of dt_set_cell",
> +         "type": "function",
> +         "value": "^set_interrupt.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description":"Property inferred as a consequence of the semantics of __p2m_get_mem_access",
> +         "type": "function",
> +         "value": "^p2m_get_mem_access.*$",
> +         "properties":{
> +            "pointee_write": "3=always",
> +            "pointee_read": "3=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "This function has alternative definitions with props {write=always,read=never} and {write=never,read=never}",
> +         "type": "function",
> +         "value": "^alloc_cpumask_var.*$",
> +         "properties":{
> +            "pointee_write": "1=maybe",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "Property inferred as a consequence of the semantics of alloc_cpumask_var",
> +         "type": "function",
> +         "value": "^xenctl_bitmap_to_cpumask.*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "The call to bitmap_and causes the pointee of dstp to be always written",
> +         "type": "function",
> +         "value": "^cpumask_(and|andnot|clear|copy|complement).*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "The call to bitmap_and causes the pointee of dstp to be always written",
> +         "type": "function",
> +         "value": "^bitmap_(andnot|complement|fill).*$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "The .*copy_(to|from).* helpers all have a memcpy-like expectation that the destination is a copy of the source. Furthermore, their uses do initialize the involved variables as needed by futher uses in the caller.",
> +         "type": "macro",
> +         "value": "^(__)?(raw_)?copy_from_(paddr|guest|compat)(_offset)?$",
> +         "properties":{
> +            "pointee_write": "1=always",
> +            "pointee_read": "1=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description":"The .*copy_(to|from).* helpers all have a memcpy-like expectation that the destination is a copy of the source. Furthermore, their uses do initialize the involved variables as needed by futher uses in the caller.",
> +         "type": "macro",
> +         "value": "^(__)?copy_to_(guest|compat)(_offset)?$",
> +         "properties":{
> +            "pointee_write": "2=always",
> +            "pointee_read": "2=never",
> +            "taken": ""
> +         }
> +      },
> +      {
> +         "description": "Functions generated by build_atomic_read cannot be considered pure since the input pointer is volatile, but they do not produce any persistent side effect.",
> +         "type": "function",
> +         "value": "^read_u(8|16|32|64|int)_atomic.*$",
> +         "properties":{
> +            "attribute": "noeffect"
> +         }
> +      },
> +      {
> +         "description": "Functions generated by TYPE_SAFE are const.",
> +         "type": "function",
> +         "value": "^(mfn|gfn|pfn)_x\\(.*$",
> +         "properties":{
> +            "attribute": "const"
> +         }
> +      },
> +      {
> +         "description": "Functions generated by TYPE_SAFE are const.",
> +         "type": "function",
> +         "value": "^_(mfn|gfn|pfn)\\(.*$",
> +         "properties":{
> +            "attribute": "const"
> +         }
> +      }
> +   ]
> +}
> diff --git a/docs/function_macro_properties.rst b/docs/function_macro_properties.rst
> new file mode 100644
> index 0000000000..ea6fb5cf1f
> --- /dev/null
> +++ b/docs/function_macro_properties.rst
> @@ -0,0 +1,58 @@
> +.. SPDX-License-Identifier: CC-BY-4.0
> +
> +Properties list for Xen
> +=======================
> +
> +Some functions and macros are found to have properties relevant to
> +the Xen codebase. For this reason, the file docs/properties.json
> +contains all the needed properties.
> +
> +Here is an example of the properties.json file::
> +
> +  {
> +     "version": "1.0",
> +     "content": [
> +        {
> +           "description": ""
> +           "type": "function",       // required
> +           "value:": "^printk*.$",   // required
> +           "properties":{
> +              "pointee_write": "1..2=never",
> +              "pointee_read": "",
> +              "taken": ""
> +              "attribute": ""
> +           }
> +        }
> +     ]
> +  }
> +
> +Here is an explanation of the fields inside an object of the "content" array:
> +
> + - description: a brief description of why the properties apply
> + - type: this is the kind of the element called: it may be either ``macro`` or ``function``
> + - value: must be a regex, starting with ^ and ending with $ and matching function fully
> +   qualified name or macro name.
> + - properties: a list of properties applied to said function.
> +   Possible values are:
> +
> +    - pointee_write: indicate the write use for call arguments that correspond to
> +      parameters whose pointee types are non-const
> +    - pointee_read: indicate the read use for call arguments that correspond to
> +      parameters whose pointee types are non-const
> +    - taken: indicates that the specified address arguments may be stored in objects
> +      that persist after the function has ceased to exist (excluding the returned value);
> +      address arguments not listed are never taken
> +    - attribute: attributes a function may have. Possible values are pure, const and noeffect.
> +
> +   pointee_read and pointee_write use a specific kind of argument, structured as pointee_arg=rw:
> +
> +    - pointee_arg: argument index for callee. Index 0 refers to the return value,
> +      the indices of the arguments start from 1. It can be either a single value or a range.
> +    - rw: a value that's either always, maybe or never
> +
> +       - always: for pointee_read: argument pointee is expected to be fully read in the function body,
> +         for pointee_write: argument pointee is fully initialized at function exit
> +       - maybe: for pointee_read: argument pointee may be expected to be read in the function body,
> +         for pointee_write: argument pointee may be written by function body
> +       - never: for pointee_read: argument pointee is not expected to be read in the function body,
> +         for pointee_write: argument pointee is never written by function body
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 01:07:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 01:07:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677224.1053689 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWPG-0006xv-E9; Wed, 07 Feb 2024 01:07:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677224.1053689; Wed, 07 Feb 2024 01:07:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWPG-0006xo-AC; Wed, 07 Feb 2024 01:07:18 +0000
Received: by outflank-mailman (input) for mailman id 677224;
 Wed, 07 Feb 2024 01:07:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aVye=JQ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rXWPF-0006xi-JL
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 01:07:17 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3ae9746f-c555-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 02:07:15 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id CAE9461762;
 Wed,  7 Feb 2024 01:07:12 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7F57C433C7;
 Wed,  7 Feb 2024 01:07:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3ae9746f-c555-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707268032;
	bh=ZNQbpnGpa2RA6YePIehVv3cgCYtIzTCaOTSI1UhwnU0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=kC5QPaTOoohRtw0qMH1ubZeUPtf8UfP0tNBMh5G6NZB2tmAAcNpYm3XoHfzKf9J38
	 Hm2aNi0TULw13BjDWBzGaaptqcMc+OYFGl7va33gybxuikp+KyDZ4jrMc/IggV1s+f
	 yD0p41qOltNrYwvjIzKagR/kjXohCq3coqNPE3gSC0bRFIgaH5JavD6TAxqotfAFt0
	 58d2TXD8Q+NujR/r2ytj94ujIqsGUlNETZ9evnAylKzbUQdAaEOS7E85anxhSqQhwV
	 ElO9R7rfO38pAhN+nLyiyAxDbO2a+85AtxMKsCbDMlMoWGGg1p4jOcfxq7hBx6ovud
	 S7eV4rj4rblAQ==
Date: Tue, 6 Feb 2024 17:07:09 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH v2 3/3] automation/eclair: add deviation for MISRA
 C:2012 Rule 16.3
In-Reply-To: <5a9ac027852dc661faac65de2caee6a7260c204c.1706259490.git.federico.serafini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402061707020.1925432@ubuntu-linux-20-04-desktop>
References: <cover.1706259490.git.federico.serafini@bugseng.com> <5a9ac027852dc661faac65de2caee6a7260c204c.1706259490.git.federico.serafini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 26 Jan 2024, Federico Serafini wrote:
> Update ECLAIR configuration to consider safe switch clauses ending
> with STATIC_ASSERT_UNREACHABLE().
> 
> Update docs/misra/deviations.rst accordingly.
> 
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 01:08:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 01:08:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677227.1053698 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWQc-0007iU-NX; Wed, 07 Feb 2024 01:08:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677227.1053698; Wed, 07 Feb 2024 01:08:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWQc-0007iN-K9; Wed, 07 Feb 2024 01:08:42 +0000
Received: by outflank-mailman (input) for mailman id 677227;
 Wed, 07 Feb 2024 01:08:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aVye=JQ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rXWQb-0007iH-4r
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 01:08:41 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6cc76b9c-c555-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 02:08:40 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id D061DCE17D5;
 Wed,  7 Feb 2024 01:08:33 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 082C6C433F1;
 Wed,  7 Feb 2024 01:08:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6cc76b9c-c555-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707268113;
	bh=RE4r+UEgZmgIeb1PNxcNr1r80Mxzi92RwFhUl438SN8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=mLzCTJYqUQgCxHYXkopkHghIM6XuR9AWPoG6ZfhKwNefxIBCL56GKEMHi4X5PeKzM
	 uEb1+//55TbjC8kra08hp8rhON05DtdmbzBp69lWeozB/v5kl/4ax2L3LK2ygBD4VR
	 KE57wHy5XsyAizkkpthv5WyrOJ60wvclF0Gb5/ClGC0ANYShMpNpkZijXEmrkjs7rY
	 gz6kn1crU5ZYxSRaF57Ynniw76/33J1iIu/pVGW3K9NWPaEqZ85y2+IeXD6XtC3+iJ
	 Iq4TZ9kybSj8QhVSDyuQvL0pe7qF+xAc38dichZmaS7ifjLeN/YepKpq6PN6AK+rSh
	 8ceort5VJyDIw==
Date: Tue, 6 Feb 2024 17:08:27 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Federico Serafini <federico.serafini@bugseng.com>, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
In-Reply-To: <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
References: <cover.1706259490.git.federico.serafini@bugseng.com> <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com> <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 6 Feb 2024, Jan Beulich wrote:
> On 26.01.2024 11:05, Federico Serafini wrote:
> > @@ -208,7 +205,7 @@ do {                                                                       \
> >      case 8:                                                                \
> >          put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
> >          break;                                                             \
> > -    default: __put_user_bad();                                             \
> > +    default: STATIC_ASSERT_UNREACHABLE();                                  \
> >      }                                                                      \
> >      clac();                                                                \
> >  } while ( false )
> > @@ -227,7 +224,7 @@ do {                                                                       \
> >      case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
> >      case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
> >      case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
> > -    default: __get_user_bad();                                             \
> > +    default: STATIC_ASSERT_UNREACHABLE();                                  \
> >      }                                                                      \
> >      clac();                                                                \
> >  } while ( false )
> 
> Related to my remark on patch 1 - how is one to know the macro this was
> invoked from, when seeing the resulting diagnostic?

I am not sure what do you mean here... we do get an error like the
following (I added a STATIC_ASSERT_UNREACHABLE for case 4):

./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 01:09:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 01:09:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677229.1053709 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWRC-0008Nz-VC; Wed, 07 Feb 2024 01:09:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677229.1053709; Wed, 07 Feb 2024 01:09:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWRC-0008Ns-SL; Wed, 07 Feb 2024 01:09:18 +0000
Received: by outflank-mailman (input) for mailman id 677229;
 Wed, 07 Feb 2024 01:09:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aVye=JQ=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rXWRB-0008Iy-Qv
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 01:09:17 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 82410d69-c555-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 02:09:16 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id AE724CE1779;
 Wed,  7 Feb 2024 01:09:13 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id AD044C433F1;
 Wed,  7 Feb 2024 01:09:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82410d69-c555-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707268153;
	bh=33P6WiDsA5f6DH606xSZnLBVDtzbCPwlPelARish9Nc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=njVSLBMGJmRWnrq3eDpfXOSkaVyYakDzELLkpLRh/culTX3YQlNe+fRiDvOggh4Fb
	 aT3px/GBg/LyxO/xgXLKY1WM5AR1xxmrnTJ5kSY13CzpiIr+YU/FHccgMEayMW2bfq
	 MoU+YDMY0cpa6DalqMMNb5+D7vBv9NzIyNNs4jEvXkHyRf5XqXXaffJmRAFpKNfY4O
	 ujaVqtZswdbiF5mojaQNPQbBAv7wG8RiTggfgTQVPyj/KBdKuKcX0BrblA/dbOXFuB
	 19ymnHd+jKZyb99x6C4Vlo1Q9q1OtqGlvfqDVYmIX/zn/33jKX+HRT0s9c0QrdHKcX
	 fXiBpUt1HDiew==
Date: Tue, 6 Feb 2024 17:09:10 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Federico Serafini <federico.serafini@bugseng.com>, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE()
In-Reply-To: <0e673e08-6fa8-4e1e-bfde-a2764b121d61@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402061708480.1925432@ubuntu-linux-20-04-desktop>
References: <cover.1706259490.git.federico.serafini@bugseng.com> <42fc6ae8d3eb802429d29c774502ff232340dc84.1706259490.git.federico.serafini@bugseng.com> <2c0d22be-0b31-4e62-a8f8-2dc82147e51a@suse.com> <0e673e08-6fa8-4e1e-bfde-a2764b121d61@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 6 Feb 2024, Jan Beulich wrote:
> On 06.02.2024 14:22, Jan Beulich wrote:
> > On 26.01.2024 11:05, Federico Serafini wrote:> --- a/xen/include/xen/compiler.h
> >> +++ b/xen/include/xen/compiler.h
> >> @@ -64,6 +64,13 @@
> >>  # define fallthrough        do {} while (0)  /* fallthrough */
> >>  #endif
> >>  
> >> +/*
> >> + * Add the following macro to check that a program point is considered
> >> + * unreachable by the static analysis performed by the compiler.
> >> + */
> >> +#define STATIC_ASSERT_UNREACHABLE() \
> >> +    asm ( ".error \"static assertion failed: unreachable\"" )
> > 
> > In the comment s/Add/Use/? The macro is there after all when this gets
> > committed. Overall maybe
> > 
> > "Use this macro at program points considered unreachable, to be checked
> >  by the compiler's static analysis."
> > 
> > ?
> > 
> > Also while asm()s without operands are implicitly volatile, I think it
> > would be a good idea to make that explicit nevertheless.
> > 
> > I'd be happy to adjust while committing, so long as you agree. If you
> > agree, and provided diagnostics resulting from this are useful (an
> > example would have been nice in the description):
> > Acked-by: Jan Beulich <jbeulich@suse.com>
> 
> Actually, having seen patch 2, I need to withdraw this right away.

To me it looks good enough but let's continue the discussion on patch
#2


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 01:18:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 01:18:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677239.1053719 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWa4-0002CV-PF; Wed, 07 Feb 2024 01:18:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677239.1053719; Wed, 07 Feb 2024 01:18:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXWa4-0002CO-MF; Wed, 07 Feb 2024 01:18:28 +0000
Received: by outflank-mailman (input) for mailman id 677239;
 Wed, 07 Feb 2024 01:18:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KALA=JQ=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXWa2-0002AG-RJ
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 01:18:26 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cade4004-c556-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 02:18:25 +0100 (CET)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-5112d5ab492so154962e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 17:18:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cade4004-c556-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707268705; x=1707873505; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=4pWn6m14NirrhfvdYf2GroV+X8QxzsnMmSoc1mjeExk=;
        b=YOZarVWhZ4mYe+DKWMnDpVFX9z5VfaRt+TOR+fEinb4vbciDOkx9ySrzLVCVQlEBZb
         hRfM7CWQX2XYVPek8Vz9CvXtxiqFQ/pABIMbDQYcNiCpvw19NOre1Ote417B9ckL8zcK
         ARoZ7ysZNScBvZLx7MpSVeCxPCsqgjrDo/9x0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707268705; x=1707873505;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=4pWn6m14NirrhfvdYf2GroV+X8QxzsnMmSoc1mjeExk=;
        b=r44gLaPKPOEZ7ie4nMaSm7slvpinUOaEtuH1+91SlDmKyOuh8FX3/ooFhgLNt+hOQh
         Vhv8i8EQSMZQKMiqvPrpc6TGrHwSPvdWCkkdTIgFFlOef2l222/c+3+gPPgk83ZbbsND
         WcdZXLS63GjiFCdaanDDxlOPKqq8lwLyucmhNbsBRO9lAfgqSbceWwiAuef95d6Ufe4m
         qCWcw8sKunLT9lsK9JxbGN06TC3xwzfQpNPAljQQLJkDQHXYy1MU5qblqFG8CWNnM2HW
         WCbGfaNEVEItGcKOHlAmnpc827f26C1UfkHzvFMzqTCuAq9Odbv2y61KhHbfLV8/sezf
         Nv0A==
X-Gm-Message-State: AOJu0YxV9Xd01JZj34YPr4YF17Em/4mA8W0XmS7uawJfrfxWnEKGUbhM
	r2lXZLZM64UtA4dX8IU9Y1tcl7sBTK1PUVwqz5EhCbbyIskw7EjlFWSsOhudQYLqdfUu/hK9LWp
	gzTSdbdOBSK7aiDc7XYisYnjBdfNq7FYqaPHvpA==
X-Google-Smtp-Source: AGHT+IENkt4ZFMs7CIcftLyruEXS+bafgv70h5gLDVhB99Nl4jfCxuEcftE3Tq+OJU1JG+13ks27XdNo+PR3eKVCQac=
X-Received: by 2002:ac2:4c0b:0:b0:511:6546:7154 with SMTP id
 t11-20020ac24c0b000000b0051165467154mr349715lfq.26.1707268705151; Tue, 06 Feb
 2024 17:18:25 -0800 (PST)
MIME-Version: 1.0
References: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
In-Reply-To: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 7 Feb 2024 01:18:13 +0000
Message-ID: <CA+zSX=beYAGb81zqD19YNAV3cXPFTk-7V68XYn59Eb3mGFH_Jg@mail.gmail.com>
Subject: Re: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor ap2m->default_access
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: xen-devel@lists.xenproject.org, Tamas K Lengyel <tamas@tklengyel.com>, 
	Alexandru Isaila <aisaila@bitdefender.com>, Petre Pircalabu <ppircalabu@bitdefender.com>, 
	Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 6, 2024 at 6:08=E2=80=AFPM Petr Bene=C5=A1 <w1benny@gmail.com> =
wrote:
>
> From: Petr Bene=C5=A1 <w1benny@gmail.com>
>
> This patch addresses a behavior discrepancy in the handling of altp2m vie=
ws,
> where upon the creation and subsequent EPT violation, the page access
> permissions were incorrectly inherited from the hostp2m instead of respec=
ting
> the altp2m default_access.
>
> Previously, when a new altp2m view was established with restrictive
> default_access permissions and activated via xc_altp2m_switch_to_view(),
> it failed to trigger an event on the first access violation.  This behavi=
or
> diverged from the intended mechanism, where the altp2m's default_access
> should dictate the initial permissions, ensuring proper event triggering =
on
> access violations.
>
> The correction involves modifying the handling mechanism to respect the
> altp2m view's default_access upon its activation, eliminating the need fo=
r
> setting memory access permissions for the entire altp2m range (e.g. withi=
n
> xen-access.c).  This change not only aligns the behavior with the expecte=
d
> access control logic but also results in a significant performance improv=
ement
> by reducing the overhead associated with setting memory access permission=
s
> across the altp2m range.
>
> Signed-off-by: Petr Bene=C5=A1 <w1benny@gmail.com>

Thanks Petr, this looks like a great change.

Two things:

- Probably worth adjusting the comment at the top of
p2m_altp2m_get_or_propagate to mention that you use the altp2m
default_access when propagating from the host p2m

- This represents a change in behavior, so probably at least worth a
mention in CHANGELOG.md?

Tamas, I guess this is OK from an interface compatibility point of
view?  In theory it should always have been behaving this way.

 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 03:07:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 03:07:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677246.1053729 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXYHd-0000R8-F2; Wed, 07 Feb 2024 03:07:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677246.1053729; Wed, 07 Feb 2024 03:07:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXYHd-0000R1-CA; Wed, 07 Feb 2024 03:07:33 +0000
Received: by outflank-mailman (input) for mailman id 677246;
 Wed, 07 Feb 2024 03:07:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MEJi=JQ=gmail.com=dunlapg@srs-se1.protection.inumbo.net>)
 id 1rXYHb-0000Qr-NH
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 03:07:31 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 05a6f3cd-c566-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 04:07:26 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-40fc22f372cso1058235e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 19:07:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05a6f3cd-c566-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=umich.edu; s=google-2016-06-03; t=1707275246; x=1707880046; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=uYZgRtm/NRg2LmrWyVBtSGliVoI0O6GINp7/LJ89NqM=;
        b=oMBT1JkT7N3mDFKsfhZjhaDJ5Uzafk9JDOX9pTPXVZ+9ct/c5nf94RIufMBKep9cPH
         oVfUU4CuqsIS3Xr7lRzHJQym+LiFk4FBvYIO8MdXm6A5GnjUEGrNUufo71GJ62pIjZ08
         ZlvaGME24GcIi/0tw6C30B2KphEhv/EgX7uoio9clDkKQmFiPQFX3TWGSL0hO48Pnt+n
         L4IYFzYYpW3pmgo8AG0xuBMMDVbVtvx8rVEYMR4M4U7jGeiUkSJ1QQjAZe2eXKfAfHmQ
         miLDgAfK+bd1WrSoYGmMFdtSo9h/J3wDv8ak4bI+chg77ohFOShzp7wzhsIyjk9fAsbX
         cBjQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707275246; x=1707880046;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=uYZgRtm/NRg2LmrWyVBtSGliVoI0O6GINp7/LJ89NqM=;
        b=YUWloTcRwMOskOk00qQ2Jz9Pvt1mJBn+I+eSAYog237SwYFTekqbkF4umIvtrX+c15
         2NaziQSxIRVT0PX+kUpcMEbcr504VcdwhBmzxswPD+3JH9fDNdM5wdNpp8tSWN4hhQM9
         FZOc8k7+RU7sAzlzlYZ17oBRo1Sz43Wk4KCR/PaLPrzL5X7jGr27jLqxfamDnrYQE4qO
         a2M0wK4xkRtKbgEESkvsTen1OlXafHxS31cxHy6W54HrQ+M5y1JuvYhZp9b8xI7TI1SS
         egVeDpB6imOfHkRIycUrnBQLXp2CkC5oxvO/P/7jwCKSyyK5zSFBNevmB37rYmr2rOdF
         pMiQ==
X-Gm-Message-State: AOJu0Yy6GXCJ/hKDxjYVsVf8C7TzdbYD3sRd2OTKj2863Gm7rlsxX/8L
	5nmEmpFW+qgBUgXSNKohMH+z3u2hP1ZwX5G2UFm/glkuMWpaP8HeUGWewZyY9Tqo1TcMm7xP9i7
	MnfsSsubl7pNK3jGYwfPGflgbgSI=
X-Google-Smtp-Source: AGHT+IGapBKOSKXmMlsTTb5PGPfOg/CbuyXGcWyhK2I00fBoRnUuydJDoQIsTgQc2FGNUbhxQyySvpjubq7LoYYTOpE=
X-Received: by 2002:a05:600c:3543:b0:40f:c1fc:e5b6 with SMTP id
 i3-20020a05600c354300b0040fc1fce5b6mr3218161wmq.38.1707275246051; Tue, 06 Feb
 2024 19:07:26 -0800 (PST)
MIME-Version: 1.0
References: <5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com>
 <7cce89f4-962e-bfbe-7d30-18fea7515bed@suse.com> <CAFLBxZZLJMWpf1fCNN4dhoDpYpW6O=V_C==-sCAZy6t4Df_yBw@mail.gmail.com>
 <47b5a1be-280c-4e8e-a5c5-6df7da657539@suse.com>
In-Reply-To: <47b5a1be-280c-4e8e-a5c5-6df7da657539@suse.com>
From: George Dunlap <dunlapg@umich.edu>
Date: Wed, 7 Feb 2024 03:07:14 +0000
Message-ID: <CAFLBxZY4hR6mxL_Zu+AYaNFsg528zmnL45K6tcE=_Lq7s5p=0w@mail.gmail.com>
Subject: Re: [PATCH v2 2/2] x86/p2m: aid the compiler in folding p2m_is_...()
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	George Dunlap <george.dunlap@cloud.com>
Content-Type: multipart/alternative; boundary="0000000000009326a10610c1fec1"

--0000000000009326a10610c1fec1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 1, 2024 at 10:15=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:

> On 01.02.2024 14:32, George Dunlap wrote:
> > On Thu, Jun 23, 2022 at 12:54=E2=80=AFPM Jan Beulich <jbeulich@suse.com=
> wrote:
> >
> >> By using | instead of || or (in the negated form) && chances increase
> >> for the compiler to recognize that both predicates can actually be
> >> folded into an expression requiring just a single branch (via OR-ing
> >> together the respective P2M_*_TYPES constants).
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >>
> >
> > Sorry for the delay.  Git complains that this patch is malformed:
> >
> > error: `git apply --index`: error: corrupt patch at line 28
> >
> > Similar complaint from patchew when it was posted:
> >
> > https://patchew.org/Xen/5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com/
>
> Not sure what to say. The patch surely is well-formed. It applies fine
> using patch (when not taken from email). When taken from email, patch
> mentions that it strips CRs (I'm running my email client on Windows),
> but the saved email still applies fine. "git am" indeed is unhappy
> when taking the plain file as saved from email, albeit here with an
> error different from yours. If I edit the saved email to retain just
> the From: and Subject: tags, all is fine.
>

That still doesn't work for me.


> I can't tell what git doesn't like. The error messages (the one you
> see and the one I got) tell me nothing.


The raw email looks like this:

```
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -379,7 +379,7 @@ struct page_info *p2m_get_page_from_gfn(
             return page;
=3D20
         /* Error path: not a suitable GFN at all */
-        if ( !p2m_is_ram(*t) && !p2m_is_paging(*t) && !p2m_is_pod(*t) &&
+        if ( !(p2m_is_ram(*t) | p2m_is_paging(*t) | p2m_is_pod(*t)) &&
              !mem_sharing_is_fork(p2m->domain) )
             return NULL;
     }
```

Note the "=3D20" at the beginning of the empty line.  Why `patch` handles i=
t
but `git am` doesn't, who knows.


> I'm also not aware of there
> being a requirement that patches I send via email need to be
> "git am"-able (unlike in xsa.git, where I edit patches enough to be
> suitable for that), nor am I aware how I would convince my email
> client and/or server to omit whatever git doesn't like or to add
> whatever git is missing.
>
> Bottom line - your response would be actionable by me only in so far
> as I could switch to using "git send-email". Which I'm afraid I'm not
> going to do unless left with no other choice. The way I've been
> sending patches has worked well for over 20 years, and for different
> projects. (I'm aware Andrew has some special "Jan" command to apply
> patches I send, but I don't know any specifics.)
>

In the general case, I'm not going to review a patch without being able to
see it in context; and it's not reasonable to expect reviewers to have
specific contributor-specific scripts for doing so.  If we run into this
issue in the future, and you want my review, you may have to post a git
tree somewhere, or attach the patch as an attachment or something.  (Or you
can try to figure out why `git am` isn't working and try to upstream a fix.=
)

That said, in this case, context isn't really necessary to understand the
change, so it won't be necessary.

The logic of the change is obviously correct; but it definitely reduces the
readability.  I kind of feel like whether this sort of optimization is
worth the benefits is more a general x86 maintainer policy decision.  Maybe
we can talk about it at the next maintainer's meeting I'll be at?

 -George

--0000000000009326a10610c1fec1
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Thu, Feb 1, 2024 at 10:15=E2=80=AFPM J=
an Beulich &lt;<a href=3D"mailto:jbeulich@suse.com">jbeulich@suse.com</a>&g=
t; wrote:<br></div><div class=3D"gmail_quote"><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,20=
4);padding-left:1ex">On 01.02.2024 14:32, George Dunlap wrote:<br>
&gt; On Thu, Jun 23, 2022 at 12:54=E2=80=AFPM Jan Beulich &lt;<a href=3D"ma=
ilto:jbeulich@suse.com" target=3D"_blank">jbeulich@suse.com</a>&gt; wrote:<=
br>
&gt; <br>
&gt;&gt; By using | instead of || or (in the negated form) &amp;&amp; chanc=
es increase<br>
&gt;&gt; for the compiler to recognize that both predicates can actually be=
<br>
&gt;&gt; folded into an expression requiring just a single branch (via OR-i=
ng<br>
&gt;&gt; together the respective P2M_*_TYPES constants).<br>
&gt;&gt;<br>
&gt;&gt; Signed-off-by: Jan Beulich &lt;<a href=3D"mailto:jbeulich@suse.com=
" target=3D"_blank">jbeulich@suse.com</a>&gt;<br>
&gt;&gt;<br>
&gt; <br>
&gt; Sorry for the delay.=C2=A0 Git complains that this patch is malformed:=
<br>
&gt; <br>
&gt; error: `git apply --index`: error: corrupt patch at line 28<br>
&gt; <br>
&gt; Similar complaint from patchew when it was posted:<br>
&gt; <br>
&gt; <a href=3D"https://patchew.org/Xen/5d6c927e-7d7c-5754-e7eb-65d1e70f622=
2@suse.com/" rel=3D"noreferrer" target=3D"_blank">https://patchew.org/Xen/5=
d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com/</a><br>
<br>
Not sure what to say. The patch surely is well-formed. It applies fine<br>
using patch (when not taken from email). When taken from email, patch<br>
mentions that it strips CRs (I&#39;m running my email client on Windows),<b=
r>
but the saved email still applies fine. &quot;git am&quot; indeed is unhapp=
y<br>
when taking the plain file as saved from email, albeit here with an<br>
error different from yours. If I edit the saved email to retain just<br>
the From: and Subject: tags, all is fine.<br></blockquote><div><br></div><d=
iv>That still doesn&#39;t work for me.</div><div>=C2=A0</div><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid=
 rgb(204,204,204);padding-left:1ex">I can&#39;t tell what git doesn&#39;t l=
ike. The error messages (the one you<br>
see and the one I got) tell me nothing.</blockquote><div><br></div><div>The=
 raw email looks like this:</div><div><br></div><div>```</div><div>--- a/xe=
n/arch/x86/mm/p2m.c<br>+++ b/xen/arch/x86/mm/p2m.c<br>@@ -379,7 +379,7 @@ s=
truct page_info *p2m_get_page_from_gfn(<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0return page;<br>=3D20<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0/* Error path: not a suitable GFN at all */<br>- =C2=A0 =C2=A0 =C2=A0 =
=C2=A0if ( !p2m_is_ram(*t) &amp;&amp; !p2m_is_paging(*t) &amp;&amp; !p2m_is=
_pod(*t) &amp;&amp;<br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0if ( !(p2m_is_ram(*t) |=
 p2m_is_paging(*t) | p2m_is_pod(*t)) &amp;&amp;<br>=C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 !mem_sharing_is_fork(p2m-&gt;domain) )<br>=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return NULL;<br>=C2=A0 =C2=A0 =C2=
=A0}<br></div><div>```</div><div><br></div><div>Note the &quot;=3D20&quot; =
at the beginning of the empty line.=C2=A0 Why `patch` handles it but `git a=
m` doesn&#39;t, who knows.</div><div>=C2=A0</div><blockquote class=3D"gmail=
_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204=
,204);padding-left:1ex"> I&#39;m also not aware of there<br>
being a requirement that patches I send via email need to be<br>
&quot;git am&quot;-able (unlike in xsa.git, where I edit patches enough to =
be<br>
suitable for that), nor am I aware how I would convince my email<br>
client and/or server to omit whatever git doesn&#39;t like or to add<br>
whatever git is missing.<br>
<br>
Bottom line - your response would be actionable by me only in so far<br>
as I could switch to using &quot;git send-email&quot;. Which I&#39;m afraid=
 I&#39;m not<br>
going to do unless left with no other choice. The way I&#39;ve been<br>
sending patches has worked well for over 20 years, and for different<br>
projects. (I&#39;m aware Andrew has some special &quot;Jan&quot; command to=
 apply<br>
patches I send, but I don&#39;t know any specifics.)<br></blockquote><div><=
br></div><div>In the general case, I&#39;m not going to review a patch with=
out being able to see it in context; and it&#39;s not reasonable to expect =
reviewers to have specific contributor-specific scripts for doing so.=C2=A0=
 If we run into this issue in the future, and you want my review, you may h=
ave to post a git tree somewhere, or attach the patch as an attachment or s=
omething.=C2=A0 (Or you can try to figure out why `git am` isn&#39;t workin=
g and try to upstream a fix.)</div><div><br></div><div>That said, in this c=
ase, context isn&#39;t really necessary to understand the change, so it won=
&#39;t be necessary.</div><div><br></div><div>The logic of the change is ob=
viously correct; but it definitely reduces the readability.=C2=A0 I kind of=
 feel like whether this sort of optimization is worth the benefits is more =
a general x86 maintainer policy decision.=C2=A0 Maybe we can talk about it =
at the next maintainer&#39;s meeting I&#39;ll be at?</div><div><br></div><d=
iv>=C2=A0-George</div></div></div>

--0000000000009326a10610c1fec1--


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 03:28:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 03:28:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677252.1053739 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXYbp-0003R2-7H; Wed, 07 Feb 2024 03:28:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677252.1053739; Wed, 07 Feb 2024 03:28:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXYbp-0003Qv-4F; Wed, 07 Feb 2024 03:28:25 +0000
Received: by outflank-mailman (input) for mailman id 677252;
 Wed, 07 Feb 2024 03:28:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXYbn-0003Qj-SI; Wed, 07 Feb 2024 03:28:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXYbn-0006ut-Jv; Wed, 07 Feb 2024 03:28:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXYbn-0002pF-7i; Wed, 07 Feb 2024 03:28:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXYbn-0001vi-77; Wed, 07 Feb 2024 03:28:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=xK6lgEp/B0DmMSwg3VOJpgud/fYt95EMK/i2n1dFxzM=; b=A8UHqyleCmtUgmO3NZGReA2Fmm
	uKFQatbm9/ksFlSZO/cEZV6J+EENyywcvzrMte7fRPNiZ0+HIcipjlXQoxpNFbiwopMosM0MkSon0
	e4DlC2NZNeKYTrOL+a0966gUNNylwX4I57jcan9K9SqTRtcYlyQYWUtT89FgX0GAk5hs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184611-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184611: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=59a69674b533a2d0c0ad5cd7fbcd655295574509
X-Osstest-Versions-That:
    xen=57a923bfd920307c5bfa533c95dba16a13ea0225
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 07 Feb 2024 03:28:23 +0000

flight 184611 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184611/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  59a69674b533a2d0c0ad5cd7fbcd655295574509
baseline version:
 xen                  57a923bfd920307c5bfa533c95dba16a13ea0225

Last test of basis   184605  2024-02-06 11:03:06 Z    0 days
Testing same since   184611  2024-02-07 01:02:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michal Orzel <michal.orzel@amd.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   57a923bfd9..59a69674b5  59a69674b533a2d0c0ad5cd7fbcd655295574509 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 07:20:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 07:20:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677261.1053750 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXcE2-0003l3-IX; Wed, 07 Feb 2024 07:20:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677261.1053750; Wed, 07 Feb 2024 07:20:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXcE2-0003kw-Ce; Wed, 07 Feb 2024 07:20:06 +0000
Received: by outflank-mailman (input) for mailman id 677261;
 Wed, 07 Feb 2024 07:20:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXcE1-0003PF-5b
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 07:20:05 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4f714469-c589-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 08:20:03 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-40fe59b22dbso2077435e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 23:20:03 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bl13-20020adfe24d000000b0033b39cba5e4sm733640wrb.116.2024.02.06.23.20.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 23:20:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4f714469-c589-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707290402; x=1707895202; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dsqIjtdJvYnDqXUdFBSMbduiUGYGKQ1mhaQP04o58Tc=;
        b=XomLSKWkxd1SpHezyzkMeWFwujpUsFMXITAv7YmCqrrYeIZ01jWkqyJ6tCRhHB+dXX
         55MYr0vC+gAp0GpuCRzPryVH6bXmUp8y2iMyBR15AYMkwiF/h/HPvls5YtjlE7Nk3S1f
         dq6C6h3GGEIVAC51A9J54oqAcJaemZsEJjv8IOAriTQiCP+3i8wdu1d95nXHL0JgCse2
         aWGUYW8MGLk/dPpjxxRJ+vtHEKUIiW04pjVMbIPgeRSMxND5YZ/i77YmM1GjR2qXiBNE
         5VsH2Ejtn+MzC1Gv6su2CikfJjm/vLSru91aJfx6GNXXBVC9vtSA/iUcfdzdC/IguOgm
         teng==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707290402; x=1707895202;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dsqIjtdJvYnDqXUdFBSMbduiUGYGKQ1mhaQP04o58Tc=;
        b=V5E1qISHDdg668WwT54FZn6rDNrAh6Hc8y3tmy6UkE7nyMcDDe73KX2YM23pzXFT+f
         2vWKmx+zHJBxXFCLCMWUSbgXhpTLDIUceFzA2HBnrfjJhYeEqKTSPdPwicuDC7WppC/p
         1Bv/uh8VGuUR7eL2iSB/iZHp0JG762ITYHpedK8136CZia6Djl1YVbBFUWInZpM8mP/w
         eqTIUuxNi0mq9NkKZmEKe2vrGtoz1hRVODVgbVqMuh9eF8wU1zfkyRW44PACd69p2G7O
         uf/tYmuB9ON6ud9Unf9CEMCJI8U1LP7sBpzPjBWWIRzEj6z6dtrb1B1Ac34mD5sn5NV3
         lWrA==
X-Gm-Message-State: AOJu0Yx+sEojKCvRG3KM42362Xnqf5nd2WH4yqELzO+fySNljclqFCGH
	KYH/YPtiJKT+hoZL9pdAEEp6Ocw6jZr12sbvMrGfXJrNJQcTX4LjQYJ+Fswkdg==
X-Google-Smtp-Source: AGHT+IFcpg0VM145aEdMVYTIcloClLOv/UjrIDtDOAG91vYxrgMpKXzsHjMR2DJZRCT2NS+S/4HIOQ==
X-Received: by 2002:a5d:5349:0:b0:33b:4d13:30a1 with SMTP id t9-20020a5d5349000000b0033b4d1330a1mr859313wrv.2.1707290402551;
        Tue, 06 Feb 2024 23:20:02 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCV24P5J20ieH0txA7sN6eEEXGwRT3nD/+ghLEH9d022U0+YJ+wTsKhMMugS7+0pvVZvLFmo9si8GH9iUDCyQYI/ErDGjc4uXZYUxYmEd3lNZkBlRWZVzhR6CKpAO4Skpx3ie8cOADWk5mNsKY637l8yvRkQDKdTK8VgVytnSwnUt3CQyYwWkPwQNRsjN2jnjN2iyVh0PMvAP37p5Mc/5ejNT7rFk8bn3nv89Fvp/9ttfQBcRP1RU37yQVADsXC8YfSX33kjkls2E7eK8EAaaWl+k0JKJ56/X9LP
Message-ID: <743801de-1fe2-4575-9987-a9fbf80afdb9@suse.com>
Date: Wed, 7 Feb 2024 08:20:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/2] x86/IOMMU: address violations of MISRA C:2012 Rule
 14.4
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Kevin Tian <kevin.tian@intel.com>,
 Simone Ballarin <simone.ballarin@bugseng.com>
References: <cover.1702310368.git.maria.celeste.cesario@bugseng.com>
 <746a33fff1386b2e76657b5f7cfb31f3b117a1fe.1702310368.git.maria.celeste.cesario@bugseng.com>
 <alpine.DEB.2.22.394.2402061651140.1925432@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402061651140.1925432@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 01:51, Stefano Stabellini wrote:
> On Wed, 13 Dec 2023, Simone Ballarin wrote:
>> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
>>
>> The xen sources contain violations of MISRA C:2012 Rule 14.4 whose
>> headline states:
>> "The controlling expression of an if statement and the controlling
>> expression of an iteration-statement shall have essentially Boolean type".
>>
>> Add comparisons to avoid using enum constants as controlling expressions
>> to comply with Rule 14.4.
>> No functional change.
>>
>> Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
>> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> However it needs an ack from an x86 maintainer

And from respective IOMMU ones.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 07:36:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 07:36:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677266.1053759 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXcTR-0005Ux-QU; Wed, 07 Feb 2024 07:36:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677266.1053759; Wed, 07 Feb 2024 07:36:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXcTR-0005Uq-MI; Wed, 07 Feb 2024 07:36:01 +0000
Received: by outflank-mailman (input) for mailman id 677266;
 Wed, 07 Feb 2024 07:36:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXcTQ-0005Uk-Hi
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 07:36:00 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 89657b1b-c58b-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 08:35:59 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-40fff96d5d7so2436025e9.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 23:35:59 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m3-20020a056000024300b0033b2799815csm775284wrz.86.2024.02.06.23.35.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 23:35:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89657b1b-c58b-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707291359; x=1707896159; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=NJ3YMq/32Qq9ggAY1KGcljB21mQ0ETSPdhbYcdJIuZ4=;
        b=Fz+df0k6/UJGfhTyeqa9sOqbfnjiZUME6h0koUdKTL3vvJjhcwiJ0BVC3e5ikFATnK
         Jbzlx0pdLsxNA7YNuDE/eBqqkCca7oSM4N+zUgqtU3HckoVD4ZrYPaTXjy0ajkakHS3N
         XfMuFvIxDyoXFITBEwq6XT/56DlNbkkqbdiHglFv6kNc82KWHUABkIIunDIfC8sdUN+U
         67f5weBHQhpisaos/Qnie2wer28jHbKDHJTAssVpSFtkE9K+dFyBdbFur0z2dH+qDMQk
         zrXStKhOF88FFwLp+6ShiUbLgV6FXGkFWHG2oikK7gVCPuV1q3Ylt+V3r/jAdwjRCl1M
         HZiw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707291359; x=1707896159;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=NJ3YMq/32Qq9ggAY1KGcljB21mQ0ETSPdhbYcdJIuZ4=;
        b=uc6Y+zYeHUHwqwIGTBlt7wdsvKZYdJ02+JDvBu6nJz4k8J+sa09w+khtFwrE6ye/Q3
         5DnAIRqzGwb3TkYNx00c+icXHmMJ1LPklVXrFH5kheAaLA3cFxsOFn8kqVyhyaYChpLF
         DvkJt+/+pIkGh38MsRHy2aAwRv+w+mFMnvkCiZ1AYPyuoaOzrMM8cKeoQxjoNJmEjG7P
         7HCtEs2S8tscJQSX79Z0jDSpvrLcqtbEVRQIDz5wmSjH/yBkDtaOrrhvVqF3VDRNNplW
         dtp7XRMQNaUSo0fo01vKRPTb3F0ackzpnzH5GPnRGsFPYq2Ef1liKNG2akRhBOfq5Nef
         NPMg==
X-Gm-Message-State: AOJu0Yw5OLvCCur+4K7xv/6Tii0SibVIAhf8qDT4iMGTbWoMVd525MxD
	u1jymUw2HUM5JRAPTXDnB38Ds5fdxSYB/auQ/o9VUdL6wt+thEtwvHEqRpXjgQ==
X-Google-Smtp-Source: AGHT+IHNUlUMaPjfhrnufYqBPn4y3HplSYGb/heWAmCYFHaSeT/5LNc2iBak0KXFhcswzcgkHtmXHQ==
X-Received: by 2002:a5d:5505:0:b0:33b:5036:8111 with SMTP id b5-20020a5d5505000000b0033b50368111mr239435wrv.11.1707291358767;
        Tue, 06 Feb 2024 23:35:58 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXeJ1d1jqul7Y4ZzqxHr6zv4LDiQXOqq5nr8o7r8dIs2Jee8e7BCse2/+5Epg+h6tzq2sY3ImVzGRGuiUQR++yDNEQ6beBc5FLmgd5Pp8T/lU1Sv9ScX57BC2DDX+T7Y6IeuE0lCjmkHyxVd6nHmqFKSgc58QmiTPw34j9xiaa97zp4iKnfVhypt43RSQMRfp3VzbiWvIZFNoBoTUQzCV9zbPN5
Message-ID: <c5de1969-32dd-4e52-b669-c1bfeae28e40@suse.com>
Date: Wed, 7 Feb 2024 08:35:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 3/4] xen/x86: address violations of MISRA C:2012
 Rule 13.1
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Simone Ballarin <simone.ballarin@bugseng.com>, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <16bb514ac0a5fe0d6e9a2c95279a8200ff4495c6.1706886631.git.simone.ballarin@bugseng.com>
 <b23ffbf2-e1b1-42f5-b0ea-9f0889a5a7af@suse.com>
 <alpine.DEB.2.22.394.2402061656080.1925432@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402061656080.1925432@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 01:56, Stefano Stabellini wrote:
> On Tue, 6 Feb 2024, Jan Beulich wrote:
>> On 02.02.2024 16:16, Simone Ballarin wrote:
>>> Rule 13.1: Initializer lists shall not contain persistent side effects
>>>
>>> This patch moves expressions with side-effects into new variables before
>>> the initializer lists.
>>>
>>> No functional changes.
>>>
>>> Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
>>
>> To be honest, I don't like this. It's more code for no gain. Really its
>> hampering clarity imo. I'm willing to be convinced otherwise, but for
>> now this gets a nack from me.
> 
> Which part of the patch you don't like? The changes to
> bad_ioapic_register?

Really all of them. bad_ioapic_register() is merely worst.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 07:38:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 07:38:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677269.1053769 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXcW1-0006CL-62; Wed, 07 Feb 2024 07:38:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677269.1053769; Wed, 07 Feb 2024 07:38:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXcW1-0006CE-2M; Wed, 07 Feb 2024 07:38:41 +0000
Received: by outflank-mailman (input) for mailman id 677269;
 Wed, 07 Feb 2024 07:38:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXcW0-0006C4-2O
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 07:38:40 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e7c25b13-c58b-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 08:38:37 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-4101995472aso196475e9.2
 for <xen-devel@lists.xenproject.org>; Tue, 06 Feb 2024 23:38:37 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 az22-20020a05600c601600b0040e813f1f31sm1142209wmb.25.2024.02.06.23.38.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 06 Feb 2024 23:38:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7c25b13-c58b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707291517; x=1707896317; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rI0/7s/AzuxGq7spPCa1cjqAnALNzOqy2OeH8OfXcuQ=;
        b=REzvvpeh9i0tz5vruaaGNQCHeuqgYHTop7b9TsYvxQEDbXKv55JrCC4IH6APYSryam
         ChlL1fBEnCW5Ux+4uawY895WdXFsLljKF4uQJi4QCOj1hMqNhLTSYXGI7qdfU2HJMS2K
         BpFRVZ0kaq80b1K57znMc5/Pf9sFwrN/WqAH6pIYgpNuaMzJFGUqCTdErh/0eDZV8eqN
         3BDrHTDcNDTix2OiYkNi2rr+GFlYDcN1WMPqGRw9+E9E5wQwSgF4/X/uhPy8SgfW/5cf
         bPXbM+PO2gEI8CxTcrEkDT+3udsX7jzzhBSNf2EAh2ozGv2URpAmblzKd6VIxejoXbp4
         Fu+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707291517; x=1707896317;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rI0/7s/AzuxGq7spPCa1cjqAnALNzOqy2OeH8OfXcuQ=;
        b=nskcOV58rMsoNG+wGLlLZfFSn3dL6pItqABoxBUNBcIF5z2of6utH3gYo0yZl5XYBp
         moeVLNXdZfsH+cavccDtD2GmwxXGKmM6BxyLoPODN4lI542NcCcIWXySdNmNk/8kSYub
         60B09/eEE9I0lAqb4TWx1csFLnICwVZZ67tWrbtL4iy1ZXVPUaA8bxYS6RgpXink5sbV
         c44Hgy6PwIrIfRdAFc+sdBFrKwzeLSUpPrBX481RXFcSnmZEaoqi/g6j5XLgjbdMH8EQ
         ZW2pm6kRx2nvAzS/JtLCSaes/OdUeDPRczIV5T673/JWpC1S83I2/0+rcoSAiT68Il6D
         O1sA==
X-Gm-Message-State: AOJu0Yy08OPjL3IrKUN89z5VwMQ1jNTlewabpd+h4aj7iMn7/rDr95Nd
	Suq6LkGJ/qxLJ5mSP3cFD7F2I/qREBJCLN0nUGbQlKfsm1Zo4v9PDCgAzKyXdg==
X-Google-Smtp-Source: AGHT+IGeKV7xqIRSaCRjH1hq394Re2a3St7OqFCYhEPM5dtgMcevgJjJzl9VvS8TO7Tm4vW+XX6eBw==
X-Received: by 2002:a05:600c:519a:b0:40f:e498:3e15 with SMTP id fa26-20020a05600c519a00b0040fe4983e15mr2713120wmb.3.1707291517068;
        Tue, 06 Feb 2024 23:38:37 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVztV3N/fBNR3HsOyqJuVnMKm6EfhypsmqMhFuhkq4zTTdicVbIC5RamJcfeNxfkBDZRXARMeRLAHJ1lQUrHRvJwzX+UPdKxZARh3vweffLx4e/DQLNqDEwmNPtqFFuoAx/tWftmCijxObHt9jZyMCvkyKpwOAXTyFBkJ4Ip7nP8NBemaUsDVi7jgHj8ncviSMTtty6FNpo33joOoO3MexNk4dH
Message-ID: <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
Date: Wed, 7 Feb 2024 08:38:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Federico Serafini <federico.serafini@bugseng.com>,
 consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 02:08, Stefano Stabellini wrote:
> On Tue, 6 Feb 2024, Jan Beulich wrote:
>> On 26.01.2024 11:05, Federico Serafini wrote:
>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>      case 8:                                                                \
>>>          put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>          break;                                                             \
>>> -    default: __put_user_bad();                                             \
>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>      }                                                                      \
>>>      clac();                                                                \
>>>  } while ( false )
>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>      case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>      case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>      case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>> -    default: __get_user_bad();                                             \
>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>      }                                                                      \
>>>      clac();                                                                \
>>>  } while ( false )
>>
>> Related to my remark on patch 1 - how is one to know the macro this was
>> invoked from, when seeing the resulting diagnostic?
> 
> I am not sure what do you mean here... we do get an error like the
> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
> 
> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable

Right - and how do I know what _user_ of the macro actually triggered
it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
__FUNCTION__ here, for that specific purpose ...

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 07:45:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 07:45:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677276.1053779 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXcci-0007ff-Rr; Wed, 07 Feb 2024 07:45:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677276.1053779; Wed, 07 Feb 2024 07:45:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXcci-0007fY-OS; Wed, 07 Feb 2024 07:45:36 +0000
Received: by outflank-mailman (input) for mailman id 677276;
 Wed, 07 Feb 2024 07:45:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=beHI=JQ=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rXcch-0007fS-Vz
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 07:45:36 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id df54ee6c-c58c-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 08:45:33 +0100 (CET)
Received: from BYAPR05CA0037.namprd05.prod.outlook.com (2603:10b6:a03:74::14)
 by SJ0PR12MB5469.namprd12.prod.outlook.com (2603:10b6:a03:37f::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.16; Wed, 7 Feb
 2024 07:45:27 +0000
Received: from SJ1PEPF00001CDE.namprd05.prod.outlook.com
 (2603:10b6:a03:74:cafe::eb) by BYAPR05CA0037.outlook.office365.com
 (2603:10b6:a03:74::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.23 via Frontend
 Transport; Wed, 7 Feb 2024 07:45:27 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ1PEPF00001CDE.mail.protection.outlook.com (10.167.242.6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Wed, 7 Feb 2024 07:45:27 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Wed, 7 Feb
 2024 01:45:22 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Wed, 7 Feb
 2024 01:45:21 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Wed, 7 Feb 2024 01:45:20 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df54ee6c-c58c-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bJFIgdrdPxi/QE7uGYChnwcI+AnhQXy1XIGgmIsAauHe0gkLMhubGVgTapmXTKXSd9wJZhEIjiX4Cdd6GkLFzv8EBKJfS/We62llfV6VfOwN+9z6tDpCv/glgfdKVBnbcjPZ+E3Ab9JpHIwltLlvfnjMrCbvibzMhXPX7TZp67yVzyMhZYP2xFqjrhnLQBjaKATnZfJsYsxxo9KTIDkQSvtaIPs5FVm0kRyazFGk8MRLqEYoTkGiX6L7elfIYmpXpXo2x4etq86T3TZudhZ1SVa4X+PEXP7TnfK5AR+Fu9YjMObubkJX28fiM2pPtC5mdUPa561mXvqdSo2wvLu0hg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=s0Au5QSoHwuPeCLBh/jovbIb0s+fFUJga4OVHXVkhzY=;
 b=jMLKnV9egpqCJCtie9D/+8leIQEYGfFvihFux5iGc2bsBMCCpGFl4zw0z3+OLnilIYYvC6VB8cGHPZZSodONGY5Eu+anATpk3vdtOSjnhQiy0nK0p3tGrbaK1NPUVmtgjFEUmhJbtnAJyLtMD2rMZjLx1fPSs6PZPqXBBomw3kZ6R6vUdaRi4L1tW60LyI1cfwFE/T7Lhy3aC4vRkIPpxD9RvzAaq2J24hhLla9gkGCFMfWLwuEgF0nkCHtQ/KAjZnv74XZvgmY2rIfX7KX3dmCTv3XEVC8J4iUlkMZFiiEBsAIhoII5BYIi/S1qF+zQNxIRGYETe5rv30cCBmRi2g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=s0Au5QSoHwuPeCLBh/jovbIb0s+fFUJga4OVHXVkhzY=;
 b=ErSgyILYn57nHG/79Uxn+ZnfVd3DlEpza2DVNaWDjIQ+bXDmQLu7RypCTZLxGq0QxIS/gGHAHG9B3/4gNdJ0gxzWH9PNTP2gqDl9MenGCvzLb+FBvUQbCJ2WqGC5Yxh3jejOAiL43zkoN+QNe1JmjJNf26VFZ95hnoYwZJ8L79w=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <8daa2461-092f-48ad-8556-3bba398ae544@amd.com>
Date: Wed, 7 Feb 2024 08:45:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v4 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
To: Julien Grall <julien@xen.org>, Ayan Kumar Halder
	<ayan.kumar.halder@amd.com>, <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <stefano.stabellini@amd.com>,
	<Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>,
	<luca.fancellu@arm.com>
References: <20240131121049.225044-1-ayan.kumar.halder@amd.com>
 <20240131121049.225044-2-ayan.kumar.halder@amd.com>
 <c7841d11-d072-4787-9d76-ee94123605bb@xen.org>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <c7841d11-d072-4787-9d76-ee94123605bb@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CDE:EE_|SJ0PR12MB5469:EE_
X-MS-Office365-Filtering-Correlation-Id: b2342e35-f5ad-4d29-d682-08dc27b0c027
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	w6SYzoZOzT/Jd4ODuWnH/j1OWCz1PK6juT/DwZFk2hFTPuzgSt4o1iSRPLVh59Gk8hfl+thos/a5kZs4CbjCLaWUbl9OFwJECWFgB/KsH5VYUNoITfClKAAeeWwx3vY77YP10zvVsnEfnThb92o6yYFz1viLQTPSg/K0DLrWdrxRHWPQGPGr4dkDAvKNhbMzF+Ng0J4IsH70cF65HFe3ejE0eOxsvqv3EpetElJukAz+O7o9sl8MLzg8cB5QU78TNv4pLrZhWZ0xZa/v2DMD/qDxfOkv19Jx7EuBYIqPXTQJ4icrUZeHTHwYASUO9Ou2SX2P1/Cr6zhoig7+AJpcxY05vC14+9odVGgLBR9j2Rs6rRXuzP8Y5bZgOvfeq6tUnR1RdnIZvtkmt+jQZQV0bvFOuxMb8166OKQOj26GK57Ry+JUOMpItfsTnE7ExGQRDl7QEDYIVQ2YIugVlFrHYHCz8lyRud81VSaYs1f/F7CYvaTLbEJepvAq/h9l8KPKh1AySQA1pVak9O8f1HslrhNyvMIzuTMQEWmthFZlf4eYemODaxOelRzpSJnbCTquBYJcX72uhFOO3TmU4ybd28aUpMSroLEdo/4Grk9uiuolZcLkfwiNu4Iibw+uxgGoIk2PhFmRIdEoR9qWexzyvYQdnidk1hyqRP5hqToobCzTt6RWJCtgSQkcpBFbLPOfTQvY1H0t4WNtDPduW2sOU5pSGehPmTnMyt7h9xPkhdbVrFiwgm7H/6mEaZ7fHeh8MsCKST9gPeDv9Y3UgHGrtFcZ8HsmpCeCEpNXu5GN77JQS875TjN7gOn7dyEP2ZdB295T/Lp27C+Fu8gYtjg8fw==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(136003)(376002)(39860400002)(230922051799003)(1800799012)(64100799003)(82310400011)(186009)(451199024)(40470700004)(36840700001)(46966006)(36860700001)(426003)(336012)(2616005)(26005)(83380400001)(47076005)(478600001)(40460700003)(53546011)(31696002)(40480700001)(966005)(86362001)(70586007)(316002)(70206006)(4326008)(8676002)(54906003)(8936002)(16576012)(110136005)(31686004)(356005)(81166007)(82740400003)(44832011)(36756003)(41300700001)(2906002)(5660300002)(43740500002)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Feb 2024 07:45:27.0081
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b2342e35-f5ad-4d29-d682-08dc27b0c027
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CDE.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB5469

Hi Julien,

On 06/02/2024 19:49, Julien Grall wrote:
> 
> 
> Hi Ayan,
> 
> On 31/01/2024 12:10, Ayan Kumar Halder wrote:
>> There can be situations when the registers cannot be emulated to their full
>> functionality. This can be due to the complexity involved. In such cases, one
>> can emulate those registers as RAZ/WI for example. We call them as partial
>> emulation.
>>
>> Some registers are non-optional and as such there is nothing preventing an OS
>> from accessing them.
>> Instead of injecting undefined exception (thus crashing a guest), one may want
>> to prefer a partial emulation to let the guest running (in some cases accepting
>> the fact that it might result in unwanted behavior).
>>
>> A suitable example of this (as seen in subsequent patches) is emulation of
>> DBGDTRTX_EL0 (on Arm64) and DBGDTRTXINT(on Arm32). These non-optional
>> registers can be emulated as RAZ/WI and they can be enclosed within
>> CONFIG_PARTIAL_EMULATION.
>>
>> Further, "partial-emulation" command line option allows us to
>> enable/disable partial emulation at run time. While CONFIG_PARTIAL_EMULATION
>> enables support for partial emulation at compile time (i.e. adds code for
>> partial emulation), this option may be enabled or disabled by Yocto or other
>> build systems. However if the build system turns this option on, users
>> can use scripts like Imagebuilder to generate uboot-script which will append
>> "partial-emulation=false" to xen command line to turn off the partial
>> emulation. Thus, it helps to avoid rebuilding xen.
>>
>> By default, "CONFIG_PARTIAL_EMULATION=y" and "partial-emulation=false".
>> This is done so that Xen supports partial emulation. However, customers are
>> fully aware when they enable partial emulation. It's important to note that
>> enabling such support might result in unwanted/non-spec compliant behavior.
> 
> Can you remind me why this is built by default? In particular...
This is the result of RFC discussion we had, where both Bertrand and Stefano agreed on having
the Kconfig enabled by default to improve user experience:
Bertrand:
https://lore.kernel.org/xen-devel/C0ADC33B-1966-4D3E-B081-A3AA0C3AE76D@arm.com/
Stefano:
https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2312081514450.1703076@ubuntu-linux-20-04-desktop/

~Michal


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 08:37:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 08:37:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677286.1053789 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXdQg-0006Bk-4k; Wed, 07 Feb 2024 08:37:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677286.1053789; Wed, 07 Feb 2024 08:37:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXdQg-0006Bd-1w; Wed, 07 Feb 2024 08:37:14 +0000
Received: by outflank-mailman (input) for mailman id 677286;
 Wed, 07 Feb 2024 08:37:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gvxy=JQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXdQf-0006BX-C7
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 08:37:13 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 156900de-c594-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 09:37:10 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a34c5ca2537so47623266b.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 00:37:10 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 ss28-20020a170907c01c00b00a36c499c935sm499058ejc.43.2024.02.07.00.37.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 00:37:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 156900de-c594-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707295029; x=1707899829; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Hl/fZj57liKmlI/xyYGVc9W5enFSRet6446WIdVaNQE=;
        b=DXaiLtKjW8AvND75EHh+/8qE9VYmMEWugGwx9VDsj8PvFugkmyY+rZ4FJPRmCK7Y0c
         KwaZ3sTS/f7xhbRv/6kMD97ltZkvMbii//ZhfIkaO7ZKIaAzoHTeKzi7Lrb04pXbBcgR
         RwjkKSTKsRMkHb1QuMNDRMV7I933WgSLNPMlU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707295029; x=1707899829;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Hl/fZj57liKmlI/xyYGVc9W5enFSRet6446WIdVaNQE=;
        b=Wf7Yj2PgMjt6Rk0xH1CzFlsMb9S/OBxXgL9TcOu16wvFGxp45zA5+cnJIxx2+nTJxq
         v18g/CAIKuY/sNR5fWXXr3+v4lpkdcxdACd5OkDMhbg9gkgQ6cEm/dj5YfytdjxAiJgu
         DJWIbpPUjX+9X63Li6SPimR4IALWK6xymxq/+6Q+xu2fnYqsdiUV+Wljd3sUdZlMTPkf
         1mkRbdcpYLvi6KMT7dUlEZ4HITpFO3MgpkDOSbuSvAqNxL5IEKIqvffXKjFeFYepRuwh
         mcR3ooUtlkOeZDp//TbWPJ8LKOUHETIu6uzpno6IC+EtcmKSnJ2X1hMOKugnr4yXvbXP
         qdUw==
X-Gm-Message-State: AOJu0Yz46FoOro/VNS6/st5gCTZnL/Dfn5sd8Vg8BS5ul+R1YZ6HCEYP
	pCxVUTO1PRTnqfCldiDKuEutO2ENAVYGgjUChiC4ClsmN7ngy6gKzuuhpdVY3FM=
X-Google-Smtp-Source: AGHT+IGnBhSGpg2iJs2U1tFtwRCYUxp2cBv8mQ+TDnSzhUgbn/mvYMrZ0M9tjMgJOyyUIvLlC4VUJw==
X-Received: by 2002:a17:906:1287:b0:a36:5e63:7b12 with SMTP id k7-20020a170906128700b00a365e637b12mr3696556ejb.37.1707295029646;
        Wed, 07 Feb 2024 00:37:09 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXfX3t49E3mLqiopEcmxNi1PlE4+1vFhHlz5WoWT+XzPAO6JCn6UAuYu+8eP7qZ3jazl92okLLrEj7nA0W8YUUyZOK5Ltlrv3IZ3mBVGAFc8vj0yNJtrsJUfciIpJA8gjfm/Sy77ar/dg==
Date: Wed, 7 Feb 2024 09:37:08 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH 2/4] iommu/x86: introduce a generic IVMD/RMRR range
 validity helper
Message-ID: <ZcNBNGO_G2zT-l9o@macbook>
References: <20240201170159.66330-1-roger.pau@citrix.com>
 <20240201170159.66330-3-roger.pau@citrix.com>
 <232baaae-5fe0-4a94-8469-d1dac520c1af@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <232baaae-5fe0-4a94-8469-d1dac520c1af@suse.com>

On Tue, Feb 06, 2024 at 12:17:29PM +0100, Jan Beulich wrote:
> On 01.02.2024 18:01, Roger Pau Monne wrote:
> > IVMD and RMRR ranges are functionally equivalent, and as so could use the same
> > validity checker.
> 
> May I suggest s/equivalent/similar/?

Sure.

> > Move the IVMD to x86 common IOMMU code and adjust the function to take a pair
> > of [start, end) mfn parameters.
> 
> [start,end) ranges generally come with the problem of not allowing to
> represent the full address space. While that isn't specifically a problem
> here, seeing that both VT-d and V-i present inclusive ranges, how about
> making the common function match that?

OK, I can adjust the code that way.  I originally did it using non
inclusive because it looked like the code would be clearer.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 08:50:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 08:50:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677291.1053798 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXddB-0000R4-73; Wed, 07 Feb 2024 08:50:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677291.1053798; Wed, 07 Feb 2024 08:50:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXddB-0000Qx-47; Wed, 07 Feb 2024 08:50:09 +0000
Received: by outflank-mailman (input) for mailman id 677291;
 Wed, 07 Feb 2024 08:50:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXddA-0000Qr-Eo
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 08:50:08 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e3eb1aed-c595-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 09:50:06 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-5115fcef9e9so419684e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 00:50:06 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e18-20020adffd12000000b0033b4acb999dsm927760wrr.98.2024.02.07.00.50.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 00:50:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3eb1aed-c595-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707295805; x=1707900605; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nZSIqQ/iatJRbl0FM8Szbv2uOJ95WfDTljckD54FTaU=;
        b=IXg4HN+LzeUWrt/L5qJR0OklFAN1EOTtRml9vuCVxCqncpcodIQYXfSDYcCPnnjlzQ
         H/cbWgU93jjXsc7uamhXzA98uBGvsCJU6Dp44R+Z3AIFhL2q9ZIIQzDq6/NiljMhmEyX
         o7mOZoFk7W68ZEs7Ud+9DA7ksvOUKx2Kf9Xa55tO0rTLFHBmrIDg7hByHI9qBHAd2BDi
         sM1gDgB7XRIMoynXyIrTrbc5x/F+76k4T4/K/8tJ5+KWGpt56vEExSHdwuCZYvxeKvOV
         5zyS37ToGsU5l79mJQe/VpRjJknfkjruCNt2Ommzf90XN6RIJ+cdhuvuEPc7HreMe2do
         SsbA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707295805; x=1707900605;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nZSIqQ/iatJRbl0FM8Szbv2uOJ95WfDTljckD54FTaU=;
        b=KdL0FLWDyoyN6xKPvVZ8k7hvVHNnfogEp36Kf9U6+bT9MGBHvIp4uFveCnLjXf8Yix
         gR75horGTl5RIJ8gy+W4Js85wZsx1RhzT9vzcO+JLD6I4zSJXVTUa5vZPPLHTemr7Nw0
         z9t9a+o+IE+o1N2R00SJP9IbLJrWo3thkw5YFXskL8WUIb/dwNJZmfFTpg4LK3rJ2H1S
         GK2T/ptI5VX6PSGZPzfyQioo5vg2vMjs82ONRxbxvGQaMHtpRkW5Qdg6HxRfj2koQvH+
         2mX5OToG1VjPfQ5hUXENf7sRTcdQR286NHOxc3ihWfKrh48t44Slse44ubc2BeRs8B7Q
         elKw==
X-Gm-Message-State: AOJu0YzHnxB1VBXKBD3DRAMED//h4cNb8ef7zfYfAeHNg90//nd/W+Ds
	enSeTk0YefVu7TWzdA7/8U8ksEfy/EsZ7GbhEe2FDcSV137BKR0jsCU9GTKcKg==
X-Google-Smtp-Source: AGHT+IGOLhNnELKlcVJ5BnGIKomDp/JfGS0UsmV51BVpAH5sfYlHO5f5ZvOTspmgQpHzlvssHuUCZA==
X-Received: by 2002:a05:6512:480a:b0:511:62fb:6664 with SMTP id eo10-20020a056512480a00b0051162fb6664mr1239502lfb.8.1707295805407;
        Wed, 07 Feb 2024 00:50:05 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUW2EGouGksObthuM4TWqNMeSxx0xTxCOstkx1/IbnyntF8KI9+vssIZwhYqKVLPnpuf9TB+dUw3FnaaYNc2rfxgi8SNQBmDQKyCi1T7bKOBSv6Pdof1J6RXjBDEw6HVGXGB38EK1kwujGcIvys58VJU6dq8ouaMGqGWT/8TcBL1t1qW7A=
Message-ID: <0ee0381b-326a-412e-91f9-bd464376b7d1@suse.com>
Date: Wed, 7 Feb 2024 09:50:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/2] x86/p2m: aid the compiler in folding p2m_is_...()
Content-Language: en-US
To: George Dunlap <dunlapg@umich.edu>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@cloud.com>
References: <5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com>
 <7cce89f4-962e-bfbe-7d30-18fea7515bed@suse.com>
 <CAFLBxZZLJMWpf1fCNN4dhoDpYpW6O=V_C==-sCAZy6t4Df_yBw@mail.gmail.com>
 <47b5a1be-280c-4e8e-a5c5-6df7da657539@suse.com>
 <CAFLBxZY4hR6mxL_Zu+AYaNFsg528zmnL45K6tcE=_Lq7s5p=0w@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAFLBxZY4hR6mxL_Zu+AYaNFsg528zmnL45K6tcE=_Lq7s5p=0w@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07.02.2024 04:07, George Dunlap wrote:
> On Thu, Feb 1, 2024 at 10:15 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 01.02.2024 14:32, George Dunlap wrote:
>>> On Thu, Jun 23, 2022 at 12:54 PM Jan Beulich <jbeulich@suse.com> wrote:
>>>
>>>> By using | instead of || or (in the negated form) && chances increase
>>>> for the compiler to recognize that both predicates can actually be
>>>> folded into an expression requiring just a single branch (via OR-ing
>>>> together the respective P2M_*_TYPES constants).
>>>>
>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>
>>>
>>> Sorry for the delay.  Git complains that this patch is malformed:
>>>
>>> error: `git apply --index`: error: corrupt patch at line 28
>>>
>>> Similar complaint from patchew when it was posted:
>>>
>>> https://patchew.org/Xen/5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com/
>>
>> Not sure what to say. The patch surely is well-formed. It applies fine
>> using patch (when not taken from email). When taken from email, patch
>> mentions that it strips CRs (I'm running my email client on Windows),
>> but the saved email still applies fine. "git am" indeed is unhappy
>> when taking the plain file as saved from email, albeit here with an
>> error different from yours. If I edit the saved email to retain just
>> the From: and Subject: tags, all is fine.
>>
> 
> That still doesn't work for me.
> 
> 
>> I can't tell what git doesn't like. The error messages (the one you
>> see and the one I got) tell me nothing.
> 
> 
> The raw email looks like this:
> 
> ```
> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -379,7 +379,7 @@ struct page_info *p2m_get_page_from_gfn(
>              return page;
> =20
>          /* Error path: not a suitable GFN at all */
> -        if ( !p2m_is_ram(*t) && !p2m_is_paging(*t) && !p2m_is_pod(*t) &&
> +        if ( !(p2m_is_ram(*t) | p2m_is_paging(*t) | p2m_is_pod(*t)) &&
>               !mem_sharing_is_fork(p2m->domain) )
>              return NULL;
>      }
> ```
> 
> Note the "=20" at the beginning of the empty line.  Why `patch` handles it
> but `git am` doesn't, who knows.

Hmm. Nothing like that seen when I save that mail. Plus I recall having
an issue with this when applying patches coming from Shawn, where those
=20 got in the way, but only if I pruned the saved email before handing
to "git am".

>> I'm also not aware of there
>> being a requirement that patches I send via email need to be
>> "git am"-able (unlike in xsa.git, where I edit patches enough to be
>> suitable for that), nor am I aware how I would convince my email
>> client and/or server to omit whatever git doesn't like or to add
>> whatever git is missing.
>>
>> Bottom line - your response would be actionable by me only in so far
>> as I could switch to using "git send-email". Which I'm afraid I'm not
>> going to do unless left with no other choice. The way I've been
>> sending patches has worked well for over 20 years, and for different
>> projects. (I'm aware Andrew has some special "Jan" command to apply
>> patches I send, but I don't know any specifics.)
>>
> 
> In the general case, I'm not going to review a patch without being able to
> see it in context; and it's not reasonable to expect reviewers to have
> specific contributor-specific scripts for doing so.  If we run into this
> issue in the future, and you want my review, you may have to post a git
> tree somewhere, or attach the patch as an attachment or something.  (Or you
> can try to figure out why `git am` isn't working and try to upstream a fix.)

Based on my own observation mentioned above, I assume "git am" is capable
of dealing with the =20, provided some specific further encoding
specification is present in the mail. Which I'd then have to assume is
missing from what Thunderbird sends, or the =20 is being introduced
without Thunderbird being involved.

> That said, in this case, context isn't really necessary to understand the
> change, so it won't be necessary.
> 
> The logic of the change is obviously correct; but it definitely reduces the
> readability.  I kind of feel like whether this sort of optimization is
> worth the benefits is more a general x86 maintainer policy decision.  Maybe
> we can talk about it at the next maintainer's meeting I'll be at?

I see no problem doing so.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 09:01:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 09:01:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677296.1053809 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXdo8-0002Eb-7U; Wed, 07 Feb 2024 09:01:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677296.1053809; Wed, 07 Feb 2024 09:01:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXdo8-0002EU-4M; Wed, 07 Feb 2024 09:01:28 +0000
Received: by outflank-mailman (input) for mailman id 677296;
 Wed, 07 Feb 2024 09:01:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gvxy=JQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXdo6-0002EO-Rq
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 09:01:26 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7916f562-c597-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 10:01:25 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a29c4bbb2f4so47303166b.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 01:01:25 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 lj24-20020a170906f9d800b00a38599ba2d1sm511259ejb.118.2024.02.07.01.01.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 01:01:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7916f562-c597-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707296485; x=1707901285; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=hRekonX+MjgbxnCNc9YwjzVl3WE2txkCIt6Y5xOA35Q=;
        b=YBj6lL+gMpMBjAmu5I0t8LWokPBb9Gz25aAyWjRDeHlXE35TGLChXanFx2xo7dkV13
         EEX3OGBiuhVKg1vWItuATR0Yhz96LI2yRB688VU39levfNZOOU8hn/xB74Biji63Fayw
         cyEsM3pxJdn9WygnJqMMzIkdc0ARn9iP979Z8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707296485; x=1707901285;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=hRekonX+MjgbxnCNc9YwjzVl3WE2txkCIt6Y5xOA35Q=;
        b=RTVkpqB/I2fnEIh0yiAO3Qvv+UoN0sAdigEg8hKSYmsQsaF4d7dVfcjOmJ1UOTvnh/
         RfE3eFFtTfA2l+h3diLUZb7pFkD81JbCPKCZDhIVVcS2TuLzhpwIKlGKOtEJEZ7T6qQj
         ZVdfBfAFD5eFlano8rw5XXSeoG698qC8Bq0OM+uupoxF+4sn22fDWqWGeayY0DNvuCbW
         IDHD74WpILcIdkAyXKvQTElGrlkeMdMqPrQdL0mgRJYTf5Kj7HIdWOIVKswjwWKPAfdQ
         lGY7GA0dRu0pD0WrSU6tt0XGShVM4J4jxZ2P5B12u5ECtk0KctE61LN6yiBpxQqSaUip
         jLKw==
X-Gm-Message-State: AOJu0Yzak14fM/DO7FNipKC2HlSw/1P7aE81jToRFZf2bzbhLPFO7TtM
	eJl/K2KMp3hzFqbPjTMzr7xVfYlLoz6sst4NcdpLLAdb5G0adfXreb0/HKOXU5w=
X-Google-Smtp-Source: AGHT+IHB3w25zugsVwLODItPAS1DcwZ2c5AvOeJdV4n/n05x+tJ3ToCBEZapJZ5bi6pTkSaQUXawEA==
X-Received: by 2002:a17:906:1296:b0:a38:537:9598 with SMTP id k22-20020a170906129600b00a3805379598mr3489470ejb.17.1707296485316;
        Wed, 07 Feb 2024 01:01:25 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWivn7J+iIFw6MXZ2wFXOkvCJjCq8G5GkAG14G6eMH2R1L9hO0E2rfMxsOe9U+WcvvwFWyMTwZGkjz1kflQfJ7WujT1UHioB0ubmkJgk4Q=
Date: Wed, 7 Feb 2024 10:01:24 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH 3/4] iommu/vt-d: switch to common RMRR checker
Message-ID: <ZcNG5C9gEzlcWJPw@macbook>
References: <20240201170159.66330-1-roger.pau@citrix.com>
 <20240201170159.66330-4-roger.pau@citrix.com>
 <90f8e183-5e78-49e8-ae2f-9898c1d7686b@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <90f8e183-5e78-49e8-ae2f-9898c1d7686b@suse.com>

On Tue, Feb 06, 2024 at 12:28:07PM +0100, Jan Beulich wrote:
> On 01.02.2024 18:01, Roger Pau Monne wrote:
> > Use the newly introduced generic unity map checker.
> > 
> > Also drop the message recommending the usage of iommu_inclusive_mapping: the
> > ranges would end up being mapped anyway even if some of the checks above
> > failed, regardless of whether iommu_inclusive_mapping is set.
> 
> I'm afraid I don't understand this: When not in an appropriate E820
> region, you now even fail IOMMU initialization. Shouldn't such
> failure only occur when inclusive mappings weren't requested? At
> which point referring to that option is still relevant?

This is now better handled, since the VT-d code will use the same
logic as the AMD-Vi logic and attempt to 'convert' such bogus RMRR
regions so they can be safely used.  iommu_unity_region_ok() signals
the RMRR region is impossible to be used, and hence not even
iommu_inclusive_mapping would help in that case.  Also note that
iommu_inclusive_mapping is only applicable to PV, so the message was
already wrong in the PVH case.

> Further to this failing - in patch 2 shouldn't the respective log
> messages then be XENLOG_ERR, matching the earlier use of
> AMD_IOMMU_ERROR()?

Oh, indeed, I've converted them all to WARN, when the first one is
indeed WARN, but the following two are ERROR.

> > --- a/xen/drivers/passthrough/vtd/dmar.c
> > +++ b/xen/drivers/passthrough/vtd/dmar.c
> > @@ -642,17 +642,9 @@ acpi_parse_one_rmrr(struct acpi_dmar_header *header)
> >             return -EEXIST;
> >         }
> >  
> > -    /* This check is here simply to detect when RMRR values are
> > -     * not properly represented in the system memory map and
> > -     * inform the user
> > -     */
> > -    if ( !e820_all_mapped(base_addr, end_addr + 1, E820_RESERVED) &&
> > -         !e820_all_mapped(base_addr, end_addr + 1, E820_NVS) &&
> > -         !e820_all_mapped(base_addr, end_addr + 1, E820_ACPI) )
> > -        printk(XENLOG_WARNING VTDPREFIX
> > -               " RMRR [%"PRIx64",%"PRIx64"] not in reserved memory;"
> > -               " need \"iommu_inclusive_mapping=1\"?\n",
> > -                base_addr, end_addr);
> > +    if ( !iommu_unity_region_ok(maddr_to_mfn(base_addr),
> > +                                maddr_to_mfn(end_addr + PAGE_SIZE)) )
> > +        return -EIO;
> 
> Hmm, noticing only here, but applicable also to the earlier patch: The
> "RMRR" (and there "IVMD") is lost, which removes some relevant context
> information from the log messages. Can you add a const char* parameter
> to the new helper, please?

I debated myself whether to keep the RMRR/IVMD prefix, but I didn't
think there was a lot of value in it, since whether it's an RMRR or an
IVMD region can be deduced from previous messages.  Anyway will pass
the prefix as a function parameter.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 09:15:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 09:15:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677301.1053819 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXe1B-00040E-Bx; Wed, 07 Feb 2024 09:14:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677301.1053819; Wed, 07 Feb 2024 09:14:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXe1B-000407-9C; Wed, 07 Feb 2024 09:14:57 +0000
Received: by outflank-mailman (input) for mailman id 677301;
 Wed, 07 Feb 2024 09:14:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gvxy=JQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXe1A-000401-0r
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 09:14:56 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5b0c483d-c599-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 10:14:54 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-560a9738081so406375a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 01:14:54 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 ev9-20020a056402540900b0055c9280dc51sm451910edb.14.2024.02.07.01.14.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 01:14:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b0c483d-c599-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707297294; x=1707902094; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=xy7k443k0GctZ/WaQZNw858yCaCIbup+H2QveNxhzYc=;
        b=BXEMDrqhs8XDSkTZAU+P+D0WQYTp3cIjVUDU/djaU7o9OrgKIAKwNuTxiv8y1m96g5
         +nnHkHeKZ9DeRKeodz8ElZojhEgRBVzKJ8XIkbygnzx9DQUNpHZaWCHDzHockO6DvUPT
         oJ+lWnpCJHZMwlIiHIL+cE2UShG17j9eg+P2o=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707297294; x=1707902094;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xy7k443k0GctZ/WaQZNw858yCaCIbup+H2QveNxhzYc=;
        b=u41gPxZeNFBCPwPCx+7L3VOa9dKCryGYQ4NjmUjE2tU4BCO4YpWRzf7/PLOoM/EC5C
         xqzfP7MTdrtcvhTKqHvmqK/tvQzYBIeupTpcSr4dERuKzx9XCMu7wyD+asKM6p+9F3Yz
         jD1IkP82dVj7OSx+LhmQDmMzWwjX+kZNr0TB9S/FB3k0CzpkSl8xHBFbpPz+TDND7RQd
         UdZW60BwzQCdl+yyebVaUUMV5Lk0hcjrF3E0isAnf+YYh1W0Hov4CxFCOJ6gk7VpLFKT
         49P64beI4L4gseP4C3oYl4rzUVnKmBsRvQ82aoDp8blb4O9a3LWX9+s5Rml5o9r1YfJ/
         xe5w==
X-Gm-Message-State: AOJu0YzZ7/8yn6xD96quGfPb2j18fuGgfoTSzh5hOU9LE26G1Mh9khU4
	t/myjbaMlMXLksimiwehxghRsZm8KivBG20Q8fVxbqfRmfX6q9yT/kcY4un5hrS9Y3UGOgPtgxu
	0
X-Google-Smtp-Source: AGHT+IH5CGQ5Do/RJiwekak5uzTI7sIocFtSiqYi4S/IwY4JiDMNDCeyksa3USmNUaiD59mja4/k0Q==
X-Received: by 2002:aa7:d693:0:b0:560:654c:157a with SMTP id d19-20020aa7d693000000b00560654c157amr3294423edr.30.1707297293961;
        Wed, 07 Feb 2024 01:14:53 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUSO2Kh3wydBNEqH5rgVzdGAg6rGkvjTNHpXwhSr3aLYfUI4TxIsXE2a1AhBQew09FzKc8RkCml5s8Sn+xTYIbRoJs0goiBDd4fPgy5Qn4=
Date: Wed, 7 Feb 2024 10:14:52 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH 4/4] iommu/x86: make unity range checking more strict
Message-ID: <ZcNKDKD4G7fpIX80@macbook>
References: <20240201170159.66330-1-roger.pau@citrix.com>
 <20240201170159.66330-5-roger.pau@citrix.com>
 <5db22ee1-40b3-4df8-88b8-446a2e224d22@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5db22ee1-40b3-4df8-88b8-446a2e224d22@suse.com>

On Tue, Feb 06, 2024 at 12:49:08PM +0100, Jan Beulich wrote:
> On 01.02.2024 18:01, Roger Pau Monne wrote:
> > Currently when a unity range overlaps with memory being used as RAM by the
> > hypervisor the result would be that the IOMMU gets disabled.  However that's
> > not enough, as even with the IOMMU disabled the device will still access the
> > affected RAM areas.
> 
> Hmm, no, I think this is going too far. Not the least because it is
> s/will/may/. But also because if we really wanted such behavior, we
> ought to also parse the respective ACPI tables when the "iommu=off".

I guessed so, hence why it's the last patch in the series.  TBH I
think it's very unlikely that such system exist.

> > Note that IVMD or RMRR ranges being placed over RAM is a firmware bug.
> 
> As written this is wrong: They're typically in RAM, just that the E820
> type for that range should not be RAM_TYPE_CONVENTIONAL.

Hm, yes, ÇI should have written 'over a RAM range in the memory map'
or similar.

> > Doing so also allows to simplify the code and use a switch over the reported
> > memory type(s).
> 
> I'm afraid this isn't right either: page_get_ram_type() can set
> multiple bits in its output.

It can indeed.  But if the only bit set is RAM_TYPE_CONVENTIONAL then
the page will be handled as RAM, and that's where Xen would be in
trouble if a device is also using such page as a unity map.

If the page however is RAM_TYPE_CONVENTIONAL | RAM_TYPE_RESERVED then
the RESERVED type will take over the whole page, and it's no longer an
issue to have a unity range covering it.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:03:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:03:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677309.1053829 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXelp-0002Fo-0A; Wed, 07 Feb 2024 10:03:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677309.1053829; Wed, 07 Feb 2024 10:03:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXelo-0002FS-SL; Wed, 07 Feb 2024 10:03:08 +0000
Received: by outflank-mailman (input) for mailman id 677309;
 Wed, 07 Feb 2024 10:03:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nQwS=JQ=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rXelo-0002FI-A7
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:03:08 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 170fdb13-c5a0-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 11:03:07 +0100 (CET)
Received: from [192.168.1.140] (unknown [81.56.166.244])
 by support.bugseng.com (Postfix) with ESMTPSA id EA6CC4EE0738;
 Wed,  7 Feb 2024 11:03:05 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 170fdb13-c5a0-11ee-8a49-1f161083a0e0
Message-ID: <851eb6ec-558b-4a89-a31a-34046730bb55@bugseng.com>
Date: Wed, 7 Feb 2024 11:03:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and
 logging effects
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com>
 <09869431-ed00-4102-93fd-84c697a5c950@suse.com>
From: Simone Ballarin <simone.ballarin@bugseng.com>
Organization: BUGSENG
In-Reply-To: <09869431-ed00-4102-93fd-84c697a5c950@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 06/02/24 13:04, Jan Beulich wrote:
> On 02.02.2024 16:16, Simone Ballarin wrote:
>> Rule 13.1: Initializer lists shall not contain persistent side effects
>>
>> Effects caused by debug/logging macros and functions (like ASSERT, __bad_atomic_size,
>> LOG, etc ...) that crash execution or produce logs are not dangerous in initializer
>> lists. The evaluation order in abnormal conditions is not relevant. Evaluation order
>> of logging effects is always safe.
> 
> I thought I said so before: When talking of just logging, evaluation order
> may very well have a impact on correctness. Therefore we shouldn't mix
> debugging and logging.

My general feeling was that changes like the following one are not supported by
the community:

- x = { .field1 = function_with_logs_effects() /*other eventual code*/ };
+ int field1 = function_with_logs_effects();
+ x = { .field1 = field1 /*other eventual code*/};

so I tried to deviate as much as possible.

If having log effects is a good reason to do changes like the above, I can
propose a patch in that sense.

> 
>> Function hvm_get_guest_tsc_fixed (indirectly) performs different side effects.
>> For example it calls hvm_get_guest_time_fixed that contains an ASSERT and calls
>> to spin_lock and spin_unlock.
>>
>> These side effects are not dangerous: they can be executed regardless of the
>> initializer list evaluation order
>>
>> This patch deviates violations using SAF commits caused by debug/logging macros and
>> functions.
> 
> DYM "comments"?
> 

Oh yes, sorry.

>> --- a/xen/arch/arm/device.c
>> +++ b/xen/arch/arm/device.c
>> @@ -331,6 +331,7 @@ int handle_device(struct domain *d, struct dt_device_node *dev, p2m_type_t p2mt,
>>           .p2mt = p2mt,
>>           .skip_mapping = !own_device ||
>>                           (is_pci_passthrough_enabled() &&
>> +                        /* SAF-3-safe effects for debugging/logging reasons are safe */
>>                           (device_get_class(dev) == DEVICE_PCI_HOSTBRIDGE)),
> 
> Taking this just as example: I think the comment is too long. Just
> saying (leaving aside my comment higher up) "debugging/logging"
> would imo be sufficient.
> 

Ok.

>> --- a/xen/arch/arm/guestcopy.c
>> +++ b/xen/arch/arm/guestcopy.c
>> @@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf, uint64_t addr, unsigned int len,
>>   unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
>>   {
>>       return copy_guest((void *)from, (vaddr_t)to, len,
>> -                      GVA_INFO(current), COPY_to_guest | COPY_linear);
>> +                      /* SAF-4-safe No persistent side effects */
>> +                      GVA_INFO(current),
> 
> I _still_ think this leaves ambiguity. The more that you need to look
> up GVA_INFO() to recognize what this is about.


Just to recap: here the point is that current reads a register with a volatile asm, so the
violation is in the expansion of GVA_INFO(current). Both GVA_INFO and current taken alone
are completely fine, so this is the only place where a SAF comment can be placed.

The exapansion is:
((copy_info_t) { .gva = { ((*({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"(&
   per_cpu__curr_vcpu)); (typeof(&per_cpu__curr_vcpu)) (__ptr + (({ uint64_t _r; asm volatile("mrs  %0, ""TPIDR_EL2" : "=r"
   (_r)); _r; }))); }))) } }), (1U << 1) | (1U << 2));

My proposals are:
1) address the violation moving the current expansion outside (extra variable);
2) put a more detailed comment to avoid the ambiguity;
3) use an ECL deviation for GVA_INFO(current).

Do you have any preference or proposal?

>> --- a/xen/arch/x86/hvm/hvm.c
>> +++ b/xen/arch/x86/hvm/hvm.c
>> @@ -800,6 +800,7 @@ static int cf_check hvm_save_cpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
>>   {
>>       struct segment_register seg;
>>       struct hvm_hw_cpu ctxt = {
>> +        /* SAF-3-safe effects for debugging/logging reasons are safe */
>>           .tsc = hvm_get_guest_tsc_fixed(v, v->domain->arch.hvm.sync_tsc),
> 
> A prereq for this imo is that the function take const struct vcpu *.
> But I'm not sure that'll suffice. The function can change at any time,
> rendering the comment here stale perhaps without anyone noticing.
>

IMO It isn't a strict prereq, but it would make everything more clear.

In any case, apart adding the const, I do not see other easy solutions.
Would you give me your ack if I change the function signature?

Another possible solutions would be documenting the function in the new
JSON file with a special attribute like only_debug_effect. Of course,
this still requires keeping the JSON up to date in case of changes.

>> --- a/xen/common/sched/core.c
>> +++ b/xen/common/sched/core.c
>> @@ -1521,6 +1521,7 @@ long vcpu_yield(void)
>>   
>>       SCHED_STAT_CRANK(vcpu_yield);
>>   
>> +    /* SAF-4-safe No persistent side effects */
>>       TRACE_2D(TRC_SCHED_YIELD, current->domain->domain_id, current->vcpu_id);
>>       raise_softirq(SCHEDULE_SOFTIRQ);
>>       return 0;
>> @@ -1899,6 +1900,7 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>           if ( copy_from_guest(&sched_shutdown, arg, 1) )
>>               break;
>>   
>> +        /* SAF-4-safe No persistent side effects */
>>           TRACE_3D(TRC_SCHED_SHUTDOWN,
>>                    current->domain->domain_id, current->vcpu_id,
>>                    sched_shutdown.reason);
>> @@ -1916,6 +1918,7 @@ ret_t do_sched_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>           if ( copy_from_guest(&sched_shutdown, arg, 1) )
>>               break;
>>   
>> +        /* SAF-4-safe No persistent side effects */
>>           TRACE_3D(TRC_SCHED_SHUTDOWN_CODE,
>>                    d->domain_id, current->vcpu_id, sched_shutdown.reason);
>>   
> 
> For all of these iirc the suggestion was to latch current into a local
> variable (named "curr" by convention).
> 

Ok.

> Jan

-- 
Simone Ballarin, M.Sc.

Field Application Engineer, BUGSENG (https://bugseng.com)



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:06:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:06:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677313.1053838 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXepJ-0002rg-Hz; Wed, 07 Feb 2024 10:06:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677313.1053838; Wed, 07 Feb 2024 10:06:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXepJ-0002rZ-FO; Wed, 07 Feb 2024 10:06:45 +0000
Received: by outflank-mailman (input) for mailman id 677313;
 Wed, 07 Feb 2024 10:06:44 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rXepI-0002rT-E5
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:06:44 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rXepH-0006sv-UF; Wed, 07 Feb 2024 10:06:43 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rXepH-0005Mc-Nr; Wed, 07 Feb 2024 10:06:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=D0dBMrZ3WAJaA4lhRMFcD0qZlUFXCJfatxEbtL5kBjc=; b=cQ+FESx5w5HdNGevGiOQm7lEIF
	H7RIvV5sIipxg9wEHCpJUWfwtK1cytKOGV6cNSnnWPkS5fBmcMmsKHheU3ubuijRWh6s2cmbbQdsF
	KuKaLyQwdbUuaFTQJTQI6OWMvR4GrBEKbCSZIZzt6AlR/aHkqgjtQngrH6uEv859hSEg=;
Message-ID: <e3a967fb-f9c5-495f-a30a-1826e3673c95@xen.org>
Date: Wed, 7 Feb 2024 10:06:41 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v4 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
Content-Language: en-GB
To: Michal Orzel <michal.orzel@amd.com>,
 Ayan Kumar Halder <ayan.kumar.halder@amd.com>, xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, luca.fancellu@arm.com
References: <20240131121049.225044-1-ayan.kumar.halder@amd.com>
 <20240131121049.225044-2-ayan.kumar.halder@amd.com>
 <c7841d11-d072-4787-9d76-ee94123605bb@xen.org>
 <8daa2461-092f-48ad-8556-3bba398ae544@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <8daa2461-092f-48ad-8556-3bba398ae544@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Michal,

On 07/02/2024 07:45, Michal Orzel wrote:
> On 06/02/2024 19:49, Julien Grall wrote:
>> On 31/01/2024 12:10, Ayan Kumar Halder wrote:
>>> There can be situations when the registers cannot be emulated to their full
>>> functionality. This can be due to the complexity involved. In such cases, one
>>> can emulate those registers as RAZ/WI for example. We call them as partial
>>> emulation.
>>>
>>> Some registers are non-optional and as such there is nothing preventing an OS
>>> from accessing them.
>>> Instead of injecting undefined exception (thus crashing a guest), one may want
>>> to prefer a partial emulation to let the guest running (in some cases accepting
>>> the fact that it might result in unwanted behavior).
>>>
>>> A suitable example of this (as seen in subsequent patches) is emulation of
>>> DBGDTRTX_EL0 (on Arm64) and DBGDTRTXINT(on Arm32). These non-optional
>>> registers can be emulated as RAZ/WI and they can be enclosed within
>>> CONFIG_PARTIAL_EMULATION.
>>>
>>> Further, "partial-emulation" command line option allows us to
>>> enable/disable partial emulation at run time. While CONFIG_PARTIAL_EMULATION
>>> enables support for partial emulation at compile time (i.e. adds code for
>>> partial emulation), this option may be enabled or disabled by Yocto or other
>>> build systems. However if the build system turns this option on, users
>>> can use scripts like Imagebuilder to generate uboot-script which will append
>>> "partial-emulation=false" to xen command line to turn off the partial
>>> emulation. Thus, it helps to avoid rebuilding xen.
>>>
>>> By default, "CONFIG_PARTIAL_EMULATION=y" and "partial-emulation=false".
>>> This is done so that Xen supports partial emulation. However, customers are
>>> fully aware when they enable partial emulation. It's important to note that
>>> enabling such support might result in unwanted/non-spec compliant behavior.
>>
>> Can you remind me why this is built by default? In particular...
> This is the result of RFC discussion we had, where both Bertrand and Stefano agreed on having
> the Kconfig enabled by default to improve user experience:
> Bertrand:
> https://lore.kernel.org/xen-devel/C0ADC33B-1966-4D3E-B081-A3AA0C3AE76D@arm.com/
> Stefano:
> https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2312081514450.1703076@ubuntu-linux-20-04-desktop/

Thanks for the pointer. I thought a bit more and per-se the default of 
the Kconfig doesn't really matter too much. So I am fine to keep it on 
by default.

That said, I think we need to detail the security support for the 
command line in SUPPORT.md. I think we want to consider to not security 
support any issue that would allow the userland to attack the guest OS 
due to a bug in the partial emulation.

I would be fine with security supporting any issue that would 
DoS/compromise Xen.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:19:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:19:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677320.1053848 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXf1N-0004pP-J2; Wed, 07 Feb 2024 10:19:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677320.1053848; Wed, 07 Feb 2024 10:19:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXf1N-0004pI-GS; Wed, 07 Feb 2024 10:19:13 +0000
Received: by outflank-mailman (input) for mailman id 677320;
 Wed, 07 Feb 2024 10:19:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXf1M-0004p8-0O; Wed, 07 Feb 2024 10:19:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXf1L-00076Q-HH; Wed, 07 Feb 2024 10:19:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXf1L-0006FL-2l; Wed, 07 Feb 2024 10:19:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXf1L-0006C1-1l; Wed, 07 Feb 2024 10:19:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=p6nwWWaVn2qpg2Gg12h660WWC9V90OIHBHZTmvhndTQ=; b=5GaYg2KnIDIEnhmQ013EMKmxvF
	f3vIbIjVBrP/+t58OVnQR0RZEtG3LLLSHM2YKEWD6IJd1eTk8k/1k5zQDJkwOypo9+vSZztS8xtgn
	kFyH8fjIKCW5djxnb8LyF7styV2BZvHFjKRfgavgOl/7eb0T+TpoowN+575mf1FSiKdg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184613-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184613: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=b051762e3e3968a1790d084264e46ddd4e56b9ec
X-Osstest-Versions-That:
    libvirt=442061583e9dc0e4e3bf314275979051345a4a93
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 07 Feb 2024 10:19:11 +0000

flight 184613 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184613/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184573
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184573
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184573
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              b051762e3e3968a1790d084264e46ddd4e56b9ec
baseline version:
 libvirt              442061583e9dc0e4e3bf314275979051345a4a93

Last test of basis   184573  2024-02-03 04:24:41 Z    4 days
Testing same since   184613  2024-02-07 04:22:23 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Göran Uddeborg <goeran@uddeborg.se>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   442061583e..b051762e3e  b051762e3e3968a1790d084264e46ddd4e56b9ec -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:24:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:24:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677326.1053858 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXf6U-0006J0-6l; Wed, 07 Feb 2024 10:24:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677326.1053858; Wed, 07 Feb 2024 10:24:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXf6U-0006It-3r; Wed, 07 Feb 2024 10:24:30 +0000
Received: by outflank-mailman (input) for mailman id 677326;
 Wed, 07 Feb 2024 10:24:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXf6S-0006In-Qd
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:24:28 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 11b9a908-c5a3-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 11:24:26 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-33b466bc363so299308f8f.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 02:24:26 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a6-20020adffac6000000b0033b4a52bfbfsm1137846wrs.57.2024.02.07.02.24.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 02:24:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11b9a908-c5a3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707301466; x=1707906266; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=DD2ZdcqNJo+L5U+OTILg3YCDJtGAHhzTdFjO+uk8Zpc=;
        b=Xx/s9ULny2oA9eqYGkMd/RFdQ8IL36JOGFVU1k0xB8jh/Mvd/oSz+BAiasKnljNJUn
         2emCT0cmUdvthgleFDTTZBSpsqsA4k0Tz2I4Am7cp08v2WJyaTyq83S3DnLnQvlmkFhm
         kY1hEZUKZaMrHUZBu3KfoE1oibNFEsL55DyB2P4kDPi4ybgjqBZxQXfW2sntauqpHFi0
         bIypqwXHA8cjlGZCmQ/ERL0WxLcfVRUegexlIF9gTtmQIFrFpSKIvts0VhmaWlkJrbHs
         dvf6XaNr1ZkhK/Z92sXCXY65KNcjXU3FOGzOm3NEHGhDErogsdZMQMi+V20xu7tBGKF8
         dzaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707301466; x=1707906266;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=DD2ZdcqNJo+L5U+OTILg3YCDJtGAHhzTdFjO+uk8Zpc=;
        b=E+ANQqx8lxUh6yXGzCCYb6/buYg7bKzXdEkWrapK3f/Rrvvo3YLvNB6PsuoIgRWDbe
         lhsUHE5neDBFeumnB7r05y+FaFzD3pIKhCjvYKdLJYnwPr0g+MTkfemdQt/XUfaXGvN7
         wbHlf1foGmGMdwrYhhU7piANKdr4yqTnhMRm1d5SWSfbDV+lEoeraZq1uJ4Gfpzdt0HZ
         M6bIHQnfSBEdgX2UORx20w2E7JvxAuOsc0kXNDOrq5bJQhf6ieGr9NPDAyMEYglolENh
         p3r+dMnZ2jNEO71kSOfLG63yhIiBO1zPdJPFP24mS03pAJPb51N2Ys8ILoRC4wgsGpCe
         7acA==
X-Forwarded-Encrypted: i=1; AJvYcCWJeY7YSQqR67n2TuxeaJCvSUFJe5USHEuAES9qAFcalDQvJ2OFQ/4FZKAEpAO3y8bDRi1wE6HGcIYGo9OU+rdN/QQ6XQOTTv6zsCcFSZw=
X-Gm-Message-State: AOJu0Yxs3PONesz3fVtccRCYnO1REgFM8zLaPBVxW8bGsBridHujmMXb
	gESzzpyFLIbId525nDk8CLZySUb2EXE5IKqfBB2hcZssV7GabySPHBpyN4h92A==
X-Google-Smtp-Source: AGHT+IEB0PXF19yXhagbO8thfBtNm1bkMy70NT9rpsqw4RzAIUUGiawgsvPYdyUUojOVmnXrd2tcrg==
X-Received: by 2002:adf:f546:0:b0:33b:4ad0:5475 with SMTP id j6-20020adff546000000b0033b4ad05475mr2091513wrp.37.1707301465785;
        Wed, 07 Feb 2024 02:24:25 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX6bIO6tUKisWhyF0XHsGxQ4yRmYU5SidGvgdgYOEZlTDuvCNPleEl7KZMt/pyTcF7b2zIuJlEom6rsBLnL1Y9Vypx9UhmWKkraELrDkeSpwpi4n/B1W4MWYTKMrYzPy48sX0gvSLet1POa98XuOLIx6FlJTwad5thX5VkxUvk1mA9wh13sq86k8Gyh+BydolsvuwA2pvgoFcyOIV1z6BzlnqKCKBzOnTkP4mTAw9qbW+7DdkfTpu3xy8I3racrXLwdwrKap3BC203h9RQyL6y5rDI5U0g0Hfb0R5EkKjNB/M8Zo8hWUsPwoD6WrdDahQhZ9S/CPCOS0nQPUBxaIVohihVC0LzY5d6FgAaVpJKm0aFK/hokJo/9o+P/Pe+nDwaOXV4Yl63Ux3Mme4yJFlEySljPfxD8MwFPmEUlKhIn+Y1WBCUZv7FMvR8Pe96VxG8=
Message-ID: <41474fe7-c65f-4244-a455-0aaa0e4315e3@suse.com>
Date: Wed, 7 Feb 2024 11:24:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and
 logging effects
Content-Language: en-US
To: Simone Ballarin <simone.ballarin@bugseng.com>
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com>
 <09869431-ed00-4102-93fd-84c697a5c950@suse.com>
 <851eb6ec-558b-4a89-a31a-34046730bb55@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <851eb6ec-558b-4a89-a31a-34046730bb55@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 11:03, Simone Ballarin wrote:
> On 06/02/24 13:04, Jan Beulich wrote:
>> On 02.02.2024 16:16, Simone Ballarin wrote:
>>> Rule 13.1: Initializer lists shall not contain persistent side effects
>>>
>>> Effects caused by debug/logging macros and functions (like ASSERT, __bad_atomic_size,
>>> LOG, etc ...) that crash execution or produce logs are not dangerous in initializer
>>> lists. The evaluation order in abnormal conditions is not relevant. Evaluation order
>>> of logging effects is always safe.
>>
>> I thought I said so before: When talking of just logging, evaluation order
>> may very well have a impact on correctness. Therefore we shouldn't mix
>> debugging and logging.
> 
> My general feeling was that changes like the following one are not supported by
> the community:
> 
> - x = { .field1 = function_with_logs_effects() /*other eventual code*/ };
> + int field1 = function_with_logs_effects();
> + x = { .field1 = field1 /*other eventual code*/};
> 
> so I tried to deviate as much as possible.
> 
> If having log effects is a good reason to do changes like the above, I can
> propose a patch in that sense.

Just to avoid misunderstandings: I'm not advocating for changes like the
one you outline above. I simply consider the rule too strict: There's
nothing at risk when there's just a single operation with side effects
in an initializer. Even when there are multiple such operations, whether
there's anything at risk depends on whether any of the side effects
actually collide. In a number of cases the compiler would actually warn
(and thus, due to -Werror, the build would fail).

The primary purpose of my comment here was that we need to please
separate debugging from logging side effects.

>>> --- a/xen/arch/arm/guestcopy.c
>>> +++ b/xen/arch/arm/guestcopy.c
>>> @@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf, uint64_t addr, unsigned int len,
>>>   unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
>>>   {
>>>       return copy_guest((void *)from, (vaddr_t)to, len,
>>> -                      GVA_INFO(current), COPY_to_guest | COPY_linear);
>>> +                      /* SAF-4-safe No persistent side effects */
>>> +                      GVA_INFO(current),
>>
>> I _still_ think this leaves ambiguity. The more that you need to look
>> up GVA_INFO() to recognize what this is about.
> 
> 
> Just to recap: here the point is that current reads a register with a volatile asm, so the
> violation is in the expansion of GVA_INFO(current). Both GVA_INFO and current taken alone
> are completely fine, so this is the only place where a SAF comment can be placed.
> 
> The exapansion is:
> ((copy_info_t) { .gva = { ((*({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"(&
>    per_cpu__curr_vcpu)); (typeof(&per_cpu__curr_vcpu)) (__ptr + (({ uint64_t _r; asm volatile("mrs  %0, ""TPIDR_EL2" : "=r"
>    (_r)); _r; }))); }))) } }), (1U << 1) | (1U << 2));
> 
> My proposals are:
> 1) address the violation moving the current expansion outside (extra variable);
> 2) put a more detailed comment to avoid the ambiguity;
> 3) use an ECL deviation for GVA_INFO(current).
> 
> Do you have any preference or proposal?

Imo 3 is not an option at all. Probably 1 wouldn't be too bad here, but
I still wouldn't like it (as matching a general pattern I try to avoid:
introducing local variables that are used just once and don't meaningfully
improve e.g. readability). Therefore out of what you list, 2 would remain.
But I'm not happy with a comment here either - as per above, there's
nothing that can go wrong here as long as there's only a single construct
with side effect(s).

>>> --- a/xen/arch/x86/hvm/hvm.c
>>> +++ b/xen/arch/x86/hvm/hvm.c
>>> @@ -800,6 +800,7 @@ static int cf_check hvm_save_cpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
>>>   {
>>>       struct segment_register seg;
>>>       struct hvm_hw_cpu ctxt = {
>>> +        /* SAF-3-safe effects for debugging/logging reasons are safe */
>>>           .tsc = hvm_get_guest_tsc_fixed(v, v->domain->arch.hvm.sync_tsc),
>>
>> A prereq for this imo is that the function take const struct vcpu *.
>> But I'm not sure that'll suffice. The function can change at any time,
>> rendering the comment here stale perhaps without anyone noticing.
>>
> 
> IMO It isn't a strict prereq, but it would make everything more clear.
> 
> In any case, apart adding the const, I do not see other easy solutions.
> Would you give me your ack if I change the function signature?

Well, as said: I'm not sure that'll suffice.

> Another possible solutions would be documenting the function in the new
> JSON file with a special attribute like only_debug_effect. Of course,
> this still requires keeping the JSON up to date in case of changes.

Exactly. So wouldn't really help.

In any event I'd like to ask that you consider splitting up this patch,
such that you won't need multiple acks for any of the parts. That'll
also allow focusing on one aspect at a time in reviews.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:31:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:31:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677334.1053872 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfDV-00080r-00; Wed, 07 Feb 2024 10:31:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677334.1053872; Wed, 07 Feb 2024 10:31:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfDU-00080j-Rz; Wed, 07 Feb 2024 10:31:44 +0000
Received: by outflank-mailman (input) for mailman id 677334;
 Wed, 07 Feb 2024 10:31:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=D7mL=JQ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXfDT-00080F-LQ
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:31:43 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 153b89b5-c5a4-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 11:31:42 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 46571221B0;
 Wed,  7 Feb 2024 10:31:41 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 14E2713931;
 Wed,  7 Feb 2024 10:31:41 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id L2aXAw1cw2UvYgAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 07 Feb 2024 10:31:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 153b89b5-c5a4-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707301901; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=ZgvFN9fVJULnhlM4tZSu8boApfM5Dt//TBSdSqybRdM=;
	b=CxfSExGkqXhwo/r8W57pEtsbznbIa/qSIvY81eSuXUNEMn1ZU3uc5RqII/HX8ifUJ0yB4D
	yF+bdRxpbRYgTlge0v44vIpf2xgyjb4yV57RYN2z5fa6MCDnsqerL3LVJgo/NNSr+etNY6
	U6MQsUeWDUNnuY2jsVFmxhhFy/Xfu/4=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707301901; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=ZgvFN9fVJULnhlM4tZSu8boApfM5Dt//TBSdSqybRdM=;
	b=CxfSExGkqXhwo/r8W57pEtsbznbIa/qSIvY81eSuXUNEMn1ZU3uc5RqII/HX8ifUJ0yB4D
	yF+bdRxpbRYgTlge0v44vIpf2xgyjb4yV57RYN2z5fa6MCDnsqerL3LVJgo/NNSr+etNY6
	U6MQsUeWDUNnuY2jsVFmxhhFy/Xfu/4=
From: Juergen Gross <jgross@suse.com>
To: minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org
Cc: samuel.thibault@ens-lyon.org,
	wl@xen.org,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Date: Wed,  7 Feb 2024 11:31:38 +0100
Message-Id: <20240207103138.26901-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=CxfSExGk
X-Spamd-Result: default: False [2.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 2.69
X-Rspamd-Queue-Id: 46571221B0
X-Spam-Level: **
X-Spam-Flag: NO
X-Spamd-Bar: ++

The .bss segment should be zeroed at very early boot.

While adding the extern declaration of __bss_start for x86, make it
together with the other linker table defined section boundaries
common for all architectures.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/setup.c      | 2 ++
 include/arm/arch_mm.h | 1 -
 include/mm.h          | 2 ++
 include/x86/arch_mm.h | 1 -
 4 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/x86/setup.c b/arch/x86/setup.c
index b27bbed7..3dddf4ad 100644
--- a/arch/x86/setup.c
+++ b/arch/x86/setup.c
@@ -184,6 +184,8 @@ arch_init(void *par)
 {
 	static char hello[] = "Bootstrapping...\n";
 
+	memset(&__bss_start, 0, &_end - &__bss_start);
+
 	hpc_init();
 	(void)HYPERVISOR_console_io(CONSOLEIO_write, strlen(hello), hello);
 
diff --git a/include/arm/arch_mm.h b/include/arm/arch_mm.h
index 79d9e05b..335eb4ff 100644
--- a/include/arm/arch_mm.h
+++ b/include/arm/arch_mm.h
@@ -3,7 +3,6 @@
 
 typedef uint64_t paddr_t;
 
-extern char _text, _etext, _erodata, _edata, _end, __bss_start;
 extern int _boot_stack[];
 extern int _boot_stack_end[];
 extern uint32_t physical_address_offset;	/* Add this to a virtual address to get the physical address (wraps at 4GB) */
diff --git a/include/mm.h b/include/mm.h
index 4fc364ff..e02e080b 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -46,6 +46,8 @@
 #define round_pgdown(_p)  ((_p) & PAGE_MASK)
 #define round_pgup(_p)    (((_p) + (PAGE_SIZE - 1)) & PAGE_MASK)
 
+extern char _text, _etext, _erodata, _edata, _end, __bss_start;
+
 extern unsigned long nr_free_pages;
 
 extern unsigned long *mm_alloc_bitmap;
diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index a1b975dc..6b398cef 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -225,7 +225,6 @@ extern unsigned long *phys_to_machine_mapping;
 #else
 extern pgentry_t page_table_base[];
 #endif
-extern char _text, _etext, _erodata, _edata, _end;
 extern unsigned long mfn_zero;
 static __inline__ maddr_t phys_to_machine(paddr_t phys)
 {
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:33:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:33:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677345.1053882 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfF2-0000Cy-DR; Wed, 07 Feb 2024 10:33:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677345.1053882; Wed, 07 Feb 2024 10:33:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfF2-0000Cr-As; Wed, 07 Feb 2024 10:33:20 +0000
Received: by outflank-mailman (input) for mailman id 677345;
 Wed, 07 Feb 2024 10:33:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXfF0-0000Cl-KP
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:33:18 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4dbe148d-c5a4-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 11:33:16 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-40fff96d5daso3837445e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 02:33:16 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h13-20020a05600c314d00b0040fdc7f4fcdsm4894934wmo.4.2024.02.07.02.33.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 02:33:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4dbe148d-c5a4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707301996; x=1707906796; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0ZGj5tm/xCQ2+UYNEw9SgAD65WI1XL19bihdd14q2NI=;
        b=UAGL6JAOgv7wy/NXnGZ6UDsuQMSEWv8b0BZVFCJPhcQyTUT0ex3AmISUd3flGOjPgJ
         YtHNtEKAWtZeCWdZdsqZODdekSNiA+g5KaNwTK8XzgmOc+fAp7aBT0xHg7DlCy+3ahzy
         1k8+Z/RHB8z4BVUivV2EatuevPtU2xfVCjZPkS4bkDM0onDc29d5TRWT2lCVczsT3Lt/
         Hkay4ogacYtiUEgB0hsoGnQHpb/VS71A1tT89x+iG2oritoK+Hy8RjRDIGxRAOTTQx52
         /lMkoOSxiva14QfwGWcLpCaXOylcI//SBvbsGdoJ6qLu7ncyKmoychHd3RpSE0t5o2bL
         We6A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707301996; x=1707906796;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0ZGj5tm/xCQ2+UYNEw9SgAD65WI1XL19bihdd14q2NI=;
        b=xM3yrSrnc8ngJTMnyw26HzicPt2yMQy6EPVQYlced/pjE9uh44fPgm/nhN3n24TUwo
         jRRo6D7rJXadsVtPuimaUYMYIyw8x7XToQiWNgV3+lpaXrLkcgappXmPNFj94sb3w9/p
         2lh01wNKKB0I1+LuxvjZulwdJjLvP2nXglRNwE6gC6e/2zFZiPMkN4zD3hO+KMwGES9u
         uhQZJt38Amso8GLcJ4cDvHgHgmse5mVjKGJ/0NVJeGiJfNdNL4vLlwLaQ0UCugCHIkiD
         uim59lHkctvXZ6gDP+iUU56T8eerB4LVvELMZH5+PshwKWV8y3BwEdT8crip4TWsfWYa
         cJNQ==
X-Gm-Message-State: AOJu0YzLV2C26qnWTKjaf/6sf81TcZAZdIKK0QH8QgcfZJhirEVrfcup
	AJ093eT6GQZCipPHdwx/nxGSragY7y8nYQZ7FhNamPGgSyVnf/Hvaok64BRBMA==
X-Google-Smtp-Source: AGHT+IE1yd8TsaNvx8WfeHmQG2itRlSrKm7BVviPV4E6vn6HI6IjGuqnYJoVT+KaS91ODyB63DPFsw==
X-Received: by 2002:a05:600c:524d:b0:410:471:a2e with SMTP id fc13-20020a05600c524d00b0041004710a2emr1378298wmb.32.1707301996045;
        Wed, 07 Feb 2024 02:33:16 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVSFwsBZF0dcIs8rE6fkwRoTJ42P87bkQzxNqCj9CjZY8+vUfP2u1aM4+J6thwyUveomgzopFtSuPTyXPGuk4u9qIkyqjFk3otdSw20yII=
Message-ID: <6ac74dfb-1899-41a9-9e1c-5a204fb5913f@suse.com>
Date: Wed, 7 Feb 2024 11:33:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/4] iommu/vt-d: switch to common RMRR checker
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240201170159.66330-1-roger.pau@citrix.com>
 <20240201170159.66330-4-roger.pau@citrix.com>
 <90f8e183-5e78-49e8-ae2f-9898c1d7686b@suse.com> <ZcNG5C9gEzlcWJPw@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcNG5C9gEzlcWJPw@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07.02.2024 10:01, Roger Pau Monné wrote:
> On Tue, Feb 06, 2024 at 12:28:07PM +0100, Jan Beulich wrote:
>> On 01.02.2024 18:01, Roger Pau Monne wrote:
>>> Use the newly introduced generic unity map checker.
>>>
>>> Also drop the message recommending the usage of iommu_inclusive_mapping: the
>>> ranges would end up being mapped anyway even if some of the checks above
>>> failed, regardless of whether iommu_inclusive_mapping is set.
>>
>> I'm afraid I don't understand this: When not in an appropriate E820
>> region, you now even fail IOMMU initialization. Shouldn't such
>> failure only occur when inclusive mappings weren't requested? At
>> which point referring to that option is still relevant?
> 
> This is now better handled, since the VT-d code will use the same
> logic as the AMD-Vi logic and attempt to 'convert' such bogus RMRR
> regions so they can be safely used.  iommu_unity_region_ok() signals
> the RMRR region is impossible to be used, and hence not even
> iommu_inclusive_mapping would help in that case.

Impossible only in so far as we don't know whether a such named region
would actually still be accessed post-boot. But yes, if it wouldn't be
accessed, there would also be no need for passing the extra option.

>  Also note that
> iommu_inclusive_mapping is only applicable to PV, so the message was
> already wrong in the PVH case.

This is a fair point, which probably wants mentioning as (partial)
justification. Plus iirc the intention was to get rid of that option
anyway, at some point.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:38:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:38:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677353.1053897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfK8-0001CB-2C; Wed, 07 Feb 2024 10:38:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677353.1053897; Wed, 07 Feb 2024 10:38:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfK7-0001C4-VP; Wed, 07 Feb 2024 10:38:35 +0000
Received: by outflank-mailman (input) for mailman id 677353;
 Wed, 07 Feb 2024 10:38:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CAtM=JQ=ens-lyon.org=samuel.thibault@bounce.ens-lyon.org>)
 id 1rXfK7-0001A0-8N
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:38:35 +0000
Received: from sonata.ens-lyon.org (sonata.ens-lyon.org [140.77.166.138])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0a5cdfa2-c5a5-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 11:38:33 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by sonata.ens-lyon.org (Postfix) with ESMTP id B5412A02E8;
 Wed,  7 Feb 2024 11:38:32 +0100 (CET)
Received: from sonata.ens-lyon.org ([127.0.0.1])
 by localhost (sonata.ens-lyon.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id W1GDuaE29GF2; Wed,  7 Feb 2024 11:38:32 +0100 (CET)
Received: from begin (nat-inria-interne-52-gw-01-bso.bordeaux.inria.fr
 [194.199.1.52])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by sonata.ens-lyon.org (Postfix) with ESMTPSA id 5443EA02D4;
 Wed,  7 Feb 2024 11:38:32 +0100 (CET)
Received: from samy by begin with local (Exim 4.97)
 (envelope-from <samuel.thibault@ens-lyon.org>)
 id 1rXfK3-00000008jDN-3xTx; Wed, 07 Feb 2024 11:38:31 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a5cdfa2-c5a5-11ee-8a49-1f161083a0e0
Date: Wed, 7 Feb 2024 11:38:31 +0100
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Message-ID: <20240207103831.dtvlyjtbmgz7kmll@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org, wl@xen.org
References: <20240207103138.26901-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240207103138.26901-1-jgross@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Juergen Gross, le mer. 07 févr. 2024 11:31:38 +0100, a ecrit:
> The .bss segment should be zeroed at very early boot.

Is that not done by the elf loader of Xen?

> While adding the extern declaration of __bss_start for x86, make it
> together with the other linker table defined section boundaries
> common for all architectures.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  arch/x86/setup.c      | 2 ++
>  include/arm/arch_mm.h | 1 -
>  include/mm.h          | 2 ++
>  include/x86/arch_mm.h | 1 -
>  4 files changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/setup.c b/arch/x86/setup.c
> index b27bbed7..3dddf4ad 100644
> --- a/arch/x86/setup.c
> +++ b/arch/x86/setup.c
> @@ -184,6 +184,8 @@ arch_init(void *par)
>  {
>  	static char hello[] = "Bootstrapping...\n";
>  
> +	memset(&__bss_start, 0, &_end - &__bss_start);
> +
>  	hpc_init();
>  	(void)HYPERVISOR_console_io(CONSOLEIO_write, strlen(hello), hello);
>  
> diff --git a/include/arm/arch_mm.h b/include/arm/arch_mm.h
> index 79d9e05b..335eb4ff 100644
> --- a/include/arm/arch_mm.h
> +++ b/include/arm/arch_mm.h
> @@ -3,7 +3,6 @@
>  
>  typedef uint64_t paddr_t;
>  
> -extern char _text, _etext, _erodata, _edata, _end, __bss_start;
>  extern int _boot_stack[];
>  extern int _boot_stack_end[];
>  extern uint32_t physical_address_offset;	/* Add this to a virtual address to get the physical address (wraps at 4GB) */
> diff --git a/include/mm.h b/include/mm.h
> index 4fc364ff..e02e080b 100644
> --- a/include/mm.h
> +++ b/include/mm.h
> @@ -46,6 +46,8 @@
>  #define round_pgdown(_p)  ((_p) & PAGE_MASK)
>  #define round_pgup(_p)    (((_p) + (PAGE_SIZE - 1)) & PAGE_MASK)
>  
> +extern char _text, _etext, _erodata, _edata, _end, __bss_start;
> +
>  extern unsigned long nr_free_pages;
>  
>  extern unsigned long *mm_alloc_bitmap;
> diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
> index a1b975dc..6b398cef 100644
> --- a/include/x86/arch_mm.h
> +++ b/include/x86/arch_mm.h
> @@ -225,7 +225,6 @@ extern unsigned long *phys_to_machine_mapping;
>  #else
>  extern pgentry_t page_table_base[];
>  #endif
> -extern char _text, _etext, _erodata, _edata, _end;
>  extern unsigned long mfn_zero;
>  static __inline__ maddr_t phys_to_machine(paddr_t phys)
>  {
> -- 
> 2.35.3


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:39:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:39:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677359.1053907 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfL4-0001k3-D6; Wed, 07 Feb 2024 10:39:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677359.1053907; Wed, 07 Feb 2024 10:39:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfL4-0001jw-90; Wed, 07 Feb 2024 10:39:34 +0000
Received: by outflank-mailman (input) for mailman id 677359;
 Wed, 07 Feb 2024 10:39:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXfL2-0001jU-QQ
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:39:32 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2ce76f7e-c5a5-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 11:39:31 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-4101fc00832so29125e9.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 02:39:31 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 q18-20020a05600c46d200b0040ff2933959sm1659650wmo.7.2024.02.07.02.39.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 02:39:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ce76f7e-c5a5-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707302370; x=1707907170; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=WT0YfbOyND1R+V/f2LYn++s/QuD8BooAGB1/mezQFbQ=;
        b=Yjs/YblvLAZAO6JM7bj9bMpsYmY+k+XHktbjxmnHLy98KrOnnG5Z2H0RBX19DL2/oY
         OG9fyHfRMxcgjUtqKENYkgGQau1KM//311ajnwNfiSWBl4xCcFfNEI3MxPMHqbzZ9ycW
         ZRP0lvo24WZegyNyDz04SDoNvjEDA6s8fClYwOuFkK9pV3oudEUUGuNFl3MkHc0X1kEn
         Ctn/LGJVEoG9y1G0R/uYIrSjQ8mLsGh/IfWglocNUPHbE9q57Ir6OkEt+iKR6eqggcd0
         cTmD3lNtYeFLktuf0u6Kc8pEKsxm27hV90CNqw3J166FmimaO+xrPL3P/yOueBq3qWp+
         S2Sw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707302370; x=1707907170;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=WT0YfbOyND1R+V/f2LYn++s/QuD8BooAGB1/mezQFbQ=;
        b=lDaZGk9YdYGV+YYFLCSZ4XE5pLfPTwJWzlHhugnfg/ZtSV1V90bu8Do7Sk7GZiBGiC
         h/kXdINQ9b8OwTmXAoA/27ovDrMXQkUjgltQUoc/p5Su45OaZh9a2Hoc8iQCDaRZvlHH
         J/yQ5GcKWp5lWjSa15h6GwE5mnb+1AbZ4qrsor77SWkxgwVjvYE8K2TVmMYgkBVy/XdW
         f7z4Z2P+7E7wa9YY2J5VK5amYJ6PvsBb1aj4gk7xJ7IZ3uvPxyNZQeHn0b4Y0gtev/C9
         a4G/QrTK1BuTGnSrCW3IaOGC3P5X4d3u3on7glCJheZTvw8zPRzjwKjLl8vtSyu+mYO8
         TiTA==
X-Forwarded-Encrypted: i=1; AJvYcCXVXaZapBjW3jcByiXbKE+Rym90X8e1MjkYp/XpXXc0mkv8+7nlBHSAsMvB7fofu5wNH4KrMquAxF6dgRwxaLFKaP5sPzyLF8TGEDkbGUk=
X-Gm-Message-State: AOJu0YzqyLP51QF3fcV47OUrmeD35pShEs6w80TQGQsuNJ/iN41vcM+2
	/RucV0NBggiTuVsiwQR9m97iE0JuH/DMITbitmM93ZJ79aSUfqjbZ3ZXa/4fvj/vhSASoOhwxAo
	=
X-Google-Smtp-Source: AGHT+IEPVrLZc7pb1qrYFWrkO/5oBB5XuvZfLjlscw3zYHX7FvhClmgJXGZQhs3J8Ywd6Sy7kkflmQ==
X-Received: by 2002:a05:600c:511f:b0:40e:d425:85a with SMTP id o31-20020a05600c511f00b0040ed425085amr4155663wms.17.1707302370449;
        Wed, 07 Feb 2024 02:39:30 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXgAH1rCCulBF6TxhZN0zix0vD42yIbGoFuIh9JlsvfBDjN00X3f0Lm0IFysT0GxR6DhhDRRwpRs13J0WOJvXgz7MCeSSM0aEGteBrid7wmkDxCaZPHtcscWOqDSbj27XStGw3b4Kn7eu3tXvm4TVvu0cnYzcyp/axwf+xyTA==
Message-ID: <6ba32907-2d35-4dcf-ba3b-409984f1d877@suse.com>
Date: Wed, 7 Feb 2024 11:39:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: samuel.thibault@ens-lyon.org, wl@xen.org,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org
References: <20240207103138.26901-1-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240207103138.26901-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 11:31, Juergen Gross wrote:
> --- a/arch/x86/setup.c
> +++ b/arch/x86/setup.c
> @@ -184,6 +184,8 @@ arch_init(void *par)
>  {
>  	static char hello[] = "Bootstrapping...\n";
>  
> +	memset(&__bss_start, 0, &_end - &__bss_start);

Doesn't / shouldn't the loader guarantee this? I ask in particular
because doing this here implies earlier assembly code may not write
to any variable in .bss (nothing like that looks to exist right now,
but who knows what may be added at some point).

> --- a/include/mm.h
> +++ b/include/mm.h
> @@ -46,6 +46,8 @@
>  #define round_pgdown(_p)  ((_p) & PAGE_MASK)
>  #define round_pgup(_p)    (((_p) + (PAGE_SIZE - 1)) & PAGE_MASK)
>  
> +extern char _text, _etext, _erodata, _edata, _end, __bss_start;

Maybe use the more conventional array form? Thus also eliminating
the need for & at the use sites?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:42:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:42:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677369.1053925 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfNr-0003Go-Rl; Wed, 07 Feb 2024 10:42:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677369.1053925; Wed, 07 Feb 2024 10:42:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfNr-0003Gh-Oy; Wed, 07 Feb 2024 10:42:27 +0000
Received: by outflank-mailman (input) for mailman id 677369;
 Wed, 07 Feb 2024 10:42:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=D7mL=JQ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXfNq-0003Eb-Cn
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:42:26 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 92cfef9d-c5a5-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 11:42:22 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 6037A221C8;
 Wed,  7 Feb 2024 10:42:21 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 36FB713931;
 Wed,  7 Feb 2024 10:42:21 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id FLLwC41ew2UzZQAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 07 Feb 2024 10:42:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92cfef9d-c5a5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707302541; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NF8R7aWuE/rrndv8O1H7/mhwfC8rWzlUKracjQGi0bo=;
	b=HLdVCeA7fWxRTsZYcKpV2g3gE/Po0UwaY7R/4mugfxk6iB2ygKuxe3UkySW5+vZQWlnFrw
	S5J61Qk8Vxzwusc0Rp5kTB4THDHR9tzvSEu3frF48ZupjvovH7C314SrQ/R0CDUkODfrbv
	XSuhazk2nk7zzfz81wC2wOKnjSM0c8w=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707302541; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NF8R7aWuE/rrndv8O1H7/mhwfC8rWzlUKracjQGi0bo=;
	b=HLdVCeA7fWxRTsZYcKpV2g3gE/Po0UwaY7R/4mugfxk6iB2ygKuxe3UkySW5+vZQWlnFrw
	S5J61Qk8Vxzwusc0Rp5kTB4THDHR9tzvSEu3frF48ZupjvovH7C314SrQ/R0CDUkODfrbv
	XSuhazk2nk7zzfz81wC2wOKnjSM0c8w=
Message-ID: <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
Date: Wed, 7 Feb 2024 11:42:20 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Content-Language: en-US
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org
References: <20240207103138.26901-1-jgross@suse.com>
 <20240207103831.dtvlyjtbmgz7kmll@begin>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240207103831.dtvlyjtbmgz7kmll@begin>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=HLdVCeA7
X-Spamd-Result: default: False [-1.95 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 BAYES_HAM(-2.42)[97.35%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.77)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 6037A221C8
X-Spam-Level: 
X-Spam-Score: -1.95
X-Spam-Flag: NO

On 07.02.24 11:38, Samuel Thibault wrote:
> Juergen Gross, le mer. 07 févr. 2024 11:31:38 +0100, a ecrit:
>> The .bss segment should be zeroed at very early boot.
> 
> Is that not done by the elf loader of Xen?

It might be done by Xen tools today, but I'm quite sure it is not part
of the ABI. The hypervisor doesn't clear .bss when loading a domain (e.g.
dom0).

I stumbled over it while implementing kexec in Mini-OS. For that I need
it for sure.


Juergen


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:42:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:42:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677374.1053936 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfOI-0003jf-5O; Wed, 07 Feb 2024 10:42:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677374.1053936; Wed, 07 Feb 2024 10:42:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfOI-0003jY-0O; Wed, 07 Feb 2024 10:42:54 +0000
Received: by outflank-mailman (input) for mailman id 677374;
 Wed, 07 Feb 2024 10:42:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXfOG-0003i8-BV
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:42:52 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a4538263-c5a5-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 11:42:51 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-40fff96d5d7so3821185e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 02:42:51 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 r18-20020adfe692000000b0033afc81fc00sm1180459wrm.41.2024.02.07.02.42.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 02:42:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a4538263-c5a5-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707302571; x=1707907371; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ps2vaANjIsrQ7LJvCFKOCFibXgAxuGnwkpFAFDL5+ks=;
        b=XR7zncJLMoIdsrurWfwDl9/bPcOn4gnJXjZsf5YAz0BRRgosEE7AYECmqX/19V5Clu
         MDzdi3DLQo5Mf8SUYJIRajmUS9iy/vfAjYMkA957L85xX6qiOnenLcDwN4XscZ1977Sk
         yisuA2RXG+ppHClMEMFwvrnshHvJkexYDWYuH3F1w9aJ1wRdOUzXtBj42rno/8q1vW16
         KYooIwntlTpKl3WWKCpXJoGIHxNlkP8k4GlWZ72MYo7tCXvVDLX3tcouuenbbkZ3fJWR
         iyGPKT/+IdmkfWLlJ96nG6lKZvj3/Gxc4Smb4VEFqbgF2Xlo4qiFO26GjON/evSEkIMf
         8CNQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707302571; x=1707907371;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ps2vaANjIsrQ7LJvCFKOCFibXgAxuGnwkpFAFDL5+ks=;
        b=ecqj8gHpXRwKoRx2EmvnLeXoLNKAAgw2SGCa2jsq8ynkidj5jmlFXoOvZt8spemzWI
         0G0my0o4gBaKzQQI/J3ur35eCi71vPzImFGmCJWevAAjdF5qewc4LSRl7DsCPA2Uh9Lx
         VvJbMqgK+ymgsMJJnnf5INZ1FVZXRtkE5dYmDon2zIzNrJZq4ecDwhPx9flk6wZgAKxI
         y07FTiZDJd5qUlqkTabS6/i3RCuP7Y0aPi6zJgz2oycGy3SwXgXMBMiHo3K+dFhejRgW
         pDdI81JHwyxb93gp+Z7eBJUdJIAzzma2424+Ifxw1ZAeTKuF8PdKczCNXZHPyWx0Er5p
         ZVAQ==
X-Gm-Message-State: AOJu0YyWUxzUuqqVxlmN+MASmLcQMGSrdWSTV8YxXuVY8FMaMmx6s6fC
	MFz6Y0MXuCtvJU2FFW6f92VglRJ8deVRtm3Dd4mWqndXqZneeFE60EB4jtY89BuQoohJSC5K6F4
	=
X-Google-Smtp-Source: AGHT+IECWXUKvv/5uALmqI/6+NRnfoWMhScEbIoXGiOarsUYOeMjrdeU+XlybQV/vnZopLAIOzFiNA==
X-Received: by 2002:a05:600c:1e1a:b0:40f:f069:8d13 with SMTP id ay26-20020a05600c1e1a00b0040ff0698d13mr2711605wmb.9.1707302570834;
        Wed, 07 Feb 2024 02:42:50 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWLnG8mEx8JVmkKWLDyi0dAG/3gCzs4Vj95Sc/EQgCgprPpZy3ohf6Eeb43L3CcPfs0WWh+8BbZEKG8yV4O2c2xNXslm39wUIxL6y7nP+E=
Message-ID: <361f91c2-af27-4506-986d-0f580edb7861@suse.com>
Date: Wed, 7 Feb 2024 11:42:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/4] iommu/x86: make unity range checking more strict
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
References: <20240201170159.66330-1-roger.pau@citrix.com>
 <20240201170159.66330-5-roger.pau@citrix.com>
 <5db22ee1-40b3-4df8-88b8-446a2e224d22@suse.com> <ZcNKDKD4G7fpIX80@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcNKDKD4G7fpIX80@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07.02.2024 10:14, Roger Pau Monné wrote:
> On Tue, Feb 06, 2024 at 12:49:08PM +0100, Jan Beulich wrote:
>> On 01.02.2024 18:01, Roger Pau Monne wrote:
>>> Currently when a unity range overlaps with memory being used as RAM by the
>>> hypervisor the result would be that the IOMMU gets disabled.  However that's
>>> not enough, as even with the IOMMU disabled the device will still access the
>>> affected RAM areas.
>>
>> Hmm, no, I think this is going too far. Not the least because it is
>> s/will/may/. But also because if we really wanted such behavior, we
>> ought to also parse the respective ACPI tables when the "iommu=off".
> 
> I guessed so, hence why it's the last patch in the series.  TBH I
> think it's very unlikely that such system exist.

And you think so despite knowing that on some systems one needs to
manually specify RMRR regions?

>>> Doing so also allows to simplify the code and use a switch over the reported
>>> memory type(s).
>>
>> I'm afraid this isn't right either: page_get_ram_type() can set
>> multiple bits in its output.
> 
> It can indeed.  But if the only bit set is RAM_TYPE_CONVENTIONAL then
> the page will be handled as RAM, and that's where Xen would be in
> trouble if a device is also using such page as a unity map.
> 
> If the page however is RAM_TYPE_CONVENTIONAL | RAM_TYPE_RESERVED then
> the RESERVED type will take over the whole page, and it's no longer an
> issue to have a unity range covering it.

Okay, if this is intentional, than saying so in a code comment would
imo help quite a bit.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:46:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:46:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677384.1053944 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfS3-0004Up-Ks; Wed, 07 Feb 2024 10:46:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677384.1053944; Wed, 07 Feb 2024 10:46:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfS3-0004Ui-IE; Wed, 07 Feb 2024 10:46:47 +0000
Received: by outflank-mailman (input) for mailman id 677384;
 Wed, 07 Feb 2024 10:46:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXfS1-0004UX-Un
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:46:45 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2ee469b4-c5a6-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 11:46:43 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-4101d096c92so831295e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 02:46:43 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 fa7-20020a05600c518700b0040ef3ae26cdsm1653253wmb.37.2024.02.07.02.46.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 02:46:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ee469b4-c5a6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707302803; x=1707907603; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Y4pT1WW3ucR/uqmeTEXzqlHGNfRbEHUD7g5eHjPYIV4=;
        b=CQYdYr7RRXCKrOM7Zk+6BpCAo3a1A5NuUb/w1nJmRFpyfYydjnwX0o6avhHVpnS5Xu
         UfWPgqNI49ghrqSMIgi2p4JFEnrfJS73odcLRgdzyv272nnQrh2/7rBJ0pgwgOr3SsVB
         N3iBC0ZliUAjkySdlFNPt8w16gDrKwhmOqY/oENr6F2aXBxop5S2RDk6TpgYP2T3LyoI
         aWMGfdcUDi2t+DDCHXaZZZaViJM/RZ8RGqp8+iK6FcGFdurMeFvOL97DpZeAPVPX1GFl
         8NZs1HBtm4R/kspChFerHgiI5kxIMIQI8cmGPIZzSb/lhyUl2kqt7BOJfoIaAwNSec9N
         VXTQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707302803; x=1707907603;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Y4pT1WW3ucR/uqmeTEXzqlHGNfRbEHUD7g5eHjPYIV4=;
        b=u3EaWz5sEVvHRlG/fHhLLxtsJQrwqcKz8bJkak7X0UulLTP+Qd3zfnKZFRK9nHGIc9
         g/0+ACFGu6rUv3UUqMaSVFV853cRhC6Er8NCB2mxgGMScAOs1LxyDxmX5I1TSi7fJohW
         m13SouJnOB5vm26KY8ccKRYM03hM/NRHaOMQMiRWuJkTHoTyiv6KCxKjOjWP3Mvurf0+
         FF7U3a0kyXpPybTbkazu5/8c0K4TeARXmsYenByD16C2iZtDtpFZ+e37CWp2x3kJQ0NA
         h7apzKe19wKRQeIQ9x5lLjnlBVUkBIRu0pS2Iwy+CsCfA6sdBsfC5Xqqt6CyBOt7jePb
         Hslw==
X-Gm-Message-State: AOJu0YxIbUoSPKltVKKgXvdanHDvxdn8BjMOKQlHls5ebZBns6bFDa/i
	x+Q/Cm/51U1AfAGSodt7R057k5ruMYsnfE9dsKIMs44NdXj6wE5FEOOeXL1NKw==
X-Google-Smtp-Source: AGHT+IGakxpF+ru2UeXDCy+ErLka3tC+imQPfnr4mVsZBYV2BqatndNONa8vuVOJ24j2SwSI7ftQBw==
X-Received: by 2002:a05:600c:35ce:b0:40e:a3b8:a2b8 with SMTP id r14-20020a05600c35ce00b0040ea3b8a2b8mr4110513wmq.26.1707302803258;
        Wed, 07 Feb 2024 02:46:43 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVGgysI5ulDnic/9qaqBpeV8Hj/wDFy6L0afUxyp6wmRRgouSMYhb6QU5OYY227PIeRPKsJNwCL3U/RkzRpaA5Fq1VAbZV5n12y2PTtCeCDBMwKSTwTd/TtvcQLSTzTkTN0B5JsJjHIuJuD23RyVqQN2m2Qk8w+MWlXYTpm/A==
Message-ID: <513796cf-7735-49f2-8d70-59c4165e6482@suse.com>
Date: Wed, 7 Feb 2024 11:46:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Content-Language: en-US
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
References: <20240207103138.26901-1-jgross@suse.com>
 <20240207103831.dtvlyjtbmgz7kmll@begin>
 <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
Cc: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07.02.2024 11:42, Jürgen Groß wrote:
> On 07.02.24 11:38, Samuel Thibault wrote:
>> Juergen Gross, le mer. 07 févr. 2024 11:31:38 +0100, a ecrit:
>>> The .bss segment should be zeroed at very early boot.
>>
>> Is that not done by the elf loader of Xen?
> 
> It might be done by Xen tools today, but I'm quite sure it is not part
> of the ABI. The hypervisor doesn't clear .bss when loading a domain (e.g.
> dom0).

libelf's elf_memcpy() looks to say otherwise.


> I stumbled over it while implementing kexec in Mini-OS. For that I need
> it for sure.

Hmm, yes, if the protocol there doesn't guarantee zeroing. But then you
probably want to give this as the reason for the change?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:47:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:47:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677387.1053960 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfSH-0004sx-WD; Wed, 07 Feb 2024 10:47:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677387.1053960; Wed, 07 Feb 2024 10:47:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfSH-0004sq-RW; Wed, 07 Feb 2024 10:47:01 +0000
Received: by outflank-mailman (input) for mailman id 677387;
 Wed, 07 Feb 2024 10:47:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=D7mL=JQ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXfSH-0004qj-7u
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:47:01 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 38d5e0b1-c5a6-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 11:47:00 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 0C97122128;
 Wed,  7 Feb 2024 10:47:00 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id CA1EF13931;
 Wed,  7 Feb 2024 10:46:59 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 12siL6Nfw2WhZgAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 07 Feb 2024 10:46:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38d5e0b1-c5a6-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707302820; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=syNOOYn63MNzXJUUlPMc/v0Jky5g42FzSmxz26qDTeA=;
	b=gyZTAKwyYUgo7No2eWiMPiUulCCx2x23X49DJsQE6wKo4ICkuzuYChzwKc+CmUxyvSkkWP
	go014aPONzZSmgzHWX6lJAApl5G8QoEzZzWe0nvUZRJq2uPVk1htUXhR7NTfBqTcLFM9ok
	M4egoOdWjhnFpScWNhOnsw1fJBkCCBw=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707302820; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=syNOOYn63MNzXJUUlPMc/v0Jky5g42FzSmxz26qDTeA=;
	b=gyZTAKwyYUgo7No2eWiMPiUulCCx2x23X49DJsQE6wKo4ICkuzuYChzwKc+CmUxyvSkkWP
	go014aPONzZSmgzHWX6lJAApl5G8QoEzZzWe0nvUZRJq2uPVk1htUXhR7NTfBqTcLFM9ok
	M4egoOdWjhnFpScWNhOnsw1fJBkCCBw=
Message-ID: <13ac1a21-8275-4e7a-859d-41d8cb0aea76@suse.com>
Date: Wed, 7 Feb 2024 11:46:59 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: samuel.thibault@ens-lyon.org, wl@xen.org,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org
References: <20240207103138.26901-1-jgross@suse.com>
 <6ba32907-2d35-4dcf-ba3b-409984f1d877@suse.com>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <6ba32907-2d35-4dcf-ba3b-409984f1d877@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------KcaRW0tAFOLurK3V49GIDlTo"
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=gyZTAKwy
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-2.53 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 TO_DN_SOME(0.00)[];
	 HAS_ATTACHMENT(0.00)[];
	 MIME_BASE64_TEXT_BOGUS(1.00)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 MID_RHS_MATCH_FROM(0.00)[];
	 BAYES_HAM(-0.13)[67.81%];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 RCVD_TLS_ALL(0.00)[];
	 RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]
X-Spam-Score: -2.53
X-Rspamd-Queue-Id: 0C97122128
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------KcaRW0tAFOLurK3V49GIDlTo
Content-Type: multipart/mixed; boundary="------------07x00YrSXcKJQerA3Dr070Wc";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: samuel.thibault@ens-lyon.org, wl@xen.org,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org
Message-ID: <13ac1a21-8275-4e7a-859d-41d8cb0aea76@suse.com>
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
References: <20240207103138.26901-1-jgross@suse.com>
 <6ba32907-2d35-4dcf-ba3b-409984f1d877@suse.com>
In-Reply-To: <6ba32907-2d35-4dcf-ba3b-409984f1d877@suse.com>
Autocrypt-Gossip: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJ3BBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AAIQkQoDSui/t3IH4WIQQ+pJkfkcoLMCa4X6CgNK6L+3cgfgn7AJ9DmMd0SMJE
 ePbc7/m22D2v04iu7ACffXTdZQhNl557tJuDXZSBxDmW/tLOwU0EWTecRBAIAIK5OMKMU5R2
 Lk2bbjgX7vyQuCFFyKf9rC/4itNwhYWFSlKzVj3WJBDsoi2KvPm7AI+XB6NIkNAkshL5C0kd
 pcNd5Xo0jRR5/WE/bT7LyrJ0OJWS/qUit5eNNvsO+SxGAk28KRa1ieVLeZi9D03NL0+HIAtZ
 tecfqwgl3Y72UpLUyt+r7LQhcI/XR5IUUaD4C/chB4Vq2QkDKO7Q8+2HJOrFIjiVli4lU+Sf
 OBp64m//Y1xys++Z4ODoKh7tkh5DxiO3QBHG7bHK0CSQsJ6XUvPVYubAuy1XfSDzSeSBl//C
 v78Fclb+gi9GWidSTG/4hsEzd1fY5XwCZG/XJJY9M/sAAwUH/09Ar9W2U1Qm+DwZeP2ii3Ou
 14Z9VlVVPhcEmR/AFykL9dw/OV2O/7cdi52+l00reUu6Nd4Dl8s4f5n8b1YFzmkVVIyhwjvU
 jxtPyUgDOt6DRa+RaDlXZZmxQyWcMv2anAgYWGVszeB8Myzsw8y7xhBEVV1S+1KloCzw4V8Z
 DSJrcsZlyMDoiTb7FyqxwQnM0f6qHxWbmOOnbzJmBqpNpFuDcz/4xNsymJylm6oXiucHQBAP
 Xb/cE1YNHpuaH4SRhIxwQilCYEznWowQphNAbJtEKOmcocY7EbSt8VjXTzmYENkIfkrHRyXQ
 dUm5AoL51XZljkCqNwrADGkTvkwsWSvCSQQYEQIACQUCWTecRAIbDAAKCRCgNK6L+3cgfuef
 AJ9wlZQNQUp0KwEf8Tl37RmcxCL4bQCcC5alCSMzUBJ5DBIcR4BY+CyQFAs=

--------------07x00YrSXcKJQerA3Dr070Wc
Content-Type: multipart/mixed; boundary="------------OUCV5dBHO7VI1wWjNLvRs0Zi"

--------------OUCV5dBHO7VI1wWjNLvRs0Zi
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMDcuMDIuMjQgMTE6MzksIEphbiBCZXVsaWNoIHdyb3RlOg0KPiBPbiAwNy4wMi4yMDI0
IDExOjMxLCBKdWVyZ2VuIEdyb3NzIHdyb3RlOg0KPj4gLS0tIGEvYXJjaC94ODYvc2V0dXAu
Yw0KPj4gKysrIGIvYXJjaC94ODYvc2V0dXAuYw0KPj4gQEAgLTE4NCw2ICsxODQsOCBAQCBh
cmNoX2luaXQodm9pZCAqcGFyKQ0KPj4gICB7DQo+PiAgIAlzdGF0aWMgY2hhciBoZWxsb1td
ID0gIkJvb3RzdHJhcHBpbmcuLi5cbiI7DQo+PiAgIA0KPj4gKwltZW1zZXQoJl9fYnNzX3N0
YXJ0LCAwLCAmX2VuZCAtICZfX2Jzc19zdGFydCk7DQo+IA0KPiBEb2Vzbid0IC8gc2hvdWxk
bid0IHRoZSBsb2FkZXIgZ3VhcmFudGVlIHRoaXM/IEkgYXNrIGluIHBhcnRpY3VsYXINCj4g
YmVjYXVzZSBkb2luZyB0aGlzIGhlcmUgaW1wbGllcyBlYXJsaWVyIGFzc2VtYmx5IGNvZGUg
bWF5IG5vdCB3cml0ZQ0KPiB0byBhbnkgdmFyaWFibGUgaW4gLmJzcyAobm90aGluZyBsaWtl
IHRoYXQgbG9va3MgdG8gZXhpc3QgcmlnaHQgbm93LA0KPiBidXQgd2hvIGtub3dzIHdoYXQg
bWF5IGJlIGFkZGVkIGF0IHNvbWUgcG9pbnQpLg0KDQpPaCwgYW0gSSBtaXNyZW1lbWJlcmlu
ZyB0aGF0IHRoZSBoeXBlcnZpc29yIGRvZXNuJ3QgZG8gdGhlIGNsZWFyaW5nPw0KDQpJIGRv
IHJlbWVtYmVyIGEgYnVnIHdoZW4gZG9tMCBMaW51eCBjcmFzaGVkIGR1ZSB0byBhIG5vdCB6
ZXJvZWQgLmJzcywNCnNvIEkgYWRkZWQgdGhlIHplcm9pbmcgaW4gdGhlIHBhcmF2aXJ0IHN0
YXJ0dXAgY29kZSBvZiB0aGUga2VybmVsLg0KDQpCdXQgbWF5YmUgdGhpcyB3YXMgZHVlIHRv
IGEgYnVnIGluIHRoZSBsaW5rZXIgc2NyaXB0IG9mIHRoZSBrZXJuZWwuDQoNCj4gDQo+PiAt
LS0gYS9pbmNsdWRlL21tLmgNCj4+ICsrKyBiL2luY2x1ZGUvbW0uaA0KPj4gQEAgLTQ2LDYg
KzQ2LDggQEANCj4+ICAgI2RlZmluZSByb3VuZF9wZ2Rvd24oX3ApICAoKF9wKSAmIFBBR0Vf
TUFTSykNCj4+ICAgI2RlZmluZSByb3VuZF9wZ3VwKF9wKSAgICAoKChfcCkgKyAoUEFHRV9T
SVpFIC0gMSkpICYgUEFHRV9NQVNLKQ0KPj4gICANCj4+ICtleHRlcm4gY2hhciBfdGV4dCwg
X2V0ZXh0LCBfZXJvZGF0YSwgX2VkYXRhLCBfZW5kLCBfX2Jzc19zdGFydDsNCj4gDQo+IE1h
eWJlIHVzZSB0aGUgbW9yZSBjb252ZW50aW9uYWwgYXJyYXkgZm9ybT8gVGh1cyBhbHNvIGVs
aW1pbmF0aW5nDQo+IHRoZSBuZWVkIGZvciAmIGF0IHRoZSB1c2Ugc2l0ZXM/DQoNClRoaXMg
d2FzIGJhc2ljYWxseSBqdXN0IGEgbW92ZW1lbnQgb2YgY29kZSB0byBhbm90aGVyIGZpbGUu
DQoNCkJ1dCBJIGNhbiBtYWtlIHRoaXMgY2hhbmdlLCBvZiBjb3Vyc2UuDQoNCg0KSnVlcmdl
bg0K
--------------OUCV5dBHO7VI1wWjNLvRs0Zi
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------OUCV5dBHO7VI1wWjNLvRs0Zi--

--------------07x00YrSXcKJQerA3Dr070Wc--

--------------KcaRW0tAFOLurK3V49GIDlTo
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXDX6MFAwAAAAAACgkQsN6d1ii/Ey+o
cQf+N/RgQ3Hdbb1MlpBfjtaI+EpKovscJWRf7j3momineQ+3QJ2huuXj67239KIsIBfOf7jaoiPq
2jHO9Xc6fieap7mvIabsjy+MxxdrP/r9bnbO+DoxoLlcP9lPA14C8oA3H3T3c9fKm4WlfO9XkwWv
fHBywsUzk8rLpypx+7x4gYtgDC6CQpVyaz0rJAp+Dse7Sz3R+51+9s4mj08Jls76XLl2GFbjUdlV
VlsYZbuLQZixfGPzX/aeJUq1gcY+/vS/UlMAjJ+219DFYrhrSzUIxpa12byd+MotvMj4B4U0WhV2
ltfIfGw5aEMk3NDYMqW1XVvSKb+dWUjSS12Us789dw==
=SCnD
-----END PGP SIGNATURE-----

--------------KcaRW0tAFOLurK3V49GIDlTo--


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:47:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:47:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677399.1053976 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfSa-0005VD-C2; Wed, 07 Feb 2024 10:47:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677399.1053976; Wed, 07 Feb 2024 10:47:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfSa-0005Ux-6P; Wed, 07 Feb 2024 10:47:20 +0000
Received: by outflank-mailman (input) for mailman id 677399;
 Wed, 07 Feb 2024 10:47:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXfSZ-0004qj-3Y
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:47:19 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 43569342-c5a6-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 11:47:18 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33929364bdaso322093f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 02:47:18 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 g16-20020adfa490000000b0033b50ed5f98sm466412wrb.72.2024.02.07.02.47.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 02:47:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43569342-c5a6-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707302837; x=1707907637; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=a00hFHtSZDXw0sz/VU/iWI0xU2cVHoZxXzkr3nG0S8c=;
        b=vbf2NKPEiervcXgpU6BH74m4RvTpE+Mj/icnSK8H/0UMnq/5UQViFfGP189gQWsTRo
         B9RLR0RpTa42etIKBGoTOoIWpRiYLAUsAnbT/NGxnXD5ExDc6Z0N4T05VJovTM5Uv+LN
         IVrxTBjmwJipzSSKG7ZO7d57DKqP+HhgoQh2I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707302837; x=1707907637;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=a00hFHtSZDXw0sz/VU/iWI0xU2cVHoZxXzkr3nG0S8c=;
        b=mHAb+73XALgyddfZnMg3e7kLxscS8Ua+c6YKO2GW2GUJwnzlDpBt+6mpcHIFetGre0
         Hf6lOJ1gZgqYZIDLU1aki6ZQtLuyh97vJFb2tNf2Wa1PaKZeJZmdQEPPiZNwaCZqP96O
         xqsAlIeQmFve4avj+/E5TRtUVremBMnvUyP+3JVHQpoWlNS8V7CD1etM1O8W0fg1ee6Z
         keAXOGQK5HM3zJAzVxOssgIUOw39WR6QVsjRcgQfMIYW/8EVzzMCQdjzVHw/DNeODrJN
         P+hx0YxUcBS9ztUAjSvVwdLX+8lVjSRwBToG46wXqXAIM78V1fyz8Ekx1LI76R/FdEw3
         1nlw==
X-Forwarded-Encrypted: i=1; AJvYcCVzkcT6mtCj7Fv5iIZj94PfarpK36GuRM2BfSs6Y1ECDnmF+I+aJxNN+wO4lphZuNKQ0lub0Wu/k09yj5Y62k61L+8azQjHpo0ZsrP0nIc=
X-Gm-Message-State: AOJu0YwD/LrL0t1Jckw4F1Jw2KaAl1Mer6lH2vigY9UHEzgPMX3jaTNm
	TdcdTQf1tuildrHk7SnHsqU8+D/xIaN0mVsRbqlPOTjqEKK7QrTUs1LgXUg9BDA=
X-Google-Smtp-Source: AGHT+IFgUksnf7g8zE7c1vTAHnNDu2i6XvXlXhModzLjCbmR0u6WpAy3IRhJOTOL1HTQaD+8Yha0Zg==
X-Received: by 2002:adf:dd8b:0:b0:33b:305c:9de7 with SMTP id x11-20020adfdd8b000000b0033b305c9de7mr3298450wrl.50.1707302837539;
        Wed, 07 Feb 2024 02:47:17 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWX6AVB4x7jVJyovcK0RZrKzWIZ2SSBv0EQtB4oNLAlhJrfQEOF1SxPBQmHqvJpHvuo7io25WabeZeIKpTD0YlR+8v3S668i10o5dT/pFIbFV3pkk8zm505QqPl97neh7XPce8GeUuxPia67UydqJD6Ib7WrH+RKqSJ09yI3LbTUu4cbORFaRFSQtckYwEG/jSGXvYqqpRo2AR2Qg==
Message-ID: <7f2da5cc-eea3-4efb-9393-bd5e47686a1c@citrix.com>
Date: Wed, 7 Feb 2024 10:47:16 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Content-Language: en-GB
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
 Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org
References: <20240207103138.26901-1-jgross@suse.com>
 <20240207103831.dtvlyjtbmgz7kmll@begin>
 <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/02/2024 10:42 am, Jürgen Groß wrote:
> On 07.02.24 11:38, Samuel Thibault wrote:
>> Juergen Gross, le mer. 07 févr. 2024 11:31:38 +0100, a ecrit:
>>> The .bss segment should be zeroed at very early boot.
>>
>> Is that not done by the elf loader of Xen?
>
> It might be done by Xen tools today, but I'm quite sure it is not part
> of the ABI. The hypervisor doesn't clear .bss when loading a domain (e.g.
> dom0).

It is a requirement of loading an ELF file.

If we're not doing it for dom0, that's a bug.

> I stumbled over it while implementing kexec in Mini-OS. For that I need
> it for sure.

A kexec kernel does need to do this, unless the kexec-loader is going to
guarantee to do it, and I don't believe it does.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 10:48:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 10:48:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677415.1053986 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfTo-0006Ww-PX; Wed, 07 Feb 2024 10:48:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677415.1053986; Wed, 07 Feb 2024 10:48:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfTo-0006Wo-Lz; Wed, 07 Feb 2024 10:48:36 +0000
Received: by outflank-mailman (input) for mailman id 677415;
 Wed, 07 Feb 2024 10:48:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXfTn-0006WV-Fa
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 10:48:35 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 70d9e490-c5a6-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 11:48:34 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-40f02b8d176so3785465e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 02:48:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f8-20020a05600c154800b0040fb03f803esm1680514wmg.24.2024.02.07.02.48.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 02:48:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 70d9e490-c5a6-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707302914; x=1707907714; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=GXwRRf/nnbvG7KmptQ7/e+l/dIAGEq4cgunLwMG1JXk=;
        b=JC7p90hRYEno9M0B+BeO8924VyJq3wCXPQgqJaOrBqpM65WcWQSIXSBCN7U2c6DHjD
         v5DqH5HtqrIVBGwzN/nIfBc9wSNw0yQBQn/K1kSKet8xGjCdPMRXSD07P3bWnpEiKHTb
         Ck1LI8acgfluONIvvKaPcOG7ewwdRbxbB8U3mLhA3L5K3DUTTiM49WYXiyTp8SLzGCuo
         aTOtHFONEK5Jb8ne3QJ9i3Amo/RRJVehhK4IfNCCe687PPnaS233qYJfQnmSDxKZYqds
         MeNiTGC8sKHVlAn2WVgzZi1SU2CZVAjBLzyPH4FJu2hBkjbD9gb968q8xkVaStgSMaao
         pOuw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707302914; x=1707907714;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GXwRRf/nnbvG7KmptQ7/e+l/dIAGEq4cgunLwMG1JXk=;
        b=Vu5G5Mdco9Rfgsu3yqt/IucXUsqUdfa/0qCE9p6dYDhU0W0TJhC14pNJR7KNjT1eU1
         sKcSn7Lg1/3U2GikoSwEor8QXafPzxLrYJ9lNG5k5JQmfCT29Iw6IXf4naED+t6JwKlY
         fUuMBaFssX8i5mcaLDLRjUChFvJJPIktUrsNz74OxlClhUM/Ic8Op808cJTHhN9BMcLt
         8fh3Z9yh9xGTWcUa48Mp2bLQ5hvWT+hws/DqKA/flEv6Vc9HbjK2xY1owKD1TWdgu5lo
         6R4dnWX7QQo9Z5A1sYc7X5BEf7JKDUCj+7BF/IpFRPMuSSOZkwx+MTn6h6FpBniyafOX
         27XQ==
X-Gm-Message-State: AOJu0Ywrke34ghe0tKWiOIxE5eIQEDLdqm2bk8NKfYfaecSzwJEIy5TD
	PFk45AESkn4ib3sK6PjfP2ktjUWNnTVfot0zxeqKVmyPLewHvRyadhW15EgMbg==
X-Google-Smtp-Source: AGHT+IGiLc4QV7DVNoUGCkQbRF7166JsL6KNp8eQa8LrGBOEb2rI7WquPcxVG5PwJ1xSLI6U01XDXQ==
X-Received: by 2002:a05:600c:45d1:b0:40f:c1f9:c884 with SMTP id s17-20020a05600c45d100b0040fc1f9c884mr4176401wmo.39.1707302914004;
        Wed, 07 Feb 2024 02:48:34 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWoiWl2zL3zvjnz1eJumKb8cZBB3U9k4GIpGgLd+mcRrw9aQU5IK/qC9zjl5a0qGiQV8cjf5cHKd60bZzDzaw9SJgDP3ZpRrLa29YeF2gzJfrDFLuzrcwIL5NB5zv0toYJx0F+xOY3YkVp9wdYxHCRVf/HV+DCvR8hcNSX6xQ==
Message-ID: <161c2e1f-0c43-4391-bee3-1f94d27ba036@suse.com>
Date: Wed, 7 Feb 2024 11:48:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: samuel.thibault@ens-lyon.org, wl@xen.org,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org
References: <20240207103138.26901-1-jgross@suse.com>
 <6ba32907-2d35-4dcf-ba3b-409984f1d877@suse.com>
 <13ac1a21-8275-4e7a-859d-41d8cb0aea76@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <13ac1a21-8275-4e7a-859d-41d8cb0aea76@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 11:46, Juergen Gross wrote:
> On 07.02.24 11:39, Jan Beulich wrote:
>> On 07.02.2024 11:31, Juergen Gross wrote:
>>> --- a/arch/x86/setup.c
>>> +++ b/arch/x86/setup.c
>>> @@ -184,6 +184,8 @@ arch_init(void *par)
>>>   {
>>>   	static char hello[] = "Bootstrapping...\n";
>>>   
>>> +	memset(&__bss_start, 0, &_end - &__bss_start);
>>
>> Doesn't / shouldn't the loader guarantee this? I ask in particular
>> because doing this here implies earlier assembly code may not write
>> to any variable in .bss (nothing like that looks to exist right now,
>> but who knows what may be added at some point).
> 
> Oh, am I misremembering that the hypervisor doesn't do the clearing?
> 
> I do remember a bug when dom0 Linux crashed due to a not zeroed .bss,
> so I added the zeroing in the paravirt startup code of the kernel.
> 
> But maybe this was due to a bug in the linker script of the kernel.

That's what I recall.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 11:00:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 11:00:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677432.1054005 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXffF-0001pa-Si; Wed, 07 Feb 2024 11:00:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677432.1054005; Wed, 07 Feb 2024 11:00:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXffF-0001pJ-Pk; Wed, 07 Feb 2024 11:00:25 +0000
Received: by outflank-mailman (input) for mailman id 677432;
 Wed, 07 Feb 2024 11:00:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CAtM=JQ=ens-lyon.org=samuel.thibault@bounce.ens-lyon.org>)
 id 1rXffD-0001nR-Va
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 11:00:23 +0000
Received: from sonata.ens-lyon.org (domu-toccata.ens-lyon.fr [140.77.166.138])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 16abafeb-c5a8-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 12:00:22 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by sonata.ens-lyon.org (Postfix) with ESMTP id D8BAFA02E8;
 Wed,  7 Feb 2024 12:00:21 +0100 (CET)
Received: from sonata.ens-lyon.org ([127.0.0.1])
 by localhost (sonata.ens-lyon.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id L-0MvcTTUOE9; Wed,  7 Feb 2024 12:00:21 +0100 (CET)
Received: from begin (nat-inria-interne-52-gw-01-bso.bordeaux.inria.fr
 [194.199.1.52])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by sonata.ens-lyon.org (Postfix) with ESMTPSA id B1A25A02D4;
 Wed,  7 Feb 2024 12:00:21 +0100 (CET)
Received: from samy by begin with local (Exim 4.97)
 (envelope-from <samuel.thibault@ens-lyon.org>)
 id 1rXffB-00000008jiV-1XA6; Wed, 07 Feb 2024 12:00:21 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16abafeb-c5a8-11ee-8a49-1f161083a0e0
Date: Wed, 7 Feb 2024 12:00:21 +0100
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Message-ID: <20240207110021.hhlbkpx3vv3p7o4j@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	=?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
	minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
References: <20240207103138.26901-1-jgross@suse.com>
 <20240207103831.dtvlyjtbmgz7kmll@begin>
 <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Jürgen Groß, le mer. 07 févr. 2024 11:42:20 +0100, a ecrit:
> while implementing kexec in Mini-OS.

Oh, nice :D

> For that I need it for sure.

It needs to be done by kexec itself then.

Samuel


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 11:17:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 11:17:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677441.1054015 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfv7-00046h-4j; Wed, 07 Feb 2024 11:16:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677441.1054015; Wed, 07 Feb 2024 11:16:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXfv7-00046a-1i; Wed, 07 Feb 2024 11:16:49 +0000
Received: by outflank-mailman (input) for mailman id 677441;
 Wed, 07 Feb 2024 11:16:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=D7mL=JQ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXfv5-00046N-AC
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 11:16:47 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 613e749b-c5aa-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 12:16:46 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 8D6D82214F;
 Wed,  7 Feb 2024 11:16:45 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5C389139D8;
 Wed,  7 Feb 2024 11:16:45 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id k1CoFJ1mw2V8dgAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 07 Feb 2024 11:16:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 613e749b-c5aa-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707304605; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=3V4gMK0rL0P1KpLJ5eRgmUh4BUvAaFdYdsrl4ibkrNc=;
	b=Mwyq9qF6JvcfCxP53ihgIFU4zPtkfLt635qZ778pWC5izkC9iesDJJAzOoo+h5qvVbXO5X
	msvm0Rgro/TjBmOGQOqR0t+mcHSzaPmSMmnB9b2OSma5k9fbH8is7K4A85dicNRdDD5eIl
	YWKKSeeikKKyi/epYu9iLBt3lW8m1Ws=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707304605; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=3V4gMK0rL0P1KpLJ5eRgmUh4BUvAaFdYdsrl4ibkrNc=;
	b=Mwyq9qF6JvcfCxP53ihgIFU4zPtkfLt635qZ778pWC5izkC9iesDJJAzOoo+h5qvVbXO5X
	msvm0Rgro/TjBmOGQOqR0t+mcHSzaPmSMmnB9b2OSma5k9fbH8is7K4A85dicNRdDD5eIl
	YWKKSeeikKKyi/epYu9iLBt3lW8m1Ws=
Message-ID: <d0fc9a2a-e1e0-4cdf-8619-378a0c829bc6@suse.com>
Date: Wed, 7 Feb 2024 12:16:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Content-Language: en-US
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org
References: <20240207103138.26901-1-jgross@suse.com>
 <20240207103831.dtvlyjtbmgz7kmll@begin>
 <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
 <20240207110021.hhlbkpx3vv3p7o4j@begin>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240207110021.hhlbkpx3vv3p7o4j@begin>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=Mwyq9qF6
X-Spamd-Result: default: False [-2.10 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 BAYES_HAM(-2.57)[98.10%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.77)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 8D6D82214F
X-Spam-Level: 
X-Spam-Score: -2.10
X-Spam-Flag: NO

On 07.02.24 12:00, Samuel Thibault wrote:
> Jürgen Groß, le mer. 07 févr. 2024 11:42:20 +0100, a ecrit:
>> while implementing kexec in Mini-OS.
> 
> Oh, nice :D
> 
>> For that I need it for sure.
> 
> It needs to be done by kexec itself then.

That's another option, yes.

The question is whether we want to support to be kexec-ed from other
systems, too. Then I'd rather do it on both sides.


Juergen


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 11:35:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 11:35:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677453.1054031 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgCh-0007w3-MR; Wed, 07 Feb 2024 11:34:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677453.1054031; Wed, 07 Feb 2024 11:34:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgCh-0007vo-J1; Wed, 07 Feb 2024 11:34:59 +0000
Received: by outflank-mailman (input) for mailman id 677453;
 Wed, 07 Feb 2024 11:34:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CAtM=JQ=ens-lyon.org=samuel.thibault@bounce.ens-lyon.org>)
 id 1rXgCh-0007vO-8Z
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 11:34:59 +0000
Received: from sonata.ens-lyon.org (sonata.ens-lyon.org [140.77.166.138])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id eb89f3a2-c5ac-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 12:34:57 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by sonata.ens-lyon.org (Postfix) with ESMTP id 031C9A00F3;
 Wed,  7 Feb 2024 12:34:57 +0100 (CET)
Received: from sonata.ens-lyon.org ([127.0.0.1])
 by localhost (sonata.ens-lyon.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id rmGv8KW6WZiz; Wed,  7 Feb 2024 12:34:56 +0100 (CET)
Received: from begin (nat-inria-interne-52-gw-01-bso.bordeaux.inria.fr
 [194.199.1.52])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by sonata.ens-lyon.org (Postfix) with ESMTPSA id BBE4FA00DE;
 Wed,  7 Feb 2024 12:34:56 +0100 (CET)
Received: from samy by begin with local (Exim 4.97)
 (envelope-from <samuel.thibault@ens-lyon.org>)
 id 1rXgCe-00000008k9i-1YKC; Wed, 07 Feb 2024 12:34:56 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb89f3a2-c5ac-11ee-8a49-1f161083a0e0
Date: Wed, 7 Feb 2024 12:34:56 +0100
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Message-ID: <20240207113456.2pxcrcazxseznyjq@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	=?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
	minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
References: <20240207103138.26901-1-jgross@suse.com>
 <20240207103831.dtvlyjtbmgz7kmll@begin>
 <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
 <20240207110021.hhlbkpx3vv3p7o4j@begin>
 <d0fc9a2a-e1e0-4cdf-8619-378a0c829bc6@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <d0fc9a2a-e1e0-4cdf-8619-378a0c829bc6@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Jürgen Groß, le mer. 07 févr. 2024 12:16:44 +0100, a ecrit:
> On 07.02.24 12:00, Samuel Thibault wrote:
> > Jürgen Groß, le mer. 07 févr. 2024 11:42:20 +0100, a ecrit:
> > > while implementing kexec in Mini-OS.
> > 
> > Oh, nice :D
> > 
> > > For that I need it for sure.
> > 
> > It needs to be done by kexec itself then.
> 
> That's another option, yes.
> 
> The question is whether we want to support to be kexec-ed from other
> systems, too.

But aren't other systems' kexec supports supposed to do the memset?

They really should.

Samuel


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 11:43:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 11:43:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677463.1054046 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgKc-0001gP-Ip; Wed, 07 Feb 2024 11:43:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677463.1054046; Wed, 07 Feb 2024 11:43:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgKc-0001gI-GE; Wed, 07 Feb 2024 11:43:10 +0000
Received: by outflank-mailman (input) for mailman id 677463;
 Wed, 07 Feb 2024 11:43:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=D7mL=JQ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXgKb-0001d2-23
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 11:43:09 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0de17bcc-c5ae-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 12:43:04 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 421E8221BF;
 Wed,  7 Feb 2024 11:43:04 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1D8B6139D8;
 Wed,  7 Feb 2024 11:43:04 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id bo/VBchsw2UgfgAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 07 Feb 2024 11:43:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0de17bcc-c5ae-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707306184; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xGzSiKPaIOOGKNiGPGbMuq7bs0z9asbp2mQfLWN4l70=;
	b=Oy+aD0CP4xZLDumskl12HgZSMMhT6xHP/M+DcRhgewNQCl0Id6Il0vFP8cwiJmuc+CuZSB
	wLHXeJkUEvrmNzY7zP1hw9hamCjiqNt8gQgopK6sxSdrokaKoAvkkkxAlPh1L0i3LbuDzm
	KR8tIjiM3cQLlRfL+Bm74dlF/ncTy9Q=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707306184; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xGzSiKPaIOOGKNiGPGbMuq7bs0z9asbp2mQfLWN4l70=;
	b=Oy+aD0CP4xZLDumskl12HgZSMMhT6xHP/M+DcRhgewNQCl0Id6Il0vFP8cwiJmuc+CuZSB
	wLHXeJkUEvrmNzY7zP1hw9hamCjiqNt8gQgopK6sxSdrokaKoAvkkkxAlPh1L0i3LbuDzm
	KR8tIjiM3cQLlRfL+Bm74dlF/ncTy9Q=
Message-ID: <f538b855-f2bb-4881-bd8a-eb8988579c85@suse.com>
Date: Wed, 7 Feb 2024 12:43:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Content-Language: en-US
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org
References: <20240207103138.26901-1-jgross@suse.com>
 <20240207103831.dtvlyjtbmgz7kmll@begin>
 <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
 <20240207110021.hhlbkpx3vv3p7o4j@begin>
 <d0fc9a2a-e1e0-4cdf-8619-378a0c829bc6@suse.com>
 <20240207113456.2pxcrcazxseznyjq@begin>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240207113456.2pxcrcazxseznyjq@begin>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -0.62
X-Spamd-Result: default: False [-0.62 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 BAYES_HAM(-0.10)[64.95%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.77)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Flag: NO

On 07.02.24 12:34, Samuel Thibault wrote:
> Jürgen Groß, le mer. 07 févr. 2024 12:16:44 +0100, a ecrit:
>> On 07.02.24 12:00, Samuel Thibault wrote:
>>> Jürgen Groß, le mer. 07 févr. 2024 11:42:20 +0100, a ecrit:
>>>> while implementing kexec in Mini-OS.
>>>
>>> Oh, nice :D
>>>
>>>> For that I need it for sure.
>>>
>>> It needs to be done by kexec itself then.
>>
>> That's another option, yes.
>>
>> The question is whether we want to support to be kexec-ed from other
>> systems, too.
> 
> But aren't other systems' kexec supports supposed to do the memset?
> 
> They really should.

I guess there is a reason why the Linux kernel does clear its .bss section
in early boot. Maybe it is due to how the boot process works (the ELF file
is encapsulated in vmlinuz), but following your reasoning they should have
cleared their .bss while unpacking the ELF contents, not while booting the
contents.

I'm not sure they do the .bss clearing in kexec either, as the kernel is
clearing it anyway.


Juergen


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 11:52:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 11:52:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677474.1054057 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgTw-0003nG-G8; Wed, 07 Feb 2024 11:52:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677474.1054057; Wed, 07 Feb 2024 11:52:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgTw-0003n9-Bt; Wed, 07 Feb 2024 11:52:48 +0000
Received: by outflank-mailman (input) for mailman id 677474;
 Wed, 07 Feb 2024 11:52:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=beHI=JQ=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rXgTv-0003n3-3v
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 11:52:47 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 67f59b53-c5af-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 12:52:45 +0100 (CET)
Received: from SA9PR13CA0173.namprd13.prod.outlook.com (2603:10b6:806:28::28)
 by BN9PR12MB5321.namprd12.prod.outlook.com (2603:10b6:408:102::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.17; Wed, 7 Feb
 2024 11:52:42 +0000
Received: from SN1PEPF00026368.namprd02.prod.outlook.com
 (2603:10b6:806:28:cafe::f) by SA9PR13CA0173.outlook.office365.com
 (2603:10b6:806:28::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.23 via Frontend
 Transport; Wed, 7 Feb 2024 11:52:41 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SN1PEPF00026368.mail.protection.outlook.com (10.167.241.133) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Wed, 7 Feb 2024 11:52:40 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Wed, 7 Feb
 2024 05:52:38 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Wed, 7 Feb 2024 05:52:37 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67f59b53-c5af-11ee-8a49-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SAZzAqUh+Tg76QtgGyCbHDGAO3okMNFfH7vaLPdUhYBba8rA2lQcZdpFzqMlHwoLjJwMddKuDV7Z1fnM1h0639Tf10TlZr/KXxqvkuPSJDq2XRW+YVXXNFth3vq755wuFs2rUSQ45w/DTQt0FcFqCcDYmjyfJ7HmfCd9QZ07DQjUAvwTZAduYzKJwZwBr7/LuXWn9FTzBNCiCMMS2QsKxNoNnAd6YusEVrh1nGl6lgqjKWaQjOvXeE14Gd5UvRZZo7oXq223hS5bxAab+x+1ElyAuZ5aTSb3pCjcaQOe9655FVVfJLWJ0Eg/IugmJQv/WqFoN7G/ic5nqOBX70kK6w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=HR5jjJqAJ2mvwGBUKuh+NtUs3U0/RVL3mMDNnR++9p8=;
 b=gSrp4j+uZQHUCKbCXpBGLjD7ed/OgM2fN0eeW3Omuhe3hs9x2xyGYkXMrW9NtDbl0YGUzMZVREdU6PzsLWZHFRKyOSK92trejwYNQdLbp6iaqgiJdhKnb5b3izUq3EDeMYu7t6m16lWItkEnutFzn7HZ8HmSftvZvLDzFREzgIXkAqT7BwHun5SaXteZu4eVmgcStCTkbQquWMqW03vzmKX8rxP3y20yHyO2/DLu1SGzTcUBI4wImeVPHCi0nSklyinQTOqJL8xWGbmKTkrhd2T8vMznwz9FGDKFqqhCAeJKsJ1iPg2IGDM9ZxMsDByoIk7Nryi5HhYfuW9zCKe2JA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=HR5jjJqAJ2mvwGBUKuh+NtUs3U0/RVL3mMDNnR++9p8=;
 b=KNGKhF50F2aH/rP6Pt3n40HhpwOPxLY7DVqjtx2cEfX9uUP4FFR8tEIXMkBGPAkxLYNg2hF0zML9tLqhBev7apzVJkmwHu9TzfZ9ohFMXDlACmsiZ5i5H1lB7ATJ0irdHwg1A1vL6kPGC70BB7wmZKCkmfuklFplVsnKtft21vU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <228a977a-c644-4ccb-b9aa-a25171d8605d@amd.com>
Date: Wed, 7 Feb 2024 12:52:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v4 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
Content-Language: en-US
To: Julien Grall <julien@xen.org>, Ayan Kumar Halder
	<ayan.kumar.halder@amd.com>, <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <stefano.stabellini@amd.com>,
	<Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>,
	<luca.fancellu@arm.com>
References: <20240131121049.225044-1-ayan.kumar.halder@amd.com>
 <20240131121049.225044-2-ayan.kumar.halder@amd.com>
 <c7841d11-d072-4787-9d76-ee94123605bb@xen.org>
 <8daa2461-092f-48ad-8556-3bba398ae544@amd.com>
 <e3a967fb-f9c5-495f-a30a-1826e3673c95@xen.org>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <e3a967fb-f9c5-495f-a30a-1826e3673c95@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF00026368:EE_|BN9PR12MB5321:EE_
X-MS-Office365-Filtering-Correlation-Id: ccbc4e49-25f5-424a-a4c3-08dc27d349f5
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	NP1UhHWz+jQlGfZr7pdT/8u9gkh31mDcQj+xP6Mqqg2kdlHCiP3VxP/lweQ9fx0e3ZniI6W+uu4TzdzNf5mtWF0aNBt1LScG0GT35UEVB5Sm3RvlBz+ulPCGg+WwCDjZVrTORlS3+V3/kWqg91FqpVD9ARHZWZQBUS/LTkkPrEcCbTxUiMtQYphuH0LDdrHM6qwmepnAT32B09wLYeVqCHXAmlC3pd6p/h0DlD/6nUiNMef8tJNnhKRrCJYBIpZWODcVMWsrkuJzrGBvZUrf4gM9188xrHd4UpLSqzUcPzQ3SZNW270hTHDunybZ7weP3p1jbcNFTgwOkSkrXrmM1fgvyrfr8YvtOFM66jTfphC59y1AwjQ/Ext3paaXES2j71yiZBvy0sUUmpTBUqbniq467g5XwWM0yOvpPXRd7LO3K4c9QCnsR4XR3qG28ZLVk6sHJW/mNfTgBAn62NBjY6n8YoOV2HBTzGVYCyUgrC5QpkJ8Lr1jk6V6xgNZOjlyckfUzYWNPZF0YmuimONgjGDIxOhT+UpOh7Spvl5YDshDIp9b3QWjzq+ZApmadLnUQS0l3U0PKJSGQLZyF9YJR5JdmYLQxFfFwvDcipUMAb7MI5GhgVuRYyOtctObiKEEHbrlWOXimjJ9S45Xu/ctH+CtAwM3wFR+PgR1mE7DLjv2YpCEajfsjoOgMOmB8/6bZnvoVKHHSJGfQ9OFIE5Z96P1tMKNF2OLnt3CLAEMaKoTjfJBw7M/fR4HV6MiY7EC62Lai/oON2tVMHYRbr5RkilZGpLW/S5ggQtopgQdpx8w92haYzXSK4Mt3GQZh1AInX1lYCXqYnjBkvcvI/hYHg==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(376002)(136003)(346002)(396003)(230922051799003)(82310400011)(186009)(64100799003)(451199024)(1800799012)(36840700001)(40470700004)(46966006)(40480700001)(31686004)(40460700003)(41300700001)(478600001)(53546011)(86362001)(36756003)(966005)(31696002)(426003)(356005)(5660300002)(81166007)(336012)(2616005)(47076005)(82740400003)(83380400001)(316002)(26005)(8936002)(36860700001)(70206006)(70586007)(44832011)(110136005)(54906003)(2906002)(16576012)(8676002)(4326008)(43740500002)(36900700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Feb 2024 11:52:40.9530
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ccbc4e49-25f5-424a-a4c3-08dc27d349f5
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF00026368.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5321



On 07/02/2024 11:06, Julien Grall wrote:
> 
> 
> Hi Michal,
> 
> On 07/02/2024 07:45, Michal Orzel wrote:
>> On 06/02/2024 19:49, Julien Grall wrote:
>>> On 31/01/2024 12:10, Ayan Kumar Halder wrote:
>>>> There can be situations when the registers cannot be emulated to their full
>>>> functionality. This can be due to the complexity involved. In such cases, one
>>>> can emulate those registers as RAZ/WI for example. We call them as partial
>>>> emulation.
>>>>
>>>> Some registers are non-optional and as such there is nothing preventing an OS
>>>> from accessing them.
>>>> Instead of injecting undefined exception (thus crashing a guest), one may want
>>>> to prefer a partial emulation to let the guest running (in some cases accepting
>>>> the fact that it might result in unwanted behavior).
>>>>
>>>> A suitable example of this (as seen in subsequent patches) is emulation of
>>>> DBGDTRTX_EL0 (on Arm64) and DBGDTRTXINT(on Arm32). These non-optional
>>>> registers can be emulated as RAZ/WI and they can be enclosed within
>>>> CONFIG_PARTIAL_EMULATION.
>>>>
>>>> Further, "partial-emulation" command line option allows us to
>>>> enable/disable partial emulation at run time. While CONFIG_PARTIAL_EMULATION
>>>> enables support for partial emulation at compile time (i.e. adds code for
>>>> partial emulation), this option may be enabled or disabled by Yocto or other
>>>> build systems. However if the build system turns this option on, users
>>>> can use scripts like Imagebuilder to generate uboot-script which will append
>>>> "partial-emulation=false" to xen command line to turn off the partial
>>>> emulation. Thus, it helps to avoid rebuilding xen.
>>>>
>>>> By default, "CONFIG_PARTIAL_EMULATION=y" and "partial-emulation=false".
>>>> This is done so that Xen supports partial emulation. However, customers are
>>>> fully aware when they enable partial emulation. It's important to note that
>>>> enabling such support might result in unwanted/non-spec compliant behavior.
>>>
>>> Can you remind me why this is built by default? In particular...
>> This is the result of RFC discussion we had, where both Bertrand and Stefano agreed on having
>> the Kconfig enabled by default to improve user experience:
>> Bertrand:
>> https://lore.kernel.org/xen-devel/C0ADC33B-1966-4D3E-B081-A3AA0C3AE76D@arm.com/
>> Stefano:
>> https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2312081514450.1703076@ubuntu-linux-20-04-desktop/
> 
> Thanks for the pointer. I thought a bit more and per-se the default of
> the Kconfig doesn't really matter too much. So I am fine to keep it on
> by default.
> 
> That said, I think we need to detail the security support for the
> command line in SUPPORT.md. I think we want to consider to not security
> support any issue that would allow the userland to attack the guest OS
> due to a bug in the partial emulation.
> 
> I would be fine with security supporting any issue that would
> DoS/compromise Xen.
Sounds good to me. Something like:
### ARM/Partial emulation

Enable partial emulation of registers, otherwise considered unimplemented,
that would normally trigger a fault injection.

    Status: Supported, with caveats

Bugs allowing the userspace to attack the guest OS will not be considered
security vulnerabilities.

Bugs that could compromise Xen will be considered security vulnerabilities.

~Michal


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 12:02:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 12:02:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677481.1054067 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgdP-0005wB-Gt; Wed, 07 Feb 2024 12:02:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677481.1054067; Wed, 07 Feb 2024 12:02:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgdP-0005w4-DO; Wed, 07 Feb 2024 12:02:35 +0000
Received: by outflank-mailman (input) for mailman id 677481;
 Wed, 07 Feb 2024 12:02:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CAtM=JQ=ens-lyon.org=samuel.thibault@bounce.ens-lyon.org>)
 id 1rXgdO-0005vm-30
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 12:02:34 +0000
Received: from sonata.ens-lyon.org (sonata.ens-lyon.org [140.77.166.138])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c54b2746-c5b0-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 13:02:31 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by sonata.ens-lyon.org (Postfix) with ESMTP id CF0F9A02F5;
 Wed,  7 Feb 2024 13:02:30 +0100 (CET)
Received: from sonata.ens-lyon.org ([127.0.0.1])
 by localhost (sonata.ens-lyon.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id Ca9USYTn4aKJ; Wed,  7 Feb 2024 13:02:30 +0100 (CET)
Received: from begin (nat-inria-interne-52-gw-01-bso.bordeaux.inria.fr
 [194.199.1.52])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by sonata.ens-lyon.org (Postfix) with ESMTPSA id 9C35BA02D4;
 Wed,  7 Feb 2024 13:02:30 +0100 (CET)
Received: from samy by begin with local (Exim 4.97)
 (envelope-from <samuel.thibault@ens-lyon.org>)
 id 1rXgdK-00000008lCO-162I; Wed, 07 Feb 2024 13:02:30 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c54b2746-c5b0-11ee-98f5-efadbce2ee36
Date: Wed, 7 Feb 2024 13:02:30 +0100
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Message-ID: <20240207120230.elgawtnwmftcdqr4@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	=?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>,
	minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
References: <20240207103138.26901-1-jgross@suse.com>
 <20240207103831.dtvlyjtbmgz7kmll@begin>
 <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
 <20240207110021.hhlbkpx3vv3p7o4j@begin>
 <d0fc9a2a-e1e0-4cdf-8619-378a0c829bc6@suse.com>
 <20240207113456.2pxcrcazxseznyjq@begin>
 <f538b855-f2bb-4881-bd8a-eb8988579c85@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <f538b855-f2bb-4881-bd8a-eb8988579c85@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Jürgen Groß, le mer. 07 févr. 2024 12:43:03 +0100, a ecrit:
> On 07.02.24 12:34, Samuel Thibault wrote:
> > Jürgen Groß, le mer. 07 févr. 2024 12:16:44 +0100, a ecrit:
> > > On 07.02.24 12:00, Samuel Thibault wrote:
> > > > Jürgen Groß, le mer. 07 févr. 2024 11:42:20 +0100, a ecrit:
> > > > > while implementing kexec in Mini-OS.
> > > > 
> > > > Oh, nice :D
> > > > 
> > > > > For that I need it for sure.
> > > > 
> > > > It needs to be done by kexec itself then.
> > > 
> > > That's another option, yes.
> > > 
> > > The question is whether we want to support to be kexec-ed from other
> > > systems, too.
> > 
> > But aren't other systems' kexec supports supposed to do the memset?
> > 
> > They really should.
> 
> I guess there is a reason why the Linux kernel does clear its .bss section
> in early boot. Maybe it is due to how the boot process works (the ELF file
> is encapsulated in vmlinuz),

Yes, the unpack prevents grub etc. from doing it.

> but following your reasoning they should have cleared their .bss while
> unpacking the ELF contents, not while booting the contents.

AIUI the decompressor itself doesn't actually know about ELF.
decompress_kernel() does call handle_relocations(), but it should indeed
clear bss itself and not leave it out to assembly indeed.

> I'm not sure they do the .bss clearing in kexec either,

AIUI they do, see kimage_load_normal_segment() which clears the page
before possibly loading some file piece into it.

Really, this is part of what "loading an ELF" means.

Samuel


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 12:07:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 12:07:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677490.1054081 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgiV-0006eJ-4B; Wed, 07 Feb 2024 12:07:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677490.1054081; Wed, 07 Feb 2024 12:07:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgiV-0006eC-19; Wed, 07 Feb 2024 12:07:51 +0000
Received: by outflank-mailman (input) for mailman id 677490;
 Wed, 07 Feb 2024 12:07:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rXgiT-0006e4-Sy
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 12:07:49 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rXgiT-0000dw-CY; Wed, 07 Feb 2024 12:07:49 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rXgiT-0002et-4Z; Wed, 07 Feb 2024 12:07:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=20KOmuWek69cldepG0n/tGQ1GtEH7espjH0QHeKzDR0=; b=lCfSdcvBGaQmgv/OmzLgcXAwm3
	BwmhkbGnAu0fc6PX1lV7V92uuECyq2ExhlERCGPQ/oBBvV0L4FxVLM2s7BuOJ1+l+545JxyyHY6QR
	KVuWavZI9stM1cFrxMfSpJy3FJf9WhfrYehE1Ha1g5q+Kc4tKSTX0uem+g6XqxUlrBKQ=;
Message-ID: <67a1ab3a-1777-4c80-9fb1-95e547d286e9@xen.org>
Date: Wed, 7 Feb 2024 12:07:46 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v4 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
Content-Language: en-GB
To: Michal Orzel <michal.orzel@amd.com>,
 Ayan Kumar Halder <ayan.kumar.halder@amd.com>, xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, luca.fancellu@arm.com
References: <20240131121049.225044-1-ayan.kumar.halder@amd.com>
 <20240131121049.225044-2-ayan.kumar.halder@amd.com>
 <c7841d11-d072-4787-9d76-ee94123605bb@xen.org>
 <8daa2461-092f-48ad-8556-3bba398ae544@amd.com>
 <e3a967fb-f9c5-495f-a30a-1826e3673c95@xen.org>
 <228a977a-c644-4ccb-b9aa-a25171d8605d@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <228a977a-c644-4ccb-b9aa-a25171d8605d@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Michal,

On 07/02/2024 11:52, Michal Orzel wrote:
> 
> 
> On 07/02/2024 11:06, Julien Grall wrote:
>>
>>
>> Hi Michal,
>>
>> On 07/02/2024 07:45, Michal Orzel wrote:
>>> On 06/02/2024 19:49, Julien Grall wrote:
>>>> On 31/01/2024 12:10, Ayan Kumar Halder wrote:
>>>>> There can be situations when the registers cannot be emulated to their full
>>>>> functionality. This can be due to the complexity involved. In such cases, one
>>>>> can emulate those registers as RAZ/WI for example. We call them as partial
>>>>> emulation.
>>>>>
>>>>> Some registers are non-optional and as such there is nothing preventing an OS
>>>>> from accessing them.
>>>>> Instead of injecting undefined exception (thus crashing a guest), one may want
>>>>> to prefer a partial emulation to let the guest running (in some cases accepting
>>>>> the fact that it might result in unwanted behavior).
>>>>>
>>>>> A suitable example of this (as seen in subsequent patches) is emulation of
>>>>> DBGDTRTX_EL0 (on Arm64) and DBGDTRTXINT(on Arm32). These non-optional
>>>>> registers can be emulated as RAZ/WI and they can be enclosed within
>>>>> CONFIG_PARTIAL_EMULATION.
>>>>>
>>>>> Further, "partial-emulation" command line option allows us to
>>>>> enable/disable partial emulation at run time. While CONFIG_PARTIAL_EMULATION
>>>>> enables support for partial emulation at compile time (i.e. adds code for
>>>>> partial emulation), this option may be enabled or disabled by Yocto or other
>>>>> build systems. However if the build system turns this option on, users
>>>>> can use scripts like Imagebuilder to generate uboot-script which will append
>>>>> "partial-emulation=false" to xen command line to turn off the partial
>>>>> emulation. Thus, it helps to avoid rebuilding xen.
>>>>>
>>>>> By default, "CONFIG_PARTIAL_EMULATION=y" and "partial-emulation=false".
>>>>> This is done so that Xen supports partial emulation. However, customers are
>>>>> fully aware when they enable partial emulation. It's important to note that
>>>>> enabling such support might result in unwanted/non-spec compliant behavior.
>>>>
>>>> Can you remind me why this is built by default? In particular...
>>> This is the result of RFC discussion we had, where both Bertrand and Stefano agreed on having
>>> the Kconfig enabled by default to improve user experience:
>>> Bertrand:
>>> https://lore.kernel.org/xen-devel/C0ADC33B-1966-4D3E-B081-A3AA0C3AE76D@arm.com/
>>> Stefano:
>>> https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2312081514450.1703076@ubuntu-linux-20-04-desktop/
>>
>> Thanks for the pointer. I thought a bit more and per-se the default of
>> the Kconfig doesn't really matter too much. So I am fine to keep it on
>> by default.
>>
>> That said, I think we need to detail the security support for the
>> command line in SUPPORT.md. I think we want to consider to not security
>> support any issue that would allow the userland to attack the guest OS
>> due to a bug in the partial emulation.
>>
>> I would be fine with security supporting any issue that would
>> DoS/compromise Xen.
> Sounds good to me. Something like:
> ### ARM/Partial emulation
> 
> Enable partial emulation of registers, otherwise considered unimplemented,
> that would normally trigger a fault injection.
> 
>      Status: Supported, with caveats
> 
> Bugs allowing the userspace to attack the guest OS will not be considered
> security vulnerabilities.
> 
> Bugs that could compromise Xen will be considered security vulnerabilities.

LGTM.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 12:12:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 12:12:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677496.1054091 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgmq-0008FV-L0; Wed, 07 Feb 2024 12:12:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677496.1054091; Wed, 07 Feb 2024 12:12:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgmq-0008FO-H8; Wed, 07 Feb 2024 12:12:20 +0000
Received: by outflank-mailman (input) for mailman id 677496;
 Wed, 07 Feb 2024 12:12:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=D7mL=JQ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXgmp-0008FD-JU
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 12:12:19 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 23664dc9-c5b2-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 13:12:18 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id D83A5220E0;
 Wed,  7 Feb 2024 12:12:17 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B242C13931;
 Wed,  7 Feb 2024 12:12:17 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id ekTuKaFzw2VaBwAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 07 Feb 2024 12:12:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23664dc9-c5b2-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707307938; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mwOSzPmFc0MyA3yvn93iNKUXA2xbWuVuWVf9ZdgYwbo=;
	b=g0i73Y6bw0VSiU7ghAJnBTULN99GGpl1JT9BZuuSI5sGuWXUC07bBTj8XuOeV44UaP7tEO
	Hzti6N1rE4zimfmL5gWO6p6LYiNIvJTzEUGM1t4Su4VGTxpMAVUucREn+Pe2VppkxDVU8G
	xvm11eaHaNTpuxXb0kAvjhShG7yPwdA=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707307937; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mwOSzPmFc0MyA3yvn93iNKUXA2xbWuVuWVf9ZdgYwbo=;
	b=ol1Scw0J9VSS99tUYSukEKpRJSDoPty/qHTu/rEWyXdycdJun1QgrDiOzGoKy3cWXSeFZz
	DykcbRzooIvlCTtFPKPB2XViFeU6+tns8zkPcF+HmtWwIfBjmcMFGWGa3YIUH2Jls/pxfn
	mo6PXUGu1QnkrI2hVKZT7oQEQ2c3pk8=
Message-ID: <64ef988e-cadf-4fd3-8927-db882c0f4185@suse.com>
Date: Wed, 7 Feb 2024 13:12:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: x86: zero out .bss segment at boot
Content-Language: en-US
To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org, wl@xen.org
References: <20240207103138.26901-1-jgross@suse.com>
 <20240207103831.dtvlyjtbmgz7kmll@begin>
 <d51d842e-5609-4c98-9a31-8d6a0b237c31@suse.com>
 <20240207110021.hhlbkpx3vv3p7o4j@begin>
 <d0fc9a2a-e1e0-4cdf-8619-378a0c829bc6@suse.com>
 <20240207113456.2pxcrcazxseznyjq@begin>
 <f538b855-f2bb-4881-bd8a-eb8988579c85@suse.com>
 <20240207120230.elgawtnwmftcdqr4@begin>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240207120230.elgawtnwmftcdqr4@begin>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=ol1Scw0J
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-2.76 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 TO_DN_SOME(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.77)[subject];
	 MID_RHS_MATCH_FROM(0.00)[];
	 BAYES_HAM(-0.53)[80.53%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 RCVD_DKIM_ARC_DNSWL_HI(-1.00)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 MIME_GOOD(-0.10)[text/plain];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 RCVD_IN_DNSWL_HI(-0.50)[2a07:de40:b281:104:10:150:64:97:from];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: -2.76
X-Rspamd-Queue-Id: D83A5220E0
X-Spam-Flag: NO

On 07.02.24 13:02, Samuel Thibault wrote:
> Jürgen Groß, le mer. 07 févr. 2024 12:43:03 +0100, a ecrit:
>> On 07.02.24 12:34, Samuel Thibault wrote:
>>> Jürgen Groß, le mer. 07 févr. 2024 12:16:44 +0100, a ecrit:
>>>> On 07.02.24 12:00, Samuel Thibault wrote:
>>>>> Jürgen Groß, le mer. 07 févr. 2024 11:42:20 +0100, a ecrit:
>>>>>> while implementing kexec in Mini-OS.
>>>>>
>>>>> Oh, nice :D
>>>>>
>>>>>> For that I need it for sure.
>>>>>
>>>>> It needs to be done by kexec itself then.
>>>>
>>>> That's another option, yes.
>>>>
>>>> The question is whether we want to support to be kexec-ed from other
>>>> systems, too.
>>>
>>> But aren't other systems' kexec supports supposed to do the memset?
>>>
>>> They really should.
>>
>> I guess there is a reason why the Linux kernel does clear its .bss section
>> in early boot. Maybe it is due to how the boot process works (the ELF file
>> is encapsulated in vmlinuz),
> 
> Yes, the unpack prevents grub etc. from doing it.
> 
>> but following your reasoning they should have cleared their .bss while
>> unpacking the ELF contents, not while booting the contents.
> 
> AIUI the decompressor itself doesn't actually know about ELF.
> decompress_kernel() does call handle_relocations(), but it should indeed
> clear bss itself and not leave it out to assembly indeed.
> 
>> I'm not sure they do the .bss clearing in kexec either,
> 
> AIUI they do, see kimage_load_normal_segment() which clears the page
> before possibly loading some file piece into it.
> 
> Really, this is part of what "loading an ELF" means.

Okay, I'll do the clearing only in the kexec code then.


Juergen


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 12:21:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 12:21:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677507.1054105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgvW-0002C1-EV; Wed, 07 Feb 2024 12:21:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677507.1054105; Wed, 07 Feb 2024 12:21:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXgvW-0002Bu-BV; Wed, 07 Feb 2024 12:21:18 +0000
Received: by outflank-mailman (input) for mailman id 677507;
 Wed, 07 Feb 2024 12:21:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nQwS=JQ=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rXgvU-0002Bo-Vc
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 12:21:16 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 62f8da3e-c5b3-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 13:21:14 +0100 (CET)
Received: from [192.168.1.140] (unknown [81.56.166.244])
 by support.bugseng.com (Postfix) with ESMTPSA id DB1734EE0738;
 Wed,  7 Feb 2024 13:21:13 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62f8da3e-c5b3-11ee-98f5-efadbce2ee36
Message-ID: <e48f4d49-274c-4bd6-8e3b-1ecaee6fa602@bugseng.com>
Date: Wed, 7 Feb 2024 13:21:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and
 logging effects
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com>
 <09869431-ed00-4102-93fd-84c697a5c950@suse.com>
 <851eb6ec-558b-4a89-a31a-34046730bb55@bugseng.com>
 <41474fe7-c65f-4244-a455-0aaa0e4315e3@suse.com>
From: Simone Ballarin <simone.ballarin@bugseng.com>
Organization: BUGSENG
In-Reply-To: <41474fe7-c65f-4244-a455-0aaa0e4315e3@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 07/02/24 11:24, Jan Beulich wrote:
> On 07.02.2024 11:03, Simone Ballarin wrote:
>> On 06/02/24 13:04, Jan Beulich wrote:
>>> On 02.02.2024 16:16, Simone Ballarin wrote:
>>>> Rule 13.1: Initializer lists shall not contain persistent side effects
>>>>
>>>> Effects caused by debug/logging macros and functions (like ASSERT, __bad_atomic_size,
>>>> LOG, etc ...) that crash execution or produce logs are not dangerous in initializer
>>>> lists. The evaluation order in abnormal conditions is not relevant. Evaluation order
>>>> of logging effects is always safe.
>>>
>>> I thought I said so before: When talking of just logging, evaluation order
>>> may very well have a impact on correctness. Therefore we shouldn't mix
>>> debugging and logging.
>>
>> My general feeling was that changes like the following one are not supported by
>> the community:
>>
>> - x = { .field1 = function_with_logs_effects() /*other eventual code*/ };
>> + int field1 = function_with_logs_effects();
>> + x = { .field1 = field1 /*other eventual code*/};
>>
>> so I tried to deviate as much as possible.
>>
>> If having log effects is a good reason to do changes like the above, I can
>> propose a patch in that sense.
> 
> Just to avoid misunderstandings: I'm not advocating for changes like the
> one you outline above. I simply consider the rule too strict: There's
> nothing at risk when there's just a single operation with side effects
> in an initializer.

I agree for the safe cases such as single item list initializers (independently
by the number of effect contained in io_apic_read).
In fact, I was about to propose in another patch to deviate cases like:

union IO_APIC_reg_01 reg_01 = { .raw = io_apic_read(idx, 1) };
union IO_APIC_reg_02 reg_02 = { .raw = io_apic_read(idx, 2) };

> Even when there are multiple such operations, whether
> there's anything at risk depends on whether any of the side effects
> actually collide. In a number of cases the compiler would actually warn
> (and thus, due to -Werror, the build would fail).
> 

I don't completely agree on that, this requires an in-depth comprehension
of the code especially when complex call chains are involved. Moreover
these deviations need to be maintained when one of the function involved changes.

> The primary purpose of my comment here was that we need to please
> separate debugging from logging side effects.
>

Ok, I will work in that sense.

>>>> --- a/xen/arch/arm/guestcopy.c
>>>> +++ b/xen/arch/arm/guestcopy.c
>>>> @@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf, uint64_t addr, unsigned int len,
>>>>    unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
>>>>    {
>>>>        return copy_guest((void *)from, (vaddr_t)to, len,
>>>> -                      GVA_INFO(current), COPY_to_guest | COPY_linear);
>>>> +                      /* SAF-4-safe No persistent side effects */
>>>> +                      GVA_INFO(current),
>>>
>>> I _still_ think this leaves ambiguity. The more that you need to look
>>> up GVA_INFO() to recognize what this is about.
>>
>>
>> Just to recap: here the point is that current reads a register with a volatile asm, so the
>> violation is in the expansion of GVA_INFO(current). Both GVA_INFO and current taken alone
>> are completely fine, so this is the only place where a SAF comment can be placed.
>>
>> The exapansion is:
>> ((copy_info_t) { .gva = { ((*({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"(&
>>     per_cpu__curr_vcpu)); (typeof(&per_cpu__curr_vcpu)) (__ptr + (({ uint64_t _r; asm volatile("mrs  %0, ""TPIDR_EL2" : "=r"
>>     (_r)); _r; }))); }))) } }), (1U << 1) | (1U << 2));
>>
>> My proposals are:
>> 1) address the violation moving the current expansion outside (extra variable);
>> 2) put a more detailed comment to avoid the ambiguity;
>> 3) use an ECL deviation for GVA_INFO(current).
>>
>> Do you have any preference or proposal?
> 
> Imo 3 is not an option at all. Probably 1 wouldn't be too bad here, but
> I still wouldn't like it (as matching a general pattern I try to avoid:
> introducing local variables that are used just once and don't meaningfully
> improve e.g. readability). Therefore out of what you list, 2 would remain.
> But I'm not happy with a comment here either - as per above, there's
> nothing that can go wrong here as long as there's only a single construct
> with side effect(s).
>
So, would be changing the SAF in:
/* SAF-<new_id>-safe single item initializer */

OK for you?

>>>> --- a/xen/arch/x86/hvm/hvm.c
>>>> +++ b/xen/arch/x86/hvm/hvm.c
>>>> @@ -800,6 +800,7 @@ static int cf_check hvm_save_cpu_ctxt(struct vcpu *v, hvm_domain_context_t *h)
>>>>    {
>>>>        struct segment_register seg;
>>>>        struct hvm_hw_cpu ctxt = {
>>>> +        /* SAF-3-safe effects for debugging/logging reasons are safe */
>>>>            .tsc = hvm_get_guest_tsc_fixed(v, v->domain->arch.hvm.sync_tsc),
>>>
>>> A prereq for this imo is that the function take const struct vcpu *.
>>> But I'm not sure that'll suffice. The function can change at any time,
>>> rendering the comment here stale perhaps without anyone noticing.
>>>
>>
>> IMO It isn't a strict prereq, but it would make everything more clear.
>>
>> In any case, apart adding the const, I do not see other easy solutions.
>> Would you give me your ack if I change the function signature?
> 
> Well, as said: I'm not sure that'll suffice.
> 
>> Another possible solutions would be documenting the function in the new
>> JSON file with a special attribute like only_debug_effect. Of course,
>> this still requires keeping the JSON up to date in case of changes.
> 
> Exactly. So wouldn't really help.
> 
> In any event I'd like to ask that you consider splitting up this patch,
> such that you won't need multiple acks for any of the parts. That'll
> also allow focusing on one aspect at a time in reviews.
> 

Ok, but please consider that the JSON file has been precisely added to deal
with these cases (avoiding __attribute__). If we are scared to use it, it becomes
meaningless.

@Stefano maybe your opinion could help.

Another more complex option would be emulating __attribute__ (in general any property)
with a comment:

/* @attribute: [only_debug_effects]
int foo(void);

This solution implies that:
- we need a script to convert these comments in ECL configurations (the script should
   parse the entire codebase);
- regexes cannot be used (the JSON instead accepts regexes);
- to be coherent, all properties that actually lives in the JSON should be moved inside
   these comments. This means a lot of changes. A hybrid approach is also possible.

> Jan

-- 
Simone Ballarin, M.Sc.

Field Application Engineer, BUGSENG (https://bugseng.com)



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 12:28:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 12:28:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677511.1054115 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXh2E-000345-5K; Wed, 07 Feb 2024 12:28:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677511.1054115; Wed, 07 Feb 2024 12:28:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXh2E-00033y-2A; Wed, 07 Feb 2024 12:28:14 +0000
Received: by outflank-mailman (input) for mailman id 677511;
 Wed, 07 Feb 2024 12:28:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nQwS=JQ=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rXh2C-00033m-PP
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 12:28:12 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5b6dba7b-c5b4-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 13:28:11 +0100 (CET)
Received: from [192.168.1.140] (unknown [81.56.166.244])
 by support.bugseng.com (Postfix) with ESMTPSA id C7B3A4EE0738;
 Wed,  7 Feb 2024 13:28:10 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b6dba7b-c5b4-11ee-8a49-1f161083a0e0
Message-ID: <f4525632-5874-48f3-8b75-49c328433c43@bugseng.com>
Date: Wed, 7 Feb 2024 13:28:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 3/4] xen/x86: address violations of MISRA C:2012
 Rule 13.1
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <16bb514ac0a5fe0d6e9a2c95279a8200ff4495c6.1706886631.git.simone.ballarin@bugseng.com>
 <b23ffbf2-e1b1-42f5-b0ea-9f0889a5a7af@suse.com>
From: Simone Ballarin <simone.ballarin@bugseng.com>
Organization: BUGSENG
In-Reply-To: <b23ffbf2-e1b1-42f5-b0ea-9f0889a5a7af@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 06/02/24 14:13, Jan Beulich wrote:
> On 02.02.2024 16:16, Simone Ballarin wrote:
>> Rule 13.1: Initializer lists shall not contain persistent side effects
>>
>> This patch moves expressions with side-effects into new variables before
>> the initializer lists.
>>
>> No functional changes.
>>
>> Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
> 
> To be honest, I don't like this. It's more code for no gain. Really its
> hampering clarity imo. I'm willing to be convinced otherwise, but for
> now this gets a nack from me.
> 
> As an aside, ...
> 
>> --- a/xen/arch/x86/io_apic.c
>> +++ b/xen/arch/x86/io_apic.c
>> @@ -2559,9 +2559,12 @@ integer_param("max_gsi_irqs", max_gsi_irqs);
>>   
>>   static __init bool bad_ioapic_register(unsigned int idx)
>>   {
>> -    union IO_APIC_reg_00 reg_00 = { .raw = io_apic_read(idx, 0) };
>> -    union IO_APIC_reg_01 reg_01 = { .raw = io_apic_read(idx, 1) };
>> -    union IO_APIC_reg_02 reg_02 = { .raw = io_apic_read(idx, 2) };
>> +    uint32_t reg_00_raw = io_apic_read(idx, 0);
>> +    uint32_t reg_01_raw = io_apic_read(idx, 1);
>> +    uint32_t reg_02_raw = io_apic_read(idx, 2);

I'll propose a deviation for these single item initializers.

> 
> ... while you properly use uint32_t here, ...
> 
>> +    union IO_APIC_reg_00 reg_00 = { .raw = reg_00_raw };
>> +    union IO_APIC_reg_01 reg_01 = { .raw = reg_01_raw };
>> +    union IO_APIC_reg_02 reg_02 = { .raw = reg_02_raw };
>>   
>>       if ( reg_00.raw == -1 && reg_01.raw == -1 && reg_02.raw == -1 )
>>       {
>> --- a/xen/arch/x86/mpparse.c
>> +++ b/xen/arch/x86/mpparse.c
>> @@ -798,11 +798,12 @@ void __init mp_register_lapic_address (
>>   
>>   int mp_register_lapic(u32 id, bool enabled, bool hotplug)
>>   {
>> +	u32 apic = apic_read(APIC_LVR);> 
> ... why the being-phased-out u32 here?
> 

It was just to be coherent with the type of id.
I will use uint32_t in the next submission.

> Jan
> 
>>   	struct mpc_config_processor processor = {
>>   		.mpc_type = MP_PROCESSOR,
>>   		/* Note: We don't fill in fields not consumed anywhere. */
>>   		.mpc_apicid = id,
>> -		.mpc_apicver = GET_APIC_VERSION(apic_read(APIC_LVR)),
>> +		.mpc_apicver = GET_APIC_VERSION(apic),
>>   		.mpc_cpuflag = (enabled ? CPU_ENABLED : 0) |
>>   			       (id == boot_cpu_physical_apicid ?
>>   				CPU_BOOTPROCESSOR : 0),
>> --- a/xen/arch/x86/setup.c
>> +++ b/xen/arch/x86/setup.c
>> @@ -885,13 +885,14 @@ static struct domain *__init create_dom0(const module_t *image,
>>   {
>>       static char __initdata cmdline[MAX_GUEST_CMDLINE];
>>   
>> +    unsigned int max_vcpus = dom0_max_vcpus();
>>       struct xen_domctl_createdomain dom0_cfg = {
>>           .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
>>           .max_evtchn_port = -1,
>>           .max_grant_frames = -1,
>>           .max_maptrack_frames = -1,
>>           .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version),
>> -        .max_vcpus = dom0_max_vcpus(),
>> +        .max_vcpus = max_vcpus,
>>           .arch = {
>>               .misc_flags = opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0,
>>           },
> 

-- 
Simone Ballarin, M.Sc.

Field Application Engineer, BUGSENG (https://bugseng.com)



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 12:40:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 12:40:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677518.1054125 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXhER-0006CL-AZ; Wed, 07 Feb 2024 12:40:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677518.1054125; Wed, 07 Feb 2024 12:40:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXhER-0006CE-7d; Wed, 07 Feb 2024 12:40:51 +0000
Received: by outflank-mailman (input) for mailman id 677518;
 Wed, 07 Feb 2024 12:40:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXhEP-0006C8-B0
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 12:40:49 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1e5a3fdf-c5b6-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 13:40:48 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-40fdc63f4feso5060855e9.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 04:40:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l37-20020a05600c1d2500b0040ee8765901sm2003884wms.43.2024.02.07.04.40.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 04:40:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e5a3fdf-c5b6-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707309647; x=1707914447; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=T1lHdXC9Jjw0YX91nZGzqRXP2VzvUdARbS5xf1vyCXU=;
        b=el1s33uqPZ/eEFae3MQDyd0oeGwh3zEGoSVZZHHTll99/JZmGS0RSkQMDT+MZaZchy
         9vBnP+y3iS9FLgVtebyNpQaXEmhvlH1Adk1TqhVphbfrKjCiOi9hiaHa/ZDIKXJKLm9N
         QNaFlRvZnlKPAUQednZALc9tqCVJX6bgyy9r0+XC543VnmL+IiTuFGhn8YVLBLjgDivp
         iN0xervDFZRMg7ZYUeW4/ffRBrW0q74D8mfenn/JoAVt/9DxTnVcdRGa8IU1KVK1okmi
         us2XSrb8ejRLmKUTUhZsPiWl9KN6Dn9jFrqRxtrbB7C4AA+Rt5wDayhY6EBPc21yB47o
         uAIA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707309647; x=1707914447;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=T1lHdXC9Jjw0YX91nZGzqRXP2VzvUdARbS5xf1vyCXU=;
        b=uiy4mUTJ5X3cY3Hd1AEfukyfsQ413x7BfYk45vEpphXJW4ONcexiphcO+3a/V3lWFT
         TZSvuj6pgXEZNOdyolQDcIz1ZCs899MAh/ZTCXS89SAtUb2ijHMOOJRjpRIqnv4JBS5w
         x/3Db8c7+kfIkOJTGPX9QPE0umBRZj19xumTnfIiPfiTmvuSdv29V4DOaN/axiFuhIq0
         Nm3Yln/mjGfJz681qtLC+iZj4uYhFW1hGIkIz7DFw8L70mBUh7vZ4ES4Zwfbk07d0ZgH
         K2wwqUbsXpBztLazlX06I+3h5TNt+5S2uJGpZKORwRTIUuMo3b70FO0hYJ4BhbTrxLFi
         gnPA==
X-Forwarded-Encrypted: i=1; AJvYcCVz9uR/HxhZkjswVBzNWW5wsENlJDY34ginNozqs9vEjIZ9ZGeIvoPyNSr9P+mr2W9M1p+ZFk45VNpOKWwHBffqHYhCR8H3WIApGpNCuJo=
X-Gm-Message-State: AOJu0YyL/eTY++89xrozugKwjovXCqFjaYqqs/VZG7slML8Q5c98wqlp
	ya74gp0dZrwWTpiU6ec21u85XvbFZ49bMUsD1fFoaHcbFWk0xNbVaZXoWwIEnA==
X-Google-Smtp-Source: AGHT+IHaM2t4FvpcMnWtFL2D6FdMm2/5mjLh64fuuyR9/jHOxEa+PHTuOxFlI3AuScnbiYY2JFZgjw==
X-Received: by 2002:a05:600c:510b:b0:40e:b9fb:103 with SMTP id o11-20020a05600c510b00b0040eb9fb0103mr3832256wms.23.1707309647487;
        Wed, 07 Feb 2024 04:40:47 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXcumd5rXaXzOauyJojIzCtGpP09YDamgBoScdfjbGQmT4YYESL5fb3I0vmOUjXbFZYpRFwE5TBni5+R9GfViw9TnRx9YjK3S29sYItVpRI7SHOTyNh2GMpyM8BoJc1gHZp88Y5XtzCmjg/nO+2WXGy0FzHcZeYKtnw4jedbPuY0HwLLOXT5jxaYAuosXrxJz/TiZl7FC3ztUsDAC343KTKVzeVWsNN1d5kqAc/j6AbYZ8k+9kc05LcnbQKhFnZbLNd0kBD96SHaEUKuzrNhAiskIJHpw/vTQyXJmfh8YkmLqhPLsnjFWwuWUnoUXO1tRVYwSc7cC7eRRe7XBSymrSOm81+Q7kNUS4vefCzHaiWDcI26dkzq3krxxXf/FgjMZNtxWoYZ3iKtuEfe9nNTylWOq0IKzUDyxBJROmx6XqmiTLCqF+HxAAaLabIbZwz43g=
Message-ID: <c6199a60-ecd2-4f8a-b742-199b90e62bac@suse.com>
Date: Wed, 7 Feb 2024 13:40:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and
 logging effects
Content-Language: en-US
To: Simone Ballarin <simone.ballarin@bugseng.com>
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com>
 <09869431-ed00-4102-93fd-84c697a5c950@suse.com>
 <851eb6ec-558b-4a89-a31a-34046730bb55@bugseng.com>
 <41474fe7-c65f-4244-a455-0aaa0e4315e3@suse.com>
 <e48f4d49-274c-4bd6-8e3b-1ecaee6fa602@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e48f4d49-274c-4bd6-8e3b-1ecaee6fa602@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 13:21, Simone Ballarin wrote:
> On 07/02/24 11:24, Jan Beulich wrote:
>> On 07.02.2024 11:03, Simone Ballarin wrote:
>>> On 06/02/24 13:04, Jan Beulich wrote:
>>>> On 02.02.2024 16:16, Simone Ballarin wrote:
>>>>> Rule 13.1: Initializer lists shall not contain persistent side effects
>>>>>
>>>>> Effects caused by debug/logging macros and functions (like ASSERT, __bad_atomic_size,
>>>>> LOG, etc ...) that crash execution or produce logs are not dangerous in initializer
>>>>> lists. The evaluation order in abnormal conditions is not relevant. Evaluation order
>>>>> of logging effects is always safe.
>>>>
>>>> I thought I said so before: When talking of just logging, evaluation order
>>>> may very well have a impact on correctness. Therefore we shouldn't mix
>>>> debugging and logging.
>>>
>>> My general feeling was that changes like the following one are not supported by
>>> the community:
>>>
>>> - x = { .field1 = function_with_logs_effects() /*other eventual code*/ };
>>> + int field1 = function_with_logs_effects();
>>> + x = { .field1 = field1 /*other eventual code*/};
>>>
>>> so I tried to deviate as much as possible.
>>>
>>> If having log effects is a good reason to do changes like the above, I can
>>> propose a patch in that sense.
>>
>> Just to avoid misunderstandings: I'm not advocating for changes like the
>> one you outline above. I simply consider the rule too strict: There's
>> nothing at risk when there's just a single operation with side effects
>> in an initializer.
> 
> I agree for the safe cases such as single item list initializers (independently
> by the number of effect contained in io_apic_read).
> In fact, I was about to propose in another patch to deviate cases like:
> 
> union IO_APIC_reg_01 reg_01 = { .raw = io_apic_read(idx, 1) };
> union IO_APIC_reg_02 reg_02 = { .raw = io_apic_read(idx, 2) };
> 
>> Even when there are multiple such operations, whether
>> there's anything at risk depends on whether any of the side effects
>> actually collide. In a number of cases the compiler would actually warn
>> (and thus, due to -Werror, the build would fail).
>>
> 
> I don't completely agree on that, this requires an in-depth comprehension
> of the code especially when complex call chains are involved. Moreover
> these deviations need to be maintained when one of the function involved changes.

Right, and I didn't really mean multiple function calls here, but e.g.
multiple ++ or --.

>>>>> --- a/xen/arch/arm/guestcopy.c
>>>>> +++ b/xen/arch/arm/guestcopy.c
>>>>> @@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf, uint64_t addr, unsigned int len,
>>>>>    unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
>>>>>    {
>>>>>        return copy_guest((void *)from, (vaddr_t)to, len,
>>>>> -                      GVA_INFO(current), COPY_to_guest | COPY_linear);
>>>>> +                      /* SAF-4-safe No persistent side effects */
>>>>> +                      GVA_INFO(current),
>>>>
>>>> I _still_ think this leaves ambiguity. The more that you need to look
>>>> up GVA_INFO() to recognize what this is about.
>>>
>>>
>>> Just to recap: here the point is that current reads a register with a volatile asm, so the
>>> violation is in the expansion of GVA_INFO(current). Both GVA_INFO and current taken alone
>>> are completely fine, so this is the only place where a SAF comment can be placed.
>>>
>>> The exapansion is:
>>> ((copy_info_t) { .gva = { ((*({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"(&
>>>     per_cpu__curr_vcpu)); (typeof(&per_cpu__curr_vcpu)) (__ptr + (({ uint64_t _r; asm volatile("mrs  %0, ""TPIDR_EL2" : "=r"
>>>     (_r)); _r; }))); }))) } }), (1U << 1) | (1U << 2));
>>>
>>> My proposals are:
>>> 1) address the violation moving the current expansion outside (extra variable);
>>> 2) put a more detailed comment to avoid the ambiguity;
>>> 3) use an ECL deviation for GVA_INFO(current).
>>>
>>> Do you have any preference or proposal?
>>
>> Imo 3 is not an option at all. Probably 1 wouldn't be too bad here, but
>> I still wouldn't like it (as matching a general pattern I try to avoid:
>> introducing local variables that are used just once and don't meaningfully
>> improve e.g. readability). Therefore out of what you list, 2 would remain.
>> But I'm not happy with a comment here either - as per above, there's
>> nothing that can go wrong here as long as there's only a single construct
>> with side effect(s).
>>
> So, would be changing the SAF in:
> /* SAF-<new_id>-safe single item initializer */
> 
> OK for you?

A comment, as said, is only the least bad of what you did enumerate. But
for this code in particular I'm not a maintainer anyway, so it's not me
you need to convince. I'm taking this only as an example for discussing
underlying aspects.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 12:50:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 12:50:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677523.1054134 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXhNk-0007jU-7I; Wed, 07 Feb 2024 12:50:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677523.1054134; Wed, 07 Feb 2024 12:50:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXhNk-0007jN-4f; Wed, 07 Feb 2024 12:50:28 +0000
Received: by outflank-mailman (input) for mailman id 677523;
 Wed, 07 Feb 2024 12:50:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXhNj-0007jH-22
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 12:50:27 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 768d2301-c5b7-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 13:50:25 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-4101d4c5772so1426225e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 04:50:25 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v16-20020a05600c471000b0040fdb17e66csm2043341wmo.4.2024.02.07.04.50.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 04:50:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 768d2301-c5b7-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707310225; x=1707915025; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ae0tUtHfV8laKYvxnKfLbizXF3IJHmTWSyh1XqNp3YA=;
        b=NAPrrJw5K6wrmczPDQvgHHktoy+mP1HYco+WYVurdRxAcUHOL1I0iIFoUtpUZTSoN4
         px4SAC654EvW/ys5MFpPVK1qi7bADGazlGzl+jHAbTiBJwJtTAygIxngA3uf0s3PG82k
         iLkCJimeqft2lBeroxMjX8VtoC55uD3LYfzfKhVEmhlc9uJ1DF4zdbr6n+mOjTXBXcmg
         22BuNCThPvUfiT/631jrAd+rTchF129hifzaTcXrPQZGZfUYV/LZYM81xa7Ji8DF4ybY
         2vNTv5PNPwKqBxxv5BP+wad94xvWCfN5mDjO+HLUiMdw2OpjgzNir0Z73H8MwWlZmn01
         QxSg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707310225; x=1707915025;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ae0tUtHfV8laKYvxnKfLbizXF3IJHmTWSyh1XqNp3YA=;
        b=EKyZ5DgsaNUVy18B9EwsiNmGNxzrK7yuHMf8/srRxGHfta9lGiL7eD3F+fzztAXtA9
         CGnBlOFw0+aUn1g1kBMEyQIckdNfwM+Vjy6AuuKZGtGAC8n5p/sDFtlut4pcml3zSPKA
         LY+Np9CAw9D4Hhhpdgc/5UA1rMe01nlONXM+9v3Cj/miYomCx0KDZqLamrVqvkRlCQL3
         w1rx/FjoouOxCRmalwNdnBzYy1OJFg6jNLQDgq1Ae2lY2gBIfzPYo2TAiJN+JSCecaDO
         85EP2qgIiIBRK2DaVvaE8YUY+l3f2hS1Aft+d1PE9JqAQPxQbXzhD5QPrgvkJSZhzn3M
         ZCbQ==
X-Gm-Message-State: AOJu0YyXRBmrqEMljLdVpyKlAgrvOGWRYvHgGFqwkGUkc1Bqr6QEOB8z
	whAOC3SLx4iBOd4F4+B5jvPQdbRDuOlKu/iVZfjN1uTSHwhv3Yu5xjHZT/JZ1w==
X-Google-Smtp-Source: AGHT+IGPSsqy7RCQQISIiIhqdSOO7fwn8IPQMQ2jF1f7L8qTSHxw6KOI0LcLphgEByloK0csgEoaOQ==
X-Received: by 2002:a05:600c:1914:b0:40f:d0a7:3fdd with SMTP id j20-20020a05600c191400b0040fd0a73fddmr9635619wmq.6.1707310224933;
        Wed, 07 Feb 2024 04:50:24 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCV1CR6eF1nWeIBfrNvfN5o0i1JQupEMG8yAWVYGm05A1t0W4P583ZOmB4oBFSDjOcM+HV1MJPg3L2p/jmTTMnQygD8NI/mGO+R53By96fTjH+0DGCDX
Message-ID: <a0145f9d-654d-4336-b5bb-69883543b919@suse.com>
Date: Wed, 7 Feb 2024 13:50:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/4] block-common: Fix same_vm for no targets
Content-Language: en-US
To: Jason Andryuk <jandryuk@gmail.com>,
 Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <20240201183024.145424-1-jandryuk@gmail.com>
 <20240201183024.145424-2-jandryuk@gmail.com>
 <cd9e0194-5814-4735-bca4-df2577102c19@perard>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <cd9e0194-5814-4735-bca4-df2577102c19@perard>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 12:45, Anthony PERARD wrote:
> On Thu, Feb 01, 2024 at 01:30:21PM -0500, Jason Andryuk wrote:
>> same_vm is broken when the two main domains do not have targets.  otvm
>> and targetvm are both missing, which means they get set to -1 and then
>> converted to empty strings:
>>
>> ++10697+ local targetvm=-1
>> ++10697+ local otvm=-1
>> ++10697+ otvm=
>> ++10697+ othervm=/vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4
>> ++10697+ targetvm=
>> ++10697+ local frontend_uuid=/vm/844dea4e-44f8-4e3e-8145-325132a31ca5
>>
>> The final comparison returns true since the two empty strings match:
>>
>> ++10697+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o '' = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = '' -o '' = '' ']'
>>
>> Replace -1 with distinct strings indicating the lack of a value and
>> remove the collescing to empty stings.  The strings themselves will no
>> longer match, and that is correct.
>>
>> ++12364+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o 'No target' = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = 'No other target' -o 'No target' = 'No other target' ']'
>>
>> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
> 
> Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

I've committed this, but I take the absence of a Fixes: tag as indication
that this doesn't want/need backporting.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:00:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:00:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677538.1054185 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXhX7-0002BH-FP; Wed, 07 Feb 2024 13:00:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677538.1054185; Wed, 07 Feb 2024 13:00:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXhX7-0002BA-C8; Wed, 07 Feb 2024 13:00:09 +0000
Received: by outflank-mailman (input) for mailman id 677538;
 Wed, 07 Feb 2024 13:00:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXhX5-0002Ay-Nv; Wed, 07 Feb 2024 13:00:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXhX5-0001ec-GU; Wed, 07 Feb 2024 13:00:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXhX4-0004UA-VA; Wed, 07 Feb 2024 13:00:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXhX4-0004ay-Un; Wed, 07 Feb 2024 13:00:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ogOjgs/O9fPSIcE3A4/uLXT5SIOIEmVtAqbRBPQvBOo=; b=guP+cnYDz+krWUrjSMdYzFRP7J
	9yLWmT8s9Ei+wf+oNV42RyHptxP+wYZzK2O6Q8OmO0kRxD2UYGEe3/X8G+uOyB6fFBwosknBp4JCc
	GbPv+ojdcnEAZnhCf1ChMgX/YGm8GWmsYBayPHU1zhCGX+U3ZkCpDFGpjPNZ0ZoREz20=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184612-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184612: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-i386-qemut-rhel6hvm-intel:guest-start/redhat.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=57a923bfd920307c5bfa533c95dba16a13ea0225
X-Osstest-Versions-That:
    xen=1cb7949b37b3d8892d8a3cea3ad1e2448352a34d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 07 Feb 2024 13:00:06 +0000

flight 184612 xen-unstable real [real]
flight 184615 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184612/
http://logs.test-lab.xenproject.org/osstest/logs/184615/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-qemut-rhel6hvm-intel 14 guest-start/redhat.repeat fail pass in 184615-retest
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install fail pass in 184615-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop  fail in 184615 like 184604
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184604
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184604
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184604
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184604
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184604
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184604
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184604
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184604
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184604
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184604
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184604
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  57a923bfd920307c5bfa533c95dba16a13ea0225
baseline version:
 xen                  1cb7949b37b3d8892d8a3cea3ad1e2448352a34d

Last test of basis   184604  2024-02-06 10:06:23 Z    1 days
Testing same since   184612  2024-02-07 01:07:15 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  David Woodhouse <dwmw2@amazon.com>
  Elias El Yandouzi <eliasely@amazon.com>
  Frediano Ziglio <frediano.ziglio@cloud.com>
  George Dunlap <george.dunlap@cloud.com>
  Hongyan Xia <hongyxia@amazon.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  oxjo <oxjo@proton.me>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>
  Wei Liu <wei.liu2@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   1cb7949b37..57a923bfd9  57a923bfd920307c5bfa533c95dba16a13ea0225 -> master


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:31:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:31:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677551.1054194 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi1E-0006Zf-0G; Wed, 07 Feb 2024 13:31:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677551.1054194; Wed, 07 Feb 2024 13:31:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi1D-0006ZY-U2; Wed, 07 Feb 2024 13:31:15 +0000
Received: by outflank-mailman (input) for mailman id 677551;
 Wed, 07 Feb 2024 13:31:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=D7mL=JQ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXi1C-0006ZS-T0
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:31:14 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 290b653f-c5bd-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 14:31:12 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id AAED0222E0;
 Wed,  7 Feb 2024 13:31:09 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5C54B139B9;
 Wed,  7 Feb 2024 13:31:09 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id jbQHFR2Gw2W/HAAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 07 Feb 2024 13:31:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 290b653f-c5bd-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707312671; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=m2KNOLFrtgTQyAwV3sA794wX+VsOWbUl4bE7mleClg4=;
	b=BHvtWdvuaomagskxwCJOLeBvflORnsvVhRAJFkyLAeAhPZ8JQ45oSOvGCJT6VodgYEGeYz
	tilF1h2xLYdYvpaZOtagheoIyFsKUFqUKuQGzafMl7FB6xisG1sMDWBL3/hUrlQSLXEuZ4
	i2y19OKbxg4a6HDLruheO4Ii2eIeDO0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707312669; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=m2KNOLFrtgTQyAwV3sA794wX+VsOWbUl4bE7mleClg4=;
	b=P0VaZMVsqHg6BvpOPYIRSKlpfuVzKsa0ADtTxmXFz3M1E4im7HsGT6fHPsiSmxrcFCAxDS
	PdUj2695VC2u/wpDF2cEcleaqYwv+zNXM5AxeDgBldyNn5dIyx1UZ617aahLfmYHKXIDGU
	OF+arwZuqC/PpLx8Vr0ebAd0cwZW7e0=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] config: update Mini-OS commit
Date: Wed,  7 Feb 2024 14:31:06 +0100
Message-Id: <20240207133106.6185-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [4.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[config.mk:url,suse.com:email,xen.org:url];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[15.33%]
X-Spam-Level: ****
X-Spam-Score: 4.90
X-Spam-Flag: NO

Update the Mini-OS upstream revision.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 Config.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Config.mk b/Config.mk
index f7d6d84847..077d841bb7 100644
--- a/Config.mk
+++ b/Config.mk
@@ -224,7 +224,7 @@ QEMU_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/qemu-xen.git
 QEMU_UPSTREAM_REVISION ?= master
 
 MINIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/mini-os.git
-MINIOS_UPSTREAM_REVISION ?= 090eeeb1631f00a9a41ebf66d9b4aacb97eb51e7
+MINIOS_UPSTREAM_REVISION ?= b119f0178fd86876d0678007dfcf435ab8bb7568
 
 SEABIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/seabios.git
 SEABIOS_UPSTREAM_REVISION ?= rel-1.16.3
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:34:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:34:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677554.1054205 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi4L-00078W-Es; Wed, 07 Feb 2024 13:34:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677554.1054205; Wed, 07 Feb 2024 13:34:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi4L-00078P-Bk; Wed, 07 Feb 2024 13:34:29 +0000
Received: by outflank-mailman (input) for mailman id 677554;
 Wed, 07 Feb 2024 13:34:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXi4J-00078H-Jv
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:34:27 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9cba63ec-c5bd-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 14:34:26 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-4101fc00832so1597055e9.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:34:26 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z12-20020a056000110c00b0033b4ebc3c8fsm1328999wrw.2.2024.02.07.05.34.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:34:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9cba63ec-c5bd-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707312866; x=1707917666; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:subject:from:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=XqtG+cLyvuCJcFuJ5L5v3nQUvmQ55x8jdSxLBkC+bBI=;
        b=Y0/Lf4hQL6BIGqAU5VJymHVp8BNntDvI1Rp1Sh7GgjHNbUW2Pz6iz6XSfAEBXLsarx
         4noocEs7dwdV6dGTIh8RMr98rpQcyzVc2zV38bCzdJvkBD0LfYQNsV+tAytrpAWbV8rJ
         IXunMToIXC4QFTv4QqCl/lT8MdIYrdpaoYatxNyaA4iY2Oe/KUEQIoImY9jog7qaWaOx
         3k0euunLQtDfxO2RjCdfOS52Aq1lVCAOFcU90J6Bs029WJFfAORKZzbC8SuVof4+hLgx
         7GzeddLzh/FVxS/HabYyhGUeKKf3/Y5alRPGd7FKCmWHqNMh1VWm3WAtw3Dzu2gwRIT8
         BMDw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707312866; x=1707917666;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:subject:from:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=XqtG+cLyvuCJcFuJ5L5v3nQUvmQ55x8jdSxLBkC+bBI=;
        b=i5PhkE6zdmgwDGG/tJbtIC7P/ZGZERyUcKEg2VuD5CfSovhqimEugigE4tGzVT/kTY
         11sz91tmpwvC6dIgA72xLvHbmnYpXwREDjFWqgXtM5IBMDxSdSZL31U0ktaONibd0wOs
         tI2R1Yi64enIDF8rQSls+H8w4VHwpUT1p5sTHvDM1NxdnQqbagLQMYNHGijTacmY6PgF
         ObOmZwxeqPfhG8mCBovzs0f8DthNHcDwxF/Kqhpg8sIG5IHXsMW9q0mOsDQz3NJtX3Dn
         kv+OQwsy3OlwUHWPh5vloUfsr9t9LSv/FB3w3WPmgV+QvxRerIQpYa8ERrCFzBpOhaQg
         IsBg==
X-Gm-Message-State: AOJu0YzZ00TZw6BXRN3gr0wxJ22tdFnqc3511dcVt1ylUOV3sZ0a8qi8
	o/hpcUu44v0IQNzteJAZ/YCoGIM22K/7OUWqWv8BH2xLPxZE40KR0VN8SEgRLPdv5Czj6dTX5xQ
	=
X-Google-Smtp-Source: AGHT+IHLnZH3SlWA6FxETBF8HPAoCyGLozRz1XspjDqS4MuHTytt5KBoz7tZz2gkAOiRfb5qLw83SQ==
X-Received: by 2002:a05:600c:3b85:b0:410:40d:6eac with SMTP id n5-20020a05600c3b8500b00410040d6eacmr1690150wms.18.1707312866076;
        Wed, 07 Feb 2024 05:34:26 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXRlJnf3no8t/086ixNowgAF7dgqO1CQAtJjDRCys5pvucapDgbJTdwv5LV7F8ccZ9xWr+dDwyMhuA5Ah8FnqXu0BMv/Sl9sDHbySfi16QHypCp7F/Jk227dZ5PWA9dkFu7l4IhdkJLS4SH
Message-ID: <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Date: Wed, 7 Feb 2024 14:34:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v6 7/7] (mostly) x86: add/convert entry point annotations
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

1: common: honor CONFIG_CC_SPLIT_SECTIONS also for assembly functions
2: SVM: convert entry point annotations
3: VMX: convert entry point annotations
4: x86/ACPI: annotate assembly functions with type and size
5: x86/kexec: convert entry point annotations
6: x86: convert misc assembly function annotations
7: x86: move ENTRY(), GLOBAL(), and ALIGN

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:36:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:36:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677558.1054214 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi5o-0007fp-O2; Wed, 07 Feb 2024 13:36:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677558.1054214; Wed, 07 Feb 2024 13:36:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi5o-0007fi-LR; Wed, 07 Feb 2024 13:36:00 +0000
Received: by outflank-mailman (input) for mailman id 677558;
 Wed, 07 Feb 2024 13:35:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXi5n-0007fa-2H
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:35:59 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d2a1add9-c5bd-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 14:35:57 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-4101fc00832so1611715e9.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:35:57 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z12-20020a056000110c00b0033b4ebc3c8fsm1328999wrw.2.2024.02.07.05.35.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:35:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2a1add9-c5bd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707312956; x=1707917756; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=6Ge7Se7yqBn7izxaE/vDZCmzRLyqnuhDzqa6J3Ad+rE=;
        b=auYVzOQCqjtSlhRP/NN8jqe5eJsWJqQybLoi9XtgxZQW17ZU1nxF8ppK6xyxCQTYVs
         4ZwoFceyhuH0djo9yDV+kjcgwuA8o8o0fy10Rns/2FXTJnC3eQhaDol37E43ipSEzDFP
         u/L29yQlAGUBs/2RRvJ9RFUwDRjbtBYdWYePinwEB3m1peX/ol1mtocQN/JZwcK+JiqY
         AW2qQ/PevsvgPx3a1cVhXx5gz5IYQ6rT3+FXWiwYG6N6w2/TKDsMuPsjil0wN4929cIW
         rZDzMGvRToNo4udW5tsbBLjJ4ir/iyyo3Wh+8EEvO/uJfLqLkFrw/IVpCqsb0l0tJV5N
         JPew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707312956; x=1707917756;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=6Ge7Se7yqBn7izxaE/vDZCmzRLyqnuhDzqa6J3Ad+rE=;
        b=EXEPE6oUbOh/FA6rBDP79ZW5Qtl182ecMwWgkJ3g6qUxrvznv1B2KK2ERTnqwND4C8
         G7Bgl3PapIEz1Wlsu4mARzOGoj80noehbJmIXwyaLh/YN0xj2XbikyBjPFLxrPMcQ5V/
         WA0kYYuy65eBqSSah9aoA2T93PVzkaQlDExLAfsWlaCWgG+unO7J+yR5xVLuqWE8PrZz
         oZJpBWtxLy3IajBIo9r3a4q6VpTPQVzVMN+2PDcFP6Ystohl4AbQ6kAMZ10uzslTkMsn
         ssVa3MFZoH4zA/aybiDCIZzSIbV50/jJyWI1aoSs2pMt8sMdwGqg+/icg6QbL4T5j6V7
         3saA==
X-Gm-Message-State: AOJu0Yx6lgjGHrUqT9yHyrW1zLM0VCDpCACtz2Z6juCAKY7i6XszWzTL
	eoT+5ZdcSuvIzEzUyLQLitYFWEfYJxIab6IHY5XbtWpPtVPRShX9yTQLZ0QRvH1WFKN2HMISaQM
	=
X-Google-Smtp-Source: AGHT+IF4zhlpSrmemj+UxZjn5oYwYy1YVpD3BmS8329z2l7ta/Ap8kUijS0TEYXCD5sRz3t6lWlvyg==
X-Received: by 2002:a5d:5963:0:b0:33b:28e0:e09c with SMTP id e35-20020a5d5963000000b0033b28e0e09cmr3420394wri.61.1707312956517;
        Wed, 07 Feb 2024 05:35:56 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUORpMt9ryxusJTp2j1gkDjQErVyTJQLGNbSk9Nk6TZNcdSdBhnTzrqfFUJglTQiwFaqbCmRea2prYQDVzKagPQa73tc29Vbji1DoZEySzRkWpXbmDff14JbRx2d35PuZ2D62JFBB8s6u3D
Message-ID: <54e8944e-b22b-4134-862c-ab9c16ce18bb@suse.com>
Date: Wed, 7 Feb 2024 14:35:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 0/7] (mostly) x86: add/convert entry point annotations
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 14:34, Jan Beulich wrote:
> 1: common: honor CONFIG_CC_SPLIT_SECTIONS also for assembly functions
> 2: SVM: convert entry point annotations
> 3: VMX: convert entry point annotations
> 4: x86/ACPI: annotate assembly functions with type and size
> 5: x86/kexec: convert entry point annotations
> 6: x86: convert misc assembly function annotations
> 7: x86: move ENTRY(), GLOBAL(), and ALIGN

Being the cover letter, the title was of course supposed to say 0/7.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:36:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:36:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677560.1054224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi6Y-0008BD-1c; Wed, 07 Feb 2024 13:36:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677560.1054224; Wed, 07 Feb 2024 13:36:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi6X-0008B6-V4; Wed, 07 Feb 2024 13:36:45 +0000
Received: by outflank-mailman (input) for mailman id 677560;
 Wed, 07 Feb 2024 13:36:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXi6W-0007fa-W5
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:36:44 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ede63ab8-c5bd-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 14:36:42 +0100 (CET)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-33b189ae5e8so419539f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:36:42 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z12-20020a056000110c00b0033b4ebc3c8fsm1328999wrw.2.2024.02.07.05.36.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:36:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ede63ab8-c5bd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707313002; x=1707917802; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=XBbuaqP2Yx3a79KtXTeTOOaIP4tI4ggPxVsGqod4xkQ=;
        b=ZVNu3O41pUetosxip8Wp5bxVP7uETem9CWyYInQNeAyeshWY7otDXTEOvQwsp7RNfX
         nHRxuDiJK1laFyurwXq3baPc/+r6R6CnYL5LcfNjWMGMcfFEMaqX+RhnIBWNrYIxzGjb
         5ZxX5JValVIYZmbS3oyczofzX5rDMiIV2xHngqbTPsjDbYyAHBfsbfNZnMuBWVKhSqdX
         2Ki9cft4Bm9p1f+k3nufz+1Ff8u7Td1+c4q6KBvc2evs2tXRvQPE1QZcVDprgN5DJ5l0
         phs76cPy/Fzr3fgTcEmvSJDSpMFRJzozNwCkSTQsFI5NSSQyhGkoN1g+AmbVdP3IF/3J
         8MJw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707313002; x=1707917802;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=XBbuaqP2Yx3a79KtXTeTOOaIP4tI4ggPxVsGqod4xkQ=;
        b=Dr9FjIEJ3NPQcqQZ53EyDt0WgHmv8AKYqXk4ikvMnMlXc6MAnZGzksQHQ68t3ft7aT
         4vRGqnlAYT8QS74t72xFLUEC1tYoNWQL7k4Jnj6wBH/nalxNO1KEWX6TbACjjL3QqDT2
         DscrzjU3B16sLGgFkPhKjSPXu6DhHN90Rg7JL/v/ONjkv/mCvlny2TRlJpdX5Civ7olW
         SBUPQxbJIOtEdjM3pViWP3uVKtAtCnDZqPjAYuML5lV3lxrPQ7UT7ZK7FWEKTY5PPj6I
         G+tDN+QBi4Syt4ipGUBmtzjx0R9pwADfOpLWwz5bgr5WKcUARZ8mW47NsHwjc8q5+ghC
         v0vg==
X-Gm-Message-State: AOJu0Yy/gsW7w7sPYjTohME0eYl2jzr2+wyNPH+qx/iWY69OC1YIglj9
	2VA8z/cLyaCN+R6UJLnlBd6GidI4qipBXaZvRbBRvW4D47Wb3SlYnn+z+UzOpyzO4MGcEOS5zp0
	=
X-Google-Smtp-Source: AGHT+IH1eErRkcqc/6EFZPgTTvBp8bktJo4gueEGOvOTR+LjDzxR9mUOdCImQ6XCeqPVisG8EIrdRQ==
X-Received: by 2002:adf:e4cd:0:b0:33b:5192:53d0 with SMTP id v13-20020adfe4cd000000b0033b519253d0mr691003wrm.46.1707313002218;
        Wed, 07 Feb 2024 05:36:42 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW4dYHoFA87bhX0+CGTEUDvc+ruSx2ZcmhS90KX/uh3b/3xj7c3BjV5uttYUYAS5RMWLObvZEAoDhUYG6oFmaM1yfFtJ2vHZhoh0kNuKeSnsObb2iVQAIuMRps6P3s5kjFq7rLnLYJPIZCEUuQP1s64mDMhnajZNHCu4TeG9pgkZgevHIbklLsjoviVOoGKusX3rcUeI6IYA7ZGq5rO8TaGP5h61OHOtxF35Bt8BdejVJAaOHq+dcOJpUzBHeA5LMtVenrgpmAJcvQ2CO3fLB+bhYyLhjXv6Npky5QArSEl8Nsx6dE6S1l7nno9xWECDx9zXg4PLw+i5mONhB4rQggbDQD9dldfMDHrmc9S3TorMtg1T5vNlUst4cufz9OpjRY7IKCTlpufutOIVDneZIgdO/LvIXDXdmUQfT3DU6ynQAzlYHbAX7JR6ek=
Message-ID: <782b911e-c40e-4e9c-b259-af287d968b3b@suse.com>
Date: Wed, 7 Feb 2024 14:36:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v6 1/7] common: honor CONFIG_CC_SPLIT_SECTIONS also for
 assembly functions
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Bobby Eshleman <bobbyeshleman@gmail.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Connor Davis <connojdavis@gmail.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Leverage the new infrastructure in xen/linkage.h to also switch to per-
function sections (when configured), deriving the specific name from the
"base" section in use at the time FUNC() is invoked.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
TBD: Since we use .subsection in UNLIKELY_START(), a perhaps not really
     wanted side effect of this change is that respective out-of-line
     code now moves much closer to its original (invoking) code.

TBD: Of course something with the same overall effect, but less
     impactful might do in Config.mk. E.g. $(filter-out -D%,$(3))
     instead of $(firstword (3)). In fact Roger wants the detection to
     be in Kconfig, for LIVEPATCH to depend on it. Yet the whole
     underlying discussion there imo would first need settling (and
     therefore reviving).

TBD: On top of Roger's respective patch (for livepatch), also respect
     CONFIG_FUNCTION_ALIGNMENT.

Note that we'd need to split DATA() in order to separate r/w and r/o
contributions. Further splitting might be needed to also support more
advanced attributes (e.g. merge), hence why this isn't done right here.
Sadly while a new section's name can be derived from the presently in
use, its attributes cannot be. Perhaps the only thing we can do is give
DATA() a 2nd mandatory parameter. Then again I guess most data
definitions could be moved to C anyway.
---
v6: Deal with x86'es entry_PF() and entry_int82() falling through to the
    next "function". Re-base.
v5: Re-base over changes earlier in the series.
v4: Re-base.
v2: Make detection properly fail on old gas (by adjusting
    cc-option-add-closure).

--- a/Config.mk
+++ b/Config.mk
@@ -102,7 +102,7 @@ cc-option = $(shell if $(1) $(2:-Wno-%=-
 # Usage: $(call cc-option-add CFLAGS,CC,-march=winchip-c6)
 cc-option-add = $(eval $(call cc-option-add-closure,$(1),$(2),$(3)))
 define cc-option-add-closure
-    ifneq ($$(call cc-option,$$($(2)),$(3),n),n)
+    ifneq ($$(call cc-option,$$($(2)),$(firstword $(3)),n),n)
         $(1) += $(3)
     endif
 endef
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -409,6 +409,9 @@ AFLAGS += -D__ASSEMBLY__
 
 $(call cc-option-add,AFLAGS,CC,-Wa$$(comma)--noexecstack)
 
+# Check to see whether the assembler supports the --sectname-subst option.
+$(call cc-option-add,AFLAGS,CC,-Wa$$(comma)--sectname-subst -DHAVE_AS_SECTNAME_SUBST)
+
 LDFLAGS-$(call ld-option,--warn-rwx-segments) += --no-warn-rwx-segments
 
 CFLAGS += $(CFLAGS-y)
--- a/xen/arch/arm/xen.lds.S
+++ b/xen/arch/arm/xen.lds.S
@@ -157,6 +157,9 @@ SECTIONS
   .init.text : {
        _sinittext = .;
        *(.init.text)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+       *(.init.text.*)
+#endif
        _einittext = .;
        . = ALIGN(PAGE_SIZE);        /* Avoid mapping alt insns executable */
        *(.altinstr_replacement)
--- a/xen/arch/ppc/xen.lds.S
+++ b/xen/arch/ppc/xen.lds.S
@@ -104,6 +104,9 @@ SECTIONS
     .init.text : {
         _sinittext = .;
         *(.init.text)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+        *(.init.text.*)
+#endif
         _einittext = .;
         . = ALIGN(PAGE_SIZE);        /* Avoid mapping alt insns executable */
     } :text
--- a/xen/arch/riscv/xen.lds.S
+++ b/xen/arch/riscv/xen.lds.S
@@ -104,6 +104,9 @@ SECTIONS
     .init.text : {
         _sinittext = .;
         *(.init.text)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+        *(.init.text.*)
+#endif
         _einittext = .;
         . = ALIGN(PAGE_SIZE);        /* Avoid mapping alt insns executable */
     } :text
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -29,6 +29,9 @@ FUNC(entry_int82)
 
         mov   %rsp, %rdi
         call  do_entry_int82
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+        jmp   compat_test_all_events
+#endif
 END(entry_int82)
 
 /* %rbx: struct vcpu */
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -723,6 +723,9 @@ END(common_interrupt)
 FUNC(entry_PF)
         ENDBR64
         movl  $X86_EXC_PF, 4(%rsp)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+        jmp   handle_exception
+#endif
 END(entry_PF)
 /* No special register assumptions. */
 FUNC(handle_exception, 0)
@@ -1023,8 +1026,11 @@ FUNC(entry_NMI)
         ENDBR64
         pushq $0
         movl  $X86_EXC_NMI, 4(%rsp)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+        jmp   handle_ist_exception
+#endif
 END(entry_NMI)
-
+/* No special register assumptions. */
 FUNC(handle_ist_exception)
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         SAVE_ALL
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -86,6 +86,9 @@ SECTIONS
        . = ALIGN(PAGE_SIZE);
        _stextentry = .;
        *(.text.entry)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+       *(.text.entry.*)
+#endif
        . = ALIGN(PAGE_SIZE);
        _etextentry = .;
 
@@ -214,6 +217,9 @@ SECTIONS
 #endif
        _sinittext = .;
        *(.init.text)
+#ifdef CONFIG_CC_SPLIT_SECTIONS
+       *(.init.text.*)
+#endif
        *(.text.startup)
        _einittext = .;
        /*
--- a/xen/include/xen/linkage.h
+++ b/xen/include/xen/linkage.h
@@ -19,6 +19,14 @@
 
 #define SYM_ALIGN(align...) .balign align
 
+#if defined(HAVE_AS_SECTNAME_SUBST) && defined(CONFIG_CC_SPLIT_SECTIONS)
+# define SYM_PUSH_SECTION(name, attr) \
+         .pushsection %S.name, attr, %progbits; \
+         .equ .Lsplit_section, 1
+#else
+# define SYM_PUSH_SECTION(name, attr)
+#endif
+
 #define SYM_L_GLOBAL(name) .globl name; .hidden name
 #define SYM_L_WEAK(name)   .weak name
 #define SYM_L_LOCAL(name)  /* nothing */
@@ -33,7 +41,14 @@
         SYM_ALIGN(align);                         \
         name:
 
-#define END(name) .size name, . - name
+#define END(name) \
+        .size name, . - name; \
+        .ifdef .Lsplit_section; \
+            .if .Lsplit_section; \
+                .popsection; \
+                .equ .Lsplit_section, 0; \
+            .endif; \
+        .endif
 
 /*
  * CODE_FILL in particular may need to expand to nothing (e.g. for RISC-V), in
@@ -47,6 +62,7 @@
 #endif
 
 #define FUNC(name, align...) \
+        SYM_PUSH_SECTION(name, "ax"); \
         SYM(name, FUNC, GLOBAL, DO_CODE_ALIGN(align))
 #define LABEL(name, align...) \
         SYM(name, NONE, GLOBAL, DO_CODE_ALIGN(align))
@@ -54,6 +70,7 @@
         SYM(name, DATA, GLOBAL, LASTARG(DATA_ALIGN, ## align), DATA_FILL)
 
 #define FUNC_LOCAL(name, align...) \
+        SYM_PUSH_SECTION(name, "ax"); \
         SYM(name, FUNC, LOCAL, DO_CODE_ALIGN(align))
 #define LABEL_LOCAL(name, align...) \
         SYM(name, NONE, LOCAL, DO_CODE_ALIGN(align))



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:37:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:37:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677564.1054234 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi6s-0000BD-FM; Wed, 07 Feb 2024 13:37:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677564.1054234; Wed, 07 Feb 2024 13:37:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi6s-0000AR-Ba; Wed, 07 Feb 2024 13:37:06 +0000
Received: by outflank-mailman (input) for mailman id 677564;
 Wed, 07 Feb 2024 13:37:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXi6r-00007V-Gq
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:37:05 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fb1c2f07-c5bd-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 14:37:04 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-33b0f36b808so485883f8f.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:37:04 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z12-20020a056000110c00b0033b4ebc3c8fsm1328999wrw.2.2024.02.07.05.37.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:37:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb1c2f07-c5bd-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707313024; x=1707917824; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Y42iGpChZ2+X/wNXWYFaYLa/ywLQuM+ANdY+in144ps=;
        b=d44bpYhtKeJew4D/SmArcLYoQG4aPDe9wAh1y4VcB2zTDoXkYUQl4XJIYZ0c4xcOh4
         AzUOMe17WRhoMEMyA/uwnyvuHhBwVmqrDCP3wONQwzdvlz3W6IneZw3fB9DC4Sc4se9r
         cGVQeb018pLHOsFLSYzmRq7CmpCJNwuhLZkI7LTdeEZhTLuwXNln7NZRmv7nXQ5SU9DB
         QFih9OYvtHIfH3naO44ZwICIWh5byCzCdiT9hZH8gPHXxaZRMxg93m55/cLdgOnHi2+y
         Ttbfuq5mIPiaX0JtEgjTzmMy3hLxkQtsy4hKJst6MnYjcqxPdKGfnsVmxIKaOtZIpAGg
         C7kQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707313024; x=1707917824;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Y42iGpChZ2+X/wNXWYFaYLa/ywLQuM+ANdY+in144ps=;
        b=Cbm5oJwWz5+as8uj6bacx4kW+kZmkLJ/MivaVI/3I6ZGrXaNAKJjuGlYzyCOTFGvKi
         MHNoe7ltIBu9RskWk+pcIrOp4TpDHhcfx5JcohcQKqA7WcwNLytB0LOZw6Gz6NFE4sjt
         3K/w5rMosXuwUoKrc77lLB/iLI8t79YLiosCrddWs2PQSXd8m4jmmApuevCtZ2E2wZfd
         pjBg9HMWx/rgHlr1aU2wBtLUlLOijwYItypWEvDZi86MYrqtXAE5YGu0lhoFxeZR2fvO
         A3slfkh9CVSYkwpYAtARa+ArvOEqVPe0JtaZGy0j/R8MtV3oaJrpiu84tYYrrvTO4nep
         D4/Q==
X-Gm-Message-State: AOJu0YxuRy8i17yZrgYJZSD4AAZTAJcOP4RkfLM33WwI/BcdOTVXoJfl
	Yfx9LLnQMc81qiBI/GbWgpn02baZByqVcnnvMA/QljAvtOT8QxUvzCewTb2HyTHn0YltsJVY7GQ
	=
X-Google-Smtp-Source: AGHT+IGqEj83M/D2sQzNE34YqcaSeXT63U2jm74FPmYefn3qGCzpuoeUh9bblWZ+cxInNq8cnsRdxw==
X-Received: by 2002:a5d:4b02:0:b0:33b:2c2a:934f with SMTP id v2-20020a5d4b02000000b0033b2c2a934fmr3907458wrq.44.1707313024408;
        Wed, 07 Feb 2024 05:37:04 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUDjdr0j/18mr6bOOe4ACIO6K6mAv7beJONhQU9byqRqWrNr6kVo35NwOvtTSCtDBbj3+umFFufEChtn8U6cOMN3v46yLOG8DnmdYv9i6dN8BbfEakK+M/vzjx00qfGLcqY210b7qPQiTVi
Message-ID: <4be8669e-bea7-49a7-9a56-f043c76d2f72@suse.com>
Date: Wed, 7 Feb 2024 14:37:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v6 2/7] SVM: convert entry point annotations
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Use the generic framework from xen/linkage.h.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v6: New.

--- a/xen/arch/x86/hvm/svm/entry.S
+++ b/xen/arch/x86/hvm/svm/entry.S
@@ -24,7 +24,7 @@
 #include <asm/asm_defns.h>
 #include <asm/page.h>
 
-ENTRY(svm_asm_do_resume)
+FUNC(svm_asm_do_resume)
         GET_CURRENT(bx)
 .Lsvm_do_resume:
         call svm_intr_assist
@@ -132,7 +132,7 @@ __UNLIKELY_END(nsvm_hap)
          * to safely resolve any Spectre-v1 concerns in the above logic.
          */
         stgi
-GLOBAL(svm_stgi_label)
+LABEL(svm_stgi_label, 0)
         call svm_vmexit_handler
         jmp  .Lsvm_do_resume
 
@@ -140,6 +140,4 @@ GLOBAL(svm_stgi_label)
         sti
         call do_softirq
         jmp  .Lsvm_do_resume
-
-        .type svm_asm_do_resume, @function
-        .size svm_asm_do_resume, . - svm_asm_do_resume
+END(svm_asm_do_resume)



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:37:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:37:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677568.1054244 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi7E-0000pK-Mx; Wed, 07 Feb 2024 13:37:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677568.1054244; Wed, 07 Feb 2024 13:37:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi7E-0000pD-KR; Wed, 07 Feb 2024 13:37:28 +0000
Received: by outflank-mailman (input) for mailman id 677568;
 Wed, 07 Feb 2024 13:37:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXi7D-00007V-44
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:37:27 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 079c2b0f-c5be-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 14:37:25 +0100 (CET)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-33b4e6972f6so428495f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:37:25 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z12-20020a056000110c00b0033b4ebc3c8fsm1328999wrw.2.2024.02.07.05.37.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:37:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 079c2b0f-c5be-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707313045; x=1707917845; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Y82MohMyrcgLEJtzsSnSC+KLl8n5qIEw59SpPMusgXQ=;
        b=AUhN+VJOqRSomyX1XdT0sXVjPEG5IJSfHLljALSePrOEe2lmMzl5RLEM3B+xB0rLv+
         qOYEY6tVW9OwvRwsUf3IJtU6ES8N2JajpZnKUjXZF2H4Xrc1d7ys+c33kIhauuYvJDZJ
         n2Sk9aA3rFkFAT4lpzMx0w4lZMFQwoaEHaZM7R2Wk5yKcCEQobS4jH0sV9wqbMiXaj/w
         0kRwncGmH5Wtew8AJG/1AK/aE4eQPA02ZamRHBZHuV5iCI77vfvwAF0ShJqLdoBKj9S7
         N+vRRbDxIlIH8+7RSF78vf0N5AqfcwTXm5cWGyf7JTyYoLLl5TfJCqArwHawO5wrgoke
         eHvQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707313045; x=1707917845;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Y82MohMyrcgLEJtzsSnSC+KLl8n5qIEw59SpPMusgXQ=;
        b=ls0qIPIjYH3GslGtlRVYvHSx1LpYHRQ4in49qn6P2cERw05nhafdqlbmB4PMhBhhSW
         Z0DaVzkBSwg545AucnXm87N52ckMtdrEUuHBF6Cnvstw32t4JbcJZX2QHBhovmWEKB4q
         eLqrDlT29IMiDbMDHBLBQzZmKvoyBO39ZarrVu8j0B8sFgHBydLvbGp4pwZofv4gUsbc
         oUNbeRntEljeKa6DF8XnAzmIWTg9hiWGczLptHkl3jm5jXq5NgXnAGFHtlJAYYFAMXiA
         bpfg3tckEHNXtbGUlmN5qfDij7LK5Y2koAb/TFiaAWc8/oRCzwPpS9gbznmG0Ue1gQ0m
         VtKg==
X-Gm-Message-State: AOJu0YyjUxxZfhp6CkYfe0x1Kgv4Rnip5DgKylsK76L35liai4GquQ3a
	EjW3jv2Bqo/R7H7nBSyDVoxQmzjGTI/+RMggkAaG2DTOtC96FYu6iUVNchIUmj5Y0uAYUdc99/0
	=
X-Google-Smtp-Source: AGHT+IFUN/uUEJTFN2yjSewFuBdgBOu537B/gJx5tTd1flGr1zhH3Prm3E38ihIpLr4G4/lXhZ0KUA==
X-Received: by 2002:a5d:6d88:0:b0:337:c454:81a8 with SMTP id l8-20020a5d6d88000000b00337c45481a8mr3845735wrs.55.1707313045482;
        Wed, 07 Feb 2024 05:37:25 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWgCqCvdwRAAtY3jDnLko+pXIr43NzcubyFc10vabDAJBB1OM609Uvscdtj9CEijIijeyy49nyWr3JVGcQR2xNtaDetaDEYK/TZMJuGerHmTXzEnJdYhO9KJFcWbumv7SOMrU0nwtnfIR8bH1RsKPtVfiSfMxG7ohnl+Xb8EXC4JR0y9r8Uu012OK9SBCuhkU8tyaB5HyPhZA==
Message-ID: <5fc304c0-be1f-46dd-a783-4030ec76a2f8@suse.com>
Date: Wed, 7 Feb 2024 14:37:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v6 3/7] VMX: convert entry point annotations
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Kevin Tian <kevin.tian@intel.com>,
 Jun Nakajima <jun.nakajima@intel.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Use the generic framework from xen/linkage.h.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v6: New.

--- a/xen/arch/x86/hvm/vmx/entry.S
+++ b/xen/arch/x86/hvm/vmx/entry.S
@@ -24,7 +24,7 @@
 #define VMRESUME     .byte 0x0f,0x01,0xc3
 #define VMLAUNCH     .byte 0x0f,0x01,0xc2
 
-ENTRY(vmx_asm_vmexit_handler)
+FUNC(vmx_asm_vmexit_handler)
         SAVE_ALL
 
         mov  %cr2,%rax
@@ -132,7 +132,7 @@ UNLIKELY_END(realmode)
         call vmx_vmentry_failure
         jmp  .Lvmx_process_softirqs
 
-ENTRY(vmx_asm_do_vmentry)
+LABEL(vmx_asm_do_vmentry)
         GET_CURRENT(bx)
         jmp  .Lvmx_do_vmentry
 
@@ -150,6 +150,4 @@ ENTRY(vmx_asm_do_vmentry)
         sti
         call do_softirq
         jmp  .Lvmx_do_vmentry
-
-        .type vmx_asm_vmexit_handler, @function
-        .size vmx_asm_vmexit_handler, . - vmx_asm_vmexit_handler
+END(vmx_asm_vmexit_handler)



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:37:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:37:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677573.1054256 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi7j-0001pO-0t; Wed, 07 Feb 2024 13:37:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677573.1054256; Wed, 07 Feb 2024 13:37:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi7i-0001pH-S3; Wed, 07 Feb 2024 13:37:58 +0000
Received: by outflank-mailman (input) for mailman id 677573;
 Wed, 07 Feb 2024 13:37:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXi7h-0000oU-Aj
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:37:57 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 195b303c-c5be-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 14:37:55 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-40f033c2e30so5412265e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:37:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z12-20020a056000110c00b0033b4ebc3c8fsm1328999wrw.2.2024.02.07.05.37.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:37:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 195b303c-c5be-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707313075; x=1707917875; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=YjmnSJwKzDpM0u5Pn+iHXjwC2VYerV7wNfCIIKzoSdk=;
        b=a/XQdWITxE3GaPINMKu5S+GDaSDA+tn5jzD+GuTUWliYQl9s5+NrwqYG5rCJwCaPda
         f6g/BbH/4WCvStQly0uIna22yFXHjUzaZxpaZ9Akw/zgmItM9kc3yfT/s3L2afMpHVV6
         uWYVwTIxnoJy2vn+VoS6T+i90k7qZn5kKAAyHeTIKkc598MW5Qcy69TtqDZIScdeljMt
         GBrJVxr9VAHX5y57yPiiN2XJ4rGXVggNDyciValHbbIYKHj30fBJEGqSuCmEIho5xmcS
         Kq9anCNVqaI7rcxTa+Clq9ayKbRpw6E9coF2F+kXYzMwZX6RzyNrCF4b07a9IEi/VK81
         ofrw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707313075; x=1707917875;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=YjmnSJwKzDpM0u5Pn+iHXjwC2VYerV7wNfCIIKzoSdk=;
        b=mQpKeqIJO/XsbDPaFR55janBjD1lVSqkAAV4l0XqaAAFlU2LMIz/l5x3hjAk4E7XLE
         s1jNP/R1CxJoWeFkI8N8U0gK7weoRU9MZmcXrWi7Tc+EacWUc0PU0WpjNQ9KgESibIqT
         Q8bpwsVhU0ciKcd419kooQNPmZxU6+xrgtiEubzW7ohCPyXhNW0rssuKo1SDqQtVorHt
         9CE4zvh+oDryfvJ6tYUIF/tSOhGZCHLCzzs33T+G3cKfHwAFitJTH43kyOpvjmTImkUU
         2DH4+mUnnm8YVnIG3GO20M/YiQWEFm+1JepoT7NDioxZ+KtMHIxriGvHZw7iEYwdT953
         LVEQ==
X-Gm-Message-State: AOJu0YwgGudOvkAgljd8xGw7UQaEAbd0Wgry7rWN8Y/itPgEnu9531Tl
	V5raSUlM++JXCgAM4BBjK2S0552FSAXs5Khqyv5eFWWEGUGxUJd+YCKoVOj35G0bDALSD30+ktg
	=
X-Google-Smtp-Source: AGHT+IFIHCSmXG0T6Q2bGWzEPV7V0mNOStg32L3BpcsG4d89iHLOwNJgqR5ZSsZMIA8DmFvMvAimpw==
X-Received: by 2002:adf:f410:0:b0:33b:3fe8:6413 with SMTP id g16-20020adff410000000b0033b3fe86413mr4042855wro.27.1707313075072;
        Wed, 07 Feb 2024 05:37:55 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXXpQ+thZHgfpMWIr7AbathifGsjYNkr2ViU0JJR0dAKbtDRfTUvw0iSXnDeixeMRkyHCKnIWgVcl6rcXp3APL74VONGyJ50qXo1p/d9EEZ+CvatvWYoNSKMznKtwYMq4Y2QFCCOEkpqVER
Message-ID: <cbc04b39-44cd-4f1f-a011-5d9d313f24d3@suse.com>
Date: Wed, 7 Feb 2024 14:37:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v6 4/7] x86/ACPI: annotate assembly functions with type and
 size
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Use the generic framework from xen/linkage.h.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
The .Lsuspend_err label is used in a cross-function manner here, but
it's not clear to me what - if anything - to do about this.
---
v6: New.

--- a/xen/arch/x86/acpi/wakeup_prot.S
+++ b/xen/arch/x86/acpi/wakeup_prot.S
@@ -7,7 +7,7 @@
         .text
         .code64
 
-ENTRY(do_suspend_lowlevel)
+FUNC(do_suspend_lowlevel)
         push    %rbp
         push    %rbx
         push    %r12
@@ -32,6 +32,7 @@ ENTRY(do_suspend_lowlevel)
 
         /* It seems we didn't suspend.  Get out of here. */
         jmp     .Lsuspend_err
+END(do_suspend_lowlevel)
 
         /*
          * do_suspend_lowlevel() is arranged to behave as a regular function
@@ -43,7 +44,7 @@ ENTRY(do_suspend_lowlevel)
          *
          * Everything else, including the stack, needs restoring.
          */
-ENTRY(s3_resume)
+FUNC(s3_resume)
         lgdt    boot_gdtr(%rip)
 
         mov     saved_cr0(%rip), %rax
@@ -132,6 +133,7 @@ ENTRY(s3_resume)
         pop     %rbx
         pop     %rbp
         ret
+END(s3_resume)
 
 .data
         .align 16
@@ -142,5 +144,4 @@ saved_cr0:      .quad   0
 saved_ssp:      .quad   0
 #endif
 
-GLOBAL(saved_magic)
-        .long   0x9abcdef0
+ASM_INT(saved_magic, 0x9abcdef0)



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:38:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:38:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677577.1054265 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi8G-0002SX-8P; Wed, 07 Feb 2024 13:38:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677577.1054265; Wed, 07 Feb 2024 13:38:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi8G-0002SQ-3k; Wed, 07 Feb 2024 13:38:32 +0000
Received: by outflank-mailman (input) for mailman id 677577;
 Wed, 07 Feb 2024 13:38:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXi8F-0002S8-Ku
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:38:31 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2d9b36f8-c5be-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 14:38:29 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-33929364bdaso429933f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:38:29 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z12-20020a056000110c00b0033b4ebc3c8fsm1328999wrw.2.2024.02.07.05.38.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:38:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d9b36f8-c5be-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707313109; x=1707917909; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=XZ8E4NCSVP15aCKtBvXW1WrLdgMyh/zRv3UNW3vt5Qc=;
        b=ejMoMTrlgFVyOskBb4n2ZZg2Gc0mJvo83SEoqeVY6SldeZx/eCDYkOppXuhkBDQ3QI
         pv/nOdVSvXlWmLWH1cw+7nPXDYMR7EaT5E0p+uzeAOyhK2jzo+7KhAy2UAjYw1LF+ofd
         rRIcbAS4XaVbahmtf6VfErYxM7YyOYvZbGL0DQY2v7IW554sp+ehuAS9MBsPaF+eX7Er
         SRqdYWXvdRdG7vv60QXnXLfOuapKMtrzOSWFo4Vf6lzwKTp36XfkWNXeXjPkYLga6K9u
         3ngDXeWF1C2aZtyP6ehovVcpOBoO1nq2TFP2LLqf3W5ng4w/FrE4s4yoNhBt9slFnU9V
         FBCA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707313109; x=1707917909;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=XZ8E4NCSVP15aCKtBvXW1WrLdgMyh/zRv3UNW3vt5Qc=;
        b=TBCQN1c/oI4PMzSLfxfYhJRCUEOgxNaswNPzwkRLRN3hknj10VS+/oiTdVDpkxCmJm
         3B0ykFs1XMyhfAMTS92bwMjZ0CBk/b4oookzxkgUGvH2Mhco/XsPmI2x/em6HhoUb7/G
         7T9ExA1C+wBadnZjTddMCP2xU8OREklX7oWeunKYIFVoQNCMhWYv3/SFF1DlrkMgpSJ2
         inHgQ6wrNh+Y+Cr8hIogY/NrnIK9+4ze6RT8PDNzXueii2ySkOhSwyDcDR1A+Tk2FNb/
         /Hh7SoIO1fPpUWTmbv7XM2m3U4HBtzuqvkZzyIMTZWHCixoVonQowLtSGcCDWzaeJmsz
         0HOQ==
X-Gm-Message-State: AOJu0YzmyjdfWZL6TBCxlBgmTVNu85bbfxXI2wsyh6iqdbtBXxAgUsqL
	FmtaoHTOgQyMGh5apZxr1GNJ21O3Ltlxf+CoFaYySeto4uZOOD0F/PVF1YlBPe7s22MkjBllX5Y
	=
X-Google-Smtp-Source: AGHT+IGR06n/2V+cnjT9rh5AURC/vp9SxGx6kbg+I20kRXd6mNNxweTRBk0aiO1Os0kvTevB74O8fA==
X-Received: by 2002:adf:ec84:0:b0:33a:e526:3a49 with SMTP id z4-20020adfec84000000b0033ae5263a49mr3451729wrn.23.1707313109062;
        Wed, 07 Feb 2024 05:38:29 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWAdXI5+RWS6xEYY7Cc2hdkIRfrrGk9gWx6/LEAc7GOqeFlyonCbnqtaapIcpNz61nL5szCdCqgqT0XgqtWnWG4wuDXyMo5e3brj26mBcIsMC+syEvtHPnxVot976HjBBmf8RcZ6By7MRL7
Message-ID: <50c0bcbb-e46e-4777-a7e8-20ee4b979b05@suse.com>
Date: Wed, 7 Feb 2024 14:38:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v6 5/7] x86/kexec: convert entry point annotations
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Use the generic framework from xen/linkage.h.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Using the linker script like this feels fragile. Maybe it's better to
suppress (#undef) CONFIG_CC_SPLIT_SECTIONS for this one file?
---
v6: New.

--- a/xen/arch/x86/x86_64/kexec_reloc.S
+++ b/xen/arch/x86/x86_64/kexec_reloc.S
@@ -21,10 +21,9 @@
 #include <asm/machine_kexec.h>
 
         .section .text.kexec, "ax", @progbits
-        .align PAGE_SIZE
         .code64
 
-ENTRY(kexec_reloc)
+FUNC(kexec_reloc, PAGE_SIZE)
         /* %rdi - code page maddr */
         /* %rsi - page table maddr */
         /* %rdx - indirection page maddr */
@@ -91,8 +90,9 @@ ENTRY(kexec_reloc)
         push    $0x10
         push    %rax
         lretq
+END(kexec_reloc)
 
-relocate_pages:
+FUNC_LOCAL(relocate_pages)
         /* %rdi - indirection page maddr */
         pushq   %rbx
 
@@ -138,10 +138,11 @@ relocate_pages:
 .L_done:
         popq    %rbx
         ret
+END(relocate_pages)
 
         .code32
 
-compatibility_mode:
+FUNC_LOCAL(compatibility_mode)
         /* Setup some sane segments. */
         movl    $0x0008, %eax
         movl    %eax, %ds
@@ -168,39 +169,29 @@ compatibility_mode:
         /* Call the image entry point.  This should never return. */
         call    *%ebp
         ud2
+END(compatibility_mode)
 
-        .align 4
-compat_mode_gdt_desc:
+DATA_LOCAL(compat_mode_gdt_desc, 4)
         .word .Lcompat_mode_gdt_end - compat_mode_gdt -1
         .quad 0x0000000000000000     /* set in call_32_bit above */
+END(compat_mode_gdt_desc)
 
-        .type compat_mode_gdt_desc, @object
-        .size compat_mode_gdt_desc, . - compat_mode_gdt_desc
-
-        .align 8
-compat_mode_gdt:
+DATA_LOCAL(compat_mode_gdt, 8)
         .quad 0x0000000000000000     /* null                              */
         .quad 0x00cf93000000ffff     /* 0x0008 ring 0 data                */
         .quad 0x00cf9b000000ffff     /* 0x0010 ring 0 code, compatibility */
 .Lcompat_mode_gdt_end:
+END(compat_mode_gdt)
 
-        .type compat_mode_gdt, @object
-        .size compat_mode_gdt, . - compat_mode_gdt
-
-compat_mode_idt:
+DATA_LOCAL(compat_mode_idt)
         .word 0                      /* limit */
         .long 0                      /* base */
-
-        .type compat_mode_idt, @object
-        .size compat_mode_idt, . - compat_mode_idt
+END(compat_mode_idt)
 
         /*
          * 16 words of stack are more than enough.
          */
-        .align 8
-reloc_stack:
+DATA_LOCAL(reloc_stack, 8)
         .fill 16,8,0
 .Lreloc_stack_base:
-
-        .type reloc_stack, @object
-        .size reloc_stack, . - reloc_stack
+END(reloc_stack)
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -92,7 +92,10 @@ SECTIONS
        . = ALIGN(PAGE_SIZE);
        _etextentry = .;
 
-       *(.text.kexec)          /* Page aligned in the object file. */
+       /* Page aligned in the object file. */
+       *(.text.kexec.kexec_reloc)
+       *(.text.kexec.*)
+       *(.text.kexec)
        kexec_reloc_end = .;
 
        *(.text.cold)



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:38:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:38:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677581.1054275 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi8b-0002y3-J7; Wed, 07 Feb 2024 13:38:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677581.1054275; Wed, 07 Feb 2024 13:38:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi8b-0002xw-G7; Wed, 07 Feb 2024 13:38:53 +0000
Received: by outflank-mailman (input) for mailman id 677581;
 Wed, 07 Feb 2024 13:38:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXi8a-0002S8-1A
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:38:52 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 39f6b48d-c5be-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 14:38:50 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33b1d7f736bso526476f8f.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:38:50 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z12-20020a056000110c00b0033b4ebc3c8fsm1328999wrw.2.2024.02.07.05.38.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:38:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39f6b48d-c5be-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707313130; x=1707917930; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=aBpoZSGBreowDhO2VF/qZrHAAevqhsPVyooMAssUAG4=;
        b=cCNK7Vm59Z9mYsC6ky0LYdvDjldX1QdIXbCpcodAtt3mZ6TiKqUBMXlQAQDAeo6y2s
         K/wXi6CTUr0u0DlZ2ZekFrzQG+0SsVh7uVfclDZBMy3L7ZyA6O2I2vBcW5hG2X5xbRKX
         IaaZBrnJvN7JMt1xZpKzzrLBFUmWgPdAINpfnXn7dSAmzKXpWr2GyvVNtXUEwN4CoUWZ
         bjKepR6sxN1dbz0CmA4YuFe9iDPfljExY6ySvxzs5ks9hyupgMFfvLnOJPgE5HMzsoEi
         t/3W98V7JRz2DNk+kfzZ3raIiujOf1dO0Rlc7hZb5Hn+Vfc/qMTVDuMCWZaCClSs0f0T
         WphQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707313130; x=1707917930;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=aBpoZSGBreowDhO2VF/qZrHAAevqhsPVyooMAssUAG4=;
        b=PowgE0pHl8BgdZCKtPZq9HUqFQ58J/anIOUozXS0HkTfidNt+76wRLgR2sL3qp0dkL
         Y/jokecZWwiG1eArsh4h1Ew6AECB90ubGjZQ1epw64nikhG9DvgZiG+wl686HG/CUIjx
         SLEgJh8NaqAV45L96M48gL0+yNqmLfvleToluELQ4ZhDRDeaMKOhH/IJfwJRSn9oTwGR
         0u5Ya+Eiszjd2JEpzdPJAm8JeKdoc4fuiFf8wFfjc2Gc8HP4wND3THWZReGBV21JDcsx
         o+wMpW7i9P8qx0JKgwngSjO7wTnSnkk/ckcbBzGEP6gmZHRThHZXG1BtM6YiOxHmYw16
         403A==
X-Gm-Message-State: AOJu0Ywr7xDswFqBW0/rEd+C6gSOf+m9l1Qne9epxWcELe4MnUlyvpfi
	ihwnM+m2RtBwJjiOVRT5Hfn/AaK6+Skv0SkkpvPb/GnT0jJkqixcDKnVUcCxT7bsUqugiA47TIo
	=
X-Google-Smtp-Source: AGHT+IGSFfXYOZS+6DKecgfnZokmK4eRCNKXhnba6Cmtvw/CVLUogCftvpuW1Dwd4a5QpImfyVFPZA==
X-Received: by 2002:a5d:468a:0:b0:33b:4382:c54 with SMTP id u10-20020a5d468a000000b0033b43820c54mr2970345wrq.62.1707313129835;
        Wed, 07 Feb 2024 05:38:49 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUxK+RGJLlk2fIeOCc43MT2CekBJEHPKSMDl3gENroZYzOzyMUa0IIWTboi+ObE+9JEuEB6PcJlIOzgRAK0W75rzR3tAQTAfspOzqfChrJlVA3f0t8iwHEpyqcy7JJgyPlY2VeufF5k4mjm
Message-ID: <4b0a581d-be2b-444d-a044-668b5e2e2279@suse.com>
Date: Wed, 7 Feb 2024 14:38:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v6 6/7] x86: convert misc assembly function annotations
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Use the generic framework from xen/linkage.h.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v6: New.

--- a/xen/arch/x86/clear_page.S
+++ b/xen/arch/x86/clear_page.S
@@ -1,8 +1,9 @@
         .file __FILE__
 
+#include <xen/linkage.h>
 #include <asm/page.h>
 
-ENTRY(clear_page_sse2)
+FUNC(clear_page_sse2)
         mov     $PAGE_SIZE/32, %ecx
         xor     %eax,%eax
 
@@ -16,6 +17,4 @@ ENTRY(clear_page_sse2)
 
         sfence
         ret
-
-        .type clear_page_sse2, @function
-        .size clear_page_sse2, . - clear_page_sse2
+END(clear_page_sse2)
--- a/xen/arch/x86/copy_page.S
+++ b/xen/arch/x86/copy_page.S
@@ -1,5 +1,6 @@
         .file __FILE__
 
+#include <xen/linkage.h>
 #include <asm/page.h>
 
 #define src_reg %rsi
@@ -10,7 +11,7 @@
 #define tmp3_reg %r10
 #define tmp4_reg %r11
 
-ENTRY(copy_page_sse2)
+FUNC(copy_page_sse2)
         mov     $PAGE_SIZE/(4*WORD_SIZE)-3, %ecx
 
         prefetchnta 2*4*WORD_SIZE(src_reg)
@@ -41,6 +42,4 @@ ENTRY(copy_page_sse2)
 
         sfence
         ret
-
-        .type copy_page_sse2, @function
-        .size copy_page_sse2, . - copy_page_sse2
+END(copy_page_sse2)
--- a/xen/arch/x86/guest/xen/hypercall_page.S
+++ b/xen/arch/x86/guest/xen/hypercall_page.S
@@ -3,13 +3,11 @@
 #include <public/xen.h>
 
         .section ".text.page_aligned", "ax", @progbits
-        .p2align PAGE_SHIFT
 
-GLOBAL(hypercall_page)
+DATA(hypercall_page, PAGE_SIZE)
          /* Poisoned with `ret` for safety before hypercalls are set up. */
         .fill PAGE_SIZE, 1, 0xc3
-        .type hypercall_page, STT_OBJECT
-        .size hypercall_page, PAGE_SIZE
+END(hypercall_page)
 
 /*
  * Identify a specific hypercall in the hypercall page
--- a/xen/arch/x86/indirect-thunk.S
+++ b/xen/arch/x86/indirect-thunk.S
@@ -35,15 +35,13 @@
 .macro GEN_INDIRECT_THUNK reg:req
         .section .text.__x86_indirect_thunk_\reg, "ax", @progbits
 
-ENTRY(__x86_indirect_thunk_\reg)
+FUNC(__x86_indirect_thunk_\reg)
         ALTERNATIVE_2 __stringify(IND_THUNK_RETPOLINE \reg),              \
         __stringify(IND_THUNK_LFENCE \reg), X86_FEATURE_IND_THUNK_LFENCE, \
         __stringify(IND_THUNK_JMP \reg),    X86_FEATURE_IND_THUNK_JMP
 
         int3 /* Halt straight-line speculation */
-
-        .size __x86_indirect_thunk_\reg, . - __x86_indirect_thunk_\reg
-        .type __x86_indirect_thunk_\reg, @function
+END(__x86_indirect_thunk_\reg)
 .endm
 
 /* Instantiate GEN_INDIRECT_THUNK for each register except %rsp. */
--- a/xen/arch/x86/pv/gpr_switch.S
+++ b/xen/arch/x86/pv/gpr_switch.S
@@ -10,7 +10,7 @@
 #include <asm/asm_defns.h>
 
 /* Load guest GPRs.  Parameter in %rdi, clobbers all registers. */
-ENTRY(load_guest_gprs)
+FUNC(load_guest_gprs)
         movq  UREGS_rdx(%rdi), %rdx
         movq  UREGS_rax(%rdi), %rax
         movq  UREGS_rbx(%rdi), %rbx
@@ -27,13 +27,10 @@ ENTRY(load_guest_gprs)
         movq  UREGS_rcx(%rdi), %rcx
         movq  UREGS_rdi(%rdi), %rdi
         ret
-
-        .size load_guest_gprs, . - load_guest_gprs
-        .type load_guest_gprs, STT_FUNC
-
+END(load_guest_gprs)
 
 /* Save guest GPRs.  Parameter on the stack above the return address. */
-ENTRY(save_guest_gprs)
+FUNC(save_guest_gprs)
         pushq %rdi
         movq  2*8(%rsp), %rdi
         movq  %rax, UREGS_rax(%rdi)
@@ -52,6 +49,4 @@ ENTRY(save_guest_gprs)
         movq  %rdx, UREGS_rdx(%rdi)
         movq  %rcx, UREGS_rcx(%rdi)
         ret
-
-        .size save_guest_gprs, . - save_guest_gprs
-        .type save_guest_gprs, STT_FUNC
+END(save_guest_gprs)
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -599,7 +599,7 @@ domain_crash_page_fault_0x8:
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         movq  %rsi,%rdi
         call  show_page_walk
-ENTRY(dom_crash_sync_extable)
+LABEL(dom_crash_sync_extable, 0)
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         # Get out of the guest-save area of the stack.
         GET_STACK_END(ax)



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:39:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:39:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677583.1054285 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi92-0003X2-RG; Wed, 07 Feb 2024 13:39:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677583.1054285; Wed, 07 Feb 2024 13:39:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXi92-0003Wv-Nl; Wed, 07 Feb 2024 13:39:20 +0000
Received: by outflank-mailman (input) for mailman id 677583;
 Wed, 07 Feb 2024 13:39:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXi91-0003WW-5C
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:39:19 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4a920ba9-c5be-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 14:39:18 +0100 (CET)
Received: by mail-wr1-x42c.google.com with SMTP id
 ffacd0b85a97d-33aea66a31cso412730f8f.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:39:18 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z12-20020a056000110c00b0033b4ebc3c8fsm1328999wrw.2.2024.02.07.05.39.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:39:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4a920ba9-c5be-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707313157; x=1707917957; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=ivg8xh0fXC1hxH27jiq7v5DWoiDVmbEsIYwYYVUpzS4=;
        b=Obgnlhr+f1zii/+AoaP0+kIaDXOavp/kmgUd0fuQwpRvGvscBnhDHuQTU8Ji1fz63r
         18n6NqP2G0KEPpKIPiZHlrOs5868yBl6Cj4czuFNRE0b/F/PbIloMZhW2mrNOn8yiwcA
         bnLBsWkMuVDiYmj/lbA6djNS0Enqm6t8S83C6nxPdmG4c/eOkxl4HmYCgzc8QKBqEZmH
         51odUB04VCewZANTdxd7N2r4plSiUHBniS3BqCYEXTzCUFX7GyAeq/bMubdbrnQFFq1L
         2iAr8oBWg692CRDUEftWx5zFjPRo2n809ICZnRkQJau+Iuc3tjLwuHPf4tU3W1tm77Q6
         oHmA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707313158; x=1707917958;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=ivg8xh0fXC1hxH27jiq7v5DWoiDVmbEsIYwYYVUpzS4=;
        b=FpDz14eODr9EKnF/opvADi0X5VyTK/VKH4nxAVZmHzzO8T+xNXWQLCeU/Ewip4Wb/4
         MAnM7WqyVTMXszkYX1Csa+Cmu01tqdYpM7ywG4P2xqBqL4VqcOqqto5U9SHWLjsT9iuK
         TtbbLbqJex2vjnUzOydojdrFpz3VOuE5MQZQXw+8vL5YRlXDi+/Ze8e0pgJxnmBzkKJE
         WFa2PJDI4obfXQeQc7WV4x6PeybrxfVcuouLbcVhzZdkQKmcp+OWGzyqCrMaN2gjc0kY
         Sfu/X/jzvtoJuCT52ynLGjMHV3FT+5LXmrsEiD3iaDchS8Xom/manFGBV+PThPpPCeyR
         ijGA==
X-Gm-Message-State: AOJu0Yyc3Zku/4YjhdsL9DRoUOnJ8+GZagq3ELpS5tzh9wf3N/JSOCBd
	7w6Q92Ds57qaajMZmmkCu4WoZT9zvQ/O4tqwvd3qPgNUKlSKqiQNIHXAygVDfU8Bf2xJlVuVA2c
	=
X-Google-Smtp-Source: AGHT+IHmlckhq6dGHNx2qssCV2e0/bynu+ZfTWcsbiepMstmyGqYumdOaKYcN9bbm1jkuYUM2agbWQ==
X-Received: by 2002:a5d:67d1:0:b0:33b:51d8:17d7 with SMTP id n17-20020a5d67d1000000b0033b51d817d7mr483477wrw.60.1707313157792;
        Wed, 07 Feb 2024 05:39:17 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXSnUDo8ufld7OrDNYBTm/ECVPFis4JzxZm2xiWvmlQlLYJxgTEiubKkMLhgYGe66j7m6cvF6G+f10orkR7VeNSWAx0sb/9K5lMdXHHlmumazQoy0FIq77hAjZyJKLAJ+qHMFDRZIkFgDcC
Message-ID: <64785449-1ca6-41f7-9012-2f432c171e3a@suse.com>
Date: Wed, 7 Feb 2024 14:39:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v6 7/7] x86: move ENTRY(), GLOBAL(), and ALIGN
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... to boot code, limiting their scope and thus allowing to drop
respective #undef-s from the linker script.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
An obvious alternative would be to convert boot code right away too, but
I think this has lower priority for now.
---
v6: New.

--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -10,6 +10,15 @@
 #include <asm/cpufeature.h>
 #include <public/elfnote.h>
 
+#define ALIGN .align CODE_ALIGN, CODE_FILL
+#define ENTRY(name)                             \
+  ALIGN;                                        \
+  GLOBAL(name)
+#define GLOBAL(name)                            \
+  .globl name;                                  \
+  .hidden name;                                 \
+  name:
+
         .section .text.header, "ax", @progbits
         .code32
 
--- a/xen/arch/x86/include/asm/config.h
+++ b/xen/arch/x86/include/asm/config.h
@@ -45,14 +45,6 @@
 #ifdef __ASSEMBLY__
 #define CODE_ALIGN 16
 #define CODE_FILL 0x90
-#define ALIGN .align CODE_ALIGN, CODE_FILL
-#define ENTRY(name)                             \
-  ALIGN;                                        \
-  GLOBAL(name)
-#define GLOBAL(name)                            \
-  .globl name;                                  \
-  .hidden name;                                 \
-  name:
 #endif
 
 #define NR_hypercalls 64
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -1199,7 +1199,7 @@ FUNC_LOCAL(autogen_stubs, 0) /* Automati
         .if vec >= FIRST_IRQ_VECTOR
 #endif
 
-        ALIGN
+        .align CODE_ALIGN, CODE_FILL
 1:
         ENDBR64
         pushq $0
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -5,8 +5,6 @@
 #include <xen/lib.h>
 #include <xen/xen.lds.h>
 #include <asm/page.h>
-#undef ENTRY
-#undef ALIGN
 
 #ifdef EFI
 



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:41:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:41:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677594.1054294 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiBA-00054R-6O; Wed, 07 Feb 2024 13:41:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677594.1054294; Wed, 07 Feb 2024 13:41:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiBA-00054K-3V; Wed, 07 Feb 2024 13:41:32 +0000
Received: by outflank-mailman (input) for mailman id 677594;
 Wed, 07 Feb 2024 13:41:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rXiB8-000548-9K
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:41:30 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rXiB7-0002QX-QF; Wed, 07 Feb 2024 13:41:29 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rXiB7-00082H-KO; Wed, 07 Feb 2024 13:41:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=zcOA5p3mqf4Dg5+yjoHYeX6pM7rYcgT5lxB287zKftI=; b=SkxGDrCG1Ijva3yNyyngIquqlt
	TiI7kMt5jMn9Ejr8v7Q4vk9i58O/MNZztTxb9PMWPC/mhKMIx5uIdhEk+MGtZxGmRk3st1KTthbDr
	N4f6hMFEDhg0VIei+Q1AQX3+6lmdIlrfv+NwPk6du9/y/m8B+IqueiGE78RJlR2Ui3JA=;
Message-ID: <13384e3e-c947-4cb3-a39a-b1ef801f0d12@xen.org>
Date: Wed, 7 Feb 2024 13:41:27 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] config: update Mini-OS commit
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20240207133106.6185-1-jgross@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240207133106.6185-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Juergen,

On 07/02/2024 13:31, Juergen Gross wrote:
> Update the Mini-OS upstream revision.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>   Config.mk | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/Config.mk b/Config.mk
> index f7d6d84847..077d841bb7 100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -224,7 +224,7 @@ QEMU_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/qemu-xen.git
>   QEMU_UPSTREAM_REVISION ?= master
>   
>   MINIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/mini-os.git
> -MINIOS_UPSTREAM_REVISION ?= 090eeeb1631f00a9a41ebf66d9b4aacb97eb51e7
> +MINIOS_UPSTREAM_REVISION ?= b119f0178fd86876d0678007dfcf435ab8bb7568

I looked at the changes and I am puzzled by one. In p9_stat(), I see a 
call with:

   free_stat(&stat);

'stat' is defined as 'struct p9_stat *stat' and 'free_stat' expects a 
'struct p9_stat'.

Wouldn't this resuilt to a build error?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:43:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:43:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677600.1054305 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiCh-0005ay-HR; Wed, 07 Feb 2024 13:43:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677600.1054305; Wed, 07 Feb 2024 13:43:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiCh-0005ar-Df; Wed, 07 Feb 2024 13:43:07 +0000
Received: by outflank-mailman (input) for mailman id 677600;
 Wed, 07 Feb 2024 13:43:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXiCf-0005af-Hj
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:43:05 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d161b6af-c5be-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 14:43:04 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-410177e41a8so3525465e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:43:04 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020a5d484a000000b0033b43d7394bsm1549474wrs.55.2024.02.07.05.43.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:43:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d161b6af-c5be-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707313384; x=1707918184; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=w35KGoAZ1nHlL9t8pX78QfY6RGIg1FbLlK/ZBQm81iA=;
        b=KYqyrHZ7QOm4nMnskFKNGUJKMbNMMScSOmjbYpxyQOONG8O8A3DUHGlBS8p8+U7XRm
         OIaFYSXsOYnIRvNMixgRY3Eb+q9lFPl5KHKY0USinnT1NRSRqNPU7ulEdivWP0PieCbt
         Go/cZvbAuKIS3iolBZkEPCuACen5ZGs+f9Jd+IIOuP9SPQh2IEcidIdyu5RIBWHU5oR4
         Y1dmIEomCehja46iiy7jX5EjDFDQqe6txff9sWGW3ZLZNORbZLo9cdelSwoxPcdrgoCW
         7WlnU0WUISC4kTTgnRbUBakXq/81iWkBEF4f7xklvoo8IxFj6XtS4I4qQ/w/Vmfh+NBp
         dgfA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707313384; x=1707918184;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=w35KGoAZ1nHlL9t8pX78QfY6RGIg1FbLlK/ZBQm81iA=;
        b=JouECdIEJucO0CbejGlbta6NOe0Q55uWCjdHcSm0kIFQiVhyUW4AVeW3/AxZPg25Lc
         XdoKipF4oxDZUwH7iAzVEihmfyk8gqVNa/o7Dh6o0OaYQUSW9Sj1Yl4HDAVYNGr1sWfT
         DUBvQkha3PlQ0BSLPTIn05QZZlNubRt5D4vMULr/whYxXMKsg18pVLMXjp7EIvaUX00V
         pQ1+Y3JG/LumuOiGZbZE3KWGkxKv2zE/DbE6QTbkkvL5NWFjPwS1rpXUUe1r5V3CpA/P
         PpHOp+IjPWhWds76d+6iaGBt690elyqESBdRaCWC9hsyWLqE7ElLH62sq44hiqc73+r/
         SRyQ==
X-Gm-Message-State: AOJu0Ywb3A/BC1M/X/8padBKN4GJjXwnTikII8LRspsdljn6Zf+1Dxhj
	ZqnnZ7owBMkk8QYdYEF2hnp0ytKziS6cPxQchRFZdhcZUG4awghssSUVgNmgTg==
X-Google-Smtp-Source: AGHT+IGhQkJZqlqi6yW+I6EXTHNWpdRdf9y0oYiaikIQg2MUuoXV5GZkrJjZvtBmPTS9Nnca+FKGOA==
X-Received: by 2002:a05:600c:3516:b0:410:273:c662 with SMTP id h22-20020a05600c351600b004100273c662mr1851611wmq.17.1707313383904;
        Wed, 07 Feb 2024 05:43:03 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWt84xqOnYEtxj/eC2S0h0GwYsOa0LvXNjrKpoDmgsIWZiod5YGAJ8XLegTHeRwf/w4SvfCK7iMZ62C8zAaGmdvMPt7HnJw52v+tA+g6ztEL+5h1XflorsaHFMJpfTPfMCq/f56cyyK0kagP6g3CmrRzucbn2AByBtuIorpDQrwxmeL/31drqczH41ExWdxGB/7JwM+dxCPbYpEsQY=
Message-ID: <9d96ef9c-1fc8-4405-914a-fcddb8ba1d4a@suse.com>
Date: Wed, 7 Feb 2024 14:43:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] config: update Mini-OS commit
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240207133106.6185-1-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240207133106.6185-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 14:31, Juergen Gross wrote:
> Update the Mini-OS upstream revision.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:45:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:45:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677605.1054315 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiFO-0006CC-UJ; Wed, 07 Feb 2024 13:45:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677605.1054315; Wed, 07 Feb 2024 13:45:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiFO-0006C5-R8; Wed, 07 Feb 2024 13:45:54 +0000
Received: by outflank-mailman (input) for mailman id 677605;
 Wed, 07 Feb 2024 13:45:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=D7mL=JQ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXiFN-0006Bz-Vu
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:45:53 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 35d76ce3-c5bf-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 14:45:53 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 547601FBEF;
 Wed,  7 Feb 2024 13:45:52 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 132D8139B9;
 Wed,  7 Feb 2024 13:45:52 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id JpoXA5CJw2V8IAAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 07 Feb 2024 13:45:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35d76ce3-c5bf-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707313552; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=btRrZ2d6dG4l77btXYu/GOJanzf/esV5Gu927V0bFqQ=;
	b=WVBtd10ThbwypMHzs1Jtko5BqmJ42eJZPdx68JxB1zIEwAMcMdld2cj5PQzBeCYzS4YdYc
	mZVo19TbaPOyB01O6J3xiMrI1cSX8LzVZp3gKfIuowS7o5k/35NZlF+BWN0MK+KdW2aBIc
	kWzqRqt4ZvYC89aPys7rqSnjdbyXECY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707313552; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=btRrZ2d6dG4l77btXYu/GOJanzf/esV5Gu927V0bFqQ=;
	b=WVBtd10ThbwypMHzs1Jtko5BqmJ42eJZPdx68JxB1zIEwAMcMdld2cj5PQzBeCYzS4YdYc
	mZVo19TbaPOyB01O6J3xiMrI1cSX8LzVZp3gKfIuowS7o5k/35NZlF+BWN0MK+KdW2aBIc
	kWzqRqt4ZvYC89aPys7rqSnjdbyXECY=
Message-ID: <fedffedf-0f0c-48f6-a1ea-5adfbf118257@suse.com>
Date: Wed, 7 Feb 2024 14:45:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] config: update Mini-OS commit
Content-Language: en-US
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20240207133106.6185-1-jgross@suse.com>
 <13384e3e-c947-4cb3-a39a-b1ef801f0d12@xen.org>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <13384e3e-c947-4cb3-a39a-b1ef801f0d12@xen.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=WVBtd10T
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-0.39 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-0.00)[43.99%];
	 MIME_GOOD(-0.10)[text/plain];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 NEURAL_HAM_SHORT(-0.20)[-0.992];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[xen.org:url,config.mk:url,suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(1.11)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[];
	 RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]
X-Spam-Score: -0.39
X-Rspamd-Queue-Id: 547601FBEF
X-Spam-Flag: NO

On 07.02.24 14:41, Julien Grall wrote:
> Hi Juergen,
> 
> On 07/02/2024 13:31, Juergen Gross wrote:
>> Update the Mini-OS upstream revision.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   Config.mk | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/Config.mk b/Config.mk
>> index f7d6d84847..077d841bb7 100644
>> --- a/Config.mk
>> +++ b/Config.mk
>> @@ -224,7 +224,7 @@ QEMU_UPSTREAM_URL ?= 
>> https://xenbits.xen.org/git-http/qemu-xen.git
>>   QEMU_UPSTREAM_REVISION ?= master
>>   MINIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/mini-os.git
>> -MINIOS_UPSTREAM_REVISION ?= 090eeeb1631f00a9a41ebf66d9b4aacb97eb51e7
>> +MINIOS_UPSTREAM_REVISION ?= b119f0178fd86876d0678007dfcf435ab8bb7568
> 
> I looked at the changes and I am puzzled by one. In p9_stat(), I see a call with:
> 
>    free_stat(&stat);
> 
> 'stat' is defined as 'struct p9_stat *stat' and 'free_stat' expects a 'struct 
> p9_stat'.
> 
> Wouldn't this resuilt to a build error?

Hmm, I'm puzzled, too. I didn't see a build error when doing "make testbuild".

But I agree that this looks very suspicious.

Oh, in test builds we don't have HAVE_LIBC defined. I just verified that
9pfront.c gets compiled, while I didn't pay attention to the parts which were
compiled during the build. Shame on me!

Fixup patch coming soon...

Thanks for noticing!


Juergen


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:48:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:48:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677615.1054325 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiHb-00075o-Ct; Wed, 07 Feb 2024 13:48:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677615.1054325; Wed, 07 Feb 2024 13:48:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiHb-00075h-AB; Wed, 07 Feb 2024 13:48:11 +0000
Received: by outflank-mailman (input) for mailman id 677615;
 Wed, 07 Feb 2024 13:48:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXiHa-00075M-BG
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:48:10 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 875a0f2e-c5bf-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 14:48:09 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-5116063585aso727783e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:48:09 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ga25-20020a170906b85900b00a389490cb05sm229112ejb.29.2024.02.07.05.48.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:48:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 875a0f2e-c5bf-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707313689; x=1707918489; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=d78TNworV8vLG7dKQT4LnDr7vWByqtZuJ/zxSC0ZijM=;
        b=nk3qRI5KwY+GBpB8nMACkcptLF3CxzMLiJ1Gqqmqn9up8PivUBK1Lc1+wYI5jeQmz+
         EhHAMdZZYFohaucWSXSeKk3ssFS2A/WPZktqRahZMsSY0zilUY7E1+TX20DQxuM1IHjJ
         NtgVnIFLUhGwBtxISP0n9kvQF+4vb2d/4/Dyg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707313689; x=1707918489;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=d78TNworV8vLG7dKQT4LnDr7vWByqtZuJ/zxSC0ZijM=;
        b=vhrBByWAeDAHEQW0/tP/ZmN93c+q5V0jBNP/w7C2vR5VaZD5RhUQPVPssLlcugEEyW
         /dci6BQ1J50NdKXd/oa9Urdy2nxrsaqGKBh/rPnzEgbxSTHJ9ceYR5u9MpRGtD5yd751
         v1riTIEZy75SNp6ZUeqG+KcdvsVmsA89fZBzS52aHpBBqf8hgQ3lWu/GchR6IYDntLYf
         b29Z/CGLngBZLa3Pl7oxZ89o8VXEAnCjLx6HoC7uMlRyPMZVGuQHMu6mUtzmFwdM1GBP
         ATB8LN00yGMs736uNFBOFEPqeSjC22twAKtMu7feBwFO+novsMPect6AYC9BTRAXre7L
         J9pA==
X-Gm-Message-State: AOJu0YzmlCW7Wr/jWu675JkHUK8HDUcxNFBa5TDy3d2xtfJfbuvSzT4k
	R7fv+TIkR3Bo3dQ88BejBBFxGEVcH/i1rLtTeeT8oj7olG8B+7xfZKF+aKDd3oY=
X-Google-Smtp-Source: AGHT+IHaiWj9h0SgoQs4TQq9G42u6Y8Xn78eiZYtRRUF7bb5JBPPG4bZamLU8N0x2URMeCeJEAKZOw==
X-Received: by 2002:a05:6512:12c2:b0:511:4e58:dd15 with SMTP id p2-20020a05651212c200b005114e58dd15mr4426104lfg.51.1707313689099;
        Wed, 07 Feb 2024 05:48:09 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXb2P4Vwf36Y3YJJygNorlZVHnhshr8FAVvqYgV2FNxveJBq843pzaw+l+qYbKM/v6Lx67dG5ufFm4e/6olTFfYiSLhld3nWiHasokI53Mic7aDmyCozrxrWi7oqkmYXBopi6sistkxQ/OWZ5h/hrAbC5/npVxhqLs8caJgmo0=
Message-ID: <d5af10fe-589b-406d-af8c-8638a93a9836@citrix.com>
Date: Wed, 7 Feb 2024 13:48:08 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 2/7] SVM: convert entry point annotations
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
 <4be8669e-bea7-49a7-9a56-f043c76d2f72@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <4be8669e-bea7-49a7-9a56-f043c76d2f72@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07/02/2024 1:37 pm, Jan Beulich wrote:
> Use the generic framework from xen/linkage.h.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:49:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:49:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677623.1054339 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiJ6-0007sI-O3; Wed, 07 Feb 2024 13:49:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677623.1054339; Wed, 07 Feb 2024 13:49:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiJ6-0007sB-LE; Wed, 07 Feb 2024 13:49:44 +0000
Received: by outflank-mailman (input) for mailman id 677623;
 Wed, 07 Feb 2024 13:49:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=D7mL=JQ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rXiJ5-0007ot-D2
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:49:43 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bccc34c4-c5bf-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 14:49:39 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id D322E1FBEF;
 Wed,  7 Feb 2024 13:49:38 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 93331139B9;
 Wed,  7 Feb 2024 13:49:38 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id qzFYInKKw2WFIQAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 07 Feb 2024 13:49:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bccc34c4-c5bf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707313779; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=qlPbapYX5kD+SBGGRiIEqtqGLDnstGGBMPRRuI48fEU=;
	b=hYp0rCdwQeQ/RQFWd0OrwmaNklD55xsppQ9GowUeL6zq6TAfjcff9qV7464eMhPrzLTP8s
	KrDnfmhMNv41e5vGdsBRzItn3n3nBJWOQjau6OV7cr31TOu6S+qkuDwl66+l/RSURzTQLX
	PrIdlQRoK+lJjk92K6Y3zFm6ladMFes=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707313778; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=qlPbapYX5kD+SBGGRiIEqtqGLDnstGGBMPRRuI48fEU=;
	b=KSsN49K88lDFt2bgPyaUVH8oz+jaYQgrkosDO9GktKdZfwETj+25w49wnp9qkreNFAE5GD
	8+TUPaCCVOGJnNk8g1OSYF847SFdVFH09DkFJZn+tEeu68pOnvsjG4XNtYChFyH9tJnuT0
	iIWJt5dVcVQuEfutXCSeFGg6Fxrgn1U=
From: Juergen Gross <jgross@suse.com>
To: minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org
Cc: samuel.thibault@ens-lyon.org,
	wl@xen.org,
	julien@xen.org,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH] Mini-OS: fix 9pfront fix
Date: Wed,  7 Feb 2024 14:49:20 +0100
Message-Id: <20240207134920.7341-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=KSsN49K8
X-Spamd-Result: default: False [4.59 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[6];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[xen.org:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.10)[65.58%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 4.59
X-Rspamd-Queue-Id: D322E1FBEF
X-Spam-Level: ****
X-Spam-Flag: NO
X-Spamd-Bar: ++++

There was a typo in the recent 9pfront fix.

Fixes: b119f0178fd8 ("Mini-OS: fix 9pfs frontend error path")
Reported-by: Julien Grall <julien@xen.org>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 9pfront.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/9pfront.c b/9pfront.c
index 042879a7..1741d600 100644
--- a/9pfront.c
+++ b/9pfront.c
@@ -729,7 +729,7 @@ static int p9_stat(struct dev_9pfs *dev, uint32_t fid, struct p9_stat *stat)
 
     ret = req->result;
     if ( ret )
-        free_stat(&stat);
+        free_stat(stat);
 
     put_free_req(dev, req);
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:51:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:51:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677630.1054349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiKu-0001R4-2w; Wed, 07 Feb 2024 13:51:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677630.1054349; Wed, 07 Feb 2024 13:51:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiKt-0001Qc-Vx; Wed, 07 Feb 2024 13:51:35 +0000
Received: by outflank-mailman (input) for mailman id 677630;
 Wed, 07 Feb 2024 13:51:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Py/x=JQ=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rXiKs-0001PJ-2X
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:51:34 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ffd88d3b-c5bf-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 14:51:31 +0100 (CET)
Received: from [192.168.1.15] (host-87-16-84-78.retail.telecomitalia.it
 [87.16.84.78])
 by support.bugseng.com (Postfix) with ESMTPSA id 39CAD4EE0738;
 Wed,  7 Feb 2024 14:51:31 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ffd88d3b-c5bf-11ee-98f5-efadbce2ee36
Message-ID: <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
Date: Wed, 7 Feb 2024 14:51:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US, it
To: Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 07/02/24 08:38, Jan Beulich wrote:
> On 07.02.2024 02:08, Stefano Stabellini wrote:
>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>       case 8:                                                                \
>>>>           put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>           break;                                                             \
>>>> -    default: __put_user_bad();                                             \
>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>       }                                                                      \
>>>>       clac();                                                                \
>>>>   } while ( false )
>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>       case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>       case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>       case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>> -    default: __get_user_bad();                                             \
>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>       }                                                                      \
>>>>       clac();                                                                \
>>>>   } while ( false )
>>>
>>> Related to my remark on patch 1 - how is one to know the macro this was
>>> invoked from, when seeing the resulting diagnostic?
>>
>> I am not sure what do you mean here... we do get an error like the
>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>
>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
> 
> Right - and how do I know what _user_ of the macro actually triggered
> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
> __FUNCTION__ here, for that specific purpose ...

To test the macro and its diagnostics,
I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
on the x86 code with STATIC_ASSERT_UNREACHABLE(),
that is in file arch/x86/alternative.c, line 312,
function _apply_alternatives().

What I got is the following build error:

...
arch/x86/alternative.c: Assembler messages:
arch/x86/alternative.c:312: Error: static assertion failed: unreachable
   CC      arch/x86/copy_page.o
make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1
make[2]: *** Waiting for unfinished jobs....
...

If I understood your requests correctly,
the only thing missing is the function name but I didn't find a way
to make __FUNCTION__ or __func__ work with the .error directive.
Do you know any tricks to make it work?

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:54:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:54:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677637.1054363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiNe-0002CL-G8; Wed, 07 Feb 2024 13:54:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677637.1054363; Wed, 07 Feb 2024 13:54:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiNe-0002CE-Cr; Wed, 07 Feb 2024 13:54:26 +0000
Received: by outflank-mailman (input) for mailman id 677637;
 Wed, 07 Feb 2024 13:54:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CAtM=JQ=ens-lyon.org=samuel.thibault@bounce.ens-lyon.org>)
 id 1rXiNc-0002AL-DC
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:54:24 +0000
Received: from sonata.ens-lyon.org (domu-toccata.ens-lyon.fr [140.77.166.138])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 64365d83-c5c0-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 14:54:20 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by sonata.ens-lyon.org (Postfix) with ESMTP id E1C85A02F3;
 Wed,  7 Feb 2024 14:54:19 +0100 (CET)
Received: from sonata.ens-lyon.org ([127.0.0.1])
 by localhost (sonata.ens-lyon.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id c8xwyZB29BqE; Wed,  7 Feb 2024 14:54:19 +0100 (CET)
Received: from begin (nat-inria-interne-52-gw-01-bso.bordeaux.inria.fr
 [194.199.1.52])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by sonata.ens-lyon.org (Postfix) with ESMTPSA id 9447DA02E8;
 Wed,  7 Feb 2024 14:54:19 +0100 (CET)
Received: from samy by begin with local (Exim 4.97)
 (envelope-from <samuel.thibault@ens-lyon.org>)
 id 1rXiNX-00000008oCE-0Z2A; Wed, 07 Feb 2024 14:54:19 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64365d83-c5c0-11ee-98f5-efadbce2ee36
Date: Wed, 7 Feb 2024 14:54:19 +0100
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org, julien@xen.org
Subject: Re: [PATCH] Mini-OS: fix 9pfront fix
Message-ID: <20240207135419.4upw334gz6trevwq@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org, wl@xen.org, julien@xen.org
References: <20240207134920.7341-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240207134920.7341-1-jgross@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Juergen Gross, le mer. 07 févr. 2024 14:49:20 +0100, a ecrit:
> There was a typo in the recent 9pfront fix.
> 
> Fixes: b119f0178fd8 ("Mini-OS: fix 9pfs frontend error path")
> Reported-by: Julien Grall <julien@xen.org>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

> ---
>  9pfront.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/9pfront.c b/9pfront.c
> index 042879a7..1741d600 100644
> --- a/9pfront.c
> +++ b/9pfront.c
> @@ -729,7 +729,7 @@ static int p9_stat(struct dev_9pfs *dev, uint32_t fid, struct p9_stat *stat)
>  
>      ret = req->result;
>      if ( ret )
> -        free_stat(&stat);
> +        free_stat(stat);
>  
>      put_free_req(dev, req);
>  
> -- 
> 2.35.3
> 


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 13:55:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 13:55:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677645.1054372 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiP3-0002ko-Pl; Wed, 07 Feb 2024 13:55:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677645.1054372; Wed, 07 Feb 2024 13:55:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiP3-0002kh-My; Wed, 07 Feb 2024 13:55:53 +0000
Received: by outflank-mailman (input) for mailman id 677645;
 Wed, 07 Feb 2024 13:55:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXiP2-0002kb-Mw
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 13:55:52 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9a2aa67a-c5c0-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 14:55:50 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-55783b7b47aso897589a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 05:55:50 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 v21-20020aa7d9d5000000b00560ada2df6dsm685560eds.45.2024.02.07.05.55.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 05:55:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9a2aa67a-c5c0-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707314150; x=1707918950; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=qZ+r8dd1O25QMrSpy4XNymKw2jbC121Nj8siOIzmWuQ=;
        b=T/N0pcIrSwBe2kdvGXThiIxF+Fjt5gjJXQJwVxdt5TsXNapWR4OBkCRNpYg2yHFSED
         tFIrY+LejbEsnWjsNaMfx9UrqQyS9dilKSDdiqn0uoHZ0Q2iEJv2KPI1zNzj6O955yL8
         PQjcMS74F7WMpK2oxO1b5jDcFnECiVXjrTeGo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707314150; x=1707918950;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qZ+r8dd1O25QMrSpy4XNymKw2jbC121Nj8siOIzmWuQ=;
        b=qGhR1WdxGmc5Yr+HbylqHoqnziwpYJ6GmG3pC53BvsOpICow05YD5NckqbLzDQm//Y
         W9lY0WMOLgTwH7lEmnr45lCEGGNDRZOG9oiEVgwxdZpJjEB4upGTzUUaSrgADIj6J0/t
         rh6cJMwrBU3SCek0qlT50aALloYVbZ7YwRW7PU4pm1U7ki6yvJk1DaPc5fmhrjBCnOAy
         aeS2x4X0XE3JOiMHi5dc5aW2GEzVaaYew9o7DrGfoB4ScxYinDu7KpWEGr4kiWB4Tgeu
         OSDoGQkRgy0L42FWHUOWlMRvKhKIYpGoCoeMeehKEtElFEj6mPTWZ5gfbFJrC5FeE+hT
         sMvg==
X-Forwarded-Encrypted: i=1; AJvYcCVOnCfQg1OaffMwY65pyaxeA0Sr6+Mc3/H1ftNbvHv8gsGHGPHCmv/Nuq63ZHihgnUzadco89A5aQOXmg7H/PirbUz8fkkKSv7JwLyY3J0=
X-Gm-Message-State: AOJu0YyuBF+ioiL8dnNEdxebK2faRftU1Uwiari7BdTCoQDuP0kXPJcV
	v8JkJ+n1apy6co9e/4aIOd0IBEKMxOEQV9VdZZFG6A6rpQL1e7cHoTy7nKzxzaP+qBd1hFx0QI8
	H
X-Google-Smtp-Source: AGHT+IFzA2Z9+raem4TmwhFovXnrhUQPRvfB1YkBIJ25NO5aOnL0y1mKqYXf27LSYBiFa9QcCbilPA==
X-Received: by 2002:aa7:df11:0:b0:560:8fb7:a721 with SMTP id c17-20020aa7df11000000b005608fb7a721mr4371322edy.13.1707314150173;
        Wed, 07 Feb 2024 05:55:50 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXpuRur2D4wGj3oiGpLI+90YZNb4PfccnuCJx30/sGSmjdniPiwPNWJ3XYIZoI/Snn7nw96IlSFU9z8Y+1JiULF0x4uqP+z7ktBk3O73EG0rmOESniMsvSR1MvCdkkE+PFVvkZLvQTVCHTNRfnKLRG6ezw3mr2EWQyYuOP4U3S7fIHQhcqw3U9CFgGJWLQvVnk1Yk/3GjgL2PK0nrajEuOm+yXlwxBe3oGZwpc1
Message-ID: <c11bd472-fc43-49c5-8482-4fc7ed836269@citrix.com>
Date: Wed, 7 Feb 2024 13:55:49 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 3/7] VMX: convert entry point annotations
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
 <5fc304c0-be1f-46dd-a783-4030ec76a2f8@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <5fc304c0-be1f-46dd-a783-4030ec76a2f8@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/02/2024 1:37 pm, Jan Beulich wrote:
> Use the generic framework from xen/linkage.h.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> v6: New.
>
> --- a/xen/arch/x86/hvm/vmx/entry.S
> +++ b/xen/arch/x86/hvm/vmx/entry.S
> @@ -24,7 +24,7 @@
>  #define VMRESUME     .byte 0x0f,0x01,0xc3
>  #define VMLAUNCH     .byte 0x0f,0x01,0xc2
>  
> -ENTRY(vmx_asm_vmexit_handler)
> +FUNC(vmx_asm_vmexit_handler)
>          SAVE_ALL
>  
>          mov  %cr2,%rax
> @@ -132,7 +132,7 @@ UNLIKELY_END(realmode)
>          call vmx_vmentry_failure
>          jmp  .Lvmx_process_softirqs
>  
> -ENTRY(vmx_asm_do_vmentry)
> +LABEL(vmx_asm_do_vmentry)

This really is a function, not a label.

xen.git/xen$ git grep vmx_asm_do_vmentry
arch/x86/hvm/vmx/entry.S:135:ENTRY(vmx_asm_do_vmentry)
arch/x86/hvm/vmx/vmcs.c:1855:void noreturn vmx_asm_do_vmentry(void);
arch/x86/hvm/vmx/vmcs.c:1929:    reset_stack_and_jump(vmx_asm_do_vmentry);

It is giant mess, of two functions forming part of the same loop.

Considering that you declines to take CODE, I don't know what to
suggest.  The point of CODE, distinct to FUNC, was to identify the
places where weird things were going on, and this absolutely counts.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:00:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:00:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677657.1054382 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiTi-0005No-EV; Wed, 07 Feb 2024 14:00:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677657.1054382; Wed, 07 Feb 2024 14:00:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiTi-0005Nh-BZ; Wed, 07 Feb 2024 14:00:42 +0000
Received: by outflank-mailman (input) for mailman id 677657;
 Wed, 07 Feb 2024 14:00:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXiTg-0005NZ-P8
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:00:40 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4692fa9d-c5c1-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 15:00:40 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a389ea940f1so9533966b.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:00:40 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 b27-20020a170906491b00b00a36d0dae491sm782384ejq.202.2024.02.07.06.00.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:00:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4692fa9d-c5c1-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707314439; x=1707919239; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=l1r7EhfbR/xK2yfox+JQNnX9ej4QNRq+KOLLJ9kK4n4=;
        b=ES6hX9IQwbCmuz8oVzSHiVYR6+oBajx6Fh0IDmKIMJ15q0xx1gbbt/CPzjPlKmW3M9
         u2h9E/uPpJZMEpBOABuDsnQJxrVCL4denfPBMd3zLVjMT/q77JcKW/Li5v6sYHZFK4jx
         eg7wItVVv5Axlct/AHE35EtBtO4dqLapE8B00=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707314439; x=1707919239;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=l1r7EhfbR/xK2yfox+JQNnX9ej4QNRq+KOLLJ9kK4n4=;
        b=BaZui8/D5gV9nc2cK/V3NiGX1NYVio2MakHU3oeZUEoNRuq3HqF+FIyO619Bz2X+gg
         AHIkav3CEQynu2LyaBDAUvWTmmFyOsfpsUZf1p9qUmXMDgZJNr+GiugivpTK0v+NYCf2
         sBBhNfSRvloHVE6RCz5wVUNTh1fDV5EhnOG8hPeQ1H3cOFBgwVX7PnFUBJK4WpYJjBAi
         WuKJbwDavl7oF8mmOAIicHEnKcKWzMKs2HFz56QyrmV4p2mFt2C5DN7HyoSl7B/AyrP0
         hfy/gANeQaseFEHaKpVwNyWVzThaSgwbn/SxRX0/XCBmNUdPH0VGMQ45dxeeuS4K+SIv
         pwqA==
X-Gm-Message-State: AOJu0YyFH6nPmQTmaPUozNwa1COirEsQmyF9YSpCwUQVkRMmDLUV8emb
	39hlj+ig7MzEGzdJOOmFstVaRf40FsnnvH9tM+gf5MdIuBpRFsdUXlOHNajR1qI=
X-Google-Smtp-Source: AGHT+IGP4tquwo6XGZR5eSrlKHi1uMaXfR4Gov7uml9Ht09snch2mEpLBPixsSRik8rARAoacJnkQg==
X-Received: by 2002:a17:906:3ca1:b0:a37:8a2c:316a with SMTP id b1-20020a1709063ca100b00a378a2c316amr3697548ejh.45.1707314439324;
        Wed, 07 Feb 2024 06:00:39 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXzSz0REHTRSob5wtiwAeffFEo7NSiBpdeOrqf/fsf1NzF1z8mCWQiOg1uXjD7ZYIHvVuLNHBbjwRhAEqUpaCz17mOAjnmWUrHyUKCJP3otfOsjIr0PzT4bYBNfD84rKosAH0fk1kEsWWaeuLUbRWOaRiOSXZaAK5AbEm+WW0s=
Message-ID: <0744256b-4584-425f-a0cb-48a84d3a7b8f@citrix.com>
Date: Wed, 7 Feb 2024 14:00:38 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 4/7] x86/ACPI: annotate assembly functions with type
 and size
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
 <cbc04b39-44cd-4f1f-a011-5d9d313f24d3@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <cbc04b39-44cd-4f1f-a011-5d9d313f24d3@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07/02/2024 1:37 pm, Jan Beulich wrote:
> Use the generic framework from xen/linkage.h.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> The .Lsuspend_err label is used in a cross-function manner here, but
> it's not clear to me what - if anything - to do about this.

Well - again like VMX, this is special CODE.

It really is one function, do_suspend_lowlevel() - see how it's caller
uses it, with a magic position in the middle that is joined by the S3 path.

s3_resume should be a CODE too, or whatever we call a not-LABEL.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:03:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:03:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677660.1054392 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiWk-000656-RT; Wed, 07 Feb 2024 14:03:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677660.1054392; Wed, 07 Feb 2024 14:03:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiWk-00064z-Os; Wed, 07 Feb 2024 14:03:50 +0000
Received: by outflank-mailman (input) for mailman id 677660;
 Wed, 07 Feb 2024 14:03:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXiWj-00064q-3p
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:03:49 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b5cee9d4-c5c1-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 15:03:46 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-40fff96d5daso5694705e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:03:46 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l9-20020a5d6749000000b0033b47cf3221sm1567726wrw.35.2024.02.07.06.03.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:03:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5cee9d4-c5c1-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707314626; x=1707919426; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=kF60STwMtJ+pWFIeKlfRnOGuA1FbXzU7nSkrXcE9m2w=;
        b=Ud7kbpL+hHMY05dHzebgmCAnHRkI4ONCZDa0vP1hmRLetFid2R8n+hkHXEYlYjoBaN
         orMLPlGaXuo15VG0iXd4yAUtsQhkDBeg/QVeID9qEaS/+h/NNTbjgL4kzlG1m9uhVcFW
         EhCLHq7urMtMa7U0Lk6GAiGNp6e6G/DBfQnLED6zcvQ37dES2sgR175Q20MwkTfdlTOI
         hND7FrB5RYWgNHT8wOibfA9hegzlmkof3llbhUUz7ByGFYBmZu1mETy2rTYuJcZzpRys
         icgoc9RKvVgRteAZDUrTS70pa7gZ/LHU3DLiFgBmEqT7vtjKtLgJ4VGIffG79easLtSk
         tmCg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707314626; x=1707919426;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kF60STwMtJ+pWFIeKlfRnOGuA1FbXzU7nSkrXcE9m2w=;
        b=kPD4/I5Ek6eIGq0jNKUawaLzbq7aVYEwt5C+mwlzfSRh7WXYk+cPNzckmTrIuYYsLm
         tDov0FUp3yfVLpuz0uN7q27G06UV5QvQESHDaEtyYGPoMMOFbxQ2TymvGcI/ed2RN5Vv
         TuRxEokgweYaY3p9rJXZ4lGvtxs0OxnJ5LTKK1AFyOpGu1PjNc/3Fri+wMZuAwFOgJkh
         zMzHCEY6yhaymi4ydzjEFgAk9Nj0pYDFqwvVVjMlSyuthetNBlK7aezEMJ0j0VwlrhuI
         Cs1un0CDSnCNIh7KVZi0hI1xgS5ExgZAP/MAnKbrxZ2VjMS4HQdT9r/mU4y8ZEWzkA/V
         bayg==
X-Gm-Message-State: AOJu0YztkwMuvEy4y+fmXkAC1xhNrF9wKUxX2BdBtPwiU+MzSvbJZA9A
	zLOIssNlGKvhUZm1+a49b5G514t/TyambjYhe332N3hUDuXVacLCCnjyiHKIvxtYG21si/djYNY
	=
X-Google-Smtp-Source: AGHT+IGTnWuDDeU5k+oFyd0GTXnI0JwIfeo8H3ha7RzKP9kwHcTkGjL91ibaL6g1s7z7ji/6Ys+oqQ==
X-Received: by 2002:a5d:658f:0:b0:33b:4e7c:8230 with SMTP id q15-20020a5d658f000000b0033b4e7c8230mr1196474wru.54.1707314626012;
        Wed, 07 Feb 2024 06:03:46 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCV58WzfdY/oXnfAxkJ4knywwEYdvpJBRsmJDsln/u7fqbFu9JrRhi+Ow005itjCHt5ch0oMfByC4v4/7xVNV3K4SwoMe8gJQMAL4WTJYbghQg==
Message-ID: <90070a25-445f-4b82-be19-d87b25fc9b8c@suse.com>
Date: Wed, 7 Feb 2024 15:03:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] hvmloader: suppress GNU ld 2.39 warning about RWX load
 segments
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

This aspect is simply meaningless for this component.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/tools/firmware/hvmloader/Makefile
+++ b/tools/firmware/hvmloader/Makefile
@@ -20,6 +20,8 @@
 XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/firmware/Rules.mk
 
+ld-option = $(shell if $(LD) -v $(1) >/dev/null 2>&1; then echo y; else echo n; fi)
+
 # SMBIOS spec requires format mm/dd/yyyy
 SMBIOS_REL_DATE ?= $(shell date +%m/%d/%Y)
 
@@ -59,6 +61,10 @@ ROMBIOS_ROM := $(ROMBIOS_DIR)/BIOS-bochs
 ROMS += $(ROMBIOS_ROM) $(STDVGA_ROM) $(CIRRUSVGA_ROM)
 endif
 
+# Suppress the warning about LOAD segments with RWX permissions, as what we
+# build isn't a normal user-mode executable.
+LDFLAGS-$(call ld-option,--warn-rwx-segments) := --no-warn-rwx-segments
+
 .PHONY: all
 all: hvmloader
 
@@ -85,7 +91,7 @@ $(DSDT_FILES): acpi
 build.o: $(DSDT_FILES)
 
 hvmloader: $(OBJS) hvmloader.lds
-	$(LD) $(LDFLAGS_DIRECT) -N -T hvmloader.lds -o $@ $(OBJS)
+	$(LD) $(LDFLAGS_DIRECT) $(LDFLAGS-y) -N -T hvmloader.lds -o $@ $(OBJS)
 
 roms.inc: $(ROMS)
 	echo "/* Autogenerated file. DO NOT EDIT */" > $@.new



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:05:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:05:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677664.1054403 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiYd-0006bS-66; Wed, 07 Feb 2024 14:05:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677664.1054403; Wed, 07 Feb 2024 14:05:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXiYd-0006bL-3S; Wed, 07 Feb 2024 14:05:47 +0000
Received: by outflank-mailman (input) for mailman id 677664;
 Wed, 07 Feb 2024 14:05:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXiYb-0006bF-Pd
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:05:45 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fc51d16a-c5c1-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 15:05:45 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-560f37e2d9bso6011a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:05:45 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 k9-20020a50c8c9000000b0056010210438sm699694edh.32.2024.02.07.06.05.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:05:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc51d16a-c5c1-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707314744; x=1707919544; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cba5wmsZcKD/2MZ7axaD0kSURXsy5xMPmG6QLNQZh/8=;
        b=m1bU1rYYBLGv8k+KN+2c923/e7FxmYYxOy+rU5pipYI5mTP12yIPrGql+KfZ6yQ79e
         Dt5H/QIrc0qktLWDQ5JuNhksiYBJz+OxHcmKujE9cfErs5QkMiQ10csQ++/4bESvSnld
         lGSb9wfM7eZk8MRbs568iPQYGnDoSLF10azzk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707314744; x=1707919544;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cba5wmsZcKD/2MZ7axaD0kSURXsy5xMPmG6QLNQZh/8=;
        b=hBB0egXOz6W/EMtpa5BqitXTpdrJD8cBjg7klfxuk6zufrAsU0+F7maMnbtxaK0vqh
         Wk/5g8hHuWzB4b8uuG0NOAti7rc+eOR6gCTJ4KeMw6BbqmECSVmySdVNY5Ybkj5eQyTW
         gmd54s7RLwL8dZRq0tzJM3+n/35MOaoncANMovVWUUrGJz8a3qGA0ridNKtjLGP5S/hF
         jwz4rA7dSASJ7shNtJP060u3qvkaHw70nE3R/yyFaPhc63PhwTYC4zCm9Y1MYn217BX2
         hn8xtX4c57V3XLEc8kbmiJEz8Md1XcBdh6MYDpVRkYuml9a6h8CuY2uKmNQIzy5nvcfn
         FgMQ==
X-Gm-Message-State: AOJu0YyRjyqqlqq/fQHipKDROLtBIm/qCwhvD0DoBxKKvERiVDgD8MKK
	fAhx+bLzOc8FgO9N0We+1Oow4Mkcr3gKReG5KxXjdZr+lkLaWW3e1Djue8lQrB8=
X-Google-Smtp-Source: AGHT+IGwk8RI+ApuOQB5GgTbsaGslyAnwWbcGkRvbyCQx3WpGeOO5xfHmIMUrm9GAyhwWIKlDONOvA==
X-Received: by 2002:aa7:cf03:0:b0:55f:4602:bf84 with SMTP id a3-20020aa7cf03000000b0055f4602bf84mr3684514edy.27.1707314744408;
        Wed, 07 Feb 2024 06:05:44 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWKxsHJki0vf10c/SUn4Vd62+l2bOliMVg9WQrAt+3Hb38/vqNC8G+KhJrd+79eBHR+c4b3f3oZ7pGX7ouVkYhYImsc6RcZfNZrqei4YE/GvjZvT6dXDwaAnNc4gohXM+r/iKuiBIWdQTeyk2Dgl4scZxzL4S6mWA12TAFlshw=
Message-ID: <a0003ed2-1f1e-431a-a12b-2717d267503f@citrix.com>
Date: Wed, 7 Feb 2024 14:05:43 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 5/7] x86/kexec: convert entry point annotations
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
 <50c0bcbb-e46e-4777-a7e8-20ee4b979b05@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <50c0bcbb-e46e-4777-a7e8-20ee4b979b05@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/02/2024 1:38 pm, Jan Beulich wrote:
> Use the generic framework from xen/linkage.h.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> Using the linker script like this feels fragile. Maybe it's better to
> suppress (#undef) CONFIG_CC_SPLIT_SECTIONS for this one file?

This is specific glue code, needing to fit in a single page.  I'd prefer
to explicitly state that SPLIT_SECTIONS is inapplciable, than to try to
undo the effects in the linker script.

Everything else looks ok.  I'll fix the preexiting errors after a rebase.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:11:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:11:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677671.1054413 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXidp-0008HT-Qi; Wed, 07 Feb 2024 14:11:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677671.1054413; Wed, 07 Feb 2024 14:11:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXidp-0008HM-N3; Wed, 07 Feb 2024 14:11:09 +0000
Received: by outflank-mailman (input) for mailman id 677671;
 Wed, 07 Feb 2024 14:11:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXidn-0008HG-RR
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:11:07 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bbb0c253-c5c2-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 15:11:05 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-511490772f6so734547e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:11:05 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 jl13-20020a17090775cd00b00a35cf6727c1sm793868ejc.105.2024.02.07.06.11.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:11:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bbb0c253-c5c2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707315065; x=1707919865; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=XRsra38j88U49UA+I68ks0cCTPJ89STFRWfeESpYjDY=;
        b=aJC0Jgtv+EhRNWbkwVCbE3RkZzEwUO5sVvYNKbKATYef52BocLuvgGnNpL6E0F6++1
         RytnzolylJjKV2oT+0D+0V9WHqtm6Ag8l6a49sMqwb6az4bTVLsaZJZkktSuDmvMgwrd
         BDdsOqa9/+VZ2ghgKBowKAyGIBKl6siUCltEQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707315065; x=1707919865;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=XRsra38j88U49UA+I68ks0cCTPJ89STFRWfeESpYjDY=;
        b=hY1PyWvEJkABb1MmOlZTsyIDYiN/IPEZijOE2TPNW4WHuWDuJTQL5FosOrlZsoBvuK
         Bs5bA8eamW8vhL/0tF3m+DS3m4H7pRdR2c/9S0KcC/2UA1JreDtXO+N0ASEG/2e46QK9
         SrKNVw1gLyfqxYcRKuoOuyaPa4UqqbvoXtAwEdW7l/Hd1XZFukt9eErDYRJygzBaCmOP
         +SgsutgEZt5yk1b1XcXtJHOpL34+L/rM4vE3gJPp/yYCxlChIfCijlvSBFPMoA3Hlfz9
         61nZUJMfQ0o7uguSNBpgCtLqOb0mg7JL5QoGHT+RgHr6W+BeT1cMKcCpdORcT9yFahno
         m1+Q==
X-Gm-Message-State: AOJu0YxC9sMiRfCbwkjTIbjJfnF9YmiCd8RaE7rBD5An3Cm4ehwi5Qi5
	ljCErsjhqP/KclrwQoK++cYkG3wYfprfsCQoHu4sUBXnEh8GhvzTFWc/9HRq6Nc=
X-Google-Smtp-Source: AGHT+IHW0GCqKtA46+FINbW52jjTqFmn0YG3NwAKTcUxwv2TxPPF4v+Jf7y7P2Okan4odSOsjNGuTg==
X-Received: by 2002:ac2:4832:0:b0:511:4268:3a54 with SMTP id 18-20020ac24832000000b0051142683a54mr3481511lft.29.1707315065299;
        Wed, 07 Feb 2024 06:11:05 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX17A/Au5nwZudVU5slV/eTKOiHK3FFZ/eosqwT4Cvd0tcU00Q9FhFJpdDuplbZI7zaW24qHRXZje8BxkCYyWF+gi/HuBWnzlRt42WzTHwWuTYx0mKk50YdKBZK55v6mIXN7N5sG5i4B08hPSIQnUKdg8fMfj41Y6IZRxwTwWM=
Message-ID: <4784f1e9-2415-45cc-ac1f-561cefb3d5c1@citrix.com>
Date: Wed, 7 Feb 2024 14:11:04 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 6/7] x86: convert misc assembly function annotations
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
 <4b0a581d-be2b-444d-a044-668b5e2e2279@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <4b0a581d-be2b-444d-a044-668b5e2e2279@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/02/2024 1:38 pm, Jan Beulich wrote:

> --- a/xen/arch/x86/x86_64/entry.S
> +++ b/xen/arch/x86/x86_64/entry.S
> @@ -599,7 +599,7 @@ domain_crash_page_fault_0x8:
>          ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
>          movq  %rsi,%rdi
>          call  show_page_walk
> -ENTRY(dom_crash_sync_extable)
> +LABEL(dom_crash_sync_extable, 0)
>          ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
>          # Get out of the guest-save area of the stack.
>          GET_STACK_END(ax)
>

This again is a function, and one even used across-TUs.

Furthermore, it's a (domain) fatal error path.  It has the least excuse
of all to not conform to a regular function-like layout.

Everything else looks fine.  If you want to split this out into a
separate patch to address its function-ness, then consider the remainder
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:16:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:16:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677677.1054422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXijJ-0000wL-G9; Wed, 07 Feb 2024 14:16:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677677.1054422; Wed, 07 Feb 2024 14:16:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXijJ-0000wE-DZ; Wed, 07 Feb 2024 14:16:49 +0000
Received: by outflank-mailman (input) for mailman id 677677;
 Wed, 07 Feb 2024 14:16:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXijI-0000w8-E5
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:16:48 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 868379c8-c5c3-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 15:16:46 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-4101d096c92so2789795e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:16:46 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f18-20020a05600c4e9200b0040fe0bb761dsm5438894wmq.16.2024.02.07.06.16.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:16:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 868379c8-c5c3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707315406; x=1707920206; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=u0h55+3hzBq5cmd5rCMnlrIJBOEVlByJPH9Vvdn2WXo=;
        b=cwqrd3VQQslY/HVCWyu8bRpQl6Pm0Ox9rmJWmP3ucFF0F0Y2ZVpIRw1GhN0U6I1T62
         K5AfhJAMPwaOMHGYqLctNVPTNQFoCq58aFmFpQTQNDnuWDFHjkUBJVX7O3nocLPDzmuW
         XXUkKF2s5x5Ed+/QDnfVMQ6+ffKqMUrAasM1CZiQig1AfnzP42pYbPKS2UwvwXwuv8/g
         GGbiuiwF0XoHwGV7SxbNz48zenyh1rLNUJiwlYyc1VVt2B1H6oUhPIgJ3xo6VyeQe3Sy
         awyQ9IKEwgjh0Zdyzi4zT9oIh0bpG2rdBaZgl3YGm0vkW08wrxpJIdbgzjayOxsKFHFG
         jdJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707315406; x=1707920206;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=u0h55+3hzBq5cmd5rCMnlrIJBOEVlByJPH9Vvdn2WXo=;
        b=IoFTyjkoBfvrKLYtIm/6mniV5TgvikX+1Fnh9hHp0Je6NG+b35t9FQL7lWA/aNrQJY
         b073GyqGr6wmY1VUig6rWECThgqS64UHLt5aioihCPGan5HSbH9gW4LQNF+9DmyI3pH5
         Olg1nO7LGLdoRPlh9NQQBzKjo903eGvFQeK8B0avFkdnJbf0nExRMEM+0mmkdE52zyFE
         h5EyS8n6wowmOf+NED/URjBDpeisSMS0G7Z8IPae+xMuct7FIQr60vpXV66SilVyITO4
         QH8DlYVcsIJtfez1tSWGSfIGABAXcDxmVGLTe9t0BkROXeuzgiXFGT40Q701ar13TLKw
         qRpQ==
X-Gm-Message-State: AOJu0YxwJ6u+nA5c8+wvO6sJSmIIRQPI5KeACMNHryW6JvBmpKGMnH5d
	OjRcLaYdb45TqFYj6dGgCTF8BzMwTs/Hhz3yIfqbqXv5AjMsUaMJb2wY5D1Bqw==
X-Google-Smtp-Source: AGHT+IF4cyD38LO7MPpt/li4lyXfPo3ydSShEix60JMXhPLdd3TY+nJzPD0qInLdjjoh607ZeX6sFQ==
X-Received: by 2002:a05:600c:3595:b0:40f:d432:c4a0 with SMTP id p21-20020a05600c359500b0040fd432c4a0mr4832957wmq.27.1707315405641;
        Wed, 07 Feb 2024 06:16:45 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUxRKDFYEFxE8ZssylmnRrQe7NqJTu2T1ra3ELKnAtZn5F97JECWQHyH4DtaMPqmFGvuHtritR303BjHJ5VdghjiIcjP+9zrgGRQgDr/ri04rUarYDIiVWvXD5ymwavwfVp2C1lspQKTC4wr8SKXU/NIC6JXCMXkp4VlL8m8L5Y4+zhwl58RYQZYfRgL7HKWmIBR3K3kqI9cpPUsIvuC3R9BdsL
Message-ID: <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
Date: Wed, 7 Feb 2024 15:16:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 14:51, Federico Serafini wrote:
> On 07/02/24 08:38, Jan Beulich wrote:
>> On 07.02.2024 02:08, Stefano Stabellini wrote:
>>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>>       case 8:                                                                \
>>>>>           put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>>           break;                                                             \
>>>>> -    default: __put_user_bad();                                             \
>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>       }                                                                      \
>>>>>       clac();                                                                \
>>>>>   } while ( false )
>>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>>       case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>>       case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>>       case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>>> -    default: __get_user_bad();                                             \
>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>       }                                                                      \
>>>>>       clac();                                                                \
>>>>>   } while ( false )
>>>>
>>>> Related to my remark on patch 1 - how is one to know the macro this was
>>>> invoked from, when seeing the resulting diagnostic?
>>>
>>> I am not sure what do you mean here... we do get an error like the
>>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>>
>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>
>> Right - and how do I know what _user_ of the macro actually triggered
>> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
>> __FUNCTION__ here, for that specific purpose ...
> 
> To test the macro and its diagnostics,
> I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
> on the x86 code with STATIC_ASSERT_UNREACHABLE(),
> that is in file arch/x86/alternative.c, line 312,
> function _apply_alternatives().
> 
> What I got is the following build error:
> 
> ...
> arch/x86/alternative.c: Assembler messages:
> arch/x86/alternative.c:312: Error: static assertion failed: unreachable
>    CC      arch/x86/copy_page.o
> make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1

But that's not what my request was about. Here sufficient context is
given, even if it would be nice if the function was also visible right
away. But that's not the same as the case above, where the new macro
is used inside another macro.

> If I understood your requests correctly,
> the only thing missing is the function name but I didn't find a way
> to make __FUNCTION__ or __func__ work with the .error directive.
> Do you know any tricks to make it work?

I didn't think any tricks would be required:

asm ( ".error " __FILE__ ":" __LINE__ ": in function " __FUNCTION__ );

Yet it looks like I was under the wrong impression that __FUNCTION__
differed from __func__ and would be like __FILE__ / __LINE__. I have
to admit I have no good idea then how to achieve helpful diagnostics.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:20:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:20:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677683.1054432 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXinJ-00045X-02; Wed, 07 Feb 2024 14:20:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677683.1054432; Wed, 07 Feb 2024 14:20:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXinI-00045Q-Tl; Wed, 07 Feb 2024 14:20:56 +0000
Received: by outflank-mailman (input) for mailman id 677683;
 Wed, 07 Feb 2024 14:20:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXinI-00045K-22
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:20:56 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1a344395-c5c4-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 15:20:54 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-40fe2ed8746so5759395e9.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:20:54 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s5-20020a05600c384500b0040ff7e31749sm3510665wmr.22.2024.02.07.06.20.53
 for <xen-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:20:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1a344395-c5c4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707315653; x=1707920453; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0engdDqSwk41ztqZ8R3j4EjvMcOaAEJGYhy9mKr3sFg=;
        b=QV2Ye6vpmfTNgpbiW8LvKNfufGnVOmoafJ8qvidhEcYKkxQvwF4IWYkJvFjnuocgMO
         8M+G8rhXgIBDYHlEIuILmi7Q8ARS1/02qv1VcegeUIIV2UHkB7Ii1UoV+ZmZjIEWqof4
         EIkJa0R7fv00Vi8R7dPK3GcD9m1qOgprFdMaarsHMV9Q1Zdzc6w5kMooMILMsFt9yvWY
         ikYRmePdyqSVWfiLOsKlQcFjjWCm4QNBpTakI81cbuDp1HvW8PEOO8S7obttF5q5r8gq
         AgeLPjKX0RpYoUeBERvX6gvkm4Qi+BnxYObf8IyVGobbeFLuDrX21dsILDNc3XDPwrcM
         DyJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707315653; x=1707920453;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0engdDqSwk41ztqZ8R3j4EjvMcOaAEJGYhy9mKr3sFg=;
        b=UADq6bf4YkMCZMFNl+4d84QlnGcpfWkPZKWAtoZwCrqcZZKcklyAB7JaDAdrPjcNhh
         5WxbTn79P+VjjU0cg9uDkTTvPcJcDAQORhMkKETOQfEweXlqOccw4g7TMFIchkECepSF
         LQeM/MysF6EvahhTdeKGJvfOmyye66wDLOCx6f8zMrTAfGU7+Kg2N0k6yFTP25hIGXdi
         sFMOsjDNjCcmMrRDTTd56h0zduyUKg343UaPme58SN/W0h7OlaVAcwIdSC/EWS6ERVh2
         b3/Q0wMEjMofD/wqxiK0bahseIfaJcIhdmHtSo05f5y9DB4BD2bvfAkz23IMI+bW81uM
         Zfaw==
X-Gm-Message-State: AOJu0YwnS2+VRL6Cnawg1lYwSFUzvUV1RUT0VOKzuvENFiPO0cSiO1yC
	pVKKPG8fikyHWKCwua5nuaeoe0VLca+G982oQY3USs7LL9wmc1PS6u1uVxz1PFxs9xWuKq7/vfg
	=
X-Google-Smtp-Source: AGHT+IE9hiifi5FmaXPGOj4tZ1ru/1c55KZHiTFedqbJ3j2bqfrscms9CjRMVEm8zDqedlQV6ueteg==
X-Received: by 2002:a05:600c:3d8d:b0:410:1e83:bb5b with SMTP id bi13-20020a05600c3d8d00b004101e83bb5bmr580146wmb.14.1707315653491;
        Wed, 07 Feb 2024 06:20:53 -0800 (PST)
Message-ID: <115f10ac-a115-40d1-a444-50e960320557@suse.com>
Date: Wed, 7 Feb 2024 15:20:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: xen | Failed pipeline for staging | f4519ee8
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <65c38d7a7ea07_2d1b5321c96792@gitlab-sidekiq-catchall-v2-779bdffdc8-kp98w.mail>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <65c38d7a7ea07_2d1b5321c96792@gitlab-sidekiq-catchall-v2-779bdffdc8-kp98w.mail>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 15:02, GitLab wrote:
> 
> 
> Pipeline #1167820960 has failed!
> 
> Project: xen ( https://gitlab.com/xen-project/hardware/xen )
> Branch: staging ( https://gitlab.com/xen-project/hardware/xen/-/commits/staging )
> 
> Commit: f4519ee8 ( https://gitlab.com/xen-project/hardware/xen/-/commit/f4519ee87ebfb74fe0451130d82c99453b75e2e1 )
> Commit Message: eclair: move function and macro properties outs...
> Commit Author: Maria Celeste Cesario
> Committed by: Jan Beulich ( https://gitlab.com/jbeulich )
> 
> 
> Pipeline #1167820960 ( https://gitlab.com/xen-project/hardware/xen/-/pipelines/1167820960 ) triggered by Jan Beulich ( https://gitlab.com/jbeulich )
> had 1 failed job.
> 
> Job #6116519549 ( https://gitlab.com/xen-project/hardware/xen/-/jobs/6116519549/raw )
> 
> Stage: test
> Name: qemu-smoke-riscv64-gcc

I have to admit that I can't connect what was pushed recently to this job
failing.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:25:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:25:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677688.1054453 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXirH-0004u0-OK; Wed, 07 Feb 2024 14:25:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677688.1054453; Wed, 07 Feb 2024 14:25:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXirH-0004tt-LI; Wed, 07 Feb 2024 14:25:03 +0000
Received: by outflank-mailman (input) for mailman id 677688;
 Wed, 07 Feb 2024 14:25:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXirG-0004ey-Ct
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:25:02 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id add73098-c5c4-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 15:25:01 +0100 (CET)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-3394b892691so401529f8f.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:25:01 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020a05600c474e00b00410232ffb2csm307940wmo.25.2024.02.07.06.25.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:25:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: add73098-c5c4-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707315901; x=1707920701; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=m5ICgbt0/W6ckB4/A7YpRur4Ixkx6v4ImUf/u2oPtk0=;
        b=d+AbupH0Zjagyp82KchuJVrVlGTq30Toef9fjDvEEyHTaXdBZepCCl5EXncuEr2p4D
         v9w53ccgkx4Eq+V+ZutM5+pvNi92utH4SRJAd22AYc3VLm+2TKyRhtVmFUqBAZ57cXuM
         8hephYDBD4WlsJztnoLLMTwcTbL25AldHxYgivyRObZ/ss3t6q93Q7dG6bM1+qObqryO
         vYuydBFCNAliGg1N9M4f1l8iiSWBZimwyoIVOn+8Hlr+GkZhu95gpZh1ipLesWtOBBIz
         EYqOu5EpsMbbuaJrbFI9IWIJKuIuV22G221xJe4M82yWKyi3S0Vk57Q3XySIW+3iqsy6
         QVYg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707315901; x=1707920701;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=m5ICgbt0/W6ckB4/A7YpRur4Ixkx6v4ImUf/u2oPtk0=;
        b=h5uOGv1Qtu/wGS2vc1/XamUsPLHXFaAgtD1UEIcm4E1q/9A4JXGqtB7KaMlWdv5Icm
         q84tHq6doUqWzQDByV0b8Q5oeg2D+WAF4RDwzQ2YC1R9R5JDwofEpEjxqEO/1L7g+4S/
         vIFxPV3mMeLSQQzozsaemViPLgzz0Z4jlQz0QV9nor16KPYcyXkotc/y6tpOE167YN56
         nUC+wooYcXd0Ru/SLhiblHaBHyIyrUbQdpaj0J5xaQ9Y3MRtznIl1/bymJCi10vKMP8Q
         1riIY/21RvDE+ZykfVhsA2Fy8aukuXUdXFWjjWB9h0tpgVU0uREf99+mOAiBvy5EvSVF
         8GoQ==
X-Gm-Message-State: AOJu0Yy2Pvs53V4J4spBr8jxku8NAWCFaFcWtzH6xesLePBQQo3yfyYT
	4XPXcCiaifiqpm2OxkvM5fBppazTLXS/HV7Yh89LhBim8cQZidnnRc2jjFOMkw==
X-Google-Smtp-Source: AGHT+IHFfYQma5nIGhVhm7U+qLj0KtfWMYEcHNVW1FkFkMLeY+PEC/w3nQTrm/dm/gS2yyCoXIZUfg==
X-Received: by 2002:adf:ea01:0:b0:33b:26c3:4fe0 with SMTP id q1-20020adfea01000000b0033b26c34fe0mr3914606wrm.6.1707315901213;
        Wed, 07 Feb 2024 06:25:01 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX2BDCNa3cAHpOW4ksLdgggKkvmRP+f1u8BJ2O5Vur0F41syCvmTmTVGZBCAM6uJAewbGJ82dsF53uF51t5iUqMAIrVKkJRhN8/L5/JC92FsN6PTNS+HcPwovO6h0yxOFAm2Ns5PMyESqqo3k8U+O6BAw1ODkis/SP0vnMJpwXVndz0h9HUHzNGRB6wrmmRqxCaCJDf0M7APkTDNLnAVozGWDU5sngnM7CqiQHY
Message-ID: <a6bf6d72-5796-4ccc-aa3e-38438f986b32@suse.com>
Date: Wed, 7 Feb 2024 15:25:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 3/7] VMX: convert entry point annotations
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
 <5fc304c0-be1f-46dd-a783-4030ec76a2f8@suse.com>
 <c11bd472-fc43-49c5-8482-4fc7ed836269@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <c11bd472-fc43-49c5-8482-4fc7ed836269@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07.02.2024 14:55, Andrew Cooper wrote:
> On 07/02/2024 1:37 pm, Jan Beulich wrote:
>> Use the generic framework from xen/linkage.h.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> v6: New.
>>
>> --- a/xen/arch/x86/hvm/vmx/entry.S
>> +++ b/xen/arch/x86/hvm/vmx/entry.S
>> @@ -24,7 +24,7 @@
>>  #define VMRESUME     .byte 0x0f,0x01,0xc3
>>  #define VMLAUNCH     .byte 0x0f,0x01,0xc2
>>  
>> -ENTRY(vmx_asm_vmexit_handler)
>> +FUNC(vmx_asm_vmexit_handler)
>>          SAVE_ALL
>>  
>>          mov  %cr2,%rax
>> @@ -132,7 +132,7 @@ UNLIKELY_END(realmode)
>>          call vmx_vmentry_failure
>>          jmp  .Lvmx_process_softirqs
>>  
>> -ENTRY(vmx_asm_do_vmentry)
>> +LABEL(vmx_asm_do_vmentry)
> 
> This really is a function, not a label.
> 
> xen.git/xen$ git grep vmx_asm_do_vmentry
> arch/x86/hvm/vmx/entry.S:135:ENTRY(vmx_asm_do_vmentry)
> arch/x86/hvm/vmx/vmcs.c:1855:void noreturn vmx_asm_do_vmentry(void);
> arch/x86/hvm/vmx/vmcs.c:1929:    reset_stack_and_jump(vmx_asm_do_vmentry);
> 
> It is giant mess, of two functions forming part of the same loop.
> 
> Considering that you declines to take CODE, I don't know what to
> suggest.  The point of CODE, distinct to FUNC, was to identify the
> places where weird things were going on, and this absolutely counts.

What's not clear to me: How would CODE() differ from both FUNC() and
LABEL()? And if the symbol is to be a function, what's wrong with
using FUNC() here as is? There's in particular no real need to have
all FUNC()s paired with END()s, afaict.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:25:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:25:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677687.1054442 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXirE-0004fB-Fx; Wed, 07 Feb 2024 14:25:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677687.1054442; Wed, 07 Feb 2024 14:25:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXirE-0004f4-D2; Wed, 07 Feb 2024 14:25:00 +0000
Received: by outflank-mailman (input) for mailman id 677687;
 Wed, 07 Feb 2024 14:24:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXirC-0004ey-Rs
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:24:58 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ab8ddbe0-c5c4-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 15:24:57 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-56001b47285so828124a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:24:57 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 v30-20020a50a45e000000b0056023efc5besm756492edb.53.2024.02.07.06.24.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:24:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab8ddbe0-c5c4-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707315897; x=1707920697; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=YXilhrGUfIHrLwjt0i+K1G2q3zjUJ7iYeyluDTomkHQ=;
        b=tcXCCoTcyxlgmP9tfnApZ5fES3CJiMTbw0b5GLO2VbzhW6Hw0y8IfsjSEcCpsQI0Jr
         y2gMaraJKWsKnmBI1FRc3e+dkQfks6Rv1rpwQu3FJKeOEM1xROD/93OHoetw7wao7Fmi
         156Q2QXz5yqwXRcuYNoYxExQBhx+oIRNGEXmc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707315897; x=1707920697;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=YXilhrGUfIHrLwjt0i+K1G2q3zjUJ7iYeyluDTomkHQ=;
        b=Bsz/kvfEVQ/vXhkRzPLG8E3CiqHGKpkim2njvQLoHtjluZmaiHMubFJnbZz/Wuqa0e
         Jqj9GpQ+d6icM52zGOA5pqpEH+PUz36D4Kf5WKHGHK4+WNdCyOMeH/la0Jerca7YNWYt
         01EAJCqo9gqO2QqfYBRQlWQWNB6iDLd7ozAsxNgPQMh+Vrf2VwjulDXxbjUr0f0NVImS
         a8cQSRzuhCo6HDGlRlWjps5HSMCT6AZtpU6UoISV6qAKRD1vF5IscWo920cbUX/iGWsP
         gt96QAOVnZ68ibaFqICRZ91zIJc62REjD/ct8d41Pp/s9OEZwoColzFD+M8BWb1CxrtV
         QWFQ==
X-Gm-Message-State: AOJu0Yxpv2uM7BhezFvB2Kp4l4J5bdjbcP1I5eV1prI7CaRH0KH/KCQS
	tGYf6EjghhwosLXsgKXg0ezmmyxgLaTFRfECSLriEF+oZzUNVDlW2BviY7ac5GU=
X-Google-Smtp-Source: AGHT+IHOSn0TVoob/vj6+vgW8+pgnuW54al1/4+1LQar4mwK9hG5hs+HolndV8AG8Omz4htzmryvgQ==
X-Received: by 2002:a05:6402:31f3:b0:55f:f2ae:20ed with SMTP id dy19-20020a05640231f300b0055ff2ae20edmr4150228edb.16.1707315897417;
        Wed, 07 Feb 2024 06:24:57 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWPudKpEgYVB1GH3Oa+Qf4lNCV0U0Z3h/Yp8hJ0kr4wS62UOm6Jt1v9LW8OdmeR3xSNxu0ebHxToC1ElSRIN+bWAzyXYfJL1p3l1L2HrEFnEObFjmCVs0FlPoVakgjt889/xeDw2X3NNP0=
Message-ID: <65665f86-cbfc-4352-9d81-67ac0b21b161@citrix.com>
Date: Wed, 7 Feb 2024 14:24:56 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: xen | Failed pipeline for staging | f4519ee8
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Anthony PERARD <anthony.perard@citrix.com>
References: <65c38d7a7ea07_2d1b5321c96792@gitlab-sidekiq-catchall-v2-779bdffdc8-kp98w.mail>
 <115f10ac-a115-40d1-a444-50e960320557@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <115f10ac-a115-40d1-a444-50e960320557@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07/02/2024 2:20 pm, Jan Beulich wrote:
> On 07.02.2024 15:02, GitLab wrote:
>>
>> Pipeline #1167820960 has failed!
>>
>> Project: xen ( https://gitlab.com/xen-project/hardware/xen )
>> Branch: staging ( https://gitlab.com/xen-project/hardware/xen/-/commits/staging )
>>
>> Commit: f4519ee8 ( https://gitlab.com/xen-project/hardware/xen/-/commit/f4519ee87ebfb74fe0451130d82c99453b75e2e1 )
>> Commit Message: eclair: move function and macro properties outs...
>> Commit Author: Maria Celeste Cesario
>> Committed by: Jan Beulich ( https://gitlab.com/jbeulich )
>>
>>
>> Pipeline #1167820960 ( https://gitlab.com/xen-project/hardware/xen/-/pipelines/1167820960 ) triggered by Jan Beulich ( https://gitlab.com/jbeulich )
>> had 1 failed job.
>>
>> Job #6116519549 ( https://gitlab.com/xen-project/hardware/xen/-/jobs/6116519549/raw )
>>
>> Stage: test
>> Name: qemu-smoke-riscv64-gcc
> I have to admit that I can't connect what was pushed recently to this job
> failing.

The qemu smoke tests for riscv and ppc intermittently fail on the
OSSTest-lab infrastructure in Gitlab.

We've never got to the bottom of it.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:27:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:27:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677695.1054463 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXitO-0005lm-6F; Wed, 07 Feb 2024 14:27:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677695.1054463; Wed, 07 Feb 2024 14:27:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXitO-0005lf-3Z; Wed, 07 Feb 2024 14:27:14 +0000
Received: by outflank-mailman (input) for mailman id 677695;
 Wed, 07 Feb 2024 14:27:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXitN-0005l0-Nl
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:27:13 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fc41b4c2-c5c4-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 15:27:13 +0100 (CET)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a28a6cef709so100989466b.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:27:13 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 bi10-20020a170906a24a00b00a371a1b40c1sm822187ejb.23.2024.02.07.06.27.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:27:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc41b4c2-c5c4-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707316033; x=1707920833; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xSTRBj9hIamysI41oHj8qdndP0w+mL+CHkqpJbH/j00=;
        b=sjXQNFtkry+lHuCvqzUEYFeUfuWIn7LHgtkyPGkLxdwHbbfN19EZ9Jz0x8Q+71mdkA
         cMORlRjyX8lXhLOC+C9FIxGSvYk77FpYSx8p9ytw4a7ppKcvAsxxXUc4Tvlmoc0X32ie
         gtXwgm0h396HX/vw8n7N4K99Q7VR6A95F+ra4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707316033; x=1707920833;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xSTRBj9hIamysI41oHj8qdndP0w+mL+CHkqpJbH/j00=;
        b=LyK+C+9lhWsu0W1Hq8eEUPvNJr1hyK+QF+TkFm1bpvR8g/OSU8Eh+3tpYLdRG72qtL
         NcoDBnwHmJV4TDVfoIO8+6Wkng69ROlgX+xqRxPJJfe87PDpZU1UR84k7MfbFJs2aupw
         Ybw3tAe7rHpewb/pl6qAw/xsUToPTcKtViU09mKj/JYotfvgS/vUzS3jz1WXFeMBUXe/
         B6H0eByHFXXbbcyGxLtzGbCau51eSfr9oFLMh0BtxE1gJcojWdgDnIqKbPn370Yf2vYV
         uXSfP9CiBKlVjg7MNkHdafDjAmFP949v72hruO9XLGNS2MlHY5BDObSpLmZ+eLQIoCWv
         WycQ==
X-Gm-Message-State: AOJu0YycwsrsF/sxX9gOh7yGJ3FYBsDkiKURMf7fC8m/hyWa4QDtMXwR
	bzyiYwMR+TLRiPutMRYMgg2HzlzloQJvcDr30HPDwg+J5crAo10LuF+OnDCPx6Q=
X-Google-Smtp-Source: AGHT+IGasbD83RRA3rAALKf36ep1GL+otVwZwTaOb2zLonyTBsAAci5d80bMs0RN0am33yXnCwdgJQ==
X-Received: by 2002:a17:906:300b:b0:a38:19d1:cddf with SMTP id 11-20020a170906300b00b00a3819d1cddfmr3170842ejz.0.1707316032828;
        Wed, 07 Feb 2024 06:27:12 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU93tv2A/QiD9sekiWd/3EgPezt704DEJLqKqV1L7oD3ultMBIpTHRvf9Raamx9Qh0VGnPaY91IWstoyFdKljQPZLlYIGYO9cnVA4CLqGg26DAjN7cOeNRi6EQuMKo9ixntQF6P5VRitCdeM6f+n1HyYZv8I7TY2Z4BK+ipDYo=
Message-ID: <c0d19a2a-3599-4f9c-ab34-c090e29760a4@citrix.com>
Date: Wed, 7 Feb 2024 14:27:12 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 7/7] x86: move ENTRY(), GLOBAL(), and ALIGN
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
 <64785449-1ca6-41f7-9012-2f432c171e3a@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <64785449-1ca6-41f7-9012-2f432c171e3a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07/02/2024 1:39 pm, Jan Beulich wrote:
> ... to boot code, limiting their scope and thus allowing to drop
> respective #undef-s from the linker script.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> An obvious alternative would be to convert boot code right away too, but
> I think this has lower priority for now.

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

It would be nice to take these out, and it doesn't look as if it's too
much work to do, but I guess we can settle for getting them out of config.h

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:31:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:31:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677703.1054473 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXixc-0007zC-MY; Wed, 07 Feb 2024 14:31:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677703.1054473; Wed, 07 Feb 2024 14:31:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXixc-0007z5-Ja; Wed, 07 Feb 2024 14:31:36 +0000
Received: by outflank-mailman (input) for mailman id 677703;
 Wed, 07 Feb 2024 14:31:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXixb-0007yz-EI
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:31:35 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 976a1e89-c5c5-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 15:31:33 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a26f73732c5so100059466b.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:31:33 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 q25-20020a17090622d900b00a37579fa8f1sm813196eja.71.2024.02.07.06.31.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:31:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 976a1e89-c5c5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707316293; x=1707921093; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jig2Fn74+PW1WgMpn+wHyKCUQJuRi09dFaicsCJwpJo=;
        b=cQABvtaz+nywmHfgjQ+XbO5j0TC6Tzu0Bpy/8E/Gsp/3bD5kjzr60F65FaNOyY7WBC
         aazGfmcaC4po2u3Zvx41PjLyMOxuDyNd0V+mxPX6CEMRyAoJ/GVPW87Of7Ey9aTNun/V
         kCFv1suqfABtckCg8j2h+6ldSM10pKSWicWnk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707316293; x=1707921093;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jig2Fn74+PW1WgMpn+wHyKCUQJuRi09dFaicsCJwpJo=;
        b=vNKHcFrvrfD5W+m+DrYYM5shJtBsME41M//PEx1FbeZ8p66WBTFhw12mKlBiWHi4Ir
         3nxRszQtrirZ5ajsLQ27WIjLsFMEvsOcqIlsO2c/jIJbkmY4C0Lzd8A5h9rvszNtevMZ
         qgzEkS5WUSvTIRDjI4Jbcosb79Us9zhCdNRZRfuJ5WG322Rb4oQPcPGueD1cdLksIeLv
         AWzcJGhO1dKlmTlvAyIdcsHA9RdiPC2T8ZXgEzDL8DLLvpzvQvc06hxqhbC1uCbs0c7d
         Z6fZctpFBarOKyDVHQfmFMasnzkSQWKcKfG68pevry2og+a/F1DHQtDuDj2GpIcVSeG5
         +Gwg==
X-Gm-Message-State: AOJu0YzJDrM8pKw6t9XfYp0AVvhrdmAh9sedtBJZauz0klprT03YXv04
	X2wULyWFMOe+QR9m2GtHN41oiffVpKWeb5WOsUXnNgRySRB33Kv2tGqUIck0Sm8=
X-Google-Smtp-Source: AGHT+IFJ7aoT4k5YGU1mtNWa08vNQDUPSwxjk2AWaCsfcTGUe0ClC0rPBxUb4i40lZ0l7TE4yIpK8g==
X-Received: by 2002:a17:906:2551:b0:a38:40d0:24db with SMTP id j17-20020a170906255100b00a3840d024dbmr2962652ejb.13.1707316293085;
        Wed, 07 Feb 2024 06:31:33 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWJP9c14Pb/gZxpaMaADoDEFwSD9RT/EsypWUqOFfhFhpnXgABsjnPA7Lt3U7aZG7QGSQK1aMzipiFjk5mjsZDk3FY5HClS8j+1f+c0eJFczykG9IzNs59VwpGeIHZKYmso18MW
Message-ID: <59b77f3b-53d8-481b-925c-b7f0dc64e2c3@citrix.com>
Date: Wed, 7 Feb 2024 14:31:32 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] hvmloader: suppress GNU ld 2.39 warning about RWX load
 segments
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <90070a25-445f-4b82-be19-d87b25fc9b8c@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <90070a25-445f-4b82-be19-d87b25fc9b8c@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07/02/2024 2:03 pm, Jan Beulich wrote:
> This aspect is simply meaningless for this component.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

In principle Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>, but

>
> --- a/tools/firmware/hvmloader/Makefile
> +++ b/tools/firmware/hvmloader/Makefile
> @@ -20,6 +20,8 @@
>  XEN_ROOT = $(CURDIR)/../../..
>  include $(XEN_ROOT)/tools/firmware/Rules.mk
>  
> +ld-option = $(shell if $(LD) -v $(1) >/dev/null 2>&1; then echo y; else echo n; fi)

The root config.mk has cc-option, and one opencoded ld-option for
handling build-id's.

This ought to live there, and ideally clean up the opencoded use while
at at.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:34:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:34:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677707.1054483 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXj0f-0000fy-3R; Wed, 07 Feb 2024 14:34:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677707.1054483; Wed, 07 Feb 2024 14:34:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXj0f-0000fr-0d; Wed, 07 Feb 2024 14:34:45 +0000
Received: by outflank-mailman (input) for mailman id 677707;
 Wed, 07 Feb 2024 14:34:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXj0e-0000fl-09
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:34:44 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 07b046fa-c5c6-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 15:34:42 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-40fd2f7ef55so10645205e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:34:42 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e10-20020a5d500a000000b0033b518f73e1sm719251wrt.71.2024.02.07.06.34.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:34:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07b046fa-c5c6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707316481; x=1707921281; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=X5rBWMcq2hTmpd869bkHfCkopsDt45VP/hnDbkBCaLo=;
        b=EpDHiOzp+EGe1ZA3imu4BHlmijmb9cDZhx+mtspQY1tUJXLz0ui437/eCEBJXzle8P
         PxFb5DN3ws9jMmyqe9cdlmbtfu1fh8MwOIS8BuE7a3Y3yuCpdy+q5ie4DTvP55zwRbFx
         RncJZT/WjYn85xwEk7DfOkbdxJ5PMXVABMx91IBi1qFcvOxxF8pqP8MojS0InC9Qzwkp
         2d9nHhMk+YZl1wEL2D+Bv+FdLSxHar399/oIo7k9zdp3aTJSZVOiL5YzAOE0YeI/9PyH
         aHA5IC6Ee9XcpnSTwhuYVQkzKoZPPgrOA1+wyMNt74EkDvKkevGcu2GI64N/v/DhV9f3
         9tWQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707316481; x=1707921281;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=X5rBWMcq2hTmpd869bkHfCkopsDt45VP/hnDbkBCaLo=;
        b=l4PF9V6lCTYnwokp1CuYkfzVQArmW9HbmL8cZPQM/k3mQ09Wtlj5kdLGJDjdqM33AM
         OMMTBIwuMDXQiiPWFgdjHe3RvZuPMvNvz7ssT9yFYQHmvVyCK18DxVwejBoDMxWWCNVU
         xMGqvOC3aOBnz1kAmo6+L4mCZ8celEKKq55ll0FoF8PeK9zWrLEE9uRtZZ3X2mJg88Zo
         umBRx2J+GQt4bG3kjmhv3NDfN0rR1qR2XsT+Br6eyUI0+/jVHgSVaWLcnZmwUeZGBZff
         8vS+2rra0QNkzYQ9DNLDSnPzhayLhdUO0TzsOWUdXyb5Fc1+F1026bnPOwzILkTOPBJx
         vluA==
X-Gm-Message-State: AOJu0YxJm9/m8FO2aDcGjLe8NGZoTFAhipG5TO9SsaHkz6qWcr4ggTtm
	vZuM4Rmh0PI4C6JjCQQLXmv6q2GxfaTzw+0E6FGHQTrsOphU3azpwOd4QCQbQw==
X-Google-Smtp-Source: AGHT+IFXY2otHOBvYt/xZnPjvK4DetTRv8P227sxQNWJUH+Wu2wHjcamg7GZduQIKMtD3FKk6cHETw==
X-Received: by 2002:a05:6000:69c:b0:33b:4e6b:e8fa with SMTP id bo28-20020a056000069c00b0033b4e6be8famr2474448wrb.29.1707316481389;
        Wed, 07 Feb 2024 06:34:41 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXW/IjbcFjN9rqJu7vaTXCHwP1y7CNO33jbqQdLnZVyElhYk8o/28zlJim08ank+hpGcd9lnqxIocKaeC3bQA7JnFbXJVGZXNxA0vCgS1fqRg2kAEcrFoFpG28wCZhcW9ypcWM4
Message-ID: <ec2eb8b8-e407-49d5-ae1e-8d8ad19e3eb4@suse.com>
Date: Wed, 7 Feb 2024 15:34:40 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] hvmloader: suppress GNU ld 2.39 warning about RWX load
 segments
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <90070a25-445f-4b82-be19-d87b25fc9b8c@suse.com>
 <59b77f3b-53d8-481b-925c-b7f0dc64e2c3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <59b77f3b-53d8-481b-925c-b7f0dc64e2c3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 15:31, Andrew Cooper wrote:
> On 07/02/2024 2:03 pm, Jan Beulich wrote:
>> This aspect is simply meaningless for this component.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> In principle Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>, but
> 
>>
>> --- a/tools/firmware/hvmloader/Makefile
>> +++ b/tools/firmware/hvmloader/Makefile
>> @@ -20,6 +20,8 @@
>>  XEN_ROOT = $(CURDIR)/../../..
>>  include $(XEN_ROOT)/tools/firmware/Rules.mk
>>  
>> +ld-option = $(shell if $(LD) -v $(1) >/dev/null 2>&1; then echo y; else echo n; fi)
> 
> The root config.mk has cc-option, and one opencoded ld-option for
> handling build-id's.
> 
> This ought to live there, and ideally clean up the opencoded use while
> at at.

Well, that's what I first meant to do. But: xen/scripts/Kconfig.include
has a different ld-option, and I didn't want to get into the business
of having two fight each other.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:34:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:34:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677708.1054493 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXj0p-0000zT-AY; Wed, 07 Feb 2024 14:34:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677708.1054493; Wed, 07 Feb 2024 14:34:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXj0p-0000zM-7S; Wed, 07 Feb 2024 14:34:55 +0000
Received: by outflank-mailman (input) for mailman id 677708;
 Wed, 07 Feb 2024 14:34:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9378=JQ=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rXj0n-0000fl-Fu
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:34:53 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0daf0bd4-c5c6-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 15:34:51 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-51168572090so912355e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:34:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0daf0bd4-c5c6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707316491; x=1707921291; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=RJXz4eOzOB/wEF6Z9P+wu/oxvAjOFut5Y1isvsL+Lo4=;
        b=mhk/jkLXgXP88JtVXxvFbelumAzF6bcjIFvTs03PK+GHwiEENg2nz7qZqQeTbloCaL
         XzmdVdDhhWoZ8Xhbexs0qfHd3+dglq/vHoTAxnjwWLjRkJKiMCCGjotOvSXpCv1d0yTd
         4jvR9KMnvtdqGdVEhxGvS1EEHR0MGPav1bGmA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707316491; x=1707921291;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=RJXz4eOzOB/wEF6Z9P+wu/oxvAjOFut5Y1isvsL+Lo4=;
        b=Qb9FrRYcn3hplmf4XqpmjzkXgnAaX6fcAMVBi2YHtondZ37yg5eiQ4Ay2d0YdYSWyi
         r4iMeW7gS37iH+e99ebEupwR9OwZR7FBO4B5jlOOBmFibAD24CRDchO+HUO7w89A7K3Z
         LfKefJowRM/Ci4p0lVIUdIqgQTt+mUrV27Zdsbx4H66lcAogHH75P7eIcEhfD7G3GaLu
         H5WHADUYUP5nXlyLsmGhCgKCQn7QBz7cVyCvtZMd+4duE/rAii36nqWpz7Q9gdJRRwqW
         HUDVBi1+3OKD/rM5YSNJdWCLVm0zFRX9Wo6Zpby0r64V/L8Mn9ExKs/fM8W3G9AE93Mi
         H34Q==
X-Gm-Message-State: AOJu0YxsK0GSXSVUjcb5VKZJj1SdjVavUiFvvqWQh5WEMwymzO4pt7c2
	3c41IyedIB3a732cJIunj5QgkEHJ3Rd8U4GBHdZng2r+Ewa+fNR69mPRYcB+RFfO3kyldW0Apws
	6EpsAzaBBB4boN8HLp14mj0aNpwJjbcT/K52MYw==
X-Google-Smtp-Source: AGHT+IG+SRYI4PlCbGiEJw48dR5iliE1UaLoAi6lzqHkmM9BvRirdMojkkXHaxa4MjbiwujUdZOssYoJfjvHFfZrQbc=
X-Received: by 2002:ac2:5a01:0:b0:511:4844:77f2 with SMTP id
 q1-20020ac25a01000000b00511484477f2mr4254008lfn.40.1707316491464; Wed, 07 Feb
 2024 06:34:51 -0800 (PST)
MIME-Version: 1.0
References: <65c38d7a7ea07_2d1b5321c96792@gitlab-sidekiq-catchall-v2-779bdffdc8-kp98w.mail>
 <115f10ac-a115-40d1-a444-50e960320557@suse.com> <65665f86-cbfc-4352-9d81-67ac0b21b161@citrix.com>
In-Reply-To: <65665f86-cbfc-4352-9d81-67ac0b21b161@citrix.com>
From: Anthony PERARD <anthony.perard@citrix.com>
Date: Wed, 7 Feb 2024 14:34:40 +0000
Message-ID: <CAKoJhMROFw1w679U9xsW6s4p34Zzm6-gaHihUHtU1w5ZH9ViUQ@mail.gmail.com>
Subject: Re: xen | Failed pipeline for staging | f4519ee8
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 7, 2024 at 2:24=E2=80=AFPM Andrew Cooper <andrew.cooper3@citrix=
.com> wrote:
> >> Stage: test
> >> Name: qemu-smoke-riscv64-gcc
> > I have to admit that I can't connect what was pushed recently to this j=
ob
> > failing.
>
> The qemu smoke tests for riscv and ppc intermittently fail on the
> OSSTest-lab infrastructure in Gitlab.
>
> We've never got to the bottom of it.

It's maybe because joubertin* and nocera* have more than one NUMA
nodes. It's look like 2 for nocera* and 4 for joubertin*.
Would that be enough of of an explanation about those tests taking a
bit longer than expected?
I could try to setup gitlab-runner/docker to have the containers
running on a single node, since my guess is Linux doesn't do that well
enough.

Cheers,

--=20
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:55:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:55:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677720.1054503 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjL0-0006II-1P; Wed, 07 Feb 2024 14:55:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677720.1054503; Wed, 07 Feb 2024 14:55:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjKz-0006IB-U0; Wed, 07 Feb 2024 14:55:45 +0000
Received: by outflank-mailman (input) for mailman id 677720;
 Wed, 07 Feb 2024 14:55:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXjKy-0006I5-D8
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:55:44 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f6e77c34-c5c8-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 15:55:42 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-5600c43caddso772845a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:55:42 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 l13-20020a056402028d00b0055f0f643ebcsm736027edv.96.2024.02.07.06.55.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 06:55:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6e77c34-c5c8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707317742; x=1707922542; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/wyz1rxYbBZ1o2WqAlufUy4smq3954bcszzF/VIZfdY=;
        b=aTvMiktYAy4UFByperYXsByvpLsxRoSd/9z4v2gseSBlIFKjBwC7X9LzZTZ8QAd9Iw
         iRVzh1UGpnzUQlapr3EY5aQGLJOeruP1QOdr154i04PNIsMcNPxEGr1nfyLbww9ilmgm
         bU19vFd8J8nUXQiYefa3rhlrJI9neagEqLdck=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707317742; x=1707922542;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/wyz1rxYbBZ1o2WqAlufUy4smq3954bcszzF/VIZfdY=;
        b=Rilr48409V8GCDZl6hqozXmv8yQM0xTb/btu2jT1y9Hts9SDvAUkSXfI3CUs+/+f3Y
         g27W8Dby0nocALMYbR4JUJt5d8UTuMfI5EOvHU3F88XGFvP27j5l+PoU2EW7H0q1jArx
         gGl63nTUKZQpLsXU8qVTlX2qiRq509ocfja0JAh0d6FSFOGva3d7WL3xbvykYn4z7SlK
         0bTxYmSnLy3B9Tg9xE91T4ClhlfDc1rO0zkqVh4p1XWeaN+pDw8Xpcb8Ko+F9gXuNgiM
         LeigjOjtZ8ykdL/BA6Y0henKafEd1cx0zcry8LzOJVZ30weBWdUodMeUUe9lXIwPIj/E
         CFaA==
X-Forwarded-Encrypted: i=1; AJvYcCWgQUQNpbSuR9An1VdUqLF+1jm7SD+v5H9LFLGvDJojKNBoOUjQ7IrjIRgeE8aHgDSNbhPesb7LgCB2Xwx80TNqSkU5sunoJOr4SJg9eko=
X-Gm-Message-State: AOJu0Yw523Y5Kw/qpsB0nfg/Voe/jP8ctosfCAGFl9Dbxqu0AIhWrOed
	QingoPADjCrSWwHWqVuhWFxCkuiOJ9AU4y/iWdWVkz2I+HQDHfDo6BVJCyh8xQ8=
X-Google-Smtp-Source: AGHT+IGgofH1Xx70r3l/kEpFu5CWzsN5HDcVrc3w7ERdv+MVLZA8vsU6DGDTqR1RA466QQDYT/bqgg==
X-Received: by 2002:aa7:d758:0:b0:55f:e9ce:7406 with SMTP id a24-20020aa7d758000000b0055fe9ce7406mr3732972eds.29.1707317741774;
        Wed, 07 Feb 2024 06:55:41 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU36Fkbno04ai0yCnpjbdxgeGkYkABer0nOo26oOJaK9CAfiRqN0dVDI82NvaG24iok0n28auGkWgU8S94o3UlWrsQSPkyT9nV7kXE0+AA=
Message-ID: <f1f74e69-8892-4615-8411-f7a1329dd76c@citrix.com>
Date: Wed, 7 Feb 2024 14:55:41 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: xen | Failed pipeline for staging | f4519ee8
Content-Language: en-GB
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <65c38d7a7ea07_2d1b5321c96792@gitlab-sidekiq-catchall-v2-779bdffdc8-kp98w.mail>
 <115f10ac-a115-40d1-a444-50e960320557@suse.com>
 <65665f86-cbfc-4352-9d81-67ac0b21b161@citrix.com>
 <CAKoJhMROFw1w679U9xsW6s4p34Zzm6-gaHihUHtU1w5ZH9ViUQ@mail.gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <CAKoJhMROFw1w679U9xsW6s4p34Zzm6-gaHihUHtU1w5ZH9ViUQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/02/2024 2:34 pm, Anthony PERARD wrote:
> On Wed, Feb 7, 2024 at 2:24 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>>> Stage: test
>>>> Name: qemu-smoke-riscv64-gcc
>>> I have to admit that I can't connect what was pushed recently to this job
>>> failing.
>> The qemu smoke tests for riscv and ppc intermittently fail on the
>> OSSTest-lab infrastructure in Gitlab.
>>
>> We've never got to the bottom of it.
> It's maybe because joubertin* and nocera* have more than one NUMA
> nodes. It's look like 2 for nocera* and 4 for joubertin*.
> Would that be enough of of an explanation about those tests taking a
> bit longer than expected?

That feels a bit like a blind guess.

What hardware are these?  They're rather old, I seem to recall?

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:56:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:56:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677722.1054512 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjLO-0006h8-Cp; Wed, 07 Feb 2024 14:56:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677722.1054512; Wed, 07 Feb 2024 14:56:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjLO-0006h1-AE; Wed, 07 Feb 2024 14:56:10 +0000
Received: by outflank-mailman (input) for mailman id 677722;
 Wed, 07 Feb 2024 14:56:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gvxy=JQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXjLN-0006gr-NG
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:56:09 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 06501841-c5c9-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 15:56:08 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a30e445602cso378717366b.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:56:08 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 m8-20020a1709066d0800b00a3785efe1c4sm840290ejr.85.2024.02.07.06.56.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 06:56:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06501841-c5c9-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707317767; x=1707922567; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=wKhbHBSZ0QAOAy6CUsvXikLiOFRiYlat1iOA7P+iYBc=;
        b=Uy5l0PldsFzKOynautWQgm6Y2piPym9fIvfstyxj/7JCWgaMnfOkKZEk8tEi1V1zRr
         IsMnai06LD4l51Mu2usuhNJEoq5/yz78y6Q+OuuuJ95CG9E8vNTj0weXzA2xea+uXi0q
         fE4OzujmXhGehQeGfUkVyIVJDrrK85jupp6p4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707317767; x=1707922567;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=wKhbHBSZ0QAOAy6CUsvXikLiOFRiYlat1iOA7P+iYBc=;
        b=REidBXBV9W0Xl32mATrAKVb66dGnzWVHqVmGp0FudJ56R2kFzSLs3Nz7RxOWlD0DGn
         jUbJNJ/fw858+NHNQfvTY7lrrRrngqVl9yRcN+JRR8FJH8c3vavQXiceTsKoJOGrdpKl
         2ymWELNGl+I3VYDesauWeqrHkkrGry8BJE7F+OdNdbahoOiM0X5HoIg6FNxk69oF9eL5
         JQJalVBEBfl9MSERTNsXLX4NwEHSo8dbzPsYtV6lJIMCzIlkyy+IykgaNj9jZAJuXkjt
         XE7ugQdJdjFQhsRg2tL7N5z4SM9NNOzG0e4g2WT/Zlxmp+ZJygV51AomfW4Blz9NbiVI
         YYpQ==
X-Gm-Message-State: AOJu0Yw0z+GgMWUa4CMjZw5kur81sFp8i890oDHfuurGUGYX1+dkWCR4
	Bs2RDZt9ug7+fPir+rWjRZMLfxEpnplFgy4b82fXU+D/Qg7AMcHvf+dY6ukaYodgySeSyGbCx2d
	c
X-Google-Smtp-Source: AGHT+IE+QavObo6WXUZV2GK0jjcNLqNq2ESEm2T1esia4XHziRsF1cQwIaDXGQ7D7vodLTPeNhisSQ==
X-Received: by 2002:a17:906:f910:b0:a38:567a:6574 with SMTP id lc16-20020a170906f91000b00a38567a6574mr3287765ejb.5.1707317767385;
        Wed, 07 Feb 2024 06:56:07 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX5e49uwKzyqKNvvQX+thy+cmHqjm4W6nMXLX4YG6juO7WiK3fZPfZbc4/fVohlExFQE77CjaqpNeb7MPZ1EFtexlOybrJwKWeBzNVeusfRVlgPH6MeEE6DpkJ3E3OSYxZVGRHkT/AFFN0+vVp8s1Nhv5QZg+yiwYAOya29fmIHJkvI6mwvNDzU/IP7OoyJpBka2WjO8UJ/ECHmw5Q9YP3oayuzPED+SajrJRcp8wMG1yAgOutQ2YCIiv1kxDXPCgJUQulhLPTET8Awbu5+nrrKGDAR9j3wxosC4FQjXGMQ4x4NKHnuylPOzCecAhtJzjDR0zGo3htJzy5QN2UQSAjkOHcD9OF6t8/yYyPf26ZkmEMuKr5ASFTMMTCcMEXiHWYOAdPROtZ3MdsU4JS91bvjAPghX79vNORAOHgnZCpSjYuzVct8lGS5MzlzauhkYaHerjhmkYw5y+NwVF9I4I+3VBmAkdDxcXs7jArJ4PCb5zWeCWb7dr0EFTPHwB17n5Sl0KHBJfQseth6nKCR2ZDHvLlbUe0=
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>
Subject: [PATCH v6 0/3] xen: introduce Kconfig function alignment option
Date: Wed,  7 Feb 2024 15:55:44 +0100
Message-ID: <20240207145547.89689-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

The following series adds an additional Kconfig option for the per-arch
code alignment.  Such alignment is to be used in all assembly code
symbols and C functions unless specified otherwise.

Last patch also uses such alignment in order to guarantee enough
distance between function entry points, so that there's always space in
order to do the instruction replacements required by livepatch.

Thanks, Roger.

Roger Pau Monne (3):
  xen: introduce Kconfig function alignment option
  xen: use explicit function alignment if supported by compiler
  xen/livepatch: align functions to ensure minimal distance between
    entry points

 xen/Kconfig                         | 22 ++++++++++++++++++++++
 xen/Makefile                        |  1 +
 xen/arch/arm/Kconfig                |  1 +
 xen/arch/arm/include/asm/config.h   |  3 +--
 xen/arch/arm/livepatch.c            |  2 ++
 xen/arch/arm/xen.lds.S              |  4 ++++
 xen/arch/ppc/Kconfig                |  1 +
 xen/arch/ppc/include/asm/config.h   |  3 ---
 xen/arch/ppc/xen.lds.S              |  4 ++++
 xen/arch/riscv/Kconfig              |  1 +
 xen/arch/riscv/include/asm/config.h |  1 -
 xen/arch/riscv/xen.lds.S            |  4 ++++
 xen/arch/x86/Kconfig                |  1 +
 xen/arch/x86/include/asm/config.h   |  3 +--
 xen/arch/x86/livepatch.c            |  4 ++++
 xen/arch/x86/xen.lds.S              |  4 ++++
 xen/common/Kconfig                  |  5 ++++-
 xen/include/xen/linkage.h           |  5 +++--
 18 files changed, 58 insertions(+), 11 deletions(-)

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:56:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:56:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677723.1054524 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjLP-0006wJ-My; Wed, 07 Feb 2024 14:56:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677723.1054524; Wed, 07 Feb 2024 14:56:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjLP-0006w6-Gn; Wed, 07 Feb 2024 14:56:11 +0000
Received: by outflank-mailman (input) for mailman id 677723;
 Wed, 07 Feb 2024 14:56:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gvxy=JQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXjLO-0006gr-ES
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:56:10 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 07532507-c5c9-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 15:56:09 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a30f7c9574eso100317366b.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:56:09 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 y13-20020a170906070d00b00a386a32a9d9sm834614ejb.194.2024.02.07.06.56.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 06:56:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07532507-c5c9-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707317768; x=1707922568; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=VvhvJ0mk4ckYFEcAiyKPBpKqoDXebryxKX/EJIJcbn4=;
        b=g6B46dlfp7q0nunTllR8NB5f4f3IRonaTzlX53W1RlNhcEIWLz90iDqNiMpBJMqNgQ
         S1NC4U6I9d0nmcVIHR5U3fE13l1ff/p+sv+q8FkgtAc6dOXaovzTjTQM7MRXTSk0rfvE
         iIFLzs5U4vHcVK1NsdIHF7Nhr8GbOPh6WFXSc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707317768; x=1707922568;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=VvhvJ0mk4ckYFEcAiyKPBpKqoDXebryxKX/EJIJcbn4=;
        b=qNHXlDGViVIHBNMFpfNtvHiRsr4T/ia41xI7GyXmIGasd1S6kUgp+y+ZFICbXi4IbL
         2BTsGbKJMu9UD7fm9h2tGmN33MPPgD+6rVR/5+E28AyqIX035h/gdvd1RNxttGkefNVb
         44I3zjU+8yDCgDowFGuzefL4BpP/ZUffhOO48DBFwz4ApOkKjBueJJo2KLwup4xU46u/
         g9N8zAuG1tx3kvxLhFlY2eQWFgTtDUAJIHIZxMJ/v3FHrXL9KhtfQaCHF+DbRCu33iO8
         Q4xE5gUBkWBCX5tA91OwpQHjsz9ZiQNNGkhRaEnHb6l2IpyNuux5B7rDuVs2tq+lLEt4
         dB2w==
X-Gm-Message-State: AOJu0YwkxDK+Z6liLxr2KKaQqxZEM0iZ6DommNGqhvz+dikyCI3FErVH
	IV8s5TmgYtmLtKc0XEw+2nWA25lo5ATni5HQ8s+BaoHEIxxtNi3hxPVuZ2ZMYc9vtydv4FBC9kP
	+
X-Google-Smtp-Source: AGHT+IEf//jnO6+wwgVwFq/WbrfkwQt0EVuxuJU/7OLvjKJPip3WS7InnQyd8gBn9YlqnI5vxEk4Nw==
X-Received: by 2002:a17:906:cc1:b0:a38:5c3c:9d1e with SMTP id l1-20020a1709060cc100b00a385c3c9d1emr1744877ejh.18.1707317768593;
        Wed, 07 Feb 2024 06:56:08 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXtnfQ4fJ1r4TYx1lFl0J8JNHk10d6cWkSfoq5n/yZRh5p5FxCnDmCFAljj8VyeWsf1vl9b16Ni+7hOCHfm/OaCfd/2bDdIpCQ+3V7O+jfokzkfiST9xH5EBMQJh3/mtrtR4PC0tXSJ/9VQzFrIg8e+BDzDEoVUbXXo6CFax+lo7fwEgCXwmOSpOMkHD2tIaTU8uSwQvg/V0vI7REPXfqnIX6WCoZmKZ0CUzfDx3nN56R6Br1EJU3WByChBz7um5utykJgmp0PqzD4r5WI/Nis9yyh/8ymQkgLQiThHwDBqKthf+cpgh6zNTwGv1shcC6SJx3dKft3Y63g49diiN2ZR05QOFfZCwn3pqQEdtIo78ydvw50BUmLlJDw0rWsn+tgTZt4gZEf08HetDF9UhjMZS7rOT38bQIq9mzHHRE2BN4ARc6kXqKmUZ+NFQqMIhvq5aZ1keLiDIbbpfmuYqgG+nrI=
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v6 1/3] xen: introduce Kconfig function alignment option
Date: Wed,  7 Feb 2024 15:55:45 +0100
Message-ID: <20240207145547.89689-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240207145547.89689-1-roger.pau@citrix.com>
References: <20240207145547.89689-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

And use it to replace CODE_ALIGN in assembly.  This allows to generalize the
way the code alignment gets set across all architectures.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v5:
 - New in this version.
---
 xen/Kconfig                         | 17 +++++++++++++++++
 xen/arch/arm/Kconfig                |  1 +
 xen/arch/arm/include/asm/config.h   |  3 +--
 xen/arch/ppc/Kconfig                |  1 +
 xen/arch/ppc/include/asm/config.h   |  3 ---
 xen/arch/riscv/Kconfig              |  1 +
 xen/arch/riscv/include/asm/config.h |  1 -
 xen/arch/x86/Kconfig                |  1 +
 xen/arch/x86/include/asm/config.h   |  3 +--
 xen/include/xen/linkage.h           |  5 +++--
 10 files changed, 26 insertions(+), 10 deletions(-)

diff --git a/xen/Kconfig b/xen/Kconfig
index 134e6e68ad84..1e1b041fd52f 100644
--- a/xen/Kconfig
+++ b/xen/Kconfig
@@ -37,6 +37,23 @@ config CC_HAS_VISIBILITY_ATTRIBUTE
 config CC_SPLIT_SECTIONS
 	bool
 
+# Set code alignment.
+#
+# Allow setting on a boolean basis, and then convert such selection to an
+# integer for the build system and code to consume more easily.
+config FUNCTION_ALIGNMENT_4B
+	bool
+config FUNCTION_ALIGNMENT_8B
+	bool
+config FUNCTION_ALIGNMENT_16B
+	bool
+config FUNCTION_ALIGNMENT
+	int
+	default 16 if FUNCTION_ALIGNMENT_16B
+	default  8 if  FUNCTION_ALIGNMENT_8B
+	default  4 if  FUNCTION_ALIGNMENT_4B
+	default  0
+
 source "arch/$(SRCARCH)/Kconfig"
 
 config DEFCONFIG_LIST
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 50e9bfae1ac8..80fb5b14f04e 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -16,6 +16,7 @@ config ARM
 	select HAS_PASSTHROUGH
 	select HAS_UBSAN
 	select IOMMU_FORCE_PT_SHARE
+	select FUNCTION_ALIGNMENT_4B
 
 config ARCH_DEFCONFIG
 	string
diff --git a/xen/arch/arm/include/asm/config.h b/xen/arch/arm/include/asm/config.h
index 3b6d829197a4..a2e22b659d53 100644
--- a/xen/arch/arm/include/asm/config.h
+++ b/xen/arch/arm/include/asm/config.h
@@ -53,8 +53,7 @@
 
 /* Linkage for ARM */
 #ifdef __ASSEMBLY__
-#define CODE_ALIGN 4
-#define ALIGN .balign CODE_ALIGN
+#define ALIGN .balign CONFIG_FUNCTION_ALIGNMENT
 #define ENTRY(name)                             \
   .globl name;                                  \
   ALIGN;                                        \
diff --git a/xen/arch/ppc/Kconfig b/xen/arch/ppc/Kconfig
index ab116ffb2a70..6b3b2bb95f56 100644
--- a/xen/arch/ppc/Kconfig
+++ b/xen/arch/ppc/Kconfig
@@ -1,6 +1,7 @@
 config PPC
 	def_bool y
 	select HAS_DEVICE_TREE
+	select FUNCTION_ALIGNMENT_4B
 
 config PPC64
 	def_bool y
diff --git a/xen/arch/ppc/include/asm/config.h b/xen/arch/ppc/include/asm/config.h
index e5d201e16c50..e0a0abfeb408 100644
--- a/xen/arch/ppc/include/asm/config.h
+++ b/xen/arch/ppc/include/asm/config.h
@@ -31,9 +31,6 @@
 #define INVALID_VCPU_ID MAX_VIRT_CPUS
 
 /* Linkage for PPC */
-#ifdef __ASSEMBLY__
-#define CODE_ALIGN 4
-#endif
 
 #define XEN_VIRT_START _AC(0xc000000000000000, UL)
 
diff --git a/xen/arch/riscv/Kconfig b/xen/arch/riscv/Kconfig
index f382b36f6c82..b4b354a7786e 100644
--- a/xen/arch/riscv/Kconfig
+++ b/xen/arch/riscv/Kconfig
@@ -1,5 +1,6 @@
 config RISCV
 	def_bool y
+	select FUNCTION_ALIGNMENT_16B
 
 config RISCV_64
 	def_bool y
diff --git a/xen/arch/riscv/include/asm/config.h b/xen/arch/riscv/include/asm/config.h
index a80cdd4f857c..99ea5635208b 100644
--- a/xen/arch/riscv/include/asm/config.h
+++ b/xen/arch/riscv/include/asm/config.h
@@ -69,7 +69,6 @@
 
 /* Linkage for RISCV */
 #ifdef __ASSEMBLY__
-#define CODE_ALIGN 16
 #define CODE_FILL /* empty */
 #endif
 
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 1acdffc51c22..3dd8f18b46ef 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -29,6 +29,7 @@ config X86
 	select HAS_UBSAN
 	select HAS_VPCI if HVM
 	select NEEDS_LIBELF
+	select FUNCTION_ALIGNMENT_16B
 
 config ARCH_DEFCONFIG
 	string
diff --git a/xen/arch/x86/include/asm/config.h b/xen/arch/x86/include/asm/config.h
index 660246d1dae5..ab7288cb3682 100644
--- a/xen/arch/x86/include/asm/config.h
+++ b/xen/arch/x86/include/asm/config.h
@@ -43,9 +43,8 @@
 
 /* Linkage for x86 */
 #ifdef __ASSEMBLY__
-#define CODE_ALIGN 16
 #define CODE_FILL 0x90
-#define ALIGN .align CODE_ALIGN, CODE_FILL
+#define ALIGN .align CONFIG_FUNCTION_ALIGNMENT, CODE_FILL
 #define ENTRY(name)                             \
   ALIGN;                                        \
   GLOBAL(name)
diff --git a/xen/include/xen/linkage.h b/xen/include/xen/linkage.h
index 0997e16810b2..770ae49963b8 100644
--- a/xen/include/xen/linkage.h
+++ b/xen/include/xen/linkage.h
@@ -41,9 +41,10 @@
  */
 #define count_args_exp(args...) count_args(args)
 #if count_args_exp(CODE_FILL)
-# define DO_CODE_ALIGN(align...) LASTARG(CODE_ALIGN, ## align), CODE_FILL
+# define DO_CODE_ALIGN(align...) LASTARG(CONFIG_FUNCTION_ALIGNMENT, ## align), \
+                                 CODE_FILL
 #else
-# define DO_CODE_ALIGN(align...) LASTARG(CODE_ALIGN, ## align)
+# define DO_CODE_ALIGN(align...) LASTARG(CONFIG_FUNCTION_ALIGNMENT, ## align)
 #endif
 
 #define FUNC(name, align...) \
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:56:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:56:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677724.1054533 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjLQ-0007C4-RN; Wed, 07 Feb 2024 14:56:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677724.1054533; Wed, 07 Feb 2024 14:56:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjLQ-0007Bx-OX; Wed, 07 Feb 2024 14:56:12 +0000
Received: by outflank-mailman (input) for mailman id 677724;
 Wed, 07 Feb 2024 14:56:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gvxy=JQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXjLP-0006gr-EA
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:56:11 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 08055efc-c5c9-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 15:56:11 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a271a28aeb4so94407466b.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:56:11 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 l16-20020a170906415000b00a3551f727d1sm830912ejk.68.2024.02.07.06.56.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 06:56:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08055efc-c5c9-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707317770; x=1707922570; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=akzbcc4EuKcp9VAzRO87Fc+0Red8uqWWSH8LHiM9Mf0=;
        b=LrE13oNnVGeF2t8ZuvKIRL/aM/6H+SDnBqPHYRVcwsUjCX+aGkGxbf7DRDLWgM3P1A
         FnM9omA+Kcd+dNFrnmokH0r93wTq8HDZY4JM22Zp1RNIyrxLdgeuFk2MkQIctbTXpPrH
         ra/7xCxG9EV/yAtitpZ0LA9IF7cjLFuVaYg2I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707317770; x=1707922570;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=akzbcc4EuKcp9VAzRO87Fc+0Red8uqWWSH8LHiM9Mf0=;
        b=iYgNg7Jl6ttsYGAnol9eiTrdhP3fNKJj4bNfu8/FbBfLwv1lKQLZVYd/H8UEJglkCC
         yV51pVbU/Mxi24U1rQCR0Z02IH0cztjs4Sz+wesV5tp6I5kmWhhRAhau3KedXMZgymM9
         Y1ir6EXq8SqOCTaJWsJmqCIKp8QWPtOMkQgVZelp4EWE0tQP2uNSyQ4sxABh0IT04rbu
         CSkdWR5WFrx71coqX2KrPJuAJpkcYNgqw7lA6qg7OZwjkIx/PU0H5WnLF4318Mp6nWGT
         13eWCnyUda8rECXzmCtTi8di23XTn8exAfXXhEI9WbLx+bPl0W3tM24L8Scs6SH9hB7W
         q6nw==
X-Gm-Message-State: AOJu0YwFMLrFCQ47Ln/xumNFCxsOERy3e85DaoTaK5gvscw4w+4azjQ9
	BoW94xL1CJR5aF20QQU3SHpin721BOm2/LVJkaM1umGivth6+R/nJvDGBHSv1ftFUHREPhKBVgA
	B
X-Google-Smtp-Source: AGHT+IF8t8/nmL4BjDrYilxRzo93JsRzkWnYIJ8/PI+xuH0RzNl3UwwXRYn8sWqNsbYCRRDf0zCEIw==
X-Received: by 2002:a17:906:69d6:b0:a38:970a:75df with SMTP id g22-20020a17090669d600b00a38970a75dfmr649647ejs.15.1707317769901;
        Wed, 07 Feb 2024 06:56:09 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUgtHNKg/CEiKIRnL5cmUT1iguKf8bV8fxszMgMhEi6x7dMwAsGFPeRh3psns0y2/JQiVz17gW6BoXCO22hyvH7+RGiZJeZkajNtS7qt47yb/wQnFgWzumfwPbBGnD5wQ9KCQXmDrgAogtwnUCSPC4ZHLSWlazXgJ3LP9CIUr+hEBrCx2cLHdpVWQp5/MFilRrbQzaxQEOMq0wd9q6VeOMCq0sMB6BDRBZpTw==
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v6 2/3] xen: use explicit function alignment if supported by compiler
Date: Wed,  7 Feb 2024 15:55:46 +0100
Message-ID: <20240207145547.89689-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240207145547.89689-1-roger.pau@citrix.com>
References: <20240207145547.89689-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Introduce a new Kconfig check for whether the compiler supports
-falign-functions and if supported use it to align functions to the per-arch
selected value, just like it's done for assembly ENTRY() and FUNC() symbols.

Note that it's possible for the compiler to end up using a higher function
alignment regardless of the passed value.  Different compilers handle the
option differently, as clang will ignore -falign-functions value if it's
smaller than the one that would be set by the optimization level, while gcc
seems to always honor the function alignment passed in -falign-functions.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v5:
 - New in this version.
---
 xen/Kconfig  | 5 +++++
 xen/Makefile | 1 +
 2 files changed, 6 insertions(+)

diff --git a/xen/Kconfig b/xen/Kconfig
index 1e1b041fd52f..040cba1b4b73 100644
--- a/xen/Kconfig
+++ b/xen/Kconfig
@@ -41,6 +41,11 @@ config CC_SPLIT_SECTIONS
 #
 # Allow setting on a boolean basis, and then convert such selection to an
 # integer for the build system and code to consume more easily.
+#
+# Requires clang >= 7.0.0
+config CC_HAS_FUNCTION_ALIGNMENT
+	def_bool $(cc-option,-falign-functions)
+
 config FUNCTION_ALIGNMENT_4B
 	bool
 config FUNCTION_ALIGNMENT_8B
diff --git a/xen/Makefile b/xen/Makefile
index 21832d640225..7c8249ab3a33 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -390,6 +390,7 @@ CFLAGS += -fomit-frame-pointer
 endif
 
 CFLAGS-$(CONFIG_CC_SPLIT_SECTIONS) += -ffunction-sections -fdata-sections
+CFLAGS-$(CONFIG_CC_HAS_FUNCTION_ALIGNMENT) += -falign-functions=$(CONFIG_FUNCTION_ALIGNMENT)
 
 CFLAGS += -nostdinc -fno-builtin -fno-common
 CFLAGS += -Werror -Wredundant-decls -Wwrite-strings -Wno-pointer-arith
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 14:56:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 14:56:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677725.1054542 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjLS-0007S2-32; Wed, 07 Feb 2024 14:56:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677725.1054542; Wed, 07 Feb 2024 14:56:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjLR-0007RU-Vd; Wed, 07 Feb 2024 14:56:13 +0000
Received: by outflank-mailman (input) for mailman id 677725;
 Wed, 07 Feb 2024 14:56:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gvxy=JQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXjLQ-0006gr-VN
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 14:56:12 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 08e3c0cd-c5c9-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 15:56:12 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-511531f03f6so895010e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 06:56:12 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 tb21-20020a1709078b9500b00a380a1ee2b0sm840342ejc.25.2024.02.07.06.56.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 06:56:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08e3c0cd-c5c9-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707317771; x=1707922571; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2ed76aC18yY98YVZXZt7AsqWqpjS5eaKS4tBHEoatpQ=;
        b=ffgy3MgAIwcB4+vvmMZL30pvZFqJ2ua4eoco8+OCeuirQid7bs9n4JkBX+p67QxzIi
         yKvKM2BDgOAYPLP2j+CQA48J0qRjoZXFeJ0ZSZ9NIG5fPlBbqxkhxadXDGe7cRviqbVd
         Tc+rLC06Fz1E516rLfOAY3EkTY7VWnhdEr3dc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707317771; x=1707922571;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=2ed76aC18yY98YVZXZt7AsqWqpjS5eaKS4tBHEoatpQ=;
        b=gqkeeJqiYkIO8bldNb2bSrb/Z0H8Y1wTwOtYk0/6D/4f8H5YMh8tj8zUCjFTucqpEm
         gmiOE3pwJfHPN4xo1jV2XG6lZLfBWkuW7f7nqw5bhPqkHnrp70OWvx9odoiN/6+bNFba
         pT/RaxPELe5IYK8e9XORp0HCsx31vEy64niHmwAlA1MykJuxoUhb2RAOJlfq+tI6s4Nn
         Ls7zjj7cKyUNipDKY74gNZtIeLrVKxmT12UFC+IDBqJS4QFy9dyu2dk4AVr37SXJIA1p
         b2q3EraLA0jzcscxUCiAsPprbBvrsFwxNvhGUxW8LItJWNBxU5cof4rFzrmFN8Yod0q+
         /AOQ==
X-Gm-Message-State: AOJu0YyHovjPoU62jeVyUrQZk8amfhdlIXTlXVGClBg75yq9ZsJDZNCI
	1RsYSyBkirGB3aa1zVoiDiAwSXqmtKq79zGjAISKHUd3CE+ZIML9XYbwyEyjAX/IeMvguJ2TlNq
	j
X-Google-Smtp-Source: AGHT+IFjeD8qDT4xiVr2PUwhzlFFk5FQqMJQTzmBOSuqXkY85fRKeoE9V31qv2BMZZ3sUhl7/cnDgQ==
X-Received: by 2002:a05:6512:64:b0:511:32cc:1c4c with SMTP id i4-20020a056512006400b0051132cc1c4cmr4090745lfo.40.1707317771143;
        Wed, 07 Feb 2024 06:56:11 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWZDc68kXxHbMNCiBjGEn/a0QM3LIn8rfYvRUQUi478C0UQqxSFN/K6AnPu1utTjEZmyXBs3qYeNFZSI7mn0fqdTkFnxCYLLJYmgFbSbdFQl0ic0LQoaALVUDGoWngQPjWDi+vKi019BuUPLzADdEX3+5oyjBA3pwz3/jqOTaKUq/tQjAdxY9WM+vLvrDtGvXPtDYc4XLRciPOXtNqEzcYVEcirEx5ueeXM+VMEcxha4hhp7TQfVuHoH2OcHGO9D9zGMdDWTb1EpuwXe4CFR5YxlbZrDTt4zOjr4XYXsY/99oH7YjUgI9Vi40MJyk3eZm2wcHJOTR81mGIbajLbX0iY02r5Pk3vxwB4dvW6CUegfJISvC4QMt3cetooXA1/Qn+5ns6RQH3fpZQnLik5g7llzi4ve2vMqAteT56qVl4JoOyr94FjXjy30YTh56Hoe60wp/wbBUkxNwF8DNs7ohgW4JT3iteWCChBm288x8oPy3ea/+skvN7auSrTHns+FDOtH8OmFXenfIAqT/es0zW/xEESneQ=
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v6 3/3] xen/livepatch: align functions to ensure minimal distance between entry points
Date: Wed,  7 Feb 2024 15:55:47 +0100
Message-ID: <20240207145547.89689-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240207145547.89689-1-roger.pau@citrix.com>
References: <20240207145547.89689-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The minimal function size requirements for an x86 livepatch are either 5 bytes
(for jmp) or 9 bytes (for endbr + jmp), and always 4 bytes on Arm.  Ensure that
distance between functions entry points is always at least of the minimal
required size for livepatch instruction replacement to be successful.

Add an additional align directive to the linker scripts, in order to ensure that
the next section placed after the .text.* (per-function sections) is also
aligned to the required boundary, so that the distance of the last function
entry point with the next symbol is also of minimal size.

Note that livepatch-build-tools will take into account each per-function
section alignment in order to decide whether there's enough padding.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v5:
 - Split chunks into pre-patches to make review easier.

Changes since v4:
 - Split from the rest of the livepatch testing series.
 - Reword and expand a bit the commit message.
 - Add a comment about falign-functions clang version requirement.

Changes since v3:
 - Test for compiler option with -falign-functions.
 - Make FUNCTION_ALIGNMENT depend on CC_HAS_FUNCTION_ALIGNMENT.
 - Set 16byte function alignment for x86 release builds.

Changes since v2:
 - Add Arm side.
 - Align end of section in the linker script to ensure enough padding for the
   last function.
 - Expand commit message and subject.
 - Rework Kconfig options.
 - Check that the compiler supports the option.

Changes since v1:
 - New in this version.
---
 xen/arch/arm/livepatch.c | 2 ++
 xen/arch/arm/xen.lds.S   | 4 ++++
 xen/arch/ppc/xen.lds.S   | 4 ++++
 xen/arch/riscv/xen.lds.S | 4 ++++
 xen/arch/x86/livepatch.c | 4 ++++
 xen/arch/x86/xen.lds.S   | 4 ++++
 xen/common/Kconfig       | 5 ++++-
 7 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/livepatch.c b/xen/arch/arm/livepatch.c
index bbca1e5a5ed3..aa8ae8c38d28 100644
--- a/xen/arch/arm/livepatch.c
+++ b/xen/arch/arm/livepatch.c
@@ -68,6 +68,8 @@ void arch_livepatch_revive(void)
 
 int arch_livepatch_verify_func(const struct livepatch_func *func)
 {
+    BUILD_BUG_ON(ARCH_PATCH_INSN_SIZE > CONFIG_FUNCTION_ALIGNMENT);
+
     /* If NOPing only do up to maximum amount we can put in the ->opaque. */
     if ( !func->new_addr && (func->new_size > LIVEPATCH_OPAQUE_SIZE ||
          func->new_size % ARCH_PATCH_INSN_SIZE) )
diff --git a/xen/arch/arm/xen.lds.S b/xen/arch/arm/xen.lds.S
index 470c8f22084f..0126a3afae53 100644
--- a/xen/arch/arm/xen.lds.S
+++ b/xen/arch/arm/xen.lds.S
@@ -46,6 +46,10 @@ SECTIONS
 #ifdef CONFIG_CC_SPLIT_SECTIONS
        *(.text.*)
 #endif
+#ifdef CONFIG_LIVEPATCH
+       /* Ensure enough distance with the next placed section. */
+       . = ALIGN(CONFIG_FUNCTION_ALIGNMENT);
+#endif
 
        *(.fixup)
        *(.gnu.warning)
diff --git a/xen/arch/ppc/xen.lds.S b/xen/arch/ppc/xen.lds.S
index 030e1ee37b55..f300c03a666f 100644
--- a/xen/arch/ppc/xen.lds.S
+++ b/xen/arch/ppc/xen.lds.S
@@ -32,6 +32,10 @@ SECTIONS
 #ifdef CONFIG_CC_SPLIT_SECTIONS
         *(.text.*)
 #endif
+#ifdef CONFIG_LIVEPATCH
+       /* Ensure enough distance with the next placed section. */
+       . = ALIGN(CONFIG_FUNCTION_ALIGNMENT);
+#endif
 
         *(.fixup)
         *(.gnu.warning)
diff --git a/xen/arch/riscv/xen.lds.S b/xen/arch/riscv/xen.lds.S
index 8510a87c4d06..1fb9af11c1cc 100644
--- a/xen/arch/riscv/xen.lds.S
+++ b/xen/arch/riscv/xen.lds.S
@@ -27,6 +27,10 @@ SECTIONS
 #ifdef CONFIG_CC_SPLIT_SECTIONS
         *(.text.*)
 #endif
+#ifdef CONFIG_LIVEPATCH
+       /* Ensure enough distance with the next placed section. */
+       . = ALIGN(CONFIG_FUNCTION_ALIGNMENT);
+#endif
 
         . = ALIGN(IDENT_AREA_SIZE);
         _ident_start = .;
diff --git a/xen/arch/x86/livepatch.c b/xen/arch/x86/livepatch.c
index ee539f001b73..b00ad7120da9 100644
--- a/xen/arch/x86/livepatch.c
+++ b/xen/arch/x86/livepatch.c
@@ -109,6 +109,10 @@ int arch_livepatch_verify_func(const struct livepatch_func *func)
          */
         uint8_t needed = ARCH_PATCH_INSN_SIZE;
 
+        BUILD_BUG_ON(ARCH_PATCH_INSN_SIZE +
+                     (IS_ENABLED(CONIFG_XEN_IBT) ? ENDBR64_LEN : 0) >
+                     CONFIG_FUNCTION_ALIGNMENT);
+
         if ( is_endbr64(func->old_addr) || is_endbr64_poison(func->old_addr) )
             needed += ENDBR64_LEN;
 
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index 8930e14fc40e..6649bc16dc48 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -101,6 +101,10 @@ SECTIONS
        *(.text.*)
 #endif
        *(.text.__x86_indirect_thunk_*)
+#ifdef CONFIG_LIVEPATCH
+       /* Ensure enough distance with the next placed section. */
+       . = ALIGN(CONFIG_FUNCTION_ALIGNMENT);
+#endif
 
        *(.fixup)
        *(.gnu.warning)
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 310ad4229cdf..63fba175c99f 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -395,8 +395,11 @@ config CRYPTO
 config LIVEPATCH
 	bool "Live patching support"
 	default X86
-	depends on "$(XEN_HAS_BUILD_ID)" = "y"
+	depends on "$(XEN_HAS_BUILD_ID)" = "y" && CC_HAS_FUNCTION_ALIGNMENT
 	select CC_SPLIT_SECTIONS
+	select FUNCTION_ALIGNMENT_16B if XEN_IBT
+	select FUNCTION_ALIGNMENT_8B  if X86
+	select FUNCTION_ALIGNMENT_4B  if ARM
 	---help---
 	  Allows a running Xen hypervisor to be dynamically patched using
 	  binary patches without rebooting. This is primarily used to binarily
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 15:08:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 15:08:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677747.1054552 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjXR-0003SL-7l; Wed, 07 Feb 2024 15:08:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677747.1054552; Wed, 07 Feb 2024 15:08:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjXR-0003SE-4y; Wed, 07 Feb 2024 15:08:37 +0000
Received: by outflank-mailman (input) for mailman id 677747;
 Wed, 07 Feb 2024 15:08:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Py/x=JQ=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rXjXP-0003S8-Uu
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 15:08:35 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c2ae3893-c5ca-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 16:08:33 +0100 (CET)
Received: from [192.168.1.15] (host-87-16-84-78.retail.telecomitalia.it
 [87.16.84.78])
 by support.bugseng.com (Postfix) with ESMTPSA id ED8F84EE0738;
 Wed,  7 Feb 2024 16:08:32 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c2ae3893-c5ca-11ee-98f5-efadbce2ee36
Message-ID: <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
Date: Wed, 7 Feb 2024 16:08:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US, it
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 07/02/24 15:16, Jan Beulich wrote:
> On 07.02.2024 14:51, Federico Serafini wrote:
>> On 07/02/24 08:38, Jan Beulich wrote:
>>> On 07.02.2024 02:08, Stefano Stabellini wrote:
>>>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>>>        case 8:                                                                \
>>>>>>            put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>>>            break;                                                             \
>>>>>> -    default: __put_user_bad();                                             \
>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>        }                                                                      \
>>>>>>        clac();                                                                \
>>>>>>    } while ( false )
>>>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>>>        case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>>>        case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>>>        case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>>>> -    default: __get_user_bad();                                             \
>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>        }                                                                      \
>>>>>>        clac();                                                                \
>>>>>>    } while ( false )
>>>>>
>>>>> Related to my remark on patch 1 - how is one to know the macro this was
>>>>> invoked from, when seeing the resulting diagnostic?
>>>>
>>>> I am not sure what do you mean here... we do get an error like the
>>>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>>>
>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>
>>> Right - and how do I know what _user_ of the macro actually triggered
>>> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
>>> __FUNCTION__ here, for that specific purpose ...
>>
>> To test the macro and its diagnostics,
>> I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
>> on the x86 code with STATIC_ASSERT_UNREACHABLE(),
>> that is in file arch/x86/alternative.c, line 312,
>> function _apply_alternatives().
>>
>> What I got is the following build error:
>>
>> ...
>> arch/x86/alternative.c: Assembler messages:
>> arch/x86/alternative.c:312: Error: static assertion failed: unreachable
>>     CC      arch/x86/copy_page.o
>> make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1
> 
> But that's not what my request was about. Here sufficient context is
> given, even if it would be nice if the function was also visible right
> away. But that's not the same as the case above, where the new macro
> is used inside another macro.

An example of that is the get_unsafe_size() macro,
whose body uses STATIC_ASSERT_UNREACHABLE().
A wrong use of get_unsafe_size() at line n
leads to a build error pointing to the line n,
isn't this the desired behavior?

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 15:08:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 15:08:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677748.1054564 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjXc-0003mM-Hg; Wed, 07 Feb 2024 15:08:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677748.1054564; Wed, 07 Feb 2024 15:08:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjXc-0003mF-Cs; Wed, 07 Feb 2024 15:08:48 +0000
Received: by outflank-mailman (input) for mailman id 677748;
 Wed, 07 Feb 2024 15:08:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXjXb-0003S8-DD
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 15:08:47 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c9d2726d-c5ca-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 16:08:45 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33b40208735so535310f8f.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 07:08:45 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p5-20020a05600c430500b0040fd3121c4asm2241587wme.46.2024.02.07.07.08.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 07:08:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c9d2726d-c5ca-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707318525; x=1707923325; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=ncbHwP6wrgBev/dEAPt4B8n9MsTfuVc0R/BDYiBxm7I=;
        b=HO0v/oNk7GgTTMwbOmP05neLcZh4mFxlA5CbkF1koMq2UUuSx2FYZV+HDez2jY//7D
         mSVtTBsQfA5imPPK3UoMCNX+zqkB0nfGwturMAZELcx6whhquy+GA2WZg4JfaX3uk0cQ
         SqIbSZN/wu93Zvj2qSBFHH8fSU8t9+F45UEVDd9sCHx13HnAioTA8iwTdTvlybs7shAB
         2kJq5wqIbvCgmhuvbS/kycLErjNQHLu35HgHRPLKIG67DxAQbVAlar8/3GzUr+7iDjVe
         r3ZY8ZqYJ1802EjvmOFp4pDpNh1EmES7kAQNmbHBG/mq1doNd1307acRDnc8aHgaPcNc
         9WyA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707318525; x=1707923325;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=ncbHwP6wrgBev/dEAPt4B8n9MsTfuVc0R/BDYiBxm7I=;
        b=qKqvOXTebVlpMk9AMA8ML+iLlJv8RABbvRmb2I1pxyP4rLr5BVuI+3m2qbVNcKXN9V
         KJSHvsYOMA3A8LuO2U9d8mOf0PAUWM26OI2JqstFpsaGygCud7xlIhE6kHn5VKiLP9Xx
         wF1dtdSiEshcHEBt+8Oh/tRtq26Ksnj7j3fNNHqly1pH0wDwregh8PKle9aoTTffEM8y
         VK4MsvwV6M3M4GGawtnV7aQaE6JFBNxSrrWffbSbcKqb1jyWgGPk4DRgwR3ealypB7Jn
         EAlvTKokwxNeb41WDMDANpK1JH+v8bfweiLpNklAxnI36ss0uC/uDQIPxzukN0PxUfdg
         eDZA==
X-Gm-Message-State: AOJu0YyDSn4uStWXAB/5GMzCDws6IU/keoewY7a8SVYtKn7888cTRhHA
	fAcWaAYm/UVqi94u+TZiNn0SvHuD4vbJ+SJK/jOophn4E7/HQKBVx0hDqLKkfQ==
X-Google-Smtp-Source: AGHT+IHpTZbVlXr6oUYAFfED3ujMa1YPEDCizZUL0qYIsnLU1QfxeZLUqnPn2oqdtgJdan/kkrVMwA==
X-Received: by 2002:adf:e50b:0:b0:33b:4f78:a5b8 with SMTP id j11-20020adfe50b000000b0033b4f78a5b8mr1538168wrm.44.1707318525010;
        Wed, 07 Feb 2024 07:08:45 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWtjlpRWbb5uFymD3YuVK3I1HwqWinyKwXYL860LzvmTS8AEkqDEqAGOkOukoo5+b3JueSTLUhSnMG9651mbS8+9e7sYP9xyxRAb8bnDNGTs7SUAhdTo3B5nNxsDVUpaFH5mF8=
Message-ID: <56c24b5b-42a0-4022-bb14-f90a3881fdc1@suse.com>
Date: Wed, 7 Feb 2024 16:08:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: hvmloader - allow_memory_relocate overlaps
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Neowutran <xen@neowutran.ovh>
Cc: xen-devel@lists.xenproject.org, Anthony Perard
 <anthony.perard@citrix.com>, George Dunlap <george.dunlap@citrix.com>
References: <7oe275z3eap7rhdwmrm4mvqnjnhrpag5cjwnfvwsf7rchhkyjv@pd3abzwdhg6v>
 <217649a9-8399-48d3-ba49-ae22cacf0d4b@suse.com>
 <lyqv62ezqqaybcfuhkvbhiltvnxyy32wzr36kclh7bzrupcvib@a5fpoe6atykl>
 <1c857c96-9f2d-4787-8804-799a63a00480@suse.com>
 <7lhrvh2ad46gjke5kvy4pbvrhstv7ihwm64suqrkle4v43tos6@oktohrjfupkc>
 <6db70696-4b40-46d2-b19c-881820338ba4@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <6db70696-4b40-46d2-b19c-881820338ba4@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 04.01.2024 14:16, Jan Beulich wrote:
> On 22.12.2023 16:49, Neowutran wrote:
>> Full logs without my patch to set allow-memory-relocate (https://github.com/neowutran/qubes-vmm-xen/blob/allowmemoryrelocate/ALLOWMEMORYRELOCATE.patch)
>> https://pastebin.com/g 
>> QGg55WZ
>> (GPU passthrough doesn't work, hvmloader overlaps with guest memory)
> 
> So there are oddities, but I can't spot any overlaps. What's odd is that
> the two blocks already above 4Gb are accounted for (and later relocated)
> when calculating total MMIO size. BARs of size 2Gb and more shouldn't be
> accounted for at all when deciding whether low RAM needs relocating, as
> those can't live below 4Gb anyway. I vaguely recall pointing this out
> years ago, but it was thought we'd get away for a fair while. What's
> further odd is where the two blocks are moved to: F800000 moves (down)
> to C00000, while the smaller FC00000 moves further up to FC80000.
> 
> I'll try to get to addressing at least the first oddity; if I can figure
> out why the second one occurs, I may try to address that as well.

Could you give the patch below a try? I don't have a device with large
enough a BAR that I could sensibly pass through to a guest, so I was
only able to build-test the change.

Jan

hvmloader/PCI: skip huge BARs in certain calculations

BARs of size 2Gb and up can't possibly fit below 4Gb: Both the bottom of
the lower 2Gb range and the top of the higher 2Gb range have special
purpose. Don't even have them influence whether to (perhaps) relocate
low RAM.

Reported-by: Neowutran <xen@neowutran.ovh>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
If we wanted to fit e.g. multiple 1Gb BARs, it would likely be prudent
to similarly avoid low RAM relocation in the first place. Yet accounting
for things differently depending on many large BARs there are would
require more intrusive code changes.

That said, I'm open to further lowering of the threshold. That'll
require different justification then, though.

--- a/tools/firmware/hvmloader/pci.c
+++ b/tools/firmware/hvmloader/pci.c
@@ -33,6 +33,13 @@ uint32_t pci_mem_start = HVM_BELOW_4G_MM
 const uint32_t pci_mem_end = RESERVED_MEMBASE;
 uint64_t pci_hi_mem_start = 0, pci_hi_mem_end = 0;
 
+/*
+ * BARs larger than this value are put in 64-bit space unconditionally.  That
+ * is, such BARs also don't play into the determination of how big the lowmem
+ * MMIO hole needs to be.
+ */
+#define HUGE_BAR_THRESH GB(1)
+
 enum virtual_vga virtual_vga = VGA_none;
 unsigned long igd_opregion_pgbase = 0;
 
@@ -286,9 +293,11 @@ void pci_setup(void)
             bars[i].bar_reg = bar_reg;
             bars[i].bar_sz  = bar_sz;
 
-            if ( ((bar_data & PCI_BASE_ADDRESS_SPACE) ==
-                  PCI_BASE_ADDRESS_SPACE_MEMORY) ||
-                 (bar_reg == PCI_ROM_ADDRESS) )
+            if ( is_64bar && bar_sz > HUGE_BAR_THRESH )
+                bar64_relocate = 1;
+            else if ( ((bar_data & PCI_BASE_ADDRESS_SPACE) ==
+                       PCI_BASE_ADDRESS_SPACE_MEMORY) ||
+                      (bar_reg == PCI_ROM_ADDRESS) )
                 mmio_total += bar_sz;
 
             nr_bars++;
@@ -367,7 +376,7 @@ void pci_setup(void)
             pci_mem_start = hvm_info->low_mem_pgend << PAGE_SHIFT;
     }
 
-    if ( mmio_total > (pci_mem_end - pci_mem_start) )
+    if ( mmio_total > (pci_mem_end - pci_mem_start) || bar64_relocate )
     {
         printf("Low MMIO hole not large enough for all devices,"
                " relocating some BARs to 64-bit\n");
@@ -446,8 +455,9 @@ void pci_setup(void)
          *   the code here assumes it to be.)
          * Should either of those two conditions change, this code will break.
          */
-        using_64bar = bars[i].is_64bar && bar64_relocate
-            && (mmio_total > (mem_resource.max - mem_resource.base));
+        using_64bar = bars[i].is_64bar && bar64_relocate &&
+            (mmio_total > (mem_resource.max - mem_resource.base) ||
+             bar_sz > HUGE_BAR_THRESH);
         bar_data = pci_readl(devfn, bar_reg);
 
         if ( (bar_data & PCI_BASE_ADDRESS_SPACE) ==
@@ -467,7 +477,8 @@ void pci_setup(void)
                 resource = &mem_resource;
                 bar_data &= ~PCI_BASE_ADDRESS_MEM_MASK;
             }
-            mmio_total -= bar_sz;
+            if ( bar_sz <= HUGE_BAR_THRESH )
+                mmio_total -= bar_sz;
         }
         else
         {




From xen-devel-bounces@lists.xenproject.org Wed Feb 07 15:10:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 15:10:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677753.1054574 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjZ2-0005d7-SF; Wed, 07 Feb 2024 15:10:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677753.1054574; Wed, 07 Feb 2024 15:10:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjZ2-0005d0-NW; Wed, 07 Feb 2024 15:10:16 +0000
Received: by outflank-mailman (input) for mailman id 677753;
 Wed, 07 Feb 2024 15:10:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9378=JQ=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rXjZ1-0005bg-BM
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 15:10:15 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fed821be-c5ca-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 16:10:14 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-5116a6fb076so313767e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 07:10:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fed821be-c5ca-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707318614; x=1707923414; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=G34MnWk3pe/7uEF/KUF82whwYiU9mlpjrdFqc2kuT6M=;
        b=n98CNTxk9feSIDUvtScvh1jWxU+JUjGaAbCzJmXbPUgkbyghPDssKrxYVrAdeWkBKq
         MDVvx3pG89hFn0kkraqimbKGRkaCSYIzDpQljG6tiaszCgtACL5mZvKvpN/WGSS4DrC6
         E0bYRhlpZxrskL2N0J17LhqnoRWDErY+32Ls4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707318614; x=1707923414;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=G34MnWk3pe/7uEF/KUF82whwYiU9mlpjrdFqc2kuT6M=;
        b=egcYraaP3ekyWqdIeZL291NlHMVsA2Y7XcrZ2ny4Tqhd3S3ftLKszOhysrXMTT5lAD
         5YkCfaE0RMlVjfHvvXza6QxwRpQYzegPntiYSmXG7wT54+mKyL8gacbJHFAR+bG+3ZAG
         LLYpOPCeabkjQS7k0Ed86cEkafkxds7KzBDwhT4zWFjpKGb9yThtyN+pLbXwDMXE8ZHI
         8p3iPs8tqypf0DTfBQ4EoRr/BCrdR+0oRShKVxzTnKxm5Us21Y5nWN223Bl6mz3IIU3P
         bxBNjrx1E2Z8+SGrhAddr3onA/aoSOrEG7xl3awRCXBqJwJJA6FFtfl5wcb1mFu3aTWa
         Jv9Q==
X-Gm-Message-State: AOJu0YytXKc0r0b0j0C5l1KO7m8wF9/d8grYwdipjPvtXDZ4/xaxslkL
	ahpPwO44jT7aTpCws+pDMdl2uJ8H9X45aiqw11mol2/BJPElAZ6dQlId9ng3brNW4b/eekIzfCI
	CoGa1jYCszlqJjOjXj8S//1gktSWYS1zbauV28g==
X-Google-Smtp-Source: AGHT+IFfKHlX57Q4POVz4oD5zc2AhMSRd7knfkjStPeBLj3w/EAUV51LDHlvqqmtM0E9nSgdSGoy9wSeF/5F/Xwl0hI=
X-Received: by 2002:a05:6512:3494:b0:511:64c2:8c0a with SMTP id
 v20-20020a056512349400b0051164c28c0amr1649805lfr.20.1707318614080; Wed, 07
 Feb 2024 07:10:14 -0800 (PST)
MIME-Version: 1.0
References: <65c38d7a7ea07_2d1b5321c96792@gitlab-sidekiq-catchall-v2-779bdffdc8-kp98w.mail>
 <115f10ac-a115-40d1-a444-50e960320557@suse.com> <65665f86-cbfc-4352-9d81-67ac0b21b161@citrix.com>
 <CAKoJhMROFw1w679U9xsW6s4p34Zzm6-gaHihUHtU1w5ZH9ViUQ@mail.gmail.com> <f1f74e69-8892-4615-8411-f7a1329dd76c@citrix.com>
In-Reply-To: <f1f74e69-8892-4615-8411-f7a1329dd76c@citrix.com>
From: Anthony PERARD <anthony.perard@citrix.com>
Date: Wed, 7 Feb 2024 15:10:02 +0000
Message-ID: <CAKoJhMTaO+LW0Mm3mn+XXgxxM-=T5QwGacG5oGKwF-egEmes0Q@mail.gmail.com>
Subject: Re: xen | Failed pipeline for staging | f4519ee8
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 7, 2024 at 2:55=E2=80=AFPM Andrew Cooper <andrew.cooper3@citrix=
.com> wrote:
>
> On 07/02/2024 2:34 pm, Anthony PERARD wrote:
> > On Wed, Feb 7, 2024 at 2:24=E2=80=AFPM Andrew Cooper <andrew.cooper3@ci=
trix.com> wrote:
> >>>> Stage: test
> >>>> Name: qemu-smoke-riscv64-gcc
> >>> I have to admit that I can't connect what was pushed recently to this=
 job
> >>> failing.
> >> The qemu smoke tests for riscv and ppc intermittently fail on the
> >> OSSTest-lab infrastructure in Gitlab.
> >>
> >> We've never got to the bottom of it.
> > It's maybe because joubertin* and nocera* have more than one NUMA
> > nodes. It's look like 2 for nocera* and 4 for joubertin*.
> > Would that be enough of of an explanation about those tests taking a
> > bit longer than expected?
>
> That feels a bit like a blind guess.

Yes, but I think I did a test earlier, and the riscv test took more
like 10 seconds, but can also easily finish in less than a second. I
did the test a while ago, so I could misremember.

> What hardware are these?  They're rather old, I seem to recall?

nocera: http://logs.test-lab.xenproject.org/osstest/results/host/nocera1.ex=
amine/
joubertin: http://logs.test-lab.xenproject.org/osstest/results/host/joubert=
in0.examine/

nocera have been here for a while, since before 2015. I don't really
know how to find out how old the machine is. joubertin* have never
been used with osstest so far, but the first mention of it I can find
of it is from 2017.

--=20
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 15:16:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 15:16:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677759.1054583 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjf8-0006cx-Fk; Wed, 07 Feb 2024 15:16:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677759.1054583; Wed, 07 Feb 2024 15:16:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjf8-0006cq-D1; Wed, 07 Feb 2024 15:16:34 +0000
Received: by outflank-mailman (input) for mailman id 677759;
 Wed, 07 Feb 2024 15:16:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXjf7-0006ck-8O
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 15:16:33 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dffd0606-c5cb-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 16:16:32 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a38392b9917so104824066b.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 07:16:32 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ty20-20020a170907c71400b00a3890e238c2sm390526ejc.49.2024.02.07.07.16.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 07:16:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dffd0606-c5cb-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707318992; x=1707923792; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Az41Ic58DspSSGy3hwXVQtcJeCUrHwtkL7/lJNxi2uQ=;
        b=czyphCbWMc36vQC4+5yFq2uk9J4qBng1RvCmalzuC+U2ZWuM9+he2iIlYyIGmxujIQ
         TVNxdjgZ0Cpt3BxdDVhUWu1Xksn0D1wGVbYVa6fFEeP6W9NYDGlB8UrCDAOnWOldt9y8
         79YisaK/A526ia6hx9Kc6eu2+0ke96D/AWDXg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707318992; x=1707923792;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Az41Ic58DspSSGy3hwXVQtcJeCUrHwtkL7/lJNxi2uQ=;
        b=TGBG1gCMLnQDz4byH5UYeDcC0mYII/7icS5rwX5f+Z9dpSmj1YTTop9LsRFwLmBels
         rY6Chn3eYXZX6x0t3t1d3H9jwUBdPrawPsKhPE3skWCgrXTSdd2sk4Ns7cxK7PqTS4t1
         NwEaPs1MxwCQHXxv0V/2mPUG117sfqsxurkwSm7zG8XZtJstcjpeT2hCyFJCGiYEytAw
         hhHRIwVDrBqe9mqmNs0dTz6xw+c6idJXzWxkgXr7HdiCJJci4vOdKkkNvJXAp+xxCByK
         P51fUwpRuJx8Gwy5xeWrvQLhBFe+4yB5ulYXpNwg898f/GHBJl7l8qJSQ8YIdZavcmvV
         xYlA==
X-Gm-Message-State: AOJu0YwCG1BQTu4mOc8c8nq96PCM3hxBX9OuUNcQfpZDWDaUqDnmuz9l
	rKGDG1A1qLUSNMEmQ8A7Sfjyu+8yZAjOSZyQzgdcIMkv2gtLeUDoJMI5TguEuAHoa3tXUfWzz/W
	D
X-Google-Smtp-Source: AGHT+IFBdvYPJCpzBG7wj1ydOARTl+Ioh7VKiMrOKV8YrO9jXEJXuC48GLArUB06hYVZ5GY2qUD5gQ==
X-Received: by 2002:a17:907:784e:b0:a36:f811:44f5 with SMTP id lb14-20020a170907784e00b00a36f81144f5mr3626245ejc.47.1707318991843;
        Wed, 07 Feb 2024 07:16:31 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVzCkHiblwylHKZvOzOcbM8UjS9DldL68rqlXMsc2w6+FXf6LD3aWkZ+JmxjEGeCg8eaOSiYffPvlsL+7d+CTcRrNmxaXWhgLQRe/B6hVg=
Message-ID: <c443cb04-177b-4207-a9a1-3a0dca32fd28@citrix.com>
Date: Wed, 7 Feb 2024 15:16:29 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: xen | Failed pipeline for staging | f4519ee8
Content-Language: en-GB
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <65c38d7a7ea07_2d1b5321c96792@gitlab-sidekiq-catchall-v2-779bdffdc8-kp98w.mail>
 <115f10ac-a115-40d1-a444-50e960320557@suse.com>
 <65665f86-cbfc-4352-9d81-67ac0b21b161@citrix.com>
 <CAKoJhMROFw1w679U9xsW6s4p34Zzm6-gaHihUHtU1w5ZH9ViUQ@mail.gmail.com>
 <f1f74e69-8892-4615-8411-f7a1329dd76c@citrix.com>
 <CAKoJhMTaO+LW0Mm3mn+XXgxxM-=T5QwGacG5oGKwF-egEmes0Q@mail.gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <CAKoJhMTaO+LW0Mm3mn+XXgxxM-=T5QwGacG5oGKwF-egEmes0Q@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/02/2024 3:10 pm, Anthony PERARD wrote:
> On Wed, Feb 7, 2024 at 2:55 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> On 07/02/2024 2:34 pm, Anthony PERARD wrote:
>>> On Wed, Feb 7, 2024 at 2:24 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>>>>> Stage: test
>>>>>> Name: qemu-smoke-riscv64-gcc
>>>>> I have to admit that I can't connect what was pushed recently to this job
>>>>> failing.
>>>> The qemu smoke tests for riscv and ppc intermittently fail on the
>>>> OSSTest-lab infrastructure in Gitlab.
>>>>
>>>> We've never got to the bottom of it.
>>> It's maybe because joubertin* and nocera* have more than one NUMA
>>> nodes. It's look like 2 for nocera* and 4 for joubertin*.
>>> Would that be enough of of an explanation about those tests taking a
>>> bit longer than expected?
>> That feels a bit like a blind guess.
> Yes, but I think I did a test earlier, and the riscv test took more
> like 10 seconds, but can also easily finish in less than a second. I
> did the test a while ago, so I could misremember.

Ok - that is a wild variation.  Something wonky is clearly going on.

>
>> What hardware are these?  They're rather old, I seem to recall?
> nocera: http://logs.test-lab.xenproject.org/osstest/results/host/nocera1.examine/

AMD Opteron(tm) Processor 4133

a.k.a. Lisbon uarch from 2010.


> joubertin: http://logs.test-lab.xenproject.org/osstest/results/host/joubertin0.examine/

AMD Opteron(tm) Processor 6376

a.k.a. Abu-Dhabi uarch from 2012.

> nocera have been here for a while, since before 2015. I don't really
> know how to find out how old the machine is. joubertin* have never
> been used with osstest so far, but the first mention of it I can find
> of it is from 2017.

They were ancient when added to OSSTest.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 15:25:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 15:25:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677768.1054593 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjnH-0000s3-Du; Wed, 07 Feb 2024 15:24:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677768.1054593; Wed, 07 Feb 2024 15:24:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjnH-0000rw-Af; Wed, 07 Feb 2024 15:24:59 +0000
Received: by outflank-mailman (input) for mailman id 677768;
 Wed, 07 Feb 2024 15:24:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXjnF-0000rq-9x
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 15:24:57 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0c402d19-c5cd-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 16:24:56 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33b29b5ea86so491883f8f.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 07:24:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 9-20020a05600c234900b0040fccf7e8easm5479789wmq.36.2024.02.07.07.24.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 07:24:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c402d19-c5cd-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707319495; x=1707924295; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=qdHBKMdgeAhwXxCrB96lMAYx4N2sOhdnLI6MGOgTijk=;
        b=ABcrKEfiHVditMCpoBP5vBRmjqIuY8+Dp+ketAm7qCZuKltIsJFNeWtIoRwFgEJ62o
         WonIT2cTxzPOc38IAAl3E2eEgBGNrP6NoiLsYO8g4B7w4Y3U/TMeJeJxF43Rq/+SiRVV
         h54gDV23CmneJaLBDD3mvEWjmQ5WM8cULZm7QULce6wppil/ilq8yKA/2aDmE/XBujNi
         DrWXYVj9tnS+T7uU9qyCzc61DbWNJmdmdoybcSkYab6zK6+gGV29rlPR3SmzxqqY6rc3
         +0Ttg13W6HT8XTzoh76eYG5QuTdcnUhmiyWUhSzU6WXlM/KKSPkwGw/jpUrSHN7asneR
         TbhQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707319495; x=1707924295;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qdHBKMdgeAhwXxCrB96lMAYx4N2sOhdnLI6MGOgTijk=;
        b=WerfaO2Hh5wKURjosAe084lwiYofj9Co2xGoAnGfwq6+4fhEOWeMTzTlf7/rJsswmC
         yN6gvE5AXC4tJy75KQuH/NALNr48LpDIRtX/FZNiCtZ0Z7XvV++NoU91LQcRzC/51lEH
         /IN7k6TmDeQ8kbZy3oPx9qpAOSRn6Nq9GG8EnqbkKiXEUtBp3SrGEWYZFSviTBevSW7U
         5Cxx068aYfvvWhRkZrQXxqK4GyzI8I6+b5qVHXO9sprosmK9oqjVoWFMFK+LQUpCVBPz
         FG55lPcp4HE1IaWb9wVfFhhV43vOQGJ6FNK6CV4M3NkYiF5QmrXrTEKnWguN/KE7KldW
         Bueg==
X-Forwarded-Encrypted: i=1; AJvYcCVqQ5Rv+P9Q0DvaVS8yivStzKwxYkeVaR5N+QfMOE5XGAqSnf40/8IWeX8AtBvOUbBrWAAWkqJs33aDs3UofjoGscUdx5YbzDRWXDzwv8w=
X-Gm-Message-State: AOJu0YyZGN0+C7wFEJP6eLFC9q+CrpTtp5KPg3LEVaENZa7W8F3uxPlK
	7bgpEkXJMpIsp1mDstl16aohrl7ScCA3TDgZdH1Vo0Iqd18g++R6PvtA+yFihQ==
X-Google-Smtp-Source: AGHT+IH3QFYSNj8AXL/icdI5j0lqVAZqHW19RzZ5QoYqXcaaamLbaL639Ac1QZhDOx4Z73cHM++YIw==
X-Received: by 2002:adf:f450:0:b0:33b:4d27:6303 with SMTP id f16-20020adff450000000b0033b4d276303mr2037353wrp.12.1707319495485;
        Wed, 07 Feb 2024 07:24:55 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWuSbTDSsh1iuvlS01Vj3cu3TGi9aYr6QPGWkG+wisW6iTTVg8jRMJrHlF5GcbzCJ0HM2lDZ14BHkxBcggyDeOVnFB4/cPca1OuV2wOg5yqQlsl1BHCxbzDYkrpbo/87nmA3B12VLIUF9VIdkxZiBQVkDl31E8xkXc3FupBQ3L7CXmMBtGV+yptHOLmp71yI/PP+/2IkVITFOYF5nCI0p5Xfdl4
Message-ID: <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
Date: Wed, 7 Feb 2024 16:24:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 16:08, Federico Serafini wrote:
> On 07/02/24 15:16, Jan Beulich wrote:
>> On 07.02.2024 14:51, Federico Serafini wrote:
>>> On 07/02/24 08:38, Jan Beulich wrote:
>>>> On 07.02.2024 02:08, Stefano Stabellini wrote:
>>>>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>>>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>>>>        case 8:                                                                \
>>>>>>>            put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>>>>            break;                                                             \
>>>>>>> -    default: __put_user_bad();                                             \
>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>        }                                                                      \
>>>>>>>        clac();                                                                \
>>>>>>>    } while ( false )
>>>>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>>>>        case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>>>>        case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>>>>        case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>>>>> -    default: __get_user_bad();                                             \
>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>        }                                                                      \
>>>>>>>        clac();                                                                \
>>>>>>>    } while ( false )
>>>>>>
>>>>>> Related to my remark on patch 1 - how is one to know the macro this was
>>>>>> invoked from, when seeing the resulting diagnostic?
>>>>>
>>>>> I am not sure what do you mean here... we do get an error like the
>>>>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>>>>
>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>
>>>> Right - and how do I know what _user_ of the macro actually triggered
>>>> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
>>>> __FUNCTION__ here, for that specific purpose ...
>>>
>>> To test the macro and its diagnostics,
>>> I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
>>> on the x86 code with STATIC_ASSERT_UNREACHABLE(),
>>> that is in file arch/x86/alternative.c, line 312,
>>> function _apply_alternatives().
>>>
>>> What I got is the following build error:
>>>
>>> ...
>>> arch/x86/alternative.c: Assembler messages:
>>> arch/x86/alternative.c:312: Error: static assertion failed: unreachable
>>>     CC      arch/x86/copy_page.o
>>> make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1
>>
>> But that's not what my request was about. Here sufficient context is
>> given, even if it would be nice if the function was also visible right
>> away. But that's not the same as the case above, where the new macro
>> is used inside another macro.
> 
> An example of that is the get_unsafe_size() macro,
> whose body uses STATIC_ASSERT_UNREACHABLE().
> A wrong use of get_unsafe_size() at line n
> leads to a build error pointing to the line n,
> isn't this the desired behavior?

Aiui this would point to the line in the header file, when what you need
to spot the bad use of the macro is the line in the source file actually
using the macro. Quoting from an earlier mail of yours:

./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable

Compare this with what we have today, where the linker will point out
the function it found the bad use in. Of course this could also be
solved by better assembler diagnostics, but I'm not sure compiler output
would actually lend itself to that. Specifically we'd then rely on the
.type directive always preceding the actual function. Plus while it may
be reasonably possible to change gas, I'm not as sure about Clang's
integrated assembler. Plus changing gas and then getting it into use by
people will take quite a bit of time.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 15:34:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 15:34:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677774.1054603 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjwh-0003SL-AZ; Wed, 07 Feb 2024 15:34:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677774.1054603; Wed, 07 Feb 2024 15:34:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjwh-0003SE-6i; Wed, 07 Feb 2024 15:34:43 +0000
Received: by outflank-mailman (input) for mailman id 677774;
 Wed, 07 Feb 2024 15:34:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gvxy=JQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXjwg-0003S3-0H
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 15:34:42 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 68c1c7c3-c5ce-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 16:34:41 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-40fdd65a9bdso6532175e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 07:34:41 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 y8-20020a1c4b08000000b0040efa7e0ef6sm1982413wma.1.2024.02.07.07.34.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 07:34:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68c1c7c3-c5ce-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707320079; x=1707924879; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=gAUGm5BQ2CDJSgo+yjmiDzO6K/rlrbqNJIMvARSp6N8=;
        b=cXCnLLh9N84OdjthCIE7V6f4S1TmpJV8f/bfd/gPL1jrSTbI+0ozoVGZ+ItjVZyow3
         8HWsFSvC0uYBvjwlhQ8dl4wR9vgEiIFVNh9kQvMN73oS58IH/W4YArZ8QRbD1Vef8jNx
         ytgE13TY4EMPuWqQGHMsirJMnTUoiKZBpH9uo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707320079; x=1707924879;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=gAUGm5BQ2CDJSgo+yjmiDzO6K/rlrbqNJIMvARSp6N8=;
        b=T5imYbETbnS+5zgvT4OqpOlrVbBZf81smlmep0sX8hqEKj85fUsiJXv398W18811GV
         Slnf3oEI/JiC+UM2XhCrLL/sUj0PlLvA0L/zZ0d3Tu8ZfV9eVc0FO/C0TXVUndsF5Y/T
         mWzQs1zVo12YdR1J6Cfb0i1UNJ4iKM6t8Ww25kX/vZQsKRA1Ri8v2B+Ovo55vNZnVhey
         Y6estN0FG8WDUwl8N00JKWF2rscBSl3lvKwxoJgUmlgvcHp5ItR9a9z/fAbbUw7HZz+f
         oKlxZvhZ0KU2kwYcZD1w8eYL9lc2Q+5Hjsh3MkeqkHcfWpD0EXZme9SO2zUjToLF27cj
         Mexg==
X-Gm-Message-State: AOJu0Ywwf36XxlYqDIOhokbTQqx7ks+UEHsOp8OzT+hGgezIfvX2ca8W
	vJd/75Z6v/7Z9LBRVz11wy0QsjHzi3XG7fkiaM3w6t6Si4A9SLYZ60TQTIzKPu+Bm2MDDeDHeOk
	w
X-Google-Smtp-Source: AGHT+IHQdJiMUElRKYkV3M2QvteScs7wP8fNpCIY8aDlKhEMYuhQkII21gP4aWfhMtW72liPaTE1zg==
X-Received: by 2002:a05:600c:1546:b0:40e:e6ce:71ff with SMTP id f6-20020a05600c154600b0040ee6ce71ffmr5121019wmg.41.1707320079483;
        Wed, 07 Feb 2024 07:34:39 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVEiYt/ZlEJtoORU7kmwIMAhM8Iw4nesL+AsoVCyQWc//fgThW/YHUyMTIdqcVkI0YEo5s3eJqOwNKl9tQpnLqYVTZNvq80ayXn9GaWa+BCvAhclfq4k69iza2SPd8Oapy1kXpxsx9EfEuj6d+QiB45Z3t36+5C
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>,
	Paul Durrant <paul@xen.org>
Subject: [PATCH v2 1/2] iommu/x86: introduce a generic IVMD/RMRR range validity helper
Date: Wed,  7 Feb 2024 16:34:16 +0100
Message-ID: <20240207153417.89975-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240207153417.89975-1-roger.pau@citrix.com>
References: <20240207153417.89975-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

IVMD and RMRR ranges are functionally equivalent, and as so could use the same
validity checker.

Move the IVMD to x86 common IOMMU code and adjust the function to take a pair
of [start, end] mfn parameters.

So far only the AMD-Vi side is adjusted to use the newly introduced helper, the
VT-d side will be adjusted in a further change.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Fix usage of ERR/WARN in messages.
 - Pass region as an inclusive range.
 - Pass log message prefix.
---
 xen/arch/x86/include/asm/iommu.h         |  3 ++
 xen/drivers/passthrough/amd/iommu_acpi.c | 38 ++------------------
 xen/drivers/passthrough/x86/iommu.c      | 46 ++++++++++++++++++++++++
 3 files changed, 52 insertions(+), 35 deletions(-)

diff --git a/xen/arch/x86/include/asm/iommu.h b/xen/arch/x86/include/asm/iommu.h
index 15a848ddc329..2b97cb442edd 100644
--- a/xen/arch/x86/include/asm/iommu.h
+++ b/xen/arch/x86/include/asm/iommu.h
@@ -135,6 +135,9 @@ struct page_info *__must_check iommu_alloc_pgtable(struct domain_iommu *hd,
                                                    uint64_t contig_mask);
 void iommu_queue_free_pgtable(struct domain_iommu *hd, struct page_info *pg);
 
+/* Check [start, end] unity map range for correctness. */
+bool iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end);
+
 #endif /* !__ARCH_X86_IOMMU_H__ */
 /*
  * Local variables:
diff --git a/xen/drivers/passthrough/amd/iommu_acpi.c b/xen/drivers/passthrough/amd/iommu_acpi.c
index ca70f4f3ae2c..3f5508eba049 100644
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -405,41 +405,9 @@ static int __init parse_ivmd_block(const struct acpi_ivrs_memory *ivmd_block)
         return 0;
     }
 
-    if ( !e820_all_mapped(base, limit + PAGE_SIZE, E820_RESERVED) )
-    {
-        paddr_t addr;
-
-        AMD_IOMMU_WARN("IVMD: [%lx,%lx) is not (entirely) in reserved memory\n",
-                       base, limit + PAGE_SIZE);
-
-        for ( addr = base; addr <= limit; addr += PAGE_SIZE )
-        {
-            unsigned int type = page_get_ram_type(maddr_to_mfn(addr));
-
-            if ( type == RAM_TYPE_UNKNOWN )
-            {
-                if ( e820_add_range(addr, addr + PAGE_SIZE,
-                                    E820_RESERVED) )
-                    continue;
-                AMD_IOMMU_ERROR("IVMD: page at %lx couldn't be reserved\n",
-                                addr);
-                return -EIO;
-            }
-
-            /*
-             * Types which aren't RAM are considered good enough.
-             * Note that a page being partially RESERVED, ACPI or UNUSABLE will
-             * force Xen into assuming the whole page as having that type in
-             * practice.
-             */
-            if ( type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
-                         RAM_TYPE_UNUSABLE) )
-                continue;
-
-            AMD_IOMMU_ERROR("IVMD: page at %lx can't be converted\n", addr);
-            return -EIO;
-        }
-    }
+    if ( !iommu_unity_region_ok("IVMD", maddr_to_mfn(base),
+                                maddr_to_mfn(limit)) )
+        return -EIO;
 
     if ( ivmd_block->header.flags & ACPI_IVMD_EXCLUSION_RANGE )
         exclusion = true;
diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index c90755ff58fa..d867490e25dc 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -792,6 +792,52 @@ static int __init cf_check adjust_irq_affinities(void)
 }
 __initcall(adjust_irq_affinities);
 
+bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
+{
+    mfn_t addr;
+
+    if ( e820_all_mapped(mfn_to_maddr(start), mfn_to_maddr(end) + PAGE_SIZE,
+                         E820_RESERVED) )
+        return true;
+
+    printk(XENLOG_WARNING "%s: [%#" PRI_mfn " ,%#" PRI_mfn
+           "] is not (entirely) in reserved memory\n",
+           prefix, mfn_x(start), mfn_x(end));
+
+    for ( addr = start; mfn_x(addr) <= mfn_x(end); mfn_add(addr, 1) )
+    {
+        unsigned int type = page_get_ram_type(addr);
+
+        if ( type == RAM_TYPE_UNKNOWN )
+        {
+            if ( e820_add_range(mfn_to_maddr(addr),
+                                mfn_to_maddr(addr) + PAGE_SIZE, E820_RESERVED) )
+                continue;
+            printk(XENLOG_ERR
+                   "%s: page at %#" PRI_mfn " couldn't be reserved\n",
+                   prefix, mfn_x(addr));
+            return false;
+        }
+
+        /*
+         * Types which aren't RAM are considered good enough.
+         * Note that a page being partially RESERVED, ACPI or UNUSABLE will
+         * force Xen into assuming the whole page as having that type in
+         * practice.
+         */
+        if ( type & (RAM_TYPE_RESERVED | RAM_TYPE_ACPI |
+                     RAM_TYPE_UNUSABLE) )
+            continue;
+
+        printk(XENLOG_ERR
+               "%s: page at %#" PRI_mfn " can't be converted\n",
+               prefix, mfn_x(addr));
+        return false;
+    }
+
+    return true;
+}
+
 /*
  * Local variables:
  * mode: C
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 15:34:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 15:34:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677776.1054623 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjwi-0003ui-QT; Wed, 07 Feb 2024 15:34:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677776.1054623; Wed, 07 Feb 2024 15:34:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjwi-0003ub-Mz; Wed, 07 Feb 2024 15:34:44 +0000
Received: by outflank-mailman (input) for mailman id 677776;
 Wed, 07 Feb 2024 15:34:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gvxy=JQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXjwh-0003S2-LK
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 15:34:43 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6986bc28-c5ce-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 16:34:42 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-40ef3f351d2so11075215e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 07:34:42 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 fa7-20020a05600c518700b0040ef3ae26cdsm2436907wmb.37.2024.02.07.07.34.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 07:34:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6986bc28-c5ce-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707320081; x=1707924881; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=xvplvaESko/5mMtO6f7pTc5NpoSjdl4OVHnVT3YtPaA=;
        b=sRNbGtjozI7jsBmiv6wWsMD9W42cQjB2QWz5wbCA4VKFP+TN9HBcQ8IamaA3cpWFxM
         vBh3HMy/o7HRZkS7s1Pt6v5FrrUjCL/t18yfpcwNDmMKPrBLLbQUZy0P8IiSulGQF9C/
         AEy1NUumpw/LAmKkKj/1iQa2diFSHpPxxCKmY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707320081; x=1707924881;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=xvplvaESko/5mMtO6f7pTc5NpoSjdl4OVHnVT3YtPaA=;
        b=XV7aUKm2qmo7OQlT/zYBr7GrKf/td5Yi0/9S4v4lRfZmfZMZQQ9dL8nsxSV2fFsCzW
         O/n/6RsOCk9dt4Kw/XRK2szFW1y9L8sUkZyxMYZmeDwo9TH46NCHUG/jK8foPGkiPMKU
         4jKkUUu5Bg/aAl/L9K9O6ovRijL14GJvQbwnardR3lY714erbBf2JroHFG0juFnL+XA3
         7+OZP8zBfoUh1TtHH4PVVthDo8+Hp3rpofQGZiNlyYS15Bek2Dzkr+1ZZ+filtHconk2
         RXpr9xbn4j0w107qdzsDSfItTOBbnS4969APXUF4XrvHwlNG5GDqy67YoRDjuP9rBSpj
         4Qmw==
X-Gm-Message-State: AOJu0YyLvZ3X92CmrUQGYBpf6AWrFqP/JDlfhO2G/OA8+HQS0Arsae/0
	/eHzmlB7Km9dWvcppQKOMcal++xFeW7/VpSHOKyBqiGkjerguUSzYSlgi0kbtNcwThz4MdqEGwD
	e
X-Google-Smtp-Source: AGHT+IFkS/KQnrbwKgRj5eFfG/1jKpAO+OKO8YN/8QkkTtfBcLgsExK9uaG+sSyEdWxyTU9ZOPpW2g==
X-Received: by 2002:a05:600c:3552:b0:40f:b69e:aa1e with SMTP id i18-20020a05600c355200b0040fb69eaa1emr9583718wmq.13.1707320080960;
        Wed, 07 Feb 2024 07:34:40 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUEuOBcJ3Alw3uB8O9HITmnlttfnBK6KRhKaEwkWf8MNCIcisvQNBCf9Jn4+syC4ScYJNdapPguiKYrkqXmARi4bV1Iq2/Bls/z+Lig+1Ydl1eFECOqrKogka1yRTAnRMYuXcKGVREpuNr8XwW3hSCPOSY=
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v2 2/2] iommu/vt-d: switch to common RMRR checker
Date: Wed,  7 Feb 2024 16:34:17 +0100
Message-ID: <20240207153417.89975-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240207153417.89975-1-roger.pau@citrix.com>
References: <20240207153417.89975-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Use the newly introduced generic unity map checker.

Also drop the message recommending the usage of iommu_inclusive_mapping: the
ranges would end up being mapped anyway even if some of the checks above
failed, regardless of whether iommu_inclusive_mapping is set.  Plus such option
is not supported for PVH, and it's deprecated.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Adjust to changes in the previous patch.
 - Expand commit message.
---
 xen/drivers/passthrough/vtd/dmar.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/xen/drivers/passthrough/vtd/dmar.c b/xen/drivers/passthrough/vtd/dmar.c
index 07772f178fe6..76aade816c08 100644
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -642,17 +642,9 @@ acpi_parse_one_rmrr(struct acpi_dmar_header *header)
            return -EEXIST;
        }
 
-    /* This check is here simply to detect when RMRR values are
-     * not properly represented in the system memory map and
-     * inform the user
-     */
-    if ( !e820_all_mapped(base_addr, end_addr + 1, E820_RESERVED) &&
-         !e820_all_mapped(base_addr, end_addr + 1, E820_NVS) &&
-         !e820_all_mapped(base_addr, end_addr + 1, E820_ACPI) )
-        printk(XENLOG_WARNING VTDPREFIX
-               " RMRR [%"PRIx64",%"PRIx64"] not in reserved memory;"
-               " need \"iommu_inclusive_mapping=1\"?\n",
-                base_addr, end_addr);
+    if ( !iommu_unity_region_ok("RMRR", maddr_to_mfn(base_addr),
+                                maddr_to_mfn(end_addr)) )
+        return -EIO;
 
     rmrru = xzalloc(struct acpi_rmrr_unit);
     if ( !rmrru )
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 15:34:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 15:34:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677775.1054607 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjwh-0003VT-Jb; Wed, 07 Feb 2024 15:34:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677775.1054607; Wed, 07 Feb 2024 15:34:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXjwh-0003Us-FD; Wed, 07 Feb 2024 15:34:43 +0000
Received: by outflank-mailman (input) for mailman id 677775;
 Wed, 07 Feb 2024 15:34:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gvxy=JQ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rXjwg-0003S2-Dr
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 15:34:42 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 67b2935c-c5ce-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 16:34:39 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-4100826df08so6601905e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 07:34:39 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 t20-20020a05600c199400b0040fff3382casm3133205wmq.46.2024.02.07.07.34.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 07:34:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67b2935c-c5ce-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707320078; x=1707924878; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=detjo7ZTC8akO4hEW6pkOY6GaqUpDczVv6k7EJU30NU=;
        b=fknb1yHW5U4Wy3O8vPJbYyHwXdgvV7A03JF8A8Xocy/0/NT/ic1zeXuou4yfSVncdj
         /6IbyqSXntwKlWMCMPgxDCCChTCi+yv66NzU3ysyvexSCAPqLNKu0e7WWj3ySJ49jkf0
         3JEoCtxMhWrpPm02JefchJFOvoG42EEzVVoFQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707320078; x=1707924878;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=detjo7ZTC8akO4hEW6pkOY6GaqUpDczVv6k7EJU30NU=;
        b=OpolSybWOozv/WlsDS7R2uHsBJiKsxWky43RtMhmsAqhyxizNIff6nQQkXjvTr7jBC
         /uFswGdRkyMPDOr8AlXYKT0YgASUq6S59ubinLAvYm/9kq0tZ4d+qHF/lcGLGMmMPeZa
         +9mMAaOoMaWY3VwMXM9mcr4adIHXpqPSVQQvOGmscGi2PSpV9sI+hviSvkvBztOvDuVn
         Q1UShU0+l34D2J/FmO89qBTKyeG1ePc2A1pwS7WYNo0R5V8oKMnHk+LHMODAeUkxizin
         3wMVkV/3Frv/WnBBRFdaQl/vo9txNs8Rw0vM4+R8UwGtu47j+JUcLeup64JFwNaLSlOt
         CR1Q==
X-Gm-Message-State: AOJu0YxHaDmIHkWExKr83YqSkGhK1yOzN4s8DOK2d3m495PKt+YcvaI8
	Ka9APwgiBrxNQAR/7tnJIeReAukCbPicBJJG47dFBFdA7AJYhiRVnT21gtEzzHsG1BB/KHhErw0
	4
X-Google-Smtp-Source: AGHT+IFVxC1HURX8b6Oyh/DOXByDQtdMY0M/niJq82QLF8VQbD9LMQbnN9Wp+hyo5CyRtZyvTBgaYA==
X-Received: by 2002:a05:600c:4f55:b0:40f:e60f:438e with SMTP id m21-20020a05600c4f5500b0040fe60f438emr4281209wmq.8.1707320078204;
        Wed, 07 Feb 2024 07:34:38 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVZ+qDkw8kp5fzF1nAmb9isBPfRk8QVpvEfPGT4TvTbMB6PBD5sKwRfskL73dx5q9TmRydqvOCgJtQXZK3S91xzIQ4/HirkO1gmpB/PINcLM5Zq8hwjPe6gqZsnH+D1YgaGRLupYC2Say6I+ZOob6R6qPKs12aoqKXVmEc2PLiOAK4jdPZc9LxfvKfFxg==
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>,
	Paul Durrant <paul@xen.org>,
	Kevin Tian <kevin.tian@intel.com>
Subject: [PATCH v2 0/2] iommu/x86: unify RMRR/IVMD range checks
Date: Wed,  7 Feb 2024 16:34:15 +0100
Message-ID: <20240207153417.89975-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

The following patches unify the check for the RMRR/IVMD ranges in a
common function, using the IVMD as the basis.

Thanks, Roger.

Roger Pau Monne (2):
  iommu/x86: introduce a generic IVMD/RMRR range validity helper
  iommu/vt-d: switch to common RMRR checker

 xen/arch/x86/include/asm/iommu.h         |  3 ++
 xen/drivers/passthrough/amd/iommu_acpi.c | 38 ++------------------
 xen/drivers/passthrough/vtd/dmar.c       | 14 ++------
 xen/drivers/passthrough/x86/iommu.c      | 46 ++++++++++++++++++++++++
 4 files changed, 55 insertions(+), 46 deletions(-)

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 15:44:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 15:44:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677791.1054632 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXk68-000739-Rg; Wed, 07 Feb 2024 15:44:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677791.1054632; Wed, 07 Feb 2024 15:44:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXk68-000732-P4; Wed, 07 Feb 2024 15:44:28 +0000
Received: by outflank-mailman (input) for mailman id 677791;
 Wed, 07 Feb 2024 15:44:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9378=JQ=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rXk67-00072w-8u
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 15:44:27 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c5a683f8-c5cf-11ee-8a49-1f161083a0e0;
 Wed, 07 Feb 2024 16:44:26 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-55fcceb5f34so812935a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 07:44:26 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 rn3-20020a170906d92300b00a3829e6015fsm879229ejb.55.2024.02.07.07.44.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 07:44:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5a683f8-c5cf-11ee-8a49-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707320665; x=1707925465; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Z0SbiVzdSKRqvmXr/5qOJD7ln6HcULYsjafGA9w2GiU=;
        b=XFEozJ++suDjF3Xz5BDHVFVi+O4LnyFlNV2n3MGEbUf4ZunC4VVu4rooIsWdxdk/nD
         sNJwXOIo1qzePOyODklDUxBBb2BsQ0ABq3Z5uf7F7VxgKnkavdrXCdDhrXH8Oftx/RfZ
         1fzJdeH3pB0Yg1NvM5TpOTaGPKoX063t+6RSU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707320665; x=1707925465;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Z0SbiVzdSKRqvmXr/5qOJD7ln6HcULYsjafGA9w2GiU=;
        b=dzYS5sG+1pJ9p7Aour0z9lZin7xvEPtMufZqkg5WiBKDZ6eKtFrlIW4bAaQ1C7XoGD
         cd4J4olLddJ0NdIAfK31plDNcpi3ZIsA3eqnC4IYGId/0Y5Q/IBeq5gubB1LHUy0gyK4
         pdwjiHIPKplniAiIcNOAFEEz9s+zIUVmtbBLfIyrMlgxk7O3ZwX1e+qLvQvHho86W+DN
         PE1/xpv7jr0dJa/fKMUK+ZGIJLc3HrpaHOSa55pALUxt32rTLdnCmgjQV9jBLqCESy7p
         CIwJ3ICUb94qaj8kosPah2HOSvtnSumn+fjBQi+GDZVYYoAU6sRSGfFAtPAgKORFISus
         lsIw==
X-Gm-Message-State: AOJu0YzJ4QirLcghgSILWqII99B0c8A3crj7L0eXXJcjCG+XjARVT6CN
	wiFSoKocyGpPj1DtoUlaB5sYVQwTFTI5EkYfKMu8C2VfathdnHzwUysx9kJWKHc=
X-Google-Smtp-Source: AGHT+IE1dUnzCIYzDNduPTL/tRNq9CGTMsHr+tgi9BU1MUh+eQHb47nQ24eX3eLZLZwuGjqXrXqVtA==
X-Received: by 2002:a17:906:ee1:b0:a37:3881:77e with SMTP id x1-20020a1709060ee100b00a373881077emr4244826eji.45.1707320665623;
        Wed, 07 Feb 2024 07:44:25 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUeeI7d3NlyXuANJ6W7RN37SHBDWakgO0WuOei3jnX36TkD8FOIdxtpEIW03N6vXUKqsO2A6KtshOTq
Date: Wed, 7 Feb 2024 15:44:24 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: zithro <slack@rabbit.lu>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] tools/xentop: add option to display dom0 first
Message-ID: <5f6c6a52-9ef1-4db2-9e5e-2ca916bdf225@perard>
References: <87846acd5b31991e38561c9765eb97730c79d0f3.1706723494.git.slack@rabbit.lu>
 <a8194dec-70f0-4f90-898d-f15152a5018b@perard>
 <1f2fab6d-6eaf-4e32-a124-cf1042c31312@rabbit.lu>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <1f2fab6d-6eaf-4e32-a124-cf1042c31312@rabbit.lu>

On Tue, Feb 06, 2024 at 03:38:05PM +0100, zithro wrote:
> On 05 Feb 2024 18:27, Anthony PERARD wrote:
> > On Wed, Jan 31, 2024 at 06:51:34PM +0100, Cyril Rbert wrote:
> > > Add a command line option to xentop to be able to display dom0 first, on top of the list.
> > > This is unconditional, so sorting domains with the S option will also ignore dom0.
> > > 
> > > Signed-off-by: Cyril Rbert (zithro) <slack@rabbit.lu>
> > Your patch looks like a good idea
> 
> Thanks, I also have a "display dom id column" in the pipes, almost ready to
> send, but I found kind of a bug doing this (field_id/fields offsets off by
> one).
> Are there "most wanted functionnalities" concerning xentop, while I'm at it
> ? There's a TODO in xentop's folder.

That TODO file is 18 year old, and never been touch since. I don't know
how relevant it is. As for wanted features, I'm not aware of such list.

> Things I'd like to add, if possible, is domain management (pause/unpause,
> maybe shutdown/destroy), columns hiding and domains hiding.

I'm slightly worried about adding domain management, what if someone hit
the wrong key and kill a domain when they just wanted to do something
else, but I guess we can make domain management work more or less
safely. In any case, any feature is welcome.

> > but xentop segv without '-z' now, when there are guest running.
> 
> I failed at the strict minimum here, should have tested ... Sorry for
> wasting your time !

No worries, and your patch was reviewed so you didn't failed to the
strict minimum ;-).

> Bug spotted ("sort_start" incorrectly initialized), will post a v2.
> 
> > Revelant part of a backtrace:
> > #0  xenstat_domain_name (domain=0x121) at xenstat.c:344
> > 344		return domain->name;
> > #6  0x00006344dd283651 in top () at xentop.c:1209
> >          i = 2
> >          num_domains = 2
> >          sort_start = 1
> >          sort_count = <optimized out>
> >          dom0_index = <optimized out>
> > 1209		qsort((domains+sort_start), (num_domains-sort_count), sizeof(xenstat_domain *),
> > 1210		      (int(*)(const void *, const void *))compare_domains);
> 
> What soft did you use to get that output ? A debugger ?
> (It's a real question, I'm a noob).

I've used `gdb`, the GNU Debugger. There's plenty of tutorial on the
internet on the few commands that are useful to get started. And you can
run your program with gdb or run it on a coredump.

What I pasted, is a mixture of running the command `bt full`, and `list`
and `up`/`down` to get to the function I wanted to print a bit of the
source code.

Cheers,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 15:58:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 15:58:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677797.1054642 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkJc-00015S-1w; Wed, 07 Feb 2024 15:58:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677797.1054642; Wed, 07 Feb 2024 15:58:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkJb-00015L-Vb; Wed, 07 Feb 2024 15:58:23 +0000
Received: by outflank-mailman (input) for mailman id 677797;
 Wed, 07 Feb 2024 15:58:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Py/x=JQ=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rXkJb-00015D-9t
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 15:58:23 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b74df7d8-c5d1-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 16:58:21 +0100 (CET)
Received: from [192.168.1.15] (host-87-16-84-78.retail.telecomitalia.it
 [87.16.84.78])
 by support.bugseng.com (Postfix) with ESMTPSA id 44D0D4EE0738;
 Wed,  7 Feb 2024 16:58:20 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b74df7d8-c5d1-11ee-98f5-efadbce2ee36
Message-ID: <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
Date: Wed, 7 Feb 2024 16:58:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US, it
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 07/02/24 16:24, Jan Beulich wrote:
> On 07.02.2024 16:08, Federico Serafini wrote:
>> On 07/02/24 15:16, Jan Beulich wrote:
>>> On 07.02.2024 14:51, Federico Serafini wrote:
>>>> On 07/02/24 08:38, Jan Beulich wrote:
>>>>> On 07.02.2024 02:08, Stefano Stabellini wrote:
>>>>>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>>>>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>>>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>>>>>         case 8:                                                                \
>>>>>>>>             put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>>>>>             break;                                                             \
>>>>>>>> -    default: __put_user_bad();                                             \
>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>         }                                                                      \
>>>>>>>>         clac();                                                                \
>>>>>>>>     } while ( false )
>>>>>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>>>>>         case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>>>>>         case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>>>>>         case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>>>>>> -    default: __get_user_bad();                                             \
>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>         }                                                                      \
>>>>>>>>         clac();                                                                \
>>>>>>>>     } while ( false )
>>>>>>>
>>>>>>> Related to my remark on patch 1 - how is one to know the macro this was
>>>>>>> invoked from, when seeing the resulting diagnostic?
>>>>>>
>>>>>> I am not sure what do you mean here... we do get an error like the
>>>>>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>>>>>
>>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>>
>>>>> Right - and how do I know what _user_ of the macro actually triggered
>>>>> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
>>>>> __FUNCTION__ here, for that specific purpose ...
>>>>
>>>> To test the macro and its diagnostics,
>>>> I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
>>>> on the x86 code with STATIC_ASSERT_UNREACHABLE(),
>>>> that is in file arch/x86/alternative.c, line 312,
>>>> function _apply_alternatives().
>>>>
>>>> What I got is the following build error:
>>>>
>>>> ...
>>>> arch/x86/alternative.c: Assembler messages:
>>>> arch/x86/alternative.c:312: Error: static assertion failed: unreachable
>>>>      CC      arch/x86/copy_page.o
>>>> make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1
>>>
>>> But that's not what my request was about. Here sufficient context is
>>> given, even if it would be nice if the function was also visible right
>>> away. But that's not the same as the case above, where the new macro
>>> is used inside another macro.
>>
>> An example of that is the get_unsafe_size() macro,
>> whose body uses STATIC_ASSERT_UNREACHABLE().
>> A wrong use of get_unsafe_size() at line n
>> leads to a build error pointing to the line n,
>> isn't this the desired behavior?
> 
> Aiui this would point to the line in the header file, when what you need
> to spot the bad use of the macro is the line in the source file actually
> using the macro. Quoting from an earlier mail of yours:
> 
> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable

It points to the header file uaccess.h because at line 262 there is
an intentional wrong use of put_guest_size(), within the body of
__copy_to_guest_pv() function.
This example can be misleading because {get,put}_unsafe_size() are
defined in the same file but the diagnostics is doing the
right thing.

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 16:00:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 16:00:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677800.1054652 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkLi-0003MT-D7; Wed, 07 Feb 2024 16:00:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677800.1054652; Wed, 07 Feb 2024 16:00:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkLi-0003MM-Ae; Wed, 07 Feb 2024 16:00:34 +0000
Received: by outflank-mailman (input) for mailman id 677800;
 Wed, 07 Feb 2024 16:00:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9378=JQ=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rXkLh-0003MF-Br
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 16:00:33 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 05ae14b9-c5d2-11ee-8a4a-1f161083a0e0;
 Wed, 07 Feb 2024 17:00:32 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-51165a488baso1029605e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 08:00:32 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ha5-20020a170906a88500b00a3667f53691sm906802ejb.86.2024.02.07.08.00.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 08:00:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05ae14b9-c5d2-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707321632; x=1707926432; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=sfJS0fcEO4vKP037vedQWZnTsRIX9mqN55AFC/lUpOc=;
        b=RERqJzwMaDKr0JdT+gRPtg2N9EKBc2CcZDmz06H4M37dCKToGI8eFNS7qKOMprCmOR
         O3LO74/EEE327loWUSKy2/veaVHsQrrOxONJc2dOsUghpCV6DSvy+phbImURJ8RzTM+A
         fDoD+ZPD6SRKakar69PrLC2IgLh54lUAqgypU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707321632; x=1707926432;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sfJS0fcEO4vKP037vedQWZnTsRIX9mqN55AFC/lUpOc=;
        b=qJbGdt1P0gPq+gd+E7LBkhMJqdu8XqIGD07hgX3SN88l/ut7nhsxchwvZT94Zi4XeB
         JjRYeX1NdA2iGV3VrdoouuhWbje6Ri0g22V7xcS1EP0hCiRfqC/06pA4sVvP6bmTycnU
         YCgQ8XiMdDnzxXhFv32tWaVC31u974GIq1gL3uxcKasVFrxCj97/3lglL+yTb8Lp9JZP
         oPRoVOmFLUmj1P2xtP0Cm+pHAOuGdfAqjmo2T6Kp02Gsi9ZSOS08zFFSQqrs6CRc8ikR
         MARXjJq3cRxV+cD8Ie8B36dEm58vKZVhhR7M4oqW7NRUxukcgtLtR1L6mQLS67mm7MZG
         pVrw==
X-Gm-Message-State: AOJu0YzOa1XRiNSWqIB1NdxkptMPx41lMTgFoyk7+Bom1hjs4eC1P0wO
	c4uqlIRYgTC39qsmFYbbGqlHmgaYaLN1JItt5UF3VwtRZnrTZhjg7kfvuo5fBFuZ71D3YeEHqoh
	6
X-Google-Smtp-Source: AGHT+IFwxmNLVP3TGVhNIbNLaXosaEJJ1aWgjEleNZBA7mvzW3rAIVL5Oi0P0dRhyewNjObOjDTS0A==
X-Received: by 2002:a19:c519:0:b0:511:6a0b:1035 with SMTP id w25-20020a19c519000000b005116a0b1035mr491315lfe.17.1707321631950;
        Wed, 07 Feb 2024 08:00:31 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXrvtUOA0CCda9FTfFUbiNHcclN0TfHmIRdz8SUT6kSrBRnz8jLMWYbkZm11s7ganzsMEi3drndyafd
Date: Wed, 7 Feb 2024 16:00:31 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: zithro / Cyril =?iso-8859-1?Q?R=E9bert?= <slack@rabbit.lu>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v2] tools/xentop: add option to display dom0 first
Message-ID: <90e6b5d1-8f0e-4596-8f55-4b06d549d58c@perard>
References: <efdca3e0c02576c624403d273c7168c9303fcc52.1707232545.git.slack@rabbit.lu>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <efdca3e0c02576c624403d273c7168c9303fcc52.1707232545.git.slack@rabbit.lu>

On Tue, Feb 06, 2024 at 04:15:45PM +0100, zithro / Cyril Rbert wrote:
> Add a command line option to xentop to be able to display dom0 first, on top of the list.
> This is unconditional, so sorting domains with the S option will also ignore dom0.
> 
> Signed-off-by: Cyril Rbert (zithro) <slack@rabbit.lu>
> 
> ---
> Changes in v2:
> - bug fix
> - add documentation

Looks like this v2 contains only changes from v1. Could you instead
squash both version and send it as v3? After that, it looks like the
patch would be ready to be committed (changes looks fine if I look at
both version).

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 16:11:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 16:11:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677807.1054663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkWM-0005L1-C1; Wed, 07 Feb 2024 16:11:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677807.1054663; Wed, 07 Feb 2024 16:11:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkWM-0005Ku-9L; Wed, 07 Feb 2024 16:11:34 +0000
Received: by outflank-mailman (input) for mailman id 677807;
 Wed, 07 Feb 2024 16:11:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXkWL-0005Ko-Fx
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 16:11:33 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8eedaa58-c5d3-11ee-8a4a-1f161083a0e0;
 Wed, 07 Feb 2024 17:11:32 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-41028900b83so961555e9.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 08:11:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bh9-20020a05600c3d0900b0041027934b19sm313220wmb.7.2024.02.07.08.11.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 08:11:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8eedaa58-c5d3-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707322292; x=1707927092; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nKE+UH5CLXyTzi6UmjufGHKXNy858Aqo/bh0dbKyexo=;
        b=aWoiiDcBCEWyYBwKWHewpOYaYF5C/GEfGY2IDi0iUlA/qQtHjEKSxkzyI5J6b471U9
         dFkvGx/CAMbueirL0PN377mGeLP++gxs8bnRiVwZ8aKJm+qCoh/yz9M2U5L96cWV4BKp
         QIgXOn7VeJhygeujG2M2Jete39I7gzadrxJ7LoSVFA4pjlHRpRBQZU9v8D9wDA+fQbZi
         oHfbpuKJ7ldUEBYvGeCRber1iK9RHOD/p5/qSrEbDw7uzbEBJ1Y+MdqD25nmmcuQRfEl
         he4qV7rR0tIFjSBkiirZfKXudLcq+jKjg/nUlxeIpwocfS2IbtVYDIn5F75Wvj5any8N
         KlxA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707322292; x=1707927092;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nKE+UH5CLXyTzi6UmjufGHKXNy858Aqo/bh0dbKyexo=;
        b=vuyXe3t/mzeMVblYeOp2MJzkErYbkr5mjMz2R2vJxo02HBieCHwPyVZrFmmPrLkh3L
         I5zxAvjldqZqYG1PMpCoRYFdodAkTpgaLDzcRCv8Wkhh1yy1Ix3dTkcxOoplMGuUUmAJ
         hDpsJUHX/hc+Ry4I+qm/Z9Q7c+ET7JW/auACPVL+GJd0us+XpxACyAzBASs00hc/WFDJ
         9bjQN+okBWc+RmCLJkwqn2inO9y07ZobhBBRnceJd6EbuvNVRZi1nIQdrTyEyOxXfQMj
         8fnv3eOgMJ+Ced11eVXYUZ/zHyBhXDCMn0n8/JJ3RBM+GBY0zbekwA+BmTxOjCcay6F2
         wPwA==
X-Gm-Message-State: AOJu0YycTDFKTF5+aD15QzxxYzdoWQPLiLQhfthaqbJP4NTfa+9zdIMU
	ZaCWVT+ZTtzYcwZ6ga+velcr1QYhXzZo88y8JrJW3sqRQ3zwXWI76pu8/wYXWA==
X-Google-Smtp-Source: AGHT+IH4EoKc/UEYyEeLUGG8Omg1WDat1nGttqddfySYnVLAOZeKI6Yv45e/ncIOeiOq7FHs8Tz1Ug==
X-Received: by 2002:a05:600c:1d99:b0:410:2058:c478 with SMTP id p25-20020a05600c1d9900b004102058c478mr762793wms.5.1707322291741;
        Wed, 07 Feb 2024 08:11:31 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXlEijOBTKL7mYoLAfPz68fujlStIzstAPPmpW512v9iylUy0Rn7g8FXA/P8oLkcSXaCeHs4fO9ZKUXiEkmI1S6km6UcQQ1IOdYhsAaP/+5l/q7HBjY7CFkFjNyo62HcukLwoq4R1DERSOT+nISlZKK
Message-ID: <d0e03f9a-83b2-4809-9b76-5612f28f2464@suse.com>
Date: Wed, 7 Feb 2024 17:11:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 1/9] x86/boot: choose AP stack based on APIC ID
Content-Language: en-US
To: Krystian Hebel <krystian.hebel@3mdeb.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <0e7dd957b6f26fa7b752bdce1ef6ebe97c825903.1699982111.git.krystian.hebel@3mdeb.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <0e7dd957b6f26fa7b752bdce1ef6ebe97c825903.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.11.2023 18:49, Krystian Hebel wrote:
> --- a/xen/arch/x86/boot/trampoline.S
> +++ b/xen/arch/x86/boot/trampoline.S
> @@ -72,6 +72,26 @@ trampoline_protmode_entry:
>          mov     $X86_CR4_PAE,%ecx
>          mov     %ecx,%cr4
>  
> +        /*
> +         * Get APIC ID while we're in non-paged mode. Start by checking if
> +         * x2APIC is enabled.
> +         */
> +        mov     $MSR_APIC_BASE, %ecx
> +        rdmsr
> +        and     $APIC_BASE_EXTD, %eax

You don't use the result, in which case "test" is to be preferred over
"and".

> +        jnz     .Lx2apic
> +
> +        /* Not x2APIC, read from MMIO */
> +        mov     0xfee00020, %esp

Please don't open-code existing constants (APIC_ID here and below,
APIC_DEFAULT_PHYS_BASE just here, and ...

> +        shr     $24, %esp

... a to-be-introduced constant here (for {G,S}ET_xAPIC_ID() to use as
well then). This is the only way of being able to easily identify all
pieces of code accessing the same piece of hardware.

> +        jmp     1f
> +
> +.Lx2apic:
> +        mov     $(MSR_X2APIC_FIRST + (0x20 >> 4)), %ecx
> +        rdmsr
> +        mov     %eax, %esp
> +1:

Overall I'm worried of the use of %esp throughout here.

> --- a/xen/arch/x86/boot/x86_64.S
> +++ b/xen/arch/x86/boot/x86_64.S
> @@ -15,7 +15,33 @@ ENTRY(__high_start)
>          mov     $XEN_MINIMAL_CR4,%rcx
>          mov     %rcx,%cr4
>  
> -        mov     stack_start(%rip),%rsp
> +        test    %ebx,%ebx

Nit (style): Elsewhere you have blanks after the commas, just here
(and once again near the end of the hunk) you don't.

> +        cmovz   stack_start(%rip), %rsp
> +        jz      .L_stack_set
> +
> +        /* APs only: get stack base from APIC ID saved in %esp. */
> +        mov     $-1, %rax

Why a 64-bit insn here and ...

> +        lea     x86_cpu_to_apicid(%rip), %rcx
> +1:
> +        add     $1, %rax

... here, when you only use (far less than) 32-bit values?

> +        cmp     $NR_CPUS, %eax
> +        jb      2f
> +        hlt
> +2:
> +        cmp     %esp, (%rcx, %rax, 4)
> +        jne     1b

Aren't you open-coding REPNE SCASD here?

> +        /* %eax is now Xen CPU index. */
> +        lea     stack_base(%rip), %rcx
> +        mov     (%rcx, %rax, 8), %rsp
> +
> +        test    %rsp,%rsp
> +        jnz     1f
> +        hlt
> +1:
> +        add     $(STACK_SIZE - CPUINFO_sizeof), %rsp

Even this post-adjustment is worrying me. Imo the stack pointer is
better set exactly once, to its final value. Keeping it at its init
value of 0 until then yields more predictable results in case it
ends up being used somewhere.

> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1951,6 +1951,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
>       */
>      if ( !pv_shim )
>      {
> +        /* Separate loop to make parallel AP bringup possible. */
>          for_each_present_cpu ( i )
>          {
>              /* Set up cpu_to_node[]. */
> @@ -1958,6 +1959,12 @@ void __init noreturn __start_xen(unsigned long mbi_p)
>              /* Set up node_to_cpumask based on cpu_to_node[]. */
>              numa_add_cpu(i);
>  
> +            if ( stack_base[i] == NULL )
> +                stack_base[i] = cpu_alloc_stack(i);
> +        }

Imo this wants accompanying by removal of the allocation in
cpu_smpboot_alloc(). Which would then make more visible that there's
error checking there, but not here (I realize there effectively is
error checking in assembly code, but that'll end in HLT with no
useful indication of what the problem is). Provided, as Julien has
pointed out, that the change is necessary in the first place.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 16:19:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 16:19:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677812.1054672 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkdz-000726-4S; Wed, 07 Feb 2024 16:19:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677812.1054672; Wed, 07 Feb 2024 16:19:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkdz-00071z-1e; Wed, 07 Feb 2024 16:19:27 +0000
Received: by outflank-mailman (input) for mailman id 677812;
 Wed, 07 Feb 2024 16:19:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXkdx-00070h-UM
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 16:19:25 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a7cd78be-c5d4-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 17:19:23 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-4101fc00832so3432625e9.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 08:19:23 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 j11-20020a05600c190b00b0040fe5994d0csm2595513wmq.0.2024.02.07.08.19.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 08:19:22 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a7cd78be-c5d4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707322763; x=1707927563; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=GaVJzzhvZXekZjvJeV5RvXknDdmJRT+EvgPzHIwBJ44=;
        b=H/AqZu9aCDj1DtDFGvxJb7UY6z8r8p9gNmkgEQ7vvrmJE7YE7QYAxJFYJDTk4p10hM
         OjMoa+I8hnagX6MhueR4OZzm2MGaG0fTyHN+lh8+jd4PRNqgNe+MH7TIepykSEW29Pxe
         b+5NmQj5A9qQJMKtBdn2AlUkYGOfP0yy7lnoVCVPv56HqmfDBiQyLlcN28DMLIqBqwF5
         2OLM8zn0Nw2Fkh7iS4/7Vvr3mvUSDAvDKsezO/PJpXAxBBCf8GRcZWGJGKIRgdhXBLiz
         b7qePe227ZQ+wr6u+HhRbYyzhceJiGnEadYWpXfd+CuJlxsVoqPkk0UUYXIePAMUpVRN
         OsGA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707322763; x=1707927563;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GaVJzzhvZXekZjvJeV5RvXknDdmJRT+EvgPzHIwBJ44=;
        b=URaSnCzipdhtXz6j+ijPvHMZtPdlrpLJWDmgG24hKtrCOhwGwSkvATvyoLiyYtkWh9
         xw4k96MnDc3S/3DRojBdDzVUx5avX31OZbreu1OfaqLWoMV/Y2JzQ6C+miGuYXnG9DLC
         86wkhspYqrIHUq8DvKMKJJo29PjPyGH77HNXadQ4YVB6mmAUzwRCFKL/irtcxzTXSOxp
         e7Ta1wd1k1F1yDMv9ACHAYeZRGgbeh0PRIw+cC8aE/oWDacM5vVKYz3K76waF0CHIIAM
         N5UuS4l6bxa05H4LMqBLz1+9ms53suv4uP87XNTcZr1GO5U8mmzaTxgEihkUpgILk9Ct
         di+Q==
X-Forwarded-Encrypted: i=1; AJvYcCUVyP9LhhDLgVQAbJ1k3lg1vfLvZbrz2QF9McYvrfEa9Y8v8t3lgCVQRUq0mUeXVL5uGbWCZ2pOjQAWQdK2YqvMBhoLY+M8v+cUK0gJ3yg=
X-Gm-Message-State: AOJu0YxSo5/ylF8FgGAL7cCct4/jESBrbJPUDLHVKL25efZMzks3SsON
	QVbNDFIbe+36ZSM39LH/rzHkXnrPd6lM2bfzJFPmIb5fKReOn1/5Jqfl54GYLQ==
X-Google-Smtp-Source: AGHT+IGv1frmZcd3w58BYSohr9vit+R8YdQ2Qb3TLfFUstDpSE/rDDd1OFxw89NdM/p3rErVUUk+uw==
X-Received: by 2002:a05:600c:3d93:b0:410:219f:121b with SMTP id bi19-20020a05600c3d9300b00410219f121bmr580614wmb.21.1707322762979;
        Wed, 07 Feb 2024 08:19:22 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUN0IamYoxtLq0CVWm8GQROqCSHQnrsGUx2grnA3KhAgdS9L0Fe03D16mneX1VZAyjoulwKsYJpXA9wdV8sP15VGJJ6faCnp2VvOfBVJ/M8E/BAFQXkGj9tWIRBykppUWp6DzmYOnm7C8IvbcBmxLRaKE7E7cPvpuUjeOdQ3Dwg7UIIm9UziOGBKChXVrlHVjzE7CW8IZcXeNKqtst6SIXUrdrd
Message-ID: <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
Date: Wed, 7 Feb 2024 17:19:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 16:58, Federico Serafini wrote:
> On 07/02/24 16:24, Jan Beulich wrote:
>> On 07.02.2024 16:08, Federico Serafini wrote:
>>> On 07/02/24 15:16, Jan Beulich wrote:
>>>> On 07.02.2024 14:51, Federico Serafini wrote:
>>>>> On 07/02/24 08:38, Jan Beulich wrote:
>>>>>> On 07.02.2024 02:08, Stefano Stabellini wrote:
>>>>>>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>>>>>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>>>>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>>>>>>         case 8:                                                                \
>>>>>>>>>             put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>>>>>>             break;                                                             \
>>>>>>>>> -    default: __put_user_bad();                                             \
>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>         }                                                                      \
>>>>>>>>>         clac();                                                                \
>>>>>>>>>     } while ( false )
>>>>>>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>>>>>>         case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>>>>>>         case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>>>>>>         case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>>>>>>> -    default: __get_user_bad();                                             \
>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>         }                                                                      \
>>>>>>>>>         clac();                                                                \
>>>>>>>>>     } while ( false )
>>>>>>>>
>>>>>>>> Related to my remark on patch 1 - how is one to know the macro this was
>>>>>>>> invoked from, when seeing the resulting diagnostic?
>>>>>>>
>>>>>>> I am not sure what do you mean here... we do get an error like the
>>>>>>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>>>>>>
>>>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>>>
>>>>>> Right - and how do I know what _user_ of the macro actually triggered
>>>>>> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
>>>>>> __FUNCTION__ here, for that specific purpose ...
>>>>>
>>>>> To test the macro and its diagnostics,
>>>>> I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
>>>>> on the x86 code with STATIC_ASSERT_UNREACHABLE(),
>>>>> that is in file arch/x86/alternative.c, line 312,
>>>>> function _apply_alternatives().
>>>>>
>>>>> What I got is the following build error:
>>>>>
>>>>> ...
>>>>> arch/x86/alternative.c: Assembler messages:
>>>>> arch/x86/alternative.c:312: Error: static assertion failed: unreachable
>>>>>      CC      arch/x86/copy_page.o
>>>>> make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1
>>>>
>>>> But that's not what my request was about. Here sufficient context is
>>>> given, even if it would be nice if the function was also visible right
>>>> away. But that's not the same as the case above, where the new macro
>>>> is used inside another macro.
>>>
>>> An example of that is the get_unsafe_size() macro,
>>> whose body uses STATIC_ASSERT_UNREACHABLE().
>>> A wrong use of get_unsafe_size() at line n
>>> leads to a build error pointing to the line n,
>>> isn't this the desired behavior?
>>
>> Aiui this would point to the line in the header file, when what you need
>> to spot the bad use of the macro is the line in the source file actually
>> using the macro. Quoting from an earlier mail of yours:
>>
>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
> 
> It points to the header file uaccess.h because at line 262 there is
> an intentional wrong use of put_guest_size(), within the body of
> __copy_to_guest_pv() function.

Yet that's again only a helper function being inlined into the ultimate
caller. That ultimate caller is what wants identifying in the diag. Not
the least because of ...

> This example can be misleading because {get,put}_unsafe_size() are
> defined in the same file but the diagnostics is doing the
> right thing.

... this. And really __copy_to_guest_pv() is the wrong place to put a
wrong put_guest_size() in, to try out how diagnostics would look like
in reality: That function falls back to copy_to_guest_ll() for all
cases it can't handle directly. You want to place a bogus put_guest()
somewhere in a .c file to see what results.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 16:28:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 16:28:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677817.1054682 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkmM-0000H4-2m; Wed, 07 Feb 2024 16:28:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677817.1054682; Wed, 07 Feb 2024 16:28:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkmL-0000Gx-Vw; Wed, 07 Feb 2024 16:28:05 +0000
Received: by outflank-mailman (input) for mailman id 677817;
 Wed, 07 Feb 2024 16:28:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXkmK-0000Gr-1c
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 16:28:04 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dd0a295c-c5d5-11ee-8a4a-1f161083a0e0;
 Wed, 07 Feb 2024 17:28:02 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-40fd2f7ef55so11557945e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 08:28:02 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 t17-20020a05600c199100b0040ef2e7041esm2634750wmq.6.2024.02.07.08.28.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 08:28:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dd0a295c-c5d5-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707323282; x=1707928082; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=q6TdYqu32u1KRS7tNYDPZhANdGBEMh0MB5q5fR4i4VA=;
        b=LwovsdSpKhZyJz7roY2hcRKwSXtjOYXjr/GCsRk968Ifx+qrzzPd4gIOEx0eObP4S9
         KdJgXLFRXh0yAW+SqPmSlCIhEl7gd56y39Yb9rnYUWUjReE2CnU8DaGd2SrgJo5sevd5
         srVHLsf6SPeimvTB4M5lMqEgULLwlC7nC0nQndLQ0Fl6pK9Cn7QrjG4Mm19BFoIEHzsA
         mTGpO22bSBOofmlWo4nBcZmDKGRwgfjpyoIxm1Im8G6+AwzVyYZa6flP31PY6MltRquX
         iGmU0VeR/kRwPMevoi9mPuYOiRw0n2tXpyLsDNb1OcQmHnMF6Wp6S6yG+W4GSTb5uEMz
         VWpg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707323282; x=1707928082;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=q6TdYqu32u1KRS7tNYDPZhANdGBEMh0MB5q5fR4i4VA=;
        b=wIdIKSnH2f1ux5klNvda3JzzsRTLY3fVExe/UJ1MVwWHCooxJavuTVg+CXq2jCxtR5
         nzyCzQ4xW2V1Am8fQscBrnPWykeo1afwIBlRBJLfq5/S1EAI3SU8F0hakVlazBVhmyqC
         G9us0tmi0JmnLe25lAFjAx+BJHvF3xEJjehqevZSsnITikRsAprDvYxpG5VtN7RDO4gg
         9cYY6Oiziw2FfuVJym4oIbwBTfRGc8shRKlzuUX0nrO9gaRsHx9x/OI7QTZ7rf/pAup+
         Qa/DZnvIJzRudO+x3vC9F/ABtb6xItaSD+Uz3bup4XiS9NWbU4xrHl3re0sjnARbsU5M
         34Mg==
X-Gm-Message-State: AOJu0Yy/IjZhdYozKJQjlDNqZ67Ua6r/nssC6TA7981VGGBefy8Iebrq
	KBtUHCcN4IcFC+dDRlWTWa3Yel+5hpQzdEY3Flg0C5SfwR9AgAwywTmWUcp2JQ==
X-Google-Smtp-Source: AGHT+IHOk3skZX8eYWutNmYm546LyurSMygvkjb1/qeQ0GpD8Y0wEKJ1tvd3lvmk4SQDG+Dxxmw8iQ==
X-Received: by 2002:a05:600c:4f96:b0:40e:fb74:c514 with SMTP id n22-20020a05600c4f9600b0040efb74c514mr5038432wmq.7.1707323281834;
        Wed, 07 Feb 2024 08:28:01 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWhaCn/fgoo3THGyDymbFMQiNxIidILnLEP3ZjuTExNdTrcV7OG6ao1iHb3v9O5TVcVrGrtQClZrLI+s63Vv8ptYUehlkIjyYc9CpjX1Rdbr3GsWugFFN9MFgO/FslDcI0V6rCOjYn77CRwuTeKfx4P
Message-ID: <904f358b-c535-4de2-b31d-e593687af11b@suse.com>
Date: Wed, 7 Feb 2024 17:28:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 2/9] x86: don't access x86_cpu_to_apicid[] directly,
 use cpu_physical_id(cpu)
Content-Language: en-US
To: Krystian Hebel <krystian.hebel@3mdeb.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <705574ddb7f18bae9ed3f60ddf2e4bda02c70388.1699982111.git.krystian.hebel@3mdeb.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <705574ddb7f18bae9ed3f60ddf2e4bda02c70388.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.11.2023 18:49, Krystian Hebel wrote:
> --- a/xen/arch/x86/apic.c
> +++ b/xen/arch/x86/apic.c
> @@ -950,7 +950,7 @@ __next:
>       */
>      if (boot_cpu_physical_apicid == -1U)
>          boot_cpu_physical_apicid = get_apic_id();
> -    x86_cpu_to_apicid[0] = get_apic_id();
> +    cpu_physical_id(0) = get_apic_id();

While personally I don't mind as much, I expect Andrew would not like
this: Something that looks like a function call on the lhs is against
what normal language structure would be.

> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -1615,7 +1615,7 @@ long do_vcpu_op(int cmd, unsigned int vcpuid, XEN_GUEST_HANDLE_PARAM(void) arg)
>              break;
>  
>          cpu_id.phys_id =
> -            (uint64_t)x86_cpu_to_apicid[v->vcpu_id] |
> +            (uint64_t)cpu_physical_id(v->vcpu_id) |
>              ((uint64_t)acpi_get_processor_id(v->vcpu_id) << 32);

While the cast on the 2nd line is necessary, the one on the 2st isn't
and would be nice to be dropped while touching the line anyway.

> --- a/xen/arch/x86/numa.c
> +++ b/xen/arch/x86/numa.c
> @@ -70,7 +70,7 @@ void __init init_cpu_to_node(void)
>  
>      for ( i = 0; i < nr_cpu_ids; i++ )
>      {
> -        u32 apicid = x86_cpu_to_apicid[i];
> +        u32 apicid = cpu_physical_id(i);
>          if ( apicid == BAD_APICID )
>              continue;
>          node = apicid < MAX_LOCAL_APIC ? apicid_to_node[apicid] : NUMA_NO_NODE;

We're in the process of phasing out u32 and friends, favoring uint32_t.
Would be nice if in code being touched anyway (i.e. not just here) the
conversion would be done right away. Then again fixed-width types are
preferably avoided where not really needed (see ./CODING_STYLE), so
quite likely it actually wants to be unsigned int here.

Furthermore our style demands that declaration(s) and statement(s) are
separated by a blank line. Inserting the missing one in cases like the
one here would be very desirable as well.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 16:41:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 16:41:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677823.1054693 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkyu-0003fK-6v; Wed, 07 Feb 2024 16:41:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677823.1054693; Wed, 07 Feb 2024 16:41:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXkyu-0003fD-4K; Wed, 07 Feb 2024 16:41:04 +0000
Received: by outflank-mailman (input) for mailman id 677823;
 Wed, 07 Feb 2024 16:41:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXkyt-0003f7-Os
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 16:41:03 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ae079667-c5d7-11ee-8a4a-1f161083a0e0;
 Wed, 07 Feb 2024 17:41:02 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-41008ab427fso7734765e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 08:41:02 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 b7-20020a05600010c700b0033b459e8f60sm1853630wrx.18.2024.02.07.08.41.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 08:41:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae079667-c5d7-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707324062; x=1707928862; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4wzTALPGPQx+krgy++JH7ll+8zWoYZ429HdJyi3Km34=;
        b=MkfSlsr3OtIVopV6auKmETRmEs6lj/ArexgEQRGltEfUx9hEb2ql1cU7tC3So6Y/fr
         cKX935yLjqux6PESRqXq7a86dF+rF6xNDVQS5y+5vBZU2bwNOgJcq7SQHcZwEXf7sQSr
         +mPWirpfFUBP3CfszHrEcOLl7yacLkopwsdg4UPcFunVcqYAiqTpkX5IOgi25bvlcHjS
         fKLpNZEfzfPxdi9kaSN1Y4hj0gjRsnc2pGABpwKRKGVQz2Dry8/GiP4Y32hzz0P4z/l1
         GRDBcvSGO/icIA8MKsp++/vrC7SppwxRdt/7DPx//Bo66b4oXK9ot8MzbJYB1OBFdPhJ
         iisA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707324062; x=1707928862;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=4wzTALPGPQx+krgy++JH7ll+8zWoYZ429HdJyi3Km34=;
        b=MJYUGYUFRaHsgQzaUjwq++WcZ1jYF4dgrEobxsdnVrOKuNbr1QwhfDnIro7b4h7FJO
         nhZpdiqJe/d/7FrgIj0tl4obqq8WLHE8gbAlVmYXPgHq/wESj3WesRgUP8u4z3hzCPny
         KnmLAiYJm7g32p0Y7nz0MyimZRMPKW6aKxGwN1V37Ueas983QSAAzizIDq5iYgb7duNO
         GHvkGNs0Y1HonnWyDNHH/9k65lWGcwqqeKuMui2WAoDAYzhiWfR0dgJcVm/jHYmPR/5l
         V/8Tackh2omF2kze32OJ3VwG1vRl8XX0iCCPMrs7VP8s1JF5UYgvwavc/qwkwjmzeVn0
         3oCg==
X-Forwarded-Encrypted: i=1; AJvYcCVutMTMm89mfomheZa4JSZj/dvS7jn1VpfR8zGB7RNouxXTr1ecEQlX3NDdprYkqWKS/eWr4wUfwS9D64JxH6iZvGVOigloQa3yrrTSvzw=
X-Gm-Message-State: AOJu0YxthC4edrrHQRwlUVrZCKUtiIa97/y0EphDp3om/FFbv6MxbTRo
	NxJ2rsywJdQYCGGMnhoW1j+YlIB3mrIX/49q2AI4J9tww1vNdEG9JqMd2Tapkg==
X-Google-Smtp-Source: AGHT+IGgj565s8RiXjL+ftfCBRVIoOOftoII7/Hw5eBGyE8NOE86vIs7J2TV19rM35GkSj2+muGtTw==
X-Received: by 2002:a5d:55c8:0:b0:33b:17c5:dcc8 with SMTP id i8-20020a5d55c8000000b0033b17c5dcc8mr3719679wrw.43.1707324061790;
        Wed, 07 Feb 2024 08:41:01 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVuh540rcao5EbXC3FOFiXKswUT41gOigIelHZ4uMpfaVY7Vy0So+8lrQIN5gKb6gvZ7IuFsTD/VdeHh9MkVfyrj+rP8Bc9BoBzFCOn1oFacawkbfAFy7j/DEP/1Uy/STRNx/XgEqpUAlV2Ex/mKQ+72AJZUv2Dci8cu2GsBG/6k30cxzWKgrY8ZM4=
Message-ID: <24dc1f13-3add-4b5f-9e2d-a18286b691eb@suse.com>
Date: Wed, 7 Feb 2024 17:41:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 3/9] x86/smp: drop x86_cpu_to_apicid, use
 cpu_data[cpu].apicid instead
Content-Language: en-US
To: Julien Grall <julien@xen.org>, Krystian Hebel <krystian.hebel@3mdeb.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <8121d9b472b305be751158aa3af3fed98ff0572e.1699982111.git.krystian.hebel@3mdeb.com>
 <b85ecee7-65ce-47f0-8e9d-cdc056d337fb@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b85ecee7-65ce-47f0-8e9d-cdc056d337fb@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 19:11, Julien Grall wrote:
> Hi,
> 
> On 14/11/2023 17:50, Krystian Hebel wrote:
>> Both fields held the same data.
>>
>> Signed-off-by: Krystian Hebel <krystian.hebel@3mdeb.com>
>> ---
>>   xen/arch/x86/boot/x86_64.S           |  8 +++++---
>>   xen/arch/x86/include/asm/asm_defns.h |  2 +-
>>   xen/arch/x86/include/asm/processor.h |  2 ++
>>   xen/arch/x86/include/asm/smp.h       |  4 ----
>>   xen/arch/x86/numa.c                  | 15 +++++++--------
>>   xen/arch/x86/smpboot.c               |  8 ++++----
>>   xen/arch/x86/x86_64/asm-offsets.c    |  4 +++-
>>   7 files changed, 22 insertions(+), 21 deletions(-)
>>
>> diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
>> index b85b47b5c1a0..195550b5c0ea 100644
>> --- a/xen/arch/x86/boot/x86_64.S
>> +++ b/xen/arch/x86/boot/x86_64.S
>> @@ -20,15 +20,17 @@ ENTRY(__high_start)
>>           jz      .L_stack_set
>>   
>>           /* APs only: get stack base from APIC ID saved in %esp. */
>> -        mov     $-1, %rax
>> -        lea     x86_cpu_to_apicid(%rip), %rcx
>> +        mov     $0, %rax
>> +        lea     cpu_data(%rip), %rcx
>> +        /* cpu_data[0] is BSP, skip it. */
>>   1:
>>           add     $1, %rax
>> +        add     $CPUINFO_X86_sizeof, %rcx
>>           cmp     $NR_CPUS, %eax
>>           jb      2f
>>           hlt
>>   2:
>> -        cmp     %esp, (%rcx, %rax, 4)
>> +        cmp     %esp, CPUINFO_X86_apicid(%rcx)
>>           jne     1b
>>   
>>           /* %eax is now Xen CPU index. */
> 
> As mentioned in an earlier patch, I think you want to re-order the 
> patches. This will avoid to modify twice the same code within the same 
> series (it is best to avoid if you can).

I second this request. Even more so that there's an unexplained move
from starting at $-1 to starting at $0 (in which case you really want
to use xor, not mov).

>> --- a/xen/arch/x86/numa.c
>> +++ b/xen/arch/x86/numa.c
>> @@ -54,14 +54,13 @@ bool __init arch_numa_unavailable(void)
>>   /*
>>    * Setup early cpu_to_node.
>>    *
>> - * Populate cpu_to_node[] only if x86_cpu_to_apicid[],
>> - * and apicid_to_node[] tables have valid entries for a CPU.
>> - * This means we skip cpu_to_node[] initialisation for NUMA
>> - * emulation and faking node case (when running a kernel compiled
>> - * for NUMA on a non NUMA box), which is OK as cpu_to_node[]
>> - * is already initialized in a round robin manner at numa_init_array,
>> - * prior to this call, and this initialization is good enough
>> - * for the fake NUMA cases.
>> + * Populate cpu_to_node[] only if cpu_data[], and apicid_to_node[]

You mean cpu_physical_id() here, and then this change wants doing when
switching to that, imo.

>> + * tables have valid entries for a CPU. This means we skip
>> + * cpu_to_node[] initialisation for NUMA emulation and faking node
>> + * case (when running a kernel compiled for NUMA on a non NUMA box),
>> + * which is OK as cpu_to_node[] is already initialized in a round
>> + * robin manner at numa_init_array, prior to this call, and this
>> + * initialization is good enough for the fake NUMA cases.
>>    */

Also if you're already re-wrapping this comment, please make better use
of line width.

>> --- a/xen/arch/x86/x86_64/asm-offsets.c
>> +++ b/xen/arch/x86/x86_64/asm-offsets.c
>> @@ -159,7 +159,9 @@ void __dummy__(void)
>>       OFFSET(IRQSTAT_softirq_pending, irq_cpustat_t, __softirq_pending);
>>       BLANK();
>>   
>> -    OFFSET(CPUINFO_features, struct cpuinfo_x86, x86_capability);
>> +    OFFSET(CPUINFO_X86_features, struct cpuinfo_x86, x86_capability);
> 
> The rename seems to be unrelated to this patch. Can you clarify?

I agree some renaming wants doing, but separately. That's because we
use CPUINFO_ as a prefix for two entirely different structure's offsets
right now. I'm not convinced of CPUINFO_X86_ as the new prefix though:
Uses are against cpu_data[], so CPUDATA_ may be better. Might be good
if Andrew and/or Roger could voice their view.

Jan

>> +    OFFSET(CPUINFO_X86_apicid, struct cpuinfo_x86, apicid);
>> +    DEFINE(CPUINFO_X86_sizeof, sizeof(struct cpuinfo_x86));
>>       BLANK();
>>   
>>       OFFSET(MB_flags, multiboot_info_t, flags);
> 
> Cheers,
> 



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 16:53:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 16:53:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677828.1054703 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXlBI-0005w0-Ay; Wed, 07 Feb 2024 16:53:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677828.1054703; Wed, 07 Feb 2024 16:53:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXlBI-0005vt-7r; Wed, 07 Feb 2024 16:53:52 +0000
Received: by outflank-mailman (input) for mailman id 677828;
 Wed, 07 Feb 2024 16:53:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXlBH-0005vn-OU
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 16:53:51 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 77259de2-c5d9-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 17:53:49 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33b28aadb28so654914f8f.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 08:53:49 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i16-20020adff310000000b003392206c808sm1839943wro.105.2024.02.07.08.53.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 08:53:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 77259de2-c5d9-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707324829; x=1707929629; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=c3/xLrT3xy8FCSaEeeaE3t51L7zMLxe+7hZs/3bSXwg=;
        b=E+hyLhNjIlr+HFQJ7j71/y0ypRp45az+vkWQFZMryc54zRcLA6UTkapYDz7jLxss5w
         y+4f5nSF/pI1VYFfPgjZR9UmK0IHdZPZEOvxO5BhQeOP2BDdQCnU18JsObRRFvTYgAB0
         8KlK7UuGoFHDjFY1iV6UTSMVZyorqMS2Vh2JdHlfk7P1OJjlZeSPo4Xm/fF+//auBOqc
         Ytu6qCGYTEfnQ/Zbs+Mg8WyLfZG4cC5y2YZLBaec1JEupCCIF6AZnJbNV9H6+X6xN+AO
         iGvoSwfjbnfoX539/u6kNbmNNzQEKmowxoevQd1nXkU0FWLUewQutiQjXTcaa+hiyn5j
         qHlA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707324829; x=1707929629;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=c3/xLrT3xy8FCSaEeeaE3t51L7zMLxe+7hZs/3bSXwg=;
        b=oejnxPGg+HllCd5XxbYshSYbde5MaWCliPrwpAYltQldPvPrQYburJvOKuQPGJ+8ga
         ZnfMyhaM+u3IxsBMoOxHz0TnPv4c21zOXCuMGkSSec1O8lbhbcL9gR7Sq1At345rXshZ
         w01DwdWs6EYBtoFF9FOa1vNQX6WKgE/V375GoXdoL+U8MvqDnOsO16b8YLrkoJxkO8qc
         wrJPFzKvmuOMhDIsp5Pt8yzeyU1nkSk3rkxJ7uX1GhUjotPe9S2nOsOdybxCDuTDLUZ0
         pfEyyARIIkazas3xb/asfaHZHeYVZA8UlvOqijHFZ6f4+kxXcrzBo7CWxRmdUKrfqT/3
         URHA==
X-Forwarded-Encrypted: i=1; AJvYcCWSf76tpWgKHGEUkJl9GYMgkHTfbm9LJ6LJBMkVPa5wjuvad6AaqyJRhClLt77n5lTaKkAc88RVtO6ZPetixtIxoBjlBpHds4yUCWOyC5U=
X-Gm-Message-State: AOJu0YwzzNqdMhHJn11DMzTF7X3Ax5gmqnYt8qA/GC9aFJEH8vWx2phN
	0ANq8/w6dJqR7M7mw9eb/ZAnLlrCmNPXq1v9cSskMrRxD1940wls+QDmggkviw==
X-Google-Smtp-Source: AGHT+IHE0umQRAIxhUfxJ/RV/+x9qs21FNAxdzKQj2sOvq3g+OvNq8qR+sj4HLL5jDnyl9V2FKe8Sw==
X-Received: by 2002:a5d:4a45:0:b0:33b:3a62:b0f2 with SMTP id v5-20020a5d4a45000000b0033b3a62b0f2mr4247278wrs.12.1707324828830;
        Wed, 07 Feb 2024 08:53:48 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU0pxRAQwOSVqCKbBVHzZzqWb3c7/5K4UAQHvRcb4N2uq0jBnpmA64O06dDIm3vYkJ7L3efVDI68t9sNnDOUGtPeIwp5j5cRaeERTaXetyvY8yZHYGIHXIC9GS+vDsVs9grCs1KkgutnxD8nnXJu2CSv0m5Qrwh46dpKFEWA2PWV8ff0TBxgK0HlkL4T/GG+pUMWwtZ6MfHR7NZMjp9K3XXL7CecTHUm2/5zcw+E7LaVfBm
Message-ID: <67fb6859-1306-4000-8b4c-fddd505ead6a@suse.com>
Date: Wed, 7 Feb 2024 17:53:47 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 4/9] x86/smp: move stack_base to cpu_data
Content-Language: en-US
To: Krystian Hebel <krystian.hebel@3mdeb.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <70e3b7c84a69a7ec52b3ed6314395165c281734c.1699982111.git.krystian.hebel@3mdeb.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <70e3b7c84a69a7ec52b3ed6314395165c281734c.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.11.2023 18:50, Krystian Hebel wrote:
> --- a/xen/arch/x86/boot/x86_64.S
> +++ b/xen/arch/x86/boot/x86_64.S
> @@ -33,9 +33,8 @@ ENTRY(__high_start)
>          cmp     %esp, CPUINFO_X86_apicid(%rcx)
>          jne     1b
>  
> -        /* %eax is now Xen CPU index. */
> -        lea     stack_base(%rip), %rcx
> -        mov     (%rcx, %rax, 8), %rsp
> +        /* %rcx is now cpu_data[cpu], read stack base from it. */
> +        mov     CPUINFO_X86_stack_base(%rcx), %rsp

Looks like you're not using the value in %eax anymore? If so, respective
code would want dropping. Which in turn again raises the question that
Julien already put up: By re-ordering the series, can't you avoid
altering the same code multiple times, in part even removing in a later
patch what an earlier one added?

That said, I remain unconvinced that ...

> --- a/xen/arch/x86/include/asm/cpufeature.h
> +++ b/xen/arch/x86/include/asm/cpufeature.h
> @@ -37,6 +37,7 @@ struct cpuinfo_x86 {
>      unsigned int phys_proc_id;         /* package ID of each logical CPU */
>      unsigned int cpu_core_id;          /* core ID of each logical CPU */
>      unsigned int compute_unit_id;      /* AMD compute unit ID of each logical CPU */
> +    void *stack_base;
>      unsigned short x86_clflush_size;
>  } __cacheline_aligned;

... this is a good place for the new data: As indicated before, it
doesn't fit (in nature) with everything else in this struct.

Additionally no matter where the data is put, I'd wonder if it
wouldn't better be const void *. You don't mean to ever write
through it, I suppose.

> @@ -1156,7 +1156,8 @@ void __init smp_prepare_cpus(void)
>      boot_cpu_physical_apicid = get_apic_id();
>      cpu_physical_id(0) = boot_cpu_physical_apicid;
>  
> -    stack_base[0] = (void *)((unsigned long)stack_start & ~(STACK_SIZE - 1));
> +    cpu_data[0].stack_base = (void *)
> +             ((unsigned long)stack_start & ~(STACK_SIZE - 1));

Nit: Too deep indentation. Each indentation level is 4 spaces. I also
think the cast would then also want to move on the 2nd line, such that
(see again ./CODING_STYLE) the assignment operator is last on the 1st
line.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 17:02:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 17:02:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677833.1054713 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXlJS-0007iY-4r; Wed, 07 Feb 2024 17:02:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677833.1054713; Wed, 07 Feb 2024 17:02:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXlJS-0007iR-1I; Wed, 07 Feb 2024 17:02:18 +0000
Received: by outflank-mailman (input) for mailman id 677833;
 Wed, 07 Feb 2024 17:02:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXlJQ-0007iH-Ex; Wed, 07 Feb 2024 17:02:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXlJQ-0006az-3U; Wed, 07 Feb 2024 17:02:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXlJP-0007Ry-Kr; Wed, 07 Feb 2024 17:02:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXlJP-000389-KR; Wed, 07 Feb 2024 17:02:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2tl5vp5abRgV0gvhy2ZIdPYibAkfjO4YjHnFUzjVrpw=; b=T7tY8Kq3XJu0I72HXKW/BOq+sp
	50G3hpflSqZxUwWa3MFgOwBTIpLjbAZBCAfHiH08LpDV3G3OnJmvMgfzkic5pfVQ0oDZWWI8lcaO7
	PYpWkFQD87E42LCxocrgmBC9m/gtduk3s9lifwD3xQ7YcW2EYKf0Ly4QUQ02plTF9zbc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184616-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184616: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f4519ee87ebfb74fe0451130d82c99453b75e2e1
X-Osstest-Versions-That:
    xen=59a69674b533a2d0c0ad5cd7fbcd655295574509
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 07 Feb 2024 17:02:15 +0000

flight 184616 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184616/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  f4519ee87ebfb74fe0451130d82c99453b75e2e1
baseline version:
 xen                  59a69674b533a2d0c0ad5cd7fbcd655295574509

Last test of basis   184611  2024-02-07 01:02:09 Z    0 days
Testing same since   184616  2024-02-07 13:03:56 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
  Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
  Simone Ballarin  <simone.ballarin@bugseng.com>
  Simone Ballarin <simone.ballarin@bugseng.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   59a69674b5..f4519ee87e  f4519ee87ebfb74fe0451130d82c99453b75e2e1 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 17:02:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 17:02:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677838.1054723 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXlK2-0008HZ-Fx; Wed, 07 Feb 2024 17:02:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677838.1054723; Wed, 07 Feb 2024 17:02:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXlK2-0008HS-D8; Wed, 07 Feb 2024 17:02:54 +0000
Received: by outflank-mailman (input) for mailman id 677838;
 Wed, 07 Feb 2024 17:02:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EHwQ=JQ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXlK0-0008Ax-Vx
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 17:02:52 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b9d1cacd-c5da-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 18:02:50 +0100 (CET)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-33b29b5eab5so578956f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 09:02:50 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l4-20020adfe9c4000000b0033afb963d0dsm1910765wrn.101.2024.02.07.09.02.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 09:02:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9d1cacd-c5da-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707325370; x=1707930170; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=L65Bs2/pfBg+seYP0MIV5oa3dLYjzOjXBK7CtBddhoM=;
        b=PHAP7xDvdEjpmsh0K868QfzPt+KVC5dAI3BROYuD/Uoxeoh1lIbo837ryXApgVXotx
         K5HSsi6ZQmJH+mHPOOO1Nn0SnIHEl/rqHQZ4slUhRjyBIT1ww/Sy5/ztgEoftUwSwCzo
         hkP9rvtdoAesPdvn1S1/eeHWORnchS8VypxpSkAZ3XoxEQzrU3MpPvM6zPYD0M7qdMIC
         lkdskUfdYeFiBw5sUF9fm8HdLsy+LcSPeWLL4n3eObIjngWuGzwzBZX11auw/rc4YGtE
         bPKql94giNjdT9ST9xrBaqwFak3s9qqCyKGnK/F98qY6qzSjQS0cKDp8KqVmwuG3WgdP
         mIBA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707325370; x=1707930170;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=L65Bs2/pfBg+seYP0MIV5oa3dLYjzOjXBK7CtBddhoM=;
        b=QDE07KcbHC/vsoeqkhk+wDD+jhk6zS2j+kCZK2nMYeSjEQCgO3HAnqmekxzdttnfZR
         QUIfwqUfH9YWrunvbinz0VTmYMYkGtfIBHpgo1g279i1DNJQaTkplOsKHOARPb66rhCk
         ruKKH6K1WfelGl8lQk+oXOH8JEz+9wG75Np1rR3ApiTNEdLamXl+g/HRTmEluVKKBZyK
         Wij6/DyAOXdTV0wMI0dGt/aC1/MQmaNC9zzQuwZvfoKrCNDfZr4ZJQ7ZIxI3yHQuq4p3
         iKA34/CwfAVNUYby/564t6oxOYjyH1uA+lk9iGF0v86jFn8dD5bXup60TWQAJqY2fErl
         zx+Q==
X-Gm-Message-State: AOJu0YyqibqYN6wfIq9fooFiKEYnxe2+4tP7G9WJf31pIx1AY1hbZ2vW
	dqqqlZkbX5tf9mrtuo9bzKI3am5xjKaydeHMdalCpJ9LIoRITphFfqAnHpRjxg==
X-Google-Smtp-Source: AGHT+IHkAx6Vu784Vhd6LABljCFiMqpRaHpKheDLXBdrcNEPy8wjzaFenR/xLN/oJGLzizGD/R7kxw==
X-Received: by 2002:a5d:4cc5:0:b0:33b:5216:ef2b with SMTP id c5-20020a5d4cc5000000b0033b5216ef2bmr890000wrt.3.1707325370216;
        Wed, 07 Feb 2024 09:02:50 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW95E5ivFZXn4pivFxjWdSVdI7m6xwtnhGkg8bTyWyPPFjM7CWV2DYwpajAshDfKGf2FzaoZcrH4MaNdChITtfbPlsc+yT1H7daxqjmEHOsVECWC53nKQf09Q/cR1jU6cGnbaO4pSTQ2V1xxIZhCZy/CIZh6EA=
Message-ID: <a9a44e5d-80cf-404a-abe0-985a11f6d668@suse.com>
Date: Wed, 7 Feb 2024 18:02:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 5/9] x86/smp: call x2apic_ap_setup() earlier
Content-Language: en-US
To: Krystian Hebel <krystian.hebel@3mdeb.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <7c13554e60cc76516922992b7faf911b91f99a2a.1699982111.git.krystian.hebel@3mdeb.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <7c13554e60cc76516922992b7faf911b91f99a2a.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.11.2023 18:50, Krystian Hebel wrote:
> It used to be called from smp_callin(), however BUG_ON() was invoked on
> multiple occasions before that. It may end up calling machine_restart()
> which tries to get APIC ID for CPU running this code. If BSP detected
> that x2APIC is enabled, get_apic_id() will try to use it for all CPUs.
> Enabling x2APIC on secondary CPUs earlier protects against an endless
> loop of #GP exceptions caused by attempts to read IA32_X2APIC_APICID
> MSR while x2APIC is disabled in IA32_APIC_BASE.
> 
> Signed-off-by: Krystian Hebel <krystian.hebel@3mdeb.com>
> ---
>  xen/arch/x86/smpboot.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
> index 8ae65ab1769f..a3895dafa267 100644
> --- a/xen/arch/x86/smpboot.c
> +++ b/xen/arch/x86/smpboot.c
> @@ -184,7 +184,6 @@ static void smp_callin(void)
>       * update until we finish. We are free to set up this CPU: first the APIC.
>       */
>      Dprintk("CALLIN, before setup_local_APIC().\n");
> -    x2apic_ap_setup();
>      setup_local_APIC(false);
>  
>      /* Save our processor parameters. */
> @@ -351,6 +350,14 @@ void start_secondary(void *unused)
>      get_cpu_info()->xen_cr3 = 0;
>      get_cpu_info()->pv_cr3 = 0;
>  
> +    /*
> +     * BUG_ON() used in load_system_tables() and later code may end up calling
> +     * machine_restart() which tries to get APIC ID for CPU running this code.
> +     * If BSP detected that x2APIC is enabled, get_apic_id() will try to use it
> +     * for _all_ CPUs. Enable x2APIC on secondary CPUs now so we won't end up
> +     * with endless #GP loop.
> +     */
> +    x2apic_ap_setup();
>      load_system_tables();

While I find the argument convincing, I seem to recall that there was a
firm plan to have load_system_tables() as early as possible. Andrew?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 17:40:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 17:40:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677848.1054753 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXluM-0005ww-NN; Wed, 07 Feb 2024 17:40:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677848.1054753; Wed, 07 Feb 2024 17:40:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXluM-0005wp-JX; Wed, 07 Feb 2024 17:40:26 +0000
Received: by outflank-mailman (input) for mailman id 677848;
 Wed, 07 Feb 2024 17:40:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0q5t=JQ=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1rXluK-0005Tj-Mx
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 17:40:24 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f80944d2-c5df-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 18:40:22 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a2d7e2e7fe0so10964766b.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 09:40:22 -0800 (PST)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 op3-20020a170906bce300b00a37669280d1sm976355ejb.141.2024.02.07.09.40.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 09:40:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f80944d2-c5df-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707327621; x=1707932421; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=mVtApRo3oDb95gESnvgdip5hWmKbe9Gt8uYW92ZcjgY=;
        b=IHKLTSYqGYMG2FbH4XhU2/4dg8HjyG7XKMChdFW0AzS2SS0+rBa/mIg1pLePXoV6Cj
         6TloVV9S5DRGe8SpK3kjoXh02ycAD5nidZo6V5bSfqYnhBKbsd41VkZRGCJD4l/J9HQR
         H7r123liUYtSaJWYGIsGgCtS3qIJxLiLhAIBg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707327621; x=1707932421;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=mVtApRo3oDb95gESnvgdip5hWmKbe9Gt8uYW92ZcjgY=;
        b=SaLmk5gg3BKOST9x3myILd3xwm54635Y2x7M7a083+S4m+ULArgLdIUNt9mEhHHZEx
         EE/rqEnhV7AmYnVOM7+CJF0D8O+hZb2tbgSgZYpp2KONPiun4ZJZXb+GzPV23fndW3R4
         NoK7zTL7ZSi9KZwvijWv+b7un7PDRwfE324OesvI9CjzQYuK/bTe8R48EpSS6ZAeKCyi
         GiVaBUErcuvNR8+fkgbVEd8ydvuk0jKsycM3Kki1xsIlIrsFCPzTLzeozbQVNWMW0CEI
         owdC0sbvieVAKSo70O/Q3qtPDI0WDZ0UA0Ze0q7ULl78ZmoYUNtpkUKjdYPvO1QTJVO1
         Jy/g==
X-Gm-Message-State: AOJu0YybHxX/YS3sJS14HVpOq+zdbp9VGCErpy6JHzYhbQfKBXpU941t
	SsB/Yt2WQwpJPLcMwTqeH95ho5J+dDKHs0nGmt7HutWxSfMxQIajGwzzqDj6RoNaebax3kTyk+f
	Q
X-Google-Smtp-Source: AGHT+IFti7lLnLxiOpm9L+6Lm+EqtJakdk0jtGAtrQZIv4hyXS+2eHGRGr7hF7VGK4dj1jA3WvSInw==
X-Received: by 2002:a17:907:7d89:b0:a38:c2f9:4ee with SMTP id oz9-20020a1709077d8900b00a38c2f904eemr337167ejc.0.1707327621345;
        Wed, 07 Feb 2024 09:40:21 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUu8JHBp8+s80ogwlQxOobccH2V6n8CB4NFjDsKsHcVsCjocjr8+LHqnciFLJV+cvz9ipel+qPrGpwUWU0rHratbFeRltuKDTAyMOW8946xPYF9IEOF5rfdEE2QGfsdEe6kvCdzeDMMsbTxduZ22i6lh+riJqeTHys8O8ScEF3Au59iVHk08WNMgf8KFBgY313lRhjvzc7aJz80UODN6Nv1XJ6A6Xi9E/la8Q==
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH 2/3] tools/xg: Streamline cpu policy serialise/deserialise calls
Date: Wed,  7 Feb 2024 17:39:56 +0000
Message-Id: <20240207173957.19811-3-alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240207173957.19811-1-alejandro.vallejo@cloud.com>
References: <20240207173957.19811-1-alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The idea is to use xc_cpu_policy_t as a single object containing both the
serialised and deserialised forms of the policy. Note that we need lengths
for the arrays, as the serialised policies may be shorter than the array
capacities.

* Add the serialised lengths to the struct so we can distinguish
  between length and capacity of the serialisation buffers.
* Remove explicit buffer+lengths in serialise/deserialise calls
  and use the internal buffer inside xc_cpu_policy_t instead.
* Refactor everything to use the new serialisation functions.
* Remove redundant serialization calls and avoid allocating dynamic
  memory aside from the policy objects in xen-cpuid. Also minor cleanup
  in the policy print call sites.

No functional change intended.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
Splitting this patch in several is quite annoying, as it implies
artificially creating duplicates of the (de)serialization functions, and
applying them little by little. The resulting diffs are not much better
than this.
---
 tools/include/xenguest.h            |  6 +--
 tools/libs/guest/xg_cpuid_x86.c     | 62 ++++++++++++++---------------
 tools/libs/guest/xg_sr_common_x86.c | 54 +++++++++----------------
 tools/misc/xen-cpuid.c              | 40 +++++--------------
 4 files changed, 61 insertions(+), 101 deletions(-)

diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
index 4e9078fdee4d..5c287b4a210c 100644
--- a/tools/include/xenguest.h
+++ b/tools/include/xenguest.h
@@ -789,7 +789,9 @@ xen_pfn_t *xc_map_m2p(xc_interface *xch,
 typedef struct xc_cpu_policy {
     struct cpu_policy policy;
     xen_cpuid_leaf_t leaves[CPUID_MAX_SERIALISED_LEAVES];
+    uint32_t nr_leaves;
     xen_msr_entry_t msrs[MSR_MAX_SERIALISED_ENTRIES];
+    uint32_t nr_msrs;
 } xc_cpu_policy_t;
 
 /* Create and free a xc_cpu_policy object. */
@@ -805,9 +807,7 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
                              xc_cpu_policy_t *policy);
 
 /* Manipulate a policy via architectural representations. */
-int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *policy,
-                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
-                            xen_msr_entry_t *msrs, uint32_t *nr_msrs);
+int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *policy);
 int xc_cpu_policy_update_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
                                const xen_cpuid_leaf_t *leaves,
                                uint32_t nr);
diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 4453178100ad..5699a26b946e 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -834,14 +834,13 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
     }
 }
 
-static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
-                              unsigned int nr_leaves, unsigned int nr_entries)
+static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
 {
     uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
     int rc;
 
     rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
-                                    nr_leaves, &err_leaf, &err_subleaf);
+                                    policy->nr_leaves, &err_leaf, &err_subleaf);
     if ( rc )
     {
         if ( err_leaf != -1 )
@@ -851,7 +850,7 @@ static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy,
     }
 
     rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
-                                  nr_entries, &err_msr);
+                                  policy->nr_msrs, &err_msr);
     if ( rc )
     {
         if ( err_msr != -1 )
@@ -878,7 +877,10 @@ int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
         return rc;
     }
 
-    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
+    policy->nr_leaves = nr_leaves;
+    policy->nr_msrs = nr_msrs;
+
+    rc = deserialize_policy(xch, policy);
     if ( rc )
     {
         errno = -rc;
@@ -903,7 +905,10 @@ int xc_cpu_policy_get_domain(xc_interface *xch, uint32_t domid,
         return rc;
     }
 
-    rc = deserialize_policy(xch, policy, nr_leaves, nr_msrs);
+    policy->nr_leaves = nr_leaves;
+    policy->nr_msrs = nr_msrs;
+
+    rc = deserialize_policy(xch, policy);
     if ( rc )
     {
         errno = -rc;
@@ -917,17 +922,14 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
                              xc_cpu_policy_t *policy)
 {
     uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    unsigned int nr_leaves = ARRAY_SIZE(policy->leaves);
-    unsigned int nr_msrs = ARRAY_SIZE(policy->msrs);
     int rc;
 
-    rc = xc_cpu_policy_serialise(xch, policy, policy->leaves, &nr_leaves,
-                                 policy->msrs, &nr_msrs);
+    rc = xc_cpu_policy_serialise(xch, policy);
     if ( rc )
         return rc;
 
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, policy->leaves,
-                                  nr_msrs, policy->msrs,
+    rc = xc_set_domain_cpu_policy(xch, domid, policy->nr_leaves, policy->leaves,
+                                  policy->nr_msrs, policy->msrs,
                                   &err_leaf, &err_subleaf, &err_msr);
     if ( rc )
     {
@@ -942,34 +944,32 @@ int xc_cpu_policy_set_domain(xc_interface *xch, uint32_t domid,
     return rc;
 }
 
-int xc_cpu_policy_serialise(xc_interface *xch, const xc_cpu_policy_t *p,
-                            xen_cpuid_leaf_t *leaves, uint32_t *nr_leaves,
-                            xen_msr_entry_t *msrs, uint32_t *nr_msrs)
+int xc_cpu_policy_serialise(xc_interface *xch, xc_cpu_policy_t *p)
 {
+    unsigned int nr_leaves = ARRAY_SIZE(p->leaves);
+    unsigned int nr_msrs = ARRAY_SIZE(p->msrs);
     int rc;
 
-    if ( leaves )
+    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
+    if ( rc )
     {
-        rc = x86_cpuid_copy_to_buffer(&p->policy, leaves, nr_leaves);
-        if ( rc )
-        {
-            ERROR("Failed to serialize CPUID policy");
-            errno = -rc;
-            return -1;
-        }
+        ERROR("Failed to serialize CPUID policy");
+        errno = -rc;
+        return -1;
     }
 
-    if ( msrs )
+    p->nr_leaves = nr_leaves;
+
+    rc = x86_msr_copy_to_buffer(&p->policy, p->msrs, &nr_msrs);
+    if ( rc )
     {
-        rc = x86_msr_copy_to_buffer(&p->policy, msrs, nr_msrs);
-        if ( rc )
-        {
-            ERROR("Failed to serialize MSR policy");
-            errno = -rc;
-            return -1;
-        }
+        ERROR("Failed to serialize MSR policy");
+        errno = -rc;
+        return -1;
     }
 
+    p->nr_msrs = nr_msrs;
+
     errno = 0;
     return 0;
 }
diff --git a/tools/libs/guest/xg_sr_common_x86.c b/tools/libs/guest/xg_sr_common_x86.c
index 563b4f016877..832047756e58 100644
--- a/tools/libs/guest/xg_sr_common_x86.c
+++ b/tools/libs/guest/xg_sr_common_x86.c
@@ -1,4 +1,5 @@
 #include "xg_sr_common_x86.h"
+#include "xg_sr_stream_format.h"
 
 int write_x86_tsc_info(struct xc_sr_context *ctx)
 {
@@ -45,54 +46,37 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)
 int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
-    struct xc_sr_record cpuid = { .type = REC_TYPE_X86_CPUID_POLICY, };
-    struct xc_sr_record msrs  = { .type = REC_TYPE_X86_MSR_POLICY, };
-    uint32_t nr_leaves = 0, nr_msrs = 0;
-    xc_cpu_policy_t *policy = NULL;
+    struct xc_sr_record record;
+    xc_cpu_policy_t *policy = xc_cpu_policy_init();
     int rc;
 
-    if ( xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs) < 0 )
-    {
-        PERROR("Unable to get CPU Policy size");
-        return -1;
-    }
-
-    cpuid.data = malloc(nr_leaves * sizeof(xen_cpuid_leaf_t));
-    msrs.data  = malloc(nr_msrs   * sizeof(xen_msr_entry_t));
-    policy = xc_cpu_policy_init();
-    if ( !cpuid.data || !msrs.data || !policy )
-    {
-        ERROR("Cannot allocate memory for CPU Policy");
-        rc = -1;
-        goto out;
-    }
-
-    if ( xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
+    if ( !policy || xc_cpu_policy_get_domain(xch, ctx->domid, policy) )
     {
         PERROR("Unable to get d%d CPU Policy", ctx->domid);
         rc = -1;
         goto out;
     }
-    if ( xc_cpu_policy_serialise(xch, policy, cpuid.data, &nr_leaves,
-                                 msrs.data, &nr_msrs) )
-    {
-        PERROR("Unable to serialize d%d CPU Policy", ctx->domid);
-        rc = -1;
-        goto out;
-    }
 
-    cpuid.length = nr_leaves * sizeof(xen_cpuid_leaf_t);
-    if ( cpuid.length )
+    record = (struct xc_sr_record) {
+        .type = REC_TYPE_X86_CPUID_POLICY,
+        .data = policy->leaves,
+        .length = policy->nr_leaves * sizeof(*policy->leaves),
+    };
+    if ( record.length )
     {
-        rc = write_record(ctx, &cpuid);
+        rc = write_record(ctx, &record);
         if ( rc )
             goto out;
     }
 
-    msrs.length = nr_msrs * sizeof(xen_msr_entry_t);
-    if ( msrs.length )
+    record = (struct xc_sr_record) {
+        .type = REC_TYPE_X86_MSR_POLICY,
+        .data = policy->msrs,
+        .length = policy->nr_msrs * sizeof(*policy->msrs),
+    };
+    if ( record.length )
     {
-        rc = write_record(ctx, &msrs);
+        rc = write_record(ctx, &record);
         if ( rc )
             goto out;
     }
@@ -100,8 +84,6 @@ int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
     rc = 0;
 
  out:
-    free(cpuid.data);
-    free(msrs.data);
     xc_cpu_policy_destroy(policy);
 
     return rc;
diff --git a/tools/misc/xen-cpuid.c b/tools/misc/xen-cpuid.c
index 46abdb914092..1d4761d9ad9f 100644
--- a/tools/misc/xen-cpuid.c
+++ b/tools/misc/xen-cpuid.c
@@ -403,17 +403,18 @@ static void dump_info(xc_interface *xch, bool detail)
     free(fs);
 }
 
-static void print_policy(const char *name,
-                         xen_cpuid_leaf_t *leaves, uint32_t nr_leaves,
-                         xen_msr_entry_t *msrs, uint32_t nr_msrs)
+static void print_policy(const char *name, xc_cpu_policy_t *policy)
 {
-    unsigned int l;
+    const xen_cpuid_leaf_t *leaves = policy->leaves;
+    const xen_msr_entry_t *msrs = policy->msrs;
+    uint32_t nr_leaves = policy->nr_leaves;
+    uint32_t nr_msrs = policy->nr_msrs;
 
     printf("%s policy: %u leaves, %u MSRs\n", name, nr_leaves, nr_msrs);
     printf(" CPUID:\n");
     printf("  %-8s %-8s -> %-8s %-8s %-8s %-8s\n",
            "leaf", "subleaf", "eax", "ebx", "ecx", "edx");
-    for ( l = 0; l < nr_leaves; ++l )
+    for ( uint32_t l = 0; l < nr_leaves; ++l )
     {
         /* Skip empty leaves. */
         if ( !leaves[l].a && !leaves[l].b && !leaves[l].c && !leaves[l].d )
@@ -426,7 +427,7 @@ static void print_policy(const char *name,
 
     printf(" MSRs:\n");
     printf("  %-8s -> %-16s\n", "index", "value");
-    for ( l = 0; l < nr_msrs; ++l )
+    for ( uint32_t l = 0; l < nr_msrs; ++l )
         printf("  %08x -> %016"PRIx64"\n",
                msrs[l].idx, msrs[l].val);
 }
@@ -533,8 +534,6 @@ int main(int argc, char **argv)
             [ XEN_SYSCTL_cpu_policy_pv_default ]   = "PV Default",
             [ XEN_SYSCTL_cpu_policy_hvm_default ]  = "HVM Default",
         };
-        xen_cpuid_leaf_t *leaves;
-        xen_msr_entry_t *msrs;
         uint32_t i, max_leaves, max_msrs;
 
         xc_interface *xch = xc_interface_open(0, 0, 0);
@@ -551,36 +550,21 @@ int main(int argc, char **argv)
             printf("Xen reports there are maximum %u leaves and %u MSRs\n",
                    max_leaves, max_msrs);
 
-        leaves = calloc(max_leaves, sizeof(xen_cpuid_leaf_t));
-        if ( !leaves )
-            err(1, "calloc(max_leaves)");
-        msrs = calloc(max_msrs, sizeof(xen_msr_entry_t));
-        if ( !msrs )
-            err(1, "calloc(max_msrs)");
-
         if ( domid != -1 )
         {
             char name[20];
-            uint32_t nr_leaves = max_leaves;
-            uint32_t nr_msrs = max_msrs;
 
             if ( xc_cpu_policy_get_domain(xch, domid, policy) )
                 err(1, "xc_cpu_policy_get_domain(, %d, )", domid);
-            if ( xc_cpu_policy_serialise(xch, policy, leaves, &nr_leaves,
-                                         msrs, &nr_msrs) )
-                err(1, "xc_cpu_policy_serialise");
 
             snprintf(name, sizeof(name), "Domain %d", domid);
-            print_policy(name, leaves, nr_leaves, msrs, nr_msrs);
+            print_policy(name, policy);
         }
         else
         {
             /* Get system policies */
             for ( i = 0; i < ARRAY_SIZE(sys_policies); ++i )
             {
-                uint32_t nr_leaves = max_leaves;
-                uint32_t nr_msrs = max_msrs;
-
                 if ( xc_cpu_policy_get_system(xch, i, policy) )
                 {
                     if ( errno == EOPNOTSUPP )
@@ -592,18 +576,12 @@ int main(int argc, char **argv)
 
                     err(1, "xc_cpu_policy_get_system(, %s, )", sys_policies[i]);
                 }
-                if ( xc_cpu_policy_serialise(xch, policy, leaves, &nr_leaves,
-                                             msrs, &nr_msrs) )
-                    err(1, "xc_cpu_policy_serialise");
 
-                print_policy(sys_policies[i], leaves, nr_leaves,
-                             msrs, nr_msrs);
+                print_policy(sys_policies[i], policy);
             }
         }
 
         xc_cpu_policy_destroy(policy);
-        free(leaves);
-        free(msrs);
         xc_interface_close(xch);
     }
     else if ( mode == MODE_INFO || mode == MODE_DETAIL )
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 17:40:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 17:40:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677846.1054733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXluJ-0005Tm-9V; Wed, 07 Feb 2024 17:40:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677846.1054733; Wed, 07 Feb 2024 17:40:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXluJ-0005Te-6P; Wed, 07 Feb 2024 17:40:23 +0000
Received: by outflank-mailman (input) for mailman id 677846;
 Wed, 07 Feb 2024 17:40:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0q5t=JQ=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1rXluI-0005TY-1N
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 17:40:22 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f6ef0306-c5df-11ee-8a4a-1f161083a0e0;
 Wed, 07 Feb 2024 18:40:20 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a30f7c9574eso123379466b.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 09:40:20 -0800 (PST)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 op3-20020a170906bce300b00a37669280d1sm976355ejb.141.2024.02.07.09.40.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 09:40:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6ef0306-c5df-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707327619; x=1707932419; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=sD3mINkQKVOArN50c8qQ7+QTNBNiB330mzDy3nnj1xU=;
        b=HTmjyvOGL9+ENIHosrIdCMbYfbtnhfcO+E0+7dDj1jAS5WsomC1pWt/4VKRo+xbIfC
         cfkcJhV74XLQep/joCC45QP7a9eO7Q+ugpeygVOuX4mQ3+QVeLp522jZ+GhIxgF2EVT5
         tFdvMEO1EAbYlmMKf7JgTEvLu42iVepqwVbZM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707327619; x=1707932419;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=sD3mINkQKVOArN50c8qQ7+QTNBNiB330mzDy3nnj1xU=;
        b=Tpjz4Fi7K1hPQ0DC3hLBtx9+uubQUHmBB9ZykKjo/8cd0BgEaB2Nj/CbfKThOFHdjN
         ltzF1gmyTYTLdWT27CfyfbwqN5aHlqEkM6uAgnBHdZ1ldqXDD6LNh34gt45xzSLtkt7L
         gwl4iuQfLaqsiplm7o9AUXgnSbS8lxCOEcEvwI6DHTeJnA1P/9XqBoZ8XfRUQkmFU19v
         wRMjXDnUGZQBmzIOrJlDFgiJr2cVmPJWMIB80JT8jPVvxQLebFkSQz/67nxbBvE6cYfe
         QhFR7oxgud0cglOdT6iDjXEsyDGnMaLt2zCKbyPvDiyTVrlgyzhMkUIYrekVTMnZedfh
         Y1eg==
X-Gm-Message-State: AOJu0Yw/84fMk0LnS0FQkrkkADudA9qfA23RUr2Z9IVhD4U3z0hvsXXV
	igFDWkVU9Tai5RNycs/mXm91Z/kmNTX3S2xBBgpO33Qb8k0g8cd/8MviXTVhX2Ia+XI2/0MnqOE
	r
X-Google-Smtp-Source: AGHT+IF11QpAkfRxY+MsanOmPWEMYD3Cq0V8iwuzsFs+nzpzBUQtbx6IVOL41xAtkbIkxubcfvhi+g==
X-Received: by 2002:a17:906:b28c:b0:a39:3bf2:fd1c with SMTP id q12-20020a170906b28c00b00a393bf2fd1cmr56641ejz.50.1707327619642;
        Wed, 07 Feb 2024 09:40:19 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWhFp+xi0PV2VrIacwi6eV5jByd9LFUZjmp7Xk81upR6OycBLOHVelDdCSlH67cg6LMs9dUo05+r0s2XkI0mYeJrObHfet/GlUkYsn7I+Dwm8Zm7RdhPOPDfCGrh5+jJf8PW66fESSrMauEewqN4d5H8lAow0Np/O/GdDXEhEq/NISGM9nwDdF4UAOJ2v9zlywujoMaAb5JXhFffjAfDzNPSL4on0LZUYFmxg==
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH 0/3] Clean the policy manipulation path in domain creation
Date: Wed,  7 Feb 2024 17:39:54 +0000
Message-Id: <20240207173957.19811-1-alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

In the context of creating a domain, we currently issue a lot of hypercalls
redundantly while populating its CPU policy; likely a side effect of
organic growth more than anything else.

However, the worst part is not the overhead (this is a glacially cold
path), but the insane amounts of boilerplate that make it really hard to
pick apart what's going on. One major contributor to this situation is the
fact that what's effectively "setup" and "teardown" phases in policy
manipulation are not factored out from the functions that perform said
manipulations, leading to the same getters and setter being invoked many
times, when once each would do.

Another big contributor is the code being unaware of when a policy is
serialised and when it's not.

This patch attempts to alleviate this situation, yielding over 200 LoC
reduction.

Patch 1: Mechanical change. Makes xc_cpu_policy_t public so it's usable
         from clients of libxc/libxg.
Patch 2: Changes the (de)serialization wrappers in xenguest so they always
         serialise to/from the internal buffers of xc_cpu_policy_t. The
         struct is suitably expanded to hold extra information required.
Patch 3: Performs the refactor of the policy manipulation code so that it
         follows a strict: PULL_POLICIES, MUTATE_POLICY (n times), PUSH_POLICY.

Alejandro Vallejo (3):
  tools/xg: Move xc_cpu_policy_t to xenguest.h
  tools/xg: Streamline cpu policy serialise/deserialise calls
  tools/xg: Clean up xend-style overrides for CPU policies

 tools/include/xenguest.h             |  14 +-
 tools/libs/guest/xg_cpuid_x86.c      | 500 +++++++++------------------
 tools/libs/guest/xg_private.h        |  10 -
 tools/libs/guest/xg_sr_common_x86.c  |  54 +--
 tools/misc/xen-cpuid.c               |  40 +--
 xen/include/xen/lib/x86/cpu-policy.h |   6 +-
 6 files changed, 197 insertions(+), 427 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 17:40:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 17:40:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677849.1054758 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXluN-00060T-1S; Wed, 07 Feb 2024 17:40:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677849.1054758; Wed, 07 Feb 2024 17:40:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXluM-0005zY-SE; Wed, 07 Feb 2024 17:40:26 +0000
Received: by outflank-mailman (input) for mailman id 677849;
 Wed, 07 Feb 2024 17:40:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0q5t=JQ=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1rXluL-0005Tj-6h
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 17:40:25 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f82012e9-c5df-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 18:40:22 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-55f50cf2021so1147683a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 09:40:22 -0800 (PST)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 op3-20020a170906bce300b00a37669280d1sm976355ejb.141.2024.02.07.09.40.21
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 09:40:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f82012e9-c5df-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707327622; x=1707932422; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ldOTwPyqzlRUxWLHaw7i5K1+RmAlnFbaE9qRDYjtAH0=;
        b=OEEhh8ObTtLBmFFcrGBrcK0UiNVrx5h45tMxZKhrzlGM3EnSHiMP9ZCpte4ySlJe56
         9Py2EaM8xNIjPk88uRU0obzCBeyc+3ug3BGosS6j9G9igXeCh5kPtWXpKDltVxv7XsXG
         q9P+lXJBSF/OW0+AHfKzQvfsyOuve10PNzesU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707327622; x=1707932422;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ldOTwPyqzlRUxWLHaw7i5K1+RmAlnFbaE9qRDYjtAH0=;
        b=XM7y/XavkxbSTvmieY7p0ACePy61zumc0oY1Cc5C5CXJ9SXSA01vFIv9Q3Pf9zAZeo
         h31OBCnfh7tr7IwH1oF+G1LShn755EEgRoM1xIGQ756u8mkHuxrO/1S8/oU8uM6BdzCG
         CMZjXhGzp5MUMF7mmMgCYFaCN/Xu3RezPqHptHlVcMJ1ceffWV3gcC1KTvo5QlWl7o3d
         FCZcxVJwpieTp7PbBg6Tq0q1ybLOIJOzxttc+9qQ5cbDlPJtB+qaNj8Yx/ygYAdKGW7L
         IIhmRIwiy+n/s3ILGkSti9jttNHM1G7iEr1i32EEEIRxqybCUfeGjZoFt6IeKS4Bp7Jd
         2TQA==
X-Gm-Message-State: AOJu0YwyCZaKHrlgtZl8guA0g2GfKLWj9dTH44Bd+H82EbTgodIojHaP
	GtWRS88Ddlo0yZpmIzxf8gVTr5KE6d7QH2FlBRzpsKJeRh2JkhmCjB9cxUD9UrkmkE/ThcKz/11
	l
X-Google-Smtp-Source: AGHT+IE3MMy7s/KceVpl0+osxKbEUSaPxMA3zbux6Npx3Vw8SZaWADt2bnx3dI6X45hSB1uwO9zP+A==
X-Received: by 2002:a17:906:168e:b0:a38:41bf:f6f6 with SMTP id s14-20020a170906168e00b00a3841bff6f6mr3284144ejd.77.1707327621876;
        Wed, 07 Feb 2024 09:40:21 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU3evZJ50jo+0bSNFkZnb7WnQH7a1v5kbysvosyVpbX6zqd3DmZFqDLPSyJfq7ElGFRcP6ky6K7IttyTpe+NbP+gPvsMEptIhhVaEDevy8pSlXI7VZG2WK+1657Su3lnKVcxrgzmro=
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH 3/3] tools/xg: Clean up xend-style overrides for CPU policies
Date: Wed,  7 Feb 2024 17:39:57 +0000
Message-Id: <20240207173957.19811-4-alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240207173957.19811-1-alejandro.vallejo@cloud.com>
References: <20240207173957.19811-1-alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Factor out policy getters/setters from both (CPUID and MSR) policy override
functions. Additionally, use host policy rather than featureset when
preparing the cur policy, saving one hypercall and several lines of
boilerplate.

No functional change intended.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
 tools/libs/guest/xg_cpuid_x86.c | 444 +++++++++-----------------------
 1 file changed, 128 insertions(+), 316 deletions(-)

diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 5699a26b946e..cee0be80ba5b 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -36,6 +36,34 @@ enum {
 #define bitmaskof(idx)      (1u << ((idx) & 31))
 #define featureword_of(idx) ((idx) >> 5)
 
+static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
+{
+    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
+    int rc;
+
+    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
+                                    policy->nr_leaves, &err_leaf, &err_subleaf);
+    if ( rc )
+    {
+        if ( err_leaf != -1 )
+            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
+                  err_leaf, err_subleaf, -rc, strerror(-rc));
+        return rc;
+    }
+
+    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
+                                  policy->nr_msrs, &err_msr);
+    if ( rc )
+    {
+        if ( err_msr != -1 )
+            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
+                  err_msr, -rc, strerror(-rc));
+        return rc;
+    }
+
+    return 0;
+}
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps)
 {
     struct xen_sysctl sysctl = {};
@@ -260,102 +288,34 @@ static int compare_leaves(const void *l, const void *r)
     return 0;
 }
 
-static xen_cpuid_leaf_t *find_leaf(
-    xen_cpuid_leaf_t *leaves, unsigned int nr_leaves,
-    const struct xc_xend_cpuid *xend)
+static xen_cpuid_leaf_t *find_leaf(xc_cpu_policy_t *p,
+                                   const struct xc_xend_cpuid *xend)
 {
     const xen_cpuid_leaf_t key = { xend->leaf, xend->subleaf };
 
-    return bsearch(&key, leaves, nr_leaves, sizeof(*leaves), compare_leaves);
+    return bsearch(&key, p->leaves, ARRAY_SIZE(p->leaves),
+                   sizeof(*p->leaves), compare_leaves);
 }
 
-static int xc_cpuid_xend_policy(
-    xc_interface *xch, uint32_t domid, const struct xc_xend_cpuid *xend)
+static int xc_cpuid_xend_policy(xc_interface *xch, uint32_t domid,
+                                const struct xc_xend_cpuid *xend,
+                                xc_cpu_policy_t *host,
+                                xc_cpu_policy_t *def,
+                                xc_cpu_policy_t *cur)
 {
-    int rc;
-    bool hvm;
-    xc_domaininfo_t di;
-    unsigned int nr_leaves, nr_msrs;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    /*
-     * Three full policies.  The host, default for the domain type,
-     * and domain current.
-     */
-    xen_cpuid_leaf_t *host = NULL, *def = NULL, *cur = NULL;
-    unsigned int nr_host, nr_def, nr_cur;
-
-    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
-    {
-        PERROR("Failed to obtain d%d info", domid);
-        rc = -errno;
-        goto fail;
-    }
-    hvm = di.flags & XEN_DOMINF_hvm_guest;
-
-    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-    if ( rc )
-    {
-        PERROR("Failed to obtain policy info size");
-        rc = -errno;
-        goto fail;
-    }
-
-    rc = -ENOMEM;
-    if ( (host = calloc(nr_leaves, sizeof(*host))) == NULL ||
-         (def  = calloc(nr_leaves, sizeof(*def)))  == NULL ||
-         (cur  = calloc(nr_leaves, sizeof(*cur)))  == NULL )
-    {
-        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
-        goto fail;
-    }
-
-    /* Get the domain's current policy. */
-    nr_msrs = 0;
-    nr_cur = nr_leaves;
-    rc = get_domain_cpu_policy(xch, domid, &nr_cur, cur, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain d%d current policy", domid);
-        rc = -errno;
-        goto fail;
-    }
-
-    /* Get the domain type's default policy. */
-    nr_msrs = 0;
-    nr_def = nr_leaves;
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_def, def, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
-        rc = -errno;
-        goto fail;
-    }
-
-    /* Get the host policy. */
-    nr_msrs = 0;
-    nr_host = nr_leaves;
-    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
-                               &nr_host, host, &nr_msrs, NULL);
-    if ( rc )
-    {
-        PERROR("Failed to obtain host policy");
-        rc = -errno;
-        goto fail;
-    }
+    if ( !xend )
+        return 0;
 
-    rc = -EINVAL;
     for ( ; xend->leaf != XEN_CPUID_INPUT_UNUSED; ++xend )
     {
-        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, nr_cur, xend);
-        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, nr_def, xend);
-        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, nr_host, xend);
+        xen_cpuid_leaf_t *cur_leaf = find_leaf(cur, xend);
+        const xen_cpuid_leaf_t *def_leaf = find_leaf(def, xend);
+        const xen_cpuid_leaf_t *host_leaf = find_leaf(host, xend);
 
         if ( cur_leaf == NULL || def_leaf == NULL || host_leaf == NULL )
         {
             ERROR("Missing leaf %#x, subleaf %#x", xend->leaf, xend->subleaf);
-            goto fail;
+            return -EINVAL;
         }
 
         for ( unsigned int i = 0; i < ARRAY_SIZE(xend->policy); i++ )
@@ -384,7 +344,7 @@ static int xc_cpuid_xend_policy(
                 {
                     ERROR("Bad character '%c' in policy[%d] string '%s'",
                           xend->policy[i][j], i, xend->policy[i]);
-                    goto fail;
+                    return -EINVAL;
                 }
 
                 clear_bit(31 - j, cur_reg);
@@ -394,25 +354,7 @@ static int xc_cpuid_xend_policy(
         }
     }
 
-    /* Feed the transformed currrent policy back up to Xen. */
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_cur, cur, 0, NULL,
-                                  &err_leaf, &err_subleaf, &err_msr);
-    if ( rc )
-    {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
-        rc = -errno;
-        goto fail;
-    }
-
-    /* Success! */
-
- fail:
-    free(cur);
-    free(def);
-    free(host);
-
-    return rc;
+    return 0;
 }
 
 static int compare_msr(const void *l, const void *r)
@@ -426,107 +368,34 @@ static int compare_msr(const void *l, const void *r)
     return lhs->idx < rhs->idx ? -1 : 1;
 }
 
-static xen_msr_entry_t *find_msr(
-    xen_msr_entry_t *msrs, unsigned int nr_msrs,
-    uint32_t index)
+static xen_msr_entry_t *find_msr(xc_cpu_policy_t *p,
+                                 uint32_t index)
 {
     const xen_msr_entry_t key = { .idx = index };
 
-    return bsearch(&key, msrs, nr_msrs, sizeof(*msrs), compare_msr);
+    return bsearch(&key, p->msrs, ARRAY_SIZE(p->msrs),
+                   sizeof(*p->msrs), compare_msr);
 }
 
-
 static int xc_msr_policy(xc_interface *xch, domid_t domid,
-                         const struct xc_msr *msr)
+                         const struct xc_msr *msr,
+                         xc_cpu_policy_t *host,
+                         xc_cpu_policy_t *def,
+                         xc_cpu_policy_t *cur)
 {
-    int rc;
-    bool hvm;
-    xc_domaininfo_t di;
-    unsigned int nr_leaves, nr_msrs;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    /*
-     * Three full policies.  The host, default for the domain type,
-     * and domain current.
-     */
-    xen_msr_entry_t *host = NULL, *def = NULL, *cur = NULL;
-    unsigned int nr_host, nr_def, nr_cur;
-
-    if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
-    {
-        PERROR("Failed to obtain d%d info", domid);
-        rc = -errno;
-        goto out;
-    }
-    hvm = di.flags & XEN_DOMINF_hvm_guest;
-
-    rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-    if ( rc )
-    {
-        PERROR("Failed to obtain policy info size");
-        rc = -errno;
-        goto out;
-    }
-
-    if ( (host = calloc(nr_msrs, sizeof(*host))) == NULL ||
-         (def  = calloc(nr_msrs, sizeof(*def)))  == NULL ||
-         (cur  = calloc(nr_msrs, sizeof(*cur)))  == NULL )
-    {
-        ERROR("Unable to allocate memory for %u CPUID leaves", nr_leaves);
-        rc = -ENOMEM;
-        goto out;
-    }
-
-    /* Get the domain's current policy. */
-    nr_leaves = 0;
-    nr_cur = nr_msrs;
-    rc = get_domain_cpu_policy(xch, domid, &nr_leaves, NULL, &nr_cur, cur);
-    if ( rc )
-    {
-        PERROR("Failed to obtain d%d current policy", domid);
-        rc = -errno;
-        goto out;
-    }
-
-    /* Get the domain type's default policy. */
-    nr_leaves = 0;
-    nr_def = nr_msrs;
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_leaves, NULL, &nr_def, def);
-    if ( rc )
-    {
-        PERROR("Failed to obtain %s def policy", hvm ? "hvm" : "pv");
-        rc = -errno;
-        goto out;
-    }
-
-    /* Get the host policy. */
-    nr_leaves = 0;
-    nr_host = nr_msrs;
-    rc = get_system_cpu_policy(xch, XEN_SYSCTL_cpu_policy_host,
-                               &nr_leaves, NULL, &nr_host, host);
-    if ( rc )
-    {
-        PERROR("Failed to obtain host policy");
-        rc = -errno;
-        goto out;
-    }
-
     for ( ; msr->index != XC_MSR_INPUT_UNUSED; ++msr )
     {
-        xen_msr_entry_t *cur_msr = find_msr(cur, nr_cur, msr->index);
-        const xen_msr_entry_t *def_msr = find_msr(def, nr_def, msr->index);
-        const xen_msr_entry_t *host_msr = find_msr(host, nr_host, msr->index);
-        unsigned int i;
+        xen_msr_entry_t *cur_msr = find_msr(cur, msr->index);
+        const xen_msr_entry_t *def_msr = find_msr(def, msr->index);
+        const xen_msr_entry_t *host_msr = find_msr(host, msr->index);
 
         if ( cur_msr == NULL || def_msr == NULL || host_msr == NULL )
         {
             ERROR("Missing MSR %#x", msr->index);
-            rc = -ENOENT;
-            goto out;
+            return -ENOENT;
         }
 
-        for ( i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
+        for ( size_t i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
         {
             bool val;
 
@@ -542,8 +411,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
             {
                 ERROR("MSR index %#x: bad character '%c' in policy string '%s'",
                       msr->index, msr->policy[i], msr->policy);
-                rc = -EINVAL;
-                goto out;
+                return -EINVAL;
             }
 
             if ( val )
@@ -553,25 +421,7 @@ static int xc_msr_policy(xc_interface *xch, domid_t domid,
         }
     }
 
-    /* Feed the transformed policy back up to Xen. */
-    rc = xc_set_domain_cpu_policy(xch, domid, 0, NULL, nr_cur, cur,
-                                  &err_leaf, &err_subleaf, &err_msr);
-    if ( rc )
-    {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
-        rc = -errno;
-        goto out;
-    }
-
-    /* Success! */
-
- out:
-    free(cur);
-    free(def);
-    free(host);
-
-    return rc;
+    return 0;
 }
 
 int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
@@ -583,14 +433,16 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     int rc;
     bool hvm;
     xc_domaininfo_t di;
-    struct xc_cpu_policy *p = xc_cpu_policy_init();
-    unsigned int i, nr_leaves = ARRAY_SIZE(p->leaves), nr_msrs = 0;
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
-    uint32_t len = ARRAY_SIZE(host_featureset);
 
-    if ( !p )
-        return -ENOMEM;
+    struct xc_cpu_policy *host = xc_cpu_policy_init();
+    struct xc_cpu_policy *def = xc_cpu_policy_init();
+    struct xc_cpu_policy *cur = xc_cpu_policy_init();
+
+    if ( !host || !def || !cur )
+    {
+        rc = -ENOMEM;
+        goto out;
+    }
 
     if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
     {
@@ -600,21 +452,19 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
     }
     hvm = di.flags & XEN_DOMINF_hvm_guest;
 
-    /* Get the host policy. */
-    rc = xc_get_cpu_featureset(xch, XEN_SYSCTL_cpu_featureset_host,
-                               &len, host_featureset);
-    /* Tolerate "buffer too small", as we've got the bits we need. */
-    if ( rc && errno != ENOBUFS )
+    /* Get the raw host policy */
+    rc = xc_cpu_policy_get_system(xch, XEN_SYSCTL_cpu_policy_host, host);
+    if ( rc )
     {
-        PERROR("Failed to obtain host featureset");
+        PERROR("Failed to obtain host policy");
         rc = -errno;
         goto out;
     }
 
     /* Get the domain's default policy. */
-    rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-                                        : XEN_SYSCTL_cpu_policy_pv_default,
-                               &nr_leaves, p->leaves, &nr_msrs, NULL);
+    rc = xc_cpu_policy_get_system(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
+                                           : XEN_SYSCTL_cpu_policy_pv_default,
+                                  def);
     if ( rc )
     {
         PERROR("Failed to obtain %s default policy", hvm ? "hvm" : "pv");
@@ -622,14 +472,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
         goto out;
     }
 
-    rc = x86_cpuid_copy_from_buffer(&p->policy, p->leaves, nr_leaves,
-                                    &err_leaf, &err_subleaf);
-    if ( rc )
-    {
-        ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
-              err_leaf, err_subleaf, -rc, strerror(-rc));
-        goto out;
-    }
+    /* Copy the deserialised default policy to modify it */
+    memcpy(cur, def, sizeof(*cur));
 
     if ( restore )
     {
@@ -647,18 +491,18 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * - Re-enable features which have become (possibly) off by default.
          */
 
-        p->policy.basic.rdrand = test_bit(X86_FEATURE_RDRAND, host_featureset);
-        p->policy.feat.hle = test_bit(X86_FEATURE_HLE, host_featureset);
-        p->policy.feat.rtm = test_bit(X86_FEATURE_RTM, host_featureset);
+        cur->policy.basic.rdrand = host->policy.basic.rdrand;
+        cur->policy.feat.hle = host->policy.feat.hle;
+        cur->policy.feat.rtm = host->policy.feat.rtm;
 
         if ( hvm )
         {
-            p->policy.feat.mpx = test_bit(X86_FEATURE_MPX, host_featureset);
+            cur->policy.feat.mpx = host->policy.feat.mpx;
         }
 
-        p->policy.basic.max_leaf = min(p->policy.basic.max_leaf, 0xdu);
-        p->policy.feat.max_subleaf = 0;
-        p->policy.extd.max_leaf = min(p->policy.extd.max_leaf, 0x8000001c);
+        cur->policy.basic.max_leaf = min(cur->policy.basic.max_leaf, 0xdu);
+        cur->policy.feat.max_subleaf = 0;
+        cur->policy.extd.max_leaf = min(cur->policy.extd.max_leaf, 0x8000001c);
     }
 
     if ( featureset )
@@ -666,7 +510,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
         uint32_t disabled_features[FEATURESET_NR_ENTRIES],
             feat[FEATURESET_NR_ENTRIES] = {};
         static const uint32_t deep_features[] = INIT_DEEP_FEATURES;
-        unsigned int i, b;
+        unsigned int b;
 
         /*
          * The user supplied featureset may be shorter or longer than
@@ -677,14 +521,14 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
 
         /* Check for truncated set bits. */
         rc = -EOPNOTSUPP;
-        for ( i = user_len; i < nr_features; ++i )
+        for ( size_t i = user_len; i < nr_features; ++i )
             if ( featureset[i] != 0 )
                 goto out;
 
         memcpy(feat, featureset, sizeof(*featureset) * user_len);
 
         /* Disable deep dependencies of disabled features. */
-        for ( i = 0; i < ARRAY_SIZE(disabled_features); ++i )
+        for ( size_t i = 0; i < ARRAY_SIZE(disabled_features); ++i )
             disabled_features[i] = ~feat[i] & deep_features[i];
 
         for ( b = 0; b < sizeof(disabled_features) * CHAR_BIT; ++b )
@@ -695,24 +539,24 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
                  !(dfs = x86_cpu_policy_lookup_deep_deps(b)) )
                 continue;
 
-            for ( i = 0; i < ARRAY_SIZE(disabled_features); ++i )
+            for ( size_t i = 0; i < ARRAY_SIZE(disabled_features); ++i )
             {
                 feat[i] &= ~dfs[i];
                 disabled_features[i] &= ~dfs[i];
             }
         }
 
-        x86_cpu_featureset_to_policy(feat, &p->policy);
+        x86_cpu_featureset_to_policy(feat, &cur->policy);
     }
     else
     {
-        p->policy.extd.itsc = itsc;
+        cur->policy.extd.itsc = itsc;
 
         if ( hvm )
         {
-            p->policy.basic.pae = pae;
-            p->policy.basic.vmx = nested_virt;
-            p->policy.extd.svm = nested_virt;
+            cur->policy.basic.pae = pae;
+            cur->policy.basic.vmx = nested_virt;
+            cur->policy.extd.svm = nested_virt;
         }
     }
 
@@ -722,8 +566,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * On hardware without CPUID Faulting, PV guests see real topology.
          * As a consequence, they also need to see the host htt/cmp fields.
          */
-        p->policy.basic.htt       = test_bit(X86_FEATURE_HTT, host_featureset);
-        p->policy.extd.cmp_legacy = test_bit(X86_FEATURE_CMP_LEGACY, host_featureset);
+        cur->policy.basic.htt       = host->policy.basic.htt;
+        cur->policy.extd.cmp_legacy = host->policy.extd.cmp_legacy;
     }
     else
     {
@@ -731,28 +575,28 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
          * Topology for HVM guests is entirely controlled by Xen.  For now, we
          * hardcode APIC_ID = vcpu_id * 2 to give the illusion of no SMT.
          */
-        p->policy.basic.htt = true;
-        p->policy.extd.cmp_legacy = false;
+        cur->policy.basic.htt = true;
+        cur->policy.extd.cmp_legacy = false;
 
         /*
          * Leaf 1 EBX[23:16] is Maximum Logical Processors Per Package.
          * Update to reflect vLAPIC_ID = vCPU_ID * 2, but make sure to avoid
          * overflow.
          */
-        if ( !p->policy.basic.lppp )
-            p->policy.basic.lppp = 2;
-        else if ( !(p->policy.basic.lppp & 0x80) )
-            p->policy.basic.lppp *= 2;
+        if ( !cur->policy.basic.lppp )
+            cur->policy.basic.lppp = 2;
+        else if ( !(cur->policy.basic.lppp & 0x80) )
+            cur->policy.basic.lppp *= 2;
 
-        switch ( p->policy.x86_vendor )
+        switch ( cur->policy.x86_vendor )
         {
         case X86_VENDOR_INTEL:
-            for ( i = 0; (p->policy.cache.subleaf[i].type &&
-                          i < ARRAY_SIZE(p->policy.cache.raw)); ++i )
+            for ( size_t i = 0; (cur->policy.cache.subleaf[i].type &&
+                                i < ARRAY_SIZE(cur->policy.cache.raw)); ++i )
             {
-                p->policy.cache.subleaf[i].cores_per_package =
-                    (p->policy.cache.subleaf[i].cores_per_package << 1) | 1;
-                p->policy.cache.subleaf[i].threads_per_cache = 0;
+                cur->policy.cache.subleaf[i].cores_per_package =
+                    (cur->policy.cache.subleaf[i].cores_per_package << 1) | 1;
+                cur->policy.cache.subleaf[i].threads_per_cache = 0;
             }
             break;
 
@@ -772,49 +616,45 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t domid, bool restore,
              * apic_id_size values greater than 7.  Limit the value to
              * 7 for now.
              */
-            if ( p->policy.extd.nc < 0x7f )
+            if ( cur->policy.extd.nc < 0x7f )
             {
-                if ( p->policy.extd.apic_id_size != 0 && p->policy.extd.apic_id_size < 0x7 )
-                    p->policy.extd.apic_id_size++;
+                if ( cur->policy.extd.apic_id_size != 0 && cur->policy.extd.apic_id_size < 0x7 )
+                    cur->policy.extd.apic_id_size++;
 
-                p->policy.extd.nc = (p->policy.extd.nc << 1) | 1;
+                cur->policy.extd.nc = (cur->policy.extd.nc << 1) | 1;
             }
             break;
         }
     }
 
-    nr_leaves = ARRAY_SIZE(p->leaves);
-    rc = x86_cpuid_copy_to_buffer(&p->policy, p->leaves, &nr_leaves);
-    if ( rc )
+    if ( xend || msr )
     {
-        ERROR("Failed to serialise CPUID (%d = %s)", -rc, strerror(-rc));
-        goto out;
+        // The overrides are over the serialised form of the policy
+        if ( (rc = xc_cpu_policy_serialise(xch, cur)) )
+            goto out;
+
+        if ( (rc = xc_cpuid_xend_policy(xch, domid, xend, host, def, cur)) )
+            goto out;
+        if ( (rc = xc_msr_policy(xch, domid, msr, host, def, cur)) )
+            goto out;
+
+        if ( (rc = deserialize_policy(xch, cur)) )
+            goto out;
     }
 
-    rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, p->leaves, 0, NULL,
-                                  &err_leaf, &err_subleaf, &err_msr);
+    rc = xc_cpu_policy_set_domain(xch, domid, cur);
     if ( rc )
     {
-        PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr %#x)",
-               domid, err_leaf, err_subleaf, err_msr);
         rc = -errno;
         goto out;
     }
 
-    if ( xend && (rc = xc_cpuid_xend_policy(xch, domid, xend)) )
-        goto out;
-
-    if ( msr )
-    {
-        rc = xc_msr_policy(xch, domid, msr);
-        if ( rc )
-            goto out;
-    }
-
     rc = 0;
 
 out:
-    xc_cpu_policy_destroy(p);
+    xc_cpu_policy_destroy(def);
+    xc_cpu_policy_destroy(host);
+    xc_cpu_policy_destroy(cur);
 
     return rc;
 }
@@ -834,34 +674,6 @@ void xc_cpu_policy_destroy(xc_cpu_policy_t *policy)
     }
 }
 
-static int deserialize_policy(xc_interface *xch, xc_cpu_policy_t *policy)
-{
-    uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-    int rc;
-
-    rc = x86_cpuid_copy_from_buffer(&policy->policy, policy->leaves,
-                                    policy->nr_leaves, &err_leaf, &err_subleaf);
-    if ( rc )
-    {
-        if ( err_leaf != -1 )
-            ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = %s)",
-                  err_leaf, err_subleaf, -rc, strerror(-rc));
-        return rc;
-    }
-
-    rc = x86_msr_copy_from_buffer(&policy->policy, policy->msrs,
-                                  policy->nr_msrs, &err_msr);
-    if ( rc )
-    {
-        if ( err_msr != -1 )
-            ERROR("Failed to deserialise MSR (err MSR %#x) (%d = %s)",
-                  err_msr, -rc, strerror(-rc));
-        return rc;
-    }
-
-    return 0;
-}
-
 int xc_cpu_policy_get_system(xc_interface *xch, unsigned int policy_idx,
                              xc_cpu_policy_t *policy)
 {
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 17:40:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 17:40:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677847.1054742 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXluL-0005iE-FU; Wed, 07 Feb 2024 17:40:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677847.1054742; Wed, 07 Feb 2024 17:40:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXluL-0005i7-CT; Wed, 07 Feb 2024 17:40:25 +0000
Received: by outflank-mailman (input) for mailman id 677847;
 Wed, 07 Feb 2024 17:40:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0q5t=JQ=cloud.com=alejandro.vallejo@srs-se1.protection.inumbo.net>)
 id 1rXluJ-0005Tj-IS
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 17:40:23 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f762fe44-c5df-11ee-98f5-efadbce2ee36;
 Wed, 07 Feb 2024 18:40:21 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a2a17f3217aso110280766b.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 09:40:21 -0800 (PST)
Received: from EMEAENGAAD19049.citrite.net
 (default-46-102-197-194.interdsl.co.uk. [46.102.197.194])
 by smtp.gmail.com with ESMTPSA id
 op3-20020a170906bce300b00a37669280d1sm976355ejb.141.2024.02.07.09.40.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 09:40:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f762fe44-c5df-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707327621; x=1707932421; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+hjQ03fb4gTy1nB5aYDQXgvwccHLoP4GL/zVeSr+oE4=;
        b=FWuphzI+t9EusMvTOLYSbPCBE0rojA0TSYYslW1S2FTkw7z459xTXBW0zQDljPPZuL
         N6PsmJ5N8V0L//WGixo1Vrr9DD7j6gNs/kIuPAuVVcivS3pJfxdMn6/xgC8KltshQMdE
         pNrwN5Y5j2QoPzDE+T/D+BcWjX/AaL+voI1KM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707327621; x=1707932421;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=+hjQ03fb4gTy1nB5aYDQXgvwccHLoP4GL/zVeSr+oE4=;
        b=MtcEy8iE0XuU8rZWhTC6SyWIwPmHa+bxwX3KT85cNca65hxtIBu/3KD4K1jB/H0LH7
         kiqnjuMzK5ZI3GuW42P1B9/S0GgZsi8cxkHacTfdrwEFsaFI2kwm6oMGmvEWDKSe7sh2
         m6ZOSPsjFP/6t+IrVl0VrA5TqNaTjNK1ArFRJ1uln0bhDDaKg+mFitDaVB+axDXxOU1y
         6G9WKZGbklHAMH7mANnTfXt3xrHr8C3fyUh+zJJxekRJNyLqXi87+tCXj8khAEgQszDm
         ZDgXAZGMJ8+ZEbZ5OSt3IDjZixSIGGWs5gA801tfDSpmdG9ekOn6v3Ywf8JYr78sFled
         ef5w==
X-Gm-Message-State: AOJu0Yweqssvhyy7pr0nN7bG/h5YTqTMV7Q5NGokLcInZ1ynpw/0X34l
	r+XOZXIFRpoqJrlmHz7+W7qXCNxCPyfedAazTVx53EbywyCUKF/dRiRrx0+xFSsKj2x+4au6/ao
	s
X-Google-Smtp-Source: AGHT+IHXF4iCI5cUyuRNUfpzz5Nygg8GIH0vsQ0EZmE3VpuusshmPnBXuPJdBt5m9k7Mcz+oFVS0XQ==
X-Received: by 2002:a17:906:1c5:b0:a35:103e:3614 with SMTP id 5-20020a17090601c500b00a35103e3614mr4214546ejj.52.1707327620723;
        Wed, 07 Feb 2024 09:40:20 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXAS9JeEvannetqUxhv+QgmoRsEdjhaEIVOAi6+SmbdeoIyM4R9mh2xgAQBkALvkDnr2n63Q83ejxNypl8yn5tIL0/CsOQh5VtR9zVzpmyTTtTWsxO+C5/DMwMb2J4TO3qp1DJaIKW/bTAqi90aHYgiJh1nZvqxa1CjWcr4Ca8brIS+dYe6vENEMMGfizXYa91EmZrSDslDEfu/BexVP3EWe6yNwrxIxz1yoA==
From: Alejandro Vallejo <alejandro.vallejo@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Alejandro Vallejo <alejandro.vallejo@cloud.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>
Subject: [PATCH 1/3] tools/xg: Move xc_cpu_policy_t to xenguest.h
Date: Wed,  7 Feb 2024 17:39:55 +0000
Message-Id: <20240207173957.19811-2-alejandro.vallejo@cloud.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20240207173957.19811-1-alejandro.vallejo@cloud.com>
References: <20240207173957.19811-1-alejandro.vallejo@cloud.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This enables a set of follow-up simplifications in the toolstack.

No functional change.

Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
---
 tools/include/xenguest.h             |  8 +++++++-
 tools/libs/guest/xg_private.h        | 10 ----------
 xen/include/xen/lib/x86/cpu-policy.h |  6 ++++--
 3 files changed, 11 insertions(+), 13 deletions(-)

diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
index e01f494b772a..4e9078fdee4d 100644
--- a/tools/include/xenguest.h
+++ b/tools/include/xenguest.h
@@ -784,7 +784,13 @@ xen_pfn_t *xc_map_m2p(xc_interface *xch,
                       unsigned long *mfn0);
 
 #if defined(__i386__) || defined(__x86_64__)
-typedef struct xc_cpu_policy xc_cpu_policy_t;
+#include <xen/lib/x86/cpu-policy.h>
+
+typedef struct xc_cpu_policy {
+    struct cpu_policy policy;
+    xen_cpuid_leaf_t leaves[CPUID_MAX_SERIALISED_LEAVES];
+    xen_msr_entry_t msrs[MSR_MAX_SERIALISED_ENTRIES];
+} xc_cpu_policy_t;
 
 /* Create and free a xc_cpu_policy object. */
 xc_cpu_policy_t *xc_cpu_policy_init(void);
diff --git a/tools/libs/guest/xg_private.h b/tools/libs/guest/xg_private.h
index d73947094f2e..d1940f1ea482 100644
--- a/tools/libs/guest/xg_private.h
+++ b/tools/libs/guest/xg_private.h
@@ -170,14 +170,4 @@ int pin_table(xc_interface *xch, unsigned int type, unsigned long mfn,
 #define M2P_SIZE(_m)    ROUNDUP(((_m) * sizeof(xen_pfn_t)), M2P_SHIFT)
 #define M2P_CHUNKS(_m)  (M2P_SIZE((_m)) >> M2P_SHIFT)
 
-#if defined(__x86_64__) || defined(__i386__)
-#include <xen/lib/x86/cpu-policy.h>
-
-struct xc_cpu_policy {
-    struct cpu_policy policy;
-    xen_cpuid_leaf_t leaves[CPUID_MAX_SERIALISED_LEAVES];
-    xen_msr_entry_t msrs[MSR_MAX_SERIALISED_ENTRIES];
-};
-#endif /* x86 */
-
 #endif /* XG_PRIVATE_H */
diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h
index d5e447e9dc06..0cd454dfc0b6 100644
--- a/xen/include/xen/lib/x86/cpu-policy.h
+++ b/xen/include/xen/lib/x86/cpu-policy.h
@@ -85,6 +85,8 @@ unsigned int x86_cpuid_lookup_vendor(uint32_t ebx, uint32_t ecx, uint32_t edx);
  */
 const char *x86_cpuid_vendor_to_str(unsigned int vendor);
 
+#define XENLIB_MAX(x, y) ((x) < (y) ? (y) : (x))
+
 #define CPUID_GUEST_NR_BASIC      (0xdu + 1)
 #define CPUID_GUEST_NR_CACHE      (5u + 1)
 #define CPUID_GUEST_NR_FEAT       (2u + 1)
@@ -92,8 +94,8 @@ const char *x86_cpuid_vendor_to_str(unsigned int vendor);
 #define CPUID_GUEST_NR_XSTATE     (62u + 1)
 #define CPUID_GUEST_NR_EXTD_INTEL (0x8u + 1)
 #define CPUID_GUEST_NR_EXTD_AMD   (0x21u + 1)
-#define CPUID_GUEST_NR_EXTD       MAX(CPUID_GUEST_NR_EXTD_INTEL, \
-                                      CPUID_GUEST_NR_EXTD_AMD)
+#define CPUID_GUEST_NR_EXTD       XENLIB_MAX(CPUID_GUEST_NR_EXTD_INTEL, \
+                                             CPUID_GUEST_NR_EXTD_AMD)
 
 /*
  * Maximum number of leaves a struct cpu_policy turns into when serialised for
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 19:02:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 19:02:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677878.1054776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXnBu-0000te-2n; Wed, 07 Feb 2024 19:02:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677878.1054776; Wed, 07 Feb 2024 19:02:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXnBu-0000tX-0H; Wed, 07 Feb 2024 19:02:38 +0000
Received: by outflank-mailman (input) for mailman id 677878;
 Wed, 07 Feb 2024 19:02:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oL7s=JQ=rabbit.lu=slack@srs-se1.protection.inumbo.net>)
 id 1rXnBs-0000tR-Tg
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 19:02:37 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 743332e9-c5eb-11ee-8a4a-1f161083a0e0;
 Wed, 07 Feb 2024 20:02:35 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-4101565d20bso7849855e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 11:02:35 -0800 (PST)
Received: from localhost.localdomain (82-64-138-184.subs.proxad.net.
 [82.64.138.184]) by smtp.googlemail.com with ESMTPSA id
 h13-20020a05600c314d00b0040fdc7f4fcdsm6226198wmo.4.2024.02.07.11.02.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 11:02:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 743332e9-c5eb-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=rabbit-lu.20230601.gappssmtp.com; s=20230601; t=1707332555; x=1707937355; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=mdaTCdI8ySFmu0E5W0G1WTxxRBPa6Pci0/ksty6Vs5I=;
        b=kgZaPz0LHeePYMcHpUHMZtV83hlGrWjIhf797dI/kS1knPZzBWehfCoeXkRDNQFGnc
         Ae8r7BgGyDlJ7RgDWtkJdWktd1zzRaOhFjFtK67Z8OpnIC4BW/ZaK0hJQVcWl8lioq2o
         pzTj4IhTH8jAjkELsUIDgivZYwxkRLhVQxJbnS5EazQ2op9Of+S/5mlYp7WrpQMmvOOA
         jA8ywzWEal11Qk47vqS4R5zw2Q3GxQvTFPR4eAjFUn6g0pAj7YiZTjHzj+/bGxUscAno
         sihy/P2pN6NGPaHm2XQa1JVMLFrD7LducrAUeHcEynu/oqxeWx397zHzqICoRM4wK5Sv
         KlLQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707332555; x=1707937355;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=mdaTCdI8ySFmu0E5W0G1WTxxRBPa6Pci0/ksty6Vs5I=;
        b=YD1NIO0ha9PXvLjWTIZTedLGNSuru8bIb/Nmg5LpdJ3eOIOH5PzeL72XafTlMRrksp
         t6esxSUKHX1HtzLX5Lnr8ddxYnWvHklxgWX3W6lCjs3RBSw+xOhpLLAqGTA4XmzQ6++9
         OCZAJ2UP8GJvQuxN+9lcBXcy4OItF50WNk7EKjDczcbGbWZHLSzExb/2O2xi41SY7NGR
         FGryN0KmwHDHol2H3mgMwZLgHRfUTatnZ7/LbmqqD4clZtATpIwQH6TLbRS9BIKbhBJV
         hcz71t4ASQ9vETTrePSHyJnu2C03w5YCf0++wZY02Go4jC/u63MiM6ahnXa3F3oyTl2+
         CrPw==
X-Gm-Message-State: AOJu0YwlZuxFR/d6sTnwf2E2YMGOm2XVXg372fijweCLACXp+2XhZ/Tr
	7OaqLTNLus/o6w1rQUrNa8ppYtetpD0hfSlT4dCUaZWgzihCaDQSBlAehf/vTdZUMegxElLahlz
	rvWI=
X-Google-Smtp-Source: AGHT+IF7eD5p5sY0u42jubYJxmZ6uK7J9JFWJ+gQzZsPSpO0aJ8KqByWvwfkeqnif513snA43+OGzg==
X-Received: by 2002:a05:600c:310d:b0:40f:d162:944c with SMTP id g13-20020a05600c310d00b0040fd162944cmr5581668wmo.31.1707332554662;
        Wed, 07 Feb 2024 11:02:34 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX6E0vfHCynkqxv+VCQGiERZIf7DKowp3PzAkd48ZDkn2wJvD8KNrswi5/SQL8DP3zScyGd80phSmCCjA+4f1EzbQbSC7MfTeF7OZAcObGPsDi87udr
From: =?UTF-8?q?zithro=20/=20Cyril=20R=C3=A9bert?= <slack@rabbit.lu>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Cyril=20R=C3=A9bert?= <slack@rabbit.lu>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v3] tools/xentop: add option to display dom0 first
Date: Wed,  7 Feb 2024 20:02:00 +0100
Message-Id: <92ef4d230e05970e1d685b03125fce44adc55010.1707331801.git.slack@rabbit.lu>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Cyril Rébert <slack@rabbit.lu>

Add a command line option to xentop to be able to display dom0 first, on top of the list.
This is unconditional, so sorting domains with the S option will also ignore dom0.

Signed-off-by: Cyril Rébert (zithro) <slack@rabbit.lu>
---
Changes in v3:
(none, just reformatting patch correctly ... hopefully ?!)
---
Changes in v2:
- bug fix
- add documentation
---
 docs/man/xentop.1.pod |  6 +++++-
 tools/xentop/xentop.c | 30 +++++++++++++++++++++++++++---
 2 files changed, 32 insertions(+), 4 deletions(-)

diff --git a/docs/man/xentop.1.pod b/docs/man/xentop.1.pod
index 126f43d2f3..593a484ce7 100644
--- a/docs/man/xentop.1.pod
+++ b/docs/man/xentop.1.pod
@@ -5,7 +5,7 @@ xentop - displays real-time information about a Xen system and domains
 =head1 SYNOPSIS
 
 B<xentop> [B<-h>] [B<-V>] [B<-d>SECONDS] [B<-n>] [B<-r>] [B<-v>] [B<-f>]
-[B<-b>] [B<-i>ITERATIONS]
+[B<-b>] [B<-i>ITERATIONS] [B<-z>]
 
 =head1 DESCRIPTION
 
@@ -57,6 +57,10 @@ output data in batch mode (to stdout)
 
 maximum number of iterations xentop should produce before ending
 
+=item B<-z>, B<--dom0-first>
+
+display dom0 first, ignoring interactive sorting
+
 =back
 
 =head1 INTERACTIVE COMMANDS
diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
index 545bd5e96d..0a2fab7f15 100644
--- a/tools/xentop/xentop.c
+++ b/tools/xentop/xentop.c
@@ -211,6 +211,7 @@ int show_networks = 0;
 int show_vbds = 0;
 int repeat_header = 0;
 int show_full_name = 0;
+int dom0_first = 0;
 #define PROMPT_VAL_LEN 80
 const char *prompt = NULL;
 char prompt_val[PROMPT_VAL_LEN];
@@ -240,6 +241,7 @@ static void usage(const char *program)
 	       "-b, --batch	     output in batch mode, no user input accepted\n"
 	       "-i, --iterations     number of iterations before exiting\n"
 	       "-f, --full-name      output the full domain name (not truncated)\n"
+	       "-z, --dom0-first     display dom0 first (ignore sorting)\n"
 	       "\n" XENTOP_BUGSTO,
 	       program);
 	return;
@@ -1163,6 +1165,8 @@ static void top(void)
 {
 	xenstat_domain **domains;
 	unsigned int i, num_domains = 0;
+	int dom0_index = -1;
+	int sort_start = 0, sort_count = 0;
 
 	/* Now get the node information */
 	if (prev_node != NULL)
@@ -1183,11 +1187,27 @@ static void top(void)
 	if(domains == NULL)
 		fail("Failed to allocate memory\n");
 
-	for (i=0; i < num_domains; i++)
+	for (i=0; i < num_domains; i++) {
 		domains[i] = xenstat_node_domain_by_index(cur_node, i);
+		if ( strcmp(xenstat_domain_name(domains[i]), "Domain-0") == 0 )
+			dom0_index = i;
+	}
+
+	/* Handle dom0 position, not for dom0-less */
+	if ( dom0_first == 1 && dom0_index != -1 ){
+		/* if dom0 is not first in domains, swap it there */
+		if ( dom0_index != 0 ){
+			xenstat_domain *tmp;
+			tmp = domains[0];
+			domains[0] = domains[dom0_index];
+			domains[dom0_index] = tmp;
+		}
+		sort_start = 1;
+		sort_count = 1;
+	}
 
 	/* Sort */
-	qsort(domains, num_domains, sizeof(xenstat_domain *),
+	qsort((domains+sort_start), (num_domains-sort_count), sizeof(xenstat_domain *),
 	      (int(*)(const void *, const void *))compare_domains);
 
 	if(first_domain_index >= num_domains)
@@ -1242,9 +1262,10 @@ int main(int argc, char **argv)
 		{ "batch",	   no_argument,	      NULL, 'b' },
 		{ "iterations",	   required_argument, NULL, 'i' },
 		{ "full-name",     no_argument,       NULL, 'f' },
+		{ "dom0-first",    no_argument,       NULL, 'z' },
 		{ 0, 0, 0, 0 },
 	};
-	const char *sopts = "hVnxrvd:bi:f";
+	const char *sopts = "hVnxrvd:bi:fz";
 
 	if (atexit(cleanup) != 0)
 		fail("Failed to install cleanup handler.\n");
@@ -1286,6 +1307,9 @@ int main(int argc, char **argv)
 		case 'f':
 			show_full_name = 1;
 			break;
+		case 'z':
+			dom0_first = 1;
+			break;
 		}
 	}
 
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 19:10:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 19:10:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677883.1054787 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXnJp-0002eN-0U; Wed, 07 Feb 2024 19:10:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677883.1054787; Wed, 07 Feb 2024 19:10:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXnJo-0002eG-TK; Wed, 07 Feb 2024 19:10:48 +0000
Received: by outflank-mailman (input) for mailman id 677883;
 Wed, 07 Feb 2024 19:10:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXnJn-0002e4-6S; Wed, 07 Feb 2024 19:10:47 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXnJm-0000YS-MI; Wed, 07 Feb 2024 19:10:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXnJm-0004CH-Ba; Wed, 07 Feb 2024 19:10:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXnJm-0002oD-B9; Wed, 07 Feb 2024 19:10:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=XJuTAcnqYhPoBUUr+hAnO9klcYSRsy+RTNaoy6gUXdk=; b=HDCDkSJ3wAC/1Br89B8Y1O3F/K
	zpzw8JDIjx73BDPVtL9m6O4RcuxWNofF6qG4eWsNWgECOuDsGYdMNvihkVibNNKMwwljFczZwCnzk
	vF5Kjr+digjxr40xJuABgWasd92y9lV2Y8EtHCYTDe84uk6WEGIEs5XNf6E5ZADwwiAg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184614-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184614: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-shadow:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=6d280f4d760e3bcb4a8df302afebf085b65ec982
X-Osstest-Versions-That:
    linux=99bd3cb0d12e85d5114425353552121ec8f93adc
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 07 Feb 2024 19:10:46 +0000

flight 184614 linux-linus real [real]
flight 184618 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184614/
http://logs.test-lab.xenproject.org/osstest/logs/184618/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-shadow 20 guest-localmigrate/x10 fail pass in 184618-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184601
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184601
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184601
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184601
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184601
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184601
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184601
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184601
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                6d280f4d760e3bcb4a8df302afebf085b65ec982
baseline version:
 linux                99bd3cb0d12e85d5114425353552121ec8f93adc

Last test of basis   184601  2024-02-06 08:13:56 Z    1 days
Testing same since   184614  2024-02-07 08:42:21 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Boris Burkov <boris@bur.io>
  David Sterba <dsterba@suse.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Qu Wenruo <wqu@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   fail    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   99bd3cb0d12e..6d280f4d760e  6d280f4d760e3bcb4a8df302afebf085b65ec982 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 21:17:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 21:17:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677891.1054796 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXpHj-0007OK-O0; Wed, 07 Feb 2024 21:16:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677891.1054796; Wed, 07 Feb 2024 21:16:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXpHj-0007OD-LD; Wed, 07 Feb 2024 21:16:47 +0000
Received: by outflank-mailman (input) for mailman id 677891;
 Wed, 07 Feb 2024 21:16:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXpHh-0007O3-Vz; Wed, 07 Feb 2024 21:16:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXpHh-0002kF-Of; Wed, 07 Feb 2024 21:16:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXpHh-0008JG-Ga; Wed, 07 Feb 2024 21:16:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXpHh-0004bi-G5; Wed, 07 Feb 2024 21:16:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=sh99rsEOK9gifGJuArcmFiJY3tRRQFGSHgXG37x4WbM=; b=BQj5gTWeDI0gU1lkrIR0D1Zci1
	COJLyVUMeH0Y7TKKrbZXxHkWRoNX1zEZXOQZIJVexAS72P/XsMcudCm0gQdq8YVHzBk8ql06MqodI
	802SUZEJ3ihDp3F8WSnBZHAR6yD2QXKkUkHt+HavrBgAJGwPzUqraX7ZfcDIsmQaWUEQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184619-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184619: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=4d1f0babe20cf757897fa43c399fd79bb6aa8a30
X-Osstest-Versions-That:
    ovmf=1d0b95f6457d225c5108302a9da74b4ed7aa5a38
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 07 Feb 2024 21:16:45 +0000

flight 184619 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184619/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 4d1f0babe20cf757897fa43c399fd79bb6aa8a30
baseline version:
 ovmf                 1d0b95f6457d225c5108302a9da74b4ed7aa5a38

Last test of basis   184609  2024-02-06 20:43:02 Z    1 days
Testing same since   184619  2024-02-07 19:12:58 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michael D Kinney <michael.d.kinney@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   1d0b95f645..4d1f0babe2  4d1f0babe20cf757897fa43c399fd79bb6aa8a30 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 07 22:05:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 22:05:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677898.1054807 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXq2L-0004QV-C1; Wed, 07 Feb 2024 22:04:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677898.1054807; Wed, 07 Feb 2024 22:04:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXq2L-0004QO-7s; Wed, 07 Feb 2024 22:04:57 +0000
Received: by outflank-mailman (input) for mailman id 677898;
 Wed, 07 Feb 2024 22:04:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Ti/e=JQ=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1rXq2J-0004QI-4j
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 22:04:55 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ec1ced47-c604-11ee-8a4a-1f161083a0e0;
 Wed, 07 Feb 2024 23:04:54 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-40fe79f1aaaso10356545e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 14:04:54 -0800 (PST)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 bi26-20020a05600c3d9a00b004102da467bcsm537972wmb.25.2024.02.07.14.04.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 07 Feb 2024 14:04:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec1ced47-c604-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707343493; x=1707948293; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=cjxg1rDFUbClUNBZj/Vc2i7YGFGmg/zn3O672UciEs0=;
        b=lpY7uVpzG/3a1wCxa+pvsnl7DJ6M+xuSa1PLLqq7zZrfwvKXos3odyxFa5RdAnmD+x
         QI2BynoQmJ7GqY+ry4ReJ0bcPihZZBIILG8vngapQAxItOPUXiNYwaWrq22Oy/glmILv
         51WF1xOODIY5hEDxXyD1BVubsZZLMT3VsWfYcfM5TvdcixjS9qyTDa/ZmaMCr+RwzuZH
         qwe8cZN1mcL4WNPb4znfY7qdxoodo35egywbwZnu7y0fD577lCoj5n2uL9IVmXonHyOI
         TY2Njt4JMYR5yocOvATNNbII+ZLCFc5ZMvGnnGf/zW5VeN+vFmahHnRHBCZiFBRJY+z0
         FHhg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707343493; x=1707948293;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=cjxg1rDFUbClUNBZj/Vc2i7YGFGmg/zn3O672UciEs0=;
        b=rmyetecoeFez/98opYs/E/quSVW51eKn7oMrV/IlnrcuK/rXC1fGGq3dPMVPKV2npo
         ici+gzENi/fKbvBVHrpNKBHMBzsVg80IyJacaxACtCCq1UimTR8Q+8ZD2jrwJDPnuGo1
         O+n8CA2kww1aRNPSpJdWvtBAKB+KllpkiCXPquBSqPAf5hU+iGrAjhudKQe18isBY3Vw
         X5Ytwxp01OSNV7/Y1tlT7A4W77Aqzyi/fEVomLXuBifQV29DK+qIon/VdgcdbTOaxF5u
         VyKlJ2fT8fZo2rKrwZ13bUWmgnhOvuwr3JVtBtAmmmu8JYZiCeUSPYYFaKQfTbGrM0CC
         4RKA==
X-Gm-Message-State: AOJu0YxTPawo5X/ndgHn0HnOpF1ek59tMHbSrLRQgLAdCCR206dWc+tL
	KkbTgdfY4o1HnT3iCkfm4wV0zPBn3aF0vYs9dpyQkaK8ziIfzFn3UGJ7dyOalr0=
X-Google-Smtp-Source: AGHT+IFP1WynVWTCZZauCNnn5yhzuHh7fzV2sktmgRQfxQMtndmYttuV0WDhxBNI6CxlgtagHSFr4w==
X-Received: by 2002:a05:600c:4fce:b0:40e:fc20:b574 with SMTP id o14-20020a05600c4fce00b0040efc20b574mr5429310wmq.10.1707343492923;
        Wed, 07 Feb 2024 14:04:52 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWFAe1nA3JmnYhDxl9Rw9q/5Igdc7GJEAifVjyeDCBAzvLWinq4bftp74tC47yFQDA68pUdDsKgIHLK8gmxPZhF24zhDJ6oyFyBKbFEDeqsU/sCxmwgYPQ8Ofi9XS/vJPzx1BitjY56hZ6wHeSdvwieloEKJzc3u0uUmSlL/VlT/c/AH8mqXV9iZJnSdkgirW0XMe1Leeo=
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com,
	=?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Christian Lindig <christian.lindig@citrix.com>,
	David Scott <dave@recoil.org>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH] tools/ocaml: Add missing vmtrace_buf_kb field
Date: Wed,  7 Feb 2024 22:04:12 +0000
Message-Id: <9b9909c9e93cb540b3488c784935acc2bc9e071e.1707343396.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

Add the missing `vmtrace_buf_kb` field to the OCaml bindings to match the
vm.cfg configuration, correcting an oversight from its initial introduction.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 tools/ocaml/libs/xc/xenctrl.ml      |  1 +
 tools/ocaml/libs/xc/xenctrl.mli     |  1 +
 tools/ocaml/libs/xc/xenctrl_stubs.c | 13 +++++++++++--
 3 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml
index d6c6eb73db..55923857ec 100644
--- a/tools/ocaml/libs/xc/xenctrl.ml
+++ b/tools/ocaml/libs/xc/xenctrl.ml
@@ -85,6 +85,7 @@ type domctl_create_config =
     max_grant_frames: int;
     max_maptrack_frames: int;
     max_grant_version: int;
+    vmtrace_buf_kb: int32;
     cpupool_id: int32;
     arch: arch_domainconfig;
   }
diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli
index 3bfc16edba..9b4b45db3a 100644
--- a/tools/ocaml/libs/xc/xenctrl.mli
+++ b/tools/ocaml/libs/xc/xenctrl.mli
@@ -77,6 +77,7 @@ type domctl_create_config = {
   max_grant_frames: int;
   max_maptrack_frames: int;
   max_grant_version: int;
+  vmtrace_buf_kb: int32;
   cpupool_id: int32;
   arch: arch_domainconfig;
 }
diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c b/tools/ocaml/libs/xc/xenctrl_stubs.c
index 3703f48c74..2b6d3c09df 100644
--- a/tools/ocaml/libs/xc/xenctrl_stubs.c
+++ b/tools/ocaml/libs/xc/xenctrl_stubs.c
@@ -210,10 +210,17 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 #define VAL_MAX_GRANT_FRAMES    Field(config, 6)
 #define VAL_MAX_MAPTRACK_FRAMES Field(config, 7)
 #define VAL_MAX_GRANT_VERSION   Field(config, 8)
-#define VAL_CPUPOOL_ID          Field(config, 9)
-#define VAL_ARCH                Field(config, 10)
+#define VAL_VMTRACE_BUF_KB      Field(config, 9)
+#define VAL_CPUPOOL_ID          Field(config, 10)
+#define VAL_ARCH                Field(config, 11)
 
 	uint32_t domid = Int_val(wanted_domid);
+	uint64_t vmtrace_size = Int32_val(VAL_VMTRACE_BUF_KB);
+
+	vmtrace_size = ROUNDUP(vmtrace_size << 10, XC_PAGE_SHIFT);
+	if ( vmtrace_size != (uint32_t)vmtrace_size )
+		caml_invalid_argument("vmtrace_buf_kb");
+
 	int result;
 	struct xen_domctl_createdomain cfg = {
 		.ssidref = Int32_val(VAL_SSIDREF),
@@ -223,6 +230,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 		.max_maptrack_frames = Int_val(VAL_MAX_MAPTRACK_FRAMES),
 		.grant_opts =
 		    XEN_DOMCTL_GRANT_version(Int_val(VAL_MAX_GRANT_VERSION)),
+		.vmtrace_size = vmtrace_size,
 		.cpupool_id = Int32_val(VAL_CPUPOOL_ID),
 	};
 
@@ -279,6 +287,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 
 #undef VAL_ARCH
 #undef VAL_CPUPOOL_ID
+#undef VAL_VMTRACE_BUF_KB
 #undef VAL_MAX_GRANT_VERSION
 #undef VAL_MAX_MAPTRACK_FRAMES
 #undef VAL_MAX_GRANT_FRAMES
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 07 22:21:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 07 Feb 2024 22:21:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677903.1054818 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXqHy-00073y-OG; Wed, 07 Feb 2024 22:21:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677903.1054818; Wed, 07 Feb 2024 22:21:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXqHy-00073r-Jm; Wed, 07 Feb 2024 22:21:06 +0000
Received: by outflank-mailman (input) for mailman id 677903;
 Wed, 07 Feb 2024 22:21:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t7Cu=JQ=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXqHx-00073l-FH
 for xen-devel@lists.xenproject.org; Wed, 07 Feb 2024 22:21:05 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2e20d5e8-c607-11ee-8a4a-1f161083a0e0;
 Wed, 07 Feb 2024 23:21:03 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a38291dbe65so134643766b.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 14:21:03 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 h1-20020a1709063c0100b00a372b38380csm1229755ejg.13.2024.02.07.14.21.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 14:21:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e20d5e8-c607-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707344463; x=1707949263; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cFdvdwPMjxP3oO7k0VUfwIFCu1vTihzkXLXv0Jg6QKc=;
        b=DHKeOYMiXRl/Ae/EvtsNFn1ewAxY6djZDQR4TdgVP43G1FPgIL5ugckIoMBduvU7S3
         AzTmrirga2zXjNCCWkGGtks3fLvSiPmPk4JTYfTEY8FjcOP7hUiv9zufdj0vwU6ymHkW
         u36fmu8CoRZLWZ3J7Ui+RbSP/FNv0X2sdEa2E=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707344463; x=1707949263;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cFdvdwPMjxP3oO7k0VUfwIFCu1vTihzkXLXv0Jg6QKc=;
        b=RIil/MR1kSPxBYDGmsvczBcngymrySsqI/BHQkbcMNnS8jLSbWKfm1lk0JJQu7TPvT
         8y1JqVAtsyj+fynQRUC8TIp+1ZgujRsV0Uv+0Ua+TMOxoAYZ8NRzDeY4tRGApba5B2HI
         Bwgrl7pA4F6VrT2qcBh3HhcmKuYUZdpUcKvi54ybLRC5yW9Tk1vZneH6j3WAaOFJC4J5
         w0it/ob820nVml/9lA60D38oIw2zPoapkyL/Wnkt9oA9R1wC0yV8tqFetpVYUbLm7aAR
         MsnMVsv/YXqdjDXria0gWklYYYH9D0VTdXzwcFRo7Ne9lnB3/eHLPlWvUe1uMJ5Z03nV
         DhYA==
X-Gm-Message-State: AOJu0YzuRHu3ofLPSvwwcMER2aHlcilVEEdP7Bc6KtZ2UaQFXtunvFvR
	v4odt34iPMQZyFZ34a4cVtSuw7YzF/HpJTem3IOBTRDc+DMowYR8DPetz2KRTPc=
X-Google-Smtp-Source: AGHT+IHN+UsHQfZD4D6AqtFB41/tALY1kKBZyOTuNlPYsV/sbqssHK17Tkm40a9joCpIr9CFfNEevQ==
X-Received: by 2002:a17:906:5a49:b0:a37:78f1:1302 with SMTP id my9-20020a1709065a4900b00a3778f11302mr4792880ejc.70.1707344463233;
        Wed, 07 Feb 2024 14:21:03 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUBXRFM0Eu8TNQm1BN3fk6gg5i4CXf+rWwbd+6U2wjI+dBDpHmdWkqeoUxbTq8GmpUnZOejFmUXqmGyyOHYjup3tPpgUYB1YjyCqOZBJlw+5oLCsU3KqJNOjeFNnQfImnLZnibzTNGrdOm/zCm/IxluOs5fwgRa6NXC4f/f0NNdD8GLCqbZBGv0WfE4VPGUTDt2kmdlh94WQB6dpuee2cjTAETaI7tMuifJYfohQt0KyJnQDG0JPHzMh+YINJwcYJWdI4Hf0OkpgTpP9+ViHxfA7+jNykFLKVx0yWk=
Message-ID: <507f29cf-7a07-4f3d-81e8-2236a8380b1a@citrix.com>
Date: Wed, 7 Feb 2024 22:21:02 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor
 ap2m->default_access
Content-Language: en-GB
To: George Dunlap <george.dunlap@cloud.com>, =?UTF-8?Q?Petr_Bene=C5=A1?=
 <w1benny@gmail.com>
Cc: xen-devel@lists.xenproject.org, Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
 <CA+zSX=beYAGb81zqD19YNAV3cXPFTk-7V68XYn59Eb3mGFH_Jg@mail.gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <CA+zSX=beYAGb81zqD19YNAV3cXPFTk-7V68XYn59Eb3mGFH_Jg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07/02/2024 1:18 am, George Dunlap wrote:
> On Tue, Feb 6, 2024 at 6:08 PM Petr Beneš <w1benny@gmail.com> wrote:
>> From: Petr Beneš <w1benny@gmail.com>
>>
>> This patch addresses a behavior discrepancy in the handling of altp2m views,
>> where upon the creation and subsequent EPT violation, the page access
>> permissions were incorrectly inherited from the hostp2m instead of respecting
>> the altp2m default_access.
>>
>> Previously, when a new altp2m view was established with restrictive
>> default_access permissions and activated via xc_altp2m_switch_to_view(),
>> it failed to trigger an event on the first access violation.  This behavior
>> diverged from the intended mechanism, where the altp2m's default_access
>> should dictate the initial permissions, ensuring proper event triggering on
>> access violations.
>>
>> The correction involves modifying the handling mechanism to respect the
>> altp2m view's default_access upon its activation, eliminating the need for
>> setting memory access permissions for the entire altp2m range (e.g. within
>> xen-access.c).  This change not only aligns the behavior with the expected
>> access control logic but also results in a significant performance improvement
>> by reducing the overhead associated with setting memory access permissions
>> across the altp2m range.
>>
>> Signed-off-by: Petr Beneš <w1benny@gmail.com>
> Thanks Petr, this looks like a great change.
>
> Two things:
>
> - Probably worth adjusting the comment at the top of
> p2m_altp2m_get_or_propagate to mention that you use the altp2m
> default_access when propagating from the host p2m
>
> - This represents a change in behavior, so probably at least worth a
> mention in CHANGELOG.md?

This is a bugfix to an tech preview feature.  It's not remotely close to
CHANGELOG material.

Tangent.  SUPPORT.md says tech preview on ARM, despite there being no
support in the slightest.  Patches were posted and never taken.

> Tamas, I guess this is OK from an interface compatibility point of
> view?  In theory it should always have been behaving this way.

Given the already-provided Ack, I expect that has been considered and
deemed ok.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 00:44:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 00:44:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677912.1054827 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsW8-0006K6-Kf; Thu, 08 Feb 2024 00:43:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677912.1054827; Thu, 08 Feb 2024 00:43:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsW8-0006Jz-I0; Thu, 08 Feb 2024 00:43:52 +0000
Received: by outflank-mailman (input) for mailman id 677912;
 Thu, 08 Feb 2024 00:43:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jwf=JR=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rXsW8-0006Jr-1o
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 00:43:52 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1fb4d451-c61b-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 01:43:49 +0100 (CET)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-55f0b2c79cdso1619217a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 16:43:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1fb4d451-c61b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707353029; x=1707957829; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=OIFfhiRwdxKtdU4NZEKTXEA42OX90qSEYfkjrTn0SAQ=;
        b=ZmARkC2D9J9qS/p4giaxn3rnzdFBzNJTI5/m4yCRtzNQenJrLwSzwY2zeyOSQxQYHF
         ee6QYnOz2lAClIp+z8HZMWKfWMUXGoSpjGIkDUcxXQElY5O+KhNscL4jjb8FxxVe7mNz
         a3BY0fH14jRcgaG2rpl2/gAqhV8CPXfgww88TIJ475758P9RcfLVgA0poXapXJUQQWhR
         kTQcV3bLoE9m4L3e1lswjDaaeizjuKonafji3z5F5D8ShtZGtU1Enma8IQUqfqYfhZRR
         HFG07/lz1nAO3bO2divA61+It4OvNSCsknvY4idRMwiDcy9DNxfpEhzRVlYyq8FgZogy
         d8gw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707353029; x=1707957829;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=OIFfhiRwdxKtdU4NZEKTXEA42OX90qSEYfkjrTn0SAQ=;
        b=jbTdSSYY32v9UCVR8GIBUku71g//Js8lX+8vlnkkZhVrADH+jgWp3Qig3nnMLAA+B9
         K+zz1Wdr4NsODYb0pasH0f1k7WBbyVzR46vsgV8su9brIN3hCYbcvehsO0DeQNayJWgV
         FcIwhrpwsx9OXkRW7XMTEcbAlYiLhZqjO1WGJTE9qp7g07CpOt7W74/zxcT36YYGkfCD
         kT/Dw85EPD9rxo53isZtYQ4shlzJZI3Y9iKijOKjemHho0oY2J7AVKcWHJpj1P9bkCW7
         IBkm30UZNL9YChv8QK9Rqz97mk+XiW2pzWfpNs80ipB4JIRVNx4F/wwD19ofJNsI6qtA
         ND3A==
X-Gm-Message-State: AOJu0YwLTAxS9ZR1jmUAyl/jDmhyAEkEeqadUmi8wS7jQtiSj8/17qi7
	au5Z/utCrMjv/7G8EqnYXzPEj83PfnwWIjKQrCiue2fhzXqYEAlQr7vNgsJj+J0JuMVvY0ZX1Jz
	ppSfF/asxrXZNRBHo6S/7n+nBjjs=
X-Google-Smtp-Source: AGHT+IHu//4l07Yk35hWdmp3FDyoPYisNXZc9i3MB/MMuWp9YUTprBwOdolCDrnYDcYEewXzJFGflGULGLcE1t1W/8w=
X-Received: by 2002:aa7:d047:0:b0:560:c6a8:e7dc with SMTP id
 n7-20020aa7d047000000b00560c6a8e7dcmr3169063edo.1.1707353028600; Wed, 07 Feb
 2024 16:43:48 -0800 (PST)
MIME-Version: 1.0
References: <20240205105001.24171-1-jgross@suse.com> <20240205105001.24171-6-jgross@suse.com>
In-Reply-To: <20240205105001.24171-6-jgross@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Wed, 7 Feb 2024 19:43:36 -0500
Message-ID: <CAKf6xpty1OJ5Tq+oZJCny=rbW-JszojoGAp-PoJziz+s90NWTA@mail.gmail.com>
Subject: Re: [PATCH v4 05/32] tools/xen-9pfsd: add 9pfs response generation support
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 5, 2024 at 5:50=E2=80=AFAM Juergen Gross <jgross@suse.com> wrot=
e:
>
> Add support for generation a 9pfs protocol response via a format based
> approach.
>
> Strings are stored in a per device string buffer and they are
> referenced via their offset in this buffer. This allows to avoid
> having to dynamically allocate memory for each single string.
>
> As a first user of the response handling add a generic p9_error()
> function which will be used to return any error to the client.
>
> Add all format parsing variants in order to avoid additional code churn
> later when adding the users of those variants. Prepare a special case
> for the "read" case already (format character 'D'): in order to avoid
> adding another buffer for read data support doing the read I/O directly
> into the response buffer.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 00:46:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 00:46:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677917.1054836 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsYQ-0006qj-2E; Thu, 08 Feb 2024 00:46:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677917.1054836; Thu, 08 Feb 2024 00:46:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsYP-0006qc-Vc; Thu, 08 Feb 2024 00:46:13 +0000
Received: by outflank-mailman (input) for mailman id 677917;
 Thu, 08 Feb 2024 00:46:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jwf=JR=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rXsYP-0006qW-9j
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 00:46:13 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 74497895-c61b-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 01:46:11 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-55ad2a47b7aso1742364a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 16:46:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74497895-c61b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707353171; x=1707957971; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/OCgb0jTRL69NrP0XVB7Vo2l0PgRnEdf6iV8TLjQ5ec=;
        b=ChEvcRnKvKNAauevlxBmKR+7YPb4LaduC8P/CjVOWLbcRm0CBasQ1DW750JsjT4b3f
         7cBnGuvBVouTqwuxMjAqIAi25spKkdH/hjHHoVUJXcZLgUtWeB+nAfhy/gz5H62X/8Gr
         ksW9czmI2dj8f5CqA3zLje9fWDl+TOD0OxIgvjoux1cAS+v7Qn7zKY/7JyyqECk2TPUM
         tf9Hzxk95WSI0QfhNCGyoMOXIlqHtBcchlsW3yas68D4nmRGxvzzWhYm8/Zvo6jjCkDh
         /HC7tWBWBKKYCgrqtxBrW2uk+mLm4iP+x2nm9akUpM037MtH4v1DCCWs6g+Tu7+OZg6C
         1+cQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707353171; x=1707957971;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=/OCgb0jTRL69NrP0XVB7Vo2l0PgRnEdf6iV8TLjQ5ec=;
        b=EUNzWlgYGgzkSIzoK6a07ZAag1aB6qcIjz9MlK2v6JDF3oUU4vqgyog0v1sAY0VYo1
         8WBzJClfWJYEEUovGq6Wub7VFDMBn2z41o4dCc298mjawF2Fad+8x1TGAgWBaFOCRuYn
         641n4zOIsPF/1twXM209EtfkTClTCJhWNZ709Sc0Lk5OJvyyUVkXG7iiVY5UyLluTS7A
         U+o7nge+72zCh+O8n3XMcX2eBWh79HfjZXTXhF5BVFnUCLA22sYQvdiqL2lnQmZNvSpj
         HsTT4fE/k1O4a/eX3+9orndv7qaDTgD+OSNEIGw1KD6zpQ3yX5cMvdNuramKiMoUy/cj
         P1Lw==
X-Gm-Message-State: AOJu0YxwaiohsZJH4ju5cqkecuPG1H6PR5h8ydr2KjiNUpynK/tSB9SL
	Ls/Qz/GrOIBx9u7TBP93kRaDnr0U12H/SNpY3FsMNoqE/8xvUkejAe8lGajn+tTn29OYu8SAg+m
	qsFAwopFmhpjJ98SNJa+UUNsF5es=
X-Google-Smtp-Source: AGHT+IEkgmgNXu84IsAIEUvw5RpvD9U5KdWagYvH1D/akR77xdE1/K3zXnDhcSEmA9nr9jXGvZhvzp3UmfjjW21W6DU=
X-Received: by 2002:a05:6402:5154:b0:560:5fbb:a148 with SMTP id
 n20-20020a056402515400b005605fbba148mr5443550edd.39.1707353170582; Wed, 07
 Feb 2024 16:46:10 -0800 (PST)
MIME-Version: 1.0
References: <20240205105001.24171-1-jgross@suse.com> <20240205105001.24171-8-jgross@suse.com>
In-Reply-To: <20240205105001.24171-8-jgross@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Wed, 7 Feb 2024 19:45:59 -0500
Message-ID: <CAKf6xpurmS-__uvPAKzhQyhZbFAMepHU4r2HPvR+rNpRRLSjUQ@mail.gmail.com>
Subject: Re: [PATCH v4 07/32] tools/xen-9pfsd: add 9pfs attach request support
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 5, 2024 at 5:57=E2=80=AFAM Juergen Gross <jgross@suse.com> wrot=
e:
>
> Add the attach request of the 9pfs protocol. This introduces the "fid"
> scheme of the 9pfs protocol.
>
> As this will be needed later, use a dedicated memory allocation
> function in alloc_fid() and prepare a fid reference count.
>
> For filling the qid data take the approach from the qemu 9pfs backend
> implementation.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 00:48:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 00:48:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677925.1054847 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsaa-0007bP-Db; Thu, 08 Feb 2024 00:48:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677925.1054847; Thu, 08 Feb 2024 00:48:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsaa-0007bI-Ai; Thu, 08 Feb 2024 00:48:28 +0000
Received: by outflank-mailman (input) for mailman id 677925;
 Thu, 08 Feb 2024 00:48:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jwf=JR=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rXsaY-0007aS-TS
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 00:48:26 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c48f7273-c61b-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 01:48:26 +0100 (CET)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-5605c7b0ca2so1378779a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 16:48:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c48f7273-c61b-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707353305; x=1707958105; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=3h02KhYcKYr4sChkqfp2MHa63RLYk/teiY9V2+4SCmA=;
        b=Q8R/ef0PRPANYxRCIRkXBH6Djn1IWdXFxCwXhjQ78OhcwZUlIDmpOIvSaa7+YOBSL2
         2cbSaebCMTTvH9SY21bSoUA2FcY6uZ/pkIn8uQ9A0A7MDB8DKpGd4TDU1ezWmV0GlY/C
         Xk/tBR08lV58RPTogqnWUMYhVHw4YxrPL/aWwg2gNFPWbDdLDjsLHpvIePDHlC1JQqtu
         NuSTJp7LspHuwcPANHLVljc1r1/3PVarrzn8QBU37Zsnq7H+JNZ8Af3EArts5U6s5J5u
         DG5cM/eE/kk01joPPSb328pZ21trecXAcyyp6a+pr40I52cwJj8G8qkppyKGOgXApIZF
         HPjQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707353305; x=1707958105;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=3h02KhYcKYr4sChkqfp2MHa63RLYk/teiY9V2+4SCmA=;
        b=IynrSe4dvGRooXgystsAE3rjzoZB8effNDpiHugWOE6tgiAUzLtGGvuHyZqw4/S3VK
         awfUyWN9Q7IMmLHV6RmnqL+sSqAP0j9M+uSl1be3Af368sX0+VC2BCi3LRn0gBMlGzHH
         8ok63LRPhlvBdXlhOC5a92lu3ZblFAWsGh+0hFrvYPhLJnQszTj7asEim54id7TRdeg9
         +DAcMgtb5QnyqhYfW6otcUgkxYnBOQIpWldkL+0lRfrMYrwiqbyJWw4tNtJOOFhwF8TB
         uHZaTbKAYyZY4knC233L0Y3mo+gqmO0ZkK800aXnd3z20po3IxTpdNX29BWT0xIP5GcH
         FUrA==
X-Gm-Message-State: AOJu0YwRPgRoNHDA8aVP/oQ+kNiFvRJgO0TW1wsRmJqzk3tAOXqtcK7+
	AYVT5T2CSBZ9iuPuP5fdi5aiaOPrJcKhpfbToeJVydxLxlWepJln7r6Mxo7P7DBxw9C3EUYrHCZ
	gg5XJ8Lj+1EjSqnLwxw9sJY1yseGRfKKn
X-Google-Smtp-Source: AGHT+IEZM91i+3Zb092PAYmXBXxU19MGbLJMvAoeCT42BoP0yED+1H5imulhiwac7qT0Z5njeg4e07TTZ8+qjPYx4nA=
X-Received: by 2002:a05:6402:1203:b0:560:64f4:cbd5 with SMTP id
 c3-20020a056402120300b0056064f4cbd5mr5150060edw.21.1707353305396; Wed, 07 Feb
 2024 16:48:25 -0800 (PST)
MIME-Version: 1.0
References: <20240205105001.24171-1-jgross@suse.com> <20240205105001.24171-9-jgross@suse.com>
In-Reply-To: <20240205105001.24171-9-jgross@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Wed, 7 Feb 2024 19:48:13 -0500
Message-ID: <CAKf6xpuLc1YjeaEH6XgAFNLWAwSJj7h=OOLPJ9FFUhvqUDD=HA@mail.gmail.com>
Subject: Re: [PATCH v4 08/32] tools/xen-9pfsd: add 9pfs walk request support
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 5, 2024 at 5:59=E2=80=AFAM Juergen Gross <jgross@suse.com> wrot=
e:
>
> Add the walk request of the 9pfs protocol.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 00:53:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 00:53:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677932.1054857 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsev-0000as-UU; Thu, 08 Feb 2024 00:52:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677932.1054857; Thu, 08 Feb 2024 00:52:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsev-0000al-RT; Thu, 08 Feb 2024 00:52:57 +0000
Received: by outflank-mailman (input) for mailman id 677932;
 Thu, 08 Feb 2024 00:52:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jwf=JR=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rXseu-0000af-2Q
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 00:52:56 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 646b8d2f-c61c-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 01:52:54 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-5605c7b0ca2so1381994a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 16:52:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 646b8d2f-c61c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707353574; x=1707958374; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=g4/6pPPW9cKDAE5xv53SDcVFW7vlkCvJ+UeinfJAcuQ=;
        b=U67m/nIYs7b8/kN0P0srCDx7qHop3qPxNJFwlv7soeq6+yC5gSHioU4vwdiJA5gzkU
         GJx7mZxkKm9uYubkFmf3GBHOQyraKiouHoHaIHK6aSFuxGEmzEdUxXFCPLKJKCKZ4wcy
         7yxHt/8KGecyWFH27yi9iSzt0ZmoujUDnEKF3jhKETgbnjtRlc24HZmAgNPtrobYsUon
         5NiQw/AIR01nSL61Iig9jppVSKyMZNpDQ9XUY0gaWkCXQ0SnAQunxtYdQUAGvxh/BM25
         6u08fNbt1VX5SrB0Sa9V4MWRSOLqT0qOkZUf6gAQwJzYaIzwQcTleOeYE3jQbhkgWS56
         sbmA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707353574; x=1707958374;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=g4/6pPPW9cKDAE5xv53SDcVFW7vlkCvJ+UeinfJAcuQ=;
        b=C9yFTcvpS8Qiy9/gvbiFTa+O1sv90j0h5Rf4tNqZYE5cmoy7+butyECwjM/wJ/Dx4x
         Jr86SPrD8DbC+yaT+ldJ9PSWXv757ezZcTUJrG5GxAYd/j43KvivBB3dX4POnY4s9Sdy
         gQ6BxGHmNP2CZeZA2zXuw9DXvx88b8bFGHS+En0iRzVYS6gMlwJ30s40/CSU7qj91QpC
         fAIBT4j1AeDYXLt/kz85OgLnoKmPnHkw14QskdAS+QAS7nTzJUUQWyfFdp+0tHzFN7sz
         xDixUy2erq5N9RX+eNR0oCAobXaXM0285RY/7CslLJQGRF2LiMZqgEVSUelZmczlfbE9
         S0EA==
X-Gm-Message-State: AOJu0YynD12RS7xVsdzvBvscI+7q8eH3n/YTW73cWPOPEX8vZKGf5RWx
	LU+tRyDaGZA9rBqdfvF2mecl+LEyZW9G1+jMvo5t4uyx5JSbupx5wkh07V8HDDOjUSJGMe6aKC8
	YLCoJpPxN2Ec1MeMVPXSw68FV1Jg=
X-Google-Smtp-Source: AGHT+IH3+EfhY+1g7chY05CpNnWKLIRsW7fjoIsA5M6RXnfJymZ/pZ16IzVTSQoLDWBabt8iVwj6LldvUcYTVBTNwf0=
X-Received: by 2002:a05:6402:149a:b0:55f:c3c1:34e with SMTP id
 e26-20020a056402149a00b0055fc3c1034emr4622599edv.15.1707353573452; Wed, 07
 Feb 2024 16:52:53 -0800 (PST)
MIME-Version: 1.0
References: <20240205105001.24171-1-jgross@suse.com> <20240205105001.24171-10-jgross@suse.com>
In-Reply-To: <20240205105001.24171-10-jgross@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Wed, 7 Feb 2024 19:52:41 -0500
Message-ID: <CAKf6xpsUqVTMF89P5Qhadt-YBV8z+O+ag+zpQx4FLiSKq8Lp1g@mail.gmail.com>
Subject: Re: [PATCH v4 09/32] tools/xen-9pfsd: add 9pfs open request support
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 5, 2024 at 6:13=E2=80=AFAM Juergen Gross <jgross@suse.com> wrot=
e:
>
> Add the open request of the 9pfs protocol.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 00:53:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 00:53:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677933.1054867 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsey-0000pe-4i; Thu, 08 Feb 2024 00:53:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677933.1054867; Thu, 08 Feb 2024 00:53:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsey-0000pX-1N; Thu, 08 Feb 2024 00:53:00 +0000
Received: by outflank-mailman (input) for mailman id 677933;
 Thu, 08 Feb 2024 00:52:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ckqh=JR=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rXsew-0000af-A9
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 00:52:58 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 65c0c6f4-c61c-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 01:52:56 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-40ef64d8955so11633545e9.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 16:52:56 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 f5-20020a056000128500b0033b3cf1ff09sm2499757wrx.29.2024.02.07.16.52.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 16:52:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65c0c6f4-c61c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707353576; x=1707958376; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3QhgJY3kbAlZ1mgX4kDbwqz2TLqCn5dKc9y1wV9Rc5I=;
        b=LHtHi2/DlK6ZE8qtbIexij5DgrN6lhlmObO2BM4isIv4cgi9br64iHNuDo0m/Z+fu4
         QWbWAfaa/UqbzYYoQUCX7OMR1GQYpYIMcRiPa4KXbx+1JYY9t4IlPxKN43Tt419Mjjdo
         nv1APl416p3XjWDlriJWPpT0mbeNbXYmD6+ic=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707353576; x=1707958376;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3QhgJY3kbAlZ1mgX4kDbwqz2TLqCn5dKc9y1wV9Rc5I=;
        b=xBrY1CtQ7Wz3D/ijJ/fLp+p0AFoBnw0tfZiHglg5cVU1+Sua4J72ytFTPtNk7nEjiF
         cMdBLAX3KGAEWzNZp33+PcdPwxjTPUJhoJue4g3Lku6gCaVX1JMJh20DRWl784IIrLQQ
         8gZxjc/TixR8dkwq3vN97IbV0iPg9DwrMcmtjXHjvqnOMbKXBgFzvFDmteJdB8Wp2IOk
         z/jaKffjwoeTXXeigjzMOFpMDn5FORIark607TR2Ee1KW10WvjF9NV65r0IalsB3JjZA
         6oFEaE1I4Ak1gQS8lIPvGArTzAQUMWKK3cuLidMpjRlj+eamNQf64h721I8c0EKcekkE
         aoRQ==
X-Gm-Message-State: AOJu0YzaZ/P/34/sxzkhpqAZICd/b72fXGdCZhTREUmiiLdmYzKeZEyR
	L4QP2OcsKP5oPsvJ/vJ/zS0Mfyh+hJuP4yu4g8mkrRhnQ2iVvM6t+th6h6IRilk=
X-Google-Smtp-Source: AGHT+IFzIX3HPjSumvwGSvBrI/7KiDVTh8GvvnnZJcOmLRN7zOjTD7gDvEBKuG66+YbWlAz5ZJzUjw==
X-Received: by 2002:adf:eed2:0:b0:33b:45f7:636b with SMTP id a18-20020adfeed2000000b0033b45f7636bmr4787012wrp.22.1707353575862;
        Wed, 07 Feb 2024 16:52:55 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUbWPmkQD9C7kTxfv/wmDHCKvaUDbAB60cVwUu+npMxorUK2fxpVaCMK9j1z8h8fK3Pj1nOCqpY4ei9GN5lyVxdpJg6Y4TTAHsUIsv12AXXne4zVG+9y5IepeWiJuk46B4cB3j7EiWbkwdimLLZQ0xlMEL6hb0bqZHxOXer/g==
Message-ID: <d5035d8e-4183-4213-8e12-86d5cb2dbfc7@citrix.com>
Date: Thu, 8 Feb 2024 00:52:55 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 02/32] tools: add a new xen logging daemon
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jason Andryuk <jandryuk@gmail.com>
References: <20240205105001.24171-1-jgross@suse.com>
 <20240205105001.24171-3-jgross@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240205105001.24171-3-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 05/02/2024 10:49 am, Juergen Gross wrote:
> Add "xen-9pfsd", a new logging daemon meant to support infrastructure
> domains (e.g. xenstore-stubdom) to access files in dom0.

I was still expecting for "logging" to disappear from this.

In both cases it could just be deleted the sentences still work,
although the subject ought to gain a xen-9pfsd part.

Happy to fix on commit.

>  tools/Makefile              |   1 +
>  tools/xen-9pfsd/.gitignore  |   1 +
>  tools/xen-9pfsd/Makefile    |  38 ++++++++++
>  tools/xen-9pfsd/xen-9pfsd.c | 147 ++++++++++++++++++++++++++++++++++++

Asking just in case...  Do we really want the "xen-" in the directory? 
tools/9pfsd is still perfectly descriptive, and more amenable to tab
completion given what else is in tools/

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 00:57:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 00:57:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677941.1054878 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsj7-0001hb-ND; Thu, 08 Feb 2024 00:57:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677941.1054878; Thu, 08 Feb 2024 00:57:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsj7-0001hU-Iu; Thu, 08 Feb 2024 00:57:17 +0000
Received: by outflank-mailman (input) for mailman id 677941;
 Thu, 08 Feb 2024 00:57:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jwf=JR=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rXsj6-0001hO-8g
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 00:57:16 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ff734226-c61c-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 01:57:14 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-560e9c7a094so1113448a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 16:57:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff734226-c61c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707353834; x=1707958634; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=rIb7EFybyD98WhmBqTwsfmtz56jlmQb0fYOwTFmQv5I=;
        b=nluBvMzaH5iei34gicA2Slkj6xUfAe6XOWrSOHKodoG9F2aXkcWZPqcmzuMkowIzVc
         WZn9xfmrjlxjv6DjrZKYAt/BRl5Ea3nP0MzDago/xBdMtres5oWMxB3hEnc2KiDkhKOU
         /VEphiD454sXLA5Q1F0NwREMxmqqCoj0FXpIbXr53HUrCG6mfqM6JL2IVdu9BGfcxj7H
         7TuUkiQ3xjEigccbdyLN7M1Hk5kum5KcW4cCOmFA3gW67niV6YuNsMfrWt0QkX4cHuR0
         wM5znN+1VGN2Zdr+HOWfP00bMrvNHtLOvbI+8LzvPkRXUY2283sb5DQj/O0ly+F8sCDt
         yPSg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707353834; x=1707958634;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=rIb7EFybyD98WhmBqTwsfmtz56jlmQb0fYOwTFmQv5I=;
        b=UkTqdd7BpJQlUQYASshib6FFPdQEDbvF7D2s+g7CY7UU9ZFSmiUunhNWaB3UVAKMqd
         HxsYMRqaXcAHnkRKN5vk+047PmobVWAPd6XwQ5QVjnuWXf1ZFOSBtY2ufn6Fs2g6VOpU
         v0/4UfDYhveyQLSAqTGXcSOWFm4+0c6qn8Ta4n7w8oMktmNzh+8VZXbyjEDQjD835WQz
         QCFOagleQQ/ZWlDtz4FVaCaZ/R9cuk93gLiFId1eLrqxNJ5wGXy+0Z6rBNHeiPauu13o
         z12T2re+Du5WYLZYwJlG5p0Zp3yHgpzXO3CGrMIhdkuD4KMQFU7RNE0HeRwYO+5pAaWL
         Ht1g==
X-Gm-Message-State: AOJu0YxOCTtEB3oKwu+u+cMvf7+YFTzvscBHeg9MpMHaFDtX7Vg9S6hY
	lFLChodI29b50/D1d8GNPx3uKLB24o4UGqp9oPJghFkqVf9i1KniuX6pHeWC7FnRRsJYYQ8AMw8
	uWPHK02lOQzZUbjNQwq5LqK1b2s8=
X-Google-Smtp-Source: AGHT+IF0FdJgtz5j29b0bEQhmoSqOqR4Oo26lLd6QN2ZRfLiNF3pezvmQ2S46al7Mhz8xI0s5WesOAskt70+A+xO2ww=
X-Received: by 2002:a05:6402:695:b0:55c:d503:764c with SMTP id
 f21-20020a056402069500b0055cd503764cmr4663889edy.22.1707353833795; Wed, 07
 Feb 2024 16:57:13 -0800 (PST)
MIME-Version: 1.0
References: <20240205105001.24171-1-jgross@suse.com> <20240205105001.24171-11-jgross@suse.com>
In-Reply-To: <20240205105001.24171-11-jgross@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Wed, 7 Feb 2024 19:57:01 -0500
Message-ID: <CAKf6xpuU8wF2Mpy0uDAMDXR7uVNWyJ8d7aDJxTJ3h9SdV=unDA@mail.gmail.com>
Subject: Re: [PATCH v4 10/32] tools/xen-9pfsd: add 9pfs clunk request support
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 5, 2024 at 5:58=E2=80=AFAM Juergen Gross <jgross@suse.com> wrot=
e:
>
> Add the clunk request of the 9pfs protocol.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 01:01:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 01:01:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677945.1054886 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsmr-0004tw-7d; Thu, 08 Feb 2024 01:01:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677945.1054886; Thu, 08 Feb 2024 01:01:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXsmr-0004tB-4x; Thu, 08 Feb 2024 01:01:09 +0000
Received: by outflank-mailman (input) for mailman id 677945;
 Thu, 08 Feb 2024 01:01:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jwf=JR=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rXsmq-0004SB-Er
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 01:01:08 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8a89d11b-c61d-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 02:01:07 +0100 (CET)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-511612e0c57so2228961e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 17:01:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a89d11b-c61d-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707354067; x=1707958867; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8/DzsEsEP9tjAI36Ecp36ox2GViiOlgStXseU82a9Xo=;
        b=BktMQJjXkvbmqhDDG5wabFNWNBc9LyLg9133Mf1q/1+2fn5Bz71fZuyEHG3cM5q6y7
         G66sAdm7ocah0cO4pHh3ggm0O2u3eOoJUUKMAWZVJ7xbm4wMrUW44tPWEdTHamqTEw2b
         Qgkgra1RLdg1v1te2QTT51GqZltSRK4U32r35kLKZku1kESuAaKcpSyDxuu3BCNdK5D2
         h7407Wy6eU0Bh2i6quacsEXe1XOOXLHQThEjHXA9cpoY0sXdKP3Qrdkxf07QwXuSoU/m
         k9QdPl8Aomd7Ky8FIfJxhRXhOxx2AakVCSWjTQmW3OGltircEiOaAA/4beY/MmGqvA4b
         JTCQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707354067; x=1707958867;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8/DzsEsEP9tjAI36Ecp36ox2GViiOlgStXseU82a9Xo=;
        b=GHWETvxHBhm7sYLF/tQggrISVwbvdxCT841lZINrxItdKL5soMEqnDQYXtM6NoJ+66
         lr5tti3jNzjSU1I//YmJ/dtY1Kk4GTC6jbflacDsh72r53oHrkJsQTJ6sS0D4PghDPN4
         QPWnxw3s8l3CgoUFiO2axyFlm39dvREAGhJT2glmLtHX4U3Y/fAq+v26btkUfoZ7dDBO
         l5FWWtskaakdKfJiEtvjI5IAvCjJcN0iQ0NW5n1fiX5ZxdtoPF4cF3fnsURbuDkicIxw
         vl9TGtXCAz441ebiELzQpEow7ASHXKYvhBCpLkJZmJkGOK/5d/HyMRco8ps2SFviQqVm
         eRcQ==
X-Gm-Message-State: AOJu0Yzeo+E7NKXGI0Z6KWVYnMsaZXEYFpk7sq15aKHyWcnwyvtF84K8
	IFpp2cZQH68Fp5cMdh1uMiAIbkTXBIDtdYBIY3tezWVVdHwAE5saWS4hM3rpQqdDrHU4v1V4xg2
	lPXIBOjYFV4J/wTBdfDrmOGXcDBA=
X-Google-Smtp-Source: AGHT+IF4hBGKy/zn4zeAzvvHT2G3779htxiQ0fcDKukuf+DkjppCAyx7m1KDLqG9qCbLlMNtIq2M+pOLFOKK1+x/0ek=
X-Received: by 2002:a2e:be0f:0:b0:2d0:bf87:b3a6 with SMTP id
 z15-20020a2ebe0f000000b002d0bf87b3a6mr5122694ljq.37.1707354067011; Wed, 07
 Feb 2024 17:01:07 -0800 (PST)
MIME-Version: 1.0
References: <20240205105001.24171-1-jgross@suse.com> <20240205105001.24171-12-jgross@suse.com>
In-Reply-To: <20240205105001.24171-12-jgross@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Wed, 7 Feb 2024 20:00:55 -0500
Message-ID: <CAKf6xps64YBzg9pRZjtZ1RiQqMVPhi4Qcwwbd+4g6wyXouzaPA@mail.gmail.com>
Subject: Re: [PATCH v4 11/32] tools/xen-9pfsd: add 9pfs create request support
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 5, 2024 at 6:21=E2=80=AFAM Juergen Gross <jgross@suse.com> wrot=
e:
>
> Add the create request of the 9pfs protocol.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 01:11:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 01:11:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677951.1054896 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXswK-0007oU-4W; Thu, 08 Feb 2024 01:10:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677951.1054896; Thu, 08 Feb 2024 01:10:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXswK-0007oN-14; Thu, 08 Feb 2024 01:10:56 +0000
Received: by outflank-mailman (input) for mailman id 677951;
 Thu, 08 Feb 2024 01:10:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXswJ-0007oD-6x; Thu, 08 Feb 2024 01:10:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXswI-0000s7-Vs; Thu, 08 Feb 2024 01:10:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXswI-0007xL-GS; Thu, 08 Feb 2024 01:10:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXswI-0006eU-Fz; Thu, 08 Feb 2024 01:10:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=t6ltz0M1ahOV5aCxRVwiNkhUpUtpG5xOXO/PTuDBuXw=; b=0FftivrdfwtIkV4n9xGRm5fAt/
	PqdPOYCmIopbG8k73QdkuvRCZ0ApJIRi3ZL2nWidUMgqCc8+PtcsLZGV/NIXWAXaniWEPLzlEfW4M
	aB88u6B9Bi15e1wS7k43RvNFEnoWZrxIRMoSp2of3jHTXhIUpnZSLwm3nFpfpqYJfY5Y=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184617-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184617: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-vhd:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=59a69674b533a2d0c0ad5cd7fbcd655295574509
X-Osstest-Versions-That:
    xen=57a923bfd920307c5bfa533c95dba16a13ea0225
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 08 Feb 2024 01:10:54 +0000

flight 184617 xen-unstable real [real]
flight 184621 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184617/
http://logs.test-lab.xenproject.org/osstest/logs/184621/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-vhd 17 guest-start/debian.repeat fail pass in 184621-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop      fail blocked in 184612
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184612
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184612
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184612
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184612
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184612
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184612
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184612
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184612
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184612
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184612
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184612
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  59a69674b533a2d0c0ad5cd7fbcd655295574509
baseline version:
 xen                  57a923bfd920307c5bfa533c95dba16a13ea0225

Last test of basis   184612  2024-02-07 01:07:15 Z    0 days
Testing same since   184617  2024-02-07 13:03:56 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michal Orzel <michal.orzel@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      fail    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   57a923bfd9..59a69674b5  59a69674b533a2d0c0ad5cd7fbcd655295574509 -> master


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 01:11:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 01:11:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677955.1054911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXswS-000867-Do; Thu, 08 Feb 2024 01:11:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677955.1054911; Thu, 08 Feb 2024 01:11:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXswS-000860-A8; Thu, 08 Feb 2024 01:11:04 +0000
Received: by outflank-mailman (input) for mailman id 677955;
 Thu, 08 Feb 2024 01:11:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jwf=JR=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rXswR-00085O-8o
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 01:11:03 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ec52e95c-c61e-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 02:11:01 +0100 (CET)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2d090c83d45so15685031fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 17:11:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec52e95c-c61e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707354660; x=1707959460; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=N8cfemsAt0BP54fzkiw1RrU2pfk5Y/Jkj4Junx0q4y8=;
        b=VihC06BTko4y0ofBqdiEhZdh+ogTyENiq7lrdP3HmiiJh+/rb99GiBaEGqRkYa4Lfn
         SmUl4dd3aKPSPYJ9uFDDyN8rQcPZuRBDLaqfZ15AWMyjALow2NU8oGHYdwSkf94ZznLW
         MmXPvVzEtVkypeD7+D8nAatCxkLrCxNXjLRaoSa3m4kiIPOyvoNnuiUBm1iaYMS60bBK
         igiTQmfiTWwqF8ioF/IJfiOAlKqRmy0M2UHy3heHLAAJ5tpIyUirAME0Pq8kd5TP+sa/
         TmW7+3ExFydL79jik+MhAxqHg61qWibZkP8r5VrGVuLWGLGWkLgyRi/z7ftb6cBe1fU8
         EuwQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707354660; x=1707959460;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=N8cfemsAt0BP54fzkiw1RrU2pfk5Y/Jkj4Junx0q4y8=;
        b=Vk2Xnn2g23Bo+AzY1V14ZcwClhM3UR+vFjjaI38LXdzNbHQPl0GkqpmigzvrNCX7cN
         u01+kOBNWtmEwDG5kex8fgDrNH9uX21w2unRcAwDzvne5irQICYbdkQLWt/RLkzxixwL
         d4Ni0Mgz8eplodlOLdc1qu4aLnrhATBBNGPuj999xMXi7wPf+yPy78eXKj/wU5b+xpu4
         Jk+jnrh0N4qvY73luGPCwx/uMPODRHdKZVtn5SQ/2FGJSJTPr3I9b/8KRv2ijcrDPdsd
         qLuJgtOkPpWD6k+dsOr1vwZlXyJsGgca/zTFw0I3wDltRcLVPX7INRAaqEWvIUzK1UmQ
         VLqA==
X-Gm-Message-State: AOJu0Yzzjyc2C1VeJ/VeCFLmz4r5XtJfOh+DUV9bMTfJuiFoXNa90xhz
	HqkxF+BKaRuiEF4aNkMEopPcCMM9U2biI8wTlbOFMz3gSa0g0yZf0loFdPZ5sND41BLqpBhP0pO
	Sfh4New0VIuzswCP1wy/Pjj+tUZI=
X-Google-Smtp-Source: AGHT+IE7TVEdTt7mSFoVIZF+adJDTfGzTJv1DCAGzGh2ypb3ql475iMlq3T7wBUuQdg87yb/lpHRgV+dz8c98nmlmNk=
X-Received: by 2002:a2e:700e:0:b0:2d0:b578:39da with SMTP id
 l14-20020a2e700e000000b002d0b57839damr4908738ljc.5.1707354660146; Wed, 07 Feb
 2024 17:11:00 -0800 (PST)
MIME-Version: 1.0
References: <20240205105001.24171-1-jgross@suse.com> <20240205105001.24171-13-jgross@suse.com>
In-Reply-To: <20240205105001.24171-13-jgross@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Wed, 7 Feb 2024 20:10:48 -0500
Message-ID: <CAKf6xpvGSXycixq5TP4gQFs+CRChP4z0+Y6Wf-LS0p4Yinmujw@mail.gmail.com>
Subject: Re: [PATCH v4 12/32] tools/xen-9pfsd: add 9pfs stat request support
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 5, 2024 at 5:57=E2=80=AFAM Juergen Gross <jgross@suse.com> wrot=
e:
>
> Add the stat request of the 9pfs protocol.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 01:21:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 01:21:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677963.1054920 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXt6i-0001yS-9P; Thu, 08 Feb 2024 01:21:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677963.1054920; Thu, 08 Feb 2024 01:21:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXt6i-0001yL-6k; Thu, 08 Feb 2024 01:21:40 +0000
Received: by outflank-mailman (input) for mailman id 677963;
 Thu, 08 Feb 2024 01:21:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9ajo=JR=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1rXt6h-0001yF-4x
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 01:21:39 +0000
Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com
 [136.143.188.12]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 65db1260-c620-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 02:21:36 +0100 (CET)
Received: from mail-yb1-f179.google.com (mail-yb1-f179.google.com
 [209.85.219.179]) by mx.zohomail.com
 with SMTPS id 1707355292168575.4124404708025;
 Wed, 7 Feb 2024 17:21:32 -0800 (PST)
Received: by mail-yb1-f179.google.com with SMTP id
 3f1490d57ef6-dc6d8f19ab3so1450572276.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 17:21:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65db1260-c620-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; t=1707355293; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=EFwFtv9vPRT/UZpBPHJAh/BaaRDx6jneI/Xhu3SB3I4tpe5krIHGc2bREayelCUoj4K5jt/t2Vgn4z4lGqVBJ9jc53XYWDVZhkHDeaotOHnH4Z/OCN3KBE0dEu3C8dZUlAyUMbTtWGlk6lDs08c0JjxYXyTKXx9EpYC94VnPfEA=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1707355293; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=rr7/ceTMxDTm0VhdI7nT/NNQsHRms8IddkEL0Jw0XmY=; 
	b=Ttqvofzp6NkOt/NxSMmOeE7qjfh/snixgSmOj5A91u0dFyZmPjM6n5hVkyW/2LngLbONq17LSQvMpmauBpsw5zidoHo1K9XO0qlPlHknJ9CsAQlNgB4YkGoPkmYX4U4JGvsU8YUOMmwg2RE2wQ+AQRvajTyNCFI2bFGnw5L/3j8=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1707355293;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=rr7/ceTMxDTm0VhdI7nT/NNQsHRms8IddkEL0Jw0XmY=;
	b=DKyiz+vqy9OEZHpVAW9NggrFZgkn8EFZieQ6F01LtBMPJUFvHL9NYT82h0Q4TDZg
	oAE6DdtLmmRwy3JAqtuFoW0wM3/g9R/8H5nzAqufBRkb1euHOLTEbKKkbAvGNushjhk
	LYKrzu8Un0F/QFBEpP1BO+VQiGeqGVvxmdaQKKWM=
X-Gm-Message-State: AOJu0YyKYn3sa0ClpyrIMOIdjHKNocli8zQzuFx0MsVJwrLuVSQBIMuM
	zaBxUKnlWQuz28Fodgs4guHGhM89OQkFPhyQLX8YuRTvxUYufvTM4dtFaY4lbRXUy8U2sYyZ7+Z
	tlcIqbjY35QuQ+y86H2ADwO3xlTw=
X-Google-Smtp-Source: AGHT+IGsJp5QI8+AzgmR/lXMCHfxTydK2C8ctmt5+QZBI/QI0XUx4PmSSXcWUFQsMto6Jj6EkAIzz5aFRg2vF/ODZTw=
X-Received: by 2002:a05:6902:1a45:b0:dc6:b8f5:50ae with SMTP id
 cy5-20020a0569021a4500b00dc6b8f550aemr7357185ybb.32.1707355291123; Wed, 07
 Feb 2024 17:21:31 -0800 (PST)
MIME-Version: 1.0
References: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
 <CA+zSX=beYAGb81zqD19YNAV3cXPFTk-7V68XYn59Eb3mGFH_Jg@mail.gmail.com> <507f29cf-7a07-4f3d-81e8-2236a8380b1a@citrix.com>
In-Reply-To: <507f29cf-7a07-4f3d-81e8-2236a8380b1a@citrix.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Wed, 7 Feb 2024 20:20:55 -0500
X-Gmail-Original-Message-ID: <CABfawh=6AHZzH1aBP2A=oRUTvjVmgd6QQNxEQAwGCBV_9-CTcA@mail.gmail.com>
Message-ID: <CABfawh=6AHZzH1aBP2A=oRUTvjVmgd6QQNxEQAwGCBV_9-CTcA@mail.gmail.com>
Subject: Re: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor ap2m->default_access
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@cloud.com>, =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>, 
	xen-devel@lists.xenproject.org, Alexandru Isaila <aisaila@bitdefender.com>, 
	Petre Pircalabu <ppircalabu@bitdefender.com>, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 7, 2024 at 5:21=E2=80=AFPM Andrew Cooper <andrew.cooper3@citrix=
.com> wrote:
>
> On 07/02/2024 1:18 am, George Dunlap wrote:
> > On Tue, Feb 6, 2024 at 6:08=E2=80=AFPM Petr Bene=C5=A1 <w1benny@gmail.c=
om> wrote:
> >> From: Petr Bene=C5=A1 <w1benny@gmail.com>
> >>
> >> This patch addresses a behavior discrepancy in the handling of altp2m =
views,
> >> where upon the creation and subsequent EPT violation, the page access
> >> permissions were incorrectly inherited from the hostp2m instead of res=
pecting
> >> the altp2m default_access.
> >>
> >> Previously, when a new altp2m view was established with restrictive
> >> default_access permissions and activated via xc_altp2m_switch_to_view(=
),
> >> it failed to trigger an event on the first access violation.  This beh=
avior
> >> diverged from the intended mechanism, where the altp2m's default_acces=
s
> >> should dictate the initial permissions, ensuring proper event triggeri=
ng on
> >> access violations.
> >>
> >> The correction involves modifying the handling mechanism to respect th=
e
> >> altp2m view's default_access upon its activation, eliminating the need=
 for
> >> setting memory access permissions for the entire altp2m range (e.g. wi=
thin
> >> xen-access.c).  This change not only aligns the behavior with the expe=
cted
> >> access control logic but also results in a significant performance imp=
rovement
> >> by reducing the overhead associated with setting memory access permiss=
ions
> >> across the altp2m range.
> >>
> >> Signed-off-by: Petr Bene=C5=A1 <w1benny@gmail.com>
> > Thanks Petr, this looks like a great change.
> >
> > Two things:
> >
> > - Probably worth adjusting the comment at the top of
> > p2m_altp2m_get_or_propagate to mention that you use the altp2m
> > default_access when propagating from the host p2m
> >
> > - This represents a change in behavior, so probably at least worth a
> > mention in CHANGELOG.md?
>
> This is a bugfix to an tech preview feature.  It's not remotely close to
> CHANGELOG material.
>
> Tangent.  SUPPORT.md says tech preview on ARM, despite there being no
> support in the slightest.  Patches were posted and never taken.
>
> > Tamas, I guess this is OK from an interface compatibility point of
> > view?  In theory it should always have been behaving this way.
>
> Given the already-provided Ack, I expect that has been considered and
> deemed ok.

Correct, this is just a bugfix.

Tamas


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 01:28:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 01:28:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677970.1054931 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXtDU-0003Bk-3j; Thu, 08 Feb 2024 01:28:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677970.1054931; Thu, 08 Feb 2024 01:28:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXtDU-0003Bd-0o; Thu, 08 Feb 2024 01:28:40 +0000
Received: by outflank-mailman (input) for mailman id 677970;
 Thu, 08 Feb 2024 01:28:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jwf=JR=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rXtDS-0003BX-P8
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 01:28:38 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 61ecfe66-c621-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 02:28:37 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-560e9c7a094so1130576a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 17:28:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61ecfe66-c621-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707355717; x=1707960517; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=P0l3fpPtOu4vPs3n5vwWMAqgu+7u7L7cruddr+XftSA=;
        b=Nbu4Fa4NfSihUveNWiem2BvGPyG8B5c55t1GgnsrnK5pn0ZU2eF2zeV9rBduZZKM8r
         s52zKPH1RulQsiR2wbHtvguDbtS9FnTJTWV7SzgLl44Iykz7KKNXnyHoCSQrsvhLBUE5
         PeUOdgm6MMMOJ+HQ2UcMsgkU100qTbyX2yJ7XUH7JOMK8hWSOBYEEjM5EsHgAhH980Q0
         W2yQ5H55sqtLWkGmUIrA62ayk8seufgKvs7oKXA/ci1i/ajskMabFoEW2+wvmngE4LiZ
         LkGaHeRQYdgiiWuqLMG9Htw2kRIGMKwCyof3AGhaw/FlBPfRbbnEw6wBXrLuGjxSgQlT
         5vDA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707355717; x=1707960517;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=P0l3fpPtOu4vPs3n5vwWMAqgu+7u7L7cruddr+XftSA=;
        b=MFs9kGmDO5n+NlTY5lrbY+HtPA715FZbB7pnD1mmx5m9xCRdftz23AocmvoA1A1fEl
         zdbO4VZqbRRtkN+MjemFgUtfP68suPdPILL2JwCOehVWkKM5Ya4kZMWur4Divh48BfSD
         /dZoV7dqyThC/jMRsWBOLSjr3I8fFioBe37zFphvef43/TZm0kBBrxOIAQ8njU2VDF7z
         jcVH5zz62WGM4bV9unkr0OY6Yu3pJCYTH1ZmGAdADS5uyg8Jh3xLz1dQJg9fAQz6RG26
         fToAmBgoHI0jKWnBi7NNoZ0pmfn2AIU9s7OJEs2Dd6JfMxu/3vvdE+6msZt2YDZDKVoJ
         sgsA==
X-Gm-Message-State: AOJu0YyFtYCt3wR7Ap2cOYJc1ZkJq77Yluj88l9zBFVpNywyyxZzXrcH
	6sEm+uV/Qr2cyESvrmuufrUdMwmwiSFCLbTxHPb2G2gjf0nUcAnfRP+sVfBqKXPR7eA0zjWE1HZ
	FU4xbZ9xp7VzufAXmkJjiCvFBIbU=
X-Google-Smtp-Source: AGHT+IF/FeEd7XrwAVCegx7mogK3i0Frooxf/vOgxw+BF076t1SLJ4ZhEwFqoKuAinu13SEkmpZKuztgHeoLyEE2/Pc=
X-Received: by 2002:aa7:d885:0:b0:55f:f1f5:1a41 with SMTP id
 u5-20020aa7d885000000b0055ff1f51a41mr3007154edq.15.1707355716854; Wed, 07 Feb
 2024 17:28:36 -0800 (PST)
MIME-Version: 1.0
References: <20240205105001.24171-1-jgross@suse.com> <20240205105001.24171-15-jgross@suse.com>
In-Reply-To: <20240205105001.24171-15-jgross@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Wed, 7 Feb 2024 20:28:24 -0500
Message-ID: <CAKf6xpvZP-kGpPMvbVm-e=f-RQ3Czb24VnmwMzTc915O9hQJcg@mail.gmail.com>
Subject: Re: [PATCH v4 14/32] tools/xen-9pfsd: add 9pfs read request support
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 5, 2024 at 5:51=E2=80=AFAM Juergen Gross <jgross@suse.com> wrot=
e:
>
> Add the read request of the 9pfs protocol.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> V2:
> - make error check more readable (Jason Andryuk)
> V4:
> - add directory read support
> ---
>  tools/xen-9pfsd/io.c | 90 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 90 insertions(+)
>
> diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
> index b763e3d8d9..cfbf3bef59 100644
> --- a/tools/xen-9pfsd/io.c
> +++ b/tools/xen-9pfsd/io.c

> +
> +    len =3D count;
> +    buf =3D ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
> +
> +    if ( fidp->isdir )
> +    {
> +        struct dirent *dirent;
> +        struct stat st;
> +        struct p9_stat p9s;
> +

"""
For directories, read returns an integral number of direc- tory
entries exactly as in stat (see stat(5)), one for each member of the
directory. The read request message must have offset equal to zero or
the value of offset in the previous read on the directory, plus the
number of bytes returned in the previous read. In other words, seeking
other than to the beginning is illegal in a directory (see seek(2)).
"""

I think you need to check `off`.  Looks like QEMU only checks for off
=3D=3D 0 and rewinds in that case.  QEMU ignores other values.

Regards,
Jason

> +        while ( len !=3D 0 )
> +        {
> +            errno =3D 0;
> +            dirent =3D readdir(fidp->data);
> +            if ( !dirent )
> +            {
> +                if ( errno )
> +                    goto err;
> +                break;
> +            }
> +            if ( fstatat(fidp->fd, dirent->d_name, &st, 0) < 0 )
> +                goto err;
> +            fill_p9_stat(device, &p9s, &st, dirent->d_name);
> +            if ( p9s.size + sizeof(p9s.size) > len )
> +            {
> +                seekdir(fidp->data, dirent->d_off);
> +                break;
> +            }
> +            fill_buffer_at(&buf, "s", &p9s);
> +            len -=3D p9s.size + sizeof(p9s.size);
> +        }
> +    }
> +    else
> +    {
> +        while ( len !=3D 0 )
> +        {
> +            ret =3D pread(fidp->fd, buf, len, off);
> +            if ( ret <=3D 0 )
> +                break;
> +            len -=3D ret;
> +            buf +=3D ret;
> +            off +=3D ret;
> +        }
> +        if ( ret < 0 && len =3D=3D count )
> +            goto err;
> +    }
> +
> +    buf =3D ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
> +    len =3D count - len;
> +    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "D", &len, buf);
> +
> + out:
> +    free_fid(device, fidp);
> +
> +    return;
> +
> + err:
> +    p9_error(ring, hdr->tag, errno);
> +    goto out;
> +}
> +


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 02:26:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 02:26:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677975.1054940 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXu6s-0003Tn-6o; Thu, 08 Feb 2024 02:25:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677975.1054940; Thu, 08 Feb 2024 02:25:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXu6s-0003Tg-4E; Thu, 08 Feb 2024 02:25:54 +0000
Received: by outflank-mailman (input) for mailman id 677975;
 Thu, 08 Feb 2024 02:25:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jwf=JR=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rXu6q-0003Ta-PU
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 02:25:52 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5f47dbd6-c629-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 03:25:49 +0100 (CET)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-33b4b121e28so684663f8f.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 18:25:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f47dbd6-c629-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707359148; x=1707963948; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=rykO7zXEcMv/WhsHVZTU+YRtV6PJ4UtB6DcUCziOnBk=;
        b=Cjkd3l8gz9OjmrwG8IcPF+xmgJ/iwiKVzjnKqjaxWzklh+nnOcl3JyTRsgDHdeXhlX
         dob3tof70NhMOXjyXKJfO7RtENRtyVaM0zRI1shPX8RIQTiF9TRWJ515963oFrIwrpto
         VhD6CzVTfUsIwJKF9IZb/Bnvunm5mv/wLWX1QI/OEKIkVVvq7BeBRjHwliTR4bXZL4fm
         0GcQyK7quO9w+RhBsZFzD6Nhml1AaLV6lYVhH5WNdPRtKn1XtEAdNZTUzK6WWv9T6URe
         7ZMCgQofMH+/cAvRIY6F3IYbnolBlHmYWw8RmIZ58KWifVQElhTyKTx8hSzyqhQ9Icdp
         9G+w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707359148; x=1707963948;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=rykO7zXEcMv/WhsHVZTU+YRtV6PJ4UtB6DcUCziOnBk=;
        b=d/tuJXpQBSH39ms2JaRDYc8m7jAFNOYQWSPwUK8hySUQdnJnBLvFCC96Y7TxhO7/6d
         PjikpXRe6J2ApT4uh4sivlMWw30kJY0R3pr4txTK4N2BKDjSw1YPt40FnTbuQbDtas9v
         aRB+REgF5nC0DnaGmj/3ymE4jpDdl3NnwMXZoqFVxHSijpCvU6/jHT9ciZom4jA9iDkw
         XAi7I0QMYY3H/xvsudI6SSRUDn6UZOWbdTpJ/j0g3EHnAdvqKgjy+xPGFL+SVS3A8ttz
         aJLT5MqO3Is+CdT2AhyGCovbWau8bqf8vUpJgnZ/1A8vCYpDtKlOpqpmVA033yFiyO5a
         SY7w==
X-Forwarded-Encrypted: i=1; AJvYcCWhr55pKOKGl3c1VXnwsax+fdZ6ydLm3gzIIEgre3obqRR8ixz5xsdG7/LHd7lzEwpJkcNpm7NKUjMeSPNbHfiFSAQM+/1hO9QkzjRZo6s=
X-Gm-Message-State: AOJu0YztRfUPB//Pa1mQZYGU1H+D5yeguKBDkyIOf2zkwZKpvxdCwj3g
	REA6Qj7H3TH/2JHw11lz9+nSs/A5B/B0V+/sXP5v131ZJEU4BdhEbDjxg17Ah98FEA7IsUxxoQq
	eUyDBkAa7QGGPJafCEK+pv5Ey27g=
X-Google-Smtp-Source: AGHT+IFcrjZG7MB3VPHR1I9O032xb/tXtJbmdYzG4p0U+Y7TJbh86vq64pppfUltlW8JQdXVmlyIAuUCe9Swx1vGdPs=
X-Received: by 2002:a5d:5f85:0:b0:33b:4acb:9999 with SMTP id
 dr5-20020a5d5f85000000b0033b4acb9999mr5329371wrb.44.1707359148390; Wed, 07
 Feb 2024 18:25:48 -0800 (PST)
MIME-Version: 1.0
References: <20240201183024.145424-1-jandryuk@gmail.com> <20240201183024.145424-2-jandryuk@gmail.com>
 <cd9e0194-5814-4735-bca4-df2577102c19@perard> <a0145f9d-654d-4336-b5bb-69883543b919@suse.com>
In-Reply-To: <a0145f9d-654d-4336-b5bb-69883543b919@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Wed, 7 Feb 2024 21:25:36 -0500
Message-ID: <CAKf6xptDuYr2mT0w39Ezb3gTETYoZBpFUQ1Zqe6Kb9UeRQw7EQ@mail.gmail.com>
Subject: Re: [PATCH 1/4] block-common: Fix same_vm for no targets
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony PERARD <anthony.perard@citrix.com>, xen-devel@lists.xenproject.org, 
	Wei Liu <wl@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 7, 2024 at 7:50=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 06.02.2024 12:45, Anthony PERARD wrote:
> > On Thu, Feb 01, 2024 at 01:30:21PM -0500, Jason Andryuk wrote:
> >> same_vm is broken when the two main domains do not have targets.  otvm
> >> and targetvm are both missing, which means they get set to -1 and then
> >> converted to empty strings:
> >>
> >> ++10697+ local targetvm=3D-1
> >> ++10697+ local otvm=3D-1
> >> ++10697+ otvm=3D
> >> ++10697+ othervm=3D/vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4
> >> ++10697+ targetvm=3D
> >> ++10697+ local frontend_uuid=3D/vm/844dea4e-44f8-4e3e-8145-325132a31ca=
5
> >>
> >> The final comparison returns true since the two empty strings match:
> >>
> >> ++10697+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 =3D /vm/cc97bc2f=
-3a91-43f7-8fbc-4cb92f90b4e4 -o '' =3D /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90=
b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 =3D '' -o '' =3D '' ']'
> >>
> >> Replace -1 with distinct strings indicating the lack of a value and
> >> remove the collescing to empty stings.  The strings themselves will no
> >> longer match, and that is correct.
> >>
> >> ++12364+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 =3D /vm/cc97bc2f=
-3a91-43f7-8fbc-4cb92f90b4e4 -o 'No target' =3D /vm/cc97bc2f-3a91-43f7-8fbc=
-4cb92f90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 =3D 'No other tar=
get' -o 'No target' =3D 'No other target' ']'
> >>
> >> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
> >
> > Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
>
> I've committed this, but I take the absence of a Fixes: tag as indication
> that this doesn't want/need backporting.

Hmmm, maybe this should have a Fixes.  Sorry I didn't investigate that
better before submission.

Looks like this would be the commit:
https://xenbits.xen.org/gitweb/?p=3Dxen.git;a=3Dcommitdiff;h=3Df3a7ca02400d=
1c416e97451b4aebfaf608fc8192

f3a7ca02400d1 ("hotplug/Linux: fix same_vm check in block script")

I need to circle back on this.  IIRC, when I set up a conflicting
assignment of a writable disk to two VMs with block-tap, it was
allowed and not denied.  That is what prompted this change.

I'll have to double check there isn't something in the regular block
that might prevent that.

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 04:32:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 04:32:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677980.1054951 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXw5S-0001bi-2N; Thu, 08 Feb 2024 04:32:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677980.1054951; Thu, 08 Feb 2024 04:32:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXw5R-0001bb-W6; Thu, 08 Feb 2024 04:32:33 +0000
Received: by outflank-mailman (input) for mailman id 677980;
 Thu, 08 Feb 2024 04:32:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gCg7=JR=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXw5P-0001bV-Uf
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 04:32:31 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0f877d14-c63b-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 05:32:28 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-5116ba772dcso1125045e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 20:32:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f877d14-c63b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707366745; x=1707971545; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=bLrj5IveP/vEqO7sfcC1oMCN9R0hIR+aMlfjkSVMrTU=;
        b=HDE4RU+s+y9vkPVs14wC7FMyqQC74mqRhGA/H861UiWy0Mq+XryWNyQWxeAAVvNly4
         XuuK9+w/ooTqBTd7wHh0dfHYMiwRgS7CYU7QyEporAFiMPWWAtyAwW9KJQooiwcBaJOq
         LxPWnL0rLPWbLMS8l3KdmE/DVgNq4rbw288xE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707366745; x=1707971545;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=bLrj5IveP/vEqO7sfcC1oMCN9R0hIR+aMlfjkSVMrTU=;
        b=JTuB6h4g1YWvQw7wfKY0j9YFHcjvOBwE6JsWo9Qo+b3BKNKauBdwJZmZt6GvRfE1Af
         44XvPLnnV1cZVVcO9zv4e3Yx1UIsVhHgAnkbHyODLSiXp5KP9dxiGH/8yA2Y1YyAz0zI
         oWpIzQfmYSUQyO8sglbv+7aIM0XIJzsHH4/9UGaXVtqvzCcFxIexEMdo1CCPgkwPkAWt
         RjVUlt2o5m7rsn+aeFyq3W92+CBXHFnTefKx3R+fNfvQzeueQq/+EC3dSvyxqq/NAtsz
         PdET/nE0kjn5HkBnYzbl7iyDt1jCCZRwA1jqQVZopRAIHl3bP41Cwa88Yy2U/+xvwiVh
         D4Pw==
X-Forwarded-Encrypted: i=1; AJvYcCWjtzmo74QWBT8XDvHPC4AmGyWdqlyygpd2JPJCVbqAErIAMo5LA+f5Gi139ifTya3q6YFSgH5Az/y6lIo6hDguzjT+m+REtK5LdWp8RuE=
X-Gm-Message-State: AOJu0YxPgoZA95PMOVb8DWhbhAOTBS3uxprxj/Ks8MDZXnCjZ3UfcT/0
	kaoDbAZRH+horjwvOHc0xiiLHkCmlaKH6zv1CjtGv9jCa5n/0LLsGYnXTu+q7h3yAFeXkyuIZEp
	OHebzpUw8V3JT7CmGpxSu3iO3l3U0iCqPQkB4sQ==
X-Google-Smtp-Source: AGHT+IEZzZlfVyEQYBNYIf9/kYGrGOIl9/4R49z6p7u5xBBP4X3qaveiFdXQg6RJyzrBtxGK6BhaaVhgvXszLKxkuqk=
X-Received: by 2002:a05:6512:10ca:b0:511:693c:976c with SMTP id
 k10-20020a05651210ca00b00511693c976cmr2087664lfg.47.1707366745623; Wed, 07
 Feb 2024 20:32:25 -0800 (PST)
MIME-Version: 1.0
References: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
 <CA+zSX=beYAGb81zqD19YNAV3cXPFTk-7V68XYn59Eb3mGFH_Jg@mail.gmail.com>
 <507f29cf-7a07-4f3d-81e8-2236a8380b1a@citrix.com> <CABfawh=6AHZzH1aBP2A=oRUTvjVmgd6QQNxEQAwGCBV_9-CTcA@mail.gmail.com>
In-Reply-To: <CABfawh=6AHZzH1aBP2A=oRUTvjVmgd6QQNxEQAwGCBV_9-CTcA@mail.gmail.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Thu, 8 Feb 2024 04:32:13 +0000
Message-ID: <CA+zSX=Zou19sjb8ToLTaMjLHLYmqptsMXCT50Y9MwGykOaTrZQ@mail.gmail.com>
Subject: Re: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor ap2m->default_access
To: Tamas K Lengyel <tamas@tklengyel.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>, 
	xen-devel@lists.xenproject.org, Alexandru Isaila <aisaila@bitdefender.com>, 
	Petre Pircalabu <ppircalabu@bitdefender.com>, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 8, 2024 at 9:21=E2=80=AFAM Tamas K Lengyel <tamas@tklengyel.com=
> wrote:
>
> On Wed, Feb 7, 2024 at 5:21=E2=80=AFPM Andrew Cooper <andrew.cooper3@citr=
ix.com> wrote:
> >
> > On 07/02/2024 1:18 am, George Dunlap wrote:
> > > On Tue, Feb 6, 2024 at 6:08=E2=80=AFPM Petr Bene=C5=A1 <w1benny@gmail=
.com> wrote:
> > >> From: Petr Bene=C5=A1 <w1benny@gmail.com>
> > >>
> > >> This patch addresses a behavior discrepancy in the handling of altp2=
m views,
> > >> where upon the creation and subsequent EPT violation, the page acces=
s
> > >> permissions were incorrectly inherited from the hostp2m instead of r=
especting
> > >> the altp2m default_access.
> > >>
> > >> Previously, when a new altp2m view was established with restrictive
> > >> default_access permissions and activated via xc_altp2m_switch_to_vie=
w(),
> > >> it failed to trigger an event on the first access violation.  This b=
ehavior
> > >> diverged from the intended mechanism, where the altp2m's default_acc=
ess
> > >> should dictate the initial permissions, ensuring proper event trigge=
ring on
> > >> access violations.
> > >>
> > >> The correction involves modifying the handling mechanism to respect =
the
> > >> altp2m view's default_access upon its activation, eliminating the ne=
ed for
> > >> setting memory access permissions for the entire altp2m range (e.g. =
within
> > >> xen-access.c).  This change not only aligns the behavior with the ex=
pected
> > >> access control logic but also results in a significant performance i=
mprovement
> > >> by reducing the overhead associated with setting memory access permi=
ssions
> > >> across the altp2m range.
> > >>
> > >> Signed-off-by: Petr Bene=C5=A1 <w1benny@gmail.com>
> > > Thanks Petr, this looks like a great change.
> > >
> > > Two things:
> > >
> > > - Probably worth adjusting the comment at the top of
> > > p2m_altp2m_get_or_propagate to mention that you use the altp2m
> > > default_access when propagating from the host p2m
> > >
> > > - This represents a change in behavior, so probably at least worth a
> > > mention in CHANGELOG.md?
> >
> > This is a bugfix to an tech preview feature.  It's not remotely close t=
o
> > CHANGELOG material.
> >
> > Tangent.  SUPPORT.md says tech preview on ARM, despite there being no
> > support in the slightest.  Patches were posted and never taken.
> >
> > > Tamas, I guess this is OK from an interface compatibility point of
> > > view?  In theory it should always have been behaving this way.
> >
> > Given the already-provided Ack, I expect that has been considered and
> > deemed ok.
>
> Correct, this is just a bugfix.

Er, ok, just one more comment: this could allow an altp2m to have more
permissions than the host; for example, the host p2m entry could be
p2m_access_r, but if the altp2m's default_access were p2m_access_rw,
it would override that.  Is that the behavior we want?  Or do we want
to do some sort of intersection of permissions?

If the former, I'd propose the comment be adjusted thus:

 * If the entry is invalid, and the host entry was valid, propagate
 * the host's entry to the altp2m, retaining page order but using the
 * altp2m's default_access, and indicate that the caller should re-try
 * the faulting instruction.

I could do that on check-in.

 -George


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 05:42:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 05:42:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677984.1054961 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXxAX-0001O6-50; Thu, 08 Feb 2024 05:41:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677984.1054961; Thu, 08 Feb 2024 05:41:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXxAX-0001Nz-0K; Thu, 08 Feb 2024 05:41:53 +0000
Received: by outflank-mailman (input) for mailman id 677984;
 Thu, 08 Feb 2024 05:41:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXxAV-0001Np-Iy; Thu, 08 Feb 2024 05:41:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXxAV-0006Qd-5x; Thu, 08 Feb 2024 05:41:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXxAU-0002yk-KU; Thu, 08 Feb 2024 05:41:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXxAU-0007bW-Jo; Thu, 08 Feb 2024 05:41:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=5qPW0VuAn/Ktt8R95ZqzWryINfi8YPMpmkfBRPNfQ7Y=; b=IVpA+lXSW/i2bA8v5ygCYiz0jY
	/2LmJxEsgR6kCTeaKMcVUBL6ghOCN15EMUdg/k6SGW3N50G6oodjAq/uVNzHOllQ+lAaSsdp0A3Dk
	j/HZBpT/unPhY5ezrzE2MjA6Et1cu+ejvXhycr/ds2s2ZWhvph2Eji5bZ9m9kyUvo/uU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184620-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184620: regressions - trouble: broken/fail/pass
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-multivcpu:<job status>:broken:regression
    linux-linus:test-armhf-armhf-xl-multivcpu:host-install(5):broken:regression
    linux-linus:test-arm64-arm64-xl-credit1:guest-start/debian.repeat:fail:regression
    linux-linus:test-armhf-armhf-xl-rtds:host-ping-check-xen:fail:allowable
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=547ab8fc4cb04a1a6b34377dd8fad34cd2c8a8e3
X-Osstest-Versions-That:
    linux=6d280f4d760e3bcb4a8df302afebf085b65ec982
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 08 Feb 2024 05:41:50 +0000

flight 184620 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184620/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-multivcpu    <job status>                 broken
 test-armhf-armhf-xl-multivcpu  5 host-install(5)       broken REGR. vs. 184614
 test-arm64-arm64-xl-credit1 18 guest-start/debian.repeat fail REGR. vs. 184614

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds     10 host-ping-check-xen      fail REGR. vs. 184614

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184614
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184614
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184614
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184614
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184614
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184614
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184614
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184614
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                547ab8fc4cb04a1a6b34377dd8fad34cd2c8a8e3
baseline version:
 linux                6d280f4d760e3bcb4a8df302afebf085b65ec982

Last test of basis   184614  2024-02-07 08:42:21 Z    0 days
Testing same since   184620  2024-02-07 19:40:30 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anup Patel <anup@brainfault.org>
  Anup Patel <apatel@ventanamicro.com>
  Bibo Mao <maobibo@loongson.cn>
  Borislav Petkov (AMD) <bp@alien8.de>
  Christian Borntraeger <borntraeger@linux.ibm.com>
  Chuck Lever <chuck.lever@oracle.com>
  David Rientjes <rientjes@google.com>
  Eric Farman <farman@linux.ibm.com>
  Huacai Chen <chenhuacai@loongson.cn>
  Janosch Frank <frankja@linux.ibm.com>
  Kees Cook <keescook@chromium.org>
  Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Nathan Chancellor <nathan@kernel.org>
  NeilBrown <neilb@suse.de>
  Paolo Bonzini <pbonzini@redhat.com>
  Tengfei Yu <moehanabichan@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  fail    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                broken  
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-armhf-armhf-xl-multivcpu broken
broken-step test-armhf-armhf-xl-multivcpu host-install(5)

Not pushing.

(No revision log; it would be 652 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 06:33:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 06:33:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677991.1054971 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXxy8-0007JG-19; Thu, 08 Feb 2024 06:33:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677991.1054971; Thu, 08 Feb 2024 06:33:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXxy7-0007J9-UJ; Thu, 08 Feb 2024 06:33:07 +0000
Received: by outflank-mailman (input) for mailman id 677991;
 Thu, 08 Feb 2024 06:33:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=T15F=JR=gmail.com=dunlapg@srs-se1.protection.inumbo.net>)
 id 1rXxy6-0007J3-R8
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 06:33:06 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e94f0fab-c64b-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 07:33:03 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-4103be6e938so1397235e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 22:33:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e94f0fab-c64b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=umich.edu; s=google-2016-06-03; t=1707373983; x=1707978783; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=RfUmzfcC+F+NtcyZx7qulQ0OpdfLAcQzOcDyTiRj2YM=;
        b=JDTreakDaWdPxhnZAjWJDugTBjfYWyo5tst3xWGv8+LHVB2IDlrS55JMVUKxiPH+Hw
         ekYtipGl9BjIB0CoqeLMqwFgvBagVA2s2nvaOpPq3KgGyfIyvamyk0godAVSlVjwrZ5o
         Ts7Lyuuo64qOGB0VwpC7100iH56nI/1bNyQ02fc2f6xxbaMPuZdiADp6uTp6y4f3nMD9
         rscwZyn5WbRyKwUiqQyRLvT27sgti9PtKBY8J6cFOuQKxhBkEHi/lwzxXCBnKSZsZ0hA
         vot++vVun5puFXfJuhMfDBFHXI+/+QKipdsFFDYUsx9sns/FuocAAVsBRyuZjlHHwRZt
         QYYQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707373983; x=1707978783;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=RfUmzfcC+F+NtcyZx7qulQ0OpdfLAcQzOcDyTiRj2YM=;
        b=eenJhOm2jFxadfqi4CbsqTgWVXoBJqIO4xBUtczgnyH7kaFU9FPZ/aiEcE8Ckev6H8
         0E6lHbnjDnauMC2SqV7ZUYZWR6iVFUXG0ykKHeuK5/TdLzuq1ho2XN/9OahMsJEJwSbK
         KDuQzZTdLkesFj6gR6RqbUyjUhXgkq/Hwul5OahmLdGwUvgTTCSLtmW8cRlglXZGy4s8
         oAxap1LpKUyS1zpaQSoaYylXoV/ME7U5Rhos4HZfiaLE9n8yHlfa1nGaUCfRqipPurzj
         MgNr530BHKdIqxVzWHam51hQPW0GPC4M+tafwZ/PQh/WZH+OY61E2IOXFZTN0GSlJzU8
         f3xw==
X-Gm-Message-State: AOJu0YzpJYPD2+9PUl7K2BchqwNC+ZedJPMtGF3aJ0VSAJYaaqNv1RFJ
	YjsbAZL8DbStKYtY9iWXs3EriE7mQgDKiRno6scMrXTbIVr7E19FdT3c49iet7Sg+T6knSro6vk
	gH7KDAFJf5Kye/Ft49yFlW0LVahM=
X-Google-Smtp-Source: AGHT+IF+LezDZx8vNVaQN9DjDCYTb+tzJSx96Hcxd97dModPN7CtIpRcUJPAZSEOhdtfcE7UkggdLTgw33pek2JILs4=
X-Received: by 2002:a5d:674f:0:b0:33a:e791:d8ad with SMTP id
 l15-20020a5d674f000000b0033ae791d8admr4882895wrw.4.1707373982804; Wed, 07 Feb
 2024 22:33:02 -0800 (PST)
MIME-Version: 1.0
References: <5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com> <8d5df1f4-74ca-27cc-99f0-7e7a82050de1@suse.com>
In-Reply-To: <8d5df1f4-74ca-27cc-99f0-7e7a82050de1@suse.com>
From: George Dunlap <dunlapg@umich.edu>
Date: Thu, 8 Feb 2024 06:32:50 +0000
Message-ID: <CAFLBxZbO2-f3rdgVhu2j_7sRWSeL5oMZt6pit=6nRvhPn=pqoQ@mail.gmail.com>
Subject: Re: [PATCH v2 1/2] x86/p2m: make p2m_get_page_from_gfn() handle grant
 case correctly
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	George Dunlap <george.dunlap@cloud.com>
Content-Type: multipart/alternative; boundary="000000000000be729a0610d8fb0f"

--000000000000be729a0610d8fb0f
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Jun 23, 2022 at 7:54=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:

> Grant P2M entries, which are covered by p2m_is_any_ram(), wouldn't pass
> the get_page() unless the grant was a local one. These need to take the
> same path as foreign entries. Just the assertion there is not valid for
> local grants, and hence it triggering needs to be avoided.
>

I think I'd say:

---
The 'fast' path of p2m_get_page_from_gfn handles three cases: normal ram,
foreign p2m entries, and grant map entries.  For normal ram and grant table
entries, get_page() is called, but for foreign entries,
page_get_owner_and_reference() is called, since the current domain is
expected not to be the owner.

Unfortunately, grant maps are *also* generally expected to be owned by
foreign domains; so this function will fail for any p2m entry containing a
grant map that doesn't happen to be local.

Have grant maps take the same path as foreign entries.  Since grants may
actually be either foreign or local, adjust the assertion to allow for this=
.
---

One more comment...


> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> Using | instead of || helps the compiler fold the two p2m_is_*().
> ---
> v2: The shared case was fine; limit to grant adjustment.
>
> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -357,11 +357,11 @@ struct page_info *p2m_get_page_from_gfn(
>               && !((q & P2M_UNSHARE) && p2m_is_shared(*t)) )
>          {
>              page =3D mfn_to_page(mfn);
> -            if ( unlikely(p2m_is_foreign(*t)) )
> +            if ( unlikely(p2m_is_foreign(*t) | p2m_is_grant(*t)) )
>

I'm not a fan of this.  If you replace it with || you can have my R-b
immediately; otherwise we'll have to wait until we can discuss our general
policy on this sort of thing at the x86 maintainer's call.

 -George

--000000000000be729a0610d8fb0f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Thu, Jun 23, 2022 at 7:54=E2=80=AF=
PM Jan Beulich &lt;<a href=3D"mailto:jbeulich@suse.com">jbeulich@suse.com</=
a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0p=
x 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Gr=
ant P2M entries, which are covered by p2m_is_any_ram(), wouldn&#39;t pass<b=
r>
the get_page() unless the grant was a local one. These need to take the<br>
same path as foreign entries. Just the assertion there is not valid for<br>
local grants, and hence it triggering needs to be avoided.<br></blockquote>=
<div><br></div><div>I think I&#39;d say:</div><div><br></div><div>---</div>=
<div>The &#39;fast&#39; path of p2m_get_page_from_gfn handles three cases: =
normal ram, foreign p2m entries, and grant map entries.=C2=A0 For normal ra=
m and grant table entries, get_page() is called, but for foreign entries, p=
age_get_owner_and_reference() is called, since the current domain is expect=
ed not to be the owner.</div><div><br></div><div>Unfortunately, grant maps =
are *also* generally expected to be owned by foreign domains; so this funct=
ion will fail for any p2m entry containing a grant map that doesn&#39;t hap=
pen to be local.</div><div><br></div><div>Have grant maps take the same pat=
h as foreign entries.=C2=A0 Since grants may actually be either foreign or =
local, adjust the assertion to allow for this.</div><div>---</div><div>=C2=
=A0</div><div>One more comment...</div><div><br></div><blockquote class=3D"=
gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20=
4,204,204);padding-left:1ex">
<br>
Signed-off-by: Jan Beulich &lt;<a href=3D"mailto:jbeulich@suse.com" target=
=3D"_blank">jbeulich@suse.com</a>&gt;<br>
---<br>
Using | instead of || helps the compiler fold the two p2m_is_*().<br>
---<br>
v2: The shared case was fine; limit to grant adjustment.<br>
<br>
--- a/xen/arch/x86/mm/p2m.c<br>
+++ b/xen/arch/x86/mm/p2m.c<br>
@@ -357,11 +357,11 @@ struct page_info *p2m_get_page_from_gfn(<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 &amp;&amp; !((q &amp; P2M_=
UNSHARE) &amp;&amp; p2m_is_shared(*t)) )<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0page =3D mfn_to_page(mfn);<=
br>
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if ( unlikely(p2m_is_foreign(*t)=
) )<br>
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if ( unlikely(p2m_is_foreign(*t)=
 | p2m_is_grant(*t)) )<br></blockquote><div><br></div><div>I&#39;m not a fa=
n of this.=C2=A0 If you replace it with || you can have my R-b immediately;=
 otherwise we&#39;ll have to wait until we can discuss our general policy o=
n this sort of thing at the x86 maintainer&#39;s call.</div><div><br></div>=
<div>=C2=A0-George</div></div></div>

--000000000000be729a0610d8fb0f--


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 07:08:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 07:08:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.677995.1054980 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXyWD-0002ZK-I5; Thu, 08 Feb 2024 07:08:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 677995.1054980; Thu, 08 Feb 2024 07:08:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXyWD-0002ZD-FQ; Thu, 08 Feb 2024 07:08:21 +0000
Received: by outflank-mailman (input) for mailman id 677995;
 Thu, 08 Feb 2024 07:08:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXyWC-0002Z3-83; Thu, 08 Feb 2024 07:08:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXyWB-0008AM-T1; Thu, 08 Feb 2024 07:08:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rXyWB-0007Y2-Ff; Thu, 08 Feb 2024 07:08:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rXyWB-0007ej-F6; Thu, 08 Feb 2024 07:08:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=24glUhnC94L/gBKCzKHCtBdTr5SdFSSbwG1SFn7aCYY=; b=Yt+0U2Q/E36BnK37YQLgzIPWa3
	1jjyjFfacS4qkV60lv3lG1TXhZGjlzY8BeMbdQP8cvxiD41NE8OTnIgzprbtJQGOOrwXSJdOvzbWd
	i8yVHVF0goUPFqyb4O6Qxx42mvGsTTxk/A0nDICIAPq48qthWajeQZj/6uzN47hnN/W4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184623-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184623: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=8f316e99ec8de9dea294f6751dd7457f9f1a828c
X-Osstest-Versions-That:
    ovmf=4d1f0babe20cf757897fa43c399fd79bb6aa8a30
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 08 Feb 2024 07:08:19 +0000

flight 184623 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184623/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 8f316e99ec8de9dea294f6751dd7457f9f1a828c
baseline version:
 ovmf                 4d1f0babe20cf757897fa43c399fd79bb6aa8a30

Last test of basis   184619  2024-02-07 19:12:58 Z    0 days
Testing same since   184623  2024-02-08 04:13:04 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ashraf Ali S <ashraf.ali.s@intel.com>
  devel@edk2.groups.io <devel@edk2.groups.io>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   4d1f0babe2..8f316e99ec  8f316e99ec8de9dea294f6751dd7457f9f1a828c -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 07:29:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 07:29:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678001.1054990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXyqO-000564-85; Thu, 08 Feb 2024 07:29:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678001.1054990; Thu, 08 Feb 2024 07:29:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXyqO-00055x-5O; Thu, 08 Feb 2024 07:29:12 +0000
Received: by outflank-mailman (input) for mailman id 678001;
 Thu, 08 Feb 2024 07:29:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXyqM-00055r-GL
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 07:29:10 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bf134073-c653-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 08:29:09 +0100 (CET)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-3392b12dd21so952276f8f.0
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 23:29:09 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bq8-20020a5d5a08000000b0033b55186901sm1243690wrb.94.2024.02.07.23.29.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 23:29:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf134073-c653-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707377348; x=1707982148; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vfjQHRPN3metDdrirwzAwxK0e7/WGzBowxCDPw/EbP4=;
        b=INrv8cxze+QA86beOJxOE3yMqSyzWnLQ6OixtF3KTLZicap4hTHDdnN1VzSj0w09+6
         luAlpOkeT9Bf+eTjTfwI2Mdco7YM1q+otKU4AwaYLoZLK9VExno8rRAp76ZvVyzuvny/
         AuLc0MhH7EKL2eW6zn6Mu6ivJFZ7MTwOTwaWiW2pOByEBR+isBkyT/iJiDLFOAM6+5NH
         ettv/wG9M4qFibMCyGZcfcpuOK0ebY2pdegsG0KOeMvnd8Ukz9wqQoOIjgPEMDjHpcXa
         AdSfsF8qwv/xwzNQQzmnf/H9iQwXPNz7matzV3S8nu9C3obM06OZUuudt4ECrfBbG/Hi
         L5pQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707377348; x=1707982148;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vfjQHRPN3metDdrirwzAwxK0e7/WGzBowxCDPw/EbP4=;
        b=m9AiAILR9NxXdVRdD3v5/5nctPNsVe5dGzFSsYNEeeREznJHGkQzMC8iDGJJmwpZ9U
         JL5FLHIvXrf2DeHvhhcjPBruLVWlbcmLUt6df29CHaNuuVp0idWxyivSNHkTD8OssiSc
         RsqvKgLD+KeefJHNurzGDSSvNU5WFU898PKmLdp/d5p/u+UIEtMDF4ngBqTviBHmUfCa
         lfKKMRBBgP3NXFSTG9HSB1UdsCtNkq/efjIr9opBImmJ3XpHVOW7MLBnkO6jJfmYq08D
         3yVHgF92UGyYYMXJ8hW4SE/v/qxfl85nw95jE9Izsu9fgXesuE1fxyqTUQWD/Z1eBFEd
         nZsw==
X-Gm-Message-State: AOJu0YyykdmOtkySS7rgd3iIv7iqqmOGSeqqHof4DUzmHGqcrdbTqi5h
	RZ1yld954WLXLDyj8cEI9+ST/1VbHo+oJFX7uYjtdm5FZXswGtnghd8Xfs38hQ==
X-Google-Smtp-Source: AGHT+IEmDUBeQgKKCT9mdpB9+e++W9/X9MZX6yqxefpE0AWm1Z0Hqpp8l4j6aa/GjMLitWWmhs2ApQ==
X-Received: by 2002:a5d:634e:0:b0:33a:edf8:b8e6 with SMTP id b14-20020a5d634e000000b0033aedf8b8e6mr4685967wrw.58.1707377348125;
        Wed, 07 Feb 2024 23:29:08 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWIRcCwDjA+mo7vfknLPuzkGyss9vN4mVn/cDhFs5SOz6q4krvoTTopMqpqEogJzAmGamh3uh/FUiORTI/fD64vKnFaEjPc2NuJCtnAGpNurBajhyvdFZAhA3DJNVGCnOwtc6r7ybQ0xAeV7dMV9icj
Message-ID: <0d776dcc-af14-436a-bf8b-9bfb39b787b8@suse.com>
Date: Thu, 8 Feb 2024 08:29:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 3/9] x86/smp: drop x86_cpu_to_apicid, use
 cpu_data[cpu].apicid instead
Content-Language: en-US
To: Krystian Hebel <krystian.hebel@3mdeb.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <8121d9b472b305be751158aa3af3fed98ff0572e.1699982111.git.krystian.hebel@3mdeb.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <8121d9b472b305be751158aa3af3fed98ff0572e.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.11.2023 18:50, Krystian Hebel wrote:
> Both fields held the same data.

Supposedly the same data only. They come from different origins, and you're
hiding this quite well by leaving all sites in place where the field is
written. Both items are also used for entirely separate purposes. So you
need to
- explain why both sources of information necessarily provide the same
  data,
- especially if there's remaining concern from the above explanation that
  the two values might end up different in corner cases (running
  virtualized ourselves comes to mind as a possible example), explain why
  nevertheless it is fine (risk free) to use the consolidated item for
  all of the originally separate purposes,
- either explain or do away with the multiple places setting this single
  remaining field.

> --- a/xen/arch/x86/smpboot.c
> +++ b/xen/arch/x86/smpboot.c
> @@ -61,10 +61,8 @@ unsigned int __read_mostly nr_sockets;
>  cpumask_t **__read_mostly socket_cpumask;
>  static cpumask_t *secondary_socket_cpumask;
>  
> -struct cpuinfo_x86 cpu_data[NR_CPUS];
> -
> -u32 x86_cpu_to_apicid[NR_CPUS] __read_mostly =
> -	{ [0 ... NR_CPUS-1] = BAD_APICID };
> +struct cpuinfo_x86 cpu_data[NR_CPUS] =
> +        { [0 ... NR_CPUS-1] .apicid = BAD_APICID };

Nit: Stray blank after closing square bracket.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 07:46:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 07:46:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678006.1055000 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXz71-0007u5-La; Thu, 08 Feb 2024 07:46:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678006.1055000; Thu, 08 Feb 2024 07:46:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXz71-0007ty-IH; Thu, 08 Feb 2024 07:46:23 +0000
Received: by outflank-mailman (input) for mailman id 678006;
 Thu, 08 Feb 2024 07:46:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXz6z-0007tb-Ue
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 07:46:21 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 25714793-c656-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 08:46:19 +0100 (CET)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-33b13332ca7so843713f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 23:46:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g2-20020adff3c2000000b00337d5cd0d8asm3013827wrp.90.2024.02.07.23.46.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 23:46:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25714793-c656-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707378379; x=1707983179; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=lE1cibqb+0alDvGvEMZl/lOukYeRYjy3aGf4NS6gOCA=;
        b=e+oRRfv8ebbjYco8ACQODap7UB+WiWNDWbJ01hMTUqaAkuCVJ/01dz0+mHANhzdbe1
         or7cynyXRHzboorQfmxDi7AAkLAKi9wAvUPgBC0pmmEzSBRtQG75dsV+Hyu131PaXVXR
         l4konenl6bHwGJl3y+M1XlXufcXVvCswJClf6uelai/RejfBxHl5ZbH8tE5au8CsDlJ1
         YE+6Top/eekDU4zljSNbcmiL1E/WT3OIm0jcQFd2UKNNVvGBfLUBLn6xUkIOpMoTrOO8
         K6AqvFiit6ONfKHKbK6uO1ONKf1e8IDnaTtoNYDbPuPxuIgo2wbc08MJBDieHX1Hw5SE
         DUaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707378379; x=1707983179;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=lE1cibqb+0alDvGvEMZl/lOukYeRYjy3aGf4NS6gOCA=;
        b=PSr374XSFv3bCDXfhvNsd9//moZFBNHNRCBRQxm2rFSGE8D+DcP12r5kHcfbqcEOnI
         SpZ6sHNGWABdCoq8fIoNLbnDAaM8B8F9U21lwT6o0iVUbPa/WfC+4LTWjjgk/u7pXByE
         oLFugHNIWiT0InMtiRCGMIsvjYClMudrQXs09WIyUEStyvbt19zZPVWJW+ElVBAOajX5
         vjocTIQf2HhWjbBPSrWSEHvOCEZvoT0pdhC1EJwX3iLeat6aVMyi2J4ufvF40ZyVcWcC
         KN0s04pKMbY90SX1763eGoh/O7xHjTmwSvSw2BFJVGSsx1wlOtYmCfEiWgM4PUBTJIxO
         7Eng==
X-Gm-Message-State: AOJu0YwBbZIR5SwAfyk+d6uKFko8HjMC1mIFbbO7LzuSZhCyhOWrrMsm
	1oPTJNcGXDNdTacbSzR5XRISXeCNTpQ763eKRKCDnk2Ux4BbZ5RkVB5wi7FGBA==
X-Google-Smtp-Source: AGHT+IF/K2UjwB5v7Rv/7I96J9LGiCChvsd/FYlS54QQ9sXjUKN/wpjVreWYDMjM3oM1rJIaUDXiqw==
X-Received: by 2002:adf:f38e:0:b0:33b:5a6c:330e with SMTP id m14-20020adff38e000000b0033b5a6c330emr398490wro.2.1707378378831;
        Wed, 07 Feb 2024 23:46:18 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUEKzWUJd3q/faJ3daOkQLt525PoRO9hdfc+IPNiPuAhT2Tdlnf7L5HiduShtsolxur4oy3hv/dNPBiu4wNtQnOHmvBY1fCdgbKbv06I4B/jp6x1N+UP1B7Ppwu4mO1+bylOvlFjUSx21fFfgdlGp9tKkYa9YLBofjPhwCGvI2goIdzBylz+sdrRtk0nvHwOQzdqE4id1zA1xgwBYkaSSgA4dKozTk2nenZCGabQXmiokbqKmOIycsqA8mpROpii3eZcVZ68BTwmmwWMokivV5hGs+R5OBoRs4bX7PZB7X2xNR7y68mra43
Message-ID: <156f2693-e178-4553-b4b4-f671b0221dfc@suse.com>
Date: Thu, 8 Feb 2024 08:46:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor
 ap2m->default_access
To: George Dunlap <george.dunlap@cloud.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, =?UTF-8?Q?Petr_Bene=C5=A1?=
 <w1benny@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 xen-devel@lists.xenproject.org, Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
 <CA+zSX=beYAGb81zqD19YNAV3cXPFTk-7V68XYn59Eb3mGFH_Jg@mail.gmail.com>
 <507f29cf-7a07-4f3d-81e8-2236a8380b1a@citrix.com>
 <CABfawh=6AHZzH1aBP2A=oRUTvjVmgd6QQNxEQAwGCBV_9-CTcA@mail.gmail.com>
 <CA+zSX=Zou19sjb8ToLTaMjLHLYmqptsMXCT50Y9MwGykOaTrZQ@mail.gmail.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CA+zSX=Zou19sjb8ToLTaMjLHLYmqptsMXCT50Y9MwGykOaTrZQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.02.2024 05:32, George Dunlap wrote:
> Er, ok, just one more comment: this could allow an altp2m to have more
> permissions than the host; for example, the host p2m entry could be
> p2m_access_r, but if the altp2m's default_access were p2m_access_rw,
> it would override that.  Is that the behavior we want?  Or do we want
> to do some sort of intersection of permissions?
> 
> If the former, I'd propose the comment be adjusted thus:
> 
>  * If the entry is invalid, and the host entry was valid, propagate
>  * the host's entry to the altp2m, retaining page order but using the
>  * altp2m's default_access, and indicate that the caller should re-try
>  * the faulting instruction.

I find it highly questionable that such blind overriding should be taking
place.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 07:50:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 07:50:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678010.1055012 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXzAs-00015s-6p; Thu, 08 Feb 2024 07:50:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678010.1055012; Thu, 08 Feb 2024 07:50:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXzAs-00015l-2T; Thu, 08 Feb 2024 07:50:22 +0000
Received: by outflank-mailman (input) for mailman id 678010;
 Thu, 08 Feb 2024 07:50:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXzAr-00015f-Ii
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 07:50:21 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b4f33ad2-c656-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 08:50:20 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-4101eb5a115so10501255e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 07 Feb 2024 23:50:20 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g2-20020adff3c2000000b00337d5cd0d8asm3019916wrp.90.2024.02.07.23.50.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 07 Feb 2024 23:50:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b4f33ad2-c656-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707378619; x=1707983419; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=TB1ao47PA2wE8Mgp6SW7hVzOtec7VfK8GjzF9zx5AOw=;
        b=SZ7S/sLzF8cqcVK1AGmoY+N29O3fmO/NzYUA4yoDTsiaFwiLM6B6u1n2UIiOHcs1hR
         sDivILuX+gUMJslpWuCFvcD3Ek4ZghzjuAacZD31dg4Jv9rrPVXzJIZXFyjhYiWh1i9z
         C0Q/Dj4gpOYU4iVO8D6VmbpaDSJAToNiMxFXiBM/FTlJMvqPb2TIjyslpHO3voIgylKS
         qsJAo3u/Ys3QbjU5XtHEtW3vsFPLUiawKolKtPS5vLKQGsXS0+8psCW20E55c8r0fYB4
         XdONoabVLnrXhzpkV9Nj7S68EuTGmzXfTNcxpRCLJu1/jHakIaqTIwhpnqqO7obAGm80
         OEYw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707378619; x=1707983419;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=TB1ao47PA2wE8Mgp6SW7hVzOtec7VfK8GjzF9zx5AOw=;
        b=jWtKLe4a3fTnjXTff7ukIbepNJDuvMfbkhlUBIb1S4jJnrg9fdmQRXYcj82eqyP5Hv
         PnXE9a0CmXM5UYRfZd+OjcI8uXApa6cMkmokv9FDM/ymbnn6dDSnsOl5RsjNSd/r7YH7
         Q5mHLdczFLbP4StmINSk/QVXdfoUQavKBqr84W2/TtqrMZb7sxPyBFcguI9FBsJmTaEH
         LqE6GEo6s3DoWYNV3Zg+Ajgq7201GM+W43GAWWQMn9lGLs2oqvfeuXlujvSGWJhro867
         uCvKp71hWeBTh/vbMsSKLePIXXPbsNYjqKLS32kO+LsDMUbdvsVZEkOSzBhukTtJGdj1
         S5og==
X-Gm-Message-State: AOJu0YzRvYVP2hDx4PwJx369QztX14HRCAUXmgNoPKfxazIDnDT9BRAB
	lI8C2ZlLE61MJhJQ4lP0Q3fHz6WecOgAsuKODrs8Z/kX8rOyKx3lGTSspY+rhQ==
X-Google-Smtp-Source: AGHT+IEDyz92JcyTX/EXoPXoUl5EKBsdtymyMl5gui+p1sB3stm1hUz6TviPwsZKn6bKLy160985Cw==
X-Received: by 2002:a5d:53d1:0:b0:33b:10de:59a2 with SMTP id a17-20020a5d53d1000000b0033b10de59a2mr4818038wrw.15.1707378619603;
        Wed, 07 Feb 2024 23:50:19 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXKWRFWgh5pgmEsONn49zqH2KXbv4Ftk0yKkFEnXOC65Jy+W7IN65aQX55aQP5ZM7RVDYyJD6NJnIZ37pgIfj89PQhmr6Ig9/7OMZEpEzmR4l5Gv1CpodvMffA=
Message-ID: <714d7de9-5a6e-4d30-bd2d-5b84ae24f3dd@suse.com>
Date: Thu, 8 Feb 2024 08:50:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/4] block-common: Fix same_vm for no targets
Content-Language: en-US
To: Jason Andryuk <jandryuk@gmail.com>
Cc: Anthony PERARD <anthony.perard@citrix.com>,
 xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <20240201183024.145424-1-jandryuk@gmail.com>
 <20240201183024.145424-2-jandryuk@gmail.com>
 <cd9e0194-5814-4735-bca4-df2577102c19@perard>
 <a0145f9d-654d-4336-b5bb-69883543b919@suse.com>
 <CAKf6xptDuYr2mT0w39Ezb3gTETYoZBpFUQ1Zqe6Kb9UeRQw7EQ@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAKf6xptDuYr2mT0w39Ezb3gTETYoZBpFUQ1Zqe6Kb9UeRQw7EQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08.02.2024 03:25, Jason Andryuk wrote:
> On Wed, Feb 7, 2024 at 7:50 AM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 06.02.2024 12:45, Anthony PERARD wrote:
>>> On Thu, Feb 01, 2024 at 01:30:21PM -0500, Jason Andryuk wrote:
>>>> same_vm is broken when the two main domains do not have targets.  otvm
>>>> and targetvm are both missing, which means they get set to -1 and then
>>>> converted to empty strings:
>>>>
>>>> ++10697+ local targetvm=-1
>>>> ++10697+ local otvm=-1
>>>> ++10697+ otvm=
>>>> ++10697+ othervm=/vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4
>>>> ++10697+ targetvm=
>>>> ++10697+ local frontend_uuid=/vm/844dea4e-44f8-4e3e-8145-325132a31ca5
>>>>
>>>> The final comparison returns true since the two empty strings match:
>>>>
>>>> ++10697+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o '' = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = '' -o '' = '' ']'
>>>>
>>>> Replace -1 with distinct strings indicating the lack of a value and
>>>> remove the collescing to empty stings.  The strings themselves will no
>>>> longer match, and that is correct.
>>>>
>>>> ++12364+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o 'No target' = /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 = 'No other target' -o 'No target' = 'No other target' ']'
>>>>
>>>> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
>>>
>>> Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
>>
>> I've committed this, but I take the absence of a Fixes: tag as indication
>> that this doesn't want/need backporting.
> 
> Hmmm, maybe this should have a Fixes.  Sorry I didn't investigate that
> better before submission.
> 
> Looks like this would be the commit:
> https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=f3a7ca02400d1c416e97451b4aebfaf608fc8192
> 
> f3a7ca02400d1 ("hotplug/Linux: fix same_vm check in block script")
> 
> I need to circle back on this.  IIRC, when I set up a conflicting
> assignment of a writable disk to two VMs with block-tap, it was
> allowed and not denied.  That is what prompted this change.
> 
> I'll have to double check there isn't something in the regular block
> that might prevent that.

Okay, I'll wait for a result here before deciding whether to queue.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 07:55:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 07:55:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678014.1055021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXzGF-0001lM-Rr; Thu, 08 Feb 2024 07:55:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678014.1055021; Thu, 08 Feb 2024 07:55:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXzGF-0001lF-OL; Thu, 08 Feb 2024 07:55:55 +0000
Received: by outflank-mailman (input) for mailman id 678014;
 Thu, 08 Feb 2024 07:55:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=z5WF=JR=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rXzGE-0001l9-Lo
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 07:55:54 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7b8a708d-c657-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 08:55:53 +0100 (CET)
Received: from nico.bugseng.com (unknown [147.123.100.131])
 by support.bugseng.com (Postfix) with ESMTPSA id 4C9114EE0739;
 Thu,  8 Feb 2024 08:55:51 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b8a708d-c657-11ee-8a4a-1f161083a0e0
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	Nicola Vetrini <nicola.vetrini@bugseng.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH] docs/misra: add x86_64 and arm64 asm-offset.c to exclude-list
Date: Thu,  8 Feb 2024 08:55:47 +0100
Message-Id: <b0c855581eed247a32b745906f84d352bf812091.1707324479.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

These two files contain several deliberate violations of MISRA C rules and
they are not linked in the final Xen binary, therefore they can be exempted
from MISRA compliance.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
Since the exclude list only contains arm64 and x86 files I reasoned that
introducing an entry that would match all architectures would not be desirable
(e.g., arm32). I'm happy to change that, though.
---
 docs/misra/exclude-list.json | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/docs/misra/exclude-list.json b/docs/misra/exclude-list.json
index 7971d0e70f5b..bd05478e03e9 100644
--- a/docs/misra/exclude-list.json
+++ b/docs/misra/exclude-list.json
@@ -17,6 +17,10 @@
             "rel_path": "arch/arm/arm64/lib/find_next_bit.c",
             "comment": "Imported from Linux, ignore for now"
         },
+        {
+          "rel_path": "arch/arm/arm64/asm-offsets.c",
+          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
+        },
         {
             "rel_path": "arch/x86/acpi/boot.c",
             "comment": "Imported from Linux, ignore for now"
@@ -97,6 +101,10 @@
             "rel_path": "arch/x86/x86_64/mmconf-fam10h.c",
             "comment": "Imported from Linux, ignore for now"
         },
+        {
+          "rel_path": "arch/x86/x86_64/asm-offsets.c",
+          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
+        },
         {
             "rel_path": "arch/x86/efi/check.c",
             "comment": "The resulting code is not included in the final Xen binary, ignore for now"
-- 
2.34.1


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 08:02:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 08:02:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678022.1055031 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXzMM-00042Q-TG; Thu, 08 Feb 2024 08:02:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678022.1055031; Thu, 08 Feb 2024 08:02:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXzMM-00042J-PE; Thu, 08 Feb 2024 08:02:14 +0000
Received: by outflank-mailman (input) for mailman id 678022;
 Thu, 08 Feb 2024 08:02:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXzML-00042D-5U
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 08:02:13 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5c64732b-c658-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 09:02:10 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-4102eae7e6aso5238985e9.3
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 00:02:10 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m3-20020a056000024300b0033b2799815csm3055415wrz.86.2024.02.08.00.02.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 00:02:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5c64732b-c658-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707379330; x=1707984130; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sA6DTUljh8AuqClGp1xj+vIuOIKVU2MgFNi3HwqW92Y=;
        b=X3NNEyAhBllCgj4f5Vt3rz/P8SAlz+Q2zOJMKMujdhiAibLDKMDEKKhkTHUtJzP6rl
         rTy2r0X+P3A3J3K2kzmpDjKSVFkyosQZfxmtN1auGuAc8N5CSn/uV8jX1RchYYn9Kwfz
         DKamEmfQM50No05IxHwpZf5WnyKdJ1tgNhLDRLbx1IaqqDRZtVUvMTbm7KFcHAFckoOa
         PI25XRYHU0Uk4U6rytEZ3/hlfDy5VIg06HT5gn6XnYBgTVEdPmDcqMG4sK4XBlW83cHp
         +VD+KhX/1JJPmCKzYk5chfTUlRQzNu290C3J8MLJKEbdLj43m24bhIYoUKMmhx9XREEq
         +/hg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707379330; x=1707984130;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sA6DTUljh8AuqClGp1xj+vIuOIKVU2MgFNi3HwqW92Y=;
        b=ctitJKhrYv5pugcTOHjU774PRF57nA77amEMuoK/8yhuiPNTYrWcZUk2YCQhoCWMrS
         x2rzXiZJ9yGuZE7fNfYbuUil43l3fs0eFMQ7/FuCnBi1007GQsra3tT0FbQ4d+b10AwA
         kM6wtLUg0D+LnPIGw3ba1r3xaQc1QJ38LolqA/AvgabZ5u5dqfOpq3omlbrnlUAFkQQj
         deFOf1lCOQjPnzDFxjiPZVqFinfybkukERnBF52f1DDG+fnd0NU0XR03o2hP8A5O9q0w
         oHNVLI94LF8unU6jh6Imn53xBXqV+tWXguCxjCUiGM6kZlpkI9z5eLYu8eu8gr8ma+f+
         Ih/A==
X-Gm-Message-State: AOJu0Yylbun0f/gNt44Z01adIEstsfW2sRmxiPfKeIvgjnXMq51Dme7L
	IYeF3hHDnM6KuENxww95dxEQp1ypbINu7LiKttcdAQ4ZaaIsp1rJynZY7xgEVg==
X-Google-Smtp-Source: AGHT+IGxlQqzhqccR5pAw12IpRkO5HU6l2U7IJY1tN6SfFuEzk7LuNMbasBODOHwaki763Q9WfVL5A==
X-Received: by 2002:a05:600c:1d9c:b0:410:3fc8:f75c with SMTP id p28-20020a05600c1d9c00b004103fc8f75cmr433965wms.36.1707379330082;
        Thu, 08 Feb 2024 00:02:10 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUhfKUGOOm413S2cJBFpu4jL9UI8tzRGXXO/2+cg1i87DxvzXXHzPsDS66B7gFMlJ3jVyrNsMd0Jt0T6s/+SMsJzVRaZJNufB4utIG6R6Dsmgw6ln/aGqF+bNmPZGbCWRhRc38/ECQ1y+1JTeZCGz9CYPVOC3MIuKQsisMXcwugjtB0ajQ=
Message-ID: <bb2533df-dbb6-4836-bfe9-3611cd11b7f9@suse.com>
Date: Thu, 8 Feb 2024 09:02:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/2] x86/p2m: make p2m_get_page_from_gfn() handle grant
 case correctly
Content-Language: en-US
To: George Dunlap <dunlapg@umich.edu>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@cloud.com>
References: <5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com>
 <8d5df1f4-74ca-27cc-99f0-7e7a82050de1@suse.com>
 <CAFLBxZbO2-f3rdgVhu2j_7sRWSeL5oMZt6pit=6nRvhPn=pqoQ@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAFLBxZbO2-f3rdgVhu2j_7sRWSeL5oMZt6pit=6nRvhPn=pqoQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08.02.2024 07:32, George Dunlap wrote:
> On Thu, Jun 23, 2022 at 7:54 PM Jan Beulich <jbeulich@suse.com> wrote:
> 
>> Grant P2M entries, which are covered by p2m_is_any_ram(), wouldn't pass
>> the get_page() unless the grant was a local one. These need to take the
>> same path as foreign entries. Just the assertion there is not valid for
>> local grants, and hence it triggering needs to be avoided.
>>
> 
> I think I'd say:
> 
> ---
> The 'fast' path of p2m_get_page_from_gfn handles three cases: normal ram,
> foreign p2m entries, and grant map entries.  For normal ram and grant table
> entries, get_page() is called, but for foreign entries,
> page_get_owner_and_reference() is called, since the current domain is
> expected not to be the owner.
> 
> Unfortunately, grant maps are *also* generally expected to be owned by
> foreign domains; so this function will fail for any p2m entry containing a
> grant map that doesn't happen to be local.
> 
> Have grant maps take the same path as foreign entries.  Since grants may
> actually be either foreign or local, adjust the assertion to allow for this.
> ---

Sure, thanks, I can use this, but then I'd perhaps ought to add your
S-o-b instead of ...

> One more comment...
> 
> 
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> Using | instead of || helps the compiler fold the two p2m_is_*().
>> ---
>> v2: The shared case was fine; limit to grant adjustment.
>>
>> --- a/xen/arch/x86/mm/p2m.c
>> +++ b/xen/arch/x86/mm/p2m.c
>> @@ -357,11 +357,11 @@ struct page_info *p2m_get_page_from_gfn(
>>               && !((q & P2M_UNSHARE) && p2m_is_shared(*t)) )
>>          {
>>              page = mfn_to_page(mfn);
>> -            if ( unlikely(p2m_is_foreign(*t)) )
>> +            if ( unlikely(p2m_is_foreign(*t) | p2m_is_grant(*t)) )
>>
> 
> I'm not a fan of this.  If you replace it with || you can have my R-b

... R-b, requiring yet someone else's ack?

> immediately; otherwise we'll have to wait until we can discuss our general
> policy on this sort of thing at the x86 maintainer's call.

I prefer to wait. Considering that even leaving aside the use of
p2m_is_...() "if ( a || b )" is equivalent to "if ( a | b )" (with a and
b of suitable types, of course), and typically requiring less branches
(on x86 at least; architectures with predicated insns of course are
different), personally I'd see us make more use of this in general.
(Hence also the post-commit-message remark.) But yes, Misra in principle
doesn't like such (we've already deviated the underlying pattern,
though).

Of course the compiler is generally in a position to do such a
transformation itself. Just that in at least this specific case I did
observe it not to. I didn't check simpler cases any time halfway
recently.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 08:06:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 08:06:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678024.1055041 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXzQ0-0004c9-B3; Thu, 08 Feb 2024 08:06:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678024.1055041; Thu, 08 Feb 2024 08:06:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXzQ0-0004c2-85; Thu, 08 Feb 2024 08:06:00 +0000
Received: by outflank-mailman (input) for mailman id 678024;
 Thu, 08 Feb 2024 08:05:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rXzPy-0004bk-Ri
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 08:05:58 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e3de74ac-c658-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 09:05:57 +0100 (CET)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-33b436dbdcfso1133089f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 00:05:57 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e3-20020a5d5003000000b0033b524e54d1sm1881204wrt.113.2024.02.08.00.05.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 00:05:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3de74ac-c658-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707379557; x=1707984357; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SJ2b+YIn4mMFi8rKTUVQ8lugGtyEfUAbFbXwWFmHkp0=;
        b=Z57msbG74UU7ll2gnCccBLCzxwUIeTMqTc/m/tb8z6G7qTgjCTTDTrSbDoIyQeeF7O
         /GBZpkgNXXNE7dUBUWHpT1DVDyIahWpuqGexk0qujJP00B0N8z7HARbFRqZ9uMEeqwaW
         ckpQ8URpGcJTFUpZ+Ff4ddKW7XDYcOrxMcYzYFljcc4i8RaYdEIqwjUc4tZkL9k17my/
         d94bRBHxKffq95F7mLRzbrGhanz499NvQdHglSelPzmN1VSXt4IMGdAMLIbgR2gjbjzp
         WEa1ZwyPU/wnhBJZve7votmIDGZlJWB73L8FfP/Jl+STcC1lodb2Xjiutzbal1xq9vFh
         WWZA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707379557; x=1707984357;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SJ2b+YIn4mMFi8rKTUVQ8lugGtyEfUAbFbXwWFmHkp0=;
        b=XYFCQKnIe3FXJg4WmOR2S8JkLwUpVTpjNXQyW64ipyb2ok0plr0f5vk9R4lerQNhCa
         cFPYV1UJWL919jZxMi57iRBHYsXD5nHgZ+UujdpyHs9bWESb5b6oV46Q1nWpUWiVNt8z
         AbPJnM8glChiOBd/y6+zOJoIhxWwVh8Yl8nZMWVgtsSOC2vPhXmRMhPV7FuSarX0PNTl
         73KcmW0BzETjwy8dzhsoz5CNftg0QdWM3ZA9mrhVuy+OUvuY7PnVb3nUcsFm3DxPbwZn
         7Du/gMiFBiHAjFHDojiPMxGlCMSw0HjnvDb2s2Mv+WrBWgAXDRpAtaFLtMucOfNXb17e
         VtIA==
X-Gm-Message-State: AOJu0YzycV/auuCXTL7aXe3y/zFlnIwCA+n0ymwIN6sBQvVOaFvyTyVk
	bwUrMcIYsSEAO6gq+HGem5MsFNIjCZBUr+GeVqIEWh7FJJUFVXppSgBT6g8vsA==
X-Google-Smtp-Source: AGHT+IFTy/rBUUqPyky62p9LP9uKd62XnUWEUWLDj5VPqSSlu1No8GXhCYFFqMTaitsyi3cthrnKLA==
X-Received: by 2002:a5d:6651:0:b0:33b:39d5:553d with SMTP id f17-20020a5d6651000000b0033b39d5553dmr5248295wrw.60.1707379557343;
        Thu, 08 Feb 2024 00:05:57 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVF2gfnpHFwWxg6D/cbPwBpYORLt3qhOipvRvoj7S66MTa/I3yTkx24JagdtrMQDe6PwfBL4dCOlgp+NXLEq9lidAO6jCklzLiSo5k3hJFwcJ1s2zrcfvn3SfcZdseEKbA3JZeJTraRhkhwioS5J5D8z6X/h342dGdb9AtUm84/OksABfmFTwTTNgDUHB9xfNQPkfwsZ39m082+xSdSZpw6KtZXT1Bg705gtRiCCUleIxmNPgORhnfpD7XhJ2MFF55zPnUbsuA4O9AABJAU3W6SN6JErFhgYI4BfepSOOumrLHXA8AGNPZyxRBFzSZhXpDfsu1dBBDbLt+ra4Y0Xm+IL3CzOofPM5wHNnBKjZSJjnSWRLISrX4BDKeEQNMCQDD/S+rHHImPXx+mDzGJMveEdCk=
Message-ID: <9b9c128f-f143-4c2c-9ee2-67a240d216d5@suse.com>
Date: Thu, 8 Feb 2024 09:05:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] docs/misra: add x86_64 and arm64 asm-offset.c to
 exclude-list
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <b0c855581eed247a32b745906f84d352bf812091.1707324479.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b0c855581eed247a32b745906f84d352bf812091.1707324479.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.02.2024 08:55, Nicola Vetrini wrote:
> These two files contain several deliberate violations of MISRA C rules and
> they are not linked in the final Xen binary, therefore they can be exempted
> from MISRA compliance.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> ---
> Since the exclude list only contains arm64 and x86 files I reasoned that
> introducing an entry that would match all architectures would not be desirable
> (e.g., arm32). I'm happy to change that, though.

Just wanted to ask - if globs are permitted, I'd favor covering all possible
architectures. It is certainly expected that they all follow suit. Just that
in the absence of sub-architectures the path would be xen/arch/*/asm-offsets.c
(and it's quite possible that we may, over time, morph x86 to a sub-arch-less
form).

Jan

> ---
>  docs/misra/exclude-list.json | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/docs/misra/exclude-list.json b/docs/misra/exclude-list.json
> index 7971d0e70f5b..bd05478e03e9 100644
> --- a/docs/misra/exclude-list.json
> +++ b/docs/misra/exclude-list.json
> @@ -17,6 +17,10 @@
>              "rel_path": "arch/arm/arm64/lib/find_next_bit.c",
>              "comment": "Imported from Linux, ignore for now"
>          },
> +        {
> +          "rel_path": "arch/arm/arm64/asm-offsets.c",
> +          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
> +        },
>          {
>              "rel_path": "arch/x86/acpi/boot.c",
>              "comment": "Imported from Linux, ignore for now"
> @@ -97,6 +101,10 @@
>              "rel_path": "arch/x86/x86_64/mmconf-fam10h.c",
>              "comment": "Imported from Linux, ignore for now"
>          },
> +        {
> +          "rel_path": "arch/x86/x86_64/asm-offsets.c",
> +          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
> +        },
>          {
>              "rel_path": "arch/x86/efi/check.c",
>              "comment": "The resulting code is not included in the final Xen binary, ignore for now"



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 08:10:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 08:10:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678031.1055051 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXzTx-0006Fg-RL; Thu, 08 Feb 2024 08:10:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678031.1055051; Thu, 08 Feb 2024 08:10:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rXzTx-0006F2-OE; Thu, 08 Feb 2024 08:10:05 +0000
Received: by outflank-mailman (input) for mailman id 678031;
 Thu, 08 Feb 2024 08:10:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gCg7=JR=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rXzTx-0005lh-89
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 08:10:05 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 74e6fe78-c659-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 09:10:01 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-511717231bfso90488e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 00:10:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74e6fe78-c659-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707379800; x=1707984600; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=dikZXvOrvcq8ZB5trjy0EP7GCY4EkWanEfHrJyxj5gc=;
        b=bW3SDMRS8wTZHoK8ZHAFXBAForZ/FYwAHVJv3Yo78FN02ioqg2PjlJMKhf3ouYjspa
         iZnNtrKurUn6T+srKVuKEea1pnS0+sQ/2XxZpUDnilyBjiZB0r+G7HB/J0s8ABXUZES5
         bqmp76CwsgUQ0TbFyVwK18BR1VUpZBCHlGRsI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707379800; x=1707984600;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=dikZXvOrvcq8ZB5trjy0EP7GCY4EkWanEfHrJyxj5gc=;
        b=QCeZmUo2KqPnEILhrOBUF0X0VjxNl1Tb6JYMqHsuG8UkJTzjn0o+Mue0UmuBKG68Q6
         cO99TA89bFf52xv3g+xk/4fyRDQJDldU7XImFceecWfcAI118OfoV9ZkGSnh6Gls+1Gj
         6xP2eZZCQ3EzolUQe+ZwvZJxBh4qXqdA19z92/CJLRG752GoXtglLOkHP3q1KHQy5QsH
         nqPtWzRX2Iio7siGW3W1StPJu4bTKpGFNPm9ciFerIA/u9QIRG2dZDoXvW1EJyfUjodY
         2WEd841nr7vhKd5fb/R6oDf1Ipxh0+eFtnIWk8KQHkuBJAHsBTcYW2M0LJ8DZATijVat
         GKqQ==
X-Forwarded-Encrypted: i=1; AJvYcCWyqH+/v8CsIw9ImfG/eviK9XCWpFrphpK5ill2FtZ4VkwPzbCF1JsgKDr93SYV0+KvkZ7YwgwER1iKSRnQg46FmMWZumS5hsxU7Izc7Z0=
X-Gm-Message-State: AOJu0YxG3dSUmPcTZqISeldpRwQ7/t+tZMWDs4fuClq4uN0LBJmlPOB2
	MfjIgBVl8ODqWa0WdBfV2H77dW8qEB9MxjCrXSlEYvvw7T6XNdIQLp37O7mWcG7FGNbj0IHTeYv
	SHRc88ArdFvojwF7Swh2JyzkLCCTeMP8C5vZV5g==
X-Google-Smtp-Source: AGHT+IHdtj7v3m80PJGjkqfdPRAXnXTQs85KLSz21ZfIeyG0K5G3j+0dj+4ruYiBRV1Wen9Nij2Ps/d7gL/Vsu8N7i4=
X-Received: by 2002:a05:6512:b88:b0:511:54ab:9bed with SMTP id
 b8-20020a0565120b8800b0051154ab9bedmr6838357lfv.2.1707379800584; Thu, 08 Feb
 2024 00:10:00 -0800 (PST)
MIME-Version: 1.0
References: <5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com>
 <8d5df1f4-74ca-27cc-99f0-7e7a82050de1@suse.com> <CAFLBxZbO2-f3rdgVhu2j_7sRWSeL5oMZt6pit=6nRvhPn=pqoQ@mail.gmail.com>
 <bb2533df-dbb6-4836-bfe9-3611cd11b7f9@suse.com>
In-Reply-To: <bb2533df-dbb6-4836-bfe9-3611cd11b7f9@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Thu, 8 Feb 2024 08:09:49 +0000
Message-ID: <CA+zSX=ZPq+AXjUqytiOCwzfBc3xYF1GfTc4hELHz0oPagZsurw@mail.gmail.com>
Subject: Re: [PATCH v2 1/2] x86/p2m: make p2m_get_page_from_gfn() handle grant
 case correctly
To: Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <dunlapg@umich.edu>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 8, 2024 at 4:02=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 08.02.2024 07:32, George Dunlap wrote:
> > On Thu, Jun 23, 2022 at 7:54=E2=80=AFPM Jan Beulich <jbeulich@suse.com>=
 wrote:
> >
> >> Grant P2M entries, which are covered by p2m_is_any_ram(), wouldn't pas=
s
> >> the get_page() unless the grant was a local one. These need to take th=
e
> >> same path as foreign entries. Just the assertion there is not valid fo=
r
> >> local grants, and hence it triggering needs to be avoided.
> >>
> >
> > I think I'd say:
> >
> > ---
> > The 'fast' path of p2m_get_page_from_gfn handles three cases: normal ra=
m,
> > foreign p2m entries, and grant map entries.  For normal ram and grant t=
able
> > entries, get_page() is called, but for foreign entries,
> > page_get_owner_and_reference() is called, since the current domain is
> > expected not to be the owner.
> >
> > Unfortunately, grant maps are *also* generally expected to be owned by
> > foreign domains; so this function will fail for any p2m entry containin=
g a
> > grant map that doesn't happen to be local.
> >
> > Have grant maps take the same path as foreign entries.  Since grants ma=
y
> > actually be either foreign or local, adjust the assertion to allow for =
this.
> > ---
>
> Sure, thanks, I can use this, but then I'd perhaps ought to add your
> S-o-b instead of ...
<snip>
> ... R-b, requiring yet someone else's ack?

Legally I think the SoB is more for the provenance of the code than
the commit messages; so it would mainly be to credit me, which I'm not
particularly fussed by.

That said, we did just put something in MAINTAINERS about how to deal
with this situation; You sending the patch implicitly approves all the
changes I made, so then if I give an R-b, that approves all the
changes you made, satisfying the requirements.

 -George


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 08:51:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 08:51:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678036.1055061 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY07x-0003gi-Sf; Thu, 08 Feb 2024 08:51:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678036.1055061; Thu, 08 Feb 2024 08:51:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY07x-0003gb-Pr; Thu, 08 Feb 2024 08:51:25 +0000
Received: by outflank-mailman (input) for mailman id 678036;
 Thu, 08 Feb 2024 08:51:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YTSP=JR=rabbit.lu=slack@srs-se1.protection.inumbo.net>)
 id 1rY07x-0003gV-07
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 08:51:25 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3be67c38-c65f-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 09:51:22 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-40fc22f372cso13031615e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 00:51:22 -0800 (PST)
Received: from [192.168.2.1] (82-64-138-184.subs.proxad.net. [82.64.138.184])
 by smtp.googlemail.com with ESMTPSA id
 q30-20020adfb19e000000b0033b2221d938sm3135967wra.76.2024.02.08.00.51.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 00:51:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3be67c38-c65f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=rabbit-lu.20230601.gappssmtp.com; s=20230601; t=1707382282; x=1707987082; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:content-language:references
         :cc:to:subject:from:user-agent:mime-version:date:message-id:from:to
         :cc:subject:date:message-id:reply-to;
        bh=JzUyytzaAKCWO+7wPw+FR11d2mjz6ZrQxww+yYXLRMY=;
        b=sI0sjlUMoM9GwkymEJmM9skDm+6JifXi0vJHUi5CVcxQzOxPVaNOaTj9ws6HNpqOOA
         PVbn57uIhgzgojR3hWQyzpDxwWV6EEGjENUosRsOpt9sZPqssBYrUOyFzdImJfgvD+K0
         90fYB+fkKmNeV4kBqNiHT9/45+FgwQV2Dlgs1OC5wQOM45aKWutVbq6SC+JfxXSWbHY2
         kItEFGrMlx+3RxY18V075vtmJRz2FKfTwnlxF7sjiEBuJIlQo79HfwlYiSF6RIim+IPe
         cTt2krGjwymVS3ZVjDFCQPq6ZzaxEakY3CIevvqFqVEHkJomcPEw7UZgSv9g7Q8aWgRm
         Nb1w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707382282; x=1707987082;
        h=content-transfer-encoding:in-reply-to:content-language:references
         :cc:to:subject:from:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JzUyytzaAKCWO+7wPw+FR11d2mjz6ZrQxww+yYXLRMY=;
        b=pYUiHSPfliU8so6NqiTFFOLpJ4KmJUpaGvhN0CoXeCb18+b27L6BkhTP7tFgi51r+r
         AmU0KYvu7bEaynIaaYyTGHQSdx4hMwiD1BYw/38pglKjHEckDHYRc/wf6xRvZCDYED4N
         i0bKBTIH9Cn7sXWtPMf7OUG6QoQlTILfRUjbLgb+U4sh98PZCQMlFU7c2CoH6AT13uLh
         5gjKEs0JDS5TpQNLCsOdyGOXgkwnSxpoM5wbnGc58M7kikcoLVTqqk+SBv2VqmikDRdh
         ZrjjUweA9rv/yY6/d299rUkUAJB3gQPwTZHIxevsez/LIbhdokGfiO6hw2vjWBcCkGPL
         pFdg==
X-Gm-Message-State: AOJu0YxqZsbAPMquN9wWNPoDeM+r6Cq9IKRsa8s1XjIhkF6wUe2IMo+3
	GbDbJK3vQE6zAIP/wpjCiMUr0p1LIRpca+iRbA4Fv5aDsFCzc9qhuTRhj9YRjIg=
X-Google-Smtp-Source: AGHT+IHJfJxk9UExK2yDzdkgiZdwUFjwmCHGho+cWobWW4Rkq/bQsWQkFdHUlk6dOapc+IHKotbp5A==
X-Received: by 2002:a05:6000:1843:b0:33b:2f8f:da32 with SMTP id c3-20020a056000184300b0033b2f8fda32mr6420762wri.10.1707382281353;
        Thu, 08 Feb 2024 00:51:21 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXuLOqCiftNaQG+dqkyCuzK1E000kZlTRwHeHU9K3zQYcb2lExkf0SoD1iaK5HzXLJqmW+VdYIvyI5W
Message-ID: <a096dab8-88c4-4450-8c55-60d68346367f@rabbit.lu>
Date: Thu, 8 Feb 2024 09:51:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: zithro <slack@rabbit.lu>
Subject: Re: [PATCH] tools/xentop: add option to display dom0 first
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <87846acd5b31991e38561c9765eb97730c79d0f3.1706723494.git.slack@rabbit.lu>
 <a8194dec-70f0-4f90-898d-f15152a5018b@perard>
 <1f2fab6d-6eaf-4e32-a124-cf1042c31312@rabbit.lu>
 <5f6c6a52-9ef1-4db2-9e5e-2ca916bdf225@perard>
Content-Language: en-US
In-Reply-To: <5f6c6a52-9ef1-4db2-9e5e-2ca916bdf225@perard>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 07 Feb 2024 16:44, Anthony PERARD wrote:
> No worries, and your patch was reviewed so you didn't failed to the 
> strict minimum ;-).

Ahah, true, just felt a bit stupid ! Thanks, and also for the gdb pointers.
But sorry for nooby "non-rebased/non-squashed" v2, fixed now ;)

>> I have a "display dom id column" in the pipes, ready to send [...]
> That TODO file is 18 year old, and never been touch since. I don't 
> know how relevant it is. As for wanted features, I'm not aware of 
> such list.

TODO is 18 y/o, but some ideas are still nice to have ! At least
for me ;)

Like adding a "domain id" column, I ask beforehand because it will
change the default, expected display of xentop in "batch mode", and it
will annoy everyone relying on a constant output. Should I worry ?

To preserve old behaviour, I'd have to add the possibility of
displaying/hiding columns -before- adding the "dom_id" column, to then
make it hidden by default.

> I'm slightly worried about adding domain management, what if someone 
> hit the wrong key and kill a domain when they just wanted to do 
> something else, but I guess we can make domain management work more 
> or less safely. In any case, any feature is welcome.

I share your concerns re. domain management, we've all messed up at
least once with the wrong term/window, could be a real PITA !
Some have seen a ":q" followed by "oops" in XenDevel recently :)

I'd create a "*M*anage domain" bottom item, as a prompt, asking
the action and domain, like "pause domu1", "r domu2","s 42" :

     Manage domain (p/r/s/d <Domain>): _
     Manage domain (p/r/s/d <Domain>): pause domu1_
     Manage domain (p/r/s/d <Domain>): s 42_

I think it's practical -and- safe enough, even the shortest form.
Would that be a good starting point ?

((( Rest of the mail is a list of alternatives, but requiring the 
ability to select a domain/line :

1) Select the domain, press <M>, enter the action in a prompt :

     *P*ause/unpause,*R*estart,*S*hutdown,*D*estroy domain 'mydomu' ? _

2) Like (2) but add a confirmation step :

     To pause 'mydomu', type [y/yes/pause/dom_name/etc]: _

This confirm step could be user-chosen with a cmdline option, like :
     -c/-m [no-confirm/y/yes/action-name/dom-id/dom-name]

3) Arrow-keys and menu driven: select the domain <Enter> select the
action <Enter> (could also be used to display more domain info/config)

)))


-- 
++
zithro / Cyril



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 09:04:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 09:04:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678042.1055071 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY0K9-0005e0-2d; Thu, 08 Feb 2024 09:04:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678042.1055071; Thu, 08 Feb 2024 09:04:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY0K8-0005dt-VI; Thu, 08 Feb 2024 09:04:00 +0000
Received: by outflank-mailman (input) for mailman id 678042;
 Thu, 08 Feb 2024 09:03:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY0K7-0005dn-8u
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 09:03:59 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fe27667f-c660-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 10:03:58 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 695F61FCE8;
 Thu,  8 Feb 2024 09:03:57 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2F68D13984;
 Thu,  8 Feb 2024 09:03:57 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id junhCf2YxGWsDwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 09:03:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe27667f-c660-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707383037; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uKzsp9H7l0J5sye6dEzLIW91ECplPM3Er8faxSpMmio=;
	b=Sy91NG6BRYzU2FLZZKa2DQASz3BSU25BEvihpZNNjTO2nYYRRoIz+K9CZU45TYBTeuC3bk
	Dw4CTFxN7Aq/E0MPYKcs03FyZ5+VyiJUfo4CSx1w8kjna3FmIzA9b1DsURvesM0qhocsDl
	ln3xUTFzi0X+DzRZ8Qzd3HM/6c0qcqY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707383037; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uKzsp9H7l0J5sye6dEzLIW91ECplPM3Er8faxSpMmio=;
	b=Sy91NG6BRYzU2FLZZKa2DQASz3BSU25BEvihpZNNjTO2nYYRRoIz+K9CZU45TYBTeuC3bk
	Dw4CTFxN7Aq/E0MPYKcs03FyZ5+VyiJUfo4CSx1w8kjna3FmIzA9b1DsURvesM0qhocsDl
	ln3xUTFzi0X+DzRZ8Qzd3HM/6c0qcqY=
Message-ID: <135d0228-56b9-4308-bf1a-5753f7c1092a@suse.com>
Date: Thu, 8 Feb 2024 10:03:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 02/32] tools: add a new xen logging daemon
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jason Andryuk <jandryuk@gmail.com>
References: <20240205105001.24171-1-jgross@suse.com>
 <20240205105001.24171-3-jgross@suse.com>
 <d5035d8e-4183-4213-8e12-86d5cb2dbfc7@citrix.com>
Content-Language: en-US
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <d5035d8e-4183-4213-8e12-86d5cb2dbfc7@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=Sy91NG6B
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-0.95 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BAYES_HAM(-0.27)[74.13%];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 NEURAL_HAM_SHORT(-0.10)[-0.477];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.71)[subject];
	 FREEMAIL_CC(0.00)[xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Score: -0.95
X-Rspamd-Queue-Id: 695F61FCE8
X-Spam-Flag: NO

On 08.02.24 01:52, Andrew Cooper wrote:
> On 05/02/2024 10:49 am, Juergen Gross wrote:
>> Add "xen-9pfsd", a new logging daemon meant to support infrastructure
>> domains (e.g. xenstore-stubdom) to access files in dom0.
> 
> I was still expecting for "logging" to disappear from this.
> 
> In both cases it could just be deleted the sentences still work,
> although the subject ought to gain a xen-9pfsd part.
> 
> Happy to fix on commit.
> 
>>   tools/Makefile              |   1 +
>>   tools/xen-9pfsd/.gitignore  |   1 +
>>   tools/xen-9pfsd/Makefile    |  38 ++++++++++
>>   tools/xen-9pfsd/xen-9pfsd.c | 147 ++++++++++++++++++++++++++++++++++++
> 
> Asking just in case...  Do we really want the "xen-" in the directory?
> tools/9pfsd is still perfectly descriptive, and more amenable to tab
> completion given what else is in tools/

I'll change it.


Juergen



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 09:04:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 09:04:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678044.1055081 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY0Km-00065J-BG; Thu, 08 Feb 2024 09:04:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678044.1055081; Thu, 08 Feb 2024 09:04:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY0Km-00065C-7X; Thu, 08 Feb 2024 09:04:40 +0000
Received: by outflank-mailman (input) for mailman id 678044;
 Thu, 08 Feb 2024 09:04:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY0Kk-0005y8-P4
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 09:04:38 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 155d51a8-c661-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 10:04:37 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 7855C1FCA1;
 Thu,  8 Feb 2024 09:04:36 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 526ED13984;
 Thu,  8 Feb 2024 09:04:36 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id uJ49ESSZxGWsDwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 09:04:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 155d51a8-c661-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707383076; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=9eANjPaCqnr9toahltFFLE8rScNrnblAcR5iml7qQCQ=;
	b=j20+MF2+2SFSq7UYrbOFkXcU5AP6aDBR9PC7cX3DclSbRGZhqC3nRovoDRtqzzOwmlpwou
	xWASntiTxNXfDBvA1MvuTa4ANf+KpDdfrMkF0P5QJrmvkeQqPMSkBlYZroxPT7mWP7WRO9
	Sg8gsDL0KJV1EFjDURKviVCH+77RVpk=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707383076; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=9eANjPaCqnr9toahltFFLE8rScNrnblAcR5iml7qQCQ=;
	b=j20+MF2+2SFSq7UYrbOFkXcU5AP6aDBR9PC7cX3DclSbRGZhqC3nRovoDRtqzzOwmlpwou
	xWASntiTxNXfDBvA1MvuTa4ANf+KpDdfrMkF0P5QJrmvkeQqPMSkBlYZroxPT7mWP7WRO9
	Sg8gsDL0KJV1EFjDURKviVCH+77RVpk=
Message-ID: <264ea079-35d2-44b9-962c-8e8427af7bcd@suse.com>
Date: Thu, 8 Feb 2024 10:04:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 14/32] tools/xen-9pfsd: add 9pfs read request support
Content-Language: en-US
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>
References: <20240205105001.24171-1-jgross@suse.com>
 <20240205105001.24171-15-jgross@suse.com>
 <CAKf6xpvZP-kGpPMvbVm-e=f-RQ3Czb24VnmwMzTc915O9hQJcg@mail.gmail.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <CAKf6xpvZP-kGpPMvbVm-e=f-RQ3Czb24VnmwMzTc915O9hQJcg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [0.57 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_DN_SOME(0.00)[];
	 BAYES_HAM(-0.01)[47.38%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FREEMAIL_TO(0.00)[gmail.com];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.67)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: 0.57

On 08.02.24 02:28, Jason Andryuk wrote:
> On Mon, Feb 5, 2024 at 5:51 AM Juergen Gross <jgross@suse.com> wrote:
>>
>> Add the read request of the 9pfs protocol.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
>> Acked-by: Anthony PERARD <anthony.perard@citrix.com>
>> ---
>> V2:
>> - make error check more readable (Jason Andryuk)
>> V4:
>> - add directory read support
>> ---
>>   tools/xen-9pfsd/io.c | 90 ++++++++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 90 insertions(+)
>>
>> diff --git a/tools/xen-9pfsd/io.c b/tools/xen-9pfsd/io.c
>> index b763e3d8d9..cfbf3bef59 100644
>> --- a/tools/xen-9pfsd/io.c
>> +++ b/tools/xen-9pfsd/io.c
> 
>> +
>> +    len = count;
>> +    buf = ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
>> +
>> +    if ( fidp->isdir )
>> +    {
>> +        struct dirent *dirent;
>> +        struct stat st;
>> +        struct p9_stat p9s;
>> +
> 
> """
> For directories, read returns an integral number of direc- tory
> entries exactly as in stat (see stat(5)), one for each member of the
> directory. The read request message must have offset equal to zero or
> the value of offset in the previous read on the directory, plus the
> number of bytes returned in the previous read. In other words, seeking
> other than to the beginning is illegal in a directory (see seek(2)).
> """
> 
> I think you need to check `off`.  Looks like QEMU only checks for off
> == 0 and rewinds in that case.  QEMU ignores other values.

Oh, indeed. Thanks for noticing. I'll use the same approach as qemu.


Juergen


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 09:14:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 09:14:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678050.1055091 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY0Tv-00085P-59; Thu, 08 Feb 2024 09:14:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678050.1055091; Thu, 08 Feb 2024 09:14:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY0Tv-00085I-2E; Thu, 08 Feb 2024 09:14:07 +0000
Received: by outflank-mailman (input) for mailman id 678050;
 Thu, 08 Feb 2024 09:14:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=N3hj=JR=cloud.com=christian.lindig@srs-se1.protection.inumbo.net>)
 id 1rY0Tt-00085C-6M
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 09:14:05 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 67423078-c662-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 10:14:03 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a26fa294e56so206641466b.0
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 01:14:03 -0800 (PST)
Received: from smtpclient.apple ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 h1-20020a1709063c0100b00a372b38380csm1715669ejg.13.2024.02.08.01.14.02
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 08 Feb 2024 01:14:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67423078-c662-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707383643; x=1707988443; darn=lists.xenproject.org;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=hVUbvbl2/tPS9JovxVYvASUjcfKuy1BNXPlNtJB8Tc8=;
        b=ZqbrBhcLJshmdOClxTRfVxUY6on6mEMzDXzLjSoz0y7588dBFpR98EulepbkdBnI1x
         Hp+ENA4C6ZsjTMXVQ3BAs9TP/z0kitWhan17rpcow0u7HwrYPE+XEjgcJ8Y986k+HaRq
         8nzgtF+WUGLvSZRn5oYugoGv0t7/d7n5phZKo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707383643; x=1707988443;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=hVUbvbl2/tPS9JovxVYvASUjcfKuy1BNXPlNtJB8Tc8=;
        b=DZlSpFWpr/S/PnDgqaQRcJ7HU/u3VoLggUA2jv/zfLcpdI4Yjc+Q9f7ELhKO3i/CIp
         4uKJJEROZG19nE5xRDEU9ppc81S9qgh34WMEF6kYtgRguv05QqDK5qupXFWQYZ1s2Nt3
         Q0mo7M8b8DeYn0DB860wsvNIX+iIb7EAQUcIE8SOKF2rmsSYgtmLup3QapqALZV1AtrD
         yabLidOQbzUuCK+4IHrjT3g9aQtUt69JnecR4vgr9LbbVOt7s/yktuVYu99tkaUoUNfz
         kLM39EhcdgMVSPJ5z6nfFawARI6p4X733pvwxHMvDEEIqDRdtrsdix2QhSmL0NQsPsoc
         /NCQ==
X-Gm-Message-State: AOJu0YyQ2zwYW9orxhzZdFq1MomycYV7DigizIith3Hj6li97kWmA3nj
	q5Yu0+cl/yZ35ZG7JhW3IHq8R6oeYJnVkJEK9WxAJuSHDXtADMN+gVAym/0YCdA=
X-Google-Smtp-Source: AGHT+IG6GKz4l5u8ca0bu6orYcFUm4XWtTikWwfW7+VDzhRstJ5EzyJeGLpFV9yqc2y+6/QdED2eww==
X-Received: by 2002:a17:906:4:b0:a37:fb7d:6f21 with SMTP id 4-20020a170906000400b00a37fb7d6f21mr7037565eja.51.1707383643338;
        Thu, 08 Feb 2024 01:14:03 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXjnLUAUCQ6o/4FF2Ub28aLOdMFIDyw99MMcvTl/+J9jrElWth2YUEmeeqzAdwYWBzK1zlR+QefcEno41yKcAcYpeRIMZsJtSVk+32g8tq30rCQuAW+1dAn7gmFz5r+4oaSHjzDNcef+oNxePC1XbJqXtJxvxagMZn/YvNzozxlR2k=
Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\))
Subject: Re: [PATCH] tools/ocaml: Add missing vmtrace_buf_kb field
From: Christian Lindig <christian.lindig@cloud.com>
In-Reply-To: <9b9909c9e93cb540b3488c784935acc2bc9e071e.1707343396.git.w1benny@gmail.com>
Date: Thu, 8 Feb 2024 09:13:51 +0000
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 David Scott <dave@recoil.org>,
 Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <3A858D7F-C953-4EF0-8919-AE96D6105AB1@cloud.com>
References: <9b9909c9e93cb540b3488c784935acc2bc9e071e.1707343396.git.w1benny@gmail.com>
To: =?utf-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
X-Mailer: Apple Mail (2.3774.200.91.1.1)



> On 7 Feb 2024, at 22:04, Petr Bene=C5=A1 <w1benny@gmail.com> wrote:
>=20
>=20
> Add the missing `vmtrace_buf_kb` field to the OCaml bindings to match =
the
> vm.cfg configuration, correcting an oversight from its initial =
introduction.
>=20
> Signed-off-by: Petr Bene=C5=A1 <w1benny@gmail.com>

Acked-by: Christian Lindig <christian.lindig@cloud.com>

This looks correct from an OCaml perspective. Why was the new field =
added in the middle of the record type domctl_create_config and thus =
forcing changes to the index of fields coming later in the record versus =
just appending the new field to the record type?

The critical bit is using the correct type in =
"Int32_val(VAL_VMTRACE_BUF_KB)=E2=80=9D that matches the type =
"vmtrace_buf_kb: int32;=E2=80=9D - which it does.

=E2=80=94- C=


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 09:17:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 09:17:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678053.1055101 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY0XF-0000Ds-Iv; Thu, 08 Feb 2024 09:17:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678053.1055101; Thu, 08 Feb 2024 09:17:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY0XF-0000Dl-Fu; Thu, 08 Feb 2024 09:17:33 +0000
Received: by outflank-mailman (input) for mailman id 678053;
 Thu, 08 Feb 2024 09:17:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mdps=JR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rY0XE-0000Cj-NV
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 09:17:32 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e11ace83-c662-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 10:17:28 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-4103fc91755so1859975e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 01:17:28 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 u7-20020a05600c19c700b004101e93d276sm1025574wmq.14.2024.02.08.01.17.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 08 Feb 2024 01:17:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e11ace83-c662-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707383848; x=1707988648; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=wifhZVtfwjB84gHV1kh+uI3PlWaqN45zcQPThhjzTNk=;
        b=BwPSLma/RuoxmzFZ4vaVOm+CiVCiSlTTh/Mo/c2rnNn43Xdfu09wN8Ekx+YCgfVWni
         pjt/iZs3NeVQGsIDmkHidpLRh8X5GQy1A0WjRFMepTbA+3nD9dm092I2TsvUXOy8Mbmh
         XfgKCBwk6+sqkCIqoy6HuzttfWClnJzujlPIg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707383848; x=1707988648;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wifhZVtfwjB84gHV1kh+uI3PlWaqN45zcQPThhjzTNk=;
        b=aaZ+huSwgsFHGATcqI9zQZNt9PjhcsNKfCnPIIiYq4SbQt5X0Koaa0mCVEWeLX9apd
         IIXbEQmxxQJ6TauNH7r73j4YAujWzV7Xc+Ht+gpUgiMn9RnampgxcU7PLKtWLmE9EHDB
         iL7VLJxi0D98tUjwNnwU2OP169+R3fMNtminc5tm1a7gpMKetFGknN6d93yQGo3jyfKm
         4Ymvmx9MgA/p7SMfnVXCdtim1fMyQPt3yapsK8BkMRJXXUCYTbfuSV6NB3E5T6SfVfSe
         WtcUqS5MXHhwQ7vmCuS5cCSGdqxY2Vbxjlo1BmJO198HMCh6PTv6XjRddml8AFYnUYBI
         clZw==
X-Gm-Message-State: AOJu0YzM9w4Db2vwu3O4anR+2Vt0os2H9sio3w5ou09JqijMRIQh4Mgm
	udyOxp/sMBKESKfzVbmnmuVWFeIkrgLSwEahpDg2qm0v2PSY/77C85m8uaSIDv4=
X-Google-Smtp-Source: AGHT+IF1DIAJ5Gd8CwZHSU73tvvB0duAIpkpweIixYv6rPbFRESMnotdJF0K3KyQQfVWHPGjggxEXQ==
X-Received: by 2002:a05:600c:35c7:b0:40f:e1e8:38f1 with SMTP id r7-20020a05600c35c700b0040fe1e838f1mr5690607wmq.1.1707383847572;
        Thu, 08 Feb 2024 01:17:27 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXcPGstiI5nHahmBpIIs2mKAmtNL/kbxUY/F0WNeXcI0GvAs7A0V7Q4pKnJ4DiYvwX+1Lw27v92Hs+tzzaW6VYrypVobY/a/N0xnJtnUnKw3iil
Date: Thu, 8 Feb 2024 10:17:26 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 1/7] VT-d: parse ACPI "SoC Integrated Address Translation
 Cache Reporting Structure"s
Message-ID: <ZcScJvAhI7CRJhAZ@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <b5a58dee-9a4c-4833-be59-b52c62f7137d@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <b5a58dee-9a4c-4833-be59-b52c62f7137d@suse.com>

On Mon, Feb 05, 2024 at 02:55:17PM +0100, Jan Beulich wrote:
> This is a prereq to us, in particular, respecting the "ATC required"
> flag.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> Should we check scope entries for appropriate types? (If so, then also
> for e.g. ATSR.)

Hm, I guess we could do so in acpi_parse_dev_scope() since that
function already gets passed a 'type' argument.

> 
> --- a/xen/drivers/passthrough/vtd/dmar.c
> +++ b/xen/drivers/passthrough/vtd/dmar.c
> @@ -47,6 +47,7 @@ LIST_HEAD_READ_MOSTLY(acpi_drhd_units);
>  LIST_HEAD_READ_MOSTLY(acpi_rmrr_units);
>  static LIST_HEAD_READ_MOSTLY(acpi_atsr_units);
>  static LIST_HEAD_READ_MOSTLY(acpi_rhsa_units);
> +static LIST_HEAD_READ_MOSTLY(acpi_satc_units);
>  
>  static struct acpi_table_header *__read_mostly dmar_table;
>  static int __read_mostly dmar_flags;
> @@ -764,6 +765,95 @@ acpi_parse_one_rhsa(struct acpi_dmar_hea
>      return ret;
>  }
>  
> +static int __init register_one_satc(struct acpi_satc_unit *satcu)
> +{
> +    bool ignore = false;
> +    unsigned int i = 0;
> +    int ret = 0;
> +
> +    /* Skip checking if segment is not accessible yet. */
> +    if ( !pci_known_segment(satcu->segment) )
> +        i = UINT_MAX;
> +
> +    for ( ; i < satcu->scope.devices_cnt; i++ )
> +    {
> +        uint8_t b = PCI_BUS(satcu->scope.devices[i]);
> +        uint8_t d = PCI_SLOT(satcu->scope.devices[i]);
> +        uint8_t f = PCI_FUNC(satcu->scope.devices[i]);
> +
> +        if ( pci_device_detect(satcu->segment, b, d, f) == 0 )

Any reason to explicitly compare against 0?

if ( !pci_device_detect(satcu->segment, b, d, f) )
...

The function returns a boolean.

> +        {
> +            dprintk(XENLOG_WARNING VTDPREFIX,
> +                    " Non-existent device (%pp) is reported in SATC scope!\n",
> +                    &PCI_SBDF(satcu->segment, b, d, f));
> +            ignore = true;

This is kind of reporting is incomplete: as soon as one device is
found the loop is exited and no further detection happens.  If we want
to print such information, we should do the full scan and avoid
exiting early when a populated device is detected.

> +        }
> +        else
> +        {
> +            ignore = false;
> +            break;
> +        }
> +    }
> +
> +    if ( ignore )
> +    {
> +        dprintk(XENLOG_WARNING VTDPREFIX,
> +                " Ignore SATC for seg %04x as no device under its scope is PCI discoverable!\n",

(I would drop the '!' at the end, here and above, I don't think they
add much to the error message)  I would also use the '#' flag to avoid
confusion, as in the weird case we have a segment '1234' then without
the zero padding I wouldn't really know whether it's decimal or hex.

> +                satcu->segment);
> +        scope_devices_free(&satcu->scope);
> +        xfree(satcu);
> +        return 1;
> +    }
> +
> +    if ( iommu_verbose )
> +        printk(VTDPREFIX " ATC required: %d\n", satcu->atc_required);
> +
> +    list_add(&satcu->list, &acpi_satc_units);
> +
> +    return ret;
> +}
> +
> +static int __init
> +acpi_parse_one_satc(const struct acpi_dmar_header *header)
> +{
> +    const struct acpi_dmar_satc *satc =
> +        container_of(header, const struct acpi_dmar_satc, header);
> +    struct acpi_satc_unit *satcu;
> +    const void *dev_scope_start, *dev_scope_end;
> +    int ret;
> +
> +    if ( (ret = acpi_dmar_check_length(header, sizeof(*satc))) != 0 )
> +        return ret;

A very stupid nit, but I would rather prefer:

int ret = acpi_dmar_check_length(header, sizeof(*satc));

if ( ret )
    return ret;

Which has the same number of lines and is IMO easier to read.

> +
> +    satcu = xzalloc(struct acpi_satc_unit);
> +    if ( !satcu )
> +        return -ENOMEM;
> +
> +    satcu->segment = satc->segment;
> +    satcu->atc_required = satc->flags & 1;

I would add this as a define in actbl2.h:

#define ACPI_DMAR_ATC_REQUIRED (1U << 0)

Or some such (maybe just using plain 1 is also fine).

> +
> +    dev_scope_start = (const void *)(satc + 1);
> +    dev_scope_end   = (const void *)satc + header->length;
> +    ret = acpi_parse_dev_scope(dev_scope_start, dev_scope_end,
> +                               &satcu->scope, SATC_TYPE, satc->segment);
> +
> +    if ( !ret && satcu->scope.devices_cnt )
> +    {
> +        ret = register_one_satc(satcu);
> +        /*
> +         * register_one_satc() returns greater than 0 when a specified
> +         * PCIe device cannot be detected. To prevent VT-d from being
> +         * disabled in such cases, reset the return value to 0 here.
> +         */
> +        if ( ret > 0 )
> +            ret = 0;
> +    }
> +    else
> +        xfree(satcu);

You can safely use scope_devices_free() even if acpi_parse_dev_scope()
failed, so you could unify the freeing here, instead of doing it in
register_one_satc() also.

Also why not make register_one_satc() return a boolean instead of 0/1?

if ( ret || !satcu->scope.devices_cnt || register_one_satc(satcu) )
{
    scope_devices_free(&satcu->scope);
    xfree(satcu);
}

> +
> +    return ret;
> +}
> +
>  static int __init cf_check acpi_parse_dmar(struct acpi_table_header *table)
>  {
>      struct acpi_table_dmar *dmar;
> @@ -817,6 +907,11 @@ static int __init cf_check acpi_parse_dm
>                  printk(VTDPREFIX "found ACPI_DMAR_RHSA:\n");
>              ret = acpi_parse_one_rhsa(entry_header);
>              break;
> +        case ACPI_DMAR_TYPE_SATC:
> +            if ( iommu_verbose )
> +                printk(VTDPREFIX "found ACPI_DMAR_SATC:\n");
> +            ret = acpi_parse_one_satc(entry_header);
> +            break;

I know the surrounding code doesn't use it, but readability would
benefit from adding a blank line after the break statement.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 09:57:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 09:57:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678061.1055111 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY19M-0006H8-Dq; Thu, 08 Feb 2024 09:56:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678061.1055111; Thu, 08 Feb 2024 09:56:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY19M-0006H1-AG; Thu, 08 Feb 2024 09:56:56 +0000
Received: by outflank-mailman (input) for mailman id 678061;
 Thu, 08 Feb 2024 09:56:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=z5WF=JR=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rY19K-0006Gr-V8
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 09:56:54 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6287b7d9-c668-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 10:56:53 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 3939F4EE0739;
 Thu,  8 Feb 2024 10:56:52 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6287b7d9-c668-11ee-8a4a-1f161083a0e0
MIME-Version: 1.0
Date: Thu, 08 Feb 2024 10:56:52 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH] docs/misra: add x86_64 and arm64 asm-offset.c to
 exclude-list
In-Reply-To: <9b9c128f-f143-4c2c-9ee2-67a240d216d5@suse.com>
References: <b0c855581eed247a32b745906f84d352bf812091.1707324479.git.nicola.vetrini@bugseng.com>
 <9b9c128f-f143-4c2c-9ee2-67a240d216d5@suse.com>
Message-ID: <2b30967d547330ed5d1b521733d19d82@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 2024-02-08 09:05, Jan Beulich wrote:
> On 08.02.2024 08:55, Nicola Vetrini wrote:
>> These two files contain several deliberate violations of MISRA C rules 
>> and
>> they are not linked in the final Xen binary, therefore they can be 
>> exempted
>> from MISRA compliance.
>> 
>> No functional change.
>> 
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>> ---
>> Since the exclude list only contains arm64 and x86 files I reasoned 
>> that
>> introducing an entry that would match all architectures would not be 
>> desirable
>> (e.g., arm32). I'm happy to change that, though.
> 
> Just wanted to ask - if globs are permitted, I'd favor covering all 
> possible
> architectures. It is certainly expected that they all follow suit. Just 
> that
> in the absence of sub-architectures the path would be 
> xen/arch/*/asm-offsets.c
> (and it's quite possible that we may, over time, morph x86 to a 
> sub-arch-less
> form).
> 
> Jan
> 

Ok, I'll modify it. Thanks,

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 10:17:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 10:17:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678067.1055120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY1Su-0001FC-1F; Thu, 08 Feb 2024 10:17:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678067.1055120; Thu, 08 Feb 2024 10:17:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY1St-0001F5-Uy; Thu, 08 Feb 2024 10:17:07 +0000
Received: by outflank-mailman (input) for mailman id 678067;
 Thu, 08 Feb 2024 10:17:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pa6k=JR=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1rY1Ss-0001Eo-GR
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 10:17:06 +0000
Received: from mail-ed1-x541.google.com (mail-ed1-x541.google.com
 [2a00:1450:4864:20::541])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 34c2c2fb-c66b-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 11:17:04 +0100 (CET)
Received: by mail-ed1-x541.google.com with SMTP id
 4fb4d7f45d1cf-5602500d1a6so1931436a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 02:17:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 34c2c2fb-c66b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707387424; x=1707992224; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=AbTGySjrskR5XP4hQZjlCOJWstAuPb2HAfUelcGxGZk=;
        b=IP8btcyJz00KIOzvLYKHSFGHr+V4/KwXdq425x6qIYTIlBfRN367/nzvtX02iTSrFT
         cPomNj1G8k8F0g6kpr2msmtgicatOwkVpWA9bVo/kKpfyUU53HXs/ienwhUcur8mcobB
         TLFh3wLkDAHWIWhJ6SOAT+YSoNWXfZj7y2/Mg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707387424; x=1707992224;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=AbTGySjrskR5XP4hQZjlCOJWstAuPb2HAfUelcGxGZk=;
        b=MWawtw5bE+fXne03kLh3S9CRM9iNq9q+OaJrVGkrJeyHx4le4B0Urhr0z5KTp8UR7+
         rGoO+XVXeUyZmTXY6GSg0IVCJ1r1e9F3ppt+ARhKAiXtEaAWK8Jwz399mWRsehH0W5oA
         HefMAj9gs3nx14JqbZP0f1Rb4qrGn7ycFL2D4xLxuufOSCFArEpTlAMbZNRroujE3ZE4
         LZShuYQBsNpYm+eX6c2LdqiRHTH1kuh3OGl+4c06cLUjIqnOVr58dnaYPBFp4v5m3L8K
         m81ip+4b9p/thc1Y2WEcPVVYVL7BE8NBUkNIA012dYerWDCU9IL1pxXI6stKQMghCfVJ
         WUNA==
X-Gm-Message-State: AOJu0YydKaEWrQrNuzBt85aW+4PTLa4BiVzN8gGlXkJXaZ6Xu2VjoMAP
	6MSEgQVK5xtF5W2aBh0NOXXMaMCFFmkQ7t2wI/6g6H5indt4aictXjyV5D/ZguOWg5Y+0lVmgMV
	cYD/mX9EfB1+S13+oE8zqTtuftjDqM0FFQMenT+hUEW30Kk1uJCgg4CrU
X-Google-Smtp-Source: AGHT+IHe6AGaed2PJn4jN/U3ncOM07HkK6LqNUDGKQihPy8aUtplZ92K1Nq9g7koR31K/hwd9Zj6lkuqHjm350FZr/A=
X-Received: by 2002:a17:906:319a:b0:a38:aa7f:bdd2 with SMTP id
 26-20020a170906319a00b00a38aa7fbdd2mr1539922ejy.59.1707387423669; Thu, 08 Feb
 2024 02:17:03 -0800 (PST)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Thu, 8 Feb 2024 11:16:28 +0100
Message-ID: <CAO-mL=wTYM8-=gUCJxew8gM+M6WvkVszroqVW8kJEiNLDZ-PHQ@mail.gmail.com>
Subject: CFP for Xen Summit 2024!
To: xen-devel@lists.xenproject.org, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000e1d6ca0610dc1cca"

--000000000000e1d6ca0610dc1cca
Content-Type: text/plain; charset="UTF-8"

Hi all,

*A reminder that our CFP for Xen Summit 2024 closes in less than a month! *

We'd love to hear from you.

Submit your talk here:
https://events.linuxfoundation.org/xen-project-summit/

Many thanks,
Kelly Choi

Community Manager
Xen Project

--000000000000e1d6ca0610dc1cca
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi all,=C2=A0<div><br></div><div><b>A reminder that our CF=
P for Xen Summit 2024 closes in less than a month!=C2=A0</b></div><div><br>=
</div><div>We&#39;d love to hear from you.</div><div><br></div><div>Submit =
your talk here:=C2=A0<a href=3D"https://events.linuxfoundation.org/xen-proj=
ect-summit/">https://events.linuxfoundation.org/xen-project-summit/</a></di=
v><div><br clear=3D"all"><div><div dir=3D"ltr" class=3D"gmail_signature" da=
ta-smartmail=3D"gmail_signature"><div dir=3D"ltr"><div>Many thanks,</div><d=
iv>Kelly Choi</div><div><br></div><div><div style=3D"color:rgb(136,136,136)=
">Community Manager</div><div style=3D"color:rgb(136,136,136)">Xen Project=
=C2=A0<br></div></div></div></div></div></div></div>

--000000000000e1d6ca0610dc1cca--


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 10:44:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 10:44:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678105.1055147 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY1ti-0006gK-Kk; Thu, 08 Feb 2024 10:44:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678105.1055147; Thu, 08 Feb 2024 10:44:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY1ti-0006gD-Gn; Thu, 08 Feb 2024 10:44:50 +0000
Received: by outflank-mailman (input) for mailman id 678105;
 Thu, 08 Feb 2024 10:44:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Up58=JR=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rY1th-0006g7-HV
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 10:44:49 +0000
Received: from NAM02-BN1-obe.outbound.protection.outlook.com
 (mail-bn1nam02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2407::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 13842ea3-c66f-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 11:44:47 +0100 (CET)
Received: from DM6PR03CA0067.namprd03.prod.outlook.com (2603:10b6:5:100::44)
 by PH7PR12MB6538.namprd12.prod.outlook.com (2603:10b6:510:1f1::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.10; Thu, 8 Feb
 2024 10:44:42 +0000
Received: from CY4PEPF0000FCBE.namprd03.prod.outlook.com
 (2603:10b6:5:100:cafe::f5) by DM6PR03CA0067.outlook.office365.com
 (2603:10b6:5:100::44) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.37 via Frontend
 Transport; Thu, 8 Feb 2024 10:44:42 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCBE.mail.protection.outlook.com (10.167.242.100) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Thu, 8 Feb 2024 10:44:41 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Thu, 8 Feb
 2024 04:44:40 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Thu, 8 Feb
 2024 02:44:40 -0800
Received: from XIR-MICHALO-L1.xilinx.com (10.180.168.240) by
 SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34
 via Frontend Transport; Thu, 8 Feb 2024 04:44:39 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13842ea3-c66f-11ee-8a4a-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=E2bcZFAkakoZypWyc7LQEQYZwWGo0Jb6fu//3u/ZZabzF/eboREq16Je2Lg+JyOmsUGptmytC2Culdtsa17p+985WdNF4sEZnLXMdNJDk/hWPAZncvhLOHYqnRXOqq57/Pji9z4n5awAXz74+16F/D9Qo8Wi1Pa/Xr5OiEQPH3KH/nnMtXK9M+kRlK0RWIKfd+YvCo+e8NKK4RQRDVncjheitfQ5TZyrJXvDy8LMCFfLBmvn4LzeMKUVOI1eeGFmUtZ9eJgnd4SeMKesKwJzaSgm8sbdHTR5ATVQUnf7jnsrUzEVPiRaAYeiuWBxghQlwNgwZ/NnAVegdmWX/lYqzg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Qe6SRiNUmo8alfoEpeT1FGLlqpS/CEq6DFcwSK5kqkw=;
 b=TWtBHGWo1nB0eVHAjLHCQjTMafIXRmFPqpUTcvurws7IDciFgk31b7hvYoxfm0HFIV08nrLwLExnLtGO8Nquo7qicuhqAnQowGvowMtnYjllOGGjdo3O4y1ixc/ZF5tfn9ciNJJwglpL6SQV+NJ9Xr80NVXV95oZlum34/AhChAY/sW7AWsMtstJtEf6gNYfG3lrm5LccE4ga2z4Lj4z+x4G22n2ia0C74/MRtOuYxjT121OHcEkH3vr5G3gyhpAFkXSM/JPQwj8ovNm6GrhL/fjxiBAyPnAAGh0Jj1gmiLfyQroXI+OMxbVIXMa5+DGoDmX9IPrFRSYVY6bsSjRwQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Qe6SRiNUmo8alfoEpeT1FGLlqpS/CEq6DFcwSK5kqkw=;
 b=kJMezAY4MCN0m0O3VdW2RYEVk56PaAB61QNmwSxYWnmMiWUGGK6lRqbHFoRwvP5j0eAgAzhKLhhHRPXMUlTsw3ooSr8sFIHLiWYJtVUwXHTHyj/B+bH3sblxzbO4rpiR9uVg+BTe8EeDHjerY1eBAD/TluUW+rlPnUz+4E0SK6M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Michal Orzel <michal.orzel@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Michal Orzel <michal.orzel@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH] xen/arm: Fix UBSAN failure in start_xen()
Date: Thu, 8 Feb 2024 11:43:39 +0100
Message-ID: <20240208104339.37826-1-michal.orzel@amd.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCBE:EE_|PH7PR12MB6538:EE_
X-MS-Office365-Filtering-Correlation-Id: 7b20c5b1-5205-4f79-1f7c-08dc2892f4c8
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dRthpoS3OagOnjDMjqB+nProMbElFIIdTlvrIY8CUsErkGRkSIHEWCDE2WnL2xMXw+GxkQkX/qUIrkB2tUPALLYJpTmzZAldFvnYXQUWIULNGrejcXv+GlyHtRUQD+VdHtibWFqtyddrAcJkw2zxCCsxEo3Z2NSGOoRmkEXISAafgoJaCkoXDlgzN9iCD/qttREKyRazJxaUWPLExOwZmy11fh0Lyp5EXS0g9rIUpful/15Nj/kddz9jRnr7+8U2uZ1WQlG+DkB2CwnbpPhTousAp+t1Sei2QGV6FLWsLtQl9DQ0aW1kariHSQBS2giKXPYKzJm83YJIOzC3smK7gLK8KvLhyx07o2YmVqPlei1x/ng+ps5KLL5OEKhWPzTCCmNo2MO7aVKU0J6Ot8o/fRu3Orrg0zw02JuXqsvIX5K7DzcqZ0Kuc5I3qwx1iADfqgUwE+trZRmmjtANGUXwy5b6J2nnSosQBJngfgoOobiP1npohNxnCGO1GLcntYDEgkIfpK7uXuJ/N9gh8zMa7nqomzxdCxxxm90zsDSgcB+SeBDTjdBLWT/GzWDPaS3kwSoK69JNrv+u5udj8i5nIRKd3iU0uoSwuDlaCmBwbwQ=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(346002)(396003)(136003)(376002)(230922051799003)(82310400011)(64100799003)(451199024)(1800799012)(186009)(40470700004)(36840700001)(46966006)(41300700001)(2906002)(8936002)(44832011)(5660300002)(8676002)(4326008)(83380400001)(1076003)(426003)(336012)(478600001)(26005)(2616005)(36756003)(86362001)(6916009)(316002)(54906003)(70586007)(70206006)(82740400003)(356005)(81166007);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2024 10:44:41.5297
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7b20c5b1-5205-4f79-1f7c-08dc2892f4c8
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCBE.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6538

When running Xen on arm32, in scenario where Xen is loaded at an address
such as boot_phys_offset >= 2GB, UBSAN reports the following:

(XEN) UBSAN: Undefined behaviour in arch/arm/setup.c:739:58
(XEN) pointer operation underflowed 00200000 to 86800000
(XEN) Xen WARN at common/ubsan/ubsan.c:172
(XEN) ----[ Xen-4.19-unstable  arm32  debug=y ubsan=y  Not tainted ]----
...
(XEN) Xen call trace:
(XEN)    [<0031b4c0>] ubsan.c#ubsan_epilogue+0x18/0xf0 (PC)
(XEN)    [<0031d134>] __ubsan_handle_pointer_overflow+0xb8/0xd4 (LR)
(XEN)    [<0031d134>] __ubsan_handle_pointer_overflow+0xb8/0xd4
(XEN)    [<004d15a8>] start_xen+0xe0/0xbe0
(XEN)    [<0020007c>] head.o#primary_switched+0x4/0x30

The failure is reported for the following line:
(paddr_t)(uintptr_t)(_start + boot_phys_offset)

This occurs because the compiler treats (ptr + size) with size bigger than
PTRDIFF_MAX as undefined behavior. To address this, switch to macro
virt_to_maddr(), given the future plans to eliminate boot_phys_offset.

Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
 xen/arch/arm/setup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 7e28f62d09f1..424744ad5e1a 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -736,7 +736,7 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset,
 
     /* Register Xen's load address as a boot module. */
     xen_bootmodule = add_boot_module(BOOTMOD_XEN,
-                             (paddr_t)(uintptr_t)(_start + boot_phys_offset),
+                             virt_to_maddr(_start),
                              (paddr_t)(uintptr_t)(_end - _start), false);
     BUG_ON(!xen_bootmodule);
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 10:45:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 10:45:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678107.1055157 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY1uN-00077R-Sl; Thu, 08 Feb 2024 10:45:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678107.1055157; Thu, 08 Feb 2024 10:45:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY1uN-00077K-Pe; Thu, 08 Feb 2024 10:45:31 +0000
Received: by outflank-mailman (input) for mailman id 678107;
 Thu, 08 Feb 2024 10:45:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w87w=JR=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rY1uM-00075l-Ll
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 10:45:30 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2c7b0b4e-c66f-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 11:45:28 +0100 (CET)
Received: from [192.168.1.15] (host-87-16-84-78.retail.telecomitalia.it
 [87.16.84.78])
 by support.bugseng.com (Postfix) with ESMTPSA id EE9674EE0739;
 Thu,  8 Feb 2024 11:45:27 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c7b0b4e-c66f-11ee-98f5-efadbce2ee36
Message-ID: <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
Date: Thu, 8 Feb 2024 11:45:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
Content-Language: en-US, it
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 07/02/24 17:19, Jan Beulich wrote:
> On 07.02.2024 16:58, Federico Serafini wrote:
>> On 07/02/24 16:24, Jan Beulich wrote:
>>> On 07.02.2024 16:08, Federico Serafini wrote:
>>>> On 07/02/24 15:16, Jan Beulich wrote:
>>>>> On 07.02.2024 14:51, Federico Serafini wrote:
>>>>>> On 07/02/24 08:38, Jan Beulich wrote:
>>>>>>> On 07.02.2024 02:08, Stefano Stabellini wrote:
>>>>>>>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>>>>>>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>>>>>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>>>>>>>          case 8:                                                                \
>>>>>>>>>>              put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>>>>>>>              break;                                                             \
>>>>>>>>>> -    default: __put_user_bad();                                             \
>>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>>          }                                                                      \
>>>>>>>>>>          clac();                                                                \
>>>>>>>>>>      } while ( false )
>>>>>>>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>>>>>>>          case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>>>>>>>          case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>>>>>>>          case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>>>>>>>> -    default: __get_user_bad();                                             \
>>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>>          }                                                                      \
>>>>>>>>>>          clac();                                                                \
>>>>>>>>>>      } while ( false )
>>>>>>>>>
>>>>>>>>> Related to my remark on patch 1 - how is one to know the macro this was
>>>>>>>>> invoked from, when seeing the resulting diagnostic?
>>>>>>>>
>>>>>>>> I am not sure what do you mean here... we do get an error like the
>>>>>>>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>>>>>>>
>>>>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>>>>
>>>>>>> Right - and how do I know what _user_ of the macro actually triggered
>>>>>>> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
>>>>>>> __FUNCTION__ here, for that specific purpose ...
>>>>>>
>>>>>> To test the macro and its diagnostics,
>>>>>> I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
>>>>>> on the x86 code with STATIC_ASSERT_UNREACHABLE(),
>>>>>> that is in file arch/x86/alternative.c, line 312,
>>>>>> function _apply_alternatives().
>>>>>>
>>>>>> What I got is the following build error:
>>>>>>
>>>>>> ...
>>>>>> arch/x86/alternative.c: Assembler messages:
>>>>>> arch/x86/alternative.c:312: Error: static assertion failed: unreachable
>>>>>>       CC      arch/x86/copy_page.o
>>>>>> make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1
>>>>>
>>>>> But that's not what my request was about. Here sufficient context is
>>>>> given, even if it would be nice if the function was also visible right
>>>>> away. But that's not the same as the case above, where the new macro
>>>>> is used inside another macro.
>>>>
>>>> An example of that is the get_unsafe_size() macro,
>>>> whose body uses STATIC_ASSERT_UNREACHABLE().
>>>> A wrong use of get_unsafe_size() at line n
>>>> leads to a build error pointing to the line n,
>>>> isn't this the desired behavior?
>>>
>>> Aiui this would point to the line in the header file, when what you need
>>> to spot the bad use of the macro is the line in the source file actually
>>> using the macro. Quoting from an earlier mail of yours:
>>>
>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>
>> It points to the header file uaccess.h because at line 262 there is
>> an intentional wrong use of put_guest_size(), within the body of
>> __copy_to_guest_pv() function.
> 
> Yet that's again only a helper function being inlined into the ultimate
> caller. That ultimate caller is what wants identifying in the diag. Not
> the least because of ...
> 
>> This example can be misleading because {get,put}_unsafe_size() are
>> defined in the same file but the diagnostics is doing the
>> right thing.
> 
> ... this. And really __copy_to_guest_pv() is the wrong place to put a
> wrong put_guest_size() in, to try out how diagnostics would look like
> in reality: That function falls back to copy_to_guest_ll() for all
> cases it can't handle directly. You want to place a bogus put_guest()
> somewhere in a .c file to see what results.

I added a bogus call to put_guest() at line 387 of
file xen/arch/x86/mm.c, inside function page_is_ram_type().
Assuming I did not choose another wrong place,
the diagnostic seems appropriate:

arch/x86/mm.c: Assembler messages:
arch/x86/mm.c:387: Error: static assertion failed: unreachable

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 11:14:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 11:14:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678125.1055167 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY2Mf-0003Ts-3o; Thu, 08 Feb 2024 11:14:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678125.1055167; Thu, 08 Feb 2024 11:14:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY2Me-0003Tl-W3; Thu, 08 Feb 2024 11:14:44 +0000
Received: by outflank-mailman (input) for mailman id 678125;
 Thu, 08 Feb 2024 11:14:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY2Md-0003Tf-DU
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 11:14:43 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 418320d3-c673-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 12:14:42 +0100 (CET)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-3394ca0c874so1272343f8f.2
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 03:14:41 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 y16-20020a5d6150000000b0033b443a7aa7sm3342432wrt.97.2024.02.08.03.14.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 03:14:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 418320d3-c673-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707390881; x=1707995681; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=WJhKsc/PhbHhvUZpY7uO3Xn7XeP9pnyGn84MJLpOOFM=;
        b=RFiTx7KOOmYCKykPdJ/nnXtnQU0xjm8QsKUUwAxWABDSGFIusZq7p3TsFHypqZ1tPv
         jDuLXhPIhDCU9bRXKaveitX1AaBOv8XfbdvC4YrtNXnl5ahhsCcPRfOWa6geU14sPp4f
         l+3/in+5LRX0CCh7xRZVN+VVW7Oet1W0OBAnxJdGjbDorIfP2mBIUuFEyCyt6e5HqYDf
         a2egjU3P9utIJApm2+zfmPfmeC23uL8EpfLdnxeu2I8JtetBP9xlXH5XAEqsY1aom7Y/
         UzGOKxF6LA2Yaw975GOpKhEyL0ZDZpS+AOn9CckdTYlOyMy9zD6MAJcl+j4MnwYY8isk
         bcNw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707390881; x=1707995681;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=WJhKsc/PhbHhvUZpY7uO3Xn7XeP9pnyGn84MJLpOOFM=;
        b=cjX5pIrzQh0VYEotzDKnnsMvKcjz7cORPQ6+FuK3dtAnF94A6Kqnfu383c8ndNCjQq
         ljjI2KVLH6+3GCnRlVeCiPVCWZZp6P1U2nDASbS3IjDYcQReS5reWET88MzUDpeWYy6X
         GuZtzRI6L4ajzs1Jtfq3giBgMj99PXxYTTpQu4r74wkLd4v+9yVdRyJz1iBG+juy6NbX
         u6Kmpp4aoXLSUGz1l9JB2V5wCSRn/OVMGm820WlmpXjuNI8GkJrHBTOHH29f+VtuEkCj
         ofQiehd+o2HfEwpVgT8xJDeRMAKxHLmDpfcFTMHCn8Sis2lWbSCzJNcoO8q6Adbm0QWp
         3xsA==
X-Forwarded-Encrypted: i=1; AJvYcCVWbVqJ8rTMx0ShCVry4r6IE0bKiW21wuTQZXDQd6JhcO4wHZWXldAH/ZB4PF9Y9xqOglfnD08GxAW6euqvXn5iUYR2QDp5iEXGuZhneFk=
X-Gm-Message-State: AOJu0YxG+mzLTEmwE1mHPkHioPbJ6hEHfgN8tRapfUh5TNTt/mM7+O1K
	IISoZ+RQOncsJO2uXSc6tMn/2AkJSHULqPqslwM+3vyqR+RLI0zQhlWrnRk9FQ==
X-Google-Smtp-Source: AGHT+IFylrrAl6aJ2C/Hb3phl/+IWyRvdAZv/JyGmvhsWE/nvhJLz1SUpOn/GZCiOcXZi4GiBY4sgg==
X-Received: by 2002:adf:fc88:0:b0:33b:304d:36b7 with SMTP id g8-20020adffc88000000b0033b304d36b7mr5248456wrr.56.1707390881320;
        Thu, 08 Feb 2024 03:14:41 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWtjuXxem4auupau0lPwAwAX5g03VI/R//n7SzPHCxoVwXoHkqFe869bWRvVay3IF07mPiPab8Onujk1gBNzFCYquihLg4emImWKenWyCIiynS7UIYryZPzNcAqcshgS7fZZwTAk8UNUZ+ao4PsxwbDu40DqycfjKiJT4wKLttkqtmeH9hHpv+Cy7uvba1pbqYtHzjjUrXjTRZBL1i19V7f5Krw
Message-ID: <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
Date: Thu, 8 Feb 2024 12:14:40 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.02.2024 11:45, Federico Serafini wrote:
> On 07/02/24 17:19, Jan Beulich wrote:
>> On 07.02.2024 16:58, Federico Serafini wrote:
>>> On 07/02/24 16:24, Jan Beulich wrote:
>>>> On 07.02.2024 16:08, Federico Serafini wrote:
>>>>> On 07/02/24 15:16, Jan Beulich wrote:
>>>>>> On 07.02.2024 14:51, Federico Serafini wrote:
>>>>>>> On 07/02/24 08:38, Jan Beulich wrote:
>>>>>>>> On 07.02.2024 02:08, Stefano Stabellini wrote:
>>>>>>>>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>>>>>>>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>>>>>>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>>>>>>>>          case 8:                                                                \
>>>>>>>>>>>              put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>>>>>>>>              break;                                                             \
>>>>>>>>>>> -    default: __put_user_bad();                                             \
>>>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>>>          }                                                                      \
>>>>>>>>>>>          clac();                                                                \
>>>>>>>>>>>      } while ( false )
>>>>>>>>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>>>>>>>>          case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>>>>>>>>          case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>>>>>>>>          case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>>>>>>>>> -    default: __get_user_bad();                                             \
>>>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>>>          }                                                                      \
>>>>>>>>>>>          clac();                                                                \
>>>>>>>>>>>      } while ( false )
>>>>>>>>>>
>>>>>>>>>> Related to my remark on patch 1 - how is one to know the macro this was
>>>>>>>>>> invoked from, when seeing the resulting diagnostic?
>>>>>>>>>
>>>>>>>>> I am not sure what do you mean here... we do get an error like the
>>>>>>>>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>>>>>>>>
>>>>>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>>>>>
>>>>>>>> Right - and how do I know what _user_ of the macro actually triggered
>>>>>>>> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
>>>>>>>> __FUNCTION__ here, for that specific purpose ...
>>>>>>>
>>>>>>> To test the macro and its diagnostics,
>>>>>>> I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
>>>>>>> on the x86 code with STATIC_ASSERT_UNREACHABLE(),
>>>>>>> that is in file arch/x86/alternative.c, line 312,
>>>>>>> function _apply_alternatives().
>>>>>>>
>>>>>>> What I got is the following build error:
>>>>>>>
>>>>>>> ...
>>>>>>> arch/x86/alternative.c: Assembler messages:
>>>>>>> arch/x86/alternative.c:312: Error: static assertion failed: unreachable
>>>>>>>       CC      arch/x86/copy_page.o
>>>>>>> make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1
>>>>>>
>>>>>> But that's not what my request was about. Here sufficient context is
>>>>>> given, even if it would be nice if the function was also visible right
>>>>>> away. But that's not the same as the case above, where the new macro
>>>>>> is used inside another macro.
>>>>>
>>>>> An example of that is the get_unsafe_size() macro,
>>>>> whose body uses STATIC_ASSERT_UNREACHABLE().
>>>>> A wrong use of get_unsafe_size() at line n
>>>>> leads to a build error pointing to the line n,
>>>>> isn't this the desired behavior?
>>>>
>>>> Aiui this would point to the line in the header file, when what you need
>>>> to spot the bad use of the macro is the line in the source file actually
>>>> using the macro. Quoting from an earlier mail of yours:
>>>>
>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>
>>> It points to the header file uaccess.h because at line 262 there is
>>> an intentional wrong use of put_guest_size(), within the body of
>>> __copy_to_guest_pv() function.
>>
>> Yet that's again only a helper function being inlined into the ultimate
>> caller. That ultimate caller is what wants identifying in the diag. Not
>> the least because of ...
>>
>>> This example can be misleading because {get,put}_unsafe_size() are
>>> defined in the same file but the diagnostics is doing the
>>> right thing.
>>
>> ... this. And really __copy_to_guest_pv() is the wrong place to put a
>> wrong put_guest_size() in, to try out how diagnostics would look like
>> in reality: That function falls back to copy_to_guest_ll() for all
>> cases it can't handle directly. You want to place a bogus put_guest()
>> somewhere in a .c file to see what results.
> 
> I added a bogus call to put_guest() at line 387 of
> file xen/arch/x86/mm.c, inside function page_is_ram_type().
> Assuming I did not choose another wrong place,
> the diagnostic seems appropriate:
> 
> arch/x86/mm.c: Assembler messages:
> arch/x86/mm.c:387: Error: static assertion failed: unreachable

Oh, okay, this looks appropriate then as to identifying where the
source construct is. However, we then still don't know where the
assertion in question is (there could be multiple in what the
original construct expands to). So I'm still inclined to ask that
__FILE__ / __LINE__ and/or the name of the invoking construct
(macro or function) be made visible in the diagnostic.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 11:30:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 11:30:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678131.1055177 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY2bg-0006W8-Eh; Thu, 08 Feb 2024 11:30:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678131.1055177; Thu, 08 Feb 2024 11:30:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY2bg-0006W1-BT; Thu, 08 Feb 2024 11:30:16 +0000
Received: by outflank-mailman (input) for mailman id 678131;
 Thu, 08 Feb 2024 11:30:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY2be-0006Vs-WE
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 11:30:15 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6bb1d238-c675-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 12:30:11 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-4103fc91755so2890585e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 03:30:11 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h18-20020adffa92000000b0033afe6968bfsm3383070wrr.64.2024.02.08.03.30.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 03:30:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6bb1d238-c675-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707391811; x=1707996611; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pdhg3xSGzFMAZb/LNRGGE+OJKIs3hiKXgSsLNVGtgZU=;
        b=Do02MTB8GlOYbkiod3I5T6fhY16nVaq0t9Vsb5sPYpjdRzT8ihDN8ECHvFdVMXjR0M
         kWaHvNjqFEnChOG8mLmJO6QrKzu9TVft3TokmSt8dJErwbseSqRt0kJ3ShmRvvmNLjhx
         4KyqWGl+F4nxGT14+WA9bldgwvMEvqQ8NRsWXl/4/17+YWaYh4xLPMQo03jvdEHQG2ys
         4ydWiBqsjTG5p2+WJLCW65vc1SAaO6U6YErQ5iuy6zOhqj2srJoRNcs6YeHP9PYqy8FX
         0Ra3/SJDmdPH7nRzN0Lwe69IhM7rulZMPTVYkF6y5cjC+uuhwBZC/Ekh7H6wkvGXY/5/
         9T3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707391811; x=1707996611;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pdhg3xSGzFMAZb/LNRGGE+OJKIs3hiKXgSsLNVGtgZU=;
        b=Uw9Re8kuuXTwZrRDu5G51mjszk6r/TJKoc9lYk5mGesATi9w8QjOUpRjdw2h3PRmFG
         vUvaNX78h54HdMCxf6IMhLh225uPxW2i2a2KBWo9edqAupwQSUSgOhvzvjfW6crDh7Pr
         iefhrnqvLwPPHn9bdXo9bwPNYMm0sjPXra614OyRzd289ZhBsCFAb4C36VaKq7EGxc45
         q6CCh+/+2B430ShxYmJqIYxxMcX/TUplQreIlwxeMHoRDzECy1cIgtW6en4nH9i2Q9oc
         zkDHLGlSeG+LaFk8r6khV8oeQfQdDi/Y2kiKur+Q1odYAeOS4uzf778PNrJbq7NaKh7A
         um4g==
X-Forwarded-Encrypted: i=1; AJvYcCX9KSr4vAY4mh3UlnFjoI6bZihLBZhAv3p5G9FkXWlQXbQuFdOr1z3SnhspeMzfxEEA73PXVwvgK4S17sU6L7HZVo53yN2+25RLkNSAzyg=
X-Gm-Message-State: AOJu0YxiwfqZeg5uAgOPGaML/QEoE8/fQPc5iULOdrw023uQHm+UCTam
	uQZ3eFwKfNprLVNyk5O9BYbI2BSHQyAPfeVbdJdH1T4YSP/w8xxkk2cAjzxz9Q==
X-Google-Smtp-Source: AGHT+IGVyLts7G7/0rMJP9J77GnwsYcyC3kh2E1fvSbU0yiPw2dAHZndzGtfkZ/7KQFAoqNVWNSpXg==
X-Received: by 2002:a5d:4cc8:0:b0:33b:381d:a71e with SMTP id c8-20020a5d4cc8000000b0033b381da71emr5010235wrt.17.1707391811150;
        Thu, 08 Feb 2024 03:30:11 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCV+cVVYw+FTFCL5I3+hLQHLJHW1THwBycf9d6eIICWEUyEfM49SUvNOr5nXRh8SIX5R+JRNJ7WP7jJ3a/aiLLxObqWh501Dwm0GXGeUMBY7nRxYqcVcNim/xtRL2yXK4/OIM4knXttjQkTZR3XkHn3+
Message-ID: <665c7f47-a218-4187-858d-562bb5b9513b@suse.com>
Date: Thu, 8 Feb 2024 12:30:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 6/9] x86/shutdown: protect against recurrent
 machine_restart()
Content-Language: en-US
To: Krystian Hebel <krystian.hebel@3mdeb.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <87b0e650f28038c2fb64c5eb607c8fdaa7b4db07.1699982111.git.krystian.hebel@3mdeb.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <87b0e650f28038c2fb64c5eb607c8fdaa7b4db07.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.11.2023 18:50, Krystian Hebel wrote:
> If multiple CPUs called machine_restart() before actual restart took
> place, but after boot CPU declared itself not online,

Can you help me please in identifying where this operation is? I can see
two places where a CPU is removed from cpu_online_map, yet neither
__stop_this_cpu() nor __cpu_disable() ought to be coming into play here.
In fact I didn't think CPU0 was ever marked not-online. Except perhaps
if we came through machine_crash_shutdown() -> nmi_shootdown_cpus(), but
I'm sure you would have mentioned such a further dependency.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 11:32:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 11:32:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678133.1055187 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY2dO-00070V-Qa; Thu, 08 Feb 2024 11:32:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678133.1055187; Thu, 08 Feb 2024 11:32:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY2dO-00070O-MX; Thu, 08 Feb 2024 11:32:02 +0000
Received: by outflank-mailman (input) for mailman id 678133;
 Thu, 08 Feb 2024 11:32:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rY2dN-00070C-4I; Thu, 08 Feb 2024 11:32:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rY2dM-00054N-Ki; Thu, 08 Feb 2024 11:32:00 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rY2dM-0000sM-3K; Thu, 08 Feb 2024 11:32:00 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rY2dM-0006hs-2n; Thu, 08 Feb 2024 11:32:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Hxvx3j2A4EoX32oBWr7lZEUZuOn3I6oGOuENWNx0OBo=; b=Cyvg8Jyg/bDDKbrSyEuiDtAnfp
	+pH9i2MfcWCGagifq9E0yr+0MsBOYTaf2ut7e+7wgZRCQeDDh6FsaIlbV6vWVNzBR6uq3AFGQLulX
	zYJR5a0vi3tdkykOmAl3kyFLsKjlDECavDn6MBXDTJXa1B+prrbM9+y5i6whgSlbQTDw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184622-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184622: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f4519ee87ebfb74fe0451130d82c99453b75e2e1
X-Osstest-Versions-That:
    xen=59a69674b533a2d0c0ad5cd7fbcd655295574509
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 08 Feb 2024 11:32:00 +0000

flight 184622 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184622/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184617
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184617
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184617
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184617
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184617
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184617
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184617
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184617
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184617
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184617
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184617
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184617
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  f4519ee87ebfb74fe0451130d82c99453b75e2e1
baseline version:
 xen                  59a69674b533a2d0c0ad5cd7fbcd655295574509

Last test of basis   184617  2024-02-07 13:03:56 Z    0 days
Testing same since   184622  2024-02-08 01:39:41 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
  Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
  Simone Ballarin  <simone.ballarin@bugseng.com>
  Simone Ballarin <simone.ballarin@bugseng.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   59a69674b5..f4519ee87e  f4519ee87ebfb74fe0451130d82c99453b75e2e1 -> master


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 11:39:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 11:39:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678147.1055212 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY2kU-0008U4-Pj; Thu, 08 Feb 2024 11:39:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678147.1055212; Thu, 08 Feb 2024 11:39:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY2kU-0008Tx-N6; Thu, 08 Feb 2024 11:39:22 +0000
Received: by outflank-mailman (input) for mailman id 678147;
 Thu, 08 Feb 2024 11:39:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY2kT-0008Tr-0z
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 11:39:21 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b1df0893-c676-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 12:39:18 +0100 (CET)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-5114b2b3b73so2111808e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 03:39:18 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v12-20020a05600c470c00b0040ff7e3170asm1367838wmo.2.2024.02.08.03.39.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 03:39:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1df0893-c676-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707392358; x=1707997158; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sks6tmWIUqZoMfS1Ef5pE1EePh6Bv5w66w0wqO1z8U0=;
        b=NO8SkXI4ahjslOvnhNGwnb0JO23+6kSITcoNAlQFquwfzQLjaObH27iLJ8/9tfeDt4
         VhTNWjUV7j0N8x5+4ou9A6gda+ePhOWsvOZeUb/R/p8fvAwRdsVms/DSEOtTERuFAQid
         DSTlemOva1hPn/L/rEg6OlOoc8iD+8zaG0e6YYGKZ0Q2RM3qSORFEF1qL2fHK0lZdxdP
         kOXd0j94sHJx2SCYabHqxbC3pf4VOhCNfZF5NzMG38hEHf87DJF3eNs6pRzlkIWAYL3D
         TkT5rneZ3KUD2ALl5FjZTtEb9mzrKU/IpMrwqvOZ2p9FmPrrYRA/Cz0dTyR/bktTELID
         cRyg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707392358; x=1707997158;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sks6tmWIUqZoMfS1Ef5pE1EePh6Bv5w66w0wqO1z8U0=;
        b=A3X0fch4Xh1Zkth0D4vLC09nMBDhEv3UX9D025/pRwffbWfZKRITlaY6X34O00SU4r
         ivlYLehBAfp4zAwNRqIfBqYl3Q4br6nvSpXGmlTBPrWAtXoJ5tF0fxtyFBIpq71GGEHu
         5h6sSDI+Z9UckR2RGG+VZr+aWiZP5iF45KoyqEvQSdrEXZSh0X1Oul0qOZnhrKPenqr8
         Mc65/JxYhaktaEU2DcmMxHqHkhnNAPrgmCTidz5xJirpA0nZn0nfsrQYvY6rK1wfhT0m
         MYzJWeRAsr+/siVls91/JYVP/m5cI8Tplf8a7PPRwoffFt2mt+/k113adEIjhkILAWPU
         POHw==
X-Forwarded-Encrypted: i=1; AJvYcCU8jm70Di6pPF6H7vIM/Uespv8o+DMlJmuGNjCds/Qc0p2xt/0cMiYJIxOeMtwjzB1OhqzwiZUTc8IjPWFzSMZt+snnsOsrLYxCOUVmyeg=
X-Gm-Message-State: AOJu0YxOtzVab6wvGruhLMQbaHTXpAx08Hkdtfw593gm6iPXDglTMYaB
	B/c/lOzELaSf+lUXc1HOSd1u8Undp7J+oVzuf9Bs2o9v5Msrz1oJJjqYV7kG9A==
X-Google-Smtp-Source: AGHT+IF+acKMiFQ/xa67kZUT2IHl9yEGJ7HvyTL+MduhRmYbf1vRgPMEi3kzV/9biDUzBso5lPQGpQ==
X-Received: by 2002:a05:6512:e99:b0:511:564e:3a80 with SMTP id bi25-20020a0565120e9900b00511564e3a80mr8391615lfb.50.1707392358310;
        Thu, 08 Feb 2024 03:39:18 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWpk2ZQwaH3IOm+iC5H/O1Vc5YzBDuggUCPq1AWY86y3olOe1Gpv+mcRsX8A+DEa+P7bmKjTAkYEY8v3gwzUlUQnCYXOpNWEqQ7PyB7FsKALIdB9fCD2EMPVFbzQibMD/kzE+zVj3gs0TzRSVA+a9ct
Message-ID: <3df6b783-ff02-4ac9-ae76-843899fa2383@suse.com>
Date: Thu, 8 Feb 2024 12:39:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 7/9] x86/smp: drop booting_cpu variable
Content-Language: en-US
To: Krystian Hebel <krystian.hebel@3mdeb.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <22109ebd7edef1140cb438a6ec5fa1726cdf2c12.1699982111.git.krystian.hebel@3mdeb.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <22109ebd7edef1140cb438a6ec5fa1726cdf2c12.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.11.2023 18:50, Krystian Hebel wrote:
> CPU id is obtained as a side effect of searching for appropriate
> stack for AP. It can be used as a parameter to start_secondary().
> Coincidentally this also makes further work on making AP bring-up
> code parallel easier.

It's not just "easier", but strictly a prereq I think? Such a global
would get in the way of having multiple CPUs make it into
start_secondary() in parallel.

> --- a/xen/arch/x86/boot/x86_64.S
> +++ b/xen/arch/x86/boot/x86_64.S
> @@ -20,20 +20,24 @@ ENTRY(__high_start)
>          jz      .L_stack_set
>  
>          /* APs only: get stack base from APIC ID saved in %esp. */
> -        mov     $0, %rax
> +        mov     $0, %rbx
>          lea     cpu_data(%rip), %rcx
>          /* cpu_data[0] is BSP, skip it. */
>  1:
> -        add     $1, %rax
> +        add     $1, %rbx
>          add     $CPUINFO_X86_sizeof, %rcx
> -        cmp     $NR_CPUS, %eax
> +        cmp     $NR_CPUS, %rbx
>          jb      2f
>          hlt
>  2:
>          cmp     %esp, CPUINFO_X86_apicid(%rcx)
>          jne     1b

Once again this is code you introduced a few patches ago. Why not use
%ebx right away for that purpose? (And yes, this explains why in the
earlier patch you retained that code. Just that again suitably ordering
the series would make this look natural. Otherwise it needs at least
mentioning why dead pieces are kept around.)

> -        /* %rcx is now cpu_data[cpu], read stack base from it. */
> +        /*
> +         * At this point:
> +         * - %rcx is cpu_data[cpu], read stack base from it,
> +         * - %rbx (callee-save) is Xen cpu number, pass it to start_secondary().
> +         */
>          mov     CPUINFO_X86_stack_base(%rcx), %rsp
>  
>          test    %rsp,%rsp
> @@ -101,6 +105,7 @@ ENTRY(__high_start)
>  .L_ap_cet_done:
>  #endif /* CONFIG_XEN_SHSTK || CONFIG_XEN_IBT */
>  
> +        mov     %rbx, %rdi
>          tailcall start_secondary

As alluded to above and as mentioned before - please stick to 32-bit
operations when you deal with 32 (or less) bits of data.

> --- a/xen/arch/x86/smpboot.c
> +++ b/xen/arch/x86/smpboot.c
> @@ -222,8 +222,6 @@ static void smp_callin(void)
>          cpu_relax();
>  }
>  
> -static int booting_cpu;
> -
>  /* CPUs for which sibling maps can be computed. */
>  static cpumask_t cpu_sibling_setup_map;
>  
> @@ -311,15 +309,14 @@ static void set_cpu_sibling_map(unsigned int cpu)
>      }
>  }
>  
> -void start_secondary(void *unused)
> +void start_secondary(unsigned int cpu)
>  {
>      struct cpu_info *info = get_cpu_info();
>  
>      /*
> -     * Dont put anything before smp_callin(), SMP booting is so fragile that we
> +     * Don't put anything before smp_callin(), SMP booting is so fragile that we
>       * want to limit the things done here to the most necessary things.
>       */
> -    unsigned int cpu = booting_cpu;
>  
>      /* Critical region without IDT or TSS.  Any fault is deadly! */
>  
> @@ -346,9 +343,9 @@ void start_secondary(void *unused)
>       */
>      spin_debug_disable();
>  
> -    get_cpu_info()->use_pv_cr3 = false;
> -    get_cpu_info()->xen_cr3 = 0;
> -    get_cpu_info()->pv_cr3 = 0;
> +    info->use_pv_cr3 = false;
> +    info->xen_cr3 = 0;
> +    info->pv_cr3 = 0;

This hunk looks unrelated. While tidying next to what's changed anyway
may be okay if suitably mentioned in the description, in this case I
think it needs splitting off.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 11:49:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 11:49:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678152.1055222 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY2u4-0001sO-Sj; Thu, 08 Feb 2024 11:49:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678152.1055222; Thu, 08 Feb 2024 11:49:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY2u4-0001sH-Pz; Thu, 08 Feb 2024 11:49:16 +0000
Received: by outflank-mailman (input) for mailman id 678152;
 Thu, 08 Feb 2024 11:49:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mdps=JR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rY2u4-0001sB-32
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 11:49:16 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1492fcfe-c678-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 12:49:14 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a2a17f3217aso213966966b.2
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 03:49:14 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 gq12-20020a170906e24c00b00a3816515cd4sm1812331ejb.116.2024.02.08.03.49.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 08 Feb 2024 03:49:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1492fcfe-c678-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707392953; x=1707997753; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=+qTxWUbBQSGBGPIVl5GZg1+uuCSi3J+TktZnHjRxE9M=;
        b=h6LJJ9qPugvfPaNhBvaM9yz9ThtDWATfd+ndD+ZZnWK86CaIbq9nxurMkeqFoEki7I
         bBm/BC3r98hK/CL38uq68Ga7SE64mf7eoVAhW32xJFPKSAjTL8E/U2x/oR8+ydTps5k+
         5eT9hVytKNuqZHtKR7pN/qZoZaGbDGx/Sm0nU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707392953; x=1707997753;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+qTxWUbBQSGBGPIVl5GZg1+uuCSi3J+TktZnHjRxE9M=;
        b=tTye5RFwIcVWckj65mPSZATHdWo99ZYO+v+veWDeGBbRHPx9JiGRO3ZFHNbhYXiF8X
         dzWa1OVMmWovpc16cIUnACb1NfdFFay+GJJGr4KLBVBgXpUMdY8Ps4lqPRyUZXI8zV3c
         DVGZ17K/HfUSOy5AjIh7w2Mll7cAC40tvL3K0TzOu7F7zzamPNlevck0LicPWJqd+5LC
         e/kWgwGXj3hFK4eon2lOD0mZ5QFPETAXbkShxurpS9fU0bAPCXoP3mQD1tR2/9oYpujF
         HoSZROmuej6Dk8stVhrvTvoz7XRsDQ4Cl8FxtcGKjwHNY+rMmu4N6gkPA+6iuoZ8X3eg
         ldSQ==
X-Gm-Message-State: AOJu0YxdHOLYwwWTE3IRAyUOW2b6kvmrpSYGDAUpgE7xseDoVMgZohkt
	LEmhWWjnTmil+WvhloMtncoCvX+XmUbl90R2GiTwLte4/XSHqOyVbFoqGSfriU0=
X-Google-Smtp-Source: AGHT+IG3YqqOqA3vyZLfdBck11XK0C/kfM3095U73ByEA4rdmtCfav1RdTGNwjLP/HSrQanUPdPy3A==
X-Received: by 2002:a17:906:bcf8:b0:a37:9aaa:9c2e with SMTP id op24-20020a170906bcf800b00a379aaa9c2emr5831131ejb.76.1707392953426;
        Thu, 08 Feb 2024 03:49:13 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWYncylOMhE0q91k9VsqA3fkEsFUDT9HNq+qpPsjOhonilGGNymndN1cQlehoVnUiWzyI0yoaP38ivPY8rEXJIzid8E+tVouyBtutKQMTCXSImJZLgsI2e00ow8bKi11tT6X8WUGqpwIw+4spAE
Date: Thu, 8 Feb 2024 12:49:12 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 2/7] IOMMU: rename and re-type ats_enabled
Message-ID: <ZcS_uC2t96Lh720Y@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <467d24e1-8ed3-4dda-a334-70ff747bf94b@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <467d24e1-8ed3-4dda-a334-70ff747bf94b@suse.com>

On Mon, Feb 05, 2024 at 02:55:43PM +0100, Jan Beulich wrote:
> Make the variable a tristate, with (as done elsewhere) a negative value
> meaning "default". Since all use sites need looking at, also rename it
> to match our usual "opt_*" pattern. While touching it, also move it to
> .data.ro_after_init.
> 
> The only place it retains boolean nature is pci_ats_device(), for now.

Why does it retain the boolean nature in pci_ats_device()?

I assume this is to avoid having to touch the line again in a further
patch, as given the current logic pci_ats_device() would also want to
treat -1 as ATS disabled.

I think this is all fine because you add additional opt_ats > 0 checks
before the call to pci_ats_device(), but would be good to know this is
the intention.

> In AMD code re-order conditionals to have the config space accesses
> after (cheaper) flag checks.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> In domain_context_mapping_one() I'm a little puzzled that translation
> type is selected based on only IOMMU and global properties, i.e. not
> taking the device itself into account.

That seems like a bug to me, we should check that the device supports
ATS (and has it enabled) before setting the translation type to
CONTEXT_TT_DEV_IOTLB unconditionally.  We should likely use
ats_device() instead of ats_enabled in domain_context_mapping_one().

There's also IMO a second bug here, which is that we possibly attempt
to flush the device IOTLB before having ATS enabled.  We flush the
device TLB in domain_context_mapping_one(), yet ATS is enabled by the
caller afterwards (see domain_context_mapping()).

> 
> --- a/xen/drivers/passthrough/amd/iommu_cmd.c
> +++ b/xen/drivers/passthrough/amd/iommu_cmd.c
> @@ -282,7 +282,7 @@ void amd_iommu_flush_iotlb(u8 devfn, con
>      struct amd_iommu *iommu;
>      unsigned int req_id, queueid, maxpend;
>  
> -    if ( !ats_enabled )
> +    if ( opt_ats <= 0 )
>          return;
>  
>      if ( !pci_ats_enabled(pdev->seg, pdev->bus, pdev->devfn) )
> @@ -340,7 +340,7 @@ static void _amd_iommu_flush_pages(struc
>          flush_command_buffer(iommu, 0);
>      }
>  
> -    if ( ats_enabled )
> +    if ( opt_ats > 0 )
>      {
>          amd_iommu_flush_all_iotlbs(d, daddr, order);
>  
> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> @@ -185,10 +185,11 @@ static int __must_check amd_iommu_setup_
>          dte->ex = ivrs_dev->dte_allow_exclusion;
>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
>  
> -        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> +        if ( opt_ats > 0 &&
>               !ivrs_dev->block_ats &&
> -             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
> -            dte->i = ats_enabled;
> +             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> +             pci_ats_device(iommu->seg, bus, pdev->devfn) )
> +            dte->i = true;
>  
>          spin_unlock_irqrestore(&iommu->lock, flags);
>  
> @@ -248,10 +249,11 @@ static int __must_check amd_iommu_setup_
>          ASSERT(dte->sys_mgt == MASK_EXTR(ivrs_dev->device_flags,
>                                           ACPI_IVHD_SYSTEM_MGMT));
>  
> -        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> +        if ( opt_ats > 0 &&
>               !ivrs_dev->block_ats &&
> -             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
> -            ASSERT(dte->i == ats_enabled);
> +             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> +             pci_ats_device(iommu->seg, bus, pdev->devfn) )
> +            ASSERT(dte->i);
>  
>          spin_unlock_irqrestore(&iommu->lock, flags);
>  
> @@ -268,9 +270,10 @@ static int __must_check amd_iommu_setup_
>  
>      ASSERT(pcidevs_locked());
>  
> -    if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> +    if ( opt_ats > 0 &&
>           !ivrs_dev->block_ats &&
>           iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> +         pci_ats_device(iommu->seg, bus, pdev->devfn) &&
>           !pci_ats_enabled(iommu->seg, bus, pdev->devfn) )

Seeing that this same set of conditions is used in 3 different checks,
could we add a wrapper for it?

opt_ats > 0 && !ivrs_dev->block_ats &&
iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
pci_ats_device(iommu->seg, bus, pdev->devfn)

pci_device_ats_capable()? or some such.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 12:13:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 12:13:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678170.1055233 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY3HF-0006X2-1U; Thu, 08 Feb 2024 12:13:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678170.1055233; Thu, 08 Feb 2024 12:13:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY3HE-0006Wv-V5; Thu, 08 Feb 2024 12:13:12 +0000
Received: by outflank-mailman (input) for mailman id 678170;
 Thu, 08 Feb 2024 12:13:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY3HD-0006Wp-GQ
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 12:13:11 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6bf774ee-c67b-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 13:13:09 +0100 (CET)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-33b0ecb1965so1124397f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 04:13:09 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bn14-20020a056000060e00b0033b5a6b4b9bsm842507wrb.71.2024.02.08.04.13.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 04:13:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6bf774ee-c67b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707394388; x=1707999188; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=zijVtzTTJMWH7R5G6lvUwxpJR1glBtEArYNbKa6Ng9s=;
        b=U9/pJMKkmmdIiKp+L7dlH2oCjEiw7EelQGbCGdobhlWXLtBjc7fT1STTARRC6Fuj4S
         g1kbLQIL6fB1pXr9Iyely0reUw8/KC8sPZDFFlLR4xFvPo04GQ3ijq+ZZcJ0s+3/ILdA
         rlpM1SIG/SMPdAcTfGKSoUYtgiKrTU9tUD2G9/ZmGuwjs+kXH/DPWq9MoReJsGt0XZPS
         NwUG2mRRLiPJiqK+WcJRUH99vu1/TvZGqaehx+AdgJH+bceocjddkWX7J4l9NKZQTyUj
         KzoauQFTp6qkd2NUT6FYiy4nvXx4aZNNAutqnYVxYKRD3pEPD47hkeIhSolcT4SBFbZj
         Y8wA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707394388; x=1707999188;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=zijVtzTTJMWH7R5G6lvUwxpJR1glBtEArYNbKa6Ng9s=;
        b=DjzUoL05jvFf+ld/UzDAji6Rj9s3DXUX+u2mSjUC3AAL9Zb+3m0g635nVOzO+FmYP3
         608PfU+aOqXzCMjrD+JHGlOrvqwPL9auLo8Bul/ndDCjpqA0nqIvdx3Uvutnmpg6KDLZ
         AevMfchbOprCjyAocsC1VSI+Grz3w0fnJ1vFAEitONRvBV+0Nm7hxxVZq+3GraPeGmm/
         mv80bZsb9CD58Uh0BaFOl3gIYeZxyGdLDgkZ4Ael86YmyYRYRVoEzTgcaZFfnP2DCiyk
         ZBnE3gW1M5/WDOK5/1mb2VVd5mBB0X3kmkY33e5ZzbpKDLtOimd5MQWmhpzXzVDcGHdT
         xs2g==
X-Gm-Message-State: AOJu0YxXx1LDIbFAEChQGaCTRzLOiS77XBCla1GWHkNiBcqk7VsRR9vo
	nGbBQ8NJ8H8bvtxkMxSXrcTXmUpVxBFutxkyrZDT2dVMojD0mNQHjjfROzQHTQ==
X-Google-Smtp-Source: AGHT+IEo8wydfTyw7UcbA5yEJSrqGRir9w5aIsKii2VVd44MvBAo1OTrUN5UfM5Ujb0liTDxGHay1g==
X-Received: by 2002:a05:6000:4014:b0:33b:597a:5018 with SMTP id cp20-20020a056000401400b0033b597a5018mr1270925wrb.71.1707394388483;
        Thu, 08 Feb 2024 04:13:08 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWi+NyBq7EoM4oTxqySKuZ37QiGAb1oaYoLZ+VagRg6TVp+EBC1dEZDY/V7+4Rd59oTetjX8fhjIEKthV/mgls3MS/lSUuHm2ZFvtm5jkB8va4qgzW8RmbWLfsx33DyLc6j6UOVgav8+g1p8QVoxDal
Message-ID: <c1e23e38-aaac-44c5-a1af-999f71909110@suse.com>
Date: Thu, 8 Feb 2024 13:13:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 8/9] x86/smp: make cpu_state per-CPU
Content-Language: en-US
To: Krystian Hebel <krystian.hebel@3mdeb.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <52083114d4cbbc75f021e8c61763ad0e166cf05b.1699982111.git.krystian.hebel@3mdeb.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <52083114d4cbbc75f021e8c61763ad0e166cf05b.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.11.2023 18:50, Krystian Hebel wrote:
> This will be used for parallel AP bring-up.
> 
> CPU_STATE_INIT changed direction.

Nit: I think you mean "changes" as you describe what the patch does, not
what has happened before. But ...

> It was previously set by BSP and never
> consumed by AP. Now it signals that AP got through assembly part of
> initialization and waits for BSP to call notifiers that set up data
> structures required for further initialization.

... all of this is, afaict, independent of what the title says the
purpose of this patch is. Since the correctness of the state change
adjustments doesn't look straightforward to prove, please split the
mechanical change from the change to the actual logic.

> --- a/xen/arch/x86/include/asm/cpufeature.h
> +++ b/xen/arch/x86/include/asm/cpufeature.h
> @@ -38,6 +38,7 @@ struct cpuinfo_x86 {
>      unsigned int cpu_core_id;          /* core ID of each logical CPU */
>      unsigned int compute_unit_id;      /* AMD compute unit ID of each logical CPU */
>      void *stack_base;
> +    unsigned int cpu_state;
>      unsigned short x86_clflush_size;
>  } __cacheline_aligned;

Is there any reason this cannot be ordinary per-CPU data?

> --- a/xen/arch/x86/smpboot.c
> +++ b/xen/arch/x86/smpboot.c
> @@ -65,15 +65,18 @@ struct cpuinfo_x86 cpu_data[NR_CPUS] =
>          { [0 ... NR_CPUS-1] .apicid = BAD_APICID };
>  
>  static int cpu_error;
> -static enum cpu_state {
> +enum cpu_state {
>      CPU_STATE_DYING,    /* slave -> master: I am dying */
>      CPU_STATE_DEAD,     /* slave -> master: I am completely dead */
> -    CPU_STATE_INIT,     /* master -> slave: Early bringup phase 1 */
> -    CPU_STATE_CALLOUT,  /* master -> slave: Early bringup phase 2 */
> +    CPU_STATE_INIT,     /* slave -> master: Early bringup phase 1 completed */
> +    CPU_STATE_CALLOUT,  /* master -> slave: Start early bringup phase 2 */

It's not really clear to me whether the adding of "Start" on the 2nd line
really adds value.

>      CPU_STATE_CALLIN,   /* slave -> master: Completed phase 2 */
>      CPU_STATE_ONLINE    /* master -> slave: Go fully online now. */
> -} cpu_state;
> -#define set_cpu_state(state) do { smp_mb(); cpu_state = (state); } while (0)
> +};
> +#define set_cpu_state(cpu, state) do { \
> +    smp_mb(); \
> +    cpu_data[cpu].cpu_state = (state); \
> +} while (0)

While you merely re-arrange it, I'd still like to ask: Does this really
need to be smp_mb(), not just smp_wmb()?

> @@ -320,6 +317,10 @@ void start_secondary(unsigned int cpu)
>  
>      /* Critical region without IDT or TSS.  Any fault is deadly! */
>  
> +    /* Wait until data set up by CPU_UP_PREPARE notifiers is ready. */
> +    while ( cpu_data[cpu].cpu_state != CPU_STATE_CALLOUT )
> +        cpu_relax();

I'm afraid I don't understand the comment (and hence whether this loop
is actually needed here): __cpu_up() is called only after those
notifiers completed.

> @@ -1161,6 +1171,12 @@ void __init smp_prepare_cpus(void)
>      cpu_data[0].stack_base = (void *)
>               ((unsigned long)stack_start & ~(STACK_SIZE - 1));
>  
> +    /* Set state as CALLOUT so APs won't change it in initialize_cpu_data() */
> +    boot_cpu_data.cpu_state = CPU_STATE_CALLOUT;

This is actually one of the reasons I don't like you putting the item
as a new field in struct cpuinfo_x86. Otherwise imo initialize_cpu_data()
ought to gain a respective assertion.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 12:37:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 12:37:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678176.1055243 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY3eR-0001Rq-St; Thu, 08 Feb 2024 12:37:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678176.1055243; Thu, 08 Feb 2024 12:37:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY3eR-0001Rj-Ps; Thu, 08 Feb 2024 12:37:11 +0000
Received: by outflank-mailman (input) for mailman id 678176;
 Thu, 08 Feb 2024 12:37:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY3eP-0001Rd-Na
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 12:37:09 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c5be4bdd-c67e-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 13:37:08 +0100 (CET)
Received: by mail-wr1-x42c.google.com with SMTP id
 ffacd0b85a97d-337d05b8942so1311711f8f.3
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 04:37:08 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f2-20020a5d50c2000000b0033b4db744e5sm3523400wrt.12.2024.02.08.04.37.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 04:37:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5be4bdd-c67e-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707395827; x=1708000627; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=g+bO2YLDTgvwGVjHmGR/iqLM3C/eV82w8BM4exB2zLQ=;
        b=c/oimUwQX3cIde2F7/ZapzwLjjbvPuFsWslEmgHCA13+eqfb2xCuwYJITmWgu5pyBz
         G15sG/UOxHjUHBe6bNgzFxQc9yevuA9xFWCp2uIU2CtMiepZD2gP2eNBrxyDvPZRT3xW
         x0LQ3p+AFsR8rdbkMSWkqbDfGVnt9tZ+q8IZ1Bxl2mY2R+/aLR2uMOSMllQ+Y5zPjK2U
         zkeIpLdP07WfdyqHHNrvSFRDC+Y6G4F8jKU9IjvyFdW6iKclVOgZO6vnXZ53kbjqOLhn
         ekW1VYZYj5kmu2+1yMvPj3oiGTLrPy476Y/UHvLP5+rM2yAJ1/gPevy9uMYD39Cko8na
         o2yQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707395827; x=1708000627;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=g+bO2YLDTgvwGVjHmGR/iqLM3C/eV82w8BM4exB2zLQ=;
        b=G2ehzVRmTI2MTM0Lyq14RI3Pw6Gkit9og/3NI1DAAqwMWbGIZElXCorFl2RmbKul6H
         yzEX5ZJlwrNXiIHj5bLTCLLrPFABRGEWSZTMhf9YSWsAmess9cz2C8INHqJxQ/HmhkTJ
         GdY14lBmZ8sqdkwctddJir94d2mlMgTPE0paSI64y9nE8CBhHKiQl2GqEJJ4Vk/0bsQR
         scodJXNWGF8hgZY1iASoRwhoHsvJUNVj08SSTL9ViHP/q0ddDiEII3PsSDKI09ER5FWv
         9gPLsQ1CDk10BJKM4hf41jh8FhVLJh4xqXHuC2IdtPzEJD3mt6hPfq+dO31yBGfiO+ys
         jdWQ==
X-Forwarded-Encrypted: i=1; AJvYcCUfwXLhEKPpIaptMdxYrZCExMeIJlNh5fFnd3NoJTULVMdIG/pDTf3Shc2yminaE1n3HUw7mxiMAfQhg8QCMfAdN0tdLNiEUrahUPjuAws=
X-Gm-Message-State: AOJu0Yw8XQqX/xzg6j8pKXAXc0AVChiQGEwvpQSuQjIHulrbiFHgn0ld
	Mq8ehL8IRzCZIglhVQEc69czOr4d06ZnDx9FJC6EuK3wDKATLiERRO3mskAI/g==
X-Google-Smtp-Source: AGHT+IHtF6AS+YG5ehp1lZuKjmubSNJjlIvb3lKyA5iPGAmMxM8bvl9IOTXg/bTSBTmiV/EIQxtdcg==
X-Received: by 2002:adf:a490:0:b0:33b:50e5:8a78 with SMTP id g16-20020adfa490000000b0033b50e58a78mr3229679wrb.22.1707395827632;
        Thu, 08 Feb 2024 04:37:07 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCV0Kz8OFu1pwmM0HnfbsUMN9XxmpXSAostBsNrndDdLc7MGttviyJo+0/+1bRoGW73kZjT0e5fsS2UVBYtRTm31h31/nv/Dj5wI3egG4Q5az6EZ9ZX/mfa0C6Xg9HtyUr3etoPIpMUG2vV4nuIen9L0
Message-ID: <dad39029-d0fc-4aa0-8562-4c7a02ca8039@suse.com>
Date: Thu, 8 Feb 2024 13:37:06 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 9/9] x86/smp: start APs in parallel during boot
Content-Language: en-US
To: Krystian Hebel <krystian.hebel@3mdeb.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <cover.1699982111.git.krystian.hebel@3mdeb.com>
 <77c9199eabf3a30ebcf89356b2dd35abd611a3a9.1699982111.git.krystian.hebel@3mdeb.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <77c9199eabf3a30ebcf89356b2dd35abd611a3a9.1699982111.git.krystian.hebel@3mdeb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.11.2023 18:50, Krystian Hebel wrote:
> Multiple delays are required when sending IPIs and waiting for
> responses. During boot, 4 such IPIs were sent per each AP. With this
> change, only one set of broadcast IPIs is sent. This reduces boot time,
> especially for platforms with large number of cores.

Yet APs do their startup work in parallel only for a brief period of
time, if I'm not mistaken. Othwerwise I can't see why you'd still have
cpu_up() in __start_xen().

> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1963,6 +1963,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
>                  cpu_data[i].stack_base = cpu_alloc_stack(i);
>          }
>  
> +        smp_send_init_sipi_sipi_allbutself();
> +
>          for_each_present_cpu ( i )
>          {
>              if ( (park_offline_cpus || num_online_cpus() < max_cpus) &&

So what about constraints on the number of CPUs to use? In such a case
you shouldn't send the IPI to all of them, at least if they're not
meant to be parked.

> --- a/xen/arch/x86/smpboot.c
> +++ b/xen/arch/x86/smpboot.c
> @@ -425,7 +425,7 @@ void start_secondary(unsigned int cpu)
>  
>  static int wakeup_secondary_cpu(int phys_apicid, unsigned long start_eip)
>  {
> -    unsigned long send_status = 0, accept_status = 0;
> +    unsigned long send_status = 0, accept_status = 0, sh = 0;

sh doesn't need to be 64 bits wide, does it?

>      int maxlvt, timeout, i;
>  
>      /*
> @@ -445,6 +445,12 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned long start_eip)
>      if ( tboot_in_measured_env() && !tboot_wake_ap(phys_apicid, start_eip) )
>          return 0;
>  
> +    /*
> +     * Use destination shorthand for broadcasting IPIs during boot.
> +     */

Nit (style): This is a single line comment.

> +    if ( phys_apicid == BAD_APICID )
> +        sh = APIC_DEST_ALLBUT;

I think the latest for this the function parameter wants changing to
unsigned int (in another prereq patch).

> @@ -573,21 +578,31 @@ static int do_boot_cpu(int apicid, int cpu)
>       */
>      mtrr_save_state();
>  
> -    start_eip = bootsym_phys(trampoline_realmode_entry);
> +    /* Check if AP is already up. */
> +    if ( cpu_data[cpu].cpu_state != CPU_STATE_INIT )
> +    {
> +        /* This grunge runs the startup process for the targeted processor. */
> +        unsigned long start_eip;
> +        start_eip = bootsym_phys(trampoline_realmode_entry);
>  
> -    /* start_eip needs be page aligned, and below the 1M boundary. */
> -    if ( start_eip & ~0xff000 )
> -        panic("AP trampoline %#lx not suitably positioned\n", start_eip);
> +        /* start_eip needs be page aligned, and below the 1M boundary. */
> +        if ( start_eip & ~0xff000 )
> +            panic("AP trampoline %#lx not suitably positioned\n", start_eip);

Isn't this redundant now with the panic() in
smp_send_init_sipi_sipi_allbutself(), at least as long as that runs
unconditionally.

> -    /* So we see what's up   */
> -    if ( opt_cpu_info )
> -        printk("Booting processor %d/%d eip %lx\n",
> -               cpu, apicid, start_eip);
> +        /* So we see what's up   */
> +        if ( opt_cpu_info )
> +            printk("AP trampoline at %lx\n", start_eip);

Why this change in log message? It makes messages for individual CPUs
indistinguishable. And like above it's redundant with what
smp_send_init_sipi_sipi_allbutself() logs.

> -    /* This grunge runs the startup process for the targeted processor. */
> +        /* mark "stuck" area as not stuck */
> +        bootsym(trampoline_cpu_started) = 0;
> +        smp_mb();
>  
> -    /* Starting actual IPI sequence... */
> -    boot_error = wakeup_secondary_cpu(apicid, start_eip);
> +        /* Starting actual IPI sequence... */
> +        boot_error = wakeup_secondary_cpu(apicid, start_eip);
> +    }
> +
> +    if ( opt_cpu_info )
> +        printk("Booting processor %d/%d\n", cpu, apicid);

Oh, here's the other half. Yet for above it still doesn't make sense
to issue the same message for all CPUs.

> @@ -646,10 +661,6 @@ static int do_boot_cpu(int apicid, int cpu)
>          rc = -EIO;
>      }
>  
> -    /* mark "stuck" area as not stuck */
> -    bootsym(trampoline_cpu_started) = 0;
> -    smp_mb();

While you move this up, it's not clear to me how you would now
identify individual stuck CPUs. I would have expected that this is
another global that needs converting up front, to be per-CPU.

> @@ -1155,6 +1166,23 @@ static struct notifier_block cpu_smpboot_nfb = {
>      .notifier_call = cpu_smpboot_callback
>  };
>  
> +void smp_send_init_sipi_sipi_allbutself(void)

__init?

> +{
> +    unsigned long start_eip;
> +    start_eip = bootsym_phys(trampoline_realmode_entry);

This can be the initializer of the variable, which would then save
me from complaining about the missing blank line between declaration
and statement(s). (Actually, as I notice only now - same for code you
move around in do_boot_cpu().)

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 12:38:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 12:38:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678179.1055253 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY3fw-00021v-5o; Thu, 08 Feb 2024 12:38:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678179.1055253; Thu, 08 Feb 2024 12:38:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY3fw-00021o-2w; Thu, 08 Feb 2024 12:38:44 +0000
Received: by outflank-mailman (input) for mailman id 678179;
 Thu, 08 Feb 2024 12:38:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UFk1=JR=amazon.de=prvs=761b9cf69=mheyne@srs-se1.protection.inumbo.net>)
 id 1rY3fu-00021i-Rg
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 12:38:42 +0000
Received: from smtp-fw-80009.amazon.com (smtp-fw-80009.amazon.com
 [99.78.197.220]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fc430d95-c67e-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 13:38:41 +0100 (CET)
Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO
 smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210])
 by smtp-border-fw-80009.pdx80.corp.amazon.com with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2024 12:38:36 +0000
Received: from EX19MTAUEA002.ant.amazon.com [10.0.44.209:48592]
 by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.4.133:2525]
 with esmtp (Farcaster)
 id 8f6b3cd2-68f2-4838-a80e-f09091bf4d06; Thu, 8 Feb 2024 12:38:35 +0000 (UTC)
Received: from EX19D008UEC002.ant.amazon.com (10.252.135.242) by
 EX19MTAUEA002.ant.amazon.com (10.252.134.9) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1118.40; Thu, 8 Feb 2024 12:38:32 +0000
Received: from EX19MTAUWA001.ant.amazon.com (10.250.64.204) by
 EX19D008UEC002.ant.amazon.com (10.252.135.242) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.2.1118.40; Thu, 8 Feb 2024 12:38:32 +0000
Received: from dev-dsk-mheyne-1b-c1362c4d.eu-west-1.amazon.com (10.15.57.183)
 by mail-relay.amazon.com (10.250.64.204) with Microsoft SMTP Server
 id
 15.2.1118.40 via Frontend Transport; Thu, 8 Feb 2024 12:38:31 +0000
Received: by dev-dsk-mheyne-1b-c1362c4d.eu-west-1.amazon.com (Postfix,
 from userid 5466572)
 id 67924956; Thu,  8 Feb 2024 12:38:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc430d95-c67e-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209;
  t=1707395921; x=1738931921;
  h=date:from:to:subject:message-id:references:mime-version:
   in-reply-to;
  bh=XpaOl328Z0sK2jVAZ5hfANv8ZvM8k9ggJCj6QG2taxk=;
  b=VETxlY73eMhY7MzEZulFGG6AycRpfM4JY89u54jYMmmRXwEMro5nvY8V
   Mta0NEJBuy4DYxv+5LDywwWZariAyr9J9jae7Uaplvar7cTA1OH8s2Rio
   K1+UPOwc1xKuafxb6g64M8QMlCYkub2woJuJwStxVbQ+ftcFyBBgILvO4
   o=;
X-IronPort-AV: E=Sophos;i="6.05,253,1701129600"; 
   d="scan'208";a="64637088"
X-Farcaster-Flow-ID: 8f6b3cd2-68f2-4838-a80e-f09091bf4d06
Date: Thu, 8 Feb 2024 12:38:31 +0000
From: Maximilian Heyne <mheyne@amazon.de>
To: Julien Grall <jgrall@amazon.com>, <stable@vger.kernel.org>, "Andrew
 Panyakin" <apanyaki@amazon.com>, Juergen Gross <jgross@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Oleksandr Tyshchenko
	<oleksandr_tyshchenko@epam.com>, Rahul Singh <rahul.singh@arm.com>, "David
 Woodhouse" <dwmw@amazon.co.uk>, Viresh Kumar <viresh.kumar@linaro.org>,
	"Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>, Jeremy Fitzhardinge
	<jeremy.fitzhardinge@citrix.com>, <xen-devel@lists.xenproject.org>,
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] xen/events: close evtchn after mapping cleanup
Message-ID: <ZcTLR9uUyfy9cNUk@amazon.de>
References: <20240124163130.31324-1-mheyne@amazon.de>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240124163130.31324-1-mheyne@amazon.de>

On Wed, Jan 24, 2024 at 04:31:28PM +0000, Maximilian Heyne wrote:
> shutdown_pirq and startup_pirq are not taking the
> irq_mapping_update_lock because they can't due to lock inversion. Both
> are called with the irq_desc->lock being taking. The lock order,
> however, is first irq_mapping_update_lock and then irq_desc->lock.
> 
> This opens multiple races:
> - shutdown_pirq can be interrupted by a function that allocates an event
>   channel:
> 
>   CPU0                        CPU1
>   shutdown_pirq {
>     xen_evtchn_close(e)
>                               __startup_pirq {
>                                 EVTCHNOP_bind_pirq
>                                   -> returns just freed evtchn e
>                                 set_evtchn_to_irq(e, irq)
>                               }
>     xen_irq_info_cleanup() {
>       set_evtchn_to_irq(e, -1)
>     }
>   }
> 
>   Assume here event channel e refers here to the same event channel
>   number.
>   After this race the evtchn_to_irq mapping for e is invalid (-1).
> 
> - __startup_pirq races with __unbind_from_irq in a similar way. Because
>   __startup_pirq doesn't take irq_mapping_update_lock it can grab the
>   evtchn that __unbind_from_irq is currently freeing and cleaning up. In
>   this case even though the event channel is allocated, its mapping can
>   be unset in evtchn_to_irq.
> 
> The fix is to first cleanup the mappings and then close the event
> channel. In this way, when an event channel gets allocated it's
> potential previous evtchn_to_irq mappings are guaranteed to be unset already.
> This is also the reverse order of the allocation where first the event
> channel is allocated and then the mappings are setup.
> 
> On a 5.10 kernel prior to commit 3fcdaf3d7634 ("xen/events: modify internal
> [un]bind interfaces"), we hit a BUG like the following during probing of NVMe
> devices. The issue is that during nvme_setup_io_queues, pci_free_irq
> is called for every device which results in a call to shutdown_pirq.
> With many nvme devices it's therefore likely to hit this race during
> boot because there will be multiple calls to shutdown_pirq and
> startup_pirq are running potentially in parallel.
> 
>   ------------[ cut here ]------------
>   blkfront: xvda: barrier or flush: disabled; persistent grants: enabled; indirect descriptors: enabled; bounce buffer: enabled
>   kernel BUG at drivers/xen/events/events_base.c:499!
>   invalid opcode: 0000 [#1] SMP PTI
>   CPU: 44 PID: 375 Comm: kworker/u257:23 Not tainted 5.10.201-191.748.amzn2.x86_64 #1
>   Hardware name: Xen HVM domU, BIOS 4.11.amazon 08/24/2006
>   Workqueue: nvme-reset-wq nvme_reset_work
>   RIP: 0010:bind_evtchn_to_cpu+0xdf/0xf0
>   Code: 5d 41 5e c3 cc cc cc cc 44 89 f7 e8 2b 55 ad ff 49 89 c5 48 85 c0 0f 84 64 ff ff ff 4c 8b 68 30 41 83 fe ff 0f 85 60 ff ff ff <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 0f 1f 44 00 00
>   RSP: 0000:ffffc9000d533b08 EFLAGS: 00010046
>   RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000006
>   RDX: 0000000000000028 RSI: 00000000ffffffff RDI: 00000000ffffffff
>   RBP: ffff888107419680 R08: 0000000000000000 R09: ffffffff82d72b00
>   R10: 0000000000000000 R11: 0000000000000000 R12: 00000000000001ed
>   R13: 0000000000000000 R14: 00000000ffffffff R15: 0000000000000002
>   FS:  0000000000000000(0000) GS:ffff88bc8b500000(0000) knlGS:0000000000000000
>   CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>   CR2: 0000000000000000 CR3: 0000000002610001 CR4: 00000000001706e0
>   DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>   DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>   Call Trace:
>    ? show_trace_log_lvl+0x1c1/0x2d9
>    ? show_trace_log_lvl+0x1c1/0x2d9
>    ? set_affinity_irq+0xdc/0x1c0
>    ? __die_body.cold+0x8/0xd
>    ? die+0x2b/0x50
>    ? do_trap+0x90/0x110
>    ? bind_evtchn_to_cpu+0xdf/0xf0
>    ? do_error_trap+0x65/0x80
>    ? bind_evtchn_to_cpu+0xdf/0xf0
>    ? exc_invalid_op+0x4e/0x70
>    ? bind_evtchn_to_cpu+0xdf/0xf0
>    ? asm_exc_invalid_op+0x12/0x20
>    ? bind_evtchn_to_cpu+0xdf/0xf0
>    ? bind_evtchn_to_cpu+0xc5/0xf0
>    set_affinity_irq+0xdc/0x1c0
>    irq_do_set_affinity+0x1d7/0x1f0
>    irq_setup_affinity+0xd6/0x1a0
>    irq_startup+0x8a/0xf0
>    __setup_irq+0x639/0x6d0
>    ? nvme_suspend+0x150/0x150
>    request_threaded_irq+0x10c/0x180
>    ? nvme_suspend+0x150/0x150
>    pci_request_irq+0xa8/0xf0
>    ? __blk_mq_free_request+0x74/0xa0
>    queue_request_irq+0x6f/0x80
>    nvme_create_queue+0x1af/0x200
>    nvme_create_io_queues+0xbd/0xf0
>    nvme_setup_io_queues+0x246/0x320
>    ? nvme_irq_check+0x30/0x30
>    nvme_reset_work+0x1c8/0x400
>    process_one_work+0x1b0/0x350
>    worker_thread+0x49/0x310
>    ? process_one_work+0x350/0x350
>    kthread+0x11b/0x140
>    ? __kthread_bind_mask+0x60/0x60
>    ret_from_fork+0x22/0x30
>   Modules linked in:
>   ---[ end trace a11715de1eee1873 ]---
> 
> Fixes: d46a78b05c0e ("xen: implement pirq type event channels")
> Cc: stable@vger.kernel.org
> Co-debugged-by: Andrew Panyakin <apanyaki@amazon.com>
> Signed-off-by: Maximilian Heyne <mheyne@amazon.de>

Friendly ping. Did anyone find time to look at this?



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879





From xen-devel-bounces@lists.xenproject.org Thu Feb 08 12:42:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 12:42:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678184.1055262 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY3jf-0003sV-Nc; Thu, 08 Feb 2024 12:42:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678184.1055262; Thu, 08 Feb 2024 12:42:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY3jf-0003sO-Ky; Thu, 08 Feb 2024 12:42:35 +0000
Received: by outflank-mailman (input) for mailman id 678184;
 Thu, 08 Feb 2024 12:42:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mdps=JR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rY3je-0003sI-Dj
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 12:42:34 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 86ea671c-c67f-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 13:42:32 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-55ff5f6a610so1887213a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 04:42:32 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 i8-20020a05640200c800b0055ffa94d6b8sm773244edu.31.2024.02.08.04.42.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 08 Feb 2024 04:42:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 86ea671c-c67f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707396152; x=1708000952; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=DhOx3zcsQMMZLsZQMGj7+ND2NZNNZJqpdgSoYj0lico=;
        b=Dq4ezVkNBlSje/3Z8fkf30xn+ZcwQgqte3hb23pt2yiLPfEobMK1Bzpg2qllcucYKH
         NE+rfKXYWcybfl5te0dJvqKv8tbtSxTVYkgPfVQ+7ZeHXQLOIfrS9jDR0gqIp8dMgGgu
         nmAHdYpcjqPz1YMxNiN34wkA2mtfDAELriJAo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707396152; x=1708000952;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=DhOx3zcsQMMZLsZQMGj7+ND2NZNNZJqpdgSoYj0lico=;
        b=IEnEjz07VvCns8j5WgNg7P1eZ3DVCJMEfP8+KddyZCUb0ZNNWeWOYESoidvt6Lndcx
         KPHupumIyXE/zBqO2e6L6iyZPPLggwCAqkBbvKow69+46Yi1pAq5TZ+Ce6snj1q7S8qx
         RnJi8ycdGBo0q183Kb0u+Hw6trZ+S6nk2JOSid5YRZnDt69NlNKIUBHu4+4oFF8bkJP5
         oJGh+J+WIxxOjY0U7La1fnacoQ4iC+c6Yshojs7h8pvnGOH3K5RpicAItd96b2XbutuI
         E+KjCvGKXVSOjx65RlaHfnRIia3ooC2DTpRsji6v8ZVaU/RsmjDdNLbOV7UKmPpG3NyW
         sNXQ==
X-Gm-Message-State: AOJu0Yzru989250/6e9p6wxSl8Xx/Ae+EVxaNk43lHZksIlVtUg6+omk
	GyIBojybqYXGnS2zABZpWI9nw29Ii8h9D2/H9wx0y+czBDxOeKL4I0juwYjvUH2UIEaLE3oTlbP
	y
X-Google-Smtp-Source: AGHT+IH6PX/HjpKnGtXKGpU4N85hWOXaM1TkZQ44xyasBqxctlGiZlTO9NaKGygB0/gfyGoMz8yFDw==
X-Received: by 2002:aa7:c2d3:0:b0:560:62e0:d28e with SMTP id m19-20020aa7c2d3000000b0056062e0d28emr6474958edp.33.1707396151778;
        Thu, 08 Feb 2024 04:42:31 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW9DuxmUHT9pGQK0ya8v8Qg6XukFUVco9e4TNyM2WTj2zXCMJ1GgwE7c1CeABgPjMMEivQAmnSDpLppiA0unmEUXSzrZjGDWsneYufh6jl3uF27
Date: Thu, 8 Feb 2024 13:42:30 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 3/7] VT-d: respect ACPI SATC's ATC_REQUIRED flag
Message-ID: <ZcTMNkGvBT5S5z7L@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <72496fb7-8835-4d45-bd92-7e7d612861c7@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <72496fb7-8835-4d45-bd92-7e7d612861c7@suse.com>

On Mon, Feb 05, 2024 at 02:56:14PM +0100, Jan Beulich wrote:
> When the flag is set, permit Dom0 to control the device (no worse than
> what we had before and in line with other "best effort" behavior we use
> when it comes to Dom0), but suppress passing through to DomU-s unless
> ATS can actually be enabled for such devices.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> Is ats_device() using acpi_find_matched_atsr_unit() unconditionally
> actually correct? Shouldn't that check be skipped for root complex
> integrated devices?

Yes, I think so, ATSR only lists root ports supporting ATS, because
the root complex is assumed to always be ATS capable.

None of this seems to be working then for PCIe endpoints directly in
the root complex, as ats_device() will always return 0?

> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -225,7 +225,10 @@ exceptions (watchdog NMIs and unexpected
>  > Default: `false`
>  
>  Permits Xen to set up and use PCI Address Translation Services.  This is a
> -performance optimisation for PCI Passthrough.
> +performance optimisation for PCI Passthrough.  Note that firmware may indicate
> +that certain devices need to have ATS enabled for proper operation. For such
> +devices ATS will be enabled by default, unless the option is used in its
> +negative form.

I'm kind of worried that we add this support while maintaining the
WARNING below.  If I was an admin I would certainly be worried whether
my system could lock-up during normal operations, even with the
devices assigned to dom0 and not a malicious domain.

I know that enabling ATS is forced on us from DMAR, but still.

>  **WARNING: Xen cannot currently safely use ATS because of its synchronous wait
>  loops for Queued Invalidation completions.**
> --- a/xen/drivers/passthrough/vtd/dmar.c
> +++ b/xen/drivers/passthrough/vtd/dmar.c
> @@ -254,6 +254,24 @@ struct acpi_atsr_unit *acpi_find_matched
>      return all_ports;
>  }
>  
> +const struct acpi_satc_unit *acpi_find_matched_satc_unit(
> +    const struct pci_dev *pdev)
> +{
> +    const struct acpi_satc_unit *satc;
> +
> +    list_for_each_entry ( satc, &acpi_satc_units, list )
> +    {
> +        if ( satc->segment != pdev->seg )
> +            continue;
> +
> +        for ( unsigned int i = 0; i < satc->scope.devices_cnt; ++i )
> +            if ( satc->scope.devices[i] == pdev->sbdf.bdf )
> +                return satc;
> +    }
> +
> +    return NULL;
> +}
> +
>  struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd)
>  {
>      struct acpi_rhsa_unit *rhsa;
> --- a/xen/drivers/passthrough/vtd/dmar.h
> +++ b/xen/drivers/passthrough/vtd/dmar.h
> @@ -109,6 +109,8 @@ struct acpi_satc_unit {
>  
>  struct acpi_drhd_unit *acpi_find_matched_drhd_unit(const struct pci_dev *);
>  struct acpi_atsr_unit *acpi_find_matched_atsr_unit(const struct pci_dev *);
> +const struct acpi_satc_unit *acpi_find_matched_satc_unit(
> +    const struct pci_dev *pdev);
>  
>  #define DMAR_TYPE 1
>  #define RMRR_TYPE 2
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -2364,6 +2364,25 @@ static int cf_check intel_iommu_add_devi
>      if ( ret )
>          dprintk(XENLOG_ERR VTDPREFIX, "%pd: context mapping failed\n",
>                  pdev->domain);
> +    else if ( !pdev->broken )
> +    {
> +        const struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
> +        const struct acpi_satc_unit *satc = acpi_find_matched_satc_unit(pdev);
> +
> +        /*
> +         * Prevent the device from getting assigned to an unprivileged domain
> +         * when firmware indicates ATS is required, but ATS could not be enabled
> +         * (e.g. because of being suppressed via command line option).
> +         */

I think a safer policy would be to prevent assigning any device that
has atc_required set unless opt_ats > 1 (ie: the user has explicitly
opted-in to the usage of ATS).

While we can't likely avoid ATS being enabled for devices having the
ATC_REQUIRED flag, we shouldn't allow passthrough to possibly
untrusted guests without notice.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 13:41:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 13:41:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678198.1055273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY4eV-0004Df-VB; Thu, 08 Feb 2024 13:41:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678198.1055273; Thu, 08 Feb 2024 13:41:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY4eV-0004DY-S7; Thu, 08 Feb 2024 13:41:19 +0000
Received: by outflank-mailman (input) for mailman id 678198;
 Thu, 08 Feb 2024 13:41:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY4eU-0004DS-C1
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 13:41:18 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bba9ccbb-c687-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 14:41:16 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-40fb3b5893eso14534245e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 05:41:16 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 iv9-20020a05600c548900b004104bc8d841sm555445wmb.13.2024.02.08.05.40.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 05:40:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bba9ccbb-c687-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707399676; x=1708004476; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=CABFFYl92xdWMf0DiklpDeUUblSm1jKtp8QnB6qJDPU=;
        b=T4SW54BtNIyMDaAwH1RI/YiBRAYiHbvrINdst9jznI7cJR61mhJfZEvWmqE1foLcpr
         RL+APBatjgL8MtCVLVW5pEyFwyI20GoRlVBCFMCO03VfHBs/PwcV+/G0bPjHdr6ayPsc
         bSvZIEMGRq7VXVQkNs68629PWV5NsgZM15gK8mV0UDAfYAKSyPZ1itrkG4QiAlWNELDQ
         dde1EfiRRJoqW+QrmczBWMkT/RMAVa9dqajqJHVOSUEawr/RCDz/dapQXy/hCV8fuyOB
         jp7S727ZzmeN479YN50gVkOZBffSlFBgdmO5qGoynnhEqnE4MvLkJKL+zUPngmsxd9gr
         tL/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707399676; x=1708004476;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=CABFFYl92xdWMf0DiklpDeUUblSm1jKtp8QnB6qJDPU=;
        b=D1z0TJ6HOU+A5zxHBCRIBD6Obs6qK7ThsfVS+bf45jdeG9aJI1DIFDA1Pb+PCGVaDk
         GYRLsmwhPocskxzJf4olCm7mv1xpkOjCxMX4gpXG18J+PF23D+OoHEecfEwBq9akeem3
         tN4wy8Rf0EVOmknqVhuGY0s2kN1AAM1Ea1kOkgAwg5j18qgz9rAyas134WTO3excLFJm
         QrvHZE5k3AQUvCWcW6lTvf2vMiX3nOaYxU4ucxMpzilSS7zZsA2rS3aCU7eo0at87tpx
         T7hZtdCsmWdTwMhn8/irKIVFHjNvkORzexZN/YAaNUgUDF/cfhaTzQoFWNIXaxX4RinL
         LXgA==
X-Forwarded-Encrypted: i=1; AJvYcCXCaJfrrgrONfp5gAOkLvCAsfJ3koMmkGXUrcSVf6ssTR3pZOoxVpw7UFSNzfbNPhtr4I14u7BVSTfRq7tQvEDu02hVcak8fYXH2ZDf+vE=
X-Gm-Message-State: AOJu0YzDO+xj4Uq4ZYhOiaEHu5PY6hn5fyuKuSvmQAe9lNO+uoinM5li
	a1mlpstgeH/rqsPTlwSx4bJaPFxxewYBjiZR0BIAwl81stGWHnEG2i6IOx6dXA==
X-Google-Smtp-Source: AGHT+IG3oBHD//gbzfxRjUQd4WkuFHz2ISynBvf6Lt65W4lSIu/vfVGC2mfXe/eyC0ASVTpSY1hfJQ==
X-Received: by 2002:a05:600c:19c7:b0:40f:dd7e:4a4d with SMTP id u7-20020a05600c19c700b0040fdd7e4a4dmr7276575wmq.31.1707399655165;
        Thu, 08 Feb 2024 05:40:55 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWGJ+SoKhdfLaHOPIBBF+cPkkrgnfV4rTLqm6ly/63qssg0T5TcKdtnWBWsHzJot/uLZJB7ViLPIawx+aLfkX1rpdvv99dwnAux04M+7ujkyzUzE10bNbyPx5NzwdfPumtzc+QOUa2EScn2aFayJtnjG7TWIAvCLO+bo36L9Rhpnzkmmt4ffha0FppO
Message-ID: <91e3fd09-8325-49b0-9d7b-43aacf2acd81@suse.com>
Date: Thu, 8 Feb 2024 14:40:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/vmx: add support for virtualize SPEC_CTRL
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240206142507.81985-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240206142507.81985-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 15:25, Roger Pau Monne wrote:
> @@ -2086,6 +2091,9 @@ void vmcs_dump_vcpu(struct vcpu *v)
>      if ( v->arch.hvm.vmx.secondary_exec_control &
>           SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY )
>          printk("InterruptStatus = %04x\n", vmr16(GUEST_INTR_STATUS));
> +    if ( cpu_has_vmx_virt_spec_ctrl )
> +        printk("SPEC_CTRL mask = %#016lx  shadow = %#016lx\n",
> +               vmr(SPEC_CTRL_MASK), vmr(SPEC_CTRL_SHADOW));

#0... doesn't make a lot of sense; only e.g. %#lx does. Seeing context
there's no 0x prefix there anyway. Having looked at the function the
other day, I know though that there's a fair mix of 0x-prefixed and
unprefixed hex numbers that are output. Personally I'd prefer if all
0x prefixes were omitted here. If you and Andrew think otherwise, I can
live with that, so long as we're at least striving towards consistent
output (I may be able to get to doing a conversion patch, once I know
which way the conversion should be).

> --- a/xen/arch/x86/hvm/vmx/vmx.c
> +++ b/xen/arch/x86/hvm/vmx/vmx.c
> @@ -823,18 +823,28 @@ static void cf_check vmx_cpuid_policy_changed(struct vcpu *v)
>      {
>          vmx_clear_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
>  
> -        rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
> -        if ( rc )
> -            goto out;
> +        if ( !cpu_has_vmx_virt_spec_ctrl )
> +        {
> +            rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
> +            if ( rc )
> +                goto out;
> +        }

I'm certainly okay with you doing it this way, but generally I'd prefer
if code churn was limited whjere possible. Here leveraging that rc is 0
on entry, a smaller change would be to

        if ( !cpu_has_vmx_virt_spec_ctrl )
            rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
        if ( rc )
            goto out;

(similarly below then).

>      else
>      {
>          vmx_set_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
>  
> -        rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
> -        if ( rc && rc != -ESRCH )
> -            goto out;
> -        rc = 0; /* Tolerate -ESRCH */
> +        /*
> +         * NB: there's no need to clear the virtualize SPEC_CTRL control, as
> +         * the MSR intercept takes precedence.
> +         */

The two VMCS values are, aiui, unused during guest entry/exit. Maybe
worth mentioning here as well, as that not being the case would also
raise correctness questions?

> --- a/xen/arch/x86/include/asm/msr.h
> +++ b/xen/arch/x86/include/asm/msr.h
> @@ -302,8 +302,13 @@ struct vcpu_msrs
>       * For PV guests, this holds the guest kernel value.  It is accessed on
>       * every entry/exit path.
>       *
> -     * For VT-x guests, the guest value is held in the MSR guest load/save
> -     * list.
> +     * For VT-x guests, the guest value is held in the MSR guest load/save list
> +     * if there's no support for virtualized SPEC_CTRL. If virtualized
> +     * SPEC_CTRL is enabled the value here signals which bits in SPEC_CTRL the
> +     * guest is not able to modify.  Note that the value for those bits used in
> +     * Xen context is also used in the guest context.  Setting a bit here
> +     * doesn't force such bit to set in the guest context unless also set in
> +     * Xen selection of SPEC_CTRL.

Hmm, this mask value is unlikely to be in need of being vCPU-specific.
I'd not even expect it to be per-domain, but simply global.

I also can't spot where you set that field; do we really mean to give
guests full control now that we have it (rather than e.g. running in
IBRS-always-on mode at least under certain conditions)? If intended to
be like this for now, this (to me at least) surprising aspect could
likely do with mentioning in the description.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 13:45:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 13:45:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678203.1055286 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY4in-0004qc-Gj; Thu, 08 Feb 2024 13:45:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678203.1055286; Thu, 08 Feb 2024 13:45:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY4in-0004qV-EC; Thu, 08 Feb 2024 13:45:45 +0000
Received: by outflank-mailman (input) for mailman id 678203;
 Thu, 08 Feb 2024 13:45:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9ajo=JR=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1rY4im-0004qP-7l
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 13:45:44 +0000
Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com
 [136.143.188.12]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 58a99e84-c688-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 14:45:41 +0100 (CET)
Received: from mail-yw1-f177.google.com (mail-yw1-f177.google.com
 [209.85.128.177]) by mx.zohomail.com
 with SMTPS id 170739993763192.12847165744085;
 Thu, 8 Feb 2024 05:45:37 -0800 (PST)
Received: by mail-yw1-f177.google.com with SMTP id
 00721157ae682-604a3d33c4dso8941557b3.1
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 05:45:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58a99e84-c688-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; t=1707399938; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=MZNOKKcfMuEihrKl9ms5pooZutHgMwlw6Tm5BWMknV2HSqL/Rr6g6TtekqFTHcPNo89kyStwYcPLvTK2/+474fkVZyqnfS+pFcJFqlxO6t+oTmecp68FXR/uGOvRWWxzPepb0YygPKI8ryzmZH8Ntu7h6LVgG0BOUbyNF+/HZsY=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1707399938; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=tjXjQaxzEOyNGIfNc6u0nIFLCWtCu/UY1BqT0L8u9LE=; 
	b=XaVQKg4QQfJ9VBUaiZef3qgDpC6xubuXCldl4keDdybC4UGdT5ZUoGUoiAwH5x99ugxPetT1ub6vhGoXLY55f2+r/YKYsJy2Jx9H+xJPFcoWHztUbXIqiLl9TJf6y7IFMad/gs/WFH7zR1NMqxP7xMTbIU8CQEABmiH0SnUzcBY=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1707399938;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=tjXjQaxzEOyNGIfNc6u0nIFLCWtCu/UY1BqT0L8u9LE=;
	b=MO7qqhIMqQ834IceewfKx4FuvZ6bTWyLjY+sfUZKdWAp0/8rFXJtNASR1zGhRgZi
	OEeUcGbkj6GP3rthqHge9MdMpMCy51Mx9u2wIAzxMaqv7r0BBVSCME6M6TJy5I/96tU
	sVVvcFNZxKORFILHIFIBPgpUnSosgWZKyu+ldylQ=
X-Forwarded-Encrypted: i=1; AJvYcCWFJCpnbglG35aW3UjVCZUvGrB5SQ9w/UpW4KXpwzuaByJVPQzI6RK4rmS+vU6QVIW1dRibKc3LJOh7pBxOCYYmUS4G13DOSOkhavBQjes=
X-Gm-Message-State: AOJu0YzLWVqXfd3MBuCWvw2S6KcUq/roX0PDAeNTPUEDYsp7q+NHazEK
	xqazl/f9cBtELFRDoLvx58cbc13X5dQyjZXjudpzBHTaKioFGq1/6FoYFNUI/YH1Ezyfa+kGfoF
	TZESzTVlnI9Z5t8lXirJ8/KFeA+c=
X-Google-Smtp-Source: AGHT+IGKwratiCXm/VVOg3+QguJ2fjWe+3MCJq+zct0YYRBxrQqTLCOxgjFMCaiZEV4n3asLC4w27OG1MaI2yH4WFcA=
X-Received: by 2002:a81:cf04:0:b0:604:4b89:bf4 with SMTP id
 u4-20020a81cf04000000b006044b890bf4mr8187743ywi.20.1707399936709; Thu, 08 Feb
 2024 05:45:36 -0800 (PST)
MIME-Version: 1.0
References: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
 <CA+zSX=beYAGb81zqD19YNAV3cXPFTk-7V68XYn59Eb3mGFH_Jg@mail.gmail.com>
 <507f29cf-7a07-4f3d-81e8-2236a8380b1a@citrix.com> <CABfawh=6AHZzH1aBP2A=oRUTvjVmgd6QQNxEQAwGCBV_9-CTcA@mail.gmail.com>
 <CA+zSX=Zou19sjb8ToLTaMjLHLYmqptsMXCT50Y9MwGykOaTrZQ@mail.gmail.com> <156f2693-e178-4553-b4b4-f671b0221dfc@suse.com>
In-Reply-To: <156f2693-e178-4553-b4b4-f671b0221dfc@suse.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Thu, 8 Feb 2024 08:45:01 -0500
X-Gmail-Original-Message-ID: <CABfawhmYaGCLN2rsi4y+-CJs-AA+01YLEZdbkGvUi3t0MM42=A@mail.gmail.com>
Message-ID: <CABfawhmYaGCLN2rsi4y+-CJs-AA+01YLEZdbkGvUi3t0MM42=A@mail.gmail.com>
Subject: Re: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor ap2m->default_access
To: Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <george.dunlap@cloud.com>, =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org, 
	Alexandru Isaila <aisaila@bitdefender.com>, Petre Pircalabu <ppircalabu@bitdefender.com>, 
	Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 8, 2024 at 2:46=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 08.02.2024 05:32, George Dunlap wrote:
> > Er, ok, just one more comment: this could allow an altp2m to have more
> > permissions than the host; for example, the host p2m entry could be
> > p2m_access_r, but if the altp2m's default_access were p2m_access_rw,
> > it would override that.  Is that the behavior we want?  Or do we want
> > to do some sort of intersection of permissions?
> >
> > If the former, I'd propose the comment be adjusted thus:

No intersection of permissions please, that needlessly complicates
things and makes it hard to reason about the state of a view where
default permissions are used. No need to force a specific type of
usecase here where the hostp2m's permissions are special just cause we
say so. No, the permissions in the hostp2m should not have more weight
then the specifically requested default permission.

> >
> >  * If the entry is invalid, and the host entry was valid, propagate
> >  * the host's entry to the altp2m, retaining page order but using the
> >  * altp2m's default_access, and indicate that the caller should re-try
> >  * the faulting instruction.
>
> I find it highly questionable that such blind overriding should be taking
> place.

It's not blind overriding, it's the requested default permission set
for a view where no entry was present before. It is the expected
behavior. It would be way harder to design applications with this
feature if it was special cased and it would take different
permissions based on what permission is set in another view.

Tamas


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 14:00:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 14:00:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678216.1055296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY4xE-00089a-Od; Thu, 08 Feb 2024 14:00:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678216.1055296; Thu, 08 Feb 2024 14:00:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY4xE-00089T-Lu; Thu, 08 Feb 2024 14:00:40 +0000
Received: by outflank-mailman (input) for mailman id 678216;
 Thu, 08 Feb 2024 14:00:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY4xE-00089N-8N
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 14:00:40 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6fbfa450-c68a-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 15:00:38 +0100 (CET)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-33b18099411so1252694f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 06:00:37 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m25-20020a056000181900b0033b4235b155sm3643276wrh.48.2024.02.08.06.00.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 06:00:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fbfa450-c68a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707400837; x=1708005637; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Uyq1ZZ40J4/aUjA9k6OM/KOmgk4jM219yrQCxmgufoY=;
        b=aLrv9O6wAPbiU/PmO/7KNKx7m5eDjDK7UB86RgEzwmYumpvL7ox3POOPVTHOz0MYWZ
         afggCJZGX53iWJSPCFc/eRCJJEfvW98u6n8/w1JIC+wjGDb81+yv+cKqk1OXooSKPiuY
         YIh7OVRgpPV8QJUQ6Q/8HSaw0BeZ75b0HhaXcWJCIhs5FU0zDuVDterzP7IAC4UXkDCF
         qYc0nrI6aWpBDddRkxVtcwlfELjCMB+YQScJiDd9aBuvF4iCivDLOgGnHCDUu2isJIt+
         PnSIgqE1uDBVLx6CbbSBtxcbUOM8sFj1MhPzNqex0VIF0mmVWp+E1QIGUCHH98VkDTeT
         RfZA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707400837; x=1708005637;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Uyq1ZZ40J4/aUjA9k6OM/KOmgk4jM219yrQCxmgufoY=;
        b=sk5qps7EypjCE4ZShHOSzcOUMEscxgAQZfYbSYL5pBxwBxQdkL7k6+Cp48NlPi9O9u
         VbJ6HsjsN43UStmk83n4qaUohpt5SNUdRz5xNOq0hqeRZV1O+iPetAjul3zzM7vdQLBh
         0+1tsFn1seED8y5wHlmhXz51Qov4uPon9/kYULZm0TUdf9ccChqo35wwkEQCyNoFEFSa
         se+3KRHKkrEet+DqbQs1gC0NLr89fTEhUN4/qTGLm8HC9LyqOaK9L14Iu6Un4Fub9ahS
         jOOZQF7PnemL95shrqpV1HejLqXe/UhgFyf3+f5Z43dDcaEsEzLPvS0+5QTYmIovlFqS
         OBCw==
X-Forwarded-Encrypted: i=1; AJvYcCWEd+n43rnq1F0iFWAO/TyK9ztqbZooXvBlPtrRbjqMdpghvsuZCvXaqOBSQI95IkWYtrNzMPLngDwBw+F0KHBtnVKss7iRW9vutqpD/S4=
X-Gm-Message-State: AOJu0YyLzZkAWpWK/4louZlJzaHmi6wLvUrR7Lo/uEPtM1C63mRHiCGh
	QBOvf3yx8H0FPZT5jRSzXiJxaW2LkStgdHA2yMb76/wgPNFEj2vvAQGKX2ZUiQ==
X-Google-Smtp-Source: AGHT+IGo3FfxugMBGnhL2hFjWpurs+bL+X5PcVpF0wKWUjkvOcfa6d4y7zv7Q/b8Wg5Vjd9KJ5t8Hg==
X-Received: by 2002:adf:ce8c:0:b0:33b:136a:95a3 with SMTP id r12-20020adfce8c000000b0033b136a95a3mr5305528wrn.28.1707400837334;
        Thu, 08 Feb 2024 06:00:37 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUwSMlnzp+unxLae35BjcAFPuSVAeK1nRuxm0adM/EQ8MTD0/F4VazSoHGbTOo/Ay8rVhLEz+3F9lcY+eRoylXMuubS/4AfVe7jZxz1py0jRZepoUg6KbEicpeInUTqnWJhcPqNzM8HmymjDY2IfgLAsilCg91+JlluJ6O/N77xfZP2H+IVml2caAU0MlLbY0pqq1ffkaThLQa+/hHEsJ3H6g+3nz+OsbS4036FtTLz8E5/MIwhjlpbQPS4zrB4vWqu9QmyyQc1XOV1giAgTbLjEBkkv63ADW3BK+/AABBVdxZVP/QqrdD29/Z+mprJ
Message-ID: <6e818fdc-defe-41dd-a931-f5abc7ffce6b@suse.com>
Date: Thu, 8 Feb 2024 15:00:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor
 ap2m->default_access
Content-Language: en-US
To: Tamas K Lengyel <tamas@tklengyel.com>
Cc: George Dunlap <george.dunlap@cloud.com>, =?UTF-8?Q?Petr_Bene=C5=A1?=
 <w1benny@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 xen-devel@lists.xenproject.org, Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
 <CA+zSX=beYAGb81zqD19YNAV3cXPFTk-7V68XYn59Eb3mGFH_Jg@mail.gmail.com>
 <507f29cf-7a07-4f3d-81e8-2236a8380b1a@citrix.com>
 <CABfawh=6AHZzH1aBP2A=oRUTvjVmgd6QQNxEQAwGCBV_9-CTcA@mail.gmail.com>
 <CA+zSX=Zou19sjb8ToLTaMjLHLYmqptsMXCT50Y9MwGykOaTrZQ@mail.gmail.com>
 <156f2693-e178-4553-b4b4-f671b0221dfc@suse.com>
 <CABfawhmYaGCLN2rsi4y+-CJs-AA+01YLEZdbkGvUi3t0MM42=A@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CABfawhmYaGCLN2rsi4y+-CJs-AA+01YLEZdbkGvUi3t0MM42=A@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08.02.2024 14:45, Tamas K Lengyel wrote:
> On Thu, Feb 8, 2024 at 2:46 AM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 08.02.2024 05:32, George Dunlap wrote:
>>> Er, ok, just one more comment: this could allow an altp2m to have more
>>> permissions than the host; for example, the host p2m entry could be
>>> p2m_access_r, but if the altp2m's default_access were p2m_access_rw,
>>> it would override that.  Is that the behavior we want?  Or do we want
>>> to do some sort of intersection of permissions?
>>>
>>> If the former, I'd propose the comment be adjusted thus:
> 
> No intersection of permissions please, that needlessly complicates
> things and makes it hard to reason about the state of a view where
> default permissions are used. No need to force a specific type of
> usecase here where the hostp2m's permissions are special just cause we
> say so. No, the permissions in the hostp2m should not have more weight
> then the specifically requested default permission.
> 
>>>
>>>  * If the entry is invalid, and the host entry was valid, propagate
>>>  * the host's entry to the altp2m, retaining page order but using the
>>>  * altp2m's default_access, and indicate that the caller should re-try
>>>  * the faulting instruction.
>>
>> I find it highly questionable that such blind overriding should be taking
>> place.
> 
> It's not blind overriding, it's the requested default permission set
> for a view where no entry was present before. It is the expected
> behavior. It would be way harder to design applications with this
> feature if it was special cased and it would take different
> permissions based on what permission is set in another view.

But the default can be only one specific value: How do you make sure that
R/O, R/X, and R/W mappings all retain their respective restrictions in the
alternative view? To not over-restrict permissions, the default would then
need to be RWX, yet then all mappings will have full permission. What am I
missing?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 14:11:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 14:11:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678239.1055320 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY586-0002RN-2l; Thu, 08 Feb 2024 14:11:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678239.1055320; Thu, 08 Feb 2024 14:11:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY586-0002RG-04; Thu, 08 Feb 2024 14:11:54 +0000
Received: by outflank-mailman (input) for mailman id 678239;
 Thu, 08 Feb 2024 14:11:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rY584-0002R6-O3; Thu, 08 Feb 2024 14:11:52 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rY584-0007uH-Gy; Thu, 08 Feb 2024 14:11:52 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rY583-00070w-VS; Thu, 08 Feb 2024 14:11:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rY583-0002Ob-Uy; Thu, 08 Feb 2024 14:11:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=QDe0ypPPc3uNhZHYDQqvST50CabnVbERsviOB2cgEfg=; b=yhDJnu6uQfOUW71sDEcMH06Tgb
	ysSiZSZZe5ytju11s7UgiTqCnnvHRp0XgVoSq/JOThbhNLFJPSXX1lBSUx2Qc0SN8CKA6DjD3N/Ze
	PWWbrcIcVFc92EJuCvRznJyrU5taYE85NrHfs+IM5CFI0Q4ZMgxUWSgJvxaviPrEuNjA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184624-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184624: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=9ef6541cecde4ebc16bcfc9c5860006f0d8fb2a9
X-Osstest-Versions-That:
    libvirt=b051762e3e3968a1790d084264e46ddd4e56b9ec
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 08 Feb 2024 14:11:51 +0000

flight 184624 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184624/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184613
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184613
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184613
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              9ef6541cecde4ebc16bcfc9c5860006f0d8fb2a9
baseline version:
 libvirt              b051762e3e3968a1790d084264e46ddd4e56b9ec

Last test of basis   184613  2024-02-07 04:22:23 Z    1 days
Testing same since   184624  2024-02-08 04:19:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Göran Uddeborg <goeran@uddeborg.se>
  Jiri Denemark <jdenemar@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   b051762e3e..9ef6541cec  9ef6541cecde4ebc16bcfc9c5860006f0d8fb2a9 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 14:16:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 14:16:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678247.1055330 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY5CJ-00037x-NW; Thu, 08 Feb 2024 14:16:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678247.1055330; Thu, 08 Feb 2024 14:16:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY5CJ-00037q-Jg; Thu, 08 Feb 2024 14:16:15 +0000
Received: by outflank-mailman (input) for mailman id 678247;
 Thu, 08 Feb 2024 14:16:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Up58=JR=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rY5CI-00037k-BO
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 14:16:14 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:200a::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9b995cf1-c68c-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 15:16:11 +0100 (CET)
Received: from BN8PR12CA0019.namprd12.prod.outlook.com (2603:10b6:408:60::32)
 by IA0PR12MB8279.namprd12.prod.outlook.com (2603:10b6:208:40c::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.14; Thu, 8 Feb
 2024 14:16:06 +0000
Received: from BN3PEPF0000B074.namprd04.prod.outlook.com
 (2603:10b6:408:60:cafe::6) by BN8PR12CA0019.outlook.office365.com
 (2603:10b6:408:60::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.36 via Frontend
 Transport; Thu, 8 Feb 2024 14:16:06 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN3PEPF0000B074.mail.protection.outlook.com (10.167.243.119) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Thu, 8 Feb 2024 14:16:05 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Thu, 8 Feb
 2024 08:16:04 -0600
Received: from XIR-MICHALO-L1.xilinx.com (10.180.168.240) by
 SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34
 via Frontend Transport; Thu, 8 Feb 2024 08:16:02 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b995cf1-c68c-11ee-8a4a-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SYYYhhuo25wqmJlmPzC791mh/BJFs6+04A6jgikJplY/se25rcbADV9B4WLT7Boua5I6uSitfTHF6g30xYt44sdBXgjr4stLBgjP2uo+GuONRJYuztKuRjoKNe9bvC8EAzVS9p1/wKZBeyNYbDDBo7BGOYdK+PFNhHNwF6hQK7leGlT8qk0I6GsUfwSH4NoBkYwSsP5KUpLoz0HSvI8NVVp7aSWaoBtVHlTP5foXnmHGXv4DAJq/2FUkXyeyMqwEBDJAFiOyXrJLeNYfW+58Rx9PTfYxS+Y5cVEGqh38zev+iH7r6TsWQrdgZrRxFaCBJ/fngF1EiSFl+RRCLw8mEA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=eYJ+n8SemN9KXBcE3LP38y3El3PaoT/CJmuWmm7LOco=;
 b=hP/hUzgAusXpAcnXh9cTyTqUBAfNJC+swY73EL3ReTYGZFKKdo5pemT1ULT1BS/8Xebabmg+caQbR4+dU5WpSu+OGPAHmtF62AQvTODYogWerlat38kf+1Kesz0vZR6gNUOCSNGglE5VKug/1N/vpmnFjd77+bhM8UyRIUrHuJmMOswzbyRIETXMLEK1xpPcTta3QSc0gLFuicuPpOyGJrbMLgVSjdOUdi7X/DTg+XBccJG6PvkoVj93k/iPX268jFDxWErRht+6N7tVP1nhsJuzOGd9Yi0rsIVBWGcGNHHuYwlM1TNXB74LsKxIpIET/fYeGCL2vLqytFulmzu8qQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eYJ+n8SemN9KXBcE3LP38y3El3PaoT/CJmuWmm7LOco=;
 b=xhq6ZtTL0OHKZwbQLlj9CHEnByOiC0DwDh1F4AGjk1XVKd6P9roQAR2VR6eRtIvwoDev98mII6vVBfUl3iIJdxSIR1EQpP37ylg11YcCqGo3QzBQH6rFah+MY0woR8XqZD+9RtC4mxFBGDKzWEzQFUzGgY7YSZwX1OqL/UB14Ps=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Michal Orzel <michal.orzel@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Michal Orzel <michal.orzel@amd.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: [PATCH] xen/serial: Drop serial_{put,get}c
Date: Thu, 8 Feb 2024 15:16:01 +0100
Message-ID: <20240208141601.78023-1-michal.orzel@amd.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN3PEPF0000B074:EE_|IA0PR12MB8279:EE_
X-MS-Office365-Filtering-Correlation-Id: db149e00-21be-4b77-4c67-08dc28b07cfe
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	qpDWTXmItqzI9oZWdkFnPh55oXhiiA8aKDDd1hh01KWCcWVY39fvK7rGvZXnO8BS0ErDwSSwULLUYhGuW3NU+tFHj5bF44K4q5SHvvj5oy4xujWyJwzE0uufpGc2bHMxMcU0eJWKB43FYEgO5MDeCSFi4T7JO1sV2iWChamzVKFqh96U2j8nS0PepjPe1vXCtCtnmpDsNR3HbPEeTEUze7igklPgHzua0psBnVbcaa1hxiYd2K3dJwpGlDE2S0lieiWPrV1+5ZUu7I24eVGX7Pph+/xIEvLIbr8+A83T/RimxJgiXVWaWCrRyMwzwQfJzKBagXHtMfg+GONGe5G/rgOby130AUul1C+sLFhMlg9in/1tQWGvy6cvecTH2G3s9eWQCxsBguw2lL94deWqqKmFp757TwoX3xxN7YYGqjesSxrJ9uubtnZnQ4GBdt8ZaiSyQYg/QxWuzocQXiwx3Et8r78/hj/gG+749nIB6ujSwKiVlUyIAKvq2jRlq6skZyp6Lr6XQftMBq4Ji4u/KPM4U/ojMPsx6VYiBzQnkZf/YU570lZDVhjp1ovq0bwSqsd5Y02aETVn5YtI0K/zjl+T8WhhzdoGtq/7TZXiBcc=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(346002)(376002)(396003)(39860400002)(230922051799003)(82310400011)(64100799003)(186009)(1800799012)(451199024)(46966006)(36840700001)(40470700004)(2616005)(1076003)(86362001)(70206006)(6916009)(70586007)(54906003)(316002)(356005)(82740400003)(26005)(336012)(426003)(81166007)(83380400001)(478600001)(4326008)(8936002)(8676002)(44832011)(36756003)(5660300002)(41300700001)(2906002);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2024 14:16:05.5069
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: db149e00-21be-4b77-4c67-08dc28b07cfe
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN3PEPF0000B074.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8279

After removal of gdbstub by commit d182f9feae6d ("x86: Remove gdbstub"),
we no longer have users of serial_{put,get}c functions, so remove them.

Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
 xen/drivers/char/serial.c | 72 ---------------------------------------
 xen/include/xen/serial.h  | 10 ------
 2 files changed, 82 deletions(-)

diff --git a/xen/drivers/char/serial.c b/xen/drivers/char/serial.c
index 324024c29a10..93e217d34859 100644
--- a/xen/drivers/char/serial.c
+++ b/xen/drivers/char/serial.c
@@ -196,36 +196,6 @@ static void __serial_putc(struct serial_port *port, char c)
     }
 }
 
-void serial_putc(int handle, char c)
-{
-    struct serial_port *port;
-    unsigned long flags;
-
-    if ( handle == -1 )
-        return;
-
-    port = &com[handle & SERHND_IDX];
-    if ( !port->driver || !port->driver->putc )
-        return;
-
-    spin_lock_irqsave(&port->tx_lock, flags);
-
-    if ( (c == '\n') && (handle & SERHND_COOKED) )
-        __serial_putc(port, '\r' | ((handle & SERHND_HI) ? 0x80 : 0x00));
-
-    if ( handle & SERHND_HI )
-        c |= 0x80;
-    else if ( handle & SERHND_LO )
-        c &= 0x7f;
-
-    __serial_putc(port, c);
-
-    if ( port->driver->flush )
-        port->driver->flush(port);
-
-    spin_unlock_irqrestore(&port->tx_lock, flags);
-}
-
 void serial_puts(int handle, const char *s, size_t nr)
 {
     struct serial_port *port;
@@ -261,48 +231,6 @@ void serial_puts(int handle, const char *s, size_t nr)
     spin_unlock_irqrestore(&port->tx_lock, flags);
 }
 
-char serial_getc(int handle)
-{
-    struct serial_port *port;
-    char c;
-    unsigned long flags;
-
-    if ( handle == -1 )
-        return '\0';
-
-    port = &com[handle & SERHND_IDX];
-    if ( !port->driver || !port->driver->getc )
-        return '\0';
-
-    do {
-        for ( ; ; )
-        {
-            spin_lock_irqsave(&port->rx_lock, flags);
-            
-            if ( port->rxbufp != port->rxbufc )
-            {
-                c = port->rxbuf[mask_serial_rxbuf_idx(port->rxbufc++)];
-                spin_unlock_irqrestore(&port->rx_lock, flags);
-                break;
-            }
-            
-            if ( port->driver->getc(port, &c) )
-            {
-                spin_unlock_irqrestore(&port->rx_lock, flags);
-                break;
-            }
-
-            spin_unlock_irqrestore(&port->rx_lock, flags);
-
-            cpu_relax();
-            udelay(100);
-        }
-    } while ( ((handle & SERHND_LO) &&  (c & 0x80)) ||
-              ((handle & SERHND_HI) && !(c & 0x80)) );
-    
-    return c & 0x7f;
-}
-
 int __init serial_parse_handle(const char *conf)
 {
     int handle, flags = 0;
diff --git a/xen/include/xen/serial.h b/xen/include/xen/serial.h
index 12ef24351d0f..7548bb73249b 100644
--- a/xen/include/xen/serial.h
+++ b/xen/include/xen/serial.h
@@ -112,19 +112,9 @@ void serial_endboot(void);
 /* Takes a config string and creates a numeric handle on the COM port. */
 int serial_parse_handle(const char *conf);
 
-/* Transmit a single character via the specified COM port. */
-void serial_putc(int handle, char c);
-
 /* Transmit a string via the specified COM port. */
 void serial_puts(int handle, const char *s, size_t nr);
 
-/*
- * An alternative to registering a character-receive hook. This function
- * will not return until a character is available. It can safely be
- * called with interrupts disabled.
- */
-char serial_getc(int handle);
-
 /* Forcibly prevent serial lockup when the system is in a bad way. */
 /* (NB. This also forces an implicit serial_start_sync()). */
 void serial_force_unlock(int handle);
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 14:20:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 14:20:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678252.1055340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY5Gl-00053J-9x; Thu, 08 Feb 2024 14:20:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678252.1055340; Thu, 08 Feb 2024 14:20:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY5Gl-00053C-5e; Thu, 08 Feb 2024 14:20:51 +0000
Received: by outflank-mailman (input) for mailman id 678252;
 Thu, 08 Feb 2024 14:20:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9ajo=JR=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1rY5Gj-000536-Qw
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 14:20:49 +0000
Received: from sender3-op-o17.zoho.com (sender3-op-o17.zoho.com
 [136.143.184.17]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4001a791-c68d-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 15:20:47 +0100 (CET)
Received: from mail-yw1-f182.google.com (mail-yw1-f182.google.com
 [209.85.128.182]) by mx.zohomail.com
 with SMTPS id 1707402043602976.6873532347577;
 Thu, 8 Feb 2024 06:20:43 -0800 (PST)
Received: by mail-yw1-f182.google.com with SMTP id
 00721157ae682-60491b1fdeaso19566497b3.3
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 06:20:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4001a791-c68d-11ee-8a4a-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; t=1707402045; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=erM+YJ+oErLz1jNKp9+2tLdezaW6rMeY2qT5KKWstrvztH4QTQLpT8+x+/eyuRQAeAvm9PDU6AXDJZ1neUiNFYyFlNLe7RrppGP3TaVx3stcGWu4AfhvEUolDtCaIGE58tYcbCn4ls3vik8tjTKJhlPsTC//0V5owQudYxriM8Y=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1707402045; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=wH+tUFLsw/E6eFvRCsiAGUx0r/1n1jC6F6nrBzmppjw=; 
	b=J9hCL+1ft5Jj/81uYp0xdbbf7XI7lD/EYRixMIiPMXA+qXxWqTSULPb9SVG1FHWJWMesg9dvJP6Z0bHkiAqVQallz2IpKr/hwDjlPUTDLyfHx572m4OK+NVkM2rFMt/to2Igo8nTSBuaiP2t3GsK5g6ltR1xPr/TjStOqNcvXNQ=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1707402045;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=wH+tUFLsw/E6eFvRCsiAGUx0r/1n1jC6F6nrBzmppjw=;
	b=AIVdhcoZMerD4D6lqqpCtfuUXWmZh4keWUJaynvcraYbrwod2ZA57iwewvLy8Joz
	HEUQ5Wvi8GztvNF5zSyaYQ4ivuZf14FmqXujk/RvyA+LfBGjSiHj5hAtrJL6ssruVEA
	/Kq8c6Y4KyooHIIAUFpZJgV426cjEVpLuW6EWOT4=
X-Gm-Message-State: AOJu0YwhLq8dVyE4rxutVvRt5Bu0lma8biLEJI6dJiMZjP9Ccv1xj551
	Q0VdZVVEtKzMSGWBCzgxUK20HgclrRQearSa6Ve2TGKJTe2St14MOfq75HO+KB7lLrpxEejEmZr
	xL9jJ6aZSAnWyOytdmPf/60CfRHE=
X-Google-Smtp-Source: AGHT+IFceimg/GjYkEUIUG53o1KeeQn4Mexb/J/g3sLk6W3oX0ZW/RiFta/u/oAk9iluY5p0qczTtxrWAvRabiFMXcA=
X-Received: by 2002:a05:6902:220f:b0:dc6:d2d9:308c with SMTP id
 dm15-20020a056902220f00b00dc6d2d9308cmr8969559ybb.36.1707402042697; Thu, 08
 Feb 2024 06:20:42 -0800 (PST)
MIME-Version: 1.0
References: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
 <CA+zSX=beYAGb81zqD19YNAV3cXPFTk-7V68XYn59Eb3mGFH_Jg@mail.gmail.com>
 <507f29cf-7a07-4f3d-81e8-2236a8380b1a@citrix.com> <CABfawh=6AHZzH1aBP2A=oRUTvjVmgd6QQNxEQAwGCBV_9-CTcA@mail.gmail.com>
 <CA+zSX=Zou19sjb8ToLTaMjLHLYmqptsMXCT50Y9MwGykOaTrZQ@mail.gmail.com>
 <156f2693-e178-4553-b4b4-f671b0221dfc@suse.com> <CABfawhmYaGCLN2rsi4y+-CJs-AA+01YLEZdbkGvUi3t0MM42=A@mail.gmail.com>
 <6e818fdc-defe-41dd-a931-f5abc7ffce6b@suse.com>
In-Reply-To: <6e818fdc-defe-41dd-a931-f5abc7ffce6b@suse.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Thu, 8 Feb 2024 09:20:07 -0500
X-Gmail-Original-Message-ID: <CABfawhmCr7B47HJyWrdZC13J3muA19AOtRK1a6Fmz5kvQ6FdYQ@mail.gmail.com>
Message-ID: <CABfawhmCr7B47HJyWrdZC13J3muA19AOtRK1a6Fmz5kvQ6FdYQ@mail.gmail.com>
Subject: Re: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor ap2m->default_access
To: Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <george.dunlap@cloud.com>, =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org, 
	Alexandru Isaila <aisaila@bitdefender.com>, Petre Pircalabu <ppircalabu@bitdefender.com>, 
	Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 8, 2024 at 9:00=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 08.02.2024 14:45, Tamas K Lengyel wrote:
> > On Thu, Feb 8, 2024 at 2:46=E2=80=AFAM Jan Beulich <jbeulich@suse.com> =
wrote:
> >>
> >> On 08.02.2024 05:32, George Dunlap wrote:
> >>> Er, ok, just one more comment: this could allow an altp2m to have mor=
e
> >>> permissions than the host; for example, the host p2m entry could be
> >>> p2m_access_r, but if the altp2m's default_access were p2m_access_rw,
> >>> it would override that.  Is that the behavior we want?  Or do we want
> >>> to do some sort of intersection of permissions?
> >>>
> >>> If the former, I'd propose the comment be adjusted thus:
> >
> > No intersection of permissions please, that needlessly complicates
> > things and makes it hard to reason about the state of a view where
> > default permissions are used. No need to force a specific type of
> > usecase here where the hostp2m's permissions are special just cause we
> > say so. No, the permissions in the hostp2m should not have more weight
> > then the specifically requested default permission.
> >
> >>>
> >>>  * If the entry is invalid, and the host entry was valid, propagate
> >>>  * the host's entry to the altp2m, retaining page order but using the
> >>>  * altp2m's default_access, and indicate that the caller should re-tr=
y
> >>>  * the faulting instruction.
> >>
> >> I find it highly questionable that such blind overriding should be tak=
ing
> >> place.
> >
> > It's not blind overriding, it's the requested default permission set
> > for a view where no entry was present before. It is the expected
> > behavior. It would be way harder to design applications with this
> > feature if it was special cased and it would take different
> > permissions based on what permission is set in another view.
>
> But the default can be only one specific value: How do you make sure that
> R/O, R/X, and R/W mappings all retain their respective restrictions in th=
e
> alternative view? To not over-restrict permissions, the default would the=
n
> need to be RWX, yet then all mappings will have full permission. What am =
I
> missing?

Why do you assume you need to retain the permissions that were set in
the hostp2m across all altp2ms? It is entirely reasonable to set
permissions for a couple entries manually in an altp2m and set the
default to _n, which may be totally different then what the hostp2m
has. When you get the event for entries you didn't manually specify
you can decide which view to switch to, which may be the hostp2m, but
may be some other view.

If you wanted to have a view with a default permission that inherits
permissions that are always at least as restrictive as the hostp2m,
you could do that, but the way to do that would be to introduce a
special mem_access permission to signify that that's the expected
behavior (similar to permissions like n2rwx). I don't think anyone is
asking for that.

Tamas


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 14:29:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 14:29:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678259.1055349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY5Oe-00063m-2f; Thu, 08 Feb 2024 14:29:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678259.1055349; Thu, 08 Feb 2024 14:29:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY5Od-00063f-W9; Thu, 08 Feb 2024 14:28:59 +0000
Received: by outflank-mailman (input) for mailman id 678259;
 Thu, 08 Feb 2024 14:28:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY5Oc-00063X-AY
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 14:28:58 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 63e0be6a-c68e-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 15:28:56 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 5CFC01FCF0;
 Thu,  8 Feb 2024 14:28:55 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 11E561326D;
 Thu,  8 Feb 2024 14:28:55 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id TgzlAiflxGWLZgAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 14:28:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 63e0be6a-c68e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707402535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=xCB83GoEaZ2jEoriM2v1h2fr0UV28YqVdRI+kWmendo=;
	b=fOozfGaTf9KAAppNDj/vKVms3IF9+KysCYWBOU1O6x3/yMrfdlSz/XKWs/XZ2Tw2g+kJzk
	NzieJwpsCTgrTzRqHt5QGeI5C/yDeov9o/xY8vrwC1JOu55LKEqI+N9OJKYc6kO6pTyQeM
	AGTGS5DuZbXjQk5PrrSOtnVE8b3Bf5c=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707402535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=xCB83GoEaZ2jEoriM2v1h2fr0UV28YqVdRI+kWmendo=;
	b=fOozfGaTf9KAAppNDj/vKVms3IF9+KysCYWBOU1O6x3/yMrfdlSz/XKWs/XZ2Tw2g+kJzk
	NzieJwpsCTgrTzRqHt5QGeI5C/yDeov9o/xY8vrwC1JOu55LKEqI+N9OJKYc6kO6pTyQeM
	AGTGS5DuZbXjQk5PrrSOtnVE8b3Bf5c=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2] config: update Mini-OS commit
Date: Thu,  8 Feb 2024 15:28:52 +0100
Message-Id: <20240208142852.3819-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [4.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[18.83%]
X-Spam-Level: ****
X-Spam-Score: 4.90
X-Spam-Flag: NO

Update the Mini-OS upstream revision.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- newer commit fixing a bug
---
 Config.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Config.mk b/Config.mk
index f7d6d84847..6f6e0425ba 100644
--- a/Config.mk
+++ b/Config.mk
@@ -224,7 +224,7 @@ QEMU_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/qemu-xen.git
 QEMU_UPSTREAM_REVISION ?= master
 
 MINIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/mini-os.git
-MINIOS_UPSTREAM_REVISION ?= 090eeeb1631f00a9a41ebf66d9b4aacb97eb51e7
+MINIOS_UPSTREAM_REVISION ?= 2bc8dbb9b6111113d27455cbca318ea337309c04
 
 SEABIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/seabios.git
 SEABIOS_UPSTREAM_REVISION ?= rel-1.16.3
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 15:07:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 15:07:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678269.1055360 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY5zC-0003xR-1W; Thu, 08 Feb 2024 15:06:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678269.1055360; Thu, 08 Feb 2024 15:06:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY5zB-0003xK-V9; Thu, 08 Feb 2024 15:06:45 +0000
Received: by outflank-mailman (input) for mailman id 678269;
 Thu, 08 Feb 2024 15:06:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IT3j=JR=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1rY5zB-0003xE-0A
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 15:06:45 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 (mail-ve1eur01on0631.outbound.protection.outlook.com
 [2a01:111:f400:fe1f::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aaa8015f-c693-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 16:06:42 +0100 (CET)
Received: from DU2PR04CA0048.eurprd04.prod.outlook.com (2603:10a6:10:234::23)
 by VI1PR08MB9984.eurprd08.prod.outlook.com (2603:10a6:800:1c7::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.34; Thu, 8 Feb
 2024 15:06:38 +0000
Received: from DU2PEPF0001E9C6.eurprd03.prod.outlook.com
 (2603:10a6:10:234:cafe::af) by DU2PR04CA0048.outlook.office365.com
 (2603:10a6:10:234::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.24 via Frontend
 Transport; Thu, 8 Feb 2024 15:06:38 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU2PEPF0001E9C6.mail.protection.outlook.com (10.167.8.75) with
 Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7249.19 via Frontend Transport; Thu, 8 Feb 2024 15:06:38 +0000
Received: ("Tessian outbound c4f080b252bb:v228");
 Thu, 08 Feb 2024 15:06:37 +0000
Received: from da2037b5c704.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 E7B4DB36-3256-424E-A5C6-8651DE28165C.1; 
 Thu, 08 Feb 2024 15:06:31 +0000
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id da2037b5c704.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 08 Feb 2024 15:06:31 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by GV1PR08MB10690.eurprd08.prod.outlook.com (2603:10a6:150:161::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.39; Thu, 8 Feb
 2024 15:06:28 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::2257:5fcb:b449:fff0]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::2257:5fcb:b449:fff0%7]) with mapi id 15.20.7249.039; Thu, 8 Feb 2024
 15:06:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aaa8015f-c693-11ee-98f5-efadbce2ee36
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=ngQuuuJPYzPpoKaLJbulVBUuCYt5veGPA6k0NTWEdIVK1XU6HMwB00fVKV/kPjPzKu0n/PXzG8K2bwH08tvkiZ5OkXvShGSHnUf+kLRTK9qFd+uJytu4vZDDhp3lBqk+nW7JqYTLiAr4JJhGR28xYxIPR3SP5nsW/hXfe/ZiuC+CiUdbOD5DK9lcNfn+wnwkikuNAdz7Cox79KthlO5/ymLCatwa9ugd6o9QyaCVlhO1OURO4Icil0rWwjTai2BmMHZCfssY/7vyGNdFSfMgZxpyZasapicdXmNyDxiEQrQXoRbay49Y1rZI1PFtpaIYy6aazm/z4IZ9AqV0FKU/iQ==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ROFXKZ8pa30sestvhraDOKnrZn0OrdE8qBJgGskW+o8=;
 b=b++lmlYEHIu8GMQjhEiX+4uCCwPq9FeUkUwNrQVET4vbu2GiXyF6EWuEfZw9uUWbxzdwhnKsl2If/fD4wGHE/Lw5Ws9ZrNtTurbplSpD+TgDgxd224MwWcIOtMPNNG1fJG7PfsNZV9ZHbKdWVJG48aLEgkaxyi8gnLuJhv+5rDRZimC151emgPyP2Tfk7zq71+xNocTbkEfNtBgzReSOTDvujB6GdmGttIachZ2lxerZMCMJS4sj5SblUVgan5h+08HXezeITOKcJGp1vykhqwyWrE0KaRU8xonyy0nW5IDFYeBu0f7XZMV2zluV0ao2ur9+xynKJuCqj+NjXNDaaQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0
 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ROFXKZ8pa30sestvhraDOKnrZn0OrdE8qBJgGskW+o8=;
 b=e3zvwqb1px67NItZCQSxlduAetYT3/vaYKStQiXH9NCKd5zmyQBK6Cz8YN6Ub1CxdF1q25AdFIbn9eeGajnpJ+z84tC1AOve9Id/9b1c7t6z414RgPB9Lk7gIQUyRBnOaNQKPHOqgJKZRCrhrQ9G6SKhWrX0t8YvgmVM/60pDew=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: c639facb120889b3
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dm6PFpSOJxzAztFFgwaeMduO5KJs2d7sZLgKEKpuaOaB07jtZoBvDRn+gDXU+vWadjR+4XXN3VkmIPYIRRLXBoYXRMTIlqBzPec+02oSrHhNX42MfjSK8j55EYI78ZQYdnGj9rj4z3BzIRRQhbuY8WLUKlrKd4cXVguHEufw3RiXSKyVf2Xsu1+cRW3/dEpG2F/jBqU/kjfffyZAJSefBXbr2DAbW1JO0DCIxiQdUUTfgovhS4Wt2epFdnQ8KaBqW5eJs2+fyXx3SaBoq5U7aSB1bLLRvqOx9KZfQu/IFEFAkPPygjOksggOJLr6QHZW9AhDGmpqgp+g6XieGv3R9g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ROFXKZ8pa30sestvhraDOKnrZn0OrdE8qBJgGskW+o8=;
 b=QKJqcOqtMEQcB0Ve/5m+OHSUFaPhnT/4CRnDDUuVxWe5a1H4EJsWu5IU5aw40U0LEd+dd52v7pgekPW/3klgcJakt2KMPsryD+vKjygUqhUrs6hrpdgqYvbZPNPQgHp3aW+arFZZxQbgWTg3U4QNfH9JwIiyJ2CJa5FytbUkY7V1J45dgYYawBoCe5mi3rlypySgeeSM3qCGWWeubx8/WffB4xgc7R+Cj6zC37O52irnl9FY9TGEud4LgvDdlUqCbm8T8H4d1bqdlXSRDUuYM8dF65G3mQsbcfDVuQtfBaR3aNLbGala2zdyGW2y8qr2TMcthnz3Vz3HJ89dhR7I/g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ROFXKZ8pa30sestvhraDOKnrZn0OrdE8qBJgGskW+o8=;
 b=e3zvwqb1px67NItZCQSxlduAetYT3/vaYKStQiXH9NCKd5zmyQBK6Cz8YN6Ub1CxdF1q25AdFIbn9eeGajnpJ+z84tC1AOve9Id/9b1c7t6z414RgPB9Lk7gIQUyRBnOaNQKPHOqgJKZRCrhrQ9G6SKhWrX0t8YvgmVM/60pDew=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, George Dunlap
	<George.Dunlap@citrix.com>, Jan Beulich <JBeulich@suse.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Julien Grall
	<julien@xen.org>
Subject: Re: [PATCH] xen: Swap order of actions in the FREE*() macros
Thread-Topic: [PATCH] xen: Swap order of actions in the FREE*() macros
Thread-Index: AQHaVXBn1d4D59eCl0GZpSsSk57lq7EAldsA
Date: Thu, 8 Feb 2024 15:06:28 +0000
Message-ID: <89DCD9B2-7D9F-4AC3-AFFC-21584C63C8F5@arm.com>
References: <20240202003942.647599-1-andrew.cooper3@citrix.com>
In-Reply-To: <20240202003942.647599-1-andrew.cooper3@citrix.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.200.91.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|GV1PR08MB10690:EE_|DU2PEPF0001E9C6:EE_|VI1PR08MB9984:EE_
X-MS-Office365-Filtering-Correlation-Id: 4da54f54-c15a-4423-874e-08dc28b78c82
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 sMzTHCfhb+culVog/3Jesi0u1JwychbFgbD/28GjfSY4Vev7gWI3ykzqhqyLHDcCHSwdIRdEqo7O27L1/MOHIm+LF/DvPjptnK+0ZGArgPVjBI77MTq6VbPaz1NMNxLtOYOCzuzFMizApia/6uOxg3fKB+dsSHJbqr+J7GJWb9XVsor18JduKdH1ywULwWXda/WrmPL7KuRxCgsEELcAlO410/5WVwzryuBDbNwbnjfzQA6nGz+n00tSuC6PYNGvzE2VgqMsOMWso6PSDGrf7R5Q7IaivdBERlROWRzCzinEWDsGoz9mQ+3ZltLhX4/MjyX2nyvJNrLoKSU9vuY1Q0a4aD+3VHRJLOXiSzdlLjNIFcSTK3BmLkhYPVzktwh6BKLfdEmV02kSCwEhb6YgsgbrqAW1Gpfz3+5cB2HYEEebl5yE/h/sBorSN13KDTJe2khab1krV/uf6FFCE2FzBCnr4G13meamgyDW8z+5SQ1j0Fm7UJDee4qqehfQd+lXza1o29H2zGCre9jWCazwQQUFrTlo0rzgdFSslf0JNnvT2zH5Nh8F3drYNJJvt1C+k/QvbUPqGKtXBHVmyznPxM1S/8GG/7g4NxIg2ABpjbGL6MbTlLrCZXgxPoYSOQvV
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(366004)(396003)(376002)(136003)(346002)(230922051799003)(1800799012)(451199024)(186009)(64100799003)(33656002)(36756003)(41300700001)(38070700009)(76116006)(66476007)(66446008)(66556008)(54906003)(6916009)(66946007)(316002)(64756008)(4326008)(5660300002)(86362001)(8936002)(8676002)(122000001)(38100700002)(26005)(6506007)(6512007)(53546011)(478600001)(83380400001)(2616005)(2906002)(71200400001)(6486002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <6620D2D817992F41A045438CD8CEB9C4@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB10690
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU2PEPF0001E9C6.eurprd03.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	2c2f181c-1b0f-44f0-9fd8-08dc28b786fc
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	mw7HwFfXtxdBfdIIoG6R3JdECHvCCpsDGOCLpEHrFKv1blTzQQoAgpNi6pjM3Ifce4NjNGdMVH8PpBx5uROIixCjGfAKyQXq+ZAIaaIVtLDoo6X97y/ZlMgEbXcHAyRIwMn1WP6HoHzYynb0k0DCc1Cy1yDUnR2JyBZYFaH3OXwp6BX72FxN70Ko6Bu5U+GO+rtpHpsF4nUh87hPp5/evzNPTLWGWdsGXoe5czuJaKrF2un/5LiygRaE0VaZYcx1KmNoeDwUJInVJoOo3Uj7ODpy3kS8SdXVFFK7wyjuPvAU+o45h9KCCId480MYBJAYLQjQL6SOZQid4/xQjKaW1WJiZknrc7+xJMHpkZMKUd5X9+A6tbGjqlbvUbnNkTfd6ynh+oSPziUXCwdr8tbHAtT3IcLPxi9C+4L2/LXQb1CXFfnpw5ye7XrL6p3NEEUe70AL77ChHpJRXz6YNGunFks+sKBkrqjMuqUo+FPSBi1GApM3zOyVf/XaF1J21KGyxD0vhBcimy/v6kGjYO3NhGY70WGWdKpR0mRfODksgR6gKCNb7vJw70FLsS9lnEwn9TS+vC0ovPU7a98csKBCaACypWx/cJkxsRKNiOxs7eA=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(39850400004)(376002)(396003)(136003)(230922051799003)(186009)(451199024)(82310400011)(1800799012)(64100799003)(40470700004)(46966006)(36840700001)(26005)(83380400001)(33656002)(336012)(6506007)(70586007)(41300700001)(86362001)(54906003)(316002)(70206006)(2906002)(5660300002)(356005)(82740400003)(478600001)(2616005)(81166007)(6512007)(4326008)(53546011)(8936002)(6486002)(8676002)(36756003)(6862004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2024 15:06:38.1258
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 4da54f54-c15a-4423-874e-08dc28b78c82
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU2PEPF0001E9C6.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB9984

Hi Andrew,


> On 2 Feb 2024, at 00:39, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>=20
> Wherever possible, it is a good idea to NULL out the visible reference to=
 an
> object prior to freeing it.  The FREE*() macros already collect together =
both
> parts, making it easy to adjust.
>=20
> This has a marginal code generation improvement, as some of the calls to =
the
> free() function can be tailcall optimised.
>=20

Could you improve a bit the commit message and explain a bit why it is a go=
od idea
and also how the code might be improved because i do not get it ?

Cheers
Bertrand

> No functional change.
>=20
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: George Dunlap <George.Dunlap@citrix.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Wei Liu <wl@xen.org>
> CC: Julien Grall <julien@xen.org>
> ---
> xen/include/xen/mm.h      | 3 ++-
> xen/include/xen/xmalloc.h | 7 ++++---
> 2 files changed, 6 insertions(+), 4 deletions(-)
>=20
> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> index 3d9b2d05a5c8..044f3f3b19c8 100644
> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -92,8 +92,9 @@ bool scrub_free_pages(void);
>=20
> /* Free an allocation, and zero the pointer to it. */
> #define FREE_XENHEAP_PAGES(p, o) do { \
> -    free_xenheap_pages(p, o);         \
> +    void *_ptr_ =3D (p);                \
>     (p) =3D NULL;                       \
> +    free_xenheap_pages(_ptr_, o);     \
> } while ( false )
> #define FREE_XENHEAP_PAGE(p) FREE_XENHEAP_PAGES(p, 0)
>=20
> diff --git a/xen/include/xen/xmalloc.h b/xen/include/xen/xmalloc.h
> index 9ecddbff5e00..1b88a83be879 100644
> --- a/xen/include/xen/xmalloc.h
> +++ b/xen/include/xen/xmalloc.h
> @@ -66,9 +66,10 @@
> extern void xfree(void *p);
>=20
> /* Free an allocation, and zero the pointer to it. */
> -#define XFREE(p) do { \
> -    xfree(p);         \
> -    (p) =3D NULL;       \
> +#define XFREE(p) do {                       \
> +    void *_ptr_ =3D (p);                      \
> +    (p) =3D NULL;                             \
> +    xfree(_ptr_);                           \
> } while ( false )
>=20
> /* Underlying functions */
>=20
> base-commit: 3f819af8a796c0e2f798dd301ec8c3f8cccbc9fc
> --=20
> 2.30.2
>=20
>=20



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 15:29:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 15:29:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678275.1055370 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6LL-0007qQ-SX; Thu, 08 Feb 2024 15:29:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678275.1055370; Thu, 08 Feb 2024 15:29:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6LL-0007qJ-Ot; Thu, 08 Feb 2024 15:29:39 +0000
Received: by outflank-mailman (input) for mailman id 678275;
 Thu, 08 Feb 2024 15:29:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY6LK-0007qD-KQ
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 15:29:38 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dda62a31-c696-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 16:29:36 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-40fdc63f4feso18361465e9.3
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 07:29:36 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bv7-20020a0560001f0700b0033b525dbc70sm2625796wrb.79.2024.02.08.07.29.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 07:29:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dda62a31-c696-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707406176; x=1708010976; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=oEYIBMj4q82gAihAcQrBobhfSf02eMg+M+fb0JV5aNw=;
        b=bXa9C0CqFR+eMmo1uvMocIt+XaG4xxus35i2dRvjNU9Gj4UA7PxETGyk+WZFF/03+8
         3aiCLHEN/tdeRSs6IpR96avDnVWSzXAiqGvNhTNlD+AmjTNEWYN1cUTB8+JTHTBqmVME
         HUslw7xhpTLaSVY3RfhNQ7zmXwdd93lDWE6YGeb1B9utZnCoS/fvlWwBfAuQs9ZuPlsD
         YZFG4mV26Q9m06V/W1Co/cwGlMCOLA+M3+FHeIDGoWteXwYSRhTzQv112ckgx65ILsxq
         G5hPXm/IDnC/QzLQvV2vL2FQmC28BFyyoP8qm6+b2vgyS15P8uKZLnmFEWtPR/daYd5S
         +QCQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707406176; x=1708010976;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=oEYIBMj4q82gAihAcQrBobhfSf02eMg+M+fb0JV5aNw=;
        b=LIi+LRTBjFJCGs2A7yYpTqC7Urfw2awr3c9Xn8856YUirG3Iz1tXwx74HMyNTIfBik
         RHRNFHr2jZGhcn1X5UkwB93nZHL3eCmer3wxVHrF7UK44LQWffBKynRDsYY+cobvwP8o
         +gvVoU3aQAgv8XSIdgbtWINENVQbSSze+iKlyaTgGjuxZzdKKrkTXa1bLJEL2qI6v6ei
         DYCNb4OtUIoKPUr40zvmIKXOzjGlCifJkhr8xC3U/AtBBTjYx2w1rc45sxqJQROJAhZx
         byihLLq8QGph3kFt6B55AGSI9xt5cHbi/RlUDDfNXzPg3zlb5K+wHKeOs2vKkLl7iUPu
         ZHlw==
X-Gm-Message-State: AOJu0YwKqDeJmpguyV/PcMLno07mx8QrJKTYKfmGfylaEEVm9d3FigqN
	tYD2aravP4NqCODKfj0GhgIHJJo9c3larMfesT4UzABaro+5+rxPNtoa7OANwQ==
X-Google-Smtp-Source: AGHT+IH0JHFkLXsmYTwGwWLjWi5WyJrKcrGATWjR0rAhQyA9G5zeCsZo+AG+8UhW2VESqJKVfAwm2w==
X-Received: by 2002:a5d:4e86:0:b0:33b:401f:bd3e with SMTP id e6-20020a5d4e86000000b0033b401fbd3emr4723804wru.41.1707406175655;
        Thu, 08 Feb 2024 07:29:35 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWSfeAq7Fjb25QTB1r/6B/lo6jcwb8sbcuBxJmJkgn8CzeFFnjobnSP8xRjnRuB+1JOBqtktkgT3FY4cdpcOm8EXBMZHGCfvMlL0i+gWd4AL7Fg
Message-ID: <a96fe270-15f1-492e-92d6-2a7ceda11dec@suse.com>
Date: Thu, 8 Feb 2024 16:29:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/7] VT-d: parse ACPI "SoC Integrated Address Translation
 Cache Reporting Structure"s
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <b5a58dee-9a4c-4833-be59-b52c62f7137d@suse.com> <ZcScJvAhI7CRJhAZ@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcScJvAhI7CRJhAZ@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08.02.2024 10:17, Roger Pau Monné wrote:
> On Mon, Feb 05, 2024 at 02:55:17PM +0100, Jan Beulich wrote:
>> This is a prereq to us, in particular, respecting the "ATC required"
>> flag.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> Should we check scope entries for appropriate types? (If so, then also
>> for e.g. ATSR.)
> 
> Hm, I guess we could do so in acpi_parse_dev_scope() since that
> function already gets passed a 'type' argument.

Right, I transiently had it there, but then dropped it again for being
inconsistent with what we have right now. I'll try to remember to add
another patch.

>> @@ -764,6 +765,95 @@ acpi_parse_one_rhsa(struct acpi_dmar_hea
>>      return ret;
>>  }
>>  
>> +static int __init register_one_satc(struct acpi_satc_unit *satcu)
>> +{
>> +    bool ignore = false;
>> +    unsigned int i = 0;
>> +    int ret = 0;
>> +
>> +    /* Skip checking if segment is not accessible yet. */
>> +    if ( !pci_known_segment(satcu->segment) )
>> +        i = UINT_MAX;
>> +
>> +    for ( ; i < satcu->scope.devices_cnt; i++ )
>> +    {
>> +        uint8_t b = PCI_BUS(satcu->scope.devices[i]);
>> +        uint8_t d = PCI_SLOT(satcu->scope.devices[i]);
>> +        uint8_t f = PCI_FUNC(satcu->scope.devices[i]);
>> +
>> +        if ( pci_device_detect(satcu->segment, b, d, f) == 0 )
> 
> Any reason to explicitly compare against 0?
> 
> if ( !pci_device_detect(satcu->segment, b, d, f) )
> ...
> 
> The function returns a boolean.

Hmm, right - simply a result of copy-and-paste.

>> +        {
>> +            dprintk(XENLOG_WARNING VTDPREFIX,
>> +                    " Non-existent device (%pp) is reported in SATC scope!\n",
>> +                    &PCI_SBDF(satcu->segment, b, d, f));
>> +            ignore = true;
> 
> This is kind of reporting is incomplete: as soon as one device is
> found the loop is exited and no further detection happens.  If we want
> to print such information, we should do the full scan and avoid
> exiting early when a populated device is detected.

Not sure I follow, but first of all - these are dprintk()s only, so
meant to only help in dev environments. Specifically ...

>> +        }
>> +        else
>> +        {
>> +            ignore = false;
>> +            break;
>> +        }
>> +    }
>> +
>> +    if ( ignore )
>> +    {
>> +        dprintk(XENLOG_WARNING VTDPREFIX,
>> +                " Ignore SATC for seg %04x as no device under its scope is PCI discoverable!\n",

... this message is then issued only bogus entries were found. IOW
when a real device was found, there's no real reason to report N
other bogus ones, I think.

Plus, whatever we change here ought to also / first change in
register_one_rmrr().

> (I would drop the '!' at the end, here and above, I don't think they
> add much to the error message)

Sure; copy-and-paste again.

>  I would also use the '#' flag to avoid
> confusion, as in the weird case we have a segment '1234' then without
> the zero padding I wouldn't really know whether it's decimal or hex.

Not really, no. If there's any place we log segment numbers in
decimal, we should change that. They ought to be possible to match
with the usual ssss:bb:dd.f coordinates we log.

>> +    satcu = xzalloc(struct acpi_satc_unit);
>> +    if ( !satcu )
>> +        return -ENOMEM;
>> +
>> +    satcu->segment = satc->segment;
>> +    satcu->atc_required = satc->flags & 1;
> 
> I would add this as a define in actbl2.h:
> 
> #define ACPI_DMAR_ATC_REQUIRED (1U << 0)
> 
> Or some such (maybe just using plain 1 is also fine).

I intended to do so, but strictly staying in line with what Linux has.
To my surprise they use a literal number and have no #define. Hence I
didn't add any either.

>> +
>> +    dev_scope_start = (const void *)(satc + 1);
>> +    dev_scope_end   = (const void *)satc + header->length;
>> +    ret = acpi_parse_dev_scope(dev_scope_start, dev_scope_end,
>> +                               &satcu->scope, SATC_TYPE, satc->segment);
>> +
>> +    if ( !ret && satcu->scope.devices_cnt )
>> +    {
>> +        ret = register_one_satc(satcu);
>> +        /*
>> +         * register_one_satc() returns greater than 0 when a specified
>> +         * PCIe device cannot be detected. To prevent VT-d from being
>> +         * disabled in such cases, reset the return value to 0 here.
>> +         */
>> +        if ( ret > 0 )
>> +            ret = 0;
>> +    }
>> +    else
>> +        xfree(satcu);
> 
> You can safely use scope_devices_free() even if acpi_parse_dev_scope()
> failed, so you could unify the freeing here, instead of doing it in
> register_one_satc() also.

Moving that out of acpi_parse_dev_scope() would feel wrong - if a
function fails, it would better not leave cleanup to its caller(s).

> Also why not make register_one_satc() return a boolean instead of 0/1?

To leave room to also return errors, like register_one_rmrr() does.

For both of these aspects you raise: I'd really like to avoid these
sibling functions going too much out of sync.

>> @@ -817,6 +907,11 @@ static int __init cf_check acpi_parse_dm
>>                  printk(VTDPREFIX "found ACPI_DMAR_RHSA:\n");
>>              ret = acpi_parse_one_rhsa(entry_header);
>>              break;
>> +        case ACPI_DMAR_TYPE_SATC:
>> +            if ( iommu_verbose )
>> +                printk(VTDPREFIX "found ACPI_DMAR_SATC:\n");
>> +            ret = acpi_parse_one_satc(entry_header);
>> +            break;
> 
> I know the surrounding code doesn't use it, but readability would
> benefit from adding a blank line after the break statement.

Well, yes, done so. I'm not generally in favor of introducing such
inconsistencies, but it looks tolerable here. (In cases like this
I do - and did here - consider this as an option, but in most cases
I end up valuing uniform look slightly higher.)

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 15:30:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 15:30:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678277.1055380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6M9-0000o2-3e; Thu, 08 Feb 2024 15:30:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678277.1055380; Thu, 08 Feb 2024 15:30:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6M9-0000nv-10; Thu, 08 Feb 2024 15:30:29 +0000
Received: by outflank-mailman (input) for mailman id 678277;
 Thu, 08 Feb 2024 15:30:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY6M7-0000nV-3r
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 15:30:27 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fa9f5d4e-c696-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 16:30:24 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-4104dee26bfso2550195e9.3
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 07:30:24 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bv7-20020a0560001f0700b0033b525dbc70sm2625796wrb.79.2024.02.08.07.30.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 07:30:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa9f5d4e-c696-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707406224; x=1708011024; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3LL+IMK6+DyY0TYkPMCbzd0twwOfkoWuFWCqCoHRBMs=;
        b=T/mIcKvHpnjdszn2gzNtrnhweIPqGFhnb0A03fQvy+SOPdh6dK+jc3WbarE4/Poi+j
         4RC6255QzbT8tSxN0Gz1pRDfcMtvQpD4NHFoaid3IkRXSz61gLjNuVSJjzpfg7YY4+GF
         XECJa8Q9NP3q9B/psevdeTH8Ttepeyjde/XJC9rgD56NOFV43+H2yGICF9qn4n22Icmn
         lAD/S+VYLcsMdex5av04DvvM3YyH4R0oI6T+r1Q6+WN6tT20cjdACwYB5T7XViLNvB9u
         JstIk7toKxRDsH9jpjpmtWQ+UMUDVKIqcFzughivtaho0Tsi2kpTrV7MjbXRaSEZ7KlV
         XMgw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707406224; x=1708011024;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3LL+IMK6+DyY0TYkPMCbzd0twwOfkoWuFWCqCoHRBMs=;
        b=iFZbmWUkIb3kwlmyjwYWZP4Iu3ruCSJZX0UFMsaw+GiVJBlDGq6qDdqtYmg6f37qwk
         f03LXHPVgmmOBkWpwMN/a3HoV0Jpx462l2UY9oLB8hnHC8S6UFEoGGDVZ+nlxH9llONL
         O3tLrM1e9DMIiKRG7C/qG8EImFi7HKkwUNOZJx2zVwDcbR+cbKNQ2+3A0f1BRen6izKb
         bc/hmUPJqb4dSobGDl4PpkRM74EcVM6UOb6hsl7rgnzoE+UAy8GKgRcl9bKTX8EtSaP7
         dM+XH+2goxJikxLLShmvkTWpjWXSjTu7RVBVnKw0IurORNI7M8ASZOVdFhbHqfnL/VN6
         k5rw==
X-Forwarded-Encrypted: i=1; AJvYcCVWU9xlIBlL7kDVXYdlBpr/TwbSabKWwrAIhGT/sdWwWTcyVzft0CpIv0YSPWwi1T/EvgZraEknjfBXbtHe+79MX/pdve/CcZWIjoyaWDY=
X-Gm-Message-State: AOJu0YyXLTG+t2Gxk0QjR2+KMGTk5XGlpGUo5KScK7EhXTzYdYjTNNXm
	C9rCpxnwBB3EdBG35Yp3Pq8Vgzz3CKkXtgwkRpArWt4seMZ+wnGpBzTdrD73uw==
X-Google-Smtp-Source: AGHT+IE0Dl/N99PnPibWJIickBNqMnZ0Fjq4u43fhDJ8c3bBfixc1L3kWFxq9qTE3ag5aV+2n6lMAA==
X-Received: by 2002:a5d:49c8:0:b0:33b:226c:ecea with SMTP id t8-20020a5d49c8000000b0033b226ceceamr6165279wrs.32.1707406224429;
        Thu, 08 Feb 2024 07:30:24 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX1iAALmx159uWm5Zw1hdhg2C5teSCMYiU0L0KLxrnGXdFTa9xu7HE7T9YUc7FY/0czozCOGafzrmmZmyC/lwWbDwONXMrSUZ0Obr/bCOhIaaLIubxj9enEp/xm6BWO86j0qs8Vefeq9byystT+761jswg3ivqTFVnWR1u0wRfuf3coyUDYMiB4nZ0ywHIFU7S9dROtKSVjiISeQSc=
Message-ID: <1a0368d3-9562-4225-b2d2-26dc13d741e2@suse.com>
Date: Thu, 8 Feb 2024 16:30:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] config: update Mini-OS commit
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240208142852.3819-1-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240208142852.3819-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.02.2024 15:28, Juergen Gross wrote:
> Update the Mini-OS upstream revision.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Thu Feb 08 15:46:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 15:46:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678290.1055390 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6bq-00030D-H9; Thu, 08 Feb 2024 15:46:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678290.1055390; Thu, 08 Feb 2024 15:46:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6bq-000306-EX; Thu, 08 Feb 2024 15:46:42 +0000
Received: by outflank-mailman (input) for mailman id 678290;
 Thu, 08 Feb 2024 15:46:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/4LT=JR=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1rY6bp-000300-Mb
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 15:46:41 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2611::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3e85d4da-c699-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 16:46:39 +0100 (CET)
Received: from AS9P250CA0021.EURP250.PROD.OUTLOOK.COM (2603:10a6:20b:532::19)
 by PAVPR08MB8821.eurprd08.prod.outlook.com (2603:10a6:102:2fc::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.24; Thu, 8 Feb
 2024 15:46:35 +0000
Received: from AMS0EPF0000019E.eurprd05.prod.outlook.com
 (2603:10a6:20b:532:cafe::2a) by AS9P250CA0021.outlook.office365.com
 (2603:10a6:20b:532::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.41 via Frontend
 Transport; Thu, 8 Feb 2024 15:46:35 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AMS0EPF0000019E.mail.protection.outlook.com (10.167.16.250) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7249.19 via Frontend Transport; Thu, 8 Feb 2024 15:46:35 +0000
Received: ("Tessian outbound a064b9944658:v228");
 Thu, 08 Feb 2024 15:46:35 +0000
Received: from 5a5bff2f51c2.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 420708D1-174A-4415-965C-B9F6B18E3072.1; 
 Thu, 08 Feb 2024 15:46:28 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 5a5bff2f51c2.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 08 Feb 2024 15:46:28 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AS8PR08MB9955.eurprd08.prod.outlook.com (2603:10a6:20b:639::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.37; Thu, 8 Feb
 2024 15:46:27 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::99ee:f28d:84bc:97cb]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::99ee:f28d:84bc:97cb%7]) with mapi id 15.20.7270.024; Thu, 8 Feb 2024
 15:46:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3e85d4da-c699-11ee-8a4a-1f161083a0e0
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=T9LhVzujfGoS9ln9vKlaGwPJZeYtlEPkuFFuRJXAXURo+Zc/qPye/Ptm2MowMTFsRxfvHtkB1R1wAc9HRqblO0pl+4JO9NufGw6I/jgoKfDENU6vsZDSloSGcUmYiPXKEYjs/iFkHQb2gJJLQC/gHH0ARtZvcxScfmsbb0lzwplVLLw/loOrU169Ssvmfkamw+Na8tRPeZbCRYtb0XIr+zmV5SzAuj6/hrDxt7wr65R9KSU4IvIq8QK5fMHZmSVdAfR/bKAlf9kSPrj1XvzWnw2Bl5HxBq9IgDgeIPT7ldwVz01mD6Be0Law3B2i6dJZeI3scOKmqtZ4a/0TKuwXcA==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=lO6Y5pvQLyttuGrsUemOR6FlRAT4hv6Onm3HApcA1A0=;
 b=El9edqR9FmFZWz5wf2zUHPLrKUCqTRMGgbQJGzavVDR2l4sMa7nr/6oNOBup0Zimkq5u8BqvRlSkD0bQH5kPxD7ekWOmTQ3J3Wh2EZiGRs5I+0WPvOw+9f1ZzKCihvV7T65xLl697Zpg3TMaKWe3u7dvIJvsH/q8ZPu+gWfb3IBPTH3+eBhKXZmOP/zW9JxIcsS1VczvA5s1lkm1Hkw2bCg+0JqI44fUJdb+EIxqyb1/UVTSKgo52E98eUz4bx63XcxWkgWLCd+ngjyjdUJ6Xo1+mN2V3XJan68EfcmulIcszub6eNEJA0dOSlo7eGGjNbYK12TY6w3ZZ4jpTN90+w==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0
 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lO6Y5pvQLyttuGrsUemOR6FlRAT4hv6Onm3HApcA1A0=;
 b=YruQUpTIC4pXaEq7UtqKnqNE85F5JNrhTV34VkvRoxNf7sheiAgp33XW8hKdCpHvUG4QWMZ0T0UF/xAPRlQGSF4+2ZUklJMb1b9xZUj4UPgCEnB6fTOU0Lt9WDUjaoWF9/mINpnNb6AoxmqqLLUsp1W14gAak9LYGSn6rFS0tg4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 742a23b4b9cd856b
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MV0m1tItlbt30wn1YWPgzpwZMJo9tr2W3kpW45mBIoCB3p44aMoc3tliMhCiCIzR0tsHTMXfK+s4lPiQAJOBWBSDuEyilVm816IfCor9myj3ccryIlY3ZFZcibJUuMGBfHBs90ifF5W48U/REGKhw2VpfxzNoFEK937Yz0sJ9MxAFt/gwKxHntxSy5Fpp7iB/sVe/vyGt3RMYxpfRuHRGnv54XRxdTDokcHJKc/CbunOwzAjNfNOZ1WIvBeDB04JRpXeCoYCCDQs9pRFt4VLkNe2t0LQBiJJfrefenuGfBK+85wQ/Gkz+0KBzjXQYjFrnM+pryJ6+ALn4f1Up+tqzg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=lO6Y5pvQLyttuGrsUemOR6FlRAT4hv6Onm3HApcA1A0=;
 b=nNEVzUMzK6JZUReeQxpkeixeJYJgAhr/npz40xJH/zUGA9d/6ZCjP+aeO5yu4U7vzQCacYa8AYbbuu1rsa/vxfqK4eSs7ymNk1xBrBzcdXppfLTnUIRVSgLhb2MlGS+B/RIAsSqxxPLEs3/VN/RECOhMcPi72Yv3JzlWdy/w0SwuCyi9AgE3GBLitEGO0dcXWxfAmb57dctAzreGryJl0q32roh+JU1je21U212JK4cfU+JSRrjtDqTyk8SaklG4c8Bl/WBfPrh3yG4ezIeHgX8jPTB1FYesr6p5kjmbCf4oqJK9vG74+u3XgWnssfxoQCMv6LdTurPIcpvZoTKBgQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=lO6Y5pvQLyttuGrsUemOR6FlRAT4hv6Onm3HApcA1A0=;
 b=YruQUpTIC4pXaEq7UtqKnqNE85F5JNrhTV34VkvRoxNf7sheiAgp33XW8hKdCpHvUG4QWMZ0T0UF/xAPRlQGSF4+2ZUklJMb1b9xZUj4UPgCEnB6fTOU0Lt9WDUjaoWF9/mINpnNb6AoxmqqLLUsp1W14gAak9LYGSn6rFS0tg4=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH] xen/arm: Fix UBSAN failure in start_xen()
Thread-Topic: [PATCH] xen/arm: Fix UBSAN failure in start_xen()
Thread-Index: AQHaWnvkg1O7hpIgTUmeZhL+Ri6ivrEAlu4A
Date: Thu, 8 Feb 2024 15:46:27 +0000
Message-ID: <990CAEEA-B5CC-42B4-8350-71FCC8A16242@arm.com>
References: <20240208104339.37826-1-michal.orzel@amd.com>
In-Reply-To: <20240208104339.37826-1-michal.orzel@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AS8PR08MB9955:EE_|AMS0EPF0000019E:EE_|PAVPR08MB8821:EE_
X-MS-Office365-Filtering-Correlation-Id: 7044a7ec-43b5-4e21-b421-08dc28bd2168
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Ne3jK/RWDCqaX1NkngyfzDR8FzLDil8ogGZLSDPhXXYU+2OusW8L0su/ntJXuYsYzpU/VxiDE2v8PpSr4pE3npu49szNYndnDfEJbkbK+V3Uk2EFlyLsp9PuwkwAwVRalGkDmudtukljU1ZAQJQVdjFDNupD+/TuqOACLZc0sqKC+97ST6kN5PsHJaOdC0fZuls+/QGMz4yiW4c+OQQ2mceiDJktlLSYV2l8D2CJx4laqAEES9WrYN9TiqwmIZ7Xvuiz7nCBkTYUiC37Vc359cOYHeinw5vct/HKKklWxIGm4oNhy+s8g+dCfHr7NE3oK1EKm10NVDTzdttLGgFpJMC+zSyeim9/A6qsJG7uxLs6LgX09tAaDCbk/MlJcx6L10uNCUe3yIUHpW+hgFon/AsL71kidmL46RJM/zx/mI/fgmlq5pLcoMA4jC5VA8bHZ5etchk+44JnF4vM578q+9GP7818NgEgv1dbRAflyYSoOZWQxU+WRzJhxZxRobLUDnszjG4+uSuG5RA/e9CB4C7eFTE8QqJEUE8xSQimp+6yno4IvUaGg3BAQTNZwcbhzc9tyKYZAuZ466yIezSCVGULE2QEPJTFAI0b4/ElwmWPeeyb7E5DTMrq98eBaaxH
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366004)(39860400002)(396003)(136003)(346002)(376002)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(83380400001)(41300700001)(122000001)(66946007)(66476007)(6486002)(71200400001)(36756003)(76116006)(6506007)(64756008)(6916009)(53546011)(54906003)(478600001)(33656002)(66446008)(8936002)(8676002)(316002)(38070700009)(6512007)(4326008)(86362001)(66556008)(26005)(2616005)(38100700002)(2906002)(5660300002);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <39F8B1AB92BE2F4FB41AC3F3F8E7C9DB@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB9955
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AMS0EPF0000019E.eurprd05.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	57bc7306-2010-4734-01f1-08dc28bd1c83
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	z+lM3os1haec8ICiyTOhVPpfoWIDN7okxUq7/2eyrNNduNdWUKCfjLiRPsa73hyquAoHOyCaKdOdPIXaOsSvwblUy4FjlzOjRttMq7o5XvD913U6StBHkbjOfWGlQBS5+3jFTRtr96mJr1mzmlU2VwArszmltbYLiKesuBrorILnS2thmo/5e+oaic7X3vfFZe1LBX4824X6M5Wn3HvPjc6TB+7LB7CdMjYaKm2TrfZiDJGiNkbiIwgibCPH0e/T/i1o5Tvhi3EJY4MkcuPkqtqHqD7q6tNiQhsMP58UAIwS51zoTbohtj4lKJjNigrtLLSeSVaSs+pwIkIEjAaiNZHUcRLB4ME4bMVhl0Hhf2luw4A1DvU3BJnno6bsYVETkOPcZve9Wz4Ky7jUalPjj5cg1vtqdv0MdJfKnbRRdvawcJwQwNrQTwFe992EeesNq51+0bxu7rHUuZCkHbDml7TMTPZKzWCRAKD2q32aXGP23k0YtyA+sWUODy3X1tcSWs/X2n3v4beTLehzw1J3lNcPP27yy9ovudlB3P17JXlAslZxWuCEUEu/sMCRdEeWjRRJomliNVTfr/62FNVRvAyHnmRYICB3MmlhUtRYfX0=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(136003)(376002)(396003)(39860400002)(230922051799003)(64100799003)(82310400011)(1800799012)(186009)(451199024)(46966006)(40470700004)(36840700001)(2906002)(54906003)(5660300002)(316002)(6862004)(41300700001)(70586007)(70206006)(6512007)(6486002)(478600001)(6506007)(53546011)(8676002)(8936002)(4326008)(2616005)(107886003)(83380400001)(86362001)(82740400003)(36756003)(336012)(26005)(356005)(81166007)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2024 15:46:35.3582
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7044a7ec-43b5-4e21-b421-08dc28bd2168
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AMS0EPF0000019E.eurprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAVPR08MB8821

DQoNCj4gT24gOCBGZWIgMjAyNCwgYXQgMTA6NDMsIE1pY2hhbCBPcnplbCA8bWljaGFsLm9yemVs
QGFtZC5jb20+IHdyb3RlOg0KPiANCj4gV2hlbiBydW5uaW5nIFhlbiBvbiBhcm0zMiwgaW4gc2Nl
bmFyaW8gd2hlcmUgWGVuIGlzIGxvYWRlZCBhdCBhbiBhZGRyZXNzDQo+IHN1Y2ggYXMgYm9vdF9w
aHlzX29mZnNldCA+PSAyR0IsIFVCU0FOIHJlcG9ydHMgdGhlIGZvbGxvd2luZzoNCj4gDQo+IChY
RU4pIFVCU0FOOiBVbmRlZmluZWQgYmVoYXZpb3VyIGluIGFyY2gvYXJtL3NldHVwLmM6NzM5OjU4
DQo+IChYRU4pIHBvaW50ZXIgb3BlcmF0aW9uIHVuZGVyZmxvd2VkIDAwMjAwMDAwIHRvIDg2ODAw
MDAwDQo+IChYRU4pIFhlbiBXQVJOIGF0IGNvbW1vbi91YnNhbi91YnNhbi5jOjE3Mg0KPiAoWEVO
KSAtLS0tWyBYZW4tNC4xOS11bnN0YWJsZSAgYXJtMzIgIGRlYnVnPXkgdWJzYW49eSAgTm90IHRh
aW50ZWQgXS0tLS0NCj4gLi4uDQo+IChYRU4pIFhlbiBjYWxsIHRyYWNlOg0KPiAoWEVOKSAgICBb
PDAwMzFiNGMwPl0gdWJzYW4uYyN1YnNhbl9lcGlsb2d1ZSsweDE4LzB4ZjAgKFBDKQ0KPiAoWEVO
KSAgICBbPDAwMzFkMTM0Pl0gX191YnNhbl9oYW5kbGVfcG9pbnRlcl9vdmVyZmxvdysweGI4LzB4
ZDQgKExSKQ0KPiAoWEVOKSAgICBbPDAwMzFkMTM0Pl0gX191YnNhbl9oYW5kbGVfcG9pbnRlcl9v
dmVyZmxvdysweGI4LzB4ZDQNCj4gKFhFTikgICAgWzwwMDRkMTVhOD5dIHN0YXJ0X3hlbisweGUw
LzB4YmUwDQo+IChYRU4pICAgIFs8MDAyMDAwN2M+XSBoZWFkLm8jcHJpbWFyeV9zd2l0Y2hlZCsw
eDQvMHgzMA0KPiANCj4gVGhlIGZhaWx1cmUgaXMgcmVwb3J0ZWQgZm9yIHRoZSBmb2xsb3dpbmcg
bGluZToNCj4gKHBhZGRyX3QpKHVpbnRwdHJfdCkoX3N0YXJ0ICsgYm9vdF9waHlzX29mZnNldCkN
Cj4gDQo+IFRoaXMgb2NjdXJzIGJlY2F1c2UgdGhlIGNvbXBpbGVyIHRyZWF0cyAocHRyICsgc2l6
ZSkgd2l0aCBzaXplIGJpZ2dlciB0aGFuDQo+IFBUUkRJRkZfTUFYIGFzIHVuZGVmaW5lZCBiZWhh
dmlvci4gVG8gYWRkcmVzcyB0aGlzLCBzd2l0Y2ggdG8gbWFjcm8NCj4gdmlydF90b19tYWRkcigp
LCBnaXZlbiB0aGUgZnV0dXJlIHBsYW5zIHRvIGVsaW1pbmF0ZSBib290X3BoeXNfb2Zmc2V0Lg0K
PiANCj4gU2lnbmVkLW9mZi1ieTogTWljaGFsIE9yemVsIDxtaWNoYWwub3J6ZWxAYW1kLmNvbT4N
Cj4gLS0tDQoNCkhpIE1pY2hhbCwNCg0KSeKAmXZlIHRlc3RlZCB0aGlzIGNoYW5nZSB3aXRoIHFl
bXUgZm9yIGFybTMyIGFuZCBhcm02NCwgbG9va3MgZ29vZCB0byBtZToNCg0KUmV2aWV3ZWQtYnk6
IEx1Y2EgRmFuY2VsbHUgPGx1Y2EuZmFuY2VsbHVAYXJtLmNvbT4NClRlc3RlZC1ieTogTHVjYSBG
YW5jZWxsdSA8bHVjYS5mYW5jZWxsdUBhcm0uY29tPg0KDQoNCg==


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 15:49:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 15:49:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678294.1055399 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6et-00043q-VM; Thu, 08 Feb 2024 15:49:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678294.1055399; Thu, 08 Feb 2024 15:49:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6et-00043j-SF; Thu, 08 Feb 2024 15:49:51 +0000
Received: by outflank-mailman (input) for mailman id 678294;
 Thu, 08 Feb 2024 15:49:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY6es-00043d-NE
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 15:49:50 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b02319f9-c699-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 16:49:48 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-5114c05806eso3012233e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 07:49:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i16-20020a05600c355000b004103ffb8cfdsm1684167wmq.10.2024.02.08.07.49.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 07:49:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b02319f9-c699-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707407388; x=1708012188; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=9IfOXbySo5kMYwKP5SIe9JdNxvzj7Wb+z/dAnAkMBRk=;
        b=N/yZHE5NRmyjoD+FcQeibpFMvHSQE8TnSFO7tKe5UELfN0bjkzM5Xxy0BhhAUjZ5CN
         seKx4jb94GlCy5/qhv9d2cn126ZrBbcu+OaGDvC7HNOigDMk/iybMotPc6v8KkgldOi5
         bRW/FwKi5+7c02Vb66zBKtRybatrHvp3q8x+3VuN2WclLBJk6YVM1R8th/fExhgBuyI8
         bjDJYKsM6VelSpCXy6rZNoMOgtvGrSdpwveG+wVzJK6j6ZSrq6swq7I8HVHBgsmNpXXK
         hbQKxl8HIX1sv1eF3GHB3h05aL2/Ye3e9u3rWIco3aTBcbREdG/R4RQRESdPGMaMoLFV
         dKTA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707407388; x=1708012188;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9IfOXbySo5kMYwKP5SIe9JdNxvzj7Wb+z/dAnAkMBRk=;
        b=jI54GNGG8B6Qe0HJR/2ICLFmr7GgBJPTVqsrZA3wE9h518A6zIaU0viPZBX4evlbl5
         W3wfp3qSdPwSjukBVemTapG2TOqWNMhMrl8uUZYC1GG60X8GM3San2/JsdXSvyk+1r97
         kMoyBFTdG5VnT2hNfXNP1sTiR9KkOAP1DArfX80XPI17pdLr4i0dtgXDU7BsFddawfwC
         9YJO1cQXeTb2inQciKEnK7eWVXHLlGJBrwrXVK0QDaMBebSFZ5wKwxjKsP1XKMvVIaQN
         YSZEe+o0QEGOI6JQ/NgXvV/HPFjwpOrNJxkFOGT9sVsCbcgq1p8yR67EnAHriICCrfrz
         lXnA==
X-Gm-Message-State: AOJu0YzwxJtOCua6XG/ojdWttpE3lCUhMyL/I/HVgTPdZsCfXKylFDSz
	nQA1efBth1CqIm++/NJSNg3jgLdhXaptyx/sPQudiXjvPja248xBYgLlgdv3rQ==
X-Google-Smtp-Source: AGHT+IGkgUj99GW2jBlzP2XdvWHMwKlracBDRcumKQS8z0i1cltN0/+nZZiLulUFE8U5F1W+DzXu6Q==
X-Received: by 2002:ac2:43b7:0:b0:511:6fe4:efdd with SMTP id t23-20020ac243b7000000b005116fe4efddmr1294133lfl.5.1707407387752;
        Thu, 08 Feb 2024 07:49:47 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUpyOH/CbRFQhYXusBikQwsLshn/7WhUIACgnYx1yIXlD3Nw1VpEH0V5STBD5Y9QMyNxhcCDesS1DoCGnLVcZRQFOVNOL9LQdsQMQMsGLW7TV9vzdEP104thltYrZC2NCgQURSYE61Ri7VgJvsO
Message-ID: <4179ded9-36b8-4d47-997b-40d056e033af@suse.com>
Date: Thu, 8 Feb 2024 16:49:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/7] IOMMU: rename and re-type ats_enabled
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <467d24e1-8ed3-4dda-a334-70ff747bf94b@suse.com> <ZcS_uC2t96Lh720Y@macbook>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcS_uC2t96Lh720Y@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08.02.2024 12:49, Roger Pau Monné wrote:
> On Mon, Feb 05, 2024 at 02:55:43PM +0100, Jan Beulich wrote:
>> Make the variable a tristate, with (as done elsewhere) a negative value
>> meaning "default". Since all use sites need looking at, also rename it
>> to match our usual "opt_*" pattern. While touching it, also move it to
>> .data.ro_after_init.
>>
>> The only place it retains boolean nature is pci_ats_device(), for now.
> 
> Why does it retain the boolean nature in pci_ats_device()?
> 
> I assume this is to avoid having to touch the line again in a further
> patch, as given the current logic pci_ats_device() would also want to
> treat -1 as ATS disabled.

No, then I would need to touch the line. The function wants to treat
-1 as "maybe enabled", so the caller can know whether a device is an
ATS device regardless of whether ATS use is fully off, or only
"soft-off".

> I think this is all fine because you add additional opt_ats > 0 checks
> before the call to pci_ats_device(), but would be good to know this is
> the intention.

Note how amd_iommu_disable_domain_device() does not gain such a
check, for exactly the reason named above: The function would better
turn off ATS whenever enabled, no matter for what reason.

And of course - none of this "soft-off" vs "fully off" matters for
AMD (which is the only user of the function) right now anyway, seeing
they don't have an equivalent of the ATC_REQUIRED flag.

>> In AMD code re-order conditionals to have the config space accesses
>> after (cheaper) flag checks.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> In domain_context_mapping_one() I'm a little puzzled that translation
>> type is selected based on only IOMMU and global properties, i.e. not
>> taking the device itself into account.
> 
> That seems like a bug to me, we should check that the device supports
> ATS (and has it enabled) before setting the translation type to
> CONTEXT_TT_DEV_IOTLB unconditionally.  We should likely use
> ats_device() instead of ats_enabled in domain_context_mapping_one().

Will try to remember to add yet another patch then.

> There's also IMO a second bug here, which is that we possibly attempt
> to flush the device IOTLB before having ATS enabled.  We flush the
> device TLB in domain_context_mapping_one(), yet ATS is enabled by the
> caller afterwards (see domain_context_mapping()).

You may be right with this; I'd need to read up on whether such
flushing is permissible.

>> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
>> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
>> @@ -185,10 +185,11 @@ static int __must_check amd_iommu_setup_
>>          dte->ex = ivrs_dev->dte_allow_exclusion;
>>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
>>  
>> -        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
>> +        if ( opt_ats > 0 &&
>>               !ivrs_dev->block_ats &&
>> -             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
>> -            dte->i = ats_enabled;
>> +             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
>> +             pci_ats_device(iommu->seg, bus, pdev->devfn) )
>> +            dte->i = true;
>>  
>>          spin_unlock_irqrestore(&iommu->lock, flags);
>>  
>> @@ -248,10 +249,11 @@ static int __must_check amd_iommu_setup_
>>          ASSERT(dte->sys_mgt == MASK_EXTR(ivrs_dev->device_flags,
>>                                           ACPI_IVHD_SYSTEM_MGMT));
>>  
>> -        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
>> +        if ( opt_ats > 0 &&
>>               !ivrs_dev->block_ats &&
>> -             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
>> -            ASSERT(dte->i == ats_enabled);
>> +             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
>> +             pci_ats_device(iommu->seg, bus, pdev->devfn) )
>> +            ASSERT(dte->i);
>>  
>>          spin_unlock_irqrestore(&iommu->lock, flags);
>>  
>> @@ -268,9 +270,10 @@ static int __must_check amd_iommu_setup_
>>  
>>      ASSERT(pcidevs_locked());
>>  
>> -    if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
>> +    if ( opt_ats > 0 &&
>>           !ivrs_dev->block_ats &&
>>           iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
>> +         pci_ats_device(iommu->seg, bus, pdev->devfn) &&
>>           !pci_ats_enabled(iommu->seg, bus, pdev->devfn) )
> 
> Seeing that this same set of conditions is used in 3 different checks,
> could we add a wrapper for it?
> 
> opt_ats > 0 && !ivrs_dev->block_ats &&
> iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> pci_ats_device(iommu->seg, bus, pdev->devfn)
> 
> pci_device_ats_capable()? or some such.

I was pondering that, yes (iirc already once when adding block_ats).
Problem is the name. "capable" isn't quite right when considering
the tristate opt_ats. And pci_device_may_use_ats() reads, well,
clumsy to me. If you have any good idea for a name that's fully
applicable and not odd or overly long, I can certainly introduce
such a helper.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 15:50:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 15:50:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678296.1055409 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6fe-0005On-6M; Thu, 08 Feb 2024 15:50:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678296.1055409; Thu, 08 Feb 2024 15:50:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6fe-0005Og-3r; Thu, 08 Feb 2024 15:50:38 +0000
Received: by outflank-mailman (input) for mailman id 678296;
 Thu, 08 Feb 2024 15:50:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=z5WF=JR=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rY6fc-0005GL-L1
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 15:50:36 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cc67806a-c699-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 16:50:35 +0100 (CET)
Received: from nico.bugseng.com (unknown [147.123.100.131])
 by support.bugseng.com (Postfix) with ESMTPSA id D54794EE0739;
 Thu,  8 Feb 2024 16:50:33 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc67806a-c699-11ee-8a4a-1f161083a0e0
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	Nicola Vetrini <nicola.vetrini@bugseng.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH v2] docs/misra: add asm-offset.c to exclude-list
Date: Thu,  8 Feb 2024 16:50:09 +0100
Message-Id: <09b659e19bf2cc6b3ee4320e019bdfa7def5f3b8.1707406598.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

These files contain several deliberate violations of MISRA C rules and
they are not linked in the final Xen binary, therefore they can be exempted
from MISRA compliance.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
Changes in v2:
- use a glob to exclude this file for all architectures.
---
 docs/misra/exclude-list.json | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/docs/misra/exclude-list.json b/docs/misra/exclude-list.json
index 7971d0e70f5b..89df966eeac9 100644
--- a/docs/misra/exclude-list.json
+++ b/docs/misra/exclude-list.json
@@ -101,6 +101,10 @@
             "rel_path": "arch/x86/efi/check.c",
             "comment": "The resulting code is not included in the final Xen binary, ignore for now"
         },
+        {
+          "rel_path": "arch/*/*/asm-offsets.c",
+          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
+        },
         {
             "rel_path": "common/coverage/*",
             "comment": "Files to support gcov, ignore for now"
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 15:56:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 15:56:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678303.1055420 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6lQ-000635-Q9; Thu, 08 Feb 2024 15:56:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678303.1055420; Thu, 08 Feb 2024 15:56:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6lQ-00062y-NY; Thu, 08 Feb 2024 15:56:36 +0000
Received: by outflank-mailman (input) for mailman id 678303;
 Thu, 08 Feb 2024 15:56:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY6lP-00062h-RW
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 15:56:35 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a1f4126f-c69a-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 16:56:34 +0100 (CET)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-33b436dbdcfso1442564f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 07:56:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 dr9-20020a5d5f89000000b0033b14f22180sm3940318wrb.20.2024.02.08.07.56.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 07:56:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1f4126f-c69a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707407793; x=1708012593; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=WoKTmhnl937M83ulqEmDhr8UT5Fq+44qVy8o8fulbP4=;
        b=AsAx/XkgwiBFqltYPM30/uLojylOdMAfWXucRLih5Tt544PNXXfZXO95/dAzAM2o2n
         YaxJlXY/cKdd3/ZgaSj9OuPZX1DNdtZG96eASwll7elNyHdQQyaYWMYkzG/25iCkePdL
         G19urwlh9sYun63U3E0kauhC4ODp0TOa0P124fythvuZ+ZICWYrSS9c9fNq1lCShelEY
         MNkh7aTcRZ9VoK0aL3Mbf8T7f66Yr+SHpRqVGpxBbD3SlFaXqDMDC058lB45NyIm136N
         RomyAOFpMI+lakuA2II7QyFPDoAkcmxVWCq0xGthUn4AlAgPID099k44fpv0u/DyF5L6
         YmNg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707407793; x=1708012593;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=WoKTmhnl937M83ulqEmDhr8UT5Fq+44qVy8o8fulbP4=;
        b=EFJUxYxltTdmUcvEgjrQaZkShd+SKbf7wyZ5SYZN9lQQoXiclSKb9jvbN+O50UlQ7j
         9tuMtFRFB3PCXvfFGM3KE4JruwZqjAZjmf8EFDJLuvwM11B9g9vzHL2avf0zRKw8NkmR
         pkvNXNpAAtu/jWIv3oK/OfyzHqdXXogm48Gn+h+J3R5oge7Y3mwk83LZn9Ex7uN5961n
         Gakh7SWHbdCtj5IBninC1VggmsIbNruwrYg6SUHDi3mcs0uQr/wnvrJ/xzW1mNQxRURt
         TbkSYykhV48vMCbCpV02i50laHZsszsEH7woNmQ2BDWGtTQOX1Psf33ijyC0R90DPWDh
         2ahg==
X-Forwarded-Encrypted: i=1; AJvYcCWgG75P9aDypPCwelVK+U+sd+IFlFCp5Id2dd2pdJpmtFNCl2Sn/1jsXfD6eStGf1mP+KIFfLC2WKTqa7f6QgHZ7fz5u04gorBus3HBy/4=
X-Gm-Message-State: AOJu0Ywv4ktaZyCqdH85BB5++jmOPYON6Ir6tcusF42E0cjcw+f4D0Tm
	LpnNW6KhI+pzsshUBn6YQmL5Ry+8ekr9zbTdc3m/WCnBNDJDjMPf86RPrPqKkg==
X-Google-Smtp-Source: AGHT+IEXxy4E4Mcg6IM9g0DANqMlEQWN12/RB5CcSE4m5hpDG2PMk8mUyGy98G4iEfWFIjzrHRrkKw==
X-Received: by 2002:a5d:42c5:0:b0:33b:1a39:e501 with SMTP id t5-20020a5d42c5000000b0033b1a39e501mr5494134wrr.28.1707407793541;
        Thu, 08 Feb 2024 07:56:33 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVPmvhsxXkUehwKER38yQsypxjca1lfVSAd0MGyfsN6IHTwHc71D/q6lz6jqcx35v2jXr13sA3N0R7dq2OTrjIdZRCL93HhxfmzsjAgGIYHyxHZVdnJWhtqyyX6XitTq3SCMbFnqOzGaBtNPoQTdS2BQ5nIA2OcNG7EHfmp+0kraQ3xLRjGk7r6h0DIsSw2ouLvcWLWkA1P5x8wRb8aK9cciNMGeDIdZ5qnVSrJjEOC3oiBY3hdC8rMMj7iUDqawteOJZbcP7wCtMopYScvWn+FSu83RCeMEq2kd8Nnz2UpYBMAwSFwMK2J7WqLPq8VprFADdkX2mdSfzuQTvYsL932KmvCOJQdQ0wOwd123xYsM0+HJPouQAIdtYdv1xlz1ZsIHSh3nlAGFujNhfVDU6h2TGI=
Message-ID: <891e8cc2-e073-4cd3-9998-4c01ae775a3b@suse.com>
Date: Thu, 8 Feb 2024 16:56:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2] docs/misra: add asm-offset.c to exclude-list
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <09b659e19bf2cc6b3ee4320e019bdfa7def5f3b8.1707406598.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <09b659e19bf2cc6b3ee4320e019bdfa7def5f3b8.1707406598.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.02.2024 16:50, Nicola Vetrini wrote:
> These files contain several deliberate violations of MISRA C rules and
> they are not linked in the final Xen binary, therefore they can be exempted
> from MISRA compliance.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
albeit ...

> --- a/docs/misra/exclude-list.json
> +++ b/docs/misra/exclude-list.json
> @@ -101,6 +101,10 @@
>              "rel_path": "arch/x86/efi/check.c",
>              "comment": "The resulting code is not included in the final Xen binary, ignore for now"
>          },
> +        {
> +          "rel_path": "arch/*/*/asm-offsets.c",
> +          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
> +        },
>          {
>              "rel_path": "common/coverage/*",
>              "comment": "Files to support gcov, ignore for now"

... something looks odd with indentation; can probably be adjusted
while committing.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 15:59:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 15:59:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678309.1055430 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6o5-0006zj-CY; Thu, 08 Feb 2024 15:59:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678309.1055430; Thu, 08 Feb 2024 15:59:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6o5-0006zc-9h; Thu, 08 Feb 2024 15:59:21 +0000
Received: by outflank-mailman (input) for mailman id 678309;
 Thu, 08 Feb 2024 15:59:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY6o3-0006zU-MT
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 15:59:19 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 041bd668-c69b-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 16:59:18 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-51168addef1so2897156e87.1
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 07:59:18 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 jl26-20020a05600c6a9a00b0040fff3382casm1965797wmb.46.2024.02.08.07.59.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 07:59:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 041bd668-c69b-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707407958; x=1708012758; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=opkDLlYlwAcgajlR7y6ntiRvdsqEsQAXt5P7B8Dgv+g=;
        b=c+3n51K/JUk79gdtph/xFjCCtBD19yk8lw4IH5sF83REetMd727xI7bKeySEj7/RXI
         7RJMJTaOQsieXWDi/5yLQTxvhl0EJxIxSoSbX58QkcFlvdAMs28Wgtob+FUrDGNsX7dx
         QbsipxS3fcfYgSCK3i9XZPwdPYU/Mn5pCq89DVvTxzkDajbJ05NxCO0X+QC/1r/aMHpu
         f1xXyTzoN1a0GjXuYD7HPn4k1MtLfNd/IR1DQL14uXFlWGVdBnKXh+3KB9Yka0SVAGMx
         3R+tZrsNqsx9n16opH0uNkdkGwT3yFJVXVGknlU7vSYTfQgb6RDp5+gYED+Vg7Z6nycg
         BaIQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707407958; x=1708012758;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=opkDLlYlwAcgajlR7y6ntiRvdsqEsQAXt5P7B8Dgv+g=;
        b=v9TbtIdZ/C0vUo4WztQiKHOFRHMdGpjuDmeRhlrB4FOIToLuT5MPrJZTwUxVsk30sm
         LvimEp2dqcktPZyGMQKoA6GIcElSCXv3hgbc6suMCWIaTjA/jWtIDvMAI9mahNIyCKty
         C5ZsUHuwL6GIq/iIkEO6uItPfOHufBt5iBRkxc6Ik/SoUCjIhMt5QtSn4gqsfkelHpM8
         EC6zqn7EEapebdyncPcAFZA3DzcRsHjhqwEfcOfMcw5q0iOuSkoVWvn79CvqYlQqR2o9
         NlNvumBIetK5U/rdc7hvdU3ue+qFfcpuU7naqwYLVeHuctNrowJPPWGUiNd6DKJtRQ7S
         p9OA==
X-Gm-Message-State: AOJu0Yzpw84zRHeHpFxLR0qVqpQbm1+D5NAsOYRQIUajQzFmAx3f+kY3
	40/wWBg52qu9iUSGTgk58rz/0s8m11UVXqNq3InDdl+TSH1k3KAR5VtpU72xE4rCF1Ip+ajjaQ0
	=
X-Google-Smtp-Source: AGHT+IFDCeuGS01OPrOQ2jug1IHE7nVncAtND3DA5AgLA/umbntR63ri6VA+u11tzQJih9c+39Rz+A==
X-Received: by 2002:a05:6512:23a3:b0:511:5322:345b with SMTP id c35-20020a05651223a300b005115322345bmr7321843lfv.7.1707407958189;
        Thu, 08 Feb 2024 07:59:18 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVqAk52yeXQyinP+0NOLH6e/ts7RO36WE2W/hIN/7MljLN1qdkDaP0FSrYnZnRv0zuYhJtXBLd1BOH5Ww1NjB5uAP5ZDBzdjxNAt5o32Wa+cUm6jGDGvGtoDCTJyFQ3PB1oRqCi6D7Uein9QU1oiL6NFyelKaI=
Message-ID: <c0a2b6fe-e64f-433a-accf-124a7cee3f91@suse.com>
Date: Thu, 8 Feb 2024 16:59:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/HVM: tidy state on hvmemul_map_linear_addr()'s error
 path
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Manuel Andreas <manuel.andreas@tum.de>, Paul Durrant <paul@xen.org>
References: <c4c1d2b3-591e-403f-879b-bbb897f7ff25@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <c4c1d2b3-591e-403f-879b-bbb897f7ff25@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 13:06, Jan Beulich wrote:
> While in the vast majority of cases failure of the function will not
> be followed by re-invocation with the same emulation context, a few
> very specific insns - involving multiple independent writes, e.g. ENTER
> and PUSHA - exist where this can happen. Since failure of the function
> only signals to the caller that it ought to try an MMIO write instead,
> such failure also cannot be assumed to result in wholesale failure of
> emulation of the current insn. Instead we have to maintain internal
> state such that another invocation of the function with the same
> emulation context remains possible. To achieve that we need to reset MFN
> slots after putting page references on the error path.
> 
> Note that all of this affects debugging code only, in causing an
> assertion to trigger (higher up in the function). There's otherwise no
> misbehavior - such a "leftover" slot would simply be overwritten by new
> contents in a release build.
> 
> Also extend the related unmap() assertion, to further check for MFN 0.
> 
> Fixes: 8cbd4fb0b7ea ("x86/hvm: implement hvmemul_write() using real mappings")
> Reported.by: Manuel Andreas <manuel.andreas@tum.de>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Just noticed that I forgot to Cc Paul.

Jan

> ---
> While probably I could be convinced to omit the #ifndef, I'm really
> considering to extend the one in hvmemul_unmap_linear_addr(), to
> eliminate the zapping from release builds: Leaving MFN 0 in place is not
> much better than leaving a (presently) guest-owned one there. And we
> can't really put/leave INVALID_MFN there, as that would conflict with
> other debug checking.
> 
> --- a/xen/arch/x86/hvm/emulate.c
> +++ b/xen/arch/x86/hvm/emulate.c
> @@ -696,7 +696,12 @@ static void *hvmemul_map_linear_addr(
>   out:
>      /* Drop all held references. */
>      while ( mfn-- > hvmemul_ctxt->mfn )
> +    {
>          put_page(mfn_to_page(*mfn));
> +#ifndef NDEBUG /* Clean slot for a subsequent map()'s error checking. */
> +        *mfn = _mfn(0);
> +#endif
> +    }
>  
>      return err;
>  }
> @@ -718,7 +723,7 @@ static void hvmemul_unmap_linear_addr(
>  
>      for ( i = 0; i < nr_frames; i++ )
>      {
> -        ASSERT(mfn_valid(*mfn));
> +        ASSERT(mfn_x(*mfn) && mfn_valid(*mfn));
>          paging_mark_dirty(currd, *mfn);
>          put_page(mfn_to_page(*mfn));
>  



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:05:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:05:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678313.1055439 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6tv-0000rT-0V; Thu, 08 Feb 2024 16:05:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678313.1055439; Thu, 08 Feb 2024 16:05:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6tu-0000rM-U0; Thu, 08 Feb 2024 16:05:22 +0000
Received: by outflank-mailman (input) for mailman id 678313;
 Thu, 08 Feb 2024 16:05:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY6tu-0000rG-1p
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:05:22 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id db03be06-c69b-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:05:19 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 733601FCF4;
 Thu,  8 Feb 2024 16:05:18 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 305931326D;
 Thu,  8 Feb 2024 16:05:18 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id id0+Cr77xGWzAwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:05:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: db03be06-c69b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707408318; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=GzoKAVtAHMHyYxKj3Jy39XsROB3PeexKhIb/Jv9U/Co=;
	b=isd/087gteg1utA3oqlCzukeuQxrMvA3AFLMem8sffCqZIQz84oWqOEqQaxs+ybM68XRK3
	k9QgBTgP2WwBVdjCWxuC/FUzDo7AZevlFGyrlZ4hC2vz3wmllKgalTpTG4H+Rgw3lgoOCM
	Q7MJEt0O11jwgSa+DQw63lbskXbX56c=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707408318; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=GzoKAVtAHMHyYxKj3Jy39XsROB3PeexKhIb/Jv9U/Co=;
	b=isd/087gteg1utA3oqlCzukeuQxrMvA3AFLMem8sffCqZIQz84oWqOEqQaxs+ybM68XRK3
	k9QgBTgP2WwBVdjCWxuC/FUzDo7AZevlFGyrlZ4hC2vz3wmllKgalTpTG4H+Rgw3lgoOCM
	Q7MJEt0O11jwgSa+DQw63lbskXbX56c=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] docs: add a best practices coding guide
Date: Thu,  8 Feb 2024 17:05:15 +0100
Message-Id: <20240208160515.9949-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b="isd/087g"
X-Spamd-Result: default: False [-3.81 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 RCVD_DKIM_ARC_DNSWL_HI(-1.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_IN_DNSWL_HI(-0.50)[2a07:de40:b281:104:10:150:64:97:from];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 733601FCF4
X-Spam-Level: 
X-Spam-Score: -3.81
X-Spam-Flag: NO

Today the CODING_STYLE contains a section "Handling unexpected
conditions" specific to the hypervisor. This section is kind of
misplaced for a coding style. It should rather be part of a "Coding
best practices" guide.

Add such a guide as docs/process/coding-best-practices.pandoc and
move the mentioned section from CODING_STYLE to the new file, while
converting the format to pandoc.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 CODING_STYLE                              | 102 ----------------------
 docs/process/coding-best-practices.pandoc |  92 +++++++++++++++++++
 2 files changed, 92 insertions(+), 102 deletions(-)
 create mode 100644 docs/process/coding-best-practices.pandoc

diff --git a/CODING_STYLE b/CODING_STYLE
index ed13ee2b66..7f6e9ad065 100644
--- a/CODING_STYLE
+++ b/CODING_STYLE
@@ -167,105 +167,3 @@ the end of files.  It should be:
  * indent-tabs-mode: nil
  * End:
  */
-
-Handling unexpected conditions
-------------------------------
-
-GUIDELINES:
-
-Passing errors up the stack should be used when the caller is already
-expecting to handle errors, and the state when the error was
-discovered isn’t broken, or isn't too hard to fix.
-
-domain_crash() should be used when passing errors up the stack is too
-difficult, and/or when fixing up state of a guest is impractical, but
-where fixing up the state of Xen will allow Xen to continue running.
-This is particularly appropriate when the guest is exhibiting behavior
-well-behaved guests shouldn't.
-
-BUG_ON() should be used when you can’t pass errors up the stack, and
-either continuing or crashing the guest would likely cause an
-information leak or privilege escalation vulnerability.
-
-ASSERT() IS NOT AN ERROR HANDLING MECHANISM.  ASSERT is a way to move
-detection of a bug earlier in the programming cycle; it is a
-more-noticeable printk.  It should only be added after one of the
-other three error-handling mechanisms has been evaluated for
-reliability and security.
-
-RATIONALE:
-
-It's frequently the case that code is written with the assumption that
-certain conditions can never happen.  There are several possible
-actions programmers can take in these situations:
-
-* Programmers can simply not handle those cases in any way, other than
-perhaps to write a comment documenting what the assumption is.
-
-* Programmers can try to handle the case gracefully -- fixing up
-in-progress state and returning an error to the user.
-
-* Programmers can crash the guest.
-
-* Programmers can use ASSERT(), which will cause the check to be
-executed in DEBUG builds, and cause the hypervisor to crash if it's
-violated
-
-* Programmers can use BUG_ON(), which will cause the check to be
-executed in both DEBUG and non-DEBUG builds, and cause the hypervisor
-to crash if it's violated.
-
-In selecting which response to use, we want to achieve several goals:
-
-- To minimize risk of introducing security vulnerabilities,
-  particularly as the code evolves over time
-
-- To efficiently spend programmer time
-
-- To detect violations of assumptions as early as possible
-
-- To minimize the impact of bugs on production use cases
-
-The guidelines above attempt to balance these:
-
-- When the caller is expecting to handle errors, and there is no
-broken state at the time the unexpected condition is discovered, or
-when fixing the state is straightforward, then fixing up the state and
-returning an error is the most robust thing to do.  However, if the
-caller isn't expecting to handle errors, or if the state is difficult
-to fix, then returning an error may require extensive refactoring,
-which is not a good use of programmer time when they're certain that
-this condition cannot occur.
-
-- BUG_ON() will stop all hypervisor action immediately.  In situations
-where continuing might allow an attacker to escalate privilege, a
-BUG_ON() can change a privilege escalation or information leak into a
-denial-of-service (an improvement).  But in situations where
-continuing (say, returning an error) might be safe, then BUG_ON() can
-change a benign failure into denial-of-service (a degradation).
-
-- domain_crash() is similar to BUG_ON(), but with a more limited
-effect: it stops that domain immediately.  In situations where
-continuing might cause guest or hypervisor corruption, but destroying
-the guest allows the hypervisor to continue, this can change a more
-serious bug into a guest denial-of-service.  But in situations where
-returning an error might be safe, then domain_crash() can change a
-benign failure into a guest denial-of-service.
-
-- ASSERT() will stop the hypervisor during development, but allow
-hypervisor action to continue during production.  In situations where
-continuing will at worst result in a denial-of-service, and at best
-may have little effect other than perhaps quirky behavior, using an
-ASSERT() will allow violation of assumptions to be detected as soon as
-possible, while not causing undue degradation in production
-hypervisors.  However, in situations where continuing could cause
-privilege escalation or information leaks, using an ASSERT() can
-introduce security vulnerabilities.
-
-Note however that domain_crash() has its own traps: callers far up the
-call stack may not realize that the domain is now dying as a result of
-an innocuous-looking operation, particularly if somewhere on the
-callstack between the initial function call and the failure, no error
-is returned.  Using domain_crash() requires careful inspection and
-documentation of the code to make sure all callers at the stack handle
-a newly-dead domain gracefully.
diff --git a/docs/process/coding-best-practices.pandoc b/docs/process/coding-best-practices.pandoc
new file mode 100644
index 0000000000..f611aa9a55
--- /dev/null
+++ b/docs/process/coding-best-practices.pandoc
@@ -0,0 +1,92 @@
+# Best Practices in the Hypervisor
+
+## Handling unexpected conditions
+
+### Guidelines
+
+Passing errors up the stack should be used when the caller is already
+expecting to handle errors, and the state when the error was
+discovered isn’t broken, or isn't too hard to fix.
+
+domain_crash() should be used when passing errors up the stack is too
+difficult, and/or when fixing up state of a guest is impractical, but
+where fixing up the state of Xen will allow Xen to continue running.
+This is particularly appropriate when the guest is exhibiting behavior
+well-behaved guests shouldn't.
+
+BUG_ON() should be used when you can’t pass errors up the stack, and
+either continuing or crashing the guest would likely cause an
+information leak or privilege escalation vulnerability.
+
+ASSERT() IS NOT AN ERROR HANDLING MECHANISM.  ASSERT is a way to move
+detection of a bug earlier in the programming cycle; it is a
+more-noticeable printk.  It should only be added after one of the
+other three error-handling mechanisms has been evaluated for
+reliability and security.
+
+### Rationale
+
+It's frequently the case that code is written with the assumption that
+certain conditions can never happen.  There are several possible
+actions programmers can take in these situations:
+
+ * Programmers can simply not handle those cases in any way, other than
+   perhaps to write a comment documenting what the assumption is.
+ * Programmers can try to handle the case gracefully -- fixing up
+   in-progress state and returning an error to the user.
+ * Programmers can crash the guest.
+ * Programmers can use ASSERT(), which will cause the check to be
+   executed in DEBUG builds, and cause the hypervisor to crash if it's
+   violated
+ * Programmers can use BUG_ON(), which will cause the check to be
+   executed in both DEBUG and non-DEBUG builds, and cause the hypervisor
+   to crash if it's violated.
+
+In selecting which response to use, we want to achieve several goals:
+
+ * To minimize risk of introducing security vulnerabilities,
+   particularly as the code evolves over time
+ * To efficiently spend programmer time
+ * To detect violations of assumptions as early as possible
+ * To minimize the impact of bugs on production use cases
+
+The guidelines above attempt to balance these:
+
+ * When the caller is expecting to handle errors, and there is no
+   broken state at the time the unexpected condition is discovered, or
+   when fixing the state is straightforward, then fixing up the state and
+   returning an error is the most robust thing to do.  However, if the
+   caller isn't expecting to handle errors, or if the state is difficult
+   to fix, then returning an error may require extensive refactoring,
+   which is not a good use of programmer time when they're certain that
+   this condition cannot occur.
+ * BUG_ON() will stop all hypervisor action immediately.  In situations
+   where continuing might allow an attacker to escalate privilege, a
+   BUG_ON() can change a privilege escalation or information leak into a
+   denial-of-service (an improvement).  But in situations where
+   continuing (say, returning an error) might be safe, then BUG_ON() can
+   change a benign failure into denial-of-service (a degradation).
+ * domain_crash() is similar to BUG_ON(), but with a more limited
+   effect: it stops that domain immediately.  In situations where
+   continuing might cause guest or hypervisor corruption, but destroying
+   the guest allows the hypervisor to continue, this can change a more
+   serious bug into a guest denial-of-service.  But in situations where
+   returning an error might be safe, then domain_crash() can change a
+   benign failure into a guest denial-of-service.
+ * ASSERT() will stop the hypervisor during development, but allow
+   hypervisor action to continue during production.  In situations where
+   continuing will at worst result in a denial-of-service, and at best
+   may have little effect other than perhaps quirky behavior, using an
+   ASSERT() will allow violation of assumptions to be detected as soon as
+   possible, while not causing undue degradation in production
+   hypervisors.  However, in situations where continuing could cause
+   privilege escalation or information leaks, using an ASSERT() can
+   introduce security vulnerabilities.
+
+Note however that domain_crash() has its own traps: callers far up the
+call stack may not realize that the domain is now dying as a result of
+an innocuous-looking operation, particularly if somewhere on the
+callstack between the initial function call and the failure, no error
+is returned.  Using domain_crash() requires careful inspection and
+documentation of the code to make sure all callers at the stack handle
+a newly-dead domain gracefully.
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:09:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:09:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678319.1055450 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6y5-0001vr-He; Thu, 08 Feb 2024 16:09:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678319.1055450; Thu, 08 Feb 2024 16:09:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6y5-0001vk-EY; Thu, 08 Feb 2024 16:09:41 +0000
Received: by outflank-mailman (input) for mailman id 678319;
 Thu, 08 Feb 2024 16:09:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rY6y4-0001vc-JA
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:09:40 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rY6y4-00022Q-4V; Thu, 08 Feb 2024 16:09:40 +0000
Received: from [15.248.3.89] (helo=[10.24.67.25])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rY6y3-0008QW-Uc; Thu, 08 Feb 2024 16:09:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=XcSRXHG2iHvCB7O/0OiQDuFBEQj38vp3DT4FIDVeE08=; b=cYl0bq7sJt1by007uENchK5p2T
	wtbH9cNvKkHtiGhvobq81Qf7jWwGVMsHB423VBYHXdIZ2NZsyn/75cBGhQQw7221uoiq73+vHCTOY
	iGrlMOaubZAGA8vEsiiyNx/6vZU6jx2utBRk6cA9WtjLvozGUm/C6H9AXCgOJp/n058M=;
Message-ID: <28dff183-bc8a-4c59-bd4a-75abba8858b8@xen.org>
Date: Thu, 8 Feb 2024 16:09:37 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Fix UBSAN failure in start_xen()
Content-Language: en-GB
To: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240208104339.37826-1-michal.orzel@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240208104339.37826-1-michal.orzel@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Michal,

On 08/02/2024 10:43, Michal Orzel wrote:
> When running Xen on arm32, in scenario where Xen is loaded at an address
> such as boot_phys_offset >= 2GB, UBSAN reports the following:
> 
> (XEN) UBSAN: Undefined behaviour in arch/arm/setup.c:739:58
> (XEN) pointer operation underflowed 00200000 to 86800000
> (XEN) Xen WARN at common/ubsan/ubsan.c:172
> (XEN) ----[ Xen-4.19-unstable  arm32  debug=y ubsan=y  Not tainted ]----
> ...
> (XEN) Xen call trace:
> (XEN)    [<0031b4c0>] ubsan.c#ubsan_epilogue+0x18/0xf0 (PC)
> (XEN)    [<0031d134>] __ubsan_handle_pointer_overflow+0xb8/0xd4 (LR)
> (XEN)    [<0031d134>] __ubsan_handle_pointer_overflow+0xb8/0xd4
> (XEN)    [<004d15a8>] start_xen+0xe0/0xbe0
> (XEN)    [<0020007c>] head.o#primary_switched+0x4/0x30
> 
> The failure is reported for the following line:
> (paddr_t)(uintptr_t)(_start + boot_phys_offset)
> 
> This occurs because the compiler treats (ptr + size) with size bigger than
> PTRDIFF_MAX as undefined behavior. To address this, switch to macro
> virt_to_maddr(), given the future plans to eliminate boot_phys_offset.
> 
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> ---
>   xen/arch/arm/setup.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 7e28f62d09f1..424744ad5e1a 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -736,7 +736,7 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset,
>   
>       /* Register Xen's load address as a boot module. */
>       xen_bootmodule = add_boot_module(BOOTMOD_XEN,
> -                             (paddr_t)(uintptr_t)(_start + boot_phys_offset),
> +                             virt_to_maddr(_start),

The two lines can now be merged. So I have done it while committing.

Also, Stefano, I think this wants to be backported.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:11:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:11:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678321.1055459 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6zf-0003Q7-Ry; Thu, 08 Feb 2024 16:11:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678321.1055459; Thu, 08 Feb 2024 16:11:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY6zf-0003Q0-PQ; Thu, 08 Feb 2024 16:11:19 +0000
Received: by outflank-mailman (input) for mailman id 678321;
 Thu, 08 Feb 2024 16:11:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=N909=JR=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rY6ze-0003Ps-Ly
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:11:18 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id afefc489-c69c-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:11:16 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-410475afeddso67095e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 08:11:16 -0800 (PST)
Received: from [192.168.10.199] (54-240-197-231.amazon.com. [54.240.197.231])
 by smtp.gmail.com with ESMTPSA id
 l9-20020a05600c16c900b004103400259fsm1967203wmn.29.2024.02.08.08.11.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 08:11:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: afefc489-c69c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707408676; x=1708013476; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:references:cc:to
         :content-language:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=+az7xZn/dvncgU4RIhCZjis7qc7EUKrlPMvS+7lsMzY=;
        b=Dixpg7eHh7A6Uljthuk6+lrV7gldaTUOyW+2HQYyGMOMIuKOAyFUoovHNJti3K0jt8
         dZkUh8svQdq+dvThDG5F++yyZ1w5Yzy5YHPwOjqvpr6Qwuf/pYz9pG4wzCiuy7nM3/EM
         cIIoGQ1DsIQbFjBGnDVDVOBykjy5nz3M/PYItFrt4KbhZJxXt/g6D18x3xXVXV9DHMxQ
         K1x3oVDCCWrmqQexBtG5Ln+1ejqOD/EBUZWL1Yl3b5DUhzkusYzuIuK+9VuODziNU2oO
         iSzk6ITfsyStU/FaXId/P9bkRD8RW6IbgE5mQw48nco3aQhrCwqRpI6SVc46idTgCcyR
         5jwQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707408676; x=1708013476;
        h=content-transfer-encoding:in-reply-to:organization:references:cc:to
         :content-language:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+az7xZn/dvncgU4RIhCZjis7qc7EUKrlPMvS+7lsMzY=;
        b=gedMfk1396LoH/DPglLS2dk6/ZrSq1ObB7CWDnteo3pnEftBP3o9XT/D+i7Mzz+2mo
         yWk0BQehvfUBUnNyj2+bGiPsS+5b62WTs6mhMexJLMsfjPkwDeDNPC0kvzamzzaJ0HEc
         8+FFli0EsLbu6PmYeHDDHh48MFCfxqM1CbVbkuUaIBDjQ/ENp+vWl7EPGjXBlVumi2bj
         KIlSfhtDRpSNR0RTK1QtO0+GXmeRNCp1gH3m8VVxnD2TYhdpZClego31RJieBDT3SEd/
         /PIdGsRioVIeE5z47R994QtOf5tfxYkcvVHg2Kdfw4sWL6E0Lkn/yKlnjmw7IXoHI01O
         z4Wg==
X-Forwarded-Encrypted: i=1; AJvYcCWr7n4WwZKEJjoB06f03dABxz8WXa8KCEuwmTf2mJ6iVLQjxpOCigmpbQk4Pm/XknQ2eWAhDXG7xhOdLelckmGn+A2tcZ+6F5cLss/nfPM=
X-Gm-Message-State: AOJu0Yz4pywjfvC6CYspF0t9p6nQk8iDZCcn1CGxLVkEqRcCenocQq5K
	4SCRrxw0qbNiOmmtC9kxg46Nw1rZNv0y/FKZd0OndoBCCWPGVoXV
X-Google-Smtp-Source: AGHT+IFYpkvr4trMVeU0gWqW1he4OnOUM+21Vvv73CpCSPxSCCye6rF2uWOL3gd9pqLSPhfPIppMUQ==
X-Received: by 2002:a05:600c:1ca4:b0:40e:e025:8d4e with SMTP id k36-20020a05600c1ca400b0040ee0258d4emr7355372wms.27.1707408675786;
        Thu, 08 Feb 2024 08:11:15 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUkfXZMKgSmW92XYuQiEmIQD9bQKUUylP5f6+z+yr+N0K3LAY6KKkSyPg3afcP5kj44XL+VFVdlxIkmvEOiSQBqMHunKaGFkqZEVgiXZvtYuzx/qVCiSakF0iYiwMgSVea6Xb2LK0MSIffP3w+vyjASvUgs4NloyxebL+E46dpSMATf9fc1CN4=
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <d4e3ea26-961a-4c30-a366-d0afba475813@xen.org>
Date: Thu, 8 Feb 2024 16:11:07 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH] x86/HVM: tidy state on hvmemul_map_linear_addr()'s error
 path
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Manuel Andreas <manuel.andreas@tum.de>
References: <c4c1d2b3-591e-403f-879b-bbb897f7ff25@suse.com>
 <c0a2b6fe-e64f-433a-accf-124a7cee3f91@suse.com>
Organization: Xen Project
In-Reply-To: <c0a2b6fe-e64f-433a-accf-124a7cee3f91@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 08/02/2024 15:59, Jan Beulich wrote:
> On 06.02.2024 13:06, Jan Beulich wrote:
>> While in the vast majority of cases failure of the function will not
>> be followed by re-invocation with the same emulation context, a few
>> very specific insns - involving multiple independent writes, e.g. ENTER
>> and PUSHA - exist where this can happen. Since failure of the function
>> only signals to the caller that it ought to try an MMIO write instead,
>> such failure also cannot be assumed to result in wholesale failure of
>> emulation of the current insn. Instead we have to maintain internal
>> state such that another invocation of the function with the same
>> emulation context remains possible. To achieve that we need to reset MFN
>> slots after putting page references on the error path.
>>
>> Note that all of this affects debugging code only, in causing an
>> assertion to trigger (higher up in the function). There's otherwise no
>> misbehavior - such a "leftover" slot would simply be overwritten by new
>> contents in a release build.
>>
>> Also extend the related unmap() assertion, to further check for MFN 0.
>>
>> Fixes: 8cbd4fb0b7ea ("x86/hvm: implement hvmemul_write() using real mappings")
>> Reported.by: Manuel Andreas <manuel.andreas@tum.de>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Just noticed that I forgot to Cc Paul.
> 
> Jan
> 
>> ---
>> While probably I could be convinced to omit the #ifndef, I'm really
>> considering to extend the one in hvmemul_unmap_linear_addr(), to
>> eliminate the zapping from release builds: Leaving MFN 0 in place is not
>> much better than leaving a (presently) guest-owned one there. And we
>> can't really put/leave INVALID_MFN there, as that would conflict with
>> other debug checking.

Would it be worth defining a sentinel value for this purpose rather than 
hardcoding _mfn(0)? (_mfn(0) seems like a reasonable sentinel... it's 
just a question of having a #define for it).

Either way...

Acked-by: Paul Durrant <paul@xen.org>

>>
>> --- a/xen/arch/x86/hvm/emulate.c
>> +++ b/xen/arch/x86/hvm/emulate.c
>> @@ -696,7 +696,12 @@ static void *hvmemul_map_linear_addr(
>>    out:
>>       /* Drop all held references. */
>>       while ( mfn-- > hvmemul_ctxt->mfn )
>> +    {
>>           put_page(mfn_to_page(*mfn));
>> +#ifndef NDEBUG /* Clean slot for a subsequent map()'s error checking. */
>> +        *mfn = _mfn(0);
>> +#endif
>> +    }
>>   
>>       return err;
>>   }
>> @@ -718,7 +723,7 @@ static void hvmemul_unmap_linear_addr(
>>   
>>       for ( i = 0; i < nr_frames; i++ )
>>       {
>> -        ASSERT(mfn_valid(*mfn));
>> +        ASSERT(mfn_x(*mfn) && mfn_valid(*mfn));
>>           paging_mark_dirty(currd, *mfn);
>>           put_page(mfn_to_page(*mfn));
>>   
> 



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:12:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:12:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678324.1055470 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY713-00046h-AL; Thu, 08 Feb 2024 16:12:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678324.1055470; Thu, 08 Feb 2024 16:12:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY713-00046a-6o; Thu, 08 Feb 2024 16:12:45 +0000
Received: by outflank-mailman (input) for mailman id 678324;
 Thu, 08 Feb 2024 16:12:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=z5WF=JR=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rY712-00046S-10
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:12:44 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e2f3166b-c69c-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:12:42 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 8DE014EE0739;
 Thu,  8 Feb 2024 17:12:41 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2f3166b-c69c-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Thu, 08 Feb 2024 17:12:41 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH v2] docs/misra: add asm-offset.c to exclude-list
In-Reply-To: <891e8cc2-e073-4cd3-9998-4c01ae775a3b@suse.com>
References: <09b659e19bf2cc6b3ee4320e019bdfa7def5f3b8.1707406598.git.nicola.vetrini@bugseng.com>
 <891e8cc2-e073-4cd3-9998-4c01ae775a3b@suse.com>
Message-ID: <6521ace1928267e8d6690265181deba8@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-08 16:56, Jan Beulich wrote:
> On 08.02.2024 16:50, Nicola Vetrini wrote:
>> These files contain several deliberate violations of MISRA C rules and
>> they are not linked in the final Xen binary, therefore they can be 
>> exempted
>> from MISRA compliance.
>> 
>> No functional change.
>> 
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> Acked-by: Jan Beulich <jbeulich@suse.com>
> albeit ...
> 
>> --- a/docs/misra/exclude-list.json
>> +++ b/docs/misra/exclude-list.json
>> @@ -101,6 +101,10 @@
>>              "rel_path": "arch/x86/efi/check.c",
>>              "comment": "The resulting code is not included in the 
>> final Xen binary, ignore for now"
>>          },
>> +        {
>> +          "rel_path": "arch/*/*/asm-offsets.c",
>> +          "comment": "The resulting code is not included in the final 
>> Xen binary, ignore for now"
>> +        },
>>          {
>>              "rel_path": "common/coverage/*",
>>              "comment": "Files to support gcov, ignore for now"
> 
> ... something looks odd with indentation; can probably be adjusted
> while committing.
> 
> Jan

Sorry, I didn't notice the wrong indentation.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:17:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:17:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678329.1055480 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY75t-0004oD-SM; Thu, 08 Feb 2024 16:17:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678329.1055480; Thu, 08 Feb 2024 16:17:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY75t-0004o6-PG; Thu, 08 Feb 2024 16:17:45 +0000
Received: by outflank-mailman (input) for mailman id 678329;
 Thu, 08 Feb 2024 16:17:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY75s-0004o0-3Z
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:17:44 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 959bd598-c69d-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:17:41 +0100 (CET)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2d0d95e8133so401031fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 08:17:41 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m13-20020adfa3cd000000b0033ae50e2c6asm3959655wrb.83.2024.02.08.08.17.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 08:17:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 959bd598-c69d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707409061; x=1708013861; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vrAgSpRzw0VF3hyy7scsIHdqo8uFRtXsBzGWlj8EKnU=;
        b=XT3nplztcGn36BbDAxxVS6l3ICt7d3N3MeJDvggIHq4XrclezkjvJpWyhA8HmfNfQ1
         hRDceUW/jDBEUXRvlZqrLYoxddITG3Aq5ycHRQWI3YzTmpuIYicN4KrC2r+4SFiERVgX
         N5kiH7VHfr2375hFNMNm7YkW8/wH+GqVCCKy01SIPfHwMDj9aRzN3rUFi3k8bqF1XClh
         Eu4GFfPGukL965+6uhkQvYJVvedAtnm1bEd1Ikbwh/LVWEfn0WgIXtmLy2WQV678RSn5
         NdixIJ77zcuZnd7Z174qhgLpr0kDrl/Xl/0lRNlrPwohJg2UPn4mwqsrdM29v5cfRGwY
         wsQg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707409061; x=1708013861;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vrAgSpRzw0VF3hyy7scsIHdqo8uFRtXsBzGWlj8EKnU=;
        b=v5qVwLkXuSPhn4SSFXcNA1d2pyWD01hMOwvwz7rcBqQ+eJA/q4nNLz1tX0MXEBd3A9
         zKXRENzT///Fx//Ik6X6uI9WxLK3yVYcGvdRlJ4864PXq/UMd5gca0QdM9W2s7rNBOup
         wbyaCfJ8vzYZisMx3shDhU6MhLrK9r813EyQbA1KTBl6e93lP4n5do4lLn4ueapW7xbj
         aqnXBeMVE4XHIu0pjmb7/+vD/YjrdaQ9tvxrmLjiZu84TSvRYEGFJl7fQePWgiUJxwoj
         1k4rtwY0x5igaSIl3VSMGsInLDc42WcZ/0Ujjth4upfqKS4DBFHkKphpZDqVvXTC7V3e
         Qzgw==
X-Gm-Message-State: AOJu0YypRAkF5ELsO8TnTkDYQgWavBnrMTRTGG9iCIsXG5eAayYbbrtb
	pzXNWIjFwOlLiEhy4QXAw0lliHMNhR9DXQg/+Hfrium5V6iW74m9Br97A1nRsA==
X-Google-Smtp-Source: AGHT+IHrxvZ/S950vrLi99fwE/2Z918+m6wqxV9EhfMJRslLQ9sDzePtryHvj51c3KmI9g4GVQs9yA==
X-Received: by 2002:a2e:9bcb:0:b0:2cd:1ca6:87bf with SMTP id w11-20020a2e9bcb000000b002cd1ca687bfmr6957020ljj.8.1707409061241;
        Thu, 08 Feb 2024 08:17:41 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUm/bnmKOGk3/+6BAzSKuBsNyCbGwQc2a1wIXAvxsLRrNr9ZPbgO2gnF7adtSg7R0mJNDOuBdvrLX2+c/+FEQzQVXHQxJrs9rb6Kqjs30I0ACbvPQf/tjkSYw/P8ofashutoj0LJKnPcABDF+HlfrRt3RoYe83Yh/EaPMDJaSlLjXIMUS8XDC4=
Message-ID: <3120d2fe-0698-4c7c-9916-45ff5a32b3a7@suse.com>
Date: Thu, 8 Feb 2024 17:17:39 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/HVM: tidy state on hvmemul_map_linear_addr()'s error
 path
Content-Language: en-US
To: paul@xen.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Manuel Andreas <manuel.andreas@tum.de>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <c4c1d2b3-591e-403f-879b-bbb897f7ff25@suse.com>
 <c0a2b6fe-e64f-433a-accf-124a7cee3f91@suse.com>
 <d4e3ea26-961a-4c30-a366-d0afba475813@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d4e3ea26-961a-4c30-a366-d0afba475813@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.02.2024 17:11, Paul Durrant wrote:
> On 08/02/2024 15:59, Jan Beulich wrote:
>> On 06.02.2024 13:06, Jan Beulich wrote:
>>> While in the vast majority of cases failure of the function will not
>>> be followed by re-invocation with the same emulation context, a few
>>> very specific insns - involving multiple independent writes, e.g. ENTER
>>> and PUSHA - exist where this can happen. Since failure of the function
>>> only signals to the caller that it ought to try an MMIO write instead,
>>> such failure also cannot be assumed to result in wholesale failure of
>>> emulation of the current insn. Instead we have to maintain internal
>>> state such that another invocation of the function with the same
>>> emulation context remains possible. To achieve that we need to reset MFN
>>> slots after putting page references on the error path.
>>>
>>> Note that all of this affects debugging code only, in causing an
>>> assertion to trigger (higher up in the function). There's otherwise no
>>> misbehavior - such a "leftover" slot would simply be overwritten by new
>>> contents in a release build.
>>>
>>> Also extend the related unmap() assertion, to further check for MFN 0.
>>>
>>> Fixes: 8cbd4fb0b7ea ("x86/hvm: implement hvmemul_write() using real mappings")
>>> Reported.by: Manuel Andreas <manuel.andreas@tum.de>
>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>
>> Just noticed that I forgot to Cc Paul.
>>
>> Jan
>>
>>> ---
>>> While probably I could be convinced to omit the #ifndef, I'm really
>>> considering to extend the one in hvmemul_unmap_linear_addr(), to
>>> eliminate the zapping from release builds: Leaving MFN 0 in place is not
>>> much better than leaving a (presently) guest-owned one there. And we
>>> can't really put/leave INVALID_MFN there, as that would conflict with
>>> other debug checking.
> 
> Would it be worth defining a sentinel value for this purpose rather than 
> hardcoding _mfn(0)? (_mfn(0) seems like a reasonable sentinel... it's 
> just a question of having a #define for it).

Perhaps, but that's for a separate patch then.

> Either way...
> 
> Acked-by: Paul Durrant <paul@xen.org>

Thanks.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:21:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:21:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678334.1055490 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY798-0006bd-9m; Thu, 08 Feb 2024 16:21:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678334.1055490; Thu, 08 Feb 2024 16:21:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY798-0006bW-72; Thu, 08 Feb 2024 16:21:06 +0000
Received: by outflank-mailman (input) for mailman id 678334;
 Thu, 08 Feb 2024 16:21:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY797-0006bQ-Fe
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:21:05 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0cc92b68-c69e-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:21:01 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-4102f273c46so207475e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 08:21:01 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 q4-20020adfea04000000b0033afcc899c1sm3927352wrm.13.2024.02.08.08.21.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 08:21:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0cc92b68-c69e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707409261; x=1708014061; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=xqG/Vmab/OeEfIeTFczQuKZrhTLj5YYY2ejvrMDjsJA=;
        b=VEFg8/2dx8tdjFSvk+B++EvYIPsHSF/kdPhtVUiFQeU5tpBVZ59AtilmeznU2VGbwQ
         ZsY9RBznivdm30iFJfGj7ARUuodbrgOnxJNjqSCzaV6FGK0uk81U4TohWLNIwhTNSp3E
         tyYUKpwICuFmBbk5XPrGtxdhUg6PZEa5oltbRlewwXuUO3Gv3cCAe3uIxoB4r4N6mbCU
         /4l4XE2GATiUnb+XwGcQ7V6LgwJxYO2JfLEmOK7IHFzRJNLTR/csEtV49WtczQmJsmLJ
         YkrFFlCfeGjhJ+1HiS1YPYCPtPob8FSnrpmLCAXzrkz+SfSuQRjDjfRVyPPdqd4Ic6B8
         GPiw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707409261; x=1708014061;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=xqG/Vmab/OeEfIeTFczQuKZrhTLj5YYY2ejvrMDjsJA=;
        b=douXm+eyBKtqHaJVkQePbtaMBYKov9v1XNSQ4qInzJdmgvARXbuOlGtIV5uIxXMRoU
         uUmOHu6XSqlwTjQ3YMUmErSw7xjjf4Ay1+y6DaKq5bdvYaYLXoZHlpUaMRLmD9gLH7IO
         SIG/TslsWx8oF5PGw/y0r+m3bnJCbt97JBiU3XQdo6ntp/1P+KKs67JD7wE2D7IG6wWT
         bQaUlg4z8nBVFf2EaYMx7zeibNPeLbhCiGiEVCPknDGlElwnXVKD5EJuMmf/Q533fxPS
         qfCsmb3kWm/9cdQakUyI1VRQq6BAU4o1uaI9T/vNxQPTGWJklkA1wwPp4PmoA4kqyz5M
         oUnw==
X-Gm-Message-State: AOJu0YwATwYqjQQ4IpcNaKd5rcCF+0Dn0RqwmEGlbRzhO13ttGei+66i
	rJevbjf2XmpzQlBJmw6uj7vdKnNbpgkwCIYVQZzlz4eWPls9mhLc+ArIKMCgHw==
X-Google-Smtp-Source: AGHT+IG5eNWYMVBH+AvsGH8q161Jguv0pvgJ5f20ENibyuAMOFGBGk/wtIpsSP9I12eNC6KtuhX94w==
X-Received: by 2002:a5d:4607:0:b0:33b:470b:a377 with SMTP id t7-20020a5d4607000000b0033b470ba377mr5863752wrq.14.1707409261220;
        Thu, 08 Feb 2024 08:21:01 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVvLvlux8ux2QjLbif3Ie6ZENmuKiV5+pfGK9rcIVdEBMUqsGZvnieP+yv6FOMUvdMSRWbND6qANQNJxupqcqRLbiFZ4Jd/BZVLYz1zsikzq5GHWUrJivXW4FWttxKWkKGhw+MOUmg1Ekso6v1Yiqr5Q5z4k9ZAOTGgcKzG9XKiGmapTWQ7x0j8uzYu8moAJo4tqJSLp8cJJC2QknPmAHiSlFQJGrl1GC+oYKdv
Message-ID: <1d3f93c0-64ac-414d-b677-ec48a1ddf046@suse.com>
Date: Thu, 8 Feb 2024 17:20:59 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 3/7] VMX: convert entry point annotations
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <e4bf47ca-2ae6-1fd4-56a6-e4e777150b64@suse.com>
 <3ba82c3a-ff95-43d0-8672-a63b23bc2cdc@suse.com>
 <5fc304c0-be1f-46dd-a783-4030ec76a2f8@suse.com>
 <c11bd472-fc43-49c5-8482-4fc7ed836269@citrix.com>
 <a6bf6d72-5796-4ccc-aa3e-38438f986b32@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a6bf6d72-5796-4ccc-aa3e-38438f986b32@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07.02.2024 15:25, Jan Beulich wrote:
> On 07.02.2024 14:55, Andrew Cooper wrote:
>> On 07/02/2024 1:37 pm, Jan Beulich wrote:
>>> Use the generic framework from xen/linkage.h.
>>>
>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>> ---
>>> v6: New.
>>>
>>> --- a/xen/arch/x86/hvm/vmx/entry.S
>>> +++ b/xen/arch/x86/hvm/vmx/entry.S
>>> @@ -24,7 +24,7 @@
>>>  #define VMRESUME     .byte 0x0f,0x01,0xc3
>>>  #define VMLAUNCH     .byte 0x0f,0x01,0xc2
>>>  
>>> -ENTRY(vmx_asm_vmexit_handler)
>>> +FUNC(vmx_asm_vmexit_handler)
>>>          SAVE_ALL
>>>  
>>>          mov  %cr2,%rax
>>> @@ -132,7 +132,7 @@ UNLIKELY_END(realmode)
>>>          call vmx_vmentry_failure
>>>          jmp  .Lvmx_process_softirqs
>>>  
>>> -ENTRY(vmx_asm_do_vmentry)
>>> +LABEL(vmx_asm_do_vmentry)
>>
>> This really is a function, not a label.
>>
>> xen.git/xen$ git grep vmx_asm_do_vmentry
>> arch/x86/hvm/vmx/entry.S:135:ENTRY(vmx_asm_do_vmentry)
>> arch/x86/hvm/vmx/vmcs.c:1855:void noreturn vmx_asm_do_vmentry(void);
>> arch/x86/hvm/vmx/vmcs.c:1929:    reset_stack_and_jump(vmx_asm_do_vmentry);
>>
>> It is giant mess, of two functions forming part of the same loop.
>>
>> Considering that you declines to take CODE, I don't know what to
>> suggest.  The point of CODE, distinct to FUNC, was to identify the
>> places where weird things were going on, and this absolutely counts.
> 
> What's not clear to me: How would CODE() differ from both FUNC() and
> LABEL()? And if the symbol is to be a function, what's wrong with
> using FUNC() here as is?

Well, I figured this one: FUNC() may switch sections following patch 1,
so indeed we'd need something that is much like FUNC(), but without the
(optional) section switch.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:27:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:27:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678340.1055500 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7Fi-0007f6-0e; Thu, 08 Feb 2024 16:27:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678340.1055500; Thu, 08 Feb 2024 16:27:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7Fh-0007ez-Sx; Thu, 08 Feb 2024 16:27:53 +0000
Received: by outflank-mailman (input) for mailman id 678340;
 Thu, 08 Feb 2024 16:27:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rY7Fg-0007et-H4
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:27:52 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rY7Fg-0002K5-85; Thu, 08 Feb 2024 16:27:52 +0000
Received: from [15.248.3.89] (helo=[10.24.67.25])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rY7Ff-0000xo-Sb; Thu, 08 Feb 2024 16:27:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=T3L/iBN307abZCiBOFfFStrhdwZOQ6iq+KEt3EZiv5A=; b=GgnmgmSm3rttNhcWsC+un4iMwq
	eBxWW45O6/WmCGqe5pfdvwHiSqE6Lic6K5dnS4NTBfcSMSyKi97R+4BQMkjPw3FaWqcDSVzSVqmqs
	orkCfDJRr6zsGmPCcn5D/iiz8jxqgs0109XOuRAfErKAu/QBa5IES4iVBoEw1pmp+Opc=;
Message-ID: <e090be7b-1ceb-4dc0-a664-089179236688@xen.org>
Date: Thu, 8 Feb 2024 16:27:49 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] docs: add a best practices coding guide
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20240208160515.9949-1-jgross@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240208160515.9949-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Juergen,

On 08/02/2024 16:05, Juergen Gross wrote:
> Today the CODING_STYLE contains a section "Handling unexpected
> conditions" specific to the hypervisor. This section is kind of
> misplaced for a coding style. It should rather be part of a "Coding
> best practices" guide.
> 
> Add such a guide as docs/process/coding-best-practices.pandoc and
> move the mentioned section from CODING_STYLE to the new file, while
> converting the format to pandoc.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:43:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:43:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678344.1055509 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7UI-0003AT-9e; Thu, 08 Feb 2024 16:42:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678344.1055509; Thu, 08 Feb 2024 16:42:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7UI-0003AM-71; Thu, 08 Feb 2024 16:42:58 +0000
Received: by outflank-mailman (input) for mailman id 678344;
 Thu, 08 Feb 2024 16:42:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=nVk9=JR=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rY7UH-0003AG-AE
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:42:57 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1c0085aa-c6a1-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:42:55 +0100 (CET)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-33b5b6236afso432331f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 08:42:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h16-20020adff4d0000000b0033af3a43e91sm3966233wrp.46.2024.02.08.08.42.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 08:42:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c0085aa-c6a1-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707410575; x=1708015375; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cGp8YxNLTClF4yl5BUibCCne27gUKs9JcvoLO4RVgmk=;
        b=RsF8/VkF7E8i1MrYQl3OFdTCULyooUcoKImaYSB98TjFcP1qggvaNeIoaNSi+FC6Ch
         9Lwaj7dEYeJKfJx5tY1m3t5u7jcnFb5M+lxyliSchXjdZXyjK2BTKKygij2eEXANTnv4
         W3aiFk5td7iEFIj1h8Ir0Urko8fa+o+tLnRY6MikZ8W/lQ+q7lKGlvRKhGLEtsT1JUgy
         OnI422e9hEKGCGPM0T/1P25nz75I+6xBl9RktkLKJUCh4dJOFhoXLVN4zQeQxcN5JYt6
         GKS1fBNbRc+O5fVmlRYDsKXdtbirBXcx8Ss+XlqU5+1iYDgMojZemvUB74JUyCAk8e1c
         ZaGg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707410575; x=1708015375;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cGp8YxNLTClF4yl5BUibCCne27gUKs9JcvoLO4RVgmk=;
        b=ChBpcWg8VQTdd8ypwC8dsYQtcRlP1uHY6k4TDhIRWiCF5psyjltgg5bEPge4GHFOfU
         O1SFfV+3CUYfpW+MX5SHG0r4/PIU36xeaxdq/76xBPXYwOFzHlGfB2JoBYMqubbS8G2Q
         smMZq2w0oOh61Gkq2hZ6empvyj0GKhYIGhqjd6J5ycxFrQMohrFSjvW/1+zLHm4ohAYu
         4riGvCzmd1C67I0/pOs49Dxjlbr1zuSN0e2vdsZMLmA3C4wxAyaq5J7dD2NZ3TFaLV6P
         2HpOmGzxH1krMAAWaFdJ6IsaasinzNQWc9MG5fk3zpsBhlJM3RPraDUAgFk1UF47oT5+
         c5Fg==
X-Forwarded-Encrypted: i=1; AJvYcCUC/qRv7369hxi8V+SKePofpldycdS0nX2HeqMXFOygH4zRjmsAjIl4epuH1NzxW0j6L4hrHbU7Msy6QmTdxmxXS0hA/QBg+9bQScvHcjE=
X-Gm-Message-State: AOJu0YyAh6ZHheFnfuBmbb+bVFDjRVOm5NMQnMk8vXTbqT+cENBjYcOt
	h745Z/nT3I8JfcAQ8qC9z25+fSURj04M0hyX3dgCo2cGyLpDkKKHfX1P5Fnz2g==
X-Google-Smtp-Source: AGHT+IEI4Fjilf81VGgEHBoyzxWBfpgn/p+uWt2LI60tH+S12zn4hmHQ/4Hk7A2kR93VX+Uyuy9Hog==
X-Received: by 2002:adf:a299:0:b0:33b:180d:edc7 with SMTP id s25-20020adfa299000000b0033b180dedc7mr2246wra.17.1707410575316;
        Thu, 08 Feb 2024 08:42:55 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXfyPAeWm4/aQiOxjc0Q0R7lp/NpvNvB2ofwcMhFnix3tVKGQaGsHkzeTfQoBU6KJ2aGclg09O15s7xSPAtF6bStt397p9LoRtwSFqZtbUGqYlD0S/0whsocK/vXUSK7SlgY7UnFAqYAfBuX8jlGbS1czYu4uxhBNK+eMFi18mmnSe/d0+f/73OUfAGsXOOxG88duDdDANV+A8wYyxjCeSRm1EOYxh5hbSjwJlbGIVDgv7pJgsgS9XFoIpP7OypAaxGl3puPrL5cO7rfsjvXzlVBsgZimn+3MpKkQQhKQYgd/UFfE/++sq2zPXat+Xb+dx/TiPsguqstPptkbcoP/K+wzPZuZrPi441tYuEJaK8lVK8ep7tn0AQ0fp/NEhstswyjeaVEEpUFBAlYOFxS9v+473d7uArLeuip6cE2Cz6D80h0MFUaaGsQKc1vShhnmrYzsKtmY1JotDE5Na+FfA7gyG5RFG4n5jYwVCGFgYtqD/oQ4s=
Message-ID: <91e58646-5420-4e1c-a8c5-b6ff7a417d13@suse.com>
Date: Thu, 8 Feb 2024 17:42:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(),
 ASSERT_UNREACHABLE() to xen/bug.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 xen-devel@lists.xenproject.org
References: <4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 19:03, Oleksii Kurochko wrote:
> Move the macros mentioned in the commit subject to their appropriate
> locations.
> Additionally, eliminate the dependency of xen/lib.h from xen/bug.h and
> include "xen/bug.h" in files where xen/bug.h macros are utilized.
> 
> Most of the changes were made because a file requires macros from xen/bug.h,
> except for some files for Arm which require definitions of BUG_OPCODE,
> BUG_INSTR, BUG_FN_REG.
> 
> xen/lib.h was added to list-sort.c ( otherwise compilation errors related
> to {d}printk occur during compilation of list-sort.c. ) as xen/lib.h was
> removed from xen/list.h. Since nothing in xen/list.h depends on xen/lib.h
> functionality and only xen/bug.h is needed.
> 
> cpufeature.h requires the inclusion of <xen/cache.h>;
> otherwise, the following error will occur:
> ld: common/monitor.o:/build/xen/./arch/x86/include/asm/cpufeature.h:41:
> multiple definitions of `__cacheline_aligned';
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
with one nit:

> --- a/xen/include/xen/cpumask.h
> +++ b/xen/include/xen/cpumask.h
> @@ -55,6 +55,7 @@
>   * for_each_present_cpu(cpu)		for-loop cpu over cpu_present_map
>   */
>  
> +#include <xen/bug.h>
>  #include <xen/bitmap.h>
>  #include <xen/kernel.h>
>  #include <xen/random.h>

I think everywhere else you nicely inserted new #include-s so respect
alphabetical ordering, just here the insertion would want to move one
line down. Likely easy to do while committing, if no other reason for
a v2 arises ...

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:55:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:55:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678350.1055520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7gn-0005bQ-Hs; Thu, 08 Feb 2024 16:55:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678350.1055520; Thu, 08 Feb 2024 16:55:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7gn-0005bJ-EG; Thu, 08 Feb 2024 16:55:53 +0000
Received: by outflank-mailman (input) for mailman id 678350;
 Thu, 08 Feb 2024 16:55:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7gl-0005bD-Al
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:55:51 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e96963d0-c6a2-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:55:50 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 616B71FCF6;
 Thu,  8 Feb 2024 16:55:49 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0B6DD1326D;
 Thu,  8 Feb 2024 16:55:49 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id trolAZUHxWWfEwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:55:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e96963d0-c6a2-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411349; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=jvl+vfXWB3anELPj+eNKY9N43mJyk/v1hE3rU19JhxI=;
	b=Ew4JmYzJ1T5Ef90BZhaYHrWje39cyQIuNkYQZPAxfA0cs9bIqQLJhcO20eeAuTiWpB0boJ
	DGkfKhcf/B26+m/3wasCLaMEqs9CYERLJ5OGzu/JClSFOIVGL6zmVf1lUvtGd7NvlANhJa
	gCYg66cR0gJhecRiFASrrUQahGgjnLc=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411349; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=jvl+vfXWB3anELPj+eNKY9N43mJyk/v1hE3rU19JhxI=;
	b=Ew4JmYzJ1T5Ef90BZhaYHrWje39cyQIuNkYQZPAxfA0cs9bIqQLJhcO20eeAuTiWpB0boJ
	DGkfKhcf/B26+m/3wasCLaMEqs9CYERLJ5OGzu/JClSFOIVGL6zmVf1lUvtGd7NvlANhJa
	gCYg66cR0gJhecRiFASrrUQahGgjnLc=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH v5 00/22] tools: enable xenstore-stubdom to use 9pfs
Date: Thu,  8 Feb 2024 17:55:24 +0100
Message-Id: <20240208165546.5715-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [1.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,ens-lyon.org];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: *
X-Spam-Score: 1.90
X-Spam-Flag: NO

This series is adding 9pfs support to Xenstore-stubdom, enabling it
to do logging to a dom0 directory.

This is a prerequisite for the final goal to add live update support
to Xenstore-stubdom, as it enables the stubdom to store its state in
a dom0 file.

The 9pfs backend is a new daemon written from scratch. Using a
dedicated 9pfs daemon has several advantages:

- it is using much less resources than a full blown qemu process
- it can serve multiple guests (the idea is to use it for other
  infrastructure domains, like qemu-stubdom or driver domains, too)
- it is designed to support several security enhancements, like
  limiting the number of files for a guest, or limiting the allocated
  file system space
- it doesn't support file links (neither hard nor soft links) or
  referencing parent directories via "..", minimizing the risk that
  a guest can "escape" from its home directory

Note that for now the daemon only contains the minimal needed
functionality to do logging from Xenstore-stubdom. I didn't want to
add all the 9pfs commands and security add-ons in the beginning, in
order to avoid needless efforts in case the idea of the daemon is
being rejected.

Please note that the pending patch for updating the Mini-OS commit
in Config.mk needs to be applied for patch "stubdom: extend xenstore
stubdom configs" and the following ones.

Changes in V5:
- 10 patches have been applied already
- rename source directory to tools/9pfsd
- addressed comments

Changes in V4:
- patch 2 of V3 was applied
- added support of reading directories
- addressed review comments

Changes in V3:
- new patches 1, 23-25
- addressed review comments

Changes in V2:
- support of multiple rings per device
- xenlogd->xen-9pfsd rename
- addressed review comments
- fixed some bugs

Juergen Gross (22):
  tools: add access macros for unaligned data
  tools: add a new xen 9pfs daemon
  tools/9pfsd: connect to frontend
  tools/9pfsd: add transport layer
  tools/9pfsd: add 9pfs response generation support
  tools/9pfsd: add 9pfs version request support
  tools/9pfsd: add 9pfs attach request support
  tools/9pfsd: add 9pfs walk request support
  tools/9pfsd: add 9pfs open request support
  tools/9pfsd: add 9pfs clunk request support
  tools/9pfsd: add 9pfs create request support
  tools/9pfsd: add 9pfs stat request support
  tools/9pfsd: add 9pfs write request support
  tools/9pfsd: add 9pfs read request support
  tools/libs/light: add backend type for 9pfs PV devices
  tools/xl: support new 9pfs backend xen_9pfsd
  stubdom: extend xenstore stubdom configs
  tools: add 9pfs device to xenstore-stubdom
  tools/xenstored: mount 9pfs device in stubdom
  tools/xenstored: add helpers for filename handling
  tools/xenstored: support complete log capabilities in stubdom
  tools/xenstored: have a single do_control_memreport()

 docs/man/xl.cfg.5.pod.in                      |   36 +-
 stubdom/xenstore-minios.cfg                   |    2 +-
 stubdom/xenstorepvh-minios.cfg                |    2 +-
 tools/9pfsd/.gitignore                        |    1 +
 tools/9pfsd/Makefile                          |   38 +
 tools/9pfsd/io.c                              | 1513 +++++++++++++++++
 tools/9pfsd/xen-9pfsd.c                       |  800 +++++++++
 tools/9pfsd/xen-9pfsd.h                       |   99 ++
 tools/Makefile                                |    1 +
 tools/golang/xenlight/helpers.gen.go          |   16 +
 tools/golang/xenlight/types.gen.go            |   13 +
 tools/helpers/init-xenstore-domain.c          |    7 +
 .../Linux/init.d/sysconfig.xencommons.in      |    1 -
 tools/hotplug/Linux/launch-xenstore.in        |    1 +
 tools/include/libxl.h                         |   22 +
 tools/include/xen-tools/common-macros.h       |   17 +
 tools/libs/light/libxl_9pfs.c                 |  193 ++-
 tools/libs/light/libxl_create.c               |    4 +-
 tools/libs/light/libxl_dm.c                   |    2 +-
 tools/libs/light/libxl_types.idl              |   11 +
 tools/libs/light/libxl_types_internal.idl     |    1 +
 tools/xenstored/control.c                     |   29 +-
 tools/xenstored/core.c                        |   15 +-
 tools/xenstored/core.h                        |   11 +-
 tools/xenstored/domain.c                      |    2 +
 tools/xenstored/lu_daemon.c                   |    4 +-
 tools/xenstored/minios.c                      |   62 +
 tools/xenstored/posix.c                       |    8 +-
 tools/xl/xl_parse.c                           |   23 +-
 29 files changed, 2892 insertions(+), 42 deletions(-)
 create mode 100644 tools/9pfsd/.gitignore
 create mode 100644 tools/9pfsd/Makefile
 create mode 100644 tools/9pfsd/io.c
 create mode 100644 tools/9pfsd/xen-9pfsd.c
 create mode 100644 tools/9pfsd/xen-9pfsd.h

-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:55:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:55:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678351.1055530 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7gt-0005qx-Ng; Thu, 08 Feb 2024 16:55:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678351.1055530; Thu, 08 Feb 2024 16:55:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7gt-0005qq-L1; Thu, 08 Feb 2024 16:55:59 +0000
Received: by outflank-mailman (input) for mailman id 678351;
 Thu, 08 Feb 2024 16:55:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7gt-0005q0-7M
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:55:59 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ec934bea-c6a2-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:55:55 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 00BB521F60;
 Thu,  8 Feb 2024 16:55:55 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B8C7A1326D;
 Thu,  8 Feb 2024 16:55:54 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id SfSmK5oHxWWmEwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:55:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec934bea-c6a2-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v5 01/22] tools: add access macros for unaligned data
Date: Thu,  8 Feb 2024 17:55:25 +0100
Message-Id: <20240208165546.5715-2-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 00BB521F60
X-Spam-Flag: NO

Add the basic access macros for unaligned data to common-macros.h.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
V3:
- new patch
---
 tools/include/xen-tools/common-macros.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/tools/include/xen-tools/common-macros.h b/tools/include/xen-tools/common-macros.h
index e5ed603904..81fba2e9f5 100644
--- a/tools/include/xen-tools/common-macros.h
+++ b/tools/include/xen-tools/common-macros.h
@@ -79,6 +79,10 @@
 #define __must_check __attribute__((__warn_unused_result__))
 #endif
 
+#ifndef __packed
+#define __packed __attribute__((__packed__))
+#endif
+
 #define container_of(ptr, type, member) ({              \
     typeof(((type *)0)->member) *mptr__ = (ptr);        \
     (type *)((char *)mptr__ - offsetof(type, member));  \
@@ -87,4 +91,17 @@
 #define __AC(X, Y)   (X ## Y)
 #define _AC(X, Y)    __AC(X, Y)
 
+#define get_unaligned_t(type, ptr) ({                               \
+    const struct { type x; } __packed *ptr_ = (typeof(ptr_))(ptr);  \
+    ptr_->x;                                                        \
+})
+
+#define put_unaligned_t(type, val, ptr) do {                        \
+    struct { type x; } __packed *ptr_ = (typeof(ptr_))(ptr);        \
+    ptr_->x = val;                                                  \
+} while (0)
+
+#define get_unaligned(ptr)      get_unaligned_t(typeof(*(ptr)), ptr)
+#define put_unaligned(val, ptr) put_unaligned_t(typeof(*(ptr)), val, ptr)
+
 #endif	/* __XEN_TOOLS_COMMON_MACROS__ */
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:56:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:56:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678352.1055540 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7gw-00067W-Ub; Thu, 08 Feb 2024 16:56:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678352.1055540; Thu, 08 Feb 2024 16:56:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7gw-00067P-RW; Thu, 08 Feb 2024 16:56:02 +0000
Received: by outflank-mailman (input) for mailman id 678352;
 Thu, 08 Feb 2024 16:56:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7gv-0005bD-Hl
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:01 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id efe0cf8d-c6a2-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:56:00 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 8D35C21F60;
 Thu,  8 Feb 2024 16:56:00 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 580A21326D;
 Thu,  8 Feb 2024 16:56:00 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id RjUNFKAHxWWtEwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: efe0cf8d-c6a2-11ee-8a4a-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v5 02/22] tools: add a new xen 9pfs daemon
Date: Thu,  8 Feb 2024 17:55:26 +0100
Message-Id: <20240208165546.5715-3-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 8D35C21F60
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add "xen-9pfsd", a new 9pfs daemon meant to support infrastructure
domains (e.g. xenstore-stubdom) to access files in dom0.

For now only add the code needed for starting the daemon and
registering it with Xenstore via a new "libxl/xen-9pfs/state" node by
writing the "running" state to it.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- rename from xenlogd to xen-9pfsd (Andrew Cooper)
- use a backend domain local Xenstore node (Jason Andryuk)
- use "volatile" for stop_me (Andrew Cooper)
V3:
- fix SPDX Identifier (Andrew Cooper)
V4:
- add strerror() test to error logging (Andrew Cooper)
- don't handle "daemon already running" as error (Anthony Perard)
V5:
- rename source directory to tools/9pfsd (Andrew Cooper)
---
 tools/9pfsd/.gitignore  |   1 +
 tools/9pfsd/Makefile    |  38 +++++++++++
 tools/9pfsd/xen-9pfsd.c | 147 ++++++++++++++++++++++++++++++++++++++++
 tools/Makefile          |   1 +
 4 files changed, 187 insertions(+)
 create mode 100644 tools/9pfsd/.gitignore
 create mode 100644 tools/9pfsd/Makefile
 create mode 100644 tools/9pfsd/xen-9pfsd.c

diff --git a/tools/9pfsd/.gitignore b/tools/9pfsd/.gitignore
new file mode 100644
index 0000000000..d0c2d223ef
--- /dev/null
+++ b/tools/9pfsd/.gitignore
@@ -0,0 +1 @@
+/xen-9pfsd
diff --git a/tools/9pfsd/Makefile b/tools/9pfsd/Makefile
new file mode 100644
index 0000000000..089cf5ae24
--- /dev/null
+++ b/tools/9pfsd/Makefile
@@ -0,0 +1,38 @@
+#
+# tools/9pfsd/Makefile
+#
+
+XEN_ROOT = $(CURDIR)/../..
+include $(XEN_ROOT)/tools/Rules.mk
+
+CFLAGS += $(PTHREAD_CFLAGS)
+LDFLAGS += $(PTHREAD_LDFLAGS)
+
+TARGETS := xen-9pfsd
+
+XEN-9PFSD_OBJS = xen-9pfsd.o
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenstore)
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenevtchn)
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
+xen-9pfsd: LDLIBS += $(call xenlibs-ldlibs,store evtchn gnttab)
+
+.PHONY: all
+all: $(TARGETS)
+
+xen-9pfsd: $(XEN-9PFSD_OBJS)
+	$(CC) $(LDFLAGS) -o $@ $(XEN-9PFSD_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
+
+.PHONY: install
+install: all
+	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
+	for i in $(TARGETS); do $(INSTALL_PROG) $$i $(DESTDIR)$(LIBEXEC_BIN); done
+
+.PHONY: uninstall
+uninstall:
+	for i in $(TARGETS); do rm -f $(DESTDIR)$(LIBEXEC_BIN)/$$i; done
+
+.PHONY: clean
+clean:
+	$(RM) *.o $(TARGETS) $(DEPS_RM)
+
+distclean: clean
diff --git a/tools/9pfsd/xen-9pfsd.c b/tools/9pfsd/xen-9pfsd.c
new file mode 100644
index 0000000000..6939d01574
--- /dev/null
+++ b/tools/9pfsd/xen-9pfsd.c
@@ -0,0 +1,147 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+/*
+ * xen-9pfsd - Xen 9pfs daemon
+ *
+ * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
+ *
+ * Daemon to enable guests to access a directory of the dom0 file system.
+ * Access is made via the 9pfs protocol (xen-9pfsd acts as a PV 9pfs backend).
+ *
+ * Usage: xen-9pfsd
+ *
+ * xen-9pfsd does NOT support writing any links (neither soft links nor hard
+ * links), and it is accepting only canonicalized file paths in order to
+ * avoid the possibility to "escape" from the guest specific directory.
+ *
+ * The backend device string is "xen_9pfs", the tag used for mounting the
+ * 9pfs device is "Xen".
+ *
+ * As an additional security measure the maximum file space used by the guest
+ * can be limited by the backend Xenstore node "max-size" specifying the size
+ * in MBytes. This size includes the size of the root directory of the guest.
+ */
+
+#include <err.h>
+#include <errno.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <syslog.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <xenevtchn.h>
+#include <xengnttab.h>
+#include <xenstore.h>
+
+static volatile bool stop_me;
+static bool daemon_running;
+static struct xs_handle *xs;
+static xengnttab_handle *xg;
+static xenevtchn_handle *xe;
+
+static void handle_stop(int sig)
+{
+    stop_me = true;
+}
+
+static void close_all(void)
+{
+    if ( daemon_running )
+        xs_rm(xs, XBT_NULL, "libxl/xen-9pfs");
+    if ( xe )
+        xenevtchn_close(xe);
+    if ( xg )
+        xengnttab_close(xg);
+    if ( xs )
+        xs_close(xs);
+    closelog();
+}
+
+static void do_err(const char *msg)
+{
+    syslog(LOG_ALERT, "%s, errno = %d, %s", msg, errno, strerror(errno));
+    close_all();
+    exit(1);
+}
+
+static void xen_connect(void)
+{
+    xs_transaction_t t;
+    char *val;
+    unsigned int len;
+
+    xs = xs_open(0);
+    if ( xs == NULL )
+        do_err("xs_open() failed");
+
+    xg = xengnttab_open(NULL, 0);
+    if ( xg == NULL )
+        do_err("xengnttab_open() failed");
+
+    xe = xenevtchn_open(NULL, 0);
+    if ( xe == NULL )
+        do_err("xenevtchn_open() failed");
+
+    while ( true )
+    {
+        t = xs_transaction_start(xs);
+        if ( t == XBT_NULL )
+            do_err("xs_transaction_start() failed");
+
+        val = xs_read(xs, t, "libxl/xen-9pfs/state", &len);
+        if ( val )
+        {
+            free(val);
+            xs_transaction_end(xs, t, true);
+            syslog(LOG_INFO, "daemon already running");
+            close_all();
+            exit(0);
+        }
+
+        if ( !xs_write(xs, t, "libxl/xen-9pfs/state", "running",
+                       strlen("running")) )
+        {
+            xs_transaction_end(xs, t, true);
+            do_err("xs_write() failed writing state");
+        }
+
+        if ( xs_transaction_end(xs, t, false) )
+            break;
+        if ( errno != EAGAIN )
+            do_err("xs_transaction_end() failed");
+    }
+
+    daemon_running = true;
+}
+
+int main(int argc, char *argv[])
+{
+    struct sigaction act = { .sa_handler = handle_stop, };
+    int syslog_mask = LOG_MASK(LOG_WARNING) | LOG_MASK(LOG_ERR) |
+                      LOG_MASK(LOG_CRIT) | LOG_MASK(LOG_ALERT) |
+                      LOG_MASK(LOG_EMERG);
+
+    umask(027);
+    if ( getenv("XEN_9PFSD_VERBOSE") )
+        syslog_mask |= LOG_MASK(LOG_NOTICE) | LOG_MASK(LOG_INFO);
+    openlog("xen-9pfsd", LOG_CONS, LOG_DAEMON);
+    setlogmask(syslog_mask);
+
+    sigemptyset(&act.sa_mask);
+    sigaction(SIGHUP, &act, NULL);
+
+    xen_connect();
+
+    while ( !stop_me )
+    {
+        sleep(60);
+    }
+
+    close_all();
+
+    return 0;
+}
diff --git a/tools/Makefile b/tools/Makefile
index 3a510663a0..7d17211782 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -32,6 +32,7 @@ SUBDIRS-y += xenpmd
 SUBDIRS-$(CONFIG_GOLANG) += golang
 SUBDIRS-y += xl
 SUBDIRS-y += helpers
+SUBDIRS-y += 9pfsd
 SUBDIRS-$(CONFIG_X86) += xenpaging
 SUBDIRS-$(CONFIG_X86) += debugger
 SUBDIRS-$(CONFIG_TESTS) += tests
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:56:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:56:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678353.1055550 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7h3-0006SP-85; Thu, 08 Feb 2024 16:56:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678353.1055550; Thu, 08 Feb 2024 16:56:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7h3-0006SE-2S; Thu, 08 Feb 2024 16:56:09 +0000
Received: by outflank-mailman (input) for mailman id 678353;
 Thu, 08 Feb 2024 16:56:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7h1-0005bD-L0
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:07 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f336e543-c6a2-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:56:06 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 28F5021F60;
 Thu,  8 Feb 2024 16:56:06 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id E86051326D;
 Thu,  8 Feb 2024 16:56:05 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id iLRUN6UHxWW6EwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f336e543-c6a2-11ee-8a4a-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 03/22] tools/9pfsd: connect to frontend
Date: Thu,  8 Feb 2024 17:55:27 +0100
Message-Id: <20240208165546.5715-4-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 28F5021F60
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add the code for connecting to frontends to xenlogd.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- support multiple rings per device (Jason Andryuk)
- don't set .revents initially (Jason Andryuk)
- call poll() with infinite timeout (Jason Andryuk)
- take mutex before calling pthread_cond_signal()
V3:
- fix SPDX indentifier (Andrew Cooper)
- better validation of host path (Jason Andryuk)
- don't hard-code dom0 in backend nodes (Jason Andryuk)
- use bool instead of int for some functions' return types
- open root directory (Jason Andryuk)
---
 tools/9pfsd/Makefile    |   2 +-
 tools/9pfsd/io.c        |  45 +++
 tools/9pfsd/xen-9pfsd.c | 653 +++++++++++++++++++++++++++++++++++++++-
 tools/9pfsd/xen-9pfsd.h |  61 ++++
 4 files changed, 757 insertions(+), 4 deletions(-)
 create mode 100644 tools/9pfsd/io.c
 create mode 100644 tools/9pfsd/xen-9pfsd.h

diff --git a/tools/9pfsd/Makefile b/tools/9pfsd/Makefile
index 089cf5ae24..50573121ed 100644
--- a/tools/9pfsd/Makefile
+++ b/tools/9pfsd/Makefile
@@ -10,7 +10,7 @@ LDFLAGS += $(PTHREAD_LDFLAGS)
 
 TARGETS := xen-9pfsd
 
-XEN-9PFSD_OBJS = xen-9pfsd.o
+XEN-9PFSD_OBJS = xen-9pfsd.o io.o
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenstore)
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenevtchn)
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
new file mode 100644
index 0000000000..eb7c136e09
--- /dev/null
+++ b/tools/9pfsd/io.c
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+/*
+ * xen-9pfsd - Xen 9pfs daemon
+ *
+ * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
+ *
+ * I/O thread handling.
+ */
+
+#include <stdbool.h>
+#include <string.h>
+#include <syslog.h>
+
+#include "xen-9pfsd.h"
+
+static bool io_work_pending(struct ring *ring)
+{
+    if ( ring->stop_thread )
+        return true;
+    return false;
+}
+
+void *io_thread(void *arg)
+{
+    struct ring *ring = arg;
+
+    while ( !ring->stop_thread )
+    {
+        pthread_mutex_lock(&ring->mutex);
+        if ( !io_work_pending(ring) )
+        {
+            if ( xenevtchn_unmask(xe, ring->evtchn) < 0 )
+                syslog(LOG_WARNING, "xenevtchn_unmask() failed");
+            pthread_cond_wait(&ring->cond, &ring->mutex);
+        }
+        pthread_mutex_unlock(&ring->mutex);
+
+        /* TODO: I/O handling. */
+    }
+
+    ring->thread_active = false;
+
+    return NULL;
+}
diff --git a/tools/9pfsd/xen-9pfsd.c b/tools/9pfsd/xen-9pfsd.c
index 6939d01574..73b6c3a30e 100644
--- a/tools/9pfsd/xen-9pfsd.c
+++ b/tools/9pfsd/xen-9pfsd.c
@@ -24,34 +24,632 @@
 
 #include <err.h>
 #include <errno.h>
+#include <fcntl.h>
+#include <poll.h>
+#include <pthread.h>
 #include <signal.h>
 #include <stdbool.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <sys/mman.h>
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <unistd.h>
-#include <xenevtchn.h>
 #include <xengnttab.h>
 #include <xenstore.h>
 
+#include "xen-9pfsd.h"
+
+/*
+ * List of currently known devices.
+ * The list itself is modified only in the main thread. When a device is being
+ * removed its memory needs to be freed after the I/O thread (if existing)
+ * has stopped.
+ */
+static XEN_TAILQ_HEAD(devhead, device) devs = XEN_TAILQ_HEAD_INITIALIZER(devs);
+
+struct path {
+    char path[100];
+};
+
 static volatile bool stop_me;
 static bool daemon_running;
 static struct xs_handle *xs;
 static xengnttab_handle *xg;
-static xenevtchn_handle *xe;
+static unsigned int now;
+
+xenevtchn_handle *xe;
 
 static void handle_stop(int sig)
 {
     stop_me = true;
 }
 
+static int check_host_path(device *device)
+{
+    struct stat statbuf;
+    char *path, *p;
+    int ret = 1;
+
+    if ( !device->host_path )
+        return 1;
+
+    /* Path must be absolute. */
+    if ( device->host_path[0] != '/' )
+        return 1;
+
+    /* No double "/". */
+    if ( strstr(device->host_path, "//") )
+        return 1;
+
+    /* No trailing "/" (includes refusing to share "/"). */
+    if ( device->host_path[strlen(device->host_path) - 1] == '/' )
+        return 1;
+
+    path = strdup(device->host_path);
+    if ( !path )
+    {
+        syslog(LOG_CRIT, "memory allocation failure!");
+        return 1;
+    }
+
+    for ( p = path; p; )
+    {
+        p = strchr(p + 1, '/');
+        if ( p )
+            *p = 0;
+        if ( !stat(path, &statbuf) )
+        {
+            if ( !(statbuf.st_mode & S_IFDIR) )
+                break;
+            if ( !p )
+            {
+                ret = 0;
+                break;
+            }
+            *p = '/';
+            continue;
+        }
+        if ( mkdir(path, 0777) )
+            break;
+        if ( p )
+            *p = '/';
+    }
+
+    free(path);
+    return ret;
+}
+
+static void construct_frontend_path(device *device, const char *node,
+                                    struct path *p)
+{
+    snprintf(p->path, sizeof(p->path), "/local/domain/%u/device/9pfs/%u/%s",
+             device->domid, device->devid, node);
+}
+
+static void construct_backend_path(device *device, const char *node,
+                                   struct path *p)
+{
+    snprintf(p->path, sizeof(p->path), "backend/xen_9pfs/%u/%u/%s",
+             device->domid, device->devid, node);
+}
+
+static char *read_backend_node(device *device, const char *node)
+{
+    struct path p;
+    char *val;
+    unsigned int len;
+
+    construct_backend_path(device, node, &p);
+    val = xs_read(xs, XBT_NULL, p.path, &len);
+
+    return val;
+}
+
+static unsigned int uint_from_string(char *string, unsigned int def)
+{
+    unsigned long val;
+    char *end;
+
+    if ( !string )
+        return def;
+
+    val = strtoul(string, &end, 10);
+    if ( *end || val > UINT_MAX )
+        val = def;
+    free(string);
+
+    return val;
+}
+
+static unsigned int read_backend_node_uint(device *device, const char *node,
+                                           unsigned int def)
+{
+    return uint_from_string(read_backend_node(device, node), def);
+}
+
+static unsigned int read_frontend_node_uint(device *device, const char *node,
+                                            unsigned int def)
+{
+    struct path p;
+    unsigned int len;
+
+    construct_frontend_path(device, node, &p);
+
+    return uint_from_string(xs_read(xs, XBT_NULL, p.path, &len), def);
+}
+
+static bool write_backend_node(device *device, const char *node,
+                               const char *val)
+{
+    struct path p;
+    unsigned int num_perms;
+    struct xs_permissions *old_perms;
+    struct xs_permissions perms[2] = {
+        { .id = 0, .perms = XS_PERM_NONE },
+        { .id = device->domid, .perms = XS_PERM_READ }
+    };
+
+    construct_backend_path(device, node, &p);
+    if ( !xs_write(xs, XBT_NULL, p.path, val, strlen(val)) )
+    {
+        syslog(LOG_ERR, "error writing bacḱend node \"%s\" for device %u/%u",
+               node, device->domid, device->devid);
+        return false;
+    }
+
+    old_perms = xs_get_permissions(xs, XBT_NULL, p.path, &num_perms);
+    if ( !old_perms )
+    {
+        syslog(LOG_ERR, "error getting permissions for \"%s\"", p.path);
+        return false;
+    }
+
+    perms[0] = old_perms[0];
+    free(old_perms);
+    if ( !xs_set_permissions(xs, XBT_NULL, p.path, perms, 2) )
+    {
+        syslog(LOG_ERR, "error setting permissions for \"%s\"", p.path);
+        return false;
+    }
+
+    return true;
+}
+
+static bool write_backend_node_uint(device *device, const char *node,
+                                   unsigned int val)
+{
+    char str[12];
+
+    snprintf(str, sizeof(str), "%u", val);
+
+    return write_backend_node(device, node, str);
+}
+
+static bool write_backend_state(device *device, enum xenbus_state state)
+{
+    struct path p;
+    char val[2];
+
+    snprintf(val, sizeof(val), "%u", state);
+    construct_backend_path(device, "state", &p);
+    if ( !xs_write(xs, XBT_NULL, p.path, val, 1) )
+    {
+        syslog(LOG_ERR, "error writing backend state %u for device %u/%u",
+               state, device->domid, device->devid);
+        return false;
+    }
+
+    device->backend_state = state;
+
+    return true;
+}
+
+static device *find_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+
+    XEN_TAILQ_FOREACH( device, &devs, list )
+    {
+        if ( domid == device->domid && devid == device->devid )
+            return device;
+    }
+
+    return NULL;
+}
+
+static void free_device(device *device)
+{
+    char token[20];
+    struct path p;
+
+    construct_frontend_path(device, "state", &p);
+    snprintf(token, sizeof(token), "%u/%u", device->domid, device->devid);
+    xs_unwatch(xs, p.path, token);
+
+    if ( device->root_fd >= 0 )
+        close(device->root_fd);
+
+    free(device->host_path);
+    free(device);
+}
+
+static device *new_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+    char token[20];
+    struct path p;
+    char *val;
+
+    device = calloc(1, sizeof(*device));
+    if ( !device )
+    {
+        syslog(LOG_CRIT, "Got no memory for new device %u/%u", domid, devid);
+        return NULL;
+    }
+
+    device->domid = domid;
+    device->devid = devid;
+    device->root_fd = -1;
+
+    construct_frontend_path(device, "state", &p);
+    snprintf(token, sizeof(token), "%u/%u", domid, devid);
+    if ( !xs_watch(xs, p.path, token) )
+    {
+        syslog(LOG_ERR, "Setting up watch for device %u/%u failed",
+               domid, devid);
+        free(device);
+        return NULL;
+    }
+
+    val = read_backend_node(device, "security_model");
+    if ( !val || strcmp(val, "none") )
+    {
+        syslog(LOG_ERR, "Security model \"%s\" for device %u/%u invalid.",
+               val, domid, devid);
+        free(val);
+        goto err;
+    }
+    free(val);
+
+    device->max_space = read_backend_node_uint(device, "max-space", 0);
+    device->max_files = read_backend_node_uint(device, "max-files", 0);
+    device->max_open_files =
+        read_backend_node_uint(device, "max-open-files", 0)
+        ?: MAX_OPEN_FILES_DEFAULT;
+    device->auto_delete = read_backend_node_uint(device, "auto-delete", 0);
+
+    device->host_path = read_backend_node(device, "path");
+    if ( check_host_path(device) )
+    {
+        syslog(LOG_ERR, "Host path \"%s\" for device %u/%u invalid.",
+               device->host_path, domid, devid);
+        goto err;
+    }
+    device->root_fd = open(device->host_path, O_RDONLY | O_DIRECTORY);
+    if ( device->root_fd < 0 )
+        goto err;
+
+    if ( !write_backend_node(device, "versions", "1") )
+        goto err;
+    if ( !write_backend_node_uint(device, "max-rings", MAX_RINGS) )
+        goto err;
+    if ( !write_backend_node_uint(device, "max-ring-page-order",
+                                 MAX_RING_ORDER) )
+        goto err;
+
+    if ( !write_backend_state(device, XenbusStateInitWait) )
+        goto err;
+
+    XEN_TAILQ_INSERT_TAIL(&devs, device, list);
+    syslog(LOG_INFO, "New device %u/%u added", domid, devid);
+
+    return device;
+
+ err:
+    free_device(device);
+    return NULL;
+}
+
+static void disconnect_ring(struct ring *ring)
+{
+    if ( !ring )
+        return;
+
+    if ( ring->thread_active )
+    {
+        ring->stop_thread = true;
+        pthread_cond_signal(&ring->cond);
+        pthread_join(ring->thread, NULL);
+        ring->stop_thread = false;
+    }
+
+    if ( ring->data.in )
+    {
+        xengnttab_unmap(xg, ring->data.in, 1 << ring->ring_order);
+        ring->data.in = NULL;
+    }
+    if ( ring->intf )
+    {
+        xengnttab_unmap(xg, ring->intf, 1 );
+        ring->intf = NULL;
+    }
+
+    if ( ring->evtchn )
+    {
+        xenevtchn_unbind(xe, ring->evtchn);
+        ring->evtchn = 0;
+    }
+
+    pthread_mutex_destroy(&ring->mutex);
+    pthread_cond_destroy(&ring->cond);
+}
+
+static void disconnect_guest(device *device)
+{
+    unsigned int ring_idx;
+
+    for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+    {
+        disconnect_ring(device->ring[ring_idx]);
+        free(device->ring[ring_idx]);
+        device->ring[ring_idx] = NULL;
+    }
+
+    device->num_rings = 0;
+}
+
+static void close_device(device *device, enum xenbus_state state)
+{
+    disconnect_guest(device);
+    write_backend_state(device, state);
+}
+
+static void connect_err(device *device, const char *msg)
+{
+    syslog(LOG_WARNING, "%s", msg);
+    close_device(device, XenbusStateClosed);
+}
+
+static void connect_device(device *device)
+{
+    unsigned int val;
+    unsigned int ring_idx;
+    char node[20];
+    struct ring *ring;
+    xenevtchn_port_or_error_t evtchn;
+
+    val = read_frontend_node_uint(device, "version", 0);
+    if ( val != 1 )
+        return connect_err(device, "frontend specifies illegal version");
+    device->num_rings = read_frontend_node_uint(device, "num-rings", 0);
+    if ( device->num_rings < 1 || device->num_rings > MAX_RINGS )
+        return connect_err(device, "frontend specifies illegal ring number");
+
+    for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+    {
+        ring = calloc(1, sizeof(*ring));
+        if ( !ring )
+            return connect_err(device, "could not allocate ring memory");
+        device->ring[ring_idx] = ring;
+        ring->device = device;
+        pthread_cond_init(&ring->cond, NULL);
+        pthread_mutex_init(&ring->mutex, NULL);
+
+        snprintf(node, sizeof(node), "event-channel-%u", ring_idx);
+        val = read_frontend_node_uint(device, node, 0);
+        if ( val == 0 )
+            return connect_err(device, "frontend specifies illegal evtchn");
+        evtchn = xenevtchn_bind_interdomain(xe, device->domid, val);
+        if ( evtchn < 0 )
+            return connect_err(device, "could not bind to event channel");
+        ring->evtchn = evtchn;
+
+        snprintf(node, sizeof(node), "ring-ref%u", ring_idx);
+        val = read_frontend_node_uint(device, node, 0);
+        if ( val == 0 )
+            return connect_err(device,
+                               "frontend specifies illegal grant for ring");
+        ring->intf = xengnttab_map_grant_ref(xg, device->domid, val,
+                                             PROT_READ | PROT_WRITE);
+        if ( !ring->intf )
+            return connect_err(device, "could not map interface page");
+        ring->ring_order = ring->intf->ring_order;
+        if ( ring->ring_order > MAX_RING_ORDER || ring->ring_order < 1 )
+            return connect_err(device, "frontend specifies illegal ring order");
+        ring->ring_size = XEN_FLEX_RING_SIZE(ring->ring_order);
+        ring->data.in = xengnttab_map_domain_grant_refs(xg,
+                                                        1 << ring->ring_order,
+                                                        device->domid,
+                                                        ring->intf->ref,
+                                                        PROT_READ | PROT_WRITE);
+        if ( !ring->data.in )
+            return connect_err(device, "could not map ring pages");
+        ring->data.out = ring->data.in + ring->ring_size;
+
+        if ( pthread_create(&ring->thread, NULL, io_thread, ring) )
+            return connect_err(device, "could not start I/O thread");
+        ring->thread_active = true;
+    }
+
+    write_backend_state(device, XenbusStateConnected);
+}
+
+static void remove_device(device *device)
+{
+    XEN_TAILQ_REMOVE(&devs, device, list);
+
+    disconnect_guest(device);
+    free_device(device);
+}
+
+static void remove_all_devices(void)
+{
+    device *device, *tmp;
+
+    XEN_TAILQ_FOREACH_SAFE( device, &devs, list, tmp )
+        remove_device(device);
+}
+
+static void frontend_changed(device *device)
+{
+    struct path p;
+    char *state, *end;
+    unsigned int len;
+    unsigned long new_state;
+
+    construct_frontend_path(device, "state", &p);
+    state = xs_read(xs, XBT_NULL, p.path, &len);
+    if ( !state )
+    {
+        close_device(device, XenbusStateClosed);
+        return;
+    }
+
+    new_state = strtoul(state, &end, 10);
+    if ( *end || new_state > XenbusStateReconfigured )
+    {
+        syslog(LOG_WARNING, "unknown state \"%s\" seen for device %u/%u", state,
+               device->domid, device->devid);
+        new_state = XenbusStateUnknown;
+    }
+    free(state);
+
+    if ( new_state == device->frontend_state )
+        return;
+
+    switch ( new_state )
+    {
+    case XenbusStateInitialising:
+        break;
+
+    case XenbusStateInitialised:
+        connect_device(device);
+        break;
+
+    case XenbusStateConnected:
+        break;
+
+    case XenbusStateClosing:
+        close_device(device, XenbusStateClosing);
+        break;
+
+    case XenbusStateClosed:
+        close_device(device, XenbusStateClosed);
+        break;
+
+    default:
+        syslog(LOG_WARNING, "not handled frontend state %lu for device %u/%u",
+               new_state, device->domid, device->devid);
+        break;
+    }
+
+    device->frontend_state = new_state;
+}
+
+static void check_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+
+    device = find_device(domid, devid);
+    if ( !device )
+    {
+        device = new_device(domid, devid);
+        if ( !device )
+            return;
+    }
+
+    device->last_seen = now;
+}
+
+static void scan_backend(void)
+{
+    char **doms;
+    unsigned int n_doms, dom;
+    char **devices;
+    unsigned int n_devs, dev;
+    char dom_path[24];
+    unsigned long domid, devid;
+    char *end;
+    device *device, *tmp;
+
+    now++;
+
+    doms = xs_directory(xs, XBT_NULL, "backend/xen_9pfs", &n_doms);
+    if ( doms == NULL )
+        return;
+
+    for ( dom = 0; dom < n_doms; dom++ )
+    {
+        errno = 0;
+        domid = strtoul(doms[dom], &end, 10);
+        if ( errno || *end || domid >= DOMID_FIRST_RESERVED )
+            continue;
+
+        snprintf(dom_path, sizeof(dom_path), "backend/xen_9pfs/%lu", domid);
+        devices = xs_directory(xs, XBT_NULL, dom_path, &n_devs);
+
+        for ( dev = 0; dev < n_devs; dev++ )
+        {
+            errno = 0;
+            devid = strtoul(devices[dev], &end, 10);
+            if ( errno || *end || devid > UINT_MAX )
+                continue;
+
+            check_device(domid, devid);
+        }
+
+        free(devices);
+    }
+
+    free(doms);
+
+    XEN_TAILQ_FOREACH_SAFE( device, &devs, list, tmp )
+    {
+        if ( device->last_seen != now )
+            remove_device(device);
+    }
+}
+
+static void handle_watch(char *path, char *token)
+{
+    unsigned int domid, devid;
+    device *device;
+
+    if ( !strcmp(token, "main") )
+    {
+        scan_backend();
+        return;
+    }
+
+    if ( sscanf(token, "%u/%u", &domid, &devid) != 2 )
+    {
+        syslog(LOG_WARNING, "unknown watch event %s %s", path, token);
+        return;
+    }
+
+    device = find_device(domid, devid);
+    if ( !device )
+    {
+        syslog(LOG_WARNING, "watch event for unknown device %u/%u",
+               domid, devid);
+        return;
+    }
+
+    frontend_changed(device);
+}
+
 static void close_all(void)
 {
     if ( daemon_running )
+    {
         xs_rm(xs, XBT_NULL, "libxl/xen-9pfs");
+        xs_unwatch(xs, "backend/xen_9pfs", "main");
+
+        remove_all_devices();
+    }
     if ( xe )
         xenevtchn_close(xe);
     if ( xg )
@@ -68,6 +666,33 @@ static void do_err(const char *msg)
     exit(1);
 }
 
+static void handle_event(void)
+{
+    xenevtchn_port_or_error_t evtchn;
+    device *device;
+    struct ring *ring;
+    unsigned int ring_idx;
+
+    evtchn = xenevtchn_pending(xe);
+    if ( evtchn < 0 )
+        do_err("xenevtchn_pending() failed");
+
+    XEN_TAILQ_FOREACH( device, &devs, list )
+    {
+        for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+        {
+            ring = device->ring[ring_idx];
+            if ( ring && ring->evtchn == evtchn )
+            {
+                pthread_mutex_lock(&ring->mutex);
+                pthread_cond_signal(&ring->cond);
+                pthread_mutex_unlock(&ring->mutex);
+                return;
+            }
+        }
+    }
+}
+
 static void xen_connect(void)
 {
     xs_transaction_t t;
@@ -124,6 +749,11 @@ int main(int argc, char *argv[])
     int syslog_mask = LOG_MASK(LOG_WARNING) | LOG_MASK(LOG_ERR) |
                       LOG_MASK(LOG_CRIT) | LOG_MASK(LOG_ALERT) |
                       LOG_MASK(LOG_EMERG);
+    char **watch;
+    struct pollfd p[2] = {
+        { .events = POLLIN },
+        { .events = POLLIN }
+    };
 
     umask(027);
     if ( getenv("XEN_9PFSD_VERBOSE") )
@@ -136,9 +766,26 @@ int main(int argc, char *argv[])
 
     xen_connect();
 
+    if ( !xs_watch(xs, "backend/xen_9pfs", "main") )
+        do_err("xs_watch() in main thread failed");
+    p[0].fd = xs_fileno(xs);
+    p[1].fd = xenevtchn_fd(xe);
+
+    scan_backend();
+
     while ( !stop_me )
     {
-        sleep(60);
+        while ( (p[0].revents & POLLIN) &&
+                (watch = xs_check_watch(xs)) != NULL )
+        {
+            handle_watch(watch[XS_WATCH_PATH], watch[XS_WATCH_TOKEN]);
+            free(watch);
+        }
+
+        if ( p[1].revents & POLLIN )
+            handle_event();
+
+        poll(p, 2, -1);
     }
 
     close_all();
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
new file mode 100644
index 0000000000..ff3562164a
--- /dev/null
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef XEN_9PFSD_H
+#define XEN_9PFSD_H
+
+#include <pthread.h>
+#include <stdbool.h>
+#include <xenevtchn.h>
+#include <xen_list.h>
+#include <xen/xen.h>
+#include <xen/io/xenbus.h>
+#include <xen/io/9pfs.h>
+
+#define MAX_RINGS                4
+#define MAX_RING_ORDER           9
+#define MAX_OPEN_FILES_DEFAULT   5
+
+typedef struct device device;
+
+struct ring {
+    device *device;
+    pthread_t thread;
+    bool thread_active;
+    bool stop_thread;
+    pthread_cond_t cond;
+    pthread_mutex_t mutex;
+
+    evtchn_port_t evtchn;
+    struct xen_9pfs_data_intf *intf;
+    unsigned int ring_order;
+    RING_IDX ring_size;
+    struct xen_9pfs_data data;
+};
+
+struct device {
+    /* Admin data. */
+    XEN_TAILQ_ENTRY(device) list;
+    unsigned int last_seen;    /* Set in scan_backend(). */
+    unsigned int domid;
+    unsigned int devid;
+
+    /* Tool side configuration data. */
+    char *host_path;
+    unsigned int max_space;
+    unsigned int max_files;
+    unsigned int max_open_files;
+    bool auto_delete;
+
+    /* Connection data. */
+    enum xenbus_state backend_state;
+    enum xenbus_state frontend_state;
+    unsigned int num_rings;
+    struct ring *ring[MAX_RINGS];
+    int root_fd;
+};
+
+extern xenevtchn_handle *xe;
+
+void *io_thread(void *arg);
+
+#endif /* XEN_9PFSD_H */
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:56:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:56:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678354.1055560 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7h9-0006yj-HX; Thu, 08 Feb 2024 16:56:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678354.1055560; Thu, 08 Feb 2024 16:56:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7h9-0006yV-EK; Thu, 08 Feb 2024 16:56:15 +0000
Received: by outflank-mailman (input) for mailman id 678354;
 Thu, 08 Feb 2024 16:56:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7h7-0005q0-UV
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:13 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f68c5f73-c6a2-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:56:12 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id BBBBB21F60;
 Thu,  8 Feb 2024 16:56:11 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8D4591326D;
 Thu,  8 Feb 2024 16:56:11 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id I4kWIasHxWXHEwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f68c5f73-c6a2-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 04/22] tools/9pfsd: add transport layer
Date: Thu,  8 Feb 2024 17:55:28 +0100
Message-Id: <20240208165546.5715-5-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: BBBBB21F60
X-Spam-Flag: NO

Add the transport layer of 9pfs. This is basically the infrastructure
to receive requests from the frontend and to send the related answers
via the rings.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- rename put_request_bytes() (Jason Andryuk)
- rename get_request_bytes() and put_response_bytes() len parameter
  (Jason Andryuk)
- don't unmask event channel if error indicator is set (Jason Andryuk)
---
 tools/9pfsd/io.c        | 143 +++++++++++++++++++++++++++++++++++++++-
 tools/9pfsd/xen-9pfsd.h |  16 +++++
 2 files changed, 156 insertions(+), 3 deletions(-)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index eb7c136e09..4312a62dfe 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -6,39 +6,176 @@
  * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
  *
  * I/O thread handling.
+ *
+ * Only handle one request at a time, pushing out the complete response
+ * before looking for the next request.
  */
 
 #include <stdbool.h>
+#include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <xenctrl.h>           /* For cpu barriers. */
+#include <xen-tools/common-macros.h>
 
 #include "xen-9pfsd.h"
 
+/*
+ * Note that the ring names "in" and "out" are from the frontend's
+ * perspective, so the "in" ring will be used for responses to the frontend,
+ * while the "out" ring is used for requests from the frontend to the
+ * backend.
+ */
+static unsigned int ring_in_free(struct ring *ring)
+{
+    unsigned int queued;
+
+    queued = xen_9pfs_queued(ring->prod_pvt_in, ring->intf->in_cons,
+                             ring->ring_size);
+    xen_rmb();
+
+    return ring->ring_size - queued;
+}
+
+static unsigned int ring_out_data(struct ring *ring)
+{
+    unsigned int queued;
+
+    queued = xen_9pfs_queued(ring->intf->out_prod, ring->cons_pvt_out,
+                             ring->ring_size);
+    xen_rmb();
+
+    return queued;
+}
+
+static unsigned int get_request_bytes(struct ring *ring, unsigned int off,
+                                      unsigned int total_len)
+{
+    unsigned int size;
+    unsigned int out_data = ring_out_data(ring);
+    RING_IDX prod, cons;
+
+    size = min(total_len - off, out_data);
+    prod = xen_9pfs_mask(ring->intf->out_prod, ring->ring_size);
+    cons = xen_9pfs_mask(ring->cons_pvt_out, ring->ring_size);
+    xen_9pfs_read_packet(ring->buffer + off, ring->data.out, size,
+                         prod, &cons, ring->ring_size);
+
+    xen_rmb();           /* Read data out before setting visible consumer. */
+    ring->cons_pvt_out += size;
+    ring->intf->out_cons = ring->cons_pvt_out;
+
+    /* Signal that more space is available now. */
+    xenevtchn_notify(xe, ring->evtchn);
+
+    return size;
+}
+
+static unsigned int put_response_bytes(struct ring *ring, unsigned int off,
+                                       unsigned int total_len)
+{
+    unsigned int size;
+    unsigned int in_data = ring_in_free(ring);
+    RING_IDX prod, cons;
+
+    size = min(total_len - off, in_data);
+    prod = xen_9pfs_mask(ring->prod_pvt_in, ring->ring_size);
+    cons = xen_9pfs_mask(ring->intf->in_cons, ring->ring_size);
+    xen_9pfs_write_packet(ring->data.in, ring->buffer + off, size,
+                          &prod, cons, ring->ring_size);
+
+    xen_wmb();           /* Write data out before setting visible producer. */
+    ring->prod_pvt_in += size;
+    ring->intf->in_prod = ring->prod_pvt_in;
+
+    return size;
+}
+
 static bool io_work_pending(struct ring *ring)
 {
     if ( ring->stop_thread )
         return true;
-    return false;
+    if ( ring->error )
+        return false;
+    return ring->handle_response ? ring_in_free(ring) : ring_out_data(ring);
 }
 
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
+    unsigned int count = 0;
+    struct p9_header hdr;
+    bool in_hdr = true;
+
+    ring->max_size = ring->ring_size;
+    ring->buffer = malloc(ring->max_size);
+    if ( !ring->buffer )
+    {
+        syslog(LOG_CRIT, "memory allocation failure!");
+        return NULL;
+    }
 
     while ( !ring->stop_thread )
     {
         pthread_mutex_lock(&ring->mutex);
         if ( !io_work_pending(ring) )
         {
-            if ( xenevtchn_unmask(xe, ring->evtchn) < 0 )
+            if ( !ring->error && xenevtchn_unmask(xe, ring->evtchn) < 0 )
                 syslog(LOG_WARNING, "xenevtchn_unmask() failed");
             pthread_cond_wait(&ring->cond, &ring->mutex);
         }
         pthread_mutex_unlock(&ring->mutex);
 
-        /* TODO: I/O handling. */
+        if ( ring->stop_thread || ring->error )
+            continue;
+
+        if ( !ring->handle_response )
+        {
+            if ( in_hdr )
+            {
+                count += get_request_bytes(ring, count, sizeof(hdr));
+                if ( count != sizeof(hdr) )
+                    continue;
+                hdr = *(struct p9_header *)ring->buffer;
+                if ( hdr.size > ring->max_size || hdr.size < sizeof(hdr) )
+                {
+                    syslog(LOG_ERR, "%u.%u specified illegal request length %u",
+                           ring->device->domid, ring->device->devid, hdr.size);
+                    ring->error = true;
+                    continue;
+                }
+                in_hdr = false;
+            }
+
+            count += get_request_bytes(ring, count, hdr.size);
+            if ( count < hdr.size )
+                continue;
+
+            /* TODO: handle request (will rewrite hdr.size). */
+
+            ring->handle_response = true;
+            hdr.size = ((struct p9_header *)ring->buffer)->size;
+            count = 0;
+        }
+
+        if ( ring->handle_response )
+        {
+            count += put_response_bytes(ring, count, hdr.size);
+
+            if ( count == hdr.size )
+            {
+                /* Signal presence of response. */
+                xenevtchn_notify(xe, ring->evtchn);
+
+                ring->handle_response = false;
+                in_hdr = true;
+                count = 0;
+            }
+        }
     }
 
+    free(ring->buffer);
+
     ring->thread_active = false;
 
     return NULL;
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index ff3562164a..d587f59a32 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -15,6 +15,12 @@
 #define MAX_RING_ORDER           9
 #define MAX_OPEN_FILES_DEFAULT   5
 
+struct p9_header {
+    uint32_t size;
+    uint8_t cmd;
+    uint16_t tag;
+} __attribute__((packed));
+
 typedef struct device device;
 
 struct ring {
@@ -29,7 +35,17 @@ struct ring {
     struct xen_9pfs_data_intf *intf;
     unsigned int ring_order;
     RING_IDX ring_size;
+
+    /* Transport layer data. */
     struct xen_9pfs_data data;
+    RING_IDX prod_pvt_in;
+    RING_IDX cons_pvt_out;
+
+    /* Request and response handling. */
+    uint32_t max_size;
+    bool error;             /* Protocol error - stop processing. */
+    bool handle_response;   /* Main loop now handling response. */
+    void *buffer;           /* Request/response buffer. */
 };
 
 struct device {
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:56:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:56:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678356.1055570 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7hE-0007RT-RD; Thu, 08 Feb 2024 16:56:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678356.1055570; Thu, 08 Feb 2024 16:56:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7hE-0007Qb-NC; Thu, 08 Feb 2024 16:56:20 +0000
Received: by outflank-mailman (input) for mailman id 678356;
 Thu, 08 Feb 2024 16:56:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7hD-0005q0-Nu
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:19 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f9ff0e2f-c6a2-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:56:17 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 52F4C1FCF6;
 Thu,  8 Feb 2024 16:56:17 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2596E1326D;
 Thu,  8 Feb 2024 16:56:17 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 2hjGB7EHxWXPEwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9ff0e2f-c6a2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411377; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=O73f+E3GNfH4W5rlXCuJYVhh6QCuB0n/PfhAGghm1+4=;
	b=nkZm9m8QVWUfkfUTsQw6NwlKpn2n5Cn6EF90D/G1d6w4ct6KfLbLI4jYOPxlo8ZiTw6ANU
	nA5DJjE84Mlnt0o2CkMZZzE61hvvBPxD3clbGzYEyi4+Cv/VHEZrb0Hj/71urEOPkHhyiB
	hN+fMPzkeZQvTPYQ2XjxXu/7rCDM4RI=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411377; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=O73f+E3GNfH4W5rlXCuJYVhh6QCuB0n/PfhAGghm1+4=;
	b=nkZm9m8QVWUfkfUTsQw6NwlKpn2n5Cn6EF90D/G1d6w4ct6KfLbLI4jYOPxlo8ZiTw6ANU
	nA5DJjE84Mlnt0o2CkMZZzE61hvvBPxD3clbGzYEyi4+Cv/VHEZrb0Hj/71urEOPkHhyiB
	hN+fMPzkeZQvTPYQ2XjxXu/7rCDM4RI=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 05/22] tools/9pfsd: add 9pfs response generation support
Date: Thu,  8 Feb 2024 17:55:29 +0100
Message-Id: <20240208165546.5715-6-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add support for generation a 9pfs protocol response via a format based
approach.

Strings are stored in a per device string buffer and they are
referenced via their offset in this buffer. This allows to avoid
having to dynamically allocate memory for each single string.

As a first user of the response handling add a generic p9_error()
function which will be used to return any error to the client.

Add all format parsing variants in order to avoid additional code churn
later when adding the users of those variants. Prepare a special case
for the "read" case already (format character 'D'): in order to avoid
adding another buffer for read data support doing the read I/O directly
into the response buffer.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- check parameter size limits (Jason Andryuk)
V3:
- use new unaligned access macros (Jason Andryuk)
V4:
- use recursion in fill_buffer() as a preparation for reading dirs
---
 tools/9pfsd/io.c        | 217 +++++++++++++++++++++++++++++++++++++++-
 tools/9pfsd/xen-9pfsd.h |   3 +
 2 files changed, 219 insertions(+), 1 deletion(-)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 4312a62dfe..4a44c70c4d 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -11,6 +11,7 @@
  * before looking for the next request.
  */
 
+#include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
@@ -20,6 +21,16 @@
 
 #include "xen-9pfsd.h"
 
+/* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
+#define P9_CMD_ERROR      107
+
+struct p9_qid {
+    uint8_t type;
+#define QID_TYPE_DIR      0x80
+    uint32_t version;
+    uint64_t path;
+};
+
 /*
  * Note that the ring names "in" and "out" are from the frontend's
  * perspective, so the "in" ring will be used for responses to the frontend,
@@ -100,6 +111,200 @@ static bool io_work_pending(struct ring *ring)
     return ring->handle_response ? ring_in_free(ring) : ring_out_data(ring);
 }
 
+static void fmt_err(const char *fmt)
+{
+    syslog(LOG_CRIT, "illegal format %s passed to fill_buffer()", fmt);
+    exit(1);
+}
+
+/*
+ * Fill buffer with response data.
+ * fmt is a sequence of format characters. Supported characters are:
+ * a: an array (2 bytes number of elements + the following format as elements)
+ *    The number of elements is passed in the first unsigned int parameter, the
+ *    next parameter is a pointer to an array of elements as denoted by the next
+ *    format character.
+ * b: 1 byte unsigned integer
+ * u: 2 byte unsigned integer
+ *    The parameter is a pointer to a uint16_t value
+ * D: Data blob (4 byte length + <length> bytes)
+ *    2 parameters are consumed, first an unsigned int for the length, then a
+ *    pointer to the first uint8_t value.
+ *    No array support.
+ * L: 8 byte unsigned integer
+ *    The parameter is a pointer to a uint64_t value
+ * Q: Qid (struct p9_qid)
+ * S: String (2 byte length + <length> characters)
+ *    The length is obtained via strlen() of the parameter, being a pointer
+ *    to the first character of the string
+ * U: 4 byte unsigned integer
+ *    The parameter is a pointer to a uint32_t value
+ */
+static void fill_buffer_at(void **data, const char *fmt, ...);
+static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
+{
+    const char *f;
+    const void *par;
+    const char *str_val;
+    const struct p9_qid *qid;
+    unsigned int len;
+    unsigned int array_sz = 0;
+    unsigned int elem_sz = 0;
+
+    for ( f = fmt; *f; f++ )
+    {
+        if ( !array_sz )
+            par = va_arg(ap, const void *);
+        else
+        {
+            par += elem_sz;
+            array_sz--;
+        }
+
+        switch ( *f )
+        {
+        case 'a':
+            f++;
+            if ( !*f || array_sz )
+                fmt_err(fmt);
+            array_sz = *(const unsigned int *)par;
+            if ( array_sz > 0xffff )
+            {
+                syslog(LOG_CRIT, "array size %u in fill_buffer()", array_sz);
+                exit(1);
+            }
+            put_unaligned(array_sz, (uint16_t *)*data);
+            *data += sizeof(uint16_t);
+            par = va_arg(ap, const void *);
+            elem_sz = 0;
+            break;
+
+        case 'b':
+            put_unaligned(*(const uint8_t *)par, (uint8_t *)*data);
+            elem_sz = sizeof(uint8_t);
+            *data += sizeof(uint8_t);
+            break;
+
+        case 'u':
+            put_unaligned(*(const uint16_t *)par, (uint16_t *)*data);
+            elem_sz = sizeof(uint16_t);
+            *data += sizeof(uint16_t);
+            break;
+
+        case 'D':
+            if ( array_sz )
+                fmt_err(fmt);
+            len = *(const unsigned int *)par;
+            put_unaligned(len, (uint32_t *)*data);
+            *data += sizeof(uint32_t);
+            par = va_arg(ap, const void *);
+            if ( *data != par )
+                memcpy(*data, par, len);
+            *data += len;
+            break;
+
+        case 'L':
+            put_unaligned(*(const uint64_t *)par, (uint64_t *)*data);
+            elem_sz = sizeof(uint64_t);
+            *data += sizeof(uint64_t);
+            break;
+
+        case 'Q':
+            qid = par;
+            elem_sz = sizeof(*qid);
+            fill_buffer_at(data, "bUL", &qid->type, &qid->version, &qid->path);
+            break;
+
+        case 'S':
+            str_val = par;
+            elem_sz = sizeof(str_val);
+            len = strlen(str_val);
+            if ( len > 0xffff )
+            {
+                syslog(LOG_CRIT, "string length %u in fill_buffer()", len);
+                exit(1);
+            }
+            put_unaligned(len, (uint16_t *)*data);
+            *data += sizeof(uint16_t);
+            memcpy(*data, str_val, len);
+            *data += len;
+            break;
+
+        case 'U':
+            put_unaligned(*(const uint32_t *)par, (uint32_t *)*data);
+            elem_sz = sizeof(uint32_t);
+            *data += sizeof(uint32_t);
+            break;
+
+        default:
+            fmt_err(fmt);
+        }
+
+        if ( array_sz )
+            f--;
+    }
+}
+
+static void fill_buffer_at(void **data, const char *fmt, ...)
+{
+    va_list ap;
+
+    va_start(ap, fmt);
+    vfill_buffer_at(data, fmt, ap);
+    va_end(ap);
+}
+
+static void fill_buffer(struct ring *ring, uint8_t cmd, uint16_t tag,
+                        const char *fmt, ...)
+{
+    struct p9_header *hdr = ring->buffer;
+    void *data = hdr + 1;
+    va_list ap;
+
+    hdr->cmd = cmd;
+    hdr->tag = tag;
+
+    va_start(ap, fmt);
+    vfill_buffer_at(&data, fmt, ap);
+    va_end(ap);
+
+    hdr->size = data - ring->buffer;
+}
+
+static unsigned int add_string(struct ring *ring, const char *str,
+                               unsigned int len)
+{
+    char *tmp;
+    unsigned int ret;
+
+    if ( ring->str_used + len + 1 > ring->str_size )
+    {
+        tmp = realloc(ring->str, ring->str_used + len + 1);
+        if ( !tmp )
+            return ~0;
+        ring->str = tmp;
+        ring->str_size = ring->str_used + len + 1;
+    }
+
+    ret = ring->str_used;
+    memcpy(ring->str + ret, str, len);
+    ring->str_used += len;
+    ring->str[ring->str_used++] = 0;
+
+    return ret;
+}
+
+static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
+{
+    unsigned int erroff;
+
+    strerror_r(err, ring->buffer, ring->ring_size);
+    erroff = add_string(ring, ring->buffer, strlen(ring->buffer));
+    fill_buffer(ring, P9_CMD_ERROR, tag, "SU",
+                erroff != ~0 ? ring->str + erroff : "cannot allocate memory",
+                &err);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -151,7 +356,16 @@ void *io_thread(void *arg)
             if ( count < hdr.size )
                 continue;
 
-            /* TODO: handle request (will rewrite hdr.size). */
+            ring->str_used = 0;
+
+            switch ( hdr.cmd )
+            {
+            default:
+                syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
+                       ring->device->domid, ring->device->devid, hdr.cmd);
+                p9_error(ring, hdr.tag, EOPNOTSUPP);
+                break;
+            }
 
             ring->handle_response = true;
             hdr.size = ((struct p9_header *)ring->buffer)->size;
@@ -174,6 +388,7 @@ void *io_thread(void *arg)
         }
     }
 
+    free(ring->str);
     free(ring->buffer);
 
     ring->thread_active = false;
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index d587f59a32..0cde0d2bb8 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -46,6 +46,9 @@ struct ring {
     bool error;             /* Protocol error - stop processing. */
     bool handle_response;   /* Main loop now handling response. */
     void *buffer;           /* Request/response buffer. */
+    char *str;              /* String work space. */
+    unsigned int str_size;  /* Size of *str. */
+    unsigned int str_used;  /* Currently used size of *str. */
 };
 
 struct device {
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:56:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:56:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678358.1055580 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7hJ-0007vA-2C; Thu, 08 Feb 2024 16:56:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678358.1055580; Thu, 08 Feb 2024 16:56:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7hI-0007v1-Uu; Thu, 08 Feb 2024 16:56:24 +0000
Received: by outflank-mailman (input) for mailman id 678358;
 Thu, 08 Feb 2024 16:56:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7hH-0005bD-Rl
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:23 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fd331859-c6a2-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:56:23 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id E25D921F60;
 Thu,  8 Feb 2024 16:56:22 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B27D81326D;
 Thu,  8 Feb 2024 16:56:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 82saKrYHxWXVEwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd331859-c6a2-11ee-8a4a-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 06/22] tools/9pfsd: add 9pfs version request support
Date: Thu,  8 Feb 2024 17:55:30 +0100
Message-Id: <20240208165546.5715-7-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: E25D921F60
X-Spam-Flag: NO

Add the version request of the 9pfs protocol. For the version use the
"9P2000.u" variant, as it is supported by Mini-OS and Linux.

For the request parsing add all format items needed even in future in
order to avoid code churn for those additions later.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V3:
- use unaligned helper macros (Jason Andryuk)
---
 tools/9pfsd/io.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 201 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 4a44c70c4d..839dd1112c 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -22,8 +22,12 @@
 #include "xen-9pfsd.h"
 
 /* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
+#define P9_CMD_VERSION    100
 #define P9_CMD_ERROR      107
 
+#define P9_MIN_MSIZE      2048
+#define P9_VERSION        "9P2000.u"
+
 struct p9_qid {
     uint8_t type;
 #define QID_TYPE_DIR      0x80
@@ -294,6 +298,169 @@ static unsigned int add_string(struct ring *ring, const char *str,
     return ret;
 }
 
+static bool chk_data(struct ring *ring, void *data, unsigned int len)
+{
+    struct p9_header *hdr = ring->buffer;
+
+    if ( data + len <= ring->buffer + hdr->size )
+        return true;
+
+    errno = E2BIG;
+
+    return false;
+}
+
+static bool fill_data_elem(void **par, void **array, unsigned int *array_sz,
+                           unsigned int elem_sz, void *data)
+{
+    if ( *array_sz && !*array )
+    {
+        *array = calloc(*array_sz, elem_sz);
+        if ( !*array )
+            return false;
+        *par = *array;
+    }
+
+    memcpy(*par, data, elem_sz);
+
+    if ( *array_sz )
+    {
+        *par += elem_sz;
+        *array_sz -= 1;
+    }
+
+    return true;
+}
+
+/*
+ * Fill variables with request data.
+ * fmt is a sequence of format characters. Supported characters are:
+ * a: an array (2 bytes number of elements + the following format as elements)
+ *    The number of elements is stored in the first unsigned int parameter, the
+ *    next parameter is a pointer to an array of elements as denoted by the next
+ *    format character. The array is allocated dynamically.
+ * b: 1 byte unsigned integer
+ *    The value is stored in the next parameter with type uint8_t.
+ * D: Data blob (4 byte length + <length> bytes)
+ *    2 parameters are consumed, first an unsigned int for the length, then a
+ *    pointer to the first uint8_t value.
+ *    No array support.
+ * L: 8 byte unsigned integer
+ *    The value is stored in the next parameter with type uint64_t.
+ * S: String (2 byte length + <length> characters)
+ *    The 0-terminated string is stored in device->str + off, off is stored in
+ *    the next parameter with type unsigned int.
+ * U: 4 byte unsigned integer
+ *    The value is stored in the next parameter with type uint32_t.
+ *
+ * Return value: number of filled variables, errno will be set in case of
+ *   error.
+ */
+static int fill_data(struct ring *ring, const char *fmt, ...)
+{
+    struct p9_header *hdr = ring->buffer;
+    void *data = hdr + 1;
+    void *par;
+    unsigned int pars = 0;
+    const char *f;
+    va_list ap;
+    unsigned int len;
+    unsigned int str_off;
+    unsigned int array_sz = 0;
+    void **array = NULL;
+
+    va_start(ap, fmt);
+
+    for ( f = fmt; *f; f++ )
+    {
+        if ( !array_sz )
+            par = va_arg(ap, void *);
+
+        switch ( *f )
+        {
+        case 'a':
+            f++;
+            if ( !*f || array_sz )
+                fmt_err(fmt);
+            if ( !chk_data(ring, data, sizeof(uint16_t)) )
+                return pars;
+            array_sz = get_unaligned((uint16_t *)data);
+            data += sizeof(uint16_t);
+            *(unsigned int *)par = array_sz;
+            array = va_arg(ap, void **);
+            *array = NULL;
+            break;
+
+        case 'b':
+            if ( !chk_data(ring, data, sizeof(uint8_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint8_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint8_t);
+            break;
+
+        case 'D':
+            if ( array_sz )
+                fmt_err(fmt);
+            if ( !chk_data(ring, data, sizeof(uint32_t)) )
+                return pars;
+            len = get_unaligned((uint32_t *)data);
+            data += sizeof(uint32_t);
+            *(unsigned int *)par = len;
+            par = va_arg(ap, void *);
+            if ( !chk_data(ring, data, len) )
+                return pars;
+            memcpy(par, data, len);
+            data += len;
+            break;
+
+        case 'L':
+            if ( !chk_data(ring, data, sizeof(uint64_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint64_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint64_t);
+            break;
+
+        case 'S':
+            if ( !chk_data(ring, data, sizeof(uint16_t)) )
+                return pars;
+            len = get_unaligned((uint16_t *)data);
+            data += sizeof(uint16_t);
+            if ( !chk_data(ring, data, len) )
+                return pars;
+            str_off = add_string(ring, data, len);
+            if ( str_off == ~0 )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(unsigned int),
+                                 &str_off) )
+                return pars;
+            data += len;
+            break;
+
+        case 'U':
+            if ( !chk_data(ring, data, sizeof(uint32_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint32_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint32_t);
+            break;
+
+        default:
+            fmt_err(fmt);
+        }
+
+        if ( array_sz )
+            f--;
+        pars++;
+    }
+
+    return pars;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -305,6 +472,36 @@ static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
                 &err);
 }
 
+static void p9_version(struct ring *ring, struct p9_header *hdr)
+{
+    uint32_t max_size;
+    unsigned int off;
+    char *version;
+    int ret;
+
+    ret = fill_data(ring, "US", &max_size, &off);
+    if ( ret != 2 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    if ( max_size < P9_MIN_MSIZE )
+    {
+        p9_error(ring, hdr->tag, EMSGSIZE);
+        return;
+    }
+
+    if ( max_size < ring->max_size )
+        ring->max_size = max_size;
+
+    version = ring->str + off;
+    if ( strcmp(version, P9_VERSION) )
+        version = "unknown";
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "US", &ring->max_size, version);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -360,6 +557,10 @@ void *io_thread(void *arg)
 
             switch ( hdr.cmd )
             {
+            case P9_CMD_VERSION:
+                p9_version(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:56:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:56:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678360.1055590 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7hP-0008TB-Fp; Thu, 08 Feb 2024 16:56:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678360.1055590; Thu, 08 Feb 2024 16:56:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7hP-0008T2-BA; Thu, 08 Feb 2024 16:56:31 +0000
Received: by outflank-mailman (input) for mailman id 678360;
 Thu, 08 Feb 2024 16:56:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7hN-0005bD-RS
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:29 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0088a281-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:56:28 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 7F6C41FCF6;
 Thu,  8 Feb 2024 16:56:28 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4EE3D1326D;
 Thu,  8 Feb 2024 16:56:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id lPLfEbwHxWXdEwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0088a281-c6a3-11ee-8a4a-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 07/22] tools/9pfsd: add 9pfs attach request support
Date: Thu,  8 Feb 2024 17:55:31 +0100
Message-Id: <20240208165546.5715-8-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 7F6C41FCF6
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add the attach request of the 9pfs protocol. This introduces the "fid"
scheme of the 9pfs protocol.

As this will be needed later, use a dedicated memory allocation
function in alloc_fid() and prepare a fid reference count.

For filling the qid data take the approach from the qemu 9pfs backend
implementation.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- make fill_qid() parameter stbuf const (Jason Andryuk)
- free fids after disconnecting guest (Jason Andryuk)
V3:
- only store relative path in fid (Jason Andryuk)
V4:
- store a path directly usable by *at() functions in fid (Jason Andryuk)
---
 tools/9pfsd/io.c        | 164 ++++++++++++++++++++++++++++++++++++++++
 tools/9pfsd/xen-9pfsd.c |   6 ++
 tools/9pfsd/xen-9pfsd.h |  14 ++++
 3 files changed, 184 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 839dd1112c..4f6acefc87 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -11,11 +11,14 @@
  * before looking for the next request.
  */
 
+#include <assert.h>
 #include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <sys/types.h>
+#include <sys/stat.h>
 #include <xenctrl.h>           /* For cpu barriers. */
 #include <xen-tools/common-macros.h>
 
@@ -23,6 +26,7 @@
 
 /* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
 #define P9_CMD_VERSION    100
+#define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
 
 #define P9_MIN_MSIZE      2048
@@ -461,6 +465,126 @@ static int fill_data(struct ring *ring, const char *fmt, ...)
     return pars;
 }
 
+static struct p9_fid *find_fid(device *device, unsigned int fid)
+{
+    struct p9_fid *fidp;
+
+    XEN_TAILQ_FOREACH(fidp, &device->fids, list)
+    {
+        if ( fidp->fid == fid )
+            return fidp;
+    }
+
+    return NULL;
+}
+
+static struct p9_fid *alloc_fid_mem(device *device, unsigned int fid,
+                                    const char *path)
+{
+    struct p9_fid *fidp;
+    size_t pathlen;
+
+    pathlen = strlen(path);
+    fidp = calloc(sizeof(*fidp) + pathlen + 1, 1);
+    if ( !fidp )
+        return NULL;
+
+    fidp->fid = fid;
+    strncpy(fidp->path, path, pathlen);
+
+    return fidp;
+}
+
+static struct p9_fid *alloc_fid(device *device, unsigned int fid,
+                                const char *path)
+{
+    struct p9_fid *fidp = NULL;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    if ( find_fid(device, fid) )
+    {
+        errno = EBADFD;
+        goto out;
+    }
+
+    if ( device->n_fids >= device->max_open_files )
+    {
+        errno = EMFILE;
+        goto out;
+    }
+
+    fidp = alloc_fid_mem(device, fid, path);
+    if ( !fidp )
+        goto out;
+
+    fidp->ref = 1;
+    XEN_TAILQ_INSERT_HEAD(&device->fids, fidp, list);
+    device->n_fids++;
+
+ out:
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    return fidp;
+}
+
+static void free_fid(device *device, struct p9_fid *fidp)
+{
+    if ( !fidp )
+        return;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    fidp->ref--;
+    if ( !fidp->ref )
+    {
+        device->n_fids--;
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        free(fidp);
+    }
+
+    pthread_mutex_unlock(&device->fid_mutex);
+}
+
+void free_fids(device *device)
+{
+    struct p9_fid *fidp;
+
+    while ( (fidp = XEN_TAILQ_FIRST(&device->fids)) != NULL )
+    {
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        free(fidp);
+    }
+}
+
+static const char *relpath_from_path(const char *path)
+{
+    if (!strcmp(path, "/"))
+        return ".";
+
+    return (path[0] == '/') ? path + 1 : path;
+}
+
+static int fill_qid(device *device, const char *path, struct p9_qid *qid,
+                    const struct stat *stbuf)
+{
+    struct stat st;
+
+    if ( !stbuf )
+    {
+        if ( fstatat(device->root_fd, path, &st, 0) )
+            return errno;
+
+        stbuf = &st;
+    }
+
+    qid->type = S_ISDIR(stbuf->st_mode) ? QID_TYPE_DIR : 0;
+    qid->version = stbuf->st_mtime ^ (stbuf->st_size << 8);
+    qid->path = stbuf->st_ino;
+
+    return 0;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -502,6 +626,42 @@ static void p9_version(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "US", &ring->max_size, version);
 }
 
+static void p9_attach(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint32_t dummy_u32;
+    unsigned int dummy_uint;
+    struct p9_qid qid;
+    int ret;
+
+    ret = fill_data(ring, "UUSSU", &fid, &dummy_u32, &dummy_uint, &dummy_uint,
+                    &dummy_u32);
+    if ( ret != 5 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    device->root_fid = alloc_fid(device, fid, relpath_from_path("/"));
+    if ( !device->root_fid )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    ret = fill_qid(device, device->root_fid->path, &qid, NULL);
+    if ( ret )
+    {
+        free_fid(device, device->root_fid);
+        device->root_fid = NULL;
+        p9_error(ring, hdr->tag, ret);
+        return;
+    }
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "Q", &qid);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -561,6 +721,10 @@ void *io_thread(void *arg)
                 p9_version(ring, &hdr);
                 break;
 
+            case P9_CMD_ATTACH:
+                p9_attach(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/9pfsd/xen-9pfsd.c b/tools/9pfsd/xen-9pfsd.c
index 73b6c3a30e..b83b896bcc 100644
--- a/tools/9pfsd/xen-9pfsd.c
+++ b/tools/9pfsd/xen-9pfsd.c
@@ -304,6 +304,9 @@ static device *new_device(unsigned int domid, unsigned int devid)
         return NULL;
     }
 
+    pthread_mutex_init(&device->fid_mutex, NULL);
+    XEN_TAILQ_INIT(&device->fids);
+
     val = read_backend_node(device, "security_model");
     if ( !val || strcmp(val, "none") )
     {
@@ -399,6 +402,8 @@ static void disconnect_guest(device *device)
     }
 
     device->num_rings = 0;
+
+    free_fids(device);
 }
 
 static void close_device(device *device, enum xenbus_state state)
@@ -482,6 +487,7 @@ static void remove_device(device *device)
     XEN_TAILQ_REMOVE(&devs, device, list);
 
     disconnect_guest(device);
+    pthread_mutex_destroy(&device->fid_mutex);
     free_device(device);
 }
 
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index 0cde0d2bb8..6c755411a7 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -21,6 +21,13 @@ struct p9_header {
     uint16_t tag;
 } __attribute__((packed));
 
+struct p9_fid {
+    XEN_TAILQ_ENTRY(struct p9_fid) list;
+    unsigned int fid;
+    unsigned int ref;
+    char path[];
+};
+
 typedef struct device device;
 
 struct ring {
@@ -71,10 +78,17 @@ struct device {
     unsigned int num_rings;
     struct ring *ring[MAX_RINGS];
     int root_fd;
+
+    /* File system handling. */
+    pthread_mutex_t fid_mutex;
+    XEN_TAILQ_HEAD(fidhead, struct p9_fid) fids;
+    struct p9_fid *root_fid;
+    unsigned int n_fids;
 };
 
 extern xenevtchn_handle *xe;
 
 void *io_thread(void *arg);
+void free_fids(device *device);
 
 #endif /* XEN_9PFSD_H */
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:56:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:56:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678364.1055600 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7hU-0000XE-NE; Thu, 08 Feb 2024 16:56:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678364.1055600; Thu, 08 Feb 2024 16:56:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7hU-0000X0-Ih; Thu, 08 Feb 2024 16:56:36 +0000
Received: by outflank-mailman (input) for mailman id 678364;
 Thu, 08 Feb 2024 16:56:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7hT-0005bD-0Q
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:35 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 03dbfd60-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:56:34 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 17FDD21F60;
 Thu,  8 Feb 2024 16:56:34 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DC9D11326D;
 Thu,  8 Feb 2024 16:56:33 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id r7tcNMEHxWXlEwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03dbfd60-c6a3-11ee-8a4a-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 08/22] tools/9pfsd: add 9pfs walk request support
Date: Thu,  8 Feb 2024 17:55:32 +0100
Message-Id: <20240208165546.5715-9-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 17FDD21F60
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add the walk request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- don't allow walking across symbolic links
---
 tools/9pfsd/io.c        | 171 ++++++++++++++++++++++++++++++++++++++++
 tools/9pfsd/xen-9pfsd.h |   1 +
 2 files changed, 172 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 4f6acefc87..5b827706b7 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -28,9 +28,11 @@
 #define P9_CMD_VERSION    100
 #define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
+#define P9_CMD_WALK       110
 
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
+#define P9_WALK_MAXELEM   16
 
 struct p9_qid {
     uint8_t type;
@@ -478,6 +480,21 @@ static struct p9_fid *find_fid(device *device, unsigned int fid)
     return NULL;
 }
 
+static struct p9_fid *get_fid_ref(device *device, unsigned int fid)
+{
+    struct p9_fid *fidp;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    fidp = find_fid(device, fid);
+    if ( fidp )
+        fidp->ref++;
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    return fidp;
+}
+
 static struct p9_fid *alloc_fid_mem(device *device, unsigned int fid,
                                     const char *path)
 {
@@ -578,6 +595,10 @@ static int fill_qid(device *device, const char *path, struct p9_qid *qid,
         stbuf = &st;
     }
 
+    /* Don't allow symbolic links. */
+    if ( S_ISLNK(stbuf->st_mode) )
+        return EMLINK;
+
     qid->type = S_ISDIR(stbuf->st_mode) ? QID_TYPE_DIR : 0;
     qid->version = stbuf->st_mtime ^ (stbuf->st_size << 8);
     qid->path = stbuf->st_ino;
@@ -585,6 +606,20 @@ static int fill_qid(device *device, const char *path, struct p9_qid *qid,
     return 0;
 }
 
+static bool name_ok(const char *str)
+{
+    if ( !*str )
+        return false;
+
+    if ( strchr(str, '/' ) )
+        return false;
+
+    if ( !strcmp(str, "..") || !strcmp(str, ".") )
+        return false;
+
+    return true;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -662,6 +697,138 @@ static void p9_attach(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "Q", &qid);
 }
 
+static void p9_walk(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint32_t newfid;
+    struct p9_fid *fidp = NULL;
+    struct p9_qid *qids = NULL;
+    unsigned int n_names = 0;
+    unsigned int *names = NULL;
+    unsigned int walked = 0;
+    unsigned int i;
+    char *path = NULL;
+    unsigned int path_len;
+    int ret;
+
+    ret = fill_data(ring, "UUaS", &fid, &newfid, &n_names, &names);
+    if ( n_names > P9_WALK_MAXELEM )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        goto out;
+    }
+    if ( ret != 3 + n_names )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        goto out;
+    }
+    if ( fidp->opened )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        goto out;
+    }
+
+    path_len = strlen(fidp->path) + 1;
+    for ( i = 0; i < n_names; i++ )
+    {
+        if ( !name_ok(ring->str + names[i]) )
+        {
+            p9_error(ring, hdr->tag, ENOENT);
+            goto out;
+        }
+        path_len += strlen(ring->str + names[i]) + 1;
+    }
+    path = calloc(path_len + 1, 1);
+    if ( !path )
+    {
+        p9_error(ring, hdr->tag, ENOMEM);
+        goto out;
+    }
+    strcpy(path, fidp->path);
+
+    if ( n_names )
+    {
+        qids = calloc(n_names, sizeof(*qids));
+        if ( !qids )
+        {
+            p9_error(ring, hdr->tag, ENOMEM);
+            goto out;
+        }
+        for ( i = 0; i < n_names; i++ )
+        {
+            strcat(path, "/");
+            strcat(path, ring->str + names[i]);
+            ret = fill_qid(device, path, qids + i, NULL);
+            if ( ret )
+            {
+                if ( !walked )
+                {
+                    p9_error(ring, hdr->tag, errno);
+                    goto out;
+                }
+                break;
+            }
+            walked++;
+        }
+    }
+
+    if ( walked == n_names )
+    {
+        bool ok = false;
+
+        if ( fid == newfid )
+        {
+            struct p9_fid *new_fidp;
+
+            pthread_mutex_lock(&device->fid_mutex);
+
+            if ( fidp->ref != 2 )
+            {
+                errno = EBUSY;
+            }
+            else
+            {
+                new_fidp = alloc_fid_mem(device, fid, path);
+                if ( new_fidp )
+                {
+                    new_fidp->ref = 2;
+                    XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+                    XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+                    free(fidp);
+                    fidp = new_fidp;
+                    ok = true;
+                }
+            }
+
+            pthread_mutex_unlock(&device->fid_mutex);
+        }
+        else
+            ok = alloc_fid(device, newfid, path);
+
+        if ( !ok )
+        {
+            p9_error(ring, hdr->tag, errno);
+            goto out;
+        }
+    }
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "aQ", &walked, qids);
+
+ out:
+    free_fid(device, fidp);
+    free(qids);
+    free(path);
+    free(names);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -725,6 +892,10 @@ void *io_thread(void *arg)
                 p9_attach(ring, &hdr);
                 break;
 
+            case P9_CMD_WALK:
+                p9_walk(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index 6c755411a7..f01fffb0bb 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -25,6 +25,7 @@ struct p9_fid {
     XEN_TAILQ_ENTRY(struct p9_fid) list;
     unsigned int fid;
     unsigned int ref;
+    bool opened;
     char path[];
 };
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:57:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:57:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678368.1055609 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7i2-0001fC-Uz; Thu, 08 Feb 2024 16:57:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678368.1055609; Thu, 08 Feb 2024 16:57:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7i2-0001f5-Ro; Thu, 08 Feb 2024 16:57:10 +0000
Received: by outflank-mailman (input) for mailman id 678368;
 Thu, 08 Feb 2024 16:57:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7i1-0005q0-L6
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:57:09 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 17d8a5dd-c6a3-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:57:07 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 987D11FCF6;
 Thu,  8 Feb 2024 16:57:07 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6A19C1326D;
 Thu,  8 Feb 2024 16:57:07 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id rdaDGOMHxWUvFAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:57:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17d8a5dd-c6a3-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 14/22] tools/9pfsd: add 9pfs read request support
Date: Thu,  8 Feb 2024 17:55:38 +0100
Message-Id: <20240208165546.5715-15-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 987D11FCF6
X-Spam-Flag: NO

Add the read request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- make error check more readable (Jason Andryuk)
V4:
- add directory read support
V5:
- rewinddir() if reading a directory and offset is 0 (Jason Andryuk)
---
 tools/9pfsd/io.c | 93 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 93 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index b763e3d8d9..df4f09356e 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -33,6 +33,7 @@
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
+#define P9_CMD_READ       116
 #define P9_CMD_WRITE      118
 #define P9_CMD_CLUNK      120
 #define P9_CMD_STAT       124
@@ -1247,6 +1248,94 @@ static void p9_stat(struct ring *ring, struct p9_header *hdr)
     free_fid(device, fidp);
 }
 
+static void p9_read(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint64_t off;
+    unsigned int len;
+    uint32_t count;
+    void *buf;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "ULU", &fid, &off, &count);
+    if ( ret != 3 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || !fidp->opened )
+    {
+        errno = EBADF;
+        goto err;
+    }
+
+    len = count;
+    buf = ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
+
+    if ( fidp->isdir )
+    {
+        struct dirent *dirent;
+        struct stat st;
+        struct p9_stat p9s;
+
+        if ( off == 0 )
+            rewinddir(fidp->data);
+
+        while ( len != 0 )
+        {
+            errno = 0;
+            dirent = readdir(fidp->data);
+            if ( !dirent )
+            {
+                if ( errno )
+                    goto err;
+                break;
+            }
+            if ( fstatat(fidp->fd, dirent->d_name, &st, 0) < 0 )
+                goto err;
+            fill_p9_stat(device, &p9s, &st, dirent->d_name);
+            if ( p9s.size + sizeof(p9s.size) > len )
+            {
+                seekdir(fidp->data, dirent->d_off);
+                break;
+            }
+            fill_buffer_at(&buf, "s", &p9s);
+            len -= p9s.size + sizeof(p9s.size);
+        }
+    }
+    else
+    {
+        while ( len != 0 )
+        {
+            ret = pread(fidp->fd, buf, len, off);
+            if ( ret <= 0 )
+                break;
+            len -= ret;
+            buf += ret;
+            off += ret;
+        }
+        if ( ret < 0 && len == count )
+            goto err;
+    }
+
+    buf = ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
+    len = count - len;
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "D", &len, buf);
+
+ out:
+    free_fid(device, fidp);
+
+    return;
+
+ err:
+    p9_error(ring, hdr->tag, errno);
+    goto out;
+}
+
 static void p9_write(struct ring *ring, struct p9_header *hdr)
 {
     device *device = ring->device;
@@ -1371,6 +1460,10 @@ void *io_thread(void *arg)
                 p9_create(ring, &hdr);
                 break;
 
+            case P9_CMD_READ:
+                p9_read(ring, &hdr);
+                break;
+
             case P9_CMD_WRITE:
                 p9_write(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:57:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:57:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678369.1055620 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7iE-00021A-5t; Thu, 08 Feb 2024 16:57:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678369.1055620; Thu, 08 Feb 2024 16:57:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7iE-000213-2E; Thu, 08 Feb 2024 16:57:22 +0000
Received: by outflank-mailman (input) for mailman id 678369;
 Thu, 08 Feb 2024 16:57:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7iD-0005q0-3S
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:57:21 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1eae7c53-c6a3-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:57:19 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 01B0221F60;
 Thu,  8 Feb 2024 16:57:19 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id C63B81326D;
 Thu,  8 Feb 2024 16:57:18 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id SIjwLu4HxWU5FAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:57:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1eae7c53-c6a3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411439; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=h0jxnSPLZTXt0UgW8EvhU376ps4C4d/D3zwFe+T6tRE=;
	b=GE9X7FRWIuvvlCILXlzc90RQS7BX1rtp/k4OCAxHYCNnLLIQdC3sL3+WrFGkM2Jlw1lRTl
	0REiy96noWlw/hcEj6AKSdxm4ojn9dPgOt0NX6kFfxM2RoPvalu+0PiLVg2x8u1hBDj6K7
	ctKIkwWjxlc1qyhBDmMErjDK+Xxpvus=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411439; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=h0jxnSPLZTXt0UgW8EvhU376ps4C4d/D3zwFe+T6tRE=;
	b=GE9X7FRWIuvvlCILXlzc90RQS7BX1rtp/k4OCAxHYCNnLLIQdC3sL3+WrFGkM2Jlw1lRTl
	0REiy96noWlw/hcEj6AKSdxm4ojn9dPgOt0NX6kFfxM2RoPvalu+0PiLVg2x8u1hBDj6K7
	ctKIkwWjxlc1qyhBDmMErjDK+Xxpvus=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 16/22] tools/xl: support new 9pfs backend xen_9pfsd
Date: Thu,  8 Feb 2024 17:55:40 +0100
Message-Id: <20240208165546.5715-17-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add support for the new 9pfs backend "xen_9pfsd". For this backend type
the tag defaults to "Xen" and the host side path to
"/var/log/xen/guests/<dom-name>".

Do most of the default settings in libxl. Unfortunately the default
path can't easily be set in libxl, as the domain name isn't available
in the related 9pfs specific function.

Settings the defaults in libxl requires to move the sanity checking
of 9pfs parameters from xl to libxl, too.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- test max_files and max_open_files, too (Jason Andryuk)
V4:
- fix man page to use the "xen_9pfsd" type due to idl limitation
  (Jason Andryuk)
- set (most of) the defaults in libxl (Anthony Perard)
---
 docs/man/xl.cfg.5.pod.in      | 36 +++++++++++++++++++++++++++++++++--
 tools/libs/light/libxl_9pfs.c | 18 ++++++++++++++++++
 tools/xl/xl_parse.c           | 23 +++++++++++++++++++---
 3 files changed, 72 insertions(+), 5 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index ea8d41727d..039e057318 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -772,10 +772,16 @@ settings, from the following list:
 
 =over 4
 
+=item B<type=TYPE>
+
+The backendtype for the PV device. Supported values are B<qemu> and
+B<xen_9pfsd>.  The default is B<qemu>.
+
 =item B<tag=STRING>
 
 9pfs tag to identify the filesystem share. The tag is needed on the
-guest side to mount it.
+guest side to mount it. For the backendtype of B<xen_9pfsd> the tag defaults to
+"Xen".
 
 =item B<security_model="none">
 
@@ -785,12 +791,38 @@ squash or remap).
 
 =item B<path=STRING>
 
-Filesystem path on the backend to export.
+Filesystem path on the backend to export. For the backendtype of B<xen_9pfsd>
+the path defaults to "@XEN_LOG_DIR@/guests/<guest-name>".
 
 =item B<backend=domain-id>
 
 Specify the backend domain name or id, defaults to dom0.
 
+=item B<max-files=NUMBER>
+
+Specify the maximum number of files below B<path>. A value of 0 (which
+is the default) doesn't limit the number of files. Only valid for
+B<type=xen_9pfsd>.
+
+=item B<max-open-files=NUMBER>
+
+Specify the maximum number of concurrently opened files below B<path>.
+Multiple opens of the same file are counted individually. Only valid for
+B<type=xen_9pfsd>, which has a default of B<max-open-files=5>.
+
+=item B<max-space=NUMBER>
+
+Specify the maximum used disk space in MiB below B<path>. A value of 0 (which
+is the default) doesn't limit the usable disk space. Only valid for
+B<type=xen_9pfsd>.
+
+=item B<auto-delete=BOOLEAN>
+
+When set the backend will delete the oldest file which is currently not
+opened by the guest in case the disk space limit set via B<max-space> or the
+file limit set via B<max-files> is being reached. Only valid for
+B<type=xen_9pfsd>.
+
 =back
 
 =item B<pvcalls=[ "backend=domain-id", ... ]>
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 68fa9947ad..7593686a90 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -20,6 +20,24 @@
 static int libxl__device_p9_setdefault(libxl__gc *gc, uint32_t domid,
                                        libxl_device_p9 *p9, bool hotplug)
 {
+    if (p9->type == LIBXL_P9_TYPE_UNKNOWN) {
+        p9->type = LIBXL_P9_TYPE_QEMU;
+    }
+    if (p9->type == LIBXL_P9_TYPE_QEMU &&
+        (p9->max_files || p9->max_open_files || p9->max_space ||
+         p9->auto_delete)) {
+        LOGD(ERROR, domid, "Illegal 9pfs parameter combination");
+        return ERROR_INVAL;
+    }
+    if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD && !p9->tag) {
+        p9->tag = libxl__strdup(NOGC, "Xen");
+    }
+
+    if (!p9->path || !p9->security_model || !p9->tag) {
+        LOGD(ERROR, domid, "9pfs spec missing required field!");
+        return ERROR_INVAL;
+    }
+
     return libxl__resolve_domid(gc, p9->backend_domname, &p9->backend_domid);
 }
 
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 9b358f11b8..80ffe85f5e 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2233,6 +2233,20 @@ void parse_config_data(const char *config_source,
                     replace_string(&p9->tag, value);
                 } else if (!strcmp(key, "backend")) {
                     replace_string(&p9->backend_domname, value);
+                } else if (!strcmp(key, "type")) {
+                    if (libxl_p9_type_from_string(value, &p9->type)) {
+                        fprintf(stderr, "failed to parse 9pfs type: %s\n",
+                                value);
+                        exit(1);
+                    }
+                } else if (!strcmp(key, "max-files")) {
+                    p9->max_files = parse_ulong(value);
+                } else if (!strcmp(key, "max-open-files")) {
+                    p9->max_open_files = parse_ulong(value);
+                } else if (!strcmp(key, "max-space")) {
+                    p9->max_space = parse_ulong(value);
+                } else if (!strcmp(key, "auto-delete")) {
+                    p9->auto_delete = strtoul(value, NULL, 0);
                 } else {
                     fprintf(stderr, "Unknown 9pfs parameter '%s'\n", key);
                     exit(1);
@@ -2243,9 +2257,12 @@ void parse_config_data(const char *config_source,
 
             libxl_string_list_dispose(&pairs);
 
-            if (!p9->path || !p9->security_model || !p9->tag) {
-                fprintf(stderr, "9pfs spec missing required field!\n");
-                exit(1);
+            if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD && !p9->path) {
+                char *path;
+
+                xasprintf(&path, XEN_LOG_DIR "/guests/%s", c_info->name);
+                replace_string(&p9->path, path);
+                free(path);
             }
         }
     }
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:57:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:57:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678370.1055629 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7if-0002t7-HJ; Thu, 08 Feb 2024 16:57:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678370.1055629; Thu, 08 Feb 2024 16:57:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7if-0002sy-Ej; Thu, 08 Feb 2024 16:57:49 +0000
Received: by outflank-mailman (input) for mailman id 678370;
 Thu, 08 Feb 2024 16:57:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7i6-0005bD-Nf
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:57:14 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1b67808d-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:57:13 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 601B621F60;
 Thu,  8 Feb 2024 16:57:13 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 11EA61326D;
 Thu,  8 Feb 2024 16:57:13 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id NbKuAukHxWUzFAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:57:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b67808d-c6a3-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411433; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4t9CaY2HdSuK4w688pTUcmMnQS10iOMn7pZg/fdTzSY=;
	b=Q892pMr6spfVOepKXkEDMY/k6It1weX3je2832BNClnIZWNYrnARW1hUBATT06ePVmj9QR
	BqAclUImJF38Arr5AjHa316tR45g0kuDxWp5ZrB0y71UQ9WFWwY2qBmbC7DOJOwTndNp3U
	M3o4LkNkzZ7AajA11YBIWrWUMqAR8Cg=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411433; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=4t9CaY2HdSuK4w688pTUcmMnQS10iOMn7pZg/fdTzSY=;
	b=Q892pMr6spfVOepKXkEDMY/k6It1weX3je2832BNClnIZWNYrnARW1hUBATT06ePVmj9QR
	BqAclUImJF38Arr5AjHa316tR45g0kuDxWp5ZrB0y71UQ9WFWwY2qBmbC7DOJOwTndNp3U
	M3o4LkNkzZ7AajA11YBIWrWUMqAR8Cg=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	George Dunlap <george.dunlap@cloud.com>
Subject: [PATCH v5 15/22] tools/libs/light: add backend type for 9pfs PV devices
Date: Thu,  8 Feb 2024 17:55:39 +0100
Message-Id: <20240208165546.5715-16-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,citrix.com,gmail.com,xen.org,cloud.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Make the backend type of 9pfs PV devices configurable. The default is
"qemu" with the related Xenstore backend-side directory being "9pfs".

Add another type "xen_9pfsd" with the related Xenstore backend-side
directory "xen_9pfs".

As additional security features it is possible to specify:
- "max-space" for limiting the maximum space consumed on the filesystem
  in MBs
- "max-files" for limiting the maximum number of files in the
  filesystem
- "max-open-files" for limiting the maximum number of concurrent open
  files

For convenience "auto-delete" is available to let the backend delete the
oldest file of the guest in case otherwise "max-space" or "max-files"
would be violated.

The xen-9pfsd daemon will be started by libxenlight automatically when
the first "xen_9pfs" device is being created.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: George Dunlap <george.dunlap@cloud.com> # Golang bits
---
V3:
- regenerate go bindings
V4:
- rename libxl_device_p9_dm_needed() to libxl__device_p9_dm_needed()
  (Anthony Perard)
- reorder span related functions (Anthony Perard)
- add comment for xen9pfsd_spawn() return values (Anthony Perard)
- add LIBXL_HAVE_XEN_9PFS to libxl.h (Anthony Perard)
- use a copy of 'p9' in xen9pfsd_spawn() (Anthony Perard)
---
 tools/golang/xenlight/helpers.gen.go      |  16 +++
 tools/golang/xenlight/types.gen.go        |  13 ++
 tools/include/libxl.h                     |   7 +
 tools/libs/light/libxl_9pfs.c             | 159 +++++++++++++++++++++-
 tools/libs/light/libxl_create.c           |   4 +-
 tools/libs/light/libxl_dm.c               |   2 +-
 tools/libs/light/libxl_types.idl          |  11 ++
 tools/libs/light/libxl_types_internal.idl |   1 +
 8 files changed, 206 insertions(+), 7 deletions(-)

diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 35e209ff1b..8f44397a4e 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1280,6 +1280,9 @@ return fmt.Errorf("converting field Rdm: %v", err)
 }
 x.RdmMemBoundaryMemkb = uint64(tmp.rdm_mem_boundary_memkb)
 x.McaCaps = uint64(tmp.mca_caps)
+if err := x.Pirq.fromC(&tmp.pirq);err != nil {
+return fmt.Errorf("converting field Pirq: %v", err)
+}
 return nil
 }
 
@@ -1613,6 +1616,9 @@ return fmt.Errorf("converting field Rdm: %v", err)
 }
 hvm.rdm_mem_boundary_memkb = C.uint64_t(tmp.RdmMemBoundaryMemkb)
 hvm.mca_caps = C.uint64_t(tmp.McaCaps)
+if err := tmp.Pirq.toC(&hvm.pirq); err != nil {
+return fmt.Errorf("converting field Pirq: %v", err)
+}
 hvmBytes := C.GoBytes(unsafe.Pointer(&hvm),C.sizeof_libxl_domain_build_info_type_union_hvm)
 copy(xc.u[:],hvmBytes)
 case DomainTypePv:
@@ -2434,6 +2440,11 @@ x.Tag = C.GoString(xc.tag)
 x.Path = C.GoString(xc.path)
 x.SecurityModel = C.GoString(xc.security_model)
 x.Devid = Devid(xc.devid)
+x.Type = P9Type(xc._type)
+x.MaxSpace = int(xc.max_space)
+x.MaxFiles = int(xc.max_files)
+x.MaxOpenFiles = int(xc.max_open_files)
+x.AutoDelete = bool(xc.auto_delete)
 
  return nil}
 
@@ -2452,6 +2463,11 @@ xc.path = C.CString(x.Path)}
 if x.SecurityModel != "" {
 xc.security_model = C.CString(x.SecurityModel)}
 xc.devid = C.libxl_devid(x.Devid)
+xc._type = C.libxl_p9_type(x.Type)
+xc.max_space = C.int(x.MaxSpace)
+xc.max_files = C.int(x.MaxFiles)
+xc.max_open_files = C.int(x.MaxOpenFiles)
+xc.auto_delete = C.bool(x.AutoDelete)
 
  return nil
  }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 7907aa8999..d31722407a 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -122,6 +122,13 @@ NicTypeVifIoemu NicType = 1
 NicTypeVif NicType = 2
 )
 
+type P9Type int
+const(
+P9TypeUnknown P9Type = 0
+P9TypeQemu P9Type = 1
+P9TypeXen9Pfsd P9Type = 2
+)
+
 type ActionOnShutdown int
 const(
 ActionOnShutdownDestroy ActionOnShutdown = 1
@@ -651,6 +658,7 @@ SerialList StringList
 Rdm RdmReserve
 RdmMemBoundaryMemkb uint64
 McaCaps uint64
+Pirq Defbool
 }
 
 func (x DomainBuildInfoTypeUnionHvm) isDomainBuildInfoTypeUnion(){}
@@ -888,6 +896,11 @@ Tag string
 Path string
 SecurityModel string
 Devid Devid
+Type P9Type
+MaxSpace int
+MaxFiles int
+MaxOpenFiles int
+AutoDelete bool
 }
 
 type DevicePvcallsif struct {
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index f1652b1664..a554f2ccd6 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -615,6 +615,13 @@
  */
 #define LIBXL_HAVE_HVM_PIRQ 1
 
+/*
+ * LIBXL_HAVE_XEN_9PFS indicates the presence of the xen-9pfsd related
+ * fields in libxl_device_p9: type, max_space, max_files, max_open_files and
+ * auto_delete.
+ */
+#define LIBXL_HAVE_XEN_9PFS 1
+
 /*
  * libxl memory management
  *
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 5ab0d3aa21..68fa9947ad 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -33,20 +33,173 @@ static int libxl__set_xenstore_p9(libxl__gc *gc, uint32_t domid,
 
     flexarray_append_pair(front, "tag", p9->tag);
 
+    if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD) {
+        flexarray_append_pair(back, "max-space",
+                              GCSPRINTF("%u", p9->max_space));
+        flexarray_append_pair(back, "max-files",
+                              GCSPRINTF("%u", p9->max_files));
+        flexarray_append_pair(back, "max-open-files",
+                              GCSPRINTF("%u", p9->max_open_files));
+        flexarray_append_pair(back, "auto-delete",
+                              p9->auto_delete ? "1" : "0");
+    }
+
+    return 0;
+}
+
+static int libxl__device_from_p9(libxl__gc *gc, uint32_t domid,
+                                 libxl_device_p9 *type, libxl__device *device)
+{
+    device->backend_devid   = type->devid;
+    device->backend_domid   = type->backend_domid;
+    device->backend_kind    = type->type == LIBXL_P9_TYPE_QEMU
+                              ? LIBXL__DEVICE_KIND_9PFS
+                              : LIBXL__DEVICE_KIND_XEN_9PFS;
+    device->devid           = type->devid;
+    device->domid           = domid;
+    device->kind            = LIBXL__DEVICE_KIND_9PFS;
+
     return 0;
 }
 
-#define libxl__add_p9s NULL
+static int libxl__device_p9_dm_needed(void *e, unsigned domid)
+{
+    libxl_device_p9 *elem = e;
+
+    return elem->type == LIBXL_P9_TYPE_QEMU && elem->backend_domid == domid;
+}
+
+typedef struct libxl__aop9_state libxl__aop9_state;
+
+struct libxl__aop9_state {
+    libxl__spawn_state spawn;
+    libxl__ao_device *aodev;
+    libxl_device_p9 p9;
+    uint32_t domid;
+    void (*callback)(libxl__egc *, libxl__aop9_state *, int);
+};
+
+static void xen9pfsd_confirm(libxl__egc *egc, libxl__spawn_state *spawn,
+                             const char *xsdata);
+static void xen9pfsd_failed(libxl__egc *egc, libxl__spawn_state *spawn, int rc);
+static void xen9pfsd_detached(libxl__egc *egc, libxl__spawn_state *spawn);
+static void xen9pfsd_spawn_outcome(libxl__egc *egc, libxl__aop9_state *aop9,
+                                   int rc);
+
+/*
+ * Spawn the xen-9pfsd daemon if needed.
+ * returns:
+ * < 0 if error
+ * 0 if no daemon needs to be spawned
+ * 1 if daemon was spawned
+ */
+static int xen9pfsd_spawn(libxl__egc *egc, uint32_t domid, libxl_device_p9 *p9,
+                         libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    struct libxl__aop9_state *aop9;
+    int rc;
+    char *args[] = { "xen-9pfsd", NULL };
+    char *path = GCSPRINTF("/local/domain/%u/libxl/xen-9pfs",
+                           p9->backend_domid);
+
+    if (p9->type != LIBXL_P9_TYPE_XEN_9PFSD ||
+        libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/state", path)))
+        return 0;
+
+    GCNEW(aop9);
+    aop9->aodev = aodev;
+    libxl_device_p9_copy(CTX, &aop9->p9, p9);
+    aop9->domid = domid;
+    aop9->callback = xen9pfsd_spawn_outcome;
+
+    aop9->spawn.ao = aodev->ao;
+    aop9->spawn.what = "xen-9pfs daemon";
+    aop9->spawn.xspath = GCSPRINTF("%s/state", path);
+    aop9->spawn.timeout_ms = LIBXL_DEVICE_MODEL_START_TIMEOUT * 1000;
+    aop9->spawn.pidpath = GCSPRINTF("%s/pid", path);
+    aop9->spawn.midproc_cb = libxl__spawn_record_pid;
+    aop9->spawn.confirm_cb = xen9pfsd_confirm;
+    aop9->spawn.failure_cb = xen9pfsd_failed;
+    aop9->spawn.detached_cb = xen9pfsd_detached;
+    rc = libxl__spawn_spawn(egc, &aop9->spawn);
+    if (rc < 0)
+        return rc;
+    if (!rc) {
+        setsid();
+        libxl__exec(gc, -1, -1, -1, LIBEXEC_BIN "/xen-9pfsd", args, NULL);
+    }
+
+    return 1;
+}
+
+static void xen9pfsd_confirm(libxl__egc *egc, libxl__spawn_state *spawn,
+                             const char *xsdata)
+{
+    STATE_AO_GC(spawn->ao);
+
+    if (!xsdata)
+        return;
+
+    if (strcmp(xsdata, "running"))
+        return;
+
+    libxl__spawn_initiate_detach(gc, spawn);
+}
+
+static void xen9pfsd_failed(libxl__egc *egc, libxl__spawn_state *spawn, int rc)
+{
+    libxl__aop9_state *aop9 = CONTAINER_OF(spawn, *aop9, spawn);
+
+    xen9pfsd_spawn_outcome(egc, aop9, rc);
+}
+
+static void xen9pfsd_detached(libxl__egc *egc, libxl__spawn_state *spawn)
+{
+    libxl__aop9_state *aop9 = CONTAINER_OF(spawn, *aop9, spawn);
+
+    xen9pfsd_spawn_outcome(egc, aop9, 0);
+}
+
+static void xen9pfsd_spawn_outcome(libxl__egc *egc, libxl__aop9_state *aop9,
+                                   int rc)
+{
+    aop9->aodev->rc = rc;
+    if (rc)
+        aop9->aodev->callback(egc, aop9->aodev);
+    else
+        libxl__device_add_async(egc, aop9->domid, &libxl__p9_devtype,
+                                &aop9->p9, aop9->aodev);
+}
+
+static void libxl__device_p9_add(libxl__egc *egc, uint32_t domid,
+                                 libxl_device_p9 *p9,
+                                 libxl__ao_device *aodev)
+{
+    int rc;
+
+    rc = xen9pfsd_spawn(egc, domid, p9, aodev);
+    if (rc == 1)
+        return;
+
+    if (rc == 0)
+        libxl__device_add_async(egc, domid, &libxl__p9_devtype, p9, aodev);
+
+    aodev->rc = rc;
+    if (rc)
+        aodev->callback(egc, aodev);
+}
+
 #define libxl_device_p9_list NULL
 #define libxl_device_p9_compare NULL
 
 static LIBXL_DEFINE_UPDATE_DEVID(p9)
-static LIBXL_DEFINE_DEVICE_FROM_TYPE(p9)
+static LIBXL_DEFINE_DEVICES_ADD(p9)
 
 LIBXL_DEFINE_DEVICE_REMOVE(p9)
 
 DEFINE_DEVICE_TYPE_STRUCT(p9, 9PFS, p9s,
-    .skip_attach = 1,
     .set_xenstore_config = (device_set_xenstore_config_fn_t)
                            libxl__set_xenstore_p9,
+    .dm_needed = libxl__device_p9_dm_needed,
 );
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 0008fac607..5546335973 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -1761,9 +1761,6 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
         libxl__device_console_dispose(&console);
     }
 
-    for (i = 0; i < d_config->num_p9s; i++)
-        libxl__device_add(gc, domid, &libxl__p9_devtype, &d_config->p9s[i]);
-
     for (i = 0; i < d_config->num_pvcallsifs; i++)
         libxl__device_add(gc, domid, &libxl__pvcallsif_devtype,
                           &d_config->pvcallsifs[i]);
@@ -1900,6 +1897,7 @@ const libxl__device_type *device_type_tbl[] = {
     &libxl__vdispl_devtype,
     &libxl__vsnd_devtype,
     &libxl__virtio_devtype,
+    &libxl__p9_devtype,
     NULL
 };
 
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 14b593110f..2aaaeb5aa7 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -3761,7 +3761,7 @@ int libxl__need_xenpv_qemu(libxl__gc *gc, libxl_domain_config *d_config)
         goto out;
     }
 
-    if (d_config->num_vfbs > 0 || d_config->num_p9s > 0) {
+    if (d_config->num_vfbs > 0) {
         ret = 1;
         goto out;
     }
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 899ad30969..682092a058 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -150,6 +150,12 @@ libxl_nic_type = Enumeration("nic_type", [
     (2, "VIF"),
     ])
 
+libxl_p9_type = Enumeration("p9_type", [
+    (0, "unknown"),
+    (1, "qemu"),
+    (2, "xen_9pfsd"),
+    ])
+
 libxl_action_on_shutdown = Enumeration("action_on_shutdown", [
     (1, "DESTROY"),
 
@@ -943,6 +949,11 @@ libxl_device_p9 = Struct("device_p9", [
     ("path",             string),
     ("security_model",   string),
     ("devid",            libxl_devid),
+    ("type",             libxl_p9_type),
+    ("max_space",        integer),
+    ("max_files",        integer),
+    ("max_open_files",   integer),
+    ("auto_delete",      bool),
 ])
 
 libxl_device_pvcallsif = Struct("device_pvcallsif", [
diff --git a/tools/libs/light/libxl_types_internal.idl b/tools/libs/light/libxl_types_internal.idl
index e24288f1a5..39da71cef5 100644
--- a/tools/libs/light/libxl_types_internal.idl
+++ b/tools/libs/light/libxl_types_internal.idl
@@ -34,6 +34,7 @@ libxl__device_kind = Enumeration("device_kind", [
     (16, "VINPUT"),
     (17, "VIRTIO_DISK"),
     (18, "VIRTIO"),
+    (19, "XEN_9PFS"),
     ])
 
 libxl__console_backend = Enumeration("console_backend", [
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:57:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:57:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678371.1055635 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7if-0002wI-P8; Thu, 08 Feb 2024 16:57:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678371.1055635; Thu, 08 Feb 2024 16:57:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7if-0002vX-LE; Thu, 08 Feb 2024 16:57:49 +0000
Received: by outflank-mailman (input) for mailman id 678371;
 Thu, 08 Feb 2024 16:57:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7hk-0005bD-8A
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:52 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0de5890d-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:56:51 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id E6A5821F60;
 Thu,  8 Feb 2024 16:56:50 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id AB4C41326D;
 Thu,  8 Feb 2024 16:56:50 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id XcdVKNIHxWUPFAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0de5890d-c6a3-11ee-8a4a-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 11/22] tools/9pfsd: add 9pfs create request support
Date: Thu,  8 Feb 2024 17:55:35 +0100
Message-Id: <20240208165546.5715-12-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: E6A5821F60
X-Spam-Flag: NO

Add the create request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- set permissions correctly (Jason Andryuk)
V3:
- use opendirat() etc. (Jason Andryuk)
- rework error handling a little bit
---
 tools/9pfsd/io.c | 151 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 151 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 74efd21c10..583eea6059 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -32,6 +32,7 @@
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
+#define P9_CMD_CREATE     114
 #define P9_CMD_CLUNK      120
 
 /* P9 protocol open flags. */
@@ -42,6 +43,12 @@
 #define P9_OTRUNC        0x10   /* or'ed in, truncate file first */
 #define P9_OREMOVE       0x40   /* or'ed in, remove file after clunk */
 
+/* P9 protocol create permission masks. */
+#define P9_CREATE_PERM_DIR        0x80000000
+#define P9_CREATE_PERM_NOTSUPP    0x03b00000   /* link, symlink, ... */
+#define P9_CREATE_PERM_DIR_MASK   0777
+#define P9_CREATE_PERM_FILE_MASK  0666
+
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
 #define P9_WALK_MAXELEM   16
@@ -963,6 +970,146 @@ static void p9_open(struct ring *ring, struct p9_header *hdr)
     p9_error(ring, hdr->tag, errno);
 }
 
+static void p9_create(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    unsigned int name_off;
+    uint32_t perm;
+    uint8_t mode;
+    unsigned int ext_off;
+    struct p9_fid *fidp;
+    struct p9_fid *new_fidp;
+    char *path;
+    struct stat st;
+    struct p9_qid qid;
+    uint32_t iounit;
+    int flags;
+    int ret;
+
+    ret = fill_data(ring, "USUbS", &fid, &name_off, &perm, &mode, &ext_off);
+    if ( ret != 5 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    if ( !name_ok(ring->str + name_off) )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( perm & P9_CREATE_PERM_NOTSUPP )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || fidp->opened )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    path = malloc(strlen(fidp->path) + strlen(ring->str + name_off) + 2);
+    if ( !path )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, ENOMEM);
+        return;
+    }
+    sprintf(path, "%s/%s", fidp->path, ring->str + name_off);
+    new_fidp = alloc_fid_mem(device, fid, path);
+    free(path);
+    if ( !new_fidp )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, ENOMEM);
+        return;
+    }
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    new_fidp->ref = fidp->ref;
+
+    if ( perm & P9_CREATE_PERM_DIR )
+    {
+        perm &= P9_CREATE_PERM_DIR_MASK & st.st_mode;
+        if ( mode != P9_OREAD )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        if ( mkdirat(device->root_fd, new_fidp->path, perm) < 0 )
+            goto err;
+
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+        free(fidp);
+        fidp = new_fidp;
+        new_fidp = NULL;
+
+        fidp->fd = openat(device->root_fd, fidp->path, O_RDONLY);
+        if ( fidp->fd < 0 )
+            goto err;
+        fidp->data = fdopendir(fidp->fd);
+        if ( !fidp->data )
+            goto err;
+    }
+    else
+    {
+        flags = open_flags_from_mode(mode);
+        if ( flags < 0 )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        perm &= P9_CREATE_PERM_FILE_MASK & st.st_mode;
+
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+        free(fidp);
+        fidp = new_fidp;
+        new_fidp = NULL;
+
+        fidp->fd = openat(device->root_fd, fidp->path, flags | O_CREAT | O_EXCL,
+                          perm);
+        if ( fidp->fd < 0 )
+            goto err;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+        goto err;
+
+    fill_qid(device, fidp->path, &qid, &st);
+    iounit = get_iounit(ring, &st);
+    fidp->opened = true;
+    fidp->mode = mode;
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "QU", &qid, &iounit);
+
+    return;
+
+ err:
+    p9_error(ring, hdr->tag, errno);
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    free(new_fidp);
+    free_fid(device, fidp);
+}
+
 static void p9_clunk(struct ring *ring, struct p9_header *hdr)
 {
     device *device = ring->device;
@@ -1072,6 +1219,10 @@ void *io_thread(void *arg)
                 p9_open(ring, &hdr);
                 break;
 
+            case P9_CMD_CREATE:
+                p9_create(ring, &hdr);
+                break;
+
             case P9_CMD_CLUNK:
                 p9_clunk(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:58:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:58:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678372.1055650 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7j0-0003tP-29; Thu, 08 Feb 2024 16:58:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678372.1055650; Thu, 08 Feb 2024 16:58:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7iz-0003tI-Uq; Thu, 08 Feb 2024 16:58:09 +0000
Received: by outflank-mailman (input) for mailman id 678372;
 Thu, 08 Feb 2024 16:58:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rY7iy-0003so-Mp
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:58:08 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rY7is-0002rV-7l; Thu, 08 Feb 2024 16:58:02 +0000
Received: from [15.248.3.89] (helo=[10.24.67.25])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rY7is-0007xl-2K; Thu, 08 Feb 2024 16:58:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=RENPLmjSW3ld/feJjMbCFg7nhfYb8CIKzF6PBoHCtBg=; b=3OJAITkVafby8TiHw9Dk77/yOy
	henoFtHGg+khlhHpRTrfsaTXTskkdb44TOR+PpYi3fmNHQW7jj/glgQXeJtavAqXD4/l8CAZR3bvQ
	5+qSHoj5j93WWXf4X0Bi9AU96DyML1Li95psdPtgGxkjmEKRwu86AslbdU5ljQd42pxw=;
Message-ID: <e18cf10d-0531-4b11-8ead-a65c853a7a90@xen.org>
Date: Thu, 8 Feb 2024 16:58:00 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen 4.19 release status tracking list
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: committers@xenproject.org, community.manager@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>
References: <CAMacjJzio2EDwudCUHoL6psKYztkHZOWwLNDknATvpN6Y0im=Q@mail.gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CAMacjJzio2EDwudCUHoL6psKYztkHZOWwLNDknATvpN6Y0im=Q@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 05/02/2024 19:23, Oleksii Kurochko wrote:
> Hello everyone,
> 
> I would like to share with you a list for status tracking based on Xen ML:
> 
> Arm:
>    * [PATCH v5 00/13] Arm cache coloring [
> https://lore.kernel.org/xen-devel/20240102095138.17933-1-carlo.nonato@minervasys.tech/
> ]
>    * [PATCH v1 repost 0/4] xen/arm64: Rework the MMU-off code (idmap)
> so it is self-contained [
> https://lore.kernel.org/xen-devel/20240116143709.86584-1-julien@xen.org/
> ]

This has been committed. You can mark it as resolved.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:58:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:58:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678373.1055660 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7jR-0004eL-A0; Thu, 08 Feb 2024 16:58:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678373.1055660; Thu, 08 Feb 2024 16:58:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7jR-0004eE-6o; Thu, 08 Feb 2024 16:58:37 +0000
Received: by outflank-mailman (input) for mailman id 678373;
 Thu, 08 Feb 2024 16:58:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7hu-0005bD-V7
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:57:02 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 149118b1-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:57:02 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 0E4841FCFC;
 Thu,  8 Feb 2024 16:57:02 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D2C561326D;
 Thu,  8 Feb 2024 16:57:01 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id hjMEMt0HxWUnFAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:57:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 149118b1-c6a3-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411422; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=6jOy4CDaubUih8kxJ4zaPc/m/f/u39ZSxwFGdu6w5z0=;
	b=rKhMrlHaQrA+P8LTKcB5TETHd3n5cwUe7Q+Ej601xC3azrCz70X34wIFbdE4NUYBgOdU9e
	kbCqi/hKMsdTKSFxUrJn3TGy0+0Lk6eaXQSrm7sP9Fl+ZcmiZNmTuxsbFJAXTQzW2zMt8U
	QFaWQObKpXTICwTm6txQclmvmnCaXmI=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411422; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=6jOy4CDaubUih8kxJ4zaPc/m/f/u39ZSxwFGdu6w5z0=;
	b=rKhMrlHaQrA+P8LTKcB5TETHd3n5cwUe7Q+Ej601xC3azrCz70X34wIFbdE4NUYBgOdU9e
	kbCqi/hKMsdTKSFxUrJn3TGy0+0Lk6eaXQSrm7sP9Fl+ZcmiZNmTuxsbFJAXTQzW2zMt8U
	QFaWQObKpXTICwTm6txQclmvmnCaXmI=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 13/22] tools/9pfsd: add 9pfs write request support
Date: Thu,  8 Feb 2024 17:55:37 +0100
Message-Id: <20240208165546.5715-14-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [0.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[citrix.com:email,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: 0.90

Add the write request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/9pfsd/io.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 031cff9f0e..b763e3d8d9 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -33,6 +33,7 @@
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
+#define P9_CMD_WRITE      118
 #define P9_CMD_CLUNK      120
 #define P9_CMD_STAT       124
 
@@ -1246,6 +1247,55 @@ static void p9_stat(struct ring *ring, struct p9_header *hdr)
     free_fid(device, fidp);
 }
 
+static void p9_write(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint64_t off;
+    unsigned int len;
+    uint32_t written;
+    void *buf;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "ULD", &fid, &off, &len, ring->buffer);
+    if ( ret != 3 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || !fidp->opened || fidp->isdir )
+    {
+        p9_error(ring, hdr->tag, EBADF);
+        goto out;
+    }
+
+    buf = ring->buffer;
+
+    while ( len != 0 )
+    {
+        ret = pwrite(fidp->fd, buf, len, off);
+        if ( ret < 0 )
+            break;
+        len -= ret;
+        buf += ret;
+        off += ret;
+    }
+
+    written = buf - ring->buffer;
+    if ( written == 0 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "U", &written);
+
+ out:
+    free_fid(device, fidp);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1321,6 +1371,10 @@ void *io_thread(void *arg)
                 p9_create(ring, &hdr);
                 break;
 
+            case P9_CMD_WRITE:
+                p9_write(ring, &hdr);
+                break;
+
             case P9_CMD_CLUNK:
                 p9_clunk(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:58:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:58:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678374.1055670 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7jT-0004vY-Kt; Thu, 08 Feb 2024 16:58:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678374.1055670; Thu, 08 Feb 2024 16:58:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7jT-0004vP-Gc; Thu, 08 Feb 2024 16:58:39 +0000
Received: by outflank-mailman (input) for mailman id 678374;
 Thu, 08 Feb 2024 16:58:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7hY-0005bD-Ip
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:40 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 072fc20f-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:56:40 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id A7D4021F60;
 Thu,  8 Feb 2024 16:56:39 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7A0D31326D;
 Thu,  8 Feb 2024 16:56:39 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id udg8HMcHxWX7EwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 072fc20f-c6a3-11ee-8a4a-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 09/22] tools/9pfsd: add 9pfs open request support
Date: Thu,  8 Feb 2024 17:55:33 +0100
Message-Id: <20240208165546.5715-10-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: A7D4021F60
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add the open request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- don't allow to open symbolic link
V3:
- use openat() (Jason Andryuk)
- use common error handling in p9_open()
---
 tools/9pfsd/io.c        | 137 ++++++++++++++++++++++++++++++++++++++++
 tools/9pfsd/xen-9pfsd.h |   4 ++
 2 files changed, 141 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 5b827706b7..2e2275e8ab 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -19,6 +19,8 @@
 #include <syslog.h>
 #include <sys/types.h>
 #include <sys/stat.h>
+#include <dirent.h>
+#include <fcntl.h>
 #include <xenctrl.h>           /* For cpu barriers. */
 #include <xen-tools/common-macros.h>
 
@@ -29,6 +31,15 @@
 #define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
+#define P9_CMD_OPEN       112
+
+/* P9 protocol open flags. */
+#define P9_OREAD            0   /* read */
+#define P9_OWRITE           1   /* write */
+#define P9_ORDWR            2   /* read and write */
+#define P9_OMODEMASK     0x03
+#define P9_OTRUNC        0x10   /* or'ed in, truncate file first */
+#define P9_OREMOVE       0x40   /* or'ed in, remove file after clunk */
 
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
@@ -829,6 +840,128 @@ static void p9_walk(struct ring *ring, struct p9_header *hdr)
     free(names);
 }
 
+static int open_flags_from_mode(uint8_t mode)
+{
+    int flags;
+
+    switch ( mode & P9_OMODEMASK )
+    {
+    case P9_OREAD:
+        flags = O_RDONLY;
+        break;
+
+    case P9_OWRITE:
+        flags = O_WRONLY;
+        break;
+
+    case P9_ORDWR:
+        flags = O_RDWR;
+        break;
+
+    default:
+        errno = EINVAL;
+        return -1;
+    }
+
+    if ( mode & P9_OTRUNC )
+        flags |= O_TRUNC;
+
+    return flags;
+}
+
+static unsigned int get_iounit(struct ring *ring, struct stat *st)
+{
+    return (ring->max_size - st->st_blksize) & ~(st->st_blksize - 1);
+}
+
+static void p9_open(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint8_t mode;
+    struct p9_fid *fidp;
+    struct stat st;
+    struct p9_qid qid;
+    uint32_t iounit;
+    int flags;
+    int ret;
+
+    ret = fill_data(ring, "Ub", &fid, &mode);
+    if ( ret != 2 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+    if ( mode & ~(P9_OMODEMASK | P9_OTRUNC | P9_OREMOVE) )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+    if ( fidp->opened )
+    {
+        errno = EINVAL;
+        goto err;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        errno = ENOENT;
+        goto err;
+    }
+
+    if ( S_ISLNK(st.st_mode) )
+    {
+        errno = EMLINK;
+        goto err;
+    }
+
+    fidp->isdir = S_ISDIR(st.st_mode);
+    fidp->mode = mode;
+    if ( fidp->isdir )
+    {
+        if ( mode != P9_OREAD )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        fidp->fd = openat(device->root_fd, fidp->path, O_RDONLY);
+        if ( fidp->fd < 0 )
+            goto err;
+        fidp->data = fdopendir(fidp->fd);
+        if ( !fidp->data )
+            goto err;
+    }
+    else
+    {
+        flags = open_flags_from_mode(mode);
+        if ( flags < 0 )
+            goto err;
+
+        fidp->fd = openat(device->root_fd, fidp->path, flags);
+        if ( fidp->fd < 0 )
+            goto err;
+    }
+
+    fill_qid(device, fidp->path, &qid, &st);
+    iounit = get_iounit(ring, &st);
+    fidp->opened = true;
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "QU", &qid, &iounit);
+
+    return;
+
+ err:
+    free_fid(device, fidp);
+    p9_error(ring, hdr->tag, errno);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -896,6 +1029,10 @@ void *io_thread(void *arg)
                 p9_walk(ring, &hdr);
                 break;
 
+            case P9_CMD_OPEN:
+                p9_open(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index f01fffb0bb..757be2da4b 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -25,7 +25,11 @@ struct p9_fid {
     XEN_TAILQ_ENTRY(struct p9_fid) list;
     unsigned int fid;
     unsigned int ref;
+    int fd;
+    uint8_t mode;
     bool opened;
+    bool isdir;
+    void *data;    /* File type specific. */
     char path[];
 };
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:59:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:59:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678378.1055680 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7jv-0005x3-Us; Thu, 08 Feb 2024 16:59:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678378.1055680; Thu, 08 Feb 2024 16:59:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7jv-0005ww-Ql; Thu, 08 Feb 2024 16:59:07 +0000
Received: by outflank-mailman (input) for mailman id 678378;
 Thu, 08 Feb 2024 16:59:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7hf-0005bD-Dk
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:47 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0b4f3aa3-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:56:46 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 5AA2A21F60;
 Thu,  8 Feb 2024 16:56:45 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1A8121326D;
 Thu,  8 Feb 2024 16:56:45 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 9/DXBM0HxWUGFAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b4f3aa3-c6a3-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411406; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pqT/NgFMhhcmocAciubVrW3PJT45hfuiNE/G/OftJpE=;
	b=fihzRyIfjJ3gNXOyEFc51eu7egRWpQEWUgALUXaQZ91zA/2HOzQAXoEcNL0/4v6BQc+zLy
	D+5aeVmcJWhEUUtIoxMeB0SzW0VTY+kKhPu2AIFCtcxgooOrpZbMAhB5OmnwPNm34+keqf
	AqjVpvAuQj+IE+wb0sPRRShjYf7VhLQ=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411405; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pqT/NgFMhhcmocAciubVrW3PJT45hfuiNE/G/OftJpE=;
	b=dadLbYC/4aS81uQ9oHg2Fx+DbM9riQMiy6RVxxOXzz/o1WDh/L5CgUXoeKoP7lB2lofvsp
	SQPo3u2294fu4waACJ+JCI3RAVcPsQxCy97GspQtQbOZegB7349+H6YL8+ki/+hdUdrDQV
	Mj4CeeXauGv6DaFwL7Wv49RKIMIOqPI=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 10/22] tools/9pfsd: add 9pfs clunk request support
Date: Thu,  8 Feb 2024 17:55:34 +0100
Message-Id: <20240208165546.5715-11-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [0.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[citrix.com:email,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[10.37%]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: 0.90

Add the clunk request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V3:
- use unlinkat() (Jason Andryuk)
---
 tools/9pfsd/io.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 2e2275e8ab..74efd21c10 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -32,6 +32,7 @@
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
+#define P9_CMD_CLUNK      120
 
 /* P9 protocol open flags. */
 #define P9_OREAD            0   /* read */
@@ -962,6 +963,44 @@ static void p9_open(struct ring *ring, struct p9_header *hdr)
     p9_error(ring, hdr->tag, errno);
 }
 
+static void p9_clunk(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "U", &fid);
+    if ( ret != 1 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( fidp->opened )
+    {
+        fidp->opened = false;
+        free_fid(device, fidp);
+        close(fidp->fd);
+        if ( fidp->mode & P9_OREMOVE )
+            unlinkat(device->root_fd, fidp->path,
+                     fidp->isdir ? AT_REMOVEDIR : 0);
+    }
+
+    /* 2 calls of free_fid(): one for our reference, and one to free it. */
+    free_fid(device, fidp);
+    free_fid(device, fidp);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "");
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1033,6 +1072,10 @@ void *io_thread(void *arg)
                 p9_open(ring, &hdr);
                 break;
 
+            case P9_CMD_CLUNK:
+                p9_clunk(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 16:59:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 16:59:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678379.1055689 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7kG-0006cG-53; Thu, 08 Feb 2024 16:59:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678379.1055689; Thu, 08 Feb 2024 16:59:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7kG-0006c9-29; Thu, 08 Feb 2024 16:59:28 +0000
Received: by outflank-mailman (input) for mailman id 678379;
 Thu, 08 Feb 2024 16:59:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7hp-0005bD-CP
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:56:57 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 11318eac-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:56:56 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 730F12204E;
 Thu,  8 Feb 2024 16:56:56 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 434F41326D;
 Thu,  8 Feb 2024 16:56:56 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id rYsDD9gHxWUgFAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:56:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11318eac-c6a3-11ee-8a4a-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 12/22] tools/9pfsd: add 9pfs stat request support
Date: Thu,  8 Feb 2024 17:55:36 +0100
Message-Id: <20240208165546.5715-13-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 730F12204E
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add the stat request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V3:
- use fstatat() (Jason Andryuk)
V4:
- add "s" format to fill_buffer() as a preparation for reading dirs
---
 tools/9pfsd/io.c | 102 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 583eea6059..031cff9f0e 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -34,6 +34,7 @@
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
 #define P9_CMD_CLUNK      120
+#define P9_CMD_STAT       124
 
 /* P9 protocol open flags. */
 #define P9_OREAD            0   /* read */
@@ -60,6 +61,25 @@ struct p9_qid {
     uint64_t path;
 };
 
+struct p9_stat {
+    uint16_t size;
+    uint16_t type;
+    uint32_t dev;
+    struct p9_qid qid;
+    uint32_t mode;
+    uint32_t atime;
+    uint32_t mtime;
+    uint64_t length;
+    const char *name;
+    const char *uid;
+    const char *gid;
+    const char *muid;
+    const char *extension;
+    uint32_t n_uid;
+    uint32_t n_gid;
+    uint32_t n_muid;
+};
+
 /*
  * Note that the ring names "in" and "out" are from the frontend's
  * perspective, so the "in" ring will be used for responses to the frontend,
@@ -166,6 +186,7 @@ static void fmt_err(const char *fmt)
  * S: String (2 byte length + <length> characters)
  *    The length is obtained via strlen() of the parameter, being a pointer
  *    to the first character of the string
+ * s: stat (struct p9_stat)
  * U: 4 byte unsigned integer
  *    The parameter is a pointer to a uint32_t value
  */
@@ -176,6 +197,8 @@ static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
     const void *par;
     const char *str_val;
     const struct p9_qid *qid;
+    const struct p9_stat *stat;
+    uint16_t tlen;
     unsigned int len;
     unsigned int array_sz = 0;
     unsigned int elem_sz = 0;
@@ -259,6 +282,18 @@ static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
             *data += len;
             break;
 
+        case 's':
+            stat = par;
+            elem_sz = sizeof(*stat);
+            tlen = stat->size + sizeof(stat->size);
+            fill_buffer_at(data, "uuuUQUUULSSSSSUUU", &tlen, &stat->size,
+                           &stat->type, &stat->dev, &stat->qid, &stat->mode,
+                           &stat->atime, &stat->mtime, &stat->length,
+                           stat->name, stat->uid, stat->gid, stat->muid,
+                           stat->extension, &stat->n_uid, &stat->n_gid,
+                           &stat->n_muid);
+            break;
+
         case 'U':
             put_unaligned(*(const uint32_t *)par, (uint32_t *)*data);
             elem_sz = sizeof(uint32_t);
@@ -1148,6 +1183,69 @@ static void p9_clunk(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "");
 }
 
+static void fill_p9_stat(device *device, struct p9_stat *p9s, struct stat *st,
+                         const char *name)
+{
+    memset(p9s, 0, sizeof(*p9s));
+    fill_qid(device, NULL, &p9s->qid, st);
+    p9s->mode = st->st_mode & 0777;
+    if ( S_ISDIR(st->st_mode) )
+        p9s->mode |= P9_CREATE_PERM_DIR;
+    p9s->atime = st->st_atime;
+    p9s->mtime = st->st_mtime;
+    p9s->length = st->st_size;
+    p9s->name = name;
+    p9s->uid = "";
+    p9s->gid = "";
+    p9s->muid = "";
+    p9s->extension = "";
+    p9s->n_uid = 0;
+    p9s->n_gid = 0;
+    p9s->n_muid = 0;
+
+    /*
+     * Size of individual fields without the size field, including 5 2-byte
+     * string length fields.
+     */
+    p9s->size = 71 + strlen(p9s->name);
+}
+
+static void p9_stat(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    struct p9_fid *fidp;
+    struct p9_stat p9s;
+    struct stat st;
+    int ret;
+
+    ret = fill_data(ring, "U", &fid);
+    if ( ret != 1 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+    fill_p9_stat(device, &p9s, &st, strrchr(fidp->path, '/') + 1);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "s", &p9s);
+
+ out:
+    free_fid(device, fidp);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1227,6 +1325,10 @@ void *io_thread(void *arg)
                 p9_clunk(ring, &hdr);
                 break;
 
+            case P9_CMD_STAT:
+                p9_stat(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 17:07:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 17:07:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678411.1055700 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7s2-0000Oj-UZ; Thu, 08 Feb 2024 17:07:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678411.1055700; Thu, 08 Feb 2024 17:07:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7s2-0000Oc-Rq; Thu, 08 Feb 2024 17:07:30 +0000
Received: by outflank-mailman (input) for mailman id 678411;
 Thu, 08 Feb 2024 17:07:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7ie-0005bD-Bj
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:57:48 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2f8186d7-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:57:47 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 1DF281FCF6;
 Thu,  8 Feb 2024 16:57:47 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id CF9611326D;
 Thu,  8 Feb 2024 16:57:46 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id JtLtMAoIxWWbFAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:57:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f8186d7-c6a3-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411467; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mx+xsXHmP+kMTvkOl3Dx/uX2x1DBpbeqqzrtIUR5SLs=;
	b=bITun6oikkB/33YNcvw3f0MZodSkbL79eFnv0oLLWJiAchpZd+1s6DNnyMBLzFMryN/ORx
	eJYirQgll/Xt5Tyna/xFapyBnmlU4JnZtPAbEg11ytnzl0xKyCQIvZ5X1fP7UdIEFKNKNO
	YZUe3ZP95G5WGXs0eu1DBxZyok6BVo4=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411467; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mx+xsXHmP+kMTvkOl3Dx/uX2x1DBpbeqqzrtIUR5SLs=;
	b=bITun6oikkB/33YNcvw3f0MZodSkbL79eFnv0oLLWJiAchpZd+1s6DNnyMBLzFMryN/ORx
	eJYirQgll/Xt5Tyna/xFapyBnmlU4JnZtPAbEg11ytnzl0xKyCQIvZ5X1fP7UdIEFKNKNO
	YZUe3ZP95G5WGXs0eu1DBxZyok6BVo4=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Julien Grall <julien@xen.org>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v5 21/22] tools/xenstored: support complete log capabilities in stubdom
Date: Thu,  8 Feb 2024 17:55:45 +0100
Message-Id: <20240208165546.5715-22-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-2.10 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: -2.10

With 9pfs being fully available in Xenstore-stubdom now, there is no
reason to not fully support all logging capabilities in stubdom.

Open the logfile on stubdom only after the 9pfs file system has been
mounted.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- remove now stale comment in sysconfig.xencommons.in (Julien Grall)
---
 .../Linux/init.d/sysconfig.xencommons.in      |  1 -
 tools/hotplug/Linux/launch-xenstore.in        |  1 +
 tools/xenstored/control.c                     | 30 +++++++++----------
 tools/xenstored/minios.c                      |  3 ++
 4 files changed, 19 insertions(+), 16 deletions(-)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index 433e4849af..1bdd830d8a 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -58,7 +58,6 @@ XENSTORED_ARGS=
 ## Default: Not defined, tracing off
 #
 # Log xenstored messages
-# Only evaluated if XENSTORETYPE is "daemon".
 #XENSTORED_TRACE=[yes|on|1]
 
 ## Type: integer
diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in
index e854ca1eb8..da4eeca7c5 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -98,6 +98,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 	[ -z "$XENSTORE_DOMAIN_SIZE" ] && XENSTORE_DOMAIN_SIZE=8
 	XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory $XENSTORE_DOMAIN_SIZE"
 	[ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem $XENSTORE_MAX_DOMAIN_SIZE"
+	[ -z "$XENSTORED_TRACE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS -T xenstored-trace.log"
 
 	echo -n Starting $XENSTORE_DOMAIN_KERNEL...
 	${LIBEXEC_BIN}/init-xenstore-domain $XENSTORE_DOMAIN_ARGS || exit 1
diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index b2f64d674f..dae23a5ac0 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -201,19 +201,6 @@ static int do_control_quota_s(const void *ctx, struct connection *conn,
 	return EINVAL;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_logfile(const void *ctx, struct connection *conn,
 			      const char **vec, int num)
 {
@@ -222,13 +209,26 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 
 	close_log();
 	talloc_free(tracefile);
-	tracefile = talloc_strdup(NULL, vec[0]);
+	tracefile = absolute_filename(NULL, vec[0]);
 	reopen_log();
 
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
 
+#ifdef __MINIOS__
+static int do_control_memreport(const void *ctx, struct connection *conn,
+				const char **vec, int num)
+{
+	if (num)
+		return EINVAL;
+
+	talloc_report_full(NULL, stdout);
+
+	send_ack(conn, XS_CONTROL);
+	return 0;
+}
+#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
@@ -309,10 +309,10 @@ static struct cmd_s cmds[] = {
 		"[-c <cmdline>] [-F] [-t <timeout>] <file>\n"
 		"    Default timeout is 60 seconds.", 5 },
 #endif
+	{ "logfile", do_control_logfile, "<file>" },
 #ifdef __MINIOS__
 	{ "memreport", do_control_memreport, "" },
 #else
-	{ "logfile", do_control_logfile, "<file>" },
 	{ "memreport", do_control_memreport, "[<file>]" },
 #endif
 	{ "print", do_control_print, "<string>" },
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index e70386f8c7..a229954cf4 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -122,6 +122,9 @@ static void mount_thread(void *p)
 	}
 
 	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+
+	/* Start logging if selected. */
+	reopen_log();
 }
 
 void mount_9pfs(void)
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 17:09:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 17:09:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678415.1055710 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7to-0001EX-CW; Thu, 08 Feb 2024 17:09:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678415.1055710; Thu, 08 Feb 2024 17:09:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7to-0001EQ-8o; Thu, 08 Feb 2024 17:09:20 +0000
Received: by outflank-mailman (input) for mailman id 678415;
 Thu, 08 Feb 2024 17:09:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7iI-0005q0-El
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:57:26 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 21fcf08e-c6a3-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:57:24 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 89EE022135;
 Thu,  8 Feb 2024 16:57:24 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6028D13985;
 Thu,  8 Feb 2024 16:57:24 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id DUQLFvQHxWU/FAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:57:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21fcf08e-c6a3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411444; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/eJaaQzJFr9iaY84hbFZ6eF6VnkmvJwoY+5FWl4+J1Y=;
	b=SfizwTsFQSmxJXg2DZCFqiTWiYAvzfRkXWEVBqvHCMPAUHhprZEeee96Oq2R8bqe0o6WNr
	UL4pQUUshGAYJoLAugvlE/LuoxOZsf8Gw/Uq8yJwE+S8580j08EsdpxWrZyFzdgs/q21tk
	CAKCiEmtdTcps6WYuy/RySIUZZCIIQc=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411444; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/eJaaQzJFr9iaY84hbFZ6eF6VnkmvJwoY+5FWl4+J1Y=;
	b=SfizwTsFQSmxJXg2DZCFqiTWiYAvzfRkXWEVBqvHCMPAUHhprZEeee96Oq2R8bqe0o6WNr
	UL4pQUUshGAYJoLAugvlE/LuoxOZsf8Gw/Uq8yJwE+S8580j08EsdpxWrZyFzdgs/q21tk
	CAKCiEmtdTcps6WYuy/RySIUZZCIIQc=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 17/22] stubdom: extend xenstore stubdom configs
Date: Thu,  8 Feb 2024 17:55:41 +0100
Message-Id: <20240208165546.5715-18-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [0.88 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 TO_DN_SOME(0.00)[];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,ens-lyon.org,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.02)[54.20%]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: 0.88

Extend the config files of the Xenstore stubdoms to include XENBUS
and 9PFRONT items in order to support file based logging.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 stubdom/xenstore-minios.cfg    | 2 +-
 stubdom/xenstorepvh-minios.cfg | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/stubdom/xenstore-minios.cfg b/stubdom/xenstore-minios.cfg
index a41704bb6b..239da519b9 100644
--- a/stubdom/xenstore-minios.cfg
+++ b/stubdom/xenstore-minios.cfg
@@ -3,7 +3,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
diff --git a/stubdom/xenstorepvh-minios.cfg b/stubdom/xenstorepvh-minios.cfg
index 6af51f5753..752b90d7d3 100644
--- a/stubdom/xenstorepvh-minios.cfg
+++ b/stubdom/xenstorepvh-minios.cfg
@@ -4,7 +4,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 17:09:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 17:09:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678416.1055714 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7to-0001Hk-Ia; Thu, 08 Feb 2024 17:09:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678416.1055714; Thu, 08 Feb 2024 17:09:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7to-0001Gk-F9; Thu, 08 Feb 2024 17:09:20 +0000
Received: by outflank-mailman (input) for mailman id 678416;
 Thu, 08 Feb 2024 17:09:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7iN-0005bD-3X
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:57:31 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2542a858-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:57:30 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 234F31FCF9;
 Thu,  8 Feb 2024 16:57:30 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id E9C571326D;
 Thu,  8 Feb 2024 16:57:29 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id sJenN/kHxWVNFAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:57:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2542a858-c6a3-11ee-8a4a-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v5 18/22] tools: add 9pfs device to xenstore-stubdom
Date: Thu,  8 Feb 2024 17:55:42 +0100
Message-Id: <20240208165546.5715-19-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 234F31FCF9
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add a 9pfs device to Xenstore stubdom in order to allow it to do e.g.
logging into a dom0 file.

Use the following parameters for the new device:

- tag = "Xen"
- type = "xen_9pfsd"
- path = "/var/lib/xen/xenstore"
- security-model = "none"

For now don't limit allowed file space or number of files.

Add a new libxl function for adding it similar to the function for
adding the console device.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- add security_model parameter to new libxl function (Jason Andryuk)
V4:
- rename function to libxl_device_9pfs_add() (Anthony Perard)
- use a libxl_device_p9 pointer as parameter (Anthony Perard)
---
 tools/helpers/init-xenstore-domain.c |  7 +++++++
 tools/include/libxl.h                | 15 +++++++++++++++
 tools/libs/light/libxl_9pfs.c        | 16 ++++++++++++++++
 3 files changed, 38 insertions(+)

diff --git a/tools/helpers/init-xenstore-domain.c b/tools/helpers/init-xenstore-domain.c
index 140ed610ae..1683438c5c 100644
--- a/tools/helpers/init-xenstore-domain.c
+++ b/tools/helpers/init-xenstore-domain.c
@@ -433,6 +433,12 @@ int main(int argc, char** argv)
     int rv, fd;
     char *maxmem_str = NULL;
     libxl_ctx *ctx;
+    libxl_device_p9 p9 = { .backend_domid = 0,
+                           .tag = "Xen",
+                           .path = XEN_LIB_DIR"/xenstore",
+                           .security_model = "none",
+                           .type = LIBXL_P9_TYPE_XEN_9PFSD,
+    };
 
     while ( (opt = getopt_long(argc, argv, "v", options, NULL)) != -1 )
     {
@@ -543,6 +549,7 @@ int main(int argc, char** argv)
     }
     libxl_console_add_xenstore(ctx, domid, 0, console_evtchn, console_gfn,
                                NULL);
+    libxl_device_9pfs_add(ctx, domid, &p9, NULL);
     libxl_ctx_free(ctx);
 
     fd = creat(XEN_RUN_DIR "/xenstored.pid", 0666);
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index a554f2ccd6..e6ab5ddb94 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -583,6 +583,13 @@
  * libxl_console_add_xenstore() in libxl.
  */
 #define LIBXL_HAVE_CONSOLE_ADD_XENSTORE 1
+
+/*
+ * LIBXL_HAVE_P9_ADD_XENSTORE indicates presence of the function
+ * libxl_device_9pfs_add() in libxl.
+ */
+#define LIBXL_HAVE_P9_ADD_XENSTORE 1
+
 /*
  * libxl ABI compatibility
  *
@@ -2074,6 +2081,14 @@ int libxl_console_add_xenstore(libxl_ctx *ctx, uint32_t domid, uint32_t backend,
                                const libxl_asyncop_how *ao_how)
                                LIBXL_EXTERNAL_CALLERS_ONLY;
 
+/* libxl_device_9pfs_add writes the Xenstore entries for a domain's
+ * primary 9pfs device based on domid, and device parameters.
+ * If needed it will start the backend daemon.
+ */
+int libxl_device_9pfs_add(libxl_ctx *ctx, uint32_t domid, libxl_device_p9 *p9,
+                          const libxl_asyncop_how *ao_how)
+                          LIBXL_EXTERNAL_CALLERS_ONLY;
+
 /* May be called with info_r == NULL to check for domain's existence.
  * Returns ERROR_DOMAIN_NOTFOUND if domain does not exist (used to return
  * ERROR_INVAL for this scenario). */
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 7593686a90..8f5d475322 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -208,6 +208,22 @@ static void libxl__device_p9_add(libxl__egc *egc, uint32_t domid,
         aodev->callback(egc, aodev);
 }
 
+int libxl_device_9pfs_add(libxl_ctx *ctx, uint32_t domid, libxl_device_p9 *p9,
+                          const libxl_asyncop_how *ao_how)
+{
+    AO_CREATE(ctx, domid, ao_how);
+    libxl__ao_device *aodev;
+
+    GCNEW(aodev);
+    libxl__prepare_ao_device(ao, aodev);
+    aodev->action = LIBXL__DEVICE_ACTION_ADD;
+    aodev->callback = device_addrm_aocomplete;
+
+    libxl__device_p9_add(egc, domid, p9, aodev);
+
+    return AO_INPROGRESS;
+}
+
 #define libxl_device_p9_list NULL
 #define libxl_device_p9_compare NULL
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 17:12:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 17:12:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678422.1055730 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7wZ-0003N7-2n; Thu, 08 Feb 2024 17:12:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678422.1055730; Thu, 08 Feb 2024 17:12:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7wY-0003N0-Ux; Thu, 08 Feb 2024 17:12:10 +0000
Received: by outflank-mailman (input) for mailman id 678422;
 Thu, 08 Feb 2024 17:12:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7iT-0005q0-Rk
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:57:37 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 28ab82da-c6a3-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 17:57:36 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id BF7C522137;
 Thu,  8 Feb 2024 16:57:35 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 803101326D;
 Thu,  8 Feb 2024 16:57:35 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id UW7WHf8HxWV5FAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:57:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28ab82da-c6a3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411455; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GuKS5JMiiFYV782dbcVdzQvPQKcEtbDic1ZOTvla/ok=;
	b=bh/zegVSwS4GGTjmidwlt9YlsWICUBp/6TeLoTe2keAR7KSbkeVfBUVxaZAgRsrqvOXc7a
	kBCPksWfA+MuZhZ7WaU2nn6nXaTTuSGABhZL+RrYyFkcrcvOJnUAPT25nsrRO8XGX9Ab4E
	KJcBuzkAE1vR04wyrTcrdjMhIhDR4GY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411455; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GuKS5JMiiFYV782dbcVdzQvPQKcEtbDic1ZOTvla/ok=;
	b=bh/zegVSwS4GGTjmidwlt9YlsWICUBp/6TeLoTe2keAR7KSbkeVfBUVxaZAgRsrqvOXc7a
	kBCPksWfA+MuZhZ7WaU2nn6nXaTTuSGABhZL+RrYyFkcrcvOJnUAPT25nsrRO8XGX9Ab4E
	KJcBuzkAE1vR04wyrTcrdjMhIhDR4GY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v5 19/22] tools/xenstored: mount 9pfs device in stubdom
Date: Thu,  8 Feb 2024 17:55:43 +0100
Message-Id: <20240208165546.5715-20-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-2.10 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: -2.10

Mount the 9pfs device in stubdom enabling it to use files.

This has to happen in a worker thread in order to allow the main thread
handling the required Xenstore accesses in parallel.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- add logging in case of errors (Julien Grall)
---
 tools/xenstored/core.h   |  6 +++++
 tools/xenstored/domain.c |  2 ++
 tools/xenstored/minios.c | 54 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+)

diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index f6af086f01..fe0ee90581 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -36,6 +36,8 @@
 #include "list.h"
 #include "hashtable.h"
 
+#define XENSTORE_LIB_DIR	XEN_LIB_DIR "/xenstore"
+
 #ifndef O_CLOEXEC
 #define O_CLOEXEC 0
 /* O_CLOEXEC support is needed for Live Update in the daemon case. */
@@ -399,6 +401,10 @@ void handle_special_fds(void);
 int get_socket_fd(void);
 void set_socket_fd(int fd);
 
+#ifdef __MINIOS__
+void mount_9pfs(void);
+#endif
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index 1a7d5e9756..64c8fd0cc3 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -1236,6 +1236,8 @@ void stubdom_init(void)
 		barf_perror("Failed to initialize stubdom");
 
 	xenevtchn_notify(xce_handle, stubdom->port);
+
+	mount_9pfs();
 #endif
 }
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 22ac8defbd..562a9b4972 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -17,10 +17,20 @@
 */
 #include <sys/types.h>
 #include <sys/mman.h>
+#include <syslog.h>
+#include "talloc.h"
 #include "core.h"
 #include "utils.h"
 #include <xen/grant_table.h>
 #include <mini-os/lib.h>
+#include <mini-os/9pfront.h>
+#include <mini-os/sched.h>
+#include <mini-os/xenbus.h>
+#include <mini-os/xmalloc.h>
+
+#define P9_STATE_PATH	"device/9pfs/0/state"
+
+static void *p9_device;
 
 void finish_daemonize(void)
 {
@@ -74,3 +84,47 @@ int get_socket_fd(void)
 void set_socket_fd(int fd)
 {
 }
+
+static void mount_thread(void *p)
+{
+	xenbus_event_queue events = NULL;
+	char *err;
+	char *dummy;
+
+	err = xenbus_watch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs", &events);
+	if (err) {
+		log("error \"%s\" when setting watch on \"%s\"\n", err,
+		    P9_STATE_PATH);
+		free(err);
+		return;
+	}
+
+	for (;;) {
+		xenbus_wait_for_watch(&events);
+
+		/*
+		 * We only care for existence of the state node.
+		 * State changes are handled in init_9pfront().
+		 */
+		err = xenbus_read(XBT_NIL, P9_STATE_PATH, &dummy);
+		if (!err)
+			break;
+		free(err);
+	}
+
+	free(dummy);
+
+	err = xenbus_unwatch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs");
+	if (err) {
+		log("error \"%s\" when unwatching \"%s\", leaking watch\n",
+		    err, P9_STATE_PATH);
+		free(err);
+	}
+
+	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+}
+
+void mount_9pfs(void)
+{
+	create_thread("mount-9pfs", mount_thread, NULL);
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 17:13:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 17:13:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678427.1055739 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7xa-0003uH-Au; Thu, 08 Feb 2024 17:13:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678427.1055739; Thu, 08 Feb 2024 17:13:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY7xa-0003uA-89; Thu, 08 Feb 2024 17:13:14 +0000
Received: by outflank-mailman (input) for mailman id 678427;
 Thu, 08 Feb 2024 17:13:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7ij-0005bD-IJ
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:57:53 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 32bbef2b-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:57:53 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id B606421F60;
 Thu,  8 Feb 2024 16:57:52 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 7F6281326D;
 Thu,  8 Feb 2024 16:57:52 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id q7qoHRAIxWWjFAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:57:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 32bbef2b-c6a3-11ee-8a4a-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v5 22/22] tools/xenstored: have a single do_control_memreport()
Date: Thu,  8 Feb 2024 17:55:46 +0100
Message-Id: <20240208165546.5715-23-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: B606421F60
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

With 9pfs now available in Xenstore-stubdom, there is no reason to
have distinct do_control_memreport() variants for the daemon and the
stubdom implementations.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 tools/xenstored/control.c | 27 +++++++--------------------
 1 file changed, 7 insertions(+), 20 deletions(-)

diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index dae23a5ac0..9561289179 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -216,23 +216,11 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 	return 0;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
 	FILE *fp;
+	const char *filename;
 	int fd;
 
 	if (num > 1)
@@ -255,8 +243,12 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 			if (!fp)
 				close(fd);
 		}
-	} else
-		fp = fopen(vec[0], "a");
+	} else {
+		filename = absolute_filename(ctx, vec[0]);
+		if (!filename)
+			return ENOMEM;
+		fp = fopen(filename, "a");
+	}
 
 	if (!fp)
 		return EBADF;
@@ -267,7 +259,6 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
-#endif
 
 static int do_control_print(const void *ctx, struct connection *conn,
 			    const char **vec, int num)
@@ -310,11 +301,7 @@ static struct cmd_s cmds[] = {
 		"    Default timeout is 60 seconds.", 5 },
 #endif
 	{ "logfile", do_control_logfile, "<file>" },
-#ifdef __MINIOS__
-	{ "memreport", do_control_memreport, "" },
-#else
 	{ "memreport", do_control_memreport, "[<file>]" },
-#endif
 	{ "print", do_control_print, "<string>" },
 	{ "quota", do_control_quota,
 		"[set <name> <val>|<domid>|max [-r]]" },
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 17:16:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 17:16:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678442.1055750 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY80z-00056M-PZ; Thu, 08 Feb 2024 17:16:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678442.1055750; Thu, 08 Feb 2024 17:16:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY80z-00056F-Mu; Thu, 08 Feb 2024 17:16:45 +0000
Received: by outflank-mailman (input) for mailman id 678442;
 Thu, 08 Feb 2024 17:16:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m/2e=JR=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rY7iY-0005bD-Ec
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 16:57:42 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2c09cc7e-c6a3-11ee-8a4a-1f161083a0e0;
 Thu, 08 Feb 2024 17:57:41 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 6637121F60;
 Thu,  8 Feb 2024 16:57:41 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2A9161326D;
 Thu,  8 Feb 2024 16:57:41 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap1.dmz-prg2.suse.org with ESMTPSA id /Lz4CAUIxWWSFAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 08 Feb 2024 16:57:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c09cc7e-c6a3-11ee-8a4a-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411461; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xUq3AE+SsnVI0DjnPSJQwNriZRE6cVmY47vrUqUrQuc=;
	b=lP+ZdDEbr3RlhqEoHdegVuyE5oJ2N4y5W47/Iv1TOBRM+ar0CH+EG8AAdnLSeSjEHklf0w
	B0ATW55ljMFguLnO9X4l1xQvfmAGoqjNJIbViFKH4Tfrcg/PoZkeQ5A/hPBusQFZcfdz5b
	zckSQ652Fw3S9/bYyQqQYhISnyj5OjI=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707411461; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xUq3AE+SsnVI0DjnPSJQwNriZRE6cVmY47vrUqUrQuc=;
	b=lP+ZdDEbr3RlhqEoHdegVuyE5oJ2N4y5W47/Iv1TOBRM+ar0CH+EG8AAdnLSeSjEHklf0w
	B0ATW55ljMFguLnO9X4l1xQvfmAGoqjNJIbViFKH4Tfrcg/PoZkeQ5A/hPBusQFZcfdz5b
	zckSQ652Fw3S9/bYyQqQYhISnyj5OjI=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v5 20/22] tools/xenstored: add helpers for filename handling
Date: Thu,  8 Feb 2024 17:55:44 +0100
Message-Id: <20240208165546.5715-21-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240208165546.5715-1-jgross@suse.com>
References: <20240208165546.5715-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Flag: NO

Add some helpers for handling filenames which might need different
implementations between stubdom and daemon environments:

- expansion of relative filenames (those are not really defined today,
  just expand them to be relative to /var/lib/xen/xenstore)
- expansion of xenstore_daemon_rundir() (used e.g. for saving the state
  file in case of live update - needs to be unchanged in the daemon
  case, but should result in /var/lib/xen/xenstore for stubdom)

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- make absolute_filename() return a pointer to const (Julien Grall)
---
 tools/xenstored/core.c      | 15 +++++++++++++--
 tools/xenstored/core.h      |  5 ++++-
 tools/xenstored/lu_daemon.c |  4 ++--
 tools/xenstored/minios.c    |  5 +++++
 tools/xenstored/posix.c     |  8 +++++++-
 5 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 48fc787ac1..bada1ad9a2 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -63,7 +63,7 @@ char **orig_argv;
 LIST_HEAD(connections);
 int tracefd = -1;
 bool keep_orphans = false;
-char *tracefile = NULL;
+const char *tracefile = NULL;
 static struct hashtable *nodes;
 unsigned int trace_flags = TRACE_OBJ | TRACE_IO;
 
@@ -137,6 +137,17 @@ void trace_destroy(const void *data, const char *type)
 		trace("obj: DESTROY %s %p\n", type, data);
 }
 
+/*
+ * Return an absolute filename.
+ * In case of a relative filename given as input, prepend XENSTORE_LIB_DIR.
+ */
+const char *absolute_filename(const void *ctx, const char *filename)
+{
+	if (filename[0] != '/')
+		return talloc_asprintf(ctx, XENSTORE_LIB_DIR "/%s", filename);
+	return talloc_strdup(ctx, filename);
+}
+
 void close_log(void)
 {
 	if (tracefd >= 0)
@@ -2759,7 +2770,7 @@ int main(int argc, char *argv[])
 #endif
 
 	if (tracefile)
-		tracefile = talloc_strdup(NULL, tracefile);
+		tracefile = absolute_filename(NULL, tracefile);
 
 #ifndef NO_LIVE_UPDATE
 	/* Read state in case of live update. */
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index fe0ee90581..e58779e88c 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -341,7 +341,7 @@ void close_log(void);
 extern int orig_argc;
 extern char **orig_argv;
 
-extern char *tracefile;
+extern const char *tracefile;
 extern int tracefd;
 
 /* Trace flag values must be kept in sync with trace_switches[] contents. */
@@ -405,6 +405,9 @@ void set_socket_fd(int fd);
 void mount_9pfs(void);
 #endif
 
+const char *xenstore_rundir(void);
+const char *absolute_filename(const void *ctx, const char *filename);
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/lu_daemon.c b/tools/xenstored/lu_daemon.c
index 71bcabadd3..6351111ab0 100644
--- a/tools/xenstored/lu_daemon.c
+++ b/tools/xenstored/lu_daemon.c
@@ -24,7 +24,7 @@ void lu_get_dump_state(struct lu_dump_state *state)
 	state->size = 0;
 
 	state->filename = talloc_asprintf(NULL, "%s/state_dump",
-					  xenstore_daemon_rundir());
+					  xenstore_rundir());
 	if (!state->filename)
 		barf("Allocation failure");
 
@@ -65,7 +65,7 @@ FILE *lu_dump_open(const void *ctx)
 	int fd;
 
 	filename = talloc_asprintf(ctx, "%s/state_dump",
-				   xenstore_daemon_rundir());
+				   xenstore_rundir());
 	if (!filename)
 		return NULL;
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 562a9b4972..e70386f8c7 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -128,3 +128,8 @@ void mount_9pfs(void)
 {
 	create_thread("mount-9pfs", mount_thread, NULL);
 }
+
+const char *xenstore_rundir(void)
+{
+	return XENSTORE_LIB_DIR;
+}
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 496329dfd1..d88c82d972 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -326,9 +326,10 @@ void early_init(bool live_update, bool dofork, const char *pidfile)
 {
 	reopen_log();
 
-	/* Make sure xenstored directory exists. */
+	/* Make sure xenstored directories exist. */
 	/* Errors ignored here, will be reported when we open files */
 	mkdir(xenstore_daemon_rundir(), 0755);
+	mkdir(XENSTORE_LIB_DIR, 0755);
 
 	if (dofork) {
 		openlog("xenstored", 0, LOG_DAEMON);
@@ -406,3 +407,8 @@ void set_socket_fd(int fd)
 {
 	sock = fd;
 }
+
+const char *xenstore_rundir(void)
+{
+	return xenstore_daemon_rundir();
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 17:31:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 17:31:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678453.1055760 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY8FU-0001qf-68; Thu, 08 Feb 2024 17:31:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678453.1055760; Thu, 08 Feb 2024 17:31:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY8FU-0001qY-2s; Thu, 08 Feb 2024 17:31:44 +0000
Received: by outflank-mailman (input) for mailman id 678453;
 Thu, 08 Feb 2024 17:31:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mdps=JR=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rY8FT-0001qS-6Y
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 17:31:43 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id eb64a81c-c6a7-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 18:31:40 +0100 (CET)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2cf5917f049so1365131fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 09:31:40 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 cx12-20020a170907168c00b00a3bcfe76170sm119527ejd.2.2024.02.08.09.31.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 08 Feb 2024 09:31:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb64a81c-c6a7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707413500; x=1708018300; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=vOKqfPtYD5AX98Cbs2+YqThcOaVb5WkLWqN42uxEHYI=;
        b=TO1JA352brXbB/BjrU1fEFcrlH508zQouWQH0+elaSClbqLJ5x5LM05mUz7594OWs3
         jt4tjIg/Ua5uP3dV/YS0iKz016tKXTzsx9Vc7/1HjOUZSLvLbxuj8JBPGoa/mSBiQZqg
         2q+22xZ7KIMgqSHq2Su/olhva/fl1TZcm6A3U=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707413500; x=1708018300;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vOKqfPtYD5AX98Cbs2+YqThcOaVb5WkLWqN42uxEHYI=;
        b=cn77FoXWdvIJqGY6RGC8i5zPKXf22B4qw7Rz9awTGGK4RpWBNyLbOkRwTmgtQJgf2T
         HgMYhgiz9pRW97YR7QgCROqXgOC5nU/zAlN9jFi1AaxZjKTbBEuNrGILEWgUgvBkRG8W
         gzHhQ7AfsVsn3H9YalRCalSIIEAcw5U+XIMKNH9RZMW5pmiLHwMSUZCdDmLtUYlVFCA+
         SnsNbg3GdrL2XPuo6d/rjQNvtyb90ModfVvH18zX7Ea7IxJk+9mVfHwMICg1A5Ktk2WS
         XuUP8X+w+Sopz8rYPPVDrsHJL3MLjc4eFP8oEgvWkoW+ccD4HedUB+Djm4DNc2animLQ
         Y7NA==
X-Gm-Message-State: AOJu0YzGXqkWYehKa1u+ZE6DLYx69k5G59xR60wmuLrhyxiLsKBdjIUp
	gouKh3HeJcyAPergPdlaUEbrFviYVLfqQtRIEzXooiqbFoecKZihlt2ljim9rkg=
X-Google-Smtp-Source: AGHT+IFxextErh7M4gWfyaT2Njuy6Op6P/1y8aX0KQoW/JbjiHrsZ2ob/M+5aa+NeuPTyWFxxNQLBA==
X-Received: by 2002:a2e:9211:0:b0:2d0:aafc:914c with SMTP id k17-20020a2e9211000000b002d0aafc914cmr28432ljg.1.1707413500051;
        Thu, 08 Feb 2024 09:31:40 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU5yA5s0TRpr2zJQPNryhdwWB1slmgWyKG9LI2WcZbw8wiLp2e9+7DVsCXp1IR4MTuOBdld1gwN/7zC1ho5Tk7RebNHZ/g6ADwmQHbhMlKOoweh
Date: Thu, 8 Feb 2024 18:31:38 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 4/7] VT-d: replace find_ats_dev_drhd()
Message-ID: <ZcUP-t5gFx2v31at@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <a75bef32-8bbb-4471-b26b-981b06173cd5@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <a75bef32-8bbb-4471-b26b-981b06173cd5@suse.com>

On Mon, Feb 05, 2024 at 02:56:36PM +0100, Jan Beulich wrote:
> All callers only care about boolean outcome. For this there's no point
> in allocating a duplicate of the respective DRHD structure; a simple
> boolean suffices (which eventually may wantg to become a count, such
                                         ^ want
> that the "any ATS devices assigned state" can also clear again). With
> that boolean, remove respective parameters from internal helper
> functions right away, as those have access to the flag through another
> parameter.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

AFAICT the intention is that this is a non-functional change?

> 
> --- a/xen/drivers/passthrough/vtd/extern.h
> +++ b/xen/drivers/passthrough/vtd/extern.h
> @@ -65,8 +65,6 @@ struct acpi_drhd_unit *ioapic_to_drhd(un
>  struct acpi_drhd_unit *hpet_to_drhd(unsigned int hpet_id);
>  struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd);
>  
> -struct acpi_drhd_unit *find_ats_dev_drhd(struct vtd_iommu *iommu);
> -
>  int ats_device(const struct pci_dev *, const struct acpi_drhd_unit *);
>  
>  int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -624,8 +624,7 @@ int cf_check vtd_flush_iotlb_reg(
>  }
>  
>  static int __must_check iommu_flush_iotlb_global(struct vtd_iommu *iommu,
> -                                                 bool flush_non_present_entry,
> -                                                 bool flush_dev_iotlb)
> +                                                 bool flush_non_present_entry)
>  {
>      int status;
>  
> @@ -633,7 +632,7 @@ static int __must_check iommu_flush_iotl
>      vtd_ops_preamble_quirk(iommu);
>  
>      status = iommu->flush.iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH,
> -                                flush_non_present_entry, flush_dev_iotlb);
> +                                flush_non_present_entry, iommu->flush_dev_iotlb);

Any reason to not also remove the parameter from here also?  As the handler
gets iommu passed as the first parameter anyway.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 18:24:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 18:24:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678457.1055770 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY94U-0003iX-UG; Thu, 08 Feb 2024 18:24:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678457.1055770; Thu, 08 Feb 2024 18:24:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rY94U-0003iQ-RY; Thu, 08 Feb 2024 18:24:26 +0000
Received: by outflank-mailman (input) for mailman id 678457;
 Thu, 08 Feb 2024 18:24:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rY94T-0003iG-D9; Thu, 08 Feb 2024 18:24:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rY94T-0004R1-36; Thu, 08 Feb 2024 18:24:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rY94S-00022i-Ms; Thu, 08 Feb 2024 18:24:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rY94S-0001RT-MM; Thu, 08 Feb 2024 18:24:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=aQSECd9Pja+1HmEIYq+oN57RYwo54R3eIwLf7qE5TSk=; b=IgnUMKSaI0oIR4ZNhQkIPTbbDO
	DKG71aZZ7o3OGcVe5FK7aKh+q2pCyhBO9RSFsaRO6m9Kig09Bu4rF7pcCIvm2JMGK7Gbyo+VjgQt+
	XpPjLjS0WAJJrqT1CyuDmTr8YGTzvfzZk6vK3g/rk8fXfVp843/RtoDGnhJmGSI2wpiI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184625-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184625: FAIL
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-multivcpu:<job status>:broken:regression
    linux-linus:test-armhf-armhf-xl-multivcpu:host-install(5):broken:heisenbug
    linux-linus:test-arm64-arm64-xl-credit1:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-rtds:host-ping-check-xen:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-amd64-amd64-libvirt-qcow2:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=547ab8fc4cb04a1a6b34377dd8fad34cd2c8a8e3
X-Osstest-Versions-That:
    linux=6d280f4d760e3bcb4a8df302afebf085b65ec982
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 08 Feb 2024 18:24:24 +0000

flight 184625 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184625/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-multivcpu    <job status>                 broken in 184620

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-multivcpu 5 host-install(5) broken in 184620 pass in 184625
 test-arm64-arm64-xl-credit1 18 guest-start/debian.repeat fail in 184620 pass in 184625
 test-armhf-armhf-xl-rtds  10 host-ping-check-xen fail in 184620 pass in 184625
 test-armhf-armhf-xl-credit1   8 xen-boot                   fail pass in 184620
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184620
 test-amd64-amd64-libvirt-qcow2 19 guest-start/debian.repeat fail pass in 184620

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit1 15 migrate-support-check fail in 184620 never pass
 test-armhf-armhf-xl-credit1 16 saverestore-support-check fail in 184620 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184614
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184614
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184614
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184614
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184614
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184614
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184614
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184614
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                547ab8fc4cb04a1a6b34377dd8fad34cd2c8a8e3
baseline version:
 linux                6d280f4d760e3bcb4a8df302afebf085b65ec982

Last test of basis   184614  2024-02-07 08:42:21 Z    1 days
Testing same since   184620  2024-02-07 19:40:30 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anup Patel <anup@brainfault.org>
  Anup Patel <apatel@ventanamicro.com>
  Bibo Mao <maobibo@loongson.cn>
  Borislav Petkov (AMD) <bp@alien8.de>
  Christian Borntraeger <borntraeger@linux.ibm.com>
  Chuck Lever <chuck.lever@oracle.com>
  David Rientjes <rientjes@google.com>
  Eric Farman <farman@linux.ibm.com>
  Huacai Chen <chenhuacai@loongson.cn>
  Janosch Frank <frankja@linux.ibm.com>
  Kees Cook <keescook@chromium.org>
  Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Nathan Chancellor <nathan@kernel.org>
  NeilBrown <neilb@suse.de>
  Paolo Bonzini <pbonzini@redhat.com>
  Tengfei Yu <moehanabichan@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               fail    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-armhf-armhf-xl-multivcpu broken

Not pushing.

(No revision log; it would be 652 lines long.)


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 19:30:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 19:30:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678465.1055779 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYA5j-0004UY-Hh; Thu, 08 Feb 2024 19:29:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678465.1055779; Thu, 08 Feb 2024 19:29:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYA5j-0004UR-FD; Thu, 08 Feb 2024 19:29:47 +0000
Received: by outflank-mailman (input) for mailman id 678465;
 Thu, 08 Feb 2024 19:29:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYA5i-0004UH-DA; Thu, 08 Feb 2024 19:29:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYA5i-0005tT-8S; Thu, 08 Feb 2024 19:29:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYA5h-0003Y9-Ob; Thu, 08 Feb 2024 19:29:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYA5h-0000YA-O6; Thu, 08 Feb 2024 19:29:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OdftXVo/SH86gvOPPWrz9EYo+Ibc27HvT/wZdJGNAiM=; b=NBZagNRUTE5f7XBLnrQ6i5hA0l
	BuvUPeTQToJRiZIq/P+z7X06LCkA3KFFmW9uhdRs8axzpjfbhOIiX/41GdICCTuugrDWQ1sRLbcrw
	o7G97GeeK3Wg3QwI6PbnGQ9UTvihxJT9hjvhI0sA4j7eLI0pzXT8f9PYynl8CBAviiFo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184627-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184627: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=e11f5766503c0ff074b4e0f888bbfc931518a169
X-Osstest-Versions-That:
    xen=f4519ee87ebfb74fe0451130d82c99453b75e2e1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 08 Feb 2024 19:29:45 +0000

flight 184627 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184627/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  e11f5766503c0ff074b4e0f888bbfc931518a169
baseline version:
 xen                  f4519ee87ebfb74fe0451130d82c99453b75e2e1

Last test of basis   184616  2024-02-07 13:03:56 Z    1 days
Testing same since   184627  2024-02-08 17:03:44 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Julien Grall <jgrall@amazon.com>
  Luca Fancellu <luca.fancellu@arm.com>
  Michal Orzel <michal.orzel@amd.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f4519ee87e..e11f576650  e11f5766503c0ff074b4e0f888bbfc931518a169 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 21:05:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 21:05:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678472.1055789 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYBaY-0000N4-Ew; Thu, 08 Feb 2024 21:05:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678472.1055789; Thu, 08 Feb 2024 21:05:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYBaY-0000Mx-C9; Thu, 08 Feb 2024 21:05:42 +0000
Received: by outflank-mailman (input) for mailman id 678472;
 Thu, 08 Feb 2024 21:05:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7cv+=JR=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rYBaX-0000Mr-Dc
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 21:05:41 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cdff2921-c6c5-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 22:05:37 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 36FE3CE1E5A;
 Thu,  8 Feb 2024 21:05:33 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 67BD0C433F1;
 Thu,  8 Feb 2024 21:05:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cdff2921-c6c5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707426332;
	bh=uXsBbejCKZT2gnWzgYDT9prt95WAobjzVIID+8hP2/w=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VkzE29WU6pxlECZMgFlcC+6TFZWyaluX3yvLieSujfVcaPXmJXaKlsuudNckYUGSw
	 Iln41SAvK//bZe3E0UnPhPVRR7fyzghTCk/l+IAKTp8rFe9LmVFVQm1t7jO2Z7WdCs
	 +GS0xlO2ZFm2YpZuz7LK1pyFKdnnBnS3/MfvubGg6em1a3nU27m6wZCbfKmKRESZHj
	 6yDRDS6p3qDIiTHawHFZDrNGN/+qOYe8yIJk8Z2VTAij1OfUzyHDdvxWyiHv74VSzC
	 8BrW3rTRqzdiY9QNLAKBPC7sDybXTYnYTrs8b9nf2rp59XO5udTErKFgBIW59tQyqb
	 9+Z/1A2mN7Anw==
Date: Thu, 8 Feb 2024 13:05:30 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Michal Orzel <michal.orzel@amd.com>
cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH] xen/serial: Drop serial_{put,get}c
In-Reply-To: <20240208141601.78023-1-michal.orzel@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402081305250.1925432@ubuntu-linux-20-04-desktop>
References: <20240208141601.78023-1-michal.orzel@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 8 Feb 2024, Michal Orzel wrote:
> After removal of gdbstub by commit d182f9feae6d ("x86: Remove gdbstub"),
> we no longer have users of serial_{put,get}c functions, so remove them.
> 
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/drivers/char/serial.c | 72 ---------------------------------------
>  xen/include/xen/serial.h  | 10 ------
>  2 files changed, 82 deletions(-)
> 
> diff --git a/xen/drivers/char/serial.c b/xen/drivers/char/serial.c
> index 324024c29a10..93e217d34859 100644
> --- a/xen/drivers/char/serial.c
> +++ b/xen/drivers/char/serial.c
> @@ -196,36 +196,6 @@ static void __serial_putc(struct serial_port *port, char c)
>      }
>  }
>  
> -void serial_putc(int handle, char c)
> -{
> -    struct serial_port *port;
> -    unsigned long flags;
> -
> -    if ( handle == -1 )
> -        return;
> -
> -    port = &com[handle & SERHND_IDX];
> -    if ( !port->driver || !port->driver->putc )
> -        return;
> -
> -    spin_lock_irqsave(&port->tx_lock, flags);
> -
> -    if ( (c == '\n') && (handle & SERHND_COOKED) )
> -        __serial_putc(port, '\r' | ((handle & SERHND_HI) ? 0x80 : 0x00));
> -
> -    if ( handle & SERHND_HI )
> -        c |= 0x80;
> -    else if ( handle & SERHND_LO )
> -        c &= 0x7f;
> -
> -    __serial_putc(port, c);
> -
> -    if ( port->driver->flush )
> -        port->driver->flush(port);
> -
> -    spin_unlock_irqrestore(&port->tx_lock, flags);
> -}
> -
>  void serial_puts(int handle, const char *s, size_t nr)
>  {
>      struct serial_port *port;
> @@ -261,48 +231,6 @@ void serial_puts(int handle, const char *s, size_t nr)
>      spin_unlock_irqrestore(&port->tx_lock, flags);
>  }
>  
> -char serial_getc(int handle)
> -{
> -    struct serial_port *port;
> -    char c;
> -    unsigned long flags;
> -
> -    if ( handle == -1 )
> -        return '\0';
> -
> -    port = &com[handle & SERHND_IDX];
> -    if ( !port->driver || !port->driver->getc )
> -        return '\0';
> -
> -    do {
> -        for ( ; ; )
> -        {
> -            spin_lock_irqsave(&port->rx_lock, flags);
> -            
> -            if ( port->rxbufp != port->rxbufc )
> -            {
> -                c = port->rxbuf[mask_serial_rxbuf_idx(port->rxbufc++)];
> -                spin_unlock_irqrestore(&port->rx_lock, flags);
> -                break;
> -            }
> -            
> -            if ( port->driver->getc(port, &c) )
> -            {
> -                spin_unlock_irqrestore(&port->rx_lock, flags);
> -                break;
> -            }
> -
> -            spin_unlock_irqrestore(&port->rx_lock, flags);
> -
> -            cpu_relax();
> -            udelay(100);
> -        }
> -    } while ( ((handle & SERHND_LO) &&  (c & 0x80)) ||
> -              ((handle & SERHND_HI) && !(c & 0x80)) );
> -    
> -    return c & 0x7f;
> -}
> -
>  int __init serial_parse_handle(const char *conf)
>  {
>      int handle, flags = 0;
> diff --git a/xen/include/xen/serial.h b/xen/include/xen/serial.h
> index 12ef24351d0f..7548bb73249b 100644
> --- a/xen/include/xen/serial.h
> +++ b/xen/include/xen/serial.h
> @@ -112,19 +112,9 @@ void serial_endboot(void);
>  /* Takes a config string and creates a numeric handle on the COM port. */
>  int serial_parse_handle(const char *conf);
>  
> -/* Transmit a single character via the specified COM port. */
> -void serial_putc(int handle, char c);
> -
>  /* Transmit a string via the specified COM port. */
>  void serial_puts(int handle, const char *s, size_t nr);
>  
> -/*
> - * An alternative to registering a character-receive hook. This function
> - * will not return until a character is available. It can safely be
> - * called with interrupts disabled.
> - */
> -char serial_getc(int handle);
> -
>  /* Forcibly prevent serial lockup when the system is in a bad way. */
>  /* (NB. This also forces an implicit serial_start_sync()). */
>  void serial_force_unlock(int handle);
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 21:20:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 21:20:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678477.1055799 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYBot-0003Tw-Qh; Thu, 08 Feb 2024 21:20:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678477.1055799; Thu, 08 Feb 2024 21:20:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYBot-0003Tp-O7; Thu, 08 Feb 2024 21:20:31 +0000
Received: by outflank-mailman (input) for mailman id 678477;
 Thu, 08 Feb 2024 21:20:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jPix=JR=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1rYBos-0003Tj-Aa
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 21:20:30 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e1f98bbd-c6c7-11ee-8a4b-1f161083a0e0;
 Thu, 08 Feb 2024 22:20:29 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-41061f068cdso633355e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 13:20:28 -0800 (PST)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 r15-20020a05600c458f00b0040e4733aecbsm488854wmo.15.2024.02.08.13.20.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 08 Feb 2024 13:20:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1f98bbd-c6c7-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707427228; x=1708032028; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Lx8PQ+gRbkIzFQZ1WKf4xwBqXzUF6EuRsqMcRxbl6vY=;
        b=MiQ0w3uDaIEKgHr3qO7jZYMqgcH/jzf2gaDD1/PXvjps69eNTcwGB0JV/Jv9R1SHix
         8nTlDVCl9XE1vo00k2imeiv/AGMA10KIyRGpZBnRDgMDXISnsAefqPq1KiysFwDnsEqg
         /5ux6aFsTD6GZvK5l2KmtXvwj2RhVlwDJr3o/F4q/D/mHEHmvS4bznTl3LyQGi11a2Xd
         giqKsR9Wh5/TW1TdaQ3BH96Pks04WB+CDifQSSUZGa2nhvET2TlK+cKre4eXC9qKrC8I
         hH+MHVEqNze8Zms67U/Ji6aVF2V2wnuHS7BF2uRs/2KDMDGPf3d+e/wUaFpwLM3CaIwI
         vuMQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707427228; x=1708032028;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Lx8PQ+gRbkIzFQZ1WKf4xwBqXzUF6EuRsqMcRxbl6vY=;
        b=G7pt9dHewFZ/N5CHd9Yd3/hjHeha8/xGTYL05mjpUTVFPLPyFYZdL1buaE/pY5SHXt
         PplAWD31ruutIIT4ypZxvvmhUeC4qbKL/M3YmyK+qBvs9WocG5L3Scykv9ZEK+Uyf5vq
         7gEjuHYVmxVWVJW2pALyNMysQ+FwJVsGxxZczdRRMndOsYgvU+rIP1CoYrd0sGkAwMKn
         1uSrXDaekkgeVt0G9AS+84CSoNMFEH3yqFn6gnph4oof3ooEHzbr8d74s5nMmIJ7TgTu
         IFH9HoeFb27zKWGbhe3UTUXHDjwZ6LI5h5mHkqslDmAYgGXujFAYZ9jtKAXjo//zryzj
         ARRA==
X-Gm-Message-State: AOJu0YyPAsC1b1SvaNHcOnTUS3XJRM38ugc17zEcRY03R8Yf/6ub2KsZ
	UXsCbDb2XdxE0F09bT1BJ++g85l4MpW1hTLThQhkzi/gKxAWjY5EtZH1SJSB/9c=
X-Google-Smtp-Source: AGHT+IH0tDqHgTsyeGubGIKfGffBDEuPdX5Xj/EoEztMoHNa1z5bdeBDppGFVQcOvglw7Pgw4r02XA==
X-Received: by 2002:a05:600c:4e88:b0:410:41f5:368a with SMTP id f8-20020a05600c4e8800b0041041f5368amr389118wmq.18.1707427227770;
        Thu, 08 Feb 2024 13:20:27 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVd3vrxY3xqYwd00BW1JbLIZBfZOVkd1koUdgN9V0pMXP6z0gAARQ2NpkmkVGGEZ1X+NrrKdG7ziU7Ze6OtHU22oMHIx6PB3hGNUp3BeHcDQ8W0Ffc6p0iP+Y4o9E71iqVl35V2IOPBarjU2xo2AnJgd+nKjofMdbSCiyMy3YKlLL5S4LbyU/7dZ/yXGcOtq1GN
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: tamas@tklengyel.com,
	=?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/hvm: Fix fast singlestep state persistence
Date: Thu,  8 Feb 2024 21:20:16 +0000
Message-Id: <fa519b9a544d3d19a31313292a909d12775e6e1f.1707427103.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

This patch addresses an issue where the fast singlestep setting would persist
despite xc_domain_debug_control being called with XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF.
Specifically, if fast singlestep was enabled in a VMI session and that session
stopped before the MTF trap occurred, the fast singlestep setting remained
active even though MTF itself was disabled.  This led to a situation where, upon
starting a new VMI session, the first event to trigger an EPT violation would
cause the corresponding EPT event callback to be skipped due to the lingering
fast singlestep setting.

The fix ensures that the fast singlestep setting is properly reset when
disabling single step debugging operations.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
---
 xen/arch/x86/hvm/hvm.c | 32 +++++++++++++++++++++++---------
 1 file changed, 23 insertions(+), 9 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index e8deeb0222..4f988de4c1 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -5160,26 +5160,40 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
 
 int hvm_debug_op(struct vcpu *v, int32_t op)
 {
-    int rc;
+    int rc = 0;
 
     switch ( op )
     {
         case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON:
         case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF:
-            rc = -EOPNOTSUPP;
             if ( !cpu_has_monitor_trap_flag )
-                break;
-            rc = 0;
-            vcpu_pause(v);
-            v->arch.hvm.single_step =
-                (op == XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON);
-            vcpu_unpause(v); /* guest will latch new state */
+                return -EOPNOTSUPP;
             break;
         default:
-            rc = -ENOSYS;
+            return -ENOSYS;
+    }
+
+    vcpu_pause(v);
+
+    switch ( op )
+    {
+        case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON:
+            v->arch.hvm.single_step = true;
+            break;
+
+        case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF:
+            v->arch.hvm.single_step = false;
+            v->arch.hvm.fast_single_step.enabled = false;
+            v->arch.hvm.fast_single_step.p2midx = 0;
             break;
+
+        default:
+            ASSERT_UNREACHABLE();
+            return -ENOSYS;
     }
 
+    vcpu_unpause(v); /* guest will latch new state */
+
     return rc;
 }
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 08 21:30:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 21:30:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678481.1055810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYByh-0005La-Ng; Thu, 08 Feb 2024 21:30:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678481.1055810; Thu, 08 Feb 2024 21:30:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYByh-0005LT-Ky; Thu, 08 Feb 2024 21:30:39 +0000
Received: by outflank-mailman (input) for mailman id 678481;
 Thu, 08 Feb 2024 21:30:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rYByg-0005LN-5j
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 21:30:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYBye-0007vN-IL; Thu, 08 Feb 2024 21:30:36 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.102])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYBye-0003d4-BD; Thu, 08 Feb 2024 21:30:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=qLuY6Xv9ODJMnAYKAyM+fFuhel2IAtcX/BmcTYKANqg=; b=zxi3l04EdWxoLyYeCvznLofZ9k
	IrFzQOUrV3i7aEXXLA5NFVBwDeoIml9rczuIrsl0157Q0XoPMUwW58CRfpeRAApWyom5HyG2ARsV8
	p6HN1BUGA0tW0gIv51xPic77OLFM3UoZ/yu3wUKj8VUlAOTGAjSj1A7tHvcnfkIAuJsg=;
Message-ID: <07e4b9b0-5ddd-4499-b280-bd5d583b8b8c@xen.org>
Date: Thu, 8 Feb 2024 21:30:33 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(),
 ASSERT_UNREACHABLE() to xen/bug.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>, Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>
References: <4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 02/02/2024 18:03, Oleksii Kurochko wrote:
> Move the macros mentioned in the commit subject to their appropriate
> locations.
> Additionally, eliminate the dependency of xen/lib.h from xen/bug.h and
> include "xen/bug.h" in files where xen/bug.h macros are utilized.
> 
> Most of the changes were made because a file requires macros from xen/bug.h,
> except for some files for Arm which require definitions of BUG_OPCODE,
> BUG_INSTR, BUG_FN_REG.
> 
> xen/lib.h was added to list-sort.c ( otherwise compilation errors related
> to {d}printk occur during compilation of list-sort.c. ) as xen/lib.h was
> removed from xen/list.h. Since nothing in xen/list.h depends on xen/lib.h
> functionality and only xen/bug.h is needed.
> 
> cpufeature.h requires the inclusion of <xen/cache.h>;
> otherwise, the following error will occur:
> ld: common/monitor.o:/build/xen/./arch/x86/include/asm/cpufeature.h:41:
> multiple definitions of `__cacheline_aligned';
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 21:33:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 21:33:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678484.1055819 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYC1O-0005tC-5U; Thu, 08 Feb 2024 21:33:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678484.1055819; Thu, 08 Feb 2024 21:33:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYC1O-0005t5-29; Thu, 08 Feb 2024 21:33:26 +0000
Received: by outflank-mailman (input) for mailman id 678484;
 Thu, 08 Feb 2024 21:33:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rYC1N-0005sC-CB
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 21:33:25 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYC1M-0007zj-OJ; Thu, 08 Feb 2024 21:33:24 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.102])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYC1M-0003pD-J3; Thu, 08 Feb 2024 21:33:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=BW5qiyt7AQfECNgwOhZqLpmZVcPreDUv8vn3ys0CSp4=; b=qkGt7yt+8Ohf49EWTyWEtWfApm
	lDzouXqEQ4pqiIJ+0cDbXWJPAyqT8DiPU6wacmYKFx/Rvxawwk8zXtY1zoFZqB4reUw+o0RDgfiRz
	NhwGUBKBBwUAXL8ja7xIow2afwf+kl5fjfL74wEL+jAE8epMwiaki81LnstTzQ7kQ0JU=;
Message-ID: <ccc568a3-d81c-482c-8768-c5934e3943a8@xen.org>
Date: Thu, 8 Feb 2024 21:33:22 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] docs/misra: add x86_64 and arm64 asm-offset.c to
 exclude-list
Content-Language: en-GB
To: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com, jbeulich@suse.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
References: <b0c855581eed247a32b745906f84d352bf812091.1707324479.git.nicola.vetrini@bugseng.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <b0c855581eed247a32b745906f84d352bf812091.1707324479.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Nicola,

On 08/02/2024 07:55, Nicola Vetrini wrote:
> These two files contain several deliberate violations of MISRA C rules and
> they are not linked in the final Xen binary, therefore they can be exempted
> from MISRA compliance.

I am curious, what are the violations you are talking about?

Cheers,

>  > No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> ---
> Since the exclude list only contains arm64 and x86 files I reasoned that
> introducing an entry that would match all architectures would not be desirable
> (e.g., arm32). I'm happy to change that, though.
> ---
>   docs/misra/exclude-list.json | 8 ++++++++
>   1 file changed, 8 insertions(+)
> 
> diff --git a/docs/misra/exclude-list.json b/docs/misra/exclude-list.json
> index 7971d0e70f5b..bd05478e03e9 100644
> --- a/docs/misra/exclude-list.json
> +++ b/docs/misra/exclude-list.json
> @@ -17,6 +17,10 @@
>               "rel_path": "arch/arm/arm64/lib/find_next_bit.c",
>               "comment": "Imported from Linux, ignore for now"
>           },
> +        {
> +          "rel_path": "arch/arm/arm64/asm-offsets.c",
> +          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
> +        },
>           {
>               "rel_path": "arch/x86/acpi/boot.c",
>               "comment": "Imported from Linux, ignore for now"
> @@ -97,6 +101,10 @@
>               "rel_path": "arch/x86/x86_64/mmconf-fam10h.c",
>               "comment": "Imported from Linux, ignore for now"
>           },
> +        {
> +          "rel_path": "arch/x86/x86_64/asm-offsets.c",
> +          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
> +        },
>           {
>               "rel_path": "arch/x86/efi/check.c",
>               "comment": "The resulting code is not included in the final Xen binary, ignore for now"

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 21:35:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 21:35:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678487.1055829 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYC3U-0006RB-HQ; Thu, 08 Feb 2024 21:35:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678487.1055829; Thu, 08 Feb 2024 21:35:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYC3U-0006R4-E6; Thu, 08 Feb 2024 21:35:36 +0000
Received: by outflank-mailman (input) for mailman id 678487;
 Thu, 08 Feb 2024 21:35:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jPix=JR=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1rYC3T-0006Qy-84
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 21:35:35 +0000
Received: from mail-oa1-x34.google.com (mail-oa1-x34.google.com
 [2001:4860:4864:20::34])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fc777f29-c6c9-11ee-98f5-efadbce2ee36;
 Thu, 08 Feb 2024 22:35:33 +0100 (CET)
Received: by mail-oa1-x34.google.com with SMTP id
 586e51a60fabf-214410e969cso557221fac.0
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 13:35:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc777f29-c6c9-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707428131; x=1708032931; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=3Mh+xWIVTtqZw4i0bVaC8y7Z/eeQ+wmiQoKLgVaf7JA=;
        b=FdgxPk5gt6pOiLnWe2Y+wqWMN1Re5Hto4ER+C3bfR5HO+8XmMe+K6/DBxz4JPCSdc3
         9jstggOc4LBhARosWJspMIv7gx2EByqZoCjfV+HzCkc2I9xJVN2OYFa1e9QJJzY6D+my
         nXaoa14p23AfoF8/iVbWPDcNUoRKP6RLU+F5FT9bQgpnsmP92E4A6RXdsGGwxq1ty/RI
         dOLBD8tffravNK/8v6nemR/q22Smdb6hocDArsp8TMbDfYndKIXODX/ZGD5jpV9osTaK
         aYxsWp2VDGo5bLHd1ButrZ7W8GfrLci/xDA4PxMUmWMab5UgcNe5uKBhiw1ytUTt9N+n
         gWLg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707428131; x=1708032931;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=3Mh+xWIVTtqZw4i0bVaC8y7Z/eeQ+wmiQoKLgVaf7JA=;
        b=lw+c/1LuWu1DKI1MSDcjh8J15DIWNylgPjx4DG/JpVrZda7H3qYhMLYe5EVOwn7Vm0
         5NWYMfigd0u58OKy+XUqezpLnLXDTs0PBn85AwFRVKX8qFb4Pa3AD+kD8bHxzF+9YFp0
         RgPomXX7P3WGsSuxK7baz8YrxoWSRNtpCH4WU4SQYPl8HRxGZOK9rAeSOhrCj+F5gcnR
         6qZnfEMXsOiK+rzkh+hlt/9rsusuAGAbswFC1bx7zVCPuk9dl8IXcO4a7Grpu6v6mkwK
         z0ndmFhNsbza1gryX2aMhP3IxeWma98Z582oc674jUyfXg3lu05lSYrCdTNdTQ/xU6YY
         vIUg==
X-Gm-Message-State: AOJu0YyEUQVtS75Bslhhlcz78NXM7lOIPvlEgKv2C9DGlWBnevQBNeFy
	DtAxjyj/+7d2g9BEs2pEAAoHxCJVyo3roVtHicyFasOkHtj+xh7mBjkwYRiqVGsRS3PWUZFJrGT
	fuhH//XZZE8tDdW5duN8/VHBS5e0=
X-Google-Smtp-Source: AGHT+IHfFjTyxsLbTPD/HdzbnDuyruvrDDAKk0A/2doxdRvMNO/PSk8n+hDuTVEr8ggOLJIpaxd/nUANy7d7AlhTa9A=
X-Received: by 2002:a05:6871:721:b0:214:fb22:c9b9 with SMTP id
 f33-20020a056871072100b00214fb22c9b9mr1994270oap.26.1707428131585; Thu, 08
 Feb 2024 13:35:31 -0800 (PST)
MIME-Version: 1.0
References: <9b9909c9e93cb540b3488c784935acc2bc9e071e.1707343396.git.w1benny@gmail.com>
 <3A858D7F-C953-4EF0-8919-AE96D6105AB1@cloud.com>
In-Reply-To: <3A858D7F-C953-4EF0-8919-AE96D6105AB1@cloud.com>
From: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Date: Thu, 8 Feb 2024 22:35:20 +0100
Message-ID: <CAKBKdXijXDcSnnm2P9kTe74924foiBxRks7mpxwE+q4rjiEbAA@mail.gmail.com>
Subject: Re: [PATCH] tools/ocaml: Add missing vmtrace_buf_kb field
To: Christian Lindig <christian.lindig@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, David Scott <dave@recoil.org>, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

> On Thu, Feb 8, 2024 at 10:14=E2=80=AFAM Christian Lindig <christian.lindi=
g@cloud.com> wrote:
>
> > On 7 Feb 2024, at 22:04, Petr Bene=C5=A1 <w1benny@gmail.com> wrote:
> >
> >
> > Add the missing `vmtrace_buf_kb` field to the OCaml bindings to match t=
he
> > vm.cfg configuration, correcting an oversight from its initial introduc=
tion.
> >
> > Signed-off-by: Petr Bene=C5=A1 <w1benny@gmail.com>
>
> Acked-by: Christian Lindig <christian.lindig@cloud.com>
>
> This looks correct from an OCaml perspective. Why was the new field added=
 in the middle of the record type domctl_create_config and thus forcing cha=
nges to the index of fields coming later in the record versus just appendin=
g the new field to the record type?
>

To match the position of the field inside of xen_domctl_createdomain.

P.


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 21:36:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 21:36:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678489.1055840 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYC4V-0006wp-RF; Thu, 08 Feb 2024 21:36:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678489.1055840; Thu, 08 Feb 2024 21:36:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYC4V-0006wi-NV; Thu, 08 Feb 2024 21:36:39 +0000
Received: by outflank-mailman (input) for mailman id 678489;
 Thu, 08 Feb 2024 21:36:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rYC4U-0006wa-Dy
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 21:36:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYC4T-00082C-D1; Thu, 08 Feb 2024 21:36:37 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.102])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYC4T-0003zo-3w; Thu, 08 Feb 2024 21:36:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=JsFidKjUcQAhbrMELN81Rxg6LTUTc51hbT9cqv8W31k=; b=2oB/P3RFCV6VYfSjlfALc3Luqb
	W00S/1BhR/Di8cAhpoaixaCrkWaxpGuZOV0XGgwsSG/v7c1wEYfpI/p1Jk9sXsg8COe75uIR6GVWJ
	uF/DHLv0SB2ejggl1BsaQiQOD9d605sIMDdVNCsZZ25x1kvR9V1RIR0HjwabBPFL0lvE=;
Message-ID: <6d1b1ec4-0502-42f3-a30f-2633ae14c553@xen.org>
Date: Thu, 8 Feb 2024 21:36:34 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 8/8] consolidate do_bug_frame() / bug_fn_t
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <f20d2cef-4435-4b9a-9ad8-61ceddb6bcba@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <f20d2cef-4435-4b9a-9ad8-61ceddb6bcba@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 05/02/2024 13:32, Jan Beulich wrote:
> The type not being used in do_bug_frame() is suspicious. Apparently
> that's solely because the type uses a pointer-to-const parameter,
> when so far run_in_exception_handler() wanted functions taking pointer-
> to-non-const. Expand use of const, in turn requiring common code's
> do_bug_frame() as well as [gs]et_irq_regs() to also gain const. This
> then brings the former function also closer to the common one, with
> Arm's use of vaddr_t remaining as a difference.
> 
> While there also replace the bogus use of hard tabs in [gs]et_irq_regs()
> (I clearly didn't mean to put it in like this).
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

For the Arm parts:

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 21:39:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 21:39:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678494.1055850 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYC6l-00089K-5V; Thu, 08 Feb 2024 21:38:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678494.1055850; Thu, 08 Feb 2024 21:38:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYC6l-00089D-2k; Thu, 08 Feb 2024 21:38:59 +0000
Received: by outflank-mailman (input) for mailman id 678494;
 Thu, 08 Feb 2024 21:38:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYC6j-000893-SK; Thu, 08 Feb 2024 21:38:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYC6j-000866-RV; Thu, 08 Feb 2024 21:38:57 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYC6j-0000yv-ED; Thu, 08 Feb 2024 21:38:57 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYC6j-0008JA-Ds; Thu, 08 Feb 2024 21:38:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Bo6YdMn6yUx7q/Y0AssrLCFCCeqiXzJCsSV+hzylBJ8=; b=xyxlgaFWm5jDR9qdgFx8M8S7S0
	z9mtz6nOts0KtfuTrJDe6oMgbCfHihnm4OI+V43BAuIIOwSDljezzh6h4G7XpFJIxcTXlphLwvh7V
	zzQVxAxZ4zhmoixXIseCZlS/cdy9pkev05ZebVzce/q9XMQLM2NuqJ20by8cAJF1zwC8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184629-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184629: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=e32b58ab5a12d37c82327f28376e7d12cccc8b3a
X-Osstest-Versions-That:
    ovmf=8f316e99ec8de9dea294f6751dd7457f9f1a828c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 08 Feb 2024 21:38:57 +0000

flight 184629 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184629/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 e32b58ab5a12d37c82327f28376e7d12cccc8b3a
baseline version:
 ovmf                 8f316e99ec8de9dea294f6751dd7457f9f1a828c

Last test of basis   184623  2024-02-08 04:13:04 Z    0 days
Testing same since   184629  2024-02-08 19:43:01 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ashraf Ali S <ashraf.ali.s@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   8f316e99ec..e32b58ab5a  e32b58ab5a12d37c82327f28376e7d12cccc8b3a -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 22:00:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 22:00:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678504.1055860 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYCRY-0004Qp-Ul; Thu, 08 Feb 2024 22:00:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678504.1055860; Thu, 08 Feb 2024 22:00:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYCRY-0004Qi-R4; Thu, 08 Feb 2024 22:00:28 +0000
Received: by outflank-mailman (input) for mailman id 678504;
 Thu, 08 Feb 2024 22:00:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rYCRX-0004Qc-Qq
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 22:00:27 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYCRW-0008SK-Dh; Thu, 08 Feb 2024 22:00:26 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.102])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYCRW-00056e-61; Thu, 08 Feb 2024 22:00:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=xqIygMEEMqNnKk6z7W/Cg0G0pnuI7QEEgqbBRap4qYE=; b=ovQw6dY4et1496NnnQyfvAFG68
	FT0o/AFzPSCdsYRtDr2JGqVxWVI+mNXmv6HBWPHohdylsgGGE366JKHl2U5N2MVGtwBV+n4Bj8pxU
	8vxEBqyi4BVROl37vnnt4gHNXRUtgt0gcc2eXSVKu8rHX4sS5ixNN+LYPn0ZpZEBczSY=;
Message-ID: <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
Date: Thu, 8 Feb 2024 22:00:24 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Marek Marczykowski <marmarek@invisiblethingslab.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 05/02/2024 13:27, Jan Beulich wrote:
> In preparation of dropping the register parameters from
> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
> register state needs making available another way for the few key
> handlers which need it. Fake IRQ-like state.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsistent with
> other console poll functions we have, and it's unclear whether that's
> actually generally correct.

Is it? Looking at ns16550_poll() we would pass guest_user_regs() if 
run_in_exception() doesn't exist. But looking at the caller, no-on seems 
to care about the 'regs'. So is this just a latent bug?

BTW, do you have an idea why the poll function is not run in an 
exception handler?

> 
> Andrew suggested to move set_irq_regs() to BUGFRAME_run_fn handling;
> it's not clear to me whether that would be (a) correct from an abstract
> pov (that's exception, not interrupt context after all) 

I agree with that.

> and (b) really beneficial.

I guess this could help to reduce the amount of churn. I can't really 
make my mind whether this is worth it or not. So I would keep it as you did.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 08 22:10:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 08 Feb 2024 22:10:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678509.1055870 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYCap-00068E-TY; Thu, 08 Feb 2024 22:10:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678509.1055870; Thu, 08 Feb 2024 22:10:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYCap-00067c-QG; Thu, 08 Feb 2024 22:10:03 +0000
Received: by outflank-mailman (input) for mailman id 678509;
 Thu, 08 Feb 2024 22:10:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rYCao-0005nd-43
 for xen-devel@lists.xenproject.org; Thu, 08 Feb 2024 22:10:02 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYCan-0000BQ-6Z; Thu, 08 Feb 2024 22:10:01 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.102])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYCan-0005XU-1O; Thu, 08 Feb 2024 22:10:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=Ahx8HJ9+2fDL1QsSyj6wZYq1mf3TI5yQerJobR7IVJg=; b=hr2M/ZU0ip6znJhYTkdVSpDZpe
	su+1kd453cjoJPby/y1DkUB1h2+fm19F9ra69PQhztjBFYhv9SSvDXfR5v1UYBEaVwA6RI552Sjap
	oLXuDHZ2rkqCfPZkG2c+etitvygVR9wz6c4xNP6Hupf4plwg1T8M80HjfOmrn/0Q4Fmo=;
Message-ID: <0bbe2028-946c-459f-b6cc-419869d22b35@xen.org>
Date: Thu, 8 Feb 2024 22:09:58 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/8] keyhandler: drop regs parameter from
 handle_keyregs()
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <5258d8a9-a7ef-4342-9b5c-fc0078139bb2@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <5258d8a9-a7ef-4342-9b5c-fc0078139bb2@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 05/02/2024 13:28, Jan Beulich wrote:
> In preparation for further removal of regs parameters, drop it here. In
> the two places where it's actually needed, retrieve IRQ context if
> available, or else guest context.
> 
> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> As an alternative to the new boolean parameter, I wonder if we couldn't
> special-case the idle vCPU case: It's only there where we would not have
> proper context retrievable via guest_cpu_user_regs().

I am trying to understand the implication. Looking at the code, it seems 
in the case where we pass NULL, we would expect to call 
run_in_exception_handler().

If I am not mistaken, at least for Arm, regs would not be the same as 
guest_cpu_user_regs(). So I think your current approach is more correct.

Did I miss anything?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 00:15:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 00:15:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678513.1055880 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYEYJ-0007HQ-LE; Fri, 09 Feb 2024 00:15:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678513.1055880; Fri, 09 Feb 2024 00:15:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYEYJ-0007HJ-I8; Fri, 09 Feb 2024 00:15:35 +0000
Received: by outflank-mailman (input) for mailman id 678513;
 Fri, 09 Feb 2024 00:15:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYEYH-0007Gp-Vg; Fri, 09 Feb 2024 00:15:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYEYH-0002ut-Mj; Fri, 09 Feb 2024 00:15:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYEYH-0005cS-6k; Fri, 09 Feb 2024 00:15:33 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYEYH-0005Bq-6K; Fri, 09 Feb 2024 00:15:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SN5qwWc2X6YFNOhoiUiXIpeDHM5pgd71OGc5r2XqBcE=; b=DXP3XcWG302IHVxW6FwlsfKW6I
	i5C6K+flZzx6Ke51SWKmkz06av2H293SYC5QbKoVBG0ZfxqQJgUj/HTGGlu4yR2eyQAt6U0SzNKvB
	kVji9OAvYeT19+Y0CEY3guYFBYaezbBYCQ/JswAydPyJLTrUPiAzSIfWzCn1phl995GM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184626-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184626: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f4519ee87ebfb74fe0451130d82c99453b75e2e1
X-Osstest-Versions-That:
    xen=f4519ee87ebfb74fe0451130d82c99453b75e2e1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 09 Feb 2024 00:15:33 +0000

flight 184626 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184626/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184622
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install    fail pass in 184622

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop fail in 184622 blocked in 184626
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184622
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184622
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184622
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184622
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184622
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184622
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184622
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184622
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184622
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184622
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184622
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  f4519ee87ebfb74fe0451130d82c99453b75e2e1
baseline version:
 xen                  f4519ee87ebfb74fe0451130d82c99453b75e2e1

Last test of basis   184626  2024-02-08 11:36:07 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 03:02:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 03:02:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678527.1055890 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYH9W-0005RI-DG; Fri, 09 Feb 2024 03:02:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678527.1055890; Fri, 09 Feb 2024 03:02:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYH9W-0005RB-9h; Fri, 09 Feb 2024 03:02:10 +0000
Received: by outflank-mailman (input) for mailman id 678527;
 Fri, 09 Feb 2024 03:02:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vW4Z=JS=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rYH9U-0005R5-Mo
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 03:02:08 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 99ee2988-c6f7-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 04:02:03 +0100 (CET)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-5601eb97b29so1021461a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 19:02:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99ee2988-c6f7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707447723; x=1708052523; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=elgHtFhGeyKWc7iSS6Fo66vWWF/m+4FwK52x2N/jEyo=;
        b=ASAMhuIZQvMPZLlPeM4nXi5mHeqS0EbbeyzgZA0Eg43zCPSKsJCo8yQF4jel89hDyu
         p2eOJJvF9g+Tsep1mkqkNqfE7MHj2BbgAikJjtR/inH08bZ7HQf0aiB7E1B9ahp4Q7/y
         iuUzYMjwsqEtperaJybVxxo7LojFr4bek1QEWccOGKFoK04FAiwtoWuGVwlf3sRj02jH
         gmuNssNklzq/20t9mzwm8xHBJVFRgf68lFvE4U5vYKfwNP12cvd6FzPJYbRiuStFurX+
         ts2mvNKIV4JPrkrP5HLGgpnMtiCEnSHcaghbPRnL1OYKm/dZWVNUxQRxb4AlP+pPKx0m
         BUVw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707447723; x=1708052523;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=elgHtFhGeyKWc7iSS6Fo66vWWF/m+4FwK52x2N/jEyo=;
        b=uwUS5ZNhcKLrHAiHWjgFnf/brG4pc5WEg3HPD2mxWniFzSTIgWJL11LQbr947Ge3ma
         1Z2wwQNNzdu5NFVZEDpdsUYiZv7eJfoff3wN8FAhJfs9CE/Y5v2KYAXhAySTyJXsQK5Q
         As0NukGFT7xZNyxg4qDrF64Gi+UvGmwhn0fuEigKd2xf4R8R/J5uXWLZtLFp4K4+OeQY
         BgN6njFSdhHKhXOa4IanaQJuF6peRNZl6L2Zjg1FY/8Op7xOFDJg/zhFPtGFN5uBuI3H
         yWa/yQ52Ce9NcL/HG+kNZjHhJx/WxljKN8dDMGhD82uj6EwgNF8kZTd5t/Oxsa9vUnFX
         ioyA==
X-Gm-Message-State: AOJu0YySWLEvlOGvA+Q9iw2ELRwXAKNIfUYuZ6hr7aCJmiBDGB/1D34Y
	2Id7TV/U3YXq/luAGZF0FJlEUYQTz9K2aUt8fNHyW8ta0cOfMii9hcgA6ffWYIZrX0seP6g7gPj
	NudcKuKprhNblpkTS9f69Yx9mHN4=
X-Google-Smtp-Source: AGHT+IFiqkleSYckTun/pHSbY+1Q05t4X07O53MEjk0VGM3pTQHciXTAXWWHRE0XESHTh5B2OFtNt5kfYxZsco6LseA=
X-Received: by 2002:a05:6402:5169:b0:55f:5197:6314 with SMTP id
 d9-20020a056402516900b0055f51976314mr259427ede.18.1707447723066; Thu, 08 Feb
 2024 19:02:03 -0800 (PST)
MIME-Version: 1.0
References: <20240208165546.5715-1-jgross@suse.com> <20240208165546.5715-15-jgross@suse.com>
In-Reply-To: <20240208165546.5715-15-jgross@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Thu, 8 Feb 2024 22:01:51 -0500
Message-ID: <CAKf6xptUVYzGyh7hnx2f=v0_tSEpJujc0hL3Qnyy=aNLu5hxVg@mail.gmail.com>
Subject: Re: [PATCH v5 14/22] tools/9pfsd: add 9pfs read request support
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 8, 2024 at 11:57=E2=80=AFAM Juergen Gross <jgross@suse.com> wro=
te:
>
> Add the read request of the 9pfs protocol.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
> V2:
> - make error check more readable (Jason Andryuk)
> V4:
> - add directory read support
> V5:
> - rewinddir() if reading a directory and offset is 0 (Jason Andryuk)

FTAOD with the v5 change:
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 03:09:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 03:09:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678531.1055900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYHGB-0006O4-04; Fri, 09 Feb 2024 03:09:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678531.1055900; Fri, 09 Feb 2024 03:09:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYHGA-0006Nx-TS; Fri, 09 Feb 2024 03:09:02 +0000
Received: by outflank-mailman (input) for mailman id 678531;
 Fri, 09 Feb 2024 03:09:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vW4Z=JS=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rYHG9-0006Nr-Tn
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 03:09:01 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 92792d0d-c6f8-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 04:09:00 +0100 (CET)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-5600c43caddso597357a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 19:09:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92792d0d-c6f8-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707448140; x=1708052940; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2OcjvyFdFfN0YTA7X6657JNcd4izQVcCS3w/UHKd7y4=;
        b=i4KCilLpH//ky8A8F5KKrGWlS1yqa9ZAzLMzF3TG4ycDnTDtXopLMNsxZ/HbHelvVL
         IHRR6Ho6Xa7d8KXiJTIh9YwIiKBk+c4DTEMbvw9/kGdu17U9nOWQBffpvxpXciwHUDqF
         J88Ns/aHUvfX1IGgfui1QbROyj0ig86PY3qo1Pv22tPMQoOKMVCG0/T4OdXoSsn9ibkv
         s8y76uSpC0MWH8JXzk2mUXR3LOcdkT7CP77TzAASjPIWuTfArsbo9a6etLxpWYxbZDVw
         fZp5N+VxIaVJKXekr6bt4QXT9imoIn4FmiuZyiGW0IUVqCjL+0dZxd3AtOXVq2ntMb3d
         brWQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707448140; x=1708052940;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=2OcjvyFdFfN0YTA7X6657JNcd4izQVcCS3w/UHKd7y4=;
        b=LIklZ3EX1M1kZZ2FsJFDk9FpQdC9rRVIgJ1nPpWwGP2sIowJ2Q7LV16cyDgkoV+d5k
         RX3E0dqg4NdpuAiMWD/akKmAAZd0bzRrkqA4REw9On1u9B+CU/lP9B/jHJPh4bFGme3z
         15CHPPx0psIXnTW49Dv14P00ER+Cd4nqWARhalCz7jOjlO6LyKfY3s0XOyzx93FRq0g/
         fMwcpkm7CNcw88sdm3LzntV7b0wt73dSKX1C2QNy1lxIIc43e7H96jqUiQiPcOwpmqSa
         x6MkTu8C/5McXTN4EvGbqLEqxdbkbQHzigITsR4QDHKOS2iguz3y5NXzWwNIIWB78e5y
         oM3w==
X-Gm-Message-State: AOJu0YyeuPKzo8D0QOth6I1OZQarRUg968Fk1jPGwCOgjQtJ4oqhXDfL
	WB/GUCEa7NZITGwaudiyv7D4PBbYJWN7emrFFVrHFMGts1mJYywz+EqNOfnrVEfXk8eNMAOF99k
	VVpehbSOHpkJ+VnU6kPeLk9iO0Fg=
X-Google-Smtp-Source: AGHT+IFe0FeP/dxtCtFwA+vo7bxsfEYQJghTdKjDR9nRJtyHL9gqBtUqdRPjRy+vC1C7XXhQU9g87Xx8iCskyGcGKO0=
X-Received: by 2002:aa7:d692:0:b0:55f:c5b7:5855 with SMTP id
 d18-20020aa7d692000000b0055fc5b75855mr285211edr.6.1707448140092; Thu, 08 Feb
 2024 19:09:00 -0800 (PST)
MIME-Version: 1.0
References: <20240208165546.5715-1-jgross@suse.com> <20240208165546.5715-19-jgross@suse.com>
In-Reply-To: <20240208165546.5715-19-jgross@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Thu, 8 Feb 2024 22:08:44 -0500
Message-ID: <CAKf6xpv-77WjgXFcJz_WgwRxetO3mACAM+uagb6V1OgWKd__-Q@mail.gmail.com>
Subject: Re: [PATCH v5 18/22] tools: add 9pfs device to xenstore-stubdom
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>, 
	Julien Grall <julien@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 8, 2024 at 12:10=E2=80=AFPM Juergen Gross <jgross@suse.com> wro=
te:
>
> Add a 9pfs device to Xenstore stubdom in order to allow it to do e.g.
> logging into a dom0 file.
>
> Use the following parameters for the new device:
>
> - tag =3D "Xen"
> - type =3D "xen_9pfsd"
> - path =3D "/var/lib/xen/xenstore"
> - security-model =3D "none"
>
> For now don't limit allowed file space or number of files.
>
> Add a new libxl function for adding it similar to the function for
> adding the console device.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 05:07:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 05:07:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678537.1055910 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYJ6E-0005cW-Vg; Fri, 09 Feb 2024 05:06:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678537.1055910; Fri, 09 Feb 2024 05:06:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYJ6E-0005cP-T0; Fri, 09 Feb 2024 05:06:54 +0000
Received: by outflank-mailman (input) for mailman id 678537;
 Fri, 09 Feb 2024 05:06:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYJ6D-0005cF-5e; Fri, 09 Feb 2024 05:06:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYJ6C-0002JJ-NP; Fri, 09 Feb 2024 05:06:52 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYJ6C-00031n-8T; Fri, 09 Feb 2024 05:06:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYJ6C-0000LY-5r; Fri, 09 Feb 2024 05:06:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=wD+4CqrSsiznS0m2ngbYSm6z+zhUfkJwm1BBpvgAq8M=; b=sZ74eRXI863PzJ9c+aEQR4qBiR
	eRcKUHyTIXYuVCcgjJqgrwuuiFvUwsNILWIz19SZcaDNUse3RGL51vvBkDHTBn9fX/o9oAcAT70mL
	21f2ydVaU4PiihGUxf0W53xv3rFj9g0Ny1wAHaPPmoprskTt2cKLQtizOu89IJDqal0o=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184628-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184628: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-start/freebsd.repeat:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=047371968ffc470769f541d6933e262dc7085456
X-Osstest-Versions-That:
    linux=6d280f4d760e3bcb4a8df302afebf085b65ec982
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 09 Feb 2024 05:06:52 +0000

flight 184628 linux-linus real [real]
flight 184631 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184628/
http://logs.test-lab.xenproject.org/osstest/logs/184631/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-freebsd12-amd64 21 guest-start/freebsd.repeat fail pass in 184631-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184614
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184614
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184614
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184614
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184614
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184614
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184614
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184614
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                047371968ffc470769f541d6933e262dc7085456
baseline version:
 linux                6d280f4d760e3bcb4a8df302afebf085b65ec982

Last test of basis   184614  2024-02-07 08:42:21 Z    1 days
Failing since        184620  2024-02-07 19:40:30 Z    1 days    3 attempts
Testing same since   184628  2024-02-08 18:42:01 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexandre Ghiti <alexghiti@rivosinc.com>
  Anup Patel <anup@brainfault.org>
  Anup Patel <apatel@ventanamicro.com>
  Bibo Mao <maobibo@loongson.cn>
  Borislav Petkov (AMD) <bp@alien8.de>
  Christian Borntraeger <borntraeger@linux.ibm.com>
  Chuck Lever <chuck.lever@oracle.com>
  David Rientjes <rientjes@google.com>
  Dennis Zhou <dennis@kernel.org>
  Eric Farman <farman@linux.ibm.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Huacai Chen <chenhuacai@loongson.cn>
  Janosch Frank <frankja@linux.ibm.com>
  John Allen <john.allen@amd.com>
  Kees Cook <keescook@chromium.org>
  Kim Phillips <kim.phillips@amd.com>
  Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Nathan Chancellor <nathan@kernel.org>
  NeilBrown <neilb@suse.de>
  Paolo Bonzini <pbonzini@redhat.com>
  Tengfei Yu <moehanabichan@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   6d280f4d760e..047371968ffc  047371968ffc470769f541d6933e262dc7085456 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 07:06:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 07:06:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678546.1055919 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYKxl-0003gT-6z; Fri, 09 Feb 2024 07:06:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678546.1055919; Fri, 09 Feb 2024 07:06:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYKxl-0003gM-4E; Fri, 09 Feb 2024 07:06:17 +0000
Received: by outflank-mailman (input) for mailman id 678546;
 Fri, 09 Feb 2024 07:06:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4TWb=JS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rYKxk-0003gG-FO
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 07:06:16 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b3672801-c719-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 08:06:09 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-3392b12dd21so306093f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 08 Feb 2024 23:06:09 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s12-20020a5d424c000000b0033b483d1abcsm1037602wrr.53.2024.02.08.23.06.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 08 Feb 2024 23:06:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3672801-c719-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707462369; x=1708067169; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rs0tHOlYbH5N6L2VIKAqxprdhJjVsv1pRMDgrzx+qgA=;
        b=fhFCmvTfnmxetMPOwC5VW6WTpy9h16yLSjNU2qFMueqxVamtkHIk620BnJlRdy2SO0
         Jqlr3bs0m54+XBAVfhJELwEeZETYZ8RxUkrSuH+Fg4cUoztUHETtzTuBzCjb+MCwVD1Y
         9YevwOZpRY7xoSIyBfdNUt2/JWv4BWT8gV/65Dithi+AQAxmJXCwUMGbYwwqZEzaxuHy
         r8pEfbcaSBr78cFNsZ7gDb8JeDdODfimpUCTvUkwRG9htAlqnF8bD7rV1GKDXFxQDBd1
         jT9aojzXaKfkRjKbth3XRWgRpkLsH0f7zjdHTDGP+MY5T8Xt9ZM1PKcQ3NURKJnBEPyV
         D2vg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707462369; x=1708067169;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rs0tHOlYbH5N6L2VIKAqxprdhJjVsv1pRMDgrzx+qgA=;
        b=ZUKEt9qsjDh3WBN6vDHf2qHPL1hr67QSzGJqnCepENK/YVoIC19xUR7Vr9mDr8OgFV
         JjtVrhtEFNg6ubzZhcdN9lFJYzx1SxZ5UryCQPlIKUgOD5IKWtqBlOl7i6zFZqG/34ny
         jkIZoAq0R2tBwQxBdG883B/o46tUy20o+SNnHCO3ZOuAsJUqiCPoONorJmp805ob19r2
         UBllLBj/OY9+8+oFoxq00GFRkH9DOWkDHIbJ4f/aujNkg8NWCYiJQe8RyjF/WrQ9iWDp
         bBBjyUl9zJeem2kxdRmQ2J4n07lPnkFpCfRYGoANIBPAIbu8vROBrvmLNGC8qn7A5FhI
         vcHQ==
X-Gm-Message-State: AOJu0YyCRJuj9w9byDjDemg7+CQCfHrZ6r2F8SOqRdo5j8yCaT/mjjv9
	Hh+/X5uN8++zwpsI5aAFBCXa9ccqetQkpxNj/q9STX1GuPjieLFhXR9Km1BGVw==
X-Google-Smtp-Source: AGHT+IH9eZqV3GlPXIDzSOMOOX+g4NdGp8sSIz7PGh8adEvRMOC2Qwzqy+k25JZCOGAzOYLc7ieG0Q==
X-Received: by 2002:adf:f5c4:0:b0:33a:e8be:51bd with SMTP id k4-20020adff5c4000000b0033ae8be51bdmr408892wrp.51.1707462368925;
        Thu, 08 Feb 2024 23:06:08 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCV659TepQ9KsiAIu5CG0PsuaG6t21m1PVIC8+VdKCg5eZkN5t74n6soJyDyPuawVH7eIflmafU2FkyRJdHqHPnSiFBtfUCg4DKy1nW2sxBzhiPD
Message-ID: <ee53bbdd-4d57-4470-9ff4-1ff6b9eec63c@suse.com>
Date: Fri, 9 Feb 2024 08:06:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/7] VT-d: replace find_ats_dev_drhd()
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <a75bef32-8bbb-4471-b26b-981b06173cd5@suse.com> <ZcUP-t5gFx2v31at@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcUP-t5gFx2v31at@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08.02.2024 18:31, Roger Pau Monné wrote:
> On Mon, Feb 05, 2024 at 02:56:36PM +0100, Jan Beulich wrote:
>> All callers only care about boolean outcome. For this there's no point
>> in allocating a duplicate of the respective DRHD structure; a simple
>> boolean suffices (which eventually may wantg to become a count, such
>                                          ^ want
>> that the "any ATS devices assigned state" can also clear again). With
>> that boolean, remove respective parameters from internal helper
>> functions right away, as those have access to the flag through another
>> parameter.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> AFAICT the intention is that this is a non-functional change?

No functional effect intended, yes. Added such a sentence.

>> --- a/xen/drivers/passthrough/vtd/extern.h
>> +++ b/xen/drivers/passthrough/vtd/extern.h
>> @@ -65,8 +65,6 @@ struct acpi_drhd_unit *ioapic_to_drhd(un
>>  struct acpi_drhd_unit *hpet_to_drhd(unsigned int hpet_id);
>>  struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd);
>>  
>> -struct acpi_drhd_unit *find_ats_dev_drhd(struct vtd_iommu *iommu);
>> -
>>  int ats_device(const struct pci_dev *, const struct acpi_drhd_unit *);
>>  
>>  int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
>> --- a/xen/drivers/passthrough/vtd/iommu.c
>> +++ b/xen/drivers/passthrough/vtd/iommu.c
>> @@ -624,8 +624,7 @@ int cf_check vtd_flush_iotlb_reg(
>>  }
>>  
>>  static int __must_check iommu_flush_iotlb_global(struct vtd_iommu *iommu,
>> -                                                 bool flush_non_present_entry,
>> -                                                 bool flush_dev_iotlb)
>> +                                                 bool flush_non_present_entry)
>>  {
>>      int status;
>>  
>> @@ -633,7 +632,7 @@ static int __must_check iommu_flush_iotl
>>      vtd_ops_preamble_quirk(iommu);
>>  
>>      status = iommu->flush.iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH,
>> -                                flush_non_present_entry, flush_dev_iotlb);
>> +                                flush_non_present_entry, iommu->flush_dev_iotlb);
> 
> Any reason to not also remove the parameter from here also?  As the handler
> gets iommu passed as the first parameter anyway.

Indeed, yet then the patch would have grown quite a bit. I think I
meant to have a respective post-commit-message remark, but then
forgot to actually put one there. Once (if) this change has gone in,
a follow-on patch could further tidy tings. (The "right away" in the
description was kind of meant to indicate that.)

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 07:52:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 07:52:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678552.1055929 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYLgj-0001oW-L7; Fri, 09 Feb 2024 07:52:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678552.1055929; Fri, 09 Feb 2024 07:52:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYLgj-0001oP-IJ; Fri, 09 Feb 2024 07:52:45 +0000
Received: by outflank-mailman (input) for mailman id 678552;
 Fri, 09 Feb 2024 07:52:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BFPS=JS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rYLgh-0001oI-Ue
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 07:52:43 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 33c85d06-c720-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 08:52:42 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 52D4F4EE0739;
 Fri,  9 Feb 2024 08:52:41 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 33c85d06-c720-11ee-8a4b-1f161083a0e0
MIME-Version: 1.0
Date: Fri, 09 Feb 2024 08:52:41 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Julien Grall <julien@xen.org>
Cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org,
 michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com,
 consulting@bugseng.com, jbeulich@suse.com, andrew.cooper3@citrix.com,
 roger.pau@citrix.com, bertrand.marquis@arm.com, George Dunlap
 <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH] docs/misra: add x86_64 and arm64 asm-offset.c to
 exclude-list
In-Reply-To: <ccc568a3-d81c-482c-8768-c5934e3943a8@xen.org>
References: <b0c855581eed247a32b745906f84d352bf812091.1707324479.git.nicola.vetrini@bugseng.com>
 <ccc568a3-d81c-482c-8768-c5934e3943a8@xen.org>
Message-ID: <a6ed69c037c23cf2b0854ef52d797921@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

Hi Julien,

On 2024-02-08 22:33, Julien Grall wrote:
> Hi Nicola,
> 
> On 08/02/2024 07:55, Nicola Vetrini wrote:
>> These two files contain several deliberate violations of MISRA C rules 
>> and
>> they are not linked in the final Xen binary, therefore they can be 
>> exempted
>> from MISRA compliance.
> 
> I am curious, what are the violations you are talking about?
> 
> Cheers,
> 

The one that prompted the exclusion is for R20.12 on arm for the macros 
DEFINE and OFFSET, where the second argument of OFFSET is a macro and is 
used as a normal parameter and a stringification operand.
However, there are other special cases (e.g., the file not being linked, 
which violates R2.1 and was already decided to deviate that aspect).

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 08:11:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 08:11:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678561.1055940 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYLyo-0005HN-I7; Fri, 09 Feb 2024 08:11:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678561.1055940; Fri, 09 Feb 2024 08:11:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYLyo-0005HG-Ee; Fri, 09 Feb 2024 08:11:26 +0000
Received: by outflank-mailman (input) for mailman id 678561;
 Fri, 09 Feb 2024 08:11:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aSO/=JS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rYLyn-0005HA-Ke
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 08:11:25 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d0dd46f9-c722-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 09:11:24 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-41047395814so5775345e9.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 00:11:24 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 g4-20020a5d46c4000000b0033b1f023af2sm1158153wrs.52.2024.02.09.00.11.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 00:11:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0dd46f9-c722-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707466284; x=1708071084; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=GVTHB6Z/WJ/dQCx8xwxfP0PgrYlWn1zft1GvNWCII9Y=;
        b=wBbHpNVKpv2oBQH68/1I0UErB83pvxFE16ifQg2BfmtwTPc4gImuAIgEFaQmfZM1eX
         jVpwK74DxO8BYIA3Ry3/YrhZjPljwwzB3gZdfSp+a8zjYffOLexHd9C7yaQh3KQx7Wrm
         hJvNse/WFWoHdDaNoqMPq3xmFMGa8AGejLUWo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707466284; x=1708071084;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GVTHB6Z/WJ/dQCx8xwxfP0PgrYlWn1zft1GvNWCII9Y=;
        b=mcskyw+VRFctiUzOYrVHbJ0MPWOv+wEXrRRknJxUbpBxlXeMlF80X4BVss/26JX3kK
         OigXnsFDtLu9jQcd55F08YDQIXesNBLFelPeM7U6ujZxDuhcEw9Qr7i1/vaFpOepzP7Q
         3oUzG9nEaMN6e1mYdOgn4GU9qF2NWKmiYdJ9l4qolY08ZMSDlZDFSuWwLp1O+aVDStrI
         p7cV4n4AXMbBfICepwzzVPDxCDCPzWluEYbHnVzTuoqofme1IBtN2vBUDIELk34BEits
         uPaI5GpaeISODbtLDxJJ2f/lzdznnC3mcxuBwmr3wFDsIibvGifZAoLkEHpdvi67FNHE
         Cjag==
X-Gm-Message-State: AOJu0YwMBzq0dNPvGRXNdidRPELDavfO05wK2uyIA0reufL9RjKpzVod
	1z5xmdz7nCFt80eqEt1UVR4ZFkGRGj8Uyag4am9o05HcWlmy2FQe4bsAYmO90cQ=
X-Google-Smtp-Source: AGHT+IGzU79p5Epp/sld2vo3kIRxLlIxaVQpLL66jc5+oNk88fQ8L4QjfR1GaqacXCjUobQXVE3ayw==
X-Received: by 2002:a05:600c:19d1:b0:40f:b691:d3c1 with SMTP id u17-20020a05600c19d100b0040fb691d3c1mr580645wmq.30.1707466283957;
        Fri, 09 Feb 2024 00:11:23 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUGNUXcW1v1eaGMXjXB8JscA07ihnYPOXlZpjTsPxKXMpCn0C5ZlLuKaoKRqrawjIHQ4q85QdDcNRUrvYqNgn3pejFFMPmAFpBVQAq1CPT0Lzze
Date: Fri, 9 Feb 2024 09:11:22 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 4/7] VT-d: replace find_ats_dev_drhd()
Message-ID: <ZcXeKvdyxwtECV4a@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <a75bef32-8bbb-4471-b26b-981b06173cd5@suse.com>
 <ZcUP-t5gFx2v31at@macbook>
 <ee53bbdd-4d57-4470-9ff4-1ff6b9eec63c@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ee53bbdd-4d57-4470-9ff4-1ff6b9eec63c@suse.com>

On Fri, Feb 09, 2024 at 08:06:07AM +0100, Jan Beulich wrote:
> On 08.02.2024 18:31, Roger Pau Monné wrote:
> > On Mon, Feb 05, 2024 at 02:56:36PM +0100, Jan Beulich wrote:
> >> All callers only care about boolean outcome. For this there's no point
> >> in allocating a duplicate of the respective DRHD structure; a simple
> >> boolean suffices (which eventually may wantg to become a count, such
> >                                          ^ want
> >> that the "any ATS devices assigned state" can also clear again). With
> >> that boolean, remove respective parameters from internal helper
> >> functions right away, as those have access to the flag through another
> >> parameter.
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> > 
> > AFAICT the intention is that this is a non-functional change?
> 
> No functional effect intended, yes. Added such a sentence.

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

> 
> >> --- a/xen/drivers/passthrough/vtd/extern.h
> >> +++ b/xen/drivers/passthrough/vtd/extern.h
> >> @@ -65,8 +65,6 @@ struct acpi_drhd_unit *ioapic_to_drhd(un
> >>  struct acpi_drhd_unit *hpet_to_drhd(unsigned int hpet_id);
> >>  struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd);
> >>  
> >> -struct acpi_drhd_unit *find_ats_dev_drhd(struct vtd_iommu *iommu);
> >> -
> >>  int ats_device(const struct pci_dev *, const struct acpi_drhd_unit *);
> >>  
> >>  int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
> >> --- a/xen/drivers/passthrough/vtd/iommu.c
> >> +++ b/xen/drivers/passthrough/vtd/iommu.c
> >> @@ -624,8 +624,7 @@ int cf_check vtd_flush_iotlb_reg(
> >>  }
> >>  
> >>  static int __must_check iommu_flush_iotlb_global(struct vtd_iommu *iommu,
> >> -                                                 bool flush_non_present_entry,
> >> -                                                 bool flush_dev_iotlb)
> >> +                                                 bool flush_non_present_entry)
> >>  {
> >>      int status;
> >>  
> >> @@ -633,7 +632,7 @@ static int __must_check iommu_flush_iotl
> >>      vtd_ops_preamble_quirk(iommu);
> >>  
> >>      status = iommu->flush.iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH,
> >> -                                flush_non_present_entry, flush_dev_iotlb);
> >> +                                flush_non_present_entry, iommu->flush_dev_iotlb);
> > 
> > Any reason to not also remove the parameter from here also?  As the handler
> > gets iommu passed as the first parameter anyway.
> 
> Indeed, yet then the patch would have grown quite a bit. I think I
> meant to have a respective post-commit-message remark, but then
> forgot to actually put one there. Once (if) this change has gone in,
> a follow-on patch could further tidy tings. (The "right away" in the
> description was kind of meant to indicate that.)

Would you mind adding a sentence to the commit message that the
vtd_iommu hooks are not modified in order to avoid the patch growing
too much?  Otherwise it it's not clear why those are not also
converted.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 08:15:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 08:15:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678565.1055950 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYM2q-0005ui-5G; Fri, 09 Feb 2024 08:15:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678565.1055950; Fri, 09 Feb 2024 08:15:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYM2q-0005ub-29; Fri, 09 Feb 2024 08:15:36 +0000
Received: by outflank-mailman (input) for mailman id 678565;
 Fri, 09 Feb 2024 08:15:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aSO/=JS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rYM2o-0005uV-H0
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 08:15:34 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 656a945b-c723-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 09:15:33 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-5116063585aso819949e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 00:15:33 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 o9-20020a05600c4fc900b004101f27737asm1797374wmq.29.2024.02.09.00.15.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 00:15:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 656a945b-c723-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707466533; x=1708071333; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=+wgU5GZe1ofFBOofLysTtEQRnNdEUJgHJSdhljtKCv8=;
        b=YSQlTbxNp3sektOKgkungTV0L8gxjeqFNO7i4WgKHNHKOlFJT4q26WAKm//Xqc4lAs
         V9rY+PrRS5AFtcaRKfnMP0CKfa+q+LBckEGPZYqL8pPGvrgFRpYlQL4PHnz/0ES+HwX6
         4yvxSPURqCzC2tYlzPkLhKB2HlmEJ2e+80I14=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707466533; x=1708071333;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+wgU5GZe1ofFBOofLysTtEQRnNdEUJgHJSdhljtKCv8=;
        b=olAyB6pG7GGBBlq5d7yC09n2LbbuRNZfSeXhrGlAlSc/FD1etwEAh1zaX4/w0e/3cd
         G+hnPEzFndTlufhpQm4WLZHIfu5K4U1y3EJBrA/GU6Thv/gSn7bC1vPZ4z75bDKIQLQs
         xKU9X9/y7vUx1AZXQSRMz73xJHWgqxmKayvOSe3QUKQeIihyp7MCQeczF8ahNi92+MqI
         D+Crs1v9HuvjAvxvYvi0tJ4ilYqJGwkhqqdhABKH2QvNnlJgt257ih547nMiFHmqAxq1
         MX9uh+UJIHYEfE+huk00/424KJKDtdT6R4lIRoB2ASlpmATB5cMCnh5g8lzJpgBGbVgK
         E29g==
X-Gm-Message-State: AOJu0Yz0fwB4mH0uaY6Dnxmbs5QNlnhaqTxrs1lcO/H8v7eNjz5fHz/b
	jGDh2yw0dZVisPI5MNxxqfvs3jg+960HUBrgCc+4s/bCVOtYKRZCkZBvIKzyKZRjK0hbwOGSBt+
	D
X-Google-Smtp-Source: AGHT+IGguNE3q5/j1aeFVp7fybAnbMOG4N2WKOXVnyeqSOBLe/5aSy8kObXgVGhY/ZJOuz6rDP4o3A==
X-Received: by 2002:ac2:4d06:0:b0:511:61ab:406e with SMTP id r6-20020ac24d06000000b0051161ab406emr450332lfi.28.1707466533278;
        Fri, 09 Feb 2024 00:15:33 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWdr+/J4ubAHMcX3+oWbVVNYVJceH08wfyzfiBQXFCa7X7Uz030Lor/bIymGJbEkTY+w80Wq2LJ+JUrrzv/PYvstaHymnZEudNR7WZs+FBBjFse
Date: Fri, 9 Feb 2024 09:15:32 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 5/7] VT-d: move ats_device() to the sole file it's used
 from
Message-ID: <ZcXfJHs6DWNvvTYw@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <c6d803ba-9478-4ab0-9ab6-a9af94808682@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <c6d803ba-9478-4ab0-9ab6-a9af94808682@suse.com>

On Mon, Feb 05, 2024 at 02:56:55PM +0100, Jan Beulich wrote:
> ..., thus allowing it to become static, and thus reducing scope overlap
> between it and pci_ats_device(). There's nothing x86-specific about this
> function anyway.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné zroger.pau@citrix.com>

FWIW, the function would better return a boolean, as none of the
callers care about the position of the ATS capability, and use it as
an indication that ATS is supported.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 08:25:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 08:25:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678570.1055960 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYMCD-0007mk-W3; Fri, 09 Feb 2024 08:25:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678570.1055960; Fri, 09 Feb 2024 08:25:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYMCD-0007md-T8; Fri, 09 Feb 2024 08:25:17 +0000
Received: by outflank-mailman (input) for mailman id 678570;
 Fri, 09 Feb 2024 08:25:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aSO/=JS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rYMCD-0007mV-18
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 08:25:17 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c062e162-c724-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 09:25:15 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a3c00c98d32so7445866b.3
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 00:25:15 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 f14-20020a170906494e00b00a3bbf725080sm520411ejt.55.2024.02.09.00.25.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 00:25:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c062e162-c724-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707467115; x=1708071915; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=AVOD5/afe6IIiSKaIDkkhptsRiqDV1ABiXu/6o56ZrU=;
        b=ruFuTpiv6Ro4SO1UL1PG5C8aTxDuCzsF5ZVGnn6SgaIiVYLLzTmshwYpJyhN0Y/YcL
         oA7VEBj/2twZ1x5nbQt6ANg/UQH84e4yfPliG9zUwoOU6+KDLeUP2aWx88ckR38aHjRk
         XNIlGSxWJl3J02oyUmDbJVokGGOoMA22tGWxs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707467115; x=1708071915;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=AVOD5/afe6IIiSKaIDkkhptsRiqDV1ABiXu/6o56ZrU=;
        b=hr6t8TVOeqeQkjOWtA9OqN6GNGZpTtXft581e2jBGveN7DpexabTUHv8RdD3XOaea2
         Wr7nMhfU1XTWrtgrH2l1unfwU85Exdc2/YGxGK3NxS9l8qiYBuZHOTgFFFENBcc1BICU
         xf7wecgaKygO6eC7jwFkxOzmuDGgZkfYRM5HEqv4kn2WeVLAF9RFEkPEOvKChFuR8xsr
         +uIbs1/aiIQD37Az4LvVdvl4uF8RDhryShDL8aDH4rTt3dI4x9JrsVEXK2xUBvGaQe9A
         9itRBqADPnoY28VxTA7KhOhT9eyTg2Bb7Hyqqp4QzRZWdL92beM1Qbq/XbmqP1IIs3BY
         gWJQ==
X-Gm-Message-State: AOJu0Yz92LqX/cirYTxEHKuffhYSIosulq2cVnkZJgIa01ZI48fUBWX5
	Z2xDKB+will+jVO/DD+8te2rpYBhap3wdmzFUCP/pt2QRwU5hI72SOSSH2GtfLE=
X-Google-Smtp-Source: AGHT+IFhxb1yfazcGUJaoCUDfb8LwXieEVumGRM4fbO7A61brk9PWm6yFfOvfQqNHm8M1xoLXixWFw==
X-Received: by 2002:a17:906:f215:b0:a38:200d:f96 with SMTP id gt21-20020a170906f21500b00a38200d0f96mr599380ejb.59.1707467115418;
        Fri, 09 Feb 2024 00:25:15 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWGVop+SmJ4DPKBtTPViQNfaWL2EolZK/02P4KZRPoExziGhRRTLEfJ8yxYc29ZcbDQgnzGfVCJT3fB62zj+QliFU/ZSNu8DAt5lusSDfwXazCs
Date: Fri, 9 Feb 2024 09:25:14 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 6/7] VT-d: move dev_invalidate_iotlb() to the sole file
 it's used from
Message-ID: <ZcXhauUax5WHD93J@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <0d14f148-32e0-4dac-b5e8-3230adfc5f9d@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <0d14f148-32e0-4dac-b5e8-3230adfc5f9d@suse.com>

On Mon, Feb 05, 2024 at 02:57:12PM +0100, Jan Beulich wrote:
> ..., thus allowing it and qinval_device_iotlb_sync() to become static.
> There's nothing x86-specific about the function anyway. While moving,
> adjust types to better match ./CODING_STYLE (albeit use of fixed-width
> types for parameters is retained to limit the effective change).
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

> 
> --- a/xen/drivers/passthrough/vtd/extern.h
> +++ b/xen/drivers/passthrough/vtd/extern.h
> @@ -65,12 +65,6 @@ struct acpi_drhd_unit *ioapic_to_drhd(un
>  struct acpi_drhd_unit *hpet_to_drhd(unsigned int hpet_id);
>  struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd);
>  
> -int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
> -                         u64 addr, unsigned int size_order, u64 type);
> -
> -int __must_check qinval_device_iotlb_sync(struct vtd_iommu *iommu,
> -                                          struct pci_dev *pdev,
> -                                          u16 did, u16 size, u64 addr);
>  
>  uint64_t alloc_pgtable_maddr(unsigned long npages, nodeid_t node);
>  void free_pgtable_maddr(u64 maddr);
> --- a/xen/drivers/passthrough/vtd/qinval.c
> +++ b/xen/drivers/passthrough/vtd/qinval.c
> @@ -251,8 +251,9 @@ static int __must_check dev_invalidate_s
>      return rc;
>  }
>  
> -int qinval_device_iotlb_sync(struct vtd_iommu *iommu, struct pci_dev *pdev,
> -                             u16 did, u16 size, u64 addr)
> +static int qinval_device_iotlb_sync(struct vtd_iommu *iommu,
> +                                    struct pci_dev *pdev, uint16_t did,
> +                                    uint16_t size, paddr_t addr)
>  {
>      unsigned long flags;
>      unsigned int index;
> @@ -282,6 +283,101 @@ int qinval_device_iotlb_sync(struct vtd_
>      return dev_invalidate_sync(iommu, pdev, did);
>  }
>  
> +static bool device_in_domain(const struct vtd_iommu *iommu,
> +                             const struct pci_dev *pdev, uint16_t did)
> +{
> +    struct root_entry *root_entry;
> +    struct context_entry *ctxt_entry = NULL;
> +    unsigned int tt;
> +    bool found = false;
> +
> +    if ( unlikely(!iommu->root_maddr) )
> +    {
> +        ASSERT_UNREACHABLE();
> +        return false;
> +    }
> +
> +    root_entry = map_vtd_domain_page(iommu->root_maddr);
> +    if ( !root_present(root_entry[pdev->bus]) )
> +        goto out;
> +
> +    ctxt_entry = map_vtd_domain_page(root_entry[pdev->bus].val);
> +    if ( context_domain_id(ctxt_entry[pdev->devfn]) != did )
> +        goto out;
> +
> +    tt = context_translation_type(ctxt_entry[pdev->devfn]);
> +    if ( tt != CONTEXT_TT_DEV_IOTLB )
> +        goto out;
> +
> +    found = true;
> + out:
> +    if ( root_entry )
> +        unmap_vtd_domain_page(root_entry);
> +
> +    if ( ctxt_entry )
> +        unmap_vtd_domain_page(ctxt_entry);
> +
> +    return found;
> +}
> +
> +static int dev_invalidate_iotlb(struct vtd_iommu *iommu, uint16_t did,
> +                                paddr_t addr, unsigned int size_order,
> +                                uint64_t type)
> +{
> +    struct pci_dev *pdev, *temp;
> +    int ret = 0;
> +
> +    if ( !ecap_dev_iotlb(iommu->ecap) )
> +        return ret;
> +
> +    list_for_each_entry_safe( pdev, temp, &iommu->ats_devices, ats.list )
> +    {
> +        bool sbit;
> +        int rc = 0;
> +
> +        switch ( type )
> +        {
> +        case DMA_TLB_DSI_FLUSH:
> +            if ( !device_in_domain(iommu, pdev, did) )
> +                break;
> +            /* fall through if DSI condition met */
> +        case DMA_TLB_GLOBAL_FLUSH:
> +            /* invalidate all translations: sbit=1,bit_63=0,bit[62:12]=1 */
> +            sbit = 1;
> +            addr = (~0UL << PAGE_SHIFT_4K) & 0x7FFFFFFFFFFFFFFF;

Given the MISRA stuff, won't it be better to append 'UL' here while
moving?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 08:39:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 08:39:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678574.1055970 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYMPb-0001m6-3d; Fri, 09 Feb 2024 08:39:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678574.1055970; Fri, 09 Feb 2024 08:39:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYMPb-0001lz-0r; Fri, 09 Feb 2024 08:39:07 +0000
Received: by outflank-mailman (input) for mailman id 678574;
 Fri, 09 Feb 2024 08:39:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aSO/=JS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rYMPZ-0001lt-PH
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 08:39:05 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id adf4c6ff-c726-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 09:39:03 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a383016f428so70404866b.2
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 00:39:03 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 vu6-20020a170907a64600b00a3be92b07d8sm382430ejc.24.2024.02.09.00.39.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 00:39:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: adf4c6ff-c726-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707467943; x=1708072743; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=whpypgL+BiX1yO/akrzi74w+kAMRpN7Y3hHHdPuVD90=;
        b=KPxXDwsIJsPmyr7PLSXOFI0sTtsJgwbjYFYjlBH0H9bcPSZL1nubW0jb0OMV6LY9E4
         QSgGkX1uTyx4XcrTAVuempd63O+HBAY204znIqhOFxxlu8m2WWlY+3jKhzWKu4TNoUC8
         FFb76u92LrUJOCBW5dmunewq8q58ptwTupKrU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707467943; x=1708072743;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=whpypgL+BiX1yO/akrzi74w+kAMRpN7Y3hHHdPuVD90=;
        b=xG/dMyzKWRfJmsZCRdRdvZ/QYqb45K1Rx2wnbs2lTxdU7PythbGpzJJkt1yITGc1sN
         eHBboVv6NKjm3JJS4pVWukEqN+Da7uGQ/SS2N1q6YT+2GX64acbi1nhMcZ91B4nzmh79
         CI4+M2j3p21U32wokBV5eMSAAkYCvuOiENbjQRn5d9Ro0NkzktsCnXdhlvgtdG8my5Wi
         mr9bjPXT/C4vd9E4AOcpuhZFaEiTs3BaIRfxaFAqKS1M6Sbf/jiDYCtWwka121xHuaFz
         wSN3BJNhY08/WgudQe9NT/B/ZBK/HJcrlp3tkCcMlTFMy5K7bXj1LPwjmSJ5KgvMrBFg
         UF8w==
X-Gm-Message-State: AOJu0Yy6KlTgYUf9TRAV3AYWBykvtQsfKmsZdoDpOGdwshNczvrg+r3i
	bK30y4MDcavu38OltsARq3Pb2dBfXWwrTwlWEMnaJFnhvNvTWiU+weKeIDwqV2U=
X-Google-Smtp-Source: AGHT+IH36LU0raEjis0t2n221WWmXo0G3jf7Y934NTrgX4sLL3nvvjhqBZlatxXXplYJp0E9+fc2iQ==
X-Received: by 2002:a17:906:4717:b0:a37:b91c:8a4e with SMTP id y23-20020a170906471700b00a37b91c8a4emr635326ejq.60.1707467943110;
        Fri, 09 Feb 2024 00:39:03 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXuD3kQ57Tz3pd5HwQpw5j+KwUJcQe7ZmvcgX1needhogh/3CGR0DbKzHpLH1cq9LsOvTZNLK0KP0YA9Da+ppS9DQIfMC3z5cOSy79EJlSXnZQm
Date: Fri, 9 Feb 2024 09:39:02 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 7/7] VT-d: move {,un}map_vtd_domain_page()
Message-ID: <ZcXkpk8XxJmGO1EM@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <22e7036c-cf49-4160-bd26-fbba6b67ff5d@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <22e7036c-cf49-4160-bd26-fbba6b67ff5d@suse.com>

On Mon, Feb 05, 2024 at 02:57:30PM +0100, Jan Beulich wrote:
> ..., thus allowing them to become static. There's nothing x86-specific
> about these functions anyway.
> 
> Since only the "iommu_inclusive_mapping" parameter declaration would be
> left in the file, move that as well. There's nothing VT-d specific about
> it (anymore?): "dom0-iommu=map-inclusive" is similarly generic, and
> documentation also doesn't say anything.

Hm, I guess documentation should at least say that
iommu_inclusive_mapping is x86 specific, because it's not parsed on
Arm and hence might give the wrong impression that it's actually
acknowledged there.

> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Albeit I think it would be better to put the parsing in generic
iommu.c, so that the option gets parsed on Arm and
arch_iommu_hwdom_init() can print a warning message about it not
supported on Arm.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 09:01:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 09:01:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678579.1055979 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYMki-0005ww-Ih; Fri, 09 Feb 2024 09:00:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678579.1055979; Fri, 09 Feb 2024 09:00:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYMki-0005wp-Ft; Fri, 09 Feb 2024 09:00:56 +0000
Received: by outflank-mailman (input) for mailman id 678579;
 Fri, 09 Feb 2024 09:00:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aSO/=JS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rYMkg-0005wi-Tr
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 09:00:54 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ba2c5091-c729-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 10:00:53 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a38392b9917so89384166b.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 01:00:52 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 m9-20020a1709061ec900b00a3bb6de59b0sm552094ejj.95.2024.02.09.01.00.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 01:00:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ba2c5091-c729-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707469252; x=1708074052; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=qj6FdNIXA1BjepOCfn21bqPPzP2VkvH25TTBSp2baOo=;
        b=rFE4S9esGUwSS9t3d8I6fnethwPV+PPELosjuYUxqsKsB4oQ5xrTR7eGyLmD7DGEU8
         eN+IYlaF1J0Xyx1rffMR+A+r2IZwM7vbmuxh+/Y48RPD1MU2UuwgVWb4uZ/tW8/gQPeT
         XbCBR4EumfTS13uXwncDrF4zTsfyu6fVBhl0A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707469252; x=1708074052;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qj6FdNIXA1BjepOCfn21bqPPzP2VkvH25TTBSp2baOo=;
        b=QVb61kUhmGNBwS34U8DU9qsnOt9CWlV3c7XewT+bqT+h7dJcW/B+kDV1bSm+oa+QRX
         oV+OAklPhT/jlflZYIlaS6m+3ZBkC3UNQhvclm6sD0aGPz3PNgbAyaqIRBgYpYpkKEaa
         GJIZyeRPpF8Gu8M3jBetNXxmsebfxdXppV+yPOsBYCTMl4sVhnm362Q3qXdSJ4mpigaf
         xBYY36VI3k2rvIU6SguJjT65hcvyLAuzLHROamuCNbipjglMlDXBNZPKRX6XHtXc6oKr
         OZLEfp6N7HbSmc6Gj9jc5wi/hrbdNTYhXeT4wwXwlc8rplw3fgFo9rCtH03x8Y55SC77
         NhlA==
X-Gm-Message-State: AOJu0YyCp+SYfx/y4UpUdy/4r11ro4kuSqMEQX455LhJk1ZmCG8HI3rL
	HSN+cQJgUfqlh34DvxGLCI3SmVz6MMMhdz1RfkyI2OubdeXR1ujvPqQtY0JEFQJ/5B3ehiC0QNm
	C
X-Google-Smtp-Source: AGHT+IGg8m7xhtES74BzUhWIq2bHc3n3L+eV/1P9cIxLX49F2jgQzblJ/3kqh+ei1VcL2me3hktlfg==
X-Received: by 2002:a17:906:2657:b0:a38:3282:baf0 with SMTP id i23-20020a170906265700b00a383282baf0mr669401ejc.75.1707469252095;
        Fri, 09 Feb 2024 01:00:52 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVZo9ujKjFzdHM3zOvmA2BwAfIP8wwKBYYIbcl0Jpkr18V4kN3jXvjco1b/zw7VUmg2a0AkK3XtK8j0PG0CEAOA/U5EAiFyQK78bzf73gxrkTup
Date: Fri, 9 Feb 2024 10:00:50 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 1/7] VT-d: parse ACPI "SoC Integrated Address Translation
 Cache Reporting Structure"s
Message-ID: <ZcXpwtIDZ5RobRtw@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <b5a58dee-9a4c-4833-be59-b52c62f7137d@suse.com>
 <ZcScJvAhI7CRJhAZ@macbook>
 <a96fe270-15f1-492e-92d6-2a7ceda11dec@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <a96fe270-15f1-492e-92d6-2a7ceda11dec@suse.com>

On Thu, Feb 08, 2024 at 04:29:34PM +0100, Jan Beulich wrote:
> On 08.02.2024 10:17, Roger Pau Monné wrote:
> > On Mon, Feb 05, 2024 at 02:55:17PM +0100, Jan Beulich wrote:
> >> This is a prereq to us, in particular, respecting the "ATC required"
> >> flag.
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >> ---
> >> Should we check scope entries for appropriate types? (If so, then also
> >> for e.g. ATSR.)
> > 
> > Hm, I guess we could do so in acpi_parse_dev_scope() since that
> > function already gets passed a 'type' argument.
> 
> Right, I transiently had it there, but then dropped it again for being
> inconsistent with what we have right now. I'll try to remember to add
> another patch.

No strict requirement - but since it's on the spec we might as well
try to honor it.

> >> +        {
> >> +            dprintk(XENLOG_WARNING VTDPREFIX,
> >> +                    " Non-existent device (%pp) is reported in SATC scope!\n",
> >> +                    &PCI_SBDF(satcu->segment, b, d, f));
> >> +            ignore = true;
> > 
> > This is kind of reporting is incomplete: as soon as one device is
> > found the loop is exited and no further detection happens.  If we want
> > to print such information, we should do the full scan and avoid
> > exiting early when a populated device is detected.
> 
> Not sure I follow, but first of all - these are dprintk()s only, so
> meant to only help in dev environments. Specifically ...
> 
> >> +        }
> >> +        else
> >> +        {
> >> +            ignore = false;
> >> +            break;
> >> +        }
> >> +    }
> >> +
> >> +    if ( ignore )
> >> +    {
> >> +        dprintk(XENLOG_WARNING VTDPREFIX,
> >> +                " Ignore SATC for seg %04x as no device under its scope is PCI discoverable!\n",
> 
> ... this message is then issued only bogus entries were found. IOW
> when a real device was found, there's no real reason to report N
> other bogus ones, I think.

I guess it's a question of taste.  I do find it odd (asymmetric
maybe?) that we stop reporting non-existing devices once a valid
device is found.  Makes me wonder what's the point of reporting them
in the first place, if the list of non-existing devices is not
complete?

> Plus, whatever we change here ought to also / first change in
> register_one_rmrr().

I could live with those looking differently, or register_one_rmrr()
can be adjusted later.  Existing examples shouldn't be an argument to
not make new additions better.

But that's only if you agree the suggested changes make the code
better, if you prefer the current implementation then there's no point
in arguing whether we should keep register_one_rmrr() and the newly
introduce function similar enough.

> >> +    satcu = xzalloc(struct acpi_satc_unit);
> >> +    if ( !satcu )
> >> +        return -ENOMEM;
> >> +
> >> +    satcu->segment = satc->segment;
> >> +    satcu->atc_required = satc->flags & 1;
> > 
> > I would add this as a define in actbl2.h:
> > 
> > #define ACPI_DMAR_ATC_REQUIRED (1U << 0)
> > 
> > Or some such (maybe just using plain 1 is also fine).
> 
> I intended to do so, but strictly staying in line with what Linux has.
> To my surprise they use a literal number and have no #define. Hence I
> didn't add any either.

I would prefer the define unless you have strong objections, even if
that means diverging from Linux.

> >> +
> >> +    dev_scope_start = (const void *)(satc + 1);
> >> +    dev_scope_end   = (const void *)satc + header->length;
> >> +    ret = acpi_parse_dev_scope(dev_scope_start, dev_scope_end,
> >> +                               &satcu->scope, SATC_TYPE, satc->segment);
> >> +
> >> +    if ( !ret && satcu->scope.devices_cnt )
> >> +    {
> >> +        ret = register_one_satc(satcu);
> >> +        /*
> >> +         * register_one_satc() returns greater than 0 when a specified
> >> +         * PCIe device cannot be detected. To prevent VT-d from being
> >> +         * disabled in such cases, reset the return value to 0 here.
> >> +         */
> >> +        if ( ret > 0 )
> >> +            ret = 0;
> >> +    }
> >> +    else
> >> +        xfree(satcu);
> > 
> > You can safely use scope_devices_free() even if acpi_parse_dev_scope()
> > failed, so you could unify the freeing here, instead of doing it in
> > register_one_satc() also.
> 
> Moving that out of acpi_parse_dev_scope() would feel wrong - if a
> function fails, it would better not leave cleanup to its caller(s).

Given that the caller here is the one that did the allocation my
preference would be to also do the cleanup there - register_one_satc()
has no need to know what needs freeing, and allows unifying the
cleanup in a single place.

> > Also why not make register_one_satc() return a boolean instead of 0/1?
> 
> To leave room to also return errors, like register_one_rmrr() does.
> 
> For both of these aspects you raise: I'd really like to avoid these
> sibling functions going too much out of sync.

I could live with those going out-of-sync (or adjusted later), but only
if you think the suggestions are an improvement.

> >> @@ -817,6 +907,11 @@ static int __init cf_check acpi_parse_dm
> >>                  printk(VTDPREFIX "found ACPI_DMAR_RHSA:\n");
> >>              ret = acpi_parse_one_rhsa(entry_header);
> >>              break;
> >> +        case ACPI_DMAR_TYPE_SATC:
> >> +            if ( iommu_verbose )
> >> +                printk(VTDPREFIX "found ACPI_DMAR_SATC:\n");
> >> +            ret = acpi_parse_one_satc(entry_header);
> >> +            break;
> > 
> > I know the surrounding code doesn't use it, but readability would
> > benefit from adding a blank line after the break statement.
> 
> Well, yes, done so. I'm not generally in favor of introducing such
> inconsistencies, but it looks tolerable here. (In cases like this
> I do - and did here - consider this as an option, but in most cases
> I end up valuing uniform look slightly higher.)

 Yeah, overall I think it's an improvement if we go switching those as
 we modify the code for other reasons.

 Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 09:07:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 09:07:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678582.1055990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYMqc-0006X9-6U; Fri, 09 Feb 2024 09:07:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678582.1055990; Fri, 09 Feb 2024 09:07:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYMqc-0006X2-3b; Fri, 09 Feb 2024 09:07:02 +0000
Received: by outflank-mailman (input) for mailman id 678582;
 Fri, 09 Feb 2024 09:07:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aSO/=JS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rYMqa-0006Ww-VT
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 09:07:00 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9456ba79-c72a-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 10:06:58 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-55a5e7fa471so845770a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 01:06:58 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 i8-20020aa7dd08000000b0056010210438sm586704edv.32.2024.02.09.01.06.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 01:06:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9456ba79-c72a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707469618; x=1708074418; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=GXtB07cRzYTvSKLC1CPOntwIqxXCSCKKbfFCwUjQa5g=;
        b=gPpTmtWIr0/M5fonpP54ceICa1iwzXakF0i/gLC8Bio6t1ct7gwS+dAgZgCXhyP1RJ
         4REU4C3CBdjK4g0PZi8/YvAdQC06Q9ZSmeUGny3z7gJWLuAXUlL9OHeuOqe5i52RIr7C
         pemABQV7Qw9gIpyh1WYwd25Mcezz1omGBedb8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707469618; x=1708074418;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GXtB07cRzYTvSKLC1CPOntwIqxXCSCKKbfFCwUjQa5g=;
        b=pSUz8u7OEDMCCupdM4ZipoCAYb8lbI3CfAfMjtFVhRoijU1o1JRH3aPYmfaqDY8NTv
         RXXUVGOQ3nSeui406eYKSYOPgf7BiE1/nQsoOR2YbolZMzrwd9hTBVTovltP+EZ70l2k
         MiZAgGStCKPBzHgiaHheqF8P4e+5gKVh+hq+qtZm9Z0VuzMXFdJdoETSsTY2f6Mk+fC5
         y0XkKyAPA2ppoaZ5H4WjSigVw8jV0P4n0uQEaxv8WokmzEuYM8mv0COjXxiXD/7/Exsz
         3PklzTl+QvD0ODARzOE3xNKJaKXLlKUcn6aA2anAUqRWUc2a9/oqhWnVygOuk0K6bdXx
         RnPQ==
X-Gm-Message-State: AOJu0Yy2aoiLdn7i8CATErvBOkdg2m0Hr2quVvcF7BMvoUW+hxKeC/7M
	fvpqptwWq19cq6/Tl2gQbfOIjX8p87Eb3LtPI9WeteUGL5E/5mSm8BVNkG/bWiU=
X-Google-Smtp-Source: AGHT+IGLYrBI+9ciaUPYERkVVgzTyf7RtlbgoMB0O128Fb6VO9lX8DY3HMzj3u0hKYz2q/GDCjji/w==
X-Received: by 2002:aa7:ccd6:0:b0:561:13eb:4ade with SMTP id y22-20020aa7ccd6000000b0056113eb4ademr837825edt.27.1707469618331;
        Fri, 09 Feb 2024 01:06:58 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWuQYCpoKkS3NtRCWsfI54g3N/M5WIBgnJ5seTsD6lyniRylK89rgYXxH6skWd9s5C5zzxv0NtmWvmGjbFQXtqU3BWu3ZooTsLO8ae292boqdo78N/v
Date: Fri, 9 Feb 2024 10:06:57 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86/PV32: restore PAE-extended-CR3 logic
Message-ID: <ZcXrMZ2Nrsna2Kbe@macbook>
References: <47ab9000-68f6-8925-d814-a3a955b7f6cc@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <47ab9000-68f6-8925-d814-a3a955b7f6cc@suse.com>

On Wed, Feb 15, 2023 at 03:54:11PM +0100, Jan Beulich wrote:
> While the PAE-extended-CR3 VM assist is a 32-bit only concept, it still
> applies to guests also when run on a 64-bit hypervisor: The "extended
> CR3" format has to be used there as well, to fit the address in the only
> 32-bit wide register there. As a result it was a mistake that the check
> was never enabled for that case, and was then mistakenly deleted in the
> course of removal of 32-bit-Xen code (218adf199e68 ["x86: We can assume
> CONFIG_PAGING_LEVELS==4"]).
> 
> Similarly during Dom0 construction kernel awareness needs to be taken
> into account, and respective code was again mistakenly never enabled for
> 32-bit Dom0 when running on 64-bit Xen (and thus wrongly deleted by
> 5d1181a5ea5e ["xen: Remove x86_32 build target"]).
> 
> At the same time restrict enabling of the assist for Dom0 to just the
> 32-bit case. Furthermore there's no need for an atomic update there.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 09:25:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 09:25:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678590.1056000 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYN87-0001da-PB; Fri, 09 Feb 2024 09:25:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678590.1056000; Fri, 09 Feb 2024 09:25:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYN87-0001dT-MG; Fri, 09 Feb 2024 09:25:07 +0000
Received: by outflank-mailman (input) for mailman id 678590;
 Fri, 09 Feb 2024 09:25:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=z5rg=JS=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rYN86-0001dJ-Ml
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 09:25:06 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1ba4439d-c72d-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 10:25:04 +0100 (CET)
Received: from [192.168.1.9] (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id B06324EE0739;
 Fri,  9 Feb 2024 10:25:03 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ba4439d-c72d-11ee-8a4b-1f161083a0e0
Message-ID: <a3586ea2-5ad7-4c64-8e05-00ea0f83518f@bugseng.com>
Date: Fri, 9 Feb 2024 10:25:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and
 logging effects
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com>
 <09869431-ed00-4102-93fd-84c697a5c950@suse.com>
 <851eb6ec-558b-4a89-a31a-34046730bb55@bugseng.com>
 <41474fe7-c65f-4244-a455-0aaa0e4315e3@suse.com>
 <e48f4d49-274c-4bd6-8e3b-1ecaee6fa602@bugseng.com>
 <c6199a60-ecd2-4f8a-b742-199b90e62bac@suse.com>
From: Simone Ballarin <simone.ballarin@bugseng.com>
Organization: BUGSENG
In-Reply-To: <c6199a60-ecd2-4f8a-b742-199b90e62bac@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 07/02/24 13:40, Jan Beulich wrote:
> On 07.02.2024 13:21, Simone Ballarin wrote:
>> On 07/02/24 11:24, Jan Beulich wrote:
>>> On 07.02.2024 11:03, Simone Ballarin wrote:
>>>> On 06/02/24 13:04, Jan Beulich wrote:
>>>>> On 02.02.2024 16:16, Simone Ballarin wrote:
>>>>>> Rule 13.1: Initializer lists shall not contain persistent side effects
>>>>>>
>>>>>> Effects caused by debug/logging macros and functions (like ASSERT, __bad_atomic_size,
>>>>>> LOG, etc ...) that crash execution or produce logs are not dangerous in initializer
>>>>>> lists. The evaluation order in abnormal conditions is not relevant. Evaluation order
>>>>>> of logging effects is always safe.
>>>>>
>>>>> I thought I said so before: When talking of just logging, evaluation order
>>>>> may very well have a impact on correctness. Therefore we shouldn't mix
>>>>> debugging and logging.
>>>>
>>>> My general feeling was that changes like the following one are not supported by
>>>> the community:
>>>>
>>>> - x = { .field1 = function_with_logs_effects() /*other eventual code*/ };
>>>> + int field1 = function_with_logs_effects();
>>>> + x = { .field1 = field1 /*other eventual code*/};
>>>>
>>>> so I tried to deviate as much as possible.
>>>>
>>>> If having log effects is a good reason to do changes like the above, I can
>>>> propose a patch in that sense.
>>>
>>> Just to avoid misunderstandings: I'm not advocating for changes like the
>>> one you outline above. I simply consider the rule too strict: There's
>>> nothing at risk when there's just a single operation with side effects
>>> in an initializer.
>>
>> I agree for the safe cases such as single item list initializers (independently
>> by the number of effect contained in io_apic_read).
>> In fact, I was about to propose in another patch to deviate cases like:
>>
>> union IO_APIC_reg_01 reg_01 = { .raw = io_apic_read(idx, 1) };
>> union IO_APIC_reg_02 reg_02 = { .raw = io_apic_read(idx, 2) };
>>
>>> Even when there are multiple such operations, whether
>>> there's anything at risk depends on whether any of the side effects
>>> actually collide. In a number of cases the compiler would actually warn
>>> (and thus, due to -Werror, the build would fail).
>>>
>>
>> I don't completely agree on that, this requires an in-depth comprehension
>> of the code especially when complex call chains are involved. Moreover
>> these deviations need to be maintained when one of the function involved changes.
> 
> Right, and I didn't really mean multiple function calls here, but e.g.
> multiple ++ or --.
> 
>>>>>> --- a/xen/arch/arm/guestcopy.c
>>>>>> +++ b/xen/arch/arm/guestcopy.c
>>>>>> @@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf, uint64_t addr, unsigned int len,
>>>>>>     unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
>>>>>>     {
>>>>>>         return copy_guest((void *)from, (vaddr_t)to, len,
>>>>>> -                      GVA_INFO(current), COPY_to_guest | COPY_linear);
>>>>>> +                      /* SAF-4-safe No persistent side effects */
>>>>>> +                      GVA_INFO(current),
>>>>>
>>>>> I _still_ think this leaves ambiguity. The more that you need to look
>>>>> up GVA_INFO() to recognize what this is about.
>>>>
>>>>
>>>> Just to recap: here the point is that current reads a register with a volatile asm, so the
>>>> violation is in the expansion of GVA_INFO(current). Both GVA_INFO and current taken alone
>>>> are completely fine, so this is the only place where a SAF comment can be placed.
>>>>
>>>> The exapansion is:
>>>> ((copy_info_t) { .gva = { ((*({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"(&
>>>>      per_cpu__curr_vcpu)); (typeof(&per_cpu__curr_vcpu)) (__ptr + (({ uint64_t _r; asm volatile("mrs  %0, ""TPIDR_EL2" : "=r"
>>>>      (_r)); _r; }))); }))) } }), (1U << 1) | (1U << 2));
>>>>
>>>> My proposals are:
>>>> 1) address the violation moving the current expansion outside (extra variable);
>>>> 2) put a more detailed comment to avoid the ambiguity;
>>>> 3) use an ECL deviation for GVA_INFO(current).
>>>>
>>>> Do you have any preference or proposal?
>>>
>>> Imo 3 is not an option at all. Probably 1 wouldn't be too bad here, but
>>> I still wouldn't like it (as matching a general pattern I try to avoid:
>>> introducing local variables that are used just once and don't meaningfully
>>> improve e.g. readability). Therefore out of what you list, 2 would remain.
>>> But I'm not happy with a comment here either - as per above, there's
>>> nothing that can go wrong here as long as there's only a single construct
>>> with side effect(s).
>>>
>> So, would be changing the SAF in:
>> /* SAF-<new_id>-safe single item initializer */
>>
>> OK for you?
> 
> A comment, as said, is only the least bad of what you did enumerate. But
> for this code in particular I'm not a maintainer anyway, so it's not me
> you need to convince. I'm taking this only as an example for discussing
> underlying aspects.
> 
> Jan
> 

I was generally thinking about the comments of this series, and I've
just realised that many of them can be summarized by the following sentence:

"We do not want changes to address violations of R13.1 that are not also violations
of R13.2"

MC3R1.R13.2	rule	The value of an expression and its persistent side effects shall be the same under all permitted evaluation orders
MC3R1.R13.1	rule	Initializer lists shall not contain persistent side effects

At this point, my proposal is to remove R13.1 from the coding standard and add
R13.2 (eventually limiting its scope to initializer lists).
Maybe it is better to re-discuss the rule adoption during the next meeting?


-- 
Simone Ballarin, M.Sc.

Field Application Engineer, BUGSENG (https://bugseng.com)



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 09:28:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 09:28:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678593.1056010 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNBj-0002hC-7k; Fri, 09 Feb 2024 09:28:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678593.1056010; Fri, 09 Feb 2024 09:28:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNBj-0002h5-4x; Fri, 09 Feb 2024 09:28:51 +0000
Received: by outflank-mailman (input) for mailman id 678593;
 Fri, 09 Feb 2024 09:28:50 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rYNBi-0002gz-E4
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 09:28:50 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYNBh-0007f1-HN; Fri, 09 Feb 2024 09:28:49 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYNBh-0004IS-Bc; Fri, 09 Feb 2024 09:28:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=r3egIsQ76+Zz7dDfZhdkZzns4xBxiI+91N1PKJEsPe0=; b=YtZsNLVyeq9aqOAqRjzZQZZN+P
	sueT610C4CWM64u+Jc6I68xBaQHQ0KDEYq1n6ehg0fZZiWtAcDBGJJlICgD3+AuN5dg+iFe0CQkc3
	SR8M2KNmn9I7xp8Lxy9OfEQbycD7aKJBy3W7UqBxKSo9I8BM5xSDuLe/mE9wlbW5Kyr8=;
Message-ID: <2a335c77-bafb-4a42-a10b-f01c9985c5f8@xen.org>
Date: Fri, 9 Feb 2024 09:28:46 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] docs/misra: add x86_64 and arm64 asm-offset.c to
 exclude-list
Content-Language: en-GB
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org,
 michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com,
 consulting@bugseng.com, jbeulich@suse.com, andrew.cooper3@citrix.com,
 roger.pau@citrix.com, bertrand.marquis@arm.com,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
References: <b0c855581eed247a32b745906f84d352bf812091.1707324479.git.nicola.vetrini@bugseng.com>
 <ccc568a3-d81c-482c-8768-c5934e3943a8@xen.org>
 <a6ed69c037c23cf2b0854ef52d797921@bugseng.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <a6ed69c037c23cf2b0854ef52d797921@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 09/02/2024 07:52, Nicola Vetrini wrote:
> Hi Julien,

Hi Nicola,

> 
> On 2024-02-08 22:33, Julien Grall wrote:
>> Hi Nicola,
>>
>> On 08/02/2024 07:55, Nicola Vetrini wrote:
>>> These two files contain several deliberate violations of MISRA C 
>>> rules and
>>> they are not linked in the final Xen binary, therefore they can be 
>>> exempted
>>> from MISRA compliance.
>>
>> I am curious, what are the violations you are talking about?
>>
>> Cheers,
>>
> 
> The one that prompted the exclusion is for R20.12 on arm for the macros 
> DEFINE and OFFSET, where the second argument of OFFSET is a macro and is 
> used as a normal parameter and a stringification operand.
> However, there are other special cases (e.g., the file not being linked, 
> which violates R2.1 and was already decided to deviate that aspect).

Thanks for the clarification. I think it would be worth to add what you 
just wrote in the commit message. This give an idea to the reviewer why 
we excluded it.

With Jan's request addressed:

Acked-by: Julien Grall <jgrall@amazon.com>

> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 09:34:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 09:34:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678598.1056021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNGv-000488-Ry; Fri, 09 Feb 2024 09:34:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678598.1056021; Fri, 09 Feb 2024 09:34:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNGv-000481-NY; Fri, 09 Feb 2024 09:34:13 +0000
Received: by outflank-mailman (input) for mailman id 678598;
 Fri, 09 Feb 2024 09:34:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BFPS=JS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rYNGu-00047v-1n
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 09:34:12 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 604abc3a-c72e-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 10:34:09 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 37C844EE0739;
 Fri,  9 Feb 2024 10:34:09 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 604abc3a-c72e-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Fri, 09 Feb 2024 10:34:09 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Julien Grall <julien@xen.org>, Jbeulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org,
 michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com,
 consulting@bugseng.com, andrew.cooper3@citrix.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, George Dunlap <george.dunlap@citrix.com>, Wei Liu
 <wl@xen.org>
Subject: Re: [XEN PATCH] docs/misra: add x86_64 and arm64 asm-offset.c to
 exclude-list
In-Reply-To: <2a335c77-bafb-4a42-a10b-f01c9985c5f8@xen.org>
References: <b0c855581eed247a32b745906f84d352bf812091.1707324479.git.nicola.vetrini@bugseng.com>
 <ccc568a3-d81c-482c-8768-c5934e3943a8@xen.org>
 <a6ed69c037c23cf2b0854ef52d797921@bugseng.com>
 <2a335c77-bafb-4a42-a10b-f01c9985c5f8@xen.org>
Message-ID: <1f0b2c1621d76884ed36e3bceb8a1989@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-09 10:28, Julien Grall wrote:
> On 09/02/2024 07:52, Nicola Vetrini wrote:
>> Hi Julien,
> 
> Hi Nicola,
> 
>> 
>> On 2024-02-08 22:33, Julien Grall wrote:
>>> Hi Nicola,
>>> 
>>> On 08/02/2024 07:55, Nicola Vetrini wrote:
>>>> These two files contain several deliberate violations of MISRA C 
>>>> rules and
>>>> they are not linked in the final Xen binary, therefore they can be 
>>>> exempted
>>>> from MISRA compliance.
>>> 
>>> I am curious, what are the violations you are talking about?
>>> 
>>> Cheers,
>>> 
>> 
>> The one that prompted the exclusion is for R20.12 on arm for the 
>> macros DEFINE and OFFSET, where the second argument of OFFSET is a 
>> macro and is used as a normal parameter and a stringification operand.
>> However, there are other special cases (e.g., the file not being 
>> linked, which violates R2.1 and was already decided to deviate that 
>> aspect).
> 
> Thanks for the clarification. I think it would be worth to add what you 
> just wrote in the commit message. This give an idea to the reviewer why 
> we excluded it.
> 
> With Jan's request addressed:
> 
> Acked-by: Julien Grall <jgrall@amazon.com>
> 
>> 

Thanks. I did send a v2 of this patch addressing Jan's request. I'll let 
him decide whether he wants a more detailed commit message.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 09:36:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 09:36:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678601.1056030 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNIc-0004eF-4v; Fri, 09 Feb 2024 09:35:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678601.1056030; Fri, 09 Feb 2024 09:35:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNIc-0004e8-22; Fri, 09 Feb 2024 09:35:58 +0000
Received: by outflank-mailman (input) for mailman id 678601;
 Fri, 09 Feb 2024 09:35:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rYNIa-0004e2-E3
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 09:35:56 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYNIZ-0007mC-5j; Fri, 09 Feb 2024 09:35:55 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYNIY-0004fR-VS; Fri, 09 Feb 2024 09:35:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=zRIQ/5cSyWBz/1iz1i/1nX4Ea1J7jiFSGIQPRw3NG+U=; b=FaNJ6+77qzWTQ1E8UBnTnXiCMh
	KLqw5iWPFjhtAlOdotix6epX6CcN7KdgcTImOc1BdVCoxEzyYF/EzFetswL9+qRZm0dDzgB3NT5rL
	hlCrOQKhFW/bpTUt8HHjSUheN37gWkRZRLddf367zVc3KDfY1gZop4OfFY3M6Wy8RQVI=;
Message-ID: <f261ffa0-73af-4839-aeea-5e290e4f80b3@xen.org>
Date: Fri, 9 Feb 2024 09:35:52 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] docs/misra: add x86_64 and arm64 asm-offset.c to
 exclude-list
Content-Language: en-GB
To: Nicola Vetrini <nicola.vetrini@bugseng.com>, Jbeulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org,
 michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com,
 consulting@bugseng.com, andrew.cooper3@citrix.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>
References: <b0c855581eed247a32b745906f84d352bf812091.1707324479.git.nicola.vetrini@bugseng.com>
 <ccc568a3-d81c-482c-8768-c5934e3943a8@xen.org>
 <a6ed69c037c23cf2b0854ef52d797921@bugseng.com>
 <2a335c77-bafb-4a42-a10b-f01c9985c5f8@xen.org>
 <1f0b2c1621d76884ed36e3bceb8a1989@bugseng.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <1f0b2c1621d76884ed36e3bceb8a1989@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 09/02/2024 09:34, Nicola Vetrini wrote:
> On 2024-02-09 10:28, Julien Grall wrote:
>> On 09/02/2024 07:52, Nicola Vetrini wrote:
>>> Hi Julien,
>>
>> Hi Nicola,
>>
>>>
>>> On 2024-02-08 22:33, Julien Grall wrote:
>>>> Hi Nicola,
>>>>
>>>> On 08/02/2024 07:55, Nicola Vetrini wrote:
>>>>> These two files contain several deliberate violations of MISRA C 
>>>>> rules and
>>>>> they are not linked in the final Xen binary, therefore they can be 
>>>>> exempted
>>>>> from MISRA compliance.
>>>>
>>>> I am curious, what are the violations you are talking about?
>>>>
>>>> Cheers,
>>>>
>>>
>>> The one that prompted the exclusion is for R20.12 on arm for the 
>>> macros DEFINE and OFFSET, where the second argument of OFFSET is a 
>>> macro and is used as a normal parameter and a stringification operand.
>>> However, there are other special cases (e.g., the file not being 
>>> linked, which violates R2.1 and was already decided to deviate that 
>>> aspect).
>>
>> Thanks for the clarification. I think it would be worth to add what 
>> you just wrote in the commit message. This give an idea to the 
>> reviewer why we excluded it.
>>
>> With Jan's request addressed:
>>
>> Acked-by: Julien Grall <jgrall@amazon.com>
>>
>>>
> 
> Thanks. I did send a v2 of this patch addressing Jan's request. I'll let 
> him decide whether he wants a more detailed commit message.

Ah I didn't spot the v2. Jan, would you be happy if I commit it with the 
updated commit message?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 09:39:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 09:39:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678605.1056039 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNMF-0005pz-Iz; Fri, 09 Feb 2024 09:39:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678605.1056039; Fri, 09 Feb 2024 09:39:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNMF-0005ps-G1; Fri, 09 Feb 2024 09:39:43 +0000
Received: by outflank-mailman (input) for mailman id 678605;
 Fri, 09 Feb 2024 09:39:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYNMD-0005pm-DW
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 09:39:41 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2575982f-c72f-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 10:39:40 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-511616b73ddso1198497e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 01:39:40 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 8-20020ac24828000000b005117ac9cd1asm50230lft.88.2024.02.09.01.39.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 01:39:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2575982f-c72f-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707471580; x=1708076380; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=s/Dyv6JD++6g81K++/rhaIf7qr4pDURdgPVxoli3z+Q=;
        b=f+ZzhumdxUhCs7z8vuJcBa7ps6nOoppJmqLpBVwPuMRkPlIriC6MTXbBERIw67Kl+E
         q1bfpY7uxHPmtna/piH5Oaqg1VaVdlah7dqxBb0AmLa5eQq2q6Gx7hdPf4d4Oso2OUQP
         KSw6NSi2h4DLjA3q3lF4OLdCUIjQwykHqv7TW12tabJArkyOhO/QLpUv8ekTKRtTzzDZ
         O1pQ2DOoRWJWhR3RrathipzzqTzis3355OvLmr8ggk7uabjQN+O8ZRnY5M5AHjm7lx19
         TaOIzQfajs8M0bJNzj1ikYiuHXHyq2cV/ynZg/nU5PxdGUSxzxayh5tBySueubSPJCEz
         tVsw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707471580; x=1708076380;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=s/Dyv6JD++6g81K++/rhaIf7qr4pDURdgPVxoli3z+Q=;
        b=JbD3QlAA9ounIX7RgPXecGDuGMlV2acSi3uLYhh3/dY4EH9cZogawCBXu6B+F1jiry
         WOOHeMDFpZQ3V8gEPgFXCuREHLGn35hUfXMZWAj00kFyz+wdRKZLr0zz5pgJtV0VrmBv
         qsv83DROoFRJopuwquqvVKzd0z/MV0VobWztf97EY+aCxEY9WHBMxM6RqmZ+UpELuxsy
         lreJY2S/Nxca61dD//jmYhKwmOf/JmOIxFw6m9kg8IPIxtfr4oLSOFqjP5hSijDgRawH
         WJEHcLG4EQOS/jgaKbofz0oXeEv4lGptC3YRlgmv3JKHAwutiZ2kgf+3fOQMTZUMAc3n
         a/2g==
X-Gm-Message-State: AOJu0YwRZxfY9fc6R4HVcrYjUGbsA3Vmt9Oe2udOZGXLAT+Mw4sXk2Yn
	I5Y8hAAur8fV6Kl6XBlwde/GySPUccFRoXZc62l/gwrGKaDUNH3B
X-Google-Smtp-Source: AGHT+IGPGglg+V/00Jrg55McyLpL1F2ELorstlFLspgxxLXJnTblCWSYFl9w/vecAD44DNktsRIpzg==
X-Received: by 2002:a19:644a:0:b0:511:4eb4:c4ed with SMTP id b10-20020a19644a000000b005114eb4c4edmr750652lfj.45.1707471579745;
        Fri, 09 Feb 2024 01:39:39 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUEvC+GK8BdtojWPmhTZpydXwX0wQPxVQaiIZbKsLQMl5FlkeVpxc4TINBKICwZVBzjekdbN+WxRBwgob9afVbADDzJ1RQUXID4/Jhl8CbnyosNTbA/0+eaF03Pa2AnfNjTz3i3TUFMPuDS6xDf39V0+Fk4s8WEeqvfsVms5HjSyLF+vxryUxqa5OfPFuR2G2hPhbu6xEEShII/uWuAulzQHRaCTsWATxxWCwdzcg1D61OEpQljG3/Ovjryqrh6NstCgDYSSDpsJ+CQq0nxxi7w9TW/lg9kKj8PsZYOZDftg6Zet6RNR+ONShiIyAXAuKaG4E713nnVkbIYKRKjyQxjnTlNu6f3tlyqcrhtW0I2Y529Lhx5tDhXh3Pn5cx/eTN1UB5Is9SDelikGRHtgcX44KnRuNOHT0Q+YSmPLojpFJ/sttVCnd1Rn3jC9kKVb6dW5LoeApAcYdDbiL1gz6eQ/Ry1cvM3ZrtowNtiwMLf4R8Myzc=
Message-ID: <4670386b86a396de7706b3d77dede92992296a92.camel@gmail.com>
Subject: Re: [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(),
 ASSERT_UNREACHABLE() to xen/bug.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>,  Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, Shawn
 Anastasio <sanastasio@raptorengineering.com>, Roger Pau
 =?ISO-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>, Konrad Rzeszutek Wilk
 <konrad.wilk@oracle.com>, Ross Lagerwall <ross.lagerwall@citrix.com>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>, 
 xen-devel@lists.xenproject.org
Date: Fri, 09 Feb 2024 10:39:38 +0100
In-Reply-To: <91e58646-5420-4e1c-a8c5-b6ff7a417d13@suse.com>
References: 
	<4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
	 <91e58646-5420-4e1c-a8c5-b6ff7a417d13@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Thu, 2024-02-08 at 17:42 +0100, Jan Beulich wrote:
> On 02.02.2024 19:03, Oleksii Kurochko wrote:
> > Move the macros mentioned in the commit subject to their
> > appropriate
> > locations.
> > Additionally, eliminate the dependency of xen/lib.h from xen/bug.h
> > and
> > include "xen/bug.h" in files where xen/bug.h macros are utilized.
> >=20
> > Most of the changes were made because a file requires macros from
> > xen/bug.h,
> > except for some files for Arm which require definitions of
> > BUG_OPCODE,
> > BUG_INSTR, BUG_FN_REG.
> >=20
> > xen/lib.h was added to list-sort.c ( otherwise compilation errors
> > related
> > to {d}printk occur during compilation of list-sort.c. ) as
> > xen/lib.h was
> > removed from xen/list.h. Since nothing in xen/list.h depends on
> > xen/lib.h
> > functionality and only xen/bug.h is needed.
> >=20
> > cpufeature.h requires the inclusion of <xen/cache.h>;
> > otherwise, the following error will occur:
> > ld:
> > common/monitor.o:/build/xen/./arch/x86/include/asm/cpufeature.h:41:
> > multiple definitions of `__cacheline_aligned';
> >=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Acked-by: Jan Beulich <jbeulich@suse.com>
> with one nit:
>=20
> > --- a/xen/include/xen/cpumask.h
> > +++ b/xen/include/xen/cpumask.h
> > @@ -55,6 +55,7 @@
> > =C2=A0 * for_each_present_cpu(cpu)		for-loop cpu over
> > cpu_present_map
> > =C2=A0 */
> > =C2=A0
> > +#include <xen/bug.h>
> > =C2=A0#include <xen/bitmap.h>
> > =C2=A0#include <xen/kernel.h>
> > =C2=A0#include <xen/random.h>
>=20
> I think everywhere else you nicely inserted new #include-s so respect
> alphabetical ordering, just here the insertion would want to move one
> line down. Likely easy to do while committing, if no other reason for
> a v2 arises ...
I'll be happy if it can be done during committing.

Thanks.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 09:40:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 09:40:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678609.1056050 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNNH-0007CR-Sj; Fri, 09 Feb 2024 09:40:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678609.1056050; Fri, 09 Feb 2024 09:40:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNNH-0007CK-Ol; Fri, 09 Feb 2024 09:40:47 +0000
Received: by outflank-mailman (input) for mailman id 678609;
 Fri, 09 Feb 2024 09:40:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYNNH-0007Bz-Am
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 09:40:47 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4c5f0482-c72f-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 10:40:45 +0100 (CET)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2cf3ed3b917so7266291fa.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 01:40:45 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a6-20020a2e88c6000000b002d0c34de739sm177438ljk.105.2024.02.09.01.40.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 01:40:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c5f0482-c72f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707471645; x=1708076445; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=nMIWIk8UrRRacEEuL0vZEYRkZB38ASc7SyjxbcTn3AU=;
        b=Jy0veO4dlENiRNAlyGC7Y2irXiK+jsKkccfl2eydz1zEiBGg/a9unYk+THrWQg/haJ
         i0vinAiO/NXFHZjasK9Peq0YujwbpSjJTkcyZCasjcjkwiM5R3ADfkr0LIcAhwjtzhIT
         MmaPE4+0j4+ja8wo2dcyWPbBCphFsUrQzTx0Dm4XbOIsCU1ir6fadrez/wkDAKGml+wF
         DJAe1zk2wHey3lWfkg4B9+hTLyDlY3gkKceWEkYJHh5d3qUEco7RIDSh+xazM3WqWpCC
         1F/YDPLXNZEA/RQtM4MBNRNLRxJnl3/R5ySjHrwO2zXzOqTdqSzluglDM7yhXLXDdhb2
         H4gQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707471645; x=1708076445;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nMIWIk8UrRRacEEuL0vZEYRkZB38ASc7SyjxbcTn3AU=;
        b=vgqxrRQbh6GAjgqUAWd+RWQBasVq5R/XZ2hZmOHz027v7Xfp49pJ1pc54AEWJShs9r
         6B99nSX+tvVoyi++yYOxyGyqFUC1Z2jY3dfSBXV1W1AHBs65O/rU6/iHY9FinVKDdNEX
         Udwpct6Qi/U0uNoms7Wgzdx4DV8UNh2/77HP8RZtIQV4NVqI3ETitBKcJ44sWLToXeEI
         BRGVU3AHePCku547qTGR+vVq7vA4LufWLqlwVJb2jECIbWkAVy1SWuA0H9Q5V3X1XmbP
         URyeWVKdIGzuDmi3FZNa8mF+GC8G254D9CjkTwM5NdMoGEPAPZ4tDJVtjV6pvRMka0Sk
         Zl5Q==
X-Gm-Message-State: AOJu0YyNRJnY6Pot7QuASG/6xLM9AoC6CXSdnC4P04hXp/Z364qoC2A2
	juKyxMg8Ib6dUc5xIs7Pwe9I+lEd321RyyMoBfUlyaYICeCqUPgo
X-Google-Smtp-Source: AGHT+IH4mqRFBjZxoDil7XDFslrGlZtiEmW3p5aKqaA72XsAf0//uQdPo1b8TgPlszCI5pJeJswWXw==
X-Received: by 2002:a2e:b616:0:b0:2d0:b3c4:5113 with SMTP id r22-20020a2eb616000000b002d0b3c45113mr1027651ljn.11.1707471644980;
        Fri, 09 Feb 2024 01:40:44 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUbpAX5B/Q4W5sTtQATc+u0xmXMqrVC9Xc20JpKktz5DkvnijQzfc5XHKySVgT4qN8Y+6fHQHwz1yvboKpiV3h57JLc844kJoBdVpBVX+aZyYZBzT+Y9+YXK8jx014Ut47fhKCuakjJ2AEIKzlIhTMzNdqdUUOOedyRYjRGmh3YxRLJBg7uUYBmSSykalZQfUoPqhEOnK0htVTrvHApbD0SDsUDO3XajCUSpEojgv1k21ct7F5W+bmYsVN7RRJR6qt+CSVdfVRoRrTqE6f5C8GAjRjerTo3fqGdxET03FjfdU1yhuPU0UbIyNK+AujZoekb3TxN1Lih1SiHqPrKwzvRGferXBc2+aOoXuEk5OMHQNeV/MRb4Yi43U5VTHLG2kY/Ye9B4/iiTIkUuSrsScmVmvQD9z0J4xXqiQlobjiHbQ0Ho9aVRO2LMRnpsLMwPO/wbqnIP9CR9p0TQ9lQGPwkUJgwO7fTpNBw9rkz3VfvhqRX38cXgbs=
Message-ID: <9d1cadd4e918960d38426a8a4d83d18dbd4aa07e.camel@gmail.com>
Subject: Re: [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(),
 ASSERT_UNREACHABLE() to xen/bug.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Jan
 Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>,  Shawn Anastasio
 <sanastasio@raptorengineering.com>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>,  Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,  "Daniel P. Smith"
 <dpsmith@apertussolutions.com>
Date: Fri, 09 Feb 2024 10:40:43 +0100
In-Reply-To: <07e4b9b0-5ddd-4499-b280-bd5d583b8b8c@xen.org>
References: 
	<4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
	 <07e4b9b0-5ddd-4499-b280-bd5d583b8b8c@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Thu, 2024-02-08 at 21:30 +0000, Julien Grall wrote:
> Hi Oleksii,
Hi Julien,

>=20
> On 02/02/2024 18:03, Oleksii Kurochko wrote:
> > Move the macros mentioned in the commit subject to their
> > appropriate
> > locations.
> > Additionally, eliminate the dependency of xen/lib.h from xen/bug.h
> > and
> > include "xen/bug.h" in files where xen/bug.h macros are utilized.
> >=20
> > Most of the changes were made because a file requires macros from
> > xen/bug.h,
> > except for some files for Arm which require definitions of
> > BUG_OPCODE,
> > BUG_INSTR, BUG_FN_REG.
> >=20
> > xen/lib.h was added to list-sort.c ( otherwise compilation errors
> > related
> > to {d}printk occur during compilation of list-sort.c. ) as
> > xen/lib.h was
> > removed from xen/list.h. Since nothing in xen/list.h depends on
> > xen/lib.h
> > functionality and only xen/bug.h is needed.
> >=20
> > cpufeature.h requires the inclusion of <xen/cache.h>;
> > otherwise, the following error will occur:
> > ld:
> > common/monitor.o:/build/xen/./arch/x86/include/asm/cpufeature.h:41:
> > multiple definitions of `__cacheline_aligned';
> >=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Acked-by: Julien Grall <jgrall@amazon.com>
Thanks.

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 09:40:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 09:40:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678610.1056060 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNNK-0007Ri-60; Fri, 09 Feb 2024 09:40:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678610.1056060; Fri, 09 Feb 2024 09:40:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNNK-0007RZ-2o; Fri, 09 Feb 2024 09:40:50 +0000
Received: by outflank-mailman (input) for mailman id 678610;
 Fri, 09 Feb 2024 09:40:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rYNNI-0007OP-K8
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 09:40:48 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYNNE-0007s9-Hc; Fri, 09 Feb 2024 09:40:44 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYNNE-0004vM-CT; Fri, 09 Feb 2024 09:40:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=WcBzPkO8rEnA0O+IN3gB0CE7cR4nQt10Lo5BDEzZa8o=; b=raVHHxnqiN5B8TmXzxyAhz2yzQ
	wILWAEIALVdaqCbt5Fz310pW1DjIoSkHN4hAS0hmPX2a+ZkU/CDpkPdMnoGyQuMQQr4VD3c7iJIQ1
	FcyxCZ64kwm7qHhxOFjef7NgqNjEdrw4OEUdFXrwGxUT8VXij7chZOWdaESk7ayv+HFE=;
Message-ID: <8977390f-9549-40d5-bfdc-5c3da81521fc@xen.org>
Date: Fri, 9 Feb 2024 09:40:42 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2] docs/misra: add asm-offset.c to exclude-list
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <09b659e19bf2cc6b3ee4320e019bdfa7def5f3b8.1707406598.git.nicola.vetrini@bugseng.com>
 <891e8cc2-e073-4cd3-9998-4c01ae775a3b@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <891e8cc2-e073-4cd3-9998-4c01ae775a3b@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

Replying on the v2 as well.

On 08/02/2024 15:56, Jan Beulich wrote:
> On 08.02.2024 16:50, Nicola Vetrini wrote:
>> These files contain several deliberate violations of MISRA C rules and
>> they are not linked in the final Xen binary, therefore they can be exempted
>> from MISRA compliance.

I'd like the commit message to be expanded a little bit to explain which 
MISRA rules are a problem. This helped me to understand why we excluded 
rather than fixed.

Base on the previous discussion, I would suggest:

These files contain several deliberate violation of MISRA C rules such as:
   * R20.12 on Arm for macros DEFINE and OFFSET, where the second 
argument of OFFSET is a macro and is used as a normal parameter and a 
stringification operand.
   * R2.1 because the file is not linked That said it was decided to 
deviate the rule itselfed to deviate that aspect).

The files are also not linked in the final Xen binary, therefore they 
can be expempted from MISRA compliance.

>>
>> No functional change.
>>
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> Acked-by: Jan Beulich <jbeulich@suse.com>
> albeit ...
> 
>> --- a/docs/misra/exclude-list.json
>> +++ b/docs/misra/exclude-list.json
>> @@ -101,6 +101,10 @@
>>               "rel_path": "arch/x86/efi/check.c",
>>               "comment": "The resulting code is not included in the final Xen binary, ignore for now"
>>           },
>> +        {
>> +          "rel_path": "arch/*/*/asm-offsets.c",
>> +          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
>> +        },
>>           {
>>               "rel_path": "common/coverage/*",
>>               "comment": "Files to support gcov, ignore for now"
> 
> ... something looks odd with indentation; can probably be adjusted
> while committing.

I am happy to take care of both the commit message and the indentation 
on commit.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 09:50:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 09:50:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678620.1056069 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNWX-0001fY-25; Fri, 09 Feb 2024 09:50:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678620.1056069; Fri, 09 Feb 2024 09:50:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYNWW-0001fR-Vc; Fri, 09 Feb 2024 09:50:20 +0000
Received: by outflank-mailman (input) for mailman id 678620;
 Fri, 09 Feb 2024 09:50:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KbLn=JS=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rYNWW-0001fG-2n
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 09:50:20 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a219e824-c730-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 10:50:18 +0100 (CET)
Received: from [192.168.1.15] (host-87-16-84-78.retail.telecomitalia.it
 [87.16.84.78])
 by support.bugseng.com (Postfix) with ESMTPSA id 84B3A4EE0739;
 Fri,  9 Feb 2024 10:50:18 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a219e824-c730-11ee-8a4b-1f161083a0e0
Message-ID: <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
Date: Fri, 9 Feb 2024 10:50:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
 <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
Content-Language: en-US, it
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 08/02/24 12:14, Jan Beulich wrote:
> On 08.02.2024 11:45, Federico Serafini wrote:
>> On 07/02/24 17:19, Jan Beulich wrote:
>>> On 07.02.2024 16:58, Federico Serafini wrote:
>>>> On 07/02/24 16:24, Jan Beulich wrote:
>>>>> On 07.02.2024 16:08, Federico Serafini wrote:
>>>>>> On 07/02/24 15:16, Jan Beulich wrote:
>>>>>>> On 07.02.2024 14:51, Federico Serafini wrote:
>>>>>>>> On 07/02/24 08:38, Jan Beulich wrote:
>>>>>>>>> On 07.02.2024 02:08, Stefano Stabellini wrote:
>>>>>>>>>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>>>>>>>>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>>>>>>>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>>>>>>>>>           case 8:                                                                \
>>>>>>>>>>>>               put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>>>>>>>>>               break;                                                             \
>>>>>>>>>>>> -    default: __put_user_bad();                                             \
>>>>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>>>>           }                                                                      \
>>>>>>>>>>>>           clac();                                                                \
>>>>>>>>>>>>       } while ( false )
>>>>>>>>>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>>>>>>>>>           case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>>>>>>>>>           case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>>>>>>>>>           case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>>>>>>>>>> -    default: __get_user_bad();                                             \
>>>>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>>>>           }                                                                      \
>>>>>>>>>>>>           clac();                                                                \
>>>>>>>>>>>>       } while ( false )
>>>>>>>>>>>
>>>>>>>>>>> Related to my remark on patch 1 - how is one to know the macro this was
>>>>>>>>>>> invoked from, when seeing the resulting diagnostic?
>>>>>>>>>>
>>>>>>>>>> I am not sure what do you mean here... we do get an error like the
>>>>>>>>>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>>>>>>>>>
>>>>>>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>>>>>>
>>>>>>>>> Right - and how do I know what _user_ of the macro actually triggered
>>>>>>>>> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
>>>>>>>>> __FUNCTION__ here, for that specific purpose ...
>>>>>>>>
>>>>>>>> To test the macro and its diagnostics,
>>>>>>>> I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
>>>>>>>> on the x86 code with STATIC_ASSERT_UNREACHABLE(),
>>>>>>>> that is in file arch/x86/alternative.c, line 312,
>>>>>>>> function _apply_alternatives().
>>>>>>>>
>>>>>>>> What I got is the following build error:
>>>>>>>>
>>>>>>>> ...
>>>>>>>> arch/x86/alternative.c: Assembler messages:
>>>>>>>> arch/x86/alternative.c:312: Error: static assertion failed: unreachable
>>>>>>>>        CC      arch/x86/copy_page.o
>>>>>>>> make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1
>>>>>>>
>>>>>>> But that's not what my request was about. Here sufficient context is
>>>>>>> given, even if it would be nice if the function was also visible right
>>>>>>> away. But that's not the same as the case above, where the new macro
>>>>>>> is used inside another macro.
>>>>>>
>>>>>> An example of that is the get_unsafe_size() macro,
>>>>>> whose body uses STATIC_ASSERT_UNREACHABLE().
>>>>>> A wrong use of get_unsafe_size() at line n
>>>>>> leads to a build error pointing to the line n,
>>>>>> isn't this the desired behavior?
>>>>>
>>>>> Aiui this would point to the line in the header file, when what you need
>>>>> to spot the bad use of the macro is the line in the source file actually
>>>>> using the macro. Quoting from an earlier mail of yours:
>>>>>
>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>
>>>> It points to the header file uaccess.h because at line 262 there is
>>>> an intentional wrong use of put_guest_size(), within the body of
>>>> __copy_to_guest_pv() function.
>>>
>>> Yet that's again only a helper function being inlined into the ultimate
>>> caller. That ultimate caller is what wants identifying in the diag. Not
>>> the least because of ...
>>>
>>>> This example can be misleading because {get,put}_unsafe_size() are
>>>> defined in the same file but the diagnostics is doing the
>>>> right thing.
>>>
>>> ... this. And really __copy_to_guest_pv() is the wrong place to put a
>>> wrong put_guest_size() in, to try out how diagnostics would look like
>>> in reality: That function falls back to copy_to_guest_ll() for all
>>> cases it can't handle directly. You want to place a bogus put_guest()
>>> somewhere in a .c file to see what results.
>>
>> I added a bogus call to put_guest() at line 387 of
>> file xen/arch/x86/mm.c, inside function page_is_ram_type().
>> Assuming I did not choose another wrong place,
>> the diagnostic seems appropriate:
>>
>> arch/x86/mm.c: Assembler messages:
>> arch/x86/mm.c:387: Error: static assertion failed: unreachable
> 
> Oh, okay, this looks appropriate then as to identifying where the
> source construct is. However, we then still don't know where the
> assertion in question is (there could be multiple in what the
> original construct expands to). So I'm still inclined to ask that
> __FILE__ / __LINE__ and/or the name of the invoking construct
> (macro or function) be made visible in the diagnostic.

Any use of __FILE__ and __LINE__ results in obtaining
the same information already reported by the assembler error message.

We could add an argument to the new macro to manually add some context
at every use of the macro, but I think this would be annoying.

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 10:46:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 10:46:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678634.1056080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYOOL-0001Zh-6b; Fri, 09 Feb 2024 10:45:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678634.1056080; Fri, 09 Feb 2024 10:45:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYOOL-0001Za-2D; Fri, 09 Feb 2024 10:45:57 +0000
Received: by outflank-mailman (input) for mailman id 678634;
 Fri, 09 Feb 2024 10:45:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aSO/=JS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rYOOJ-0001ZU-7M
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 10:45:55 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 656706bb-c738-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 11:45:53 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33adec41b55so274989f8f.0
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 02:45:53 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 bt2-20020a056000080200b0033b4db744e5sm1500245wrb.12.2024.02.09.02.45.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 02:45:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 656706bb-c738-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707475552; x=1708080352; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Xw8I61+V3ko8S2OozdYFKiO1k+Am7u32/BltW5Y0w3s=;
        b=HfoklA2cNpGlqSmn/3DZMLWKYy+WFy/nl5hwt0RQBKTH1R92uufjaCQ1CWLRCwnco8
         hJqTm0/8rT6FYMoEeZi8Gi6tWc5zmoIFAvc0mHHeVvEAZfqKiR6v2GAw3/rP++jwjqR4
         uZ0fWHp1XVUfOAi4cXkXsFoNjjYHLT1I1O+ac=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707475552; x=1708080352;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Xw8I61+V3ko8S2OozdYFKiO1k+Am7u32/BltW5Y0w3s=;
        b=MQLhiCKWIiqRKWfM5jm8T9OzivIZA8nV7qHf5xXFFadVNUZxnUNiZpCKf2D9TsTiuk
         Yq8LVrl3gRUGsRRoNqob5Nm+6CEANmt/5U9Q/vG+oUCVzoOSQK9cwrYdjJpuRXoZfTaj
         Z0M4u5TOHIQ7Tpcg3eWDsFoYxmQu2zuGhGqgTaOhyDLtvaDm3NGzweUQr6lNKA288pMt
         7mCmpvmg0z91MZg7fLRxKIv9mRK49CSDjJmPndVAoJtp++7TBwstN+aKC5wYQphH//b9
         7UIK/Q13xmBlZTV4z7P/G/urTj5CcK+TvE1tNn+KTl0s0/Kgp/5fFGBlZLrsZUNTq1sN
         Mf1w==
X-Gm-Message-State: AOJu0YzOSJn3xIL/xDy9Rqth+YQ0qAX3ciSeymwamIhMObPHc3x0J0Gn
	T4qQyaGfMx8W8k2mxI71CV5qqsAcfwEJoo8klsSB79Bj5lswOH+UOadkK2bXbD8=
X-Google-Smtp-Source: AGHT+IHdwuWu+yU+Hg22tV6XSuSR4iPdVWj841mjdjldKM/cfP8a+J2NSacP9Od1RqLsKp22DeJHgw==
X-Received: by 2002:adf:b1d7:0:b0:33a:ea26:6a91 with SMTP id r23-20020adfb1d7000000b0033aea266a91mr833540wra.0.1707475552330;
        Fri, 09 Feb 2024 02:45:52 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWI4bwmJOjY0Gdkt/bCrPBk2/sbLSWa4p3OfWmTCxkDbPNXoh8vHe/dcMymFnMQm8Y6zZcxjHkoXhVZlYOQUsK2sMqmIdWQEpXbYkJUcU806l37TUfPKSlJ9madGiKVDIkMOUVd8F+yhZkERS1SgZ18QLHOWFVgNQkc3HJUI+ekTIeDlOPIOptfbkKh
Date: Fri, 9 Feb 2024 11:45:50 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] x86/vmx: add support for virtualize SPEC_CTRL
Message-ID: <ZcYCXrEaOyxZUb2I@macbook>
References: <20240206142507.81985-1-roger.pau@citrix.com>
 <91e3fd09-8325-49b0-9d7b-43aacf2acd81@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <91e3fd09-8325-49b0-9d7b-43aacf2acd81@suse.com>

On Thu, Feb 08, 2024 at 02:40:53PM +0100, Jan Beulich wrote:
> On 06.02.2024 15:25, Roger Pau Monne wrote:
> > @@ -2086,6 +2091,9 @@ void vmcs_dump_vcpu(struct vcpu *v)
> >      if ( v->arch.hvm.vmx.secondary_exec_control &
> >           SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY )
> >          printk("InterruptStatus = %04x\n", vmr16(GUEST_INTR_STATUS));
> > +    if ( cpu_has_vmx_virt_spec_ctrl )
> > +        printk("SPEC_CTRL mask = %#016lx  shadow = %#016lx\n",
> > +               vmr(SPEC_CTRL_MASK), vmr(SPEC_CTRL_SHADOW));
> 
> #0... doesn't make a lot of sense; only e.g. %#lx does. Seeing context
> there's no 0x prefix there anyway. Having looked at the function the
> other day, I know though that there's a fair mix of 0x-prefixed and
> unprefixed hex numbers that are output.

For consistency with how other MSRs are printed I should use the '0x'
prefix.

> Personally I'd prefer if all
> 0x prefixes were omitted here. If you and Andrew think otherwise, I can
> live with that, so long as we're at least striving towards consistent
> output (I may be able to get to doing a conversion patch, once I know
> which way the conversion should be).

I usually prefer the '0x' to avoid ambiguity.  However this being all
hardware registers, I might be fine with dropping the '0x' on the
grounds that all registers are always printed as hex.

> > --- a/xen/arch/x86/hvm/vmx/vmx.c
> > +++ b/xen/arch/x86/hvm/vmx/vmx.c
> > @@ -823,18 +823,28 @@ static void cf_check vmx_cpuid_policy_changed(struct vcpu *v)
> >      {
> >          vmx_clear_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
> >  
> > -        rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
> > -        if ( rc )
> > -            goto out;
> > +        if ( !cpu_has_vmx_virt_spec_ctrl )
> > +        {
> > +            rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
> > +            if ( rc )
> > +                goto out;
> > +        }
> 
> I'm certainly okay with you doing it this way, but generally I'd prefer
> if code churn was limited whjere possible. Here leveraging that rc is 0
> on entry, a smaller change would be to
> 
>         if ( !cpu_has_vmx_virt_spec_ctrl )
>             rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
>         if ( rc )
>             goto out;
> 
> (similarly below then).

That looks odd to me, and is not how I would write that code.  I can
however adjust if you insist.  Given it's just a two line difference I
think it was worth having the more usual form.

> >      else
> >      {
> >          vmx_set_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
> >  
> > -        rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
> > -        if ( rc && rc != -ESRCH )
> > -            goto out;
> > -        rc = 0; /* Tolerate -ESRCH */
> > +        /*
> > +         * NB: there's no need to clear the virtualize SPEC_CTRL control, as
> > +         * the MSR intercept takes precedence.
> > +         */
> 
> The two VMCS values are, aiui, unused during guest entry/exit. Maybe
> worth mentioning here as well, as that not being the case would also
> raise correctness questions?

Hm, yes indeed, I've double checked and the value is not loaded, so
will expand the message.

> > --- a/xen/arch/x86/include/asm/msr.h
> > +++ b/xen/arch/x86/include/asm/msr.h
> > @@ -302,8 +302,13 @@ struct vcpu_msrs
> >       * For PV guests, this holds the guest kernel value.  It is accessed on
> >       * every entry/exit path.
> >       *
> > -     * For VT-x guests, the guest value is held in the MSR guest load/save
> > -     * list.
> > +     * For VT-x guests, the guest value is held in the MSR guest load/save list
> > +     * if there's no support for virtualized SPEC_CTRL. If virtualized
> > +     * SPEC_CTRL is enabled the value here signals which bits in SPEC_CTRL the
> > +     * guest is not able to modify.  Note that the value for those bits used in
> > +     * Xen context is also used in the guest context.  Setting a bit here
> > +     * doesn't force such bit to set in the guest context unless also set in
> > +     * Xen selection of SPEC_CTRL.
> 
> Hmm, this mask value is unlikely to be in need of being vCPU-specific.
> I'd not even expect it to be per-domain, but simply global.

This is mostly to keep the logic in-sync with the one used on AMD.

> I also can't spot where you set that field; do we really mean to give
> guests full control now that we have it (rather than e.g. running in
> IBRS-always-on mode at least under certain conditions)? If intended to
> be like this for now, this (to me at least) surprising aspect could
> likely do with mentioning in the description.

Yes, so far I didn't set any bit before the guest back, that should be
done in a separate patch.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 11:24:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 11:24:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678640.1056090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYOzJ-0007Ww-1E; Fri, 09 Feb 2024 11:24:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678640.1056090; Fri, 09 Feb 2024 11:24:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYOzI-0007Wp-UW; Fri, 09 Feb 2024 11:24:08 +0000
Received: by outflank-mailman (input) for mailman id 678640;
 Fri, 09 Feb 2024 11:24:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYOzG-0007Wf-TS; Fri, 09 Feb 2024 11:24:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYOzG-0001Xt-J5; Fri, 09 Feb 2024 11:24:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYOzG-0008Jo-6H; Fri, 09 Feb 2024 11:24:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYOzG-0007N5-5d; Fri, 09 Feb 2024 11:24:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=IiZlAI2/IpjvzD02O3p2uq56RI7zlufrhVVgZMMlXwo=; b=BniQNV37MPS0/EielcizDHxOpr
	hNG6PHJfQobaYlMRAciJH2V2dUn1Vsl0bbEem47neLK/+A0q2e/P8gHaNrVjhTgEaChM4Ql6ZDuuy
	UyvSgsHpgb+99PzZvG6FM3+omdT0X69yBBc+RyW3vuEU4PFcTzjAmGIWIiKVqMpMtgGg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184634-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184634: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
X-Osstest-Versions-That:
    xen=e11f5766503c0ff074b4e0f888bbfc931518a169
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 09 Feb 2024 11:24:06 +0000

flight 184634 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184634/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
baseline version:
 xen                  e11f5766503c0ff074b4e0f888bbfc931518a169

Last test of basis   184627  2024-02-08 17:03:44 Z    0 days
Testing same since   184634  2024-02-09 08:02:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Christian Lindig <christian.lindig@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Michal Orzel <michal.orzel@amd.com>
  Petr Beneš <w1benny@gmail.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   e11f576650..4e5bd87d24  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369 -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 11:41:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 11:41:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678647.1056099 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPG0-00026J-Df; Fri, 09 Feb 2024 11:41:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678647.1056099; Fri, 09 Feb 2024 11:41:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPG0-00026C-Am; Fri, 09 Feb 2024 11:41:24 +0000
Received: by outflank-mailman (input) for mailman id 678647;
 Fri, 09 Feb 2024 11:41:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aSO/=JS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rYPFy-000266-GY
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 11:41:22 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2540a61f-c740-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 12:41:21 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a38392b9917so111224366b.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 03:41:21 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 q18-20020a17090609b200b00a3a40b7ecdbsm677798eje.69.2024.02.09.03.41.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 03:41:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2540a61f-c740-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707478880; x=1708083680; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Xkhp1PJ17XuWxmN1RggcbPkPzxtBkxTMYnDQw1aIVAM=;
        b=Uw7KCBF8dBYFj/20xPwIJxYZAEOEALfOVhiQkx0Ws2ZGolXprz9ntXaIO+P15yp3kA
         DR4WuDZUbTIEp+t/1dt7/6/iDCLzWmr4nLy2WYLZHqkHnAMT9dD7t5JCKJzYqkR5nesO
         ZATP+kiTXkQMj0af8LdAP/0FMyGt1G5tnmyLU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707478880; x=1708083680;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Xkhp1PJ17XuWxmN1RggcbPkPzxtBkxTMYnDQw1aIVAM=;
        b=LMrwVngn+AZf0Isj4sxXtuElMsrgawgiV93SxbRGlSw1mjgXbZGR1iiuuhJwnSCrju
         de5G3HjJ0TNGcAywi52ZtDkKX7nMyZgHIVaSrGbbEbBAZck5HpIUUAUrRUKzXFAfjIwd
         BGqIFLb758rYb8QOSgSYtWcuREM7zJ+QHKm234DWZ77vNq69cHZNujC4COkWFR5MBRqE
         FFAuprn5sX4UcZAoj1bO9e41YRskesU0xOJp6bdN3D6w0r1agfhjYPtmQRJ6vLLP47+Q
         UYYZY74OTwfoSsII8PSp3aMKMqzc307QaAIc+vnxtiXCquc0w2j8T+oSvQZVMPYC2TwG
         wo9g==
X-Gm-Message-State: AOJu0Yys8oSwKV04+Q0SAq2RHDtpQhwAGGmhvVSLZUkWHXsEHlQu+cF1
	d7ybtpNtMj3/FpAvJ72IJuIRiSrMsIQ9fRs6CFk9C9RLlBslOKdqafdopSM6DQd3MQG6veZbhcj
	F
X-Google-Smtp-Source: AGHT+IE4JIDuMxyjgMZ5SdPB9b0tR1hC6YJtZnZJl4k55QtaKWT0EYEICeUJ3qhFkuomVDPXt2yZLQ==
X-Received: by 2002:a17:906:3455:b0:a3c:267:112a with SMTP id d21-20020a170906345500b00a3c0267112amr437012ejb.12.1707478880492;
        Fri, 09 Feb 2024 03:41:20 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXywGqP+ITP3OKYjlAbchXWUwZku3PQPv7BHHiY2H9HWGPOeSw6Kg/PE29R0UffrXY/ZwrorLXC6EDn1CvT7jM7hs4K/FJ75p1Z3ycK+SQURUdR49uM8dQ+KmKnIIEvz8YanJqPl9RnBPShACOgFWaa4SXxmblQLuXiKsA9XDEXrVBmQJUjTODf8ikE+TJkYUS9vegz4Wo=
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2] x86/vmx: add support for virtualize SPEC_CTRL
Date: Fri,  9 Feb 2024 12:40:45 +0100
Message-ID: <20240209114045.97005-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The feature is defined in the tertiary exec control, and is available starting
from Sapphire Rapids and Alder Lake CPUs.

When enabled, two extra VMCS fields are used: SPEC_CTRL mask and shadow.  Bits
set in mask are not allowed to be toggled by the guest (either set or clear)
and the value in the shadow field is the value the guest expects to be in the
SPEC_CTRL register.

By using it the hypervisor can force the value of SPEC_CTRL bits behind the
guest back without having to trap all accesses to SPEC_CTRL, note that no bits
are forced into the guest as part of this patch.  It also allows getting rid of
SPEC_CTRL in the guest MSR load list, since the value in the shadow field will
be loaded by the hardware on vmentry.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Expand commit message and code comments.
 - Prefix the output of the VMCS dump with '0x'.
---
 xen/arch/x86/hvm/vmx/vmcs.c             | 10 +++++-
 xen/arch/x86/hvm/vmx/vmx.c              | 41 ++++++++++++++++++++-----
 xen/arch/x86/include/asm/hvm/vmx/vmcs.h |  5 +++
 xen/arch/x86/include/asm/msr.h          |  9 ++++--
 4 files changed, 55 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 9e016634ab5c..dc46adb02595 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -202,6 +202,7 @@ static void __init vmx_display_features(void)
     P(cpu_has_vmx_tsc_scaling, "TSC Scaling");
     P(cpu_has_vmx_bus_lock_detection, "Bus Lock Detection");
     P(cpu_has_vmx_notify_vm_exiting, "Notify VM Exit");
+    P(cpu_has_vmx_virt_spec_ctrl, "Virtualize SPEC_CTRL");
 #undef P
 
     if ( !printed )
@@ -365,7 +366,7 @@ static int vmx_init_vmcs_config(bool bsp)
 
     if ( _vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS )
     {
-        uint64_t opt = 0;
+        uint64_t opt = TERTIARY_EXEC_VIRT_SPEC_CTRL;
 
         _vmx_tertiary_exec_control = adjust_vmx_controls2(
             "Tertiary Exec Control", 0, opt,
@@ -1378,6 +1379,10 @@ static int construct_vmcs(struct vcpu *v)
         rc = vmx_add_msr(v, MSR_PRED_CMD, PRED_CMD_IBPB,
                          VMX_MSR_HOST);
 
+    /* Set any bits we don't allow toggling in the mask field. */
+    if ( cpu_has_vmx_virt_spec_ctrl && v->arch.msrs->spec_ctrl.raw )
+        __vmwrite(SPEC_CTRL_MASK, v->arch.msrs->spec_ctrl.raw);
+
  out:
     vmx_vmcs_exit(v);
 
@@ -2086,6 +2091,9 @@ void vmcs_dump_vcpu(struct vcpu *v)
     if ( v->arch.hvm.vmx.secondary_exec_control &
          SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY )
         printk("InterruptStatus = %04x\n", vmr16(GUEST_INTR_STATUS));
+    if ( cpu_has_vmx_virt_spec_ctrl )
+        printk("SPEC_CTRL mask = 0x%016lx  shadow = 0x%016lx\n",
+               vmr(SPEC_CTRL_MASK), vmr(SPEC_CTRL_SHADOW));
 
     printk("*** Host State ***\n");
     printk("RIP = 0x%016lx (%ps)  RSP = 0x%016lx\n",
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 48376cc32751..33cffb4f8747 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -823,18 +823,29 @@ static void cf_check vmx_cpuid_policy_changed(struct vcpu *v)
     {
         vmx_clear_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
 
-        rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
-        if ( rc )
-            goto out;
+        if ( !cpu_has_vmx_virt_spec_ctrl )
+        {
+            rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
+            if ( rc )
+                goto out;
+        }
     }
     else
     {
         vmx_set_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
 
-        rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
-        if ( rc && rc != -ESRCH )
-            goto out;
-        rc = 0; /* Tolerate -ESRCH */
+        /*
+         * NB: there's no need to clear the virtualize SPEC_CTRL control, as
+         * the MSR intercept takes precedence.  The SPEC_CTRL shadow VMCS field
+         * is also not loaded on guest entry/exit if the intercept is set.
+         */
+        if ( !cpu_has_vmx_virt_spec_ctrl )
+        {
+            rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
+            if ( rc && rc != -ESRCH )
+                goto out;
+            rc = 0; /* Tolerate -ESRCH */
+        }
     }
 
     /* MSR_PRED_CMD is safe to pass through if the guest knows about it. */
@@ -2629,6 +2640,9 @@ static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
     switch ( reg )
     {
     case MSR_SPEC_CTRL:
+        if ( cpu_has_vmx_virt_spec_ctrl )
+            /* Requires remote VMCS loaded - fetched below. */
+            break;
         rc = vmx_read_guest_msr(v, reg, &val);
         if ( rc )
         {
@@ -2652,6 +2666,11 @@ static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
     vmx_vmcs_enter(v);
     switch ( reg )
     {
+    case MSR_SPEC_CTRL:
+        ASSERT(cpu_has_vmx_virt_spec_ctrl);
+        __vmread(SPEC_CTRL_SHADOW, &val);
+        break;
+
     case MSR_IA32_BNDCFGS:
         __vmread(GUEST_BNDCFGS, &val);
         break;
@@ -2678,6 +2697,9 @@ static void cf_check vmx_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
     switch ( reg )
     {
     case MSR_SPEC_CTRL:
+        if ( cpu_has_vmx_virt_spec_ctrl )
+            /* Requires remote VMCS loaded - fetched below. */
+            break;
         rc = vmx_write_guest_msr(v, reg, val);
         if ( rc )
         {
@@ -2698,6 +2720,11 @@ static void cf_check vmx_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
     vmx_vmcs_enter(v);
     switch ( reg )
     {
+    case MSR_SPEC_CTRL:
+        ASSERT(cpu_has_vmx_virt_spec_ctrl);
+        __vmwrite(SPEC_CTRL_SHADOW, val);
+        break;
+
     case MSR_IA32_BNDCFGS:
         __vmwrite(GUEST_BNDCFGS, val);
         break;
diff --git a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
index a7dd2eeffcad..58140af69153 100644
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -270,6 +270,9 @@ extern u32 vmx_secondary_exec_control;
 #define TERTIARY_EXEC_VIRT_SPEC_CTRL            BIT(7, UL)
 extern uint64_t vmx_tertiary_exec_control;
 
+#define cpu_has_vmx_virt_spec_ctrl \
+     (vmx_tertiary_exec_control & TERTIARY_EXEC_VIRT_SPEC_CTRL)
+
 #define VMX_EPT_EXEC_ONLY_SUPPORTED                         0x00000001
 #define VMX_EPT_WALK_LENGTH_4_SUPPORTED                     0x00000040
 #define VMX_EPT_MEMORY_TYPE_UC                              0x00000100
@@ -436,6 +439,8 @@ enum vmcs_field {
     XSS_EXIT_BITMAP                 = 0x0000202c,
     TSC_MULTIPLIER                  = 0x00002032,
     TERTIARY_VM_EXEC_CONTROL        = 0x00002034,
+    SPEC_CTRL_MASK                  = 0x0000204a,
+    SPEC_CTRL_SHADOW                = 0x0000204c,
     GUEST_PHYSICAL_ADDRESS          = 0x00002400,
     VMCS_LINK_POINTER               = 0x00002800,
     GUEST_IA32_DEBUGCTL             = 0x00002802,
diff --git a/xen/arch/x86/include/asm/msr.h b/xen/arch/x86/include/asm/msr.h
index 1d8ea9f26faa..eed7b36cd992 100644
--- a/xen/arch/x86/include/asm/msr.h
+++ b/xen/arch/x86/include/asm/msr.h
@@ -302,8 +302,13 @@ struct vcpu_msrs
      * For PV guests, this holds the guest kernel value.  It is accessed on
      * every entry/exit path.
      *
-     * For VT-x guests, the guest value is held in the MSR guest load/save
-     * list.
+     * For VT-x guests, the guest value is held in the MSR guest load/save list
+     * if there's no support for virtualized SPEC_CTRL. If virtualized
+     * SPEC_CTRL is enabled the value here signals which bits in SPEC_CTRL the
+     * guest is not able to modify.  Note that the value for those bits used in
+     * Xen context is also used in the guest context.  Setting a bit here
+     * doesn't force such bit to set in the guest context unless also set in
+     * Xen selection of SPEC_CTRL.
      *
      * For SVM, the guest value lives in the VMCB, and hardware saves/restores
      * the host value automatically.  However, guests run with the OR of the
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 11:47:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 11:47:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678652.1056109 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPLK-0002kl-3h; Fri, 09 Feb 2024 11:46:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678652.1056109; Fri, 09 Feb 2024 11:46:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPLK-0002ke-1B; Fri, 09 Feb 2024 11:46:54 +0000
Received: by outflank-mailman (input) for mailman id 678652;
 Fri, 09 Feb 2024 11:46:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4TWb=JS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rYPLI-0002kX-B4
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 11:46:52 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e99f18bc-c740-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 12:46:51 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-33b2960ff60so1103957f8f.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 03:46:51 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v3-20020a5d59c3000000b0033b66c2d61esm918936wry.48.2024.02.09.03.46.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 09 Feb 2024 03:46:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e99f18bc-c740-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707479210; x=1708084010; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PiEferPHOA1KnMDY4S9rwu+csM6PS8dV3Ai9ITuH0uc=;
        b=UrNZ5/NVWzrwV7cU0ejqGfizR3Qo9CR+rR+OpGzZq7jcZoKBD64C678fR3ONRlM+P9
         pm9P3NpfUaC3uDxt3D2oK+qDZYxko4KQZ+oGWkP0H2KJC9JIKicEjP87eXTZxfITxvMZ
         Xoz+6sFlpuAwF75yVglLKquKO2vKpYfMVHhmnHbRXhiQjAIEDXg9HvgU9AEO1eNl4366
         4kbIIajkKrdT0bfORmQTrSdEf8F8zAhbK2MXlHWsSeffqR0XpmJ2283GrDbrpO5e4Y01
         P2Lm9fvpkbvPm0uMXfn9L4dN0uLvuqgUb8/MqPvlQ5NxVF6hdgPLwPANmD0yUZ7EoKsz
         mV7w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707479210; x=1708084010;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PiEferPHOA1KnMDY4S9rwu+csM6PS8dV3Ai9ITuH0uc=;
        b=koNGDOG7WkwhnMgzojcBwv03QOylVqCrcqx+U3MkEDS5ydYAE4lUyXBcNVOoXOXlVK
         ZbI3r/qdm5I8mF+wwat7A6Wm5MdTmh9k6GOAvXIWnJgDrTKALZu8bn04rsYqaKdUbzmg
         OSFm0n8kBFnO53ELN6u7AKeeSYut15MboK6x1JwVsJY/EglahWTE+GPWOHaWA0w3coyd
         2r6W3PmG6d5HitiwkA42AdP2m/+nYcMjss2Rb2WP2UBa5vLQIqv/2EkKANVQIdi/M4Cl
         LNIU8l4P6fOlphmD9Y5JMkF2Ln29sNRoMMeDMmg7Nzjbk+KzkzESIg5KTDvpDq1fFFPV
         Mpsg==
X-Forwarded-Encrypted: i=1; AJvYcCUOdBoii3tnl/Dfm8L1rS5IlceDa5YTGiBNN7K59Nk4zTkrAeu/C1zWgMSte+iDAmfVHiRBeNWOYWrq8jrm7vMuF4pXVPtz+3gcI2bnNQ4=
X-Gm-Message-State: AOJu0YwbuThbw0IaQwzUMcYT9QBWlrTReM+FSOFZ9/V/X16e3cgs0pyR
	7utByMnpi86dpud9JTBOZQ2EzADyWrjPIB2YWIybz6kRfdGmV4RFi81XIR2C1w==
X-Google-Smtp-Source: AGHT+IGr8l83hIgRTnIZkYDs1ZxRuNO9ekH1PsLv7Ku/lMQTgf3mbYHT2NFBp/LTcc9kbdX+8O0adA==
X-Received: by 2002:adf:fb81:0:b0:33b:3c05:cafb with SMTP id a1-20020adffb81000000b0033b3c05cafbmr856871wrr.16.1707479210201;
        Fri, 09 Feb 2024 03:46:50 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU71sj4KFkgLkNzmL98QHZA6LHuxDn4AwXtnAd+39Pl+4HZu0HIzHDTOkPhr1wQ3Q2dMthMlVYoiU/YOM2wuVU3bD3hKP9tvyaBsnRru2r1JVO3nVTmfG8XC/PncsTD+l0eUA9gTpXY1wfApPChKYFPjDc/lyhLsIbLscS0vDtzPyvXcosEO5/rNOI2eMWSv72bQDqigjljjmDrjbBYCQLlmhd2s5OfmVEDAOcvJ/sbBcBEdWNcUJaq4mlmHRw5XGjLg4zohWkjstHJc7ozjF7ZZj2DUVxO0qimEiMXjpq2okFvnmVzvoLx8IANbwaCu4ok5G59n8M2gLouCJywiyT6u3xf06S7BS36Lpn5PPwgLVn1oOwQiRsims6KrwNF0JFWCJa+gEM712X3/ePxV0h1ncFzzU5NzMVwkwdr7Ec=
Message-ID: <efc0eee4-9903-4228-b33e-cb50d4c545ee@suse.com>
Date: Fri, 9 Feb 2024 12:46:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2] docs/misra: add asm-offset.c to exclude-list
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Nicola Vetrini <nicola.vetrini@bugseng.com>
References: <09b659e19bf2cc6b3ee4320e019bdfa7def5f3b8.1707406598.git.nicola.vetrini@bugseng.com>
 <891e8cc2-e073-4cd3-9998-4c01ae775a3b@suse.com>
 <8977390f-9549-40d5-bfdc-5c3da81521fc@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <8977390f-9549-40d5-bfdc-5c3da81521fc@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 09.02.2024 10:40, Julien Grall wrote:
> Replying on the v2 as well.

And answering here despite the respective question was raised on the
v1 thread: I'm certainly okay with the more detailed commit message.
A few nits, though:

> On 08/02/2024 15:56, Jan Beulich wrote:
>> On 08.02.2024 16:50, Nicola Vetrini wrote:
>>> These files contain several deliberate violations of MISRA C rules and
>>> they are not linked in the final Xen binary, therefore they can be exempted
>>> from MISRA compliance.
> 
> I'd like the commit message to be expanded a little bit to explain which 
> MISRA rules are a problem. This helped me to understand why we excluded 
> rather than fixed.
> 
> Base on the previous discussion, I would suggest:
> 
> These files contain several deliberate violation of MISRA C rules such as:

violations

>    * R20.12 on Arm for macros DEFINE and OFFSET, where the second 
> argument of OFFSET is a macro and is used as a normal parameter and a 
> stringification operand.

Is this really for Arm only?

>    * R2.1 because the file is not linked That said it was decided to 
> deviate the rule itselfed to deviate that aspect).

There look to be punctuation issues here. Also s/itselfed/itself/, and
the duplicate "deviate" is also a little odd to read (maybe "deal with"
or "address" in place of the 2nd instance).

> The files are also not linked in the final Xen binary, therefore they 
> can be expempted from MISRA compliance.

Looks to duplicate what the latter half of the 2nd bullet point has.
If to be kept: s/expempted/exempted/.

>>> --- a/docs/misra/exclude-list.json
>>> +++ b/docs/misra/exclude-list.json
>>> @@ -101,6 +101,10 @@
>>>               "rel_path": "arch/x86/efi/check.c",
>>>               "comment": "The resulting code is not included in the final Xen binary, ignore for now"
>>>           },
>>> +        {
>>> +          "rel_path": "arch/*/*/asm-offsets.c",
>>> +          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
>>> +        },
>>>           {
>>>               "rel_path": "common/coverage/*",
>>>               "comment": "Files to support gcov, ignore for now"
>>
>> ... something looks odd with indentation; can probably be adjusted
>> while committing.
> 
> I am happy to take care of both the commit message and the indentation 
> on commit.

Okay, I'll leave that to you then.

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 11:51:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 11:51:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678658.1056120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPQ1-0004bn-Lh; Fri, 09 Feb 2024 11:51:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678658.1056120; Fri, 09 Feb 2024 11:51:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPQ1-0004bg-Iy; Fri, 09 Feb 2024 11:51:45 +0000
Received: by outflank-mailman (input) for mailman id 678658;
 Fri, 09 Feb 2024 11:51:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4TWb=JS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rYPQ0-0004ba-UD
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 11:51:44 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 984462ac-c741-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 12:51:43 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-33b29b5ea96so443108f8f.0
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 03:51:43 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 r15-20020a5d4e4f000000b0033b17880eacsm1611218wrt.56.2024.02.09.03.51.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 09 Feb 2024 03:51:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 984462ac-c741-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707479503; x=1708084303; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=eX0TodQJ+EiGGJzfw7RdCoXaDa3ajnzOtvqsNFlyrgY=;
        b=BO0PotZBGvVj6MXDfHbZqdZ2FNnj6NWIehisKvevB05cFyB/dAlPGJmYNJDTcWNVJQ
         iJ8x17ugee04/WGIh3efkWhaHbHwozECuJ1DJcp5Hikq8WtvSUaAnqfJJ4iYwZXVQbJs
         rQv/A01JTKhfHSTmWRw153+vdBl0dDG9rVjZo3qn6zn5V1tWVhZv5wwtRugRVDMIFCC/
         LYEjMHNUCcGYuyJ+k/61xyikRq+6tUE+6EzzKG9lYAgjmd6QBKlm3OvVNMGsndQQEw7t
         KhlmjHYjRG3ZgRRoMxxYcOQ6P9K3yzwoRhL/RH6JU+MVLKBrYiFB7i+1mQLALJEAiWSD
         yEhQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707479503; x=1708084303;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=eX0TodQJ+EiGGJzfw7RdCoXaDa3ajnzOtvqsNFlyrgY=;
        b=B72z2wW6/xqyJQhn18+a14Tk1sqyl5CWtxks/U22ocvbl30x8ZIL0QD85/ghsWs1dX
         5CxpL+GdNhACwjsRzxdUU73nJ62AFqwUpMNS8OXb5VdRyhiu4z3Tj6gnmsyRDt8ivJRs
         fo1fy0WbcKfwAbqS89mjij7f5wZi4eR8tcOQJU4fQsF60jbXKm24GYZHCFi5c8LMcrhs
         JG4JtLEZsOnOPdMlF4licmBq4oyXNK+nCMEM89kmHi9leZWBBzUNtn71xh4Va0CHliOf
         LD0NLVb1n0pm0U/n+mytZ49r/xuuewrd8gJwWubovXDAG+Fen188KS8KTQTozUe0aoMO
         gxPw==
X-Gm-Message-State: AOJu0Yz6Hr5r6UTAXhqs1N9iXqjh6RmMQLeOnjIA7QJqDWfNlS8zKq9I
	Z8lOeCNSfRSIArpdSHsoPe4ujulkniZ6pfmsoA3v9pVHNnhyHXsSMe6/cjhQsg==
X-Google-Smtp-Source: AGHT+IGL2wtyj8JoCK4/ne/iuallM0OrjB8j6qE4CPlaM7cJSsRVwv/ka4bYV9bfUzw/f//SV5VwtA==
X-Received: by 2002:adf:a3c2:0:b0:33b:65d3:cd91 with SMTP id m2-20020adfa3c2000000b0033b65d3cd91mr795146wrb.3.1707479503259;
        Fri, 09 Feb 2024 03:51:43 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWZakR/jEUPbKEmlFMnd1jemDbED3meq/I3zwxkAfk1Yf026SnWEFA6ZRBpetI4i3pDQs990GKmy357tSbDgwaajsZf/wWCpMmef0fedhN0LVAYOXlLCWn4nJLqHHlGGPAoJIHEzwN7hIoHg0AEIFCJfZI/2VBrGZUuVBvfC8E9L2NMvltBxvjAeqDt
Message-ID: <fe22b91e-227c-45df-a39a-dcccd8726cd3@suse.com>
Date: Fri, 9 Feb 2024 12:51:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/vmx: add support for virtualize SPEC_CTRL
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240206142507.81985-1-roger.pau@citrix.com>
 <91e3fd09-8325-49b0-9d7b-43aacf2acd81@suse.com> <ZcYCXrEaOyxZUb2I@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcYCXrEaOyxZUb2I@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 09.02.2024 11:45, Roger Pau Monné wrote:
> On Thu, Feb 08, 2024 at 02:40:53PM +0100, Jan Beulich wrote:
>> On 06.02.2024 15:25, Roger Pau Monne wrote:
>>> @@ -2086,6 +2091,9 @@ void vmcs_dump_vcpu(struct vcpu *v)
>>>      if ( v->arch.hvm.vmx.secondary_exec_control &
>>>           SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY )
>>>          printk("InterruptStatus = %04x\n", vmr16(GUEST_INTR_STATUS));
>>> +    if ( cpu_has_vmx_virt_spec_ctrl )
>>> +        printk("SPEC_CTRL mask = %#016lx  shadow = %#016lx\n",
>>> +               vmr(SPEC_CTRL_MASK), vmr(SPEC_CTRL_SHADOW));
>>
>> #0... doesn't make a lot of sense; only e.g. %#lx does. Seeing context
>> there's no 0x prefix there anyway. Having looked at the function the
>> other day, I know though that there's a fair mix of 0x-prefixed and
>> unprefixed hex numbers that are output.
> 
> For consistency with how other MSRs are printed I should use the '0x'
> prefix.

MSRs? It's VMCS fields which are printed here.

>> Personally I'd prefer if all
>> 0x prefixes were omitted here. If you and Andrew think otherwise, I can
>> live with that, so long as we're at least striving towards consistent
>> output (I may be able to get to doing a conversion patch, once I know
>> which way the conversion should be).
> 
> I usually prefer the '0x' to avoid ambiguity.  However this being all
> hardware registers, I might be fine with dropping the '0x' on the
> grounds that all registers are always printed as hex.
> 
>>> --- a/xen/arch/x86/hvm/vmx/vmx.c
>>> +++ b/xen/arch/x86/hvm/vmx/vmx.c
>>> @@ -823,18 +823,28 @@ static void cf_check vmx_cpuid_policy_changed(struct vcpu *v)
>>>      {
>>>          vmx_clear_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
>>>  
>>> -        rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
>>> -        if ( rc )
>>> -            goto out;
>>> +        if ( !cpu_has_vmx_virt_spec_ctrl )
>>> +        {
>>> +            rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
>>> +            if ( rc )
>>> +                goto out;
>>> +        }
>>
>> I'm certainly okay with you doing it this way, but generally I'd prefer
>> if code churn was limited whjere possible. Here leveraging that rc is 0
>> on entry, a smaller change would be to
>>
>>         if ( !cpu_has_vmx_virt_spec_ctrl )
>>             rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
>>         if ( rc )
>>             goto out;
>>
>> (similarly below then).
> 
> That looks odd to me, and is not how I would write that code.  I can
> however adjust if you insist.  Given it's just a two line difference I
> think it was worth having the more usual form.

As said, I'm okay with the change as is.

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 11:54:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 11:54:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678661.1056130 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPSp-0005AM-3v; Fri, 09 Feb 2024 11:54:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678661.1056130; Fri, 09 Feb 2024 11:54:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPSp-0005AF-03; Fri, 09 Feb 2024 11:54:39 +0000
Received: by outflank-mailman (input) for mailman id 678661;
 Fri, 09 Feb 2024 11:54:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4TWb=JS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rYPSn-0005A7-L1
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 11:54:37 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id feb7d93f-c741-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 12:54:35 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-51168addef1so1585494e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 03:54:35 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bi16-20020a05600c3d9000b004103400259fsm411469wmb.29.2024.02.09.03.54.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 09 Feb 2024 03:54:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: feb7d93f-c741-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707479675; x=1708084475; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=fPLiNtHGgwBjYK/1AImqtz9UhjZ/Xjnm0e5OdT1IOEI=;
        b=S8e0ioipGyf8en9Z/nLUKg34kWOqL+0ud47ndYBdRSLaFy4KSrgk8a1qQLpBJxJnxR
         GB69/11FINTNU8rbm4kXn+fbtRJJHDhoxyOaXCCZ04S5MD5tzlzQViwQ53CefjdeDEyJ
         R2V2i0+Jhw29ly5cFu0uvE9oJJ25IONVC3Pr4nNSVln3slYE9pcBCzcmYk6rbcLB+Cdg
         6NFjvvWppmkbSgYPqsAhPEELOXffvg0b8SWLUOmEUwb2j5HRZjcZu4YCbIWJnVWjk82i
         wpW6+jbMb9iKz+9pDSpBcWOpIN3ZzH6WTqdmOpsN7i6JXa2fyiZOE5/8pQW4U/vYr6IL
         zvXA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707479675; x=1708084475;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=fPLiNtHGgwBjYK/1AImqtz9UhjZ/Xjnm0e5OdT1IOEI=;
        b=Objobosae7y94CrCpppkESx+64zcdBltiiZCbjMpaHqDiixxudVcML0q6Z2JAM46XQ
         /VT3m/xsyD0te0SqWMLADu4UWV+WuxENtSeGkiYMZeJfqja4P65k5H45oOSBDjpF+qY8
         h8stRYYKt6eoK5TgGdQteFcRZfyRFvc8q/Uxc05kukQhl1asl62Xb08dKsCFmHIBCtsP
         0zIvZIhg7LEy6UTsPtSxDXEICi+G8ifgnQxFGf5LTyOw2j6TSEtKabxz3Bpo2TFDS1us
         Ve4rLYF5o9wM0OS+IMAv4lBET+Ckd7Rw4Uvi+INds1zsCm13Pm4IPMW3aP5CcuPYv5U8
         UoWw==
X-Forwarded-Encrypted: i=1; AJvYcCXY3UJhZxT0e8EsvjaGByh7c2HIPaL86OnZNP/vhRO4hwJssvIaQ/t/76bpmBgmM3Ytx0y1y2RawDi5S+wyCNgMHrmvgHz6UJ+xZfUZTQ0=
X-Gm-Message-State: AOJu0YxUUQwTFreJlOArCWpyG5vl2NUFswtt6UPEtFxFrGKuLSw4SbOV
	XYGxFxT+UPHT0OxA5+ZDSX4Jg6B27OKt2ABCllQa85rWfMCgVN5ueVep4OVOZB9gxA0O1OO7q98
	=
X-Google-Smtp-Source: AGHT+IH2uz1ket5tUekJsMpCNATRPtPHaKONfnvdHaiGDhnr0c3hL+mFe8M7H8nlkdhiSsrOjsLrhQ==
X-Received: by 2002:ac2:52b1:0:b0:511:6512:9d3e with SMTP id r17-20020ac252b1000000b0051165129d3emr943125lfm.33.1707479674988;
        Fri, 09 Feb 2024 03:54:34 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWV9zs4+9ZcLYVw/LFnDDpAh3Cb5rgJEkJxKxJv1FXgZ38ve7QAQAIOIi+j0UVrU1EhxMpLUXW0Yq89SegSrIrqFY9+STLeK0wbDns6OFdTQPgldl4j0wfyjgzLcN0EBwuk3quN6LBgGQGBuM7MRrjcmDIBZJiCr0Wg7JAiIfRXgbInzIUxhlW3M5H5B8dDH3NakPGCxqrgiusl9INLyMB9xjQZKI/U/TKkL47DPeCjMlCwi9PTdJR/vuEaxAsqapxpbup2+ho7Nuf22pGR743qFP2I+p24fl3BY4vj0h9KoQ1bgm2UpoLSwC+ZHbw8/07lXLZTVWSB28SG1ycGfqJNHNf0p2SUHmDf5MhKac85M/uQ6nTtBbIpuRr31zvhkFhuzHEKh7ktoCqDKogPj7dLTjYaK4BdQ3BrJfyzPhx4O3dF/NxmcPNhs/DgMk5ztCM=
Message-ID: <c341164a-0bd9-41f1-a984-718ccb986053@suse.com>
Date: Fri, 9 Feb 2024 12:54:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and
 logging effects
Content-Language: en-US
To: Simone Ballarin <simone.ballarin@bugseng.com>, sstabellini@kernel.org
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com>
 <09869431-ed00-4102-93fd-84c697a5c950@suse.com>
 <851eb6ec-558b-4a89-a31a-34046730bb55@bugseng.com>
 <41474fe7-c65f-4244-a455-0aaa0e4315e3@suse.com>
 <e48f4d49-274c-4bd6-8e3b-1ecaee6fa602@bugseng.com>
 <c6199a60-ecd2-4f8a-b742-199b90e62bac@suse.com>
 <a3586ea2-5ad7-4c64-8e05-00ea0f83518f@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a3586ea2-5ad7-4c64-8e05-00ea0f83518f@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 09.02.2024 10:25, Simone Ballarin wrote:
> On 07/02/24 13:40, Jan Beulich wrote:
>> On 07.02.2024 13:21, Simone Ballarin wrote:
>>> On 07/02/24 11:24, Jan Beulich wrote:
>>>> On 07.02.2024 11:03, Simone Ballarin wrote:
>>>>> On 06/02/24 13:04, Jan Beulich wrote:
>>>>>> On 02.02.2024 16:16, Simone Ballarin wrote:
>>>>>>> --- a/xen/arch/arm/guestcopy.c
>>>>>>> +++ b/xen/arch/arm/guestcopy.c
>>>>>>> @@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf, uint64_t addr, unsigned int len,
>>>>>>>     unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
>>>>>>>     {
>>>>>>>         return copy_guest((void *)from, (vaddr_t)to, len,
>>>>>>> -                      GVA_INFO(current), COPY_to_guest | COPY_linear);
>>>>>>> +                      /* SAF-4-safe No persistent side effects */
>>>>>>> +                      GVA_INFO(current),
>>>>>>
>>>>>> I _still_ think this leaves ambiguity. The more that you need to look
>>>>>> up GVA_INFO() to recognize what this is about.
>>>>>
>>>>>
>>>>> Just to recap: here the point is that current reads a register with a volatile asm, so the
>>>>> violation is in the expansion of GVA_INFO(current). Both GVA_INFO and current taken alone
>>>>> are completely fine, so this is the only place where a SAF comment can be placed.
>>>>>
>>>>> The exapansion is:
>>>>> ((copy_info_t) { .gva = { ((*({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"(&
>>>>>      per_cpu__curr_vcpu)); (typeof(&per_cpu__curr_vcpu)) (__ptr + (({ uint64_t _r; asm volatile("mrs  %0, ""TPIDR_EL2" : "=r"
>>>>>      (_r)); _r; }))); }))) } }), (1U << 1) | (1U << 2));
>>>>>
>>>>> My proposals are:
>>>>> 1) address the violation moving the current expansion outside (extra variable);
>>>>> 2) put a more detailed comment to avoid the ambiguity;
>>>>> 3) use an ECL deviation for GVA_INFO(current).
>>>>>
>>>>> Do you have any preference or proposal?
>>>>
>>>> Imo 3 is not an option at all. Probably 1 wouldn't be too bad here, but
>>>> I still wouldn't like it (as matching a general pattern I try to avoid:
>>>> introducing local variables that are used just once and don't meaningfully
>>>> improve e.g. readability). Therefore out of what you list, 2 would remain.
>>>> But I'm not happy with a comment here either - as per above, there's
>>>> nothing that can go wrong here as long as there's only a single construct
>>>> with side effect(s).
>>>>
>>> So, would be changing the SAF in:
>>> /* SAF-<new_id>-safe single item initializer */
>>>
>>> OK for you?
>>
>> A comment, as said, is only the least bad of what you did enumerate. But
>> for this code in particular I'm not a maintainer anyway, so it's not me
>> you need to convince. I'm taking this only as an example for discussing
>> underlying aspects.
> 
> I was generally thinking about the comments of this series, and I've
> just realised that many of them can be summarized by the following sentence:
> 
> "We do not want changes to address violations of R13.1 that are not also violations
> of R13.2"
> 
> MC3R1.R13.2	rule	The value of an expression and its persistent side effects shall be the same under all permitted evaluation orders
> MC3R1.R13.1	rule	Initializer lists shall not contain persistent side effects
> 
> At this point, my proposal is to remove R13.1 from the coding standard and add
> R13.2 (eventually limiting its scope to initializer lists).

I'm afraid I don't understand the "eventually limiting" part.

> Maybe it is better to re-discuss the rule adoption during the next meeting?

Perhaps best. Stefano, could you take note of this?

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 11:57:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 11:57:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678672.1056144 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPVm-0005tj-Km; Fri, 09 Feb 2024 11:57:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678672.1056144; Fri, 09 Feb 2024 11:57:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPVm-0005tc-I1; Fri, 09 Feb 2024 11:57:42 +0000
Received: by outflank-mailman (input) for mailman id 678672;
 Fri, 09 Feb 2024 11:57:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYPVl-0005t2-8R; Fri, 09 Feb 2024 11:57:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYPVl-00025h-67; Fri, 09 Feb 2024 11:57:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYPVk-0000jO-Ol; Fri, 09 Feb 2024 11:57:40 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYPVk-0002KF-ON; Fri, 09 Feb 2024 11:57:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=JM0DPCzmgjGann9AL18Cf41BZ8w3BgjbQO2S2PHh2YY=; b=XeSP4GKCTheUZ/z/2iJIoAOBnQ
	sFH2XDTHCUIChJpdy/3lCoa4E/Mo1mPTLUpRCCw1TzBSy1CCFeWEFstULX7dXsePOXi+0Go1gxjLJ
	RHzAtQxzJUT8f20nX++a+kTSB4onfD7TF1xx8sC3PWXs7q8z+OxMQUu/m3iU2GRxvtMQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184630-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184630: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=e11f5766503c0ff074b4e0f888bbfc931518a169
X-Osstest-Versions-That:
    xen=f4519ee87ebfb74fe0451130d82c99453b75e2e1
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 09 Feb 2024 11:57:40 +0000

flight 184630 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184630/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184626
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184626
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184626
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184626
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184626
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184626
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail like 184626
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184626
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184626
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184626
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184626
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184626
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install       fail like 184626
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  e11f5766503c0ff074b4e0f888bbfc931518a169
baseline version:
 xen                  f4519ee87ebfb74fe0451130d82c99453b75e2e1

Last test of basis   184626  2024-02-08 11:36:07 Z    1 days
Testing same since   184630  2024-02-09 00:40:40 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Julien Grall <jgrall@amazon.com>
  Luca Fancellu <luca.fancellu@arm.com>
  Michal Orzel <michal.orzel@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f4519ee87e..e11f576650  e11f5766503c0ff074b4e0f888bbfc931518a169 -> master


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 12:18:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 12:18:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678687.1056154 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPpP-0001CS-JD; Fri, 09 Feb 2024 12:17:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678687.1056154; Fri, 09 Feb 2024 12:17:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYPpP-0001CL-Fx; Fri, 09 Feb 2024 12:17:59 +0000
Received: by outflank-mailman (input) for mailman id 678687;
 Fri, 09 Feb 2024 12:17:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rYPpN-0001CF-Kd
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 12:17:57 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYPpI-0002Rm-L1; Fri, 09 Feb 2024 12:17:52 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYPpI-0008K4-9g; Fri, 09 Feb 2024 12:17:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=mPlY5oDjvzGiefQwBflm3VOMFbQHhIMdeYij+5vSI0w=; b=QxIRbKVNWQJbqvvdUeukf/om9s
	o+29VZzcGoTb9ZOU0097Nx9+p4/G8es1hRUGur+dmWgD3SZjT4kvgTWGlHGxgAYYVy4hv+zhRqq07
	LWRCqGXEVtQB8sRqxHe7PDBLX9YW1L8U/qDbAykfaov4sbmGy1VM5t9BRzUC6HTzpYQ8=;
Message-ID: <5cdae9ce-d2ff-40bd-bc07-aa1965275833@xen.org>
Date: Fri, 9 Feb 2024 12:17:50 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2] docs/misra: add asm-offset.c to exclude-list
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Nicola Vetrini <nicola.vetrini@bugseng.com>
References: <09b659e19bf2cc6b3ee4320e019bdfa7def5f3b8.1707406598.git.nicola.vetrini@bugseng.com>
 <891e8cc2-e073-4cd3-9998-4c01ae775a3b@suse.com>
 <8977390f-9549-40d5-bfdc-5c3da81521fc@xen.org>
 <efc0eee4-9903-4228-b33e-cb50d4c545ee@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <efc0eee4-9903-4228-b33e-cb50d4c545ee@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 09/02/2024 11:46, Jan Beulich wrote:
> On 09.02.2024 10:40, Julien Grall wrote:
>> Replying on the v2 as well.
> 
> And answering here despite the respective question was raised on the
> v1 thread: I'm certainly okay with the more detailed commit message.

Ah yes. Sorry, I replied to v1 first and then realized it may have been 
easier to comment on v2.

> A few nits, though:
> 
>> On 08/02/2024 15:56, Jan Beulich wrote:
>>> On 08.02.2024 16:50, Nicola Vetrini wrote:
>>>> These files contain several deliberate violations of MISRA C rules and
>>>> they are not linked in the final Xen binary, therefore they can be exempted
>>>> from MISRA compliance.
>>
>> I'd like the commit message to be expanded a little bit to explain which
>> MISRA rules are a problem. This helped me to understand why we excluded
>> rather than fixed.
>>
>> Base on the previous discussion, I would suggest:
>>
>> These files contain several deliberate violation of MISRA C rules such as:
> 
> violations
> 
>>     * R20.12 on Arm for macros DEFINE and OFFSET, where the second
>> argument of OFFSET is a macro and is used as a normal parameter and a
>> stringification operand.
> 
> Is this really for Arm only?

I don't exactly know. I took Nicola's comment and massage it for the 
commit message. I am assuming that this was also not exhaustive list, so 
my aim was to only provide some example.

Thinking of it, I don't see why it would only be a problem on Arm. I can 
drop the "on Arm".

> 
>>     * R2.1 because the file is not linked That said it was decided to
>> deviate the rule itselfed to deviate that aspect).
> 
> There look to be punctuation issues here. Also s/itselfed/itself/, and
> the duplicate "deviate" is also a little odd to read (maybe "deal with"
> or "address" in place of the 2nd instance).

Doh, indeed. This wants to be:

"R2.1 because the file is not linked. That said, it was decided to 
deviate the ruule itself to address that aspect."
>> The files are also not linked in the final Xen binary, therefore they
>> can be expempted from MISRA compliance.
> 
> Looks to duplicate what the latter half of the 2nd bullet point has.
> If to be kept: s/expempted/exempted/.

I will remove.

> 
>>>> --- a/docs/misra/exclude-list.json
>>>> +++ b/docs/misra/exclude-list.json
>>>> @@ -101,6 +101,10 @@
>>>>                "rel_path": "arch/x86/efi/check.c",
>>>>                "comment": "The resulting code is not included in the final Xen binary, ignore for now"
>>>>            },
>>>> +        {
>>>> +          "rel_path": "arch/*/*/asm-offsets.c",
>>>> +          "comment": "The resulting code is not included in the final Xen binary, ignore for now"
>>>> +        },
>>>>            {
>>>>                "rel_path": "common/coverage/*",
>>>>                "comment": "Files to support gcov, ignore for now"
>>>
>>> ... something looks odd with indentation; can probably be adjusted
>>> while committing.
>>
>> I am happy to take care of both the commit message and the indentation
>> on commit.
> 
> Okay, I'll leave that to you then.

Thanks. I will do it shortly.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 12:45:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 12:45:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678692.1056163 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYQFd-0005pN-Ko; Fri, 09 Feb 2024 12:45:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678692.1056163; Fri, 09 Feb 2024 12:45:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYQFd-0005pG-I7; Fri, 09 Feb 2024 12:45:05 +0000
Received: by outflank-mailman (input) for mailman id 678692;
 Fri, 09 Feb 2024 12:45:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=z5rg=JS=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rYQFc-0005p9-E4
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 12:45:04 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0afb2d32-c749-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 13:45:03 +0100 (CET)
Received: from beta.station (net-188-218-67-100.cust.vodafonedsl.it
 [188.218.67.100])
 by support.bugseng.com (Postfix) with ESMTPSA id 0E17B4EE0739;
 Fri,  9 Feb 2024 13:45:01 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0afb2d32-c749-11ee-8a4b-1f161083a0e0
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH] eclair: remove wrongly added -eval_file
Date: Fri,  9 Feb 2024 13:44:12 +0100
Message-Id: <976ea35a8682adb2991d249a48bb2b41e834e4e6.1707482589.git.simone.ballarin@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

properties.ecl does not exist and it is not
even generated by propertyparser.py. Loading
this file causes empty ECLAIR analysis.

Fixes: f4519ee8 ("eclair: move function and macro properties outside ECLAIR")
Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
---
 automation/eclair_analysis/ECLAIR/analysis.ecl | 1 -
 1 file changed, 1 deletion(-)

diff --git a/automation/eclair_analysis/ECLAIR/analysis.ecl b/automation/eclair_analysis/ECLAIR/analysis.ecl
index 684c5b0b39..a604582da3 100644
--- a/automation/eclair_analysis/ECLAIR/analysis.ecl
+++ b/automation/eclair_analysis/ECLAIR/analysis.ecl
@@ -30,7 +30,6 @@ if(not(scheduled_analysis),
 -eval_file=deviations.ecl
 -eval_file=call_properties.ecl
 -eval_file=tagging.ecl
--eval_file=properties.ecl
 -eval_file=concat(set,".ecl")
 
 -doc="Hide reports in external code."
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 13:13:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 13:13:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678697.1056174 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYQgn-00024D-P5; Fri, 09 Feb 2024 13:13:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678697.1056174; Fri, 09 Feb 2024 13:13:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYQgn-000246-MG; Fri, 09 Feb 2024 13:13:09 +0000
Received: by outflank-mailman (input) for mailman id 678697;
 Fri, 09 Feb 2024 13:13:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aSO/=JS=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rYQgm-000240-Uy
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 13:13:08 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f6c8658c-c74c-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 14:13:06 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a3810e975aaso109417266b.3
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 05:13:06 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 ps2-20020a170906bf4200b00a39e0196eecsm754537ejb.26.2024.02.09.05.13.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 05:13:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6c8658c-c74c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707484386; x=1708089186; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=7WGxuJXfyeGxLs1xzSfe3Sg7BNZubTJxTioPX7QPVyw=;
        b=JE4SBvyXMIJB3kiJ2eDLb9raXGIgE65QDjIJiE7C/+8RNoSbfQ1cDy1sz7ZuHCiRnd
         +flTQDUSvFW6+eG/3CyfdmgjPTgTOXsDcEH7NPrgGLCa7u4gZQ+RmsLeTwQJ7jmUaagC
         Z6y5rTJTL3T89sifVMKUsxeS4CPErm/m9Ki/Q=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707484386; x=1708089186;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=7WGxuJXfyeGxLs1xzSfe3Sg7BNZubTJxTioPX7QPVyw=;
        b=Kwtp7Z5P9ml+LUG7VLn6WkNW/oQPZeuVSt5y8wGHVHz5zmIsArLf/dWln5PmFqsRq1
         IMUPSYubKto94D3Z6apwvMmOuMEjHqHNqzD9ia8P10T1XJQ1+biAyccoPPrEPHe4GYKs
         +gRFPQTBpgOsNe7URYKTdSgJyR4SxKv44eH4nJ7NJnvwiRh3b9rMZksK51OVXj43HjpK
         v5nlEcP3dMgf8ifGOU4px3BKMmVtDfkHGWF/vtvtlJm3M/WH+sfzcURBPSPzmsYP6Se7
         RddBEDDBuZshufAzUSP1tl8aHKHaJjJRchID84T2V4x1ZBkD79xea3sVoSZxN0LtSGzM
         0VYg==
X-Forwarded-Encrypted: i=1; AJvYcCWhSVvOghYB6bcZ7xkkBJzGxV4yMEdu7J9mTEwmamXZPEGJ5Ic78swQy3KInguoEnQ19L3Dw+e9gbvP05v1YIMthnXIFMkF6NQAHRdZu3s=
X-Gm-Message-State: AOJu0Yx/4wZxHyZtAdkAlCMlYfxx1S4MUdBQ3eibZ/oM7UvgtJ/iOyQE
	Iy2KiRAAKpc/csFyjw2gu1trOPWK1B8FvDGX4hD84C0CGPbl6m0PeuSAls4Zcz0=
X-Google-Smtp-Source: AGHT+IHpFTghhTKnw9mRC2bUTQmLlBHyEe3xfXY86lBBR9rWXOa9Ilp4nACA4nc5c5CZuU3mdWPFTw==
X-Received: by 2002:a17:906:2793:b0:a3b:dd99:c2c3 with SMTP id j19-20020a170906279300b00a3bdd99c2c3mr1315934ejc.73.1707484386297;
        Fri, 09 Feb 2024 05:13:06 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWO8pSvxSYIXApXZgEibC4MdKaXSKjQUZCoix+sGF+SPci3Xc3LtkYD18GwDKfL/mXYPdOD3zZSqAQTZry8E/r5oPpTgeUZ7sQXxgWBLhS4qDvI3aUJQVUloYQ4RGI4trEi2ao8Y3kIyDydO0UYTlofgo5SW2JKFhBWGzTiYwRFjZ/KKGT+qMEqUaxQ
Date: Fri, 9 Feb 2024 14:13:05 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] x86/vmx: add support for virtualize SPEC_CTRL
Message-ID: <ZcYk4bHo0Cytndwf@macbook>
References: <20240206142507.81985-1-roger.pau@citrix.com>
 <91e3fd09-8325-49b0-9d7b-43aacf2acd81@suse.com>
 <ZcYCXrEaOyxZUb2I@macbook>
 <fe22b91e-227c-45df-a39a-dcccd8726cd3@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <fe22b91e-227c-45df-a39a-dcccd8726cd3@suse.com>

On Fri, Feb 09, 2024 at 12:51:41PM +0100, Jan Beulich wrote:
> On 09.02.2024 11:45, Roger Pau Monné wrote:
> > On Thu, Feb 08, 2024 at 02:40:53PM +0100, Jan Beulich wrote:
> >> On 06.02.2024 15:25, Roger Pau Monne wrote:
> >>> @@ -2086,6 +2091,9 @@ void vmcs_dump_vcpu(struct vcpu *v)
> >>>      if ( v->arch.hvm.vmx.secondary_exec_control &
> >>>           SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY )
> >>>          printk("InterruptStatus = %04x\n", vmr16(GUEST_INTR_STATUS));
> >>> +    if ( cpu_has_vmx_virt_spec_ctrl )
> >>> +        printk("SPEC_CTRL mask = %#016lx  shadow = %#016lx\n",
> >>> +               vmr(SPEC_CTRL_MASK), vmr(SPEC_CTRL_SHADOW));
> >>
> >> #0... doesn't make a lot of sense; only e.g. %#lx does. Seeing context
> >> there's no 0x prefix there anyway. Having looked at the function the
> >> other day, I know though that there's a fair mix of 0x-prefixed and
> >> unprefixed hex numbers that are output.
> > 
> > For consistency with how other MSRs are printed I should use the '0x'
> > prefix.
> 
> MSRs? It's VMCS fields which are printed here.

Well, yes, but it represents a MSR value.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 13:14:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 13:14:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678701.1056184 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYQiW-0002as-2r; Fri, 09 Feb 2024 13:14:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678701.1056184; Fri, 09 Feb 2024 13:14:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYQiW-0002al-0B; Fri, 09 Feb 2024 13:14:56 +0000
Received: by outflank-mailman (input) for mailman id 678701;
 Fri, 09 Feb 2024 13:14:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fiXy=JS=actia.se=john.ernberg@srs-se1.protection.inumbo.net>)
 id 1rYQiU-0002af-SH
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 13:14:54 +0000
Received: from mail.actia.se (mail.actia.se [212.181.117.226])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3609f27b-c74d-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 14:14:52 +0100 (CET)
Received: from S036ANL.actianordic.se (10.12.31.117) by S035ANL.actianordic.se
 (10.12.31.116) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 9 Feb
 2024 14:14:51 +0100
Received: from S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69]) by
 S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69%4]) with mapi id
 15.01.2507.035; Fri, 9 Feb 2024 14:14:51 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3609f27b-c74d-11ee-98f5-efadbce2ee36
From: John Ernberg <john.ernberg@actia.se>
To: Julien Grall <julien@xen.org>, Peng Fan <peng.fan@nxp.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
CC: Jonas Blixt <jonas.blixt@actia.se>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Topic: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Index: AQHaVDuqVQ/kgxLZgUK/m8/hE0ez5bDzxxsAgAA1KwCAAVx5AIAAQjeAgAEdtQCAAyp5AIABm3uAgAZ8E4A=
Date: Fri, 9 Feb 2024 13:14:51 +0000
Message-ID: <08ae764c-3490-4dd1-ab70-7a855a98d16d@actia.se>
References: <20240131114952.305805-1-john.ernberg@actia.se>
 <20240131114952.305805-2-john.ernberg@actia.se>
 <a2f726f5-df4c-4d15-90af-7d59c0f1f513@xen.org>
 <494d4961-ad8a-4d1d-aaa6-d1bfb9d6a137@actia.se>
 <167f0c7a-e037-446c-82f8-2584e35a7af1@xen.org>
 <a265ea2d-9b5f-4726-9395-448b1b669839@actia.se>
 <012b5f83-2f9b-4477-965e-07b8506c0052@xen.org>
 <DU0PR04MB94170CB1F77E3D594A6D0E9488402@DU0PR04MB9417.eurprd04.prod.outlook.com>
 <5a0c8f09-4f01-45e9-892c-86342c0d0ec5@xen.org>
In-Reply-To: <5a0c8f09-4f01-45e9-892c-86342c0d0ec5@xen.org>
Accept-Language: en-US, sv-SE
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.12.12.30]
x-esetresult: clean, is OK
x-esetid: 37303A2921D72955617360
Content-Type: text/plain; charset="utf-8"
Content-ID: <C41CBFE19B385A469101D13966200049@actia.se>
Content-Transfer-Encoding: base64
MIME-Version: 1.0

SGkgSnVsaWVuLA0KDQpBcG9sb2dpZXMgZm9yIHRoZSBkZWxheSwgSSB3YXMgcHVsbGVkIGF3YXkg
Zm9yIGEgYml0Lg0KDQpPbiAyLzUvMjQgMTE6MTMsIEp1bGllbiBHcmFsbCB3cm90ZToNCj4gSGks
DQo+IA0KPiBPbiAwNC8wMi8yMDI0IDA5OjQwLCBQZW5nIEZhbiB3cm90ZToNCj4+DQo+Pg0KPj4+
IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+Pj4gRnJvbTogSnVsaWVuIEdyYWxsIDxqdWxp
ZW5AeGVuLm9yZz4NCj4+PiBTZW50OiAyMDI05bm0MuaciDLml6UgMTc6MjANCj4+PiBUbzogSm9o
biBFcm5iZXJnIDxqb2huLmVybmJlcmdAYWN0aWEuc2U+OyBTdGVmYW5vIFN0YWJlbGxpbmkNCj4+
PiA8c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IEJlcnRyYW5kIE1hcnF1aXMgPGJlcnRyYW5kLm1h
cnF1aXNAYXJtLmNvbT47DQo+Pj4gTWljaGFsIE9yemVsIDxtaWNoYWwub3J6ZWxAYW1kLmNvbT47
IFZvbG9keW15ciBCYWJjaHVrDQo+Pj4gPFZvbG9keW15cl9CYWJjaHVrQGVwYW0uY29tPjsgUGVu
ZyBGYW4gPHBlbmcuZmFuQG54cC5jb20+DQo+Pj4gQ2M6IEpvbmFzIEJsaXh0IDxqb25hcy5ibGl4
dEBhY3RpYS5zZT47IHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZw0KPj4+IFN1YmplY3Q6
IFJlOiBbUEFUQ0ggMS8yXSB4ZW4vYXJtOiBBZGQgaW14OHF7bSx4fSBwbGF0Zm9ybSBnbHVlDQo+
Pj4NCj4+PiBPbiAwMS8wMi8yMDI0IDE2OjE3LCBKb2huIEVybmJlcmcgd3JvdGU6DQo+Pj4+IE9u
IDIvMS8yNCAxMzoyMCwgSnVsaWVuIEdyYWxsIHdyb3RlOg0KPj4+Pj4NCj4+Pj4+DQo+Pj4+PiBP
biAzMS8wMS8yMDI0IDE1OjMyLCBKb2huIEVybmJlcmcgd3JvdGU6DQo+Pj4+Pj4gSGkgSnVsaWVu
LA0KPj4+Pj4NCj4+Pj4+IEhpIEpvaG4sDQo+Pj4+Pg0KPj4+Pj4+IE9uIDEvMzEvMjQgMTM6MjIs
IEp1bGllbiBHcmFsbCB3cm90ZToNCj4+Pj4+Pj4gSGksDQo+Pj4+Pj4+DQo+Pj4+Pj4+IE9uIDMx
LzAxLzIwMjQgMTE6NTAsIEpvaG4gRXJuYmVyZyB3cm90ZToNClsgLi4uIF0NCj4+Pj4+DQo+Pj4+
Pj4+IEJ1dCBldmVuIGlmIHdlIHJlc3RyaWN0IHRvIGRvbTAsIGhhdmUgeW91IGNoZWNrZWQgdGhh
dCBub25lIG9mIHRoZQ0KPj4+Pj4+PiBTTUNzIHVzZSBidWZmZXJzPw0KPj4+Pj4+IEkgaGF2ZW4n
dCBmb3VuZCBhbnkgc3VjaCBpbnN0YW5jZXMgaW4gdGhlIExpbnV4IGtlcm5lbCB3aGVyZSBhDQo+
Pj4+Pj4gYnVmZmVyIGlzIHVzZWQuIEFkZGluZyBhIGNhbGwgZmlsdGVyaW5nIGxpa2Ugc3VnZ2Vz
dGVkIGJlbG93DQo+Pj4+Pj4gYWRkaXRpb25zIG9mIHN1Y2ggZnVuY3Rpb25zIGNhbiBiZSBkaXNj
b3ZlcmVkIGFuZCBhZGFwdGVkIGZvciBpZiB0aGV5DQo+Pj4gd291bGQgc2hvdyB1cCBsYXRlci4N
Cj4+Pj4+Pj4NCj4+Pj4+Pj4gUmF0aGVyIHRoYW4gcHJvdmlkaW5nIGEgYmxhbmtldCBmb3J3YXJk
LCB0byBtZSBpdCBzb3VuZHMgbW9yZSBsaWtlDQo+Pj4+Pj4+IHlvdSB3YW50IHRvIHByb3ZpZGUg
YW4gYWxsb3dsaXN0IG9mIHRoZSBTTUNzLiBUaGlzIGlzIG1vcmUNCj4+Pj4+Pj4gZnV0dXJlcHJv
b2YgYW5kIGF2b2lkIHRoZSByaXNrIHRvIGV4cG9zZSB1bnNhZmUgU01DcyB0byBhbnkgZG9tYWlu
Lg0KPj4+Pj4+Pg0KPj4+Pj4+PiBGb3IgYW4gZXhhbXBsZSwgeW91IGNhbiBoYXZlIGEgbG9vayBh
dCB0aGUgRUVNSSBtZWRpYXRvciBmb3IgWGlsaW54Lg0KPj4+Pj4+DQo+Pj4+Pj4gQWNrLiBEbyB5
b3UgcHJlZmVyIHRvIHNlZSBvbmx5IG9uIFNNQ0NDIHNlcnZpY2UgbGV2ZWwgb3IgYWxzbyBvbg0K
Pj4+Pj4+IGZ1bmN0aW9uIGxldmVsPyAoYTEgcmVnaXN0ZXIsIHBlciBkZXNjcmlwdGlvbiBlYXJs
aWVyKQ0KPj4+Pj4NCj4+Pj4+IEkgYW0gbm90IHN1cmUuIEl0IHdpbGwgZGVwZW5kIG9uIHdoZXRo
ZXIgaXQgaXMgY29ycmVjdCB0byBleHBvc2UNCj4+Pj4+ICphbGwqIHRoZSBmdW5jdGlvbnMgd2l0
aGluIGEgc2VydmljZSBsZXZlbCBhbmQgdGhleSBoYXZlIHRoZSBzYW1lIA0KPj4+Pj4gZm9ybWF0
Lg0KPj4+Pj4NCj4+Pj4+IElmIHlvdSBjYW4ndCBndWFyYW50ZWUgdGhhdCwgdGhlbiB5b3Ugd2ls
bCBtb3N0IGxpa2VseSBuZWVkIHRvDQo+Pj4+PiBhbGxvd2xpc3QgYXQgdGhlIGZ1bmN0aW9uIGxl
dmVsLg0KPj4+Pj4NCj4+Pj4+IEFsc28sIGRvIHlvdSBoYXZlIGEgc3BlYyBpbiBoYW5kIHRoYXQg
d291bGQgaGVscCB0byB1bmRlcnN0YW5kIHdoaWNoDQo+Pj4+PiBzZXJ2aWNlL2Z1bmN0aW9uIGlz
IGltcGxlbWVudGVkIHZpYSB0aG9zZSBTTUNzPw0KPj4+Pg0KPj4+PiBJIGRvbid0IGhhdmUgdGhl
IHNwZWMgdW5mb3J0dW5hdGVseSwgYnV0IEkgd2lsbCBhZGQgYSBmaWx0ZXIgb24gYm90aA0KPj4+
PiBzZXJ2aWNlIGFuZCBmdW5jdGlvbiBmb3IgVjIgYW5kIHdlJ2xsIHRha2UgaXQgZnJvbSB0aGVy
ZS4NCj4+Pg0KPj4+IEBQZW5nLCBkbyB5b3UgaGF2ZSBhbnkgc3BlY2lmaWNhdGlvbiB5b3UgY291
bGQgc2hhcmU/IEhvdyBzdGFibGUgaXMgdGhlDQo+Pj4gaW50ZXJmYWNlPw0KPj4NCj4+IE5vIHNw
ZWNpZmljYXRpb24sIHRoZSB1c2UgaXMgSU1YX1NJUF9YIGluIGxpbnV4IGtlcm5lbCBzb3VyY2Uu
DQo+Pg0KPj4gU3VjaCBhcyBJTVhfU0lQX1JUQywgSU1YX1NJUF9USU1FUg0KPj4NCj4+IEl0IGlz
IHN0YWJsZSBhbmQgbm8gY2hhbmdlLCB3ZSBvbmx5IGFkZCBuZXcgU0lQIG1hY3JvIGlmIG5lZWRl
ZA0KPj4gYW5kIG5vIGNoYW5nZSB0aGUgbWVhbmluZyBvciByZXVzZSBvbGQgU0lQLg0KPiANCj4g
VGhhbmtzIGZvciB0aGUgYW5zd2VyLiBJdCBpcyBhIGJpdCB1bmZvcnR1bmF0ZSB0aGVyZSBhcmUg
bm8gDQo+IHNwZWNpZmljYXRpb24sIGJ1dCBhdCBsZWFzdCB0aGV5IGFyZSBzdGFibGUuDQo+IA0K
PiBJIGhhdmUgc2VhcmNoZWQgSU1YX1NJUCBpbiBMaW51eCwgdGhlcmUgZG9lc24ndCBzZWVtIG1h
bnkgc28gd2UgY291bGQgDQo+IGFsbG93bGlzdCB0aGVtIChzZWUgbW9yZSBiZWxvdykuIERvIHlv
dSBrbm93IGlmIHRoZXJlIGFyZSBtb3JlIG5lY2Vzc2FyeSANCj4gdGhhdCBhcmUgbm90IHlldCB1
cHN0cmVhbWVkIGluIExpbnV4Pw0KDQpJIHRvb2sgYSBkaXZlIGludG8gYm90aCB1cHN0cmVhbSBr
ZXJuZWwgYW5kIHRoZSB2ZW5kb3IgdHJlZSBhbmQgZm91bmQgDQp0aGUgZm9sbG93aW5nIGxpc3Qg
YW5kIGZvciB3aGljaCBTb0NzIHRoZXkgYXJlIGFwcGxpY2FibGUgKFBsZWFzZSANCmNvcnJlY3Qg
bWUgaWYgeW91IGNhbiBQZW5nKQ0KDQoweDgyMDAwMDA2IElNWF9TSVBfQlVTRlJFUV9DSEFOR0Ug
W3Vuc3VyZSwgcHJvYmFibHkgbm90IGlteDhdDQoweEMyMDAwMDAwIElNWF9TSVBfR1BDIFtvbmx5
IGlteDhtIHNlcmllc10NCjB4QzIwMDAwMDEgSU1YX1NJUF9DUFVGUkVRIFtvbmx5IGlteDhxe3gs
bX0gc2VyaWVzXQ0KMHhDMjAwMDAwMiBJTVhfU0lQX1NSVEMgLyBJTVhfU0lQX1RJTUVSIFtvbmx5
IGlteDhxe3gsbX0gc2VyaWVzXQ0KMHhDMjAwMDAwNCBJTVhfU0lQX0REUl9EVkZTIFtvbmx5IGlt
eDhtIHNlcmllc10NCjB4QzIwMDAwMDUgSU1YX1NJUF9SUFJPQyAvIElNWF9TSVBfU1JDIFtvbmx5
IGlteDhtIHNlcmllc10NCjB4QzIwMDAwMDYgSU1YX1NJUF9HRVRfU09DX0lORk8gW29ubHkgaW14
OG0gc2VyaWVzXQ0KMHhDMjAwMDAwOCBJTVhfU0lQX05PQyBbb25seSBpbXg4bSBzZXJpZXNdDQow
eEMyMDAwMDA5IElNWF9TSVBfV0FLRVVQX1NSQyBbb25seSBpbXg4cXt4LG19IHNlcmllc10NCjB4
QzIwMDAwMEIgSU1YX1NJUF9PVFBfV1JJVEUgW29ubHkgaW14OHF7eCxtfSBzZXJpZXNdDQo+IA0K
PiANCj4gTG9va2luZyB0aHJvdWdoIHRoZSBsaXN0LCB0aGVyZSBhcmUgc29tZSB0aGF0IHByb2Jh
Ymx5IHdhbnQgYSBiaXQgbW9yZSANCj4gZGlzY3Vzc2lvbiBvbiB3aGV0aGVyIHdlIHdhbnQgdG8g
ZXhwb3NlIHRoZW06DQo+ICDCoCAqIElNWF9TSVBfQ1BVRlJFUTogUmlnaHQgbm93LCBkb20wIGlz
IG5vdCBhd2FyZSBvZiB0aGUgZnVsbCBzeXN0ZW0uIA0KPiBTbyBpdCBtYXkgdGFrZSB3cm9uZyBk
ZWNpc2lvbi4NClRoZSBjcHVmcmVxIGZ1bmN0aW9uIG9wZXJhdGVzIG9uIHRoZSBjbHVzdGVyIGxl
dmVsLCBpdCBwZXJmb3JtcyBubyBlcnJvciANCmNoZWNraW5nIHNvIGJsb2NraW5nIGl0IHdpbGwg
YmUgaW52aXNpYmxlIHRvIExpbnV4LiBJIGRvbid0IGtub3cgeWV0IA0Kd2hhdCBraW5kIG9mIGlt
cGFjdCB0aGF0IGNvdWxkIGhhdmUuIExvb2tpbmcgZm9yIGhpbnRzIGFib3V0IGNwdWZyZXEgaW4g
DQpYZW4gZm9yIEFSTSBJIGZvdW5kIFsxXSBhbmQgWzJdLg0KDQpJbiBvdXIgY3VycmVudCBzZXR1
cCB3ZSBkbyBub3QgdXNlIGNwdWZyZXEgYW5kIGl0IGlzIHN0YWJsZSBlbm91Z2ggZm9yDQpvdXIg
dXNlY2FzZS4NCg0KIEZyb20gbXkgcG9pbnQgb2YgdmlldyB3ZSBjYW4gYmxvY2sgaXQgZm9yIG5v
dywgYW5kIHdoZW4gY3B1ZnJlcSANCmZ1bmN0aW9uYWxpdHkgZ2V0cyBwaWNrZWQgdXAgYWdhaW4s
IHdlIGNhbiByZXZpc2l0Lg0KDQo+ICDCoCAqIElNWF9TSVBfRERSX0RWRlM6IFNvbWUgb3BlcmF0
aW9uIHNlZW1zIHRvIHRha2UgdGhlIG51bWJlciBvZiBvbmxpbmUgDQo+IENQVXMuIERvbTAgZG9l
c24ndCBrbm93IHRoYXQNCg0KVGhlIGlNWDhRe1gsTX0gc2VyaWVzIGNvbWVzIHdpdGggYSBzeXN0
ZW0gY29udHJvbGxlciBjYWxsZWQgdGhlIFNDVSwgDQpydW5uaW5nIGluIGEgc21hbGwgTS1jb3Jl
LiBUaGlzIGNvbnRyb2xsZXIgdGFrZXMgY2FyZSBvZiBhbGwgdGhlIEREUiANCmJpdHMsIHNvIHdl
IGRvIG5vdCBuZWVkIHRvIGNhcmUgZm9yIHRoZXNlIFNvQ3MuDQoNCj4gIMKgICogSU1YX1NJUF9U
SU1FUl8qOsKgIFRoaXMgc2VlbXMgdG8gYmUgcmVsYXRlZCB0byB0aGUgd2F0Y2hkb2cuIA0KPiBT
aG91bGRuJ3QgZG9tMCByZWx5IG9uIHRoZSB3YXRjaGRvZyBwcm92aWRlZCBieSBYZW4gaW5zdGVh
ZD8gU28gdGhvc2UgDQo+IGNhbGwgd2lsbCBiZSB1c2VkIGJ5IFhlbi4NCg0KVGhhdCBpcyBpbmRl
ZWQgYSB3YXRjaGRvZyBTSVAsIGFuZCBhbHNvIGZvciBzZXR0aW5nIHRoZSBTb0MgaW50ZXJuYWwg
UlRDDQppZiBpdCBpcyBiZWluZyB1c2VkLg0KDQpJIGxvb2tlZCBhcm91bmQgaWYgdGhlcmUgd2Fz
IHByZXZpb3VzIGRpc2N1c3Npb24gYW5kIG9ubHkgcmVhbGx5IGZvdW5kIFszXS4NCklzIHRoZSB4
ZW4veGVuL2luY2x1ZGUvd2F0Y2hkb2cuaCBoZWFkZXIgbWVhbnQgdG8gYmUgZm9yIHRoaXMga2lu
ZCBvZiANCndhdGNoZG9nIHN1cHBvcnQgb3IgaXMgdGhhdCBtb3JlIGZvciB0aGUgVk0gd2F0Y2hk
b2c/IExvb2tpbmcgYXQgdGhlIHg4NiANCkFDUEkgTk1JIHdhdGNoZG9nIGl0IHNlZW1zIGxpa2Ug
dGhlIGZvcm1lciwgYnV0IEkgaGF2ZSBuZXZlciB3b3JrZWQgd2l0aCANCng4NiBub3IgQUNQSS4N
Cg0KQ3VycmVudGx5IGZvcndhcmRpbmcgaXQgdG8gRG9tMCBoYXMgbm90IGNhdXNlZCBhbnkgd2F0
Y2hkb2cgcmVzZXRzIHdpdGggDQpvdXIgd2F0Y2hkb2cgdGltZW91dCBzZXR0aW5ncywgb3VyIHNw
ZWNpZmljIERvbTAgc2V0dXAgYW5kIFZNIGNvdW50Lg0KDQpGb3IgdGhlIHJlbWFpbmluZyBiaXRz
Og0KDQpUaGUgd2FrZXVwIHNyYyBpcyBub3QgaW4gdGhlIHVwc3RyZWFtIGtlcm5lbCB5ZXQgYW5k
IHJlbGF0ZWQgdG8gc3lzdGVtIA0KcmVzdW1lIGZyb20gc3VzcGVuZCB3aGljaCBpc24ndCBzdXBw
b3J0ZWQgaW4gWGVuIG9uIEFSTSB5ZXQgLSBzbyB0aGlzIHdlIA0KY2FuIGJsb2NrIHNhZmVseS4N
Cg0KVGhlIE9UUCB3cml0ZSBpcyBmdXNlIHByb2dyYW1taW5nIGluIHRoZSBTb0MgZnVzZSBiYW5r
cywgYW5kIHNob3VsZCANCnByb2JhYmx5IGJlIGFsbG93ZWQgYnV0IHJlc2VydmVkIGZvciBEb20w
LCBhcyB0aGlzIGNhbiBzZXQgZnVzZXMgdGhhdA0KYWZmZWN0cyB0aGUgU29DIGJvb3QuDQoNCj4g
DQo+IEFsc28sIHdoYXQgaGFwcGVuIGlmIHdlIGRvbid0IGV4cG9zZSB0aG9zZSBTTUMgdG8gZG9t
MD8NCj4gDQo+IENoZWVycywNCj4gDQoNClsxXTogDQpodHRwczovL2xvcmUua2VybmVsLm9yZy94
ZW4tZGV2ZWwvMTUxMDI0NzQyMS0yNDA5NC0xLWdpdC1zZW5kLWVtYWlsLW9sZWtzdHlzaEBnbWFp
bC5jb20vDQpbMl06IA0KaHR0cHM6Ly93d3cuc2xpZGVzaGFyZS5uZXQveGVuX2NvbV9tZ3IveHBk
ZHMxOC1jcHVmcmVxLWluLXhlbi1vbi1hcm0tb2xla3NhbmRyLXR5c2hjaGVua28tZXBhbS1zeXN0
ZW1zDQpbM106IGh0dHBzOi8veGVuLXVzZXJzLm5hcmtpdmUuY29tL0lTWG5sbUIwL3dhdGNoZG9n
LXN1cHBvcnQtaW4teGVuDQoNClRoYW5rcyEgLy8gSm9obiBFcm5iZXJn


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 13:20:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 13:20:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678706.1056194 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYQoB-0004Pq-MP; Fri, 09 Feb 2024 13:20:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678706.1056194; Fri, 09 Feb 2024 13:20:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYQoB-0004Pj-Jc; Fri, 09 Feb 2024 13:20:47 +0000
Received: by outflank-mailman (input) for mailman id 678706;
 Fri, 09 Feb 2024 13:20:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BFPS=JS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rYQoA-0004Ot-JM
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 13:20:46 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 085edf5a-c74e-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 14:20:45 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 5A2D74EE0739;
 Fri,  9 Feb 2024 14:20:45 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 085edf5a-c74e-11ee-8a4b-1f161083a0e0
MIME-Version: 1.0
Date: Fri, 09 Feb 2024 14:20:45 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Julien Grall <julien@xen.org>
Cc: Jan Beulich <jbeulich@suse.com>, sstabellini@kernel.org,
 michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com,
 consulting@bugseng.com, andrew.cooper3@citrix.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, George Dunlap <george.dunlap@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH v2] docs/misra: add asm-offset.c to exclude-list
In-Reply-To: <5cdae9ce-d2ff-40bd-bc07-aa1965275833@xen.org>
References: <09b659e19bf2cc6b3ee4320e019bdfa7def5f3b8.1707406598.git.nicola.vetrini@bugseng.com>
 <891e8cc2-e073-4cd3-9998-4c01ae775a3b@suse.com>
 <8977390f-9549-40d5-bfdc-5c3da81521fc@xen.org>
 <efc0eee4-9903-4228-b33e-cb50d4c545ee@suse.com>
 <5cdae9ce-d2ff-40bd-bc07-aa1965275833@xen.org>
Message-ID: <f6c91540dbb5a83d3172bb65b6d4ca4b@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-09 13:17, Julien Grall wrote:
> Hi Jan,
> 
> On 09/02/2024 11:46, Jan Beulich wrote:
>> On 09.02.2024 10:40, Julien Grall wrote:
>>> Replying on the v2 as well.
>> 
>> And answering here despite the respective question was raised on the
>> v1 thread: I'm certainly okay with the more detailed commit message.
> 
> Ah yes. Sorry, I replied to v1 first and then realized it may have been 
> easier to comment on v2.
> 
>> A few nits, though:
>> 
>>> On 08/02/2024 15:56, Jan Beulich wrote:
>>>> On 08.02.2024 16:50, Nicola Vetrini wrote:
>>>>> These files contain several deliberate violations of MISRA C rules 
>>>>> and
>>>>> they are not linked in the final Xen binary, therefore they can be 
>>>>> exempted
>>>>> from MISRA compliance.
>>> 
>>> I'd like the commit message to be expanded a little bit to explain 
>>> which
>>> MISRA rules are a problem. This helped me to understand why we 
>>> excluded
>>> rather than fixed.
>>> 
>>> Base on the previous discussion, I would suggest:
>>> 
>>> These files contain several deliberate violation of MISRA C rules 
>>> such as:
>> 
>> violations
>> 
>>>     * R20.12 on Arm for macros DEFINE and OFFSET, where the second
>>> argument of OFFSET is a macro and is used as a normal parameter and a
>>> stringification operand.
>> 
>> Is this really for Arm only?
> 
> I don't exactly know. I took Nicola's comment and massage it for the 
> commit message. I am assuming that this was also not exhaustive list, 
> so my aim was to only provide some example.
> 

Yes, indeed. I mentioned the first two examples that I remembered.

> Thinking of it, I don't see why it would only be a problem on Arm. I 
> can drop the "on Arm".
> 

My bad, I missed it in the output. Please drop the "on Arm" part.

>> 
>>>     * R2.1 because the file is not linked That said it was decided to
>>> deviate the rule itselfed to deviate that aspect).
>> 
>> There look to be punctuation issues here. Also s/itselfed/itself/, and
>> the duplicate "deviate" is also a little odd to read (maybe "deal 
>> with"
>> or "address" in place of the 2nd instance).
> 
> Doh, indeed. This wants to be:
> 
> "R2.1 because the file is not linked. That said, it was decided to 
> deviate the ruule itself to address that aspect."
                                                                          
         ^ rule
>>> The files are also not linked in the final Xen binary, therefore they
>>> can be expempted from MISRA compliance.
>> 
>> Looks to duplicate what the latter half of the 2nd bullet point has.
>> If to be kept: s/expempted/exempted/.
> 
> I will remove.
> 
>> 
>>>>> --- a/docs/misra/exclude-list.json
>>>>> +++ b/docs/misra/exclude-list.json
>>>>> @@ -101,6 +101,10 @@
>>>>>                "rel_path": "arch/x86/efi/check.c",
>>>>>                "comment": "The resulting code is not included in 
>>>>> the final Xen binary, ignore for now"
>>>>>            },
>>>>> +        {
>>>>> +          "rel_path": "arch/*/*/asm-offsets.c",
>>>>> +          "comment": "The resulting code is not included in the 
>>>>> final Xen binary, ignore for now"
>>>>> +        },
>>>>>            {
>>>>>                "rel_path": "common/coverage/*",
>>>>>                "comment": "Files to support gcov, ignore for now"
>>>> 
>>>> ... something looks odd with indentation; can probably be adjusted
>>>> while committing.
>>> 
>>> I am happy to take care of both the commit message and the 
>>> indentation
>>> on commit.
>> 
>> Okay, I'll leave that to you then.
> 
> Thanks. I will do it shortly.

Thanks,

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 13:54:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 13:54:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678716.1056204 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYRKE-0000Qj-91; Fri, 09 Feb 2024 13:53:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678716.1056204; Fri, 09 Feb 2024 13:53:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYRKE-0000Qc-5U; Fri, 09 Feb 2024 13:53:54 +0000
Received: by outflank-mailman (input) for mailman id 678716;
 Fri, 09 Feb 2024 13:53:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1vtu=JS=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1rYRKC-0000QW-0v
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 13:53:52 +0000
Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com
 [136.143.188.12]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a4bd1674-c752-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 14:53:47 +0100 (CET)
Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com
 [209.85.128.171]) by mx.zohomail.com
 with SMTPS id 1707486822674974.7715314425574;
 Fri, 9 Feb 2024 05:53:42 -0800 (PST)
Received: by mail-yw1-f171.google.com with SMTP id
 00721157ae682-604b23fc6a7so10861517b3.0
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 05:53:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a4bd1674-c752-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; t=1707486823; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=mtRQSytfSnfrgPL8u7ZP0DpsqIEqZRXtfJb5z+UAaO0QKy4Ffirhp1RetJVBeJeClcVKNnozZBT+i9htT2SwzAceAbzUSewExNrlETYnvJonKqO2D/VoEwSjep87x8qq6T9eJEmCVf0a4VhbPSTwIOeWi8kfzQmVUyWT8+L0awI=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1707486823; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=oqnESpiVyk/PjxgNGOrqwPCDLj/GqlaTMXewMMadClQ=; 
	b=PIMDBG9Ilua4/ePvMs7EPx7+6fQckoUYLTJDugsSO6gfMyNKPQN15oN6X50e2OWiLPG/x8kRklCOvehU6ZFAuxIikpz5QoXU9hCf2OZppcRexgKb31AIeQ4k91tvXkddNW6jpetWX1ldwwjb7GiuSGPmKfxjihzfnpore0e9MOA=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1707486823;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=oqnESpiVyk/PjxgNGOrqwPCDLj/GqlaTMXewMMadClQ=;
	b=P98QSSeJKx/ujZTdCQUXp4EzWG94DJHp174SDFUF9YiqkBqnTYc32UrK7LPD/LuG
	9TbmeyGcV5KBhK6tdIEiFeU3+4T5rhoHvQJYjtQ36NHiI0uiCpTrUyVlA6JPhepUMlM
	V2SP8xiLKPm63b6qnyfldLLvnvp8WVz8aNxikS2c=
X-Gm-Message-State: AOJu0YxKUlI7ovlV9fYgyRP5ST9lLOarbUrrivee1eJoh6aAZfYhvRY1
	6q4fmq77dYuvT3L8cb5t+lcj8lOePgF9kSv/7AKIB/B2LhHary8c0iAT+MXBXrPj6TZchRvC3Hm
	QQvoYD72mrcAsw1ctnhZlIejLivE=
X-Google-Smtp-Source: AGHT+IEi9KoJlvzKr2w6/bJN14FwerfDBAfbJqruA7jmCavob4uS/RB5aAUOXUhc/zHN64EFifwnNh5X3dKtiUOVcac=
X-Received: by 2002:a81:4e83:0:b0:5eb:de3c:fbff with SMTP id
 c125-20020a814e83000000b005ebde3cfbffmr1727657ywb.27.1707486821808; Fri, 09
 Feb 2024 05:53:41 -0800 (PST)
MIME-Version: 1.0
References: <fa519b9a544d3d19a31313292a909d12775e6e1f.1707427103.git.w1benny@gmail.com>
In-Reply-To: <fa519b9a544d3d19a31313292a909d12775e6e1f.1707427103.git.w1benny@gmail.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Fri, 9 Feb 2024 08:53:07 -0500
X-Gmail-Original-Message-ID: <CABfawhnqjTVh6=YCjN1CHEQOEh_ji_6dMzDk=sdXnzBony3g+Q@mail.gmail.com>
Message-ID: <CABfawhnqjTVh6=YCjN1CHEQOEh_ji_6dMzDk=sdXnzBony3g+Q@mail.gmail.com>
Subject: Re: [PATCH] x86/hvm: Fix fast singlestep state persistence
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 8, 2024 at 4:20=E2=80=AFPM Petr Bene=C5=A1 <w1benny@gmail.com> =
wrote:
>
> From: Petr Bene=C5=A1 <w1benny@gmail.com>
>
> This patch addresses an issue where the fast singlestep setting would per=
sist
> despite xc_domain_debug_control being called with XEN_DOMCTL_DEBUG_OP_SIN=
GLE_STEP_OFF.
> Specifically, if fast singlestep was enabled in a VMI session and that se=
ssion
> stopped before the MTF trap occurred, the fast singlestep setting remaine=
d
> active even though MTF itself was disabled.  This led to a situation wher=
e, upon
> starting a new VMI session, the first event to trigger an EPT violation w=
ould
> cause the corresponding EPT event callback to be skipped due to the linge=
ring
> fast singlestep setting.
>
> The fix ensures that the fast singlestep setting is properly reset when
> disabling single step debugging operations.
>
> Signed-off-by: Petr Bene=C5=A1 <w1benny@gmail.com>

Thanks, this has been a known bug that awaited a fix for a long time.

Reviewed-by: Tamas K Lengyel <tamas@tklengyel.com>


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 14:21:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 14:21:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678726.1056213 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYRkO-00057w-BS; Fri, 09 Feb 2024 14:20:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678726.1056213; Fri, 09 Feb 2024 14:20:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYRkO-00057p-8T; Fri, 09 Feb 2024 14:20:56 +0000
Received: by outflank-mailman (input) for mailman id 678726;
 Fri, 09 Feb 2024 14:20:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYRkN-00057f-Pg; Fri, 09 Feb 2024 14:20:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYRkN-0004Xf-C8; Fri, 09 Feb 2024 14:20:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYRkN-0006A2-04; Fri, 09 Feb 2024 14:20:55 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYRkM-0005i4-Vj; Fri, 09 Feb 2024 14:20:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TS4jVk9b+wv6wqkP5OMSRqHVzEWXicBXgYmwPthqu0U=; b=X4Ad5I9zQKBnfjQ85e7JazeQzQ
	WNU89C9lZ/C1L2Q/U5e7A9lwgruncLDNDEy5ebKMH9iyFJF1EAZm3RDDazBlSwFhJqA5jE0ue5YnP
	vU6igBM8bIy9H9ldfPzGWYEPzjoCRFOMuIsIxFjy8so+hTZwPUuGdKEtdxzXAHIh5vaI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184632-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184632: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=3f01ecd830204f128ef1d5f85c6b316b5f2cffdd
X-Osstest-Versions-That:
    libvirt=9ef6541cecde4ebc16bcfc9c5860006f0d8fb2a9
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 09 Feb 2024 14:20:54 +0000

flight 184632 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184632/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184624
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184624
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184624
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              3f01ecd830204f128ef1d5f85c6b316b5f2cffdd
baseline version:
 libvirt              9ef6541cecde4ebc16bcfc9c5860006f0d8fb2a9

Last test of basis   184624  2024-02-08 04:19:11 Z    1 days
Testing same since   184632  2024-02-09 04:18:49 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrea Bolognani <abologna@redhat.com>
  Peter Krempa <pkrempa@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   9ef6541cec..3f01ecd830  3f01ecd830204f128ef1d5f85c6b316b5f2cffdd -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 14:44:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 14:44:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678733.1056224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYS78-0008Om-4C; Fri, 09 Feb 2024 14:44:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678733.1056224; Fri, 09 Feb 2024 14:44:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYS78-0008Of-1F; Fri, 09 Feb 2024 14:44:26 +0000
Received: by outflank-mailman (input) for mailman id 678733;
 Fri, 09 Feb 2024 14:44:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4TWb=JS=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rYS77-0008OZ-Ft
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 14:44:25 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b6cfae60-c759-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 15:44:23 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-41068e36cbbso5686685e9.2
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 06:44:23 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p6-20020adfcc86000000b0033afb963d0dsm1965253wrj.101.2024.02.09.06.44.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 09 Feb 2024 06:44:22 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b6cfae60-c759-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707489862; x=1708094662; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+3Yj7hO4QutSZqSdyo1ZNc+oTsKD4Eomw49ZUTIRolA=;
        b=bvk7ZAbKkTDTYMnHRjBsTzj7uZlNzJ+eoLdh1Wd6yZyASIiUDF5Uwpkq7L0wtACCir
         sWhGbc4mdsvfnd8EoAAc4kA6IZ8epxf3HIYPRWwR0ew27MynCMrW2nsCtksk1BkHeX+z
         +dTnsK7u7mvtVi1owvwbS1GSfQnETZQasCJnvb4vPaQlI80YlGjAKJTrJ0FQGEkqlVoI
         MSJA0iBnH927UfpdJ+s3diwxQgK4MiGhpYn60eVFE3kOi7S31O5dcvsVOtKQU+v5wwP+
         ab5zJmqWlapfbJZq1ipSn/h5YcMhraON1wSjL/XK5jvG1ogekEfdHlZDpt9IgmW79ADw
         reVQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707489862; x=1708094662;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+3Yj7hO4QutSZqSdyo1ZNc+oTsKD4Eomw49ZUTIRolA=;
        b=f7h+8m+to9sBtuirRPh5Ch5y6/5ZIzbiWsC9GwZU6QYpChyUoGEkyI9T9GaQhW0DFo
         ENAp5Ksp+iymS5qSuutpDUBcaktpjh4Z0defp5OIkhRFQDYTsXlPdCqmT5zcSspils0o
         UwQ7eCxZbs98szLo+VXRTjsPdYRotZoHnvTXUYBxNL5GD9qktiBU6v0kf5vLSZ6Enuix
         bc4hvFqlwd1PpdJn76lvcMqM8jBx6KhuzNpTftyKXGBpNbVxv2qw26y03Kfaw9W5u8nk
         TOAOsD/R5kJ9CCk4PPy2nnWEwZdiBu/G+EEDHXpN/8e3Eye4t1DrPGZJAJxjqQmat7PQ
         lX5g==
X-Gm-Message-State: AOJu0YwrFGBDViLmlGbefIb18PHohaGczNPIxcO4WcMhxKQAurm0uR0H
	xv3bO30evLZNy9r0ouOphb1ds0ISqiZVq2At2G3logOkZAcYpa73vgqs312I2Q==
X-Google-Smtp-Source: AGHT+IFueh0D+GOWMixyDaV3FDC80b7a5zZIwWVjD+liFaVSmCBP5njLHbpXK5LvQPRzZMHyonFmEw==
X-Received: by 2002:a05:600c:3d97:b0:40f:e806:2f19 with SMTP id bi23-20020a05600c3d9700b0040fe8062f19mr1642822wmb.22.1707489862377;
        Fri, 09 Feb 2024 06:44:22 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCV1lJsHYrv2gf15lpbCN/IZ1SZasveslzAbbahxHRlMReJAtAxioSVU0WlVb4c7jrvJQ26BStecsUVqL/MAmOJhJ4WVHmYR4Sw6zsBW20gAWnPE
Message-ID: <5a495da2-f012-4411-8620-7e765a92bd1b@suse.com>
Date: Fri, 9 Feb 2024 15:44:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 6/7] VT-d: move dev_invalidate_iotlb() to the sole file
 it's used from
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <0d14f148-32e0-4dac-b5e8-3230adfc5f9d@suse.com> <ZcXhauUax5WHD93J@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcXhauUax5WHD93J@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 09.02.2024 09:25, Roger Pau Monné wrote:
> On Mon, Feb 05, 2024 at 02:57:12PM +0100, Jan Beulich wrote:
>> ..., thus allowing it and qinval_device_iotlb_sync() to become static.
>> There's nothing x86-specific about the function anyway. While moving,
>> adjust types to better match ./CODING_STYLE (albeit use of fixed-width
>> types for parameters is retained to limit the effective change).
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks.

>> +static int dev_invalidate_iotlb(struct vtd_iommu *iommu, uint16_t did,
>> +                                paddr_t addr, unsigned int size_order,
>> +                                uint64_t type)
>> +{
>> +    struct pci_dev *pdev, *temp;
>> +    int ret = 0;
>> +
>> +    if ( !ecap_dev_iotlb(iommu->ecap) )
>> +        return ret;
>> +
>> +    list_for_each_entry_safe( pdev, temp, &iommu->ats_devices, ats.list )
>> +    {
>> +        bool sbit;
>> +        int rc = 0;
>> +
>> +        switch ( type )
>> +        {
>> +        case DMA_TLB_DSI_FLUSH:
>> +            if ( !device_in_domain(iommu, pdev, did) )
>> +                break;
>> +            /* fall through if DSI condition met */
>> +        case DMA_TLB_GLOBAL_FLUSH:
>> +            /* invalidate all translations: sbit=1,bit_63=0,bit[62:12]=1 */
>> +            sbit = 1;
>> +            addr = (~0UL << PAGE_SHIFT_4K) & 0x7FFFFFFFFFFFFFFF;
> 
> Given the MISRA stuff, won't it be better to append 'UL' here while
> moving?

Sure, done.

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 14:58:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 14:58:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678737.1056235 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYSKr-0002Vj-Bx; Fri, 09 Feb 2024 14:58:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678737.1056235; Fri, 09 Feb 2024 14:58:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYSKr-0002Vc-7e; Fri, 09 Feb 2024 14:58:37 +0000
Received: by outflank-mailman (input) for mailman id 678737;
 Fri, 09 Feb 2024 14:58:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BZDb=JS=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rYSKp-0002VW-Sa
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 14:58:35 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b13ebfda-c75b-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 15:58:32 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a370e7e1e02so116650566b.0
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 06:58:32 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 v26-20020a170906489a00b00a385dd961bbsm831640ejq.192.2024.02.09.06.58.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 09 Feb 2024 06:58:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b13ebfda-c75b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707490712; x=1708095512; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dFexxYaH8CObRk0trpAT4UEPECRJcHoYoTUKZ3cvh+s=;
        b=m50IsjLL4OI031eMPqhtkWLXeIgY8reWSdyhU2GJkNOUbkh6LoiyMkL48HD9JHuarN
         DNBLSKCf2KmPjey5U3FBz2Gu98VU8Kgh+u0vN/IUgXIdwSgA6iuqiCotASeR5oHQMumv
         2ZF6Q+R94OIxcuS4qTujwjXC69j1I320+fHyk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707490712; x=1708095512;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dFexxYaH8CObRk0trpAT4UEPECRJcHoYoTUKZ3cvh+s=;
        b=hXU5kS8JMNv8RBpGCjxpQhiZhrVj/vX8s409KmDj60Slz0CkCqaN2o8CsHeSW10oiu
         357HqNcVEVGE+Kt8G8bba5M6nu/W1MhiptefzAvaX+g/vVrH3x4MGjBcyu6ByRouOSZ6
         b46k9aAVv5Hnr6iYzJkMs4dv/wUq55e9gbQ5m2JOE5yGD58wQDbxe7GoCgnCwO7y0Iu3
         Y2T3TMKVznkxtk61ceeV/TeQeOPkh9j1HsVDU5RpWs2Zw8eC0MxQfNSpjDhUozz6vA/E
         CEsbxirzxkBS/KpTGaD0vTP7rdRSXrkA5MLKa+I89+fCVKgpjNrDLLqgKfyEa3xLjiYn
         fDpQ==
X-Gm-Message-State: AOJu0YyVeWknGPBEmAYNHpoZPSaXTqv0sLsX/RMpsiCp/u1GlUKZJtKd
	FhH0dJNzu+klh2UAu6IjmlHzB5RmDB+BF1B0B45Os75O+3re3eyO/gb/Cbgns6I=
X-Google-Smtp-Source: AGHT+IHIP8N+GpuNf3670pKBXb1L718uMBLYyC3UyhckHJkTnohTtHbAUT34acr/EBw+9S8/zHwXmQ==
X-Received: by 2002:a17:906:af11:b0:a37:d286:8c98 with SMTP id lx17-20020a170906af1100b00a37d2868c98mr1497119ejb.14.1707490712084;
        Fri, 09 Feb 2024 06:58:32 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXhNrBl2I5wtjF1L5dbxM92vLxJxEd1QirF68a5C7MG8466bTi0tcvKABpoAymnDFzwh62RJ9YQwWb77IE6muuHRqXwYaBT2y9aWkO2qfs9XTe+9ZFrEBhbcB49bF9fvJwcDO1LE65KBw+gD1bmm/oP15SEohVfFXahRXuOwJUJj2K69w==
Message-ID: <eba39016-7d7a-4d35-865c-bfa03674617a@citrix.com>
Date: Fri, 9 Feb 2024 14:58:31 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/hvm: Fix fast singlestep state persistence
Content-Language: en-GB
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: tamas@tklengyel.com, Jan Beulich <jbeulich@suse.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu <wl@xen.org>
References: <fa519b9a544d3d19a31313292a909d12775e6e1f.1707427103.git.w1benny@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <fa519b9a544d3d19a31313292a909d12775e6e1f.1707427103.git.w1benny@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08/02/2024 9:20 pm, Petr Beneš wrote:
> From: Petr Beneš <w1benny@gmail.com>
>
> This patch addresses an issue where the fast singlestep setting would persist
> despite xc_domain_debug_control being called with XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF.
> Specifically, if fast singlestep was enabled in a VMI session and that session
> stopped before the MTF trap occurred, the fast singlestep setting remained
> active even though MTF itself was disabled.  This led to a situation where, upon
> starting a new VMI session, the first event to trigger an EPT violation would
> cause the corresponding EPT event callback to be skipped due to the lingering
> fast singlestep setting.
>
> The fix ensures that the fast singlestep setting is properly reset when
> disabling single step debugging operations.
>
> Signed-off-by: Petr Beneš <w1benny@gmail.com>
> ---
>  xen/arch/x86/hvm/hvm.c | 32 +++++++++++++++++++++++---------
>  1 file changed, 23 insertions(+), 9 deletions(-)
>
> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> index e8deeb0222..4f988de4c1 100644
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -5160,26 +5160,40 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
>  
>  int hvm_debug_op(struct vcpu *v, int32_t op)
>  {
> -    int rc;
> +    int rc = 0;
>  
>      switch ( op )
>      {
>          case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON:
>          case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF:
> -            rc = -EOPNOTSUPP;
>              if ( !cpu_has_monitor_trap_flag )
> -                break;
> -            rc = 0;
> -            vcpu_pause(v);
> -            v->arch.hvm.single_step =
> -                (op == XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON);
> -            vcpu_unpause(v); /* guest will latch new state */
> +                return -EOPNOTSUPP;
>              break;
>          default:
> -            rc = -ENOSYS;
> +            return -ENOSYS;
> +    }
> +
> +    vcpu_pause(v);
> +
> +    switch ( op )
> +    {
> +        case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON:
> +            v->arch.hvm.single_step = true;
> +            break;
> +
> +        case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF:
> +            v->arch.hvm.single_step = false;
> +            v->arch.hvm.fast_single_step.enabled = false;
> +            v->arch.hvm.fast_single_step.p2midx = 0;
>              break;
> +
> +        default:
> +            ASSERT_UNREACHABLE();

Two things.

First, this reads as if it's reachable, and therefore wrong.  You
probably want an /* Excluded above */ comment to point out why it's safe
in this case.

Second, I know you're copying the existing switch(), but it wasn't
compliant with Xen's coding style.  The cases and their clauses should
have one fewer indentation level.

I'm happy to fix up both on commit.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 15:46:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 15:46:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678748.1056243 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYT5P-0001Pf-Re; Fri, 09 Feb 2024 15:46:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678748.1056243; Fri, 09 Feb 2024 15:46:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYT5P-0001PY-Ou; Fri, 09 Feb 2024 15:46:43 +0000
Received: by outflank-mailman (input) for mailman id 678748;
 Fri, 09 Feb 2024 15:46:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=v369=JS=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1rYT5O-0001PS-7J
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 15:46:42 +0000
Received: from mail-oa1-x2f.google.com (mail-oa1-x2f.google.com
 [2001:4860:4864:20::2f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 69994740-c762-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 16:46:40 +0100 (CET)
Received: by mail-oa1-x2f.google.com with SMTP id
 586e51a60fabf-2191b085639so402922fac.0
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 07:46:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 69994740-c762-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707493598; x=1708098398; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=pGLniU/Nd8/gkAQj+uYixOpsybWYqA3IabfJTeP8PU0=;
        b=Q3h/BOL8FRlsLtKVnt9UKmgWWEYlxMgf5bjkKxRRS/1KQGGiYQGEGnfjgYilwyrgOY
         dF9eXVpeIAoKizoodnP6SF4LX/8+1R8FATiH9p0KN5rdPSFRwqRuRxCB7e/EiOF+YpFi
         J6NVbWMTHWFxVpjmkc37ivcjqTnGdyAKUXoyLEufvEzmYD74Q/sMGWK4bpZHYJO1iWSu
         ylq75HBLsHds5JQWlZ7BCwlqW8P2icE6LgGL6T/BTMKh4ltH2doc7AIu5zT2Lrckg6kF
         pHNp5VhD5JJleAfwCiYG7WZuwLMmue7RwiKMonNVN76+z8FZ+d0UXdai8VnLyWUsb/va
         BULQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707493598; x=1708098398;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=pGLniU/Nd8/gkAQj+uYixOpsybWYqA3IabfJTeP8PU0=;
        b=IM43gpo1cnbuweaU6ufDsSbPXF2cspYgzInvRKnJzz0l32QpeBnXlKkIUP+7cWdPki
         RVdr517ays/UTcpSEL+yFNZ/wQWB6s1F7jW2pTqhy68uIcgD0UbPvH/f6b5rNiXKDIRp
         A+J63NcZJNv+nqZJBMPtOyVlpRlj9wjUATfaShlrJxfpXlu+SSUHzDSjYijZl0mHUK52
         CqbzAp3mAZBKPXqAeN+g0ItIc14u1dYNu6K3uU1oe0Fd6UzBc7fZiriOm5b1RtJirKUJ
         X/AXO5W2pNS30/TbZSIAu+cTQqXTLAwwLwBa17rDYpL3IAJGi95RH7n8XN6pJjhaW1UH
         1Apg==
X-Gm-Message-State: AOJu0YwQ+c8LkEgcrOsdb1I5p4xnhh6SvL3BXWD9eFZ48Qfui5ggV/fB
	+Fhy8hG5r4UbfhnafRHgDO79Ar94yUdFeUhjhYXvp7oNj6UMSv7gjDubVXefsMXaqnCJ6xO6ci4
	SfiOOmOv1oSqeSAV5bgPD6N+n2Q8=
X-Google-Smtp-Source: AGHT+IGLhm3d/5UqOMS3N8mjaV72I+nJgfGEbPykyE53OIH3Fo+4fezabWWTAiM/5/hAhR3sFWFVXLWO7q01gA+8ka0=
X-Received: by 2002:a05:6870:a9aa:b0:206:6db:dff9 with SMTP id
 ep42-20020a056870a9aa00b0020606dbdff9mr2085043oab.58.1707493598201; Fri, 09
 Feb 2024 07:46:38 -0800 (PST)
MIME-Version: 1.0
References: <fa519b9a544d3d19a31313292a909d12775e6e1f.1707427103.git.w1benny@gmail.com>
 <eba39016-7d7a-4d35-865c-bfa03674617a@citrix.com>
In-Reply-To: <eba39016-7d7a-4d35-865c-bfa03674617a@citrix.com>
From: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Date: Fri, 9 Feb 2024 16:46:26 +0100
Message-ID: <CAKBKdXhs5Wx8U5iaLWnRuVxMMTJ20zyWO6Nnku06nDg90jtCMA@mail.gmail.com>
Subject: Re: [PATCH] x86/hvm: Fix fast singlestep state persistence
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, tamas@tklengyel.com, 
	Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

> On Fri, Feb 9, 2024 at 3:58=E2=80=AFPM Andrew Cooper <andrew.cooper3@citr=
ix.com> wrote:
>
> On 08/02/2024 9:20 pm, Petr Bene=C5=A1 wrote:
> > From: Petr Bene=C5=A1 <w1benny@gmail.com>
> >
> > This patch addresses an issue where the fast singlestep setting would p=
ersist
> > despite xc_domain_debug_control being called with XEN_DOMCTL_DEBUG_OP_S=
INGLE_STEP_OFF.
> > Specifically, if fast singlestep was enabled in a VMI session and that =
session
> > stopped before the MTF trap occurred, the fast singlestep setting remai=
ned
> > active even though MTF itself was disabled.  This led to a situation wh=
ere, upon
> > starting a new VMI session, the first event to trigger an EPT violation=
 would
> > cause the corresponding EPT event callback to be skipped due to the lin=
gering
> > fast singlestep setting.
> >
> > The fix ensures that the fast singlestep setting is properly reset when
> > disabling single step debugging operations.
> >
> > Signed-off-by: Petr Bene=C5=A1 <w1benny@gmail.com>
> > ---
> >  xen/arch/x86/hvm/hvm.c | 32 +++++++++++++++++++++++---------
> >  1 file changed, 23 insertions(+), 9 deletions(-)
> >
> > diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> > index e8deeb0222..4f988de4c1 100644
> > --- a/xen/arch/x86/hvm/hvm.c
> > +++ b/xen/arch/x86/hvm/hvm.c
> > @@ -5160,26 +5160,40 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HAND=
LE_PARAM(void) arg)
> >
> >  int hvm_debug_op(struct vcpu *v, int32_t op)
> >  {
> > -    int rc;
> > +    int rc =3D 0;
> >
> >      switch ( op )
> >      {
> >          case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON:
> >          case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF:
> > -            rc =3D -EOPNOTSUPP;
> >              if ( !cpu_has_monitor_trap_flag )
> > -                break;
> > -            rc =3D 0;
> > -            vcpu_pause(v);
> > -            v->arch.hvm.single_step =3D
> > -                (op =3D=3D XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON);
> > -            vcpu_unpause(v); /* guest will latch new state */
> > +                return -EOPNOTSUPP;
> >              break;
> >          default:
> > -            rc =3D -ENOSYS;
> > +            return -ENOSYS;
> > +    }
> > +
> > +    vcpu_pause(v);
> > +
> > +    switch ( op )
> > +    {
> > +        case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON:
> > +            v->arch.hvm.single_step =3D true;
> > +            break;
> > +
> > +        case XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF:
> > +            v->arch.hvm.single_step =3D false;
> > +            v->arch.hvm.fast_single_step.enabled =3D false;
> > +            v->arch.hvm.fast_single_step.p2midx =3D 0;
> >              break;
> > +
> > +        default:
> > +            ASSERT_UNREACHABLE();
>
> Two things.
>
> First, this reads as if it's reachable, and therefore wrong.  You
> probably want an /* Excluded above */ comment to point out why it's safe
> in this case.
>
> Second, I know you're copying the existing switch(), but it wasn't
> compliant with Xen's coding style.  The cases and their clauses should
> have one fewer indentation level.
>
> I'm happy to fix up both on commit.
>
> ~Andrew

Thanks for the feedback. If it's not too much of a hassle, I'll be
happy if you fix it.

P.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 15:52:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 15:52:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678753.1056254 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYTAf-0003LP-EF; Fri, 09 Feb 2024 15:52:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678753.1056254; Fri, 09 Feb 2024 15:52:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYTAf-0003LI-BH; Fri, 09 Feb 2024 15:52:09 +0000
Received: by outflank-mailman (input) for mailman id 678753;
 Fri, 09 Feb 2024 15:52:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BFPS=JS=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rYTAd-0003LC-Dc
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 15:52:07 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2bf907fc-c763-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 16:52:05 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 917894EE0739;
 Fri,  9 Feb 2024 16:52:04 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2bf907fc-c763-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Fri, 09 Feb 2024 16:52:04 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Xen Devel <xen-devel@lists.xenproject.org>
Cc: Consulting <consulting@bugseng.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Jbeulich
 <jbeulich@suse.com>, Andrew Cooper3 <andrew.cooper3@citrix.com>, Roger Pau
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Wei Liu
 <wl@xen.org>
Subject: Return type of clean_and_invalidate_dcache_va_range
Message-ID: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

Hi all,

In the context of violations of MISRA C:2012 Rule 17.7: "The value 
returned by a function having non-void return type shall be used", I was 
looking at the function "clean_and_invalidate_dcache_va_range". It has 
the following signature on both arm and x86:

static inline int clean_and_invalidate_dcache_va_range
     (const void *p, unsigned long size)

The commit that introduced it for Arm ~9 years ago (71d64afe3e12: "arm: 
return int from *_dcache_va_range") [1] mentions that on Arm it can't 
fail, but supposedly it can on x86.

However, as far as I can tell, for both arch-es the implementation now 
always returns 0 [2][3], so perhaps the mention of -EOPNOTSUPP for x86 
is no longer true (I wasn't able to reconstruct if there was a time at 
which this was true, even in the same commit that changed the return 
type to int).

The question is: should the return type be void, since it appears that 
every user is ignoring the returned value (violating the rule), except 
the one in common/grant_table.c [4]?

The other two resolution paths are either allowing this function's 
result to be ignored or cast all ignored invocations to void, with the 
first being cleaner from a code readability perspective.

[1] These functions cannot really fail on ARM, but their x86 equivalents 
can (-EOPNOTSUPP). Change the prototype to return int.
[2] 
https://gitlab.com/xen-project/xen/-/blob/staging/xen/arch/arm/include/asm/page.h#L218
[3] 
https://gitlab.com/xen-project/xen/-/blob/staging/xen/arch/x86/include/asm/flushtlb.h#L188
[4] 
https://gitlab.com/xen-project/xen/-/blob/staging/xen/common/grant_table.c#L3576

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 17:57:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 17:57:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678772.1056264 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYV7o-0004Fz-1o; Fri, 09 Feb 2024 17:57:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678772.1056264; Fri, 09 Feb 2024 17:57:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYV7n-0004Fs-Uz; Fri, 09 Feb 2024 17:57:19 +0000
Received: by outflank-mailman (input) for mailman id 678772;
 Fri, 09 Feb 2024 17:57:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYV7m-0004Fm-Eb
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 17:57:18 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a976fcce-c774-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 18:57:17 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-55f50cf2021so1803494a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 09:57:17 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 fi11-20020a056402550b00b0056104738371sm951350edb.65.2024.02.09.09.57.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 09:57:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a976fcce-c774-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501436; x=1708106236; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=/Jlz1d+vTDoRNkUP9lFNds5ZN+DZ59WWMyV4kqtOIRE=;
        b=CvdekzTGSGAA9DjFnsmkD35iPMNvL+y30UR7ZsJVfKmbjb7zpG/KqmW0YsGwWHMWy9
         xWOYeklaR0cJyI8UdGBsl23+jMHbrdnsv72KXp8eu5UpVNRVgVrBbErGId1SAIN0PrE1
         r/GKJ30XPWEn7WMpbBanvDLG0Emhd7VuQElBdlKX7QV39WEk3u1WQcoSo9vIQ7EUwDhp
         l925IOd/MjxsNEa0vttSjh17W97pO+1PmSpp05DCyRxYOYWC53xqifosqRDeEMvvcA9Y
         19ZRasmpuQiZlMhW5N4CsVKQD8T1vVp2JRBFADohCzhYouUB8ue/S2r8ZmeWIMgobzfh
         +e1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501436; x=1708106236;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/Jlz1d+vTDoRNkUP9lFNds5ZN+DZ59WWMyV4kqtOIRE=;
        b=I84yx5XK4TgC+FOr3IpCDZ/vb6ZY3f7nCXCCetmr8uFWoTokTCUfIIxV5T1p9sUNNf
         ZmQ4AFmBy50emIpu/1gZd8fbxgYrDVyRgttcaWWDn/2PpyRdcpa40Gf2wn+ICa6iIQ51
         HRBN+LZWgCnFEfNy9joQxMdfVSy+BRd9zN23CJn3j5viyiSRdYNP+VX7OPhDUn5N4RKX
         cjWjQaNutAuqedEiasW+eAFBpllH754CwCyDKjErK/lDTMRcaZ/HKZ7lcLM7ufDzFjq7
         mc48MlrRzu2pG5khJ5kdqFGourmH3IWvVgVPVnFW2WIxWz+1S8TFMWrvPjzz8+fSnvVk
         0hLA==
X-Forwarded-Encrypted: i=1; AJvYcCX3dmWeF5K04cv8pwg3kbxbjmogTJT0s43aGcAlsTr3rPXO7zW05O6uMbgFSKkA6cgQb2FSb1YZ2rE/CGSpDWACXqUPkFJ8xCZhf4HFB3A=
X-Gm-Message-State: AOJu0YyMERpL3UVbt8WWBD8Se/D24SXgoJkXmX1/7Kx1XtjKnuUWioiH
	Uu02caegnk/iFu3RC64an4AvEmHoBWP1w4GcIkU6+iMY8UStJB/TqPNlwFfh
X-Google-Smtp-Source: AGHT+IHVjzmNsUoeIO9O/oSNeaqHZOR6tal36ZgZZ4RHHYDXpHVwaGM79oAVWFPNC5diqcqWTRiQng==
X-Received: by 2002:aa7:c35a:0:b0:561:2711:68cd with SMTP id j26-20020aa7c35a000000b00561271168cdmr1966085edr.3.1707501436108;
        Fri, 09 Feb 2024 09:57:16 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCULGPQtV3ONfFXgBrfTjNZlITToognPUNgvyOzVb/BsGnZA4YW2AyOwZ5mKptSgQZEq7O0hok+OdcX/PI+gcbSs3cO5mFplgLTg4K3Z4Q+CNNLTYh75LrMz7SN2uRCkE2aUAaOS+CfOarIzHcQQycTfKDEzpUk58HblbO+ZGlY2HsTQRfXlEg==
Message-ID: <189af0b744223de054d06962e3a86d3c0955f084.camel@gmail.com>
Subject: Re: Xen 4.19 release status tracking list
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: committers@xenproject.org, community.manager@xenproject.org, Kelly Choi
	 <kelly.choi@cloud.com>, Xen-devel <xen-devel@lists.xenproject.org>
Date: Fri, 09 Feb 2024 18:57:14 +0100
In-Reply-To: <7799755f-fad2-4f73-a2c1-de97bcce95fa@suse.com>
References: 
	<CAMacjJzio2EDwudCUHoL6psKYztkHZOWwLNDknATvpN6Y0im=Q@mail.gmail.com>
	 <7799755f-fad2-4f73-a2c1-de97bcce95fa@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Tue, 2024-02-06 at 10:24 +0100, Jan Beulich wrote:
> On 05.02.2024 20:23, Oleksii Kurochko wrote:
> > Hello everyone,
> >=20
> > I would like to share with you a list for status tracking based on
> > Xen ML:
> >=20
> > Arm:
> > =C2=A0 * [PATCH v5 00/13] Arm cache coloring [
> > https://lore.kernel.org/xen-devel/20240102095138.17933-1-carlo.nonato@m=
inervasys.tech/
> > ]
> > =C2=A0 * [PATCH v1 repost 0/4] xen/arm64: Rework the MMU-off code
> > (idmap)
> > so it is self-contained [
> > https://lore.kernel.org/xen-devel/20240116143709.86584-1-julien@xen.org=
/
> > ]
> > =C2=A0 * [PATCH v13 00/14] PCI devices passthrough on Arm, part 3 [
> > https://lore.kernel.org/xen-devel/20240202213321.1920347-1-stewart.hild=
ebrand@amd.com/
> > ]
> >=20
> > PPC:
> > =C2=A0 *=C2=A0 [PATCH v2 0/7] Early Boot Allocation on Power [
> > https://lore.kernel.org/xen-devel/cover.1702607884.git.sanastasio@rapto=
rengineering.com/
> > ]
> >=20
> > RISC-V:
> > =C2=A0 *=C2=A0 [PATCH v4 00/30]=C2=A0 Enable build of full Xen for RISC=
-V [
> > https://lore.kernel.org/xen-devel/cover.1707146506.git.oleksii.kurochko=
@gmail.com/
> > ]
> >=20
> > x86:
> > =C2=A0 * [PATCH 0/4] iommu/x86: fixes/improvements for unity range
> > checks [
> > https://lore.kernel.org/xen-devel/20240201170159.66330-1-roger.pau@citr=
ix.com/
> > ]
> > =C2=A0 * [PATCH 0/8] x86: support AVX10.1 [
> > https://lore.kernel.org/xen-devel/298db76f-d0ee-4d47-931f-1baa1a7546cf@=
suse.com/
> > ]
> > =C2=A0 * [PATCH v13 00/35] x86: enable FRED for x86-64 [
> > https://lore.kernel.org/xen-devel/20231205105030.8698-1-xin3.li@intel.c=
om/
> > ]
>=20
> This is a kernel series. I don't expect FRED support is in scope for
> 4.19.
Thanks, overlooked that. I will drop it in the next status tracking
mail.

>=20
> > =C2=A0 * [PATCH v4 0/8] x86emul: misc additions
> > https://lore.kernel.org/xen-devel/9dd23064-c79e-4a50-9c71-c0e73b189944@=
suse.com/
> > =C2=A0 * [PATCH 0/7] VT-d: SATC handling and ATS tidying [
> > https://lore.kernel.org/xen-devel/25506838-b818-4686-8c16-3a198338af44@=
suse.com/
> > ]
> > =C2=A0 * [XEN PATCH 0/9] x86: parallelize AP bring-up during boot [
> > https://lore.kernel.org/xen-devel/cover.1699982111.git.krystian.hebel@3=
mdeb.com/
> > ]
> > =C2=A0 * [PATCH v2 00/12] x86: memcpy() / memset() (non-)ERMS flavors
> > plus
> > fallout [
> > https://lore.kernel.org/xen-devel/8f56a8f4-0482-932f-96a9-c791bebb4610@=
suse.com/
> > ]
> >=20
> > common:
> > =C2=A0 * annotate entry points with type and size" series [
> > https://lore.kernel.org/xen-devel/5e74a501-2824-42af-804b-47b515aa9bcf@=
suse.com/
> > ]
>=20
> The bulk of this has gone in, but there'll want to be follow-ups.
>=20
> > =C2=A0 * [PATCH v2 0/8] limit passing around of cpu_user_regs [
> > https://lore.kernel.org/xen-devel/33db57a2-b569-4fa8-b0b0-36ea92528d7b@=
suse.com/
> > ]
> > =C2=A0 * [PATCH v2 (resend) 00/27] Remove the directmap [
> > https://lore.kernel.org/xen-devel/20240116192611.41112-1-eliasely@amazo=
n.com/
> > ]
> > =C2=A0 * [PATCH v4 00/32] tools: enable xenstore-stubdom to use 9pfs [
> > https://lore.kernel.org/xen-devel/20240205105001.24171-1-jgross@suse.co=
m/
> > ]
> > =C2=A0 * [PATCH v7 0/7] Introduce generic headers [
> > https://lore.kernel.org/xen-devel/cover.1706281994.git.oleksii.kurochko=
@gmail.com/
> > ]
> > =C2=A0 * [PATCH] move __read_mostly to xen/cache.h
> > [
> > https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@=
suse.com
> > /]
> > =C2=A0 * [PATCH v2] NUMA: no need for asm/numa.h when !NUMA [
> > https://lore.kernel.org/xen-devel/94cffa48-5e31-4557-827d-c6dec6031d05@=
suse.com/
> > ]
>=20
> This went in already.
I just pulled staging today, so missed that it was merged.

>=20
> > =C2=A0 * [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE() =
[
> > https://lore.kernel.org/xen-devel/42fc6ae8d3eb802429d29c774502ff232340d=
c84.1706259490.git.federico.serafini@bugseng.com/
> > ]
> > =C2=A0 * [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(),
> > ASSERT_UNREACHABLE() to xen/bug.h [
> > https://lore.kernel.org/xen-devel/4887b2d91a4bf2e8b4b66f039642596519814=
03b.1706897023.git.oleksii.kurochko@gmail.com/
> > ]
> > =C2=A0 * [PATCH] xen/lib: introduce generic find next bit operations [
> > https://lore.kernel.org/xen-devel/52730e6314210ba4164a9934a720c4fda2014=
47b.1706266854.git.oleksii.kurochko@gmail.com/
> > ]
> >=20
> > Please reply with items you would like to see in 4.19 so that
> > people
> > know what is happening and prioritize accordingly.
>=20
> Beyond (on x86) AVX10 I'd like to see whether we can also make some
> progress towards supporting APX. That'll be quite a bit of work,
> though,
> including quite a bit of reviews that are likely to end up on the
> backburner (as has been commonly happening for, especially, more
> involved emulator changes).
>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 17:57:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 17:57:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678775.1056274 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYV8L-0004xS-8h; Fri, 09 Feb 2024 17:57:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678775.1056274; Fri, 09 Feb 2024 17:57:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYV8L-0004xL-65; Fri, 09 Feb 2024 17:57:53 +0000
Received: by outflank-mailman (input) for mailman id 678775;
 Fri, 09 Feb 2024 17:57:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYV8K-0004vz-Ne
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 17:57:52 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bd9c17e8-c774-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 18:57:50 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a30e445602cso452899166b.0
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 09:57:50 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 qx10-20020a170906fcca00b00a384810f371sm963915ejb.190.2024.02.09.09.57.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 09:57:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd9c17e8-c774-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501470; x=1708106270; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=X8dA/oVMrRThJ+rwgG6q9S6a3f0WD1rU2/Q4oEE47Ko=;
        b=cJctG8WZfuhgfGjG8obXjNsgDfAWF7eob8tEK1v6GTl5N9dHr+t4JtvEpkDRKhqEYv
         z6DnPOb4ay6j47WH41oRxCJcZHeQ5jYP04qjxpehP4wx+Ww0cYgb7lFFtgY8BATgGLZY
         EG4JYQRmY8OQxJ6GkTBvJF9qTfDWdLp+0VkmiEXNpzv/jcTUmGWXcfn/z75T12DPs1Ye
         ULiaqmdMoWmrQJGuDoTsCPvjkWubRnbleRaqOMG+V/Sh0Wm/pVj/dfb+p4UzAf5XRvhG
         N3i0lPNKcrOfSLalonEqkAAJc/Ee1vCH31aLsOiuMvoJ55nT9gXEVxzW5FHosymPYhhh
         30Vg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501470; x=1708106270;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=X8dA/oVMrRThJ+rwgG6q9S6a3f0WD1rU2/Q4oEE47Ko=;
        b=tK0apS8Uo1XQOnLTbMP9Bvzfvb2cUMGJi9Mi8WgD1NH1BHu5wBIaG9q7OU0PbCr574
         FvzeRrfL+Jq2sxmJFGhAY7nXqN9D2gU+OJeY2uSU3jv8yz+BseuST4xabHNcSS+F2HMV
         VqUdOUGGyKrILNvAGp30A81hX7mDG+fYfGVsLHInBJXu2iqMqu1QNaiVkAkdomZbib26
         tXuXGQuUB+hgrl+J4Noec7RVyFNSgR/n5zd3BrNGsRjonMdF7ToaJEOa936HxTTlQfqk
         edBCelnjB+yOy87jgkXxPCrnwTWX/zZng8ImHsnA0vnuX2itviL2T0tFg8x0mgwl3bnU
         apRA==
X-Forwarded-Encrypted: i=1; AJvYcCXMQfOjyybWtt68rWFY9mixUqbRk42RnB50fT0yjgq0MZI4Et2Vbw9yFC1JAXgpc1SH3Kxc2uPZLEgnzos+K4527+8bvELV4EB34b2aBo4=
X-Gm-Message-State: AOJu0YwUGiUrGzTybIHcT80hxN5Sy/GZccUq0DmCUtsR/eGw+xP06PkW
	jcHqr+cU5u4AFqbRN9/P7xAnspaPYw1TsP6IME9akToOGJNPkYLhf/ZCL4Yo
X-Google-Smtp-Source: AGHT+IH45wM1JKOEWOemDFLbi1dbLAoCuLTw92E5wJpGOoifnT/kiBRtA7ZmKMB4At6bKr2YfIivPQ==
X-Received: by 2002:a17:906:3589:b0:a3c:1111:4683 with SMTP id o9-20020a170906358900b00a3c11114683mr604238ejb.11.1707501470365;
        Fri, 09 Feb 2024 09:57:50 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW9PS5yNZlk5xlqoUR2iWdLtTds/Xvc+8UkF8ZwhLbR+NLy54yVGgMQD5zQyljpNowP7+ai7+vLyLe/C3u805CJZCGKrcdFmA1/Pgzu7nKJ3/zK9+ceJ5viYGdb3hoYeEr+eJONb0NIMyg5k4/02Qk0ewe9XWWiVGRH5K4KxHaW0944FzXUsA==
Message-ID: <b9e334235f4caf391521019b3e65c671246d1f12.camel@gmail.com>
Subject: Re: Xen 4.19 release status tracking list
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
Cc: committers@xenproject.org, community.manager@xenproject.org, Kelly Choi
	 <kelly.choi@cloud.com>
Date: Fri, 09 Feb 2024 18:57:49 +0100
In-Reply-To: <e18cf10d-0531-4b11-8ead-a65c853a7a90@xen.org>
References: 
	<CAMacjJzio2EDwudCUHoL6psKYztkHZOWwLNDknATvpN6Y0im=Q@mail.gmail.com>
	 <e18cf10d-0531-4b11-8ead-a65c853a7a90@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Thu, 2024-02-08 at 16:58 +0000, Julien Grall wrote:
> Hi Oleksii,
Hi Julien,

>=20
> On 05/02/2024 19:23, Oleksii Kurochko wrote:
> > Hello everyone,
> >=20
> > I would like to share with you a list for status tracking based on
> > Xen ML:
> >=20
> > Arm:
> > =C2=A0=C2=A0 * [PATCH v5 00/13] Arm cache coloring [
> > https://lore.kernel.org/xen-devel/20240102095138.17933-1-carlo.nonato@m=
inervasys.tech/
> > ]
> > =C2=A0=C2=A0 * [PATCH v1 repost 0/4] xen/arm64: Rework the MMU-off code
> > (idmap)
> > so it is self-contained [
> > https://lore.kernel.org/xen-devel/20240116143709.86584-1-julien@xen.org=
/
> > ]
>=20
> This has been committed. You can mark it as resolved.
Great! Thanks for the update!

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 17:58:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 17:58:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678777.1056283 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYV93-0005Zl-I1; Fri, 09 Feb 2024 17:58:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678777.1056283; Fri, 09 Feb 2024 17:58:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYV93-0005Zc-Ev; Fri, 09 Feb 2024 17:58:37 +0000
Received: by outflank-mailman (input) for mailman id 678777;
 Fri, 09 Feb 2024 17:58:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYV92-0005ZO-L5
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 17:58:36 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d7bf9aa8-c774-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 18:58:34 +0100 (CET)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a383016f428so137429566b.2
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 09:58:34 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 ty15-20020a170907c70f00b00a358b6242fcsm966999ejc.114.2024.02.09.09.58.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 09:58:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7bf9aa8-c774-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501514; x=1708106314; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=2lrkJ9V+mE1GUJ7aII/vGn7VoenhC2UAArd0NmS/noQ=;
        b=bZCp6P9EgGhWltelv1EW481bmqBLxnOIgPSaJam2HoEi1OOX16qkgcrhu/tAn57f7O
         5ivrYGmrpMgWcVk41+pw4j3Tguw64IjZ9Q3cI0VRtKUEzGGNPmU0ouc7jj4R7q16Etui
         aRo0ldFaYI5J7sNi9ERDymfWjdDzKKwWTY3GmV+zNmj1hDkNuBxm2gqDwbjWe9f4hMzv
         CWPsa1FbjsptMcjdZm6Q9aF7vPXKLpb0kKjr2kAztDfVqiyLL4UJk+h7r9AuZzXeWkYk
         ARwl+miB19FyNPt31N3EuA+C31cu+cNoy2dAYwHpj3fl1UjsrmXjVlR+zKMVa9XIYjCF
         lwaw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501514; x=1708106314;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=2lrkJ9V+mE1GUJ7aII/vGn7VoenhC2UAArd0NmS/noQ=;
        b=V0m4583EvKKLaKRE/bhtqu01bvfNN0dRqEZC36iJgq9u72Y6PzYMGJGOISGZRmZLYf
         JjFbZ0Ec6agUKaq2aqvXHQA4S4EZqmSR7qeghe7TPSsKMf/2moF1vJ9Ucwyz4yqrTsJS
         DFsoEOvzrUAJMLKVCsv1MIvom97L524GTws+V8ercrWcdaJvtwL6woQYCvbTRbOBoUO+
         J/frfGhV9LWGw1zTj84FM00txy3GFlosCMcl3UDLeeFzhiwiWI/4YzqZ64epCxjJTKNp
         uucXzUUVdGqfOZHgy7WRtF1YhXZWpOchmenXU9TsPpTl2YLAMqJ/xm9ykWqNm+A5qNfh
         +C/g==
X-Gm-Message-State: AOJu0YyOjm/nF5e0MzevVwQ3Xz3/jROXxwnrad0iEjZErX9J/cmvOehY
	qkYDguYf3QmQSBA5N8m9MC+kXMSo5Uj3VcoTc5kstngDkPP0BEJxUSprwOeG
X-Google-Smtp-Source: AGHT+IG4/yVaVqTmTUdvj85AiCmykXPfvOVQfW7vtw+raXGRW3duqrmrkW9WeBZDTJwX3xHCQp2kmA==
X-Received: by 2002:a17:906:138e:b0:a37:2566:e248 with SMTP id f14-20020a170906138e00b00a372566e248mr2059724ejc.37.1707501513785;
        Fri, 09 Feb 2024 09:58:33 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUwj+GLoZzXGKtudUJE+LOLgvk5tKyT1bhuhpO0NzBiT0I0ed1BpOpkwHUUJMMq87ZqBtWZfCHn5kB2ke/v2lh3fuymnZ26krLQpoF+cJLXCEGqzuDwFbCBtLkNz7mKxmjJaRfLna5djXTwDeGDhZRYNt3TuOwxckZpg2ogxybFAz1gWR/HazKkToDwN8o/5jnGsq8TmrDz4ise2ZzaykMsPoTXKOysAftzFpk1oUsQqkG3dxEybwr/88+rWZvEMUYNoENaqrXrEzbD2N7gEMIeX/fhq5ant398QiTegeBJh5GOVXB9HlbjCRlVmPuPWhKflfs/h9DKS4WOoxWeASwbV0uVRJyjDUk3ZcAK9a2HD79L+6T1SE7Y
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>
Subject: [PATCH v2] xen/lib: introduce generic find next bit operations
Date: Fri,  9 Feb 2024 18:58:30 +0100
Message-ID: <fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

find-next-bit.c is common for Arm64, PPC and RISCV64,
so it is moved to xen/lib.

PPC has been transitioned to generic functions from find-next-bit.c
since it now shares the same implementation as the PPC-specific code.

The MISRA exclude list has been updated to verify
lib/find-next-bit.c instead of Arm's find_next_bit.c,
as Arm's find_next_bit.c has been relocated to xen/lib/.

Despite CONFIG_GENERIC_FIND_FIRST_BIT not currently being used in
Xen and being removed from the Linux kernel [1], it could theoretically
prove useful for Xen. This is because the Linux kernel transitioned
Arm to the generic version of find_first_bit() and
find_first_zero_bit() due to improvements [1] in both performance
and .text size.
It would be prudent to investigate whether [1] is applicable to Xen
as well and, if so, implement the necessary changes in a separate patch.

[1] https://lore.kernel.org/linux-arch/20211005054059.475634-5-yury.norov@gmail.com/

Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 docs/misra/exclude-list.json                  |   8 +-
 xen/arch/arm/arm64/lib/Makefile               |   2 +-
 xen/arch/arm/include/asm/arm64/bitops.h       |  46 +------
 xen/arch/ppc/include/asm/bitops.h             | 115 ------------------
 xen/include/xen/bitops.h                      |  48 ++++++++
 xen/lib/Makefile                              |   1 +
 .../find_next_bit.c => lib/find-next-bit.c}   |   0
 7 files changed, 55 insertions(+), 165 deletions(-)
 rename xen/{arch/arm/arm64/lib/find_next_bit.c => lib/find-next-bit.c} (100%)

diff --git a/docs/misra/exclude-list.json b/docs/misra/exclude-list.json
index 7971d0e70f..b28174d0eb 100644
--- a/docs/misra/exclude-list.json
+++ b/docs/misra/exclude-list.json
@@ -13,10 +13,6 @@
             "rel_path": "arch/arm/arm64/insn.c",
             "comment": "Imported on Linux, ignore for now"
         },
-        {
-            "rel_path": "arch/arm/arm64/lib/find_next_bit.c",
-            "comment": "Imported from Linux, ignore for now"
-        },
         {
             "rel_path": "arch/x86/acpi/boot.c",
             "comment": "Imported from Linux, ignore for now"
@@ -221,6 +217,10 @@
             "rel_path": "include/xen/decompress.h",
             "comment": "Imported from Linux, ignore for now"
         },
+        {
+            "rel_path": "lib/find-next-bit.c",
+            "comment": "Imported from Linux, ignore for now"
+        },
         {
             "rel_path": "lib/list-sort.c",
             "comment": "Imported from Linux, ignore for now"
diff --git a/xen/arch/arm/arm64/lib/Makefile b/xen/arch/arm/arm64/lib/Makefile
index 1b9c7a95e6..66cfac435a 100644
--- a/xen/arch/arm/arm64/lib/Makefile
+++ b/xen/arch/arm/arm64/lib/Makefile
@@ -1,4 +1,4 @@
 obj-y += memcpy.o memcmp.o memmove.o memset.o memchr.o
 obj-y += clear_page.o
-obj-y += bitops.o find_next_bit.o
+obj-y += bitops.o
 obj-y += strchr.o strcmp.o strlen.o strncmp.o strnlen.o strrchr.o
diff --git a/xen/arch/arm/include/asm/arm64/bitops.h b/xen/arch/arm/include/asm/arm64/bitops.h
index d85a49bca4..0efde29068 100644
--- a/xen/arch/arm/include/asm/arm64/bitops.h
+++ b/xen/arch/arm/include/asm/arm64/bitops.h
@@ -36,57 +36,13 @@ static inline int flsl(unsigned long x)
 
 /* Based on linux/include/asm-generic/bitops/find.h */
 
-#ifndef find_next_bit
-/**
- * find_next_bit - find the next set bit in a memory region
- * @addr: The address to base the search on
- * @offset: The bitnumber to start searching at
- * @size: The bitmap size in bits
- */
-extern unsigned long find_next_bit(const unsigned long *addr, unsigned long
-		size, unsigned long offset);
-#endif
-
-#ifndef find_next_zero_bit
-/**
- * find_next_zero_bit - find the next cleared bit in a memory region
- * @addr: The address to base the search on
- * @offset: The bitnumber to start searching at
- * @size: The bitmap size in bits
- */
-extern unsigned long find_next_zero_bit(const unsigned long *addr, unsigned
-		long size, unsigned long offset);
-#endif
-
-#ifdef CONFIG_GENERIC_FIND_FIRST_BIT
-
-/**
- * find_first_bit - find the first set bit in a memory region
- * @addr: The address to start the search at
- * @size: The maximum size to search
- *
- * Returns the bit number of the first set bit.
- */
-extern unsigned long find_first_bit(const unsigned long *addr,
-				    unsigned long size);
-
-/**
- * find_first_zero_bit - find the first cleared bit in a memory region
- * @addr: The address to start the search at
- * @size: The maximum size to search
- *
- * Returns the bit number of the first cleared bit.
- */
-extern unsigned long find_first_zero_bit(const unsigned long *addr,
-					 unsigned long size);
-#else /* CONFIG_GENERIC_FIND_FIRST_BIT */
+#ifndef CONFIG_GENERIC_FIND_FIRST_BIT
 
 #define find_first_bit(addr, size) find_next_bit((addr), (size), 0)
 #define find_first_zero_bit(addr, size) find_next_zero_bit((addr), (size), 0)
 
 #endif /* CONFIG_GENERIC_FIND_FIRST_BIT */
 
-
 #endif /* _ARM_ARM64_BITOPS_H */
 /*
  * Local variables:
diff --git a/xen/arch/ppc/include/asm/bitops.h b/xen/arch/ppc/include/asm/bitops.h
index 5e7f36c21d..5820b9ce7b 100644
--- a/xen/arch/ppc/include/asm/bitops.h
+++ b/xen/arch/ppc/include/asm/bitops.h
@@ -217,119 +217,4 @@ static always_inline unsigned long __ffs(unsigned long word)
  */
 #define find_first_set_bit(x) (ffsl(x) - 1)
 
-/*
- * Find the first set bit in a memory region.
- */
-static inline unsigned long find_first_bit(const unsigned long *addr,
-                                           unsigned long size)
-{
-    const unsigned long *p = addr;
-    unsigned long result = 0;
-    unsigned long tmp;
-
-    while ( size & ~(BITS_PER_LONG - 1) )
-    {
-        if ( (tmp = *(p++)) )
-            goto found;
-        result += BITS_PER_LONG;
-        size -= BITS_PER_LONG;
-    }
-    if ( !size )
-        return result;
-
-    tmp = (*p) & (~0UL >> (BITS_PER_LONG - size));
-    if ( tmp == 0UL )         /* Are any bits set? */
-        return result + size; /* Nope. */
- found:
-    return result + __ffs(tmp);
-}
-
-static inline unsigned long find_next_bit(const unsigned long *addr,
-                                          unsigned long size,
-                                          unsigned long offset)
-{
-    const unsigned long *p = addr + BITOP_WORD(offset);
-    unsigned long result = offset & ~(BITS_PER_LONG - 1);
-    unsigned long tmp;
-
-    if ( offset >= size )
-        return size;
-    size -= result;
-    offset %= BITS_PER_LONG;
-    if ( offset )
-    {
-        tmp = *(p++);
-        tmp &= (~0UL << offset);
-        if ( size < BITS_PER_LONG )
-            goto found_first;
-        if ( tmp )
-            goto found_middle;
-        size -= BITS_PER_LONG;
-        result += BITS_PER_LONG;
-    }
-    while ( size & ~(BITS_PER_LONG - 1) )
-    {
-        if ( (tmp = *(p++)) )
-            goto found_middle;
-        result += BITS_PER_LONG;
-        size -= BITS_PER_LONG;
-    }
-    if ( !size )
-        return result;
-    tmp = *p;
-
- found_first:
-    tmp &= (~0UL >> (BITS_PER_LONG - size));
-    if ( tmp == 0UL )         /* Are any bits set? */
-        return result + size; /* Nope. */
- found_middle:
-    return result + __ffs(tmp);
-}
-
-/*
- * This implementation of find_{first,next}_zero_bit was stolen from
- * Linus' asm-alpha/bitops.h.
- */
-static inline unsigned long find_next_zero_bit(const unsigned long *addr,
-                                               unsigned long size,
-                                               unsigned long offset)
-{
-    const unsigned long *p = addr + BITOP_WORD(offset);
-    unsigned long result = offset & ~(BITS_PER_LONG - 1);
-    unsigned long tmp;
-
-    if ( offset >= size )
-        return size;
-    size -= result;
-    offset %= BITS_PER_LONG;
-    if ( offset )
-    {
-        tmp = *(p++);
-        tmp |= ~0UL >> (BITS_PER_LONG - offset);
-        if ( size < BITS_PER_LONG )
-            goto found_first;
-        if ( ~tmp )
-            goto found_middle;
-        size -= BITS_PER_LONG;
-        result += BITS_PER_LONG;
-    }
-    while ( size & ~(BITS_PER_LONG - 1) )
-    {
-        if ( ~(tmp = *(p++)) )
-            goto found_middle;
-        result += BITS_PER_LONG;
-        size -= BITS_PER_LONG;
-    }
-    if ( !size )
-        return result;
-    tmp = *p;
-
- found_first:
-    tmp |= ~0UL << size;
-    if ( tmp == ~0UL )        /* Are any bits zero? */
-        return result + size; /* Nope. */
- found_middle:
-    return result + ffz(tmp);
-}
-
 #endif /* _ASM_PPC_BITOPS_H */
diff --git a/xen/include/xen/bitops.h b/xen/include/xen/bitops.h
index 2cb7892bcc..e3c5a4ccf3 100644
--- a/xen/include/xen/bitops.h
+++ b/xen/include/xen/bitops.h
@@ -105,6 +105,54 @@ static inline int generic_flsl(unsigned long x)
  */
 #include <asm/bitops.h>
 
+#ifndef find_next_bit
+/**
+ * find_next_bit - find the next set bit in a memory region
+ * @addr: The address to base the search on
+ * @offset: The bitnumber to start searching at
+ * @size: The bitmap size in bits
+ */
+extern unsigned long find_next_bit(const unsigned long *addr,
+                                   unsigned long size,
+                                   unsigned long offset);
+#endif
+
+#ifndef find_next_zero_bit
+/**
+ * find_next_zero_bit - find the next cleared bit in a memory region
+ * @addr: The address to base the search on
+ * @offset: The bitnumber to start searching at
+ * @size: The bitmap size in bits
+ */
+extern unsigned long find_next_zero_bit(const unsigned long *addr,
+                                        unsigned long size,
+                                        unsigned long offset);
+#endif
+
+#ifndef find_first_bit
+/**
+ * find_first_bit - find the first set bit in a memory region
+ * @addr: The address to start the search at
+ * @size: The maximum size to search
+ *
+ * Returns the bit number of the first set bit.
+ */
+extern unsigned long find_first_bit(const unsigned long *addr,
+                                    unsigned long size);
+#endif
+
+#ifndef find_first_zero_bit
+/**
+ * find_first_zero_bit - find the first cleared bit in a memory region
+ * @addr: The address to start the search at
+ * @size: The maximum size to search
+ *
+ * Returns the bit number of the first cleared bit.
+ */
+extern unsigned long find_first_zero_bit(const unsigned long *addr,
+                                         unsigned long size);
+#endif
+
 #if BITS_PER_LONG == 64
 # define fls64 flsl
 # define ffs64 ffsl
diff --git a/xen/lib/Makefile b/xen/lib/Makefile
index 2d9ebb945f..e63798e1d4 100644
--- a/xen/lib/Makefile
+++ b/xen/lib/Makefile
@@ -3,6 +3,7 @@ obj-$(CONFIG_X86) += x86/
 lib-y += bsearch.o
 lib-y += ctors.o
 lib-y += ctype.o
+lib-y += find-next-bit.o
 lib-y += list-sort.o
 lib-y += memchr.o
 lib-y += memchr_inv.o
diff --git a/xen/arch/arm/arm64/lib/find_next_bit.c b/xen/lib/find-next-bit.c
similarity index 100%
rename from xen/arch/arm/arm64/lib/find_next_bit.c
rename to xen/lib/find-next-bit.c
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 18:00:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 18:00:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678780.1056293 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBK-0007B7-1i; Fri, 09 Feb 2024 18:00:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678780.1056293; Fri, 09 Feb 2024 18:00:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBJ-0007B0-VO; Fri, 09 Feb 2024 18:00:57 +0000
Received: by outflank-mailman (input) for mailman id 678780;
 Fri, 09 Feb 2024 18:00:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYVBI-0007AX-P7
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 18:00:56 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2ada27f8-c775-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 19:00:54 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-5600c43caddso1548855a12.2
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 10:00:54 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 fd1-20020a056402388100b00561103f29fbsm984503edb.75.2024.02.09.10.00.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 10:00:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ada27f8-c775-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501653; x=1708106453; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Bs9ogOGLG/U26z7bEm15alnNocp6JCN0/JehCJi+Wgw=;
        b=gBBZPeAiP+kiEh30FVBtaKdDAuoDW/wb5u/Yl3zccLWPBTcyFuvcIzn1kPShYQcV5L
         QXvR5jTH0R40Yw2YHzz9kQbhQlP1vxThf+7+dpsA0HmU78jwnUcivAEtHLRzGS63DkPx
         88jDeGCWCHW7vSZh/sHWjurSH2DrLmhAnyJha0xZ6bBpqh24HgyIZicKLace36Fuwpmm
         +MSUJqVbQd83Rnf77yljBGTfPu/EWBfcO1+yBUVJW6VAdX5Z4bGWDXdV1Q+AxJbaFw2/
         UYosfHCGfWW6M6dt8xVsuFNqGbUa1BZq7iwkc4wuKB8s1YF9AnAbE5wmAtKxYDFodhYc
         I9Cg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501653; x=1708106453;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Bs9ogOGLG/U26z7bEm15alnNocp6JCN0/JehCJi+Wgw=;
        b=qL+BvMw5QN0Q43NwIZF60GS+JBDpGPxvjrpWqVuEEN2q9RVI4PhL3c/YPrqiGJp5Fh
         oMMzWp8mAiQFbBbAR4otaPVrZjm/kx0ZUGdipVCkpyixFnEAsJis7OroyPcEK79O6xNp
         bj21mhWqzC+zxOHoULFV5tdXCvAPvw/+psI/a0l55wZdK3lNj73+t6z6OEtmBrJlqIG+
         rPbZC0H+5ZFIgZltkrBd9ychPfN7CSShILtY71d+cClonPPNOY9Q/nLygGhV83UFdlRb
         kQHuGB1IVQHq8e46an6AbVp4ghrmdwXtsolJHmcLt69h2k/pifPCH6gN35Aa5DpTr/XW
         8tWw==
X-Gm-Message-State: AOJu0Ywp+tdBgCKuSrpz1hrLDlHOOfy5PTnrrQLNw1exccA3F1gT6CSj
	tVVaS4r09VmyXxT7UOnrlakUN0A1+aGLMlXXpbscGcuyv9DN4RoXSEulDug8
X-Google-Smtp-Source: AGHT+IF/prAYCCgRd01hVTel3l+zsf3uytU0uSrtVjBhByLiQLPt5hGw3OqXEjpzyK9BHHubiAm5JA==
X-Received: by 2002:a05:6402:5172:b0:55f:e2be:ec0e with SMTP id d18-20020a056402517200b0055fe2beec0emr1915143ede.1.1707501652539;
        Fri, 09 Feb 2024 10:00:52 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVvZgV6duVjqfdo8lft4AMLopEZ2qr4Yma8GPzb13TkJdRfdQvsUhogHBzMFLpZgw9KPvUbbgSTqZbb7zKp5Z0PTpN3v6yJPkVlDNSQi3Ozo4mRW5qJG3FBBhN0ZQ1mMqLPXavX3bVimPVw/NVBQHbqJl+3yAPuIX8w10JwVhu051BbhkwdWAmA7uohdieve/47Sr0OfjNZ8bhukCpHxw/aNZpGtM2n5L6YOWHNNLcX+k+WeyUBABLbCZ/gzBAzpiJWriSLBVHz1lhR7QkCu6EkL2ZT70rVyb75xeQiw3A1DAAnJXd9aRXuVsQTY1Jj0g9721J2cbSWu+fJ2ZvvL6dNQCNAjelhmV9Hpc+WwCw0LTAfb2aJOL2KGeiaSwrS+uMtyjNyFhWzGbAW/lQXwYiLUNumuKYS8ejFyRm/sJDzTUSURp0b7DcNlFQXrK+E5hcan8dUkJpGsa9NSo80SXUzwcV8M9m6loyFyt0+PrcRH84LrKB4lQ3mQa3AsPbKA/l4huy2dVvhoIQ5zPbIFnKZdzAShEao+Omyyq7M8f5bv3h1wRTrzpZG7JYcAzuovd5ITRgGY9O7uxvLIoR/sQc=
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v8 0/7]  Introduce generic headers
Date: Fri,  9 Feb 2024 19:00:42 +0100
Message-ID: <cover.1707499278.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Some headers are common between several architectures, so the current patch series
provide them.

Another one reason to have them as generic is a simplification of adding support
necessary to make a complete Xen build as it was/is being done in the patch series [1]
and [2].

Also, instead of providing generic/stub headers, it was used
"#ifdef CONFIG_* #include <asm/*.h> #endif" instead of providing empty headers.

This patch series is a pre-requisite for "Enable build of full Xen for RISC-V" [3].

[1] https://lore.kernel.org/xen-devel/cover.1694543103.git.sanastasio@raptorengineering.com/
[2] https://lore.kernel.org/xen-devel/cover.1692181079.git.oleksii.kurochko@gmail.com/
[3] https://lore.kernel.org/xen-devel/cover.1700761381.git.oleksii.kurochko@gmail.com/
---
Changes in V8:
 - Add Acked-by Acked-by: Tamas K Lengyel <tamas@tklengyel.com> for patches:
    [PATCH v7 4/7] xen/asm-generic: ifdef inclusion of <asm/mem_access.h>
    [PATCH v7 3/7] xen/asm-generic: introduce stub header monitor.h
 - Update the commit messages for patches:
    [PATCH v7 5/7] xen/asm-generic: introduce generic device.h
    [PATCH v7 6/7] xen/arm: switch Arm to use asm-generic/device.h
 - Drop #ifdef HAS_PCI in asm-generic/device.h to be consistent with other pci-related
   things in the header.
 - Code styles fixes.    
---
Changes in V7:
 - The following patches were dropped because of rebasing as they were merged
   to staging:
   - [PATCH v6 3/9] xen/asm-generic: introduce generic div64.h header
   - [PATCH v6 6/9] xen/asm-generic: introduce stub header softirq.h
   - [PATCH v6 7/9] xen: ifdef inclusion of <asm/grant_table.h> in <xen/grant_table.h>
   - [PATCH v6 8/9] xen/asm-generic: ifdef inclusion of <asm/mem_access.h>
- [PATCH v6 5/9] xen/asm-generic: introduce stub header numa.h was dropped becaus of
  the patch: "[PATCH v2] NUMA: no need for asm/numa.h when !NUMA"
- Drop definition of arch_monitor_domctl_event for PPC.
- define arch_monitor_domctl_event in asm-generic/monitor.h.
- Add "define HAS_ARCH_MONITOR_DOMCTL_EVENT" in arm/.../monitor.h as it has arch
  specific implementation.
- keeping DEVICE_PCI_HOSTBRIDGE available for every build based on the reply:
    https://lore.kernel.org/xen-devel/926a5c12-7f02-42ec-92a8-1c82d060c710@xen.org/
- add comment above enum device_type.h with explanation about DEV_TYPE_MAX.        
- drop #ifdef HAS_PCI around "(device_get_class(dev) == DEVICE_PCI_HOSTBRIDGE))" in ARM code.
- separate patch  "[PATCH v6 9/9] xen/asm-generic: introduce generic device.h" into 3 patches.
---
Changes in V6:
 - Fix the build script to work properly with EXTRA_FIXED_RANDCONFIG.
 - Introduce separate randconfig yaml with fixed configs for RISC-V.
 - Disable CONFIG_GRANT_TABLE and CONFIG_MEM_ACCESS for PPC and RISC-V.
 - Remove change in Kconfig/common for CONFIG_GRANT_TABLE and CONFIG_MEM_ACCESS.
 - Rebase on top of the latest staging. 
---
Changes in V5:
 - Update the patch series message as patch related to delay.h was merged.
 - Rebase on top of staging because half of the patches of the patch series were
   merged to staging branch.
 - Add A-by for some of the patches.
 - Add "depends on X86 || Arm" for CONFIG_GRANT_TABLE and CONFIG_MEM_ACCESS to be
   sure it won't be turned on by randconfig in CI.
 - Partly switch Arm and PPC to asm-generic/monitor.h.
 - Some other minor changes
---
Changes in V4:
 - Update the cover letter message
 - Add Reviewed-by/Acked-by for patches:
    [PATCH v3 01/14] xen/asm-generic: introduce stub header paging.h
    [PATCH v3 03/14] xen/asm-generic: introduce generic hypercall.h
    [PATCH v3 04/14] xen/asm-generic: introduce generic header iocap.h
    [PATCH v3 05/14] xen/asm-generic: introduce stub header <asm/random.h>
    [PATCH v3 06/14] xen/asm-generic: introduce generic header percpu.h
    [PATCH v3 07/14] xen/asm-generic: introduce generalized hardirq.h
    [PATCH v3 08/14] xen/asm-generic: introduce generic div64.h header
    [PATCH v3 09/14] xen/asm-generic: introduce generic header altp2m.h
    [PATCH v3 10/14] xen/asm-generic: introduce stub header monitor.h
    [PATCH v3 11/14] xen/asm-generic: introduce stub header numa.h
    [PATCH v3 12/14] xen/asm-generic: introduce stub header softirq.h
 - Fix some code style and minor issues.
 - Use asm-generic version of device.h for Arm and PPC.
---
Changes in V3:
 - Update the commit message of the cover letter.
 - Drop the following patch as it can be arch-specific enough:
   * [PATCH v2 09/15] xen/asm-generic: introduce generic header smp.h
 - Drop correspondent arch specific headers and use asm-generic version of
   a header.
 - Back to the patch series patches:
   * xen: ifdef inclusion of <asm/grant_table.h> in <xen/grant_table.h>
   * xen/asm-generic: ifdef inclusion of <asm/mem_access.h>
---
Changes in V2:
 - Update the commit message of the cover letter.
 - Drop the following patches because they are arch-specific or was sent as a separate patch:
   - xen/asm-generic: introduce stub header event.h
	 - xen/asm-generic: introduce stub header spinlock.h
	 - [PATCH v1 03/29] xen/asm-generic: introduce stub header cpufeature.h
	 - [PATCH v1 07/29] xen/asm-generic: introduce stub header guest_atomics.h
	 - [PATCH v1 10/29] xen/asm-generic: introduce stub header iommu.h
	 - [PATCH v1 12/29] xen/asm-generic: introduce stub header pci.h because separate patch was sent [5]
	 - [PATCH v1 14/29] xen/asm-generic: introduce stub header setup.h
	 - [PATCH v1 15/29] xen/asm-generic: introduce stub header xenoprof.h because of [3].
	 - [PATCH v1 16/29] xen/asm-generic: introduce stub header flushtlb.h
	 - [PATCH v1 22/29] xen/asm-generic: introduce stub header delay.h because of [3]
	 - [PATCH v1 23/29] xen/asm-generic: introduce stub header domain.h
	 - [PATCH v1 24/29] xen/asm-generic: introduce stub header guest_access.h
	 - [PATCH v1 25/29] xen/asm-generic: introduce stub header irq.h ( probably not so generic as I expected, I'll back to it if it will be necessary in the future )
	 - [PATCH v1 28/29] xen/asm-generic: introduce stub header p2m.h ( probably not so generic as I expected, I'll back to it if it will be necessary in the future )
 - For the rest of the patches please look at changes for each patch separately.
---

Oleksii Kurochko (7):
  automation: ensure values in EXTRA_FIXED_RANDCONFIG are separated by
    new line
  automation: introduce fixed randconfig for RISC-V
  xen/asm-generic: introduce stub header monitor.h
  xen/asm-generic: ifdef inclusion of <asm/mem_access.h>
  xen/asm-generic: introduce generic device.h
  xen/arm: switch Arm to use asm-generic/device.h
  xen/ppc: switch PPC to use asm-generic/device.h

 automation/gitlab-ci/build.yaml               |   8 +-
 .../gitlab-ci/riscv-fixed-randconfig.yaml     |   7 ++
 automation/scripts/build                      |   6 +-
 xen/arch/arm/device.c                         |   5 +
 xen/arch/arm/domain_build.c                   |   2 +-
 xen/arch/arm/gic-v2.c                         |   4 +-
 xen/arch/arm/gic-v3.c                         |   6 +-
 xen/arch/arm/gic.c                            |   4 +-
 xen/arch/arm/include/asm/Makefile             |   1 +
 xen/arch/arm/include/asm/monitor.h            |  25 +---
 xen/arch/arm/p2m.c                            |   1 +
 xen/arch/arm/traps.c                          |   1 +
 xen/arch/ppc/configs/ppc64_defconfig          |   1 +
 xen/arch/ppc/include/asm/Makefile             |   1 +
 xen/arch/ppc/include/asm/device.h             |  53 --------
 xen/arch/ppc/include/asm/mem_access.h         |   5 -
 xen/arch/ppc/include/asm/monitor.h            |  28 +----
 xen/arch/ppc/stubs.c                          |   8 --
 xen/arch/riscv/configs/tiny64_defconfig       |   1 +
 .../asm => include/asm-generic}/device.h      | 117 +++++++++++-------
 xen/include/asm-generic/monitor.h             |  64 ++++++++++
 xen/include/xen/mem_access.h                  |   2 +
 22 files changed, 174 insertions(+), 176 deletions(-)
 create mode 100644 automation/gitlab-ci/riscv-fixed-randconfig.yaml
 delete mode 100644 xen/arch/ppc/include/asm/device.h
 delete mode 100644 xen/arch/ppc/include/asm/mem_access.h
 rename xen/{arch/arm/include/asm => include/asm-generic}/device.h (66%)
 create mode 100644 xen/include/asm-generic/monitor.h

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 18:00:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 18:00:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678781.1056301 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBK-0007Dd-CN; Fri, 09 Feb 2024 18:00:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678781.1056301; Fri, 09 Feb 2024 18:00:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBK-0007CX-5R; Fri, 09 Feb 2024 18:00:58 +0000
Received: by outflank-mailman (input) for mailman id 678781;
 Fri, 09 Feb 2024 18:00:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYVBJ-0007AX-06
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 18:00:57 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2b4587ba-c775-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 19:00:54 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-56001d49cc5so1574177a12.2
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 10:00:54 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 fd1-20020a056402388100b00561103f29fbsm984503edb.75.2024.02.09.10.00.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 10:00:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b4587ba-c775-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501653; x=1708106453; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8B0Pj3x9NNKyskGfWUDgYAd1f6cJ4ZaavTN0FBUQZB0=;
        b=hVJR1BqBFs+33LuACE+BtHluv4asgv0Ii/hfFeuXPieavH8xQ90r4/p7jQyitCAj6j
         03CXLm5zD7s89cCV0UmwNP9XAtfx8ufnXnFC52nSBR30mFGn8m81whnEia/KVJiKduL8
         f5rfYy4KSYx+G0m2bORWWEB8Y4uBgXv5LGt5FRvks+uKztwoKkt13nDoI3OyLBvqtFXQ
         yFAsFWVmouLEXwvIwTciPHBBVnAi5vIouo+bQ090GSzaU6QaNsO5TyLX90h2guN+65ym
         evHVe2C1N9wgVjknBhoQlHEHsbW2hRdHclGx7PKGLI79gJWEl3zYLMbq3/UBjTwi58kN
         +YNg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501653; x=1708106453;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8B0Pj3x9NNKyskGfWUDgYAd1f6cJ4ZaavTN0FBUQZB0=;
        b=nOu+JehDBDcdanz65c4oRk3obYkkzGhGTbf87JEfjjRO3nTPWNz4ftRe2R/QVtNNly
         MMDFoxhi8BnBx5Lb9XTek1ujAuEJe0H7gwxx08iW7O8m8fkN9IqKyQxhq+xrytjSUa78
         eAXKkErcVxWrOxdIj+d8jpN1yBwaixhw2UmOhCo4vzezIiuVp7luORAmRDuZHmQRerrJ
         UryMZGRIEhCsTuvTyKKJpVNU04FKSDb1zIkfW5J4pS6ELruCPq0w3Jk8EjNe6sQLUFEB
         1uI846sP4dbPxGR/xmKFYRdrFHW6FE5BohsyAskPDuYM8fizdbw4GWUyZCRKgrUfoVRD
         lBjw==
X-Gm-Message-State: AOJu0YyD15moGf6cFDTKKjvXS2WAKpevQsSqRFrGXgXvOXOggeJJTdV0
	dRE1MS2qzHEPpnI3sHe4e4Y/9hrDK7beNycWPcFfjwR5P9pjPCAWdlvxuVOy
X-Google-Smtp-Source: AGHT+IG7YVXB7f4PptTAFpGa8e2Ebe1L2b/BYhzaG1OzjD3gFAhpsghcd0eV447A5hEkZZ+cTgF6yw==
X-Received: by 2002:a05:6402:148a:b0:561:c35:dad7 with SMTP id e10-20020a056402148a00b005610c35dad7mr2005215edv.32.1707501653167;
        Fri, 09 Feb 2024 10:00:53 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVJq/ORgYEshu0agU36A7K2vaCtwYElOyG89uaAaZbIGprzWaCHQZSxuiaWXmvYNC53dDnvYqCoRA5D/DMmtg47GnJuanhYSEj3pySLOrY3r6CTI82SQ6TwgQ==
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v8 1/7] automation: ensure values in EXTRA_FIXED_RANDCONFIG are separated by new line
Date: Fri,  9 Feb 2024 19:00:43 +0100
Message-ID: <8b6f208f0b4327392cfddbfa6809117cc7686750.1707499278.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707499278.git.oleksii.kurochko@gmail.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Kconfig tool expects each configuration to be on a new line.

The current version of the build script puts all of ${EXTRA_FIXED_RANDCONFIG}
in a single line and configs are seperated by spaces.

As a result, only the first configuration in ${EXTRA_FIXED_RANDCONFIG} will
be used.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V8:
 - Nothing changed. Only rebase
---
Changes in V7:
 - Nothing changed. Only rebase
---
Changes in V6:
 - The patch was introduced in this version of patch series.
---
 automation/scripts/build | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/automation/scripts/build b/automation/scripts/build
index b3c71fb6fb..13b043923d 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -14,7 +14,7 @@ if [[ "${RANDCONFIG}" == "y" ]]; then
 
     # Append job-specific fixed configuration
     if [[ -n "${EXTRA_FIXED_RANDCONFIG}" ]]; then
-        echo "${EXTRA_FIXED_RANDCONFIG}" >> xen/tools/kconfig/allrandom.config
+        sed "s/ /\n/g" <<< "${EXTRA_FIXED_RANDCONFIG}" > xen/tools/kconfig/allrandom.config
     fi
 
     make -j$(nproc) -C xen KCONFIG_ALLCONFIG=tools/kconfig/allrandom.config randconfig
@@ -28,9 +28,11 @@ else
     echo "CONFIG_DEBUG=${debug}" >> xen/.config
 
     if [[ -n "${EXTRA_XEN_CONFIG}" ]]; then
-        echo "${EXTRA_XEN_CONFIG}" >> xen/.config
+        sed "s/ /\n/g" <<< "${EXTRA_XEN_CONFIG}" >> xen/.config
     fi
 
+    cat xen/.config
+
     make -j$(nproc) -C xen olddefconfig
 fi
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 18:00:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 18:00:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678782.1056314 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBL-0007eL-Hp; Fri, 09 Feb 2024 18:00:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678782.1056314; Fri, 09 Feb 2024 18:00:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBL-0007eE-DJ; Fri, 09 Feb 2024 18:00:59 +0000
Received: by outflank-mailman (input) for mailman id 678782;
 Fri, 09 Feb 2024 18:00:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYVBJ-0007AX-Nd
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 18:00:57 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2c0f6149-c775-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 19:00:56 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d066b82658so17478961fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 10:00:56 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 fd1-20020a056402388100b00561103f29fbsm984503edb.75.2024.02.09.10.00.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 10:00:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c0f6149-c775-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501655; x=1708106455; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2FAxJg0gmB5DWDjQyTT6C98jISHS9vZdHw6mxETNKmo=;
        b=E27mnS4bZ1BDoOq5d3Dg2Dc+ocZRlrmtvuoMKCToE/lbvm9r8W6BOGfH0UfZ49kbP5
         sl1/h+hGtvdc6CDaDKaqDbaZ7NdPS3eb2w/LAZLERLgS8QSHceUV7XEAno39EbwWAz/Y
         oI4R9NWNQ28FsDVZpOP/8mda5dUzM3DY+IZ+8g3RyYQHOWSncXAJSWIQoF4C8XgShI1h
         sQNrf+QP83I7Od+xiwsJtBkQg1TCTX1Hxm/29Ut7C+kOCf7BRh4ea7EOUDwhtin9UW+N
         yjcMwvkD5R5VIj7ukMlA5fMQqzBGvQZw4ua5tq7jQdqAL3AYwUKNIBeWVykP9K87IYTd
         kaRA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501655; x=1708106455;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=2FAxJg0gmB5DWDjQyTT6C98jISHS9vZdHw6mxETNKmo=;
        b=lZyKt/p0BxpuEU5RbmGQiFunTFlbUqlmOcm5xTxRbh9MTqAiWv0DS+luj4XzZC2tf3
         EdgI3PZkG41olls/66mR5RAsCEZkvWKB3qhRFesogHRulX9pbM/uD3EqR+nFVXD6ak7C
         azJTTZUD8BsVNBeFLntZ/gOCphlpaIGecY0vpN4Uico9skKnaw3gw5mSZGJkOwkOQLk/
         nZ+vHVKGQcKGViFiixWHtS3aokY7eaEMgyjHHlmxKBgDolNe4ccTyvt77UlKvGHfIhIy
         JAYzzU6IIZPoSBU+7E7V2ZyTWowFITkAKtIXGQvW9HpM5rXSGQH1/mxDVoHLpfrwhNqw
         z3LQ==
X-Gm-Message-State: AOJu0Yxs9r2AlDunwz+Qoejf2pYADzzyc8+5DctKUFdoM7ibYzc2xJU/
	5Oc2UKfJQg38hLLQHF6E2A2pagT9oFeyuc6/RzCkMBZeiBmuzUsQzmLYObX5
X-Google-Smtp-Source: AGHT+IFbSuQHrFD+pCaWosSAT51WKM2wswg64lILWQSiNhb+QTAe+Zz7P5j3ad0mIicMH3BJdo6Qeg==
X-Received: by 2002:a2e:9888:0:b0:2d0:a2ad:bb5 with SMTP id b8-20020a2e9888000000b002d0a2ad0bb5mr1691612ljj.44.1707501654857;
        Fri, 09 Feb 2024 10:00:54 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU2UNPXe/bQPGqfYVR+SWxZqR7Zi8dQvlVA1YR82f6V8MlZ84d6/8d410wm+zJH0xq9zzODG8LCTm+bzx6fbU7UQQRCXjVZ0tDE/77AZUqwDTDa5nH/MhdxWA==
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v8 2/7] automation: introduce fixed randconfig for RISC-V
Date: Fri,  9 Feb 2024 19:00:44 +0100
Message-ID: <3f662e3b4ded36d982ff04ed5cdfdd8e36f83594.1707499278.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707499278.git.oleksii.kurochko@gmail.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch introduces the file riscv-fixed-randconfig.yaml,
which includes all configurations that should be disabled for
randconfig builds.

Suggested-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
The patch were introduced after discussion in a topic:
 https://lore.kernel.org/xen-devel/cover.1701966261.git.oleksii.kurochko@gmail.com/
 ---
Changes in V8:
 - Nothing changed. Only rebase
---
Changes in V7:
 - Nothing changed. Only rebase
---
Changes in V6:
 - The patch was introduced in this version of patch series.
---
 automation/gitlab-ci/build.yaml                  | 8 ++++----
 automation/gitlab-ci/riscv-fixed-randconfig.yaml | 7 +++++++
 2 files changed, 11 insertions(+), 4 deletions(-)
 create mode 100644 automation/gitlab-ci/riscv-fixed-randconfig.yaml

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 6d2cb18b88..376eb17f9c 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -512,6 +512,8 @@ alpine-3.18-gcc-debug-arm64-boot-cpupools:
       CONFIG_BOOT_TIME_CPUPOOLS=y
 
 # RISC-V 64 cross-build
+include: 'automation/gitlab-ci/riscv-fixed-randconfig.yaml'
+
 archlinux-current-gcc-riscv64:
   extends: .gcc-riscv64-cross-build
   variables:
@@ -532,8 +534,7 @@ archlinux-current-gcc-riscv64-randconfig:
     CONTAINER: archlinux:current-riscv64
     KBUILD_DEFCONFIG: tiny64_defconfig
     RANDCONFIG: y
-    EXTRA_FIXED_RANDCONFIG:
-      CONFIG_COVERAGE=n
+    EXTRA_FIXED_RANDCONFIG: !reference [.riscv-fixed-randconfig, variables, EXTRA_FIXED_RANDCONFIG]
 
 archlinux-current-gcc-riscv64-debug-randconfig:
   extends: .gcc-riscv64-cross-build-debug
@@ -541,8 +542,7 @@ archlinux-current-gcc-riscv64-debug-randconfig:
     CONTAINER: archlinux:current-riscv64
     KBUILD_DEFCONFIG: tiny64_defconfig
     RANDCONFIG: y
-    EXTRA_FIXED_RANDCONFIG:
-      CONFIG_COVERAGE=n
+    EXTRA_FIXED_RANDCONFIG: !reference [.riscv-fixed-randconfig, variables, EXTRA_FIXED_RANDCONFIG]
 
 # Power cross-build
 debian-bullseye-gcc-ppc64le:
diff --git a/automation/gitlab-ci/riscv-fixed-randconfig.yaml b/automation/gitlab-ci/riscv-fixed-randconfig.yaml
new file mode 100644
index 0000000000..f1282b40c9
--- /dev/null
+++ b/automation/gitlab-ci/riscv-fixed-randconfig.yaml
@@ -0,0 +1,7 @@
+.riscv-fixed-randconfig:
+  variables:
+    EXTRA_FIXED_RANDCONFIG:
+      CONFIG_COVERAGE=n
+      CONFIG_EXPERT=y
+      CONFIG_GRANT_TABLE=n
+      CONFIG_MEM_ACCESS=n
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 18:01:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 18:01:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678783.1056318 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBL-0007hx-SI; Fri, 09 Feb 2024 18:00:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678783.1056318; Fri, 09 Feb 2024 18:00:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBL-0007h7-Mf; Fri, 09 Feb 2024 18:00:59 +0000
Received: by outflank-mailman (input) for mailman id 678783;
 Fri, 09 Feb 2024 18:00:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYVBK-0007AX-Nw
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 18:00:58 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2c7e2771-c775-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 19:00:56 +0100 (CET)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2d0e20b7375so5703001fa.2
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 10:00:56 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 fd1-20020a056402388100b00561103f29fbsm984503edb.75.2024.02.09.10.00.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 10:00:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c7e2771-c775-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501656; x=1708106456; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=dDTJW+iXXU/pmzqRlWGMRI2OkRV6YQ3UWt7y5c2dJ2o=;
        b=EIUakyFiLdSVq/sWfKoNBlnT7tFZchSqvfm2C7aEkZm3ki4EpOOlphHBBriNoXU9xq
         vUeQY21nqMsPeW3ZCSBR+ioBPMaD8jtati7RO1SX98q8DHH3F3nbmBZ43JXgUSHqTdPS
         A4Js4UHI4BLi0a72hMIze8jPxkFpLbgw3V4Zt+L+phOQKTjO8FE+RH8AiE18sSW061de
         3y2UPa43y7dWK8LbNvzsJVeDq4Ye+GCVyRr+UXZO+DwslV3v0qIUhyNPz/ClnN2HXth7
         /E2OaspXRfyriZ9O9WKzY3+ga8lfaUvO0ncOFC8CTIY2BENqRSQHQxSWtME9C4f0LJ54
         wJ6w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501656; x=1708106456;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=dDTJW+iXXU/pmzqRlWGMRI2OkRV6YQ3UWt7y5c2dJ2o=;
        b=YQMKYpEbLoPMTIB803B0X88CuAQsj6YRCxa0H6iiJ3kXo+Yzx8eISaBwZZi5qOUk3N
         EY/ecIuK3EYEe5xqJ+83wJRG569ykrqXfjfIklnGBp8VXgF8dPL0/JotA/48YWMu4fCY
         eprEPt2B8Mal78O6oA6bKAWHqF+KXEv4MbQ7gWijaT6j+QcCbtWh9VVhzfcdol3DRK0t
         Xi0Aw0sDO7kwaFX0XRi0gSRNBaBNtGcCF4xiqZUoBZCGwTcaRs9qD6835o8hGtVEWbHN
         60TN7xhShRhzYd8PRWXWf5Bi0op4UGjb9xbK9YzD880ddTJImzRB9SknyhuE5cygoaIz
         hN3Q==
X-Gm-Message-State: AOJu0YxfHzcGDflwiyXPDefhMDDudITX0agaXmsCVLu0PCqzjaeJkXKM
	78thgR7jqiHTQFzQMfvYcjqTYUsi+DH/fPy+QspeBsZtAE5Wf3PPokhi1iBo
X-Google-Smtp-Source: AGHT+IGFHZ+cyVVb2L4PXW3RObTBtjdUa76Y9tfpmFyQ15SpxFnJRcY7kMAciBx9g1Gmhpoh26N8PQ==
X-Received: by 2002:a2e:9b58:0:b0:2d0:9fac:bdff with SMTP id o24-20020a2e9b58000000b002d09facbdffmr1844374ljj.38.1707501655965;
        Fri, 09 Feb 2024 10:00:55 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUUH3gne00j+NV48jUiaBFjkD6wtBIFi6UDcL+hC2C9BHR0akOBxRpJk06JaF27iQgDmJYSoKiw271l71n38cPcehtHkDw4GZeN4/o0UZoZgiwYUkuGOc1SUnN3Stf2IaofQtbDzaPYd7Y6ZxGgICYhuMNzD3aaqZunBY9mfvY5oAliHqO16bNb1OPdX+e7TGhlvcPybwopcD4QWABy7Bhpib587TZh5KdDOdHo7SdTczil7JdsBtxzu58zcRsIzI2g5DZ4wsIUkg9MEwRiyRPbMDbRp/gKsuRTF0EMutMoCI6PGIoytQXrveu/NZc+Fle8nnXLnZbXbVJb+8GauwM48vLahZ/f5CmoNpnodJ2mPnnzlgziBc9jr0hYXijxMvOi
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v8 3/7] xen/asm-generic: introduce stub header monitor.h
Date: Fri,  9 Feb 2024 19:00:45 +0100
Message-ID: <84568b0c24a5ec96244f3f34537e9a148367facf.1707499278.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707499278.git.oleksii.kurochko@gmail.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The header is shared between several archs so it is
moved to asm-generic.

Switch partly Arm and PPC to asm-generic/monitor.h and only
arch_monitor_get_capabilities() left in arch-specific/monitor.h.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
---
Changes in V8:
 - Add Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
---
Changes in V7:
 - Drop definition of arch_monitor_domctl_event for PPC.
 - define arch_monitor_domctl_event in asm-generic/monitor.h.
 - add "define HAS_ARCH_MONITOR_DOMCTL_EVENT" in arm/.../monitor.h as it has arch specific implementation.
---
Changes in V6:
 - Rebase only.
---
Changes in V5:
  - Switched partly Arm and PPC to asm-generic monitor.h only
    arch_monitor_get_capabilities() left in arch-specific/monitor.h.
  - Updated the commit message.
---
Changes in V4:
 - Removed the double blank line.
 - Added Acked-by: Jan Beulich <jbeulich@suse.com>.
 - Update the commit message
---
Changes in V3:
 - Use forward-declaration of struct domain instead of " #include <xen/sched.h> ".
 - Add ' include <xen/errno.h> '
 - Drop PPC's monitor.h.
---
Changes in V2:
	- remove inclusion of "+#include <public/domctl.h>"
	- add "struct xen_domctl_monitor_op;"
	- remove one of SPDX tags.
---
 xen/arch/arm/include/asm/monitor.h | 25 +-----------
 xen/arch/ppc/include/asm/monitor.h | 28 +------------
 xen/arch/ppc/stubs.c               |  8 ----
 xen/include/asm-generic/monitor.h  | 64 ++++++++++++++++++++++++++++++
 4 files changed, 67 insertions(+), 58 deletions(-)
 create mode 100644 xen/include/asm-generic/monitor.h

diff --git a/xen/arch/arm/include/asm/monitor.h b/xen/arch/arm/include/asm/monitor.h
index 7567be66bd..77a3c1a36c 100644
--- a/xen/arch/arm/include/asm/monitor.h
+++ b/xen/arch/arm/include/asm/monitor.h
@@ -25,34 +25,13 @@
 #include <xen/sched.h>
 #include <public/domctl.h>
 
-static inline
-void arch_monitor_allow_userspace(struct domain *d, bool allow_userspace)
-{
-}
+#define HAS_ARCH_MONITOR_DOMCTL_EVENT
 
-static inline
-int arch_monitor_domctl_op(struct domain *d, struct xen_domctl_monitor_op *mop)
-{
-    /* No arch-specific monitor ops on ARM. */
-    return -EOPNOTSUPP;
-}
+#include <asm-generic/monitor.h>
 
 int arch_monitor_domctl_event(struct domain *d,
                               struct xen_domctl_monitor_op *mop);
 
-static inline
-int arch_monitor_init_domain(struct domain *d)
-{
-    /* No arch-specific domain initialization on ARM. */
-    return 0;
-}
-
-static inline
-void arch_monitor_cleanup_domain(struct domain *d)
-{
-    /* No arch-specific domain cleanup on ARM. */
-}
-
 static inline uint32_t arch_monitor_get_capabilities(struct domain *d)
 {
     uint32_t capabilities = 0;
diff --git a/xen/arch/ppc/include/asm/monitor.h b/xen/arch/ppc/include/asm/monitor.h
index e5b0282bf1..89000dacc6 100644
--- a/xen/arch/ppc/include/asm/monitor.h
+++ b/xen/arch/ppc/include/asm/monitor.h
@@ -6,33 +6,7 @@
 #include <public/domctl.h>
 #include <xen/errno.h>
 
-static inline
-void arch_monitor_allow_userspace(struct domain *d, bool allow_userspace)
-{
-}
-
-static inline
-int arch_monitor_domctl_op(struct domain *d, struct xen_domctl_monitor_op *mop)
-{
-    /* No arch-specific monitor ops on PPC. */
-    return -EOPNOTSUPP;
-}
-
-int arch_monitor_domctl_event(struct domain *d,
-                              struct xen_domctl_monitor_op *mop);
-
-static inline
-int arch_monitor_init_domain(struct domain *d)
-{
-    /* No arch-specific domain initialization on PPC. */
-    return 0;
-}
-
-static inline
-void arch_monitor_cleanup_domain(struct domain *d)
-{
-    /* No arch-specific domain cleanup on PPC. */
-}
+#include <asm-generic/monitor.h>
 
 static inline uint32_t arch_monitor_get_capabilities(struct domain *d)
 {
diff --git a/xen/arch/ppc/stubs.c b/xen/arch/ppc/stubs.c
index a96e45626d..da193839bd 100644
--- a/xen/arch/ppc/stubs.c
+++ b/xen/arch/ppc/stubs.c
@@ -95,14 +95,6 @@ void arch_get_domain_info(const struct domain *d,
     BUG_ON("unimplemented");
 }
 
-/* monitor.c */
-
-int arch_monitor_domctl_event(struct domain *d,
-                              struct xen_domctl_monitor_op *mop)
-{
-    BUG_ON("unimplemented");
-}
-
 /* smp.c */
 
 void arch_flush_tlb_mask(const cpumask_t *mask)
diff --git a/xen/include/asm-generic/monitor.h b/xen/include/asm-generic/monitor.h
new file mode 100644
index 0000000000..1ade289099
--- /dev/null
+++ b/xen/include/asm-generic/monitor.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * include/asm-generic/monitor.h
+ *
+ * Arch-specific monitor_op domctl handler.
+ *
+ * Copyright (c) 2015 Tamas K Lengyel (tamas@tklengyel.com)
+ * Copyright (c) 2016, Bitdefender S.R.L.
+ *
+ */
+
+#ifndef __ASM_GENERIC_MONITOR_H__
+#define __ASM_GENERIC_MONITOR_H__
+
+#include <xen/errno.h>
+#include <xen/lib.h>
+
+struct domain;
+struct xen_domctl_monitor_op;
+
+static inline
+void arch_monitor_allow_userspace(struct domain *d, bool allow_userspace)
+{
+}
+
+static inline
+int arch_monitor_domctl_op(struct domain *d, struct xen_domctl_monitor_op *mop)
+{
+    /* No arch-specific monitor ops on GENERIC. */
+    return -EOPNOTSUPP;
+}
+
+#ifndef HAS_ARCH_MONITOR_DOMCTL_EVENT
+static inline
+int arch_monitor_domctl_event(struct domain *d,
+                              struct xen_domctl_monitor_op *mop)
+{
+    BUG_ON("unimplemented");
+}
+#endif
+
+static inline
+int arch_monitor_init_domain(struct domain *d)
+{
+    /* No arch-specific domain initialization on GENERIC. */
+    return 0;
+}
+
+static inline
+void arch_monitor_cleanup_domain(struct domain *d)
+{
+    /* No arch-specific domain cleanup on GENERIC. */
+}
+
+#endif /* __ASM_GENERIC_MONITOR_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: BSD
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 18:01:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 18:01:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678784.1056326 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBM-0007rJ-Cj; Fri, 09 Feb 2024 18:01:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678784.1056326; Fri, 09 Feb 2024 18:01:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBM-0007mT-32; Fri, 09 Feb 2024 18:01:00 +0000
Received: by outflank-mailman (input) for mailman id 678784;
 Fri, 09 Feb 2024 18:00:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYVBL-0007U9-6Q
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 18:00:59 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2d6343f5-c775-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 19:00:58 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-561519f8698so603693a12.0
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 10:00:58 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 fd1-20020a056402388100b00561103f29fbsm984503edb.75.2024.02.09.10.00.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 10:00:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2d6343f5-c775-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501657; x=1708106457; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=enFGeu4zcdTjkPATANnCevJL8MNitOguXGYXr6w8aDE=;
        b=lWzkJE1/eBHbrk0JVrrYzlB1xrw/XxdQapp+y5EQvqxEd0mAf8LG3A8FS2kgfG5Cgl
         bnL+Dsetmn+1YXbRYTbDAQWaSYGryAkS16pNiTi8tihKp3pdAwFI+B6x5GW5Ohk+k5AD
         roOZqBJJJ/CcGs+P2BcT8A1Wz+wm2Gb6bdQfOd9Mkk9NHAufGmEsMwxp6x9cm187haQ3
         Oqpi9Xw+M+mDdTsIAxgVAPUQZSPs6M/D0z0XeV7MpeDk2+1f9dSbDd0WRhAXsffUUFaA
         r1UBZzirKlpQ3ASTZW8P2f7Z4aTvpFDsPcvi67P2hOesghVDfavvfe83n7rp1txlFdDD
         cPcg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501657; x=1708106457;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=enFGeu4zcdTjkPATANnCevJL8MNitOguXGYXr6w8aDE=;
        b=OOE7/phfUcHi4HBqoS5K4AePa7q2ndB/1Bkv5M3RLP0OsRP/NcJcFw6lRFuMiaztY8
         ZuTs64A5NuqJNl7AoSBj+S1Ce4aEK2+UuvMVmmw0o7t6EdmIJrwXBDKVoq14azlHM6g0
         mjmgMeia+gWwYpYVht8k4FifJKz4VakdWkGwKfKk4F15kE+Kuy04pdqwsSiBisw6mNWe
         dVITSgohStnCxA3XYYbORbg2GOTIal7tY9NchKZc06ETSg8q8ZhbziJwdzI/N/Fq+R1w
         G1g1PlvuT4rUe82vVBXZkniz1ZZdttTFwwwDu/xUn08RoRBJ4bZChKnWAqEN2WKO4BPe
         IQ4w==
X-Gm-Message-State: AOJu0Ywfj0md9IaZeQEN5DJIrLArrYqo++/KRgCgi2YdqCU2tqBef2oo
	RG+SKKKWWnvixQOYh8ZvH9IhSMlAkDcRkx+JeghWm/lY/FhHUus/WKp6p6fY
X-Google-Smtp-Source: AGHT+IFYFhz9toHU1yUGZQf/W/6R7wMN3qe/PwM2zpmugwrVd6BkNnTFjWyNy6TK0+lm/sJAdQDYyw==
X-Received: by 2002:aa7:d4d5:0:b0:560:269c:214f with SMTP id t21-20020aa7d4d5000000b00560269c214fmr2020823edr.19.1707501657084;
        Fri, 09 Feb 2024 10:00:57 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW020EtyrHen7sM4ysquTcqWN2lu/YXqgBtqrhn12PZMbnUApXKGPJoyceKy8+JP5xggN32sxpKIJEa2WEjmkXghB8tQrmvNfk97NjuvDlc9sUsFrW5TqpIekjRRltkgJANojZC+d/M8AFo7eCqEP5CwWt9BJSswaDSGtiy7ZxV6eHW8XMYJzlw5Y+K29XngggJQlay437ulOOnMxbnPqZt7N6NgIWwAq7MJqo7tBZBy7VjXwYOlB0h8pepc391WS6ErYNyBZkaDkquAGt6bYOm21mabVakYwSLfGsrz+2/z/8K776zOaf2+BZPDZvwlHGr+uyyVcGC83tTnSxGxPVKHCp3mphuyUm1Ikiu8OQyQG//CtSndxmzts1LTJUJh1lcpCvATu4zNFESpWU/d54AB99fnRbwPZk/1Dhc68hDUChAvrKotRg4xBzyjmEm0tzNZHarxgMGXm7tz6BIur6wxNng4O+2Qv8JMpY=
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v8 4/7] xen/asm-generic: ifdef inclusion of <asm/mem_access.h>
Date: Fri,  9 Feb 2024 19:00:46 +0100
Message-ID: <e1927bde3fa27a934e8ccaa8969b38e8a5dff52e.1707499278.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707499278.git.oleksii.kurochko@gmail.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

ifdefing inclusion of <asm/mem_access.h> in <xen/mem_access.h>
allows to avoid generation of empty <asm/mem_access.h> header
for the case when !CONFIG_MEM_ACCESS.

For Arm it was explicitly added inclusion of <asm/mem_access.h> for p2m.c
and traps.c because they require some functions from <asm/mem_access.h> which
aren't available in case of !CONFIG_MEM_ACCESS.

Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
---
Changes in V8:
 - Add Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
---
Changes in V7:
 - Nothing changed. Only rebase
---
Changes in V6:
 - Remove the way how CONFIG_MEM_ACCESS is disabled for PPC and RISC-V.
 - Disable the config in ppc64_defconfig and tiny64_defconfig (RISC-V).
---
Changes in V5:
 - Added dependencies for "Config MEM_ACCESS" to be sure that randconfig will not
   turn on the config.
---
Changes in V4:
 - Nothing changed. Only rebase.
---
Changes in V3:
 - Remove unnecessary comment.
---
 xen/arch/arm/p2m.c                      | 1 +
 xen/arch/arm/traps.c                    | 1 +
 xen/arch/ppc/configs/ppc64_defconfig    | 1 +
 xen/arch/ppc/include/asm/mem_access.h   | 5 -----
 xen/arch/riscv/configs/tiny64_defconfig | 1 +
 xen/include/xen/mem_access.h            | 2 ++
 6 files changed, 6 insertions(+), 5 deletions(-)
 delete mode 100644 xen/arch/ppc/include/asm/mem_access.h

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index b991b76ce4..2465c266e9 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -7,6 +7,7 @@
 #include <asm/event.h>
 #include <asm/flushtlb.h>
 #include <asm/guest_walk.h>
+#include <asm/mem_access.h>
 #include <asm/page.h>
 #include <asm/traps.h>
 
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 9c10e8f78c..8ddca643d4 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -35,6 +35,7 @@
 #include <asm/cpufeature.h>
 #include <asm/event.h>
 #include <asm/hsr.h>
+#include <asm/mem_access.h>
 #include <asm/mmio.h>
 #include <asm/regs.h>
 #include <asm/smccc.h>
diff --git a/xen/arch/ppc/configs/ppc64_defconfig b/xen/arch/ppc/configs/ppc64_defconfig
index f7cc075e45..48a053237a 100644
--- a/xen/arch/ppc/configs/ppc64_defconfig
+++ b/xen/arch/ppc/configs/ppc64_defconfig
@@ -6,6 +6,7 @@
 # CONFIG_HYPFS is not set
 # CONFIG_GRANT_TABLE is not set
 # CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
+# CONFIG_MEM_ACCESS is not set
 
 CONFIG_PPC64=y
 CONFIG_DEBUG=y
diff --git a/xen/arch/ppc/include/asm/mem_access.h b/xen/arch/ppc/include/asm/mem_access.h
deleted file mode 100644
index e7986dfdbd..0000000000
--- a/xen/arch/ppc/include/asm/mem_access.h
+++ /dev/null
@@ -1,5 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-#ifndef __ASM_PPC_MEM_ACCESS_H__
-#define __ASM_PPC_MEM_ACCESS_H__
-
-#endif /* __ASM_PPC_MEM_ACCESS_H__ */
diff --git a/xen/arch/riscv/configs/tiny64_defconfig b/xen/arch/riscv/configs/tiny64_defconfig
index 3c9a2ff941..09defe236b 100644
--- a/xen/arch/riscv/configs/tiny64_defconfig
+++ b/xen/arch/riscv/configs/tiny64_defconfig
@@ -6,6 +6,7 @@
 # CONFIG_HYPFS is not set
 # CONFIG_GRANT_TABLE is not set
 # CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
+# CONFIG_MEM_ACCESS is not set
 
 CONFIG_RISCV_64=y
 CONFIG_DEBUG=y
diff --git a/xen/include/xen/mem_access.h b/xen/include/xen/mem_access.h
index 4e4811680d..87d93b31f6 100644
--- a/xen/include/xen/mem_access.h
+++ b/xen/include/xen/mem_access.h
@@ -33,7 +33,9 @@
  */
 struct vm_event_st;
 
+#ifdef CONFIG_MEM_ACCESS
 #include <asm/mem_access.h>
+#endif
 
 /*
  * Additional access types, which are used to further restrict
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 18:01:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 18:01:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678785.1056343 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBN-0008O3-U8; Fri, 09 Feb 2024 18:01:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678785.1056343; Fri, 09 Feb 2024 18:01:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBN-0008Nq-QM; Fri, 09 Feb 2024 18:01:01 +0000
Received: by outflank-mailman (input) for mailman id 678785;
 Fri, 09 Feb 2024 18:01:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYVBM-0007AX-Nh
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 18:01:00 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2ddff2de-c775-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 19:00:59 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-55a5e7fa471so1616253a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 10:00:59 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 fd1-20020a056402388100b00561103f29fbsm984503edb.75.2024.02.09.10.00.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 10:00:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ddff2de-c775-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501658; x=1708106458; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vOVmqMl/hPpGvSwcX+N7nK/RY+xiJRv2+tQc7IlhjtY=;
        b=Idrm/AYIkfpGgJW6TYYHdiw/pDE3ZYx0GrWFGdULZyEfk9nLF5hhkl9EAK/prfXFQm
         hMcB1X/GPV7CqTmIpndbaPL5qRcyISWW0UaxaAUcHh63drVZpf4yd312kGnY2iZM7v8x
         sw4CpKNl4cH2f6iEQ4m54AdMEbx4KD5QmmYjVw8qGTllQFHzivUtk6L7BSofHP3atlzX
         tzc8Tz0Zmj8OUtrm/jEFR8vF60N4hZ/XWenS2yKET5N19P826V6jVlg6QfM6GcC3rZUp
         AJ+dZGpPT7803us1XDg3jRTsu0jkaLllxt8C6/hsS02Wr9Vr2inJx+XEr8mjrbZn3ftR
         DpOA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501658; x=1708106458;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=vOVmqMl/hPpGvSwcX+N7nK/RY+xiJRv2+tQc7IlhjtY=;
        b=W6g5Jv3ysciWJEAvNxq3SMYW0zgsDuDq3Q8VwDk65yCUtp7ZWetQhItx0+pqDGhGhm
         VR4zWFFePWAx5YPX4TGMgppjYh5Z3UGMI5ERg7LBXcz2T+DsHleGgzZWVjoXOPMJqYgW
         ZqXtBw+IPo5Qke56At6ynIsUbYGE7lE/7KctuNFp6rDjwe3FndfT7NHt+6b0A2ehOkax
         /R5ETEq6gRnj7PZcd9ElAVnAzIPku3DGASSrtZRQXABFrtJdc3kLFhL/Z65lp3dM1rOV
         /gopH4hNcNH/ToIoqoG0PGBrVFU4tTmxyqFQ5iCSRYxdEnYf7YOH84wK3MABj2exQGAa
         bRJQ==
X-Gm-Message-State: AOJu0YyEymcsMQ+zsqv85zEj1PeycLBOoY7h6t+rfM8O/BvT6tU4m+2n
	FQ+oU4FN7AQ/B8z2hcuLNy40VZf+ui37wNKyKtiC6ZHn/kzPM0hIhRp32WG7
X-Google-Smtp-Source: AGHT+IGZNhtlYMhLN+J3mwCuT+VlPUB2Wjdy1BQurwwsaXJBlp4ljXSYf5RPXU8CDNNMBxOHwgV+6g==
X-Received: by 2002:a05:6402:784:b0:55f:ec52:73c4 with SMTP id d4-20020a056402078400b0055fec5273c4mr1966265edy.34.1707501658375;
        Fri, 09 Feb 2024 10:00:58 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXNBSTd3EyIu7gvwXGI3lavhZQHbrnmmpBNhyJw51rTjTslLUNeFbc8AGpni/IrW6EYgoT3AAP/JdD4/bZDkBci240carThCRmd4TrdB3kel+XrUwUAf/XI1l8uqTU4zV18I2qh1txcoAMFDHm2t5I0xn5+OTdXOLtgqDKa4GgaHgAXm4XNlegFjZeIbDGUJT+VL4TNHGt7Cm+kGKlsSK3q/RQ183fHfNMcFw==
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v8 5/7] xen/asm-generic: introduce generic device.h
Date: Fri,  9 Feb 2024 19:00:47 +0100
Message-ID: <4ecb5e5f4a1dbf585ac7a9ebe40b5fd8e4709182.1707499278.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707499278.git.oleksii.kurochko@gmail.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Arm, PPC and RISC-V introduce the same things in asm/device.h, so
generic device.h was introduced.
Arm's device.h was taken as a base with the following changes:
 - #ifdef ACPI related things.
 - Rename #ifdef guards.
 - Add SPDX tag.
 - #ifdef CONFIG_HAS_DEVICE_TREE related things.
 - #ifdef-ing iommu related things with CONFIG_HAS_PASSTHROUGH.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V8:
 - drop #ifdef HAS_PCI around DEV_PCI in enum device_type to be consistent with DEVICE_PCI_HOSTBRIDGE
 - drop the comment above enum device_type, after DEV_PCI is alaways present in enum device_type, so
   there is no need anymore in comment and DEV_TYPE_MAX.
 - drop #ifdef HAS_PCI around dev_is_pci() as DEV_PCI is alwayds define now.
 - rename args _name, _namestr, _class to name, namestr, class.
 - drop unnessary backslash after .class = class in defines ACPI_DEVICE_START() and DT_DEVICE_START()
 - update the commit message 
---
Changes in V7:
 - keeping DEVICE_PCI_HOSTBRIDGE available for every build based on the reply:
    https://lore.kernel.org/xen-devel/926a5c12-7f02-42ec-92a8-1c82d060c710@xen.org/
 - add comment above enum device_type.h with explanation about DEV_TYPE_MAX.        
 - separate patch  "[PATCH v6 9/9] xen/asm-generic: introduce generic device.h" into 3 patches.
---
Changes in V6:
 - Rebase only.
---
Changes in V5:
  - Removed generated file: xen/include/headers++.chk.new
  - Removed pointless #ifdef CONFIG_HAS_DEVICE_TREE ... #endif for PPC as
    CONFIG_HAS_DEVICE_TREE will be always used for PPC.
---
Changes in V4:
 - Updated the commit message
 - Switched Arm and PPC to asm-generic version of device.h
 - Replaced HAS_PCI with CONFIG_HAS_PCI
 - ifdef-ing iommu filed of dev_archdata struct with CONFIG_HAS_PASSTHROUGH
 - ifdef-ing iommu_fwspec of device struct with CONFIG_HAS_PASSTHROUGH
 - ifdef-ing DT related things with CONFIG_HAS_DEVICE_TREE
 - Updated the commit message ( remove a note with question about
   if device.h should be in asm-generic or not )
 - Replaced DEVICE_IC with DEVICE_INTERRUPT_CONTROLLER
 - Rationalized usage of CONFIG_HAS_* in device.h
 - Fixed indents for ACPI_DEVICE_START and ACPI_DEVICE_END
---
Changes in V3:
 - ifdef device tree related things.
 - update the commit message
---
Changes in V2:
	- take ( as common ) device.h from Arm as PPC and RISC-V use it as a base.
	- #ifdef PCI related things.
	- #ifdef ACPI related things.
	- rename DEVICE_GIC to DEVIC_IC.
	- rename #ifdef guards.
	- switch Arm and PPC to generic device.h
	- add SPDX tag
	- update the commit message
---
 xen/include/asm-generic/device.h | 149 +++++++++++++++++++++++++++++++
 1 file changed, 149 insertions(+)
 create mode 100644 xen/include/asm-generic/device.h

diff --git a/xen/include/asm-generic/device.h b/xen/include/asm-generic/device.h
new file mode 100644
index 0000000000..6e56658271
--- /dev/null
+++ b/xen/include/asm-generic/device.h
@@ -0,0 +1,149 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_GENERIC_DEVICE_H__
+#define __ASM_GENERIC_DEVICE_H__
+
+#include <xen/stdbool.h>
+
+enum device_type
+{
+#ifdef CONFIG_HAS_DEVICE_TREE
+    DEV_DT,
+#endif
+    DEV_PCI
+};
+
+enum device_class
+{
+    DEVICE_SERIAL,
+    DEVICE_IOMMU,
+    DEVICE_INTERRUPT_CONTROLLER,
+    DEVICE_PCI_HOSTBRIDGE,
+    /* Use for error */
+    DEVICE_UNKNOWN,
+};
+
+struct dev_archdata {
+#ifdef CONFIG_HAS_PASSTHROUGH
+    void *iommu;    /* IOMMU private data */
+#endif
+};
+
+/* struct device - The basic device structure */
+struct device
+{
+    enum device_type type;
+#ifdef CONFIG_HAS_DEVICE_TREE
+    struct dt_device_node *of_node; /* Used by drivers imported from Linux */
+#endif
+    struct dev_archdata archdata;
+#ifdef CONFIG_HAS_PASSTHROUGH
+    struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
+#endif
+};
+
+typedef struct device device_t;
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+
+#include <xen/device_tree.h>
+
+#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
+
+/**
+ *  device_init - Initialize a device
+ *  @dev: device to initialize
+ *  @class: class of the device (serial, network...)
+ *  @data: specific data for initializing the device
+ *
+ *  Return 0 on success.
+ */
+int device_init(struct dt_device_node *dev, enum device_class class,
+                const void *data);
+
+/**
+ * device_get_type - Get the type of the device
+ * @dev: device to match
+ *
+ * Return the device type on success or DEVICE_ANY on failure
+ */
+enum device_class device_get_class(const struct dt_device_node *dev);
+
+#define DT_DEVICE_START(name_, namestr_, class_)            \
+static const struct device_desc __dev_desc_##name_ __used   \
+__section(".dev.info") = {                                  \
+    .name = namestr_,                                       \
+    .class = class_,
+
+#define DT_DEVICE_END                                       \
+};
+
+#else /* !CONFIG_HAS_DEVICE_TREE */
+#define dev_is_dt(dev) ((void)(dev), false)
+#endif /* CONFIG_HAS_DEVICE_TREE */
+
+#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
+
+struct device_desc {
+    /* Device name */
+    const char *name;
+    /* Device class */
+    enum device_class class;
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+
+    /* List of devices supported by this driver */
+    const struct dt_device_match *dt_match;
+    /*
+     * Device initialization.
+     *
+     * -EAGAIN is used to indicate that device probing is deferred.
+     */
+    int (*init)(struct dt_device_node *dev, const void *data);
+
+#endif
+};
+
+#ifdef CONFIG_ACPI
+
+struct acpi_device_desc {
+    /* Device name */
+    const char *name;
+    /* Device class */
+    enum device_class class;
+    /* type of device supported by the driver */
+    const int class_type;
+    /* Device initialization */
+    int (*init)(const void *data);
+};
+
+/**
+ *  acpi_device_init - Initialize a device
+ *  @class: class of the device (serial, network...)
+ *  @data: specific data for initializing the device
+ *
+ *  Return 0 on success.
+ */
+int acpi_device_init(enum device_class class,
+                     const void *data, int class_type);
+
+#define ACPI_DEVICE_START(name_, namestr_, class_)              \
+static const struct acpi_device_desc __dev_desc_##name_ __used  \
+__section(".adev.info") = {                                     \
+    .name = namestr_,                                           \
+    .class = class_,
+
+#define ACPI_DEVICE_END                                         \
+};
+
+#endif /* CONFIG_ACPI */
+
+#endif /* __ASM_GENERIC_DEVICE_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 18:01:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 18:01:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678786.1056350 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBO-0008UO-Fa; Fri, 09 Feb 2024 18:01:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678786.1056350; Fri, 09 Feb 2024 18:01:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBO-0008Sa-7q; Fri, 09 Feb 2024 18:01:02 +0000
Received: by outflank-mailman (input) for mailman id 678786;
 Fri, 09 Feb 2024 18:01:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYVBN-0007U9-6o
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 18:01:01 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2eb5f52c-c775-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 19:01:00 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-560c696ccffso4008420a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 10:01:00 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 fd1-20020a056402388100b00561103f29fbsm984503edb.75.2024.02.09.10.00.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 10:00:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2eb5f52c-c775-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501659; x=1708106459; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=JZxnCL12H+ZDFR5vLYR2Fa8H1VelnALrYXWhEz3aNg4=;
        b=N6FuLSU318UjDePVp4JLJu6GT/2hhh2apCe9NgTUVd0UjSSudeJCfua2CmdNr6i1ch
         KbXvwj1AQgetQUCZkDAG0klJPSLbe8VMkQ6PyRSLsOKXaETFsSWwyqWud3cfWwlyLZuA
         yZRlpScPN+VG7oFyycYizcKA91Tu3Jk5BJPMt2vgbitE7DStTNOMYDcvHUZgUjh26z18
         GwNoPG7f+G3oswwoDeCv82z2/3ldrNLRN1XddFc3BjYMQ8BcfcuQnt4DJO9MWVTN7CFk
         yNf5C+jlXAoxu5BWvTiCDpM9hJvv3ltpUPhwrdmd4xEvWfGtGxg1AKCjTYymAxX5miX9
         DKVQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501659; x=1708106459;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=JZxnCL12H+ZDFR5vLYR2Fa8H1VelnALrYXWhEz3aNg4=;
        b=qr09f4NVEjrYHmIqZvZ2iN+1MVhK4Lo5avln1cMJHYEWxJjcSWLBWQ+pQnEdAaB0bm
         zYZGEAF4Ac8EBP9T2P28Gr6v1ZBnnqNDGkxDWoFGCRAxLCTYVbVGvc5ZvU2c1hmAcddp
         JeI9IoSx663gtNV4qZx1zHUopGR+7ipq2CD/8NsVNzkpinkd+mkWYfiy6hU6M/k1TwxE
         x4UtznmAYZexqxLIsVwVDCifc+6MWgzWTYWKITjmltsh36KqVdXE7M/DwdqKnKUuiE4u
         FMWYlP64UmUUexR/kwohMvHRwf4RiJEY/51ZNjH2y9V//jFtSZyy76Ug6e8tcKpaH5Dx
         V5CQ==
X-Gm-Message-State: AOJu0YzZ7dMbn/YW4bSZY2D6NBTvFSvNKJvnAwaBZCM2frRLB+wG9L0J
	Aq95o79W/fuqcz/LGXD/mnyznr9GYBsUoohg4+3EJ3BR9UKUK+1HMwLJyRQS
X-Google-Smtp-Source: AGHT+IGW1gnVN8wsBtZuIzYKExcTmqqfEH1F9xffamnzp5f18zkBRyLMCfeGTgTtHd+FxfHWMDTdSg==
X-Received: by 2002:a05:6402:3410:b0:55f:5197:6314 with SMTP id k16-20020a056402341000b0055f51976314mr1764168edc.18.1707501659252;
        Fri, 09 Feb 2024 10:00:59 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVfF3SdaH0Hp4Jo9YeYcEvD7pVnwVeVJuG1jZmmx8lgUsmz+i3i1ZTu0HGjlH/cg2uiXiCeCn5o8z6wNqAIlnnvoIdXt0nyvj7wESsm3GQClePV3re9qKFxWtHwZuMhpSFtFSf2ebV1F13JWfqHQA/qVS0hjVqCIwG2F5fGkqp4G82gzINpKtK1CK38FVsHl7cC5JyO6TtvUrnqgD1xj4wfiiE=
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH v8 6/7] xen/arm: switch Arm to use asm-generic/device.h
Date: Fri,  9 Feb 2024 19:00:48 +0100
Message-ID: <ae256e804cbfd3c5d17ef9d17f100ebbf17c48fc.1707499278.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707499278.git.oleksii.kurochko@gmail.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The following changes were done as a result of switching to
asm-generic/device.h:
 * DEVICE_GIC was renamed to DEVICE_INTERRUPT_CONTROLLER according
   to definition of enum device_class in asm-generic/device.h.
 * acpi-related things in Arm code were guarded by #ifdef CONFIG_ACPI
   as struct acpi_device_desc was guarded in asm-generic, also functions
   acpi_device_init() was guarded too as they are using structure
   acpi_device_desc inside.
 * drop arm/include/asm/device.h and update arm/include/asm/Makefile
   to use asm-generic/device.h instead.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V8:
 - update the commit message
---
Changes in V7:
 - newly introduced patch which is based on the previous version of the patch:
     [PATCH v6 9/9] xen/asm-generic: introduce generic device.h
---
 xen/arch/arm/device.c             |   5 ++
 xen/arch/arm/domain_build.c       |   2 +-
 xen/arch/arm/gic-v2.c             |   4 +-
 xen/arch/arm/gic-v3.c             |   6 +-
 xen/arch/arm/gic.c                |   4 +-
 xen/arch/arm/include/asm/Makefile |   1 +
 xen/arch/arm/include/asm/device.h | 124 ------------------------------
 7 files changed, 14 insertions(+), 132 deletions(-)
 delete mode 100644 xen/arch/arm/include/asm/device.h

diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
index 1f631d3274..3e02cff008 100644
--- a/xen/arch/arm/device.c
+++ b/xen/arch/arm/device.c
@@ -16,7 +16,10 @@
 #include <xen/lib.h>
 
 extern const struct device_desc _sdevice[], _edevice[];
+
+#ifdef CONFIG_ACPI
 extern const struct acpi_device_desc _asdevice[], _aedevice[];
+#endif
 
 int __init device_init(struct dt_device_node *dev, enum device_class class,
                        const void *data)
@@ -45,6 +48,7 @@ int __init device_init(struct dt_device_node *dev, enum device_class class,
     return -EBADF;
 }
 
+#ifdef CONFIG_ACPI
 int __init acpi_device_init(enum device_class class, const void *data, int class_type)
 {
     const struct acpi_device_desc *desc;
@@ -61,6 +65,7 @@ int __init acpi_device_init(enum device_class class, const void *data, int class
 
     return -EBADF;
 }
+#endif
 
 enum device_class device_get_class(const struct dt_device_node *dev)
 {
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 46161848dc..085d88671e 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1651,7 +1651,7 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
      * Replace these nodes with our own. Note that the original may be
      * used_by DOMID_XEN so this check comes first.
      */
-    if ( device_get_class(node) == DEVICE_GIC )
+    if ( device_get_class(node) == DEVICE_INTERRUPT_CONTROLLER )
         return make_gic_node(d, kinfo->fdt, node);
     if ( dt_match_node(timer_matches, node) )
         return make_timer_node(kinfo);
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index cf392bfd1c..5d6885e389 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -1366,7 +1366,7 @@ static const struct dt_device_match gicv2_dt_match[] __initconst =
     { /* sentinel */ },
 };
 
-DT_DEVICE_START(gicv2, "GICv2", DEVICE_GIC)
+DT_DEVICE_START(gicv2, "GICv2", DEVICE_INTERRUPT_CONTROLLER)
         .dt_match = gicv2_dt_match,
         .init = gicv2_dt_preinit,
 DT_DEVICE_END
@@ -1381,7 +1381,7 @@ static int __init gicv2_acpi_preinit(const void *data)
     return 0;
 }
 
-ACPI_DEVICE_START(agicv2, "GICv2", DEVICE_GIC)
+ACPI_DEVICE_START(agicv2, "GICv2", DEVICE_INTERRUPT_CONTROLLER)
         .class_type = ACPI_MADT_GIC_VERSION_V2,
         .init = gicv2_acpi_preinit,
 ACPI_DEVICE_END
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index bf0e5c1b75..1cb1360606 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1909,7 +1909,7 @@ static const struct dt_device_match gicv3_dt_match[] __initconst =
     { /* sentinel */ },
 };
 
-DT_DEVICE_START(gicv3, "GICv3", DEVICE_GIC)
+DT_DEVICE_START(gicv3, "GICv3", DEVICE_INTERRUPT_CONTROLLER)
         .dt_match = gicv3_dt_match,
         .init = gicv3_dt_preinit,
 DT_DEVICE_END
@@ -1924,12 +1924,12 @@ static int __init gicv3_acpi_preinit(const void *data)
     return 0;
 }
 
-ACPI_DEVICE_START(agicv3, "GICv3", DEVICE_GIC)
+ACPI_DEVICE_START(agicv3, "GICv3", DEVICE_INTERRUPT_CONTROLLER)
         .class_type = ACPI_MADT_GIC_VERSION_V3,
         .init = gicv3_acpi_preinit,
 ACPI_DEVICE_END
 
-ACPI_DEVICE_START(agicv4, "GICv4", DEVICE_GIC)
+ACPI_DEVICE_START(agicv4, "GICv4", DEVICE_INTERRUPT_CONTROLLER)
         .class_type = ACPI_MADT_GIC_VERSION_V4,
         .init = gicv3_acpi_preinit,
 ACPI_DEVICE_END
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index d922ea67aa..b5a9c8266c 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -234,7 +234,7 @@ static void __init gic_dt_preinit(void)
         if ( !dt_get_parent(node) )
             continue;
 
-        rc = device_init(node, DEVICE_GIC, NULL);
+        rc = device_init(node, DEVICE_INTERRUPT_CONTROLLER, NULL);
         if ( !rc )
         {
             /* NOTE: Only one GIC is supported */
@@ -262,7 +262,7 @@ static void __init gic_acpi_preinit(void)
 
     dist = container_of(header, struct acpi_madt_generic_distributor, header);
 
-    if ( acpi_device_init(DEVICE_GIC, NULL, dist->version) )
+    if ( acpi_device_init(DEVICE_INTERRUPT_CONTROLLER, NULL, dist->version) )
         panic("Unable to find compatible GIC in the ACPI table\n");
 }
 #else
diff --git a/xen/arch/arm/include/asm/Makefile b/xen/arch/arm/include/asm/Makefile
index 505cb49090..4a4036c951 100644
--- a/xen/arch/arm/include/asm/Makefile
+++ b/xen/arch/arm/include/asm/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
 generic-y += altp2m.h
+generic-y += device.h
 generic-y += hardirq.h
 generic-y += iocap.h
 generic-y += paging.h
diff --git a/xen/arch/arm/include/asm/device.h b/xen/arch/arm/include/asm/device.h
deleted file mode 100644
index b5d451e087..0000000000
--- a/xen/arch/arm/include/asm/device.h
+++ /dev/null
@@ -1,124 +0,0 @@
-#ifndef __ASM_ARM_DEVICE_H
-#define __ASM_ARM_DEVICE_H
-
-enum device_type
-{
-    DEV_DT,
-    DEV_PCI,
-};
-
-struct dev_archdata {
-    void *iommu;    /* IOMMU private data */
-};
-
-/* struct device - The basic device structure */
-struct device
-{
-    enum device_type type;
-#ifdef CONFIG_HAS_DEVICE_TREE
-    struct dt_device_node *of_node; /* Used by drivers imported from Linux */
-#endif
-    struct dev_archdata archdata;
-    struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
-};
-
-typedef struct device device_t;
-
-#include <xen/device_tree.h>
-
-#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
-#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
-
-enum device_class
-{
-    DEVICE_SERIAL,
-    DEVICE_IOMMU,
-    DEVICE_GIC,
-    DEVICE_PCI_HOSTBRIDGE,
-    /* Use for error */
-    DEVICE_UNKNOWN,
-};
-
-struct device_desc {
-    /* Device name */
-    const char *name;
-    /* Device class */
-    enum device_class class;
-    /* List of devices supported by this driver */
-    const struct dt_device_match *dt_match;
-    /*
-     * Device initialization.
-     *
-     * -EAGAIN is used to indicate that device probing is deferred.
-     */
-    int (*init)(struct dt_device_node *dev, const void *data);
-};
-
-struct acpi_device_desc {
-    /* Device name */
-    const char *name;
-    /* Device class */
-    enum device_class class;
-    /* type of device supported by the driver */
-    const int class_type;
-    /* Device initialization */
-    int (*init)(const void *data);
-};
-
-/**
- *  acpi_device_init - Initialize a device
- *  @class: class of the device (serial, network...)
- *  @data: specific data for initializing the device
- *
- *  Return 0 on success.
- */
-int acpi_device_init(enum device_class class,
-                     const void *data, int class_type);
-
-/**
- *  device_init - Initialize a device
- *  @dev: device to initialize
- *  @class: class of the device (serial, network...)
- *  @data: specific data for initializing the device
- *
- *  Return 0 on success.
- */
-int device_init(struct dt_device_node *dev, enum device_class class,
-                const void *data);
-
-/**
- * device_get_type - Get the type of the device
- * @dev: device to match
- *
- * Return the device type on success or DEVICE_ANY on failure
- */
-enum device_class device_get_class(const struct dt_device_node *dev);
-
-#define DT_DEVICE_START(_name, _namestr, _class)                    \
-static const struct device_desc __dev_desc_##_name __used           \
-__section(".dev.info") = {                                          \
-    .name = _namestr,                                               \
-    .class = _class,                                                \
-
-#define DT_DEVICE_END                                               \
-};
-
-#define ACPI_DEVICE_START(_name, _namestr, _class)                    \
-static const struct acpi_device_desc __dev_desc_##_name __used           \
-__section(".adev.info") = {                       \
-    .name = _namestr,                                               \
-    .class = _class,                                                \
-
-#define ACPI_DEVICE_END                                               \
-};
-
-#endif /* __ASM_ARM_DEVICE_H */
-
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * indent-tabs-mode: nil
- * End:
- */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 18:01:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 18:01:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678787.1056364 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBQ-0000XG-6L; Fri, 09 Feb 2024 18:01:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678787.1056364; Fri, 09 Feb 2024 18:01:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVBQ-0000W1-1a; Fri, 09 Feb 2024 18:01:04 +0000
Received: by outflank-mailman (input) for mailman id 678787;
 Fri, 09 Feb 2024 18:01:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Qnec=JS=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rYVBO-0007AX-Rd
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 18:01:02 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2f2a28ce-c775-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 19:01:01 +0100 (CET)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-55a5e7fa471so1616287a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 09 Feb 2024 10:01:01 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 fd1-20020a056402388100b00561103f29fbsm984503edb.75.2024.02.09.10.00.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Feb 2024 10:00:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f2a28ce-c775-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707501660; x=1708106460; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7zSKxQR6kP5WdiwzAdOSjjJwHaQkMg7XYEJePcorDtE=;
        b=e7fpMLN5yYexyw2OYS91p/mZWEZbijXxpJP0vfpegj1CfsDG6mp6yBkmtU7YewqIdU
         xdstB7qVzx/qVzsUcLsp+I0q5NqkooccHXdkq/1X9hPKUh9vdkr9/IlbMRdZ3gQaV6qV
         olB1eNLLK0cXvTG7855zKxJDCuN3so4ImHE7+N97g/e+ovKrZ0bpbV5urPn+1ME03f6y
         bhEfW9R4QWEiSBUwcs/7v1nbo2cR+OKXq5NnUq4+/10L4pC3h4PWQxjBgT+zUytaGJBB
         nnbefpdzhFaLeU3AHIT0aochgPWAI2AAP9CwdQFxjvhhhAFK4fd0SKz9+3qKwcR15zQU
         gKTQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707501660; x=1708106460;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=7zSKxQR6kP5WdiwzAdOSjjJwHaQkMg7XYEJePcorDtE=;
        b=pI/KPXRGVvakDWIXIdH40bI8Ny9uGps9mH8nvtf4FMUjjb3eRlsg/yqX1SUC2RdTDZ
         YuWl/ZkfKhSVi2HrSbGMRbb2X53liIGklXVgGy3gZlDiY+QQGSQBSZ2Z5U/XB9d5E4Ku
         RHVKz0FqU5WYATZXcWfPG+KWQOhyyrZlPko3iucbeXJtYNB6eG0nER9Q5TM+hGjxrgm0
         q6cb/A+reMxtVKV12OFxmAvkseGXP0y1QVnKI/KV/0fXb1gwcQUVqZ+BPoPqcUR77MRc
         JRgM3TeJfS2F5wlEUA4Wd+kRt+8elFje4XCweico93K0BPW3EGoJ5iV7+xYytekFUpNu
         OZcg==
X-Gm-Message-State: AOJu0YxoL5G/Z64mSMAGSyj1zSVzy15BWzBOjo4jh6qs5uHbErbRfItZ
	aNfbXdH1pE7VTnmD83+b2a2w1DMvvJ3OJHPoF/e97PUQEyPc/fpugsEROabU
X-Google-Smtp-Source: AGHT+IFc6L4aoReIcUQhz63N8frIOcSmyo05U0y2uM2pvWezG7stFnF+wJT2FYx1hW0trc4A8v2d5g==
X-Received: by 2002:aa7:c653:0:b0:560:1652:e7cb with SMTP id z19-20020aa7c653000000b005601652e7cbmr1823428edr.16.1707501660563;
        Fri, 09 Feb 2024 10:01:00 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCV2XfEhF2CITrZWAeUUDnRihfsVEoFhTeQGycPMEtkmXn9i7lvJyqFe8+PpuUYDgOTj1Tl9SGQGh72urX/KEKYbgmPzIYhb76sjP94Z7FALQg==
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>
Subject: [PATCH v8 7/7] xen/ppc: switch PPC to use asm-generic/device.h
Date: Fri,  9 Feb 2024 19:00:49 +0100
Message-ID: <a7741646285594dcd5e00eb1fb108539f95b4139.1707499278.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1707499278.git.oleksii.kurochko@gmail.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V7:
 - newly introduced patch which is based on the previous version of the patch:
     [PATCH v6 9/9] xen/asm-generic: introduce generic device.h
---

 xen/arch/ppc/include/asm/Makefile |  1 +
 xen/arch/ppc/include/asm/device.h | 53 -------------------------------
 2 files changed, 1 insertion(+), 53 deletions(-)
 delete mode 100644 xen/arch/ppc/include/asm/device.h

diff --git a/xen/arch/ppc/include/asm/Makefile b/xen/arch/ppc/include/asm/Makefile
index 3fd893f3e0..ced02e26ed 100644
--- a/xen/arch/ppc/include/asm/Makefile
+++ b/xen/arch/ppc/include/asm/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
 generic-y += altp2m.h
+generic-y += device.h
 generic-y += div64.h
 generic-y += hardirq.h
 generic-y += hypercall.h
diff --git a/xen/arch/ppc/include/asm/device.h b/xen/arch/ppc/include/asm/device.h
deleted file mode 100644
index 8253e61d51..0000000000
--- a/xen/arch/ppc/include/asm/device.h
+++ /dev/null
@@ -1,53 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-#ifndef __ASM_PPC_DEVICE_H__
-#define __ASM_PPC_DEVICE_H__
-
-enum device_type
-{
-    DEV_DT,
-    DEV_PCI,
-};
-
-struct device {
-    enum device_type type;
-#ifdef CONFIG_HAS_DEVICE_TREE
-    struct dt_device_node *of_node; /* Used by drivers imported from Linux */
-#endif
-};
-
-enum device_class
-{
-    DEVICE_SERIAL,
-    DEVICE_IOMMU,
-    DEVICE_PCI_HOSTBRIDGE,
-    /* Use for error */
-    DEVICE_UNKNOWN,
-};
-
-struct device_desc {
-    /* Device name */
-    const char *name;
-    /* Device class */
-    enum device_class class;
-    /* List of devices supported by this driver */
-    const struct dt_device_match *dt_match;
-    /*
-     * Device initialization.
-     *
-     * -EAGAIN is used to indicate that device probing is deferred.
-     */
-    int (*init)(struct dt_device_node *dev, const void *data);
-};
-
-typedef struct device device_t;
-
-#define DT_DEVICE_START(name_, namestr_, class_)                    \
-static const struct device_desc __dev_desc_##name_ __used           \
-__section(".dev.info") = {                                          \
-    .name = namestr_,                                               \
-    .class = class_,                                                \
-
-#define DT_DEVICE_END                                               \
-};
-
-#endif /* __ASM_PPC_DEVICE_H__ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 18:02:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 18:02:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678798.1056373 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVD2-0002rS-GK; Fri, 09 Feb 2024 18:02:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678798.1056373; Fri, 09 Feb 2024 18:02:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYVD2-0002rL-D3; Fri, 09 Feb 2024 18:02:44 +0000
Received: by outflank-mailman (input) for mailman id 678798;
 Fri, 09 Feb 2024 18:02:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=H7xO=JS=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rYVD1-0002r1-4N
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 18:02:43 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2060e.outbound.protection.outlook.com
 [2a01:111:f400:7e88::60e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6a3399a9-c775-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 19:02:41 +0100 (CET)
Received: from BYAPR11CA0056.namprd11.prod.outlook.com (2603:10b6:a03:80::33)
 by IA0PR12MB8713.namprd12.prod.outlook.com (2603:10b6:208:48e::7)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.14; Fri, 9 Feb
 2024 18:02:36 +0000
Received: from MWH0EPF000989EB.namprd02.prod.outlook.com
 (2603:10b6:a03:80:cafe::c8) by BYAPR11CA0056.outlook.office365.com
 (2603:10b6:a03:80::33) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.24 via Frontend
 Transport; Fri, 9 Feb 2024 18:02:35 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 MWH0EPF000989EB.mail.protection.outlook.com (10.167.241.138) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Fri, 9 Feb 2024 18:02:34 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 9 Feb
 2024 12:02:33 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Fri, 9 Feb
 2024 10:02:33 -0800
Received: from [172.23.114.18] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend
 Transport; Fri, 9 Feb 2024 12:02:31 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a3399a9-c775-11ee-8a4b-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=IzMZ1Ca2IYHlvrWIujB0hs9yCQj67lf3gOjhj6Fv7BaLnA0vQ7DNP/pNjk9IsfqpzlF8qFxKAeEkqA0Lt96kOr5auQaT91x2+6E0R2SOxCPk0NumWIAVmjNOlLmjhc8UjdEaNNXvU5D77zG0SLBTj6Tr3dleoreRy7eMRiqqwZUS1xYYhdJo83qm34nHiGil6YrsX6M8wxWrL1YqoggiXFVlkh8eUGdq32MX0Val0N8qy6PUTv3xGZMcGsGLu6ucEc4G/KJnWVKCYzSJv0qWv0EkpXNiTRJlLzs7iXHC1twDBG66v1lCQu7ItMzabXUM/dEs0+mdDB0x2ykFQhUM2A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=kEBTveNL/p7e41ml1pnFpKoFY8Pak/QF4WgXpYUVY7k=;
 b=eleNqGrAGim1zv+bxHNA7Zv1XQ5WtktW8AmBlT2wzNowt0v1f5rxw9j5ZilqOTVHHQVgUFg9RgIzInWFevcSnQj8wC1UMDFLOvDKyX+EIesVRsAHTcYzmD9544Lj7IxRZrzzJ1cENGqZkBQ8a7LuaQuiXdyQMiRkms1VDqJAkIV090+U6MA5rLBhlX17oY7GOdGjvmM+E9VkmqAyjjOZEXOhhlIJM1KlbAdsjYzykIJ2tKxSKQ1/D2KWZck2W0ypC5NdDIBYUFf91KSvUIY4w44BH3L4hQNKBGTiVLjMYxV0kPOwbftkXnlI2AmV5qxxIco9UR1EPZWYcnMG5lb+QQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kEBTveNL/p7e41ml1pnFpKoFY8Pak/QF4WgXpYUVY7k=;
 b=bx0f06QzMAziKc14xcQbeAWhVJ+97KYNQVFPomMLrqXzArxZVlhhT5iXzsxNX8bWKdCYWmRvhmOXn40Fn4zFL68w/EdQZPGXfaQakTk8aNx/hUF6Vs3swjyjBDto+HPTpNnnESWv8uSO/PpXxBess7RtEeWhDVidYgaOwu3D6sE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <9b4ce4c7-6f4e-47ec-b3a4-0b417cd2660c@amd.com>
Date: Fri, 9 Feb 2024 13:02:31 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC XEN PATCH v5 1/5] xen/vpci: Clear all vpci status of device
To: Jiqian Chen <Jiqian.Chen@amd.com>, <xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Anthony PERARD <anthony.perard@citrix.com>, Juergen
 Gross <jgross@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>,
	Huang Rui <Ray.Huang@amd.com>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-2-Jiqian.Chen@amd.com>
Content-Language: en-US
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <20240112061317.418658-2-Jiqian.Chen@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MWH0EPF000989EB:EE_|IA0PR12MB8713:EE_
X-MS-Office365-Filtering-Correlation-Id: 3062045a-09ac-4e2d-ffcc-08dc29994afa
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	pNfbM3qkI3JWWAFuJWPt00zmQYV3JHci1VH/0Ad67oEAPivQfw91RLA+Pkrxjemvu3GC/Pcl38I0EKneaGfWYePSTLmHq6Es0gzfxcNdODpF63Ah3butC1i+nMa6BdeqmxagNxnaeHDGRM+8E28lO6/1EY35JAwsIWHpe7K4O8wvTgDwePPG/nBy6gkggGVPsp6gTiAZujx+PVViSRdQAHWmseQFT0vn/6RG95z68aVLXcEdS/RxCY559lNYdYDC1ds8qI8wc9Pbve9G9s9C2gSgtf37LHkoUxrHU1NVWWzdP7lhygA6F64jbV1LRthjyemVad9/VQJQ7K4+pxgPQ0srT7S7uHzSiKQJLYISWi2NkiyKep45hOo2Mn+limfKVikRERrc5VLYLWg7++IYxEWnoumGZsj5aQUUm2UzM8IS3kUfVtusNLDJIi9mGzDn7FItOaOV6O94nGXC+vrj5QrylxhBFFivAADGbeN9yJcZ8znECHMUGcIbGnTX1T2xqsnh2OeBOQKVk7VAbc3neqcXOyQPdzYYqotuf0nQUflKxGLzSo1DFOoBAZuXjXupMOj5T+TnaLbudUTp6ewrdw==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(396003)(346002)(39860400002)(136003)(230922051799003)(64100799003)(186009)(82310400011)(451199024)(1800799012)(36840700001)(40470700004)(46966006)(4326008)(478600001)(53546011)(966005)(31686004)(41300700001)(5660300002)(7416002)(44832011)(2906002)(54906003)(110136005)(70586007)(70206006)(16576012)(316002)(8936002)(8676002)(336012)(426003)(83380400001)(26005)(31696002)(81166007)(82740400003)(356005)(86362001)(2616005)(36756003);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2024 18:02:34.2747
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3062045a-09ac-4e2d-ffcc-08dc29994afa
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MWH0EPF000989EB.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8713

On 1/12/24 01:13, Jiqian Chen wrote:
> When a device has been reset on dom0 side, the vpci on Xen
> side won't get notification, so the cached state in vpci is
> all out of date compare with the real device state.
> To solve that problem, add a new hypercall to clear all vpci
> device state. When the state of device is reset on dom0 side,
> dom0 can call this hypercall to notify vpci.
> 
> Co-developed-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>

Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

If you send another version, the RFC tag may be dropped.

One thing to keep an eye out for below (not requesting any changes).

> ---
> diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
> index 72ef277c4f8e..c6df2c6a9561 100644
> --- a/xen/drivers/vpci/vpci.c
> +++ b/xen/drivers/vpci/vpci.c
> @@ -107,6 +107,16 @@ int vpci_add_handlers(struct pci_dev *pdev)
>  
>      return rc;
>  }
> +
> +int vpci_reset_device_state(struct pci_dev *pdev)
> +{
> +    ASSERT(pcidevs_locked());
> +    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
> +
> +    vpci_remove_device(pdev);
> +    return vpci_add_handlers(pdev);

Note that these two functions may be renamed soon by the patch at [1].
Whichever patch goes in later will need to be rebased to account for the
rename.

[1] https://lists.xenproject.org/archives/html/xen-devel/2024-02/msg00134.html

> +}
> +
>  #endif /* __XEN__ */
>  
>  static int vpci_register_cmp(const struct vpci_register *r1,



From xen-devel-bounces@lists.xenproject.org Fri Feb 09 19:54:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 19:54:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678830.1056383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYWx0-0005QQ-FT; Fri, 09 Feb 2024 19:54:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678830.1056383; Fri, 09 Feb 2024 19:54:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYWx0-0005QJ-Cp; Fri, 09 Feb 2024 19:54:18 +0000
Received: by outflank-mailman (input) for mailman id 678830;
 Fri, 09 Feb 2024 19:54:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYWwy-0005Q9-L6; Fri, 09 Feb 2024 19:54:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYWwy-0002VP-Bm; Fri, 09 Feb 2024 19:54:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYWwx-0002LV-Uh; Fri, 09 Feb 2024 19:54:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYWwx-0004cI-UJ; Fri, 09 Feb 2024 19:54:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=A6RjE5cv8TGhgouYuBtCIc48PDv3jw+JkfAV6L5MRnc=; b=kQbKPwqWmG1gzvXcTaXI7sEuoK
	xR9/Cxi2oiqY3JZ+yds9dVaX56rFpV57FQJXJ6d6eHpp+QLIMLL6JkJihGZwarRiieQ6RrpLG9A61
	1Y/wBLFFvel+/wYQtFselVLxh8PAQccWEK0YACVr7w6X0d3k+Ij0tQtkliGC/cC34UF8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184633-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184633: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-vhd:host-ping-check-xen:fail:heisenbug
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=1f719a2f3fa67665578c759ac34fd3d3690c1a20
X-Osstest-Versions-That:
    linux=047371968ffc470769f541d6933e262dc7085456
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 09 Feb 2024 19:54:15 +0000

flight 184633 linux-linus real [real]
flight 184636 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184633/
http://logs.test-lab.xenproject.org/osstest/logs/184636/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-vhd      10 host-ping-check-xen fail pass in 184636-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-vhd     14 migrate-support-check fail in 184636 never pass
 test-armhf-armhf-xl-vhd 15 saverestore-support-check fail in 184636 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184628
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184628
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184628
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184628
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184628
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184628
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184628
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184628
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 linux                1f719a2f3fa67665578c759ac34fd3d3690c1a20
baseline version:
 linux                047371968ffc470769f541d6933e262dc7085456

Last test of basis   184628  2024-02-08 18:42:01 Z    1 days
Testing same since   184633  2024-02-09 05:12:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alexandra Winter <wintera@linux.ibm.com>
  Allison Henderson <allison.henderson@oracle.com>
  Antoine Tenart <atenart@kernel.org>
  Arend van Spriel <arend.vanspriel@broadcom.com>
  Benjamin Berg <benjamin.berg@intel.com>
  Brad Spengler <spender@grsecurity.net>
  Breno Leitao <leitao@debian.org>
  Christian Heusel <christian@heusel.eu>
  Daniel Basilio <daniel.basilio@corigine.com>
  Daniel de Villiers <daniel.devilliers@corigine.com>
  David Howells <dhowells@redhat.com>
  David S. Miller <davem@davemloft.net>
  Dmitry Safonov <dima@arista.com>
  Emmanuel Grumbach <emmanuel.grumbach@intel.com>
  Eric Dumazet <edumazet@google.com>
  Felix Fietkau <nbd@nbd.name>
  Felix Huettner <felix.huettner@mail.schwarz>
  Florian Westphal <fw@strlen.de>
  Furong Xu <0x1207@gmail.com>
  Gerhard Engleder <gerhard@engleder-embedded.com>
  Gregory Greenman <gregory.greenman@intel.com>
  Ivan Vecera <ivecera@redhat.com>
  Jakub Kicinski <kuba@kernel.org>
  James Hershaw <james.hershaw@corigine.com>
  Jesse Brandeburg <jesse.brandeburg@intel.com>
  Jiri Pirko <jiri@nvidia.com>
  Johannes Berg <johannes.berg@intel.com>
  Jozsef Kadlecsik <kadlec@netfilter.org>
  Kalle Valo <kvalo@kernel.org>
  Kees Cook <keescook@chromium.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Walleij <linus.walleij@linaro.org>
  Louis Peens <louis.peens@corigine.com>
  Mario Limonciello <mario.limonciello@amd.com>
  Michael Lass <bevan@bi-co.net>
  Miri Korenblit <miriam.rachel.korenblit@intel.com>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Paolo Abeni <pabeni@redhat.com>
  Petr Tesarik <petr@tesarici.cz>
  Rafael J. Wysocki <rafael@kernel.org>
  Ratheesh Kannoth <rkannoth@marvell.com>
  Shigeru Yoshida <syoshida@redhat.com>
  Shradha Gupta <shradhagupta@linux.microsoft.com>
  Sinthu Raja <sinthu.raja@ti.com>
  Xin Long <lucien.xin@gmail.com>
  Zhipeng Lu <alexious@zju.edu.cn>
  Стас Ничипорович <stasn77@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   047371968ffc..1f719a2f3fa6  1f719a2f3fa67665578c759ac34fd3d3690c1a20 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 20:59:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 20:59:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678837.1056393 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYXxy-0006Aw-An; Fri, 09 Feb 2024 20:59:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678837.1056393; Fri, 09 Feb 2024 20:59:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYXxy-0006Ap-8E; Fri, 09 Feb 2024 20:59:22 +0000
Received: by outflank-mailman (input) for mailman id 678837;
 Fri, 09 Feb 2024 20:59:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYXxx-0006Af-3h; Fri, 09 Feb 2024 20:59:21 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYXxw-0003gG-V0; Fri, 09 Feb 2024 20:59:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYXxw-00042u-JR; Fri, 09 Feb 2024 20:59:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYXxw-0005qt-J0; Fri, 09 Feb 2024 20:59:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=vM0DAlRH+cahJ9WzGQ/mDgYn1oXl92dg1sR11TFeBlE=; b=ukRW+5aR6QNQE3km6DYKISQ9Nw
	cnkqHgMiU5l79Q+82jTVkkfAsJL936wSmzMxG1sOlIZ8JoPazihMQqBNX+7VhXJgZ0DucftJmNt0D
	NHp74zbLcNF9BfRo5DjG/p17PbAs4wAPM2fLeApmtfuPL/vtdbp7upFG8s6M8fqbhoyY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184637-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184637: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=dcf2e39dce00a5978f1c1a53a0fce04d0fdc9a3c
X-Osstest-Versions-That:
    ovmf=e32b58ab5a12d37c82327f28376e7d12cccc8b3a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 09 Feb 2024 20:59:20 +0000

flight 184637 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184637/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 dcf2e39dce00a5978f1c1a53a0fce04d0fdc9a3c
baseline version:
 ovmf                 e32b58ab5a12d37c82327f28376e7d12cccc8b3a

Last test of basis   184629  2024-02-08 19:43:01 Z    1 days
Testing same since   184637  2024-02-09 18:41:14 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jayaprakash N <n.jayaprakash@intel.com>
  Michael D Kinney <michael.d.kinney@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   e32b58ab5a..dcf2e39dce  dcf2e39dce00a5978f1c1a53a0fce04d0fdc9a3c -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 21:05:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 21:05:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678843.1056404 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYY4K-0007fn-2T; Fri, 09 Feb 2024 21:05:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678843.1056404; Fri, 09 Feb 2024 21:05:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYY4J-0007fg-Ug; Fri, 09 Feb 2024 21:05:55 +0000
Received: by outflank-mailman (input) for mailman id 678843;
 Fri, 09 Feb 2024 21:05:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=40k0=JS=kernel.org=helgaas@srs-se1.protection.inumbo.net>)
 id 1rYY4I-0007fa-JE
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 21:05:54 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 01f3ebe4-c78f-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 22:05:53 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id C545862190;
 Fri,  9 Feb 2024 21:05:51 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F711C433C7;
 Fri,  9 Feb 2024 21:05:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01f3ebe4-c78f-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707512751;
	bh=rjVvpgkdjij/VYXtLbzigiHQtkw04AlRvlQ3kPwWPAA=;
	h=Date:From:To:Cc:Subject:In-Reply-To:From;
	b=kwQK9UoMLNlCNW4mnILL/fjh6js/5cqzhXmOYR58/NZ49GCoEo35noV1N0Rexcjxg
	 lAo1KlEs60klZiyOyPrxgCKJTJAnrWYuu+tl+4qSVBQtv8xU+/SujJcpOgyxKVu+sh
	 KmtcuwLxpMN1rKL4twh466o9n/Qy3jLRTlt/hdebS8LRqcnolN4XCPDOcKoHGhNyf4
	 g7NULVMt5QtAfP+Hj7xejLKShQinuhM6827OqwAbYFZM0Y8BKAxgPkOfVHljo5k/4h
	 Z2HqW/rZs2uP/V42MoFVA1hXFtNXb0Zhp+pPSPlH//BjNALkVszex5H72/M+Iy1IJB
	 QpisPIJPit19w==
Date: Fri, 9 Feb 2024 15:05:49 -0600
From: Bjorn Helgaas <helgaas@kernel.org>
To: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
Cc: "Chen, Jiqian" <Jiqian.Chen@amd.com>,
	"Rafael J . Wysocki" <rafael@kernel.org>,
	Len Brown <lenb@kernel.org>, Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
	"Huang, Ray" <Ray.Huang@amd.com>,
	"Ragiadakou, Xenia" <Xenia.Ragiadakou@amd.com>
Subject: Re: [RFC KERNEL PATCH v4 3/3] PCI/sysfs: Add gsi sysfs for pci_dev
Message-ID: <20240209210549.GA884438@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZbtY1R15pYZz3F3B@macbook>

On Thu, Feb 01, 2024 at 09:39:49AM +0100, Roger Pau Monné wrote:
> On Wed, Jan 31, 2024 at 01:00:14PM -0600, Bjorn Helgaas wrote:
> > On Wed, Jan 31, 2024 at 09:58:19AM +0100, Roger Pau Monné wrote:
> > > On Tue, Jan 30, 2024 at 02:44:03PM -0600, Bjorn Helgaas wrote:
> > > > On Tue, Jan 30, 2024 at 10:07:36AM +0100, Roger Pau Monné wrote:
> > > > > On Mon, Jan 29, 2024 at 04:01:13PM -0600, Bjorn Helgaas wrote:
> > > > > > On Thu, Jan 25, 2024 at 07:17:24AM +0000, Chen, Jiqian wrote:
> > > > > > > On 2024/1/24 00:02, Bjorn Helgaas wrote:
> > > > > > > > On Tue, Jan 23, 2024 at 10:13:52AM +0000, Chen, Jiqian wrote:
> > > > > > > >> On 2024/1/23 07:37, Bjorn Helgaas wrote:
> > > > > > > >>> On Fri, Jan 05, 2024 at 02:22:17PM +0800, Jiqian Chen wrote:
> > > > > > > >>>> There is a need for some scenarios to use gsi sysfs.
> > > > > > > >>>> For example, when xen passthrough a device to dumU, it will
> > > > > > > >>>> use gsi to map pirq, but currently userspace can't get gsi
> > > > > > > >>>> number.
> > > > > > > >>>> So, add gsi sysfs for that and for other potential scenarios.
> > > > > > > >> ...
> > > > > > > > 
> > > > > > > >>> I don't know enough about Xen to know why it needs the GSI in
> > > > > > > >>> userspace.  Is this passthrough brand new functionality that can't be
> > > > > > > >>> done today because we don't expose the GSI yet?
> > > > > > 
> > > > > > I assume this must be new functionality, i.e., this kind of
> > > > > > passthrough does not work today, right?
> > > > > > 
> > > > > > > >> has ACPI support and is responsible for detecting and controlling
> > > > > > > >> the hardware, also it performs privileged operations such as the
> > > > > > > >> creation of normal (unprivileged) domains DomUs. When we give to a
> > > > > > > >> DomU direct access to a device, we need also to route the physical
> > > > > > > >> interrupts to the DomU. In order to do so Xen needs to setup and map
> > > > > > > >> the interrupts appropriately.
> > > > > > > > 
> > > > > > > > What kernel interfaces are used for this setup and mapping?
> > > > > > >
> > > > > > > For passthrough devices, the setup and mapping of routing physical
> > > > > > > interrupts to DomU are done on Xen hypervisor side, hypervisor only
> > > > > > > need userspace to provide the GSI info, see Xen code:
> > > > > > > xc_physdev_map_pirq require GSI and then will call hypercall to pass
> > > > > > > GSI into hypervisor and then hypervisor will do the mapping and
> > > > > > > routing, kernel doesn't do the setup and mapping.
> > > > > > 
> > > > > > So we have to expose the GSI to userspace not because userspace itself
> > > > > > uses it, but so userspace can turn around and pass it back into the
> > > > > > kernel?
> > > > > 
> > > > > No, the point is to pass it back to Xen, which doesn't know the
> > > > > mapping between GSIs and PCI devices because it can't execute the ACPI
> > > > > AML resource methods that provide such information.
> > > > > 
> > > > > The (Linux) kernel is just a proxy that forwards the hypercalls from
> > > > > user-space tools into Xen.
> > > > 
> > > > But I guess Xen knows how to interpret a GSI even though it doesn't
> > > > have access to AML?
> > > 
> > > On x86 Xen does know how to map a GSI into an IO-APIC pin, in order
> > > configure the RTE as requested.
> > 
> > IIUC, mapping a GSI to an IO-APIC pin requires information from the
> > MADT.  So I guess Xen does use the static ACPI tables, but not the AML
> > _PRT methods that would connect a GSI with a PCI device?
> 
> Yes, Xen can parse the static tables, and knows the base GSI of
> IO-APICs from the MADT.
> 
> > I guess this means Xen would not be able to deal with _MAT methods,
> > which also contains MADT entries?  I don't know the implications of
> > this -- maybe it means Xen might not be able to use with hot-added
> > devices?
> 
> It's my understanding _MAT will only be present on some very specific
> devices (IO-APIC or CPU objects).  Xen doesn't support hotplug of
> IO-APICs, but hotplug of CPUs should in principle be supported with
> cooperation from the control domain OS (albeit it's not something that
> we tests on our CI).  I don't expect however that a CPU object _MAT
> method will return IO APIC entries.
> 
> > The tables (including DSDT and SSDTS that contain the AML) are exposed
> > to userspace via /sys/firmware/acpi/tables/, but of course that
> > doesn't mean Xen knows how to interpret the AML, and even if it did,
> > Xen probably wouldn't be able to *evaluate* it since that could
> > conflict with the host kernel's use of AML.
> 
> Indeed, there can only be a single OSPM, and that's the dom0 OS (Linux
> in our context).
> 
> Getting back to our context though, what would be a suitable place for
> exposing the GSI assigned to each device?

IIUC, the Xen hypervisor:

  - Interprets /sys/firmware/acpi/tables/APIC (or gets this via
    something running on the Dom0 kernel) to find the physical base
    address and GSI base, e.g., from I/O APIC, I/O SAPIC.

  - Needs the GSI to locate the APIC and pin within the APIC.  The
    Dom0 kernel is the OSPM, so only it can evaluate the AML _PRT to
    learn the PCI device -> GSI mapping.

  - Has direct access to the APIC physical base address to program the
    Redirection Table.

The patch seems a little messy to me because the PCI core has to keep
track of the GSI even though it doesn't need it itself.  And the
current patch exposes it on all arches, even non-ACPI ones or when
ACPI is disabled (easily fixable).

We only call acpi_pci_irq_enable() in the pci_enable_device() path, so
we don't know the GSI unless a Dom0 driver has claimed the device and
called pci_enable_device() for it, which seems like it might not be
desirable.

I was hoping we could put it in /sys/firmware/acpi/interrupts, but
that looks like it's only for SCI statistics.  I guess we could moot a
new /sys/firmware/acpi/gsi/ directory, but then each file there would
have to identify a device, which might not be as convenient as the
/sys/devices/ directory that already exists.  I guess there may be
GSIs for things other than PCI devices; will you ever care about any
of those?

Bjorn


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 21:54:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 21:54:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678850.1056414 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYYos-0006eB-IA; Fri, 09 Feb 2024 21:54:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678850.1056414; Fri, 09 Feb 2024 21:54:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYYos-0006e4-FH; Fri, 09 Feb 2024 21:54:02 +0000
Received: by outflank-mailman (input) for mailman id 678850;
 Fri, 09 Feb 2024 21:54:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=H1sd=JS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rYYoq-0006dv-Pi
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 21:54:00 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b9ca1f9e-c795-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 22:53:58 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 2B9E0621A7;
 Fri,  9 Feb 2024 21:53:57 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 25859C433C7;
 Fri,  9 Feb 2024 21:53:56 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9ca1f9e-c795-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707515636;
	bh=8ju2yy5RFnHwcW52knD89kxNaAND8c1d1mv15ahob0o=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=c3uEbhhbfBKY6TSAJiaJ8elG9KSXET75X++W0bW1URkVYKtj2lCs0d107E37upZzv
	 3EX0y3qmglWULOWeD2hceYjzqA82FCcOujoidWQ9JmPtV2rlIuR2XAwPEhNpU2avZB
	 FVFQ2RoF9feZL7+K18PEurJZYSUd+J08C5kQ4zaN90bMeRTznLnunGZk7CRLnGsMo/
	 F6+dgS4lfLZDSM8HQI7JMIGsQ46LQyojQZdjURpZUhidBWSs0CK5G7GAxZccCequDu
	 TV8QBA9qUTqYMGacNpLb1/apk7/egpzePn3oyJCWzVimwbgh6HQb8H/R0tvDNvy7Tm
	 T8C4d8VMgMEoA==
Date: Fri, 9 Feb 2024 13:53:54 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Simone Ballarin <simone.ballarin@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH] eclair: remove wrongly added -eval_file
In-Reply-To: <976ea35a8682adb2991d249a48bb2b41e834e4e6.1707482589.git.simone.ballarin@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402091353490.1925432@ubuntu-linux-20-04-desktop>
References: <976ea35a8682adb2991d249a48bb2b41e834e4e6.1707482589.git.simone.ballarin@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 8 Feb 2024, Simone Ballarin wrote:
> properties.ecl does not exist and it is not
> even generated by propertyparser.py. Loading
> this file causes empty ECLAIR analysis.
> 
> Fixes: f4519ee8 ("eclair: move function and macro properties outside ECLAIR")
> Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/eclair_analysis/ECLAIR/analysis.ecl | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/analysis.ecl b/automation/eclair_analysis/ECLAIR/analysis.ecl
> index 684c5b0b39..a604582da3 100644
> --- a/automation/eclair_analysis/ECLAIR/analysis.ecl
> +++ b/automation/eclair_analysis/ECLAIR/analysis.ecl
> @@ -30,7 +30,6 @@ if(not(scheduled_analysis),
>  -eval_file=deviations.ecl
>  -eval_file=call_properties.ecl
>  -eval_file=tagging.ecl
> --eval_file=properties.ecl
>  -eval_file=concat(set,".ecl")
>  
>  -doc="Hide reports in external code."
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 22:02:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 22:02:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678855.1056424 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYYxJ-0000By-B5; Fri, 09 Feb 2024 22:02:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678855.1056424; Fri, 09 Feb 2024 22:02:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYYxJ-0000Br-7i; Fri, 09 Feb 2024 22:02:45 +0000
Received: by outflank-mailman (input) for mailman id 678855;
 Fri, 09 Feb 2024 22:02:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=H1sd=JS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rYYxI-0000Bc-U4
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 22:02:44 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f1eb0b65-c796-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 23:02:43 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id B7D82CE212A;
 Fri,  9 Feb 2024 22:02:37 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7BC27C433F1;
 Fri,  9 Feb 2024 22:02:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1eb0b65-c796-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707516157;
	bh=OkDQU3kSODec1b1x5RiJrZTr3MsFnUPpdlIyO0LN6Ag=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=tAw4wVBCeN6BxDlKoyXb/3f3Bn++j49BOn8gu8QegyPyeAOztj0OGD9wWKOv5CSWr
	 +YotrJhBA/7U+iqGQL5vPxndfC1Z1TKsNcyQWib3UYPU/GWCgGeWulR2DRTrumLx5w
	 OA+mt2fWk+UrpVZeOg5BYHKIJcF+VjzFzvWOxO2hqPi6IBuVxKP9RMlXZiEDpfTMkk
	 8dH1y8HhaQ9TfCZArDd3ifHixGLwXPcU50kKoc0yufKKKC5u1gyVbOvcDNxHJZZKVG
	 iryeGvIFwhmTJ6qKr0x03fwG8EftyIYZ29Kq/gJUfMQgpGJLrevDgXVOHghDeABFJ8
	 GprsJMJgBgw2g==
Date: Fri, 9 Feb 2024 14:02:34 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: Xen Devel <xen-devel@lists.xenproject.org>, 
    Consulting <consulting@bugseng.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, Jbeulich <jbeulich@suse.com>, 
    Andrew Cooper3 <andrew.cooper3@citrix.com>, 
    Roger Pau <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>, 
    Wei Liu <wl@xen.org>
Subject: Re: Return type of clean_and_invalidate_dcache_va_range
In-Reply-To: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402091402080.1925432@ubuntu-linux-20-04-desktop>
References: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 9 Feb 2024, Nicola Vetrini wrote:
> Hi all,
> 
> In the context of violations of MISRA C:2012 Rule 17.7: "The value returned by
> a function having non-void return type shall be used", I was looking at the
> function "clean_and_invalidate_dcache_va_range". It has the following
> signature on both arm and x86:
> 
> static inline int clean_and_invalidate_dcache_va_range
>     (const void *p, unsigned long size)
> 
> The commit that introduced it for Arm ~9 years ago (71d64afe3e12: "arm: return
> int from *_dcache_va_range") [1] mentions that on Arm it can't fail, but
> supposedly it can on x86.
> 
> However, as far as I can tell, for both arch-es the implementation now always
> returns 0 [2][3], so perhaps the mention of -EOPNOTSUPP for x86 is no longer
> true (I wasn't able to reconstruct if there was a time at which this was true,
> even in the same commit that changed the return type to int).
> 
> The question is: should the return type be void, since it appears that every
> user is ignoring the returned value (violating the rule), except the one in
> common/grant_table.c [4]?

Looking at the implementation on both ARM and x86, I am in favor of
changing the return type to void



> The other two resolution paths are either allowing this function's result to
> be ignored or cast all ignored invocations to void, with the first being
> cleaner from a code readability perspective.
> 
> [1] These functions cannot really fail on ARM, but their x86 equivalents can
> (-EOPNOTSUPP). Change the prototype to return int.
> [2]
> https://gitlab.com/xen-project/xen/-/blob/staging/xen/arch/arm/include/asm/page.h#L218
> [3]
> https://gitlab.com/xen-project/xen/-/blob/staging/xen/arch/x86/include/asm/flushtlb.h#L188
> [4]
> https://gitlab.com/xen-project/xen/-/blob/staging/xen/common/grant_table.c#L3576
> 
> -- 
> Nicola Vetrini, BSc
> Software Engineer, BUGSENG srl (https://bugseng.com)
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 22:08:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 22:08:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678859.1056434 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYZ2z-0001Rd-V1; Fri, 09 Feb 2024 22:08:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678859.1056434; Fri, 09 Feb 2024 22:08:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYZ2z-0001RW-SC; Fri, 09 Feb 2024 22:08:37 +0000
Received: by outflank-mailman (input) for mailman id 678859;
 Fri, 09 Feb 2024 22:08:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=H1sd=JS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rYZ2y-0001RQ-It
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 22:08:36 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c36321f0-c797-11ee-98f5-efadbce2ee36;
 Fri, 09 Feb 2024 23:08:34 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id C168CCE2119;
 Fri,  9 Feb 2024 22:08:31 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 32D0FC433F1;
 Fri,  9 Feb 2024 22:08:29 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c36321f0-c797-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707516511;
	bh=JR84V5h2foy3AGZ/3LhZFm2YpXNh/5hUeY0cRzKZta4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=flNHGLlWv9stvSwhs1uJE5oIQvu/le9uLrOgG7xA8rntiikk5g6zg2+CYNccMwuJH
	 xyJJ/UyV+uTTyRvuZuGtxOjj+1rRJ7Jg0i5my9qXInqCu91qAjr8UmZd5Zq0/oUk8H
	 cOAPpSaD6zf4ix3Gs56LTURjSjVx4CsFNZlY740j1oCskyKRhpK6lGc7S5QNCt330J
	 6nTkX6laGQFvAleVdfWpyfP27UIgAZ3Lhs9GqzCynt41n9bEiG0aBvN7dnDTqQd7LB
	 st3p6Jmj1bKBeKrZxo4q6DSFxGz0ogLTzSKpBvEwLpvX4pzsHVKaV7TAad5b9rNTgE
	 HbzP0F7XqRkZg==
Date: Fri, 9 Feb 2024 14:08:27 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Simone Ballarin <simone.ballarin@bugseng.com>, sstabellini@kernel.org, 
    consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Dario Faggioli <dfaggioli@suse.com>, 
    Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>, 
    xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and
 logging effects
In-Reply-To: <c341164a-0bd9-41f1-a984-718ccb986053@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402091406470.1925432@ubuntu-linux-20-04-desktop>
References: <cover.1706886631.git.simone.ballarin@bugseng.com> <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com> <09869431-ed00-4102-93fd-84c697a5c950@suse.com> <851eb6ec-558b-4a89-a31a-34046730bb55@bugseng.com>
 <41474fe7-c65f-4244-a455-0aaa0e4315e3@suse.com> <e48f4d49-274c-4bd6-8e3b-1ecaee6fa602@bugseng.com> <c6199a60-ecd2-4f8a-b742-199b90e62bac@suse.com> <a3586ea2-5ad7-4c64-8e05-00ea0f83518f@bugseng.com> <c341164a-0bd9-41f1-a984-718ccb986053@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 9 Feb 2024, Jan Beulich wrote:
> On 09.02.2024 10:25, Simone Ballarin wrote:
> > On 07/02/24 13:40, Jan Beulich wrote:
> >> On 07.02.2024 13:21, Simone Ballarin wrote:
> >>> On 07/02/24 11:24, Jan Beulich wrote:
> >>>> On 07.02.2024 11:03, Simone Ballarin wrote:
> >>>>> On 06/02/24 13:04, Jan Beulich wrote:
> >>>>>> On 02.02.2024 16:16, Simone Ballarin wrote:
> >>>>>>> --- a/xen/arch/arm/guestcopy.c
> >>>>>>> +++ b/xen/arch/arm/guestcopy.c
> >>>>>>> @@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf, uint64_t addr, unsigned int len,
> >>>>>>>     unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
> >>>>>>>     {
> >>>>>>>         return copy_guest((void *)from, (vaddr_t)to, len,
> >>>>>>> -                      GVA_INFO(current), COPY_to_guest | COPY_linear);
> >>>>>>> +                      /* SAF-4-safe No persistent side effects */
> >>>>>>> +                      GVA_INFO(current),
> >>>>>>
> >>>>>> I _still_ think this leaves ambiguity. The more that you need to look
> >>>>>> up GVA_INFO() to recognize what this is about.
> >>>>>
> >>>>>
> >>>>> Just to recap: here the point is that current reads a register with a volatile asm, so the
> >>>>> violation is in the expansion of GVA_INFO(current). Both GVA_INFO and current taken alone
> >>>>> are completely fine, so this is the only place where a SAF comment can be placed.
> >>>>>
> >>>>> The exapansion is:
> >>>>> ((copy_info_t) { .gva = { ((*({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : "0"(&
> >>>>>      per_cpu__curr_vcpu)); (typeof(&per_cpu__curr_vcpu)) (__ptr + (({ uint64_t _r; asm volatile("mrs  %0, ""TPIDR_EL2" : "=r"
> >>>>>      (_r)); _r; }))); }))) } }), (1U << 1) | (1U << 2));
> >>>>>
> >>>>> My proposals are:
> >>>>> 1) address the violation moving the current expansion outside (extra variable);
> >>>>> 2) put a more detailed comment to avoid the ambiguity;
> >>>>> 3) use an ECL deviation for GVA_INFO(current).
> >>>>>
> >>>>> Do you have any preference or proposal?
> >>>>
> >>>> Imo 3 is not an option at all. Probably 1 wouldn't be too bad here, but
> >>>> I still wouldn't like it (as matching a general pattern I try to avoid:
> >>>> introducing local variables that are used just once and don't meaningfully
> >>>> improve e.g. readability). Therefore out of what you list, 2 would remain.
> >>>> But I'm not happy with a comment here either - as per above, there's
> >>>> nothing that can go wrong here as long as there's only a single construct
> >>>> with side effect(s).
> >>>>
> >>> So, would be changing the SAF in:
> >>> /* SAF-<new_id>-safe single item initializer */
> >>>
> >>> OK for you?
> >>
> >> A comment, as said, is only the least bad of what you did enumerate. But
> >> for this code in particular I'm not a maintainer anyway, so it's not me
> >> you need to convince. I'm taking this only as an example for discussing
> >> underlying aspects.
> > 
> > I was generally thinking about the comments of this series, and I've
> > just realised that many of them can be summarized by the following sentence:
> > 
> > "We do not want changes to address violations of R13.1 that are not also violations
> > of R13.2"
> > 
> > MC3R1.R13.2	rule	The value of an expression and its persistent side effects shall be the same under all permitted evaluation orders
> > MC3R1.R13.1	rule	Initializer lists shall not contain persistent side effects
> > 
> > At this point, my proposal is to remove R13.1 from the coding standard and add
> > R13.2 (eventually limiting its scope to initializer lists).
> 
> I'm afraid I don't understand the "eventually limiting" part.
> 
> > Maybe it is better to re-discuss the rule adoption during the next meeting?
> 
> Perhaps best. Stefano, could you take note of this?

Yes. I think Simone might be right: it looks like the cases we care
about the most are the ones covered by 13.2, which appear to be a subset
of the ones covered by 13.1.

Assuming we accept 13.2, the question is whether there is anything left
in 13.1 that is of interest. We can discuss during the next MISRA call.


From xen-devel-bounces@lists.xenproject.org Fri Feb 09 22:13:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 09 Feb 2024 22:13:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678863.1056444 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYZ7e-0002vE-Kp; Fri, 09 Feb 2024 22:13:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678863.1056444; Fri, 09 Feb 2024 22:13:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYZ7e-0002v7-GY; Fri, 09 Feb 2024 22:13:26 +0000
Received: by outflank-mailman (input) for mailman id 678863;
 Fri, 09 Feb 2024 22:13:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=H1sd=JS=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rYZ7d-0002uG-KK
 for xen-devel@lists.xenproject.org; Fri, 09 Feb 2024 22:13:25 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6fd96c7e-c798-11ee-8a4b-1f161083a0e0;
 Fri, 09 Feb 2024 23:13:23 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 8C921CE1C09;
 Fri,  9 Feb 2024 22:13:19 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id D6226C433C7;
 Fri,  9 Feb 2024 22:13:16 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fd96c7e-c798-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707516798;
	bh=V0oWTTBFMbD4rVZopw9sOpBhmE7SuVVs5CGtzmZopm8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=dSuWLsbJneDujD+n65e6/pdQqipGuJmdMDxJP0yGOsYxbLqn3ugaXyTsD1SmNy2Bu
	 tyhrrQSiHSv+s2KROR8DyCSRdZr5zB9qDSsEETPYZgY1O+xxD8lZ7cPZV8ugK3CjdV
	 xTcf/PogdG7LOh6WXaei2/XPOrERc6GLWnx0MRcjtK/Ed4bsOlZk18q76Iv+t6Kqx6
	 /+7Wa53/ih5qS6lnofWppj31MhXk0xWRxWbW8Bp6zllUgtqsqkkMWO9/Y1vUeFOzrk
	 WNFWAtfEWyhVBXglTMwvoXSR5AgJyrEQHtJ1Zcr+5nCbL7lRLXGYYpp15lKxB4K+yv
	 ttd5YA87KfP4Q==
Date: Fri, 9 Feb 2024 14:13:15 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Simone Ballarin <simone.ballarin@bugseng.com>
cc: Jan Beulich <jbeulich@suse.com>, consulting@bugseng.com, 
    sstabellini@kernel.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Dario Faggioli <dfaggioli@suse.com>, 
    Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>, 
    xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and
 logging effects
In-Reply-To: <e48f4d49-274c-4bd6-8e3b-1ecaee6fa602@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402091410490.1925432@ubuntu-linux-20-04-desktop>
References: <cover.1706886631.git.simone.ballarin@bugseng.com> <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com> <09869431-ed00-4102-93fd-84c697a5c950@suse.com> <851eb6ec-558b-4a89-a31a-34046730bb55@bugseng.com>
 <41474fe7-c65f-4244-a455-0aaa0e4315e3@suse.com> <e48f4d49-274c-4bd6-8e3b-1ecaee6fa602@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 7 Feb 2024, Simone Ballarin wrote:
> On 07/02/24 11:24, Jan Beulich wrote:
> > On 07.02.2024 11:03, Simone Ballarin wrote:
> > > On 06/02/24 13:04, Jan Beulich wrote:
> > > > On 02.02.2024 16:16, Simone Ballarin wrote:
> > > > > Rule 13.1: Initializer lists shall not contain persistent side effects
> > > > > 
> > > > > Effects caused by debug/logging macros and functions (like ASSERT,
> > > > > __bad_atomic_size,
> > > > > LOG, etc ...) that crash execution or produce logs are not dangerous
> > > > > in initializer
> > > > > lists. The evaluation order in abnormal conditions is not relevant.
> > > > > Evaluation order
> > > > > of logging effects is always safe.
> > > > 
> > > > I thought I said so before: When talking of just logging, evaluation
> > > > order
> > > > may very well have a impact on correctness. Therefore we shouldn't mix
> > > > debugging and logging.
> > > 
> > > My general feeling was that changes like the following one are not
> > > supported by
> > > the community:
> > > 
> > > - x = { .field1 = function_with_logs_effects() /*other eventual code*/ };
> > > + int field1 = function_with_logs_effects();
> > > + x = { .field1 = field1 /*other eventual code*/};
> > > 
> > > so I tried to deviate as much as possible.
> > > 
> > > If having log effects is a good reason to do changes like the above, I can
> > > propose a patch in that sense.
> > 
> > Just to avoid misunderstandings: I'm not advocating for changes like the
> > one you outline above. I simply consider the rule too strict: There's
> > nothing at risk when there's just a single operation with side effects
> > in an initializer.
> 
> I agree for the safe cases such as single item list initializers
> (independently
> by the number of effect contained in io_apic_read).
> In fact, I was about to propose in another patch to deviate cases like:
> 
> union IO_APIC_reg_01 reg_01 = { .raw = io_apic_read(idx, 1) };
> union IO_APIC_reg_02 reg_02 = { .raw = io_apic_read(idx, 2) };
> 
> > Even when there are multiple such operations, whether
> > there's anything at risk depends on whether any of the side effects
> > actually collide. In a number of cases the compiler would actually warn
> > (and thus, due to -Werror, the build would fail).
> > 
> 
> I don't completely agree on that, this requires an in-depth comprehension
> of the code especially when complex call chains are involved. Moreover
> these deviations need to be maintained when one of the function involved
> changes.
> 
> > The primary purpose of my comment here was that we need to please
> > separate debugging from logging side effects.
> > 
> 
> Ok, I will work in that sense.
> 
> > > > > --- a/xen/arch/arm/guestcopy.c
> > > > > +++ b/xen/arch/arm/guestcopy.c
> > > > > @@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf,
> > > > > uint64_t addr, unsigned int len,
> > > > >    unsigned long raw_copy_to_guest(void *to, const void *from,
> > > > > unsigned int len)
> > > > >    {
> > > > >        return copy_guest((void *)from, (vaddr_t)to, len,
> > > > > -                      GVA_INFO(current), COPY_to_guest |
> > > > > COPY_linear);
> > > > > +                      /* SAF-4-safe No persistent side effects */
> > > > > +                      GVA_INFO(current),
> > > > 
> > > > I _still_ think this leaves ambiguity. The more that you need to look
> > > > up GVA_INFO() to recognize what this is about.
> > > 
> > > 
> > > Just to recap: here the point is that current reads a register with a
> > > volatile asm, so the
> > > violation is in the expansion of GVA_INFO(current). Both GVA_INFO and
> > > current taken alone
> > > are completely fine, so this is the only place where a SAF comment can be
> > > placed.
> > > 
> > > The exapansion is:
> > > ((copy_info_t) { .gva = { ((*({ unsigned long __ptr; __asm__ ("" :
> > > "=r"(__ptr) : "0"(&
> > >     per_cpu__curr_vcpu)); (typeof(&per_cpu__curr_vcpu)) (__ptr + (({
> > > uint64_t _r; asm volatile("mrs  %0, ""TPIDR_EL2" : "=r"
> > >     (_r)); _r; }))); }))) } }), (1U << 1) | (1U << 2));
> > > 
> > > My proposals are:
> > > 1) address the violation moving the current expansion outside (extra
> > > variable);
> > > 2) put a more detailed comment to avoid the ambiguity;
> > > 3) use an ECL deviation for GVA_INFO(current).
> > > 
> > > Do you have any preference or proposal?
> > 
> > Imo 3 is not an option at all. Probably 1 wouldn't be too bad here, but
> > I still wouldn't like it (as matching a general pattern I try to avoid:
> > introducing local variables that are used just once and don't meaningfully
> > improve e.g. readability). Therefore out of what you list, 2 would remain.
> > But I'm not happy with a comment here either - as per above, there's
> > nothing that can go wrong here as long as there's only a single construct
> > with side effect(s).
> > 
> So, would be changing the SAF in:
> /* SAF-<new_id>-safe single item initializer */
> 
> OK for you?
> 
> > > > > --- a/xen/arch/x86/hvm/hvm.c
> > > > > +++ b/xen/arch/x86/hvm/hvm.c
> > > > > @@ -800,6 +800,7 @@ static int cf_check hvm_save_cpu_ctxt(struct vcpu
> > > > > *v, hvm_domain_context_t *h)
> > > > >    {
> > > > >        struct segment_register seg;
> > > > >        struct hvm_hw_cpu ctxt = {
> > > > > +        /* SAF-3-safe effects for debugging/logging reasons are safe
> > > > > */
> > > > >            .tsc = hvm_get_guest_tsc_fixed(v,
> > > > > v->domain->arch.hvm.sync_tsc),
> > > > 
> > > > A prereq for this imo is that the function take const struct vcpu *.
> > > > But I'm not sure that'll suffice. The function can change at any time,
> > > > rendering the comment here stale perhaps without anyone noticing.
> > > > 
> > > 
> > > IMO It isn't a strict prereq, but it would make everything more clear.
> > > 
> > > In any case, apart adding the const, I do not see other easy solutions.
> > > Would you give me your ack if I change the function signature?
> > 
> > Well, as said: I'm not sure that'll suffice.
> > 
> > > Another possible solutions would be documenting the function in the new
> > > JSON file with a special attribute like only_debug_effect. Of course,
> > > this still requires keeping the JSON up to date in case of changes.
> > 
> > Exactly. So wouldn't really help.
> > 
> > In any event I'd like to ask that you consider splitting up this patch,
> > such that you won't need multiple acks for any of the parts. That'll
> > also allow focusing on one aspect at a time in reviews.
> > 
> 
> Ok, but please consider that the JSON file has been precisely added to deal
> with these cases (avoiding __attribute__). If we are scared to use it, it
> becomes
> meaningless.
> 
> @Stefano maybe your opinion could help.

I do think the JSON file is a useful tool in our toolbox. It is also
fair to say that it is one that we should be careful about using as it
requires manual updates from time to time.

So, I certainly consider it an option, but I prefer your other
suggestion of dropping 13.1 in favor of 13.2. With that change, would
the need for a deviation in this function go away?


From xen-devel-bounces@lists.xenproject.org Sat Feb 10 00:47:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 10 Feb 2024 00:47:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678879.1056454 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYbW3-0000gN-48; Sat, 10 Feb 2024 00:46:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678879.1056454; Sat, 10 Feb 2024 00:46:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYbW3-0000gG-1S; Sat, 10 Feb 2024 00:46:47 +0000
Received: by outflank-mailman (input) for mailman id 678879;
 Sat, 10 Feb 2024 00:46:45 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYbW1-0000g6-Aq; Sat, 10 Feb 2024 00:46:45 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYbW1-00084y-3C; Sat, 10 Feb 2024 00:46:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYbW0-0004mB-JY; Sat, 10 Feb 2024 00:46:44 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYbW0-0005Vx-J1; Sat, 10 Feb 2024 00:46:44 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=I80YNzDUd2StXaA8rsKQfb0uUAipbdB9Kw55m18BVow=; b=F3gzu3z654MRFxBJoWepcI1cIK
	cCcSihIXiOoPY9awcYEybf4XHgVA1lFDqrXIAWSfvbt47pUy2/9I3MtPBJ7rtVp91KL1Qwqnbjz/a
	X/icNbPUQIF3NOLy6ARuFJJ9WK/lEIMF3Y2d+SsPjWFgGVE+uwCPaArDHfaltgsOl+E4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184639-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184639: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=8801c75b4d77c2e6e06b3ddc8560e0db590f6342
X-Osstest-Versions-That:
    ovmf=dcf2e39dce00a5978f1c1a53a0fce04d0fdc9a3c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 10 Feb 2024 00:46:44 +0000

flight 184639 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184639/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 8801c75b4d77c2e6e06b3ddc8560e0db590f6342
baseline version:
 ovmf                 dcf2e39dce00a5978f1c1a53a0fce04d0fdc9a3c

Last test of basis   184637  2024-02-09 18:41:14 Z    0 days
Testing same since   184639  2024-02-09 22:41:18 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Suqiang Ren <suqiangx.ren@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   dcf2e39dce..8801c75b4d  8801c75b4d77c2e6e06b3ddc8560e0db590f6342 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sat Feb 10 01:00:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 10 Feb 2024 01:00:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678888.1056464 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYbjJ-0005Qx-AN; Sat, 10 Feb 2024 01:00:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678888.1056464; Sat, 10 Feb 2024 01:00:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYbjJ-0005QD-6o; Sat, 10 Feb 2024 01:00:29 +0000
Received: by outflank-mailman (input) for mailman id 678888;
 Sat, 10 Feb 2024 01:00:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=L918=JT=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rYbjH-0004Xu-CB
 for xen-devel@lists.xenproject.org; Sat, 10 Feb 2024 01:00:27 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c5313440-c7af-11ee-8a4b-1f161083a0e0;
 Sat, 10 Feb 2024 02:00:25 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 91A11CE21B7;
 Sat, 10 Feb 2024 01:00:22 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7390CC433F1;
 Sat, 10 Feb 2024 01:00:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5313440-c7af-11ee-8a4b-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707526821;
	bh=JTkq67hnvWdxWDQL1xzqnFjH30QdB2GW5uVXDakbOlo=;
	h=Date:From:To:cc:Subject:From;
	b=PgYseM3ZYlYverGvoQSJ9lZ/maULAmoWGP093b82AO+lCZMLGDEgZ282jRBgWwqeY
	 amdTHTJRhxsiI8hJf1qA6jrFUwuYrxf54Xb6ZDGDI3Yah5Ja2XhkG8t39MMzPzHhrK
	 ztbEB7xuPRfWZCs1Su9ozvQfO0kmP+c/cGnwNFfxvMiBsIW6E2QAGN/sRKj1PA+FZr
	 rjILcporlY9biqyWTCujDEyHaQU81qQbwydTcf9JNV6+yLSzZEHOtO8m3lpXcqDD+O
	 4cY9SFaLBm96SV9+iVRTItHSTqrsBeOlPODk1VMK7+HE7aUPZeCGatey8gqn6ig8y6
	 WgCHcAhiz4KeA==
Date: Fri, 9 Feb 2024 17:00:19 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: xen-devel@lists.xenproject.org
cc: sstabellini@kernel.org, andrew.cooper3@citrix.com, jbeulich@suse.com, 
    george.dunlap@citrix.com, julien@xen.org, sstabellini@kernel.org, 
    bertrand.marquis@arm.com, roger.pau@citrix.com, 
    roberto.bagnara@bugseng.com, federico.serafini@bugseng.com
Subject: [PATCH v1] docs/misra/rules.rst: catch up with accepted rules
Message-ID: <alpine.DEB.2.22.394.2402091653110.1925432@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

Update docs/misra/rules.rst to reflect the MISRA C rules accepted in the
last couple of months.

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
---

In the notes section I added some info about the deviations, but in any
case the appropriate info will also be added to deviations.rst,
safe.json, etc.

I also added Rule 14.4, which is older, but when I first tried to add it
to rules.rst, Jan had a question I couldn't reply clearly:
https://marc.info/?l=xen-devel&m=169828285627163

I think now with this series, the impact of Rule 14.4 is clearer:
https://marc.info/?l=xen-devel&m=170194257326186

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index 182331089d..0cb87ebc35 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -181,6 +181,12 @@ maintainers if you want to suggest a change.
        headers (xen/include/public/) are allowed to retain longer
        identifiers for backward compatibility.
 
+   * - `Rule 5.5 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_05_05.c>`_
+     - Required
+     - Identifiers shall be distinct from macro names
+     - Clashes between function-like macros and non-callable entities
+       are allowed. The pattern #define x x is also allowed.
+
    * - `Rule 5.6 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_05_06.c>`_
      - Required
      - A typedef name shall be a unique identifier
@@ -462,6 +468,13 @@ maintainers if you want to suggest a change.
 
        while(0) and while(1) and alike are allowed.
 
+   * - `Rule 14.4 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_14_04.c>`_
+     - Required
+     - The controlling expression of an if-statement and the controlling
+       expression of an iteration-statement shall have essentially
+       Boolean type
+     -
+
    * - `Rule 16.3 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_16_03.c>`_
      - Required
      - An unconditional break statement shall terminate every
@@ -478,6 +491,12 @@ maintainers if you want to suggest a change.
        adhere to Rule 16.2 would result in increased complexity and
        maintenance difficulty, and could potentially introduce bugs. 
 
+   * - `Rule 16.6 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_16_06.c>`_
+     - Required
+     - Every switch statement shall have at least two switch-clauses
+     - Single-clause switches are allowed when they do not involve a
+       default label.
+
    * - `Rule 16.7 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_16_07.c>`_
      - Required
      - A switch-expression shall not have essentially Boolean type
@@ -554,6 +573,13 @@ maintainers if you want to suggest a change.
        evaluation
      -
 
+   * - `Rule 20.12 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_12.c>`_
+     - A macro parameter used as an operand to the # or ## operators,
+       which is itself subject to further macro replacement, shall only
+       be used as an operand to these operators
+     - Required
+     - Variadic macros are allowed to violate the rule.
+
    * - `Rule 20.13 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_13.c>`_
      - Required
      - A line whose first token is # shall be a valid preprocessing


From xen-devel-bounces@lists.xenproject.org Sat Feb 10 02:48:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 10 Feb 2024 02:48:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678901.1056494 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYdPj-00044y-1u; Sat, 10 Feb 2024 02:48:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678901.1056494; Sat, 10 Feb 2024 02:48:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYdPi-00044r-UA; Sat, 10 Feb 2024 02:48:22 +0000
Received: by outflank-mailman (input) for mailman id 678901;
 Sat, 10 Feb 2024 02:48:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYdPi-00044h-J8; Sat, 10 Feb 2024 02:48:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYdPi-0004LM-Gn; Sat, 10 Feb 2024 02:48:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYdPh-0007YK-8y; Sat, 10 Feb 2024 02:48:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYdPh-000664-7Z; Sat, 10 Feb 2024 02:48:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=xUld/JsCOanU1BaFHgWBHyf20jZIRynHN0DhuNmwlko=; b=kQt1gF+zA73bW4kvnnJMSRgxZK
	AGJbOgOnx1w1/tr4MzGZ4e2zI+GyClptWNNnIhldJINwZicq8sTaa+T2+du3cTLU/nf3viK6JSdqG
	BLY74dHpE8uO8mcsh7cIwn+J6mbb0iGIZinSuYikWIWih0pH+GAM4YgOOKc1RYRzvaUw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184635-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184635: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt-qcow2:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
X-Osstest-Versions-That:
    xen=e11f5766503c0ff074b4e0f888bbfc931518a169
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 10 Feb 2024 02:48:21 +0000

flight 184635 xen-unstable real [real]
flight 184640 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184635/
http://logs.test-lab.xenproject.org/osstest/logs/184640/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-qcow2 17 guest-start/debian.repeat fail pass in 184640-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop      fail blocked in 184630
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184630
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184630
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184630
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184630
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184630
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184630
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail like 184630
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184630
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184630
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184630
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184630
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184630
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
baseline version:
 xen                  e11f5766503c0ff074b4e0f888bbfc931518a169

Last test of basis   184630  2024-02-09 00:40:40 Z    1 days
Testing same since   184635  2024-02-09 12:00:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Christian Lindig <christian.lindig@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Michal Orzel <michal.orzel@amd.com>
  Petr Beneš <w1benny@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               fail    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   e11f576650..4e5bd87d24  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369 -> master


From xen-devel-bounces@lists.xenproject.org Sat Feb 10 06:17:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 10 Feb 2024 06:17:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678913.1056503 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYgfQ-0006jM-QG; Sat, 10 Feb 2024 06:16:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678913.1056503; Sat, 10 Feb 2024 06:16:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYgfQ-0006jF-MF; Sat, 10 Feb 2024 06:16:48 +0000
Received: by outflank-mailman (input) for mailman id 678913;
 Sat, 10 Feb 2024 06:16:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYgfP-0006j5-Lh; Sat, 10 Feb 2024 06:16:47 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYgfP-0000N1-Dl; Sat, 10 Feb 2024 06:16:47 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYgfO-0002RF-Up; Sat, 10 Feb 2024 06:16:47 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYgfO-0007Dp-UL; Sat, 10 Feb 2024 06:16:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=e0jG1FGYXPooVxocnCgl1jMh7Poltsuq/AAuCMPSKuM=; b=G8cPPBVJEcAOMhKgY/V4Iflh7m
	5ONRbq1z3p0j0YwUoL8v4BKlhgy1tZB4CYz8aMpfLQFOoQ+092VkGPy6ABDqGABBY60MDHy/xPArf
	vPd3KtiNkY6pjHTS3V/+KfaZPIVUbGdl4jMc/VA0II+FbNnhMLZLwSdNGU4FCPRO96tA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184638-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184638: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=9ed18b0b7765ec3aa5f10843d1c3daaa1a225337
X-Osstest-Versions-That:
    linux=1f719a2f3fa67665578c759ac34fd3d3690c1a20
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 10 Feb 2024 06:16:46 +0000

flight 184638 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184638/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184633
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184633
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184633
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184633
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184633
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184633
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184633
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184633
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                9ed18b0b7765ec3aa5f10843d1c3daaa1a225337
baseline version:
 linux                1f719a2f3fa67665578c759ac34fd3d3690c1a20

Last test of basis   184633  2024-02-09 05:12:11 Z    1 days
Testing same since   184638  2024-02-09 20:12:33 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abhinav Kumar <quic_abhinavk@quicinc.com>
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Hansen <alexander.hansen@9elements.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexandre Ghiti <alexghiti@rivosinc.com>
  Alvin Lee <alvin.lee2@amd.com>
  Andrew Bresticker <abrestic@rivosinc.com>
  Ard Biesheuvel <ardb@kernel.org>
  Arnd Bergmann <arnd@arndb.de>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  Ben Dooks <ben.dooks@codethink.co.uk>
  Bjorn Helgaas <bhelgaas@google.com>
  Brian Welty <brian.welty@intel.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Danilo Krummrich <dakr@redhat.com>
  Dave Airlie <airlied@redhat.com>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Eugen Hristev <eugen.hristev@collabora.com>
  Fangzhi Zuo <jerry.zuo@amd.com>
  Fred Ai <fred.ai@bayhubtech.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Grzegorz Trzebiatowski <grzegorz.trzebiatowski@intel.com>
  Guenter Roeck <linux@roeck-us.net>
  Hamza Mahfooz <hamza.mahfooz@amd.com>
  Ira Weiny <ira.weiny@intel.com>
  Jacek Lawrynowicz <jacek.lawrynowicz@linux.intel.com>
  Jan Kiszka <jan.kiszka@siemens.com>
  Jani Nikula <jani.nikula@intel.com>
  Jiri Olsa <jolsa@kernel.org>
  Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
  Konrad Dybcio <konrad.dybcio@linaro.org>
  Krystian Pradzynski <krystian.pradzynski@intel.com>
  Kuogee Hsieh <quic_khsieh@quicinc.com>
  Li Ma <li.ma@amd.com>
  Li zeming <zeming@nfschina.com>
  Lijo Lazar <lijo.lazar@amd.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Loic Prylli <lprylli@netflix.com>
  Lucas De Marchi <lucas.demarchi@intel.com>
  Mario Limonciello <mario.limonciello@amd.com>
  Mark Rutland <mark.rutland@arm.com> [arm64]
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Matthew Auld <matthew.auld@intel.com>
  Matthew Brost <matthew.brost@intel.com>
  Maxime Ripard <mripard@kernel.org>
  Maxime Ripard <mripard@redhat.com>
  Nathan Chancellor <nathan@kernel.org>
  Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
  Palmer Dabbelt <palmer@rivosinc.com>
  Prike Liang <Prike.Liang@amd.com>
  Randy Dunlap <rdunlap@infradead.org>
  Richard Acayan <mailingradian@gmail.com>
  Rob Clark <robdclark@gmail.com>
  Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
  Saleemkhan Jamadar <saleemkhan.jamadar@amd.com>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Stanley.Yang <Stanley.Yang@amd.com>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Thomas Hellstrom <thomas.hellstrom@linux.intel.com>
  Thomas Hellström <thomas.hellstrom@linux.intel.com>
  Timur Tabi <ttabi@nvidia.com>
  Ulf Hansson <ulf.hansson@linaro.org>
  Vincent Chen <vincent.chen@sifive.com>
  Wachowski, Karol <karol.wachowski@intel.com>
  Wenjing Liu <wenjing.liu@amd.com>
  Xiaoming Wang <xiaoming.wang@intel.com>
  Yang Li <yang.lee@linux.alibaba.com>
  Zhang Rui <rui.zhang@intel.com>
  Zhenyu Wang <zhenyuw@linux.intel.com>
  Zhi Wang <zhi.wang.linux@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   1f719a2f3fa6..9ed18b0b7765  9ed18b0b7765ec3aa5f10843d1c3daaa1a225337 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sat Feb 10 10:17:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 10 Feb 2024 10:17:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678943.1056514 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYkQ7-0004N7-1k; Sat, 10 Feb 2024 10:17:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678943.1056514; Sat, 10 Feb 2024 10:17:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYkQ6-0004N0-Ug; Sat, 10 Feb 2024 10:17:14 +0000
Received: by outflank-mailman (input) for mailman id 678943;
 Sat, 10 Feb 2024 10:17:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rYkQ6-0004Mu-96
 for xen-devel@lists.xenproject.org; Sat, 10 Feb 2024 10:17:14 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYkQ4-0004uC-NN; Sat, 10 Feb 2024 10:17:12 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.102])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rYkQ4-0001QQ-HE; Sat, 10 Feb 2024 10:17:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=/Qsi4giPadtc5U+s50tYn6g6Lx4Qk4pRkfIOnkIZT7I=; b=vE4dzjNxWSnxfAXDpZ2PZH7H3L
	OqgnNIrhlq72+1n49H8yevSlpsaGLdGl13GAHB258SB6NdI1lA2WprxqiCtDBha9y9hbxXwt/QGIi
	bjGVlBz/Y7XBd4ouv6TEs7UAk25oUHCWAg9DS0FALvf1XHVxI69AbvUtXeplM0HTfJPM=;
Message-ID: <f86baad0-f113-4156-9c10-6910e8c63492@xen.org>
Date: Sat, 10 Feb 2024 10:17:10 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Return type of clean_and_invalidate_dcache_va_range
Content-Language: en-GB
To: Stefano Stabellini <sstabellini@kernel.org>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: Xen Devel <xen-devel@lists.xenproject.org>,
 Consulting <consulting@bugseng.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>, Jbeulich <jbeulich@suse.com>,
 Andrew Cooper3 <andrew.cooper3@citrix.com>, Roger Pau
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>
References: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
 <alpine.DEB.2.22.394.2402091402080.1925432@ubuntu-linux-20-04-desktop>
From: Julien Grall <julien@xen.org>
In-Reply-To: <alpine.DEB.2.22.394.2402091402080.1925432@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 09/02/2024 22:02, Stefano Stabellini wrote:
> On Fri, 9 Feb 2024, Nicola Vetrini wrote:
>> Hi all,
>>
>> In the context of violations of MISRA C:2012 Rule 17.7: "The value returned by
>> a function having non-void return type shall be used", I was looking at the
>> function "clean_and_invalidate_dcache_va_range". It has the following
>> signature on both arm and x86:
>>
>> static inline int clean_and_invalidate_dcache_va_range
>>      (const void *p, unsigned long size)
>>
>> The commit that introduced it for Arm ~9 years ago (71d64afe3e12: "arm: return
>> int from *_dcache_va_range") [1] mentions that on Arm it can't fail, but
>> supposedly it can on x86.
>>
>> However, as far as I can tell, for both arch-es the implementation now always
>> returns 0 [2][3], so perhaps the mention of -EOPNOTSUPP for x86 is no longer
>> true (I wasn't able to reconstruct if there was a time at which this was true,
>> even in the same commit that changed the return type to int).
>>
>> The question is: should the return type be void, since it appears that every
>> user is ignoring the returned value (violating the rule), except the one in
>> common/grant_table.c [4]?
> 
> Looking at the implementation on both ARM and x86, I am in favor of
> changing the return type to void
I think we need some consistency between all the cache flush helpers 
(clean_and_invalidate_dcache_va_range, invalidate_dcache_va_range() and 
clean_dcache_va_range()). They should all return a values or not return any.

That said, we have two other architectures in development. Are we saying 
this helpers will not need to (initially) return -EOPNOTSUPP?

If we need to return a value for any of the cache helpers, then I would 
consider to provide an extra helpers that would only be used by the 
grant-table code and return a value. Then transform all the existing 
helpers to return void.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sat Feb 10 12:24:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 10 Feb 2024 12:24:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678958.1056524 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYmOc-0004wJ-2V; Sat, 10 Feb 2024 12:23:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678958.1056524; Sat, 10 Feb 2024 12:23:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYmOb-0004wC-V4; Sat, 10 Feb 2024 12:23:49 +0000
Received: by outflank-mailman (input) for mailman id 678958;
 Sat, 10 Feb 2024 12:23:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYmOb-0004w1-6W; Sat, 10 Feb 2024 12:23:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYmOa-0007D1-Vy; Sat, 10 Feb 2024 12:23:49 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYmOa-0000Rn-Gl; Sat, 10 Feb 2024 12:23:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYmOa-0008G3-GF; Sat, 10 Feb 2024 12:23:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6+dREGC4fiyGGq6TpHaHGPQq1j28kqQeGzfX6PdVlZY=; b=uyWjXrtYsL8LjTKlxnP8HZ8wfp
	mJQYd7EendZb/8OIEBw1PPRVZ5/tEwUyyqxtZ6ekujpGyadlExpCtcCpl9Wxg90cEzPDl68IdL0d5
	cDxcbNFuLmQ7nylZNOpl9MXfByCUT3OnFu4mGq+/XZ/bRrfhR+zYrv59JWQB0saYAzrI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184641-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184641: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt-qcow2:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-i386-libvirt:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
X-Osstest-Versions-That:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 10 Feb 2024 12:23:48 +0000

flight 184641 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184641/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail in 184635 pass in 184641
 test-armhf-armhf-libvirt-qcow2 17 guest-start/debian.repeat fail in 184635 pass in 184641
 test-amd64-i386-libvirt      20 guest-start/debian.repeat  fail pass in 184635
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install    fail pass in 184635

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop fail in 184635 blocked in 184641
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184635
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184635
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184635
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184635
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184635
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184635
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184635
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184635
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184635
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184635
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184635
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
baseline version:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369

Last test of basis   184641  2024-02-10 02:50:46 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      fail    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sat Feb 10 15:16:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 10 Feb 2024 15:16:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678981.1056534 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYp5l-0001eN-CE; Sat, 10 Feb 2024 15:16:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678981.1056534; Sat, 10 Feb 2024 15:16:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYp5l-0001eG-9A; Sat, 10 Feb 2024 15:16:33 +0000
Received: by outflank-mailman (input) for mailman id 678981;
 Sat, 10 Feb 2024 15:16:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYp5k-0001e6-TS; Sat, 10 Feb 2024 15:16:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYp5k-0001sN-Hl; Sat, 10 Feb 2024 15:16:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYp5j-0004Wu-W9; Sat, 10 Feb 2024 15:16:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYp5j-0002Z6-Vc; Sat, 10 Feb 2024 15:16:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Hv5UeqKggUW1gYiPxRtzPju6DyrU/Y+Oh6bn86cBJm4=; b=ni/lI55YGrEegVgZdiqWiByKmg
	7zTIUAxMaVqY/cQo/FrmtmR4aD5pw+X2r7ceWSkk75ItuuUPxtel5Vi+CoBTU00tU86hVvJWyH8lS
	oVwZlZZcowb4Dn8duTuQeb4RMvW+WFajY+y+fU/SvXiqRdRrX+p/bpB1EYSMrK0l7rZs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184642-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184642: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=bdece5518d36b70ab18cf0157cb326025c07fe34
X-Osstest-Versions-That:
    libvirt=3f01ecd830204f128ef1d5f85c6b316b5f2cffdd
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 10 Feb 2024 15:16:31 +0000

flight 184642 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184642/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184632
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184632
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184632
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              bdece5518d36b70ab18cf0157cb326025c07fe34
baseline version:
 libvirt              3f01ecd830204f128ef1d5f85c6b316b5f2cffdd

Last test of basis   184632  2024-02-09 04:18:49 Z    1 days
Testing same since   184642  2024-02-10 04:20:51 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jonathon Jongsma <jjongsma@redhat.com>
  Ján Tomko <jtomko@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   3f01ecd830..bdece5518d  bdece5518d36b70ab18cf0157cb326025c07fe34 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sat Feb 10 19:25:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 10 Feb 2024 19:25:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.678997.1056543 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYsyB-0000Lb-F8; Sat, 10 Feb 2024 19:24:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 678997.1056543; Sat, 10 Feb 2024 19:24:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYsyB-0000LU-CS; Sat, 10 Feb 2024 19:24:59 +0000
Received: by outflank-mailman (input) for mailman id 678997;
 Sat, 10 Feb 2024 19:24:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYsy9-0000LK-Jf; Sat, 10 Feb 2024 19:24:57 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYsy9-0006oz-5j; Sat, 10 Feb 2024 19:24:57 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYsy8-0001lU-Nd; Sat, 10 Feb 2024 19:24:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYsy8-0007l8-N6; Sat, 10 Feb 2024 19:24:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=85p9SNt0xTYYIvmTAWWxtxarjwSitNgahPiuQoF3utY=; b=2srRYDeyK+v4skisskXubDanLV
	AhqmvVIwlenRQ1dc448Klkh32RwTDkJY2hZ55zkjb7czvq6uLakiVSmzTXe4aJxve3QWG0x3X5D2g
	67roJTPoPO5CFrcypAeWwrxXVy/lapjHibqN5d6QnG68VF01wb96NrojuZ6+Xva3iUJ4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184643-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184643: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4a7bbe7519b6a5d189feeba1d417c8ce9f7e852b
X-Osstest-Versions-That:
    linux=9ed18b0b7765ec3aa5f10843d1c3daaa1a225337
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 10 Feb 2024 19:24:56 +0000

flight 184643 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184643/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184638
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184638
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184638
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184638
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184638
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184638
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184638
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184638
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4a7bbe7519b6a5d189feeba1d417c8ce9f7e852b
baseline version:
 linux                9ed18b0b7765ec3aa5f10843d1c3daaa1a225337

Last test of basis   184638  2024-02-09 20:12:33 Z    0 days
Testing same since   184643  2024-02-10 06:21:50 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alice Chao <alice.chao@mediatek.com>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Wagner <dwagner@suse.de>
  Edward Adam Davis <eadavis@qq.com>
  Hannes Reinecke <hare@suse.de>
  Ilya Dryomov <idryomov@gmail.com>
  Ism Hong <ism.hong@gmail.com>
  Jeff Layton <jlayton@kernel.org>
  Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Ming Lei <ming.lei@redhat.com>
  Nekun <nekokun@firemail.cc>
  Paulo Alcantara <pc@manguebit.com>
  Rishabh Dave <ridave@redhat.com>
  SEO HOYOUNG <hy50.seo@samsung.com>
  Shyam Prasad N <sprasad@microsoft.com>
  Steve French <stfrench@microsoft.com>
  Xiubo Li <xiubli@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   9ed18b0b7765..4a7bbe7519b6  4a7bbe7519b6a5d189feeba1d417c8ce9f7e852b -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun Feb 11 02:50:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 11 Feb 2024 02:50:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679017.1056554 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYzui-0000Qd-Rc; Sun, 11 Feb 2024 02:49:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679017.1056554; Sun, 11 Feb 2024 02:49:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rYzui-0000QW-Mr; Sun, 11 Feb 2024 02:49:52 +0000
Received: by outflank-mailman (input) for mailman id 679017;
 Sun, 11 Feb 2024 02:49:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYzuh-0000QM-82; Sun, 11 Feb 2024 02:49:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYzug-0000xO-SV; Sun, 11 Feb 2024 02:49:50 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rYzug-0007HT-Ae; Sun, 11 Feb 2024 02:49:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rYzug-0002eo-AH; Sun, 11 Feb 2024 02:49:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/32th7cdrCdkBEUoU4Dc46e2uMoBLDePgrw+8DRugq8=; b=dqd1wJ1wY6XD8OdzMyAc2FhvUe
	dxv1+PTU91i15olqUbmnCYxOM/Lm/cmL+4cnqJrzihLKDiljjUJ5Tr++6UHn2fV6etyPdn1yp4tCE
	KfkymvlEUN3ny+bHC+DMWspYRSXQulHvF6rxbqjkOKysfejqeFbxYfml+lbhsmYaNUew=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184644-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184644: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=a5b6244cf87c50358f5562b8f07f7ac35fc7f6b0
X-Osstest-Versions-That:
    linux=4a7bbe7519b6a5d189feeba1d417c8ce9f7e852b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 11 Feb 2024 02:49:50 +0000

flight 184644 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184644/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184643
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184643
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184643
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184643
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184643
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184643
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184643
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184643
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                a5b6244cf87c50358f5562b8f07f7ac35fc7f6b0
baseline version:
 linux                4a7bbe7519b6a5d189feeba1d417c8ce9f7e852b

Last test of basis   184643  2024-02-10 06:21:50 Z    0 days
Testing same since   184644  2024-02-10 19:43:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Adam Goldman <adamg@pobox.com>
  Chaitanya Kulkarni <kch@nvidia.com>
  Fedor Pchelkin <pchelkin@ispras.ru>
  Francis Pravin <francis.p@samsung.com>
  Jan Kara <jack@suse.cz>
  Jens Axboe <axboe@kernel.dk>
  Keith Busch <kbusch@kernel.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Maurizio Lombardi <mlombard@redhat.com>
  Namjae Jeon <linkinjeon@kernel.org>
  Randy Dunlap <rdunlap@infradead.org>
  Sathyavathi M <sathya.m@samsung.com>
  Steve French <stfrench@microsoft.com>
  Takashi Sakamoto <o-takashi@sakamocchi.jp>
  Tasmiya Nalatwad <tasmiya@linux.vnet.ibm.com>
  Tejun Heo <tj@kernel.org>
  Yang Li <yang.lee@linux.alibaba.com>
  Yi Sun <yi.sun@unisoc.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   4a7bbe7519b6..a5b6244cf87c  a5b6244cf87c50358f5562b8f07f7ac35fc7f6b0 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun Feb 11 09:20:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 11 Feb 2024 09:20:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679038.1056564 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZ60F-00053r-3h; Sun, 11 Feb 2024 09:19:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679038.1056564; Sun, 11 Feb 2024 09:19:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZ60F-00053k-0t; Sun, 11 Feb 2024 09:19:59 +0000
Received: by outflank-mailman (input) for mailman id 679038;
 Sun, 11 Feb 2024 09:19:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZ60C-00053a-TT; Sun, 11 Feb 2024 09:19:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZ60C-0000Yl-Ia; Sun, 11 Feb 2024 09:19:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZ60C-00020l-3q; Sun, 11 Feb 2024 09:19:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZ60C-0005BT-3L; Sun, 11 Feb 2024 09:19:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=b1Wwbp+ON9ZErE/PlYhXsqggLFrHwKYlxH2Cwcs1Ork=; b=ObC+vAd9iFDpqLXiY6wPWXy5ts
	g8nM+GDDKvi67co5QOwkqe12KF8l0DQxM6kuBwWJ+3q05aQtAiNXZydtoDYnMLbsJyTPdOMfcZx9f
	YCH2GfFPCxV+dj9L0otcNrO1mMILmQqag+8peycSJQSP1442zhc4HA8azMVORfn++eBE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184645-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184645: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-i386-libvirt:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-multivcpu:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
X-Osstest-Versions-That:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 11 Feb 2024 09:19:56 +0000

flight 184645 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184645/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-libvirt 20 guest-start/debian.repeat fail in 184641 pass in 184645
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install fail in 184641 pass in 184645
 test-armhf-armhf-xl-multivcpu  8 xen-boot                  fail pass in 184641

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail in 184641 never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail in 184641 never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184635
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184641
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184641
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184641
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184641
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184641
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184641
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184641
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184641
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184641
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184641
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184641
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
baseline version:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369

Last test of basis   184645  2024-02-11 01:53:55 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun Feb 11 13:05:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 11 Feb 2024 13:05:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679061.1056573 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZ9Wd-0005Bw-Qa; Sun, 11 Feb 2024 13:05:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679061.1056573; Sun, 11 Feb 2024 13:05:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZ9Wd-0005Bp-Nw; Sun, 11 Feb 2024 13:05:39 +0000
Received: by outflank-mailman (input) for mailman id 679061;
 Sun, 11 Feb 2024 13:05:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZ9Wb-0005Ba-T1; Sun, 11 Feb 2024 13:05:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZ9Wb-0004Kc-Cd; Sun, 11 Feb 2024 13:05:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZ9Wa-0007Tp-Vj; Sun, 11 Feb 2024 13:05:37 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZ9Wa-0000t2-VD; Sun, 11 Feb 2024 13:05:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pjNkW7DGOfJpUjTzGwKHJ6ZQ8LTDGYKtX4Z5QilwHcE=; b=pCpzEl9gwb6yenwFKBTj30/N1k
	G7eJ1hi9OEkNwiaf71ECqqfW62Y2Td1jZsND416u4tQJiNuRgE8vPSiXzV1YlUS/Fu0SI1uT8fiGm
	Aea7TJH5aPSr6B7WCBUpA+KYGf5PMPwR4hM0Jd/MrQ/4ESPi+xeVJasjzReFRX3Vzr/0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184646-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184646: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=7521f258ea303c827434c101884b62a2b137a942
X-Osstest-Versions-That:
    linux=a5b6244cf87c50358f5562b8f07f7ac35fc7f6b0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 11 Feb 2024 13:05:36 +0000

flight 184646 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184646/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184644
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184644
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184644
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184644
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184644
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184644
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184644
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184644
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                7521f258ea303c827434c101884b62a2b137a942
baseline version:
 linux                a5b6244cf87c50358f5562b8f07f7ac35fc7f6b0

Last test of basis   184644  2024-02-10 19:43:36 Z    0 days
Testing same since   184646  2024-02-11 02:51:18 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Morton <akpm@linux-foundation.org>
  Chengming Zhou <zhouchengming@bytedance.com>
  Dylan Hatch <dylanbhatch@google.com>
  Johannes Weiner <hannes@cmpxchg.org>
  John Moon <john@jmoon.dev>
  kernel test robot <oliver.sang@intel.com>
  Leo Yan <leo.yan@linux.dev>
  Linus Torvalds <torvalds@linux-foundation.org>
  Miaohe Lin <linmiaohe@huawei.com>
  Michal Hocko <mhocko@suse.com>
  Muchun Song <muchun.song@linux.dev>
  Muhammad Usama Anjum <usama.anjum@collabora.com>
  Oleg Nesterov <oleg@redhat.com>
  Oscar Salvador <osalvador@suse.de>
  Prakash Sangappa <prakash.sangappa@oracle.com>
  Ryan Roberts <ryan.roberts@arm.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  SeongJae Park <sj@kernel.org>
  Sergey Senozhatsky <senozhatsky@chromium.org>
  Shakeel Butt <shakeelb@google.com>
  Suren Baghdasaryan <surenb@google.com>
  Trilok Soni <quic_tsoni@quicinc.com>
  Yosry Ahmed <yosryahmed@google.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   a5b6244cf87c..7521f258ea30  7521f258ea303c827434c101884b62a2b137a942 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun Feb 11 15:38:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 11 Feb 2024 15:38:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679070.1056584 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZBuG-0004wY-G4; Sun, 11 Feb 2024 15:38:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679070.1056584; Sun, 11 Feb 2024 15:38:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZBuG-0004wR-DC; Sun, 11 Feb 2024 15:38:12 +0000
Received: by outflank-mailman (input) for mailman id 679070;
 Sun, 11 Feb 2024 15:10:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ZVOd=JU=aladdin.ru=D.Dulov@srs-se1.protection.inumbo.net>)
 id 1rZBTI-0002dl-Dr
 for xen-devel@lists.xenproject.org; Sun, 11 Feb 2024 15:10:20 +0000
Received: from mail-out.aladdin-rd.ru (mail-out.aladdin-rd.ru [91.199.251.16])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a6f06610-c8ef-11ee-98f5-efadbce2ee36;
 Sun, 11 Feb 2024 16:10:13 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a6f06610-c8ef-11ee-98f5-efadbce2ee36
From: Daniil Dulov <d.dulov@aladdin.ru>
To: Juergen Gross <jgross@suse.com>
CC: Daniil Dulov <d.dulov@aladdin.ru>, Boris Ostrovsky
	<boris.ostrovsky@oracle.com>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Jeremy Fitzhardinge
	<jeremy.fitzhardinge@citrix.com>, <xen-devel@lists.xenproject.org>,
	<iommu@lists.linux-foundation.org>, <linux-kernel@vger.kernel.org>,
	<lvc-project@linuxtesting.org>
Subject: [PATCH] swiotlb-xen: remove redundant NULL check
Date: Sun, 11 Feb 2024 07:09:58 -0800
Message-ID: <20240211150958.4112-1-d.dulov@aladdin.ru>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.0.20.125]
X-ClientProxiedBy: EXCH-2016-03.aladdin.ru (192.168.1.103) To
 EXCH-2016-01.aladdin.ru (192.168.1.101)

In this case hwdev cannot be NULL, so remove redundant NULL check.

Found by Linux Verification Center (linuxtesting.org) with SVACE.

Fixes: b097186fd29d ("swiotlb-xen: SWIOTLB library for Xen PV guest with PCI passthrough.")
Signed-off-by: Daniil Dulov <d.dulov@aladdin.ru>
---
 drivers/xen/swiotlb-xen.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 2b385c1b4a99..b166f6efea26 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -305,7 +305,7 @@ xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 	if (!ret)
 		return ret;
 
-	if (hwdev && hwdev->coherent_dma_mask)
+	if (hwdev->coherent_dma_mask)
 		dma_mask = hwdev->coherent_dma_mask;
 
 	/* At this point dma_handle is the dma address, next we are
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 12 03:00:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 03:00:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679136.1056610 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZMYK-0001Yb-PD; Mon, 12 Feb 2024 03:00:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679136.1056610; Mon, 12 Feb 2024 03:00:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZMYK-0001YU-K2; Mon, 12 Feb 2024 03:00:16 +0000
Received: by outflank-mailman (input) for mailman id 679136;
 Mon, 12 Feb 2024 03:00:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZMYJ-0001YH-Al; Mon, 12 Feb 2024 03:00:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZMYJ-0006J3-0H; Mon, 12 Feb 2024 03:00:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZMYI-0004ff-Gl; Mon, 12 Feb 2024 03:00:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZMYI-0003aB-GD; Mon, 12 Feb 2024 03:00:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=GQJGo5cx/gA0IK7Prn2ajynSnG7JkxCul9Ov8RLlTqY=; b=o+zaXe+ztSRSnj3yI0mxwVgeKg
	2dJ+W7i+O1wHGcsbAV+DwsVsDdmFkBbgkUVuA5vn8tzEPNtC65x6P6AIvdC5cD8K4oDtt84bKJkzm
	YOj/zQ7+260XoKqfwmBxzo7K5JgDTVU4wFEB8gTTZioUSbbq6jN6EN4cwEppn9kk3/P4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184647-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184647: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=2766f59ca44e517a1d6226979c784b026f0e89c2
X-Osstest-Versions-That:
    linux=7521f258ea303c827434c101884b62a2b137a942
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 12 Feb 2024 03:00:14 +0000

flight 184647 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184647/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184646
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184646
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184646
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184646
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184646
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184646
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184646
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184646
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                2766f59ca44e517a1d6226979c784b026f0e89c2
baseline version:
 linux                7521f258ea303c827434c101884b62a2b137a942

Last test of basis   184646  2024-02-11 02:51:18 Z    1 days
Testing same since   184647  2024-02-11 20:13:59 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aleksander Mazur <deweloper@wp.pl>
  Andrei Vagin <avagin@google.com>
  Borislav Petkov (AMD) <bp@alien8.de>
  Dave Hansen <dave.hansen@linux.intel.com>
  Frederic Weisbecker <frederic@kernel.org>
  H. Peter Anvin <hpa@zytor.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Paul E. McKenney <paulmck@kernel.org>
  Qiuxu Zhuo <qiuxu.zhuo@intel.com>
  Thomas Gleixner <tglx@linutronix.de>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   7521f258ea30..2766f59ca44e  2766f59ca44e517a1d6226979c784b026f0e89c2 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 03:55:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 03:55:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679143.1056620 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZNPP-0007NG-RY; Mon, 12 Feb 2024 03:55:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679143.1056620; Mon, 12 Feb 2024 03:55:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZNPP-0007N9-OS; Mon, 12 Feb 2024 03:55:07 +0000
Received: by outflank-mailman (input) for mailman id 679143;
 Mon, 12 Feb 2024 03:55:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lYCt=JV=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1rZNPO-0007Mx-2e
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 03:55:06 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7facc492-c95a-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 04:55:02 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-55a035669d5so4005002a12.2
 for <xen-devel@lists.xenproject.org>; Sun, 11 Feb 2024 19:55:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7facc492-c95a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707710102; x=1708314902; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=qxZerN7J+zgn6JEepd90bhcVWF4c3VwoeTHfy8Xj0xw=;
        b=m7bb+paexxUW2Et7fsRmGpFIXZrTCY7xrL7ZILQPb23t49rikfEwOdz6eyGrWryof8
         wcTDVnwdAJRxsEHdbmb0kx7EINDis4mVewOWqfdQpfHhn+PIzWHJR96604h6r2d+kvju
         4tdG/KFxxWpkr+yTwvQtub+Zez0D/HrqgqIYqsUnvUSTTS7wd8Ubd8QMoZ2YdxEWrtl2
         kwHOC0LPlGFzjnN5xD32kqILpUwqkEpVuPKJW/Gs8gQXwrtrvvwx9lxRxty3B133FlJk
         lozOkD9ybFGlDkeLg/F2EYARVhWv6i7EYr4QwHn2IhluugYAgvRodROQ/XCkTacD3VnW
         odgQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707710102; x=1708314902;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=qxZerN7J+zgn6JEepd90bhcVWF4c3VwoeTHfy8Xj0xw=;
        b=dR9LTSzW+2E8s/JSp/tKbaxCRrboiHU8F9mcMDGkMT9vRbVPWCOSDxn3WbZwKS8aCg
         YYdLD7/TdMnY4gs8l2aO2P98+C9UQsqQs7XNkwyvV3oUw0393MXMIKWt37c2rNj0ZUSg
         CON4IfI6kdptmxqzJDxg/wuzXfrqzgn1M4TnCcNA4E1uiB/XhNk/FV8hqSbHuCwMBhfB
         OXFTyVBUIrxo/2O2TQJn/LCsoXAJm6gV6SwgaXhAKfbv2TDD4eYP9c8EqhPyEXUU6TBZ
         fx7NfHXO6nl61lQcoZF1GZYnDQj7O9Pz19nm0q7v5yawwCbnaZjandUObtGf1plrpw1T
         wYIQ==
X-Gm-Message-State: AOJu0YwS+Sira9+fLx+Nm/B6d2FlaizViljxp564y3+g6mCrI1WM1Oub
	6xcYyGLVtmLDc8C16KLJ41tgIFy0L86ZNV/Dhnn4GYaoRBKnwzfrTS2fi1tabr8G9sqRvwNc381
	BxVzSUlqkN+KC8te2ZiTaPRtN+QQ=
X-Google-Smtp-Source: AGHT+IERMDd1vAiNLNAWk93evwyJt0ttAqbO00uciJjOSmu2gVKfz0QGiuGwgjYZ9lK/+Rjp4AOkdlQrvrrWf9jG7FA=
X-Received: by 2002:aa7:d685:0:b0:55f:c930:8b8a with SMTP id
 d5-20020aa7d685000000b0055fc9308b8amr3488671edr.6.1707710101436; Sun, 11 Feb
 2024 19:55:01 -0800 (PST)
MIME-Version: 1.0
References: <20240201183024.145424-1-jandryuk@gmail.com> <20240201183024.145424-2-jandryuk@gmail.com>
 <cd9e0194-5814-4735-bca4-df2577102c19@perard> <a0145f9d-654d-4336-b5bb-69883543b919@suse.com>
 <CAKf6xptDuYr2mT0w39Ezb3gTETYoZBpFUQ1Zqe6Kb9UeRQw7EQ@mail.gmail.com> <714d7de9-5a6e-4d30-bd2d-5b84ae24f3dd@suse.com>
In-Reply-To: <714d7de9-5a6e-4d30-bd2d-5b84ae24f3dd@suse.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Sun, 11 Feb 2024 22:54:49 -0500
Message-ID: <CAKf6xptMs8TRmFnOMiCFaMZ8aiNnaNdgmtxk+bT8-MDsSJ4yjw@mail.gmail.com>
Subject: Re: [PATCH 1/4] block-common: Fix same_vm for no targets
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony PERARD <anthony.perard@citrix.com>, xen-devel@lists.xenproject.org, 
	Wei Liu <wl@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 8, 2024 at 2:50=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wrot=
e:
>
> On 08.02.2024 03:25, Jason Andryuk wrote:
> > On Wed, Feb 7, 2024 at 7:50=E2=80=AFAM Jan Beulich <jbeulich@suse.com> =
wrote:
> >>
> >> On 06.02.2024 12:45, Anthony PERARD wrote:
> >>> On Thu, Feb 01, 2024 at 01:30:21PM -0500, Jason Andryuk wrote:
> >>>> same_vm is broken when the two main domains do not have targets.  ot=
vm
> >>>> and targetvm are both missing, which means they get set to -1 and th=
en
> >>>> converted to empty strings:
> >>>>
> >>>> ++10697+ local targetvm=3D-1
> >>>> ++10697+ local otvm=3D-1
> >>>> ++10697+ otvm=3D
> >>>> ++10697+ othervm=3D/vm/cc97bc2f-3a91-43f7-8fbc-4cb92f90b4e4
> >>>> ++10697+ targetvm=3D
> >>>> ++10697+ local frontend_uuid=3D/vm/844dea4e-44f8-4e3e-8145-325132a31=
ca5
> >>>>
> >>>> The final comparison returns true since the two empty strings match:
> >>>>
> >>>> ++10697+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 =3D /vm/cc97bc=
2f-3a91-43f7-8fbc-4cb92f90b4e4 -o '' =3D /vm/cc97bc2f-3a91-43f7-8fbc-4cb92f=
90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 =3D '' -o '' =3D '' ']'
> >>>>
> >>>> Replace -1 with distinct strings indicating the lack of a value and
> >>>> remove the collescing to empty stings.  The strings themselves will =
no
> >>>> longer match, and that is correct.
> >>>>
> >>>> ++12364+ '[' /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 =3D /vm/cc97bc=
2f-3a91-43f7-8fbc-4cb92f90b4e4 -o 'No target' =3D /vm/cc97bc2f-3a91-43f7-8f=
bc-4cb92f90b4e4 -o /vm/844dea4e-44f8-4e3e-8145-325132a31ca5 =3D 'No other t=
arget' -o 'No target' =3D 'No other target' ']'
> >>>>
> >>>> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
> >>>
> >>> Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
> >>
> >> I've committed this, but I take the absence of a Fixes: tag as indicat=
ion
> >> that this doesn't want/need backporting.
> >
> > Hmmm, maybe this should have a Fixes.  Sorry I didn't investigate that
> > better before submission.
> >
> > Looks like this would be the commit:
> > https://xenbits.xen.org/gitweb/?p=3Dxen.git;a=3Dcommitdiff;h=3Df3a7ca02=
400d1c416e97451b4aebfaf608fc8192
> >
> > f3a7ca02400d1 ("hotplug/Linux: fix same_vm check in block script")
> >
> > I need to circle back on this.  IIRC, when I set up a conflicting
> > assignment of a writable disk to two VMs with block-tap, it was
> > allowed and not denied.  That is what prompted this change.
> >
> > I'll have to double check there isn't something in the regular block
> > that might prevent that.
>
> Okay, I'll wait for a result here before deciding whether to queue.

Yes, it should be backported.  This patch prevents sharing writable
block devs.  Files are still broken because of issues identified
previously here:
https://lore.kernel.org/xen-devel/CAKf6xpv-U91nF2Fik7GRN3SFeOWWcdR5R+ZcK5fg=
ojE+-D43sg@mail.gmail.com/

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 07:23:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 07:23:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679242.1056678 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZQew-0000oY-31; Mon, 12 Feb 2024 07:23:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679242.1056678; Mon, 12 Feb 2024 07:23:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZQev-0000oR-Vb; Mon, 12 Feb 2024 07:23:21 +0000
Received: by outflank-mailman (input) for mailman id 679242;
 Mon, 12 Feb 2024 07:23:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=u6L+=JV=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rZQev-0000oL-El
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 07:23:21 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 98525ea5-c977-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 08:23:19 +0100 (CET)
Received: from [192.168.1.140] (unknown [81.56.166.244])
 by support.bugseng.com (Postfix) with ESMTPSA id 0DCF64EE0737;
 Mon, 12 Feb 2024 08:23:18 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 98525ea5-c977-11ee-98f5-efadbce2ee36
Message-ID: <923c959c-8938-46d4-a45b-ed3d3409b58c@bugseng.com>
Date: Mon, 12 Feb 2024 08:23:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v4 1/4] xen: add SAF deviation for debugging and
 logging effects
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Jan Beulich <jbeulich@suse.com>, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Dario Faggioli <dfaggioli@suse.com>,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 xen-devel@lists.xenproject.org
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <7c6aeedac626b171ed44df50ce5e3e2c76593f60.1706886631.git.simone.ballarin@bugseng.com>
 <09869431-ed00-4102-93fd-84c697a5c950@suse.com>
 <851eb6ec-558b-4a89-a31a-34046730bb55@bugseng.com>
 <41474fe7-c65f-4244-a455-0aaa0e4315e3@suse.com>
 <e48f4d49-274c-4bd6-8e3b-1ecaee6fa602@bugseng.com>
 <alpine.DEB.2.22.394.2402091410490.1925432@ubuntu-linux-20-04-desktop>
Content-Language: en-US
From: Simone Ballarin <simone.ballarin@bugseng.com>
Organization: BUGSENG
In-Reply-To: <alpine.DEB.2.22.394.2402091410490.1925432@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 09/02/24 23:13, Stefano Stabellini wrote:
> On Wed, 7 Feb 2024, Simone Ballarin wrote:
>> On 07/02/24 11:24, Jan Beulich wrote:
>>> On 07.02.2024 11:03, Simone Ballarin wrote:
>>>> On 06/02/24 13:04, Jan Beulich wrote:
>>>>> On 02.02.2024 16:16, Simone Ballarin wrote:
>>>>>> Rule 13.1: Initializer lists shall not contain persistent side effects
>>>>>>
>>>>>> Effects caused by debug/logging macros and functions (like ASSERT,
>>>>>> __bad_atomic_size,
>>>>>> LOG, etc ...) that crash execution or produce logs are not dangerous
>>>>>> in initializer
>>>>>> lists. The evaluation order in abnormal conditions is not relevant.
>>>>>> Evaluation order
>>>>>> of logging effects is always safe.
>>>>>
>>>>> I thought I said so before: When talking of just logging, evaluation
>>>>> order
>>>>> may very well have a impact on correctness. Therefore we shouldn't mix
>>>>> debugging and logging.
>>>>
>>>> My general feeling was that changes like the following one are not
>>>> supported by
>>>> the community:
>>>>
>>>> - x = { .field1 = function_with_logs_effects() /*other eventual code*/ };
>>>> + int field1 = function_with_logs_effects();
>>>> + x = { .field1 = field1 /*other eventual code*/};
>>>>
>>>> so I tried to deviate as much as possible.
>>>>
>>>> If having log effects is a good reason to do changes like the above, I can
>>>> propose a patch in that sense.
>>>
>>> Just to avoid misunderstandings: I'm not advocating for changes like the
>>> one you outline above. I simply consider the rule too strict: There's
>>> nothing at risk when there's just a single operation with side effects
>>> in an initializer.
>>
>> I agree for the safe cases such as single item list initializers
>> (independently
>> by the number of effect contained in io_apic_read).
>> In fact, I was about to propose in another patch to deviate cases like:
>>
>> union IO_APIC_reg_01 reg_01 = { .raw = io_apic_read(idx, 1) };
>> union IO_APIC_reg_02 reg_02 = { .raw = io_apic_read(idx, 2) };
>>
>>> Even when there are multiple such operations, whether
>>> there's anything at risk depends on whether any of the side effects
>>> actually collide. In a number of cases the compiler would actually warn
>>> (and thus, due to -Werror, the build would fail).
>>>
>>
>> I don't completely agree on that, this requires an in-depth comprehension
>> of the code especially when complex call chains are involved. Moreover
>> these deviations need to be maintained when one of the function involved
>> changes.
>>
>>> The primary purpose of my comment here was that we need to please
>>> separate debugging from logging side effects.
>>>
>>
>> Ok, I will work in that sense.
>>
>>>>>> --- a/xen/arch/arm/guestcopy.c
>>>>>> +++ b/xen/arch/arm/guestcopy.c
>>>>>> @@ -110,26 +110,34 @@ static unsigned long copy_guest(void *buf,
>>>>>> uint64_t addr, unsigned int len,
>>>>>>     unsigned long raw_copy_to_guest(void *to, const void *from,
>>>>>> unsigned int len)
>>>>>>     {
>>>>>>         return copy_guest((void *)from, (vaddr_t)to, len,
>>>>>> -                      GVA_INFO(current), COPY_to_guest |
>>>>>> COPY_linear);
>>>>>> +                      /* SAF-4-safe No persistent side effects */
>>>>>> +                      GVA_INFO(current),
>>>>>
>>>>> I _still_ think this leaves ambiguity. The more that you need to look
>>>>> up GVA_INFO() to recognize what this is about.
>>>>
>>>>
>>>> Just to recap: here the point is that current reads a register with a
>>>> volatile asm, so the
>>>> violation is in the expansion of GVA_INFO(current). Both GVA_INFO and
>>>> current taken alone
>>>> are completely fine, so this is the only place where a SAF comment can be
>>>> placed.
>>>>
>>>> The exapansion is:
>>>> ((copy_info_t) { .gva = { ((*({ unsigned long __ptr; __asm__ ("" :
>>>> "=r"(__ptr) : "0"(&
>>>>      per_cpu__curr_vcpu)); (typeof(&per_cpu__curr_vcpu)) (__ptr + (({
>>>> uint64_t _r; asm volatile("mrs  %0, ""TPIDR_EL2" : "=r"
>>>>      (_r)); _r; }))); }))) } }), (1U << 1) | (1U << 2));
>>>>
>>>> My proposals are:
>>>> 1) address the violation moving the current expansion outside (extra
>>>> variable);
>>>> 2) put a more detailed comment to avoid the ambiguity;
>>>> 3) use an ECL deviation for GVA_INFO(current).
>>>>
>>>> Do you have any preference or proposal?
>>>
>>> Imo 3 is not an option at all. Probably 1 wouldn't be too bad here, but
>>> I still wouldn't like it (as matching a general pattern I try to avoid:
>>> introducing local variables that are used just once and don't meaningfully
>>> improve e.g. readability). Therefore out of what you list, 2 would remain.
>>> But I'm not happy with a comment here either - as per above, there's
>>> nothing that can go wrong here as long as there's only a single construct
>>> with side effect(s).
>>>
>> So, would be changing the SAF in:
>> /* SAF-<new_id>-safe single item initializer */
>>
>> OK for you?
>>
>>>>>> --- a/xen/arch/x86/hvm/hvm.c
>>>>>> +++ b/xen/arch/x86/hvm/hvm.c
>>>>>> @@ -800,6 +800,7 @@ static int cf_check hvm_save_cpu_ctxt(struct vcpu
>>>>>> *v, hvm_domain_context_t *h)
>>>>>>     {
>>>>>>         struct segment_register seg;
>>>>>>         struct hvm_hw_cpu ctxt = {
>>>>>> +        /* SAF-3-safe effects for debugging/logging reasons are safe
>>>>>> */
>>>>>>             .tsc = hvm_get_guest_tsc_fixed(v,
>>>>>> v->domain->arch.hvm.sync_tsc),
>>>>>
>>>>> A prereq for this imo is that the function take const struct vcpu *.
>>>>> But I'm not sure that'll suffice. The function can change at any time,
>>>>> rendering the comment here stale perhaps without anyone noticing.
>>>>>
>>>>
>>>> IMO It isn't a strict prereq, but it would make everything more clear.
>>>>
>>>> In any case, apart adding the const, I do not see other easy solutions.
>>>> Would you give me your ack if I change the function signature?
>>>
>>> Well, as said: I'm not sure that'll suffice.
>>>
>>>> Another possible solutions would be documenting the function in the new
>>>> JSON file with a special attribute like only_debug_effect. Of course,
>>>> this still requires keeping the JSON up to date in case of changes.
>>>
>>> Exactly. So wouldn't really help.
>>>
>>> In any event I'd like to ask that you consider splitting up this patch,
>>> such that you won't need multiple acks for any of the parts. That'll
>>> also allow focusing on one aspect at a time in reviews.
>>>
>>
>> Ok, but please consider that the JSON file has been precisely added to deal
>> with these cases (avoiding __attribute__). If we are scared to use it, it
>> becomes
>> meaningless.
>>
>> @Stefano maybe your opinion could help.
> 
> I do think the JSON file is a useful tool in our toolbox. It is also
> fair to say that it is one that we should be careful about using as it
> requires manual updates from time to time.
> 
> So, I certainly consider it an option, but I prefer your other
> suggestion of dropping 13.1 in favor of 13.2. With that change, would
> the need for a deviation in this function go away?
> 

This is only a violation of 13.1, so dropping it in favour of 13.2 it's
enough.

-- 
Simone Ballarin, M.Sc.

Field Application Engineer, BUGSENG (https://bugseng.com)



From xen-devel-bounces@lists.xenproject.org Mon Feb 12 08:13:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 08:13:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679310.1056720 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZRR4-0002YF-LQ; Mon, 12 Feb 2024 08:13:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679310.1056720; Mon, 12 Feb 2024 08:13:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZRR4-0002Y8-IM; Mon, 12 Feb 2024 08:13:06 +0000
Received: by outflank-mailman (input) for mailman id 679310;
 Mon, 12 Feb 2024 08:13:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tvA+=JV=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rZRR3-0002Y2-Pb
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 08:13:05 +0000
Received: from NAM04-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2408::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8a6c7cee-c97e-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 09:13:03 +0100 (CET)
Received: from CY5PR16CA0013.namprd16.prod.outlook.com (2603:10b6:930:10::32)
 by PH0PR12MB8176.namprd12.prod.outlook.com (2603:10b6:510:290::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.24; Mon, 12 Feb
 2024 08:12:57 +0000
Received: from CY4PEPF0000E9D5.namprd05.prod.outlook.com
 (2603:10b6:930:10:cafe::52) by CY5PR16CA0013.outlook.office365.com
 (2603:10b6:930:10::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.35 via Frontend
 Transport; Mon, 12 Feb 2024 08:12:57 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D5.mail.protection.outlook.com (10.167.241.76) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Mon, 12 Feb 2024 08:12:57 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 12 Feb
 2024 02:12:55 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 12 Feb 2024 02:12:54 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a6c7cee-c97e-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oYm53e209R3JrgrW1CTzsKrKZP1yEenUZjedMzKG6wYacBUPyWqwHEIMgB7AunkPa8nBt10T7VhGj9thQlGcw5ZtXkyrwbJJhkM3yHllNfS27B6W/VqzWEWfhsFrtTxLe/v1MRQoPlnemvEgA0MQ0r6/rpB6oWomQKeSEq3iZLKY1M+V7yfXWifZVu0ztiuF2w3tD/A9PTqUKNIzo0csOYy2p2oZY/cZ/WCLqgxIgu5v6VluQFBr3TL1fyzG8PZ+2s/SvKpbcQP4zKvR/RjHwQTy0PCayEuuvPtWZG38Yz8axTrb5+R2QTifmlUA5qVYxA8HyOOsh5KyYfstKtFbew==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=2BCuVF2LjiJkAqCSItuhKHrn8l6x2R3Y3ZiE8ZP9DFY=;
 b=NKszdVUFHyyYOIz9LxtboZ6fvA+7rwWCn/VxGkDokGF/VYiKkWmVr/qD8HMs4Feed7u+tPnnsPLWzmSjGX5oOyRb3k9T7ECHGj2hzHAPkjbkRJ9NK+nuZN95lmrQzrcOp1kLSNCeXekOuz9rizEGQOCajeVsTdrf5AbeuEzChuF2l5owx9GJLSEIFtSffGZDJQbTXxfPG7+AhDLHHbiAdIo8iUeYw8nwd90FqKbjnS8QTmI96dxGpsBAJmV5esLOSHT0JxGxhCUVus20py7PVStmVR5D7+TAyAmnuvjWMcMe/YODIEqfvn8ais9C8Bj43nFHbAlwNg60+8LMl7C+2g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=gmail.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2BCuVF2LjiJkAqCSItuhKHrn8l6x2R3Y3ZiE8ZP9DFY=;
 b=RPXMFCOCZ3fUvD+3EmLcclcXUFgjJqPhdr7l+qtCD2MrLkWNMa/BSDaHVgXtVDY4jH8/EIt38LwRQBhJgTIj91ez/wKP2ODtzpUsgGmGsQ+LyGC2Y2dmVrr3KIsIejFOgaICSasoBuyHKrknJJ43tqqeUWJPGQ4f6GQlI9EFG60=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <c516259d-46f1-4bef-b4d0-4cfe18a3cb8a@amd.com>
Date: Mon, 12 Feb 2024 09:12:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 1/7] automation: ensure values in
 EXTRA_FIXED_RANDCONFIG are separated by new line
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Doug Goldstein <cardoe@cardoe.com>, Stefano Stabellini
	<sstabellini@kernel.org>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
 <8b6f208f0b4327392cfddbfa6809117cc7686750.1707499278.git.oleksii.kurochko@gmail.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <8b6f208f0b4327392cfddbfa6809117cc7686750.1707499278.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D5:EE_|PH0PR12MB8176:EE_
X-MS-Office365-Filtering-Correlation-Id: b4eb8e14-8a30-4254-86a1-08dc2ba26bdf
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	1DGtTL6MzxyVMhp9WpbPsVBO3lMxrB2ntAqCsMijAmr1ChWd6bBnXkBN6Ngul6DjLvpteBO4wvw6EDTRILRmh5wcF8u85UN2Rzc4k7w/NqhA8A9T8hoOpsC1bv9xiJjhbn7nwQ52caZp5t9WDr8Go4U0zsnqkbOQbGifXv10u17xVd4gGUC5+HEXc0q/cUdEcuR4k7tc/FN6vgTOH1S6Xv7gclprItibTu1r2z/P0Kn/wt4N1zo6HulwyCtZA9vLfZTUWVs9BSpFa0mseePQ9iUrBTMsUwIey0Z5aphwDYMCqHckni8J4uct4j6I7f+3HW4wzQAeTi6V+bcSSsBY8aaNiS7Sy1+wlZnl6u+aWlVS4q1+/lprLKRuH1OPl5osoNrCDJKdWazN2mkBhMMTVslxzqgqi8pb5Ivz8KZI4DQ41AUC5cT6JFmkuIbqB4cMsltv0nEQf/nKkmvMvVfkARFF7ge1VxIXUcSn5+njazzTvh1qD2lbgNww+/cr3MKXqBcxysFOFCsuxa/jcVsHxgcjrHwtMr/Nfkc64glpCL9I4gE5ZEYMOr/Jhb/UR9D1X+AzoGLFWcLRpyJEKQb5rK+b463HZa4SfhwAEp4xAOw=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(376002)(396003)(346002)(39860400002)(230922051799003)(186009)(82310400011)(1800799012)(64100799003)(451199024)(40470700004)(46966006)(36840700001)(81166007)(31686004)(82740400003)(356005)(41300700001)(86362001)(31696002)(4744005)(426003)(5660300002)(336012)(44832011)(8676002)(8936002)(70586007)(70206006)(4326008)(110136005)(54906003)(16576012)(316002)(2616005)(26005)(36756003)(478600001)(2906002)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Feb 2024 08:12:57.3337
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b4eb8e14-8a30-4254-86a1-08dc2ba26bdf
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D5.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8176

Hi Oleksii,

On 09/02/2024 19:00, Oleksii Kurochko wrote:
> 
> 
> Kconfig tool expects each configuration to be on a new line.
> 
> The current version of the build script puts all of ${EXTRA_FIXED_RANDCONFIG}
> in a single line and configs are seperated by spaces.
> 
> As a result, only the first configuration in ${EXTRA_FIXED_RANDCONFIG} will
> be used.
There is no need for this patch. If you want the variables to be separated by new lines,
just use a '|' symbol after EXTRA_FIXED_RANDCONFIG (see all definitions of EXTRA_XEN_CONFIG).

~Michal



From xen-devel-bounces@lists.xenproject.org Mon Feb 12 08:21:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 08:21:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679313.1056730 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZRYm-0004ep-Db; Mon, 12 Feb 2024 08:21:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679313.1056730; Mon, 12 Feb 2024 08:21:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZRYm-0004ei-9o; Mon, 12 Feb 2024 08:21:04 +0000
Received: by outflank-mailman (input) for mailman id 679313;
 Mon, 12 Feb 2024 08:21:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZRYk-0004ec-Qy
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 08:21:02 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a70aba47-c97f-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 09:21:00 +0100 (CET)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-33aeb088324so1892254f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 00:20:59 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ch8-20020a5d5d08000000b0033b75b39aebsm4590733wrb.11.2024.02.12.00.20.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 00:20:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a70aba47-c97f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707726059; x=1708330859; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=QxAED0Upw3UUaTw/PsXjs9hNGEU9EnOi2nrcGLgAIxM=;
        b=A2W/Ayp0d8nJl3EOjg1Wdd9B2V5omEcVHTSYRKQ5LO8fwkC5h3AK/ePd2jcl6QG6uw
         ASGyDRpZjaCYmnzBhV0/qXMzl8JO6FR8VcjstPqpZuDGOlLBiaQPYSL/YqmsNcbsfv5F
         frpbUIzg7MUBW7p+oheXrYzIMn+J/Oq4SEQltdbp2HLRd9bC5CiHfWTnYb+iVsB6KnYq
         mkia4UC7GOqlYnqu3NYN8qLdYgmQiOa0y/Cd9YojKGydZTxKqaqTnrgwWb/fIB1ikWHo
         YceYoMhA/yrn4sZcVs+QYmgtAptqKgy9OoP0MQLBgzru6tipIC8Y02+TDyoT3LWo8UyH
         u01w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707726059; x=1708330859;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=QxAED0Upw3UUaTw/PsXjs9hNGEU9EnOi2nrcGLgAIxM=;
        b=cfKVZX/je9TQKNFJLQwxVmug88bgVS84yWGDBZQLFnxn8Lk9nYh7/gtBn/CYIPj51J
         XaN6Cp94WZW1fATbwP3rSnLcDEu9DkidRQq+h34TXesYjH83ZmuzhkqE8t2l20GGHKbu
         Ya7v+fp/7LRj5AsggAvwA4QA/uCYGFUUnX5FK33HC4gCAlOcNp3eWeYkqf9L5DcsmRX+
         mZaFBkfUpk1JqgNlsjybov6cUmEyv9bAEL4V0skpzTakLbPwXoI/KrtFmmyRyrFl7VKf
         tEGfjAlO5Iom9V58qGWfXg9bQydA+776Z3N0EdUHB0MBPAlDBjWYzOAFXPk1R0eKZ1I9
         RFpQ==
X-Gm-Message-State: AOJu0YwjtSJHZOyz/UR5Dv8kDWx/TwY2/b4i1KjHDxtdD+rqqSM3Q+hB
	PPPdFxG7sV+CQLOrQAfh2NFkxOl2BmT2cU35F+Un7dqtHUfRHa8u9/y5xTjDNw==
X-Google-Smtp-Source: AGHT+IGx2zqaW6JEG3NSgsQFKqNrjcogYI42tO8A6DBU4NB0ynUvwdNIrwbIB8JjMeTrlCv1lxLjxQ==
X-Received: by 2002:adf:e6cf:0:b0:33b:6d59:61cd with SMTP id y15-20020adfe6cf000000b0033b6d5961cdmr436163wrm.11.1707726059066;
        Mon, 12 Feb 2024 00:20:59 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW0SN4jQE371CBe6vPPZpj768cGL5NO3XajAMUN7PKjOREoPX4nZ75xhifvAosgrZEWz3blgFwb1KUgTHw2dWwzQujdPxmwW+hKEBvWaEknqrJn1dQnVz6ttkUotV72KDV9sV2+4iijJaAG95cdnNC5OK1o6XhWGAD2D6OAhwxK2JKP+fHgsku7/fNA28+ZqeefRbFZgJ8VEH9PxXDmau8xpClwiJHOHb5YDJV9q8nSblwVedZz12Uj03fKIe40BPrD/xj92uP7zNhDjofW8bjB20jQ4BmwUtVzaLjw/ILUNKJllWk=
Message-ID: <9504e77d-6f52-489c-a91a-f4d1a6ce9a33@suse.com>
Date: Mon, 12 Feb 2024 09:20:59 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v1] docs/misra/rules.rst: catch up with accepted rules
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, julien@xen.org,
 bertrand.marquis@arm.com, roger.pau@citrix.com, roberto.bagnara@bugseng.com,
 federico.serafini@bugseng.com, xen-devel@lists.xenproject.org
References: <alpine.DEB.2.22.394.2402091653110.1925432@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402091653110.1925432@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 10.02.2024 02:00, Stefano Stabellini wrote:
> Update docs/misra/rules.rst to reflect the MISRA C rules accepted in the
> last couple of months.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> ---
> 
> In the notes section I added some info about the deviations, but in any
> case the appropriate info will also be added to deviations.rst,
> safe.json, etc.
> 
> I also added Rule 14.4, which is older, but when I first tried to add it
> to rules.rst, Jan had a question I couldn't reply clearly:
> https://marc.info/?l=xen-devel&m=169828285627163
> 
> I think now with this series, the impact of Rule 14.4 is clearer:
> https://marc.info/?l=xen-devel&m=170194257326186

This series is about enums only afaics. Yet the rule is much wider, and iirc
we had agreed that for integer and pointer types the normal language
conversion to boolean meaning is fine as well. Not only do you not mention
this case in the entry, but it also continue to mean that effectively we
limit the rule to a very narrow case. Which continue to leave open the
question of whether the rule is worthwhile to accept in the first place.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 08:26:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 08:26:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679318.1056740 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZReO-0005JR-3j; Mon, 12 Feb 2024 08:26:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679318.1056740; Mon, 12 Feb 2024 08:26:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZReO-0005JK-11; Mon, 12 Feb 2024 08:26:52 +0000
Received: by outflank-mailman (input) for mailman id 679318;
 Mon, 12 Feb 2024 08:26:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZReN-0005JE-1E
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 08:26:51 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 773211d3-c980-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 09:26:49 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33b18099411so1667912f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 00:26:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e27-20020a5d595b000000b0033b80a0d002sm2718176wri.57.2024.02.12.00.26.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 00:26:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 773211d3-c980-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707726408; x=1708331208; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=od8jxRrRiKzPFlw+nYsHZiXd4nL03CPY77R8l36qlnM=;
        b=f2UwtqaIOspn4q0/sYdGbNCtnD05Vz4laB2SeQu3UeXBSrLAcyyRgN9Me4QsEvmC+7
         c/pT8/h6oPlHEJ8KIiRZPZRFd0EXk4/QZrtw+LFvNkmIgxSlSNFOAwpLNXLedrUC8+li
         q8ruW99diFdyG7UbXWyhq4lqjvp/wySgKLq4ev8IiVVsBjVNiHi22QJ86ykfTkurMHse
         wxYR+8//Xy0PIU5HwBs0cN0yWHzNj/hjs6cB65I0p1HnLE6V9OLXRBNrHCAvgeJe9nPf
         bMH8tGbhS5vHpexvBx0GAnsBD2KY0yeqUclDJXAx4gOqWg/pc1CyBtjuhM095c7i7jJU
         z9vw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707726408; x=1708331208;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=od8jxRrRiKzPFlw+nYsHZiXd4nL03CPY77R8l36qlnM=;
        b=DCpDqXdiRi3iQYJsR4ouDhnNEaBllPrZ2qnCppvtebdnJrlNOhuRbTRoQyLEftff1N
         MSkE7/ksud3LvN2f4D4NWM61RbVHq8rleaOQbWICx/R1pS0AM0QmlE0NEXSMKw1ISycO
         6WtBM7eUfD1k8m/idi7x0HCi6DlMG0rE0CTU4HWK/QKaffl0oWEUZ44izM2X+enY2p9f
         cZxCRwdp5YPmYgLff5SasKd1H4glsajc8TWmTmLhD+0XSargOS7PjhwD8CKGlY3A4mfk
         cYw/LRLLnONdUZY0NCThKU0u3Q1qzbUPBR0P2bc4vzvgZw88NugfF6JXZOkkT2ylLvTD
         XmPQ==
X-Gm-Message-State: AOJu0YwsK+XzgQBi8Rr/YdUabRpGSy/nKdYEJ9XodRwZDgmFKjaGK+c2
	+hEOJscCQtWuw8Gshvi99RrHQgQ612LBpDgB5FklpEEKqblRo40Oj+xFjZIx4g==
X-Google-Smtp-Source: AGHT+IF3yJxQ1J/0ELE+vneMMG4VTpysjk6zLpvtcmFNo7kZ5SfDsAIvntxPehFeu3eBU9ns9niXmA==
X-Received: by 2002:a5d:43cc:0:b0:33b:43d7:14 with SMTP id v12-20020a5d43cc000000b0033b43d70014mr4280629wrr.61.1707726408231;
        Mon, 12 Feb 2024 00:26:48 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWt3TCUre5foUEvMAxJuqGUJiLN8XhBKjOtWUVGbafAsrlSLv8veySYZkSFV2rzeOXYrZi/lG+j3vkBFyOj9ILGlheyGOJHj8hzSdoTKSHbR9Hdsfe6paf+wB1mS1jyYfeow5ZIlc1xnBTZKz7E/LfK+emfnT84HTZfOsKhr/PSpPaZrcI5OpYi2UJDHweQXxcnz4hd636AwgjKOGdhIpuVmlXi4Rgcq7TGPnnf8TgOMBk9xJ+/uVdhqSX8z4HJiIWfizO7IktltiYriqeHzNWm0XUUqtHQ7ZOw7qTcP2/6FBYasxx1Bbe7N6i5lJqEQvg7YHCdorgcCYqwYRBRtnonHw==
Message-ID: <02446758-a5a9-4a8a-8ce6-15abdc9ede65@suse.com>
Date: Mon, 12 Feb 2024 09:26:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Return type of clean_and_invalidate_dcache_va_range
To: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Cc: Xen Devel <xen-devel@lists.xenproject.org>,
 Consulting <consulting@bugseng.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper3 <andrew.cooper3@citrix.com>, Roger Pau
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>, Nicola Vetrini <nicola.vetrini@bugseng.com>
References: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
 <alpine.DEB.2.22.394.2402091402080.1925432@ubuntu-linux-20-04-desktop>
 <f86baad0-f113-4156-9c10-6910e8c63492@xen.org>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f86baad0-f113-4156-9c10-6910e8c63492@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 10.02.2024 11:17, Julien Grall wrote:
> Hi,
> 
> On 09/02/2024 22:02, Stefano Stabellini wrote:
>> On Fri, 9 Feb 2024, Nicola Vetrini wrote:
>>> Hi all,
>>>
>>> In the context of violations of MISRA C:2012 Rule 17.7: "The value returned by
>>> a function having non-void return type shall be used", I was looking at the
>>> function "clean_and_invalidate_dcache_va_range". It has the following
>>> signature on both arm and x86:
>>>
>>> static inline int clean_and_invalidate_dcache_va_range
>>>      (const void *p, unsigned long size)
>>>
>>> The commit that introduced it for Arm ~9 years ago (71d64afe3e12: "arm: return
>>> int from *_dcache_va_range") [1] mentions that on Arm it can't fail, but
>>> supposedly it can on x86.
>>>
>>> However, as far as I can tell, for both arch-es the implementation now always
>>> returns 0 [2][3], so perhaps the mention of -EOPNOTSUPP for x86 is no longer
>>> true (I wasn't able to reconstruct if there was a time at which this was true,
>>> even in the same commit that changed the return type to int).
>>>
>>> The question is: should the return type be void, since it appears that every
>>> user is ignoring the returned value (violating the rule), except the one in
>>> common/grant_table.c [4]?
>>
>> Looking at the implementation on both ARM and x86, I am in favor of
>> changing the return type to void
> I think we need some consistency between all the cache flush helpers 
> (clean_and_invalidate_dcache_va_range, invalidate_dcache_va_range() and 
> clean_dcache_va_range()). They should all return a values or not return any.

+1

> That said, we have two other architectures in development. Are we saying 
> this helpers will not need to (initially) return -EOPNOTSUPP?

For "(initially)" that's not an issue - such a stub can as well be filled
for BUG_ON("unimplemented"). The question there is what the ultimate
implementations are going to look like.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 08:39:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 08:39:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679323.1056750 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZRqq-0008G0-60; Mon, 12 Feb 2024 08:39:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679323.1056750; Mon, 12 Feb 2024 08:39:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZRqq-0008Ft-3M; Mon, 12 Feb 2024 08:39:44 +0000
Received: by outflank-mailman (input) for mailman id 679323;
 Mon, 12 Feb 2024 08:39:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tvA+=JV=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rZRqo-0008Fn-QP
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 08:39:42 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 42b909ac-c982-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 09:39:41 +0100 (CET)
Received: from BL1PR13CA0156.namprd13.prod.outlook.com (2603:10b6:208:2bd::11)
 by IA1PR12MB6283.namprd12.prod.outlook.com (2603:10b6:208:3e5::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.23; Mon, 12 Feb
 2024 08:39:35 +0000
Received: from BL02EPF0001A0FD.namprd03.prod.outlook.com
 (2603:10b6:208:2bd:cafe::5a) by BL1PR13CA0156.outlook.office365.com
 (2603:10b6:208:2bd::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.22 via Frontend
 Transport; Mon, 12 Feb 2024 08:39:35 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL02EPF0001A0FD.mail.protection.outlook.com (10.167.242.104) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Mon, 12 Feb 2024 08:39:35 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 12 Feb
 2024 02:39:35 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 12 Feb 2024 02:39:34 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42b909ac-c982-11ee-8a4c-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HI7eeQk5UNzIYRwUMH2l5HT+BEs5/WWav6ZSd3y++YBze+ylAvpayKWiYDNrv+lBW28vMQWrHx/cIYPUnNhvXsJ9Vd2Tf6mKkTlydHSoLwyEvjwweLmCVyiU7kN/q206Yywuh2AnjTVSCfsK/8U23cYwBZTbbpcVb038huvZsdq/vqzeOf5Iu42If1sPpDOhRr/B9mp6LY0PX9vhQVeyM8n6ZiMFLM9Fnt3qv4WsnJqJ0/wYesx+mAQbxpCw/PdpKthfNbhskhSc0QETC092JddwV07CQ8Nff4WgV7rGjHK52QLaIJx7HM5TXhR62+V+t/jImy94ThHxrymZLRvlHQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=uIe52IwIOMwVyiVD4eWenPEgNR+wB6cKnopL2Nq3piI=;
 b=VfSITWes851O4+5oMWU4KnLL+nTDyn42y4geMOa5zL808Jctr6Zq9Ex9Hp3yur4GfySGtmADxnrysTFnKKRjp+36pHDYcI+k1u4sWoUSQgtC851GjftEE8peeUChJT8M7PqbU0gzwM5BHXTtx7y/bWhZ4fbjvov/rh5EjdL61OMSj+rKW4YJd9V9s3UjJaQzbd+la1Rne6U2U+bQoa+YkRpdxahXtny73SW6JmOGaR6v5b0tkEPFvu1aCYTu/Dqq54SGylIzmncPf2a4FxHi/r8Y5v0CeR6hdzfcld9Q+z4EwUV8ONBiDXxpC+vdAo3n8+Se6wanZmg9uUGxNkVa5Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=gmail.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uIe52IwIOMwVyiVD4eWenPEgNR+wB6cKnopL2Nq3piI=;
 b=k+dD+GXmx1F1BpWFjv4Q09BSPNYRi8/sTCgsqxrld1+voRxWqkEPFB0sMwywV/6TJreowURpRyQsfHlgBKeWUGhuISl1UBSH9m2SoK/QV00/5HnE50RQuoVE7adamuzIsxtfjuPo+ONjHyMtssWqmNGeIXafc52PoHWM7d4kAJw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <738a0947-0c54-4c13-b632-a5e711894e7c@amd.com>
Date: Mon, 12 Feb 2024 09:39:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 2/7] automation: introduce fixed randconfig for RISC-V
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Doug Goldstein <cardoe@cardoe.com>, Stefano Stabellini
	<sstabellini@kernel.org>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
 <3f662e3b4ded36d982ff04ed5cdfdd8e36f83594.1707499278.git.oleksii.kurochko@gmail.com>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <3f662e3b4ded36d982ff04ed5cdfdd8e36f83594.1707499278.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FD:EE_|IA1PR12MB6283:EE_
X-MS-Office365-Filtering-Correlation-Id: 0e22862b-9ab1-4e6d-16ec-08dc2ba6246e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	KneJ1uLDWSZjObExO1xDi7wq/8W03BsSkUibBh9cLFdvSl4R5Cu0S60eFbKZPggYiDetJfdvSJo0K2W9G8hjCoIv+7aXoK2rBMOcPm3uCa4IawqGfDz3YNwWsrcY6G+/kebHjAOUsNL2CD2Yr6uuhuesNaxz6xYij6J+AW4882Y++4yN4zSG97P02s6hc23ZLpoXwXL50xXRm9yHjmxYXqfNNdINBxMBmxUXIjj0x5b9ucque2mxvS+/DNbccduBVFa1UmuEdh8G4akCBAS020D8qGK5vnfulM4RAW8b5UzIaSVqcU0KmM2BYaPOMX7kyAAj4M8QjsqsCCh7KfmjrZwEwQqPe5H7qnSSYG71X8oQFvfcYiyb6eOzuSVZ8WD6im9vg7F75izXUkFPrJHz2BkxhneKNLexYjf29nzXsj4PqcEMQ1I7kjAZRArkaR1fGTF6BTQRmd6IlKsM9NU59Jv4Dw4JAMO60OQrfEiggnLnk64bJaJJWdzKGcZSRd01hZRKYN20pAtiJyMJxaO0fmNhIYdteblS0ilS3Eu+2MkzioP47tFXer/8OcIgyE87nldse0ZdiSG3YMVzTK147g==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(346002)(396003)(136003)(39860400002)(230922051799003)(186009)(82310400011)(1800799012)(451199024)(64100799003)(40470700004)(46966006)(36840700001)(81166007)(82740400003)(31686004)(356005)(41300700001)(86362001)(83380400001)(31696002)(4326008)(5660300002)(336012)(426003)(44832011)(8936002)(70206006)(70586007)(8676002)(54906003)(110136005)(966005)(316002)(16576012)(26005)(478600001)(2616005)(2906002)(53546011)(36756003);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Feb 2024 08:39:35.5059
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0e22862b-9ab1-4e6d-16ec-08dc2ba6246e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL02EPF0001A0FD.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6283

Hi Oleksii,

On 09/02/2024 19:00, Oleksii Kurochko wrote:
> 
> 
> This patch introduces the file riscv-fixed-randconfig.yaml,
> which includes all configurations that should be disabled for
> randconfig builds.
> 
> Suggested-by: Stefano Stabellini <sstabellini@kernel.org>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> The patch were introduced after discussion in a topic:
>  https://lore.kernel.org/xen-devel/cover.1701966261.git.oleksii.kurochko@gmail.com/
>  ---
> Changes in V8:
>  - Nothing changed. Only rebase
> ---
> Changes in V7:
>  - Nothing changed. Only rebase
> ---
> Changes in V6:
>  - The patch was introduced in this version of patch series.
> ---
>  automation/gitlab-ci/build.yaml                  | 8 ++++----
>  automation/gitlab-ci/riscv-fixed-randconfig.yaml | 7 +++++++
>  2 files changed, 11 insertions(+), 4 deletions(-)
>  create mode 100644 automation/gitlab-ci/riscv-fixed-randconfig.yaml
> 
> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> index 6d2cb18b88..376eb17f9c 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -512,6 +512,8 @@ alpine-3.18-gcc-debug-arm64-boot-cpupools:
>        CONFIG_BOOT_TIME_CPUPOOLS=y
> 
>  # RISC-V 64 cross-build
> +include: 'automation/gitlab-ci/riscv-fixed-randconfig.yaml'
I don't think there is a need to create a new file for that.
You could define an anchor in build.yaml:

# RISC-V 64 cross-build
.riscv-fixed-randconfig:
  variables: &riscv-fixed-randconfig
    EXTRA_FIXED_RANDCONFIG: |
      CONFIG_COVERAGE=n
      CONFIG_EXPERT=y
      CONFIG_GRANT_TABLE=n
      CONFIG_MEM_ACCESS=n

and reference it in the job:

archlinux-current-gcc-riscv64-randconfig:
  extends: .gcc-riscv64-cross-build
  variables:
    CONTAINER: archlinux:current-riscv64
    KBUILD_DEFCONFIG: tiny64_defconfig
    RANDCONFIG: y
    <<: *riscv-fixed-randconfig

~Michal



From xen-devel-bounces@lists.xenproject.org Mon Feb 12 08:43:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 08:43:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679327.1056760 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZRuB-0001EY-MA; Mon, 12 Feb 2024 08:43:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679327.1056760; Mon, 12 Feb 2024 08:43:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZRuB-0001ER-Iq; Mon, 12 Feb 2024 08:43:11 +0000
Received: by outflank-mailman (input) for mailman id 679327;
 Mon, 12 Feb 2024 08:43:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZRu9-0001EL-OX
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 08:43:09 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bf1bdfc8-c982-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 09:43:08 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33aea66a31cso1587073f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 00:43:08 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z7-20020adfe547000000b0033b55186901sm6119983wrm.94.2024.02.12.00.43.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 00:43:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf1bdfc8-c982-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707727388; x=1708332188; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=aP9fMsgUKGvnPVDMOXT2RWnzWCXiUjjPupGL0ruEJok=;
        b=OEs06w8696q6zbVnykyY6wUnOOmykvl6Q0RFA8Zv6AQscM+ehczVTJUBho4+acyE/M
         GZaXE11M+D7CCGXazwdFli2hQwZ1NK+899hRhrTmn4+yE4HyJzcaFtt9NvI5RCmWvy3X
         K2t1I8OKX5qikO6Sybo7htBv6Rt3gKP8g2GVJxDX/y1UDBZEAeZsmx0Ls5uT+Kp5d9ZQ
         S7rzx/d28RgFZU2J44DzZ5Y5vaCcOU0/wME0aW42yOtzxS5iI7nfOMBk/EZUhZynvyLH
         v+KuBQr1lDzZJ7uGEA1I7P+PrKW4D1Kcc3fssiCL7fJCD7RH357DdczeeK9StZh6xpQE
         hIfA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707727388; x=1708332188;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=aP9fMsgUKGvnPVDMOXT2RWnzWCXiUjjPupGL0ruEJok=;
        b=JhmvKNI26wGIcAIuhgR1y57CaNGauLigz4pneBXQo7T/eQlI7VpM2CyWxpW3vSBDic
         xHbxT0m48lSr2qF5L1fOW93Mnr9xCSBX+fhwXulItyPLAIJIkW7RLxXR3J/mCsbwppWM
         MMtb2U+apHIbsGivUVtm9cKQ1dWeH7g1zLmU4QlUcnTHLGFMOQo3SNshENBhA3EjY+Y3
         w6qSgbewdbEGkpJzNVv988C0TDWAzYKzsRgn2PLU0AsECW6+CHxUHgfJsKD2N+KGrY3N
         k14K19GdFy/ApZSfzEO1dxwkIV0hBn4mewzbgOaHlOMHbXSWMvaXaIIuwToY+3RjUguM
         sYYw==
X-Forwarded-Encrypted: i=1; AJvYcCXBRiYO5LZmvM46TxaWf7RY8nS4Q8xrQbpqIPeTxEXCuwEB1wMjDJg/aGouJIaJPZYJX0CxlnJpKs34lTpV2pu2lJRL2kDsqtApc0qIAQw=
X-Gm-Message-State: AOJu0YwwaCKBDd2aN0DJMeeU8GBYX3mmrZ4jJ3Qb++8aAi962WUG71ZV
	hyzO6bx17J7bd83z/M1Xvt7QEHSNdmcopXhFZqNwUV0zHkztz/a7ez8eyyfxzw==
X-Google-Smtp-Source: AGHT+IHVEIk+VyenrCqwitaE1FENGrLleeO9PJxWGbYUNFeyNM9StYvxwHtzruCUzHMHVOcwUM1QWg==
X-Received: by 2002:a5d:43c6:0:b0:33b:4ac6:f73b with SMTP id v6-20020a5d43c6000000b0033b4ac6f73bmr3521268wrr.23.1707727387975;
        Mon, 12 Feb 2024 00:43:07 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXBbEbAD4fECDi7VesOnTlg2bnYtcMwNvYr7gntLWaQa4l6RoUJ02tLg8vO8Saq+XSMGf2cNBOmvYYL/rREYlBYJl+S85wr/yw6AsQMp/y0XhK2BmFUS4K8G2Upvt5LHUykY3UZBvRHyTJsu7t/B1Vg+PjXFWo+ewGnZXI1OIRaYS67NGpZvk4GwXDIIxvSCPRBhwRaefUQlGnT/nXpqJiRSc/0
Message-ID: <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com>
Date: Mon, 12 Feb 2024 09:43:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
 <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
 <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 09.02.2024 10:50, Federico Serafini wrote:
> On 08/02/24 12:14, Jan Beulich wrote:
>> On 08.02.2024 11:45, Federico Serafini wrote:
>>> On 07/02/24 17:19, Jan Beulich wrote:
>>>> On 07.02.2024 16:58, Federico Serafini wrote:
>>>>> On 07/02/24 16:24, Jan Beulich wrote:
>>>>>> On 07.02.2024 16:08, Federico Serafini wrote:
>>>>>>> On 07/02/24 15:16, Jan Beulich wrote:
>>>>>>>> On 07.02.2024 14:51, Federico Serafini wrote:
>>>>>>>>> On 07/02/24 08:38, Jan Beulich wrote:
>>>>>>>>>> On 07.02.2024 02:08, Stefano Stabellini wrote:
>>>>>>>>>>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>>>>>>>>>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>>>>>>>>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>>>>>>>>>>           case 8:                                                                \
>>>>>>>>>>>>>               put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>>>>>>>>>>               break;                                                             \
>>>>>>>>>>>>> -    default: __put_user_bad();                                             \
>>>>>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>>>>>           }                                                                      \
>>>>>>>>>>>>>           clac();                                                                \
>>>>>>>>>>>>>       } while ( false )
>>>>>>>>>>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>>>>>>>>>>           case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>>>>>>>>>>           case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>>>>>>>>>>           case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>>>>>>>>>>> -    default: __get_user_bad();                                             \
>>>>>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>>>>>           }                                                                      \
>>>>>>>>>>>>>           clac();                                                                \
>>>>>>>>>>>>>       } while ( false )
>>>>>>>>>>>>
>>>>>>>>>>>> Related to my remark on patch 1 - how is one to know the macro this was
>>>>>>>>>>>> invoked from, when seeing the resulting diagnostic?
>>>>>>>>>>>
>>>>>>>>>>> I am not sure what do you mean here... we do get an error like the
>>>>>>>>>>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>>>>>>>>>>
>>>>>>>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>>>>>>>
>>>>>>>>>> Right - and how do I know what _user_ of the macro actually triggered
>>>>>>>>>> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
>>>>>>>>>> __FUNCTION__ here, for that specific purpose ...
>>>>>>>>>
>>>>>>>>> To test the macro and its diagnostics,
>>>>>>>>> I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
>>>>>>>>> on the x86 code with STATIC_ASSERT_UNREACHABLE(),
>>>>>>>>> that is in file arch/x86/alternative.c, line 312,
>>>>>>>>> function _apply_alternatives().
>>>>>>>>>
>>>>>>>>> What I got is the following build error:
>>>>>>>>>
>>>>>>>>> ...
>>>>>>>>> arch/x86/alternative.c: Assembler messages:
>>>>>>>>> arch/x86/alternative.c:312: Error: static assertion failed: unreachable
>>>>>>>>>        CC      arch/x86/copy_page.o
>>>>>>>>> make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1
>>>>>>>>
>>>>>>>> But that's not what my request was about. Here sufficient context is
>>>>>>>> given, even if it would be nice if the function was also visible right
>>>>>>>> away. But that's not the same as the case above, where the new macro
>>>>>>>> is used inside another macro.
>>>>>>>
>>>>>>> An example of that is the get_unsafe_size() macro,
>>>>>>> whose body uses STATIC_ASSERT_UNREACHABLE().
>>>>>>> A wrong use of get_unsafe_size() at line n
>>>>>>> leads to a build error pointing to the line n,
>>>>>>> isn't this the desired behavior?
>>>>>>
>>>>>> Aiui this would point to the line in the header file, when what you need
>>>>>> to spot the bad use of the macro is the line in the source file actually
>>>>>> using the macro. Quoting from an earlier mail of yours:
>>>>>>
>>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>>
>>>>> It points to the header file uaccess.h because at line 262 there is
>>>>> an intentional wrong use of put_guest_size(), within the body of
>>>>> __copy_to_guest_pv() function.
>>>>
>>>> Yet that's again only a helper function being inlined into the ultimate
>>>> caller. That ultimate caller is what wants identifying in the diag. Not
>>>> the least because of ...
>>>>
>>>>> This example can be misleading because {get,put}_unsafe_size() are
>>>>> defined in the same file but the diagnostics is doing the
>>>>> right thing.
>>>>
>>>> ... this. And really __copy_to_guest_pv() is the wrong place to put a
>>>> wrong put_guest_size() in, to try out how diagnostics would look like
>>>> in reality: That function falls back to copy_to_guest_ll() for all
>>>> cases it can't handle directly. You want to place a bogus put_guest()
>>>> somewhere in a .c file to see what results.
>>>
>>> I added a bogus call to put_guest() at line 387 of
>>> file xen/arch/x86/mm.c, inside function page_is_ram_type().
>>> Assuming I did not choose another wrong place,
>>> the diagnostic seems appropriate:
>>>
>>> arch/x86/mm.c: Assembler messages:
>>> arch/x86/mm.c:387: Error: static assertion failed: unreachable
>>
>> Oh, okay, this looks appropriate then as to identifying where the
>> source construct is. However, we then still don't know where the
>> assertion in question is (there could be multiple in what the
>> original construct expands to). So I'm still inclined to ask that
>> __FILE__ / __LINE__ and/or the name of the invoking construct
>> (macro or function) be made visible in the diagnostic.
> 
> Any use of __FILE__ and __LINE__ results in obtaining
> the same information already reported by the assembler error message.

Hmm, yes, since put_guest() is itself a macro.

> We could add an argument to the new macro to manually add some context
> at every use of the macro, but I think this would be annoying.

That's a last resort. An alternative would be to see about converting
from macros to inline functions, where this would make a difference
here.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 09:04:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 09:04:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679362.1056785 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSEz-0006Jc-Ou; Mon, 12 Feb 2024 09:04:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679362.1056785; Mon, 12 Feb 2024 09:04:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSEz-0006JV-M4; Mon, 12 Feb 2024 09:04:41 +0000
Received: by outflank-mailman (input) for mailman id 679362;
 Mon, 12 Feb 2024 09:04:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZSEy-0006JP-Kp
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 09:04:40 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bfe02ff5-c985-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 10:04:38 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-337cc8e72f5so2163997f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 01:04:38 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bp9-20020a5d5a89000000b0033b4796641asm6227691wrb.22.2024.02.12.01.04.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 01:04:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfe02ff5-c985-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707728678; x=1708333478; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=s3mCtQgf9bplc7MhJ5AYousP7UL1yZNDiGkHf1pEsFQ=;
        b=Oa6JA38J2e33S3kljNE16DzDvhzuczdc8YsIrpfLPxVhfFPopEd/9TXFDKQOHilLyI
         uDOPLg8Rn+ux+mgjehuandQYi4cY/b/M8xL4pBYatN5E4ldBrsOF/MGci1Pk/Q2wTTun
         MRd1mk7qU0l9GHo3pJT8xwbatBNibDuN/ydjU8YMEUNRV7jxoIABjQmzzEuUCwKnV7fN
         D537eW8qtjU43iVJ3L0kkeEEqWYKPxszqw+AEPPItGa889wanjDEVNznK6lsuGcJn4+Z
         4J23G7tDaiPBfygZU6FD/tU94LuTRHGQRonpzIDui9mY8TwoncA7+gwViFq7i0Tsopet
         FmFg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707728678; x=1708333478;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=s3mCtQgf9bplc7MhJ5AYousP7UL1yZNDiGkHf1pEsFQ=;
        b=GUKbUTIScAdY/Ni9h2Bf/1GSsM5DUQ0TjTP5/1mWzY6TkUr06xafSkdMGTrfkbO9h5
         283DCKMh7ZZKyS15fZQwLwEltNAhWC2Ha7lk1yL08iVpyJTS2mf/SCj4rvPPqBy2PbRn
         wByyQpAHX+0o9FciiBPdJrbphe8tnHm0z410rrBLoUcnKTkuENNYG3/ohGkap24xl1Rd
         MaCnC/SHbbwTN5fu6PU7F+s3pQUgzOF3uQ9JRcdg5SGCy36NU0Ut47T2senqQgdA0DOX
         GVF1M4GZln78xbf4cFtqRzlFTLaQ47pNTCTuxe4F+GfwU9CDLNdeRv9NdDCjRSftM46s
         M7UA==
X-Gm-Message-State: AOJu0Yzcrko82VBtxywRAL93f8D6Wn8FdCOkFgxyaT5y5AYfujK659Tm
	iFmel4OxqLO7Ptfb+FQNtGnqRTaExbc8wo4raX/+RWuPHNs6hLfJJb9X+xhy6g==
X-Google-Smtp-Source: AGHT+IHtNGPldtkREn0sSMd9qikx3orZ5nEqCaEq3vWfLsWawau+rPH88zLz6eiwaAsOpWNWSfqfiQ==
X-Received: by 2002:a5d:4808:0:b0:33b:4382:c4b with SMTP id l8-20020a5d4808000000b0033b43820c4bmr4157355wrq.63.1707728677743;
        Mon, 12 Feb 2024 01:04:37 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUnpvoVW4d/Z6a0IYJ5thZwsRQAoK7y/B7chg8/wTEZag+/uqqhNVBU7yfOx2Fl60CZn4xE/YlJ6uLAbkuPnE4ZAOACQmQ8/RPYuVmAbo9c8VzXfMtwoEPqUdlwl1E1Zrt+15sSEK9qy2IhnL/tiLcHKNYvyPWcV3NY9AUEq2LfF0hGiZqObcSf/3o08HGIt9wY3o0mCK5dvSp2hwJhaeeAP+ujD/spoeYe35+u5A==
Message-ID: <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com>
Date: Mon, 12 Feb 2024 10:04:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Content-Language: en-US
To: Julien Grall <julien@xen.org>,
 Marek Marczykowski <marmarek@invisiblethingslab.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
 <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.02.2024 23:00, Julien Grall wrote:
> On 05/02/2024 13:27, Jan Beulich wrote:
>> In preparation of dropping the register parameters from
>> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
>> register state needs making available another way for the few key
>> handlers which need it. Fake IRQ-like state.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsistent with
>> other console poll functions we have, and it's unclear whether that's
>> actually generally correct.
> 
> Is it? Looking at ns16550_poll() we would pass guest_user_regs() if 
> run_in_exception() doesn't exist. But looking at the caller, no-on seems 
> to care about the 'regs'. So is this just a latent bug?

What do you mean by "doesn't exist"? ns16550_poll() assumes it exists.
And I can spot any use of guest_user_regs() on the respective generic
or Arm-specific bug.c paths.

> BTW, do you have an idea why the poll function is not run in an 
> exception handler?

"The poll function" being which one? If you mean the one in xhci-dbc.c
then that's why I had Cc-ed Marek. Moving him to To: - maybe that
manages to finally catch his attention.

>> Andrew suggested to move set_irq_regs() to BUGFRAME_run_fn handling;
>> it's not clear to me whether that would be (a) correct from an abstract
>> pov (that's exception, not interrupt context after all) 
> 
> I agree with that.
> 
>> and (b) really beneficial.
> 
> I guess this could help to reduce the amount of churn. I can't really 
> make my mind whether this is worth it or not. So I would keep it as you did.

Good, thanks.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 09:14:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 09:14:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679369.1056796 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSNZ-0001rI-Dc; Mon, 12 Feb 2024 09:13:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679369.1056796; Mon, 12 Feb 2024 09:13:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSNZ-0001rA-94; Mon, 12 Feb 2024 09:13:33 +0000
Received: by outflank-mailman (input) for mailman id 679369;
 Mon, 12 Feb 2024 09:13:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IbWo=JV=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rZSNY-0001r4-2H
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 09:13:32 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fcc2cb92-c986-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 10:13:30 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-55a5e7fa471so3780252a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 01:13:29 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 y3-20020aa7c243000000b005612ac47d85sm2539889edo.82.2024.02.12.01.13.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 01:13:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fcc2cb92-c986-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707729209; x=1708334009; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=kpndX5C2Mii0Flwlx+5AtvY+2C+NxX9epFi0dvTnTAk=;
        b=ICxh1bTBNIRP/Wrh4PP3jWs6L4XUaqvIbfG/7WS9QciYVf2E2Eh0HKNfktLifF5GRn
         fpq9V5TFWo8PDKGaLJQfc7jpZM6NTkHS3f1NtjsOVcOmrl3eROHTpWFSfa8/q/G9PgVC
         9Q2mo1fsncTXxfV/L6CXIBWvrKDs4SJ50i3Es=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707729209; x=1708334009;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kpndX5C2Mii0Flwlx+5AtvY+2C+NxX9epFi0dvTnTAk=;
        b=dQlccyYyK8dMSpWCbCwc0hgTja6sC1VSe8DjyNpRB306vF+2b3Sq5O2HZhpqAbfPHt
         /iOP8baTXjDVNwb/DZU5I+pjZtMNYkYk0RnKOavamCGbGr/6MtdyAP9gD8nGJU30swVG
         f33o19B3lVNMKxQ1SD7IZKLmb/AvvCXCyS5KM/dL6jwSFGFCi41y+sqNfWelI4TYKrr3
         VcA7QpeR1R66vNIb/TJkF9vS/hEKZrlGSbfqpu6WIovcKGwotEu4JjedcB1q7NeuRbru
         YeCq9f5DYNH1EPwpXSSe1VCqGcd1enorOjlDvYBFusjvPt3zNpqnHpKqfOUAiC84lIXS
         h0Ww==
X-Gm-Message-State: AOJu0Yyt2m+8g8KIVIuSVf4cgpS72oSn/09NRliZhgbFAbOGMjpWHED9
	DEgtYCoo4nEDUw52xepOd0WltY10TyPytzbR0WF8KudpPJOMhIZB5ts9uRcLBSA=
X-Google-Smtp-Source: AGHT+IFGM2App413SIKcCc2Ur0ABS1uXMeYaU9EoPH8ldn/XY6VDUheFoubSE+uiG+infjMyonjofA==
X-Received: by 2002:aa7:d3cb:0:b0:560:93b1:a351 with SMTP id o11-20020aa7d3cb000000b0056093b1a351mr4796353edr.37.1707729209255;
        Mon, 12 Feb 2024 01:13:29 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUi3cSiwbKOwpf9FXAXb3ol8cK21LF8oKTT7zqFHCYqkgcXjwYmpCjbQMm3YKWVBH0wL86GTtfkE0i71Wno6hJgmt+n7jCwj2ThW0BisPIfQ9BSjJ3P/9EBUj8hO4d+sYwI+fl9m6GT0ua3Ox1pNPX7VAhJzufiSulJnviWa7/EOypfxKpbGGnncIAlMdGui2MMtGMDfqBpBLO8n3REJrswc3RQISy4K2jPKesDs7g2VZ3q0NEQLmNizGjzCA67Ju4VddytRfukHTZfMGKvlTY9RDFYR7GrkRHU1/hVzORPAOSWyG1y0gtq04IbVpZNMTko1Xmz0KZVrkmZCyPUjiONYOx3qrezQhpvHry9fw5wK5nTj0snXr1yUQ1iD4ss1Nf0x9YWGtsKGTQ9mR0L8z3HHiCTjSmC8+ol4A7mwG+oApaXZvTUGqk/+B1CF2OES/et67TnEkF84wM1h3O5GUBRMqd0yODERzxI+mqk8FUavSZFQnV9sWAW5/Ay76BMMWPFvRpYW9HQB3Y=
Date: Mon, 12 Feb 2024 10:13:28 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: "Chen, Jiqian" <Jiqian.Chen@amd.com>,
	"Rafael J . Wysocki" <rafael@kernel.org>,
	Len Brown <lenb@kernel.org>, Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
	"Huang, Ray" <Ray.Huang@amd.com>,
	"Ragiadakou, Xenia" <Xenia.Ragiadakou@amd.com>
Subject: Re: [RFC KERNEL PATCH v4 3/3] PCI/sysfs: Add gsi sysfs for pci_dev
Message-ID: <ZcnhOEjnTgbYFPVl@macbook>
References: <ZbtY1R15pYZz3F3B@macbook>
 <20240209210549.GA884438@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240209210549.GA884438@bhelgaas>

On Fri, Feb 09, 2024 at 03:05:49PM -0600, Bjorn Helgaas wrote:
> On Thu, Feb 01, 2024 at 09:39:49AM +0100, Roger Pau Monné wrote:
> > On Wed, Jan 31, 2024 at 01:00:14PM -0600, Bjorn Helgaas wrote:
> > > On Wed, Jan 31, 2024 at 09:58:19AM +0100, Roger Pau Monné wrote:
> > > > On Tue, Jan 30, 2024 at 02:44:03PM -0600, Bjorn Helgaas wrote:
> > > > > On Tue, Jan 30, 2024 at 10:07:36AM +0100, Roger Pau Monné wrote:
> > > > > > On Mon, Jan 29, 2024 at 04:01:13PM -0600, Bjorn Helgaas wrote:
> > > > > > > On Thu, Jan 25, 2024 at 07:17:24AM +0000, Chen, Jiqian wrote:
> > > > > > > > On 2024/1/24 00:02, Bjorn Helgaas wrote:
> > > > > > > > > On Tue, Jan 23, 2024 at 10:13:52AM +0000, Chen, Jiqian wrote:
> > > > > > > > >> On 2024/1/23 07:37, Bjorn Helgaas wrote:
> > > > > > > > >>> On Fri, Jan 05, 2024 at 02:22:17PM +0800, Jiqian Chen wrote:
> > > > > > > > >>>> There is a need for some scenarios to use gsi sysfs.
> > > > > > > > >>>> For example, when xen passthrough a device to dumU, it will
> > > > > > > > >>>> use gsi to map pirq, but currently userspace can't get gsi
> > > > > > > > >>>> number.
> > > > > > > > >>>> So, add gsi sysfs for that and for other potential scenarios.
> > > > > > > > >> ...
> > > > > > > > > 
> > > > > > > > >>> I don't know enough about Xen to know why it needs the GSI in
> > > > > > > > >>> userspace.  Is this passthrough brand new functionality that can't be
> > > > > > > > >>> done today because we don't expose the GSI yet?
> > > > > > > 
> > > > > > > I assume this must be new functionality, i.e., this kind of
> > > > > > > passthrough does not work today, right?
> > > > > > > 
> > > > > > > > >> has ACPI support and is responsible for detecting and controlling
> > > > > > > > >> the hardware, also it performs privileged operations such as the
> > > > > > > > >> creation of normal (unprivileged) domains DomUs. When we give to a
> > > > > > > > >> DomU direct access to a device, we need also to route the physical
> > > > > > > > >> interrupts to the DomU. In order to do so Xen needs to setup and map
> > > > > > > > >> the interrupts appropriately.
> > > > > > > > > 
> > > > > > > > > What kernel interfaces are used for this setup and mapping?
> > > > > > > >
> > > > > > > > For passthrough devices, the setup and mapping of routing physical
> > > > > > > > interrupts to DomU are done on Xen hypervisor side, hypervisor only
> > > > > > > > need userspace to provide the GSI info, see Xen code:
> > > > > > > > xc_physdev_map_pirq require GSI and then will call hypercall to pass
> > > > > > > > GSI into hypervisor and then hypervisor will do the mapping and
> > > > > > > > routing, kernel doesn't do the setup and mapping.
> > > > > > > 
> > > > > > > So we have to expose the GSI to userspace not because userspace itself
> > > > > > > uses it, but so userspace can turn around and pass it back into the
> > > > > > > kernel?
> > > > > > 
> > > > > > No, the point is to pass it back to Xen, which doesn't know the
> > > > > > mapping between GSIs and PCI devices because it can't execute the ACPI
> > > > > > AML resource methods that provide such information.
> > > > > > 
> > > > > > The (Linux) kernel is just a proxy that forwards the hypercalls from
> > > > > > user-space tools into Xen.
> > > > > 
> > > > > But I guess Xen knows how to interpret a GSI even though it doesn't
> > > > > have access to AML?
> > > > 
> > > > On x86 Xen does know how to map a GSI into an IO-APIC pin, in order
> > > > configure the RTE as requested.
> > > 
> > > IIUC, mapping a GSI to an IO-APIC pin requires information from the
> > > MADT.  So I guess Xen does use the static ACPI tables, but not the AML
> > > _PRT methods that would connect a GSI with a PCI device?
> > 
> > Yes, Xen can parse the static tables, and knows the base GSI of
> > IO-APICs from the MADT.
> > 
> > > I guess this means Xen would not be able to deal with _MAT methods,
> > > which also contains MADT entries?  I don't know the implications of
> > > this -- maybe it means Xen might not be able to use with hot-added
> > > devices?
> > 
> > It's my understanding _MAT will only be present on some very specific
> > devices (IO-APIC or CPU objects).  Xen doesn't support hotplug of
> > IO-APICs, but hotplug of CPUs should in principle be supported with
> > cooperation from the control domain OS (albeit it's not something that
> > we tests on our CI).  I don't expect however that a CPU object _MAT
> > method will return IO APIC entries.
> > 
> > > The tables (including DSDT and SSDTS that contain the AML) are exposed
> > > to userspace via /sys/firmware/acpi/tables/, but of course that
> > > doesn't mean Xen knows how to interpret the AML, and even if it did,
> > > Xen probably wouldn't be able to *evaluate* it since that could
> > > conflict with the host kernel's use of AML.
> > 
> > Indeed, there can only be a single OSPM, and that's the dom0 OS (Linux
> > in our context).
> > 
> > Getting back to our context though, what would be a suitable place for
> > exposing the GSI assigned to each device?
> 
> IIUC, the Xen hypervisor:
> 
>   - Interprets /sys/firmware/acpi/tables/APIC (or gets this via
>     something running on the Dom0 kernel) to find the physical base
>     address and GSI base, e.g., from I/O APIC, I/O SAPIC.

No, Xen parses the MADT directly from memory, before stating dom0.
That's a static table so it's fine for Xen to parse it and obtain the
I/O APIC GSI base.

>   - Needs the GSI to locate the APIC and pin within the APIC.  The
>     Dom0 kernel is the OSPM, so only it can evaluate the AML _PRT to
>     learn the PCI device -> GSI mapping.

Yes, Xen doesn't know the PCI device -> GSI mapping.  Dom0 needs to
parse the ACPI methods and signal Xen to configure a GSI with a
given trigger and polarity.

>   - Has direct access to the APIC physical base address to program the
>     Redirection Table.

Yes, the hardware (native) I/O APIC is owned by Xen, and not directly
accessible by dom0.

> The patch seems a little messy to me because the PCI core has to keep
> track of the GSI even though it doesn't need it itself.  And the
> current patch exposes it on all arches, even non-ACPI ones or when
> ACPI is disabled (easily fixable).
> 
> We only call acpi_pci_irq_enable() in the pci_enable_device() path, so
> we don't know the GSI unless a Dom0 driver has claimed the device and
> called pci_enable_device() for it, which seems like it might not be
> desirable.

I think that's always the case, as on dom0 devices to be passed
through are handled by pciback which does enable them.

I agree it might be best to not tie exposing the node to
pci_enable_device() having been called.  Is _PRT only evaluated as
part of acpi_pci_irq_enable()? (or pci_enable_device()).

> I was hoping we could put it in /sys/firmware/acpi/interrupts, but
> that looks like it's only for SCI statistics.  I guess we could moot a
> new /sys/firmware/acpi/gsi/ directory, but then each file there would
> have to identify a device, which might not be as convenient as the
> /sys/devices/ directory that already exists.  I guess there may be
> GSIs for things other than PCI devices; will you ever care about any
> of those?

We only support passthrough of PCI devices so far, but I guess if any
of such non-PCI devices ever appear and those use a GSI, and Xen
supports passthrough for them, then yes, we would need to fetch such
GSI somehow.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 09:14:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 09:14:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679370.1056805 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSNq-000298-Jd; Mon, 12 Feb 2024 09:13:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679370.1056805; Mon, 12 Feb 2024 09:13:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSNq-000291-H2; Mon, 12 Feb 2024 09:13:50 +0000
Received: by outflank-mailman (input) for mailman id 679370;
 Mon, 12 Feb 2024 09:13:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZSNp-00028V-Eq
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 09:13:49 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 07bd684b-c987-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 10:13:48 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-410980dfddeso11438635e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 01:13:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w17-20020a05600c475100b00411062c123esm588355wmo.26.2024.02.12.01.13.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 01:13:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07bd684b-c987-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707729228; x=1708334028; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HFjzuwMBDZJ7f9kgbBbpIVc5q05kTOXXs6u4l/tVAr4=;
        b=er1W4W1S1DXov3u50QhFmcGMSoctZnp9D147U//DkbWUsUAuTncc+9f7EqnXPD7nHb
         TK38uFRHAVarz3jM/3LUcpB0SWXfClYMcgZ4Mnr7GpWJTFU9cIU1gY4Rbc22fHMo4cSl
         yELRAWxfuM/uVAY++NkRzY37P5ftZMKa5VLCRULsaVURpDM/aE93ZKASvRTYTDPR3E45
         sMIQxfEK6kmvZJ9gTMZmgl53F+EUfi3brB9jJbJYQO7yvLCtdrwSIKccDWrFv1ZviGsy
         wB1y8OmNIb1AtCEob9syf+a6b5DOm40eOz/TcRlBrwv6P3ltpVIvrI1AMPMV3iCRTAcy
         i8zA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707729228; x=1708334028;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HFjzuwMBDZJ7f9kgbBbpIVc5q05kTOXXs6u4l/tVAr4=;
        b=VvP604fvSBLcinX7oDP/q18e/yrYdMZcEkmJBG7KJWMP8tn/T5lTWvSQF1+AIko26/
         kToc9YfRdzB63Xn1nVubS68z2EeldRAVGwAYcEl76uojtTjd8XjemWVfk3XkyfOHyjWf
         mDoLIgB47yhZUUpmVb+VRVThlhHb6T5YC09QhZ0eHFI5C1ml2RLM5Bg3LdMkwMuaAb3H
         8LcdLRaU5mfeuLosEz3Ip0HHFsWKAq/UNpW9wx+iPCdxA2Tq+z5KhrOlDajtoGFn/o05
         uj1EoAQYcL4CVxZf6+67VmEiSx7kSnOdrQvCX821PabNrE/Zl3GbnpRMoNDy7jFbjGHA
         QuCA==
X-Forwarded-Encrypted: i=1; AJvYcCVrRYyTwxbmyXZO8vdBb0G2Oxh1k4lFMgemIxaHwz9i8WI/zUNYGqOibtlJfoT/tU3XytSRAIdf4N7yS3ymH1qCZx+o+CTKrMldpIDAKT4=
X-Gm-Message-State: AOJu0YzGxjaAsg2+Ii1vYijmmFx1K6hQO3//FI/noFLT+ILZTaiNi4we
	IvLluglDBJiKB+d73tbP9QTWtiP3Z0dVBoSm62VZxAKwfUQ6JPx8dOkB1nTalg==
X-Google-Smtp-Source: AGHT+IETY46VMWDlS4gCkA+GYk7DDmwNianw74Rrxna2C6hWdU6DMK5nXa/pUSy3PxCS66Y7koD27g==
X-Received: by 2002:a05:600c:190f:b0:410:c804:e891 with SMTP id j15-20020a05600c190f00b00410c804e891mr2266232wmq.16.1707729227827;
        Mon, 12 Feb 2024 01:13:47 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXlC8c60wPosj/b3crQMiWAXZStE4Rc78XWIyH+1ooLP/Ob+tz5aac8CJmY8Ycnvmw2MuNKQ6wd6xevx7TJrMXZVK9IPUoLCeZ9GwK5HlFy+9yBbbhOwZBq1QihhyystEOeUUfF+0/oZGTfpdRMmt2bLbx7A2hzDWVM3NpktF6JIe2pkIH2+X6Sc53dsQ==
Message-ID: <bdba33b7-7fc2-4836-8b78-0d56edb30871@suse.com>
Date: Mon, 12 Feb 2024 10:13:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/8] keyhandler: drop regs parameter from
 handle_keyregs()
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <5258d8a9-a7ef-4342-9b5c-fc0078139bb2@suse.com>
 <0bbe2028-946c-459f-b6cc-419869d22b35@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <0bbe2028-946c-459f-b6cc-419869d22b35@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 08.02.2024 23:09, Julien Grall wrote:
> On 05/02/2024 13:28, Jan Beulich wrote:
>> In preparation for further removal of regs parameters, drop it here. In
>> the two places where it's actually needed, retrieve IRQ context if
>> available, or else guest context.
>>
>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> As an alternative to the new boolean parameter, I wonder if we couldn't
>> special-case the idle vCPU case: It's only there where we would not have
>> proper context retrievable via guest_cpu_user_regs().
> 
> I am trying to understand the implication. Looking at the code, it seems 
> in the case where we pass NULL, we would expect to call 
> run_in_exception_handler().

Right, when NULL was passed so far, and when true is passed now, that's
to indicate to invoke run_in_exception_handler().

> If I am not mistaken, at least for Arm, regs would not be the same as 
> guest_cpu_user_regs(). So I think your current approach is more correct.
> 
> Did I miss anything?

Whether regs are the same isn't overly relevant here. The thing that's
relevant is whether what would be logged actually makes sense. And
invoking guest_cpu_user_regs() in idle vCPU context makes no sense.
Whereas in other contexts its result is good enough to show the present
state of the CPU; there's no real need in such a case to go through
run_in_exception_handler().

The present approach therefore isn't necessarily "more correct", but it
is closer to prior behavior.

The corner case that makes me prefer the presently chosen approach is
when a CPU is in the middle of scheduling, especially considering x86's
lazy switching (when current != this_cpu(curr_vcpu). The main reason to
mention the alternative is because iirc Andrew had suggested to move
more in that direction.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 09:15:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 09:15:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679377.1056816 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSPb-00039x-1A; Mon, 12 Feb 2024 09:15:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679377.1056816; Mon, 12 Feb 2024 09:15:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSPa-00039q-UM; Mon, 12 Feb 2024 09:15:38 +0000
Received: by outflank-mailman (input) for mailman id 679377;
 Mon, 12 Feb 2024 09:15:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=JouK=JV=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rZSPZ-00039i-IK
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 09:15:37 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 47a5563a-c987-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 10:15:35 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 170D22219E
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 09:15:35 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 053A313985
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 09:15:34 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id NANAO7bhyWX8fgAAD6G6ig
 (envelope-from <jgross@suse.com>)
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 09:15:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47a5563a-c987-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707729335; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=JRdQMcodzFSdYrOamMCxcGPN5Rd6DHQ4WUwCyZUyz0I=;
	b=esFuAI39UNAenLvfYM+R4AH78o1uTGME7Tbu5r93W9pUOqoKMfOXTJuCePXHwWksXPBkpA
	HcOkB+nFlmzAGMh85OTqZ5OHKXrLaSdEQ1kKFXAlSyP8k63ZngZNQPGKkJgin48YDY00Cm
	E0FeCEAeTG84eZCcMzbUcbrp6LYpzI4=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707729335; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=JRdQMcodzFSdYrOamMCxcGPN5Rd6DHQ4WUwCyZUyz0I=;
	b=esFuAI39UNAenLvfYM+R4AH78o1uTGME7Tbu5r93W9pUOqoKMfOXTJuCePXHwWksXPBkpA
	HcOkB+nFlmzAGMh85OTqZ5OHKXrLaSdEQ1kKFXAlSyP8k63ZngZNQPGKkJgin48YDY00Cm
	E0FeCEAeTG84eZCcMzbUcbrp6LYpzI4=
Message-ID: <99873f03-217d-4799-bc4b-df841cf4b039@suse.com>
Date: Mon, 12 Feb 2024 10:15:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Subject: Errors when trying to create VM with "channel" device(s)
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.46
X-Spamd-Result: default: False [-3.46 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_GOOD(-0.10)[text/plain];
	 PREVIOUSLY_DELIVERED(0.00)[xen-devel@lists.xenproject.org];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCPT_COUNT_ONE(0.00)[1];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 TO_DN_EQ_ADDR_ALL(0.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.83)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Flag: NO

A customer is complaining to be unable to create multiple "channel" devices
in a HVM domain. I've verified the same happens with upstream Xen.

The "channel" configuration is:

channel = [ "connection=socket, name=test.ch1, path=/run/testsocket1", 
"connection=socket, name=test.ch2, path=/run/testsocket2" ]

When creating the domain, I see:

libxl: error: libxl_console.c:285:libxl__device_console_add: Domain 4:Primary 
console has invalid configuration

Both /run/testsocket1 and /run/testsocket2 are being created in dom0.

"xl channel-list 4" gives me:
Idx BE state evt-ch ring-ref connection
0   0  1     -1     -1       socket

"xenstore-ls /local/domain/0/backend/console/4" prints only one channel:
1 = ""
  frontend = "/local/domain/4/device/console/1"
  frontend-id = "4"
  online = "1"
  state = "2"
  protocol = "vt100"
  name = "test.ch2"
  connection = "socket"
  path = "/run/testsocket2"
  hotplug-status = "connected"
0 = ""
  frontend = "/local/domain/4/console"
  frontend-id = "4"
  online = "1"
  state = "1"
  protocol = "vt100"

While qemu seems to have been started with the information for both.
Output of "xl -vvv create":

...
libxl: debug: libxl_dm.c:2994:libxl__spawn_local_dm: Domain 4:  -chardev
libxl: debug: libxl_dm.c:2994:libxl__spawn_local_dm: Domain 4: 
socket,id=libxl-channel0,path=/run/testsocket1,server=on,wait=off
libxl: debug: libxl_dm.c:2994:libxl__spawn_local_dm: Domain 4:  -chardev
libxl: debug: libxl_dm.c:2994:libxl__spawn_local_dm: Domain 4: 
socket,id=libxl-channel1,path=/run/testsocket2,server=on,wait=off
...

"xenstore-ls /local/domain/4/device/console" only shows:
1 = ""
  backend = "/local/domain/0/backend/console/4/1"
  backend-id = "0"
  state = "1"
  protocol = "vt100"
  name = "test.ch2"
  limit = "1048576"
  type = "ioemu"
  output = "chardev:libxl-channel1"
  tty = ""

So the console AND the first channel are missing.

"xenstore-ls /local/domain/4/error" shows:
device = ""
  console = ""
   1 = ""
    error = "22 xenbus_dev_probe on device/console/1"

Now doing the same with only one "channel" device shows the same error when
creating the guest. The guest has no channel device after that and the Xenstore
error node is not present.

I'm not that familiar with this code area, so I thought I post this here in the
hope someone has an idea where to look first.


Juergen


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 09:31:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 09:31:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679414.1056842 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSeY-0008EE-Ob; Mon, 12 Feb 2024 09:31:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679414.1056842; Mon, 12 Feb 2024 09:31:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSeY-0008E7-Ke; Mon, 12 Feb 2024 09:31:06 +0000
Received: by outflank-mailman (input) for mailman id 679414;
 Mon, 12 Feb 2024 09:31:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZSeY-0008Cx-4W; Mon, 12 Feb 2024 09:31:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZSeY-0000pe-1W; Mon, 12 Feb 2024 09:31:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZSeX-0000Ne-K4; Mon, 12 Feb 2024 09:31:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZSeX-0003Fz-JV; Mon, 12 Feb 2024 09:31:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=fdT+fyLI5EDm6Gcwp8tWltd63DVlDf+qAKNrgVUSZWo=; b=3Mwhc8cdX4+l/iIiViolBa1hnT
	wjc9cuoEO2GrOERkIswuBRqrf/ubYlOd7Roa9G05sx4CpQP/oUwoDuond2khn6NxTBwV+1un2emPb
	zJKPg9QiNkYi7hbmwAMiBFtwx+G7MknNEnOHW2E63mHZb/7UtRIwe3O7FDF05gIBACCo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184648-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184648: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
X-Osstest-Versions-That:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 12 Feb 2024 09:31:05 +0000

flight 184648 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184648/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184645
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184645
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184645
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184645
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184645
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184645
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184645
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184645
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184645
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184645
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184645
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184645
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
baseline version:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369

Last test of basis   184648  2024-02-12 01:53:55 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Mon Feb 12 09:32:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 09:32:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679420.1056852 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSfT-0000TN-1P; Mon, 12 Feb 2024 09:32:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679420.1056852; Mon, 12 Feb 2024 09:32:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSfS-0000TG-Tx; Mon, 12 Feb 2024 09:32:02 +0000
Received: by outflank-mailman (input) for mailman id 679420;
 Mon, 12 Feb 2024 09:32:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZSfR-0000JS-5T
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 09:32:01 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9267e74d-c989-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 10:32:00 +0100 (CET)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-33b728b6981so846408f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 01:32:00 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bk26-20020a0560001d9a00b0033b1277e95dsm6335462wrb.77.2024.02.12.01.31.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 01:31:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9267e74d-c989-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707730319; x=1708335119; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HfaETnidYyphApd9wup2kjg1zF8iLQfSJLe/NLyFEpE=;
        b=RvzCe1BHNDwgWT6t+yM3DiMroUoVMSTfyVNrjhctneJMp68gjPMQRXwdDejoJ9VZhV
         gagJ4UaNn1h0jwT2FEekCcFy6vfU5L2xifBEg/6nuO34WaafH6Qz3FrtJFKASJ1aj6lD
         1Yq3oH52fPT2/FdwsL5dRcLvWrO3wMoyvZ+4/G/ZOgkGUT6MC/HJyoAtLLXIAWeYALf6
         ky9pOmfLlpLa5GZfEEtTLIJoVocib5fi1chjZhJP+OfZbKHZPeAsCU3TZnQVFchppMbk
         CYEfcr9zHSdT1PYjT3Na2c//8peLU/RAhJHCWy+FB/Ff6fEWHey4KbCWMgdWHjqUF7NK
         3lkg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707730319; x=1708335119;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HfaETnidYyphApd9wup2kjg1zF8iLQfSJLe/NLyFEpE=;
        b=wjL+eW+ETkUA/McS//Te9ReA+AtaGa0zwTa4Le2Pk8YVW8sRlJN1BDBXOoQmQBbjbQ
         T9gu4T+McLxMF1ju8AhqqgaRbwdNY4T5Rt1rVL1RT9OPnWJKkd8KArhuYHhYY12R4tLi
         /ukaIjIG+kCwnqftxd1wtnG1XkSggvJNcyFHS2lzpiL8Hz6aTX+yAeLcDvVlBTOe4CzE
         HTH2Ju/2fr9L9g2dIMiQVw77Ro/IBokj6tHoSUw24oR3j83JQ62+TlXCel/JqyIE6q2H
         Bdo5Omv2sPR0NSpboK+Gvjo4RPOjrKx9v0yJB9frcvIURMhei9C3sJ1edXd8CcdDPvoE
         +a/Q==
X-Gm-Message-State: AOJu0Yww0F+A9wAsLYf7Nr6S+5BfYNl17HB46BcSJ3XJ/c5tcaFwFCo8
	h9NQTFCq7km7zDCUb7NoWuFlpsE3wIYTNUcLIpbOv8jQeJ6wUcdVXcb0KYWZcA==
X-Google-Smtp-Source: AGHT+IE6l/bTmUMaI4Bp5FXIpZF81zidKgkOFOGL0Fwh0XBA+cBWJvX5iIhfvnlLnAsLI+EzQdxETQ==
X-Received: by 2002:adf:ec8d:0:b0:33b:5563:3502 with SMTP id z13-20020adfec8d000000b0033b55633502mr3995626wrn.17.1707730319505;
        Mon, 12 Feb 2024 01:31:59 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUCcoJ+U8ziTA8mFG5wPVz7ZY2bSjGwCpmeYKy4m/RKdz4cr7dtxfDcBGQCLkfdANYEII/g8NgauGGKVA/Fvp4JD82DxvCsiGvFKHV2Eqhq7jno
Message-ID: <8715f268-be7b-43c6-bb55-4839427014f5@suse.com>
Date: Mon, 12 Feb 2024 10:32:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/7] VT-d: parse ACPI "SoC Integrated Address Translation
 Cache Reporting Structure"s
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <b5a58dee-9a4c-4833-be59-b52c62f7137d@suse.com> <ZcScJvAhI7CRJhAZ@macbook>
 <a96fe270-15f1-492e-92d6-2a7ceda11dec@suse.com> <ZcXpwtIDZ5RobRtw@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcXpwtIDZ5RobRtw@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 09.02.2024 10:00, Roger Pau Monné wrote:
> On Thu, Feb 08, 2024 at 04:29:34PM +0100, Jan Beulich wrote:
>> On 08.02.2024 10:17, Roger Pau Monné wrote:
>>> On Mon, Feb 05, 2024 at 02:55:17PM +0100, Jan Beulich wrote:
>>>> +        {
>>>> +            dprintk(XENLOG_WARNING VTDPREFIX,
>>>> +                    " Non-existent device (%pp) is reported in SATC scope!\n",
>>>> +                    &PCI_SBDF(satcu->segment, b, d, f));
>>>> +            ignore = true;
>>>
>>> This is kind of reporting is incomplete: as soon as one device is
>>> found the loop is exited and no further detection happens.  If we want
>>> to print such information, we should do the full scan and avoid
>>> exiting early when a populated device is detected.
>>
>> Not sure I follow, but first of all - these are dprintk()s only, so
>> meant to only help in dev environments. Specifically ...
>>
>>>> +        }
>>>> +        else
>>>> +        {
>>>> +            ignore = false;
>>>> +            break;
>>>> +        }
>>>> +    }
>>>> +
>>>> +    if ( ignore )
>>>> +    {
>>>> +        dprintk(XENLOG_WARNING VTDPREFIX,
>>>> +                " Ignore SATC for seg %04x as no device under its scope is PCI discoverable!\n",
>>
>> ... this message is then issued only bogus entries were found. IOW
>> when a real device was found, there's no real reason to report N
>> other bogus ones, I think.
> 
> I guess it's a question of taste.  I do find it odd (asymmetric
> maybe?) that we stop reporting non-existing devices once a valid
> device is found.  Makes me wonder what's the point of reporting them
> in the first place, if the list of non-existing devices is not
> complete?

Since you look to not be taking this into account, let me re-emphasize
that these are dprintk() only. In the event of an issue, seeing the
log messages you at least get a hint of one device that poses a
problem. That may or may not be enough of an indication for figuring
what's wrong. Making the loop run for longer than necessary when
especially in a release build there's not going to be any change (but
the logic would become [slightly] more complex, as after setting
"ignore" to true we'd need to avoid clearing it back to false) is just
pointless imo. IOW I view this 1st message as merely a courtesy for
the case where the 2nd one would end up also being logged.

>> Plus, whatever we change here ought to also / first change in
>> register_one_rmrr().
> 
> I could live with those looking differently, or register_one_rmrr()
> can be adjusted later.  Existing examples shouldn't be an argument to
> not make new additions better.

While I generally agree with this principle, in cases like this one it
needs weighing against consistency. Which I consider more important
here, to reduce the chance of making mistakes when fiddling with this
code later again.

>>>> +    satcu = xzalloc(struct acpi_satc_unit);
>>>> +    if ( !satcu )
>>>> +        return -ENOMEM;
>>>> +
>>>> +    satcu->segment = satc->segment;
>>>> +    satcu->atc_required = satc->flags & 1;
>>>
>>> I would add this as a define in actbl2.h:
>>>
>>> #define ACPI_DMAR_ATC_REQUIRED (1U << 0)
>>>
>>> Or some such (maybe just using plain 1 is also fine).
>>
>> I intended to do so, but strictly staying in line with what Linux has.
>> To my surprise they use a literal number and have no #define. Hence I
>> didn't add any either.
> 
> I would prefer the define unless you have strong objections, even if
> that means diverging from Linux.

I could probably accept such a #define living in one of dmar.[ch]. I'd
rather not see it go into actbl2.h.

>>>> +
>>>> +    dev_scope_start = (const void *)(satc + 1);
>>>> +    dev_scope_end   = (const void *)satc + header->length;
>>>> +    ret = acpi_parse_dev_scope(dev_scope_start, dev_scope_end,
>>>> +                               &satcu->scope, SATC_TYPE, satc->segment);
>>>> +
>>>> +    if ( !ret && satcu->scope.devices_cnt )
>>>> +    {
>>>> +        ret = register_one_satc(satcu);
>>>> +        /*
>>>> +         * register_one_satc() returns greater than 0 when a specified
>>>> +         * PCIe device cannot be detected. To prevent VT-d from being
>>>> +         * disabled in such cases, reset the return value to 0 here.
>>>> +         */
>>>> +        if ( ret > 0 )
>>>> +            ret = 0;
>>>> +    }
>>>> +    else
>>>> +        xfree(satcu);
>>>
>>> You can safely use scope_devices_free() even if acpi_parse_dev_scope()
>>> failed, so you could unify the freeing here, instead of doing it in
>>> register_one_satc() also.
>>
>> Moving that out of acpi_parse_dev_scope() would feel wrong - if a
>> function fails, it would better not leave cleanup to its caller(s).
> 
> Given that the caller here is the one that did the allocation my
> preference would be to also do the cleanup there - register_one_satc()
> has no need to know what needs freeing, and allows unifying the
> cleanup in a single place.

Hmm, you're right about the odd freeing behavior. I guess I really
ought to change that, but then first for register_one_rmrr() (seeing
that DRHD and ATSR parsing also do it that way). Which then of course
means also touching add_one_user_rmrr().

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 09:40:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 09:40:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679425.1056862 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSn8-0001nX-SW; Mon, 12 Feb 2024 09:39:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679425.1056862; Mon, 12 Feb 2024 09:39:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSn8-0001nQ-Pk; Mon, 12 Feb 2024 09:39:58 +0000
Received: by outflank-mailman (input) for mailman id 679425;
 Mon, 12 Feb 2024 09:39:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IbWo=JV=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rZSn8-0001nK-Dr
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 09:39:58 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ae47314e-c98a-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 10:39:56 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a36126ee41eso386296466b.2
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 01:39:56 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 st10-20020a170907c08a00b00a3c5e6515d0sm26099ejc.24.2024.02.12.01.39.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 01:39:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae47314e-c98a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707730796; x=1708335596; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=3VAFCSbbbrHScA37MngxsDzGnmc1khZJpSHHyywTSog=;
        b=tUuQ62yTwbNhIQww6WBHN0xgahdMRQyTZAP5W9rTj83vbT1sOeRRtlQcUeaZ3ZUGch
         hl6Ki1WkO8vozOb8/mBdJFxVH9Blm/B+R8ZoZbqMPz4lvO6JMtUhhM5Hxe6H5a/Y8rZ9
         qL98muGkNqxQKal3qDfj2PoMZbgHg54QWr+x4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707730796; x=1708335596;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3VAFCSbbbrHScA37MngxsDzGnmc1khZJpSHHyywTSog=;
        b=fhVX+LGJ8QPnAaEdsHhJG3TUpXXy/aQTovmvPhGJWC7116OfPXPoYu8Q1g79UKkf7m
         A91pKWEMCKQgHA6+3znq3OVK5Emqv/g3G81w9nPM5CQQsj4uCzHhpSL7w64LW1NwbeQ8
         iaQxJhVmpmEEcghJIuUonHg3f/fOQT77HwnHNAiHEOfONl365KqdaIq+pGzDEJ2IcdJu
         2T6fHty9VqpFnIB6IOl+kUS9TZBe3T2/yvNtilZA2oe840ysPLynbbh4O5xBS7gxQaIH
         fDa+3vE0f0cuxeMCwLa2Axr0A3LPf5LYn3RFTln8sLMYJEOko9LPykJHIeMU2yJcb0xI
         G5BA==
X-Gm-Message-State: AOJu0YzBNgK4nDIXTFpIZM3KbdWCY94SncqFo60BL9mVyNzpyMnFn/3V
	5hEqv2Vw1jthqCkE39aYBnTTX3XstDUSNriSm6vcRH0Aswf6dvaj+nWcDwY3vWc=
X-Google-Smtp-Source: AGHT+IEAahnidk9VcHFd4s75uh1e1phYpGWuYesYKYDabpgBPcVHh2tWxvLeUj8U/9id8HT2kUoStw==
X-Received: by 2002:a17:906:1999:b0:a36:50fb:37ef with SMTP id g25-20020a170906199900b00a3650fb37efmr4117291ejd.28.1707730795717;
        Mon, 12 Feb 2024 01:39:55 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVgFtFFtEtD2mHGf2yGl1jBRoh/JgKaK6IJ7Qh0XNhSqV1ZUz3TZwsb3OUpjAHhU1V1yDlCa4qTQt1oYrImWTYNbqMTZoz5Svq5/5gfhXG2+6m3i+pV0Ex2XeoN372u31d+WYU+282rRabGcXhy
Date: Mon, 12 Feb 2024 10:39:54 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 2/7] IOMMU: rename and re-type ats_enabled
Message-ID: <ZcnnautuDD2qluQm@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <467d24e1-8ed3-4dda-a334-70ff747bf94b@suse.com>
 <ZcS_uC2t96Lh720Y@macbook>
 <4179ded9-36b8-4d47-997b-40d056e033af@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <4179ded9-36b8-4d47-997b-40d056e033af@suse.com>

On Thu, Feb 08, 2024 at 04:49:46PM +0100, Jan Beulich wrote:
> On 08.02.2024 12:49, Roger Pau Monné wrote:
> > On Mon, Feb 05, 2024 at 02:55:43PM +0100, Jan Beulich wrote:
> >> Make the variable a tristate, with (as done elsewhere) a negative value
> >> meaning "default". Since all use sites need looking at, also rename it
> >> to match our usual "opt_*" pattern. While touching it, also move it to
> >> .data.ro_after_init.
> >>
> >> The only place it retains boolean nature is pci_ats_device(), for now.
> > 
> > Why does it retain the boolean nature in pci_ats_device()?
> > 
> > I assume this is to avoid having to touch the line again in a further
> > patch, as given the current logic pci_ats_device() would also want to
> > treat -1 as ATS disabled.
> 
> No, then I would need to touch the line. The function wants to treat
> -1 as "maybe enabled", so the caller can know whether a device is an
> ATS device regardless of whether ATS use is fully off, or only
> "soft-off".

I have to admit I'm slightly concerned about this soft-off.  Given the
current status of ATS itself in Xen, and the technology itself, I
think a user should always opt-in to ATS usage.

> > I think this is all fine because you add additional opt_ats > 0 checks
> > before the call to pci_ats_device(), but would be good to know this is
> > the intention.
> 
> Note how amd_iommu_disable_domain_device() does not gain such a
> check, for exactly the reason named above: The function would better
> turn off ATS whenever enabled, no matter for what reason.
> 
> And of course - none of this "soft-off" vs "fully off" matters for
> AMD (which is the only user of the function) right now anyway, seeing
> they don't have an equivalent of the ATC_REQUIRED flag.
> 
> >> In AMD code re-order conditionals to have the config space accesses
> >> after (cheaper) flag checks.
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >> ---
> >> In domain_context_mapping_one() I'm a little puzzled that translation
> >> type is selected based on only IOMMU and global properties, i.e. not
> >> taking the device itself into account.
> > 
> > That seems like a bug to me, we should check that the device supports
> > ATS (and has it enabled) before setting the translation type to
> > CONTEXT_TT_DEV_IOTLB unconditionally.  We should likely use
> > ats_device() instead of ats_enabled in domain_context_mapping_one().
> 
> Will try to remember to add yet another patch then.
> 
> > There's also IMO a second bug here, which is that we possibly attempt
> > to flush the device IOTLB before having ATS enabled.  We flush the
> > device TLB in domain_context_mapping_one(), yet ATS is enabled by the
> > caller afterwards (see domain_context_mapping()).
> 
> You may be right with this; I'd need to read up on whether such
> flushing is permissible.
> 
> >> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> >> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> >> @@ -185,10 +185,11 @@ static int __must_check amd_iommu_setup_
> >>          dte->ex = ivrs_dev->dte_allow_exclusion;
> >>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
> >>  
> >> -        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> >> +        if ( opt_ats > 0 &&
> >>               !ivrs_dev->block_ats &&
> >> -             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
> >> -            dte->i = ats_enabled;
> >> +             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> >> +             pci_ats_device(iommu->seg, bus, pdev->devfn) )
> >> +            dte->i = true;
> >>  
> >>          spin_unlock_irqrestore(&iommu->lock, flags);
> >>  
> >> @@ -248,10 +249,11 @@ static int __must_check amd_iommu_setup_
> >>          ASSERT(dte->sys_mgt == MASK_EXTR(ivrs_dev->device_flags,
> >>                                           ACPI_IVHD_SYSTEM_MGMT));
> >>  
> >> -        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> >> +        if ( opt_ats > 0 &&
> >>               !ivrs_dev->block_ats &&
> >> -             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
> >> -            ASSERT(dte->i == ats_enabled);
> >> +             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> >> +             pci_ats_device(iommu->seg, bus, pdev->devfn) )
> >> +            ASSERT(dte->i);
> >>  
> >>          spin_unlock_irqrestore(&iommu->lock, flags);
> >>  
> >> @@ -268,9 +270,10 @@ static int __must_check amd_iommu_setup_
> >>  
> >>      ASSERT(pcidevs_locked());
> >>  
> >> -    if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> >> +    if ( opt_ats > 0 &&
> >>           !ivrs_dev->block_ats &&
> >>           iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> >> +         pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> >>           !pci_ats_enabled(iommu->seg, bus, pdev->devfn) )
> > 
> > Seeing that this same set of conditions is used in 3 different checks,
> > could we add a wrapper for it?
> > 
> > opt_ats > 0 && !ivrs_dev->block_ats &&
> > iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> > pci_ats_device(iommu->seg, bus, pdev->devfn)
> > 
> > pci_device_ats_capable()? or some such.
> 
> I was pondering that, yes (iirc already once when adding block_ats).
> Problem is the name. "capable" isn't quite right when considering
> the tristate opt_ats. And pci_device_may_use_ats() reads, well,
> clumsy to me. If you have any good idea for a name that's fully
> applicable and not odd or overly long, I can certainly introduce
> such a helper.

But if ATS is soft-disabled (-1) or hard disabled (0), it's fine to
consider the devices as not ATS capable for the context here?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 09:42:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 09:42:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679430.1056871 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSp9-0003SB-77; Mon, 12 Feb 2024 09:42:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679430.1056871; Mon, 12 Feb 2024 09:42:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSp9-0003S4-4P; Mon, 12 Feb 2024 09:42:03 +0000
Received: by outflank-mailman (input) for mailman id 679430;
 Mon, 12 Feb 2024 09:42:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hp+9=JV=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rZSp8-0003Rw-En
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 09:42:02 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f87053d4-c98a-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 10:42:00 +0100 (CET)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2d0ce22b5f3so34869121fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 01:42:00 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 t11-20020ac2548b000000b005116aed81a6sm819654lfk.294.2024.02.12.01.41.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 01:41:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f87053d4-c98a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707730920; x=1708335720; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Aw416LoobCcO1ExpuJjj5JbWkkcVqBczmGfK13E16y0=;
        b=V4ZeBI/UvjNKYV/HCbm9ubPfkii6CpdTGHJQdK+I/jGWiPR/hICy10++t5GwJJ+Ufl
         uBGxoiY9Ux4p91sNT/uMqXHKgQndKML0/Ll8AT9WD8yV9AIvjqUXzYYpt2briBABwtSd
         NzHoHPtgjA+xSNNRetxy3lpl/t/KcG3eNyCqrGrZHJjwZsBeianVlg3OneXFuIWvEN8j
         tMGhsUQtL5MZHi8fgJRhWhkHwwaaW9DATWRZ257sHgh3loqVC27h0ltfliCFnUSJEq6Q
         fv+uI9GGKo3WFSMgyWrVNzt9/cWvsNZ1Fq7/wUCn+JpP/y9pvYOKH1/zcBHgsDNWI9Bx
         37Ng==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707730920; x=1708335720;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Aw416LoobCcO1ExpuJjj5JbWkkcVqBczmGfK13E16y0=;
        b=bPGsj22PpIcXZe724MYWvAFEksRbJ1GTlC3JBcDd2TR6dtwh775nhnEs/9osUn2RhW
         eRp9Z//600e8um+0ONwyPi9gBVyifZYZx4sdzrg3koeJI3sInSuK3nlD7tQre8ozqkBr
         TdsC3vcPJwAj5mQ2q8HSM/y3bhRufcXEAiNK3NN3x0rzNSP9WDrjgeSD9Lz7GPkyClzn
         bY9gRUSBjbMuU8GUbKVrRzKgl7qWFSct1w+x8/gPG1XUsHyJgRwUv/AP7N4LlhCm4BMT
         cczf/Uu+oWOrLX/en0cfV+ywyIFSyWyhHxP8+JmyXjEZF1+lPNiZHaZOtk3X3OrMiwuH
         2hNw==
X-Forwarded-Encrypted: i=1; AJvYcCVXtu8sqTcZ955vlnPm/sLzJ5AaOBb2OF3cE6IT8bwImad9GPyQAQKCU6eYAepmPSoyv0SspxixxtJiJ3HDB9q7xVh4sBTzY76Sd8xIS4o=
X-Gm-Message-State: AOJu0YyUdd0IIx8bXWsZXlTLzUE/dnfDKtfm67Ntwl3K9EsQm5uu5gmP
	YbyhNuOd40fjLFNXIGo3YFWlcXgV5AE999JvHsqokywzXv+Zl0H2
X-Google-Smtp-Source: AGHT+IHdvS4gEegSgL/6+sDf8IPuXOIXwVa7hhQ2WPFrsnL4oP/M4fIhSqzL12lFXpUb14wOGMWgaw==
X-Received: by 2002:a05:6512:78c:b0:511:93c1:a142 with SMTP id x12-20020a056512078c00b0051193c1a142mr223687lfr.25.1707730919758;
        Mon, 12 Feb 2024 01:41:59 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVlsQECgiRKcEVDbVxVyMgTCLNeFijmv3TkWqikOz+6Yewei+xXC9QUBzY4x6zDZKhg/QPXnftShGsvLrnWoJE4+k/Iz/E7j5CSc3ePNKVMS1wLGh8w+qc0ZanAiNtMg1j0+2O6jic=
Message-ID: <a75199e3e66c76ae48af2bde80ee913a9c39fa75.camel@gmail.com>
Subject: Re: [PATCH v8 1/7] automation: ensure values in
 EXTRA_FIXED_RANDCONFIG are separated by new line
From: Oleksii <oleksii.kurochko@gmail.com>
To: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org
Cc: Doug Goldstein <cardoe@cardoe.com>, Stefano Stabellini
	 <sstabellini@kernel.org>
Date: Mon, 12 Feb 2024 10:41:58 +0100
In-Reply-To: <c516259d-46f1-4bef-b4d0-4cfe18a3cb8a@amd.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
	 <8b6f208f0b4327392cfddbfa6809117cc7686750.1707499278.git.oleksii.kurochko@gmail.com>
	 <c516259d-46f1-4bef-b4d0-4cfe18a3cb8a@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 09:12 +0100, Michal Orzel wrote:
> Hi Oleksii,
Hi Michal,

>=20
> On 09/02/2024 19:00, Oleksii Kurochko wrote:
> >=20
> >=20
> > Kconfig tool expects each configuration to be on a new line.
> >=20
> > The current version of the build script puts all of
> > ${EXTRA_FIXED_RANDCONFIG}
> > in a single line and configs are seperated by spaces.
> >=20
> > As a result, only the first configuration in
> > ${EXTRA_FIXED_RANDCONFIG} will
> > be used.
> There is no need for this patch. If you want the variables to be
> separated by new lines,
> just use a '|' symbol after EXTRA_FIXED_RANDCONFIG (see all
> definitions of EXTRA_XEN_CONFIG).

Thanks a lot. I'll drop this path then.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 09:47:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 09:47:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679458.1056897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZStv-0004yP-50; Mon, 12 Feb 2024 09:46:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679458.1056897; Mon, 12 Feb 2024 09:46:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZStv-0004yI-2E; Mon, 12 Feb 2024 09:46:59 +0000
Received: by outflank-mailman (input) for mailman id 679458;
 Mon, 12 Feb 2024 09:46:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZStt-0004vd-2A
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 09:46:57 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a80160b3-c98b-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 10:46:55 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-410cd2c7f27so5544385e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 01:46:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 q18-20020a7bce92000000b00410c04e5455sm3525390wmj.20.2024.02.12.01.46.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 01:46:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a80160b3-c98b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707731214; x=1708336014; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nlhs7BLlVe73pNQjDo+s/qQOaUld7mWaqTgNZBzwLcg=;
        b=OExSBxOcwYrYI04ZJyKvmTepKCSI2uSgjugnscNJ8hCumU/quA74CQkQmNVQfd4P27
         AKVdWF3BHNjjmSz3B87VldF7NyctvGOwCisIlT77ROaVOK6h7AyuLj0EtDy4ZZDQYBj8
         pKjt/0aI58r/IbuM1VXd4SyuiEFG2kWRjo/wdkrxVA4Sj68vZsQUqThnqdAb5ynPMJUh
         HxtodAjjvvw1k/4XvCzuufqrE9PgXYVUzkmEMZjOyUX5pE6ySUdNNxn3emyBpyEk6aaB
         caFZbAOLF+mgVYKboORiTREj130GQRJtBBJj+Lldu+O3Y2yibO5FKSnVsDUT/Bzy5Mcg
         Bbvg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707731214; x=1708336014;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nlhs7BLlVe73pNQjDo+s/qQOaUld7mWaqTgNZBzwLcg=;
        b=IRlF0NdrfMYbZA9A1dfoG5t0wBf3nq0L6KXCMIMXfNlo/E2l1NQ2a4pqBgWOMDJRwf
         8v9JI2nq+JLGldcy71mHpmIja9MA0A5iC4Zjg3aemASHfV1ut8IYbqj/p7uz2ozR3/13
         d8s/NLc1o2EO2WAbjC2f9+74lxToi7cD7cGTT0rgmZH6hRSEpF7Gf/E8zz95UsEzwfA7
         2EauSzzPBrsRAung84SR8CLRF610Lq8cNxfZamv7gCtoje1QTws15xNh5HsYJ8kkJWnC
         CTD2XSorsSatQdi6tm0Zd/M2JQHCHft19VrJhlUIS19lNJhsFk3oN4FKUuySEK1lx+vD
         n9lw==
X-Gm-Message-State: AOJu0YwhP6mMEyVz74jM8y9Djjq1OVL4wUVOpW1L/Yw3M2UlLCANjKoY
	Z9JW52HQ5zzCMg8T0LUld8JZucCOdzlFiQQV8YX7FLwiHB//BvKBjfAAa1l1jg==
X-Google-Smtp-Source: AGHT+IG48c5IbieNOatakkD4O/WCk/J8CeowBdUAo2oqvZo352aCDln0/1C4u+/lmpnB6KGGiYRhqw==
X-Received: by 2002:a05:600c:3596:b0:410:e4fd:8259 with SMTP id p22-20020a05600c359600b00410e4fd8259mr929792wmq.0.1707731214656;
        Mon, 12 Feb 2024 01:46:54 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVWPZYyCV+TN4o5xYRUk+Vekv7ZHGhdn2eL6/xjYpeCrx+mvZTTTn4O/UFe6IuTKmgE/9QKkaUuhFPnDPiHxLJLjn+9TyvA2xOytUYHLimY4K9nOw1/6NRqhsZC+W/91CVpIVdPW5CkkADkLOX1L64BL7yFrcuozqZwrvpemcEdskDOMBNf2KYFFtiIkp/gsQ++DhKuhtKppgLiXFLyFsb2SgNO3GCdMK5EfQ==
Message-ID: <561102e1-7f02-4928-86c1-9e058a405c4b@suse.com>
Date: Mon, 12 Feb 2024 10:46:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 7/7] VT-d: move {,un}map_vtd_domain_page()
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <22e7036c-cf49-4160-bd26-fbba6b67ff5d@suse.com> <ZcXkpk8XxJmGO1EM@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcXkpk8XxJmGO1EM@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 09.02.2024 09:39, Roger Pau Monné wrote:
> On Mon, Feb 05, 2024 at 02:57:30PM +0100, Jan Beulich wrote:
>> ..., thus allowing them to become static. There's nothing x86-specific
>> about these functions anyway.
>>
>> Since only the "iommu_inclusive_mapping" parameter declaration would be
>> left in the file, move that as well. There's nothing VT-d specific about
>> it (anymore?): "dom0-iommu=map-inclusive" is similarly generic, and
>> documentation also doesn't say anything.
> 
> Hm, I guess documentation should at least say that
> iommu_inclusive_mapping is x86 specific, because it's not parsed on
> Arm and hence might give the wrong impression that it's actually
> acknowledged there.

In v2 I'm adding "(x86)" there.

>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks.

> Albeit I think it would be better to put the parsing in generic
> iommu.c, so that the option gets parsed on Arm and
> arch_iommu_hwdom_init() can print a warning message about it not
> supported on Arm.

Hmm, I would have considered doing things the other way around - make
that part of parsing in parse_dom0_iommu_param() x86-only. I would
feel odd to introduce an option to Arm, just to be able to report
that it's unsupported. The more when generic option parsing code will
already log unrecognized options (sadly such log messages aren't seen
in the serial log, for being issued too early). But let's ask Arm
folks what they'd prefer, by adding all of them to To:.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 09:47:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 09:47:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679461.1056907 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSuX-0005TF-CJ; Mon, 12 Feb 2024 09:47:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679461.1056907; Mon, 12 Feb 2024 09:47:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZSuX-0005T8-97; Mon, 12 Feb 2024 09:47:37 +0000
Received: by outflank-mailman (input) for mailman id 679461;
 Mon, 12 Feb 2024 09:47:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Hp+9=JV=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rZSuW-0005Sp-12
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 09:47:36 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bfe01837-c98b-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 10:47:35 +0100 (CET)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2d0a0e40672so26129741fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 01:47:35 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 w21-20020a05651c103500b002d0fad1ed27sm9899ljm.44.2024.02.12.01.47.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 01:47:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfe01837-c98b-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707731255; x=1708336055; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=9gSSYFcIMJkZXjr0kQ/zifKsz/U8RjjYGbhYK+dZAz0=;
        b=ljogK0G8liayvBLzqHBHJB6bEjugjMf30yLnrTdpuL+veKWOv5E1J90KXqITrHKbgr
         Mvdo4+bW/kzFpzeXI3ELjMYlXNW8XhRbYPz5H/hwpfJ0J5OTPhJVhTn8ZW06RCOKAFvd
         JoWFjzrbQkfL594q6FL3onwu+mJe2GGxmvlyZtqZ4cqWlGN09LtI+ttEVKkzdxRb4/EV
         kxUvWCIdHSR58umOh6Rx1w8ZWku/eQLLem/ayegr7XVtvRO1BE/bkZQc2sFa3my4cthH
         t80kAFA2Xs/YfHROsHAKBPdbyRkBe5EP6zcexaVh5o5X/JnZacpBmjyTuVgtnBwL+CF0
         CABA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707731255; x=1708336055;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9gSSYFcIMJkZXjr0kQ/zifKsz/U8RjjYGbhYK+dZAz0=;
        b=wkrsssBP8zEprAN1nPiazAUASHPWgdqDScFnwDw1jJutuHJIHyJJ8YySg0ZwT0pZel
         JYDpN3zQjZ0NTnth8IhOuNLIip3qAfZLTmON5Jg87TadOkR6KePlNok+ZnQnU6YM+FGo
         jairu0YrZRo6cwWOov9LB/MxBMylDTZZXFokP+68LLhfGdJ4cMaq7XQUmw659cN8yRXr
         SNdnMGN1N0ZMbxCBK4g76hfPTp+7g3rMVKTjjKin1hZI8FQRndmS4CDBURJ9wT/SDiQH
         gs8r7YA6mwiLFvG9UxUM7X40sahDl8NnFkYj7KeAgEO+Hr3Y986Z/mXL+5/nlz85S7fG
         bYuQ==
X-Gm-Message-State: AOJu0Yx/e/LuQZaY8hAUKVeMTbbGg0SA7mKLue4XFMs5f0RoRlsiMSyY
	n7z00gphTZ8rjHMiMG494T2ls2MAknIlkL2AVjtRpjzEAitmM1lm
X-Google-Smtp-Source: AGHT+IFMX/YroMfcqFxRcBbyvnmYQrWw03ALhQuj2JmQFdD7Ae3k3HrIR7wQ7B8fRBK6tvjcB4PxnQ==
X-Received: by 2002:a2e:964b:0:b0:2d0:7e5b:31d2 with SMTP id z11-20020a2e964b000000b002d07e5b31d2mr3022182ljh.4.1707731254447;
        Mon, 12 Feb 2024 01:47:34 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWqdxyraeD2aFfW0n4zHvHCreKfJzMI8JziIhm3rhgBYW1gfIaECAYyosUja7SxHCd2JxUeJlkZIG1mjj/JmnjcW838SMGcWb6nwLRZkPemcyvUYs0m2zBMMK5EHmRwXmY6Ebunp1w=
Message-ID: <3d7dd7aa52191752dfbe92374f06fd9742cd55df.camel@gmail.com>
Subject: Re: [PATCH v8 2/7] automation: introduce fixed randconfig for RISC-V
From: Oleksii <oleksii.kurochko@gmail.com>
To: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org
Cc: Doug Goldstein <cardoe@cardoe.com>, Stefano Stabellini
	 <sstabellini@kernel.org>
Date: Mon, 12 Feb 2024 10:47:33 +0100
In-Reply-To: <738a0947-0c54-4c13-b632-a5e711894e7c@amd.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
	 <3f662e3b4ded36d982ff04ed5cdfdd8e36f83594.1707499278.git.oleksii.kurochko@gmail.com>
	 <738a0947-0c54-4c13-b632-a5e711894e7c@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 09:39 +0100, Michal Orzel wrote:
> Hi Oleksii,
Hi Michal,

>=20
> On 09/02/2024 19:00, Oleksii Kurochko wrote:
> >=20
> >=20
> > This patch introduces the file riscv-fixed-randconfig.yaml,
> > which includes all configurations that should be disabled for
> > randconfig builds.
> >=20
> > Suggested-by: Stefano Stabellini <sstabellini@kernel.org>
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > The patch were introduced after discussion in a topic:
> > =C2=A0
> > https://lore.kernel.org/xen-devel/cover.1701966261.git.oleksii.kurochko=
@gmail.com
> > /
> > =C2=A0---
> > Changes in V8:
> > =C2=A0- Nothing changed. Only rebase
> > ---
> > Changes in V7:
> > =C2=A0- Nothing changed. Only rebase
> > ---
> > Changes in V6:
> > =C2=A0- The patch was introduced in this version of patch series.
> > ---
> > =C2=A0automation/gitlab-ci/build.yaml=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 8 +=
+++----
> > =C2=A0automation/gitlab-ci/riscv-fixed-randconfig.yaml | 7 +++++++
> > =C2=A02 files changed, 11 insertions(+), 4 deletions(-)
> > =C2=A0create mode 100644 automation/gitlab-ci/riscv-fixed-
> > randconfig.yaml
> >=20
> > diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-
> > ci/build.yaml
> > index 6d2cb18b88..376eb17f9c 100644
> > --- a/automation/gitlab-ci/build.yaml
> > +++ b/automation/gitlab-ci/build.yaml
> > @@ -512,6 +512,8 @@ alpine-3.18-gcc-debug-arm64-boot-cpupools:
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 CONFIG_BOOT_TIME_CPUPOOLS=3Dy
> >=20
> > =C2=A0# RISC-V 64 cross-build
> > +include: 'automation/gitlab-ci/riscv-fixed-randconfig.yaml'
> I don't think there is a need to create a new file for that.
> You could define an anchor in build.yaml:
>=20
> # RISC-V 64 cross-build
> .riscv-fixed-randconfig:
> =C2=A0 variables: &riscv-fixed-randconfig
> =C2=A0=C2=A0=C2=A0 EXTRA_FIXED_RANDCONFIG: |
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 CONFIG_COVERAGE=3Dn
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 CONFIG_EXPERT=3Dy
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 CONFIG_GRANT_TABLE=3Dn
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 CONFIG_MEM_ACCESS=3Dn
>=20
> and reference it in the job:
>=20
> archlinux-current-gcc-riscv64-randconfig:
> =C2=A0 extends: .gcc-riscv64-cross-build
> =C2=A0 variables:
> =C2=A0=C2=A0=C2=A0 CONTAINER: archlinux:current-riscv64
> =C2=A0=C2=A0=C2=A0 KBUILD_DEFCONFIG: tiny64_defconfig
> =C2=A0=C2=A0=C2=A0 RANDCONFIG: y
> =C2=A0=C2=A0=C2=A0 <<: *riscv-fixed-randconfig
I've created a new file just for convenience, build.yaml. It is pretty
large, and it's not always easy to navigate, especially when you don't
remember a specific name.
This is not directly related to this patch, but it seems to me that it
would be better to create arch-specific files and include them in
build.yaml.

If it would be better not to create a new file, I am okay to drop it
and add everything to build.yaml.

Thanks.

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Mon Feb 12 10:06:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 10:06:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679508.1056934 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTDA-0003KA-CG; Mon, 12 Feb 2024 10:06:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679508.1056934; Mon, 12 Feb 2024 10:06:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTDA-0003K3-9X; Mon, 12 Feb 2024 10:06:52 +0000
Received: by outflank-mailman (input) for mailman id 679508;
 Mon, 12 Feb 2024 10:06:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IbWo=JV=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rZTD9-0003Jw-8C
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 10:06:51 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7033cf91-c98e-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 11:06:50 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-410c804e933so6481855e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 02:06:50 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 u8-20020a056000038800b0033ae50e2c6asm6370248wrf.83.2024.02.12.02.06.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 02:06:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7033cf91-c98e-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707732409; x=1708337209; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=BBAtL6PJR+0fE4z+LXz79KwAgSwje9B6k+Smiv1M6RI=;
        b=PlTGrG+9C2/owH+a0UpvRgQAF6mGjH+sqCJFDwZZJDpfUTXxrROedb6cwlUtKLJtl/
         lFMTKcZ7UM0xcVHGqmkyJPPdCWfBIiIIMo06RLvTcnH8/PtDrpA+1QQOiioX6FU5GKtZ
         LbYgepy4WNZTMzP1MyK6szX5vaaTCbQu0CEE8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707732409; x=1708337209;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=BBAtL6PJR+0fE4z+LXz79KwAgSwje9B6k+Smiv1M6RI=;
        b=U0/r5pre4ice8+UyWUWkWqzI6ZYpaK1xFUgG4aa/SN1rzWeCu9I3L8WqokzB1qY4Ug
         k3Frib+vv8QWnt2bEmJqO2RfAgV02uyHjfTE3+fbM/OlXTmKCYPv2QzmMc+6ObZEUrx8
         bmwL4RW2uepR7hjMJVihU4pFxl9xnXkcLmKZFlR6z+WDshlkX3K4OsEt7T1o7bbu+QMU
         HobhBpp0IpzcOdvjD87u1UmmKNtDFiXCMcwi0OTFPkIUbA7EZwFAQDfkNwC7DZeJ8g5e
         mFpYV0d5GspgkLVGLUaHJEw2JmkOLzmaoHs21KqTJcRzqgV1mlJVli23cWGpklw/2yUw
         F3fA==
X-Gm-Message-State: AOJu0YwY2lhtSO94TvIAxB6WmHIX+PAMa4zudytd0HmQibrfhlBA5yHE
	FHcC8qHCPROjoLybXoaLLW7RQuluvq/I2RTJLOBxKG38ev8HesPGzX3PhVvFS/Y=
X-Google-Smtp-Source: AGHT+IEKKKFCjRii5wStWyhwJpjWmekFqTQkCxQt4mTmEblJSrWNBSaOWZj/LhgEZvUJDCdCtJ6d+A==
X-Received: by 2002:adf:a19a:0:b0:33b:3cc9:1e6e with SMTP id u26-20020adfa19a000000b0033b3cc91e6emr4732286wru.7.1707732409559;
        Mon, 12 Feb 2024 02:06:49 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXq9Kk/X13iYCumfzpiVJvVw5wL/nja6Mfk3a1MEP7C7kx5Ipl4H+1xQiZwU6jH82XQdw1ux6uHxJgHOtA7luhcAjOS/3/IXGL64Eix4wbMn98j
Date: Mon, 12 Feb 2024 11:06:48 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 1/7] VT-d: parse ACPI "SoC Integrated Address Translation
 Cache Reporting Structure"s
Message-ID: <ZcntuO3reX_HcYem@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <b5a58dee-9a4c-4833-be59-b52c62f7137d@suse.com>
 <ZcScJvAhI7CRJhAZ@macbook>
 <a96fe270-15f1-492e-92d6-2a7ceda11dec@suse.com>
 <ZcXpwtIDZ5RobRtw@macbook>
 <8715f268-be7b-43c6-bb55-4839427014f5@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <8715f268-be7b-43c6-bb55-4839427014f5@suse.com>

On Mon, Feb 12, 2024 at 10:32:00AM +0100, Jan Beulich wrote:
> On 09.02.2024 10:00, Roger Pau Monné wrote:
> > On Thu, Feb 08, 2024 at 04:29:34PM +0100, Jan Beulich wrote:
> >> On 08.02.2024 10:17, Roger Pau Monné wrote:
> >>> On Mon, Feb 05, 2024 at 02:55:17PM +0100, Jan Beulich wrote:
> >>>> +        {
> >>>> +            dprintk(XENLOG_WARNING VTDPREFIX,
> >>>> +                    " Non-existent device (%pp) is reported in SATC scope!\n",
> >>>> +                    &PCI_SBDF(satcu->segment, b, d, f));
> >>>> +            ignore = true;
> >>>
> >>> This is kind of reporting is incomplete: as soon as one device is
> >>> found the loop is exited and no further detection happens.  If we want
> >>> to print such information, we should do the full scan and avoid
> >>> exiting early when a populated device is detected.
> >>
> >> Not sure I follow, but first of all - these are dprintk()s only, so
> >> meant to only help in dev environments. Specifically ...
> >>
> >>>> +        }
> >>>> +        else
> >>>> +        {
> >>>> +            ignore = false;
> >>>> +            break;
> >>>> +        }
> >>>> +    }
> >>>> +
> >>>> +    if ( ignore )
> >>>> +    {
> >>>> +        dprintk(XENLOG_WARNING VTDPREFIX,
> >>>> +                " Ignore SATC for seg %04x as no device under its scope is PCI discoverable!\n",
> >>
> >> ... this message is then issued only bogus entries were found. IOW
> >> when a real device was found, there's no real reason to report N
> >> other bogus ones, I think.
> > 
> > I guess it's a question of taste.  I do find it odd (asymmetric
> > maybe?) that we stop reporting non-existing devices once a valid
> > device is found.  Makes me wonder what's the point of reporting them
> > in the first place, if the list of non-existing devices is not
> > complete?
> 
> Since you look to not be taking this into account, let me re-emphasize
> that these are dprintk() only. In the event of an issue, seeing the
> log messages you at least get a hint of one device that poses a
> problem. That may or may not be enough of an indication for figuring
> what's wrong. Making the loop run for longer than necessary when
> especially in a release build there's not going to be any change (but
> the logic would become [slightly] more complex, as after setting
> "ignore" to true we'd need to avoid clearing it back to false) is just
> pointless imo. IOW I view this 1st message as merely a courtesy for
> the case where the 2nd one would end up also being logged.

I will not insist anymore.

> >>>> +    satcu = xzalloc(struct acpi_satc_unit);
> >>>> +    if ( !satcu )
> >>>> +        return -ENOMEM;
> >>>> +
> >>>> +    satcu->segment = satc->segment;
> >>>> +    satcu->atc_required = satc->flags & 1;
> >>>
> >>> I would add this as a define in actbl2.h:
> >>>
> >>> #define ACPI_DMAR_ATC_REQUIRED (1U << 0)
> >>>
> >>> Or some such (maybe just using plain 1 is also fine).
> >>
> >> I intended to do so, but strictly staying in line with what Linux has.
> >> To my surprise they use a literal number and have no #define. Hence I
> >> didn't add any either.
> > 
> > I would prefer the define unless you have strong objections, even if
> > that means diverging from Linux.
> 
> I could probably accept such a #define living in one of dmar.[ch]. I'd
> rather not see it go into actbl2.h.

Fine.  I think the current open coding of 1 in Linux is wrong.  Other
flag fields in DMAR structures have the related defines.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 10:11:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 10:11:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679513.1056943 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTHl-0005hj-Tm; Mon, 12 Feb 2024 10:11:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679513.1056943; Mon, 12 Feb 2024 10:11:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTHl-0005hc-RH; Mon, 12 Feb 2024 10:11:37 +0000
Received: by outflank-mailman (input) for mailman id 679513;
 Mon, 12 Feb 2024 10:11:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IbWo=JV=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rZTHk-0005hW-Iq
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 10:11:36 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 19efdc7c-c98f-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 11:11:34 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-410ed9f8621so2262925e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 02:11:34 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 t6-20020a05600c450600b0040fc56712e8sm8177384wmo.17.2024.02.12.02.11.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 02:11:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19efdc7c-c98f-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707732694; x=1708337494; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=VJn1GjhiDpMwzcbKVeGFs/v2i5m08gdiaVt7JZm4M4s=;
        b=tA5p/ssR3yUF/z799+h/zhiZqEqg7c7+2bpbHvgVGK//y5eVjmU6ccVzg6fhHdX/LF
         j1qQdjvarwqmViUsNagad21ydPHURKzCWPoRcwk2UpnxaZ1pju8/yh8N/ceW+cchAia8
         30KjY3QGUDMDKzlyr31z8RbQIb6B2UyKj6img=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707732694; x=1708337494;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=VJn1GjhiDpMwzcbKVeGFs/v2i5m08gdiaVt7JZm4M4s=;
        b=vyg+9fZFrnIlTC7kr9ldWUhy0/3Os7C0wmO1htID45qgRNXxAXPXPDTLQKBp1DfEJv
         peCUbbBZVRhPbNATIZ7SQmpS0X21//Ho7OZ+G0VhzuaaRhy57+FA5ljXNBxvsOQc70l3
         5jm4a46KYQ6zd4uNufPVGelcyAUGuziiWmeIqUVXR8V+XqMc8/Vs2R6lPT+qymcYbgj2
         S4MZaMYmjBhZjcg23Zfon9as6H6NUONRg9AV4/LEOWBYdhfMZMGifSvqsxxvZ0/H4dAe
         1kZiohylA3YG/laIcpWpGGjeV7w5WwLgherYb7UCP/Z4RwEYtFYPfvXfVDjz8eQxivcv
         w7Ig==
X-Forwarded-Encrypted: i=1; AJvYcCVcpw2Sg7u7FeETR8Pdx6J+gIR/YSp5baAW6G6gzAmYR0blSSO+s4TxBUbf0igwV0WcD8jkM0td/wx5EP/amCM2MWd0Qqr+SKktoRbiPVE=
X-Gm-Message-State: AOJu0YzvNLrjtqfnBwqxw0pjrftnr/8RA4+ECEUROAUF247Gku7KoOKZ
	Ew9mOcEXn+PPwOcVtxmKi8zaEe8ZNQ07TPL7aZUHxqFne933b2qBsThE7jB4OOM=
X-Google-Smtp-Source: AGHT+IEWL0sbI65hxb2TgU0UIhxc32dWpWvcXEdwJaP0/NnAazqkqgcoJbwS+n+8VibdyQLNH17eXQ==
X-Received: by 2002:a05:600c:1c85:b0:410:ec61:5e90 with SMTP id k5-20020a05600c1c8500b00410ec615e90mr605542wms.10.1707732694419;
        Mon, 12 Feb 2024 02:11:34 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU1H3xHWQJBENnH1PPk6F9ckQh7S6OSQL9yqehdIubhvNS7iB3/AiJrgTjHaINjrWTwTYds6JXlFfl1kZkArKQqvJcTimXfz1j778Br0WmMmT2ObThQUAOuGYh8yMsjlhYCv7CW6Mn+C8urm+idbo+FtljfPNlUlKFGKx170+P/D/g1rxqyzYfntnQGD2YeEADGkptNkII7Bbkb/O4BuHnhybBYPuNb9p4/F79rth0c3OB+vJQpaH2GO4M=
Date: Mon, 12 Feb 2024 11:11:33 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
Subject: Re: [PATCH 7/7] VT-d: move {,un}map_vtd_domain_page()
Message-ID: <Zcnu1en6aZuYoVpZ@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <22e7036c-cf49-4160-bd26-fbba6b67ff5d@suse.com>
 <ZcXkpk8XxJmGO1EM@macbook>
 <561102e1-7f02-4928-86c1-9e058a405c4b@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <561102e1-7f02-4928-86c1-9e058a405c4b@suse.com>

On Mon, Feb 12, 2024 at 10:46:55AM +0100, Jan Beulich wrote:
> On 09.02.2024 09:39, Roger Pau Monné wrote:
> > On Mon, Feb 05, 2024 at 02:57:30PM +0100, Jan Beulich wrote:
> >> ..., thus allowing them to become static. There's nothing x86-specific
> >> about these functions anyway.
> >>
> >> Since only the "iommu_inclusive_mapping" parameter declaration would be
> >> left in the file, move that as well. There's nothing VT-d specific about
> >> it (anymore?): "dom0-iommu=map-inclusive" is similarly generic, and
> >> documentation also doesn't say anything.
> > 
> > Hm, I guess documentation should at least say that
> > iommu_inclusive_mapping is x86 specific, because it's not parsed on
> > Arm and hence might give the wrong impression that it's actually
> > acknowledged there.
> 
> In v2 I'm adding "(x86)" there.

Maybe part of this should be done as a separate patch so it can be
backported to stable branches?

> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> > 
> > Acked-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Thanks.
> 
> > Albeit I think it would be better to put the parsing in generic
> > iommu.c, so that the option gets parsed on Arm and
> > arch_iommu_hwdom_init() can print a warning message about it not
> > supported on Arm.
> 
> Hmm, I would have considered doing things the other way around - make
> that part of parsing in parse_dom0_iommu_param() x86-only. I would
> feel odd to introduce an option to Arm, just to be able to report
> that it's unsupported. The more when generic option parsing code will
> already log unrecognized options (sadly such log messages aren't seen
> in the serial log, for being issued too early). But let's ask Arm
> folks what they'd prefer, by adding all of them to To:.

FWIW, that (moving part of the parsing of parse_dom0_iommu_param() to
x86-only code) would also be fine.  As long as both
dom0-iommu=map-inclusive and iommu_inclusive_mapping are handled
equally and the documentation is updated to reflect that.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 10:28:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 10:28:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679517.1056954 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTYK-0000Gr-AW; Mon, 12 Feb 2024 10:28:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679517.1056954; Mon, 12 Feb 2024 10:28:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTYK-0000Gk-6e; Mon, 12 Feb 2024 10:28:44 +0000
Received: by outflank-mailman (input) for mailman id 679517;
 Mon, 12 Feb 2024 10:28:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9jSY=JV=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZTYI-0000Ge-U9
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 10:28:42 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7d629936-c991-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 11:28:40 +0100 (CET)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2d09b21a8bbso34190071fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 02:28:40 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a22-20020aa7d756000000b0056140d51b7asm2613355eds.83.2024.02.12.02.28.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 02:28:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d629936-c991-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707733720; x=1708338520; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=jUUI3GfqaHKmy5SV4Q2DqQy4jjrcpq08WVUneH3czzI=;
        b=g0G3GSbFzZNoeHS4JZEqe0ndmDg/hoUr+i74C9sTIMDGCKwSYMaIEULLCgefMVMo6x
         f4zUHKptaWXDzLumbzf8TrgH4qShjdmT20eVCbClfzggFEc3fZLRJ18RnyuhCttfpofm
         dmzQedmF6l/o3Co110/O13uPHLMHnBYhjzLeQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707733720; x=1708338520;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jUUI3GfqaHKmy5SV4Q2DqQy4jjrcpq08WVUneH3czzI=;
        b=LClOmOo64DonfvxTWbPXYrTQyh7VFASpR1HdMZn4c3tsDS/CWKWMX/9P1l0yUeCeXs
         kz2+5fneexN/aj2DyeR0g47Flgy6sY+UsEqpR9HVdmnke/tZEoC287AUoSOec3FLnkdU
         34G4El4VgCdVq8voxVYprnR08hvOU9GbFZBeM+r/Wgj0in8eX7wMOiW8pyaOPevj0cLP
         jKAU9VMXG/aBewr9Rbinc2LD9qc54Pa6Q9XMH62RgKK0CirmZN8nzMhIhSpLrO7mJhI7
         5zY6g/Vv6aIETXSHGuv8zCGav4PL44T/71S/CcqMXy/OCZvQ489b/HXe4wjBS5+tBlKG
         nWlg==
X-Gm-Message-State: AOJu0Yypysln8UbGZqwH7TEd4lt5YNmW1t8zD7INF/nyY2WOX56+DQFo
	AikvKAxZtSi+cIpNiPrVRtKiqMXskoIbNqwYYd0Wne2ckvngkvjx8FmWH+G1eXk=
X-Google-Smtp-Source: AGHT+IFpRB2TBiEfByAU22ZXMlRhKnmIAe28zTZbsTzbv+YFU43V3QvCc36UDjwfHqMF7B03L5CsVw==
X-Received: by 2002:a2e:960b:0:b0:2d0:a8f6:c882 with SMTP id v11-20020a2e960b000000b002d0a8f6c882mr4234067ljh.42.1707733720146;
        Mon, 12 Feb 2024 02:28:40 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXM1r3UoT4T13pdtBMst4pWYpM/jLdTKkhnmNmIM0GB72q18iqC1mnVzTMUW8E9jGrrBf/tHg5kq2PonCkiQZJ/uGwyax0QZj6xGzo=
Date: Mon, 12 Feb 2024 10:28:39 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 2/4] libxl: Add support for blktap vbd3
Message-ID: <12968873-6fc7-4c76-9bf9-6beff73a714e@perard>
References: <20240201183024.145424-1-jandryuk@gmail.com>
 <20240201183024.145424-3-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240201183024.145424-3-jandryuk@gmail.com>

On Thu, Feb 01, 2024 at 01:30:22PM -0500, Jason Andryuk wrote:
> This patch re-introduces blktap support to libxl.  Unlike earlier
> versions, it does not link against any blktap library.  libxl changes
> are needed to write to the vbd3 backend XenStore nodes.
> 
> blktap has three components.  tapdisk is a daemon implementing the disk
> IO, NBD (Network Block Device), and Xen PV interfaces.  tap-ctl is a
> tool to control tapdisks - creating, starting, stopping and freeing.
> tapback manages the XenStore operations and instructs tapdisk to
> connect.
> 
> It is notable that tapdisk performs the grant and event channel ops, but
> doesn't interact with XenStore.  tapback performs XenStore operations
> and notifies tapdisks of values and changes.
> 
> The flow is: libxl writes to the "vbd3" XenStore nodes and runs the
> block-tap script.  The block-tap script runs tap-ctl to create a tapdisk
> instance as the physical device.  tapback then sees the tapdisk and
> instructs the tapdisk to connect up the PV blkif interface.
> 
> This is expected to work without the kernel blktap driver, so the
> block-tap script is modified accordingly to write the UNIX NBD path.
> 
> backendtype=tap was not fully removed previously, but it would never
> succeed since it would hit the hardcoded error in disk_try_backend().
> It is reused now.
> 
> An example command to attach a vhd:
> xl block-attach vm 'vdev=xvdf,backendtype=tap,format=vhd,target=/srv/target.vhd'
> 
> Format raw also works to run an "aio:" tapdisk.
> 
> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 10:37:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 10:37:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679521.1056964 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTgU-0002qK-0o; Mon, 12 Feb 2024 10:37:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679521.1056964; Mon, 12 Feb 2024 10:37:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTgT-0002qD-Tu; Mon, 12 Feb 2024 10:37:09 +0000
Received: by outflank-mailman (input) for mailman id 679521;
 Mon, 12 Feb 2024 10:37:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9jSY=JV=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZTgS-0002q7-OZ
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 10:37:08 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ab92ffc3-c992-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 11:37:07 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-5116b017503so4096668e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 02:37:07 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 r4-20020a05600c298400b0040fdc7f4fcdsm8240866wmd.4.2024.02.12.02.37.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 02:37:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab92ffc3-c992-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707734227; x=1708339027; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=ZNeG2pIJ4FHUiCpE/SuqfX3/MiVauHYmdF0xmWZokYQ=;
        b=nSY7v9ixmTo2v+22hzbOmAbVRVDs29TUND+uYKEUBACTj/ur5FL6ITxq4lDOjdoI0T
         y8J4UfisWjO0NQke52RUVsBHI7dEk0iD7gvfuTVrxzT8pEZUZisCRAbITUkX7A2w63sX
         nAXr7KvXNasM+fpBNf21A+bJI9Vf8sDiMbTkM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707734227; x=1708339027;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ZNeG2pIJ4FHUiCpE/SuqfX3/MiVauHYmdF0xmWZokYQ=;
        b=HyxW+Dt627KHyAX7hexHRDwY3fYjEYt2SOLkSQZpZ/Yp2FsYSWWbtn/HFENXFEk41C
         keL7iVrCgS0C3NBLyPYa2xML9i5/oJuYSTR+KtQdx/4i+aHe75iOYEeSWyO27zlhW95X
         OfKmV/uQHPcWE3bjT8B6tcyg0yDi/QCNmyJiAm5cZH4vpwGMrHCmfQCtAz1Wi6kAmAHy
         KP0OEQvtS21cOSACOFGAAON2y4Q2u8Xi2+Gdh+9AU060jk7sh/jQEJoXNbOO7aSbXYLH
         ZJOxdseJ+5JYP7Sz6laV3/Smo0PmLfxQM6+X0uBDDMZrvE3GE/qXZOonWN1SalVhtT6n
         JhwQ==
X-Gm-Message-State: AOJu0Yz13zZRby1iUbZ8A8v2zVgoloClWotdVvCgX9/Pc3ceECib4dWJ
	h3+Q6QBfsWId08VKP2dzfgiM041wFaKGzINhZiI0SdC9GqMltec4C0Vf5XLzab4=
X-Google-Smtp-Source: AGHT+IGuil5j5HYCfxQUEtAulBqlU+/v7x1mcVJhJ9zcIduYoikBTsc38dlhHQuvabsBpvFbx2XXyg==
X-Received: by 2002:a05:6512:31d6:b0:511:51a2:e9c4 with SMTP id j22-20020a05651231d600b0051151a2e9c4mr5481039lfe.15.1707734227180;
        Mon, 12 Feb 2024 02:37:07 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWrnCdYKWqscqZmyxTKeM2xjcGSBoicKRQgKOdfaJeO2PJ7Woj17Z7dVioF3V7DE1jC+ZQYdBzt0OKr
Date: Mon, 12 Feb 2024 10:37:06 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
Subject: Re: [PATCH 3/4] hotplug: Update block-tap
Message-ID: <ea8cc573-bcb1-4f6e-b1d1-0e2e22476515@perard>
References: <20240201183024.145424-1-jandryuk@gmail.com>
 <20240201183024.145424-4-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240201183024.145424-4-jandryuk@gmail.com>

On Thu, Feb 01, 2024 at 01:30:23PM -0500, Jason Andryuk wrote:
> Implement a sharing check like the regular block script.
> 
> Checking tapback inside block-tap is too late since it needs to be
> running to transition the backend to InitWait before block-tap is run.
> 
> tap-ctl check will be removed when the requirement for the blktap kernel
> driver is removed.  Remove it now as it is of limited use.
> 
> find_device() needs to be non-fatal allow a sharing check.
> 
> Only write physical-device-path because that is all that tapback needs.
> Also write_dev doesn't handled files and would incorrectly store
> physical-device as 0:0 which would confuse the minor inside tapback

Missing SOB.


Is `block-tap` still going to work with the example given in the script
header? That is:
    "script=block-tap,vdev=xvda,target=<type>:<file>"
Or maybe, that example is already broken?

> ---
> diff --git a/tools/hotplug/Linux/block-tap b/tools/hotplug/Linux/block-tap
> index 89247921b9..5eca09f0f6 100755
> --- a/tools/hotplug/Linux/block-tap
> +++ b/tools/hotplug/Linux/block-tap
> +count_using()
> +{
> +  local file="$1"
> +  local f
> +
> +  local i=0
> +  local base_path="$XENBUS_BASE_PATH/$XENBUS_TYPE"
> +  for dom in $(xenstore-list "$base_path")
> +  do
> +    for dev in $(xenstore-list "$base_path/$dom")

This function is probably missing "local dom dev".

> +    do
> +      f=$(xenstore_read_default "$base_path/$dom/$dev/params" "")
> +      f=$(echo "$f" | cut -d ":" -f 2)
> +
> +      if [ -n "$f" ] && [ "$file" = $f ] ; then
> +          i=$(( i + 1 ))
> +      fi
> +    done
> +  done
> +
> +  echo "$i"
> +}
> +

> +check_tap_sharing()
> +{
> +  local file="$1"
> +  local mode="$2"
> +  local dev
> +
> +  local base_path="$XENBUS_BASE_PATH/$XENBUS_TYPE"
> +  for dom in $(xenstore-list "$base_path") ; do

Should we add "local dom" to the function?

> +    for dev in $(xenstore-list "$base_path/$dom") ; do
> +      f=$(xenstore_read_default "$base_path/$dom/$dev/params" "")

Same here, maybe "local f" would be good to have too.

> @@ -89,15 +183,57 @@ find_device()
>  # the device
>  add()
>  {
> -    dev=$(tap-ctl create -a $target)
> -    write_dev $dev
> +    local minor
> +    local pid
> +    local res
> +
> +    claim_lock "block"
> +
> +    if find_device; then
> +        result=$( check_tap_sharing "$file" "$mode" )
> +        if [ "$result" != "ok" ] ; then
> +            do_ebusy "tap $type file $file in use " "$mode" "${result%% *}"
> +        fi
> +    else
> +        tap_create

The new function tap_create() is doing something similar to the replace
`tap-ctl create` call, right?


>  # Disconnects the device
>  remove()
>  {
> -    find_device
> -    do_or_die tap-ctl destroy -p ${pid} -m ${minor} > /dev/null
> +    local minor
> +    local pid
> +
> +    claim_lock "block"
> +
> +    if tap_shared ; then
> +        return
> +    fi
> +
> +    minor=$( xenstore_read "$XENBUS_PATH/minor" )
> +    pid=$( xenstore_read "$XENBUS_PATH/pid" )
> +
> +    [ -n "$minor" ] || fatal "minor missing"
> +    [ -n "$pid" ] || fatal "pid missing"
> +    do_or_die tap-ctl close -p "$pid" -m "$minor" > /dev/null
> +    do_or_die tap-ctl detach -p "$pid" -m "$minor" > /dev/null

Should we also call `tap-ctl free`, like `tap-ctl destroy` seems to do?

> +
> +    release_lock "block"
>  }
>  
>  command=$1

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 10:42:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 10:42:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679524.1056974 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTlF-0005CO-JT; Mon, 12 Feb 2024 10:42:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679524.1056974; Mon, 12 Feb 2024 10:42:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTlF-0005CH-Ft; Mon, 12 Feb 2024 10:42:05 +0000
Received: by outflank-mailman (input) for mailman id 679524;
 Mon, 12 Feb 2024 10:42:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9jSY=JV=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZTlE-0005C3-5f
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 10:42:04 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5b25dd41-c993-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 11:42:02 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-5114fa38434so3261928e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 02:42:02 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 c6-20020a05600c0a4600b004117e45f12esm296569wmq.22.2024.02.12.02.42.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 02:42:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b25dd41-c993-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707734522; x=1708339322; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=u5pnzoiiByE6csyB3DAuVZmfY6ARX/aQY8Wab7e9s5M=;
        b=gUfeKNIqgJdgQwmGl4GnG0VKTud+2xx3Q6McYOhn/jPEboH+YBF3+X3F6CU7Zif0pN
         dz1jb1MlLOgoRa9SnZBnVD9yCsMd/ecytSBVveXPLHDGZTZEFwOqHmlhEWY1UbuXdTjS
         yoUI4uB9/pU4crbL075yj2D1TuK1u7StbC6dI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707734522; x=1708339322;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=u5pnzoiiByE6csyB3DAuVZmfY6ARX/aQY8Wab7e9s5M=;
        b=wfNOPHih2p9P971Eeqz8V1x861OiQhf+JXixeMxGm7Ma379noJUSI+e+nC/Hxd03Uv
         J4qMcQzDyqMtZ5Pt4gxbz4P3HrhhDhL1KaJtYZj9XgCNxn7Jn0XUN8P3ItuqgVrJmHmg
         cKh5xhoWn3KayvdTx1Y7Kkzm/nZa/MU60/7EV/n6XTIUWhaBijEjq9mfNJNrlFoA6uRu
         v+fWzs45xGVggv5/yberkGlthNRThdse4bnXcP/62f5FsUoKXKyFKxWZhtazMM9QkzLC
         3s+GLLWu69Z+GQ//HKgrmc4jTCrdR6hH3koMNqocTOJ3rSjhRvMp8hxdXHyfYQhvvmll
         Nohw==
X-Gm-Message-State: AOJu0Yw1UnJHIjMymhul1GJNnR0uy4WzsYR644miKvhebCHj+Sjk79G+
	eU3bEsBIB9PwZKdKDMhXrYlvu7CMrLCF6fMlp/MeqN64jS8wUduOHAf07TwDa/Y=
X-Google-Smtp-Source: AGHT+IGqMYP9ZEyfpamnXStOhse1sBmr908Q1Wv8RksxnxNGxCIsdFLZGinfEbtbrGQWBhoihK0Pmw==
X-Received: by 2002:ac2:4bd1:0:b0:511:87eb:8dc4 with SMTP id o17-20020ac24bd1000000b0051187eb8dc4mr3856284lfq.32.1707734521813;
        Mon, 12 Feb 2024 02:42:01 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWg5BzQbNQr+x2B7UtXHLnWkRCk+kkuQQHOtj7Qh0LbMTNYYXny02UdCP3eRE8NUhQR6DgRim7mqNWFuSgJSMNk4iaDdrcmYTQQDcc=
Date: Mon, 12 Feb 2024 10:42:01 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 4/4] libxl: Support blktap with HVM device model
Message-ID: <8f5ec1c2-4730-479f-b878-1409b3bc21f1@perard>
References: <20240201183024.145424-1-jandryuk@gmail.com>
 <20240201183024.145424-5-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240201183024.145424-5-jandryuk@gmail.com>

On Thu, Feb 01, 2024 at 01:30:24PM -0500, Jason Andryuk wrote:
> diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
> @@ -1337,10 +1338,18 @@ char *libxl__device_disk_find_local_path(libxl__gc *gc,
>          LOGD(DEBUG, guest_domid, "Attempting to read node %s", pdpath);
>          path = libxl__xs_read(gc, XBT_NULL, pdpath);
>  
> -        if (path)
> +        if (path) {
>              LOGD(DEBUG, guest_domid, "Accessing cooked block device %s", path);
> -        else
> -            LOGD(DEBUG, guest_domid, "No physical-device-path, can't access locally.");
> +
> +            /* tapdisk exposes disks locally over UNIX socket NBD. */
> +            if (disk->backend == LIBXL_DISK_BACKEND_TAP) {
> +                path = libxl__sprintf(gc, "nbd+unix:///?socket=%s", path);
> +                LOGD(DEBUG, guest_domid,
> +                     "Directly accessing local TAP target %s", path);
> +            }
> +        } else
> +            LOGD(DEBUG, guest_domid,
> +                "No physical-device-path, can't access locally.");

Here, the coding style call to have both side of the if..else to have { }
or none of them. Could you add {}-block for the else side?


Beside that, patch looks fine to me:
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 10:45:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 10:45:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679529.1056983 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTod-0005qi-40; Mon, 12 Feb 2024 10:45:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679529.1056983; Mon, 12 Feb 2024 10:45:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTod-0005qb-1S; Mon, 12 Feb 2024 10:45:35 +0000
Received: by outflank-mailman (input) for mailman id 679529;
 Mon, 12 Feb 2024 10:45:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZTob-0005qR-Qk
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 10:45:33 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d89ab291-c993-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 11:45:32 +0100 (CET)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2d09b21a8bbso34370291fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 02:45:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 b8-20020a05600003c800b0033b5a6b4b9bsm6574107wrg.71.2024.02.12.02.45.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 02:45:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d89ab291-c993-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707734732; x=1708339532; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ozEedm038kZzzThqZKsqIqIgWeSfHq5VvZACBL6sBvs=;
        b=YhTTVFw8oDaGGBa1Ji1k+lh0q4/mZjyza6L5JaJIlGQM2Bqo3UWaMq6jlL3olqcq3/
         2wdU0j3rwn6GtnKDCevCq2GNpL2DBx0w+hPRZKJ9ezcHV+51HSUOglYhOkw7HGEtB04M
         Lj4i8X/e5gJNaY1iYmmBbdEinfVX8N/uvynZCxw+v92RlLL4S1mEfLoIFm3mR1YLpvPK
         ww2fspilTm1wYS/0395F1EFQHuQb1syfXhpMkl6a2vdMDC3QELAmuXYIgOZkiwreDanY
         l/e1jv9Io0TTr3pO5JftrnYjasFJ+t2DdwTTxaV1jBRJ0EZcx6OmYpqId0TGcpiUzB4L
         7+Nw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707734732; x=1708339532;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ozEedm038kZzzThqZKsqIqIgWeSfHq5VvZACBL6sBvs=;
        b=beRK2QqF+DmEJfP8XeXaSUMTxulTVFJGzeIFBwFNkeit9aQiO1G0EnbodcnrsAGgkt
         8Sgq7Ke+G/5/ZC24n+Tc4NP31CKPnWbIAsGpwMVgT0tWtQv3dPLZaHMVoPyz7miguN4Z
         e9c7If2eA1J7nKeajK6ejnfekKZI1oXYH2h50Tjy+mW2aceKugx4Fa5sx4QqTDmTBqer
         b9E988PHst11qxEPlwBHyJdLmwEtuqdVH2tPKPBeSx++W26kaG/VY4ScJMFhQmWdekYQ
         J9YSE8DJpk/0Ls6pyJHZchCLF53xbp3Smre4SdMvym48Ss4oS6e/1TXnE1Xkh9UK278L
         mU7w==
X-Gm-Message-State: AOJu0Yzb675q42W302nBJNn4j0wCba9QMVK1Nm6PI6sZ9SDB0573nJkq
	cJ1t9FwZW5/aKNN3B+UcCcnhWvpwbDSnzc+OGQ2vqtmMkXioVgIFXb4EAJo6IA==
X-Google-Smtp-Source: AGHT+IGEf03lNC/YM35ygrRiY65g8fa2ADByICiNTEqp6Ufzb0xbsA/qkTqP5bScCv/0VX2LvvuNfA==
X-Received: by 2002:a2e:8ecf:0:b0:2d0:cd25:e569 with SMTP id e15-20020a2e8ecf000000b002d0cd25e569mr4223034ljl.5.1707734732232;
        Mon, 12 Feb 2024 02:45:32 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWtU1BZ9cSDCeobDz29G86DNTvnEr6e4pRzBXBdeFJuNalAwGKIjM84tjLD1tHiP7DeBA25lBQZUOCwK3jjQBK7KXeZjIVEyRsUSE+KfNMHUPngrSeM20oH8zrFsTpNwRuYwMuaU45gQcruPrhk
Message-ID: <2fccd004-b35a-4cde-afbb-722cb2413902@suse.com>
Date: Mon, 12 Feb 2024 11:45:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/7] IOMMU: rename and re-type ats_enabled
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <467d24e1-8ed3-4dda-a334-70ff747bf94b@suse.com> <ZcS_uC2t96Lh720Y@macbook>
 <4179ded9-36b8-4d47-997b-40d056e033af@suse.com> <ZcnnautuDD2qluQm@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcnnautuDD2qluQm@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 12.02.2024 10:39, Roger Pau Monné wrote:
> On Thu, Feb 08, 2024 at 04:49:46PM +0100, Jan Beulich wrote:
>> On 08.02.2024 12:49, Roger Pau Monné wrote:
>>> On Mon, Feb 05, 2024 at 02:55:43PM +0100, Jan Beulich wrote:
>>>> Make the variable a tristate, with (as done elsewhere) a negative value
>>>> meaning "default". Since all use sites need looking at, also rename it
>>>> to match our usual "opt_*" pattern. While touching it, also move it to
>>>> .data.ro_after_init.
>>>>
>>>> The only place it retains boolean nature is pci_ats_device(), for now.
>>>
>>> Why does it retain the boolean nature in pci_ats_device()?
>>>
>>> I assume this is to avoid having to touch the line again in a further
>>> patch, as given the current logic pci_ats_device() would also want to
>>> treat -1 as ATS disabled.
>>
>> No, then I would need to touch the line. The function wants to treat
>> -1 as "maybe enabled", so the caller can know whether a device is an
>> ATS device regardless of whether ATS use is fully off, or only
>> "soft-off".
> 
> I have to admit I'm slightly concerned about this soft-off.  Given the
> current status of ATS itself in Xen, and the technology itself, I
> think a user should always opt-in to ATS usage.

The plan is to follow your suggestion in patch 3 and require explicit
enabling for passing through of such devices. For Dom0, however, I
think it is important that we respect the firmware request by default.
The only viable(?!) alternative would be to panic() instead.

>>>> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
>>>> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
>>>> @@ -185,10 +185,11 @@ static int __must_check amd_iommu_setup_
>>>>          dte->ex = ivrs_dev->dte_allow_exclusion;
>>>>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
>>>>  
>>>> -        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
>>>> +        if ( opt_ats > 0 &&
>>>>               !ivrs_dev->block_ats &&
>>>> -             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
>>>> -            dte->i = ats_enabled;
>>>> +             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
>>>> +             pci_ats_device(iommu->seg, bus, pdev->devfn) )
>>>> +            dte->i = true;
>>>>  
>>>>          spin_unlock_irqrestore(&iommu->lock, flags);
>>>>  
>>>> @@ -248,10 +249,11 @@ static int __must_check amd_iommu_setup_
>>>>          ASSERT(dte->sys_mgt == MASK_EXTR(ivrs_dev->device_flags,
>>>>                                           ACPI_IVHD_SYSTEM_MGMT));
>>>>  
>>>> -        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
>>>> +        if ( opt_ats > 0 &&
>>>>               !ivrs_dev->block_ats &&
>>>> -             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
>>>> -            ASSERT(dte->i == ats_enabled);
>>>> +             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
>>>> +             pci_ats_device(iommu->seg, bus, pdev->devfn) )
>>>> +            ASSERT(dte->i);
>>>>  
>>>>          spin_unlock_irqrestore(&iommu->lock, flags);
>>>>  
>>>> @@ -268,9 +270,10 @@ static int __must_check amd_iommu_setup_
>>>>  
>>>>      ASSERT(pcidevs_locked());
>>>>  
>>>> -    if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
>>>> +    if ( opt_ats > 0 &&
>>>>           !ivrs_dev->block_ats &&
>>>>           iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
>>>> +         pci_ats_device(iommu->seg, bus, pdev->devfn) &&
>>>>           !pci_ats_enabled(iommu->seg, bus, pdev->devfn) )
>>>
>>> Seeing that this same set of conditions is used in 3 different checks,
>>> could we add a wrapper for it?
>>>
>>> opt_ats > 0 && !ivrs_dev->block_ats &&
>>> iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
>>> pci_ats_device(iommu->seg, bus, pdev->devfn)
>>>
>>> pci_device_ats_capable()? or some such.
>>
>> I was pondering that, yes (iirc already once when adding block_ats).
>> Problem is the name. "capable" isn't quite right when considering
>> the tristate opt_ats. And pci_device_may_use_ats() reads, well,
>> clumsy to me. If you have any good idea for a name that's fully
>> applicable and not odd or overly long, I can certainly introduce
>> such a helper.
> 
> But if ATS is soft-disabled (-1) or hard disabled (0), it's fine to
> consider the devices as not ATS capable for the context here?

I don't like mixing capability and policy aspects into a resulting
"capable".

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 10:46:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 10:46:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679532.1056994 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTpW-0006LV-CG; Mon, 12 Feb 2024 10:46:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679532.1056994; Mon, 12 Feb 2024 10:46:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZTpW-0006LO-9X; Mon, 12 Feb 2024 10:46:30 +0000
Received: by outflank-mailman (input) for mailman id 679532;
 Mon, 12 Feb 2024 10:46:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=9jSY=JV=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZTpU-0006DK-IK
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 10:46:28 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f8e96f68-c993-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 11:46:27 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-40fd2f7ef55so22515615e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 02:46:27 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 1-20020a05600c024100b0041061f094a2sm8156202wmj.11.2024.02.12.02.46.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 02:46:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f8e96f68-c993-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707734786; x=1708339586; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=IKaRny37rlggD7+Q7v7k3tUewmO/LOXkeak5vLz7CEc=;
        b=GUnsRnJj5y7MxElOGsViBgvTWnLtey5MdrM6kgw0gX0t7Ztl0Y5KU+lW0gxUz0UEP4
         fBVFwka98gmvdHng3gv/Pr+LIAaoCxSjj04vlydtnoS8HdiX2+KwJzYuP6gE7kQq5Beg
         A8fxJFILn2ZJNbDqifpVXke+t62ZopXRVQwyo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707734786; x=1708339586;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=IKaRny37rlggD7+Q7v7k3tUewmO/LOXkeak5vLz7CEc=;
        b=efuJjTrXkC1axW8LEyrUvrlmta/9+n/5zSUcGmAOOnBlcwBw60/yIFKlox6OjV56VM
         jpck5RGBDbtbXT/runDdejcZ0cXmstqragGQ2H3FWVR+UBdxNd6+Pf425r1Ke+kfjzn7
         tbJZQNj9Trea5mzJKL9AejBiepOTa2HsXznMP3jr9clPfeVMlBS4vvwoDnOMRkW7BCHP
         xCuCiFZAODGZVQxA465sUwiO4Ki9rhN2mVCQVY1QiD8+4o7XsydRfSdnqTKIZtgmEN+h
         FgQu+EA2fQJM5DZoqbEQNrTfa4uNsgk7GukKLIlxQ+fB6c0qPgkoErOmS2FYt/gj/aDU
         dh2Q==
X-Gm-Message-State: AOJu0Yz7ANOI8WKdXPKq9DYQN0PB2y++kSYLC5J4ULn/jjq79KyCkTw1
	InsGJwg0hL1NC6eVnC1OWyQaL1I/BOe5OhXRHMpS8gV6OLCp7zQxM4eELlhhMVE=
X-Google-Smtp-Source: AGHT+IE168UjhgjYQRZvqUwbFsc0agU2d3CEYELFR69OlqwkMqUHxJPjRjtMK7QAwatLifjoq5ntxQ==
X-Received: by 2002:a05:600c:46cf:b0:410:4b4:dc7f with SMTP id q15-20020a05600c46cf00b0041004b4dc7fmr6260437wmo.4.1707734786398;
        Mon, 12 Feb 2024 02:46:26 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXwu3EgZRT95uQW2pjaMqIQDUx31/paspiMeKQR2oI0onoJV3YVSTUuOOyOLE0hIGq0BsroqKKUXlcffYnEbO2Ecww0hiaHy37ITGI=
Date: Mon, 12 Feb 2024 10:46:25 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 0/4] libxl: blktap/tapback support
Message-ID: <792bceb5-e39e-4684-b320-825c6782af0e@perard>
References: <20240201183024.145424-1-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240201183024.145424-1-jandryuk@gmail.com>

On Thu, Feb 01, 2024 at 01:30:20PM -0500, Jason Andryuk wrote:
> This patch re-introduces blktap support to libxl.  Unlike earlier
> versions, it does not link against any blktap library.  libxl changes
> are needed to write to the vbd3 backend XenStore nodes.
> 
> blktap has three components.  tapdisk is a daemon implementing the disk
> IO, NBD (Network Block Device), and Xen PV interfaces.  tap-ctl is a
> tool to control tapdisks - creating, starting, stopping and freeing.
> tapback manages the XenStore operations and instructs tapdisk to
> connect.
> 
> It is notable that tapdisk performs the grant and event channel ops, but
> doesn't interact with XenStore.  tapback performs XenStore operations
> and notifies tapdisks of values and changes.
> 
> The flow is: libxl writes to the "vbd3" XenStore nodes and runs the
> block-tap script.  The block-tap script runs tap-ctl to create a tapdisk
> instance as the physical device.  tapback then sees the tapdisk and
> instructs the tapdisk to connect up the PV blkif interface.
> 
> This is expected to work without the kernel blktap driver, so the
> block-tap script is modified accordingly to write the UNIX NBD path.
> (It works with the kernel blktap driver as well - upstream blktap hasn't
> removed the blktap driver requirement yet -
> https://github.com/xapi-project/blktap/pull/364)
> 
> An example command to attach a vhd:
> xl block-attach vm 'vdev=xvdf,backendtype=tap,format=vhd,target=/srv/target.vhd'
> 
> VHD support is important for OpenXT since there are lots of existing
> VHDs which still need supporting.  tapdisk also supports encrypted VHDs
> which is not available in QEMU.
> 
> blktap's tapback needs minimal changes to work with libxl:
> https://github.com/xapi-project/blktap/pull/394

It might be nice to have an entry in the ChangeLog. But it's looks like
the blktap project isn't ready yet, so we might need to point that out.

Cheers,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 11:06:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 11:06:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679539.1057003 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZU8n-0002O9-V9; Mon, 12 Feb 2024 11:06:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679539.1057003; Mon, 12 Feb 2024 11:06:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZU8n-0002O2-Sa; Mon, 12 Feb 2024 11:06:25 +0000
Received: by outflank-mailman (input) for mailman id 679539;
 Mon, 12 Feb 2024 11:06:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZU8n-0002Nw-Ff
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 11:06:25 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c1d82173-c996-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 12:06:23 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-410da137230so4689415e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 03:06:23 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l13-20020a056000022d00b0033b1b1da2adsm6508190wrz.99.2024.02.12.03.06.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 03:06:22 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c1d82173-c996-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707735982; x=1708340782; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=qUnkmWNBQc/+wtjuqNVBCSD2zqxzcbd7I2CDtGO4YNQ=;
        b=I7jz3jihBxy2uf/tNfkf7Mdatv71ZvexW2kZG+EccyiM3c/nWQONYhzu9SKKlFyCUk
         ahytaqWnPX84fmP5kSO8l/vGG6N16O4SaafFfJ9EE9w3v0zvLdiyDNvxBeLVIdFEjlhm
         4Gs+lLQul5sNTFH3OZutyCSV6C3Z6qHARlDgeF5nTz5gWQl8pRfe080K97EqhTt6LcwT
         mFHiM7C+xdG5l5c7KJTmCoBL/AHIlYuljYSjcZFtNg+b1ZUOdxXXEtcC93meHS8pISpy
         F1EzMlcPyxzZWGt4jAGKulTHP+095ucRtcWM6KlfI2PFHTW5VC7Q0rYDHFCeJ2lrveDa
         MjjQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707735982; x=1708340782;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qUnkmWNBQc/+wtjuqNVBCSD2zqxzcbd7I2CDtGO4YNQ=;
        b=wj7EACvHYzaTxPh3Ai96fwsBtz8qREKI5zqjXy75y9fYNTtaOijet4APog6knOnNuG
         NkDs5pbDy5KG5ABT5Tz841G7zrkd+Dc6HfjX8tletbONxcHTUW+qDV74lRN6vmnH+NXu
         /3EWdiKqxpgxBo3uYdo/6HhZUeQwm5rjBMwludR0ljys+m4IUH4Je+1YdymhupCr0tD9
         58zlEl8JSRuluHxG9fIMeijpJ8kgB+dwW+WPUTzs1mV6zEChjQNO1vQrqCFLb5/kT3Bi
         NYItuzz+pYlWowtrpHQa5L7efF7KHXL8xZTRw8fGT4kg41XnVkRIgyD+DqKprossutf1
         5q2A==
X-Gm-Message-State: AOJu0Yw5JzLWNzly9QCvq2M7GtGAvqa+0SVdRKql5+1NjN+CCXBsgybv
	RVRi/1ozg0/2Ig0yMoEUIwB4fbFf1hRbu3GJBIDVAhvIHPJs04hqzClKiVPZBQ==
X-Google-Smtp-Source: AGHT+IGg30SJb2SYCqeEXMX4oYCrPVQOqaOxZ4UENOU93suGmLWaV2lB474W2j6Dn272i1p0ZjbDyQ==
X-Received: by 2002:a5d:4987:0:b0:33b:3ad8:22c5 with SMTP id r7-20020a5d4987000000b0033b3ad822c5mr4085479wrq.68.1707735982544;
        Mon, 12 Feb 2024 03:06:22 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWhYkGuE6gMRxFt+7QVih3RnX5OHvBWaUKfIRHMupXDTKBrFNjPJi9I2LjusY5PVfFSCIfnnUyPxm2HLGX/7esm/9ZdkEllKZGxUYAg/ZaIVfZ7
Message-ID: <f4ed0cbb-0a3a-400f-afc7-19716b403ca1@suse.com>
Date: Mon, 12 Feb 2024 12:06:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/7] VT-d: respect ACPI SATC's ATC_REQUIRED flag
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <72496fb7-8835-4d45-bd92-7e7d612861c7@suse.com> <ZcTMNkGvBT5S5z7L@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcTMNkGvBT5S5z7L@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08.02.2024 13:42, Roger Pau Monné wrote:
> On Mon, Feb 05, 2024 at 02:56:14PM +0100, Jan Beulich wrote:
>> When the flag is set, permit Dom0 to control the device (no worse than
>> what we had before and in line with other "best effort" behavior we use
>> when it comes to Dom0), but suppress passing through to DomU-s unless
>> ATS can actually be enabled for such devices.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> Is ats_device() using acpi_find_matched_atsr_unit() unconditionally
>> actually correct? Shouldn't that check be skipped for root complex
>> integrated devices?
> 
> Yes, I think so, ATSR only lists root ports supporting ATS, because
> the root complex is assumed to always be ATS capable.
> 
> None of this seems to be working then for PCIe endpoints directly in
> the root complex, as ats_device() will always return 0?

That's my understanding. I've now added a bugfix patch near the front of
the series.

>> --- a/docs/misc/xen-command-line.pandoc
>> +++ b/docs/misc/xen-command-line.pandoc
>> @@ -225,7 +225,10 @@ exceptions (watchdog NMIs and unexpected
>>  > Default: `false`
>>  
>>  Permits Xen to set up and use PCI Address Translation Services.  This is a
>> -performance optimisation for PCI Passthrough.
>> +performance optimisation for PCI Passthrough.  Note that firmware may indicate
>> +that certain devices need to have ATS enabled for proper operation. For such
>> +devices ATS will be enabled by default, unless the option is used in its
>> +negative form.
> 
> I'm kind of worried that we add this support while maintaining the
> WARNING below.  If I was an admin I would certainly be worried whether
> my system could lock-up during normal operations, even with the
> devices assigned to dom0 and not a malicious domain.
> 
> I know that enabling ATS is forced on us from DMAR, but still.

I'm with you; see below.

>> --- a/xen/drivers/passthrough/vtd/iommu.c
>> +++ b/xen/drivers/passthrough/vtd/iommu.c
>> @@ -2364,6 +2364,25 @@ static int cf_check intel_iommu_add_devi
>>      if ( ret )
>>          dprintk(XENLOG_ERR VTDPREFIX, "%pd: context mapping failed\n",
>>                  pdev->domain);
>> +    else if ( !pdev->broken )
>> +    {
>> +        const struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
>> +        const struct acpi_satc_unit *satc = acpi_find_matched_satc_unit(pdev);
>> +
>> +        /*
>> +         * Prevent the device from getting assigned to an unprivileged domain
>> +         * when firmware indicates ATS is required, but ATS could not be enabled
>> +         * (e.g. because of being suppressed via command line option).
>> +         */
> 
> I think a safer policy would be to prevent assigning any device that
> has atc_required set unless opt_ats > 1 (ie: the user has explicitly
> opted-in to the usage of ATS).
> 
> While we can't likely avoid ATS being enabled for devices having the
> ATC_REQUIRED flag, we shouldn't allow passthrough to possibly
> untrusted guests without notice.

Switched to that model, including respective wording in the cmdline doc.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 11:35:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 11:35:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679544.1057014 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZUak-0007w9-0H; Mon, 12 Feb 2024 11:35:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679544.1057014; Mon, 12 Feb 2024 11:35:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZUaj-0007w2-Tl; Mon, 12 Feb 2024 11:35:17 +0000
Received: by outflank-mailman (input) for mailman id 679544;
 Mon, 12 Feb 2024 11:35:17 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZUaj-0007vs-Gs; Mon, 12 Feb 2024 11:35:17 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZUaj-0003Os-FA; Mon, 12 Feb 2024 11:35:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZUaj-0003Xt-1U; Mon, 12 Feb 2024 11:35:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZUaj-0003Ve-10; Mon, 12 Feb 2024 11:35:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=kgmUF9n79q8qvoNpcROxqvgH/W0LnXu5CnQ76HB8EwI=; b=qlhwLkDbCLFibLskrdJUekst5i
	HQOiAw5kJRoy0QqcTo0d99JkEfOCG2VqlxM6ZaUNzCvLA1ifetB/kFkPimpcgcXl+y0bhEhG0cWOg
	eBi46l4KgrCuX8fQ2x3SCI6WiHv5bSfoowPWmcOLfk8GTpTxfYeQ9IC/zVwl3aeAslVw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184650-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184650: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=772380f7547bac7e4c10c9ceff87af3f54151ae5
X-Osstest-Versions-That:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 12 Feb 2024 11:35:17 +0000

flight 184650 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184650/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  772380f7547bac7e4c10c9ceff87af3f54151ae5
baseline version:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369

Last test of basis   184634  2024-02-09 08:02:06 Z    3 days
Testing same since   184650  2024-02-12 09:02:17 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Petr Beneš <w1benny@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Simone Ballarin <simone.ballarin@bugseng.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   4e5bd87d24..772380f754  772380f7547bac7e4c10c9ceff87af3f54151ae5 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 12:29:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 12:29:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679560.1057024 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZVQm-00076Z-1G; Mon, 12 Feb 2024 12:29:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679560.1057024; Mon, 12 Feb 2024 12:29:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZVQl-00076S-Ts; Mon, 12 Feb 2024 12:29:03 +0000
Received: by outflank-mailman (input) for mailman id 679560;
 Mon, 12 Feb 2024 12:29:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=guxz=JV=arm.com=Bertrand.Marquis@srs-se1.protection.inumbo.net>)
 id 1rZVQl-000763-E5
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 12:29:03 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05on20604.outbound.protection.outlook.com
 [2a01:111:f400:7e1a::604])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4bf55653-c9a2-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 13:28:59 +0100 (CET)
Received: from AS4PR10CA0025.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:5d8::14)
 by DB9PR08MB9516.eurprd08.prod.outlook.com (2603:10a6:10:451::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.37; Mon, 12 Feb
 2024 12:28:56 +0000
Received: from AM4PEPF00027A5F.eurprd04.prod.outlook.com
 (2603:10a6:20b:5d8:cafe::50) by AS4PR10CA0025.outlook.office365.com
 (2603:10a6:20b:5d8::14) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.38 via Frontend
 Transport; Mon, 12 Feb 2024 12:28:56 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM4PEPF00027A5F.mail.protection.outlook.com (10.167.16.74) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7249.19 via Frontend Transport; Mon, 12 Feb 2024 12:28:55 +0000
Received: ("Tessian outbound 1076c872ecc6:v228");
 Mon, 12 Feb 2024 12:28:55 +0000
Received: from 60c213bfa62b.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 F504CA10-0E81-48DC-9AD7-3B6178588798.1; 
 Mon, 12 Feb 2024 12:28:48 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 60c213bfa62b.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Mon, 12 Feb 2024 12:28:48 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com (2603:10a6:10:25a::24)
 by AS8PR08MB6376.eurprd08.prod.outlook.com (2603:10a6:20b:33e::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.35; Mon, 12 Feb
 2024 12:28:47 +0000
Received: from DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::2257:5fcb:b449:fff0]) by DB9PR08MB6588.eurprd08.prod.outlook.com
 ([fe80::2257:5fcb:b449:fff0%7]) with mapi id 15.20.7270.033; Mon, 12 Feb 2024
 12:28:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4bf55653-c9a2-11ee-98f5-efadbce2ee36
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=cJJe/6zkuFTvIeq2GsnFKYIPig/Rom4bRRnvU/ZyKci/BpanJtv13mRXMwNMTx1OZxe/XSOY/kelj5bnQa+1UhGyHurI/ma29exA+R32SeDRM4oAz76hPahKBPT+6nrQKjcteHCuWXFSdVUDT+tMKMkGuWLK6PPuMB1u9JaSCZKwWvWfDQRD2TgpWFdOOZyaMQE9dvurC1AtT99LL2882XtYtJuOr0l1VnsKSwIEAztKyKvjrDP5aKWfVlbzW0cANE/08awlx7HYML5+dRmpy0vH5fzuJFMe738CYzvsKLejPbxckMLfx8fjCkD6ccIjDS7v3GBIMSV+7LGWq+z8eA==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=4qugOlm4aia2D+S/VPVQ1la0Qd6ycqcbzWeEv3p8IOE=;
 b=Az3zXPxkBphvgWeDU8MBm07AUs8LHEdVCYqQy13ciCFxNCq3y4fC5CnquTqapmzISwM6gR5HkLcgwD0QO7DioX+eTPBujZfbsyrhWCZeXgQEYA0EgJHaxHy+C19sO0q0I0T6irnD2gwNAxAvZSbAz1D5WmIcjnhME2bFtAupkN/a+IylBjJHQHn9FzXFQ+IrFiZ+ezp6DPxiF7pLkCVMWIeEUVGLj89y3j6wkiQDfhhwcBGJILDNKlOhYWE66KwGJd8KRYsqOa5FsiVRHNQDUeoj+BkhmQAaVdVN2h5l3huw6DMqnapVZVW0QKqxSK3rS7yoKFIDSlw/hkp+4M+F0Q==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0
 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4qugOlm4aia2D+S/VPVQ1la0Qd6ycqcbzWeEv3p8IOE=;
 b=KRpRV+pDHaFMDfgjNQf2E2YGte8R3yXYs0Ry9IZGPTz8FiJYBudJSnlrTlpQ5Qqwwwt2ZrCKiREajVRcpTFRW8o3p+2t1izc4Rc/oVviqw+kUZqvWAfj6GtE+kvMu+pgQ2k0IvLZ+4EkiFqH8Qn0dFENfm7v3osKGx/ZUJQDwZs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 85e76e124748d97e
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LdRI8OO/K+n7yfm3UaWYOJ11L3iu6P7Woez5UQ+5MkW2GMM/hQS5U5oFFsMSAK5NSgmxSpvVZ59ypqQazSTclRLIFFWV0RHW/i9nEpvPJq6F8wxy/iyX9SjYvvTFvvw+iUZCvXhkDXHNIKJq+C8krV3J2K1tv8c//MUECSsl8NmFdg0KCjvilp/Ygss0Il26wIw5+SNjlbna/jKn3HHpDa2OZZOeV+ebK10q57Q/kP5HTj5fMUFxq57N6ZtDJYr92DxrcjtKlLSRTJZoWOrXJuZj94/lXL/2gpjEAodN/QiXacAzVtiZvD/3bQaK3hXBmmZdH5TO5dkmMSOWwMif1g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=4qugOlm4aia2D+S/VPVQ1la0Qd6ycqcbzWeEv3p8IOE=;
 b=c7z6u1HqxxiAyPx7vZDH7nfNGeK6OKHLZa64nltQF1jL7ZbHmFgWIVIvfd23WUTbuwwIUf1OdBB56nUgMdWbeASarA+j41kmABi/S9CPj3O4nH4EMcxqKLXkKesIiphZdpmDeYuHZTX8tU7iO/cXUMme+eGLuYphVAF4DDqlggZ6H/eSc28aPo5JQWWKD/rVsA9Th3K4vLWxUI5LXGsDyMZDnZMxaG5MmqG1+pzMGEPnhkD122lRWFcr4v3DptGBk6PDa1nMm/ulWcJwH+dSXP8pfhxiZpycHv1qDFSYiROpuUnO24CHTLdJwjw6Do63o/kNOWmLfFdL8D8sffTfzQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4qugOlm4aia2D+S/VPVQ1la0Qd6ycqcbzWeEv3p8IOE=;
 b=KRpRV+pDHaFMDfgjNQf2E2YGte8R3yXYs0Ry9IZGPTz8FiJYBudJSnlrTlpQ5Qqwwwt2ZrCKiREajVRcpTFRW8o3p+2t1izc4Rc/oVviqw+kUZqvWAfj6GtE+kvMu+pgQ2k0IvLZ+4EkiFqH8Qn0dFENfm7v3osKGx/ZUJQDwZs=
From: Bertrand Marquis <Bertrand.Marquis@arm.com>
To: Jens Wiklander <jens.wiklander@linaro.org>
CC: Xen-devel <xen-devel@lists.xenproject.org>, "patches@linaro.org"
	<patches@linaro.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Michal
 Orzel <michal.orzel@amd.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [XEN PATCH v4] xen/arm: ffa: reclaim shared memory on guest
 destroy
Thread-Topic: [XEN PATCH v4] xen/arm: ffa: reclaim shared memory on guest
 destroy
Thread-Index: AQHaWErq04RDxa7M60+0c7FHpMlvF7EGrWkA
Date: Mon, 12 Feb 2024 12:28:46 +0000
Message-ID: <CA6D5F9E-9FA6-440D-BD0B-7005E3F7A08E@arm.com>
References: <20240205154909.2392231-1-jens.wiklander@linaro.org>
In-Reply-To: <20240205154909.2392231-1-jens.wiklander@linaro.org>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.200.91.1.1)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	DB9PR08MB6588:EE_|AS8PR08MB6376:EE_|AM4PEPF00027A5F:EE_|DB9PR08MB9516:EE_
X-MS-Office365-Filtering-Correlation-Id: a0987e13-92a7-40f8-8195-08dc2bc62e23
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 gDhSj2gnGtuWV/SDWLDfAl7vrWWNOIxR9PATdnRQyekMzuau65OXaDoE7afyOwBcEKqrtvsThlEwdJ//sNO2CMzxG4LHIAqVsAqzKgmvw57E8ACeUs580vwBekQ8pdTZQWAroMbglS1cApch3Eev/WDIWol9CZSZLSGOxYPi0SibLrmrbRpjEClV8TnsiJHQhk6UuyaN7OTEgSFf9QJhOUT+mxwSkANNWgbTTG4kGHTmqUJkDPV6Ll54Gq3UzZjcVzXhC14ijNcaR/IhCur0XpaguXe2vq1Jn/Pfa1Mhz22e4RpnsqOUt/zVtka0fUAJTW9Tax13E9HqizAkCOUEtIElAmbD0Hls8o7Uzych/4m/elBO/DIy7yWSW01fzx9IPOZ3crYa4/zy4YELNwK5GY9GExrToeVqIs60kbM8glH6TEwnAPy5rExoGPSHTAiIpyejglO87c2bFiZkstQoPDEy7DHWZIzF8L6rQMoicLDjwYAVmwftHWYRpz6iZpj4Rr+gu7nFPZERNbZLaklBCCM3Hg7C9uhnDWhdLVEUVIClJAisKIKH9Bw8CUaEpzIdefXSFJyIJkoMtsxJcIto9OIGvbCVf7cpVEgw77oLODW8chU8otjuBgAXpzBIb6wo
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB9PR08MB6588.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(136003)(346002)(39860400002)(366004)(396003)(230922051799003)(186009)(1800799012)(451199024)(64100799003)(5660300002)(8676002)(71200400001)(41300700001)(2616005)(66899024)(30864003)(64756008)(2906002)(4326008)(66946007)(66476007)(66556008)(8936002)(66446008)(76116006)(6916009)(478600001)(83380400001)(38070700009)(53546011)(6486002)(6512007)(54906003)(6506007)(316002)(86362001)(38100700002)(122000001)(33656002)(36756003);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="us-ascii"
Content-ID: <BD33D6EFC95A97448CA4DC6FCBDD8792@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6376
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM4PEPF00027A5F.eurprd04.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	b786eb9b-2d4c-43cc-3405-08dc2bc628df
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	nFjG9NHRSh9oOAr/NIILRsgmehZHisBcT9DJpQSbbwwKCcFKxYw5l9YZVOcxaxLbFIxcEFJsMilIx3/gTNUqm4pGT73VCkqpf17VBT0MMEx+AemXlThnJQmLLCyDLrtaDlWSias6hIs4iasDXp379SHOvFEjAZur2ZMxL5KGvvgh07dt0e+yuhPbQSEoIweAfXz2bFBbpRoKb3DcmmFuXQBWzH2F9fBQMM2HqXYLa42RdasxaDf08QCuG173UtNVfE15EJ08GOlA4l17f0ynPoofCYISLAiEOLrhLnYiWDbK4vjUpc6ppPXRRwn6K5LGtt7oGgRjYaFqdDX5Pr0ioh741Tue4BeoNXX4mrQw/WYI8e+TOCec9VR3lf1TjgEG0HwEQg/KZ0rzn9vTS9ChRDc0lXtmAS/DOmCQ4rCq8QPK3vIT/XHkUjRyWHIGGKyr0GRVW6EJVMCG8fmHPyzrNlD/R066EbSUbRspPNekPCIKPxgDAYLRxs0tBhiZQFDNSiEg/5CBBuEB7x6AuB18/QhXVDTWGSZ4EMmpOsWJYgxeql6d6UmAoZisdEwJDiBRgoP7Z4Slo0Tx4DjSvz4FBz4QWU61MO9f96XaqTWK9pE=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(396003)(346002)(136003)(39860400002)(230922051799003)(451199024)(64100799003)(1800799012)(82310400011)(186009)(36840700001)(46966006)(40470700004)(41300700001)(66899024)(356005)(5660300002)(82740400003)(33656002)(30864003)(2906002)(36756003)(336012)(70586007)(316002)(8936002)(70206006)(478600001)(6486002)(8676002)(6862004)(54906003)(83380400001)(4326008)(2616005)(107886003)(81166007)(86362001)(6506007)(26005)(6512007)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Feb 2024 12:28:55.6584
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: a0987e13-92a7-40f8-8195-08dc2bc62e23
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM4PEPF00027A5F.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB9516

Hi Jens,

> On 5 Feb 2024, at 16:49, Jens Wiklander <jens.wiklander@linaro.org> wrote=
:
>=20
> When an FF-A enabled guest is destroyed it may leave behind memory
> shared with SPs. This memory must be reclaimed before it's reused or an
> SP may make changes to memory used by a new unrelated guest. So when the
> domain is teared down add FF-A requests to reclaim all remaining shared
> memory.
>=20
> SPs in the secure world are notified using VM_DESTROYED that a guest has
> been destroyed. An SP is supposed to relinquish all shared memory to allo=
w
> reclaiming the memory. The relinquish operation may need to be delayed if
> the shared memory is for instance part of a DMA operation.
>=20
> The domain reference counter is increased when the first FF-A shared
> memory is registered and the counter is decreased again when the last
> shared memory is reclaimed. If FF-A shared memory registrations remain
> at the end of of ffa_domain_teardown() a timer is set to try to reclaim
> the shared memory every second until the memory is reclaimed.
>=20
> A few minor style fixes with a removed empty line here and an added new
> line there.
>=20
> Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

>From my point of view all looks good now so with the changes requested
by Andrew handled, feel free to add my R-b in the v5:

Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

Cheers
Bertrand

> ---
>=20
> v4:
> - Retry FFA_MSG_SEND_VM_DESTROYED on the returned errors
>  FFA_RET_INTERRUPTED and FFA_RET_RETRY even after all shared memory
>  handles has been reclaimed.
> - Retry ffa_mem_reclaim() only on temporary returned errors, permanent
>  errors like FFA_RET_INVALID_PARAMETERS is dealt with in recovery mode
>  instead since the SPMC isn't expected to use that error under normal
>  circumstances.
>=20
> v3:
> - Mentioning in the commit message that there are some style fixes
> - Addressing review comments
> - Refactor the ffa_domain_teardown() path to let
>  ffa_domain_teardown_continue() do most of the work.
>=20
> v2:
> - Update commit message to match the new implementation
> - Using a per domain bitfield to keep track of which SPs has been notifie=
d
>  with VM_DESTROYED
> - Holding a domain reference counter to keep the domain as a zombie domai=
n
>  while there still is shared memory registrations remaining to be reclaim=
ed
> - Using a timer to retry reclaiming remaining shared memory registrations
> ---
> xen/arch/arm/tee/ffa.c | 285 ++++++++++++++++++++++++++++++++++-------
> 1 file changed, 236 insertions(+), 49 deletions(-)
>=20
> diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
> index 0793c1c7585d..bbb6b819ee2b 100644
> --- a/xen/arch/arm/tee/ffa.c
> +++ b/xen/arch/arm/tee/ffa.c
> @@ -54,6 +54,7 @@
> #include <xen/mm.h>
> #include <xen/sched.h>
> #include <xen/sizes.h>
> +#include <xen/timer.h>
> #include <xen/types.h>
>=20
> #include <asm/event.h>
> @@ -144,6 +145,12 @@
>  */
> #define FFA_MAX_SHM_COUNT               32
>=20
> +/*
> + * The time we wait until trying to tear down a domain again if it was
> + * blocked initially.
> + */
> +#define FFA_CTX_TEARDOWN_DELAY          SECONDS(1)
> +
> /* FF-A-1.1-REL0 section 10.9.2 Memory region handle, page 167 */
> #define FFA_HANDLE_HYP_FLAG             BIT(63, ULL)
> #define FFA_HANDLE_INVALID              0xffffffffffffffffULL
> @@ -384,11 +391,6 @@ struct ffa_ctx {
>     unsigned int page_count;
>     /* FF-A version used by the guest */
>     uint32_t guest_vers;
> -    /*
> -     * Number of SPs that we have sent a VM created signal to, used in
> -     * ffa_domain_teardown() to know which SPs need to be signalled.
> -     */
> -    uint16_t create_signal_count;
>     bool rx_is_free;
>     /* Used shared memory objects, struct ffa_shm_mem */
>     struct list_head shm_list;
> @@ -402,6 +404,15 @@ struct ffa_ctx {
>     spinlock_t tx_lock;
>     spinlock_t rx_lock;
>     spinlock_t lock;
> +    /* Used if domain can't be torn down immediately */
> +    struct domain *teardown_d;
> +    struct list_head teardown_list;
> +    s_time_t teardown_expire;
> +    /*
> +     * Used for ffa_domain_teardown() to keep track of which SPs should =
be
> +     * notified that this guest is being destroyed.
> +     */
> +    unsigned long vm_destroy_bitmap[];
> };
>=20
> struct ffa_shm_mem {
> @@ -436,6 +447,12 @@ static void *ffa_tx __read_mostly;
> static DEFINE_SPINLOCK(ffa_rx_buffer_lock);
> static DEFINE_SPINLOCK(ffa_tx_buffer_lock);
>=20
> +
> +/* Used to track domains that could not be torn down immediately. */
> +static struct timer ffa_teardown_timer;
> +static struct list_head ffa_teardown_head;
> +static DEFINE_SPINLOCK(ffa_teardown_lock);
> +
> static bool ffa_get_version(uint32_t *vers)
> {
>     const struct arm_smccc_1_2_regs arg =3D {
> @@ -853,7 +870,6 @@ static int32_t handle_partition_info_get(uint32_t w1,=
 uint32_t w2, uint32_t w3,
>             goto out_rx_release;
>         }
>=20
> -
>         memcpy(ctx->rx, ffa_rx, sz);
>     }
>     ctx->rx_is_free =3D false;
> @@ -992,53 +1008,75 @@ static void put_shm_pages(struct ffa_shm_mem *shm)
>     }
> }
>=20
> -static bool inc_ctx_shm_count(struct ffa_ctx *ctx)
> +static bool inc_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
> {
>     bool ret =3D true;
>=20
>     spin_lock(&ctx->lock);
> +
> +    /*
> +     * If this is the first shm added, increase the domain reference
> +     * counter as we need to keep domain around a bit longer to reclaim =
the
> +     * shared memory in the teardown path.
> +     */
> +    if ( !ctx->shm_count )
> +        get_knownalive_domain(d);
> +
>     if (ctx->shm_count >=3D FFA_MAX_SHM_COUNT)
>         ret =3D false;
>     else
>         ctx->shm_count++;
> +
>     spin_unlock(&ctx->lock);
>=20
>     return ret;
> }
>=20
> -static void dec_ctx_shm_count(struct ffa_ctx *ctx)
> +static void dec_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
> {
>     spin_lock(&ctx->lock);
> +
>     ASSERT(ctx->shm_count > 0);
>     ctx->shm_count--;
> +
> +    /*
> +     * If this was the last shm removed, let go of the domain reference =
we
> +     * took in inc_ctx_shm_count() above.
> +     */
> +    if ( !ctx->shm_count )
> +        put_domain(d);
> +
>     spin_unlock(&ctx->lock);
> }
>=20
> -static struct ffa_shm_mem *alloc_ffa_shm_mem(struct ffa_ctx *ctx,
> +static struct ffa_shm_mem *alloc_ffa_shm_mem(struct domain *d,
>                                              unsigned int page_count)
> {
> +    struct ffa_ctx *ctx =3D d->arch.tee;
>     struct ffa_shm_mem *shm;
>=20
>     if ( page_count >=3D FFA_MAX_SHM_PAGE_COUNT )
>         return NULL;
> -    if ( !inc_ctx_shm_count(ctx) )
> +    if ( !inc_ctx_shm_count(d, ctx) )
>         return NULL;
>=20
>     shm =3D xzalloc_flex_struct(struct ffa_shm_mem, pages, page_count);
>     if ( shm )
>         shm->page_count =3D page_count;
>     else
> -        dec_ctx_shm_count(ctx);
> +        dec_ctx_shm_count(d, ctx);
>=20
>     return shm;
> }
>=20
> -static void free_ffa_shm_mem(struct ffa_ctx *ctx, struct ffa_shm_mem *sh=
m)
> +static void free_ffa_shm_mem(struct domain *d, struct ffa_shm_mem *shm)
> {
> +    struct ffa_ctx *ctx =3D d->arch.tee;
> +
>     if ( !shm )
>         return;
>=20
> -    dec_ctx_shm_count(ctx);
> +    dec_ctx_shm_count(d, ctx);
>     put_shm_pages(shm);
>     xfree(shm);
> }
> @@ -1306,7 +1344,7 @@ static void handle_mem_share(struct cpu_user_regs *=
regs)
>         goto out_unlock;
>     }
>=20
> -    shm =3D alloc_ffa_shm_mem(ctx, page_count);
> +    shm =3D alloc_ffa_shm_mem(d, page_count);
>     if ( !shm )
>     {
>         ret =3D FFA_RET_NO_MEMORY;
> @@ -1350,7 +1388,7 @@ static void handle_mem_share(struct cpu_user_regs *=
regs)
>=20
> out:
>     if ( ret )
> -        free_ffa_shm_mem(ctx, shm);
> +        free_ffa_shm_mem(d, shm);
> out_unlock:
>     spin_unlock(&ctx->tx_lock);
>=20
> @@ -1401,7 +1439,7 @@ static int handle_mem_reclaim(uint64_t handle, uint=
32_t flags)
>     }
>     else
>     {
> -        free_ffa_shm_mem(ctx, shm);
> +        free_ffa_shm_mem(d, shm);
>     }
>=20
>     return ret;
> @@ -1486,6 +1524,41 @@ static bool ffa_handle_call(struct cpu_user_regs *=
regs)
>     }
> }
>=20
> +static bool is_in_subscr_list(const uint16_t *subscr, uint16_t start,
> +                              uint16_t end, uint16_t sp_id)
> +{
> +    unsigned int n;
> +
> +    for ( n =3D start; n < end; n++ )
> +    {
> +        if ( subscr[n] =3D=3D sp_id )
> +            return true;
> +    }
> +
> +    return false;
> +}
> +
> +static void vm_destroy_bitmap_init(struct ffa_ctx *ctx,
> +                                   unsigned int create_signal_count)
> +{
> +    unsigned int n;
> +
> +    for ( n =3D 0; n < subscr_vm_destroyed_count; n++ )
> +    {
> +        /*
> +         * Skip SPs subscribed to the VM created event that never was
> +         * notified of the VM creation due to an error during
> +         * ffa_domain_init().
> +         */
> +        if ( is_in_subscr_list(subscr_vm_created, create_signal_count,
> +                               subscr_vm_created_count,
> +                               subscr_vm_destroyed[n]) )
> +            continue;
> +
> +        set_bit(n, ctx->vm_destroy_bitmap);
> +    }
> +}
> +
> static int ffa_domain_init(struct domain *d)
> {
>     struct ffa_ctx *ctx;
> @@ -1501,11 +1574,14 @@ static int ffa_domain_init(struct domain *d)
>     if ( d->domain_id >=3D UINT16_MAX)
>         return -ERANGE;
>=20
> -    ctx =3D xzalloc(struct ffa_ctx);
> +    ctx =3D xzalloc_flex_struct(struct ffa_ctx, vm_destroy_bitmap,
> +                              BITS_TO_LONGS(subscr_vm_destroyed_count));
>     if ( !ctx )
>         return -ENOMEM;
>=20
>     d->arch.tee =3D ctx;
> +    ctx->teardown_d =3D d;
> +    INIT_LIST_HEAD(&ctx->shm_list);
>=20
>     for ( n =3D 0; n < subscr_vm_created_count; n++ )
>     {
> @@ -1515,65 +1591,173 @@ static int ffa_domain_init(struct domain *d)
>         {
>             printk(XENLOG_ERR "ffa: Failed to report creation of vm_id %u=
 to  %u: res %d\n",
>                    get_vm_id(d), subscr_vm_created[n], res);
> -            ctx->create_signal_count =3D n;
> -            return -EIO;
> +            break;
>         }
>     }
> -    ctx->create_signal_count =3D subscr_vm_created_count;
> -
> -    INIT_LIST_HEAD(&ctx->shm_list);
> +    vm_destroy_bitmap_init(ctx, n);
> +    if ( n !=3D subscr_vm_created_count )
> +        return -EIO;
>=20
>     return 0;
> }
>=20
> -static bool is_in_subscr_list(const uint16_t *subscr, uint16_t start,
> -                              uint16_t end, uint16_t sp_id)
> +static void send_vm_destroyed(struct domain *d)
> {
> +    struct ffa_ctx *ctx =3D d->arch.tee;
>     unsigned int n;
> +    int32_t res;
>=20
> -    for ( n =3D start; n < end; n++ )
> +    for ( n =3D 0; n < subscr_vm_destroyed_count; n++ )
>     {
> -        if ( subscr[n] =3D=3D sp_id )
> -            return true;
> -    }
> +        if ( !test_bit(n, ctx->vm_destroy_bitmap) )
> +            continue;
>=20
> -    return false;
> +        res =3D ffa_direct_req_send_vm(subscr_vm_destroyed[n], get_vm_id=
(d),
> +                                     FFA_MSG_SEND_VM_DESTROYED);
> +
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "%pd: ffa: Failed to report destruction of=
 vm_id %u to %u: res %d\n",
> +                   d, get_vm_id(d), subscr_vm_destroyed[n], res);
> +        }
> +
> +        /*
> +         * For these two error codes the hypervisor is expected to resen=
d
> +         * the destruction message. For the rest it is expected that the
> +         * error is permanent and that is doesn't help to resend the
> +         * destruction message.
> +         */
> +        if ( res !=3D FFA_RET_INTERRUPTED && res !=3D FFA_RET_RETRY )
> +            clear_bit(n, ctx->vm_destroy_bitmap);
> +    }
> }
>=20
> -/* This function is supposed to undo what ffa_domain_init() has done */
> -static int ffa_domain_teardown(struct domain *d)
> +static void reclaim_shms(struct domain *d)
> {
>     struct ffa_ctx *ctx =3D d->arch.tee;
> -    unsigned int n;
> +    struct ffa_shm_mem *shm, *tmp;
>     int32_t res;
>=20
> -    if ( !ctx )
> -        return 0;
> +    list_for_each_entry_safe(shm, tmp, &ctx->shm_list, list)
> +    {
> +        register_t handle_hi;
> +        register_t handle_lo;
> +
> +        uint64_to_regpair(&handle_hi, &handle_lo, shm->handle);
> +        res =3D ffa_mem_reclaim(handle_lo, handle_hi, 0);
> +        switch ( res ) {
> +        case FFA_RET_OK:
> +            printk(XENLOG_G_DEBUG "%pd: ffa: Reclaimed handle %#lx\n",
> +                   d, shm->handle);
> +            list_del(&shm->list);
> +            free_ffa_shm_mem(d, shm);
> +            break;
> +        case FFA_RET_DENIED:
> +            /*
> +             * A temporary error that may get resolved a bit later, it's
> +             * worth retrying.
> +             */
> +            printk(XENLOG_G_INFO "%pd: ffa: Failed to reclaim handle %#l=
x : %d\n",
> +                   d, shm->handle, res);
> +            break; /* We will retry later */
> +        default:
> +            /*
> +             * The rest of the error codes are not expected and are assu=
med
> +             * to be of a permanent nature. It not in our control to han=
dle
> +             * the error properly so the object in this case is to try t=
o
> +             * minimize the damage.
> +             *
> +             * FFA_RET_NO_MEMORY might be a temporary error as it it cou=
ld
> +             * succeed if retried later, but treat it as permanent for n=
ow.
> +             */
> +            printk(XENLOG_G_INFO "%pd: ffa: Permanent failure to reclaim=
 handle %#lx : %d\n",
> +                   d, shm->handle, res);
>=20
> -    for ( n =3D 0; n < subscr_vm_destroyed_count; n++ )
> +            /*
> +             * Remove the shm from the list and free it, but don't drop
> +             * references. This results in having the shared physical pa=
ges
> +             * permanently allocate and also keeps the domain as a zombi=
e
> +             * domain.
> +             */
> +            list_del(&shm->list);
> +            xfree(shm);
> +            break;
> +        }
> +    }
> +}
> +
> +static void ffa_domain_teardown_continue(struct ffa_ctx *ctx, bool first=
_time)
> +{
> +    struct ffa_ctx *next_ctx =3D NULL;
> +
> +    send_vm_destroyed(ctx->teardown_d);
> +    reclaim_shms(ctx->teardown_d);
> +
> +    if ( ctx->shm_count ||
> +         !bitmap_empty(ctx->vm_destroy_bitmap, subscr_vm_destroyed_count=
) )
> +    {
> +        printk(XENLOG_G_INFO "%pd: ffa: Remaining cleanup, retrying\n", =
ctx->teardown_d);
> +
> +        ctx->teardown_expire =3D NOW() + FFA_CTX_TEARDOWN_DELAY;
> +
> +        spin_lock(&ffa_teardown_lock);
> +        list_add_tail(&ctx->teardown_list, &ffa_teardown_head);
> +        /* Need to set a new timer for the next ctx in line */
> +        next_ctx =3D list_first_entry(&ffa_teardown_head, struct ffa_ctx=
,
> +                                    teardown_list);
> +        spin_unlock(&ffa_teardown_lock);
> +    }
> +    else
>     {
>         /*
> -         * Skip SPs subscribed to the VM created event that never was
> -         * notified of the VM creation due to an error during
> -         * ffa_domain_init().
> +         * domain_destroy() might have been called (via put_domain() in
> +         * reclaim_shms()), so we can't touch the domain structure anymo=
re.
>          */
> -        if ( is_in_subscr_list(subscr_vm_created, ctx->create_signal_cou=
nt,
> -                               subscr_vm_created_count,
> -                               subscr_vm_destroyed[n]) )
> -            continue;
> +        xfree(ctx);
>=20
> -        res =3D ffa_direct_req_send_vm(subscr_vm_destroyed[n], get_vm_id=
(d),
> -                                     FFA_MSG_SEND_VM_DESTROYED);
> -
> -        if ( res )
> -            printk(XENLOG_ERR "ffa: Failed to report destruction of vm_i=
d %u to  %u: res %d\n",
> -                   get_vm_id(d), subscr_vm_destroyed[n], res);
> +        /* Only check if there has been a change to the teardown queue *=
/
> +        if ( !first_time )
> +        {
> +            spin_lock(&ffa_teardown_lock);
> +            next_ctx =3D list_first_entry_or_null(&ffa_teardown_head,
> +                                                struct ffa_ctx, teardown=
_list);
> +            spin_unlock(&ffa_teardown_lock);
> +        }
>     }
>=20
> +    if ( next_ctx )
> +        set_timer(&ffa_teardown_timer, next_ctx->teardown_expire);
> +}
> +
> +static void ffa_teardown_timer_callback(void *arg)
> +{
> +    struct ffa_ctx *ctx;
> +
> +    spin_lock(&ffa_teardown_lock);
> +    ctx =3D list_first_entry_or_null(&ffa_teardown_head, struct ffa_ctx,
> +                                   teardown_list);
> +    if ( ctx )
> +        list_del(&ctx->teardown_list);
> +    spin_unlock(&ffa_teardown_lock);
> +
> +    if ( ctx )
> +        ffa_domain_teardown_continue(ctx, false /* !first_time */);
> +    else
> +        printk(XENLOG_G_ERR "%s: teardown list is empty\n", __func__);
> +}
> +
> +/* This function is supposed to undo what ffa_domain_init() has done */
> +static int ffa_domain_teardown(struct domain *d)
> +{
> +    struct ffa_ctx *ctx =3D d->arch.tee;
> +
> +    if ( !ctx )
> +        return 0;
> +
>     if ( ctx->rx )
>         rxtx_unmap(ctx);
>=20
> -    XFREE(d->arch.tee);
> +    ffa_domain_teardown_continue(ctx, true /* first_time */);
>=20
>     return 0;
> }
> @@ -1739,6 +1923,9 @@ static bool ffa_probe(void)
>     if ( !init_sps() )
>         goto err_free_ffa_tx;
>=20
> +    INIT_LIST_HEAD(&ffa_teardown_head);
> +    init_timer(&ffa_teardown_timer, ffa_teardown_timer_callback, NULL, 0=
);
> +
>     return true;
>=20
> err_free_ffa_tx:
> --=20
> 2.34.1
>=20



From xen-devel-bounces@lists.xenproject.org Mon Feb 12 12:52:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 12:52:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679566.1057034 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZVn6-0002jE-Us; Mon, 12 Feb 2024 12:52:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679566.1057034; Mon, 12 Feb 2024 12:52:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZVn6-0002j7-R1; Mon, 12 Feb 2024 12:52:08 +0000
Received: by outflank-mailman (input) for mailman id 679566;
 Mon, 12 Feb 2024 12:52:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZVn5-0002iz-IN
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 12:52:07 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8677e6ad-c9a5-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 13:52:05 +0100 (CET)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2d1025717c7so2046211fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 04:52:05 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v9-20020a5d4b09000000b0033b843786e1sm1973101wrq.51.2024.02.12.04.52.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 04:52:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8677e6ad-c9a5-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707742325; x=1708347125; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=KdemOTI8r+uOmKiumhE3Wv6qyJwPjzda+1Srtk+j2xg=;
        b=ZtkZdYOCnQQEAXxiLG5SMyogv8JUg5QfA39THFUayEVgfzJNICjoUQOXPvdTdEMgYy
         FHvQqWzPd0DII9lfW5DNNhPDBz8duOsMmCX2qt6piskZh6b04pjkK2GpLmuzotGVy7i7
         DSkWPEVynQalk9AIHS4kZmvgkALEU/wOUKs7cE1C/PYQjN8iMZxSv52ovPMGpGz2ljOh
         vl6b2cBZV1fFyq1OqkD/7sUMUebySuI2YIYaaQGA16NpKKDPL/F0NGruxr6rjbw0QgXd
         iZ0fOO7EY6Xt8fWpVgPwGxoAmZA+JLSUe8PokwzC9UMYE1KaxRsLmLiI2YQ6DKAwxqpe
         7svA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707742325; x=1708347125;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KdemOTI8r+uOmKiumhE3Wv6qyJwPjzda+1Srtk+j2xg=;
        b=ajNlW7d+mrdxvXt9fJfEWlL6SEDkDq6FnQWVl1gy/yoNDyjkZ/xVQIydKZRDLfF9UA
         G8csr97M1EvJoanH9CaOKQJ0x/VmwZiXIZdcqYMly6iEc8Efz9BaA0FO88BpdQ9aXL2v
         con2O1R+I3OOqC7ere9FJ7N36gDrz4dLRtpTFoarLQY6c9PGj6lzLMYa7vg7WzwOuJUI
         8vulhNm3a0FyPSEeYGmB+HpCDeq1Z6FKbadMOek+WUK0X+IooFcelHXba0wmNb4fN2j1
         tBXvKUqin0f8hX3LRYJ7cZP97HVGlXTJ6WpkGv0/U7sAP0I9n0NwVkQOjqWQZ6YOvOsO
         BiTw==
X-Gm-Message-State: AOJu0YyoT1WDmuk+5OYqAAnh04pqnsQp45JydG0VBPGq4qjCYWYa54mp
	0e7FigDWe7UVNiff+hRNEABh3Rbkb1Q6/hkDdiVpwU1Rwzn17ELZ37783tY9fGGxehi/KpDh0s0
	=
X-Google-Smtp-Source: AGHT+IFW7C85nRhY8exNhc3hIB82Hp6Jo83BehuiwxeViGO2OPfqCk2CwZT7oQszZPAN6eoEbEZB3Q==
X-Received: by 2002:a2e:920f:0:b0:2d0:f934:999d with SMTP id k15-20020a2e920f000000b002d0f934999dmr1628832ljg.5.1707742325407;
        Mon, 12 Feb 2024 04:52:05 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX1MsEDLrSLU2Js6B3IzV2Tvc07XWA1blvk8dDcLcxRkHiCMwCs5xCcSx9383bqdpXwFx0L5xr8C8fp/8hSwR3iWkcBmPQBgS4h8blVw5UXpg==
Message-ID: <4c4ff91e-ce66-428f-a360-6ffc55659e79@suse.com>
Date: Mon, 12 Feb 2024 13:52:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/2] x86: cpu_data[] populating / consuming
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Kind of fallout from looking through the AP bringup acceleration series.

1: CPU: re-work populating of cpu_data[]
2: MCE: adjust x86_mc_get_cpu_info()

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 12:53:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 12:53:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679568.1057044 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZVoY-0003FW-8P; Mon, 12 Feb 2024 12:53:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679568.1057044; Mon, 12 Feb 2024 12:53:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZVoY-0003FP-4F; Mon, 12 Feb 2024 12:53:38 +0000
Received: by outflank-mailman (input) for mailman id 679568;
 Mon, 12 Feb 2024 12:53:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZVoX-0003FJ-5Q
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 12:53:37 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bb41ee4b-c9a5-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 13:53:34 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-33b960e3b8fso51397f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 04:53:35 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v9-20020a5d4b09000000b0033b843786e1sm1973101wrq.51.2024.02.12.04.53.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 04:53:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bb41ee4b-c9a5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707742414; x=1708347214; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=gnaXKYsqx4CnSa2eo1JbWiq89d/Jw0i23q5wGyWRgbc=;
        b=KeJf1p5pfk9h8ebY9OnKDYvFrjgj6zx7bho1RQnE30Y5gTP7CBbdhDGekv3EqaWU/6
         7p4OKrxnqRXoZG0ni8PgrA+hBQ1HzUbxKml+KDRJLHkVDiNYXTyCdUN/NByufEgWwuPp
         umF5BQ+2RS29+ON9HDZnX5SDsY4I6W1F35KJy+mU/4p+0Qu7c4GaRRN6n7SAItw8j3NS
         LTcciUdAWkwbzA/3Yw9WqE/MIJo3QnliW4elrrmPduOw3dQjbqzCu2HuAirOteklLXtb
         vbyJTjbatDyVCuizu08WlhemCk3A5HLXA/2VeH1U/Y3GSNg2PAd/cyfwGoP/IG+HVmNJ
         uM7A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707742414; x=1708347214;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=gnaXKYsqx4CnSa2eo1JbWiq89d/Jw0i23q5wGyWRgbc=;
        b=H/JxARpgCQWM8R0GOfao2mVLm004pFMoihf9QhIosVuwqmN2H/CPJY0FunjQmf4+9k
         Y6ujhnwPUosvrLGsSAfcahANxHa5GJRKSk8hw/a3Fpg/H0zfy0W98OETeeoiPg7uei+z
         Lqqn2rYVoR+Vqh7IdV0Zt1RlQroJ0z61jN13MP0uXjfFGPwoumJhSdJEPiTjXM9Qd44n
         6gBVhn7x9rQHw8UwM0Qdm2Apmj37y0nq/G6WY4lYOZe771P2CQd6XyTSRPw6T+wKkE14
         prCmoBkp1R0YYsfTD2cKD5l8a6UGC/hj3pBfJdVsrwzbO0spg1e0gvbRRF8N2Ft+VQvD
         39uQ==
X-Gm-Message-State: AOJu0YxQP0HM+sun7Lz9QFMrPAopdg9uTATcrhN09x9ROqd601lbW86w
	nyRjTs8P8XJZ9BFRKwxNN3WW+I2ayLbFvCRlS37S5sxDGbBKxR688CSdJS9fP65WLaiwkDecOP0
	=
X-Google-Smtp-Source: AGHT+IH9e8X3xpPzR5iMc18oxKPa1rgHP4+2e41uFq18jBhg4QQQjj0sEJ7GicSTGEPvifwctQ8Fmg==
X-Received: by 2002:a05:6000:12ce:b0:33b:14f8:7331 with SMTP id l14-20020a05600012ce00b0033b14f87331mr4359069wrx.4.1707742414367;
        Mon, 12 Feb 2024 04:53:34 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWs+UAElik26OQ7RClJIH1gUUq4z4NF/7jYzfGkiaLL0aUky03fnC69655YmBGr1vT5WBMIQboJp06qd7HPH2R+mCLTHrvGDD1WQUjCT6UZOw==
Message-ID: <f89d1c20-1967-451e-a6dc-510421604427@suse.com>
Date: Mon, 12 Feb 2024 13:53:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 1/2] x86/CPU: re-work populating of cpu_data[]
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <4c4ff91e-ce66-428f-a360-6ffc55659e79@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4c4ff91e-ce66-428f-a360-6ffc55659e79@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Forever since its introduction x86_mc_get_cpu_info() has been checking a
structure field against BAD_APICID which would never have been set to
that particular value. Arrange for this to actually be true ahead of
CPU detection having run, which then also includes pruning after CPU
removal.

For this,
- introduce a new helper function splitting reset logic out of
  identify_cpu(), and going slightly farther than the original code to
  achieve consistent results,
- introduce a new helper macro for both initializing and resetting
  various fields.

For boot_cpu_data this in particular means that cpuid_level will no
longer be set to the bogus value of -1. Even before removal of 32-bit
support we already assumed basic CPUID leaves to be available.

Furthermore make sure initialize_cpu_data() doesn't populate fields with
BSP's values which clearly aren't going to be applicable to APs.

Fixes: e16fc5ba591c ("x86: mce: Provide extended physical CPU info")
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -413,6 +413,19 @@ void __init early_cpu_init(bool verbose)
 	initialize_cpu_data(0);
 }
 
+void reset_cpuinfo(struct cpuinfo_x86 *c, bool keep_basic)
+{
+	if (!keep_basic) {
+		c->x86 = c->x86_vendor = 0;	/* So far unknown... */
+		c->x86_model = c->x86_mask = 0;	/* So far unknown... */
+		memset(&c->x86_capability, 0, sizeof(c->x86_capability));
+		memset(&c->x86_vendor_id, 0, sizeof(c->x86_vendor_id));
+		memset(&c->x86_model_id, 0, sizeof(c->x86_model_id));
+	}
+
+	CPU_DATA_INIT((*c));
+}
+
 static void generic_identify(struct cpuinfo_x86 *c)
 {
 	u32 eax, ebx, ecx, edx, tmp;
@@ -518,16 +531,7 @@ void identify_cpu(struct cpuinfo_x86 *c)
 {
 	int i;
 
-	c->x86_cache_size = -1;
-	c->x86_model = c->x86_mask = 0;	/* So far unknown... */
-	c->x86_model_id[0] = '\0';  /* Unset */
-	c->x86_max_cores = 1;
-	c->x86_num_siblings = 1;
-	c->x86_clflush_size = 0;
-	c->cpu_core_id = XEN_INVALID_CORE_ID;
-	c->compute_unit_id = INVALID_CUID;
-	memset(&c->x86_capability, 0, sizeof c->x86_capability);
-
+	reset_cpuinfo(c, false);
 	generic_identify(c);
 
 #ifdef NOISY_CAPS
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -40,6 +40,24 @@ struct cpuinfo_x86 {
     unsigned short x86_clflush_size;
 } __cacheline_aligned;
 
+#define CPU_DATA_INIT(what...)                     \
+        what.cpuid_level = 1,                      \
+        what.extended_cpuid_level = 0,             \
+        what.x86_cache_size = -1,                  \
+        what.x86_max_cores = 1,                    \
+        what.x86_num_siblings = 1,                 \
+        what.apicid = BAD_APICID,                  \
+        what.phys_proc_id = XEN_INVALID_SOCKET_ID, \
+        what.cpu_core_id = XEN_INVALID_CORE_ID,    \
+        what.compute_unit_id = INVALID_CUID
+
+/*
+ * @keep_basic set to true retains data firmly assumed to be symmetric
+ * across all CPUs.  With it set to false only CPU_DATA_INIT() will be
+ * invoked on the passed structure.
+ */
+void reset_cpuinfo(struct cpuinfo_x86 *c, bool keep_basic);
+
 extern struct cpuinfo_x86 boot_cpu_data;
 
 static inline bool cpu_has(const struct cpuinfo_x86 *info, unsigned int feat)
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -157,7 +157,7 @@ void *stack_start = cpu0_stack + STACK_S
 /* Used by the boot asm to stash the relocated multiboot info pointer. */
 unsigned int asmlinkage __initdata multiboot_ptr;
 
-struct cpuinfo_x86 __read_mostly boot_cpu_data = { 0, 0, 0, 0, -1 };
+struct cpuinfo_x86 __read_mostly boot_cpu_data = { CPU_DATA_INIT() };
 
 unsigned long __read_mostly mmu_cr4_features = XEN_MINIMAL_CR4;
 
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -61,7 +61,8 @@ unsigned int __read_mostly nr_sockets;
 cpumask_t **__read_mostly socket_cpumask;
 static cpumask_t *secondary_socket_cpumask;
 
-struct cpuinfo_x86 cpu_data[NR_CPUS];
+struct cpuinfo_x86 __read_mostly cpu_data[NR_CPUS] =
+    { [0 ... NR_CPUS - 1] = { CPU_DATA_INIT() } };
 
 u32 x86_cpu_to_apicid[NR_CPUS] __read_mostly =
 	{ [0 ... NR_CPUS-1] = BAD_APICID };
@@ -81,7 +82,12 @@ void *stack_base[NR_CPUS];
 
 void initialize_cpu_data(unsigned int cpu)
 {
-    cpu_data[cpu] = boot_cpu_data;
+    struct cpuinfo_x86 c = boot_cpu_data;
+
+    /* Must not partially clear the BSP's collected data. */
+    if ( cpu || system_state > SYS_STATE_smp_boot )
+        reset_cpuinfo(&c, true);
+    cpu_data[cpu] = c;
 }
 
 static bool smp_store_cpu_info(unsigned int id)
@@ -954,7 +960,6 @@ static void cleanup_cpu_root_pgt(unsigne
 static void cpu_smpboot_free(unsigned int cpu, bool remove)
 {
     unsigned int socket = cpu_to_socket(cpu);
-    struct cpuinfo_x86 *c = cpu_data;
 
     if ( cpumask_empty(socket_cpumask[socket]) )
     {
@@ -966,9 +971,7 @@ static void cpu_smpboot_free(unsigned in
 
     if ( remove )
     {
-        c[cpu].phys_proc_id = XEN_INVALID_SOCKET_ID;
-        c[cpu].cpu_core_id = XEN_INVALID_CORE_ID;
-        c[cpu].compute_unit_id = INVALID_CUID;
+        reset_cpuinfo(&cpu_data[cpu], false);
 
         FREE_CPUMASK_VAR(per_cpu(cpu_sibling_mask, cpu));
         FREE_CPUMASK_VAR(per_cpu(cpu_core_mask, cpu));



From xen-devel-bounces@lists.xenproject.org Mon Feb 12 12:54:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 12:54:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679570.1057053 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZVov-0003g9-EK; Mon, 12 Feb 2024 12:54:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679570.1057053; Mon, 12 Feb 2024 12:54:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZVov-0003g2-Ba; Mon, 12 Feb 2024 12:54:01 +0000
Received: by outflank-mailman (input) for mailman id 679570;
 Mon, 12 Feb 2024 12:53:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZVot-0003YI-Dh
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 12:53:59 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c9c20310-c9a5-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 13:53:58 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-410deab9c56so4679755e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 04:53:58 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v9-20020a5d4b09000000b0033b843786e1sm1973101wrq.51.2024.02.12.04.53.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 04:53:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c9c20310-c9a5-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707742438; x=1708347238; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=mbVyp1i6IDxX6RGOTJDic76W4aj3SGd/6f+hawIv8lU=;
        b=cw5W3Hbon3d3BuN1Uueb0qsFBRmqi37mbAM0O1VeADl7BxtAFGpjtc0/9tNYbPklNn
         Zbkt6FR5lqOFRpCtfwGpIByX2O0epYLG6Sx++DjIwy+b1BMJQogIcOW/rcL7WFHrrgPQ
         R9glZkiDDnbKp5VrS3HQFO7RULZ/u9NwY4EfLryJwYcBXtjKvg7BaqQi7uuUWm91rDLq
         4zfAvGx4lEK3nMfp+0zD9eL7ATR8psQbIBDt7jK3RpU9StBnTkTVxsnjLYeKjp1CY0PW
         beXfHHfwfViXR96e5BmQuQIBOcNZLQ+5l69foFPRmMSsxXkTI4T2KNQprgFKIWZSaVDi
         K4vw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707742438; x=1708347238;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=mbVyp1i6IDxX6RGOTJDic76W4aj3SGd/6f+hawIv8lU=;
        b=O0scs78ng9I42IiPlspWTlEE3QwU+3K3yINqzeRd3aZ0Y6x8BXNOH0iN/MhDUeBGMN
         wrhfZe9V16O7vK1hMkoYpHrPXeU9t4Fx9ayF40wzIUOGDZrP7EGAp5Wr6YXHzrmfHrjA
         WxdG+fcXAn4Jc267dQIMrItZu8i4yNEmcmFSMk2JfL/Q1qlbGZvyhaLgCCFq9HJQPbId
         Vq0WAb/lGZY0cmvFizrU7X3WxmiA9esdKzUD4aLqskbRrMH4pFPyoCeI8twpI2RxQLkF
         qnnb5dS4818dX2bZ9X/uXHsYpCnJ75srWuvSp8D+NwB/WoSFZnOam3v+hW5fLPEud4v1
         uMQw==
X-Gm-Message-State: AOJu0Yz1ohq+iaerwL8UPYeIz6+FTIAGNr3DWhRfV1epuCDUoQK5ajqj
	ePGDpI1cU1UldsKb+ghn/mwqRfH2EijVESRjD7zuEiG4Xo/iP+eyceWw9YgApLgHY3oj6nxzfTk
	=
X-Google-Smtp-Source: AGHT+IEemPcHCzneBQ2LN1NYHNpP9iPDwNmQ8qR5MtXty/HpN3B/rmFG+l3d+UAx6jbLaq0hJyG+AA==
X-Received: by 2002:a05:600c:3555:b0:410:61ce:f5c3 with SMTP id i21-20020a05600c355500b0041061cef5c3mr5453655wmq.19.1707742438328;
        Mon, 12 Feb 2024 04:53:58 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXD9gvlE/YKV/DqFTKJZ3cNvsnJambtTG3sUz1PYinFr9oBn8t7KnNa/npJJ2/PlH1JL2fswvNNo0RMPqJIC/YRrjUJjiFBzJbeDoyIzbp8qQ==
Message-ID: <53bb631d-134c-4670-8bc2-aa5b5c4b558a@suse.com>
Date: Mon, 12 Feb 2024 13:53:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 2/2] x86/MCE: adjust x86_mc_get_cpu_info()
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <4c4ff91e-ce66-428f-a360-6ffc55659e79@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4c4ff91e-ce66-428f-a360-6ffc55659e79@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Move logic independent of c->apicid's initialization status out of
the if/else, leveraging that cpu_data[] now doesn't start out zero-
initialized. Constify c and have it have an initializer.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -992,21 +992,14 @@ void x86_mc_get_cpu_info(unsigned cpu, u
                          unsigned *ncores, unsigned *ncores_active,
                          unsigned *nthreads)
 {
-    struct cpuinfo_x86 *c;
+    const struct cpuinfo_x86 *c = &cpu_data[cpu];
 
     *apicid = cpu_physical_id(cpu);
-    c = &cpu_data[cpu];
     if ( c->apicid == BAD_APICID )
     {
         *chipid = cpu;
         *coreid = 0;
         *threadid = 0;
-        if ( ncores != NULL )
-            *ncores = 1;
-        if ( ncores_active != NULL )
-            *ncores_active = 1;
-        if ( nthreads != NULL )
-            *nthreads = 1;
     }
     else
     {
@@ -1016,13 +1009,16 @@ void x86_mc_get_cpu_info(unsigned cpu, u
         else
             *coreid = 0;
         *threadid = c->apicid & ((1 << (c->x86_num_siblings - 1)) - 1);
-        if ( ncores != NULL )
-            *ncores = c->x86_max_cores;
-        if ( ncores_active != NULL )
-            *ncores_active = c->booted_cores;
-        if ( nthreads != NULL )
-            *nthreads = c->x86_num_siblings;
     }
+
+    if ( ncores )
+        *ncores = c->x86_max_cores ?: 1;
+
+    if ( ncores_active )
+        *ncores_active = c->booted_cores ?: 1;
+
+    if ( nthreads )
+        *nthreads = c->x86_num_siblings ?: 1;
 }
 
 #define INTPOSE_NENT 50



From xen-devel-bounces@lists.xenproject.org Mon Feb 12 14:09:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 14:09:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679583.1057064 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZWzZ-0004zK-Q0; Mon, 12 Feb 2024 14:09:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679583.1057064; Mon, 12 Feb 2024 14:09:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZWzZ-0004zD-MW; Mon, 12 Feb 2024 14:09:05 +0000
Received: by outflank-mailman (input) for mailman id 679583;
 Mon, 12 Feb 2024 14:09:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZWzY-0004yj-Q6
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 14:09:04 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 466e0dad-c9b0-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 15:09:03 +0100 (CET)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-33b66883de9so2280839f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 06:09:03 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m10-20020adff38a000000b0033b0d2ba3a1sm6909668wro.63.2024.02.12.06.09.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 06:09:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 466e0dad-c9b0-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707746942; x=1708351742; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=P5ICFZRzFXL4KVkiHMEHxd8Eh85QmsegTMwijrGZIIA=;
        b=HNDkO1VyXgaCj/WyrKZLBBTMkA1H+7JhqrUBY+WxJ/5KnH3qC5cPhy2Sd0q0SIDQ+9
         LOnJweooSIFP7abuNhCmtE+zbSt1/X4R3shjToc0PdGryFGgc/iJi4yBkyV0wkh37slR
         ICsyg1WrhLCMGl7TZdDgz9ay0wm99U/G804eVUbW6getkOe/zPH6VcSXYu5I9YsInD6f
         HHf2NwOD8aFY7yf5iOEiN9Hmi2TXaPBGGyHQDZVApfqD55i41h3F1eav1Q9p4mJCOiUp
         2PhwPwVRRB3Iria1LB00Ez5hnLBp7AMmXLBqzhTKDuvqd/a0fY1mQGheeMKWA0vjm+Ux
         D2CQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707746942; x=1708351742;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=P5ICFZRzFXL4KVkiHMEHxd8Eh85QmsegTMwijrGZIIA=;
        b=lEaRlZWBNuHwmMonBl/JMPNYnzwuZ0G4Bvp8HZEXhV8+iLg9pSRPpSwE3TmOaslhtN
         B1pdDWBNGXTlwSVwvl1h23zN+ZCzWnbjeAXwjtOdkb/Z/ZZw9aWoiIbpWRuKzxGBuvfR
         nHaJ769MXwC2wavKDs2lSbS6jDzaV8pFFRpAPjJUjCkgjPxcuSz7uYEpcfs7up08MZfo
         h1/H3y6hLICRx5r57ba4USOM4/X+RaLeNFYcjUYdC/Y446dzI5sOSEhzhn/g0/GYPnzV
         WCFn2nn6+Wa5y4218ixkQDOb/B1erMA/RKebzQ5JE507idhiZpZYBEk153R3ikhcxl8S
         GdMg==
X-Forwarded-Encrypted: i=1; AJvYcCVi3sci/oNZ9Ly8pPeDOHlfKznMcRiFfd/4AsFV53dSwVKKRkI4viBinWO7uYO7rA3YuzJ4zJYqUKoBfqCTnitsDbwRsnmgSLcB+MNDv24=
X-Gm-Message-State: AOJu0Yyn92H/uhEh2Afwi+6WT3xUxkVVtGo81TaxmTVC2DKcAf43vqbv
	HwPctKcJsh+0LuW1qz4XPqX36yUpGQtasl1vE77q4DRKJb9+VsDpi35NNqEXdA==
X-Google-Smtp-Source: AGHT+IGzT++F0IcboumE1r9sKb6bUjhKYE7kHDwfPOGsSNMZxDm128KvJLTsJ8cJ+BPzk7WFXBbNvQ==
X-Received: by 2002:a05:6000:10cd:b0:33b:81cc:bd15 with SMTP id b13-20020a05600010cd00b0033b81ccbd15mr1899634wrx.37.1707746942359;
        Mon, 12 Feb 2024 06:09:02 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX6btzaimQNkFVtdPOXU1hh1fqtVloBK8mcR2K1lEHOh8EcFpZchVpaQqflFp7xlhEGamD9dpEdYbwCSBkpcBDpkCCsGaIoX+5QLvXN7pVqMcyy6jGyGYmxFxFXw0GwUgI4+hviMhDJY2Dxyd05XOEQNyoF2XpvrHz+5znEM9zeTeCAZ/eyEroy893H
Message-ID: <09cf678e-bdbd-47a9-8c9d-533e7bee9450@suse.com>
Date: Mon, 12 Feb 2024 15:09:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/vmx: add support for virtualize SPEC_CTRL
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240209114045.97005-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240209114045.97005-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 09.02.2024 12:40, Roger Pau Monne wrote:
> @@ -1378,6 +1379,10 @@ static int construct_vmcs(struct vcpu *v)
>          rc = vmx_add_msr(v, MSR_PRED_CMD, PRED_CMD_IBPB,
>                           VMX_MSR_HOST);
>  
> +    /* Set any bits we don't allow toggling in the mask field. */
> +    if ( cpu_has_vmx_virt_spec_ctrl && v->arch.msrs->spec_ctrl.raw )
> +        __vmwrite(SPEC_CTRL_MASK, v->arch.msrs->spec_ctrl.raw);

The right side of the conditional isn't strictly necessary here, is it?
Might it be better to omit it, for clarity?

> --- a/xen/arch/x86/hvm/vmx/vmx.c
> +++ b/xen/arch/x86/hvm/vmx/vmx.c
> @@ -823,18 +823,29 @@ static void cf_check vmx_cpuid_policy_changed(struct vcpu *v)
>      {
>          vmx_clear_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
>  
> -        rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
> -        if ( rc )
> -            goto out;
> +        if ( !cpu_has_vmx_virt_spec_ctrl )
> +        {
> +            rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
> +            if ( rc )
> +                goto out;
> +        }
>      }
>      else
>      {
>          vmx_set_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
>  
> -        rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
> -        if ( rc && rc != -ESRCH )
> -            goto out;
> -        rc = 0; /* Tolerate -ESRCH */
> +        /*
> +         * NB: there's no need to clear the virtualize SPEC_CTRL control, as
> +         * the MSR intercept takes precedence.  The SPEC_CTRL shadow VMCS field
> +         * is also not loaded on guest entry/exit if the intercept is set.
> +         */

It wasn't so much the shadow field than the mask one that I was concerned
might be used in some way. The shadow one clearly is used only during
guest RDMSR/WRMSR processing. To not focus on "shadow", maybe simple say
"The SPEC_CTRL shadow VMCS fields are also not ..."?

> +        if ( !cpu_has_vmx_virt_spec_ctrl )
> +        {
> +            rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
> +            if ( rc && rc != -ESRCH )
> +                goto out;
> +            rc = 0; /* Tolerate -ESRCH */
> +        }
>      }
>  
>      /* MSR_PRED_CMD is safe to pass through if the guest knows about it. */
> @@ -2629,6 +2640,9 @@ static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
>      switch ( reg )
>      {
>      case MSR_SPEC_CTRL:
> +        if ( cpu_has_vmx_virt_spec_ctrl )
> +            /* Requires remote VMCS loaded - fetched below. */

I could see what "fetch" refers to here, but ...

> +            break;
>          rc = vmx_read_guest_msr(v, reg, &val);
>          if ( rc )
>          {
> @@ -2652,6 +2666,11 @@ static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
>      vmx_vmcs_enter(v);
>      switch ( reg )
>      {
> +    case MSR_SPEC_CTRL:
> +        ASSERT(cpu_has_vmx_virt_spec_ctrl);
> +        __vmread(SPEC_CTRL_SHADOW, &val);
> +        break;
> +
>      case MSR_IA32_BNDCFGS:
>          __vmread(GUEST_BNDCFGS, &val);
>          break;
> @@ -2678,6 +2697,9 @@ static void cf_check vmx_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
>      switch ( reg )
>      {
>      case MSR_SPEC_CTRL:
> +        if ( cpu_has_vmx_virt_spec_ctrl )
> +            /* Requires remote VMCS loaded - fetched below. */

... since you also use the word here, I'm not sure it's really
the VMREAD up there.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 14:15:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 14:15:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679588.1057073 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZX5j-0006eF-IE; Mon, 12 Feb 2024 14:15:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679588.1057073; Mon, 12 Feb 2024 14:15:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZX5j-0006e8-FZ; Mon, 12 Feb 2024 14:15:27 +0000
Received: by outflank-mailman (input) for mailman id 679588;
 Mon, 12 Feb 2024 14:15:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZX5i-0006e2-A8
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 14:15:26 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2a4af423-c9b1-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 15:15:25 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-410cb9315f6so7878815e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 06:15:25 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i5-20020a05600c290500b0040ef95e1c78sm8730015wmd.3.2024.02.12.06.15.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 06:15:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a4af423-c9b1-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707747325; x=1708352125; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=R+KMW4T5CXrIUyiSmI3rIXQ/wdpb/3grpC8b6xH+lVQ=;
        b=IcFJVmj0yGCLp0jhkCK/VngEximrav1vfpCTv+8eL2miRC2qEVoq4Vd26wTGmlXJly
         by+mQlHrXPY5zXaSlw/TEbQrSaFaYl77IIvbWwKrJGvE+9qbqabgwxVWv9jUMevcR1x4
         doec1t0i+ZEC5K+zGlNX3f/AALrzyyw/7ST5YzfoQXz2vLJfqFAPeRqYQm/wzzeGs9d7
         dMV/q+U2rJC550h44xebMmJt9aX49/4fCgmbHlh/k3+OoLz3EKZm+UCpcGapSdSFt1Az
         MAI+ARDmLElUBt06SNqDa4h7KVpcyrk15+QQSmhu/fywx2baIJ2wPy+Cp22q8PUwrDjg
         dVUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707747325; x=1708352125;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=R+KMW4T5CXrIUyiSmI3rIXQ/wdpb/3grpC8b6xH+lVQ=;
        b=iuhbkbrFJyqn2O0OwCuRzd4UdDa3/2bXvw9y75gA5ysUc/hdpiYRwQL6jvo2Y2UPcg
         EhCKVUNtrI9wFZZ6ol/J8uebDWkR0RyB0CdM07usWq0bL8BOSJPSxgOPFKkOCoYacRSL
         ZDtKFoB+OollPuHKxecVdbuitphOxa3hwgltqOO79m2uOq1dIkOT5bqPsfELYxqv0zCh
         /8jTbLccIw1SQwhHFt5qQIApb9FU7OsyEcXCSc7OOxllN2R7GHoRCAStlWMxG7HdzYeb
         DT0yADhsmKYn1iedBLJ4Hg7i9JprfntfutODmmpg6Xh2Gi6nlM16UKkyr+UOYZuZfzzR
         NhAw==
X-Forwarded-Encrypted: i=1; AJvYcCWPJVFK1FEMhmWrxyQisfYQQ6HXDbfVhXvRD/8pwtaRy3xGpnETrJnhGOhFummW8lSHEOJP1i3PVfIOfomSDT6nB7G38VbaODwZaeRnDQQ=
X-Gm-Message-State: AOJu0YyA6usNLzVxjHxJzOguvtfZXVgBWAdR8KdsepcWw8zQl9CDexCx
	qDRgqs94KFYiqGQhmajippODKjh6G4tCR8fv4s5jC+EAhy81pqNsYy5sVEixlQ==
X-Google-Smtp-Source: AGHT+IGFyUVQOyoozkrsjSw7+TkFJF7kEZL4SsDrnBm2NJ4BTiN5CrPoXlBjhDInwNXhvoNz6A5P8g==
X-Received: by 2002:a05:600c:4e52:b0:410:a0ef:75f9 with SMTP id e18-20020a05600c4e5200b00410a0ef75f9mr4668923wmq.5.1707747324755;
        Mon, 12 Feb 2024 06:15:24 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXX1bsjBSwgnrPb1mo+z8QhHhhYoB5PLGHNRVQTzfq6VlOcwCKvBl/AwKeZXJ2gwVgmuURp9EO91xDjBVO5GXhGyVtogsmVvRS3/a3Dbm+2ZUjrSBiEnn+v8T9QIrpVCjiE8gEv7qNsFtaE00p1p1GniZTZU0/NZe9w+hBHX/blaUypc/4KpryfqRicVfBSE3n3CfGgNo9v/ypWjJN4drlIs3QiBmP18YNSHcTG5yszzGR+NDyag6f3YhMIatZOUe1pjrP000cDqS45y6JWF4QaSg9i+zSWBlxf8A095rLNq07LMdaSluPkPLgUdxVs/BOiBvKjIpK7FoGlyUjTqzF8n+k4wUPTBwW0tg==
Message-ID: <f8534623-39bf-4c4a-9236-ed5bd3d655c2@suse.com>
Date: Mon, 12 Feb 2024 15:15:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] xen/lib: introduce generic find next bit operations
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 xen-devel@lists.xenproject.org
References: <fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 09.02.2024 18:58, Oleksii Kurochko wrote:
> find-next-bit.c is common for Arm64, PPC and RISCV64,
> so it is moved to xen/lib.
> 
> PPC has been transitioned to generic functions from find-next-bit.c
> since it now shares the same implementation as the PPC-specific code.
> 
> The MISRA exclude list has been updated to verify
> lib/find-next-bit.c instead of Arm's find_next_bit.c,
> as Arm's find_next_bit.c has been relocated to xen/lib/.
> 
> Despite CONFIG_GENERIC_FIND_FIRST_BIT not currently being used in
> Xen and being removed from the Linux kernel [1], it could theoretically
> prove useful for Xen. This is because the Linux kernel transitioned
> Arm to the generic version of find_first_bit() and
> find_first_zero_bit() due to improvements [1] in both performance
> and .text size.
> It would be prudent to investigate whether [1] is applicable to Xen
> as well and, if so, implement the necessary changes in a separate patch.
> 
> [1] https://lore.kernel.org/linux-arch/20211005054059.475634-5-yury.norov@gmail.com/
> 
> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

A revision log would have been nice, though.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 14:19:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 14:19:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679591.1057084 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZX9v-0007Ui-36; Mon, 12 Feb 2024 14:19:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679591.1057084; Mon, 12 Feb 2024 14:19:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZX9v-0007Ub-0R; Mon, 12 Feb 2024 14:19:47 +0000
Received: by outflank-mailman (input) for mailman id 679591;
 Mon, 12 Feb 2024 14:19:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZX9t-0007UV-U2
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 14:19:45 +0000
Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com
 [2a00:1450:4864:20::436])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c4d7957f-c9b1-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 15:19:44 +0100 (CET)
Received: by mail-wr1-x436.google.com with SMTP id
 ffacd0b85a97d-33aeb088324so2118998f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 06:19:44 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ch16-20020a5d5d10000000b0033b58cbf3ebsm6953562wrb.26.2024.02.12.06.19.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 06:19:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c4d7957f-c9b1-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707747584; x=1708352384; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=JulKx+SVt0hyNzIkagwD5rYYirJi6m5LCwZwtfnX1AQ=;
        b=UXhindaLqirT+xi/Pnh2KSMbmo7+IUNo38H39SN4PdHwhTXiZsExUhv/7zG0q12Ha4
         rQG/qhNo4vBCmJapcptVONz6S875I4dZWB/r1s9FOfdWtxWnwAEi9wjaRqC/jLbuE6Vz
         7ovyxYtbXzsKEPUoO1KOOrYe6oD6j9I6o5UbhLPcPt2VefBrpBKNdW1iAP0TybRWMphn
         oyMM7/ePhzCnN8DWXDKnBZF0F6g+TEYurVy3qMg1JWmPgdNyK8rWiPex1mECnN9+1wae
         4xuibFXVasS+SPcK0ZlOHZONmFLXJuah4kq98x+m2SHe05EQkxZeHwouKYLqG8Ee4vnu
         /7fA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707747584; x=1708352384;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JulKx+SVt0hyNzIkagwD5rYYirJi6m5LCwZwtfnX1AQ=;
        b=vE9vEvKsSEm29UAWwNzf7ohqBh828xEajrgIPvHzg6b/mq9USKub1IZ2eXGqvE/S4+
         2bDpsB2Vaj5kTBrnt+OrpTWVGFAHl33ZslpVZeQsMwqcws6HdWHbrOafNzmYd8gxdIcv
         3L9V2Ni+0QjFYAYy0w+zej1MKh37SBpWgJoSnfqTjZXpdBEH7EFKUwZ592bbW7dp+ccW
         oXcYo+tWbGDaruUVSsQs9LTaMV3jNlzuNi/dG6epuPL/xooe2SiAPKHwD72gf/bAnN8z
         I+N0aO72kuKOpQxUP/uwPjpN7IaOYhcAwj9SnJ985dmmqp+SKJFUfdMPpy39f+t9mAZ1
         xl7g==
X-Gm-Message-State: AOJu0YxQnxSDTautVwqr77oyC43UtxWMLNgTKtquskkfYGSTL4T0tJIg
	JRjs3tCiRWumfZdVzfa1TmhIAl/2oMB2ehMdXRXKr7ZCBj2NrbSuvmE1qnfaAQ==
X-Google-Smtp-Source: AGHT+IEODM4Kfye65a1Z1bbCV33+qT+Zr0+2Dpo5GuSlAqGhr8OHowtoUCjbHZBo8E7wA/6ox9QSTA==
X-Received: by 2002:adf:fd10:0:b0:33b:3c9b:683e with SMTP id e16-20020adffd10000000b0033b3c9b683emr5264964wrr.64.1707747583834;
        Mon, 12 Feb 2024 06:19:43 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVfmGKlIKpAF5erHXbhdwRgDRXH1aL4ShKCnBQVK1Wph4wkO9kzzRM/OcnD3zvRKLNq+UU6UhuiQLMRFrD/VNxYE/2C6ouihnG6/36WedtCgsAeUGuxoDHSBHX/hjxHMwEJelG87t0qTEjKL+1bPELd0dJ2JM5CcmcJIq9p7RtGLDJOWiLsVADfWQIzcNIaoEUO3JTYoNqYt4rbs5E=
Message-ID: <94a9353f-1dd0-4957-af2c-281f9b3bbe7c@suse.com>
Date: Mon, 12 Feb 2024 15:19:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 5/7] xen/asm-generic: introduce generic device.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
 <4ecb5e5f4a1dbf585ac7a9ebe40b5fd8e4709182.1707499278.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4ecb5e5f4a1dbf585ac7a9ebe40b5fd8e4709182.1707499278.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 09.02.2024 19:00, Oleksii Kurochko wrote:
> --- /dev/null
> +++ b/xen/include/asm-generic/device.h
> @@ -0,0 +1,149 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef __ASM_GENERIC_DEVICE_H__
> +#define __ASM_GENERIC_DEVICE_H__
> +
> +#include <xen/stdbool.h>
> +
> +enum device_type
> +{
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +    DEV_DT,
> +#endif
> +    DEV_PCI
> +};
> +
> +enum device_class
> +{
> +    DEVICE_SERIAL,
> +    DEVICE_IOMMU,
> +    DEVICE_INTERRUPT_CONTROLLER,
> +    DEVICE_PCI_HOSTBRIDGE,
> +    /* Use for error */
> +    DEVICE_UNKNOWN,
> +};
> +
> +struct dev_archdata {
> +#ifdef CONFIG_HAS_PASSTHROUGH
> +    void *iommu;    /* IOMMU private data */
> +#endif
> +};
> +
> +/* struct device - The basic device structure */
> +struct device
> +{
> +    enum device_type type;
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +    struct dt_device_node *of_node; /* Used by drivers imported from Linux */
> +#endif
> +    struct dev_archdata archdata;
> +#ifdef CONFIG_HAS_PASSTHROUGH
> +    struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
> +#endif
> +};
> +
> +typedef struct device device_t;
> +
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +
> +#include <xen/device_tree.h>
> +
> +#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
> +
> +/**
> + *  device_init - Initialize a device
> + *  @dev: device to initialize
> + *  @class: class of the device (serial, network...)
> + *  @data: specific data for initializing the device
> + *
> + *  Return 0 on success.
> + */
> +int device_init(struct dt_device_node *dev, enum device_class class,
> +                const void *data);
> +
> +/**
> + * device_get_type - Get the type of the device
> + * @dev: device to match
> + *
> + * Return the device type on success or DEVICE_ANY on failure
> + */
> +enum device_class device_get_class(const struct dt_device_node *dev);
> +
> +#define DT_DEVICE_START(name_, namestr_, class_)            \

I don't think the trailing underscores are needed or helpful here
(or in the ACPI counterpart), ...

> +static const struct device_desc __dev_desc_##name_ __used   \
> +__section(".dev.info") = {                                  \
> +    .name = namestr_,                                       \
> +    .class = class_,

... seeing this all it would have taken was to avoid the two words
"name" and "class" (by e.g. using "ident" and "cls").

Nevertheless:
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 14:22:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 14:22:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679595.1057094 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXCu-0000UV-GP; Mon, 12 Feb 2024 14:22:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679595.1057094; Mon, 12 Feb 2024 14:22:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXCu-0000UO-DK; Mon, 12 Feb 2024 14:22:52 +0000
Received: by outflank-mailman (input) for mailman id 679595;
 Mon, 12 Feb 2024 14:22:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZXCs-0000UH-TP
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 14:22:50 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 33573286-c9b2-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 15:22:50 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-411a595de0cso1582815e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 06:22:49 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs25-20020a056000071900b0033b45bdb2a1sm7039751wrb.4.2024.02.12.06.22.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 06:22:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 33573286-c9b2-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707747769; x=1708352569; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jKP3l0e1bgYYbVAbkkRW0/BNXPryqSD8TvJ/QIxwWN8=;
        b=AzVIElOCQt+uS1z5g/28lR1CItZ5RV7Q1o3xFuunvh5wzQemgEfs0zZkgnTjpMUFPQ
         bwOQKZ08I/Jw2GnfLcyBAxFH0oqRtVwxiNZf/jTxxLtEjhNHjbBo2A+VvnI/3RGdktjs
         SIdS6uvDaPiKbiXqDMCkqR54YH0qCWdPZYuwDX5mMuqQgdPtllFmqo+QD9G36t28qOon
         F46MXhebvrqhcrThK5ro3NM+t2XM1A2FQB0AOYb3yhKpRxtG7/qX5C/JmnBoHE3GGFHl
         dhsCBzm43oirMoWN+vJDLUPUHrdMhqW52iuzhpugmLmfPBgB9CsGGvlZf45iaPkffM1H
         7UfQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707747769; x=1708352569;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jKP3l0e1bgYYbVAbkkRW0/BNXPryqSD8TvJ/QIxwWN8=;
        b=jNB+KVl+RgIly/HBhOOUW8Wl4VrzzW/gWAd1JVepVhya3wyMNnzovDrossgl6tbn6+
         cbWY/8cGxYZ6/fhOghU3aaYeW5Nr0C7nUdihA0T3jhBs6tMd1UlAfuAIk1xlMTF+d03g
         Ut+Kl1NLGDIdx3G8HHrMWt3QNW1rW8RHlXVQyZjHmdMPHdaZETr75EKGZbdqz0U/xtpf
         0PgVz6tQmiRbyGflCTBfPBMnpH0Q/oZrypMhwJ3je+OFPddAs8XOtOWPSi2VoSCScjUw
         0NqWhP9S2JAnNafmrBp2O9M44zqNDHBSiOWTZHl1NlMghKTQ3b1DdG53pWxnC00MDRpA
         HJzg==
X-Gm-Message-State: AOJu0YzAJoO70+8AEkCJ9lCkBMjk/CS/Spt1Ld/FNL4F8a4+NdhsoW9Y
	b+GZYyKc03Fjv90izoA7rsyqq3JiVS8G2lPr2mlcPFmSdGIm/KIb0Id2xjQJyw==
X-Google-Smtp-Source: AGHT+IGXr29XZekbJIVUk/FByaeh5PSMdov+TjQC9v03jhyFNIF2Ma23lcCBpXi7hiDh3/SzTZTm4Q==
X-Received: by 2002:a5d:5585:0:b0:33a:edaf:13ec with SMTP id i5-20020a5d5585000000b0033aedaf13ecmr5158701wrv.14.1707747769344;
        Mon, 12 Feb 2024 06:22:49 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUbugZf3s+Vgg3ojPArSBhLZ7ax9qKKQ0Lj5MUQdtkAnS+sABtO0z26GlRFDKBiGzSBMV7uMQt+IjNVdvQ6aVwDI1l/PZiNBP0TT5A3Gpw1g1mH+l3j4/MFhkqKz2VlIiO/N82xGqTmXvBqfgaHN6oOz7Iiq03P4tHKWh0OSN0FPnKexqE=
Message-ID: <59d7106d-21e8-4e5d-aa8d-bc2fc859a004@suse.com>
Date: Mon, 12 Feb 2024 15:22:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/2] x86/p2m: aid the compiler in folding p2m_is_...()
Content-Language: en-US
To: George Dunlap <dunlapg@umich.edu>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@cloud.com>
References: <5d6c927e-7d7c-5754-e7eb-65d1e70f6222@suse.com>
 <7cce89f4-962e-bfbe-7d30-18fea7515bed@suse.com>
 <CAFLBxZZLJMWpf1fCNN4dhoDpYpW6O=V_C==-sCAZy6t4Df_yBw@mail.gmail.com>
 <47b5a1be-280c-4e8e-a5c5-6df7da657539@suse.com>
 <CAFLBxZY4hR6mxL_Zu+AYaNFsg528zmnL45K6tcE=_Lq7s5p=0w@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAFLBxZY4hR6mxL_Zu+AYaNFsg528zmnL45K6tcE=_Lq7s5p=0w@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07.02.2024 04:07, George Dunlap wrote:
> On Thu, Feb 1, 2024 at 10:15 PM Jan Beulich <jbeulich@suse.com> wrote:
> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -379,7 +379,7 @@ struct page_info *p2m_get_page_from_gfn(
>              return page;
> =20
>          /* Error path: not a suitable GFN at all */
> -        if ( !p2m_is_ram(*t) && !p2m_is_paging(*t) && !p2m_is_pod(*t) &&
> +        if ( !(p2m_is_ram(*t) | p2m_is_paging(*t) | p2m_is_pod(*t)) &&
>               !mem_sharing_is_fork(p2m->domain) )
>              return NULL;
>      }
> ```
> 
> Note the "=20" at the beginning of the empty line.  Why `patch` handles it
> but `git am` doesn't, who knows.
> 
> 
>> I'm also not aware of there
>> being a requirement that patches I send via email need to be
>> "git am"-able (unlike in xsa.git, where I edit patches enough to be
>> suitable for that), nor am I aware how I would convince my email
>> client and/or server to omit whatever git doesn't like or to add
>> whatever git is missing.
>>
>> Bottom line - your response would be actionable by me only in so far
>> as I could switch to using "git send-email". Which I'm afraid I'm not
>> going to do unless left with no other choice. The way I've been
>> sending patches has worked well for over 20 years, and for different
>> projects. (I'm aware Andrew has some special "Jan" command to apply
>> patches I send, but I don't know any specifics.)
>>
> 
> In the general case, I'm not going to review a patch without being able to
> see it in context; and it's not reasonable to expect reviewers to have
> specific contributor-specific scripts for doing so.  If we run into this
> issue in the future, and you want my review, you may have to post a git
> tree somewhere, or attach the patch as an attachment or something.  (Or you
> can try to figure out why `git am` isn't working and try to upstream a fix.)
> 
> That said, in this case, context isn't really necessary to understand the
> change, so it won't be necessary.
> 
> The logic of the change is obviously correct; but it definitely reduces the
> readability.  I kind of feel like whether this sort of optimization is
> worth the benefits is more a general x86 maintainer policy decision.  Maybe
> we can talk about it at the next maintainer's meeting I'll be at?

While you weren't able to be there, I brought this up nevertheless, and
both Andrew and Roger agreed with you. Therefore I'll drop this patch
and adjust the other one.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 14:26:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 14:26:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679599.1057104 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXGj-00015W-3l; Mon, 12 Feb 2024 14:26:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679599.1057104; Mon, 12 Feb 2024 14:26:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXGi-00015P-TU; Mon, 12 Feb 2024 14:26:48 +0000
Received: by outflank-mailman (input) for mailman id 679599;
 Mon, 12 Feb 2024 14:26:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZXGi-00015E-BD; Mon, 12 Feb 2024 14:26:48 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZXGi-0006QE-5r; Mon, 12 Feb 2024 14:26:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZXGh-0002KA-N2; Mon, 12 Feb 2024 14:26:47 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZXGh-00020j-Ma; Mon, 12 Feb 2024 14:26:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=i7VYN4melc+kjGorpZv+OeJa3aHcSDT9BBIZiw4XdfE=; b=GIGWubMQe0KCqYHaLGY2Wc1ySD
	TCZ4hcTTnm4tnIAHicmkwgdSS9nSwftGtG9p6mgz0emIcTuOaYlySRspMMDLphc8Pfml+qXHe+xUf
	kJKNv0Z5yho0jjqzR9a09UawN1WZgYhzTDTOGxepHG4hmeu+K6SkRB7pAE1PT3DtdUSU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184649-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184649: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=841c35169323cd833294798e58b9bf63fa4fa1de
X-Osstest-Versions-That:
    linux=2766f59ca44e517a1d6226979c784b026f0e89c2
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 12 Feb 2024 14:26:47 +0000

flight 184649 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184649/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184647
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184647
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184647
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184647
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184647
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184647
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184647
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184647
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                841c35169323cd833294798e58b9bf63fa4fa1de
baseline version:
 linux                2766f59ca44e517a1d6226979c784b026f0e89c2

Last test of basis   184647  2024-02-11 20:13:59 Z    0 days
Testing same since   184649  2024-02-12 03:12:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Linus Torvalds <torvalds@linux-foundation.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   2766f59ca44e..841c35169323  841c35169323cd833294798e58b9bf63fa4fa1de -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 14:35:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 14:35:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679607.1057115 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXOh-00035o-Tx; Mon, 12 Feb 2024 14:35:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679607.1057115; Mon, 12 Feb 2024 14:35:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXOh-00035h-Pk; Mon, 12 Feb 2024 14:35:03 +0000
Received: by outflank-mailman (input) for mailman id 679607;
 Mon, 12 Feb 2024 14:35:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZXOh-00035b-Dn
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 14:35:03 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e755cb15-c9b3-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 15:35:01 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-51168addef1so6051704e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 06:35:01 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g19-20020a05600c4ed300b004103e15441dsm9030017wmq.6.2024.02.12.06.35.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 06:35:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e755cb15-c9b3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707748501; x=1708353301; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6gEYb1jxsMWXtKwF6u0NDJaCWqocr8nDIy4nudqDl5o=;
        b=QCwWLs3SYK1hacYmpOTRmKg3ve5s4PPbWTakRIldQFSsmhXPgJksaC6y1rZuM/Yu1n
         WV25Byosr8u4AVGCayWeHZBxjnQRaJXQfh43JRGS119AgPixZbPpb5GHoNSkdapHBlns
         sT3hoN8Cg+itMDpg1pXem3dDpT51zL4RlvV5+GeiRYti2yhzOGUibDjSmjZhUE83DcSo
         caOeUtFBceLJsurnA+R82PmdFLNUm2WVPc2L8HjWs3kyIM7ASEv8CyJOaTLAeux4ORNv
         Om2C91/F2yp9FiJQk0it7QfA+zIMEoThibXkVkg2HR5kQole7JTb5VBEIOWmkj3jQFS7
         H+DA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707748501; x=1708353301;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6gEYb1jxsMWXtKwF6u0NDJaCWqocr8nDIy4nudqDl5o=;
        b=ju2yKGQQUsI5AMaRg5cTOsfQsKO/QXhAJdX6G6q7UORXyc+PvWJqquHzwC1xBCvKq1
         aXifo788SbTeAINVPKZ/95954TxLFt36yHPIrw030QArD8tJWvo1a7Xz3x2hrcz1dbxD
         EK1o9931PGXi1LjvgcN3zQ6+2czfKZ5Yl2cf8QFCkzMuDMTOoLm3h8OLEmdUTKOYhUtQ
         KuJ98KkzGGMurDp4odiYm1S3huiNgqJMgtD3Vf+nHdqMzhL/ECABuZY9ulESrbCHLDnK
         kElWhjQvbYj7TFSgS8x6bJRkrmjT4njxNMT6JFpkn3tILjU7o9avtKL9KGgA4UUIB4qW
         8tSQ==
X-Gm-Message-State: AOJu0YzRdyvc7+e/ViERL84K8CZc6kmTriR27fNR9hYOeTCtdJpsJsp3
	BEpP9hBX8vAN7JKhJeY9WWshtL/kWVCOraw5QJ4N/rLPgIz6ZW6+vDeV4Dr2Fw==
X-Google-Smtp-Source: AGHT+IFk6QGSFeMFijkPxDn5Z+1qmzUQp12IRBkVxhoA/KxuR8U066Oq17YtzYPGOQznQ8bCYa9Qfw==
X-Received: by 2002:a05:6512:2806:b0:511:5482:398c with SMTP id cf6-20020a056512280600b005115482398cmr6114520lfb.17.1707748500923;
        Mon, 12 Feb 2024 06:35:00 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU7ImEJEN58yuvB/XCHbfcR3WQbJ79EmNJdB+NKIqVwCup7XbgbvYuXLlPM0ZkG3dbtq/XM4zMH6t7IEzBoQnjZ1lqa7cUpm20XXGXtD/QYZYNEMJHChrH/GUCbacUs6y7RUfNMNBhQhg==
Message-ID: <ee4b4d25-e6db-4265-adc4-4080784495cf@suse.com>
Date: Mon, 12 Feb 2024 15:34:59 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/2] iommu/x86: introduce a generic IVMD/RMRR range
 validity helper
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-2-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240207153417.89975-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07.02.2024 16:34, Roger Pau Monne wrote:
> IVMD and RMRR ranges are functionally equivalent, and as so could use the same
> validity checker.
> 
> Move the IVMD to x86 common IOMMU code and adjust the function to take a pair
> of [start, end] mfn parameters.
> 
> So far only the AMD-Vi side is adjusted to use the newly introduced helper, the
> VT-d side will be adjusted in a further change.
> 
> No functional change intended.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with one minor remark:

> --- a/xen/drivers/passthrough/x86/iommu.c
> +++ b/xen/drivers/passthrough/x86/iommu.c
> @@ -792,6 +792,52 @@ static int __init cf_check adjust_irq_affinities(void)
>  }
>  __initcall(adjust_irq_affinities);
>  
> +bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
> +{
> +    mfn_t addr;
> +
> +    if ( e820_all_mapped(mfn_to_maddr(start), mfn_to_maddr(end) + PAGE_SIZE,
> +                         E820_RESERVED) )
> +        return true;
> +
> +    printk(XENLOG_WARNING "%s: [%#" PRI_mfn " ,%#" PRI_mfn
> +           "] is not (entirely) in reserved memory\n",

Would you mind if I re-flowed this to

    printk(XENLOG_WARNING
           "%s: [%#" PRI_mfn " ,%#" PRI_mfn "] is not (entirely) in reserved memory\n",

while committing?

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 14:38:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 14:38:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679609.1057124 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXRm-0003d1-9t; Mon, 12 Feb 2024 14:38:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679609.1057124; Mon, 12 Feb 2024 14:38:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXRm-0003cu-7D; Mon, 12 Feb 2024 14:38:14 +0000
Received: by outflank-mailman (input) for mailman id 679609;
 Mon, 12 Feb 2024 14:38:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZXRl-0003cm-FC
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 14:38:13 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 56fb7638-c9b4-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 15:38:08 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-4114e0a2978so3319805e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 06:38:08 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 4-20020a05600c028400b00411a0477755sm715668wmk.9.2024.02.12.06.38.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 06:38:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56fb7638-c9b4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707748688; x=1708353488; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=GJVMzOCjIbr/pyn9HLEbGwESHTTN32845yRzZszXAtE=;
        b=H9SR2h8mleY7fAiBdiPrMb/kn+dH6AA99V58SfsiLfypV2MAJc+t+DOGTrYJFLm5fX
         w2xcYPOKMgTU30HYP1PVWjO94X87KCDyf2ONssP6p3EGmLH6tvWW87lAoti5bTU2NlNn
         foZV9JUuZA2Q2CY+PdzOIhrDt7O5XDy3ztV21UmyHGQNMOisuqIM4RUPoHNT8KkY0YmY
         /7p4wF598SezgPNwyJ7u9HRoqKS3/Uop6ABZDA8bi5uGF0CfNl/XckWB7+5t/rLa+Xsc
         AozeBFejjSxLtKb+bBAhfap+psbMuWRDdGayAntxyRuDONsrabRKEc/aiGFsZTF4D2xJ
         BDag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707748688; x=1708353488;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GJVMzOCjIbr/pyn9HLEbGwESHTTN32845yRzZszXAtE=;
        b=tNJN4tvfny753WR87nT8/N/iFdkCj5TcUH6u+C3awhXbyiuXu882y3xrO+6INRiZBb
         m3uGdlv98BUB4X5ULzr4JZwiBv70OEo6jWd9fDLXqvCYsUPsDy7SNS1z4puo5oy/+56A
         6gBFUyYja8npeZCKv2XcP5gVkq6WknJ6Dwh4/8ewbxyqHZeuMxnwN2eAY9E36WIF7ek1
         U8ftHH6OMDS1APPbTA2Hopn67apy+ifw7W6bLWtXeTq6CpLdzyMLClPdN7GG/YT0TsU+
         i/HkOsX4eXTx4Vo0eBKFlbbtQ1LibBqh+E642Aq8Yy/yVhBvgGeJDpfVyVd8spAR2clU
         oSxg==
X-Gm-Message-State: AOJu0YwGE31lf95DD7pWWpl7smx1FyexJnQgApEvqSZl9lNc3unj8lc3
	NazrM+gXBeBCE8aTQZ7SY7n/LSSV9lTrFXCGb5MZldtGLi8mlLsVfyV1UNqhmA==
X-Google-Smtp-Source: AGHT+IGQmMT3rU8pgfK33CuiCMI6y2NWgL4bQF1gSCYRsd3A9ZULN/Qku12uVRSMVMs08FCAzO0EBw==
X-Received: by 2002:a05:600c:5204:b0:410:6d23:dd6e with SMTP id fb4-20020a05600c520400b004106d23dd6emr6154922wmb.13.1707748688285;
        Mon, 12 Feb 2024 06:38:08 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXBHge1QXw8XP8/mR3tE9FM0xoFbEqJGOG92pcNv3IKQ3/xkCQ3BMQWxYaJe2yfWSzEDvpMKu+q093A0Z1ijC1di2ZTgInNrAbtjfOhqpfvGwiD/lIEz9KgZ/Z6cBpAom1zcBcigjsuNLM=
Message-ID: <c27c76ec-36cd-43cb-b76f-e8f95fb27ed8@suse.com>
Date: Mon, 12 Feb 2024 15:38:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/2] iommu/vt-d: switch to common RMRR checker
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Kevin Tian <kevin.tian@intel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-3-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240207153417.89975-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 07.02.2024 16:34, Roger Pau Monne wrote:
> Use the newly introduced generic unity map checker.
> 
> Also drop the message recommending the usage of iommu_inclusive_mapping: the
> ranges would end up being mapped anyway even if some of the checks above
> failed, regardless of whether iommu_inclusive_mapping is set.  Plus such option
> is not supported for PVH, and it's deprecated.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon Feb 12 14:56:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 14:56:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679619.1057134 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXj8-00075f-R9; Mon, 12 Feb 2024 14:56:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679619.1057134; Mon, 12 Feb 2024 14:56:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXj8-00075Y-OF; Mon, 12 Feb 2024 14:56:10 +0000
Received: by outflank-mailman (input) for mailman id 679619;
 Mon, 12 Feb 2024 14:56:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kTqq=JV=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rZXj6-00075S-Ug
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 14:56:08 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d9b4b0fa-c9b6-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 15:56:07 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 9704D4EE0737;
 Mon, 12 Feb 2024 15:56:06 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d9b4b0fa-c9b6-11ee-8a4c-1f161083a0e0
MIME-Version: 1.0
Date: Mon, 12 Feb 2024 15:56:06 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>, Stefano Stabellini
 <sstabellini@kernel.org>, Xen Devel <xen-devel@lists.xenproject.org>,
 Consulting <consulting@bugseng.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Andrew
 Cooper3 <andrew.cooper3@citrix.com>, Roger Pau <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: Return type of clean_and_invalidate_dcache_va_range
In-Reply-To: <02446758-a5a9-4a8a-8ce6-15abdc9ede65@suse.com>
References: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
 <alpine.DEB.2.22.394.2402091402080.1925432@ubuntu-linux-20-04-desktop>
 <f86baad0-f113-4156-9c10-6910e8c63492@xen.org>
 <02446758-a5a9-4a8a-8ce6-15abdc9ede65@suse.com>
Message-ID: <99faac70440a68824a17fcaaea55ef48@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-12 09:26, Jan Beulich wrote:
> On 10.02.2024 11:17, Julien Grall wrote:
>> Hi,
>> 
>> On 09/02/2024 22:02, Stefano Stabellini wrote:
>>> On Fri, 9 Feb 2024, Nicola Vetrini wrote:
>>>> Hi all,
>>>> 
>>>> In the context of violations of MISRA C:2012 Rule 17.7: "The value 
>>>> returned by
>>>> a function having non-void return type shall be used", I was looking 
>>>> at the
>>>> function "clean_and_invalidate_dcache_va_range". It has the 
>>>> following
>>>> signature on both arm and x86:
>>>> 
>>>> static inline int clean_and_invalidate_dcache_va_range
>>>>      (const void *p, unsigned long size)
>>>> 
>>>> The commit that introduced it for Arm ~9 years ago (71d64afe3e12: 
>>>> "arm: return
>>>> int from *_dcache_va_range") [1] mentions that on Arm it can't fail, 
>>>> but
>>>> supposedly it can on x86.
>>>> 
>>>> However, as far as I can tell, for both arch-es the implementation 
>>>> now always
>>>> returns 0 [2][3], so perhaps the mention of -EOPNOTSUPP for x86 is 
>>>> no longer
>>>> true (I wasn't able to reconstruct if there was a time at which this 
>>>> was true,
>>>> even in the same commit that changed the return type to int).
>>>> 
>>>> The question is: should the return type be void, since it appears 
>>>> that every
>>>> user is ignoring the returned value (violating the rule), except the 
>>>> one in
>>>> common/grant_table.c [4]?
>>> 
>>> Looking at the implementation on both ARM and x86, I am in favor of
>>> changing the return type to void
>> I think we need some consistency between all the cache flush helpers
>> (clean_and_invalidate_dcache_va_range, invalidate_dcache_va_range() 
>> and
>> clean_dcache_va_range()). They should all return a values or not 
>> return any.
> 
> +1
> 

I agree. I took this helper as an example, but e.g. 
invalidate_dcache_va_range returns -EOPNOTSUPP on x86 and it's only used 
in common/grant_table.
Perhaps the signatures should remain as is for consistency, especially 
given the remark below about the other architectures, and this would 
entail a deviation.

>> That said, we have two other architectures in development. Are we 
>> saying
>> this helpers will not need to (initially) return -EOPNOTSUPP?
> 
> For "(initially)" that's not an issue - such a stub can as well be 
> filled
> for BUG_ON("unimplemented"). The question there is what the ultimate
> implementations are going to look like.
> 

Should I CC them in this thread?

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:03:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:03:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679623.1057144 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXqd-0000JO-Js; Mon, 12 Feb 2024 15:03:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679623.1057144; Mon, 12 Feb 2024 15:03:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXqd-0000JH-Gt; Mon, 12 Feb 2024 15:03:55 +0000
Received: by outflank-mailman (input) for mailman id 679623;
 Mon, 12 Feb 2024 15:03:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZXqb-0000JB-KV
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:03:53 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ef05e113-c9b7-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 16:03:52 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33aeb088324so2151984f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:03:52 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 b8-20020a05600003c800b0033b65818667sm7152688wrg.76.2024.02.12.07.03.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:03:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ef05e113-c9b7-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707750232; x=1708355032; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sDMVOYplVYaTemfb+geuyF7pUqmwx0HbxEjrQg17lM8=;
        b=UJAM06nzrwBObTvV+trEBmgdo6tAD2m/KUB7VNS/PBDrppdiyACJ7K0ZRSCBcSO5JG
         Fe188/kgoTsRH7HmTXxd9U3IEQ1Q9Go6u/qdxQi8fodFy1JBUqiYLl6HGx4mkXSjk5CM
         lFoJn9+M6OVlLjWM17euwjQT6PvwUfxvHiSGTMWxrp3PJoJEuY+Ny3GJK83fgIItSj8B
         2DMC5AqSzwwTRzoKAY1z/QCsT6wETiOj0ivi+onWdGiz1y7q6lAvCKF03O1fObQUSjEj
         WI2uO4hU3tU/Zfpf8DKUJDIeEM4mLgAwkv5GBkhQJXEdeG/J4Qn4uf95bFCv/Ctwv/ip
         iKJQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707750232; x=1708355032;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sDMVOYplVYaTemfb+geuyF7pUqmwx0HbxEjrQg17lM8=;
        b=pl52iF9/blVOZNwrxQ2AgYNuNuKW8Lz6D33o2qLR8KS23SnHf6w0XZXnj8FXnWts7P
         6hHkGViY7AvAvxhu4C12mpXB7ZWvaFp6kQQyzSbNkFnupiG4HJRbopgE4Ba0kdYPZg/h
         YLizGmuFcNMQhfju1eoZIeioQNUMQYoeEDJmXlk4AkWzsAARuNbxPTUX6W6/DlO4+DX0
         yp7r25vfoI7+KPV4vCjdny/mKD41RlNA3bfKsIOX+4XIXnomA8R4ixrBKgKAKfM6lx1d
         2VdUdqhYbCiH1Q4UmsdpHiOh38XiOvcXHAaJhiRR6n5lTIpjzpWzjIV0rwnwbPDVs3fR
         vsew==
X-Forwarded-Encrypted: i=1; AJvYcCWCGKNx2iJ92zrgsKDuc6aZbGuXvZO7wRAhIQwgx7Y7Agj0VaVpvfGGvH9VYrJ+sQyIksEMrQZ5XOoidemreZhG7rBjFRcWHig+NHo9kXc=
X-Gm-Message-State: AOJu0Ywv4ep+M/BjsiLcasPLTbEcnt+/hfC897jEDxTSG8RkEEWt9+li
	Hflf1DK0Dka9ZxOJBp/k50eWiwPhm9B3gGHK6/IORp89D2i625Z/5qBB21ZyVQ==
X-Google-Smtp-Source: AGHT+IH4fSAMXTt/POP73CskPlVA8KKufGUmIGZ81uurJ1uoaJZSJbCtSlpoy+5d0pqjJgaSUZ1c4g==
X-Received: by 2002:adf:f40f:0:b0:33b:6dd9:a6a with SMTP id g15-20020adff40f000000b0033b6dd90a6amr5618089wro.22.1707750231754;
        Mon, 12 Feb 2024 07:03:51 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU8VkbH/BnRylsn8L7ryl/KhrCVNiULvDzTc+kMwoxLHybWQShasYc6MKJ7Lfex4Do8SKOyGVn+z7hlMMg/UA7S18N/IweNVNOF0CcfQkmR35n6WP/+syWrWCJw3fT+rQ+9lJQJP+b+saBJtfIk6p/tojjnLKeZakAzZFAS8avOBGp21cNe2RSEL/WPLmm1JfrJCA9dLMgtyrDGkhA+T6vKPNyqGAvD53rMDynvaXXfEGPWqodgOAfYVoY1Jk60TMtxsWfv+WVwdzoTbnZgWLaCEXVPAWcZFd88WUhJZ7C0Df0kGwqxXEo=
Message-ID: <16baca98-44fe-42ba-b61d-ff1945e0d2b5@suse.com>
Date: Mon, 12 Feb 2024 16:03:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 02/30] xen/riscv: use some asm-generic headers
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <a721f0c092306b589fae5f44bdaafcd94c60ed14.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a721f0c092306b589fae5f44bdaafcd94c60ed14.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> Some headers are the same as asm-generic verions of them
> so use them instead of arch-specific headers.

Just to mention it (I'll commit this as is, unless asked to do otherwise):
With this description I'd expect those "some headers" to be removed by
this patch. Yet you're not talking about anything that exists; instead I
think you mean "would end up the same". Yet that's precisely what
asm-generic/ is for. Hence I would have said something along the lines of
"don't need any customization".

> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> ---
>  As [PATCH v6 0/9] Introduce generic headers
>  (https://lore.kernel.org/xen-devel/cover.1703072575.git.oleksii.kurochko@gmail.com/)
>  is not stable, the list in asm/Makefile can be changed, but the changes will
>  be easy.

Or wait - doesn't this mean the change here can't be committed yet? I
know the cover letter specifies dependencies, yet I think we need to come
to a point where this large series won't need re-posting again and again.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:05:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:05:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679625.1057153 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXsM-0000tR-TE; Mon, 12 Feb 2024 15:05:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679625.1057153; Mon, 12 Feb 2024 15:05:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXsM-0000tK-QV; Mon, 12 Feb 2024 15:05:42 +0000
Received: by outflank-mailman (input) for mailman id 679625;
 Mon, 12 Feb 2024 15:05:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZXsL-0000qI-Hk
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:05:41 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2fbf85c3-c9b8-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 16:05:40 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-411a5b8765bso1823275e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:05:40 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n16-20020a5d67d0000000b0033af26545c8sm6987897wrw.50.2024.02.12.07.05.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:05:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2fbf85c3-c9b8-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707750340; x=1708355140; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=G7EAqjlnRFxD16UrnMJajlEeaimgLOuCMVU+W7dSDyU=;
        b=MOyRXtpqMqNBT2hiM5EmXZ1VbNep9WLhbjrzQ79HD4tPiwsRzRHtJMaHLAFczfN3eT
         +MH8YI0gxnKEjT3xzRJwhh+yaRCzdaA1mKXmxijmvoIj23ecNR9RxUxT6yQy6cZrbQyB
         XKZajsfEDCkYd02r/TM8293iCPvXzJuSyPvQMdCYXKE7neCNYDURmEumst2MyMlKRBea
         R+CWCqvd87Gjs/w6rrZYWebtpNzY2NPFekGeHB+DgkJVHmY3teOWifCMkYBxOqSGakYs
         LL9FoyqohLt2ByGvTpoC2YXpLEVQco80BOiWJgT7xdsxKfTA49LJQpFXgF/JyyqrFlrG
         tITQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707750340; x=1708355140;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=G7EAqjlnRFxD16UrnMJajlEeaimgLOuCMVU+W7dSDyU=;
        b=umiTwJ7f8xPpyHyZPQ3gdYm29QDEyUIU/SVAWIIIZAojd2ZkQrDafgbsJyO5IEkDFf
         I+QxtQusvm/XqTo57mYoCf7Ona9wL/CTPlvQYJyZmKIKbBotsR/UU6FeZN1/6U3ky1Po
         9XJSMs1qiif0xKwH+0aEBqRpXFFHBOiwpjrb8v5/eR4AiDVasVCZ9ygBSk9siRsa9sJB
         x6ily5i/GfY4rDM+a8dqsNTQNlrkCtmDh8vYzJwNIKSCqu59fziyOaBGjG/eEc8ctud/
         TyHNg4CW45SSWIcL5kkf03ckihgT72Zxmh8b1bMOToqeqJeo5PWGCnykrOPj7nfIiaK4
         bF8w==
X-Forwarded-Encrypted: i=1; AJvYcCW8OXnJHE/gUCe2My37TlywTlVMqC2DWb5hHZ7UhoM7cFoFjy0ut7U5SdOYo8bSIr8OrF387ZaE0pkbeHvqZtIIG/UuxCvB1rH5yMy9VBQ=
X-Gm-Message-State: AOJu0Yxix3l245BXInCSX2OomO/Z+WzBolaIP3DRO7xCfRdpAv+62uV/
	CvInOfIhJgBxQaY98r45sGP34EImGoH6h8EYgnuEx7D7oQxYdHQXTH6mPeUU3g==
X-Google-Smtp-Source: AGHT+IGy0VFkhkTLhNt+G9H60u9TGF9DW/ZeWApp2ury9xxkqzHDCeCyUqAp+QrPN/PzuU7+3LbV8g==
X-Received: by 2002:a05:600c:4f4b:b0:410:ce61:8609 with SMTP id m11-20020a05600c4f4b00b00410ce618609mr2883617wmq.11.1707750340389;
        Mon, 12 Feb 2024 07:05:40 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWfbj8Vmg0UNwF4rBg47khSVQE2Ff5q8ggcypWcRXorFraBcAV+sX+8QuU98MAz2x4rin8cQtpazQBxvhBuT0lhrKW0T6XNy8buvWBP96xYvF0tPXJ5f7D8FQponHNdUH5uWR8jfkjPRjmdNOfydXQte/k3EHixWcV9qSsrCUnst2Auc9eXmj29ccYEM6WEbgNQHBl1XIwc/TNfTRM=
Message-ID: <b3141aff-32ad-4e2c-aa2c-a5803924951f@suse.com>
Date: Mon, 12 Feb 2024 16:05:39 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC
 and RISC-V
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <c929f784466e1ba57c19663e88f22c598d961709.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <c929f784466e1ba57c19663e88f22c598d961709.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> No specific header is needed to include in public/hvm/save.h for
> PPC and RISC-V for now.
> 
> Code related to PPC was changed based on the comment:
> https://lore.kernel.org/xen-devel/c2f3280e-2208-496b-a0b5-fda1a2076b3a@raptorengineering.com/
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Albeit I don't see why ...

> --- a/xen/include/public/hvm/save.h
> +++ b/xen/include/public/hvm/save.h
> @@ -89,8 +89,8 @@ DECLARE_HVM_SAVE_TYPE(END, 0, struct hvm_save_end);
>  #include "../arch-x86/hvm/save.h"
>  #elif defined(__arm__) || defined(__aarch64__)
>  #include "../arch-arm/hvm/save.h"
> -#elif defined(__powerpc64__)
> -#include "../arch-ppc.h"
> +#elif defined(__powerpc64__) || defined(__riscv)
> +/* no specific header to include */
>  #else

... this isn't simply

#elif !defined(__powerpc64__) && !defined(__riscv)

Jan

>  #error "unsupported architecture"
>  #endif



From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:07:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:07:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679628.1057164 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXu8-0001ZP-Bv; Mon, 12 Feb 2024 15:07:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679628.1057164; Mon, 12 Feb 2024 15:07:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXu8-0001ZI-8y; Mon, 12 Feb 2024 15:07:32 +0000
Received: by outflank-mailman (input) for mailman id 679628;
 Mon, 12 Feb 2024 15:07:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZXu7-0001ZA-5K
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:07:31 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 70fc8082-c9b8-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 16:07:30 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33b2fba3176so2011088f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:07:30 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 c14-20020adfe70e000000b0033afe6968bfsm7010255wrm.64.2024.02.12.07.07.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:07:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 70fc8082-c9b8-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707750450; x=1708355250; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xx6RQITngQNSwNElmlNiSCVTEr0o1+m656emyig+vpk=;
        b=RVKksXhGOyJ2HgPO7OY5a0StgQRBuVqbk414bszVE61ZOabJIyfyyXm8Fiakh3ssrq
         DGJgMZoMK3MSUdWe+N84QcOtKR+yPfWXqKQb8YqTI85rGttqYxWDI6l25ZXRMpOX/FjK
         Dxf2sIQzEL2vsjWvOzbOZGSdvZ4xSOMMEh+UXnYckQOeJcqi6XF/eW0bsoIt1h9IokiN
         ttnTvfJSDpOL7MugP7j1zU3N+TIABVcjSC04mgJenDVGItWt1Gi4wqzoEt3g9eGNKqKg
         5BJxE0CigNJiMJt+vMbRhoc6OGJ9s5iBgsucAHQYc1cp0IMaNBtRp0KQUCc3Z05w6i98
         0zjA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707750450; x=1708355250;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xx6RQITngQNSwNElmlNiSCVTEr0o1+m656emyig+vpk=;
        b=WC3wvIcHLEL+fu6+8G7X/Ab9cB4V/0vxe/kZsr0S1pN7gHLxhvuCRBlbrfu0W09BZm
         ztwFY62D948M0AOnE9FmxADqL6POZ6hCQj5hErre6gP+mjzdI/2SeHtfE1seXuo5g5v1
         HQWV1+gVMF00zsl6zCafDWsntN6tHLqUkAsqCxh1pCh8ZASte57EIYnCO9LMFWtn5VzH
         nh/6B5SIrH+Arpa9GrsTBr4ajr/28Fs6R684oup2yk3gntZc9Eb5UFHO2pwm3nJy8FjR
         Yw7uEi4E25sH2WAWbsahMgvfnFvgfwbr1/6FAoIvesMxh3gcs8mitHQL+A7tZN6kJg+M
         RtOw==
X-Gm-Message-State: AOJu0YzNkGCLLqsJctUDgRINDAO/wGhtLiwj7EmV9NaUcZWdmNa6lp9f
	MkUU1W+k1YhPo7bgLnQz2PNm3YaTzTsRl91b44xlm3J/n13CxL9+Dzpx2BvW2Q==
X-Google-Smtp-Source: AGHT+IGDUu38GjGFZAmi+ROQD3XMA31L4ICUeYy/rFtr6wkGL+UgxYpFyX5zKu34mt6H7kbQTYy8VQ==
X-Received: by 2002:adf:f84c:0:b0:33b:87fb:7106 with SMTP id d12-20020adff84c000000b0033b87fb7106mr1080013wrq.55.1707750449791;
        Mon, 12 Feb 2024 07:07:29 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVfkaL3V42iqGsWTiCtvzEICpcWqDLN22JNi+m45yDPedPcufz9jza3B9QCfjfzyMNn9j1jUbdguWCQP3PoDeo0g0KiutceyG/3U5EuKVrbRymM3wKjfPY1GTOC4OJwIEAwh9VVCxSfht7j41fr4t8VkAu0JKUPvx/qVrSH8fnYatH30gatjq4Dmx57fGesASmpFJOUgUP/9/XXTWGnO1wsSOhqq3s3Z+szm2m0Vva2tFd4QFCA8BD9Q5p+fkoecsNrx7sH71oq6U5/HiT4JjKjUWE5/HpMpKJF0MDEiWfyoaHvbuVcl7w=
Message-ID: <ec76c372-8c17-4550-9896-eab3e0ab0c66@suse.com>
Date: Mon, 12 Feb 2024 16:07:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <542c61229e1056c3fdec3a5c086136e3b5909b07.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <542c61229e1056c3fdec3a5c086136e3b5909b07.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:11:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:11:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679632.1057174 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXxL-00034y-Pu; Mon, 12 Feb 2024 15:10:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679632.1057174; Mon, 12 Feb 2024 15:10:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXxL-00034r-Ms; Mon, 12 Feb 2024 15:10:51 +0000
Received: by outflank-mailman (input) for mailman id 679632;
 Mon, 12 Feb 2024 15:10:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZXxK-00034l-JD
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:10:50 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e7159183-c9b8-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 16:10:48 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-410ed9f8621so5077925e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:10:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 jp25-20020a05600c559900b00410c7912c6esm3977735wmb.14.2024.02.12.07.10.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:10:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7159183-c9b8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707750648; x=1708355448; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PWeD61159hXUGcIsdFXpvkgqfedGul7/HeNUVYarMMQ=;
        b=KQev6PZH+sQqZrmzo2KgMF03SPk+2IuzU2KamJY5oM9IsY2p1ZJTS/M1M4lCbH8plh
         1VY++XzNB7RAkRXZhpQBysPCDvjCDkww24EvQJnqphVdUO9JL3IWz7SNPFq025CF+qE8
         RD9Icy8yzj/lV/KY7i60+rQqY0i2WvQEZfyNdGvEGHRKcmy5R0xldXCeGjCM2D1fTjyy
         7aNz1rBZy5+lq8mIqutZYwqXZi8AOCCu2c2g4d/6XJTVWpPjfeB5DMI3GSexh1kcOCct
         C6D0eWjBX5U0T+gNhELyu+iB5efZwpF7fAnFMBCl84C9kulrD90Aicxj1KpQMJGMkBqy
         bRWg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707750648; x=1708355448;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PWeD61159hXUGcIsdFXpvkgqfedGul7/HeNUVYarMMQ=;
        b=ATzslDNKj7OMQ7TjLo+iRdJgAV3aqfsU8Day0w6DLJH2e2pVG+23YrnBxXzOIXyvsf
         VHUchgxUT4fVS/pRVpN+4S5nRIUky2ieSK/AmxswDcWvifwY5EnznKzxLvmb3sWi0fE7
         os6Uon6kwmo/lEEJWPIRf3zHJrAbXuBF8DBr+h/AoQ04MHVLVQrefN8l1BMvJdg9kV86
         6a7VPLshutQNXWJs7AQXCH1eXJyqlvUAR8ERS0VasF9cnzbU84Q67+r2c3za8esB+tcO
         eK3ow+t3sHxt0yQL+NvmkEkJtuAwnQqgTibP1Ez1Tb0J8z+rQep10NFFpLwvTkLDPmGq
         /Bqg==
X-Gm-Message-State: AOJu0YxWUxF7uuT9RjIjapY5ToBEXeq/hNtxGZeKyf7Jc6PAC7C2Jb7E
	t5IUZkT9eq11Od4CVX6Ki4b7bPaaXrB6RCsxZysf7wpJqh7v5JCRGKxAMZ+GFg==
X-Google-Smtp-Source: AGHT+IHTBWN9pEuUnMEI8ckRbRAmLZVeM2ltdT/fSBmjMK/yfMnWAdKozXOnY+uyNQfBOxDcvAfvGw==
X-Received: by 2002:a05:600c:314b:b0:410:f5d2:cbe5 with SMTP id h11-20020a05600c314b00b00410f5d2cbe5mr1173620wmo.37.1707750647934;
        Mon, 12 Feb 2024 07:10:47 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWLRMd+lCfigK2sOdzukez4iwfcFBG5W7IpKgHUurK/roAWrSNqZxe7OcDoh1acFtgzYrX8RebpWh0Aeh6vDC3bQNtNuwCk7TZ/IltQgglhApD/DOfJthekbXJHN3Oncwhzu5ZZ
Message-ID: <81593ac4-7714-4dd1-89f9-c22e88223d34@suse.com>
Date: Mon, 12 Feb 2024 16:10:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Paul Durrant <paul@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <a94aba19d32bbe8d2fa78d6d8931f9f0ca51160a.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a94aba19d32bbe8d2fa78d6d8931f9f0ca51160a.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> asm/iommu.h shouldn't

... need to ...

> be included when CONFIG_HAS_PASSTHROUGH
> isn't enabled.
> As <asm/iommu.h> is ifdef-ed by CONFIG_HAS_PASSTHROUGH it should
> be also ifdef-ed field "struct arch_iommu arch" in struct domain_iommu
> as definition of arch_iommu is located in <asm/iommu.h>.
> 
> These amount of changes are enough to avoid generation of empty
> asm/iommu.h for now.

I'm also inclined to insert "just" here, to make more obvious why e.g.
...

> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes in V4:
>  - Update the commit message.
> ---
> Changes in V3:
>  - new patch.
> ---
>  xen/include/xen/iommu.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
> index a21f25df9f..7aa6a77209 100644
> --- a/xen/include/xen/iommu.h
> +++ b/xen/include/xen/iommu.h
> @@ -337,7 +337,9 @@ extern int iommu_add_extra_reserved_device_memory(unsigned long start,
>  extern int iommu_get_extra_reserved_device_memory(iommu_grdm_t *func,
>                                                    void *ctxt);
>  
> +#ifdef CONFIG_HAS_PASSTHROUGH
>  #include <asm/iommu.h>
> +#endif
>  
>  #ifndef iommu_call
>  # define iommu_call(ops, fn, args...) ((ops)->fn(args))
> @@ -345,7 +347,9 @@ extern int iommu_get_extra_reserved_device_memory(iommu_grdm_t *func,
>  #endif
>  
>  struct domain_iommu {
> +#ifdef CONFIG_HAS_PASSTHROUGH
>      struct arch_iommu arch;
> +#endif
>  
>      /* iommu_ops */
>      const struct iommu_ops *platform_ops;

... this is left visible despite quite likely being meaningless without
HAS_PASSTHROUGH.

Then (happy to make the small edits while committing):
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:13:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:13:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679637.1057184 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXzW-0004BW-5L; Mon, 12 Feb 2024 15:13:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679637.1057184; Mon, 12 Feb 2024 15:13:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZXzW-0004BP-2Y; Mon, 12 Feb 2024 15:13:06 +0000
Received: by outflank-mailman (input) for mailman id 679637;
 Mon, 12 Feb 2024 15:13:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZXzV-0004Ay-5k
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:13:05 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 37f7168c-c9b9-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 16:13:04 +0100 (CET)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-51187830d6dso1306635e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:13:04 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e11-20020a05600c218b00b00410b98a5c77sm4697261wme.32.2024.02.12.07.13.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:13:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 37f7168c-c9b9-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707750783; x=1708355583; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xx6RQITngQNSwNElmlNiSCVTEr0o1+m656emyig+vpk=;
        b=dlT89y9gYv5+qUojVoj1LKGvrv+aWYK+HF3FqzVazMunkIprEUeWYqv0DzZjr1oabI
         AoeqhEKTHWCQJnDH+wfhui+eD71DKEcYmm0FakUiCeUL5gkqlQfTAe+LLq/JX4Wr1Sue
         1le66k/xR6n19KCPJSzm4AMsuLBOTcPY2wHYMgSTFkpekQIDd9YfCG6/WoK/wpoG/Kna
         /+6wJP2nyh92ToUYtSmwTrtI6u1/vMGY9fnRZZV4znaiiZ3qSBStqgWBF3/A8sxTiF1O
         RdiB0FPgP0FU6pOXJK3SAMOvv+3k186qFwS0+tCU6TNozGEnj//yckmioW3dIeTjYsmk
         bYtw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707750783; x=1708355583;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xx6RQITngQNSwNElmlNiSCVTEr0o1+m656emyig+vpk=;
        b=djSqQ/olfbsOAuB4eUYsV11kWmxvyZH/0c2u60LSa+qyzIEvt54A0OBGd3O49QSwXW
         UnZvLJcX7laXgJDuPKgXxLcRHKOrCUT7Xuh/Jio7uxmgFH7mhDWRASeI3kBUM69ZGGJF
         es/1avY23xFNPiHSc2tlr1ckjwdWH7fg0R+MRnZQGuFu+YR8Mq8vXog7fg6kO1G1a7eZ
         po6OgzKtaQ8v3f0JhFU60tno+oDmOw530TMfi3QbXrXIgPAbfQbDichZzhOGgr7qz3ZF
         /jLqkpEMI4ewxpVqwEr+fDyrXnrXMlBbT4zETYQkaN87Zz3F/isn+I5wsRDCoLjyHnCe
         qjdQ==
X-Gm-Message-State: AOJu0YyJA7IVUNIP5jpkPyNWS4fM7BOmvKx1jg+KOItmrSn6J8jz6CCi
	GJQJwYzEtn8RN3Qt0xbQDW1HgOIE6O4sVD8rAi4MNoOrDhsfqpdnL+DEOH3sTQ==
X-Google-Smtp-Source: AGHT+IG12MZySDOfZEwetu/MxoPJMuy6CDtVGBF5fvhPMZZZPE6K6f3rnFoOuII+sJXPkRFnrIKW9g==
X-Received: by 2002:a05:6512:400c:b0:511:3b8b:6eb2 with SMTP id br12-20020a056512400c00b005113b8b6eb2mr6203557lfb.42.1707750783600;
        Mon, 12 Feb 2024 07:13:03 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU8WGAkqHrhp1u1aeAdciGiK7Mq2qXGa6Uu41dYi+pHY4IsyLsxwpCRJIMSXN1r32VhnrX0tFbnqq2VQZSXNAw+SUIzCZOpIUJGS4wbCKdVMrGmcZ9RIMafMF6K2nqhSVUqGPOr3bI/CJ+OJEk355YAyJOLIDTONTE7WO94e2Qj3/w8AbpQlOzeSHuQGarP1BJMZ1x5Wp2GDYSMDa3QwNLJEU2a2sAINR94qVY72rfq96ebaPk1kzoLjTLcY8nNMG1V7ix1UekPGvekTyhYGov2fLy/1jA/MlVcuQJtAYmxMqy05sWQRPE=
Message-ID: <3a74458a-809f-40e4-9c86-e58e339fe96c@suse.com>
Date: Mon, 12 Feb 2024 16:13:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 11/30] xen/riscv: introduce smp.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <4d0a8aba3ec17fa428f8d18f0008f863481b1f67.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4d0a8aba3ec17fa428f8d18f0008f863481b1f67.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:16:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:16:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679640.1057194 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZY2c-0004oI-Hh; Mon, 12 Feb 2024 15:16:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679640.1057194; Mon, 12 Feb 2024 15:16:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZY2c-0004oB-EZ; Mon, 12 Feb 2024 15:16:18 +0000
Received: by outflank-mailman (input) for mailman id 679640;
 Mon, 12 Feb 2024 15:16:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZY2b-0004o5-6T
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:16:17 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id aa45ac2d-c9b9-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 16:16:16 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-33b670d8a74so2166328f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:16:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i9-20020a5d5229000000b0033ae7d768b2sm7048670wra.117.2024.02.12.07.16.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:16:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa45ac2d-c9b9-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707750975; x=1708355775; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cFRmbZWrJgN5eXjLOclQO6ULKvl6rSK34mkGuilIuAQ=;
        b=AlYeKHbZwx9karbB53Wx8W2JwiUtnEO8zCcHjJ+R5zEK0dr7gsZ27geWWB7//zQhZJ
         xyiUi7+nXJhASD72hvfHNJcG/wd4eArHhozvbVu8cUGNaNDxFykAfxYWjZsm1WPozkPV
         dwkcFxkjhf+CojH3gVfJsFrkiERo3x38OdU8QYGX4vzBraQ1zmiyZyGn+GlDdy7D0+P5
         TIBj+gLUfSbMv+25tXZvFJg1nkI+Xpq01SNzzRy7TS7+3qt0DZhILX9x6RHN4XjADgsM
         8ibY3VpO3eMEIE6rkfKP6madmjvG+hN/iWHsusBo6VUaYgO8zPHZHC80ZW7apcmaZaGv
         L76g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707750975; x=1708355775;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cFRmbZWrJgN5eXjLOclQO6ULKvl6rSK34mkGuilIuAQ=;
        b=OHy18Yc24HDZrUrwUgdbVYafyuNOJUicj6CfVzto0kh7yePjnO8Um62GyB4Osybq6W
         d4Puc2UHSJy8hV99ruz5PP4dHd5DU8i65BdvZA9SOMcR9FGjG4IFEtOB03GgGvSHXNvj
         +QwyBcWfCmNf/BdEJ5SyCB/7j/Y4UiCBp43Qx1gXKT3VNPY64zUpDNY17gpaZU3wZGNS
         B9e1WXfUAvRUlkp41bvBbjKAS7Xxj/Zh77XYy87VUqhrKR8cc799ixtAMWPJD4cTN8Om
         qwrpJoH+V5eaMOCNLEXpZBJG71b5kyXaf5ek+TJw9956GIDYMbHMPCDZDs2ePLa02DYa
         mQwQ==
X-Forwarded-Encrypted: i=1; AJvYcCWyws+LGE/YdWfXmiNiXS1N1Y7ucS3eXfz0dHoGEIy7rAC/JSrY1OFUCalTNQdTrKYY/EiP9NxOntLCsBn/NVl5QOiXw2AvB3o49LNWIkg=
X-Gm-Message-State: AOJu0YwaHpM7LXl93LVrVjHZ4KNdSYfPMYSZmaThxoPfV37YaiDKV4Un
	1vb7449dE+egH9ZQlH0kckemgPFySAbfcuHwOyWKg+aaGdab3SFGYiBMRJreoA==
X-Google-Smtp-Source: AGHT+IHfZsUS2D21RtC1Q/mgzNtE4jQAnQbI5bG/jSLGt+RgtwnvTRmsmfyJWihOGBUIE3zHkWBJPw==
X-Received: by 2002:adf:e492:0:b0:33b:87d8:678a with SMTP id i18-20020adfe492000000b0033b87d8678amr998045wrm.10.1707750975395;
        Mon, 12 Feb 2024 07:16:15 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXu+wjTYZmPVwLmaXJRWmcWHzhO6pokM/82hlU/FifjGoiR8iP9Z8UAByHlHCZ1PXQtU6ugVpGeL6CSninP9SNvX9sO7JMUIKzrsW+NngTiU2ONglPETSH5qCbHsQzofvvCs3sf3LOD/w25nyCLmj8L4b1gIz5wVJSfNp+7IrtV6DupmeADkxEEuKcJOXxVycQofk/5WiXrik+20DTo0Ln/ZvQnv89+FBXlE3Gtd8AJXnZp67PvlTuagbB6tIYh4MR3106Th0tczmb9bEeEqjOThKCxLDUrKyL0qdBFVqhRAyQGzo988A8=
Message-ID: <5e5887b2-5eb4-4a81-90f6-246553c28863@suse.com>
Date: Mon, 12 Feb 2024 16:16:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 16/30] xen/riscv: introduce p2m.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <c2a2e872f8c32d81c3d3f428f0273819dd7df081.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <c2a2e872f8c32d81c3d3f428f0273819dd7df081.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
with two more nits:

> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/p2m.h
> @@ -0,0 +1,102 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef __ASM_RISCV_P2M_H__
> +#define __ASM_RISCV_P2M_H__
> +
> +#include <asm/page-bits.h>
> +
> +#define paddr_bits PADDR_BITS
> +
> +/*
> + * List of possible type for each page in the p2m entry.
> + * The number of available bit per page in the pte for this purpose is 2 bits.
> + * So it's possible to only have 4 fields. If we run out of value in the
> + * future, it's possible to use higher value for pseudo-type and don't store
> + * them in the p2m entry.
> + */
> +typedef enum {
> +    p2m_invalid = 0,    /* Nothing mapped here */
> +    p2m_ram_rw,         /* Normal read/write domain RAM */
> +} p2m_type_t;
> +
> +#include <xen/p2m-common.h>
> +
> +static inline int get_page_and_type(struct page_info *page,
> +                                    struct domain *domain,
> +                                    unsigned long type)
> +{
> +    BUG_ON("unimplemented");
> +    return -EINVAL;
> +}
> +
> +/* Look up a GFN and take a reference count on the backing page. */
> +typedef unsigned int p2m_query_t;
> +#define P2M_ALLOC    (1u<<0)   /* Populate PoD and paged-out entries */
> +#define P2M_UNSHARE  (1u<<1)   /* Break CoW sharing */
> +
> +static inline struct page_info *get_page_from_gfn(
> +    struct domain *d, unsigned long gfn, p2m_type_t *t, p2m_query_t q)
> +{
> +    BUG_ON("unimplemented");
> +    return NULL;
> +}
> +
> +static inline void memory_type_changed(struct domain *d)
> +{
> +    BUG_ON("unimplemented");
> +}
> +
> +
> +static inline int guest_physmap_mark_populate_on_demand(struct domain *d, unsigned long gfn,

This line looks to be too long.

> +                                                        unsigned int order)
> +{
> +    return -EOPNOTSUPP;
> +}
> +
> +static inline int guest_physmap_add_entry(struct domain *d,
> +                            gfn_t gfn,
> +                            mfn_t mfn,
> +                            unsigned long page_order,
> +                            p2m_type_t t)

Indentation isn't quite right here.

I'll see about dealing with those while committing.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:18:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:18:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679646.1057203 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZY4k-0005PL-0V; Mon, 12 Feb 2024 15:18:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679646.1057203; Mon, 12 Feb 2024 15:18:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZY4j-0005PE-U0; Mon, 12 Feb 2024 15:18:29 +0000
Received: by outflank-mailman (input) for mailman id 679646;
 Mon, 12 Feb 2024 15:18:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZY4i-0005P6-VW
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:18:28 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f8fb6ba9-c9b9-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 16:18:28 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-4108cbd92b9so14961555e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:18:28 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 cc19-20020a5d5c13000000b0033b728190c1sm6495099wrb.79.2024.02.12.07.18.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:18:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f8fb6ba9-c9b9-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707751107; x=1708355907; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0Ggy6Dnpfi6Dgb+gTBMGbmPZ/TYNLlwNkwiEMz341IM=;
        b=c6MJ5A5ZMvulhk8oLFL51IobS5AhERtcFcSbl5rHR/LkDqmej29CEp1GHc78mPtj7r
         tph4RAiblaGLuMcsGhczPlXwu4X7ygQHE+RoaqBBQvLRdqPejjtr0IkJvtZkcd7b3w1V
         rNweIQ7xJvHg21RIaZ1mYD8J5bvsupCrbHH4Ven2JI1/ccmEA4Yns5PGliIngg2i8ycE
         8lisViL++wd8T8FmhbyPoK1RpWO9pTEJiuFVvEiRkZ+m+EDobC/oV6uMyMq6hY+9HbRh
         qcAPjth+ih8K0Y99EF/wmTUjXR2XBOzbiWgLFsiIgKACrtx09rxhP/0Q9K+nluViVR34
         Xb5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707751107; x=1708355907;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0Ggy6Dnpfi6Dgb+gTBMGbmPZ/TYNLlwNkwiEMz341IM=;
        b=Ctf4irIXSnP8IJb+wxGSkoAlI5uzGZ18jzJx5NMHAVdYk5/UDaYY0oyz6Unor9U6Bb
         1M1U97uVjhoFa8Cs5QiE1sYTq7GSvBfFa0cOf5FnESngYTfJXmPcovUG8CHAd+FEuCtH
         JejGrbpZ2rWQ15Efk8i5k+WerZ91l8Mkirxh84Ce2Fg7Db2eS3sAbhfIKVUgswEFw19M
         tYuouGWCAIQbzAdXuIxT+vT7YOpa4dXm8xr7A64HX2EeLAIOkX0T6u8m2vlxC5NJzya+
         cKT0IrSPrFY+QM3ufq0fv+L5ID9VKbxXooOkmllbvCoL5uC2ebvNUD/hVqnj0WnU/vYg
         RM2g==
X-Gm-Message-State: AOJu0YxHRN1QAm0Ou6//cxv67/3W8d2tB+XIoVaTTj3/PyUd7eMcpVFD
	FjYRLbR2n48uOj/YO/4te/pO06kMYTNUqiQVAFr5uZvgVM22osZdBjbZQ9OTDA==
X-Google-Smtp-Source: AGHT+IHwugeM2iLrEzHqBY7Xru4PyBDGo+16Xh99k9svqexfzulWmBNOmbLkVNVh0HZ0tV8Qtzkpyg==
X-Received: by 2002:a05:600c:34d5:b0:410:cd66:e2ec with SMTP id d21-20020a05600c34d500b00410cd66e2ecmr2618394wmq.10.1707751107461;
        Mon, 12 Feb 2024 07:18:27 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVGMWfSFxSy0gbEyiXdz0laXnAdETi0GG+Mp9rnNqzilSnAjLq/zySJGkeDxOLR/rdtOmRzf8KPe9qTONFozlbtfsWFIVxK9r+YkM275GpdcCEFmfbrUrE50lcbUstYyk4BTvs18GxC9NWpHy8XqpWAgQ+ojWu0KBuPfT2fObF6rsiSiXQU+GPGi8m3JC1hF2jmvqeHA6A6ZkcSrQKzQOkTqDn/1JYUW8lDwNzOmUTJgSERs068VjApygz7/tbezicxq0oKy0TnjnZzIDr7FuWrw0M+6JmXRopo+gmHoWPwrDROJRXkpbk=
Message-ID: <2962005e-252f-453e-a8fe-25856b16fe57@suse.com>
Date: Mon, 12 Feb 2024 16:18:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 18/30] xen/riscv: introduce time.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <f0a4566c185469fa9f5de41b5bbbc7234cfa9912.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f0a4566c185469fa9f5de41b5bbbc7234cfa9912.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>

Nevertheless ...

> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/time.h
> @@ -0,0 +1,29 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef __ASM_RISCV_TIME_H__
> +#define __ASM_RISCV_TIME_H__
> +
> +#include <xen/bug.h>
> +#include <asm/csr.h>
> +
> +struct vcpu;
> +
> +/* TODO: implement */
> +static inline void force_update_vcpu_system_time(struct vcpu *v) { BUG_ON("unimplemented"); }

... nit: Too long line. The comment also doesn't look to serve any purpose
anymore, with the BUG_ON() now taking uniform shape.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:21:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:21:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679649.1057214 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZY7D-0006ql-DX; Mon, 12 Feb 2024 15:21:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679649.1057214; Mon, 12 Feb 2024 15:21:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZY7D-0006qe-Az; Mon, 12 Feb 2024 15:21:03 +0000
Received: by outflank-mailman (input) for mailman id 679649;
 Mon, 12 Feb 2024 15:21:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZY7C-0006qY-RP
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:21:02 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 53c0862c-c9ba-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 16:21:00 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-4114e0a2978so3776785e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:21:00 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v9-20020a5d4b09000000b0033b843786e1sm2304795wrq.51.2024.02.12.07.20.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:20:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 53c0862c-c9ba-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707751260; x=1708356060; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=5sGDdGFbGFwzm+aPUvtN/8Pp/keIpjg6C/TDKw8U/4Q=;
        b=O0A/kBg05tF5jo6jPRTw+mTfNRw6HwOV7v4eVPdqi5JlliiSeEC9EluirvpLqsMmTI
         ye1amG7Gf/9B1RYVxOU44d89hYuejgYB+e5Xq9JJ4xMFxXglPsLQQq4APUw8t6EMmQFo
         yP1a9PtBDRgIhDCl1JIGxjiJk3VEu4WalvQOimYaS0KS+Ajqk8cSu7RQ6Z/dBwhK2Ira
         elPS6PfsSyv0OEQZeT51g1SWIPFTjpQkTE+Dn7psBRDalAIn/aqX2TzZeDwqFU6INOZ/
         ZWcbtWcHhzfbKLhJ4HB6A2Zdq+LpWWUrwUYKu/sM34o1Z5RBPQcqczc7mMUpRksKvcMM
         kgvw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707751260; x=1708356060;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5sGDdGFbGFwzm+aPUvtN/8Pp/keIpjg6C/TDKw8U/4Q=;
        b=EvnDIswk9vwMos3Ye2DoiCYCLyk4k02OzoUSqVMH5RWted8FpJwf5zHzsiYb9G1Ss9
         f2oMqt/Cc5rfw8foU16IpHmD4eQN3Qoq6tu084T8eL7lUO5/J2gDOlr7OyD47Gx2wMyE
         /GQi+gLg/0X1ZDIQBm+TKfSGY0Fltd4hwyYxpeoF5+nPANi/FpVXVEq8Z+yoQGnim/b5
         9a51L+9FcHy+npB3l5djWkDIzP3cxWG34GeSigBXnx2tHxXwUlPuTP5ILvinzKc2NQZw
         pId3IsYpY1MLoFOC+xBL2HFRIIMWQIfNz0+1AniqCcGOQlJLqHPnA1foTJcfcBj9sF/l
         dROg==
X-Gm-Message-State: AOJu0YzXmj+huHE49tdI1diw7q6ZROXAWQjdnjsfOzUsy3W++ARN9+Kg
	7R0bcz44JDONBtsgkFD4GbSfTkgT3Pk7prkMuK+jwfAScwH20q1O/kjQSGHQGQ==
X-Google-Smtp-Source: AGHT+IFJvvuu18s0cMR+Dh9YvdxR8HezWBwfu/RQAsKhGjyLjOlWmH8Et4IuTx+TGBXArGAylkJt0Q==
X-Received: by 2002:a05:600c:4e92:b0:40e:c06a:3ed5 with SMTP id f18-20020a05600c4e9200b0040ec06a3ed5mr6721973wmq.2.1707751259776;
        Mon, 12 Feb 2024 07:20:59 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUuJF3n4iEsUd1ZEMa2hAJT/cz79uCX9B82+0d6zQRG6fxobMOOWDOEJVbFetfExG5kiNW1pAEzzUPXR2jK0nzCcmmqLIBFJnD/rqf1cN+l0if0SYv3hApaPcaz8krq79wSebv6wBINxJ56U1Gxp0nbBDWuGlP1dl+Ex9hvM8OfvTy1dqApJe4DsjMvwc2ytgX5sR0PIN1TiUSadbZcay3XD5snomaFWM33TmPyqjJipI2U0KAKv5Ur4nhpjI7mh0kV6GOrmX76NVFHPJbNJNu9kbG71R8omEv1toNL/Bk4kjhIX1rkDxc=
Message-ID: <36b25710-cf95-4250-bf4a-6ccf4f418909@suse.com>
Date: Mon, 12 Feb 2024 16:20:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 19/30] xen/riscv: introduce event.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <a57aed8d2d8424c63b1f27d007b2d5cfe35460cc.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a57aed8d2d8424c63b1f27d007b2d5cfe35460cc.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
again with a nit, though:

> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/event.h
> @@ -0,0 +1,40 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef __ASM_RISCV_EVENT_H__
> +#define __ASM_RISCV_EVENT_H__
> +
> +#include <xen/lib.h>
> +
> +void vcpu_mark_events_pending(struct vcpu *v);
> +
> +static inline int vcpu_event_delivery_is_enabled(struct vcpu *v)
> +{
> +    BUG_ON("unimplemented");
> +    return 0;
> +}
> +
> +static inline int local_events_need_delivery(void)
> +{
> +    BUG_ON("unimplemented");
> +    return 0;
> +}
> +
> +static inline void local_event_delivery_enable(void)
> +{
> +    BUG_ON("unimplemented");
> +}
> +
> +/* No arch specific virq definition now. Default to global. */
> +static inline bool arch_virq_is_global(unsigned int virq)
> +{
> +    return true;
> +}
> +
> +#endif

This want to gain the usual comment.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:25:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:25:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679655.1057224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZYB4-0007lB-S6; Mon, 12 Feb 2024 15:25:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679655.1057224; Mon, 12 Feb 2024 15:25:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZYB4-0007l3-PS; Mon, 12 Feb 2024 15:25:02 +0000
Received: by outflank-mailman (input) for mailman id 679655;
 Mon, 12 Feb 2024 15:25:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZYB3-0007jg-FH
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:25:01 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e2e9852b-c9ba-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 16:25:00 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-3392b045e0aso2299551f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:25:00 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bw28-20020a0560001f9c00b0033b66ce7ae9sm7150916wrb.84.2024.02.12.07.24.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:24:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e2e9852b-c9ba-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707751500; x=1708356300; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0m91h5Pd6oFyMd+ETMzjc1rjEXaw9O+0243geydUybA=;
        b=gj5sk/+3KeUNU2/Qh+0Zy8Zj3tcgAt9N+pgy7qJE1BM+ivM1YuBul9j+OMAUrXPYBP
         YQkUqESP6fPFaZa4qGG8zVY7frFziwzweAfzj1DUuZBqknO2VKtfxSF8mUb976EgpoMt
         a5ndPYeljShNQGx7+iwPa4mPKam97GkOu/12bLKV87wwbdBB2hlJjEa5zoRHQG0hFZ+d
         bJLE9jcpgPhadxK2+b9R0tkeXFzisSj/PB6siPnq+RLrau9KJog45rApcwDJ7guJcQ4N
         /p7k6WtctVXfigxnICRvacqk/uDa/0TU2u98n0YlIE480KzBv1gqflI30/9jPMVW9yWJ
         LI0A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707751500; x=1708356300;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0m91h5Pd6oFyMd+ETMzjc1rjEXaw9O+0243geydUybA=;
        b=crunKnMkxm0vAhAmHLmnBvytbQ8HoG9w91A/ek1SQaeJVNqdT2Vo9cK7Uu2uuH8/jM
         8WEvcg5AotmrZXCh23DIA0117hjXGmCniza1PDY5A5L7U64D2lYcxkkdAaXTxMU663E2
         R6GfWAYHaicLwxRsWivcjjhRroeEPKPu24NefQah7Ieq1XMk9ElEZiicuZqvGY5ZlSO9
         1ZU5PSUyP2747wSonEmjZLAWePa78cegKactX+1exPhnflL5KTlD63jr1+nG120w297u
         W0/1J21/okTiVvEuAHv0yIZEVMEY4IsFjQfdsOhaO3LFoFIjpG+1n4hKQEccVJDzLeKl
         khiQ==
X-Forwarded-Encrypted: i=1; AJvYcCVlfYM4tcEx7xAY0LTiwboTumhJ7ulvvuwEEGuCn1JsYW6hjbmexiMpDcBQG3OMNsnJyWDbllOL1VqltqqviB/6+k2nB5ZQSq/t4ttGgJA=
X-Gm-Message-State: AOJu0YwM8wXB6ZwTDhsYq+MtAe3s0VJB14cWX3xO70ndGZJKWi+zT4Cd
	5XoC3YPLbiLpIqFxS/LPU3cm+P0LVrJXo9p9c9PIKizT2CzTPj2MuPVc0Coe+w==
X-Google-Smtp-Source: AGHT+IENHIfHqkyoz+xsHs5lihxLeI+65XdaV7IqidpXCp2fefSuF9fD7lm3ldCtP+j4l2R95V8S1w==
X-Received: by 2002:adf:ea48:0:b0:33b:26d7:e6dd with SMTP id j8-20020adfea48000000b0033b26d7e6ddmr5262955wrn.48.1707751499926;
        Mon, 12 Feb 2024 07:24:59 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWfeB7wDDaFzaUwEgxFK9wkwtgDdzGbB2ulZvcAGubdtD+uqnbNQ7Se9upKACT0Mq26JUe4gCf8HTB0AA6KyQgYmLOWvwHxAIJ0EAil+wP9rMrBrWZJ3//R3ZIycZO9v8HherSJOqqtklpAuq2/UAaHr/wUBZwJ+/IgMhXUR49MnMiLQC+NcU85Rc3tJRNm4HkSlfG4jILPF5n2g225oIMVKuAfTFgE+M/UD3Z4PTEk3yjKAOPDNn0W9e4OkH8pnXtzdsuKPru5XPEmh53ZZ/Xc6NkhF9cdgJ2o9L9cl/RPfe2/xbRdP40=
Message-ID: <cb20dcd4-ff74-46ea-aed2-8a19fbe17cd5@suse.com>
Date: Mon, 12 Feb 2024 16:24:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 28/30] xen/rirscv: add minimal amount of stubs to build
 full Xen
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <e25d55d1cb95c1b2b8ad0b4eae9794b5b6f84075.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e25d55d1cb95c1b2b8ad0b4eae9794b5b6f84075.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/early_printk.c
> +++ b/xen/arch/riscv/early_printk.c
> @@ -207,4 +207,3 @@ void printk(const char *format, ...)
>  }
>  
>  #endif
> -

Unrelated change?

> --- a/xen/arch/riscv/traps.c
> +++ b/xen/arch/riscv/traps.c
> @@ -4,6 +4,10 @@
>   *
>   * RISC-V Trap handlers
>   */
> +
> +#include <xen/lib.h>
> +#include <xen/sched.h>
> +
>  #include <asm/processor.h>
>  #include <asm/traps.h>
>  
> @@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs)
>  {
>      die();
>  }
> +
> +void vcpu_show_execution_state(struct vcpu *v)
> +{
> +    assert_failed("need to be implented");
> +}
> +
> +void show_execution_state(const struct cpu_user_regs *regs)
> +{
> +    printk("implement show_execution_state(regs)\n");
> +}
> +
> +void arch_hypercall_tasklet_result(struct vcpu *v, long res)
> +{
> +    assert_failed("need to be implented");
> +}
> +
> +enum mc_disposition arch_do_multicall_call(struct mc_state *state)
> +{
> +    assert_failed("need to be implented");
> +    return mc_continue;
> +}

The assert_failed() here want switching to the "canonical" BUG_ON().

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:27:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:27:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679658.1057233 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZYD1-0000GC-6U; Mon, 12 Feb 2024 15:27:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679658.1057233; Mon, 12 Feb 2024 15:27:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZYD1-0000G5-3t; Mon, 12 Feb 2024 15:27:03 +0000
Received: by outflank-mailman (input) for mailman id 679658;
 Mon, 12 Feb 2024 15:27:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=JouK=JV=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rZYD0-0000Fx-0U
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:27:02 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2ab221d5-c9bb-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 16:27:01 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 1B77B22198;
 Mon, 12 Feb 2024 15:27:00 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DB71213985;
 Mon, 12 Feb 2024 15:26:59 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id ir+0M8M4ymUpYwAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 12 Feb 2024 15:26:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ab221d5-c9bb-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707751620; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=JDnbsSmLRToMshpOSrPEVaIi1UWXzwXyk73ONXdsI50=;
	b=nDr1bsb0Iu16Uh7TCzio89xgas3oPSoVyfbmEZr2KyPVZTcCvAyb5hPgyjoeSDsZrA96wn
	yESU4olWRcmqBgOpaPGfjH+k3EWRCMjznxvgsaOCpOKeIbm3vf1291c1keDOnt/XhRBL4C
	j73JGeDrpmhr8LGtZMHcI7QPlMEGg9Y=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707751620; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=JDnbsSmLRToMshpOSrPEVaIi1UWXzwXyk73ONXdsI50=;
	b=nDr1bsb0Iu16Uh7TCzio89xgas3oPSoVyfbmEZr2KyPVZTcCvAyb5hPgyjoeSDsZrA96wn
	yESU4olWRcmqBgOpaPGfjH+k3EWRCMjznxvgsaOCpOKeIbm3vf1291c1keDOnt/XhRBL4C
	j73JGeDrpmhr8LGtZMHcI7QPlMEGg9Y=
Message-ID: <b6652732-d408-4ec9-bcf3-fdf7f84dd00a@suse.com>
Date: Mon, 12 Feb 2024 16:26:59 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/xenbus: document will_handle argument for
 xenbus_watch_path()
Content-Language: en-US
To: SeongJae Park <sj@kernel.org>
Cc: sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org,
 kernel test robot <lkp@intel.com>
References: <20240112185903.83737-1-sj@kernel.org>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <20240112185903.83737-1-sj@kernel.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------gLgkIRd1a9czmCD5XBFjNM0S"
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.36
X-Spamd-Result: default: False [-3.36 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-1.18)[88.96%];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 HAS_ATTACHMENT(0.00)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 MIME_BASE64_TEXT_BOGUS(1.00)[];
	 RCPT_COUNT_FIVE(0.00)[6];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-0.982];
	 MIME_BASE64_TEXT(0.10)[];
	 SIGNED_PGP(-2.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[];
	 MIME_UNKNOWN(0.10)[application/pgp-keys]
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------gLgkIRd1a9czmCD5XBFjNM0S
Content-Type: multipart/mixed; boundary="------------gl8sQ4R0va4tfmtD2baX32Ek";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: SeongJae Park <sj@kernel.org>
Cc: sstabellini@kernel.org, oleksandr_tyshchenko@epam.com,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org,
 kernel test robot <lkp@intel.com>
Message-ID: <b6652732-d408-4ec9-bcf3-fdf7f84dd00a@suse.com>
Subject: Re: [PATCH] xen/xenbus: document will_handle argument for
 xenbus_watch_path()
References: <20240112185903.83737-1-sj@kernel.org>
In-Reply-To: <20240112185903.83737-1-sj@kernel.org>

--------------gl8sQ4R0va4tfmtD2baX32Ek
Content-Type: multipart/mixed; boundary="------------4Q4sVHUfGM1suk8NTO0UN9cB"

--------------4Q4sVHUfGM1suk8NTO0UN9cB
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMTIuMDEuMjQgMTk6NTksIFNlb25nSmFlIFBhcmsgd3JvdGU6DQo+IENvbW1pdCAyZTg1
ZDMyYjFjODYgKCJ4ZW4veGVuYnVzOiBBZGQgJ3dpbGxfaGFuZGxlJyBjYWxsYmFjayBzdXBw
b3J0IGluDQo+IHhlbmJ1c193YXRjaF9wYXRoKCkiKSBhZGRlZCB3aWxsX2hhbmRsZSBhcmd1
bWVudCB0byB4ZW5idXNfd2F0Y2hfcGF0aCgpDQo+IGFuZCBpdHMgd3JhcHBlciwgeGVuYnVz
X3dhdGNoX3BhdGhmbXQoKSwgYnV0IGRpZG4ndCBkb2N1bWVudCBpdCBvbiB0aGUNCj4ga2Vy
bmVsZG9jIGNvbW1lbnRzIG9mIHRoZSBmdW5jdGlvbi4gIFRoaXMgaXMgY2F1c2luZyB3YXJu
aW5ncyB0aGF0DQo+IHJlcG9ydGVkIGJ5IGtlcm5lbCB0ZXN0IHJvYm90LiAgQWRkIHRoZSBk
b2N1bWVudGF0aW9uIHRvIGZpeCBpdC4NCj4gDQo+IEZpeGVzOiAyZTg1ZDMyYjFjODYgKCJ4
ZW4veGVuYnVzOiBBZGQgJ3dpbGxfaGFuZGxlJyBjYWxsYmFjayBzdXBwb3J0IGluIHhlbmJ1
c193YXRjaF9wYXRoKCkiKQ0KPiBSZXBvcnRlZC1ieToga2VybmVsIHRlc3Qgcm9ib3QgPGxr
cEBpbnRlbC5jb20+DQo+IENsb3NlczogaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcvb2Uta2J1
aWxkLWFsbC8yMDI0MDExMjExNTQuRkk4akRHdW4tbGtwQGludGVsLmNvbS8NCj4gU2lnbmVk
LW9mZi1ieTogU2VvbmdKYWUgUGFyayA8c2pAa2VybmVsLm9yZz4NCg0KUmV2aWV3ZWQtYnk6
IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT4NCg0KDQpKdWVyZ2VuDQo=
--------------4Q4sVHUfGM1suk8NTO0UN9cB
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------4Q4sVHUfGM1suk8NTO0UN9cB--

--------------gl8sQ4R0va4tfmtD2baX32Ek--

--------------gLgkIRd1a9czmCD5XBFjNM0S
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXKOMMFAwAAAAAACgkQsN6d1ii/Ey9n
Sgf+NJ2PWK5T2ouqRnkoHicpulBdtJr+OvcxkqfQlW/OXRHmjq4yhnDByxnSvV7z+BaB/KrgD8jJ
grkzED+/oTrEWRpPmJek8hc4co/lWwssV9lEnOodqhIhc1YOXFC5rsHn2SgrjBFozsj3QyHqpHX3
uDhvqMsMbtGIIPTRUR3cNL5AOj5kmkb3Ll2ZZvaK4rqUU3Ov/lVHczmK1gTHJPmCmBy//XQfyise
tQXQbgQ19Je8rfqQyXswIJLbUAreMBi+xZlOAHoB/cOJQ5u60bUCnIq6HZv7tu0B/GjZ7bdC5Wp+
m9fqvukRujv/A9Z8PH6TnNMhkBGuGEj5awP03/Z6gw==
=ITPR
-----END PGP SIGNATURE-----

--------------gLgkIRd1a9czmCD5XBFjNM0S--


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:38:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:38:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679666.1057243 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZYOL-0002sn-9M; Mon, 12 Feb 2024 15:38:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679666.1057243; Mon, 12 Feb 2024 15:38:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZYOL-0002sg-6e; Mon, 12 Feb 2024 15:38:45 +0000
Received: by outflank-mailman (input) for mailman id 679666;
 Mon, 12 Feb 2024 15:38:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=IbWo=JV=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rZYOK-0002sX-54
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:38:44 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cb9b5521-c9bc-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 16:38:40 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-410db155e57so6696735e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:38:40 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 je11-20020a05600c1f8b00b00410885ba8casm7714743wmb.39.2024.02.12.07.38.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Feb 2024 07:38:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb9b5521-c9bc-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707752320; x=1708357120; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=sN0aDPuErzB0rLLjDfP31AhY1wWx/5dUNIrbT7KkRJw=;
        b=CT9xRslVu2v3E+DosIo9bwvEKaiYZu6kLNWLEVTrU+z4HHzfaALMUFnIcnfgwiZrpE
         y4jfbFEy166xN2FrO3I9u758IDGwJNrRqkYRId+KcjQCfP9lDGsU0wM/p6Pe4Q5REO3w
         YV11hWGQESY6C7kEJo8jcc+KEDaNEN+nDcjNw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707752320; x=1708357120;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sN0aDPuErzB0rLLjDfP31AhY1wWx/5dUNIrbT7KkRJw=;
        b=PheIxaHGsTF66MsC94DLVzcNHhwk7vgqL2GFZ1JtJkpDRFOfnz7vcwaf28gTYjS7zO
         xROp9YY5PWcs13HdmefGTVfeNMU0dItkjThNI5kBJMEiAFkgIUL1ENQPVZmrKofVAGql
         Kq2V4p2IMviaBDqiXG9kQzptT6fJEjth5Sxr+BGRNYmZ0MrFdfbNqLwcnR32GrMKzOLU
         UyZR/jZFb+QN5UX/d8zVD+kTBxJx5+x3cj/lI83OUJK0upQBGhx4WZfMO0WPUMSh7D6f
         BQ1nYIPaLXrsdnL0M3ESOMuh+MYi9PctUJL6I1/RnzDxysM7CpTpQ22dmxFGrHWVz4pa
         z5ZQ==
X-Gm-Message-State: AOJu0YwISjfFz+oUkGjwcDZBKPjLVGUPGYUS84kYNO8mGCI5H0r3NcTQ
	LaLSINdjUcEYBF3Bj8vK5hp3N3Pi87K+zW75jq3FOt139X10J7Y6WCMs5imq2Gp83+3T1xke0sL
	x
X-Google-Smtp-Source: AGHT+IF+J3FtfNQv4zbYmq83MibN1zsuNiyYiDAmjrf4dQuGKs+g0k9GzS85PPU8eILbUVGUdRCReQ==
X-Received: by 2002:a05:600c:c12:b0:410:a138:7447 with SMTP id fm18-20020a05600c0c1200b00410a1387447mr5093101wmb.27.1707752319800;
        Mon, 12 Feb 2024 07:38:39 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCV+J78BSTrMzJQSAXwiU/iguBJkeBo6HhU5EJU/na9F2LPoPPEZvHZKql/Xen3EdnS6ckTGHtpZMXfKuuEEsLl/GIXtepWiHqOCP9xU3Qg14RCN0xfFSX8EBUC+Q7lSP91pYsAuGYVDqW4RpTQq
Date: Mon, 12 Feb 2024 16:38:38 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH 2/7] IOMMU: rename and re-type ats_enabled
Message-ID: <Zco7fvp1nPVlLtRy@macbook>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <467d24e1-8ed3-4dda-a334-70ff747bf94b@suse.com>
 <ZcS_uC2t96Lh720Y@macbook>
 <4179ded9-36b8-4d47-997b-40d056e033af@suse.com>
 <ZcnnautuDD2qluQm@macbook>
 <2fccd004-b35a-4cde-afbb-722cb2413902@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <2fccd004-b35a-4cde-afbb-722cb2413902@suse.com>

On Mon, Feb 12, 2024 at 11:45:33AM +0100, Jan Beulich wrote:
> On 12.02.2024 10:39, Roger Pau Monné wrote:
> > On Thu, Feb 08, 2024 at 04:49:46PM +0100, Jan Beulich wrote:
> >> On 08.02.2024 12:49, Roger Pau Monné wrote:
> >>> On Mon, Feb 05, 2024 at 02:55:43PM +0100, Jan Beulich wrote:
> >>>> Make the variable a tristate, with (as done elsewhere) a negative value
> >>>> meaning "default". Since all use sites need looking at, also rename it
> >>>> to match our usual "opt_*" pattern. While touching it, also move it to
> >>>> .data.ro_after_init.
> >>>>
> >>>> The only place it retains boolean nature is pci_ats_device(), for now.
> >>>
> >>> Why does it retain the boolean nature in pci_ats_device()?
> >>>
> >>> I assume this is to avoid having to touch the line again in a further
> >>> patch, as given the current logic pci_ats_device() would also want to
> >>> treat -1 as ATS disabled.
> >>
> >> No, then I would need to touch the line. The function wants to treat
> >> -1 as "maybe enabled", so the caller can know whether a device is an
> >> ATS device regardless of whether ATS use is fully off, or only
> >> "soft-off".
> > 
> > I have to admit I'm slightly concerned about this soft-off.  Given the
> > current status of ATS itself in Xen, and the technology itself, I
> > think a user should always opt-in to ATS usage.
> 
> The plan is to follow your suggestion in patch 3 and require explicit
> enabling for passing through of such devices. For Dom0, however, I
> think it is important that we respect the firmware request by default.
> The only viable(?!) alternative would be to panic() instead.

Or assign to domIO?

> >>>> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> >>>> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> >>>> @@ -185,10 +185,11 @@ static int __must_check amd_iommu_setup_
> >>>>          dte->ex = ivrs_dev->dte_allow_exclusion;
> >>>>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
> >>>>  
> >>>> -        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> >>>> +        if ( opt_ats > 0 &&
> >>>>               !ivrs_dev->block_ats &&
> >>>> -             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
> >>>> -            dte->i = ats_enabled;
> >>>> +             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> >>>> +             pci_ats_device(iommu->seg, bus, pdev->devfn) )
> >>>> +            dte->i = true;
> >>>>  
> >>>>          spin_unlock_irqrestore(&iommu->lock, flags);
> >>>>  
> >>>> @@ -248,10 +249,11 @@ static int __must_check amd_iommu_setup_
> >>>>          ASSERT(dte->sys_mgt == MASK_EXTR(ivrs_dev->device_flags,
> >>>>                                           ACPI_IVHD_SYSTEM_MGMT));
> >>>>  
> >>>> -        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> >>>> +        if ( opt_ats > 0 &&
> >>>>               !ivrs_dev->block_ats &&
> >>>> -             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
> >>>> -            ASSERT(dte->i == ats_enabled);
> >>>> +             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> >>>> +             pci_ats_device(iommu->seg, bus, pdev->devfn) )
> >>>> +            ASSERT(dte->i);
> >>>>  
> >>>>          spin_unlock_irqrestore(&iommu->lock, flags);
> >>>>  
> >>>> @@ -268,9 +270,10 @@ static int __must_check amd_iommu_setup_
> >>>>  
> >>>>      ASSERT(pcidevs_locked());
> >>>>  
> >>>> -    if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> >>>> +    if ( opt_ats > 0 &&
> >>>>           !ivrs_dev->block_ats &&
> >>>>           iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> >>>> +         pci_ats_device(iommu->seg, bus, pdev->devfn) &&
> >>>>           !pci_ats_enabled(iommu->seg, bus, pdev->devfn) )
> >>>
> >>> Seeing that this same set of conditions is used in 3 different checks,
> >>> could we add a wrapper for it?
> >>>
> >>> opt_ats > 0 && !ivrs_dev->block_ats &&
> >>> iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
> >>> pci_ats_device(iommu->seg, bus, pdev->devfn)
> >>>
> >>> pci_device_ats_capable()? or some such.
> >>
> >> I was pondering that, yes (iirc already once when adding block_ats).
> >> Problem is the name. "capable" isn't quite right when considering
> >> the tristate opt_ats. And pci_device_may_use_ats() reads, well,
> >> clumsy to me. If you have any good idea for a name that's fully
> >> applicable and not odd or overly long, I can certainly introduce
> >> such a helper.
> > 
> > But if ATS is soft-disabled (-1) or hard disabled (0), it's fine to
> > consider the devices as not ATS capable for the context here?
> 
> I don't like mixing capability and policy aspects into a resulting
> "capable".

IMO we should prefer avoiding code repetition, even if at the cost
of having a handler that have a maybe not ideal naming, but I can't
force you to do that.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:58:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:58:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679671.1057253 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZYhB-000723-Re; Mon, 12 Feb 2024 15:58:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679671.1057253; Mon, 12 Feb 2024 15:58:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZYhB-00071w-OW; Mon, 12 Feb 2024 15:58:13 +0000
Received: by outflank-mailman (input) for mailman id 679671;
 Mon, 12 Feb 2024 15:58:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZYhB-00071p-CA
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:58:13 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8542707e-c9bf-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 16:58:10 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-410cb93185dso9300315e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:58:10 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f5-20020a5d58e5000000b0033b44456463sm7107513wrd.106.2024.02.12.07.58.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:58:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8542707e-c9bf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707753490; x=1708358290; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dxyytBCybEJ9xwwyC0iUeDNb8txuTqOL2xSCNBB8rOo=;
        b=FbH9dB81LHPFuLjsandLrCc6xujj42xiZMRkt4PF2kL/rVYgovguWUP4lYWi5mis7C
         CobMyQyklcdblmL7gVtsTqK/MGjTmU+/WUFNwkBmPRDyQp6/HyDyXE8KZ51f7OLyDTzv
         AD0t+zvgvX2s0ueCOEuticJzOmXE2+UJFJ72B3s+yFEd7Oe30R4NrpEC6AdP9nfnO31a
         YnglSN7jQUIUnkM+Hnr+2AreWNxBSeLRLJqlE3LkO2CqQo7aERE0wQOxl/m1ZPEwg6CL
         VEPnEPSm+a4ixb5RI87r/bHiTRcyFJtk/gvHapzRShgGrBnZenTjg94TTIoT9kRxei13
         XERQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707753490; x=1708358290;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dxyytBCybEJ9xwwyC0iUeDNb8txuTqOL2xSCNBB8rOo=;
        b=nSA6cFSUvGC9CDKdzvXs9/ZR3ZUp64vEcMfcB4TneXMRf5IjF8J6PjAurWygwKi99T
         fMFB+ow4DAzvZ2KbjnJrm7iGrNKsQy0yvkB8qNs1Be0J12/CvZ/ob1XCs20xgkSf1pzF
         FOjW8ny3uf1z5qoZ1bvRsr8l5D09QDNT75YblP7p0ugF5Lj1/vc20jT3g4I/IjO+GqbM
         8V68vO0+7OG445VA+TGdIfpv5GP8f93XZkGn1gPRcKM42AqaOVUQTZ/v/gYCqb/W1D6r
         5oI9qfIszp+JwgLux509lphQ+EjLHZ23whynREGvn1fVYlgChXrpKwnGJNyZ93fsu3rt
         BcWQ==
X-Gm-Message-State: AOJu0Yz4n6xYK/UYp7YOuLa9uYBhgilkv4CAuLohN7YaBRJvsMh6O9TQ
	ywZ90N8gYV47UeSjuDVgsAzX0RUvtoZklcSuj77Zxe8gsrYd0pj+XQE65oxKUA==
X-Google-Smtp-Source: AGHT+IHWg0jS2UcCDrCg5lV5iPqpp0sgSYV32ELxcgnNGrp6LTvsrTyRgRJWA2idT/sw70wCNM6gkQ==
X-Received: by 2002:a5d:4485:0:b0:33b:3275:2721 with SMTP id j5-20020a5d4485000000b0033b32752721mr5470532wrq.40.1707753490091;
        Mon, 12 Feb 2024 07:58:10 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWdtcBZyUzsTeFDiAuEO3DU+heyss5o3FtycotVQHvFBPKmDNKp+EB4Rr/blNR5hc8/SI///WNKG6XQxFoa8LzucGlfygFITNX5Ny4xXxK3ebQQ7ErmQ51QR5gd5rYBXWxDw2nrGD4zMfGVl4tHE2ltCvbBa8RZ4vq+54UeU2LIBx98qlJX1P4efLbUb4I/BUJrhxLyH+WBjZYdj47mL6RXQvNnDtjGIu7w9C4vnn+LAvpUWXFX5mR8M429qCQW/+Z5Y944xvmBJNDvzudB1lVW1jJUefRX9vW7ifoRgyJocQZsratqAkA=
Message-ID: <b13fd044-c6db-45ab-83d0-3d02221967ed@suse.com>
Date: Mon, 12 Feb 2024 16:58:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 09/30] xen/riscv: introduce bitops.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <f76bd85f4b64a47c59c0b306ce425036819fa380.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f76bd85f4b64a47c59c0b306ce425036819fa380.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/bitops.h
> @@ -0,0 +1,164 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/* Copyright (C) 2012 Regents of the University of California */
> +
> +#ifndef _ASM_RISCV_BITOPS_H
> +#define _ASM_RISCV_BITOPS_H
> +
> +#include <asm/system.h>
> +
> +#include <asm-generic/bitops/bitops-bits.h>

Especially with ...

> +/* Based on linux/arch/include/linux/bits.h */
> +
> +#define BIT_MASK(nr)        (1UL << ((nr) % BITS_PER_LONG))
> +#define BIT_WORD(nr)        ((nr) / BITS_PER_LONG)

... these it's not entirely obvious why bitops-bits.h would be needed
here.

> +#define __set_bit(n,p)      set_bit(n,p)
> +#define __clear_bit(n,p)    clear_bit(n,p)

Nit (as before?): Missing blanks after commas.

> +/* Based on linux/arch/include/asm/bitops.h */
> +
> +#if ( BITS_PER_LONG == 64 )

Imo the parentheses here make things only harder to read.

> +#define __AMO(op)   "amo" #op ".d"
> +#elif ( BITS_PER_LONG == 32 )
> +#define __AMO(op)   "amo" #op ".w"
> +#else
> +#error "Unexpected BITS_PER_LONG"
> +#endif
> +
> +#define __test_and_op_bit_ord(op, mod, nr, addr, ord)   \

The revision log says __test_and_* were renamed. Same anomaly for
__test_and_op_bit() then.

> +({                                                      \
> +    unsigned long __res, __mask;                        \

Leftover leading underscores?

> +    __mask = BIT_MASK(nr);                              \
> +    __asm__ __volatile__ (                              \
> +        __AMO(op) #ord " %0, %2, %1"                    \
> +        : "=r" (__res), "+A" (addr[BIT_WORD(nr)])       \
> +        : "r" (mod(__mask))                             \
> +        : "memory");                                    \
> +    ((__res & __mask) != 0);                            \
> +})
> +
> +#define __op_bit_ord(op, mod, nr, addr, ord)    \
> +    __asm__ __volatile__ (                      \
> +        __AMO(op) #ord " zero, %1, %0"          \
> +        : "+A" (addr[BIT_WORD(nr)])             \
> +        : "r" (mod(BIT_MASK(nr)))               \
> +        : "memory");
> +
> +#define __test_and_op_bit(op, mod, nr, addr)    \
> +    __test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
> +#define __op_bit(op, mod, nr, addr) \
> +    __op_bit_ord(op, mod, nr, addr, )
> +
> +/* Bitmask modifiers */
> +#define __NOP(x)    (x)
> +#define __NOT(x)    (~(x))

Here the (double) leading underscores are truly worrying: Simple
names like this aren't impossible to be assigned meaninb by a compiler.

> +/**
> + * __test_and_set_bit - Set a bit and return its old value
> + * @nr: Bit to set
> + * @addr: Address to count from
> + *
> + * This operation may be reordered on other architectures than x86.
> + */
> +static inline int test_and_set_bit(int nr, volatile void *p)
> +{
> +    volatile uint32_t *addr = p;

With BIT_WORD() / BIT_MASK() being long-based, is the use of uint32_t
here actually correct?

> +    return __test_and_op_bit(or, __NOP, nr, addr);
> +}
> +
> +/**
> + * __test_and_clear_bit - Clear a bit and return its old value
> + * @nr: Bit to clear
> + * @addr: Address to count from
> + *
> + * This operation can be reordered on other architectures other than x86.

Nit: double "other" (and I think it's the 1st one that wants dropping,
not - as the earlier comment suggests - the 2nd one). Question is: Are
the comments correct? Both resolve to something which is (also) at
least a compiler barrier. Same concern also applies further down, to
at least set_bit() and clear_bit().

> + */
> +static inline int test_and_clear_bit(int nr, volatile void *p)
> +{
> +    volatile uint32_t *addr = p;
> +
> +    return __test_and_op_bit(and, __NOT, nr, addr);
> +}
> +
> +/**
> + * set_bit - Atomically set a bit in memory
> + * @nr: the bit to set
> + * @addr: the address to start counting from
> + *
> + * Note: there are no guarantees that this function will not be reordered
> + * on non x86 architectures, so if you are writing portable code,
> + * make sure not to rely on its reordering guarantees.
> + *
> + * Note that @nr may be almost arbitrarily large; this function is not
> + * restricted to acting on a single-word quantity.
> + */
> +static inline void set_bit(int nr, volatile void *p)
> +{
> +    volatile uint32_t *addr = p;
> +
> +    __op_bit(or, __NOP, nr, addr);
> +}
> +
> +/**
> + * clear_bit - Clears a bit in memory
> + * @nr: Bit to clear
> + * @addr: Address to start counting from
> + *
> + * Note: there are no guarantees that this function will not be reordered
> + * on non x86 architectures, so if you are writing portable code,
> + * make sure not to rely on its reordering guarantees.
> + */
> +static inline void clear_bit(int nr, volatile void *p)
> +{
> +    volatile uint32_t *addr = p;
> +
> +    __op_bit(and, __NOT, nr, addr);
> +}
> +
> +/**
> + * test_and_change_bit - Change a bit and return its old value

How come this one's different? I notice the comments are the same (and
hence as confusing) in Linux; are you sure they're applicable there?

> + * @nr: Bit to change
> + * @addr: Address to count from
> + *
> + * This operation is atomic and cannot be reordered.
> + * It also implies a memory barrier.
> + */
> +static inline int test_and_change_bit(int nr, volatile unsigned long *addr)
> +{
> +	return __test_and_op_bit(xor, __NOP, nr, addr);
> +}
> +
> +#undef __test_and_op_bit
> +#undef __op_bit
> +#undef __NOP
> +#undef __NOT
> +#undef __AMO
> +
> +#include <asm-generic/bitops/generic-non-atomic.h>
> +
> +#define __test_and_set_bit generic___test_and_set_bit
> +#define __test_and_clear_bit generic___test_and_clear_bit
> +#define __test_and_change_bit generic___test_and_change_bit
> +
> +#include <asm-generic/bitops/fls.h>
> +#include <asm-generic/bitops/flsl.h>
> +#include <asm-generic/bitops/__ffs.h>
> +#include <asm-generic/bitops/ffs.h>
> +#include <asm-generic/bitops/ffsl.h>
> +#include <asm-generic/bitops/ffz.h>
> +#include <asm-generic/bitops/find-first-set-bit.h>
> +#include <asm-generic/bitops/hweight.h>
> +#include <asm-generic/bitops/test-bit.h>

To be honest there's too much stuff being added here to asm-generic/,
all in one go. I'll see about commenting on the remaining parts here,
but I'd like to ask that you seriously consider splitting.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 15:59:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 15:59:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679673.1057264 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZYij-0007a9-4c; Mon, 12 Feb 2024 15:59:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679673.1057264; Mon, 12 Feb 2024 15:59:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZYij-0007a2-1m; Mon, 12 Feb 2024 15:59:49 +0000
Received: by outflank-mailman (input) for mailman id 679673;
 Mon, 12 Feb 2024 15:59:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6iVD=JV=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZYii-0007Zw-EL
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 15:59:48 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id be3b6f15-c9bf-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 16:59:46 +0100 (CET)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-3394bec856fso2613140f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 07:59:46 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f5-20020a5d58e5000000b0033b44456463sm7107513wrd.106.2024.02.12.07.59.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 07:59:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: be3b6f15-c9bf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707753586; x=1708358386; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=w1pAg6FxT+cR1Eho1V2+i3gOiEnvQ3J3KTlL22Urghc=;
        b=DPWCA6C8F+q+v41PtAriQSO5lhCG441DFZ+wpnm1HPKaDrxiBS98SV6i5n8oh9iaXc
         h1c58Y/yAQlQyuU8eIqfJddc2apkidAnKHpCjuKRHul5XCRa+kYSqUJrXlZgj9uueC2q
         Fbz6cs9kpuxkwRVpNVWGTFzhaO7mDqYfl4lwjrc+AX1H54s3Sc8iBpH91zuhs+LMcbM/
         LijvmGHYhbYl3kgnwlQ53X9zmLkmpIH4gYH9tm+JquPiZjSPa1fZLpjI+xutp0T+Nr4P
         yRE1U/4ujTiXrqfFQcUJutRjE/eCsaMZDU8QSMAtp7YLIsg4nEuam1f1CD/RP2RWksdu
         7hcw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707753586; x=1708358386;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=w1pAg6FxT+cR1Eho1V2+i3gOiEnvQ3J3KTlL22Urghc=;
        b=fHdV1fZ8YaGAEwIDaZ1tpM1LMZrycP5dOPO7AKL/C51F0BYUn/TTuDq9m6Q8nv+BfF
         wvvRshmmXu0QbdD41CbrmCy2e6xHb7KXTMRJIKwnV5WIDeT0uypRuTykffjEfVxlkaOp
         ijEpP8Uf1jEdlQkdZb0boESxsU9Yib08lRqUMmmyJI1T9FaTEu45clCdRGUcakTjWWbs
         Ug/nEojbwPeQnmBDe8r89tJrqp64mPRCzgNRND6C0e0BJHaXZd/JiI1FqkIioFpOzWJ7
         gKfR64YTobAMuQfNr3U5YDiHYoijJvnmjYnBaQrLQyrKdghrFU8o9xn4G2p2Z4JggaO0
         kGFA==
X-Gm-Message-State: AOJu0YzfMKS9zbZhGbk0FovKYsj1jZjsIqRVlTp70ix2qFDQQQ69gXRH
	o5ZU3xpRX2irqV/BVEmOq2nL8m6l+il4elGAr6PFza9ElVnYO9N88OBHqKCHwA==
X-Google-Smtp-Source: AGHT+IHyW/vf9JlvmfcD4f49Dw3hRIsdCTempaarV+B70ZaAiZ6y5Qai+ON4+VqNJhbra3/YjGZQsw==
X-Received: by 2002:adf:f2c6:0:b0:33b:17cd:d252 with SMTP id d6-20020adff2c6000000b0033b17cdd252mr5843803wrp.19.1707753585900;
        Mon, 12 Feb 2024 07:59:45 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXmif11QdnilgWxubBlobrTI+uGsson6BKpUFbSFGthgf6MGEufqEnXPhTvD2dcXrGNKQC9yOROW4aVdd6Ge0OmXDynZfDk8rb+3JNCryLsdzxrZ1v3ibCbylDdzjbAmoBVZeUe9534F1YvHDXr
Message-ID: <6689fa15-a155-4771-bb65-dae57ee4dcf8@suse.com>
Date: Mon, 12 Feb 2024 16:59:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/7] IOMMU: rename and re-type ats_enabled
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <25506838-b818-4686-8c16-3a198338af44@suse.com>
 <467d24e1-8ed3-4dda-a334-70ff747bf94b@suse.com> <ZcS_uC2t96Lh720Y@macbook>
 <4179ded9-36b8-4d47-997b-40d056e033af@suse.com> <ZcnnautuDD2qluQm@macbook>
 <2fccd004-b35a-4cde-afbb-722cb2413902@suse.com> <Zco7fvp1nPVlLtRy@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zco7fvp1nPVlLtRy@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 12.02.2024 16:38, Roger Pau Monné wrote:
> On Mon, Feb 12, 2024 at 11:45:33AM +0100, Jan Beulich wrote:
>> On 12.02.2024 10:39, Roger Pau Monné wrote:
>>> On Thu, Feb 08, 2024 at 04:49:46PM +0100, Jan Beulich wrote:
>>>> On 08.02.2024 12:49, Roger Pau Monné wrote:
>>>>> On Mon, Feb 05, 2024 at 02:55:43PM +0100, Jan Beulich wrote:
>>>>>> Make the variable a tristate, with (as done elsewhere) a negative value
>>>>>> meaning "default". Since all use sites need looking at, also rename it
>>>>>> to match our usual "opt_*" pattern. While touching it, also move it to
>>>>>> .data.ro_after_init.
>>>>>>
>>>>>> The only place it retains boolean nature is pci_ats_device(), for now.
>>>>>
>>>>> Why does it retain the boolean nature in pci_ats_device()?
>>>>>
>>>>> I assume this is to avoid having to touch the line again in a further
>>>>> patch, as given the current logic pci_ats_device() would also want to
>>>>> treat -1 as ATS disabled.
>>>>
>>>> No, then I would need to touch the line. The function wants to treat
>>>> -1 as "maybe enabled", so the caller can know whether a device is an
>>>> ATS device regardless of whether ATS use is fully off, or only
>>>> "soft-off".
>>>
>>> I have to admit I'm slightly concerned about this soft-off.  Given the
>>> current status of ATS itself in Xen, and the technology itself, I
>>> think a user should always opt-in to ATS usage.
>>
>> The plan is to follow your suggestion in patch 3 and require explicit
>> enabling for passing through of such devices. For Dom0, however, I
>> think it is important that we respect the firmware request by default.
>> The only viable(?!) alternative would be to panic() instead.
> 
> Or assign to domIO?

Not behind Dom0's back - I can't see how that would work if then Dom0
tried to load a driver for the device.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 17:08:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 17:08:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679679.1057273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZZmT-0002oI-Rf; Mon, 12 Feb 2024 17:07:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679679.1057273; Mon, 12 Feb 2024 17:07:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZZmT-0002oB-P7; Mon, 12 Feb 2024 17:07:45 +0000
Received: by outflank-mailman (input) for mailman id 679679;
 Mon, 12 Feb 2024 17:07:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=JouK=JV=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rZZmT-0002o5-1v
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 17:07:45 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3c4d298e-c9c9-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 18:07:43 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 8F1A221D12;
 Mon, 12 Feb 2024 17:07:40 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1B34813985;
 Mon, 12 Feb 2024 17:07:40 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id BiVBBFxQymUkfQAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 12 Feb 2024 17:07:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3c4d298e-c9c9-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707757662; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=kk7122btO7pJ42ZiiE74pdI4cmybsYzCru/PHh986qs=;
	b=SN8JMEgEOz30p+j1nMINkZg6dyfkpPb2CMhSDFGPu8oPYrNyizob7ZwC93s59ivC7K6xlz
	7alhMTNwFHB/Fqse6gDZjtvnvcvY//uQYdSrtug2N0Cdt6G4YXefshmHIi3V0JUC1ti7Bg
	P/XVcwjFgfFcWNmWbdSzfn/U0MtnECM=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707757660; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=kk7122btO7pJ42ZiiE74pdI4cmybsYzCru/PHh986qs=;
	b=RCkSHHkV6z/61pPjeP0ht4CS53I20jbd1R9DfpOck0S/qEMdBrNHFf5/QIxYGK0P6SiJpT
	fCzhw91HNnucBnZEXqxY/vk+nPFiLrXdrL4PkK70DmnKmezvgecLqPTS05u/uTQ9O1XGwu
	kiB5ix7gIEbSbayA0JYxOAPiwRTInlo=
Message-ID: <0d65f145-e086-44e5-9ae7-8acb3c3b05e2@suse.com>
Date: Mon, 12 Feb 2024 18:07:39 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] x86/xen: Add some null pointer checking to smp.c
Content-Language: en-US
To: Kunwu Chan <chentao@kylinos.cn>, boris.ostrovsky@oracle.com,
 tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
 dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org,
 kernel test robot <lkp@intel.com>, Markus Elfring <Markus.Elfring@web.de>
References: <20240119094948.275390-1-chentao@kylinos.cn>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <20240119094948.275390-1-chentao@kylinos.cn>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------oyqcERLWqXsESpaVe0Z0OcRJ"
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.55
X-Spamd-Result: default: False [-3.55 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 TO_DN_SOME(0.00)[];
	 HAS_ATTACHMENT(0.00)[];
	 MIME_BASE64_TEXT_BOGUS(1.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MIME_BASE64_TEXT(0.10)[];
	 SIGNED_PGP(-2.00)[];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 BAYES_HAM(-1.36)[90.57%];
	 MID_RHS_MATCH_FROM(0.00)[];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[web.de];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_TWELVE(0.00)[12];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[intel.com:email,kylinos.cn:email,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[lists.xenproject.org,vger.kernel.org,intel.com,web.de];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------oyqcERLWqXsESpaVe0Z0OcRJ
Content-Type: multipart/mixed; boundary="------------GJFwjHAkohOh3F0kON0n9v9D";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Kunwu Chan <chentao@kylinos.cn>, boris.ostrovsky@oracle.com,
 tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
 dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com
Cc: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org,
 kernel test robot <lkp@intel.com>, Markus Elfring <Markus.Elfring@web.de>
Message-ID: <0d65f145-e086-44e5-9ae7-8acb3c3b05e2@suse.com>
Subject: Re: [PATCH v3] x86/xen: Add some null pointer checking to smp.c
References: <20240119094948.275390-1-chentao@kylinos.cn>
In-Reply-To: <20240119094948.275390-1-chentao@kylinos.cn>

--------------GJFwjHAkohOh3F0kON0n9v9D
Content-Type: multipart/mixed; boundary="------------zpA84tdYN08SBrQKGSgADu0z"

--------------zpA84tdYN08SBrQKGSgADu0z
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMTkuMDEuMjQgMTA6NDksIEt1bnd1IENoYW4gd3JvdGU6DQo+IGthc3ByaW50ZigpIHJl
dHVybnMgYSBwb2ludGVyIHRvIGR5bmFtaWNhbGx5IGFsbG9jYXRlZCBtZW1vcnkNCj4gd2hp
Y2ggY2FuIGJlIE5VTEwgdXBvbiBmYWlsdXJlLiBFbnN1cmUgdGhlIGFsbG9jYXRpb24gd2Fz
IHN1Y2Nlc3NmdWwNCj4gYnkgY2hlY2tpbmcgdGhlIHBvaW50ZXIgdmFsaWRpdHkuDQo+IA0K
PiBTaWduZWQtb2ZmLWJ5OiBLdW53dSBDaGFuIDxjaGVudGFvQGt5bGlub3MuY24+DQo+IFJl
cG9ydGVkLWJ5OiBrZXJuZWwgdGVzdCByb2JvdCA8bGtwQGludGVsLmNvbT4NCj4gQ2xvc2Vz
OiBodHRwczovL2xvcmUua2VybmVsLm9yZy9vZS1rYnVpbGQtYWxsLzIwMjQwMTE2MTExOS5p
b2Y2QlFzZi1sa3BAaW50ZWwuY29tLw0KPiBTdWdnZXN0ZWQtYnk6IE1hcmt1cyBFbGZyaW5n
IDxNYXJrdXMuRWxmcmluZ0B3ZWIuZGU+DQoNClJldmlld2VkLWJ5OiBKdWVyZ2VuIEdyb3Nz
IDxqZ3Jvc3NAc3VzZS5jb20+DQoNCg0KSnVlcmdlbg0KDQo=
--------------zpA84tdYN08SBrQKGSgADu0z
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------zpA84tdYN08SBrQKGSgADu0z--

--------------GJFwjHAkohOh3F0kON0n9v9D--

--------------oyqcERLWqXsESpaVe0Z0OcRJ
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXKUFsFAwAAAAAACgkQsN6d1ii/Ey/F
RAf+NgPRf6Dj0Cv4DXxeakG2Nvg7K+3I26Sc7zRy1Ut3kK62Ftla3hGKHGVUigMg359/MLtzZFG0
3qv1WLip4xsendb+uBo/OGD8JGFsiEHasZeggaW/NRLM4iQFM3pNmMzoYb4P2xDHOeYmadeSHEXG
5VXCZihtBjfdLpSW2r/bqQXgo7/bXAElorBcKBas6NfR6ANifMVuFeuvT2N95RvYDGKl4E53Sxdm
g8okFfHNN1BbHdcJHLO6ja3//Ilo98LUu/DMCKOG9uroMBLFBd9btYzj2lnVJBxuroL0wFgIQLpL
fT4aGaalVVa0ocHzelzBFAlWCFd8VasHRRa21NXgWQ==
=ZddQ
-----END PGP SIGNATURE-----

--------------oyqcERLWqXsESpaVe0Z0OcRJ--


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 17:27:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 17:27:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679684.1057284 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZa5D-0005xa-IA; Mon, 12 Feb 2024 17:27:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679684.1057284; Mon, 12 Feb 2024 17:27:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZa5D-0005xT-FC; Mon, 12 Feb 2024 17:27:07 +0000
Received: by outflank-mailman (input) for mailman id 679684;
 Mon, 12 Feb 2024 17:27:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=tUo2=JV=rabbit.lu=slack@srs-se1.protection.inumbo.net>)
 id 1rZa5B-0005x4-U7
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 17:27:06 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ef78a95a-c9cb-11ee-98f5-efadbce2ee36;
 Mon, 12 Feb 2024 18:27:03 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-410e7b08252so19345e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 09:27:03 -0800 (PST)
Received: from [192.168.2.1] (82-64-138-184.subs.proxad.net. [82.64.138.184])
 by smtp.googlemail.com with ESMTPSA id
 ch16-20020a5d5d10000000b0033b58cbf3ebsm7355426wrb.26.2024.02.12.09.27.01
 for <xen-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 09:27:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ef78a95a-c9cb-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=rabbit-lu.20230601.gappssmtp.com; s=20230601; t=1707758822; x=1708363622; darn=lists.xenproject.org;
        h=content-transfer-encoding:to:subject:from:content-language
         :user-agent:mime-version:date:message-id:from:to:cc:subject:date
         :message-id:reply-to;
        bh=yjZj30nDMcqp2zKXpCE5Uei4AaXUao7Snh8f9+MbwJQ=;
        b=ZH1/GjXXQPZW9Q6maJYpab8IIHBUqBgVoMR9pU8aoMzhXDgDWHIw0wfj4reSCYehYi
         BvJCTfU58c/eR3BZ4lUiEn8rAns1D3qfbf9owYgxgKoiHVt4eAiClh/RRxxFWF9tGUmU
         6AUQWaFAlo2g113Lttfbp8C9f6mECgSgiim4LlWj4KaqCaNdStbb3XV6Fx9VgFoe0lQd
         TfWS5uH+W7IVV7PDQPjp0U5xhLPmzCEyzPrOj2s/U21G9uiG3XsbmUeFAJwKiIVOwEgO
         5JdKqHa97VoOMoaDdU7CSIJCoHXAafRu2U7o4AuKIj/oD6ypV8DbuAQ+k62j7u//BHrQ
         bMAw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707758822; x=1708363622;
        h=content-transfer-encoding:to:subject:from:content-language
         :user-agent:mime-version:date:message-id:x-gm-message-state:from:to
         :cc:subject:date:message-id:reply-to;
        bh=yjZj30nDMcqp2zKXpCE5Uei4AaXUao7Snh8f9+MbwJQ=;
        b=OSatZCA3WH61xVWMYxgaTbh5dARJLH/cMUk7pTgFagvNUGgBesbBC4eL/O/UfR+VJA
         DIfQUxwOLdEHKFdbZwPEnnYUdGt9ZWCo2vS7qNPx8v9wIVzd1uK1rPIL03ipzB02f/m0
         c8oe9CqAZ7E5mSSEoMXomuP6t91jrG02Y3gK9Tz3BGJ3as9L8zUGRhXRqiNgh0Al2cVh
         hY7G3JY5N73lTc23SS8xItrumVZ5MKmTSFOEFNyee/Z6s3iTdXs6O/NZFBsr+D96wNX3
         orrS/do6sBcfYHXKHrEHarXloKJw3nSTSXzJsNbXZYGSIKdsD2HB2z/cqKBbTWJHOr0r
         b7Sg==
X-Gm-Message-State: AOJu0Ywz63h1iiVTk3J7Wlsf3LPKKhnVD7pvLhhtv0lUqQ7/h5qb1rmY
	YAIrmz8qLptLk1pWpR7wty43FIpAhSQeM/laownns69Bxo+zTvLevWpd/+jYKXjvQfaiIpfsTNP
	iOV0=
X-Google-Smtp-Source: AGHT+IHdHjLZXca1fAAhTcJzF5V7Wlw/r89uGooEU+IUUgXYeY++DtZRzmuKVuACVlL5O6kuN/IAww==
X-Received: by 2002:a5d:6890:0:b0:33c:2e8a:ddee with SMTP id h16-20020a5d6890000000b0033c2e8addeemr330101wru.4.1707758822281;
        Mon, 12 Feb 2024 09:27:02 -0800 (PST)
Message-ID: <51e993f4-c250-400c-bbf4-017a1dbf1c2d@rabbit.lu>
Date: Mon, 12 Feb 2024 18:27:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
From: zithro <slack@rabbit.lu>
Subject: The "64bits time_t transition" in Debian/Xen
To: xen-devel@lists.xenproject.org
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hey all,

the Debian project is focused on the "2038 time_t" switch.
So the maintainers of the Debian Xen package must ensure that all
imported Xen code conforms to the new Debian standards.

I was asked by Andrew Cooper to post here about this, I'll quote him :
"So I had been idly wondering whether Xen would match up to Debian's new
policy, and it appears not
this topic really needs to be brought up on the xen-devel mailing list
do you have any more details as to what has gone wrong?
this is something we ought to arrange to happen in CI by default
but it sounds like there's some work needed first"

(Not answering the question because I'm just a messenger).

Relevant links:
https://wiki.debian.org/ReleaseGoals/64bit-time
https://bugs.debian.org/1063270
   (xen: NMU diff for 64-bit time_t transition)

Thanks

-- 
++
zithro / Cyril


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 17:43:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 17:43:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679688.1057293 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZaL3-0000eM-O2; Mon, 12 Feb 2024 17:43:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679688.1057293; Mon, 12 Feb 2024 17:43:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZaL3-0000eF-LP; Mon, 12 Feb 2024 17:43:29 +0000
Received: by outflank-mailman (input) for mailman id 679688;
 Mon, 12 Feb 2024 17:43:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ADcg=JV=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rZaL2-0000e9-F5
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 17:43:28 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3a164212-c9ce-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 18:43:27 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a29c4bbb2f4so451620566b.1
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 09:43:27 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 l19-20020a1709065a9300b00a3bd745325dsm415932ejq.150.2024.02.12.09.43.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 09:43:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a164212-c9ce-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707759806; x=1708364606; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HZrgOrSulvUKIjdRXKdZeWdaJEscFFMgBz2wVCOdlB4=;
        b=OeySRw5LcMGb7VpNl6GkiVLCC7r5PKeYGeb+dCZzdIj1uvwska61My7NCxWZSbBy6k
         EKc+Ij+VLbLcQGAlhex2H6NXL9+lgCMd0HuvJUK6+0EhRIRg55B+fI/WTbjOrvKbKv5c
         Am25I42OWlgi6W8wEzcMYGdkthFwqGjPUhX9s=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707759806; x=1708364606;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HZrgOrSulvUKIjdRXKdZeWdaJEscFFMgBz2wVCOdlB4=;
        b=EXy1NxrWKjZYzpzeke3kcqk77aIE6C2IFLpOKTo4+oRTxDTHqUyyr/leFa3rAPwZju
         BAcBlBNxcuF6he8VzZIGnXmjH+g8cWuTjHE5p0wYurW7tWjz6LOu6aD3ESc6M1SyK5fX
         MXeyIPAWsblqQ2T01tMd6WvzmuVtlSEoI26u4/ZB7hei7AuumzIekvuVdYDUyUskOJzf
         F23HZ3DvZvLXjatgP6wur7Y3gmFUhnKCxKEGmpY15l4tGjb/p6CpFvnL4p87PyqHdsTd
         W6a4lEYQWTNrG7O5cYrMHu7J4a8HMrDxzVFV1beWuAXdYVFSR+DteMa41lWhZTWFr9AB
         gTog==
X-Forwarded-Encrypted: i=1; AJvYcCWBXVy5g0D1FSIV9bCS1p0kLUy3lWd60A1LPQENPV/qCeJQNWF7S0hiz3wXrdXI46LxK15zo0JGq7q1sZublAJGNG6DKuBeNzzlWo0tiuQ=
X-Gm-Message-State: AOJu0YwKROjMq/U4law6BIHbMYjKJREIdnpDSdFZlK7tRXMLG7WSDIrZ
	FSAiXTp2EeeGVO7p9P3arHroWjVCNwC0+Ug+jw+u0QIucalgCjmpCRP09gJNJYzbRQ/McEm9i4v
	W
X-Google-Smtp-Source: AGHT+IHg4e9KswN9J3Z3eWFwEJk17ArjpbYFyZF6ZN6ntMWEa/S3umpsDbR25BeO+YWZ21lQm9zLMg==
X-Received: by 2002:a17:906:6c89:b0:a3c:5766:58d9 with SMTP id s9-20020a1709066c8900b00a3c576658d9mr3422994ejr.4.1707759806659;
        Mon, 12 Feb 2024 09:43:26 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW5d7M5KHAcOpPD4tUS3vaMdHSIz0ZCUjfsfhPepDFmU+gj/qeo3A6JodhmTmEg5Te+naFOiz+GvsghylR0PSuQ2PDW/e4vC+MVnrMe7fbBvYM2aLXu3O5s7kcAi+j+gThtAQi1GF7b
Message-ID: <2922b179-571a-4745-b204-7dc7e2beeeea@citrix.com>
Date: Mon, 12 Feb 2024 17:43:26 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: The "64bits time_t transition" in Debian/Xen
Content-Language: en-GB
To: zithro <slack@rabbit.lu>, xen-devel@lists.xenproject.org,
 1063270@bugs.debian.org
References: <51e993f4-c250-400c-bbf4-017a1dbf1c2d@rabbit.lu>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <51e993f4-c250-400c-bbf4-017a1dbf1c2d@rabbit.lu>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 12/02/2024 5:27 pm, zithro wrote:
> Hey all,
>
> the Debian project is focused on the "2038 time_t" switch.
> So the maintainers of the Debian Xen package must ensure that all
> imported Xen code conforms to the new Debian standards.
>
> I was asked by Andrew Cooper to post here about this, I'll quote him :
> "So I had been idly wondering whether Xen would match up to Debian's new
> policy, and it appears not
> this topic really needs to be brought up on the xen-devel mailing list
> do you have any more details as to what has gone wrong?
> this is something we ought to arrange to happen in CI by default
> but it sounds like there's some work needed first"
>
> (Not answering the question because I'm just a messenger).

xen.git/xen$ git grep -w time_t -- :/
../tools/console/client/main.c:106:     time_t start, now;
../tools/console/daemon/io.c:272:       time_t now = time(NULL);
../tools/libs/light/libxl_qmp.c:116:    time_t timeout;
../tools/libs/light/libxl_qmp.c:585:                               
time_t ask_timeout)
../tools/libs/light/libxl_x86.c:516:        time_t t;
../tools/libs/toollog/xtl_logger_stdio.c:61:        time_t now = time(0);
../tools/tests/xenstore/test-xenstore.c:453:    time_t stop;
../tools/xenmon/xenbaked.c:98:time_t start_time;
../tools/xenstored/core.c:109:  time_t now;
../tools/xenstored/core.h:150:  time_t ta_start_time;
../tools/xenstored/domain.c:143:        time_t mem_last_msg;
../tools/xenstored/domain.c:188:static time_t wrl_log_last_warning; /*
0: no previous warning */
../tools/xenstored/domain.c:1584:       time_t now;
../tools/xenstored/lu.c:160:    time_t now = time(NULL);
../tools/xenstored/lu.c:185:    time_t now = time(NULL);
../tools/xenstored/lu.c:292:    time_t now = time(NULL);
../tools/xenstored/lu.h:32:     time_t started_at;
../tools/xentop/xentop.c:947:   time_t curt;
../tools/xl/xl_info.c:742:static char *current_time_to_string(time_t now)
../tools/xl/xl_info.c:759:static void print_dom0_uptime(int short_mode,
time_t now)
../tools/xl/xl_info.c:810:static void print_domU_uptime(uint32_t domuid,
int short_mode, time_t now)
../tools/xl/xl_info.c:847:    time_t now;
../tools/xl/xl_vmcontrol.c:336:        time_t start;
../tools/xl/xl_vmcontrol.c:495:    time_t now;
../tools/xl/xl_vmcontrol.c:504:    if (now == ((time_t) -1)) {
../tools/xs-clients/xenstore_control.c:33:    time_t time_start;
arch/x86/cpu/mcheck/mce.h:224:    uint64_t time;     /* wall time_t when
error was detected */
arch/x86/time.c:1129: * machines were long is 32-bit! (However, as
time_t is signed, we


I don't see any ABI problems from using a 64bit time_t.  The only header
file with a time_t is xenstored/lu.h which is a private header and not a
public ABI.

I guess we fell into the "could not be analysed via
abi-compliance-checker" case?

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 18:38:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 18:38:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679696.1057304 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZbCM-0007lm-Mo; Mon, 12 Feb 2024 18:38:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679696.1057304; Mon, 12 Feb 2024 18:38:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZbCM-0007lf-JV; Mon, 12 Feb 2024 18:38:34 +0000
Received: by outflank-mailman (input) for mailman id 679696;
 Mon, 12 Feb 2024 18:38:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rZbCL-0007lZ-N7
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 18:38:33 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZbCK-0003AO-7F; Mon, 12 Feb 2024 18:38:32 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZbCJ-0000w0-Tz; Mon, 12 Feb 2024 18:38:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=wW9OZL67jLnN0yjB6N2SrzdWDC25QQCKYvxPL9HRe+c=; b=Ree2ebO9LjHh/2MtfcGEl2j7A2
	qEGHTKgF16h8PMQSa6cqOYuq8/unUW47PvBdRfmObcBImqabWN45S6AttRgv2/vcxPNX1YIbHY65D
	GNkL9Dj6oivnnxP65HkHU0MqSUrSUo8oa74Q8m3x/pgd2G25y7Rk7oN74UF14FaHxV1M=;
Message-ID: <09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org>
Date: Mon, 12 Feb 2024 18:38:29 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Return type of clean_and_invalidate_dcache_va_range
Content-Language: en-GB
To: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Xen Devel <xen-devel@lists.xenproject.org>,
 Consulting <consulting@bugseng.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper3 <andrew.cooper3@citrix.com>, Roger Pau
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>
References: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
 <alpine.DEB.2.22.394.2402091402080.1925432@ubuntu-linux-20-04-desktop>
 <f86baad0-f113-4156-9c10-6910e8c63492@xen.org>
 <02446758-a5a9-4a8a-8ce6-15abdc9ede65@suse.com>
 <99faac70440a68824a17fcaaea55ef48@bugseng.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <99faac70440a68824a17fcaaea55ef48@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Nicola,

On 12/02/2024 14:56, Nicola Vetrini wrote:
> On 2024-02-12 09:26, Jan Beulich wrote:
>> On 10.02.2024 11:17, Julien Grall wrote:
>>> Hi,
>>>
>>> On 09/02/2024 22:02, Stefano Stabellini wrote:
>>>> On Fri, 9 Feb 2024, Nicola Vetrini wrote:
>>>>> Hi all,
>>>>>
>>>>> In the context of violations of MISRA C:2012 Rule 17.7: "The value 
>>>>> returned by
>>>>> a function having non-void return type shall be used", I was 
>>>>> looking at the
>>>>> function "clean_and_invalidate_dcache_va_range". It has the following
>>>>> signature on both arm and x86:
>>>>>
>>>>> static inline int clean_and_invalidate_dcache_va_range
>>>>>      (const void *p, unsigned long size)
>>>>>
>>>>> The commit that introduced it for Arm ~9 years ago (71d64afe3e12: 
>>>>> "arm: return
>>>>> int from *_dcache_va_range") [1] mentions that on Arm it can't 
>>>>> fail, but
>>>>> supposedly it can on x86.
>>>>>
>>>>> However, as far as I can tell, for both arch-es the implementation 
>>>>> now always
>>>>> returns 0 [2][3], so perhaps the mention of -EOPNOTSUPP for x86 is 
>>>>> no longer
>>>>> true (I wasn't able to reconstruct if there was a time at which 
>>>>> this was true,
>>>>> even in the same commit that changed the return type to int).
>>>>>
>>>>> The question is: should the return type be void, since it appears 
>>>>> that every
>>>>> user is ignoring the returned value (violating the rule), except 
>>>>> the one in
>>>>> common/grant_table.c [4]?
>>>>
>>>> Looking at the implementation on both ARM and x86, I am in favor of
>>>> changing the return type to void
>>> I think we need some consistency between all the cache flush helpers
>>> (clean_and_invalidate_dcache_va_range, invalidate_dcache_va_range() and
>>> clean_dcache_va_range()). They should all return a values or not 
>>> return any.
>>
>> +1
>>
> 
> I agree. I took this helper as an example, but e.g. 
> invalidate_dcache_va_range returns -EOPNOTSUPP on x86 and it's only used 
> in common/grant_table.
> Perhaps the signatures should remain as is for consistency, especially 
> given the remark below about the other architectures, and this would 
> entail a deviation.

In general, I am not in favor of adding a deviation if the code can be 
changed. In this case, we could have (untested and just a hack to show 
my point):

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 5721eab22561..ae9ccf5388fc 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3573,7 +3573,13 @@ static int _cache_flush(const 
gnttab_cache_flush_t *cflush, grant_ref_t *cur_ref
      v += cflush->offset;

      if ( (cflush->op & GNTTAB_CACHE_INVAL) && (cflush->op & 
GNTTAB_CACHE_CLEAN) )
-        ret = clean_and_invalidate_dcache_va_range(v, cflush->length);
+    {
+#ifdef clean_and_invalidate_dcache_va_range
+        clean_and_invalidate_dcache_va_range(v, cflush->length);
+        ret = 0;
+#else
+        ret = -ENOSYS;
+#endif
      else if ( cflush->op & GNTTAB_CACHE_INVAL )
          ret = invalidate_dcache_va_range(v, cflush->length);
      else if ( cflush->op & GNTTAB_CACHE_CLEAN )

An alternative would be to introduced arch_grant_cache_flush() and move 
the if/else logic there. Something like:

diff --git a/xen/arch/arm/include/asm/page.h 
b/xen/arch/arm/include/asm/page.h
index 69f817d1e68a..4a3de49762a1 100644
--- a/xen/arch/arm/include/asm/page.h
+++ b/xen/arch/arm/include/asm/page.h
@@ -281,6 +281,19 @@ static inline void write_pte(lpae_t *p, lpae_t pte)
      dsb(sy);
  }

+static inline arch_grant_cache_flush(unsigned int op, const void *p, 
unsigned long size)
+{
+    unsigned int order = get_order_from_bytes(size);
+
+    if ( (cflush->op & GNTTAB_CACHE_INVAL) && (cflush->op & 
GNTTAB_CACHE_CLEAN) )
+        clean_and_invalidate_dcache_va_range(v, cflush->length);
+    else if ( cflush->op & GNTTAB_CACHE_INVAL )
+        invalidate_dcache_va_range(v, cflush->length);
+    else if ( cflush->op & GNTTAB_CACHE_CLEAN )
+        clean_dcache_va_range(v, cflush->length);
+
+    return 0;
+}

  /* Flush the dcache for an entire page. */
  void flush_page_to_ram(unsigned long mfn, bool sync_icache);
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 424744ad5e1a..647e1522466d 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -735,8 +735,7 @@ void asmlinkage __init start_xen(unsigned long 
boot_phys_offset,
                fdt_paddr);

      /* Register Xen's load address as a boot module. */
-    xen_bootmodule = add_boot_module(BOOTMOD_XEN,
-                             virt_to_maddr(_start),
+    xen_bootmodule = add_boot_module(BOOTMOD_XEN, virt_to_maddr(_start),
                               (paddr_t)(uintptr_t)(_end - _start), false);
      BUG_ON(!xen_bootmodule);

diff --git a/xen/arch/x86/include/asm/flushtlb.h 
b/xen/arch/x86/include/asm/flushtlb.h
index bb0ad58db49b..dfe51cddde90 100644
--- a/xen/arch/x86/include/asm/flushtlb.h
+++ b/xen/arch/x86/include/asm/flushtlb.h
@@ -182,23 +182,22 @@ void flush_area_mask(const cpumask_t *mask, const 
void *va,
  }

  static inline void flush_page_to_ram(unsigned long mfn, bool 
sync_icache) {}
-static inline int invalidate_dcache_va_range(const void *p,
-                                             unsigned long size)
-{ return -EOPNOTSUPP; }
-static inline int clean_and_invalidate_dcache_va_range(const void *p,
-                                                       unsigned long size)
+
+unsigned int guest_flush_tlb_flags(const struct domain *d);
+void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask);
+
+static inline arch_grant_cache_flush(unsigned int op, const void *p, 
unsigned long size)
  {
-    unsigned int order = get_order_from_bytes(size);
+    unsigned int order;
+
+    if ( !(cflush->op & GNTTAB_CACHE_CLEAN) )
+        return -EOPNOTSUPP;
+
+    order = get_order_from_bytes(size);
      /* sub-page granularity support needs to be added if necessary */
      flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
+
      return 0;
  }
-static inline int clean_dcache_va_range(const void *p, unsigned long size)
-{
-    return clean_and_invalidate_dcache_va_range(p, size);
-}
-
-unsigned int guest_flush_tlb_flags(const struct domain *d);
-void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask);

  #endif /* __FLUSHTLB_H__ */

I have a slight preference for the latter. I would like to hear the 
opinion of the others.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 19:19:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 19:19:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679702.1057315 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZbpd-0004jc-PQ; Mon, 12 Feb 2024 19:19:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679702.1057315; Mon, 12 Feb 2024 19:19:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZbpd-0004jV-KZ; Mon, 12 Feb 2024 19:19:09 +0000
Received: by outflank-mailman (input) for mailman id 679702;
 Mon, 12 Feb 2024 19:19:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=G6LY=JV=kernel.org=helgaas@srs-se1.protection.inumbo.net>)
 id 1rZbpc-0004j6-Eh
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 19:19:08 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9684ec17-c9db-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 20:19:06 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 9CC4ACE13B7;
 Mon, 12 Feb 2024 19:19:00 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95ECEC433F1;
 Mon, 12 Feb 2024 19:18:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9684ec17-c9db-11ee-8a4c-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707765539;
	bh=KtY+2vwykgYx0ZhCuIUifzWpXvaxOyQLOoq8eVe0/yU=;
	h=Date:From:To:Cc:Subject:In-Reply-To:From;
	b=jwRehpBB1mbek9OHSZcQG4LLIahO11fkowPwY+RDEnhfuRZO2S52shbq7/dcY+qfE
	 iEYB8aVjIU3buYbYfh9x6CFS1/B6CnDOvzAv9g3Jn96LPZl9XjXRj3ZlnLmvhiC1jb
	 j7E5z5nUMRSMYvlK8qNk2Js+O3eopRzA9Uux1dLjoVTcAYhb541gdyXAHlbb1u+4E0
	 02afHZ6L5zYFBoNbCq9Hf2fB4NBQzsa1nHeCculaTjKqbKwDppZ7P0VG8PXOrOHt3r
	 xcq2+FbZ0RSijcp6poSdUYCOPMyZiyUoLrxj0tejWKyGB9ClrtU0x41aXcP01RaS4o
	 8SuuSZ9KNU8+g==
Date: Mon, 12 Feb 2024 13:18:58 -0600
From: Bjorn Helgaas <helgaas@kernel.org>
To: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
Cc: "Chen, Jiqian" <Jiqian.Chen@amd.com>,
	"Rafael J . Wysocki" <rafael@kernel.org>,
	Len Brown <lenb@kernel.org>, Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
	"Huang, Ray" <Ray.Huang@amd.com>,
	"Ragiadakou, Xenia" <Xenia.Ragiadakou@amd.com>
Subject: Re: [RFC KERNEL PATCH v4 3/3] PCI/sysfs: Add gsi sysfs for pci_dev
Message-ID: <20240212191858.GA1137351@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZcnhOEjnTgbYFPVl@macbook>

On Mon, Feb 12, 2024 at 10:13:28AM +0100, Roger Pau Monné wrote:
> On Fri, Feb 09, 2024 at 03:05:49PM -0600, Bjorn Helgaas wrote:
> > On Thu, Feb 01, 2024 at 09:39:49AM +0100, Roger Pau Monné wrote:
> > > On Wed, Jan 31, 2024 at 01:00:14PM -0600, Bjorn Helgaas wrote:
> > > > On Wed, Jan 31, 2024 at 09:58:19AM +0100, Roger Pau Monné wrote:
> > > > > On Tue, Jan 30, 2024 at 02:44:03PM -0600, Bjorn Helgaas wrote:
> > > > > > On Tue, Jan 30, 2024 at 10:07:36AM +0100, Roger Pau Monné wrote:
> > > > > > > On Mon, Jan 29, 2024 at 04:01:13PM -0600, Bjorn Helgaas wrote:
> > > > > > > > On Thu, Jan 25, 2024 at 07:17:24AM +0000, Chen, Jiqian wrote:
> > > > > > > > > On 2024/1/24 00:02, Bjorn Helgaas wrote:
> > > > > > > > > > On Tue, Jan 23, 2024 at 10:13:52AM +0000, Chen, Jiqian wrote:
> > > > > > > > > >> On 2024/1/23 07:37, Bjorn Helgaas wrote:
> > > > > > > > > >>> On Fri, Jan 05, 2024 at 02:22:17PM +0800, Jiqian Chen wrote:
> > > > > > > > > >>>> There is a need for some scenarios to use gsi sysfs.
> > > > > > > > > >>>> For example, when xen passthrough a device to dumU, it will
> > > > > > > > > >>>> use gsi to map pirq, but currently userspace can't get gsi
> > > > > > > > > >>>> number.
> > > > > > > > > >>>> So, add gsi sysfs for that and for other potential scenarios.
> > > > > > > > > >> ...
> > > > > > > > > > 
> > > > > > > > > >>> I don't know enough about Xen to know why it needs the GSI in
> > > > > > > > > >>> userspace.  Is this passthrough brand new functionality that can't be
> > > > > > > > > >>> done today because we don't expose the GSI yet?
> > > > > > > > 
> > > > > > > > I assume this must be new functionality, i.e., this kind of
> > > > > > > > passthrough does not work today, right?
> > > > > > > > 
> > > > > > > > > >> has ACPI support and is responsible for detecting and controlling
> > > > > > > > > >> the hardware, also it performs privileged operations such as the
> > > > > > > > > >> creation of normal (unprivileged) domains DomUs. When we give to a
> > > > > > > > > >> DomU direct access to a device, we need also to route the physical
> > > > > > > > > >> interrupts to the DomU. In order to do so Xen needs to setup and map
> > > > > > > > > >> the interrupts appropriately.
> > > > > > > > > > 
> > > > > > > > > > What kernel interfaces are used for this setup and mapping?
> > > > > > > > >
> > > > > > > > > For passthrough devices, the setup and mapping of routing physical
> > > > > > > > > interrupts to DomU are done on Xen hypervisor side, hypervisor only
> > > > > > > > > need userspace to provide the GSI info, see Xen code:
> > > > > > > > > xc_physdev_map_pirq require GSI and then will call hypercall to pass
> > > > > > > > > GSI into hypervisor and then hypervisor will do the mapping and
> > > > > > > > > routing, kernel doesn't do the setup and mapping.
> > > > > > > > 
> > > > > > > > So we have to expose the GSI to userspace not because userspace itself
> > > > > > > > uses it, but so userspace can turn around and pass it back into the
> > > > > > > > kernel?
> > > > > > > 
> > > > > > > No, the point is to pass it back to Xen, which doesn't know the
> > > > > > > mapping between GSIs and PCI devices because it can't execute the ACPI
> > > > > > > AML resource methods that provide such information.
> > > > > > > 
> > > > > > > The (Linux) kernel is just a proxy that forwards the hypercalls from
> > > > > > > user-space tools into Xen.
> > > > > > 
> > > > > > But I guess Xen knows how to interpret a GSI even though it doesn't
> > > > > > have access to AML?
> > > > > 
> > > > > On x86 Xen does know how to map a GSI into an IO-APIC pin, in order
> > > > > configure the RTE as requested.
> > > > 
> > > > IIUC, mapping a GSI to an IO-APIC pin requires information from the
> > > > MADT.  So I guess Xen does use the static ACPI tables, but not the AML
> > > > _PRT methods that would connect a GSI with a PCI device?
> > > 
> > > Yes, Xen can parse the static tables, and knows the base GSI of
> > > IO-APICs from the MADT.
> > > 
> > > > I guess this means Xen would not be able to deal with _MAT methods,
> > > > which also contains MADT entries?  I don't know the implications of
> > > > this -- maybe it means Xen might not be able to use with hot-added
> > > > devices?
> > > 
> > > It's my understanding _MAT will only be present on some very specific
> > > devices (IO-APIC or CPU objects).  Xen doesn't support hotplug of
> > > IO-APICs, but hotplug of CPUs should in principle be supported with
> > > cooperation from the control domain OS (albeit it's not something that
> > > we tests on our CI).  I don't expect however that a CPU object _MAT
> > > method will return IO APIC entries.
> > > 
> > > > The tables (including DSDT and SSDTS that contain the AML) are exposed
> > > > to userspace via /sys/firmware/acpi/tables/, but of course that
> > > > doesn't mean Xen knows how to interpret the AML, and even if it did,
> > > > Xen probably wouldn't be able to *evaluate* it since that could
> > > > conflict with the host kernel's use of AML.
> > > 
> > > Indeed, there can only be a single OSPM, and that's the dom0 OS (Linux
> > > in our context).
> > > 
> > > Getting back to our context though, what would be a suitable place for
> > > exposing the GSI assigned to each device?
> > 
> > IIUC, the Xen hypervisor:
> > 
> >   - Interprets /sys/firmware/acpi/tables/APIC (or gets this via
> >     something running on the Dom0 kernel) to find the physical base
> >     address and GSI base, e.g., from I/O APIC, I/O SAPIC.
> 
> No, Xen parses the MADT directly from memory, before stating dom0.
> That's a static table so it's fine for Xen to parse it and obtain the
> I/O APIC GSI base.

It's an interesting split to consume ACPI static tables directly but
put the AML interpreter elsewhere.  I doubt the ACPI spec envisioned
that, which makes me wonder what other things we could trip over, but
that's just a tangent.

> >   - Needs the GSI to locate the APIC and pin within the APIC.  The
> >     Dom0 kernel is the OSPM, so only it can evaluate the AML _PRT to
> >     learn the PCI device -> GSI mapping.
> 
> Yes, Xen doesn't know the PCI device -> GSI mapping.  Dom0 needs to
> parse the ACPI methods and signal Xen to configure a GSI with a
> given trigger and polarity.
> 
> >   - Has direct access to the APIC physical base address to program the
> >     Redirection Table.
> 
> Yes, the hardware (native) I/O APIC is owned by Xen, and not directly
> accessible by dom0.
> 
> > The patch seems a little messy to me because the PCI core has to keep
> > track of the GSI even though it doesn't need it itself.  And the
> > current patch exposes it on all arches, even non-ACPI ones or when
> > ACPI is disabled (easily fixable).
> > 
> > We only call acpi_pci_irq_enable() in the pci_enable_device() path, so
> > we don't know the GSI unless a Dom0 driver has claimed the device and
> > called pci_enable_device() for it, which seems like it might not be
> > desirable.
> 
> I think that's always the case, as on dom0 devices to be passed
> through are handled by pciback which does enable them.

pcistub_init_device() labels the pci_enable_device() as a "HACK"
related to determining the IRQ, which makes me think there's not
really a requirement for the device to be *enabled* (BAR decoding
enabled) by dom0.

> I agree it might be best to not tie exposing the node to
> pci_enable_device() having been called.  Is _PRT only evaluated as
> part of acpi_pci_irq_enable()? (or pci_enable_device()).

Yes.  AFAICT, acpi_pci_irq_enable() is the only path that evaluates
_PRT (except for a debugger interface).  I don't think it *needs* to
be that way, and the fact that we do it per-device like that means we
evaluate _PRT many times even though I think the results never change.

I could imagine evaluating _PRT once as part of enumerating a PCI host
bridge (and maybe PCI-PCI bridge, per acpi_pci_irq_find_prt_entry()
comment), but that looks like a fair bit of work to implement.  And of
course it doesn't really affect the question of how to expose the
result, although it does suggest /sys/bus/acpi/devices/PNP0A03:00/ as
a possible location.

Bjorn


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 20:13:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 20:13:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679707.1057324 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZcfa-0003hP-Pi; Mon, 12 Feb 2024 20:12:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679707.1057324; Mon, 12 Feb 2024 20:12:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZcfa-0003hI-MF; Mon, 12 Feb 2024 20:12:50 +0000
Received: by outflank-mailman (input) for mailman id 679707;
 Mon, 12 Feb 2024 20:12:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZcfZ-0003h8-4a; Mon, 12 Feb 2024 20:12:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZcfY-0004q1-Lq; Mon, 12 Feb 2024 20:12:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZcfY-0000ds-45; Mon, 12 Feb 2024 20:12:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZcfY-00089S-3d; Mon, 12 Feb 2024 20:12:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PuK0zCfYeoiY9NWzlfS6LlNDtM8l6NNHFieDZDJ0D6M=; b=On9OMk7M5EkRUW48mH2CWEGH0A
	5h/TQvySxbRAqqou7K3AdLJA+J6eKDoYg1W9+/gFKzpFHfrcZSeHv+Kqdlh0oecpnWUtlZMYLAydM
	heiyI6GTxUjNaC9SBfpH9O1GbvoOHI/kQL18IGYU3tR6rGN+7xeIxWUvqGs3r/oUiLfA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184651-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184651: regressions - trouble: broken/fail/pass
X-Osstest-Failures:
    xen-unstable:test-xtf-amd64-amd64-3:<job status>:broken:regression
    xen-unstable:test-xtf-amd64-amd64-3:host-install(5):broken:regression
    xen-unstable:test-amd64-coresched-i386-xl:guest-localmigrate/x10:fail:regression
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=772380f7547bac7e4c10c9ceff87af3f54151ae5
X-Osstest-Versions-That:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 12 Feb 2024 20:12:48 +0000

flight 184651 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184651/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-xtf-amd64-amd64-3          <job status>                 broken
 test-xtf-amd64-amd64-3        5 host-install(5)        broken REGR. vs. 184648
 test-amd64-coresched-i386-xl 20 guest-localmigrate/x10   fail REGR. vs. 184648

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184648
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184648
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184648
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184648
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184648
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184648
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184648
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184648
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184648
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184648
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184648
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184648
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  772380f7547bac7e4c10c9ceff87af3f54151ae5
baseline version:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369

Last test of basis   184648  2024-02-12 01:53:55 Z    0 days
Testing same since   184651  2024-02-12 11:39:03 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Petr Beneš <w1benny@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Simone Ballarin <simone.ballarin@bugseng.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       broken  
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-xtf-amd64-amd64-3 broken
broken-step test-xtf-amd64-amd64-3 host-install(5)

Not pushing.

------------------------------------------------------------
commit 772380f7547bac7e4c10c9ceff87af3f54151ae5
Author: Simone Ballarin <simone.ballarin@bugseng.com>
Date:   Mon Feb 12 09:38:22 2024 +0100

    eclair: remove wrongly added -eval_file
    
    properties.ecl does not exist and it is not
    even generated by propertyparser.py. Loading
    this file causes empty ECLAIR analysis.
    
    Fixes: f4519ee8 ("eclair: move function and macro properties outside ECLAIR")
    Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
    Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

commit 897def94b56175ce569673a05909d2f223e1e749
Author: Petr Beneš <w1benny@gmail.com>
Date:   Mon Feb 12 09:37:58 2024 +0100

    x86/hvm: Fix fast singlestep state persistence
    
    This patch addresses an issue where the fast singlestep setting would persist
    despite xc_domain_debug_control being called with XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF.
    Specifically, if fast singlestep was enabled in a VMI session and that session
    stopped before the MTF trap occurred, the fast singlestep setting remained
    active even though MTF itself was disabled.  This led to a situation where, upon
    starting a new VMI session, the first event to trigger an EPT violation would
    cause the corresponding EPT event callback to be skipped due to the lingering
    fast singlestep setting.
    
    The fix ensures that the fast singlestep setting is properly reset when
    disabling single step debugging operations.
    
    Signed-off-by: Petr Beneš <w1benny@gmail.com>
    Reviewed-by: Tamas K Lengyel <tamas@tklengyel.com>

commit a773adedbd913966e812bddcd6b223c2e4ce3061
Author: Jan Beulich <jbeulich@suse.com>
Date:   Mon Feb 12 09:37:18 2024 +0100

    x86/PV32: restore PAE-extended-CR3 logic
    
    While the PAE-extended-CR3 VM assist is a 32-bit only concept, it still
    applies to guests also when run on a 64-bit hypervisor: The "extended
    CR3" format has to be used there as well, to fit the address in the only
    32-bit wide register there. As a result it was a mistake that the check
    was never enabled for that case, and was then mistakenly deleted in the
    course of removal of 32-bit-Xen code (218adf199e68 ["x86: We can assume
    CONFIG_PAGING_LEVELS==4"]).
    
    Similarly during Dom0 construction kernel awareness needs to be taken
    into account, and respective code was again mistakenly never enabled for
    32-bit Dom0 when running on 64-bit Xen (and thus wrongly deleted by
    5d1181a5ea5e ["xen: Remove x86_32 build target"]).
    
    At the same time restrict enabling of the assist for Dom0 to just the
    32-bit case. Furthermore there's no need for an atomic update there.
    
    Signed-off-by: Jan Beulich <jbeulich@suse.com>
    Acked-by: Roger Pau Monné <roger.pau@citrix.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 21:44:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 21:44:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679714.1057335 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZe6H-0005ZJ-BC; Mon, 12 Feb 2024 21:44:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679714.1057335; Mon, 12 Feb 2024 21:44:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZe6H-0005ZC-70; Mon, 12 Feb 2024 21:44:29 +0000
Received: by outflank-mailman (input) for mailman id 679714;
 Mon, 12 Feb 2024 21:44:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rW7E=JV=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rZe6F-0005Z6-Gv
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 21:44:27 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e30eb873-c9ef-11ee-8a4d-1f161083a0e0;
 Mon, 12 Feb 2024 22:44:25 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 5753BCE17B3;
 Mon, 12 Feb 2024 21:44:20 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3CBECC433F1;
 Mon, 12 Feb 2024 21:44:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e30eb873-c9ef-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707774259;
	bh=k+2oYHJQOv9VW4SSvSHPH8ermcNU7OsDSJypaa4rsnE=;
	h=Date:From:To:cc:Subject:From;
	b=mrgOvrG3SJQaCz8Xag+wzHTp/D8XzIGNT7PH4/JiG8v1iXfH95gCIilljmXhDJX2/
	 zviYYHc8TbjNA6RRB1+uJdiDsIJm6y8gRXtufrlrBNFidCpV/oyjPKTQVQg74BMqVV
	 foAz2KssgZXhilxojaNxLNTsZXN8FLn58+4DXeFrxVBlZSi/w5sxPrAy2C+SwEUBye
	 oFkUixhRvZYMyrpw1lqrcngtKSfP1jzci+GklfviTJ8CG0mvtrwC5zFu5esY6YpKb5
	 z8FuYJuPQSlq9Ag0TJa9t4x2LvzNpwV9hSXCPrSku4C4K4oEYx+O9lTGkI1+4KE/10
	 X3TtfXavR7dSw==
Date: Mon, 12 Feb 2024 13:44:16 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>, andrew.cooper3@citrix.com, julien@xen.org, 
    bertrand.marquis@arm.com, roger.pau@citrix.com
cc: sstabellini@kernel.org, xen-devel@lists.xenproject.org, s@baer.zone, 
    sebastian.baer@ursus-minor.de, christoph.pirkl@gmail.com, 
    roberto.bagnara@bugseng.com
Subject: OpenFastTrace presentation for the Xen Community 
Message-ID: <alpine.DEB.2.22.394.2402121331540.1925432@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

Hi all,

As we continue our journey toward Xen Safety Certifiability, one of the
next challenges is safety requirements and their traceability. We need
to be able to link high level requirements documents to mid and low
level requirements documents, to test specifications and to gitlab
tests.

Sebastian and Christoph from OpenFastTrace have kindly volunteered to
give a presentation about their tool for the Xen Community.

https://github.com/itsallcode/openfasttrace

Please join the Xen FuSa meeting on Feb 19 at 9AM California time / 5PM
UK to watch the presentation live and for the opportunity to ask
questions to the speakers: https://xilinx.zoom.us/j/92166771973?from=addon

Cheers,

Stefano


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 23:18:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 23:18:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679720.1057344 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZfZT-0007cO-Qe; Mon, 12 Feb 2024 23:18:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679720.1057344; Mon, 12 Feb 2024 23:18:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZfZT-0007cH-Nj; Mon, 12 Feb 2024 23:18:43 +0000
Received: by outflank-mailman (input) for mailman id 679720;
 Mon, 12 Feb 2024 23:18:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rW7E=JV=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rZfZR-0007cB-Rp
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 23:18:41 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0cce5940-c9fd-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 00:18:38 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 0F21C61034;
 Mon, 12 Feb 2024 23:18:37 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7B632C433F1;
 Mon, 12 Feb 2024 23:18:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0cce5940-c9fd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707779916;
	bh=AK4By05/Rz1nzh+xhqhGrd2EuQFerLTXdb2Ja4v9+7M=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=nIltDJqN3NcWVQJHBwfzmbPTESutfe/f6LyAvKdLtACWY3anOYO1Ad4MDn3I0ySvL
	 OGvFpZ2GNyxxoN5I3UArfhOTxpQXU+PnAMLeD+xT/EWrEQ/0T8BCbgTSwPP7Y72pvY
	 d7IOorqgHXvIiReLW91Apga4c2jNmB8/n6ooGeBDLky62v2wp4yMfSwnZQGdyBV1Pr
	 z7EdI/TuitOx8xkRopu3Fqgw0gkcVD79HlT1jkSlheCevQIfD+bLurUb97yWP4wTqk
	 RBSCWQG371RE/H9jeNTCTyX19pFnnJlO576bERCNUiiM/eYeYObhi7EHx+7rZJWKzJ
	 Ujpe7lGm5pHdA==
Date: Mon, 12 Feb 2024 15:18:34 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, andrew.cooper3@citrix.com, 
    george.dunlap@citrix.com, julien@xen.org, bertrand.marquis@arm.com, 
    roger.pau@citrix.com, roberto.bagnara@bugseng.com, 
    federico.serafini@bugseng.com, xen-devel@lists.xenproject.org
Subject: Re: [PATCH v1] docs/misra/rules.rst: catch up with accepted rules
In-Reply-To: <9504e77d-6f52-489c-a91a-f4d1a6ce9a33@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402121512050.1925432@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2402091653110.1925432@ubuntu-linux-20-04-desktop> <9504e77d-6f52-489c-a91a-f4d1a6ce9a33@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 12 Feb 2024, Jan Beulich wrote:
> On 10.02.2024 02:00, Stefano Stabellini wrote:
> > Update docs/misra/rules.rst to reflect the MISRA C rules accepted in the
> > last couple of months.
> > 
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> > ---
> > 
> > In the notes section I added some info about the deviations, but in any
> > case the appropriate info will also be added to deviations.rst,
> > safe.json, etc.
> > 
> > I also added Rule 14.4, which is older, but when I first tried to add it
> > to rules.rst, Jan had a question I couldn't reply clearly:
> > https://marc.info/?l=xen-devel&m=169828285627163
> > 
> > I think now with this series, the impact of Rule 14.4 is clearer:
> > https://marc.info/?l=xen-devel&m=170194257326186
> 
> This series is about enums only afaics. Yet the rule is much wider, and iirc
> we had agreed that for integer and pointer types the normal language
> conversion to boolean meaning is fine as well. Not only do you not mention
> this case in the entry,

I can add a note about it.


> but it also continue to mean that effectively we
> limit the rule to a very narrow case. Which continue to leave open the
> question of whether the rule is worthwhile to accept in the first place.

When someone does a safety certification, there is a difference between
deviating a rule as a whole or accepting the rule and only deviating
certain aspects of it (simply ignoring the rule is typically not an
option in safety certification context.) So here I think it would help
downstreams interested in safety if we added the rule, with specific
deviations.

Do you have any comments on the other parts of this patch? If not, I
would be happy to resent the rest unmodified, and update only 14.4 in
its own separate patch where we can discuss further.


From xen-devel-bounces@lists.xenproject.org Mon Feb 12 23:23:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 23:23:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679724.1057353 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZfdt-0000lw-Ey; Mon, 12 Feb 2024 23:23:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679724.1057353; Mon, 12 Feb 2024 23:23:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZfdt-0000lp-C6; Mon, 12 Feb 2024 23:23:17 +0000
Received: by outflank-mailman (input) for mailman id 679724;
 Mon, 12 Feb 2024 23:23:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eRG0=JV=m5p.com=ehem@srs-se1.protection.inumbo.net>)
 id 1rZfds-0000lj-MS
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 23:23:16 +0000
Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b15e3ec3-c9fd-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 00:23:14 +0100 (CET)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.17.1/8.15.2) with ESMTPS id 41CNN0xY065828
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Mon, 12 Feb 2024 18:23:06 -0500 (EST) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.17.1/8.15.2/Submit) id 41CNN0mY065827;
 Mon, 12 Feb 2024 15:23:00 -0800 (PST) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b15e3ec3-c9fd-11ee-8a4d-1f161083a0e0
Date: Mon, 12 Feb 2024 15:23:00 -0800
From: Elliott Mitchell <ehem+xen@m5p.com>
To: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
        Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: Serious AMD-Vi issue
Message-ID: <ZcqoVBnsgUJw8G0l@mattapan.m5p.com>
References: <ZbLDlRi0vctlhsNp@mattapan.m5p.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZbLDlRi0vctlhsNp@mattapan.m5p.com>
X-Spam-Status: No, score=0.0 required=10.0 tests=KHOP_HELO_FCRDNS
	autolearn=unavailable autolearn_force=no version=4.0.0
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on mattapan.m5p.com

On Thu, Jan 25, 2024 at 12:24:53PM -0800, Elliott Mitchell wrote:
> Apparently this was first noticed with 4.14, but more recently I've been
> able to reproduce the issue:
> 
> https://bugs.debian.org/988477
> 
> The original observation features MD-RAID1 using a pair of Samsung
> SATA-attached flash devices.  The main line shows up in `xl dmesg`:
> 
> (XEN) AMD-Vi: IO_PAGE_FAULT: DDDD:bb:dd.f d0 addr ffffff???????000 flags 0x8 I
> 
> Where the device points at the SATA controller.  I've ended up
> reproducing this with some noticable differences.
> 
> A major goal of RAID is to have different devices fail at different
> times.  Hence my initial run had a Samsung device plus a device from
> another reputable flash manufacturer.
> 
> I initially noticed this due to messages in domain 0's dmesg about
> errors from the SATA device.  Wasn't until rather later that I noticed
> the IOMMU warnings in Xen's dmesg (perhaps post-domain 0 messages should
> be duplicated into domain 0's dmesg?).
> 
> All of the failures consistently pointed at the Samsung device.  Due to
> the expectation it would fail first (lower quality offering with
> lesser guarantees), I proceeded to replace it with a NVMe device.
> 
> With some monitoring I discovered the NVMe device was now triggering
> IOMMU errors, though not nearly as many as the Samsung SATA device did.
> As such looks like AMD-Vi plus MD-RAID1 appears to be exposing some sort
> of IOMMU issue with Xen.
> 
> 
> All I can do is offer speculation about the underlying cause.  There
> does seem to be a pattern of higher-performance flash storage devices
> being more severely effected.
> 
> I was speculating about the issue being the MD-RAID1 driver abusing
> Linux's DMA infrastructure in some fashion.
> 
> Upon further consideration, I'm wondering if this is perhaps a latency
> issue.  I imagine there is some sort of flush after the IOMMU tables are
> modified.  Perhaps the Samsung SATA (and all NVMe) devices were trying to
> execute commands before reloading the IOMMU tables is complete.

Ping!

The recipe seems to be Linux MD RAID1, plus Samsung SATA or any NVMe.

To make it explicit, when I tried Crucial SATA + Samsung SATA.  IOMMU
errors matched the Samsung SATA (a number of times the SATA driver
complained).

As stated, I'm speculating lower latency devices starting to execute
commands before IOMMU tables have finished reloading.  When originally
implemented fast flash devices were rare.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Mon Feb 12 23:29:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 12 Feb 2024 23:29:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679729.1057363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZfjX-0001Qk-2s; Mon, 12 Feb 2024 23:29:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679729.1057363; Mon, 12 Feb 2024 23:29:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZfjX-0001Qd-0F; Mon, 12 Feb 2024 23:29:07 +0000
Received: by outflank-mailman (input) for mailman id 679729;
 Mon, 12 Feb 2024 23:29:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rW7E=JV=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rZfjW-0001NL-94
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 23:29:06 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8277d347-c9fe-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 00:29:05 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id F151A60ECD;
 Mon, 12 Feb 2024 23:29:03 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 606FCC433F1;
 Mon, 12 Feb 2024 23:29:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8277d347-c9fe-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707780543;
	bh=OZmWRyAZ96aIiqGUnApBj9/GzU1LECVikVIx9OJtviQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=sUhjCLeyJQSC+hYpQL8kqbT2BVhPqwCcgcfsMisIAG+uQqimRkMNRF9PL9cfIRKWh
	 yRUbEFI3XOWQKFB1UR0dJ93ghozW/fXhgrK30ChKeAve0qshx7Wc1y+sXH7Ay3TZnO
	 wkgJg07KcZaPhuYqRXw7wJrwi4jp95t7xecibBbo8B8ZPlhcnJ68jwo9iuOql/9IrY
	 EopbupN+xxGkeFL29wrjUI15kTwGaCaygI4nAdrKVLMaTQVfdOsNmov+B3+ehDu+Im
	 iQIqRFZP5eT+ZBuUJjvfoMHuqhyiGdZ78Uza/0kXjFxS7X1f6EzKp7sLoDlhMyjDst
	 ncPa8BgzLi+mQ==
Date: Mon, 12 Feb 2024 15:29:00 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Daniil Dulov <d.dulov@aladdin.ru>
cc: Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>, 
    xen-devel@lists.xenproject.org, iommu@lists.linux-foundation.org, 
    linux-kernel@vger.kernel.org, lvc-project@linuxtesting.org
Subject: Re: [PATCH] swiotlb-xen: remove redundant NULL check
In-Reply-To: <20240211150958.4112-1-d.dulov@aladdin.ru>
Message-ID: <alpine.DEB.2.22.394.2402121528340.1925432@ubuntu-linux-20-04-desktop>
References: <20240211150958.4112-1-d.dulov@aladdin.ru>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

Hi Daniil,

Against which Linux branch was this patch generated?

Cheers,

Stefano


On Sun, 11 Feb 2024, Daniil Dulov wrote:
> In this case hwdev cannot be NULL, so remove redundant NULL check.
> 
> Found by Linux Verification Center (linuxtesting.org) with SVACE.
> 
> Fixes: b097186fd29d ("swiotlb-xen: SWIOTLB library for Xen PV guest with PCI passthrough.")
> Signed-off-by: Daniil Dulov <d.dulov@aladdin.ru>
> ---
>  drivers/xen/swiotlb-xen.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
> index 2b385c1b4a99..b166f6efea26 100644
> --- a/drivers/xen/swiotlb-xen.c
> +++ b/drivers/xen/swiotlb-xen.c
> @@ -305,7 +305,7 @@ xen_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
>  	if (!ret)
>  		return ret;
>  
> -	if (hwdev && hwdev->coherent_dma_mask)
> +	if (hwdev->coherent_dma_mask)
>  		dma_mask = hwdev->coherent_dma_mask;
>  
>  	/* At this point dma_handle is the dma address, next we are
> -- 
> 2.25.1
> 


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 02:57:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 02:57:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679735.1057373 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZiyr-0007LQ-Kb; Tue, 13 Feb 2024 02:57:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679735.1057373; Tue, 13 Feb 2024 02:57:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZiyr-0007LJ-HQ; Tue, 13 Feb 2024 02:57:09 +0000
Received: by outflank-mailman (input) for mailman id 679735;
 Tue, 13 Feb 2024 02:57:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZiyq-0007L7-E6; Tue, 13 Feb 2024 02:57:08 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZiyq-0003L8-BZ; Tue, 13 Feb 2024 02:57:08 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZiyp-0001lb-Vj; Tue, 13 Feb 2024 02:57:08 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZiyp-0004Vx-Tu; Tue, 13 Feb 2024 02:57:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=U8LNlYZpwoPnBKr45SRVRTXA6SD0Fj9AowiW8jjcycU=; b=AIovK6aaVomQyBVccYD9J19+ua
	NIHCb4Z+A69VTxrOzKVf8HfyINqukpzqOwSxvuuD0pXkapgq5V0176sg7QOHLIL9R2lfk2CSpa4Uk
	UveWJy59leQbkUC7iR8JocGjXEZ58opXZbGql6x/dvFIBMg0QVYjQDmlUcKhXOg3VHm8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184652-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184652: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=716f4aaa7b48a55c73d632d0657b35342b1fefd7
X-Osstest-Versions-That:
    linux=841c35169323cd833294798e58b9bf63fa4fa1de
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 13 Feb 2024 02:57:07 +0000

flight 184652 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184652/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184649
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184649
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184649
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184649
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184649
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184649
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184649
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184649
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                716f4aaa7b48a55c73d632d0657b35342b1fefd7
baseline version:
 linux                841c35169323cd833294798e58b9bf63fa4fa1de

Last test of basis   184649  2024-02-12 03:12:36 Z    0 days
Testing same since   184652  2024-02-12 17:43:54 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Amir Goldstein <amir73il@gmail.com>
  Christian Brauner <brauner@kernel.org>
  Linus Torvalds <torvalds@linux-foundation.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   841c35169323..716f4aaa7b48  716f4aaa7b48a55c73d632d0657b35342b1fefd7 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 03:44:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 03:44:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679741.1057384 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZjiD-0004yW-5k; Tue, 13 Feb 2024 03:44:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679741.1057384; Tue, 13 Feb 2024 03:44:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZjiD-0004yP-1i; Tue, 13 Feb 2024 03:44:01 +0000
Received: by outflank-mailman (input) for mailman id 679741;
 Tue, 13 Feb 2024 03:43:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zAib=JW=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1rZjiB-0004yJ-50
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 03:43:59 +0000
Received: from wfout2-smtp.messagingengine.com
 (wfout2-smtp.messagingengine.com [64.147.123.145])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1b0a1464-ca22-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 04:43:56 +0100 (CET)
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
 by mailfout.west.internal (Postfix) with ESMTP id BAE641C00091;
 Mon, 12 Feb 2024 22:43:50 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute3.internal (MEProxy); Mon, 12 Feb 2024 22:43:51 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 12 Feb 2024 22:43:48 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b0a1464-ca22-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1707795830;
	 x=1707882230; bh=96KYMZfSEqyIRUJtvm5SXgTmTOy/Iu9skirSS8EXVb4=; b=
	DPQgQPdtfTlaJyhbGKM4tr/ZhScmN6LhBk+Xw/P/lVo4iEMjxp3yeFVeY2HWVl8s
	qEUgNwBtiA4td7gOLOaNEcXu6KUN4zgs+FuL8APSD47WwqYxe7U5x+VtE5uhnr1J
	uNtpPnaUY8f2BAStdtUFr2+Al/drQdDFtX/NVP8+YNeYe68/9EaGaA6FPSjP3T4z
	9cZrlJH3J0+vUCTV4T7OBwoBA2cDl+sKeBCRs4m3rQTcVoVManHyCmNK9R8SM92X
	4Sof+3V4lcvpmD9/LybVohtWOvQxzVCZQR8lJdx31CO76mwHjzurvMmcHgSfBR+b
	hqaln//SfXZfSW5yCtPmSg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1707795830; x=1707882230; bh=96KYMZfSEqyIRUJtvm5SXgTmTOy/
	Iu9skirSS8EXVb4=; b=dQRgZojgePfM8Oyemh8Qg/L5RmwccRzn/CniCfJ4g+uu
	4srrUEGKwmqzoDSYQdtPLrLDvudrfuDJ2L5TC4IK4bwsYqbTT5gOPxe+6C5xJ4wC
	gGg1oNRhfu/loUo5tMDwYd5mxIUwt+guHylIOSmmG7k7Cw/2cUnGVxmuSpaCrSFB
	dqUS+Ly4IQAqD1LT6zAsYE+b3iCnfa8km59bvIZo8EzjM51NAECJPGflBqlhNoZr
	MY3Qm/L0DxWU2SGI6Eii0s/9HY2VlCwH/P9LFZbvogO6MMgmXM5G77AVwIJEY6v2
	n5t2LHCoQywUHPLNEQUJgp59vNLFUjATqgf5sch9eQ==
X-ME-Sender: <xms:deXKZTQVoadRmEaaulA5DuREve69b0Rr_uutm0ByHcLxRSTs9VYN7A>
    <xme:deXKZUzbiATMMGB9kwQsZRhgSmMAB7jmMifiuALrVRm6z4YwBy-X27eG9Oh_I7rqk
    p3nISuErc0dPw>
X-ME-Received: <xmr:deXKZY3_u-xjzpSan45SkMadHxlkHl7ItRMFkVCPN3N2OHsVUf3W4gcugYXqSuoo-Q0SVsISEImyT54uzskyxBlgpgp9dAXyYA>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeggdeihecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpeffhffvvefukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihcuoehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthh
    hinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeeihfelhfekhefgveejjeet
    vedtgfduveelgeekteehueeufeefudegveejueefhfenucffohhmrghinhepghhithhhuh
    gsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho
    mhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:deXKZTDpg9T_hG2IRhBUkDmCkij6tIT_aom9wYLD2UaYU6i-Gf1Ohg>
    <xmx:deXKZcgtL9SmBpZq91hS0trEUg4y1m3rWieQ9hMYJ-X-18yMxj0JwA>
    <xmx:deXKZXrKfuA0EUe3r9AzHBjr8HGJp6V23qgMpKMJ0PzP1rJMZV5iNw>
    <xmx:duXKZXUvDpmU-uZmB8GsmfleiRGV0ZwdnKY4xTr36PtakUV7D-FsUzXG_CM>
Feedback-ID: i1568416f:Fastmail
Date: Tue, 13 Feb 2024 04:43:46 +0100
From: Marek Marczykowski <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Message-ID: <ZcrlcuNU9y8WymiK@mail-itl>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
 <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
 <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="VRYmdlhezWG4ObHs"
Content-Disposition: inline
In-Reply-To: <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com>


--VRYmdlhezWG4ObHs
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 13 Feb 2024 04:43:46 +0100
From: Marek Marczykowski <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers

On Mon, Feb 12, 2024 at 10:04:38AM +0100, Jan Beulich wrote:
> On 08.02.2024 23:00, Julien Grall wrote:
> > On 05/02/2024 13:27, Jan Beulich wrote:
> >> In preparation of dropping the register parameters from
> >> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
> >> register state needs making available another way for the few key
> >> handlers which need it. Fake IRQ-like state.
> >>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >> ---
> >> The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsistent wi=
th
> >> other console poll functions we have, and it's unclear whether that's
> >> actually generally correct.
> >=20
> > Is it? Looking at ns16550_poll() we would pass guest_user_regs() if=20
> > run_in_exception() doesn't exist. But looking at the caller, no-on seem=
s=20
> > to care about the 'regs'. So is this just a latent bug?
>=20
> What do you mean by "doesn't exist"? ns16550_poll() assumes it exists.
> And I can spot any use of guest_user_regs() on the respective generic
> or Arm-specific bug.c paths.
>=20
> > BTW, do you have an idea why the poll function is not run in an=20
> > exception handler?
>=20
> "The poll function" being which one? If you mean the one in xhci-dbc.c
> then that's why I had Cc-ed Marek. Moving him to To: - maybe that
> manages to finally catch his attention.

TBH, I don't know. That's part of the original xue patch at
https://github.com/connojd/xue/blob/master/patches/xen-xue-dbgp.patch
and it works for me as it is.

> >> Andrew suggested to move set_irq_regs() to BUGFRAME_run_fn handling;
> >> it's not clear to me whether that would be (a) correct from an abstract
> >> pov (that's exception, not interrupt context after all)=20
> >=20
> > I agree with that.
> >=20
> >> and (b) really beneficial.
> >=20
> > I guess this could help to reduce the amount of churn. I can't really=
=20
> > make my mind whether this is worth it or not. So I would keep it as you=
 did.
>=20
> Good, thanks.
>=20
> Jan

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--VRYmdlhezWG4ObHs
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXK5XIACgkQ24/THMrX
1yxyxQf/fYWlS65gDO42ftVko/W0ePGo4k8FIO3VFzxzBulu0VIh3Lf+L9CxIjgj
1flsR3OZ9r06z6CvcZocuBRnb6miXJzB0LrcLjroLVLrGvrMVpUfukTVrDSxfty4
XM0AFAFnyAwe60XMjmWdKjBW6r8a5pSA56jCNuwpiH4vlMK2kyAUhOPwwl1QU0vd
D4qo2ufxzJ2zEgcFjTg7JEEmiUhoSM46rNteCC84lT8+nl04msauN9UlWkDIKdhw
yJ67zRtAJ80k1RJY/ZSaHOqqZIAE6nw7dkXOFLQV+kE9J+2ZXqPinHZ3IGEmQCMq
9w/Tus0VKAIKiuX/OpI0bZVwHVRJlQ==
=dVoS
-----END PGP SIGNATURE-----

--VRYmdlhezWG4ObHs--


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 06:13:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 06:13:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679749.1057406 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZm2w-0005qS-MI; Tue, 13 Feb 2024 06:13:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679749.1057406; Tue, 13 Feb 2024 06:13:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZm2w-0005qL-J1; Tue, 13 Feb 2024 06:13:34 +0000
Received: by outflank-mailman (input) for mailman id 679749;
 Tue, 13 Feb 2024 06:13:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZm2v-0005qB-78; Tue, 13 Feb 2024 06:13:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZm2v-0007VZ-4I; Tue, 13 Feb 2024 06:13:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZm2u-0001hi-Mi; Tue, 13 Feb 2024 06:13:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZm2u-0006SA-M9; Tue, 13 Feb 2024 06:13:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=FiS1cicKEJgVE/AvssZh1tvQebhds4EM0e+5Xc+C6KU=; b=agMMHYZ1s409yKERucnenAqsYJ
	5ZXTuQojfs1yoYts9adpvRwMdviIuv3Gd4y+cYW6Y6eY69ZF8Wc5GBVzyeNZeSAFhttK1tqU2KuxH
	g8MbgDBeV9/TKk3WeiKKrQe7qifcicce723GftRWvZ8h83qiu0U7cBvShFwaZ4UKxrPE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184653-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184653: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=772380f7547bac7e4c10c9ceff87af3f54151ae5
X-Osstest-Versions-That:
    xen=4e5bd87d24ac2b08f33c6a9c0d530daef99ed369
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 13 Feb 2024 06:13:32 +0000

flight 184653 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184653/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184648
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184648
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184648
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184648
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184648
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184648
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184648
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184648
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184648
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184648
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184648
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184648
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  772380f7547bac7e4c10c9ceff87af3f54151ae5
baseline version:
 xen                  4e5bd87d24ac2b08f33c6a9c0d530daef99ed369

Last test of basis   184648  2024-02-12 01:53:55 Z    1 days
Testing same since   184651  2024-02-12 11:39:03 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Petr Beneš <w1benny@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Simone Ballarin <simone.ballarin@bugseng.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   4e5bd87d24..772380f754  772380f7547bac7e4c10c9ceff87af3f54151ae5 -> master


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 06:16:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 06:16:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679700.1057415 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZm5y-0006Ms-4U; Tue, 13 Feb 2024 06:16:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679700.1057415; Tue, 13 Feb 2024 06:16:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZm5y-0006Ml-1m; Tue, 13 Feb 2024 06:16:42 +0000
Received: by outflank-mailman (input) for mailman id 679700;
 Mon, 12 Feb 2024 18:51:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=jcaA=JV=knorrie.org=hans@srs-se1.protection.inumbo.net>)
 id 1rZbOZ-0002Ce-1a
 for xen-devel@lists.xenproject.org; Mon, 12 Feb 2024 18:51:11 +0000
Received: from albatros.knorrie.org (albatros.knorrie.org [35.156.0.23])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ae91aca5-c9d7-11ee-8a4c-1f161083a0e0;
 Mon, 12 Feb 2024 19:51:08 +0100 (CET)
Received: from [IPV6:4000:0:fb:a::15]
 (2a02-a213-2bc5-6a80-0000-0000-0000-0004.cable.dynamic.v6.ziggo.nl
 [IPv6:2a02:a213:2bc5:6a80::4])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384)
 (No client certificate requested)
 by albatros.knorrie.org (Postfix) with ESMTPSA id 0E66D30BA8;
 Mon, 12 Feb 2024 19:51:06 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae91aca5-c9d7-11ee-8a4c-1f161083a0e0
Message-ID: <509f476b-1c0a-401a-a564-28490c96e2ab@knorrie.org>
Date: Mon, 12 Feb 2024 19:51:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Bug#1063270: The "64bits time_t transition" in Debian/Xen
To: Andrew Cooper <andrew.cooper3@citrix.com>, 1063270@bugs.debian.org,
 zithro <slack@rabbit.lu>, xen-devel@lists.xenproject.org,
 Maximilian Engelhardt <maxi@daemonizer.de>
References: <51e993f4-c250-400c-bbf4-017a1dbf1c2d@rabbit.lu>
 <170717408685.1943167.14081331758881904940.reportbug@homer.dodds.net>
 <2922b179-571a-4745-b204-7dc7e2beeeea@citrix.com>
Content-Language: en-US
From: Hans van Kranenburg <hans@knorrie.org>
In-Reply-To: <2922b179-571a-4745-b204-7dc7e2beeeea@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hi,

On 2/12/24 18:43, Andrew Cooper wrote:
> On 12/02/2024 5:27 pm, zithro wrote:
>> Hey all,
>>
>> the Debian project is focused on the "2038 time_t" switch.
>> So the maintainers of the Debian Xen package must ensure that all
>> imported Xen code conforms to the new Debian standards.
>>
>> I was asked by Andrew Cooper to post here about this, I'll quote him :
>> "So I had been idly wondering whether Xen would match up to Debian's new
>> policy, and it appears not
>> this topic really needs to be brought up on the xen-devel mailing list
>> do you have any more details as to what has gone wrong?
>> this is something we ought to arrange to happen in CI by default
>> but it sounds like there's some work needed first"
>>
>> (Not answering the question because I'm just a messenger).
> 
> xen.git/xen$ git grep -w time_t -- :/
> ../tools/console/client/main.c:106:     time_t start, now;
> ../tools/console/daemon/io.c:272:       time_t now = time(NULL);
> ../tools/libs/light/libxl_qmp.c:116:    time_t timeout;
> ../tools/libs/light/libxl_qmp.c:585:                               
> time_t ask_timeout)
> ../tools/libs/light/libxl_x86.c:516:        time_t t;
> ../tools/libs/toollog/xtl_logger_stdio.c:61:        time_t now = time(0);
> ../tools/tests/xenstore/test-xenstore.c:453:    time_t stop;
> ../tools/xenmon/xenbaked.c:98:time_t start_time;
> ../tools/xenstored/core.c:109:  time_t now;
> ../tools/xenstored/core.h:150:  time_t ta_start_time;
> ../tools/xenstored/domain.c:143:        time_t mem_last_msg;
> ../tools/xenstored/domain.c:188:static time_t wrl_log_last_warning; /*
> 0: no previous warning */
> ../tools/xenstored/domain.c:1584:       time_t now;
> ../tools/xenstored/lu.c:160:    time_t now = time(NULL);
> ../tools/xenstored/lu.c:185:    time_t now = time(NULL);
> ../tools/xenstored/lu.c:292:    time_t now = time(NULL);
> ../tools/xenstored/lu.h:32:     time_t started_at;
> ../tools/xentop/xentop.c:947:   time_t curt;
> ../tools/xl/xl_info.c:742:static char *current_time_to_string(time_t now)
> ../tools/xl/xl_info.c:759:static void print_dom0_uptime(int short_mode,
> time_t now)
> ../tools/xl/xl_info.c:810:static void print_domU_uptime(uint32_t domuid,
> int short_mode, time_t now)
> ../tools/xl/xl_info.c:847:    time_t now;
> ../tools/xl/xl_vmcontrol.c:336:        time_t start;
> ../tools/xl/xl_vmcontrol.c:495:    time_t now;
> ../tools/xl/xl_vmcontrol.c:504:    if (now == ((time_t) -1)) {
> ../tools/xs-clients/xenstore_control.c:33:    time_t time_start;
> arch/x86/cpu/mcheck/mce.h:224:    uint64_t time;     /* wall time_t when
> error was detected */
> arch/x86/time.c:1129: * machines were long is 32-bit! (However, as
> time_t is signed, we
> 
> 
> I don't see any ABI problems from using a 64bit time_t.  The only header
> file with a time_t is xenstored/lu.h which is a private header and not a
> public ABI.
> 
> I guess we fell into the "could not be analysed via
> abi-compliance-checker" case?

Thanks for also looking into this!

Maximilian mentioned in #debian-xen that doing a Debian package build
with DEB_BUILD_OPTIONS=abi=+lfs and _FILE_OFFSET_BITS=64 and
_TIME_BITS=64 resulted in the exact same binaries for shared libs.

What we also found is these reports:

1. Enabling lfs, which has no effect:
https://adrien.dcln.fr/misc/armhf-time_t/2024-02-06T16%3A48%3A00/compat_reports/libxen-dev/base_to_lfs/compat_report.html

2. Enabling the 64-bit time_t as well:
https://adrien.dcln.fr/misc/armhf-time_t/2024-02-06T16%3A48%3A00/compat_reports/libxen-dev/lfs_to_time_t/compat_report.html
In there, see "Problems with Data Types, Low Severity  2 " about
struct_timeval:

---- >8 ----

  [+] struct timeval
    Change -> Effect
    1 Type of field tv_sec has been changed from __time_t to __time64_t.
-> Recompilation of a client program may be broken.
    2 Type of field tv_usec has been changed from __suseconds_t to
__suseconds64_t. -> Recompilation of a client program may be broken.

  [+] affected symbols: 3 (0.2%)
    * libxl_osevent_afterpoll ( libxl_ctx* ctx, int nfds, struct pollfd
const* fds, struct timeval now ) -> 4th parameter 'now' is of type
'struct timeval'.
    * libxl_osevent_beforepoll ( libxl_ctx* ctx, int* nfds_io, struct
pollfd* fds, int* timeout_upd, struct timeval now ) -> 5th parameter
'now' is of type 'struct timeval'.
    * libxl_osevent_register_hooks ( libxl_ctx* ctx, libxl_osevent_hooks
const* hooks, void* user ) -> Field 'hooks.timeout_modify.p2' in 2nd
parameter 'hooks' (pointer) has base type 'struct timeval'.

---- >8 ----

So, the question is, is this correct and would it cause a problem.

If so, it also means that those functions are in a versioned lib,
libxenlight.so.4.17.0 (in binary package libxenmisc4.17).

Coincidentally, we are currently preparing the upload to switch from Xen
4.17 to Xen 4.18 in Debian unstable. So, if we just go ahead with doing
that, and make sure it's built in the new way already...

then...

tada.wav!

We just immediately have the correct libxenmisc4.18, and no other
(stable lib) packages have to be renamed.

Hans


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 07:14:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 07:14:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679759.1057425 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZmzT-0004qb-9T; Tue, 13 Feb 2024 07:14:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679759.1057425; Tue, 13 Feb 2024 07:14:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZmzT-0004qU-6W; Tue, 13 Feb 2024 07:14:03 +0000
Received: by outflank-mailman (input) for mailman id 679759;
 Tue, 13 Feb 2024 07:14:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZmzS-0004qM-5a
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 07:14:02 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 74eba9bb-ca3f-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 08:13:59 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-40efcb37373so32067255e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 23:13:59 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 dx9-20020a05600c63c900b0040d5ae2906esm10679670wmb.30.2024.02.12.23.13.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 23:13:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74eba9bb-ca3f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707808439; x=1708413239; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vy6WUO8JrMmJ5Heu8bApmAce3OPcq0UizfnHuhdaLT8=;
        b=Q2o8wqGayXJK+15CQc0D0o/y+KnC0vafh9vZ3H3Y2rOfUYsxnjNOHJXp0wgsD0VVrK
         KfUtCjaS8zzWdQLlmVAFU2Cyl889dBuh/qd+s0jChHZ3JWEWwYg2zoXsEv0mXuElY8mJ
         7TCMldHWVwvEuEOBk8YXhAgsr8QITnmBdDYWeHYhFyhHbQX/UVl1+SL/mOIABS2qVKsA
         xfAjdhO2rLEU4kXXiOHfkBe6DJsDUNtWAQ+UEWtBUkC5MWwcr8iLjR+jPx7Jkpy/Q0Hy
         h6Il0vBirukYo/oPeeSDj7Sp4oHHFx03TGQfJ2sSqP/XeijGZg3JwfhV1YTOSBmQeV/O
         5pIQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707808439; x=1708413239;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vy6WUO8JrMmJ5Heu8bApmAce3OPcq0UizfnHuhdaLT8=;
        b=Ie65z4Co0W/n0ZPnRcXKUFflf6nJP16/NOSwX5fiEQ4GktSMnzN8VNyeLHsQSK3wIh
         0RrIaVFCs//BNc04WiyqODkn8Yl+dZCEMRsH7HA9HPp+CZxIZEBXZdmpM+oKNDKZObnL
         2sPo1Oztt4oUzUUDL2wc0j6DRoL9s6yKcpowrgVTWjKaCQmuJjaKFbwm86a8uuVHvTDn
         xv5Dl7RWIlUlryrNNphAI/AzGKCtsBk2zb62r0xrSWP26+AoYztBMJO5Y61gJJdRvQd1
         fquv1UdgtXCuVUAEngA455fJp3N9OQ7ICE0ILaK3RJBUbbzedAYX0c7XWiIEydnNscMY
         eDjQ==
X-Forwarded-Encrypted: i=1; AJvYcCX7NE7ySpdcte6fvoufRfjkIS+neqs4TtZe/Euu7t7XfdbKj8z+tVEmRF+s/8wU0DvnY2z2zKku3Flt+HG2q9F2CYJ4qVyl349YBLrSucw=
X-Gm-Message-State: AOJu0YyW73h03lMYonkBhMDJGnfaEkcmVZVKk/4lEPEOvh8//O7BUnJW
	4o8nFp2stZmZeBW/GNmSh4jiPVE4/69r6axqio2EFJugDYvZuKBE6eFbYaVQ+w==
X-Google-Smtp-Source: AGHT+IGz4wUCsMCCEoETO//hhIM4n7UB+6AjU0NUagPLKVEqUn7qp+DCLv5j/UXdULcAoaXmxKnNyQ==
X-Received: by 2002:a05:600c:45c8:b0:410:d71b:19ae with SMTP id s8-20020a05600c45c800b00410d71b19aemr3107896wmo.25.1707808438946;
        Mon, 12 Feb 2024 23:13:58 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUD7WMYI4qxZ0Tq+n0m+NYSyGzvc5d+fRERJL/LsaTvSPLC3qCdcATlX6KA4LQPYId3E81go0toNdjAgMDW8/yhc21IfTGDZWW7VQPwi3HD92AYtRSeEbdHQ86sGVLwR4z1D9MTbAk7zoYmTON/YRmWFQEdoMIat8YNgD/yAk+BVHsbhuwmr6T0n2n4gv3pKK1RX7Y9JmTy6x4PdzVGAw/PVwO7h4hrzbzBjuCnyMRB+jwyKjPRTRhlJY9mhMgQUabI5jR6/gXRBwHqg+o7AKjbNTUWryHUn3B/W6ZawWjddHDOOKYotar2/xmGOX5WHqpWZjESn3+ISC8Qqzj8FjwnGhbwyDUtkIg0
Message-ID: <c0b8f2ec-a3bb-45a6-b748-052d55a6fd5c@suse.com>
Date: Tue, 13 Feb 2024 08:13:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Return type of clean_and_invalidate_dcache_va_range
Content-Language: en-US
To: Julien Grall <julien@xen.org>, Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Xen Devel <xen-devel@lists.xenproject.org>,
 Consulting <consulting@bugseng.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper3 <andrew.cooper3@citrix.com>, Roger Pau
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>
References: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
 <alpine.DEB.2.22.394.2402091402080.1925432@ubuntu-linux-20-04-desktop>
 <f86baad0-f113-4156-9c10-6910e8c63492@xen.org>
 <02446758-a5a9-4a8a-8ce6-15abdc9ede65@suse.com>
 <99faac70440a68824a17fcaaea55ef48@bugseng.com>
 <09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 12.02.2024 19:38, Julien Grall wrote:
> An alternative would be to introduced arch_grant_cache_flush() and move 
> the if/else logic there. Something like:
> 
> diff --git a/xen/arch/arm/include/asm/page.h 
> b/xen/arch/arm/include/asm/page.h
> index 69f817d1e68a..4a3de49762a1 100644
> --- a/xen/arch/arm/include/asm/page.h
> +++ b/xen/arch/arm/include/asm/page.h
> @@ -281,6 +281,19 @@ static inline void write_pte(lpae_t *p, lpae_t pte)
>       dsb(sy);
>   }
> 
> +static inline arch_grant_cache_flush(unsigned int op, const void *p, 
> unsigned long size)
> +{
> +    unsigned int order = get_order_from_bytes(size);
> +
> +    if ( (cflush->op & GNTTAB_CACHE_INVAL) && (cflush->op & 
> GNTTAB_CACHE_CLEAN) )
> +        clean_and_invalidate_dcache_va_range(v, cflush->length);
> +    else if ( cflush->op & GNTTAB_CACHE_INVAL )
> +        invalidate_dcache_va_range(v, cflush->length);
> +    else if ( cflush->op & GNTTAB_CACHE_CLEAN )
> +        clean_dcache_va_range(v, cflush->length);
> +
> +    return 0;
> +}
> 
>   /* Flush the dcache for an entire page. */
>   void flush_page_to_ram(unsigned long mfn, bool sync_icache);
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 424744ad5e1a..647e1522466d 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -735,8 +735,7 @@ void asmlinkage __init start_xen(unsigned long 
> boot_phys_offset,
>                 fdt_paddr);
> 
>       /* Register Xen's load address as a boot module. */
> -    xen_bootmodule = add_boot_module(BOOTMOD_XEN,
> -                             virt_to_maddr(_start),
> +    xen_bootmodule = add_boot_module(BOOTMOD_XEN, virt_to_maddr(_start),
>                                (paddr_t)(uintptr_t)(_end - _start), false);
>       BUG_ON(!xen_bootmodule);
> 
> diff --git a/xen/arch/x86/include/asm/flushtlb.h 
> b/xen/arch/x86/include/asm/flushtlb.h
> index bb0ad58db49b..dfe51cddde90 100644
> --- a/xen/arch/x86/include/asm/flushtlb.h
> +++ b/xen/arch/x86/include/asm/flushtlb.h
> @@ -182,23 +182,22 @@ void flush_area_mask(const cpumask_t *mask, const 
> void *va,
>   }
> 
>   static inline void flush_page_to_ram(unsigned long mfn, bool 
> sync_icache) {}
> -static inline int invalidate_dcache_va_range(const void *p,
> -                                             unsigned long size)
> -{ return -EOPNOTSUPP; }
> -static inline int clean_and_invalidate_dcache_va_range(const void *p,
> -                                                       unsigned long size)
> +
> +unsigned int guest_flush_tlb_flags(const struct domain *d);
> +void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask);
> +
> +static inline arch_grant_cache_flush(unsigned int op, const void *p, 
> unsigned long size)
>   {
> -    unsigned int order = get_order_from_bytes(size);
> +    unsigned int order;
> +
> +    if ( !(cflush->op & GNTTAB_CACHE_CLEAN) )
> +        return -EOPNOTSUPP;
> +
> +    order = get_order_from_bytes(size);
>       /* sub-page granularity support needs to be added if necessary */
>       flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
> +
>       return 0;
>   }
> -static inline int clean_dcache_va_range(const void *p, unsigned long size)
> -{
> -    return clean_and_invalidate_dcache_va_range(p, size);
> -}
> -
> -unsigned int guest_flush_tlb_flags(const struct domain *d);
> -void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask);
> 
>   #endif /* __FLUSHTLB_H__ */
> 
> I have a slight preference for the latter. I would like to hear the 
> opinion of the others.

I would prefer this 2nd form, too, assuming the setup.c change wasn't
really meant to be there. The one thing that doesn't become clear: In
the sketch above arch_grant_cache_flush() has no return type, yet has
"return 0". This raises a question towards the one that's at the root
of this thread: Do you mean the function to have a return value, and
if so will it be (sensibly) used?

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 07:38:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 07:38:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679764.1057436 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZnMo-0007mF-Am; Tue, 13 Feb 2024 07:38:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679764.1057436; Tue, 13 Feb 2024 07:38:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZnMo-0007m8-89; Tue, 13 Feb 2024 07:38:10 +0000
Received: by outflank-mailman (input) for mailman id 679764;
 Tue, 13 Feb 2024 07:38:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZnMn-0007m2-Ai
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 07:38:09 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d494857d-ca42-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 08:38:08 +0100 (CET)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-337d05b8942so3357945f8f.3
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 23:38:08 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bt25-20020a056000081900b0033b792ed609sm6602714wrb.91.2024.02.12.23.38.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 23:38:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d494857d-ca42-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707809887; x=1708414687; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=g2TUnxqUFEUuds5VoosuLXLTLztNDPp3bX3riYGqOcU=;
        b=UIVn8kEqWVMUUwQ9tAT2zyZobITnbsym+pqVwf8MWrD/Vyt489CCVLheJLTdQPklaP
         TApozAVn6tAHmqLkRLJu3In9Ok5f3qv0vQi7W7YQe6gew2H+MB5wMDfKLKuZwYc9lHZQ
         KASaRtKVZ1Ya/kLjpCy04gXe+a++3AwJTYkg/rhD5U2+XYfa1quAiyVGmVnO1a3JLBDn
         8jQFQk8K6vcrB7FVf3DpfU4o3BGUt4ibTGUIPoKwv7AwknBJVuXmZH+5ibmLDtEv6OoC
         0kkUfrcPCix1n07LdOfJgHYtV4F1uinnpDFIfvdF/uQNxqGRqUJfXqt5RDwmrRa5bXR3
         a59Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707809887; x=1708414687;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=g2TUnxqUFEUuds5VoosuLXLTLztNDPp3bX3riYGqOcU=;
        b=pEAUQaEVz6nb9bTaGX/snxEW4z64dz+4wT+72+d4m5/ILIbzKGbsZvzYMeIjSq5MY4
         0t0f9t1NVWHjTOG18+dMro9nwODHxMkz1liPy0UTGdxE9To/RjQoUevfxrr7KozcLvq9
         EwdHWyOhCCCfUpz9zIHIKOGI87YtRo/NBRhDiiG6iWvL049IiCb4qzZYRPrknL7qQZ/v
         WZnc2KUPN9m9cZz7p8mHxIIPG3dyQQzdJg8mSqfofas4AfmZrHjrLVuv2AJ5DVk1eulS
         lqOvXUsHOhWJws2aAFMLyJk9PBIuluPlXhVb3BEG7yl6XAVG9FrSqxAob2bIujOf8aBh
         KFQg==
X-Gm-Message-State: AOJu0Yy8XNeGyUMxlwrUj93sBikVJmjCuCK1VojR2Hd2QSw6IkwhfAOX
	5EFMxDJjPtLEyyCRnftGRSiVsep0SpHB6ZZdfta/XJS+6BBaTBs7+1q6exAjOQ==
X-Google-Smtp-Source: AGHT+IFVmCk/GcVZ1U5c6YbSrUVLcj5Me8xy5p8vSQCPar7Dm48mdRqUMDtFPSVxXpCJZluTWtHsWQ==
X-Received: by 2002:adf:e68a:0:b0:33b:433d:e1d7 with SMTP id r10-20020adfe68a000000b0033b433de1d7mr6639764wrm.1.1707809887381;
        Mon, 12 Feb 2024 23:38:07 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVVzJVHDs1sbXPfI79/DDojUlFZPHXEugtMnUi5pwNgHTUHFCWQp6fVVDkmeX0L2+lWo4n3WP1pLk0B7QbdDk4J3RMlf/zFB9lGVdCHNWbJdl55WdbggEmpJPeY4WzWL4hGjp4QCnWBAVMiTTOiGgFut0K6KvXShuA8g1FQLlyJYdH9aC3ewBaPv0P3x+Qk2ez7W4Jm+//8Gt4n/s8hdz7Bz2Bgtdjl/hrhltiDxRyevoM0Mrew5YENCZv9IE6+EPwnv0ZkutXatal9t5Kqxy2AiiD0Gjs9QSxr+V9gGv6E5/nJxcs=
Message-ID: <d7295c43-c415-4113-8d9c-e6e3d688763e@suse.com>
Date: Tue, 13 Feb 2024 08:38:06 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v1] docs/misra/rules.rst: catch up with accepted rules
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, julien@xen.org,
 bertrand.marquis@arm.com, roger.pau@citrix.com, roberto.bagnara@bugseng.com,
 federico.serafini@bugseng.com, xen-devel@lists.xenproject.org
References: <alpine.DEB.2.22.394.2402091653110.1925432@ubuntu-linux-20-04-desktop>
 <9504e77d-6f52-489c-a91a-f4d1a6ce9a33@suse.com>
 <alpine.DEB.2.22.394.2402121512050.1925432@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402121512050.1925432@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.02.2024 00:18, Stefano Stabellini wrote:
> On Mon, 12 Feb 2024, Jan Beulich wrote:
>> On 10.02.2024 02:00, Stefano Stabellini wrote:
>>> Update docs/misra/rules.rst to reflect the MISRA C rules accepted in the
>>> last couple of months.
>>>
>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
>>> ---
>>>
>>> In the notes section I added some info about the deviations, but in any
>>> case the appropriate info will also be added to deviations.rst,
>>> safe.json, etc.
>>>
>>> I also added Rule 14.4, which is older, but when I first tried to add it
>>> to rules.rst, Jan had a question I couldn't reply clearly:
>>> https://marc.info/?l=xen-devel&m=169828285627163
>>>
>>> I think now with this series, the impact of Rule 14.4 is clearer:
>>> https://marc.info/?l=xen-devel&m=170194257326186
>>
>> This series is about enums only afaics. Yet the rule is much wider, and iirc
>> we had agreed that for integer and pointer types the normal language
>> conversion to boolean meaning is fine as well. Not only do you not mention
>> this case in the entry,
> 
> I can add a note about it.
> 
> 
>> but it also continue to mean that effectively we
>> limit the rule to a very narrow case. Which continue to leave open the
>> question of whether the rule is worthwhile to accept in the first place.
> 
> When someone does a safety certification, there is a difference between
> deviating a rule as a whole or accepting the rule and only deviating
> certain aspects of it (simply ignoring the rule is typically not an
> option in safety certification context.) So here I think it would help
> downstreams interested in safety if we added the rule, with specific
> deviations.

Yet then in other cases you refer to Bertrand's general statement of it
not being helpful when too little of a rule is left by deviating.

> Do you have any comments on the other parts of this patch? If not, I
> would be happy to resent the rest unmodified, and update only 14.4 in
> its own separate patch where we can discuss further.

Well. We're in territory now where I'm not really happy anymore with the
full scope of what is being added to the "accepted" list. Leaving 14.4
aside, what you have in the patch all looks like what was agreed upon,
but then I'm not taking notes during meetings, and hence I can't help
the impression that e.g. for 5.5 there was more than just the one
"permitted" pattern. Therefore, while I deliberately didn't comment
there (for not having a concrete case in mind), I'm afraid I also don't
feel anymore like acking such multi-rule patches. If you strictly went
one by one, it is certainly possible that I might ack this and that.

As attempted to voice several times during the meetings, I pretty
strongly disagree with many of the "developer confusion" aspects, when
they take away options the language quite obviously and naturally
provides. We're talking about hypervisor code here, not some random
tool that was thrown together in a haste. At the risk of sounding
arrogant, people being easily confused by what I'd call normal code
should simply not touch code like this. Whereas the spirit of many of
these rules looks to rather go in the direction that basically anyone
knowing a little bit of C should be qualified enough to maintain code
made subject to all of these rules.

I'm sorry, Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 07:46:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 07:46:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679767.1057446 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZnUN-000124-3f; Tue, 13 Feb 2024 07:45:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679767.1057446; Tue, 13 Feb 2024 07:45:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZnUN-00011x-0F; Tue, 13 Feb 2024 07:45:59 +0000
Received: by outflank-mailman (input) for mailman id 679767;
 Tue, 13 Feb 2024 07:45:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZnUL-00011r-Ce
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 07:45:57 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ebc14e9a-ca43-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 08:45:56 +0100 (CET)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2d0d799b55cso49989751fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 12 Feb 2024 23:45:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h13-20020a2e9ecd000000b002d0f768a0cesm402971ljk.12.2024.02.12.23.45.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 12 Feb 2024 23:45:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebc14e9a-ca43-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707810356; x=1708415156; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ZrvzPUVeg3QniT/kEFrnvxAk6x7E45FOL+PUkf3vrpU=;
        b=fbjlcEfEUY/KfkVi1ad57bbF8quY5yjKUEi3p7JrFJPl3B8AQWPr+k096/lbvZQpLi
         pw66TfaP7J0MTmZ2SYF/EqXQqHsC0k5qYCHH9AkVq7bpybcfIcdablkBoceXXqrPsVvz
         6lNpwWpmX+xf4EsC1p+ysioAbQ2npjfDsPXY3j2XWBOh14nweNIkMvqMbTnNjyX8WPTZ
         e8nGZlNWSqBEFfibq3kiwdD1rhq5fFu51KDHtfBcquyB6XiowxoQQwOG74Zsat2j4lik
         V2lQUIjj7lBsIgtJXooyZ9+t6u8Er+4KocfKCYEwc8MWQDbr1kgiuASk53JmD1n16BMV
         CuyQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707810356; x=1708415156;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ZrvzPUVeg3QniT/kEFrnvxAk6x7E45FOL+PUkf3vrpU=;
        b=J4qXRSEzeUWW/PX1Rd70dDJ7rLqqSDMSuzex60fVUU+YSnI+wvnPO8rCX8sElTc5ns
         9d8zS3GwUrX5D4KWGLb9GHoN8Gf0Ha0xolKEvkLm9OeulwJhhtlPs7DV7B0gZMZzuLwQ
         Tb4m2kYOsngD+2QQL5AKJd+0v34hB5Oy6gdXfwmcd76k/VqUkocqcrilbG1y0vZVWqsM
         M75m4rr9dpkYdflM8B2t3VrbhMKm9y9HugpcTJ4I0/v0s66Vr+Gwg4V1zF4veY7xwC9H
         9cYAzyR6FCtF0PPa135NbdOSA6hYhXVHRyCWeRxsjIaPDTQVcisxaQ+ljj119Qxb56Z2
         JzFA==
X-Forwarded-Encrypted: i=1; AJvYcCVRx9x94uGx8f0b/BySA+/GZndWFCpsHSAxPm1iaXulAnoEM0mS0wupRQZqNBSXl+aJ9Pkc23hDdFRglzQzo/gBu8X1bsPcrJ3lFJVJSxA=
X-Gm-Message-State: AOJu0YzSLIsncGPbDiFUJol9PsJ0XgoofVC4BSmhZ6AvBRh5RfllL119
	0m2JzoDqftwiZcAF/imCjMo6zjvkFvUl6azeCHryoQNi0dj4z1uHFUkpTjtuOA==
X-Google-Smtp-Source: AGHT+IGlLnoaOgo1CgxxXSJc3lxnmZH5xHHHjewhH70rxNqk6dwMkwLyzVt9ae7BVi/Y7bpGCLMYIg==
X-Received: by 2002:a2e:a366:0:b0:2d0:cd6a:ecf7 with SMTP id i6-20020a2ea366000000b002d0cd6aecf7mr5695177ljn.50.1707810355794;
        Mon, 12 Feb 2024 23:45:55 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW9TDiGJ8ZRAt7xOmssENkVCgrV3R2d49NRcaAgS9SpvGf3Rqq/px6eptyrbqTsfdRmezR9KRICUWS9Z3f6JaVIsWbWFRdJGTxRDA84cUSKdwNf1z6HLoagLHoBUk1J3EVVBr8ckj7b4c9t4kU/aMsE/I3etACS87HslQQkzwtlZgoRcrRvPZpZ87QpjJ5jBQrinegX8gUoV9Hfv9UCE4MpV9UqKbCOlw==
Message-ID: <817ceec4-ed12-4360-a6ce-f87732f62251@suse.com>
Date: Tue, 13 Feb 2024 08:45:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Content-Language: en-US
To: Marek Marczykowski <marmarek@invisiblethingslab.com>
Cc: Julien Grall <julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
 <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
 <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com> <ZcrlcuNU9y8WymiK@mail-itl>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcrlcuNU9y8WymiK@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.02.2024 04:43, Marek Marczykowski wrote:
> On Mon, Feb 12, 2024 at 10:04:38AM +0100, Jan Beulich wrote:
>> On 08.02.2024 23:00, Julien Grall wrote:
>>> On 05/02/2024 13:27, Jan Beulich wrote:
>>>> In preparation of dropping the register parameters from
>>>> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
>>>> register state needs making available another way for the few key
>>>> handlers which need it. Fake IRQ-like state.
>>>>
>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>> ---
>>>> The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsistent with
>>>> other console poll functions we have, and it's unclear whether that's
>>>> actually generally correct.
>>>
>>> Is it? Looking at ns16550_poll() we would pass guest_user_regs() if 
>>> run_in_exception() doesn't exist. But looking at the caller, no-on seems 
>>> to care about the 'regs'. So is this just a latent bug?
>>
>> What do you mean by "doesn't exist"? ns16550_poll() assumes it exists.
>> And I can spot any use of guest_user_regs() on the respective generic
>> or Arm-specific bug.c paths.
>>
>>> BTW, do you have an idea why the poll function is not run in an 
>>> exception handler?
>>
>> "The poll function" being which one? If you mean the one in xhci-dbc.c
>> then that's why I had Cc-ed Marek. Moving him to To: - maybe that
>> manages to finally catch his attention.
> 
> TBH, I don't know. That's part of the original xue patch at
> https://github.com/connojd/xue/blob/master/patches/xen-xue-dbgp.patch
> and it works for me as it is.

"Works" meaning what? Doesn't crash on you? Or does also provide
sensible output in _all_ cases (i.e. including when e.g. the poll
happens to run on an idle vCPU)?

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 07:47:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 07:47:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679770.1057455 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZnVt-0001Xi-Dq; Tue, 13 Feb 2024 07:47:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679770.1057455; Tue, 13 Feb 2024 07:47:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZnVt-0001Xb-As; Tue, 13 Feb 2024 07:47:33 +0000
Received: by outflank-mailman (input) for mailman id 679770;
 Tue, 13 Feb 2024 07:47:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m+vO=JW=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rZnVs-0001XT-94
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 07:47:32 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2453cd48-ca44-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 08:47:31 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id E209D218A0;
 Tue, 13 Feb 2024 07:47:28 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A16101370C;
 Tue, 13 Feb 2024 07:47:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 4xFxJZAey2X0PgAAD6G6ig
 (envelope-from <jgross@suse.com>); Tue, 13 Feb 2024 07:47:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2453cd48-ca44-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707810450; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=QfQmtk1+J1WqxyDnAU2S6IE5dzVgSPOm4JsDugmN/7k=;
	b=EUrt/4vsBIAt0W6gBjxu1S3l7L2lw6m9bEBqqi9YyAh0XhY/zZUtAHEnCOiuzZRz/2Ouqu
	n60Fw5hmP31zn1ooUwC+02pQ/KeJc6yklYH3Ppy+M9b04yrnEmKtTK1HaNz/jepaxcohq5
	KUysepAt5l9RaWvSyiLqdC45QRSaag4=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707810448; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=QfQmtk1+J1WqxyDnAU2S6IE5dzVgSPOm4JsDugmN/7k=;
	b=PK8SjE6w0pmdQrLk4wY3jcBWokUVl0A5gh60QWvH6eeRsnXzw5isDVNK0BdNfdLNFxrPDS
	WxsWu6+jgR/g/G3wE4C0sB/6oocWiIXlxmUAhVydW8HeRGKPMAvst3DTrLAVJg9Gbrhrk+
	hfZ9Z00O2XxqrW6zIBfoaVRfqxgW7Dw=
Message-ID: <2fbbdd50-4861-49d3-bb1a-f6454f2c54be@suse.com>
Date: Tue, 13 Feb 2024 08:47:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/privcmd: Use memdup_array_user() in alloc_ioreq()
To: Markus Elfring <Markus.Elfring@web.de>, xen-devel@lists.xenproject.org,
 kernel-janitors@vger.kernel.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: LKML <linux-kernel@vger.kernel.org>
References: <41e333f7-1f3a-41b6-a121-a3c0ae54e36f@web.de>
Content-Language: en-US
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <41e333f7-1f3a-41b6-a121-a3c0ae54e36f@web.de>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------WiQA2j8GJY9CCfJx2SJGgtC6"
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=PK8SjE6w
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.62 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 TO_DN_SOME(0.00)[];
	 HAS_ATTACHMENT(0.00)[];
	 RCPT_COUNT_FIVE(0.00)[6];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 FREEMAIL_TO(0.00)[web.de,lists.xenproject.org,vger.kernel.org,epam.com,kernel.org];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_RHS_MATCH_FROM(0.00)[];
	 BAYES_HAM(-1.22)[89.34%];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[web.de];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: -4.62
X-Rspamd-Queue-Id: E209D218A0
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------WiQA2j8GJY9CCfJx2SJGgtC6
Content-Type: multipart/mixed; boundary="------------DmiSYnlCNvTM8sKEknfFhIWp";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Markus Elfring <Markus.Elfring@web.de>, xen-devel@lists.xenproject.org,
 kernel-janitors@vger.kernel.org,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Message-ID: <2fbbdd50-4861-49d3-bb1a-f6454f2c54be@suse.com>
Subject: Re: [PATCH] xen/privcmd: Use memdup_array_user() in alloc_ioreq()
References: <41e333f7-1f3a-41b6-a121-a3c0ae54e36f@web.de>
In-Reply-To: <41e333f7-1f3a-41b6-a121-a3c0ae54e36f@web.de>

--------------DmiSYnlCNvTM8sKEknfFhIWp
Content-Type: multipart/mixed; boundary="------------AP8IZ9CK0yKq2yIKKiYrhJn0"

--------------AP8IZ9CK0yKq2yIKKiYrhJn0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMjguMDEuMjQgMTg6MDksIE1hcmt1cyBFbGZyaW5nIHdyb3RlOg0KPiBGcm9tOiBNYXJr
dXMgRWxmcmluZyA8ZWxmcmluZ0B1c2Vycy5zb3VyY2Vmb3JnZS5uZXQ+DQo+IERhdGU6IFN1
biwgMjggSmFuIDIwMjQgMTc6NTA6NDMgKzAxMDANCj4gDQo+ICogVGhlIGZ1bmN0aW9uIOKA
nG1lbWR1cF9hcnJheV91c2Vy4oCdIHdhcyBhZGRlZCB3aXRoIHRoZQ0KPiAgICBjb21taXQg
MzEzZWJlNDdkNzU1NTg1MTFhYTEyMzdiNmUzNWM2NjNiNWMwZWM2ZiAoInN0cmluZy5oOiBh
ZGQNCj4gICAgYXJyYXktd3JhcHBlcnMgZm9yICh2KW1lbWR1cF91c2VyKCkiKS4NCj4gICAg
VGh1cyB1c2UgaXQgYWNjb3JkaW5nbHkuDQo+IA0KPiAgICBUaGlzIGlzc3VlIHdhcyBkZXRl
Y3RlZCBieSB1c2luZyB0aGUgQ29jY2luZWxsZSBzb2Z0d2FyZS4NCj4gDQo+ICogRGVsZXRl
IGEgbGFiZWwgd2hpY2ggYmVjYW1lIHVubmVjZXNzYXJ5IHdpdGggdGhpcyByZWZhY3Rvcmlu
Zy4NCj4gDQo+IFNpZ25lZC1vZmYtYnk6IE1hcmt1cyBFbGZyaW5nIDxlbGZyaW5nQHVzZXJz
LnNvdXJjZWZvcmdlLm5ldD4NCg0KUmV2aWV3ZWQtYnk6IEp1ZXJnZW4gR3Jvc3MgPGpncm9z
c0BzdXNlLmNvbT4NCg0KDQpKdWVyZ2VuDQoNCg==
--------------AP8IZ9CK0yKq2yIKKiYrhJn0
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------AP8IZ9CK0yKq2yIKKiYrhJn0--

--------------DmiSYnlCNvTM8sKEknfFhIWp--

--------------WiQA2j8GJY9CCfJx2SJGgtC6
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXLHpAFAwAAAAAACgkQsN6d1ii/Ey9+
kAf/SK2su8rrXx43V6udWcRfoy5PTdnkmEmn5jTw4CTvbsdbfIz3Oujsf/vD9ax06Ah0HW9aSIGH
bK7DLwg467NgD8TR19lVDVTBAtQeEg15uSNI3FEEC/jmZoEO3UueV/OmzV0spqjxvuR+AWLHJ4Xb
cog7OnDr/hA4jNac5lixJf8c1wLiRLUxeCgRI9NwmymNodhpv8oiLZDsExZlgv1166kxcgRBWmAs
5N0Vo2scFGvHakI6gFCLA/7JiOLiBqxkLfCdW56XvvfHO0DFuYS0qm3rehz0707z+K8XNacd6z9j
3WVMZ/Guze0Wya8FMO2Nkt1UzE44vGaF1ffSnuykqQ==
=tn04
-----END PGP SIGNATURE-----

--------------WiQA2j8GJY9CCfJx2SJGgtC6--


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 08:11:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 08:11:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679782.1057466 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZnsj-0006cl-QO; Tue, 13 Feb 2024 08:11:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679782.1057466; Tue, 13 Feb 2024 08:11:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZnsj-0006ce-Nj; Tue, 13 Feb 2024 08:11:09 +0000
Received: by outflank-mailman (input) for mailman id 679782;
 Tue, 13 Feb 2024 08:11:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=iJZn=JW=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rZnsi-0006cX-DI
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 08:11:08 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6f56a1ca-ca47-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 09:11:05 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-410e7b08252so4289735e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 00:11:05 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 y12-20020a056000108c00b0033b40a3f92asm8871466wrw.25.2024.02.13.00.11.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 00:11:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f56a1ca-ca47-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707811865; x=1708416665; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=WyMBnM0DuyKb1MqB/cj3eU9Z2BMuh3mD8a7quXtJ5tc=;
        b=uR8I/6cuW+3BT3C8FKNbZy3S8JUrBKzvRmV3R1hHJh4d2lQIaCwdlK8pKDElyEkD87
         aB/aNdz7GfYUDGnm7MCxxsCFuYiECLtG6XlhtPc5p4xT5hC6DEa2QuQlMqfjFBabJexY
         ZALucrIosqWdhDWPLXeNZYCOI3KP3dis6atkk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707811865; x=1708416665;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=WyMBnM0DuyKb1MqB/cj3eU9Z2BMuh3mD8a7quXtJ5tc=;
        b=Jfwa5X1q0ymgFyPXU/z1u7gM3TB+/AjnmAZPmVXwsApjQGD/G+5mMjBIVfEGntHTrX
         5aSdOgUnfu/s8n8X3WY/PUcj2lyuuO8JEhVktmA0CcwGHk49u2z8Oeb72RFp4tntB5PO
         41UuRP4t1nLIi9S0PVEGYQ4bO3p8tmzOtVMxCQO4AJ5iz7Mc7YZAoNDMgXY0K8Ce7k10
         8L8APoQXmlpJ9qyv03PALeNC/CZLy1IapUtx0fAXlmZnj3zokOaD0Xo/SnJSenECWMa/
         pUXmLkLfjG1hD1Kw7+yDP44SznkTRjbvlCKYZtvwXiAClYiLVCIyUmFRQqD/R2MzKqRO
         nH+Q==
X-Forwarded-Encrypted: i=1; AJvYcCXKlPrVimP5CIcMXRfoIBcWoq+LPGlvCrvLLfqqBc+ZQpKC5G2WBy7Sa6Y7QfZ+NKm2BOljKghHPEGH4zuX3JcqX/NxeLLZf46c2mI0YKE=
X-Gm-Message-State: AOJu0YzX7hJQc4cDMfFZaKwgKyo+x8+Rbwc6itMUcJJzYqhHOcna7YUY
	a+Ark/RmuCRpkeLtE4EZ5OuXJDPMChzJf9FCEYtmldKnyGMJZwnZ6N3ZI0W8wGg=
X-Google-Smtp-Source: AGHT+IEzMyoEFgrOk1qX4+lP3za8Xq5oZmu2egfj/E857RKJEvEUqXHObwpz34VFG+j7RWXktr3PwA==
X-Received: by 2002:a05:600c:2342:b0:40f:20e:bf7d with SMTP id 2-20020a05600c234200b0040f020ebf7dmr7280059wmq.22.1707811865022;
        Tue, 13 Feb 2024 00:11:05 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXb8EnQy3Yxro3K6/q+b+bx8RlRsNFbUrgw9FyQzsuzVA0rdIuUNVgViFRoIvRliv4QsqqLe2V2V++lTX8cbS71EOueOEu+f1zqkuJbUCn7KpltP5zkSqrZGKxonIIPvubZRj1eWQaNiw==
Date: Tue, 13 Feb 2024 09:11:04 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH v2 1/2] iommu/x86: introduce a generic IVMD/RMRR range
 validity helper
Message-ID: <ZcskGNrX7y3aFWes@macbook>
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-2-roger.pau@citrix.com>
 <ee4b4d25-e6db-4265-adc4-4080784495cf@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ee4b4d25-e6db-4265-adc4-4080784495cf@suse.com>

On Mon, Feb 12, 2024 at 03:34:59PM +0100, Jan Beulich wrote:
> On 07.02.2024 16:34, Roger Pau Monne wrote:
> > IVMD and RMRR ranges are functionally equivalent, and as so could use the same
> > validity checker.
> > 
> > Move the IVMD to x86 common IOMMU code and adjust the function to take a pair
> > of [start, end] mfn parameters.
> > 
> > So far only the AMD-Vi side is adjusted to use the newly introduced helper, the
> > VT-d side will be adjusted in a further change.
> > 
> > No functional change intended.
> > 
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> with one minor remark:
> 
> > --- a/xen/drivers/passthrough/x86/iommu.c
> > +++ b/xen/drivers/passthrough/x86/iommu.c
> > @@ -792,6 +792,52 @@ static int __init cf_check adjust_irq_affinities(void)
> >  }
> >  __initcall(adjust_irq_affinities);
> >  
> > +bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
> > +{
> > +    mfn_t addr;
> > +
> > +    if ( e820_all_mapped(mfn_to_maddr(start), mfn_to_maddr(end) + PAGE_SIZE,
> > +                         E820_RESERVED) )
> > +        return true;
> > +
> > +    printk(XENLOG_WARNING "%s: [%#" PRI_mfn " ,%#" PRI_mfn
> > +           "] is not (entirely) in reserved memory\n",
> 
> Would you mind if I re-flowed this to
> 
>     printk(XENLOG_WARNING
>            "%s: [%#" PRI_mfn " ,%#" PRI_mfn "] is not (entirely) in reserved memory\n",
> 
> while committing?

Sure.  FWIW, when the line contains format specifiers I take that as a
point where splitting might be acceptable, since it already prevents
proper grepping.  Will try to remember to attempt to not split the
line.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 08:35:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 08:35:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679788.1057475 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZoGU-0001Yp-Ko; Tue, 13 Feb 2024 08:35:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679788.1057475; Tue, 13 Feb 2024 08:35:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZoGU-0001Yi-IH; Tue, 13 Feb 2024 08:35:42 +0000
Received: by outflank-mailman (input) for mailman id 679788;
 Tue, 13 Feb 2024 08:35:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=iJZn=JW=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rZoGT-0001Yc-3D
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 08:35:41 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dcc0e8ca-ca4a-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 09:35:38 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-411c93e6bfaso729755e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 00:35:37 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 a4-20020a05600c224400b00410ebcf8180sm3888029wmm.43.2024.02.13.00.35.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 00:35:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dcc0e8ca-ca4a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707813337; x=1708418137; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=YFlIjH8etZU1AYd/9gLFsW9bhhhfD3RrNOQVWytW6GY=;
        b=fjFemcnOB+9+6qqA8QI25rl2Kt6rabyaql5vzGb9P7I7P1P4bpl3uGxRS9GQROx+lc
         rcHKsvnhiZ2DvEp/2yX6KU+406QfqvvHlU6OWSbcXHpI3+1l/TDq8GPMJHJyPmvsXImp
         NfirmO8NbdlzbajcNXvpBu9L/mwYp81yvDa3E=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707813337; x=1708418137;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=YFlIjH8etZU1AYd/9gLFsW9bhhhfD3RrNOQVWytW6GY=;
        b=YHBQQ4R2b19M93DlcI61jrTExljUGRR60QdW47md3CsKG15dDl4D1wPKghCBWSFNL0
         RERtR/wk/X1EqsfKrKQr1Qx9tPBy82JKiJDXH465yK3bU2hRqJ4Tj4gH9nMTSX1Dqlbp
         rJgyrErYGXaU0NGL58R1baNKHrYoCUQ8XO5agRYeJq2fUUTBotgRnIh0cTfZwqKO0INJ
         YwmhoY5eMj/xT6jClN4JmzmW8sVi2wbcDYvCHyEg823B1fvGxG9bgcgunp4HlgiaH3OK
         /yALV96qFWBHmiJ50UnufJ+5dlpvqpGV5X32IU1qnxuYHuEF/fh2kx4GEnVo5YNkMEyx
         hw0g==
X-Gm-Message-State: AOJu0YxMHF375DdmoNJFGnCPfH2aHnFmkcinArzSv3trLVCzGK0Fif5j
	DsYmWwxxaAT2fLr+WcwzL/RZS4TW/V8BfC0LhznZa08ycA0vA6Jbt7eP0Gq4sr0=
X-Google-Smtp-Source: AGHT+IEmuh2d/UmIQlG34Qjs8sefw4XcCVT6x31IOzqRfiJIN+IPZAC3rYi9GSgmVYKxUb6jBTvgNg==
X-Received: by 2002:a05:600c:1e1e:b0:411:e7d:9086 with SMTP id ay30-20020a05600c1e1e00b004110e7d9086mr2345558wmb.15.1707813336922;
        Tue, 13 Feb 2024 00:35:36 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVJSI45pSbT8JX/1gkn01KuoBoMK8naFDhRNEObIJAlVJXM1+XE/cBPLEQwb9PRFIpGWzzkyVS6aR5wxSld8m0dlGOcDSKAkURKMFXYiYx2+O99Il2P/nqNIq1OyzfyErRa/qF3Uyo3CuvjGDNgEEsMzI5YD7uZr3fbCl8dPyC6RTbFPUgHvOzruKWQ2134SaEX7dAxKGdf//mCN++iaod+uwGsTIOsr6cknQo3Vq+zO41oCZh1gB9fMZykgrOIrHtxmp6QVF0y5WO0+htMckzZcGyRRv14lj8IrdMUKlAgKcK9OUMI/Vw4utsacfa0R9g0yOH0PGypt4NOiimDERrFHkhxjMfLC2jziebNz8hzvDztZPKrhBufSCEndQ8GIWEmiw9L
Date: Tue, 13 Feb 2024 09:35:35 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: Re: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Message-ID: <Zcsp15Aqve11Icjb@macbook>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240202213321.1920347-2-stewart.hildebrand@amd.com>

On Fri, Feb 02, 2024 at 04:33:05PM -0500, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Use the per-domain PCI read/write lock to protect the presence of the
> pci device vpci field. This lock can be used (and in a few cases is used
> right away) so that vpci removal can be performed while holding the lock
> in write mode. Previously such removal could race with vpci_read for
> example.
> 
> When taking both d->pci_lock and pdev->vpci->lock, they should be
> taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
> possible deadlock situations.
> 
> 1. Per-domain's pci_lock is used to protect pdev->vpci structure
> from being removed.
> 
> 2. Writing the command register and ROM BAR register may trigger
> modify_bars to run, which in turn may access multiple pdevs while
> checking for the existing BAR's overlap. The overlapping check, if
> done under the read lock, requires vpci->lock to be acquired on both
> devices being compared, which may produce a deadlock. It is not
> possible to upgrade read lock to write lock in such a case. So, in
> order to prevent the deadlock, use d->pci_lock in write mode instead.
> 
> All other code, which doesn't lead to pdev->vpci destruction and does
> not access multiple pdevs at the same time, can still use a
> combination of the read lock and pdev->vpci->lock.
> 
> 3. Drop const qualifier where the new rwlock is used and this is
> appropriate.
> 
> 4. Do not call process_pending_softirqs with any locks held. For that
> unlock prior the call and re-acquire the locks after. After
> re-acquiring the lock there is no need to check if pdev->vpci exists:
>  - in apply_map because of the context it is called (no race condition
>    possible)
>  - for MSI/MSI-X debug code because it is called at the end of
>    pdev->vpci access and no further access to pdev->vpci is made
> 
> 5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
> while accessing pdevs in vpci code.
> 
> 6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
> do not go away. The vPCI functions call several MSI-related functions
> which already have existing non-vPCI callers. Change those MSI-related
> functions to allow using either pcidevs_lock() or d->pci_lock for
> ensuring pdevs do not go away. Holding d->pci_lock in read mode is
> sufficient. Note that this pdev protection mechanism does not protect
> other state or critical sections. These MSI-related functions already
> have other race condition and state protection mechanims (e.g.
> d->event_lock and msixtbl RCU), so we deduce that the use of the global
> pcidevs_lock() is to ensure that pdevs do not go away. Existing non-vPCI
> callers of these MSI-related functions will remain (ab)using the global
> pcidevs_lock() to ensure pdevs do not go away so as to minimize changes
> to existing non-vPCI call paths.
> 
> 7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
> that pdevs do not go away. The purpose of this wrapper is to aid
> readability and document the intent of the pdev protection mechanism.

I would add that when possible, the existing callers haven't been
switched to use the newly introduced per-domain pci_lock, and will
continue to use the global pcidevs lock.  This is done to reduce the
risk of the new locking scheme introducing regressions.  Those users
will be adjusted in due time.

IIRC Jan had concerns about why some existing use-cases are not
switched straight to use the new per-domain pci_lock in this patch.

> 
> Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> ---
> Changes in v13:
>  - hold off adding Roger's R-b tag even though it was provided on v12.2
>  - use a wrapper construct to ease readability of odd-looking ASSERTs
>  - new placement of ASSERT in __pci_enable_msix(), __pci_enable_msi(),
>    and pci_enable_msi(). Rearrange/add pdev NULL check.
>  - expand commit description with details about using either
>    pcidevs_lock() or d->pci_lock
> 
> Changes in v12.2:
>  - drop Roger's R-b
>  - drop both locks on error paths in vpci_msix_arch_print()
>  - add another ASSERT in vpci_msix_arch_print(), to enforce the
>    expectation both locks are held before calling vpci_msix_arch_print()
>  - move pdev_done label in vpci_dump_msi()
>  - update comments in vpci_dump_msi() to say locks (plural)
> 
> Changes in v12.1:
>  - use read_trylock() in vpci_msix_arch_print()
>  - fixup in-code comments (revert double space, use DomXEN) in
>    vpci_{read,write}()
>  - minor updates in commit message
>  - add Roger's R-b
> 
> Changes in v12:
>  - s/pci_rwlock/pci_lock/ in commit message
>  - expand comment about scope of pci_lock in sched.h
>  - in vpci_{read,write}, if hwdom is trying to access a device assigned
>    to dom_xen, holding hwdom->pci_lock is sufficient (no need to hold
>    dom_xen->pci_lock)
>  - reintroduce ASSERT in vmx_pi_update_irte()
>  - reintroduce ASSERT in __pci_enable_msi{x}()
>  - delete note 6. in commit message about removing ASSERTs since we have
>    reintroduced them
> 
> Changes in v11:
>  - Fixed commit message regarding possible spinlocks
>  - Removed parameter from allocate_and_map_msi_pirq(), which was added
>  in the prev version. Now we are taking pcidevs_lock in
>  physdev_map_pirq()
>  - Returned ASSERT to pci_enable_msi
>  - Fixed case when we took read lock instead of write one
>  - Fixed label indentation
> 
> Changes in v10:
>  - Moved printk pas locked area
>  - Returned back ASSERTs
>  - Added new parameter to allocate_and_map_msi_pirq() so it knows if
>  it should take the global pci lock
>  - Added comment about possible improvement in vpci_write
>  - Changed ASSERT(rw_is_locked()) to rw_is_write_locked() in
>    appropriate places
>  - Renamed release_domain_locks() to release_domain_write_locks()
>  - moved domain_done label in vpci_dump_msi() to correct place
> Changes in v9:
>  - extended locked region to protect vpci_remove_device and
>    vpci_add_handlers() calls
>  - vpci_write() takes lock in the write mode to protect
>    potential call to modify_bars()
>  - renamed lock releasing function
>  - removed ASSERT()s from msi code
>  - added trylock in vpci_dump_msi
> 
> Changes in v8:
>  - changed d->vpci_lock to d->pci_lock
>  - introducing d->pci_lock in a separate patch
>  - extended locked region in vpci_process_pending
>  - removed pcidevs_lockis vpci_dump_msi()
>  - removed some changes as they are not needed with
>    the new locking scheme
>  - added handling for hwdom && dom_xen case
> ---
>  xen/arch/x86/hvm/vmsi.c       | 31 +++++++++++++--------
>  xen/arch/x86/hvm/vmx/vmx.c    |  2 +-
>  xen/arch/x86/irq.c            |  8 +++---
>  xen/arch/x86/msi.c            | 20 +++++++++-----
>  xen/arch/x86/physdev.c        |  2 ++
>  xen/drivers/passthrough/pci.c |  9 +++---
>  xen/drivers/vpci/header.c     | 18 ++++++++++++
>  xen/drivers/vpci/msi.c        | 30 +++++++++++++++++---
>  xen/drivers/vpci/msix.c       | 52 ++++++++++++++++++++++++++++++-----
>  xen/drivers/vpci/vpci.c       | 24 ++++++++++++++--
>  xen/include/xen/sched.h       | 15 +++++++++-
>  11 files changed, 170 insertions(+), 41 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
> index 128f23636279..f29089178a59 100644
> --- a/xen/arch/x86/hvm/vmsi.c
> +++ b/xen/arch/x86/hvm/vmsi.c
> @@ -468,7 +468,7 @@ int msixtbl_pt_register(struct domain *d, struct pirq *pirq, uint64_t gtable)
>      struct msixtbl_entry *entry, *new_entry;
>      int r = -EINVAL;
>  
> -    ASSERT(pcidevs_locked());
> +    ASSERT(pdev_list_is_read_locked(d));
>      ASSERT(rw_is_write_locked(&d->event_lock));
>  
>      if ( !msixtbl_initialised(d) )
> @@ -538,7 +538,7 @@ void msixtbl_pt_unregister(struct domain *d, struct pirq *pirq)
>      struct pci_dev *pdev;
>      struct msixtbl_entry *entry;
>  
> -    ASSERT(pcidevs_locked());
> +    ASSERT(pdev_list_is_read_locked(d));
>      ASSERT(rw_is_write_locked(&d->event_lock));
>  
>      if ( !msixtbl_initialised(d) )
> @@ -684,7 +684,7 @@ static int vpci_msi_update(const struct pci_dev *pdev, uint32_t data,
>  {
>      unsigned int i;
>  
> -    ASSERT(pcidevs_locked());
> +    ASSERT(rw_is_locked(&pdev->domain->pci_lock));

Any reason to not use the newly introduced helper here?  I know the
pcidevs will never be locked here given the new lock usage, but still
it would be less confusing if the new helper was used consistently.

Otherwise we need a comment here as to why the helper can't be used,
in order to avoid confusion in the future.

>> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
> index 9da91e0e6244..c3adec1aca3c 100644
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -462,7 +462,8 @@ struct domain
>  #ifdef CONFIG_HAS_PCI
>      struct list_head pdev_list;
>      /*
> -     * pci_lock protects access to pdev_list.
> +     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
> +     * structure from being removed.
>       *
>       * Any user *reading* from pdev_list, or from devices stored in pdev_list,
>       * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,
> @@ -628,6 +629,18 @@ struct domain
>      unsigned int cdf;
>  };
>  
> +/*
> + * Check for use in ASSERTs to ensure that:
> + *   1. we can *read* d->pdev_list
> + *   2. pdevs (belonging to this domain) do not go away
> + *   3. pdevs (belonging to this domain) do not get assigned to other domains

I think you can just state that this check ensures there will be no
changes to the entries in d->pdev_list, but not the contents of each
entry.  No changes to d->pdev_list already ensures not devices can be
deassigned or removed from the system, and obviously makes the list
safe to iterate against.

I would also drop the explicitly mention this is intended for ASSERT
usage: there's nothing specific in the code that prevents it from
being used in other places (albeit I think that's unlikely).

> + * This check is not suitable for protecting other state or critical regions.
> + */
> +#define pdev_list_is_read_locked(d) ({                           \

I would be tempted to drop at least the '_read_' part from the name,
the name is getting a bit too long for my taste.

> +        struct domain *d_ = (d);                                 \

Why do you need this local domain variable?  Can't you use the d
parameter directly?

Such assign will prevent using a const 'd' parameter, and 'd_' itself
should be const IMO (iff we really need this).

Also sched.h is not the best place, can't you just place it in
pci.h?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 08:45:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 08:45:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679792.1057485 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZoPZ-0003Rs-Fh; Tue, 13 Feb 2024 08:45:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679792.1057485; Tue, 13 Feb 2024 08:45:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZoPZ-0003Rl-Cl; Tue, 13 Feb 2024 08:45:05 +0000
Received: by outflank-mailman (input) for mailman id 679792;
 Tue, 13 Feb 2024 08:45:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZoPX-0003Ra-D0
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 08:45:03 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2c234c44-ca4c-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 09:45:00 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-51147d0abd1so4606517e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 00:45:00 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m27-20020a05600c3b1b00b00410794ddfc6sm11182448wms.35.2024.02.13.00.44.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 00:44:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c234c44-ca4c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707813899; x=1708418699; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RaA3YG3+iVtdQj6ISOklP2N0D5IWZSmrDFPfH5W5pOU=;
        b=CT4+9vKgWiHx3l1oCdQYUE7Cqpg9p6AiJUwf4kcc0q1HD6EwVBQhuilgeiPKg2MbjS
         oBS8pbcjRVoqAOJpXVkn5nTx0ENKJWKHTB8dPLTSDwV7Ow+pr2ROtGgwwreLZUAEpd3n
         WoDqMdLczd+jgEDj419XkeDPc5BoyV70pvjyyc5g4nTtxkv2AMUvk85gnZ6Hz1EDcOZ7
         Ufe4zLXpsVot5n6xSOUNzaFKTBVeNuHQM2faecktP1sO6RwIUwpvNdHZ2Lw8QjzcqnxO
         G2WR6FdDfq+JAyDQzAr9TARdc6JaIHPcdusc0knjVKMIogDFs61sY21yj9O2r5PqT4/n
         YmAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707813899; x=1708418699;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RaA3YG3+iVtdQj6ISOklP2N0D5IWZSmrDFPfH5W5pOU=;
        b=ORhheAt+zMydio5y5xZwcic9JZnaf+o1eeImyKOPvFeM5OWU9r522rqnzAO9SXPLgh
         s9kdOAaNMZF/KymDp9D3DAUDNGJ9wd1ckggepQfd6PUCY8Y0cLVVnUDVnVs0G7kn/sTI
         cAcdMctRgdm4MV3xLBG/vlovEjGLdm3yaM5OzE7mbNyuKGAVwzU6UucLNJFjVm9ejE3w
         pkQyvAgy3JpytcRRjqCvBde1i4NmFfDbNNRnoCxPGLbUUaS3DHLgSCGN1Up2Ouxv0VAW
         Mv7It4I16RQNYhxNrHZ9HkktVk/bV29MkKnp1Ume3Fr/dA74yKYPVu7oiHkOFR+nVNGi
         e2Gg==
X-Gm-Message-State: AOJu0Yw3a0T55q//N18ab/j0HyqiqebjpYvv68854wUPhi+A1OJdMc/Q
	o2KgMIevtiWD4OYjdGG4fCV4QxYIKtRZn3f5hJOHjwuWOzHx10lRvrz9UzMPPA==
X-Google-Smtp-Source: AGHT+IGG5MSvMInP5gKdQmHKgK3+l2RLwuKnOzx/5X0XjqcDof/YSb+IWYTjo4fZrA9J9asFfup20A==
X-Received: by 2002:a05:6512:e83:b0:511:4e8c:7d02 with SMTP id bi3-20020a0565120e8300b005114e8c7d02mr7463008lfb.48.1707813899622;
        Tue, 13 Feb 2024 00:44:59 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX29TO1eHQlZIG2h4eomrkCqEQmXgqYgTPLf8ZfpFgsmrvBjrEUfZWtf520g/nO2Omy36yktZB9RAyHfULFq5As1Iov0uG4sEsD1Lsow795bz6TRL6MMxlD/VzchE0Q2wmlR/bHhAzKGf9cp1cIAGLV05KQm1zAjEaRSlQtJaHte2RwU+pue2glSWf/AkyiO1bK9keL2KABK78Jj8l0vqGVqP4C7EnldxmqJDH6FpPLCMXKuDlfyR+FLbOLw9kAyah4Upy5R8AAnLOsl/X7Orm7YruanwdqysjUYg7Yx2RilIdz7xvxIqvuZ7h64EjUkFeEwLZpgNMGQx19PwdglCdpmoIV5GrCMN2OKyWny1NPGb+7y8ld+NxBa79Cj9f4V+w0h4tnf5Ca/CZ9syhQ
Message-ID: <ec5d0c39-1559-4f10-9574-98cfa0542993@suse.com>
Date: Tue, 13 Feb 2024 09:44:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Paul Durrant <paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <Zcsp15Aqve11Icjb@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zcsp15Aqve11Icjb@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.02.2024 09:35, Roger Pau Monné wrote:
> On Fri, Feb 02, 2024 at 04:33:05PM -0500, Stewart Hildebrand wrote:
>> --- a/xen/include/xen/sched.h
>> +++ b/xen/include/xen/sched.h
>> @@ -462,7 +462,8 @@ struct domain
>>  #ifdef CONFIG_HAS_PCI
>>      struct list_head pdev_list;
>>      /*
>> -     * pci_lock protects access to pdev_list.
>> +     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
>> +     * structure from being removed.
>>       *
>>       * Any user *reading* from pdev_list, or from devices stored in pdev_list,
>>       * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,
>> @@ -628,6 +629,18 @@ struct domain
>>      unsigned int cdf;
>>  };
>>  
>> +/*
>> + * Check for use in ASSERTs to ensure that:
>> + *   1. we can *read* d->pdev_list
>> + *   2. pdevs (belonging to this domain) do not go away
>> + *   3. pdevs (belonging to this domain) do not get assigned to other domains
> 
> I think you can just state that this check ensures there will be no
> changes to the entries in d->pdev_list, but not the contents of each
> entry.  No changes to d->pdev_list already ensures not devices can be
> deassigned or removed from the system, and obviously makes the list
> safe to iterate against.
> 
> I would also drop the explicitly mention this is intended for ASSERT
> usage: there's nothing specific in the code that prevents it from
> being used in other places (albeit I think that's unlikely).

But pcidevs_locked(), resolving to spin_is_locked(), isn't reliable. The
assertion usage is best-effort only, without a guarantee that all wrong
uses would be caught.

>> + * This check is not suitable for protecting other state or critical regions.
>> + */
>> +#define pdev_list_is_read_locked(d) ({                           \
> 
> I would be tempted to drop at least the '_read_' part from the name,
> the name is getting a bit too long for my taste.

While I agree with the long-ish aspect, I'm afraid the "read" part is
crucial. As a result I see no room for shortening.

>> +        struct domain *d_ = (d);                                 \
> 
> Why do you need this local domain variable?  Can't you use the d
> parameter directly?

It would be evaluated then somewhere between 0 and 2 times.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 09:02:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 09:02:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679807.1057499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZofy-0006mq-8j; Tue, 13 Feb 2024 09:02:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679807.1057499; Tue, 13 Feb 2024 09:02:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZofy-0006mj-4o; Tue, 13 Feb 2024 09:02:02 +0000
Received: by outflank-mailman (input) for mailman id 679807;
 Tue, 13 Feb 2024 09:02:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=iJZn=JW=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rZofw-0006mZ-Gp
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 09:02:00 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8af675a1-ca4e-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 10:01:58 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-337d05b8942so3409517f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 01:01:58 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 t24-20020adfa2d8000000b0033b86b38611sm3673211wra.10.2024.02.13.01.01.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 01:01:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8af675a1-ca4e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707814918; x=1708419718; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=feKTui5WrC3pt5o+89E72lmj+xsEAgwKtWT8NHN4qDc=;
        b=DfpNI9wiV3SNKE+gqRVIjlgIzNXSbEvzXhjaldezvymVmgELzpuNVh5cv/wfr11xbL
         NrSPI9rDB2I+TDhDNivg+3cE/AwHoTsV1bnkPgBJmj1Jz+XcGO0EwUjfxgp8HcYcL4BZ
         /vdzjnzX1pJB1MfuTSwtFnKF34UXFTVcaEQ7A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707814918; x=1708419718;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=feKTui5WrC3pt5o+89E72lmj+xsEAgwKtWT8NHN4qDc=;
        b=EHmgZ0XZ3RGCJ7eAbDqG7o7dTP/viXmwsXfG52BR+FX7Ri2POOW6+0GOAPt6Hjq6Mn
         a4CmvL9wday1UQuTzxOdjeurHgAQv8NDGQ4OBZsFSutpsqybLjue6V/yTzXY7NqtXiS7
         LEN1FtC45bFnu09eclIGTxv86oyiV+QgDGhp3rh0sOFade4YnUz/++1O6xwAsb/MKoPD
         Rg+y16fRONveVi2mMjF/BersLgTrabLrswvttldY5JD1F1L+3B548LMa02cKUKj2NyIq
         7CH7gWO6lq2dSrx0j/3oF31OKxqxTtSLBnRRohLL5GiJu5mLl5R49oAW8mFieu/lXRME
         jUSA==
X-Forwarded-Encrypted: i=1; AJvYcCXzbRBf3BMtOODrvB9rSTWwklOTyTSNWzZfnOIdpCTHnRmcG0MG/yz8CvJrYehiw8/jPmWfbojORE4/YYRZKxyIXRkAUtc5RDP+yyVbzY4=
X-Gm-Message-State: AOJu0YwX2BpHozTletIyVDzXkr8rV6215ZoXGlqiBqi3qpiO4SficnZ1
	aAAajrBVoql8wZzlvQGu96NSRN2RP2QWnTfko3ZgNvrtxHHD8x8Nhlf8BN0i210=
X-Google-Smtp-Source: AGHT+IGYTpcKocUJWvd762xsqzOhlx6dFzHcSshIQEl/AgbS4YsEyP0xWoyCW2c70I+C559clI1hsw==
X-Received: by 2002:adf:f805:0:b0:33b:87a0:3af with SMTP id s5-20020adff805000000b0033b87a003afmr3258678wrp.67.1707814917823;
        Tue, 13 Feb 2024 01:01:57 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXVShLoI1tAGaUPMm+bvlLMkUiMlM9Q+xa9MgzvqBByO0OJP8h2lJt+vd63o87NfJ0jkYvq3XvFZtQnbYK8bqiFsaJouUHuhsNYDtLneWQhsMOA9riw+KhikJnyznqlHFSOS/IEwUpFHJiCe74IzFCyFfFbOsvlZFleKWvNXTskB16/xRour18MO/nFKGg9jPc8NLBUTBQhnBOxnd49qphEn8H6e5QPsLfa3HKfFGVrDxJk3e3pcGYU1HnNVLaQaGN15nwXCIFrebD7TkqVgkIO++Onl5uFveliUO7dkGlbq6f3kPGOjDs1je3og68jo798jxwYm40uh6X7hQyAcsX+QGXCSQd7fwII6iI8pva5u61ELpJjVTWi7vsDj/qmpp813ljqUQ6VAUGdDLfZtYnKbg==
Date: Tue, 13 Feb 2024 10:01:56 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stewart Hildebrand <stewart.hildebrand@amd.com>,
	xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: Re: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Message-ID: <ZcswBFHtINB1XMAS@macbook>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <Zcsp15Aqve11Icjb@macbook>
 <ec5d0c39-1559-4f10-9574-98cfa0542993@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ec5d0c39-1559-4f10-9574-98cfa0542993@suse.com>

On Tue, Feb 13, 2024 at 09:44:58AM +0100, Jan Beulich wrote:
> On 13.02.2024 09:35, Roger Pau Monné wrote:
> > On Fri, Feb 02, 2024 at 04:33:05PM -0500, Stewart Hildebrand wrote:
> >> --- a/xen/include/xen/sched.h
> >> +++ b/xen/include/xen/sched.h
> >> @@ -462,7 +462,8 @@ struct domain
> >>  #ifdef CONFIG_HAS_PCI
> >>      struct list_head pdev_list;
> >>      /*
> >> -     * pci_lock protects access to pdev_list.
> >> +     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
> >> +     * structure from being removed.
> >>       *
> >>       * Any user *reading* from pdev_list, or from devices stored in pdev_list,
> >>       * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,
> >> @@ -628,6 +629,18 @@ struct domain
> >>      unsigned int cdf;
> >>  };
> >>  
> >> +/*
> >> + * Check for use in ASSERTs to ensure that:
> >> + *   1. we can *read* d->pdev_list
> >> + *   2. pdevs (belonging to this domain) do not go away
> >> + *   3. pdevs (belonging to this domain) do not get assigned to other domains
> > 
> > I think you can just state that this check ensures there will be no
> > changes to the entries in d->pdev_list, but not the contents of each
> > entry.  No changes to d->pdev_list already ensures not devices can be
> > deassigned or removed from the system, and obviously makes the list
> > safe to iterate against.
> > 
> > I would also drop the explicitly mention this is intended for ASSERT
> > usage: there's nothing specific in the code that prevents it from
> > being used in other places (albeit I think that's unlikely).
> 
> But pcidevs_locked(), resolving to spin_is_locked(), isn't reliable. The
> assertion usage is best-effort only, without a guarantee that all wrong
> uses would be caught.

Do we want to protect this with !NDEBUG guards then?

> >> + * This check is not suitable for protecting other state or critical regions.
> >> + */
> >> +#define pdev_list_is_read_locked(d) ({                           \
> > 
> > I would be tempted to drop at least the '_read_' part from the name,
> > the name is getting a bit too long for my taste.
> 
> While I agree with the long-ish aspect, I'm afraid the "read" part is
> crucial. As a result I see no room for shortening.

OK, if you think that's crucial then I'm not going to argue.

> >> +        struct domain *d_ = (d);                                 \
> > 
> > Why do you need this local domain variable?  Can't you use the d
> > parameter directly?
> 
> It would be evaluated then somewhere between 0 and 2 times.

It's ASSERT code only, so I don't see that as an issue.  Otherwise d_
needs to be made const.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 09:05:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 09:05:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679811.1057508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZojj-0007Nc-MM; Tue, 13 Feb 2024 09:05:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679811.1057508; Tue, 13 Feb 2024 09:05:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZojj-0007NV-Jt; Tue, 13 Feb 2024 09:05:55 +0000
Received: by outflank-mailman (input) for mailman id 679811;
 Tue, 13 Feb 2024 09:05:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZoji-0007NN-6D
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 09:05:54 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 167a202c-ca4f-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 10:05:52 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-410c1ebf5e4so16856165e9.2
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 01:05:52 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w17-20020a05600c475100b00411062c123esm3745234wmo.26.2024.02.13.01.05.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 01:05:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 167a202c-ca4f-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707815152; x=1708419952; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=BMyicW2A6mPqi/kvC1Q8vUVbx9Hqce/XZo/ttcaCmF0=;
        b=XM+gOXkGWSN8hfim9caARBdWvil1NJ6iksG4dH7+3n3TXRf5FyxK2IES+uxii6jR31
         jR4lRFyzNg4yKrV54U/aREyfcqG8Pvf4/GM/b1Aj0K3C6VwCpQ5stR1usOx2duaiMln4
         tctRvmHUuEgCLfr2bmIzxLNss1G0C5Z0JkWvlxVLQFRjptDi1mImU0IrghsJ51+haEDl
         9y4nq6zSLXWUVcA+jMH5XioZhK/NjXfWnOkV1fURusdNQxI8zRhMJSKXZrSiAMq9mfjD
         B/yhmUYdYNdIanes+ILRvlp8d2BtBCex62Qvymi6qsbKOlWsjFi/ywxjTm5qGv89Sjd6
         yKWQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707815152; x=1708419952;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=BMyicW2A6mPqi/kvC1Q8vUVbx9Hqce/XZo/ttcaCmF0=;
        b=FChC6Z9HN8O8QSB4nAOrT5AgyE9Jkw3qVktpEfxi9RTPE0y0JdG3+Kd7h60ln4jGvV
         xISnJT5/vcAYcDzlOcx6ooZxpIWxel3sHV/vGQx/l+Gjj+oJrlCD9SiUdHn2ucqRUbp/
         GliRqseZBh4XxZIoj1L+7vIoqAiHBcahSO91RZxC3JXVz8mOBOBPcEMFNcjZd+iC5Dxu
         xPxfOyvnUh3Inf8LuHh2MnljUL20TZ78AcIg1z0Q/T9HvkiTth1Ugb2qzENtMuIq9kVI
         YY7ddH+G7CwWtS7ptJknKzX9zRP+ByQ4Bl5qwyD9MHJWYyTZZRsLv6fLp9d1dAfhx6gR
         r25A==
X-Forwarded-Encrypted: i=1; AJvYcCXqxIEAYf9rKkYnsNsqfvOFrPnkPxgnJ5EET9kQYYw+qmiwf/EDWhv8iE8q80lxI0CJ9OkFfBZtEvwJrkgCXN/szmD2gut3mKyyax2rsBc=
X-Gm-Message-State: AOJu0Yww1f6LhtiWHEvzs1+WYHMK5w/8sG1yqw6W+xZYqRgkGjsBaHTQ
	Gle+xmCcNl2UWGajcHKV6RyBPzPZt3F6EgY9ESLoLPZ1a5UIzilW4NgRmdFNUg==
X-Google-Smtp-Source: AGHT+IE0BKijW7sROLdJocDekKlqhIY9zlVP0lOuaLywO6CtKW6F/5/pMUCBAjesEG/kX+P91kB8zA==
X-Received: by 2002:a05:600c:1551:b0:410:7bfe:86b5 with SMTP id f17-20020a05600c155100b004107bfe86b5mr9513836wmg.27.1707815151912;
        Tue, 13 Feb 2024 01:05:51 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVCQ6Q0kE60HE89gVS9f/sIDq6LykJltqjGwOIAqVccRu8v+F1a1+xBCaXOlcniPr//Xncj2NaH1llg4Icg2EungtikNDFFHgbgvzor+BJRBfe1aOjP1LPPROBXzyLonm020CzS4SU1n/AcF+Ku8ZN0xXCjpbMOLMpMdM7sW34W4ggqMx5YZIjkvd2Tkp63gVyldM1olZzYJVTdiIxUMhLLHsryp1ci13lSYBTjFAx+UTb0/tpp4OfhWH3YsrpPtNSuV9V0dfQDVUNLDmxQF250O7NtrlOpx5syKJQOJL1yBt+Rs1K6MrnDUoxchYqq3NDRgu2XRnix+HAqq2Sk8GWnY2BvfmNqexhea/4C7xy0ORpzbXBzTx3dMXXRtMerBN/IcwSqT462AZGRdjUevIBs3w==
Message-ID: <db0ce984-88e3-4394-b270-c5db585f8209@suse.com>
Date: Tue, 13 Feb 2024 10:05:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Stewart Hildebrand <stewart.hildebrand@amd.com>,
 xen-devel@lists.xenproject.org,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Paul Durrant <paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <Zcsp15Aqve11Icjb@macbook> <ec5d0c39-1559-4f10-9574-98cfa0542993@suse.com>
 <ZcswBFHtINB1XMAS@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcswBFHtINB1XMAS@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.02.2024 10:01, Roger Pau Monné wrote:
> On Tue, Feb 13, 2024 at 09:44:58AM +0100, Jan Beulich wrote:
>> On 13.02.2024 09:35, Roger Pau Monné wrote:
>>> On Fri, Feb 02, 2024 at 04:33:05PM -0500, Stewart Hildebrand wrote:
>>>> --- a/xen/include/xen/sched.h
>>>> +++ b/xen/include/xen/sched.h
>>>> @@ -462,7 +462,8 @@ struct domain
>>>>  #ifdef CONFIG_HAS_PCI
>>>>      struct list_head pdev_list;
>>>>      /*
>>>> -     * pci_lock protects access to pdev_list.
>>>> +     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
>>>> +     * structure from being removed.
>>>>       *
>>>>       * Any user *reading* from pdev_list, or from devices stored in pdev_list,
>>>>       * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,
>>>> @@ -628,6 +629,18 @@ struct domain
>>>>      unsigned int cdf;
>>>>  };
>>>>  
>>>> +/*
>>>> + * Check for use in ASSERTs to ensure that:
>>>> + *   1. we can *read* d->pdev_list
>>>> + *   2. pdevs (belonging to this domain) do not go away
>>>> + *   3. pdevs (belonging to this domain) do not get assigned to other domains
>>>
>>> I think you can just state that this check ensures there will be no
>>> changes to the entries in d->pdev_list, but not the contents of each
>>> entry.  No changes to d->pdev_list already ensures not devices can be
>>> deassigned or removed from the system, and obviously makes the list
>>> safe to iterate against.
>>>
>>> I would also drop the explicitly mention this is intended for ASSERT
>>> usage: there's nothing specific in the code that prevents it from
>>> being used in other places (albeit I think that's unlikely).
>>
>> But pcidevs_locked(), resolving to spin_is_locked(), isn't reliable. The
>> assertion usage is best-effort only, without a guarantee that all wrong
>> uses would be caught.
> 
> Do we want to protect this with !NDEBUG guards then?

Yes, that would look to be desirable.

>>>> + * This check is not suitable for protecting other state or critical regions.
>>>> + */
>>>> +#define pdev_list_is_read_locked(d) ({                           \
>>>
>>> I would be tempted to drop at least the '_read_' part from the name,
>>> the name is getting a bit too long for my taste.
>>
>> While I agree with the long-ish aspect, I'm afraid the "read" part is
>> crucial. As a result I see no room for shortening.
> 
> OK, if you think that's crucial then I'm not going to argue.
> 
>>>> +        struct domain *d_ = (d);                                 \
>>>
>>> Why do you need this local domain variable?  Can't you use the d
>>> parameter directly?
>>
>> It would be evaluated then somewhere between 0 and 2 times.
> 
> It's ASSERT code only, so I don't see that as an issue.

Fair point.

>  Otherwise d_ needs to be made const.

Indeed, but for assert-only code I agree the option is slightly better,
ideally suitably commented upon.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 09:10:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 09:10:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679818.1057520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZooK-0000ml-99; Tue, 13 Feb 2024 09:10:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679818.1057520; Tue, 13 Feb 2024 09:10:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZooK-0000me-4i; Tue, 13 Feb 2024 09:10:40 +0000
Received: by outflank-mailman (input) for mailman id 679818;
 Tue, 13 Feb 2024 09:10:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=m+vO=JW=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rZooJ-0000mY-1R
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 09:10:39 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bfc5c15a-ca4f-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 10:10:36 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id BB412211A3;
 Tue, 13 Feb 2024 09:10:34 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 59BDC13404;
 Tue, 13 Feb 2024 09:10:34 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id vXMTFAoyy2UvVAAAD6G6ig
 (envelope-from <jgross@suse.com>); Tue, 13 Feb 2024 09:10:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfc5c15a-ca4f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707815435; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2dBHyPlf5pqwHmlpBwT/zIVP0+EJA4iC+DUxlKWaHfo=;
	b=IAj6MF1EgCsSh15iPa9hS0zQwWHahdobn0sXraThyI/Gn+L1/Iy46uSfvw3XG+Q8GQ1y7V
	YFMpQSmXkcZfcf/vhPE221h9MvV3zOTXuDsPKtg42KQtPnuLw+LtDotnJ0OmECOz1seJyp
	MlF2Bo5TgjNCGErnVMTW0kkEsOH52Og=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707815434; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2dBHyPlf5pqwHmlpBwT/zIVP0+EJA4iC+DUxlKWaHfo=;
	b=ScCDbNo+7YVxtanr4RTnJ3Kh5K8Ou9076cv3r2+jRgjTXLBEsnVFk3/OX2Zaylpe0nQFex
	bexyFOoR0ufhcNF0md8VJitf7Fu6H86qOUjY7zcA+Gh+gV9SstaDsM+kKHMB0kMr/zDBZc
	TFLu57g9WDc2jKKx2NMxT6N/ScTUks0=
Message-ID: <49c3795f-7827-4e30-90d6-e49b2aa1a5bc@suse.com>
Date: Tue, 13 Feb 2024 10:10:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/events: close evtchn after mapping cleanup
Content-Language: en-US
To: Maximilian Heyne <mheyne@amazon.de>
Cc: Julien Grall <jgrall@amazon.com>, stable@vger.kernel.org,
 Andrew Panyakin <apanyaki@amazon.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Rahul Singh <rahul.singh@arm.com>, David Woodhouse <dwmw@amazon.co.uk>,
 Viresh Kumar <viresh.kumar@linaro.org>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>,
 xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
References: <20240124163130.31324-1-mheyne@amazon.de>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <20240124163130.31324-1-mheyne@amazon.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.38
X-Spamd-Result: default: False [-3.38 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_TWELVE(0.00)[13];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[11.amazon:url,amazon.de:email,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.91)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Flag: NO

On 24.01.24 17:31, Maximilian Heyne wrote:
> shutdown_pirq and startup_pirq are not taking the
> irq_mapping_update_lock because they can't due to lock inversion. Both
> are called with the irq_desc->lock being taking. The lock order,
> however, is first irq_mapping_update_lock and then irq_desc->lock.
> 
> This opens multiple races:
> - shutdown_pirq can be interrupted by a function that allocates an event
>    channel:
> 
>    CPU0                        CPU1
>    shutdown_pirq {
>      xen_evtchn_close(e)
>                                __startup_pirq {
>                                  EVTCHNOP_bind_pirq
>                                    -> returns just freed evtchn e
>                                  set_evtchn_to_irq(e, irq)
>                                }
>      xen_irq_info_cleanup() {
>        set_evtchn_to_irq(e, -1)
>      }
>    }
> 
>    Assume here event channel e refers here to the same event channel
>    number.
>    After this race the evtchn_to_irq mapping for e is invalid (-1).
> 
> - __startup_pirq races with __unbind_from_irq in a similar way. Because
>    __startup_pirq doesn't take irq_mapping_update_lock it can grab the
>    evtchn that __unbind_from_irq is currently freeing and cleaning up. In
>    this case even though the event channel is allocated, its mapping can
>    be unset in evtchn_to_irq.
> 
> The fix is to first cleanup the mappings and then close the event
> channel. In this way, when an event channel gets allocated it's
> potential previous evtchn_to_irq mappings are guaranteed to be unset already.
> This is also the reverse order of the allocation where first the event
> channel is allocated and then the mappings are setup.
> 
> On a 5.10 kernel prior to commit 3fcdaf3d7634 ("xen/events: modify internal
> [un]bind interfaces"), we hit a BUG like the following during probing of NVMe
> devices. The issue is that during nvme_setup_io_queues, pci_free_irq
> is called for every device which results in a call to shutdown_pirq.
> With many nvme devices it's therefore likely to hit this race during
> boot because there will be multiple calls to shutdown_pirq and
> startup_pirq are running potentially in parallel.
> 
>    ------------[ cut here ]------------
>    blkfront: xvda: barrier or flush: disabled; persistent grants: enabled; indirect descriptors: enabled; bounce buffer: enabled
>    kernel BUG at drivers/xen/events/events_base.c:499!
>    invalid opcode: 0000 [#1] SMP PTI
>    CPU: 44 PID: 375 Comm: kworker/u257:23 Not tainted 5.10.201-191.748.amzn2.x86_64 #1
>    Hardware name: Xen HVM domU, BIOS 4.11.amazon 08/24/2006
>    Workqueue: nvme-reset-wq nvme_reset_work
>    RIP: 0010:bind_evtchn_to_cpu+0xdf/0xf0
>    Code: 5d 41 5e c3 cc cc cc cc 44 89 f7 e8 2b 55 ad ff 49 89 c5 48 85 c0 0f 84 64 ff ff ff 4c 8b 68 30 41 83 fe ff 0f 85 60 ff ff ff <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 0f 1f 44 00 00
>    RSP: 0000:ffffc9000d533b08 EFLAGS: 00010046
>    RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000006
>    RDX: 0000000000000028 RSI: 00000000ffffffff RDI: 00000000ffffffff
>    RBP: ffff888107419680 R08: 0000000000000000 R09: ffffffff82d72b00
>    R10: 0000000000000000 R11: 0000000000000000 R12: 00000000000001ed
>    R13: 0000000000000000 R14: 00000000ffffffff R15: 0000000000000002
>    FS:  0000000000000000(0000) GS:ffff88bc8b500000(0000) knlGS:0000000000000000
>    CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>    CR2: 0000000000000000 CR3: 0000000002610001 CR4: 00000000001706e0
>    DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>    DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>    Call Trace:
>     ? show_trace_log_lvl+0x1c1/0x2d9
>     ? show_trace_log_lvl+0x1c1/0x2d9
>     ? set_affinity_irq+0xdc/0x1c0
>     ? __die_body.cold+0x8/0xd
>     ? die+0x2b/0x50
>     ? do_trap+0x90/0x110
>     ? bind_evtchn_to_cpu+0xdf/0xf0
>     ? do_error_trap+0x65/0x80
>     ? bind_evtchn_to_cpu+0xdf/0xf0
>     ? exc_invalid_op+0x4e/0x70
>     ? bind_evtchn_to_cpu+0xdf/0xf0
>     ? asm_exc_invalid_op+0x12/0x20
>     ? bind_evtchn_to_cpu+0xdf/0xf0
>     ? bind_evtchn_to_cpu+0xc5/0xf0
>     set_affinity_irq+0xdc/0x1c0
>     irq_do_set_affinity+0x1d7/0x1f0
>     irq_setup_affinity+0xd6/0x1a0
>     irq_startup+0x8a/0xf0
>     __setup_irq+0x639/0x6d0
>     ? nvme_suspend+0x150/0x150
>     request_threaded_irq+0x10c/0x180
>     ? nvme_suspend+0x150/0x150
>     pci_request_irq+0xa8/0xf0
>     ? __blk_mq_free_request+0x74/0xa0
>     queue_request_irq+0x6f/0x80
>     nvme_create_queue+0x1af/0x200
>     nvme_create_io_queues+0xbd/0xf0
>     nvme_setup_io_queues+0x246/0x320
>     ? nvme_irq_check+0x30/0x30
>     nvme_reset_work+0x1c8/0x400
>     process_one_work+0x1b0/0x350
>     worker_thread+0x49/0x310
>     ? process_one_work+0x350/0x350
>     kthread+0x11b/0x140
>     ? __kthread_bind_mask+0x60/0x60
>     ret_from_fork+0x22/0x30
>    Modules linked in:
>    ---[ end trace a11715de1eee1873 ]---
> 
> Fixes: d46a78b05c0e ("xen: implement pirq type event channels")
> Cc: stable@vger.kernel.org
> Co-debugged-by: Andrew Panyakin <apanyaki@amazon.com>
> Signed-off-by: Maximilian Heyne <mheyne@amazon.de>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen



From xen-devel-bounces@lists.xenproject.org Tue Feb 13 09:19:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 09:19:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679822.1057528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZowl-0001de-1S; Tue, 13 Feb 2024 09:19:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679822.1057528; Tue, 13 Feb 2024 09:19:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZowk-0001dX-Uo; Tue, 13 Feb 2024 09:19:22 +0000
Received: by outflank-mailman (input) for mailman id 679822;
 Tue, 13 Feb 2024 09:19:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZowj-0001br-4y
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 09:19:21 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f766fdf1-ca50-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 10:19:19 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-5118290fcd8so2793126e87.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 01:19:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 jh3-20020a05600ca08300b0040fc2f07ed5sm11216065wmb.27.2024.02.13.01.19.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 01:19:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f766fdf1-ca50-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707815959; x=1708420759; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nKj/+wciyxJwE0z01LK1QsewmS+8/rBBWDtjfT3M9Q0=;
        b=JLQraWFi7q6gWCGvcNuNWGqL3AzoRGtbGE9T4WEfjFP2/gSmQi+yqg4bh8IR4gLauU
         ayhLpEJ2qLQNNNw2XH+An4kNkVKmBZUQpoehV10PblIGNe7x20AI3eZ7mwW9xN3lY421
         e8wAVhK7fwjRXoROtsTaROA6xqCDo06Ubf007hvK94AmtEHV0uqK2vVUGiOI0inz8Ous
         1f0G99sl9Ir33wBtOin05ahCZDlCK1MTSZqLCi78E7TtDWR3yY+Ynf1GCSVx6SU28Tzb
         svYXl5xpONcJAu6yuwiU1nRzrnSk57Pk9HFS7tAce5jaGU9JZaPYw7JfhieFXRHJLu23
         8BpQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707815959; x=1708420759;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nKj/+wciyxJwE0z01LK1QsewmS+8/rBBWDtjfT3M9Q0=;
        b=AYEppFVr4jre+652/fTGryKzZTHIq7EwgdjwZIQyctpc88amcjD+y9hRK3oyIkC8C1
         iB47n5hjLlOY3tMw2VNdUYb8ddH9BzeYBoY10RJ8L8+FopsEjXUzklze4RWl4De9LTuF
         FctndcgV4bOZPBPiZt/KwLJ1HnZeHXSfate55Z3S7363vfM6OkRzwf+jv2nK0T7PBD+Y
         ZoYqCOK8qD/kZRDm540y98eY5eBTRge/0MmrdLsrAMY+MgswYV12EszEhMUREIvyu4A1
         dMS/WCLkp4kja7WwhSiywl0EPdNLLSWc44tg5VutvuBZ5FZaI4x8CLJjQs0JUtUtPpQB
         lSCQ==
X-Forwarded-Encrypted: i=1; AJvYcCXnCgknNLj8iFdlw5azZoPZHMzhv2yP55VNJ8IF+3qzRdhtyuIMZcbFfBGrJmYHp7EL1TwJqo6C4LezvuimQrtUscyeDurcdCDHy4IVnE4=
X-Gm-Message-State: AOJu0Yx/0bVGsPe0OL/FXpYLHOJQi32PbfnG0q1XH7+lq6Ro2CArWQDZ
	24vpI/7yPT0U63Hgku94lEQbLYTekoSZGnwYlVbxnQR8PQmWKzwZXF+WPN3yqA==
X-Google-Smtp-Source: AGHT+IFT20TzYtnYSW2IvS5yNTIqq1fIVMYvqpDzh9tdyUr70orVtw/WpHosKChqq8Hv8psiSAJZFg==
X-Received: by 2002:ac2:5312:0:b0:511:79f3:c217 with SMTP id c18-20020ac25312000000b0051179f3c217mr5748067lfh.39.1707815958701;
        Tue, 13 Feb 2024 01:19:18 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXtMCFHiUKWCIpFwkdxYIPbHjbeLMLUU0+uIL3tj5yVJDtiGRkN5dO1egUKZ3jogj0T6AlFCfE1lIXMCnIrWhu45vjHuaTLDPEYK7az5csXgXG+zCjpJu27iOombErRQMnxRjrbRxMT87Br71X6by8Cij9xkqd9/pMsR2utBGKEhP5u30JIa5BwxeFJR69bfco/iRA5zvbXUPBbtu94/JvQJpkQTh6bYMZBnmzbtvar8ndqWgOtF/KFOEvntVyspNy5Nz6OQNP5+hKFtT6Nf7pCa+ujEhHXKIXxT+mNv3BaOjwAwmF0v5k=
Message-ID: <28b843a6-cf88-4443-b9c8-54561b9e753a@suse.com>
Date: Tue, 13 Feb 2024 10:19:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 09/30] xen/riscv: introduce bitops.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <f76bd85f4b64a47c59c0b306ce425036819fa380.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f76bd85f4b64a47c59c0b306ce425036819fa380.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/include/asm/config.h
> +++ b/xen/arch/riscv/include/asm/config.h
> @@ -50,6 +50,8 @@
>  # error "Unsupported RISCV variant"
>  #endif
>  
> +#define BITS_PER_BYTE 8
> +
>  #define BYTES_PER_LONG (1 << LONG_BYTEORDER)
>  #define BITS_PER_LONG  (BYTES_PER_LONG << 3)
>  #define POINTER_ALIGN  BYTES_PER_LONG

How does this change relate to this patch? I can't see the new symbol
being used anywhere.

> --- /dev/null
> +++ b/xen/include/asm-generic/bitops/__ffs.h
> @@ -0,0 +1,47 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _ASM_GENERIC_BITOPS___FFS_H_
> +#define _ASM_GENERIC_BITOPS___FFS_H_
> +
> +/**
> + * ffs - find first bit in word.

__ffs ? Or wait, ...

> + * @word: The word to search
> + *
> + * Returns 0 if no bit exists, otherwise returns 1-indexed bit location.

... this actually describes ffs(), not __ffs(), and the implementation
doesn't match the description. The correct description for this function
(as Linux also has it)

 * Undefined if no bit exists, so code should check against 0 first.

Which raises a question regarding "Taken from Linux-6.4.0-rc1" in the
description. ffs.h pretty clearly also doesn't come from there.

I first I thought I might withdraw my earlier request to split all of
this up. But with just these two observations I now feel it's even
more important that you do, so every piece can be properly attributed
to (and then checked for) its origin.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 10:23:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 10:23:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679832.1057539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZpwY-00033r-Oj; Tue, 13 Feb 2024 10:23:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679832.1057539; Tue, 13 Feb 2024 10:23:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZpwY-00033k-M1; Tue, 13 Feb 2024 10:23:14 +0000
Received: by outflank-mailman (input) for mailman id 679832;
 Tue, 13 Feb 2024 10:23:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rZpwX-00033e-Mm
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 10:23:13 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZpwS-000441-Dx; Tue, 13 Feb 2024 10:23:08 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZpwS-0004Ug-56; Tue, 13 Feb 2024 10:23:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=M1qRgO2FVblnULgmMiZEVD2wNb+nBPTJELbkpwcrLc8=; b=z7ry7R40n5ijqKaUbAzU+cxCPH
	Be7RkjDbOj9ZEfTYmQGuxkVPXB+qLogh9cBZhxzLR2RUyFhd5GUpxDZ8vNR6WFGHqMf5S4wJDF3va
	EzBK7WLI2Hid8NDUv8DoGXucKrEGaP2KWAhwcgi63pcJFzsu8BravS4Zl/tTTjejL+No=;
Message-ID: <19e6204f-9adc-47ef-92ca-e7f941035ca6@xen.org>
Date: Tue, 13 Feb 2024 10:23:05 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2] docs/misra: add asm-offset.c to exclude-list
Content-Language: en-GB
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: Jan Beulich <jbeulich@suse.com>, sstabellini@kernel.org,
 michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com,
 consulting@bugseng.com, andrew.cooper3@citrix.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <09b659e19bf2cc6b3ee4320e019bdfa7def5f3b8.1707406598.git.nicola.vetrini@bugseng.com>
 <891e8cc2-e073-4cd3-9998-4c01ae775a3b@suse.com>
 <8977390f-9549-40d5-bfdc-5c3da81521fc@xen.org>
 <efc0eee4-9903-4228-b33e-cb50d4c545ee@suse.com>
 <5cdae9ce-d2ff-40bd-bc07-aa1965275833@xen.org>
 <f6c91540dbb5a83d3172bb65b6d4ca4b@bugseng.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <f6c91540dbb5a83d3172bb65b6d4ca4b@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Nicola,

On 09/02/2024 13:20, Nicola Vetrini wrote:
> On 2024-02-09 13:17, Julien Grall wrote:
>> Hi Jan,
>>
>> On 09/02/2024 11:46, Jan Beulich wrote:
>>> On 09.02.2024 10:40, Julien Grall wrote:
>>>> Replying on the v2 as well.
>>>
>>> And answering here despite the respective question was raised on the
>>> v1 thread: I'm certainly okay with the more detailed commit message.
>>
>> Ah yes. Sorry, I replied to v1 first and then realized it may have 
>> been easier to comment on v2.
>>
>>> A few nits, though:
>>>
>>>> On 08/02/2024 15:56, Jan Beulich wrote:
>>>>> On 08.02.2024 16:50, Nicola Vetrini wrote:
>>>>>> These files contain several deliberate violations of MISRA C rules 
>>>>>> and
>>>>>> they are not linked in the final Xen binary, therefore they can be 
>>>>>> exempted
>>>>>> from MISRA compliance.
>>>>
>>>> I'd like the commit message to be expanded a little bit to explain 
>>>> which
>>>> MISRA rules are a problem. This helped me to understand why we excluded
>>>> rather than fixed.
>>>>
>>>> Base on the previous discussion, I would suggest:
>>>>
>>>> These files contain several deliberate violation of MISRA C rules 
>>>> such as:
>>>
>>> violations
>>>
>>>>     * R20.12 on Arm for macros DEFINE and OFFSET, where the second
>>>> argument of OFFSET is a macro and is used as a normal parameter and a
>>>> stringification operand.
>>>
>>> Is this really for Arm only?
>>
>> I don't exactly know. I took Nicola's comment and massage it for the 
>> commit message. I am assuming that this was also not exhaustive list, 
>> so my aim was to only provide some example.
>>
> 
> Yes, indeed. I mentioned the first two examples that I remembered.
> 
>> Thinking of it, I don't see why it would only be a problem on Arm. I 
>> can drop the "on Arm".
>>
> 
> My bad, I missed it in the output. Please drop the "on Arm" part.
> 
>>>
>>>>     * R2.1 because the file is not linked That said it was decided to
>>>> deviate the rule itselfed to deviate that aspect).
>>>
>>> There look to be punctuation issues here. Also s/itselfed/itself/, and
>>> the duplicate "deviate" is also a little odd to read (maybe "deal with"
>>> or "address" in place of the 2nd instance).
>>
>> Doh, indeed. This wants to be:
>>
>> "R2.1 because the file is not linked. That said, it was decided to 
>> deviate the ruule itself to address that aspect."
>          ^ rule
>>>> The files are also not linked in the final Xen binary, therefore they
>>>> can be expempted from MISRA compliance.
>>>
>>> Looks to duplicate what the latter half of the 2nd bullet point has.
>>> If to be kept: s/expempted/exempted/.
>>
>> I will remove.
>>
>>>
>>>>>> --- a/docs/misra/exclude-list.json
>>>>>> +++ b/docs/misra/exclude-list.json
>>>>>> @@ -101,6 +101,10 @@
>>>>>>                "rel_path": "arch/x86/efi/check.c",
>>>>>>                "comment": "The resulting code is not included in 
>>>>>> the final Xen binary, ignore for now"
>>>>>>            },
>>>>>> +        {
>>>>>> +          "rel_path": "arch/*/*/asm-offsets.c",
>>>>>> +          "comment": "The resulting code is not included in the 
>>>>>> final Xen binary, ignore for now"
>>>>>> +        },
>>>>>>            {
>>>>>>                "rel_path": "common/coverage/*",
>>>>>>                "comment": "Files to support gcov, ignore for now"
>>>>>
>>>>> ... something looks odd with indentation; can probably be adjusted
>>>>> while committing.
>>>>
>>>> I am happy to take care of both the commit message and the indentation
>>>> on commit.
>>>
>>> Okay, I'll leave that to you then.
>>
>> Thanks. I will do it shortly.
> 
> Thanks,

I have committed the patch now.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 10:37:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 10:37:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679836.1057549 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZqAb-0004wZ-0T; Tue, 13 Feb 2024 10:37:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679836.1057549; Tue, 13 Feb 2024 10:37:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZqAa-0004wS-St; Tue, 13 Feb 2024 10:37:44 +0000
Received: by outflank-mailman (input) for mailman id 679836;
 Tue, 13 Feb 2024 10:37:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZqAZ-0004wM-E8
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 10:37:43 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e99cd0ed-ca5b-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 11:37:40 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-33aeb088324so2652101f8f.2
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 02:37:40 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 k6-20020a5d66c6000000b00337d5cd0d8asm9147439wrw.90.2024.02.13.02.37.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 02:37:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e99cd0ed-ca5b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707820660; x=1708425460; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=EN1HYj0UUp3SXtRVAvgiI8RUuy9KS8jyzA8NlNUJeUE=;
        b=ZPmRvx/aCYY/lbdb+ez/nTyR6pjcKiIiqw9nNpAHem/MklUSDd+DEkcCP4OIWpVyuA
         Wxacn0Rr06DiS30TZyZu/r0Qfv4PHpPrKYK8ohORwKn5Rxxw35GclZwitn4pp1dSeQBv
         Ov5Qw+RWhYqmsYgvQsh7QPE41o5MRza44ix0PtPHX9/Vm7GtnJRD0FYEaTrTaUhGIHVP
         YbQZtkv78o0FUKpMs0QNNnxT/2i6NUP//ufaU9fs+FvHOw+6HoY/24fjFAVHjP5dCSJQ
         DNp7Z8RL0qIvzTSl8hWOQDEYzvgkoBh5qpRGLJtx1JrriN00GYEN1WfCVFDZ6OIIsAAZ
         g+LQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707820660; x=1708425460;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=EN1HYj0UUp3SXtRVAvgiI8RUuy9KS8jyzA8NlNUJeUE=;
        b=sdUoq1O6RuOLufJMPyfaHqimIL7nO2pG6BA6O1pCGLOy75Tux54HlP+jmYqeIfj9aO
         jlYFcYlPGhb09/SXmTJVUCXPv/MKKBv/iN2funVXTUoArML2WyKLpoQAGpZd0UFedvZS
         sSI+lwmiFHQPTG0X4gDxrTGh6SozVw2fexK3bT89Z6K4LauQ6sm458KNYiG3x8ToJYTi
         Mj4xXB9hK7LKrjpXA2CApugxO0dkhXft+1ii3tFnzwAobgmU8jd45JQqdEWYi/GFjYfV
         B0zlwLhrPRBzehIk8e0Xk/RcbzLaYuSu6s11vuQEx/X6HF+cesubs0peNmhgtMCx8Hx7
         AupQ==
X-Forwarded-Encrypted: i=1; AJvYcCVE7P5v7DAFXf7JO10u/lU/I+LzTBvytNBWtGffmCxqJybzYsFsrO6zrQUY4M7USxtVtHBf+SJ6qarBylei/mh49kujyqgwZdY8oXsHRyo=
X-Gm-Message-State: AOJu0YzsRXEqyMUc9PtHlfeOrTCKZUdI87EMMyA0hxomILs+2Hv70XXW
	9+dszNDcIf50Ksaqjjr+KjF5Sa70rqdCREaEK0nxZ1DoeIuSUhskP994mij+Tg==
X-Google-Smtp-Source: AGHT+IFyOyLTzs7H6GCMbIvO7pJX3sch+i0SKAU/kb0VQlej+IDBZLbuZurg1XeS+kkfW7XdjctoZg==
X-Received: by 2002:a5d:6382:0:b0:33b:1fda:5226 with SMTP id p2-20020a5d6382000000b0033b1fda5226mr6696231wru.38.1707820660018;
        Tue, 13 Feb 2024 02:37:40 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU2o2muSE/byK9n1YDKqsKUbcB9vqVK39ZWJR2OIjJToIh44JI1FWVBANZvJY9Qvfhty21RNDtJOUv9W/TJzphj98zHmWhc3wbLbK3ipXetVn4vu8PMeoBAYADhmyRAUhrxOanTL0dSl42cCl8gBicn2axNR95o1sIkaJ6bx5kr9bLA5tovGXPnDISofc9PTmEwiSP/N7jDZ8MBtP2mj84pySEmk5ZasfqV1zmE8nSoHrHfeVZ1iDTRWDYIMmioXYkt6062OIdGzZEzgJwEGaFRjcl/jJSSJPFwH5e+Lo1XHu/aquZx83w=
Message-ID: <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
Date: Tue, 13 Feb 2024 11:37:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/cmpxchg.h
> @@ -0,0 +1,237 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/* Copyright (C) 2014 Regents of the University of California */
> +
> +#ifndef _ASM_RISCV_CMPXCHG_H
> +#define _ASM_RISCV_CMPXCHG_H
> +
> +#include <xen/compiler.h>
> +#include <xen/lib.h>
> +
> +#include <asm/fence.h>
> +#include <asm/io.h>
> +#include <asm/system.h>
> +
> +#define ALIGN_DOWN(addr, size)  ((addr) & (~((size) - 1)))

This feels risky: Consider what happens when someone passes 2U as 2nd argument.
The cheapest adjustment to make would be to use 1UL in the expression.

> +#define __amoswap_generic(ptr, new, ret, sfx, release_barrier, acquire_barrier) \
> +({ \
> +    asm volatile( \
> +        release_barrier \
> +        " amoswap" sfx " %0, %2, %1\n" \

While I won't insist, the revision log says \n were dropped from asm()
where not needed. A separator is needed here only if ...

> +        acquire_barrier \

... this isn't blank. Which imo suggests that the separator should be
part of the argument passed in. But yes, one can view this differently,
hence why I said I won't insist.

As to the naming of the two  - I'd generally suggest to make as litte
implications as possible: It doesn't really matter here whether it's
acquire or release; that matters at the use sites. What matters here
is that one is a "pre" barrier and the other is a "post" one.

> +        : "=r" (ret), "+A" (*ptr) \
> +        : "r" (new) \
> +        : "memory" ); \
> +})
> +
> +#define emulate_xchg_1_2(ptr, new, ret, release_barrier, acquire_barrier) \
> +({ \
> +    uint32_t *ptr_32b_aligned = (uint32_t *)ALIGN_DOWN((unsigned long)ptr, 4); \

You now appear to assume that this macro is only used with inputs not
crossing word boundaries. That's okay as long as suitably guaranteed
at the use sites, but imo wants saying in a comment.

> +    uint8_t mask_l = ((unsigned long)(ptr) & (0x8 - sizeof(*ptr))) * BITS_PER_BYTE; \

Why 0x8 (i.e. spanning 64 bits), not 4 (matching the uint32_t use
above)?

> +    uint8_t mask_size = sizeof(*ptr) * BITS_PER_BYTE; \
> +    uint8_t mask_h = mask_l + mask_size - 1; \
> +    unsigned long mask = GENMASK(mask_h, mask_l); \

Personally I find this confusing, naming-wise: GENMASK() takes bit
positions as inputs, not masks. (Initially, because of this, I
thought the calculations all can't be quite right.)

> +    unsigned long new_ = (unsigned long)(new) << mask_l; \
> +    unsigned long ret_; \
> +    unsigned long rc; \

Similarly, why unsigned long here?

I also wonder about the mix of underscore suffixed (or not) variable
names here.

> +    \
> +    asm volatile( \

Nit: Missing blank before opening parenthesis.

> +        release_barrier \
> +        "0: lr.d %0, %2\n" \

Even here it's an 8-byte access. Even if - didn't check - the insn was
okay to use with just a 4-byte aligned pointer, wouldn't it make sense
then to 8-byte align it, and be consistent throughout this macro wrt
the base unit acted upon? Alternatively, why not use lr.w here, thus
reducing possible collisions between multiple CPUs accessing the same
cache line?

> +        "   and  %1, %0, %z4\n" \
> +        "   or   %1, %1, %z3\n" \
> +        "   sc.d %1, %1, %2\n" \
> +        "   bnez %1, 0b\n" \
> +        acquire_barrier \
> +        : "=&r" (ret_), "=&r" (rc), "+A" (*ptr_32b_aligned) \
> +        : "rJ" (new_), "rJ" (~mask) \

I think that as soon as there are more than 2 or maybe 3 operands,
legibility is vastly improved by using named asm() operands.

> +        : "memory"); \

Nit: Missing blank before closing parenthesis.

> +    \
> +    ret = (__typeof__(*(ptr)))((ret_ & mask) >> mask_l); \
> +})

Why does "ret" need to be a macro argument? If you had only the
expression here, not the the assigment, ...

> +#define __xchg_generic(ptr, new, size, sfx, release_barrier, acquire_barrier) \
> +({ \
> +    __typeof__(ptr) ptr__ = (ptr); \

Is this local variable really needed? Can't you use "ptr" directly
in the three macro invocations?

> +    __typeof__(*(ptr)) new__ = (new); \
> +    __typeof__(*(ptr)) ret__; \
> +    switch (size) \
> +    { \
> +    case 1: \
> +    case 2: \
> +        emulate_xchg_1_2(ptr__, new__, ret__, release_barrier, acquire_barrier); \

... this would become

        ret__ = emulate_xchg_1_2(ptr__, new__, release_barrier, acquire_barrier); \

But, unlike assumed above, there's no enforcement here that a 2-byte
quantity won't cross a word, double-word, cache line, or even page
boundary. That might be okay if then the code would simply crash (like
the AMO insns emitted further down would), but aiui silent misbehavior
would result.

Also nit: The switch() higher up is (still/again) missing blanks.

> +        break; \
> +    case 4: \
> +        __amoswap_generic(ptr__, new__, ret__,\
> +                          ".w" sfx,  release_barrier, acquire_barrier); \
> +        break; \
> +    case 8: \
> +        __amoswap_generic(ptr__, new__, ret__,\
> +                          ".d" sfx,  release_barrier, acquire_barrier); \
> +        break; \
> +    default: \
> +        STATIC_ASSERT_UNREACHABLE(); \
> +    } \
> +    ret__; \
> +})
> +
> +#define xchg_relaxed(ptr, x) \
> +({ \
> +    __typeof__(*(ptr)) x_ = (x); \
> +    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)), "", "", ""); \
> +})
> +
> +#define xchg_acquire(ptr, x) \
> +({ \
> +    __typeof__(*(ptr)) x_ = (x); \
> +    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)), \
> +                                       "", "", RISCV_ACQUIRE_BARRIER); \
> +})
> +
> +#define xchg_release(ptr, x) \
> +({ \
> +    __typeof__(*(ptr)) x_ = (x); \
> +    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)),\
> +                                       "", RISCV_RELEASE_BARRIER, ""); \
> +})
> +
> +#define xchg(ptr,x) \
> +({ \
> +    __typeof__(*(ptr)) ret__; \
> +    ret__ = (__typeof__(*(ptr))) \
> +            __xchg_generic(ptr, (unsigned long)(x), sizeof(*(ptr)), \
> +                           ".aqrl", "", ""); \

The .aqrl doesn't look to affect the (emulated) 1- and 2-byte cases.

Further, amoswap also exists in release-only and acquire-only forms.
Why do you prefer explicit barrier insns over those? (Looks to
similarly apply to the emulation path as well as to the cmpxchg
machinery then, as both lr and sc also come in all four possible
acquire/release forms. Perhaps for the emulation path using
explicit barriers is better, in case the acquire/release forms of
lr/sc - being used inside the loop - might perform worse.)

> +    ret__; \
> +})
> +
> +#define __generic_cmpxchg(ptr, old, new, ret, lr_sfx, sc_sfx, release_barrier, acquire_barrier)	\
> + ({ \
> +    register unsigned int rc; \
> +    asm volatile( \
> +        release_barrier \
> +        "0: lr" lr_sfx " %0, %2\n" \
> +        "   bne  %0, %z3, 1f\n" \
> +        "   sc" sc_sfx " %1, %z4, %2\n" \
> +        "   bnez %1, 0b\n" \
> +        acquire_barrier \
> +        "1:\n" \
> +        : "=&r" (ret), "=&r" (rc), "+A" (*ptr) \
> +        : "rJ" (old), "rJ" (new) \
> +        : "memory"); \
> + })
> +
> +#define emulate_cmpxchg_1_2(ptr, old, new, ret, sc_sfx, release_barrier, acquire_barrier) \
> +({ \
> +    uint32_t *ptr_32b_aligned = (uint32_t *)ALIGN_DOWN((unsigned long)ptr, 4); \
> +    uint8_t mask_l = ((unsigned long)(ptr) & (0x8 - sizeof(*ptr))) * BITS_PER_BYTE; \
> +    uint8_t mask_size = sizeof(*ptr) * BITS_PER_BYTE; \
> +    uint8_t mask_h = mask_l + mask_size - 1; \
> +    unsigned long mask = GENMASK(mask_h, mask_l); \
> +    unsigned long old_ = (unsigned long)(old) << mask_l; \
> +    unsigned long new_ = (unsigned long)(new) << mask_l; \
> +    unsigned long ret_; \
> +    unsigned long rc; \
> +    \
> +    __asm__ __volatile__ ( \
> +        release_barrier \
> +        "0: lr.d %0, %2\n" \
> +        "   and  %1, %0, %z5\n" \
> +        "   bne  %1, %z3, 1f\n" \
> +        "   and  %1, %0, %z6\n" \

Isn't this equivalent to

        "   xor  %1, %1, %0\n" \

this eliminating one (likely register) input?

Furthermore with the above and ...

> +        "   or   %1, %1, %z4\n" \
> +        "   sc.d" sc_sfx " %1, %1, %2\n" \
> +        "   bnez %1, 0b\n" \

... this re-written to

        "   xor  %0, %1, %0\n" \
        "   or   %0, %0, %z4\n" \
        "   sc.d" sc_sfx " %0, %0, %2\n" \
        "   bnez %0, 0b\n" \

you'd then no longer clobber the ret_ & mask you've already calculated
in %1, so ...

> +        acquire_barrier \
> +        "1:\n" \
> +        : "=&r" (ret_), "=&r" (rc), "+A" (*ptr_32b_aligned) \
> +        : "rJ" (old_), "rJ" (new_), \
> +          "rJ" (mask), "rJ" (~mask) \
> +        : "memory"); \
> +    \
> +    ret = (__typeof__(*(ptr)))((ret_ & mask) >> mask_l); \

... you could use rc here. (Of course variable naming or use then may
want changing, assuming I understand why "rc" is named the way it is.)

> +})
> +
> +/*
> + * Atomic compare and exchange.  Compare OLD with MEM, if identical,
> + * store NEW in MEM.  Return the initial value in MEM.  Success is
> + * indicated by comparing RETURN with OLD.
> + */
> +#define __cmpxchg_generic(ptr, old, new, size, sc_sfx, release_barrier, acquire_barrier) \
> +({ \
> +    __typeof__(ptr) ptr__ = (ptr); \
> +    __typeof__(*(ptr)) old__ = (__typeof__(*(ptr)))(old); \
> +    __typeof__(*(ptr)) new__ = (__typeof__(*(ptr)))(new); \
> +    __typeof__(*(ptr)) ret__; \
> +    switch (size) \
> +    { \
> +    case 1: \
> +    case 2: \
> +        emulate_cmpxchg_1_2(ptr, old, new, ret__,\
> +                            sc_sfx, release_barrier, acquire_barrier); \
> +        break; \
> +    case 4: \
> +        __generic_cmpxchg(ptr__, old__, new__, ret__, \
> +                          ".w", ".w"sc_sfx, release_barrier, acquire_barrier); \
> +        break; \
> +    case 8: \
> +        __generic_cmpxchg(ptr__, old__, new__, ret__, \
> +                          ".d", ".d"sc_sfx, release_barrier, acquire_barrier); \
> +        break; \
> +    default: \
> +        STATIC_ASSERT_UNREACHABLE(); \
> +    } \
> +    ret__; \
> +})
> +
> +#define cmpxchg_relaxed(ptr, o, n) \
> +({ \
> +    __typeof__(*(ptr)) o_ = (o); \
> +    __typeof__(*(ptr)) n_ = (n); \
> +    (__typeof__(*(ptr)))__cmpxchg_generic(ptr, \
> +                    o_, n_, sizeof(*(ptr)), "", "", ""); \
> +})
> +
> +#define cmpxchg_acquire(ptr, o, n) \
> +({ \
> +    __typeof__(*(ptr)) o_ = (o); \
> +    __typeof__(*(ptr)) n_ = (n); \
> +    (__typeof__(*(ptr)))__cmpxchg_generic(ptr, o_, n_, sizeof(*(ptr)), \
> +                                          "", "", RISCV_ACQUIRE_BARRIER); \
> +})
> +
> +#define cmpxchg_release(ptr, o, n) \
> +({ \
> +    __typeof__(*(ptr)) o_ = (o); \
> +    __typeof__(*(ptr)) n_ = (n); \
> +    (__typeof__(*(ptr)))__cmpxchg_release(ptr, o_, n_, sizeof(*(ptr)), \
> +                                          "", RISCV_RELEASE_BARRIER, ""); \
> +})
> +
> +#define cmpxchg(ptr, o, n) \
> +({ \
> +    __typeof__(*(ptr)) ret__; \
> +    ret__ = (__typeof__(*(ptr))) \
> +            __cmpxchg_generic(ptr, (unsigned long)(o), (unsigned long)(n), \
> +                              sizeof(*(ptr)), ".rl", "", " fence rw, rw\n"); \

No RISCV_..._BARRIER for use here and ...

> +    ret__; \
> +})
> +
> +#define __cmpxchg(ptr, o, n, s) \
> +({ \
> +    __typeof__(*(ptr)) ret__; \
> +    ret__ = (__typeof__(*(ptr))) \
> +            __cmpxchg_generic(ptr, (unsigned long)(o), (unsigned long)(n), \
> +                              s, ".rl", "", " fence rw, rw\n"); \

... here? And anyway, wouldn't it make sense to have

#define cmpxchg(ptr, o, n) __cmpxchg(ptr, o, n, sizeof(*(ptr))

to limit redundancy?

Plus wouldn't

#define __cmpxchg(ptr, o, n, s) \
    ((__typeof__(*(ptr))) \
     __cmpxchg_generic(ptr, (unsigned long)(o), (unsigned long)(n), \
                       s, ".rl", "", " fence rw, rw\n"))

be shorter and thus easier to follow as well? As I notice only now,
this would apparently apply further up as well.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 11:05:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 11:05:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679841.1057558 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZqb4-0000rP-0S; Tue, 13 Feb 2024 11:05:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679841.1057558; Tue, 13 Feb 2024 11:05:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZqb3-0000rI-UD; Tue, 13 Feb 2024 11:05:05 +0000
Received: by outflank-mailman (input) for mailman id 679841;
 Tue, 13 Feb 2024 11:05:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZqb2-0000rC-Jq
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 11:05:04 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bc88ecfb-ca5f-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 12:05:03 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-339289fead2so2709705f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 03:05:03 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e27-20020a5d595b000000b0033b80a0d002sm5924940wri.57.2024.02.13.03.05.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 03:05:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc88ecfb-ca5f-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707822302; x=1708427102; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=zkomEXbfXI/7ZZptfA2/rfk5DOgP+KKcci0WVktYpTg=;
        b=XlmRjHhbbPDEQW0vEyA96Hci/fnwPxs3jQ9EhFmOUcY1w4MnvNLKxSrLxPYTHaGyIg
         zBC2UXg6aE4ZY52kz8usVVngrqxPgSspaP44I7CeaZ4L5JpdtfTAZ02/H5V6x9XYkv3S
         hFf54TZ/YKQ3C/F8aNhydR9UqrRBFMCDCT85KvSiWPlLL+VeRDf1q+sf8p+hrU8CAq8d
         BKncisOUl0x417mH96Tr4W5S072MPe70kUTEdTtrySEqtlf5C7r99MqXNvcm/Ea01bXr
         Q+6fHuegSJkiDMuNadYxWas61dcvf/JMUj7eKnIK7Id3SVLkZvGVhekeG8R2fj5p9keS
         AxdA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707822302; x=1708427102;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=zkomEXbfXI/7ZZptfA2/rfk5DOgP+KKcci0WVktYpTg=;
        b=j06++yjHGJ8t/WosxNjBYoKlMbQSms7Be2dHLBAoEVXhkHuID3q58HhLdjhwoHZGov
         RspPZkbkhskP7iSMZRKeXR2Sj3JqkRzu9yA6f7A1/ZkXIwCYEtqUZPDGjC8QfCiEWA1a
         jvhXPpCafzS8ajlKUAPH4ofB4o0C5faasKPgxO4JtXcFTm/Jz8mOUYhM7EluTM4q6ole
         htCvP/OtG9Yh0qrP4GsSpU3+/5fZrtzKw9P09+wMHXL0/wy8hQCD+g81J4f7v3/rjNkz
         4qMNTauFPGxfiluORSG1qC/68buKZrUquXgP/DvUUAQbcLoiZ+GeIJAtRNuV9VsPwPd/
         6Vqw==
X-Forwarded-Encrypted: i=1; AJvYcCVdBcpCgE0QqsV7ygKpZ3miJiOVYmD8yRcQFm13E8AYzHTaGi5Uay7Cgy8yyAtAcpBqa0hqSJ8DZ7Zk9vPKguVE+gFnXIFXXbhVkisKNdI=
X-Gm-Message-State: AOJu0YzwKvYzAiTCORQUjJ51tF4BftU/jbXeZUktoPCcI637tugsu3y1
	rWR48IxubF0X0ktBcABHEdDxady07BPndMdTSQjt4c5v5Akz96SmPatSyK8jpg==
X-Google-Smtp-Source: AGHT+IE3xHY382busbeHjY3skFDuUEygR1VQgBNmtSRZE4gZNUZYy+VQyqLy6GYYJUVZqJrTeel+JQ==
X-Received: by 2002:adf:f582:0:b0:33b:237b:22c2 with SMTP id f2-20020adff582000000b0033b237b22c2mr6585437wro.21.1707822302459;
        Tue, 13 Feb 2024 03:05:02 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWO3F7Xxte6KTRfftuOyC9eDRbfSjQpnbSkP+/aPmWruLpS1+uPKV+03x21BsacpHKWDAemcb/ouz04b4YXeTzhzRLjnlntRaLjvASnKy37ShHumnkNZoVx93jX+R2l0uDnSgOKZwAcxdZXv7ed+rBQAwDbS14/4lI0FkZffFwCJOp6iN2tTzzfMVJHi6Ndu5sax7neQhd/DxU/QJ3pwVtnzKjOzV3GZXBdLKlwB6BomW8mDTF+xy8GI0T6bz2hKxr7J2S1n9bBh8oBBLatWX9cXpZr3rRqhQw+eBpc50JaHuY6s3/Wd7I=
Message-ID: <5508296c-2721-43e4-83d3-2603ce31d010@suse.com>
Date: Tue, 13 Feb 2024 12:05:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 13/30] xen/riscv: introduce io.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <64afc021f680e55d486f8429c02513bc9bf4b9ea.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64afc021f680e55d486f8429c02513bc9bf4b9ea.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> The header taken form Linux 6.4.0-rc1 and is based on
> arch/riscv/include/asm/mmio.h.
> 
> Addionally, to the header was added definions of ioremap_*().
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes in V4:
>  - delete inner parentheses in macros.
>  - s/u<N>/uint<N>.
> ---
> Changes in V3:
>  - re-sync with linux kernel
>  - update the commit message
> ---
> Changes in V2:
>  - Nothing changed. Only rebase.
> ---
>  xen/arch/riscv/include/asm/io.h | 142 ++++++++++++++++++++++++++++++++
>  1 file changed, 142 insertions(+)
>  create mode 100644 xen/arch/riscv/include/asm/io.h
> 
> diff --git a/xen/arch/riscv/include/asm/io.h b/xen/arch/riscv/include/asm/io.h
> new file mode 100644
> index 0000000000..1e61a40522
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/io.h
> @@ -0,0 +1,142 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * {read,write}{b,w,l,q} based on arch/arm64/include/asm/io.h
> + *   which was based on arch/arm/include/io.h
> + *
> + * Copyright (C) 1996-2000 Russell King
> + * Copyright (C) 2012 ARM Ltd.
> + * Copyright (C) 2014 Regents of the University of California
> + */
> +
> +
> +#ifndef _ASM_RISCV_IO_H
> +#define _ASM_RISCV_IO_H
> +
> +#include <asm/byteorder.h>
> +
> +/*
> + * The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't
> + * change the properties of memory regions.  This should be fixed by the
> + * upcoming platform spec.
> + */
> +#define ioremap_nocache(addr, size) ioremap(addr, size)
> +#define ioremap_wc(addr, size) ioremap(addr, size)
> +#define ioremap_wt(addr, size) ioremap(addr, size)
> +
> +/* Generic IO read/write.  These perform native-endian accesses. */
> +#define __raw_writeb __raw_writeb

What use are this and the similar other #define-s?

> +static inline void __raw_writeb(uint8_t val, volatile void __iomem *addr)
> +{
> +	asm volatile("sb %0, 0(%1)" : : "r" (val), "r" (addr));

Nit (throughout): Missing blanks. Or wait - is this file intended to
be Linux style? If so, it's just one blank that's missing.

> +/*
> + * Unordered I/O memory access primitives.  These are even more relaxed than
> + * the relaxed versions, as they don't even order accesses between successive
> + * operations to the I/O regions.
> + */
> +#define readb_cpu(c)		({ uint8_t  __r = __raw_readb(c); __r; })
> +#define readw_cpu(c)		({ uint16_t __r = le16_to_cpu((__force __le16)__raw_readw(c)); __r; })
> +#define readl_cpu(c)		({ uint32_t __r = le32_to_cpu((__force __le32)__raw_readl(c)); __r; })

Didn't we settle on the little-endian stuff to be dropped from here?
No matter what CPU endianness, what endianness a particular device
(and hence its MMIO region(s)) is using is entirely independent. Hence
conversion, where necessary, needs to occur at a layer up.

Also, what good do the __r variables do here? If they weren't here,
we also wouldn't need to discuss their naming.

> +#define writeb_cpu(v,c)		((void)__raw_writeb(v,c))
> +#define writew_cpu(v,c)		((void)__raw_writew((__force uint16_t)cpu_to_le16(v),c))
> +#define writel_cpu(v,c)		((void)__raw_writel((__force uint32_t)cpu_to_le32(v),c))

Nit: Blanks after commas please (also again further down).

> +#ifdef CONFIG_64BIT
> +#define readq_cpu(c)		({ u64 __r = le64_to_cpu((__force __le64)__raw_readq(c)); __r; })
> +#define writeq_cpu(v,c)		((void)__raw_writeq((__force u64)cpu_to_le64(v),c))

uint64_t (twice)

> +#endif
> +
> +/*
> + * I/O memory access primitives. Reads are ordered relative to any
> + * following Normal memory access. Writes are ordered relative to any prior
> + * Normal memory access.  The memory barriers here are necessary as RISC-V
> + * doesn't define any ordering between the memory space and the I/O space.
> + */
> +#define __io_br()	do {} while (0)

Nit: This and ...

> +#define __io_ar(v)	__asm__ __volatile__ ("fence i,r" : : : "memory");
> +#define __io_bw()	__asm__ __volatile__ ("fence w,o" : : : "memory");
> +#define __io_aw()	do { } while (0)

... this want to be spelled exactly the same.

Also, why does __io_ar() have a parameter (which it then doesn't use)?

Finally at least within a single file please be consistent about asm()
vs __asm__() use.

> +#define readb(c)	({ uint8_t  __v; __io_br(); __v = readb_cpu(c); __io_ar(__v); __v; })
> +#define readw(c)	({ uint16_t __v; __io_br(); __v = readw_cpu(c); __io_ar(__v); __v; })
> +#define readl(c)	({ uint32_t __v; __io_br(); __v = readl_cpu(c); __io_ar(__v); __v; })

Here the local variables are surely needed. Still they would preferably
not have any underscores as prefixes.

> +#define writeb(v,c)	({ __io_bw(); writeb_cpu(v,c); __io_aw(); })
> +#define writew(v,c)	({ __io_bw(); writew_cpu(v,c); __io_aw(); })
> +#define writel(v,c)	({ __io_bw(); writel_cpu(v,c); __io_aw(); })
> +
> +#ifdef CONFIG_64BIT
> +#define readq(c)	({ u64 __v; __io_br(); __v = readq_cpu(c); __io_ar(__v); __v; })

uint64_t again

> +#define writeq(v,c)	({ __io_bw(); writeq_cpu((v),(c)); __io_aw(); })

Inner parentheses still left?

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 11:13:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 11:13:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679846.1057568 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZqjW-0002fp-UL; Tue, 13 Feb 2024 11:13:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679846.1057568; Tue, 13 Feb 2024 11:13:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZqjW-0002fi-R1; Tue, 13 Feb 2024 11:13:50 +0000
Received: by outflank-mailman (input) for mailman id 679846;
 Tue, 13 Feb 2024 11:13:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8AlW=JW=linaro.org=jens.wiklander@srs-se1.protection.inumbo.net>)
 id 1rZqjV-0002fZ-R9
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 11:13:50 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f5120cfb-ca60-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 12:13:47 +0100 (CET)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-55a179f5fa1so5705961a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 03:13:47 -0800 (PST)
Received: from localhost.localdomain (h-217-31-164-171.A175.priv.bahnhof.se.
 [217.31.164.171]) by smtp.gmail.com with ESMTPSA id
 by18-20020a170906a2d200b00a3c8494b9c9sm1187640ejb.9.2024.02.13.03.13.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 03:13:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5120cfb-ca60-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1707822826; x=1708427626; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Ah2W+Jqw0gjGODmbPx4KrR3Ac+wyRbbfIMKbSN3TmiY=;
        b=AylmvOT9sRFbXq/8AtJh+bOcZACl/bo8symGdGEaHhNedpdtl8VvP8kaWszbGulvEf
         4yQpeQLEyPhcQBmhDZIMHNzlWZoIuTutNnG0NcCrHdaXUF/Dj0yeiCBJ6AmNaY5AIEED
         juFw986z+jzQbE/Cbt1cRdVhf8tcF5zer6J9SFTZ6ZMWIdyM0cYWnUCcKsQrfecnPw7y
         qAHjvwgcj9Y5HOnqvCtb5mac6Opm34ci3OxjNtj1pT8s5y3/4EQH0O6BbTfc28ODz2Fx
         48akDqUSvpHfe6YaZMU7QI2GwFvkwt/9mcc66U9igJX92UD8VuqfDX6hFINe6+7rzTIH
         cFCQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707822826; x=1708427626;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Ah2W+Jqw0gjGODmbPx4KrR3Ac+wyRbbfIMKbSN3TmiY=;
        b=Hu2H/bdmuiH4prWYkqntOWnMZqPG8zFl0yJ0+NpJ1e4XqjqHUzXCdhmqzGoIt7XtuM
         RwDx/q0QWY89WGkNzC5BPdhSLUUndlw4F8IkkXrdzcYaY0Xuyn+enbdjAlIYs8aws5cd
         cepHoz6fkXOxC2UbX6zf1XC/dvS2kYCC4FHt6AB3krThJc3byFiGHscdK1PCPDuhJ4kM
         GeUtvdmqd9P4II8qaxEuZji/9sY2+VAa4/1EztDivCaluAZdQ+8/XHc+qHj9+cr3dQkL
         P0o0zvTnWkFFZJ7lGpyd3Ys0QcW6hb7LzPKIA3j2335LlsBfnb1fvQR2UkQC8S/6XzMr
         xaew==
X-Gm-Message-State: AOJu0Yzvl0x/NFxy5/oa4HDNWFnAWxY51td/0tWx8Lukw5hHQHpqEyTN
	7+JGFf/oKYvbmA+ayVgo79uw8mdwFYzNZI1315POOcZS+d40H5UDFVyaGTtCAJbjPFq4Omposre
	b
X-Google-Smtp-Source: AGHT+IFFZptaPu4QRJPbnM0mWZMAJhROTmtogdtbhPiBM9zg9t+ljajdWElSymRRacZsnFPnZGYuPA==
X-Received: by 2002:a17:906:114b:b0:a3d:18e3:9c52 with SMTP id i11-20020a170906114b00b00a3d18e39c52mr321079eja.4.1707822826200;
        Tue, 13 Feb 2024 03:13:46 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUWldfdzYm8TNZxEH0eSrKa9Cy8suwSs0b7eRlxZ99FBZ1+FgNdKzIy/z1m24Inrp7R0XDj21xQ8cO8LsU4K2QHEzp0bC0XSFlu5VbhHBdIhry0r3OdZwrhzOgv1G5+EIFb5uQW9DR7escthAb67ygm5DXEE48PdHk7LmXIuF7D91vImu5hVKg254m4RXgRYgNnYrbSg5Xp56IWLc8jVbzT4ZVHVxaiEp6Wsr1Vcsoeh9EqJ4mz6chnXtJ+MR4OltRPgYIPJQMYcrmwgnYNcLfH/4EC+GQMoiI=
From: Jens Wiklander <jens.wiklander@linaro.org>
To: xen-devel@lists.xenproject.org
Cc: patches@linaro.org,
	Jens Wiklander <jens.wiklander@linaro.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [XEN PATCH v5] xen/arm: ffa: reclaim shared memory on guest destroy
Date: Tue, 13 Feb 2024 12:13:36 +0100
Message-Id: <20240213111336.881934-1-jens.wiklander@linaro.org>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

When an FF-A enabled guest is destroyed it may leave behind memory
shared with SPs. This memory must be reclaimed before it's reused or an
SP may make changes to memory used by a new unrelated guest. So when the
domain is teared down add FF-A requests to reclaim all remaining shared
memory.

SPs in the secure world are notified using VM_DESTROYED that a guest has
been destroyed. An SP is supposed to relinquish all shared memory to allow
reclaiming the memory. The relinquish operation may need to be delayed if
the shared memory is for instance part of a DMA operation.

The domain reference counter is increased when the first FF-A shared
memory is registered and the counter is decreased again when the last
shared memory is reclaimed. If FF-A shared memory registrations remain
at the end of of ffa_domain_teardown() a timer is set to try to reclaim
the shared memory every second until the memory is reclaimed.

A few minor style fixes with a removed empty line here and an added new
line there.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
v5:
- Move the put_domain() call after the spinlock section in dec_ctx_shm_count()
- Move the get_knownalive_domain() call into the else statement in
  inc_ctx_shm_count() to make sure it's only performed if ctx->shm_count
  is to be increased

v4:
- Retry FFA_MSG_SEND_VM_DESTROYED on the returned errors
  FFA_RET_INTERRUPTED and FFA_RET_RETRY even after all shared memory
  handles has been reclaimed.
- Retry ffa_mem_reclaim() only on temporary returned errors, permanent
  errors like FFA_RET_INVALID_PARAMETERS is dealt with in recovery mode
  instead since the SPMC isn't expected to use that error under normal
  circumstances.

v3:
- Mentioning in the commit message that there are some style fixes
- Addressing review comments
- Refactor the ffa_domain_teardown() path to let
  ffa_domain_teardown_continue() do most of the work.

v2:
- Update commit message to match the new implementation
- Using a per domain bitfield to keep track of which SPs has been notified
  with VM_DESTROYED
- Holding a domain reference counter to keep the domain as a zombie domain
  while there still is shared memory registrations remaining to be reclaimed
- Using a timer to retry reclaiming remaining shared memory registrations
---
 xen/arch/arm/tee/ffa.c | 295 ++++++++++++++++++++++++++++++++++-------
 1 file changed, 245 insertions(+), 50 deletions(-)

diff --git a/xen/arch/arm/tee/ffa.c b/xen/arch/arm/tee/ffa.c
index 0793c1c7585d..9a05dcede17a 100644
--- a/xen/arch/arm/tee/ffa.c
+++ b/xen/arch/arm/tee/ffa.c
@@ -54,6 +54,7 @@
 #include <xen/mm.h>
 #include <xen/sched.h>
 #include <xen/sizes.h>
+#include <xen/timer.h>
 #include <xen/types.h>
 
 #include <asm/event.h>
@@ -144,6 +145,12 @@
  */
 #define FFA_MAX_SHM_COUNT               32
 
+/*
+ * The time we wait until trying to tear down a domain again if it was
+ * blocked initially.
+ */
+#define FFA_CTX_TEARDOWN_DELAY          SECONDS(1)
+
 /* FF-A-1.1-REL0 section 10.9.2 Memory region handle, page 167 */
 #define FFA_HANDLE_HYP_FLAG             BIT(63, ULL)
 #define FFA_HANDLE_INVALID              0xffffffffffffffffULL
@@ -384,11 +391,6 @@ struct ffa_ctx {
     unsigned int page_count;
     /* FF-A version used by the guest */
     uint32_t guest_vers;
-    /*
-     * Number of SPs that we have sent a VM created signal to, used in
-     * ffa_domain_teardown() to know which SPs need to be signalled.
-     */
-    uint16_t create_signal_count;
     bool rx_is_free;
     /* Used shared memory objects, struct ffa_shm_mem */
     struct list_head shm_list;
@@ -402,6 +404,15 @@ struct ffa_ctx {
     spinlock_t tx_lock;
     spinlock_t rx_lock;
     spinlock_t lock;
+    /* Used if domain can't be torn down immediately */
+    struct domain *teardown_d;
+    struct list_head teardown_list;
+    s_time_t teardown_expire;
+    /*
+     * Used for ffa_domain_teardown() to keep track of which SPs should be
+     * notified that this guest is being destroyed.
+     */
+    unsigned long vm_destroy_bitmap[];
 };
 
 struct ffa_shm_mem {
@@ -436,6 +447,12 @@ static void *ffa_tx __read_mostly;
 static DEFINE_SPINLOCK(ffa_rx_buffer_lock);
 static DEFINE_SPINLOCK(ffa_tx_buffer_lock);
 
+
+/* Used to track domains that could not be torn down immediately. */
+static struct timer ffa_teardown_timer;
+static struct list_head ffa_teardown_head;
+static DEFINE_SPINLOCK(ffa_teardown_lock);
+
 static bool ffa_get_version(uint32_t *vers)
 {
     const struct arm_smccc_1_2_regs arg = {
@@ -853,7 +870,6 @@ static int32_t handle_partition_info_get(uint32_t w1, uint32_t w2, uint32_t w3,
             goto out_rx_release;
         }
 
-
         memcpy(ctx->rx, ffa_rx, sz);
     }
     ctx->rx_is_free = false;
@@ -992,53 +1008,83 @@ static void put_shm_pages(struct ffa_shm_mem *shm)
     }
 }
 
-static bool inc_ctx_shm_count(struct ffa_ctx *ctx)
+static bool inc_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
 {
     bool ret = true;
 
     spin_lock(&ctx->lock);
-    if (ctx->shm_count >= FFA_MAX_SHM_COUNT)
+
+    if ( ctx->shm_count >= FFA_MAX_SHM_COUNT )
+    {
         ret = false;
+    }
     else
+    {
+        /*
+         * If this is the first shm added, increase the domain reference
+         * counter as we need to keep domain around a bit longer to reclaim
+         * the shared memory in the teardown path.
+         */
+        if ( !ctx->shm_count )
+            get_knownalive_domain(d);
+
         ctx->shm_count++;
+    }
+
     spin_unlock(&ctx->lock);
 
     return ret;
 }
 
-static void dec_ctx_shm_count(struct ffa_ctx *ctx)
+static void dec_ctx_shm_count(struct domain *d, struct ffa_ctx *ctx)
 {
+    bool drop_ref;
+
     spin_lock(&ctx->lock);
+
     ASSERT(ctx->shm_count > 0);
     ctx->shm_count--;
+
+    /*
+     * If this was the last shm removed, let go of the domain reference we
+     * took in inc_ctx_shm_count() above.
+     */
+    drop_ref = !ctx->shm_count;
+
     spin_unlock(&ctx->lock);
+
+    if ( drop_ref )
+        put_domain(d);
 }
 
-static struct ffa_shm_mem *alloc_ffa_shm_mem(struct ffa_ctx *ctx,
+static struct ffa_shm_mem *alloc_ffa_shm_mem(struct domain *d,
                                              unsigned int page_count)
 {
+    struct ffa_ctx *ctx = d->arch.tee;
     struct ffa_shm_mem *shm;
 
     if ( page_count >= FFA_MAX_SHM_PAGE_COUNT )
         return NULL;
-    if ( !inc_ctx_shm_count(ctx) )
+    if ( !inc_ctx_shm_count(d, ctx) )
         return NULL;
 
     shm = xzalloc_flex_struct(struct ffa_shm_mem, pages, page_count);
     if ( shm )
         shm->page_count = page_count;
     else
-        dec_ctx_shm_count(ctx);
+        dec_ctx_shm_count(d, ctx);
 
     return shm;
 }
 
-static void free_ffa_shm_mem(struct ffa_ctx *ctx, struct ffa_shm_mem *shm)
+static void free_ffa_shm_mem(struct domain *d, struct ffa_shm_mem *shm)
 {
+    struct ffa_ctx *ctx = d->arch.tee;
+
     if ( !shm )
         return;
 
-    dec_ctx_shm_count(ctx);
+    dec_ctx_shm_count(d, ctx);
     put_shm_pages(shm);
     xfree(shm);
 }
@@ -1306,7 +1352,7 @@ static void handle_mem_share(struct cpu_user_regs *regs)
         goto out_unlock;
     }
 
-    shm = alloc_ffa_shm_mem(ctx, page_count);
+    shm = alloc_ffa_shm_mem(d, page_count);
     if ( !shm )
     {
         ret = FFA_RET_NO_MEMORY;
@@ -1350,7 +1396,7 @@ static void handle_mem_share(struct cpu_user_regs *regs)
 
 out:
     if ( ret )
-        free_ffa_shm_mem(ctx, shm);
+        free_ffa_shm_mem(d, shm);
 out_unlock:
     spin_unlock(&ctx->tx_lock);
 
@@ -1401,7 +1447,7 @@ static int handle_mem_reclaim(uint64_t handle, uint32_t flags)
     }
     else
     {
-        free_ffa_shm_mem(ctx, shm);
+        free_ffa_shm_mem(d, shm);
     }
 
     return ret;
@@ -1486,6 +1532,41 @@ static bool ffa_handle_call(struct cpu_user_regs *regs)
     }
 }
 
+static bool is_in_subscr_list(const uint16_t *subscr, uint16_t start,
+                              uint16_t end, uint16_t sp_id)
+{
+    unsigned int n;
+
+    for ( n = start; n < end; n++ )
+    {
+        if ( subscr[n] == sp_id )
+            return true;
+    }
+
+    return false;
+}
+
+static void vm_destroy_bitmap_init(struct ffa_ctx *ctx,
+                                   unsigned int create_signal_count)
+{
+    unsigned int n;
+
+    for ( n = 0; n < subscr_vm_destroyed_count; n++ )
+    {
+        /*
+         * Skip SPs subscribed to the VM created event that never was
+         * notified of the VM creation due to an error during
+         * ffa_domain_init().
+         */
+        if ( is_in_subscr_list(subscr_vm_created, create_signal_count,
+                               subscr_vm_created_count,
+                               subscr_vm_destroyed[n]) )
+            continue;
+
+        set_bit(n, ctx->vm_destroy_bitmap);
+    }
+}
+
 static int ffa_domain_init(struct domain *d)
 {
     struct ffa_ctx *ctx;
@@ -1501,11 +1582,14 @@ static int ffa_domain_init(struct domain *d)
     if ( d->domain_id >= UINT16_MAX)
         return -ERANGE;
 
-    ctx = xzalloc(struct ffa_ctx);
+    ctx = xzalloc_flex_struct(struct ffa_ctx, vm_destroy_bitmap,
+                              BITS_TO_LONGS(subscr_vm_destroyed_count));
     if ( !ctx )
         return -ENOMEM;
 
     d->arch.tee = ctx;
+    ctx->teardown_d = d;
+    INIT_LIST_HEAD(&ctx->shm_list);
 
     for ( n = 0; n < subscr_vm_created_count; n++ )
     {
@@ -1515,65 +1599,173 @@ static int ffa_domain_init(struct domain *d)
         {
             printk(XENLOG_ERR "ffa: Failed to report creation of vm_id %u to  %u: res %d\n",
                    get_vm_id(d), subscr_vm_created[n], res);
-            ctx->create_signal_count = n;
-            return -EIO;
+            break;
         }
     }
-    ctx->create_signal_count = subscr_vm_created_count;
-
-    INIT_LIST_HEAD(&ctx->shm_list);
+    vm_destroy_bitmap_init(ctx, n);
+    if ( n != subscr_vm_created_count )
+        return -EIO;
 
     return 0;
 }
 
-static bool is_in_subscr_list(const uint16_t *subscr, uint16_t start,
-                              uint16_t end, uint16_t sp_id)
+static void send_vm_destroyed(struct domain *d)
 {
+    struct ffa_ctx *ctx = d->arch.tee;
     unsigned int n;
+    int32_t res;
 
-    for ( n = start; n < end; n++ )
+    for ( n = 0; n < subscr_vm_destroyed_count; n++ )
     {
-        if ( subscr[n] == sp_id )
-            return true;
-    }
+        if ( !test_bit(n, ctx->vm_destroy_bitmap) )
+            continue;
 
-    return false;
+        res = ffa_direct_req_send_vm(subscr_vm_destroyed[n], get_vm_id(d),
+                                     FFA_MSG_SEND_VM_DESTROYED);
+
+        if ( res )
+        {
+            printk(XENLOG_ERR "%pd: ffa: Failed to report destruction of vm_id %u to %u: res %d\n",
+                   d, get_vm_id(d), subscr_vm_destroyed[n], res);
+        }
+
+        /*
+         * For these two error codes the hypervisor is expected to resend
+         * the destruction message. For the rest it is expected that the
+         * error is permanent and that is doesn't help to resend the
+         * destruction message.
+         */
+        if ( res != FFA_RET_INTERRUPTED && res != FFA_RET_RETRY )
+            clear_bit(n, ctx->vm_destroy_bitmap);
+    }
 }
 
-/* This function is supposed to undo what ffa_domain_init() has done */
-static int ffa_domain_teardown(struct domain *d)
+static void reclaim_shms(struct domain *d)
 {
     struct ffa_ctx *ctx = d->arch.tee;
-    unsigned int n;
+    struct ffa_shm_mem *shm, *tmp;
     int32_t res;
 
-    if ( !ctx )
-        return 0;
+    list_for_each_entry_safe(shm, tmp, &ctx->shm_list, list)
+    {
+        register_t handle_hi;
+        register_t handle_lo;
+
+        uint64_to_regpair(&handle_hi, &handle_lo, shm->handle);
+        res = ffa_mem_reclaim(handle_lo, handle_hi, 0);
+        switch ( res ) {
+        case FFA_RET_OK:
+            printk(XENLOG_G_DEBUG "%pd: ffa: Reclaimed handle %#lx\n",
+                   d, shm->handle);
+            list_del(&shm->list);
+            free_ffa_shm_mem(d, shm);
+            break;
+        case FFA_RET_DENIED:
+            /*
+             * A temporary error that may get resolved a bit later, it's
+             * worth retrying.
+             */
+            printk(XENLOG_G_INFO "%pd: ffa: Failed to reclaim handle %#lx : %d\n",
+                   d, shm->handle, res);
+            break; /* We will retry later */
+        default:
+            /*
+             * The rest of the error codes are not expected and are assumed
+             * to be of a permanent nature. It not in our control to handle
+             * the error properly so the object in this case is to try to
+             * minimize the damage.
+             *
+             * FFA_RET_NO_MEMORY might be a temporary error as it it could
+             * succeed if retried later, but treat it as permanent for now.
+             */
+            printk(XENLOG_G_INFO "%pd: ffa: Permanent failure to reclaim handle %#lx : %d\n",
+                   d, shm->handle, res);
 
-    for ( n = 0; n < subscr_vm_destroyed_count; n++ )
+            /*
+             * Remove the shm from the list and free it, but don't drop
+             * references. This results in having the shared physical pages
+             * permanently allocate and also keeps the domain as a zombie
+             * domain.
+             */
+            list_del(&shm->list);
+            xfree(shm);
+            break;
+        }
+    }
+}
+
+static void ffa_domain_teardown_continue(struct ffa_ctx *ctx, bool first_time)
+{
+    struct ffa_ctx *next_ctx = NULL;
+
+    send_vm_destroyed(ctx->teardown_d);
+    reclaim_shms(ctx->teardown_d);
+
+    if ( ctx->shm_count ||
+         !bitmap_empty(ctx->vm_destroy_bitmap, subscr_vm_destroyed_count) )
+    {
+        printk(XENLOG_G_INFO "%pd: ffa: Remaining cleanup, retrying\n", ctx->teardown_d);
+
+        ctx->teardown_expire = NOW() + FFA_CTX_TEARDOWN_DELAY;
+
+        spin_lock(&ffa_teardown_lock);
+        list_add_tail(&ctx->teardown_list, &ffa_teardown_head);
+        /* Need to set a new timer for the next ctx in line */
+        next_ctx = list_first_entry(&ffa_teardown_head, struct ffa_ctx,
+                                    teardown_list);
+        spin_unlock(&ffa_teardown_lock);
+    }
+    else
     {
         /*
-         * Skip SPs subscribed to the VM created event that never was
-         * notified of the VM creation due to an error during
-         * ffa_domain_init().
+         * domain_destroy() might have been called (via put_domain() in
+         * reclaim_shms()), so we can't touch the domain structure anymore.
          */
-        if ( is_in_subscr_list(subscr_vm_created, ctx->create_signal_count,
-                               subscr_vm_created_count,
-                               subscr_vm_destroyed[n]) )
-            continue;
-
-        res = ffa_direct_req_send_vm(subscr_vm_destroyed[n], get_vm_id(d),
-                                     FFA_MSG_SEND_VM_DESTROYED);
+        xfree(ctx);
 
-        if ( res )
-            printk(XENLOG_ERR "ffa: Failed to report destruction of vm_id %u to  %u: res %d\n",
-                   get_vm_id(d), subscr_vm_destroyed[n], res);
+        /* Only check if there has been a change to the teardown queue */
+        if ( !first_time )
+        {
+            spin_lock(&ffa_teardown_lock);
+            next_ctx = list_first_entry_or_null(&ffa_teardown_head,
+                                                struct ffa_ctx, teardown_list);
+            spin_unlock(&ffa_teardown_lock);
+        }
     }
 
+    if ( next_ctx )
+        set_timer(&ffa_teardown_timer, next_ctx->teardown_expire);
+}
+
+static void ffa_teardown_timer_callback(void *arg)
+{
+    struct ffa_ctx *ctx;
+
+    spin_lock(&ffa_teardown_lock);
+    ctx = list_first_entry_or_null(&ffa_teardown_head, struct ffa_ctx,
+                                   teardown_list);
+    if ( ctx )
+        list_del(&ctx->teardown_list);
+    spin_unlock(&ffa_teardown_lock);
+
+    if ( ctx )
+        ffa_domain_teardown_continue(ctx, false /* !first_time */);
+    else
+        printk(XENLOG_G_ERR "%s: teardown list is empty\n", __func__);
+}
+
+/* This function is supposed to undo what ffa_domain_init() has done */
+static int ffa_domain_teardown(struct domain *d)
+{
+    struct ffa_ctx *ctx = d->arch.tee;
+
+    if ( !ctx )
+        return 0;
+
     if ( ctx->rx )
         rxtx_unmap(ctx);
 
-    XFREE(d->arch.tee);
+    ffa_domain_teardown_continue(ctx, true /* first_time */);
 
     return 0;
 }
@@ -1739,6 +1931,9 @@ static bool ffa_probe(void)
     if ( !init_sps() )
         goto err_free_ffa_tx;
 
+    INIT_LIST_HEAD(&ffa_teardown_head);
+    init_timer(&ffa_teardown_timer, ffa_teardown_timer_callback, NULL, 0);
+
     return true;
 
 err_free_ffa_tx:
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 13 11:36:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 11:36:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679852.1057579 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZr4o-0005x6-Nd; Tue, 13 Feb 2024 11:35:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679852.1057579; Tue, 13 Feb 2024 11:35:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZr4o-0005wy-Jj; Tue, 13 Feb 2024 11:35:50 +0000
Received: by outflank-mailman (input) for mailman id 679852;
 Tue, 13 Feb 2024 11:35:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gnrE=JW=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZr4n-0005wr-Ev
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 11:35:49 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 07f686aa-ca64-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 12:35:47 +0100 (CET)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-561f8b8c058so732430a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 03:35:47 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 st10-20020a170907c08a00b00a3c5e6515d0sm1202894ejc.24.2024.02.13.03.35.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 03:35:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 07f686aa-ca64-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707824147; x=1708428947; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=UflNTsRNumGC6EKCakxsXONmRQPRfVLwHRKxdQ7Uasw=;
        b=LhPByerq4w9KIHi4FBH2UjLEO2W2ebqjSlgBYMMrqJy8U0Io9CF8ETIHz8F4S79W84
         kdhrchjNwMA2ACbsChEiH5nrtRulQefDo66SSqT4RTtO1I0kQsUZ7kbru6azbPqeXySn
         Mny+0qArMoN+ReT9IUTYblLmCze2sQ1yXxqiw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707824147; x=1708428947;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=UflNTsRNumGC6EKCakxsXONmRQPRfVLwHRKxdQ7Uasw=;
        b=Qg7dpao1DzuHpnHUgV2LOct2FdikhrZJ8bc+9TeN5o7RoIwte7JN1bW+E2D2mPurkM
         Ih3Jk4JOmmlU19KQEgyuaALWgE3m+Ft+Vh+9pZJJYD+SIPShnigFZ03a0esPE5cYlq7N
         Z5l0oPSpAZ0J0gxIWr6JuOVzxXcwZXvIKbWobcmP6C9vT0LteGh2WlOEH/phu6HPgdbf
         SHL2Fk3tR3lZP85QFOv5sPKmp7EGPw5mFW6l6zQghalPB5w390g6hnteZ1Rhsev2pby6
         WwVDFhqRa7qSuGVWqpMFwpuFo8hTKKucdyoOPl/tG+mhsSAnFr1VHD4i2K9XWaaw5pvw
         cvow==
X-Gm-Message-State: AOJu0YwUF6z915EFpIJrv+6EGnzF/vI7zOCKb2Z+yLBVR+inBTphBbwc
	zevOllIcy4oMP/n+ZvDdA8O8NwRjYj13+xKKzrSKhxEgBvTMDc99cky6QpqYpRk=
X-Google-Smtp-Source: AGHT+IFyCtpkFk8rlXxzlthUKOlmGfwl/2FIK71k6UD/iRyd7h7yu0CVCqNe7aiSJZIBbHnk3VgAhg==
X-Received: by 2002:a17:906:eb57:b0:a3d:20eb:1ee8 with SMTP id mc23-20020a170906eb5700b00a3d20eb1ee8mr122655ejb.2.1707824146907;
        Tue, 13 Feb 2024 03:35:46 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW3o1s2Mw2xAHUVK1yPKXDz/wb3yK0/qKKBdbwYakl85/erl6/nbep4p25Z3l9fS4f6U3v3EUGrw0HMtZWum7yWtVjYFRbRiYLl+PTAvRbOAMGDSkLIQLx0b3GHHPTzDD1WTe0JPd4lfOeK8klB/3/IDC41qkayH8eM1I+N9BbnjHcyCgnmcbhKgWwPEQJHzsSNEbkzBSopSvgEW1le4Z27YE4+BUrDBYku0h+UXudIP3re1TEBw+3qwU/y6R1d4fw3IKeLERvo2/KHoIoaCcBc0JYi/VBoEn/u1TMG
Date: Tue, 13 Feb 2024 11:35:46 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, marmarek@invisiblethingslab.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2 3/3] libxl: Enable stubdom cdrom changing
Message-ID: <ff5f5321-5cb2-4a39-9a68-d5465098a7ff@perard>
References: <20240201214004.238858-1-jandryuk@gmail.com>
 <20240201214004.238858-4-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240201214004.238858-4-jandryuk@gmail.com>

On Thu, Feb 01, 2024 at 04:40:04PM -0500, Jason Andryuk wrote:
> To change the cd-rom medium, libxl will:
>  - QMP eject the medium from QEMU
>  - block-detach the old PV disk
>  - block-attach the new PV disk
>  - QMP change the medium to the new PV disk by fdset-id
> 
> The QMP code is reused, and remove and attach are implemented here.
> 
> The stubdom must internally handle adding /dev/xvdc to the appropriate
> fdset.  libxl in dom0 doesn't see the result of adding to the fdset as
> that is internal to the stubdom, so a delay and retries are added to
> around calling cdrom_insert_addfd_cb().

Why do you use "fdset" in linux stubdom, can't QEMU open /dev/xvdc
directly? And so why don't libxl tell QEMU to open /dev/xvdc?

I think the only reason I've implemented cd-insert via fdset in libxl
was because that was the only way to have libxl open a file on behalf of
QEMU when it's deprivileged and don't have access to any file. But that
doesn't mean we need to use an fdset with the stubdom.


And BTW, there's a "query-fdsets" command that could be used to findout
when and fdset is available, there's even an "opaque" field that could
be used to communicate which fdset to use instead of needing to rely on
a static fdset-id.

> For cd-eject, we still need to attach the empty vbd.  This is necessary
> since xenstore is used to determine that hdc exists.  Otherwise after
> eject, hdc would be gone and the cd-insert would fail to find the drive
> to insert new media.
> 
> The ERROR_JSON_CONFIG_EMPTY check in cdrom_insert_inserted() is because
> a stubdom don't have a json config.
> 
> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
> ---
> diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
> index 09082ffb58..6354982c05 100644
> --- a/tools/libs/light/libxl_disk.c
> +++ b/tools/libs/light/libxl_disk.c
> @@ -905,6 +938,18 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
>          disk->format = LIBXL_DISK_FORMAT_EMPTY;
>      }
>  
> +#define STUBDOM_FDSET_CD 8000
> +    if (strncmp(disk->vdev, "hd", 2) == 0) {
> +        cis->stubdom_fdset = STUBDOM_FDSET_CD + disk->vdev[2] - 'a';
> +    } else if (strncmp(disk->vdev, "xvd", 3) == 0) {
> +        cis->stubdom_fdset = STUBDOM_FDSET_CD + disk->vdev[3] - 'a';

Currently, cd-rom eject/insert also works if one use "sdc", so we
probably want to check for "sd".

Also, you could use device_virtdisk_matches() like
libxl__device_disk_dev_number(). I'm not sure if having vdev been
something like "xvdaa" is going to work well, and that case isn't going
to produce the expected fdset. (Without the patch, "xvdaa" produce a
warning when trying to generate QEMU's command line, so maybe it's fine
to not handle this case.)

> +    } else {
> +        LOGD(ERROR, cis->domid, "disk->vdev \"%s\" isn't hdX or xvdY",
> +             disk->vdev);
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
>  out:
>      libxl__device_list_free(&libxl__disk_devtype, disks, num);
>      if (rc) {
> @@ -923,6 +968,7 @@ static void cdrom_insert_lock_acquired(libxl__egc *egc,
>      libxl__cdrom_insert_state *cis = CONTAINER_OF(lock, *cis, qmp_lock);
>      STATE_AO_GC(cis->ao);
>  
> +    LOGD(DEBUG, cis->domid, "rc=%d", rc);

Left over debug logging? The new one in cdrom_insert_done() might be
enough? Also, I think libxl__ao_complete() does log this rc value as
well.

>      if (rc) goto out;
>  
>      rc = libxl__ev_time_register_rel(ao, &cis->time,


> +static void cdrom_insert_stubdom_ejected(libxl__egc *egc, libxl__ev_qmp *qmp,
> +                                         const libxl__json_object *response,
> +                                         int rc)
> +{
> +    libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp);
> +    libxl__device *device;
> +    STATE_AO_GC(cis->ao);
> +    domid_t stubdomid = libxl_get_stubdom_id(CTX, cis->domid);

cis->disk_domid?

> +
> +    LOGD(DEBUG, cis->domid, "rc=%d", rc);
> +    /* cis->stubdom_fdset is initially empty, so remove-fd fails the first
> +     * call with:
> +     * {"error": {"class": "GenericError",
> +     *            "desc": "File descriptor named 'fdset-id:8675' not found"}}
> +     * Carry on in that case. */
> +    if (rc && rc != ERROR_QMP_GENERIC_ERROR) goto out;
> +
> +    GCNEW(device);
> +    rc = libxl__device_from_disk(gc, stubdomid, cis->disk, device);
> +    if (rc != 0) goto out;

If (rc) is more common without "!= 0".


> +
> +    /* block dev eject */
> +    /* Below is basically an open coding of:
> +     * libxl_device_disk__remove(CTX, cis->domid, cis->disk, 0);

One too many '_'?

> +     * ...since we can't call it from within libxl.

Well, not exactly, we also need to set our own callback and own struct
here.

> +     */
> +    libxl__prepare_ao_device(ao, &cis->aodev_del);
> +    cis->aodev_del.action = LIBXL__DEVICE_ACTION_REMOVE;
> +    cis->aodev_del.dev = device;
> +    cis->aodev_del.callback = cdrom_insert_stubdom_disk_ejected_aocomplete;
> +    cis->aodev_del.force.flag = LIBXL__FORCE_OFF;
> +    libxl__initiate_device_generic_remove(egc, &cis->aodev_del);

It doesn't looks like you need an extra `aodev_del` field, and you can
probably reuse the same `aodev` field twice.

> +    return;
> +
> + out:
> +    cdrom_insert_done(egc, cis, rc); /* must be last */
> +}
> +
> +static void cdrom_insert_stubdom_disk_ejected_aocomplete(libxl__egc *egc,
> +                                                         libxl__ao_device *aodev)
> +{
> +    STATE_AO_GC(aodev->ao);
> +    libxl__cdrom_insert_state *cis = CONTAINER_OF(aodev, *cis, aodev_del);
> +
> +    LOGD(DEBUG, cis->domid, "rc=%d", aodev->rc);
> +    if (aodev->rc) {
> +        if (aodev->dev) {

I'm pretty sure "aodev->dev" should be set, so this if doesn't seems
needed.

> +            LOGD(ERROR, aodev->dev->domid, "Unable to %s %s with id %u",
> +                        libxl__device_action_to_string(aodev->action),

We already know the action is "remove".

> +                        libxl__device_kind_to_string(aodev->dev->kind),
> +                        aodev->dev->devid);

This whole message could be personalised to what we are trying to
achieve, that is, remove the cd-rom from the stubdomain.

> +        } else {
> +            LOG(ERROR, "unable to %s device",
> +                       libxl__device_action_to_string(aodev->action));
> +        }
> +        goto out;
> +    }
> +
> +    cdrom_insert_stubdom_disk_ejected(egc, &cis->qmp, NULL, aodev->rc);
> +    return;
> +
> + out:
> +    cdrom_insert_done(egc, cis, aodev->rc);
> +}
> +
> +static void cdrom_insert_stubdom_disk_ejected(libxl__egc *egc,
> +                                              libxl__ev_qmp *qmp,
> +                                              const libxl__json_object *response,
> +                                              int rc)

This functions prototype looks like a ev_qmp callback but it's never
used like that. Also, it could be merged into the previous function.


>  static void cdrom_insert_inserted(libxl__egc *egc,
>                                    libxl__ev_qmp *qmp,
>                                    const libxl__json_object *response,
>                                    int rc)
>  {
> -    EGC_GC;
>      libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp);
> +    STATE_AO_GC(cis->ao);

Why do you replace EGC_GC by AO_GC ? Is there an allocation that need to
last longer in the current function? If it's just to have "ao" in the
current scope, you can use "cis->ao" below.

>      libxl__flock *data_lock = NULL;
>      libxl_domain_config d_config;
>      flexarray_t *insert = NULL;
> @@ -1171,9 +1401,22 @@ static void cdrom_insert_inserted(libxl__egc *egc,
>  
>      libxl_domain_config_init(&d_config);
>  
> -    if (rc) goto out;
> +    LOGD(DEBUG, cis->domid, "rc=%d", rc);
> +
> +    if (rc) {
> +        if (cis->retries++ < 10 ) {
> +            LOGD(DEBUG, qmp->domid, "Retrying QMP cdrom change\n");
> +            rc = libxl__ev_time_register_rel(ao, &cis->timeout_retry,
> +                                             cdrom_insert_addfd_retry, 100);
> +            if (rc) goto out;
>  
> -    rc = libxl__device_from_disk(gc, domid, disk, &device);
> +            return;
> +        } else {
> +            goto out;
> +        }
> +    }
> +
> +    rc = libxl__device_from_disk(gc, cis->disk_domid, disk, &device);
>      if (rc) goto out;
>      be_path = libxl__device_backend_path(gc, &device);
>      libxl_path = libxl__device_libxl_path(gc, &device);
> @@ -1257,7 +1500,10 @@ static void cdrom_insert_done(libxl__egc *egc,
>  {
>      EGC_GC;
>  
> +    LOGD(DEBUG, cis->domid, "rc=%d", rc);

I don't think this is useful because libxl__ao_complete() will print rc
as well.

>      libxl__ev_time_deregister(gc, &cis->time);
> +    libxl__ev_time_deregister(gc, &cis->timeout_retry);
>      libxl__ev_qmp_dispose(gc, &cis->qmp);
>      if (cis->qmp.payload_fd >= 0) close(cis->qmp.payload_fd);
>      libxl__ev_slowlock_unlock(gc, &cis->qmp_lock);

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 11:37:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 11:37:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679855.1057588 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZr5y-0006Vw-5M; Tue, 13 Feb 2024 11:37:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679855.1057588; Tue, 13 Feb 2024 11:37:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZr5y-0006Vo-2K; Tue, 13 Feb 2024 11:37:02 +0000
Received: by outflank-mailman (input) for mailman id 679855;
 Tue, 13 Feb 2024 11:37:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZr5w-0005wr-O6
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 11:37:00 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 32e71da8-ca64-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 12:36:59 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-410ca9e851bso14953925e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 03:36:59 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 j8-20020adfe508000000b0033aedaea1b2sm9357452wrm.30.2024.02.13.03.36.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 03:36:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 32e71da8-ca64-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707824219; x=1708429019; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Z8HVVCKdR2JQf6NX1jGQ1/KKxoyowqEyolTxOAblmNk=;
        b=EFdk5SHSdLeY4T8F8pjwINywM3Fq77AaDJ7xNXsRMYkdUt3tBPr/nDBSI+MzLPL+Uw
         Co7plS4WSHUzYWS/FeuYXLJpz8rHJ7nSOfrA7SIJR8+WLBdTSkGhXEdIFrrtZCCedPUo
         z8y2McT97aGtRQeSFVoKAyBitDTzI/vtHqp2roI4mLr7U9FoRs61jyabMNVGV3o+Mis6
         FS5pDL0lW607bGftxj8qs6ru8j0Ma/0a0RSeKCA9vkNwbefEhqk4o+avcYe7DHCEMHQk
         O24INN3Ak/I6AviJNM0jSC2evtNOLMNUICqCGSmuaKeTD/WGUqOgf1A7ZFZA80erM1gU
         jq+Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707824219; x=1708429019;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Z8HVVCKdR2JQf6NX1jGQ1/KKxoyowqEyolTxOAblmNk=;
        b=k4zPBmwmWeqigsFLauxfGot+DERfbetSaez4ftTRSDp5B3icYIIuF1t1rEPnc2zIbO
         hiLIuryY8tcQj3kCo7/vUn9bx7jUf0ov3fITGn9reOxP6QfNeMEuwoTMZPkni+x444F1
         o0EWEJKTbM0maKzMb+Dd/5i1Jmc+eyLyovp7YNC7wrbIYkNqhpOjpwovXxh4glQEXnUz
         Gc3sCh0rPTNcDvtRHuXPVjj1gXuxdvXH5MN7F1UbnfWVJF1TIGCVE/jPoqrfoPmbnVrE
         d98YHFVIIIAiQUsduPSqVtR34xg6DS94ZA3HBqdREwbFoaiBRvkuew00oTrHybrY/h8E
         ZDGQ==
X-Forwarded-Encrypted: i=1; AJvYcCWnd0A86nl6z7iqj9iAF2u25gGmOLAOcwd4R+zjIOanTS9pYzvQ0pmB/5FBoKaajt2sIcwg3wunaMyH4qjKUE9LFETH6ukwmPErcmdrHXo=
X-Gm-Message-State: AOJu0YxU2rb9EgMSjYmksa6iA0RbgLLmfDMYgbSEa5zG20P9Wkwx7XgK
	Vi8yCBrgCcZX/ePM7sEucxmgH+3DbX/4Qui78SMlQmvh84JISz7vPPqZLOXGzA==
X-Google-Smtp-Source: AGHT+IEIhXSxrrgCus0ENH/r51WtOjRNxoISbc+g0Bv+dnXpqSJvu+KEDWUzn5JH3T41RG+guh020Q==
X-Received: by 2002:a05:600c:1d07:b0:410:cf4f:68e1 with SMTP id l7-20020a05600c1d0700b00410cf4f68e1mr4387021wms.2.1707824219045;
        Tue, 13 Feb 2024 03:36:59 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUsRC6LTP5gVZDasTnIopEXjnB4sjt/oeJvEJ/4Yj55HsOmZkAro/1XFhO3AI967qv/RJlZUgDmkLnIShacMVWG29zJ2ujQI6owvTNYEZMuKk3SEfaJbf5HeQAqqct5KbNcBrrp5tyArfcDt0XyYJvhueCwid2ubeQNa9LOCjdsnD2ZG7/t1FA5LML+Kne72MFw4/EhJil14munIW7CZalpleJo7xozM7xmYJ6DcpIUuES3LNOP77u3w/1n1Z210XYcvOeHEcJCJQuWPyyeH7byKx1DOoZtOEAMQYiNi8W1rleQy3TGyGww
Message-ID: <134e09c9-776a-4261-a28b-137d109b62ef@suse.com>
Date: Tue, 13 Feb 2024 12:36:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 14/30] xen/riscv: introduce atomic.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Bobby Eshleman <bobbyeshleman@gmail.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Connor Davis <connojdavis@gmail.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <6554f2479e19ed3eae6de842ac1568c31d236461.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <6554f2479e19ed3eae6de842ac1568c31d236461.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> From: Bobby Eshleman <bobbyeshleman@gmail.com>
> 
> Additionally, this patch introduces macros in fence.h,
> which are utilized in atomic.h.

These are used in an earlier patch already, so either you want to
re-order the series, or you want to move that introduction ahead.

> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/atomic.h
> @@ -0,0 +1,395 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Taken and modified from Linux.
> + *
> + * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated to use
> + * __*xchg_generic()
> + * 
> + * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
> + * Copyright (C) 2012 Regents of the University of California
> + * Copyright (C) 2017 SiFive
> + * Copyright (C) 2021 Vates SAS
> + */
> +
> +#ifndef _ASM_RISCV_ATOMIC_H
> +#define _ASM_RISCV_ATOMIC_H
> +
> +#include <xen/atomic.h>
> +#include <asm/cmpxchg.h>
> +#include <asm/fence.h>
> +#include <asm/io.h>
> +#include <asm/system.h>
> +
> +void __bad_atomic_size(void);
> +
> +static always_inline void read_atomic_size(const volatile void *p,
> +                                           void *res,
> +                                           unsigned int size)
> +{
> +    switch ( size )
> +    {
> +    case 1: *(uint8_t *)res = readb(p); break;
> +    case 2: *(uint16_t *)res = readw(p); break;
> +    case 4: *(uint32_t *)res = readl(p); break;
> +    case 8: *(uint32_t *)res  = readq(p); break;

Why is it the MMIO primitives you use here, i.e. not read<X>_cpu()?
It's RAM you're accessing after all.

Also - no CONFIG_64BIT conditional here (like you have in the other
patch)?

> +    default: __bad_atomic_size(); break;
> +    }
> +}
> +
> +#define read_atomic(p) ({                               \
> +    union { typeof(*p) val; char c[0]; } x_;            \
> +    read_atomic_size(p, x_.c, sizeof(*p));              \

I'll be curious for how much longer gcc will tolerate this accessing
of a zero-length array, without issuing at least a warning. I'd
recommend using sizeof(*(p)) as the array dimension right away. (From
this not also the missing parentheses in what you have.)

> +    x_.val;                                             \
> +})
> +
> +#define write_atomic(p, x)                              \
> +({                                                      \
> +    typeof(*p) x__ = (x);                               \
> +    switch ( sizeof(*p) )                               \
> +    {                                                   \
> +    case 1: writeb((uint8_t)x__,  p); break;            \
> +    case 2: writew((uint16_t)x__, p); break;            \
> +    case 4: writel((uint32_t)x__, p); break;            \
> +    case 8: writeq((uint64_t)x__, p); break;            \

Are the casts actually necessary here?

> +    default: __bad_atomic_size(); break;                \
> +    }                                                   \
> +    x__;                                                \
> +})
> +
> +#define add_sized(p, x)                                 \
> +({                                                      \
> +    typeof(*(p)) x__ = (x);                             \
> +    switch ( sizeof(*(p)) )                             \
> +    {                                                   \
> +    case 1: writeb(read_atomic(p) + x__, p); break;     \
> +    case 2: writew(read_atomic(p) + x__, p); break;     \
> +    case 4: writel(read_atomic(p) + x__, p); break;     \
> +    default: __bad_atomic_size(); break;                \
> +    }                                                   \
> +})
> +
> +/*
> + *  __unqual_scalar_typeof(x) - Declare an unqualified scalar type, leaving
> + *               non-scalar types unchanged.
> + *
> + * Prefer C11 _Generic for better compile-times and simpler code. Note: 'char'
> + * is not type-compatible with 'signed char', and we define a separate case.
> + */
> +#define __scalar_type_to_expr_cases(type)               \
> +    unsigned type:  (unsigned type)0,                   \
> +    signed type:    (signed type)0
> +
> +#define __unqual_scalar_typeof(x) typeof(               \
> +    _Generic((x),                                       \
> +        char:  (char)0,                                 \
> +        __scalar_type_to_expr_cases(char),              \
> +        __scalar_type_to_expr_cases(short),             \
> +        __scalar_type_to_expr_cases(int),               \
> +        __scalar_type_to_expr_cases(long),              \
> +        __scalar_type_to_expr_cases(long long),         \
> +        default: (x)))

This isn't RISC-V specific, is it? In which case it wants moving to,
perhaps, xen/macros.h (and then also have the leading underscores
dropped).

> +#define READ_ONCE(x)  (*(const volatile __unqual_scalar_typeof(x) *)&(x))
> +#define WRITE_ONCE(x, val)                                      \
> +    do {                                                        \
> +        *(volatile typeof(x) *)&(x) = (val);                    \
> +    } while (0)

In Xen we use ACCESS_ONCE(); any reason you need to introduce
{READ,WRITE}_ONCE() in addition? Without them, __unqual_scalar_typeof()
may then also not be needed (or, if there's a need to enhance it, may
then be needed for ACCESS_ONCE()). Which in turn raises the question
why only READ_ONCE() uses it here.

> +#define __atomic_acquire_fence() \
> +    __asm__ __volatile__( RISCV_ACQUIRE_BARRIER "" ::: "memory" )

Missing blank here and ...

> +#define __atomic_release_fence() \
> +    __asm__ __volatile__( RISCV_RELEASE_BARRIER "" ::: "memory" );

... here, and stray semicolon additionally just here.

> +static inline int atomic_read(const atomic_t *v)
> +{
> +    return READ_ONCE(v->counter);
> +}
> +
> +static inline int _atomic_read(atomic_t v)
> +{
> +    return v.counter;
> +}
> +
> +static inline void atomic_set(atomic_t *v, int i)
> +{
> +    WRITE_ONCE(v->counter, i);
> +}
> +
> +static inline void _atomic_set(atomic_t *v, int i)
> +{
> +    v->counter = i;
> +}
> +
> +static inline int atomic_sub_and_test(int i, atomic_t *v)
> +{
> +    return atomic_sub_return(i, v) == 0;
> +}
> +
> +static inline void atomic_inc(atomic_t *v)
> +{
> +    atomic_add(1, v);
> +}
> +
> +static inline int atomic_inc_return(atomic_t *v)
> +{
> +    return atomic_add_return(1, v);
> +}
> +
> +static inline void atomic_dec(atomic_t *v)
> +{
> +    atomic_sub(1, v);
> +}
> +
> +static inline int atomic_dec_return(atomic_t *v)
> +{
> +    return atomic_sub_return(1, v);
> +}
> +
> +static inline int atomic_dec_and_test(atomic_t *v)
> +{
> +    return atomic_sub_return(1, v) == 0;
> +}
> +
> +static inline int atomic_add_negative(int i, atomic_t *v)
> +{
> +    return atomic_add_return(i, v) < 0;
> +}
> +
> +static inline int atomic_inc_and_test(atomic_t *v)
> +{
> +    return atomic_add_return(1, v) == 0;
> +}

None of these look RISC-V-specific. Perhaps worth having something in
asm-generic/ that can be utilized here?

> +/*
> + * First, the atomic ops that have no ordering constraints and therefor don't
> + * have the AQ or RL bits set.  These don't return anything, so there's only
> + * one version to worry about.
> + */
> +#define ATOMIC_OP(op, asm_op, I, asm_type, c_type, prefix)  \
> +static inline                                               \
> +void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \
> +{                                                           \
> +    __asm__ __volatile__ (                                  \
> +        "   amo" #asm_op "." #asm_type " zero, %1, %0"      \
> +        : "+A" (v->counter)                                 \
> +        : "r" (I)                                           \
> +        : "memory" );                                       \
> +}                                                           \
> +
> +#define ATOMIC_OPS(op, asm_op, I)                           \
> +        ATOMIC_OP (op, asm_op, I, w, int,   )

So the last three parameters are to be ready to also support
atomic64, without actually doing so right now?

> +ATOMIC_OPS(add, add,  i)
> +ATOMIC_OPS(sub, add, -i)
> +ATOMIC_OPS(and, and,  i)
> +ATOMIC_OPS( or,  or,  i)
> +ATOMIC_OPS(xor, xor,  i)
> +
> +#undef ATOMIC_OP
> +#undef ATOMIC_OPS
> +
> +/*
> + * Atomic ops that have ordered, relaxed, acquire, and release variants.
> + * There's two flavors of these: the arithmatic ops have both fetch and return
> + * versions, while the logical ops only have fetch versions.
> + */

I'm somewhat confused by the comment: It first talks of 4 variants, but
then says there are only 2 (arithmetic) or 1 (logical) ones.

> +#define ATOMIC_FETCH_OP(op, asm_op, I, asm_type, c_type, prefix)    \
> +static inline                                                       \
> +c_type atomic##prefix##_fetch_##op##_relaxed(c_type i,              \
> +                         atomic##prefix##_t *v)                     \
> +{                                                                   \
> +    register c_type ret;                                            \
> +    __asm__ __volatile__ (                                          \
> +        "   amo" #asm_op "." #asm_type " %1, %2, %0"                \
> +        : "+A" (v->counter), "=r" (ret)                             \
> +        : "r" (I)                                                   \
> +        : "memory" );                                               \
> +    return ret;                                                     \
> +}                                                                   \
> +static inline                                                       \
> +c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t *v) \
> +{                                                                   \
> +    register c_type ret;                                            \
> +    __asm__ __volatile__ (                                          \
> +        "   amo" #asm_op "." #asm_type ".aqrl  %1, %2, %0"          \
> +        : "+A" (v->counter), "=r" (ret)                             \
> +        : "r" (I)                                                   \
> +        : "memory" );                                               \
> +    return ret;                                                     \
> +}
> +
> +#define ATOMIC_OP_RETURN(op, asm_op, c_op, I, asm_type, c_type, prefix) \
> +static inline                                                           \
> +c_type atomic##prefix##_##op##_return_relaxed(c_type i,                 \
> +                          atomic##prefix##_t *v)                        \
> +{                                                                       \
> +        return atomic##prefix##_fetch_##op##_relaxed(i, v) c_op I;      \
> +}                                                                       \
> +static inline                                                           \
> +c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t *v)  \
> +{                                                                       \
> +        return atomic##prefix##_fetch_##op(i, v) c_op I;                \
> +}
> +
> +#define ATOMIC_OPS(op, asm_op, c_op, I)                                 \
> +        ATOMIC_FETCH_OP( op, asm_op,       I, w, int,   )               \
> +        ATOMIC_OP_RETURN(op, asm_op, c_op, I, w, int,   )
> +
> +ATOMIC_OPS(add, add, +,  i)
> +ATOMIC_OPS(sub, add, +, -i)
> +
> +#define atomic_add_return_relaxed   atomic_add_return_relaxed
> +#define atomic_sub_return_relaxed   atomic_sub_return_relaxed
> +#define atomic_add_return   atomic_add_return
> +#define atomic_sub_return   atomic_sub_return
> +
> +#define atomic_fetch_add_relaxed    atomic_fetch_add_relaxed
> +#define atomic_fetch_sub_relaxed    atomic_fetch_sub_relaxed
> +#define atomic_fetch_add    atomic_fetch_add
> +#define atomic_fetch_sub    atomic_fetch_sub

What are all of these #define-s (any yet more further down) about?

> +static inline int atomic_sub_if_positive(atomic_t *v, int offset)
> +{
> +       int prev, rc;
> +
> +    __asm__ __volatile__ (
> +        "0: lr.w     %[p],  %[c]\n"
> +        "   sub      %[rc], %[p], %[o]\n"
> +        "   bltz     %[rc], 1f\n"
> +        "   sc.w.rl  %[rc], %[rc], %[c]\n"
> +        "   bnez     %[rc], 0b\n"
> +        "   fence    rw, rw\n"
> +        "1:\n"
> +        : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter)
> +        : [o]"r" (offset)

Nit: Blanks please between ] and ".

> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/fence.h
> @@ -0,0 +1,8 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +#ifndef _ASM_RISCV_FENCE_H
> +#define _ASM_RISCV_FENCE_H
> +
> +#define RISCV_ACQUIRE_BARRIER   "\tfence r , rw\n"
> +#define RISCV_RELEASE_BARRIER   "\tfence rw,  w\n"

Seeing that another "fence rw, rw" appears in this patch, I'm now pretty
sure you want to add e.g. RISCV_FULL_BARRIER here as well.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 11:37:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 11:37:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679856.1057599 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZr6I-0006wC-DC; Tue, 13 Feb 2024 11:37:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679856.1057599; Tue, 13 Feb 2024 11:37:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZr6I-0006w5-AT; Tue, 13 Feb 2024 11:37:22 +0000
Received: by outflank-mailman (input) for mailman id 679856;
 Tue, 13 Feb 2024 11:37:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=oPHc=JW=gmail.com=xadimgnik@srs-se1.protection.inumbo.net>)
 id 1rZr6G-0005wr-Q7
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 11:37:20 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3f3066ea-ca64-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 12:37:20 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33b28aadb28so2675574f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 03:37:20 -0800 (PST)
Received: from [192.168.8.23] (54-240-197-234.amazon.com. [54.240.197.234])
 by smtp.gmail.com with ESMTPSA id
 p16-20020adfcc90000000b0033b2497fdaesm9276695wrj.95.2024.02.13.03.37.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 03:37:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f3066ea-ca64-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707824240; x=1708429040; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:organization:references:cc:to
         :content-language:subject:reply-to:user-agent:mime-version:date
         :message-id:from:from:to:cc:subject:date:message-id:reply-to;
        bh=sQis1qvp/U8auXJAKP/yrBgCFkOt5KYyAk393lVPOdY=;
        b=A/qaFvjRaEk9VPB003SzkFxMiqXqQF8U5aejURJ8RMLIF62S3dVo13ZocTrddcUH62
         Y9JsKA4yZ6Kh/JfAdt6uayffK59/NPlhbEqo83/sDaFwIIEzJTiAS9vSMGKbScPN2AWH
         kKUGEDHQhWRGgw6uAQazkPohwVEFYOzU7U2kBcPntd6z7HSsoaAed7YhfawYoMRTL7+0
         17558DQNgncwUuoAre6MO59PnVK8rlITo3AvFnx20Sm34ue7lw5M8gqwjbQnNxlxrKCN
         k/WuGBG6kZ7ufRhfpQXYiiT53uiM4OKGhn1fX52g3j1yUMb4XToy2/6WPl8qJusBLQiW
         HEAg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707824240; x=1708429040;
        h=content-transfer-encoding:in-reply-to:organization:references:cc:to
         :content-language:subject:reply-to:user-agent:mime-version:date
         :message-id:from:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=sQis1qvp/U8auXJAKP/yrBgCFkOt5KYyAk393lVPOdY=;
        b=V92P9JXmePywvcuj1/xz5ZQpxJepkAOo4BIk4NCj6+wvXKidvCTPKhn3g7d9ZPJ/IT
         MFI4BCpNl5Jh8iiaNcwKP6yD0qtvW4wDraUGUlifIAjJsca1KDokev5FftncTbq5lzox
         Rhni9ZpuBqb/DXT+IYO57APeX8fXUIWmg/ysCmM7Z71tFE59XuRSkTB7J1G5HwlKYBlA
         13Zfrdcnk4l5bwJ15z229pxpbTGCxb6aErSS5mfshBLhS5f0enGv/1F4Sl5DZIQiXqmj
         WqXIJIZMHB7wbffUg0g1Vnluq8Rt/LPvnT3VRCaMl9/wUElFGbgn1U8YPecO8WS3wvB+
         cIzQ==
X-Forwarded-Encrypted: i=1; AJvYcCWczQWpPNEnXcy4Ekpvg1hcUvPC8zhmhYMA0aCtCjeytkI6A3/MnJDBdchxJWAkQsNW65HvbAJ0LWVyLbgBZWIBOoNB8ntIl4eQJf7NqN0=
X-Gm-Message-State: AOJu0YywD5ZNkvK/1HqgvCNela6Yy1k7k6v9m5a8bLZM1wdRNNpG6tWe
	0DOieewGWIUSRGRhgbtkYaYPHVdwnWFy+EcDcIw+Ay4TvndIjM0l
X-Google-Smtp-Source: AGHT+IFK4Uezp+NtfTRO82kWqc1JFIQYUrKoKWoO8VA6xRW8Sfp6GwechICHtRD/UhF7cWc5l61FwQ==
X-Received: by 2002:a5d:4a04:0:b0:33b:615a:19b7 with SMTP id m4-20020a5d4a04000000b0033b615a19b7mr6889589wrq.22.1707824239404;
        Tue, 13 Feb 2024 03:37:19 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWYwXCX5Zan6qTGfNR6oFZZAnPtvBVbtDkgy46zuE6MWtOG79IXS1Xg9FMxBxaQ7ljLpjj5Ino1dZ5cTTXXzuRaRgAqN7nfmWbWCawfFsjD3OnIU+f1iKFB4NLJPwLZTuCKBi/Hl5A05/yKAyo/xndTcJhSqOrYindSC8FpLWr0mabO7E0cefKm8+Q9hCr0UdrUP3Any7Pb3YlzFxBMjLm5k9P7Slfu3WToV5gVxzRogu1S//5oFy3GHJRaTUp1wuhbbl0XSif73wXn1ehw4DLDyiU7eRD1jXG34FMDTzKKUw==
From: Paul Durrant <xadimgnik@gmail.com>
X-Google-Original-From: Paul Durrant <paul@xen.org>
Message-ID: <fb3fb0b0-c11e-49a8-9a28-f3d3bb0fb6ee@xen.org>
Date: Tue, 13 Feb 2024 11:37:06 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: paul@xen.org
Subject: Re: [PATCH net 1/7] net: fill in MODULE_DESCRIPTION()s for
 xen-netback
Content-Language: en-US
To: Breno Leitao <leitao@debian.org>, kuba@kernel.org, davem@davemloft.net,
 pabeni@redhat.com, edumazet@google.com, Wei Liu <wei.liu@kernel.org>
Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, horms@kernel.org,
 "moderated list:XEN NETWORK BACKEND DRIVER" <xen-devel@lists.xenproject.org>
References: <20240213112122.404045-1-leitao@debian.org>
 <20240213112122.404045-2-leitao@debian.org>
Organization: Xen Project
In-Reply-To: <20240213112122.404045-2-leitao@debian.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 13/02/2024 11:21, Breno Leitao wrote:
> W=1 builds now warn if module is built without a MODULE_DESCRIPTION().
> Add descriptions to the Xen backend network module.
> 
> Signed-off-by: Breno Leitao <leitao@debian.org>
> ---
>   drivers/net/xen-netback/netback.c | 1 +
>   1 file changed, 1 insertion(+)
> 

Acked-by: Paul Durrant <paul@xen.org>

> diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
> index fab361a250d6..ef76850d9bcd 100644
> --- a/drivers/net/xen-netback/netback.c
> +++ b/drivers/net/xen-netback/netback.c
> @@ -1778,5 +1778,6 @@ static void __exit netback_fini(void)
>   }
>   module_exit(netback_fini);
>   
> +MODULE_DESCRIPTION("Xen backend network device module");
>   MODULE_LICENSE("Dual BSD/GPL");
>   MODULE_ALIAS("xen-backend:vif");



From xen-devel-bounces@lists.xenproject.org Tue Feb 13 11:41:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 11:41:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679864.1057609 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrAH-0000fV-TN; Tue, 13 Feb 2024 11:41:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679864.1057609; Tue, 13 Feb 2024 11:41:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrAH-0000fO-QR; Tue, 13 Feb 2024 11:41:29 +0000
Received: by outflank-mailman (input) for mailman id 679864;
 Tue, 13 Feb 2024 11:41:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gnrE=JW=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZrAG-0000fG-5w
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 11:41:28 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d2726e06-ca64-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 12:41:27 +0100 (CET)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-33b1d7f736bso566658f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 03:41:27 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a15-20020adffacf000000b0033b39cba5e4sm9296098wrs.116.2024.02.13.03.41.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 03:41:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2726e06-ca64-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707824487; x=1708429287; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=XQhla5REKvPeHXu4c0wBu7qVePgmPWZor8lCzIG6MU0=;
        b=IUtfs6zDiet98ej2bk51AYrgiGwX1IYoBJ5iBveCDpqeF0tFxnHgPxqeR4/OVE8noo
         +g/66vr71ziPDMrL+F58qIJIf+iLpDEuCJmv5f4Zu3iKRpddmuQsMOekyIsP5mhudc1i
         z5SKYtbc5RL6qAdH5grpVbKRpuwJjjY87Xwlg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707824487; x=1708429287;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XQhla5REKvPeHXu4c0wBu7qVePgmPWZor8lCzIG6MU0=;
        b=JM/doAnlrSREhyYf0YNwxJkd6fODR2Vbllwvh0C5f/qsnRzL6iIfLyRJ84kumwGclD
         L5K9H9GHB44ErR6ZpYOVdzwmMHtbye81hDPeiWP1USLWpSeLQTCd7idZdRh0iQdOI/kf
         ETvWhtayLVwfpWxMM19PkWbIZAtwsKmSAY5iiMwAUj2J0FHWaImfpvis60AI04XyfNho
         fCkqgjASqvbB1TyZhBoVs3Xv9LWQ3YLRTQ8bk9KQDsJfYKNhU5xUxIM1vFv82FWLWR9b
         1U/zBbRZuzG1VyBQp/2mPQ/cwwPSCkTwmIVwxSMJViGlHnDnnhlibQIjmoOBRIXh5lht
         PE0g==
X-Gm-Message-State: AOJu0YyRnoeh053gxMP37X4KR7Pt/f84IaTS5Lwvy6FTlIX/KQFvxYgm
	wZa44cTwSGf2U8koN1hnNa9yftQ7iEr0fVYGFEzXz7L3QE2CkIdzG7wCsBnbxXMEA7ZOP70irxk
	2
X-Google-Smtp-Source: AGHT+IGV3cIEfwQqvQVIastam2kN3OinMwDNNobqiijwCBlNaF3PoRMck2XUEJ/9Jwo6KwMn7/p6HA==
X-Received: by 2002:a5d:4b52:0:b0:33a:eec4:c0c6 with SMTP id w18-20020a5d4b52000000b0033aeec4c0c6mr5924287wrs.12.1707824486783;
        Tue, 13 Feb 2024 03:41:26 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU3P/PvkkIUB75s0vD3f6OIrkC2jNbDYJf/cqYWZcadXCjyq4gNKdViNo7/zIqaqE8JTjYz8xXXLwZ3G1j7b102BAuvBG0AZi1FoShqH9+cFkoT4DcODIC0Chx+96K3lSzzZ233vnIo6MFLGaiBts394JhEePLCOHGv1OhYItIpjunIkf5OPnkNVf8rMYFuoEwnAqlw2FOKd/T7xWJjZr4EB+bJum6t3YYUI+jLgfZZaRv0nnd+HC2rvvy2he38UlvflNqwH183TUSi4ae18wagSsMi58RSq+Q8FKoc
Date: Tue, 13 Feb 2024 11:41:26 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, marmarek@invisiblethingslab.com,
	Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH v2 0/3] libxl: Stubdom cd-rom changing support
Message-ID: <e5df0cde-3855-45c3-9000-cf34baf28616@perard>
References: <20240201214004.238858-1-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240201214004.238858-1-jandryuk@gmail.com>

On Thu, Feb 01, 2024 at 04:40:01PM -0500, Jason Andryuk wrote:
> Linux locks the cd-rom by default?  That means the QMP eject commands
> fail, but then Linux unlocks.  Re-running a second time works.  Windows
> doesn't do that.

Yes, looks like it. It would be possible to have libxl retry
automatically since Linux eject and unlock the cd-rom tray, then QEMU
does send an event via QMP about this, so libxl could retry after
receiving the event. Waiting for events is just not implemented in
libxl, but that something that could be done.

The event is, for example:
    'event': 'DEVICE_TRAY_MOVED', 'data': {'device': 'ide-5632', 'tray-open': True, 'id': 'ide-5632'}

Cheers,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 11:42:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 11:42:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679867.1057619 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrAx-0001D3-A4; Tue, 13 Feb 2024 11:42:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679867.1057619; Tue, 13 Feb 2024 11:42:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrAx-0001Cw-69; Tue, 13 Feb 2024 11:42:11 +0000
Received: by outflank-mailman (input) for mailman id 679867;
 Tue, 13 Feb 2024 11:42:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gnrE=JW=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZrAw-0000fG-8w
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 11:42:10 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ebc41058-ca64-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 12:42:09 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-3394b892691so2793568f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 03:42:09 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 t17-20020a05600c199100b0041069adbd87sm11681216wmq.21.2024.02.13.03.42.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 03:42:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ebc41058-ca64-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707824529; x=1708429329; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=BqKGq5Rihofp0wbN6VyPB/1V+A38BSfVOvDXOT3Q0hY=;
        b=Kip6C0i8V5nXcQpiNmrVVUfYPT1ACXwVltBBNzYQ+Q+w4C68kbVGthkbW1I8Lh42Tp
         nUwqAjyySmKGyUWg5PesBZi+ulkbsZcDU4IHKp6kbC1E2CpFhNVgDZxWm3lM5G4VAMeF
         sVuGNfIVUJpyrH89eBoqykh5c9n18LVsj/MwQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707824529; x=1708429329;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=BqKGq5Rihofp0wbN6VyPB/1V+A38BSfVOvDXOT3Q0hY=;
        b=HONacPQR2Hh4jixRvhiYg6btIrTSCxvfmbEd+pttfp7yetztEi1PMx0m96foiOoXUK
         zH+Y2U8D2oTv59+5eMYV7FWyisa0m/zBBUklU4cChMGgRYBuMY1g6/2KL2GduBxBeWGa
         ay5dPaqu2TyIgMEqX4Kzl11qr+KOtICfbtPKAGpswtmQ7w/X+JUecuQA6cufL9Lu0z3c
         tukvOYXE9M5SBM8xH4L2+fu4zNbpo0nUvJJl4NRRAbbGr4B3J/4g93hk/L2oTsKTmwJ1
         J1cNwPWcIy6HikJ1BHnZ3Lgw66HRJyKsw0wecMXizlq3EwAWkQTjpOGgTeYCnwlkCSE1
         TeFQ==
X-Gm-Message-State: AOJu0YyJfuOMB++7GtT3O5SOnjNL0xVi6iHaADLGQnHLkgh0ooDG6HnI
	SU40Ro1CxRl2UPaCPyqmF8rDFoWJB1dKaSuwI5ly3F10rdZLXuvvq0C0EdV6DVaP+btC40tS8zi
	i
X-Google-Smtp-Source: AGHT+IFJldikD4pD+qgUgBILN2mzEnuJ2H738k1u2ZLCaKjs/EPrgDobC1Aw+icE6v8e5yOULLV/OA==
X-Received: by 2002:a05:6000:136c:b0:33b:51d5:bd1c with SMTP id q12-20020a056000136c00b0033b51d5bd1cmr1722315wrz.12.1707824529148;
        Tue, 13 Feb 2024 03:42:09 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXQuorW5H5DgdRpYxnAhRhSajAwwNhS6IdZmw7ScIIVCL2fEZaONcIqdYkUvB+EXm5mb1Z0+Q5UGV7Wwj/mVXjd9bSIgfQVdhmWbGNfgos1SL1J2lUgP981puFk/mkuyI1bGilVYVtPCl4rCWM=
Date: Tue, 13 Feb 2024 11:42:08 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, marmarek@invisiblethingslab.com,
	Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2 1/3] libxl: Create empty file for Phy cdrom
Message-ID: <5fac8d52-c07a-4e1d-b6e8-c47fcb4f12d0@perard>
References: <20240201214004.238858-1-jandryuk@gmail.com>
 <20240201214004.238858-2-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240201214004.238858-2-jandryuk@gmail.com>

On Thu, Feb 01, 2024 at 04:40:02PM -0500, Jason Andryuk wrote:
> With a device model stubdom, dom0 exports a PV disk to the stubdom.
> Inside the stubdom, QEMU emulates a cdrom to the guest with a
> host_device pointing at the PV frontend (/dev/xvdc)
> 
> An empty cdrom drive causes problems booting the stubdom.  The PV disk
> protocol isn't designed to support no media.  That can be partially
> hacked around, but the stubdom kernel waits for all block devices to
> transition to Connected.  Since the backend never connects empty media,
> stubdom launch times out and it is destroyed.
> 
> Empty media and the PV disks not connecting is fine at runtime since the
> stubdom keeps running irrespective of the disk state.
> 
> Empty media can be worked around my providing an empty file to the
> stubdom for the PV disk source.  This works as the disk is exposed as a
> zero-size disk.  Dynamically create the empty file as needed and remove
> in the stubdom cleanup.
> 
> libxl__device_disk_set_backend() needs to allow through these "empty"
> disks with a pdev_path.
> 
> Fixup the params writing since scripts have trouble with an empty params
> field.
> 
> This works for non-stubdom HVMs as well.
> 
> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 11:42:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 11:42:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679868.1057629 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrBI-0001fT-I6; Tue, 13 Feb 2024 11:42:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679868.1057629; Tue, 13 Feb 2024 11:42:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrBI-0001fM-EF; Tue, 13 Feb 2024 11:42:32 +0000
Received: by outflank-mailman (input) for mailman id 679868;
 Tue, 13 Feb 2024 11:42:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gnrE=JW=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZrBG-0000fG-VD
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 11:42:30 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f838589c-ca64-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 12:42:30 +0100 (CET)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2d073b54359so62895951fa.0
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 03:42:30 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 r11-20020a05600c458b00b00410504b582csm11558849wmo.11.2024.02.13.03.42.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 03:42:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f838589c-ca64-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707824550; x=1708429350; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=SFOxlptOy6yPH4rsd8K2htw0DvvMc2uXfBIk/uVlwbs=;
        b=RShpAisH7H105h734oYjHnECSid7P82n+5EKyUyo9eBrQbXrrcgouEiW0h4h9SMeOW
         EgElGS06ioKF5lYK5mVkosRBxH9aB8aeiRtIgURRZrt0vzoxkOX3sgi/0vMs34DIj3D1
         /UWRJ65PaDZcQFqjp9m5EhUTcv6EnuKl6H+fY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707824550; x=1708429350;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SFOxlptOy6yPH4rsd8K2htw0DvvMc2uXfBIk/uVlwbs=;
        b=FsxUnQ5Cu4XY6Q+0Aasylo5t2Ulss3cBbwGiWQpyQANOfisxcFa9B7xjmSg793J38U
         DzMwlWVG5WEtLb6OIwEJRdOP2P3hzImDYIeg+mXt2ZfbQRn+rk3EGeR38uHNsx2QMDcZ
         YMm8lVJMZ+AQOf6PpA79ZIBD5ggvD4B1KqQ6ObrKgAes8+jafGNtt92HBsYztrvdPO3U
         pYjqhUjnfh7ANi8Zbf/K0pvxy2zWGKxbP5jBYG1tg4XBq8wS/GzJ/6BA0t2G4FijaLk5
         fCov3m8/rQcKIvqlsMU8NufZWN8hLRnptiYo/qqwTTHiaOnOMDE2TW3yDXxMl7qlG02v
         ExAg==
X-Gm-Message-State: AOJu0YzZHo/ZTr2onOnAdu3v+QJXt5CZAVBGN1AGy4+PA08v4fvXQO5s
	F+6/1iWC3DDWVxa8xxJk+alvxlSWI9L6azPAH9V7qTNlQooCVmxPNGCBQSw1Ou6HS05RHleHtFL
	O
X-Google-Smtp-Source: AGHT+IFupjNruEJrK6M00oYJtsFjanhK06mAzHaDcS7siv7Dz+vr+ljh9/JeONto5LOKC7c8sK48Vg==
X-Received: by 2002:a2e:a548:0:b0:2d0:be0f:96ff with SMTP id e8-20020a2ea548000000b002d0be0f96ffmr7808300ljn.17.1707824550099;
        Tue, 13 Feb 2024 03:42:30 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWwInivjc+fhkoQTGh52mlruKbbsI5ah9NIG04rizDPVZ82wnE3qp4h6epP2fF30zUQkgMzYuUPzoYmsWtZ44XWKHxirhdbX7b0nFILI4bCAdFWAMyw2iqIaV9JK560kH+hftJdOKPniuBL0S0=
Date: Tue, 13 Feb 2024 11:42:29 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, marmarek@invisiblethingslab.com,
	Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2 2/3] libxl: Allow Phy backend for CDROM devices
Message-ID: <d1c987d1-1f2d-40bd-9e28-21415cdb84ce@perard>
References: <20240201214004.238858-1-jandryuk@gmail.com>
 <20240201214004.238858-3-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240201214004.238858-3-jandryuk@gmail.com>

On Thu, Feb 01, 2024 at 04:40:03PM -0500, Jason Andryuk wrote:
> A Linux HVM domain ignores PV block devices with type cdrom.  The
> Windows PV drivers also ignore device-type != "disk".  Therefore QEMU's
> emulated CD-ROM support is used.  This allows ejection and other CD-ROM
> features to work.
> 
> With a stubdom, QEMU is running in the stubdom.  A PV disk is still
> connected into the stubdom, and then QEMU can emulate the CD-ROM into
> the guest.  Phy support has been enhanced to provide a placeholder file
> forempty disks, so it is usable as a CDROM backend as well.  Allow Phy
> to pass the check as well.
> 
> (Bypassing just for a linux-based stubdom doesn't work because
> libxl__device_disk_setdefault() gets called early in domain creation
> before xenstore is populated with relevant information for the stubdom
> type.  The build information isn't readily available and won't exist in
> some call trees, so it isn't usable either.)
> 
> Let disk_try_backend() allow format empty for Phy cdrom drives.
> 
> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 11:54:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 11:54:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679879.1057638 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrN8-000485-Id; Tue, 13 Feb 2024 11:54:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679879.1057638; Tue, 13 Feb 2024 11:54:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrN8-00047y-GB; Tue, 13 Feb 2024 11:54:46 +0000
Received: by outflank-mailman (input) for mailman id 679879;
 Tue, 13 Feb 2024 11:54:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gnrE=JW=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZrN7-00047s-A7
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 11:54:45 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id acf1939e-ca66-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 12:54:43 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-411c779ab2dso2488435e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 03:54:43 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 dn7-20020a05600c654700b00411b7c91470sm1783573wmb.12.2024.02.13.03.54.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 03:54:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: acf1939e-ca66-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707825283; x=1708430083; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=GHforX524n+IhGAmSToWCoKgL9eYJr/QjPXWxywpgPU=;
        b=V348cra5z2uhxcDFt9+oUILKAqCADHf/gjFtYbZC2nRjihNkQCTV+mAWlKC6uYfv4I
         vtndk0zMJ7BtDVMfBLoTxcQwWWH0dM9xqXCwfeJmH5W4Ovx74xByvu3cvv5XBn0SU/7L
         UIuHCK/CAT8zml/J+GV/AfNwMHZOhhr8p7r4k=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707825283; x=1708430083;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GHforX524n+IhGAmSToWCoKgL9eYJr/QjPXWxywpgPU=;
        b=ZlQEzbPK/bncv7HtPs7l8PxTEl0NTUDcDfrb8juVs/b9uQRZvHUVSKTgRQyl9pEH4v
         wXyouZkLt0Y8zIgHTNoZ7d1ajgTFL7JBNC7soqbmGwCPXOVvmeoKBLb32q6e2EHnx+eB
         B9MB9sYVd5mwq7yb1gFH4xjS7lXpYpeqiyu48xxf6tCi6tu0j6I8uPk8aJTv8esVduva
         +WMZWkh3739IkL42pIeTYOLm348SDxzKs1K+yeBRsRnSZUjKXnyrLj3v0/Hp3Jk3Vy+q
         djd912ODMwHNVVIF2p1pnOMIW1N645/lxPb//Z+mc1ynqJdX9sFJs9Dj9c/WYb1TL0ay
         HiYw==
X-Gm-Message-State: AOJu0YzynHrnlEMLb9zdSKyfcOWhGSbVxxNMxu23ng9UisLBaE41gfz6
	RhzpFwmBQkCNRYgEVNVEV0qXKNxOoYWwouFERA9O0X63oI5qDw00LpPmFN4KQJ4=
X-Google-Smtp-Source: AGHT+IFXm5l5BOck2+xxXz4SHFCtloMRqyP8/lk9nejBFwo0ZDoQP0BamW4yCbmxWygh/G5ZhZQ+3Q==
X-Received: by 2002:a05:600c:4e06:b0:411:c25d:7018 with SMTP id b6-20020a05600c4e0600b00411c25d7018mr890540wmq.12.1707825282890;
        Tue, 13 Feb 2024 03:54:42 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUJyp4aSECUFfTXYBDc1lPM94AYeKAJ2OGu24M4QNLTIiJ6Hhf2U6oa+OUnDUE1cDb/nHUl7CZGV7uzgQqjKYETAXa5aLg69+jF2WMqF2nWv4RBtu5wGdRsZVLKyCVIa21AOecM2r8=
Date: Tue, 13 Feb 2024 11:54:42 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Petr =?utf-8?B?QmVuZcWh?= <w1benny@gmail.com>
Cc: xen-devel@lists.xenproject.org, andrew.cooper3@citrix.com,
	Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] libxl: Fix comment for LIBXL_HAVE_VMTRACE_BUF_KB
Message-ID: <54abae4a-1a39-464f-8ae7-f8e806557017@perard>
References: <2bb71b2ba88e6eb6177c27dd65f2af608a634ac2.1707261567.git.w1benny@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <2bb71b2ba88e6eb6177c27dd65f2af608a634ac2.1707261567.git.w1benny@gmail.com>

On Tue, Feb 06, 2024 at 11:23:45PM +0000, Petr Beneš wrote:
> From: Petr Beneš <w1benny@gmail.com>
> 
> It's located in libxl_domain_build_info, not libxl_domain_create_info.

Hi Petr,

You need a "Signed-off-by:" tag when sending patches.

We could probably add also a fixes tag
Fixes: 45ba9a7d7688 ("tools/[lib]xl: Add vmtrace_buf_size parameter")

Otherwise, patch looks fine, so with the SOB: Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 12:14:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 12:14:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679884.1057649 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrgS-00088E-B5; Tue, 13 Feb 2024 12:14:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679884.1057649; Tue, 13 Feb 2024 12:14:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrgS-000887-7A; Tue, 13 Feb 2024 12:14:44 +0000
Received: by outflank-mailman (input) for mailman id 679884;
 Tue, 13 Feb 2024 12:14:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gnrE=JW=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZrgR-000881-0O
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 12:14:43 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 774d2e24-ca69-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 13:14:41 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d0d7985dfdso58760021fa.2
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 04:14:41 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 bu3-20020a056000078300b0033b46b1b6adsm9549152wrb.21.2024.02.13.04.14.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 04:14:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 774d2e24-ca69-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707826481; x=1708431281; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=WjeH00YjSJrsCeSkjG2978yTN0pqv0tSBf/xh0Gh5/E=;
        b=JqDCVc2KbmrHpKPZgFb75QFGEWvtyCqsLnASWD8NgJ/uQFGFW+zxdbYXQ8nDbER28x
         r4RYF+OT+cBPXyN6d05nEVpvYuRTXsqYfyhIBnFY3IAAF/0sPYwp5Z04pLR7bW21ldP2
         QEXULs593WxVvOzdnx+u6tfdroyBsEPeA4hNk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707826481; x=1708431281;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=WjeH00YjSJrsCeSkjG2978yTN0pqv0tSBf/xh0Gh5/E=;
        b=dzVyF9U5QUM8fhGa6bkE6uNJnsqQgV0Uy+cNLPUHYUKjwnON7OEEiQub6h5SXEaq3/
         nSDZpZ01Sm57DtT8I+dfEIQszdFSwzHh0OCnD7JH8eo+goACnJlTNwKfNLn7lasMoqU6
         vvLnmCb6+aboAbgxMEkR4XwhMSbHeeMb2Ocae41CF14HWSc+TqcI42HxWbe6ENMFTOkF
         5sVp3eo7urrQEq/uQa/gnWxljHOkDwRf9Hv9mCA7dNCWl9dCP+fB2j44f+OyKVJ7bx/b
         fDn0reraOyKH5Q30j7weocLbRB5+8yaTbwHnRHrrPQso1rX3n/cCbLCkK8zaA1rmrUav
         ERfg==
X-Gm-Message-State: AOJu0YxQa/OF0Eakab9+mIBKkWhWSaA3jaIF9vJOOpCIys8eu5N6dERS
	OB93vEyv5bty9qmPOtftACRB/jQA4tpXQYnFT0nd+sQEYyyCd8HYcPkklewS8SkGJ/zrDjNy5fz
	Q
X-Google-Smtp-Source: AGHT+IFoWK0rRjnCTM4tih7mKaCydZAU6wlay2sX/rQQlic0PdScjzmuUuTCJTh/XRynPEBmsWobUA==
X-Received: by 2002:a2e:a36e:0:b0:2d0:b646:3bc9 with SMTP id i14-20020a2ea36e000000b002d0b6463bc9mr5889870ljn.23.1707826481426;
        Tue, 13 Feb 2024 04:14:41 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXPC1/zh/10KrCX/mdPTU2TXkzcOE11lYmSMk/IzDZqYQ+ueFsYp/qBjB97BmhYl3PtFm+YuPAGeCAoB+rmKn+HWsNzQ0OhGqR2F/efUa660QoAiS7QmBHgHagERI/ueaNpZbvdTaJfFb5hOq9Rtt+Ge+oXoI2FEVlKOE5JLPhZ3jFrWtTHncEs0piirTeck6ffnhyjXJYDn77y9CZ5U7Iu9mFchGvWohDuRWkqDNeK3ZiA
Date: Tue, 13 Feb 2024 12:14:40 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Juergen Gross <jgross@suse.com>,
	Viresh Kumar <viresh.kumar@linaro.org>
Subject: Re: [PATCH V2] libxl: Add "grant_usage" parameter for virtio disk
 devices
Message-ID: <a12d924d-b570-4795-8ed6-f62f3604c361@perard>
References: <20240206123814.2308837-1-olekstysh@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240206123814.2308837-1-olekstysh@gmail.com>

On Tue, Feb 06, 2024 at 02:38:14PM +0200, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> Allow administrators to control whether Xen grant mappings for
> the virtio disk devices should be used. By default (when new
> parameter is not specified), the existing behavior is retained
> (we enable grants if backend-domid != 0).
> 
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
> In addition to "libxl: arm: Add grant_usage parameter for virtio devices"
> https://github.com/xen-project/xen/commit/c14254065ff4826e34f714e1790eab5217368c38
> 
> I wonder, whether I had to also include autogenerated changes to:
>  - tools/libs/util/libxlu_disk_l.c
>  - tools/libs/util/libxlu_disk_l.h

Well, that could be done on commit. The changes are going to be needed
to be committed as they may not be regenerated to include the new feature
in a build.

> ---
> diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
> index ea3623dd6f..ed02b655a3 100644
> --- a/tools/libs/light/libxl_disk.c
> +++ b/tools/libs/light/libxl_disk.c
> @@ -623,6 +628,15 @@ static int libxl__disk_from_xenstore(libxl__gc *gc, const char *libxl_path,
>              goto cleanup;
>          }
>          disk->irq = strtoul(tmp, NULL, 10);
> +
> +        tmp = libxl__xs_read(gc, XBT_NULL,
> +                             GCSPRINTF("%s/grant_usage", libxl_path));
> +        if (!tmp) {
> +            LOG(DEBUG, "Missing xenstore node %s/grant_usage, using default value", libxl_path);

Is this information useful for debugging?

It should be easy to find out if the grant_usage node is present or not
by looking at xenstore, and I don't think libxl is going to make use of
that information after this point, so I don't think that's going to be
very useful.

> +            libxl_defbool_set(&disk->grant_usage,
> +                              disk->backend_domid != LIBXL_TOOLSTACK_DOMID);
> +        } else
> +            libxl_defbool_set(&disk->grant_usage, strtoul(tmp, NULL, 0));

Per coding style, it's better to have both side of an if..else to have
{}-block or none of them. So could you add a {} block in the else, or
remove the {} from the true side if we remove the LOG()?

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 12:17:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 12:17:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679850.1057659 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrjK-0000EL-OU; Tue, 13 Feb 2024 12:17:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679850.1057659; Tue, 13 Feb 2024 12:17:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZrjK-0000EE-Lh; Tue, 13 Feb 2024 12:17:42 +0000
Received: by outflank-mailman (input) for mailman id 679850;
 Tue, 13 Feb 2024 11:21:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DGbd=JW=gmail.com=breno.debian@srs-se1.protection.inumbo.net>)
 id 1rZqr2-0004Sn-SF
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 11:21:36 +0000
Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com
 [209.85.208.53]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0b931cc6-ca62-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 12:21:34 +0100 (CET)
Received: by mail-ed1-f53.google.com with SMTP id
 4fb4d7f45d1cf-55a5e7fa471so5527101a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 03:21:34 -0800 (PST)
Received: from localhost (fwdproxy-lla-005.fbsv.net.
 [2a03:2880:30ff:5::face:b00c]) by smtp.gmail.com with ESMTPSA id
 vw3-20020a170907a70300b00a3c66ac512asm1191224ejc.214.2024.02.13.03.21.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 03:21:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b931cc6-ca62-11ee-98f5-efadbce2ee36
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707823294; x=1708428094;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=cuiaVQ6xHYwzgr9cteCqN0D6TVSa51dvKVi3pXWKJaE=;
        b=iYT59SR82LjDRbcm+rFHvSLwzipnK53Fg/PhBv2zOF2sfUeb45KK+XF4nql6rrjZEh
         d0H9tb9ud6SgmC+dhtGYWHRIIaNVyGuAXCaVHQMNKdSxqpbJacbn/frkADZIbbIb7OwD
         3mnAJaQZjD4UhxXzjrqyDDZgzIuDAApFHVmIbHwDyCC1ufoWML8lkiqU5ptk2iA3+8ot
         fgmDarmv18Ssbhu5C3Z6ibuh5K6f3VaslwBMf0obkCyFr897Wj3FonDDm5pCIhhRW3Rx
         2JsYdW4xFywtE9/IiVafuyYc+bCUZ71PQvYLGYMihziaHThdJsWPX8pU0IkktaNGbDNR
         2ZQA==
X-Gm-Message-State: AOJu0Yzn/VvlilsCaX/14+IPJhZYt39/wmEHUwNbUlUAJxTah2nWa47t
	KydpEC0+29D8OP+7ckSJCNHPGoPlGBeoj8ZcyLb6hGgaQZEP7w+c
X-Google-Smtp-Source: AGHT+IEv53qb/jU/q9VIX3Gx9EPG0zlDnaS4y/im5dQM+F2RnxQvedrUeNJK514EezeM/+8URcxAEw==
X-Received: by 2002:a17:906:4144:b0:a3c:82ed:5b46 with SMTP id l4-20020a170906414400b00a3c82ed5b46mr5065767ejk.59.1707823293939;
        Tue, 13 Feb 2024 03:21:33 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUz0v9+Mi6zmFUyJoPQn8nNZKJnBjRetK0SFlIJ9CNm4LQ1UNy0UiKtLBlEEtGLEbzUxs3YRg/JptpG2rh0DJINfUNyx2noHYPNlsDiQmwFHDXWKoZSgsnQ4RZtu7I6wjLW58bcNXhXpRMADteqtr+y93bZTemChK1U2JwCroTv22PFl2FuYECkpkBvL7i0HoO2Ut3SJY1eF7os1cD1p9uRly8yNiU0MyAMaMNhc1e7QX9TrPkHGBrPIq/r0sV6P1rBiw2HpoB8NIQE0rfgYl/BO201iAj6M7U5m2OsRA==
From: Breno Leitao <leitao@debian.org>
To: kuba@kernel.org,
	davem@davemloft.net,
	pabeni@redhat.com,
	edumazet@google.com,
	Wei Liu <wei.liu@kernel.org>,
	Paul Durrant <paul@xen.org>
Cc: netdev@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	horms@kernel.org,
	xen-devel@lists.xenproject.org (moderated list:XEN NETWORK BACKEND DRIVER)
Subject: [PATCH net 1/7] net: fill in MODULE_DESCRIPTION()s for xen-netback
Date: Tue, 13 Feb 2024 03:21:16 -0800
Message-Id: <20240213112122.404045-2-leitao@debian.org>
X-Mailer: git-send-email 2.39.3
In-Reply-To: <20240213112122.404045-1-leitao@debian.org>
References: <20240213112122.404045-1-leitao@debian.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

W=1 builds now warn if module is built without a MODULE_DESCRIPTION().
Add descriptions to the Xen backend network module.

Signed-off-by: Breno Leitao <leitao@debian.org>
---
 drivers/net/xen-netback/netback.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index fab361a250d6..ef76850d9bcd 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -1778,5 +1778,6 @@ static void __exit netback_fini(void)
 }
 module_exit(netback_fini);
 
+MODULE_DESCRIPTION("Xen backend network device module");
 MODULE_LICENSE("Dual BSD/GPL");
 MODULE_ALIAS("xen-backend:vif");
-- 
2.39.3



From xen-devel-bounces@lists.xenproject.org Tue Feb 13 13:07:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 13:07:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679893.1057668 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZsVO-0008Rh-Bd; Tue, 13 Feb 2024 13:07:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679893.1057668; Tue, 13 Feb 2024 13:07:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZsVO-0008Ra-8h; Tue, 13 Feb 2024 13:07:22 +0000
Received: by outflank-mailman (input) for mailman id 679893;
 Tue, 13 Feb 2024 13:07:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZsVN-0008RU-8k
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 13:07:21 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d0dabed7-ca70-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 14:07:18 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-40fb3b5893eso34095165e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 05:07:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 hi7-20020a05600c534700b00410be9b8b96sm7140957wmb.37.2024.02.13.05.07.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 05:07:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0dabed7-ca70-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707829638; x=1708434438; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Sd18wvrol6QQ+0l525rSim8sbUKG0K0n74QLzBOifzc=;
        b=UacjFxd6eSQMp7NQLGX+3RqcyJIAt9OCSLKY8LGTDdGlzJOZ5s6AHL5Q+asNpRuNos
         S87zbpEOFxPMLNClnB+0nnPPXaqAuFvsHpvF7ACaWVdbYF/1dTqrcqAJut53EKMZ+LFq
         uLWvQ25o9oFGMu96FCbI/LMfuGRgELUFLxZAZkh8l0+bsWLCh3HvZuKdkPeswdKFMpOz
         moZ9gYTV378wQ3PFHD/+zv+G1kZMrHccBNqKA0ExkbtqMjsKLAMajpPysYEQPVRK5Vnp
         hs2dm2n1L5tir5h6W+P274dXEn7rwIJizZKxDYODfRGX0L7qua6Ek9zz/2LESw9DWizy
         YchA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707829638; x=1708434438;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Sd18wvrol6QQ+0l525rSim8sbUKG0K0n74QLzBOifzc=;
        b=lsG3jIOA5GSoWIeNO3aZ18iF4swBwvCHmZJBYg/11s+7zoklLIQSHNwGaY7T4ETXHK
         rNcBNzxvzgG/5v4bFjKstpLMU4x1xiVyyocoyGX5ue6ZLxnuXKYLi0h/iND6y+UG0e1E
         BUsSfckPkqHGLbfE21U6axD7SamAA6qqaJ3uj7oiS18pNMTstrCtdulRGOWeO40u+P0v
         NIq+FVSEZgr69yfDybntdeFgu2eTMSQ/kNaiY2Jp2AE2pjyUmtZwSaHLhR+7wJ9dHNcJ
         uRWKO1FwckxC8LgZMnRa7IXdjNL908rVvdoHQMH/mPBFu+slequxhO/rFVdnm9kdmt79
         zD4w==
X-Forwarded-Encrypted: i=1; AJvYcCU/JqPXl8zVgpztjZNtEYuouu/7hs0uhtDMW5YvFTpFKyK7GKHYcXcWJnknM/iBvj9QGId/D/rfMqVet7DCcDI/L1jE0OErQ4A+/+bnqHM=
X-Gm-Message-State: AOJu0YyGqWVQbgiKYcaVQofoRnb+ZYyP3DD3FnzbohJd60XVeStGkL+7
	LRlwBuD2OT2suMQcJZFbg2y9FdU3Ph3qHicmiecPs0sCUb/u7f5KsWcUlrqL9Q==
X-Google-Smtp-Source: AGHT+IHF2bz+1gU3NCdTxSAzlHFdMJgOvqByXMvg8DYY9nFLuGACwqXJbFazgy7jqd+8HcXOWHM1SA==
X-Received: by 2002:a05:600c:5190:b0:411:a596:a2ac with SMTP id fa16-20020a05600c519000b00411a596a2acmr2379849wmb.1.1707829638397;
        Tue, 13 Feb 2024 05:07:18 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVLx5DMliR3ShmZwHBYzspQgEwhYnk2RMLP4zqsT8Rp8yC2DPH03nA71rHaOFMsqiuzLnkYeGE8xBySOa0LbbW7q2poCW+Bg2b7bLWpw0u2rnPRXCuk1yWpM8T5g1oTQxL0BnuDgJFGmL3dVatAyUxjXDYUe1ahc+VWWm+NqZchr84SXn9zzHkJPlyHPDVDjWNMAjdgDsRe1gfTrQuu8coAD8oY0RQlFY5aKz5FH6CZ/hODjK5d0/17kBXX4/ymUxrVMxgcbBE0jUx5z5hnidExQCvSGieNYy+Xvq26qfVnRLL/yHYb9aA=
Message-ID: <d6078dea-6bb4-447a-9d4e-5f223e3472b8@suse.com>
Date: Tue, 13 Feb 2024 14:07:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 22/30] xen/riscv: define an address of frame table
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <796853f19b883c474998ba97c1ee66cf224b1bde.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <796853f19b883c474998ba97c1ee66cf224b1bde.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> Also, the patch adds some helpful macros that assist in avoiding
> the redefinition of memory layout for each MMU mode.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Tue Feb 13 13:12:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 13:12:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679895.1057680 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZsa3-0001pZ-UU; Tue, 13 Feb 2024 13:12:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679895.1057680; Tue, 13 Feb 2024 13:12:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZsa3-0001pS-Px; Tue, 13 Feb 2024 13:12:11 +0000
Received: by outflank-mailman (input) for mailman id 679895;
 Tue, 13 Feb 2024 13:12:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZsa2-0001pI-DU; Tue, 13 Feb 2024 13:12:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZsa2-0007Ic-Ag; Tue, 13 Feb 2024 13:12:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZsa1-0002nu-Tn; Tue, 13 Feb 2024 13:12:10 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZsa1-0007V5-TR; Tue, 13 Feb 2024 13:12:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=FdE192zvb5HjU3ZX9eNxbUISApdqka50qBPp6p7GWSo=; b=r+YEdAs9gp/SYUpUOvKawWeR18
	a4PdMNyJqip/v2pVfaup8vvazMTuSblJpozsWeypbbUObjZ7aqwl5NPN8O0+yEdamZrD/fNRMdKJO
	TF8g33lhcjui70/m9k+CvmTERo74ZoqPZ3cHF7eomBrci2LtphG9EIG7KKHh6jb7rOqU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184657-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184657: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=a185f88e9858987e05d91eb3e9bda68d4a6b4f1d
X-Osstest-Versions-That:
    xen=772380f7547bac7e4c10c9ceff87af3f54151ae5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 13 Feb 2024 13:12:09 +0000

flight 184657 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184657/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  a185f88e9858987e05d91eb3e9bda68d4a6b4f1d
baseline version:
 xen                  772380f7547bac7e4c10c9ceff87af3f54151ae5

Last test of basis   184650  2024-02-12 09:02:17 Z    1 days
Testing same since   184657  2024-02-13 09:02:20 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  George Dunlap <george.dunlap@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Paul Durrant <paul@xen.org>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   772380f754..a185f88e98  a185f88e9858987e05d91eb3e9bda68d4a6b4f1d -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 13:33:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 13:33:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679903.1057688 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZsv0-00056n-K3; Tue, 13 Feb 2024 13:33:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679903.1057688; Tue, 13 Feb 2024 13:33:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZsv0-00056g-H8; Tue, 13 Feb 2024 13:33:50 +0000
Received: by outflank-mailman (input) for mailman id 679903;
 Tue, 13 Feb 2024 13:33:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZsuz-00056X-H8
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 13:33:49 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8302d66e-ca74-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 14:33:45 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-511898b6c9eso2613430e87.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 05:33:46 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 k9-20020a05600c1c8900b00410d7e55e5asm5979328wms.3.2024.02.13.05.33.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 05:33:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8302d66e-ca74-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707831226; x=1708436026; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vkTO5zSlzkJtpY0MPB6f8Repo9RXVis+7FLHryO9m30=;
        b=fLN8BB9MmuNYY4W640TLWe6hrPOhRn8KHjjN9tEUoQMopXGcLi/X7qxOAzzyIUZV/8
         QKGaNVM4Zp7UYtVdDjkDiLGI+IbICMb7usyoiWSJQMd8I51Yo6giEfq1BJsIqG/K0koL
         nSsWsqOVb7NSyf9VkSmR/fm1BDvfyKpdVVBs3/mKn1b/JbOtTKmVcjUkixAIFRP0+F1o
         8yQ2opB2q5sRyijMxyrJlzH9/fF84v0fPHvHgIHJZJx7dSR+pBAcE6BqKv14BAPcqw35
         n/YkzNEC7lWouLPVdT/bviW+kelf20bgSteg5rhVUnuGWcoat61WLmxy9f0YS7Txyglg
         Ymsg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707831226; x=1708436026;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vkTO5zSlzkJtpY0MPB6f8Repo9RXVis+7FLHryO9m30=;
        b=tB7WrU9NM4ZGrW6OrrRSHly8vQFnS+ugbOG5BnwTvzIUI4Oa5BmqkGUhwbZPertYEY
         nuBHqjwxMPcwL7QTgqElWDMxFWUibOKaRciT+ET+FvdYm6guTn2rKKW/gBY8NFeisR58
         1HZF5ngnYG0J1E6yYwZ/vFnYlg/qV6kkz52oDwaiBG2JUgYnBkmB67/3hjpY8oFP7XY/
         ifkwTKk/Mo8LJq/bCI6mq/KfxXyQUn1Mzfx6SVAqZHVtIxqdZJkv1iFEkmQAU0WafXtd
         GRUYuqVJH8NQ+NRSzJRbd9vPK+GKQQKQ2fCpMzzgROlUO+qOooiU5xQW1hOIoFM29vrl
         81XA==
X-Forwarded-Encrypted: i=1; AJvYcCUTfaAFPUcDcHM1DsTAzkbDoSZAwbAKLBNHcrk5zsPI+k7/l9mm60xH2H2BUuQndY77ShsRIrdfKqY7u95Xl+vXfq+dxYr9JFdPAgfSVX8=
X-Gm-Message-State: AOJu0Yz7bYy3UDkpqQSV2QBCi8o+yCl7YKvzi3yIcKD1ZnMSjXqQTM3m
	tcaAJwjTHZ9K9VYvzwjJhR8rTUIVm2aAvlBTAdIdaXsV4heBhFHs+6MBo5H6Dg==
X-Google-Smtp-Source: AGHT+IHcj3x6nvAL7HVr4bPP42Ixi1io8v+s4J0nTS7NJr1uE6k67EtoqOMHc/uhJlmhI382/1Qk+g==
X-Received: by 2002:a05:6512:5d5:b0:511:87b5:7ddb with SMTP id o21-20020a05651205d500b0051187b57ddbmr4862133lfo.37.1707831226053;
        Tue, 13 Feb 2024 05:33:46 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVd1A1yc0QgMvdpXaMOJvsnTVfODSzpxYia1m5rE8gIR46x6vRTXbXxIY+5kN2QS1bVAu+hRhgt/3r3PjSIe0wZo3oUPqp8ldnVl5gM2r8Yufv3PlJWkFGmhQ1sPwLwnfT1db04twQkEfUnGJVuzn40KmW+ZQqQbnpXaTn1Y64GYhdgiUeD9i1l9rYU94lR89Z/PYd/L6DzU4T04AtpGnk7jQaoPbFUvD/c80t+aQztkRUhDUZXvOwo7g/QSarbbFiZCsifkQ+caDvyXma8gh6EGhS6GXtsyWWZtpYc3UB3NtAibLn1gQ==
Message-ID: <6be5102a-624c-463a-9821-c618d110ce7a@suse.com>
Date: Tue, 13 Feb 2024 14:33:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 25/30] xen/riscv: add minimal stuff to processor.h to
 build full Xen
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <9efbc232f64b6192cf83f865b8987846fe082720.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <9efbc232f64b6192cf83f865b8987846fe082720.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/Kconfig
> +++ b/xen/arch/riscv/Kconfig
> @@ -45,6 +45,13 @@ config RISCV_ISA_C
>  
>  	  If unsure, say Y.
>  
> +config TOOLCHAIN_HAS_ZIHINTPAUSE
> +	bool
> +	default y

Shorter as "def_bool y".

> +	depends on !64BIT || $(cc-option,-mabi=lp64 -march=rv64ima_zihintpause)
> +	depends on !32BIT || $(cc-option,-mabi=ilp32 -march=rv32ima_zihintpause)

So for a reason I cannot really see -mabi= is indeed required here,
or else the compiler sees an issue with the D extension. But enabling
both M and A shouldn't really be needed in this check, as being
unrelated?

> +	depends on LLD_VERSION >= 150000 || LD_VERSION >= 23600

What's the linker dependency here? Depending on the answer I might further
ask why "TOOLCHAIN" when elsewhere we use CC_HAS_ or HAS_CC_ or HAS_AS_.

That said, you may or may not be aware that personally I'm against
encoding such in Kconfig, and my repeated attempts to get the respective
discussion unstuck have not led anywhere. Therefore if you keep this, I'll
be in trouble whether to actually ack the change as a whole.

> --- a/xen/arch/riscv/include/asm/processor.h
> +++ b/xen/arch/riscv/include/asm/processor.h
> @@ -12,6 +12,9 @@
>  
>  #ifndef __ASSEMBLY__
>  
> +/* TODO: need to be implemeted */
> +#define smp_processor_id() 0
> +
>  /* On stack VCPU state */
>  struct cpu_user_regs
>  {
> @@ -53,6 +56,26 @@ struct cpu_user_regs
>      unsigned long pregs;
>  };
>  
> +/* TODO: need to implement */
> +#define cpu_to_core(cpu)   (0)
> +#define cpu_to_socket(cpu) (0)
> +
> +static inline void cpu_relax(void)
> +{
> +#ifdef __riscv_zihintpause
> +    /*
> +     * Reduce instruction retirement.
> +     * This assumes the PC changes.
> +     */
> +    __asm__ __volatile__ ("pause");
> +#else
> +    /* Encoding of the pause instruction */
> +    __asm__ __volatile__ (".insn 0x100000F");
> +#endif

Like elsewhere, nit: Missing blanks immediately inside the parentheses.

> +    barrier();

It's probably okay to be separate, but I'd suggest folding this right
into the asm()-s.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 14:01:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 14:01:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679911.1057702 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZtLm-000183-Q7; Tue, 13 Feb 2024 14:01:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679911.1057702; Tue, 13 Feb 2024 14:01:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZtLm-00017w-Mx; Tue, 13 Feb 2024 14:01:30 +0000
Received: by outflank-mailman (input) for mailman id 679911;
 Tue, 13 Feb 2024 14:01:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZtLk-00017m-RZ; Tue, 13 Feb 2024 14:01:28 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZtLk-0008Ir-Ox; Tue, 13 Feb 2024 14:01:28 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZtLk-00040A-2o; Tue, 13 Feb 2024 14:01:28 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZtLk-0006Ng-2G; Tue, 13 Feb 2024 14:01:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=gC7qFur+PxTSh+wyfTl+PLjOWIp+cJzoYiGgdzaQzj8=; b=tzepTAeT3WtJkrwYNZ6GFtciuN
	WE6UTymqcNRoVJzYR88SIuseenC0l73z8ynzaeGG4UecOeZglAWX3n793pwaJ4YrGGxg8+fjE0ceR
	rCA53IXpNGs5mw8IswGLZr0l174ntDuCIWTOGtFp3JVYJiNB79nId5U7f+YHDHkEC0Zc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184654-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184654: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=c664e16bb1ba1c8cf1d7ecf3df5fd83bbb8ac15a
X-Osstest-Versions-That:
    linux=716f4aaa7b48a55c73d632d0657b35342b1fefd7
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 13 Feb 2024 14:01:28 +0000

flight 184654 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184654/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184652
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184652
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184652
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184652
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184652
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184652
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184652
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184652
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                c664e16bb1ba1c8cf1d7ecf3df5fd83bbb8ac15a
baseline version:
 linux                716f4aaa7b48a55c73d632d0657b35342b1fefd7

Last test of basis   184652  2024-02-12 17:43:54 Z    0 days
Testing same since   184654  2024-02-13 03:02:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jonathan Corbet <corbet@lwn.net>
  Linus Torvalds <torvalds@linux-foundation.org>
  Vegard Nossum <vegard.nossum@oracle.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   716f4aaa7b48..c664e16bb1ba  c664e16bb1ba1c8cf1d7ecf3df5fd83bbb8ac15a -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 14:19:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 14:19:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679918.1057712 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZtdN-0002zu-3J; Tue, 13 Feb 2024 14:19:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679918.1057712; Tue, 13 Feb 2024 14:19:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZtdN-0002zn-0h; Tue, 13 Feb 2024 14:19:41 +0000
Received: by outflank-mailman (input) for mailman id 679918;
 Tue, 13 Feb 2024 14:19:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZtdL-0002zh-Nk
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 14:19:39 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id eb8c4604-ca7a-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 15:19:38 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-411bfe83d11so4724155e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 06:19:38 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z17-20020a05600c221100b00411d1ce4f9dsm458180wml.34.2024.02.13.06.19.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 06:19:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb8c4604-ca7a-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707833978; x=1708438778; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=q28WFJt1gUUX76ctbVk+6aRIcLNdBp2Id4KQAL34lb0=;
        b=OkCzWZQkcrmIEDF2nmWM85Wqa2XWnIUOqV+B4X2gsZpOjL6R0qo7VqkvvwkAdjttmj
         rFJJo6PSId13geWSktVeMZ6jMQlFGf2bG3Z6WJpePF2MkWoGa6/OrJDKGjTvn7yABDNm
         YfT+0GI2dQFhn1wnX6YqEgSePwueO7qu5aZdvxDiidV3mE5ONLtEGG5UMBkYqsHaNEhM
         eeNs/LRzYiSECsPoAiPR+aLBDqLD306p1KPLfrI+tkCsKTJLjBRzPYYb8aE+p3U5CDgz
         JirD3jPyEWpbtbvTG6OovkiEyqD2BF4/3cfg3vtAHLCkiwtsXbvg3L6jExvxV+WgM7VD
         4mOw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707833978; x=1708438778;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=q28WFJt1gUUX76ctbVk+6aRIcLNdBp2Id4KQAL34lb0=;
        b=sl/4oV+wFYxMgR/BYFLPxPeaR7zseu1OEav76AHkkQKBikGwo0zA2Q2RN8ApR6PVjR
         lO1GvtxDRYLMhCNhxG0U4GFY/7X5CoGyVsTNgjBeHYXtYHb8dq8S0Ycq0nvLxKPoPa1z
         dpkH0fyGnkonarl46zoy/24rbkFcoXuenf3mW4oT/yTg1yurvKb9/4xZexRfsAfISKIC
         zriSVME7AAVFnw5WfMsjfJcbYTbRi04engaDhN1KeScYmjCpRT5N3Y2VyFBvbas2XZPD
         tn7UhPQCyCjCy7n1FhwwnvJlrcefv+9T5M81VJWGxsmFDLHrUuG/QP5KzYc2iJ7sqswu
         MsdQ==
X-Forwarded-Encrypted: i=1; AJvYcCVVoySzY4xY3aH0D2B7gAqAeyPa4HMcdWXtlI3Ci2lTvsO/kIMVcLRRw18OXBPEV408/WU/G33WzNoKASKIZEYLGTjMLilJSFGWZJ96Ouw=
X-Gm-Message-State: AOJu0YydIv1sBmQEr9r3e8/xdG0YmrSx7uu5LRFh4wtJec+DltfWZrtn
	+hywEs6bC7HwY4uJSA7aZEIUOI6P0EwqqwvAEopJPms5rAJedIWyibSbsFOa+w==
X-Google-Smtp-Source: AGHT+IFwl/QOPMjPGYr06ECinbuGCEvvReSAXMzYp8VgkZtWMRcMs1iWMZapu42zQ8mqirAN/05VDQ==
X-Received: by 2002:a05:600c:470d:b0:40e:dbdf:9fb4 with SMTP id v13-20020a05600c470d00b0040edbdf9fb4mr8920640wmo.23.1707833977687;
        Tue, 13 Feb 2024 06:19:37 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW47c1Wmyuw0FQ5fzOVt2h8RAZe5yaTYEF+6Hg5ijmoMZG1SO0EVsbVWS59gQBJCq0PBOHNAZCj1l8Mab0OiwdNBG9vaq3WuTzDBs1p5YGw1MOtfGJ1GmyyqLe/7z+aFjPipNeXF03RuWTvB2hVfHxm8sYA4iNrYv84NURVbBIY0FwStHOe5t577ZC+ev4pd8gLoQyO4cFm63ezcCXGUDtC5t8P0+1KSPZZyKgFJxSw3Yhc0vEIzjxUfeZ2d4+P4HQl2Tr5kh+cY6YyFmpZEGmo6mKcTZ9on8wpTOHnZS25zZXSd9T7y1s=
Message-ID: <6f5e7e21-067e-4323-88bc-f9b5975e95c5@suse.com>
Date: Tue, 13 Feb 2024 15:19:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 26/30] xen/riscv: add minimal stuff to mm.h to build
 full Xen
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <369fe9e80d11c5e499810a6819fc1c282ce60628.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <369fe9e80d11c5e499810a6819fc1c282ce60628.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes in V4:
>  - update an argument name of PFN_ORDERN macros.
>  - drop pad at the end of 'struct page_info'.
>  - Change message -> subject in "Changes in V3"
>  - delete duplicated macros from riscv/mm.h
>  - fix identation in struct page_info

I'm sorry, but how does this fit ...

> +struct page_info
> +{
> +    /* Each frame can be threaded onto a doubly-linked list. */
> +    struct page_list_entry list;
> +
> +    /* Reference count and various PGC_xxx flags and fields. */
> +    unsigned long count_info;
> +
> +    /* Context-dependent fields follow... */
> +    union {
> +        /* Page is in use: ((count_info & PGC_count_mask) != 0). */
> +        struct {
> +            /* Type reference count and various PGT_xxx flags and fields. */
> +            unsigned long type_info;
> +        } inuse;
> +        /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
> +        union {
> +            struct {
> +                /*
> +                 * Index of the first *possibly* unscrubbed page in the buddy.
> +                 * One more bit than maximum possible order to accommodate
> +                 * INVALID_DIRTY_IDX.
> +                 */
> +#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1)
> +                unsigned long first_dirty:MAX_ORDER + 1;
> +
> +                /* Do TLBs need flushing for safety before next page use? */
> +                bool need_tlbflush:1;
> +
> +#define BUDDY_NOT_SCRUBBING    0
> +#define BUDDY_SCRUBBING        1
> +#define BUDDY_SCRUB_ABORT      2
> +                unsigned long scrub_state:2;
> +            };
> +
> +                unsigned long val;

... this?

> +        } free;
> +    } u;
> +
> +    union {
> +        /* Page is in use, but not as a shadow. */

I'm also pretty sure I asked before what shadow this comment alludes to.

> +        struct {
> +            /* Owner of this page (zero if page is anonymous). */
> +            struct domain *domain;

Seeing this is a pointer, I find "zero" in the comment a little
misleading. Better say NULL?

> +        } inuse;
> +
> +        /* Page is on a free list. */
> +        struct {
> +            /* Order-size of the free chunk this page is the head of. */
> +            unsigned int order;
> +        } free;
> +    } v;
> +
> +    union {
> +        /*
> +         * Timestamp from 'TLB clock', used to avoid extra safety flushes.
> +         * Only valid for: a) free pages, and b) pages with zero type count
> +         */
> +        uint32_t tlbflush_timestamp;
> +    };
> +};
> +
> +#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
> +
> +/* PDX of the first page in the frame table. */
> +extern unsigned long frametable_base_pdx;
> +
> +/* Convert between machine frame numbers and page-info structures. */
> +#define mfn_to_page(mfn)                                            \
> +    (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
> +#define page_to_mfn(pg)                                             \
> +    pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)
> +
> +static inline void *page_to_virt(const struct page_info *pg)
> +{
> +    return mfn_to_virt(mfn_x(page_to_mfn(pg)));
> +}

Would be nice if this and the inverse function would live closer to
one another.

> +/*
> + * Common code requires get_page_type and put_page_type.
> + * We don't care about typecounts so we just do the minimum to make it
> + * happy.
> + */
> +static inline int get_page_type(struct page_info *page, unsigned long type)
> +{
> +    return 1;
> +}
> +
> +static inline void put_page_type(struct page_info *page)
> +{
> +}
> +
> +static inline void put_page_and_type(struct page_info *page)
> +{
> +    put_page_type(page);
> +    put_page(page);
> +}
> +
> +/*
> + * RISC-V does not have an M2P, but common code expects a handful of
> + * M2P-related defines and functions. Provide dummy versions of these.
> + */
> +#define INVALID_M2P_ENTRY        (~0UL)
> +#define SHARED_M2P_ENTRY         (~0UL - 1UL)
> +#define SHARED_M2P(_e)           ((_e) == SHARED_M2P_ENTRY)
> +
> +#define set_gpfn_from_mfn(mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0)
> +#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
> +
> +#define PDX_GROUP_SHIFT (16 + 5)

Where are these magic numbers coming from? None of the other three
architectures use literal numbers here, thus making clear what
values are actually meant. If you can't use suitable constants,
please add a comment.

> +static inline unsigned long domain_get_maximum_gpfn(struct domain *d)
> +{
> +    BUG_ON("unimplemented");
> +    return 0;
> +}
> +
> +static inline long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
> +{
> +    BUG_ON("unimplemented");
> +    return 0;
> +}
> +
> +/*
> + * On RISCV, all the RAM is currently direct mapped in Xen.
> + * Hence return always true.
> + */
> +static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
> +{
> +    return true;
> +}
> +
> +#define PG_shift(idx)   (BITS_PER_LONG - (idx))
> +#define PG_mask(x, idx) (x ## UL << PG_shift(idx))
> +
> +#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
> +#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
> +#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
> +
> + /* Count of uses of this frame as its current type. */

Imo the PGC_ related revision log item should have covered this one, too.

> +#define PGT_count_width   PG_shift(2)
> +#define PGT_count_mask    ((1UL << PGT_count_width) - 1)
> +
> +/*
> + * Page needs to be scrubbed. Since this bit can only be set on a page that is
> + * free (i.e. in PGC_state_free) we can reuse PGC_allocated bit.
> + */
> +#define _PGC_need_scrub   _PGC_allocated
> +#define PGC_need_scrub    PGC_allocated
> +
> +/* Cleared when the owning guest 'frees' this page. */
> +#define _PGC_allocated    PG_shift(1)
> +#define PGC_allocated     PG_mask(1, 1)
> +/* Page is Xen heap? */
> +#define _PGC_xen_heap     PG_shift(2)
> +#define PGC_xen_heap      PG_mask(1, 2)
> +/* Page is broken? */
> +#define _PGC_broken       PG_shift(7)
> +#define PGC_broken        PG_mask(1, 7)
> +/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */
> +#define PGC_state         PG_mask(3, 9)
> +#define PGC_state_inuse   PG_mask(0, 9)
> +#define PGC_state_offlining PG_mask(1, 9)
> +#define PGC_state_offlined PG_mask(2, 9)
> +#define PGC_state_free    PG_mask(3, 9)
> +#define page_state_is(pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st)
> +
> +/* Count of references to this frame. */
> +#define PGC_count_width   PG_shift(9)
> +#define PGC_count_mask    ((1UL << PGC_count_width) - 1)
> +
> +#define _PGC_extra        PG_shift(10)
> +#define PGC_extra         PG_mask(1, 10)
> +
> +#define is_xen_heap_page(page) ((page)->count_info & PGC_xen_heap)
> +#define is_xen_heap_mfn(mfn) \
> +    (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn)))
> +
> +#define is_xen_fixed_mfn(mfn)                                   \
> +    ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) &&   \
> +     (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1)))
> +
> +#define page_get_owner(_p)    (_p)->v.inuse.domain
> +#define page_set_owner(_p,_d) ((_p)->v.inuse.domain = (_d))

Unnecessary (leading) underscores again.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 14:34:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 14:34:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679923.1057721 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZtrT-0005o6-EF; Tue, 13 Feb 2024 14:34:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679923.1057721; Tue, 13 Feb 2024 14:34:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZtrT-0005nz-BD; Tue, 13 Feb 2024 14:34:15 +0000
Received: by outflank-mailman (input) for mailman id 679923;
 Tue, 13 Feb 2024 14:34:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SqM6=JW=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1rZtrS-0005nn-2j
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 14:34:14 +0000
Received: from mail-ej1-x643.google.com (mail-ej1-x643.google.com
 [2a00:1450:4864:20::643])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f5008d04-ca7c-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 15:34:13 +0100 (CET)
Received: by mail-ej1-x643.google.com with SMTP id
 a640c23a62f3a-a3d2587116aso17470466b.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 06:34:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5008d04-ca7c-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707834852; x=1708439652; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=m/tLFueGTPOjDp5fpr95SEip2U5yJw1eI2KrdoB48zQ=;
        b=fA4T3L0WQlI1HdjZRsbFrTjbEhgA71e3t5XF2hSnf/XkBf1AccT56XNF9yAmviCO1i
         7E2i0mAiDJIwvxB3rMq3meVRhy41dGrXaIywPYsoYYjeGNLquE9+6RWBd6bzSdhdW4N3
         BXkO7mugdEInfnepgB8Z71Durk004H/uMh+Ag=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707834852; x=1708439652;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=m/tLFueGTPOjDp5fpr95SEip2U5yJw1eI2KrdoB48zQ=;
        b=luYEDfB0nvpUAgCyPujsLM4CJZwpA2P5XfRboHuMnJ5o6j2MBmDyBBnhJkg/sgbLFH
         Y0nMt/5zj0RK4qn9o027dSLetJHfLx3N+s298weig3GM0r7IVqp204lUBQI2i09t6hxr
         RGEH1kgN6A4QnGaNgC4k1gcRxhhtn0/Y5mR4Ki//6XNolgQQD8qPFHIJeFdD7hjZs7ye
         LhwNsqb0W3RLtz7WWkdq0FnHwZavu75oc9kW6mPT+SdaHiwTOo7bC84GntheBJ8LrqzM
         qpuH4hA3U8JTyPG9/1FVIbOuLLjbTcetyd7dG21ghaCJRRLDMBw/uJBNLK1p1uyFt6UC
         kD9g==
X-Gm-Message-State: AOJu0YwOHIvO09OKH5U29tJgXoNEd+AX/n43TE1RZ7lC3bvXCMvI/5SS
	N2atOhF1So5hqjf30Waodeg/LbX1UFCV/NzHKJw330o4hSYVuS0wiskKhEM6nY5T4CMzIFWsr/E
	M80eHgjdoYb4IVzV3iqq327zSZuyUNTNvNz+xtwQZRraCqbPwAIKnXb4q
X-Google-Smtp-Source: AGHT+IFIdg7CBNnXtDrwZKaVLcqInhj5HPnWpbNTL4wO9GUhQFOTKUh0XTL6HeglRrdaaFv6cjdEC9qPp3Ad08b372k=
X-Received: by 2002:a17:906:40ca:b0:a35:e7a1:66ec with SMTP id
 a10-20020a17090640ca00b00a35e7a166ecmr6496969ejk.44.1707834852247; Tue, 13
 Feb 2024 06:34:12 -0800 (PST)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Tue, 13 Feb 2024 14:33:36 +0000
Message-ID: <CAO-mL=wFR=jz7sJdBsqVxesg9VQ4mZ4OeHcV-TdtB6mBP81T4Q@mail.gmail.com>
Subject: Xen Summit 2024 - Registrations now open!
To: xen-devel@lists.xenproject.org, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000b4351c0611444966"

--000000000000b4351c0611444966
Content-Type: text/plain; charset="UTF-8"

Hello Xen Community,

Our Xen Summit 2024 registrations are now open! Secure your early bird
rates today.
https://events.linuxfoundation.org/xen-project-summit/register/

If you're an academic studying for a relevant degree, entry is free!
(subject to a social media or blog post after the event)

Please make sure to submit your talks early, as CFPs will close at the end
of the month.

If you have any questions, please let me know or email
cfp@linuxfoundation.org

Many thanks,
Kelly Choi

Community Manager
Xen Project

--000000000000b4351c0611444966
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hello Xen Community,=C2=A0<div><br></div><div>Our Xen Summ=
it 2024 registrations are now open! Secure your early bird rates today.=C2=
=A0</div><div><a href=3D"https://events.linuxfoundation.org/xen-project-sum=
mit/register/">https://events.linuxfoundation.org/xen-project-summit/regist=
er/</a><br></div><div><br></div><div>If you&#39;re an academic studying for=
 a relevant degree, entry is free!=C2=A0=C2=A0</div><div>(subject to a soci=
al media or blog post after the event)</div><div><br></div><div>Please make=
 sure to submit your talks early, as CFPs will close at the end of the mont=
h.=C2=A0</div><div><br></div><div>If you have=C2=A0any questions, please=C2=
=A0let me know or email=C2=A0<a href=3D"mailto:cfp@linuxfoundation.org">cfp=
@linuxfoundation.org</a></div><div><br></div><div>Many thanks,<div><div dir=
=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div =
dir=3D"ltr"><div>Kelly Choi</div><div><br></div><div><div style=3D"color:rg=
b(136,136,136)">Community Manager</div><div style=3D"color:rgb(136,136,136)=
">Xen Project=C2=A0<br></div></div></div></div></div></div></div>

--000000000000b4351c0611444966--


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 14:53:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 14:53:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679961.1057747 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZu9o-0001gc-DL; Tue, 13 Feb 2024 14:53:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679961.1057747; Tue, 13 Feb 2024 14:53:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZu9o-0001gV-An; Tue, 13 Feb 2024 14:53:12 +0000
Received: by outflank-mailman (input) for mailman id 679961;
 Tue, 13 Feb 2024 14:53:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zAib=JW=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1rZu9m-0001gP-NT
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 14:53:10 +0000
Received: from fout4-smtp.messagingengine.com (fout4-smtp.messagingengine.com
 [103.168.172.147]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 988242ac-ca7f-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 15:53:07 +0100 (CET)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfout.nyi.internal (Postfix) with ESMTP id E468F13800B5;
 Tue, 13 Feb 2024 09:53:05 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Tue, 13 Feb 2024 09:53:05 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 13 Feb 2024 09:53:04 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 988242ac-ca7f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1707835985;
	 x=1707922385; bh=DFfeuBoVBxksTdKSx0VpgsUDX+NEgO6od69uHWLszc0=; b=
	dcz9yctKQC0vh9OoSyvra9SoskehQuDACtmPWzo6Iury+EMkGFgqOPT8tm3hN7Sc
	rmOmnCWR5ONkEXRdRSJbwdRR1Lq+416lLvrvphTYLQElaqqzgEod/lGRDPjwlk3B
	lfKpF+BD7L1PT3Sx4zPvibiY3eogxuTvgeA0nUlPLeyaidoKw8FJE/nUldE+uFfD
	8M5fVF0nn+5CCfCv41nwawp2+fsCd2z7aHW0nMq0LIaovzIBkUoLyqcxK/y6ebvS
	R+MAkIUpxFH4YdrihzRUHfkFowNUTCQdnGdyLgOz6RglgFU+7MdmZE8HY3YVvhnf
	O9f+yWFFyeMFN3AOLxtzEw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1707835985; x=1707922385; bh=DFfeuBoVBxksTdKSx0VpgsUDX+NE
	gO6od69uHWLszc0=; b=vEpqm2BMOT/Dz/ZyFO5EAwtLXtwPBgt5+MN+X/bKQ+IS
	GxjY8A9m9U3rczWo9wJqSYIGnuZVjMP2QllmRZJEJZGr2u44E5sfsOklhpWsflNg
	3KVABOa3qCZsWaBaS+B2CrTG0Jcm5NQHwQO6jmsc4eMyjEfz+pIk8TWwQEnNKxxs
	4n6wItv4EEBjwGYwLPCYfqxWRQd7J0jpTtOUXDVpvMqX+6lsXXg2yCC+Ng4phx4n
	DcjipDz+vi23AVpfW+ps+iUyfQ8AH6X0lR3P/z303vIEy1bycj7HSOnFJ62UGoLb
	xAwfI3GYXilDDovNYVnAj8EgF3ZQqziO66uOBRQM7Q==
X-ME-Sender: <xms:UYLLZWJAFdVZEw8RJqE--gSe47LTjlTL0RAzT8fXLsMqU6g4ex8BhQ>
    <xme:UYLLZeKRMnCvrS6cguEGGqBztDraVZ6qy6hzjmMuaS1BeQQG9ogUhjObcsuW5ldfP
    dABrPFof4DL8Q>
X-ME-Received: <xmr:UYLLZWuBUaRqfFhbTi-Gwvh1h9DSh84oerHIUqMfmbs_PLEMakTP-PvEbpc6b5jHYMjbTd9OJ35mqGIwU9iIpTKK1b6NZSJcdw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehgdeilecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpeffhffvvefukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihcuoehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthh
    hinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeeihfelhfekhefgveejjeet
    vedtgfduveelgeekteehueeufeefudegveejueefhfenucffohhmrghinhepghhithhhuh
    gsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho
    mhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:UYLLZbaogY7ggXl2BOfwWfo0ershlv-HFr79opgr4HEn8YjB5ANykg>
    <xmx:UYLLZdYd1ML_VZam_rCLaCxsGjq66F_0pVPDt-lbN95PqfFnUEC-AA>
    <xmx:UYLLZXCUTR3UKq6Ov22mKdFsCmBAGE2drAEDwCeazCF3ZW1MLDAn0g>
    <xmx:UYLLZSOCu9y3ZerXbEghDFBdbNL1NtCBCGtrrwQNt7cnUq5eSS6z-w>
Feedback-ID: i1568416f:Fastmail
Date: Tue, 13 Feb 2024 15:53:02 +0100
From: Marek Marczykowski <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Message-ID: <ZcuCTnMpFib4ArHW@mail-itl>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
 <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
 <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com>
 <ZcrlcuNU9y8WymiK@mail-itl>
 <817ceec4-ed12-4360-a6ce-f87732f62251@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="mt9SbzcEVrn24iEJ"
Content-Disposition: inline
In-Reply-To: <817ceec4-ed12-4360-a6ce-f87732f62251@suse.com>


--mt9SbzcEVrn24iEJ
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 13 Feb 2024 15:53:02 +0100
From: Marek Marczykowski <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers

On Tue, Feb 13, 2024 at 08:45:54AM +0100, Jan Beulich wrote:
> On 13.02.2024 04:43, Marek Marczykowski wrote:
> > On Mon, Feb 12, 2024 at 10:04:38AM +0100, Jan Beulich wrote:
> >> On 08.02.2024 23:00, Julien Grall wrote:
> >>> On 05/02/2024 13:27, Jan Beulich wrote:
> >>>> In preparation of dropping the register parameters from
> >>>> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
> >>>> register state needs making available another way for the few key
> >>>> handlers which need it. Fake IRQ-like state.
> >>>>
> >>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >>>> ---
> >>>> The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsistent =
with
> >>>> other console poll functions we have, and it's unclear whether that's
> >>>> actually generally correct.
> >>>
> >>> Is it? Looking at ns16550_poll() we would pass guest_user_regs() if=
=20
> >>> run_in_exception() doesn't exist. But looking at the caller, no-on se=
ems=20
> >>> to care about the 'regs'. So is this just a latent bug?
> >>
> >> What do you mean by "doesn't exist"? ns16550_poll() assumes it exists.
> >> And I can spot any use of guest_user_regs() on the respective generic
> >> or Arm-specific bug.c paths.
> >>
> >>> BTW, do you have an idea why the poll function is not run in an=20
> >>> exception handler?
> >>
> >> "The poll function" being which one? If you mean the one in xhci-dbc.c
> >> then that's why I had Cc-ed Marek. Moving him to To: - maybe that
> >> manages to finally catch his attention.
> >=20
> > TBH, I don't know. That's part of the original xue patch at
> > https://github.com/connojd/xue/blob/master/patches/xen-xue-dbgp.patch
> > and it works for me as it is.
>=20
> "Works" meaning what? Doesn't crash on you? Or does also provide
> sensible output in _all_ cases (i.e. including when e.g. the poll
> happens to run on an idle vCPU)?

Generally provides sensible output, for example during boot (it is using
idle vCPU then, right?).

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--mt9SbzcEVrn24iEJ
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXLgk4ACgkQ24/THMrX
1yxYqQf/QZCLPTxbKmXZI+CXVOH0exzR0oHacrk7Sb7hN5FhSBcbMsCiQ5HZ4IdF
eumV1VDdDKw0piFz5vnLk/X++4j9N37ySeyE26jUpzp4aB0JHzBqU+ItedpinVGe
MauH1EdhqsBJ+kKUVP5FTUQCbaLLowdsTzHl2fP07/xOTRNWXyhxiMJSLrrbuCkN
ul2kKAmkTiRCQt3pGcYo9kSEMieQOckHwhZ5TokbzlTYTEJ40Ehp15lSQmDX04an
ILN2Z6DtpimhQyUgnia6+/wgzrLXYVjkrtdYpnbe2wG4trf+Sqpv8OYlDFgiUjJN
HtDj59AJbaCgFGJvODIHlcHC3On/Aw==
=fsI6
-----END PGP SIGNATURE-----

--mt9SbzcEVrn24iEJ--


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 14:56:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 14:56:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679963.1057758 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZuCj-0002FP-RL; Tue, 13 Feb 2024 14:56:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679963.1057758; Tue, 13 Feb 2024 14:56:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZuCj-0002FI-O7; Tue, 13 Feb 2024 14:56:13 +0000
Received: by outflank-mailman (input) for mailman id 679963;
 Tue, 13 Feb 2024 14:56:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+jOn=JW=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1rZuCi-0002FA-6D
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 14:56:12 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 06add267-ca80-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 15:56:11 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-411d5dd7924so772675e9.2
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 06:56:11 -0800 (PST)
Received: from lab.home
 (dynamic-2a00-1028-83a4-4bca-c0bb-96ff-feed-9d50.ipv6.o2.cz.
 [2a00:1028:83a4:4bca:c0bb:96ff:feed:9d50])
 by smtp.gmail.com with ESMTPSA id
 p7-20020a05600c468700b004105528c61fsm12158012wmo.35.2024.02.13.06.56.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 06:56:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06add267-ca80-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707836170; x=1708440970; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=YDTkZB2YzDAxv60jxW/Vmfom7fptyEEc7QTL/Z9bmnA=;
        b=JhLOVmhF36zlo0u/ZASQ0UkArLuCZimJyh5Rrlvn4ZqC7dTLYPyIOcwEhflag0tC+3
         7adUKMjLjHzERdbhzb/AGUDAP0gAD7arUDQxpnc7hyMcArYh+Ndx26eHdX336e+5cd6J
         zHgnmSDssR5Shz1HimzjXdS/4LksyllqQm7RQeP24Quht+phSwcozldX82PIWuWeedvF
         WnwJ2AakPm++qGlUMYEzvR6nJD4fnOoZIp+ALjV4SCwHvavAs0b+6zuKZ9VltxMf6G7J
         JX2K4ZEWMfk8Phk9n6NQ997aG0eHEGNQN6tne3L6RGrO3cdfTrWJVuqznfRZSQQYcsmp
         DN/A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707836170; x=1708440970;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=YDTkZB2YzDAxv60jxW/Vmfom7fptyEEc7QTL/Z9bmnA=;
        b=dPWND9szvBFVcMDcmLm3sJ2Fg8HKSF/gcPi0KXES3aPWDjF8NDr0kc3Trd7zrPFvbn
         cjH5FeePZLGP9p0+cFAS8tli+RIcBIBl8vJAZevRJIXxAMfW8ESrgBlRAxjLbSnccCkQ
         feeJ55UXLQSi3iXSKLEe+OQY3bDg00jEgABjSCb1SjJQw4o0Uqj7wVT8QzQrBiUNjasH
         ZnkIT7anzZRZlDNipeThRy+QHzBTNoK6rd5rcDeV1ZCfZRaX/QtDZk83VbwHA17w6Sb/
         Rmz84piQY7q5iFZ92lLiy/r1/Ksls9V7dK/HuilsMKyavcPJdHgBmLqYisalXuVasT9D
         /7ew==
X-Gm-Message-State: AOJu0YydlVY5MwrWCifZ6W28S/69alvsg7ywKFict3UHqn11aSRh0TmX
	e/Gg2WBR/5avvy3TDtiXBmupXGo9kESu+1SeDiSoDry3mvF6iA/bLWkqwDL3uDs=
X-Google-Smtp-Source: AGHT+IFoxvp/vvZFIi4rUaYqQSSRHJrvDLAu1xi6CHM/HjrhE2cuWAeP9FSkLEHBWtXnV7/qItXAfA==
X-Received: by 2002:a05:600c:46c9:b0:410:7980:72a3 with SMTP id q9-20020a05600c46c900b00410798072a3mr7612274wmo.35.1707836170269;
        Tue, 13 Feb 2024 06:56:10 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUf9ykDLhf1RWA3IqzPhQH2cYKl+8TF1tr1iCOim9kOsMnRpE8ss1yhdVWEv7KlRQzXHyHWI/qIzi+wGkSbNiLYTPKmSq6FvZcZ+U1MpkXwqQbzDxOZbxs9s3+VFS62OIBTDu9DJT4=
From: "=?UTF-8?q?Petr=20Bene=C5=A1?=" <w1benny@gmail.com>
X-Google-Original-From: =?UTF-8?q?Petr=20Bene=C5=A1?= <petr.benes@gendigital.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Petr=20Bene=C5=A1?= <w1benny@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2] libxl: Fix comment for LIBXL_HAVE_VMTRACE_BUF_KB
Date: Tue, 13 Feb 2024 14:56:07 +0000
Message-Id: <4ced67935b2e1a2fc79ccda7624c0849bea6cd87.1707836102.git.w1benny@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Petr Beneš <w1benny@gmail.com>

It's located in libxl_domain_build_info, not libxl_domain_create_info.

Signed-off-by: Petr Beneš <w1benny@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/include/libxl.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index f1652b1664..46bc774126 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -519,7 +519,7 @@
 #define LIBXL_HAVE_PHYSINFO_CAP_VMTRACE 1
 
 /*
- * LIBXL_HAVE_VMTRACE_BUF_KB indicates that libxl_domain_create_info has a
+ * LIBXL_HAVE_VMTRACE_BUF_KB indicates that libxl_domain_build_info has a
  * vmtrace_buf_kb parameter, which allows to enable pre-allocation of
  * processor tracing buffers of given size.
  */
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:00:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:00:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679993.1057780 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZuGz-0004EK-Eu; Tue, 13 Feb 2024 15:00:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679993.1057780; Tue, 13 Feb 2024 15:00:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZuGz-0004ED-Bk; Tue, 13 Feb 2024 15:00:37 +0000
Received: by outflank-mailman (input) for mailman id 679993;
 Tue, 13 Feb 2024 15:00:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZuGx-0004E6-Lb
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:00:35 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a3a09786-ca80-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 16:00:34 +0100 (CET)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2d0e5212559so39193291fa.0
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 07:00:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 jr6-20020a05600c560600b0041079d336c7sm11993364wmb.39.2024.02.13.07.00.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 07:00:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3a09786-ca80-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707836434; x=1708441234; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=qp8gYD7O+QLqEXZHQVoRHvrqAvzQPcT/HD2ZQ9g+l3k=;
        b=TRSUqaB8id5dH3oRFaSzo8n9FHFlJDM8DrZALtOfjPXwj5h/CF6pUxg2v6PShgTsPz
         RGROIi8Ewrhqfq3TWn6isltxupK1fcFyOH0mngAc1Vdg97hCzDnlw+Ij3aj2wZ93PYLX
         uL0qWAvaatP9QBht5+CX6IeqZB2xKrssp3bOd/dv8zpOnc9tLpCWZgOs07CZdcHxwK0s
         GfgGzy1LYhgJ0OMFEYDjSEVEJ2y4GD+25bc13Sj3VkEtChN1UYQmltDSOGoSqXyH6iPV
         FNZ8urMrcVbOpIEX9r1L8yiMK3af6HFflsyrkJ3rBwtuh34U2pvl7z3dl8lu1yeEymro
         Y2tA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707836434; x=1708441234;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qp8gYD7O+QLqEXZHQVoRHvrqAvzQPcT/HD2ZQ9g+l3k=;
        b=iHleIL5bvfpjJBJI3/ytB9GVxZBQuEu/hyvnDz0FpsaFSXGNMylLoJT3djsXTJBZ5T
         OL+TVSVNxkSHVnDKyz6ao6LRz1NGi+skeRc1xKGCC+mdqTEDTEbL0Xf/9WZpk4Fg2ANu
         FiTMZm0BHgG9yvXDE2YfqiKGmuErLlwwMdoMgxrDCtY5sbV28NzluLXrcXkTzNq3lgjc
         +sEmWGE832SjQPgxUXZsakoRIdZg9E7HsnGBFv+wQfamWHXsq/c6pu7YhOPE+kVEWQ14
         PjPWOoxXF8aH3mp7s2cRLCOL5vZclpL6NA2RNjdscSa//gRRI4B8Y1qVDRcghS0Tepo3
         /lVA==
X-Forwarded-Encrypted: i=1; AJvYcCW8M2sboR7yofMd294cb325mvZwU1Vu7wt4JNQXQkAtdBYCQ9FABli/75fJsqgkmAy8u/txv/Fn6jPf0u6CUczTdLLynWAkxKmR4zjQR8s=
X-Gm-Message-State: AOJu0YwoUaQ8ehUN3KqvxS19jUA0xjqTMYln0I5sROinwMjlOGCovy9Q
	ZKmKkpM7uv6zHAVEqYHd6Ym8ei2ab3ZL1RXAVkAtdnXL51JWU9JrecVRPgvpig==
X-Google-Smtp-Source: AGHT+IEW0LRyWOugDwmxCAMt1Vz9zfO26lIQSxLT91xCk/TnHmMnxlofgeWTn0TnA2NlTiM4vFTOdw==
X-Received: by 2002:a2e:9216:0:b0:2d0:b70a:a947 with SMTP id k22-20020a2e9216000000b002d0b70aa947mr6130487ljg.23.1707836434072;
        Tue, 13 Feb 2024 07:00:34 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWSM46mWivw5EBZMc0j2OqjvbHCh3R7v4fArnJrmG1gHZeqPhxYfyM7PcDOjHBHENw/Le+WQochRwy6bQFJ1E0z6h26QaUcOI8y4dTUV6LHytPuEu8G09X0K/5VDTbQ0rMv/Rf66/AdjAkcdbyPdxhoZei2WkeYB72+pkvSox71rq43IIzZfL50fZBAsqRn6Y4C0UGM3DJv+ciu7Ma0fWImxDtGozP3Hw==
Message-ID: <b52fa3fa-4e33-4211-af18-f00aaa0b365b@suse.com>
Date: Tue, 13 Feb 2024 16:00:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Content-Language: en-US
To: Marek Marczykowski <marmarek@invisiblethingslab.com>
Cc: Julien Grall <julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
 <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
 <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com> <ZcrlcuNU9y8WymiK@mail-itl>
 <817ceec4-ed12-4360-a6ce-f87732f62251@suse.com> <ZcuCTnMpFib4ArHW@mail-itl>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcuCTnMpFib4ArHW@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.02.2024 15:53, Marek Marczykowski wrote:
> On Tue, Feb 13, 2024 at 08:45:54AM +0100, Jan Beulich wrote:
>> On 13.02.2024 04:43, Marek Marczykowski wrote:
>>> On Mon, Feb 12, 2024 at 10:04:38AM +0100, Jan Beulich wrote:
>>>> On 08.02.2024 23:00, Julien Grall wrote:
>>>>> On 05/02/2024 13:27, Jan Beulich wrote:
>>>>>> In preparation of dropping the register parameters from
>>>>>> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
>>>>>> register state needs making available another way for the few key
>>>>>> handlers which need it. Fake IRQ-like state.
>>>>>>
>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>> ---
>>>>>> The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsistent with
>>>>>> other console poll functions we have, and it's unclear whether that's
>>>>>> actually generally correct.
>>>>>
>>>>> Is it? Looking at ns16550_poll() we would pass guest_user_regs() if 
>>>>> run_in_exception() doesn't exist. But looking at the caller, no-on seems 
>>>>> to care about the 'regs'. So is this just a latent bug?
>>>>
>>>> What do you mean by "doesn't exist"? ns16550_poll() assumes it exists.
>>>> And I can spot any use of guest_user_regs() on the respective generic
>>>> or Arm-specific bug.c paths.
>>>>
>>>>> BTW, do you have an idea why the poll function is not run in an 
>>>>> exception handler?
>>>>
>>>> "The poll function" being which one? If you mean the one in xhci-dbc.c
>>>> then that's why I had Cc-ed Marek. Moving him to To: - maybe that
>>>> manages to finally catch his attention.
>>>
>>> TBH, I don't know. That's part of the original xue patch at
>>> https://github.com/connojd/xue/blob/master/patches/xen-xue-dbgp.patch
>>> and it works for me as it is.
>>
>> "Works" meaning what? Doesn't crash on you? Or does also provide
>> sensible output in _all_ cases (i.e. including when e.g. the poll
>> happens to run on an idle vCPU)?
> 
> Generally provides sensible output, for example during boot (it is using
> idle vCPU then, right?).

Before Dom0 is started: Yes. With the exception of the phase where PV
Dom0's page tables are constructed, albeit in that time window
guest_cpu_user_regs() shouldn't yield sensible data either. I can only
say I'm surprised; since I have no way to properly test with an XHCI
debug port, I'd have to see about faking something to convince myself
(unless you were to supply example output).

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:02:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:02:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.679996.1057789 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZuIu-0004nP-PJ; Tue, 13 Feb 2024 15:02:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 679996.1057789; Tue, 13 Feb 2024 15:02:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZuIu-0004nI-Mi; Tue, 13 Feb 2024 15:02:36 +0000
Received: by outflank-mailman (input) for mailman id 679996;
 Tue, 13 Feb 2024 15:02:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZuIs-0004n9-Tg
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:02:34 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ea04e00b-ca80-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 16:02:32 +0100 (CET)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-33ce32ae879so82325f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 07:02:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l1-20020a5d6d81000000b0033cda3d3916sm1727871wrs.88.2024.02.13.07.02.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 07:02:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea04e00b-ca80-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707836552; x=1708441352; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=E+yKBwC79Wsd4Nq9xIpwBQRWZaPPy5YuOEJdJjpWzNk=;
        b=gPD6R2w8ittZhhNymNhgzO326nmrPozThTbWMuvI8C0aI4cKm8o3cMWvvNsX4TdZ60
         djxblONsm9C9X4eUBINrhGiS5zzS46NytSK+xjCXtroGonaci9zAySUpOUB0uEIzbmMD
         yweQcdXo7/VDyFds5yz29sHNv6uqVclLI2OM7kQATIBt/IqH+UIl6ATCMAmaZuAJnqEg
         1ekQBbfaHt+nTvDZtFS2+RVIe9erAs2U6rsyRuvMtP+ba9v93sSqRnoms/GG91pplr1p
         1HWNvTTP/RIdesOpkLPecRgeEAjWnpnYVceyVrXjcawKozdILxyNJ8ZLWu/CdKwj3EtS
         +r6w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707836552; x=1708441352;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=E+yKBwC79Wsd4Nq9xIpwBQRWZaPPy5YuOEJdJjpWzNk=;
        b=nQjklxiv3VYf/URTGxnnQMX17K3PCq98D/Oh2V8cZKbxfE/wQodamNf6cVm35LJBCx
         Avacx4EPrYd6g6uCEHUGKFOC0r2NRJLP9jyKCPExb1Oa7j4Y/Mb6LGOzYZOwj6m/jbyy
         l3/pLPN4yNTvdltjM3h/1o04EUWTW9S4R6Ug5KryXeCYCSTKIfZEykdOW+f4Z3dHk23X
         jNqdIjAsTtby/qbb0mUK1yjy0kOZtYBosTn7+aRxPSzLKTFgYgf6VDM7WYhQ/DAzK7Dg
         ahWrpQ2Q9/yeV78l1TfKWVNx8mOoa91/WU6P52tozcrKHTGw+y0N5A9Sv0nm2giOPN7x
         ZvGg==
X-Forwarded-Encrypted: i=1; AJvYcCVcQuK7d3enRAznYYpAGDgwkX0XjW7FWL8kAJdSLT9irpYKQUsAQghjoAi2Lg42KvZHyq54xGTjmHziwr2fjMCk235JTHBBEYH+aaiICGM=
X-Gm-Message-State: AOJu0YwIm35u6cjp+XdK+JKqBHHvQyxW0EEL3v5yAjcEoFwiR4VoUJ4O
	KdarIby8Y61gN4Nx22/YWzh1ES2dym+rqilvFb9a9YuRfZ46a6M0H8y1+Qv7ig==
X-Google-Smtp-Source: AGHT+IEmlnyAJCXKGOvzygehvDKS7rR0AXLgBYen2u5eslRQx1i/Sxx+1+jxvyGdRg/juA+bGoItEg==
X-Received: by 2002:a5d:5688:0:b0:33b:649a:19f7 with SMTP id f8-20020a5d5688000000b0033b649a19f7mr7111439wrv.53.1707836552232;
        Tue, 13 Feb 2024 07:02:32 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCViPa1Hye7AdFJt48+9JuK6B+V6AzACWBaBkYQcygSAOFQC6muvtI5W+RWMuL9PUwa0VWz/bU9bsozUimUNobZwPVn2YMpNORSaWBCuv2y8KRdDqCXHOrBEPEWq4Fk8XapeC0QwdUsuBhSVQX6X/7ItGHdCpDmozia3ng==
Message-ID: <34bfa65b-5537-4bd3-a017-c9a3b01fb331@suse.com>
Date: Tue, 13 Feb 2024 16:02:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] libxl: Fix comment for LIBXL_HAVE_VMTRACE_BUF_KB
Content-Language: en-US
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
References: <4ced67935b2e1a2fc79ccda7624c0849bea6cd87.1707836102.git.w1benny@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4ced67935b2e1a2fc79ccda7624c0849bea6cd87.1707836102.git.w1benny@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.02.2024 15:56, Petr Beneš wrote:
> From: Petr Beneš <w1benny@gmail.com>
> 
> It's located in libxl_domain_build_info, not libxl_domain_create_info.
> 
> Signed-off-by: Petr Beneš <w1benny@gmail.com>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Any reason you didn't also add the Fixes: tag that Anthony suggested
to put there (for me to recognize that this may need backporting;
didn't check yet when that earlier commit went in)?

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:09:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:09:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680006.1057800 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZuPG-0005hI-IM; Tue, 13 Feb 2024 15:09:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680006.1057800; Tue, 13 Feb 2024 15:09:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZuPG-0005hB-Ft; Tue, 13 Feb 2024 15:09:10 +0000
Received: by outflank-mailman (input) for mailman id 680006;
 Tue, 13 Feb 2024 15:09:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+jOn=JW=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1rZuPF-0005gR-Id
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:09:09 +0000
Received: from mail-oo1-xc35.google.com (mail-oo1-xc35.google.com
 [2607:f8b0:4864:20::c35])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d5d8349f-ca81-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 16:09:09 +0100 (CET)
Received: by mail-oo1-xc35.google.com with SMTP id
 006d021491bc7-595b3644acbso2333950eaf.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 07:09:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5d8349f-ca81-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707836947; x=1708441747; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=K5RylAlv7OWrL6a01VMui+e25V6uawgAKvykDJbNmtg=;
        b=KI6hYkZ5Y9xf1kNUia6IYRBGwTfw9xeOSBtgxRm00VMxunH142Tbw3If5j+f1Fv7ZJ
         f8CszrJX0+0HClmfPub37lkzZuTtpoZNOsKgXf1LmmYFrsItKEC0a3HqQ/t8YsbaT/jz
         lbJ02tsBsGVE9dLStcG+4JE0bv3rJEY4eDZdJPXq5WAMkSGZtY99KKpmWNvqwpq9xmQ3
         BEGLqO95Z22EUXwy8BiePFAqQPQ6UBcjPqSdbhkMgRYnhcCD3d4/iNsZ69sYPzeQ4D8o
         I0qw9DhRMPsuTpYqrzt4nisAPYRMwxIUC/Bb8o2Kxs9KcW2aPDsE4XSFnJJ8RfNQXfXc
         De9g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707836947; x=1708441747;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=K5RylAlv7OWrL6a01VMui+e25V6uawgAKvykDJbNmtg=;
        b=EkTxQKdIZTLp4nIRM1iQ+MFaVsSMoUgP679uiUY3thrb4GPO93dwuUegp/ZaiP6NmX
         bCUyP4SGUejRyYXrXbA5RljWY19IInmWj4jEglMYLLR8pBKSAXr4w2xHc4u7B7jCYFnU
         Xm/3neZ5waISwthhTE3UF93lbYUJstjKDOCI1/0BEapQlayywwUNmW60VrKtr2h6Vftm
         Z5aJ+EQNMEU/gm2Iz4rH64Sh4vBYn0c8+6FVmy4XSh6XFrNsv2GGWbQLlAw464AT1fwr
         bpH9lSSGW6paUaI5y1Q8KYOm9k/jicXBHO+S6/oL9T9U71Sbo8SJvPunxNB78zXFBuD6
         UdHA==
X-Forwarded-Encrypted: i=1; AJvYcCVNqDRKNfuTvPddSkO5DlMtfXrPUxtHj7eF2ewj78zsWBbh8846fuJCs0cci9UialSSAv3veFIQDiptKI2VpHjRFZatfJ5YyePQMzj+73M=
X-Gm-Message-State: AOJu0Yy55eDbS7L4lD8hHOevNpnkfoV3+hXLGFYRtOYWu1+41djHJJTI
	TRNBUoOEu5QYhA6HQNRMmNS3zYeTNq47y84ucIhIh8cw7ECy2P7NhFeLnCH3uTUoH15e+IT03kb
	omMKBR+IKcSO94wbZpapa0fv571A=
X-Google-Smtp-Source: AGHT+IHXcSiXV8TKgh5H1Ka4FnpH0/FfcxF93dyUqN6KpyJd4ieGd949LuGJpaXVKA0D3Xk/lFAwXVQ1vUSF7SQb6XQ=
X-Received: by 2002:a05:6870:1494:b0:21a:1e58:2ce0 with SMTP id
 k20-20020a056870149400b0021a1e582ce0mr1147218oab.29.1707836947629; Tue, 13
 Feb 2024 07:09:07 -0800 (PST)
MIME-Version: 1.0
References: <4ced67935b2e1a2fc79ccda7624c0849bea6cd87.1707836102.git.w1benny@gmail.com>
 <34bfa65b-5537-4bd3-a017-c9a3b01fb331@suse.com>
In-Reply-To: <34bfa65b-5537-4bd3-a017-c9a3b01fb331@suse.com>
From: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Date: Tue, 13 Feb 2024 16:08:56 +0100
Message-ID: <CAKBKdXiZ1wcz7_GnUHUmuboTHHVEE03QPSeWhdRKMoV=o7nf3w@mail.gmail.com>
Subject: Re: [PATCH v2] libxl: Fix comment for LIBXL_HAVE_VMTRACE_BUF_KB
To: Jan Beulich <jbeulich@suse.com>
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, 
	Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

> On Tue, Feb 13, 2024 at 4:02=E2=80=AFPM Jan Beulich <jbeulich@suse.com> w=
rote:
>
> On 13.02.2024 15:56, Petr Bene=C5=A1 wrote:
> > From: Petr Bene=C5=A1 <w1benny@gmail.com>
> >
> > It's located in libxl_domain_build_info, not libxl_domain_create_info.
> >
> > Signed-off-by: Petr Bene=C5=A1 <w1benny@gmail.com>
> > Acked-by: Anthony PERARD <anthony.perard@citrix.com>
>
> Any reason you didn't also add the Fixes: tag that Anthony suggested
> to put there (for me to recognize that this may need backporting;
> didn't check yet when that earlier commit went in)?
>
> Jan

Apologies. No other reason that it's my first experience with creating
v2 patch and I honestly didn't know that the "Fixes: tag" suggestion
was directed at me. How should I proceed from here?

P.


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:11:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:11:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680008.1057810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZuRg-0007aD-UZ; Tue, 13 Feb 2024 15:11:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680008.1057810; Tue, 13 Feb 2024 15:11:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZuRg-0007a6-Rk; Tue, 13 Feb 2024 15:11:40 +0000
Received: by outflank-mailman (input) for mailman id 680008;
 Tue, 13 Feb 2024 15:11:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zAib=JW=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1rZuRg-0007Zy-1j
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:11:40 +0000
Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com
 [66.111.4.28]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2dd130ed-ca82-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 16:11:37 +0100 (CET)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailout.nyi.internal (Postfix) with ESMTP id 707CF5C00D7;
 Tue, 13 Feb 2024 10:11:35 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Tue, 13 Feb 2024 10:11:35 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
 13 Feb 2024 10:11:33 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2dd130ed-ca82-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1707837095;
	 x=1707923495; bh=SWskn1JQAhzyV9QJqVJcUm7BudG2qUjNcEm7Vmf1B/w=; b=
	FLOxx8R+H43Xw5yZIl8YDsd8/E1lktzMLtW1uFXWZ3rYTWCfbF49xh8Ubl0AT7tK
	PikSv/cLzflegrWFYZsE4BnJVXZGQvnnWnMCa1nqPyKVucKr6yLK9uH7Wl98wCju
	rgR6QzLOw3JLWuIoaTn22Ex9i41hQzPgzQhAFL0eGHL53DW+BiZC1DsNYT7UJIwe
	UdoorrHF4kuENmcXgSqFyLEtrJxYNpJjbyLSDyS2ITEGTdcfBH4LKskzY2lNYDAl
	D6Y8zGOX6PqSJO6ht6rWfkzH7k1BD9oykO9mngqSn+e1ri7nEYEAeyjWSfs9YkAN
	5t/P1IF3S5Xs6x53VgHjjA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1707837095; x=1707923495; bh=SWskn1JQAhzyV9QJqVJcUm7BudG2
	qUjNcEm7Vmf1B/w=; b=iS1cULKvrMlmcohLuVHADSQfuzFtZB866GBtCbsJDH4K
	22+T7tJRpGsmjkWmXJlWZf4seyszuJUzWfWo5SABgNsN6voGhHnLswO4/m+y2Zlr
	9kXx0LaMOrK/dAWPCsKxduAbxDFZyxNCqB2YoDlbz39bcQ9vhnRdiLuDzYuJqyXC
	5t7wed+j0encZrFJNST0EpBJXA1LoxYBj19eLKRwfp0K7bkA7x68D1DHnivmNi49
	iwVsHNCxH5jmNJ/kw46I0wUd/NAFe07qSYoi/+p/kl7d4ThJrRS50iRCx/a5Bwer
	rVBw1TunF6CDrqdKgSpW4GB+ZDj5SlA+ufEpWeKeCA==
X-ME-Sender: <xms:pobLZRhi36H75kiXbIVCh0ACcSUHmOLTmOn6gliYvmHDm0eWVYRRHQ>
    <xme:pobLZWBwVCSoaRbuABfOKOu3Vm1bUYKiRRGi5LXqD_PrKIcGbt3THUPiBhohh7ufJ
    rWD3hZX0yo7Jg>
X-ME-Received: <xmr:pobLZRFxyiQN8Hj8M3dFt9ZthFoDeDPtpiH_urdftZk2exLwoOMAgEi7W8fEh5V0jjJpftmCnZqAAM1pq4ZeRlOQOujS5N-WJg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehgdejfecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpeffhffvvefukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihcuoehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthh
    hinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeeihfelhfekhefgveejjeet
    vedtgfduveelgeekteehueeufeefudegveejueefhfenucffohhmrghinhepghhithhhuh
    gsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho
    mhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:pobLZWTWVso8mXcWQ9Gk8DxeNpNdffMG1zHio8FVufFK0OB_BSrtgA>
    <xmx:pobLZezkCsRK1TCOPGphXT-aWm4EuytTtFOaziYtCz-g3pqGP8b-dg>
    <xmx:pobLZc4cxnQex18a6-M-abp_7SILuLqB7TAWDsVGCKBgefmlSeQZLQ>
    <xmx:p4bLZVq6Rwjqe_944VaRk4Xxk1D67S4GxrY6QCUuxHfQN-IkvrYiRA>
Feedback-ID: i1568416f:Fastmail
Date: Tue, 13 Feb 2024 16:11:31 +0100
From: Marek Marczykowski <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Message-ID: <ZcuGo_vQhiTAQep0@mail-itl>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
 <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
 <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com>
 <ZcrlcuNU9y8WymiK@mail-itl>
 <817ceec4-ed12-4360-a6ce-f87732f62251@suse.com>
 <ZcuCTnMpFib4ArHW@mail-itl>
 <b52fa3fa-4e33-4211-af18-f00aaa0b365b@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="m4B4VxtdGezXXxDY"
Content-Disposition: inline
In-Reply-To: <b52fa3fa-4e33-4211-af18-f00aaa0b365b@suse.com>


--m4B4VxtdGezXXxDY
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 13 Feb 2024 16:11:31 +0100
From: Marek Marczykowski <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers

On Tue, Feb 13, 2024 at 04:00:32PM +0100, Jan Beulich wrote:
> On 13.02.2024 15:53, Marek Marczykowski wrote:
> > On Tue, Feb 13, 2024 at 08:45:54AM +0100, Jan Beulich wrote:
> >> On 13.02.2024 04:43, Marek Marczykowski wrote:
> >>> On Mon, Feb 12, 2024 at 10:04:38AM +0100, Jan Beulich wrote:
> >>>> On 08.02.2024 23:00, Julien Grall wrote:
> >>>>> On 05/02/2024 13:27, Jan Beulich wrote:
> >>>>>> In preparation of dropping the register parameters from
> >>>>>> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
> >>>>>> register state needs making available another way for the few key
> >>>>>> handlers which need it. Fake IRQ-like state.
> >>>>>>
> >>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >>>>>> ---
> >>>>>> The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsisten=
t with
> >>>>>> other console poll functions we have, and it's unclear whether tha=
t's
> >>>>>> actually generally correct.
> >>>>>
> >>>>> Is it? Looking at ns16550_poll() we would pass guest_user_regs() if=
=20
> >>>>> run_in_exception() doesn't exist. But looking at the caller, no-on =
seems=20
> >>>>> to care about the 'regs'. So is this just a latent bug?
> >>>>
> >>>> What do you mean by "doesn't exist"? ns16550_poll() assumes it exist=
s.
> >>>> And I can spot any use of guest_user_regs() on the respective generic
> >>>> or Arm-specific bug.c paths.
> >>>>
> >>>>> BTW, do you have an idea why the poll function is not run in an=20
> >>>>> exception handler?
> >>>>
> >>>> "The poll function" being which one? If you mean the one in xhci-dbc=
=2Ec
> >>>> then that's why I had Cc-ed Marek. Moving him to To: - maybe that
> >>>> manages to finally catch his attention.
> >>>
> >>> TBH, I don't know. That's part of the original xue patch at
> >>> https://github.com/connojd/xue/blob/master/patches/xen-xue-dbgp.patch
> >>> and it works for me as it is.
> >>
> >> "Works" meaning what? Doesn't crash on you? Or does also provide
> >> sensible output in _all_ cases (i.e. including when e.g. the poll
> >> happens to run on an idle vCPU)?
> >=20
> > Generally provides sensible output, for example during boot (it is using
> > idle vCPU then, right?).
>=20
> Before Dom0 is started: Yes. With the exception of the phase where PV
> Dom0's page tables are constructed, albeit in that time window
> guest_cpu_user_regs() shouldn't yield sensible data either. I can only
> say I'm surprised; since I have no way to properly test with an XHCI
> debug port, I'd have to see about faking something to convince myself
> (unless you were to supply example output).

Would you like me to test this series with xhci console? Or maybe add
some extra debug prints and include their output? But note, printk from
inside console code generally leads to deadlocks. What I did for some
debugging was to log into some separate buffer and dump it later.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--m4B4VxtdGezXXxDY
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXLhqMACgkQ24/THMrX
1yyQAgf7B/1ICb7+VpGmREInGiIW2AOY5R1/IcNPN0lHd5QGLZQqSm7pbhYHUwBT
1rH0cMVn6hhyqA7t1D6Hmuj37SESeC1Pcj5l3jtbXbMjU15gtWzGQh6jJ14L9Ezh
fgUx/7Be076ajBvHz8tL9OqSOIyI/YVRFh6gxU4tAhVIfTS2Kj+fzyfBZMgI/019
a5FxDpP+iTyNeRCKXJO2oAGuxYnvDbLHHcpLcvPmKfeVtNbshjHkrrQyLLhurP3G
rYgrVeAmdvy3eIuII9f1WWuBqvLKAZMr/Si4DyicCFynhjeNpmen9eVI+0J2aUjN
RgnaJhuIn009kJ0dZSqjjdGtW/xt1Q==
=kqUh
-----END PGP SIGNATURE-----

--m4B4VxtdGezXXxDY--


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:25:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:25:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680017.1057820 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZueg-0001Rr-4N; Tue, 13 Feb 2024 15:25:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680017.1057820; Tue, 13 Feb 2024 15:25:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZueg-0001Rk-08; Tue, 13 Feb 2024 15:25:06 +0000
Received: by outflank-mailman (input) for mailman id 680017;
 Tue, 13 Feb 2024 15:25:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZuef-0001Re-75
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:25:05 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0eb2b1f1-ca84-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 16:25:02 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-411d715c401so600775e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 07:25:02 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v8-20020a5d43c8000000b0033b512b2031sm9778889wrr.114.2024.02.13.07.25.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 07:25:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0eb2b1f1-ca84-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707837902; x=1708442702; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=C0R4a2pKzNuVqPKYAumSSAkLZHs8wRdapQvHR8QtOf0=;
        b=IIa8IlSBLIDiaqhRrPiz+/VfPdiT4Rg1bqRZggMC39UJD852+jPe0GIKh9Zio4xDQC
         zL5650MPugQf4OQARnDa7nInpVg1oUc4neQzBJTibrDmSzgkfaXOG8TUMb84/bkzq6rM
         u8EHqzjFiKl3YcvmRe7h7GdetR4U1vcBMPQyyO+sxfMbY8iB3Wn6wOOzdApZtM3F5oc/
         r7PEY/DMpXfWlsjX2Cavy8c9FoF6ysd3AiRI5LSZynYZbLyrXmhPxjcEr9CYLuW1zEvx
         /RCR2O8ZbCKV6e/KR3FZDD2796tv/SLdbns7BIZL/HT+SQ09cyGLwArsxS3qtEE6NZPF
         MsKg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707837902; x=1708442702;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=C0R4a2pKzNuVqPKYAumSSAkLZHs8wRdapQvHR8QtOf0=;
        b=Glw/cO22Dd7HdPkqu2cUDGa6UPCvK/1mh/YkXDrDr9B3hStCacl6T0H8ytC5/piJ6/
         t8f1L+Wk0qZpiDAC9re5UhO5Phkxdf0jV4+3w55TiWhxdhqHMS3cWPv6pVOaI58ISFSN
         tqIzEx/6KG3oqHMSi0gD2K/p/Ww4H34w7j8xjIpMRRBYfvzkDSRRs4OvNIGiaG0TG05+
         zX7RYQ4HSj0SJzNaenIbvKspXC6IC1G2hMKO5WLQRxscrhUTahFjvZKt+/akF8AqC2mj
         /vtPyLM0m4E4Dd2fgRIiSeiLaERFPBSvOJp30GmRYXMjRdnsX5mzO6zALdYrIwPWX8jl
         k2hw==
X-Forwarded-Encrypted: i=1; AJvYcCWcvMxGiUjVcyVZdX6R64FMRrCkAi79EbzwXGP2Ec2Y55Qs1NdeWgezbeJRAjsBpR/l8Fq3wMG58Zw4PE41jL+svXtSHrrMZ5Clu2fQJj4=
X-Gm-Message-State: AOJu0YwgZXE7rtk1wyutEh+aL2pusfzqy1jpdzgslwUNrpAcNJGIWbmj
	MCmVbLX8VFAaHYCYOe5mX5pH5itYdhnMuQpwcRiZ2WuQFvZhxEsUAokm/2afMA==
X-Google-Smtp-Source: AGHT+IE5rWeYLXHvDtCRv8YZFlk6pzhyOe5psstwr/EoNYQxNq+plaWvrLAnHC8AMnKx3dsOqhGuMA==
X-Received: by 2002:a05:600c:1912:b0:411:b66a:5bf6 with SMTP id j18-20020a05600c191200b00411b66a5bf6mr1736939wmq.5.1707837902246;
        Tue, 13 Feb 2024 07:25:02 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU1mO7Voo81TUSeuXr37oga6+ibvkHl0yaF9EV6FJo+eSSjyeKwDm7iuPnbkHvsEssrCuXNfSV4MZ7au25U+HzrZvD2h0P9jD1gA4wf257PobFD5E5nc3RKFtVEBxmvYykoKN8I1WrAgE3EiYDQCdYM8eWq8ouFgrhDGNq1EUu/vNDYDvmh26Gx+FvdlSCvGfoE1hXtaifK4Xppa0qIgQGK28zJNZfK8jm+8ns149I30GFWKvMCPhJyyansHRwaR7iAm+06wApyw2rNMP3N0MFNkv5W8IOlqQcQcqlz10E02tCuBzSJ6urbQqG5wy1QWExw+KJnjojlxNokqVuIOKvzbk5AwLHl+JZSkE3dDD6R1ee0+bFgxEfr3dDGYqPyhFHYuw==
Message-ID: <a66e3131-1de1-49cb-9b26-5fb1fd77bc20@suse.com>
Date: Tue, 13 Feb 2024 16:25:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 14/15] xen/arm: add cache coloring support for Xen
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-15-carlo.nonato@minervasys.tech>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240129171811.21382-15-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.01.2024 18:18, Carlo Nonato wrote:
> @@ -218,9 +230,44 @@ static void xen_pt_enforce_wnx(void)
>      flush_xen_tlb_local();
>  }
>  
> +static void __init create_llc_coloring_mappings(void)
> +{
> +    lpae_t pte;
> +    unsigned int i;
> +    struct bootmodule *xen_bootmodule = boot_module_find_by_kind(BOOTMOD_XEN);
> +    mfn_t mfn = maddr_to_mfn(xen_bootmodule->start);
> +
> +    for_each_xen_colored_mfn( mfn, i )

Nit: Either you consider the construct a pseudo-keyword, then

    for_each_xen_colored_mfn ( mfn, i )

or you don't, then

    for_each_xen_colored_mfn(mfn, i)

please.

> --- a/xen/common/llc-coloring.c
> +++ b/xen/common/llc-coloring.c
> @@ -29,6 +29,8 @@ static unsigned int __ro_after_init xen_num_colors;
>  
>  #define mfn_color_mask              (max_nr_colors - 1)
>  #define mfn_to_color(mfn)           (mfn_x(mfn) & mfn_color_mask)
> +#define mfn_set_color(mfn, color)   (_mfn((mfn_x(mfn) & ~mfn_color_mask) | \
> +                                     (color)))

Nit: The wrapped line wants further indenting, such that it becomes
immediately clear what parentheses are still open. Alternatively:

#define mfn_set_color(mfn, color) \
    (_mfn((mfn_x(mfn) & ~mfn_color_mask) | (color)))

This is certainly an "interesting" construct: I, for one, wouldn't expect
that setting the color actually changes the MFN.

> @@ -316,6 +318,27 @@ unsigned int get_max_nr_llc_colors(void)
>      return max_nr_colors;
>  }
>  
> +paddr_t __init xen_colored_map_size(void)
> +{
> +    return ROUNDUP((_end - _start) * max_nr_colors, XEN_PADDR_ALIGN);
> +}
> +
> +mfn_t __init xen_colored_mfn(mfn_t mfn)
> +{
> +    unsigned int i, color = mfn_to_color(mfn);
> +
> +    for( i = 0; i < xen_num_colors; i++ )

Nit: Missing blank.

> +    {
> +        if ( color == xen_colors[i] )
> +            return mfn;
> +        else if ( color < xen_colors[i] )
> +            return mfn_set_color(mfn, xen_colors[i]);

How do you know that this or ...

> +    }
> +
> +    /* Jump to next color space (max_nr_colors mfns) and use the first color */
> +    return mfn_set_color(mfn_add(mfn, max_nr_colors), xen_colors[0]);

... this MFN are actually valid and in (available) RAM?

> --- a/xen/include/xen/llc-coloring.h
> +++ b/xen/include/xen/llc-coloring.h
> @@ -24,6 +24,17 @@ static inline void domain_llc_coloring_free(struct domain *d) {}
>  static inline void domain_dump_llc_colors(const struct domain *d) {}
>  #endif
>  
> +/**
> + * Iterate over each Xen mfn in the colored space.
> + * @mfn:    the current mfn. The first non colored mfn must be provided as the
> + *          starting point.
> + * @i:      loop index.
> + */
> +#define for_each_xen_colored_mfn(mfn, i)        \
> +    for ( i = 0, mfn = xen_colored_mfn(mfn);    \
> +          i < (_end - _start) >> PAGE_SHIFT;    \
> +          i++, mfn = xen_colored_mfn(mfn_add(mfn, 1)) )

While the comment mentions it, I still consider it problematic that
- unlike other for_each_* constructs we have - this requires one of
the iteration variables to be set up front. Question is why it needs
to be that way: Isn't it the MFN underlying _start which you mean to
start from?

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:27:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:27:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680020.1057829 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZugh-0001xj-E4; Tue, 13 Feb 2024 15:27:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680020.1057829; Tue, 13 Feb 2024 15:27:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZugh-0001xc-BJ; Tue, 13 Feb 2024 15:27:11 +0000
Received: by outflank-mailman (input) for mailman id 680020;
 Tue, 13 Feb 2024 15:27:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZugf-0001xS-Ml
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:27:09 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5928f5e7-ca84-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 16:27:07 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-411c779ab2dso4488425e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 07:27:07 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bq10-20020a5d5a0a000000b0033b6e26f0f9sm2002007wrb.42.2024.02.13.07.27.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 07:27:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5928f5e7-ca84-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707838027; x=1708442827; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xWw3RjzWL7BVtLSN+eAGRRnnzqGow5bxLyMvvHGmVZk=;
        b=HwO/8psb605sr1w9ksIlyKaG9mZxJIJxU3XZ37R42CfAVFYHgXfd6zXGz4KUQm98dd
         8Sfp9dWTbykjLJGJ8vdcuo7CieQ1KKab5VCviVAWfMbsYAIaL8BWQMETjpAtNKmdc30l
         RoKsw/0mQ0jh6LU5qYLUhz9u6QNKxl6roYuRVyOlYrdq3SyQMg/6DyvzVuhIAzGdhmv0
         mV5dyOpmCOpuibcOCKsVBdgjejOSTmTFbJHwh/oTSpC98o1yydJUC0h4bVTVK4z+u3B/
         f8Qo31e65G4aI7Onb303kkeDo0U1qnPLCf0FVCdTbCJfgKGSYGQUeYbRj07sGCE+f9gc
         C6/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707838027; x=1708442827;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xWw3RjzWL7BVtLSN+eAGRRnnzqGow5bxLyMvvHGmVZk=;
        b=Dk3KBMH712L6JBehRXXzhONxtJ3m5fjFzi18pTZBI2cVSR8ysuNz1AIBtmLmJybelJ
         stXY2MKUsdOcGZ4/+9i3Eqv8eHS2VCCLGtHtnf+/nk/mcEqHAOpG4+PyltOxf0CQn2Y7
         wsED86T2KCvYzv9SIT67fpYtvC7U6sm3BfIklG+KQ1pOo6yzuJoexwFLxKxyILDrp2gV
         bg7YS8Ju2JVpcGxhCoFVR1a81j1A58icXT/QcACOacDfWShcxP98afMyqxy71QQXNYBe
         pQqRBLXw7BJYsf3uqG3ZUcTuIfw6KFI2xTRAPqhnMGTZ4Pz/tjhuDMIl+w6fwsWXG2cB
         zGAQ==
X-Gm-Message-State: AOJu0YwR/N/IFDn2pyemzwcevTJPJAzG/YE1bw0m1h++tRiICchV8OoN
	pCqND6Gwpe5mBnR+k+pMhVahYjXVgr+eTi94ySFabiryofrAvkiIXZOWTSBMvA==
X-Google-Smtp-Source: AGHT+IFJyq39jzuL7LrUlo8EweDtCnE0p23r4+FycNl3B7WWaJ77ubK6JzDYIpa0HkepggsPACCATg==
X-Received: by 2002:adf:b312:0:b0:33c:e321:b285 with SMTP id j18-20020adfb312000000b0033ce321b285mr281999wrd.12.1707838027246;
        Tue, 13 Feb 2024 07:27:07 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWL5z0m5RIQgbYAMQ4bqNvZhWXpBlL5kq2GgWKOPS/BUaxav89uJ0XCL8SitHVUHv+y5AQDr/cu5wHSy6P2wOimkWjHd+fEo6d2da+/paCIa1ZIeeYebtpTQW5IokyJ5tLNCf9Hm4Vg+pFUdNyVLMaF/6xO9fD8w/5OTA==
Message-ID: <c8d747a9-e164-460a-9e50-e976580d10ef@suse.com>
Date: Tue, 13 Feb 2024 16:27:06 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] libxl: Fix comment for LIBXL_HAVE_VMTRACE_BUF_KB
Content-Language: en-US
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
References: <4ced67935b2e1a2fc79ccda7624c0849bea6cd87.1707836102.git.w1benny@gmail.com>
 <34bfa65b-5537-4bd3-a017-c9a3b01fb331@suse.com>
 <CAKBKdXiZ1wcz7_GnUHUmuboTHHVEE03QPSeWhdRKMoV=o7nf3w@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAKBKdXiZ1wcz7_GnUHUmuboTHHVEE03QPSeWhdRKMoV=o7nf3w@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.02.2024 16:08, Petr Beneš wrote:
>> On Tue, Feb 13, 2024 at 4:02 PM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 13.02.2024 15:56, Petr Beneš wrote:
>>> From: Petr Beneš <w1benny@gmail.com>
>>>
>>> It's located in libxl_domain_build_info, not libxl_domain_create_info.
>>>
>>> Signed-off-by: Petr Beneš <w1benny@gmail.com>
>>> Acked-by: Anthony PERARD <anthony.perard@citrix.com>
>>
>> Any reason you didn't also add the Fixes: tag that Anthony suggested
>> to put there (for me to recognize that this may need backporting;
>> didn't check yet when that earlier commit went in)?
> 
> Apologies. No other reason that it's my first experience with creating
> v2 patch and I honestly didn't know that the "Fixes: tag" suggestion
> was directed at me. How should I proceed from here?

I'll try to remember to insert it while committing.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:36:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:36:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680025.1057840 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZupo-0004Ah-D8; Tue, 13 Feb 2024 15:36:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680025.1057840; Tue, 13 Feb 2024 15:36:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZupo-0004Aa-9n; Tue, 13 Feb 2024 15:36:36 +0000
Received: by outflank-mailman (input) for mailman id 680025;
 Tue, 13 Feb 2024 15:36:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CpMr=JW=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1rZupn-0004AU-K9
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:36:35 +0000
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
 (mail-db5eur01on0626.outbound.protection.outlook.com
 [2a01:111:f400:fe02::626])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a9576d4a-ca85-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 16:36:32 +0100 (CET)
Received: from AS8PR07CA0001.eurprd07.prod.outlook.com (2603:10a6:20b:451::10)
 by DU5PR08MB10756.eurprd08.prod.outlook.com (2603:10a6:10:51e::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.32; Tue, 13 Feb
 2024 15:36:28 +0000
Received: from AM1PEPF000252DF.eurprd07.prod.outlook.com
 (2603:10a6:20b:451:cafe::68) by AS8PR07CA0001.outlook.office365.com
 (2603:10a6:20b:451::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend
 Transport; Tue, 13 Feb 2024 15:36:28 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 AM1PEPF000252DF.mail.protection.outlook.com (10.167.16.57) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7292.25 via Frontend Transport; Tue, 13 Feb 2024 15:36:28 +0000
Received: ("Tessian outbound 1076c872ecc6:v228");
 Tue, 13 Feb 2024 15:36:27 +0000
Received: from bc95c49f6eb3.1
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 4996C959-A3ED-42DB-8C69-781C565FFD7D.1; 
 Tue, 13 Feb 2024 15:36:21 +0000
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id bc95c49f6eb3.1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Tue, 13 Feb 2024 15:36:21 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by DB8PR08MB5371.eurprd08.prod.outlook.com (2603:10a6:10:114::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Tue, 13 Feb
 2024 15:36:17 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::99ee:f28d:84bc:97cb]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::99ee:f28d:84bc:97cb%7]) with mapi id 15.20.7270.036; Tue, 13 Feb 2024
 15:36:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a9576d4a-ca85-11ee-98f5-efadbce2ee36
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=l32j02c7yzlBQMcBPHy2Re3d53WDECK3MF4qZ/m1vb5CDwkfNuhwOsn6dEeM7vhNUWBl+1anSpSQTBPkxWOTD/yiFhp4i5irykJAvnjh2mXyHqpS55lgC7YTNnSVI2S5R5wDTlHolZ8EdQxB0KHqxfCAqJE669rrf1NW10W3SEqbzqjcggg3xCv7kDYApE8AkobsF+8IrWM7qjtrP0mmM18V3WRoueDgdSxt+RmbnHzoW2yzuPPzkaNSWr3007fHVLSx5y24xWuwa9k51N9u1J0IfAUEAwkkqxger3MFuKLOIv6mZV+TL2BAx0L5kaquHXWMJgAeO/Keu8+Bd0T7rQ==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wMDqOAOqUlR6SMcP0KYloozBSpPOQU4mqbd8n5tfTiE=;
 b=afl9eFcYAaAxjGWCX7Az2SQXaQ4I6lDcGJVXUV8VcmBYyoFn9eC6XqciZ2xqNPY22eC/MrrQc3qWI6t5eJhAsF9KOa+/ADnCWU/jjMj5iVMiiGv+6BeB2XTFr9W9aY4anJauXqp95UZWAZdDn02icNcwngi1uUgqIZO9FHRyJZP0g7uCLOuGua2oyeTIqoCAh30MNfykuIaqdsrFMyaCOfH1Cmh3SUrbACsgQjZ6HXc/IvLKvrKlUO/Zp0o8abllO9AD49cF+vOlIvGRRDe5iEW0YPNSQY6fobPTjy0uGxSr4z3ElAp1ZWpBs6HtYiHX8J9P+NGrS6kmw05FeklNeA==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0
 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wMDqOAOqUlR6SMcP0KYloozBSpPOQU4mqbd8n5tfTiE=;
 b=YTtbr3r1Mte8XjdhY8ritlb9Ulm+M566u7rCLYI10syC+w7ma/oawZYCa9AtTSxASdyto3aBxq7CpReUrMIfuLCmp0yWUIdnL5ETaP7A6IyvHcocxkuzxZz/ov+tTwQuUXQXBqrYf87q/Mr2UiPh2bUsjv6Q/1KUw6hxc50yupA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: bd3bca1f42d31f29
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KHDE1E43bva9Wq4iUYDFZ3JMidMcYw0kTxRCAKpD6zyXtdqEKDQhMHPJUO3kwH8Qju2xr6/66m3rjLsUJi8C6GFOcOGm6IW6tWN226k88TrEyuE3GLMkcYRtpMMFYHz7m0rBNrgFbf3PnZe0PEVgLzrFdnMUSB5jb9H2AzR3nQS2X/kkoUYrLJoMja3rfPDrsXOwHWecMdFRHXgL6mS1WdDcy/EnK7tjBnxEQojG0hnXmGWiOv87EHNwAgIp2gEC1ETG/N48vySvwvbLJmpLiFDjRNrI2W5zHd8JsVDlBYlUjBSh723Ju/xNuk6ck5VpgDicXN4L5VGXA/TiKhO5IA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=wMDqOAOqUlR6SMcP0KYloozBSpPOQU4mqbd8n5tfTiE=;
 b=myjdHFmIlbJ+RNGWUHkwID5ByeRM/RD/zYm2sTNlZPsPwfg+lfAaETCmPQGXaEuk9PBIjZBnD0hmyeD7GEct3lZdO1n7Gk34pXaK/yD97Ts+JBwlu8oHnHxKe6VV7AwfJr9jQqGenHqFK7NAjP1ZJIpmrlFDtZi3m5UXEIvRLUpnmTZts7Qw++403E+XXzNVZoDXzAgT/ZjHecKxUzrHsGV7H/Y4dtbUN0omYGPxTirBUkt5nmVfedFnuC1K8XOoUoAymNJ/v700WUPEAAmNAFBqdMma1xgZyQqETiP3lgg77LoQRMTi6Gk95B/5Yj/r+7pQIpgIRmtiBTcjOQ1KtQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=wMDqOAOqUlR6SMcP0KYloozBSpPOQU4mqbd8n5tfTiE=;
 b=YTtbr3r1Mte8XjdhY8ritlb9Ulm+M566u7rCLYI10syC+w7ma/oawZYCa9AtTSxASdyto3aBxq7CpReUrMIfuLCmp0yWUIdnL5ETaP7A6IyvHcocxkuzxZz/ov+tTwQuUXQXBqrYf87q/Mr2UiPh2bUsjv6Q/1KUw6hxc50yupA=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: =?utf-8?B?UGV0ciBCZW5lxaE=?= <w1benny@gmail.com>
CC: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, Juergen
 Gross <jgross@suse.com>, Xen-devel <xen-devel@lists.xenproject.org>, Jan
 Beulich <jbeulich@suse.com>
Subject: Re: [PATCH v2] libxl: Fix comment for LIBXL_HAVE_VMTRACE_BUF_KB
Thread-Topic: [PATCH v2] libxl: Fix comment for LIBXL_HAVE_VMTRACE_BUF_KB
Thread-Index: AQHaXo2+7VlqRIHhmU23d/RZTQZ3VLEIYAUAgAAFEwCAAAKDgA==
Date: Tue, 13 Feb 2024 15:36:16 +0000
Message-ID: <8D75402C-D476-40FD-86E6-E188CA8DB6F0@arm.com>
References:
 <4ced67935b2e1a2fc79ccda7624c0849bea6cd87.1707836102.git.w1benny@gmail.com>
 <34bfa65b-5537-4bd3-a017-c9a3b01fb331@suse.com>
 <CAKBKdXiZ1wcz7_GnUHUmuboTHHVEE03QPSeWhdRKMoV=o7nf3w@mail.gmail.com>
 <c8d747a9-e164-460a-9e50-e976580d10ef@suse.com>
In-Reply-To: <c8d747a9-e164-460a-9e50-e976580d10ef@suse.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|DB8PR08MB5371:EE_|AM1PEPF000252DF:EE_|DU5PR08MB10756:EE_
X-MS-Office365-Filtering-Correlation-Id: 53e84576-299f-48bf-7972-08dc2ca98b93
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 Ua/LQtnZhsD7Ni7g74oL1d1Rze728RB6GNOeTLs7uFSNq8CQAwGqOWj1uzKLNRiPMT4NNy3mgxjjOnkNg6vJ5P2JkLFq3WlsfoxvN0EauiZu//HGNoH2Z70RrFodQD9Xjei/qR/8GyrI2AEiMMrR0f1R7FHXVFlBhg9WiacE1Xo12WH6Oouhbedq5JwJkkV0wyIByAtf/gmL2zzjC3wtkFnwd+yrLsNDwock//ACkgBbpHU8++VyCy3ppcnDflq0vuic+ugAEWeun46/0zzByF72n6+HSjNqM+BWbDdXOe3b8oEYiXSbfcGtOnTaFLNL172Lpr1gBjygIlKZzLzqbnnXz5rzJcm8Mm4+noXtJE7LV+kmMTwIkJFyXdbW0fSdtKox7u7ULivLGrMMfx4ihV3Y2i1w+WS6Eoy1fPNvBCVxkFr+T0Y6JOLCut8Jh5dQMg0+KW3fgy2R3d3YgTq+30L3US4qpKtO06UnjCW+b0TA2BeXM43yK/ckCN+bSn2IuIklSr9q1x9pWxL7U2EFPND0sFaBbG7PkZWy2WfvsL6oyT1QRLSPMKtMzm3LB8gGSispQLugbK95y1utUKpmmY12PkRQRIWy75rGPvmyZ5g=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(346002)(39860400002)(376002)(366004)(230922051799003)(64100799003)(186009)(1800799012)(451199024)(966005)(76116006)(6486002)(41300700001)(2906002)(5660300002)(66476007)(6916009)(54906003)(66556008)(86362001)(83380400001)(66946007)(8936002)(4326008)(8676002)(316002)(64756008)(33656002)(71200400001)(36756003)(66446008)(6506007)(2616005)(26005)(6512007)(53546011)(38100700002)(478600001)(122000001)(38070700009);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <C3DFC9F6817EEF48BDE130E031D7979D@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5371
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 AM1PEPF000252DF.eurprd07.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	6f263f6e-fc3d-4161-900a-08dc2ca98462
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	sspevaa0TUMmpRrUOPjULW9h/FX2v841QQPDe3212eiGEGXscQZNfmhvEfKdN98ZVTjZTANRXrd+4PpIriDKHxtjGizK8rDQIfZOm0fVNVgCeWJslMAiW9Mw/P+fTMjU3SekV+8iCh2EhqrIA/CZnknqE4yBP9I4Q2yt212Hr4eLayTdZu1Q/gxfsgg+GdWPpd/5wPE5H4ePEgTg4RC7jqwiLKFHyiYY/K1oFlYwuiX0mHnX6lHaQaDoE7ggCK2MUzx+N9JgKDgmAmp3EBtUMLhA9hdD6Lq3iJT0MzLjIrRSXWhKMgoU+nJ4397lzLe31c94B5XqIUzu97kFpZjCG3TxcUbOM6MtK70p27J9UEtr6yvHJFQachdxhePd1EisRJOtdauYxXpm7IVEAdnUI5JMiwtayLBsuC85f/Y7nnxa9QF9AiH1+7YIDrVz87HHXTJYwiVdcc0N9TSOfztaDe/uRLyQRNMWYcV76K2EBk5L1l6e2nND04sbM4KmPk2KeulRF70I0ka1Lz6YzSKsVEmkga0kJHD75nUY8ue7iPKZnWRECg7ngCV+W2TKEP2zm1uG49SgGr3W3eF/uGXXjQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(136003)(396003)(376002)(346002)(39860400002)(230922051799003)(451199024)(1800799012)(82310400011)(186009)(64100799003)(46966006)(40470700004)(36840700001)(26005)(336012)(41300700001)(2616005)(4326008)(70586007)(6862004)(8936002)(2906002)(5660300002)(8676002)(70206006)(54906003)(83380400001)(316002)(6512007)(966005)(6486002)(478600001)(53546011)(6506007)(86362001)(107886003)(82740400003)(356005)(36756003)(81166007)(33656002);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 15:36:28.2092
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 53e84576-299f-48bf-7972-08dc2ca98b93
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	AM1PEPF000252DF.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU5PR08MB10756

DQoNCj4gT24gMTMgRmViIDIwMjQsIGF0IDE1OjI3LCBKYW4gQmV1bGljaCA8amJldWxpY2hAc3Vz
ZS5jb20+IHdyb3RlOg0KPiANCj4gT24gMTMuMDIuMjAyNCAxNjowOCwgUGV0ciBCZW5lxaEgd3Jv
dGU6DQo+Pj4gT24gVHVlLCBGZWIgMTMsIDIwMjQgYXQgNDowMuKAr1BNIEphbiBCZXVsaWNoIDxq
YmV1bGljaEBzdXNlLmNvbT4gd3JvdGU6DQo+Pj4gDQo+Pj4gT24gMTMuMDIuMjAyNCAxNTo1Niwg
UGV0ciBCZW5lxaEgd3JvdGU6DQo+Pj4+IEZyb206IFBldHIgQmVuZcWhIDx3MWJlbm55QGdtYWls
LmNvbT4NCj4+Pj4gDQo+Pj4+IEl0J3MgbG9jYXRlZCBpbiBsaWJ4bF9kb21haW5fYnVpbGRfaW5m
bywgbm90IGxpYnhsX2RvbWFpbl9jcmVhdGVfaW5mby4NCj4+Pj4gDQo+Pj4+IFNpZ25lZC1vZmYt
Ynk6IFBldHIgQmVuZcWhIDx3MWJlbm55QGdtYWlsLmNvbT4NCj4+Pj4gQWNrZWQtYnk6IEFudGhv
bnkgUEVSQVJEIDxhbnRob255LnBlcmFyZEBjaXRyaXguY29tPg0KPj4+IA0KPj4+IEFueSByZWFz
b24geW91IGRpZG4ndCBhbHNvIGFkZCB0aGUgRml4ZXM6IHRhZyB0aGF0IEFudGhvbnkgc3VnZ2Vz
dGVkDQo+Pj4gdG8gcHV0IHRoZXJlIChmb3IgbWUgdG8gcmVjb2duaXplIHRoYXQgdGhpcyBtYXkg
bmVlZCBiYWNrcG9ydGluZzsNCj4+PiBkaWRuJ3QgY2hlY2sgeWV0IHdoZW4gdGhhdCBlYXJsaWVy
IGNvbW1pdCB3ZW50IGluKT8NCj4+IA0KPj4gQXBvbG9naWVzLiBObyBvdGhlciByZWFzb24gdGhh
dCBpdCdzIG15IGZpcnN0IGV4cGVyaWVuY2Ugd2l0aCBjcmVhdGluZw0KPj4gdjIgcGF0Y2ggYW5k
IEkgaG9uZXN0bHkgZGlkbid0IGtub3cgdGhhdCB0aGUgIkZpeGVzOiB0YWciIHN1Z2dlc3Rpb24N
Cj4+IHdhcyBkaXJlY3RlZCBhdCBtZS4gSG93IHNob3VsZCBJIHByb2NlZWQgZnJvbSBoZXJlPw0K
DQpIaSBQZXRyLA0KDQpJbiB0aGlzIGNhc2UgSmFuIHdpbGwgdHJ5IHRvIHRha2UgY2FyZSBvZiB0
aGF0LCBpZiB5b3UgYXJlIGludGVyZXN0ZWQgb24gaG93IHRvIGRvIHRoYXQsDQp0aGUgc2VjdGlv
biDigJgxLjIgVGhlIGNvbW1pdCBtZXNzYWdl4oCZIG9mIGh0dHBzOi8veGVuYml0cy54ZW4ub3Jn
L2RvY3MvdW5zdGFibGUvcHJvY2Vzcy9zZW5kaW5nLXBhdGNoZXMuaHRtbA0Kd2lsbCBleHBsYWlu
IGhvdyB0byBzZXR1cCBnaXQgdG8gaGF2ZSB0aGUgY29ycmVjdCBjb25maWd1cmF0aW9uIGZvciB0
aGUgZml4ZXMgdGFnLCBhZnRlcndhcmRzIHlvdSBjYW4ganVzdDoNCg0KMSkgdGFrZSB0aGUgU0hB
IG9mIHRoZSBjb21taXQgd2hvIGludHJvZHVjZWQgdGhlIGlzc3VlOg0KICAgIGdpdCBibGFtZSAt
TDxsaW5lIG51bSBmcm9tPiw8bGluZSBudW0gdG8+IC0tIDxwYXRoIHRvIHRoZSBmaWxlPg0KDQoy
KSBHZXQgdGhlIEZpeGVzIHRhZyB0byBiZSBwdXQgaW4gdGhlIGNvbW1pdCBtZXNzYWdlOg0KICAg
IGdpdCBzaG93IC0tcHJldHR5PWZpeGVzIDxTSEE+DQogICAgDQogICAgWW91IHdpbGwgc2VlIGFz
IGZpcnN0IGxpbmUgc29tZXRoaW5nIGFsb25nIHRoZSBsaW5lOiDigJxGaXhlczogNmU5Y2NkYTVl
NDc1ICjigJxibGEgYmxhIGJsYSBibGEgYmxhIGJsYSIpIg0KDQpUaGVyZSBjb3VsZCBiZSBiZXR0
ZXIgZ2l0IGNvbW1hbmQgdG8gb2J0YWluIHRoZSBzYW1lIHRoaW5nLCBidXQgSSBhbHdheXMgdXNl
IHRoZSBhYm92ZS4NCg0KQ2hlZXJzLA0KTHVjYQ0KDQoNCg==


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:44:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:44:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680030.1057849 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZux7-0006Ud-3c; Tue, 13 Feb 2024 15:44:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680030.1057849; Tue, 13 Feb 2024 15:44:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZux7-0006UW-0o; Tue, 13 Feb 2024 15:44:09 +0000
Received: by outflank-mailman (input) for mailman id 680030;
 Tue, 13 Feb 2024 15:44:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZux6-0006UQ-KA
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:44:08 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b83655a2-ca86-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 16:44:06 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-411d76c9ce3so787055e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 07:44:06 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 hi7-20020a05600c534700b00410be9b8b96sm7509886wmb.37.2024.02.13.07.44.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 07:44:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b83655a2-ca86-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707839045; x=1708443845; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1Uyx6HzAtaWMW1Wl2nJI/9FPFKqQgwt7hBFmnn/vgPc=;
        b=SPyfV8Lr2Q6AMchCqv8X0WpNbXGt/kWbFzg64cnnnL2xt/TD9iVBV/71yC2E08oiql
         LiUvAHqGRb9x8ZFU85f3K8RuG9ZuehJ2m409ZvNXesqsflHLp0J/aOwIaiUfBuYkx8aR
         0haILqFOc7HcCCH7E56ET5pkxvwh/podfInvl4PZXrp7EMd7nOenSVJY8eQz3y7uV+0M
         ZrRa9Hv4ymChUnQuiAlYGnp+cwHpsQnOZ9JUjkPz0QYRAtbH55C0PJ4mvrK/XneC0yaZ
         9cM7nc2gSRXb8r66C4pjTaK3seLeE2vSB1+ABCTj8dN+mVLOoWLYAAdx/RwIfP0hDRgm
         vpig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707839045; x=1708443845;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1Uyx6HzAtaWMW1Wl2nJI/9FPFKqQgwt7hBFmnn/vgPc=;
        b=ZY/ygZkOaKbXHkTJvqiF1yME4369dXg0JxIo8Cw1JoQp1MFCyW+xVdSv0e8NMtzULZ
         2jn1PTEX6kLeNBzffnulUHblr2fIw89yEdIeCeI96qk+0X7Et9v8r++Jt9NpCd4wRWCz
         8oCN57ninPKkI2F73Vp0kLjTNyqDqtgRJhjYmLWPDxyJddYhFsHQb5AP7NoExbuG05dC
         m4IvUS0yPeGDBIs/J7KPW4p15mRgH/EUZe7LDKtYXLMJvS8I6a5rzlUWAzdAt2V+/FLg
         y4NgyYE3niGTUQTGULxPgllXdGBzM30P26EhBahHlIWGszpR3O+THHqMizjA+cz7uXmQ
         0uHw==
X-Forwarded-Encrypted: i=1; AJvYcCWU95QvO6/bRu5zFs0N2SbYj6YUtrqoqy5c4hvX91v2UIJ3FACxHEOKu5Oh6dO5cFl5gjnt6OmoCJB8rcZg//N9yzCrOSFsRy6LkhM3w3U=
X-Gm-Message-State: AOJu0Yw5fjFMF8b0DirgmJGLP8BWdCQlr9nD+o4aEelx3qLcWWet0SLS
	aB9AS8PdXOXzoLEBYRftoVQ6zTSxD2pRpZ5PzD3qv/bi9AnaVr92nmCLDU40HQ==
X-Google-Smtp-Source: AGHT+IGrWldOTkLMd59SXhe55YIx6EGkiHqT60aSWXgc7pF4uc5BZOQPWyQM/ESRdL8Pso+OF317iw==
X-Received: by 2002:adf:fa8d:0:b0:33c:e29c:69a1 with SMTP id h13-20020adffa8d000000b0033ce29c69a1mr719272wrr.56.1707839045684;
        Tue, 13 Feb 2024 07:44:05 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUJCGarz7YC7YbtDgciXIE0377vq9J7yc+UHhpLtxjHzkl9b8LtHhtB4R1cUfQ+OQsuHcddnuV/DrR+J1maRanJkUKzSE+c7pnJ7WZBWJuODpgaELbpWUjK7zu2NrI1S5itROl2bcxVWO7q7fA+Jr5YmWKNu1tm715mKmD4wPbhSWFzIfvKv9sqoKa/jEipPqLDrQxLq0eZGT+x05JPy0RVSVo/Tzk4pQ==
Message-ID: <23bcd506-f843-42f7-a541-0d8db7c319ab@suse.com>
Date: Tue, 13 Feb 2024 16:44:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Content-Language: en-US
To: Marek Marczykowski <marmarek@invisiblethingslab.com>
Cc: Julien Grall <julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
 <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
 <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com> <ZcrlcuNU9y8WymiK@mail-itl>
 <817ceec4-ed12-4360-a6ce-f87732f62251@suse.com> <ZcuCTnMpFib4ArHW@mail-itl>
 <b52fa3fa-4e33-4211-af18-f00aaa0b365b@suse.com> <ZcuGo_vQhiTAQep0@mail-itl>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZcuGo_vQhiTAQep0@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.02.2024 16:11, Marek Marczykowski wrote:
> On Tue, Feb 13, 2024 at 04:00:32PM +0100, Jan Beulich wrote:
>> On 13.02.2024 15:53, Marek Marczykowski wrote:
>>> On Tue, Feb 13, 2024 at 08:45:54AM +0100, Jan Beulich wrote:
>>>> On 13.02.2024 04:43, Marek Marczykowski wrote:
>>>>> On Mon, Feb 12, 2024 at 10:04:38AM +0100, Jan Beulich wrote:
>>>>>> On 08.02.2024 23:00, Julien Grall wrote:
>>>>>>> On 05/02/2024 13:27, Jan Beulich wrote:
>>>>>>>> In preparation of dropping the register parameters from
>>>>>>>> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
>>>>>>>> register state needs making available another way for the few key
>>>>>>>> handlers which need it. Fake IRQ-like state.
>>>>>>>>
>>>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>>>>>> ---
>>>>>>>> The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsistent with
>>>>>>>> other console poll functions we have, and it's unclear whether that's
>>>>>>>> actually generally correct.
>>>>>>>
>>>>>>> Is it? Looking at ns16550_poll() we would pass guest_user_regs() if 
>>>>>>> run_in_exception() doesn't exist. But looking at the caller, no-on seems 
>>>>>>> to care about the 'regs'. So is this just a latent bug?
>>>>>>
>>>>>> What do you mean by "doesn't exist"? ns16550_poll() assumes it exists.
>>>>>> And I can spot any use of guest_user_regs() on the respective generic
>>>>>> or Arm-specific bug.c paths.
>>>>>>
>>>>>>> BTW, do you have an idea why the poll function is not run in an 
>>>>>>> exception handler?
>>>>>>
>>>>>> "The poll function" being which one? If you mean the one in xhci-dbc.c
>>>>>> then that's why I had Cc-ed Marek. Moving him to To: - maybe that
>>>>>> manages to finally catch his attention.
>>>>>
>>>>> TBH, I don't know. That's part of the original xue patch at
>>>>> https://github.com/connojd/xue/blob/master/patches/xen-xue-dbgp.patch
>>>>> and it works for me as it is.
>>>>
>>>> "Works" meaning what? Doesn't crash on you? Or does also provide
>>>> sensible output in _all_ cases (i.e. including when e.g. the poll
>>>> happens to run on an idle vCPU)?
>>>
>>> Generally provides sensible output, for example during boot (it is using
>>> idle vCPU then, right?).
>>
>> Before Dom0 is started: Yes. With the exception of the phase where PV
>> Dom0's page tables are constructed, albeit in that time window
>> guest_cpu_user_regs() shouldn't yield sensible data either. I can only
>> say I'm surprised; since I have no way to properly test with an XHCI
>> debug port, I'd have to see about faking something to convince myself
>> (unless you were to supply example output).
> 
> Would you like me to test this series with xhci console?

The behavior shouldn't really be connected to this series. But yes, 'd'
debug key output (just the part for the CPU the key handling was
actually invoked from) with the xhci debug console would be of
interest, for the case where that CPU at that time runs an idle vCPU.

> Or maybe add
> some extra debug prints and include their output? But note, printk from
> inside console code generally leads to deadlocks. What I did for some
> debugging was to log into some separate buffer and dump it later.

Right, this would be more involved.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:51:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:51:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680035.1057860 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZv42-00083n-Pa; Tue, 13 Feb 2024 15:51:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680035.1057860; Tue, 13 Feb 2024 15:51:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZv42-00083g-My; Tue, 13 Feb 2024 15:51:18 +0000
Received: by outflank-mailman (input) for mailman id 680035;
 Tue, 13 Feb 2024 15:51:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZv41-00083a-PF
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:51:17 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b8257428-ca87-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 16:51:15 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-41166710058so13244845e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 07:51:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bn8-20020a056000060800b0033b198efbedsm10093929wrb.15.2024.02.13.07.51.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 07:51:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b8257428-ca87-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707839475; x=1708444275; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2i2aUtxxyZJ4vBmTsnj4Ousq0q5qhRJ9YTgfeT1TVAA=;
        b=B5gwgUfmktwHRmfHNjg6Z2lMTOmqj232C+CKxeQgKVnppZWQlsmTsdzOwdjsS87rwA
         jVEXLkZtd9xeP2uzyy5VLWcm07d2rALpqQj0Oi4gyUdg/QDPjlYbdyS1qkblEknSq9N9
         8bFX51qQuZLTaJmh2WZjaps6ANiZOW5BSQV5kmrfPtuJz/Khpj4dXsxRZYXsWBwTJluT
         QKRU77wJ8ST6ewwsxuid7IoutV5HJWuBXQxpsK26/PmtElzr4FL3wrbYt1xTiOCmc7sr
         4QTqRUhve5qO19FefDIpa8Dd9m7NbLDpp5xThR4IucMCSu3K/UE9WuulNyjgsazmAmyb
         O8zA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707839475; x=1708444275;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2i2aUtxxyZJ4vBmTsnj4Ousq0q5qhRJ9YTgfeT1TVAA=;
        b=aEI9Gafc+38F6YAmqLn+FZzMtENG/1kMDp939V2YKYFBcOjvjvYArsQ/W5tBRJnSOB
         pc9jC6hqhPdVWCXDPxMUrM4pjLHyWdMPM75pNxydw42NyRAS+yP83dMrweSeJW9l/0BJ
         +f6Znd/lJ0o9eT6MBA3+sl7xA7mx/1T/asKncRxPVvgTbsKUDjh6uC31zBbaNtQNDO62
         4QgFZk4ZHK79muDNn/4kMnVbX07cZjsvQK8AqqXtZbdBQ7SXIhKAeu8KGnUJ+88fc4mn
         i1/++Yu2rpgdiXdf3G7AqELeAK2Xcr0BhfttANz2pMFzbXnU0WctvInngS2ZwcHVwAWR
         kO1A==
X-Forwarded-Encrypted: i=1; AJvYcCXp/4TsLM55+hSyIUeO34bnYPvolA9+asKBUjKtQKGmGDmRmNjg0X7A3dbS/FFbUogCwMM6ovGl0ooVuBXFtniUeNnX9/mJbvxzO760GXE=
X-Gm-Message-State: AOJu0Yw6z1PxOkEM+qG8EYkojeRV9/V6rjTjenjmFMaWN8ZkGUphCFLv
	9LCo+yheXjwGHNIF/jzwgE48fQ0D8Eu37wYVSi97rs/WPnRiUTqU03VaYiB4eg==
X-Google-Smtp-Source: AGHT+IHIMkvCzVWYYMDIAdMujc8qZuROpTA11IDUBbei6qnLnq+MmCoUVMiJgUjgROFdJxiaMGLb5g==
X-Received: by 2002:a05:600c:1549:b0:410:f582:6675 with SMTP id f9-20020a05600c154900b00410f5826675mr28695wmg.26.1707839475070;
        Tue, 13 Feb 2024 07:51:15 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWKLdOcWVRZCP7QRXi2ZQLUChxHB7DqjozFgnYcpHXwbEWEJWBvGLS9HhgdSv9SeOco86Tia5Z2PjyroHGklsQCJHivy/O5D33B2u0R0xy+ReuPe9LYiqD+kXIfooY9ul20kiWsTz8bjjCy8No9xLoVoYO5CDVu7p8VjW2DLC/7+tI2ZbFI+B8nxUwKj2JDIug7G3WyPE6v40E0rmiiinZqIa0FdXHUbcfVedNV/C7GrKQ7CORPWiww4UfPTPP7v3HX3t6r16w1mA0ssyc2UD7p/lNi9lB/VPnLKgi73L2jRaUyR18/FANMv7HVxpZOehSviDNU4ueembdYXRANETC+7Gbn8Y8MEAMfwuWV8e2VNugrH0lwkJlpmTw00R9SPt1UvP9hdp4crypBR/2tpP6W3CfzRNXcYvDR2q8f9USpt1N3ut4YosuUGq74BmygevuigpOR
Message-ID: <ab71e3e7-c032-454a-b9eb-4a8e9e064b8c@suse.com>
Date: Tue, 13 Feb 2024 16:51:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 1/3] xen: introduce Kconfig function alignment option
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-2-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240207145547.89689-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 15:55, Roger Pau Monne wrote:
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -29,6 +29,7 @@ config X86
>  	select HAS_UBSAN
>  	select HAS_VPCI if HVM
>  	select NEEDS_LIBELF
> +	select FUNCTION_ALIGNMENT_16B

With the insertion here as well as for Arm and PPC obeying alphabetic
sorting:
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:52:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:52:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680038.1057869 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZv4s-00006a-4x; Tue, 13 Feb 2024 15:52:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680038.1057869; Tue, 13 Feb 2024 15:52:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZv4s-00006T-2L; Tue, 13 Feb 2024 15:52:10 +0000
Received: by outflank-mailman (input) for mailman id 680038;
 Tue, 13 Feb 2024 15:52:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gnrE=JW=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZv4q-0008Ou-UM
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:52:08 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d77a4329-ca87-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 16:52:08 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33b2960ff60so3179737f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 07:52:08 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 k23-20020a5d5257000000b0033b444a39a9sm9812280wrc.54.2024.02.13.07.52.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 07:52:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d77a4329-ca87-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707839527; x=1708444327; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=gn/Nq45doFN+kZvWV6FkvFrv+P+quhq51iNM/ndYjPQ=;
        b=QVzWV9VYDxBX2EahYst3dLjP71x3k0JmyDfceba2rTLbJ8W4pmm8uL1n6c73jhx8rS
         HEgScyzGtxi0n0kemTb3DtLO52DUMiiF+c32nr6tY8qtH77yAI8hnwUxdB5u3BgoKC/Y
         Cuf10X+vG9+0oYgli7x/J74X4rj9SydGFqwUM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707839527; x=1708444327;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gn/Nq45doFN+kZvWV6FkvFrv+P+quhq51iNM/ndYjPQ=;
        b=JJ9SYYpoIZICdl/qbpD/fGtJjtgHDiwr4zLqKbxwa29RUH+LyOPBk5WQErB6RRKIKg
         ddVnnNx4shstBQhAoPgQya+07zuqiIH6R+xh9BiSDxYQymIAby0ApH0z2WK8mqzU/jPF
         8RRJBrTl3G0g3DUU5NS4jvj/v9VU1I/bNVXXDB0pz+Yq2g1zEj0QufKMRY/IwHL9I/CN
         2VxgkgTmi/tND0TXQofdNlwXN/ugO9BhD26IvsSwwCaz3U0C8DGoejpLtD34QaAJm8hH
         00FLTCrNLQBqZty08zdToe5AuspnlBVtTbkXX/tjElKiqa6gVmw4W8c1wh4wP/yXUEbp
         Jplg==
X-Gm-Message-State: AOJu0YxQlbY+bVo6FDz/NzooHJjeAJL0+AVhST9owL4Nr5VP9px/bt73
	Z9AfXDLEju37c2hpmHXDVvhRoPoXqZNYnbIiSRucuw+pRkCw5hgIbc3EbgRG8EhMRZx7OeuKntI
	P
X-Google-Smtp-Source: AGHT+IGlLEezUnFM7ZXj8c0YYVImsOl706OxW525doLLnWSi59JwpxLpMdyDbFIG/25gDMymh48KHQ==
X-Received: by 2002:adf:b351:0:b0:33c:e310:8e3e with SMTP id k17-20020adfb351000000b0033ce3108e3emr476390wrd.10.1707839527490;
        Tue, 13 Feb 2024 07:52:07 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWi1FFZhYHgXC6bdcEAvAkjbfKbFImdnyldR4WZsJx9BNo8nH0yFNlyHqFxJl5U9W7UZr9Y81a0S+BL
Date: Tue, 13 Feb 2024 15:52:06 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: zithro / Cyril =?iso-8859-1?Q?R=E9bert?= <slack@rabbit.lu>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v3] tools/xentop: add option to display dom0 first
Message-ID: <7c4a7031-828a-4d4a-bb85-a5250cf2545e@perard>
References: <92ef4d230e05970e1d685b03125fce44adc55010.1707331801.git.slack@rabbit.lu>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <92ef4d230e05970e1d685b03125fce44adc55010.1707331801.git.slack@rabbit.lu>

On Wed, Feb 07, 2024 at 08:02:00PM +0100, zithro / Cyril Rbert wrote:
> From: Cyril Rbert <slack@rabbit.lu>
> 
> Add a command line option to xentop to be able to display dom0 first, on top of the list.
> This is unconditional, so sorting domains with the S option will also ignore dom0.
> 
> Signed-off-by: Cyril Rbert (zithro) <slack@rabbit.lu>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:57:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:57:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680053.1057884 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZv9o-0001FC-NP; Tue, 13 Feb 2024 15:57:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680053.1057884; Tue, 13 Feb 2024 15:57:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZv9o-0001F5-Kn; Tue, 13 Feb 2024 15:57:16 +0000
Received: by outflank-mailman (input) for mailman id 680053;
 Tue, 13 Feb 2024 15:57:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZv9n-0001Ev-AH; Tue, 13 Feb 2024 15:57:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZv9n-0002Cc-6k; Tue, 13 Feb 2024 15:57:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZv9m-0006jJ-Py; Tue, 13 Feb 2024 15:57:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZv9m-0000Zu-PN; Tue, 13 Feb 2024 15:57:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OuF47j3BjjkFjOygJUlQRCd/4vsqXUpDdu1rRgt8pUU=; b=cmlVdnVJHuQmnV7vf9qtdkW6fx
	a8Vn33qEd62K4tgTfrCbbVqqgOiRZeFjVdbOW9HNLeSmMXEU8zJM6AaDLEnc5eq7iC2WWxEwcLzQ0
	n+ERntPzREK4/tIgDcESOWeIpI4O9hwzjaJEPzOJB3wHWPnjlFUaH6ktYczWEhcvgaWQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184655-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184655: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=94365a48710227a5fec836baadd12d8ff5a5fe8a
X-Osstest-Versions-That:
    libvirt=bdece5518d36b70ab18cf0157cb326025c07fe34
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 13 Feb 2024 15:57:14 +0000

flight 184655 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184655/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184642
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184642
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184642
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              94365a48710227a5fec836baadd12d8ff5a5fe8a
baseline version:
 libvirt              bdece5518d36b70ab18cf0157cb326025c07fe34

Last test of basis   184642  2024-02-10 04:20:51 Z    3 days
Testing same since   184655  2024-02-13 04:20:35 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrea Bolognani <abologna@redhat.com>
  Jonathon Jongsma <jjongsma@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   bdece5518d..94365a4871  94365a48710227a5fec836baadd12d8ff5a5fe8a -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 15:58:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 15:58:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680057.1057894 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZvBO-0001sT-2x; Tue, 13 Feb 2024 15:58:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680057.1057894; Tue, 13 Feb 2024 15:58:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZvBN-0001sM-WE; Tue, 13 Feb 2024 15:58:54 +0000
Received: by outflank-mailman (input) for mailman id 680057;
 Tue, 13 Feb 2024 15:58:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=W/Jz=JW=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rZvBM-0001sG-7n
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 15:58:52 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c7016564-ca88-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 16:58:50 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-411a5a86078so11383415e9.2
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 07:58:50 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020a05600c4f8a00b0040e4733aecbsm12237247wmq.15.2024.02.13.07.58.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 07:58:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7016564-ca88-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707839929; x=1708444729; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=46qlRpKLlGDPBxcSwlRclGnt3SqxTcaij+FFoWn28VE=;
        b=WKsciw83xvrxx9aONVc4qIigXiWwmCJTJtBI+/Vw4lGyrCgWsoBCpDgSwwUGB4L7E8
         ZtZ497N9WdCiMEYMSxxnKBuZ7Yfeu026X8nwHgNcb/RojVDEQzc1grodVCCJBa8OlKxq
         i3MOx5AIpqjIUD6UJNVEEkdn4GHkSlAUwdg5IDnU4jroD832bez3dsSuQpg8jXlNWd+P
         CvChG6OyVfEyUrXUyDLB3zBE/IB7o3s0hA8xA9MjFvPbrJQ7o56c8zIukR18XCOp1Suf
         wRJ9/IkW2lrxURfLqS2p8QjRW7n+4wFDX8e78kE1g/uDmRMrJ/7RIzo/zpEcpKpsRMrw
         fhrA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707839929; x=1708444729;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=46qlRpKLlGDPBxcSwlRclGnt3SqxTcaij+FFoWn28VE=;
        b=dkbJ+EAr0T9XGlr1WbxousPRsZuN5eFrjc1Mn0CNK9yFu5BuseWXdjyJMVmuYCOlCd
         zlXzR2i5L/leVE/oMBc6imwWpCzpcbnRvwuo+QwIXPJoud9EZmDvGSHKYp6IYezCtoKj
         98A7PW4gtVEmU3+Wg4ABjZHOWkQcHVT4medgZ1Xmb1B/TsQLW1tDXgoeIUvFwUtK67VT
         yHpQw7uAYL9xiFeKi+N7VosRWl30qX48iQsIJoyhOwYDTv+jHUYXlo/b5UeAh+uQVaeC
         Eaosp8VjdiQNkIf+/2WgmiMMGIcgjAfInMwjSKHT+Tv89Tv4QS+5tsEXZSMwBO/bF3gg
         k/Ow==
X-Forwarded-Encrypted: i=1; AJvYcCVao7FvZ5ILicqywVpZY6yKUVp1Oe4NC5OL/t6kAClG0gxQFUYV5LCCPR0H9WtvAjHcqDPjZxAtc5MbWddE/4JHa6MNy+jo7XHAi0Q7eXU=
X-Gm-Message-State: AOJu0YwBB/o0CDQTlzwKXbmRDNqRgMmSCEFEehB2tsNC0haJUN0WGsTh
	m3hcWk6yYwVfxgGf1Kft8+XI7hCWHgyd9L3Hb29IbGnizDtccv9XsoLRum+0lA==
X-Google-Smtp-Source: AGHT+IHk1ZBAYmLVzQ/Acr3FJP59lDkjl6eW+oyCgTX5SwY7MqG0wCbYKTlYBInV82yN6WgpOPmeRw==
X-Received: by 2002:a05:600c:3b26:b0:410:e860:c01f with SMTP id m38-20020a05600c3b2600b00410e860c01fmr60169wms.1.1707839929480;
        Tue, 13 Feb 2024 07:58:49 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVq8Kx/M2Wb5E9YaMlrrrxvXKpp7GCaBLqiP3iqPztSOig55ZC3p6ToVpWeqtN03biH6d48YBnGkN31San7L5OaMsEsa2zpIC/r2q6ZVGyudpQbztGXk0WGJQVqqWYijxR/80lHFyDa/zrf6l8nXr/3ZjTxrMK1cBCrxqHbiNsh7yqMtjQdZncnyo+7BLixg0aDFvkqhkilXsqPH8TGSQ4LiEk43DHsrqhufQTyDtsihExsRar83cQHVSCU4GlMeqv3s2DK5ttU8RDsc5x3nABbcUD/4Thi77JhAdthDeK/LNq/pAGYuPrMaxLSvmvFEH3l7zAkbh4ll4j/8bDvpuH/cvpGTn9WErZd6whYFPViHirJ0p87UqWEi9hDw+w6xd7E5NE1odw6LcqI6NEdeZHsH+fys0e0t++Wvw6f1C+jsMKX/AVnNrPdQGJS854IdojWxxE6D5moRyPsOHe8ARDNSg6Yi6BktoXyS9iBS6IDdFbOaGTo9RmFr1b0BWQbnX6xiVX6QRblNoqp
Message-ID: <670e4a5e-4eec-4a2c-9ed1-fcc164b1d76c@suse.com>
Date: Tue, 13 Feb 2024 16:58:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 3/3] xen/livepatch: align functions to ensure minimal
 distance between entry points
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-4-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240207145547.89689-4-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 07.02.2024 15:55, Roger Pau Monne wrote:
> The minimal function size requirements for an x86 livepatch are either 5 bytes
> (for jmp) or 9 bytes (for endbr + jmp), and always 4 bytes on Arm.  Ensure that
> distance between functions entry points is always at least of the minimal
> required size for livepatch instruction replacement to be successful.
> 
> Add an additional align directive to the linker scripts, in order to ensure that
> the next section placed after the .text.* (per-function sections) is also
> aligned to the required boundary, so that the distance of the last function
> entry point with the next symbol is also of minimal size.

Perhaps "... minimal required size"?

> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -395,8 +395,11 @@ config CRYPTO
>  config LIVEPATCH
>  	bool "Live patching support"
>  	default X86
> -	depends on "$(XEN_HAS_BUILD_ID)" = "y"
> +	depends on "$(XEN_HAS_BUILD_ID)" = "y" && CC_HAS_FUNCTION_ALIGNMENT
>  	select CC_SPLIT_SECTIONS
> +	select FUNCTION_ALIGNMENT_16B if XEN_IBT
> +	select FUNCTION_ALIGNMENT_8B  if X86
> +	select FUNCTION_ALIGNMENT_4B  if ARM

This isn't strictly needed, is it? Would be nice to avoid re-selection
of what the default for an arch is anyway, as otherwise this will start
looking clumsy when a couple more architectures are added. Preferably
with that dropped (or it being clarified why it's still desirable to
have):
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 16:29:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 16:29:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680073.1057908 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZvf5-0000Uu-Fa; Tue, 13 Feb 2024 16:29:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680073.1057908; Tue, 13 Feb 2024 16:29:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZvf5-0000Un-Bo; Tue, 13 Feb 2024 16:29:35 +0000
Received: by outflank-mailman (input) for mailman id 680073;
 Tue, 13 Feb 2024 16:29:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZvf3-0000TD-9c; Tue, 13 Feb 2024 16:29:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZvf3-0003I3-6q; Tue, 13 Feb 2024 16:29:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZvf2-0007V6-Qf; Tue, 13 Feb 2024 16:29:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZvf2-0002Re-Q5; Tue, 13 Feb 2024 16:29:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TSFCUZOQKsSfkks2dOFPusDGggwxmoZCLqsEbGTxLTw=; b=Tfbg9lGeibc2DUG3F1j7K/c3sf
	mwHcrpQRs6t4HZ0qUbzRO87Vy5cnuEyIr9/HBFEd9zwIameeCsnwK2snhpXnLNEbtLcHssDJT6QtW
	yr70dBBL6YoL36hAvPA7xIo2Jej4NzoLqZbg8cRIqEwcd2wXb6vZD1L9riOkK2qBWf1w=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184658-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184658: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=b3bea912a737a0db39a89b0da0033c4bfda82f03
X-Osstest-Versions-That:
    xen=a185f88e9858987e05d91eb3e9bda68d4a6b4f1d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 13 Feb 2024 16:29:32 +0000

flight 184658 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184658/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  b3bea912a737a0db39a89b0da0033c4bfda82f03
baseline version:
 xen                  a185f88e9858987e05d91eb3e9bda68d4a6b4f1d

Last test of basis   184657  2024-02-13 09:02:20 Z    0 days
Testing same since   184658  2024-02-13 14:00:26 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   a185f88e98..b3bea912a7  b3bea912a737a0db39a89b0da0033c4bfda82f03 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 16:57:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 16:57:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680081.1057919 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZw66-0005Y0-42; Tue, 13 Feb 2024 16:57:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680081.1057919; Tue, 13 Feb 2024 16:57:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZw65-0005Xt-VT; Tue, 13 Feb 2024 16:57:29 +0000
Received: by outflank-mailman (input) for mailman id 680081;
 Tue, 13 Feb 2024 16:57:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SLZZ=JW=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rZw64-0005Xn-RK
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 16:57:29 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:240a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f6433a8d-ca90-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 17:57:26 +0100 (CET)
Received: from BYAPR05CA0042.namprd05.prod.outlook.com (2603:10b6:a03:74::19)
 by DM6PR12MB4548.namprd12.prod.outlook.com (2603:10b6:5:2a1::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25; Tue, 13 Feb
 2024 16:57:21 +0000
Received: from CO1PEPF000044F4.namprd05.prod.outlook.com
 (2603:10b6:a03:74:cafe::34) by BYAPR05CA0042.outlook.office365.com
 (2603:10b6:a03:74::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26 via Frontend
 Transport; Tue, 13 Feb 2024 16:57:20 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CO1PEPF000044F4.mail.protection.outlook.com (10.167.241.74) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 13 Feb 2024 16:57:20 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 10:57:18 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 10:57:18 -0600
Received: from [172.27.131.30] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 13 Feb 2024 10:57:16 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6433a8d-ca90-11ee-8a4d-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EjikUuXkg8CEtizKiAHEG97sG4p+pQ/F5wIhzcFINQzn9t5PqQfM/Zkwi5xwN+cLcTLfHXpgEGm7n9FKpsxi99vd/1oDQ+ENg0atURzppSezVcFg0MrZrBtm1fZFiO2sFf6U4ZPcVshVEzKVoYTzFeBxVMvFHfNdoT9yYSim/nhVD7oPlXPYyTbhldjYv0zlJnZbCHlW3Y8l3xJZo3aMHgdNLAG4hfv60OUIupOfMKiCfLjx+MiI2aAJLMt/yAqOriZwcZCX8eXNlDkUaqJ+ROdVnF/uTRAxf1CIp5Miz6W3ogjQA/FH+muk7k8UDJRe74SBVJUQnxsrNzJYbQ3hDQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=tZFUcy/BkWZSpm4tijbRMhFxw02Dw2CSWZd7dmO1ADQ=;
 b=OpEVc2WYKG7wVhX7YuxT8hPA2r9NBir/Gcf5hJiHwT+20Zl/X+EEyHXx3eRog9Sn7wtHx55KGEGZrdpMTVNEE6pZD3Gtbxe6ykLkaj05FfJ9y1GXJaLY3/EOFxprnZQD/yLsMSnfPF80aIEZQ4+aYDNyRF2Doize49jdZekDVGB55HLBacele6k0cNIMC7TmRZDHBmGpAgyTQYdW7fkXmBmzbYo9ZNQ9sZ2mNx9wGN7He/7sC1KIA/4tK4qp9tGZ1TJS2lvwoPkxneuuzRi0n0IC2/k1osAsC+zKpz4axALvJxkPYt4Aqc/Gia6f5f/1NIFNS63vH4xmB561sKNH+A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=citrix.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=tZFUcy/BkWZSpm4tijbRMhFxw02Dw2CSWZd7dmO1ADQ=;
 b=o1/MS5zHrIHngsLygCAEBmyOTJQxVyxTv8kXjBxO2Z2609vCRLWH1RtRZ2CasFroFhdPXYECL+IrK7JNyPaa6EMuyf3PEJeMHs5hCftsbP+4aDD4Eit6tpKzLlRl6SweSIFkeNORRXOd+2uUILFbAiDsqUrmU3y+xkD+4vmP3K8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <e5342d84-f527-4d50-81be-600b7aafbf4d@amd.com>
Date: Tue, 13 Feb 2024 11:57:16 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
CC: <xen-devel@lists.xenproject.org>, Oleksandr Andrushchenko
	<oleksandr_andrushchenko@epam.com>, Jan Beulich <jbeulich@suse.com>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian
	<kevin.tian@intel.com>, Paul Durrant <paul@xen.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <Zcsp15Aqve11Icjb@macbook>
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <Zcsp15Aqve11Icjb@macbook>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CO1PEPF000044F4:EE_|DM6PR12MB4548:EE_
X-MS-Office365-Filtering-Correlation-Id: ddcae1c9-9f10-42ae-5d26-08dc2cb4d7ae
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8d+jv+F0d6mM0IVw4/k/8XDY0uXi4+u5lwVlg08Mub7HRD4fKhzLZyQs/g+SaphmhkH6HF21omI6LqqXRFMiBE9fQm5mCaLLADENLfr4WeB87Bg1e7YD3fGjm9KggfxWLJXNAgNA71osF9Id6TpsImVWhI4d/Hju1fD6MQuIVRlUtQ4IEX4IrN7CVLqoGcEVtuG/PWp8zKVM1LuEvpJWeru7iCPPlR1aAx4WxU4n8511hV6Q8ovMhow4D2wd9Y7uqaTIxn2o/lfe6LsZIj+5wesT6u9EtGkSTz7Dek8pog4ctf+s8JaT6kBANn0kv4YVKOUiCodVV2sjImo02P4sVIJwu4boGd8TwvJt9C6yFteqdED6VDj4a52uikXBq0NNn3Xx+6Wj2XqFw1vrmYucv2KilhETBDiASNzGuw4Y8xuYjUChE0BCaGraTrfmnnLladBc0SzrTMJVawFjCjWLhE932ewE0v4XFp7vWsoGf+a9jqXZ6gBHxKATIcyf7d4tNnkTSZTtH68cLm5sjZnANriAkBVuKly68OfMrFWsBWYnmzZXuDKr3aU+cVOIFfT9Cf3iBdUwX0zRdzj7+9FGavxL2u12m1K8rL3n6JXUwx4=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(396003)(376002)(346002)(39860400002)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(82310400011)(36840700001)(46966006)(40470700004)(30864003)(44832011)(2906002)(8676002)(8936002)(7416002)(5660300002)(4326008)(336012)(426003)(2616005)(83380400001)(26005)(36756003)(82740400003)(81166007)(356005)(86362001)(31696002)(53546011)(54906003)(16576012)(70586007)(316002)(70206006)(6916009)(478600001)(31686004)(41300700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 16:57:20.2469
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ddcae1c9-9f10-42ae-5d26-08dc2cb4d7ae
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CO1PEPF000044F4.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4548

On 2/13/24 03:35, Roger Pau Monné wrote:
> On Fri, Feb 02, 2024 at 04:33:05PM -0500, Stewart Hildebrand wrote:
>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>
>> Use the per-domain PCI read/write lock to protect the presence of the
>> pci device vpci field. This lock can be used (and in a few cases is used
>> right away) so that vpci removal can be performed while holding the lock
>> in write mode. Previously such removal could race with vpci_read for
>> example.
>>
>> When taking both d->pci_lock and pdev->vpci->lock, they should be
>> taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
>> possible deadlock situations.
>>
>> 1. Per-domain's pci_lock is used to protect pdev->vpci structure
>> from being removed.
>>
>> 2. Writing the command register and ROM BAR register may trigger
>> modify_bars to run, which in turn may access multiple pdevs while
>> checking for the existing BAR's overlap. The overlapping check, if
>> done under the read lock, requires vpci->lock to be acquired on both
>> devices being compared, which may produce a deadlock. It is not
>> possible to upgrade read lock to write lock in such a case. So, in
>> order to prevent the deadlock, use d->pci_lock in write mode instead.
>>
>> All other code, which doesn't lead to pdev->vpci destruction and does
>> not access multiple pdevs at the same time, can still use a
>> combination of the read lock and pdev->vpci->lock.
>>
>> 3. Drop const qualifier where the new rwlock is used and this is
>> appropriate.
>>
>> 4. Do not call process_pending_softirqs with any locks held. For that
>> unlock prior the call and re-acquire the locks after. After
>> re-acquiring the lock there is no need to check if pdev->vpci exists:
>>  - in apply_map because of the context it is called (no race condition
>>    possible)
>>  - for MSI/MSI-X debug code because it is called at the end of
>>    pdev->vpci access and no further access to pdev->vpci is made
>>
>> 5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
>> while accessing pdevs in vpci code.
>>
>> 6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
>> do not go away. The vPCI functions call several MSI-related functions
>> which already have existing non-vPCI callers. Change those MSI-related
>> functions to allow using either pcidevs_lock() or d->pci_lock for
>> ensuring pdevs do not go away. Holding d->pci_lock in read mode is
>> sufficient. Note that this pdev protection mechanism does not protect
>> other state or critical sections. These MSI-related functions already
>> have other race condition and state protection mechanims (e.g.
>> d->event_lock and msixtbl RCU), so we deduce that the use of the global
>> pcidevs_lock() is to ensure that pdevs do not go away. Existing non-vPCI
>> callers of these MSI-related functions will remain (ab)using the global
>> pcidevs_lock() to ensure pdevs do not go away so as to minimize changes
>> to existing non-vPCI call paths.
>>
>> 7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
>> that pdevs do not go away. The purpose of this wrapper is to aid
>> readability and document the intent of the pdev protection mechanism.
> 
> I would add that when possible, the existing callers haven't been
> switched to use the newly introduced per-domain pci_lock, and will
> continue to use the global pcidevs lock.  This is done to reduce the
> risk of the new locking scheme introducing regressions.  Those users
> will be adjusted in due time.

I'll use this wording, thanks

> 
> IIRC Jan had concerns about why some existing use-cases are not
> switched straight to use the new per-domain pci_lock in this patch.

I hope the clarified commit description addresses this

> 
>>
>> Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
>> Suggested-by: Jan Beulich <jbeulich@suse.com>
>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
>> ---
>> Changes in v13:
>>  - hold off adding Roger's R-b tag even though it was provided on v12.2
>>  - use a wrapper construct to ease readability of odd-looking ASSERTs
>>  - new placement of ASSERT in __pci_enable_msix(), __pci_enable_msi(),
>>    and pci_enable_msi(). Rearrange/add pdev NULL check.
>>  - expand commit description with details about using either
>>    pcidevs_lock() or d->pci_lock
>>
>> Changes in v12.2:
>>  - drop Roger's R-b
>>  - drop both locks on error paths in vpci_msix_arch_print()
>>  - add another ASSERT in vpci_msix_arch_print(), to enforce the
>>    expectation both locks are held before calling vpci_msix_arch_print()
>>  - move pdev_done label in vpci_dump_msi()
>>  - update comments in vpci_dump_msi() to say locks (plural)
>>
>> Changes in v12.1:
>>  - use read_trylock() in vpci_msix_arch_print()
>>  - fixup in-code comments (revert double space, use DomXEN) in
>>    vpci_{read,write}()
>>  - minor updates in commit message
>>  - add Roger's R-b
>>
>> Changes in v12:
>>  - s/pci_rwlock/pci_lock/ in commit message
>>  - expand comment about scope of pci_lock in sched.h
>>  - in vpci_{read,write}, if hwdom is trying to access a device assigned
>>    to dom_xen, holding hwdom->pci_lock is sufficient (no need to hold
>>    dom_xen->pci_lock)
>>  - reintroduce ASSERT in vmx_pi_update_irte()
>>  - reintroduce ASSERT in __pci_enable_msi{x}()
>>  - delete note 6. in commit message about removing ASSERTs since we have
>>    reintroduced them
>>
>> Changes in v11:
>>  - Fixed commit message regarding possible spinlocks
>>  - Removed parameter from allocate_and_map_msi_pirq(), which was added
>>  in the prev version. Now we are taking pcidevs_lock in
>>  physdev_map_pirq()
>>  - Returned ASSERT to pci_enable_msi
>>  - Fixed case when we took read lock instead of write one
>>  - Fixed label indentation
>>
>> Changes in v10:
>>  - Moved printk pas locked area
>>  - Returned back ASSERTs
>>  - Added new parameter to allocate_and_map_msi_pirq() so it knows if
>>  it should take the global pci lock
>>  - Added comment about possible improvement in vpci_write
>>  - Changed ASSERT(rw_is_locked()) to rw_is_write_locked() in
>>    appropriate places
>>  - Renamed release_domain_locks() to release_domain_write_locks()
>>  - moved domain_done label in vpci_dump_msi() to correct place
>> Changes in v9:
>>  - extended locked region to protect vpci_remove_device and
>>    vpci_add_handlers() calls
>>  - vpci_write() takes lock in the write mode to protect
>>    potential call to modify_bars()
>>  - renamed lock releasing function
>>  - removed ASSERT()s from msi code
>>  - added trylock in vpci_dump_msi
>>
>> Changes in v8:
>>  - changed d->vpci_lock to d->pci_lock
>>  - introducing d->pci_lock in a separate patch
>>  - extended locked region in vpci_process_pending
>>  - removed pcidevs_lockis vpci_dump_msi()
>>  - removed some changes as they are not needed with
>>    the new locking scheme
>>  - added handling for hwdom && dom_xen case
>> ---
>>  xen/arch/x86/hvm/vmsi.c       | 31 +++++++++++++--------
>>  xen/arch/x86/hvm/vmx/vmx.c    |  2 +-
>>  xen/arch/x86/irq.c            |  8 +++---
>>  xen/arch/x86/msi.c            | 20 +++++++++-----
>>  xen/arch/x86/physdev.c        |  2 ++
>>  xen/drivers/passthrough/pci.c |  9 +++---
>>  xen/drivers/vpci/header.c     | 18 ++++++++++++
>>  xen/drivers/vpci/msi.c        | 30 +++++++++++++++++---
>>  xen/drivers/vpci/msix.c       | 52 ++++++++++++++++++++++++++++++-----
>>  xen/drivers/vpci/vpci.c       | 24 ++++++++++++++--
>>  xen/include/xen/sched.h       | 15 +++++++++-
>>  11 files changed, 170 insertions(+), 41 deletions(-)
>>
>> diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
>> index 128f23636279..f29089178a59 100644
>> --- a/xen/arch/x86/hvm/vmsi.c
>> +++ b/xen/arch/x86/hvm/vmsi.c
>> @@ -468,7 +468,7 @@ int msixtbl_pt_register(struct domain *d, struct pirq *pirq, uint64_t gtable)
>>      struct msixtbl_entry *entry, *new_entry;
>>      int r = -EINVAL;
>>  
>> -    ASSERT(pcidevs_locked());
>> +    ASSERT(pdev_list_is_read_locked(d));
>>      ASSERT(rw_is_write_locked(&d->event_lock));
>>  
>>      if ( !msixtbl_initialised(d) )
>> @@ -538,7 +538,7 @@ void msixtbl_pt_unregister(struct domain *d, struct pirq *pirq)
>>      struct pci_dev *pdev;
>>      struct msixtbl_entry *entry;
>>  
>> -    ASSERT(pcidevs_locked());
>> +    ASSERT(pdev_list_is_read_locked(d));
>>      ASSERT(rw_is_write_locked(&d->event_lock));
>>  
>>      if ( !msixtbl_initialised(d) )
>> @@ -684,7 +684,7 @@ static int vpci_msi_update(const struct pci_dev *pdev, uint32_t data,
>>  {
>>      unsigned int i;
>>  
>> -    ASSERT(pcidevs_locked());
>> +    ASSERT(rw_is_locked(&pdev->domain->pci_lock));
> 
> Any reason to not use the newly introduced helper here?  I know the
> pcidevs will never be locked here given the new lock usage, but still
> it would be less confusing if the new helper was used consistently.

Nope. Agreed, it would help readability, I'll switch to the helper. In
addition to vpci_msi_update(), I assume this comment also applies to the
remaining occurrences:

xen/arch/x86/hvm/vmsi.c:vpci_msi_arch_update()
xen/arch/x86/hvm/vmsi.c:vpci_msi_arch_enable()
xen/arch/x86/hvm/vmsi.c:vpci_msi_disable()
xen/arch/x86/hvm/vmsi.c:vpci_msix_arch_enable_entry()
xen/drivers/vpci/msix.c:msix_find()

But not:

xen/arch/x86/hvm/vmsi.c:vpci_msix_arch_print()

I'll add a suitable comment to this one exception.

> 
> Otherwise we need a comment here as to why the helper can't be used,
> in order to avoid confusion in the future.
> 
>>> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
>> index 9da91e0e6244..c3adec1aca3c 100644
>> --- a/xen/include/xen/sched.h
>> +++ b/xen/include/xen/sched.h
>> @@ -462,7 +462,8 @@ struct domain
>>  #ifdef CONFIG_HAS_PCI
>>      struct list_head pdev_list;
>>      /*
>> -     * pci_lock protects access to pdev_list.
>> +     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
>> +     * structure from being removed.
>>       *
>>       * Any user *reading* from pdev_list, or from devices stored in pdev_list,
>>       * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,
>> @@ -628,6 +629,18 @@ struct domain
>>      unsigned int cdf;
>>  };
>>  
>> +/*
>> + * Check for use in ASSERTs to ensure that:
>> + *   1. we can *read* d->pdev_list
>> + *   2. pdevs (belonging to this domain) do not go away
>> + *   3. pdevs (belonging to this domain) do not get assigned to other domains
> 
> I think you can just state that this check ensures there will be no
> changes to the entries in d->pdev_list, but not the contents of each
> entry.  No changes to d->pdev_list already ensures not devices can be
> deassigned or removed from the system, and obviously makes the list
> safe to iterate against.

OK, I'll simplify the comment

> 
> I would also drop the explicitly mention this is intended for ASSERT
> usage: there's nothing specific in the code that prevents it from
> being used in other places (albeit I think that's unlikely).
> 
>> + * This check is not suitable for protecting other state or critical regions.
>> + */
>> +#define pdev_list_is_read_locked(d) ({                           \
> 
> I would be tempted to drop at least the '_read_' part from the name,
> the name is getting a bit too long for my taste.
> 
>> +        struct domain *d_ = (d);                                 \
> 
> Why do you need this local domain variable?  Can't you use the d
> parameter directly?
> 
> Such assign will prevent using a const 'd' parameter, and 'd_' itself
> should be const IMO (iff we really need this).
> 
> Also sched.h is not the best place, can't you just place it in
> pci.h?

Yes, I'll move it to xen/include/xen/pci.h


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 16:58:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 16:58:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680084.1057928 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZw72-000674-Ep; Tue, 13 Feb 2024 16:58:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680084.1057928; Tue, 13 Feb 2024 16:58:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZw72-00066x-Bn; Tue, 13 Feb 2024 16:58:28 +0000
Received: by outflank-mailman (input) for mailman id 680084;
 Tue, 13 Feb 2024 16:58:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SLZZ=JW=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rZw71-0005tv-K7
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 16:58:27 +0000
Received: from NAM04-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2408::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 19d03c51-ca91-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 17:58:25 +0100 (CET)
Received: from DM6PR03CA0068.namprd03.prod.outlook.com (2603:10b6:5:100::45)
 by SA1PR12MB8142.namprd12.prod.outlook.com (2603:10b6:806:334::5) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25; Tue, 13 Feb
 2024 16:58:20 +0000
Received: from DS2PEPF0000343E.namprd02.prod.outlook.com
 (2603:10b6:5:100:cafe::89) by DM6PR03CA0068.outlook.office365.com
 (2603:10b6:5:100::45) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.41 via Frontend
 Transport; Tue, 13 Feb 2024 16:58:20 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS2PEPF0000343E.mail.protection.outlook.com (10.167.18.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 13 Feb 2024 16:58:20 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 10:58:19 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 08:58:19 -0800
Received: from [172.27.131.30] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 13 Feb 2024 10:58:18 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19d03c51-ca91-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FMvpHEGFBJyCWvdLFWcxeGU6T7rjS/krUXHMqzEEM8mekmoFwsSZgbOmHCDjSuRi5UjwBJhlcvveftArZ7MWrQkuPx+Emf8yKmt7DITw9r7pfdYdzXJAZwRSwlGQDbc+mjhHuLg8A8knqyJ2Jt+vJMJiirlKT4H4R+ykdyPgKT58IsOxNOgYYN2AdYv/9MHKWswGK4euHP5Yc6SOxHAO5k/ZhivrSBSOPee/QY2ncWw710N+KvrAOMZNT7lby4Ozv6OtEG8Zr5HUucGBwAkXoOSfOnPfKm7NzL/ganGAEF7IKw4HpENXBuulmywadFj+9qRUow6v1vNq57kyliaifA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=cF4kq4/gZXL/ma7LW3N023XPKQzfYSmH+abcvn4PgpU=;
 b=gvb9b+hlJVX4b7SoMCdHcB6WJlXukZfdXpAdVxrCgyqmR4EtGJfN9gk+jgaf3vdj/WLFQ+yc4Sdl01k15J4heY5SvUDOxLxbAvHiOfx4zCBW8zHJ4VQfF5sHFlPV7H8qL+Y1KHNTkUzcmd+BMvFAFtGGp99SSRFJkBDwUdc+3TGzmi4DElKM7gslxrmXnsasHYYv8q7w0MR3s6jcv4SP5PGgQ70i6cDLcCwew5uBwPwzOyYUI0ivf6p2AX7EmSBIcaQdvuhRWPO368D2K456+QMLX1LsTZd3Hq9yKGB980s3bGf9m0lL0h3X4rU+ikWJQcLrTvsP5+nZqNX3udBdZA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cF4kq4/gZXL/ma7LW3N023XPKQzfYSmH+abcvn4PgpU=;
 b=ZF/r17fWPFNSHDjRV9+YrU2Cjk5R/47ZcwZHOS3xDPhjw77taM0tK+gnh7Q7ye6ojWLmCKqHPoT5dfy5VLLXmROWofWuVXENxp4X0I/MOanwVpW95XOe1SPy8+Ekza7OlB/U85pE90H8TNpaWidY3gOt4m49W01lfGEpsy3QHCM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <f3b34d97-17b5-44ef-ad9e-ad04a96754d0@amd.com>
Date: Tue, 13 Feb 2024 11:58:18 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>
CC: <xen-devel@lists.xenproject.org>, Oleksandr Andrushchenko
	<oleksandr_andrushchenko@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian
	<kevin.tian@intel.com>, Paul Durrant <paul@xen.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <Zcsp15Aqve11Icjb@macbook> <ec5d0c39-1559-4f10-9574-98cfa0542993@suse.com>
 <ZcswBFHtINB1XMAS@macbook> <db0ce984-88e3-4394-b270-c5db585f8209@suse.com>
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <db0ce984-88e3-4394-b270-c5db585f8209@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS2PEPF0000343E:EE_|SA1PR12MB8142:EE_
X-MS-Office365-Filtering-Correlation-Id: 6e8fa776-338c-4bd5-1613-08dc2cb4fb82
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Zlaq9/D9S5FWqYmKuBbEc+RpPpjuvLAzS8py/4pFm+zxFLKtzT7550/posokZMJhOyEZx9wv+ryOP6i12AfGcFKcXlJJi7T9RWAVwMm1jHO0tPM3VH7LhZzCt+BNwuMvq9uS2MQeeNM0t9dNg7gLfyCLbmsQAuaXTn8HDJj6K2MLHpCs5Gu7b1WDrqZW/0CFA5Pbmrgo4qtIQ/c9FBK0MF0bvsfi9IGkFf1B4n1+Jyn3U//PuhfAVPgvLI9/xPE03ULsadDLcpHtYevsJzQK/UQV8nLPXXumOFpMGClUoTRmsrokj3a0YFJZ9L/CphHAVv20tPSKhbPvNucS5E9cwhRMOUt4ZJpfmVequy/TG8/pzaHTglGqOYn6qSuyx0NRhkWNkl289SyptrqC2K2tE2SS9DjHSz85c7g8HcLTTxNJwjssU2EIDJX4Xx/UNeUYZC3r2zwyCv32oXGRYjSUil419Ei5IovvAy2CcWKVs+xqyHAy7UCwvYgRxmJhbqc5vAVfEVEDBGxKqegvz07k0tDjc3AcHr7w7wlhRC+g3YikV/XVCE306pq33mO/Gaj235WTDAWki0eq5fC8C9AeyOZUl5EfNcUJnJi5/aMTLwI=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(396003)(346002)(39860400002)(376002)(230922051799003)(451199024)(1800799012)(82310400011)(186009)(64100799003)(46966006)(36840700001)(40470700004)(31696002)(86362001)(31686004)(8676002)(70586007)(8936002)(5660300002)(70206006)(82740400003)(4326008)(7416002)(2906002)(478600001)(44832011)(426003)(336012)(26005)(2616005)(83380400001)(81166007)(356005)(66899024)(36756003)(54906003)(316002)(16576012)(110136005)(41300700001)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 16:58:20.4328
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6e8fa776-338c-4bd5-1613-08dc2cb4fb82
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS2PEPF0000343E.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8142

On 2/13/24 04:05, Jan Beulich wrote:
> On 13.02.2024 10:01, Roger Pau Monné wrote:
>> On Tue, Feb 13, 2024 at 09:44:58AM +0100, Jan Beulich wrote:
>>> On 13.02.2024 09:35, Roger Pau Monné wrote:
>>>> On Fri, Feb 02, 2024 at 04:33:05PM -0500, Stewart Hildebrand wrote:
>>>>> --- a/xen/include/xen/sched.h
>>>>> +++ b/xen/include/xen/sched.h
>>>>> @@ -462,7 +462,8 @@ struct domain
>>>>>  #ifdef CONFIG_HAS_PCI
>>>>>      struct list_head pdev_list;
>>>>>      /*
>>>>> -     * pci_lock protects access to pdev_list.
>>>>> +     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
>>>>> +     * structure from being removed.
>>>>>       *
>>>>>       * Any user *reading* from pdev_list, or from devices stored in pdev_list,
>>>>>       * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,
>>>>> @@ -628,6 +629,18 @@ struct domain
>>>>>      unsigned int cdf;
>>>>>  };
>>>>>  
>>>>> +/*
>>>>> + * Check for use in ASSERTs to ensure that:
>>>>> + *   1. we can *read* d->pdev_list
>>>>> + *   2. pdevs (belonging to this domain) do not go away
>>>>> + *   3. pdevs (belonging to this domain) do not get assigned to other domains
>>>>
>>>> I think you can just state that this check ensures there will be no
>>>> changes to the entries in d->pdev_list, but not the contents of each
>>>> entry.  No changes to d->pdev_list already ensures not devices can be
>>>> deassigned or removed from the system, and obviously makes the list
>>>> safe to iterate against.
>>>>
>>>> I would also drop the explicitly mention this is intended for ASSERT
>>>> usage: there's nothing specific in the code that prevents it from
>>>> being used in other places (albeit I think that's unlikely).
>>>
>>> But pcidevs_locked(), resolving to spin_is_locked(), isn't reliable. The
>>> assertion usage is best-effort only, without a guarantee that all wrong
>>> uses would be caught.
>>
>> Do we want to protect this with !NDEBUG guards then?
> 
> Yes, that would look to be desirable.

We will then also need a definition of pdev_list_is_read_locked() in the
#else case so we don't risk running into "error: implicit declaration of
function 'pdev_list_is_read_locked'".

Such a definition might look like:

#define pdev_list_is_read_locked(d) ({ (void)d; ASSERT_UNREACHABLE(); false; })

so that we still evaluate d exactly once in the NDEBUG case.

>>>>> + * This check is not suitable for protecting other state or critical regions.
>>>>> + */
>>>>> +#define pdev_list_is_read_locked(d) ({                           \
>>>>
>>>> I would be tempted to drop at least the '_read_' part from the name,
>>>> the name is getting a bit too long for my taste.
>>>
>>> While I agree with the long-ish aspect, I'm afraid the "read" part is
>>> crucial. As a result I see no room for shortening.
>>
>> OK, if you think that's crucial then I'm not going to argue.
>>
>>>>> +        struct domain *d_ = (d);                                 \
>>>>
>>>> Why do you need this local domain variable?  Can't you use the d
>>>> parameter directly?
>>>
>>> It would be evaluated then somewhere between 0 and 2 times.
>>
>> It's ASSERT code only, so I don't see that as an issue.
> 
> Fair point.
> 
>>  Otherwise d_ needs to be made const.
> 
> Indeed, but for assert-only code I agree the option is slightly better,
> ideally suitably commented upon.

Is "the option" here referring to making d_ const, or using d directly
(with suitable comment)?


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 17:14:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 17:14:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680092.1057937 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwMC-00016z-Ok; Tue, 13 Feb 2024 17:14:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680092.1057937; Tue, 13 Feb 2024 17:14:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwMC-00016s-Le; Tue, 13 Feb 2024 17:14:08 +0000
Received: by outflank-mailman (input) for mailman id 680092;
 Tue, 13 Feb 2024 17:14:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rZwMB-00016m-3m
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 17:14:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZwM9-00046Z-N8; Tue, 13 Feb 2024 17:14:05 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZwM9-0000NK-Gy; Tue, 13 Feb 2024 17:14:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=UIP48nUwwMrqsuxoJDDTPWHrQufev/aB3TJXws/WLhw=; b=t5e/1vumrpBF3zeRcfuIHHFMr1
	hSd1KW8KKmGfb22yk6MPe66LlTS/F7N7FZQq5Pu3BB8wg15DubAJht1humSNOxIrc64y+BEAzkG0V
	8a4BNAcH7eYzrn4z+VtQQWHZJ50f3wXjPAzxfOPVJHTf6i9mD0we4QMjM/EzkBXSMZUU=;
Message-ID: <9eb25d90-9f10-44e5-b9aa-32e3f898389a@xen.org>
Date: Tue, 13 Feb 2024 17:14:02 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Return type of clean_and_invalidate_dcache_va_range
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Xen Devel <xen-devel@lists.xenproject.org>,
 Consulting <consulting@bugseng.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper3 <andrew.cooper3@citrix.com>, Roger Pau
 <roger.pau@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>
References: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
 <alpine.DEB.2.22.394.2402091402080.1925432@ubuntu-linux-20-04-desktop>
 <f86baad0-f113-4156-9c10-6910e8c63492@xen.org>
 <02446758-a5a9-4a8a-8ce6-15abdc9ede65@suse.com>
 <99faac70440a68824a17fcaaea55ef48@bugseng.com>
 <09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org>
 <c0b8f2ec-a3bb-45a6-b748-052d55a6fd5c@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <c0b8f2ec-a3bb-45a6-b748-052d55a6fd5c@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 13/02/2024 07:13, Jan Beulich wrote:
> On 12.02.2024 19:38, Julien Grall wrote:
>> An alternative would be to introduced arch_grant_cache_flush() and move
>> the if/else logic there. Something like:
>>
>> diff --git a/xen/arch/arm/include/asm/page.h
>> b/xen/arch/arm/include/asm/page.h
>> index 69f817d1e68a..4a3de49762a1 100644
>> --- a/xen/arch/arm/include/asm/page.h
>> +++ b/xen/arch/arm/include/asm/page.h
>> @@ -281,6 +281,19 @@ static inline void write_pte(lpae_t *p, lpae_t pte)
>>        dsb(sy);
>>    }
>>
>> +static inline arch_grant_cache_flush(unsigned int op, const void *p,
>> unsigned long size)
>> +{
>> +    unsigned int order = get_order_from_bytes(size);
>> +
>> +    if ( (cflush->op & GNTTAB_CACHE_INVAL) && (cflush->op &
>> GNTTAB_CACHE_CLEAN) )
>> +        clean_and_invalidate_dcache_va_range(v, cflush->length);
>> +    else if ( cflush->op & GNTTAB_CACHE_INVAL )
>> +        invalidate_dcache_va_range(v, cflush->length);
>> +    else if ( cflush->op & GNTTAB_CACHE_CLEAN )
>> +        clean_dcache_va_range(v, cflush->length);
>> +
>> +    return 0;
>> +}
>>
>>    /* Flush the dcache for an entire page. */
>>    void flush_page_to_ram(unsigned long mfn, bool sync_icache);
>> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
>> index 424744ad5e1a..647e1522466d 100644
>> --- a/xen/arch/arm/setup.c
>> +++ b/xen/arch/arm/setup.c
>> @@ -735,8 +735,7 @@ void asmlinkage __init start_xen(unsigned long
>> boot_phys_offset,
>>                  fdt_paddr);
>>
>>        /* Register Xen's load address as a boot module. */
>> -    xen_bootmodule = add_boot_module(BOOTMOD_XEN,
>> -                             virt_to_maddr(_start),
>> +    xen_bootmodule = add_boot_module(BOOTMOD_XEN, virt_to_maddr(_start),
>>                                 (paddr_t)(uintptr_t)(_end - _start), false);
>>        BUG_ON(!xen_bootmodule);
>>
>> diff --git a/xen/arch/x86/include/asm/flushtlb.h
>> b/xen/arch/x86/include/asm/flushtlb.h
>> index bb0ad58db49b..dfe51cddde90 100644
>> --- a/xen/arch/x86/include/asm/flushtlb.h
>> +++ b/xen/arch/x86/include/asm/flushtlb.h
>> @@ -182,23 +182,22 @@ void flush_area_mask(const cpumask_t *mask, const
>> void *va,
>>    }
>>
>>    static inline void flush_page_to_ram(unsigned long mfn, bool
>> sync_icache) {}
>> -static inline int invalidate_dcache_va_range(const void *p,
>> -                                             unsigned long size)
>> -{ return -EOPNOTSUPP; }
>> -static inline int clean_and_invalidate_dcache_va_range(const void *p,
>> -                                                       unsigned long size)
>> +
>> +unsigned int guest_flush_tlb_flags(const struct domain *d);
>> +void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask);
>> +
>> +static inline arch_grant_cache_flush(unsigned int op, const void *p,
>> unsigned long size)
>>    {
>> -    unsigned int order = get_order_from_bytes(size);
>> +    unsigned int order;
>> +
>> +    if ( !(cflush->op & GNTTAB_CACHE_CLEAN) )
>> +        return -EOPNOTSUPP;
>> +
>> +    order = get_order_from_bytes(size);
>>        /* sub-page granularity support needs to be added if necessary */
>>        flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
>> +
>>        return 0;
>>    }
>> -static inline int clean_dcache_va_range(const void *p, unsigned long size)
>> -{
>> -    return clean_and_invalidate_dcache_va_range(p, size);
>> -}
>> -
>> -unsigned int guest_flush_tlb_flags(const struct domain *d);
>> -void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask);
>>
>>    #endif /* __FLUSHTLB_H__ */
>>
>> I have a slight preference for the latter. I would like to hear the
>> opinion of the others.
> 
> I would prefer this 2nd form, too, assuming the setup.c change wasn't
> really meant to be there. 

Indeed. I had another previous change I didn't and forgot to remove it.

> The one thing that doesn't become clear: In
> the sketch above arch_grant_cache_flush() has no return type, yet has
> "return 0". This raises a question towards the one that's at the root
> of this thread: Do you mean the function to have a return value, and
> if so will it be (sensibly) used?

Sorry I should have double checked the code before sending it. 
arch_grant_cache_flush() should return a value. So each arch can decide 
if they handle a given operation.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 17:18:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 17:18:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680097.1057947 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwQ9-0001fB-7z; Tue, 13 Feb 2024 17:18:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680097.1057947; Tue, 13 Feb 2024 17:18:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwQ9-0001f4-5R; Tue, 13 Feb 2024 17:18:13 +0000
Received: by outflank-mailman (input) for mailman id 680097;
 Tue, 13 Feb 2024 17:18:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rZwQ7-0001ey-LB
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 17:18:11 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZwQ6-0004BR-KZ; Tue, 13 Feb 2024 17:18:10 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZwQ6-0000Sb-Dj; Tue, 13 Feb 2024 17:18:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=VNnM4VQF0vjcdTR7KXseCdKfZl7qNpyp0ViprkBLpws=; b=TZVIEQjS0rEVeNqWXb8Hra5dO4
	9Zn3BGXU82HEde78hfVwcmsJS/RM3JSlUI3KitOwpa8oRy9wHNB02GUA0VBQVq/nT1I5loWWsHqyb
	f36LupXy1/7hGT+LI2rT9MwctPW+eZeXn022/WTvUL7dUdofJiRUpDEnq4rOdSVyw0+g=;
Message-ID: <01167c19-d1ff-41ea-8852-3eef307f2156@xen.org>
Date: Tue, 13 Feb 2024 17:18:08 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] xen/lib: introduce generic find next bit operations
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>
References: <fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 09/02/2024 17:58, Oleksii Kurochko wrote:
> find-next-bit.c is common for Arm64, PPC and RISCV64,
> so it is moved to xen/lib.
> 
> PPC has been transitioned to generic functions from find-next-bit.c
> since it now shares the same implementation as the PPC-specific code.
> 
> The MISRA exclude list has been updated to verify
> lib/find-next-bit.c instead of Arm's find_next_bit.c,
> as Arm's find_next_bit.c has been relocated to xen/lib/.
> 
> Despite CONFIG_GENERIC_FIND_FIRST_BIT not currently being used in
> Xen and being removed from the Linux kernel [1], it could theoretically
> prove useful for Xen. This is because the Linux kernel transitioned
> Arm to the generic version of find_first_bit() and
> find_first_zero_bit() due to improvements [1] in both performance
> and .text size.
> It would be prudent to investigate whether [1] is applicable to Xen
> as well and, if so, implement the necessary changes in a separate patch.
> 
> [1] https://lore.kernel.org/linux-arch/20211005054059.475634-5-yury.norov@gmail.com/
> 
> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 17:29:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 17:29:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680101.1057957 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwaw-0003u4-8Y; Tue, 13 Feb 2024 17:29:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680101.1057957; Tue, 13 Feb 2024 17:29:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwaw-0003tx-5k; Tue, 13 Feb 2024 17:29:22 +0000
Received: by outflank-mailman (input) for mailman id 680101;
 Tue, 13 Feb 2024 17:29:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+p2Q=JW=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rZwau-0003tr-O1
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 17:29:20 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6a6a19c7-ca95-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 18:29:18 +0100 (CET)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-55a5e7fa471so6062771a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 09:29:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a6a19c7-ca95-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1707845357; x=1708450157; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=muIebV+W/RdZ27uw8bvqR+a4qPrA/i3ZssrE3Qct38I=;
        b=wrSWeN23NZLJahgIqlbFN5CLe0TvK7Rcn1eYCyHXAk6uPwB9sz/rt0aX4EFycKuWgX
         bcZrD2aG1Ps8+w0nx6ijtLY+396qSiLnTrlYR/SYeD4aAjROZpS7D1kiblOFhcs3XpBZ
         pV6WAbyvl32aodtZ1qXywMZWyHUj6weSi8O0hO3G8z/SsL/kQZ5VaTIeyIShsZg+hYaK
         9jKRZBe2UC65y4SJ9p+NtynhUa/jiLnlVyrjYacVj50c2nypOReDZnSgyv6ry1Eq3Bv8
         E9k0EAPo9Nrwwogc05dHV7GTq1UrDqwpCwwhoMyzl+7dDTUythClGaCFuahQVOSW2x8v
         UzMw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707845357; x=1708450157;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=muIebV+W/RdZ27uw8bvqR+a4qPrA/i3ZssrE3Qct38I=;
        b=VmYTqLTqfFf0dsQ1XupvV78QU84DclUoX3GBYKeRSBak4upeq2C2h5pMpUdpLhwQ7c
         oaTeWYWhEUXyG4dMHed9G6mjXTvRThvrSkTSJ/EneNCcFbwtK+bEJO7WxpulGKzPgk5e
         4LbUG4+oejTkhHOVstXW7VPHBw8WmNceHqCkcyuZ+AGz7+L1kgy3NlL5ifUMz9SNglmu
         FOO7Y7qsenHI1sVHEykKhcTbFmqJrboK3gTFKkW1GoGp5oQULmA+uYYNgI9O7dveEUPs
         QmemPimNQnjamupAl9CkbYfiXI/usfbxeA9P+KdD33P9VEo58kChGsj7HNDQBtkSxkqc
         C0Sg==
X-Forwarded-Encrypted: i=1; AJvYcCUZsku4JeXq1vVS/sJ84geOgKX1wzdfEDXfP+KI2bH5fjzoCj9T8x4pibMnqvtW7E+owz9gz6k/u6ymC53U6CazvDl9vkn4dpFPmjdCyNQ=
X-Gm-Message-State: AOJu0YwUsMUpsVIOkGOankHrACU8W7f2+TXDZ9IkAlvrOvhhonDk1Urt
	7mbJn0UZC/0uGDfwQcPr5uo1al76+vjpOLuU99jRs3POF7iGxfvGcYNyHiIRUqpb5Iud6Y0NyIr
	88sFa1BqtDLTaF/x4iSU7HjllN1bXZXGJEWZpsw==
X-Google-Smtp-Source: AGHT+IHP0agWiBWig7PXo8/j41xPowJvxfMpGGOdBeAE7QxRcWvzJFwcKyI63AV7GDDKpSXkkYCLX6w/TO0sphSD9s4=
X-Received: by 2002:a05:6402:88c:b0:561:8918:9f5f with SMTP id
 e12-20020a056402088c00b0056189189f5fmr232398edy.24.1707845357478; Tue, 13 Feb
 2024 09:29:17 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-15-carlo.nonato@minervasys.tech> <a66e3131-1de1-49cb-9b26-5fb1fd77bc20@suse.com>
In-Reply-To: <a66e3131-1de1-49cb-9b26-5fb1fd77bc20@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Tue, 13 Feb 2024 18:29:06 +0100
Message-ID: <CAG+AhRWv5MazB-txmPkcb3CAuWFQvM97HH3D-_bn6r3kdB360A@mail.gmail.com>
Subject: Re: [PATCH v6 14/15] xen/arm: add cache coloring support for Xen
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, 
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, 
	Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Tue, Feb 13, 2024 at 4:25=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 29.01.2024 18:18, Carlo Nonato wrote:
> > @@ -218,9 +230,44 @@ static void xen_pt_enforce_wnx(void)
> >      flush_xen_tlb_local();
> >  }
> >
> > +static void __init create_llc_coloring_mappings(void)
> > +{
> > +    lpae_t pte;
> > +    unsigned int i;
> > +    struct bootmodule *xen_bootmodule =3D boot_module_find_by_kind(BOO=
TMOD_XEN);
> > +    mfn_t mfn =3D maddr_to_mfn(xen_bootmodule->start);
> > +
> > +    for_each_xen_colored_mfn( mfn, i )
>
> Nit: Either you consider the construct a pseudo-keyword, then
>
>     for_each_xen_colored_mfn ( mfn, i )
>
> or you don't, then
>
>     for_each_xen_colored_mfn(mfn, i)
>
> please.
>
> > --- a/xen/common/llc-coloring.c
> > +++ b/xen/common/llc-coloring.c
> > @@ -29,6 +29,8 @@ static unsigned int __ro_after_init xen_num_colors;
> >
> >  #define mfn_color_mask              (max_nr_colors - 1)
> >  #define mfn_to_color(mfn)           (mfn_x(mfn) & mfn_color_mask)
> > +#define mfn_set_color(mfn, color)   (_mfn((mfn_x(mfn) & ~mfn_color_mas=
k) | \
> > +                                     (color)))
>
> Nit: The wrapped line wants further indenting, such that it becomes
> immediately clear what parentheses are still open. Alternatively:
>
> #define mfn_set_color(mfn, color) \
>     (_mfn((mfn_x(mfn) & ~mfn_color_mask) | (color)))
>
> This is certainly an "interesting" construct: I, for one, wouldn't expect
> that setting the color actually changes the MFN.

Would something like mfn_with_color() be a better name? I need something th=
at
expresses clearly that something will be returned. Maybe colored_mfn() is e=
ven
better?

> > @@ -316,6 +318,27 @@ unsigned int get_max_nr_llc_colors(void)
> >      return max_nr_colors;
> >  }
> >
> > +paddr_t __init xen_colored_map_size(void)
> > +{
> > +    return ROUNDUP((_end - _start) * max_nr_colors, XEN_PADDR_ALIGN);
> > +}
> > +
> > +mfn_t __init xen_colored_mfn(mfn_t mfn)
> > +{
> > +    unsigned int i, color =3D mfn_to_color(mfn);
> > +
> > +    for( i =3D 0; i < xen_num_colors; i++ )
>
> Nit: Missing blank.
>
> > +    {
> > +        if ( color =3D=3D xen_colors[i] )
> > +            return mfn;
> > +        else if ( color < xen_colors[i] )
> > +            return mfn_set_color(mfn, xen_colors[i]);
>
> How do you know that this or ...
>
> > +    }
> > +
> > +    /* Jump to next color space (max_nr_colors mfns) and use the first=
 color */
> > +    return mfn_set_color(mfn_add(mfn, max_nr_colors), xen_colors[0]);
>
> ... this MFN are actually valid and in (available) RAM?

Xen must be relocated in a valid and available physically colored space.
In arm we do that by searching in RAM for a contiguous region that can cont=
ain
the colored version of Xen (including "holes" of memory that is skipped due=
 to
coloring). So we know that if mfn is in this region then the computed color=
ed
MFN is in the same valid region as well. I should ASSERT that somehow.
This should be something like virt_to_mfn(_start) < mfn < virt_to_mfn(_end)
(abusing a bit of syntax), but the problem is that this function is called
also when page tables are still not set so I can't count on virt_to_mfn().

> > --- a/xen/include/xen/llc-coloring.h
> > +++ b/xen/include/xen/llc-coloring.h
> > @@ -24,6 +24,17 @@ static inline void domain_llc_coloring_free(struct d=
omain *d) {}
> >  static inline void domain_dump_llc_colors(const struct domain *d) {}
> >  #endif
> >
> > +/**
> > + * Iterate over each Xen mfn in the colored space.
> > + * @mfn:    the current mfn. The first non colored mfn must be provide=
d as the
> > + *          starting point.
> > + * @i:      loop index.
> > + */
> > +#define for_each_xen_colored_mfn(mfn, i)        \
> > +    for ( i =3D 0, mfn =3D xen_colored_mfn(mfn);    \
> > +          i < (_end - _start) >> PAGE_SHIFT;    \
> > +          i++, mfn =3D xen_colored_mfn(mfn_add(mfn, 1)) )
>
> While the comment mentions it, I still consider it problematic that
> - unlike other for_each_* constructs we have - this requires one of
> the iteration variables to be set up front. Question is why it needs
> to be that way: Isn't it the MFN underlying _start which you mean to
> start from?

As said above, this is used also when page tables setup isn't complete
so I can't easily find the first MFN.

Thanks.

> Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 17:32:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 17:32:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680106.1057967 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwdV-0005Zz-PJ; Tue, 13 Feb 2024 17:32:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680106.1057967; Tue, 13 Feb 2024 17:32:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwdV-0005Zs-MV; Tue, 13 Feb 2024 17:32:01 +0000
Received: by outflank-mailman (input) for mailman id 680106;
 Tue, 13 Feb 2024 17:32:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gnrE=JW=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZwdU-0005Zk-D0
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 17:32:00 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c9e646b3-ca95-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 18:31:58 +0100 (CET)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a28a6cef709so626183966b.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 09:31:58 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 s7-20020a17090699c700b00a3cb136aef0sm1477683ejn.224.2024.02.13.09.31.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 09:31:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c9e646b3-ca95-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707845518; x=1708450318; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=jadmpGMsfnY6CDwqqwa7KbW7cyrm8fvm/Nnghv6GeuQ=;
        b=TtnlSpui+615qs+RhHELJPKRt8w2rfue8/50mzClIBdrvbeapiLsu7yXAytXJK6Awc
         YCdyO5UNaj6S5O45aUUvN6XkqKI2cCRc6UoaPNNn9ZS+GW+DCNAiyanNTPr5tUrU/1zE
         xPzV7RjvsEaJRqePVxmMg9Mm28bPmpSbIBHOM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707845518; x=1708450318;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jadmpGMsfnY6CDwqqwa7KbW7cyrm8fvm/Nnghv6GeuQ=;
        b=KXRZ+iPLapZoB4y51DOherKBNfNh0fMRMi1L+7/sUJguCuBpq8hws1kobe5HnZtBqK
         Hgf2S9e3NCzHth5KeZcIaopJUofU5B59KIXzlOSGyldVeWKs3xl5M65pIvS8S6WtJ469
         5Ql9eNHAHEq+bRqq2vJX2nGcAmK/095r3oqxmQyzZQlkP/X07YMibvIoMVnpei5WaFqg
         8Z6GQUUA7FlGKLZ2CF21FlCSs4746htoDA2EYC3wo5B/2L4WEcoNyvjn+g8jTrNJhl+4
         U/2A+EyakOZvMjTIVdm53orCCssjNClxujLckqpKST7FfoAEJ8PevcCyEBbvehhUopHJ
         UCjw==
X-Gm-Message-State: AOJu0YxCTP98lCCK6vMyNmsntURkooQ8ubhBH8bcCeWStptC7rPz/SA/
	maUV+hy1HhjzGPFWecZVZw+0rGwKPosvzH7pZIhH/nlpPEmjH+wYZaXKqzDE3Gs=
X-Google-Smtp-Source: AGHT+IFwvSL7a+s1VENs5br/aOQ+1YbL9bztJ0i7nbmvlFiQMdqFGZZhncRXvN7r64w7a6hXxX6WgQ==
X-Received: by 2002:a17:906:1c0b:b0:a3c:8f4c:b1d with SMTP id k11-20020a1709061c0b00b00a3c8f4c0b1dmr5284ejg.63.1707845517876;
        Tue, 13 Feb 2024 09:31:57 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWkbJvurJ1HMNuLQ77BBU8gUccyON2xQeCW9LRCXgZJ1/8VnAaBF8v74VOYN3h+VmgLSoUYzo5Ra/mpuQnLjyAO99WrcrjIHuxdRup83IfXitaAixuz
Date: Tue, 13 Feb 2024 17:31:57 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v5 01/22] tools: add access macros for unaligned data
Message-ID: <92250210-dede-44be-81ce-3db2bcb5b04a@perard>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-2-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240208165546.5715-2-jgross@suse.com>

On Thu, Feb 08, 2024 at 05:55:25PM +0100, Juergen Gross wrote:
> Add the basic access macros for unaligned data to common-macros.h.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 17:40:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 17:40:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680112.1057977 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwlg-0007aG-Eb; Tue, 13 Feb 2024 17:40:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680112.1057977; Tue, 13 Feb 2024 17:40:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwlg-0007a9-Bz; Tue, 13 Feb 2024 17:40:28 +0000
Received: by outflank-mailman (input) for mailman id 680112;
 Tue, 13 Feb 2024 17:40:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rZwle-0007a3-VR
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 17:40:27 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZwld-0004XT-Kd; Tue, 13 Feb 2024 17:40:25 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZwld-0001OJ-Ad; Tue, 13 Feb 2024 17:40:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=dqFCjYFXnPyxVCcP1xlWDNnuPp5d/WLL6xFEIoLefso=; b=1YrEfDIUxU0+YJ8PL4SHmJK3Jb
	DZE0BmwpZgEppVxEklWSxaNKSUOd4OUC3cYGN7DtHmed09dn7L70GRiRAi5Mk/byXQeu1MbvZ4xmS
	wE0orfrnO/kDdeW5tfr2mGT1NNA9BxflbLMXheiWeFnG0qKNFbIsJ4PVfforkWsSz0dk=;
Message-ID: <e12914b8-5c6a-4085-b5e8-03c5bdcc2c27@xen.org>
Date: Tue, 13 Feb 2024 17:40:22 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 3/7] xen/asm-generic: introduce stub header monitor.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Jan Beulich <jbeulich@suse.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
 <84568b0c24a5ec96244f3f34537e9a148367facf.1707499278.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <84568b0c24a5ec96244f3f34537e9a148367facf.1707499278.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 09/02/2024 18:00, Oleksii Kurochko wrote:
> The header is shared between several archs so it is
> moved to asm-generic.
> 
> Switch partly Arm and PPC to asm-generic/monitor.h and only
> arch_monitor_get_capabilities() left in arch-specific/monitor.h.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
> ---
> Changes in V8:
>   - Add Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
> ---
> Changes in V7:
>   - Drop definition of arch_monitor_domctl_event for PPC.
>   - define arch_monitor_domctl_event in asm-generic/monitor.h.
>   - add "define HAS_ARCH_MONITOR_DOMCTL_EVENT" in arm/.../monitor.h as it has arch specific implementation.
> ---
> Changes in V6:
>   - Rebase only.
> ---
> Changes in V5:
>    - Switched partly Arm and PPC to asm-generic monitor.h only
>      arch_monitor_get_capabilities() left in arch-specific/monitor.h.
>    - Updated the commit message.
> ---
> Changes in V4:
>   - Removed the double blank line.
>   - Added Acked-by: Jan Beulich <jbeulich@suse.com>.
>   - Update the commit message
> ---
> Changes in V3:
>   - Use forward-declaration of struct domain instead of " #include <xen/sched.h> ".
>   - Add ' include <xen/errno.h> '
>   - Drop PPC's monitor.h.
> ---
> Changes in V2:
> 	- remove inclusion of "+#include <public/domctl.h>"
> 	- add "struct xen_domctl_monitor_op;"
> 	- remove one of SPDX tags.
> ---
>   xen/arch/arm/include/asm/monitor.h | 25 +-----------
>   xen/arch/ppc/include/asm/monitor.h | 28 +------------

Looking at MAINTAINERS, monitor.h was covered by "VM EVENT, MEM ACCESS 
and MONITOR". As you move to ...

>   xen/arch/ppc/stubs.c               |  8 ----
>   xen/include/asm-generic/monitor.h  | 64 ++++++++++++++++++++++++++++++

... asm-generic/, I believe it will now fall under "THE REST". So I 
think MAINTAINERS needs to be updated to also cover asm-generic/monitor.h.

Looking at what was already committed, I think asm-generic/vm_event.h 
should also be added in MAINTAINERS. Can you send a separate patch for that?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 17:44:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 17:44:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680114.1057988 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwph-0008M3-V0; Tue, 13 Feb 2024 17:44:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680114.1057988; Tue, 13 Feb 2024 17:44:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwph-0008Lw-RG; Tue, 13 Feb 2024 17:44:37 +0000
Received: by outflank-mailman (input) for mailman id 680114;
 Tue, 13 Feb 2024 17:44:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rZwpg-0008Ll-7h
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 17:44:36 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZwpf-0004cW-IF; Tue, 13 Feb 2024 17:44:35 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZwpf-0001dn-CH; Tue, 13 Feb 2024 17:44:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	References:Cc:To:From:Subject:MIME-Version:Date:Message-ID;
	bh=cdiRVi64bQsKB3cUU5Cr8wNIj2g723MEfU6OziPU80o=; b=wUtZdsgXsYU0bEefUNNKy8Koaq
	p+EBtxsJcmfYU2Vf/qM7xmeRqgEuz0wwtaN4ofP3IgWNSbTks1yMgeHJebQn16AyGQ77FN8O1MV+2
	arZ9/Mc5ynPVZJMX4xcJN9xi2dMXccKZfoyK/oqCHuCwAKuQtmqywla18LxKKDSwgmhg=;
Message-ID: <4cc1eb90-e3ef-4225-aa98-441c11c3fc8a@xen.org>
Date: Tue, 13 Feb 2024 17:44:33 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 3/7] xen/asm-generic: introduce stub header monitor.h
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Jan Beulich <jbeulich@suse.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
 <84568b0c24a5ec96244f3f34537e9a148367facf.1707499278.git.oleksii.kurochko@gmail.com>
 <e12914b8-5c6a-4085-b5e8-03c5bdcc2c27@xen.org>
In-Reply-To: <e12914b8-5c6a-4085-b5e8-03c5bdcc2c27@xen.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 13/02/2024 17:40, Julien Grall wrote:
> Hi Oleksii,
> 
> On 09/02/2024 18:00, Oleksii Kurochko wrote:
>> The header is shared between several archs so it is
>> moved to asm-generic.
>>
>> Switch partly Arm and PPC to asm-generic/monitor.h and only
>> arch_monitor_get_capabilities() left in arch-specific/monitor.h.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> Acked-by: Jan Beulich <jbeulich@suse.com>
>> Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
>> ---
>> Changes in V8:
>>   - Add Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
>> ---
>> Changes in V7:
>>   - Drop definition of arch_monitor_domctl_event for PPC.
>>   - define arch_monitor_domctl_event in asm-generic/monitor.h.
>>   - add "define HAS_ARCH_MONITOR_DOMCTL_EVENT" in arm/.../monitor.h as 
>> it has arch specific implementation.
>> ---
>> Changes in V6:
>>   - Rebase only.
>> ---
>> Changes in V5:
>>    - Switched partly Arm and PPC to asm-generic monitor.h only
>>      arch_monitor_get_capabilities() left in arch-specific/monitor.h.
>>    - Updated the commit message.
>> ---
>> Changes in V4:
>>   - Removed the double blank line.
>>   - Added Acked-by: Jan Beulich <jbeulich@suse.com>.
>>   - Update the commit message
>> ---
>> Changes in V3:
>>   - Use forward-declaration of struct domain instead of " #include 
>> <xen/sched.h> ".
>>   - Add ' include <xen/errno.h> '
>>   - Drop PPC's monitor.h.
>> ---
>> Changes in V2:
>>     - remove inclusion of "+#include <public/domctl.h>"
>>     - add "struct xen_domctl_monitor_op;"
>>     - remove one of SPDX tags.
>> ---
>>   xen/arch/arm/include/asm/monitor.h | 25 +-----------
>>   xen/arch/ppc/include/asm/monitor.h | 28 +------------
> 
> Looking at MAINTAINERS, monitor.h was covered by "VM EVENT, MEM ACCESS 
> and MONITOR". As you move to ...
> 
>>   xen/arch/ppc/stubs.c               |  8 ----
>>   xen/include/asm-generic/monitor.h  | 64 ++++++++++++++++++++++++++++++
> 
> ... asm-generic/, I believe it will now fall under "THE REST". So I 
> think MAINTAINERS needs to be updated to also cover asm-generic/monitor.h.
> 
> Looking at what was already committed, I think asm-generic/vm_event.h 
> should also be added in MAINTAINERS. Can you send a separate patch for 
> that?

Oh, I just noticed that we have the following entry:

xen/include/*/monitor.h

So the header is already covered. Sorry for the noise.

As the code is falling under the "VM EVENT..." subsystem, then Tamas's 
acked is technically sufficient for this patch (even for the PPC change).

But just in case you need one for Arm:

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 17:53:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 17:53:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680121.1057998 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwxs-0001Zj-Oc; Tue, 13 Feb 2024 17:53:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680121.1057998; Tue, 13 Feb 2024 17:53:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZwxs-0001Zc-L3; Tue, 13 Feb 2024 17:53:04 +0000
Received: by outflank-mailman (input) for mailman id 680121;
 Tue, 13 Feb 2024 17:53:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rZwxq-0001ZW-QC
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 17:53:02 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZwxq-0004li-1K; Tue, 13 Feb 2024 17:53:02 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZwxp-0001qV-Oy; Tue, 13 Feb 2024 17:53:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=q6lzQ6MWSQ5gloczL0BcGlo6D8EKQQsfqXwuuhPjESw=; b=HMfsA7mfy5XAz7N3UTuGaPzoiX
	614eZPDWIckCLspw0uuB7t1rpJ7DZzLN0CGQlogRzt/6PEXERRsSrfAvkhKNqNsCZeM+qnlMX0HJg
	Un7zVBbMXAk9LfY5ABzhh9qAuoaHhIs7lgpmnujBywZsIs/195C8fYbeDsuZJhuRaS2s=;
Message-ID: <f3a7613e-0a3d-49f9-ac3a-57c8e51392b1@xen.org>
Date: Tue, 13 Feb 2024 17:52:58 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 4/7] xen/asm-generic: ifdef inclusion of
 <asm/mem_access.h>
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Jan Beulich <jbeulich@suse.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
 <e1927bde3fa27a934e8ccaa8969b38e8a5dff52e.1707499278.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <e1927bde3fa27a934e8ccaa8969b38e8a5dff52e.1707499278.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 09/02/2024 18:00, Oleksii Kurochko wrote:
> ifdefing inclusion of <asm/mem_access.h> in <xen/mem_access.h>
> allows to avoid generation of empty <asm/mem_access.h> header
> for the case when !CONFIG_MEM_ACCESS.
> 
> For Arm it was explicitly added inclusion of <asm/mem_access.h> for p2m.c
> and traps.c because they require some functions from <asm/mem_access.h> which
> aren't available in case of !CONFIG_MEM_ACCESS.

Ideally we should protect the code relying on <asm/mem_access.h>. But 
that's a separate improvement for the Arm folks to deal with. So:

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 18:03:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 18:03:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680125.1058008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZx8L-0004JJ-MN; Tue, 13 Feb 2024 18:03:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680125.1058008; Tue, 13 Feb 2024 18:03:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZx8L-0004JC-Jl; Tue, 13 Feb 2024 18:03:53 +0000
Received: by outflank-mailman (input) for mailman id 680125;
 Tue, 13 Feb 2024 18:03:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gnrE=JW=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rZx8J-0004J6-Vt
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 18:03:51 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3dc0628e-ca9a-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 19:03:50 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-411d231ea71so3441065e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 10:03:50 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 w17-20020a05600c475100b00411062c123esm5119041wmo.26.2024.02.13.10.03.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Feb 2024 10:03:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3dc0628e-ca9a-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707847430; x=1708452230; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=7M0bgXJxih24HFHY2drPmFWNVAVIDuvtRembp5cfhk4=;
        b=clOMFFegaros0gGvTx+lk+0QJv0arS9oXy7CLbTTRAClyOugFQ+sJ/chf/fwIJwR+W
         Rrwt2En5zEWc2PRTBpJEa21fF1kkzlEU2DC4sesg0UeEbhOvUDa4Y0t+NsSD08vcAar0
         dNMt1vGSjzUOi6T5qM2Rzv+WV/WI/7DZ4PQE0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707847430; x=1708452230;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7M0bgXJxih24HFHY2drPmFWNVAVIDuvtRembp5cfhk4=;
        b=KmSxEEDBmEak16siSFgsRhNICK1r0vi+Wq9YRFHUnEJg7whs61sj2ySJAjED0tqraY
         NMfWtVshncK9ifhvK/+ZWxO3PP1/MtsGAJ2OPgPG5QP1d8WFH7yPXG//jHcqtB3GATNY
         sAfYX3XdjEFk3gjm5CgYaJeUdYHLnDwG90aVjap/2Y6qzt9fNzNuGiqtT5d9NEyHBhVn
         qb3+6DauflE8r6Eun3dApT0F54HmlPElRktFfUdToV7U7Oiq5HJYoQhKM2uNDJzlWua7
         4t+iow/1/J2JovqEbl8i1sdDy2uzDhVQwYWoO0Rmdq7mEKuZoss95GAUfN8BPieNUxNX
         LT1A==
X-Gm-Message-State: AOJu0Ywt4lsJlCH+FZ9O2eEfHno9PWGtGeBIfsNureTnEtuX72/gbi6E
	Yi5akxnYWN7+rOTO3SZ/RWrgslGDyztQMPRh+R8BwxnQvS2lQy/7HI0SFKAAdCY=
X-Google-Smtp-Source: AGHT+IGE8XquhgKhtZWY7Ryp3SKnVu0sGy0jbAjZYde57mPrefI9TkJFhwQCYDSbkxr5ofxCBHUflQ==
X-Received: by 2002:a05:600c:5021:b0:40f:d3e4:8f85 with SMTP id n33-20020a05600c502100b0040fd3e48f85mr339895wmr.30.1707847430123;
        Tue, 13 Feb 2024 10:03:50 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX3kcDbpGD7EEotgPByO7VcuHu02Vk48EzChJ3t6vmm6/uy5cgCT+HW6mEGkB9YiYqGDp/Eu+Pw86+4Bl4ClksIidlFdUlYzN7jNRu1VWGPmFkxPRgcaXBIamZLEyiwBqBPmtC6YAn8oIwuf6p06TQLu9R1sVa8rS+uC0jvcT8aiBysrC0Jf22XwzcagBfrM4dyzuuEoVg=
Date: Tue, 13 Feb 2024 18:03:49 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>, Wei Liu <wl@xen.org>,
	Jason Andryuk <jandryuk@gmail.com>,
	George Dunlap <george.dunlap@cloud.com>
Subject: Re: [PATCH v5 15/22] tools/libs/light: add backend type for 9pfs PV
 devices
Message-ID: <5e6893dd-5280-4a14-8d00-4ed76e8c8e98@perard>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-16-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240208165546.5715-16-jgross@suse.com>

On Thu, Feb 08, 2024 at 05:55:39PM +0100, Juergen Gross wrote:
> +struct libxl__aop9_state {
> +    libxl__spawn_state spawn;
> +    libxl__ao_device *aodev;
> +    libxl_device_p9 p9;
> +    uint32_t domid;
> +    void (*callback)(libxl__egc *, libxl__aop9_state *, int);

This "callback" is never used, right?

> diff --git a/tools/libs/light/libxl_types_internal.idl b/tools/libs/light/libxl_types_internal.idl
> index e24288f1a5..39da71cef5 100644
> --- a/tools/libs/light/libxl_types_internal.idl
> +++ b/tools/libs/light/libxl_types_internal.idl
> @@ -34,6 +34,7 @@ libxl__device_kind = Enumeration("device_kind", [
>      (16, "VINPUT"),
>      (17, "VIRTIO_DISK"),
>      (18, "VIRTIO"),
> +    (19, "XEN_9PFS"),

That's going to need to be rebased ;-).

Patch looks fine otherwise: Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 18:09:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 18:09:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680128.1058018 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZxDl-00051K-8a; Tue, 13 Feb 2024 18:09:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680128.1058018; Tue, 13 Feb 2024 18:09:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZxDl-00051D-5o; Tue, 13 Feb 2024 18:09:29 +0000
Received: by outflank-mailman (input) for mailman id 680128;
 Tue, 13 Feb 2024 18:09:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rZxDj-000517-VA
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 18:09:27 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZxDj-00057P-B5; Tue, 13 Feb 2024 18:09:27 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZxDi-0002nI-W2; Tue, 13 Feb 2024 18:09:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=WH4dDpmR8qnYftqOLkvo3sWofuuVs8eP8PxR39ySo/w=; b=j8ySydJOi/m1UjolBaJNPA5Yd5
	CBBZ0CG+ECESxpRmWjbWoVGn7OQfQnfz+gKuG9Guo6l+WrernBS1lTY/ARhYGQ1+ijGVMclFeHtWD
	Zf8nsGq7CGlLEowwh7/450L3lBYAngA4nvvbIvTwd+2u9jg05pP7KPcFdyfrHN3I64dI=;
Message-ID: <319dcf7a-88cd-488e-ae51-6c20c19d8f22@xen.org>
Date: Tue, 13 Feb 2024 18:09:25 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 5/7] xen/asm-generic: introduce generic device.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
 <4ecb5e5f4a1dbf585ac7a9ebe40b5fd8e4709182.1707499278.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <4ecb5e5f4a1dbf585ac7a9ebe40b5fd8e4709182.1707499278.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 09/02/2024 18:00, Oleksii Kurochko wrote:
> diff --git a/xen/include/asm-generic/device.h b/xen/include/asm-generic/device.h
> new file mode 100644
> index 0000000000..6e56658271
> --- /dev/null
> +++ b/xen/include/asm-generic/device.h
> @@ -0,0 +1,149 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef __ASM_GENERIC_DEVICE_H__
> +#define __ASM_GENERIC_DEVICE_H__
> +
> +#include <xen/stdbool.h>
> +
> +enum device_type
> +{
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +    DEV_DT,
> +#endif
> +    DEV_PCI
> +};
> +
> +enum device_class
> +{
> +    DEVICE_SERIAL,
> +    DEVICE_IOMMU,
> +    DEVICE_INTERRUPT_CONTROLLER,
> +    DEVICE_PCI_HOSTBRIDGE,
> +    /* Use for error */
> +    DEVICE_UNKNOWN,
> +};
> +
> +struct dev_archdata {
> +#ifdef CONFIG_HAS_PASSTHROUGH
> +    void *iommu;    /* IOMMU private data */
> +#endif > +};

It is a bit too late to change, but I thought I would point it if 
someone wants to send a follow-up. It is a bit odd to have a structure 
dev_archdata that, if I am not mistaken, is only used ...

> +
> +/* struct device - The basic device structure */
> +struct device
> +{
> +    enum device_type type;
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +    struct dt_device_node *of_node; /* Used by drivers imported from Linux */
> +#endif
> +    struct dev_archdata archdata;

... in struct device. Looking at the use, I believe this was only 
introduced to try to keep that SMMU code close to Linux. I would 
consider to fold the other structure and update dev_archdata() in 
drivers/passthrough/arm/smmu.c.

> +#ifdef CONFIG_HAS_PASSTHROUGH
> +    struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
> +#endif
> +};
> +
> +typedef struct device device_t;
> +
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +
> +#include <xen/device_tree.h>
> +
> +#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
> +
> +/**
> + *  device_init - Initialize a device
> + *  @dev: device to initialize
> + *  @class: class of the device (serial, network...)
> + *  @data: specific data for initializing the device
> + *
> + *  Return 0 on success.
> + */
> +int device_init(struct dt_device_node *dev, enum device_class class,
> +                const void *data);
> +
> +/**
> + * device_get_type - Get the type of the device
> + * @dev: device to match
> + *
> + * Return the device type on success or DEVICE_ANY on failure
> + */
> +enum device_class device_get_class(const struct dt_device_node *dev);
> +
> +#define DT_DEVICE_START(name_, namestr_, class_)            \
> +static const struct device_desc __dev_desc_##name_ __used   \
> +__section(".dev.info") = {                                  \
> +    .name = namestr_,                                       \
> +    .class = class_,
> +
> +#define DT_DEVICE_END                                       \
> +};
> +
> +#else /* !CONFIG_HAS_DEVICE_TREE */
> +#define dev_is_dt(dev) ((void)(dev), false)
> +#endif /* CONFIG_HAS_DEVICE_TREE */
> +
> +#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
> +
> +struct device_desc {
> +    /* Device name */
> +    const char *name;
> +    /* Device class */
> +    enum device_class class;
> +
> +#ifdef CONFIG_HAS_DEVICE_TREE
> +
> +    /* List of devices supported by this driver */
> +    const struct dt_device_match *dt_match;
> +    /*
> +     * Device initialization.
> +     *
> +     * -EAGAIN is used to indicate that device probing is deferred.
> +     */
> +    int (*init)(struct dt_device_node *dev, const void *data);
> +
> +#endif
> +};
I am not sure I fully understand why "device_desc" is not protected by 
CONFIG_HAS_DEVICE_TREE. The structure doesn't mean much when the config 
is disabled. Can you clarify?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 18:12:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 18:12:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680131.1058027 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZxGt-0006wv-MS; Tue, 13 Feb 2024 18:12:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680131.1058027; Tue, 13 Feb 2024 18:12:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZxGt-0006wo-Ju; Tue, 13 Feb 2024 18:12:43 +0000
Received: by outflank-mailman (input) for mailman id 680131;
 Tue, 13 Feb 2024 18:12:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rZxGs-0006wi-P6
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 18:12:42 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZxGs-0005By-Ez; Tue, 13 Feb 2024 18:12:42 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rZxGs-00033G-9Z; Tue, 13 Feb 2024 18:12:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=LKJro8zqBQJ4q5uU55hTVf+fEuQUlrtlmKmeOYT/kX8=; b=psr2WBKRe/nS73rUEJzGNt4yuR
	Wa+kVCkw+K/87353qkBHejRkxiS7y510Q/U7752bwRgXuBRS7NPU5m1zmtDVUlhSDGwCLUviRhUtN
	tnP00N4Ptz42NU5HfYRcSf64wTtQHyJttfhWv7IWxBKBpRguCzqweE4yO2xffdqDdRug=;
Message-ID: <cd2091d2-ac71-4d6f-a9a2-e7444889af69@xen.org>
Date: Tue, 13 Feb 2024 18:12:40 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 6/7] xen/arm: switch Arm to use asm-generic/device.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
 <ae256e804cbfd3c5d17ef9d17f100ebbf17c48fc.1707499278.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <ae256e804cbfd3c5d17ef9d17f100ebbf17c48fc.1707499278.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 09/02/2024 18:00, Oleksii Kurochko wrote:
> The following changes were done as a result of switching to
> asm-generic/device.h:
>   * DEVICE_GIC was renamed to DEVICE_INTERRUPT_CONTROLLER according
>     to definition of enum device_class in asm-generic/device.h.
>   * acpi-related things in Arm code were guarded by #ifdef CONFIG_ACPI
>     as struct acpi_device_desc was guarded in asm-generic, also functions
>     acpi_device_init() was guarded too as they are using structure
>     acpi_device_desc inside.
>   * drop arm/include/asm/device.h and update arm/include/asm/Makefile
>     to use asm-generic/device.h instead.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes in V8:
>   - update the commit message
> ---
> Changes in V7:
>   - newly introduced patch which is based on the previous version of the patch:
>       [PATCH v6 9/9] xen/asm-generic: introduce generic device.h
> ---
>   xen/arch/arm/device.c             |   5 ++
>   xen/arch/arm/domain_build.c       |   2 +-
>   xen/arch/arm/gic-v2.c             |   4 +-
>   xen/arch/arm/gic-v3.c             |   6 +-
>   xen/arch/arm/gic.c                |   4 +-
>   xen/arch/arm/include/asm/Makefile |   1 +
>   xen/arch/arm/include/asm/device.h | 124 ------------------------------
>   7 files changed, 14 insertions(+), 132 deletions(-)
>   delete mode 100644 xen/arch/arm/include/asm/device.h
> 
> diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
> index 1f631d3274..3e02cff008 100644
> --- a/xen/arch/arm/device.c
> +++ b/xen/arch/arm/device.c
> @@ -16,7 +16,10 @@
>   #include <xen/lib.h>
>   
>   extern const struct device_desc _sdevice[], _edevice[];
> +
> +#ifdef CONFIG_ACPI
>   extern const struct acpi_device_desc _asdevice[], _aedevice[];
> +#endif

Can you also update the linker script to protect the following code? I.e

#ifdef CONFIG_ACPI
   . = ALIGN(8);
   .adev.info : {
       _asdevice = .;
       *(.adev.info)
       _aedevice = .;
   } :text
#endif

With this change:

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 20:33:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 20:33:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680143.1058038 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZzSb-0000gn-Do; Tue, 13 Feb 2024 20:32:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680143.1058038; Tue, 13 Feb 2024 20:32:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rZzSb-0000gg-Ax; Tue, 13 Feb 2024 20:32:57 +0000
Received: by outflank-mailman (input) for mailman id 680143;
 Tue, 13 Feb 2024 20:32:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZzSa-0000gW-RU; Tue, 13 Feb 2024 20:32:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZzSa-0007wy-P0; Tue, 13 Feb 2024 20:32:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rZzSa-0007fH-5m; Tue, 13 Feb 2024 20:32:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rZzSa-0002JO-58; Tue, 13 Feb 2024 20:32:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=JIOx80HWuviCtfcLaQairPomwVPWNG2g3qbC87Eck2o=; b=kkq6iXyHVzfk7MeToqVfErA/rl
	A0vYsILIcqsKBwejPCMV5expdxPWkiYi2eS82/4UrnP3Df0Wh1qejnho/Nsp5rwSbHDKJ89r4tM3I
	paZdfQMaXZOgB3wBgRLiPG2qdpM7ksyh9d/wMmrFD0CV5XG/kkKnYnJNyupihclABogI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184656-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184656: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=772380f7547bac7e4c10c9ceff87af3f54151ae5
X-Osstest-Versions-That:
    xen=772380f7547bac7e4c10c9ceff87af3f54151ae5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 13 Feb 2024 20:32:56 +0000

flight 184656 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184656/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184653
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184653
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184653
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184653
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184653
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184653
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184653
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184653
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184653
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184653
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184653
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184653
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  772380f7547bac7e4c10c9ceff87af3f54151ae5
baseline version:
 xen                  772380f7547bac7e4c10c9ceff87af3f54151ae5

Last test of basis   184656  2024-02-13 06:18:03 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Tue Feb 13 21:46:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 21:46:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680149.1058048 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra0bo-0000Uk-KP; Tue, 13 Feb 2024 21:46:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680149.1058048; Tue, 13 Feb 2024 21:46:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra0bo-0000Ud-HL; Tue, 13 Feb 2024 21:46:32 +0000
Received: by outflank-mailman (input) for mailman id 680149;
 Tue, 13 Feb 2024 21:46:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mM3Z=JW=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1ra0bn-0000UX-6d
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 21:46:31 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5688b718-cab9-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 22:46:28 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 66644CE1F39;
 Tue, 13 Feb 2024 21:46:22 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2ACBEC433C7;
 Tue, 13 Feb 2024 21:46:20 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5688b718-cab9-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707860781;
	bh=vz6VjlFd/DQuexXekHi8OgvwCROKnOnDwpEmjiujEE0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=oyZPWs4CXxvrr9ci/9ubQzd+kJ4ekY6ty6C4tg5xs2fJ+OAmVcYdRgObFjEFLiD5O
	 S0d+yYw0iRdMu4caFbhnTQWRutzWENAXr14xkYEtLo7CQ2XMjhlcGbeXwdID6ruO0p
	 y7OzI7sImuGiraxflnpDuhHC8mrc372IlK9hM4osE9/Bm1eHmqt/E0s3AzSxc8Yfy3
	 VLYGdLDIH5SyTcG5htfTvZtyujo2aXC37hR4y+mf84PBK2JtzKhVoFcpNVwJQj47F7
	 lBlRwn/gcjd7slgJSaC9z8I/2UYyXUg0kG36jqFPTKvl7NauzK4sknUVwhC5PA1+14
	 b5EFWtBBO4uaQ==
Date: Tue, 13 Feb 2024 13:46:18 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, andrew.cooper3@citrix.com, 
    george.dunlap@citrix.com, julien@xen.org, bertrand.marquis@arm.com, 
    roger.pau@citrix.com, roberto.bagnara@bugseng.com, 
    federico.serafini@bugseng.com, xen-devel@lists.xenproject.org
Subject: Re: [PATCH v1] docs/misra/rules.rst: catch up with accepted rules
In-Reply-To: <d7295c43-c415-4113-8d9c-e6e3d688763e@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402131330160.1925432@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2402091653110.1925432@ubuntu-linux-20-04-desktop> <9504e77d-6f52-489c-a91a-f4d1a6ce9a33@suse.com> <alpine.DEB.2.22.394.2402121512050.1925432@ubuntu-linux-20-04-desktop> <d7295c43-c415-4113-8d9c-e6e3d688763e@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 13 Feb 2024, Jan Beulich wrote:
> On 13.02.2024 00:18, Stefano Stabellini wrote:
> > On Mon, 12 Feb 2024, Jan Beulich wrote:
> >> On 10.02.2024 02:00, Stefano Stabellini wrote:
> >>> Update docs/misra/rules.rst to reflect the MISRA C rules accepted in the
> >>> last couple of months.
> >>>
> >>> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> >>> ---
> >>>
> >>> In the notes section I added some info about the deviations, but in any
> >>> case the appropriate info will also be added to deviations.rst,
> >>> safe.json, etc.
> >>>
> >>> I also added Rule 14.4, which is older, but when I first tried to add it
> >>> to rules.rst, Jan had a question I couldn't reply clearly:
> >>> https://marc.info/?l=xen-devel&m=169828285627163
> >>>
> >>> I think now with this series, the impact of Rule 14.4 is clearer:
> >>> https://marc.info/?l=xen-devel&m=170194257326186
> >>
> >> This series is about enums only afaics. Yet the rule is much wider, and iirc
> >> we had agreed that for integer and pointer types the normal language
> >> conversion to boolean meaning is fine as well. Not only do you not mention
> >> this case in the entry,
> > 
> > I can add a note about it.
> > 
> > 
> >> but it also continue to mean that effectively we
> >> limit the rule to a very narrow case. Which continue to leave open the
> >> question of whether the rule is worthwhile to accept in the first place.
> > 
> > When someone does a safety certification, there is a difference between
> > deviating a rule as a whole or accepting the rule and only deviating
> > certain aspects of it (simply ignoring the rule is typically not an
> > option in safety certification context.) So here I think it would help
> > downstreams interested in safety if we added the rule, with specific
> > deviations.
> 
> Yet then in other cases you refer to Bertrand's general statement of it
> not being helpful when too little of a rule is left by deviating.

I think it is a case-by-case judgement call. Also, keep in mind that
Bertrand's goal was to encourage us to accept more rules in their
entirety without any deviations, not reject more rules. Especially as
rejecting rules it is typically not possible for people doing safety, so
the alternative is to write a wider deviation -- deviate even more, not
less.


> > Do you have any comments on the other parts of this patch? If not, I
> > would be happy to resent the rest unmodified, and update only 14.4 in
> > its own separate patch where we can discuss further.
> 
> Well. We're in territory now where I'm not really happy anymore with the
> full scope of what is being added to the "accepted" list. Leaving 14.4
> aside, what you have in the patch all looks like what was agreed upon,
> but then I'm not taking notes during meetings, and hence I can't help
> the impression that e.g. for 5.5 there was more than just the one
> "permitted" pattern. Therefore, while I deliberately didn't comment
> there (for not having a concrete case in mind), I'm afraid I also don't
> feel anymore like acking such multi-rule patches. If you strictly went
> one by one, it is certainly possible that I might ack this and that.

That's understandable especially as this time I was slower to send out
the patch to update docs/misra/rules.rst (apologies for that.)

Specifically for 5.5, I have the following notes from Roberto:
"""
We tag as deliberate all violations involving clashes
between function-like macros and non-callable entities.
We also tag as deliberate the violations matching the pattern #define x x.
In the future, once string.h will be restricted to the <string.h> functionality
of the C standard, string.h will be deviated further.
Further deviations are yet to be agreed upon (e.g., on ARM64 the deviation
of TYPE_SAFE resolves most of the violation; on x86_64 the deviation
of TYPE_SAFE resolves half of the violations but the others depend
on only about a dozen of macros).
"""

I added to the notes section only the part that I thought would make
sense to keep there. For the rest, I think it would be best to leave it
to future updates of documenting-violations.rst and/or safe.json as
appropriate.

I'll split out 14.4 and 5.5, and keep the rest in one patch.


> As attempted to voice several times during the meetings, I pretty
> strongly disagree with many of the "developer confusion" aspects, when
> they take away options the language quite obviously and naturally
> provides. We're talking about hypervisor code here, not some random
> tool that was thrown together in a haste. At the risk of sounding
> arrogant, people being easily confused by what I'd call normal code
> should simply not touch code like this. Whereas the spirit of many of
> these rules looks to rather go in the direction that basically anyone
> knowing a little bit of C should be qualified enough to maintain code
> made subject to all of these rules.

At some point there will be new people working on Xen (among others,
AMD has been hiring). New people need to start from somewhere. So the
less confusing the code the better it is because it is also less error
prone. There is code reviews and tests, but if the code was less prone
to misinterpretations, then there would be fewer chances of errors in
the first place.


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 22:33:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 22:33:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680153.1058058 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra1LL-0006m3-RH; Tue, 13 Feb 2024 22:33:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680153.1058058; Tue, 13 Feb 2024 22:33:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra1LL-0006lw-OL; Tue, 13 Feb 2024 22:33:35 +0000
Received: by outflank-mailman (input) for mailman id 680153;
 Tue, 13 Feb 2024 22:33:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kFBu=JW=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1ra1LJ-0006lq-Ry
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 22:33:33 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2405::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ea1af306-cabf-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 23:33:32 +0100 (CET)
Received: from BYAPR21CA0022.namprd21.prod.outlook.com (2603:10b6:a03:114::32)
 by SJ2PR12MB7919.namprd12.prod.outlook.com (2603:10b6:a03:4cc::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.17; Tue, 13 Feb
 2024 22:33:29 +0000
Received: from MWH0EPF000971E6.namprd02.prod.outlook.com
 (2603:10b6:a03:114:cafe::7e) by BYAPR21CA0022.outlook.office365.com
 (2603:10b6:a03:114::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.9 via Frontend
 Transport; Tue, 13 Feb 2024 22:33:29 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 MWH0EPF000971E6.mail.protection.outlook.com (10.167.243.74) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 13 Feb 2024 22:33:28 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 16:33:27 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 16:33:27 -0600
Received: from ubuntu-20.04.2-arm64.shared (10.180.168.240) by
 SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.35 via Frontend Transport; Tue, 13 Feb 2024 16:33:26 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea1af306-cabf-11ee-8a4d-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FSCe+JYEPgiprKCi3bFbNmeUzlMCwtW6FRSBzVVa1+f+q5/0+byzSyb4KW6obrZyY1v9BMkksFdwYnX8eGKf9+v+S14z1tHFQn2Zm1NN0udYSxJon38wBFtwMvQ9uPnI1IEoAJvl8jmQtW28RRuwvm10VFrSZI4JuCfP68MQyiEz2XvMTIUPwTE6EwsVAoUJBdjHgbyvb6pRdXChaFX9hZL+amkW+nZDprZKgc+Bw8WLRIcsvoXda/PSl/7EHpAETCRwhBSYLzRWZyo00690r0AM6RtvPdHA7PAYAH9TGLECz/ybPCyFgFM92zGcXK3Pcr4QhTn8FpBpxzx0mTzbdw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=K2EGIVFayJXte1/ifsz9IeqstFIZA97IZnNrIWiSUO8=;
 b=ffYN7oUEP/2JMO3/D4SKVIm8zGhQDCwVRP0hlCubxgkQsBrohtrcy9QVaa8eZ095EjJIMiBEVlyPw45j9QSBtvqpD67A8Sv2Gi1oezUHKk8D3QU+gjiO1Jq3lFC4wBhJtXmLOgCY1t1WCA5BEc4BkOqhhvWYzVNc3hxgW0Ka68VOu4dZYJou/MFWWXq31XsYO6BITbdqyeo4RtJjwn3Mm3Dt1lGFJ6S9ofaYtJiAl5Hd+1ER4FVdEFDPwXvuiBhr97R/HsdYh1/ihIEJK5SNZL8T66cEqBW2e9KJtynl15uksO2R3ZCknvfWIhKJfj6kyFE1MhZwTXbGBi+4AM29Zw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=K2EGIVFayJXte1/ifsz9IeqstFIZA97IZnNrIWiSUO8=;
 b=0zifFnoNG5qvh6Ho1qg6ASJAAoQvuGWWNbtHsr9810xy1wSREV/kJBmNQDq7jxbld0ReDaUoQMTbc6+yDfb4PlSgChXG3rvRkqkreZo0pP3hLgSI7KnVOglvH8P80AuW7Uw/wLBZXrrDPkVKUILaoxzhkVP3Dt3EhndHilTZIug=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Date: Tue, 13 Feb 2024 14:33:25 -0800
From: Stefano Stabellini <stefano.stabellini@amd.com>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: <xen-devel@lists.xenproject.org>
CC: <stefano.stabellini@amd.com>, <andrew.cooper3@citrix.com>,
	<jbeulich@suse.com>, <george.dunlap@citrix.com>, <julien@xen.org>,
	<bertrannd.marquis@arm.com>, <roger.pau@citrix.com>,
	<roberto.bagnara@bugseng.com>, <federico.serafini@bugseng.com>
Subject: [PATCH v2] docs/misra/rules.rst update
Message-ID: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MWH0EPF000971E6:EE_|SJ2PR12MB7919:EE_
X-MS-Office365-Filtering-Correlation-Id: 00aaf083-51bb-4d79-af24-08dc2ce3ccd4
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ZiEO4KT4b13Zdt0wO0WEfb7hdyagyVn3c89Yuy3aspG892wVYyExd4/BjnCQ4rfS2HHaGCHgCvuQ2K7hTT/hW0h4SiUtqGdQ0nL6EykzNt4ciC97LORo1PxM6aCzHs6Kh/tqhHDPRgt3V++VxdCoXi5PVXmiXG+Ls9QjWlidm51aSVaXzhItIK6omBALnleMv0WlNj2paRUPBrNsjr/wMfoYOautU2z6S2C2Ad/JBX+ADTdmGSwm9sYvIlOZBceeeVbna00sEucI2K1neI26HGiuU4lk8hXp5gUoT4myKW9AnwuUr4nrV6rnWXkkl87fegXGZUkjY23cpOIZCSlgbmdwcCjE87tzNsRsNOFGxJP+IEoAfwjZuY8T+LBQiKuikUxk2ALY/s9AOrnkNh/x06kJKE8hQQHMvfDjUb8XeKfVhJU1ND5QTGvP74pXj76NE8uVdcjTsx0eG6k8DHpr8xhZmqrEkfmV1+EsB7OFemBMeqRVVVDVaG1v1y6uFX3qLtVsvgXiwOfgOl8IJtmUCnAFRaYcZOUcvb6KIr03OV0CRrHcux5Zm1WBT+EOpQp9WQHKaeZFsEQVZyU2FAwFNqr8S4kV+Xa/X22l68rPPIA=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(7916004)(396003)(39860400002)(136003)(376002)(346002)(230922051799003)(82310400011)(64100799003)(451199024)(186009)(1800799012)(46966006)(36840700001)(40470700004)(4326008)(8936002)(44832011)(8676002)(15650500001)(5660300002)(33716001)(336012)(426003)(83380400001)(81166007)(558084003)(26005)(356005)(70206006)(86362001)(82740400003)(6916009)(70586007)(316002)(54906003)(9686003)(2906002)(478600001)(41300700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 22:33:28.3894
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 00aaf083-51bb-4d79-af24-08dc2ce3ccd4
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MWH0EPF000971E6.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7919

Hi all,

Update docs/misra/rules.rst to reflect the MISRA C rules accepted in the
last couple of months.

Cheers,

Stefano


From xen-devel-bounces@lists.xenproject.org Tue Feb 13 22:33:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 22:33:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680154.1058067 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra1LU-00072S-8H; Tue, 13 Feb 2024 22:33:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680154.1058067; Tue, 13 Feb 2024 22:33:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra1LU-00072K-4x; Tue, 13 Feb 2024 22:33:44 +0000
Received: by outflank-mailman (input) for mailman id 680154;
 Tue, 13 Feb 2024 22:33:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kFBu=JW=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1ra1LT-0006lq-60
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 22:33:43 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f01e1dd0-cabf-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 23:33:42 +0100 (CET)
Received: from MW4PR03CA0167.namprd03.prod.outlook.com (2603:10b6:303:8d::22)
 by LV2PR12MB5989.namprd12.prod.outlook.com (2603:10b6:408:171::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.24; Tue, 13 Feb
 2024 22:33:38 +0000
Received: from MWH0EPF000971E3.namprd02.prod.outlook.com (2603:10b6:303:8d::4)
 by MW4PR03CA0167.outlook.office365.com (2603:10b6:303:8d::22) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7270.39 via Frontend Transport; Tue, 13 Feb 2024 22:33:37 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 MWH0EPF000971E3.mail.protection.outlook.com (10.167.243.70) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 13 Feb 2024 22:33:37 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 16:33:36 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 16:33:36 -0600
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 13 Feb 2024 16:33:35 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f01e1dd0-cabf-11ee-8a4d-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=CpUVo+0Ov3DJce4tEQemvjKriQZTUEv2ohOAFSqeQu41NBH3t4ui2ybyzAacf6qOD0iID1GMW12hoReM1jo3K2WgfU38OzG8xaarYZkIWI7R8cABj7cYTmAQvSZCbgkiZT5NtBC8Edl/wjR4/nPTzWMXS8IaKgBFMJx0aVabVqfQGvJ7HP7iAx8+DMf9Bp53pkQPb8GqBjWy5jUhzANEzSqzpPiK0ohHxct7KqjhCLt4ci4NT9Z7w8vCNSO9zVLLXF/nj0J26f836LFo4Ka+RSkONO3sy2maVaiRlB2emzj/k1VEQPCkNc+sq5eXvTv6QVLG3F+0ldUhSA0SO4VeNA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=p3zfWGoUqrVwRJH/yoaYwBvU4X2LEZOeMLmtpjXSpRM=;
 b=SjHTdDUzkwzHQZjvwfqQ4qPXYc84OZrSMpTuFJDKStvNHZCTy/8UZOuCJSu9FDF6j3tO5ZxfZxEsO0F3DzbJqVpMwqswD35xsAcgyPNIS9aJ3wDZJ9pOakEbJxBtKzDdfjPCW+biA9yPBTVH3nLdkJ8d9ZtRBWvS7WUz2GBM5IJ7DuCiPh6I7rlkEO7jx+EfFynPhXbX/NGkRWYv+y9ezvGdtULW/akYoLsRZ4//NZEEVBCvXz6yFvzE6pLTVq4dl6QmMJ1jRja/A9MFhdJUyrPoxwFuRXC/mhBQGOob/M5LXq9f0NVMdlAVzLUpxchWwMPRAgmgBRVf6ceMqb5pKg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=p3zfWGoUqrVwRJH/yoaYwBvU4X2LEZOeMLmtpjXSpRM=;
 b=eSbQ8sdRUu2GhCjyVh3Lp10zLLDgeQZ5CpGEwRKn2jG45RPltBqjj4Y26Uf2J8aTu4sEIXVdejrCc42E5tnPHyEacq4gO4GraG8j/iSGzYthG13zffXAMPI7HEsrhoc7Bhs57LVVH0+Y06rxhCRCDj0tp89vSyyHElww2fLpfNw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <stefano.stabellini@amd.com>, <andrew.cooper3@citrix.com>,
	<jbeulich@suse.com>, <george.dunlap@citrix.com>, <julien@xen.org>,
	<bertrannd.marquis@arm.com>, <roger.pau@citrix.com>,
	<roberto.bagnara@bugseng.com>, <federico.serafini@bugseng.com>
Subject: [PATCH v2 1/3] docs/misra/rules.rst: add rule 16.6 and 20.12
Date: Tue, 13 Feb 2024 14:33:32 -0800
Message-ID: <20240213223334.3693410-1-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MWH0EPF000971E3:EE_|LV2PR12MB5989:EE_
X-MS-Office365-Filtering-Correlation-Id: 21879c1e-281e-44bd-0aee-08dc2ce3d21e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TjsqiEo5Rs1uKXDoH6YyqaZA3Rb4eP44xMs5DU61x45jQy4n+QCvJbn3A3yfaiN0VhlTi6eOwcngJWK8bF9m/qD16QlWA+JEvYOjfNQ1C7X66ZyJ6fLgQm2tKelPenBnlFNIxRbF7ptud/k/+oc6IVWezqqBbMu1zsO3JyrZvNl68xUdncWgVScCKhfZMwJcOVbI9bkLaBFJbFiHUJjRCihTLelQNW+Yu2P8R6CC6vqfx3AvpBI0HTZEM0+YUBFeiiN0Jstn0XldUhzqtRWZiB2ICbnowUVAxQ7TU3MrnGnM/gnPstrUPP+OHr6n0VB0npqL0ve7rF9CZFw1BrdYBnCyT1hPOY1Q932CA3fI6eZSsChdHFV3G+uAvA9oTBrEJ0RStAM5EF8+c96N0iO/Bst0nxdxs+WjHCxejcPEF4JVZibrHTE7nri8NJJkaOeOYSyBOs29BAPTyTfxbTorN48JZefVcJzSJpELXJgQVvyRdLgeQWO+7GhtGUkembNCLDnmDKYABCBcsLYXkVL6T3Wf4P/ssE/yoa4TYHe+R5qQO3uPkQxrqHm8/IiQnI6sb/kT5K7+PXMOpY4VtmUmLzvhniTYo9qzxfGxuvDpBzpr2KRF7ynb2JovwD0qDEhr
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(136003)(39860400002)(396003)(376002)(230922051799003)(82310400011)(186009)(1800799012)(64100799003)(451199024)(40470700004)(36840700001)(46966006)(36756003)(8676002)(426003)(2616005)(26005)(1076003)(41300700001)(478600001)(70206006)(8936002)(5660300002)(4326008)(6916009)(54906003)(70586007)(6666004)(316002)(86362001)(82740400003)(81166007)(356005)(336012)(44832011)(2906002);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 22:33:37.2744
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 21879c1e-281e-44bd-0aee-08dc2ce3d21e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MWH0EPF000971E3.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5989

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
---
 docs/misra/rules.rst | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index 182331089d..c185366966 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -478,6 +478,12 @@ maintainers if you want to suggest a change.
        adhere to Rule 16.2 would result in increased complexity and
        maintenance difficulty, and could potentially introduce bugs. 
 
+   * - `Rule 16.6 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_16_06.c>`_
+     - Required
+     - Every switch statement shall have at least two switch-clauses
+     - Single-clause switches are allowed when they do not involve a
+       default label.
+
    * - `Rule 16.7 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_16_07.c>`_
      - Required
      - A switch-expression shall not have essentially Boolean type
@@ -554,6 +560,13 @@ maintainers if you want to suggest a change.
        evaluation
      -
 
+   * - `Rule 20.12 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_12.c>`_
+     - A macro parameter used as an operand to the # or ## operators,
+       which is itself subject to further macro replacement, shall only
+       be used as an operand to these operators
+     - Required
+     - Variadic macros are allowed to violate the rule.
+
    * - `Rule 20.13 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_20_13.c>`_
      - Required
      - A line whose first token is # shall be a valid preprocessing
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 13 22:33:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 22:33:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680155.1058078 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra1LV-0007Hv-Fv; Tue, 13 Feb 2024 22:33:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680155.1058078; Tue, 13 Feb 2024 22:33:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra1LV-0007Hn-CU; Tue, 13 Feb 2024 22:33:45 +0000
Received: by outflank-mailman (input) for mailman id 680155;
 Tue, 13 Feb 2024 22:33:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kFBu=JW=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1ra1LU-0006lq-6L
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 22:33:44 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2409::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f10b30f7-cabf-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 23:33:42 +0100 (CET)
Received: from MW4PR03CA0174.namprd03.prod.outlook.com (2603:10b6:303:8d::29)
 by MN6PR12MB8541.namprd12.prod.outlook.com (2603:10b6:208:47a::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26; Tue, 13 Feb
 2024 22:33:39 +0000
Received: from MWH0EPF000971E3.namprd02.prod.outlook.com
 (2603:10b6:303:8d:cafe::dc) by MW4PR03CA0174.outlook.office365.com
 (2603:10b6:303:8d::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.41 via Frontend
 Transport; Tue, 13 Feb 2024 22:33:39 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 MWH0EPF000971E3.mail.protection.outlook.com (10.167.243.70) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 13 Feb 2024 22:33:39 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 16:33:38 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 16:33:38 -0600
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 13 Feb 2024 16:33:37 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f10b30f7-cabf-11ee-8a4d-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=mOwfAcMZPgHCWxG/koGBqR4f3sex3ECXOf/H6hVdb4rVQyYgDQ6mtxr9Z2GOprDQ7+vWanoftorv7YJkSopXVXVU+/AmjOnMMRaz1GWhVA74fyBCUMMhQG7HLCHO3H8EnraataC4R9zheSTffg55lBbf6zuK7XkSiR6S77UliKsQb23xxiG+UP6ZNsEz4iJ/MXvKOXWXM5YjLevydoomDmPK/qmsFkjtZh6GdDpmZkkABWrjdyFTV/mQkxupcw1PZMyCYyAVkdkm3PFwNnTAB7CdR8252v1+8toUzlTlIZFXGssQSX18grwYP22xjHPImpu+k6lmTGPxnjVnlm/ebA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=fMlULJhvmyTxrcJKAvuzeEpM4IUs0jtR0Nuwpoxpqos=;
 b=m2gllIaQRPdwmog3iuLQgUsehqvLIQIQDtj4gpEDrIzop8afxHCQXjhOB35Vrb9q/yGZj2eaCx/PZ/+tyN2i8AXGoC+KxmmPiSXZ05Da4d5YYxG6C2K6xG0BXFUwCn5dLI0jibQBCB7kJ2EtYVdzShXTTgG1pWarAlmf5Im0YqoZTT0Zf30JPDxooZUwojhKlWcfGoWjah70sJPxP5wfpMqOlFT9dn8tiugmlwR83tFk6yn7dEFPXLFbd/ptuanVchwIT+uCM0Hc8JvAz5ZF70ncyVON4M8MHu6wrLFzGu/Gu1kDBIQiEliewII0pEV409oY6Ie71gbdGG6KyBJpjQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fMlULJhvmyTxrcJKAvuzeEpM4IUs0jtR0Nuwpoxpqos=;
 b=X7eLJEDNXlB6nzEFZESC94f81TK51C55nUd0LfKdkTSl54bs99YIpMDIsC4tLRS1iqq2kfl2UV/dKbn9Gcr/dM/v4RLdicUfL1zGCYH3ipYVY4cdg+ilVHi054wtO3svQgwQFba8tNw8UhVg4/2+QvuMQ/cgfmK7yZ47aVsyDQ0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <stefano.stabellini@amd.com>, <andrew.cooper3@citrix.com>,
	<jbeulich@suse.com>, <george.dunlap@citrix.com>, <julien@xen.org>,
	<bertrannd.marquis@arm.com>, <roger.pau@citrix.com>,
	<roberto.bagnara@bugseng.com>, <federico.serafini@bugseng.com>
Subject: [PATCH v2 3/3] docs/misra/rules.rst: add rule 14.4
Date: Tue, 13 Feb 2024 14:33:34 -0800
Message-ID: <20240213223334.3693410-3-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MWH0EPF000971E3:EE_|MN6PR12MB8541:EE_
X-MS-Office365-Filtering-Correlation-Id: 3da4ff8e-5b8a-499c-36bd-08dc2ce3d36e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HMqEZAmaelDMIsmHF6y+F9dPA0Qd4cWV6apIzEhAm4KJkQNFalJ2grwndN+/h/Jx+kWPbMxPkj1hcbAcTXr0ex/tkjPGVksvg3rGFc/AdegOHtUUFb8V46EOUmY2uVfgzn8qxJb3Czdcs4UuV8dsFbhK7vnb4BvffvRseI/SD6Dbp14+PYyhqO0cbQ0upL0OcwldN2ruX+qX2XWB5BTcM4yHyFmaprXwbnlBJ3uKNhislW8NxbIF76BgpukfBL/LxAy+doLsZz618Njl9Af4vDX5DRMtLbp9THhrQeG6RVx6bi4DYRTy81uBAdmkMgjfEjexinq1aWwmal6wUMHeQJxuu4k2MusGCo08w0Us3gLwwCYcDpJNBhqDrxq2ZJwQu6SK1g+SJyidknH2YrQ97i16uQUDfaU30EKCZqkzoRyFGwYXsUWs7gwTDS8OYONeTpr62AuwSiZcOm2hVhLh8ivwR7+kIEsvJFRpm/QHg6iPO0rYshigFMunu1L55OhUeEbaLIgp+hCpRlyWkEbJLWZeVE2Zhm0o8vArNIKW0KTLDp7nxno10pEja69H/KdMGvKwS5YqjOIQl3R06R0nGV4ieWiR+2NrFGRjhic8SIM0cCYuuZbNq5Unol7gXuWx
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(376002)(136003)(39860400002)(230922051799003)(1800799012)(186009)(82310400011)(451199024)(64100799003)(40470700004)(46966006)(36840700001)(2906002)(5660300002)(4326008)(41300700001)(8936002)(44832011)(8676002)(356005)(86362001)(82740400003)(81166007)(36756003)(2616005)(54906003)(316002)(6916009)(336012)(6666004)(426003)(1076003)(70586007)(70206006)(478600001)(26005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 22:33:39.4619
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3da4ff8e-5b8a-499c-36bd-08dc2ce3d36e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MWH0EPF000971E3.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN6PR12MB8541

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
---
 docs/misra/rules.rst | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index 931158b354..3e6f94d7bd 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -468,6 +468,15 @@ maintainers if you want to suggest a change.
 
        while(0) and while(1) and alike are allowed.
 
+   * - `Rule 14.4 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_14_04.c>`_
+     - Required
+     - The controlling expression of an if-statement and the controlling
+       expression of an iteration-statement shall have essentially
+       Boolean type
+     - Automatic conversions of integer types to bool are permitted.
+       Automatic conversions of pointer types to bool are permitted.
+       This rule still applies to enum types.
+
    * - `Rule 16.3 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_16_03.c>`_
      - Required
      - An unconditional break statement shall terminate every
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 13 22:33:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 22:33:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680156.1058084 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra1LV-0007LV-Si; Tue, 13 Feb 2024 22:33:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680156.1058084; Tue, 13 Feb 2024 22:33:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra1LV-0007Kx-M8; Tue, 13 Feb 2024 22:33:45 +0000
Received: by outflank-mailman (input) for mailman id 680156;
 Tue, 13 Feb 2024 22:33:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kFBu=JW=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1ra1LU-00072L-F8
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 22:33:44 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f0048d57-cabf-11ee-98f5-efadbce2ee36;
 Tue, 13 Feb 2024 23:33:42 +0100 (CET)
Received: from BL1PR13CA0329.namprd13.prod.outlook.com (2603:10b6:208:2c1::34)
 by PH8PR12MB7349.namprd12.prod.outlook.com (2603:10b6:510:217::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26; Tue, 13 Feb
 2024 22:33:38 +0000
Received: from MN1PEPF0000ECD7.namprd02.prod.outlook.com
 (2603:10b6:208:2c1:cafe::86) by BL1PR13CA0329.outlook.office365.com
 (2603:10b6:208:2c1::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.38 via Frontend
 Transport; Tue, 13 Feb 2024 22:33:38 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 MN1PEPF0000ECD7.mail.protection.outlook.com (10.167.242.136) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 13 Feb 2024 22:33:37 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 16:33:37 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 13 Feb
 2024 16:33:37 -0600
Received: from smtp.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 13 Feb 2024 16:33:36 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0048d57-cabf-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KLTmGALbAyL+wry9LrcYzL0YwM1WbKDxmGMJXU11NW2l3zY3jeZAi4u+b7yy5wG33wT2Nt1R5KxCG7apn3Kg5gvw1DEFKpUljtrikZu3OXsx3TrfW/ZBUg/OjS2JhgeDVZD05nJUrKe7nTe333hi9o//G3KuytcPHAIjhACRU8bFT0DzkMF8FjqAKfvjDu5v96lVYQOPaAGCHwy2w9hQagDcgCFaLaGfQnEYVV2NUN1rFpQ3tUJ5tCLaJ6kV3idRGnrBmBB1+AC8apR5lX/2EW/alzB1BIgrbuOYhdSkIdsIZx7jlk7HKdwucWT7j/Ai/mj10Rj3j6entrKYw4tMVQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=cIqzI8kYaY2Zo0nCOY7w4aQD8+SXQqH7qpdvGMs6SHo=;
 b=nCTjsaxSXnPvX0cxEgGQghEbSaTF82yRvL5Qd+ieWIBUKp9/bgFK34ARPn/t+PTqx53Vji4OqgcmEuwt5NDXxVZCNvootdevI3zs5+tUuxdDoST5+tgDBbBevEytpK0ZJRrECV3r7pdYEN8pvcCf6hf73vWSRH+I8sBfwtKH4yH6+DMdOkSYIF2KIN5/TRFNJ/eMysjAq536Fo0cT+4xtF9Q8Du9oc4gnV7u2+WgtGqnmW8bKTm3sVpH7vRNF+aMisFgxit20gkH73hwbgLwOMN+Gw+wNsZG/N56fk/ehZjgOPuPTeDFFEo8ILNOiVQg1cwBZl4RdoGxcMZzIB1BhQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cIqzI8kYaY2Zo0nCOY7w4aQD8+SXQqH7qpdvGMs6SHo=;
 b=kuIya99sCkprdU4SPSRSRpDgeeuDnrbHAvc/5KyIdCcIkVb67fOk1rPykqR5vgG1DS3Svr/njZXYyQ+d2ervfwQbDywzXyXd6M3MNCiD/5mpVrZyrzcEF4TfZWcWiAgPWQbyzXHhmBw3ygurUJAUff46a6KpzFoYebVJkHvB87E=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stefano Stabellini <stefano.stabellini@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <stefano.stabellini@amd.com>, <andrew.cooper3@citrix.com>,
	<jbeulich@suse.com>, <george.dunlap@citrix.com>, <julien@xen.org>,
	<bertrannd.marquis@arm.com>, <roger.pau@citrix.com>,
	<roberto.bagnara@bugseng.com>, <federico.serafini@bugseng.com>
Subject: [PATCH v2 2/3] docs/misra/rules.rst: add rule 5.5
Date: Tue, 13 Feb 2024 14:33:33 -0800
Message-ID: <20240213223334.3693410-2-stefano.stabellini@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD7:EE_|PH8PR12MB7349:EE_
X-MS-Office365-Filtering-Correlation-Id: 7f264f48-f1a9-4513-2662-08dc2ce3d270
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9KyuUhoLSi1EwbI6V3sfrIB/Pt8Mw0bCs8WvgzeJb2Uj3X/YSbKyX+91Bs7UHWxKHxDPuoxGCPs0Tdl+sR06tpbYX9vI5qsoNDQnuKMlMT2TQhSzprSf7FjhVND+X0bOemTrpJKHi6xtPI6jM9PMj1VI0U627D8eCBrf3c0SR8sKybpBdWuEF7EJ+uTemZJ7JA3lX3YWbdOj4x28CPUSqxbXXtlrPVYp0tRWErU3idTSjRwyOe3PLSU0XSbjqpnOXdmbPrbpV3SxItnUaDbEN8g9BqinlU59eWJwgRwDsyxbxk8p+06dB9aYMl6J5dzdnKDcwGZ+FsAbcvq7n+FbOttgf2cmg3Fy+UDA8LZiJ92uoNisAvVi7ONd180CNu0IrGdoaoq7FGupTpV3wdR+kDQSKOONpuBG9kgbkvhDOXRaC+85JbyqjeYop/wCQAnIiZc/DBYMVoe+5veMxtLUTODe63uPCaxmPqUtIco9PKOsBvqKyPA5TLEFKkbF1JQ66RM4dEDaA/sdp1NuTBXK4Saiwdjff7uzbbwRw87afOS8Lmp/qoo3Cu+zOBT1h3NbkG02qQ3KDGbZ4lMg6uwvaQ==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(396003)(346002)(39860400002)(376002)(230922051799003)(451199024)(1800799012)(82310400011)(186009)(64100799003)(46966006)(36840700001)(40470700004)(86362001)(8936002)(8676002)(5660300002)(70586007)(82740400003)(4326008)(4744005)(6916009)(36756003)(2906002)(478600001)(44832011)(426003)(336012)(2616005)(26005)(356005)(81166007)(70206006)(1076003)(316002)(54906003)(6666004)(41300700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Feb 2024 22:33:37.9062
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7f264f48-f1a9-4513-2662-08dc2ce3d270
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MN1PEPF0000ECD7.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7349

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
---
 docs/misra/rules.rst | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index c185366966..931158b354 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -181,6 +181,12 @@ maintainers if you want to suggest a change.
        headers (xen/include/public/) are allowed to retain longer
        identifiers for backward compatibility.
 
+   * - `Rule 5.5 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_05_05.c>`_
+     - Required
+     - Identifiers shall be distinct from macro names
+     - Clashes between function-like macros and non-callable entities
+       are allowed. The pattern #define x x is also allowed.
+
    * - `Rule 5.6 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_05_06.c>`_
      - Required
      - A typedef name shall be a unique identifier
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 13 22:37:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 13 Feb 2024 22:37:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680165.1058097 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra1Ox-0000TA-9g; Tue, 13 Feb 2024 22:37:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680165.1058097; Tue, 13 Feb 2024 22:37:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra1Ox-0000T3-6v; Tue, 13 Feb 2024 22:37:19 +0000
Received: by outflank-mailman (input) for mailman id 680165;
 Tue, 13 Feb 2024 22:37:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QnSS=JW=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1ra1Ov-0000Sx-D5
 for xen-devel@lists.xenproject.org; Tue, 13 Feb 2024 22:37:17 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7068462d-cac0-11ee-8a4d-1f161083a0e0;
 Tue, 13 Feb 2024 23:37:16 +0100 (CET)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2d1080cb9easo15804811fa.1
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 14:37:16 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 p25-20020a05620a22f900b0078719b3b55bsm1040202qki.14.2024.02.13.14.37.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 14:37:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7068462d-cac0-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707863836; x=1708468636; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1Th9gFrU9rSiUUgBV5Ym3ThorHf7Gql3Eb2J172Mslg=;
        b=f0KfZeXWuZ5Xh+wm93jjOLsguosI6bWSDfWVv4DEWMec7cxW2LVlNVn2nq7P5hBR55
         10mUFscOor/ssZrwluOJ/SA+D6b1Cw27k1ta9KorunyuBln3Ry7Rw2U4n8PTCvjomoEb
         KEW5D9ZWwnrOCM7j/LfInlF1vRhoz4AZCj5pY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707863836; x=1708468636;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1Th9gFrU9rSiUUgBV5Ym3ThorHf7Gql3Eb2J172Mslg=;
        b=r00ey8tEOUm2rMOXkZwExsm1+oAcnYcZvveNg0WZDBY95A+TO80e55bLXlx4m9FUOT
         5YDPIrAMerXGf2AlahX5E3RgeRE6NRT0gfyRy4ruG7iQxCIvkLtPZY/HN9LIRwADuf7N
         VJdzp9MMT4vAxLWa0d52aZzsRFt1qicGGSF9HZusM+s/7GGBbK3lyqySrspPsccXpAYy
         9YXO4uuzNpjwr0w4ck5ACMhng1UE06EnKSvab8GINWVjxGxKp1/Ed/xE+5qhuYAxIOnl
         qdCkIPaAhMpTyVoeJwK6uRSQJ9q0Wq+Yvj4YCF0ODE1hXszGF9hJkCiQ6GJYu7+4fSuh
         upcw==
X-Forwarded-Encrypted: i=1; AJvYcCWSODyjKWa0ZyJjVrmybTuzQr8CL5lgR9R4HuoJejuZrzcxLdmL+miVgFxisjhZD+eVsEyieMoPh7zUUghPWbMOLOlqyYKdAAYSn1TvhPM=
X-Gm-Message-State: AOJu0YzqSPaeHdwzzg1UQIAg8Ase+k6B3IrtqxBvh7gVAHwW3CoQDI+L
	7WXPbTDckNanSynYt/NmvLDScXAeFhhZ/7ouLE+rJ6JydRtb7r1BjwcOnKDS1ns=
X-Google-Smtp-Source: AGHT+IFTsUWdeoXcIf2vJi+R8gAQp/ik0q75Yunwoln4AVqdh2VHf0ukfIqJEDbX1AEU4sP3tG5Bqw==
X-Received: by 2002:a2e:aaa4:0:b0:2d1:e45:d083 with SMTP id bj36-20020a2eaaa4000000b002d10e45d083mr601375ljb.3.1707863835898;
        Tue, 13 Feb 2024 14:37:15 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUMWDPItydWzEYtfLr5TrTXujspOimM6RSvwQC6JjIpRkuyQjavCsn/x7u6i3wb8uSPqvcuJiX0uJvEStcbNBop1w8RKn+4+mB0xQrCm7027HbFlMoI+eE3/KT3ymih52nf4vGV
Message-ID: <c4f27180-86bf-45fd-8641-bd160c6de229@citrix.com>
Date: Tue, 13 Feb 2024 22:37:13 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [regression] Re: [PATCH v2 2/2] iommu/vt-d: switch to common RMRR
 checker
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>, Roger Pau Monne <roger.pau@citrix.com>
Cc: Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-3-roger.pau@citrix.com>
 <c27c76ec-36cd-43cb-b76f-e8f95fb27ed8@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <c27c76ec-36cd-43cb-b76f-e8f95fb27ed8@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 12/02/2024 2:38 pm, Jan Beulich wrote:
> On 07.02.2024 16:34, Roger Pau Monne wrote:
>> Use the newly introduced generic unity map checker.
>>
>> Also drop the message recommending the usage of iommu_inclusive_mapping: the
>> ranges would end up being mapped anyway even if some of the checks above
>> failed, regardless of whether iommu_inclusive_mapping is set.  Plus such option
>> is not supported for PVH, and it's deprecated.
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

XenRT says no.

It's not clear exactly what's going on here, but the latest resync with
staging (covering only today's pushed changes) suffered 4 failures to
boot, on a mix of Intel hardware (SNB, SKL, SKX and CLX).

All 4 triple-fault-like things where following a log message about an RMRR:

(XEN) RMRR: [0x0e8 ,0x0e8] is not (entirely) in reserved memory

not being in reserved memory.


First of all - fix this printk() to print full addresses, not frame
numbers.  It's obnoxious to cross reference with the E820.

In the example above, 0xe8000 is regular RAM in:

(XEN)  [0000000000000000, 000000000009d3ff] (usable)


In another example,

(XEN) RMRR: [0x4d800 ,0x4ffff] is not (entirely) in reserved memory

is a hole between:

(XEN)  [000000004d3ff000, 000000004d3fffff] (usable)
(XEN)  [00000000e0000000, 00000000efffffff] (reserved)

We should also explicitly render holes when printing the E820, because
that's also unnecessarily hard to spot.


It's very likely something in this series, but the link to Intel might
just be chance of which hardware got selected, and I've got no clue why
there's a reset with no further logging out of Xen...

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 02:43:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 02:43:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680180.1058108 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra5Ex-0005oE-Fk; Wed, 14 Feb 2024 02:43:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680180.1058108; Wed, 14 Feb 2024 02:43:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra5Ex-0005o7-CG; Wed, 14 Feb 2024 02:43:15 +0000
Received: by outflank-mailman (input) for mailman id 680180;
 Wed, 14 Feb 2024 02:43:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ra5Ev-0005nC-Ro; Wed, 14 Feb 2024 02:43:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ra5Ev-0006Up-O9; Wed, 14 Feb 2024 02:43:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ra5Eu-00077M-QZ; Wed, 14 Feb 2024 02:43:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1ra5Eu-00015u-Oj; Wed, 14 Feb 2024 02:43:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=QVkXCr40HEH1o60+R7edl1aq8PXraMYfGYeDeUFHrk0=; b=yqg2WE6WjQxATT90sojI5GEQVg
	6gZ/GfLQvlW7X6P5LqxeQQpfOyJieI6V7ewlJEOkVvaipoZrIxvbkEdNNz73l0NQXsC+djtPN3sbz
	EZGFtJkxjwS8Y8JDbXtmpRd6SHpoEX2p2S76H31kJOQa5WIsuLX/hwyLpe90Pb0qDb/o=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184659-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184659: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=7e90b5c295ec1e47c8ad865429f046970c549a66
X-Osstest-Versions-That:
    linux=c664e16bb1ba1c8cf1d7ecf3df5fd83bbb8ac15a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 14 Feb 2024 02:43:12 +0000

flight 184659 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184659/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184654
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184654
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184654
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184654
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184654
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184654
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184654
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184654
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                7e90b5c295ec1e47c8ad865429f046970c549a66
baseline version:
 linux                c664e16bb1ba1c8cf1d7ecf3df5fd83bbb8ac15a

Last test of basis   184654  2024-02-13 03:02:10 Z    0 days
Testing same since   184659  2024-02-13 17:43:35 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Daniel Bristot de Oliveira <bristot@kernel.org>
  John Kacur <jkacur@redhat.com>
  limingming3 <limingming3@lixiang.com>
  limingming3 <limingming890315@gmail.com>
  Linus Torvalds <torvalds@linux-foundation.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   c664e16bb1ba..7e90b5c295ec  7e90b5c295ec1e47c8ad865429f046970c549a66 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 04:58:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 04:58:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680186.1058117 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra7LF-0004Hu-L1; Wed, 14 Feb 2024 04:57:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680186.1058117; Wed, 14 Feb 2024 04:57:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra7LF-0004Hn-IO; Wed, 14 Feb 2024 04:57:53 +0000
Received: by outflank-mailman (input) for mailman id 680186;
 Wed, 14 Feb 2024 04:57:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ra7LE-0004Hd-Lv; Wed, 14 Feb 2024 04:57:52 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ra7LE-00016X-KJ; Wed, 14 Feb 2024 04:57:52 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ra7LE-0002cp-85; Wed, 14 Feb 2024 04:57:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1ra7LE-0003fv-7f; Wed, 14 Feb 2024 04:57:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=246dTh4CG7PrSCUfvOgBxOkwuHTInLXfpeRoPtXVaF4=; b=EHTtNrbUO0zrtboY5NL9WmQcSF
	lyehgcraxQPYFcNUxwShZVeogQN4khRF+8cl49MZyixmNEOPZF3pd91FFfLCMyXgtb/utmWGspHSG
	R/2nnZgCUXGGSoGMbQxX5m/L1bkPrlAmfov7cqAJ7z/yqC37XLgzYN+qR6ds8nbYv09I=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184661-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184661: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=a1c426e8440b55ee12d92cefa7114fd6f7b599cb
X-Osstest-Versions-That:
    ovmf=8801c75b4d77c2e6e06b3ddc8560e0db590f6342
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 14 Feb 2024 04:57:52 +0000

flight 184661 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184661/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 a1c426e8440b55ee12d92cefa7114fd6f7b599cb
baseline version:
 ovmf                 8801c75b4d77c2e6e06b3ddc8560e0db590f6342

Last test of basis   184639  2024-02-09 22:41:18 Z    4 days
Testing same since   184661  2024-02-14 02:46:55 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michael D Kinney <michael.d.kinney@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   8801c75b4d..a1c426e844  a1c426e8440b55ee12d92cefa7114fd6f7b599cb -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 07:12:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 07:12:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680199.1058127 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra9R6-0003gX-4G; Wed, 14 Feb 2024 07:12:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680199.1058127; Wed, 14 Feb 2024 07:12:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra9R6-0003gQ-1b; Wed, 14 Feb 2024 07:12:04 +0000
Received: by outflank-mailman (input) for mailman id 680199;
 Wed, 14 Feb 2024 07:12:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ra9R5-0003gK-0V
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 07:12:03 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 58a13ae7-cb08-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 08:12:00 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-337d05b8942so4175656f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 23:12:00 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m10-20020adff38a000000b0033b0d2ba3a1sm11474149wro.63.2024.02.13.23.11.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 23:11:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58a13ae7-cb08-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707894720; x=1708499520; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9mwylvGvro3hpcMcMavYrFkdtc5DTZNEOn4vv4OzDsk=;
        b=PUlb353dmLu0GLar4rVv6l4U5hAUJGDxl8MFzSyDUHyOISjnjQYXFt8UJyD5QwnOlB
         DFZIyckKGsXVeqP5cWbKJ/8Ycpl57SpQH6jVhRrt+sntXgVS95Wg7xg9A77nE7nr4EQv
         4xmj32eXrRqAoHszEjPUc/cPS1ePpvsaawqZYnex2O+0r7SwWgl0vTKtC7iSIu8Zui+6
         ifBv8gf3PHFJ2nyv5lRE84mE9xfq1Vrpd2PERbBdU7uF2pegdjHv90kClYAp/Qo3a8vr
         DZXQBMvhOW/2+YmOFvJhlZA0/iruCXTMCzbmYhSmMhFx6Tr3NE/ssNMPDVD1+9veiuqM
         FKAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707894720; x=1708499520;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9mwylvGvro3hpcMcMavYrFkdtc5DTZNEOn4vv4OzDsk=;
        b=iqoDGYJchWLl8lrT8TJ8iL4Uscr7aTfptZE63tGstYgkiuNxdZplgq6dGraFKzuXf+
         bvufPOkhsImnBWL5MUB0VhkJ1c7j31uJE/fOdleCF2WYzkKTgGvx1ztkHWzT+IV6MpUc
         hbP8yZgLlxWONmen0AotP8nciq7F9tAUktFzTWFxdgRp4dLJuKWe7PD0CYjnIJlwhIzJ
         ugQJ+LLqccoQy8mR+q8sYH354taJPKShmAkhe5Je8Q9gAy5R8+UTUft7TtDwBp0xtti4
         wqssBEFSAoyo67yPl2Kixu56q7sgDlQMZh50vUONlbCH9+/0bzAu/KmXqzTz342OjAl4
         5hhA==
X-Gm-Message-State: AOJu0Yy3lBlVDV6jwAeTp4t6Lpegsvfl5xTY5h0YLoslaTIj/yvm6dhk
	QDbxm5V8x8ZAaQfpFYgXW//cv2m+LcK8c+sUS4s+QkwY9YX2VixFfQHGiiCuBw==
X-Google-Smtp-Source: AGHT+IE2X3rlRVYl/9IP6r89WArUDBOBhDVerPOYLp1imhyefS+FF/MqnIEmTvIgz6moTAVosg1aKw==
X-Received: by 2002:adf:e783:0:b0:33b:cef:5b1f with SMTP id n3-20020adfe783000000b0033b0cef5b1fmr910838wrm.64.1707894719783;
        Tue, 13 Feb 2024 23:11:59 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVy8kXndOaQUvnwC8/b+dgZe/boo0v8T+BvvVDCXZ7ZogzkhYICLahOfecEtCzSsQfFRnwg3Ze3EPG2kmzCQXGgBz1/gLTt5UDkvyjzCgoRrkP/Le0zf3FuLZ6gj3nvYeO6gxAs+ceirVE27SF1iFOkFxXXnww9xEnEQ6BtDavVuL5pYnyPD+Dzx/MU88jO3+rPpkn0
Message-ID: <01b0d902-1903-4618-ad43-f625e57b61e1@suse.com>
Date: Wed, 14 Feb 2024 08:11:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/ocaml: Add missing vmtrace_buf_kb field
Content-Language: en-US
To: Christian Lindig <christian.lindig@cloud.com>,
 =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, David Scott <dave@recoil.org>,
 Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>
References: <9b9909c9e93cb540b3488c784935acc2bc9e071e.1707343396.git.w1benny@gmail.com>
 <3A858D7F-C953-4EF0-8919-AE96D6105AB1@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3A858D7F-C953-4EF0-8919-AE96D6105AB1@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08.02.2024 10:13, Christian Lindig wrote:
>> On 7 Feb 2024, at 22:04, Petr Beneš <w1benny@gmail.com> wrote:
>> Add the missing `vmtrace_buf_kb` field to the OCaml bindings to match the
>> vm.cfg configuration, correcting an oversight from its initial introduction.
>>
>> Signed-off-by: Petr Beneš <w1benny@gmail.com>
> 
> Acked-by: Christian Lindig <christian.lindig@cloud.com>
> 
> This looks correct from an OCaml perspective. Why was the new field added in the middle of the record type domctl_create_config and thus forcing changes to the index of fields coming later in the record versus just appending the new field to the record type?
> 
> The critical bit is using the correct type in "Int32_val(VAL_VMTRACE_BUF_KB)” that matches the type "vmtrace_buf_kb: int32;” - which it does.

Is this then perhaps also lacking a

Fixes: 45ba9a7d7688 ("tools/[lib]xl: Add vmtrace_buf_size parameter")

and hence wanting backporting?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 07:16:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 07:16:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680202.1058138 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra9VY-0004FX-NP; Wed, 14 Feb 2024 07:16:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680202.1058138; Wed, 14 Feb 2024 07:16:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra9VY-0004FQ-Jj; Wed, 14 Feb 2024 07:16:40 +0000
Received: by outflank-mailman (input) for mailman id 680202;
 Wed, 14 Feb 2024 07:16:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ra9VW-0004FE-Tm; Wed, 14 Feb 2024 07:16:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ra9VW-00043T-PR; Wed, 14 Feb 2024 07:16:38 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ra9VW-0006PO-D1; Wed, 14 Feb 2024 07:16:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1ra9VW-0001jG-CZ; Wed, 14 Feb 2024 07:16:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Yt9u5lkIi0gpspp0qBhz20NjS3a78si62busT+qKfKc=; b=beZzzP+G0ztTKmSwWaIAfvYqxi
	bPR6ZCampret/Gg5acgLHzCnIwha0GivIzTgqObt0muicoXYXwVBVeF0vYJvjrWEgcA/IT3oAwIPX
	saoa0Xzpu7oa3LJ61arf/Tq0hyuCRiCxdmyAIfoVUV8XAVn4hD1xTDwSYNzt4y8rx0cg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184663-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184663: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=5fd3078a2e08f607dc86a16c1b184b6e30a34a49
X-Osstest-Versions-That:
    ovmf=a1c426e8440b55ee12d92cefa7114fd6f7b599cb
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 14 Feb 2024 07:16:38 +0000

flight 184663 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184663/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 5fd3078a2e08f607dc86a16c1b184b6e30a34a49
baseline version:
 ovmf                 a1c426e8440b55ee12d92cefa7114fd6f7b599cb

Last test of basis   184661  2024-02-14 02:46:55 Z    0 days
Testing same since   184663  2024-02-14 05:11:15 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Doug Flick <dougflick@microsoft.com>
  Doug Flick [MSFT] <doug.edk2@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   a1c426e844..5fd3078a2e  5fd3078a2e08f607dc86a16c1b184b6e30a34a49 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 07:45:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 07:45:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680213.1058148 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra9xW-0008Ko-U7; Wed, 14 Feb 2024 07:45:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680213.1058148; Wed, 14 Feb 2024 07:45:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ra9xW-0008Kh-RL; Wed, 14 Feb 2024 07:45:34 +0000
Received: by outflank-mailman (input) for mailman id 680213;
 Wed, 14 Feb 2024 07:45:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ra9xV-0008Kb-P8
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 07:45:33 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 06a85fa0-cb0d-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 08:45:30 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-41166710058so18732375e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 23:45:30 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 j12-20020a05600c190c00b00410bca333b7sm1081377wmq.27.2024.02.13.23.45.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 23:45:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06a85fa0-cb0d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707896730; x=1708501530; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=agwqrUROehPCJu5b9QY6Us3Xd1MVOfemUwJf+50z7ek=;
        b=FLI+p2WZaLbDBrXMsjMxyDh1qRpeaTFWZEJsrEtb+FukTEoV7zWDcFuMChnrHoXD2x
         uKe3uwVVByKOUX02/2hUBlcRic4oWvjIUJO5hGCrF+XQX5cP8EN8QUtD2GahNBY6Sixh
         l4GZuKzsYKV8AH09/y/7e2SAJuHujSBuG3luCKZh9CUhX0pJyYT6no+xJthyLDHl3Esf
         NJzGhAjPF8VbBCS/wMbRDRGltFerhWpPwGnyqK/Kx4IJdvUvlIuCiQBy8XVPKpdaGAum
         xc1ztAttEzgYTWWf2PKHKlrl4jgJX94tOeLZA4VTTBmYlkm+HyizrWiqv4W9oZ7XkZV+
         aprA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707896730; x=1708501530;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=agwqrUROehPCJu5b9QY6Us3Xd1MVOfemUwJf+50z7ek=;
        b=LN5nSoX1s5o8kZRram3WAEE4d3ZDUWP0mjozLZB5vBao9jKdnjHVlfUrNlL2oDR4FQ
         xwBKb+iP2hWq+gMWxS+5x2LqwX+37F8Nq87mMgDgCIpVpAyX73rwAMXKlJUV7rdmYPWQ
         9WXCCWGH3QG8cGRywwvxJ8zDCgRFhxnuVJHIFB+eQi0NFUKxttmjVh2WIHAiaPAufieG
         f5RAz16kV1EyXDSQJHVLq+QC3JLoLA1x8pW/qUhvoKsPYUmQ1QucQQ55fjojsSXDVb7m
         xD/NbiDLD02vciJcANpCtG3l9MMk++s3IA03Zxl9J2N0KR++ZeolJSnxACjDVCNzU5PL
         fRhQ==
X-Forwarded-Encrypted: i=1; AJvYcCXOIttP17awcVgUXYMaw0/0D1iyxtwKF8DOndA2NI8Eumx38DVHF309AHRhongQcwSqAc18IU5rzB8AjYulTzQcAz8JsjOPMfc9ZGP7pqg=
X-Gm-Message-State: AOJu0YxDuO/EJPlEbKbIF+VLQcd93apVnLqoJpr3rqcYfSWKeTf5n/eR
	WPBcTkFXQxkbIZfPgbQO9+6bH4wKH9PfqniPEztss9qpQw4tPNLBgFvHdfq1ZQ==
X-Google-Smtp-Source: AGHT+IHTGISP9QRCS0Z0n4QXu2RNyVuKZ36/5rMpqnEfNE9NY6ayD47MB5PkFBSvWNhZmZjH9CLVfg==
X-Received: by 2002:a05:600c:511f:b0:40f:d2ff:85b1 with SMTP id o31-20020a05600c511f00b0040fd2ff85b1mr1452328wms.25.1707896729780;
        Tue, 13 Feb 2024 23:45:29 -0800 (PST)
Message-ID: <37ed1abe-afcc-4a76-8a86-623282ca37a3@suse.com>
Date: Wed, 14 Feb 2024 08:45:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [regression] Re: [PATCH v2 2/2] iommu/vt-d: switch to common RMRR
 checker
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>,
 Roger Pau Monne <roger.pau@citrix.com>
Cc: Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-3-roger.pau@citrix.com>
 <c27c76ec-36cd-43cb-b76f-e8f95fb27ed8@suse.com>
 <c4f27180-86bf-45fd-8641-bd160c6de229@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <c4f27180-86bf-45fd-8641-bd160c6de229@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.02.2024 23:37, Andrew Cooper wrote:
> On 12/02/2024 2:38 pm, Jan Beulich wrote:
>> On 07.02.2024 16:34, Roger Pau Monne wrote:
>>> Use the newly introduced generic unity map checker.
>>>
>>> Also drop the message recommending the usage of iommu_inclusive_mapping: the
>>> ranges would end up being mapped anyway even if some of the checks above
>>> failed, regardless of whether iommu_inclusive_mapping is set.  Plus such option
>>> is not supported for PVH, and it's deprecated.
>>>
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> XenRT says no.
> 
> It's not clear exactly what's going on here, but the latest resync with
> staging (covering only today's pushed changes) suffered 4 failures to
> boot, on a mix of Intel hardware (SNB, SKL, SKX and CLX).
> 
> All 4 triple-fault-like things where following a log message about an RMRR:
> 
> (XEN) RMRR: [0x0e8 ,0x0e8] is not (entirely) in reserved memory
> 
> not being in reserved memory.
> 
> 
> First of all - fix this printk() to print full addresses, not frame
> numbers.  It's obnoxious to cross reference with the E820.

Perhaps better indeed. The stray blank before the comma also wants dropping.
And while looking over the patch again, "mfn_t addr;" also isn't very
helpful - the variable would better be named mfn.

> In the example above, 0xe8000 is regular RAM in:
> 
> (XEN)  [0000000000000000, 000000000009d3ff] (usable)

Well, no, E8000 is outside of that range, and I'm inclined to guess it's
the SNB where you saw that. Iirc my SNB has such an RMRR range, too. (Or
was it the Westmere?)

> In another example,
> 
> (XEN) RMRR: [0x4d800 ,0x4ffff] is not (entirely) in reserved memory
> 
> is a hole between:
> 
> (XEN)  [000000004d3ff000, 000000004d3fffff] (usable)
> (XEN)  [00000000e0000000, 00000000efffffff] (reserved)
> 
> We should also explicitly render holes when printing the E820, because
> that's also unnecessarily hard to spot.

I disagree here - both "ends" of a hole are easily visible from the
neighboring ranges.

> It's very likely something in this series, but the link to Intel might
> just be chance of which hardware got selected, and I've got no clue why
> there's a reset with no further logging out of Xen...

I second this - even after looking closely at the patches again, I can't
make a connection between them and the observed behavior. Didn't yet look
at what, if anything, osstest may have to say. Do I understand correctly
that the cited log messages are the last sign of life prior to the
systems rebooting?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 07:55:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 07:55:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680218.1058158 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raA7F-0001ca-Rl; Wed, 14 Feb 2024 07:55:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680218.1058158; Wed, 14 Feb 2024 07:55:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raA7F-0001cT-On; Wed, 14 Feb 2024 07:55:37 +0000
Received: by outflank-mailman (input) for mailman id 680218;
 Wed, 14 Feb 2024 07:55:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raA7E-0001cN-Mv
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 07:55:36 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6f576c08-cb0e-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 08:55:35 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d10ae127ffso17361251fa.3
 for <xen-devel@lists.xenproject.org>; Tue, 13 Feb 2024 23:55:35 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 be15-20020a05600c1e8f00b004107219c664sm1128506wmb.32.2024.02.13.23.55.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 13 Feb 2024 23:55:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f576c08-cb0e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707897335; x=1708502135; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xSPml6Psa7uiTdppooK2P7rnh6GtisF82Sju1HjR1cM=;
        b=HjmIweBSidcHgyEfSIUje2Knop47gsuB3b/f6nBIZM6m47Do2ZStOys/+pUMJ6NdOh
         JmSBWfO7yEPcuVW4hzrXlDKFbL1gdoY1G6QwPhPG3f/8SWBp2eNWboMehme8bcwCxUTQ
         rELg1Pt2QqTL906FAk4Jioq/h4ktUpQXWqRhjlXiBgcL8N5sWkr/lmQkPYgs2sVEGxIl
         FyhhC0hhMBMI8wACXGXRCZ8CP2Ai/lMFdg/cJiPVlRUmLxSNvaqQTgKpPVGFeaAdY4MW
         KIE2g4gOqx5Bqs0mlFY7/Id5soiTfJaE28HUUc6LckxJ3okaA7hcgemZ31hj9gKVV5pN
         C6Bw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707897335; x=1708502135;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xSPml6Psa7uiTdppooK2P7rnh6GtisF82Sju1HjR1cM=;
        b=IeTP9RARIsaBz1j3BVINm0Pq4vC04jmQkzgmt4PkxCnjKWKI78B28yWjooHD4qdItT
         Ja8xZokrJ/bOZ3tzVHNNzmB2qdVoSZ9dyNaSXWr6+1/cT7rOjkaO1bqRvAyVv33+Qpin
         AIMHw5S59+Wbp9GMEW83Fblc+8jkC8uR/5oUjh/Agd9C8+4O1gmNNJh1Vn3LIcrkz8xm
         +80EWWRkFwbyaiXHk1dlpemESGvf6L0xpx3p+bq7Wxz1l/ctI9vsbs5d7uZW5qe82K6N
         F+JrIhPeHebmVQPQc3efVzIQFCo9sbydA9zFxUsxgj22NfIgFI5AMkJ/hq6jVpPekAOh
         nBhA==
X-Forwarded-Encrypted: i=1; AJvYcCWeQnX+nGyS7C3ZqeStmNkT1LNHVnKXeqD6uQIoXuxezMTSQ/W4iY+B5wIm8ACzqLltVw5HmcNfDtVF1t7+eqUctg3q5xBqnldKbZg/obQ=
X-Gm-Message-State: AOJu0YxnBDUHsAAHtKQqAjFTEps/MJEJ4J4F/ceI0LrAYTcgW2gTLc0b
	CUrSUpC9BmhQEJ9BPyIFcnvvjs7Bzzsx3ZjbZ0wNHoPOggQswCsiKQVRTj2JKQ==
X-Google-Smtp-Source: AGHT+IE1I7uvjPAQwHiFNLhx8rXHlBejPaJhEvtocVIB8P9EQt8HmO6zWYW2eV7GhpktNfvLE6PAvQ==
X-Received: by 2002:a05:651c:314:b0:2d0:b6c4:2180 with SMTP id a20-20020a05651c031400b002d0b6c42180mr1127696ljp.53.1707897334858;
        Tue, 13 Feb 2024 23:55:34 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWA8IIhENIcDXpPiSFA6Oq7K4eycReCf2JC9q+q6h+yZ9EUMhORtHvR4OBt9yUxJDLMnecZ6NLfnFNOnKOUcryS8zEilioXVeFIEdMbAhUgbIf5HtMQ8TCu7s/RmAOBwjPmtjLHYm8kK1oU04RhEVxtwOup+QivyST6a4uDD4ugzDpwaBBFQrYX2qfe1UpcvZoi4aM1yBKyTWIMhCWNdwio2YCjWs2rvwKeF0ljBxJVygb7wMHxsj7e4TS40YKJk24e3l6NCAxPJHFD5MbEUQTuDVxf9TyVyMw/cCfC63LKhxjBYf0YV1oFLwrHQphJvrSLT0cD3VzeVW9L1bklCCAQyMGvfXscONDXBLfMGkgbi3mt6z3ptWOiapIcG58UIZkIOw==
Message-ID: <0feb6a90-2c3e-489f-90de-b4b2979997dd@suse.com>
Date: Wed, 14 Feb 2024 08:55:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 14/15] xen/arm: add cache coloring support for Xen
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: andrea.bastoni@minervasys.tech,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-15-carlo.nonato@minervasys.tech>
 <a66e3131-1de1-49cb-9b26-5fb1fd77bc20@suse.com>
 <CAG+AhRWv5MazB-txmPkcb3CAuWFQvM97HH3D-_bn6r3kdB360A@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CAG+AhRWv5MazB-txmPkcb3CAuWFQvM97HH3D-_bn6r3kdB360A@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.02.2024 18:29, Carlo Nonato wrote:
> On Tue, Feb 13, 2024 at 4:25 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 29.01.2024 18:18, Carlo Nonato wrote:
>>> @@ -218,9 +230,44 @@ static void xen_pt_enforce_wnx(void)
>>> --- a/xen/common/llc-coloring.c
>>> +++ b/xen/common/llc-coloring.c
>>> @@ -29,6 +29,8 @@ static unsigned int __ro_after_init xen_num_colors;
>>>
>>>  #define mfn_color_mask              (max_nr_colors - 1)
>>>  #define mfn_to_color(mfn)           (mfn_x(mfn) & mfn_color_mask)
>>> +#define mfn_set_color(mfn, color)   (_mfn((mfn_x(mfn) & ~mfn_color_mask) | \
>>> +                                     (color)))
>>
>> Nit: The wrapped line wants further indenting, such that it becomes
>> immediately clear what parentheses are still open. Alternatively:
>>
>> #define mfn_set_color(mfn, color) \
>>     (_mfn((mfn_x(mfn) & ~mfn_color_mask) | (color)))
>>
>> This is certainly an "interesting" construct: I, for one, wouldn't expect
>> that setting the color actually changes the MFN.
> 
> Would something like mfn_with_color() be a better name? I need something that
> expresses clearly that something will be returned. Maybe colored_mfn() is even
> better?

The latter reads as if it was a predicate, not a transformation. The former
or get_mfn_with_color() _may_ be okay. Without the get_ it's still a little
predicate-like, while the get_ itself somewhat collides with other uses of
that prefix, specifically e.g. get_page{,_type}(). So I'm still not overly
happy, yet e.g. mfn_from_mfn_and_color() feels clumsy to me.

>>> --- a/xen/include/xen/llc-coloring.h
>>> +++ b/xen/include/xen/llc-coloring.h
>>> @@ -24,6 +24,17 @@ static inline void domain_llc_coloring_free(struct domain *d) {}
>>>  static inline void domain_dump_llc_colors(const struct domain *d) {}
>>>  #endif
>>>
>>> +/**
>>> + * Iterate over each Xen mfn in the colored space.
>>> + * @mfn:    the current mfn. The first non colored mfn must be provided as the
>>> + *          starting point.
>>> + * @i:      loop index.
>>> + */
>>> +#define for_each_xen_colored_mfn(mfn, i)        \
>>> +    for ( i = 0, mfn = xen_colored_mfn(mfn);    \
>>> +          i < (_end - _start) >> PAGE_SHIFT;    \
>>> +          i++, mfn = xen_colored_mfn(mfn_add(mfn, 1)) )
>>
>> While the comment mentions it, I still consider it problematic that
>> - unlike other for_each_* constructs we have - this requires one of
>> the iteration variables to be set up front. Question is why it needs
>> to be that way: Isn't it the MFN underlying _start which you mean to
>> start from?
> 
> As said above, this is used also when page tables setup isn't complete
> so I can't easily find the first MFN.

Did you consider making the initial value a macro parameter then?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 08:19:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 08:19:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680228.1058167 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raATq-0005G9-2p; Wed, 14 Feb 2024 08:18:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680228.1058167; Wed, 14 Feb 2024 08:18:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raATp-0005G2-W8; Wed, 14 Feb 2024 08:18:57 +0000
Received: by outflank-mailman (input) for mailman id 680228;
 Wed, 14 Feb 2024 08:18:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raATo-0005Fs-OH; Wed, 14 Feb 2024 08:18:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raATo-0005qv-Jv; Wed, 14 Feb 2024 08:18:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raATo-0007uH-7k; Wed, 14 Feb 2024 08:18:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raATo-0003Yv-7D; Wed, 14 Feb 2024 08:18:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7KywH9OwxhvOspodKfrEqKxfAgonJfrOdpCGC1Lnji4=; b=Zf16PePpIr4kgG3r7LX2YP4xK9
	Vfr2L+3apl/rCS4Pcw74GTm8VKo8uw2Wj/h0A1MVNJx2EhStqo2UflUDbZ2Q4+C/bSH/yTHc1IhVm
	TLdAE5o8G+JbdhbRvcU4sqCnZVc+wBqc94EBsDdVmzns7Fb1j9Y+9g+La4cLFs0pBD3E=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184660-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184660: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:guest-start/debianhvm.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=b3bea912a737a0db39a89b0da0033c4bfda82f03
X-Osstest-Versions-That:
    xen=772380f7547bac7e4c10c9ceff87af3f54151ae5
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 14 Feb 2024 08:18:56 +0000

flight 184660 xen-unstable real [real]
flight 184664 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184660/
http://logs.test-lab.xenproject.org/osstest/logs/184664/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 20 guest-start/debianhvm.repeat fail pass in 184664-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184656
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184656
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184656
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184656
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184656
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184656
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184656
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184656
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184656
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184656
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184656
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184656
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  b3bea912a737a0db39a89b0da0033c4bfda82f03
baseline version:
 xen                  772380f7547bac7e4c10c9ceff87af3f54151ae5

Last test of basis   184656  2024-02-13 06:18:03 Z    1 days
Testing same since   184660  2024-02-13 20:40:56 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  George Dunlap <george.dunlap@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Julien Grall <jgrall@amazon.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Paul Durrant <paul@xen.org>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             fail    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   772380f754..b3bea912a7  b3bea912a737a0db39a89b0da0033c4bfda82f03 -> master


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 08:20:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 08:20:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680233.1058178 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raAVS-0006gz-JQ; Wed, 14 Feb 2024 08:20:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680233.1058178; Wed, 14 Feb 2024 08:20:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raAVS-0006gs-FY; Wed, 14 Feb 2024 08:20:38 +0000
Received: by outflank-mailman (input) for mailman id 680233;
 Wed, 14 Feb 2024 08:20:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kQr2=JX=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1raAVR-0006gk-AX
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 08:20:37 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2062e.outbound.protection.outlook.com
 [2a01:111:f400:7e88::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ec7eabbf-cb11-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 09:20:34 +0100 (CET)
Received: from CH2PR07CA0058.namprd07.prod.outlook.com (2603:10b6:610:5b::32)
 by SJ0PR12MB5423.namprd12.prod.outlook.com (2603:10b6:a03:301::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25; Wed, 14 Feb
 2024 08:20:26 +0000
Received: from CH2PEPF000000A0.namprd02.prod.outlook.com
 (2603:10b6:610:5b:cafe::ba) by CH2PR07CA0058.outlook.office365.com
 (2603:10b6:610:5b::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26 via Frontend
 Transport; Wed, 14 Feb 2024 08:20:26 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CH2PEPF000000A0.mail.protection.outlook.com (10.167.244.26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 08:20:26 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb
 2024 02:20:25 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb
 2024 00:20:25 -0800
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 14 Feb 2024 02:20:23 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec7eabbf-cb11-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=g9fzC9shUIwRFu4dwtkY5S/bFQIK6B6BGBcw5lECcKj08plEcchBlWysUgcfutGssmkDuXgHesV2XmakHdl8HIrVkOKQiYFM+fWro5LJ77SOK/9+s/O4e63dh3UcS32gjq348oz4EXmOeQnTQNUtWTVb185hBZJg7pc4mdlsQ6EDX6oRxmNCFRbYMzsrkMPNogO3ql//uc81bY3gfXtiWMwhYANk9QFZk1WoZ3izxH9BqDrTCLlb5yRSWtsspBNdV9jI5SXeWSpr+PfdqiDRDlpVL10S96HROiS15ciBJN1/DmUdRss43mXxb4bgBkCVROj0FDW0qoNJGpjaK+E1qg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=IxyRVjkHNGh7/pGUHjKlkb4va6U54AzjGMRoKXWMtj8=;
 b=ONTmn4J3fukbNwdjCmxyfHZBktC/rmTznKaHey6rUn3pkNag0622+5Q7VR4C35uizKWTzvjV1O4BstvXGJF8UEcKqO2w0NnovbOXgQyyxvuU/pcIEfeuWYglcwp/7RT1GwY85YzzDyByHmkm7sG2QvyjlIKpwuF3mnph50R4H0udUPo8tZ9Dzv1REu0fLQM/0vfci9Zap+FhCFS8HrmlW/INZnsmAmQyl4AZMgnjerq/yT0EN/wvBorwGInvQQ2s/O4DZXU6Gt3yP/UgYNaMYvFg/2MDM2xWDeYJ83dvXlupgWp48P39N0KAS906FdZaTU3DDdcyuaDk28VxCw/w8g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=citrix.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=IxyRVjkHNGh7/pGUHjKlkb4va6U54AzjGMRoKXWMtj8=;
 b=IzVKHgzOptxEg2GJO/814yf83wIvrG0wNuNakBX2LxBrzdkvszO6hfFl9HjljjFdArGPebpVPTjHS9QmvBCzu4rPdXKzMVxxijFLbSk4onPnTTNwSuKKJbwfgBJkFKdqOjhNfhDHBVX08EXvwlPC4g57r8FZuvRcyq4q2KPb5FQ=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <902d75e7-7717-4af1-93f0-e4115aecaa54@amd.com>
Date: Wed, 14 Feb 2024 09:20:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 1/3] xen: introduce Kconfig function alignment option
To: Roger Pau Monne <roger.pau@citrix.com>, <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	<wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Shawn Anastasio
	<sanastasio@raptorengineering.com>, Alistair Francis
	<alistair.francis@wdc.com>, Bob Eshleman <bobbyeshleman@gmail.com>, "Connor
 Davis" <connojdavis@gmail.com>
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-2-roger.pau@citrix.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240207145547.89689-2-roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF000000A0:EE_|SJ0PR12MB5423:EE_
X-MS-Office365-Filtering-Correlation-Id: 43549c80-f66a-4e84-6853-08dc2d35cc46
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	V0D+QWGgvZzjwZXPEALKVctmzljIHJYHqL3++2mVi2O1XufBkhXirjZXX2wK03dUzJ2TC0opndZdubSnUzOP4LHZzYUnYzzdntUAjDQKSi0jh5aWCy21zgQI5L0N6u02F9pwznpeJ4uOcmSKabWqc903X4Tn0IBeJi24uKNpkoXwB0+AX9kfieJI2myEN37gBdo4WaDgjjvtMwORz8RUXpHnA7TTU1HtSqFjwFHZnX0/RVz9YQFrDdMyWOG44mlfqDR4kAq69Q3L5wEdRwlQKqxQhj51Hfs5lvolPKcAJo2G6Piak7TwE15E/IPJYLuiH7jklAlzu04iHC7kLNz2Uo5r1q/LStSIUkLrxGgZWg1Xf4FbPkol9QABXXL9JZE4LpmnNDzUQ1N5prrk2/gT4qIBy5kgHuxlambkVvmYdVH6PVdvz5u3hK5xF+7+/BRibJaxcTPahhvOOZ2rb8XGum4pU5V0qdHE0Bs8Y+JrHEsMO0yvlyUt73wQSNRHUt6LK7XJWihJWcI3NZ3TK+EBQr3Y+/bnQSx029ZHX7M+XVA54Y0C7u+edQhKwfuErfBaSBLI+3wP8tBf2J4HdQw5EYKlepseSWK/zfzVd+lMI9g=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(396003)(376002)(39860400002)(346002)(136003)(230922051799003)(64100799003)(82310400011)(186009)(451199024)(1800799012)(40470700004)(36840700001)(46966006)(31686004)(2906002)(44832011)(4744005)(7416002)(5660300002)(336012)(26005)(426003)(2616005)(41300700001)(4326008)(53546011)(8676002)(70586007)(70206006)(8936002)(31696002)(478600001)(110136005)(16576012)(316002)(54906003)(81166007)(82740400003)(356005)(36756003)(86362001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 08:20:26.2825
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 43549c80-f66a-4e84-6853-08dc2d35cc46
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF000000A0.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB5423

Hi,

On 07/02/2024 15:55, Roger Pau Monne wrote:
> 
> 
> And use it to replace CODE_ALIGN in assembly.  This allows to generalize the
> way the code alignment gets set across all architectures.
> 
> No functional change intended.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
In xen/linkage.h, there is still a comment at the top mentioning that CODE_ALIGN needs to
be specified by each arch. I think this wants to be removed now. With that and Jan's remark
addressed, for Arm:
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 08:29:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 08:29:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680255.1058260 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raAde-0008SK-2R; Wed, 14 Feb 2024 08:29:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680255.1058260; Wed, 14 Feb 2024 08:29:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raAdd-0008SD-Vr; Wed, 14 Feb 2024 08:29:05 +0000
Received: by outflank-mailman (input) for mailman id 680255;
 Wed, 14 Feb 2024 08:29:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raAdc-0008Qq-PA
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 08:29:04 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1b44bafc-cb13-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 09:29:02 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-4108cbd92b9so29435425e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 00:29:02 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 hi25-20020a05600c535900b0040fd3121c4asm1185044wmb.46.2024.02.14.00.29.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 00:29:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b44bafc-cb13-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707899341; x=1708504141; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=ra+BUjoqFbebrpixPvBYqShNEgsG4wCfyFJmG/mPNRg=;
        b=EKyT6PrJEkeoNjyk1/S2xZmuffY/pjPPjGi0Jrubpxym9ft8shRUnYEjUzJJLkXsyW
         0S6h6O7upTq39Lna2Hwx720MPOEUkxSfLPY7y2WVBWCVw7vivMyoWTh1eQJQA5ajY6D5
         U8hhmCILpxGPQqVEl/8VN86Ntf/xUgTE4spLpQG+w2wLOgwalzAE5Ze1i04uTK/jgAxP
         L7Xtac/hI9Hki93XJZbCD7w44XBW3LGO0GRxCmHp+HQ60RaSz/O7HeaIt26yzEd8iDEV
         OQ4w+jBj8zIXP9axZ5VkThMLx0TeW30AkrWhrGN0Qf0BlVVU1KAwUWkWYajUh5V98f1g
         RH5w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707899341; x=1708504141;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ra+BUjoqFbebrpixPvBYqShNEgsG4wCfyFJmG/mPNRg=;
        b=M0GnM931cjlPMBKnXletJyxuxRncxhknIbVZjY6mt2bzMVF5w8ypzpF6FbrsuGqSm0
         KLKEcWiKlYNOF6GWZ2LheH7XYWiRBh2xR/IRN4v8/c1bcp/n8sbmVgRnwSbf32E9Li4r
         KPhaTk5NBGXeiDF5v/C3RyuOjPb+EbmG8ThU+VZvw6WIgifKQ0ppiDwrAUun6ZJG6oZo
         V0xbhOuly5VBA4B8ubCGAoYePmBhdvH0b//kUyJKiyXNg0vK4iShBYcD8MO04k6bqC74
         SIueVl9iajoBxF13C0zVMTm/rvjGu8HPyhAW+ErK2Npqyw8BK2LhWzauCgF50R5kjn6u
         ctQA==
X-Forwarded-Encrypted: i=1; AJvYcCXDqIpKLYg8d/3VrP0r3biwTrzMpiBfskZJ7UixodSoAwngP6rtl1jkjQ+rv43LB6DOkRZwB/ZZsa8tK4lLZtdIAyAPmMS4EUW3lnB69jM=
X-Gm-Message-State: AOJu0YyXhTPg671g8tPBjlNfAs53DLriFEFoDwNPDVhB8+w/NgLGiePi
	2H8+UEdSJ81GJ7jfnLRJhaEaPJb6yNgqhDTyRModCcvxlWthKDx9ss7ckFwfaw==
X-Google-Smtp-Source: AGHT+IFnno1UBPaJ4K7EleUo7t/e7wRRvzlM5+EbOUYbN8t5h5oU32x4QhbgKqGPgBefKm8EP8CI5Q==
X-Received: by 2002:a05:600c:4689:b0:411:a5f9:26f5 with SMTP id p9-20020a05600c468900b00411a5f926f5mr1582618wmo.38.1707899341390;
        Wed, 14 Feb 2024 00:29:01 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXJ4SQgpx3agcDB8YWuq6GEl7AdHDReF0n0qBwHNbuRt/lv62Uce3BV8WQzihlMvdj2GHJ+PMkwlpTF6rcpaZfoLImEC6m4fHXLIXvJpxIluUJBIyQzazeFvxMVo46iJ9RKUWx2JT96EyyYaxRyN/e8RCqpwuFPluUA3IfU1As7DKIdyG881WTHslo2L37hx8P6JqPSbddpENOmm21zdGEL1poeZEfjdMMANOXe07ByKs686XnrYM6asYoa6kQDIYYZUycQy648nKKf7frv/3G19UEOANw6yEodX02ykvviZ71tsnM7
Message-ID: <50719397-b053-43e1-9cf7-cc9eae9098ed@suse.com>
Date: Wed, 14 Feb 2024 09:28:59 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/3] docs/misra/rules.rst: add rule 5.5
To: Stefano Stabellini <stefano.stabellini@amd.com>,
 roberto.bagnara@bugseng.com
Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, julien@xen.org,
 bertrannd.marquis@arm.com, roger.pau@citrix.com,
 federico.serafini@bugseng.com, xen-devel@lists.xenproject.org
References: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
 <20240213223334.3693410-2-stefano.stabellini@amd.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240213223334.3693410-2-stefano.stabellini@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 13.02.2024 23:33, Stefano Stabellini wrote:
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> ---
>  docs/misra/rules.rst | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
> index c185366966..931158b354 100644
> --- a/docs/misra/rules.rst
> +++ b/docs/misra/rules.rst
> @@ -181,6 +181,12 @@ maintainers if you want to suggest a change.
>         headers (xen/include/public/) are allowed to retain longer
>         identifiers for backward compatibility.
>  
> +   * - `Rule 5.5 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_05_05.c>`_
> +     - Required
> +     - Identifiers shall be distinct from macro names
> +     - Clashes between function-like macros and non-callable entities
> +       are allowed. The pattern #define x x is also allowed.

Just for me to know what exactly is covered (hence also a question
to Roberto as to [to be] implemented Eclair behavior): Even when
the above would be sufficient (and imo better) people frequently
write

#define a(x, y) b(x, y)

which, transformed to the specific case here, would then be

#define a(x, y) a(x, y)

I'd assume such ought to also be covered, but that's not clear
from the spelling above.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 08:46:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 08:46:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680262.1058270 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raAu0-0003Ck-FS; Wed, 14 Feb 2024 08:46:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680262.1058270; Wed, 14 Feb 2024 08:46:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raAu0-0003Cd-BW; Wed, 14 Feb 2024 08:46:00 +0000
Received: by outflank-mailman (input) for mailman id 680262;
 Wed, 14 Feb 2024 08:45:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raAtz-0003CX-DW
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 08:45:59 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 792ba0bb-cb15-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 09:45:58 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-3394b892691so328748f8f.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 00:45:58 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 63-20020a250b42000000b00dc6f4c19d93sm434175ybl.7.2024.02.14.00.45.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 00:45:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 792ba0bb-cb15-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707900358; x=1708505158; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=dfAI7cQQVOxHv9jurTTv0zzrNIKmWJ0ubUC7DbbAu64=;
        b=SCX6ShSqDq7eHbiQ0OYF65rjsmo7rL8RyE392PPpt0nSgMMTOC2R/aIx/sWvaS1xCF
         YG3hsSVNEPxWQEwJP6pgeuaLlU95uarDq8qvRyEHHmieML4OD/exEhpJqNLUiWybCYQu
         wFOQFHpyPGXCpRd/cx1AqbuUQygRHeFxRIy+4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707900358; x=1708505158;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dfAI7cQQVOxHv9jurTTv0zzrNIKmWJ0ubUC7DbbAu64=;
        b=o+RAlSvINeC65C6uFWFc+QrilYcvgySY60jhSw3Z3+8StZLvvXyyfyrsofgXfK4Kb1
         lKR+c/XPhImCcSEn6/Me3jpaUb0tNcr8yhOAdV0aPe7o/C6Fuh8gwH+D8IsZ65X61wT1
         FkWY7CtyeGiKh2SeIH6ntTrdpGZ/3gELvLSEy91dBXraoGHu0YYHSqs6nXCGkCvPR32x
         lU6u8bFavLkiiIBalax4YBKTlN3U++p9OZEAUwYie+c0u1VdKEdANdfXeD2NIR8L08V5
         kydC8zMuNw3PwQD8mYjY5Hh3AU5JVOwAt6YZJrgsaDwo65FdMJu3UFnWW6ubKpduUi7n
         O9XA==
X-Forwarded-Encrypted: i=1; AJvYcCXTZ8vbYliviTk0PwGgO8FGBfWwPBlBded3/JtFtm8k/xPRsAMIXTjjx3jaL9jdeJ9bqpVvIkpcE9Z7zyN3qzQ84Jiryj+BMympZqTUaoo=
X-Gm-Message-State: AOJu0Yx9x5ihoXE8C6Go1FZsttyMG/R87PdI2pAzAjUrasf11M6Qd5ol
	LTracUSL79YaHXd56tqVtYnsfAI9vT1Tso+cYcFZ5hMo3FeIUR9DEUxOxxcviFo=
X-Google-Smtp-Source: AGHT+IHby6RD/J6D0zkIY7ePysX4BDBO1uavDgUwY4RnR4fbOWJEBOS/OgbdKnR/+jioIU9IbfbPpg==
X-Received: by 2002:a5d:44cd:0:b0:33c:e395:bd8c with SMTP id z13-20020a5d44cd000000b0033ce395bd8cmr1194547wrr.7.1707900357810;
        Wed, 14 Feb 2024 00:45:57 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW7aKGKIH1ppKlzjvGI4dLNho4ZRO1XHBF7632bexTgtKFLvT1Jiroi8o2zAYFGxPNDw78xB8zp2/XE7Amz8Wo4eK/c4CEXdPxWjsB1uDaH2+rVR7FArKusz6+XA6zoaFp2Jmf+
Date: Wed, 14 Feb 2024 09:45:55 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
Subject: Re: [regression] Re: [PATCH v2 2/2] iommu/vt-d: switch to common
 RMRR checker
Message-ID: <Zcx9w5eZkXiN9s66@macbook>
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-3-roger.pau@citrix.com>
 <c27c76ec-36cd-43cb-b76f-e8f95fb27ed8@suse.com>
 <c4f27180-86bf-45fd-8641-bd160c6de229@citrix.com>
 <37ed1abe-afcc-4a76-8a86-623282ca37a3@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <37ed1abe-afcc-4a76-8a86-623282ca37a3@suse.com>

On Wed, Feb 14, 2024 at 08:45:28AM +0100, Jan Beulich wrote:
> On 13.02.2024 23:37, Andrew Cooper wrote:
> > On 12/02/2024 2:38 pm, Jan Beulich wrote:
> >> On 07.02.2024 16:34, Roger Pau Monne wrote:
> >>> Use the newly introduced generic unity map checker.
> >>>
> >>> Also drop the message recommending the usage of iommu_inclusive_mapping: the
> >>> ranges would end up being mapped anyway even if some of the checks above
> >>> failed, regardless of whether iommu_inclusive_mapping is set.  Plus such option
> >>> is not supported for PVH, and it's deprecated.
> >>>
> >>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> >> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> > 
> > XenRT says no.
> > 
> > It's not clear exactly what's going on here, but the latest resync with
> > staging (covering only today's pushed changes) suffered 4 failures to
> > boot, on a mix of Intel hardware (SNB, SKL, SKX and CLX).
> > 
> > All 4 triple-fault-like things where following a log message about an RMRR:
> > 
> > (XEN) RMRR: [0x0e8 ,0x0e8] is not (entirely) in reserved memory
> > 
> > not being in reserved memory.
> > 
> > 
> > First of all - fix this printk() to print full addresses, not frame
> > numbers.  It's obnoxious to cross reference with the E820.
> 
> Perhaps better indeed. The stray blank before the comma also wants dropping.
> And while looking over the patch again, "mfn_t addr;" also isn't very
> helpful - the variable would better be named mfn.

I can adjust those in the fix, see below.

> > It's very likely something in this series, but the link to Intel might
> > just be chance of which hardware got selected, and I've got no clue why
> > there's a reset with no further logging out of Xen...
> 
> I second this - even after looking closely at the patches again, I can't
> make a connection between them and the observed behavior. Didn't yet look
> at what, if anything, osstest may have to say. Do I understand correctly
> that the cited log messages are the last sign of life prior to the
> systems rebooting?

I've found it:

    for ( addr = start; mfn_x(addr) <= mfn_x(end); mfn_add(addr, 1) )

Should be:

    for ( addr = start; mfn_x(addr) <= mfn_x(end); addr = mfn_add(addr, 1) )

mfn_add() doesn't modify the parameter.  Will see about making those
helpers __must_check in order to avoid this happening in the future.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:00:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:00:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680267.1058280 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raB83-0006CA-JN; Wed, 14 Feb 2024 09:00:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680267.1058280; Wed, 14 Feb 2024 09:00:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raB83-0006C3-GD; Wed, 14 Feb 2024 09:00:31 +0000
Received: by outflank-mailman (input) for mailman id 680267;
 Wed, 14 Feb 2024 09:00:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Sn1=JX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1raB82-0006Bx-Qx
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:00:30 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7f2f5d57-cb17-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 10:00:28 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-33b4e6972f6so2684540f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:00:27 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 u14-20020a056000038e00b0033ce40eee9fsm1798089wrf.58.2024.02.14.01.00.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 01:00:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7f2f5d57-cb17-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707901227; x=1708506027; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RwQM1N90LXQ5crfNmukPzfyN4MW+nE2j5dkoQyyKrvo=;
        b=uOKXYwtHM546LmsXu8DZqX5Qnh7AWC5eeKYG/gLzCpyVqE9ngPPgaGDQrp3IRcvTBt
         H0I77Ilk6q4bPHVY9giMzbymfJwrvtKVBKe0O+B0QGsvkHKBOKHyjU3iOR2QjOOGNRPp
         E5ZOaV3vwWGwl6GcI74Cx20qI70TyJMx2wgVw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707901227; x=1708506027;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RwQM1N90LXQ5crfNmukPzfyN4MW+nE2j5dkoQyyKrvo=;
        b=I5CmtJJyahUPPYnlbewO5viAiYKz/WCioccwKMf8kHUJeabnsxm/Z7LMk/bu9w5Q8+
         8/sJsqVqqt1Mav5mFxePwIRUm8hVDPdSmAnSV/BNQ8hBIA3xik4cfzJf8StBwbngwdCO
         LDMpRWfefM7eRJ3Pu8BlHd2WHQ4v1LNJWjpZm6zLgzIZdvi1qiwtThco3l6QpWbZnlWl
         kImKJFsrgyLVh0lZhD1G3fCPr6Z5UgsutTwmo4KGDm+zy7JKRmVga2Z1JI8KocUoQcZd
         RyIl6kyViitDCe7jpP6Eta4FStc4HbqYshTbO7sKWFqUyOjsOWUW74XzcGmWMeQ/z3w8
         44Eg==
X-Forwarded-Encrypted: i=1; AJvYcCVi5CgvV83QvVS8Q6XLgRKY/8LK3qSFcCr3KXxBvgSynvMFwAAyPkFbYvUUtRPkj6gO/SkxP40JDEveU93U4GHoZ+XHRv9YI+Z3fyfPDC4=
X-Gm-Message-State: AOJu0YwM8B7h4Wr4a4s+lZCyJV3n7MFH84ILj0nxbe1dNBrq24jJQRef
	w5/UFr11yzuToLjDDjQmKee5CyxzGZJczlYY3KIKBn0FEFmxkLwOXL4vLedrey0=
X-Google-Smtp-Source: AGHT+IEg8k9qQcWB/80LknhA8qSv44R3lHhwppCC9PHFfdJm8V+FLB7InMkyqC4zgrWZyDlGhYypMA==
X-Received: by 2002:adf:f106:0:b0:33b:6369:6485 with SMTP id r6-20020adff106000000b0033b63696485mr1374153wro.24.1707901227002;
        Wed, 14 Feb 2024 01:00:27 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXM1DXQZ1ouIIfSmPq7iSf1qy489WAwMvyjKOP5NF+WnbNn/q0Dxu5msb7WQ6O3rbu+WAFdsG7+YPoRIplFQ16R6hPzt4VFVHAqPoGyctJi0hqsZXwn9iWXG5V6Ry/ZKdhK
Message-ID: <c84a640e-3332-4f2d-9c4a-c6806bb2da67@citrix.com>
Date: Wed, 14 Feb 2024 09:00:25 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [regression] Re: [PATCH v2 2/2] iommu/vt-d: switch to common RMRR
 checker
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jan Beulich <jbeulich@suse.com>
Cc: Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-3-roger.pau@citrix.com>
 <c27c76ec-36cd-43cb-b76f-e8f95fb27ed8@suse.com>
 <c4f27180-86bf-45fd-8641-bd160c6de229@citrix.com>
 <37ed1abe-afcc-4a76-8a86-623282ca37a3@suse.com> <Zcx9w5eZkXiN9s66@macbook>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <Zcx9w5eZkXiN9s66@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/02/2024 8:45 am, Roger Pau Monné wrote:
> On Wed, Feb 14, 2024 at 08:45:28AM +0100, Jan Beulich wrote:
>> On 13.02.2024 23:37, Andrew Cooper wrote:
>>> On 12/02/2024 2:38 pm, Jan Beulich wrote:
>>>> On 07.02.2024 16:34, Roger Pau Monne wrote:
>>>>> Use the newly introduced generic unity map checker.
>>>>>
>>>>> Also drop the message recommending the usage of iommu_inclusive_mapping: the
>>>>> ranges would end up being mapped anyway even if some of the checks above
>>>>> failed, regardless of whether iommu_inclusive_mapping is set.  Plus such option
>>>>> is not supported for PVH, and it's deprecated.
>>>>>
>>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>> XenRT says no.
>>>
>>> It's not clear exactly what's going on here, but the latest resync with
>>> staging (covering only today's pushed changes) suffered 4 failures to
>>> boot, on a mix of Intel hardware (SNB, SKL, SKX and CLX).
>>>
>>> All 4 triple-fault-like things where following a log message about an RMRR:
>>>
>>> (XEN) RMRR: [0x0e8 ,0x0e8] is not (entirely) in reserved memory
>>>
>>> not being in reserved memory.
>>>
>>>
>>> First of all - fix this printk() to print full addresses, not frame
>>> numbers.  It's obnoxious to cross reference with the E820.
>> Perhaps better indeed. The stray blank before the comma also wants dropping.
>> And while looking over the patch again, "mfn_t addr;" also isn't very
>> helpful - the variable would better be named mfn.
> I can adjust those in the fix, see below.
>
>>> It's very likely something in this series, but the link to Intel might
>>> just be chance of which hardware got selected, and I've got no clue why
>>> there's a reset with no further logging out of Xen...
>> I second this - even after looking closely at the patches again, I can't
>> make a connection between them and the observed behavior. Didn't yet look
>> at what, if anything, osstest may have to say. Do I understand correctly
>> that the cited log messages are the last sign of life prior to the
>> systems rebooting?
> I've found it:
>
>     for ( addr = start; mfn_x(addr) <= mfn_x(end); mfn_add(addr, 1) )
>
> Should be:
>
>     for ( addr = start; mfn_x(addr) <= mfn_x(end); addr = mfn_add(addr, 1) )
>
> mfn_add() doesn't modify the parameter.  Will see about making those
> helpers __must_check in order to avoid this happening in the future.

There's only a single thing in this function which wants an mfn_t. 
Everything else is operating on raw paddr_t's.  I'd suggest converting
types at the start and using plain numbers.

Also, while I hate to nitpick, iommu_unity_region_ok() really ought to
be iommu_check_unity_region().  It is not a predicate (given the
additional fixups), so the function name shouldn't read as one.

Also, the "not (entirely) in reserved memory" line ought to have an ";
adjusting" on the end to make it clear that it's making an adjustment in
light of finding the range not reserved.

Finally, the "can't be converted" error should render type, even if only
in numeric form.

What do we do when there's a region that's marked as RAM?

As to the triple-fault-like nature, given that it's an infinite loop, I
expect that it was our test automation getting unhappy and power cycling
the systems after seeing no signs of starting the installer.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:01:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:01:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680270.1058290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raB9J-0006sV-2F; Wed, 14 Feb 2024 09:01:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680270.1058290; Wed, 14 Feb 2024 09:01:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raB9I-0006sO-VW; Wed, 14 Feb 2024 09:01:48 +0000
Received: by outflank-mailman (input) for mailman id 680270;
 Wed, 14 Feb 2024 09:01:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raB9H-0006sI-P8
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:01:47 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id add02d99-cb17-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 10:01:45 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-511a45f6a57so729302e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:01:45 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 hi25-20020a05600c535900b0040fd3121c4asm1276799wmb.46.2024.02.14.01.01.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 01:01:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: add02d99-cb17-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707901305; x=1708506105; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Q0ur9nELkuWSS0YY4di7Ol5dbQl/XWQkAoCQYw4HsM8=;
        b=CgUF6PMkHj4jyslE9Ujt5g7O12JOBK6Q+jSYaLkOYAVk2N//jvBBNvYsFdZ2guMw6n
         eK94HzxksKIuU/7MDUbRi2ALYiRPJeq+C5jJbsMfpx4MOs6Rdp8Jy99sHSoA+iX7M2GJ
         ZPQC0O/jSCnqUsQT07TXPQSFGu00LfEI72YxeLVNSdtaz6IuoQXTiAoLJlaaZAmx3B1g
         9AvUVgwXfM4gA2NfSAlFCVdDB+b6V7f/tA7Pd8OnIqMxdMYD20wmmLoza5tfdFOecg05
         Q92aLL/YkHdy70oyq42ggaJyHAFG8drdG0GgFIGNen5SDliXFku0FD5656EgL8qu5ubs
         HVNQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707901305; x=1708506105;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Q0ur9nELkuWSS0YY4di7Ol5dbQl/XWQkAoCQYw4HsM8=;
        b=YzzqX3QMDMFtKNC4N8Z3w3VB3VpMdsjdhcsGsHAPBtkvnyXxdh1jqrKpGT1rdIkB2K
         KQGU3WE4KqVfNpGfj/Dr343ExD/uyhYlNLKDkREKGUklJnubZktj7n6yn/eh6bilFWso
         k/JZi+ILTc3wUzodp9qSG2Y3JeGaK3XU4cexU36juzozgAgoCz3wYhjuviUvmQ7cEpz5
         3fPwung8a0QTsNUnruav80of3BVMeQikjSatZPtnIwYWruhvcRmnPeoFffiJxZ8hPrpH
         Qc/LXVBh07ce/7/QloVFz+tdN6Z33dAkBto9ZbZ+7Noms2houLuTvYg0FyiyRPbfqmK7
         gApA==
X-Forwarded-Encrypted: i=1; AJvYcCWl7XGsFuadZF2LoyZvwQyOJb0hjTNmaHGUehBXTmGAhtSo6ZmZyUXd4BmNOxejr3jgz8XHfDhmiXV9P2VtJVJGdCQGQNp1dJoGlqqtIBM=
X-Gm-Message-State: AOJu0YztFuIWfujnjyyUnth+qxrGa4OS9RLmGHBdJPPRGKZwecbW9kCJ
	kxFicFX+YQNP9ZYYmrWBOdwJITZ4V4JuXw7XB7Pg7Z81qplW/KFl9g+x7S8L7A==
X-Google-Smtp-Source: AGHT+IHFIFktTo3watGZnMVNWV8V+fi0IZlCpQIqRa2rZDhNnXIvNAt6/dEk023SkjPqJYIqdFv06g==
X-Received: by 2002:ac2:5e8d:0:b0:511:5537:fb26 with SMTP id b13-20020ac25e8d000000b005115537fb26mr1532705lfq.39.1707901305139;
        Wed, 14 Feb 2024 01:01:45 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWKtHMzFPmTacNM0Q8yxgcyTf7HpN/58W9N2pL/IllLC4GHcTv9TxuNJU8c8Cm+rgpl3yOg5bs1bKZM3HGALBhXiFf1T5GBCz8Xuz5KRuSAEuFu9I8sSm4Z1ifcZoW3XaJ9j0oZ
Message-ID: <790c81c8-04f4-435b-937f-87fa1cd54998@suse.com>
Date: Wed, 14 Feb 2024 10:01:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [regression] Re: [PATCH v2 2/2] iommu/vt-d: switch to common RMRR
 checker
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-3-roger.pau@citrix.com>
 <c27c76ec-36cd-43cb-b76f-e8f95fb27ed8@suse.com>
 <c4f27180-86bf-45fd-8641-bd160c6de229@citrix.com>
 <37ed1abe-afcc-4a76-8a86-623282ca37a3@suse.com> <Zcx9w5eZkXiN9s66@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zcx9w5eZkXiN9s66@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 09:45, Roger Pau Monné wrote:
> On Wed, Feb 14, 2024 at 08:45:28AM +0100, Jan Beulich wrote:
>> On 13.02.2024 23:37, Andrew Cooper wrote:
>>> It's very likely something in this series, but the link to Intel might
>>> just be chance of which hardware got selected, and I've got no clue why
>>> there's a reset with no further logging out of Xen...
>>
>> I second this - even after looking closely at the patches again, I can't
>> make a connection between them and the observed behavior. Didn't yet look
>> at what, if anything, osstest may have to say. Do I understand correctly
>> that the cited log messages are the last sign of life prior to the
>> systems rebooting?
> 
> I've found it:
> 
>     for ( addr = start; mfn_x(addr) <= mfn_x(end); mfn_add(addr, 1) )
> 
> Should be:
> 
>     for ( addr = start; mfn_x(addr) <= mfn_x(end); addr = mfn_add(addr, 1) )
> 
> mfn_add() doesn't modify the parameter.  Will see about making those
> helpers __must_check in order to avoid this happening in the future.

Hmm, yes, it's not the first time this has happened. But even seeing the
flaw I still can't explain the observed behavior: The system ought to
hang then, not instantly reboot?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:08:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:08:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680273.1058300 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBFI-0007ex-Ok; Wed, 14 Feb 2024 09:08:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680273.1058300; Wed, 14 Feb 2024 09:08:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBFI-0007eq-KZ; Wed, 14 Feb 2024 09:08:00 +0000
Received: by outflank-mailman (input) for mailman id 680273;
 Wed, 14 Feb 2024 09:07:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raBFH-0007ei-AQ
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:07:59 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8bdec29d-cb18-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 10:07:58 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-5114b2b3b73so6118635e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:07:58 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 dx2-20020a05600c63c200b0040fdd7cbc8dsm1286762wmb.47.2024.02.14.01.07.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 01:07:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8bdec29d-cb18-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707901678; x=1708506478; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=fmjmg7VEdxkzSuaPj3H56MSSyKHu4neqLtCFh8PmLdI=;
        b=Kvxm76aG0yfUysKfhVHegf49UblaDinBvvejnJyeCyS2KWgs9LAjwhCNYs2OCYht56
         K+SuRZLubz96pScrAg44CQZFHDOqHJmCRzw+PvGiK3+8iECAJpTwjzsZ0lYNTGTuOpxK
         FvNXIovAqDCnJ9Y2J8WQvK8nTxXqBzhoTuvVv6JktReea/lfLWrwsOjhCvAxrmHzSgKr
         rDNVN8ACy9WusQgPgWYELgJWF0jtKeH1es8+7FKKY0O3XfnhSOxR7EaCP3NVEvzDP5/Y
         lfufatdrDUUGLJq5IwCPLLbI00dmw23vdZWzubMEsp+K3blXd5bDdOFGk1lhC7sQuPMY
         bBhg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707901678; x=1708506478;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=fmjmg7VEdxkzSuaPj3H56MSSyKHu4neqLtCFh8PmLdI=;
        b=Dh9XHIFIdLJYyUeu26JJm0yOaNcdXHp2Ho0WsdakLoczZG9Z5bwqtf0ropaFBZIa9s
         ExvO7YeBE4VV0No8Dz2rFi6xNNOh4bzWW/SW8nrx7jZsCsJarxmq2R5UsS7tKihvAAhX
         25ZoJcCPNWyoHOOe6lxfJN/FetXLy8y0ed6j5AwnFmqngUHunZaF6ngIp9HcHHHjAEzm
         wFohqy3zCoxY53Bjox6IeH/uq85wHW/cWfbkI3QWJGwHPRcZQaJ15xnecTq1Fw7weFnE
         JCcz1Ahn57jhoWe8A6JNP6JUZhgxPFa0r+wFu3lQ35kL0wkDbffigHV6c4xhgDG0DcCn
         IOpg==
X-Gm-Message-State: AOJu0Yw8IS8mGrWYT6aszYk/PzojcW/gJX8n51a6AgAIwzn7E268EQZL
	lXwJJimjUhfX3+94gkOBc7WX4FZve967n4PSzvxYRhxjvMFyNgKqi4TREO/+NA==
X-Google-Smtp-Source: AGHT+IEjxo5i4Wyw5qFN5W4+zvKJIIXcGgiCjZo2OD6lBnEUdzXTZSAINmJUi25V/KD+ALXjhjNmYg==
X-Received: by 2002:ac2:4bcd:0:b0:511:a024:dbaa with SMTP id o13-20020ac24bcd000000b00511a024dbaamr2117616lfq.3.1707901677731;
        Wed, 14 Feb 2024 01:07:57 -0800 (PST)
Message-ID: <911f37dc-b4d9-42c1-9575-98ef93099976@suse.com>
Date: Wed, 14 Feb 2024 10:07:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
 Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Paul Durrant <paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <Zcsp15Aqve11Icjb@macbook> <ec5d0c39-1559-4f10-9574-98cfa0542993@suse.com>
 <ZcswBFHtINB1XMAS@macbook> <db0ce984-88e3-4394-b270-c5db585f8209@suse.com>
 <f3b34d97-17b5-44ef-ad9e-ad04a96754d0@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f3b34d97-17b5-44ef-ad9e-ad04a96754d0@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 13.02.2024 17:58, Stewart Hildebrand wrote:
> On 2/13/24 04:05, Jan Beulich wrote:
>> On 13.02.2024 10:01, Roger Pau Monné wrote:
>>> On Tue, Feb 13, 2024 at 09:44:58AM +0100, Jan Beulich wrote:
>>>> On 13.02.2024 09:35, Roger Pau Monné wrote:
>>>>> On Fri, Feb 02, 2024 at 04:33:05PM -0500, Stewart Hildebrand wrote:
>>>>>> --- a/xen/include/xen/sched.h
>>>>>> +++ b/xen/include/xen/sched.h
>>>>>> @@ -462,7 +462,8 @@ struct domain
>>>>>>  #ifdef CONFIG_HAS_PCI
>>>>>>      struct list_head pdev_list;
>>>>>>      /*
>>>>>> -     * pci_lock protects access to pdev_list.
>>>>>> +     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
>>>>>> +     * structure from being removed.
>>>>>>       *
>>>>>>       * Any user *reading* from pdev_list, or from devices stored in pdev_list,
>>>>>>       * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,
>>>>>> @@ -628,6 +629,18 @@ struct domain
>>>>>>      unsigned int cdf;
>>>>>>  };
>>>>>>  
>>>>>> +/*
>>>>>> + * Check for use in ASSERTs to ensure that:
>>>>>> + *   1. we can *read* d->pdev_list
>>>>>> + *   2. pdevs (belonging to this domain) do not go away
>>>>>> + *   3. pdevs (belonging to this domain) do not get assigned to other domains
>>>>>
>>>>> I think you can just state that this check ensures there will be no
>>>>> changes to the entries in d->pdev_list, but not the contents of each
>>>>> entry.  No changes to d->pdev_list already ensures not devices can be
>>>>> deassigned or removed from the system, and obviously makes the list
>>>>> safe to iterate against.
>>>>>
>>>>> I would also drop the explicitly mention this is intended for ASSERT
>>>>> usage: there's nothing specific in the code that prevents it from
>>>>> being used in other places (albeit I think that's unlikely).
>>>>
>>>> But pcidevs_locked(), resolving to spin_is_locked(), isn't reliable. The
>>>> assertion usage is best-effort only, without a guarantee that all wrong
>>>> uses would be caught.
>>>
>>> Do we want to protect this with !NDEBUG guards then?
>>
>> Yes, that would look to be desirable.
> 
> We will then also need a definition of pdev_list_is_read_locked() in the
> #else case so we don't risk running into "error: implicit declaration of
> function 'pdev_list_is_read_locked'".
> 
> Such a definition might look like:
> 
> #define pdev_list_is_read_locked(d) ({ (void)d; ASSERT_UNREACHABLE(); false; })
> 
> so that we still evaluate d exactly once in the NDEBUG case.

Except that ASSERT_UNREACHABLE() use is bogus in the NDEBUG case. The
way our ASSERT() works in the NDEBUG case looks to make it sufficient
for there to be

bool pdev_list_is_read_locked(const struct domain *d);

in the #else case (with no implementation anywhere).

>>>>>> + * This check is not suitable for protecting other state or critical regions.
>>>>>> + */
>>>>>> +#define pdev_list_is_read_locked(d) ({                           \
>>>>>
>>>>> I would be tempted to drop at least the '_read_' part from the name,
>>>>> the name is getting a bit too long for my taste.
>>>>
>>>> While I agree with the long-ish aspect, I'm afraid the "read" part is
>>>> crucial. As a result I see no room for shortening.
>>>
>>> OK, if you think that's crucial then I'm not going to argue.
>>>
>>>>>> +        struct domain *d_ = (d);                                 \
>>>>>
>>>>> Why do you need this local domain variable?  Can't you use the d
>>>>> parameter directly?
>>>>
>>>> It would be evaluated then somewhere between 0 and 2 times.
>>>
>>> It's ASSERT code only, so I don't see that as an issue.
>>
>> Fair point.
>>
>>>  Otherwise d_ needs to be made const.
>>
>> Indeed, but for assert-only code I agree the option is slightly better,
>> ideally suitably commented upon.
> 
> Is "the option" here referring to making d_ const, or using d directly
> (with suitable comment)?

The latter.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:09:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:09:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680278.1058310 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBGt-0008FZ-12; Wed, 14 Feb 2024 09:09:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680278.1058310; Wed, 14 Feb 2024 09:09:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBGs-0008FS-Ua; Wed, 14 Feb 2024 09:09:38 +0000
Received: by outflank-mailman (input) for mailman id 680278;
 Wed, 14 Feb 2024 09:09:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raBGr-0008Dg-MM
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:09:37 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c68fdf81-cb18-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 10:09:36 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a2f22bfb4e6so714674366b.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:09:36 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 hu18-20020a170907a09200b00a3d1cce7c6fsm1024204ejc.62.2024.02.14.01.09.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:09:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c68fdf81-cb18-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707901776; x=1708506576; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=sPUlJDhf2pZebErIfa7t3QKc2ftExv3cCl/QvvNC50o=;
        b=gISCj7BfF1c78he4gxssnNYmwwCzAr0ZRQelcUFMIA8thH1EcGwURvSaCXc+EZ8GCW
         kknLSzcg5kh7HRAbDUo4EfizupgTG6mK9RvIbUwRvYlHicrq1CG+94cZde+P3bgqCEBS
         PWt+XLcUvndn5LhcMVyPfW5LrxL39cKr2milQ26QVYJZU1Mn4eCJ9hqVW5r5Z4jx7G36
         c1JEGms/GzIkdBmYXn1Q3wh7W7vqcGXbTpPZn6CfF1VCP2GekgZ/Cw/85FZofp8unwzN
         icGdsgWTNap/aoukI5h0B/AOyfRkZkAr1A2CEDk8GI58TeAnCyXve/DGgD85uMa1KYBN
         vMgw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707901776; x=1708506576;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sPUlJDhf2pZebErIfa7t3QKc2ftExv3cCl/QvvNC50o=;
        b=YfRHcXpJG4DeoAnxxSjh2sDFvk6vqPfVgF1ipoFB/xOK3/ap8TP41kHE+4t76i+gN2
         ZEr1s8gWK7tlLLqafb7p180P9leDu2Omf5pw0MR1nKSXWZlYIISM43kLTcCamcKrD9pE
         +/i6Exdf94qo4IBYyITqYYC6XjOdwQ4XQfhtGbvzuFSn8wU4UGsfr8sNlGfW/4v1HrzU
         aYTHOMPtr+479vXB0irEBqGU14iMkJN0scrlY0D0LPpqsBQiml9ziM4R7SrnkJcl4Jel
         VucLnLuTD+H/WjzNPUFuODzVN8/Wt1p7TKFyRK7ekRklclR/nng0hMbS819o/3gqf+IA
         y2Fg==
X-Forwarded-Encrypted: i=1; AJvYcCVuxpSnYmPIVF7Z0+h6S5FnwLOQoDoQeX5JB6XMDrNh/shNa5COjFf5kW4g8GshcMUH+DfVQMdhByWJ+thZzRttTKvWCoBeRDmHBNEAW9s=
X-Gm-Message-State: AOJu0YwVKGrsgQvnOiWUQdqw5jG9brGsoQaoB+IuBzT8w9OYRqAE7yf/
	7AbFqP5snCUREn1PRsRPSIAXnvhELKUKYQWXyQYfX/V/ht7N/o6S
X-Google-Smtp-Source: AGHT+IESqlnjGRol16m9Vchh8ZbjQ+C5afhSr/N8b7cDb4rSA9Yhnp1Dun2mWKHC4K9iEQmM8gH4HA==
X-Received: by 2002:a17:906:3710:b0:a3d:50ee:e636 with SMTP id d16-20020a170906371000b00a3d50eee636mr649809ejc.27.1707901775866;
        Wed, 14 Feb 2024 01:09:35 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVbjpdaGJg30YO0XsjCmow2nrWqrzzM3gnru8Cx6Ihw/0x9/FyiAysjnP1mSm1H72ywq1/82bJ36C8jCgWQ4Gv0kK5xsd4K9REQ72V2RFS0B1xtBkOv0B6z83IbmDxulRrv8YPN5ypXoF8Qr1UfjmED9jkJaEHxww8ptnbkqXUj40lqwY4+erLORrABVvXHzYYsM4HDBGquET/GKyn9Jy5u/QbmRn3kw/pq0rVQZNH+JBAUggFvnKQ1kjWlL8ABFhHmfqXE3GKappvhMEz2jRwE4kBXIzklb34t9xoGxU7iQzxLH9sJOlmaBNr75aHxCxRY7fUX2Ge81P7S1aAUxpdqbXTi4KXQe57U7qbEggnnHinvRq3TySgDS3kByA==
Message-ID: <ea20431452c5e7ec7efc02b1d2832bb29d33a769.camel@gmail.com>
Subject: Re: [PATCH v8 3/7] xen/asm-generic: introduce stub header monitor.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Tamas K Lengyel <tamas@tklengyel.com>, Alexandru Isaila
 <aisaila@bitdefender.com>, Petre Pircalabu <ppircalabu@bitdefender.com>, 
 Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>,  Shawn Anastasio
 <sanastasio@raptorengineering.com>, Jan Beulich <jbeulich@suse.com>
Date: Wed, 14 Feb 2024 10:09:34 +0100
In-Reply-To: <4cc1eb90-e3ef-4225-aa98-441c11c3fc8a@xen.org>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
	 <84568b0c24a5ec96244f3f34537e9a148367facf.1707499278.git.oleksii.kurochko@gmail.com>
	 <e12914b8-5c6a-4085-b5e8-03c5bdcc2c27@xen.org>
	 <4cc1eb90-e3ef-4225-aa98-441c11c3fc8a@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0


Hi Julien,

> On 13/02/2024 17:40, Julien Grall wrote:
> > Hi Oleksii,
> >=20
> > On 09/02/2024 18:00, Oleksii Kurochko wrote:
> > > The header is shared between several archs so it is
> > > moved to asm-generic.
> > >=20
> > > Switch partly Arm and PPC to asm-generic/monitor.h and only
> > > arch_monitor_get_capabilities() left in arch-specific/monitor.h.
> > >=20
> > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > > Acked-by: Jan Beulich <jbeulich@suse.com>
> > > Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
> > > ---
> > > Changes in V8:
> > > =C2=A0 - Add Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
> > > ---
> > > Changes in V7:
> > > =C2=A0 - Drop definition of arch_monitor_domctl_event for PPC.
> > > =C2=A0 - define arch_monitor_domctl_event in asm-generic/monitor.h.
> > > =C2=A0 - add "define HAS_ARCH_MONITOR_DOMCTL_EVENT" in
> > > arm/.../monitor.h as=20
> > > it has arch specific implementation.
> > > ---
> > > Changes in V6:
> > > =C2=A0 - Rebase only.
> > > ---
> > > Changes in V5:
> > > =C2=A0=C2=A0 - Switched partly Arm and PPC to asm-generic monitor.h o=
nly
> > > =C2=A0=C2=A0=C2=A0=C2=A0 arch_monitor_get_capabilities() left in arch=
-
> > > specific/monitor.h.
> > > =C2=A0=C2=A0 - Updated the commit message.
> > > ---
> > > Changes in V4:
> > > =C2=A0 - Removed the double blank line.
> > > =C2=A0 - Added Acked-by: Jan Beulich <jbeulich@suse.com>.
> > > =C2=A0 - Update the commit message
> > > ---
> > > Changes in V3:
> > > =C2=A0 - Use forward-declaration of struct domain instead of "
> > > #include=20
> > > <xen/sched.h> ".
> > > =C2=A0 - Add ' include <xen/errno.h> '
> > > =C2=A0 - Drop PPC's monitor.h.
> > > ---
> > > Changes in V2:
> > > =C2=A0=C2=A0=C2=A0=C2=A0- remove inclusion of "+#include <public/domc=
tl.h>"
> > > =C2=A0=C2=A0=C2=A0=C2=A0- add "struct xen_domctl_monitor_op;"
> > > =C2=A0=C2=A0=C2=A0=C2=A0- remove one of SPDX tags.
> > > ---
> > > =C2=A0 xen/arch/arm/include/asm/monitor.h | 25 +-----------
> > > =C2=A0 xen/arch/ppc/include/asm/monitor.h | 28 +------------
> >=20
> > Looking at MAINTAINERS, monitor.h was covered by "VM EVENT, MEM
> > ACCESS=20
> > and MONITOR". As you move to ...
> >=20
> > > =C2=A0 xen/arch/ppc/stubs.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 8 ----
> > > =C2=A0 xen/include/asm-generic/monitor.h=C2=A0 | 64
> > > ++++++++++++++++++++++++++++++
> >=20
> > ... asm-generic/, I believe it will now fall under "THE REST". So I
> > think MAINTAINERS needs to be updated to also cover asm-
> > generic/monitor.h.
> >=20
> > Looking at what was already committed, I think asm-
> > generic/vm_event.h=20
> > should also be added in MAINTAINERS. Can you send a separate patch
> > for=20
> > that?
>=20
> Oh, I just noticed that we have the following entry:
>=20
> xen/include/*/monitor.h
>=20
> So the header is already covered. Sorry for the noise.
>=20
> As the code is falling under the "VM EVENT..." subsystem, then
> Tamas's=20
> acked is technically sufficient for this patch (even for the PPC
> change).
>=20
> But just in case you need one for Arm:
>=20
> Acked-by: Julien Grall <jgrall@amazon.com>
Thanks.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:12:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:12:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680283.1058320 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBJj-0001TX-E6; Wed, 14 Feb 2024 09:12:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680283.1058320; Wed, 14 Feb 2024 09:12:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBJj-0001TQ-BN; Wed, 14 Feb 2024 09:12:35 +0000
Received: by outflank-mailman (input) for mailman id 680283;
 Wed, 14 Feb 2024 09:12:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raBJi-0001TK-6n
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:12:34 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2ecd64f7-cb19-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 10:12:31 +0100 (CET)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-5638dbf1417so10572a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:12:31 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 ps9-20020a170906bf4900b00a3cdd35688fsm2039481ejb.80.2024.02.14.01.12.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:12:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ecd64f7-cb19-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707901951; x=1708506751; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=xyyuoeLMupPgoFpyYAnfAIdumibcYl5ELRc3XTnzK9c=;
        b=TTeVL64UUj44UXt+RP508ZYKhCz0jUPnYyH9+eH8MBWAOVciIXpTxbZYRmz+alGxwl
         95bALRCyzbvXaQuCbbaS8fiNG0Y0sCs5Pm98D/H2M4cnyOBFwXtDo56/KE7sBpNwE3ev
         +RLqzknhHw0Ez9yy7FOU+jLJ/BkNZ2bFgZm4oYkRhF+tICgUZcgU+m/Uiq09qhMk0o2j
         1nfED2dWjnmCH7121tJiwgZtkDMZMErjGqR0eQHpuEWlyN6PLrQhL4pwZ4xAakoU738z
         /HGcIArcI/ZQ4fGOC1PohZTZ7m1hQaP6oJYXtoDnEdIPaloQK6tqSBMFKiDQI1DIMKT5
         rm5A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707901951; x=1708506751;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xyyuoeLMupPgoFpyYAnfAIdumibcYl5ELRc3XTnzK9c=;
        b=JPUbjHjI676QrkyVGzZUQysjpPAVXMLTOfPXT1IQVV/6ouqiSQPzXmz23ZHs9aN6Ax
         lBDz6mTLpvE/ArM7u4nvvkIE84j+zrxoEbFRtGP2Bz/pP4hfYmOwsp8hwYx2ucxqJDsm
         Naf6Sn8zlz5Nlt87Zq2ZnIvKHjbR0hw9FkowYg+yzu2j649GT6voDyxV1l1/piu7LiYL
         H0YYMFZDrcOSlR8JpiEzeknLtDqNbPVguZKS5IUinEDXTk1AlF0uvQTRQF5OmJ/TFQXk
         PU4WatdR2rFj6BXCj84hssyH2S3MsAOhzogPrOHS/acqM35haPfGAkVKXYCdPjIy2aK8
         VK9Q==
X-Forwarded-Encrypted: i=1; AJvYcCUaAJsrhyERK354rzSBAT4u7CYQw/Y7+9ye+/uUTanMEBrkoE3oQxXPzglUSXArmwl3p1LSd1OqpdlMMjhIbYbqA4vSpqyjtu3BU/de5q8=
X-Gm-Message-State: AOJu0YzkdAtiGiMoCHAoNpqMvf0+W55+o92pNg46jd+6+X+ryM1k8E76
	zcQxXrdKn4kBjrpGUrloA2HeoQmT5QBgCbdpH5p/GEv6mhmriFsK
X-Google-Smtp-Source: AGHT+IHRhf7lE++svNOn0+fduiF7pAWV6HXGKjlBhoIlSX0hrJ2KRNs7/+0B3OBn8vSZFiW+94heEw==
X-Received: by 2002:a17:906:cecc:b0:a3c:ae0d:793c with SMTP id si12-20020a170906cecc00b00a3cae0d793cmr1198445ejb.20.1707901950771;
        Wed, 14 Feb 2024 01:12:30 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCV7d9o0DQLmGf7CPUw0sOm84sXcnvRgP3t/5lWamMqHUMn3WOlaHaHXOwSikA1R1vufT5KsJrJ6tYWODlDjneieQWmw/qN8cGCrf/ipDD5t78AgjdUHMLtDv8CEJo8Wqxuinh5H2uLyeNLf6jap7EbFC+fS20Q8uX132YDJt1bjCtZ8lsnjDtVyEMeZA/mX5ifot2SvCH1FCMJnCOQ=
Message-ID: <3754521b357f4cd4d6636551f6438856a984acab.camel@gmail.com>
Subject: Re: [PATCH v8 5/7] xen/asm-generic: introduce generic device.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 10:12:29 +0100
In-Reply-To: <94a9353f-1dd0-4957-af2c-281f9b3bbe7c@suse.com>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
	 <4ecb5e5f4a1dbf585ac7a9ebe40b5fd8e4709182.1707499278.git.oleksii.kurochko@gmail.com>
	 <94a9353f-1dd0-4957-af2c-281f9b3bbe7c@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 15:19 +0100, Jan Beulich wrote:
> On 09.02.2024 19:00, Oleksii Kurochko wrote:
> > --- /dev/null
> > +++ b/xen/include/asm-generic/device.h
> > @@ -0,0 +1,149 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +#ifndef __ASM_GENERIC_DEVICE_H__
> > +#define __ASM_GENERIC_DEVICE_H__
> > +
> > +#include <xen/stdbool.h>
> > +
> > +enum device_type
> > +{
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +=C2=A0=C2=A0=C2=A0 DEV_DT,
> > +#endif
> > +=C2=A0=C2=A0=C2=A0 DEV_PCI
> > +};
> > +
> > +enum device_class
> > +{
> > +=C2=A0=C2=A0=C2=A0 DEVICE_SERIAL,
> > +=C2=A0=C2=A0=C2=A0 DEVICE_IOMMU,
> > +=C2=A0=C2=A0=C2=A0 DEVICE_INTERRUPT_CONTROLLER,
> > +=C2=A0=C2=A0=C2=A0 DEVICE_PCI_HOSTBRIDGE,
> > +=C2=A0=C2=A0=C2=A0 /* Use for error */
> > +=C2=A0=C2=A0=C2=A0 DEVICE_UNKNOWN,
> > +};
> > +
> > +struct dev_archdata {
> > +#ifdef CONFIG_HAS_PASSTHROUGH
> > +=C2=A0=C2=A0=C2=A0 void *iommu;=C2=A0=C2=A0=C2=A0 /* IOMMU private dat=
a */
> > +#endif
> > +};
> > +
> > +/* struct device - The basic device structure */
> > +struct device
> > +{
> > +=C2=A0=C2=A0=C2=A0 enum device_type type;
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +=C2=A0=C2=A0=C2=A0 struct dt_device_node *of_node; /* Used by drivers =
imported
> > from Linux */
> > +#endif
> > +=C2=A0=C2=A0=C2=A0 struct dev_archdata archdata;
> > +#ifdef CONFIG_HAS_PASSTHROUGH
> > +=C2=A0=C2=A0=C2=A0 struct iommu_fwspec *iommu_fwspec; /* per-device IO=
MMU
> > instance data */
> > +#endif
> > +};
> > +
> > +typedef struct device device_t;
> > +
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +
> > +#include <xen/device_tree.h>
> > +
> > +#define dev_is_dt(dev)=C2=A0 ((dev)->type =3D=3D DEV_DT)
> > +
> > +/**
> > + *=C2=A0 device_init - Initialize a device
> > + *=C2=A0 @dev: device to initialize
> > + *=C2=A0 @class: class of the device (serial, network...)
> > + *=C2=A0 @data: specific data for initializing the device
> > + *
> > + *=C2=A0 Return 0 on success.
> > + */
> > +int device_init(struct dt_device_node *dev, enum device_class
> > class,
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 const void *data);
> > +
> > +/**
> > + * device_get_type - Get the type of the device
> > + * @dev: device to match
> > + *
> > + * Return the device type on success or DEVICE_ANY on failure
> > + */
> > +enum device_class device_get_class(const struct dt_device_node
> > *dev);
> > +
> > +#define DT_DEVICE_START(name_, namestr_, class_)=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
>=20
> I don't think the trailing underscores are needed or helpful here
> (or in the ACPI counterpart), ...
>=20
> > +static const struct device_desc __dev_desc_##name_ __used=C2=A0=C2=A0 =
\
> > +__section(".dev.info") =3D {=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 .name =3D namestr_,=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 .class =3D class_,
>=20
> ... seeing this all it would have taken was to avoid the two words
> "name" and "class" (by e.g. using "ident" and "cls").
>=20
> Nevertheless:
> Acked-by: Jan Beulich <jbeulich@suse.com>
Thanks.

I'll apply your comments in the next patch version.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:15:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:15:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680287.1058329 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBMt-0002GW-Uc; Wed, 14 Feb 2024 09:15:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680287.1058329; Wed, 14 Feb 2024 09:15:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBMt-0002GP-S6; Wed, 14 Feb 2024 09:15:51 +0000
Received: by outflank-mailman (input) for mailman id 680287;
 Wed, 14 Feb 2024 09:15:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raBMt-0002GJ-7x
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:15:51 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a4a34282-cb19-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 10:15:49 +0100 (CET)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2d09cf00214so69162791fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:15:49 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 x2-20020a05620a14a200b00785d9e3b744sm2072612qkj.108.2024.02.14.01.15.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:15:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a4a34282-cb19-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707902149; x=1708506949; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=MtFyai8Qihp5DcPzX+dkaiqwqpJO2yVolS7zMjPGrvU=;
        b=V/585Iop6H77TGC9dbYcs3vp4RcSzlPb40E5HCvt71R28VhgMn6yrcAT/7JlvOJHcS
         ZDvrwGwszlAoQ3ysphgndBr9pmtMZ+eME7y0w7yK3SUF6dNoqaqUnhCp06xnYKDRFFhj
         eUn1PjWA+8OBZqpIiOivQyQQoOMqk+l8wH+Vk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707902149; x=1708506949;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MtFyai8Qihp5DcPzX+dkaiqwqpJO2yVolS7zMjPGrvU=;
        b=vQgp+5wzQvcWg7I8H0vMHWF11u8drwZqOLUU8ycM2G1nuQJyAKhsy+MUfWtHwXDtHf
         Ve4/L3p+0KVCIb/kSAp1WVc0mApUmu3bhRSVAPhk5hQ+0kK5SFDgUEPdk368FPP+N2ez
         sOCMMQOUk8KF/O+jn2dRmpjC0aZz4ztRLcZ1GSxeDT7fLME7j0gF8jcaYFhNOMB6Eev6
         /z7jClJpxhI4AA9D5HZspN4htAdhY+47Ns4tnNMF9oDkll6CvNxXqaGvoOwc+M3zHxs6
         P0Dm71iCGKt1E9mF+x5QF80vH/p73JHm96RkAjyoq10wAZgKD3t7BMnVA/XcK84CU0vN
         qVgA==
X-Forwarded-Encrypted: i=1; AJvYcCX6okKLIVmuAdghzzZCSX0zhMAFCU9xJVq4TtWsT5D0Q9qtNW4qUL1PbGgVi09sqPSb0/nu+HnXlfkDgh2B9ICWkxP4C9JEYQ3XPBVTpm0=
X-Gm-Message-State: AOJu0YwmogpnXQDrK47lwl29W92VGM/2xDIl4e/DAdolr7QZzDE5SvWx
	FQBwB5FchUA5QZ4FPdmKP1DcHsjkLV2zUREVw9Y3fgh7nBsSONqA0Mta/IrV0FrbJNb2OvWWWXx
	Y
X-Google-Smtp-Source: AGHT+IHSXKVIDkSsXyEIlRBTAaSPerLebQTsgWytleH0LMQ6e93O+91p1P7Z1s6AaaXuTRpwq0me6w==
X-Received: by 2002:a05:651c:626:b0:2d0:9210:fd57 with SMTP id k38-20020a05651c062600b002d09210fd57mr1220276lje.18.1707902148777;
        Wed, 14 Feb 2024 01:15:48 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVjitlyM0tWC7SMGh2/X9TO0Enpkp2cmEhuNWxHasZ2zImagTLqxjSgOC2uiLoiffg5G0AuJ6CzlJewGMMOnJDTjWE+CyBWSw6J0+2i17E9rrJAio0SMND3lU09Q8AwlLeAVNyC
Date: Wed, 14 Feb 2024 10:15:46 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
Subject: Re: [regression] Re: [PATCH v2 2/2] iommu/vt-d: switch to common
 RMRR checker
Message-ID: <ZcyEwjnu3igWl1xT@macbook>
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-3-roger.pau@citrix.com>
 <c27c76ec-36cd-43cb-b76f-e8f95fb27ed8@suse.com>
 <c4f27180-86bf-45fd-8641-bd160c6de229@citrix.com>
 <37ed1abe-afcc-4a76-8a86-623282ca37a3@suse.com>
 <Zcx9w5eZkXiN9s66@macbook>
 <790c81c8-04f4-435b-937f-87fa1cd54998@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <790c81c8-04f4-435b-937f-87fa1cd54998@suse.com>

On Wed, Feb 14, 2024 at 10:01:43AM +0100, Jan Beulich wrote:
> On 14.02.2024 09:45, Roger Pau Monné wrote:
> > On Wed, Feb 14, 2024 at 08:45:28AM +0100, Jan Beulich wrote:
> >> On 13.02.2024 23:37, Andrew Cooper wrote:
> >>> It's very likely something in this series, but the link to Intel might
> >>> just be chance of which hardware got selected, and I've got no clue why
> >>> there's a reset with no further logging out of Xen...
> >>
> >> I second this - even after looking closely at the patches again, I can't
> >> make a connection between them and the observed behavior. Didn't yet look
> >> at what, if anything, osstest may have to say. Do I understand correctly
> >> that the cited log messages are the last sign of life prior to the
> >> systems rebooting?
> > 
> > I've found it:
> > 
> >     for ( addr = start; mfn_x(addr) <= mfn_x(end); mfn_add(addr, 1) )
> > 
> > Should be:
> > 
> >     for ( addr = start; mfn_x(addr) <= mfn_x(end); addr = mfn_add(addr, 1) )
> > 
> > mfn_add() doesn't modify the parameter.  Will see about making those
> > helpers __must_check in order to avoid this happening in the future.
> 
> Hmm, yes, it's not the first time this has happened. But even seeing the
> flaw I still can't explain the observed behavior: The system ought to
> hang then, not instantly reboot?

AFAICT, it was stuck in a loop without making progress until the CI
controller decided to reboot it.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:24:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:24:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680294.1058340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBV4-0004St-Pf; Wed, 14 Feb 2024 09:24:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680294.1058340; Wed, 14 Feb 2024 09:24:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBV4-0004Sm-Lr; Wed, 14 Feb 2024 09:24:18 +0000
Received: by outflank-mailman (input) for mailman id 680294;
 Wed, 14 Feb 2024 09:24:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raBV3-0004Se-LN
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:24:17 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d30e0114-cb1a-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 10:24:16 +0100 (CET)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2d0e520362cso19236881fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:24:16 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 y20-20020a05620a44d400b00783f70bc497sm3629136qkp.115.2024.02.14.01.24.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:24:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d30e0114-cb1a-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707902656; x=1708507456; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=1Y++7tDJal4EDST6cp56TTjHOaOCdTKOVIVgLKMXSuY=;
        b=EK6/FvGifrH4Z+/itAC/pLEvUbZPtQIjgBd8tircgeFHXQi9C3KG4sG8pB3DIEe++F
         5YN/WktGsC1cnF9LYZDn7EbZfPnU3M5J52hbVmNZILHn+Y4gAEpoZ/D+5Ix+EVZvB9Dp
         L4/I5P7gRg/a+h9OI/+4TRBF3lkShzILMmERc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707902656; x=1708507456;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1Y++7tDJal4EDST6cp56TTjHOaOCdTKOVIVgLKMXSuY=;
        b=rB2yXeLRtrWCeCWB3oh2Wk1Q/RcJb58GDfNvaH3x3azvFb77h0QQBXru7q1LnzkW/M
         R+0NSNTFuDyyYKXqSGrFP7i9Rp/vKwlzKqTDT4YEvlPgFxWRJuDlcYEWLrYjXuDfHZhd
         F2+3fXR7nmT1Pxp3nB6RURSc7Hzg7w/RgKfUhZblmZlWQkHsa0/4hcooKolVf/dzYViS
         htZG0V5KhMkBA6tKKKIHUP4zvKwF5vhGjovHcraIvDmjDlDq/RgBhbWfbxIyLCQ/W9kN
         km4UVhmUqqIrAfhvWZww/Fxw6lZx+tx4yIhede1Fd901oVqOArWrgn+b1Ru9pxmEV6mL
         H6AA==
X-Forwarded-Encrypted: i=1; AJvYcCV1/yd252cC+WMHXQWrbGReRem559Z8fShYCg3bKlXFzI4R0rz4XKg3kNEm7VCQnr1dQc2IuZbVI2GscQcZY1wdfJeBUy1pLhdyZIhx/Ok=
X-Gm-Message-State: AOJu0YxGz3JhcX38m1hwsDnQbNp48FXQ40BfyesRMGD6fUHskCbOSETz
	mJrg7sWvziWGAymP8KLMgmIhulHt3fSvw2C+ql9Ms7QUqM8AslVCsVtLkIm++O4=
X-Google-Smtp-Source: AGHT+IH4RukR/OWcByYv3+3UWDmNJd5sUZtyUEuQ04CvQHsT/0RVvywXmlkH89OguYCuRFdKuZpHSw==
X-Received: by 2002:a2e:9f02:0:b0:2d0:84b1:bd3e with SMTP id u2-20020a2e9f02000000b002d084b1bd3emr1375628ljk.12.1707902656122;
        Wed, 14 Feb 2024 01:24:16 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXAu9ugG0QXE34tjUd9SThII5hnHJeI20JYgU9dZIwNuSXh+92YqBWleousBSKQW16cWJVtGxHK/gH7AxBATxmq82rgphCn/eALdiICSnz9BOM9Cq6U5pzZTOT52oJaJ5u3hpz4
Date: Wed, 14 Feb 2024 10:24:13 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>, Kevin Tian <kevin.tian@intel.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [regression] Re: [PATCH v2 2/2] iommu/vt-d: switch to common
 RMRR checker
Message-ID: <ZcyGvW-Q4MFvDTSu@macbook>
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-3-roger.pau@citrix.com>
 <c27c76ec-36cd-43cb-b76f-e8f95fb27ed8@suse.com>
 <c4f27180-86bf-45fd-8641-bd160c6de229@citrix.com>
 <37ed1abe-afcc-4a76-8a86-623282ca37a3@suse.com>
 <Zcx9w5eZkXiN9s66@macbook>
 <c84a640e-3332-4f2d-9c4a-c6806bb2da67@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <c84a640e-3332-4f2d-9c4a-c6806bb2da67@citrix.com>

On Wed, Feb 14, 2024 at 09:00:25AM +0000, Andrew Cooper wrote:
> On 14/02/2024 8:45 am, Roger Pau Monné wrote:
> > On Wed, Feb 14, 2024 at 08:45:28AM +0100, Jan Beulich wrote:
> >> On 13.02.2024 23:37, Andrew Cooper wrote:
> >>> On 12/02/2024 2:38 pm, Jan Beulich wrote:
> >>>> On 07.02.2024 16:34, Roger Pau Monne wrote:
> >>>>> Use the newly introduced generic unity map checker.
> >>>>>
> >>>>> Also drop the message recommending the usage of iommu_inclusive_mapping: the
> >>>>> ranges would end up being mapped anyway even if some of the checks above
> >>>>> failed, regardless of whether iommu_inclusive_mapping is set.  Plus such option
> >>>>> is not supported for PVH, and it's deprecated.
> >>>>>
> >>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> >>>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> >>> XenRT says no.
> >>>
> >>> It's not clear exactly what's going on here, but the latest resync with
> >>> staging (covering only today's pushed changes) suffered 4 failures to
> >>> boot, on a mix of Intel hardware (SNB, SKL, SKX and CLX).
> >>>
> >>> All 4 triple-fault-like things where following a log message about an RMRR:
> >>>
> >>> (XEN) RMRR: [0x0e8 ,0x0e8] is not (entirely) in reserved memory
> >>>
> >>> not being in reserved memory.
> >>>
> >>>
> >>> First of all - fix this printk() to print full addresses, not frame
> >>> numbers.  It's obnoxious to cross reference with the E820.
> >> Perhaps better indeed. The stray blank before the comma also wants dropping.
> >> And while looking over the patch again, "mfn_t addr;" also isn't very
> >> helpful - the variable would better be named mfn.
> > I can adjust those in the fix, see below.
> >
> >>> It's very likely something in this series, but the link to Intel might
> >>> just be chance of which hardware got selected, and I've got no clue why
> >>> there's a reset with no further logging out of Xen...
> >> I second this - even after looking closely at the patches again, I can't
> >> make a connection between them and the observed behavior. Didn't yet look
> >> at what, if anything, osstest may have to say. Do I understand correctly
> >> that the cited log messages are the last sign of life prior to the
> >> systems rebooting?
> > I've found it:
> >
> >     for ( addr = start; mfn_x(addr) <= mfn_x(end); mfn_add(addr, 1) )
> >
> > Should be:
> >
> >     for ( addr = start; mfn_x(addr) <= mfn_x(end); addr = mfn_add(addr, 1) )
> >
> > mfn_add() doesn't modify the parameter.  Will see about making those
> > helpers __must_check in order to avoid this happening in the future.
> 
> There's only a single thing in this function which wants an mfn_t. 
> Everything else is operating on raw paddr_t's.  I'd suggest converting
> types at the start and using plain numbers.

I don't have a strong opinion, can do (but then likely as a followup
patch).

> Also, while I hate to nitpick, iommu_unity_region_ok() really ought to
> be iommu_check_unity_region().  It is not a predicate (given the
> additional fixups), so the function name shouldn't read as one.

I'm afraid those two read the same to me.  I can change, but I don't
see how the additional fixups modify how the function should be
named.

> Also, the "not (entirely) in reserved memory" line ought to have an ";
> adjusting" on the end to make it clear that it's making an adjustment in
> light of finding the range not reserved.
> 
> Finally, the "can't be converted" error should render type, even if only
> in numeric form.

This was all inherited from the previous IVMD code.

> What do we do when there's a region that's marked as RAM?

We fail to initialize the IOMMU, which is what we did previously.  In
v1 of this series there was a further patch that would panic Xen if
such overlap was found.  That however raises the question if we need
to parse IVMD/RMRR regions even when the IOMMU is disabled, so that
the panic would also be triggered even when not using the IOMMU (as
the device will still be accessing the regions in the RMRR/IVMD
ranges).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:32:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:32:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680298.1058351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBcp-0006cd-IF; Wed, 14 Feb 2024 09:32:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680298.1058351; Wed, 14 Feb 2024 09:32:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBcp-0006cW-Dr; Wed, 14 Feb 2024 09:32:19 +0000
Received: by outflank-mailman (input) for mailman id 680298;
 Wed, 14 Feb 2024 09:32:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raBcn-0006cQ-Vl
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:32:17 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f0a8a5c1-cb1b-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 10:32:15 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-5638e05e27cso23574a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:32:15 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 r25-20020a170906351900b00a38a476fc08sm2130704eja.87.2024.02.14.01.32.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:32:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0a8a5c1-cb1b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707903135; x=1708507935; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Y6NEVowgluYgwOryGHAXwKJNPlcvZz1aYR6jpDCRVg4=;
        b=HcTc6mNhjl/CqRtHzS2yDxnccM5BNJr+pLWs3EemBQ1sJIoRoK83ZONjgO+Hy0FsAj
         GOFB4qt2+kmwS0hY+LAEholm8fSNy2uzrpXGfzJ9J1AgP9Q/D3hOm5myGvMpdyP9hDK+
         xcO7JeauriuipmUQqfz9+Y1s0M+qt2wZpd8RvFvDJoInuYhTiJ77rExCvpdouEnQUHhL
         XiAQBbv1I2m54g6V6SZ9ZDEDdJ60wtN7rE61BPNBB2EgQxwBYCpN6DIhbr2Jn6cV62/k
         JXSKIsly+JvucnB43kUjvbr7Ug1ImtmdtDcObYjf/laUnAU2oGanAabgue/IEaJUAp5f
         OO8w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707903135; x=1708507935;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Y6NEVowgluYgwOryGHAXwKJNPlcvZz1aYR6jpDCRVg4=;
        b=rs54Z4XnHEXs3OW1Au06ujf5FuEY0PiVuItwIHrBUoKymWzSydmC7VaXpCRASRLeyT
         lcNTN0k1/xvVst6loT835cjLP3/AiQ8O+ka5+3tIXgNDoHmgl2yUUCiMVG+s9vv3D4RG
         TxYkfGSkkGDB4Y2QSGlIS77BYIyo1kd8r1x2+CQW8DBXaxk0KhkDh+3nxwjBFQFi5NUO
         O3ZUivxRCfStmz4+3ZpFzbIUh2do3Yr6ntCp+vt6r5drbZFB1gH1JpCE5VlCmvpD9bOX
         korPu3deS/q8SZGanz21IOOKsfxeozfp9UfdMQIJCuyQEMYubLUa8qwXhmBLJrDkyBaD
         6mNw==
X-Forwarded-Encrypted: i=1; AJvYcCVbHzwHZvZYtQ2UbzrdVA2RS+XQiEiQZZvbgj5RXT2VgNJuvHqsDjTOUoqb6QnofXNY+feuDHwNeP00S/kNTy7rjjBdX+CbCgNLhyLHXEc=
X-Gm-Message-State: AOJu0YyPhGFjD49G48V/qycATD1/iD2XCBCmmRWAiRTyUfG6mjSrwcDw
	QlCbG5cc7BkCA8YA+S+4JQbzfpBniMmUktlkcIV4QJpLWMiMw+26
X-Google-Smtp-Source: AGHT+IGJYReGNpPG0mSW4iRTIJMjms9h4jQUDHM3HfL0poUHEHeItnjIBWPBMcrt8v75D6Pnfl5ndA==
X-Received: by 2002:a17:906:b7c4:b0:a3d:14e0:9608 with SMTP id fy4-20020a170906b7c400b00a3d14e09608mr1359456ejb.39.1707903135075;
        Wed, 14 Feb 2024 01:32:15 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXRU981lFmJ3PfeKhhHJpKDoQCRIlOzXTsuaxadu7o25wtgrt3q40wdKbvCHx2GtELYIks8t5EbSvWt3tAgm12+qHVShbx7PKR2TUZCQ9LpsIq9GUsGqoeRYG4v92ZWw5EEvEDCvCi/ljJFRIqlL3ilBRM6I4fstWBRp8w9dF3w9AcPuFs0Tlhu+BSeBq3ATrhyez22k+0LXpegtq2eZuA=
Message-ID: <9d4f3093de6f95b09e44bc66fe9a9d4b0009dc43.camel@gmail.com>
Subject: Re: [PATCH v8 5/7] xen/asm-generic: introduce generic device.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Date: Wed, 14 Feb 2024 10:32:13 +0100
In-Reply-To: <319dcf7a-88cd-488e-ae51-6c20c19d8f22@xen.org>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
	 <4ecb5e5f4a1dbf585ac7a9ebe40b5fd8e4709182.1707499278.git.oleksii.kurochko@gmail.com>
	 <319dcf7a-88cd-488e-ae51-6c20c19d8f22@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Tue, 2024-02-13 at 18:09 +0000, Julien Grall wrote:
> Hi Oleksii,
>=20
> On 09/02/2024 18:00, Oleksii Kurochko wrote:
> > diff --git a/xen/include/asm-generic/device.h b/xen/include/asm-
> > generic/device.h
> > new file mode 100644
> > index 0000000000..6e56658271
> > --- /dev/null
> > +++ b/xen/include/asm-generic/device.h
> > @@ -0,0 +1,149 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +#ifndef __ASM_GENERIC_DEVICE_H__
> > +#define __ASM_GENERIC_DEVICE_H__
> > +
> > +#include <xen/stdbool.h>
> > +
> > +enum device_type
> > +{
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +=C2=A0=C2=A0=C2=A0 DEV_DT,
> > +#endif
> > +=C2=A0=C2=A0=C2=A0 DEV_PCI
> > +};
> > +
> > +enum device_class
> > +{
> > +=C2=A0=C2=A0=C2=A0 DEVICE_SERIAL,
> > +=C2=A0=C2=A0=C2=A0 DEVICE_IOMMU,
> > +=C2=A0=C2=A0=C2=A0 DEVICE_INTERRUPT_CONTROLLER,
> > +=C2=A0=C2=A0=C2=A0 DEVICE_PCI_HOSTBRIDGE,
> > +=C2=A0=C2=A0=C2=A0 /* Use for error */
> > +=C2=A0=C2=A0=C2=A0 DEVICE_UNKNOWN,
> > +};
> > +
> > +struct dev_archdata {
> > +#ifdef CONFIG_HAS_PASSTHROUGH
> > +=C2=A0=C2=A0=C2=A0 void *iommu;=C2=A0=C2=A0=C2=A0 /* IOMMU private dat=
a */
> > +#endif > +};
>=20
> It is a bit too late to change, but I thought I would point it if=20
> someone wants to send a follow-up. It is a bit odd to have a
> structure=20
> dev_archdata that, if I am not mistaken, is only used ...
>=20
> > +
> > +/* struct device - The basic device structure */
> > +struct device
> > +{
> > +=C2=A0=C2=A0=C2=A0 enum device_type type;
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +=C2=A0=C2=A0=C2=A0 struct dt_device_node *of_node; /* Used by drivers =
imported
> > from Linux */
> > +#endif
> > +=C2=A0=C2=A0=C2=A0 struct dev_archdata archdata;
>=20
> ... in struct device. Looking at the use, I believe this was only=20
> introduced to try to keep that SMMU code close to Linux. I would=20
> consider to fold the other structure and update dev_archdata() in=20
> drivers/passthrough/arm/smmu.c.
I can do that in separate patch. struct dev_archdata was left because
of drivers/passthrough/arm/smmu.c.

>=20
> > +#ifdef CONFIG_HAS_PASSTHROUGH
> > +=C2=A0=C2=A0=C2=A0 struct iommu_fwspec *iommu_fwspec; /* per-device IO=
MMU
> > instance data */
> > +#endif
> > +};
> > +
> > +typedef struct device device_t;
> > +
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +
> > +#include <xen/device_tree.h>
> > +
> > +#define dev_is_dt(dev)=C2=A0 ((dev)->type =3D=3D DEV_DT)
> > +
> > +/**
> > + *=C2=A0 device_init - Initialize a device
> > + *=C2=A0 @dev: device to initialize
> > + *=C2=A0 @class: class of the device (serial, network...)
> > + *=C2=A0 @data: specific data for initializing the device
> > + *
> > + *=C2=A0 Return 0 on success.
> > + */
> > +int device_init(struct dt_device_node *dev, enum device_class
> > class,
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 const void *data);
> > +
> > +/**
> > + * device_get_type - Get the type of the device
> > + * @dev: device to match
> > + *
> > + * Return the device type on success or DEVICE_ANY on failure
> > + */
> > +enum device_class device_get_class(const struct dt_device_node
> > *dev);
> > +
> > +#define DT_DEVICE_START(name_, namestr_, class_)=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +static const struct device_desc __dev_desc_##name_ __used=C2=A0=C2=A0 =
\
> > +__section(".dev.info") =3D {=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 .name =3D namestr_,=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 .class =3D class_,
> > +
> > +#define DT_DEVICE_END=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +};
> > +
> > +#else /* !CONFIG_HAS_DEVICE_TREE */
> > +#define dev_is_dt(dev) ((void)(dev), false)
> > +#endif /* CONFIG_HAS_DEVICE_TREE */
> > +
> > +#define dev_is_pci(dev) ((dev)->type =3D=3D DEV_PCI)
> > +
> > +struct device_desc {
> > +=C2=A0=C2=A0=C2=A0 /* Device name */
> > +=C2=A0=C2=A0=C2=A0 const char *name;
> > +=C2=A0=C2=A0=C2=A0 /* Device class */
> > +=C2=A0=C2=A0=C2=A0 enum device_class class;
> > +
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +
> > +=C2=A0=C2=A0=C2=A0 /* List of devices supported by this driver */
> > +=C2=A0=C2=A0=C2=A0 const struct dt_device_match *dt_match;
> > +=C2=A0=C2=A0=C2=A0 /*
> > +=C2=A0=C2=A0=C2=A0=C2=A0 * Device initialization.
> > +=C2=A0=C2=A0=C2=A0=C2=A0 *
> > +=C2=A0=C2=A0=C2=A0=C2=A0 * -EAGAIN is used to indicate that device pro=
bing is
> > deferred.
> > +=C2=A0=C2=A0=C2=A0=C2=A0 */
> > +=C2=A0=C2=A0=C2=A0 int (*init)(struct dt_device_node *dev, const void =
*data);
> > +
> > +#endif
> > +};
> I am not sure I fully understand why "device_desc" is not protected
> by=20
> CONFIG_HAS_DEVICE_TREE. The structure doesn't mean much when the
> config=20
> is disabled. Can you clarify?
I thought that one day struct device_desc and acpi_device_desc will be
"merged", and so decided just to #ifdef only DEVICE_TREE specific
fields.
Another one reason it is if to protect fully struct device_desc then it
would be needed more #ifdef in arm/device.c ( for example,
device_init() should be all protected then ) what will require to ifdef
all calls of device_init(). As an option device_init can can be defined
in case when !CONFIG_HAS_DEVICE_TREE as:
   int __init device_init(struct dt_device_node *dev, enum device_class
   class,
                          const void *data)
   {
    return -EBADF;
   }
  =20
The similar thing will be needed for device_get_class() in Arm's device.c.

Would it be better to ifdef full struct device_desc ?

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:33:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:33:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680300.1058360 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBe1-00079R-Pu; Wed, 14 Feb 2024 09:33:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680300.1058360; Wed, 14 Feb 2024 09:33:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBe1-00079K-NO; Wed, 14 Feb 2024 09:33:33 +0000
Received: by outflank-mailman (input) for mailman id 680300;
 Wed, 14 Feb 2024 09:33:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raBdz-00078o-L5; Wed, 14 Feb 2024 09:33:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raBdz-0007Is-Hk; Wed, 14 Feb 2024 09:33:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raBdz-0001FF-50; Wed, 14 Feb 2024 09:33:31 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raBdz-0001ua-4F; Wed, 14 Feb 2024 09:33:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3ThWo06mZ8aQGQUClVzmM27Rjkvtd/bmyhdZf2/5ORk=; b=JAkIJGkTsJ1mCCOFz7PR7U8GdR
	ZjAEjx+pNZ1xZTj1UH5GiCzB1s6P5BCVhIvHtIFPkc9SSIso1Y4/FXDI+zZ9EgGXw4fSj0n/9FuMj
	dIlXmyG9MrnGdwEBCSxcqYrc8KBsRWbsAEcJIcT6o59QOPKgntVfj6eFJrvv8L0jUgLw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184662-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184662: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=40ebade891a0d2c08e82dac126822d5ee57f4281
X-Osstest-Versions-That:
    libvirt=94365a48710227a5fec836baadd12d8ff5a5fe8a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 14 Feb 2024 09:33:31 +0000

flight 184662 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184662/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184655
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184655
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184655
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              40ebade891a0d2c08e82dac126822d5ee57f4281
baseline version:
 libvirt              94365a48710227a5fec836baadd12d8ff5a5fe8a

Last test of basis   184655  2024-02-13 04:20:35 Z    1 days
Testing same since   184662  2024-02-14 04:20:33 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jiri Denemark <jdenemar@redhat.com>
  Michal Privoznik <mprivozn@redhat.com>
  Peter Krempa <pkrempa@redhat.com>
  Timothée Ravier <tim@siosm.fr>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   94365a4871..40ebade891  40ebade891a0d2c08e82dac126822d5ee57f4281 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:34:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:34:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680306.1058370 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBev-0007iU-6Z; Wed, 14 Feb 2024 09:34:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680306.1058370; Wed, 14 Feb 2024 09:34:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBev-0007iN-3f; Wed, 14 Feb 2024 09:34:29 +0000
Received: by outflank-mailman (input) for mailman id 680306;
 Wed, 14 Feb 2024 09:34:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raBet-0007fU-M5
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:34:27 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3e496152-cb1c-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 10:34:26 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-5638dbf1417so43352a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:34:26 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 j2-20020aa7de82000000b005621b45daffsm651262edv.28.2024.02.14.01.34.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:34:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3e496152-cb1c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707903265; x=1708508065; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=hP4kZLnN1WW8YJLFPK9+BmBruAYbYOdwq2j5yxpv070=;
        b=a3typs2NwxCM4y+xt0TNH5sBEE0sqCWI0wxru9dAqpmGJVIaC7vOi6Bn5GAtkpXim6
         /TD4LL6b+pnla4sM/+iHPMYogGWVvJ2JPFvuAJEaWZ//5oaAcHeYHE2IkkJDaPNHNRbU
         XADkqqa06KjkB52wczFGg2N3RHbwgE/5hAyJpcnLib1QljvMa4EGNJW5Ndyfu+pyfhTp
         x1ygGDB516LVexd91VluVobKf4bc/MMixdC1PRVzO7ytoklpJ9opPx5gbUhX237RFcsU
         vlFaiGY4XzTJPVjC80igexwRkcgaXvdEnHvfVwOA9HIHAbyiJK/mER9JnmQnK6jBz2oo
         3rgQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707903265; x=1708508065;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=hP4kZLnN1WW8YJLFPK9+BmBruAYbYOdwq2j5yxpv070=;
        b=L5karuSn3nrQz86Rs3Ff4cMyxq6B3KNrIhEaxWT83RPnv2VLtysG3WQ6B4gTmNy3Sv
         EzMpJi/9B5bHI4HNYRe0AN0owJBmsLzfv+09aLW1G3gwmd2i9B/7TYcKungubpsplbgL
         kabY+UdgkF7r7FnlyRIF8AcprPvO+4WGRty30oJCPKrBZrsDs4P07Eo3RlcYWbkZAnZU
         mYTJKylnEhzKfcfHMEMpT3NOjSEo1UPam0lwZqgyPMXxLKGoYjfXOe7BWIF95hu6+sDg
         oGCAr+2kAW+TFsVM/HqicpDLFVZZ/ZCv6hFeKYLYgfDZXJyT/1OBdL+M59cqa6By0NGd
         v5Ww==
X-Forwarded-Encrypted: i=1; AJvYcCU1SBKttrOlV8p95t0ta6Lzg9SXqUyPzjyHkMCerWdi1Q0un5V5HvtwZjCkNgnU89Hu/5G5cVXzKexU0rs+0NELnzwqa57EKaaS5PamN+w=
X-Gm-Message-State: AOJu0YyLrDNO5Zj/ocDjrW/DmUeXDVQpmqCIaqm/WtMyLx2iWbifuu+T
	kCn7MmjetVYXqAbEUlmtPcsmPBOSwQLVL6vqrEB/xP0MngBvhtZkRDtrLoqK
X-Google-Smtp-Source: AGHT+IEIlYGC6H37Zpz4V7TnbqcQepqhVKppa5hQl8CnG6YSmb/T2tgIUGKG3uV/EsJndymRRQ+oUg==
X-Received: by 2002:aa7:d1cb:0:b0:55f:99:c895 with SMTP id g11-20020aa7d1cb000000b0055f0099c895mr1574912edp.20.1707903265455;
        Wed, 14 Feb 2024 01:34:25 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUGZGyJOAetSdRCzRT8wk7AEWUt1WwA9F6F0d6JO62L8G+SVP5gz5fd9xAH6hnuyx31zMb7nLB86lhZLtdmpjAFjGyQS5XCrb0/xgbT8XMSBPL/P+2QVSFnTbIdlvtLDzIPDQSY0x8ykf5T6py311EfzO+VDmgDkuDfn01IXe+t7feuPJy9ifcN4eyISs5BErRtfbng//hR+ssp
Message-ID: <f76b81ca2e4c057e266597de2a00179c039a20d9.camel@gmail.com>
Subject: Re: [PATCH v8 6/7] xen/arm: switch Arm to use asm-generic/device.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
	Babchuk <Volodymyr_Babchuk@epam.com>
Date: Wed, 14 Feb 2024 10:34:24 +0100
In-Reply-To: <cd2091d2-ac71-4d6f-a9a2-e7444889af69@xen.org>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
	 <ae256e804cbfd3c5d17ef9d17f100ebbf17c48fc.1707499278.git.oleksii.kurochko@gmail.com>
	 <cd2091d2-ac71-4d6f-a9a2-e7444889af69@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

Hi Julien,

On Tue, 2024-02-13 at 18:12 +0000, Julien Grall wrote:
> Hi Oleksii,
>=20
> On 09/02/2024 18:00, Oleksii Kurochko wrote:
> > The following changes were done as a result of switching to
> > asm-generic/device.h:
> > =C2=A0 * DEVICE_GIC was renamed to DEVICE_INTERRUPT_CONTROLLER accordin=
g
> > =C2=A0=C2=A0=C2=A0 to definition of enum device_class in asm-generic/de=
vice.h.
> > =C2=A0 * acpi-related things in Arm code were guarded by #ifdef
> > CONFIG_ACPI
> > =C2=A0=C2=A0=C2=A0 as struct acpi_device_desc was guarded in asm-generi=
c, also
> > functions
> > =C2=A0=C2=A0=C2=A0 acpi_device_init() was guarded too as they are using=
 structure
> > =C2=A0=C2=A0=C2=A0 acpi_device_desc inside.
> > =C2=A0 * drop arm/include/asm/device.h and update
> > arm/include/asm/Makefile
> > =C2=A0=C2=A0=C2=A0 to use asm-generic/device.h instead.
> >=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > Changes in V8:
> > =C2=A0 - update the commit message
> > ---
> > Changes in V7:
> > =C2=A0 - newly introduced patch which is based on the previous version
> > of the patch:
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [PATCH v6 9/9] xen/asm-generic: introduc=
e generic device.h
> > ---
> > =C2=A0 xen/arch/arm/device.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 5 ++
> > =C2=A0 xen/arch/arm/domain_build.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
|=C2=A0=C2=A0 2 +-
> > =C2=A0 xen/arch/arm/gic-v2.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 4 +-
> > =C2=A0 xen/arch/arm/gic-v3.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 6 +-
> > =C2=A0 xen/arch/arm/gic.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 4 +-
> > =C2=A0 xen/arch/arm/include/asm/Makefile |=C2=A0=C2=A0 1 +
> > =C2=A0 xen/arch/arm/include/asm/device.h | 124 ------------------------=
-
> > -----
> > =C2=A0 7 files changed, 14 insertions(+), 132 deletions(-)
> > =C2=A0 delete mode 100644 xen/arch/arm/include/asm/device.h
> >=20
> > diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
> > index 1f631d3274..3e02cff008 100644
> > --- a/xen/arch/arm/device.c
> > +++ b/xen/arch/arm/device.c
> > @@ -16,7 +16,10 @@
> > =C2=A0 #include <xen/lib.h>
> > =C2=A0=20
> > =C2=A0 extern const struct device_desc _sdevice[], _edevice[];
> > +
> > +#ifdef CONFIG_ACPI
> > =C2=A0 extern const struct acpi_device_desc _asdevice[], _aedevice[];
> > +#endif
>=20
> Can you also update the linker script to protect the following code?
Sure, I'll update that. Also please look at my comment to the previous
patch. Perhaps, it will be needed to update this patch to ifdef
device_init() and device_get_class(), and provide stubs for them in
case of !CONFIG_HAS_DEVICE_TREE.


> I.e
>=20
> #ifdef CONFIG_ACPI
> =C2=A0=C2=A0 . =3D ALIGN(8);
> =C2=A0=C2=A0 .adev.info : {
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 _asdevice =3D .;
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 *(.adev.info)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 _aedevice =3D .;
> =C2=A0=C2=A0 } :text
> #endif
>=20
> With this change:
>=20
> Reviewed-by: Julien Grall <jgrall@amazon.com>
Thanks.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:35:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:35:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680309.1058379 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBgG-0008Gg-Ft; Wed, 14 Feb 2024 09:35:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680309.1058379; Wed, 14 Feb 2024 09:35:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBgG-0008GZ-Cj; Wed, 14 Feb 2024 09:35:52 +0000
Received: by outflank-mailman (input) for mailman id 680309;
 Wed, 14 Feb 2024 09:35:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raBgF-0008GT-BC
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:35:51 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6fe8bea1-cb1c-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 10:35:49 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a3cede53710so201321166b.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:35:49 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 jw18-20020a170906e95200b00a3d606ad367sm84807ejb.38.2024.02.14.01.35.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:35:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fe8bea1-cb1c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707903349; x=1708508149; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Jm4piMvAEf9h/Ke1toGz6mjDyUEI8hyJyycjFH3AcA8=;
        b=hwy02Xb5MIPRCO/SgBHuM8RcuVDh3TWRM6q2UHQY1Z3WjkxvAAY6s4zjVi5xjqQ/ci
         0Uq2rgl6QJE1Fn2tqHIglE4ywYOphBcFO4/vnkwqUDGLopqaLaRPlpSLBJt+FhBaRFeD
         x7/sJHM1cBsGc9BTFZwz0fwnLXDZbSjPUN0z+kJY2OsilZg9SaT7vn5hq5NoZuYQlRTF
         2ndBHctLBfGdsTc/9Kly/1R5plo6bG0rI9lvbT0gUINb8JazwOOZYJF7nRSMCF9CMX/s
         Q1XMdKqVHeW1YT2CQNX5qmmFGO5Lds+T8QfFmoPcQWj9mFOQQOSpYXFFpc4L7qPDW9rQ
         YbRA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707903349; x=1708508149;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Jm4piMvAEf9h/Ke1toGz6mjDyUEI8hyJyycjFH3AcA8=;
        b=lD5THyaIT0vkccL89qELpRY8aGuCyVp8/WFKzoX5eBCsk4lq/ezf48UvzgeKiO5ITw
         wfmYbKz5l7Vbvlr06HWvxYBOOgXe3YtMdKVH/iHgij0BMMehLtxFzpNZF2u1lk8hrnZj
         Oj6kNkI0kJmPTk97PGV/HKPmtFMFF2ISn1eeKCWmCUeeCEsJjwVELvtvSVtI9dDa/P/c
         Y4nMtZufJkoZx6LBBGVJYwZqZROV/dpEj+ivRWWXSVO9oN/prFYBtjI8DJFT1HirDPor
         jh27aqdW36Z4qx8jE9I4nrWSEeocPOSi6u/3agUzVKvbP57vkBENHu5zEq1Q1RvL1aEd
         awHw==
X-Forwarded-Encrypted: i=1; AJvYcCXI7VFS+j5JGRcUDUeiwv3rDfGTc1ZPWg7aXdsS7LxSW1KxsJNGIufKZOIi/quFhp/nNk70YFF7s52aErppI+aqSw5h782ltmuS2U7xGyM=
X-Gm-Message-State: AOJu0Yz7LGsIvQusG1havwqxTReCdJho156yR2OkJXtt3iKRn2FHOkDu
	vR9CzTEwuIRL3JV2CgJc3vqKLqejdcDl1aj9ujJTWLvtLWNMu4JN
X-Google-Smtp-Source: AGHT+IGkgu301vkQpKrx9qs6p7JGMW0OKBRVP+Cbe3+5a/ekBvf6YqTf1Pr4DlpXqMtIUYeyrYAfEQ==
X-Received: by 2002:a17:906:48e:b0:a3c:cd89:edb2 with SMTP id f14-20020a170906048e00b00a3ccd89edb2mr1571243eja.22.1707903348684;
        Wed, 14 Feb 2024 01:35:48 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUFM0XDq6DySXxtBtLMUU1ynIy5FiLs2U7AxeH2JLG01oTja95kZlisaQYITSzE0XvSi3RCe/nm3mgCLjretUYfwzag5ks/LfyZwhdf24Z7K52CFTz7Ic9DWE54yE92sC2yOvGzl2wMwE/x1SEjOSg708CF5Pkykk6/Azqx1OU09JgVYkJObqQLueNJOfhVww0OfKhSJYM7YlYjEJFpqm17yJw5PcNAWcZRV8GQGrDTZ7vIXUu/XYai6KcLD2DFT6m7UOMu6bgPNGs7P6ddUSXCK0vkeMaaesPXaw6Yv380JDXflcjdhjMIEyDZS2+KVNWUf03yE1ipxy2nnBWLLflwqcuQOc9f1wb0OQ==
Message-ID: <656dfb55b67e78061e83ca5863b515fa9b6de24f.camel@gmail.com>
Subject: Re: [PATCH v2] xen/lib: introduce generic find next bit operations
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>,  xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 10:35:47 +0100
In-Reply-To: <f8534623-39bf-4c4a-9236-ed5bd3d655c2@suse.com>
References: 
	<fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
	 <f8534623-39bf-4c4a-9236-ed5bd3d655c2@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 15:15 +0100, Jan Beulich wrote:
> On 09.02.2024 18:58, Oleksii Kurochko wrote:
> > find-next-bit.c is common for Arm64, PPC and RISCV64,
> > so it is moved to xen/lib.
> >=20
> > PPC has been transitioned to generic functions from find-next-bit.c
> > since it now shares the same implementation as the PPC-specific
> > code.
> >=20
> > The MISRA exclude list has been updated to verify
> > lib/find-next-bit.c instead of Arm's find_next_bit.c,
> > as Arm's find_next_bit.c has been relocated to xen/lib/.
> >=20
> > Despite CONFIG_GENERIC_FIND_FIRST_BIT not currently being used in
> > Xen and being removed from the Linux kernel [1], it could
> > theoretically
> > prove useful for Xen. This is because the Linux kernel transitioned
> > Arm to the generic version of find_first_bit() and
> > find_first_zero_bit() due to improvements [1] in both performance
> > and .text size.
> > It would be prudent to investigate whether [1] is applicable to Xen
> > as well and, if so, implement the necessary changes in a separate
> > patch.
> >=20
> > [1]
> > https://lore.kernel.org/linux-arch/20211005054059.475634-5-yury.norov@g=
mail.com/
> >=20
> > Suggested-by: Jan Beulich <jbeulich@suse.com>
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Thanks!

>=20
> A revision log would have been nice, though.
Missed that. In case if it will be needed a new patch version, I'll
back the log.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:36:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:36:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680311.1058390 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBgr-0000Kn-Nb; Wed, 14 Feb 2024 09:36:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680311.1058390; Wed, 14 Feb 2024 09:36:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBgr-0000Kg-Kh; Wed, 14 Feb 2024 09:36:29 +0000
Received: by outflank-mailman (input) for mailman id 680311;
 Wed, 14 Feb 2024 09:36:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raBgq-0000KY-M7
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:36:28 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8636fc87-cb1c-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 10:36:26 +0100 (CET)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-5638c4a8d6eso118340a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:36:26 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 a22-20020aa7d756000000b0056140d51b7asm4506800eds.83.2024.02.14.01.36.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:36:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8636fc87-cb1c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707903386; x=1708508186; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=G+McCBgeiVepxG0NOfFdl2eJgbLlMCg4Y4XQAubfVVc=;
        b=AdHfj2IK2mlixAXrW4YOobfuOaH/87jIHdM4Aum0Is+fXeoiHdWS4+xp7wu0KWIsQx
         EdBHoU1qLaheLGP5AiDUE4Wj4b4Jx/OHM40lAZFhFHzKinzai2/BPMTuuqvKyIq2HV8y
         xtRcj0my8WARrFzF10BLL02tTZ0cqIunyMZDJGEX8A15tdHNLuNawUdlTmWa2MxITBJh
         /DVfg97pn/mN24M7IM7RIcxvVQ88ReB4jD3K/6yMI+riebffTmWMJXODhQPgLXZBRTIJ
         h8aXdrReZnzJAx7HtH2eFZXXcIqEqeTM9SNKqVvgIC8RoY7+2/2gBPdjfSE3nQHHdAwr
         pWPw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707903386; x=1708508186;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=G+McCBgeiVepxG0NOfFdl2eJgbLlMCg4Y4XQAubfVVc=;
        b=B1ceaIX8R40udD+L8k9RdiUa0uJ8RtTTQvJlUTI9KhNGf/i5031XJs/G2k3tEY9ERg
         d9sC2jZeHnkPJQ1BzbWQsyUSLBpamQq9yc3Ec4XWVWT39NKJqEWULvfqKSOs8e4VUnMT
         0wKSScXfhUufYI5NLWLJKvKXy6Rp4ZPSgdibJRaaFLwNzgHj6HJhYgKm1a3oyKn2KqwK
         YrbRI5+SJWgfJqE90f/Xg3+mfrwL29ygxHM8FzaScJ6DJLf1NFUHzfAfA36gjV4Crd3p
         Spg+DgvaneKxtbDJ396iEhnRBBnJSerli2Qlmpl2PY5iUhoUZh8B6Tk45FBzX6riBiwO
         bG0g==
X-Forwarded-Encrypted: i=1; AJvYcCWZuIhtbgc4WPaX83aTBZ8FvYxVsepDJ3UtJx+GoWqZ/BaNmdzCDHYqVa6yu3t+/8XKUWD7SyFOE5gJ+cQLuI6Rh8/LN7dJEW0gHPWluy4=
X-Gm-Message-State: AOJu0Yy0RwAMaqYFhSR1d0SmSZ6ojuATUN3G3w212vsoeGuR/EEXtwgi
	mRWroJS5HmDylcWnLxPjed9Hm/8W55VET8noPTK1Ai2pUfKxMCdK
X-Google-Smtp-Source: AGHT+IHm3+UnxFr/XtmNY2Vh2ClOPwNoU5a4saYaiCzlzcoiReXd+E6ytl0Y6xn9BGpcn3FNwmIJGw==
X-Received: by 2002:aa7:cf11:0:b0:561:c157:8074 with SMTP id a17-20020aa7cf11000000b00561c1578074mr1564869edy.0.1707903385298;
        Wed, 14 Feb 2024 01:36:25 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUzCI9ocNyODcwb6gjPujWe1cnr5W9naUiY54RmP15a3iIOCg7dBspAo5qtfxXVyXpwt5sssK6vJM/vspZQ6wyzz+zWg+kNDnDN4HAjHPtWjl8H+7FgB0svm0Ybb62iBe6Uq19qnAXQKOlqbKmREYa1szcaO+6PeknOKi/RnOKdQPvgFiwsSnCPYliNLKDdg0xWqlerRvPFAlQ79HK6aipFsmEOygQI+DZKJjyKXwZndk+rJOXhzegGGLm84qOXUt7oNbcWYJQiNKe6V2kM6bcPA6SZWtbVeLLHw6oWZk6Xwa9ZdzMzL3hqy1e2gdU4q/w9CQE87Rg7GDz/gPQqaxw6UnCdfS1bDVYZnXZFtQ==
Message-ID: <e4c9c8133a993b3ca3beeae027ffc393457e7632.camel@gmail.com>
Subject: Re: [PATCH v2] xen/lib: introduce generic find next bit operations
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Shawn Anastasio
 <sanastasio@raptorengineering.com>
Date: Wed, 14 Feb 2024 10:36:24 +0100
In-Reply-To: <01167c19-d1ff-41ea-8852-3eef307f2156@xen.org>
References: 
	<fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
	 <01167c19-d1ff-41ea-8852-3eef307f2156@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

Hi Julien,

On Tue, 2024-02-13 at 17:18 +0000, Julien Grall wrote:
> Hi Oleksii,
>=20
> On 09/02/2024 17:58, Oleksii Kurochko wrote:
> > find-next-bit.c is common for Arm64, PPC and RISCV64,
> > so it is moved to xen/lib.
> >=20
> > PPC has been transitioned to generic functions from find-next-bit.c
> > since it now shares the same implementation as the PPC-specific
> > code.
> >=20
> > The MISRA exclude list has been updated to verify
> > lib/find-next-bit.c instead of Arm's find_next_bit.c,
> > as Arm's find_next_bit.c has been relocated to xen/lib/.
> >=20
> > Despite CONFIG_GENERIC_FIND_FIRST_BIT not currently being used in
> > Xen and being removed from the Linux kernel [1], it could
> > theoretically
> > prove useful for Xen. This is because the Linux kernel transitioned
> > Arm to the generic version of find_first_bit() and
> > find_first_zero_bit() due to improvements [1] in both performance
> > and .text size.
> > It would be prudent to investigate whether [1] is applicable to Xen
> > as well and, if so, implement the necessary changes in a separate
> > patch.
> >=20
> > [1]
> > https://lore.kernel.org/linux-arch/20211005054059.475634-5-yury.norov@g=
mail.com/
> >=20
> > Suggested-by: Jan Beulich <jbeulich@suse.com>
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Reviewed-by: Julien Grall <jgrall@amazon.com>
Thanks for review.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:48:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:48:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680321.1058400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBs7-0002na-M5; Wed, 14 Feb 2024 09:48:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680321.1058400; Wed, 14 Feb 2024 09:48:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBs7-0002nT-Iy; Wed, 14 Feb 2024 09:48:07 +0000
Received: by outflank-mailman (input) for mailman id 680321;
 Wed, 14 Feb 2024 09:48:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=luzD=JX=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1raBs6-0002nN-2w
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:48:06 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 266a492b-cb1e-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 10:48:05 +0100 (CET)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-5638c4a8d6eso136572a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:48:05 -0800 (PST)
Received: from localhost.localdomain (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 en14-20020a056402528e00b0055fef53460bsm4533356edb.0.2024.02.14.01.48.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:48:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 266a492b-cb1e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707904084; x=1708508884; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Ig7a6sezO7NNmP42wrTc3CZvvVY9+4eNvCDfnEN1+HM=;
        b=OCblBN70QJfBS2nmi+H+VAfed9opb11UrCAe8fmJn2JFUKtnzo3I81EQb5aRrJlCcP
         egIeorqyPaa4ggBE5T0ntKM3ICJTUCt+e1wt9YR357t3fYrLuaq4UU95MLzHDhTy849R
         ru4fCYaKKBbvCv8PShix1+ATBB8pK4obCf7xU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707904084; x=1708508884;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Ig7a6sezO7NNmP42wrTc3CZvvVY9+4eNvCDfnEN1+HM=;
        b=WJ30B4ui+6SemLmAR6PSxRuiCfStPxesMmSGw9D9JLFDfwe1pX2ZjverlIp1LKfiL0
         pSsVgGxWvGbSDWuliySphhie0eMjmRKFO9kgDd1hJ0f3tTZim8Q1nsr5DxqNt8yzWXo3
         oW0jHPk7woQgT715+1GHdKzKwh34F5H7ONQomUQPoiwzRUH2RtXoYxfA+L8aB7cNWFJ0
         tNM/dsHJ7AQqOSfK/sdvjSxyZX3h56f64HQEnI1DHU6EgzlEzztCO1HZrJVV8b7XT6/z
         5gUaIOvcihzoeOaBYL/a33m5Kctvk/SRveY/c6qKWOnX1KWN9arnFx4Ss5VrHuq0H9ei
         dSKQ==
X-Gm-Message-State: AOJu0Yxrk/eNjqedPayfGcNQ0EWE69SQLXq1lTHjtdOlBfILGqdA14yA
	Ku5PQe5iirTigPdC7JA9VgaXwMbQCHSgrax3xHDd28B+aRQWkGNG5qkISqZITuXRVQFXRK93CIY
	eHL8cHw==
X-Google-Smtp-Source: AGHT+IHhPGxThXgl+Cl+4R/HAmEAJWeQORbF2Hq40kMHECeMXUANbn8YZYl48tmPJZb3SE0lHQXvzA==
X-Received: by 2002:aa7:cc0f:0:b0:561:fe3f:695c with SMTP id q15-20020aa7cc0f000000b00561fe3f695cmr1534882edt.36.1707904084169;
        Wed, 14 Feb 2024 01:48:04 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXVHOdwH2E+s6sG0Is3Wplpaj7z45Q1NnJHErQnEBqkP8hwhcjCFfT2cL8IQMnmLkzSafatoS0gV/tle5Rp7jEtqRrvlhVUoYhsMwKulcVIY7p/nGjC/a7I/wyoIcU2NNMCsZmMPGin9BwN1kJUe5knvIiTUnLlyVPavoit+bv/nifEjdSz1x/z+DFgGmwT/KkYsTyj6KKXEOAiVCaHlq8bDtkjTyVdddRfRxRjh9fQ5f10AZCLizLDjsrh6QzMmtvn0Gp1K4MvGbN6
From: Frediano Ziglio <frediano.ziglio@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: Frediano Ziglio <frediano.ziglio@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Dario Faggioli <dfaggioli@suse.com>,
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Subject: [PATCH v2] Constify some parameters
Date: Wed, 14 Feb 2024 09:47:34 +0000
Message-Id: <20240214094734.13533-1-frediano.ziglio@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Make clear they are not changed in the functions.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
--
v2:
- fixed typo in commit message.
---
 xen/arch/x86/pv/callback.c | 4 ++--
 xen/common/sched/compat.c  | 2 +-
 xen/common/sched/core.c    | 2 +-
 xen/xsm/flask/flask_op.c   | 8 ++++----
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/pv/callback.c b/xen/arch/x86/pv/callback.c
index 17829304fe..caec4fb16f 100644
--- a/xen/arch/x86/pv/callback.c
+++ b/xen/arch/x86/pv/callback.c
@@ -48,7 +48,7 @@ static void unregister_guest_nmi_callback(void)
     memset(t, 0, sizeof(*t));
 }
 
-static long register_guest_callback(struct callback_register *reg)
+static long register_guest_callback(const struct callback_register *reg)
 {
     long ret = 0;
     struct vcpu *curr = current;
@@ -102,7 +102,7 @@ static long register_guest_callback(struct callback_register *reg)
     return ret;
 }
 
-static long unregister_guest_callback(struct callback_unregister *unreg)
+static long unregister_guest_callback(const struct callback_unregister *unreg)
 {
     long ret;
 
diff --git a/xen/common/sched/compat.c b/xen/common/sched/compat.c
index dd97593630..a02204ec9a 100644
--- a/xen/common/sched/compat.c
+++ b/xen/common/sched/compat.c
@@ -26,7 +26,7 @@ CHECK_sched_shutdown;
 CHECK_sched_remote_shutdown;
 #undef xen_sched_remote_shutdown
 
-static int compat_poll(struct compat_sched_poll *compat)
+static int compat_poll(const struct compat_sched_poll *compat)
 {
     struct sched_poll native;
 
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index d177c675c8..c5db373972 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -1431,7 +1431,7 @@ static void vcpu_block_enable_events(void)
     vcpu_block();
 }
 
-static long do_poll(struct sched_poll *sched_poll)
+static long do_poll(const struct sched_poll *sched_poll)
 {
     struct vcpu   *v = current;
     struct domain *d = v->domain;
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index b866e8d05f..ea7dd10dc8 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -469,7 +469,7 @@ static int flask_security_load(struct xen_flask_load *load)
     return ret;
 }
 
-static int flask_devicetree_label(struct xen_flask_devicetree_label *arg)
+static int flask_devicetree_label(const struct xen_flask_devicetree_label *arg)
 {
     int rv;
     char *buf;
@@ -492,7 +492,7 @@ static int flask_devicetree_label(struct xen_flask_devicetree_label *arg)
 
 #ifndef COMPAT
 
-static int flask_ocontext_del(struct xen_flask_ocontext *arg)
+static int flask_ocontext_del(const struct xen_flask_ocontext *arg)
 {
     int rv;
 
@@ -506,7 +506,7 @@ static int flask_ocontext_del(struct xen_flask_ocontext *arg)
     return security_ocontext_del(arg->ocon, arg->low, arg->high);
 }
 
-static int flask_ocontext_add(struct xen_flask_ocontext *arg)
+static int flask_ocontext_add(const struct xen_flask_ocontext *arg)
 {
     int rv;
 
@@ -550,7 +550,7 @@ static int flask_get_peer_sid(struct xen_flask_peersid *arg)
     return rv;
 }
 
-static int flask_relabel_domain(struct xen_flask_relabel *arg)
+static int flask_relabel_domain(const struct xen_flask_relabel *arg)
 {
     int rc;
     struct domain *d;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:52:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:52:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680325.1058409 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBwo-00059S-6B; Wed, 14 Feb 2024 09:52:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680325.1058409; Wed, 14 Feb 2024 09:52:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raBwo-00059L-3X; Wed, 14 Feb 2024 09:52:58 +0000
Received: by outflank-mailman (input) for mailman id 680325;
 Wed, 14 Feb 2024 09:52:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raBwn-00059F-3K
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:52:57 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d38852a1-cb1e-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 10:52:55 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-411f165ef9bso1848125e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:52:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 cl3-20020a5d5f03000000b0033b7ce8b496sm9409787wrb.108.2024.02.14.01.52.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 01:52:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d38852a1-cb1e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707904375; x=1708509175; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=t+SdJxmD+d33X6aL2leTE53nVcqdhOkniVqz8xJ49Vg=;
        b=UwwEiZ3fWREdlxa3gloatinAG/0fEwBnM7Q42AAsdtD9wSGmAqoDPmruNmFN0Tn0GG
         PV04r1l61MfYyegkqJ4ngbuayL7GjUadZxUBZheG8Fto29rks44nzk7M2/wmhK51m0l6
         A6QEyVPT3gd3xqHBDPDfu2BXAQA5RaxbKaVZuk+C1kjeVee+UYuQX2WsmebIa4T+aMOs
         PLCmi5tPPdH1GDi3hGVtwcVDutCXreUfB+SYTf6h3xLhW8P+gDtTN1UKQiB8UwAjiON4
         5rsq/f48Mi4avOJnRjpx78uRdtBMQIvQztCGR94FENiEP5wQ9IJmbZK6ij26lKozkSDT
         1C9Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707904375; x=1708509175;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=t+SdJxmD+d33X6aL2leTE53nVcqdhOkniVqz8xJ49Vg=;
        b=DnNvgmNNRVzFP/sPMAWO7d+KxJtRU1x2l6VR/R9Fh9dXO/xaNg4EVe9MyYGLJu7qmV
         6mVqBed8R81Wq5mVZ8oB0ol1GaOObvLSji8GKXMXKKRPWvfmU36igsifHN5/Bs/XI7mg
         9W7CtwUxBAWb+fRAnVxZCEwCcPCIuz2bhNFLBz95orfZjupo6os6Aw0PMFVFirZA9anU
         4dKesUyeXY8idcgbpf2vlZcTyBck72yewwGGlTOA0HiaQUmw0ywj8Q1kc2VgfxcVDWi3
         xLjbfuNKQFz5KfrOkF99AkCAAlFpBoYOV3c5tWG0lhOkGFFrWm4/ZGUbxkGVRMy7oXIj
         kjAQ==
X-Forwarded-Encrypted: i=1; AJvYcCVN/J2qJPzQzvqazjRChyyr4XBLv0+ROeC9AD0rHCvySfMkKSVXrmnAW+CjQ68Jm52prMiJfehV395kqzju4UjnoYRAX3Nrn/bRxY06baw=
X-Gm-Message-State: AOJu0YySNwTdv4sO2jHaOdpWP313qaUAV64eNueFXwW/QJ/eF31cfyPD
	j937C5C092tjEzqL3LxgrZ4kDlKDWetiexvTbMInYMxYCq6OewRFxJq6Ei0oeQ==
X-Google-Smtp-Source: AGHT+IEjSooyQXKvxKax9gFUr69DJuFtV9lB1d26QbFddkQ3hTT9jkoZqPZSBl8Eb5GYWd62+g4Vzw==
X-Received: by 2002:a5d:5913:0:b0:33b:51a0:4dd3 with SMTP id v19-20020a5d5913000000b0033b51a04dd3mr1407884wrd.17.1707904374925;
        Wed, 14 Feb 2024 01:52:54 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVVkTblXFrYNjf3hQM/EvXuMn/MEOz8xEwKJcBGhyAZv9nCCyts5+xq0m8eAPeEesuvI5gYDjwJwFYtN9wmxF6l044RFGKKowEnsB9VDl691Vav230M3odx/CMMbIb3tNdPbijmLu33/1eVd7tapJ4zzzzyD0Zc9tj2T7Nvo76icKdGQGZ+PG42tfVyZVD9TG7nMEaqqQ7TVDyL9n0=
Message-ID: <16162577-dc0f-4c4b-acd5-9c2519f94c9a@suse.com>
Date: Wed, 14 Feb 2024 10:52:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 30/30] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <d62dff38ee661f3fb713554d544c966fa889fd83.1707146506.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d62dff38ee661f3fb713554d544c966fa889fd83.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 16:32, Oleksii Kurochko wrote:
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  Changes in V4:
>   - Update version of GCC (12.2) and GNU Binutils (2.39) to the version
>     which are in Xen's contrainter for RISC-V
> ---
>  Changes in V3:
>   - new patch
> ---
>  README | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/README b/README
> index c8a108449e..9a898125e1 100644
> --- a/README
> +++ b/README
> @@ -48,6 +48,9 @@ provided by your OS distributor:
>        - For ARM 64-bit:
>          - GCC 5.1 or later
>          - GNU Binutils 2.24 or later
> +      - For RISC-V 64-bit:
> +        - GCC 12.2 or later
> +        - GNU Binutils 2.39 or later

And neither gcc 12.1 nor binutils 2.38 are good enough? Once again the
question likely wouldn't have needed raising if there was a non-empty
description ...

Also - Clang pretty certainly supports RISC-V, too. Any information on
a minimally required version there?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:54:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:54:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680328.1058419 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raByA-0005kQ-IR; Wed, 14 Feb 2024 09:54:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680328.1058419; Wed, 14 Feb 2024 09:54:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raByA-0005kJ-Fv; Wed, 14 Feb 2024 09:54:22 +0000
Received: by outflank-mailman (input) for mailman id 680328;
 Wed, 14 Feb 2024 09:54:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raBy9-0005kB-Eg
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:54:21 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0641b1a7-cb1f-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 10:54:20 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a3d002bc6f7so172163766b.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:54:20 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 rp1-20020a170906d96100b00a3d623d1adasm70066ejb.118.2024.02.14.01.54.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:54:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0641b1a7-cb1f-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707904460; x=1708509260; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=ciF9uY8fLw9+WCGFP+zSgB0zLXEE1BJOysGzHHC8bH8=;
        b=k5j3TkBmQoyHIghxQUnRbhMIOVCB+S/B4O+sZG9N3yuSxHLhYvvx6J3mVx4ay9Awb+
         V36usUqA5VSze9Abxtbk78aVI1v8EGfRrlsbNU4ffzuJJddKlrV1nxG0GVIleK+W7WEe
         JRdPuHghp7zyCVkeLqIlN29JuTC34AkPMfhH7N1AHVA9uYew1E+BmWmbhHClTMPCgj/E
         MmCmBKZRIe4AvweOVY0mJfBO/HeKlf3GiOGlk7ZpRcbAcP4YL7ZlQP5dyWlYVDijWCPw
         l7VydVVEi4hVcVcmQ7E8K52gLbIKgt+YRjzdquBWYo07zArQq8KEL/bQzcBAnXYHqcxp
         E6jA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707904460; x=1708509260;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ciF9uY8fLw9+WCGFP+zSgB0zLXEE1BJOysGzHHC8bH8=;
        b=fbVaf+8CmEpGtPvvc+7/vLYpRSPIGwd51YjH2Aq8dKrfJjAXhaZrKwoSJqkTapQQnG
         CWLcfgDGPrb8WuTJPCfT3MlYiiNoEEJ0XQqy2f2AJRwyNke8oOsHxpE5KwoRurHw6Rns
         QVOSPry07tLuZnCG6O7SmlnnSDtPiFadwfTjB6c7JQck9Sc81ib1bPX7LXvGHwtoSrob
         Hqvz2AYqSSo554t1xURKsksmvQSZr96W+iaQOkdXyJ4J1deOU8WK8BH2vvbFarcT0NRT
         bL4O7YgFGMteO8eWZxZISZhESR6BcBWBx0plEFcGAp/8tRcgnDQkqMlZAdOJnTjflR6c
         STnw==
X-Forwarded-Encrypted: i=1; AJvYcCXTGgsergyHlqXwEKQlvBm5tNrYsyNJremzD5xUCSTosZD6htxBwymVghAK9FsD2RUHBiGpmS0UAUV+eLmEGm44mu17H9jYVXzmdDo65ec=
X-Gm-Message-State: AOJu0Yw0T3acv5kH82wrsnu9J3WqH5+nmtUAsv+RxDomCZ06qhbaRz37
	THltt5vJUCbpAQNY1/97qIf5H/9kFgPNrOqnEyNzgHLywH8gRolw
X-Google-Smtp-Source: AGHT+IEGBqEHnWDDVGm61NE7A4JWctMc8w2m6KIKPNa0uGSjqY99sc2Qv9Rl9AYBEyc1wIcqb6CXWQ==
X-Received: by 2002:a17:906:339b:b0:a3c:aa02:3ca9 with SMTP id v27-20020a170906339b00b00a3caa023ca9mr1332642eja.25.1707904459939;
        Wed, 14 Feb 2024 01:54:19 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCULjTBf2Kahqdje0oeHa4jxD9/x9djAwka9xe5k3AfaWnG+f20SefpImYYjZWacD8lm8IpM5WHELcOI/66ery4fYOutA0I3TnJXf81TReSa2ZiNF21UWHx3uRcZP3cC4wP1dikAzLEo3SdVzIsmktL+1jv6alB/R/nVSn2p3LOC9K7J0HPBsgOeGnuiKRp5V3DOAAtrvgEqFqAoWgmXqxOsm9BJFRwcepgRvQHPKXBOr+zwWE3gJvRd18YTCdRL/OVQm6WXOwqYxVwZS4ZgFyEyh1yFbjkc96dCUVJ/BieCkLnRsmDAt/s=
Message-ID: <d2cc798462de9e0d7a89777ea0743fe6dce4d061.camel@gmail.com>
Subject: Re: [PATCH v4 02/30] xen/riscv: use some asm-generic headers
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 10:54:18 +0100
In-Reply-To: <16baca98-44fe-42ba-b61d-ff1945e0d2b5@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <a721f0c092306b589fae5f44bdaafcd94c60ed14.1707146506.git.oleksii.kurochko@gmail.com>
	 <16baca98-44fe-42ba-b61d-ff1945e0d2b5@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 16:03 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > Some headers are the same as asm-generic verions of them
> > so use them instead of arch-specific headers.
>=20
> Just to mention it (I'll commit this as is, unless asked to do
> otherwise):
> With this description I'd expect those "some headers" to be removed
> by
> this patch. Yet you're not talking about anything that exists;
> instead I
> think you mean "would end up the same". Yet that's precisely what
> asm-generic/ is for. Hence I would have said something along the
> lines of
> "don't need any customization".
Agree that "some headers" isn't the best one option to describe the
changes. Perhaps, it would be better to specify the headers which would
end up the same as asm-generic's version.
Thanks for such notes!

>=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > Acked-by: Jan Beulich <jbeulich@suse.com>
> > ---
> > =C2=A0As [PATCH v6 0/9] Introduce generic headers
> > =C2=A0(
> > https://lore.kernel.org/xen-devel/cover.1703072575.git.oleksii.kurochko=
@gmail.com
> > /)
> > =C2=A0is not stable, the list in asm/Makefile can be changed, but the
> > changes will
> > =C2=A0be easy.
>=20
> Or wait - doesn't this mean the change here can't be committed yet? I
> know the cover letter specifies dependencies, yet I think we need to
> come
> to a point where this large series won't need re-posting again and
> again.
We can't committed it now because asm-generic version of device.h,
which is not commited yet.

We can drop the change " generic-y +=3D device.h ", and commit the
current one patch, but it sill will require to create a new patch for
using of asm-generic/device.h. Or as an option, I can merge "generic-y
+=3D device.h" into PATCH 29/30 xen/riscv: enable full Xen build.

I don't expect that the of asm-generic headers will changed in
riscv/include/asm/Makefile, but it looks to me that it is better to
wait until asm-generic/device.h will be in staging branch.


If you have better ideas, please share it with me.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 09:57:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 09:57:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680332.1058430 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raC0q-0006IW-Vv; Wed, 14 Feb 2024 09:57:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680332.1058430; Wed, 14 Feb 2024 09:57:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raC0q-0006IP-St; Wed, 14 Feb 2024 09:57:08 +0000
Received: by outflank-mailman (input) for mailman id 680332;
 Wed, 14 Feb 2024 09:57:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raC0p-0006IF-2F
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 09:57:07 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 68593be0-cb1f-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 10:57:05 +0100 (CET)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-55ad2a47b7aso2657575a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 01:57:05 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 cx16-20020a05640222b000b00560f3954ffdsm4508159edb.24.2024.02.14.01.57.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 01:57:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68593be0-cb1f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707904624; x=1708509424; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=8VTXaA9NHvNzxxP4cL3FyLxwNAcpvbKK4kzi8ZQPxLk=;
        b=Q2ObylHOgIRja36gAWAuB0EKxyk6pEZi5AELpGtx0jyp/S/DUPwwjdiVZBG4K0/l63
         FY8Lm86s/ulOxGVROW8BWP3X5hLxJZ7JkLgm/DYZiOHX463TuUmT8bVCkB4h4JVDDImZ
         Ggnu7Db8k3j6Ci22uWxeFLDMVTwrvF52WU85km0NmtEWO9l2MRL2YJtfw8vC1njQj7Tn
         x51tHo2GtcGEg6oBLMUU0qSMUG15w5G6nD+jIERYvXIur2uSS6t11PcVzrD/IZqGUt9n
         yNF6ORV/hguGbINUf+VuC9hxrmF6J6vRM42ardbFkozKpSkDp9zwyu4nqSB51a+zdMkI
         GBgA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707904624; x=1708509424;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8VTXaA9NHvNzxxP4cL3FyLxwNAcpvbKK4kzi8ZQPxLk=;
        b=U2ZzdnvoejbhqJdjwqTwO4jEIXagrkEbCnTqFKsQ3vl0Ew4FpuAvEC6QplWw8KiaET
         f+5Vn8j2O+uo7WAL+D1nt1UUMl9K7Kqp1TrqepXMUrnLtI1jKbsY3Z1k9xJXb+0ayRRJ
         XMoTU4R+F7jWOIJw9PPPswizjdwyf3ZiPbhSTOphsoXAvqdTrWFRQQWysZF/Ptb3jrhw
         GsynxHfZ6JfCIXM2rFar8UNu6OSAsev8lUHU4NPIfW4NhwtmbWyTeHXkDwNqVi6zf6s2
         2utCTUP9T61UX1hE/yw7Z5PA3iHORY+Tu2piOePvHda0eMFcuboiqMfXY7DLZhxzmqry
         ldbg==
X-Forwarded-Encrypted: i=1; AJvYcCV57mpWJ+8DifRh1KGPgRwsFk7UU2MvYZehQXg9CFNLCqNNfg26X2Wyym2pIGJ+S2xfSmXbW5hJnWpNHmdl1pkTu/EwQK9EBO/fhYP/ZE4=
X-Gm-Message-State: AOJu0Ywpe0Wx7qVgdfvcOtE0GHAbfPcax8qSNICUpW7jw0xY5Rqdq3OB
	L5SLDw2n5dMw50Alx3E/U6z2RSCtV2pQAkuCJg6pyidLo3yIKQjX
X-Google-Smtp-Source: AGHT+IGsJDjTz7uiJjtqrQwNB6Nfloln+A0dP9Lm4d+meBqI6Rf+AGv/WMcK2ufapqkuA/c0frf6hQ==
X-Received: by 2002:a05:6402:68b:b0:562:129d:8dc with SMTP id f11-20020a056402068b00b00562129d08dcmr1490972edy.32.1707904624545;
        Wed, 14 Feb 2024 01:57:04 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXr6KNC7iQ+a7AzRT4go3zaQi3lkRUotNGMBvMVdU3Mu9WZ6UGCI3JdvZ5ku4KvSXW9Kd4s5O68xazz+UXCvLmRQ6bI9hllbL+iB4BQxr1Xwiqnr9rshMcm/gR5K2E5mLePnz/f/gMJ9rTc6uOnXUEBYO7U0WYHCL3/IuIyrZbZy1Ub6NnI17KlhdMCHt6/wyH8Lm/SQxt8EL8EX8o=
Message-ID: <f1486f3e6e5694dba12518a664a562d0ad907684.camel@gmail.com>
Subject: Re: [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC
 and RISC-V
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 10:57:03 +0100
In-Reply-To: <b3141aff-32ad-4e2c-aa2c-a5803924951f@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <c929f784466e1ba57c19663e88f22c598d961709.1707146506.git.oleksii.kurochko@gmail.com>
	 <b3141aff-32ad-4e2c-aa2c-a5803924951f@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 16:05 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > No specific header is needed to include in public/hvm/save.h for
> > PPC and RISC-V for now.
> >=20
> > Code related to PPC was changed based on the comment:
> > https://lore.kernel.org/xen-devel/c2f3280e-2208-496b-a0b5-fda1a2076b3a@=
raptorengineering.com/
> >=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Acked-by: Jan Beulich <jbeulich@suse.com>
>=20
> Albeit I don't see why ...
>=20
> > --- a/xen/include/public/hvm/save.h
> > +++ b/xen/include/public/hvm/save.h
> > @@ -89,8 +89,8 @@ DECLARE_HVM_SAVE_TYPE(END, 0, struct
> > hvm_save_end);
> > =C2=A0#include "../arch-x86/hvm/save.h"
> > =C2=A0#elif defined(__arm__) || defined(__aarch64__)
> > =C2=A0#include "../arch-arm/hvm/save.h"
> > -#elif defined(__powerpc64__)
> > -#include "../arch-ppc.h"
> > +#elif defined(__powerpc64__) || defined(__riscv)
> > +/* no specific header to include */
> > =C2=A0#else
>=20
> ... this isn't simply
>=20
> #elif !defined(__powerpc64__) && !defined(__riscv)
I can change that to your option in the next patch version if the patch
won't be merged now.

~ Oleksii
>=20
> > =C2=A0#error "unsupported architecture"
> > =C2=A0#endif
>=20



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 10:01:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 10:01:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680336.1058440 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raC5J-00006x-Gq; Wed, 14 Feb 2024 10:01:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680336.1058440; Wed, 14 Feb 2024 10:01:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raC5J-00006q-DJ; Wed, 14 Feb 2024 10:01:45 +0000
Received: by outflank-mailman (input) for mailman id 680336;
 Wed, 14 Feb 2024 10:01:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raC5I-00006k-Fi
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 10:01:44 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0d9a4aa2-cb20-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 11:01:42 +0100 (CET)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-5638de1d953so90932a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 02:01:42 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 cu3-20020a170906ba8300b00a3ce8093f6esm1921867ejd.179.2024.02.14.02.01.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 02:01:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0d9a4aa2-cb20-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707904902; x=1708509702; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=gtTT6VXl3pTBbwRSdecHEoBPSziWRDHJr7gxYr2gngk=;
        b=eclVRYznRnwvqY22iUKkqvf7kwo79TcIoQLPyD+grU9t3bNxmTVlaNRE8vav8XydDE
         VR5ySWkB3+0Dibj/PVmvjSDkntswhSotL7rZVn/d8Ts9glbg+7AXhrG2iH04yayA87rE
         WV1wQx3om68JCZ/RZH+D090huUtViCLddWgct8oAlsCG52paCedmzB5Jw3pqOKjW+oBd
         OPugX5wZCWw2AUFOtGh1ubG6MShKhqzTPYE6ylA84FyeI0kZORjSDmbuTBivv5QwW2gf
         ejCt6snDL+pYNm6h7OQTfnvTv3BNbB+V2PdP+0jG05/mRz0eu5LJWM2uTglXPqDfiK2V
         4qPQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707904902; x=1708509702;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gtTT6VXl3pTBbwRSdecHEoBPSziWRDHJr7gxYr2gngk=;
        b=YSbtHSAgj0UgdmjMUV3TGN1MvZpizbrGdzrkXrRKuWYzDG62AiNAwe5d2177gS/hH0
         FYsgxOBwMARPSWBmmfCyIJlo1DoHDsrjXspLFISvXd5GcyafpxozE496TSzx6DJFRbPq
         ccOwJPCtPqObc3hBIOLTgWF67rByHRm4Aydn9lbXEzjXI+UviUletf6/9IJFmN/Yo/hW
         3IEPrYisQyl7KICEbwHFrWomMfN7lvJ4HFr4JIsKXqTBi5lIcUoSVclViRcRLy0pU+Mo
         GE+MRVhujfWUG1qT5MQ9JRof2xc2XKtTLWcfDboE1EMP3edDrV+CMqxNwsjm1yW9QYKG
         /TAA==
X-Forwarded-Encrypted: i=1; AJvYcCU3+7WIOKb0v7LoJ32CIdPVtRXcSve2builc5a03vK2K87drRABlGHIfRbGlbQd1aEH3NbWs8rAfZwgWnquyCBrtGEb4YLoIPymGVpWND0=
X-Gm-Message-State: AOJu0YwoVZsQzat5CiJFLMhpQhmsBXxgNYWwNMSP2ta40wsQ+39XnGMS
	vYpcmutABmSfsE+T/H7QN7yAJUb8lHZyYBKfQeR1CJoodHeOYfa6
X-Google-Smtp-Source: AGHT+IH4jARuR/kt65qvT/4tWtF+Zj4SxshcNkM+rOpBqsTUL0FY9t8J3TK17B05EfsXDk+n3xKOUg==
X-Received: by 2002:a17:906:f0d1:b0:a3d:3a5:f7ad with SMTP id dk17-20020a170906f0d100b00a3d03a5f7admr1373318ejb.10.1707904901861;
        Wed, 14 Feb 2024 02:01:41 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXbqQq5r/fobu6cUCg0A+SDak9YjxdSky3yOE5mqSIZomcMvJtSMwOWDCsQKcJzXwLEQ5jJm5/TrgCGIiMJT12WiX+cncgWHwFFLGIA7vuJ7YduxlAzjnwa+HEV1vl7cu7+LCNXbt4CeXd6IQ8xOim4f+6RwRa13G/YM6t01wqb7VDbT2UxFAMoDcSrX2reUn5MmsNk214Dy0kcrQ9yx3R1nyYY0pW/7HMoqp5dlVReEXtj7FgbuKLhDpkDF21yD0wRy3p7UaCWahkLHwji9xmKXyY3hv1Fe/W6CSPK67lMdHomCiALbSI=
Message-ID: <dae11b4707f35469a0e9da798e5cab9cd5af280e.camel@gmail.com>
Subject: Re: [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 11:01:40 +0100
In-Reply-To: <ec76c372-8c17-4550-9896-eab3e0ab0c66@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <542c61229e1056c3fdec3a5c086136e3b5909b07.1707146506.git.oleksii.kurochko@gmail.com>
	 <ec76c372-8c17-4550-9896-eab3e0ab0c66@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 16:07 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Acked-by: Jan Beulich <jbeulich@suse.com>
Thanks!

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 10:03:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 10:03:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680340.1058450 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raC6k-0000oc-S0; Wed, 14 Feb 2024 10:03:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680340.1058450; Wed, 14 Feb 2024 10:03:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raC6k-0000oV-P2; Wed, 14 Feb 2024 10:03:14 +0000
Received: by outflank-mailman (input) for mailman id 680340;
 Wed, 14 Feb 2024 10:03:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raC6j-0000mj-RK
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 10:03:13 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4376f6eb-cb20-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 11:03:12 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51147d0abd1so6144201e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 02:03:12 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h8-20020a05600c350800b00410be9b8b96sm1462633wmq.37.2024.02.14.02.03.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 02:03:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4376f6eb-cb20-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707904992; x=1708509792; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rVYS5LtrkhFUYl8yNGaKUXrdEmZ4yJyiR3U8lVEBs9Q=;
        b=Gl8iUqWI3ZuFGbROC5sh3VB0UF54y9Z8rToxj7bvUZlmtHAcreMdsdC9xWu00X1Uvk
         xA4A7q/zRiJ0vJtyCox2tv/wXCCZfWNymvtZ8MtqkZFyC34M8Xl4WneGaVKr8DVF7Y0a
         ahbKfsCJhYX34VrMzErj45VYDHys5a9o1VKJS9GP6IjogpkboocfMOi5lFAuiboQuwiN
         GDKWYnnGr/tQQHUqU4ySgWh3bUcbzqrXDBb0Q4ZrHA6NaT2kxOA8PG0DV58/SXM9R9MW
         ujAky2PL6AXgHWtl/UKeuMvmXNL0Y00MEMa6lNIyuZGylpMcXwv6DMHA6MmN4DYoJ1Oy
         wUqg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707904992; x=1708509792;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rVYS5LtrkhFUYl8yNGaKUXrdEmZ4yJyiR3U8lVEBs9Q=;
        b=bvZfqz+aPiiGEcG1PadBDJv8cgoSSpLeAPAIJoJL8mCyx88Wfmn7hLYVESOOta+oKk
         S3OZ/wlewf8ifGUJxvYNIY35HSLiiKMrR9VGq7choILVumnbIOUQ/jYPM261xbgXJq9T
         Isjkf1h2N7fEXTsXW3YXstx1qzJaHjfwI/Hp/QeU7V2hHqn06AEagQCDYyQeNYSsO9gz
         bMbuv4jyjtV/GpFl9WykQa+4so9NGoy9KAdokWElJxZoEsLeaXu1Tr+z2/kPNyvXgxoy
         hNfLdtr4vcNS/r6F+S5rEUyYz0OJVT24LkK/pT4PHOQYrNoq0r7emUzIa0/Un516NJxh
         3Ceg==
X-Forwarded-Encrypted: i=1; AJvYcCUBPK4HPQWtBZCBrTEJRS7NE5rKOgyeCn79yChcPHOkqeEcO/zr40AX+1J4Xrxg6c5JJATVznJBPfNvVDTY7Pk7v8YEiXQyvktyprrj8dU=
X-Gm-Message-State: AOJu0YwdzRQG4GPKa6E5uOOzYEBXGN6vgLfvqIYKBvw1YEQzKE6puZV+
	EkLRYpHmmjM0HobwKpBlqpujvRtGM/2Z/r/3r6WrV5Xx7/iPWqlbJvmzFdQWfg==
X-Google-Smtp-Source: AGHT+IERo3d2VJJspHfumjA76VZoVoo4KA7Tw7S+yO7+JRI7+/MNIfz+j7lMtrb+y6+p3kciW8nZDw==
X-Received: by 2002:a19:ee0e:0:b0:511:4e6a:12e7 with SMTP id g14-20020a19ee0e000000b005114e6a12e7mr1294700lfb.58.1707904992204;
        Wed, 14 Feb 2024 02:03:12 -0800 (PST)
Message-ID: <5b8d6ff3-508c-49aa-8b25-3624d165eae1@suse.com>
Date: Wed, 14 Feb 2024 11:03:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 02/30] xen/riscv: use some asm-generic headers
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <a721f0c092306b589fae5f44bdaafcd94c60ed14.1707146506.git.oleksii.kurochko@gmail.com>
 <16baca98-44fe-42ba-b61d-ff1945e0d2b5@suse.com>
 <d2cc798462de9e0d7a89777ea0743fe6dce4d061.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d2cc798462de9e0d7a89777ea0743fe6dce4d061.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 10:54, Oleksii wrote:
> On Mon, 2024-02-12 at 16:03 +0100, Jan Beulich wrote:
>> On 05.02.2024 16:32, Oleksii Kurochko wrote:
>>>  As [PATCH v6 0/9] Introduce generic headers
>>>  (
>>> https://lore.kernel.org/xen-devel/cover.1703072575.git.oleksii.kurochko@gmail.com
>>> /)
>>>  is not stable, the list in asm/Makefile can be changed, but the
>>> changes will
>>>  be easy.
>>
>> Or wait - doesn't this mean the change here can't be committed yet? I
>> know the cover letter specifies dependencies, yet I think we need to
>> come
>> to a point where this large series won't need re-posting again and
>> again.
> We can't committed it now because asm-generic version of device.h,
> which is not commited yet.
> 
> We can drop the change " generic-y += device.h ", and commit the
> current one patch, but it sill will require to create a new patch for
> using of asm-generic/device.h. Or as an option, I can merge "generic-y
> += device.h" into PATCH 29/30 xen/riscv: enable full Xen build.
> 
> I don't expect that the of asm-generic headers will changed in
> riscv/include/asm/Makefile, but it looks to me that it is better to
> wait until asm-generic/device.h will be in staging branch.
> 
> 
> If you have better ideas, please share it with me.

My main point was that the interdependencies here have grown too far,
imo. The more that while having dependencies stated in the cover letter
is useful, while committing (and also reviewing) I for one would
typically only look at the individual patches.

For this patch alone, maybe it would be more obvious that said
dependency exists if it was last on the asm-generic series, rather
than part of the series here (which depends on that other series
anyway). That series now looks to be making some progress, and it being
a prereq for here it may be prudent to focus on getting that one in,
before re-posting here.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 10:05:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 10:05:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680344.1058459 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raC9G-0001WU-7E; Wed, 14 Feb 2024 10:05:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680344.1058459; Wed, 14 Feb 2024 10:05:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raC9G-0001WN-4j; Wed, 14 Feb 2024 10:05:50 +0000
Received: by outflank-mailman (input) for mailman id 680344;
 Wed, 14 Feb 2024 10:05:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raC9F-0001WH-2T
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 10:05:49 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a00f967c-cb20-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 11:05:48 +0100 (CET)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a293f2280c7so745919366b.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 02:05:48 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 w18-20020a17090652d200b00a3d296f46besm833671ejn.120.2024.02.14.02.05.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 02:05:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a00f967c-cb20-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707905147; x=1708509947; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=2xgJgiFEZzPqRcLFJ9aO6MdPD1MeRpyCIrCOfZZoQhg=;
        b=OVz3aumlTIiCb6kiLzIYessgbwYnvoStaYPsp+q0xniLTlHHAB2igYm4A+DKB2NSq+
         sruFrZ1TsJNd8g/4YGxjmwUl4gLLI+aYZnFkALlgScspPk6ER7xh0B+csUhnlpgp/c6S
         2g+eRtosG/eR/6zl8arDON6g0DzEsk2IetsYN3N/bmgr2qoTfpMifTVjcGBTd/2KUIzy
         yTiq0Sau0ZijGQdvFsd1AsC9o8fMJ9KJvDKZ19ydURhUcw75nKc0poswt5AYOLLlAqHn
         oaOUJpqZeFu+ZdwYMEENEtmd4GmgcWLYQ+hPBVhIYHiqQN/yfNekNchJcPE8G831X/NI
         wPYQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707905147; x=1708509947;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2xgJgiFEZzPqRcLFJ9aO6MdPD1MeRpyCIrCOfZZoQhg=;
        b=YIjZMbbfrS1jPGDC9612FFCDFNhahn/OuqOpuIFvXTWeIPoAfMvogzfL4pwTzZ80RK
         Oz9qppS+pxmYQ3eSp8ph2uQK8IghcxgJR/LL9vmM3iLmHxrME8U8a1T+N5S1Xg3e2uxT
         0JXV5M2/2TYn6vM9440PUwSPqtCV0G8xrb2TnJD6LAbXTy4hSasmHiykAneZj5nj2da6
         zPc21JbQy2eyEiPYt6ICOwYyuWL0WeSh9nbmHMHbRPxN6uRzz4/BrubALuTNp3OiJWx/
         B0RgKGneKf2bdLWjec9w4LWmAAPdPBsH/i8YPSfD1T7xbbvaCxZA6hVozxgp502z7yQc
         vatA==
X-Forwarded-Encrypted: i=1; AJvYcCUyJQLGJ+iTTeI/p3dCfUDD7VNgAwWtwjnGHXK2QSVJgRTr1ddRqDkZqns+RPoOyyXcNaSpLhLkVGWpAUD9cL+nHaYtK85AdBKmF7H5PAw=
X-Gm-Message-State: AOJu0YxSiy0vb40yhd8t/RhgrpoDuX5+LKFy5l5Etk+C6xwonG2x8l+W
	XIVoPsGJf7PU9e3KDPbavIKuz3L1N/Gitt3lHR3EJoN/vVfuUW4S
X-Google-Smtp-Source: AGHT+IGrupi4bYNxUawyMlYYQFOZPDtgIVlXbiuYH52DUepVKSXM6ir5SpKdZVBt9G81Z07Hm+W3Xw==
X-Received: by 2002:a17:906:b752:b0:a31:1154:89a9 with SMTP id fx18-20020a170906b75200b00a31115489a9mr1566077ejb.26.1707905147475;
        Wed, 14 Feb 2024 02:05:47 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWUxhguMStd+NQ7temXCLaFabUEHam1aG9lNR7xE70CTVAaOp9AgYfN2uIGKLoJNIFscjcAWioTDJ/c4stElEmLD3F/Wo/0emY6vomYsvfpjh9BgsX2yKTb39xDAv06bQoR3PjB
Message-ID: <4ffa7f8e003ecaab8251f4534f5d81669d77fcd6.camel@gmail.com>
Subject: Re: [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Paul Durrant <paul@xen.org>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
	 <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 11:05:46 +0100
In-Reply-To: <81593ac4-7714-4dd1-89f9-c22e88223d34@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <a94aba19d32bbe8d2fa78d6d8931f9f0ca51160a.1707146506.git.oleksii.kurochko@gmail.com>
	 <81593ac4-7714-4dd1-89f9-c22e88223d34@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 16:10 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > asm/iommu.h shouldn't
>=20
> ... need to ...
>=20
> > be included when CONFIG_HAS_PASSTHROUGH
> > isn't enabled.
> > As <asm/iommu.h> is ifdef-ed by CONFIG_HAS_PASSTHROUGH it should
> > be also ifdef-ed field "struct arch_iommu arch" in struct
> > domain_iommu
> > as definition of arch_iommu is located in <asm/iommu.h>.
> >=20
> > These amount of changes are enough to avoid generation of empty
> > asm/iommu.h for now.
>=20
> I'm also inclined to insert "just" here, to make more obvious why
> e.g.
> ...
>=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > Changes in V4:
> > =C2=A0- Update the commit message.
> > ---
> > Changes in V3:
> > =C2=A0- new patch.
> > ---
> > =C2=A0xen/include/xen/iommu.h | 4 ++++
> > =C2=A01 file changed, 4 insertions(+)
> >=20
> > diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
> > index a21f25df9f..7aa6a77209 100644
> > --- a/xen/include/xen/iommu.h
> > +++ b/xen/include/xen/iommu.h
> > @@ -337,7 +337,9 @@ extern int
> > iommu_add_extra_reserved_device_memory(unsigned long start,
> > =C2=A0extern int iommu_get_extra_reserved_device_memory(iommu_grdm_t
> > *func,
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 void *ctxt);
> > =C2=A0
> > +#ifdef CONFIG_HAS_PASSTHROUGH
> > =C2=A0#include <asm/iommu.h>
> > +#endif
> > =C2=A0
> > =C2=A0#ifndef iommu_call
> > =C2=A0# define iommu_call(ops, fn, args...) ((ops)->fn(args))
> > @@ -345,7 +347,9 @@ extern int
> > iommu_get_extra_reserved_device_memory(iommu_grdm_t *func,
> > =C2=A0#endif
> > =C2=A0
> > =C2=A0struct domain_iommu {
> > +#ifdef CONFIG_HAS_PASSTHROUGH
> > =C2=A0=C2=A0=C2=A0=C2=A0 struct arch_iommu arch;
> > +#endif
> > =C2=A0
> > =C2=A0=C2=A0=C2=A0=C2=A0 /* iommu_ops */
> > =C2=A0=C2=A0=C2=A0=C2=A0 const struct iommu_ops *platform_ops;
>=20
> ... this is left visible despite quite likely being meaningless
> without
> HAS_PASSTHROUGH.
>=20
> Then (happy to make the small edits while committing):
I'll be happy with that. Thanks.

> Acked-by: Jan Beulich <jbeulich@suse.com>
Thanks for Ack.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 10:11:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 10:11:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680350.1058469 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCEa-0003tF-Uq; Wed, 14 Feb 2024 10:11:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680350.1058469; Wed, 14 Feb 2024 10:11:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCEa-0003t8-S5; Wed, 14 Feb 2024 10:11:20 +0000
Received: by outflank-mailman (input) for mailman id 680350;
 Wed, 14 Feb 2024 10:11:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Sn1=JX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1raCEZ-0003t2-OI
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 10:11:19 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6524e94b-cb21-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 11:11:18 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a3d5181b3a7so50866866b.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 02:11:18 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 xa4-20020a170907b9c400b00a3d669a2055sm3100ejc.88.2024.02.14.02.11.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 02:11:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6524e94b-cb21-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707905478; x=1708510278; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ttnhX1vyDb1plETASpPah46BfJRyMiz9hrM2P7CaeCM=;
        b=RICXzomta55cEyb6zJFJ4WeF7edSddCQMdH59flILy1Hwfk06eRLfwhUEw9zzCi8WK
         Q9TcEjcGxfwjLXEG4+T7wJrVwbVT9sSYZqG+OyCMI6k+sE7IP6wgR5geKwiosTZNdOZl
         EoOLa0WfDcSC2Tjow+fMkb/lFk/I1qGSBy56I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707905478; x=1708510278;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ttnhX1vyDb1plETASpPah46BfJRyMiz9hrM2P7CaeCM=;
        b=g+3Lde6+nERI5JcvfL7D5EwnKUGox+4Efb1puKlPPMCvAystxsv3oOhkzSeHvvlukW
         m68Uy5mtIHJnWO64Z5pJWchtmCE4EF1SW9ZoRUODZVwrRi1K19pN9GXTuKgawRcNnhlv
         wyFh13X6w0eOkMCToQJvSMXXkgXzpYLxgw/Ilrz44mx7vJQLhEIQv/YUPsdjKzrqRyEl
         G53hIx0bWxkssFsDNf+eiXeHSMcvZOwJCTVROk5JFHOOJTRE8ARMvOPaA/POQ/Dwpw/L
         vQmBQxyugm4TgjIA1dZIk0KFa/Jy1AdbBFhvmzZUl7Sn+nyDrPCIdhKoqi6UcDXarOhd
         1vRw==
X-Forwarded-Encrypted: i=1; AJvYcCUecL6JDVqaNpJXqoIG96RRDCuvbwQto8eaPr0SQIj/v2Pyl3UabYXV0fObK7ESothKTiZ4Qa9PfFIuNJhtTEFUTDjfskUH8LGiiswa7Xk=
X-Gm-Message-State: AOJu0YwMUHnyI9I48oyK3fzUpgsNMm0ZhszavL1T3x0B9qSdMna0aQy4
	OixGjxSWkHGvo2cgdyvPVAn5KoOOTUkwg84NZAG/WMCxMfsqLm4amIo75cYTG1w=
X-Google-Smtp-Source: AGHT+IE37UagxXXvcmh1o1MD2lFSW3g3hwZe/7We13S26Nj0bRk7PcIZKX2lBOut/8Evqnyu0OmRlg==
X-Received: by 2002:a17:906:ce2b:b0:a38:5b3c:7c9b with SMTP id sd11-20020a170906ce2b00b00a385b3c7c9bmr1396476ejb.18.1707905478199;
        Wed, 14 Feb 2024 02:11:18 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXo81/8DafWu+PSgb3Xs9IgEMAhwMPB3lvphudh2aKegwdEknXsXUsG5hgKpoHyR6ml+qN3PNWFdQpdjYmJwXopJHiFnvUtn1gOlPYqDlgWcgr2/EB+2xD9GJGvkLnOXm6v
Message-ID: <863655b7-c889-4b7c-bb21-3d93b28b7ef8@citrix.com>
Date: Wed, 14 Feb 2024 10:11:17 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [regression] Re: [PATCH v2 2/2] iommu/vt-d: switch to common RMRR
 checker
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jan Beulich <jbeulich@suse.com>
Cc: Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240207153417.89975-1-roger.pau@citrix.com>
 <20240207153417.89975-3-roger.pau@citrix.com>
 <c27c76ec-36cd-43cb-b76f-e8f95fb27ed8@suse.com>
 <c4f27180-86bf-45fd-8641-bd160c6de229@citrix.com>
 <37ed1abe-afcc-4a76-8a86-623282ca37a3@suse.com> <Zcx9w5eZkXiN9s66@macbook>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <Zcx9w5eZkXiN9s66@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/02/2024 8:45 am, Roger Pau Monné wrote:
> I've found it:
>
>     for ( addr = start; mfn_x(addr) <= mfn_x(end); mfn_add(addr, 1) )
>
> Should be:
>
>     for ( addr = start; mfn_x(addr) <= mfn_x(end); addr = mfn_add(addr, 1) )

Coverity did end up spotting this.

> New defect(s) Reported-by: Coverity Scan
> Showing 1 of 1 defect(s)
>
>
> ** CID 1592056:  Incorrect expression  (USELESS_CALL)
>
>
> ________________________________________________________________________________________________________
> *** CID 1592056:  Incorrect expression  (USELESS_CALL)
> /xen/drivers/passthrough/x86/iommu.c: 807 in iommu_unity_region_ok()
> 801             return true;
> 802     
> 803         printk(XENLOG_WARNING
> 804                "%s: [%#" PRI_mfn " ,%#" PRI_mfn "] is not (entirely) in reserved memory\n",
> 805                prefix, mfn_x(start), mfn_x(end));
> 806     
>>>>     CID 1592056:  Incorrect expression  (USELESS_CALL)
>>>>     Calling "mfn_add(addr, 1UL)" is only useful for its return value, which is ignored.
> 807         for ( addr = start; mfn_x(addr) <= mfn_x(end); mfn_add(addr, 1) )
> 808         {
> 809             unsigned int type = page_get_ram_type(addr);
> 810     
> 811             if ( type == RAM_TYPE_UNKNOWN )
> 812             {


~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 10:12:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 10:12:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680352.1058479 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCFl-0004jb-7O; Wed, 14 Feb 2024 10:12:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680352.1058479; Wed, 14 Feb 2024 10:12:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCFl-0004jU-4P; Wed, 14 Feb 2024 10:12:33 +0000
Received: by outflank-mailman (input) for mailman id 680352;
 Wed, 14 Feb 2024 10:12:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raCFj-0004jO-SD
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 10:12:31 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8f7b5a7c-cb21-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 11:12:29 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-50eac018059so6887278e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 02:12:29 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p13-20020a05600c204d00b00410df4bf22esm1453519wmg.38.2024.02.14.02.12.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 02:12:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8f7b5a7c-cb21-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707905549; x=1708510349; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Pnu4VXy5X8iZt6vvrCxKxi+hnaJLFi62JTayobfCEu0=;
        b=I5JKJyd4eEgh2DkRqvO9Rurzvp4PuY403IiVanUFTHTnrZOaRGEWVMQ1Llc1KtD+F3
         iF309RF5QxtagorDftdhzyNJXDbLT6fselTbniky33aJ1i/Hi5SYEz3QasPBfSBhtdIL
         iD4al0pyFV48udaHJxM8v75yMqRtiLx6ZZiGK1EQ7sbvhU2iCxz/bdhO2S97tcOVdSqh
         M30DSb4LCcU+4vsVTnnCsxPA4jvxH7D2CEioioX5oU09HiUqRnyJU6QhR53zNKjRD/hG
         TUkVnGvzaDKdq7KTbUaVMSGKwBZUB8tDmnVDustmxwur5I3sOqVOETI7BLtFlA+iSMmG
         Gz5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707905549; x=1708510349;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Pnu4VXy5X8iZt6vvrCxKxi+hnaJLFi62JTayobfCEu0=;
        b=AZHyecZKruWNxJpHWXpO++5IINejiicd3qg6dEbHEA8aBeF+zk/kphrHsH1fkNsSye
         aYFz3TLA1IgC/jW7hwFluCClTpgomwZZOoIdak7o40dCFUMmKnbVPtRCOvXElMLnddy1
         dw4QOZgaji89Zn4MGe/D/FRXqcCahC07t0pPL8brPdpc0VKg5LZf4UQtnDSzuK4nZ2kU
         dCyMw4heAn5N8cmKUjMs38T0V7cWO/z2MyHQhJYlF/oyTaZ4GmVa/4P5EQ8CT75fH5bN
         GIhcuGc+h44Nxya0OX9J8v9G5EQ6T1DoTZdf8F6NxavXj6zVefyoGRLsD1qo67T+BYHT
         yX5Q==
X-Gm-Message-State: AOJu0YzVaM+aTeckJMzRVkeAe3j86hZNtllhKgnhaZF9Z3faDRqJNy+o
	/vp4c8Jck48ZDGABxgaTQ8K/XuqW5vKi+FJ5Nr79ScoMJpbKlGvHPZmYWYRWcuKowd35dODIWDA
	=
X-Google-Smtp-Source: AGHT+IG3QnsVOHp4kjee9gR7OshWqgfwQNgC1rLlP678YEJa158UljafE+ez5qWAho9mMNWzhZ0gjQ==
X-Received: by 2002:a05:6512:234f:b0:511:a90b:5334 with SMTP id p15-20020a056512234f00b00511a90b5334mr986271lfu.20.1707905549268;
        Wed, 14 Feb 2024 02:12:29 -0800 (PST)
Message-ID: <79ed633d-b0bd-4a7d-a0c6-37a034e1ee96@suse.com>
Date: Wed, 14 Feb 2024 11:12:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2] Argo: don't obtain excess page references
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Christopher Clark <christopher.w.clark@gmail.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

find_ring_mfn() already holds a page reference when trying to obtain a
writable type reference. We shouldn't make assumptions on the general
reference count limit being effectively "infinity". Obtain merely a type
ref, re-using the general ref by only dropping the previously acquired
one in the case of an error.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I further question the log-dirty check there: The present P2M type of a
page doesn't really matter for writing to the page (plus it's stale by
the time it is looked at). Instead I think every write to such a page
needs to be accompanied by a call to paging_mark_dirty().
---
v2: Re-base.

--- a/xen/common/argo.c
+++ b/xen/common/argo.c
@@ -1426,7 +1426,7 @@ find_ring_mfn(struct domain *d, gfn_t gf
     switch ( p2mt )
     {
     case p2m_ram_rw:
-        if ( !get_page_and_type(page, d, PGT_writable_page) )
+        if ( !get_page_type(page, PGT_writable_page) )
             ret = -EINVAL;
         break;
 
@@ -1441,7 +1441,8 @@ find_ring_mfn(struct domain *d, gfn_t gf
         break;
     }
 
-    put_page(page);
+    if ( unlikely(ret) )
+        put_page(page);
 
     return ret;
 }


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 10:14:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 10:14:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680356.1058490 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCHH-0005I4-Hm; Wed, 14 Feb 2024 10:14:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680356.1058490; Wed, 14 Feb 2024 10:14:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCHH-0005Hx-Es; Wed, 14 Feb 2024 10:14:07 +0000
Received: by outflank-mailman (input) for mailman id 680356;
 Wed, 14 Feb 2024 10:14:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kQr2=JX=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1raCHG-0005Hp-7W
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 10:14:06 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:200a::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c704ebb1-cb21-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 11:14:04 +0100 (CET)
Received: from BN1PR14CA0010.namprd14.prod.outlook.com (2603:10b6:408:e3::15)
 by IA1PR12MB7615.namprd12.prod.outlook.com (2603:10b6:208:428::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25; Wed, 14 Feb
 2024 10:14:00 +0000
Received: from BN2PEPF000044A1.namprd02.prod.outlook.com
 (2603:10b6:408:e3:cafe::97) by BN1PR14CA0010.outlook.office365.com
 (2603:10b6:408:e3::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26 via Frontend
 Transport; Wed, 14 Feb 2024 10:14:00 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN2PEPF000044A1.mail.protection.outlook.com (10.167.243.152) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 10:14:00 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb
 2024 04:13:59 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb
 2024 04:13:59 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 14 Feb 2024 04:13:57 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c704ebb1-cb21-11ee-8a4d-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=YVHonfe+A5YykeHNuU40H2cRfbQMMvG0HCfx5CyOtd7pBH0K5gk7pwqWmxZAhBHwOuRV4gELHeecddi+6GQU6vLWnP8x/RcGUAi8a6G2OpbneZSKu+tOBE39iGNXcswfJudkPBZdM9JyIwm8r2zv9QUNlNSR6QUuxDwk7juWNU+SHTsIjljngPe9XmOkho0GA+QI850eMNphZE1ZgKXvmocnOf4WJEilvQfdshhS04YhcrSF/60YnupETTqBRm/iFUzcMA4GStr/UxI9hzTdia1Pe6YTC9XfcxNFRYLQqDvOunTtPLL7FfNWuYWltRCzyiYFrPkoTI54wF2Pu+LqIg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=U/xOuPKjtqCJRT1ppKqXNV5kBGA3BCkrVZqPvDGj4uM=;
 b=FnwbpKxeTUzxXzFjiJ2u9oCZtCm8YjYYRCcwzjIbNFrNISpmoqsvfAoxxzWmesLcym2BQxyWnQiuDAaUfF23sPy3prtC0vIA1mVx/NZTQ3f73rtfaWs2EiHydfNGbvlEVy1EbKI0xBgT8l40Zm6O5RXxVNC8vjsZlhkA6xUPXen9iUPlT476K/aqPOgg3YmgfbjMQdlhzRg4iBPeU6XRMqYdvyLF13+k/BuUSvIEDoATp36sbJ9/4nx6yFdy1iwWE9H3PaODfastu5YSRS/1dUk16U+FEYuDhcHkUulXhYaPnZiUdCADi7wDV2gyIhHsmoblomTSm9iEbvdsEdXCUQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=minervasys.tech smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=U/xOuPKjtqCJRT1ppKqXNV5kBGA3BCkrVZqPvDGj4uM=;
 b=tYAUOx3VRJVOsVJBPbwkXJmt+vhDel6Lo3BAz1E8arLgn/2ROqnTknCj5cTh6Obj7ycE+S+LFPdnZ7543g1OJiz30XqYaKuhsYZVckoQ1x0rF5diWL0JsRTcF7a78sw6b1rwhj0xIgIKWO9JlAhX2OHx7LV4lT4gFzmkEx6r1Cw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <bb28c8a2-c2bd-4e1d-ad28-9bc632e3e474@amd.com>
Date: Wed, 14 Feb 2024 11:13:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 02/15] xen/arm: add initial support for LLC coloring on
 arm64
Content-Language: en-US
To: Carlo Nonato <carlo.nonato@minervasys.tech>,
	<xen-devel@lists.xenproject.org>
CC: <andrea.bastoni@minervasys.tech>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Marco Solieri <marco.solieri@minervasys.tech>
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-3-carlo.nonato@minervasys.tech>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240129171811.21382-3-carlo.nonato@minervasys.tech>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF000044A1:EE_|IA1PR12MB7615:EE_
X-MS-Office365-Filtering-Correlation-Id: afa9ee0f-e569-47c7-50de-08dc2d45a9ac
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	IYiqGcde8qfU0YIGXKH8pqfV23ILvwvultsvGwyKha+tO+pUgjBuASYReDZ2o9YgLK6E75t8Dg9GferqWHyyDKLTppCnUJTOw/oMq2JvsWx/pzUW56oPH/c9i/7FT9X2ugbTYXqwE+tmQHan8j4pQRKGqd23aBzsRCGau0gdoiz6bkiq0be3MydXF+TfJ0W9QcLs9xIFV2vjvsAal4ue1xJcUj1vUTLK4gPFhK/fe7dcBeuKL8Tl9gsgwJPpKMV5W7XLn+O8rAeOQOfIrNdl6ku5o69g9+43lATYA+tM3aeH69rgLKbm+JJmcrpOVaDmjEfpP70nUxoWFHrBXtsyq55BLG8aPmMADobdKF3OMOpkAZsstIxWOd/pXgPmTyzIsYF9r+YqVe5Hb56KixD6vFFAJ6UPrdNcdHXW9KUoHNTfGT4XCVpvC4BBjMa7CSBUdA95Grp7kZ/uA3MrSK8V1M4l9hvJvSzSYRfN7FLsM6hteGwkrqWwG686UV5OVP8p/V0inGNauTDUAYWUOTmE4WssDcNejzQfilZ5FdyVKfGuTJNjEeH0gkOjNPfhGkX9PZyQloum35S3xT4g4a86DuvKbl96gSdP4xd3k2+B4s4=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(396003)(39860400002)(376002)(346002)(230922051799003)(186009)(64100799003)(1800799012)(451199024)(82310400011)(40470700004)(36840700001)(46966006)(31686004)(81166007)(82740400003)(356005)(83380400001)(31696002)(41300700001)(86362001)(70206006)(7416002)(5660300002)(70586007)(336012)(44832011)(54906003)(8936002)(8676002)(2906002)(426003)(4326008)(110136005)(26005)(316002)(16576012)(36756003)(2616005)(478600001)(53546011);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 10:14:00.1934
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: afa9ee0f-e569-47c7-50de-08dc2d45a9ac
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF000044A1.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7615

Hi Carlo,

On 29/01/2024 18:17, Carlo Nonato wrote:
> 
> 
> LLC coloring needs to know the last level cache layout in order to make the
> best use of it. This can be probed by inspecting the CLIDR_EL1 register,
> so the Last Level is defined as the last level visible by this register.
> Note that this excludes system caches in some platforms.
> 
> Static memory allocation and cache coloring are incompatible because static
> memory can't be guaranteed to use only colors assigned to the domain.
> Panic during DomUs creation when both are enabled.
> 
> Based on original work from: Luca Miccio <lucmiccio@gmail.com>
> 
> Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
> Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
> ---
> v6:
> - get_llc_way_size() now checks for at least separate I/D caches
> v5:
> - used - instead of _ for filenames
> - moved static-mem check in this patch
> - moved dom0 colors parsing in next patch
> - moved color allocation and configuration in next patch
> - moved check_colors() in next patch
> - colors are now printed in short form
> v4:
> - added "llc-coloring" cmdline option for the boot-time switch
> - dom0 colors are now checked during domain init as for any other domain
> - fixed processor.h masks bit width
> - check for overflow in parse_color_config()
> - check_colors() now checks also that colors are sorted and unique
> ---
>  docs/misc/cache-coloring.rst         | 11 ++++
>  xen/arch/arm/Kconfig                 |  1 +
>  xen/arch/arm/Makefile                |  1 +
>  xen/arch/arm/dom0less-build.c        |  6 +++
>  xen/arch/arm/include/asm/processor.h | 16 ++++++
>  xen/arch/arm/llc-coloring.c          | 75 ++++++++++++++++++++++++++++
>  xen/arch/arm/setup.c                 |  3 ++
>  7 files changed, 113 insertions(+)
>  create mode 100644 xen/arch/arm/llc-coloring.c
> 
> diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rst
> index 9fe01e99e1..0535b5c656 100644
> --- a/docs/misc/cache-coloring.rst
> +++ b/docs/misc/cache-coloring.rst
> @@ -85,3 +85,14 @@ More specific documentation is available at `docs/misc/xen-command-line.pandoc`.
>  +----------------------+-------------------------------+
>  | ``llc-way-size``     | set the LLC way size          |
>  +----------------------+-------------------------------+
> +
> +Known issues and limitations
> +****************************
> +
> +"xen,static-mem" isn't supported when coloring is enabled
> +#########################################################
> +
> +In the domain configuration, "xen,static-mem" allows memory to be statically
> +allocated to the domain. This isn't possible when LLC coloring is enabled,
> +because that memory can't be guaranteed to use only colors assigned to the
> +domain.
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index 50e9bfae1a..55143f86a9 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -8,6 +8,7 @@ config ARM_64
>         depends on !ARM_32
>         select 64BIT
>         select HAS_FAST_MULTIPLY
> +       select HAS_LLC_COLORING
> 
>  config ARM
>         def_bool y
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 33c677672f..c9a1cd298d 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -37,6 +37,7 @@ obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
>  obj-y += irq.o
>  obj-y += kernel.init.o
>  obj-$(CONFIG_LIVEPATCH) += livepatch.o
> +obj-$(CONFIG_LLC_COLORING) += llc-coloring.o
>  obj-y += mem_access.o
>  obj-y += mm.o
>  obj-y += monitor.o
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index fb63ec6fd1..1142f7f74a 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -5,6 +5,7 @@
>  #include <xen/grant_table.h>
>  #include <xen/iocap.h>
>  #include <xen/libfdt/libfdt.h>
> +#include <xen/llc-coloring.h>
>  #include <xen/sched.h>
>  #include <xen/serial.h>
>  #include <xen/sizes.h>
> @@ -879,7 +880,12 @@ void __init create_domUs(void)
>              panic("No more domain IDs available\n");
> 
>          if ( dt_find_property(node, "xen,static-mem", NULL) )
> +        {
> +            if ( llc_coloring_enabled )
> +                panic("LLC coloring and static memory are incompatible\n");
> +
>              flags |= CDF_staticmem;
> +        }
> 
>          if ( dt_property_read_bool(node, "direct-map") )
>          {
> diff --git a/xen/arch/arm/include/asm/processor.h b/xen/arch/arm/include/asm/processor.h
> index 8e02410465..336933ee62 100644
> --- a/xen/arch/arm/include/asm/processor.h
> +++ b/xen/arch/arm/include/asm/processor.h
> @@ -18,6 +18,22 @@
>  #define CTR_IDC_SHIFT       28
>  #define CTR_DIC_SHIFT       29
> 
> +/* CCSIDR Current Cache Size ID Register */
> +#define CCSIDR_LINESIZE_MASK            _AC(0x7, ULL)
Why ULL and not UL? ccsidr is of register_t type

> +#define CCSIDR_NUMSETS_SHIFT            13
> +#define CCSIDR_NUMSETS_MASK             _AC(0x3fff, ULL)
> +#define CCSIDR_NUMSETS_SHIFT_FEAT_CCIDX 32
> +#define CCSIDR_NUMSETS_MASK_FEAT_CCIDX  _AC(0xffffff, ULL)
> +
> +/* CSSELR Cache Size Selection Register */
> +#define CSSELR_LEVEL_MASK  _AC(0x7, UL)
> +#define CSSELR_LEVEL_SHIFT 1
> +
> +/* CLIDR Cache Level ID Register */
> +#define CLIDR_CTYPEn_SHIFT(n) (3 * (n - 1))
n should be within parentheses

> +#define CLIDR_CTYPEn_MASK     _AC(0x7, UL)
> +#define CLIDR_CTYPEn_LEVELS   7
> +
>  #define ICACHE_POLICY_VPIPT  0
>  #define ICACHE_POLICY_AIVIVT 1
>  #define ICACHE_POLICY_VIPT   2
> diff --git a/xen/arch/arm/llc-coloring.c b/xen/arch/arm/llc-coloring.c
> new file mode 100644
> index 0000000000..eee1e80e2d
> --- /dev/null
> +++ b/xen/arch/arm/llc-coloring.c
> @@ -0,0 +1,75 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Last Level Cache (LLC) coloring support for ARM
> + *
> + * Copyright (C) 2022 Xilinx Inc.
> + */
> +#include <xen/llc-coloring.h>
> +#include <xen/types.h>
> +
> +#include <asm/processor.h>
> +#include <asm/sysregs.h>
> +
> +/* Return the LLC way size by probing the hardware */
> +unsigned int __init get_llc_way_size(void)
> +{
> +    register_t ccsidr_el1;
> +    register_t clidr_el1 = READ_SYSREG(CLIDR_EL1);
> +    register_t csselr_el1 = READ_SYSREG(CSSELR_EL1);
> +    register_t id_aa64mmfr2_el1 = READ_SYSREG(ID_AA64MMFR2_EL1);
> +    uint32_t ccsidr_numsets_shift = CCSIDR_NUMSETS_SHIFT;
> +    uint32_t ccsidr_numsets_mask = CCSIDR_NUMSETS_MASK;
> +    unsigned int n, line_size, num_sets;
> +
> +    for ( n = CLIDR_CTYPEn_LEVELS; n != 0; n-- )
> +    {
> +        uint8_t ctype_n = (clidr_el1 >> CLIDR_CTYPEn_SHIFT(n)) &
> +                          CLIDR_CTYPEn_MASK;
> +
> +        /* At least separate I/D caches (see Arm ARM DDI 0487H.a D13.2.27) */
I'm a bit confused here given that this comment does not reflect the line below (also please refer to the latest spec).
Since 0b011 is "Separate instruction and data caches" you would break only for Unified cache.
That said, we care about last level cache that is visible to SW and I'm not aware of any Arm CPU where L2,L3 is not unified.

> +        if ( ctype_n > 0b011 )
> +            break;
> +    }
> +
> +    if ( n == 0 )
> +        return 0;
> +
> +    WRITE_SYSREG((n - 1) << CSSELR_LEVEL_SHIFT, CSSELR_EL1);
> +
no need for this empty line

> +    isb();
> +
> +    ccsidr_el1 = READ_SYSREG(CCSIDR_EL1);
> +
> +    /* Arm ARM: (Log2(Number of bytes in cache line)) - 4 */
> +    line_size = 1U << ((ccsidr_el1 & CCSIDR_LINESIZE_MASK) + 4);
> +
> +    /* If FEAT_CCIDX is enabled, CCSIDR_EL1 has a different bit layout */
> +    if ( (id_aa64mmfr2_el1 >> ID_AA64MMFR2_CCIDX_SHIFT) & 0x7 )
> +    {
> +        ccsidr_numsets_shift = CCSIDR_NUMSETS_SHIFT_FEAT_CCIDX;
> +        ccsidr_numsets_mask = CCSIDR_NUMSETS_MASK_FEAT_CCIDX;
> +    }
empty line here please

> +    /* Arm ARM: (Number of sets in cache) - 1 */
> +    num_sets = ((ccsidr_el1 >> ccsidr_numsets_shift) & ccsidr_numsets_mask) + 1;
> +
> +    printk(XENLOG_INFO "LLC found: L%u (line size: %u bytes, sets num: %u)\n",
> +           n, line_size, num_sets);
> +
> +    /* Restore value in CSSELR_EL1 */
> +    WRITE_SYSREG(csselr_el1, CSSELR_EL1);
> +    isb();
> +
> +    return line_size * num_sets;
> +}
> +
> +void __init arch_llc_coloring_init(void) {}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 59dd9bb25a..14cb023783 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -12,6 +12,7 @@
>  #include <xen/device_tree.h>
>  #include <xen/domain_page.h>
>  #include <xen/grant_table.h>
> +#include <xen/llc-coloring.h>
>  #include <xen/types.h>
>  #include <xen/string.h>
>  #include <xen/serial.h>
> @@ -746,6 +747,8 @@ void asmlinkage __init start_xen(unsigned long boot_phys_offset,
>      printk("Command line: %s\n", cmdline);
>      cmdline_parse(cmdline);
> 
> +    llc_coloring_init();
I think a check with llc_coloring_enabled is missing, given there is none in llc_coloring_init

~Michal



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 10:15:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 10:15:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680359.1058499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCJ1-0005sH-Vi; Wed, 14 Feb 2024 10:15:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680359.1058499; Wed, 14 Feb 2024 10:15:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCJ1-0005sA-T1; Wed, 14 Feb 2024 10:15:55 +0000
Received: by outflank-mailman (input) for mailman id 680359;
 Wed, 14 Feb 2024 10:15:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raCJ0-0005s0-Fx
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 10:15:54 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 08e6ee03-cb22-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 11:15:53 +0100 (CET)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2d094bc2244so68847641fa.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 02:15:53 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i7-20020a05600c290700b00410add3af79sm1463796wmd.23.2024.02.14.02.15.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 02:15:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08e6ee03-cb22-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707905753; x=1708510553; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=4KlJ7kp1NtwjByIFJtrdaNCRwIi1KERZlr5Saiakv8g=;
        b=ZNjIjfGaRjur/98ZY6sray/PP5V7yNkEonKkygX9nYv0w4FXt2vKSqYCGC9blBcJT6
         umVK2+pmr6Sk5XQZANmTANDJ98m82y5B+UUdPJ9ZyMm7KmrQefd1QLSTvLWiRnu49c7R
         3mWqDo1otRUwgJDSnzN5/Vc/xlij7+9D4P3kV+oHYeGlNZyfh5NumwdP25cXy0PqLq4a
         y06q5qpY8dvYHjJY+l2jAQTDFM9H2p1u3hsvj9cjoMDbvESrAJbd3EFj3an5IvAaYg6J
         y8Xdw1y2MfwO0CRcZYk0du+Im7PNNer9IwdUz94pVnxh24Kfv6GykSia7bzsTRb2pDwp
         tj/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707905753; x=1708510553;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4KlJ7kp1NtwjByIFJtrdaNCRwIi1KERZlr5Saiakv8g=;
        b=Xc6QFhC0BqUxPrVVIk3k5imMvtcfklds2Qkg8mYOo9WQdcbbyWc0uIEo617rEdc1M4
         2/Trt7REo0Wp1JksIvCZTtqRy0k2hQB+Hd6sINpXoUCRgXEsBSYXzP649Hgrgy2oPlMS
         5Q6a8lB9ggms0BYpW0otOnIrt6TwY926gBV3thAwMQcWPHqN+Uv17unuooZZyvDJNijz
         FavQKycZyFuv0Hvf877hioo6mYMPePpZ/NkhwK0oSoSYPXTGz0l0mVHxlX4QNgHL+lMG
         zWcFMEtzRD989NxLjQ1nJ4daogwR6Plw9Gcw3ZH05/X2MjEpD2gadQRT1/cjJiByxB66
         GRMA==
X-Gm-Message-State: AOJu0YwppzWKBTehPWyokQO4yaSbNIeOuNYPwscdUdI4XO2E7k8c/+Ne
	FWBzQzSSf3EgMYqQTpcl6p/XFKPcbGgu86TSgxGE4q0cK8OYxFGSvZ7WjVNdr83Ak6c1jYVIte0
	=
X-Google-Smtp-Source: AGHT+IEcNpbWnQmLEF+7pKuEWC1eEnPFDHeiiwaDEdpI8BGwIlrRg6zHzCD+YoakY86fi/XGTyMHdA==
X-Received: by 2002:a05:651c:2112:b0:2d1:1d21:147f with SMTP id a18-20020a05651c211200b002d11d21147fmr843095ljq.39.1707905752982;
        Wed, 14 Feb 2024 02:15:52 -0800 (PST)
Message-ID: <549909b7-e34c-4a5c-aa21-9892a1724042@suse.com>
Date: Wed, 14 Feb 2024 11:15:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2] x86: amend 'n' debug-key output with SMI count
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... if available only, of course.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I don't really like issuing an IPI (and having another cf_check
function) here, yet then again this is issued only when the debug key
is actually used, and given how simple the handling function is
(including that it doesn't use its parameter) it also looks difficult
to abuse.
---
v2: Actually read each CPU's SMI count in do_nmi_stats().

--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -407,9 +407,15 @@ void __init early_cpu_init(bool verbose)
 		paddr_bits -= (ebx >> 6) & 0x3f;
 	}
 
-	if (!(c->x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON)))
+	if (!(c->x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON))) {
+		uint64_t smi_count;
+
 		park_offline_cpus = opt_mce;
 
+		if (!verbose && !rdmsr_safe(MSR_SMI_COUNT, smi_count))
+			setup_force_cpu_cap(X86_FEATURE_SMI_COUNT);
+	}
+
 	initialize_cpu_data(0);
 }
 
--- a/xen/arch/x86/include/asm/cpufeatures.h
+++ b/xen/arch/x86/include/asm/cpufeatures.h
@@ -24,7 +24,7 @@ XEN_CPUFEATURE(APERFMPERF,        X86_SY
 XEN_CPUFEATURE(MFENCE_RDTSC,      X86_SYNTH( 9)) /* MFENCE synchronizes RDTSC */
 XEN_CPUFEATURE(XEN_SMEP,          X86_SYNTH(10)) /* SMEP gets used by Xen itself */
 XEN_CPUFEATURE(XEN_SMAP,          X86_SYNTH(11)) /* SMAP gets used by Xen itself */
-/* Bit 12 unused. */
+XEN_CPUFEATURE(SMI_COUNT,         X86_SYNTH(12)) /* MSR_SMI_COUNT exists */
 XEN_CPUFEATURE(IND_THUNK_LFENCE,  X86_SYNTH(13)) /* Use IND_THUNK_LFENCE */
 XEN_CPUFEATURE(IND_THUNK_JMP,     X86_SYNTH(14)) /* Use IND_THUNK_JMP */
 XEN_CPUFEATURE(SC_NO_BRANCH_HARDEN, X86_SYNTH(15)) /* (Disable) Conditional branch hardening */
--- a/xen/arch/x86/include/asm/msr-index.h
+++ b/xen/arch/x86/include/asm/msr-index.h
@@ -28,6 +28,8 @@
 #define  TEST_CTRL_SPLITLOCK_DETECT         (_AC(1, ULL) << 29)
 #define  TEST_CTRL_SPLITLOCK_DISABLE        (_AC(1, ULL) << 31)
 
+#define MSR_SMI_COUNT                       0x00000034
+
 #define MSR_INTEL_CORE_THREAD_COUNT         0x00000035
 #define  MSR_CTC_THREAD_MASK                0x0000ffff
 #define  MSR_CTC_CORE_MASK                  _AC(0xffff0000, U)
--- a/xen/arch/x86/nmi.c
+++ b/xen/arch/x86/nmi.c
@@ -585,15 +585,34 @@ static void cf_check do_nmi_trigger(unsi
     self_nmi();
 }
 
+static DEFINE_PER_CPU(unsigned int, smi_count);
+
+static void cf_check read_smi_count(void *unused)
+{
+    unsigned int dummy;
+
+    rdmsr(MSR_SMI_COUNT, this_cpu(smi_count), dummy);
+}
+
 static void cf_check do_nmi_stats(unsigned char key)
 {
     const struct vcpu *v;
     unsigned int cpu;
     bool pend, mask;
 
-    printk("CPU\tNMI\n");
+    printk("CPU\tNMI%s\n", boot_cpu_has(X86_FEATURE_SMI_COUNT) ? "\tSMI" : "");
+
+    if ( boot_cpu_has(X86_FEATURE_SMI_COUNT) )
+        on_each_cpu(read_smi_count, NULL, 1);
+
     for_each_online_cpu ( cpu )
-        printk("%3u\t%3u\n", cpu, per_cpu(nmi_count, cpu));
+    {
+        printk("%3u\t%3u", cpu, per_cpu(nmi_count, cpu));
+        if ( boot_cpu_has(X86_FEATURE_SMI_COUNT) )
+            printk("\t%3u\n", per_cpu(smi_count, cpu));
+        else
+            printk("\n");
+    }
 
     if ( !hardware_domain || !(v = domain_vcpu(hardware_domain, 0)) )
         return;


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 10:18:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 10:18:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680365.1058510 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCLP-0006Sd-CU; Wed, 14 Feb 2024 10:18:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680365.1058510; Wed, 14 Feb 2024 10:18:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCLP-0006SW-8s; Wed, 14 Feb 2024 10:18:23 +0000
Received: by outflank-mailman (input) for mailman id 680365;
 Wed, 14 Feb 2024 10:18:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=bjrt=JX=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raCLN-0006SQ-QI
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 10:18:21 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6010f79e-cb22-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 11:18:19 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 41AFD1F7F1;
 Wed, 14 Feb 2024 10:18:19 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D109813A72;
 Wed, 14 Feb 2024 10:18:18 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id QlrgMGqTzGUxRwAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 14 Feb 2024 10:18:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6010f79e-cb22-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707905899; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=8/6JbNae1pkhQSK0y4p7SlNAiChOdbOk1ALjnpF+Zsk=;
	b=qPHYVAo3riRAMejZ+0nR6aKtekwcMZWGlBYrxDxX3uUJQHKclFHbsOjf/qv4twdffdXmKr
	IL9eFBtZ9W4DqEJoDUIK4dOljcB8DNvgVY9zpBOahYF4DIWHH2phvtWZPVtcm8Xlftv7lB
	clAFwbinMNUhzGM0NmdtYqCo83uulLs=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707905899; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=8/6JbNae1pkhQSK0y4p7SlNAiChOdbOk1ALjnpF+Zsk=;
	b=qPHYVAo3riRAMejZ+0nR6aKtekwcMZWGlBYrxDxX3uUJQHKclFHbsOjf/qv4twdffdXmKr
	IL9eFBtZ9W4DqEJoDUIK4dOljcB8DNvgVY9zpBOahYF4DIWHH2phvtWZPVtcm8Xlftv7lB
	clAFwbinMNUhzGM0NmdtYqCo83uulLs=
Message-ID: <c6af9dd1-01d4-48a2-86b9-ccb2967136f7@suse.com>
Date: Wed, 14 Feb 2024 11:18:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 15/22] tools/libs/light: add backend type for 9pfs PV
 devices
Content-Language: en-US
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, George Dunlap <george.dunlap@citrix.com>,
 Nick Rosbrook <rosbrookn@gmail.com>, Wei Liu <wl@xen.org>,
 Jason Andryuk <jandryuk@gmail.com>, George Dunlap <george.dunlap@cloud.com>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-16-jgross@suse.com>
 <5e6893dd-5280-4a14-8d00-4ed76e8c8e98@perard>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <5e6893dd-5280-4a14-8d00-4ed76e8c8e98@perard>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-1.48 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BAYES_HAM(-1.98)[94.92%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.59)[subject];
	 FREEMAIL_CC(0.00)[lists.xenproject.org,citrix.com,gmail.com,xen.org,cloud.com];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: -1.48

On 13.02.24 19:03, Anthony PERARD wrote:
> On Thu, Feb 08, 2024 at 05:55:39PM +0100, Juergen Gross wrote:
>> +struct libxl__aop9_state {
>> +    libxl__spawn_state spawn;
>> +    libxl__ao_device *aodev;
>> +    libxl_device_p9 p9;
>> +    uint32_t domid;
>> +    void (*callback)(libxl__egc *, libxl__aop9_state *, int);
> 
> This "callback" is never used, right?

Why do you think so?

In xen9pfsd_spawn() it is used:

   aop9->callback = xen9pfsd_spawn_outcome;

> 
>> diff --git a/tools/libs/light/libxl_types_internal.idl b/tools/libs/light/libxl_types_internal.idl
>> index e24288f1a5..39da71cef5 100644
>> --- a/tools/libs/light/libxl_types_internal.idl
>> +++ b/tools/libs/light/libxl_types_internal.idl
>> @@ -34,6 +34,7 @@ libxl__device_kind = Enumeration("device_kind", [
>>       (16, "VINPUT"),
>>       (17, "VIRTIO_DISK"),
>>       (18, "VIRTIO"),
>> +    (19, "XEN_9PFS"),
> 
> That's going to need to be rebased ;-).

Yes.


Juergen


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 10:36:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 10:36:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680545.1058520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCcm-0003Am-Qc; Wed, 14 Feb 2024 10:36:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680545.1058520; Wed, 14 Feb 2024 10:36:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raCcm-0003Af-Mr; Wed, 14 Feb 2024 10:36:20 +0000
Received: by outflank-mailman (input) for mailman id 680545;
 Wed, 14 Feb 2024 10:36:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=luzD=JX=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1raCcl-0003AZ-4E
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 10:36:19 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e28012ce-cb24-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 11:36:17 +0100 (CET)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-561f8b8c058so1232302a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 02:36:17 -0800 (PST)
Received: from localhost.localdomain (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 vh2-20020a170907d38200b00a3d08f3283fsm1437202ejc.104.2024.02.14.02.36.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 02:36:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e28012ce-cb24-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707906977; x=1708511777; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=FosWLS1dcg2XMXFMITky5HoMZurHyMN4qiTpqnCFJHU=;
        b=BXu+Ra4YAqWuoCwXIYS6e0dQxpLmf8dRZ6GTl7johYMv68xeBL5mJEPXl93XvkMpy0
         QBTxrqkZL0vtMeQZomnOAe2Tt7hhmUw+m3PUo1nJItj5PfP6pWtehmlFsbjJVxijD5AQ
         Xb1gfXx5CkMqTv+Z6KcWcjxbDPq+WwAl2m3xc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707906977; x=1708511777;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=FosWLS1dcg2XMXFMITky5HoMZurHyMN4qiTpqnCFJHU=;
        b=WEh2//GW7o+vs9zxBJ3/0e2XWbrLZy3bo312zFdmJ2az2EpwiaNa6OR9urW4ApJbJ9
         x3qlP+dBU02ABKYbIX9wgQ1Z/B013NajZWs0hV47Lx+5x2xIEMbRejCiyDGK2rJv6Szv
         KxpN7PfBlLcGPzNorlpr744MMHeJmploJ8c0kvzaBDJN/qUsE0aafnalhgRGc7ZF68Mh
         6arO205fH1+vURlukNVEZao4yZVD0U1quGB3Z7L/qPMXBplGxfk7z/hH4daugGX6Uk9X
         VwzziG/JixvqZjGrgvd8iLlEUGPqTAOTwdGcAbocEhOlwG6hcFs0r1nI2xqOiTFkH29j
         JJ+g==
X-Gm-Message-State: AOJu0Yy0scIae/1EDXrjistaIex/MuN2DS1wTScqjTrT4YkrSAuahLh3
	bwwof6X04LJaQcaCq25qqUfCT+Ec3DQRQ+a/+svrygFf8kJyGJQhurCAoEWVftVQNJD37tyaFbs
	smkpdIQ==
X-Google-Smtp-Source: AGHT+IFYL14JZHJiAjJ54j01NpkKVY3G50fkLJmaKJUjKyX80A4jCZBJ9aXXNLAX90ypZ/+tW4A6kw==
X-Received: by 2002:a17:906:c10f:b0:a38:107a:94e9 with SMTP id do15-20020a170906c10f00b00a38107a94e9mr1683054ejc.2.1707906976664;
        Wed, 14 Feb 2024 02:36:16 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXEiAW5NqTmW2pUYsOgjJuWfhOp1l4pUApWWsxWNtt1Twn/S6i5fWLkcbwjUKZD8zHlJjcCuo3bEqCCj4B8ysf0pkxOgdzoh1ObgytPJ/GAotxDWAI2+SPDoO68y/dOP7na0NSLMcJVb0HraFRJgGymovFYaT7bWIQiDBMyGsw=
From: Frediano Ziglio <frediano.ziglio@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: Frediano Ziglio <frediano.ziglio@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2] Reduce assembly code size of exception entry points
Date: Wed, 14 Feb 2024 10:35:58 +0000
Message-Id: <20240214103558.13655-1-frediano.ziglio@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

We just pushed a 8-bytes zero and exception constants are
small so we can just write a single byte saving 3 bytes for
instruction.
With ENDBR64 this reduces the size of many entry points from 32 to
16 bytes (due to alignment).
Similar code is already used in autogen_stubs.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
--
v2:
- added missing entry points;
- add mention to autogen_stubs code, as suggested.
---
 xen/arch/x86/x86_64/entry.S | 40 ++++++++++++++++++-------------------
 1 file changed, 20 insertions(+), 20 deletions(-)

diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index ecdd6e5b47..a28a0d4044 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -389,7 +389,7 @@ FUNC(entry_int80)
         ENDBR64
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         pushq $0
-        movl  $0x80, 4(%rsp)
+        movb  $0x80, 4(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -653,7 +653,7 @@ END(ret_from_intr)
         .section .init.text, "ax", @progbits
 FUNC(early_page_fault)
         ENDBR64
-        movl  $X86_EXC_PF, 4(%rsp)
+        movb  $X86_EXC_PF, 4(%rsp)
         SAVE_ALL
         movq  %rsp, %rdi
         call  do_early_page_fault
@@ -898,105 +898,105 @@ END(handle_exception)
 FUNC(entry_DE)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_DE, 4(%rsp)
+        movb  $X86_EXC_DE, 4(%rsp)
         jmp   handle_exception
 END(entry_DE)
 
 FUNC(entry_MF)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_MF, 4(%rsp)
+        movb  $X86_EXC_MF, 4(%rsp)
         jmp   handle_exception
 END(entry_MF)
 
 FUNC(entry_XM)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_XM, 4(%rsp)
+        movb  $X86_EXC_XM, 4(%rsp)
         jmp   handle_exception
 END(entry_XM)
 
 FUNC(entry_NM)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_NM, 4(%rsp)
+        movb  $X86_EXC_NM, 4(%rsp)
         jmp   handle_exception
 END(entry_NM)
 
 FUNC(entry_DB)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_DB, 4(%rsp)
+        movb  $X86_EXC_DB, 4(%rsp)
         jmp   handle_ist_exception
 END(entry_DB)
 
 FUNC(entry_BP)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_BP, 4(%rsp)
+        movb  $X86_EXC_BP, 4(%rsp)
         jmp   handle_exception
 END(entry_BP)
 
 FUNC(entry_OF)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_OF, 4(%rsp)
+        movb  $X86_EXC_OF, 4(%rsp)
         jmp   handle_exception
 END(entry_OF)
 
 FUNC(entry_BR)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_BR, 4(%rsp)
+        movb  $X86_EXC_BR, 4(%rsp)
         jmp   handle_exception
 END(entry_BR)
 
 FUNC(entry_UD)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_UD, 4(%rsp)
+        movb  $X86_EXC_UD, 4(%rsp)
         jmp   handle_exception
 END(entry_UD)
 
 FUNC(entry_TS)
         ENDBR64
-        movl  $X86_EXC_TS, 4(%rsp)
+        movb  $X86_EXC_TS, 4(%rsp)
         jmp   handle_exception
 END(entry_TS)
 
 FUNC(entry_NP)
         ENDBR64
-        movl  $X86_EXC_NP, 4(%rsp)
+        movb  $X86_EXC_NP, 4(%rsp)
         jmp   handle_exception
 END(entry_NP)
 
 FUNC(entry_SS)
         ENDBR64
-        movl  $X86_EXC_SS, 4(%rsp)
+        movb  $X86_EXC_SS, 4(%rsp)
         jmp   handle_exception
 END(entry_SS)
 
 FUNC(entry_GP)
         ENDBR64
-        movl  $X86_EXC_GP, 4(%rsp)
+        movb  $X86_EXC_GP, 4(%rsp)
         jmp   handle_exception
 END(entry_GP)
 
 FUNC(entry_AC)
         ENDBR64
-        movl  $X86_EXC_AC, 4(%rsp)
+        movb  $X86_EXC_AC, 4(%rsp)
         jmp   handle_exception
 END(entry_AC)
 
 FUNC(entry_CP)
         ENDBR64
-        movl  $X86_EXC_CP, 4(%rsp)
+        movb  $X86_EXC_CP, 4(%rsp)
         jmp   handle_exception
 END(entry_CP)
 
 FUNC(entry_DF)
         ENDBR64
-        movl  $X86_EXC_DF, 4(%rsp)
+        movb  $X86_EXC_DF, 4(%rsp)
         /* Set AC to reduce chance of further SMAP faults */
         ALTERNATIVE "", stac, X86_FEATURE_XEN_SMAP
         SAVE_ALL
@@ -1022,7 +1022,7 @@ END(entry_DF)
 FUNC(entry_NMI)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_NMI, 4(%rsp)
+        movb  $X86_EXC_NMI, 4(%rsp)
 END(entry_NMI)
 
 FUNC(handle_ist_exception)
@@ -1158,7 +1158,7 @@ END(handle_ist_exception)
 FUNC(entry_MC)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_MC, 4(%rsp)
+        movb  $X86_EXC_MC, 4(%rsp)
         jmp   handle_ist_exception
 END(entry_MC)
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:06:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:06:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680550.1058530 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raD5m-00015X-2r; Wed, 14 Feb 2024 11:06:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680550.1058530; Wed, 14 Feb 2024 11:06:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raD5m-00015Q-01; Wed, 14 Feb 2024 11:06:18 +0000
Received: by outflank-mailman (input) for mailman id 680550;
 Wed, 14 Feb 2024 11:06:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raD5k-00015K-Fw
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:06:16 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 11ef4286-cb29-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 12:06:15 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a2d7e2e7fe0so120082166b.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:06:15 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 sn24-20020a170906629800b00a3d192cd673sm1176508ejc.23.2024.02.14.03.06.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 03:06:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11ef4286-cb29-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707908774; x=1708513574; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=c0cfZ3TFaEvZA/ZDs4WJzdUjn9E8fJmzc/CjQGrqM9U=;
        b=EjIS3rVEeKqGPn5qs4k2spfERI/IdEFxCL0qlHUQPGMNq4Bk2RZ2JYCF9x5QcOmxMe
         HvOoZFA+zUkFEY32TV0l+dSDiUN8yDgWcCR5tpUqENaS61gLcgL7y3HJhKJluNc44MGK
         NKUyK9u2m7kY5vKljl7kQ5BAKxSFR2BbmWcTc0pic/CnoK+UHNejh/9aYTLg1jvvmhwW
         LBRaBN+vVdJsv0xGlGzN9GR0rc12oE4mveihjMjdlJA/+fIVhLfxqaTybzIpb+mxNfe9
         G2JgF84+sk1dCxMWVefDwzCYoyJBuIr+nPaXUHP0b0ojLZam7icgteiuOBVtbcu3VSWe
         nXgA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707908774; x=1708513574;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=c0cfZ3TFaEvZA/ZDs4WJzdUjn9E8fJmzc/CjQGrqM9U=;
        b=mpPjuu+YF/+IX4D7P8RYPefjOhpFehj8JN78hiPz2YNIPuB64WcU3uDfrg0oHJQt6q
         ZwqxMgm/Xva4pn1KDOR+eSBeAyTY191OmvINyodE0VXOrb2QqJTVE5LnxbaWqSy9QAbY
         XCc49ixCwsqgsjBspCP8n2KfxyQTPmFt66iNPeHVy2i/eNpG27XN+Ea2WSzF3zTMnt3W
         Ztb/E7wAnLvSFXyvm5ihaLRJ3FcIcRKMTE6QprG5u4euZOH+kOA0MhnlB4r7RXi3ddAf
         tkIqsAUawepliaI2/HZUA/mGdBEBzYNBNp0jGX2jHcMTi6ihjUvTMUPCHAGkxXuYjnCZ
         T3Pw==
X-Forwarded-Encrypted: i=1; AJvYcCV3+EK5XISs8HP0R1n7xvCd0eIg4ODAlcM4cGvDkVAhUYNHxnTpy/jM0vYVR3vmgoNqnDdsWPoAHpCfjyty6lZB1VC/hebxJNvWHzeiNAM=
X-Gm-Message-State: AOJu0YyQMK0Mz71gn4uI3H7QV7oRiCs34g8HNR8TUqU8nQ8QzNLtMtBO
	6A0Zl6wfOrQODQ58XQEag91Squ3z251GlnLkeOQ9ydrDAc8x7bSmTts621X9
X-Google-Smtp-Source: AGHT+IH+K7ErbMMneHIoG5Z/2ZxZplbe52w3+arJvZx3aaUz/6oPItHxP2Z2cg4zSWnYwERG+7+YpA==
X-Received: by 2002:a17:907:6d0c:b0:a3d:37a4:809d with SMTP id sa12-20020a1709076d0c00b00a3d37a4809dmr1819373ejc.7.1707908774249;
        Wed, 14 Feb 2024 03:06:14 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCU8pPfDepDBwwOJMrC623U/2BvCS6mQvqUQf81BppRwqu5dhEGrJ3meOlqQ9Jo4YPuoJa3vrz48oC53M62PKxIteHUhHfASNElY2NlVrs+mm7e+dmm5txZCvijN4508Z6Y1Ag3U/7Aomuy66rfeMlkA8Vp6B1s3LmfMz4QIA2e7anVBB9hOPCgtFmEfqguDzIq2ah1GzIW/qonvAXxYJM1uvBSx/Fy2W3I5ImA0WcHMNTTbloBzG9K0XNSlselUNc3ihXGeQpR1GqZNZcL6fGA2S8EKsvPWFOWKUiiKxDRQ7W1W+LWFv98=
Message-ID: <655eed7d1af08936ee3f42a0b88eeb2e197434e2.camel@gmail.com>
Subject: Re: [PATCH v4 09/30] xen/riscv: introduce bitops.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 12:06:12 +0100
In-Reply-To: <b13fd044-c6db-45ab-83d0-3d02221967ed@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <f76bd85f4b64a47c59c0b306ce425036819fa380.1707146506.git.oleksii.kurochko@gmail.com>
	 <b13fd044-c6db-45ab-83d0-3d02221967ed@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 16:58 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > --- /dev/null
> > +++ b/xen/arch/riscv/include/asm/bitops.h
> > @@ -0,0 +1,164 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/* Copyright (C) 2012 Regents of the University of California */
> > +
> > +#ifndef _ASM_RISCV_BITOPS_H
> > +#define _ASM_RISCV_BITOPS_H
> > +
> > +#include <asm/system.h>
> > +
> > +#include <asm-generic/bitops/bitops-bits.h>
>=20
> Especially with ...
>=20
> > +/* Based on linux/arch/include/linux/bits.h */
> > +
> > +#define BIT_MASK(nr)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (1UL <<=
 ((nr) % BITS_PER_LONG))
> > +#define BIT_WORD(nr)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ((nr) /=
 BITS_PER_LONG)
>=20
> ... these it's not entirely obvious why bitops-bits.h would be needed
> here.
They are needed as __test_and_op_bit_ord(), __op_bit_ord() macros are
used them, but probably it makes sense to drop BIT_MASK() and
BIT_WORD(), and just use BITOPS_MASK() and BITOPS_WORD() from asm-
generic/bitops-bits.h or re-define BITOPS_MASK() and BITOPS_WORD()
before inclusion of bitops-bits.h in the way as BIT_MASK and BIT_WORD
macros are defined now to be aligned with Linux.

>=20
> > +#define __set_bit(n,p)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 set_bit(n,p)
> > +#define __clear_bit(n,p)=C2=A0=C2=A0=C2=A0 clear_bit(n,p)
>=20
> Nit (as before?): Missing blanks after commas.
Thanks. I'll add blanks.

>=20
> > +/* Based on linux/arch/include/asm/bitops.h */
> > +
> > +#if ( BITS_PER_LONG =3D=3D 64 )
>=20
> Imo the parentheses here make things only harder to read.
I can drop them, this part was copied from Linux, so it was decided to
leave it as is.

>=20
> > +#define __AMO(op)=C2=A0=C2=A0 "amo" #op ".d"
> > +#elif ( BITS_PER_LONG =3D=3D 32 )
> > +#define __AMO(op)=C2=A0=C2=A0 "amo" #op ".w"
> > +#else
> > +#error "Unexpected BITS_PER_LONG"
> > +#endif
> > +
> > +#define __test_and_op_bit_ord(op, mod, nr, addr, ord)=C2=A0=C2=A0 \
>=20
> The revision log says __test_and_* were renamed. Same anomaly for
> __test_and_op_bit() then.
I'll double check the namings. Thanks.
>=20
> > +({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 unsigned long __res, __mask;=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
>=20
> Leftover leading underscores?
It is how it was defined in Linux, so I thought that I've to leave it
as it, but I am OK to rename this variables in next patch version.

>=20
> > +=C2=A0=C2=A0=C2=A0 __mask =3D BIT_MASK(nr);=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 __asm__ __volatile__ (=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __AMO(op) #ord " %0, %2, %1=
"=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "=3Dr" (__res), "+A" (add=
r[BIT_WORD(nr)])=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "r" (mod(__mask))=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "memory");=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 ((__res & __mask) !=3D 0);=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +})
> > +
> > +#define __op_bit_ord(op, mod, nr, addr, ord)=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 __asm__ __volatile__ (=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __AMO(op) #ord " zero, %1, =
%0"=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "+A" (addr[BIT_WORD(nr)])=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "r" (mod(BIT_MASK(nr)))=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "memory");
> > +
> > +#define __test_and_op_bit(op, mod, nr, addr)=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 __test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
> > +#define __op_bit(op, mod, nr, addr) \
> > +=C2=A0=C2=A0=C2=A0 __op_bit_ord(op, mod, nr, addr, )
> > +
> > +/* Bitmask modifiers */
> > +#define __NOP(x)=C2=A0=C2=A0=C2=A0 (x)
> > +#define __NOT(x)=C2=A0=C2=A0=C2=A0 (~(x))
>=20
> Here the (double) leading underscores are truly worrying: Simple
> names like this aren't impossible to be assigned meaninb by a
> compiler.
I am not really understand what is the difference for a compiler
between NOP and __NOP? Do you mean that the leading double underscores
(__) are often used to indicate that these macros are implementation-
specific and might be reserved for the compiler or the standard
library?

>=20
> > +/**
> > + * __test_and_set_bit - Set a bit and return its old value
> > + * @nr: Bit to set
> > + * @addr: Address to count from
> > + *
> > + * This operation may be reordered on other architectures than
> > x86.
> > + */
> > +static inline int test_and_set_bit(int nr, volatile void *p)
> > +{
> > +=C2=A0=C2=A0=C2=A0 volatile uint32_t *addr =3D p;
>=20
> With BIT_WORD() / BIT_MASK() being long-based, is the use of uint32_t
> here actually correct?
No, it is not correct. It seems to me it would be better to use
BITOPS_WORD(), BITOPS_MASK() and bitops_uint_t() and just redefine them
before inclusion of bitops-bit.h to be aligned with Linux
implementation.

>=20
> > +=C2=A0=C2=A0=C2=A0 return __test_and_op_bit(or, __NOP, nr, addr);
> > +}
> > +
> > +/**
> > + * __test_and_clear_bit - Clear a bit and return its old value
> > + * @nr: Bit to clear
> > + * @addr: Address to count from
> > + *
> > + * This operation can be reordered on other architectures other
> > than x86.
>=20
> Nit: double "other" (and I think it's the 1st one that wants
> dropping,
> not - as the earlier comment suggests - the 2nd one). Question is:
> Are
> the comments correct? Both resolve to something which is (also) at
> least a compiler barrier. Same concern also applies further down, to
> at least set_bit() and clear_bit().
It looks like comments aren't correct as operation inside is atomic,
also it implies compiler memory barrier. So the comments related to
'reordering' should be dropped.
I am not sure that I know why in Linux these comments were left.

>=20
> > + */
> > +static inline int test_and_clear_bit(int nr, volatile void *p)
> > +{
> > +=C2=A0=C2=A0=C2=A0 volatile uint32_t *addr =3D p;
> > +
> > +=C2=A0=C2=A0=C2=A0 return __test_and_op_bit(and, __NOT, nr, addr);
> > +}
> > +
> > +/**
> > + * set_bit - Atomically set a bit in memory
> > + * @nr: the bit to set
> > + * @addr: the address to start counting from
> > + *
> > + * Note: there are no guarantees that this function will not be
> > reordered
> > + * on non x86 architectures, so if you are writing portable code,
> > + * make sure not to rely on its reordering guarantees.
> > + *
> > + * Note that @nr may be almost arbitrarily large; this function is
> > not
> > + * restricted to acting on a single-word quantity.
> > + */
> > +static inline void set_bit(int nr, volatile void *p)
> > +{
> > +=C2=A0=C2=A0=C2=A0 volatile uint32_t *addr =3D p;
> > +
> > +=C2=A0=C2=A0=C2=A0 __op_bit(or, __NOP, nr, addr);
> > +}
> > +
> > +/**
> > + * clear_bit - Clears a bit in memory
> > + * @nr: Bit to clear
> > + * @addr: Address to start counting from
> > + *
> > + * Note: there are no guarantees that this function will not be
> > reordered
> > + * on non x86 architectures, so if you are writing portable code,
> > + * make sure not to rely on its reordering guarantees.
> > + */
> > +static inline void clear_bit(int nr, volatile void *p)
> > +{
> > +=C2=A0=C2=A0=C2=A0 volatile uint32_t *addr =3D p;
> > +
> > +=C2=A0=C2=A0=C2=A0 __op_bit(and, __NOT, nr, addr);
> > +}
> > +
> > +/**
> > + * test_and_change_bit - Change a bit and return its old value
>=20
> How come this one's different? I notice the comments are the same
> (and
> hence as confusing) in Linux; are you sure they're applicable there?
No, I am not sure. As I mentioned above, all this functions are atomic
and uses compiler memory barrier, so it looks like the comment for=20
clear_bit isn't really correct.
In case if all such functions are uses __test_and_op_bit_ord() and
__op_bit_ord() which are atomic and with compiler barrier, it seems
that we can just drop all such comments or even all the comments. I am
not sure that anyone is needed the comment as by default this function
is safe to use:
 * This operation is atomic and cannot be reordered.
 * It also implies a memory barrier.

>=20
> > + * @nr: Bit to change
> > + * @addr: Address to count from
> > + *
> > + * This operation is atomic and cannot be reordered.
> > + * It also implies a memory barrier.
> > + */
> > +static inline int test_and_change_bit(int nr, volatile unsigned
> > long *addr)
> > +{
> > +	return __test_and_op_bit(xor, __NOP, nr, addr);
> > +}
> > +
> > +#undef __test_and_op_bit
> > +#undef __op_bit
> > +#undef __NOP
> > +#undef __NOT
> > +#undef __AMO
> > +
> > +#include <asm-generic/bitops/generic-non-atomic.h>
> > +
> > +#define __test_and_set_bit generic___test_and_set_bit
> > +#define __test_and_clear_bit generic___test_and_clear_bit
> > +#define __test_and_change_bit generic___test_and_change_bit
> > +
> > +#include <asm-generic/bitops/fls.h>
> > +#include <asm-generic/bitops/flsl.h>
> > +#include <asm-generic/bitops/__ffs.h>
> > +#include <asm-generic/bitops/ffs.h>
> > +#include <asm-generic/bitops/ffsl.h>
> > +#include <asm-generic/bitops/ffz.h>
> > +#include <asm-generic/bitops/find-first-set-bit.h>
> > +#include <asm-generic/bitops/hweight.h>
> > +#include <asm-generic/bitops/test-bit.h>
>=20
> To be honest there's too much stuff being added here to asm-generic/,
> all in one go. I'll see about commenting on the remaining parts here,
> but I'd like to ask that you seriously consider splitting.
Would it be better to send it outside of this patch series? I can
create a separate patch series with a separate patch for each asm-
generic/bitops/*.h

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:06:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:06:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680552.1058539 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raD66-0001Su-EO; Wed, 14 Feb 2024 11:06:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680552.1058539; Wed, 14 Feb 2024 11:06:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raD66-0001Sn-Bo; Wed, 14 Feb 2024 11:06:38 +0000
Received: by outflank-mailman (input) for mailman id 680552;
 Wed, 14 Feb 2024 11:06:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raD65-00015K-QM
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:06:37 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1eb210d4-cb29-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 12:06:36 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-50eac018059so6955457e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:06:36 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 t18-20020ac243b2000000b0051023149df3sm1629611lfl.248.2024.02.14.03.06.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 03:06:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1eb210d4-cb29-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707908796; x=1708513596; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=PebPhg0dxWmlkc99eA6OqdfYUBB2ia8H+qi2Oe/OHd0=;
        b=UTtim5GGj794AU//ShgF7mVjJ3TuaOXMb5/ZazJtJOutbRONjpplJWGtdcr9cukRmL
         G/UhU4FyTfhU+VUDg9Y5cY2YQNIAxgWa6Cq1QtTXsuVzzWpoW/jX2EYPyi0gYMU/nTYA
         TmzGpsX/hSAbIG/zwPPDLn/gcJdcQrDMfS7iPyP0ENngDAR9exklSFmRzJFYCwaDGnZy
         w7cKR1q5z+G1N/ZyfvJSIWQGRNmhTRvf0Gx800MU1ipkGKPv7CMDNE47V/d4NvZ2/i3+
         zou13cCEcIWp6PBrfcMb3Jjm4IEC1EoIGS4f83Ubqy/GU9F6lCRl8Pkol/CKARrSxGHW
         X40Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707908796; x=1708513596;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PebPhg0dxWmlkc99eA6OqdfYUBB2ia8H+qi2Oe/OHd0=;
        b=rC4QI1OsJc90E9WEguFJOMAo6sFzWKlbnIY3VE61U8s+BQWKpbYBY44nr3e/2PBB5Y
         Qn4QtmeZiqg8rjG87TaCimqWdYQ46CxA9KT4bLxBiUEhR10NjXWGjoj+YTGMS3CfN6X8
         +Gc+zy74JL6xdLjFapMngt3khvOtgj5EQEtC7m0R/8I/qjrt/jQLiye93aN4N9EZDsBD
         CKyQa49hosgiXMVtIED6NWV3G35XynwVtjeQswEOYK6ofkFH8eAZxVKLcYwn5eQ9ND1q
         WqGd5XOJ+XOyugyaGQ424zw46W2iYEQ2L4ZnULTy93YuZUUIbWcdKAvoi5GuVVQTSJlr
         +xGw==
X-Forwarded-Encrypted: i=1; AJvYcCUP5az+BkbZSVoM65yxh6zJ7669i5nPe6gDnFVYD+2ZnehxRR3R77sUh6khVVrbC15C6GaHUvqCXVnFsnk/xe/brXa7oeNNRprJQkZb3FE=
X-Gm-Message-State: AOJu0Yz7435oj1H+WglI/TXwqWbNTajpg1/YBuazylDt/tIAHi4B+eg+
	g+WauuuX4pcYTwAQGVsxHUBT9sX3KwQ8hocqiXCx+1rYSMbajDlS
X-Google-Smtp-Source: AGHT+IH74VzRoZlUgSmmLukG8bUvEV6IAP8dCtDHosVeasK26VBMduoXjVpCnPkDmlj/X/6sKejQ/A==
X-Received: by 2002:a19:e001:0:b0:511:737d:6619 with SMTP id x1-20020a19e001000000b00511737d6619mr1613696lfg.0.1707908795714;
        Wed, 14 Feb 2024 03:06:35 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUGaM66fX+CjNFH6wGLj9gf4ayLknaGlkMh8pkrM2GnUMjWsuctX0Tg7CIwrgvH9fAsy/zNfiI+bjFjbhSTIYpk4giLkSJyK1XI2eAPRSzP08kevrxV6s5Sp9JO29nwOI8jO+2FkDBpfCl02bAdchWn4oF7Yeha0UfdTXX6zZZ4Y5U6Nr3X7TSFxctaWDa4HRXZYBsBBsSF9MGRd/vNN83dnA/BTM3ozkuNSXas9pHxXWIZxTHoJtmirXnK04KnlFx8EC+31jWVMmE3KEGSttuV5URXB7KGecBIxHuhp9WAnbABXI4hxZ0=
Message-ID: <16a84caf539d90f1712d931ae24c70c5368ed7cd.camel@gmail.com>
Subject: Re: [PATCH v4 11/30] xen/riscv: introduce smp.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 12:06:34 +0100
In-Reply-To: <3a74458a-809f-40e4-9c86-e58e339fe96c@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <4d0a8aba3ec17fa428f8d18f0008f863481b1f67.1707146506.git.oleksii.kurochko@gmail.com>
	 <3a74458a-809f-40e4-9c86-e58e339fe96c@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 16:13 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Acked-by: Jan Beulich <jbeulich@suse.com>
Thanks for Ack.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:14:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:14:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680577.1058563 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDDP-0004rE-CN; Wed, 14 Feb 2024 11:14:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680577.1058563; Wed, 14 Feb 2024 11:14:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDDP-0004r7-9l; Wed, 14 Feb 2024 11:14:11 +0000
Received: by outflank-mailman (input) for mailman id 680577;
 Wed, 14 Feb 2024 11:14:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=FFc2=JX=redhat.com=mst@srs-se1.protection.inumbo.net>)
 id 1raDDO-0004r1-1b
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:14:10 +0000
Received: from us-smtp-delivery-124.mimecast.com
 (us-smtp-delivery-124.mimecast.com [170.10.133.124])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2b531974-cb2a-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 12:14:08 +0100 (CET)
Received: from mail-ej1-f70.google.com (mail-ej1-f70.google.com
 [209.85.218.70]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-426-R1r3iXM5OYagLtERndrv8Q-1; Wed, 14 Feb 2024 06:14:04 -0500
Received: by mail-ej1-f70.google.com with SMTP id
 a640c23a62f3a-a30f9374db7so67610166b.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:14:04 -0800 (PST)
Received: from redhat.com ([2.52.26.67]) by smtp.gmail.com with ESMTPSA id
 gv24-20020a170906f11800b00a3bd8a34b1bsm2212432ejb.164.2024.02.14.03.13.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 03:14:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b531974-cb2a-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1707909246;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references;
	bh=c1T4mZW1Lt2fEcvTpJpGMJKvoac/0r1T6DraMR3Q2iM=;
	b=Z0S/q/Hkq2hpqo+Ygzs19Z1NaYy7ISdPxio7KCmGppHj8+qybZ0NFYXH2aKKxRa629oTxC
	XZnUDCukMDBA4QXy2gtYPFonultbGhHPGpI8SI789gTm/PVqMUdAT42aJEYWoIQeGoQm5J
	VPwxSorK6tvFSRqplKOr4YTafjolMAc=
X-MC-Unique: R1r3iXM5OYagLtERndrv8Q-1
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707909243; x=1708514043;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=c1T4mZW1Lt2fEcvTpJpGMJKvoac/0r1T6DraMR3Q2iM=;
        b=kRpQ0pQZFMtN5oPgs5KcZQWGSKA16ldZyQTRZEeFAD3kXqvLkr/KrNtjtyuH+F3FXu
         DHNnLYPTsyc7jBYw8Fd4jOKHWlYPGTkgszTFGbPF50NyuzNI0JjxhDwN3VC4zJpPXlhS
         XrlJJ3SRNQ/BDRtwnYTdSJVJbl8C89nHPt6p5qlowxS3W9j0f7GR75aLPCj0h1/7vF+6
         2rB14F6uihRLKMJexuNr17zcemCONp/veiwt9TLpDFEqzSSjiYBpSQ/j3qr1bu/RIylK
         dkbrMedpUv3FQH0+P1xuD/u8t2x9K0vPJaoKbEllZKFNIaJqktb2sBQRP6Y9VqF3+c7m
         uQPA==
X-Forwarded-Encrypted: i=1; AJvYcCVjzwZbEZFeXSGMtRuZmFupdHxtO51Y32lWcFC2XmclElPFEZ9/MwERatH0VKCafeviJWdu357XDGZlR6fsqmEi8za0t58uZnWS6ASOrUs=
X-Gm-Message-State: AOJu0YxdsbXRoV8l6xAp6JDbO2I4j17CkHQC2ZMez/3SJg6Uo92Ym/lG
	ZYjHXPXDPpXAi37s1GUFc1z44qLYEBF3ZuOR4HfGSgu8WTd+pM+5bgNpvVDaG3+/ihm2gExH8eZ
	Os5kSVHLr3OAmflaiCa0fMDQLr0KYgWPlzqjUXjmDEQ0k+qmZSj+7OM4P6GqJIZ0M
X-Received: by 2002:a17:906:aacb:b0:a3c:d0d0:4602 with SMTP id kt11-20020a170906aacb00b00a3cd0d04602mr1474604ejb.6.1707909243471;
        Wed, 14 Feb 2024 03:14:03 -0800 (PST)
X-Google-Smtp-Source: AGHT+IH8PGa8fe+YzlQNsaQxLZwEWdxHc893kIKDULDI4XVDmy08H/IvVqWGbByMSYsf44RIBF6RxQ==
X-Received: by 2002:a17:906:aacb:b0:a3c:d0d0:4602 with SMTP id kt11-20020a170906aacb00b00a3cd0d04602mr1474559ejb.6.1707909242253;
        Wed, 14 Feb 2024 03:14:02 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXzRh2VKZgo8A+woJ9nPu2okdk+XErDylKqAwVKf+sraUsUJAczLmragZ3ncuj+Z6gPCv/6xLKry8uh0vcpB7oLoDWlHlKxDV1VU2kqdG7uW+KLSabxeWUYT0SJzIB8y5XTgOIhWIJoZyEJWTP7pIVzIPL9arbFi8oUEQJWNIwMy37kApC/rRlAPXUHbU75UO8npXBekk5cXAm+Zo7QQfoKktb99+icWJ8zFp6+HqJcuTP6Z95etyaO2pW2Hlji3fBUmakvB0UwdaD9NzPFIOknAMjRD/w9UM2GtuI+bD0a+H3DWT3O9EuqWWeL9phyZZ4edKF2UUA2k46f1aQhrj08zjpcdxPAXKMBomkkOgSwmAl40S95V6ImclLcuJ6meqzuR+YL2w==
Date: Wed, 14 Feb 2024 06:13:57 -0500
From: "Michael S. Tsirkin" <mst@redhat.com>
To: qemu-devel@nongnu.org
Cc: Peter Maydell <peter.maydell@linaro.org>,
	Bui Quang Minh <minhquangbui99@gmail.com>,
	Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	Eduardo Habkost <eduardo@habkost.net>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	xen-devel@lists.xenproject.org
Subject: [PULL 14/60] apic, i386/tcg: add x2apic transitions
Message-ID: <774204cf9874e58dc7fc13394a505452357750ad.1707909001.git.mst@redhat.com>
References: <cover.1707909001.git.mst@redhat.com>
MIME-Version: 1.0
In-Reply-To: <cover.1707909001.git.mst@redhat.com>
X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1
X-Mutt-Fcc: =sent
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

From: Bui Quang Minh <minhquangbui99@gmail.com>

This commit adds support for x2APIC transitions when writing to
MSR_IA32_APICBASE register and finally adds CPUID_EXT_X2APIC to
TCG_EXT_FEATURES.

The set_base in APICCommonClass now returns an integer to indicate error in
execution. apic_set_base return -1 on invalid APIC state transition,
accelerator can use this to raise appropriate exception.

Signed-off-by: Bui Quang Minh <minhquangbui99@gmail.com>
Message-Id: <20240111154404.5333-4-minhquangbui99@gmail.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/i386/apic.h               |  2 +-
 include/hw/i386/apic_internal.h      |  2 +-
 target/i386/cpu.h                    |  4 ++
 hw/i386/kvm/apic.c                   |  3 +-
 hw/i386/xen/xen_apic.c               |  3 +-
 hw/intc/apic.c                       | 62 +++++++++++++++++++++++++++-
 hw/intc/apic_common.c                | 13 +++---
 target/i386/cpu.c                    |  9 ++--
 target/i386/tcg/sysemu/misc_helper.c | 14 ++++++-
 target/i386/whpx/whpx-apic.c         |  3 +-
 10 files changed, 96 insertions(+), 19 deletions(-)

diff --git a/include/hw/i386/apic.h b/include/hw/i386/apic.h
index c8ca41ab44..f6e7489f2d 100644
--- a/include/hw/i386/apic.h
+++ b/include/hw/i386/apic.h
@@ -8,7 +8,7 @@ int apic_accept_pic_intr(DeviceState *s);
 void apic_deliver_pic_intr(DeviceState *s, int level);
 void apic_deliver_nmi(DeviceState *d);
 int apic_get_interrupt(DeviceState *s);
-void cpu_set_apic_base(DeviceState *s, uint64_t val);
+int cpu_set_apic_base(DeviceState *s, uint64_t val);
 uint64_t cpu_get_apic_base(DeviceState *s);
 void cpu_set_apic_tpr(DeviceState *s, uint8_t val);
 uint8_t cpu_get_apic_tpr(DeviceState *s);
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index e796e6cae3..d6e85833da 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -137,7 +137,7 @@ struct APICCommonClass {
 
     DeviceRealize realize;
     DeviceUnrealize unrealize;
-    void (*set_base)(APICCommonState *s, uint64_t val);
+    int (*set_base)(APICCommonState *s, uint64_t val);
     void (*set_tpr)(APICCommonState *s, uint8_t val);
     uint8_t (*get_tpr)(APICCommonState *s);
     void (*enable_tpr_reporting)(APICCommonState *s, bool enable);
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 08eaa61c56..dfe43b8204 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -379,6 +379,10 @@ typedef enum X86Seg {
 #define MSR_IA32_APICBASE_ENABLE        (1<<11)
 #define MSR_IA32_APICBASE_EXTD          (1 << 10)
 #define MSR_IA32_APICBASE_BASE          (0xfffffU<<12)
+#define MSR_IA32_APICBASE_RESERVED \
+        (~(uint64_t)(MSR_IA32_APICBASE_BSP | MSR_IA32_APICBASE_ENABLE \
+                     | MSR_IA32_APICBASE_EXTD | MSR_IA32_APICBASE_BASE))
+
 #define MSR_IA32_FEATURE_CONTROL        0x0000003a
 #define MSR_TSC_ADJUST                  0x0000003b
 #define MSR_IA32_SPEC_CTRL              0x48
diff --git a/hw/i386/kvm/apic.c b/hw/i386/kvm/apic.c
index 1e89ca0899..a72c28e8a7 100644
--- a/hw/i386/kvm/apic.c
+++ b/hw/i386/kvm/apic.c
@@ -95,9 +95,10 @@ void kvm_get_apic_state(DeviceState *dev, struct kvm_lapic_state *kapic)
     apic_next_timer(s, s->initial_count_load_time);
 }
 
-static void kvm_apic_set_base(APICCommonState *s, uint64_t val)
+static int kvm_apic_set_base(APICCommonState *s, uint64_t val)
 {
     s->apicbase = val;
+    return 0;
 }
 
 static void kvm_apic_set_tpr(APICCommonState *s, uint8_t val)
diff --git a/hw/i386/xen/xen_apic.c b/hw/i386/xen/xen_apic.c
index 7c7a60b166..101e16a766 100644
--- a/hw/i386/xen/xen_apic.c
+++ b/hw/i386/xen/xen_apic.c
@@ -49,8 +49,9 @@ static void xen_apic_realize(DeviceState *dev, Error **errp)
     msi_nonbroken = true;
 }
 
-static void xen_apic_set_base(APICCommonState *s, uint64_t val)
+static int xen_apic_set_base(APICCommonState *s, uint64_t val)
 {
+    return 0;
 }
 
 static void xen_apic_set_tpr(APICCommonState *s, uint8_t val)
diff --git a/hw/intc/apic.c b/hw/intc/apic.c
index 178fb26b47..1d887d66b8 100644
--- a/hw/intc/apic.c
+++ b/hw/intc/apic.c
@@ -308,8 +308,49 @@ bool is_x2apic_mode(DeviceState *dev)
     return s->apicbase & MSR_IA32_APICBASE_EXTD;
 }
 
-static void apic_set_base(APICCommonState *s, uint64_t val)
+static int apic_set_base_check(APICCommonState *s, uint64_t val)
 {
+    /* Enable x2apic when x2apic is not supported by CPU */
+    if (!cpu_has_x2apic_feature(&s->cpu->env) &&
+        val & MSR_IA32_APICBASE_EXTD) {
+        return -1;
+    }
+
+    /*
+     * Transition into invalid state
+     * (s->apicbase & MSR_IA32_APICBASE_ENABLE == 0) &&
+     * (s->apicbase & MSR_IA32_APICBASE_EXTD) == 1
+     */
+    if (!(val & MSR_IA32_APICBASE_ENABLE) &&
+        (val & MSR_IA32_APICBASE_EXTD)) {
+        return -1;
+    }
+
+    /* Invalid transition from disabled mode to x2APIC */
+    if (!(s->apicbase & MSR_IA32_APICBASE_ENABLE) &&
+        !(s->apicbase & MSR_IA32_APICBASE_EXTD) &&
+        (val & MSR_IA32_APICBASE_ENABLE) &&
+        (val & MSR_IA32_APICBASE_EXTD)) {
+        return -1;
+    }
+
+    /* Invalid transition from x2APIC to xAPIC */
+    if ((s->apicbase & MSR_IA32_APICBASE_ENABLE) &&
+        (s->apicbase & MSR_IA32_APICBASE_EXTD) &&
+        (val & MSR_IA32_APICBASE_ENABLE) &&
+        !(val & MSR_IA32_APICBASE_EXTD)) {
+        return -1;
+    }
+
+    return 0;
+}
+
+static int apic_set_base(APICCommonState *s, uint64_t val)
+{
+    if (apic_set_base_check(s, val) < 0) {
+        return -1;
+    }
+
     s->apicbase = (val & 0xfffff000) |
         (s->apicbase & (MSR_IA32_APICBASE_BSP | MSR_IA32_APICBASE_ENABLE));
     /* if disabled, cannot be enabled again */
@@ -318,6 +359,25 @@ static void apic_set_base(APICCommonState *s, uint64_t val)
         cpu_clear_apic_feature(&s->cpu->env);
         s->spurious_vec &= ~APIC_SV_ENABLE;
     }
+
+    /* Transition from disabled mode to xAPIC */
+    if (!(s->apicbase & MSR_IA32_APICBASE_ENABLE) &&
+        (val & MSR_IA32_APICBASE_ENABLE)) {
+        s->apicbase |= MSR_IA32_APICBASE_ENABLE;
+        cpu_set_apic_feature(&s->cpu->env);
+    }
+
+    /* Transition from xAPIC to x2APIC */
+    if (cpu_has_x2apic_feature(&s->cpu->env) &&
+        !(s->apicbase & MSR_IA32_APICBASE_EXTD) &&
+        (val & MSR_IA32_APICBASE_EXTD)) {
+        s->apicbase |= MSR_IA32_APICBASE_EXTD;
+
+        s->log_dest = ((s->initial_apic_id & 0xffff0) << 16) |
+                      (1 << (s->initial_apic_id & 0xf));
+    }
+
+    return 0;
 }
 
 static void apic_set_tpr(APICCommonState *s, uint8_t val)
diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index 3c43ac9a1d..16ab40a35f 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -35,20 +35,19 @@
 
 bool apic_report_tpr_access;
 
-void cpu_set_apic_base(DeviceState *dev, uint64_t val)
+int cpu_set_apic_base(DeviceState *dev, uint64_t val)
 {
     trace_cpu_set_apic_base(val);
 
     if (dev) {
         APICCommonState *s = APIC_COMMON(dev);
         APICCommonClass *info = APIC_COMMON_GET_CLASS(s);
-        /* switching to x2APIC, reset possibly modified xAPIC ID */
-        if (!(s->apicbase & MSR_IA32_APICBASE_EXTD) &&
-            (val & MSR_IA32_APICBASE_EXTD)) {
-            s->id = s->initial_apic_id;
-        }
-        info->set_base(s, val);
+        /* Reset possibly modified xAPIC ID */
+        s->id = s->initial_apic_id;
+        return info->set_base(s, val);
     }
+
+    return 0;
 }
 
 uint64_t cpu_get_apic_base(DeviceState *dev)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index ef46755a50..2126b0e589 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -631,8 +631,8 @@ void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,
  * in CPL=3; remove them if they are ever implemented for system emulation.
  */
 #if defined CONFIG_USER_ONLY
-#define CPUID_EXT_KERNEL_FEATURES (CPUID_EXT_PCID | CPUID_EXT_TSC_DEADLINE_TIMER | \
-                                 CPUID_EXT_X2APIC)
+#define CPUID_EXT_KERNEL_FEATURES \
+          (CPUID_EXT_PCID | CPUID_EXT_TSC_DEADLINE_TIMER)
 #else
 #define CPUID_EXT_KERNEL_FEATURES 0
 #endif
@@ -642,12 +642,13 @@ void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,
           CPUID_EXT_XSAVE | /* CPUID_EXT_OSXSAVE is dynamic */   \
           CPUID_EXT_MOVBE | CPUID_EXT_AES | CPUID_EXT_HYPERVISOR | \
           CPUID_EXT_RDRAND | CPUID_EXT_AVX | CPUID_EXT_F16C | \
-          CPUID_EXT_FMA | CPUID_EXT_KERNEL_FEATURES)
+          CPUID_EXT_FMA | CPUID_EXT_X2APIC | CPUID_EXT_KERNEL_FEATURES)
           /* missing:
           CPUID_EXT_DTES64, CPUID_EXT_DSCPL, CPUID_EXT_VMX, CPUID_EXT_SMX,
           CPUID_EXT_EST, CPUID_EXT_TM2, CPUID_EXT_CID,
           CPUID_EXT_XTPR, CPUID_EXT_PDCM, CPUID_EXT_PCID, CPUID_EXT_DCA,
-          CPUID_EXT_X2APIC, CPUID_EXT_TSC_DEADLINE_TIMER */
+          CPUID_EXT_TSC_DEADLINE_TIMER
+          */
 
 #ifdef TARGET_X86_64
 #define TCG_EXT2_X86_64_FEATURES CPUID_EXT2_LM
diff --git a/target/i386/tcg/sysemu/misc_helper.c b/target/i386/tcg/sysemu/misc_helper.c
index 1c43a9f4f7..7de0a6e866 100644
--- a/target/i386/tcg/sysemu/misc_helper.c
+++ b/target/i386/tcg/sysemu/misc_helper.c
@@ -158,9 +158,19 @@ void helper_wrmsr(CPUX86State *env)
     case MSR_IA32_SYSENTER_EIP:
         env->sysenter_eip = val;
         break;
-    case MSR_IA32_APICBASE:
-        cpu_set_apic_base(env_archcpu(env)->apic_state, val);
+    case MSR_IA32_APICBASE: {
+        int ret;
+
+        if (val & MSR_IA32_APICBASE_RESERVED) {
+            goto error;
+        }
+
+        ret = cpu_set_apic_base(env_archcpu(env)->apic_state, val);
+        if (ret < 0) {
+            goto error;
+        }
         break;
+    }
     case MSR_EFER:
         {
             uint64_t update_mask;
diff --git a/target/i386/whpx/whpx-apic.c b/target/i386/whpx/whpx-apic.c
index 8710e37567..7e14ded978 100644
--- a/target/i386/whpx/whpx-apic.c
+++ b/target/i386/whpx/whpx-apic.c
@@ -90,9 +90,10 @@ static void whpx_get_apic_state(APICCommonState *s,
     apic_next_timer(s, s->initial_count_load_time);
 }
 
-static void whpx_apic_set_base(APICCommonState *s, uint64_t val)
+static int whpx_apic_set_base(APICCommonState *s, uint64_t val)
 {
     s->apicbase = val;
+    return 0;
 }
 
 static void whpx_put_apic_base(CPUState *cpu, uint64_t val)
-- 
MST



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:20:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:20:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680582.1058572 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDJC-0006cv-0L; Wed, 14 Feb 2024 11:20:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680582.1058572; Wed, 14 Feb 2024 11:20:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDJB-0006co-Ty; Wed, 14 Feb 2024 11:20:09 +0000
Received: by outflank-mailman (input) for mailman id 680582;
 Wed, 14 Feb 2024 11:20:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raDJA-0006ch-3V
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:20:08 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 00b9312b-cb2b-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 12:20:05 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-410e676c6bbso14394615e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:20:05 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g14-20020a05600c310e00b00410e90e82ebsm1677669wmo.4.2024.02.14.03.20.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 03:20:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00b9312b-cb2b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707909605; x=1708514405; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Rt1gKRJfgTQ1hdAu1e9zYcBvohmGTmI3u928D67LkQ0=;
        b=CSWAlZktdEkolTR8fqCO73GNfmDKxuAwlG8JNVI8OLQLscM6DLQgthO2r0ikMBWcBo
         9XRdmRL7EAwOmEyFFkdi7rVf79KO5bAVCieeG8vAVP8i1/w8WxcN/HfpclQF8hg73LH9
         oOHLzPDKEeFuQbH9ws3XPdNa4kbZxIpnh0tlx5pYtzVwMfAeHLYVJd4tK2SNtfQHL3LN
         bK9rrqqgiPesTICCT+h+ATOy27SA8mzeCetAaVDTmm1m/BIE5B1Qw70JtLzOWmm6y1DI
         m0MgUg22T8jhT8aOAbrh+LuS49YjknSCt2BvXw3o9saf517UkzvGGnoJ7h2AeEkcx1BG
         XRvg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707909605; x=1708514405;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Rt1gKRJfgTQ1hdAu1e9zYcBvohmGTmI3u928D67LkQ0=;
        b=bQu9N0du5Dm3vYp0zDaO2ALwprlSKHpbICBtSsSV2sRFGNh6BhxWMsS6506dp2XJCa
         UHS90flxKs3Y1KKzmxhp52FLHaLNyzFHg8woQxD0Gla/uYv8K3N816X8FhYxySECJmgE
         5EUcatwk03CghEP3EN8NvMJhTyYfldCgZNlqdFYWukFZ7laQhYRwqX8MdqsAZ0NKT0qg
         h1J3YJnmDlyKbsT4flQHWCmQVkUedSE5sWQJUo+Ju++AsuajcQWFjS1LRyMUlvYvm0M3
         x9U8fGA9sLAPy6sthIArEKYpTtirC3pSlklA3NRUc86MqVCi7BD9yjsLUnKQzA6/ssoI
         Kxxg==
X-Forwarded-Encrypted: i=1; AJvYcCUbL0QVJLsGw9iPs6g6g/FecAWNercJHKvc+A4ZUhGlxZpO0exUuz/vXhgEN6cqJWBaDaoijdEwFar1rsXWgXNGRjv+ZajLEKE2DniozAE=
X-Gm-Message-State: AOJu0Yy5oGQADecwv7ruL5GJ4tMprw4pvwsjNwwYA6sQtrHmjjHn04ut
	dMXr/nc2DIffMbsiCCzNjUwiztnAH1AUvALYmRYkRvFVzjM8La6V271HZ8aOEQ==
X-Google-Smtp-Source: AGHT+IFjLkaqewCMOPDe0RsfnfaBR25sKq8cXcWypbnvWnvEYgaOsIUqdyzDYP8F9K2FW/QHXrfq9g==
X-Received: by 2002:a05:600c:518e:b0:411:e398:9eff with SMTP id fa14-20020a05600c518e00b00411e3989effmr1255518wmb.37.1707909604664;
        Wed, 14 Feb 2024 03:20:04 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX6Iz8ndJ/Xn1eGE1NzMirEKgaX2wdgH8i6OY9Uc73NOAjdGUtguzagfxiBlafKXAd0pCoGbgf0IxHtZosM2i3CM1PCeTAk8G8M2FZkt+FjLyVd4EYUkjU2HxMchex7A3xauro0wc3BAM6eytLhdrc6v7rt2Y5S83iHsXigbOp41RvgZjO6SyF4LSL32F10KMTiIniJ0Q8dCcgi+VpdapXt6ManFKe9SJTJjhLeLK+4hXxrlBymN3tzPhERPkM3SzD5fKsJH6cwfgxZRjy1SZR+OT99bd7Un2wUH6AGaM3Qu7c9RmphLXA=
Message-ID: <905ad2e9-ed7f-41f1-a660-456241dd059c@suse.com>
Date: Wed, 14 Feb 2024 12:20:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 09/30] xen/riscv: introduce bitops.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <f76bd85f4b64a47c59c0b306ce425036819fa380.1707146506.git.oleksii.kurochko@gmail.com>
 <b13fd044-c6db-45ab-83d0-3d02221967ed@suse.com>
 <655eed7d1af08936ee3f42a0b88eeb2e197434e2.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <655eed7d1af08936ee3f42a0b88eeb2e197434e2.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 12:06, Oleksii wrote:
> On Mon, 2024-02-12 at 16:58 +0100, Jan Beulich wrote:
>> On 05.02.2024 16:32, Oleksii Kurochko wrote:
>>> +({                                                      \
>>> +    unsigned long __res, __mask;                        \
>>
>> Leftover leading underscores?
> It is how it was defined in Linux, so I thought that I've to leave it
> as it, but I am OK to rename this variables in next patch version.

My view: If you retain Linux style, retaining such names is also (kind
of) okay. If you convert to Xen style, then name changes are to occur
as part of that conversion.

>>> +    __mask = BIT_MASK(nr);                              \
>>> +    __asm__ __volatile__ (                              \
>>> +        __AMO(op) #ord " %0, %2, %1"                    \
>>> +        : "=r" (__res), "+A" (addr[BIT_WORD(nr)])       \
>>> +        : "r" (mod(__mask))                             \
>>> +        : "memory");                                    \
>>> +    ((__res & __mask) != 0);                            \
>>> +})
>>> +
>>> +#define __op_bit_ord(op, mod, nr, addr, ord)    \
>>> +    __asm__ __volatile__ (                      \
>>> +        __AMO(op) #ord " zero, %1, %0"          \
>>> +        : "+A" (addr[BIT_WORD(nr)])             \
>>> +        : "r" (mod(BIT_MASK(nr)))               \
>>> +        : "memory");
>>> +
>>> +#define __test_and_op_bit(op, mod, nr, addr)    \
>>> +    __test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
>>> +#define __op_bit(op, mod, nr, addr) \
>>> +    __op_bit_ord(op, mod, nr, addr, )
>>> +
>>> +/* Bitmask modifiers */
>>> +#define __NOP(x)    (x)
>>> +#define __NOT(x)    (~(x))
>>
>> Here the (double) leading underscores are truly worrying: Simple
>> names like this aren't impossible to be assigned meaninb by a
>> compiler.
> I am not really understand what is the difference for a compiler
> between NOP and __NOP? Do you mean that the leading double underscores
> (__) are often used to indicate that these macros are implementation-
> specific and might be reserved for the compiler or the standard
> library?

It's not "often used". Identifiers starting with two underscores or an
underscore and a capital letter are reserved for the implementation
(i.e. for the compiler's internal use). When not overly generic we
stand a fair chance of getting away. But NOP and NOT are pretty generic.

>>> +#include <asm-generic/bitops/fls.h>
>>> +#include <asm-generic/bitops/flsl.h>
>>> +#include <asm-generic/bitops/__ffs.h>
>>> +#include <asm-generic/bitops/ffs.h>
>>> +#include <asm-generic/bitops/ffsl.h>
>>> +#include <asm-generic/bitops/ffz.h>
>>> +#include <asm-generic/bitops/find-first-set-bit.h>
>>> +#include <asm-generic/bitops/hweight.h>
>>> +#include <asm-generic/bitops/test-bit.h>
>>
>> To be honest there's too much stuff being added here to asm-generic/,
>> all in one go. I'll see about commenting on the remaining parts here,
>> but I'd like to ask that you seriously consider splitting.
> Would it be better to send it outside of this patch series? I can
> create a separate patch series with a separate patch for each asm-
> generic/bitops/*.h

Not sure. Depends in part on whether then you'd effectively introduce
dead code. If the introduction was such that RISC-V used the new ones
right away, then yes, that would quite likely be better.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:27:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:27:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680591.1058583 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDPt-0007mo-RO; Wed, 14 Feb 2024 11:27:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680591.1058583; Wed, 14 Feb 2024 11:27:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDPt-0007mh-No; Wed, 14 Feb 2024 11:27:05 +0000
Received: by outflank-mailman (input) for mailman id 680591;
 Wed, 14 Feb 2024 11:27:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=83ZH=JX=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1raDPs-0007mb-Ur
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:27:04 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f95bdfbd-cb2b-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 12:27:02 +0100 (CET)
Received: from nico.bugseng.com (unknown [147.123.100.131])
 by support.bugseng.com (Postfix) with ESMTPSA id DE65B4EE0738;
 Wed, 14 Feb 2024 12:26:59 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f95bdfbd-cb2b-11ee-98f5-efadbce2ee36
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	Nicola Vetrini <nicola.vetrini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH] automation/eclair_analysis: deviate certain macros for Rule 20.12
Date: Wed, 14 Feb 2024 12:26:55 +0100
Message-Id: <892be5c403ca7a20b35fb9facacb6a38bc7f6bfe.1707900742.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Certain macros are allowed to violate the Rule, since their meaning and
intended use is well-known to all Xen developers.

Variadic macros that rely on the GCC extension for removing a trailing
comma when token pasting the variable argument are similarly
well-understood and therefore allowed.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
This patch is intended as a follow-up to the patch accepting this Rule for Xen
---
 .../eclair_analysis/ECLAIR/deviations.ecl      | 10 ++++++++++
 docs/misra/deviations.rst                      | 18 ++++++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
index fd32ff8a9cae..98e5ad146b8d 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -387,6 +387,16 @@ in assignments."
 {safe, "left_right(^[(,\\[]$,^[),\\]]$)"}
 -doc_end
 
+-doc_begin="The token pasting in variadic macros cannot be replaced."
+-config=MC3R1.R20.12,macros+={deliberate, "variadic()"}
+-doc_end
+
+-doc_begin="Uses of # and ## operators within the following macros are
+deliberate."
+-config=MC3R1.R20.12,macros+={deliberate, "name(ASSERT||BUILD_BUG_ON||BUILD_BUG_ON_ZERO||GENERATE_CASE)"}
+-doc_end
+
+
 #
 # General
 #
diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
index 123c78e20a01..855c1c5579bb 100644
--- a/docs/misra/deviations.rst
+++ b/docs/misra/deviations.rst
@@ -330,6 +330,24 @@ Deviations related to MISRA C:2012 Rules:
        (4) as lhs in assignments.
      - Tagged as `safe` for ECLAIR.
 
+   * - R20.12
+     - Variadic macros that use token pasting often employ the gcc extension
+       `ext_paste_comma`, as detailed in `C-language-toolchain.rst`, which is
+       not easily replaceable.
+     - Tagged as `deliberate` for ECLAIR.
+
+   * - R20.12
+     - Macros that are used for runtime or build-time assertions contain
+       deliberate uses of an argument as both a regular argument and a
+       stringification token, to provide useful diagnostic messages.
+     - Tagged as `deliberate` for ECLAIR.
+
+   * - R20.12
+     - GENERATE_CASE is a special-purpose macro that allows some selected switch
+       statements to be more compact and readable. As such, the risk of
+       developer confusion in using such macro is deemed negligible.
+     - Tagged as `deliberate` for ECLAIR.
+
 Other deviations:
 -----------------
 
-- 
2.34.1


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:27:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:27:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680594.1058593 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDQU-0008GX-2A; Wed, 14 Feb 2024 11:27:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680594.1058593; Wed, 14 Feb 2024 11:27:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDQT-0008GQ-Ve; Wed, 14 Feb 2024 11:27:41 +0000
Received: by outflank-mailman (input) for mailman id 680594;
 Wed, 14 Feb 2024 11:27:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3HRx=JX=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1raDQS-0007mb-KI
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:27:40 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0f1c83a9-cb2c-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 12:27:39 +0100 (CET)
Received: from [172.20.10.2] (unknown [37.163.90.143])
 by support.bugseng.com (Postfix) with ESMTPSA id 4B8374EE0738;
 Wed, 14 Feb 2024 12:27:37 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f1c83a9-cb2c-11ee-98f5-efadbce2ee36
Message-ID: <40b7465f-4966-43c7-8db3-e28a6cc48445@bugseng.com>
Date: Wed, 14 Feb 2024 12:27:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/3] docs/misra/rules.rst: add rule 5.5
Content-Language: en-US, it
To: Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <stefano.stabellini@amd.com>, roberto.bagnara@bugseng.com
Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, julien@xen.org,
 bertrannd.marquis@arm.com, roger.pau@citrix.com,
 xen-devel@lists.xenproject.org
References: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
 <20240213223334.3693410-2-stefano.stabellini@amd.com>
 <50719397-b053-43e1-9cf7-cc9eae9098ed@suse.com>
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <50719397-b053-43e1-9cf7-cc9eae9098ed@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 14/02/24 09:28, Jan Beulich wrote:
> On 13.02.2024 23:33, Stefano Stabellini wrote:
>> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
>> ---
>>   docs/misra/rules.rst | 6 ++++++
>>   1 file changed, 6 insertions(+)
>>
>> diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
>> index c185366966..931158b354 100644
>> --- a/docs/misra/rules.rst
>> +++ b/docs/misra/rules.rst
>> @@ -181,6 +181,12 @@ maintainers if you want to suggest a change.
>>          headers (xen/include/public/) are allowed to retain longer
>>          identifiers for backward compatibility.
>>   
>> +   * - `Rule 5.5 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_05_05.c>`_
>> +     - Required
>> +     - Identifiers shall be distinct from macro names
>> +     - Clashes between function-like macros and non-callable entities
>> +       are allowed. The pattern #define x x is also allowed.
> 
> Just for me to know what exactly is covered (hence also a question
> to Roberto as to [to be] implemented Eclair behavior): Even when
> the above would be sufficient (and imo better) people frequently
> write
> 
> #define a(x, y) b(x, y)
> 
> which, transformed to the specific case here, would then be
> 
> #define a(x, y) a(x, y)
> 
> I'd assume such ought to also be covered, but that's not clear
> from the spelling above.

I list what happens in some different situations,
then we can find the right words for the documentation and/or
refine the configuration:

If you
#define x x
and then use `x' as identifier,
the resulting violation is deviated (allowed pattern).

If you
#define a(x, y) a(x, y)
and then use `a' as identifier for a non-callable entity,
the resulting violation is deviated (no clash with non-callable
entities).
If you use identifier `a' for a callable entity, the resulting violation
is reported: the allowed pattern covers only macros expanding to their
own name, in this case the macro name is considered to be
`a' only, not a(x, y).

If you
#define a(x, y) b(x, y)
and then use `a' as identifier for a non-callable entity,
the resulting violation is deviated (no clash with non-callable
entities).
If you use `a' as identifier for a callable entity,
this is not a violation because after the preprocessing phase,
identifier `a' no longer exists.

As far as I know, this is what was agreed upon in one of the recent
MISRA meetings.

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:30:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:30:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680599.1058602 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDTX-0001Px-FM; Wed, 14 Feb 2024 11:30:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680599.1058602; Wed, 14 Feb 2024 11:30:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDTX-0001Pq-CW; Wed, 14 Feb 2024 11:30:51 +0000
Received: by outflank-mailman (input) for mailman id 680599;
 Wed, 14 Feb 2024 11:30:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=VWPZ=JX=gmail.com=w1benny@srs-se1.protection.inumbo.net>)
 id 1raDTW-0001Pk-KE
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:30:50 +0000
Received: from mail-oa1-x34.google.com (mail-oa1-x34.google.com
 [2001:4860:4864:20::34])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7fdcf12e-cb2c-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 12:30:48 +0100 (CET)
Received: by mail-oa1-x34.google.com with SMTP id
 586e51a60fabf-2185739b64cso3403076fac.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:30:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7fdcf12e-cb2c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707910247; x=1708515047; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=6g+2n+lFd1EoSzGvfq+eeDzfB8zXGJFLjsEcNhQw+uw=;
        b=My5atkWn9hVxmW7l1BnNCO7hD7Z4q8xe80XRMO/3/0+g77Q6g8Ier9GUDJypnHyT2h
         vafaGzWgeyFWycJsKOQUvaEZA+sf8ZGv7dkazgsYaz3gsm2XQMFeFkVmyJxz2IQBPfXF
         v9MbEGj3+/5JyoOaoJeM5qNpvCzIPa7S8iRtzKL1bCUZ+6Z+GYrWxjmjqw9TC3QZ3bzo
         Mn/Ih1y7qaFFHjgghjAR/n8b/3Tt2MdgURctjPjj5m/xrPyfIR5MzyXD+O9+K7voZDSZ
         MX2zGHKbNHtiA3Jvs0Mez0TGLcGGVd753t5SIsfr1hr75FjxOESM31fmlPhKAjUD9mlN
         mVug==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707910247; x=1708515047;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=6g+2n+lFd1EoSzGvfq+eeDzfB8zXGJFLjsEcNhQw+uw=;
        b=Hek6lHutQwGNu5RLtL54cKm1ge95TrTEJgwb2OdEVPKb3dnBYhdfZ0MDeAkkTEtza6
         yjA90stCVNdX1nnydZOse5Dm0itvl7BBErrodnMrVoD9CeXuHAO1HheTeQNGCBbackyW
         rP2fU5K9RhqdyjJv0Lum6U0FlyS0LjPjJ0eYIOvG+AQPPGnejOcAlY3UV0Ds9sbVdeAM
         WLtDSYBRgP0XNMMBVO/PpZj7s34l7Ldu+5WyW7J3bwlHd60MGS+RjFLeu79PgjbikrhU
         nBzY5D4kVESJPSo52YDHV2/G/WzwdTLEdiOrkYMQrj8nvI+sX67oSAB58Pri9hVsgfC7
         1H9w==
X-Forwarded-Encrypted: i=1; AJvYcCXlXTuohuuu/nR3xUz9O/Vy8dyMm2omoJqmq0OjbjLZQcQNor7c4a+ADHl5F0Ltk7OH+5/Y5qRi0VcGf6suG4D5ZJdWjwm/6Z38IBdMgF8=
X-Gm-Message-State: AOJu0YxzKVZAKNSU3zVFt+TGQPfND2pOZIOemKRjCDdJfUInHG/CL71o
	7v+cKFAiiDDIeS6oGeb2DugXgEVD5okH3+1TWdmYM5gyWYcft7digIQhOV8vDECKJSCHhYQOofn
	+UeRitS/mhWEEDbgnNAMEm+pJF6M=
X-Google-Smtp-Source: AGHT+IFRBf+8YTuGQPtDmqCnI+rSbaLJuYzTauzsLB1Fco8wJ+FQbHxHl4JbLzq7cRWpTLrJ1nuoC3X+IIZOhSuYfxQ=
X-Received: by 2002:a05:6870:d628:b0:21a:6563:f4d3 with SMTP id
 a40-20020a056870d62800b0021a6563f4d3mr2398037oaq.49.1707910247408; Wed, 14
 Feb 2024 03:30:47 -0800 (PST)
MIME-Version: 1.0
References: <9b9909c9e93cb540b3488c784935acc2bc9e071e.1707343396.git.w1benny@gmail.com>
 <3A858D7F-C953-4EF0-8919-AE96D6105AB1@cloud.com> <01b0d902-1903-4618-ad43-f625e57b61e1@suse.com>
In-Reply-To: <01b0d902-1903-4618-ad43-f625e57b61e1@suse.com>
From: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>
Date: Wed, 14 Feb 2024 12:30:36 +0100
Message-ID: <CAKBKdXjdOoGSbN_c4gyBqAHni+D4+o+gC=LNc4VEaaezYYki+g@mail.gmail.com>
Subject: Re: [PATCH] tools/ocaml: Add missing vmtrace_buf_kb field
To: Jan Beulich <jbeulich@suse.com>
Cc: Christian Lindig <christian.lindig@cloud.com>, Xen-devel <xen-devel@lists.xenproject.org>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, David Scott <dave@recoil.org>, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 14, 2024 at 8:12=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 08.02.2024 10:13, Christian Lindig wrote:
> >> On 7 Feb 2024, at 22:04, Petr Bene=C5=A1 <w1benny@gmail.com> wrote:
> >> Add the missing `vmtrace_buf_kb` field to the OCaml bindings to match =
the
> >> vm.cfg configuration, correcting an oversight from its initial introdu=
ction.
> >>
> >> Signed-off-by: Petr Bene=C5=A1 <w1benny@gmail.com>
> >
> > Acked-by: Christian Lindig <christian.lindig@cloud.com>
> >
> > This looks correct from an OCaml perspective. Why was the new field add=
ed in the middle of the record type domctl_create_config and thus forcing c=
hanges to the index of fields coming later in the record versus just append=
ing the new field to the record type?
> >
> > The critical bit is using the correct type in "Int32_val(VAL_VMTRACE_BU=
F_KB)=E2=80=9D that matches the type "vmtrace_buf_kb: int32;=E2=80=9D - whi=
ch it does.
>
> Is this then perhaps also lacking a
>
> Fixes: 45ba9a7d7688 ("tools/[lib]xl: Add vmtrace_buf_size parameter")
>
> and hence wanting backporting?
>
> Jan

In my opinion, yes.

P.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:34:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:34:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680606.1058614 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDWy-0002c5-US; Wed, 14 Feb 2024 11:34:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680606.1058614; Wed, 14 Feb 2024 11:34:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDWy-0002by-Pv; Wed, 14 Feb 2024 11:34:24 +0000
Received: by outflank-mailman (input) for mailman id 680606;
 Wed, 14 Feb 2024 11:34:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raDWx-0002bs-63
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:34:23 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fede3ab4-cb2c-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 12:34:21 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-561587ce966so1233530a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:34:21 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 p5-20020a056402074500b005638caeff4bsm134737edy.22.2024.02.14.03.34.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 03:34:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fede3ab4-cb2c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707910460; x=1708515260; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=tPi62fcobDHUKuWkdKdit8MbEtdoP0CMKdDYvUv8Ruk=;
        b=FWILDKlrbfSDAkF7bVnzrets3XH6w5rb7stIGVtQRvpd+bpEWz4ACU/stu0xxvmlmm
         Tu3HTfHDAtBPiqvf9WeRLmSFq5cy0/2sYbXM9LRcaa6A8+yh3oEZBPd1v/GnUq4BgfN6
         W8CgD2ZspI1XSPRVZGtW6Pauzdp594DAkdMmM/UbomyPMjKj/RYqyOz9920CsCMMGqqb
         ZVI2iB8ZJ4WULLF0RJsEjMBRQSDsRzSL2LwuS/dFyPluB9lL83TFGrbI0ugZnPSC2lvs
         Nfi3530BKKtPv4ZFUSfAPk/gUxt64gIArtw2g2YgRp2TeiCEUkRsi7fXuecxIy8zERHO
         PbvA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707910460; x=1708515260;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=tPi62fcobDHUKuWkdKdit8MbEtdoP0CMKdDYvUv8Ruk=;
        b=XjzVEY+PWrZ4Ao7jMS/BwmChTcehSqi3+4fRn0ld6+cKRvs1rF1Pk5wSkszCW7bsVq
         LExFxBve27OCBW7tjMxLfBsAr+efND//EnD5wql68U9SDIGC9g/67R+D7OeESoMYLGr1
         DxbDvKKTWTrcDhbvq1nlNoOFjNQF8fURvYu7AXvJCSOG9LkkNCwCUOd+FfQC2h9YGvA3
         mxxuKw3kVIY5fI1TgbncsjtLjiZ3P1RBZr9hF69S4MYMgCNMj5oAmylBULgS13IoV0jz
         z7yhBgLo4TZdqBnS5OQehwkPROyUM50Dp4nqKrNHZSrCGgirQcVm2kjF1SsNHDgO+tpi
         dd/g==
X-Forwarded-Encrypted: i=1; AJvYcCU7SQ7h25DqIUeXfdLDYks6I9deoapbRGzs3r5nkK9pQTZ1m9EnmO0vM8Y9AjZHILUiMcjWBwdgdK+JNG/i619vU+lNW/P/3zru2dp9+TQ=
X-Gm-Message-State: AOJu0Yz/smJJp+DNCdw6iu7/CM2HmNasEm3EH5DCiMQN+1h1JE/cHeO8
	YY6O0v7MsjYySy3HD2+ZWVmmeKmu8qecVvCBtWtBaroYObFjJRSx
X-Google-Smtp-Source: AGHT+IGATqMdBmVHPZd4BquITBOSNOO+olDNRbN603USHMv1ECqd4FXOQrgi8E4A31B43xKgK3ZGfg==
X-Received: by 2002:a05:6402:c4f:b0:561:f2b1:a68 with SMTP id cs15-20020a0564020c4f00b00561f2b10a68mr1447174edb.20.1707910460411;
        Wed, 14 Feb 2024 03:34:20 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW4CO0VlsdTuynp/FsKIqyTeeoE24v9VZNXvg/ZgaPWBVazpoEVB8zq9jeBGJyrDxIyNXG7qT6pUHPZjTb+wrL2Hkj09m1TSApwd+iL3R7NCyD/lfTpI8UOEJst4TGUuVwLhFtdCX5k/GC7MpHIDsfA3ZjwMoaHEX3wUnIJFDuIS7LoLCh3vhrURbEltxXVZhTFqacJn15BLSIhLBNJMY2RNzzLuS64NngsoPq7JESZfMiJBbBgkOoBD7ctCvglzDnhL3vJZEK/FFWvMoDmrXfqVG3Q8XYfL5Bktnw2/Kz3idDUizv9Wsw=
Message-ID: <c390cfc4b2993befc5ef5cd5f41fe34a7c85eda4.camel@gmail.com>
Subject: Re: [PATCH v4 13/30] xen/riscv: introduce io.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 12:34:19 +0100
In-Reply-To: <5508296c-2721-43e4-83d3-2603ce31d010@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <64afc021f680e55d486f8429c02513bc9bf4b9ea.1707146506.git.oleksii.kurochko@gmail.com>
	 <5508296c-2721-43e4-83d3-2603ce31d010@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Tue, 2024-02-13 at 12:05 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > The header taken form Linux 6.4.0-rc1 and is based on
> > arch/riscv/include/asm/mmio.h.
> >=20
> > Addionally, to the header was added definions of ioremap_*().
> >=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > Changes in V4:
> > =C2=A0- delete inner parentheses in macros.
> > =C2=A0- s/u<N>/uint<N>.
> > ---
> > Changes in V3:
> > =C2=A0- re-sync with linux kernel
> > =C2=A0- update the commit message
> > ---
> > Changes in V2:
> > =C2=A0- Nothing changed. Only rebase.
> > ---
> > =C2=A0xen/arch/riscv/include/asm/io.h | 142
> > ++++++++++++++++++++++++++++++++
> > =C2=A01 file changed, 142 insertions(+)
> > =C2=A0create mode 100644 xen/arch/riscv/include/asm/io.h
> >=20
> > diff --git a/xen/arch/riscv/include/asm/io.h
> > b/xen/arch/riscv/include/asm/io.h
> > new file mode 100644
> > index 0000000000..1e61a40522
> > --- /dev/null
> > +++ b/xen/arch/riscv/include/asm/io.h
> > @@ -0,0 +1,142 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * {read,write}{b,w,l,q} based on arch/arm64/include/asm/io.h
> > + *=C2=A0=C2=A0 which was based on arch/arm/include/io.h
> > + *
> > + * Copyright (C) 1996-2000 Russell King
> > + * Copyright (C) 2012 ARM Ltd.
> > + * Copyright (C) 2014 Regents of the University of California
> > + */
> > +
> > +
> > +#ifndef _ASM_RISCV_IO_H
> > +#define _ASM_RISCV_IO_H
> > +
> > +#include <asm/byteorder.h>
> > +
> > +/*
> > + * The RISC-V ISA doesn't yet specify how to query or modify PMAs,
> > so we can't
> > + * change the properties of memory regions.=C2=A0 This should be fixed
> > by the
> > + * upcoming platform spec.
> > + */
> > +#define ioremap_nocache(addr, size) ioremap(addr, size)
> > +#define ioremap_wc(addr, size) ioremap(addr, size)
> > +#define ioremap_wt(addr, size) ioremap(addr, size)
> > +
> > +/* Generic IO read/write.=C2=A0 These perform native-endian accesses.
> > */
> > +#define __raw_writeb __raw_writeb
>=20
> What use are this and the similar other #define-s?
I don't know specific reason for that, this file fully based on Linux's
arch/riscv/include/asm/mmio.h, so we can drop such defines.

>=20
> > +static inline void __raw_writeb(uint8_t val, volatile void __iomem
> > *addr)
> > +{
> > +	asm volatile("sb %0, 0(%1)" : : "r" (val), "r" (addr));
>=20
> Nit (throughout): Missing blanks. Or wait - is this file intended to
> be Linux style? If so, it's just one blank that's missing.
I started to update the code style, so I am OK to add missing blanks.
Thanks.
>=20
> > +/*
> > + * Unordered I/O memory access primitives.=C2=A0 These are even more
> > relaxed than
> > + * the relaxed versions, as they don't even order accesses between
> > successive
> > + * operations to the I/O regions.
> > + */
> > +#define readb_cpu(c)		({ uint8_t=C2=A0 __r =3D __raw_readb(c);
> > __r; })
> > +#define readw_cpu(c)		({ uint16_t __r =3D
> > le16_to_cpu((__force __le16)__raw_readw(c)); __r; })
> > +#define readl_cpu(c)		({ uint32_t __r =3D
> > le32_to_cpu((__force __le32)__raw_readl(c)); __r; })
>=20
> Didn't we settle on the little-endian stuff to be dropped from here?
> No matter what CPU endianness, what endianness a particular device
> (and hence its MMIO region(s)) is using is entirely independent.
> Hence
> conversion, where necessary, needs to occur at a layer up.
Yes, just missed to remove that.

>=20
> Also, what good do the __r variables do here? If they weren't here,
> we also wouldn't need to discuss their naming.
I don't see to much sense in __r, so it could be dropped.

>=20
> > +#define writeb_cpu(v,c)		((void)__raw_writeb(v,c))
> > +#define
> > writew_cpu(v,c)		((void)__raw_writew((__force uint16_t)cpu_to_le16(v),c=
))
> > +#define
> > writel_cpu(v,c)		((void)__raw_writel((__force uint32_t)cpu_to_le32(v),c=
))
>=20
> Nit: Blanks after commas please (also again further down).
Thanks, I'll update that.

>=20
> > +#ifdef CONFIG_64BIT
> > +#define readq_cpu(c)		({ u64 __r =3D le64_to_cpu((__force
> > __le64)__raw_readq(c)); __r; })
> > +#define
> > writeq_cpu(v,c)		((void)__raw_writeq((__force u64)cpu_to_le64(v),c))
>=20
> uint64_t (twice)
>=20
> > +#endif
> > +
> > +/*
> > + * I/O memory access primitives. Reads are ordered relative to any
> > + * following Normal memory access. Writes are ordered relative to
> > any prior
> > + * Normal memory access.=C2=A0 The memory barriers here are necessary
> > as RISC-V
> > + * doesn't define any ordering between the memory space and the
> > I/O space.
> > + */
> > +#define __io_br()	do {} while (0)
>=20
> Nit: This and ...
>=20
> > +#define __io_ar(v)	__asm__ __volatile__ ("fence i,r" : : :
> > "memory");
> > +#define __io_bw()	__asm__ __volatile__ ("fence w,o" : : :
> > "memory");
> > +#define __io_aw()	do { } while (0)
>=20
> ... this want to be spelled exactly the same.
Oh, overlooked that. Thanks.

>=20
> Also, why does __io_ar() have a parameter (which it then doesn't
> use)?
In case of Xen and RISC-V, it can be droped. in case of Linux,
__io_ar() is also defined, at least, for Arm where this parameter is
used, so I assume that intention was to have the same API for
__io_ar().

>=20
> Finally at least within a single file please be consistent about
> asm()
> vs __asm__() use.
>=20
> > +#define readb(c)	({ uint8_t=C2=A0 __v; __io_br(); __v =3D
> > readb_cpu(c); __io_ar(__v); __v; })
> > +#define readw(c)	({ uint16_t __v; __io_br(); __v =3D
> > readw_cpu(c); __io_ar(__v); __v; })
> > +#define readl(c)	({ uint32_t __v; __io_br(); __v =3D
> > readl_cpu(c); __io_ar(__v); __v; })
>=20
> Here the local variables are surely needed. Still they would
> preferably
> not have any underscores as prefixes.
Thanks. This header was left untouched as it was mostly just copy of
Linux's mmio.h, but I'll update it according to Xen code style. Thanks.

>=20
> > +#define writeb(v,c)	({ __io_bw(); writeb_cpu(v,c); __io_aw();
> > })
> > +#define writew(v,c)	({ __io_bw(); writew_cpu(v,c); __io_aw();
> > })
> > +#define writel(v,c)	({ __io_bw(); writel_cpu(v,c); __io_aw();
> > })
> > +
> > +#ifdef CONFIG_64BIT
> > +#define readq(c)	({ u64 __v; __io_br(); __v =3D readq_cpu(c);
> > __io_ar(__v); __v; })
>=20
> uint64_t again
Thanks. I'll update that too.

>=20
> > +#define writeq(v,c)	({ __io_bw(); writeq_cpu((v),(c));
> > __io_aw(); })
>=20
> Inner parentheses still left?
Overlooked. Thanks, I'll update that in the next patch version.

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:38:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:38:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680611.1058623 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDam-0003Bp-Cq; Wed, 14 Feb 2024 11:38:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680611.1058623; Wed, 14 Feb 2024 11:38:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDam-0003Bi-9j; Wed, 14 Feb 2024 11:38:20 +0000
Received: by outflank-mailman (input) for mailman id 680611;
 Wed, 14 Feb 2024 11:38:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raDak-0003Bc-Fk
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:38:18 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8afe8940-cb2d-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 12:38:16 +0100 (CET)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-33cd57b86bfso1221597f8f.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:38:16 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e20-20020adf9bd4000000b0033cee68c31asm1334895wrc.61.2024.02.14.03.38.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 03:38:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8afe8940-cb2d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707910696; x=1708515496; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rdxTMeHXrManeT3nC+19C4mu58aFUrs+icQiSiXCw3c=;
        b=PtVifr+hq7VFaSnvMekFZ1wif23xLDY2Xmnb8uiwO1e4kDxUhbFkVtxhewPQvBny+z
         ugZ2XeS07q3/GQhCDc92IQowpILjsYCoT2YhbYfts+lrKjkc6puH3iVNNBGfHFert2dy
         qBG61yqZzLzXeURv7MRTzR9UBkZ7eUeOUyw1V03RgxlJyAWQ9e+p4lOWPe9/5Ky/lDUr
         vzcYgo1vZYupm8X0U71YjP6Flr/55/WxsIXdGi/YydT4fzK3n8GfKgRr6sTWZ48SXN+X
         kTeKjyd/t9aRPu0t/KKTxuJrYPMSp86nQXTSCB3z0hF1ktVVI6wM9Xt7cwmQfTjcL171
         vsmQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707910696; x=1708515496;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rdxTMeHXrManeT3nC+19C4mu58aFUrs+icQiSiXCw3c=;
        b=OfR9wHCltiAwZfEEuA55vHIZkXelrafuJ4D6zk6/g7n6BPfYv5nh+st2riWJG5Dz2Q
         M6IO+YFT3iwADWOQc1NvEgEIiCxysHhxwYfTwOaBt5Ew6k0dzw+Eq+1osmBAdV90RVjF
         67tblBTqiJ1X3nZ3RrEhfsXJ3V09fVk/iS8q2l+IdX14F4wNN2QU329Gc4Pb/BPScyYm
         sOeVYFQR2Y6R3m2IYKmqKChWafLMBKLkwEU/HNol0jT0wR7AUtyTXHAAsGpIVklSBaI1
         puaoxXemEja0J723cXRQidv5L5EnPM69KRETLi/0i2EbNG5FFoQZRuid5nNBnvcuO1hK
         Wlxg==
X-Forwarded-Encrypted: i=1; AJvYcCU1JGosb2hYihCGGriBtUcwbKF+AY8asBC6BH83i/WpRcxOGDFIdFrCD/ZSdOC63mqQQTnxrJha+iQ0AIEXb0DDw/mckBWr2mQFqRrwRzY=
X-Gm-Message-State: AOJu0YwUcJCeHtZI84vT73xSTGnXmfgTVcMdyuBA/4fy8Um6S37X8QNH
	4CQaSFMD5CeCpC5+T2eEu7cketjazmnm8/wYHLH4+V0RZTOkOkJFveIpFH6ZTA==
X-Google-Smtp-Source: AGHT+IFqfA5CdqS3bvqUK+IQR1iD91ies1WuS2Yd8o8RiqnDYywGhDViQfrSCrJecn3s2aN1pG3gjg==
X-Received: by 2002:a5d:5108:0:b0:33b:8260:ecbf with SMTP id s8-20020a5d5108000000b0033b8260ecbfmr1496488wrt.14.1707910695667;
        Wed, 14 Feb 2024 03:38:15 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCViaO4MGoPO7IZg5plLx5X+U3+toIuqfG1lYuuzj9CUD77LaktBSUtvzhgRLsb5l2NYWUZcjum4gPNPdHEq9hLtZmAdaclXTVYwc64NTXsnjv4sD14/DFjaNchspjRwmo+LmrOnqf8mP49F3/Ua3462HYq8l97ccNp/EgYGQ8LlnrlPHn1+bOXImcUITlcBryu+f5VpD+KNjfq8Awfkpw+u0i9bYCu/WOUy26nNLd5iR0kLimcbIAwuDq29nwEPeP4Hu07mfhrRPq77YVhIJuPi+gAJCX6QYr2/yZop521Cgw5ooA/Wte69ZDq9VFTe8juiPO8wUcpXNIfDBAllj0CDyGmmKrZn5nU3rKcoXlq7Z8clbIJjIzAckTmCxxKgDwCEYEIMjA==
Message-ID: <123f4cc2-8fc7-4e2a-b44c-78a81b3a1a3e@suse.com>
Date: Wed, 14 Feb 2024 12:38:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Paul Durrant <paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240202213321.1920347-2-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 22:33, Stewart Hildebrand wrote:
> --- a/xen/arch/x86/physdev.c
> +++ b/xen/arch/x86/physdev.c
> @@ -123,7 +123,9 @@ int physdev_map_pirq(domid_t domid, int type, int *index, int *pirq_p,
>  
>      case MAP_PIRQ_TYPE_MSI:
>      case MAP_PIRQ_TYPE_MULTI_MSI:
> +        pcidevs_lock();
>          ret = allocate_and_map_msi_pirq(d, *index, pirq_p, type, msi);
> +        pcidevs_unlock();
>          break;

I'm afraid I need to come back to this: This is the only place where this
patch retains (moves) use of the global lock. By moving, its scope is
actually extended. It was previously said that conversion doesn't happen
to limit the scope of what is changing. But with allocate_and_map_msi_pirq()
being happy about either lock being held, I'm having a hard time seeing why
here the global lock would continue to need using. To me doing so suggests
uncertainty whether the checking in the function is actually correct.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:41:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:41:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680617.1058642 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeH-0005eC-4q; Wed, 14 Feb 2024 11:41:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680617.1058642; Wed, 14 Feb 2024 11:41:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeH-0005e5-2E; Wed, 14 Feb 2024 11:41:57 +0000
Received: by outflank-mailman (input) for mailman id 680617;
 Wed, 14 Feb 2024 11:41:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raDeG-0005Pc-C4
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:41:56 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0de73d36-cb2e-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 12:41:55 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-33b2960ff60so394416f8f.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:41:55 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 x4-20020ac84d44000000b0042c4f46a994sm1934179qtv.75.2024.02.14.03.41.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 03:41:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0de73d36-cb2e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707910914; x=1708515714; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=94Zht/z5Je72hcNo+d2tbsOKFvcKaoSfFgtITaQ3n5U=;
        b=SW360cwWTfLDuLpOY9V1ckMfL2Kt97p8VYEqHCE8pqMVfaE6hsZHYQMrD+T0H7dhFY
         R1frgzZVSJ/a4zzRpWUQPnS0YaJ0c8dx1OXB84ZoUYgdBOuSsYrTtlfm5xtZkyUa6eED
         2w9JAire+NWB+QQSPaLF0ToIOgXaQDOVCondY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707910914; x=1708515714;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=94Zht/z5Je72hcNo+d2tbsOKFvcKaoSfFgtITaQ3n5U=;
        b=S6t8SlIC7Vhg5tOcOsb0mange8t+HrBDC/r6xf/09kMdvo87zofCXyTGSAtIsUK+YZ
         +gfNZgIZqvOOlRXdpUtYc1Xvfk6WSlvcaNFsilEUAHflAGtVVvPEW7eBSy2RVT6nZ4Po
         XoLYqyV/5cr0sZVyS2IHPoSKYgsWtzDWYcbllmbxjzOV7GqJZE4vfaE4FkaohxTvZL30
         T1GE+RQMEAYxh0AlaAOBvJsKA/HMjkt4mkx5G9jyXQkUD4khs/nzCyyDvAm/LfG7Qe4S
         KIp8tW/NXWBp3aOclv4xVDrFGTtBWcQeauqnTiWckX7txbJqvCNcRxPkx41LBlJ0j6uv
         VNqw==
X-Gm-Message-State: AOJu0YytBTQlVOIDwgEkuscssnvhAR0SK7iDjUtoIlnUCI11XlX/2Vct
	4+VeSndL/cVZtwpB6HUI7Q/gW+kg0QDWMTevJSWNQcsrO7XPY2xNqx+JmDiMafmA+O0F+lZ7sd6
	B
X-Google-Smtp-Source: AGHT+IEkEoDh/AVgNK/zEUbyRj4HeKXLiM43nhLSJLRhxeMK+hvruqQDYd8uOhDgGtSNqa7Q9Z+CbA==
X-Received: by 2002:a5d:6387:0:b0:33b:81b3:78a8 with SMTP id p7-20020a5d6387000000b0033b81b378a8mr1777186wru.18.1707910914756;
        Wed, 14 Feb 2024 03:41:54 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWey/ui9yHUSZHbl2yekfFjQ4UyqIIZVpK+rsjQvNtNmInYGZTW6tKzTJKzBHZ6pdX8SDkn/d9cysiCciMLbPln1flWtSSE/UZYrUgeRLnI
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>
Subject: [PATCH 1/5] iommu/x86: fix IVMD/RMRR range checker loop increment
Date: Wed, 14 Feb 2024 11:37:37 +0100
Message-ID: <20240214103741.16189-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240214103741.16189-1-roger.pau@citrix.com>
References: <20240214103741.16189-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

mfn_add() doesn't store the incremented value in the parameter, and instead
returns it to the caller.  As a result, the loop in iommu_unity_region_ok()
didn't make progress.  Fix it by storing the incremented value.

Fixes: e45801dea17b ('iommu/x86: introduce a generic IVMD/RMRR range validity helper')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/drivers/passthrough/x86/iommu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index 1c8cf3271a09..a3fa0aef7c37 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -804,7 +804,7 @@ bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
            "%s: [%#" PRI_mfn " ,%#" PRI_mfn "] is not (entirely) in reserved memory\n",
            prefix, mfn_x(start), mfn_x(end));
 
-    for ( addr = start; mfn_x(addr) <= mfn_x(end); mfn_add(addr, 1) )
+    for ( addr = start; mfn_x(addr) <= mfn_x(end); addr = mfn_add(addr, 1) )
     {
         unsigned int type = page_get_ram_type(addr);
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:42:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:42:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680616.1058634 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeE-0005Pr-W7; Wed, 14 Feb 2024 11:41:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680616.1058634; Wed, 14 Feb 2024 11:41:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeE-0005Pk-S5; Wed, 14 Feb 2024 11:41:54 +0000
Received: by outflank-mailman (input) for mailman id 680616;
 Wed, 14 Feb 2024 11:41:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raDeE-0005Pc-EI
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:41:54 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0c9b3f0d-cb2e-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 12:41:53 +0100 (CET)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2d10ad265d5so18624511fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:41:53 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 pj35-20020a05620a1da300b00783ce19f9e6sm3686636qkn.57.2024.02.14.03.41.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 03:41:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c9b3f0d-cb2e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707910912; x=1708515712; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=c4EJXnL9GIcqm26Ox9Uwndbf51pNFjXvmgGVjTtia4M=;
        b=by+3+J4eXUIoLQJCjeI9EmGEMpAfyhihY7sXaUIvjzYSpweCE2bnipjitfk00MMIwn
         K7VNVYbwNvJ52mz1suI0hy39FroVrBTkvaaqhajk/AY5sRkg/hLHNZebUZcPcykjDgu5
         Pqpa/+IVyZbbwLXQBid4g+rhNlVv7RCte4x24=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707910912; x=1708515712;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=c4EJXnL9GIcqm26Ox9Uwndbf51pNFjXvmgGVjTtia4M=;
        b=mNA2Gq/oRkTtajE4vJjHqtStC0iWfg7NnRJ31RxVo1+qhLfMgqgaAidXw0VMRBReBM
         qtVn3tRNh1tQmz/TmIbOsJAHqIf+rR37ZP7uN68zokj8zcDFucn1kYqg+8l8wbicZhTO
         HxsKTjpHt/UHi7zOu6Ljuvt78QoGIrzgreapZ/2QikTtW6OmNaM4rRrJb6z4LM94nJai
         7K4XumSRaykW+UTs4eeH3vWWwalCCbnBdjgfp6aiNwJzYLBs2TCmuPCMDiDvbFE/Ofl6
         A2mKIHILIRhKiIH4A5815L8TaPcOEh6VTgrwqVAL+jfjm87wrt5V+6W9HJLfHcLsgUY5
         Hjbw==
X-Gm-Message-State: AOJu0Yy0Nr5T1YJFfX5nC++ke2DRKbgC3ffqJBk3s448Q6wBDEvGvG0B
	tsP8h9DfRFJQbnlj//ywHpW48vx3mm8fLQ1gy+9PToJZOFPLlyvpJtKSPS8FZJ/Xj0CrgrNOY6B
	4
X-Google-Smtp-Source: AGHT+IEPAJe00LaxPFcLum2WOIrjiERL8ZZyqWfPMdQ/7UY4MX5TO7wyMcau+RjaKHzOaGBGxpscMQ==
X-Received: by 2002:a2e:3614:0:b0:2d0:d336:d144 with SMTP id d20-20020a2e3614000000b002d0d336d144mr1724428lja.14.1707910912414;
        Wed, 14 Feb 2024 03:41:52 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWgEpQj5AGX4u5g1D6mTmXMmV9dp+XBVo/w+lsneoU8l3aPkI9FGSSOqi297O0H1lww856b1s128dEhbxuBNbkn2csMLBS+bU2yPFyRF5VJB9CJ6VBhAswKseMvIIojs8otL6mGcHuzPuIn8TlXQXYSD36NRMDP4aeKEpscA1as6jLNATfg8IitIYQz0Nbsdy8XcqGRP4J3HZ5ZXV3gFEsy1YgIYNN32rKAF0ivVZeC9EdcqqYe8g38
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 0/5] Fix fallout from IVMD/RMRR unification checks
Date: Wed, 14 Feb 2024 11:37:36 +0100
Message-ID: <20240214103741.16189-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

First patch is a fix for a silly mistake I introduced in
iommu_unity_region_ok().  The rest are additional chances requested in
that context.  Last patch adds __must_check to the gfn/mfn addition
handlers.

Thanks, Roger.

Roger Pau Monne (5):
  iommu/x86: fix IVMD/RMRR range checker loop increment
  iommu/x86: print RMRR/IVMD ranges using full addresses
  iommu/x86: use full addresses internally for the IVMD/RMRR range
    checks
  iommu/x86: print page type in IVMD/RMRR check in case of error
  mm: add the __must_check attribute to {gfn,mfn}_add()

 xen/drivers/passthrough/x86/iommu.c | 25 +++++++++++--------------
 xen/include/xen/mm-frame.h          |  4 ++--
 2 files changed, 13 insertions(+), 16 deletions(-)

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:42:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:42:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680618.1058653 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeK-0005uk-Ct; Wed, 14 Feb 2024 11:42:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680618.1058653; Wed, 14 Feb 2024 11:42:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeK-0005ub-9W; Wed, 14 Feb 2024 11:42:00 +0000
Received: by outflank-mailman (input) for mailman id 680618;
 Wed, 14 Feb 2024 11:41:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raDeI-0005Pc-LU
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:41:58 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0f56b977-cb2e-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 12:41:58 +0100 (CET)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-5116b540163so8124261e87.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:41:58 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 qj14-20020a056214320e00b0068caf901c9bsm1083852qvb.17.2024.02.14.03.41.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 03:41:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f56b977-cb2e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707910917; x=1708515717; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=73EI9rqYbQp7NZhmMdmjrgOEumsFjhm+bWFGhs3mCc8=;
        b=u5nNhFq49z7LkTqpl0/tr54Y5FMVddD654IQu/VYyV9xtTV8ZQFMeYXlS3nwLhnez8
         cMoTanBtXJnYdaP//rRViKZBzeQVjg5uJ495Z48zx8C24D8Gz0Z6Q1NPdvmhx10R7Zrt
         GDjrPCZtxC9eybwXVArQJxg6VInHzw1U0DK48=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707910917; x=1708515717;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=73EI9rqYbQp7NZhmMdmjrgOEumsFjhm+bWFGhs3mCc8=;
        b=h9B8XvAe3EVSjluAWItPVqFS0CUp3Twxq9DZuSQ2vP2oyz5DCdCI22r2APXBKGfP+Q
         3nYHQJivrr1WgWqAi5VPk8UDmNXe261jiZ6Xnuepw45QMOguNiYXsU2Y3QZBMvs+MGfI
         yC/24Q7YOBVl+SvZldb8retpwMvqAxx4yvOvGz5LNJpfQkPdd13m3ZS/qZck2liqdBPu
         3ds1euWKGMfhRcDWAdfW2U10gbJjKpb0OC0Y+yZKOqgE4tk0sW/scVZPRtiZGos2eFqu
         QLwcXUjIN5zT+pMBYwRv3Ru1z9Am0chWZG7zrbbLdAU2hFlTAf5PQaIJj6vPv+/dDWex
         eonw==
X-Gm-Message-State: AOJu0YzqwTYR2QQf13xYbo/77A7q1jtq9Yudjk688DAajN+RVj0D6N5f
	BSn5Z7baBB4sQpZ6I7+lK26N5JdwjhvJ0DFK2ie8wE3EJwSrDrBuJVS4E/UBjHR27lcrI0dznHA
	C
X-Google-Smtp-Source: AGHT+IG0tJEU3fhJjIJ8NVl/QPwMLbE9iPLosJcRNbPHOIyIxybONmcE5i6MHKjeI9QyOeaQmhEJBg==
X-Received: by 2002:a05:6512:4843:b0:511:7f28:f8e8 with SMTP id ep3-20020a056512484300b005117f28f8e8mr1510839lfb.63.1707910917127;
        Wed, 14 Feb 2024 03:41:57 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVfH3/g35RuDyhvzIfATZTTLIR2rJynU8TdLm5oAGqUbVOCHC9mYTayvxWXVzvpn9gwU/oGC2j9x/ljvXoGP3Z7IqmXEDmRGsK+oixE8XlwVsVqE9OtSiJ38X+XqfSohaj1ZzbBpfu+Re8j
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH 2/5] iommu/x86: print RMRR/IVMD ranges using full addresses
Date: Wed, 14 Feb 2024 11:37:38 +0100
Message-ID: <20240214103741.16189-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240214103741.16189-1-roger.pau@citrix.com>
References: <20240214103741.16189-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It's easier to correlate with the physical memory map if the addresses are
fully printed, instead of using frame numbers.

Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/drivers/passthrough/x86/iommu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index a3fa0aef7c37..304a2f5480c7 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -801,8 +801,8 @@ bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
         return true;
 
     printk(XENLOG_WARNING
-           "%s: [%#" PRI_mfn " ,%#" PRI_mfn "] is not (entirely) in reserved memory\n",
-           prefix, mfn_x(start), mfn_x(end));
+           "%s: [%#lx, %#lx] is not (entirely) in reserved memory\n",
+           prefix, mfn_to_maddr(start), mfn_to_maddr(end));
 
     for ( addr = start; mfn_x(addr) <= mfn_x(end); addr = mfn_add(addr, 1) )
     {
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:42:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:42:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680620.1058663 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeN-0006DT-Ka; Wed, 14 Feb 2024 11:42:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680620.1058663; Wed, 14 Feb 2024 11:42:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeN-0006DK-GY; Wed, 14 Feb 2024 11:42:03 +0000
Received: by outflank-mailman (input) for mailman id 680620;
 Wed, 14 Feb 2024 11:42:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raDeM-0006Bc-Jn
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:42:02 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 10b392fe-cb2e-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 12:42:00 +0100 (CET)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2d0fd07ba8bso35152041fa.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:42:00 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 bi38-20020a05620a31a600b007872ed24739sm180347qkb.31.2024.02.14.03.41.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 03:41:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 10b392fe-cb2e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707910919; x=1708515719; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SKL9Cv8WHcDaH4CP4Lonu8yBimilzz26H15juTt/ZxA=;
        b=nEe4aLtez9IXm9cX4kSlhp+yFykMDFn90VDMvr6IQYhSFu7r4oYvd8Gr5PBSRrD4J6
         nbXzmt2sasTjduN8+bY2oS03WoEHuz7nVCXV9EYgrRMF9vXAD/7py+ZJk03Umc60trLp
         TxhHGiRAphBwZSVlHTFC658VQaB24Agv9yepY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707910919; x=1708515719;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=SKL9Cv8WHcDaH4CP4Lonu8yBimilzz26H15juTt/ZxA=;
        b=wId2IcD2gy8n4kVxlDO9PM2YrmO14VGM/Y70zTtxK/rdVAR8wW01ypWXNy+RpIErj3
         4UxMJ422hImjkY6jtDjg5SpCg2vZ3xmrMoXIaC3gOOvItrrtZJNiyQoNomu3DO39wrni
         xVpwx0oJxFIMkZmiGXn7B17GujxouVHdBE50dn2gx0dBxaLFu9M8gkH7tBV1SOW0SZd9
         pwt7OSsudSAaTCj6ZunBNPDWjVpPhjGKd4VDuKI+c/p9ZFbkoCYH93I7P+HQoCeYVDeP
         BmPWR4gcLAfSXnEvLNLbhoFjrumWxLuOYEwP28N3+m3oUCUI/sQfrhO7bcNhK3D2TXnF
         Nixw==
X-Gm-Message-State: AOJu0YyVgWot4iQhzV/4/w5tjh9qvkeXhjPzBuRQuX9Ivc/VLDXNDus/
	VjgMuUXC3VfqSGiCVYicTb3IUyOk8gbExgw2TxEBkOktENOEk5h7cRv1jR70WtqK+oQ6NC6WhcR
	o
X-Google-Smtp-Source: AGHT+IHUsXy8zpz6Pkru0Y+FJ4gDPCuKjJ3J88fc19kB1QXCFNBgs5JRwin6PaWa+OJZ0ev+OgzL6g==
X-Received: by 2002:a2e:9bd7:0:b0:2d0:b464:a9b1 with SMTP id w23-20020a2e9bd7000000b002d0b464a9b1mr1782333ljj.4.1707910919383;
        Wed, 14 Feb 2024 03:41:59 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVoumRfLLegYsfIk0OlRRhL1cxE/bp5SDNpmxtiua8EizwJKwUX8BArraMIUl0wj0hJ2jKP0AtR05o5kO9FmvnRBd9+V+rSutBzGOkNv+cfPzEtV1HNQV1uxM2VRTQllPnC+ZptTBTenXNm
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH 3/5] iommu/x86: use full addresses internally for the IVMD/RMRR range checks
Date: Wed, 14 Feb 2024 11:37:39 +0100
Message-ID: <20240214103741.16189-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240214103741.16189-1-roger.pau@citrix.com>
References: <20240214103741.16189-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Adjust the code in the checker to use full addresses rather than frame numbers,
as it's only page_get_ram_type() that requires an mfn parameter.

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/drivers/passthrough/x86/iommu.c | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)

diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index 304a2f5480c7..e713cf803e8a 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -794,28 +794,26 @@ __initcall(adjust_irq_affinities);
 
 bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
 {
-    mfn_t addr;
+    paddr_t s = mfn_to_maddr(start), e = mfn_to_maddr(end);
 
-    if ( e820_all_mapped(mfn_to_maddr(start), mfn_to_maddr(end) + PAGE_SIZE,
-                         E820_RESERVED) )
+    if ( e820_all_mapped(s, e + PAGE_SIZE, E820_RESERVED) )
         return true;
 
     printk(XENLOG_WARNING
            "%s: [%#lx, %#lx] is not (entirely) in reserved memory\n",
-           prefix, mfn_to_maddr(start), mfn_to_maddr(end));
+           prefix, s, e);
 
-    for ( addr = start; mfn_x(addr) <= mfn_x(end); addr = mfn_add(addr, 1) )
+    for ( paddr_t addr = s; addr <= e; addr += PAGE_SIZE )
     {
-        unsigned int type = page_get_ram_type(addr);
+        unsigned int type = page_get_ram_type(maddr_to_mfn(addr));
 
         if ( type == RAM_TYPE_UNKNOWN )
         {
-            if ( e820_add_range(mfn_to_maddr(addr),
-                                mfn_to_maddr(addr) + PAGE_SIZE, E820_RESERVED) )
+            if ( e820_add_range(addr, addr + PAGE_SIZE, E820_RESERVED) )
                 continue;
             printk(XENLOG_ERR
-                   "%s: page at %#" PRI_mfn " couldn't be reserved\n",
-                   prefix, mfn_x(addr));
+                   "%s: page at %#lx couldn't be reserved\n",
+                   prefix, paddr_to_pfn(addr));
             return false;
         }
 
@@ -829,9 +827,8 @@ bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
                      RAM_TYPE_UNUSABLE) )
             continue;
 
-        printk(XENLOG_ERR
-               "%s: page at %#" PRI_mfn " can't be converted\n",
-               prefix, mfn_x(addr));
+        printk(XENLOG_ERR "%s: page at %#lx can't be converted\n",
+               prefix, paddr_to_pfn(addr));
         return false;
     }
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:42:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:42:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680621.1058673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeO-0006Tx-T6; Wed, 14 Feb 2024 11:42:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680621.1058673; Wed, 14 Feb 2024 11:42:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeO-0006Tq-PI; Wed, 14 Feb 2024 11:42:04 +0000
Received: by outflank-mailman (input) for mailman id 680621;
 Wed, 14 Feb 2024 11:42:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raDeN-0005Pc-1p
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:42:03 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 11f41b25-cb2e-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 12:42:02 +0100 (CET)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2d0a4e1789cso63429811fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:42:02 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 t27-20020a05620a035b00b00785367e5a93sm3725905qkm.77.2024.02.14.03.42.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 03:42:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 11f41b25-cb2e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707910921; x=1708515721; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zbHWtGQxe+lF1Oh/RMVx/RWGcnqoQ1bvQB9vtnF0JLs=;
        b=EYpQ//rDxIqwy1B+4GD0x8hPSRnVtDTLHzBFTJ2Hnx5DJNTCU9yFTcH4C1thxsqL6a
         W4SRlMeEmgwD1Mhyu/N0Nq4UqaD672Yf0Ajev47TOmgsd8iHmTEtSg+o33jgDcLud5DU
         4n9Wa2knLdio+MxODqUcvz/LU7NHkt5UnHL2Y=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707910921; x=1708515721;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=zbHWtGQxe+lF1Oh/RMVx/RWGcnqoQ1bvQB9vtnF0JLs=;
        b=mKVgHBj5O+Cj9CtuHET/JOD7OQyhhCCvUUIBefapmqvqH6Abp3r7Fo5fCNJWLDfLTC
         Kvvh97/D5IXSSAQJcspcTCpxkD3IKLyWx1REP1j8vAOUt26ElA21ibfDLJ4+BAWKDN7d
         Ba6mpFct3wro5GiPIxje7kSjJaDlhJhxKYavvdu1qwUBzJ6dTiKPOCvBKwHfx8VzplxL
         NCj5/5/oYyFYd+XfIx/jjIk0OsXtHBRkoFIuYVHg/1J1YhqDQbchg9ZNLcqQZI1qjqCp
         Q6wFcX6pHczbyeL/2QDS6yl7WCcorbnnPwOtA35X3nnX78ZlEK2dqj/RKCEfNxcqTL0z
         z53g==
X-Gm-Message-State: AOJu0Yz/LRmKR4oVSN5pQ3YcAyR9lkgNnjwXYHMWaJWo+4c7wTpi5qGk
	PdnTQMKSAaFYvnSNxgw+zMBCEmUBNt46c87Dw20esoaRgbeWychd9CXPlIJETwb7rKTg3WdL73K
	C
X-Google-Smtp-Source: AGHT+IGit4GnXlDQsJF49FMFMw89YetTn+GHy8FchTC0tzsg9sECFBGs3fbenu35rI3rLBMV31tpzA==
X-Received: by 2002:a05:651c:151:b0:2d0:f8d7:a91c with SMTP id c17-20020a05651c015100b002d0f8d7a91cmr1741911ljd.52.1707910921603;
        Wed, 14 Feb 2024 03:42:01 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUIoLUKXv+O3lO+DQm8w35xbRtpWa1TiHlLqcDtv8HdUSV3S5HOln3aAlDvmfHXGpKEXTs7ptD7YkFg0uKVcLfTuoxR2AZ6VnQO3W3/WF61dzMvtunXuzIzsKFv35Xl+m2yov/Lms2Klb44
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH 4/5] iommu/x86: print page type in IVMD/RMRR check in case of error
Date: Wed, 14 Feb 2024 11:37:40 +0100
Message-ID: <20240214103741.16189-5-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240214103741.16189-1-roger.pau@citrix.com>
References: <20240214103741.16189-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Provide more information in case the page can't be converted, and print the
original type(s).

Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/drivers/passthrough/x86/iommu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index e713cf803e8a..217409c29644 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -827,8 +827,8 @@ bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
                      RAM_TYPE_UNUSABLE) )
             continue;
 
-        printk(XENLOG_ERR "%s: page at %#lx can't be converted\n",
-               prefix, paddr_to_pfn(addr));
+        printk(XENLOG_ERR "%s: page at %#lx can't be converted (type %#x)\n",
+               prefix, paddr_to_pfn(addr), type);
         return false;
     }
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:42:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:42:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680622.1058683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeR-0006nM-78; Wed, 14 Feb 2024 11:42:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680622.1058683; Wed, 14 Feb 2024 11:42:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDeR-0006n7-23; Wed, 14 Feb 2024 11:42:07 +0000
Received: by outflank-mailman (input) for mailman id 680622;
 Wed, 14 Feb 2024 11:42:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raDeQ-0006Bc-07
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:42:06 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 13158367-cb2e-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 12:42:04 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d0d7985dfdso75380661fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:42:04 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 bl20-20020a05620a1a9400b007871de8ec59sm1213626qkb.129.2024.02.14.03.42.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 03:42:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13158367-cb2e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707910924; x=1708515724; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GJNvqG2IGk9PsE23BxVL4PP6tdgbjCcG8XCSUoGj9SA=;
        b=fYam6ZkLBgALIdNuWAFx7UiX271DdFtImeyeOUyx0kknu/6atDJK797eneyGEVVYhS
         Re7FTwNIL6a7pRFuGgV2Iyr+2hpPlEB7IXFqUl91lG075/qjkC0lMAiyo+yDzpdRYj7E
         dH2BKL9+62Ej1Omt6nIJkUIPx44P07ojnFS/4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707910924; x=1708515724;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GJNvqG2IGk9PsE23BxVL4PP6tdgbjCcG8XCSUoGj9SA=;
        b=Jv1OUFNCs97A6mBuOwtiX8IwfECfW+cxMgtARfOKdbg2I09j5QnwKkGSmdrSJot9Yo
         fvym4KYl37Zbb+HKlqKILlXnPuLn4CkwTs1wEIq+Pq0VabNapjYSYBT44eFEwW5sdHhh
         JE3R5NwmPgUy6z5JenAXkSARBZWWtacd/GMsz1cZ3XXBQHYfiDaTNtGotEVobFMqQt/2
         5+n1EDtMe8HXspF77yEV7Ju5Tzerm15h+WMBUnDaJujgdKkMpNTidvJRr5f797q/w/CQ
         ZpEi/FA6DWJEcbyZLlIphEI04JRhEQIop7w2AIYDDDFx0Q6B3lsi0/9BYzzmPldanmik
         zXQg==
X-Gm-Message-State: AOJu0YwNYjinrEf71GABLCv30e9q9xxP6NHdHcR/nxqbKtKKfV6Oi9oZ
	6jXy6f7JvR0ER0b/TQlDMgAkSpO9mCuBBhA6g8pb5qToX6b6s+yRRxPgxmdQYrZoB9zCEJJQbm7
	V
X-Google-Smtp-Source: AGHT+IERu49AGLIkPi283NJAq1+/A4S0ucosD9AQdE9Pv8SPP/1s0udEm7HZ9T+ks4n26Nx+Dh+bSQ==
X-Received: by 2002:a05:651c:1425:b0:2d0:aa28:6f77 with SMTP id u37-20020a05651c142500b002d0aa286f77mr1534669lje.45.1707910923934;
        Wed, 14 Feb 2024 03:42:03 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUiClLFOh55PWam1Jc8L+bOL+KWodINot4ElxIAPkcWyyWduLUh/Prog/oIShPDXpmPyM3yEHr7dTmShha8Q3352O8rjla6S86V9p2imJdeYf6w26ogCW+bDGotQ1PWpRU9QYo/3P1p28QE3F9akc6c0XsZnqZDq11Gfk7rmbmUlusoxxZmcFHZ6gcDyeeTzaTBbc1XbKZZoEFkRSmCp1gcSLuILQsOZvtiYQ==
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 5/5] mm: add the __must_check attribute to {gfn,mfn}_add()
Date: Wed, 14 Feb 2024 11:37:41 +0100
Message-ID: <20240214103741.16189-6-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240214103741.16189-1-roger.pau@citrix.com>
References: <20240214103741.16189-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It's not obvious from the function itself whether the incremented value will be
stored in the parameter, or returned to the caller.  That has leads to bugs in
the past as callers assume the incremented value is stored in the parameter.

Add the __must_check attribute to the function to easily spot callers that
don't consume the returned value, which signals an error in the caller logic.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/include/xen/mm-frame.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/include/xen/mm-frame.h b/xen/include/xen/mm-frame.h
index 922ae418807a..c25e836f255a 100644
--- a/xen/include/xen/mm-frame.h
+++ b/xen/include/xen/mm-frame.h
@@ -23,7 +23,7 @@ TYPE_SAFE(unsigned long, mfn);
 #undef mfn_x
 #endif
 
-static inline mfn_t mfn_add(mfn_t mfn, unsigned long i)
+static inline mfn_t __must_check mfn_add(mfn_t mfn, unsigned long i)
 {
     return _mfn(mfn_x(mfn) + i);
 }
@@ -62,7 +62,7 @@ TYPE_SAFE(unsigned long, gfn);
 #undef gfn_x
 #endif
 
-static inline gfn_t gfn_add(gfn_t gfn, unsigned long i)
+static inline gfn_t __must_check gfn_add(gfn_t gfn, unsigned long i)
 {
     return _gfn(gfn_x(gfn) + i);
 }
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:46:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:46:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680646.1058692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDiB-0000EL-VO; Wed, 14 Feb 2024 11:45:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680646.1058692; Wed, 14 Feb 2024 11:45:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDiB-0000EE-Sg; Wed, 14 Feb 2024 11:45:59 +0000
Received: by outflank-mailman (input) for mailman id 680646;
 Wed, 14 Feb 2024 11:45:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Sn1=JX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1raDiA-0000E8-6J
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:45:58 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9dd9986c-cb2e-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 12:45:57 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a38271c0bd5so684742966b.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:45:57 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 s22-20020a170906285600b00a3d6590e4d7sm98842ejc.84.2024.02.14.03.45.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 03:45:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9dd9986c-cb2e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707911157; x=1708515957; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:subject:from:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=w/Mu3JHm0P4wjycZ6WMNTmkQvj92o0r1X1GhbBbxUyM=;
        b=Mgbf4zoHumiITNIOsF+rLnHDAs8bDXi9LORQPDvLXusjyJD+paF+5RczfQVuz0q3sg
         Sbx1Z9PtEY6fftKGUbUoXTMCh0Yd9badFGG4JU32/psk604cslQwY0/nWFzpPWKPL63T
         FrTmWvphXfAHc+CyOMMq3M6KXBMObXfk/d1ho=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707911157; x=1708515957;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:subject:from:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=w/Mu3JHm0P4wjycZ6WMNTmkQvj92o0r1X1GhbBbxUyM=;
        b=TTCEuloNAHVz0x3kfaMUyOM0OBVbPQKzULBaEDVjXtS6bR3eg5hN8BW3QsqupBXiXG
         XAdMdTQLMiMgaGnypRC251Umrxw6Y2F+pf2BXsGcm6sQWVqM4z+3roS6ihOIhheBpVGA
         ACrjY2B8k7Y4EbM+8Nje37xp3oAS6rRwY1g7JpHs1w7XxVJdoFK5DKYCXUrxAw2GOGVa
         aPaDlgJUmJY+R0WF58MXC49EstX63stcRN8eEezbxitwMMhYNDbfNXH01QNTOf5AFLs+
         WEqq8L9fvoYdxQ4EJmVekzDZKjcpXXMb3BbExLJ/6kzQiCqgdf3at/8ik18xWevymRWx
         Wilw==
X-Gm-Message-State: AOJu0YwoTfTXkUqNelOcQA5IZuvLUOx3s41GdsbStkZCTlN62cN98x6C
	D1PlKZUyd4pQ0hnuZEJ/fVVuMKvzRkLFyQHvV0unXLYQXVBbdGz3YtAzgm7jMls=
X-Google-Smtp-Source: AGHT+IHBHws9hnAQs/21HEXnOV9LVtBy31PavabBJuaj1V/U/poOWA7qiTU93uV/O1IfJKvJjOy5/g==
X-Received: by 2002:a17:906:dc0a:b0:a3c:c980:5936 with SMTP id yy10-20020a170906dc0a00b00a3cc9805936mr1722493ejb.24.1707911156876;
        Wed, 14 Feb 2024 03:45:56 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUm87J+Fx14s8ChPQl/T6DKxtoD7gX1Xc1TPuV6lwXF2azO1Ebgm5YdR78n9regKIMlwSW30ieN2hXohYsOqnNMVUm+QLwHL8IEjpgyHoLS882LsdAy/TJEvMIgq9LQ7vZGEKBp00illLGl+Am92Agf0Kql4+3frpytGzjnsj9uM9AH7X81XUgR5YchVfaNfjAWCv18Tw==
Message-ID: <046415d5-d4f4-4870-a588-60637ea3bc3a@citrix.com>
Date: Wed, 14 Feb 2024 11:45:56 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH] tools/ocaml: Add missing vmtrace_buf_kb field
To: Jan Beulich <jbeulich@suse.com>,
 Christian Lindig <christian.lindig@cloud.com>, =?UTF-8?Q?Petr_Bene=C5=A1?=
 <w1benny@gmail.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, David Scott
 <dave@recoil.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>
References: <9b9909c9e93cb540b3488c784935acc2bc9e071e.1707343396.git.w1benny@gmail.com>
 <3A858D7F-C953-4EF0-8919-AE96D6105AB1@cloud.com>
 <01b0d902-1903-4618-ad43-f625e57b61e1@suse.com>
Content-Language: en-GB
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <01b0d902-1903-4618-ad43-f625e57b61e1@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/02/2024 7:11 am, Jan Beulich wrote:
> On 08.02.2024 10:13, Christian Lindig wrote:
>>> On 7 Feb 2024, at 22:04, Petr Beneš <w1benny@gmail.com> wrote:
>>> Add the missing `vmtrace_buf_kb` field to the OCaml bindings to match the
>>> vm.cfg configuration, correcting an oversight from its initial introduction.
>>>
>>> Signed-off-by: Petr Beneš <w1benny@gmail.com>
>> Acked-by: Christian Lindig <christian.lindig@cloud.com>
>>
>> This looks correct from an OCaml perspective. Why was the new field added in the middle of the record type domctl_create_config and thus forcing changes to the index of fields coming later in the record versus just appending the new field to the record type?
>>
>> The critical bit is using the correct type in "Int32_val(VAL_VMTRACE_BUF_KB)” that matches the type "vmtrace_buf_kb: int32;” - which it does.
> Is this then perhaps also lacking a
>
> Fixes: 45ba9a7d7688 ("tools/[lib]xl: Add vmtrace_buf_size parameter")
>
> and hence wanting backporting?

It ought to have been part of that at the time, but I suggest not
backporting.

API changes in Ocaml are all breaking (module compatibility is checked
by hash).

Xapi is the only consumer of this interface.  I've fixed up the build
against staging, but we're not going to be running KFX under Xapi any
time soon.

Ultimately it's Christian's call.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:50:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:50:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680652.1058702 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDm2-00015Z-FV; Wed, 14 Feb 2024 11:49:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680652.1058702; Wed, 14 Feb 2024 11:49:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDm2-00015S-Cb; Wed, 14 Feb 2024 11:49:58 +0000
Received: by outflank-mailman (input) for mailman id 680652;
 Wed, 14 Feb 2024 11:49:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raDm1-00015M-8I
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:49:57 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2b44b4d6-cb2f-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 12:49:54 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33ce2121d5dso635717f8f.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:49:54 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 x8-20020a5d6508000000b0033b1b01e4fcsm12195893wru.96.2024.02.14.03.49.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 03:49:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b44b4d6-cb2f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707911394; x=1708516194; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SJH9x7u+fZSVOBTrC0UwZ3YggKvS80omYArXBXtxJHU=;
        b=H1EieQ7bMo3kgSbhTxZhvq7sHWInGzcbJpidQ4imgAbGlAkQtUk4Ecs2NarwMt4PLR
         b+IL+vFKaho+ATeInkCCtzFGME4NnxT4yCOixixiP0Qxz6fXLki3VprRYr8uZsi2OYA2
         mZnLlVrQFdvb7ohFLesCxGn6+/0fVwx8oKrSSdBXntH2VNnqc8Dca9IiIXaJz/jcICoS
         6IqRo7NiqDtPy3L0Le3IEjK2JwFryKD1qofCQeCPVDT4QQ2hYd5H/YzIgm7FcHI+4RvY
         1vU2M8ycUSyfykPU4J3NAyoJ9bzJ7O3dn647WzYb7wCw/Umq14VU01i+x+7Z2eUBpE15
         tSNQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707911394; x=1708516194;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SJH9x7u+fZSVOBTrC0UwZ3YggKvS80omYArXBXtxJHU=;
        b=Z1dYehu7ywytVxIPUwB01c5scqgoaD3i8DqbNByvtJve00z7enlkaNEeKdl8HwekSN
         RvUnV31kcNpfcJgnsp4Scg3sZaosk42TsTe5wcOpXINC5rFl1OGM03oeBk4kdQorfEiW
         c/ZBZsDmC3GFb1EPfEl4Ka17e4GGcDJwFykeLc5A7QCetoXJc/PRs4OQAtZ1GuU4PGd5
         F7PPeABuZa0I4rsLeVqpLynhAtn4x9YKSVWF5th3/Wz5OGLQQ1jVOOxzDzEid3TZn/sJ
         846iDfAX5Ws9whzQQCJOrqbdFY0Zq0W2Sfn0OmSEdsWHmzq48k3o0ofXKV1hNEqQL7ih
         5OdQ==
X-Forwarded-Encrypted: i=1; AJvYcCVV5IO79RufB2fXN5vwumdORDVSKjEnl9Dxvqdy7hgfYNmGozWcXdafbd+kF3SG07IaJzT6b9Az/OU54NN+wn5ouhqVuZ+Y90KRuJCIdTw=
X-Gm-Message-State: AOJu0YzBV1gBmO+Oq+4yBr/6ifP8p0i4d/mGNZrIzE9ggPO3BZ3ETqxu
	2H+4HkhqgnwvLXT2GhRZfCSQ/+cEHm9ev457HoySft9Qkh02RW8FFwkSiWvWqQ==
X-Google-Smtp-Source: AGHT+IHwwKbUV9JDCU4JG0tCYI09ZrfDDscYPkxxinZsIrhZt9zxwYnZgSI8Jux4udD6KYtbd7xJzg==
X-Received: by 2002:a5d:464a:0:b0:33c:e48e:8cc1 with SMTP id j10-20020a5d464a000000b0033ce48e8cc1mr1621430wrs.0.1707911394130;
        Wed, 14 Feb 2024 03:49:54 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVvFw6M3o1YUycVkoeVW/vwPFZATxkgr8PEPtSLLZYJlcDh6/JEDzyt/IF7A6if4mocJJQgeh/rYMEbEX6LCAGk0eCW1pCM6sSu3uBuyamp5hIgrQ8QOj69LAWgYuqsKxp6AATfqT5MbaAKNvUW9n0ruzviRK0/qlbRqYFUXNidsF8Gl/oOTswp0N3plNe6MX7sdGJxI/UaV5/qx5gv/yQInR9td9YBnmlAZeJrex8h75GQ01Lu+WH8f7Y9pYYFrQ50OV1/jsiYG1EZcfWMxbdLHoCI2q8B1NsHR5heHoVQvOGaba3cRtfXv7h5dXmZk9Q3nBZRjJTfaCelDpoesrc/XPEtpCcap3c0CCgYtQKtLyy7Lrpi2T5lTNRoYCoSmUARhh0XcjpbxorV0ET4clF+w0vAW8fj7yuO1hETuetFw3KrSoilMRKmldLOyWPQhifUZBdrQixelwjcdIAGyGCCvkk=
Message-ID: <3519db82-6126-4aa0-9d04-795edf6f2bca@suse.com>
Date: Wed, 14 Feb 2024 12:49:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair_analysis: deviate certain macros
 for Rule 20.12
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>, George Dunlap
 <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <892be5c403ca7a20b35fb9facacb6a38bc7f6bfe.1707900742.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <892be5c403ca7a20b35fb9facacb6a38bc7f6bfe.1707900742.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.02.2024 12:26, Nicola Vetrini wrote:
> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> @@ -387,6 +387,16 @@ in assignments."
>  {safe, "left_right(^[(,\\[]$,^[),\\]]$)"}
>  -doc_end
>  
> +-doc_begin="The token pasting in variadic macros cannot be replaced."
> +-config=MC3R1.R20.12,macros+={deliberate, "variadic()"}
> +-doc_end
> +
> +-doc_begin="Uses of # and ## operators within the following macros are
> +deliberate."

I don't think this is a good explanation. The use of those operators is
deliberate elsewhere as well.

> +-config=MC3R1.R20.12,macros+={deliberate, "name(ASSERT||BUILD_BUG_ON||BUILD_BUG_ON_ZERO||GENERATE_CASE)"}

I said in another context already that it is necessary to separate
global scope macros from local helper ones. Any CU can introduce another
GENERATE_CASE(), and would be deviated here right away. In fact I
question applicability of the deviation to arm/arm64/vsysreg.c; I only
see it as applicable to arm/vcpreg.c.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 11:51:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 11:51:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680655.1058713 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDng-0002ST-QE; Wed, 14 Feb 2024 11:51:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680655.1058713; Wed, 14 Feb 2024 11:51:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDng-0002SM-Nb; Wed, 14 Feb 2024 11:51:40 +0000
Received: by outflank-mailman (input) for mailman id 680655;
 Wed, 14 Feb 2024 11:51:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raDnf-0002SG-7D
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 11:51:39 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 68d69a21-cb2f-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 12:51:38 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-411f439fb93so2318265e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 03:51:38 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 t9-20020a7bc3c9000000b004107dfa6aebsm1693947wmj.28.2024.02.14.03.51.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 03:51:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68d69a21-cb2f-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707911497; x=1708516297; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nNUAQ+mMR1wBg/kRyeaLtF6OpyXmqNra9TWwqKRTVJI=;
        b=e9Xy29ufvnl2Ro8Me2Yrb8DqM//uzvzQrujA1XO2kaatVoicg6VFUM5eEQF2flTJH0
         JvKfRUsU6O+2nHU3WTjbUJhAmZ+yd1dA57nLJwdeTcJvL42i0YVTPURI7OzX768oPtzA
         hcRauZzZe0tJC7aUzzFN723ae45/6r8/tDEpPF00ovfxoYDRO0vTW5IBFUX81VwtvI2m
         owKr30jctx5bISLHwUxKpxmfvUiKHSaLNG7nWuh9OCPD9Kjm1y0xFNxeJES/WPaJxizX
         +xWGjQeZaR2vElgOCdCGp+QnsnDZNNCpkilj0GFTxtTHSn2iyLYeQGUu0AZ9H9G7uqpP
         YOFQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707911497; x=1708516297;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nNUAQ+mMR1wBg/kRyeaLtF6OpyXmqNra9TWwqKRTVJI=;
        b=JpkoTvmkgW20BE2cJ2uAF5X0locE4lvDYFqlohZK/HyNxcaQfZoZ4IdxkPcRXSHcrM
         gqe2iLmKfUPRfhlH3cIaFdQawSzDaBY8DUmZDhbZDvdnEJjr1sdh0qwZ6n6d97QIlrW7
         lhtHGQRmKhY3fHkax/YRk+b/VGFFw8Qnc4eKRpzGNMa8cgnGRRxzZ+Dtp1fZSGUyAQHG
         PQV5srx4iTuWXyYgd0Fz+6nkF3YXiUJykLfWuoH9tbgjHKQxiiW6c1twG7sWmIgalgEM
         9QQBhXWaoVeD7+dk0+qrzePmY3lg29HW4g4K5aKnMiEy8dZa7XQUj2mZWr/N+qsrUsYJ
         GNpQ==
X-Forwarded-Encrypted: i=1; AJvYcCUCzJiqhNKE5RYEzD6x4gtt2oTLb1ao0me4hsgexNJll+RSHGRfcrsmz24htaC3zENJQIlOpqRi+6to6CUxqzAM/ojDbPrKtWYWlNoQuq0=
X-Gm-Message-State: AOJu0YxW5T28EoEi9H5CfGKhOLXjmWBTyg8Mkkoe1prHzHffSfgxQDWQ
	kBieX1G9wOUJd/HTXjvzs2oySzIx2BawiMpHP/TeGx4ZU9FxUioseTTsU2RJlw==
X-Google-Smtp-Source: AGHT+IGqj1X5ZifK2e592AURiNdU0gNycxGKhQ7W/Kety7J2/a2zIcH5Hn7464nYugMpurUB4cN2Qg==
X-Received: by 2002:a05:600c:1911:b0:410:cf70:c537 with SMTP id j17-20020a05600c191100b00410cf70c537mr2178143wmq.4.1707911497435;
        Wed, 14 Feb 2024 03:51:37 -0800 (PST)
Message-ID: <b2433393-1a04-4570-abe6-0356db9976f8@suse.com>
Date: Wed, 14 Feb 2024 12:51:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/5] iommu/x86: fix IVMD/RMRR range checker loop increment
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-2-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214103741.16189-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 11:37, Roger Pau Monne wrote:
> mfn_add() doesn't store the incremented value in the parameter, and instead
> returns it to the caller.  As a result, the loop in iommu_unity_region_ok()
> didn't make progress.  Fix it by storing the incremented value.
> 
> Fixes: e45801dea17b ('iommu/x86: introduce a generic IVMD/RMRR range validity helper')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

Should Andrew get a Reported-by here? And surely we want to list the
Coverity ID as well? (Happy to take of both while committing, so long
as you agree.)

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 12:02:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 12:02:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680677.1058722 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDy0-0005eU-4h; Wed, 14 Feb 2024 12:02:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680677.1058722; Wed, 14 Feb 2024 12:02:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDy0-0005eN-29; Wed, 14 Feb 2024 12:02:20 +0000
Received: by outflank-mailman (input) for mailman id 680677;
 Wed, 14 Feb 2024 12:02:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1raDxy-0005eH-BK
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 12:02:18 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1raDxy-0002Ab-54; Wed, 14 Feb 2024 12:02:18 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1raDxx-0006Lh-TW; Wed, 14 Feb 2024 12:02:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=YRM/j+EA91QLN+z8xGAnc5ab9MB8UUZAVjVKosB1qus=; b=54PH/F8O8KJeuElzWEZahHU/Nd
	wW8LIxFepqRis+nSmkltT/h+8B4EV3DcsXKxCHnOCo9PqM5oeTkh+vRK5DfafHzmX+amwCnS5xCPV
	IUPv68/gsmgg5QYSRsvtgX7T4gmSmYjpeU+ji2D0avZmUMs3yy8U8nSWB5EZl4N0pP2I=;
Message-ID: <aa89e2b8-7251-4cd4-a2f9-cfb32a642af3@xen.org>
Date: Wed, 14 Feb 2024 12:02:15 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/5] mm: add the __must_check attribute to {gfn,mfn}_add()
Content-Language: en-GB
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-6-roger.pau@citrix.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240214103741.16189-6-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 14/02/2024 10:37, Roger Pau Monne wrote:
> It's not obvious from the function itself whether the incremented value will be
> stored in the parameter, or returned to the caller.  That has leads to bugs in
> the past as callers assume the incremented value is stored in the parameter.
> 
> Add the __must_check attribute to the function to easily spot callers that
> don't consume the returned value, which signals an error in the caller logic.

I like the patch. We should add more __must_check in the code :). I 
think ECLAIR will also help us to spot any place where we don't use the 
returned value.

> No functional change intended.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Acked-by: Julien Grall <jgrall@amazon.com>

> ---
>   xen/include/xen/mm-frame.h | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/include/xen/mm-frame.h b/xen/include/xen/mm-frame.h
> index 922ae418807a..c25e836f255a 100644
> --- a/xen/include/xen/mm-frame.h
> +++ b/xen/include/xen/mm-frame.h
> @@ -23,7 +23,7 @@ TYPE_SAFE(unsigned long, mfn);
>   #undef mfn_x
>   #endif
>   
> -static inline mfn_t mfn_add(mfn_t mfn, unsigned long i)
> +static inline mfn_t __must_check mfn_add(mfn_t mfn, unsigned long i)
>   {
>       return _mfn(mfn_x(mfn) + i);
>   }
> @@ -62,7 +62,7 @@ TYPE_SAFE(unsigned long, gfn);
>   #undef gfn_x
>   #endif
>   
> -static inline gfn_t gfn_add(gfn_t gfn, unsigned long i)
> +static inline gfn_t __must_check gfn_add(gfn_t gfn, unsigned long i)
>   {
>       return _gfn(gfn_x(gfn) + i);
>   }

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 12:04:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 12:04:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680683.1058733 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDzt-0006Of-FV; Wed, 14 Feb 2024 12:04:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680683.1058733; Wed, 14 Feb 2024 12:04:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raDzt-0006OY-Cw; Wed, 14 Feb 2024 12:04:17 +0000
Received: by outflank-mailman (input) for mailman id 680683;
 Wed, 14 Feb 2024 12:04:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raDzs-0006OP-Dm
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 12:04:16 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2c7f8980-cb31-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 13:04:15 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-5118d65cf9cso3940626e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 04:04:15 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 h22-20020ac85056000000b0042dbbe27a55sm430962qtm.66.2024.02.14.04.04.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 04:04:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c7f8980-cb31-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707912255; x=1708517055; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=UDsEfgwp8wcDvHYNMbE3kESDb0kSmC9ET3bXxl+wzto=;
        b=G9itvYDx3F3wE38tqRhfCDpkblVEsUkxk/ctvLLQf2zAVgbUo6EV3oZ3QFM6TQ32Lo
         erFk9i9IMqgwrpAYqPCQ3ERDZCIi/6nZfWYbjionmhfZiCUT+gdz0b/hY+ob8/MHTrPW
         HhVBn7pU3pZ7NCPLKJJEey+N/wtfXlybvKNa8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707912255; x=1708517055;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=UDsEfgwp8wcDvHYNMbE3kESDb0kSmC9ET3bXxl+wzto=;
        b=WSw8vc0MYsqZXx8TxuifdhClK3TR4nR39Eme1Edoj8G49sRXQDOuPhM2HDvLt7F730
         kNweqSI7UlzRrFWtXCpCzVTdmbDEfCukoxS5zN563d5HFPOXqccoIjYuAk+gWx8QC31C
         6P7drq39paL3PK/7xl9fzJo1hEuv0H2r5ek1g5ML/mutdy9sNxcMSMLeM3pA2OybgJbN
         nkDae/PgMsbA9dBMzDYvmoLmPEMZMoufPK3tBLNpZYh6ZsF23zL4NP0kO4v+9O9e4tRW
         kJ2negvayz1pK5ESAuqI3mid7LUeffdJOUXw5OpSZ0jd5yBk0fJdrtDUL0jy613aKdpf
         n97A==
X-Forwarded-Encrypted: i=1; AJvYcCUI5rTlYk0EYI3ZJrTruLA9NLKhEf3YGNhKzili38rDEdb1Qi745uUV46EZB8HjNeNugs4RRMhaTN0Fz/bUffs4L00Taz6ze4FpyIt9wHM=
X-Gm-Message-State: AOJu0YwTy0H6rW+hRzdCXBWhS8Z75l3uwmeNDfU3p4rFWcCKLmwMs+NF
	xX45K6S59TLoY1ppvcABnoszSz8MBIJAs2yg8woEp3t3c/EGr1hagYPdYUvT3Ss=
X-Google-Smtp-Source: AGHT+IGxiGlwLPi/moC75NIUet2FgAUQsi4O6r4RrNMf8WHSEYZFYneKm2D/gkoO4rnvgXhM/iCATw==
X-Received: by 2002:a05:6512:2026:b0:511:69b3:a47e with SMTP id s6-20020a056512202600b0051169b3a47emr1582330lfs.52.1707912255065;
        Wed, 14 Feb 2024 04:04:15 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXspfXe6IhdjYPjAiKekEE0zoaF41AO8//AX/HQGibEgLkhVA8YUvUizB0h63p56RUZQM0ZwEGLYPN3Wj+XgiOdoYni3T+bdXC/lp/oWf4=
Date: Wed, 14 Feb 2024 13:04:12 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH 1/5] iommu/x86: fix IVMD/RMRR range checker loop increment
Message-ID: <ZcysPDDHvkEuFrgJ@macbook>
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-2-roger.pau@citrix.com>
 <b2433393-1a04-4570-abe6-0356db9976f8@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <b2433393-1a04-4570-abe6-0356db9976f8@suse.com>

On Wed, Feb 14, 2024 at 12:51:36PM +0100, Jan Beulich wrote:
> On 14.02.2024 11:37, Roger Pau Monne wrote:
> > mfn_add() doesn't store the incremented value in the parameter, and instead
> > returns it to the caller.  As a result, the loop in iommu_unity_region_ok()
> > didn't make progress.  Fix it by storing the incremented value.
> > 
> > Fixes: e45801dea17b ('iommu/x86: introduce a generic IVMD/RMRR range validity helper')
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> Should Andrew get a Reported-by here? And surely we want to list the
> Coverity ID as well? (Happy to take of both while committing, so long
> as you agree.)

Oh, I didn't add those here, yes, sure, feel free to add.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 12:10:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 12:10:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680690.1058743 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raE5S-0007vr-4F; Wed, 14 Feb 2024 12:10:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680690.1058743; Wed, 14 Feb 2024 12:10:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raE5S-0007vS-0k; Wed, 14 Feb 2024 12:10:02 +0000
Received: by outflank-mailman (input) for mailman id 680690;
 Wed, 14 Feb 2024 12:09:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1raE5P-0007lT-SE
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 12:09:59 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1raE5P-0002Id-5L; Wed, 14 Feb 2024 12:09:59 +0000
Received: from [15.248.2.25] (helo=[10.24.67.29])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1raE5O-0006hn-VS; Wed, 14 Feb 2024 12:09:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=8qXRNpdDTjwZAcgVlkdMOP4VDuS2Ax6ZCZLPyYIR4Ek=; b=14jT2pM60cNLfNeVtJrWesgQDc
	VjPTb79kNOReN0uJZnFekK/W/C0s0amKDqKvu0BbK17jccMJ7i5xbr4jE9uEpEZsUgppF6oeMIuR4
	c77D7Iy+5aW9aj5HL+caMiEhk8SKgh6WflTN1MgeJopibh8zuj271wELhSps5x2NVqxg=;
Message-ID: <3171ef24-bcb3-4387-ba32-f1c4d6c7220a@xen.org>
Date: Wed, 14 Feb 2024 12:09:56 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 5/7] xen/asm-generic: introduce generic device.h
Content-Language: en-GB
To: Oleksii <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
 <4ecb5e5f4a1dbf585ac7a9ebe40b5fd8e4709182.1707499278.git.oleksii.kurochko@gmail.com>
 <319dcf7a-88cd-488e-ae51-6c20c19d8f22@xen.org>
 <9d4f3093de6f95b09e44bc66fe9a9d4b0009dc43.camel@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <9d4f3093de6f95b09e44bc66fe9a9d4b0009dc43.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit


Hi Oleksii,

On 14/02/2024 09:32, Oleksii wrote:
> On Tue, 2024-02-13 at 18:09 +0000, Julien Grall wrote:
>>> +#ifdef CONFIG_HAS_PASSTHROUGH
>>> +    struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU
>>> instance data */
>>> +#endif
>>> +};
>>> +
>>> +typedef struct device device_t;
>>> +
>>> +#ifdef CONFIG_HAS_DEVICE_TREE
>>> +
>>> +#include <xen/device_tree.h>
>>> +
>>> +#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
>>> +
>>> +/**
>>> + *  device_init - Initialize a device
>>> + *  @dev: device to initialize
>>> + *  @class: class of the device (serial, network...)
>>> + *  @data: specific data for initializing the device
>>> + *
>>> + *  Return 0 on success.
>>> + */
>>> +int device_init(struct dt_device_node *dev, enum device_class
>>> class,
>>> +                const void *data);
>>> +
>>> +/**
>>> + * device_get_type - Get the type of the device
>>> + * @dev: device to match
>>> + *
>>> + * Return the device type on success or DEVICE_ANY on failure
>>> + */
>>> +enum device_class device_get_class(const struct dt_device_node
>>> *dev);
>>> +
>>> +#define DT_DEVICE_START(name_, namestr_, class_)            \
>>> +static const struct device_desc __dev_desc_##name_ __used   \
>>> +__section(".dev.info") = {                                  \
>>> +    .name = namestr_,                                       \
>>> +    .class = class_,
>>> +
>>> +#define DT_DEVICE_END                                       \
>>> +};
>>> +
>>> +#else /* !CONFIG_HAS_DEVICE_TREE */
>>> +#define dev_is_dt(dev) ((void)(dev), false)
>>> +#endif /* CONFIG_HAS_DEVICE_TREE */
>>> +
>>> +#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
>>> +
>>> +struct device_desc {
>>> +    /* Device name */
>>> +    const char *name;
>>> +    /* Device class */
>>> +    enum device_class class;
>>> +
>>> +#ifdef CONFIG_HAS_DEVICE_TREE
>>> +
>>> +    /* List of devices supported by this driver */
>>> +    const struct dt_device_match *dt_match;
>>> +    /*
>>> +     * Device initialization.
>>> +     *
>>> +     * -EAGAIN is used to indicate that device probing is
>>> deferred.
>>> +     */
>>> +    int (*init)(struct dt_device_node *dev, const void *data);
>>> +
>>> +#endif
>>> +};
>> I am not sure I fully understand why "device_desc" is not protected
>> by
>> CONFIG_HAS_DEVICE_TREE. The structure doesn't mean much when the
>> config
>> is disabled. Can you clarify?
> I thought that one day struct device_desc and acpi_device_desc will be
> "merged", and so decided just to #ifdef only DEVICE_TREE specific
> fields.

It might be possible to merge the two if we were using an union for the 
ACPI/DT specific part. However the majority of the parsing code needs to 
differ. So I am not convinced there would be any value to merge the two 
structures.

> Another one reason it is if to protect fully struct device_desc then it
> would be needed more #ifdef in arm/device.c ( for example,
> device_init() should be all protected then ) what will require to ifdef
> all calls of device_init(). As an option device_init can can be defined
> in case when !CONFIG_HAS_DEVICE_TREE as:
>     int __init device_init(struct dt_device_node *dev, enum device_class
>     class,
>                            const void *data)
>     {
>      return -EBADF;
>     }
>     
> The similar thing will be needed for device_get_class() in Arm's device.c.

I agree that in theory device_init() & co should be protected with 
CONFIG_HAS_DEVICE_TREE. However, it is not possible to compile Xen on 
Arm without the Device-Tree part today. So I don't view adding the 
#ifdef or any extra stub as necessary today.

This may be useful in the future though. Note this is not a request to 
modify the patch more than...

> 
> Would it be better to ifdef full struct device_desc ?
.. moving structure within the #ifdef.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 12:11:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 12:11:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680693.1058752 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raE6v-0000xg-Du; Wed, 14 Feb 2024 12:11:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680693.1058752; Wed, 14 Feb 2024 12:11:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raE6v-0000xZ-Ae; Wed, 14 Feb 2024 12:11:33 +0000
Received: by outflank-mailman (input) for mailman id 680693;
 Wed, 14 Feb 2024 12:11:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raE6u-0000xR-GK
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 12:11:32 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 301809b7-cb32-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 13:11:31 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-562117d3dbfso1255369a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 04:11:31 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 th8-20020a1709078e0800b00a3cfa56d186sm1690833ejc.183.2024.02.14.04.11.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 04:11:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 301809b7-cb32-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707912691; x=1708517491; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=6+e+D74wcSMT5q1p0U9/h3H1M+/W8RseIIBXf+tCI1k=;
        b=lG/E7eWPKhsoEAVD05d/+p7g9RNNEz2AFEuev4HW34rUNU4XlG4aGZqRgrRakgvW6x
         Sdod2Rkdpus6mYB2nDUQXGzb2ODEzOHvpUFSTXRc48oWHmtgGKdwa9XRg3Qsz8ZkpeX7
         1rga8jGBoO357YhdZNcuFpviD6NZmMrr9/KOwmWpJHKZiv+gFFoJ9muVvs8prFcAbGKp
         ZmXC/XjSGQX0lTcRbBPYu6VrvsC4MVasmna6F+6lpcNeELHQii5WLG04djtU2UEHUbZx
         p39FqIR/yQpcSoJRfM/SnYHNOGHTrStseHSCrgCYUOag3DCFvFl2GAr77Cvy6PqR/cci
         TR4w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707912691; x=1708517491;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6+e+D74wcSMT5q1p0U9/h3H1M+/W8RseIIBXf+tCI1k=;
        b=igP0iItk8alR6Oo5P/i07MMfnfroyVcMGeyt8+HHOeFSQLKpKSeqeJWy05oRixyTmk
         tl/IhPCs6m3ktA/7aB4l83aYqFykgNjataduTjLp8yYmZrKmv57Y9CMNTJtIALA/rU0Y
         pJq1bmALX0/C0HhuZ8rsLQWs6rJOLLdrvzfmPoXOFRaI8b1sSX/JCYbKW7Hs9xJ5PZoL
         oe7ApzuF84+3ndTQ3IJNgPUmOKo8kA7eYg+z1PRyMGT+SY/V/Li5avas3y/8H9g/wsbH
         rWgoPDPosIjo9muN0WubgMyGeMpt4em/Th9cy8K4x6iNYGdstVX7Utu2+8S/wvAktKup
         yJSQ==
X-Forwarded-Encrypted: i=1; AJvYcCUHtv0M0KAOgPQMyMOyaT3zdjRUG/CcPoj24ULqGN57fjmebKrhAzqjjG66AGskUwuc3r0m/MCV8aY317J73NmmP0Ofwbuq7TTorBCiuO0=
X-Gm-Message-State: AOJu0Yx8zlLITO3nxeOSeiZLFZa86Tf3m9YxxwFJrrCIjaKhDoOOdFCE
	XEBpAyr5DZgC2xcMLVfUb+3pRKsn7J3UU2R2W6mOSWdou8yK0FiD
X-Google-Smtp-Source: AGHT+IGFpTP35bowhkxNhc6W+1de5r2OYHlcqTVc7mxQsEt2HhxJa8H0zb8tlvgrEtyoqmWkoJsu3g==
X-Received: by 2002:a17:906:e090:b0:a38:187c:2a9f with SMTP id gh16-20020a170906e09000b00a38187c2a9fmr1603192ejb.36.1707912690498;
        Wed, 14 Feb 2024 04:11:30 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVefQeWFKyl4v+JwhUg/F+SQgJLtbuJEu/2whcc7G867tcCtexBkaJm66hfs03O2kOILe0z7X95UAFQg4H9tSBF/XsCOBKL0mMgz/2tGXC5xJ/wPkARmOQP62O8Gnj4NIzZsXJUZooKsMdWMkmFbN1n+Iqlz0SE2r5qhTwZSYodvLnjiNcQOlYZOkAiH3uG+1jrnrHMqSlUX5On54fi5zlJ33fqi6NbGFnuozwc+MmW5kfPPpBMiifu80ELM9IefxH4TuPpE32sJ3KeEDLUJbL1WfYx4a7i0rm3BiFRHQx9xEtYKesOOOQ1
Message-ID: <8af3bbf27c51f4b5d8663b537c3123c6e05e74e7.camel@gmail.com>
Subject: Re: [PATCH v4 14/30] xen/riscv: introduce atomic.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Bobby Eshleman <bobbyeshleman@gmail.com>, Alistair Francis
 <alistair.francis@wdc.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 13:11:29 +0100
In-Reply-To: <134e09c9-776a-4261-a28b-137d109b62ef@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <6554f2479e19ed3eae6de842ac1568c31d236461.1707146506.git.oleksii.kurochko@gmail.com>
	 <134e09c9-776a-4261-a28b-137d109b62ef@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

T24gVHVlLCAyMDI0LTAyLTEzIGF0IDEyOjM2ICswMTAwLCBKYW4gQmV1bGljaCB3cm90ZToKPiBP
biAwNS4wMi4yMDI0IDE2OjMyLCBPbGVrc2lpIEt1cm9jaGtvIHdyb3RlOgo+ID4gRnJvbTogQm9i
YnkgRXNobGVtYW4gPGJvYmJ5ZXNobGVtYW5AZ21haWwuY29tPgo+ID4gCj4gPiBBZGRpdGlvbmFs
bHksIHRoaXMgcGF0Y2ggaW50cm9kdWNlcyBtYWNyb3MgaW4gZmVuY2UuaCwKPiA+IHdoaWNoIGFy
ZSB1dGlsaXplZCBpbiBhdG9taWMuaC4KPiAKPiBUaGVzZSBhcmUgdXNlZCBpbiBhbiBlYXJsaWVy
IHBhdGNoIGFscmVhZHksIHNvIGVpdGhlciB5b3Ugd2FudCB0bwo+IHJlLW9yZGVyIHRoZSBzZXJp
ZXMsIG9yIHlvdSB3YW50IHRvIG1vdmUgdGhhdCBpbnRyb2R1Y3Rpb24gYWhlYWQuCj4gCj4gPiAt
LS0gL2Rldi9udWxsCj4gPiArKysgYi94ZW4vYXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9hdG9taWMu
aAo+ID4gQEAgLTAsMCArMSwzOTUgQEAKPiA+ICsvKiBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjog
R1BMLTIuMC1vbmx5ICovCj4gPiArLyoKPiA+ICsgKiBUYWtlbiBhbmQgbW9kaWZpZWQgZnJvbSBM
aW51eC4KPiA+ICsgKgo+ID4gKyAqIGF0b21pYyMjcHJlZml4IyNfKnhjaGdfKihhdG9taWMjI3By
ZWZpeCMjX3QgKnYsIGNfdCBuKSB3ZXJlCj4gPiB1cGRhdGVkIHRvIHVzZQo+ID4gKyAqIF9fKnhj
aGdfZ2VuZXJpYygpCj4gPiArICogCj4gPiArICogQ29weXJpZ2h0IChDKSAyMDA3IFJlZCBIYXQs
IEluYy4gQWxsIFJpZ2h0cyBSZXNlcnZlZC4KPiA+ICsgKiBDb3B5cmlnaHQgKEMpIDIwMTIgUmVn
ZW50cyBvZiB0aGUgVW5pdmVyc2l0eSBvZiBDYWxpZm9ybmlhCj4gPiArICogQ29weXJpZ2h0IChD
KSAyMDE3IFNpRml2ZQo+ID4gKyAqIENvcHlyaWdodCAoQykgMjAyMSBWYXRlcyBTQVMKPiA+ICsg
Ki8KPiA+ICsKPiA+ICsjaWZuZGVmIF9BU01fUklTQ1ZfQVRPTUlDX0gKPiA+ICsjZGVmaW5lIF9B
U01fUklTQ1ZfQVRPTUlDX0gKPiA+ICsKPiA+ICsjaW5jbHVkZSA8eGVuL2F0b21pYy5oPgo+ID4g
KyNpbmNsdWRlIDxhc20vY21weGNoZy5oPgo+ID4gKyNpbmNsdWRlIDxhc20vZmVuY2UuaD4KPiA+
ICsjaW5jbHVkZSA8YXNtL2lvLmg+Cj4gPiArI2luY2x1ZGUgPGFzbS9zeXN0ZW0uaD4KPiA+ICsK
PiA+ICt2b2lkIF9fYmFkX2F0b21pY19zaXplKHZvaWQpOwo+ID4gKwo+ID4gK3N0YXRpYyBhbHdh
eXNfaW5saW5lIHZvaWQgcmVhZF9hdG9taWNfc2l6ZShjb25zdCB2b2xhdGlsZSB2b2lkICpwLAo+
ID4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB2b2lkICpyZXMsCj4gPiArwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgIHVuc2lnbmVkIGludCBzaXplKQo+ID4gK3sKPiA+ICvCoMKgwqAg
c3dpdGNoICggc2l6ZSApCj4gPiArwqDCoMKgIHsKPiA+ICvCoMKgwqAgY2FzZSAxOiAqKHVpbnQ4
X3QgKilyZXMgPSByZWFkYihwKTsgYnJlYWs7Cj4gPiArwqDCoMKgIGNhc2UgMjogKih1aW50MTZf
dCAqKXJlcyA9IHJlYWR3KHApOyBicmVhazsKPiA+ICvCoMKgwqAgY2FzZSA0OiAqKHVpbnQzMl90
ICopcmVzID0gcmVhZGwocCk7IGJyZWFrOwo+ID4gK8KgwqDCoCBjYXNlIDg6ICoodWludDMyX3Qg
KilyZXPCoCA9IHJlYWRxKHApOyBicmVhazsKPiAKPiBXaHkgaXMgaXQgdGhlIE1NSU8gcHJpbWl0
aXZlcyB5b3UgdXNlIGhlcmUsIGkuZS4gbm90IHJlYWQ8WD5fY3B1KCk/Cj4gSXQncyBSQU0geW91
J3JlIGFjY2Vzc2luZyBhZnRlciBhbGwuCkxlZ2FjeSBmcm9tIExpbnV4IGtlcm5lbC4gRm9yIHNv
bWUgcmVhc29uIHRoZXkgd2FudGVkIHRvIGhhdmUgb3JkZXJlZApyZWFkL3dyaXRlIGFjY2Vzcy4K
Cj4gCj4gQWxzbyAtIG5vIENPTkZJR182NEJJVCBjb25kaXRpb25hbCBoZXJlIChsaWtlIHlvdSBo
YXZlIGluIHRoZSBvdGhlcgo+IHBhdGNoKT8KQWdyZWUsIGl0IHNob3VsZCBiZSBhZGRlZC4KCj4g
Cj4gPiArwqDCoMKgIGRlZmF1bHQ6IF9fYmFkX2F0b21pY19zaXplKCk7IGJyZWFrOwo+ID4gK8Kg
wqDCoCB9Cj4gPiArfQo+ID4gKwo+ID4gKyNkZWZpbmUgcmVhZF9hdG9taWMocCkgKHvCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+
ID4gK8KgwqDCoCB1bmlvbiB7IHR5cGVvZigqcCkgdmFsOyBjaGFyIGNbMF07IH0geF87wqDCoMKg
wqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIHJlYWRfYXRvbWljX3NpemUocCwgeF8uYywg
c2l6ZW9mKCpwKSk7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+IAo+IEknbGwgYmUgY3Vy
aW91cyBmb3IgaG93IG11Y2ggbG9uZ2VyIGdjYyB3aWxsIHRvbGVyYXRlIHRoaXMgYWNjZXNzaW5n
Cj4gb2YgYSB6ZXJvLWxlbmd0aCBhcnJheSwgd2l0aG91dCBpc3N1aW5nIGF0IGxlYXN0IGEgd2Fy
bmluZy4gSSdkCj4gcmVjb21tZW5kIHVzaW5nIHNpemVvZigqKHApKSBhcyB0aGUgYXJyYXkgZGlt
ZW5zaW9uIHJpZ2h0IGF3YXkuIChGcm9tCj4gdGhpcyBub3QgYWxzbyB0aGUgbWlzc2luZyBwYXJl
bnRoZXNlcyBpbiB3aGF0IHlvdSBoYXZlLikKVGhhbmtzLiBJJ2xsIHVwZGF0ZSB0aGF0LgoKPiAK
PiA+ICvCoMKgwqAgeF8udmFsO8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4g
K30pCj4gPiArCj4gPiArI2RlZmluZSB3cml0ZV9hdG9taWMocCwgeCnCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICsoe8KgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDC
oCB0eXBlb2YoKnApIHhfXyA9ICh4KTvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCBzd2l0Y2ggKCBzaXplb2Yo
KnApICnCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqAgXAo+ID4gK8KgwqDCoCB7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIGNhc2UgMTogd3JpdGViKCh1aW50OF90KXhfXyzCoCBw
KTsgYnJlYWs7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIGNhc2UgMjogd3Jp
dGV3KCh1aW50MTZfdCl4X18sIHApOyBicmVhazvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+
ICvCoMKgwqAgY2FzZSA0OiB3cml0ZWwoKHVpbnQzMl90KXhfXywgcCk7IGJyZWFrO8KgwqDCoMKg
wqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCBjYXNlIDg6IHdyaXRlcSgodWludDY0X3QpeF9f
LCBwKTsgYnJlYWs7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gCj4gQXJlIHRoZSBjYXN0cyBh
Y3R1YWxseSBuZWNlc3NhcnkgaGVyZT8KTm90IHJlYWxseSwgd2UgY2FuIGRyb3AgdGhlbS4KCj4g
Cj4gPiArwqDCoMKgIGRlZmF1bHQ6IF9fYmFkX2F0b21pY19zaXplKCk7IGJyZWFrO8KgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIH3CoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqAgeF9fO8KgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK30pCj4gPiArCj4gPiArI2RlZmluZSBhZGRf
c2l6ZWQocCwgeCnCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICsoe8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCB0eXBlb2YoKihwKSkgeF9fID0gKHgp
O8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
IFwKPiA+ICvCoMKgwqAgc3dpdGNoICggc2l6ZW9mKCoocCkpICnCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIHvCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqAgY2Fz
ZSAxOiB3cml0ZWIocmVhZF9hdG9taWMocCkgKyB4X18sIHApOyBicmVhazvCoMKgwqDCoCBcCj4g
PiArwqDCoMKgIGNhc2UgMjogd3JpdGV3KHJlYWRfYXRvbWljKHApICsgeF9fLCBwKTsgYnJlYWs7
wqDCoMKgwqAgXAo+ID4gK8KgwqDCoCBjYXNlIDQ6IHdyaXRlbChyZWFkX2F0b21pYyhwKSArIHhf
XywgcCk7IGJyZWFrO8KgwqDCoMKgIFwKPiA+ICvCoMKgwqAgZGVmYXVsdDogX19iYWRfYXRvbWlj
X3NpemUoKTsgYnJlYWs7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKg
wqAgfcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK30p
Cj4gPiArCj4gPiArLyoKPiA+ICsgKsKgIF9fdW5xdWFsX3NjYWxhcl90eXBlb2YoeCkgLSBEZWNs
YXJlIGFuIHVucXVhbGlmaWVkIHNjYWxhcgo+ID4gdHlwZSwgbGVhdmluZwo+ID4gKyAqwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBub24tc2NhbGFyIHR5cGVzIHVuY2hhbmdlZC4KPiA+ICsg
Kgo+ID4gKyAqIFByZWZlciBDMTEgX0dlbmVyaWMgZm9yIGJldHRlciBjb21waWxlLXRpbWVzIGFu
ZCBzaW1wbGVyIGNvZGUuCj4gPiBOb3RlOiAnY2hhcicKPiA+ICsgKiBpcyBub3QgdHlwZS1jb21w
YXRpYmxlIHdpdGggJ3NpZ25lZCBjaGFyJywgYW5kIHdlIGRlZmluZSBhCj4gPiBzZXBhcmF0ZSBj
YXNlLgo+ID4gKyAqLwo+ID4gKyNkZWZpbmUgX19zY2FsYXJfdHlwZV90b19leHByX2Nhc2VzKHR5
cGUpwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIHVuc2lnbmVkIHR5
cGU6wqAgKHVuc2lnbmVkIHR5cGUpMCzCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqAgXAo+ID4gK8KgwqDCoCBzaWduZWQgdHlwZTrCoMKgwqAgKHNpZ25lZCB0eXBlKTAKPiA+ICsK
PiA+ICsjZGVmaW5lIF9fdW5xdWFsX3NjYWxhcl90eXBlb2YoeCkgdHlwZW9mKMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCBfR2VuZXJpYygoeCkswqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoCBcCj4gPiArwqDCoMKgwqDCoMKgwqAgY2hhcjrCoCAoY2hhcikwLMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+
ID4gK8KgwqDCoMKgwqDCoMKgIF9fc2NhbGFyX3R5cGVfdG9fZXhwcl9jYXNlcyhjaGFyKSzCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgwqDCoMKgwqAgX19zY2FsYXJfdHlw
ZV90b19leHByX2Nhc2VzKHNob3J0KSzCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8Kg
wqDCoMKgwqDCoMKgIF9fc2NhbGFyX3R5cGVfdG9fZXhwcl9jYXNlcyhpbnQpLMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoMKgwqDCoMKgIF9fc2NhbGFyX3R5cGVfdG9f
ZXhwcl9jYXNlcyhsb25nKSzCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKg
wqDCoMKgwqAgX19zY2FsYXJfdHlwZV90b19leHByX2Nhc2VzKGxvbmcgbG9uZykswqDCoMKgwqDC
oMKgwqDCoCBcCj4gPiArwqDCoMKgwqDCoMKgwqAgZGVmYXVsdDogKHgpKSkKPiAKPiBUaGlzIGlz
bid0IFJJU0MtViBzcGVjaWZpYywgaXMgaXQ/IEluIHdoaWNoIGNhc2UgaXQgd2FudHMgbW92aW5n
IHRvLAo+IHBlcmhhcHMsIHhlbi9tYWNyb3MuaCAoYW5kIHRoZW4gYWxzbyBoYXZlIHRoZSBsZWFk
aW5nIHVuZGVyc2NvcmVzCj4gZHJvcHBlZCkuCk5vLCBhdCBhbGwuIEJ1dCB0aGlzIHRoaW5nIGlz
IG9ubHkgdXNlZCBpbiBSSVNDLVYgcGFydCwgYnV0IGlmIGl0IHdvdWxkCmJlIGJldHRlciB0byBt
b3ZlIGl0IHRvIHhlbi9tYWNyb3MuaCBJIHdpbGwgaGFwcHkgdG8gc2VudCBzZXBhcmF0ZQpwYXRj
aC4KCj4gCj4gPiArI2RlZmluZSBSRUFEX09OQ0UoeCnCoCAoKihjb25zdCB2b2xhdGlsZSBfX3Vu
cXVhbF9zY2FsYXJfdHlwZW9mKHgpCj4gPiAqKSYoeCkpCj4gPiArI2RlZmluZSBXUklURV9PTkNF
KHgsIHZhbCnCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIGRvIHvCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgwqDC
oMKgwqAgKih2b2xhdGlsZSB0eXBlb2YoeCkgKikmKHgpID0gKHZhbCk7wqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCB9IHdoaWxlICgwKQo+IAo+IElu
IFhlbiB3ZSB1c2UgQUNDRVNTX09OQ0UoKTsgYW55IHJlYXNvbiB5b3UgbmVlZCB0byBpbnRyb2R1
Y2UKPiB7UkVBRCxXUklURX1fT05DRSgpIGluIGFkZGl0aW9uPyBXaXRob3V0IHRoZW0sCj4gX191
bnF1YWxfc2NhbGFyX3R5cGVvZigpCj4gbWF5IHRoZW4gYWxzbyBub3QgYmUgbmVlZGVkIChvciwg
aWYgdGhlcmUncyBhIG5lZWQgdG8gZW5oYW5jZSBpdCwgbWF5Cj4gdGhlbiBiZSBuZWVkZWQgZm9y
IEFDQ0VTU19PTkNFKCkpLiBXaGljaCBpbiB0dXJuIHJhaXNlcyB0aGUgcXVlc3Rpb24KPiB3aHkg
b25seSBSRUFEX09OQ0UoKSB1c2VzIGl0IGhlcmUuCkhtbSwgUkVBRF9PTkNFKCkgYW5kIFdSSVRF
X09OQ0UoKSBjYW4gYmUgZHJvcHBlZCB0aGVuLCBJJ2xsIHN3aXRjaApldmVyeXRoaW5nIGluIG15
IGNvZGUgdG8gQUNDRVNTX09OQ0UoKS4KCj4gCj4gPiArI2RlZmluZSBfX2F0b21pY19hY3F1aXJl
X2ZlbmNlKCkgXAo+ID4gK8KgwqDCoCBfX2FzbV9fIF9fdm9sYXRpbGVfXyggUklTQ1ZfQUNRVUlS
RV9CQVJSSUVSICIiIDo6OiAibWVtb3J5IiApCj4gCj4gTWlzc2luZyBibGFuayBoZXJlIGFuZCAu
Li4KPiAKPiA+ICsjZGVmaW5lIF9fYXRvbWljX3JlbGVhc2VfZmVuY2UoKSBcCj4gPiArwqDCoMKg
IF9fYXNtX18gX192b2xhdGlsZV9fKCBSSVNDVl9SRUxFQVNFX0JBUlJJRVIgIiIgOjo6ICJtZW1v
cnkiICk7Cj4gCj4gLi4uIGhlcmUsIGFuZCBzdHJheSBzZW1pY29sb24gYWRkaXRpb25hbGx5IGp1
c3QgaGVyZS4KVGhhbmtzLiBJJ2xsIGFwcGx5IHlvdXIgY29tbWVudHMgdG8gdGhpcyBwYXJ0IG9m
IGNvZGUuCgo+IAo+ID4gK3N0YXRpYyBpbmxpbmUgaW50IGF0b21pY19yZWFkKGNvbnN0IGF0b21p
Y190ICp2KQo+ID4gK3sKPiA+ICvCoMKgwqAgcmV0dXJuIFJFQURfT05DRSh2LT5jb3VudGVyKTsK
PiA+ICt9Cj4gPiArCj4gPiArc3RhdGljIGlubGluZSBpbnQgX2F0b21pY19yZWFkKGF0b21pY190
IHYpCj4gPiArewo+ID4gK8KgwqDCoCByZXR1cm4gdi5jb3VudGVyOwo+ID4gK30KPiA+ICsKPiA+
ICtzdGF0aWMgaW5saW5lIHZvaWQgYXRvbWljX3NldChhdG9taWNfdCAqdiwgaW50IGkpCj4gPiAr
ewo+ID4gK8KgwqDCoCBXUklURV9PTkNFKHYtPmNvdW50ZXIsIGkpOwo+ID4gK30KPiA+ICsKPiA+
ICtzdGF0aWMgaW5saW5lIHZvaWQgX2F0b21pY19zZXQoYXRvbWljX3QgKnYsIGludCBpKQo+ID4g
K3sKPiA+ICvCoMKgwqAgdi0+Y291bnRlciA9IGk7Cj4gPiArfQo+ID4gKwo+ID4gK3N0YXRpYyBp
bmxpbmUgaW50IGF0b21pY19zdWJfYW5kX3Rlc3QoaW50IGksIGF0b21pY190ICp2KQo+ID4gK3sK
PiA+ICvCoMKgwqAgcmV0dXJuIGF0b21pY19zdWJfcmV0dXJuKGksIHYpID09IDA7Cj4gPiArfQo+
ID4gKwo+ID4gK3N0YXRpYyBpbmxpbmUgdm9pZCBhdG9taWNfaW5jKGF0b21pY190ICp2KQo+ID4g
K3sKPiA+ICvCoMKgwqAgYXRvbWljX2FkZCgxLCB2KTsKPiA+ICt9Cj4gPiArCj4gPiArc3RhdGlj
IGlubGluZSBpbnQgYXRvbWljX2luY19yZXR1cm4oYXRvbWljX3QgKnYpCj4gPiArewo+ID4gK8Kg
wqDCoCByZXR1cm4gYXRvbWljX2FkZF9yZXR1cm4oMSwgdik7Cj4gPiArfQo+ID4gKwo+ID4gK3N0
YXRpYyBpbmxpbmUgdm9pZCBhdG9taWNfZGVjKGF0b21pY190ICp2KQo+ID4gK3sKPiA+ICvCoMKg
wqAgYXRvbWljX3N1YigxLCB2KTsKPiA+ICt9Cj4gPiArCj4gPiArc3RhdGljIGlubGluZSBpbnQg
YXRvbWljX2RlY19yZXR1cm4oYXRvbWljX3QgKnYpCj4gPiArewo+ID4gK8KgwqDCoCByZXR1cm4g
YXRvbWljX3N1Yl9yZXR1cm4oMSwgdik7Cj4gPiArfQo+ID4gKwo+ID4gK3N0YXRpYyBpbmxpbmUg
aW50IGF0b21pY19kZWNfYW5kX3Rlc3QoYXRvbWljX3QgKnYpCj4gPiArewo+ID4gK8KgwqDCoCBy
ZXR1cm4gYXRvbWljX3N1Yl9yZXR1cm4oMSwgdikgPT0gMDsKPiA+ICt9Cj4gPiArCj4gPiArc3Rh
dGljIGlubGluZSBpbnQgYXRvbWljX2FkZF9uZWdhdGl2ZShpbnQgaSwgYXRvbWljX3QgKnYpCj4g
PiArewo+ID4gK8KgwqDCoCByZXR1cm4gYXRvbWljX2FkZF9yZXR1cm4oaSwgdikgPCAwOwo+ID4g
K30KPiA+ICsKPiA+ICtzdGF0aWMgaW5saW5lIGludCBhdG9taWNfaW5jX2FuZF90ZXN0KGF0b21p
Y190ICp2KQo+ID4gK3sKPiA+ICvCoMKgwqAgcmV0dXJuIGF0b21pY19hZGRfcmV0dXJuKDEsIHYp
ID09IDA7Cj4gPiArfQo+IAo+IE5vbmUgb2YgdGhlc2UgbG9vayBSSVNDLVYtc3BlY2lmaWMuIFBl
cmhhcHMgd29ydGggaGF2aW5nIHNvbWV0aGluZyBpbgo+IGFzbS1nZW5lcmljLyB0aGF0IGNhbiBi
ZSB1dGlsaXplZCBoZXJlPwpMb29rcyBsaWtlIHdlIGNhbiwgYXQgbGVhc3QsIFBQQyBoYXMgdGhl
IHNpbWlsYXIgZGVmaW5pdGlvbnMuCgo+IAo+ID4gKy8qCj4gPiArICogRmlyc3QsIHRoZSBhdG9t
aWMgb3BzIHRoYXQgaGF2ZSBubyBvcmRlcmluZyBjb25zdHJhaW50cyBhbmQKPiA+IHRoZXJlZm9y
IGRvbid0Cj4gPiArICogaGF2ZSB0aGUgQVEgb3IgUkwgYml0cyBzZXQuwqAgVGhlc2UgZG9uJ3Qg
cmV0dXJuIGFueXRoaW5nLCBzbwo+ID4gdGhlcmUncyBvbmx5Cj4gPiArICogb25lIHZlcnNpb24g
dG8gd29ycnkgYWJvdXQuCj4gPiArICovCj4gPiArI2RlZmluZSBBVE9NSUNfT1Aob3AsIGFzbV9v
cCwgSSwgYXNtX3R5cGUsIGNfdHlwZSwgcHJlZml4KcKgIFwKPiA+ICtzdGF0aWMgaW5saW5lwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK3ZvaWQgYXRvbWljIyNwcmVm
aXgjI18jI29wKGNfdHlwZSBpLCBhdG9taWMjI3ByZWZpeCMjX3QgKnYpIFwKPiA+ICt7wqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+
ID4gK8KgwqDCoCBfX2FzbV9fIF9fdm9sYXRpbGVfXyAowqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqDC
oMKgwqDCoCAiwqDCoCBhbW8iICNhc21fb3AgIi4iICNhc21fdHlwZSAiIHplcm8sICUxLCAlMCLC
oMKgwqDCoMKgIFwKPiA+ICvCoMKgwqDCoMKgwqDCoCA6ICIrQSIgKHYtPmNvdW50ZXIpwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oCBcCj4gPiArwqDCoMKgwqDCoMKgwqAgOiAiciIgKEkpwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgIFwKPiA+ICvCoMKgwqDCoMKgwqDCoCA6ICJtZW1vcnkiICk7wqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oCBcCj4gPiArfcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgIFwKPiA+ICsKPiA+ICsjZGVmaW5lIEFUT01JQ19PUFMob3AsIGFzbV9vcCwg
SSnCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwK
PiA+ICvCoMKgwqDCoMKgwqDCoCBBVE9NSUNfT1AgKG9wLCBhc21fb3AsIEksIHcsIGludCzCoMKg
ICkKPiAKPiBTbyB0aGUgbGFzdCB0aHJlZSBwYXJhbWV0ZXJzIGFyZSB0byBiZSByZWFkeSB0byBh
bHNvIHN1cHBvcnQKPiBhdG9taWM2NCwgd2l0aG91dCBhY3R1YWxseSBkb2luZyBzbyByaWdodCBu
b3c/ClllcywgaXQgaXMgcmVhZHkgdG8gc3VwcG9ydC4KCj4gCj4gPiArQVRPTUlDX09QUyhhZGQs
IGFkZCzCoCBpKQo+ID4gK0FUT01JQ19PUFMoc3ViLCBhZGQsIC1pKQo+ID4gK0FUT01JQ19PUFMo
YW5kLCBhbmQswqAgaSkKPiA+ICtBVE9NSUNfT1BTKCBvcizCoCBvcizCoCBpKQo+ID4gK0FUT01J
Q19PUFMoeG9yLCB4b3IswqAgaSkKPiA+ICsKPiA+ICsjdW5kZWYgQVRPTUlDX09QCj4gPiArI3Vu
ZGVmIEFUT01JQ19PUFMKPiA+ICsKPiA+ICsvKgo+ID4gKyAqIEF0b21pYyBvcHMgdGhhdCBoYXZl
IG9yZGVyZWQsIHJlbGF4ZWQsIGFjcXVpcmUsIGFuZCByZWxlYXNlCj4gPiB2YXJpYW50cy4KPiA+
ICsgKiBUaGVyZSdzIHR3byBmbGF2b3JzIG9mIHRoZXNlOiB0aGUgYXJpdGhtYXRpYyBvcHMgaGF2
ZSBib3RoCj4gPiBmZXRjaCBhbmQgcmV0dXJuCj4gPiArICogdmVyc2lvbnMsIHdoaWxlIHRoZSBs
b2dpY2FsIG9wcyBvbmx5IGhhdmUgZmV0Y2ggdmVyc2lvbnMuCj4gPiArICovCj4gCj4gSSdtIHNv
bWV3aGF0IGNvbmZ1c2VkIGJ5IHRoZSBjb21tZW50OiBJdCBmaXJzdCB0YWxrcyBvZiA0IHZhcmlh
bnRzLAo+IGJ1dAo+IHRoZW4gc2F5cyB0aGVyZSBhcmUgb25seSAyIChhcml0aG1ldGljKSBvciAx
IChsb2dpY2FsKSBvbmVzLgpQcm9iYWJseSB0aGV5IG1lYW4gdGhhdCB1c3VhbGx5IHRoZXkgaGF2
ZSA0IHZhcmlhbnRzLCBidXQgaXQgd2FzCmltcGxlbWVudGVkIG9ubHkgMiAoYXJpdGgpIGFuZCAx
IChsb2dpY2FsKS4KCj4gCj4gPiArI2RlZmluZSBBVE9NSUNfRkVUQ0hfT1Aob3AsIGFzbV9vcCwg
SSwgYXNtX3R5cGUsIGNfdHlwZSwKPiA+IHByZWZpeCnCoMKgwqAgXAo+ID4gK3N0YXRpYwo+ID4g
aW5saW5lwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
IFwKPiA+ICtjX3R5cGUgYXRvbWljIyNwcmVmaXgjI19mZXRjaF8jI29wIyNfcmVsYXhlZChjX3R5
cGUKPiA+IGkswqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBhdG9taWMjI3ByZWZpeCMjX3QKPiA+
ICp2KcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK3vCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgCj4gPiDCoCBcCj4gPiArwqDCoMKgIHJlZ2lzdGVyIGNfdHlwZQo+ID4gcmV0
O8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqAgX19hc21fXyBfX3Zv
bGF0aWxlX18KPiA+ICjCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqDCoMKg
wqDCoCAiwqDCoCBhbW8iICNhc21fb3AgIi4iICNhc21fdHlwZSAiICUxLCAlMiwKPiA+ICUwIsKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgwqDCoMKgwqAgOiAiK0Ei
ICh2LT5jb3VudGVyKSwgIj1yIgo+ID4gKHJldCnCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgwqDCoMKgwqAgOiAiciIK
PiA+IChJKcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4g
K8KgwqDCoMKgwqDCoMKgIDogIm1lbW9yeSIKPiA+ICk7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCByZXR1cm4KPiA+IHJldDvCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArfcKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAK
PiA+IMKgIFwKPiA+ICtzdGF0aWMKPiA+IGlubGluZcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArY190eXBlIGF0b21pYyMjcHJlZml4IyNf
ZmV0Y2hfIyNvcChjX3R5cGUgaSwgYXRvbWljIyNwcmVmaXgjI190Cj4gPiAqdikgXAo+ID4gK3vC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgCj4gPiDCoCBcCj4gPiArwqDCoMKgIHJlZ2lzdGVyIGNfdHlwZQo+ID4g
cmV0O8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqAgX19hc21fXyBf
X3ZvbGF0aWxlX18KPiA+ICjCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqDC
oMKgwqDCoCAiwqDCoCBhbW8iICNhc21fb3AgIi4iICNhc21fdHlwZSAiLmFxcmzCoCAlMSwgJTIs
Cj4gPiAlMCLCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoMKgwqDCoMKgIDogIitBIiAo
di0+Y291bnRlciksICI9ciIKPiA+IChyZXQpwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoMKgwqDCoMKgIDogInIiCj4g
PiAoSSnCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvC
oMKgwqDCoMKgwqDCoCA6ICJtZW1vcnkiCj4gPiApO8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgIFwKPiA+ICvCoMKgwqAgcmV0dXJuCj4gPiByZXQ7wqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK30KPiA+ICsKPiA+ICsjZGVmaW5l
IEFUT01JQ19PUF9SRVRVUk4ob3AsIGFzbV9vcCwgY19vcCwgSSwgYXNtX3R5cGUsIGNfdHlwZSwK
PiA+IHByZWZpeCkgXAo+ID4gK3N0YXRpYwo+ID4gaW5saW5lwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK2NfdHlwZSBhdG9t
aWMjI3ByZWZpeCMjXyMjb3AjI19yZXR1cm5fcmVsYXhlZChjX3R5cGUKPiA+IGkswqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGF0b21pYyMjcHJlZml4IyNfdAo+ID4gKnYpwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiAre8KgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqAKPiA+IMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoMKgwqDCoMKgIHJldHVybiBhdG9t
aWMjI3ByZWZpeCMjX2ZldGNoXyMjb3AjI19yZWxheGVkKGksIHYpIGNfb3AKPiA+IEk7wqDCoMKg
wqDCoCBcCj4gPiArfcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAKPiA+IMKgwqDCoMKgwqAgXAo+ID4gK3N0YXRp
Ywo+ID4gaW5saW5lwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqAgXAo+ID4gK2NfdHlwZSBhdG9taWMjI3ByZWZpeCMjXyMjb3AjI19yZXR1
cm4oY190eXBlIGksIGF0b21pYyMjcHJlZml4IyNfdAo+ID4gKnYpwqAgXAo+ID4gK3vCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgCj4gPiDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqDCoMKgwqDCoCByZXR1cm4gYXRv
bWljIyNwcmVmaXgjI19mZXRjaF8jI29wKGksIHYpIGNfb3AKPiA+IEk7wqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgIFwKPiA+ICt9Cj4gPiArCj4gPiArI2RlZmluZSBBVE9NSUNfT1BTKG9w
LCBhc21fb3AsIGNfb3AsCj4gPiBJKcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoMKgwqDCoMKgIEFUT01J
Q19GRVRDSF9PUCggb3AsIGFzbV9vcCzCoMKgwqDCoMKgwqAgSSwgdywgaW50LMKgwqAKPiA+ICnC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqDCoMKgwqDCoCBBVE9NSUNf
T1BfUkVUVVJOKG9wLCBhc21fb3AsIGNfb3AsIEksIHcsIGludCzCoMKgICkKPiA+ICsKPiA+ICtB
VE9NSUNfT1BTKGFkZCwgYWRkLCArLMKgIGkpCj4gPiArQVRPTUlDX09QUyhzdWIsIGFkZCwgKywg
LWkpCj4gPiArCj4gPiArI2RlZmluZSBhdG9taWNfYWRkX3JldHVybl9yZWxheGVkwqDCoCBhdG9t
aWNfYWRkX3JldHVybl9yZWxheGVkCj4gPiArI2RlZmluZSBhdG9taWNfc3ViX3JldHVybl9yZWxh
eGVkwqDCoCBhdG9taWNfc3ViX3JldHVybl9yZWxheGVkCj4gPiArI2RlZmluZSBhdG9taWNfYWRk
X3JldHVybsKgwqAgYXRvbWljX2FkZF9yZXR1cm4KPiA+ICsjZGVmaW5lIGF0b21pY19zdWJfcmV0
dXJuwqDCoCBhdG9taWNfc3ViX3JldHVybgo+ID4gKwo+ID4gKyNkZWZpbmUgYXRvbWljX2ZldGNo
X2FkZF9yZWxheGVkwqDCoMKgIGF0b21pY19mZXRjaF9hZGRfcmVsYXhlZAo+ID4gKyNkZWZpbmUg
YXRvbWljX2ZldGNoX3N1Yl9yZWxheGVkwqDCoMKgIGF0b21pY19mZXRjaF9zdWJfcmVsYXhlZAo+
ID4gKyNkZWZpbmUgYXRvbWljX2ZldGNoX2FkZMKgwqDCoCBhdG9taWNfZmV0Y2hfYWRkCj4gPiAr
I2RlZmluZSBhdG9taWNfZmV0Y2hfc3ViwqDCoMKgIGF0b21pY19mZXRjaF9zdWIKPiAKPiBXaGF0
IGFyZSBhbGwgb2YgdGhlc2UgI2RlZmluZS1zIChhbnkgeWV0IG1vcmUgZnVydGhlciBkb3duKSBh
Ym91dD8KPiAKPiA+ICtzdGF0aWMgaW5saW5lIGludCBhdG9taWNfc3ViX2lmX3Bvc2l0aXZlKGF0
b21pY190ICp2LCBpbnQgb2Zmc2V0KQo+ID4gK3sKPiA+ICvCoMKgwqDCoMKgwqAgaW50IHByZXYs
IHJjOwo+ID4gKwo+ID4gK8KgwqDCoCBfX2FzbV9fIF9fdm9sYXRpbGVfXyAoCj4gPiArwqDCoMKg
wqDCoMKgwqAgIjA6IGxyLnfCoMKgwqDCoCAlW3BdLMKgICVbY11cbiIKPiA+ICvCoMKgwqDCoMKg
wqDCoCAiwqDCoCBzdWLCoMKgwqDCoMKgICVbcmNdLCAlW3BdLCAlW29dXG4iCj4gPiArwqDCoMKg
wqDCoMKgwqAgIsKgwqAgYmx0esKgwqDCoMKgICVbcmNdLCAxZlxuIgo+ID4gK8KgwqDCoMKgwqDC
oMKgICLCoMKgIHNjLncucmzCoCAlW3JjXSwgJVtyY10sICVbY11cbiIKPiA+ICvCoMKgwqDCoMKg
wqDCoCAiwqDCoCBibmV6wqDCoMKgwqAgJVtyY10sIDBiXG4iCj4gPiArwqDCoMKgwqDCoMKgwqAg
IsKgwqAgZmVuY2XCoMKgwqAgcncsIHJ3XG4iCj4gPiArwqDCoMKgwqDCoMKgwqAgIjE6XG4iCj4g
PiArwqDCoMKgwqDCoMKgwqAgOiBbcF0iPSZyIiAocHJldiksIFtyY10iPSZyIiAocmMpLCBbY10i
K0EiICh2LT5jb3VudGVyKQo+ID4gK8KgwqDCoMKgwqDCoMKgIDogW29dInIiIChvZmZzZXQpCj4g
Cj4gTml0OiBCbGFua3MgcGxlYXNlIGJldHdlZW4gXSBhbmQgIi4KVGhhbmtzLiBJJ2xsIHVwZGF0
ZSB0aGF0LgoKPiAKPiA+IC0tLSAvZGV2L251bGwKPiA+ICsrKyBiL3hlbi9hcmNoL3Jpc2N2L2lu
Y2x1ZGUvYXNtL2ZlbmNlLmgKPiA+IEBAIC0wLDAgKzEsOCBAQAo+ID4gKy8qIFNQRFgtTGljZW5z
ZS1JZGVudGlmaWVyOiBHUEwtMi4wLW9yLWxhdGVyICovCj4gPiArI2lmbmRlZiBfQVNNX1JJU0NW
X0ZFTkNFX0gKPiA+ICsjZGVmaW5lIF9BU01fUklTQ1ZfRkVOQ0VfSAo+ID4gKwo+ID4gKyNkZWZp
bmUgUklTQ1ZfQUNRVUlSRV9CQVJSSUVSwqDCoCAiXHRmZW5jZSByICwgcndcbiIKPiA+ICsjZGVm
aW5lIFJJU0NWX1JFTEVBU0VfQkFSUklFUsKgwqAgIlx0ZmVuY2UgcncswqAgd1xuIgo+IAo+IFNl
ZWluZyB0aGF0IGFub3RoZXIgImZlbmNlIHJ3LCBydyIgYXBwZWFycyBpbiB0aGlzIHBhdGNoLCBJ
J20gbm93Cj4gcHJldHR5Cj4gc3VyZSB5b3Ugd2FudCB0byBhZGQgZS5nLiBSSVNDVl9GVUxMX0JB
UlJJRVIgaGVyZSBhcyB3ZWxsLgpJdCBtYWtlcyBzZW5zZS4gSSdsbCBkbyB0aGF0LiBUaGFua3Mu
Cj4gCj4gSmFuCgo=



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 12:12:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 12:12:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680697.1058762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raE8C-0001dN-Qa; Wed, 14 Feb 2024 12:12:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680697.1058762; Wed, 14 Feb 2024 12:12:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raE8C-0001dG-Nv; Wed, 14 Feb 2024 12:12:52 +0000
Received: by outflank-mailman (input) for mailman id 680697;
 Wed, 14 Feb 2024 12:12:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raE8B-0001d1-AT
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 12:12:51 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5f42ab65-cb32-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 13:12:50 +0100 (CET)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-5600c43caddso6514553a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 04:12:50 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 s11-20020a170906454b00b00a36f9941d6dsm2236035ejq.112.2024.02.14.04.12.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 04:12:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f42ab65-cb32-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707912770; x=1708517570; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=9/LsC7QZmntGdTY3iDGNcgIvQvzAOpsEBfxLTw7/29I=;
        b=gPkK0O0m38Ew3Mhmm3Z6AKrUidHPF0MOcgvvnuLo0Svy41GK8sfDA+GakAtcgZsJNs
         kanCaZ6pxMDKDpT9Z47dmSmTc2o4CbPlgJaItGA1eyUYeWt57siBMaSFVc1Ogp5xIFaN
         3jgm/JfoyUrY9agNPWwSyOQiAyb7IOLRgdg6kLTpyIN+4epY5ShahiEBicwU3vr9adZj
         h5EKQo8wjKvF69J8MVHqXzupuBwajd6ky4TbopZ0Px1nfzjT/3dWEqsTlsrGuVEI1nRP
         xiFEgqj7Aq6toAYz873umpY5GsIp+Plwp/yFMlHmFBv3vfxsZnTsubmAyEghR+U0oQsN
         2aQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707912770; x=1708517570;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9/LsC7QZmntGdTY3iDGNcgIvQvzAOpsEBfxLTw7/29I=;
        b=pZobQJ932YSQBzxEeyO79f/pNov8OkYKZlMqKv7kKxnEUN7Q5n3I4Joiel2HPOehZe
         8BN1xoAWUgg1cZwTm3J672Rx56bUYrY1C2uZwQ46ShnfBUhl2VWVHlCin121RfQp0JqB
         TsTAhXLpNgffEKSrc5zLk3Ea0UueFEF9gUpoFRPfgeIcbP//+k52fJ8osV+kzDP3mSzX
         csLVAIa2pmEd+sNSe96fI9o6SxoadZdLoDgHNEBInuNQsv+ZFzzIIg/1b537ADU4/BOI
         wrVx2mjhbe+460eZ/nsYwV5Xqs8/1DVFgqKlJZoJoS5kqtIbF6X3mGpp6UQUjqYRcozc
         c8wQ==
X-Forwarded-Encrypted: i=1; AJvYcCUqTwDRT5C61lqnGUj2pxKOeYpcg6ZJ2XysJSHVFTC3H4hjVAi8qlHr1mTe4KMFY/KHOTOlrdE5oaCaaQ5n2P1IFC0B/PwPDZl5GlnQ67E=
X-Gm-Message-State: AOJu0Yxk/X5jflqZFIlcmGBLGq5g1XLOXkdmwJHU7zf6zhP3oDNHhFl7
	/PJXDrvX/VMdt5deESObmPVNqDnXQZeAEbOo/ma/q8or67Dfm291
X-Google-Smtp-Source: AGHT+IHag1yX4ZW/iL0HTPZQ8Kj4HYJdQD4SfJ8rblj0HMB9qTLW5Ksl2gRR5jGjmyStx8KZDtN+Cg==
X-Received: by 2002:a17:906:3b56:b0:a3c:8a77:e67d with SMTP id h22-20020a1709063b5600b00a3c8a77e67dmr1536965ejf.59.1707912769731;
        Wed, 14 Feb 2024 04:12:49 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXKJd49HtDBikmzlizfcS+OvKQPFXHBflQuxd8gKnmO5Yin9Eb/V2C7nZygU3NU2sOxQ8gT8lVI6JQbIH0CSJZgSJLD4myyTDCVJLFdt6EfvsA/WL+gcZiI8Pfu02oReRAfUSkVnApJ79EjrH2oXDDqg9WAtWvks0wuE/o/XZV5iOBCO9CNo2dlAuipOY8hAkcrlcdsPMUmh+lANfWnDZMYbQ8q+T5QSYbkSLh1cMKiYgFvv8am3AHjUExDv+fGwHsKaZRw69psLV4KDdIZVvczOozx3eGyrjhnSxEjHCiVAuHaamHBY1o=
Message-ID: <7b52a0ea4630abdaf3704f82c2619215e5bfa694.camel@gmail.com>
Subject: Re: [PATCH v4 16/30] xen/riscv: introduce p2m.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 13:12:48 +0100
In-Reply-To: <5e5887b2-5eb4-4a81-90f6-246553c28863@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <c2a2e872f8c32d81c3d3f428f0273819dd7df081.1707146506.git.oleksii.kurochko@gmail.com>
	 <5e5887b2-5eb4-4a81-90f6-246553c28863@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

T24gTW9uLCAyMDI0LTAyLTEyIGF0IDE2OjE2ICswMTAwLCBKYW4gQmV1bGljaCB3cm90ZToKPiBP
biAwNS4wMi4yMDI0IDE2OjMyLCBPbGVrc2lpIEt1cm9jaGtvIHdyb3RlOgo+ID4gU2lnbmVkLW9m
Zi1ieTogT2xla3NpaSBLdXJvY2hrbyA8b2xla3NpaS5rdXJvY2hrb0BnbWFpbC5jb20+Cj4gCj4g
QWNrZWQtYnk6IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KPiB3aXRoIHR3byBtb3Jl
IG5pdHM6Cj4gCj4gPiAtLS0gL2Rldi9udWxsCj4gPiArKysgYi94ZW4vYXJjaC9yaXNjdi9pbmNs
dWRlL2FzbS9wMm0uaAo+ID4gQEAgLTAsMCArMSwxMDIgQEAKPiA+ICsvKiBTUERYLUxpY2Vuc2Ut
SWRlbnRpZmllcjogR1BMLTIuMC1vbmx5ICovCj4gPiArI2lmbmRlZiBfX0FTTV9SSVNDVl9QMk1f
SF9fCj4gPiArI2RlZmluZSBfX0FTTV9SSVNDVl9QMk1fSF9fCj4gPiArCj4gPiArI2luY2x1ZGUg
PGFzbS9wYWdlLWJpdHMuaD4KPiA+ICsKPiA+ICsjZGVmaW5lIHBhZGRyX2JpdHMgUEFERFJfQklU
Uwo+ID4gKwo+ID4gKy8qCj4gPiArICogTGlzdCBvZiBwb3NzaWJsZSB0eXBlIGZvciBlYWNoIHBh
Z2UgaW4gdGhlIHAybSBlbnRyeS4KPiA+ICsgKiBUaGUgbnVtYmVyIG9mIGF2YWlsYWJsZSBiaXQg
cGVyIHBhZ2UgaW4gdGhlIHB0ZSBmb3IgdGhpcwo+ID4gcHVycG9zZSBpcyAyIGJpdHMuCj4gPiAr
ICogU28gaXQncyBwb3NzaWJsZSB0byBvbmx5IGhhdmUgNCBmaWVsZHMuIElmIHdlIHJ1biBvdXQg
b2YgdmFsdWUKPiA+IGluIHRoZQo+ID4gKyAqIGZ1dHVyZSwgaXQncyBwb3NzaWJsZSB0byB1c2Ug
aGlnaGVyIHZhbHVlIGZvciBwc2V1ZG8tdHlwZSBhbmQKPiA+IGRvbid0IHN0b3JlCj4gPiArICog
dGhlbSBpbiB0aGUgcDJtIGVudHJ5Lgo+ID4gKyAqLwo+ID4gK3R5cGVkZWYgZW51bSB7Cj4gPiAr
wqDCoMKgIHAybV9pbnZhbGlkID0gMCzCoMKgwqAgLyogTm90aGluZyBtYXBwZWQgaGVyZSAqLwo+
ID4gK8KgwqDCoCBwMm1fcmFtX3J3LMKgwqDCoMKgwqDCoMKgwqAgLyogTm9ybWFsIHJlYWQvd3Jp
dGUgZG9tYWluIFJBTSAqLwo+ID4gK30gcDJtX3R5cGVfdDsKPiA+ICsKPiA+ICsjaW5jbHVkZSA8
eGVuL3AybS1jb21tb24uaD4KPiA+ICsKPiA+ICtzdGF0aWMgaW5saW5lIGludCBnZXRfcGFnZV9h
bmRfdHlwZShzdHJ1Y3QgcGFnZV9pbmZvICpwYWdlLAo+ID4gK8KgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc3RydWN0
IGRvbWFpbiAqZG9tYWluLAo+ID4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdW5zaWduZWQgbG9uZyB0eXBlKQo+
ID4gK3sKPiA+ICvCoMKgwqAgQlVHX09OKCJ1bmltcGxlbWVudGVkIik7Cj4gPiArwqDCoMKgIHJl
dHVybiAtRUlOVkFMOwo+ID4gK30KPiA+ICsKPiA+ICsvKiBMb29rIHVwIGEgR0ZOIGFuZCB0YWtl
IGEgcmVmZXJlbmNlIGNvdW50IG9uIHRoZSBiYWNraW5nIHBhZ2UuCj4gPiAqLwo+ID4gK3R5cGVk
ZWYgdW5zaWduZWQgaW50IHAybV9xdWVyeV90Owo+ID4gKyNkZWZpbmUgUDJNX0FMTE9DwqDCoMKg
ICgxdTw8MCnCoMKgIC8qIFBvcHVsYXRlIFBvRCBhbmQgcGFnZWQtb3V0Cj4gPiBlbnRyaWVzICov
Cj4gPiArI2RlZmluZSBQMk1fVU5TSEFSRcKgICgxdTw8MSnCoMKgIC8qIEJyZWFrIENvVyBzaGFy
aW5nICovCj4gPiArCj4gPiArc3RhdGljIGlubGluZSBzdHJ1Y3QgcGFnZV9pbmZvICpnZXRfcGFn
ZV9mcm9tX2dmbigKPiA+ICvCoMKgwqAgc3RydWN0IGRvbWFpbiAqZCwgdW5zaWduZWQgbG9uZyBn
Zm4sIHAybV90eXBlX3QgKnQsCj4gPiBwMm1fcXVlcnlfdCBxKQo+ID4gK3sKPiA+ICvCoMKgwqAg
QlVHX09OKCJ1bmltcGxlbWVudGVkIik7Cj4gPiArwqDCoMKgIHJldHVybiBOVUxMOwo+ID4gK30K
PiA+ICsKPiA+ICtzdGF0aWMgaW5saW5lIHZvaWQgbWVtb3J5X3R5cGVfY2hhbmdlZChzdHJ1Y3Qg
ZG9tYWluICpkKQo+ID4gK3sKPiA+ICvCoMKgwqAgQlVHX09OKCJ1bmltcGxlbWVudGVkIik7Cj4g
PiArfQo+ID4gKwo+ID4gKwo+ID4gK3N0YXRpYyBpbmxpbmUgaW50IGd1ZXN0X3BoeXNtYXBfbWFy
a19wb3B1bGF0ZV9vbl9kZW1hbmQoc3RydWN0Cj4gPiBkb21haW4gKmQsIHVuc2lnbmVkIGxvbmcg
Z2ZuLAo+IAo+IFRoaXMgbGluZSBsb29rcyB0byBiZSB0b28gbG9uZy4KPiAKPiA+ICvCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB1bnNpZ25lZAo+
ID4gaW50IG9yZGVyKQo+ID4gK3sKPiA+ICvCoMKgwqAgcmV0dXJuIC1FT1BOT1RTVVBQOwo+ID4g
K30KPiA+ICsKPiA+ICtzdGF0aWMgaW5saW5lIGludCBndWVzdF9waHlzbWFwX2FkZF9lbnRyeShz
dHJ1Y3QgZG9tYWluICpkLAo+ID4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCBnZm5fdCBnZm4sCj4gPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIG1mbl90IG1mbiwKPiA+ICvCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdW5zaWduZWQg
bG9uZyBwYWdlX29yZGVyLAo+ID4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCBwMm1fdHlwZV90IHQpCj4gCj4gSW5kZW50YXRpb24gaXNuJ3Qg
cXVpdGUgcmlnaHQgaGVyZS4KPiAKPiBJJ2xsIHNlZSBhYm91dCBkZWFsaW5nIHdpdGggdGhvc2Ug
d2hpbGUgY29tbWl0dGluZy4KVGhhbmtzIGEgbG90LgoKfiBPbGVrc2lpCg==



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 12:14:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 12:14:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680705.1058773 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raE9q-0002Jr-4u; Wed, 14 Feb 2024 12:14:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680705.1058773; Wed, 14 Feb 2024 12:14:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raE9q-0002Jk-1z; Wed, 14 Feb 2024 12:14:34 +0000
Received: by outflank-mailman (input) for mailman id 680705;
 Wed, 14 Feb 2024 12:14:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raE9p-0002JY-40
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 12:14:33 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9b6f47c0-cb32-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 13:14:31 +0100 (CET)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2d1094b5568so21165271fa.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 04:14:31 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 c3-20020a50d643000000b0056022d78141sm4929633edj.56.2024.02.14.04.14.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 04:14:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b6f47c0-cb32-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707912871; x=1708517671; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=cgqebQZxQ/sWMWCoqyhd5yCUbhDXysR/gEfB1BllkAs=;
        b=TwOuTzhB6X5t1UkKgWvdk/voJJ/gQeoQr6KUMuMsdyR3M4cICh4A7C+f4V0lKC4vCI
         vYgdk4GTodr0LY2aHoKMXaTfKRrLIDY0jHNpJJuijf3Loh6aYp6W0LCXolfLu+s1qYQP
         c7fLJRR91xCYGHSGRLkANExckY4DXCL3Woj17WfABz0mf2qUHs8JfYYcBGBSEAo0G8Nn
         KyDSZIGxH7yZUus2aNGZaUXCvAe5/lQ3Rn/VlP7lQ3/AUUMSi9bRxw6CylqJzrrEtKii
         jYcTB1Q2213R2Jw7XvJvWfA6dUjwjS67w03KqBxUcarRb3DBOrnEbrmmc+rAdzbAx14N
         gWdw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707912871; x=1708517671;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cgqebQZxQ/sWMWCoqyhd5yCUbhDXysR/gEfB1BllkAs=;
        b=rG7wBK3gbJUwwlhxSPMTMtOyufmDiyNenkh+Ot6l3i5ERVXBt2mXfyQ41dbjl/Naj4
         TzbWrNe69o2fAEHqUH32ek/zqNEBSw432COWwP1z3SFa3MDdw5GNPlLhoG0D7c/7bvm3
         IqXq1hrej8gjgHmNYoO8kL7DCW2XpaDHrTDRWsTk+Lf/B+PwVl9XjYGFa4yQB6GcBH9t
         4ibsHfAjqKg5/qLWkmNV0Y6m32etXx4WsosT+RAGojUnQwIiqtwy439UHLA3revEQGh/
         V0FvGoVOYUaqv4dFtTLxpuC41b3n8txsdq2WjmKaaIF8m4+XsJR6zkB3PSv2z5JdLu01
         DxmQ==
X-Forwarded-Encrypted: i=1; AJvYcCX9NSlNPNLd5l7+7nv1ra0m09og/HCfODmMsFGl3W3Tr8WE4XLtzNZe9ND1OIi1l6du+FFUBQd/zsVbBiuDZLwkyiM9XF2ZEuzN0aMvtPI=
X-Gm-Message-State: AOJu0YzrtFH7JT9oUgcyDNJAn7pOloKpyXUbsekYjbC/2aVUFcOJNicW
	tMUFHhjAB4801X00qAffK3WKEURwTcorHZX+gr0QsEQeItz7tzqk
X-Google-Smtp-Source: AGHT+IF5CyL+SH5i3qCBgxsWPnLmvcabRCjWbUwfRz4O4qY8vosdT3eLJrNOnyCZwbSa/Fi3oI0fZg==
X-Received: by 2002:a2e:91ce:0:b0:2d0:f6c7:906b with SMTP id u14-20020a2e91ce000000b002d0f6c7906bmr1756615ljg.42.1707912870499;
        Wed, 14 Feb 2024 04:14:30 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVI3ODmz/PC6/hPyQakbG6DiO9oplU06Zy+GMF4mlQ3HFb9vqgi/cietpi8rJw+HUNBZQk8Tavho9Sntu3aMnXYYvRUwqCXnaYphaMivGDeCL4qfJ8U1omq/gJnZf6K8/seUJP/d5Yra/uQIyD0/BD/+JCHVLDYIR1d+NTCSttRU4wZIerbkzTwuiImJEfkI7KIZiRLLdJeij7WP2Nbv0xkqXxqfuf2AT2uBCcGxgl40YSBeH8B7hSfN7CooZ5iMd3q9Qt02T/KARd0/iGlZSs4WZxmvk1aDvy+emBdhT+6g/xEg1cM1Ls=
Message-ID: <7dd960c8bdce416f7cecc29f8794b5e5a35ee8cb.camel@gmail.com>
Subject: Re: [PATCH v4 18/30] xen/riscv: introduce time.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 13:14:29 +0100
In-Reply-To: <2962005e-252f-453e-a8fe-25856b16fe57@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <f0a4566c185469fa9f5de41b5bbbc7234cfa9912.1707146506.git.oleksii.kurochko@gmail.com>
	 <2962005e-252f-453e-a8fe-25856b16fe57@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 16:18 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > Acked-by: Jan Beulich <jbeulich@suse.com>
>=20
> Nevertheless ...
>=20
> > --- /dev/null
> > +++ b/xen/arch/riscv/include/asm/time.h
> > @@ -0,0 +1,29 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +#ifndef __ASM_RISCV_TIME_H__
> > +#define __ASM_RISCV_TIME_H__
> > +
> > +#include <xen/bug.h>
> > +#include <asm/csr.h>
> > +
> > +struct vcpu;
> > +
> > +/* TODO: implement */
> > +static inline void force_update_vcpu_system_time(struct vcpu *v) {
> > BUG_ON("unimplemented"); }
>=20
> ... nit: Too long line. The comment also doesn't look to serve any
> purpose
> anymore, with the BUG_ON() now taking uniform shape.
I'll drop the comment and move "BUG_ON(...)" to new line.


~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 12:16:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 12:16:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680710.1058782 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raEBg-0002vv-Ed; Wed, 14 Feb 2024 12:16:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680710.1058782; Wed, 14 Feb 2024 12:16:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raEBg-0002vo-CA; Wed, 14 Feb 2024 12:16:28 +0000
Received: by outflank-mailman (input) for mailman id 680710;
 Wed, 14 Feb 2024 12:16:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raEBe-0002vi-VJ
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 12:16:26 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dfdf60cd-cb32-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 13:16:26 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-5638c4a8d6eso338139a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 04:16:26 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 vs2-20020a170907a58200b00a3d5b5221bbsm301885ejc.141.2024.02.14.04.16.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 04:16:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dfdf60cd-cb32-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707912985; x=1708517785; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=E/IXyESE4Z5lTt9/WS9WESTAg/XUURSl0UZdLXssyhg=;
        b=PMLBJ4Ew3PuNBkMkziUSb0VImnXBA0s+Zf3oNMBElcBGOEYHpqdVmXGVBUTh1L9wG2
         3qHhzbCSgbwi46TK1QWlHopzYjWb5J4FzgMkQMzsfQiutsUC0jVjwJaCVfqc5xVNGux6
         5QGfhgU1Xwju04Sj9qlxLxGTFgZVp7tqfihJWGTrTe3XddWpxU6Wzxc1+y99h4GX0zMt
         sMn+DCCVMTBiKJRzBzULwmswbphQATRkdQVR1YPQmLUq3lZCmp1QUI1dp6us7HXB/gsA
         ugl2l7e9Hq/9uUGeLXj91+1rbowLS8/lfAoadkstRCnjWYnnVBW8erSDMuF8moBKHMh8
         mM8g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707912985; x=1708517785;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=E/IXyESE4Z5lTt9/WS9WESTAg/XUURSl0UZdLXssyhg=;
        b=jvTQsita8P8L5i480fm0UWhYDo0vtIQpnMSecS0USLAeYmEX6E3Ax0CiNDBMToIFlW
         wZree41hhFS78auzdZGtKYd1FCu/DnL8PDM61/AYCIP86xK/pgrbAdjWbiuR6xfDBb41
         J+Ih7Ixx+BgqPU9HT1tL4nNRuxpZkfJjix4kb7Je2ZHmAil18bHoz4/w3ne3pLR3Pm7F
         lvQQv5QMO1JU7KtB3VjvR5TAmzYD8vjHF3UHiqxr4naww+ryKLXYNTtnFVjU5tfqSFFd
         CgGY1GeXEIlLsds/SwHSdDYnmYoUU5/2iv7LFV4lfEzcbuHx8ESO1zEQIR6lQhtMdk0W
         zoRQ==
X-Forwarded-Encrypted: i=1; AJvYcCXHOgdPp99wOaZQAfSRMYP2P13GmZmF6hMZMiW+IVvg2smZyspNNE+fMplZRHcqxeUoy860mEcNhpV8BaZnmNG2nAjU446PqvQgGjpUOwc=
X-Gm-Message-State: AOJu0Yz0GzgKHPmqF6CJd6fRSZKftTPiEax/ZCX/zHaYuEoTAJCAdree
	ZUd7dWdYZzgcCFsBN6wnp3DngB1Qdq8CrxcddqXw/GsELwExCf+R
X-Google-Smtp-Source: AGHT+IER8QuamIdkDMFDgBnmG3CnFbvCkYKlE1vob/BlPUua6lxzawWnI5Yb39sHCH0dJfyvQb2UiQ==
X-Received: by 2002:a17:906:5596:b0:a3c:eeef:2283 with SMTP id y22-20020a170906559600b00a3ceeef2283mr1555308ejp.43.1707912985462;
        Wed, 14 Feb 2024 04:16:25 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXL9iMGe537taVKj48hcR93UOO5CjYv7ZC5hcdZ4NvHjBIUHn5RjHTjMTaupSFI1MWgQfPWemwXLRNRmJxMJe90BGOqIurDeOzZS1bDyZOgi5t8j0zCV8NTh8n0J1OXIAjbQD+DcTEOKQl6rd7FKbIi/QPz/q3yAAYdk9wm5cd45mzgR0uv3cLBUquD64D8E/yVft3mvHOMw4uDluw6TxESOBsRHyMBwfMzHFGpInnK6Pu9aiWs6z3nUr7skHZGCOf9VLgRbTCh7oXhcwEE83Ry4fOknj2OFeRe2z92XkKyQBMyhnNpaNQ=
Message-ID: <39e07917c237a2d2e8879a66511edef4590d9fba.camel@gmail.com>
Subject: Re: [PATCH v4 19/30] xen/riscv: introduce event.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 13:16:24 +0100
In-Reply-To: <36b25710-cf95-4250-bf4a-6ccf4f418909@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <a57aed8d2d8424c63b1f27d007b2d5cfe35460cc.1707146506.git.oleksii.kurochko@gmail.com>
	 <36b25710-cf95-4250-bf4a-6ccf4f418909@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-12 at 16:20 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Acked-by: Jan Beulich <jbeulich@suse.com>
> again with a nit, though:
>=20
> > --- /dev/null
> > +++ b/xen/arch/riscv/include/asm/event.h
> > @@ -0,0 +1,40 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +#ifndef __ASM_RISCV_EVENT_H__
> > +#define __ASM_RISCV_EVENT_H__
> > +
> > +#include <xen/lib.h>
> > +
> > +void vcpu_mark_events_pending(struct vcpu *v);
> > +
> > +static inline int vcpu_event_delivery_is_enabled(struct vcpu *v)
> > +{
> > +=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> > +=C2=A0=C2=A0=C2=A0 return 0;
> > +}
> > +
> > +static inline int local_events_need_delivery(void)
> > +{
> > +=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> > +=C2=A0=C2=A0=C2=A0 return 0;
> > +}
> > +
> > +static inline void local_event_delivery_enable(void)
> > +{
> > +=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> > +}
> > +
> > +/* No arch specific virq definition now. Default to global. */
> > +static inline bool arch_virq_is_global(unsigned int virq)
> > +{
> > +=C2=A0=C2=A0=C2=A0 return true;
> > +}
> > +
> > +#endif
>=20
> This want to gain the usual comment.
Do you mean that commit messag should be updated?

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 12:21:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 12:21:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680716.1058792 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raEGf-0004t8-1K; Wed, 14 Feb 2024 12:21:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680716.1058792; Wed, 14 Feb 2024 12:21:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raEGe-0004t1-Uw; Wed, 14 Feb 2024 12:21:36 +0000
Received: by outflank-mailman (input) for mailman id 680716;
 Wed, 14 Feb 2024 12:21:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C2D8=JX=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raEGd-0004rR-Fz
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 12:21:35 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 97001d7e-cb33-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 13:21:33 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a2d7e2e7fe0so131400366b.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 04:21:33 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 qb35-20020a1709077ea300b00a3840fbeef9sm2257224ejc.70.2024.02.14.04.21.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 04:21:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 97001d7e-cb33-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1707913293; x=1708518093; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Aqu1pzReXGCcQGJ62RxsKSX1lKOwf9R4esB0hrUWg7E=;
        b=Fu8bbS/DWa0slUos+Kz1cRsJUhrCyX2p0EBiT2srf12lAW5OAEzmehwnxetENrTFVp
         WaNiHmSqufJQ3unxr6qZ1SQczgezWJL1UPCSsLMf8K6PXUCJZKIuE48XHOvztTBNpMf3
         PoIO7w1DD8ZtBvK0WoPySoc//OS1yvCSNXBJooZRL4KpD7RvXi5Nh25fKU+GPNPVKdD3
         U9/riYSSbcw9vdIr8XIfPTpBf0i11JQyGSJCzdScIlk0m6OIzKkbVxrHc9Uha0QSft9V
         D4Ia8avh6g/SXblMIFI1sx4OMBXIv/EzDtqVni+23bhJwJ+qeM4v01ieSMME5BGC7CdZ
         H/KA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707913293; x=1708518093;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Aqu1pzReXGCcQGJ62RxsKSX1lKOwf9R4esB0hrUWg7E=;
        b=jSZ80dFphu4CNLah9MVPFNgCSC5VKzGt9oDaRe/7fZUpJ03YIgV74zXWwxHsGv5wjS
         RZT6eFGbZXy59jleMx1VjbIIdUstznhA3wmMosd4BhPYS0Umw4Ctwag2npf9kDnr41gk
         wU5g76LNSWtYHDJiXcHQJTPZTojEm1ZTvl1toj72Hu+uL5Rc9yHg0NBLLwkAE6jaEFQt
         c+U2DWU2LvfEY5R7Ch5gl1cf4MXSAH4NTPegpydTh3j4uvPaQ1fUfr224Z9IEW+0+nQu
         gGY4g0cRq1wi3iRdTPQe5XzLOnTamXAWrtWtvmtmglabfnpW+poMoth9qeoK0TSwbpm+
         OIiQ==
X-Forwarded-Encrypted: i=1; AJvYcCWtraxhbAGVUdN2pu+SCKvFzs/GNOTaAPHuVCMcRVaaGmrl+HoJLTJASk8GURalnIKV1MzaxTP4CCUp7H6eW+oVEE8UCsV4HQL21bgHaIs=
X-Gm-Message-State: AOJu0YzISFhGiBEIa//DytAEsBFHbaaqSTWZDOrJMEGHqVw1A/CFU9NJ
	U9TyNckG46O2QF/N+/04unz7gVu5/w5sR2ydnSYLjUM0hAj6WmDk
X-Google-Smtp-Source: AGHT+IHgSyZjknsoS3eVmO0IjIwoFtHC61etgbM/wfT5AYEt/1d/GZ+pG3DoS/J76TIZoHf2lxZ1CQ==
X-Received: by 2002:a17:906:2614:b0:a3c:698:635f with SMTP id h20-20020a170906261400b00a3c0698635fmr1496045ejc.28.1707913292778;
        Wed, 14 Feb 2024 04:21:32 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXNkUa3gWujoZTuqWzPYaCRP+QxwMjBUyik8pAgbXd5daCCThMKgTC5mnmWY1Abwue02IuKSfGq89s40F3Y86IUNbekuOOIpaUqDCfjI0dsri2nzp+O40i/JEFSq/WgGy3UxpZiNOo05qv6TuruKO0DyjL2J/a2m1f9SayTZbxOvAqlnosDUvQcUKTKM9f0fAxAVvyhrdBqZWEx3nY=
Message-ID: <4b7ff366d58089a1c755c01cef21c503f408464b.camel@gmail.com>
Subject: Re: [PATCH v4 30/30] xen/README: add compiler and binutils versions
 for RISC-V64
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Date: Wed, 14 Feb 2024 13:21:31 +0100
In-Reply-To: <16162577-dc0f-4c4b-acd5-9c2519f94c9a@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <d62dff38ee661f3fb713554d544c966fa889fd83.1707146506.git.oleksii.kurochko@gmail.com>
	 <16162577-dc0f-4c4b-acd5-9c2519f94c9a@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Wed, 2024-02-14 at 10:52 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > =C2=A0Changes in V4:
> > =C2=A0 - Update version of GCC (12.2) and GNU Binutils (2.39) to the
> > version
> > =C2=A0=C2=A0=C2=A0 which are in Xen's contrainter for RISC-V
> > ---
> > =C2=A0Changes in V3:
> > =C2=A0 - new patch
> > ---
> > =C2=A0README | 3 +++
> > =C2=A01 file changed, 3 insertions(+)
> >=20
> > diff --git a/README b/README
> > index c8a108449e..9a898125e1 100644
> > --- a/README
> > +++ b/README
> > @@ -48,6 +48,9 @@ provided by your OS distributor:
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - For ARM 64-bit:
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GCC 5.1 or later
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GNU Binutils 2.24 or=
 later
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - For RISC-V 64-bit:
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GCC 12.2 or later
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GNU Binutils 2.39 or late=
r
>=20
> And neither gcc 12.1 nor binutils 2.38 are good enough? Once again
> the
> question likely wouldn't have needed raising if there was a non-empty
> description ...
I haven't verified gcc 12.1 and binutils 2.38. gcc 12.2 and binutils
2.39 were chosen because this veriosn is used in Xen contrainer for
RISC-V, on my system I have newer versions. So this is the minimal
versions which would be always tested and I can't be sure that the
lessser version will work fine, as there is not any compilation testing
for that.

>=20
> Also - Clang pretty certainly supports RISC-V, too. Any information
> on
> a minimally required version there?
I haven't verified that. I am only testing gcc for now.
I can add this information to commit message.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 12:55:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 12:55:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680727.1058802 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raEmr-0003Yv-Fz; Wed, 14 Feb 2024 12:54:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680727.1058802; Wed, 14 Feb 2024 12:54:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raEmr-0003Yo-DU; Wed, 14 Feb 2024 12:54:53 +0000
Received: by outflank-mailman (input) for mailman id 680727;
 Wed, 14 Feb 2024 12:54:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raEmp-0003Yi-IL
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 12:54:51 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3d3d227b-cb38-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 13:54:50 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d0d799b55cso68004141fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 04:54:50 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p15-20020a05600c1d8f00b0040ecdd672fasm1951305wms.13.2024.02.14.04.54.49
 for <xen-devel@lists.xenproject.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 04:54:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d3d227b-cb38-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707915289; x=1708520089; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0nGc33uiEkM41QRyRyElvnJ93fT7k0IZrbbvFNTqVG8=;
        b=UnWMWT2MAAvg++1t/JJPApbbRtqLZlQchQ5w/FB5jwd5d1NppE0p6iM0dFj1PS9ReF
         /D3P0jQoTvxfKozpwDRp/tneFHKY6JcTMxbxneiGcF2yqeEVini18cxdE1lkzVYuEW9y
         7SOQ6wZazBUKy9Sc4fvGuCmXDX4oKTqUB87RZyh9S5zCxKWB3ZSNPFNNOfmWWJTt+xNv
         yrc899NMm45kbRsVyCKd3zDxKyQAzBPEPHRmWGIAElSMbZyBOrpz88YjtmgJpCrmfB2d
         vWVHNMZmAjGNgrj5m0WKTe4rhtNMVsxfNMIvfdDNa7ysXiR682HaiHaRJfksH8PuYQe8
         c/Ig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707915289; x=1708520089;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0nGc33uiEkM41QRyRyElvnJ93fT7k0IZrbbvFNTqVG8=;
        b=FFrwv5gNthWUErIS/YL9sREvn/SkZk+cjUFsIui4pHQr2Bz2C1OTaNf6ZtC8wW6Wwi
         WU8E85VRU+OGUmD+Stzn9cuqPW2TXoeoEQlmqPRmu8bN4FPMFssx7H83tyQ0oEAgo8Ot
         tAv7fXx63pctinW80M3qr9TiP5vXN67FqPvIW6byPlw4lxMmdmMFZm2tkI7ZLYi8mgqn
         4cKwBou6vH+bLqtzUN/fBjqds+0z6yYE8+dkaZ3T0JDj7vI/TfhOUY/aUncGL9my3/js
         b67ylESLWOZq2yD5UoFgvMhxdJdi+6RXXODolKIPia44VXsUAlrKfiPZnSaPt458COed
         vqiA==
X-Gm-Message-State: AOJu0YxNUqo5Euqbu5Wr6sCj+C458QVakSxzqeY3W/gqVmVy2JqEJ3gE
	KJ+MPh2pMrLhjw9TdOMzRhOtAl1KEt3nkrQZah3GcM27dgNKNshD91cggVyLIvXoBR+9CHUf6UM
	=
X-Google-Smtp-Source: AGHT+IHHjOGUJLNnpIg4wkrzXEXBnk8IrLSTpeoy2+hNtpJz2R5NBfFWLzOVMhayeKREwo1FJXvq1w==
X-Received: by 2002:a2e:b0d2:0:b0:2d0:c1df:b4df with SMTP id g18-20020a2eb0d2000000b002d0c1dfb4dfmr2213413ljl.39.1707915289652;
        Wed, 14 Feb 2024 04:54:49 -0800 (PST)
Message-ID: <12ce9813-d9a3-4770-b7ab-a5ccfef65b23@suse.com>
Date: Wed, 14 Feb 2024 13:54:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: xen | Failed pipeline for staging | d670c1a3
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <65ccb539c5f56_2c88e1496072@gitlab-sidekiq-catchall-v2-5d948c44f-qdsrj.mail>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <65ccb539c5f56_2c88e1496072@gitlab-sidekiq-catchall-v2-5d948c44f-qdsrj.mail>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 13:42, GitLab wrote:
> 
> 
> Pipeline #1176167215 has failed!
> 
> Project: xen ( https://gitlab.com/xen-project/xen )
> Branch: staging ( https://gitlab.com/xen-project/xen/-/commits/staging )
> 
> Commit: d670c1a3 ( https://gitlab.com/xen-project/xen/-/commit/d670c1a38ba3561296f68c0079209365760b3001 )
> Commit Message: libxl: Fix comment for LIBXL_HAVE_VMTRACE_BUF_K...
> Commit Author: Petr Beneš
> Committed by: Jan Beulich ( https://gitlab.com/jbeulich )
> 
> 
> Pipeline #1176167215 ( https://gitlab.com/xen-project/xen/-/pipelines/1176167215 ) triggered by Ganis ( https://gitlab.com/ganis )
> had 1 failed job.
> 
> Job #6169539038 ( https://gitlab.com/xen-project/xen/-/jobs/6169539038/raw )
> 
> Stage: build
> Name: yocto-qemux86-64

Hmm, pretty odd: Dom0 crashes during boot with

[   10.642958] segment-related general protection fault: e030 [#1] PREEMPT SMP NOPTI
...
[   10.642958] RIP: e030:native_irq_return_iret+0x0/0x2

when E030 is also what's reported to be in CS (with the RPL bits stripped
off). According to the stack trace it's in the middle of applying
alternatives, with xen_irq_enable_direct() also on the stack. So maybe a
badly timed interrupt that normally we get away with?

Sadly modern Linux doesn't dump raw stack contents by default anymore, so
it's impossible to see what exactly was on the stack, for the IRET to
consume.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:06:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:06:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680733.1058814 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raEyE-0005lX-N8; Wed, 14 Feb 2024 13:06:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680733.1058814; Wed, 14 Feb 2024 13:06:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raEyE-0005lQ-Io; Wed, 14 Feb 2024 13:06:38 +0000
Received: by outflank-mailman (input) for mailman id 680733;
 Wed, 14 Feb 2024 13:06:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raEyD-0005lJ-OL
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:06:37 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e1907fcc-cb39-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 14:06:35 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-410e676c6bbso15157975e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:06:35 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n9-20020a05600c294900b00410cc2f5550sm1874323wmd.19.2024.02.14.05.06.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 05:06:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1907fcc-cb39-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707915995; x=1708520795; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/+SBJH3tyuiPYvHw7rIW+L4RszvBgnNudqwloHmO/Us=;
        b=Z0X/x97xmj+4UdYCpSsnwuC887C7oKk2ZJksQV1XVL9VxwJUyrCnnwmYIiSjew+0/5
         sWIrzwarAsVjGcIeML/IUccxP1p0f4Lmkl32HfZLo5i5bqzuKs69JKM8YHJlBGLw9EIm
         H2jRZSCAqkpG6cgSDLRYcipL8YPWrFDFfp659p9xAvtKTLeuCeU2InAG+HB+cOObnxVo
         dGs2C0ycRvXychrPFVoV8YxE/csx963efJbWTuWbVg7NZVJepVX6uLbL8SWinY18bJW0
         DQWPzG8IyV8I2Rys0XDAVsUH+Szyos5E1NqccHZ65LuDM4XB2ibLCpv/0nr5csvzZ+bc
         yDVA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707915995; x=1708520795;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/+SBJH3tyuiPYvHw7rIW+L4RszvBgnNudqwloHmO/Us=;
        b=AtkyizKuShkqI02vuzsSSQCLM0im6VuiPAxeCaphrG2DsCiscXwqiInkY//T/deJaw
         nIGIpLemDyrkJ2PwBLsvjvtb6S2enSzj4tr+oKIOYePLRwHpCy+Gliibv5Vl9mJeEtSI
         q+7P0f2fYieDy4YnaaI5zapdc42x4XaKPc5BW9vJilAfXVua6q/Sgp8i2pMFDXqQtFrY
         O6YGDiOBiqqXD9HnoGHLsM4YZDwPBU/u2+clmXM3jqxo0LI2cclzU8LDs/rpZGho3RKH
         VNPcwFdf/EpsCAwTKOy3xmrg4klylU8iy67LUirLGMP6IePnudhVu3M4q/W79ve1DoUK
         plDQ==
X-Forwarded-Encrypted: i=1; AJvYcCXjGizBXl7/cplOlDonep/U9xMg29PTUE/7CpEU+MNwQuFEqoD1ocvrDgokrxlM8KNid/tUVRejJtvxMNAC+2DYqRmzqwxYW/gGgI3FE4M=
X-Gm-Message-State: AOJu0YzjmPWXWhf3DorJZ28KX9AyphJq5ZLR837JuPQTAWTOE8KVmns4
	elf6iTMvgl++k/x0AqB514wbdLiggu9yICYJy3AX9DxhphAoBJ8mToPK7YrMEQ==
X-Google-Smtp-Source: AGHT+IHD4CcEeoZH1+OX4/V+HxpW3HboiNhhdvuL2DWjtyLH0Y08NzWxImaf33FkaHhye4qtMByUaw==
X-Received: by 2002:a05:600c:470f:b0:410:d1bd:150e with SMTP id v15-20020a05600c470f00b00410d1bd150emr1869377wmo.14.1707915994916;
        Wed, 14 Feb 2024 05:06:34 -0800 (PST)
Message-ID: <d63e0139-8f75-4c14-b612-27b003592d3d@suse.com>
Date: Wed, 14 Feb 2024 14:06:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 30/30] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <d62dff38ee661f3fb713554d544c966fa889fd83.1707146506.git.oleksii.kurochko@gmail.com>
 <16162577-dc0f-4c4b-acd5-9c2519f94c9a@suse.com>
 <4b7ff366d58089a1c755c01cef21c503f408464b.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4b7ff366d58089a1c755c01cef21c503f408464b.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 13:21, Oleksii wrote:
> On Wed, 2024-02-14 at 10:52 +0100, Jan Beulich wrote:
>> On 05.02.2024 16:32, Oleksii Kurochko wrote:
>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>> ---
>>>  Changes in V4:
>>>   - Update version of GCC (12.2) and GNU Binutils (2.39) to the
>>> version
>>>     which are in Xen's contrainter for RISC-V
>>> ---
>>>  Changes in V3:
>>>   - new patch
>>> ---
>>>  README | 3 +++
>>>  1 file changed, 3 insertions(+)
>>>
>>> diff --git a/README b/README
>>> index c8a108449e..9a898125e1 100644
>>> --- a/README
>>> +++ b/README
>>> @@ -48,6 +48,9 @@ provided by your OS distributor:
>>>        - For ARM 64-bit:
>>>          - GCC 5.1 or later
>>>          - GNU Binutils 2.24 or later
>>> +      - For RISC-V 64-bit:
>>> +        - GCC 12.2 or later
>>> +        - GNU Binutils 2.39 or later
>>
>> And neither gcc 12.1 nor binutils 2.38 are good enough? Once again
>> the
>> question likely wouldn't have needed raising if there was a non-empty
>> description ...
> I haven't verified gcc 12.1 and binutils 2.38. gcc 12.2 and binutils
> 2.39 were chosen because this veriosn is used in Xen contrainer for
> RISC-V, on my system I have newer versions. So this is the minimal
> versions which would be always tested and I can't be sure that the
> lessser version will work fine, as there is not any compilation testing
> for that.
> 
>>
>> Also - Clang pretty certainly supports RISC-V, too. Any information
>> on
>> a minimally required version there?
> I haven't verified that. I am only testing gcc for now.
> I can add this information to commit message.

Yes please. And if this isn't a firm lower bound, that fact imo wants
reflecting in README itself as well.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:09:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:09:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680736.1058823 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raF1A-0006Uy-3c; Wed, 14 Feb 2024 13:09:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680736.1058823; Wed, 14 Feb 2024 13:09:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raF1A-0006Ur-0Y; Wed, 14 Feb 2024 13:09:40 +0000
Received: by outflank-mailman (input) for mailman id 680736;
 Wed, 14 Feb 2024 13:09:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raF18-0006Uh-8O
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:09:38 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4c9641d1-cb3a-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 14:09:34 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-511898b6c9eso4093275e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:09:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m10-20020a05600c4f4a00b00411e6243e70sm1933730wmq.12.2024.02.14.05.09.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 05:09:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c9641d1-cb3a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707916174; x=1708520974; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2WgV/KpmqVIoQsd4n3Phk/UVFdI4GBNBCLy9AxSUPqY=;
        b=H0YUrLmI4NQLCBCq1kpjJtOSPJbB9g+KwXFzbqWbFZ1JPN1YUvujMtY1QLZVPluU8Y
         SitW0vxXa94DG/wYXxQ/ec4v2n0QilLKfdAC1gnMBxDnQz1wkjrG0yRuxiTeMikviENl
         rm6gn6F+TTnT5P7tLmDhrPQ6deWgvQ4hE1vqqyORfzEGCjKZDDd3Ec3gAwwHBWSYPRgm
         VVJ8/KjMr+9DcViMgpK+Opoxm07W6aIJNsrNTQ/+/1DCDJhTwtazB9ZYN7SzpkgWE5TE
         bAWyIr5a5tAMLz269XxYFhPq72tllLpLdZa5BoTmrOPcbYiD1fo0B7qKfp9gdgRXQXMY
         SyKw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707916174; x=1708520974;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2WgV/KpmqVIoQsd4n3Phk/UVFdI4GBNBCLy9AxSUPqY=;
        b=bTqWAx94WzW0e5K/aNk2QLjLHhgkp6z42kTPu17ncQC9fFlfdZC6eg6Q2FD6dHuhqh
         f4NSCPfsVUSqT1nqwHN5T0ZHQa6/VCvKCbLOWZCIyrvgGMGLeEDovjSHT9uVgN+jDDbi
         msO4MDogkApRud7H36xvrC2r2ojrp7Lw5T/pLhvgpXfbOV23JGXP9m41IuXy8Y4iCly3
         ukbx0AEqE//bIqVLwvkXzzd8ds0ohwUBlhNA3a3JSJIFh3zKRROjh44OMlTyJYQi3AKV
         zFgCnbRUu5qFptsoFicoYiiA3jxgm8eP1gThhp6cDtGIIRqgn8r//MGvubd/xuOpHAVG
         ZNzg==
X-Forwarded-Encrypted: i=1; AJvYcCVN9+Ktyd56Mt7ZYmQlzbJ2/I2ezeTJcaxu1lHb0iPVw7Twavu4Ggq+CbBq3ILHnfBHI0CijRgzgTQoElAbLjph4MeESvZFl/67jMIOeqs=
X-Gm-Message-State: AOJu0Yyw9OxBLrSwRlildbzYjmGnjQtrA/Q8fjAF8/vJWn3uAJHTseCX
	lVYSmLg+yQjxclLIyrLCp2qt5dy6vmLyrn92h9tpG26NhFRHRP1Ity7vHmaTOg==
X-Google-Smtp-Source: AGHT+IF5tn5KKuczaaH54Vs9gcimwfvNX1ijsYcJrHo4SP7BOAdGX2/EFM1sFDhtXCRHWyVqJugnOg==
X-Received: by 2002:a05:6512:3983:b0:511:7c0a:da3 with SMTP id j3-20020a056512398300b005117c0a0da3mr2193477lfu.25.1707916174436;
        Wed, 14 Feb 2024 05:09:34 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWhE/UnIeLJH/KKuV5WbfZRXqwFo+AXyrgTSNRG9t6b8cuT925ZRg3D3b4+EuKebpSxPQucivq8Wbw0r/g9/9FnK/TXZOLsRsQ0IuLdGJeHKCxd2zgqiNcL6UDTxnNpeBuJ65ipVDZPxg0w0uJFAwj+1C5uNCa+GnnF+qGe2o0fwksvF2uAjbHZhwzhRY9krD3qwYFO2tCH/lU/usvyfzZQ+FdWlItTChqwy4/zj/pnrW21QtojjCk74Xy/lpE88/v1c/hLW76HIdIezWBLJnwfEldudTUXV4ckVaoGpU9HrLGTcBjydg8U
Message-ID: <59312f50-d36c-48d8-8d95-e1abe48d9bc3@suse.com>
Date: Wed, 14 Feb 2024 14:09:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 14/30] xen/riscv: introduce atomic.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Bobby Eshleman <bobbyeshleman@gmail.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Connor Davis <connojdavis@gmail.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <6554f2479e19ed3eae6de842ac1568c31d236461.1707146506.git.oleksii.kurochko@gmail.com>
 <134e09c9-776a-4261-a28b-137d109b62ef@suse.com>
 <8af3bbf27c51f4b5d8663b537c3123c6e05e74e7.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <8af3bbf27c51f4b5d8663b537c3123c6e05e74e7.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 13:11, Oleksii wrote:
> On Tue, 2024-02-13 at 12:36 +0100, Jan Beulich wrote:
>> On 05.02.2024 16:32, Oleksii Kurochko wrote:
>>> --- /dev/null
>>> +++ b/xen/arch/riscv/include/asm/atomic.h
>>> @@ -0,0 +1,395 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +/*
>>> + * Taken and modified from Linux.
>>> + *
>>> + * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were
>>> updated to use
>>> + * __*xchg_generic()
>>> + * 
>>> + * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
>>> + * Copyright (C) 2012 Regents of the University of California
>>> + * Copyright (C) 2017 SiFive
>>> + * Copyright (C) 2021 Vates SAS
>>> + */
>>> +
>>> +#ifndef _ASM_RISCV_ATOMIC_H
>>> +#define _ASM_RISCV_ATOMIC_H
>>> +
>>> +#include <xen/atomic.h>
>>> +#include <asm/cmpxchg.h>
>>> +#include <asm/fence.h>
>>> +#include <asm/io.h>
>>> +#include <asm/system.h>
>>> +
>>> +void __bad_atomic_size(void);
>>> +
>>> +static always_inline void read_atomic_size(const volatile void *p,
>>> +                                           void *res,
>>> +                                           unsigned int size)
>>> +{
>>> +    switch ( size )
>>> +    {
>>> +    case 1: *(uint8_t *)res = readb(p); break;
>>> +    case 2: *(uint16_t *)res = readw(p); break;
>>> +    case 4: *(uint32_t *)res = readl(p); break;
>>> +    case 8: *(uint32_t *)res  = readq(p); break;
>>
>> Why is it the MMIO primitives you use here, i.e. not read<X>_cpu()?
>> It's RAM you're accessing after all.
> Legacy from Linux kernel. For some reason they wanted to have ordered
> read/write access.

Wants expressing in a comment then, or at the very least in the patch
description.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:11:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:11:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680740.1058832 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raF34-00085B-DD; Wed, 14 Feb 2024 13:11:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680740.1058832; Wed, 14 Feb 2024 13:11:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raF34-000854-Ac; Wed, 14 Feb 2024 13:11:38 +0000
Received: by outflank-mailman (input) for mailman id 680740;
 Wed, 14 Feb 2024 13:11:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raF32-00083h-EF
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:11:36 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 93b38bfb-cb3a-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 14:11:34 +0100 (CET)
Received: by mail-wr1-x42c.google.com with SMTP id
 ffacd0b85a97d-33ce2121d5dso671916f8f.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:11:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bp9-20020a5d5a89000000b0033b4796641asm12593638wrb.22.2024.02.14.05.11.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 05:11:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93b38bfb-cb3a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707916294; x=1708521094; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ukviu3LuM/V8muzTRtwHLcezGkJt0UeVaScTyjdwqDg=;
        b=B6gHuwa90bTm2Ed1vm4KRhAHfY9B7AYN+A100oITJ+OShAyFuyyh4VXpn4Y/Ckrwfm
         KgdDIPPpazk7uOHaA94yf74WOGJI3YAuhshdi5ptlrcN4kzW9vrLiqsW1lxoT088WKQ4
         GRCUKiTdzIh1n6J3DDD0S9d7UA98l/x2uYfCue21LVybROxZW4l9ZCaiC4YoHzXNCDEx
         6b1DnYNAolrQcIZ6VkWa8fhPnN541njeD3ySrTTKFWd/ByT9yqdekdTJFaSUcpKnmlaa
         yLRoxMF4QD9czR0KTEZK6LQMCiVmivPqDKabVQ7yVZ7najl73zlzUFkHrUwsMEgUgE5h
         mBDQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707916294; x=1708521094;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ukviu3LuM/V8muzTRtwHLcezGkJt0UeVaScTyjdwqDg=;
        b=JY77KXte2YRTh+tOolSpU2CkvcQIbluylnkoxVrvthbsV/LchHW5ikBU/HPVLLxln3
         bRh/GBCy/lOLX9GlAp1hZD9hG0FXf0/aqfSlS/pbwZchzmf9j6JwRxsGt0e9XbSZkgwm
         8+iqJ8PxrgR4NEmsiq8qK8cYB2171bEOqSMt1gTL5kzSjYvAwj0lWCZedL3vrYnX6CrP
         6/wCfpXPIAzfIFtqDjEA7FYZHgSVffb+zgFUVqyXvGrsMJmbn6WUUmFmN/oh1U032gpA
         c1b0Hjb5/1hLu7ri0JojlrFTbo+ABlrjmzxT8umDVevJwcjoJuZrQj8h5zqIZiFZgwXn
         AtZA==
X-Forwarded-Encrypted: i=1; AJvYcCWZulEYhqy+Thk3fsBSZBomYEMwa8US78MA+frjnsbswh4rLy0B+/G/QwA+tklw36OqDpNgFexVXluFJ+O73dIJe0fG2DspyN+5vqGFUyg=
X-Gm-Message-State: AOJu0YzQS8V+U03I13b3HELgJ1mXwY9SYXkKKSp18v7BSv8R3T7OHL6o
	tJWAmd658aSe3zlj05PF2c1HuAm2MgN/pLtvsJMvKeD29raCc8Y1hVpKrVIJlg==
X-Google-Smtp-Source: AGHT+IF8GBh8485TPMDJWdvJhzAzeZzpHPe+Kh2EwqXpAcazV/83Xs+K1KUzJsDA03Av4OIuNS1WIA==
X-Received: by 2002:a5d:58eb:0:b0:33b:3a90:13ea with SMTP id f11-20020a5d58eb000000b0033b3a9013eamr1765462wrd.4.1707916293756;
        Wed, 14 Feb 2024 05:11:33 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXSmeageTS8VktOPPX3Uu18z0f9YjJFoRX+zVHgdsfL8KQDl9Jy4F0Bsh7bRdntwnMaW8pdG/VriPHgJZEXyDx3dkb1O8b8eETCGQMwco399XYKgko2o34Zf+Va1GuKGfJTaosLMEP3I3RdD/WFiCOmIEVATBJ0GOjOSuKb6sGj10SY9G/4CYPhUAInz3C5zocB5VoorcuUCTJj1p24XtDRy1ji+HcPjPcwgxAf7Oa8vT11TyKO40zWnYZ97SiTdQGEm5dPJCsNBNbrWRniu1HyYvMf5MBx+uvZ0gxXcsnX+RYYDjJaUeQ=
Message-ID: <be27de24-a6e8-4ea4-b835-9da365eac1b0@suse.com>
Date: Wed, 14 Feb 2024 14:11:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 19/30] xen/riscv: introduce event.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <a57aed8d2d8424c63b1f27d007b2d5cfe35460cc.1707146506.git.oleksii.kurochko@gmail.com>
 <36b25710-cf95-4250-bf4a-6ccf4f418909@suse.com>
 <39e07917c237a2d2e8879a66511edef4590d9fba.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <39e07917c237a2d2e8879a66511edef4590d9fba.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 13:16, Oleksii wrote:
> On Mon, 2024-02-12 at 16:20 +0100, Jan Beulich wrote:
>> On 05.02.2024 16:32, Oleksii Kurochko wrote:
>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>
>> Acked-by: Jan Beulich <jbeulich@suse.com>
>> again with a nit, though:
>>
>>> --- /dev/null
>>> +++ b/xen/arch/riscv/include/asm/event.h
>>> @@ -0,0 +1,40 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +#ifndef __ASM_RISCV_EVENT_H__
>>> +#define __ASM_RISCV_EVENT_H__
>>> +
>>> +#include <xen/lib.h>
>>> +
>>> +void vcpu_mark_events_pending(struct vcpu *v);
>>> +
>>> +static inline int vcpu_event_delivery_is_enabled(struct vcpu *v)
>>> +{
>>> +    BUG_ON("unimplemented");
>>> +    return 0;
>>> +}
>>> +
>>> +static inline int local_events_need_delivery(void)
>>> +{
>>> +    BUG_ON("unimplemented");
>>> +    return 0;
>>> +}
>>> +
>>> +static inline void local_event_delivery_enable(void)
>>> +{
>>> +    BUG_ON("unimplemented");
>>> +}
>>> +
>>> +/* No arch specific virq definition now. Default to global. */
>>> +static inline bool arch_virq_is_global(unsigned int virq)
>>> +{
>>> +    return true;
>>> +}
>>> +
>>> +#endif
>>
>> This want to gain the usual comment.
> Do you mean that commit messag should be updated?

No, I indeed mean "comment". Just go look what I committed yesterday.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:15:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:15:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680748.1058842 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raF6u-0000d4-SP; Wed, 14 Feb 2024 13:15:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680748.1058842; Wed, 14 Feb 2024 13:15:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raF6u-0000cx-Pq; Wed, 14 Feb 2024 13:15:36 +0000
Received: by outflank-mailman (input) for mailman id 680748;
 Wed, 14 Feb 2024 13:15:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raF6t-0000cX-8N
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:15:35 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 21ffd7b5-cb3b-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 14:15:33 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-33cddf4b4b5so1133841f8f.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:15:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 u14-20020a056000038e00b0033ce40eee9fsm2480791wrf.58.2024.02.14.05.15.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 05:15:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21ffd7b5-cb3b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707916532; x=1708521332; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=n6C0PY+0B/eU5cH4iEmzlZHG2r8SQ67nXHcdU7UpiIc=;
        b=Uo2ohDe9olc0H30zJXOR4eyjx/0TWyYNkDlkYdp5Njq2xYcMZMfqN0DJUc1eB0E414
         2rFUKRbomymgkx1DRkAy/14JzOUFO8EJ1NzZBgWabeXshdWmumLaqoSohH+0nQoOiAom
         RvfkSucXip8n2v2fNAiwIMtEd2qpC19W9NUv8CI7Szb9aBt60wSU0Kom/C3C8+ft46wm
         WXHuxa9/BtKp836qRwc1Qmo/5a9ACjBoaQJhOl+dlN9khXx6zlo4hKL6Zd9q5XV2zc7J
         7TI2BjC4m8gGVbQI+bo9ewWj/6bAGTVnZerMDmTadRFeJ1K5ZnXSsN7pA++MWg91nZ1Q
         P/tg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707916532; x=1708521332;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=n6C0PY+0B/eU5cH4iEmzlZHG2r8SQ67nXHcdU7UpiIc=;
        b=JS7amntXNn3OfSIDnrDUNxdOJJvwiHx7jo1QUEvDzJp3b+OFmodZpxdWVWQURYFGqq
         vTAcVY0/w8WjUsrY99IzYYcIcCdif5iuSA8UQqfT/3dcrCuQYtqnhLn3ItgUtobq+sLc
         bGXmflZs/DUFWjVbZvjg60E5wRjS+69wJQ+aiOQQVFWdc/t0jISumnjlxEMgz7x6mqQ/
         Tjd2dZLdjMsGg9dMMhBfm82gCgmelgybDZAVBxJBwzBQeQdVEI0vDV5Hrs5TgVNlTFj1
         2ytoFQpIRqviLvMRenXz9dxzNnmPjQIkpne/wmrRQA0B77l/VRrAiYRu2hklF7IqYDnh
         JzDQ==
X-Forwarded-Encrypted: i=1; AJvYcCVVsSAF2c0SP61nWU5ndLD6f49s9x5MaF3OLmOMeOI8yS+5gPqFPgMroDemDoSNpbw1YKrRpyaf/5QHzTz+cwNj0avNxYptj0EzPb+tK6M=
X-Gm-Message-State: AOJu0YzmplKyCiH2xR2yWx2UZc34t93bltS9BWIHo0jfmbjdbmcDlAGd
	zSsG+cGrCNW7BVofsicxP7KoRKGhsuOj6CiwlaTcNnzVji1hLJAiTsAJ3dlMIg==
X-Google-Smtp-Source: AGHT+IETcjpYO34DCIQMnCwd3dcohCnHjfiabDEyWguteJHFU/1oc9WLVjCVx15tzM05d31Q94hxBA==
X-Received: by 2002:a5d:51c7:0:b0:33b:7d9b:4592 with SMTP id n7-20020a5d51c7000000b0033b7d9b4592mr2017179wrv.5.1707916532491;
        Wed, 14 Feb 2024 05:15:32 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUnBSHL+PgeDPoIK1o7DgQ+w8jxB50essMLshAyXMIKxv84cLdLxu08JpZ4DfrOJOkvB5XeLDVlAbQoTkOg2NCHI7pw223r3112dHXS1Lo5q+zQEXR+AOKDiDw9S0uofvKNGCoh6tGB3vwsqtsbqoDLgjFJe1Gp3X5smkmHPrz/fW5VHK4UvRGx1cPh3UcIX3By0RsGlevLaR863pEhBveHUvHxUD3SnWpZPWG/tDC4IV265NIQCLsrmyKi185ih7X5ezzdekdbQj3+5n+72gcAGokX29jyEG1J8V2qxfdK7pg=
Message-ID: <2da73259-a86e-4a13-9fd4-4217372f4b76@suse.com>
Date: Wed, 14 Feb 2024 14:15:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/3] docs/misra/rules.rst: add rule 5.5
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, julien@xen.org,
 bertrand.marquis@arm.com, roger.pau@citrix.com,
 xen-devel@lists.xenproject.org,
 Stefano Stabellini <stefano.stabellini@amd.com>, roberto.bagnara@bugseng.com
References: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
 <20240213223334.3693410-2-stefano.stabellini@amd.com>
 <50719397-b053-43e1-9cf7-cc9eae9098ed@suse.com>
 <40b7465f-4966-43c7-8db3-e28a6cc48445@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <40b7465f-4966-43c7-8db3-e28a6cc48445@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.02.2024 12:27, Federico Serafini wrote:
> On 14/02/24 09:28, Jan Beulich wrote:
>> On 13.02.2024 23:33, Stefano Stabellini wrote:
>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
>>> ---
>>>   docs/misra/rules.rst | 6 ++++++
>>>   1 file changed, 6 insertions(+)
>>>
>>> diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
>>> index c185366966..931158b354 100644
>>> --- a/docs/misra/rules.rst
>>> +++ b/docs/misra/rules.rst
>>> @@ -181,6 +181,12 @@ maintainers if you want to suggest a change.
>>>          headers (xen/include/public/) are allowed to retain longer
>>>          identifiers for backward compatibility.
>>>   
>>> +   * - `Rule 5.5 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_05_05.c>`_
>>> +     - Required
>>> +     - Identifiers shall be distinct from macro names
>>> +     - Clashes between function-like macros and non-callable entities
>>> +       are allowed. The pattern #define x x is also allowed.
>>
>> Just for me to know what exactly is covered (hence also a question
>> to Roberto as to [to be] implemented Eclair behavior): Even when
>> the above would be sufficient (and imo better) people frequently
>> write
>>
>> #define a(x, y) b(x, y)
>>
>> which, transformed to the specific case here, would then be
>>
>> #define a(x, y) a(x, y)
>>
>> I'd assume such ought to also be covered, but that's not clear
>> from the spelling above.
> 
> I list what happens in some different situations,
> then we can find the right words for the documentation and/or
> refine the configuration:
> 
> If you
> #define x x
> and then use `x' as identifier,
> the resulting violation is deviated (allowed pattern).
> 
> If you
> #define a(x, y) a(x, y)
> and then use `a' as identifier for a non-callable entity,
> the resulting violation is deviated (no clash with non-callable
> entities).
> If you use identifier `a' for a callable entity, the resulting violation
> is reported: the allowed pattern covers only macros expanding to their
> own name, in this case the macro name is considered to be
> `a' only, not a(x, y).
> 
> If you
> #define a(x, y) b(x, y)
> and then use `a' as identifier for a non-callable entity,
> the resulting violation is deviated (no clash with non-callable
> entities).

I'm afraid I don't see what violation there is in this case, to
deviate. As a result I'm also not sure I correctly understand the
rest of your reply.

> If you use `a' as identifier for a callable entity,
> this is not a violation because after the preprocessing phase,
> identifier `a' no longer exists.
> 
> As far as I know, this is what was agreed upon in one of the recent
> MISRA meetings.

That was my fear (of being what I'd call a wrong [insufficient]
interpretation of what was meant).

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:22:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:22:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680755.1058852 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFDJ-0002wg-Jt; Wed, 14 Feb 2024 13:22:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680755.1058852; Wed, 14 Feb 2024 13:22:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFDJ-0002wZ-HO; Wed, 14 Feb 2024 13:22:13 +0000
Received: by outflank-mailman (input) for mailman id 680755;
 Wed, 14 Feb 2024 13:22:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raFDJ-0002wT-0p
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:22:13 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0f0ff012-cb3c-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 14:22:10 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-41102f140b4so15212975e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:22:10 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n9-20020a05600c294900b00410cc2f5550sm1912910wmd.19.2024.02.14.05.22.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 05:22:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f0ff012-cb3c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707916930; x=1708521730; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Za8dYxhmQv5z0d1hW7+w4ZKDD4wdXQ+pZGwuqrALwhE=;
        b=RFrYuHBNprrJusHQtb35S50UoHul1BdBFFq2asEf2+c93jYKOR3BYeHcR9SkhH8Ri1
         VRlrGh04Rf4dK1wKiofmbB9P+Jb+mbqtnHAvKj5SS5DUBKf5YO05IcXkpK1PGpglbeq4
         aWJgcmquP4l8lXMtHlGRuhhAgV9i+xxwRD/DP7PxYB420FxM3LHAMF45Ef2I3VvFuP8s
         wFbWdqeMoBiMqmMqzCeZAk5dxCLrxASuNozgva8/xIrllFHMwaWnjjXPqTyN/edMmXKT
         g90Ai85XGSzcw0nhDDSvbiLHvvndDBwqz07+dqVKCN+zBNNZM0+nWa/hJXzNdqfOdwib
         Quig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707916930; x=1708521730;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Za8dYxhmQv5z0d1hW7+w4ZKDD4wdXQ+pZGwuqrALwhE=;
        b=GYcaqjqSXwaD/1k4ErOceOi12plxFoqfo/QVzkG56SyrRcrGeNNdrVrWVO6Jq6EhcO
         8SOa1YcA+irhZCbZcx+m9qCdtJpD3WSR7N9U5pKdHePYnbsErdPwqQwk+jirvCUYO6bF
         sUEMkdT3gmRKRl64kdYcviTmw7JrdB+CB1G5ir+bpyOx1OabbsM0hIIipB8+qgZNw9aP
         W5Vpw2ZXu8Jcbi8HcPaqpFpsMMTTofz9/lIsvBsyNwtGHPMSrVMjVrI+ksuzcDoTaPIq
         MvO5/l9NtD1edkTW5Pl3xaCkfxbEXPTTTSRfIoxorvSefFiw1q2eP5H7GlIs4Me7OX+r
         WaoQ==
X-Forwarded-Encrypted: i=1; AJvYcCWE7bQLD+KeZcPFDEiIGY74kMoPYwwHV1PPE442sKvC4YSRaeJLh7PHeg2TWKxjtL2XXnBTFgU+amKd7XECNaEcsu+aX5mNUmrbgBPZkHE=
X-Gm-Message-State: AOJu0YzmrN1cJajYtft3l0mhP3nAoREuwlDMicxjrrwosEHdmCH0xfbg
	kurP6U5JJTu6ow8uMXQ0eblgRcAOCy6/1StN28O1p5BvbBbpVYpLCUXPWbyjCQ==
X-Google-Smtp-Source: AGHT+IEIEbq2klUpJohxXfOhdp9gJ2tr+ijQD1FJ9Iec9F35vHtTYaF8uBWf8ishWFtWSTYNEESZsw==
X-Received: by 2002:a05:600c:1da3:b0:411:c390:ea43 with SMTP id p35-20020a05600c1da300b00411c390ea43mr2020263wms.10.1707916930246;
        Wed, 14 Feb 2024 05:22:10 -0800 (PST)
Message-ID: <2416e41e-0b59-4e63-a9c2-550983b71c1c@suse.com>
Date: Wed, 14 Feb 2024 14:22:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/5] iommu/x86: print RMRR/IVMD ranges using full
 addresses
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Paul Durrant <paul@xen.org>, xen-devel@lists.xenproject.org
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-3-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214103741.16189-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 11:37, Roger Pau Monne wrote:
> It's easier to correlate with the physical memory map if the addresses are
> fully printed, instead of using frame numbers.
> 
> Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

In principle
Reviewed-by: Jan Beulich <jbeulich@suse.com>
I'm not sure though that this fully matches Andrew's intentions:

> --- a/xen/drivers/passthrough/x86/iommu.c
> +++ b/xen/drivers/passthrough/x86/iommu.c
> @@ -801,8 +801,8 @@ bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
>          return true;
>  
>      printk(XENLOG_WARNING
> -           "%s: [%#" PRI_mfn " ,%#" PRI_mfn "] is not (entirely) in reserved memory\n",
> -           prefix, mfn_x(start), mfn_x(end));
> +           "%s: [%#lx, %#lx] is not (entirely) in reserved memory\n",
> +           prefix, mfn_to_maddr(start), mfn_to_maddr(end));

e820.c uses [%016Lx, %016Lx] instead, i.e. full 16-digit numbers. For
easiest cross matching it may be desirable to do the same here. Then
of course the 0x prefixes may also better disappear.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:29:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:29:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680761.1058863 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFKV-0003kp-Bg; Wed, 14 Feb 2024 13:29:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680761.1058863; Wed, 14 Feb 2024 13:29:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFKV-0003ki-7g; Wed, 14 Feb 2024 13:29:39 +0000
Received: by outflank-mailman (input) for mailman id 680761;
 Wed, 14 Feb 2024 13:29:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raFKU-0003jM-9M
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:29:38 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1917a68d-cb3d-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 14:29:37 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-411f42986e1so3367605e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:29:37 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h16-20020a056000001000b0033cdedb3e84sm3964907wrx.18.2024.02.14.05.29.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 05:29:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1917a68d-cb3d-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707917376; x=1708522176; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rPbjGDf8TwEUCNn9c+jGy03coUPmzqiLsxJcqNOBLPE=;
        b=cO0gHFnJbO38dUcHGjrsHqy953SgC4JmpPeR1J44HtL+wX2DL6SO+1ZAHu2blnFhjN
         he59y/ES86KorzAZM5mzh8Fr2apq7Um0+WOQR/mXtrqFvTe9p1p8jM+Xap4oqUlWwm/9
         /DY5y6gSwV5B6ExsaZLGmrAPEI3V4IHrzh4EZVJKlA7+rLmPvTpi0MxV8cqG4LrdOBUg
         5q9Lnp6Nx91dMXFRI+YI28CibAzf0X+yg7vF55k+i0gDhFsczxrZPEjrTt6ZadiykB9e
         zhO1jyHQS6CVcCmewJ9OPUXKYpDz71ONwDzVtb24ia2qqFd4X0DjqYBdk9KJRPedrv95
         p60g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707917376; x=1708522176;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rPbjGDf8TwEUCNn9c+jGy03coUPmzqiLsxJcqNOBLPE=;
        b=iV40rX/G2rclRZWySjc6Ne0Trwuygau0jZb7eyNQ6cgJqU08pebRf64YVia5JQvZMh
         DG0sQkDmEul1ifYaYOj1bhMi0Z0At6VH2SuUZKmvSIlJEffoy3hJnzez/3woJF9TzkUJ
         9gdG9nrsqM5E2s8vtRoQ+JA0zxnxk22AUBaRDqfURCrq4A7DPBLF5tit5LYw2884Nl0N
         AWnK3gTZMS4Xhiqt110QAJvQrXlE3KkB5Ya0cdoq401r4AdrLJI4pq4gU8CUksb5pp0I
         N8a69otqQ3NLqHSvnscln9jHuTAdDU70xI9cGcitmWsXfnpjhK7kSt8+UmN+eNFW8IZg
         SrqA==
X-Forwarded-Encrypted: i=1; AJvYcCXGXSMLoNDxDanzHma+FjeVpNK6XeFJSo9T7kJ/IPvc8t837IaFMRy8q7cyAAr4gdIRgm953qXYBxUPVsMjx1wIJKCWRdg9qfxHNntFKns=
X-Gm-Message-State: AOJu0YyKXGygmKfOHuIhY8mXMR0QXE8MpnWaM6bfOp8BQ61Hbkp6NcEy
	lsAxtuyGF27UoS4mhptzTblSuBHMEOXXN5AvVeiFxEIPlCT6f2BuaKRjKzEpaMqei0uZTx96fSo
	=
X-Google-Smtp-Source: AGHT+IGAsy+86rwubyJeg5mFDTN0xo4pqz5SrPlCddDPRfhGWZmUSg9+3Et6lxyUZcAW5bBN6Z6LxQ==
X-Received: by 2002:a5d:5143:0:b0:33c:d5a7:4fc with SMTP id u3-20020a5d5143000000b0033cd5a704fcmr1824072wrt.16.1707917376565;
        Wed, 14 Feb 2024 05:29:36 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXE6OYxzKnF0xybTpGLJlSQkqfp2K+HA1hLK2pP9CIzpw5Xt354hg83W3CtRriNp6Yr4sQ1tozj7z+PW6tTr1vB/CK/kzWcL5orO32b84EbTP26RTW3rfyf48G40wN6nh1Z1golSpScUlQ=
Message-ID: <16de2e3e-7807-4754-852b-e486c1fde95c@suse.com>
Date: Wed, 14 Feb 2024 14:29:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/5] iommu/x86: use full addresses internally for the
 IVMD/RMRR range checks
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Paul Durrant <paul@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 xen-devel@lists.xenproject.org
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-4-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214103741.16189-4-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 11:37, Roger Pau Monne wrote:
> Adjust the code in the checker to use full addresses rather than frame numbers,
> as it's only page_get_ram_type() that requires an mfn parameter.
> 
> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

In this very shape I'd like to leave this to Paul or Andrew: I'm not outright
opposed, but I think this either goes too far (most type-safety being lost
again), or not far enough (both callers convert addresses to MFNs, just for
them to be converted back here).

Jan

> --- a/xen/drivers/passthrough/x86/iommu.c
> +++ b/xen/drivers/passthrough/x86/iommu.c
> @@ -794,28 +794,26 @@ __initcall(adjust_irq_affinities);
>  
>  bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
>  {
> -    mfn_t addr;
> +    paddr_t s = mfn_to_maddr(start), e = mfn_to_maddr(end);
>  
> -    if ( e820_all_mapped(mfn_to_maddr(start), mfn_to_maddr(end) + PAGE_SIZE,
> -                         E820_RESERVED) )
> +    if ( e820_all_mapped(s, e + PAGE_SIZE, E820_RESERVED) )
>          return true;
>  
>      printk(XENLOG_WARNING
>             "%s: [%#lx, %#lx] is not (entirely) in reserved memory\n",
> -           prefix, mfn_to_maddr(start), mfn_to_maddr(end));
> +           prefix, s, e);
>  
> -    for ( addr = start; mfn_x(addr) <= mfn_x(end); addr = mfn_add(addr, 1) )
> +    for ( paddr_t addr = s; addr <= e; addr += PAGE_SIZE )
>      {
> -        unsigned int type = page_get_ram_type(addr);
> +        unsigned int type = page_get_ram_type(maddr_to_mfn(addr));
>  
>          if ( type == RAM_TYPE_UNKNOWN )
>          {
> -            if ( e820_add_range(mfn_to_maddr(addr),
> -                                mfn_to_maddr(addr) + PAGE_SIZE, E820_RESERVED) )
> +            if ( e820_add_range(addr, addr + PAGE_SIZE, E820_RESERVED) )
>                  continue;
>              printk(XENLOG_ERR
> -                   "%s: page at %#" PRI_mfn " couldn't be reserved\n",
> -                   prefix, mfn_x(addr));
> +                   "%s: page at %#lx couldn't be reserved\n",
> +                   prefix, paddr_to_pfn(addr));
>              return false;
>          }
>  
> @@ -829,9 +827,8 @@ bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
>                       RAM_TYPE_UNUSABLE) )
>              continue;
>  
> -        printk(XENLOG_ERR
> -               "%s: page at %#" PRI_mfn " can't be converted\n",
> -               prefix, mfn_x(addr));
> +        printk(XENLOG_ERR "%s: page at %#lx can't be converted\n",
> +               prefix, paddr_to_pfn(addr));
>          return false;
>      }
>  



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:30:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:30:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680765.1058873 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFLL-0005B6-JG; Wed, 14 Feb 2024 13:30:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680765.1058873; Wed, 14 Feb 2024 13:30:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFLL-0005Az-GR; Wed, 14 Feb 2024 13:30:31 +0000
Received: by outflank-mailman (input) for mailman id 680765;
 Wed, 14 Feb 2024 13:30:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mIjL=JX=cloud.com=christian.lindig@srs-se1.protection.inumbo.net>)
 id 1raFLK-00058m-Ka
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:30:30 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 38a16fe8-cb3d-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 14:30:30 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a26ed1e05c7so305539866b.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:30:30 -0800 (PST)
Received: from smtpclient.apple ([160.101.139.1])
 by smtp.gmail.com with ESMTPSA id
 ca23-20020a170906a3d700b00a3caccb8f66sm2320867ejb.44.2024.02.14.05.30.29
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Wed, 14 Feb 2024 05:30:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 38a16fe8-cb3d-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707917429; x=1708522229; darn=lists.xenproject.org;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Uw6dNung5bnivI0pZ1zZjo9g75/H0AXcN8gPbhDiXss=;
        b=KSo68wxhpFY6jjzwDIyT6ozfSAkuwqh8KzRJnNRX7Je61RD5NJobTWhhuidZUaXLpa
         BiaB8J6H7yYTkyz3ravt4HpJvrghY6NbuQND9W6wEQkEW+NuOrPD+wpjHUnMI1SljsfZ
         y64Vmglew8euNCMgAoE0UMXkiUd1HA6Jx9o2g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707917429; x=1708522229;
        h=to:references:message-id:content-transfer-encoding:cc:date
         :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Uw6dNung5bnivI0pZ1zZjo9g75/H0AXcN8gPbhDiXss=;
        b=d345bHiE4DMl8x3JphXKfU5ZrcibwAPA/yzX334OJd6y+z0ikzPCYOnJfRA04D56zB
         aDVa2LdgDKDR3yLtb4594N2VUYnZAe/RKiYlJLchiB5/zwJLba38jg4ZLZPKivC9TjHi
         Xnnw+5lJIFucsi8gjW1qs2LEXMeAERfJXFgFaH1pt6W+BxK4+zqfdul19schV3uYwUUz
         RnAydoTp8gJ6CrwY41aWcelbZOQjEjs84f6FMUKTnVEBaMtuK3r9JQFXSnL0Cw9189RH
         lxOl0iNIY6fkSHL+9aMhT1Fw0kL4DtbLXrbd3ESiGEuo797pfJdEHD9uLjwRpHXExBfy
         usmQ==
X-Forwarded-Encrypted: i=1; AJvYcCXW7cXsD0CJcTnBEkmpV+ma6HC8ZgjIL/jprZGiiI/JKR1G8rp/jqZpUSlRhkhRWoxZO2QzXUmIxRL7nmIo1uHv9PjViLtqv0qLpizi2GU=
X-Gm-Message-State: AOJu0YyS809yo+3bfC/Flb3pFKbRQGQ6xej8CHlB8Tzj6m31Uo8NnzzP
	vvCGGg7fMznVRgO/0+xo0p0/lcgkuRID2wZJ7lqh/9b8bcjvCJj634hVoEk6O4o=
X-Google-Smtp-Source: AGHT+IGxz8DX0Ag72bUYpwxMmZMfMVxz43iA5Rv2wIQbj18PV9FEV9Y0y5opL0oobHowAIW3877dkA==
X-Received: by 2002:a17:906:fa94:b0:a3c:d7a5:6ab1 with SMTP id lt20-20020a170906fa9400b00a3cd7a56ab1mr2002258ejb.0.1707917429565;
        Wed, 14 Feb 2024 05:30:29 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWsjtZTSMqkhrsgb+rgBtIdXw8ZqLqhmTSLZzqNeyaea4TG2jy2ZoBnWcf0TINsd7Pbc7eK5OKfhFYjLDwCNh/udgooIPVdHROw5myT6uurom4RxDxKgChm446IWt403keBZUwCe6tTCWpLAHMVpDCmP9Qjk0uXCQsZP0DwfCU2pxLJCA5wp/chtLfFRU/yWpFdzB33eqXKJtk=
Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\))
Subject: Re: [PATCH] tools/ocaml: Add missing vmtrace_buf_kb field
From: Christian Lindig <christian.lindig@cloud.com>
In-Reply-To: <046415d5-d4f4-4870-a588-60637ea3bc3a@citrix.com>
Date: Wed, 14 Feb 2024 13:30:18 +0000
Cc: Jan Beulich <jbeulich@suse.com>,
 =?utf-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>,
 Xen-devel <xen-devel@lists.xenproject.org>,
 David Scott <dave@recoil.org>,
 Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <217314CC-5717-4253-9DEC-61D5DA1E52AC@cloud.com>
References: <9b9909c9e93cb540b3488c784935acc2bc9e071e.1707343396.git.w1benny@gmail.com>
 <3A858D7F-C953-4EF0-8919-AE96D6105AB1@cloud.com>
 <01b0d902-1903-4618-ad43-f625e57b61e1@suse.com>
 <046415d5-d4f4-4870-a588-60637ea3bc3a@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
X-Mailer: Apple Mail (2.3774.200.91.1.1)



> On 14 Feb 2024, at 11:45, Andrew Cooper <andrew.cooper3@citrix.com> =
wrote:
>=20
> Xapi is the only consumer of this interface.  I've fixed up the build
> against staging, but we're not going to be running KFX under Xapi any
> time soon.
>=20
> Ultimately it's Christian's call.


After a discussion with Andrew, we will not back port for now as we =
expect Xapi to use a new version that has it when it becomes relevant.

=E2=80=94 C=


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:30:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:30:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680767.1058882 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFLa-0005Wj-QN; Wed, 14 Feb 2024 13:30:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680767.1058882; Wed, 14 Feb 2024 13:30:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFLa-0005Wa-Nf; Wed, 14 Feb 2024 13:30:46 +0000
Received: by outflank-mailman (input) for mailman id 680767;
 Wed, 14 Feb 2024 13:30:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raFLZ-0005Ti-5P
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:30:45 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 40ac8e04-cb3d-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 14:30:43 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-411d2836b20so9707015e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:30:43 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h16-20020a056000001000b0033cdedb3e84sm3964907wrx.18.2024.02.14.05.30.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 05:30:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 40ac8e04-cb3d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707917443; x=1708522243; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6Jnwz2PcsLE2CA+AOjY9sKso1knmUH7H5GPWbGxMJW0=;
        b=IZVHLb5fc8tTzv7xQ/4ljwvMq2reyLEZYScWi4EpjBUkqtQp+VJcmlSmRq09etvbqV
         QWh6pXCBCFVzpLleUPmN85DvPdQAHGX7HpdfWrw5ZgcERtG0AwCvyV7i9o3ozqssz1y/
         4FZxS8iSisiTpcy3YasX8A4eO+U3ffl3b2QMVs3IdjephBr+LmVAFkXco/EXScwcTKxR
         iODciQ/uutPWFsj7E0L2MaRAaGqGGoNXXzVhwAl+HMNlkVoJU3qLLAAMFWK/CMefGR9P
         gwlTslts8lTeZ7/TWwGgtnNTKE5/Vr2VuavMcpNDA1BTMhKWtJwkcfMzM52oYiB2B9N/
         dqOg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707917443; x=1708522243;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6Jnwz2PcsLE2CA+AOjY9sKso1knmUH7H5GPWbGxMJW0=;
        b=GgMAf/oqRqFNt/f5ja/DcCf9f5n/SD/X6KGuu1+1X8RXZpKX7ojxhkTcMQ1O93JBBT
         /hxQ3adfakn0+Xumr3I3b+JOXyTYSvcvgIUfDOoQLyXZQswjpAPiyONdy+mpMcsnBBoJ
         RxUMkmXGdRbvXoXGlaem21EB0/y+9K7dmmWz0L9p04ad78LFsbmPiIPDDbm96NOnIBbx
         Di7Cj6o+5jBk1jMOhvrcjlGlFdQV9axg6GDPmieZmGf7sn8otWH2BmYYwKoTewL5yI8S
         UVwHm42uru4Jxziv8fk/eURV+US3Y6LHgCNnOF4R1savMkdNIpMEy2reTOF4BOh8JfZm
         Ji2Q==
X-Forwarded-Encrypted: i=1; AJvYcCUGGfL0Vx781LouIb8WYWhUIard4fcdoxSvcBAGuKptfGkqX5Ies6ytxIQnmO9knETw+lX/CSaAgEIlDg+T0XsncUmwmfcjRioCnwXeyVo=
X-Gm-Message-State: AOJu0Yx/geq9DLgNdtI7S1IjqGJFWj3m7NaFtKIEsVZGd1GIRj5bY5wU
	JUOtaCtCRcZBrbDutkBvTCFS8O3lXYL/63u+bJSilKuqY7U7NuOjYsbxp6pFkA==
X-Google-Smtp-Source: AGHT+IETjPPahs5Lk0gw+/zX+JTKxLg6vZib0nEN5qtQG7aWtiQOIY2IgTqwjZjher/6Aj4Oze4nfA==
X-Received: by 2002:a05:600c:190c:b0:410:72e0:d391 with SMTP id j12-20020a05600c190c00b0041072e0d391mr1936886wmq.6.1707917443002;
        Wed, 14 Feb 2024 05:30:43 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVEbwcv2Itggz1byTFXoqmFvryxBwoK0qGcM4+DV+ofhGpbIC2FMiVA/aG2xicto4KgD8HnqgMsscM/6AWpi2Gw2Ajv70DDku874btkgyqPOmZjSRtO1wD+sn4Le5fPRiVoX3sXAaM7FHY=
Message-ID: <d62df112-6cb8-46aa-ad50-d7e3da14e43b@suse.com>
Date: Wed, 14 Feb 2024 14:30:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/5] iommu/x86: print page type in IVMD/RMRR check in case
 of error
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Paul Durrant <paul@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 xen-devel@lists.xenproject.org
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-5-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214103741.16189-5-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 11:37, Roger Pau Monne wrote:
> Provide more information in case the page can't be converted, and print the
> original type(s).
> 
> Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:42:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:42:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680780.1058893 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFXH-0000AD-Rj; Wed, 14 Feb 2024 13:42:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680780.1058893; Wed, 14 Feb 2024 13:42:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFXH-0000A0-Ox; Wed, 14 Feb 2024 13:42:51 +0000
Received: by outflank-mailman (input) for mailman id 680780;
 Wed, 14 Feb 2024 13:42:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raFXG-00009s-8w
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:42:50 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f0cc0df4-cb3e-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 14:42:48 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-33ce2d1ab86so627057f8f.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:42:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 x4-20020a5d6504000000b0033cda80f54dsm4352308wru.83.2024.02.14.05.42.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 05:42:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0cc0df4-cb3e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707918168; x=1708522968; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=m5g4/QT6by55SLstFmMALxYwyJ8bfsQFdcdXYiqLh8I=;
        b=aPvo/f2qaaAB8X6zGY7cL0IL0nN2MOLm0xez2yDOzCUHTdrp4hQJIIRBOZ7DlVlTU4
         oZ3tdTgBL3CTyULVViNUI/KKqrKubdOLxXlUtU1eNHRUrE5Xh+YSaMWMXTdykFanfTo7
         L7vFxRgph6PDGfclo+5STrCId0Vbzf0hfRsqG04euwskvvqK18LD3kBnJ3cHOjot23NQ
         b0xhK0NsHldefGD8q3YCIQVFxzlhtAz2Wlq2plMssyY23VnulHYHtdL84L6+YyzBX/Nt
         pD4GiXFAWZPrkvEMpU7cG6Ye6xsdI6PdijjHGoKsK1rMFY7dqDL6EX9X4xYUXp7pDYoh
         zmYQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707918168; x=1708522968;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=m5g4/QT6by55SLstFmMALxYwyJ8bfsQFdcdXYiqLh8I=;
        b=Smhezx+vUREP5+yHUMhoV6sur27cmy6ZL7A+QUkI+1bZpw+pr6vHyko4h1Nv8+CSXo
         dvbTqFeT9d2WCanja01BGlZVoeUVnzvhAoq85Mbtob7E7IlGO0NJbzJLAqdr30X8AfHH
         Z0ovlDkk3TXPaFogwJMmIkMH4QtwLApeararPVY1rLbQ2JjO3G/ogTSNyD0KGwi2Dhwy
         a/KddK/rhqhm1IUhyKyio9G9WT0DDYzgqs6R9N89bbZmw3efLK6hZZ5Lmr8JMLwDNEeq
         /I+wecna3bLHgp8Pb5pgbajhqR1ltAGVwcopfZFxGCts/i0IJqg5KpCca6LfJDPSu0IV
         q0kQ==
X-Forwarded-Encrypted: i=1; AJvYcCWK9jjarSdaZv+Whh3wwYlVOBwQzH9W4bs5W96M3Q0XNsdt982iDoe6QPwErN5jvi5b+aDIvY7flNmcrIfo8FGUJywpyszzHp4zLnzLkNM=
X-Gm-Message-State: AOJu0Yy6JNC/KxcxK6b6mKkNRDRLJw3IuD6C02MPRvszVLD4xpkvuA2A
	TgY203d2X8vsfqeWov5XF+JH2gM9P6VvtwA3TPtY7DpQGmRXW1SnJ+Ner5vNxg==
X-Google-Smtp-Source: AGHT+IFJNLmYlbCvCqgwmaXxTLYz2vinYknPHEKtZ5v+4/O7ksSn2YGbL7CH8iZI/AKeu91mpPsxCg==
X-Received: by 2002:a5d:60c7:0:b0:33a:edaf:13ec with SMTP id x7-20020a5d60c7000000b0033aedaf13ecmr1681734wrt.14.1707918167947;
        Wed, 14 Feb 2024 05:42:47 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUwwxBeq/YFk/SLTfoX7WP/4MOc5gzA9qiWaUMNmPQZCMrr2+qrVNXEe+btaSLu1jPTWIDnsqKm7JtJcfgFCjqNVC/tqIQVMeoX0lzR92586/WWzSTT2l+7eqlAjHDp1FZhI3FEpyo86jkdLg0Z2aYSwIr8GQwLQbhDlqDwuoLdSOjWNf27/vl49ttb2g8efbH7M60ZSbiVO9iw2DY=
Message-ID: <ff45456b-87bf-4e6c-b2c9-6f3d5258d478@suse.com>
Date: Wed, 14 Feb 2024 14:42:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/5] mm: add the __must_check attribute to {gfn,mfn}_add()
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-6-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214103741.16189-6-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.02.2024 11:37, Roger Pau Monne wrote:
> It's not obvious from the function itself whether the incremented value will be

s/the function itself/just the function name/ ?

> stored in the parameter, or returned to the caller.  That has leads to bugs in
> the past as callers assume the incremented value is stored in the parameter.

... callers may assume ... ?

> --- a/xen/include/xen/mm-frame.h
> +++ b/xen/include/xen/mm-frame.h
> @@ -23,7 +23,7 @@ TYPE_SAFE(unsigned long, mfn);
>  #undef mfn_x
>  #endif
>  
> -static inline mfn_t mfn_add(mfn_t mfn, unsigned long i)
> +static inline mfn_t __must_check mfn_add(mfn_t mfn, unsigned long i)
>  {
>      return _mfn(mfn_x(mfn) + i);
>  }
> @@ -62,7 +62,7 @@ TYPE_SAFE(unsigned long, gfn);
>  #undef gfn_x
>  #endif
>  
> -static inline gfn_t gfn_add(gfn_t gfn, unsigned long i)
> +static inline gfn_t __must_check gfn_add(gfn_t gfn, unsigned long i)
>  {
>      return _gfn(gfn_x(gfn) + i);
>  }

What about dfn_add() (in xen/iommu.h)?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:52:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:52:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680786.1058902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFgj-0002Q1-S1; Wed, 14 Feb 2024 13:52:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680786.1058902; Wed, 14 Feb 2024 13:52:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFgj-0002Pu-PH; Wed, 14 Feb 2024 13:52:37 +0000
Received: by outflank-mailman (input) for mailman id 680786;
 Wed, 14 Feb 2024 13:52:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=U0HE=JX=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1raFgi-0002Po-M8
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:52:36 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4dce86a2-cb40-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 14:52:34 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-55a035669d5so7997520a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:52:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4dce86a2-cb40-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1707918753; x=1708523553; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=1NaW3+T/ol5E+Y89aSouGRO7U38Mk+SYTtVoYAoju/Q=;
        b=wblwJ0wKiGY0MZsjq5lVQJmgvqRelOU9/oQDrpBUjceVLOG2fSbZl4DJs5K21mth72
         RhQnJMFXVF06sdjHZF4q8z29FCR9+dY2ry4zuwqyrxAbWK9jpakS4+XRzflaTDiLgJfa
         R9Txg+gdfQzqUvcy1I/EiS8XJthgqp8n+pcnNrQGLB/FeuTJbLI7wvEP1zeAgdFQ+ThN
         r9Y3pdew4bVwa3ljRW9z1lphM2S/ggbHdMbnqfhOK1ia29GT0majahgs9SbOSZC0T7s7
         CvU0lSWohqiFMe8DCM5S/XE6b1YkKH9zAVVgHJ0fD6JE0/egPHtMuFQhAmkMxCfvs0pV
         H7Bw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707918753; x=1708523553;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=1NaW3+T/ol5E+Y89aSouGRO7U38Mk+SYTtVoYAoju/Q=;
        b=iGHoB2tK0GnAvzBCYWbEoO8+p2u/FfdnIiJ7pi57OG6Tr/IagRVTRq6gnVyKuKx3ty
         VxeB6nZAViqpuWqfbqGiyPzxCWbIYim1JFk6QAtc/4MHi/9lVbtrKHllWCVl3rV5rXNO
         mb5lqL2FbTaZ/1WkBgCEMnxbvLFftY9fN5+wBYUrvKbIwPW1WTXZ3OFEZCJAWJufj6GW
         iw0HdZs4gx1ESpLnGCKOILL1WkObVffU6oV16ywFkwi1dKzycJF2nC1saYeR6aoHLiZv
         DBw5zBJ3qc6wGIq7FBBd0mv3kfn0nLMr6qI4BCF0UN2oTXX/a9Yj1wi9gm/+OdEnxMPb
         IVEg==
X-Gm-Message-State: AOJu0Yy3uUY0N+Xxs5IZBL2pXHJc3cJ6o+2BjXpPlrsYKlRekdVMrlFq
	li/bIncjiDL45HWLmLKcrgc1/chVcI39dInDAeqRCy+729oPE7xWcu8GtydAsVcmHCQfPFiRhGr
	ZZyo/wYXNYhOdD7ic7If7OC1PZxuPAilM5fsGLg==
X-Google-Smtp-Source: AGHT+IGjh/jJ7IWOkhA0kpBufUqMaFMNEUojI7L46nCdB1H3t7WN2gPAV59OPAr9WCcuBnmB0VBzDRQwHnax6+SCM5w=
X-Received: by 2002:aa7:d7cb:0:b0:562:1441:81f0 with SMTP id
 e11-20020aa7d7cb000000b00562144181f0mr1941792eds.7.1707918753324; Wed, 14 Feb
 2024 05:52:33 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-3-carlo.nonato@minervasys.tech> <bb28c8a2-c2bd-4e1d-ad28-9bc632e3e474@amd.com>
In-Reply-To: <bb28c8a2-c2bd-4e1d-ad28-9bc632e3e474@amd.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Wed, 14 Feb 2024 14:52:22 +0100
Message-ID: <CAG+AhRXsZC7Pjdce42yYTvBy7MyjS1axrDGsY0U=o+APNyxdYA@mail.gmail.com>
Subject: Re: [PATCH v6 02/15] xen/arm: add initial support for LLC coloring on arm64
To: Michal Orzel <michal.orzel@amd.com>
Cc: xen-devel@lists.xenproject.org, andrea.bastoni@minervasys.tech, 
	Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, 
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
	Marco Solieri <marco.solieri@minervasys.tech>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Michal,

On Wed, Feb 14, 2024 at 11:14=E2=80=AFAM Michal Orzel <michal.orzel@amd.com=
> wrote:
>
> Hi Carlo,
>
> On 29/01/2024 18:17, Carlo Nonato wrote:
> >
> >
> > LLC coloring needs to know the last level cache layout in order to make=
 the
> > best use of it. This can be probed by inspecting the CLIDR_EL1 register=
,
> > so the Last Level is defined as the last level visible by this register=
.
> > Note that this excludes system caches in some platforms.
> >
> > Static memory allocation and cache coloring are incompatible because st=
atic
> > memory can't be guaranteed to use only colors assigned to the domain.
> > Panic during DomUs creation when both are enabled.
> >
> > Based on original work from: Luca Miccio <lucmiccio@gmail.com>
> >
> > Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
> > Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
> > ---
> > v6:
> > - get_llc_way_size() now checks for at least separate I/D caches
> > v5:
> > - used - instead of _ for filenames
> > - moved static-mem check in this patch
> > - moved dom0 colors parsing in next patch
> > - moved color allocation and configuration in next patch
> > - moved check_colors() in next patch
> > - colors are now printed in short form
> > v4:
> > - added "llc-coloring" cmdline option for the boot-time switch
> > - dom0 colors are now checked during domain init as for any other domai=
n
> > - fixed processor.h masks bit width
> > - check for overflow in parse_color_config()
> > - check_colors() now checks also that colors are sorted and unique
> > ---
> >  docs/misc/cache-coloring.rst         | 11 ++++
> >  xen/arch/arm/Kconfig                 |  1 +
> >  xen/arch/arm/Makefile                |  1 +
> >  xen/arch/arm/dom0less-build.c        |  6 +++
> >  xen/arch/arm/include/asm/processor.h | 16 ++++++
> >  xen/arch/arm/llc-coloring.c          | 75 ++++++++++++++++++++++++++++
> >  xen/arch/arm/setup.c                 |  3 ++
> >  7 files changed, 113 insertions(+)
> >  create mode 100644 xen/arch/arm/llc-coloring.c
> >
> > diff --git a/docs/misc/cache-coloring.rst b/docs/misc/cache-coloring.rs=
t
> > index 9fe01e99e1..0535b5c656 100644
> > --- a/docs/misc/cache-coloring.rst
> > +++ b/docs/misc/cache-coloring.rst
> > @@ -85,3 +85,14 @@ More specific documentation is available at `docs/mi=
sc/xen-command-line.pandoc`.
> >  +----------------------+-------------------------------+
> >  | ``llc-way-size``     | set the LLC way size          |
> >  +----------------------+-------------------------------+
> > +
> > +Known issues and limitations
> > +****************************
> > +
> > +"xen,static-mem" isn't supported when coloring is enabled
> > +#########################################################
> > +
> > +In the domain configuration, "xen,static-mem" allows memory to be stat=
ically
> > +allocated to the domain. This isn't possible when LLC coloring is enab=
led,
> > +because that memory can't be guaranteed to use only colors assigned to=
 the
> > +domain.
> > diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> > index 50e9bfae1a..55143f86a9 100644
> > --- a/xen/arch/arm/Kconfig
> > +++ b/xen/arch/arm/Kconfig
> > @@ -8,6 +8,7 @@ config ARM_64
> >         depends on !ARM_32
> >         select 64BIT
> >         select HAS_FAST_MULTIPLY
> > +       select HAS_LLC_COLORING
> >
> >  config ARM
> >         def_bool y
> > diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> > index 33c677672f..c9a1cd298d 100644
> > --- a/xen/arch/arm/Makefile
> > +++ b/xen/arch/arm/Makefile
> > @@ -37,6 +37,7 @@ obj-$(CONFIG_IOREQ_SERVER) +=3D ioreq.o
> >  obj-y +=3D irq.o
> >  obj-y +=3D kernel.init.o
> >  obj-$(CONFIG_LIVEPATCH) +=3D livepatch.o
> > +obj-$(CONFIG_LLC_COLORING) +=3D llc-coloring.o
> >  obj-y +=3D mem_access.o
> >  obj-y +=3D mm.o
> >  obj-y +=3D monitor.o
> > diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-buil=
d.c
> > index fb63ec6fd1..1142f7f74a 100644
> > --- a/xen/arch/arm/dom0less-build.c
> > +++ b/xen/arch/arm/dom0less-build.c
> > @@ -5,6 +5,7 @@
> >  #include <xen/grant_table.h>
> >  #include <xen/iocap.h>
> >  #include <xen/libfdt/libfdt.h>
> > +#include <xen/llc-coloring.h>
> >  #include <xen/sched.h>
> >  #include <xen/serial.h>
> >  #include <xen/sizes.h>
> > @@ -879,7 +880,12 @@ void __init create_domUs(void)
> >              panic("No more domain IDs available\n");
> >
> >          if ( dt_find_property(node, "xen,static-mem", NULL) )
> > +        {
> > +            if ( llc_coloring_enabled )
> > +                panic("LLC coloring and static memory are incompatible=
\n");
> > +
> >              flags |=3D CDF_staticmem;
> > +        }
> >
> >          if ( dt_property_read_bool(node, "direct-map") )
> >          {
> > diff --git a/xen/arch/arm/include/asm/processor.h b/xen/arch/arm/includ=
e/asm/processor.h
> > index 8e02410465..336933ee62 100644
> > --- a/xen/arch/arm/include/asm/processor.h
> > +++ b/xen/arch/arm/include/asm/processor.h
> > @@ -18,6 +18,22 @@
> >  #define CTR_IDC_SHIFT       28
> >  #define CTR_DIC_SHIFT       29
> >
> > +/* CCSIDR Current Cache Size ID Register */
> > +#define CCSIDR_LINESIZE_MASK            _AC(0x7, ULL)
> Why ULL and not UL? ccsidr is of register_t type

Julien, while reviewing an earlier version:

> Please use ULL here otherwise someone using MASK << SHIFT will have the
> expected result.

https://patchew.org/Xen/20220826125111.152261-1-carlo.nonato@minervasys.tec=
h/20220826125111.152261-2-carlo.nonato@minervasys.tech/#08956082-c194-8bae-=
cb25-44e4e3227689@xen.org

>
> > +#define CCSIDR_NUMSETS_SHIFT            13
> > +#define CCSIDR_NUMSETS_MASK             _AC(0x3fff, ULL)
> > +#define CCSIDR_NUMSETS_SHIFT_FEAT_CCIDX 32
> > +#define CCSIDR_NUMSETS_MASK_FEAT_CCIDX  _AC(0xffffff, ULL)
> > +
> > +/* CSSELR Cache Size Selection Register */
> > +#define CSSELR_LEVEL_MASK  _AC(0x7, UL)
> > +#define CSSELR_LEVEL_SHIFT 1
> > +
> > +/* CLIDR Cache Level ID Register */
> > +#define CLIDR_CTYPEn_SHIFT(n) (3 * (n - 1))
> n should be within parentheses
>
> > +#define CLIDR_CTYPEn_MASK     _AC(0x7, UL)
> > +#define CLIDR_CTYPEn_LEVELS   7
> > +
> >  #define ICACHE_POLICY_VPIPT  0
> >  #define ICACHE_POLICY_AIVIVT 1
> >  #define ICACHE_POLICY_VIPT   2
> > diff --git a/xen/arch/arm/llc-coloring.c b/xen/arch/arm/llc-coloring.c
> > new file mode 100644
> > index 0000000000..eee1e80e2d
> > --- /dev/null
> > +++ b/xen/arch/arm/llc-coloring.c
> > @@ -0,0 +1,75 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Last Level Cache (LLC) coloring support for ARM
> > + *
> > + * Copyright (C) 2022 Xilinx Inc.
> > + */
> > +#include <xen/llc-coloring.h>
> > +#include <xen/types.h>
> > +
> > +#include <asm/processor.h>
> > +#include <asm/sysregs.h>
> > +
> > +/* Return the LLC way size by probing the hardware */
> > +unsigned int __init get_llc_way_size(void)
> > +{
> > +    register_t ccsidr_el1;
> > +    register_t clidr_el1 =3D READ_SYSREG(CLIDR_EL1);
> > +    register_t csselr_el1 =3D READ_SYSREG(CSSELR_EL1);
> > +    register_t id_aa64mmfr2_el1 =3D READ_SYSREG(ID_AA64MMFR2_EL1);
> > +    uint32_t ccsidr_numsets_shift =3D CCSIDR_NUMSETS_SHIFT;
> > +    uint32_t ccsidr_numsets_mask =3D CCSIDR_NUMSETS_MASK;
> > +    unsigned int n, line_size, num_sets;
> > +
> > +    for ( n =3D CLIDR_CTYPEn_LEVELS; n !=3D 0; n-- )
> > +    {
> > +        uint8_t ctype_n =3D (clidr_el1 >> CLIDR_CTYPEn_SHIFT(n)) &
> > +                          CLIDR_CTYPEn_MASK;
> > +
> > +        /* At least separate I/D caches (see Arm ARM DDI 0487H.a D13.2=
.27) */
> I'm a bit confused here given that this comment does not reflect the line=
 below (also please refer to the latest spec).
> Since 0b011 is "Separate instruction and data caches" you would break onl=
y for Unified cache.
> That said, we care about last level cache that is visible to SW and I'm n=
ot aware of any Arm CPU where L2,L3 is not unified.

You're right, that should have been >=3D.
Anyway I can check more explicitly for =3D=3D 0b100.

> > +        if ( ctype_n > 0b011 )
> > +            break;
> > +    }
> > +
> > +    if ( n =3D=3D 0 )
> > +        return 0;
> > +
> > +    WRITE_SYSREG((n - 1) << CSSELR_LEVEL_SHIFT, CSSELR_EL1);
> > +
> no need for this empty line
>
> > +    isb();
> > +
> > +    ccsidr_el1 =3D READ_SYSREG(CCSIDR_EL1);
> > +
> > +    /* Arm ARM: (Log2(Number of bytes in cache line)) - 4 */
> > +    line_size =3D 1U << ((ccsidr_el1 & CCSIDR_LINESIZE_MASK) + 4);
> > +
> > +    /* If FEAT_CCIDX is enabled, CCSIDR_EL1 has a different bit layout=
 */
> > +    if ( (id_aa64mmfr2_el1 >> ID_AA64MMFR2_CCIDX_SHIFT) & 0x7 )
> > +    {
> > +        ccsidr_numsets_shift =3D CCSIDR_NUMSETS_SHIFT_FEAT_CCIDX;
> > +        ccsidr_numsets_mask =3D CCSIDR_NUMSETS_MASK_FEAT_CCIDX;
> > +    }
> empty line here please
>
> > +    /* Arm ARM: (Number of sets in cache) - 1 */
> > +    num_sets =3D ((ccsidr_el1 >> ccsidr_numsets_shift) & ccsidr_numset=
s_mask) + 1;
> > +
> > +    printk(XENLOG_INFO "LLC found: L%u (line size: %u bytes, sets num:=
 %u)\n",
> > +           n, line_size, num_sets);
> > +
> > +    /* Restore value in CSSELR_EL1 */
> > +    WRITE_SYSREG(csselr_el1, CSSELR_EL1);
> > +    isb();
> > +
> > +    return line_size * num_sets;
> > +}
> > +
> > +void __init arch_llc_coloring_init(void) {}
> > +
> > +/*
> > + * Local variables:
> > + * mode: C
> > + * c-file-style: "BSD"
> > + * c-basic-offset: 4
> > + * tab-width: 4
> > + * indent-tabs-mode: nil
> > + * End:
> > + */
> > diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> > index 59dd9bb25a..14cb023783 100644
> > --- a/xen/arch/arm/setup.c
> > +++ b/xen/arch/arm/setup.c
> > @@ -12,6 +12,7 @@
> >  #include <xen/device_tree.h>
> >  #include <xen/domain_page.h>
> >  #include <xen/grant_table.h>
> > +#include <xen/llc-coloring.h>
> >  #include <xen/types.h>
> >  #include <xen/string.h>
> >  #include <xen/serial.h>
> > @@ -746,6 +747,8 @@ void asmlinkage __init start_xen(unsigned long boot=
_phys_offset,
> >      printk("Command line: %s\n", cmdline);
> >      cmdline_parse(cmdline);
> >
> > +    llc_coloring_init();
> I think a check with llc_coloring_enabled is missing, given there is none=
 in llc_coloring_init

You're right. It should have been in llc_coloring_init(), my bad.

Thanks.

> ~Michal
>


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:56:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:56:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680791.1058913 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFkE-00030S-A4; Wed, 14 Feb 2024 13:56:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680791.1058913; Wed, 14 Feb 2024 13:56:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFkE-00030L-5l; Wed, 14 Feb 2024 13:56:14 +0000
Received: by outflank-mailman (input) for mailman id 680791;
 Wed, 14 Feb 2024 13:56:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raFkC-000309-V1
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:56:12 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cf8c7eb2-cb40-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 14:56:11 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-411d231ea71so10654635e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 05:56:11 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v13-20020a05600c214d00b0040fdc7f4fcdsm2059764wml.4.2024.02.14.05.56.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 05:56:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cf8c7eb2-cb40-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707918971; x=1708523771; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=yOCG6AC8/5eWg5KRc470FQdsEV72KMeSVzBIyWoh9L8=;
        b=T1buZEVrpVapSS8OkCo0HGNIP69er+R5RE9ibEF7XZEzgG3lFDpK3IBmhogSRbpSeh
         8e1qMl4hPwuwD2LAhYyQEgXYpRlubldtXhke68y0AR1fLToctBVFeTzUu5KVFlSJfqQY
         0cAW9Bi1U1LpF+G3KavaBZwqkZTNKfQHfpr3ow8rSBuQLde5s6F00JPYlQz7hOLd8Rkg
         5yn2rwxOxbYYsxRPjTAwoUryOHOEBiH+mR1yk84T569awKetuoFBQAjbOd1/lasN/VQI
         pZN7S0MPAsCAx4hrtuWXqTBB2Z7Ln5iAzdnI54mWUBalZPTrtAkiAaKGZfZGJS+1fysH
         9BzQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707918971; x=1708523771;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=yOCG6AC8/5eWg5KRc470FQdsEV72KMeSVzBIyWoh9L8=;
        b=iCytT18TnECMUq9U3upRBIKvDg5FhfXoHVXYbJA8vgerfdiln0M2kj0E6N4HimJizn
         Qrm7lsLVjHi8ya0z/rRzJ3kzGEy56fv1N2br1TLQdzfX/+kZ+ukVZcW/rx0APi8FuoPa
         yNsztzZs9vlTSQx+jAXal6/nQMJhMagqjNSjSyXqobozwswFRmI4bdmAVsctrDvc6vtS
         QTIPgDyzg14P/+cSrVLltcRnI3NWO+kWnab5oJZUJwg8RmlFcqoMZhno50TtATq3bVpM
         B8esYv/eyBl9E1pSJD801EQ4rU4uN4w70B4xaTmEQ04Pzrnmk54g4ZIv5fS8W+cOcivT
         cLow==
X-Forwarded-Encrypted: i=1; AJvYcCXMUdGmQLLnltKeBXQAg1TylJhauZi80n08d3WIPmp539M5CTc7W8TAWZL/I+0aPA9s9ycaQ5Th+6rl+k6UX8aJYFwK9lZ89qWT7sn7/SA=
X-Gm-Message-State: AOJu0YyD24JiQ5RBrBjPmmmL9ivOVKVFpa+WIQ7OiM5OfhmFA3zXc7tS
	uQfXLCE0VziuvGL0TcVFEQ5xeAfoWuV0EV+KrsdvsjyvkjnFuAIpKnY3Fif+5g==
X-Google-Smtp-Source: AGHT+IEIyO2SrYHZETCq+nEvS/fJ82llkSjE6Vc1sYeaP1vTJsUvR3NkDey+VH8BvnWwWeGLm2J7YQ==
X-Received: by 2002:a05:600c:1383:b0:410:adc2:6d1c with SMTP id u3-20020a05600c138300b00410adc26d1cmr2415233wmf.12.1707918971186;
        Wed, 14 Feb 2024 05:56:11 -0800 (PST)
Message-ID: <d3de481d-bb66-4ee2-8805-d5ded203d8c5@suse.com>
Date: Wed, 14 Feb 2024 14:56:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] Reduce assembly code size of exception entry points
Content-Language: en-US
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <20240214103558.13655-1-frediano.ziglio@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214103558.13655-1-frediano.ziglio@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.02.2024 11:35, Frediano Ziglio wrote:
> We just pushed a 8-bytes zero

This part is now somewhat stale.

> and exception constants are
> small so we can just write a single byte saving 3 bytes for
> instruction.
> With ENDBR64 this reduces the size of many entry points from 32 to
> 16 bytes (due to alignment).
> Similar code is already used in autogen_stubs.
> 
> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
> --
> v2:
> - added missing entry points;

What about entry_int82?

> @@ -653,7 +653,7 @@ END(ret_from_intr)
>          .section .init.text, "ax", @progbits
>  FUNC(early_page_fault)
>          ENDBR64
> -        movl  $X86_EXC_PF, 4(%rsp)
> +        movb  $X86_EXC_PF, 4(%rsp)
>          SAVE_ALL
>          movq  %rsp, %rdi
>          call  do_early_page_fault

Between this and the next hunk there's also entry_PF.

Jan

> @@ -898,105 +898,105 @@ END(handle_exception)
>  FUNC(entry_DE)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_DE, 4(%rsp)
> +        movb  $X86_EXC_DE, 4(%rsp)
>          jmp   handle_exception
>  END(entry_DE)
>  
>  FUNC(entry_MF)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_MF, 4(%rsp)
> +        movb  $X86_EXC_MF, 4(%rsp)
>          jmp   handle_exception
>  END(entry_MF)
>  
>  FUNC(entry_XM)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_XM, 4(%rsp)
> +        movb  $X86_EXC_XM, 4(%rsp)
>          jmp   handle_exception
>  END(entry_XM)
>  
>  FUNC(entry_NM)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_NM, 4(%rsp)
> +        movb  $X86_EXC_NM, 4(%rsp)
>          jmp   handle_exception
>  END(entry_NM)
>  
>  FUNC(entry_DB)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_DB, 4(%rsp)
> +        movb  $X86_EXC_DB, 4(%rsp)
>          jmp   handle_ist_exception
>  END(entry_DB)
>  
>  FUNC(entry_BP)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_BP, 4(%rsp)
> +        movb  $X86_EXC_BP, 4(%rsp)
>          jmp   handle_exception
>  END(entry_BP)
>  
>  FUNC(entry_OF)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_OF, 4(%rsp)
> +        movb  $X86_EXC_OF, 4(%rsp)
>          jmp   handle_exception
>  END(entry_OF)
>  
>  FUNC(entry_BR)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_BR, 4(%rsp)
> +        movb  $X86_EXC_BR, 4(%rsp)
>          jmp   handle_exception
>  END(entry_BR)
>  
>  FUNC(entry_UD)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_UD, 4(%rsp)
> +        movb  $X86_EXC_UD, 4(%rsp)
>          jmp   handle_exception
>  END(entry_UD)
>  
>  FUNC(entry_TS)
>          ENDBR64
> -        movl  $X86_EXC_TS, 4(%rsp)
> +        movb  $X86_EXC_TS, 4(%rsp)
>          jmp   handle_exception
>  END(entry_TS)
>  
>  FUNC(entry_NP)
>          ENDBR64
> -        movl  $X86_EXC_NP, 4(%rsp)
> +        movb  $X86_EXC_NP, 4(%rsp)
>          jmp   handle_exception
>  END(entry_NP)
>  
>  FUNC(entry_SS)
>          ENDBR64
> -        movl  $X86_EXC_SS, 4(%rsp)
> +        movb  $X86_EXC_SS, 4(%rsp)
>          jmp   handle_exception
>  END(entry_SS)
>  
>  FUNC(entry_GP)
>          ENDBR64
> -        movl  $X86_EXC_GP, 4(%rsp)
> +        movb  $X86_EXC_GP, 4(%rsp)
>          jmp   handle_exception
>  END(entry_GP)
>  
>  FUNC(entry_AC)
>          ENDBR64
> -        movl  $X86_EXC_AC, 4(%rsp)
> +        movb  $X86_EXC_AC, 4(%rsp)
>          jmp   handle_exception
>  END(entry_AC)
>  
>  FUNC(entry_CP)
>          ENDBR64
> -        movl  $X86_EXC_CP, 4(%rsp)
> +        movb  $X86_EXC_CP, 4(%rsp)
>          jmp   handle_exception
>  END(entry_CP)
>  
>  FUNC(entry_DF)
>          ENDBR64
> -        movl  $X86_EXC_DF, 4(%rsp)
> +        movb  $X86_EXC_DF, 4(%rsp)
>          /* Set AC to reduce chance of further SMAP faults */
>          ALTERNATIVE "", stac, X86_FEATURE_XEN_SMAP
>          SAVE_ALL
> @@ -1022,7 +1022,7 @@ END(entry_DF)
>  FUNC(entry_NMI)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_NMI, 4(%rsp)
> +        movb  $X86_EXC_NMI, 4(%rsp)
>  END(entry_NMI)
>  
>  FUNC(handle_ist_exception)
> @@ -1158,7 +1158,7 @@ END(handle_ist_exception)
>  FUNC(entry_MC)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_MC, 4(%rsp)
> +        movb  $X86_EXC_MC, 4(%rsp)
>          jmp   handle_ist_exception
>  END(entry_MC)
>  



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 13:56:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 13:56:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680794.1058922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFkS-0003IQ-Gj; Wed, 14 Feb 2024 13:56:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680794.1058922; Wed, 14 Feb 2024 13:56:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFkS-0003IC-Da; Wed, 14 Feb 2024 13:56:28 +0000
Received: by outflank-mailman (input) for mailman id 680794;
 Wed, 14 Feb 2024 13:56:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3HRx=JX=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1raFkQ-000309-Pp
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 13:56:26 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d7ff73ea-cb40-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 14:56:25 +0100 (CET)
Received: from [192.168.1.229] (93-36-216-194.ip62.fastwebnet.it
 [93.36.216.194])
 by support.bugseng.com (Postfix) with ESMTPSA id 23AF04EE0738;
 Wed, 14 Feb 2024 14:56:25 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7ff73ea-cb40-11ee-8a4d-1f161083a0e0
Message-ID: <3632611e-61ff-455a-9cc1-990a1f663d83@bugseng.com>
Date: Wed, 14 Feb 2024 14:56:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/3] docs/misra/rules.rst: add rule 5.5
Content-Language: en-US, it
To: Jan Beulich <jbeulich@suse.com>
Cc: andrew.cooper3@citrix.com, george.dunlap@citrix.com, julien@xen.org,
 bertrand.marquis@arm.com, roger.pau@citrix.com,
 xen-devel@lists.xenproject.org,
 Stefano Stabellini <stefano.stabellini@amd.com>, roberto.bagnara@bugseng.com
References: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
 <20240213223334.3693410-2-stefano.stabellini@amd.com>
 <50719397-b053-43e1-9cf7-cc9eae9098ed@suse.com>
 <40b7465f-4966-43c7-8db3-e28a6cc48445@bugseng.com>
 <2da73259-a86e-4a13-9fd4-4217372f4b76@suse.com>
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <2da73259-a86e-4a13-9fd4-4217372f4b76@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 14/02/24 14:15, Jan Beulich wrote:
> On 14.02.2024 12:27, Federico Serafini wrote:
>> On 14/02/24 09:28, Jan Beulich wrote:
>>> On 13.02.2024 23:33, Stefano Stabellini wrote:
>>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
>>>> ---
>>>>    docs/misra/rules.rst | 6 ++++++
>>>>    1 file changed, 6 insertions(+)
>>>>
>>>> diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
>>>> index c185366966..931158b354 100644
>>>> --- a/docs/misra/rules.rst
>>>> +++ b/docs/misra/rules.rst
>>>> @@ -181,6 +181,12 @@ maintainers if you want to suggest a change.
>>>>           headers (xen/include/public/) are allowed to retain longer
>>>>           identifiers for backward compatibility.
>>>>    
>>>> +   * - `Rule 5.5 <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_05_05.c>`_
>>>> +     - Required
>>>> +     - Identifiers shall be distinct from macro names
>>>> +     - Clashes between function-like macros and non-callable entities
>>>> +       are allowed. The pattern #define x x is also allowed.
>>>
>>> Just for me to know what exactly is covered (hence also a question
>>> to Roberto as to [to be] implemented Eclair behavior): Even when
>>> the above would be sufficient (and imo better) people frequently
>>> write
>>>
>>> #define a(x, y) b(x, y)
>>>
>>> which, transformed to the specific case here, would then be
>>>
>>> #define a(x, y) a(x, y)
>>>
>>> I'd assume such ought to also be covered, but that's not clear
>>> from the spelling above.
>>
>> I list what happens in some different situations,
>> then we can find the right words for the documentation and/or
>> refine the configuration:
>>
>> If you
>> #define x x
>> and then use `x' as identifier,
>> the resulting violation is deviated (allowed pattern).
>>
>> If you
>> #define a(x, y) a(x, y)
>> and then use `a' as identifier for a non-callable entity,
>> the resulting violation is deviated (no clash with non-callable
>> entities).
>> If you use identifier `a' for a callable entity, the resulting violation
>> is reported: the allowed pattern covers only macros expanding to their
>> own name, in this case the macro name is considered to be
>> `a' only, not a(x, y).
>>
>> If you
>> #define a(x, y) b(x, y)
>> and then use `a' as identifier for a non-callable entity,
>> the resulting violation is deviated (no clash with non-callable
>> entities).
> 
> I'm afraid I don't see what violation there is in this case, to
> deviate. As a result I'm also not sure I correctly understand the
> rest of your reply.

#define a(x, y) b(x, y)

int a; // Violation of Rule 5.5.

The macro name `a' that exist before the preprocessing phase,
still exists after the preprocessing phase as identifier for the integer
variable and this is a violation.

>> If you use `a' as identifier for a callable entity,
>> this is not a violation because after the preprocessing phase,
>> identifier `a' no longer exists.
I correct myself:
if you use `a' as identifier for a *function*,
it is not a violation because after the preprocessing phase
the identifier `a' no longer exists, for example:

#define a(x, y) b(x, y)

void a(int x, int y); // Ok.

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 14:00:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 14:00:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680803.1058933 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFoG-0005WM-5d; Wed, 14 Feb 2024 14:00:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680803.1058933; Wed, 14 Feb 2024 14:00:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFoG-0005WF-2j; Wed, 14 Feb 2024 14:00:24 +0000
Received: by outflank-mailman (input) for mailman id 680803;
 Wed, 14 Feb 2024 14:00:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raFoE-0005W7-PH
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 14:00:22 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 64082c3f-cb41-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 15:00:20 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d0cdbd67f0so77878371fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 06:00:20 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 h7-20020ac85147000000b0042c21e75aeesm1997900qtn.64.2024.02.14.06.00.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 06:00:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64082c3f-cb41-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707919220; x=1708524020; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=SK+ljDOJFwB1Pcm4QUM+mo6y8GGH+iEANQ6BOvMG5o4=;
        b=OPAIm1cSzndgve/pIdyXGGQBu9nBuqoZkwWnpv08h4YmolOXGpEBqBRIHk/4skYhHr
         oXkozQWex10B3KAofW9+4ueYAsIDX6AH5vWXEo/d/yjyi2W64SLkmfDDnRNxWE0HnUe2
         xJFl4v+Ua0ALPOFZKsS96pwZGtTk5EQersU38=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707919220; x=1708524020;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SK+ljDOJFwB1Pcm4QUM+mo6y8GGH+iEANQ6BOvMG5o4=;
        b=mEpOq4a9AqfATfVCHH6IlTHPhWolBxdLPoqtnySna7lUhbq6rr4ZlnpoS6NUz57wCa
         7Gr8/ZR3gp2MySNDvu0+zONrWxHBuezNq2UmpwyNhQb/hLyOKz4Nrro/GlpdjbIQFs/Z
         TKTxYOE1N7zphhGTz4B/2nsOSIMKKvN4dlmUpZYnjkcfMJC+SycPJZMiT71RJemyW5mO
         vZFzNNQ4h6AvOM8bJVFMJCGYdraXta4MTYnFNrZux9ZBWjChqCJlzvVkyqPnco/pJC2m
         OJA6uy4TdpIau3b8FiIha5B7mTXUcBKoFYxEbEdguKKs8dfkUOtBStpcMMfr2xt6JvWN
         TNcQ==
X-Forwarded-Encrypted: i=1; AJvYcCU9Xw5bYV44vkRaI5Ru/Rej1BzmEOC3BixNny/8Ic6TaX0yxubCTDKJDhcojdky5GGUoyMJ8QL0D9RH/coPyOr0fWggXH7uPJFidbJ+7xE=
X-Gm-Message-State: AOJu0YzNQd5QpQtE94V6BX0BqeMdo6gcEAmd/S+XA9ZhDUQRp4Q+fiKl
	xlTSStm0D5xCpjU+c7r+Zgy7PRC5koBb6x1JX6O+tJBro8PN5cQ5Yxsylr4qXrI=
X-Google-Smtp-Source: AGHT+IF9d+rjt2dxgOmfcpJTuI+BCrZDmQTK1bnJgLnJ1TTDXqNVcaeetIu/UdJF7IN7GK8rAgTkLQ==
X-Received: by 2002:a2e:a686:0:b0:2d0:b3c4:5113 with SMTP id q6-20020a2ea686000000b002d0b3c45113mr2037359lje.11.1707919220207;
        Wed, 14 Feb 2024 06:00:20 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUgl3+Ub4/7Ry97ZnjZsHzIm4TgTAqX/UW8dqsL846sT9HzTo7JG+4/GxQFegO4BD/wrImCxnOK4mFV9vLlwRuMsRZFjThQtSO4NLzAa5DaS0p2g4tkVKhnLJd2YyzyRAQ0d6an805H6+miWPVgMB9mlt88jtuew7bNCxildDvgwZdWaA9IFpnhkIiMElOyowX3Ty1OwjloGaXyPHg=
Date: Wed, 14 Feb 2024 15:00:17 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH 5/5] mm: add the __must_check attribute to {gfn,mfn}_add()
Message-ID: <ZczHcTSaxM3KJOR_@macbook>
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-6-roger.pau@citrix.com>
 <ff45456b-87bf-4e6c-b2c9-6f3d5258d478@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <ff45456b-87bf-4e6c-b2c9-6f3d5258d478@suse.com>

On Wed, Feb 14, 2024 at 02:42:46PM +0100, Jan Beulich wrote:
> On 14.02.2024 11:37, Roger Pau Monne wrote:
> > It's not obvious from the function itself whether the incremented value will be
> 
> s/the function itself/just the function name/ ?
> 
> > stored in the parameter, or returned to the caller.  That has leads to bugs in
> > the past as callers assume the incremented value is stored in the parameter.
> 
> ... callers may assume ... ?
> 
> > --- a/xen/include/xen/mm-frame.h
> > +++ b/xen/include/xen/mm-frame.h
> > @@ -23,7 +23,7 @@ TYPE_SAFE(unsigned long, mfn);
> >  #undef mfn_x
> >  #endif
> >  
> > -static inline mfn_t mfn_add(mfn_t mfn, unsigned long i)
> > +static inline mfn_t __must_check mfn_add(mfn_t mfn, unsigned long i)
> >  {
> >      return _mfn(mfn_x(mfn) + i);
> >  }
> > @@ -62,7 +62,7 @@ TYPE_SAFE(unsigned long, gfn);
> >  #undef gfn_x
> >  #endif
> >  
> > -static inline gfn_t gfn_add(gfn_t gfn, unsigned long i)
> > +static inline gfn_t __must_check gfn_add(gfn_t gfn, unsigned long i)
> >  {
> >      return _gfn(gfn_x(gfn) + i);
> >  }
> 
> What about dfn_add() (in xen/iommu.h)?

Oh, didn't spot that one.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 14:04:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 14:04:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680807.1058943 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFrk-0006Ga-Jr; Wed, 14 Feb 2024 14:04:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680807.1058943; Wed, 14 Feb 2024 14:04:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFrk-0006GT-Gl; Wed, 14 Feb 2024 14:04:00 +0000
Received: by outflank-mailman (input) for mailman id 680807;
 Wed, 14 Feb 2024 14:04:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raFrk-0006GN-7W
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 14:04:00 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e5ad1855-cb41-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 15:03:58 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-51171c9f4c0so6713159e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 06:03:58 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 kf24-20020a05622a2a9800b0042c6c5bc47fsm2019131qtb.83.2024.02.14.06.03.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 06:03:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e5ad1855-cb41-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707919438; x=1708524238; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=lhWgYXhhqFI+/ERbgKPrgT4K9Zc+cpdMIudB9KDFJgw=;
        b=XaRaRTZX+iyQPDJvcDhHN3rwsD875k1X9AFnF2vAMn8Q5PN7WaqPwcwCDATQUoebwx
         LsP/1iCdA7sekEiYngDabFMQh5BdGLv671q5SB3n61GDXq0bzK+JrlY79YT+Klcuap/3
         Fn/0DZMNXWe/fx/erDikYYXgBP09E0VqqF+0o=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707919438; x=1708524238;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=lhWgYXhhqFI+/ERbgKPrgT4K9Zc+cpdMIudB9KDFJgw=;
        b=uFYGi5zAOIriVcVQSP4FXgNfaW4lWUz5kenrrII+LLQCsw0QVcr+Mi9+6JITgBRadj
         kN1u7z9rYza3fQvo2QJcUoR0HjBrFty3inviUpETzksBRC1v6HGHRWlHZTZu2EjvPxkW
         BukmsZUlhH4Bw0Ih+CXE0mMxC22RBY4QOiXSlX8QmuL183ZwuMlCXud3pGwWi99o+I+c
         kJKbAHZbwAQjSSlq/8lWkCpmTaJpaTj0XmkPfqvNOCYwHLDpLlRiFR88KS1WzXLSF94B
         KoxeYAIdzfst6IZr8pWF4lVL9OgiFLZe/pLpPCV4r65ph8RwDxLKfICIxwJwG7/HpcDK
         8GXw==
X-Forwarded-Encrypted: i=1; AJvYcCXJVp+bZ8bVBUJf9pHoW/LdagbeC8CTPR4RHydsyewAGOgaOOklBHAVQk8KT8EEueEJZWyUhXIdfOPoWcTV55lSJeMWFHgQUdAOlZSULIo=
X-Gm-Message-State: AOJu0YyJ6weGD/dPO8mSbrmXX8k7QQNu0xxrINfnw9vFcP/1xxTl6tHy
	5n1YeJR2zaz1CMNq7Jjm7VBithoXhTxeQ4TPr8SPhVz1KtBdQYhspGZqVfZ1fZE=
X-Google-Smtp-Source: AGHT+IGUd5Z04e/CMB3mAtU8dI0tbzt3pzLK0HxkmBTX6Qx0lnugbV/I/b/fa+ekAMdd2ICNTFFboQ==
X-Received: by 2002:a05:6512:2002:b0:511:a587:dc27 with SMTP id a2-20020a056512200200b00511a587dc27mr1396934lfb.43.1707919437787;
        Wed, 14 Feb 2024 06:03:57 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVR8mstF/miSV5POx81ZRwthvKfX2tMF+AFNWL2ksnzsfyEDMhejVVgCtvGmCx6D2xGvEiF93oTh436WXjv5l39rTOiuOLpiNJeyMJzDMqVdFEQw2kYpXiDrh0odw==
Date: Wed, 14 Feb 2024 15:03:55 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH 2/5] iommu/x86: print RMRR/IVMD ranges using full
 addresses
Message-ID: <ZczIS7Rlnt0xeVGF@macbook>
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-3-roger.pau@citrix.com>
 <2416e41e-0b59-4e63-a9c2-550983b71c1c@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <2416e41e-0b59-4e63-a9c2-550983b71c1c@suse.com>

On Wed, Feb 14, 2024 at 02:22:09PM +0100, Jan Beulich wrote:
> On 14.02.2024 11:37, Roger Pau Monne wrote:
> > It's easier to correlate with the physical memory map if the addresses are
> > fully printed, instead of using frame numbers.
> > 
> > Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> In principle
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> I'm not sure though that this fully matches Andrew's intentions:
> 
> > --- a/xen/drivers/passthrough/x86/iommu.c
> > +++ b/xen/drivers/passthrough/x86/iommu.c
> > @@ -801,8 +801,8 @@ bool __init iommu_unity_region_ok(const char *prefix, mfn_t start, mfn_t end)
> >          return true;
> >  
> >      printk(XENLOG_WARNING
> > -           "%s: [%#" PRI_mfn " ,%#" PRI_mfn "] is not (entirely) in reserved memory\n",
> > -           prefix, mfn_x(start), mfn_x(end));
> > +           "%s: [%#lx, %#lx] is not (entirely) in reserved memory\n",
> > +           prefix, mfn_to_maddr(start), mfn_to_maddr(end));
> 
> e820.c uses [%016Lx, %016Lx] instead, i.e. full 16-digit numbers. For
> easiest cross matching it may be desirable to do the same here. Then
> of course the 0x prefixes may also better disappear.

Yes, I also saw that format, but wasn't sure whether it was desirable
to use here, as for example I would expect RMRR/IVMD regions to be
below the 4GB boundary.  Also the leading zeros are useful to have a
uniform table when printing the memory map that contains more than
one entry, but here I expect printing will be limited to a very small
set of entries, or maybe just one (as we only print the misplaced
ones).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 14:05:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 14:05:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680813.1058953 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFt2-0006rq-Tf; Wed, 14 Feb 2024 14:05:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680813.1058953; Wed, 14 Feb 2024 14:05:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFt2-0006rj-Qa; Wed, 14 Feb 2024 14:05:20 +0000
Received: by outflank-mailman (input) for mailman id 680813;
 Wed, 14 Feb 2024 14:05:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raFt1-0006rZ-NF
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 14:05:19 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1511a45a-cb42-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 15:05:17 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-3394bec856fso492935f8f.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 06:05:17 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 od9-20020a0562142f0900b0068c9d0561d1sm2291587qvb.83.2024.02.14.06.05.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 06:05:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1511a45a-cb42-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707919517; x=1708524317; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=wgXaN7odOXC9XR/4OcmjtChcknL7ns+w+h62EojdQ10=;
        b=QMW8tbhRiMYtmo79dFyWWlmFvpsxA7f401p2HEeh5MUdGvNE7T1aucLT5EXTzrYtZg
         C2xVMk0yjiTIh0XVKj4MMETqUFH5oPrcDnf+LCRJgUwMzjO3MYErxSFPYERDcVMf7MtM
         f1kCdacuMPiFB8SFJ7GqsmmSMtllXsJs/dOVU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707919517; x=1708524317;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wgXaN7odOXC9XR/4OcmjtChcknL7ns+w+h62EojdQ10=;
        b=srDLQNNKvyw58ELaminWBUdLiXKmP8IWqUKm7o+fYYXzWpudVg0h0D8TErYXejBao5
         6cNPncA+MQd0vYVFaqvME9yjfULDQVTkbnlBbWKqB9r7Sr/Mk6iH23/5g+8to0hlCBJR
         iYV8dzRrHU9kqVSSxf7/JcWv9I+CBap7TdKd05UkXYjWjHLueJ5idv6QCBWyR9RPARuO
         cmr6PEpIwEljP6XYXKcUiGVSn+MkwUx2+73g5JQCNG4o3leOo5MdRDB6JKo+bcavQubJ
         i4Zc35ERGu//x3V9vuzbYc18Jnid/lmxUouDFJPuzzaYqDIU1LE8U3LGihMFmr1ITqv+
         Sp1A==
X-Forwarded-Encrypted: i=1; AJvYcCWBGg2DWW92wm6UIEwgZOBnPG6i8CtPI/Zl7OV23hH3nh8pvv7G6Bk7GGMSRb+7wUsvxJiXVDnlirVYJJwTgOzCQisNAFJZXw9V50L2px0=
X-Gm-Message-State: AOJu0Yznp0pdWXxwHwl4LbsabMh/fKEWNTMnUqKx1q6G93nOqAm3dsLs
	btZoVWGxgAI/lcFzikYGedKZdSknrNT76Y3AzfGIvJl+x77roSdVCeEpiz7s8Gk=
X-Google-Smtp-Source: AGHT+IFfhnBW5Cv3eeME0Tt2w1udCJ0ptH2yzKhV56dZwsR8vCb38gLXD2z+JGwZLYIXAfw/MqvRmg==
X-Received: by 2002:adf:e9c4:0:b0:33c:ddf1:4e70 with SMTP id l4-20020adfe9c4000000b0033cddf14e70mr2269967wrn.32.1707919517233;
        Wed, 14 Feb 2024 06:05:17 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXfWNz7jxTVb6OdCztegW9/VSpPzCjU7AD9ZQCvOTqYejcUTOFTQVy+3oTmyfkajgOS44SbIMIOouqpffVJgUhhCKXvpW2WkkdyOQel9NZK1B65GgK1J41BMF4VCNpevapAnfzo2TmDmQQ=
Date: Wed, 14 Feb 2024 15:05:15 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Paul Durrant <paul@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH 3/5] iommu/x86: use full addresses internally for the
 IVMD/RMRR range checks
Message-ID: <ZczIm37J3WBQPYsx@macbook>
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-4-roger.pau@citrix.com>
 <16de2e3e-7807-4754-852b-e486c1fde95c@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <16de2e3e-7807-4754-852b-e486c1fde95c@suse.com>

On Wed, Feb 14, 2024 at 02:29:35PM +0100, Jan Beulich wrote:
> On 14.02.2024 11:37, Roger Pau Monne wrote:
> > Adjust the code in the checker to use full addresses rather than frame numbers,
> > as it's only page_get_ram_type() that requires an mfn parameter.
> > 
> > Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> In this very shape I'd like to leave this to Paul or Andrew: I'm not outright
> opposed, but I think this either goes too far (most type-safety being lost
> again), or not far enough (both callers convert addresses to MFNs, just for
> them to be converted back here).

I don't have a strong opinion, given this would you like me to adjust
4/5 so it can applied without this patch?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 14:09:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 14:09:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680819.1058963 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFwj-0007Z4-Cs; Wed, 14 Feb 2024 14:09:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680819.1058963; Wed, 14 Feb 2024 14:09:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFwj-0007Yx-9Y; Wed, 14 Feb 2024 14:09:09 +0000
Received: by outflank-mailman (input) for mailman id 680819;
 Wed, 14 Feb 2024 14:09:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raFwi-0007Yn-LJ; Wed, 14 Feb 2024 14:09:08 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raFwi-0004cl-Jb; Wed, 14 Feb 2024 14:09:08 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raFwi-0006dn-7w; Wed, 14 Feb 2024 14:09:08 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raFwi-0002ex-7T; Wed, 14 Feb 2024 14:09:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hC1FdIkwoMFcbfHPgDcH/HWqnzrsR/eT1QAGsw/6NSA=; b=JrVAWf55QVaVnfQpqA063i4hKI
	MHdF44N3/UFORfQ3Cn8M9en88tzDl/ilFCxTClBQ8JptHsQjXju87js80KiPGRSerp731MhGVJqTW
	o6331qLVx3nB8Lt94/AAzX+OByXNvuIHXfmIafbQMOsTfPE1RaqkJGrcd0f/bl7EbPMc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184666-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184666: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=d670c1a38ba3561296f68c0079209365760b3001
X-Osstest-Versions-That:
    xen=b3bea912a737a0db39a89b0da0033c4bfda82f03
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 14 Feb 2024 14:09:08 +0000

flight 184666 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184666/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  d670c1a38ba3561296f68c0079209365760b3001
baseline version:
 xen                  b3bea912a737a0db39a89b0da0033c4bfda82f03

Last test of basis   184658  2024-02-13 14:00:26 Z    1 days
Testing same since   184666  2024-02-14 10:04:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Cyril Rébert (zithro) <slack@rabbit.lu>
  Cyril Rébert <slack@rabbit.lu>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Juergen Gross <jgross@suse.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Petr Beneš <w1benny@gmail.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   b3bea912a7..d670c1a38b  d670c1a38ba3561296f68c0079209365760b3001 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 14:12:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 14:12:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680824.1058973 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFzU-00010I-Qx; Wed, 14 Feb 2024 14:12:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680824.1058973; Wed, 14 Feb 2024 14:12:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raFzU-00010B-O9; Wed, 14 Feb 2024 14:12:00 +0000
Received: by outflank-mailman (input) for mailman id 680824;
 Wed, 14 Feb 2024 14:11:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raFzT-000105-U3
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 14:11:59 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 03a6a007-cb43-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 15:11:58 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-511821e60c2so5087454e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 06:11:58 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 pa6-20020a056214480600b0068ca40389besm2276097qvb.101.2024.02.14.06.11.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 06:11:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03a6a007-cb43-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707919917; x=1708524717; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YCIVxdokNXSrLnsPOXEL8IpvANxXGvqRhyL2hkP1P4s=;
        b=qMS6D3NGdU924jOZsM8teCjdz0XsihA86XqIh3hGBkYrukO0o0bV7dUdmUsUsuWwqo
         okcYGj/09BkcHRZGvxCAmkMEgp/hkvUVRqeaRM8ue/uJe7rjXpDW2jZxseSSObs4L9+i
         PahWjGQoe6BbKL5OvZJhuVNbE87kHKxFdmW4I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707919917; x=1708524717;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=YCIVxdokNXSrLnsPOXEL8IpvANxXGvqRhyL2hkP1P4s=;
        b=XMeXJVEKEW5b5W7JeS2k94qfrfxS1QfHBS+MWeymjvCckEWVf+ky6EI7H3zsoNchoH
         xqtoBr6UvejleUSiZNnvnfm74bJF2xfFmTO665QmjxVDlfyMr+AxCtP8mgOXbETU7vBW
         RUNYdub81z2PrzR9grIojWHM1kkIMAx1XmBUlo+S5IXKsakZN0FER8eVdWIbJ3Nu6rMU
         IL0RG0qdGxTyUWq8sy2F9UP8djZ5YL1rFhi1Pn/b5d7YJwjxKjtUMyj8aMEWp9aRiS1S
         55BKoc7eLIVGuGO9E80qU7HmPkeHnQBHB9IqnRexd0apljT7JlTCoR7PhKo0lUmwbhWI
         jGlw==
X-Gm-Message-State: AOJu0Yw+ukDECB1dQhqYPscpp6g2mDtk7avVy9uCBvHch6F7hF9vUP7x
	KycwJyTzOdWkYpGcy+Y5NXO8mnb4Kikd3TdQDb5P7xiJGrPkCSjkXcKdBzjub7TgtZkD0wr2wwR
	o
X-Google-Smtp-Source: AGHT+IG8hwFfJ0slL4KEWZFWJ0kTzDzauVvaUmHlAl9WH66eSLFsrN3Zx3/gd3dbEVol7fa3MC5t3g==
X-Received: by 2002:a05:6512:3444:b0:511:a100:b30d with SMTP id j4-20020a056512344400b00511a100b30dmr1751915lfr.56.1707919916870;
        Wed, 14 Feb 2024 06:11:56 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXHINi5l0kFrPuwrx9PFTumxxwpHs7i+OrAvFfe4Z/7BOomKgJCRdDI8LtSZ7TPVzqPU2BsmnbLz09HQ6qPXYGOUptNHRqyBBhZsZU5zVFKsmDfGNHQRF4vX3PLeQm0M0qAeulnMxRfTEl/zNWRtlHrBkOWsoHJ31N4ijFP9jKa9N1kdUit6H0P7UWz7ujZYkoqe0Q3lh4e3PvsVFI4C33Seoh//zsnYSAH2hje88maJ3pAo29lUH8/MnUBU3rLeNkTQtbEBAzuXvt0RA==
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v2 5/5] mm: add the __must_check attribute to {gfn,mfn,dfn}_add()
Date: Wed, 14 Feb 2024 15:11:37 +0100
Message-ID: <20240214141137.17787-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240214103741.16189-6-roger.pau@citrix.com>
References: <20240214103741.16189-6-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It's not obvious from just the function name whether the incremented value will
be stored in the parameter, or returned to the caller.  That has leads to bugs
in the past as callers may assume the incremented value is stored in the
parameter.

Add the __must_check attribute to the function to easily spot callers that
don't consume the returned value, which signals an error in the caller logic.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
Changes since v1:
 - Also add attribute to dfn_add().
---
 xen/include/xen/iommu.h    | 2 +-
 xen/include/xen/mm-frame.h | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 7aa6a7720977..9621459c63ee 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -42,7 +42,7 @@ TYPE_SAFE(uint64_t, dfn);
 #undef dfn_x
 #endif
 
-static inline dfn_t dfn_add(dfn_t dfn, unsigned long i)
+static inline dfn_t __must_check dfn_add(dfn_t dfn, unsigned long i)
 {
     return _dfn(dfn_x(dfn) + i);
 }
diff --git a/xen/include/xen/mm-frame.h b/xen/include/xen/mm-frame.h
index 922ae418807a..c25e836f255a 100644
--- a/xen/include/xen/mm-frame.h
+++ b/xen/include/xen/mm-frame.h
@@ -23,7 +23,7 @@ TYPE_SAFE(unsigned long, mfn);
 #undef mfn_x
 #endif
 
-static inline mfn_t mfn_add(mfn_t mfn, unsigned long i)
+static inline mfn_t __must_check mfn_add(mfn_t mfn, unsigned long i)
 {
     return _mfn(mfn_x(mfn) + i);
 }
@@ -62,7 +62,7 @@ TYPE_SAFE(unsigned long, gfn);
 #undef gfn_x
 #endif
 
-static inline gfn_t gfn_add(gfn_t gfn, unsigned long i)
+static inline gfn_t __must_check gfn_add(gfn_t gfn, unsigned long i)
 {
     return _gfn(gfn_x(gfn) + i);
 }
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 14:20:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 14:20:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680834.1058982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raG7S-0003CS-JT; Wed, 14 Feb 2024 14:20:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680834.1058982; Wed, 14 Feb 2024 14:20:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raG7S-0003CL-Gf; Wed, 14 Feb 2024 14:20:14 +0000
Received: by outflank-mailman (input) for mailman id 680834;
 Wed, 14 Feb 2024 14:20:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raG7R-0003CD-BB
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 14:20:13 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2a22eb70-cb44-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 15:20:12 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d09bdddfc9so73769421fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 06:20:12 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 jl24-20020a05600c6a9800b00410ac2d6b40sm2083113wmb.8.2024.02.14.06.20.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 06:20:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a22eb70-cb44-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707920411; x=1708525211; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IBzfqUtIcuE9KowMDf8kv7FMyo2GLcIHsuFAzM9TdPw=;
        b=UkVI4WlLKH5jxHgfcB6wHHFPJIRaeoYRI1pneItK/YQpaZtYOxjc1HP0f4gGFWI69b
         8X3X8o83BTKGUXfAXe0RPUrHZiJrB9ODJgPhHEZikRCks2xeDVvYFko3N/Ja1cwZU1fX
         jgUUBjxu+XgQ12wsBIo2Z1LmL1KhEY03zfsGw/u3Brle8NuD4TAGaCX0mtAA5M9OuKd7
         f84Q2lCfjJdz5oLYa3FwXySvMp4YKpVaceBSuhXIYCIXTplFJgwG9hGBQYV2l0SVrhiN
         A3F3tGJqTlZDUfJmdU6RBWULBymhRtrrSZ1M8RaeuYIxVsl5I2S8Y3BYYKRIab/rjXpZ
         wl9Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707920411; x=1708525211;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IBzfqUtIcuE9KowMDf8kv7FMyo2GLcIHsuFAzM9TdPw=;
        b=FJFvstIH2soZCbOUiXwCdwndLVt1IHGOWemOjgcY/ecL4RuQOjwocNbwSNiS5m1dzL
         NEI5UZ5mTvJrBTB3KWV+pWqKyjYeyK/UL8EtlAc+uAJZh3yYk2al/SWUHmHyVVX7TJ74
         41NHh61n1UYuXqRL/thlk1HgEK0XlRC4wYFKT3MJTFmrORIcAH5lRAE5G15w18pYCL6a
         1RCCX5/JhVbLM1Vjj/sPOo1vug9bUnkTEHJLU1xVrhgUN+sAZdltW3u+Clq/A/jC+Wzm
         q4+FSLZALbtAK0jaM4Wjrqqvx9twK6rK3JlcMOX/oHIji8KIKUU52N4jn24yF22sHZmK
         PIyg==
X-Forwarded-Encrypted: i=1; AJvYcCUi8/4jLIez09HKkwGykbcjNaWDSduQ5DPtLLnln8g1OXTXyAMIlIspfDnUA5xt8c4BCm7UHZVZn1epTQVbuQJ5reHp/X+vrIlLQWraMOI=
X-Gm-Message-State: AOJu0YxSES91JiTWJDrL4fbcRSAfYd6zH2u6VNN7Cmw8ux1WUzlaBgR4
	2p+RU3GJMfXrjZUE7vwk2aPBjgqxaULebj85u0qdYGeh8/73cK4JMSvi/vQDPA==
X-Google-Smtp-Source: AGHT+IGER8DJUrbp5BpfnCNDkDghfV05beEkWFdp/Nbsh3e3IFfemTP5bMsizuHw/c7AvuA8jI5OEg==
X-Received: by 2002:a05:6512:32a2:b0:511:4c52:ff53 with SMTP id q2-20020a05651232a200b005114c52ff53mr1958684lfe.53.1707920411570;
        Wed, 14 Feb 2024 06:20:11 -0800 (PST)
Message-ID: <5e13d688-3cc3-48dc-ad7b-81f94734f5f3@suse.com>
Date: Wed, 14 Feb 2024 15:20:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] Reduce assembly code size of exception entry points
Content-Language: en-US
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
References: <20240214103558.13655-1-frediano.ziglio@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214103558.13655-1-frediano.ziglio@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.02.2024 11:35, Frediano Ziglio wrote:
> @@ -898,105 +898,105 @@ END(handle_exception)
>  FUNC(entry_DE)
>          ENDBR64
>          pushq $0
> -        movl  $X86_EXC_DE, 4(%rsp)
> +        movb  $X86_EXC_DE, 4(%rsp)

As we're trying to compact things: This writes 0 over the previously
pushed 0. The insn therefore could be replaced by
"BUILD_BUG_ON X86_EXC_DE".

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 14:20:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 14:20:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680836.1058993 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raG7l-0003gC-Ue; Wed, 14 Feb 2024 14:20:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680836.1058993; Wed, 14 Feb 2024 14:20:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raG7l-0003g5-RW; Wed, 14 Feb 2024 14:20:33 +0000
Received: by outflank-mailman (input) for mailman id 680836;
 Wed, 14 Feb 2024 14:20:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=U0HE=JX=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1raG7k-0003eC-Go
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 14:20:32 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 35063d27-cb44-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 15:20:30 +0100 (CET)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-5638ae09ec1so683773a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 06:20:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 35063d27-cb44-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1707920430; x=1708525230; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=rQOqQYiwVvevA4SrtadQQkQhVXKeL4AfeaO3c9Od7bg=;
        b=o/TsWxFM6leERX5swIRzzYAtcFf1wNYZ8Mv/C1wMWlw5VwRdaDNHdEKJRgF4ZmTwuy
         9pkItyrh2KdLe7uZXFcfrrttfMsWCRR0KLNdfQ95PGlaDzhZ+fVqBi1GPghiNCVEpgg6
         aXAULiH3O9dP/PYrl2dxZqpKcf51Wxa9z808JEJfYq3KP2qQ7khRGqAKrj+8RJ7rti1+
         b6uGcgAqYQXnblK4CwAz9UWSxCEhF3rA6hPYeBy/CAwHuGhFyx92h1v1RDhS/YPqQw+h
         r2O9lslawo09yGlURLUdFtxBsoHmCqXI1/zmuDelUMxJRFj8bPBZrWsHgM+asXEnN2cq
         8dKg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707920430; x=1708525230;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=rQOqQYiwVvevA4SrtadQQkQhVXKeL4AfeaO3c9Od7bg=;
        b=G14Q+rCPFnd1syA2ebQyCPzvY9GX7SxpzwT//LdtEM8kUOXMMXF22DggXJ2VNol1kC
         mdj/4FQiCLxM7MLHZdO0EteRfuvuh1PvtOCwKmDrgxfo2r58h7ual3tIIZDLbpMHqNgq
         2F3KAr1wgbtwSd8QQTAG2FGTawxQHn6/jmekAYNf5MdlXWOo3LsreBOHL1neAf0DcNgk
         jtJKa797fuCHZ66Po5MRwU5p9gAK2qrPkHuj7VwVafJK4lL3reuYZoPhXrVEH5G2yrZX
         2To4TegO0NXbAS3J5k+01Eieh4uDKzEeBNg2D39LqQCCgU8JS5Mgr7EJTzO5J8ETvOjk
         s/sg==
X-Forwarded-Encrypted: i=1; AJvYcCXArkL0ljF0zC3N4cwRbtLqeDBF9DDf7YCvQFnuQnGg7tEqjQcj/6Ru9SM93WVcUkrZTbMdcIGgz3X+UIMnfixLgV1393KMlZqQx+0QVuw=
X-Gm-Message-State: AOJu0YyWHvTVDfYTaI19GxU7mhcYXU+oV135M0qc3i7WksWVVcmrCxg0
	tTZtsPgY5EbZ/7x+CJGm1Z3EUHwPMpV/qlleXRDKDVEyqxzSo8vxrnifmkYrB2XUpEaCVBIE9AN
	5zTC844xhyvEG/FDqx1eeQqj8UQ43sJFI9VG7cMdYXQh7AzmOWiE=
X-Google-Smtp-Source: AGHT+IEy10UAcKZPzraZU2Omj/AkivtWdzVsN1J9VieFBGFIuiXzUlRy2hsXf2UkVocfOuvrm3TVzm6lDqE4s+0v5xg=
X-Received: by 2002:aa7:c1d8:0:b0:561:f7d9:d77e with SMTP id
 d24-20020aa7c1d8000000b00561f7d9d77emr1877818edp.10.1707920429896; Wed, 14
 Feb 2024 06:20:29 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-15-carlo.nonato@minervasys.tech> <a66e3131-1de1-49cb-9b26-5fb1fd77bc20@suse.com>
 <CAG+AhRWv5MazB-txmPkcb3CAuWFQvM97HH3D-_bn6r3kdB360A@mail.gmail.com> <0feb6a90-2c3e-489f-90de-b4b2979997dd@suse.com>
In-Reply-To: <0feb6a90-2c3e-489f-90de-b4b2979997dd@suse.com>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Wed, 14 Feb 2024 15:20:18 +0100
Message-ID: <CAG+AhRW-MGkGcXUA5=TtiMFqvDpsYmdGN1d2nCPfCPqXNKA9Gw@mail.gmail.com>
Subject: Re: [PATCH v6 14/15] xen/arm: add cache coloring support for Xen
To: Jan Beulich <jbeulich@suse.com>
Cc: andrea.bastoni@minervasys.tech, 
	Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, 
	Marco Solieri <marco.solieri@minervasys.tech>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jan,

On Wed, Feb 14, 2024 at 8:55=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 13.02.2024 18:29, Carlo Nonato wrote:
> > On Tue, Feb 13, 2024 at 4:25=E2=80=AFPM Jan Beulich <jbeulich@suse.com>=
 wrote:
> >> On 29.01.2024 18:18, Carlo Nonato wrote:
> >>> @@ -218,9 +230,44 @@ static void xen_pt_enforce_wnx(void)
> >>> --- a/xen/common/llc-coloring.c
> >>> +++ b/xen/common/llc-coloring.c
> >>> @@ -29,6 +29,8 @@ static unsigned int __ro_after_init xen_num_colors;
> >>>
> >>>  #define mfn_color_mask              (max_nr_colors - 1)
> >>>  #define mfn_to_color(mfn)           (mfn_x(mfn) & mfn_color_mask)
> >>> +#define mfn_set_color(mfn, color)   (_mfn((mfn_x(mfn) & ~mfn_color_m=
ask) | \
> >>> +                                     (color)))
> >>
> >> Nit: The wrapped line wants further indenting, such that it becomes
> >> immediately clear what parentheses are still open. Alternatively:
> >>
> >> #define mfn_set_color(mfn, color) \
> >>     (_mfn((mfn_x(mfn) & ~mfn_color_mask) | (color)))
> >>
> >> This is certainly an "interesting" construct: I, for one, wouldn't exp=
ect
> >> that setting the color actually changes the MFN.
> >
> > Would something like mfn_with_color() be a better name? I need somethin=
g that
> > expresses clearly that something will be returned. Maybe colored_mfn() =
is even
> > better?
>
> The latter reads as if it was a predicate, not a transformation. The form=
er
> or get_mfn_with_color() _may_ be okay. Without the get_ it's still a litt=
le
> predicate-like, while the get_ itself somewhat collides with other uses o=
f
> that prefix, specifically e.g. get_page{,_type}(). So I'm still not overl=
y
> happy, yet e.g. mfn_from_mfn_and_color() feels clumsy to me.

For the moment get_mfn_with_color() seems the best option.

> >>> --- a/xen/include/xen/llc-coloring.h
> >>> +++ b/xen/include/xen/llc-coloring.h
> >>> @@ -24,6 +24,17 @@ static inline void domain_llc_coloring_free(struct=
 domain *d) {}
> >>>  static inline void domain_dump_llc_colors(const struct domain *d) {}
> >>>  #endif
> >>>
> >>> +/**
> >>> + * Iterate over each Xen mfn in the colored space.
> >>> + * @mfn:    the current mfn. The first non colored mfn must be provi=
ded as the
> >>> + *          starting point.
> >>> + * @i:      loop index.
> >>> + */
> >>> +#define for_each_xen_colored_mfn(mfn, i)        \
> >>> +    for ( i =3D 0, mfn =3D xen_colored_mfn(mfn);    \
> >>> +          i < (_end - _start) >> PAGE_SHIFT;    \
> >>> +          i++, mfn =3D xen_colored_mfn(mfn_add(mfn, 1)) )
> >>
> >> While the comment mentions it, I still consider it problematic that
> >> - unlike other for_each_* constructs we have - this requires one of
> >> the iteration variables to be set up front. Question is why it needs
> >> to be that way: Isn't it the MFN underlying _start which you mean to
> >> start from?
> >
> > As said above, this is used also when page tables setup isn't complete
> > so I can't easily find the first MFN.
>
> Did you consider making the initial value a macro parameter then?

Yes, this is better.

Thanks.

> Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 14:31:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 14:31:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680847.1059002 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGI4-00068W-Rt; Wed, 14 Feb 2024 14:31:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680847.1059002; Wed, 14 Feb 2024 14:31:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGI4-00068P-PL; Wed, 14 Feb 2024 14:31:12 +0000
Received: by outflank-mailman (input) for mailman id 680847;
 Wed, 14 Feb 2024 14:31:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raGI3-00068J-IR
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 14:31:11 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b1c9bd60-cb45-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 15:31:09 +0100 (CET)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2d109e82bd0so24605351fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 06:31:09 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 dd18-20020a0560001e9200b0033b3d28ef1fsm12486651wrb.65.2024.02.14.06.31.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 06:31:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1c9bd60-cb45-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707921069; x=1708525869; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=34Uxz0mARJ0/GDBpjpC+NYRes3/sgTgBpejJlKX0Dfs=;
        b=RQaFTE3m9Jgb70pU0TEfX5guRwiOE2h15YYk8RzCRHr8w1Le2Xmsyxy0R+N9+tDULY
         X0t3x+ioiHGNks7eSYzAv0XSOCPZ7a/9X/UGQINadhk0otsHP8+6+6jV84EXpO18p9RU
         7unkP7+uU6YRJeckloNHnUO4uHtrGQzLuR1wtrE475RHVr3sYh6+v6TuaDYH4MY/zJc4
         HndLaCpFFWFjvu8yH+GeKoLZ5G9iu1bh2cpefnm9XXR91VqQluzjrwqt76JNXeGmsYUw
         ip8rQttagE1gnjfXkXXtgH6zq5qJxI7KRAumqXZwRKlMxmhdjDjp9lx3PqUwetovEOfl
         Vo2w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707921069; x=1708525869;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=34Uxz0mARJ0/GDBpjpC+NYRes3/sgTgBpejJlKX0Dfs=;
        b=AY4dd+G0gf89hWib0pHwsbAXF+SZxL1QtfapgeslFLWQiXscKMauKFDbM+/PwGpdc5
         jmpiUo1XQKRczt0N0Bt8Bf1zqsPlvkGW6xqCnuupYGvkq/cDEKx7Tq4royXAUU77iOe3
         pNOxuGCQRhAq6IuuiBV9wbrcwO1c8PacxBJFLxIfyKbt6Cw674XuOKi4KDw/MFz9dY6V
         LH8DJ/UPyBDS/V4L+sMB0mGzsvA277vUZsOSggtY4hypM403s7T5hdQ3ypAwA0v2r7kL
         sn6Mj4rY1wIShiPBU9AqvUrywBjylm75Jw8vjc70ZZoZr+BVSZTHhFQEG1BovYYFjVi/
         6wVA==
X-Forwarded-Encrypted: i=1; AJvYcCWmYR4GU9BKiMgFMgmCPJYErpG982ipzCCrsL2SDJWaiScsxt3wL9/1NxIfnczCitv+R+SHNI6Y/g1BSUhKdPsfChA0ZXXH4AlKOoBTJjw=
X-Gm-Message-State: AOJu0Yx1OCDbB10ybbdwOSOnlx96v40F/MgWxXck4yxzeFjgG0BGDe3g
	6Lgv9B54sRBav2rUzYecaSbGzUStQ4adr2KVLFkWO6sg/Q133bqJPUv5gBSwXQ==
X-Google-Smtp-Source: AGHT+IEkeU9GnF959IYjUG8fah4p9oMZwLWLxWxecKhuf+Gv8grjypqmmcLconweJCD9q4YJjz7JyA==
X-Received: by 2002:a2e:a601:0:b0:2d1:1cd3:7dfc with SMTP id v1-20020a2ea601000000b002d11cd37dfcmr1167736ljp.41.1707921068705;
        Wed, 14 Feb 2024 06:31:08 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUaeW4XDJFW2NHtOECwidw1AWLu7uZp/unKchGSaF8uu1FKKeO5fcRnMvX/mpnz5AXS/rY9ZLRsP0otMDVTDkjY5VICSQzUKf482m+zX+ek1mUmCKJ0kUNX0d8PCQ/4teyGa/n8hvfzB+Q=
Message-ID: <898fe19c-1f6c-4efe-a4bd-ac83c61757a2@suse.com>
Date: Wed, 14 Feb 2024 15:31:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/5] iommu/x86: use full addresses internally for the
 IVMD/RMRR range checks
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Paul Durrant <paul@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 xen-devel@lists.xenproject.org
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-4-roger.pau@citrix.com>
 <16de2e3e-7807-4754-852b-e486c1fde95c@suse.com> <ZczIm37J3WBQPYsx@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZczIm37J3WBQPYsx@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 15:05, Roger Pau Monné wrote:
> On Wed, Feb 14, 2024 at 02:29:35PM +0100, Jan Beulich wrote:
>> On 14.02.2024 11:37, Roger Pau Monne wrote:
>>> Adjust the code in the checker to use full addresses rather than frame numbers,
>>> as it's only page_get_ram_type() that requires an mfn parameter.
>>>
>>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>
>> In this very shape I'd like to leave this to Paul or Andrew: I'm not outright
>> opposed, but I think this either goes too far (most type-safety being lost
>> again), or not far enough (both callers convert addresses to MFNs, just for
>> them to be converted back here).
> 
> I don't have a strong opinion, given this would you like me to adjust
> 4/5 so it can applied without this patch?

That or we first wait what Paul/Andrew think.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 14:32:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 14:32:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680851.1059013 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGJk-0006vg-6y; Wed, 14 Feb 2024 14:32:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680851.1059013; Wed, 14 Feb 2024 14:32:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGJk-0006vZ-3w; Wed, 14 Feb 2024 14:32:56 +0000
Received: by outflank-mailman (input) for mailman id 680851;
 Wed, 14 Feb 2024 14:32:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raGJj-0006v6-6a
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 14:32:55 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id eff7622f-cb45-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 15:32:53 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d1094b549cso24090701fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 06:32:53 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 o19-20020a05600c379300b00411c3c2fc55sm2077941wmr.45.2024.02.14.06.32.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 06:32:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eff7622f-cb45-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707921173; x=1708525973; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=mfAKw+F2nfgxWcvONfxDaoSEpMBCqgHxAXhJu6BPULI=;
        b=EJy7Dcwjqm0o5D5A/6m90140WMif6lRvvJd0ShE+DB+8z+IREbJ5LQfaQ6JlWRyYZl
         dmIsyXZZSOgp+iFW91nA/EPA/pOgXPab16xl5ch0Lk4ZO9s48mmXM2qG9tniC0xxjq+i
         3/TzQ4oafJtZm+Hat2QHniPh553ysVygQSKZ4NQZNJjAxxaSuJ4ijrYhlSvyvPO1IEV3
         V4Ym8gf0qSHUl3bxLJCIpzjPh3m0tEKv+HrgWNaoip64DdZ1t/O4NRiTp9kjDoUS74aQ
         LwdImonWu6nPMBEZbSm3uNJJerQt7s11igk4ACDtRZkYX5kKuuKJYMG+1rONIF1HZHgZ
         IavQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707921173; x=1708525973;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=mfAKw+F2nfgxWcvONfxDaoSEpMBCqgHxAXhJu6BPULI=;
        b=U1ov6KSui45co4ZhKA8NDBcjVIxKEY0KKGMqApDWQVNtoZcvnefrQPSta/DBefstWh
         wqq1MiHnf5hXe/xbQM5pvrwtnWi3CXTGAL5wrT5i80EKwZu2ytoeO/pz+d5/yjrhTYT3
         LIEFQEGnkQx42Kz2OEQemAXxWtwl2teVL2djOO8pTM1cwtEpdqjohHp6ShremvEvEc3X
         9bE1M5/e29VFlEJAxC21g2zUc3Y/aweVWBvzwUsdoYlHMWsCdVebdEptO8TVUl09AB2/
         WxSO/lSTBLGf1GilgP+mVESDKquSDwgV1F+/pFzSnV0EkpqqCpUkxsOOZRY+lCT0H84x
         MInw==
X-Forwarded-Encrypted: i=1; AJvYcCX2DCQ4G1Ih9WBBlVVRks61GqcGjtesvrzfh1S3AsLjX+ssCE0AE/Q8lINFOIq6xqgAPmmcqWqbRKkjFxMFAtxMct5P+Wy0Edta4IMHpew=
X-Gm-Message-State: AOJu0YwQMNWPDjguBZ/LnJATK7LETAT98e9z1FQ7s0DHE8WF+HzvKR7+
	9zLvMP98c+J/zjfVYL+LucapHz8GXjFUJo3eIKKN9v29TT4XP37P3GANlk7r7Q==
X-Google-Smtp-Source: AGHT+IEDx2kqRtO2K23EIDLdaM1+lH4hgB8CyCDwInVk7b+FbVQagq0UpR2TzzDZ5PJJ8zahB5/41A==
X-Received: by 2002:a2e:8552:0:b0:2d0:f641:3c09 with SMTP id u18-20020a2e8552000000b002d0f6413c09mr2245926ljj.30.1707921173085;
        Wed, 14 Feb 2024 06:32:53 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVAI6a9zaDAzo/WZ9ZbQdvDlqzxZOWymY1r8K+krM0OoC+C4NB0LXX1elaCnjV60+YaOPqdsyQhraR8FiHax76JvcXkoMsV8dkfBSqqztpP5zljVuadpvusw1SWTJk77oyU10vOyKHRjC5eOpx+JKCLjlSXJ0AzTD7judwc21YhvpHTH7KY/3r6qPG9PCZYswM0GCZ5iBkUTWsSzJfXEZWujajLbiozYaNtTmjNYMtcD78tB59jedpsqdhoncIMP9oMZoNcLDpWhc0S
Message-ID: <825b18b4-ec7c-4bd1-8408-b63491fcfaaa@suse.com>
Date: Wed, 14 Feb 2024 15:32:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 5/5] mm: add the __must_check attribute to
 {gfn,mfn,dfn}_add()
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Paul Durrant <paul@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240214103741.16189-6-roger.pau@citrix.com>
 <20240214141137.17787-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214141137.17787-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 15:11, Roger Pau Monne wrote:
> It's not obvious from just the function name whether the incremented value will
> be stored in the parameter, or returned to the caller.  That has leads to bugs
> in the past as callers may assume the incremented value is stored in the
> parameter.
> 
> Add the __must_check attribute to the function to easily spot callers that
> don't consume the returned value, which signals an error in the caller logic.
> 
> No functional change intended.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Acked-by: Julien Grall <jgrall@amazon.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Wed Feb 14 14:34:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 14:34:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680857.1059024 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGLa-0007Vc-JI; Wed, 14 Feb 2024 14:34:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680857.1059024; Wed, 14 Feb 2024 14:34:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGLa-0007VT-Er; Wed, 14 Feb 2024 14:34:50 +0000
Received: by outflank-mailman (input) for mailman id 680857;
 Wed, 14 Feb 2024 14:34:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hroU=JX=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1raGLZ-0007VN-Pd
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 14:34:49 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 341d25fd-cb46-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 15:34:47 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a3d484a58f6so93067166b.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 06:34:47 -0800 (PST)
Received: from localhost.localdomain ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 s6-20020a17090699c600b00a3ced84c1e5sm1998360ejn.169.2024.02.14.06.34.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 06:34:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 341d25fd-cb46-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707921286; x=1708526086; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=JSeGZrKW45NOij+0OCKsMaecydGRYMcVxuo54PO/m0Q=;
        b=R3TEKUSMc5Nsj2u4etKDPZPE9pSn0j78XyerQFDrTsXf4OE39tDEQr2tsJw0oWlhM5
         5SjwkaOlRtOflSXUgn0Olu8L+S1d0YF1/mGpDVWeqfyC3AB/KN0ShKOE7AX8f0Mo1vzA
         5xjVjUr4WtEftwInECBZsZmDvbBinc++nfotg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707921286; x=1708526086;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=JSeGZrKW45NOij+0OCKsMaecydGRYMcVxuo54PO/m0Q=;
        b=ThXhQa4xk5IMREvMseePKaQPbtODu5fqVIPue3JcJNe2nnbBoSS64Wjyj0qcKvBK5z
         aiIZA3BAzf59lHnMxletcFKeFZvt0RS8g6dmNri5AjqT2VWxKZ6eBte0Tb7tpVcDkK/t
         CeUecgWBWB+hYCo56peo00XfTjAIgvQZxl7YV2eFa3iJDmSQWs3/cwDyq1jVjySwjFwI
         Yaw4FoQdpjWRx6KHnJeEQG4hN3t7fWRZojVF4dZhIiKwporVKnWeaVvXxVtbei7GfxOH
         yaSFnekzaOAImphkVrt/KUFC+5ocXpfINtdf6JY9wdOVj6hu/mAVarHN3es56d8IjOJq
         GQxA==
X-Gm-Message-State: AOJu0YzcILT6wcYFbSlPIVTGYUYN056+CgtxEP+IKBiEMugHD8QdPnrE
	b4bT2+ToPY2T1STgD4C64p0bzsdLRKPgngbE1A9fCYG9WnZxJO0a+Dy5HCvUzJKSWrZa1kYKyrH
	r
X-Google-Smtp-Source: AGHT+IGgKUN0QpGnEXykWKM7akyMU9FS+TnDZ8F136CPjK7aIkwLnADrTk4VAwWJkZpy1d7mdAQl1g==
X-Received: by 2002:a17:906:1cd7:b0:a3c:e8f6:a399 with SMTP id i23-20020a1709061cd700b00a3ce8f6a399mr1753239ejh.29.1707921286594;
        Wed, 14 Feb 2024 06:34:46 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXACRwUnl35To02OMat2HxDZyplCK0waE6I2DmrnvECw7kSBKhCP84nl1lR4vwW+WV+PkKSYs/l87OOlAgdPojU69cpaOFv6L1JCGDcCa4ZniD3N3RuINBLeHEV2pAMD0CZzi047bwjzautDMZkkfWm5ert7gTCQkKpQtYoT6+Jlto5qHLpGLHoOdIJ52GkwjfSPQeNpZqJjMVCfniYH8kHHMUb8hd3S92gDA==
From: Anthony PERARD <anthony.perard@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH] build: Replace `which` with `command -v`
Date: Wed, 14 Feb 2024 14:34:11 +0000
Message-Id: <20240214143411.37957-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The `which` command is not standard, may not exist on the build host,
or may not behave as expected. It is recommanded to use `command -v`
to find out if a command exist and have it's path, and it's part of a
POSIX shell standard.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 21832d6402..767e47d6c7 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -25,8 +25,8 @@ export XEN_BUILD_HOST	:= $(shell hostname)
 endif
 
 # Best effort attempt to find a python interpreter, defaulting to Python 3 if
-# available.  Fall back to just `python` if `which` is nowhere to be found.
-PYTHON_INTERPRETER	:= $(word 1,$(shell which python3 python python2 2>/dev/null) python)
+# available.  Fall back to just `python`.
+PYTHON_INTERPRETER	:= $(word 1,$(shell command -v python3 || command -v python || command -v python2) python)
 export PYTHON		?= $(PYTHON_INTERPRETER)
 
 export CHECKPOLICY	?= checkpolicy
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:02:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:02:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680914.1059229 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGmN-0006tT-Tf; Wed, 14 Feb 2024 15:02:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680914.1059229; Wed, 14 Feb 2024 15:02:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGmN-0006tM-R3; Wed, 14 Feb 2024 15:02:31 +0000
Received: by outflank-mailman (input) for mailman id 680914;
 Wed, 14 Feb 2024 15:02:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raGmN-0006sm-9e
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:02:31 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 12327a8e-cb4a-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 16:02:28 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-5114b2b3b73so6577262e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 07:02:28 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 oj11-20020a056214440b00b0068c71b78980sm2325105qvb.118.2024.02.14.07.02.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 07:02:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12327a8e-cb4a-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707922948; x=1708527748; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=BuGYPWEJOUTaH5rf6DAQz91oXgKrMbqT0YkEhfKJGuc=;
        b=SXTo5CRM5BgfBuEXKcn/qWZ1gvRq5XOqnjXuURulUCwXwMfivdcoIRvz0IUtoJutYX
         04UMnh9pjIHFsjWDhaXdrs4Hi0lj9KMHd78fIhOQuabi8s+fQ86XF+C4NSI2Xng7wfkp
         aJeK2ei0MKdK4thpp5Mbp1iI6XIILz1/fpLTw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707922948; x=1708527748;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=BuGYPWEJOUTaH5rf6DAQz91oXgKrMbqT0YkEhfKJGuc=;
        b=ivO45OCBKtWPp67+9ITAzHCygiqZK6iQeielpek91jQmny2wC4UkV+eSzAQE7Jybnh
         W2jFdNoz7JbNCbiYKJDHv2epEGy5AGtsWdYD7b7LnmQvuTc0dwIDK4nE16mW0bO8l0pC
         fgi5ivj5GG78To0muUbvHJbX5X26GlQ4l0d/Fd4Ug7qQJaohvVsYUCe7N2J6OrbdCeRp
         +p+u8Oo6UnKNLCKJmWXMzOl2pVp2zPQvbtdYiojM27FCkOR/G3qWyJG70o0h1RPTvSYg
         VrqxpePwYlJSdcg6QyMwslUJIxo0XPhHaHDUVHRN+j2VDw27R0xtMIomE7e3A2W2cuWE
         ye9A==
X-Gm-Message-State: AOJu0Yx1DwGoPtGPHTL+CDJLLnyjcUkRYbdicPGTALh8D1G0z2s2+dcc
	+7ysYmrSHazwmOpxs6fK8CVz8pKIQv/Ram3HVC84IabPNFjXaYhXhfX9+qIEQdM=
X-Google-Smtp-Source: AGHT+IHNFmecXRH2jOezFs+xLBqoWq0zVXTtUbqa+I4cM5vGILI35YTnPjzxTvvMRnrpve4xr8FTSw==
X-Received: by 2002:a05:6512:1391:b0:511:a2ed:f6c5 with SMTP id fc17-20020a056512139100b00511a2edf6c5mr2382930lfb.18.1707922948343;
        Wed, 14 Feb 2024 07:02:28 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXVYYtCtrG9pGYKKSvFMKWFE+/qIvEICmzua5q4ESHeNrm9MWt1QlnrEny9trbfH7G1y+aoz5TC5VgiBOAPKnbiwgqyrwwfD+ApOti4QfTklBDCJuXVdjgjj43SWfgHipjZCeLsieff+w==
Date: Wed, 14 Feb 2024 16:02:26 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v2] Reduce assembly code size of exception entry points
Message-ID: <ZczWAg-zmoLXiWAr@macbook>
References: <20240214103558.13655-1-frediano.ziglio@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240214103558.13655-1-frediano.ziglio@cloud.com>

On Wed, Feb 14, 2024 at 10:35:58AM +0000, Frediano Ziglio wrote:
> We just pushed a 8-bytes zero and exception constants are
> small so we can just write a single byte saving 3 bytes for
> instruction.
> With ENDBR64 this reduces the size of many entry points from 32 to
> 16 bytes (due to alignment).
> Similar code is already used in autogen_stubs.

Will using movb instead of movl have any performance impact?  I don't
think we should trade speed for code size, so this needs to be
mentioned in the commit message.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:08:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:08:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680954.1059375 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGrz-0000m9-74; Wed, 14 Feb 2024 15:08:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680954.1059375; Wed, 14 Feb 2024 15:08:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGrz-0000m2-3d; Wed, 14 Feb 2024 15:08:19 +0000
Received: by outflank-mailman (input) for mailman id 680954;
 Wed, 14 Feb 2024 15:08:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raGrx-0000ls-ED
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:08:17 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e09a6d5b-cb4a-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 16:08:15 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d107900457so27149241fa.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 07:08:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m22-20020a7bcb96000000b0040fafd84095sm2130736wmi.41.2024.02.14.07.08.13
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 07:08:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e09a6d5b-cb4a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707923295; x=1708528095; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IPp1XQ7LPeZ5bV2atiaBj+y5LoeQvJFr62RZlyniSWY=;
        b=Kk0br5/V8uNO7+CUoUll7GQQ174rpKfQDcneqVI1GOrbpvVbFvJVznDSCaD4j8HwAO
         xk33Wf1j+D4bAYoyXqqLYhv2YvDHooBqx1s+oL+Y8kns2+eeAW67PDH9rfHzostGai6m
         P0yqpDUc78nMD0pW17w7G3BPe7b9YPX0PwUEGnBppBLyypYAl9o7IB97ECq7TgHRUduM
         Isn3jtMLSvjWrSkXFAVm4oLy0n3Ylxs1/F19pTrwm8b8Kmo1oISe1XO52vTAZsQE/NgW
         20a1fgNqT5HSAYVM8zstinZNgDIQnx1y93I6285iX1lA2MoObndDbcorFzkQxjVf8klq
         Oskg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707923295; x=1708528095;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IPp1XQ7LPeZ5bV2atiaBj+y5LoeQvJFr62RZlyniSWY=;
        b=ly1xLtU1CTZpkoLNi2jgLxzfwwIDGuFkGiEyNhUhyQxgSIsiuzUbVOOGlQP5CuP80u
         233A11wRdoj5MxSyPk87bCkqp7G/SeihsjJGAQqxw+Nt8gVyEHH9uzGBgJiYEZR0mpbp
         G/uG8YRrvvcXpfjwhcC6+vBUm7Cu7THufKtruFC8qf6RCUIAM02vSiDpaEanTpKhhTXK
         fUU05qSURn2LsqsdMxQ4PUsKJ5vVPrWUr42qWCh5j4WnzUkFqelLgnHKuPfzHwGHySMg
         MmL8LA7mOJKlO/YDk2bw9FX+4dse40pi/6c9tD/DAfVNeO6XmUSd+4MoVeHBi0KZpoe0
         gm/w==
X-Gm-Message-State: AOJu0YyWXv72IHXTfHz9wok1K0yM+usL67KMH4T8ue2EKjyivCiDFGt0
	p/MhET4gWDpmOt3rzFXycBbQkECOS1QpUX/g7MsH3EqhS3AEFf3XZGaWEuwGpQ==
X-Google-Smtp-Source: AGHT+IFkQd8efQKrrrO0G5mJ5ipb1YMZqMoY4w2jgGgWks+8Vbfs9ELcrurcGm2884ewN5XzOcli2w==
X-Received: by 2002:a05:651c:198a:b0:2d1:1d89:347a with SMTP id bx10-20020a05651c198a00b002d11d89347amr1266023ljb.17.1707923294717;
        Wed, 14 Feb 2024 07:08:14 -0800 (PST)
Message-ID: <ce3ae020-92b2-44fd-8055-694e374f68a9@suse.com>
Date: Wed, 14 Feb 2024 16:08:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] Reduce assembly code size of exception entry points
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 Frediano Ziglio <frediano.ziglio@cloud.com>
References: <20240214103558.13655-1-frediano.ziglio@cloud.com>
 <ZczWAg-zmoLXiWAr@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZczWAg-zmoLXiWAr@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 16:02, Roger Pau Monné wrote:
> On Wed, Feb 14, 2024 at 10:35:58AM +0000, Frediano Ziglio wrote:
>> We just pushed a 8-bytes zero and exception constants are
>> small so we can just write a single byte saving 3 bytes for
>> instruction.
>> With ENDBR64 this reduces the size of many entry points from 32 to
>> 16 bytes (due to alignment).
>> Similar code is already used in autogen_stubs.
> 
> Will using movb instead of movl have any performance impact?  I don't
> think we should trade speed for code size, so this needs to be
> mentioned in the commit message.

That's really what the last sentence is about (it could have been said
more explicitly though): If doing so on interrupt paths is fine, it
ought to be fine on exception paths as well. Plus, no matter what, we
have two overlapping stores in every one of these places. But yes,
their sizes may still be relevant to the overall result.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:13:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:13:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680966.1059421 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGwY-0002ix-Ab; Wed, 14 Feb 2024 15:13:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680966.1059421; Wed, 14 Feb 2024 15:13:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raGwY-0002iq-7c; Wed, 14 Feb 2024 15:13:02 +0000
Received: by outflank-mailman (input) for mailman id 680966;
 Wed, 14 Feb 2024 15:13:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raGwX-0002ic-25
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:13:01 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8a53ea85-cb4b-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 16:12:59 +0100 (CET)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2d0a4e8444dso63364111fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 07:12:59 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 hn32-20020a05600ca3a000b00410ebcf8180sm2206447wmb.43.2024.02.14.07.12.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 07:12:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8a53ea85-cb4b-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707923579; x=1708528379; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SmBQtWqYk9CvBmizB1i7LwVJxy/3yjASVs1Xa7mYRYo=;
        b=Q7yEbJSDvxJ8eZ151M2V6f1uY8KzqJXsIe3CQmRBThHtcUb44Tra1fXTuJtcXdZe7F
         HEeVdkJqq80js02BPiTbElWHXoRAONORVkzt5eJp9nhbWxsnFKNGUy/ZL6+YwuaCuU+d
         hqg/pNorY98PN+s+7nNfAoMeHNB5S4JY9idsHVhVo4InkgOnjHu6Qvv1PoqhKktwSAjy
         3RSg+WtTjxN0wBgCZeynH1n/VW/Jj+BhmyFf0PS/D7iK1l6hCje8Ybl9PbfJGxiJFjOu
         fp0xG3s4srJH9qi0VwKdUXI6T9gCCXL/cCGderLieNEFpus+/pIvCLSR92mv7Xf0ggS+
         drIA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707923579; x=1708528379;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SmBQtWqYk9CvBmizB1i7LwVJxy/3yjASVs1Xa7mYRYo=;
        b=W270bQSB1FDxLESnMHSYf7R/u0+j+rPaI6EB6UQWh4zToX+uNI69IgvCt9VLNmJ7gX
         tKjJDbeQLT40QO0KJKJnoDDfGhpFcjUR3o3evYqRIWlAjJTF7eiyNN0prNYTFks6D261
         M1Pj0mvnEpyqMObCbiY/F4OApT9hWyQPYkICpMX0Wdz2pRTQv+3bAVN7mgUUmlo2Zf5Y
         75Bw2SE+2x3snnE6TS00j5tHN08oOViuIMT3kE4As+eJOfoP1J8aKDu1mnjdpEGOoE7Z
         K0jNYEU0DOzoPR4MZBd2dmYgPzxiv0RBIuMg0gF4uZ7V0HuvVXasbht+6Fh5gPzlFUsL
         fTZw==
X-Forwarded-Encrypted: i=1; AJvYcCUXF85wIExMJK/CYKZAd0/mT9L1hPJkrF/bbEQV/weBu/E9mBBBNdxmj7DGZgaWM+cMB5T7F6mNnf4giPpw1pW1pCQPfSgAmdkYgNG8RUg=
X-Gm-Message-State: AOJu0YynOojG3URMmXG5Y2/0JIQlE2XmCiizmXpC8uls47lE9Sz5AV+V
	PdFbdWppBX8uf5TOkBpoF+wrdB/rvUTvBSYJLnHzHx4xyJwcF+TPytalAR0L3w==
X-Google-Smtp-Source: AGHT+IEM3+DZ1fCRA+5ZVBlBnelcscSOrVF8Zji8HWdFkqEXOcyMkelxT+fivlwQoZ/vBVpnfCv6lA==
X-Received: by 2002:a05:651c:212:b0:2d0:c490:affe with SMTP id y18-20020a05651c021200b002d0c490affemr2240015ljn.19.1707923579513;
        Wed, 14 Feb 2024 07:12:59 -0800 (PST)
Message-ID: <560ff594-97fd-4cee-a9e4-70d153bf3a53@suse.com>
Date: Wed, 14 Feb 2024 16:12:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] build: Replace `which` with `command -v`
Content-Language: en-US
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240214143411.37957-1-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214143411.37957-1-anthony.perard@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.02.2024 15:34, Anthony PERARD wrote:
> The `which` command is not standard, may not exist on the build host,
> or may not behave as expected. It is recommanded to use `command -v`
> to find out if a command exist and have it's path, and it's part of a
> POSIX shell standard.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
>  xen/Makefile | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/Makefile b/xen/Makefile
> index 21832d6402..767e47d6c7 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -25,8 +25,8 @@ export XEN_BUILD_HOST	:= $(shell hostname)
>  endif
>  
>  # Best effort attempt to find a python interpreter, defaulting to Python 3 if
> -# available.  Fall back to just `python` if `which` is nowhere to be found.
> -PYTHON_INTERPRETER	:= $(word 1,$(shell which python3 python python2 2>/dev/null) python)
> +# available.  Fall back to just `python`.
> +PYTHON_INTERPRETER	:= $(word 1,$(shell command -v python3 || command -v python || command -v python2) python)
>  export PYTHON		?= $(PYTHON_INTERPRETER)

Shouldn't we go farther with this and also switch the use in
xen/build.mk (which looks to be the only other one I can find under
xen/ and config/)?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:21:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:21:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680984.1059467 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raH4x-0005NR-FU; Wed, 14 Feb 2024 15:21:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680984.1059467; Wed, 14 Feb 2024 15:21:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raH4x-0005NK-Ce; Wed, 14 Feb 2024 15:21:43 +0000
Received: by outflank-mailman (input) for mailman id 680984;
 Wed, 14 Feb 2024 15:21:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raH4v-0005NE-K4
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:21:41 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bfd2886d-cb4c-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 16:21:39 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-40efcb37373so44274045e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 07:21:39 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l12-20020a1c790c000000b00410bc833c2csm1596919wme.1.2024.02.14.07.21.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 07:21:38 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfd2886d-cb4c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707924099; x=1708528899; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=5oyh52a4Z6LZhtZWfw+fMmAMgPOOv0etXEzm5yIXrbg=;
        b=MAfvTt8tQS7CWSTGF3naHFTPj367adXokWHYLYxFMhUOmMEj/CECHjajtUMZgyAEUt
         gYE5eS1jtuJNvOTmWG2YslsP+zXnWKu6NpB7DMfRP+w5LQCADI3ZKdPyH3okLZtxObPW
         Zej/9cstPFEvt8+Ha2iBDqDprOZ9plTSL/kQAdD8lFAw7QC65FAkWJWhz9UkU2JvLhNa
         m0ckko/g3D6I2g9jPPJFDe0RO6t0OQKDpvgoCbTltFecqEjfAKJuh03JsmXpPp2GXNCO
         rqkrg0KSmS0ppoRwiC4iNv1WYjjff9Ao/DMyC+sfbylvf84Kjn4/7L5/QDS3mfZadOLX
         IwJw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707924099; x=1708528899;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5oyh52a4Z6LZhtZWfw+fMmAMgPOOv0etXEzm5yIXrbg=;
        b=G+Gg1AjfqVV6SOeJ7TSOpzbOGhHDkqcCh4HUtBDnHImKEklLaxW12UjRW8U9zI/DtW
         demjIAuLng6OjvaGiYXiWwFaSx5E9dzwTvdv7GxqtqW/tnvBL62Os6t2DF7+F8wyFLSl
         +6U1V1zm8PLPgJ0Dyu+GxZtr7De6P7fs0OWZOxuIrH/5BQd2ZdeGDTP3zaSETYbetstf
         DKcmjGImHBqPuPa4CcM+maCvkCIPdsspnvkjbutnNo9mKcsQul0hAsGXhlo+v8uXYKfD
         PaDA/UYowK79fU0l7PGJ7rzQE7Sla+s2Zy9vGsqulnsN1sACLX4SCmfi8ep8b9qQomQi
         yQiQ==
X-Forwarded-Encrypted: i=1; AJvYcCV4aKVOZqiTx/zXmRrnrDZOT0Asc3C1VOZKOOv8E8byqgGhHRa8RI6B/TgxT3J0iqwr4PyA3hEhHDmWY025cyqPKmIRNhq6D4rrVBstL/M=
X-Gm-Message-State: AOJu0Yx0c0uPs6sJdrkFKrzemPdbi03cdrqFnySn26hyBkupds+mGPsr
	zUs6qVwrvXipuOhZqUcAHyDcfld920VD6p2z+217Thr+WME41axLul98kFn7Cw==
X-Google-Smtp-Source: AGHT+IHzO2QdQXydBdzZ3QSdjAVJIJ6Urlan047+Md4GAyZvGTlau+Puts0aj9uMNb7FnW4uDYOaoQ==
X-Received: by 2002:a05:600c:5198:b0:411:f03e:588b with SMTP id fa24-20020a05600c519800b00411f03e588bmr1504258wmb.0.1707924098732;
        Wed, 14 Feb 2024 07:21:38 -0800 (PST)
Message-ID: <5e0c9d51-b630-4f03-809a-19ad570b4fb4@suse.com>
Date: Wed, 14 Feb 2024 16:21:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] build: Replace `which` with `command -v`
Content-Language: en-US
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240214143411.37957-1-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214143411.37957-1-anthony.perard@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.02.2024 15:34, Anthony PERARD wrote:
> The `which` command is not standard, may not exist on the build host,
> or may not behave as expected. It is recommanded to use `command -v`
> to find out if a command exist and have it's path, and it's part of a
> POSIX shell standard.

Just to mention it: It hasn't always been. In IEEE Std 1003.1-2004 it was
still an optional thing. First mention of it being non-optional looks to
be in 1003.1-2008, which is slightly newer than the oldest gcc/binutils
we presently support. Still even on the oldest systems I'm still building
Xen on every once in a while, the command form is supported. So not an
objection, merely a request to be a little more precise in the description
of v2.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:21:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:21:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680986.1059476 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raH55-0005f7-NE; Wed, 14 Feb 2024 15:21:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680986.1059476; Wed, 14 Feb 2024 15:21:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raH55-0005f0-Ki; Wed, 14 Feb 2024 15:21:51 +0000
Received: by outflank-mailman (input) for mailman id 680986;
 Wed, 14 Feb 2024 15:21:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ltoc=JX=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1raH54-0005NE-Uf
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:21:51 +0000
Received: from wfout8-smtp.messagingengine.com
 (wfout8-smtp.messagingengine.com [64.147.123.151])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c49a2fbc-cb4c-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 16:21:48 +0100 (CET)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.48])
 by mailfout.west.internal (Postfix) with ESMTP id 5614F1C0007E;
 Wed, 14 Feb 2024 10:21:45 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute7.internal (MEProxy); Wed, 14 Feb 2024 10:21:46 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 14 Feb 2024 10:21:43 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c49a2fbc-cb4c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1707924104;
	 x=1708010504; bh=B9oNuKADKFiXlV8IGd26mv+ZZa9laBxiIk3H/egj2d0=; b=
	Eqq9Ue8/lwaNXBOQzmRoA3SIHBYCIjMYDqBOC61QNozvW+wYGEhmYwGoagysFV7L
	2Oyke7afD55W2SqYyQOVxT7TEYtIZPg9Z7jvdjJFeHpWUwpZrAKStMzaahsZJRzj
	m/rXWk7e524tTL5s5WR67nrv4EQXLQEEeCkYZRznBdusEnc4+gisP9J6uSFPpxrE
	/8KczKQTJoSYqR62/25m4mpiuDkMVDiwpphYRojo+sItrL95AEqV4F9QaxnZRGX0
	P3gy1wnUnwcj+SQ4KUaThBigjV3mA2Gv9SSjlVjuFbsmupR4/uI7B+22CAhAmCP1
	14KIJFmctTOOqBb/bzrnLQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1707924104; x=1708010504; bh=B9oNuKADKFiXlV8IGd26mv+ZZa9l
	aBxiIk3H/egj2d0=; b=xXPXzmXhfkKRVA/xPjBjdp62CSOm2Unwb6xQCYthT23a
	Kt6HUzbmoEePi03+FbssDokHk1ULmVkflqjgqWKRwS0xFaOC7/+Re9NP/w/c8L+U
	xa8MdbMVZCsXOFHBMCg+YfUU/Aa4a8bQC0ZczcOPGZ4Rs77lBrNRRELv+up/0Vnn
	U5fia9VpJmlDo35bjZMXd3u4sqCQCebtvSNpRBjDOMenZAaEQfDaCnmhK5JnePcL
	p8WMvNieGSU8u9HLS6cWV0oQaF/t4tFXsiBCZrbWjsJMN9XaaoMRusjEfg1Zxklk
	wcBxIe+p5K34L4qSB7R3W9mVt/XmiMy6ZAeiEfGV4A==
X-ME-Sender: <xms:iNrMZWBTogZ708axZ-ySUAGTtVfOUYqJDGtZogSwh02w08Q1YZ5M_A>
    <xme:iNrMZQjcew6phf7yyPi2FihawuhPhwj8TUz8D4JdfpaJjuYzOomVOoAdPOyTKgLXC
    MJUgWeFziWiRg>
X-ME-Received: <xmr:iNrMZZk3LDFaOnih7XSuJpi20v0nuASr2x69ydBjsdi3BBoV0I8hhoDGy1AEWWtF--vOw8bGebZ0dr0kJO2E6uS_UwEPmkXpaQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudejgdejiecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpeffhffvvefukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefgudel
    teefvefhfeehieetleeihfejhfeludevteetkeevtedtvdegueetfeejudenucevlhhush
    htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhes
    ihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:iNrMZUyOtt3CHxWfybPj5StYI22EheGNfthX7FxVLGewjrFi550HLQ>
    <xmx:iNrMZbSi5rz5ynHmZokAi4_q1-7WjfFjVRi34ZZNpVUdOm7f-S2EuA>
    <xmx:iNrMZfaaNztUhZphyyLAHNa7ywC40doJl_P52ofVHl4vwHqrNibyaA>
    <xmx:iNrMZUR4NEvjs5H7845_KouI9fwzQOlYi1ZYA-Wb80jMHImYjN78391EnKc>
Feedback-ID: i1568416f:Fastmail
Date: Wed, 14 Feb 2024 16:21:40 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH] build: Replace `which` with `command -v`
Message-ID: <ZczaheI7kIVrluHh@mail-itl>
References: <20240214143411.37957-1-anthony.perard@citrix.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="b2ha0LXU7iibO0rD"
Content-Disposition: inline
In-Reply-To: <20240214143411.37957-1-anthony.perard@citrix.com>


--b2ha0LXU7iibO0rD
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Wed, 14 Feb 2024 16:21:40 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH] build: Replace `which` with `command -v`

On Wed, Feb 14, 2024 at 02:34:11PM +0000, Anthony PERARD wrote:
> The `which` command is not standard, may not exist on the build host,
> or may not behave as expected. It is recommanded to use `command -v`
> to find out if a command exist and have it's path, and it's part of a
> POSIX shell standard.
>=20
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

This fixes build on fedora 39:

Tested-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com>

> ---
>  xen/Makefile | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>=20
> diff --git a/xen/Makefile b/xen/Makefile
> index 21832d6402..767e47d6c7 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -25,8 +25,8 @@ export XEN_BUILD_HOST	:=3D $(shell hostname)
>  endif
> =20
>  # Best effort attempt to find a python interpreter, defaulting to Python=
 3 if
> -# available.  Fall back to just `python` if `which` is nowhere to be fou=
nd.
> -PYTHON_INTERPRETER	:=3D $(word 1,$(shell which python3 python python2 2>=
/dev/null) python)
> +# available.  Fall back to just `python`.
> +PYTHON_INTERPRETER	:=3D $(word 1,$(shell command -v python3 || command -=
v python || command -v python2) python)
>  export PYTHON		?=3D $(PYTHON_INTERPRETER)
> =20
>  export CHECKPOLICY	?=3D checkpolicy
> --=20
> Anthony PERARD
>=20
>=20

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--b2ha0LXU7iibO0rD
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXM2oUACgkQ24/THMrX
1yyMcwgAkqphKADojZKY+urm5UcOwU44kgQgjSwAt+Ud1TVvanBliaW8eKUMGyjz
yB6I94Uzxj0eN2n9w+ohTbWDAQzS/CbngocKKHIR1gzSywv08GUSi96w8qlc8uZY
0sKWmZZ/kntBbnnlP9c8q/b5B2jEkx13LBTrBPfi0OlpTWd1KH9F9cOf3Rn+2VJt
c/8K4mbOd9j5OTJTGX7GoKRMR5Ft31myzZb4S+HU17MmW6XX+HA1+7GH9YDV/utd
5GNOaC3ikyJtoPy1miwkmLRMwrJHnY4egD6L2732iMVvYoKvAE6NaA9HyR7he5jD
MDowZkl90yM5zJMmZNpjskcTxSx+EQ==
=wmkT
-----END PGP SIGNATURE-----

--b2ha0LXU7iibO0rD--


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:29:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:29:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680998.1059494 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHC2-0006nh-H9; Wed, 14 Feb 2024 15:29:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680998.1059494; Wed, 14 Feb 2024 15:29:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHC2-0006na-EM; Wed, 14 Feb 2024 15:29:02 +0000
Received: by outflank-mailman (input) for mailman id 680998;
 Wed, 14 Feb 2024 15:29:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zwwQ=JX=gmail.com=breno.debian@srs-se1.protection.inumbo.net>)
 id 1raHC1-0006nU-4K
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:29:01 +0000
Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com
 [209.85.218.49]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c5df4c39-cb4d-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 16:28:59 +0100 (CET)
Received: by mail-ej1-f49.google.com with SMTP id
 a640c23a62f3a-a3d6d160529so31590766b.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 07:28:58 -0800 (PST)
Received: from localhost (fwdproxy-lla-003.fbsv.net.
 [2a03:2880:30ff:3::face:b00c]) by smtp.gmail.com with ESMTPSA id
 kt19-20020a1709079d1300b00a3d6b13c4efsm189745ejc.180.2024.02.14.07.28.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 07:28:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c5df4c39-cb4d-11ee-98f5-efadbce2ee36
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707924538; x=1708529338;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Ywgijd0AB1UdEP/G8o0p2+lNq+pzPJZCGntMh1zrtqQ=;
        b=qHN7Cy8VeXkEibg6UGca7oMwefPJ7NIRMaKueODpspre6p4YGCjbpUoeW+xNM/Sc9T
         CT5MZJYqo1LR9D7/XFdgC3IDKsmCixu+vJNESd0/SVGF8EjEEMT3mUi1KWa5f1jWKNeo
         x5HAkWPm/Xt/2joOowT4W56mrw1oMbO8Q6q4hZaBsvwIwEOTQtktudCXdLP8vtLDPw19
         pFJ6DkaImJ7h+aPQsihXJswjPkqkQ0qt2fioy7ziTyqdxEmEDfH4FeQKG3bqEzWdFuvi
         YYsfgdiwjdv0TXeRxxMk3/9s9Zi/cOnty7gD+t6GbOLM2Ed8RH13TB7Q9P/sPjiJAbWp
         OCXg==
X-Forwarded-Encrypted: i=1; AJvYcCWrnINzW2chnuFJhWaSUOSqEDVdlRRHa1ATsSOGgU9kTg8G63sIuL4GKivHgeqCmzOy6L5jF03nSd4wqJ164S3V/7HSmajtgf2UFq6fsfI=
X-Gm-Message-State: AOJu0Yw5WIx0PPRlrXhiH4UzkgFWikRz4F64D86JimImyrDphcKJX2Bk
	OC9iTxBhVkAkb/juE/GCQqa6kxnTkcIBfYItMcm2SCKE7tBqfqwH
X-Google-Smtp-Source: AGHT+IFtJU0LK8XMLjaJSuQ2WCKqrywmYokSm2MZ7f9DhVf79vcqxli9RGGYxwNOsedtO8MxADI7CQ==
X-Received: by 2002:a17:906:bc59:b0:a38:916e:a4fe with SMTP id s25-20020a170906bc5900b00a38916ea4femr2111177ejv.39.1707924538262;
        Wed, 14 Feb 2024 07:28:58 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVfMLVUFkbuHzQC2OC9s91+gjpxuK1mhdiXAmM1GCwrXLwkEtoLQ/eAjQgNpnuenBaafZecJiI+fj5O3FrEWMo0RdLIhaRBa7Gbwp0QJMtysEWtRtMjhGoi78wZDfEfd6cEWMKtsajM97m0ienpP+djQBkIXcvGXSudvpg3bh519cHjBM3u//oK/24GkJejh+fTi9dUaiVhQMEz3DsLuY0YCPDocCexAJA50AtvWEgi0O6d5eyyQjOviJzqjcuAHP3vSiov30OMzy57ODllVpl5L5sQz5oWHB7SBaPP9g==
From: Breno Leitao <leitao@debian.org>
To: kuba@kernel.org,
	davem@davemloft.net,
	pabeni@redhat.com,
	edumazet@google.com,
	Wei Liu <wei.liu@kernel.org>,
	Paul Durrant <paul@xen.org>
Cc: netdev@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	horms@kernel.org,
	xen-devel@lists.xenproject.org (moderated list:XEN NETWORK BACKEND DRIVER)
Subject: [PATCH net v2 1/7] net: fill in MODULE_DESCRIPTION()s for xen-netback
Date: Wed, 14 Feb 2024 07:27:35 -0800
Message-Id: <20240214152741.670178-2-leitao@debian.org>
X-Mailer: git-send-email 2.39.3
In-Reply-To: <20240214152741.670178-1-leitao@debian.org>
References: <20240214152741.670178-1-leitao@debian.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

W=1 builds now warn if module is built without a MODULE_DESCRIPTION().
Add descriptions to the Xen backend network module.

Signed-off-by: Breno Leitao <leitao@debian.org>
Acked-by: Paul Durrant <paul@xen.org>
---
 drivers/net/xen-netback/netback.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index fab361a250d6..ef76850d9bcd 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -1778,5 +1778,6 @@ static void __exit netback_fini(void)
 }
 module_exit(netback_fini);
 
+MODULE_DESCRIPTION("Xen backend network device module");
 MODULE_LICENSE("Dual BSD/GPL");
 MODULE_ALIAS("xen-backend:vif");
-- 
2.39.3



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:29:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:29:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.680999.1059505 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHC8-00073t-Sp; Wed, 14 Feb 2024 15:29:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 680999.1059505; Wed, 14 Feb 2024 15:29:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHC8-00073m-OV; Wed, 14 Feb 2024 15:29:08 +0000
Received: by outflank-mailman (input) for mailman id 680999;
 Wed, 14 Feb 2024 15:29:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raHC7-00072s-1z
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:29:07 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ca0ac154-cb4d-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 16:29:05 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d10ad265d5so21571621fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 07:29:05 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 x8-20020a05622a000800b0042c6c103f7bsm2086260qtw.37.2024.02.14.07.29.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 07:29:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ca0ac154-cb4d-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707924545; x=1708529345; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=y0x+jTbx8fMoAuRbaxP8v0E7jzDJuyaaY44PCp8u//Y=;
        b=QiiKJpOcPe6GjVROB78/agi2tyAZJ+KpwiEsE58jIlur8lc0QAgJtpU8tNEdOF/yso
         4tEN9ffBhP/7CqeVwIdPG64CMbPXr/n8+4nS4xDzWgxvK2/jljroj7qKTa5HLikZLAZz
         5Mku+NV73ejfIcryiZ+usr2lnnypC2jnnFPRY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707924545; x=1708529345;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=y0x+jTbx8fMoAuRbaxP8v0E7jzDJuyaaY44PCp8u//Y=;
        b=oDMgVYemuOhek+PlfOISOhPNFheLQ65uG656pJC0SVE9HslG3Hx5HYERt/VebIrHTS
         eOWft4CVCcEBTTiNy+eavQC2ARS8aTmrXSlaj5jGSviIoTGfVxq8imaoShzhtZ4a1Z21
         Oa9ARhsFRKcvW5LZaK6ViW+9kZpxMXfQsd9rW+IgLAaWkfLksOgAXiUd39/wy8T+teWB
         xEZvTO3Hp55glampO707hkse90VEOh+uVozFOsFGRTxKIAw6TcMcXG8L056pQWodRgxd
         ADFOIRjEmRJznb27rlutzHhag5CLuZEGNRdUOAfJi9f+I8pfjww3mSQGgNBeUtjKmZ2X
         eTpg==
X-Gm-Message-State: AOJu0Yx1eLEQ461R/8zNsjYZnpHKkGYFRrYQktTyICRdrjZKnGzcGDtV
	AlpMgy8WsVoQNb6WXVvH6Au732YoeTFRTlsHJQqF7U8rnMtS6guOO/H1fr1nMmg=
X-Google-Smtp-Source: AGHT+IGWgXFe467NBjTQNZyxtkG+dkBrckGoZ3SUMb3Cvn1fK1d2Nq0QSwV/7n2RFrGw3H3RLDEa1g==
X-Received: by 2002:a2e:7818:0:b0:2d1:8fb:1b35 with SMTP id t24-20020a2e7818000000b002d108fb1b35mr2250905ljc.16.1707924545249;
        Wed, 14 Feb 2024 07:29:05 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXig/OACCIjeXM1ftSPa+G34KJe7Nk3mTiy/Ud/aSukvWfORq7v1/LUeajVSK6KUEwFxYOcnmlUMGGhAJXh268TWERfB1S3l6FO045vHmulBEBVAYXKmrJMQhI0R/xqUV+ngnNKOOCesmxxmCnnClNh
Date: Wed, 14 Feb 2024 16:29:02 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	Frediano Ziglio <frediano.ziglio@cloud.com>
Subject: Re: [PATCH v2] Reduce assembly code size of exception entry points
Message-ID: <ZczcPhpcWuAO7OYh@macbook>
References: <20240214103558.13655-1-frediano.ziglio@cloud.com>
 <ZczWAg-zmoLXiWAr@macbook>
 <ce3ae020-92b2-44fd-8055-694e374f68a9@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ce3ae020-92b2-44fd-8055-694e374f68a9@suse.com>

On Wed, Feb 14, 2024 at 04:08:12PM +0100, Jan Beulich wrote:
> On 14.02.2024 16:02, Roger Pau Monné wrote:
> > On Wed, Feb 14, 2024 at 10:35:58AM +0000, Frediano Ziglio wrote:
> >> We just pushed a 8-bytes zero and exception constants are
> >> small so we can just write a single byte saving 3 bytes for
> >> instruction.
> >> With ENDBR64 this reduces the size of many entry points from 32 to
> >> 16 bytes (due to alignment).
> >> Similar code is already used in autogen_stubs.
> > 
> > Will using movb instead of movl have any performance impact?  I don't
> > think we should trade speed for code size, so this needs to be
> > mentioned in the commit message.
> 
> That's really what the last sentence is about (it could have been said
> more explicitly though): If doing so on interrupt paths is fine, it
> ought to be fine on exception paths as well.

I might view it the other way around: maybe it's autogen_stubs that
needs changing to use movl instead of movb for performance reasons?

I think this needs to be clearly stated, and ideally some kind of
benchmarks should be provided to demonstrate no performance change if
there are doubts whether movl and movb might perform differently.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:29:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:29:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681004.1059515 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHCX-0007j5-4m; Wed, 14 Feb 2024 15:29:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681004.1059515; Wed, 14 Feb 2024 15:29:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHCW-0007i3-W9; Wed, 14 Feb 2024 15:29:32 +0000
Received: by outflank-mailman (input) for mailman id 681004;
 Wed, 14 Feb 2024 15:29:31 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raHCV-0007gT-AP; Wed, 14 Feb 2024 15:29:31 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raHCV-0006NV-7n; Wed, 14 Feb 2024 15:29:31 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raHCU-0000B7-Ph; Wed, 14 Feb 2024 15:29:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raHCU-00070W-PG; Wed, 14 Feb 2024 15:29:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=0j6c9YReBVdSsdNnDOkc3VQeybIB9ochIsKTbzvoqkA=; b=10oxMlsP/ZDUVzIPxMl7ctPyKU
	SeVOmtTeXpUmq1P15zNTenvIXa651lbl8qE66Xfjh/USYrbHRaN0RWAtjZhmdvtd5HZ1K40jVqd3Q
	FRSVBez94sRyV6ueV9NLEEbCUVMYHpzWZ2FX+vVuecjy4NJZ4+qpwkLcn6vxrOaSokaw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184665-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184665: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:guest-start/debianhvm.repeat:fail:heisenbug
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:guest-localmigrate/x10:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-vhd:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=b3bea912a737a0db39a89b0da0033c4bfda82f03
X-Osstest-Versions-That:
    xen=b3bea912a737a0db39a89b0da0033c4bfda82f03
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 14 Feb 2024 15:29:30 +0000

flight 184665 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184665/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 20 guest-start/debianhvm.repeat fail in 184660 pass in 184665
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184660
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184660
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 18 guest-localmigrate/x10 fail pass in 184660
 test-armhf-armhf-xl-vhd      17 guest-start/debian.repeat  fail pass in 184660

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184660
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184660
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184660
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184660
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184660
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184660
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184660
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184660
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184660
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184660
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184660
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184660
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  b3bea912a737a0db39a89b0da0033c4bfda82f03
baseline version:
 xen                  b3bea912a737a0db39a89b0da0033c4bfda82f03

Last test of basis   184665  2024-02-14 08:21:31 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      fail    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:31:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:31:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681011.1059525 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHEY-00016J-G0; Wed, 14 Feb 2024 15:31:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681011.1059525; Wed, 14 Feb 2024 15:31:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHEY-00016C-C3; Wed, 14 Feb 2024 15:31:38 +0000
Received: by outflank-mailman (input) for mailman id 681011;
 Wed, 14 Feb 2024 15:31:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=83ZH=JX=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1raHEX-00015h-Eq
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:31:37 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2319b43a-cb4e-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 16:31:35 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id AD8004EE0737;
 Wed, 14 Feb 2024 16:31:34 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2319b43a-cb4e-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Wed, 14 Feb 2024 16:31:34 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Simone Ballarin <simone.ballarin@bugseng.com>, Doug
 Goldstein <cardoe@cardoe.com>, George Dunlap <george.dunlap@citrix.com>, Wei
 Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH] automation/eclair_analysis: deviate certain macros
 for Rule 20.12
In-Reply-To: <3519db82-6126-4aa0-9d04-795edf6f2bca@suse.com>
References: <892be5c403ca7a20b35fb9facacb6a38bc7f6bfe.1707900742.git.nicola.vetrini@bugseng.com>
 <3519db82-6126-4aa0-9d04-795edf6f2bca@suse.com>
Message-ID: <b87f53b8e39fdbd7f7aefe63f227fe7f@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-14 12:49, Jan Beulich wrote:
> On 14.02.2024 12:26, Nicola Vetrini wrote:
>> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
>> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
>> @@ -387,6 +387,16 @@ in assignments."
>>  {safe, "left_right(^[(,\\[]$,^[),\\]]$)"}
>>  -doc_end
>> 
>> +-doc_begin="The token pasting in variadic macros cannot be replaced."
>> +-config=MC3R1.R20.12,macros+={deliberate, "variadic()"}
>> +-doc_end
>> +
>> +-doc_begin="Uses of # and ## operators within the following macros 
>> are
>> +deliberate."
> 
> I don't think this is a good explanation. The use of those operators is
> deliberate elsewhere as well.
> 

Implicitly this was intended to be delimited to its applicability scope 
- violations of Rule 20.12. I detailed the reasons that were brought up 
to keep using these macros as is in the deviations.rst file. I'm open to 
further refinements, though.

>> +-config=MC3R1.R20.12,macros+={deliberate, 
>> "name(ASSERT||BUILD_BUG_ON||BUILD_BUG_ON_ZERO||GENERATE_CASE)"}
> 
> I said in another context already that it is necessary to separate
> global scope macros from local helper ones. Any CU can introduce 
> another
> GENERATE_CASE(), and would be deviated here right away. In fact I
> question applicability of the deviation to arm/arm64/vsysreg.c; I only
> see it as applicable to arm/vcpreg.c.
> 

Why wouldn't this be applicable for vsysreg.c? I can certainly fine-tune 
the deviation, if needed

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:41:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:41:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681027.1059543 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHNx-0003Gv-Iu; Wed, 14 Feb 2024 15:41:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681027.1059543; Wed, 14 Feb 2024 15:41:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHNx-0003Go-FC; Wed, 14 Feb 2024 15:41:21 +0000
Received: by outflank-mailman (input) for mailman id 681027;
 Wed, 14 Feb 2024 15:41:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raHNv-0003Gg-P8
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:41:19 +0000
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com
 [2a00:1450:4864:20::331])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7eb53ca1-cb4f-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 16:41:18 +0100 (CET)
Received: by mail-wm1-x331.google.com with SMTP id
 5b1f17b1804b1-411f17700ffso5152155e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 07:41:18 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z1-20020a05600c0a0100b00411ce6def3fsm2345857wmp.38.2024.02.14.07.41.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 07:41:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7eb53ca1-cb4f-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707925278; x=1708530078; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=H5VUANlO1xV6UYsQayWjOtnZr9ZMmIC4eD4CnxviuJk=;
        b=NGrJJ/IZzZt1FZij4MTchjGnE0IxAPp0O9N9WzrOtjTeZgHAXDugX/ILzkLfT9JjVp
         Rd3aCDxq5SMHVu19QjzkgcJApVMhVEXmmPhaU6GzDGE+gmT5XBgSzbl86qhTvttx38v3
         Svl0tOXf99shij426gZ+vMdFApZrK461NklDyI6o2KMd0/u9dn7kWri18NoHXf4d0WbW
         DAXAW0+JOb6cNy2limWoCehV/tcf+mpG5bLh4rfS3eour8SROVXV96pgRET4k1dZq2K1
         VZprS+0h/FbD6vIJzEJP0cHB0BtaD17FbXH+WzrohyvoeW9V5NGztF1VXFXjG5+cB0oM
         xhiQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707925278; x=1708530078;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=H5VUANlO1xV6UYsQayWjOtnZr9ZMmIC4eD4CnxviuJk=;
        b=R61CI4IaKcjbeXZChJ2TWEIfyf4vUwLPUbMFDT5bNG6rNV3bH8SUQ0WuApSwPPDT8U
         ouCP5Q7GZWBdaPFV+yZWy6d+Hu4thkzzpZujOUNSK1IiLvE0b/LLIO9RKd1kwFctPCR6
         dUmp1GO+0ATXEcSPKe2YTQZH+f840rAFjop/bh8Wi+gI1ExdFJ+f9OjSBb52WT7usCbG
         lSGM19b+63awL8yBtF+KTaCf6LgduP+bcIGDQRkv1OG6GjhkB/olbWMJIqpSX24tUQZO
         H7L6YXlOceub/MZVj6Qn1DSapQS6m7OEEOHyhgOicv+EgHN7jn5hSBmvhhICMvq+7U0R
         +wuw==
X-Forwarded-Encrypted: i=1; AJvYcCXLtVIoyg9JcNBcqiNgGTWWaMynI5KSFoesoEQSeZHJWWINp0Lco9VvHdrk+wQxunJs1/mJG5r1dOQaXA958qRISSUnH8PGncjWWS4xsYQ=
X-Gm-Message-State: AOJu0YxU6MyzMQj4ky+rjVUhe9RKTKFdF2cHHb0IMHHqIuex4gqIPElr
	vGsROewaen1WMg5vSG42CZUHWIa0xXfGZ2YQb67/ZF95sC13/DOtLxwNAvI5nA==
X-Google-Smtp-Source: AGHT+IEBT8gmWi1+xReFfTrYPz8d/ygqnr42j1mWH+xNHEkxZU0eURsKajhdhsMuptbcDV+00FO+yw==
X-Received: by 2002:a05:600c:1991:b0:411:ffcd:8f81 with SMTP id t17-20020a05600c199100b00411ffcd8f81mr444660wmq.18.1707925277883;
        Wed, 14 Feb 2024 07:41:17 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXbxt9DfIj066IoyEYa8+sAqo/Ls0rTvE2CQloCvqDBkLJVafjyucxcwVZ77MJLuYRfAxG1zwm3IDfUKl2zxALM06y03vNIvbv5vewqz3kNc76cq82N6ApMyyyznhVSZSL818KQtNgPoBg71BhQ7+mjN5YwJVrfV450TtmW2oMbPfSIwNwXF3TTfXGm7g0CVQgAPB5024FJZDEUt9OdOBN+CJQcAOPuoPkLvqOYX2nbx/KXQgSPP/y9uQZ/xDEb0TfDo+kFWv4ICj2nUZvXLjFVU4ZXbS/Fyd7Le1stBgAB5/9XMalQccR6Xg==
Message-ID: <180cac00-e7c4-4bea-bd13-c5983da707d7@suse.com>
Date: Wed, 14 Feb 2024 16:41:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13 10/14] vpci/header: emulate PCI_COMMAND register for
 guests
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-11-stewart.hildebrand@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240202213321.1920347-11-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 02.02.2024 22:33, Stewart Hildebrand wrote:
> @@ -836,9 +870,20 @@ static int cf_check init_header(struct pci_dev *pdev)
>      if ( pdev->ignore_bars )
>          return 0;
>  
> -    /* Disable memory decoding before sizing. */
>      cmd = pci_conf_read16(pdev->sbdf, PCI_COMMAND);
> -    if ( cmd & PCI_COMMAND_MEMORY )
> +
> +    /*
> +     * Clear PCI_COMMAND_MEMORY and PCI_COMMAND_IO for DomUs, so they will
> +     * always start with memory decoding disabled and to ensure that we will not
> +     * call modify_bars() at the end of this function.

To achieve this, fiddling with PCI_COMMAND_IO isn't necessary. Which isn't
to say its clearing should go away; quite the other way around: Why would
we leave e.g. PCI_COMMAND_MASTER enabled? In fact wasn't it in an earlier
version of the series that the guest view simply started out as zero? The
patch description still says so.

> --- a/xen/drivers/vpci/msi.c
> +++ b/xen/drivers/vpci/msi.c
> @@ -70,6 +70,13 @@ static void cf_check control_write(
>  
>          if ( vpci_msi_arch_enable(msi, pdev, vectors) )
>              return;
> +
> +        /* Make sure domU doesn't enable INTx while enabling MSI. */
> +        if ( !is_hardware_domain(pdev->domain) )
> +        {
> +            pci_intx(pdev, false);
> +            pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
> +        }

While here we're inside "if ( new_enabled )", ...

> --- a/xen/drivers/vpci/msix.c
> +++ b/xen/drivers/vpci/msix.c
> @@ -135,6 +135,13 @@ static void cf_check control_write(
>          }
>      }
>  
> +    /* Make sure domU doesn't enable INTx while enabling MSI-X. */
> +    if ( new_enabled && !msix->enabled && !is_hardware_domain(pdev->domain) )
> +    {
> +        pci_intx(pdev, false);
> +        pdev->vpci->header.guest_cmd |= PCI_COMMAND_INTX_DISABLE;
> +    }

.. here you further check that it's actually a 0->1 transition? Why
not alike for MSI?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:45:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:45:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681035.1059561 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHRl-0004AU-21; Wed, 14 Feb 2024 15:45:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681035.1059561; Wed, 14 Feb 2024 15:45:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHRk-0004AN-Vj; Wed, 14 Feb 2024 15:45:16 +0000
Received: by outflank-mailman (input) for mailman id 681035;
 Wed, 14 Feb 2024 15:45:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=l+oD=JX=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raHRk-0004AG-Jg
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:45:16 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0bdd5a6c-cb50-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 16:45:15 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-33cddf4b4b5so1276942f8f.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 07:45:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 r2-20020a056000014200b0033b4acb999dsm12623407wrx.98.2024.02.14.07.45.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 07:45:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0bdd5a6c-cb50-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707925515; x=1708530315; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=TvQiN1hpDNMqbnpgIeCWsGY/7CQtMyl74GRbryjZrhU=;
        b=QtGzVoEolyqwJQ4mgrN3ms+gcBTLROeP0h9tqyJcuKHlIZ7wNws+quVcdRAJcgSGLB
         ZqD5/3f/I/htTndpR5WCOVsZ95/4T5E6+DtaKULiEXOzI/aR6X06X9t5sWT5U0YIdUJd
         mYu6cf8cwSjWiWbfkMa+NxFHM0Tg1MgKX/QKobBi3ByIfFAuP3qUx7YcNEH9DVMfS0du
         hd65X2vyLxA629/hVR6YApOcoz4BxGt4SjTQO6NLrTnxas72mZ/0r99oEoHBtN50+19y
         tFBUED4TxUXvvLKUpx8lRBwFs9rMXoO6Biu/y/MyrTjCkHuPS/Zmy48t6+ZNVpXhYveh
         SNaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707925515; x=1708530315;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=TvQiN1hpDNMqbnpgIeCWsGY/7CQtMyl74GRbryjZrhU=;
        b=hmvie1pjNYhfZRcnstsSUKEHvMB4wTqSmuwoJaAGwhm/TOY2/yKDZzrIzMwhrw1hgb
         RmTucG/bqVfqhVmoQQPIToSugtUtyJlqQAWto+dorlbQEi63aps45Of9DKEPetvJTBPv
         mfvIBrNakZhHjJ47ITZ/i7YK5lU1kooO5pMlBkQlYju4ixWnyOOBpAVvsV3EYqN9aj9H
         0LeFNJzO/jXVVyZOxE48X8lorGRblHPGZer/mg7xiFdoNTfhlJ5Ym+h43+iNuS3vLX5G
         FiIVYc/EHL25+l7tn60EYQwJBdgGBsUq2bOGm15W6cSyGzskcFTLlfP0FRZdCaG7nYm7
         5Ziw==
X-Forwarded-Encrypted: i=1; AJvYcCV5XSRI9gDRukaX7zv4ArY/XiXqLnQggV1HbTzMZb76UX2T4nTznt1cS4nVqC75y45ABRVW5hEI4z/nptND5ZwDpyox4pGK74GemZVUr0Y=
X-Gm-Message-State: AOJu0YzejrTItAbckapNAl0E6hpnUKvv5NbzpBAAk1YkletL3ZUgwlVr
	FAM4Uc3M7dU/LHcNzzNP+25NEfiiOQHVGm6VXmhvOfG0RY5Z7gPcWENdxLBycg==
X-Google-Smtp-Source: AGHT+IGkg8kCfd4nk4XfXVp4V6XVfyn4kJHXtM/gVD52cfZiOycE0Dr3AS4P/JJpJCgNLRibPzCaHA==
X-Received: by 2002:adf:e90a:0:b0:33b:649a:1a1b with SMTP id f10-20020adfe90a000000b0033b649a1a1bmr2311195wrm.49.1707925514743;
        Wed, 14 Feb 2024 07:45:14 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWftMTWQ3K0SjNZPcs9pCAHpczQraU0ZfpLWzM3CFYWy4da1f0HtN+OO5YOcNL36BV/ZAVTWfCYXLfs9LsBif5tLiVxgHM4T5g3cJNx95ENxvqb7EmtvR/Jm/8pYlPabJXalyB/T47KVrDL2mMg14LYvW9wBHcLmHKoAGTDlHmGGqxsAEjwjzWZQEne4wIfMbbL1udiwJ5eoUnI4c1kVJLNDO+w0TKMyTOG3PmkWa6RXAl41hHV0o1OAd9VCb5WFQzLj3zjpbY1PNkIyL7Zvh5ttqAvsUm7wky8eOYXtHfpIDA7dc0TpMWLnoY21RG2SwnizoXzPmloAEKY992Ff4+n1klBQUF7nPMwP5OPWN2KY2rlc0fE7hfnS5EwBnxScL+L72tD3IG+mK0E5+MoaJep5T0w/gtgx4RbVgsEshnseRC1U+irTyWF+8z3p3kXZ/XfUvJnMEtyU92+CqDBbj3mNxI=
Message-ID: <447b1843-f28c-4633-9e74-d695bc37208a@suse.com>
Date: Wed, 14 Feb 2024 16:45:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair_analysis: deviate certain macros
 for Rule 20.12
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>, George Dunlap
 <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <892be5c403ca7a20b35fb9facacb6a38bc7f6bfe.1707900742.git.nicola.vetrini@bugseng.com>
 <3519db82-6126-4aa0-9d04-795edf6f2bca@suse.com>
 <b87f53b8e39fdbd7f7aefe63f227fe7f@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b87f53b8e39fdbd7f7aefe63f227fe7f@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.02.2024 16:31, Nicola Vetrini wrote:
> On 2024-02-14 12:49, Jan Beulich wrote:
>> On 14.02.2024 12:26, Nicola Vetrini wrote:
>>> +-config=MC3R1.R20.12,macros+={deliberate, 
>>> "name(ASSERT||BUILD_BUG_ON||BUILD_BUG_ON_ZERO||GENERATE_CASE)"}
>>
>> I said in another context already that it is necessary to separate
>> global scope macros from local helper ones. Any CU can introduce 
>> another
>> GENERATE_CASE(), and would be deviated here right away. In fact I
>> question applicability of the deviation to arm/arm64/vsysreg.c; I only
>> see it as applicable to arm/vcpreg.c.
> 
> Why wouldn't this be applicable for vsysreg.c? I can certainly fine-tune 
> the deviation, if needed

There the sole macro parameter is used in only one of the possible ways,
with ##, unless I'm overlooking something. Aiui it is mixed use which
would need a deviation.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 15:53:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 15:53:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681041.1059571 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHZe-00067n-Pq; Wed, 14 Feb 2024 15:53:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681041.1059571; Wed, 14 Feb 2024 15:53:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHZe-00067g-NF; Wed, 14 Feb 2024 15:53:26 +0000
Received: by outflank-mailman (input) for mailman id 681041;
 Wed, 14 Feb 2024 15:53:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Sn1=JX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1raHZd-00067a-W4
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 15:53:25 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2fe1fe1d-cb51-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 16:53:25 +0100 (CET)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2d0a4e1789cso66924841fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 07:53:25 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 p2-20020a056402044200b0056200715130sm1428241edw.54.2024.02.14.07.53.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 07:53:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2fe1fe1d-cb51-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707926005; x=1708530805; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bD9rKCpEMpxhCq5S1MQTvrpi48yycMfbflB2M7btCJ4=;
        b=XOr1zU+NULIKGhXKOp54VwTsIM28/4FH5tzLXOvr/JtuaEUMheUjI10eV6wFvSKkdr
         s305CMpbzvR1Vx7ksnjjK1IKIDAWInDDgbxD4GwC36dlet37G8Gu7jyn1wFAXcw3Dp8d
         189gfVmZcEHNWk3bTGXfLbK/dSkZbgVmbWsaA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707926005; x=1708530805;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bD9rKCpEMpxhCq5S1MQTvrpi48yycMfbflB2M7btCJ4=;
        b=wpxGxe8MSLHcX1ww92y3Nl7e9W4CvsZqf0Y4gli9W9HyyH7MurzSM1+TM4KHW5mxt8
         1/bkfgMFPCrhK75Pp5IybuZiDhVs8R1LtiBFw60Uy8uf6/2Kj2clAv8af+QUtsM5SajF
         4/CsptMe+znuCD+rPI32jvr0SPxtLcINx1FOtoYqRR0pLEDOGH0O6E8J1l1fA+B4+GLy
         NwPsZ3f+BD0ndAesozrnWvuKb3Pwh9J3KZHfx+mlQaPxWvJscO8qTiJUs555edyYil54
         fz2JtPOsQn93riE2cw1sjePTxsKRrP8kXirQNdZ7ZE7CGgBc0kWFU50PB38AASLeO3FH
         bWaA==
X-Gm-Message-State: AOJu0YwWDBVOFrRwrHqzeTj0Cl0nLRjYGE1Il+3gr8tHJTyo9xrbcHYa
	kATriLsbc+VQEDN5TPDdwMbx3DqVz7UFfb43LWp1gPGh55dQkW3VSPYjBPz9IOA=
X-Google-Smtp-Source: AGHT+IHF1vaPf0hF5WCQth0xAJj+9SA8dKAbXKqoYWt91IqshkJUxmycF3u7z4/QLZJVsNghR/B09w==
X-Received: by 2002:a2e:b0cb:0:b0:2d0:9bcb:6248 with SMTP id g11-20020a2eb0cb000000b002d09bcb6248mr2193963ljl.39.1707926004685;
        Wed, 14 Feb 2024 07:53:24 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCW4Uv1Nrz0eQnERdJ83gzeXaQq8Lqy9hCB8Mqt8H2jKph1nnnpmUWTE/4/J6znao81F8tS4C6Z2Sh+hr8Aqsa1sRU5lwIx+jpGdKWOcOGo38LHioCGWgJxs+kRN3lPUGnkryR4zAtORsA==
Message-ID: <7f92673f-fa33-478f-ab27-bbec2ead5c40@citrix.com>
Date: Wed, 14 Feb 2024 15:53:24 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] Reduce assembly code size of exception entry points
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
 Frediano Ziglio <frediano.ziglio@cloud.com>
References: <20240214103558.13655-1-frediano.ziglio@cloud.com>
 <ZczWAg-zmoLXiWAr@macbook> <ce3ae020-92b2-44fd-8055-694e374f68a9@suse.com>
 <ZczcPhpcWuAO7OYh@macbook>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZczcPhpcWuAO7OYh@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/02/2024 3:29 pm, Roger Pau Monné wrote:
> On Wed, Feb 14, 2024 at 04:08:12PM +0100, Jan Beulich wrote:
>> On 14.02.2024 16:02, Roger Pau Monné wrote:
>>> On Wed, Feb 14, 2024 at 10:35:58AM +0000, Frediano Ziglio wrote:
>>>> We just pushed a 8-bytes zero and exception constants are
>>>> small so we can just write a single byte saving 3 bytes for
>>>> instruction.
>>>> With ENDBR64 this reduces the size of many entry points from 32 to
>>>> 16 bytes (due to alignment).
>>>> Similar code is already used in autogen_stubs.
>>> Will using movb instead of movl have any performance impact?  I don't
>>> think we should trade speed for code size, so this needs to be
>>> mentioned in the commit message.
>> That's really what the last sentence is about (it could have been said
>> more explicitly though): If doing so on interrupt paths is fine, it
>> ought to be fine on exception paths as well.
> I might view it the other way around: maybe it's autogen_stubs that
> needs changing to use movl instead of movb for performance reasons?
>
> I think this needs to be clearly stated, and ideally some kind of
> benchmarks should be provided to demonstrate no performance change if
> there are doubts whether movl and movb might perform differently.

The push and the mov are overlapping stores either way.  Swapping
between movl and movb will make no difference at all.

However, the shorter instruction ends up halving the size of the entry
stub when alignment is considered, and that will make a marginal
difference.  Fewer cache misses (to a first approximation, even #PF will
be L1-cold), and better utilisation of branch prediction resource (~>
less likely to be BP-cold).

I doubt you'll be able to see a difference without perf counters
(whatever difference is covered here will be dwarfed by the speculation
workarounds), but a marginal win is still a win.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 16:06:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 16:06:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681052.1059597 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHlh-0000Gp-8B; Wed, 14 Feb 2024 16:05:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681052.1059597; Wed, 14 Feb 2024 16:05:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHlh-0000Gi-4e; Wed, 14 Feb 2024 16:05:53 +0000
Received: by outflank-mailman (input) for mailman id 681052;
 Wed, 14 Feb 2024 16:05:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WArD=JX=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raHlf-0000Ga-FW
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 16:05:51 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ec0879f7-cb52-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 17:05:50 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d073b54359so82906611fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 08:05:50 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 q19-20020ac84513000000b0042c6e5192e6sm2099376qtn.60.2024.02.14.08.05.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 08:05:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec0879f7-cb52-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707926750; x=1708531550; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Ee2PqXgtqlaFAmPhieq1rCz/wnguvI78ivIy080wfc8=;
        b=BaQRnTfp2a1IXrkxgDdSHcxxFcQexVBIvcn6InJSiUvdScTBuN9bjtuoLwQW+9KzE5
         I24VfDb+AsDmpyLQRfVyP/hJm0LbtRKUSyM4iw4cB578hNMeCIH+0DdRTchPQdPOukz8
         VOb3HKJ7MLGjWwKZ1V4mjEHBUdUmvwKAmiylo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707926750; x=1708531550;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Ee2PqXgtqlaFAmPhieq1rCz/wnguvI78ivIy080wfc8=;
        b=uA1h2cEMQrQuCcIJCOGBx+rVEhLtnEMMn9cQTV832Y0gzgFTXwdmzqYfR82/1b18i2
         X/yWXPEst1bqcBl5BxIP4wD10Vd1YAQFT65izsWGzlRLzjVTRdMV+dGoKI8Yu0u4bTHQ
         NqEO7ZU7tQdYNjzN3Fujgf97fkT7j1ATNvK+zo3UBnX7nHGNoXX3BciwHyO2xa6UUDhv
         FI0bx6LGpzx/WPb2bXUTsZxI2eRICAUfqYCvf/f+cUJVaNd82XLhdy2mxt+oWdZM2115
         S94+AcI5zLcTgrK5j5vH96PlhIh8eFZVpdm6CmJnKZZFxAWBG0mNSBAk7Mg5z1DiMKwL
         yAGA==
X-Forwarded-Encrypted: i=1; AJvYcCVohhLd6OZeXEU4h6mZ4A82TjrjCD+lBxZbf7txHpfzGkJhRWG/hriVsPMNXgX9/z5iSdqAOytrb8j9YypJ746iRiTVJstD9A2CEKSIC7E=
X-Gm-Message-State: AOJu0YyfFp7vYJk2QnDn8oGyHSBgOwQn9XukH1CNHDNts9BIfllRwAYw
	COejGpLGtrVIh7zh2o0k+AakFj8LUzV1Xt9WcGNJamRcVfPpjnt5qLWe79L6klA=
X-Google-Smtp-Source: AGHT+IHRWxlXoOK62vctLZxPl61VPJarAdCAPHPH+0kN2sfqq03shMl6M1jmieBJUtbNM8PvaxlrIg==
X-Received: by 2002:a2e:a452:0:b0:2d1:1c11:38b with SMTP id v18-20020a2ea452000000b002d11c11038bmr1253094ljn.13.1707926749662;
        Wed, 14 Feb 2024 08:05:49 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWPTgJ42MSi5x5/UAPm56vwhiDtwZFUEZweHmgrxKlzZgeZ6i9JJX5O935ukZHL8/C3sSIGs212weuQ6Goh/8pFgXFZdAo/jHcNNfKVOEAtLjo/U8ZT5F/kWu1WkidyKARj/Hu6AeaAnvY3rde2ECgtw10k/VI=
Date: Wed, 14 Feb 2024 17:05:47 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>, xen-devel@lists.xenproject.org,
	Wei Liu <wl@xen.org>, Frediano Ziglio <frediano.ziglio@cloud.com>
Subject: Re: [PATCH v2] Reduce assembly code size of exception entry points
Message-ID: <Zczk2-YaeOwk1MUI@macbook>
References: <20240214103558.13655-1-frediano.ziglio@cloud.com>
 <ZczWAg-zmoLXiWAr@macbook>
 <ce3ae020-92b2-44fd-8055-694e374f68a9@suse.com>
 <ZczcPhpcWuAO7OYh@macbook>
 <7f92673f-fa33-478f-ab27-bbec2ead5c40@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <7f92673f-fa33-478f-ab27-bbec2ead5c40@citrix.com>

On Wed, Feb 14, 2024 at 03:53:24PM +0000, Andrew Cooper wrote:
> On 14/02/2024 3:29 pm, Roger Pau Monné wrote:
> > On Wed, Feb 14, 2024 at 04:08:12PM +0100, Jan Beulich wrote:
> >> On 14.02.2024 16:02, Roger Pau Monné wrote:
> >>> On Wed, Feb 14, 2024 at 10:35:58AM +0000, Frediano Ziglio wrote:
> >>>> We just pushed a 8-bytes zero and exception constants are
> >>>> small so we can just write a single byte saving 3 bytes for
> >>>> instruction.
> >>>> With ENDBR64 this reduces the size of many entry points from 32 to
> >>>> 16 bytes (due to alignment).
> >>>> Similar code is already used in autogen_stubs.
> >>> Will using movb instead of movl have any performance impact?  I don't
> >>> think we should trade speed for code size, so this needs to be
> >>> mentioned in the commit message.
> >> That's really what the last sentence is about (it could have been said
> >> more explicitly though): If doing so on interrupt paths is fine, it
> >> ought to be fine on exception paths as well.
> > I might view it the other way around: maybe it's autogen_stubs that
> > needs changing to use movl instead of movb for performance reasons?
> >
> > I think this needs to be clearly stated, and ideally some kind of
> > benchmarks should be provided to demonstrate no performance change if
> > there are doubts whether movl and movb might perform differently.
> 
> The push and the mov are overlapping stores either way.  Swapping
> between movl and movb will make no difference at all.
> 
> However, the shorter instruction ends up halving the size of the entry
> stub when alignment is considered, and that will make a marginal
> difference.  Fewer cache misses (to a first approximation, even #PF will
> be L1-cold), and better utilisation of branch prediction resource (~>
> less likely to be BP-cold).
> 
> I doubt you'll be able to see a difference without perf counters
> (whatever difference is covered here will be dwarfed by the speculation
> workarounds), but a marginal win is still a win.

I'm happy just stating in the commit message that the change doesn't
make any performance difference.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 16:07:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 16:07:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681054.1059607 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHmn-0000no-N1; Wed, 14 Feb 2024 16:07:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681054.1059607; Wed, 14 Feb 2024 16:07:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHmn-0000nk-Ix; Wed, 14 Feb 2024 16:07:01 +0000
Received: by outflank-mailman (input) for mailman id 681054;
 Wed, 14 Feb 2024 16:07:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xNvu=JX=actia.se=john.ernberg@srs-se1.protection.inumbo.net>)
 id 1raHmm-0000fn-Uj
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 16:07:00 +0000
Received: from mail.actia.se (mail.actia.se [212.181.117.226])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 15bb1de6-cb53-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 17:07:00 +0100 (CET)
Received: from S036ANL.actianordic.se (10.12.31.117) by S036ANL.actianordic.se
 (10.12.31.117) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb
 2024 17:06:59 +0100
Received: from S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69]) by
 S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69%4]) with mapi id
 15.01.2507.035; Wed, 14 Feb 2024 17:06:59 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15bb1de6-cb53-11ee-8a4d-1f161083a0e0
From: John Ernberg <john.ernberg@actia.se>
To: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Peng
 Fan" <peng.fan@nxp.com>, Jonas Blixt <jonas.blixt@actia.se>, John Ernberg
	<john.ernberg@actia.se>
Subject: [PATCH v2 0/2] Xen: ARM: Improved NXP iMX8 platform support
Thread-Topic: [PATCH v2 0/2] Xen: ARM: Improved NXP iMX8 platform support
Thread-Index: AQHaX1/WsH0KS4ZeU0CO5OqKnDdxdQ==
Date: Wed, 14 Feb 2024 16:06:59 +0000
Message-ID: <20240214160644.3418228-1-john.ernberg@actia.se>
Accept-Language: en-US, sv-SE
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: git-send-email 2.43.0
x-originating-ip: [10.12.12.30]
x-esetresult: clean, is OK
x-esetid: 37303A2958D72955607464
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

The iMX lpuart driver added at 44e17aa60d47 ("xen/arm: Add i.MX lpuart driv=
er")
is not enough to boot a Linux based dom0 when certain drivers, such as the
watchdog driver, are enabled.

We're also fixing compatibles in imx-lpuart to allow Xen to use the UART
on the QXP variant as well.

NOTE:
There's still an open point regarding the watchdog from v1, but I wanted
to post this series to show the filtering changes mostly because they also
clarify what exists and what we can definitely get away with blocking /
ignoring out of all the SMC SIP calls.

v2: (See each path for complete changelog)
 - Added SIP call filtering (Julien Grall)
 - Replace lpuart compatible instead (Julien Grall)

v1: https://lore.kernel.org/xen-devel/20240131114952.305805-1-john.ernberg@=
actia.se

John Ernberg (2):
  xen/arm: Add imx8q{m,x} platform glue
  xen/drivers: imx-lpuart: Replace iMX8QM compatible with iMX8QXP

 xen/arch/arm/platforms/Makefile |   1 +
 xen/arch/arm/platforms/imx8qm.c | 143 ++++++++++++++++++++++++++++++++
 xen/drivers/char/imx-lpuart.c   |   2 +-
 3 files changed, 145 insertions(+), 1 deletion(-)
 create mode 100644 xen/arch/arm/platforms/imx8qm.c

--=20
2.43.0


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 16:07:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 16:07:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681055.1059617 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHmo-00012g-Tj; Wed, 14 Feb 2024 16:07:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681055.1059617; Wed, 14 Feb 2024 16:07:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHmo-00012X-QL; Wed, 14 Feb 2024 16:07:02 +0000
Received: by outflank-mailman (input) for mailman id 681055;
 Wed, 14 Feb 2024 16:07:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xNvu=JX=actia.se=john.ernberg@srs-se1.protection.inumbo.net>)
 id 1raHmn-0000fn-Uk
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 16:07:01 +0000
Received: from mail.actia.se (mail.actia.se [212.181.117.226])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 15bd22fa-cb53-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 17:07:00 +0100 (CET)
Received: from S036ANL.actianordic.se (10.12.31.117) by S035ANL.actianordic.se
 (10.12.31.116) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb
 2024 17:06:59 +0100
Received: from S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69]) by
 S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69%4]) with mapi id
 15.01.2507.035; Wed, 14 Feb 2024 17:06:59 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15bd22fa-cb53-11ee-8a4d-1f161083a0e0
From: John Ernberg <john.ernberg@actia.se>
To: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Peng
 Fan" <peng.fan@nxp.com>, Jonas Blixt <jonas.blixt@actia.se>, John Ernberg
	<john.ernberg@actia.se>
Subject: [PATCH v2 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Topic: [PATCH v2 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Index: AQHaX1/Xeq0xXmpbZkiZsYYSHlNyGA==
Date: Wed, 14 Feb 2024 16:06:59 +0000
Message-ID: <20240214160644.3418228-2-john.ernberg@actia.se>
References: <20240214160644.3418228-1-john.ernberg@actia.se>
In-Reply-To: <20240214160644.3418228-1-john.ernberg@actia.se>
Accept-Language: en-US, sv-SE
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: git-send-email 2.43.0
x-originating-ip: [10.12.12.30]
x-esetresult: clean, is OK
x-esetid: 37303A2921D72955607464
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

When using Linux for dom0 there are a bunch of drivers that need to do SMC
SIP calls into the firmware to enable certain hardware bits like the
watchdog.

Provide a basic platform glue that implements the needed SMC forwarding.

The format of these calls are as follows:
 - reg 0: service ID
 - reg 1: function ID
 remaining regs: args

For now we only allow Dom0 to make these calls as they are all managing
hardware. There is no specification for these SIP calls, the IDs and names
have been extracted from the upstream linux kernel and the vendor kernel.

Most of the SIP calls are only available for the iMX8M series of SoCs, so
they are easy to reject and they need to be revisited when iMX8M series
support is added.

>From the other calls we can reject CPUFREQ because Dom0 cannot make an
informed decision regarding CPU frequency scaling, WAKEUP_SRC is to wake
up from suspend, which Xen doesn't support at this time.

This leaves the TIME SIP and the OTP_WRITE SIP, which for now are allowed
to Dom0.

NOTE: This code is based on code found in NXP Xen tree located here:
https://github.com/nxp-imx/imx-xen/blob/lf-5.10.y_4.13/xen/arch/arm/platfor=
ms/imx8qm.c

Signed-off-by: Peng Fan <peng.fan@nxp.com>
[jernberg: Add SIP call filtering]
Signed-off-by: John Ernberg <john.ernberg@actia.se>

---

v2:
 - Reword the commit message to be a bit clearer
 - Include the link previously added as a context note to the commit messag=
e (Julien Grall)
 - Add Pengs signed off (Julien Grall, Peng Fan)
 - Add basic SIP call filter (Julien Grall)
 - Expand the commit message a whole bunch because of the changes to the co=
de
---
 xen/arch/arm/platforms/Makefile |   1 +
 xen/arch/arm/platforms/imx8qm.c | 143 ++++++++++++++++++++++++++++++++
 2 files changed, 144 insertions(+)
 create mode 100644 xen/arch/arm/platforms/imx8qm.c

diff --git a/xen/arch/arm/platforms/Makefile b/xen/arch/arm/platforms/Makef=
ile
index 8632f4115f..bec6e55d1f 100644
--- a/xen/arch/arm/platforms/Makefile
+++ b/xen/arch/arm/platforms/Makefile
@@ -9,5 +9,6 @@ obj-$(CONFIG_ALL_PLAT)   +=3D sunxi.o
 obj-$(CONFIG_ALL64_PLAT) +=3D thunderx.o
 obj-$(CONFIG_ALL64_PLAT) +=3D xgene-storm.o
 obj-$(CONFIG_ALL64_PLAT) +=3D brcm-raspberry-pi.o
+obj-$(CONFIG_ALL64_PLAT) +=3D imx8qm.o
 obj-$(CONFIG_MPSOC_PLATFORM)  +=3D xilinx-zynqmp.o
 obj-$(CONFIG_MPSOC_PLATFORM)  +=3D xilinx-zynqmp-eemi.o
diff --git a/xen/arch/arm/platforms/imx8qm.c b/xen/arch/arm/platforms/imx8q=
m.c
new file mode 100644
index 0000000000..4515c75935
--- /dev/null
+++ b/xen/arch/arm/platforms/imx8qm.c
@@ -0,0 +1,143 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * xen/arch/arm/platforms/imx8qm.c
+ *
+ * i.MX 8QM setup
+ *
+ * Copyright (c) 2016 Freescale Inc.
+ * Copyright 2018-2019 NXP
+ *
+ *
+ * Peng Fan <peng.fan@nxp.com>
+ */
+
+#include <xen/sched.h>
+#include <asm/platform.h>
+#include <asm/smccc.h>
+
+static const char * const imx8qm_dt_compat[] __initconst =3D
+{
+    "fsl,imx8qm",
+    "fsl,imx8qxp",
+    NULL
+};
+
+#define IMX_SIP_GPC          0xC2000000
+#define IMX_SIP_CPUFREQ      0xC2000001
+#define IMX_SIP_TIME         0xC2000002
+#define IMX_SIP_DDR_DVFS     0xC2000004
+#define IMX_SIP_SRC          0xC2000005
+#define IMX_SIP_GET_SOC_INFO 0xC2000006
+#define IMX_SIP_NOC          0xC2000008
+#define IMX_SIP_WAKEUP_SRC   0xC2000009
+#define IMX_SIP_OTP_WRITE    0xC200000B
+
+#define IMX_SIP_TIME_F_RTC_SET_TIME     0x00
+#define IMX_SIP_TIME_F_WDOG_START       0x01
+#define IMX_SIP_TIME_F_WDOG_STOP        0x02
+#define IMX_SIP_TIME_F_WDOG_SET_ACT     0x03
+#define IMX_SIP_TIME_F_WDOG_PING        0x04
+#define IMX_SIP_TIME_F_WDOG_SET_TIMEOUT 0x05
+#define IMX_SIP_TIME_F_WDOG_GET_STAT    0x06
+#define IMX_SIP_TIME_F_WDOG_SET_PRETIME 0x07
+
+static bool imx8qm_is_sip_time_call_ok(uint32_t function_id)
+{
+    switch ( function_id )
+    {
+    case IMX_SIP_TIME_F_RTC_SET_TIME:
+        return true;
+    case IMX_SIP_TIME_F_WDOG_START:
+    case IMX_SIP_TIME_F_WDOG_STOP:
+    case IMX_SIP_TIME_F_WDOG_SET_ACT:
+    case IMX_SIP_TIME_F_WDOG_PING:
+    case IMX_SIP_TIME_F_WDOG_SET_TIMEOUT:
+    case IMX_SIP_TIME_F_WDOG_GET_STAT:
+    case IMX_SIP_TIME_F_WDOG_SET_PRETIME:
+        return true;
+    default:
+        printk(XENLOG_WARNING "imx8qm: smc: time: Unknown function id %x\n=
", function_id);
+        return false;
+    }
+}
+
+static bool imx8qm_smc(struct cpu_user_regs *regs)
+{
+    uint32_t service_id =3D get_user_reg(regs, 0);
+    uint32_t function_id =3D get_user_reg(regs, 1);
+    struct arm_smccc_res res;
+
+    if ( !cpus_have_const_cap(ARM_SMCCC_1_1) )
+    {
+        printk_once(XENLOG_WARNING "no SMCCC 1.1 support. Disabling firmwa=
re calls\n");
+
+        return false;
+    }
+
+    /* Only hardware domain may use the SIP calls */
+    if ( !is_hardware_domain(current->domain) )
+    {
+        gprintk(XENLOG_WARNING, "imx8qm: smc: No access\n");
+        return false;
+    }
+
+    switch ( service_id )
+    {
+    case IMX_SIP_GPC: /* Only available on imx8m series */
+        return false;
+    case IMX_SIP_CPUFREQ: /* Dom0 can't take any informed descision here *=
/
+        return false;
+    case IMX_SIP_TIME:
+        if ( imx8qm_is_sip_time_call_ok(function_id) )
+            goto allow_call;
+        return false;
+    case IMX_SIP_DDR_DVFS: /* Only available on imx8m series */
+        return false;
+    case IMX_SIP_SRC: /* Only available on imx8m series */
+        return false;
+    case IMX_SIP_GET_SOC_INFO: /* Only available on imx8m series */
+        return false;
+    case IMX_SIP_NOC: /* Only available on imx8m series */
+        return false;
+    case IMX_SIP_WAKEUP_SRC: /* Xen doesn't have suspend support */
+        return false;
+    case IMX_SIP_OTP_WRITE:
+        /* function_id is the fuse number, no sensible check possible */
+        goto allow_call;
+    default:
+        printk(XENLOG_WARNING "imx8qm: smc: Unknown service id %x\n", serv=
ice_id);
+        return false;
+    }
+
+allow_call:
+    arm_smccc_1_1_smc(service_id,
+                      function_id,
+                      get_user_reg(regs, 2),
+                      get_user_reg(regs, 3),
+                      get_user_reg(regs, 4),
+                      get_user_reg(regs, 5),
+                      get_user_reg(regs, 6),
+                      get_user_reg(regs, 7),
+                      &res);
+
+    set_user_reg(regs, 0, res.a0);
+    set_user_reg(regs, 1, res.a1);
+    set_user_reg(regs, 2, res.a2);
+    set_user_reg(regs, 3, res.a3);
+
+    return true;
+}
+
+PLATFORM_START(imx8qm, "i.MX 8")
+    .compatible =3D imx8qm_dt_compat,
+    .smc =3D imx8qm_smc,
+PLATFORM_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--=20
2.43.0


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 16:07:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 16:07:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681056.1059627 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHmq-0001Hz-5L; Wed, 14 Feb 2024 16:07:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681056.1059627; Wed, 14 Feb 2024 16:07:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHmq-0001Ho-1w; Wed, 14 Feb 2024 16:07:04 +0000
Received: by outflank-mailman (input) for mailman id 681056;
 Wed, 14 Feb 2024 16:07:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xNvu=JX=actia.se=john.ernberg@srs-se1.protection.inumbo.net>)
 id 1raHmo-0000fn-Up
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 16:07:02 +0000
Received: from mail.actia.se (mail.actia.se [212.181.117.226])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 163d58a9-cb53-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 17:07:01 +0100 (CET)
Received: from S036ANL.actianordic.se (10.12.31.117) by S035ANL.actianordic.se
 (10.12.31.116) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb
 2024 17:07:00 +0100
Received: from S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69]) by
 S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69%4]) with mapi id
 15.01.2507.035; Wed, 14 Feb 2024 17:07:00 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 163d58a9-cb53-11ee-8a4d-1f161083a0e0
From: John Ernberg <john.ernberg@actia.se>
To: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	<michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Peng
 Fan" <peng.fan@nxp.com>, Jonas Blixt <jonas.blixt@actia.se>, John Ernberg
	<john.ernberg@actia.se>, Julien Grall <jgrall@amazon.com>
Subject: [PATCH v2 2/2] xen/drivers: imx-lpuart: Replace iMX8QM compatible
 with iMX8QXP
Thread-Topic: [PATCH v2 2/2] xen/drivers: imx-lpuart: Replace iMX8QM
 compatible with iMX8QXP
Thread-Index: AQHaX1/XC0kFM5mNB0K7IOFYEeejEA==
Date: Wed, 14 Feb 2024 16:06:59 +0000
Message-ID: <20240214160644.3418228-3-john.ernberg@actia.se>
References: <20240214160644.3418228-1-john.ernberg@actia.se>
In-Reply-To: <20240214160644.3418228-1-john.ernberg@actia.se>
Accept-Language: en-US, sv-SE
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: git-send-email 2.43.0
x-originating-ip: [10.12.12.30]
x-esetresult: clean, is OK
x-esetid: 37303A2921D72955607464
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Allow the uart to probe also with iMX8QXP. The ip-block is the same as in
the QM.

Since the fsl,imx8qm-lpuart compatible in Linux exists in name only and is
not used in the driver any iMX8QM device tree that can boot Linux must set
fsl,imx8qxp-lpuart compatible as well as the QM one.

Thus we replace the compatible rather than adding just another one.

Signed-off-by: John Ernberg <john.ernberg@actia.se>
Acked-by: Julien Grall <jgrall@amazon.com>

---

v2:
 - Replace the compatible rather than adding to the list (Julien Grall)
 - Reword commit message to reflect the above.
 - Collect Julien's ack
---
 xen/drivers/char/imx-lpuart.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c
index 77f70c2719..7a9c2e3878 100644
--- a/xen/drivers/char/imx-lpuart.c
+++ b/xen/drivers/char/imx-lpuart.c
@@ -256,7 +256,7 @@ static int __init imx_lpuart_init(struct dt_device_node=
 *dev,
=20
 static const struct dt_device_match imx_lpuart_dt_compat[] __initconst =3D
 {
-    DT_MATCH_COMPATIBLE("fsl,imx8qm-lpuart"),
+    DT_MATCH_COMPATIBLE("fsl,imx8qxp-lpuart"),
     { /* sentinel */ },
 };
=20
--=20
2.43.0


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 16:10:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 16:10:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681067.1059637 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHpr-0003YF-Ji; Wed, 14 Feb 2024 16:10:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681067.1059637; Wed, 14 Feb 2024 16:10:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHpr-0003Y8-FV; Wed, 14 Feb 2024 16:10:11 +0000
Received: by outflank-mailman (input) for mailman id 681067;
 Wed, 14 Feb 2024 16:10:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=83ZH=JX=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1raHpq-0003Xu-1d
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 16:10:10 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 85d7b54c-cb53-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 17:10:08 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id CDA7A4EE0737;
 Wed, 14 Feb 2024 17:10:07 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85d7b54c-cb53-11ee-8a4d-1f161083a0e0
MIME-Version: 1.0
Date: Wed, 14 Feb 2024 17:10:07 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Simone Ballarin <simone.ballarin@bugseng.com>, Doug
 Goldstein <cardoe@cardoe.com>, George Dunlap <george.dunlap@citrix.com>, Wei
 Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH] automation/eclair_analysis: deviate certain macros
 for Rule 20.12
In-Reply-To: <447b1843-f28c-4633-9e74-d695bc37208a@suse.com>
References: <892be5c403ca7a20b35fb9facacb6a38bc7f6bfe.1707900742.git.nicola.vetrini@bugseng.com>
 <3519db82-6126-4aa0-9d04-795edf6f2bca@suse.com>
 <b87f53b8e39fdbd7f7aefe63f227fe7f@bugseng.com>
 <447b1843-f28c-4633-9e74-d695bc37208a@suse.com>
Message-ID: <0733c2e990bca64f4c9a8a47a05b4795@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-14 16:45, Jan Beulich wrote:
> On 14.02.2024 16:31, Nicola Vetrini wrote:
>> On 2024-02-14 12:49, Jan Beulich wrote:
>>> On 14.02.2024 12:26, Nicola Vetrini wrote:
>>>> +-config=MC3R1.R20.12,macros+={deliberate,
>>>> "name(ASSERT||BUILD_BUG_ON||BUILD_BUG_ON_ZERO||GENERATE_CASE)"}
>>> 
>>> I said in another context already that it is necessary to separate
>>> global scope macros from local helper ones. Any CU can introduce
>>> another
>>> GENERATE_CASE(), and would be deviated here right away. In fact I
>>> question applicability of the deviation to arm/arm64/vsysreg.c; I 
>>> only
>>> see it as applicable to arm/vcpreg.c.
>> 
>> Why wouldn't this be applicable for vsysreg.c? I can certainly 
>> fine-tune
>> the deviation, if needed
> 
> There the sole macro parameter is used in only one of the possible 
> ways,
> with ##, unless I'm overlooking something. Aiui it is mixed use which
> would need a deviation.

I see. In that case I'll restrict the deviation, thanks.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 16:11:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 16:11:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681074.1059655 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHqr-0004C3-7X; Wed, 14 Feb 2024 16:11:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681074.1059655; Wed, 14 Feb 2024 16:11:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raHqr-0004BR-1x; Wed, 14 Feb 2024 16:11:13 +0000
Received: by outflank-mailman (input) for mailman id 681074;
 Wed, 14 Feb 2024 16:11:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3HRx=JX=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1raHqp-0004Au-Jb
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 16:11:11 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id aacd8894-cb53-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 17:11:10 +0100 (CET)
Received: from [192.168.1.229] (93-36-216-194.ip62.fastwebnet.it
 [93.36.216.194])
 by support.bugseng.com (Postfix) with ESMTPSA id 0A2EF4EE0737;
 Wed, 14 Feb 2024 17:11:10 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aacd8894-cb53-11ee-8a4d-1f161083a0e0
Message-ID: <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
Date: Wed, 14 Feb 2024 17:11:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US, it
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
 <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
 <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
 <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com>
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 12/02/24 09:43, Jan Beulich wrote:
> On 09.02.2024 10:50, Federico Serafini wrote:
>> On 08/02/24 12:14, Jan Beulich wrote:
>>> On 08.02.2024 11:45, Federico Serafini wrote:
>>>> On 07/02/24 17:19, Jan Beulich wrote:
>>>>> On 07.02.2024 16:58, Federico Serafini wrote:
>>>>>> On 07/02/24 16:24, Jan Beulich wrote:
>>>>>>> On 07.02.2024 16:08, Federico Serafini wrote:
>>>>>>>> On 07/02/24 15:16, Jan Beulich wrote:
>>>>>>>>> On 07.02.2024 14:51, Federico Serafini wrote:
>>>>>>>>>> On 07/02/24 08:38, Jan Beulich wrote:
>>>>>>>>>>> On 07.02.2024 02:08, Stefano Stabellini wrote:
>>>>>>>>>>>> On Tue, 6 Feb 2024, Jan Beulich wrote:
>>>>>>>>>>>>> On 26.01.2024 11:05, Federico Serafini wrote:
>>>>>>>>>>>>>> @@ -208,7 +205,7 @@ do {                                                                       \
>>>>>>>>>>>>>>            case 8:                                                                \
>>>>>>>>>>>>>>                put_unsafe_asm(x, ptr, grd, retval, "q",  "", "ir", errret);       \
>>>>>>>>>>>>>>                break;                                                             \
>>>>>>>>>>>>>> -    default: __put_user_bad();                                             \
>>>>>>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>>>>>>            }                                                                      \
>>>>>>>>>>>>>>            clac();                                                                \
>>>>>>>>>>>>>>        } while ( false )
>>>>>>>>>>>>>> @@ -227,7 +224,7 @@ do {                                                                       \
>>>>>>>>>>>>>>            case 2: get_unsafe_asm(x, ptr, grd, retval, "w", "=r", errret); break; \
>>>>>>>>>>>>>>            case 4: get_unsafe_asm(x, ptr, grd, retval, "k", "=r", errret); break; \
>>>>>>>>>>>>>>            case 8: get_unsafe_asm(x, ptr, grd, retval,  "", "=r", errret); break; \
>>>>>>>>>>>>>> -    default: __get_user_bad();                                             \
>>>>>>>>>>>>>> +    default: STATIC_ASSERT_UNREACHABLE();                                  \
>>>>>>>>>>>>>>            }                                                                      \
>>>>>>>>>>>>>>            clac();                                                                \
>>>>>>>>>>>>>>        } while ( false )
>>>>>>>>>>>>>
>>>>>>>>>>>>> Related to my remark on patch 1 - how is one to know the macro this was
>>>>>>>>>>>>> invoked from, when seeing the resulting diagnostic?
>>>>>>>>>>>>
>>>>>>>>>>>> I am not sure what do you mean here... we do get an error like the
>>>>>>>>>>>> following (I added a STATIC_ASSERT_UNREACHABLE for case 4):
>>>>>>>>>>>>
>>>>>>>>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>>>>>>>>
>>>>>>>>>>> Right - and how do I know what _user_ of the macro actually triggered
>>>>>>>>>>> it? ISTR suggesting to use one or more of __FILE__ / __LINE__ /
>>>>>>>>>>> __FUNCTION__ here, for that specific purpose ...
>>>>>>>>>>
>>>>>>>>>> To test the macro and its diagnostics,
>>>>>>>>>> I modified the first "git grep" occurrence of ASSERT_UNREACHABLE()
>>>>>>>>>> on the x86 code with STATIC_ASSERT_UNREACHABLE(),
>>>>>>>>>> that is in file arch/x86/alternative.c, line 312,
>>>>>>>>>> function _apply_alternatives().
>>>>>>>>>>
>>>>>>>>>> What I got is the following build error:
>>>>>>>>>>
>>>>>>>>>> ...
>>>>>>>>>> arch/x86/alternative.c: Assembler messages:
>>>>>>>>>> arch/x86/alternative.c:312: Error: static assertion failed: unreachable
>>>>>>>>>>         CC      arch/x86/copy_page.o
>>>>>>>>>> make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error 1
>>>>>>>>>
>>>>>>>>> But that's not what my request was about. Here sufficient context is
>>>>>>>>> given, even if it would be nice if the function was also visible right
>>>>>>>>> away. But that's not the same as the case above, where the new macro
>>>>>>>>> is used inside another macro.
>>>>>>>>
>>>>>>>> An example of that is the get_unsafe_size() macro,
>>>>>>>> whose body uses STATIC_ASSERT_UNREACHABLE().
>>>>>>>> A wrong use of get_unsafe_size() at line n
>>>>>>>> leads to a build error pointing to the line n,
>>>>>>>> isn't this the desired behavior?
>>>>>>>
>>>>>>> Aiui this would point to the line in the header file, when what you need
>>>>>>> to spot the bad use of the macro is the line in the source file actually
>>>>>>> using the macro. Quoting from an earlier mail of yours:
>>>>>>>
>>>>>>> ./arch/x86/include/asm/uaccess.h:262: Error: static assertion failed: unreachable
>>>>>>
>>>>>> It points to the header file uaccess.h because at line 262 there is
>>>>>> an intentional wrong use of put_guest_size(), within the body of
>>>>>> __copy_to_guest_pv() function.
>>>>>
>>>>> Yet that's again only a helper function being inlined into the ultimate
>>>>> caller. That ultimate caller is what wants identifying in the diag. Not
>>>>> the least because of ...
>>>>>
>>>>>> This example can be misleading because {get,put}_unsafe_size() are
>>>>>> defined in the same file but the diagnostics is doing the
>>>>>> right thing.
>>>>>
>>>>> ... this. And really __copy_to_guest_pv() is the wrong place to put a
>>>>> wrong put_guest_size() in, to try out how diagnostics would look like
>>>>> in reality: That function falls back to copy_to_guest_ll() for all
>>>>> cases it can't handle directly. You want to place a bogus put_guest()
>>>>> somewhere in a .c file to see what results.
>>>>
>>>> I added a bogus call to put_guest() at line 387 of
>>>> file xen/arch/x86/mm.c, inside function page_is_ram_type().
>>>> Assuming I did not choose another wrong place,
>>>> the diagnostic seems appropriate:
>>>>
>>>> arch/x86/mm.c: Assembler messages:
>>>> arch/x86/mm.c:387: Error: static assertion failed: unreachable
>>>
>>> Oh, okay, this looks appropriate then as to identifying where the
>>> source construct is. However, we then still don't know where the
>>> assertion in question is (there could be multiple in what the
>>> original construct expands to). So I'm still inclined to ask that
>>> __FILE__ / __LINE__ and/or the name of the invoking construct
>>> (macro or function) be made visible in the diagnostic.
>>
>> Any use of __FILE__ and __LINE__ results in obtaining
>> the same information already reported by the assembler error message.
> 
> Hmm, yes, since put_guest() is itself a macro.
> 
>> We could add an argument to the new macro to manually add some context
>> at every use of the macro, but I think this would be annoying.
> 
> That's a last resort. An alternative would be to see about converting
> from macros to inline functions, where this would make a difference
> here.

I did some tries with example programs
and the assembler error always points to file and line
of the most enclosing function that caused the failure.
If I am not missing something, using __FILE__ and __LINE__ does not add
any information.

Therefore, if the new macro is used within the body of other macros,
then the resulting assembler error will point to the source of
the problem (e.g., the site of a bogus call to put_guest()).

In my opinion, converting put_guest() &Co. to inline functions is not
convenient: the assembler error will point to the most enclosing
function that would be put_unsafe_size(), instead of pointing to the
source of the problem.

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 17:13:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 17:13:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681088.1059665 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raIoi-0004Cw-RP; Wed, 14 Feb 2024 17:13:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681088.1059665; Wed, 14 Feb 2024 17:13:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raIoi-0004Cp-ON; Wed, 14 Feb 2024 17:13:04 +0000
Received: by outflank-mailman (input) for mailman id 681088;
 Wed, 14 Feb 2024 17:13:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hroU=JX=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1raIoh-0004Cj-9J
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 17:13:03 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4e639de0-cb5c-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 18:13:01 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-411d231ea71so12515475e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 09:13:01 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 o14-20020a5d684e000000b0033cf4727a46sm762932wrw.25.2024.02.14.09.12.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 09:13:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e639de0-cb5c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707930780; x=1708535580; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=7UKpQCyWcv7wnt3+/fGe9aC//4icjIiWztF/pEJGKqQ=;
        b=NeuySOY+IW7w8TLslWg4XDZ9XenIBjvMuhy3jtwg5P76oWqjJOMH5/OH4P6I/LrA3K
         FRl3inxK3ytCb6H6HBeS4X46VE5yne+042IiPEE7kU5KY8Wnx2Ro9+ez4YR0fDcKIPcE
         GS/AOaS/7T66jw5ueBZeRVZAZGvJlMccYQt7U=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707930780; x=1708535580;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7UKpQCyWcv7wnt3+/fGe9aC//4icjIiWztF/pEJGKqQ=;
        b=V12eU/XhPZfseDFRDdRJCLy3CEP6mylt/6C4az5pHD2bfGVPxun7f74h33qI5n0e1/
         W5PgloF97FdVkzyGnw9ZtITfP9qKOEunWmV17517VB8wdAbXZmAtZE4tbCJigFo/KXJk
         ZPiN+gIM/TKJWb4FDJyN1kyGzMcYsbKsU7509YbgsUEpwt2IrI8ul4HI8xzmJYu7ZqdZ
         3dQOvuT6o+HGeSHtIa7rdQqOwvFosw39x4+n2gpUsw06lyILvmD7PW8ppH60kqME7fUd
         tS4VSz2FSJhtsjMiJh3/hRhQPE0QQcYRtdlroprIYbM4JCR7rn/xNliE+8JexhNxxW2l
         T5Tg==
X-Gm-Message-State: AOJu0YxERbuxPGTd2C85oxzB3viSgnM0h8fLOd1D2ujDQGO1DZbD1p8q
	MQ4we0z2DQe+yc14vkMev1hyBO4Dw8dxwofHgoe2y94pjNb1PRnbk40JjyivmwU=
X-Google-Smtp-Source: AGHT+IEA51Z61t8WM/hpI8eJDUqTxNiktdX0cB8JdpQY3OlFQRL15XPUPegiOA2JpRjggIqgeY4WKg==
X-Received: by 2002:a05:600c:474c:b0:411:de5c:8abe with SMTP id w12-20020a05600c474c00b00411de5c8abemr2524354wmo.1.1707930780427;
        Wed, 14 Feb 2024 09:13:00 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWsJrmCaeEK8Y8dL0icIRmymZrxFKqsIq7ZcExI9SAB795YcH4FqO2We5lt/lGhdFNsHAgnnz+NMAi6GsxKIVjXpje883JySx/RQEXCqJc=
Date: Wed, 14 Feb 2024 17:12:59 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: Re: [PATCH v5 16/22] tools/xl: support new 9pfs backend xen_9pfsd
Message-ID: <678e5655-8f17-48b6-9a87-a3647aa61de9@perard>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-17-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240208165546.5715-17-jgross@suse.com>

On Thu, Feb 08, 2024 at 05:55:40PM +0100, Juergen Gross wrote:
> Add support for the new 9pfs backend "xen_9pfsd". For this backend type
> the tag defaults to "Xen" and the host side path to
> "/var/log/xen/guests/<dom-name>".
> 
> Do most of the default settings in libxl. Unfortunately the default
> path can't easily be set in libxl, as the domain name isn't available
> in the related 9pfs specific function.

It might be possible to do that in libxl__domain_config_setdefault(), or
maybe in initiate_domain_create(), but I guess that's fine to set the
default path in xl. Users of libxl might want to have a different
default path.

> Settings the defaults in libxl requires to move the sanity checking
> of 9pfs parameters from xl to libxl, too.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
> ---
> V2:
> - test max_files and max_open_files, too (Jason Andryuk)
> V4:
> - fix man page to use the "xen_9pfsd" type due to idl limitation
>   (Jason Andryuk)
> - set (most of) the defaults in libxl (Anthony Perard)

Patch looks good to me now:
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 17:19:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 17:19:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681092.1059674 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raIv4-0004qj-FY; Wed, 14 Feb 2024 17:19:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681092.1059674; Wed, 14 Feb 2024 17:19:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raIv4-0004qc-CK; Wed, 14 Feb 2024 17:19:38 +0000
Received: by outflank-mailman (input) for mailman id 681092;
 Wed, 14 Feb 2024 17:19:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hroU=JX=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1raIv3-0004qT-6o
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 17:19:37 +0000
Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com
 [2a00:1450:4864:20::436])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3a1370e4-cb5d-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 18:19:36 +0100 (CET)
Received: by mail-wr1-x436.google.com with SMTP id
 ffacd0b85a97d-33ce55ab993so679705f8f.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 09:19:36 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 bn8-20020a056000060800b0033b198efbedsm13229061wrb.15.2024.02.14.09.19.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 09:19:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a1370e4-cb5d-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707931176; x=1708535976; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=iQcbDB//Q5fEWkhqNQ+NEk9bkYANdAUf2ims+W6W1y8=;
        b=fJ5fdTuDmdzmEuaFe7xNp8+TIOmT4OwYuUGAjip67uIUTo8HA82U3IvDqASblHCjFP
         Pz71g6JCom7py9b6VTFbCeBfoy7BlV/gO1MJtUvs9sQijvo+tQeKwzxy8/32iMdE2CjB
         9p79rvPkv0KPXxo/PRWgX5UaYtCjBXL0WNs2o=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707931176; x=1708535976;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iQcbDB//Q5fEWkhqNQ+NEk9bkYANdAUf2ims+W6W1y8=;
        b=ZD6HG0V3izJRsQ3++wp1XX5bjjFqd/On5NP7oG1++/ez9bssrecn1M1lLMMUFsQql1
         uwrQPBFWJP2xX+vwSy2Tq9MKzOIABQO7pZAhB7/PZRyL6QkHbZusu+OeD+s1TLPt8SCk
         6caKB/wHK0IHrKCjtCdKLUyMWyWjfU3ZOApDe/+FP15brI2MUcBxdyV3A4LBvTzEXPhe
         ePAaFtsgaRBM10j4nT50kj3UyJ5pbG6A8jivK0ZhbuEuewvOfk5Cp/fcsvPjwBLGpCeu
         C1U26Fc1/4U/rjKLvgEutSO4afhPCRDnIm6lqA+in5nwbe+TnDaUBrNZuF13znrFSW4w
         UNUA==
X-Gm-Message-State: AOJu0Yx4lBiNJkmugmxXY25IvkV1s72qb2AwniWEdl+wDeYk9p0QESHQ
	hAGRJk31Jfk5Y7V8lRPIaUpBMpyYwtIRMZ5ayeSPChOOY0+LpyqEZBIm36vWrnQ=
X-Google-Smtp-Source: AGHT+IGQHujsKd+TkpPpdAStmas1ncFYEdPaB6NCKbc9x79pcuCfNA5h/2zMDikzymk1wT4XvoUjxA==
X-Received: by 2002:adf:e683:0:b0:33c:df3e:a5a3 with SMTP id r3-20020adfe683000000b0033cdf3ea5a3mr2004566wrm.69.1707931175808;
        Wed, 14 Feb 2024 09:19:35 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUDZL+ThuTqpkUQyCXiL4xr9Pg6MhBU4aIwXth0DOsXZWGT80OjWXvyTAwhFTMHiuI8RO7U6sOe4PdivF/3dvmZ8olU/b87VU/J4g==
Date: Wed, 14 Feb 2024 17:19:35 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>
Subject: Re: [PATCH v5 18/22] tools: add 9pfs device to xenstore-stubdom
Message-ID: <1d87da18-6dcc-4eae-8453-3eef4ffee9c9@perard>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-19-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240208165546.5715-19-jgross@suse.com>

On Thu, Feb 08, 2024 at 05:55:42PM +0100, Juergen Gross wrote:
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index a554f2ccd6..e6ab5ddb94 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -583,6 +583,13 @@
>   * libxl_console_add_xenstore() in libxl.
>   */
>  #define LIBXL_HAVE_CONSOLE_ADD_XENSTORE 1
> +
> +/*
> + * LIBXL_HAVE_P9_ADD_XENSTORE indicates presence of the function
> + * libxl_device_9pfs_add() in libxl.
> + */
> +#define LIBXL_HAVE_P9_ADD_XENSTORE 1

Nit, we could probably remove the _XENSTORE suffix from the define, but
it's ok to keep it as well, just to avoid a round trip.

In any case, patch looks good:
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 17:22:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 17:22:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681097.1059684 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raIxW-0006P1-Qs; Wed, 14 Feb 2024 17:22:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681097.1059684; Wed, 14 Feb 2024 17:22:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raIxW-0006Ou-OK; Wed, 14 Feb 2024 17:22:10 +0000
Received: by outflank-mailman (input) for mailman id 681097;
 Wed, 14 Feb 2024 17:22:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hroU=JX=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1raIxU-0006Oj-Qv
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 17:22:08 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 93cede92-cb5d-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 18:22:06 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-562178003a1so1625415a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 09:22:06 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 tx27-20020a1709078e9b00b00a3d4f282cb3sm699632ejc.142.2024.02.14.09.22.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 09:22:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93cede92-cb5d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707931326; x=1708536126; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=YuH53H9XoLYsAVEgEXhUxfFUh+6OlXcG/Rp2nhnU/0M=;
        b=hs2+l0vlk6InyBmXcB4ZiMiKQiC6xBNNYKuGQ0DkOKU8UFP+4UQ8Q9alvgJST4wMZC
         fgaxn5q+6yd4ta1b7l8wnIKaF7QmH4zcDR/6oEV/9ME8sdOOu7ct9iuRF3Lox9WAhH1R
         MfKZommxNAcSIMTFHMCsciGNJlN5LtDlEvPxg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707931326; x=1708536126;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YuH53H9XoLYsAVEgEXhUxfFUh+6OlXcG/Rp2nhnU/0M=;
        b=XGV5OgDTaAQ/sZSOh198ml7FAFRRztAwPQC9LgunJhA+CqAazKNYMyA2CF3JgAzYel
         g7UuZFWxoQf/YfiUJxhDXQy/x/sdNX5qUVHo6HF/n6cOJ9y8H0tBwYrIGXDc1N+l0s1M
         KsZZCQ1cM/fKgPYinkKUasdNhDeTrergCeZEqRv+urokAru4lDLP5KXbxxEDdO7ksCCl
         7melbIayf6p+wlh+xXCX0ENhNQpq1IN4M2+qZgPkog3fs3P/w4hUGWgamk2zS+9XbFm1
         pGPSsf9MZlcQ/0xeg/2tyrqn43Ob8EdFX4LFsPxeCyLtEGbNgi5NfrF8YF8jUuNydHk8
         RFGw==
X-Gm-Message-State: AOJu0Yz5WW+6ej/urcntz78qJQbxxRR9wwqZGuoc3HTdky2KSlTP5rTM
	hyeCj9x2WLfXfeaLCFt+KJFZsFeeycwLJvjzQbs0Ij/EwnfQ+C3Er1PYF5bZfyc=
X-Google-Smtp-Source: AGHT+IF/tfQEqh38OZc58enlDR6I+2+ljBZ8klwKoC7ABRhruMlGDAYxi5KlTorL+RwSKMe10XLybg==
X-Received: by 2002:a17:906:d95:b0:a3c:ea61:9359 with SMTP id m21-20020a1709060d9500b00a3cea619359mr1922273eji.77.1707931326434;
        Wed, 14 Feb 2024 09:22:06 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCWPsAPBJoggk9uu9JbXWwmXDf5ZkOyOhROzcPXfqOKGfVriL3/TvJjLflOqtnPb0foFFjLQBpaxCE9fEe7Qva3+mWW2DxNTWfCf118fj48GFPyY55NGHjM7SBJwaUmkq9GKlMhaADzOrGXKNHDfaEG0vA==
Date: Wed, 14 Feb 2024 17:22:05 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>, Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: Re: [PATCH v5 21/22] tools/xenstored: support complete log
 capabilities in stubdom
Message-ID: <b162a4c8-b113-4174-89e6-f0f9d8f1eae4@perard>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-22-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240208165546.5715-22-jgross@suse.com>

On Thu, Feb 08, 2024 at 05:55:45PM +0100, Juergen Gross wrote:
> With 9pfs being fully available in Xenstore-stubdom now, there is no
> reason to not fully support all logging capabilities in stubdom.
> 
> Open the logfile on stubdom only after the 9pfs file system has been
> mounted.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
> Reviewed-by: Julien Grall <jgrall@amazon.com>

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 17:26:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 17:26:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681102.1059695 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJ1M-0006zs-Bu; Wed, 14 Feb 2024 17:26:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681102.1059695; Wed, 14 Feb 2024 17:26:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJ1M-0006zl-7Z; Wed, 14 Feb 2024 17:26:08 +0000
Received: by outflank-mailman (input) for mailman id 681102;
 Wed, 14 Feb 2024 17:26:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raJ1L-0006zb-84; Wed, 14 Feb 2024 17:26:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raJ1L-0000T9-3S; Wed, 14 Feb 2024 17:26:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raJ1K-0004Qw-PO; Wed, 14 Feb 2024 17:26:06 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raJ1K-0000xn-Ow; Wed, 14 Feb 2024 17:26:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=10ukcym1FCa8BPrKAEpgVEJmGNuaiRjZPve2FxtLLv0=; b=Cpvw3zy9aaHNfhGFJRsgwhhnWc
	xIP2z9RI0tt5K2QIA8ghUfStibkmUcbZ8/Kejdu0iXDlzrRFq/rMqjZ5AktlOPV0s3mAOZLFJuIQU
	2c0hrBhbni2e9i3oq/irctIwFqW3Wjwei8+WvfBBPnvOVDIfEFL+dlupdaOhDdu2kB10=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184667-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184667: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=d9bfc2810e1cbe4cd27555bdc48315d36ee91f14
X-Osstest-Versions-That:
    xen=d670c1a38ba3561296f68c0079209365760b3001
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 14 Feb 2024 17:26:06 +0000

flight 184667 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184667/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  d9bfc2810e1cbe4cd27555bdc48315d36ee91f14
baseline version:
 xen                  d670c1a38ba3561296f68c0079209365760b3001

Last test of basis   184666  2024-02-14 10:04:10 Z    0 days
Testing same since   184667  2024-02-14 15:00:24 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   d670c1a38b..d9bfc2810e  d9bfc2810e1cbe4cd27555bdc48315d36ee91f14 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 17:27:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 17:27:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681107.1059705 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJ34-0007W3-Lz; Wed, 14 Feb 2024 17:27:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681107.1059705; Wed, 14 Feb 2024 17:27:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJ34-0007Vw-Ii; Wed, 14 Feb 2024 17:27:54 +0000
Received: by outflank-mailman (input) for mailman id 681107;
 Wed, 14 Feb 2024 17:27:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hroU=JX=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1raJ33-0007Vq-1l
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 17:27:53 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 61a16c62-cb5e-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 18:27:52 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-3394bec856fso659065f8f.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 09:27:52 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 fm7-20020a05600c0c0700b00411062c123esm2609487wmb.26.2024.02.14.09.27.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 09:27:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61a16c62-cb5e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707931671; x=1708536471; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=+IxHnV8/9K1qQ9cTZTKCjiDJk/W8DnR94KHlGZYkQ0M=;
        b=sJPW+cImXSJ6f1GprIJj48R08zWg7xoWTuhcQe2Rmzi6PiemiguKJhW7lDzWSewENC
         PzoMV2jl32LphAaFlaGZhe6OoHqafcYD8IQbfst7G4b307qCsaeBuvQ2ySLTSugEVA9o
         nlHvvFjV8p/zbXWRwPBQgYx/QqQo7BEeAioJk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707931671; x=1708536471;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+IxHnV8/9K1qQ9cTZTKCjiDJk/W8DnR94KHlGZYkQ0M=;
        b=CmybW1SbOMtIAKv/J31TGgFK9lsobbEY5771wIN3SmU7icZL1HjEKrjcaml9iqi81u
         NSJC2wodPgNF7N8Tq8T5x4jU3e0TVA4qtqfdiQd5+ELw2XMy1LFBQJjKJ2XPk2mr23rl
         X629vcQUzP5SnalaN+AZ7oVG0jaoOH9aOXboLQMSlkUbWYW0S5rptB34pUcEi1iT7Qra
         rwNfcB2Cc60mT/OMUZYeKrlcI8xWlQbeA+CNkyLTS8b86o2ugLMgGM2M6cY1rYtDH6IO
         4QNPVb9XnZ1bL0aQ+O2MQs4xhGNE2qhnv8RlvcNWOZw7HRrcwLOHwKFVsZuSr8saegV8
         I6XA==
X-Gm-Message-State: AOJu0Yylj0ZzLC3b5+TiLLadE1GKDsZ5eR8qNllrrcVJApjLhiSksQan
	A+VTvndj2KZ+CdG7HWAsGsIvJmPOQsnDeSwPDxWZlrN4Kbi7b5S89/PuSlK8XF4=
X-Google-Smtp-Source: AGHT+IEWpTGUH6XrpQRC5lTcS8bV31/yc5/WTMnxibCgSv+EyL6vBw25kdlGBui518ylwHEhHhVseQ==
X-Received: by 2002:a5d:4048:0:b0:33b:2138:623c with SMTP id w8-20020a5d4048000000b0033b2138623cmr2576466wrp.16.1707931671686;
        Wed, 14 Feb 2024 09:27:51 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX+eRRBXsyqDNpx2sZmEfNi0pFI7QkUWFr5OJ37smIpWL1sUtGWRinygUlMssD8YxzEmjQox3GCq906JRst3S3oqjCC+MoGT832nn0T3FSYvaHr5sKSTkPF0JoDa/1Qn7pPM9OGotAraVby1Yh08+HDQXrXz4jTjpFSWWEBAQ+HCfE/l09qC7huMJ8eKlk6ty6tkvCyzwE=
Date: Wed, 14 Feb 2024 17:27:50 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: =?iso-8859-1?Q?J=FCrgen_Gro=DF?= <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>, Wei Liu <wl@xen.org>,
	Jason Andryuk <jandryuk@gmail.com>,
	George Dunlap <george.dunlap@cloud.com>
Subject: Re: [PATCH v5 15/22] tools/libs/light: add backend type for 9pfs PV
 devices
Message-ID: <a3a0d9eb-97ae-4e5d-ad08-cfe5319836d7@perard>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-16-jgross@suse.com>
 <5e6893dd-5280-4a14-8d00-4ed76e8c8e98@perard>
 <c6af9dd1-01d4-48a2-86b9-ccb2967136f7@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <c6af9dd1-01d4-48a2-86b9-ccb2967136f7@suse.com>

On Wed, Feb 14, 2024 at 11:18:18AM +0100, Jrgen Gro wrote:
> On 13.02.24 19:03, Anthony PERARD wrote:
> > On Thu, Feb 08, 2024 at 05:55:39PM +0100, Juergen Gross wrote:
> > > +struct libxl__aop9_state {
> > > +    libxl__spawn_state spawn;
> > > +    libxl__ao_device *aodev;
> > > +    libxl_device_p9 p9;
> > > +    uint32_t domid;
> > > +    void (*callback)(libxl__egc *, libxl__aop9_state *, int);
> > 
> > This "callback" is never used, right?
> 
> Why do you think so?
> 
> In xen9pfsd_spawn() it is used:
> 
>   aop9->callback = xen9pfsd_spawn_outcome;

By never used, I mean that nothing is reading the value, their is no
"aop9->callback(egc, aop9, rc)" call.
It might have been useful if a caller of xen9pfsd_spawn() was actually
setting this field, but that's not an option here. And callbacks of
xen9pfsd_spawn() knows to call xen9pfsd_spawn_outcome() when done.

Cheers,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 17:40:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 17:40:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681116.1059714 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJFQ-0002HK-O2; Wed, 14 Feb 2024 17:40:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681116.1059714; Wed, 14 Feb 2024 17:40:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJFQ-0002HD-LW; Wed, 14 Feb 2024 17:40:40 +0000
Received: by outflank-mailman (input) for mailman id 681116;
 Wed, 14 Feb 2024 17:40:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Sn1=JX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1raJFQ-0002H7-4N
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 17:40:40 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2ab6d35d-cb60-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 18:40:39 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a3d01a9a9a2so232713166b.1
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 09:40:39 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 vi16-20020a170907d41000b00a3ce31d3ffdsm2365838ejc.93.2024.02.14.09.40.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 09:40:38 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ab6d35d-cb60-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707932438; x=1708537238; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6aQhl2xza4kDvBJT8FjINNxnWqWGuSlxm9Fy5soVkgw=;
        b=ad5+Q5bGEdnWshFtiLT9/RJyDkjH+Lu745byYwUS3D8WF6ywOvVtATbhwPDyMAaYHw
         KfU4ZilmWLj2rZRGfEEcKI5oKwR6BS2z6NlqQpC3qstAxKy0eUVQ7m0g0oq1TcTYZYuu
         NLa/OW9H11/ZGDkqdM1CKV8hv9PJYNTHSxlNE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707932438; x=1708537238;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6aQhl2xza4kDvBJT8FjINNxnWqWGuSlxm9Fy5soVkgw=;
        b=T1qbQZqTWHQWmULqPPMf0KvLPb+huIQCHzawI/iaCCOe53rtZSlwZzhvw7vxjLFgaW
         dVKEZWQgaJPkfboUGx2hvGFSAJeizRuVOKP0JXprMFw4jEgGZQ1B6XCo2hw9WKXdqHnw
         CPuEDwlWM5qEoF8G2qXto+f0INhgu2HQF1cZ5BS38i4C47JZx8sTKvPHaTZh3xmP+/ih
         5B9gz0aE/R9c92syWqnlbGNgdkVB0dll06A8B+cJr/DQNJf9srzwuJQE+Ofy0AGfrRJY
         OG4/c4hSGiFJqX/jgAJNrioI0g/AZ1cL63sycw/iw/Ne91082fFMLP2qPeLqlQaoUBTR
         CE2g==
X-Forwarded-Encrypted: i=1; AJvYcCWmBfXfj6jSv1LlN1KzhzZ6hStWiPt/fzOWNITqugYXKqAL0uLt8rKvLFXdNnAuSBPjAueDZQY0iQNcMs+2aI3nXr96pDeJpK2SRl4ObJk=
X-Gm-Message-State: AOJu0YwWMrRxK6QAhQFGiETM947Pn1nLB1aPOq96ytOugLurbdjDqrOg
	X/Hf9TTtlp5z2C5uP05fErQnCPfr/ot8e1zBqAZtoyJJ1GNNzI+DsRF4GAfBDiTtUilRCTFkIvi
	p
X-Google-Smtp-Source: AGHT+IHHHayKrVIUAC/GRYv65r4Qcoqr/R2fQt0T6z9BTgKrggS1InhOJvMWu3MQ5zpDeQko+V0XnQ==
X-Received: by 2002:a17:906:6606:b0:a3d:11a4:6f90 with SMTP id b6-20020a170906660600b00a3d11a46f90mr2166833ejp.43.1707932438534;
        Wed, 14 Feb 2024 09:40:38 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCVyP7Yi/LGMnS+Lq1JxwBm5l/IasMJpl/BxowRmPay0R/i7FL8J2tObgihqBmArhdW5jhMhHkLbJd+AQArSMFcU6BJwPM8ZlZEec1iV9NPhTD7uv2Z2xo8DuHlrv1nCUfYixenVHugV+4BacMkFieeXjE74KZ8mkyEU2Fs4OQ==
Message-ID: <a628abf7-a04d-4c32-ad49-a2fe073f3286@citrix.com>
Date: Wed, 14 Feb 2024 17:40:37 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 07/22] tools/9pfsd: add 9pfs attach request support
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jason Andryuk <jandryuk@gmail.com>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-8-jgross@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240208165546.5715-8-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08/02/2024 4:55 pm, Juergen Gross wrote:
> +static struct p9_fid *alloc_fid_mem(device *device, unsigned int fid,
> +                                    const char *path)
> +{
> +    struct p9_fid *fidp;
> +    size_t pathlen;
> +
> +    pathlen = strlen(path);
> +    fidp = calloc(sizeof(*fidp) + pathlen + 1, 1);
> +    if ( !fidp )
> +        return NULL;
> +
> +    fidp->fid = fid;
> +    strncpy(fidp->path, path, pathlen);
> +
> +    return fidp;
> +}

GitlabCI has something to say about this. 
https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1176787593

I think they're all variations of:

io.c: In function 'alloc_fid_mem.isra.8':
io.c:566:5: error: 'strncpy' output truncated before terminating nul
copying as many bytes from a string as its length
[-Werror=stringop-truncation]
     strncpy(fidp->path, path, pathlen);
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
io.c:560:15: note: length computed here
     pathlen = strlen(path);
               ^~~~~~~~~~~~

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 17:48:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 17:48:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681122.1059725 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJMs-0002zu-JZ; Wed, 14 Feb 2024 17:48:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681122.1059725; Wed, 14 Feb 2024 17:48:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJMs-0002zn-Gb; Wed, 14 Feb 2024 17:48:22 +0000
Received: by outflank-mailman (input) for mailman id 681122;
 Wed, 14 Feb 2024 17:48:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Sn1=JX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1raJMr-0002zf-TL
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 17:48:21 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3d641b48-cb61-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 18:48:19 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-511acd26c2bso950025e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 09:48:19 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 un8-20020a170907cb8800b00a3d0a094574sm1753623ejc.66.2024.02.14.09.48.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 09:48:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d641b48-cb61-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707932899; x=1708537699; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vxfOdf0wghA/C5YJTyJEu20TyJgflNsMVkLrPyiVqYM=;
        b=GkWZovQGopacuErvpLga9wkXIAXXqg/XXVUhcMSiQ6PNRuSCf+HVQOnoBuo/z0MIzY
         GN6WNU+n6KnVGW3Xukswqx7lmbgrunmSxGsdjZmzq5fWBfg86XFVcDLcfPTBOaZXswuj
         bFhqirTRBH+Lh/OzHxTZqG6r6/CFGzJ/4jIrg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707932899; x=1708537699;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vxfOdf0wghA/C5YJTyJEu20TyJgflNsMVkLrPyiVqYM=;
        b=eqZaJscPBs5Z03A+q+2+XfMjLtkhHob8pSEDHsR4atoOMTd5lLcOfg1yYHnONpNkyT
         UgNsdRwKX09VSFVdLTHo4rc5f3oVVgYDd3uUme1wKDh+O2Sw+MhGy9abe88MQFgMIPI8
         qWMaxnmvM25/utayEz1c1xX/Smv1QBCNRjE97Bp2UidoDgHB8E1QPf0FstKA8Z6Bc7Tq
         Z10k0bS3h8NIVlA7gGL+w1/h/3+HwQENWY9bVFaGE78/iZ8jIJiU6x42Jd8Dt8uP4fVS
         YtSJNKeISnKA/NJKEmXwG5Mtmwyg1zpZSkXVoclxjxnd+fY3Fcq7TZ4D5HVm0UsBmew5
         S1sw==
X-Forwarded-Encrypted: i=1; AJvYcCXxEPl2YVSu6fhbBxdPc4YWKP06hXv8bahALjkmXM7iru+8SEqxHwPfCXrlRANS6m8GqZU7s0l13INx6tchcgxCZ7m0pwPgG2rYgzMKH2g=
X-Gm-Message-State: AOJu0YzrPBZEVGzOayKYjxk3pIwf/MTF4TLgr+m8OXJdZYDc3xfX7/f5
	9vvpbrkIA7HFHgZzS17B4dL6/lDQJmNPRqkUbNyvcWRS/A1+UPMjPN7YrlncqCk=
X-Google-Smtp-Source: AGHT+IHcCsjGSYit/C03zWXaZ+zt34W2264AYLdpxnqSLBMBDoKwJuSmN+irHoRO6Jxb9OT/u5b2/g==
X-Received: by 2002:a05:6512:3b8f:b0:511:42b5:5616 with SMTP id g15-20020a0565123b8f00b0051142b55616mr3401152lfv.17.1707932899325;
        Wed, 14 Feb 2024 09:48:19 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUU6XyYERw9vJpORWmi4nDk9bV4b4yFrGZsy1IiB4ZDy0RD90kFYlKu1MHkcOIbzVNuq35CFh6fsZ3v7xCrZwm16CMGzVtvxSvRmQQHmKGBAnaOrgpE5W9GAq2d49DVJ+dndnF7tTZ2Cl/VsOvAUmIVsIiQg9t4Nvxx3r+l4A==
Message-ID: <4af65d74-b465-4f2b-9c4e-b1dab9a4ea7a@citrix.com>
Date: Wed, 14 Feb 2024 17:48:18 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 07/22] tools/9pfsd: add 9pfs attach request support
Content-Language: en-GB
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jason Andryuk <jandryuk@gmail.com>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-8-jgross@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240208165546.5715-8-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 08/02/2024 4:55 pm, Juergen Gross wrote:
> +static struct p9_fid *alloc_fid(device *device, unsigned int fid,
> +                                const char *path)
> +{
> +    struct p9_fid *fidp = NULL;
> +
> +    pthread_mutex_lock(&device->fid_mutex);
> +
> +    if ( find_fid(device, fid) )
> +    {
> +        errno = EBADFD;

Also, FreeBSD says no.

https://cirrus-ci.com/task/6634697753624576

io.c:580:17: error: use of undeclared identifier 'EBADFD'
        errno = EBADFD;
                ^
1 error generated.

Need to use EBADF.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 18:03:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 18:03:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681127.1059734 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJbC-00063y-RT; Wed, 14 Feb 2024 18:03:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681127.1059734; Wed, 14 Feb 2024 18:03:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJbC-00063r-Ol; Wed, 14 Feb 2024 18:03:10 +0000
Received: by outflank-mailman (input) for mailman id 681127;
 Wed, 14 Feb 2024 18:03:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hroU=JX=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1raJbB-00063l-FY
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 18:03:09 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4ec180c2-cb63-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 19:03:08 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a389ea940f1so692839366b.3
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 10:03:07 -0800 (PST)
Received: from localhost.localdomain ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 vo10-20020a170907a80a00b00a3cfd838f32sm1945011ejc.178.2024.02.14.10.03.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 14 Feb 2024 10:03:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ec180c2-cb63-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707933787; x=1708538587; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Xqnw7qzhukxFfthztomyRM3QJeXI218bAfnlsrQi+Vw=;
        b=XdRYBPi/HVnndhkmA9QSR+gKFc9VXaqa6d6kleweZ8bJtg8GsPBMMuDaVFLskjRN18
         pBhNvJZFOTlFANNb0J5DJeen4vlvGn0jM/v1M5AW3cCaj3FTiDfdhn+22wbkf9mIeTob
         STLrEUumUqhgAbauNrSUG3PT3msVXh0AGVQDg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707933787; x=1708538587;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Xqnw7qzhukxFfthztomyRM3QJeXI218bAfnlsrQi+Vw=;
        b=IvMp3GQ+Oj8Zpl+UeXin2YDzSXMd0iN7oEoTmf88hlzJ5WayN3znFvTX1Kv91IHG/s
         t720/xQGDKKEX7HttzUpC7LIXMBV9uukZPFtei2kiNxzf4uy70vIMe0Wjs/JHdlrIp0w
         sxOn4shVhYWvytNvnwcXHVqFeG0b0qMmzxvpz5MdjtzV8k1n46bGBV43fWmewg7oOq27
         LS+GmXcf5rlbBzwhz50IvQOqTaOEgkfLuke0DHgoqFwIcdZePOKtBNCC/87nm8udQtNz
         XNGsDOsW/DdScLiltFKltTLg2AHfQOqVDhNOnjxTu2YIRC9P+BZmEZ2st3lxakpJbuxe
         sq0g==
X-Gm-Message-State: AOJu0YyrrHcxemjzVTFNpOMcgUAe7SAWoZKQvVZL9JnAw2LgPZk6QNeB
	AdJ8pyeN8JiLgfJGwXNwUWZh78Vc5fhfBxxcp9OMb04FcV0KpRu7pErRHgbUGiGnT40fci1Y3gj
	J
X-Google-Smtp-Source: AGHT+IEAVzvrd5KoAqINQyYVTbYT321zhmHgHBRgOmlJfwbvml1mfCIdG+A7w2OYYdiMNArfJmnf7Q==
X-Received: by 2002:a17:906:eb50:b0:a3d:765:7bdc with SMTP id mc16-20020a170906eb5000b00a3d07657bdcmr2281592ejb.25.1707933786731;
        Wed, 14 Feb 2024 10:03:06 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCUTgblWqrgd79opw33d5uqe+v5ZQD1gL9JC+/97ThOvIRJaTKWYyhWaz/zDzkOVRlNNCrI3Q9JfK2H/k/2FFLqWj9SdPSm38zKOCJEZ1WTnuyPOBbsAOnYku50sktTL4yiMPdDJMQhmc6fFUe6KpEKxIN7bxABgOFyo3N7IEbYquAm4PWOxF43+oT3af7QQZO+ilDAwtY+HeuJXF6SUhEuuyI3e2B2ftrcbdeK4IAZ4MsxhA6D+2jWwH9qlAxEIkeIbtFeuBSeHX0kXqw==
From: Anthony PERARD <anthony.perard@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony.perard@citrix.com>,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH v2] build: Replace `which` with `command -v`
Date: Wed, 14 Feb 2024 18:00:39 +0000
Message-Id: <20240214180039.57527-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The `which` command is not standard, may not exist on the build host,
or may not behave as expected by the build system. It is recommended
to use `command -v` to find out if a command exist and have its path,
and it's part of a POSIX shell standard (at least, it seems to be
mandatory since IEEE Std 1003.1-2008, but was optional before).

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---

Notes:
    v2:
    - also replace it in xen/build.mk
    - mention that it's a mandatory feature since edition 2008.
    - some other rework of the commit message.

 xen/Makefile | 4 ++--
 xen/build.mk | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 21832d6402..767e47d6c7 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -25,8 +25,8 @@ export XEN_BUILD_HOST	:= $(shell hostname)
 endif
 
 # Best effort attempt to find a python interpreter, defaulting to Python 3 if
-# available.  Fall back to just `python` if `which` is nowhere to be found.
-PYTHON_INTERPRETER	:= $(word 1,$(shell which python3 python python2 2>/dev/null) python)
+# available.  Fall back to just `python`.
+PYTHON_INTERPRETER	:= $(word 1,$(shell command -v python3 || command -v python || command -v python2) python)
 export PYTHON		?= $(PYTHON_INTERPRETER)
 
 export CHECKPOLICY	?= checkpolicy
diff --git a/xen/build.mk b/xen/build.mk
index 26dd5a8e87..0f490ca71b 100644
--- a/xen/build.mk
+++ b/xen/build.mk
@@ -1,6 +1,6 @@
 quiet_cmd_banner = BANNER  $@
 define cmd_banner
-    if which figlet >/dev/null 2>&1 ; then \
+    if command -v figlet >/dev/null 2>&1 ; then \
 	echo " Xen $(XEN_FULLVERSION)" | figlet -f $< > $@.tmp; \
     else \
 	echo " Xen $(XEN_FULLVERSION)" > $@.tmp; \
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Wed Feb 14 18:06:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 18:06:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681130.1059745 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJec-0006qH-9x; Wed, 14 Feb 2024 18:06:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681130.1059745; Wed, 14 Feb 2024 18:06:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJec-0006qA-7M; Wed, 14 Feb 2024 18:06:42 +0000
Received: by outflank-mailman (input) for mailman id 681130;
 Wed, 14 Feb 2024 18:06:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Sn1=JX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1raJea-0006q2-K8
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 18:06:40 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cced29aa-cb63-11ee-8a4d-1f161083a0e0;
 Wed, 14 Feb 2024 19:06:39 +0100 (CET)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-5600c43caddso30573a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 10:06:39 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 eo14-20020a056402530e00b00561f0ce1c91sm1786848edb.31.2024.02.14.10.06.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 10:06:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cced29aa-cb63-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707933999; x=1708538799; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=37j6eElZ6gYL2fWuf2fnGhbQJvRalCcw79uaKd1ALlw=;
        b=JAc8LLh86WIxxku4lOApAMgb/C85k9X/zMTuveFRFDwo6n4icKp9oIsJOUAIV1LTOJ
         QWf3km481KJt0ieQIAD5uKazK4Q1MpUozuMIq/TrZhBJewvIvucrx4hA9WYbFVfPMeO9
         BYISmklRBWfUhRJhy84RBWL4leT/cMeYqm6Do=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707933999; x=1708538799;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=37j6eElZ6gYL2fWuf2fnGhbQJvRalCcw79uaKd1ALlw=;
        b=IUqo7M6Kx9bzV4USvzQF9+jGYcymKbqYEDuH/Btp7q2QB2PiDEh5YHWu6ZeUNDrrN7
         HAFPTR1jCiXSgKRR+DP/YH4SsdsOieeR/oWp7fSYhNYNlskTvqUqBYIG0U3cg0T8B8Ya
         1R6PSejocEglkelO7S5wAhRSLcwxQlflIG8LVA4n0Nl+VbfZm+18GCmD3tSaRmQ3R8rS
         zmP58f8QXdlDPx8E8Jp4wxvB1fcvC3djU71/uegu4NRzII4ZLnNNpeOf3kyy3vXjTRNj
         0iANeY4KuNhBd5DXys6iVmVBPAIn9BNAOpe5h7DD5Y5gTBMNW97o3rtY0kQDAw2Cco/t
         TQrQ==
X-Forwarded-Encrypted: i=1; AJvYcCVfkdBgfpSIHNg1Xf5fBU8wpP9SrSnDQC3kCQtL9FWFUC6tqQHSNvZEMxoPf1tjjqk6RRyPXkc/aPfI358NAg0RMemlo200uKq5M4/vxx0=
X-Gm-Message-State: AOJu0YyAn+CHiYxtBUt3W2Um83APa6eXCcjUxPdAkcTdo5S59bEpIABl
	p38C39Gii4jfNqhws1nDewjb3M94DNCXoWbzL0pZnA8VnoTWbfyw6TmNvQnMQ6A=
X-Google-Smtp-Source: AGHT+IGzt4oUARh6ueYiUVpLLVllsr4kP3w3oBbf9fwqwCZ2I4oM75yyOkTbYlKZ9wnR54OtBVVViQ==
X-Received: by 2002:a05:6402:1491:b0:562:f95:e501 with SMTP id e17-20020a056402149100b005620f95e501mr2381635edv.26.1707933999219;
        Wed, 14 Feb 2024 10:06:39 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCXo/MSza71iXwHT+vkWpnQm6ZwkM9qHl0NOkakjXvyemALHU/7ByWkGhEXkG9Y0SEIe3ki/l+Ax9A1SDP4eJbGclybSirec4qdgt8J4PWCzTzfXZ5qUrX37BeL/XKVsnWmHTEIjJ/lzwcRORwZgQ9Ns2aIHTfwH5ULo8xMQ2jGKBEp/xprc3VPhgBh3C2SqESrrqOM+VDO+xl9rFCPSHiKo4xVoTZygea3tQhOItCcM
Message-ID: <dca166f1-6b94-46bb-b576-411ef9cce127@citrix.com>
Date: Wed, 14 Feb 2024 18:06:38 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2] build: Replace `which` with `command -v`
Content-Language: en-GB
To: Anthony PERARD <anthony.perard@citrix.com>, xen-devel@lists.xenproject.org
Cc: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20240214180039.57527-1-anthony.perard@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240214180039.57527-1-anthony.perard@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14/02/2024 6:00 pm, Anthony PERARD wrote:
> The `which` command is not standard, may not exist on the build host,
> or may not behave as expected by the build system. It is recommended
> to use `command -v` to find out if a command exist and have its path,
> and it's part of a POSIX shell standard (at least, it seems to be
> mandatory since IEEE Std 1003.1-2008, but was optional before).
>
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

I expect we want fixes tags for c8a8645f1ef (python) and 3b47bcdb6d383
(figlet).  Both my fault it turns out...

Happy to fix on commit.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 18:15:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 18:15:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681136.1059754 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJmj-0000VA-3P; Wed, 14 Feb 2024 18:15:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681136.1059754; Wed, 14 Feb 2024 18:15:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raJmj-0000V3-0o; Wed, 14 Feb 2024 18:15:05 +0000
Received: by outflank-mailman (input) for mailman id 681136;
 Wed, 14 Feb 2024 18:15:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=4Sn1=JX=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1raJmh-0000Uu-PP
 for xen-devel@lists.xenproject.org; Wed, 14 Feb 2024 18:15:03 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f7d028da-cb64-11ee-98f5-efadbce2ee36;
 Wed, 14 Feb 2024 19:15:01 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a3d5e77cfbeso14120366b.0
 for <xen-devel@lists.xenproject.org>; Wed, 14 Feb 2024 10:15:01 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 vs2-20020a170907a58200b00a3d5b5221bbsm587020ejc.141.2024.02.14.10.14.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Feb 2024 10:15:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7d028da-cb64-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707934500; x=1708539300; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KiNv5W7Gl51WblfWE3TFKqwR1YncW99F/nk5SJm2mOI=;
        b=Hg0H++UNCmGYYTZW1Zl5F241dn9OI1Sy/dgsJPG7s9TZP1aKDzPlDcnftu0ErnDvLs
         5+oIO7FF2ON5dNvgkE0Hlq7ybE+rScIhgIDvngQeHXHADJ1dQM9/dSdLN8jIBYjZ0Zde
         V2qZwIl+tbZcdscjPQIztXWxCws2n/0cWoZ7c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707934500; x=1708539300;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KiNv5W7Gl51WblfWE3TFKqwR1YncW99F/nk5SJm2mOI=;
        b=sZ5TuHaGx5/dIHi8sK0enhEb+NuT8eDOAUw3Ix279tKhKlm1IRgP0g+rT+8cZc9CDa
         Z1LcWhP7I2/Tnzz1spmplkM641vwyXkftHBDdv/qLEX8ZTk9jv7PtURvK9h5pw1egyuB
         jexh445g2MZfbZE61As3QYok+W/kOuYyjgxKsVrG2y01hJ240MGl6dmaOZtfliOewlum
         2RXIG/6o/TLAunVaPYeqGfeiJr2xMtwZFan+FzjPXcDxxI8Z4oZqcNv1xqWw6w9FUN66
         VZ7onNhf/yUgDpUFXjdNWgS2eZc8pHYKgR+SkTO8D4AoEIOLrO4IqOtnOZC/4BZ+W4Ft
         DkUg==
X-Forwarded-Encrypted: i=1; AJvYcCW7fnJkuDeLdzQ6Hhg42gF7NonPXIUVoiaSYPCp6WswFQpSOKIDSiIYAXKs3gzTj27dKPiFGYp7MKJsM0kU/ufqlNjcQodyUaITqSecU50=
X-Gm-Message-State: AOJu0YyaLoJ0WW7188YXfjTExp7BvPeuAk1Q9Urpsrok2ms4rdtZFHg2
	URdvGXPRGPRETxvStSBqUx376seprbaQk2anzlBPQAVPP+dpteEUVEvNsGY2MRo=
X-Google-Smtp-Source: AGHT+IGQr9AYSCNGpfF0jsRcXPNupd0f8qz3oUzdVSQu/jaNPrwaZx5MpUKINwnu4QoVOo1r6SHYZw==
X-Received: by 2002:a17:906:ce2b:b0:a38:5b3c:7c9b with SMTP id sd11-20020a170906ce2b00b00a385b3c7c9bmr2622254ejb.18.1707934500614;
        Wed, 14 Feb 2024 10:15:00 -0800 (PST)
X-Forwarded-Encrypted: i=1; AJvYcCX1Ozpg+kqyWXYNMWuHBvH3WNB66Wci6cgRvadLAb6f4BJueS//7AaV8cH6cIAbQDuoKwaYF9NQzPVD1sEwnDEfVo9YAXTn0KpuL3gEBesy8/US1jK6DRrebH/qxeFnbH6ve5Q7wBtkdduiuMzBlR+4RH90NB4mjk5+ehDsS26ZFbaM7lt88u9uNO/NOyz2B2QsTBZIGgZV+fruIqG5JMMWRQwJ/Z6RowvbS8a1w6vU3+EQwmK/Lwms0bS8qFGnvWV2kdIzfH3kScF/x1xz7AZsvPO5MmZ/zN0ndKBNWGwNAkwS/8KZc2A0MOWai00=
Message-ID: <219340f5-8378-42aa-84cf-bbb0bb0fb486@citrix.com>
Date: Wed, 14 Feb 2024 18:14:59 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/altp2m: p2m_altp2m_get_or_propagate() should honor
 ap2m->default_access
Content-Language: en-GB
To: =?UTF-8?Q?Petr_Bene=C5=A1?= <w1benny@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Tamas K Lengyel <tamas@tklengyel.com>,
 Alexandru Isaila <aisaila@bitdefender.com>,
 Petre Pircalabu <ppircalabu@bitdefender.com>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <1b854c6b38787675294c58eea25556ce241b2b4f.1707213023.git.w1benny@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06/02/2024 10:08 am, Petr Beneš wrote:
> From: Petr Beneš <w1benny@gmail.com>
>
> This patch addresses a behavior discrepancy in the handling of altp2m views,
> where upon the creation and subsequent EPT violation, the page access
> permissions were incorrectly inherited from the hostp2m instead of respecting
> the altp2m default_access.
>
> Previously, when a new altp2m view was established with restrictive
> default_access permissions and activated via xc_altp2m_switch_to_view(),
> it failed to trigger an event on the first access violation.  This behavior
> diverged from the intended mechanism, where the altp2m's default_access
> should dictate the initial permissions, ensuring proper event triggering on
> access violations.
>
> The correction involves modifying the handling mechanism to respect the
> altp2m view's default_access upon its activation, eliminating the need for
> setting memory access permissions for the entire altp2m range (e.g. within
> xen-access.c).  This change not only aligns the behavior with the expected
> access control logic but also results in a significant performance improvement
> by reducing the overhead associated with setting memory access permissions
> across the altp2m range.
>
> Signed-off-by: Petr Beneš <w1benny@gmail.com>

It appears that all discussion has completed.

So unless anyone screams, I'm going to commit this with Tamas' ack as
he's the major user of this functionality.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 21:03:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 21:03:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681175.1059781 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raMPg-0004z1-7R; Wed, 14 Feb 2024 21:03:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681175.1059781; Wed, 14 Feb 2024 21:03:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raMPg-0004yu-3f; Wed, 14 Feb 2024 21:03:28 +0000
Received: by outflank-mailman (input) for mailman id 681175;
 Wed, 14 Feb 2024 21:03:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raMPe-0004yk-N5; Wed, 14 Feb 2024 21:03:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raMPe-0004lc-Gh; Wed, 14 Feb 2024 21:03:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raMPe-00077t-6a; Wed, 14 Feb 2024 21:03:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raMPe-0000gA-63; Wed, 14 Feb 2024 21:03:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/bQv16bfRlzBdR1cmVwS0m26N+NNAL7TuQmWZNBIzYw=; b=RmcWm2BXPxGlEbKMOd/tDM0urQ
	4aUr9bN8g6u5Ekx8jtJBE5j3IJax9yCd1dbgiIFf4vvXjBlqv7s6xYq4AQZB3xHB3cdnUH0mvwal5
	+jV1CUJgbhNDBlGysCUdo+5kLr668gBM8KfRaCRX/fV2EEeZGCfx7KeOUbau3HwNb2Aw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184669-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184669: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=edc6681206c1a8791981a2f911d2fb8b3d2f5768
X-Osstest-Versions-That:
    ovmf=5fd3078a2e08f607dc86a16c1b184b6e30a34a49
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 14 Feb 2024 21:03:26 +0000

flight 184669 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184669/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 edc6681206c1a8791981a2f911d2fb8b3d2f5768
baseline version:
 ovmf                 5fd3078a2e08f607dc86a16c1b184b6e30a34a49

Last test of basis   184663  2024-02-14 05:11:15 Z    0 days
Testing same since   184669  2024-02-14 17:44:06 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Gerd Hoffmann <kraxel@redhat.com>
  Laszlo Ersek <lersek@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   5fd3078a2e..edc6681206  edc6681206c1a8791981a2f911d2fb8b3d2f5768 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 14 22:21:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 14 Feb 2024 22:21:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681183.1059791 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raNco-0004wN-JQ; Wed, 14 Feb 2024 22:21:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681183.1059791; Wed, 14 Feb 2024 22:21:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raNco-0004wG-Gu; Wed, 14 Feb 2024 22:21:06 +0000
Received: by outflank-mailman (input) for mailman id 681183;
 Wed, 14 Feb 2024 22:21:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raNcn-0004w6-JD; Wed, 14 Feb 2024 22:21:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raNcn-00066f-Bl; Wed, 14 Feb 2024 22:21:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raNcm-0000Xp-Uo; Wed, 14 Feb 2024 22:21:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raNcm-0005k8-UM; Wed, 14 Feb 2024 22:21:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=RpOVPftEfVXx2o58behvJoy4vH6XsjwZzqKv2gN1MQo=; b=zewT7B3+wIRKhO27Lmy/bx2IvX
	c1zo+F7TwrcaTX2Af4nFUtj6n+V5B++DQ/gKjGRHGmRrQ6+1tDW3kYoNWM0vnYsy1sTbPdcZJ4RMn
	+5Cucm3CCDewpcv3GxAtAOeejQGj3BaS4kiifSuxFENV7FBNMpp1EtdJUCL8iyo9TvMY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184668-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184668: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=d670c1a38ba3561296f68c0079209365760b3001
X-Osstest-Versions-That:
    xen=b3bea912a737a0db39a89b0da0033c4bfda82f03
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 14 Feb 2024 22:21:04 +0000

flight 184668 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184668/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184665
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184665
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184665
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184665
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184665
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail like 184665
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail like 184665
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184665
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184665
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184665
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184665
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184665
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184665
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184665
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  d670c1a38ba3561296f68c0079209365760b3001
baseline version:
 xen                  b3bea912a737a0db39a89b0da0033c4bfda82f03

Last test of basis   184665  2024-02-14 08:21:31 Z    0 days
Testing same since   184668  2024-02-14 15:38:57 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Cyril Rébert (zithro) <slack@rabbit.lu>
  Cyril Rébert <slack@rabbit.lu>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Juergen Gross <jgross@suse.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Petr Beneš <w1benny@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   b3bea912a7..d670c1a38b  d670c1a38ba3561296f68c0079209365760b3001 -> master


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 00:05:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 00:05:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681199.1059829 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raPFa-00086s-As; Thu, 15 Feb 2024 00:05:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681199.1059829; Thu, 15 Feb 2024 00:05:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raPFa-00086l-7t; Thu, 15 Feb 2024 00:05:14 +0000
Received: by outflank-mailman (input) for mailman id 681199;
 Thu, 15 Feb 2024 00:05:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Am97=JY=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1raPFY-00086f-OU
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 00:05:12 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e1c3f1bf-cb95-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 01:05:10 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id B27AF61B8D;
 Thu, 15 Feb 2024 00:05:08 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F729C433C7;
 Thu, 15 Feb 2024 00:05:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e1c3f1bf-cb95-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707955508;
	bh=/FQHaS5JjlDs8rnJQ4Q0ybU/7VHuFGl/Pi3LlWgiNd0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=UzJcSESGa9MyYaAqNz+M/YBc44FnFVjPkJSHy2W3axre3xcMqrL4epk9cFBBlEmIv
	 I21Fyd0yeUC2ESzwjin8untDhvZaDKr7yH95zZmlYdG+V/wJfsXVSNwbgPkRFgLD1R
	 1ao5f1GAwmv644dJxAGL70qG6pz41QmdMu97iNJr/1ioLmhe8Uckxk5xu+sAaIGp6r
	 JyRZ/mo6og2qnhRD1OIHGHln5qeUgwni4a+0jhWC1rJpCfS4as2IaHmdTqbCoWAusS
	 uJK4pFwBy4mP3VJzH/uHEeZRejmppIexR3xnJBN5QDy7jTAqB7wp2nMrm5+Ki/Rswo
	 +sysDLz/NyGkg==
Date: Wed, 14 Feb 2024 16:05:05 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: Jan Beulich <jbeulich@suse.com>, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
In-Reply-To: <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402141600010.1925432@ubuntu-linux-20-04-desktop>
References: <cover.1706259490.git.federico.serafini@bugseng.com> <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com> <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com> <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com> <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com> <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com> <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com> <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com> <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com> <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com> <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com> <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
 <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com> <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 14 Feb 2024, Federico Serafini wrote:
> On 12/02/24 09:43, Jan Beulich wrote:
> > On 09.02.2024 10:50, Federico Serafini wrote:
> > > On 08/02/24 12:14, Jan Beulich wrote:
> > > > On 08.02.2024 11:45, Federico Serafini wrote:
> > > > > On 07/02/24 17:19, Jan Beulich wrote:
> > > > > > On 07.02.2024 16:58, Federico Serafini wrote:
> > > > > > > On 07/02/24 16:24, Jan Beulich wrote:
> > > > > > > > On 07.02.2024 16:08, Federico Serafini wrote:
> > > > > > > > > On 07/02/24 15:16, Jan Beulich wrote:
> > > > > > > > > > On 07.02.2024 14:51, Federico Serafini wrote:
> > > > > > > > > > > On 07/02/24 08:38, Jan Beulich wrote:
> > > > > > > > > > > > On 07.02.2024 02:08, Stefano Stabellini wrote:
> > > > > > > > > > > > > On Tue, 6 Feb 2024, Jan Beulich wrote:
> > > > > > > > > > > > > > On 26.01.2024 11:05, Federico Serafini wrote:
> > > > > > > > > > > > > > > @@ -208,7 +205,7 @@ do {
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > >            case 8:
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > >                put_unsafe_asm(x, ptr, grd, retval,
> > > > > > > > > > > > > > > "q",  "", "ir", errret);       \
> > > > > > > > > > > > > > >                break;
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > > -    default: __put_user_bad();
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > > +    default: STATIC_ASSERT_UNREACHABLE();
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > >            }
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > >            clac();
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > >        } while ( false )
> > > > > > > > > > > > > > > @@ -227,7 +224,7 @@ do {
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > >            case 2: get_unsafe_asm(x, ptr, grd,
> > > > > > > > > > > > > > > retval, "w", "=r", errret); break; \
> > > > > > > > > > > > > > >            case 4: get_unsafe_asm(x, ptr, grd,
> > > > > > > > > > > > > > > retval, "k", "=r", errret); break; \
> > > > > > > > > > > > > > >            case 8: get_unsafe_asm(x, ptr, grd,
> > > > > > > > > > > > > > > retval,  "", "=r", errret); break; \
> > > > > > > > > > > > > > > -    default: __get_user_bad();
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > > +    default: STATIC_ASSERT_UNREACHABLE();
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > >            }
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > >            clac();
> > > > > > > > > > > > > > > \
> > > > > > > > > > > > > > >        } while ( false )
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > Related to my remark on patch 1 - how is one to know
> > > > > > > > > > > > > > the macro this was
> > > > > > > > > > > > > > invoked from, when seeing the resulting diagnostic?
> > > > > > > > > > > > > 
> > > > > > > > > > > > > I am not sure what do you mean here... we do get an
> > > > > > > > > > > > > error like the
> > > > > > > > > > > > > following (I added a STATIC_ASSERT_UNREACHABLE for
> > > > > > > > > > > > > case 4):
> > > > > > > > > > > > > 
> > > > > > > > > > > > > ./arch/x86/include/asm/uaccess.h:262: Error: static
> > > > > > > > > > > > > assertion failed: unreachable
> > > > > > > > > > > > 
> > > > > > > > > > > > Right - and how do I know what _user_ of the macro
> > > > > > > > > > > > actually triggered
> > > > > > > > > > > > it? ISTR suggesting to use one or more of __FILE__ /
> > > > > > > > > > > > __LINE__ /
> > > > > > > > > > > > __FUNCTION__ here, for that specific purpose ...
> > > > > > > > > > > 
> > > > > > > > > > > To test the macro and its diagnostics,
> > > > > > > > > > > I modified the first "git grep" occurrence of
> > > > > > > > > > > ASSERT_UNREACHABLE()
> > > > > > > > > > > on the x86 code with STATIC_ASSERT_UNREACHABLE(),
> > > > > > > > > > > that is in file arch/x86/alternative.c, line 312,
> > > > > > > > > > > function _apply_alternatives().
> > > > > > > > > > > 
> > > > > > > > > > > What I got is the following build error:
> > > > > > > > > > > 
> > > > > > > > > > > ...
> > > > > > > > > > > arch/x86/alternative.c: Assembler messages:
> > > > > > > > > > > arch/x86/alternative.c:312: Error: static assertion
> > > > > > > > > > > failed: unreachable
> > > > > > > > > > >         CC      arch/x86/copy_page.o
> > > > > > > > > > > make[2]: *** [Rules.mk:247: arch/x86/alternative.o] Error
> > > > > > > > > > > 1
> > > > > > > > > > 
> > > > > > > > > > But that's not what my request was about. Here sufficient
> > > > > > > > > > context is
> > > > > > > > > > given, even if it would be nice if the function was also
> > > > > > > > > > visible right
> > > > > > > > > > away. But that's not the same as the case above, where the
> > > > > > > > > > new macro
> > > > > > > > > > is used inside another macro.
> > > > > > > > > 
> > > > > > > > > An example of that is the get_unsafe_size() macro,
> > > > > > > > > whose body uses STATIC_ASSERT_UNREACHABLE().
> > > > > > > > > A wrong use of get_unsafe_size() at line n
> > > > > > > > > leads to a build error pointing to the line n,
> > > > > > > > > isn't this the desired behavior?
> > > > > > > > 
> > > > > > > > Aiui this would point to the line in the header file, when what
> > > > > > > > you need
> > > > > > > > to spot the bad use of the macro is the line in the source file
> > > > > > > > actually
> > > > > > > > using the macro. Quoting from an earlier mail of yours:
> > > > > > > > 
> > > > > > > > ./arch/x86/include/asm/uaccess.h:262: Error: static assertion
> > > > > > > > failed: unreachable
> > > > > > > 
> > > > > > > It points to the header file uaccess.h because at line 262 there
> > > > > > > is
> > > > > > > an intentional wrong use of put_guest_size(), within the body of
> > > > > > > __copy_to_guest_pv() function.
> > > > > > 
> > > > > > Yet that's again only a helper function being inlined into the
> > > > > > ultimate
> > > > > > caller. That ultimate caller is what wants identifying in the diag.
> > > > > > Not
> > > > > > the least because of ...
> > > > > > 
> > > > > > > This example can be misleading because {get,put}_unsafe_size() are
> > > > > > > defined in the same file but the diagnostics is doing the
> > > > > > > right thing.
> > > > > > 
> > > > > > ... this. And really __copy_to_guest_pv() is the wrong place to put
> > > > > > a
> > > > > > wrong put_guest_size() in, to try out how diagnostics would look
> > > > > > like
> > > > > > in reality: That function falls back to copy_to_guest_ll() for all
> > > > > > cases it can't handle directly. You want to place a bogus
> > > > > > put_guest()
> > > > > > somewhere in a .c file to see what results.
> > > > > 
> > > > > I added a bogus call to put_guest() at line 387 of
> > > > > file xen/arch/x86/mm.c, inside function page_is_ram_type().
> > > > > Assuming I did not choose another wrong place,
> > > > > the diagnostic seems appropriate:
> > > > > 
> > > > > arch/x86/mm.c: Assembler messages:
> > > > > arch/x86/mm.c:387: Error: static assertion failed: unreachable
> > > > 
> > > > Oh, okay, this looks appropriate then as to identifying where the
> > > > source construct is. However, we then still don't know where the
> > > > assertion in question is (there could be multiple in what the
> > > > original construct expands to). So I'm still inclined to ask that
> > > > __FILE__ / __LINE__ and/or the name of the invoking construct
> > > > (macro or function) be made visible in the diagnostic.
> > > 
> > > Any use of __FILE__ and __LINE__ results in obtaining
> > > the same information already reported by the assembler error message.
> > 
> > Hmm, yes, since put_guest() is itself a macro.
> > 
> > > We could add an argument to the new macro to manually add some context
> > > at every use of the macro, but I think this would be annoying.
> > 
> > That's a last resort. An alternative would be to see about converting
> > from macros to inline functions, where this would make a difference
> > here.
> 
> I did some tries with example programs
> and the assembler error always points to file and line
> of the most enclosing function that caused the failure.
> If I am not missing something, using __FILE__ and __LINE__ does not add
> any information.
> 
> Therefore, if the new macro is used within the body of other macros,
> then the resulting assembler error will point to the source of
> the problem (e.g., the site of a bogus call to put_guest()).
> 
> In my opinion, converting put_guest() &Co. to inline functions is not
> convenient: the assembler error will point to the most enclosing
> function that would be put_unsafe_size(), instead of pointing to the
> source of the problem.

I don't think is a good idea to add further changes to this patch. I
think we should go ahead with it as-is.


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 01:08:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 01:08:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681210.1059839 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raQEc-0004zl-5b; Thu, 15 Feb 2024 01:08:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681210.1059839; Thu, 15 Feb 2024 01:08:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raQEc-0004ze-0v; Thu, 15 Feb 2024 01:08:18 +0000
Received: by outflank-mailman (input) for mailman id 681210;
 Thu, 15 Feb 2024 01:08:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Am97=JY=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1raQEa-0004zW-W7
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 01:08:16 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b1696e61-cb9e-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 02:08:14 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id E6BB961BAF;
 Thu, 15 Feb 2024 01:08:12 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 79F9FC433F1;
 Thu, 15 Feb 2024 01:08:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1696e61-cb9e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1707959292;
	bh=5Q/RVqOudlFSQhp6a5DJPyfkNbZS1Px8Ch1jDrJZJXQ=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hJyHDvv5Rkx/xrV47j2ITxJTKBQpiOiAVM90lGBTIg7Ufoz08YQ1nNy7Y8YU6lWWB
	 Ethdt5SL124D9Fy/ydkXO3TgtEHXgoR+8P+p4I9A1X+bGI4/3a/p41htTYNFTyp3A3
	 gOw1/PYiNpvMyKSrCV/Heza+rtTSeNQTpeRMHMEJJ3qz04Fri9wBT0/akej4pe+3hr
	 3IWCKIutn5IoN/91c4reuZ4N4F1cXCJE7b+WhI107265WvZlnT7iUuc4HNhzTf0NJe
	 7OAGb/cpuNxtNijqKzmOvrACVkxloLBjjDBKWIboZuYnd3j/5/2WsZFn8fxQ+56um5
	 ayOoEmEkyB4Kw==
Date: Wed, 14 Feb 2024 17:08:10 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Luca Fancellu <luca.fancellu@arm.com>
cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Wei Liu <wl@xen.org>
Subject: Re: [PATCH 0/3] xen-analysis.py: Enable Xen deviation tag at the
 end of the line
In-Reply-To: <20240131110241.3951995-1-luca.fancellu@arm.com>
Message-ID: <alpine.DEB.2.22.394.2402141707290.1925432@ubuntu-linux-20-04-desktop>
References: <20240131110241.3951995-1-luca.fancellu@arm.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 31 Jan 2024, Luca Fancellu wrote:
> This serie is enabling the xen-analysis tool to parse and substitute correctly
> a deviation tag put at the end of the line containing a deviation to be deviated.
> 
> Before this serie the only way to deviate a violation was to put the tag in the
> line above:
> 
> /* SAF-<id>-safe deviate the bla bla bla */
> <line containing the violation>
> 
> But there are places in the code base where using the tag in the line above is
> not convinient, for example:
> 
> if ( (expression) &&
>      ((expression with violation) ||
>      (expression) )
> {
>   [...]
> }
> 
> In the above example is better to have the suppression comment at the end of the
> line:
> 
> if ( (expression) &&
>      ((expression with violation) || /* SAF-<id>-safe deviate the bla bla bla */
>      (expression) )
> {
>   [...]
> }
> 
> This clearly brings up the question about the code style line length, which in
> this case needs to be amended for Xen deviation tags that goes above the limit.


Hi Luca,

I tested the series in a number of configurations and everything works
as expected. Great!

For the whole series:

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 02:20:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 02:20:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681216.1059849 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raRLv-0004lw-4X; Thu, 15 Feb 2024 02:19:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681216.1059849; Thu, 15 Feb 2024 02:19:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raRLv-0004lp-0x; Thu, 15 Feb 2024 02:19:55 +0000
Received: by outflank-mailman (input) for mailman id 681216;
 Thu, 15 Feb 2024 02:19:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jew=JY=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1raRLt-0004lj-2w
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 02:19:53 +0000
Received: from wfout7-smtp.messagingengine.com
 (wfout7-smtp.messagingengine.com [64.147.123.150])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b09a6bbe-cba8-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 03:19:48 +0100 (CET)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfout.west.internal (Postfix) with ESMTP id 6A0E81C0007E;
 Wed, 14 Feb 2024 21:19:45 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Wed, 14 Feb 2024 21:19:46 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
 14 Feb 2024 21:19:43 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b09a6bbe-cba8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1707963584;
	 x=1708049984; bh=06HGHn4qudP95v7fKy2/rJ9whPfa1kXdI0kdXfazAos=; b=
	KMU4j/Gc3lvPxbPvT56BKfJegMpwLNfhYum+q+OSc3o03xAq1nv+1uANTvfgl2eI
	959NpA7kfIjU9xi/FNrJj+62DbRn4RmPyLpIV2J+H0+YXmyANZoALrk1uSQtsJLP
	+LRhi2TobcHffhKDYWhJ/eh49Yesx1HhwMT5NdmTnB+XJH3MuR9LhjL687YVii8u
	O5LKvX3N79H/gN2cxc2oUHppkgW/cBRbdzMC1Fdij1+h7lANDpi6DCJkNPPQBx32
	g8OG957UiKxcK+d/Yg7gkhnFtaJvPlLe4uVxowUUTrOGVkZ/fGJveBViikgIMmcD
	NK9h7HVBGrnx8SeazIP/VA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm3; t=1707963584; x=1708049984; bh=06HGHn4qudP95v7fKy2/rJ9whPfa
	1kXdI0kdXfazAos=; b=j3bGFNF3/NY2umt82z1K86+QpCdS+Yn3EtXigbqjT4DD
	3c/rOwJu3BQ9YIDzCaz6RwhHQfA6paBEKNXvX34rckkhmn/LQFVHBeWAVHjnjTU+
	VgsH9J3LXSp5dQv8g6nGRAxzWPvEtgld2UTfbn0ky8Bs4KxVOtjYSx2zSo2ony41
	KB4GSPVttWxwv77GDm2+pVb+jb+VXVEd2gIfWCgkqD/hVE8jXEZgT/mD3mo55bCN
	GunA5fVs8uLUCrXjePEc4JlBtPMBwrrT11RdfGw3NriTEl0WSI5fS9FnAzBcDYrk
	ilcK6P5mWGZtc7evIbmaT/Rl+gAdw/BAkgQmaqmLmw==
X-ME-Sender: <xms:wHTNZW-BwOeR_hTCn5LNHyVyFVpAUbJjiQOvXyewS9MgonvOc_D_JQ>
    <xme:wHTNZWvyin0j6HZkRLBufKYgsj5NEM8x_ttLzmGqA7w0QcKg56HR4DaVD1udR9Sp5
    VKPaDUCAguPjQ>
X-ME-Received: <xmr:wHTNZcCx-YVMHH1-9eS3rorTPOLUn8c8GL5zL_gD8FVTNqkVXKOWRduGNuf93xeV62khK4hg55xgKhlauWLQhi-IUTu7af9lZQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudekgdegfecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpeffhffvvefukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihcuoehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthh
    hinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeeihfelhfekhefgveejjeet
    vedtgfduveelgeekteehueeufeefudegveejueefhfenucffohhmrghinhepghhithhhuh
    gsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho
    mhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:wHTNZeea5gKxh_42nSv0JFfY-qXbRtZ18TANY2oqtl0UjJ4pf__K-Q>
    <xmx:wHTNZbNCk8aNY41ub-vOoODjHV0XJEdaG7fuj2X3GSqLzEl68HMlkw>
    <xmx:wHTNZYnUihLP8ZTcyTuJlVYt51oDfZ6sjRLtzwr9K9TYj6QoPz7w1w>
    <xmx:wHTNZXAgnUqUuFowZvi-1UIQIn8-ndt6A0LC5q3d_fPrsrO2wO4CpdcHvwI>
Feedback-ID: i1568416f:Fastmail
Date: Thu, 15 Feb 2024 03:19:40 +0100
From: Marek Marczykowski <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Message-ID: <Zc10vFQtG40isDHK@mail-itl>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
 <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
 <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com>
 <ZcrlcuNU9y8WymiK@mail-itl>
 <817ceec4-ed12-4360-a6ce-f87732f62251@suse.com>
 <ZcuCTnMpFib4ArHW@mail-itl>
 <b52fa3fa-4e33-4211-af18-f00aaa0b365b@suse.com>
 <ZcuGo_vQhiTAQep0@mail-itl>
 <23bcd506-f843-42f7-a541-0d8db7c319ab@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="PMzvEAhlaWIXeocq"
Content-Disposition: inline
In-Reply-To: <23bcd506-f843-42f7-a541-0d8db7c319ab@suse.com>


--PMzvEAhlaWIXeocq
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Thu, 15 Feb 2024 03:19:40 +0100
From: Marek Marczykowski <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers

On Tue, Feb 13, 2024 at 04:44:04PM +0100, Jan Beulich wrote:
> On 13.02.2024 16:11, Marek Marczykowski wrote:
> > On Tue, Feb 13, 2024 at 04:00:32PM +0100, Jan Beulich wrote:
> >> On 13.02.2024 15:53, Marek Marczykowski wrote:
> >>> On Tue, Feb 13, 2024 at 08:45:54AM +0100, Jan Beulich wrote:
> >>>> On 13.02.2024 04:43, Marek Marczykowski wrote:
> >>>>> On Mon, Feb 12, 2024 at 10:04:38AM +0100, Jan Beulich wrote:
> >>>>>> On 08.02.2024 23:00, Julien Grall wrote:
> >>>>>>> On 05/02/2024 13:27, Jan Beulich wrote:
> >>>>>>>> In preparation of dropping the register parameters from
> >>>>>>>> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
> >>>>>>>> register state needs making available another way for the few key
> >>>>>>>> handlers which need it. Fake IRQ-like state.
> >>>>>>>>
> >>>>>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >>>>>>>> ---
> >>>>>>>> The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsist=
ent with
> >>>>>>>> other console poll functions we have, and it's unclear whether t=
hat's
> >>>>>>>> actually generally correct.
> >>>>>>>
> >>>>>>> Is it? Looking at ns16550_poll() we would pass guest_user_regs() =
if=20
> >>>>>>> run_in_exception() doesn't exist. But looking at the caller, no-o=
n seems=20
> >>>>>>> to care about the 'regs'. So is this just a latent bug?
> >>>>>>
> >>>>>> What do you mean by "doesn't exist"? ns16550_poll() assumes it exi=
sts.
> >>>>>> And I can spot any use of guest_user_regs() on the respective gene=
ric
> >>>>>> or Arm-specific bug.c paths.
> >>>>>>
> >>>>>>> BTW, do you have an idea why the poll function is not run in an=
=20
> >>>>>>> exception handler?
> >>>>>>
> >>>>>> "The poll function" being which one? If you mean the one in xhci-d=
bc.c
> >>>>>> then that's why I had Cc-ed Marek. Moving him to To: - maybe that
> >>>>>> manages to finally catch his attention.
> >>>>>
> >>>>> TBH, I don't know. That's part of the original xue patch at
> >>>>> https://github.com/connojd/xue/blob/master/patches/xen-xue-dbgp.pat=
ch
> >>>>> and it works for me as it is.
> >>>>
> >>>> "Works" meaning what? Doesn't crash on you? Or does also provide
> >>>> sensible output in _all_ cases (i.e. including when e.g. the poll
> >>>> happens to run on an idle vCPU)?
> >>>
> >>> Generally provides sensible output, for example during boot (it is us=
ing
> >>> idle vCPU then, right?).
> >>
> >> Before Dom0 is started: Yes. With the exception of the phase where PV
> >> Dom0's page tables are constructed, albeit in that time window
> >> guest_cpu_user_regs() shouldn't yield sensible data either. I can only
> >> say I'm surprised; since I have no way to properly test with an XHCI
> >> debug port, I'd have to see about faking something to convince myself
> >> (unless you were to supply example output).
> >=20
> > Would you like me to test this series with xhci console?
>=20
> The behavior shouldn't really be connected to this series. But yes, 'd'
> debug key output (just the part for the CPU the key handling was
> actually invoked from) with the xhci debug console would be of
> interest, for the case where that CPU at that time runs an idle vCPU.

I managed to press 'd' before dom0 started. Full output at
https://gist.github.com/marmarek/a495cd666f4aafed3a5cfcb8393f515b (it's
Alder Lake, and smt=3Doff, so CPU numbering is weird).
Interestingly, I do _not_ see output for CPU0, where I'd expect the
key handler to run... I see all the idle ones, plus one doing memory
scrubbing.
But also, I don't see info about the handling CPU when doing `xl
debug-key d`. At one time, with `xl debug-key d` I got this:

(XEN) *** Dumping CPU6 guest state (d0v7): ***
(XEN) ----[ Xen-4.18-unstable  x86_64  debug=3Dy  Tainted:  M     ]----
(XEN) CPU:    6
(XEN) RIP:    e033:[<ffffffff81e1546a>]
(XEN) RFLAGS: 0000000000000286   EM: 0   CONTEXT: pv guest (d0v7)
(XEN) rax: 0000000000000023   rbx: 0000000000000005   rcx: ffffffff81e1546a
(XEN) rdx: 0000000000000000   rsi: 0000000000000000   rdi: 000079147611e010
(XEN) rbp: ffff88810db53200   rsp: ffffc90041c6bde0   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000286
(XEN) r12: 0000000000305000   r13: 00007ffc61097f40   r14: ffff88810db53200
(XEN) r15: 0000000000000000   cr0: 0000000080050033   cr4: 0000000000b526e0
(XEN) cr3: 00000004ae2a7000   cr2: 00000000006d3118
(XEN) fsb: 0000791475b8a380   gsb: ffff8881897c0000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
(XEN) Guest stack trace from rsp=3Dffffc90041c6bde0:
(XEN)    0000000000000001 0000000000000000 ffffffffc02905a6 0000000000000023
(XEN)    000079147611e010 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 8064129fc747f100 ffffffffc0291568 0000000000305000
(XEN)    8064129fc747f100 0000000000000005 ffffffff813f7d4d ffffc90041c6bf58
(XEN)    ffffc90041c6bf48 0000000000000000 0000000000000000 0000000000000000
(XEN)    ffffffff81e16158 00000000006d3118 ffffc90041c6bf58 0000000000000040
(XEN)    ffffffff8132f6bb 0000000000000006 ffffc90041c6bf58 00000000006d3118
(XEN)    0000000000000255 ffff888102cf8880 ffff888102cf88f0 ffffffff8108746f
(XEN)    0000000000000000 0000000000000002 ffffc90041c6bf58 ffffc90041c6bf58
(XEN)    00000000006d3118 0000000000000000 0000000000000006 0000000000000000
(XEN)    0000000000000000 ffffffff81e1a975 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 ffffffff8200009b 000000000043d9b0
(XEN)    000000000043d990 00007ffc61097f90 00007ffc61097fc0 00007ffc61099d16
(XEN)    00000000006cab40 0000000000000246 0000000000000001 0000000000000000
(XEN)    0000000000000006 ffffffffffffffda 0000791475f1ed6f 00007ffc61097f40
(XEN)    0000000000305000 0000000000000005 0000000000000010 0000791475f1ed6f
(XEN)    0000000000000033 0000000000000246 00007ffc61097ed0 000000000000002b
(XEN)     Fault while accessing guest memory.
(XEN)=20
(XEN) *** Dumping CPU0 host state: ***
(XEN) ----[ Xen-4.18-unstable  x86_64  debug=3Dy  Tainted:  M     ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d04022c07e>] _spin_unlock_irqrestore+0x21/0x27
(XEN) RFLAGS: 0000000000000206   CONTEXT: hypervisor
(XEN) rax: ffff82d0405c1038   rbx: 0000000000000200   rcx: 0000000000000008
(XEN) rdx: ffff830856d07fff   rsi: ffff8308529d5b28   rdi: ffff8308529d5b20
(XEN) rbp: ffff830856d07dc8   rsp: ffff830856d07dc0   r8:  0000000000000001
(XEN) r9:  ffff8308529d5b20   r10: ffff82d0405c13a0   r11: 000000d091e62221
(XEN) r12: ffff82d040476898   r13: 0000000000000296   r14: ffff82d040476918
(XEN) r15: ffff82cffff04700   cr0: 0000000080050033   cr4: 0000000000b526e0
(XEN) cr3: 000000082e7ff000   cr2: ffff888109538618
(XEN) fsb: 0000000000000000   gsb: ffff888189600000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen code around <ffff82d04022c07e> (_spin_unlock_irqrestore+0x21/0x27=
):
(XEN)  fd ff ff 48 09 1c 24 9d <48> 8b 5d f8 c9 c3 0f b7 47 04 66 25 ff 0f =
66 3d
(XEN) Xen stack trace from rsp=3Dffff830856d07dc0:
(XEN)    ffff82d0405d0c80 ffff830856d07e08 ffff82d040257c3f 0000000040476898
(XEN)    ffff82d0405c1280 ffff82d040257bca ffff82d040476898 000000d0911fcbc4
(XEN)    0000000000000000 ffff830856d07e30 ffff82d04022d55c ffff82d0405c1280
(XEN)    ffff8308529d5f00 ffff82d0405d0d68 ffff830856d07e70 ffff82d04022de59
(XEN)    ffff830856d07ef8 ffff82d0405c7f00 ffffffffffffffff ffff82d0405c7f00
(XEN)    ffff830856d07fff 0000000000000000 ffff830856d07ea8 ffff82d04022b53e
(XEN)    0000000000000000 0000000000007fff ffff82d0405c7f00 ffff82d0405c11d0
(XEN)    ffff82d0405db2a0 ffff830856d07eb8 ffff82d04022b5d1 ffff830856d07ef0
(XEN)    ffff82d0402fcd15 ffff82d0402fcc88 ffff8308528cb000 ffff830856d07ef8
(XEN)    ffff830856ce2000 0000000000000000 ffff830856d07e18 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 ffffffff82c1aa40
(XEN)    0000000000000000 0000000000000246 0000000000007ff0 0000000000000000
(XEN)    000000000fd109eb 0000000000000000 ffffffff81e153aa 4000000000000000
(XEN)    deadbeefdeadf00d deadbeefdeadf00d 0000010000000000 ffffffff81e153aa
(XEN)    000000000000e033 0000000000000246 ffffffff82c03dd0 000000000000e02b
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000e01000000000 ffff830856ce1000 0000000000000000 0000000000b526e0
(XEN)    0000000000000000 0000000000000000 0006030300000000 0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff82d04022c07e>] R _spin_unlock_irqrestore+0x21/0x27
(XEN)    [<ffff82d040257c3f>] F xhci-dbc.c#dbc_uart_poll+0x75/0x17c
(XEN)    [<ffff82d04022d55c>] F timer.c#execute_timer+0x45/0x5c
(XEN)    [<ffff82d04022de59>] F timer.c#timer_softirq_action+0x71/0x275
(XEN)    [<ffff82d04022b53e>] F softirq.c#__do_softirq+0x94/0xbe
(XEN)    [<ffff82d04022b5d1>] F do_softirq+0x13/0x15
(XEN)    [<ffff82d0402fcd15>] F domain.c#idle_loop+0x8d/0xe6

(other CPUs in mwait-idle)

> > Or maybe add
> > some extra debug prints and include their output? But note, printk from
> > inside console code generally leads to deadlocks. What I did for some
> > debugging was to log into some separate buffer and dump it later.
>=20
> Right, this would be more involved.
>=20
> Jan

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--PMzvEAhlaWIXeocq
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXNdLwACgkQ24/THMrX
1yxqlgf+NOQt0926sb4bVt3vGsTb1ua6SF6ggpCt1mqSjifDwDpKQslyOtL5sWCv
y2LkaiqiSj+iLHRnyt/9XR9l/NKwe83o7nNd1p3Z/2x0EpaIwdWJkC04Oc3/YXup
GRqgHOiOoG+eJQqRQnLsu+6+Z8KckINvYt14RycdwpqUBhY+nn+Jm/sl8hJ44QdP
grW2rYoyxMh1WIzs5d90FEVAKsUIUMp1OuWykKMcGWPewxg6lq10zItryhXQiu3f
mei1wRfbjjb4oME7jc8tB8rs8DOU2/iYQ32WjcioAiayb0FoPLgse8uQYPSm5Pys
Ew0zoWYRcql1wO3ACqPtl7Qcc+BDdQ==
=1IKe
-----END PGP SIGNATURE-----

--PMzvEAhlaWIXeocq--


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 05:27:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 05:27:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681222.1059859 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raUGs-0008Gl-44; Thu, 15 Feb 2024 05:26:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681222.1059859; Thu, 15 Feb 2024 05:26:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raUGs-0008Ge-0x; Thu, 15 Feb 2024 05:26:54 +0000
Received: by outflank-mailman (input) for mailman id 681222;
 Thu, 15 Feb 2024 05:26:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RjNl=JY=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1raUGr-0008GY-0F
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 05:26:53 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d1acbc87-cbc2-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 06:26:51 +0100 (CET)
Received: from BL1PR13CA0394.namprd13.prod.outlook.com (2603:10b6:208:2c2::9)
 by SA1PR12MB7150.namprd12.prod.outlook.com (2603:10b6:806:2b4::6)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25; Thu, 15 Feb
 2024 05:26:47 +0000
Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com
 (2603:10b6:208:2c2:cafe::19) by BL1PR13CA0394.outlook.office365.com
 (2603:10b6:208:2c2::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.29 via Frontend
 Transport; Thu, 15 Feb 2024 05:26:46 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 15 Feb 2024 05:26:46 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb
 2024 23:26:46 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb
 2024 23:26:46 -0600
Received: from [172.27.131.30] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 14 Feb 2024 23:26:44 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1acbc87-cbc2-11ee-8a4d-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hY37+ZY0yvxYqFDLS8Qi3Wl42jClpieRu+pd677bL/zqlNHi7Rbp5wxvj5AuPgtLYx7Bv3QQqRG5dqCGStI4o4SrF49bbfG7yy3Oy9mc3qtLW3gSyfd6273oU/w0sZpPPvwniNfRS8KgH0YLsaQ3FB/Z1s9XD8VyperxOop9J/ve0emG1GeGA1S/rn0Is6YClBUlbfsEHB3BnG7jrJvVd4iTALGrRUtFRzk2b5VhdT4bevZo3qLo37fcVWOs5wNvIhc27zGZ7Uy/FaipbiYuZczStVEuKGpZIZS4oqyY2RFf9vOvnpJDE8UOMUt+pRNAUlLy7dxUzMYI34IbtLSmBw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=LGcn4dr4r7yoZtaNsJN7hiO4rVa2xVLAOV1RLsTcJwo=;
 b=DD6rywfTSn1aGTrV+5hiQGg5lV45xUriqI3JfzHHJnLKouWiVt5eywOZHmJXqh9OBsfFtZK27SvgvlKPxom05leLviISh/Y/AQ2/XGEG7dx0I9cNTDF2R6nSMmNb5T69WR5dXCqyTOPY3HJO59qw7gekVwhQONKqjnp1lldpoeuAZx2cnOvl0z1NUyDM8t+sSduuFHouQFuHbY++JdvUFbg0YeM4c4RXfCpwJsDO2fui3Tw1gwzLiw6Ep9ZM4fPfl0Hlly50jQQ2EVjIKwuQ20nsVaNyrIWHKb63jQnXAcyguH8HbceK72jElWcdEIIfHwFsZ74+BdOp+lM3Xx54CQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=LGcn4dr4r7yoZtaNsJN7hiO4rVa2xVLAOV1RLsTcJwo=;
 b=0V6Qy8ZA2IrgPINUUjlvKXfzGp0L1IDautNpyWyalAoVh098D6KLoQcVCxwabgf5OsmC/Yxt9DryUvt0bF6GZIs1jwXE2p6dQdymQsiCgpQQPmSMUDBTFpYkDz6DDIGk3wDeRteaFWIRKh6e9NYshZugC9dCbYcT5ef0Dmb50s0=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <8b6fa7eb-cead-4758-b9d8-d1b4bf8b4c77@amd.com>
Date: Thu, 15 Feb 2024 00:26:44 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian
	<kevin.tian@intel.com>, Paul Durrant <paul@xen.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>, <xen-devel@lists.xenproject.org>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <123f4cc2-8fc7-4e2a-b44c-78a81b3a1a3e@suse.com>
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <123f4cc2-8fc7-4e2a-b44c-78a81b3a1a3e@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|SA1PR12MB7150:EE_
X-MS-Office365-Filtering-Correlation-Id: e85fdc8d-dba4-4de0-247c-08dc2de6b429
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	k67mp/fBZdMl0WwPXZ7VPSJxOQbYyj/Kdm3KlxSYXxcUnU+HqwRLSNQ4YWOv/9v5lxDaoHY2HKZFmM58i9ql/EorosL9D5E5fpOx1bI2xu8/wL8sDMxdIuf4IdM1oAv47Y8TWd5r3mWVAiGH16EvQ3BsXsHpSFWmpE16qi5Q+rZVWxQuViHhJYAH05DzmRrNpZQmak88KTS0w1aeFT8bied1XLywA8m0OiLrK/RxZHq2+r0vsVwFFBV6yYC5h68DMLFrDBC4ciTUyraCA2xsMeJYulVJXaubg2tQ1COb+/tN1SOnOQIRbS1Ky296pQGbUqttcEv0vol34o6J3ZBZqq5nn+O6EAPZnbVfk0dsmq/i6ST6KZSIYPomxeO5RJOSJbJuHe7ZbsxfQRIkHQsVq4pW9FVN4JEjtv5FuveDtS2AVN9BZaStrVnQt6JxLUtiAzOZgmqFxTLAwGUksAA7qBW0ekr4yIro1AoDhXT8r4/a0t1vpA9UX+KQIWcO0T8+HrT8ShnlOrTLfTsZWkyq0yFu5ss6jxoFFTH/wK0l7lQemEXb353ncKTi42jEo5O9TlfABoAcaJS+v6uZX4iAw6UauaGf5FeOCL5Rj5gR3hbk+Y7VZ7DuBxXVAZuY9hvvy6wY3Mm3mPhH92Ufnn4mNA==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(136003)(346002)(376002)(396003)(230922051799003)(1800799012)(36860700004)(82310400011)(186009)(64100799003)(451199024)(46966006)(40470700004)(54906003)(7416002)(2906002)(478600001)(336012)(426003)(26005)(83380400001)(356005)(2616005)(44832011)(966005)(53546011)(81166007)(41300700001)(82740400003)(316002)(70586007)(70206006)(8936002)(8676002)(110136005)(16576012)(36756003)(31686004)(5660300002)(4326008)(86362001)(31696002);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 05:26:46.7622
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e85fdc8d-dba4-4de0-247c-08dc2de6b429
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MN1PEPF0000ECD9.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7150

On 2/14/24 06:38, Jan Beulich wrote:
> On 02.02.2024 22:33, Stewart Hildebrand wrote:
>> --- a/xen/arch/x86/physdev.c
>> +++ b/xen/arch/x86/physdev.c
>> @@ -123,7 +123,9 @@ int physdev_map_pirq(domid_t domid, int type, int *index, int *pirq_p,
>>  
>>      case MAP_PIRQ_TYPE_MSI:
>>      case MAP_PIRQ_TYPE_MULTI_MSI:
>> +        pcidevs_lock();
>>          ret = allocate_and_map_msi_pirq(d, *index, pirq_p, type, msi);
>> +        pcidevs_unlock();
>>          break;
> 
> I'm afraid I need to come back to this: This is the only place where this
> patch retains (moves) use of the global lock. By moving, its scope is
> actually extended. It was previously said that conversion doesn't happen
> to limit the scope of what is changing. But with allocate_and_map_msi_pirq()
> being happy about either lock being held, I'm having a hard time seeing why
> here the global lock would continue to need using. To me doing so suggests
> uncertainty whether the checking in the function is actually correct.

I understand the concern. I've gone back and forth on this one in
particular myself [1]. I've tested it both ways (i.e. as shown here with
pcidevs_lock() and replacing it with read_lock(&d->pci_lock)). In the
analysis I've done, I also cannot find a need to continue using
pcidevs_lock() here. read_lock(&d->pci_lock) is sufficient. Let's be
clear: both ways are correct. The only reason I left it at
pcidevs_lock() for v13 was to make sure the code was doing what the
commit description and notes in the cover letter say.

allocate_and_map_msi_pirq() acquires write_lock(&d->event_lock), and
accesses to non-domain-specific data fall in the category of reading
__read_mostly globals or are appropriately protected by desc->lock
and/or vector_lock and/or cmpxchg.

I'll change it to read_lock(&d->pci_lock) in this one particular
instance (and update the description appropriately).

[1] https://lore.kernel.org/xen-devel/3c1023c4-25fb-41f1-83eb-03cbc1c3720e@amd.com/


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:40:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:40:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681229.1059868 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVPz-0000PT-J9; Thu, 15 Feb 2024 06:40:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681229.1059868; Thu, 15 Feb 2024 06:40:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVPz-0000PM-Gh; Thu, 15 Feb 2024 06:40:23 +0000
Received: by outflank-mailman (input) for mailman id 681229;
 Thu, 15 Feb 2024 06:40:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVPy-0000PG-2l
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:40:22 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 15a7a0e1-cbcd-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:40:19 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 2CBAA1F850;
 Thu, 15 Feb 2024 06:40:18 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D98C313A72;
 Thu, 15 Feb 2024 06:40:17 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 4lbyMtGxzWXUcQAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:40:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15a7a0e1-cbcd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707979218; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=jMgwWIROcUiEtJ/aRznmaKwaecq/6qWAjHktvBCBXaA=;
	b=sd2poRr/QuYUSEmDU6CToMvt16E+dQVgAGsOx05//vwj+mVe+5NmraDuamP3xSipfVHrnT
	0HtMqVyu65WTSWz00Ws9nlohg6zpJNGY+vki+z8dLUgpH6UiZd1D06iwg2m6TqQM+4Lc3I
	3SQ3PuNvk7SyWUCuFFNmCWpyIzkSxsk=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707979218; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=jMgwWIROcUiEtJ/aRznmaKwaecq/6qWAjHktvBCBXaA=;
	b=sd2poRr/QuYUSEmDU6CToMvt16E+dQVgAGsOx05//vwj+mVe+5NmraDuamP3xSipfVHrnT
	0HtMqVyu65WTSWz00Ws9nlohg6zpJNGY+vki+z8dLUgpH6UiZd1D06iwg2m6TqQM+4Lc3I
	3SQ3PuNvk7SyWUCuFFNmCWpyIzkSxsk=
Message-ID: <f8a24054-4d19-489a-bcc5-e465a391b2f7@suse.com>
Date: Thu, 15 Feb 2024 07:40:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 15/22] tools/libs/light: add backend type for 9pfs PV
 devices
Content-Language: en-US
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, George Dunlap <george.dunlap@citrix.com>,
 Nick Rosbrook <rosbrookn@gmail.com>, Wei Liu <wl@xen.org>,
 Jason Andryuk <jandryuk@gmail.com>, George Dunlap <george.dunlap@cloud.com>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-16-jgross@suse.com>
 <5e6893dd-5280-4a14-8d00-4ed76e8c8e98@perard>
 <c6af9dd1-01d4-48a2-86b9-ccb2967136f7@suse.com>
 <a3a0d9eb-97ae-4e5d-ad08-cfe5319836d7@perard>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <a3a0d9eb-97ae-4e5d-ad08-cfe5319836d7@perard>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: 0.31
X-Spamd-Result: default: False [0.31 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_SPAM_SHORT(1.16)[0.387];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 BAYES_HAM(-0.35)[76.45%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.59)[subject];
	 FREEMAIL_CC(0.00)[lists.xenproject.org,citrix.com,gmail.com,xen.org,cloud.com];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Flag: NO

On 14.02.24 18:27, Anthony PERARD wrote:
> On Wed, Feb 14, 2024 at 11:18:18AM +0100, Jürgen Groß wrote:
>> On 13.02.24 19:03, Anthony PERARD wrote:
>>> On Thu, Feb 08, 2024 at 05:55:39PM +0100, Juergen Gross wrote:
>>>> +struct libxl__aop9_state {
>>>> +    libxl__spawn_state spawn;
>>>> +    libxl__ao_device *aodev;
>>>> +    libxl_device_p9 p9;
>>>> +    uint32_t domid;
>>>> +    void (*callback)(libxl__egc *, libxl__aop9_state *, int);
>>>
>>> This "callback" is never used, right?
>>
>> Why do you think so?
>>
>> In xen9pfsd_spawn() it is used:
>>
>>    aop9->callback = xen9pfsd_spawn_outcome;
> 
> By never used, I mean that nothing is reading the value, their is no
> "aop9->callback(egc, aop9, rc)" call.
> It might have been useful if a caller of xen9pfsd_spawn() was actually
> setting this field, but that's not an option here. And callbacks of
> xen9pfsd_spawn() knows to call xen9pfsd_spawn_outcome() when done.

Oh, right. This must be a relict of a previous attempt to make it work.

I'll remove the callback.


Juergen



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:43:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:43:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681232.1059878 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVSu-0000y9-0m; Thu, 15 Feb 2024 06:43:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681232.1059878; Thu, 15 Feb 2024 06:43:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVSt-0000y2-US; Thu, 15 Feb 2024 06:43:23 +0000
Received: by outflank-mailman (input) for mailman id 681232;
 Thu, 15 Feb 2024 06:43:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVSs-0000xw-JM
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:43:22 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 81ff370a-cbcd-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:43:20 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 388E821FDA;
 Thu, 15 Feb 2024 06:43:20 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0255313A72;
 Thu, 15 Feb 2024 06:43:19 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id sPUEOoeyzWVscgAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:43:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81ff370a-cbcd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707979400; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=KJBtwRgqYykFvijX68nK+SNPxgtK4+lGewN5IH7jEo8=;
	b=fgW+Z+CI46bCAMamx3AX/z9XlErTdkQY4CTh4DvzAgXZp/Vm+zdt9PmDDWLcJNl+tPjKTg
	vCUlVEJxffxXuOT92u1+CZfeLEfELbZQpMPhKtjyEHhFueUh3PR7yJoDaF65qHqyindzSF
	w5axZbXmLpfZGGxJeTgp5ngZC+t9CLs=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707979400; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=KJBtwRgqYykFvijX68nK+SNPxgtK4+lGewN5IH7jEo8=;
	b=fgW+Z+CI46bCAMamx3AX/z9XlErTdkQY4CTh4DvzAgXZp/Vm+zdt9PmDDWLcJNl+tPjKTg
	vCUlVEJxffxXuOT92u1+CZfeLEfELbZQpMPhKtjyEHhFueUh3PR7yJoDaF65qHqyindzSF
	w5axZbXmLpfZGGxJeTgp5ngZC+t9CLs=
Message-ID: <5ca334ca-3bb0-427e-bbe2-845a931dcec7@suse.com>
Date: Thu, 15 Feb 2024 07:43:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 07/22] tools/9pfsd: add 9pfs attach request support
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jason Andryuk <jandryuk@gmail.com>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-8-jgross@suse.com>
 <a628abf7-a04d-4c32-ad49-a2fe073f3286@citrix.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <a628abf7-a04d-4c32-ad49-a2fe073f3286@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [0.61 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 BAYES_HAM(-0.01)[50.84%];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[gitlab.com:url];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.71)[subject];
	 FREEMAIL_CC(0.00)[xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: 0.61

On 14.02.24 18:40, Andrew Cooper wrote:
> On 08/02/2024 4:55 pm, Juergen Gross wrote:
>> +static struct p9_fid *alloc_fid_mem(device *device, unsigned int fid,
>> +                                    const char *path)
>> +{
>> +    struct p9_fid *fidp;
>> +    size_t pathlen;
>> +
>> +    pathlen = strlen(path);
>> +    fidp = calloc(sizeof(*fidp) + pathlen + 1, 1);
>> +    if ( !fidp )
>> +        return NULL;
>> +
>> +    fidp->fid = fid;
>> +    strncpy(fidp->path, path, pathlen);
>> +
>> +    return fidp;
>> +}
> 
> GitlabCI has something to say about this.
> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1176787593
> 
> I think they're all variations of:
> 
> io.c: In function 'alloc_fid_mem.isra.8':
> io.c:566:5: error: 'strncpy' output truncated before terminating nul
> copying as many bytes from a string as its length
> [-Werror=stringop-truncation]
>       strncpy(fidp->path, path, pathlen);
>       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> io.c:560:15: note: length computed here
>       pathlen = strlen(path);
>                 ^~~~~~~~~~~~

In the end the result is fine, as the buffer is large enough and it is
zeroed on allocation.

I'll change it nevertheless as it is a bad code pattern.


Juergen



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:45:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:45:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681238.1059889 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVV6-0001XE-By; Thu, 15 Feb 2024 06:45:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681238.1059889; Thu, 15 Feb 2024 06:45:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVV6-0001X7-95; Thu, 15 Feb 2024 06:45:40 +0000
Received: by outflank-mailman (input) for mailman id 681238;
 Thu, 15 Feb 2024 06:45:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVV4-0001X1-4o
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:45:38 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d2b9dc2c-cbcd-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:45:36 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 820ED1FDB6;
 Thu, 15 Feb 2024 06:45:35 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 538C713A72;
 Thu, 15 Feb 2024 06:45:35 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id N3aNEg+zzWUCcwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:45:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d2b9dc2c-cbcd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707979535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=R9+emDP1lJIkMFxao7dr1SXZIkl3gMyxU58knvfJfVc=;
	b=s154+xbSLPSopT26tAEPttYRLQ2hkaZtap5rNCaOCgH5dexnNE/hXn+xGX0JQkto1aZjbT
	Ywv7wiJSZ+SN9IpVfJ4dX3PspYB8rPPrYzxYYrdVtS7B5/YY1WX6yyOYM3rjX5Te+p5Whr
	j8CpA22vJxXkJLi6S11514jzMXoRnZ4=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707979535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=R9+emDP1lJIkMFxao7dr1SXZIkl3gMyxU58knvfJfVc=;
	b=s154+xbSLPSopT26tAEPttYRLQ2hkaZtap5rNCaOCgH5dexnNE/hXn+xGX0JQkto1aZjbT
	Ywv7wiJSZ+SN9IpVfJ4dX3PspYB8rPPrYzxYYrdVtS7B5/YY1WX6yyOYM3rjX5Te+p5Whr
	j8CpA22vJxXkJLi6S11514jzMXoRnZ4=
Message-ID: <a5a29a13-fe9c-46be-b955-1ad3f450b2cd@suse.com>
Date: Thu, 15 Feb 2024 07:45:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 07/22] tools/9pfsd: add 9pfs attach request support
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jason Andryuk <jandryuk@gmail.com>
References: <20240208165546.5715-1-jgross@suse.com>
 <20240208165546.5715-8-jgross@suse.com>
 <4af65d74-b465-4f2b-9c4e-b1dab9a4ea7a@citrix.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <4af65d74-b465-4f2b-9c4e-b1dab9a4ea7a@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=s154+xbS
X-Spamd-Result: default: False [-0.99 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 BAYES_HAM(-1.41)[90.96%];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.71)[subject];
	 FREEMAIL_CC(0.00)[xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: -0.99
X-Rspamd-Queue-Id: 820ED1FDB6
X-Spam-Level: 
X-Spam-Flag: NO
X-Spamd-Bar: /

On 14.02.24 18:48, Andrew Cooper wrote:
> On 08/02/2024 4:55 pm, Juergen Gross wrote:
>> +static struct p9_fid *alloc_fid(device *device, unsigned int fid,
>> +                                const char *path)
>> +{
>> +    struct p9_fid *fidp = NULL;
>> +
>> +    pthread_mutex_lock(&device->fid_mutex);
>> +
>> +    if ( find_fid(device, fid) )
>> +    {
>> +        errno = EBADFD;
> 
> Also, FreeBSD says no.
> 
> https://cirrus-ci.com/task/6634697753624576
> 
> io.c:580:17: error: use of undeclared identifier 'EBADFD'
>          errno = EBADFD;
>                  ^
> 1 error generated.
> 
> Need to use EBADF.

Yes.


Juergen



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:55:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:55:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681245.1059909 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVez-0003ce-GM; Thu, 15 Feb 2024 06:55:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681245.1059909; Thu, 15 Feb 2024 06:55:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVez-0003cX-DT; Thu, 15 Feb 2024 06:55:53 +0000
Received: by outflank-mailman (input) for mailman id 681245;
 Thu, 15 Feb 2024 06:55:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVey-0003c6-Km
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:55:52 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 40c68447-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:55:50 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id A9D5921DA6;
 Thu, 15 Feb 2024 06:55:49 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 697BC1346A;
 Thu, 15 Feb 2024 06:55:49 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id t9P1F3W1zWVFSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:55:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 40c68447-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980149; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=rQPRnnpvHJ9iAPaCFRu4161m+4eTjuy3Ob9j9LfvB8M=;
	b=mX49LHZRDV++ILXrTayh6jGziie0EhtNzoDzJHCzVWLUgfXbnr8Q6NKSQnodKKXx/ailRY
	q2ZADXS2iHrer5GzVIB+o2y8yndP5bSHmunEuQjXXJ86GY0XOwG2z6wp/tzkVkNMVPY812
	NTKClGhBqoKlcLM9SspHl/ca9TbheMM=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980149; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=rQPRnnpvHJ9iAPaCFRu4161m+4eTjuy3Ob9j9LfvB8M=;
	b=mX49LHZRDV++ILXrTayh6jGziie0EhtNzoDzJHCzVWLUgfXbnr8Q6NKSQnodKKXx/ailRY
	q2ZADXS2iHrer5GzVIB+o2y8yndP5bSHmunEuQjXXJ86GY0XOwG2z6wp/tzkVkNMVPY812
	NTKClGhBqoKlcLM9SspHl/ca9TbheMM=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v6 01/21] tools: add a new xen 9pfs daemon
Date: Thu, 15 Feb 2024 07:55:21 +0100
Message-Id: <20240215065541.21067-2-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=mX49LHZR
X-Spamd-Result: default: False [1.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[6];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 1.69
X-Rspamd-Queue-Id: A9D5921DA6
X-Spam-Level: *
X-Spam-Flag: NO
X-Spamd-Bar: +

Add "xen-9pfsd", a new 9pfs daemon meant to support infrastructure
domains (e.g. xenstore-stubdom) to access files in dom0.

For now only add the code needed for starting the daemon and
registering it with Xenstore via a new "libxl/xen-9pfs/state" node by
writing the "running" state to it.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- rename from xenlogd to xen-9pfsd (Andrew Cooper)
- use a backend domain local Xenstore node (Jason Andryuk)
- use "volatile" for stop_me (Andrew Cooper)
V3:
- fix SPDX Identifier (Andrew Cooper)
V4:
- add strerror() test to error logging (Andrew Cooper)
- don't handle "daemon already running" as error (Anthony Perard)
V5:
- rename source directory to tools/9pfsd (Andrew Cooper)
---
 tools/9pfsd/.gitignore  |   1 +
 tools/9pfsd/Makefile    |  38 +++++++++++
 tools/9pfsd/xen-9pfsd.c | 147 ++++++++++++++++++++++++++++++++++++++++
 tools/Makefile          |   1 +
 4 files changed, 187 insertions(+)
 create mode 100644 tools/9pfsd/.gitignore
 create mode 100644 tools/9pfsd/Makefile
 create mode 100644 tools/9pfsd/xen-9pfsd.c

diff --git a/tools/9pfsd/.gitignore b/tools/9pfsd/.gitignore
new file mode 100644
index 0000000000..d0c2d223ef
--- /dev/null
+++ b/tools/9pfsd/.gitignore
@@ -0,0 +1 @@
+/xen-9pfsd
diff --git a/tools/9pfsd/Makefile b/tools/9pfsd/Makefile
new file mode 100644
index 0000000000..089cf5ae24
--- /dev/null
+++ b/tools/9pfsd/Makefile
@@ -0,0 +1,38 @@
+#
+# tools/9pfsd/Makefile
+#
+
+XEN_ROOT = $(CURDIR)/../..
+include $(XEN_ROOT)/tools/Rules.mk
+
+CFLAGS += $(PTHREAD_CFLAGS)
+LDFLAGS += $(PTHREAD_LDFLAGS)
+
+TARGETS := xen-9pfsd
+
+XEN-9PFSD_OBJS = xen-9pfsd.o
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenstore)
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenevtchn)
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
+xen-9pfsd: LDLIBS += $(call xenlibs-ldlibs,store evtchn gnttab)
+
+.PHONY: all
+all: $(TARGETS)
+
+xen-9pfsd: $(XEN-9PFSD_OBJS)
+	$(CC) $(LDFLAGS) -o $@ $(XEN-9PFSD_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
+
+.PHONY: install
+install: all
+	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
+	for i in $(TARGETS); do $(INSTALL_PROG) $$i $(DESTDIR)$(LIBEXEC_BIN); done
+
+.PHONY: uninstall
+uninstall:
+	for i in $(TARGETS); do rm -f $(DESTDIR)$(LIBEXEC_BIN)/$$i; done
+
+.PHONY: clean
+clean:
+	$(RM) *.o $(TARGETS) $(DEPS_RM)
+
+distclean: clean
diff --git a/tools/9pfsd/xen-9pfsd.c b/tools/9pfsd/xen-9pfsd.c
new file mode 100644
index 0000000000..6939d01574
--- /dev/null
+++ b/tools/9pfsd/xen-9pfsd.c
@@ -0,0 +1,147 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+/*
+ * xen-9pfsd - Xen 9pfs daemon
+ *
+ * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
+ *
+ * Daemon to enable guests to access a directory of the dom0 file system.
+ * Access is made via the 9pfs protocol (xen-9pfsd acts as a PV 9pfs backend).
+ *
+ * Usage: xen-9pfsd
+ *
+ * xen-9pfsd does NOT support writing any links (neither soft links nor hard
+ * links), and it is accepting only canonicalized file paths in order to
+ * avoid the possibility to "escape" from the guest specific directory.
+ *
+ * The backend device string is "xen_9pfs", the tag used for mounting the
+ * 9pfs device is "Xen".
+ *
+ * As an additional security measure the maximum file space used by the guest
+ * can be limited by the backend Xenstore node "max-size" specifying the size
+ * in MBytes. This size includes the size of the root directory of the guest.
+ */
+
+#include <err.h>
+#include <errno.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <syslog.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <xenevtchn.h>
+#include <xengnttab.h>
+#include <xenstore.h>
+
+static volatile bool stop_me;
+static bool daemon_running;
+static struct xs_handle *xs;
+static xengnttab_handle *xg;
+static xenevtchn_handle *xe;
+
+static void handle_stop(int sig)
+{
+    stop_me = true;
+}
+
+static void close_all(void)
+{
+    if ( daemon_running )
+        xs_rm(xs, XBT_NULL, "libxl/xen-9pfs");
+    if ( xe )
+        xenevtchn_close(xe);
+    if ( xg )
+        xengnttab_close(xg);
+    if ( xs )
+        xs_close(xs);
+    closelog();
+}
+
+static void do_err(const char *msg)
+{
+    syslog(LOG_ALERT, "%s, errno = %d, %s", msg, errno, strerror(errno));
+    close_all();
+    exit(1);
+}
+
+static void xen_connect(void)
+{
+    xs_transaction_t t;
+    char *val;
+    unsigned int len;
+
+    xs = xs_open(0);
+    if ( xs == NULL )
+        do_err("xs_open() failed");
+
+    xg = xengnttab_open(NULL, 0);
+    if ( xg == NULL )
+        do_err("xengnttab_open() failed");
+
+    xe = xenevtchn_open(NULL, 0);
+    if ( xe == NULL )
+        do_err("xenevtchn_open() failed");
+
+    while ( true )
+    {
+        t = xs_transaction_start(xs);
+        if ( t == XBT_NULL )
+            do_err("xs_transaction_start() failed");
+
+        val = xs_read(xs, t, "libxl/xen-9pfs/state", &len);
+        if ( val )
+        {
+            free(val);
+            xs_transaction_end(xs, t, true);
+            syslog(LOG_INFO, "daemon already running");
+            close_all();
+            exit(0);
+        }
+
+        if ( !xs_write(xs, t, "libxl/xen-9pfs/state", "running",
+                       strlen("running")) )
+        {
+            xs_transaction_end(xs, t, true);
+            do_err("xs_write() failed writing state");
+        }
+
+        if ( xs_transaction_end(xs, t, false) )
+            break;
+        if ( errno != EAGAIN )
+            do_err("xs_transaction_end() failed");
+    }
+
+    daemon_running = true;
+}
+
+int main(int argc, char *argv[])
+{
+    struct sigaction act = { .sa_handler = handle_stop, };
+    int syslog_mask = LOG_MASK(LOG_WARNING) | LOG_MASK(LOG_ERR) |
+                      LOG_MASK(LOG_CRIT) | LOG_MASK(LOG_ALERT) |
+                      LOG_MASK(LOG_EMERG);
+
+    umask(027);
+    if ( getenv("XEN_9PFSD_VERBOSE") )
+        syslog_mask |= LOG_MASK(LOG_NOTICE) | LOG_MASK(LOG_INFO);
+    openlog("xen-9pfsd", LOG_CONS, LOG_DAEMON);
+    setlogmask(syslog_mask);
+
+    sigemptyset(&act.sa_mask);
+    sigaction(SIGHUP, &act, NULL);
+
+    xen_connect();
+
+    while ( !stop_me )
+    {
+        sleep(60);
+    }
+
+    close_all();
+
+    return 0;
+}
diff --git a/tools/Makefile b/tools/Makefile
index 3a510663a0..7d17211782 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -32,6 +32,7 @@ SUBDIRS-y += xenpmd
 SUBDIRS-$(CONFIG_GOLANG) += golang
 SUBDIRS-y += xl
 SUBDIRS-y += helpers
+SUBDIRS-y += 9pfsd
 SUBDIRS-$(CONFIG_X86) += xenpaging
 SUBDIRS-$(CONFIG_X86) += debugger
 SUBDIRS-$(CONFIG_TESTS) += tests
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:55:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:55:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681244.1059898 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVeu-0003Ng-7y; Thu, 15 Feb 2024 06:55:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681244.1059898; Thu, 15 Feb 2024 06:55:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVeu-0003NZ-4z; Thu, 15 Feb 2024 06:55:48 +0000
Received: by outflank-mailman (input) for mailman id 681244;
 Thu, 15 Feb 2024 06:55:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVet-0003NA-8e
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:55:47 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3e5f860f-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:55:46 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id EE26D21DA6;
 Thu, 15 Feb 2024 06:55:43 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id A95C41346A;
 Thu, 15 Feb 2024 06:55:43 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id hvnfJ2+1zWVASQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:55:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3e5f860f-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980144; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=ECSeBWWjKQHdyNMNTJRjvPW6+HyR8VrvmXmI6Qt0kLI=;
	b=ecFi8c1VNfIHhOyrUKXzC5qmYy7NobQrWIQvr8VvGXOioVIZo96IlqFP+DRGYZJyU7T/wY
	suBDeePN3+4StP7UnTreaQcOKX18Ene05c5/MCAVR+BstAPbV6FCjXHMQFj9pxwOt+ArYA
	6yhRh+q9Yzy04+ZPm2lXBA/w82yPt50=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980143; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=ECSeBWWjKQHdyNMNTJRjvPW6+HyR8VrvmXmI6Qt0kLI=;
	b=nv5DNbUemAO/zdgoljnsrZvKBtBJVcKw3AI5qMOqP5B6KgjJYwzRqbjLU9hEiPgDbsC9WN
	7q9IzZkJ2EEqAhSE9VVxyjeAS9IvBpbmYz+YmkdgkYVxQBJEQlJ1OGef/fiEdcKhn7sgl2
	dxLZv8wRfxlcqye8HdYONRgU6oDIdyE=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH v6 00/21] tools: enable xenstore-stubdom to use 9pfs
Date: Thu, 15 Feb 2024 07:55:20 +0100
Message-Id: <20240215065541.21067-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=nv5DNbUe
X-Spamd-Result: default: False [1.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,ens-lyon.org];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 1.69
X-Rspamd-Queue-Id: EE26D21DA6
X-Spam-Level: *
X-Spam-Flag: NO
X-Spamd-Bar: +

This series is adding 9pfs support to Xenstore-stubdom, enabling it
to do logging to a dom0 directory.

This is a prerequisite for the final goal to add live update support
to Xenstore-stubdom, as it enables the stubdom to store its state in
a dom0 file.

The 9pfs backend is a new daemon written from scratch. Using a
dedicated 9pfs daemon has several advantages:

- it is using much less resources than a full blown qemu process
- it can serve multiple guests (the idea is to use it for other
  infrastructure domains, like qemu-stubdom or driver domains, too)
- it is designed to support several security enhancements, like
  limiting the number of files for a guest, or limiting the allocated
  file system space
- it doesn't support file links (neither hard nor soft links) or
  referencing parent directories via "..", minimizing the risk that
  a guest can "escape" from its home directory

Note that for now the daemon only contains the minimal needed
functionality to do logging from Xenstore-stubdom. I didn't want to
add all the 9pfs commands and security add-ons in the beginning, in
order to avoid needless efforts in case the idea of the daemon is
being rejected.

Please note that the pending patch for updating the Mini-OS commit
in Config.mk needs to be applied for patch "stubdom: extend xenstore
stubdom configs" and the following ones.

Changes in V6:
- patch 1 of V5 has been applied
- rebase
- addressed comments

Changes in V5:
- 10 patches have been applied already
- rename source directory to tools/9pfsd
- addressed comments

Changes in V4:
- patch 2 of V3 was applied
- added support of reading directories
- addressed review comments

Changes in V3:
- new patches 1, 23-25
- addressed review comments

Changes in V2:
- support of multiple rings per device
- xenlogd->xen-9pfsd rename
- addressed review comments
- fixed some bugs

Juergen Gross (21):
  tools: add a new xen 9pfs daemon
  tools/xen-9pfsd: connect to frontend
  tools/xen-9pfsd: add transport layer
  tools/xen-9pfsd: add 9pfs response generation support
  tools/xen-9pfsd: add 9pfs version request support
  tools/xen-9pfsd: add 9pfs attach request support
  tools/xen-9pfsd: add 9pfs walk request support
  tools/xen-9pfsd: add 9pfs open request support
  tools/xen-9pfsd: add 9pfs clunk request support
  tools/xen-9pfsd: add 9pfs create request support
  tools/xen-9pfsd: add 9pfs stat request support
  tools/xen-9pfsd: add 9pfs write request support
  tools/xen-9pfsd: add 9pfs read request support
  tools/libs/light: add backend type for 9pfs PV devices
  tools/xl: support new 9pfs backend xen_9pfsd
  stubdom: extend xenstore stubdom configs
  tools: add 9pfs device to xenstore-stubdom
  tools/xenstored: mount 9pfs device in stubdom
  tools/xenstored: add helpers for filename handling
  tools/xenstored: support complete log capabilities in stubdom
  tools/xenstored: have a single do_control_memreport()

 docs/man/xl.cfg.5.pod.in                      |   36 +-
 stubdom/xenstore-minios.cfg                   |    2 +-
 stubdom/xenstorepvh-minios.cfg                |    2 +-
 tools/9pfsd/.gitignore                        |    1 +
 tools/9pfsd/Makefile                          |   38 +
 tools/9pfsd/io.c                              | 1513 +++++++++++++++++
 tools/9pfsd/xen-9pfsd.c                       |  800 +++++++++
 tools/9pfsd/xen-9pfsd.h                       |   99 ++
 tools/Makefile                                |    1 +
 tools/golang/xenlight/helpers.gen.go          |   10 +
 tools/golang/xenlight/types.gen.go            |   12 +
 tools/helpers/init-xenstore-domain.c          |    7 +
 .../Linux/init.d/sysconfig.xencommons.in      |    1 -
 tools/hotplug/Linux/launch-xenstore.in        |    1 +
 tools/include/libxl.h                         |   22 +
 tools/libs/light/libxl_9pfs.c                 |  191 ++-
 tools/libs/light/libxl_create.c               |    4 +-
 tools/libs/light/libxl_dm.c                   |    2 +-
 tools/libs/light/libxl_types.idl              |   11 +
 tools/libs/light/libxl_types_internal.idl     |    1 +
 tools/xenstored/control.c                     |   29 +-
 tools/xenstored/core.c                        |   15 +-
 tools/xenstored/core.h                        |   11 +-
 tools/xenstored/domain.c                      |    2 +
 tools/xenstored/lu_daemon.c                   |    4 +-
 tools/xenstored/minios.c                      |   62 +
 tools/xenstored/posix.c                       |    8 +-
 tools/xl/xl_parse.c                           |   23 +-
 28 files changed, 2866 insertions(+), 42 deletions(-)
 create mode 100644 tools/9pfsd/.gitignore
 create mode 100644 tools/9pfsd/Makefile
 create mode 100644 tools/9pfsd/io.c
 create mode 100644 tools/9pfsd/xen-9pfsd.c
 create mode 100644 tools/9pfsd/xen-9pfsd.h

-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:55:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:55:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681246.1059919 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVf4-0003vQ-TH; Thu, 15 Feb 2024 06:55:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681246.1059919; Thu, 15 Feb 2024 06:55:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVf4-0003vJ-Q4; Thu, 15 Feb 2024 06:55:58 +0000
Received: by outflank-mailman (input) for mailman id 681246;
 Thu, 15 Feb 2024 06:55:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVf2-0003NA-VK
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:55:57 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 44145466-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:55:55 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 50EC321DA6;
 Thu, 15 Feb 2024 06:55:55 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 1AB961346A;
 Thu, 15 Feb 2024 06:55:55 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id Xsj3BHu1zWVHSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:55:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 44145466-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980155; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2rcpEbJbgUxiNrBORk+pKJOPpR9mCjteYZqR1AIlbbY=;
	b=RXa/+oJmlrhUZ4j9T89gUioYq4P7YJzotxNKHG+nSrFaYhD8yZNP0x414pmjcN2jJAcbGx
	QAD+4MxtUws68+7H1XZBTj7yh+CO8W8T1ipuA/1G1fJYcsRBfBJy8PB97wGKWJJ1jwg2dN
	Hfr1B0BWyfQ0qohfhyOSCsKcgI9L+4A=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980155; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2rcpEbJbgUxiNrBORk+pKJOPpR9mCjteYZqR1AIlbbY=;
	b=RXa/+oJmlrhUZ4j9T89gUioYq4P7YJzotxNKHG+nSrFaYhD8yZNP0x414pmjcN2jJAcbGx
	QAD+4MxtUws68+7H1XZBTj7yh+CO8W8T1ipuA/1G1fJYcsRBfBJy8PB97wGKWJJ1jwg2dN
	Hfr1B0BWyfQ0qohfhyOSCsKcgI9L+4A=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 02/21] tools/xen-9pfsd: connect to frontend
Date: Thu, 15 Feb 2024 07:55:22 +0100
Message-Id: <20240215065541.21067-3-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b="RXa/+oJm"
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-3.51 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Score: -3.51
X-Rspamd-Queue-Id: 50EC321DA6
X-Spam-Flag: NO

Add the code for connecting to frontends to xenlogd.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- support multiple rings per device (Jason Andryuk)
- don't set .revents initially (Jason Andryuk)
- call poll() with infinite timeout (Jason Andryuk)
- take mutex before calling pthread_cond_signal()
V3:
- fix SPDX indentifier (Andrew Cooper)
- better validation of host path (Jason Andryuk)
- don't hard-code dom0 in backend nodes (Jason Andryuk)
- use bool instead of int for some functions' return types
- open root directory (Jason Andryuk)
---
 tools/9pfsd/Makefile    |   2 +-
 tools/9pfsd/io.c        |  45 +++
 tools/9pfsd/xen-9pfsd.c | 653 +++++++++++++++++++++++++++++++++++++++-
 tools/9pfsd/xen-9pfsd.h |  61 ++++
 4 files changed, 757 insertions(+), 4 deletions(-)
 create mode 100644 tools/9pfsd/io.c
 create mode 100644 tools/9pfsd/xen-9pfsd.h

diff --git a/tools/9pfsd/Makefile b/tools/9pfsd/Makefile
index 089cf5ae24..50573121ed 100644
--- a/tools/9pfsd/Makefile
+++ b/tools/9pfsd/Makefile
@@ -10,7 +10,7 @@ LDFLAGS += $(PTHREAD_LDFLAGS)
 
 TARGETS := xen-9pfsd
 
-XEN-9PFSD_OBJS = xen-9pfsd.o
+XEN-9PFSD_OBJS = xen-9pfsd.o io.o
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenstore)
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenevtchn)
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
new file mode 100644
index 0000000000..eb7c136e09
--- /dev/null
+++ b/tools/9pfsd/io.c
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+/*
+ * xen-9pfsd - Xen 9pfs daemon
+ *
+ * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
+ *
+ * I/O thread handling.
+ */
+
+#include <stdbool.h>
+#include <string.h>
+#include <syslog.h>
+
+#include "xen-9pfsd.h"
+
+static bool io_work_pending(struct ring *ring)
+{
+    if ( ring->stop_thread )
+        return true;
+    return false;
+}
+
+void *io_thread(void *arg)
+{
+    struct ring *ring = arg;
+
+    while ( !ring->stop_thread )
+    {
+        pthread_mutex_lock(&ring->mutex);
+        if ( !io_work_pending(ring) )
+        {
+            if ( xenevtchn_unmask(xe, ring->evtchn) < 0 )
+                syslog(LOG_WARNING, "xenevtchn_unmask() failed");
+            pthread_cond_wait(&ring->cond, &ring->mutex);
+        }
+        pthread_mutex_unlock(&ring->mutex);
+
+        /* TODO: I/O handling. */
+    }
+
+    ring->thread_active = false;
+
+    return NULL;
+}
diff --git a/tools/9pfsd/xen-9pfsd.c b/tools/9pfsd/xen-9pfsd.c
index 6939d01574..73b6c3a30e 100644
--- a/tools/9pfsd/xen-9pfsd.c
+++ b/tools/9pfsd/xen-9pfsd.c
@@ -24,34 +24,632 @@
 
 #include <err.h>
 #include <errno.h>
+#include <fcntl.h>
+#include <poll.h>
+#include <pthread.h>
 #include <signal.h>
 #include <stdbool.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <sys/mman.h>
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <unistd.h>
-#include <xenevtchn.h>
 #include <xengnttab.h>
 #include <xenstore.h>
 
+#include "xen-9pfsd.h"
+
+/*
+ * List of currently known devices.
+ * The list itself is modified only in the main thread. When a device is being
+ * removed its memory needs to be freed after the I/O thread (if existing)
+ * has stopped.
+ */
+static XEN_TAILQ_HEAD(devhead, device) devs = XEN_TAILQ_HEAD_INITIALIZER(devs);
+
+struct path {
+    char path[100];
+};
+
 static volatile bool stop_me;
 static bool daemon_running;
 static struct xs_handle *xs;
 static xengnttab_handle *xg;
-static xenevtchn_handle *xe;
+static unsigned int now;
+
+xenevtchn_handle *xe;
 
 static void handle_stop(int sig)
 {
     stop_me = true;
 }
 
+static int check_host_path(device *device)
+{
+    struct stat statbuf;
+    char *path, *p;
+    int ret = 1;
+
+    if ( !device->host_path )
+        return 1;
+
+    /* Path must be absolute. */
+    if ( device->host_path[0] != '/' )
+        return 1;
+
+    /* No double "/". */
+    if ( strstr(device->host_path, "//") )
+        return 1;
+
+    /* No trailing "/" (includes refusing to share "/"). */
+    if ( device->host_path[strlen(device->host_path) - 1] == '/' )
+        return 1;
+
+    path = strdup(device->host_path);
+    if ( !path )
+    {
+        syslog(LOG_CRIT, "memory allocation failure!");
+        return 1;
+    }
+
+    for ( p = path; p; )
+    {
+        p = strchr(p + 1, '/');
+        if ( p )
+            *p = 0;
+        if ( !stat(path, &statbuf) )
+        {
+            if ( !(statbuf.st_mode & S_IFDIR) )
+                break;
+            if ( !p )
+            {
+                ret = 0;
+                break;
+            }
+            *p = '/';
+            continue;
+        }
+        if ( mkdir(path, 0777) )
+            break;
+        if ( p )
+            *p = '/';
+    }
+
+    free(path);
+    return ret;
+}
+
+static void construct_frontend_path(device *device, const char *node,
+                                    struct path *p)
+{
+    snprintf(p->path, sizeof(p->path), "/local/domain/%u/device/9pfs/%u/%s",
+             device->domid, device->devid, node);
+}
+
+static void construct_backend_path(device *device, const char *node,
+                                   struct path *p)
+{
+    snprintf(p->path, sizeof(p->path), "backend/xen_9pfs/%u/%u/%s",
+             device->domid, device->devid, node);
+}
+
+static char *read_backend_node(device *device, const char *node)
+{
+    struct path p;
+    char *val;
+    unsigned int len;
+
+    construct_backend_path(device, node, &p);
+    val = xs_read(xs, XBT_NULL, p.path, &len);
+
+    return val;
+}
+
+static unsigned int uint_from_string(char *string, unsigned int def)
+{
+    unsigned long val;
+    char *end;
+
+    if ( !string )
+        return def;
+
+    val = strtoul(string, &end, 10);
+    if ( *end || val > UINT_MAX )
+        val = def;
+    free(string);
+
+    return val;
+}
+
+static unsigned int read_backend_node_uint(device *device, const char *node,
+                                           unsigned int def)
+{
+    return uint_from_string(read_backend_node(device, node), def);
+}
+
+static unsigned int read_frontend_node_uint(device *device, const char *node,
+                                            unsigned int def)
+{
+    struct path p;
+    unsigned int len;
+
+    construct_frontend_path(device, node, &p);
+
+    return uint_from_string(xs_read(xs, XBT_NULL, p.path, &len), def);
+}
+
+static bool write_backend_node(device *device, const char *node,
+                               const char *val)
+{
+    struct path p;
+    unsigned int num_perms;
+    struct xs_permissions *old_perms;
+    struct xs_permissions perms[2] = {
+        { .id = 0, .perms = XS_PERM_NONE },
+        { .id = device->domid, .perms = XS_PERM_READ }
+    };
+
+    construct_backend_path(device, node, &p);
+    if ( !xs_write(xs, XBT_NULL, p.path, val, strlen(val)) )
+    {
+        syslog(LOG_ERR, "error writing bacḱend node \"%s\" for device %u/%u",
+               node, device->domid, device->devid);
+        return false;
+    }
+
+    old_perms = xs_get_permissions(xs, XBT_NULL, p.path, &num_perms);
+    if ( !old_perms )
+    {
+        syslog(LOG_ERR, "error getting permissions for \"%s\"", p.path);
+        return false;
+    }
+
+    perms[0] = old_perms[0];
+    free(old_perms);
+    if ( !xs_set_permissions(xs, XBT_NULL, p.path, perms, 2) )
+    {
+        syslog(LOG_ERR, "error setting permissions for \"%s\"", p.path);
+        return false;
+    }
+
+    return true;
+}
+
+static bool write_backend_node_uint(device *device, const char *node,
+                                   unsigned int val)
+{
+    char str[12];
+
+    snprintf(str, sizeof(str), "%u", val);
+
+    return write_backend_node(device, node, str);
+}
+
+static bool write_backend_state(device *device, enum xenbus_state state)
+{
+    struct path p;
+    char val[2];
+
+    snprintf(val, sizeof(val), "%u", state);
+    construct_backend_path(device, "state", &p);
+    if ( !xs_write(xs, XBT_NULL, p.path, val, 1) )
+    {
+        syslog(LOG_ERR, "error writing backend state %u for device %u/%u",
+               state, device->domid, device->devid);
+        return false;
+    }
+
+    device->backend_state = state;
+
+    return true;
+}
+
+static device *find_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+
+    XEN_TAILQ_FOREACH( device, &devs, list )
+    {
+        if ( domid == device->domid && devid == device->devid )
+            return device;
+    }
+
+    return NULL;
+}
+
+static void free_device(device *device)
+{
+    char token[20];
+    struct path p;
+
+    construct_frontend_path(device, "state", &p);
+    snprintf(token, sizeof(token), "%u/%u", device->domid, device->devid);
+    xs_unwatch(xs, p.path, token);
+
+    if ( device->root_fd >= 0 )
+        close(device->root_fd);
+
+    free(device->host_path);
+    free(device);
+}
+
+static device *new_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+    char token[20];
+    struct path p;
+    char *val;
+
+    device = calloc(1, sizeof(*device));
+    if ( !device )
+    {
+        syslog(LOG_CRIT, "Got no memory for new device %u/%u", domid, devid);
+        return NULL;
+    }
+
+    device->domid = domid;
+    device->devid = devid;
+    device->root_fd = -1;
+
+    construct_frontend_path(device, "state", &p);
+    snprintf(token, sizeof(token), "%u/%u", domid, devid);
+    if ( !xs_watch(xs, p.path, token) )
+    {
+        syslog(LOG_ERR, "Setting up watch for device %u/%u failed",
+               domid, devid);
+        free(device);
+        return NULL;
+    }
+
+    val = read_backend_node(device, "security_model");
+    if ( !val || strcmp(val, "none") )
+    {
+        syslog(LOG_ERR, "Security model \"%s\" for device %u/%u invalid.",
+               val, domid, devid);
+        free(val);
+        goto err;
+    }
+    free(val);
+
+    device->max_space = read_backend_node_uint(device, "max-space", 0);
+    device->max_files = read_backend_node_uint(device, "max-files", 0);
+    device->max_open_files =
+        read_backend_node_uint(device, "max-open-files", 0)
+        ?: MAX_OPEN_FILES_DEFAULT;
+    device->auto_delete = read_backend_node_uint(device, "auto-delete", 0);
+
+    device->host_path = read_backend_node(device, "path");
+    if ( check_host_path(device) )
+    {
+        syslog(LOG_ERR, "Host path \"%s\" for device %u/%u invalid.",
+               device->host_path, domid, devid);
+        goto err;
+    }
+    device->root_fd = open(device->host_path, O_RDONLY | O_DIRECTORY);
+    if ( device->root_fd < 0 )
+        goto err;
+
+    if ( !write_backend_node(device, "versions", "1") )
+        goto err;
+    if ( !write_backend_node_uint(device, "max-rings", MAX_RINGS) )
+        goto err;
+    if ( !write_backend_node_uint(device, "max-ring-page-order",
+                                 MAX_RING_ORDER) )
+        goto err;
+
+    if ( !write_backend_state(device, XenbusStateInitWait) )
+        goto err;
+
+    XEN_TAILQ_INSERT_TAIL(&devs, device, list);
+    syslog(LOG_INFO, "New device %u/%u added", domid, devid);
+
+    return device;
+
+ err:
+    free_device(device);
+    return NULL;
+}
+
+static void disconnect_ring(struct ring *ring)
+{
+    if ( !ring )
+        return;
+
+    if ( ring->thread_active )
+    {
+        ring->stop_thread = true;
+        pthread_cond_signal(&ring->cond);
+        pthread_join(ring->thread, NULL);
+        ring->stop_thread = false;
+    }
+
+    if ( ring->data.in )
+    {
+        xengnttab_unmap(xg, ring->data.in, 1 << ring->ring_order);
+        ring->data.in = NULL;
+    }
+    if ( ring->intf )
+    {
+        xengnttab_unmap(xg, ring->intf, 1 );
+        ring->intf = NULL;
+    }
+
+    if ( ring->evtchn )
+    {
+        xenevtchn_unbind(xe, ring->evtchn);
+        ring->evtchn = 0;
+    }
+
+    pthread_mutex_destroy(&ring->mutex);
+    pthread_cond_destroy(&ring->cond);
+}
+
+static void disconnect_guest(device *device)
+{
+    unsigned int ring_idx;
+
+    for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+    {
+        disconnect_ring(device->ring[ring_idx]);
+        free(device->ring[ring_idx]);
+        device->ring[ring_idx] = NULL;
+    }
+
+    device->num_rings = 0;
+}
+
+static void close_device(device *device, enum xenbus_state state)
+{
+    disconnect_guest(device);
+    write_backend_state(device, state);
+}
+
+static void connect_err(device *device, const char *msg)
+{
+    syslog(LOG_WARNING, "%s", msg);
+    close_device(device, XenbusStateClosed);
+}
+
+static void connect_device(device *device)
+{
+    unsigned int val;
+    unsigned int ring_idx;
+    char node[20];
+    struct ring *ring;
+    xenevtchn_port_or_error_t evtchn;
+
+    val = read_frontend_node_uint(device, "version", 0);
+    if ( val != 1 )
+        return connect_err(device, "frontend specifies illegal version");
+    device->num_rings = read_frontend_node_uint(device, "num-rings", 0);
+    if ( device->num_rings < 1 || device->num_rings > MAX_RINGS )
+        return connect_err(device, "frontend specifies illegal ring number");
+
+    for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+    {
+        ring = calloc(1, sizeof(*ring));
+        if ( !ring )
+            return connect_err(device, "could not allocate ring memory");
+        device->ring[ring_idx] = ring;
+        ring->device = device;
+        pthread_cond_init(&ring->cond, NULL);
+        pthread_mutex_init(&ring->mutex, NULL);
+
+        snprintf(node, sizeof(node), "event-channel-%u", ring_idx);
+        val = read_frontend_node_uint(device, node, 0);
+        if ( val == 0 )
+            return connect_err(device, "frontend specifies illegal evtchn");
+        evtchn = xenevtchn_bind_interdomain(xe, device->domid, val);
+        if ( evtchn < 0 )
+            return connect_err(device, "could not bind to event channel");
+        ring->evtchn = evtchn;
+
+        snprintf(node, sizeof(node), "ring-ref%u", ring_idx);
+        val = read_frontend_node_uint(device, node, 0);
+        if ( val == 0 )
+            return connect_err(device,
+                               "frontend specifies illegal grant for ring");
+        ring->intf = xengnttab_map_grant_ref(xg, device->domid, val,
+                                             PROT_READ | PROT_WRITE);
+        if ( !ring->intf )
+            return connect_err(device, "could not map interface page");
+        ring->ring_order = ring->intf->ring_order;
+        if ( ring->ring_order > MAX_RING_ORDER || ring->ring_order < 1 )
+            return connect_err(device, "frontend specifies illegal ring order");
+        ring->ring_size = XEN_FLEX_RING_SIZE(ring->ring_order);
+        ring->data.in = xengnttab_map_domain_grant_refs(xg,
+                                                        1 << ring->ring_order,
+                                                        device->domid,
+                                                        ring->intf->ref,
+                                                        PROT_READ | PROT_WRITE);
+        if ( !ring->data.in )
+            return connect_err(device, "could not map ring pages");
+        ring->data.out = ring->data.in + ring->ring_size;
+
+        if ( pthread_create(&ring->thread, NULL, io_thread, ring) )
+            return connect_err(device, "could not start I/O thread");
+        ring->thread_active = true;
+    }
+
+    write_backend_state(device, XenbusStateConnected);
+}
+
+static void remove_device(device *device)
+{
+    XEN_TAILQ_REMOVE(&devs, device, list);
+
+    disconnect_guest(device);
+    free_device(device);
+}
+
+static void remove_all_devices(void)
+{
+    device *device, *tmp;
+
+    XEN_TAILQ_FOREACH_SAFE( device, &devs, list, tmp )
+        remove_device(device);
+}
+
+static void frontend_changed(device *device)
+{
+    struct path p;
+    char *state, *end;
+    unsigned int len;
+    unsigned long new_state;
+
+    construct_frontend_path(device, "state", &p);
+    state = xs_read(xs, XBT_NULL, p.path, &len);
+    if ( !state )
+    {
+        close_device(device, XenbusStateClosed);
+        return;
+    }
+
+    new_state = strtoul(state, &end, 10);
+    if ( *end || new_state > XenbusStateReconfigured )
+    {
+        syslog(LOG_WARNING, "unknown state \"%s\" seen for device %u/%u", state,
+               device->domid, device->devid);
+        new_state = XenbusStateUnknown;
+    }
+    free(state);
+
+    if ( new_state == device->frontend_state )
+        return;
+
+    switch ( new_state )
+    {
+    case XenbusStateInitialising:
+        break;
+
+    case XenbusStateInitialised:
+        connect_device(device);
+        break;
+
+    case XenbusStateConnected:
+        break;
+
+    case XenbusStateClosing:
+        close_device(device, XenbusStateClosing);
+        break;
+
+    case XenbusStateClosed:
+        close_device(device, XenbusStateClosed);
+        break;
+
+    default:
+        syslog(LOG_WARNING, "not handled frontend state %lu for device %u/%u",
+               new_state, device->domid, device->devid);
+        break;
+    }
+
+    device->frontend_state = new_state;
+}
+
+static void check_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+
+    device = find_device(domid, devid);
+    if ( !device )
+    {
+        device = new_device(domid, devid);
+        if ( !device )
+            return;
+    }
+
+    device->last_seen = now;
+}
+
+static void scan_backend(void)
+{
+    char **doms;
+    unsigned int n_doms, dom;
+    char **devices;
+    unsigned int n_devs, dev;
+    char dom_path[24];
+    unsigned long domid, devid;
+    char *end;
+    device *device, *tmp;
+
+    now++;
+
+    doms = xs_directory(xs, XBT_NULL, "backend/xen_9pfs", &n_doms);
+    if ( doms == NULL )
+        return;
+
+    for ( dom = 0; dom < n_doms; dom++ )
+    {
+        errno = 0;
+        domid = strtoul(doms[dom], &end, 10);
+        if ( errno || *end || domid >= DOMID_FIRST_RESERVED )
+            continue;
+
+        snprintf(dom_path, sizeof(dom_path), "backend/xen_9pfs/%lu", domid);
+        devices = xs_directory(xs, XBT_NULL, dom_path, &n_devs);
+
+        for ( dev = 0; dev < n_devs; dev++ )
+        {
+            errno = 0;
+            devid = strtoul(devices[dev], &end, 10);
+            if ( errno || *end || devid > UINT_MAX )
+                continue;
+
+            check_device(domid, devid);
+        }
+
+        free(devices);
+    }
+
+    free(doms);
+
+    XEN_TAILQ_FOREACH_SAFE( device, &devs, list, tmp )
+    {
+        if ( device->last_seen != now )
+            remove_device(device);
+    }
+}
+
+static void handle_watch(char *path, char *token)
+{
+    unsigned int domid, devid;
+    device *device;
+
+    if ( !strcmp(token, "main") )
+    {
+        scan_backend();
+        return;
+    }
+
+    if ( sscanf(token, "%u/%u", &domid, &devid) != 2 )
+    {
+        syslog(LOG_WARNING, "unknown watch event %s %s", path, token);
+        return;
+    }
+
+    device = find_device(domid, devid);
+    if ( !device )
+    {
+        syslog(LOG_WARNING, "watch event for unknown device %u/%u",
+               domid, devid);
+        return;
+    }
+
+    frontend_changed(device);
+}
+
 static void close_all(void)
 {
     if ( daemon_running )
+    {
         xs_rm(xs, XBT_NULL, "libxl/xen-9pfs");
+        xs_unwatch(xs, "backend/xen_9pfs", "main");
+
+        remove_all_devices();
+    }
     if ( xe )
         xenevtchn_close(xe);
     if ( xg )
@@ -68,6 +666,33 @@ static void do_err(const char *msg)
     exit(1);
 }
 
+static void handle_event(void)
+{
+    xenevtchn_port_or_error_t evtchn;
+    device *device;
+    struct ring *ring;
+    unsigned int ring_idx;
+
+    evtchn = xenevtchn_pending(xe);
+    if ( evtchn < 0 )
+        do_err("xenevtchn_pending() failed");
+
+    XEN_TAILQ_FOREACH( device, &devs, list )
+    {
+        for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+        {
+            ring = device->ring[ring_idx];
+            if ( ring && ring->evtchn == evtchn )
+            {
+                pthread_mutex_lock(&ring->mutex);
+                pthread_cond_signal(&ring->cond);
+                pthread_mutex_unlock(&ring->mutex);
+                return;
+            }
+        }
+    }
+}
+
 static void xen_connect(void)
 {
     xs_transaction_t t;
@@ -124,6 +749,11 @@ int main(int argc, char *argv[])
     int syslog_mask = LOG_MASK(LOG_WARNING) | LOG_MASK(LOG_ERR) |
                       LOG_MASK(LOG_CRIT) | LOG_MASK(LOG_ALERT) |
                       LOG_MASK(LOG_EMERG);
+    char **watch;
+    struct pollfd p[2] = {
+        { .events = POLLIN },
+        { .events = POLLIN }
+    };
 
     umask(027);
     if ( getenv("XEN_9PFSD_VERBOSE") )
@@ -136,9 +766,26 @@ int main(int argc, char *argv[])
 
     xen_connect();
 
+    if ( !xs_watch(xs, "backend/xen_9pfs", "main") )
+        do_err("xs_watch() in main thread failed");
+    p[0].fd = xs_fileno(xs);
+    p[1].fd = xenevtchn_fd(xe);
+
+    scan_backend();
+
     while ( !stop_me )
     {
-        sleep(60);
+        while ( (p[0].revents & POLLIN) &&
+                (watch = xs_check_watch(xs)) != NULL )
+        {
+            handle_watch(watch[XS_WATCH_PATH], watch[XS_WATCH_TOKEN]);
+            free(watch);
+        }
+
+        if ( p[1].revents & POLLIN )
+            handle_event();
+
+        poll(p, 2, -1);
     }
 
     close_all();
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
new file mode 100644
index 0000000000..ff3562164a
--- /dev/null
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef XEN_9PFSD_H
+#define XEN_9PFSD_H
+
+#include <pthread.h>
+#include <stdbool.h>
+#include <xenevtchn.h>
+#include <xen_list.h>
+#include <xen/xen.h>
+#include <xen/io/xenbus.h>
+#include <xen/io/9pfs.h>
+
+#define MAX_RINGS                4
+#define MAX_RING_ORDER           9
+#define MAX_OPEN_FILES_DEFAULT   5
+
+typedef struct device device;
+
+struct ring {
+    device *device;
+    pthread_t thread;
+    bool thread_active;
+    bool stop_thread;
+    pthread_cond_t cond;
+    pthread_mutex_t mutex;
+
+    evtchn_port_t evtchn;
+    struct xen_9pfs_data_intf *intf;
+    unsigned int ring_order;
+    RING_IDX ring_size;
+    struct xen_9pfs_data data;
+};
+
+struct device {
+    /* Admin data. */
+    XEN_TAILQ_ENTRY(device) list;
+    unsigned int last_seen;    /* Set in scan_backend(). */
+    unsigned int domid;
+    unsigned int devid;
+
+    /* Tool side configuration data. */
+    char *host_path;
+    unsigned int max_space;
+    unsigned int max_files;
+    unsigned int max_open_files;
+    bool auto_delete;
+
+    /* Connection data. */
+    enum xenbus_state backend_state;
+    enum xenbus_state frontend_state;
+    unsigned int num_rings;
+    struct ring *ring[MAX_RINGS];
+    int root_fd;
+};
+
+extern xenevtchn_handle *xe;
+
+void *io_thread(void *arg);
+
+#endif /* XEN_9PFSD_H */
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:56:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:56:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681247.1059929 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfB-0004I3-5w; Thu, 15 Feb 2024 06:56:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681247.1059929; Thu, 15 Feb 2024 06:56:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfB-0004Hw-2j; Thu, 15 Feb 2024 06:56:05 +0000
Received: by outflank-mailman (input) for mailman id 681247;
 Thu, 15 Feb 2024 06:56:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVf9-0003c6-9Z
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:03 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 47689714-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:56:01 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 021DB1F855;
 Thu, 15 Feb 2024 06:56:01 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id BAC861346A;
 Thu, 15 Feb 2024 06:56:00 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id m40KLIC1zWVKSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47689714-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980161; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1Cx2zmy0kYHTdVFW0XAu4VXbYGYjbC4IGhNjDXCFbLc=;
	b=Xtk3g5xuR3Vh36Sdmx54bP9VKA0xFnPnQnkR2e4QsADoN4OSH99lVehaLMmT6V/Iyqp3A6
	u03WJuvTR49Gfc8vWj3JJed/irP3CM1LPLCDJmEqjGPVOcHIDt8Rjaxlf+iBxMcbzWV1M0
	+3wZtNOzWta6gawDNUa1/SD4/zYmf0I=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980161; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1Cx2zmy0kYHTdVFW0XAu4VXbYGYjbC4IGhNjDXCFbLc=;
	b=Xtk3g5xuR3Vh36Sdmx54bP9VKA0xFnPnQnkR2e4QsADoN4OSH99lVehaLMmT6V/Iyqp3A6
	u03WJuvTR49Gfc8vWj3JJed/irP3CM1LPLCDJmEqjGPVOcHIDt8Rjaxlf+iBxMcbzWV1M0
	+3wZtNOzWta6gawDNUa1/SD4/zYmf0I=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 03/21] tools/xen-9pfsd: add transport layer
Date: Thu, 15 Feb 2024 07:55:23 +0100
Message-Id: <20240215065541.21067-4-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [1.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: *
X-Spam-Score: 1.90
X-Spam-Flag: NO

Add the transport layer of 9pfs. This is basically the infrastructure
to receive requests from the frontend and to send the related answers
via the rings.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- rename put_request_bytes() (Jason Andryuk)
- rename get_request_bytes() and put_response_bytes() len parameter
  (Jason Andryuk)
- don't unmask event channel if error indicator is set (Jason Andryuk)
---
 tools/9pfsd/io.c        | 143 +++++++++++++++++++++++++++++++++++++++-
 tools/9pfsd/xen-9pfsd.h |  16 +++++
 2 files changed, 156 insertions(+), 3 deletions(-)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index eb7c136e09..4312a62dfe 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -6,39 +6,176 @@
  * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
  *
  * I/O thread handling.
+ *
+ * Only handle one request at a time, pushing out the complete response
+ * before looking for the next request.
  */
 
 #include <stdbool.h>
+#include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <xenctrl.h>           /* For cpu barriers. */
+#include <xen-tools/common-macros.h>
 
 #include "xen-9pfsd.h"
 
+/*
+ * Note that the ring names "in" and "out" are from the frontend's
+ * perspective, so the "in" ring will be used for responses to the frontend,
+ * while the "out" ring is used for requests from the frontend to the
+ * backend.
+ */
+static unsigned int ring_in_free(struct ring *ring)
+{
+    unsigned int queued;
+
+    queued = xen_9pfs_queued(ring->prod_pvt_in, ring->intf->in_cons,
+                             ring->ring_size);
+    xen_rmb();
+
+    return ring->ring_size - queued;
+}
+
+static unsigned int ring_out_data(struct ring *ring)
+{
+    unsigned int queued;
+
+    queued = xen_9pfs_queued(ring->intf->out_prod, ring->cons_pvt_out,
+                             ring->ring_size);
+    xen_rmb();
+
+    return queued;
+}
+
+static unsigned int get_request_bytes(struct ring *ring, unsigned int off,
+                                      unsigned int total_len)
+{
+    unsigned int size;
+    unsigned int out_data = ring_out_data(ring);
+    RING_IDX prod, cons;
+
+    size = min(total_len - off, out_data);
+    prod = xen_9pfs_mask(ring->intf->out_prod, ring->ring_size);
+    cons = xen_9pfs_mask(ring->cons_pvt_out, ring->ring_size);
+    xen_9pfs_read_packet(ring->buffer + off, ring->data.out, size,
+                         prod, &cons, ring->ring_size);
+
+    xen_rmb();           /* Read data out before setting visible consumer. */
+    ring->cons_pvt_out += size;
+    ring->intf->out_cons = ring->cons_pvt_out;
+
+    /* Signal that more space is available now. */
+    xenevtchn_notify(xe, ring->evtchn);
+
+    return size;
+}
+
+static unsigned int put_response_bytes(struct ring *ring, unsigned int off,
+                                       unsigned int total_len)
+{
+    unsigned int size;
+    unsigned int in_data = ring_in_free(ring);
+    RING_IDX prod, cons;
+
+    size = min(total_len - off, in_data);
+    prod = xen_9pfs_mask(ring->prod_pvt_in, ring->ring_size);
+    cons = xen_9pfs_mask(ring->intf->in_cons, ring->ring_size);
+    xen_9pfs_write_packet(ring->data.in, ring->buffer + off, size,
+                          &prod, cons, ring->ring_size);
+
+    xen_wmb();           /* Write data out before setting visible producer. */
+    ring->prod_pvt_in += size;
+    ring->intf->in_prod = ring->prod_pvt_in;
+
+    return size;
+}
+
 static bool io_work_pending(struct ring *ring)
 {
     if ( ring->stop_thread )
         return true;
-    return false;
+    if ( ring->error )
+        return false;
+    return ring->handle_response ? ring_in_free(ring) : ring_out_data(ring);
 }
 
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
+    unsigned int count = 0;
+    struct p9_header hdr;
+    bool in_hdr = true;
+
+    ring->max_size = ring->ring_size;
+    ring->buffer = malloc(ring->max_size);
+    if ( !ring->buffer )
+    {
+        syslog(LOG_CRIT, "memory allocation failure!");
+        return NULL;
+    }
 
     while ( !ring->stop_thread )
     {
         pthread_mutex_lock(&ring->mutex);
         if ( !io_work_pending(ring) )
         {
-            if ( xenevtchn_unmask(xe, ring->evtchn) < 0 )
+            if ( !ring->error && xenevtchn_unmask(xe, ring->evtchn) < 0 )
                 syslog(LOG_WARNING, "xenevtchn_unmask() failed");
             pthread_cond_wait(&ring->cond, &ring->mutex);
         }
         pthread_mutex_unlock(&ring->mutex);
 
-        /* TODO: I/O handling. */
+        if ( ring->stop_thread || ring->error )
+            continue;
+
+        if ( !ring->handle_response )
+        {
+            if ( in_hdr )
+            {
+                count += get_request_bytes(ring, count, sizeof(hdr));
+                if ( count != sizeof(hdr) )
+                    continue;
+                hdr = *(struct p9_header *)ring->buffer;
+                if ( hdr.size > ring->max_size || hdr.size < sizeof(hdr) )
+                {
+                    syslog(LOG_ERR, "%u.%u specified illegal request length %u",
+                           ring->device->domid, ring->device->devid, hdr.size);
+                    ring->error = true;
+                    continue;
+                }
+                in_hdr = false;
+            }
+
+            count += get_request_bytes(ring, count, hdr.size);
+            if ( count < hdr.size )
+                continue;
+
+            /* TODO: handle request (will rewrite hdr.size). */
+
+            ring->handle_response = true;
+            hdr.size = ((struct p9_header *)ring->buffer)->size;
+            count = 0;
+        }
+
+        if ( ring->handle_response )
+        {
+            count += put_response_bytes(ring, count, hdr.size);
+
+            if ( count == hdr.size )
+            {
+                /* Signal presence of response. */
+                xenevtchn_notify(xe, ring->evtchn);
+
+                ring->handle_response = false;
+                in_hdr = true;
+                count = 0;
+            }
+        }
     }
 
+    free(ring->buffer);
+
     ring->thread_active = false;
 
     return NULL;
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index ff3562164a..d587f59a32 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -15,6 +15,12 @@
 #define MAX_RING_ORDER           9
 #define MAX_OPEN_FILES_DEFAULT   5
 
+struct p9_header {
+    uint32_t size;
+    uint8_t cmd;
+    uint16_t tag;
+} __attribute__((packed));
+
 typedef struct device device;
 
 struct ring {
@@ -29,7 +35,17 @@ struct ring {
     struct xen_9pfs_data_intf *intf;
     unsigned int ring_order;
     RING_IDX ring_size;
+
+    /* Transport layer data. */
     struct xen_9pfs_data data;
+    RING_IDX prod_pvt_in;
+    RING_IDX cons_pvt_out;
+
+    /* Request and response handling. */
+    uint32_t max_size;
+    bool error;             /* Protocol error - stop processing. */
+    bool handle_response;   /* Main loop now handling response. */
+    void *buffer;           /* Request/response buffer. */
 };
 
 struct device {
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:56:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:56:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681248.1059939 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfG-0004jj-JM; Thu, 15 Feb 2024 06:56:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681248.1059939; Thu, 15 Feb 2024 06:56:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfG-0004jY-FP; Thu, 15 Feb 2024 06:56:10 +0000
Received: by outflank-mailman (input) for mailman id 681248;
 Thu, 15 Feb 2024 06:56:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVfF-0003c6-9Z
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:09 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4acbe281-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:56:07 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 98AD021DA6;
 Thu, 15 Feb 2024 06:56:06 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 5CCC01346A;
 Thu, 15 Feb 2024 06:56:06 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id iYgtFYa1zWVNSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4acbe281-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980166; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=O73f+E3GNfH4W5rlXCuJYVhh6QCuB0n/PfhAGghm1+4=;
	b=CEwJR+KoK/c7xZ16rrOpsTN5YHy9Aq3nXeAA+yuGbdcTcEhrbaSva+cMOaEBSvmtqDt+oD
	6ghufNZQcmA7o/2rLfsK3QJBMXtPeNjjllaMit4r8FA7gnxnxD4WSLj9zSbvw0v1RfMMsS
	AnDu5B6vxgKrULAebn+mc4xzDfuAf4A=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980166; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=O73f+E3GNfH4W5rlXCuJYVhh6QCuB0n/PfhAGghm1+4=;
	b=CEwJR+KoK/c7xZ16rrOpsTN5YHy9Aq3nXeAA+yuGbdcTcEhrbaSva+cMOaEBSvmtqDt+oD
	6ghufNZQcmA7o/2rLfsK3QJBMXtPeNjjllaMit4r8FA7gnxnxD4WSLj9zSbvw0v1RfMMsS
	AnDu5B6vxgKrULAebn+mc4xzDfuAf4A=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 04/21] tools/xen-9pfsd: add 9pfs response generation support
Date: Thu, 15 Feb 2024 07:55:24 +0100
Message-Id: <20240215065541.21067-5-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=CEwJR+Ko
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [0.49 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 0.49
X-Rspamd-Queue-Id: 98AD021DA6
X-Spam-Flag: NO

Add support for generation a 9pfs protocol response via a format based
approach.

Strings are stored in a per device string buffer and they are
referenced via their offset in this buffer. This allows to avoid
having to dynamically allocate memory for each single string.

As a first user of the response handling add a generic p9_error()
function which will be used to return any error to the client.

Add all format parsing variants in order to avoid additional code churn
later when adding the users of those variants. Prepare a special case
for the "read" case already (format character 'D'): in order to avoid
adding another buffer for read data support doing the read I/O directly
into the response buffer.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- check parameter size limits (Jason Andryuk)
V3:
- use new unaligned access macros (Jason Andryuk)
V4:
- use recursion in fill_buffer() as a preparation for reading dirs
---
 tools/9pfsd/io.c        | 217 +++++++++++++++++++++++++++++++++++++++-
 tools/9pfsd/xen-9pfsd.h |   3 +
 2 files changed, 219 insertions(+), 1 deletion(-)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 4312a62dfe..4a44c70c4d 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -11,6 +11,7 @@
  * before looking for the next request.
  */
 
+#include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
@@ -20,6 +21,16 @@
 
 #include "xen-9pfsd.h"
 
+/* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
+#define P9_CMD_ERROR      107
+
+struct p9_qid {
+    uint8_t type;
+#define QID_TYPE_DIR      0x80
+    uint32_t version;
+    uint64_t path;
+};
+
 /*
  * Note that the ring names "in" and "out" are from the frontend's
  * perspective, so the "in" ring will be used for responses to the frontend,
@@ -100,6 +111,200 @@ static bool io_work_pending(struct ring *ring)
     return ring->handle_response ? ring_in_free(ring) : ring_out_data(ring);
 }
 
+static void fmt_err(const char *fmt)
+{
+    syslog(LOG_CRIT, "illegal format %s passed to fill_buffer()", fmt);
+    exit(1);
+}
+
+/*
+ * Fill buffer with response data.
+ * fmt is a sequence of format characters. Supported characters are:
+ * a: an array (2 bytes number of elements + the following format as elements)
+ *    The number of elements is passed in the first unsigned int parameter, the
+ *    next parameter is a pointer to an array of elements as denoted by the next
+ *    format character.
+ * b: 1 byte unsigned integer
+ * u: 2 byte unsigned integer
+ *    The parameter is a pointer to a uint16_t value
+ * D: Data blob (4 byte length + <length> bytes)
+ *    2 parameters are consumed, first an unsigned int for the length, then a
+ *    pointer to the first uint8_t value.
+ *    No array support.
+ * L: 8 byte unsigned integer
+ *    The parameter is a pointer to a uint64_t value
+ * Q: Qid (struct p9_qid)
+ * S: String (2 byte length + <length> characters)
+ *    The length is obtained via strlen() of the parameter, being a pointer
+ *    to the first character of the string
+ * U: 4 byte unsigned integer
+ *    The parameter is a pointer to a uint32_t value
+ */
+static void fill_buffer_at(void **data, const char *fmt, ...);
+static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
+{
+    const char *f;
+    const void *par;
+    const char *str_val;
+    const struct p9_qid *qid;
+    unsigned int len;
+    unsigned int array_sz = 0;
+    unsigned int elem_sz = 0;
+
+    for ( f = fmt; *f; f++ )
+    {
+        if ( !array_sz )
+            par = va_arg(ap, const void *);
+        else
+        {
+            par += elem_sz;
+            array_sz--;
+        }
+
+        switch ( *f )
+        {
+        case 'a':
+            f++;
+            if ( !*f || array_sz )
+                fmt_err(fmt);
+            array_sz = *(const unsigned int *)par;
+            if ( array_sz > 0xffff )
+            {
+                syslog(LOG_CRIT, "array size %u in fill_buffer()", array_sz);
+                exit(1);
+            }
+            put_unaligned(array_sz, (uint16_t *)*data);
+            *data += sizeof(uint16_t);
+            par = va_arg(ap, const void *);
+            elem_sz = 0;
+            break;
+
+        case 'b':
+            put_unaligned(*(const uint8_t *)par, (uint8_t *)*data);
+            elem_sz = sizeof(uint8_t);
+            *data += sizeof(uint8_t);
+            break;
+
+        case 'u':
+            put_unaligned(*(const uint16_t *)par, (uint16_t *)*data);
+            elem_sz = sizeof(uint16_t);
+            *data += sizeof(uint16_t);
+            break;
+
+        case 'D':
+            if ( array_sz )
+                fmt_err(fmt);
+            len = *(const unsigned int *)par;
+            put_unaligned(len, (uint32_t *)*data);
+            *data += sizeof(uint32_t);
+            par = va_arg(ap, const void *);
+            if ( *data != par )
+                memcpy(*data, par, len);
+            *data += len;
+            break;
+
+        case 'L':
+            put_unaligned(*(const uint64_t *)par, (uint64_t *)*data);
+            elem_sz = sizeof(uint64_t);
+            *data += sizeof(uint64_t);
+            break;
+
+        case 'Q':
+            qid = par;
+            elem_sz = sizeof(*qid);
+            fill_buffer_at(data, "bUL", &qid->type, &qid->version, &qid->path);
+            break;
+
+        case 'S':
+            str_val = par;
+            elem_sz = sizeof(str_val);
+            len = strlen(str_val);
+            if ( len > 0xffff )
+            {
+                syslog(LOG_CRIT, "string length %u in fill_buffer()", len);
+                exit(1);
+            }
+            put_unaligned(len, (uint16_t *)*data);
+            *data += sizeof(uint16_t);
+            memcpy(*data, str_val, len);
+            *data += len;
+            break;
+
+        case 'U':
+            put_unaligned(*(const uint32_t *)par, (uint32_t *)*data);
+            elem_sz = sizeof(uint32_t);
+            *data += sizeof(uint32_t);
+            break;
+
+        default:
+            fmt_err(fmt);
+        }
+
+        if ( array_sz )
+            f--;
+    }
+}
+
+static void fill_buffer_at(void **data, const char *fmt, ...)
+{
+    va_list ap;
+
+    va_start(ap, fmt);
+    vfill_buffer_at(data, fmt, ap);
+    va_end(ap);
+}
+
+static void fill_buffer(struct ring *ring, uint8_t cmd, uint16_t tag,
+                        const char *fmt, ...)
+{
+    struct p9_header *hdr = ring->buffer;
+    void *data = hdr + 1;
+    va_list ap;
+
+    hdr->cmd = cmd;
+    hdr->tag = tag;
+
+    va_start(ap, fmt);
+    vfill_buffer_at(&data, fmt, ap);
+    va_end(ap);
+
+    hdr->size = data - ring->buffer;
+}
+
+static unsigned int add_string(struct ring *ring, const char *str,
+                               unsigned int len)
+{
+    char *tmp;
+    unsigned int ret;
+
+    if ( ring->str_used + len + 1 > ring->str_size )
+    {
+        tmp = realloc(ring->str, ring->str_used + len + 1);
+        if ( !tmp )
+            return ~0;
+        ring->str = tmp;
+        ring->str_size = ring->str_used + len + 1;
+    }
+
+    ret = ring->str_used;
+    memcpy(ring->str + ret, str, len);
+    ring->str_used += len;
+    ring->str[ring->str_used++] = 0;
+
+    return ret;
+}
+
+static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
+{
+    unsigned int erroff;
+
+    strerror_r(err, ring->buffer, ring->ring_size);
+    erroff = add_string(ring, ring->buffer, strlen(ring->buffer));
+    fill_buffer(ring, P9_CMD_ERROR, tag, "SU",
+                erroff != ~0 ? ring->str + erroff : "cannot allocate memory",
+                &err);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -151,7 +356,16 @@ void *io_thread(void *arg)
             if ( count < hdr.size )
                 continue;
 
-            /* TODO: handle request (will rewrite hdr.size). */
+            ring->str_used = 0;
+
+            switch ( hdr.cmd )
+            {
+            default:
+                syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
+                       ring->device->domid, ring->device->devid, hdr.cmd);
+                p9_error(ring, hdr.tag, EOPNOTSUPP);
+                break;
+            }
 
             ring->handle_response = true;
             hdr.size = ((struct p9_header *)ring->buffer)->size;
@@ -174,6 +388,7 @@ void *io_thread(void *arg)
         }
     }
 
+    free(ring->str);
     free(ring->buffer);
 
     ring->thread_active = false;
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index d587f59a32..0cde0d2bb8 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -46,6 +46,9 @@ struct ring {
     bool error;             /* Protocol error - stop processing. */
     bool handle_response;   /* Main loop now handling response. */
     void *buffer;           /* Request/response buffer. */
+    char *str;              /* String work space. */
+    unsigned int str_size;  /* Size of *str. */
+    unsigned int str_used;  /* Currently used size of *str. */
 };
 
 struct device {
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:56:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:56:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681252.1059949 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfK-0005FU-Th; Thu, 15 Feb 2024 06:56:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681252.1059949; Thu, 15 Feb 2024 06:56:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfK-0005FK-Pf; Thu, 15 Feb 2024 06:56:14 +0000
Received: by outflank-mailman (input) for mailman id 681252;
 Thu, 15 Feb 2024 06:56:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVfJ-0003NA-Pg
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:13 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4e2dc186-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:56:12 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 3E03821DA6;
 Thu, 15 Feb 2024 06:56:12 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 0136B1346A;
 Thu, 15 Feb 2024 06:56:11 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id M05jOou1zWVUSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e2dc186-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980172; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Ep5GCYtOAcPi9Z1A4O1nBYAfiV8kAZfwfen9awb2d94=;
	b=AViuvk/q5wD2sGrPfa+YKUMCAxWbfDnm6j86JHxbAPz9pblBx1uuNjaOhx7GXSIVkaMQAr
	UYWYQYXLIGmvB9KCPlJH2f6dYVTXC84ev9Q6mxLxVzxtGGXbfgZ5zUFk/BnpwHISSyJj/T
	oKnnf5QRzNIqXvpSK8pMvxyyo//RJ5o=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980172; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Ep5GCYtOAcPi9Z1A4O1nBYAfiV8kAZfwfen9awb2d94=;
	b=AViuvk/q5wD2sGrPfa+YKUMCAxWbfDnm6j86JHxbAPz9pblBx1uuNjaOhx7GXSIVkaMQAr
	UYWYQYXLIGmvB9KCPlJH2f6dYVTXC84ev9Q6mxLxVzxtGGXbfgZ5zUFk/BnpwHISSyJj/T
	oKnnf5QRzNIqXvpSK8pMvxyyo//RJ5o=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 05/21] tools/xen-9pfsd: add 9pfs version request support
Date: Thu, 15 Feb 2024 07:55:25 +0100
Message-Id: <20240215065541.21067-6-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [1.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: *
X-Spam-Score: 1.90
X-Spam-Flag: NO

Add the version request of the 9pfs protocol. For the version use the
"9P2000.u" variant, as it is supported by Mini-OS and Linux.

For the request parsing add all format items needed even in future in
order to avoid code churn for those additions later.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V3:
- use unaligned helper macros (Jason Andryuk)
---
 tools/9pfsd/io.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 201 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 4a44c70c4d..839dd1112c 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -22,8 +22,12 @@
 #include "xen-9pfsd.h"
 
 /* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
+#define P9_CMD_VERSION    100
 #define P9_CMD_ERROR      107
 
+#define P9_MIN_MSIZE      2048
+#define P9_VERSION        "9P2000.u"
+
 struct p9_qid {
     uint8_t type;
 #define QID_TYPE_DIR      0x80
@@ -294,6 +298,169 @@ static unsigned int add_string(struct ring *ring, const char *str,
     return ret;
 }
 
+static bool chk_data(struct ring *ring, void *data, unsigned int len)
+{
+    struct p9_header *hdr = ring->buffer;
+
+    if ( data + len <= ring->buffer + hdr->size )
+        return true;
+
+    errno = E2BIG;
+
+    return false;
+}
+
+static bool fill_data_elem(void **par, void **array, unsigned int *array_sz,
+                           unsigned int elem_sz, void *data)
+{
+    if ( *array_sz && !*array )
+    {
+        *array = calloc(*array_sz, elem_sz);
+        if ( !*array )
+            return false;
+        *par = *array;
+    }
+
+    memcpy(*par, data, elem_sz);
+
+    if ( *array_sz )
+    {
+        *par += elem_sz;
+        *array_sz -= 1;
+    }
+
+    return true;
+}
+
+/*
+ * Fill variables with request data.
+ * fmt is a sequence of format characters. Supported characters are:
+ * a: an array (2 bytes number of elements + the following format as elements)
+ *    The number of elements is stored in the first unsigned int parameter, the
+ *    next parameter is a pointer to an array of elements as denoted by the next
+ *    format character. The array is allocated dynamically.
+ * b: 1 byte unsigned integer
+ *    The value is stored in the next parameter with type uint8_t.
+ * D: Data blob (4 byte length + <length> bytes)
+ *    2 parameters are consumed, first an unsigned int for the length, then a
+ *    pointer to the first uint8_t value.
+ *    No array support.
+ * L: 8 byte unsigned integer
+ *    The value is stored in the next parameter with type uint64_t.
+ * S: String (2 byte length + <length> characters)
+ *    The 0-terminated string is stored in device->str + off, off is stored in
+ *    the next parameter with type unsigned int.
+ * U: 4 byte unsigned integer
+ *    The value is stored in the next parameter with type uint32_t.
+ *
+ * Return value: number of filled variables, errno will be set in case of
+ *   error.
+ */
+static int fill_data(struct ring *ring, const char *fmt, ...)
+{
+    struct p9_header *hdr = ring->buffer;
+    void *data = hdr + 1;
+    void *par;
+    unsigned int pars = 0;
+    const char *f;
+    va_list ap;
+    unsigned int len;
+    unsigned int str_off;
+    unsigned int array_sz = 0;
+    void **array = NULL;
+
+    va_start(ap, fmt);
+
+    for ( f = fmt; *f; f++ )
+    {
+        if ( !array_sz )
+            par = va_arg(ap, void *);
+
+        switch ( *f )
+        {
+        case 'a':
+            f++;
+            if ( !*f || array_sz )
+                fmt_err(fmt);
+            if ( !chk_data(ring, data, sizeof(uint16_t)) )
+                return pars;
+            array_sz = get_unaligned((uint16_t *)data);
+            data += sizeof(uint16_t);
+            *(unsigned int *)par = array_sz;
+            array = va_arg(ap, void **);
+            *array = NULL;
+            break;
+
+        case 'b':
+            if ( !chk_data(ring, data, sizeof(uint8_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint8_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint8_t);
+            break;
+
+        case 'D':
+            if ( array_sz )
+                fmt_err(fmt);
+            if ( !chk_data(ring, data, sizeof(uint32_t)) )
+                return pars;
+            len = get_unaligned((uint32_t *)data);
+            data += sizeof(uint32_t);
+            *(unsigned int *)par = len;
+            par = va_arg(ap, void *);
+            if ( !chk_data(ring, data, len) )
+                return pars;
+            memcpy(par, data, len);
+            data += len;
+            break;
+
+        case 'L':
+            if ( !chk_data(ring, data, sizeof(uint64_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint64_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint64_t);
+            break;
+
+        case 'S':
+            if ( !chk_data(ring, data, sizeof(uint16_t)) )
+                return pars;
+            len = get_unaligned((uint16_t *)data);
+            data += sizeof(uint16_t);
+            if ( !chk_data(ring, data, len) )
+                return pars;
+            str_off = add_string(ring, data, len);
+            if ( str_off == ~0 )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(unsigned int),
+                                 &str_off) )
+                return pars;
+            data += len;
+            break;
+
+        case 'U':
+            if ( !chk_data(ring, data, sizeof(uint32_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint32_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint32_t);
+            break;
+
+        default:
+            fmt_err(fmt);
+        }
+
+        if ( array_sz )
+            f--;
+        pars++;
+    }
+
+    return pars;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -305,6 +472,36 @@ static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
                 &err);
 }
 
+static void p9_version(struct ring *ring, struct p9_header *hdr)
+{
+    uint32_t max_size;
+    unsigned int off;
+    char *version;
+    int ret;
+
+    ret = fill_data(ring, "US", &max_size, &off);
+    if ( ret != 2 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    if ( max_size < P9_MIN_MSIZE )
+    {
+        p9_error(ring, hdr->tag, EMSGSIZE);
+        return;
+    }
+
+    if ( max_size < ring->max_size )
+        ring->max_size = max_size;
+
+    version = ring->str + off;
+    if ( strcmp(version, P9_VERSION) )
+        version = "unknown";
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "US", &ring->max_size, version);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -360,6 +557,10 @@ void *io_thread(void *arg)
 
             switch ( hdr.cmd )
             {
+            case P9_CMD_VERSION:
+                p9_version(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:56:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:56:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681258.1059959 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfS-0005o1-58; Thu, 15 Feb 2024 06:56:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681258.1059959; Thu, 15 Feb 2024 06:56:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfS-0005nu-1v; Thu, 15 Feb 2024 06:56:22 +0000
Received: by outflank-mailman (input) for mailman id 681258;
 Thu, 15 Feb 2024 06:56:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVfQ-0003c6-5i
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:20 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 517aa5e1-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:56:18 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id C9E2F21DA6;
 Thu, 15 Feb 2024 06:56:17 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 97AE61346A;
 Thu, 15 Feb 2024 06:56:17 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 1tWJI5G1zWVaSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 517aa5e1-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980177; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=BYco3d2a7SkA/imW9UK5aS4CeR/UeIsWQgnqFK/FENQ=;
	b=g9fBmyWZdHM3PCYTBMucFnsB5BRZGYNacaQPLmGhTu13H8jBOWPdd82YR5pknXlUZnK3la
	233vcY2eL06TPujrWhsotVa5NqRFBDrPLm9tP2mCiJSj2E1xce9lM19Ht5W/gT5rpsT8IE
	qz7nTNGnDccWGH7bj0O4cEqoO8DrqNE=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980177; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=BYco3d2a7SkA/imW9UK5aS4CeR/UeIsWQgnqFK/FENQ=;
	b=g9fBmyWZdHM3PCYTBMucFnsB5BRZGYNacaQPLmGhTu13H8jBOWPdd82YR5pknXlUZnK3la
	233vcY2eL06TPujrWhsotVa5NqRFBDrPLm9tP2mCiJSj2E1xce9lM19Ht5W/gT5rpsT8IE
	qz7nTNGnDccWGH7bj0O4cEqoO8DrqNE=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 06/21] tools/xen-9pfsd: add 9pfs attach request support
Date: Thu, 15 Feb 2024 07:55:26 +0100
Message-Id: <20240215065541.21067-7-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=g9fBmyWZ
X-Spamd-Result: default: False [1.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 1.69
X-Rspamd-Queue-Id: C9E2F21DA6
X-Spam-Level: *
X-Spam-Flag: NO
X-Spamd-Bar: +

Add the attach request of the 9pfs protocol. This introduces the "fid"
scheme of the 9pfs protocol.

As this will be needed later, use a dedicated memory allocation
function in alloc_fid() and prepare a fid reference count.

For filling the qid data take the approach from the qemu 9pfs backend
implementation.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- make fill_qid() parameter stbuf const (Jason Andryuk)
- free fids after disconnecting guest (Jason Andryuk)
V3:
- only store relative path in fid (Jason Andryuk)
V4:
- store a path directly usable by *at() functions in fid (Jason Andryuk)
---
 tools/9pfsd/io.c        | 164 ++++++++++++++++++++++++++++++++++++++++
 tools/9pfsd/xen-9pfsd.c |   6 ++
 tools/9pfsd/xen-9pfsd.h |  14 ++++
 3 files changed, 184 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 839dd1112c..4f6acefc87 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -11,11 +11,14 @@
  * before looking for the next request.
  */
 
+#include <assert.h>
 #include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <sys/types.h>
+#include <sys/stat.h>
 #include <xenctrl.h>           /* For cpu barriers. */
 #include <xen-tools/common-macros.h>
 
@@ -23,6 +26,7 @@
 
 /* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
 #define P9_CMD_VERSION    100
+#define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
 
 #define P9_MIN_MSIZE      2048
@@ -461,6 +465,126 @@ static int fill_data(struct ring *ring, const char *fmt, ...)
     return pars;
 }
 
+static struct p9_fid *find_fid(device *device, unsigned int fid)
+{
+    struct p9_fid *fidp;
+
+    XEN_TAILQ_FOREACH(fidp, &device->fids, list)
+    {
+        if ( fidp->fid == fid )
+            return fidp;
+    }
+
+    return NULL;
+}
+
+static struct p9_fid *alloc_fid_mem(device *device, unsigned int fid,
+                                    const char *path)
+{
+    struct p9_fid *fidp;
+    size_t pathlen;
+
+    pathlen = strlen(path);
+    fidp = calloc(sizeof(*fidp) + pathlen + 1, 1);
+    if ( !fidp )
+        return NULL;
+
+    fidp->fid = fid;
+    strncpy(fidp->path, path, pathlen);
+
+    return fidp;
+}
+
+static struct p9_fid *alloc_fid(device *device, unsigned int fid,
+                                const char *path)
+{
+    struct p9_fid *fidp = NULL;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    if ( find_fid(device, fid) )
+    {
+        errno = EBADFD;
+        goto out;
+    }
+
+    if ( device->n_fids >= device->max_open_files )
+    {
+        errno = EMFILE;
+        goto out;
+    }
+
+    fidp = alloc_fid_mem(device, fid, path);
+    if ( !fidp )
+        goto out;
+
+    fidp->ref = 1;
+    XEN_TAILQ_INSERT_HEAD(&device->fids, fidp, list);
+    device->n_fids++;
+
+ out:
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    return fidp;
+}
+
+static void free_fid(device *device, struct p9_fid *fidp)
+{
+    if ( !fidp )
+        return;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    fidp->ref--;
+    if ( !fidp->ref )
+    {
+        device->n_fids--;
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        free(fidp);
+    }
+
+    pthread_mutex_unlock(&device->fid_mutex);
+}
+
+void free_fids(device *device)
+{
+    struct p9_fid *fidp;
+
+    while ( (fidp = XEN_TAILQ_FIRST(&device->fids)) != NULL )
+    {
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        free(fidp);
+    }
+}
+
+static const char *relpath_from_path(const char *path)
+{
+    if (!strcmp(path, "/"))
+        return ".";
+
+    return (path[0] == '/') ? path + 1 : path;
+}
+
+static int fill_qid(device *device, const char *path, struct p9_qid *qid,
+                    const struct stat *stbuf)
+{
+    struct stat st;
+
+    if ( !stbuf )
+    {
+        if ( fstatat(device->root_fd, path, &st, 0) )
+            return errno;
+
+        stbuf = &st;
+    }
+
+    qid->type = S_ISDIR(stbuf->st_mode) ? QID_TYPE_DIR : 0;
+    qid->version = stbuf->st_mtime ^ (stbuf->st_size << 8);
+    qid->path = stbuf->st_ino;
+
+    return 0;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -502,6 +626,42 @@ static void p9_version(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "US", &ring->max_size, version);
 }
 
+static void p9_attach(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint32_t dummy_u32;
+    unsigned int dummy_uint;
+    struct p9_qid qid;
+    int ret;
+
+    ret = fill_data(ring, "UUSSU", &fid, &dummy_u32, &dummy_uint, &dummy_uint,
+                    &dummy_u32);
+    if ( ret != 5 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    device->root_fid = alloc_fid(device, fid, relpath_from_path("/"));
+    if ( !device->root_fid )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    ret = fill_qid(device, device->root_fid->path, &qid, NULL);
+    if ( ret )
+    {
+        free_fid(device, device->root_fid);
+        device->root_fid = NULL;
+        p9_error(ring, hdr->tag, ret);
+        return;
+    }
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "Q", &qid);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -561,6 +721,10 @@ void *io_thread(void *arg)
                 p9_version(ring, &hdr);
                 break;
 
+            case P9_CMD_ATTACH:
+                p9_attach(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/9pfsd/xen-9pfsd.c b/tools/9pfsd/xen-9pfsd.c
index 73b6c3a30e..b83b896bcc 100644
--- a/tools/9pfsd/xen-9pfsd.c
+++ b/tools/9pfsd/xen-9pfsd.c
@@ -304,6 +304,9 @@ static device *new_device(unsigned int domid, unsigned int devid)
         return NULL;
     }
 
+    pthread_mutex_init(&device->fid_mutex, NULL);
+    XEN_TAILQ_INIT(&device->fids);
+
     val = read_backend_node(device, "security_model");
     if ( !val || strcmp(val, "none") )
     {
@@ -399,6 +402,8 @@ static void disconnect_guest(device *device)
     }
 
     device->num_rings = 0;
+
+    free_fids(device);
 }
 
 static void close_device(device *device, enum xenbus_state state)
@@ -482,6 +487,7 @@ static void remove_device(device *device)
     XEN_TAILQ_REMOVE(&devs, device, list);
 
     disconnect_guest(device);
+    pthread_mutex_destroy(&device->fid_mutex);
     free_device(device);
 }
 
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index 0cde0d2bb8..6c755411a7 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -21,6 +21,13 @@ struct p9_header {
     uint16_t tag;
 } __attribute__((packed));
 
+struct p9_fid {
+    XEN_TAILQ_ENTRY(struct p9_fid) list;
+    unsigned int fid;
+    unsigned int ref;
+    char path[];
+};
+
 typedef struct device device;
 
 struct ring {
@@ -71,10 +78,17 @@ struct device {
     unsigned int num_rings;
     struct ring *ring[MAX_RINGS];
     int root_fd;
+
+    /* File system handling. */
+    pthread_mutex_t fid_mutex;
+    XEN_TAILQ_HEAD(fidhead, struct p9_fid) fids;
+    struct p9_fid *root_fid;
+    unsigned int n_fids;
 };
 
 extern xenevtchn_handle *xe;
 
 void *io_thread(void *arg);
+void free_fids(device *device);
 
 #endif /* XEN_9PFSD_H */
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:56:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:56:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681261.1059969 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfY-0006Pg-Hn; Thu, 15 Feb 2024 06:56:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681261.1059969; Thu, 15 Feb 2024 06:56:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfY-0006PV-Dx; Thu, 15 Feb 2024 06:56:28 +0000
Received: by outflank-mailman (input) for mailman id 681261;
 Thu, 15 Feb 2024 06:56:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVfX-0003c6-FI
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:27 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 55f39920-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:56:25 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 666731F850;
 Thu, 15 Feb 2024 06:56:23 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 35FDE1346A;
 Thu, 15 Feb 2024 06:56:23 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id gxCSC5e1zWVfSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 55f39920-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980185; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=K0xuqXiEC4iAEod/g/QSMGJfwxMyyt5jqb3GT9oAAzI=;
	b=teRIGcmK2cVwHGBZH1h5M+3LE7TS/gZRbBIMHLjKZCGKIhMsJxQqDzi15bmm7MSbRorJ49
	mc/W2n78GeQI4Nm/+ZT2Fe+XASQbCzJVTutORpery0/SRvMF0qQPMHHV+YVfSNweSfLldU
	TFRuAEhsyrmQranIMYfLiFsJN5F+ViE=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980183; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=K0xuqXiEC4iAEod/g/QSMGJfwxMyyt5jqb3GT9oAAzI=;
	b=mhjihxDU/oekcJz7V/y7aJiB3WnFsCBDLK0/7+XmFpt01zxvXOrFUeFBPc1zgWu3zmG8Hb
	9PDXEajRYxggB9BMt/tW+PS7PjI9fpu5kRqdmdX//hyuOtbL9PYcO6pJKZrtAY7fPmvze3
	6UKb3fZc/bahwzeoPqcYXlTcIdsbjVg=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 07/21] tools/xen-9pfsd: add 9pfs walk request support
Date: Thu, 15 Feb 2024 07:55:27 +0100
Message-Id: <20240215065541.21067-8-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: 0.70
X-Spamd-Result: default: False [0.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-0.999];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[citrix.com:email,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add the walk request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- don't allow walking across symbolic links
V6:
- use EBADF instead of EBADFD (Andrew Cooper)
- use strncpy() with strlen() + 1 (Andrew Cooper)
---
 tools/9pfsd/io.c        | 177 +++++++++++++++++++++++++++++++++++++++-
 tools/9pfsd/xen-9pfsd.h |   1 +
 2 files changed, 175 insertions(+), 3 deletions(-)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 4f6acefc87..498f275a27 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -28,9 +28,11 @@
 #define P9_CMD_VERSION    100
 #define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
+#define P9_CMD_WALK       110
 
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
+#define P9_WALK_MAXELEM   16
 
 struct p9_qid {
     uint8_t type;
@@ -478,14 +480,29 @@ static struct p9_fid *find_fid(device *device, unsigned int fid)
     return NULL;
 }
 
+static struct p9_fid *get_fid_ref(device *device, unsigned int fid)
+{
+    struct p9_fid *fidp;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    fidp = find_fid(device, fid);
+    if ( fidp )
+        fidp->ref++;
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    return fidp;
+}
+
 static struct p9_fid *alloc_fid_mem(device *device, unsigned int fid,
                                     const char *path)
 {
     struct p9_fid *fidp;
     size_t pathlen;
 
-    pathlen = strlen(path);
-    fidp = calloc(sizeof(*fidp) + pathlen + 1, 1);
+    pathlen = strlen(path) + 1;
+    fidp = calloc(sizeof(*fidp) + pathlen, 1);
     if ( !fidp )
         return NULL;
 
@@ -504,7 +521,7 @@ static struct p9_fid *alloc_fid(device *device, unsigned int fid,
 
     if ( find_fid(device, fid) )
     {
-        errno = EBADFD;
+        errno = EBADF;
         goto out;
     }
 
@@ -578,6 +595,10 @@ static int fill_qid(device *device, const char *path, struct p9_qid *qid,
         stbuf = &st;
     }
 
+    /* Don't allow symbolic links. */
+    if ( S_ISLNK(stbuf->st_mode) )
+        return EMLINK;
+
     qid->type = S_ISDIR(stbuf->st_mode) ? QID_TYPE_DIR : 0;
     qid->version = stbuf->st_mtime ^ (stbuf->st_size << 8);
     qid->path = stbuf->st_ino;
@@ -585,6 +606,20 @@ static int fill_qid(device *device, const char *path, struct p9_qid *qid,
     return 0;
 }
 
+static bool name_ok(const char *str)
+{
+    if ( !*str )
+        return false;
+
+    if ( strchr(str, '/' ) )
+        return false;
+
+    if ( !strcmp(str, "..") || !strcmp(str, ".") )
+        return false;
+
+    return true;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -662,6 +697,138 @@ static void p9_attach(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "Q", &qid);
 }
 
+static void p9_walk(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint32_t newfid;
+    struct p9_fid *fidp = NULL;
+    struct p9_qid *qids = NULL;
+    unsigned int n_names = 0;
+    unsigned int *names = NULL;
+    unsigned int walked = 0;
+    unsigned int i;
+    char *path = NULL;
+    unsigned int path_len;
+    int ret;
+
+    ret = fill_data(ring, "UUaS", &fid, &newfid, &n_names, &names);
+    if ( n_names > P9_WALK_MAXELEM )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        goto out;
+    }
+    if ( ret != 3 + n_names )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        goto out;
+    }
+    if ( fidp->opened )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        goto out;
+    }
+
+    path_len = strlen(fidp->path) + 1;
+    for ( i = 0; i < n_names; i++ )
+    {
+        if ( !name_ok(ring->str + names[i]) )
+        {
+            p9_error(ring, hdr->tag, ENOENT);
+            goto out;
+        }
+        path_len += strlen(ring->str + names[i]) + 1;
+    }
+    path = calloc(path_len + 1, 1);
+    if ( !path )
+    {
+        p9_error(ring, hdr->tag, ENOMEM);
+        goto out;
+    }
+    strcpy(path, fidp->path);
+
+    if ( n_names )
+    {
+        qids = calloc(n_names, sizeof(*qids));
+        if ( !qids )
+        {
+            p9_error(ring, hdr->tag, ENOMEM);
+            goto out;
+        }
+        for ( i = 0; i < n_names; i++ )
+        {
+            strcat(path, "/");
+            strcat(path, ring->str + names[i]);
+            ret = fill_qid(device, path, qids + i, NULL);
+            if ( ret )
+            {
+                if ( !walked )
+                {
+                    p9_error(ring, hdr->tag, errno);
+                    goto out;
+                }
+                break;
+            }
+            walked++;
+        }
+    }
+
+    if ( walked == n_names )
+    {
+        bool ok = false;
+
+        if ( fid == newfid )
+        {
+            struct p9_fid *new_fidp;
+
+            pthread_mutex_lock(&device->fid_mutex);
+
+            if ( fidp->ref != 2 )
+            {
+                errno = EBUSY;
+            }
+            else
+            {
+                new_fidp = alloc_fid_mem(device, fid, path);
+                if ( new_fidp )
+                {
+                    new_fidp->ref = 2;
+                    XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+                    XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+                    free(fidp);
+                    fidp = new_fidp;
+                    ok = true;
+                }
+            }
+
+            pthread_mutex_unlock(&device->fid_mutex);
+        }
+        else
+            ok = alloc_fid(device, newfid, path);
+
+        if ( !ok )
+        {
+            p9_error(ring, hdr->tag, errno);
+            goto out;
+        }
+    }
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "aQ", &walked, qids);
+
+ out:
+    free_fid(device, fidp);
+    free(qids);
+    free(path);
+    free(names);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -725,6 +892,10 @@ void *io_thread(void *arg)
                 p9_attach(ring, &hdr);
                 break;
 
+            case P9_CMD_WALK:
+                p9_walk(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index 6c755411a7..f01fffb0bb 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -25,6 +25,7 @@ struct p9_fid {
     XEN_TAILQ_ENTRY(struct p9_fid) list;
     unsigned int fid;
     unsigned int ref;
+    bool opened;
     char path[];
 };
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:56:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:56:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681264.1059979 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfc-0006po-Ou; Thu, 15 Feb 2024 06:56:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681264.1059979; Thu, 15 Feb 2024 06:56:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfc-0006pg-Lf; Thu, 15 Feb 2024 06:56:32 +0000
Received: by outflank-mailman (input) for mailman id 681264;
 Thu, 15 Feb 2024 06:56:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVfb-0003c6-VJ
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:31 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5833e3f2-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:56:29 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 2376021FDA;
 Thu, 15 Feb 2024 06:56:29 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id D488B1346A;
 Thu, 15 Feb 2024 06:56:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id /bMZMpy1zWViSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5833e3f2-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980189; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AKhABKxA9hqZ48SJ4B4hqEBlmi+Z1rvQ2y1Mz+PFJGg=;
	b=g36ZdPB8/9IwNXogQbi7Cv6+BiII7+qh/nq5lySeefhhpS7FjILXWFuy42JAxRyO5AoWyZ
	1e0/SZgs7S978S9/LNWPH3XYVR80N7pjuC41gwORG73YZqroS7SEGBP3jEx+6P2qZq6VJ4
	eUcuDmumdrKG2VpF0vQq/tfWJIc+ubw=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980189; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=AKhABKxA9hqZ48SJ4B4hqEBlmi+Z1rvQ2y1Mz+PFJGg=;
	b=g36ZdPB8/9IwNXogQbi7Cv6+BiII7+qh/nq5lySeefhhpS7FjILXWFuy42JAxRyO5AoWyZ
	1e0/SZgs7S978S9/LNWPH3XYVR80N7pjuC41gwORG73YZqroS7SEGBP3jEx+6P2qZq6VJ4
	eUcuDmumdrKG2VpF0vQq/tfWJIc+ubw=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 08/21] tools/xen-9pfsd: add 9pfs open request support
Date: Thu, 15 Feb 2024 07:55:28 +0100
Message-Id: <20240215065541.21067-9-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=g36ZdPB8
X-Spamd-Result: default: False [1.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[citrix.com:email,suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 1.69
X-Rspamd-Queue-Id: 2376021FDA
X-Spam-Level: *
X-Spam-Flag: NO
X-Spamd-Bar: +

Add the open request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- don't allow to open symbolic link
V3:
- use openat() (Jason Andryuk)
- use common error handling in p9_open()
---
 tools/9pfsd/io.c        | 137 ++++++++++++++++++++++++++++++++++++++++
 tools/9pfsd/xen-9pfsd.h |   4 ++
 2 files changed, 141 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 498f275a27..139e177ad1 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -19,6 +19,8 @@
 #include <syslog.h>
 #include <sys/types.h>
 #include <sys/stat.h>
+#include <dirent.h>
+#include <fcntl.h>
 #include <xenctrl.h>           /* For cpu barriers. */
 #include <xen-tools/common-macros.h>
 
@@ -29,6 +31,15 @@
 #define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
+#define P9_CMD_OPEN       112
+
+/* P9 protocol open flags. */
+#define P9_OREAD            0   /* read */
+#define P9_OWRITE           1   /* write */
+#define P9_ORDWR            2   /* read and write */
+#define P9_OMODEMASK     0x03
+#define P9_OTRUNC        0x10   /* or'ed in, truncate file first */
+#define P9_OREMOVE       0x40   /* or'ed in, remove file after clunk */
 
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
@@ -829,6 +840,128 @@ static void p9_walk(struct ring *ring, struct p9_header *hdr)
     free(names);
 }
 
+static int open_flags_from_mode(uint8_t mode)
+{
+    int flags;
+
+    switch ( mode & P9_OMODEMASK )
+    {
+    case P9_OREAD:
+        flags = O_RDONLY;
+        break;
+
+    case P9_OWRITE:
+        flags = O_WRONLY;
+        break;
+
+    case P9_ORDWR:
+        flags = O_RDWR;
+        break;
+
+    default:
+        errno = EINVAL;
+        return -1;
+    }
+
+    if ( mode & P9_OTRUNC )
+        flags |= O_TRUNC;
+
+    return flags;
+}
+
+static unsigned int get_iounit(struct ring *ring, struct stat *st)
+{
+    return (ring->max_size - st->st_blksize) & ~(st->st_blksize - 1);
+}
+
+static void p9_open(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint8_t mode;
+    struct p9_fid *fidp;
+    struct stat st;
+    struct p9_qid qid;
+    uint32_t iounit;
+    int flags;
+    int ret;
+
+    ret = fill_data(ring, "Ub", &fid, &mode);
+    if ( ret != 2 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+    if ( mode & ~(P9_OMODEMASK | P9_OTRUNC | P9_OREMOVE) )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+    if ( fidp->opened )
+    {
+        errno = EINVAL;
+        goto err;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        errno = ENOENT;
+        goto err;
+    }
+
+    if ( S_ISLNK(st.st_mode) )
+    {
+        errno = EMLINK;
+        goto err;
+    }
+
+    fidp->isdir = S_ISDIR(st.st_mode);
+    fidp->mode = mode;
+    if ( fidp->isdir )
+    {
+        if ( mode != P9_OREAD )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        fidp->fd = openat(device->root_fd, fidp->path, O_RDONLY);
+        if ( fidp->fd < 0 )
+            goto err;
+        fidp->data = fdopendir(fidp->fd);
+        if ( !fidp->data )
+            goto err;
+    }
+    else
+    {
+        flags = open_flags_from_mode(mode);
+        if ( flags < 0 )
+            goto err;
+
+        fidp->fd = openat(device->root_fd, fidp->path, flags);
+        if ( fidp->fd < 0 )
+            goto err;
+    }
+
+    fill_qid(device, fidp->path, &qid, &st);
+    iounit = get_iounit(ring, &st);
+    fidp->opened = true;
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "QU", &qid, &iounit);
+
+    return;
+
+ err:
+    free_fid(device, fidp);
+    p9_error(ring, hdr->tag, errno);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -896,6 +1029,10 @@ void *io_thread(void *arg)
                 p9_walk(ring, &hdr);
                 break;
 
+            case P9_CMD_OPEN:
+                p9_open(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index f01fffb0bb..757be2da4b 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -25,7 +25,11 @@ struct p9_fid {
     XEN_TAILQ_ENTRY(struct p9_fid) list;
     unsigned int fid;
     unsigned int ref;
+    int fd;
+    uint8_t mode;
     bool opened;
+    bool isdir;
+    void *data;    /* File type specific. */
     char path[];
 };
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:56:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:56:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681269.1059989 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfk-0007MI-1Q; Thu, 15 Feb 2024 06:56:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681269.1059989; Thu, 15 Feb 2024 06:56:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfj-0007M3-UA; Thu, 15 Feb 2024 06:56:39 +0000
Received: by outflank-mailman (input) for mailman id 681269;
 Thu, 15 Feb 2024 06:56:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVfi-0003c6-LH
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:38 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5cb21b87-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:56:37 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id B52221F850;
 Thu, 15 Feb 2024 06:56:34 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 837DF1346A;
 Thu, 15 Feb 2024 06:56:34 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id nHmmHqK1zWVmSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5cb21b87-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980196; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=DaaDElEF/OIaHBHxGvY2JgXcGRlyuI6TRhgNYcijioI=;
	b=q2G/y8B97JaGOfd2pd1lKNQA+G+bT5Ahdft6BsjgV/qXHKnB+jtuR0hbE5OIlvxmQ+ZppT
	tHRUIO3d6F/0QUsdQbfdL5/Z5oaut9BqlqeC12RUCIrD47CElQ8FRtdYYIunRBAQQuVJmB
	yrYnBtBcKMnJJgvJE1IrCDXv53mjnjY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980194; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=DaaDElEF/OIaHBHxGvY2JgXcGRlyuI6TRhgNYcijioI=;
	b=s+RLDcDZksVGJmbVK/2KOHr6vGzwCOXuCQdvAJdQcD0/VMpNs0sfR76AdyhamL65qzZdBn
	YqaPyUDnUe4dqPWUTnQSVUNWOgjjNsH1WAPKTWEPV28an0uOaWMvrnrYMtd+B3c3l7jSCh
	t3+UGMjKSJe6NFACM8mYeNfiLjvghCA=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 09/21] tools/xen-9pfsd: add 9pfs clunk request support
Date: Thu, 15 Feb 2024 07:55:29 +0100
Message-Id: <20240215065541.21067-10-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: ***
X-Spam-Score: 3.70
X-Spamd-Result: default: False [3.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-0.999];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[citrix.com:email,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[14.33%]
X-Spam-Flag: NO

Add the clunk request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V3:
- use unlinkat() (Jason Andryuk)
---
 tools/9pfsd/io.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 139e177ad1..b98ea23b2e 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -32,6 +32,7 @@
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
+#define P9_CMD_CLUNK      120
 
 /* P9 protocol open flags. */
 #define P9_OREAD            0   /* read */
@@ -962,6 +963,44 @@ static void p9_open(struct ring *ring, struct p9_header *hdr)
     p9_error(ring, hdr->tag, errno);
 }
 
+static void p9_clunk(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "U", &fid);
+    if ( ret != 1 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( fidp->opened )
+    {
+        fidp->opened = false;
+        free_fid(device, fidp);
+        close(fidp->fd);
+        if ( fidp->mode & P9_OREMOVE )
+            unlinkat(device->root_fd, fidp->path,
+                     fidp->isdir ? AT_REMOVEDIR : 0);
+    }
+
+    /* 2 calls of free_fid(): one for our reference, and one to free it. */
+    free_fid(device, fidp);
+    free_fid(device, fidp);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "");
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1033,6 +1072,10 @@ void *io_thread(void *arg)
                 p9_open(ring, &hdr);
                 break;
 
+            case P9_CMD_CLUNK:
+                p9_clunk(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:56:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:56:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681270.1059999 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfo-0007tS-B8; Thu, 15 Feb 2024 06:56:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681270.1059999; Thu, 15 Feb 2024 06:56:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVfo-0007tL-7k; Thu, 15 Feb 2024 06:56:44 +0000
Received: by outflank-mailman (input) for mailman id 681270;
 Thu, 15 Feb 2024 06:56:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVfm-0003c6-Ex
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:42 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5ee4c382-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:56:40 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 5988222019;
 Thu, 15 Feb 2024 06:56:40 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 276E61346A;
 Thu, 15 Feb 2024 06:56:40 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id pWUiCKi1zWV0SQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5ee4c382-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980200; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Myjqrdq55cb4MHOivuO+imYqV/xiNZ2vBEYvRnzwiJg=;
	b=I+Sq2WoOmqIAPS7aINbiyJxk5xBFwIiM6NfXVhRq0/vCj+5l0IiJ1n7klJ94DxO+6gojZG
	i9w5aAQyzhW9B/dzDfMizrtBHNhjEIUodeVflLkyxawMNhZB5djdJLeVuIpmuTK4RToned
	Xbf6JAl4iXP5RbWDa5rJozPlG1pvLfs=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980200; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Myjqrdq55cb4MHOivuO+imYqV/xiNZ2vBEYvRnzwiJg=;
	b=I+Sq2WoOmqIAPS7aINbiyJxk5xBFwIiM6NfXVhRq0/vCj+5l0IiJ1n7klJ94DxO+6gojZG
	i9w5aAQyzhW9B/dzDfMizrtBHNhjEIUodeVflLkyxawMNhZB5djdJLeVuIpmuTK4RToned
	Xbf6JAl4iXP5RbWDa5rJozPlG1pvLfs=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 10/21] tools/xen-9pfsd: add 9pfs create request support
Date: Thu, 15 Feb 2024 07:55:30 +0100
Message-Id: <20240215065541.21067-11-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=I+Sq2WoO
X-Spamd-Result: default: False [4.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email,citrix.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 4.69
X-Rspamd-Queue-Id: 5988222019
X-Spam-Level: ****
X-Spam-Flag: NO
X-Spamd-Bar: ++++

Add the create request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- set permissions correctly (Jason Andryuk)
V3:
- use opendirat() etc. (Jason Andryuk)
- rework error handling a little bit
---
 tools/9pfsd/io.c | 151 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 151 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index b98ea23b2e..65ff4dab73 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -32,6 +32,7 @@
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
+#define P9_CMD_CREATE     114
 #define P9_CMD_CLUNK      120
 
 /* P9 protocol open flags. */
@@ -42,6 +43,12 @@
 #define P9_OTRUNC        0x10   /* or'ed in, truncate file first */
 #define P9_OREMOVE       0x40   /* or'ed in, remove file after clunk */
 
+/* P9 protocol create permission masks. */
+#define P9_CREATE_PERM_DIR        0x80000000
+#define P9_CREATE_PERM_NOTSUPP    0x03b00000   /* link, symlink, ... */
+#define P9_CREATE_PERM_DIR_MASK   0777
+#define P9_CREATE_PERM_FILE_MASK  0666
+
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
 #define P9_WALK_MAXELEM   16
@@ -963,6 +970,146 @@ static void p9_open(struct ring *ring, struct p9_header *hdr)
     p9_error(ring, hdr->tag, errno);
 }
 
+static void p9_create(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    unsigned int name_off;
+    uint32_t perm;
+    uint8_t mode;
+    unsigned int ext_off;
+    struct p9_fid *fidp;
+    struct p9_fid *new_fidp;
+    char *path;
+    struct stat st;
+    struct p9_qid qid;
+    uint32_t iounit;
+    int flags;
+    int ret;
+
+    ret = fill_data(ring, "USUbS", &fid, &name_off, &perm, &mode, &ext_off);
+    if ( ret != 5 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    if ( !name_ok(ring->str + name_off) )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( perm & P9_CREATE_PERM_NOTSUPP )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || fidp->opened )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    path = malloc(strlen(fidp->path) + strlen(ring->str + name_off) + 2);
+    if ( !path )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, ENOMEM);
+        return;
+    }
+    sprintf(path, "%s/%s", fidp->path, ring->str + name_off);
+    new_fidp = alloc_fid_mem(device, fid, path);
+    free(path);
+    if ( !new_fidp )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, ENOMEM);
+        return;
+    }
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    new_fidp->ref = fidp->ref;
+
+    if ( perm & P9_CREATE_PERM_DIR )
+    {
+        perm &= P9_CREATE_PERM_DIR_MASK & st.st_mode;
+        if ( mode != P9_OREAD )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        if ( mkdirat(device->root_fd, new_fidp->path, perm) < 0 )
+            goto err;
+
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+        free(fidp);
+        fidp = new_fidp;
+        new_fidp = NULL;
+
+        fidp->fd = openat(device->root_fd, fidp->path, O_RDONLY);
+        if ( fidp->fd < 0 )
+            goto err;
+        fidp->data = fdopendir(fidp->fd);
+        if ( !fidp->data )
+            goto err;
+    }
+    else
+    {
+        flags = open_flags_from_mode(mode);
+        if ( flags < 0 )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        perm &= P9_CREATE_PERM_FILE_MASK & st.st_mode;
+
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+        free(fidp);
+        fidp = new_fidp;
+        new_fidp = NULL;
+
+        fidp->fd = openat(device->root_fd, fidp->path, flags | O_CREAT | O_EXCL,
+                          perm);
+        if ( fidp->fd < 0 )
+            goto err;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+        goto err;
+
+    fill_qid(device, fidp->path, &qid, &st);
+    iounit = get_iounit(ring, &st);
+    fidp->opened = true;
+    fidp->mode = mode;
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "QU", &qid, &iounit);
+
+    return;
+
+ err:
+    p9_error(ring, hdr->tag, errno);
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    free(new_fidp);
+    free_fid(device, fidp);
+}
+
 static void p9_clunk(struct ring *ring, struct p9_header *hdr)
 {
     device *device = ring->device;
@@ -1072,6 +1219,10 @@ void *io_thread(void *arg)
                 p9_open(ring, &hdr);
                 break;
 
+            case P9_CMD_CREATE:
+                p9_create(ring, &hdr);
+                break;
+
             case P9_CMD_CLUNK:
                 p9_clunk(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:57:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:57:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681275.1060009 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVgi-0000gM-RQ; Thu, 15 Feb 2024 06:57:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681275.1060009; Thu, 15 Feb 2024 06:57:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVgi-0000gF-NN; Thu, 15 Feb 2024 06:57:40 +0000
Received: by outflank-mailman (input) for mailman id 681275;
 Thu, 15 Feb 2024 06:57:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVgi-0000g1-B8
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:57:40 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8130f278-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:57:38 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id A39CD1F854;
 Thu, 15 Feb 2024 06:57:36 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 5E36D1346A;
 Thu, 15 Feb 2024 06:57:36 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id evc3FeC1zWWgSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:57:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8130f278-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980257; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mx+xsXHmP+kMTvkOl3Dx/uX2x1DBpbeqqzrtIUR5SLs=;
	b=fObL+CA6ySmC2T/FqauWKvLa59dP9zpP6NTJF0RCg5W42s09TwWq5UAwOVnBvH/OBEgYmO
	a8M6DX5dwD8cY6e6ZdmKDcHyuVKukfMY02gZMGCA+OOl3rLm9eRkK3e9FsINt2Zd9M4Aci
	tp/y7IskdBFuLnrP6ZBqn0TBL8A1A3o=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980256; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mx+xsXHmP+kMTvkOl3Dx/uX2x1DBpbeqqzrtIUR5SLs=;
	b=O3AZhhaNdnX7sEu5uFCLddgGFleA/F1YZ7FtX646dZmiDwtf7epAxsdL0LPWw4s1Q7wzI0
	nKNMqpctmD8NqwiNkm/cQAdlEmtu+LsqUM8CHNmy9UlO/NEd1+OKw52lOeFgC/WJq6mDuV
	WkOnQkUaD6F1utJJM1775iOglj0kZuM=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Julien Grall <julien@xen.org>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v6 20/21] tools/xenstored: support complete log capabilities in stubdom
Date: Thu, 15 Feb 2024 07:55:40 +0100
Message-Id: <20240215065541.21067-21-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=O3AZhhaN
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [0.49 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 0.49
X-Rspamd-Queue-Id: A39CD1F854
X-Spam-Flag: NO

With 9pfs being fully available in Xenstore-stubdom now, there is no
reason to not fully support all logging capabilities in stubdom.

Open the logfile on stubdom only after the 9pfs file system has been
mounted.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- remove now stale comment in sysconfig.xencommons.in (Julien Grall)
---
 .../Linux/init.d/sysconfig.xencommons.in      |  1 -
 tools/hotplug/Linux/launch-xenstore.in        |  1 +
 tools/xenstored/control.c                     | 30 +++++++++----------
 tools/xenstored/minios.c                      |  3 ++
 4 files changed, 19 insertions(+), 16 deletions(-)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index 433e4849af..1bdd830d8a 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -58,7 +58,6 @@ XENSTORED_ARGS=
 ## Default: Not defined, tracing off
 #
 # Log xenstored messages
-# Only evaluated if XENSTORETYPE is "daemon".
 #XENSTORED_TRACE=[yes|on|1]
 
 ## Type: integer
diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in
index e854ca1eb8..da4eeca7c5 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -98,6 +98,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 	[ -z "$XENSTORE_DOMAIN_SIZE" ] && XENSTORE_DOMAIN_SIZE=8
 	XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory $XENSTORE_DOMAIN_SIZE"
 	[ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem $XENSTORE_MAX_DOMAIN_SIZE"
+	[ -z "$XENSTORED_TRACE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS -T xenstored-trace.log"
 
 	echo -n Starting $XENSTORE_DOMAIN_KERNEL...
 	${LIBEXEC_BIN}/init-xenstore-domain $XENSTORE_DOMAIN_ARGS || exit 1
diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index b2f64d674f..dae23a5ac0 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -201,19 +201,6 @@ static int do_control_quota_s(const void *ctx, struct connection *conn,
 	return EINVAL;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_logfile(const void *ctx, struct connection *conn,
 			      const char **vec, int num)
 {
@@ -222,13 +209,26 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 
 	close_log();
 	talloc_free(tracefile);
-	tracefile = talloc_strdup(NULL, vec[0]);
+	tracefile = absolute_filename(NULL, vec[0]);
 	reopen_log();
 
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
 
+#ifdef __MINIOS__
+static int do_control_memreport(const void *ctx, struct connection *conn,
+				const char **vec, int num)
+{
+	if (num)
+		return EINVAL;
+
+	talloc_report_full(NULL, stdout);
+
+	send_ack(conn, XS_CONTROL);
+	return 0;
+}
+#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
@@ -309,10 +309,10 @@ static struct cmd_s cmds[] = {
 		"[-c <cmdline>] [-F] [-t <timeout>] <file>\n"
 		"    Default timeout is 60 seconds.", 5 },
 #endif
+	{ "logfile", do_control_logfile, "<file>" },
 #ifdef __MINIOS__
 	{ "memreport", do_control_memreport, "" },
 #else
-	{ "logfile", do_control_logfile, "<file>" },
 	{ "memreport", do_control_memreport, "[<file>]" },
 #endif
 	{ "print", do_control_print, "<string>" },
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index e70386f8c7..a229954cf4 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -122,6 +122,9 @@ static void mount_thread(void *p)
 	}
 
 	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+
+	/* Start logging if selected. */
+	reopen_log();
 }
 
 void mount_9pfs(void)
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:59:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:59:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681282.1060024 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVi7-0001XU-Cg; Thu, 15 Feb 2024 06:59:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681282.1060024; Thu, 15 Feb 2024 06:59:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVi7-0001Wn-7r; Thu, 15 Feb 2024 06:59:07 +0000
Received: by outflank-mailman (input) for mailman id 681282;
 Thu, 15 Feb 2024 06:59:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVfw-0003NA-LY
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:52 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 65a9918a-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:56:52 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 96E261F850;
 Thu, 15 Feb 2024 06:56:51 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 6433B1346A;
 Thu, 15 Feb 2024 06:56:51 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 9jf0FrO1zWV/SQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 65a9918a-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980211; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=H7h59ZNJ4K9MODJOtjiUCqzaEY8VYjleLWRYedUkilk=;
	b=S8VPO92Ez26e2+oDhdKDN5boHGtPEopilM9SSRoFUS+zUiSi/cthv9UVw4CmGw7dKd0wdT
	YvmGqQM6NttCQQn495jXUBJNLFVxK71iTCvp3UTg5VM8TbPIdxaSKihFpV0tgzSfs4k80q
	46o5sQjRhsSXZ4PCQGDRLKLloq+HWT0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980211; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=H7h59ZNJ4K9MODJOtjiUCqzaEY8VYjleLWRYedUkilk=;
	b=S8VPO92Ez26e2+oDhdKDN5boHGtPEopilM9SSRoFUS+zUiSi/cthv9UVw4CmGw7dKd0wdT
	YvmGqQM6NttCQQn495jXUBJNLFVxK71iTCvp3UTg5VM8TbPIdxaSKihFpV0tgzSfs4k80q
	46o5sQjRhsSXZ4PCQGDRLKLloq+HWT0=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 12/21] tools/xen-9pfsd: add 9pfs write request support
Date: Thu, 15 Feb 2024 07:55:32 +0100
Message-Id: <20240215065541.21067-13-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=S8VPO92E
X-Spamd-Result: default: False [4.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email,citrix.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[25.67%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 4.69
X-Rspamd-Queue-Id: 96E261F850
X-Spam-Level: ****
X-Spam-Flag: NO
X-Spamd-Bar: ++++

Add the write request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/9pfsd/io.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index d08c4b1283..358b7f0781 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -33,6 +33,7 @@
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
+#define P9_CMD_WRITE      118
 #define P9_CMD_CLUNK      120
 #define P9_CMD_STAT       124
 
@@ -1246,6 +1247,55 @@ static void p9_stat(struct ring *ring, struct p9_header *hdr)
     free_fid(device, fidp);
 }
 
+static void p9_write(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint64_t off;
+    unsigned int len;
+    uint32_t written;
+    void *buf;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "ULD", &fid, &off, &len, ring->buffer);
+    if ( ret != 3 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || !fidp->opened || fidp->isdir )
+    {
+        p9_error(ring, hdr->tag, EBADF);
+        goto out;
+    }
+
+    buf = ring->buffer;
+
+    while ( len != 0 )
+    {
+        ret = pwrite(fidp->fd, buf, len, off);
+        if ( ret < 0 )
+            break;
+        len -= ret;
+        buf += ret;
+        off += ret;
+    }
+
+    written = buf - ring->buffer;
+    if ( written == 0 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "U", &written);
+
+ out:
+    free_fid(device, fidp);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1321,6 +1371,10 @@ void *io_thread(void *arg)
                 p9_create(ring, &hdr);
                 break;
 
+            case P9_CMD_WRITE:
+                p9_write(ring, &hdr);
+                break;
+
             case P9_CMD_CLUNK:
                 p9_clunk(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:59:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:59:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681281.1060018 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVi7-0001V3-3s; Thu, 15 Feb 2024 06:59:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681281.1060018; Thu, 15 Feb 2024 06:59:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVi7-0001Uw-1M; Thu, 15 Feb 2024 06:59:07 +0000
Received: by outflank-mailman (input) for mailman id 681281;
 Thu, 15 Feb 2024 06:59:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVgD-0003NA-Hr
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:57:09 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6faa9a00-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:57:08 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 7EB0421DA6;
 Thu, 15 Feb 2024 06:57:08 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 4DC381346A;
 Thu, 15 Feb 2024 06:57:08 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id bAWLEcS1zWWKSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:57:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6faa9a00-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980228; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=RvgJykXrxBqZTlWS0F7IlE2o6lxF6w2XfzGIljjECLU=;
	b=VEYyLXGZts5i286cp136sufUGMTjF85yd5/sEv9MCLKNryZE8UrVW4I9koQ/nlQIzYvqW8
	Bs+PPerXyimJjA/QxpK1Xl0SvqGVPIIBH92Vo5UKFPS+mO/Kw8Eypz3dSJd/bfbZzVWsd3
	tPmfexS31tbN4tGmih3GLj/WJH/7SOY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980228; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=RvgJykXrxBqZTlWS0F7IlE2o6lxF6w2XfzGIljjECLU=;
	b=VEYyLXGZts5i286cp136sufUGMTjF85yd5/sEv9MCLKNryZE8UrVW4I9koQ/nlQIzYvqW8
	Bs+PPerXyimJjA/QxpK1Xl0SvqGVPIIBH92Vo5UKFPS+mO/Kw8Eypz3dSJd/bfbZzVWsd3
	tPmfexS31tbN4tGmih3GLj/WJH/7SOY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 15/21] tools/xl: support new 9pfs backend xen_9pfsd
Date: Thu, 15 Feb 2024 07:55:35 +0100
Message-Id: <20240215065541.21067-16-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=VEYyLXGZ
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [0.49 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 0.49
X-Rspamd-Queue-Id: 7EB0421DA6
X-Spam-Flag: NO

Add support for the new 9pfs backend "xen_9pfsd". For this backend type
the tag defaults to "Xen" and the host side path to
"/var/log/xen/guests/<dom-name>".

Do most of the default settings in libxl. Unfortunately the default
path can't easily be set in libxl, as the domain name isn't available
in the related 9pfs specific function.

Settings the defaults in libxl requires to move the sanity checking
of 9pfs parameters from xl to libxl, too.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- test max_files and max_open_files, too (Jason Andryuk)
V4:
- fix man page to use the "xen_9pfsd" type due to idl limitation
  (Jason Andryuk)
- set (most of) the defaults in libxl (Anthony Perard)
---
 docs/man/xl.cfg.5.pod.in      | 36 +++++++++++++++++++++++++++++++++--
 tools/libs/light/libxl_9pfs.c | 18 ++++++++++++++++++
 tools/xl/xl_parse.c           | 23 +++++++++++++++++++---
 3 files changed, 72 insertions(+), 5 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index ea8d41727d..039e057318 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -772,10 +772,16 @@ settings, from the following list:
 
 =over 4
 
+=item B<type=TYPE>
+
+The backendtype for the PV device. Supported values are B<qemu> and
+B<xen_9pfsd>.  The default is B<qemu>.
+
 =item B<tag=STRING>
 
 9pfs tag to identify the filesystem share. The tag is needed on the
-guest side to mount it.
+guest side to mount it. For the backendtype of B<xen_9pfsd> the tag defaults to
+"Xen".
 
 =item B<security_model="none">
 
@@ -785,12 +791,38 @@ squash or remap).
 
 =item B<path=STRING>
 
-Filesystem path on the backend to export.
+Filesystem path on the backend to export. For the backendtype of B<xen_9pfsd>
+the path defaults to "@XEN_LOG_DIR@/guests/<guest-name>".
 
 =item B<backend=domain-id>
 
 Specify the backend domain name or id, defaults to dom0.
 
+=item B<max-files=NUMBER>
+
+Specify the maximum number of files below B<path>. A value of 0 (which
+is the default) doesn't limit the number of files. Only valid for
+B<type=xen_9pfsd>.
+
+=item B<max-open-files=NUMBER>
+
+Specify the maximum number of concurrently opened files below B<path>.
+Multiple opens of the same file are counted individually. Only valid for
+B<type=xen_9pfsd>, which has a default of B<max-open-files=5>.
+
+=item B<max-space=NUMBER>
+
+Specify the maximum used disk space in MiB below B<path>. A value of 0 (which
+is the default) doesn't limit the usable disk space. Only valid for
+B<type=xen_9pfsd>.
+
+=item B<auto-delete=BOOLEAN>
+
+When set the backend will delete the oldest file which is currently not
+opened by the guest in case the disk space limit set via B<max-space> or the
+file limit set via B<max-files> is being reached. Only valid for
+B<type=xen_9pfsd>.
+
 =back
 
 =item B<pvcalls=[ "backend=domain-id", ... ]>
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 900c0d46a0..ddeb4f20a7 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -20,6 +20,24 @@
 static int libxl__device_p9_setdefault(libxl__gc *gc, uint32_t domid,
                                        libxl_device_p9 *p9, bool hotplug)
 {
+    if (p9->type == LIBXL_P9_TYPE_UNKNOWN) {
+        p9->type = LIBXL_P9_TYPE_QEMU;
+    }
+    if (p9->type == LIBXL_P9_TYPE_QEMU &&
+        (p9->max_files || p9->max_open_files || p9->max_space ||
+         p9->auto_delete)) {
+        LOGD(ERROR, domid, "Illegal 9pfs parameter combination");
+        return ERROR_INVAL;
+    }
+    if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD && !p9->tag) {
+        p9->tag = libxl__strdup(NOGC, "Xen");
+    }
+
+    if (!p9->path || !p9->security_model || !p9->tag) {
+        LOGD(ERROR, domid, "9pfs spec missing required field!");
+        return ERROR_INVAL;
+    }
+
     return libxl__resolve_domid(gc, p9->backend_domname, &p9->backend_domid);
 }
 
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 9b358f11b8..80ffe85f5e 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2233,6 +2233,20 @@ void parse_config_data(const char *config_source,
                     replace_string(&p9->tag, value);
                 } else if (!strcmp(key, "backend")) {
                     replace_string(&p9->backend_domname, value);
+                } else if (!strcmp(key, "type")) {
+                    if (libxl_p9_type_from_string(value, &p9->type)) {
+                        fprintf(stderr, "failed to parse 9pfs type: %s\n",
+                                value);
+                        exit(1);
+                    }
+                } else if (!strcmp(key, "max-files")) {
+                    p9->max_files = parse_ulong(value);
+                } else if (!strcmp(key, "max-open-files")) {
+                    p9->max_open_files = parse_ulong(value);
+                } else if (!strcmp(key, "max-space")) {
+                    p9->max_space = parse_ulong(value);
+                } else if (!strcmp(key, "auto-delete")) {
+                    p9->auto_delete = strtoul(value, NULL, 0);
                 } else {
                     fprintf(stderr, "Unknown 9pfs parameter '%s'\n", key);
                     exit(1);
@@ -2243,9 +2257,12 @@ void parse_config_data(const char *config_source,
 
             libxl_string_list_dispose(&pairs);
 
-            if (!p9->path || !p9->security_model || !p9->tag) {
-                fprintf(stderr, "9pfs spec missing required field!\n");
-                exit(1);
+            if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD && !p9->path) {
+                char *path;
+
+                xasprintf(&path, XEN_LOG_DIR "/guests/%s", c_info->name);
+                replace_string(&p9->path, path);
+                free(path);
             }
         }
     }
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:59:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:59:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681288.1060040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raViN-0002aa-KK; Thu, 15 Feb 2024 06:59:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681288.1060040; Thu, 15 Feb 2024 06:59:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raViN-0002aT-F8; Thu, 15 Feb 2024 06:59:23 +0000
Received: by outflank-mailman (input) for mailman id 681288;
 Thu, 15 Feb 2024 06:59:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVg3-0003c6-6u
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:59 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 68ed70d7-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:56:57 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 3B44422019;
 Thu, 15 Feb 2024 06:56:57 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 09A311346A;
 Thu, 15 Feb 2024 06:56:57 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id JBfSALm1zWWCSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68ed70d7-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980217; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=C35q6n+20s9i1tUBUpKfNxMi8oO6a19E8Aw3Ulu+bss=;
	b=JHQcBFvCkr3he2R2YQiyIOhnEDRNK232Gk9HoA8OOhyg9kiiGHry+NSO1ax/hr/BKvZdUO
	H0vwNRJQrfbRcAkUD0msGwRCENnd+lW/0/ss6W69taEXaO0a0c6cgdRKNZ/L9p6j33euXk
	5Mi3uNao0qZkkC9Z6lmi6lkGVSUzOpk=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980217; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=C35q6n+20s9i1tUBUpKfNxMi8oO6a19E8Aw3Ulu+bss=;
	b=JHQcBFvCkr3he2R2YQiyIOhnEDRNK232Gk9HoA8OOhyg9kiiGHry+NSO1ax/hr/BKvZdUO
	H0vwNRJQrfbRcAkUD0msGwRCENnd+lW/0/ss6W69taEXaO0a0c6cgdRKNZ/L9p6j33euXk
	5Mi3uNao0qZkkC9Z6lmi6lkGVSUzOpk=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 13/21] tools/xen-9pfsd: add 9pfs read request support
Date: Thu, 15 Feb 2024 07:55:33 +0100
Message-Id: <20240215065541.21067-14-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: ***
X-Spam-Score: 3.70
X-Spamd-Result: default: False [3.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-0.999];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email,citrix.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[10.98%]
X-Spam-Flag: NO

Add the read request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- make error check more readable (Jason Andryuk)
V4:
- add directory read support
V5:
- rewinddir() if reading a directory and offset is 0 (Jason Andryuk)
---
 tools/9pfsd/io.c | 93 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 93 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 358b7f0781..28b43eb992 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -33,6 +33,7 @@
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
+#define P9_CMD_READ       116
 #define P9_CMD_WRITE      118
 #define P9_CMD_CLUNK      120
 #define P9_CMD_STAT       124
@@ -1247,6 +1248,94 @@ static void p9_stat(struct ring *ring, struct p9_header *hdr)
     free_fid(device, fidp);
 }
 
+static void p9_read(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint64_t off;
+    unsigned int len;
+    uint32_t count;
+    void *buf;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "ULU", &fid, &off, &count);
+    if ( ret != 3 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || !fidp->opened )
+    {
+        errno = EBADF;
+        goto err;
+    }
+
+    len = count;
+    buf = ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
+
+    if ( fidp->isdir )
+    {
+        struct dirent *dirent;
+        struct stat st;
+        struct p9_stat p9s;
+
+        if ( off == 0 )
+            rewinddir(fidp->data);
+
+        while ( len != 0 )
+        {
+            errno = 0;
+            dirent = readdir(fidp->data);
+            if ( !dirent )
+            {
+                if ( errno )
+                    goto err;
+                break;
+            }
+            if ( fstatat(fidp->fd, dirent->d_name, &st, 0) < 0 )
+                goto err;
+            fill_p9_stat(device, &p9s, &st, dirent->d_name);
+            if ( p9s.size + sizeof(p9s.size) > len )
+            {
+                seekdir(fidp->data, dirent->d_off);
+                break;
+            }
+            fill_buffer_at(&buf, "s", &p9s);
+            len -= p9s.size + sizeof(p9s.size);
+        }
+    }
+    else
+    {
+        while ( len != 0 )
+        {
+            ret = pread(fidp->fd, buf, len, off);
+            if ( ret <= 0 )
+                break;
+            len -= ret;
+            buf += ret;
+            off += ret;
+        }
+        if ( ret < 0 && len == count )
+            goto err;
+    }
+
+    buf = ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
+    len = count - len;
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "D", &len, buf);
+
+ out:
+    free_fid(device, fidp);
+
+    return;
+
+ err:
+    p9_error(ring, hdr->tag, errno);
+    goto out;
+}
+
 static void p9_write(struct ring *ring, struct p9_header *hdr)
 {
     device *device = ring->device;
@@ -1371,6 +1460,10 @@ void *io_thread(void *arg)
                 p9_create(ring, &hdr);
                 break;
 
+            case P9_CMD_READ:
+                p9_read(ring, &hdr);
+                break;
+
             case P9_CMD_WRITE:
                 p9_write(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:59:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:59:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681289.1060045 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raViN-0002dB-UU; Thu, 15 Feb 2024 06:59:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681289.1060045; Thu, 15 Feb 2024 06:59:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raViN-0002cW-MO; Thu, 15 Feb 2024 06:59:23 +0000
Received: by outflank-mailman (input) for mailman id 681289;
 Thu, 15 Feb 2024 06:59:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVgI-0003NA-W9
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:57:14 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 72fab3dd-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:57:14 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 1384021FDA;
 Thu, 15 Feb 2024 06:57:14 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id DF95A1346A;
 Thu, 15 Feb 2024 06:57:13 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id kNglNcm1zWWQSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:57:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 72fab3dd-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980234; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/eJaaQzJFr9iaY84hbFZ6eF6VnkmvJwoY+5FWl4+J1Y=;
	b=axG/GU5r8nvaK53ajF2jKDuDkiFtHqlD1qsrrZrcwk2oky5VSZmVHK+TzMYvhPUNux4i9s
	ledpQ15g4dCFgTB/uc04t6HaBdTx+IAzC8FkQJ/BYXxo3XdzxId5f121GlW6uzLiLbQIY7
	1E6UJwdWbI3ugiXvz8Tpg3BhAC+8Hs8=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980234; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/eJaaQzJFr9iaY84hbFZ6eF6VnkmvJwoY+5FWl4+J1Y=;
	b=axG/GU5r8nvaK53ajF2jKDuDkiFtHqlD1qsrrZrcwk2oky5VSZmVHK+TzMYvhPUNux4i9s
	ledpQ15g4dCFgTB/uc04t6HaBdTx+IAzC8FkQJ/BYXxo3XdzxId5f121GlW6uzLiLbQIY7
	1E6UJwdWbI3ugiXvz8Tpg3BhAC+8Hs8=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 16/21] stubdom: extend xenstore stubdom configs
Date: Thu, 15 Feb 2024 07:55:36 +0100
Message-Id: <20240215065541.21067-17-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b="axG/GU5r"
X-Spamd-Result: default: False [3.42 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 TO_DN_SOME(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,ens-lyon.org,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-1.27)[89.83%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 3.42
X-Rspamd-Queue-Id: 1384021FDA
X-Spam-Level: ***
X-Spam-Flag: NO
X-Spamd-Bar: +++

Extend the config files of the Xenstore stubdoms to include XENBUS
and 9PFRONT items in order to support file based logging.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 stubdom/xenstore-minios.cfg    | 2 +-
 stubdom/xenstorepvh-minios.cfg | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/stubdom/xenstore-minios.cfg b/stubdom/xenstore-minios.cfg
index a41704bb6b..239da519b9 100644
--- a/stubdom/xenstore-minios.cfg
+++ b/stubdom/xenstore-minios.cfg
@@ -3,7 +3,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
diff --git a/stubdom/xenstorepvh-minios.cfg b/stubdom/xenstorepvh-minios.cfg
index 6af51f5753..752b90d7d3 100644
--- a/stubdom/xenstorepvh-minios.cfg
+++ b/stubdom/xenstorepvh-minios.cfg
@@ -4,7 +4,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:59:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:59:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681299.1060058 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVia-0003iZ-AI; Thu, 15 Feb 2024 06:59:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681299.1060058; Thu, 15 Feb 2024 06:59:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVia-0003hP-7E; Thu, 15 Feb 2024 06:59:36 +0000
Received: by outflank-mailman (input) for mailman id 681299;
 Thu, 15 Feb 2024 06:59:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVgU-0003NA-BB
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:57:26 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 79b167d6-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:57:25 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 50A5621DA6;
 Thu, 15 Feb 2024 06:57:25 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 12B551346A;
 Thu, 15 Feb 2024 06:57:25 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 6igQA9W1zWWZSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:57:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 79b167d6-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980245; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GuKS5JMiiFYV782dbcVdzQvPQKcEtbDic1ZOTvla/ok=;
	b=P7GtBwrbZO9zwT84qY4tKKhl6jd1WkQb9v44S2nnWvDc2UsQIy+t92iy87b5Weq9DIEk5z
	TRmSSPVuGJG1J6gD2gyuaEIyWVmYVYC/pSNYdyOJChHjpMn8UsIeAe7xY0GRDaYXXBCZ8s
	pA9phvl1LByMdjNDgOa7RD3aZ5JA3+4=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980245; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GuKS5JMiiFYV782dbcVdzQvPQKcEtbDic1ZOTvla/ok=;
	b=P7GtBwrbZO9zwT84qY4tKKhl6jd1WkQb9v44S2nnWvDc2UsQIy+t92iy87b5Weq9DIEk5z
	TRmSSPVuGJG1J6gD2gyuaEIyWVmYVYC/pSNYdyOJChHjpMn8UsIeAe7xY0GRDaYXXBCZ8s
	pA9phvl1LByMdjNDgOa7RD3aZ5JA3+4=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v6 18/21] tools/xenstored: mount 9pfs device in stubdom
Date: Thu, 15 Feb 2024 07:55:38 +0100
Message-Id: <20240215065541.21067-19-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=P7GtBwrb
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [0.49 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 0.49
X-Rspamd-Queue-Id: 50A5621DA6
X-Spam-Flag: NO

Mount the 9pfs device in stubdom enabling it to use files.

This has to happen in a worker thread in order to allow the main thread
handling the required Xenstore accesses in parallel.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- add logging in case of errors (Julien Grall)
---
 tools/xenstored/core.h   |  6 +++++
 tools/xenstored/domain.c |  2 ++
 tools/xenstored/minios.c | 54 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+)

diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index f6af086f01..fe0ee90581 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -36,6 +36,8 @@
 #include "list.h"
 #include "hashtable.h"
 
+#define XENSTORE_LIB_DIR	XEN_LIB_DIR "/xenstore"
+
 #ifndef O_CLOEXEC
 #define O_CLOEXEC 0
 /* O_CLOEXEC support is needed for Live Update in the daemon case. */
@@ -399,6 +401,10 @@ void handle_special_fds(void);
 int get_socket_fd(void);
 void set_socket_fd(int fd);
 
+#ifdef __MINIOS__
+void mount_9pfs(void);
+#endif
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index 1a7d5e9756..64c8fd0cc3 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -1236,6 +1236,8 @@ void stubdom_init(void)
 		barf_perror("Failed to initialize stubdom");
 
 	xenevtchn_notify(xce_handle, stubdom->port);
+
+	mount_9pfs();
 #endif
 }
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 22ac8defbd..562a9b4972 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -17,10 +17,20 @@
 */
 #include <sys/types.h>
 #include <sys/mman.h>
+#include <syslog.h>
+#include "talloc.h"
 #include "core.h"
 #include "utils.h"
 #include <xen/grant_table.h>
 #include <mini-os/lib.h>
+#include <mini-os/9pfront.h>
+#include <mini-os/sched.h>
+#include <mini-os/xenbus.h>
+#include <mini-os/xmalloc.h>
+
+#define P9_STATE_PATH	"device/9pfs/0/state"
+
+static void *p9_device;
 
 void finish_daemonize(void)
 {
@@ -74,3 +84,47 @@ int get_socket_fd(void)
 void set_socket_fd(int fd)
 {
 }
+
+static void mount_thread(void *p)
+{
+	xenbus_event_queue events = NULL;
+	char *err;
+	char *dummy;
+
+	err = xenbus_watch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs", &events);
+	if (err) {
+		log("error \"%s\" when setting watch on \"%s\"\n", err,
+		    P9_STATE_PATH);
+		free(err);
+		return;
+	}
+
+	for (;;) {
+		xenbus_wait_for_watch(&events);
+
+		/*
+		 * We only care for existence of the state node.
+		 * State changes are handled in init_9pfront().
+		 */
+		err = xenbus_read(XBT_NIL, P9_STATE_PATH, &dummy);
+		if (!err)
+			break;
+		free(err);
+	}
+
+	free(dummy);
+
+	err = xenbus_unwatch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs");
+	if (err) {
+		log("error \"%s\" when unwatching \"%s\", leaking watch\n",
+		    err, P9_STATE_PATH);
+		free(err);
+	}
+
+	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+}
+
+void mount_9pfs(void)
+{
+	create_thread("mount-9pfs", mount_thread, NULL);
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:59:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:59:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681300.1060064 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVia-0003kv-Mr; Thu, 15 Feb 2024 06:59:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681300.1060064; Thu, 15 Feb 2024 06:59:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVia-0003ja-EF; Thu, 15 Feb 2024 06:59:36 +0000
Received: by outflank-mailman (input) for mailman id 681300;
 Thu, 15 Feb 2024 06:59:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVga-0003NA-08
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:57:32 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7d0fac3e-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:57:31 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id ECE0A21FF3;
 Thu, 15 Feb 2024 06:57:30 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id AD63E1346A;
 Thu, 15 Feb 2024 06:57:30 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id lNDEKNq1zWWeSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:57:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7d0fac3e-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980251; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xUq3AE+SsnVI0DjnPSJQwNriZRE6cVmY47vrUqUrQuc=;
	b=jqcVKPMpWufFO0TwxgB+8mTv+zIq/586V1AGNg25zlfX+XSj3Iokrxs/kvHI0L9NRwid1Y
	59MyBja/v9Q+I3qQ6vlhxhNmLTs7elFnV3gLmT8JFrx2VL7WQbMuG+JB2UiCJ7vsTSfx/Y
	nQjn3nOr/vH1tqONF26USXVWBmCeg/8=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980250; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xUq3AE+SsnVI0DjnPSJQwNriZRE6cVmY47vrUqUrQuc=;
	b=ILytG+GZXPWroDz1IUIW5TG1OUc8M2PUtRiK6mbiRnJQY57lhkPOcvnA70QUOpQ0f6UWYA
	vgBj093YJhveGTjL4NvqEj+Pk9WrQVKpQVwNImPHy83gHmdDD0RGxT0fIPEl3a9x+9D6Wu
	DszT9Z88VeWbgbwWUohYyc/tf5pC8S4=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v6 19/21] tools/xenstored: add helpers for filename handling
Date: Thu, 15 Feb 2024 07:55:39 +0100
Message-Id: <20240215065541.21067-20-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=ILytG+GZ
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [0.49 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 0.49
X-Rspamd-Queue-Id: ECE0A21FF3
X-Spam-Flag: NO

Add some helpers for handling filenames which might need different
implementations between stubdom and daemon environments:

- expansion of relative filenames (those are not really defined today,
  just expand them to be relative to /var/lib/xen/xenstore)
- expansion of xenstore_daemon_rundir() (used e.g. for saving the state
  file in case of live update - needs to be unchanged in the daemon
  case, but should result in /var/lib/xen/xenstore for stubdom)

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- make absolute_filename() return a pointer to const (Julien Grall)
---
 tools/xenstored/core.c      | 15 +++++++++++++--
 tools/xenstored/core.h      |  5 ++++-
 tools/xenstored/lu_daemon.c |  4 ++--
 tools/xenstored/minios.c    |  5 +++++
 tools/xenstored/posix.c     |  8 +++++++-
 5 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 48fc787ac1..bada1ad9a2 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -63,7 +63,7 @@ char **orig_argv;
 LIST_HEAD(connections);
 int tracefd = -1;
 bool keep_orphans = false;
-char *tracefile = NULL;
+const char *tracefile = NULL;
 static struct hashtable *nodes;
 unsigned int trace_flags = TRACE_OBJ | TRACE_IO;
 
@@ -137,6 +137,17 @@ void trace_destroy(const void *data, const char *type)
 		trace("obj: DESTROY %s %p\n", type, data);
 }
 
+/*
+ * Return an absolute filename.
+ * In case of a relative filename given as input, prepend XENSTORE_LIB_DIR.
+ */
+const char *absolute_filename(const void *ctx, const char *filename)
+{
+	if (filename[0] != '/')
+		return talloc_asprintf(ctx, XENSTORE_LIB_DIR "/%s", filename);
+	return talloc_strdup(ctx, filename);
+}
+
 void close_log(void)
 {
 	if (tracefd >= 0)
@@ -2759,7 +2770,7 @@ int main(int argc, char *argv[])
 #endif
 
 	if (tracefile)
-		tracefile = talloc_strdup(NULL, tracefile);
+		tracefile = absolute_filename(NULL, tracefile);
 
 #ifndef NO_LIVE_UPDATE
 	/* Read state in case of live update. */
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index fe0ee90581..e58779e88c 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -341,7 +341,7 @@ void close_log(void);
 extern int orig_argc;
 extern char **orig_argv;
 
-extern char *tracefile;
+extern const char *tracefile;
 extern int tracefd;
 
 /* Trace flag values must be kept in sync with trace_switches[] contents. */
@@ -405,6 +405,9 @@ void set_socket_fd(int fd);
 void mount_9pfs(void);
 #endif
 
+const char *xenstore_rundir(void);
+const char *absolute_filename(const void *ctx, const char *filename);
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/lu_daemon.c b/tools/xenstored/lu_daemon.c
index 71bcabadd3..6351111ab0 100644
--- a/tools/xenstored/lu_daemon.c
+++ b/tools/xenstored/lu_daemon.c
@@ -24,7 +24,7 @@ void lu_get_dump_state(struct lu_dump_state *state)
 	state->size = 0;
 
 	state->filename = talloc_asprintf(NULL, "%s/state_dump",
-					  xenstore_daemon_rundir());
+					  xenstore_rundir());
 	if (!state->filename)
 		barf("Allocation failure");
 
@@ -65,7 +65,7 @@ FILE *lu_dump_open(const void *ctx)
 	int fd;
 
 	filename = talloc_asprintf(ctx, "%s/state_dump",
-				   xenstore_daemon_rundir());
+				   xenstore_rundir());
 	if (!filename)
 		return NULL;
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 562a9b4972..e70386f8c7 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -128,3 +128,8 @@ void mount_9pfs(void)
 {
 	create_thread("mount-9pfs", mount_thread, NULL);
 }
+
+const char *xenstore_rundir(void)
+{
+	return XENSTORE_LIB_DIR;
+}
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 496329dfd1..d88c82d972 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -326,9 +326,10 @@ void early_init(bool live_update, bool dofork, const char *pidfile)
 {
 	reopen_log();
 
-	/* Make sure xenstored directory exists. */
+	/* Make sure xenstored directories exist. */
 	/* Errors ignored here, will be reported when we open files */
 	mkdir(xenstore_daemon_rundir(), 0755);
+	mkdir(XENSTORE_LIB_DIR, 0755);
 
 	if (dofork) {
 		openlog("xenstored", 0, LOG_DAEMON);
@@ -406,3 +407,8 @@ void set_socket_fd(int fd)
 {
 	sock = fd;
 }
+
+const char *xenstore_rundir(void)
+{
+	return xenstore_daemon_rundir();
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:59:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:59:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681304.1060078 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVie-0004iN-S6; Thu, 15 Feb 2024 06:59:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681304.1060078; Thu, 15 Feb 2024 06:59:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVie-0004hX-Os; Thu, 15 Feb 2024 06:59:40 +0000
Received: by outflank-mailman (input) for mailman id 681304;
 Thu, 15 Feb 2024 06:59:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVgl-0003NA-5X
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:57:43 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 83c200a5-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:57:42 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 435131F854;
 Thu, 15 Feb 2024 06:57:42 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 0E1961346A;
 Thu, 15 Feb 2024 06:57:42 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id z9/jAea1zWWxSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:57:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83c200a5-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980262; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GGQ2M6ZDlC8Q8t5jjlaB/u1rTlT60Df0gemVV+Z+fCo=;
	b=MZuXh1/rRnf54vH7uhs4GK42aaJ1w/jf7bFQUZgbLMegMuJKE8VLTydKayhMWcTsTXjDhh
	qdDJmW/Ol1CEUz1s0i39PZ7bn4WaX4DUIVUD8mg5+NIgwzmm+yE4X14LdvnO6mEHalwiNs
	8HYBPPJmb266QncFMxRP4JOtNtaXx54=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980262; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GGQ2M6ZDlC8Q8t5jjlaB/u1rTlT60Df0gemVV+Z+fCo=;
	b=MZuXh1/rRnf54vH7uhs4GK42aaJ1w/jf7bFQUZgbLMegMuJKE8VLTydKayhMWcTsTXjDhh
	qdDJmW/Ol1CEUz1s0i39PZ7bn4WaX4DUIVUD8mg5+NIgwzmm+yE4X14LdvnO6mEHalwiNs
	8HYBPPJmb266QncFMxRP4JOtNtaXx54=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 21/21] tools/xenstored: have a single do_control_memreport()
Date: Thu, 15 Feb 2024 07:55:41 +0100
Message-Id: <20240215065541.21067-22-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [4.89 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[6];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.01)[50.66%]
X-Spam-Level: ****
X-Spam-Score: 4.89
X-Spam-Flag: NO

With 9pfs now available in Xenstore-stubdom, there is no reason to
have distinct do_control_memreport() variants for the daemon and the
stubdom implementations.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 tools/xenstored/control.c | 27 +++++++--------------------
 1 file changed, 7 insertions(+), 20 deletions(-)

diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index dae23a5ac0..9561289179 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -216,23 +216,11 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 	return 0;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
 	FILE *fp;
+	const char *filename;
 	int fd;
 
 	if (num > 1)
@@ -255,8 +243,12 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 			if (!fp)
 				close(fd);
 		}
-	} else
-		fp = fopen(vec[0], "a");
+	} else {
+		filename = absolute_filename(ctx, vec[0]);
+		if (!filename)
+			return ENOMEM;
+		fp = fopen(filename, "a");
+	}
 
 	if (!fp)
 		return EBADF;
@@ -267,7 +259,6 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
-#endif
 
 static int do_control_print(const void *ctx, struct connection *conn,
 			    const char **vec, int num)
@@ -310,11 +301,7 @@ static struct cmd_s cmds[] = {
 		"    Default timeout is 60 seconds.", 5 },
 #endif
 	{ "logfile", do_control_logfile, "<file>" },
-#ifdef __MINIOS__
-	{ "memreport", do_control_memreport, "" },
-#else
 	{ "memreport", do_control_memreport, "[<file>]" },
-#endif
 	{ "print", do_control_print, "<string>" },
 	{ "quota", do_control_quota,
 		"[set <name> <val>|<domid>|max [-r]]" },
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:59:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:59:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681305.1060085 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVif-0004lG-As; Thu, 15 Feb 2024 06:59:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681305.1060085; Thu, 15 Feb 2024 06:59:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVif-0004kZ-1P; Thu, 15 Feb 2024 06:59:41 +0000
Received: by outflank-mailman (input) for mailman id 681305;
 Thu, 15 Feb 2024 06:59:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVgB-0003c6-7l
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:57:07 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6d830d0c-cbcf-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 07:57:05 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id E2C0A21DA6;
 Thu, 15 Feb 2024 06:57:02 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 9DC1D1346A;
 Thu, 15 Feb 2024 06:57:02 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id AAERJb61zWWFSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:57:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d830d0c-cbcf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980224; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zgmWzK99RZBrySpVIOBOfTuvIg1qZCvB7PuUVcrLtSw=;
	b=EK3z198yve7MbJRT7dEmsy4CiPeaLjJEYm+sHHSh4a7EiicOyuJTno9t2I9RXbuI7c/Ym1
	zHcDB7GV5lTpcA62W1QD65UTZ4rPpjYd5NV/ndG4oZPQdcoQIcKTaVaQ9s/6ZfB4Rc1IP2
	svNPqsT23YLjNZA53kfBLOwzJwqy+Vk=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980222; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zgmWzK99RZBrySpVIOBOfTuvIg1qZCvB7PuUVcrLtSw=;
	b=I+QjrAQy2C+9nBmuBNeKebLOYu2+QHy3Ql4hm7PHovlTELsxg6B3zVOScRj29YnTIrsyvJ
	qlo2aIzOUhrCvOrITc7zM7M7wDbYWGwO9+WDEzkgPvCkDrt0cjuerNRmNmrGCkUNvKGiSt
	DONhnAOuetia27L+Kv+pF5S8OEmdQwM=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	George Dunlap <george.dunlap@cloud.com>
Subject: [PATCH v6 14/21] tools/libs/light: add backend type for 9pfs PV devices
Date: Thu, 15 Feb 2024 07:55:34 +0100
Message-Id: <20240215065541.21067-15-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [1.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email,citrix.com:email,cloud.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,citrix.com,gmail.com,xen.org,cloud.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: *
X-Spam-Score: 1.90
X-Spam-Flag: NO

Make the backend type of 9pfs PV devices configurable. The default is
"qemu" with the related Xenstore backend-side directory being "9pfs".

Add another type "xen_9pfsd" with the related Xenstore backend-side
directory "xen_9pfs".

As additional security features it is possible to specify:
- "max-space" for limiting the maximum space consumed on the filesystem
  in MBs
- "max-files" for limiting the maximum number of files in the
  filesystem
- "max-open-files" for limiting the maximum number of concurrent open
  files

For convenience "auto-delete" is available to let the backend delete the
oldest file of the guest in case otherwise "max-space" or "max-files"
would be violated.

The xen-9pfsd daemon will be started by libxenlight automatically when
the first "xen_9pfs" device is being created.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: George Dunlap <george.dunlap@cloud.com> # Golang bits
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
---
V3:
- regenerate go bindings
V4:
- rename libxl_device_p9_dm_needed() to libxl__device_p9_dm_needed()
  (Anthony Perard)
- reorder span related functions (Anthony Perard)
- add comment for xen9pfsd_spawn() return values (Anthony Perard)
- add LIBXL_HAVE_XEN_9PFS to libxl.h (Anthony Perard)
- use a copy of 'p9' in xen9pfsd_spawn() (Anthony Perard)
V6:
- rebase (Anthony Perard)
- drop callback from struct libxl__aop9_state (Anthony Perard)
---
 tools/golang/xenlight/helpers.gen.go      |  10 ++
 tools/golang/xenlight/types.gen.go        |  12 ++
 tools/include/libxl.h                     |   7 +
 tools/libs/light/libxl_9pfs.c             | 157 +++++++++++++++++++++-
 tools/libs/light/libxl_create.c           |   4 +-
 tools/libs/light/libxl_dm.c               |   2 +-
 tools/libs/light/libxl_types.idl          |  11 ++
 tools/libs/light/libxl_types_internal.idl |   1 +
 8 files changed, 197 insertions(+), 7 deletions(-)

diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 0f8e23773c..8f44397a4e 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -2440,6 +2440,11 @@ x.Tag = C.GoString(xc.tag)
 x.Path = C.GoString(xc.path)
 x.SecurityModel = C.GoString(xc.security_model)
 x.Devid = Devid(xc.devid)
+x.Type = P9Type(xc._type)
+x.MaxSpace = int(xc.max_space)
+x.MaxFiles = int(xc.max_files)
+x.MaxOpenFiles = int(xc.max_open_files)
+x.AutoDelete = bool(xc.auto_delete)
 
  return nil}
 
@@ -2458,6 +2463,11 @@ xc.path = C.CString(x.Path)}
 if x.SecurityModel != "" {
 xc.security_model = C.CString(x.SecurityModel)}
 xc.devid = C.libxl_devid(x.Devid)
+xc._type = C.libxl_p9_type(x.Type)
+xc.max_space = C.int(x.MaxSpace)
+xc.max_files = C.int(x.MaxFiles)
+xc.max_open_files = C.int(x.MaxOpenFiles)
+xc.auto_delete = C.bool(x.AutoDelete)
 
  return nil
  }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 9c8b7b81f6..d31722407a 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -122,6 +122,13 @@ NicTypeVifIoemu NicType = 1
 NicTypeVif NicType = 2
 )
 
+type P9Type int
+const(
+P9TypeUnknown P9Type = 0
+P9TypeQemu P9Type = 1
+P9TypeXen9Pfsd P9Type = 2
+)
+
 type ActionOnShutdown int
 const(
 ActionOnShutdownDestroy ActionOnShutdown = 1
@@ -889,6 +896,11 @@ Tag string
 Path string
 SecurityModel string
 Devid Devid
+Type P9Type
+MaxSpace int
+MaxFiles int
+MaxOpenFiles int
+AutoDelete bool
 }
 
 type DevicePvcallsif struct {
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 46bc774126..9a3e702557 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -615,6 +615,13 @@
  */
 #define LIBXL_HAVE_HVM_PIRQ 1
 
+/*
+ * LIBXL_HAVE_XEN_9PFS indicates the presence of the xen-9pfsd related
+ * fields in libxl_device_p9: type, max_space, max_files, max_open_files and
+ * auto_delete.
+ */
+#define LIBXL_HAVE_XEN_9PFS 1
+
 /*
  * libxl memory management
  *
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 5ab0d3aa21..900c0d46a0 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -33,20 +33,171 @@ static int libxl__set_xenstore_p9(libxl__gc *gc, uint32_t domid,
 
     flexarray_append_pair(front, "tag", p9->tag);
 
+    if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD) {
+        flexarray_append_pair(back, "max-space",
+                              GCSPRINTF("%u", p9->max_space));
+        flexarray_append_pair(back, "max-files",
+                              GCSPRINTF("%u", p9->max_files));
+        flexarray_append_pair(back, "max-open-files",
+                              GCSPRINTF("%u", p9->max_open_files));
+        flexarray_append_pair(back, "auto-delete",
+                              p9->auto_delete ? "1" : "0");
+    }
+
+    return 0;
+}
+
+static int libxl__device_from_p9(libxl__gc *gc, uint32_t domid,
+                                 libxl_device_p9 *type, libxl__device *device)
+{
+    device->backend_devid   = type->devid;
+    device->backend_domid   = type->backend_domid;
+    device->backend_kind    = type->type == LIBXL_P9_TYPE_QEMU
+                              ? LIBXL__DEVICE_KIND_9PFS
+                              : LIBXL__DEVICE_KIND_XEN_9PFS;
+    device->devid           = type->devid;
+    device->domid           = domid;
+    device->kind            = LIBXL__DEVICE_KIND_9PFS;
+
     return 0;
 }
 
-#define libxl__add_p9s NULL
+static int libxl__device_p9_dm_needed(void *e, unsigned domid)
+{
+    libxl_device_p9 *elem = e;
+
+    return elem->type == LIBXL_P9_TYPE_QEMU && elem->backend_domid == domid;
+}
+
+typedef struct libxl__aop9_state libxl__aop9_state;
+
+struct libxl__aop9_state {
+    libxl__spawn_state spawn;
+    libxl__ao_device *aodev;
+    libxl_device_p9 p9;
+    uint32_t domid;
+};
+
+static void xen9pfsd_confirm(libxl__egc *egc, libxl__spawn_state *spawn,
+                             const char *xsdata);
+static void xen9pfsd_failed(libxl__egc *egc, libxl__spawn_state *spawn, int rc);
+static void xen9pfsd_detached(libxl__egc *egc, libxl__spawn_state *spawn);
+static void xen9pfsd_spawn_outcome(libxl__egc *egc, libxl__aop9_state *aop9,
+                                   int rc);
+
+/*
+ * Spawn the xen-9pfsd daemon if needed.
+ * returns:
+ * < 0 if error
+ * 0 if no daemon needs to be spawned
+ * 1 if daemon was spawned
+ */
+static int xen9pfsd_spawn(libxl__egc *egc, uint32_t domid, libxl_device_p9 *p9,
+                         libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    struct libxl__aop9_state *aop9;
+    int rc;
+    char *args[] = { "xen-9pfsd", NULL };
+    char *path = GCSPRINTF("/local/domain/%u/libxl/xen-9pfs",
+                           p9->backend_domid);
+
+    if (p9->type != LIBXL_P9_TYPE_XEN_9PFSD ||
+        libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/state", path)))
+        return 0;
+
+    GCNEW(aop9);
+    aop9->aodev = aodev;
+    libxl_device_p9_copy(CTX, &aop9->p9, p9);
+    aop9->domid = domid;
+
+    aop9->spawn.ao = aodev->ao;
+    aop9->spawn.what = "xen-9pfs daemon";
+    aop9->spawn.xspath = GCSPRINTF("%s/state", path);
+    aop9->spawn.timeout_ms = LIBXL_DEVICE_MODEL_START_TIMEOUT * 1000;
+    aop9->spawn.pidpath = GCSPRINTF("%s/pid", path);
+    aop9->spawn.midproc_cb = libxl__spawn_record_pid;
+    aop9->spawn.confirm_cb = xen9pfsd_confirm;
+    aop9->spawn.failure_cb = xen9pfsd_failed;
+    aop9->spawn.detached_cb = xen9pfsd_detached;
+    rc = libxl__spawn_spawn(egc, &aop9->spawn);
+    if (rc < 0)
+        return rc;
+    if (!rc) {
+        setsid();
+        libxl__exec(gc, -1, -1, -1, LIBEXEC_BIN "/xen-9pfsd", args, NULL);
+    }
+
+    return 1;
+}
+
+static void xen9pfsd_confirm(libxl__egc *egc, libxl__spawn_state *spawn,
+                             const char *xsdata)
+{
+    STATE_AO_GC(spawn->ao);
+
+    if (!xsdata)
+        return;
+
+    if (strcmp(xsdata, "running"))
+        return;
+
+    libxl__spawn_initiate_detach(gc, spawn);
+}
+
+static void xen9pfsd_failed(libxl__egc *egc, libxl__spawn_state *spawn, int rc)
+{
+    libxl__aop9_state *aop9 = CONTAINER_OF(spawn, *aop9, spawn);
+
+    xen9pfsd_spawn_outcome(egc, aop9, rc);
+}
+
+static void xen9pfsd_detached(libxl__egc *egc, libxl__spawn_state *spawn)
+{
+    libxl__aop9_state *aop9 = CONTAINER_OF(spawn, *aop9, spawn);
+
+    xen9pfsd_spawn_outcome(egc, aop9, 0);
+}
+
+static void xen9pfsd_spawn_outcome(libxl__egc *egc, libxl__aop9_state *aop9,
+                                   int rc)
+{
+    aop9->aodev->rc = rc;
+    if (rc)
+        aop9->aodev->callback(egc, aop9->aodev);
+    else
+        libxl__device_add_async(egc, aop9->domid, &libxl__p9_devtype,
+                                &aop9->p9, aop9->aodev);
+}
+
+static void libxl__device_p9_add(libxl__egc *egc, uint32_t domid,
+                                 libxl_device_p9 *p9,
+                                 libxl__ao_device *aodev)
+{
+    int rc;
+
+    rc = xen9pfsd_spawn(egc, domid, p9, aodev);
+    if (rc == 1)
+        return;
+
+    if (rc == 0)
+        libxl__device_add_async(egc, domid, &libxl__p9_devtype, p9, aodev);
+
+    aodev->rc = rc;
+    if (rc)
+        aodev->callback(egc, aodev);
+}
+
 #define libxl_device_p9_list NULL
 #define libxl_device_p9_compare NULL
 
 static LIBXL_DEFINE_UPDATE_DEVID(p9)
-static LIBXL_DEFINE_DEVICE_FROM_TYPE(p9)
+static LIBXL_DEFINE_DEVICES_ADD(p9)
 
 LIBXL_DEFINE_DEVICE_REMOVE(p9)
 
 DEFINE_DEVICE_TYPE_STRUCT(p9, 9PFS, p9s,
-    .skip_attach = 1,
     .set_xenstore_config = (device_set_xenstore_config_fn_t)
                            libxl__set_xenstore_p9,
+    .dm_needed = libxl__device_p9_dm_needed,
 );
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 0008fac607..5546335973 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -1761,9 +1761,6 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
         libxl__device_console_dispose(&console);
     }
 
-    for (i = 0; i < d_config->num_p9s; i++)
-        libxl__device_add(gc, domid, &libxl__p9_devtype, &d_config->p9s[i]);
-
     for (i = 0; i < d_config->num_pvcallsifs; i++)
         libxl__device_add(gc, domid, &libxl__pvcallsif_devtype,
                           &d_config->pvcallsifs[i]);
@@ -1900,6 +1897,7 @@ const libxl__device_type *device_type_tbl[] = {
     &libxl__vdispl_devtype,
     &libxl__vsnd_devtype,
     &libxl__virtio_devtype,
+    &libxl__p9_devtype,
     NULL
 };
 
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 14b593110f..2aaaeb5aa7 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -3761,7 +3761,7 @@ int libxl__need_xenpv_qemu(libxl__gc *gc, libxl_domain_config *d_config)
         goto out;
     }
 
-    if (d_config->num_vfbs > 0 || d_config->num_p9s > 0) {
+    if (d_config->num_vfbs > 0) {
         ret = 1;
         goto out;
     }
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 899ad30969..682092a058 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -150,6 +150,12 @@ libxl_nic_type = Enumeration("nic_type", [
     (2, "VIF"),
     ])
 
+libxl_p9_type = Enumeration("p9_type", [
+    (0, "unknown"),
+    (1, "qemu"),
+    (2, "xen_9pfsd"),
+    ])
+
 libxl_action_on_shutdown = Enumeration("action_on_shutdown", [
     (1, "DESTROY"),
 
@@ -943,6 +949,11 @@ libxl_device_p9 = Struct("device_p9", [
     ("path",             string),
     ("security_model",   string),
     ("devid",            libxl_devid),
+    ("type",             libxl_p9_type),
+    ("max_space",        integer),
+    ("max_files",        integer),
+    ("max_open_files",   integer),
+    ("auto_delete",      bool),
 ])
 
 libxl_device_pvcallsif = Struct("device_pvcallsif", [
diff --git a/tools/libs/light/libxl_types_internal.idl b/tools/libs/light/libxl_types_internal.idl
index 56dccac153..0425e9b6b0 100644
--- a/tools/libs/light/libxl_types_internal.idl
+++ b/tools/libs/light/libxl_types_internal.idl
@@ -35,6 +35,7 @@ libxl__device_kind = Enumeration("device_kind", [
     (17, "VIRTIO_DISK"),
     (18, "VIRTIO"),
     (19, "VBD3"),
+    (20, "XEN_9PFS"),
     ])
 
 libxl__console_backend = Enumeration("console_backend", [
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:59:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:59:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681306.1060097 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVig-0005N7-RX; Thu, 15 Feb 2024 06:59:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681306.1060097; Thu, 15 Feb 2024 06:59:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVig-0005MK-Kp; Thu, 15 Feb 2024 06:59:42 +0000
Received: by outflank-mailman (input) for mailman id 681306;
 Thu, 15 Feb 2024 06:59:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVfr-0003NA-3y
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:56:47 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6247e130-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:56:46 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id EF9361F850;
 Thu, 15 Feb 2024 06:56:45 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id BED311346A;
 Thu, 15 Feb 2024 06:56:45 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id EtIiLa21zWV6SQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:56:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6247e130-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980206; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Vc2VSUYI94Us0Ei3rvbkISjz40iZ7FXy+b6mZ9EcBLc=;
	b=LybbqxRT5+qLQsi0FAiLmPsWJolEPqW2kVYrinmbaCkkyexNAxZ5Lt2nnZyTl9j/r0nI4W
	ERHwRiS+XiSp61SRlqp6CR7Goi8yd2MAWbSCt153IHwUD3eOgRuPaTxBYMBovLqYtLNfK/
	0E7Udu+JNrjyWHktM6N2r44Pg1PYnrw=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980205; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Vc2VSUYI94Us0Ei3rvbkISjz40iZ7FXy+b6mZ9EcBLc=;
	b=EvBu3DKerQ2dxTgL4JAg7cdOJko2RS0G3t3Rt9PbgxQvAdVsS0tb8x6eIEg+Hjrgkcg4QO
	JGSLTZOCcJo1RE1CXBMHIpyDu+WbYsSvdfl9ES4ZNOa6hZSZflQEIamHgBBHgpT14kDwbw
	wHN9B42zbKsQfAkeJdExhloIVXFm4co=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v6 11/21] tools/xen-9pfsd: add 9pfs stat request support
Date: Thu, 15 Feb 2024 07:55:31 +0100
Message-Id: <20240215065541.21067-12-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: 0.70
X-Spamd-Result: default: False [0.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-0.999];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[citrix.com:email,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add the stat request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V3:
- use fstatat() (Jason Andryuk)
V4:
- add "s" format to fill_buffer() as a preparation for reading dirs
---
 tools/9pfsd/io.c | 102 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 65ff4dab73..d08c4b1283 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -34,6 +34,7 @@
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
 #define P9_CMD_CLUNK      120
+#define P9_CMD_STAT       124
 
 /* P9 protocol open flags. */
 #define P9_OREAD            0   /* read */
@@ -60,6 +61,25 @@ struct p9_qid {
     uint64_t path;
 };
 
+struct p9_stat {
+    uint16_t size;
+    uint16_t type;
+    uint32_t dev;
+    struct p9_qid qid;
+    uint32_t mode;
+    uint32_t atime;
+    uint32_t mtime;
+    uint64_t length;
+    const char *name;
+    const char *uid;
+    const char *gid;
+    const char *muid;
+    const char *extension;
+    uint32_t n_uid;
+    uint32_t n_gid;
+    uint32_t n_muid;
+};
+
 /*
  * Note that the ring names "in" and "out" are from the frontend's
  * perspective, so the "in" ring will be used for responses to the frontend,
@@ -166,6 +186,7 @@ static void fmt_err(const char *fmt)
  * S: String (2 byte length + <length> characters)
  *    The length is obtained via strlen() of the parameter, being a pointer
  *    to the first character of the string
+ * s: stat (struct p9_stat)
  * U: 4 byte unsigned integer
  *    The parameter is a pointer to a uint32_t value
  */
@@ -176,6 +197,8 @@ static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
     const void *par;
     const char *str_val;
     const struct p9_qid *qid;
+    const struct p9_stat *stat;
+    uint16_t tlen;
     unsigned int len;
     unsigned int array_sz = 0;
     unsigned int elem_sz = 0;
@@ -259,6 +282,18 @@ static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
             *data += len;
             break;
 
+        case 's':
+            stat = par;
+            elem_sz = sizeof(*stat);
+            tlen = stat->size + sizeof(stat->size);
+            fill_buffer_at(data, "uuuUQUUULSSSSSUUU", &tlen, &stat->size,
+                           &stat->type, &stat->dev, &stat->qid, &stat->mode,
+                           &stat->atime, &stat->mtime, &stat->length,
+                           stat->name, stat->uid, stat->gid, stat->muid,
+                           stat->extension, &stat->n_uid, &stat->n_gid,
+                           &stat->n_muid);
+            break;
+
         case 'U':
             put_unaligned(*(const uint32_t *)par, (uint32_t *)*data);
             elem_sz = sizeof(uint32_t);
@@ -1148,6 +1183,69 @@ static void p9_clunk(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "");
 }
 
+static void fill_p9_stat(device *device, struct p9_stat *p9s, struct stat *st,
+                         const char *name)
+{
+    memset(p9s, 0, sizeof(*p9s));
+    fill_qid(device, NULL, &p9s->qid, st);
+    p9s->mode = st->st_mode & 0777;
+    if ( S_ISDIR(st->st_mode) )
+        p9s->mode |= P9_CREATE_PERM_DIR;
+    p9s->atime = st->st_atime;
+    p9s->mtime = st->st_mtime;
+    p9s->length = st->st_size;
+    p9s->name = name;
+    p9s->uid = "";
+    p9s->gid = "";
+    p9s->muid = "";
+    p9s->extension = "";
+    p9s->n_uid = 0;
+    p9s->n_gid = 0;
+    p9s->n_muid = 0;
+
+    /*
+     * Size of individual fields without the size field, including 5 2-byte
+     * string length fields.
+     */
+    p9s->size = 71 + strlen(p9s->name);
+}
+
+static void p9_stat(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    struct p9_fid *fidp;
+    struct p9_stat p9s;
+    struct stat st;
+    int ret;
+
+    ret = fill_data(ring, "U", &fid);
+    if ( ret != 1 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+    fill_p9_stat(device, &p9s, &st, strrchr(fidp->path, '/') + 1);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "s", &p9s);
+
+ out:
+    free_fid(device, fidp);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1227,6 +1325,10 @@ void *io_thread(void *arg)
                 p9_clunk(ring, &hdr);
                 break;
 
+            case P9_CMD_STAT:
+                p9_stat(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 06:59:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 06:59:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681312.1060108 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVii-0005lN-3q; Thu, 15 Feb 2024 06:59:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681312.1060108; Thu, 15 Feb 2024 06:59:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVih-0005kw-VO; Thu, 15 Feb 2024 06:59:43 +0000
Received: by outflank-mailman (input) for mailman id 681312;
 Thu, 15 Feb 2024 06:59:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raVgO-0003NA-LR
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 06:57:20 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7652083e-cbcf-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 07:57:20 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id A7D561F850;
 Thu, 15 Feb 2024 06:57:19 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 74D641346A;
 Thu, 15 Feb 2024 06:57:19 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 9NgDG8+1zWWUSQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 06:57:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7652083e-cbcf-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980239; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=t9xMwpWGoJR7DxJ7ry2LqpTZcYdrlw/DDzPveaCxkX4=;
	b=KIHCF+j/4GL4sKcW5wSB2W+XS4sAf+TaOS3Sl8denIp/8U/RjmXqmfLrOvPwkja/tZdejQ
	omJcMNze1dKNnZ6non7aim4ph2CXDRMEv1c6ZbvRMOOk4DCuwwHfJQUWCsR4KpxDHeISaj
	MgrEhM6jAhxXSw/TkAWFXjT8r6HxuFg=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707980239; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=t9xMwpWGoJR7DxJ7ry2LqpTZcYdrlw/DDzPveaCxkX4=;
	b=KIHCF+j/4GL4sKcW5wSB2W+XS4sAf+TaOS3Sl8denIp/8U/RjmXqmfLrOvPwkja/tZdejQ
	omJcMNze1dKNnZ6non7aim4ph2CXDRMEv1c6ZbvRMOOk4DCuwwHfJQUWCsR4KpxDHeISaj
	MgrEhM6jAhxXSw/TkAWFXjT8r6HxuFg=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v6 17/21] tools: add 9pfs device to xenstore-stubdom
Date: Thu, 15 Feb 2024 07:55:37 +0100
Message-Id: <20240215065541.21067-18-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215065541.21067-1-jgross@suse.com>
References: <20240215065541.21067-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b="KIHCF+j/"
X-Spamd-Result: default: False [4.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 4.69
X-Rspamd-Queue-Id: A7D561F850
X-Spam-Level: ****
X-Spam-Flag: NO
X-Spamd-Bar: ++++

Add a 9pfs device to Xenstore stubdom in order to allow it to do e.g.
logging into a dom0 file.

Use the following parameters for the new device:

- tag = "Xen"
- type = "xen_9pfsd"
- path = "/var/lib/xen/xenstore"
- security-model = "none"

For now don't limit allowed file space or number of files.

Add a new libxl function for adding it similar to the function for
adding the console device.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- add security_model parameter to new libxl function (Jason Andryuk)
V4:
- rename function to libxl_device_9pfs_add() (Anthony Perard)
- use a libxl_device_p9 pointer as parameter (Anthony Perard)
---
 tools/helpers/init-xenstore-domain.c |  7 +++++++
 tools/include/libxl.h                | 15 +++++++++++++++
 tools/libs/light/libxl_9pfs.c        | 16 ++++++++++++++++
 3 files changed, 38 insertions(+)

diff --git a/tools/helpers/init-xenstore-domain.c b/tools/helpers/init-xenstore-domain.c
index 140ed610ae..1683438c5c 100644
--- a/tools/helpers/init-xenstore-domain.c
+++ b/tools/helpers/init-xenstore-domain.c
@@ -433,6 +433,12 @@ int main(int argc, char** argv)
     int rv, fd;
     char *maxmem_str = NULL;
     libxl_ctx *ctx;
+    libxl_device_p9 p9 = { .backend_domid = 0,
+                           .tag = "Xen",
+                           .path = XEN_LIB_DIR"/xenstore",
+                           .security_model = "none",
+                           .type = LIBXL_P9_TYPE_XEN_9PFSD,
+    };
 
     while ( (opt = getopt_long(argc, argv, "v", options, NULL)) != -1 )
     {
@@ -543,6 +549,7 @@ int main(int argc, char** argv)
     }
     libxl_console_add_xenstore(ctx, domid, 0, console_evtchn, console_gfn,
                                NULL);
+    libxl_device_9pfs_add(ctx, domid, &p9, NULL);
     libxl_ctx_free(ctx);
 
     fd = creat(XEN_RUN_DIR "/xenstored.pid", 0666);
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 9a3e702557..44a2205d2b 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -583,6 +583,13 @@
  * libxl_console_add_xenstore() in libxl.
  */
 #define LIBXL_HAVE_CONSOLE_ADD_XENSTORE 1
+
+/*
+ * LIBXL_HAVE_P9_ADD_XENSTORE indicates presence of the function
+ * libxl_device_9pfs_add() in libxl.
+ */
+#define LIBXL_HAVE_P9_ADD_XENSTORE 1
+
 /*
  * libxl ABI compatibility
  *
@@ -2074,6 +2081,14 @@ int libxl_console_add_xenstore(libxl_ctx *ctx, uint32_t domid, uint32_t backend,
                                const libxl_asyncop_how *ao_how)
                                LIBXL_EXTERNAL_CALLERS_ONLY;
 
+/* libxl_device_9pfs_add writes the Xenstore entries for a domain's
+ * primary 9pfs device based on domid, and device parameters.
+ * If needed it will start the backend daemon.
+ */
+int libxl_device_9pfs_add(libxl_ctx *ctx, uint32_t domid, libxl_device_p9 *p9,
+                          const libxl_asyncop_how *ao_how)
+                          LIBXL_EXTERNAL_CALLERS_ONLY;
+
 /* May be called with info_r == NULL to check for domain's existence.
  * Returns ERROR_DOMAIN_NOTFOUND if domain does not exist (used to return
  * ERROR_INVAL for this scenario). */
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index ddeb4f20a7..48f894f070 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -206,6 +206,22 @@ static void libxl__device_p9_add(libxl__egc *egc, uint32_t domid,
         aodev->callback(egc, aodev);
 }
 
+int libxl_device_9pfs_add(libxl_ctx *ctx, uint32_t domid, libxl_device_p9 *p9,
+                          const libxl_asyncop_how *ao_how)
+{
+    AO_CREATE(ctx, domid, ao_how);
+    libxl__ao_device *aodev;
+
+    GCNEW(aodev);
+    libxl__prepare_ao_device(ao, aodev);
+    aodev->action = LIBXL__DEVICE_ACTION_ADD;
+    aodev->callback = device_addrm_aocomplete;
+
+    libxl__device_p9_add(egc, domid, p9, aodev);
+
+    return AO_INPROGRESS;
+}
+
 #define libxl_device_p9_list NULL
 #define libxl_device_p9_compare NULL
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 07:13:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 07:13:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681361.1060124 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVwJ-0003fp-CB; Thu, 15 Feb 2024 07:13:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681361.1060124; Thu, 15 Feb 2024 07:13:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raVwJ-0003fi-6y; Thu, 15 Feb 2024 07:13:47 +0000
Received: by outflank-mailman (input) for mailman id 681361;
 Thu, 15 Feb 2024 07:13:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raVwI-0003fW-AH; Thu, 15 Feb 2024 07:13:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raVwI-0007eR-7v; Thu, 15 Feb 2024 07:13:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raVwH-0007HD-NH; Thu, 15 Feb 2024 07:13:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raVwH-0001ii-Lb; Thu, 15 Feb 2024 07:13:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=QQFXMlyN+9lhtSobD/dbpFFN58pIFtU65ztCfW8Pg5c=; b=wW9YzxIAPS6w1lEAa/BIdjehhi
	UNwOABsYkJ9R7GBAoia+rMe6pszOoGwNpWPCOpdO9VENAT4IXXYSK4ZzQp36vyDLgIa5Jjew4pI7X
	2UTkCuHCAOQGnp7EyXFWnf4MCenIAmP0Z0d5yi8XHWK+6CCw+kir176HFj/84AYRHpVQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184670-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184670: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-dom0pvh-xl-amd:guest-start:fail:heisenbug
    xen-unstable:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=d9bfc2810e1cbe4cd27555bdc48315d36ee91f14
X-Osstest-Versions-That:
    xen=d670c1a38ba3561296f68c0079209365760b3001
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 15 Feb 2024 07:13:45 +0000

flight 184670 xen-unstable real [real]
flight 184672 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184670/
http://logs.test-lab.xenproject.org/osstest/logs/184672/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-dom0pvh-xl-amd 14 guest-start      fail pass in 184672-retest
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 12 debian-hvm-install fail pass in 184672-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184668
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184668
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184668
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184668
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184668
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184668
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184668
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184668
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184668
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184668
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184668
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184668
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  d9bfc2810e1cbe4cd27555bdc48315d36ee91f14
baseline version:
 xen                  d670c1a38ba3561296f68c0079209365760b3001

Last test of basis   184668  2024-02-14 15:38:57 Z    0 days
Testing same since   184670  2024-02-14 22:40:46 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   d670c1a38b..d9bfc2810e  d9bfc2810e1cbe4cd27555bdc48315d36ee91f14 -> master


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 08:11:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 08:11:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681376.1060132 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raWpR-0001hx-09; Thu, 15 Feb 2024 08:10:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681376.1060132; Thu, 15 Feb 2024 08:10:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raWpQ-0001hq-Tp; Thu, 15 Feb 2024 08:10:44 +0000
Received: by outflank-mailman (input) for mailman id 681376;
 Thu, 15 Feb 2024 08:10:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raWpQ-0001hk-21
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 08:10:44 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b5cf3e8b-cbd9-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 09:10:41 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-411e5f21c0bso5066725e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 00:10:41 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bk30-20020a0560001d9e00b0033cff6da10bsm977600wrb.89.2024.02.15.00.10.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 00:10:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5cf3e8b-cbd9-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707984641; x=1708589441; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=hFKq6tUiKQaY/aBQSjO+MQHA3hkBWmNRwqZcUU0D4Q0=;
        b=Pz7xpWHAzZhv047d1m0i/n35X8ALZVpiZXvbfLu/3Td2Od9La2bnJ+dc8AuD/gGIAX
         toMe/w27snwwDEXx+9JO92uvsygbAd1CVo+BTeTmsqgOQ19uUrjaIEuRNusrZ25gP1nM
         9fWV5ld+fISWZod04ZMO97KTRpPG2eA07HkuZxCzf6WLxpBwPLVgyqzTGKPjKNSr4LQd
         PtKj5ETIBxWufO+ZeA9XMhW+omuknrUIJTWvPbz74YuM5fZjhkiHT34fD0Qx6aEFFbq1
         T0D+J5QWX+YoN9D2S9uIG1XZ6IlF/A+rX09/AFv7xcTK7lJpAwooh/OhjvYOypS6MC57
         UteA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707984641; x=1708589441;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=hFKq6tUiKQaY/aBQSjO+MQHA3hkBWmNRwqZcUU0D4Q0=;
        b=oMwI69iuSX/YYjCeeBAJdNqESov2mtODTdvazzkH8/+ZePB05S++bFDNwsGlrRse5a
         S9vymSIOw4usjjJPopk42KWTCSKcruBHSDZCmQgEZ5hYMZexh5R2jba8+q+wQL4ps5jD
         TbgfRejPAZpxFbb52bn8DNQGMOPb7I6X+3ELZJn34POkq9Q4CYdvbS4lGaGlURQDbdmY
         0VEV43w3K+wEWYCJbv0Vv8D28dEUS21Svh3PD396Wkcoy+/YMnN4zEN9vHs0Ekl0pHUB
         UJqoR8O2ROnC7n7bLLnh/kNhRGmmun1chH4cs/b6luizvyqn78mFiU96zXWtLZFStvVc
         8sBg==
X-Forwarded-Encrypted: i=1; AJvYcCUYA1Kzh7hXUFWUH1ydjxyDE+1BP9yICG8gPO5UAVNqCNQc/bUuRWBHGWNFuqB+9+a3kin+GdQp4e6HxJeqQFWhRbugvifSSEFY3zOtheA=
X-Gm-Message-State: AOJu0YwCOBdG7orBBxDaaDEqKYM/IDp6ESNzCr0fvDERgr287l49oR43
	0NYxJ+VzjrCNgmK0C7jaECxIzbxTiobImeqcZwnDAlrqX3ZsfW6f+OaftDqDjQ==
X-Google-Smtp-Source: AGHT+IGOWyJqS9NYs0B/yS14vLtYA8JD0PB4LWgwqp0KCMZxQGnQiKAnjGQKwk2sWWiT8XyLDe2uHw==
X-Received: by 2002:a05:600c:46cd:b0:410:cf60:d857 with SMTP id q13-20020a05600c46cd00b00410cf60d857mr762245wmo.18.1707984641008;
        Thu, 15 Feb 2024 00:10:41 -0800 (PST)
Message-ID: <45c19a87-5833-4350-a374-3a70fd3aa9ae@suse.com>
Date: Thu, 15 Feb 2024 09:10:39 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
 <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
 <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
 <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com>
 <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 14.02.2024 17:11, Federico Serafini wrote:
> I did some tries with example programs
> and the assembler error always points to file and line
> of the most enclosing function that caused the failure.
> If I am not missing something, using __FILE__ and __LINE__ does not add
> any information.
> 
> Therefore, if the new macro is used within the body of other macros,
> then the resulting assembler error will point to the source of
> the problem (e.g., the site of a bogus call to put_guest()).
> 
> In my opinion, converting put_guest() &Co. to inline functions is not
> convenient: the assembler error will point to the most enclosing
> function that would be put_unsafe_size(), instead of pointing to the
> source of the problem.

The assembler error will point to where the inline function was expanded,
sure. __FILE__ / __LINE__ ought to point to that inline function (where
the macro was used) then, though?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 08:37:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 08:37:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681391.1060142 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raXF4-0005MW-0b; Thu, 15 Feb 2024 08:37:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681391.1060142; Thu, 15 Feb 2024 08:37:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raXF3-0005MP-U6; Thu, 15 Feb 2024 08:37:13 +0000
Received: by outflank-mailman (input) for mailman id 681391;
 Thu, 15 Feb 2024 08:37:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wY2D=JY=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raXF2-0005MJ-Fl
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 08:37:12 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6965ab9a-cbdd-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 09:37:11 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d0cdbd67f0so8049681fa.3
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 00:37:11 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 g8-20020a05620a108800b00785d6c811cdsm420962qkk.70.2024.02.15.00.37.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 00:37:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6965ab9a-cbdd-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707986231; x=1708591031; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=p0Tr3DlNIUsbNl7Zybl/OUv7Beh0KoEzVQNaV1wEscA=;
        b=C6cBX1OTLsjXo7C5FnWa3SKRt6t1uSM/hPyP7wzhRUhRY+aNVYBy/88vdOHO/hzFte
         eVAVNOPojfpZCvk2zbfI+6qiaETMhlSyMimjVjyV2iMyJuzrHsHI8wzubfCOaPEM0xgv
         KFTaURM4vjuYNhm6CqOhw9oYQiacD3QYZ7Tsg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707986231; x=1708591031;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=p0Tr3DlNIUsbNl7Zybl/OUv7Beh0KoEzVQNaV1wEscA=;
        b=drdOh6uhZMnm+czGLvtTtf6gkvHzkNHJnkFRLu9lzDTKiXhMr0mF1HkoO5/7Lyh5Yl
         zFq9HPwipORrfIxWouXasKDWAO5xM2mvbHsqMyCGJQID7Oy05fOUTj+h9m13zM8uCF+4
         ChmoFCx1dsKaRzrsRi2mO9AZKdwlcgD/I476KIrSYpANW5CPbhIp52tTBzwq7eT32InX
         SIo0lmmUWORYHzt1ssPbh2XoqOpbW1qMCPn5w8a0hAZ29ltwguDQM70HNIlEGDOCjzag
         F1AvarJSRFI9QoBBLwBZZOl6AVmzGfFfyoQTZkmDHQmzJaG+FhlH9b6NuBJKruP9ahnR
         yQsg==
X-Forwarded-Encrypted: i=1; AJvYcCXEyjFx/VYy6Ix8Dm6s1R4E5PcqvzOWpMPTO5f9BeMlExggcoYwtiG43NSx2szAys/iZBcrxt9yXSTMW2YobvdNMxNPPyZNAv13In256Bc=
X-Gm-Message-State: AOJu0YwrDlwTq2vuMqRrQnSZrvDf6A33ENzI46433UKjQR45EA+x9ixi
	jPixGnrsKZdZ72YqOM7kI0s8uzvsDuCtowDbDVHA4BM+p6hufyRcog7bt88cVwU=
X-Google-Smtp-Source: AGHT+IG6L8UDyDxpMu9X27hfF38b95OqoJaWrWoeoQPrfZTqYcZ+N/S/qJDlQGyYvbnfKjPT22kAlw==
X-Received: by 2002:a2e:a9a0:0:b0:2d0:fe83:bc87 with SMTP id x32-20020a2ea9a0000000b002d0fe83bc87mr1234033ljq.10.1707986230631;
        Thu, 15 Feb 2024 00:37:10 -0800 (PST)
Date: Thu, 15 Feb 2024 09:37:08 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: "Chen, Jiqian" <Jiqian.Chen@amd.com>,
	"Rafael J . Wysocki" <rafael@kernel.org>,
	Len Brown <lenb@kernel.org>, Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
	"Huang, Ray" <Ray.Huang@amd.com>,
	"Ragiadakou, Xenia" <Xenia.Ragiadakou@amd.com>
Subject: Re: [RFC KERNEL PATCH v4 3/3] PCI/sysfs: Add gsi sysfs for pci_dev
Message-ID: <Zc3NNOEN-NPR9jjI@macbook>
References: <ZcnhOEjnTgbYFPVl@macbook>
 <20240212191858.GA1137351@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240212191858.GA1137351@bhelgaas>

On Mon, Feb 12, 2024 at 01:18:58PM -0600, Bjorn Helgaas wrote:
> On Mon, Feb 12, 2024 at 10:13:28AM +0100, Roger Pau Monné wrote:
> > On Fri, Feb 09, 2024 at 03:05:49PM -0600, Bjorn Helgaas wrote:
> > > On Thu, Feb 01, 2024 at 09:39:49AM +0100, Roger Pau Monné wrote:
> > > > On Wed, Jan 31, 2024 at 01:00:14PM -0600, Bjorn Helgaas wrote:
> > > > > On Wed, Jan 31, 2024 at 09:58:19AM +0100, Roger Pau Monné wrote:
> > > > > > On Tue, Jan 30, 2024 at 02:44:03PM -0600, Bjorn Helgaas wrote:
> > > > > > > On Tue, Jan 30, 2024 at 10:07:36AM +0100, Roger Pau Monné wrote:
> > > > > > > > On Mon, Jan 29, 2024 at 04:01:13PM -0600, Bjorn Helgaas wrote:
> > > > > > > > > On Thu, Jan 25, 2024 at 07:17:24AM +0000, Chen, Jiqian wrote:
> > > > > > > > > > On 2024/1/24 00:02, Bjorn Helgaas wrote:
> > > > > > > > > > > On Tue, Jan 23, 2024 at 10:13:52AM +0000, Chen, Jiqian wrote:
> > > > > > > > > > >> On 2024/1/23 07:37, Bjorn Helgaas wrote:
> > > > > > > > > > >>> On Fri, Jan 05, 2024 at 02:22:17PM +0800, Jiqian Chen wrote:
> > > > > > > > > > >>>> There is a need for some scenarios to use gsi sysfs.
> > > > > > > > > > >>>> For example, when xen passthrough a device to dumU, it will
> > > > > > > > > > >>>> use gsi to map pirq, but currently userspace can't get gsi
> > > > > > > > > > >>>> number.
> > > > > > > > > > >>>> So, add gsi sysfs for that and for other potential scenarios.
> > > > > > > > > > >> ...
> > > > > > > > > > > 
> > > > > > > > > > >>> I don't know enough about Xen to know why it needs the GSI in
> > > > > > > > > > >>> userspace.  Is this passthrough brand new functionality that can't be
> > > > > > > > > > >>> done today because we don't expose the GSI yet?
> > > > > > > > > 
> > > > > > > > > I assume this must be new functionality, i.e., this kind of
> > > > > > > > > passthrough does not work today, right?
> > > > > > > > > 
> > > > > > > > > > >> has ACPI support and is responsible for detecting and controlling
> > > > > > > > > > >> the hardware, also it performs privileged operations such as the
> > > > > > > > > > >> creation of normal (unprivileged) domains DomUs. When we give to a
> > > > > > > > > > >> DomU direct access to a device, we need also to route the physical
> > > > > > > > > > >> interrupts to the DomU. In order to do so Xen needs to setup and map
> > > > > > > > > > >> the interrupts appropriately.
> > > > > > > > > > > 
> > > > > > > > > > > What kernel interfaces are used for this setup and mapping?
> > > > > > > > > >
> > > > > > > > > > For passthrough devices, the setup and mapping of routing physical
> > > > > > > > > > interrupts to DomU are done on Xen hypervisor side, hypervisor only
> > > > > > > > > > need userspace to provide the GSI info, see Xen code:
> > > > > > > > > > xc_physdev_map_pirq require GSI and then will call hypercall to pass
> > > > > > > > > > GSI into hypervisor and then hypervisor will do the mapping and
> > > > > > > > > > routing, kernel doesn't do the setup and mapping.
> > > > > > > > > 
> > > > > > > > > So we have to expose the GSI to userspace not because userspace itself
> > > > > > > > > uses it, but so userspace can turn around and pass it back into the
> > > > > > > > > kernel?
> > > > > > > > 
> > > > > > > > No, the point is to pass it back to Xen, which doesn't know the
> > > > > > > > mapping between GSIs and PCI devices because it can't execute the ACPI
> > > > > > > > AML resource methods that provide such information.
> > > > > > > > 
> > > > > > > > The (Linux) kernel is just a proxy that forwards the hypercalls from
> > > > > > > > user-space tools into Xen.
> > > > > > > 
> > > > > > > But I guess Xen knows how to interpret a GSI even though it doesn't
> > > > > > > have access to AML?
> > > > > > 
> > > > > > On x86 Xen does know how to map a GSI into an IO-APIC pin, in order
> > > > > > configure the RTE as requested.
> > > > > 
> > > > > IIUC, mapping a GSI to an IO-APIC pin requires information from the
> > > > > MADT.  So I guess Xen does use the static ACPI tables, but not the AML
> > > > > _PRT methods that would connect a GSI with a PCI device?
> > > > 
> > > > Yes, Xen can parse the static tables, and knows the base GSI of
> > > > IO-APICs from the MADT.
> > > > 
> > > > > I guess this means Xen would not be able to deal with _MAT methods,
> > > > > which also contains MADT entries?  I don't know the implications of
> > > > > this -- maybe it means Xen might not be able to use with hot-added
> > > > > devices?
> > > > 
> > > > It's my understanding _MAT will only be present on some very specific
> > > > devices (IO-APIC or CPU objects).  Xen doesn't support hotplug of
> > > > IO-APICs, but hotplug of CPUs should in principle be supported with
> > > > cooperation from the control domain OS (albeit it's not something that
> > > > we tests on our CI).  I don't expect however that a CPU object _MAT
> > > > method will return IO APIC entries.
> > > > 
> > > > > The tables (including DSDT and SSDTS that contain the AML) are exposed
> > > > > to userspace via /sys/firmware/acpi/tables/, but of course that
> > > > > doesn't mean Xen knows how to interpret the AML, and even if it did,
> > > > > Xen probably wouldn't be able to *evaluate* it since that could
> > > > > conflict with the host kernel's use of AML.
> > > > 
> > > > Indeed, there can only be a single OSPM, and that's the dom0 OS (Linux
> > > > in our context).
> > > > 
> > > > Getting back to our context though, what would be a suitable place for
> > > > exposing the GSI assigned to each device?
> > > 
> > > IIUC, the Xen hypervisor:
> > > 
> > >   - Interprets /sys/firmware/acpi/tables/APIC (or gets this via
> > >     something running on the Dom0 kernel) to find the physical base
> > >     address and GSI base, e.g., from I/O APIC, I/O SAPIC.
> > 
> > No, Xen parses the MADT directly from memory, before stating dom0.
> > That's a static table so it's fine for Xen to parse it and obtain the
> > I/O APIC GSI base.
> 
> It's an interesting split to consume ACPI static tables directly but
> put the AML interpreter elsewhere.

Well, static tables can be consumed by Xen, because thye don't require
an AML parser (obviously), and parsing them doesn't have any
side-effects that would prevent dom0 from being the OSPM (no methods
or similar are evaluated).

> I doubt the ACPI spec envisioned
> that, which makes me wonder what other things we could trip over, but
> that's just a tangent.

Indeed, ACPI is not be best interface for the Xen/dom0 split model.

> > >   - Needs the GSI to locate the APIC and pin within the APIC.  The
> > >     Dom0 kernel is the OSPM, so only it can evaluate the AML _PRT to
> > >     learn the PCI device -> GSI mapping.
> > 
> > Yes, Xen doesn't know the PCI device -> GSI mapping.  Dom0 needs to
> > parse the ACPI methods and signal Xen to configure a GSI with a
> > given trigger and polarity.
> > 
> > >   - Has direct access to the APIC physical base address to program the
> > >     Redirection Table.
> > 
> > Yes, the hardware (native) I/O APIC is owned by Xen, and not directly
> > accessible by dom0.
> > 
> > > The patch seems a little messy to me because the PCI core has to keep
> > > track of the GSI even though it doesn't need it itself.  And the
> > > current patch exposes it on all arches, even non-ACPI ones or when
> > > ACPI is disabled (easily fixable).
> > > 
> > > We only call acpi_pci_irq_enable() in the pci_enable_device() path, so
> > > we don't know the GSI unless a Dom0 driver has claimed the device and
> > > called pci_enable_device() for it, which seems like it might not be
> > > desirable.
> > 
> > I think that's always the case, as on dom0 devices to be passed
> > through are handled by pciback which does enable them.
> 
> pcistub_init_device() labels the pci_enable_device() as a "HACK"
> related to determining the IRQ, which makes me think there's not
> really a requirement for the device to be *enabled* (BAR decoding
> enabled) by dom0.

No, there's no need for memory decoding to be enabled for getting the
GSI from the ACPI method I would assume.  I'm confused by that
pci_enable_device() call.  Is maybe the purpose to make sure the
device is powered up so that reading the PCI header Interrupt Line and
Pin fields returns valid values?  No idea whether reading those fields
requires the device to be in certain (active) power states.

> > I agree it might be best to not tie exposing the node to
> > pci_enable_device() having been called.  Is _PRT only evaluated as
> > part of acpi_pci_irq_enable()? (or pci_enable_device()).
> 
> Yes.  AFAICT, acpi_pci_irq_enable() is the only path that evaluates
> _PRT (except for a debugger interface).  I don't think it *needs* to
> be that way, and the fact that we do it per-device like that means we
> evaluate _PRT many times even though I think the results never change.
> 
> I could imagine evaluating _PRT once as part of enumerating a PCI host
> bridge (and maybe PCI-PCI bridge, per acpi_pci_irq_find_prt_entry()
> comment), but that looks like a fair bit of work to implement.  And of
> course it doesn't really affect the question of how to expose the
> result, although it does suggest /sys/bus/acpi/devices/PNP0A03:00/ as
> a possible location.

So you suggest exposing the GSI as part of the PCI host bridge?  I'm
afraid I'm not following how we could then map PCI SBDFs from devices
to their assigned GSI.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 08:39:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 08:39:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681395.1060153 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raXHV-0006C6-Cq; Thu, 15 Feb 2024 08:39:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681395.1060153; Thu, 15 Feb 2024 08:39:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raXHV-0006Bz-9q; Thu, 15 Feb 2024 08:39:45 +0000
Received: by outflank-mailman (input) for mailman id 681395;
 Thu, 15 Feb 2024 08:39:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raXHU-0006Bp-As
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 08:39:44 +0000
Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com
 [2a00:1450:4864:20::436])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c3d007e2-cbdd-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 09:39:43 +0100 (CET)
Received: by mail-wr1-x436.google.com with SMTP id
 ffacd0b85a97d-33cf6266c76so354826f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 00:39:43 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i6-20020a5d5226000000b0033940016d6esm1027881wra.93.2024.02.15.00.39.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 00:39:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3d007e2-cbdd-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707986382; x=1708591182; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=v1/Q95sj4WA2FF9J6ud+YMn0QgbGrf4qbedowJ2Con4=;
        b=gkVgqdQj1X/8DXitT5Z8cO3/UuDrtupO5SjNdlo7+tUJ9lHgZZ6JBPQyk5rUnzzmqU
         CjPkEMMDEFSexAMPF14+F2OUrpmWN/HLs9vb3Nfoeknt/FNlhmn1DLdZbyYpMEz+4Cnr
         hRuQJfqUCoiRHWMpkADm4zy6O2/idqbgtBlFmiv38LYnG27CLRZ4GHQLk/fDIMHkN3be
         lohh277K9Y10aOeA1PaMM0jPKJr/BxVQK0uK02HG71RqRK79o4YzUxn3ZqUakVJQuddL
         s0asBmRiKdKPdR/S6JrlD5+vNsPcur3qtv3dPry40mm4JGO+/BNmnbXP3bgGWyJ/JygY
         dXfg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707986382; x=1708591182;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=v1/Q95sj4WA2FF9J6ud+YMn0QgbGrf4qbedowJ2Con4=;
        b=tEYoVgHYcTrrE/K9FrQAKguRBEf/gEP9jPdrfvlSGt9Apjpf79dlzIfcfxPp0vMoV0
         GNavF2ML/dtJibPWAXcE6cAhb6LLzo0RW8b3qUqNOBxzCMO3ogvUGjLgWhmy2y56F+9k
         cvY11xYrX5syjqk2pagXNGGe6cHj7jhT8u34tcLRPGPrVmq+GPO3x6GRqP/sKOq0JIFY
         tWLW7Bgng1YMj3R+7zhlPceIbJk6KrblzAnMQMmvwDpCmGAxZcRy+we2wpFay+tcizD6
         QIbUoJf03u/u5CVdgz6S++pXZAvz4W3OWCQNAAt+UHDRXgJQ1zQRHzYklJc5elhXKYaq
         uaSA==
X-Forwarded-Encrypted: i=1; AJvYcCXVjL5VJt1Wnx0xeNEjHtrWzvUnjLAWOiCEx+sCf6rggqYv+tQNTMluna0GX1QusR6XIMsnE62QMcVmiwn9KgDbtR3Bq/YwjvfQx7ZpIf0=
X-Gm-Message-State: AOJu0YzMC+/LnEXCCmECYmMaQes+ivuPjuRbCPmXtz6osd72zpGJRvXz
	920/8cMPtP5xCvxHOAaEFePxEavi8p/sEyjVhs4YPt+2RnGtbgVdkojl2tl7VA==
X-Google-Smtp-Source: AGHT+IGOW30j0aeCwbqfiKfzN6D8QEHzDpFUPdT8rVTsOH6aSMYzYKa5RBJmGf69RzhMTCNx2EKkvw==
X-Received: by 2002:adf:fd43:0:b0:33c:efad:c6a2 with SMTP id h3-20020adffd43000000b0033cefadc6a2mr860687wrs.61.1707986382439;
        Thu, 15 Feb 2024 00:39:42 -0800 (PST)
Message-ID: <6dbb77a9-0a2d-4eb2-a484-d43d007e4182@suse.com>
Date: Thu, 15 Feb 2024 09:39:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Content-Language: en-US
To: Marek Marczykowski <marmarek@invisiblethingslab.com>
Cc: Julien Grall <julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
 <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
 <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com> <ZcrlcuNU9y8WymiK@mail-itl>
 <817ceec4-ed12-4360-a6ce-f87732f62251@suse.com> <ZcuCTnMpFib4ArHW@mail-itl>
 <b52fa3fa-4e33-4211-af18-f00aaa0b365b@suse.com> <ZcuGo_vQhiTAQep0@mail-itl>
 <23bcd506-f843-42f7-a541-0d8db7c319ab@suse.com> <Zc10vFQtG40isDHK@mail-itl>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zc10vFQtG40isDHK@mail-itl>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.02.2024 03:19, Marek Marczykowski wrote:
> On Tue, Feb 13, 2024 at 04:44:04PM +0100, Jan Beulich wrote:
>> On 13.02.2024 16:11, Marek Marczykowski wrote:
>>> On Tue, Feb 13, 2024 at 04:00:32PM +0100, Jan Beulich wrote:
>>>> On 13.02.2024 15:53, Marek Marczykowski wrote:
>>>>> Generally provides sensible output, for example during boot (it is using
>>>>> idle vCPU then, right?).
>>>>
>>>> Before Dom0 is started: Yes. With the exception of the phase where PV
>>>> Dom0's page tables are constructed, albeit in that time window
>>>> guest_cpu_user_regs() shouldn't yield sensible data either. I can only
>>>> say I'm surprised; since I have no way to properly test with an XHCI
>>>> debug port, I'd have to see about faking something to convince myself
>>>> (unless you were to supply example output).
>>>
>>> Would you like me to test this series with xhci console?
>>
>> The behavior shouldn't really be connected to this series. But yes, 'd'
>> debug key output (just the part for the CPU the key handling was
>> actually invoked from) with the xhci debug console would be of
>> interest, for the case where that CPU at that time runs an idle vCPU.
> 
> I managed to press 'd' before dom0 started. Full output at
> https://gist.github.com/marmarek/a495cd666f4aafed3a5cfcb8393f515b (it's
> Alder Lake, and smt=off, so CPU numbering is weird).
> Interestingly, I do _not_ see output for CPU0, where I'd expect the
> key handler to run... I see all the idle ones, plus one doing memory
> scrubbing.

Which is precisely the problem, just in not exactly the manifestation
I expected. In dump_execstate() we dump host state only if the
incoming regs don't indicate guest state. Yet for the idle vCPU they
(wrongly) do here - see how guest_mode() calculates the delta to what
guest_cpu_user_regs() returns, i.e. 0 when what guest_cpu_user_regs()
returned is passed in.

Guest state dumping is suppressed for idle vCPU-s. Hence no output
at all for the CPU where the key processing was actually invoked
from.

> But also, I don't see info about the handling CPU when doing `xl
> debug-key d`.

I'm afraid I'm confused, ...

> At one time, with `xl debug-key d` I got this:
> 
> (XEN) *** Dumping CPU6 guest state (d0v7): ***
> (XEN) ----[ Xen-4.18-unstable  x86_64  debug=y  Tainted:  M     ]----
> (XEN) CPU:    6
> (XEN) RIP:    e033:[<ffffffff81e1546a>]
> (XEN) RFLAGS: 0000000000000286   EM: 0   CONTEXT: pv guest (d0v7)
> (XEN) rax: 0000000000000023   rbx: 0000000000000005   rcx: ffffffff81e1546a
> (XEN) rdx: 0000000000000000   rsi: 0000000000000000   rdi: 000079147611e010
> (XEN) rbp: ffff88810db53200   rsp: ffffc90041c6bde0   r8:  0000000000000000
> (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000286
> (XEN) r12: 0000000000305000   r13: 00007ffc61097f40   r14: ffff88810db53200
> (XEN) r15: 0000000000000000   cr0: 0000000080050033   cr4: 0000000000b526e0
> (XEN) cr3: 00000004ae2a7000   cr2: 00000000006d3118
> (XEN) fsb: 0000791475b8a380   gsb: ffff8881897c0000   gss: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> (XEN) Guest stack trace from rsp=ffffc90041c6bde0:
> (XEN)    0000000000000001 0000000000000000 ffffffffc02905a6 0000000000000023
> (XEN)    000079147611e010 0000000000000000 0000000000000000 0000000000000000
> (XEN)    0000000000000000 8064129fc747f100 ffffffffc0291568 0000000000305000
> (XEN)    8064129fc747f100 0000000000000005 ffffffff813f7d4d ffffc90041c6bf58
> (XEN)    ffffc90041c6bf48 0000000000000000 0000000000000000 0000000000000000
> (XEN)    ffffffff81e16158 00000000006d3118 ffffc90041c6bf58 0000000000000040
> (XEN)    ffffffff8132f6bb 0000000000000006 ffffc90041c6bf58 00000000006d3118
> (XEN)    0000000000000255 ffff888102cf8880 ffff888102cf88f0 ffffffff8108746f
> (XEN)    0000000000000000 0000000000000002 ffffc90041c6bf58 ffffc90041c6bf58
> (XEN)    00000000006d3118 0000000000000000 0000000000000006 0000000000000000
> (XEN)    0000000000000000 ffffffff81e1a975 0000000000000000 0000000000000000
> (XEN)    0000000000000000 0000000000000000 ffffffff8200009b 000000000043d9b0
> (XEN)    000000000043d990 00007ffc61097f90 00007ffc61097fc0 00007ffc61099d16
> (XEN)    00000000006cab40 0000000000000246 0000000000000001 0000000000000000
> (XEN)    0000000000000006 ffffffffffffffda 0000791475f1ed6f 00007ffc61097f40
> (XEN)    0000000000305000 0000000000000005 0000000000000010 0000791475f1ed6f
> (XEN)    0000000000000033 0000000000000246 00007ffc61097ed0 000000000000002b
> (XEN)     Fault while accessing guest memory.
> (XEN) 
> (XEN) *** Dumping CPU0 host state: ***
> (XEN) ----[ Xen-4.18-unstable  x86_64  debug=y  Tainted:  M     ]----
> (XEN) CPU:    0
> (XEN) RIP:    e008:[<ffff82d04022c07e>] _spin_unlock_irqrestore+0x21/0x27
> (XEN) RFLAGS: 0000000000000206   CONTEXT: hypervisor
> (XEN) rax: ffff82d0405c1038   rbx: 0000000000000200   rcx: 0000000000000008
> (XEN) rdx: ffff830856d07fff   rsi: ffff8308529d5b28   rdi: ffff8308529d5b20
> (XEN) rbp: ffff830856d07dc8   rsp: ffff830856d07dc0   r8:  0000000000000001
> (XEN) r9:  ffff8308529d5b20   r10: ffff82d0405c13a0   r11: 000000d091e62221
> (XEN) r12: ffff82d040476898   r13: 0000000000000296   r14: ffff82d040476918
> (XEN) r15: ffff82cffff04700   cr0: 0000000080050033   cr4: 0000000000b526e0
> (XEN) cr3: 000000082e7ff000   cr2: ffff888109538618
> (XEN) fsb: 0000000000000000   gsb: ffff888189600000   gss: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen code around <ffff82d04022c07e> (_spin_unlock_irqrestore+0x21/0x27):
> (XEN)  fd ff ff 48 09 1c 24 9d <48> 8b 5d f8 c9 c3 0f b7 47 04 66 25 ff 0f 66 3d
> (XEN) Xen stack trace from rsp=ffff830856d07dc0:
> (XEN)    ffff82d0405d0c80 ffff830856d07e08 ffff82d040257c3f 0000000040476898
> (XEN)    ffff82d0405c1280 ffff82d040257bca ffff82d040476898 000000d0911fcbc4
> (XEN)    0000000000000000 ffff830856d07e30 ffff82d04022d55c ffff82d0405c1280
> (XEN)    ffff8308529d5f00 ffff82d0405d0d68 ffff830856d07e70 ffff82d04022de59
> (XEN)    ffff830856d07ef8 ffff82d0405c7f00 ffffffffffffffff ffff82d0405c7f00
> (XEN)    ffff830856d07fff 0000000000000000 ffff830856d07ea8 ffff82d04022b53e
> (XEN)    0000000000000000 0000000000007fff ffff82d0405c7f00 ffff82d0405c11d0
> (XEN)    ffff82d0405db2a0 ffff830856d07eb8 ffff82d04022b5d1 ffff830856d07ef0
> (XEN)    ffff82d0402fcd15 ffff82d0402fcc88 ffff8308528cb000 ffff830856d07ef8
> (XEN)    ffff830856ce2000 0000000000000000 ffff830856d07e18 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 ffffffff82c1aa40
> (XEN)    0000000000000000 0000000000000246 0000000000007ff0 0000000000000000
> (XEN)    000000000fd109eb 0000000000000000 ffffffff81e153aa 4000000000000000
> (XEN)    deadbeefdeadf00d deadbeefdeadf00d 0000010000000000 ffffffff81e153aa
> (XEN)    000000000000e033 0000000000000246 ffffffff82c03dd0 000000000000e02b
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN)    0000e01000000000 ffff830856ce1000 0000000000000000 0000000000b526e0
> (XEN)    0000000000000000 0000000000000000 0006030300000000 0000000000000000
> (XEN) Xen call trace:
> (XEN)    [<ffff82d04022c07e>] R _spin_unlock_irqrestore+0x21/0x27
> (XEN)    [<ffff82d040257c3f>] F xhci-dbc.c#dbc_uart_poll+0x75/0x17c
> (XEN)    [<ffff82d04022d55c>] F timer.c#execute_timer+0x45/0x5c
> (XEN)    [<ffff82d04022de59>] F timer.c#timer_softirq_action+0x71/0x275
> (XEN)    [<ffff82d04022b53e>] F softirq.c#__do_softirq+0x94/0xbe
> (XEN)    [<ffff82d04022b5d1>] F do_softirq+0x13/0x15
> (XEN)    [<ffff82d0402fcd15>] F domain.c#idle_loop+0x8d/0xe6

... this looks to be the issuing CPU. What I don't understand is why we
are in _spin_unlock_irqrestore() here, called out of dbc_uart_poll().

Btw - was any/all of this with or without the series here applied?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 08:46:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 08:46:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681406.1060164 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raXNY-0007sZ-5p; Thu, 15 Feb 2024 08:46:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681406.1060164; Thu, 15 Feb 2024 08:46:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raXNY-0007sS-1P; Thu, 15 Feb 2024 08:46:00 +0000
Received: by outflank-mailman (input) for mailman id 681406;
 Thu, 15 Feb 2024 08:45:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raXNW-0007sM-QP
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 08:45:58 +0000
Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com
 [2a00:1450:4864:20::333])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a28a0528-cbde-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 09:45:56 +0100 (CET)
Received: by mail-wm1-x333.google.com with SMTP id
 5b1f17b1804b1-411fc50e4e9so4697785e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 00:45:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s7-20020a5d69c7000000b0033cf453f2bbsm1051894wrw.35.2024.02.15.00.45.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 00:45:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a28a0528-cbde-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707986756; x=1708591556; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=ZCoAMOp9fEFTVJ/m8pBXny7hymQmiLKidoko2xjvH6g=;
        b=F7ZLH9Peg41Pdc28H/kdVSPy8Ru7nk4NYJ+pL0lqlbnDUy2oYrnOH3JTnzyaqeieWl
         +TDRTKNK5H29Y0LG3WBUxl7+ncPeS6cLHBEPTlzT8wfN4sVqKqvQ/kwLqJn8BBURlR/l
         jkkbErpri8uflBeOmmZofUEZ6DNrF8+/5ylP5regLppypwK4yb/J199v7GBDC5DAMztt
         JIaHassuIiTFgYEqAWlYcYTXpaxWPJCT6bMnF3SBPkxsPs3eiC9qcxfG5MWNu1WGvZ34
         N2xvAzoGHFJcJiGR/iEzjaxSrlrTZ94BbDcMLcdzCTDOK9VQebSCMS0H98e9fWL1QoIN
         sDDw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707986756; x=1708591556;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=ZCoAMOp9fEFTVJ/m8pBXny7hymQmiLKidoko2xjvH6g=;
        b=F9s+2EsaCayssT9Ns34MloRPYpq447TiT+S09pwpFQN205Lz+Htujzq5Tj19k1OA9N
         LHk9ozKMI118tnRxm+5+2d2Q8nS3XPvDoLwti8IEliF6v35fViTNd06F5OF7z3t86sk4
         a+Up6VjfU22lbNo59nO/grPJEt2AQQZd61oHdebT8DAjuwnYlYMBYXs7u9QcLbvkd8hz
         PQgxoBlJMBNoJXCXatKLr/bt/OsHmedqvPdQuNY9bsXBpX3dfiLw91eFn0qEcXcCKvu0
         n0cKuGBgKcGQPhn6L0JObnhJMrH2fuOO9J/MV/xBrRdgaRiUFskPbvCQMfbLqmR7j/Ha
         7Tmg==
X-Gm-Message-State: AOJu0YzHwD/2lboWJxMCS9T/BNZncKLE3qCQPA6ARW9TN2R047LEE9iz
	izef8+Y1ot5OUmWRnLoYlDLF0VTxxsp8yLxkrkD4X/V1gKAYShTrm+E9XadVYmASgpzPFZQaBzk
	=
X-Google-Smtp-Source: AGHT+IENqY9eMdOvS9qPBmPFYGfhQKZ0mvrIkIPsWF76XX13onNr2wfhId11Xw/KBFkS9sPv7NxRgw==
X-Received: by 2002:a05:600c:4706:b0:410:27e7:4c5 with SMTP id v6-20020a05600c470600b0041027e704c5mr1040756wmo.29.1707986756104;
        Thu, 15 Feb 2024 00:45:56 -0800 (PST)
Message-ID: <5d8f8dac-f2df-4760-8130-dec97e486578@suse.com>
Date: Thu, 15 Feb 2024 09:45:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Marek Marczykowski <marmarek@invisiblethingslab.com>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <893be03d-22cc-4b8c-8a54-6479961c5aa2@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 05.02.2024 14:27, Jan Beulich wrote:
> In preparation of dropping the register parameters from
> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
> register state needs making available another way for the few key
> handlers which need it. Fake IRQ-like state.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsistent with
> other console poll functions we have, and it's unclear whether that's
> actually generally correct.

It occurs to me that due to this behavior, ...

> --- a/xen/drivers/char/xhci-dbc.c
> +++ b/xen/drivers/char/xhci-dbc.c
> @@ -1164,6 +1164,7 @@ static void cf_check dbc_uart_poll(void
>      struct dbc_uart *uart = port->uart;
>      struct dbc *dbc = &uart->dbc;
>      unsigned long flags = 0;
> +    struct cpu_user_regs *old_regs;
>  
>      if ( spin_trylock_irqsave(&port->tx_lock, flags) )
>      {
> @@ -1175,10 +1176,15 @@ static void cf_check dbc_uart_poll(void
>          spin_unlock_irqrestore(&port->tx_lock, flags);
>      }
>  
> +    /* Mimic interrupt context. */
> +    old_regs = set_irq_regs(guest_cpu_user_regs());
> +
>      while ( dbc_work_ring_size(&dbc->dbc_iwork) )
>          serial_rx_interrupt(port, guest_cpu_user_regs());
>  
>      serial_tx_interrupt(port, guest_cpu_user_regs());
> +
> +    set_irq_regs(old_regs);
>      set_timer(&uart->timer, NOW() + MICROSECS(DBC_POLL_INTERVAL));
>  }
>  
> 

... this ought to be unnecessary, considering what dump_registers()
is changed to in patch 2.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 09:02:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 09:02:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681414.1060173 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raXdj-0002ll-Ew; Thu, 15 Feb 2024 09:02:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681414.1060173; Thu, 15 Feb 2024 09:02:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raXdj-0002le-B1; Thu, 15 Feb 2024 09:02:43 +0000
Received: by outflank-mailman (input) for mailman id 681414;
 Thu, 15 Feb 2024 09:02:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DeW9=JY=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1raXdi-0002lY-41
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 09:02:42 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f70152a9-cbe0-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 10:02:38 +0100 (CET)
Received: from BL1PR13CA0209.namprd13.prod.outlook.com (2603:10b6:208:2be::34)
 by CH3PR12MB9343.namprd12.prod.outlook.com (2603:10b6:610:1c0::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.24; Thu, 15 Feb
 2024 09:02:32 +0000
Received: from BL6PEPF0001AB76.namprd02.prod.outlook.com
 (2603:10b6:208:2be:cafe::ce) by BL1PR13CA0209.outlook.office365.com
 (2603:10b6:208:2be::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.10 via Frontend
 Transport; Thu, 15 Feb 2024 09:02:32 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL6PEPF0001AB76.mail.protection.outlook.com (10.167.242.169) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 15 Feb 2024 09:02:31 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 15 Feb
 2024 03:02:31 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 15 Feb
 2024 03:02:31 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 15 Feb 2024 03:02:29 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f70152a9-cbe0-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=RV7Yi9u81PozEB1sph3v7r9+/uP09Eyy+/pkpwr/fCz4rV0cQ9GcA08PifpDFqRpoKjU+TjC4FUEuF3POjdjljyZMQoXS94kECgcvAUqu0bxcd1YeS/Fv5vvX7TYypbl7YkeYm+3s39W9wd5HzRmm5I8ebitGKYYiCEc6gMldmUDNejqC0SMTzy8pCPTT+pfUzf9dmOue96yyFXCKyt9EMKSILf4BX65x9A+oQGGpubCplfNrpC0Lz3Sgyp+gyfjomWV1qiOHvTyFwEG808u1yN6SzhZ7toVTwShgI7wRe9iZuB8k9WXNH6bUyF1MWmmQUnBCuTVi0XpW9TLFM2QJA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=7SMTQcb7Cy9OfOmmMhTCyRGm7XoY2iIL/SVb7fVFy+o=;
 b=Qcbjqv9wVgsle8uBCJPQA7i/Tc2liGQ2TvsP5CyWD1mcsxBbWvo8Q4B2Y8tDeBvYAMAympbi36oSQzib7oMB6f5mjOCFO3f7yKwQSMH8Z+ne4xrIRT07wGWo8WSd3ZBi3RViwuhrJHAkn6cV+Q/3Ap3gWFVE+QrQzNFq+ikwCC18en5q3n/VHWts7T6l2knjYsv5SbvSibEP2OtYJ8FfnT2WqOeL2bUvH4Y9C3AZPlAaJ6TMv0LfawR0u7PLP28XryLxjDwICdDBKqbutcnXWA0ABiIGsDH6Kmhlg+h6oHcAJGtsZBw7xW+3zPGpfg1k6psXLPXDw8O05zyti6clNA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=actia.se smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7SMTQcb7Cy9OfOmmMhTCyRGm7XoY2iIL/SVb7fVFy+o=;
 b=F3Qi3PK6eGPF39PnImzF8G29J7QbiY7Gqi9vRiDLpYhv1lp8LIQyJqmEBTE817auUvdAsHVKvdfBlAhFRJMuWeJyh6NbOhf+exMUwE/F6hSmHQ5mJf/zbIhzAB5O5+Rx5nigHMeU35NVLmCZ1OdbNM/ql5CLs8L9qxZDorvb+FM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <3fe044c4-5e60-4e11-a99e-38534610640b@amd.com>
Date: Thu, 15 Feb 2024 10:02:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/2] xen/arm: Add imx8q{m,x} platform glue
To: John Ernberg <john.ernberg@actia.se>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Peng
 Fan" <peng.fan@nxp.com>, Jonas Blixt <jonas.blixt@actia.se>
References: <20240214160644.3418228-1-john.ernberg@actia.se>
 <20240214160644.3418228-2-john.ernberg@actia.se>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240214160644.3418228-2-john.ernberg@actia.se>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB76:EE_|CH3PR12MB9343:EE_
X-MS-Office365-Filtering-Correlation-Id: ea5d16c2-dd65-4af7-1e8f-08dc2e04d7fe
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	fCC6vWkXuZigwe4HrrjzJ97Spf7d4QR+3Fu3+yBGGsqkLy7PgrF9qIqsB9H9nwbnnGBYU4AoYR+Sn3mBTG9CYyHKvKcgoYW2pOCYFIvg34G320j3x21P5iCGYxk5Y4Lmq3u4d9r7HhCCDMsv02m9Y/jv5ocfJ9DloGYzLpY5fg9Fl7ZGg8VS4gUFq4vCHCmF0MOokfpnXJdjqP7gVTkrrRx3o9m7FIDeJx2YCqhLr2br66AmxMfOtudYXcOY4ouU6x3RZaxPF1+cVOSa+Xg8rCUr1ZzGyKmzV/5XABadgjpvwZYrjiz6G7RwnRGnT3gEwnxUCjCH/9kEnVo5A+GpMqjeV9MprZN0YpoA5yuYJetnc7XBTAnd9mwveCkC+o0OC2d5M1KfGnwzdlFBZziXeOSz1w0QjrJwCs1P8c/BcAL4HD1adiueYbvKiiTE1hk9eRBhez34Tftjfez65hAuZk3WZQ2MGdAz/jF3RhWszwL/0qrvrmGR0asd5fn/QbalBGUrRltTQe+qBlF04RCHECFDClzctw14ibMvU0MQ7ommNugsRobcfqLeNOayPWJP+h3gVIKr47jtAzGX+iyUbQtSmFvUcIsKKNbYh6zuKcruuJb9H64v1T40OVft4zBQmC075m26DEVuFyB6X6q2oQ==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(39860400002)(376002)(136003)(230273577357003)(230922051799003)(451199024)(36860700004)(186009)(1800799012)(82310400011)(64100799003)(40470700004)(46966006)(36756003)(110136005)(8936002)(26005)(2616005)(426003)(336012)(4326008)(70206006)(83380400001)(8676002)(53546011)(966005)(54906003)(5660300002)(70586007)(31696002)(316002)(86362001)(82740400003)(81166007)(356005)(41300700001)(16576012)(478600001)(31686004)(2906002)(44832011);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 09:02:31.6395
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ea5d16c2-dd65-4af7-1e8f-08dc2e04d7fe
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB76.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9343

Hi,

On 14/02/2024 17:06, John Ernberg wrote:
> 
> 
> When using Linux for dom0 there are a bunch of drivers that need to do SMC
> SIP calls into the firmware to enable certain hardware bits like the
> watchdog.
> 
> Provide a basic platform glue that implements the needed SMC forwarding.
> 
> The format of these calls are as follows:
>  - reg 0: service ID
>  - reg 1: function ID
>  remaining regs: args
> 
> For now we only allow Dom0 to make these calls as they are all managing
> hardware. There is no specification for these SIP calls, the IDs and names
> have been extracted from the upstream linux kernel and the vendor kernel.
> 
> Most of the SIP calls are only available for the iMX8M series of SoCs, so
> they are easy to reject and they need to be revisited when iMX8M series
> support is added.
Given that you named your driver *qm and you search for dt compatible *qm, *qxp, does it really
make sense to add SIP calls for iMX8M?

> 
> From the other calls we can reject CPUFREQ because Dom0 cannot make an
> informed decision regarding CPU frequency scaling, WAKEUP_SRC is to wake
> up from suspend, which Xen doesn't support at this time.
> 
> This leaves the TIME SIP and the OTP_WRITE SIP, which for now are allowed
> to Dom0.
> 
> NOTE: This code is based on code found in NXP Xen tree located here:
> https://github.com/nxp-imx/imx-xen/blob/lf-5.10.y_4.13/xen/arch/arm/platforms/imx8qm.c
> 
> Signed-off-by: Peng Fan <peng.fan@nxp.com>
> [jernberg: Add SIP call filtering]
> Signed-off-by: John Ernberg <john.ernberg@actia.se>
> 
> ---
> 
> v2:
>  - Reword the commit message to be a bit clearer
>  - Include the link previously added as a context note to the commit message (Julien Grall)
>  - Add Pengs signed off (Julien Grall, Peng Fan)
>  - Add basic SIP call filter (Julien Grall)
>  - Expand the commit message a whole bunch because of the changes to the code
> ---
>  xen/arch/arm/platforms/Makefile |   1 +
>  xen/arch/arm/platforms/imx8qm.c | 143 ++++++++++++++++++++++++++++++++
>  2 files changed, 144 insertions(+)
>  create mode 100644 xen/arch/arm/platforms/imx8qm.c
> 
> diff --git a/xen/arch/arm/platforms/Makefile b/xen/arch/arm/platforms/Makefile
> index 8632f4115f..bec6e55d1f 100644
> --- a/xen/arch/arm/platforms/Makefile
> +++ b/xen/arch/arm/platforms/Makefile
> @@ -9,5 +9,6 @@ obj-$(CONFIG_ALL_PLAT)   += sunxi.o
>  obj-$(CONFIG_ALL64_PLAT) += thunderx.o
>  obj-$(CONFIG_ALL64_PLAT) += xgene-storm.o
>  obj-$(CONFIG_ALL64_PLAT) += brcm-raspberry-pi.o
> +obj-$(CONFIG_ALL64_PLAT) += imx8qm.o
>  obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp.o
>  obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp-eemi.o
> diff --git a/xen/arch/arm/platforms/imx8qm.c b/xen/arch/arm/platforms/imx8qm.c
> new file mode 100644
> index 0000000000..4515c75935
> --- /dev/null
> +++ b/xen/arch/arm/platforms/imx8qm.c
> @@ -0,0 +1,143 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * xen/arch/arm/platforms/imx8qm.c
> + *
> + * i.MX 8QM setup
> + *
> + * Copyright (c) 2016 Freescale Inc.
> + * Copyright 2018-2019 NXP
> + *
> + *
> + * Peng Fan <peng.fan@nxp.com>
> + */
> +
> +#include <xen/sched.h>
> +#include <asm/platform.h>
> +#include <asm/smccc.h>
> +
> +static const char * const imx8qm_dt_compat[] __initconst =
> +{
> +    "fsl,imx8qm",
> +    "fsl,imx8qxp",
> +    NULL
> +};
> +
> +#define IMX_SIP_GPC          0xC2000000
NIT: A matter of personal opinion but all the Arm64 SIP services starts with 0xc2000000,
so you could just define a function id i.e. 1, 2, ... and use a macro similar to EEMI_FID(fid).
This is just a suggestion.

> +#define IMX_SIP_CPUFREQ      0xC2000001
> +#define IMX_SIP_TIME         0xC2000002
> +#define IMX_SIP_DDR_DVFS     0xC2000004
> +#define IMX_SIP_SRC          0xC2000005
> +#define IMX_SIP_GET_SOC_INFO 0xC2000006
> +#define IMX_SIP_NOC          0xC2000008
> +#define IMX_SIP_WAKEUP_SRC   0xC2000009
> +#define IMX_SIP_OTP_WRITE    0xC200000B
Looking at ATF, for QM,QXP there are also:
BUILDINFO 0xC2000003
OTP_READ  0xC200000A
SET_TEMP  0xC200000C

> +
> +#define IMX_SIP_TIME_F_RTC_SET_TIME     0x00
> +#define IMX_SIP_TIME_F_WDOG_START       0x01
> +#define IMX_SIP_TIME_F_WDOG_STOP        0x02
> +#define IMX_SIP_TIME_F_WDOG_SET_ACT     0x03
> +#define IMX_SIP_TIME_F_WDOG_PING        0x04
> +#define IMX_SIP_TIME_F_WDOG_SET_TIMEOUT 0x05
> +#define IMX_SIP_TIME_F_WDOG_GET_STAT    0x06
> +#define IMX_SIP_TIME_F_WDOG_SET_PRETIME 0x07
> +
> +static bool imx8qm_is_sip_time_call_ok(uint32_t function_id)
> +{
> +    switch ( function_id )
> +    {
> +    case IMX_SIP_TIME_F_RTC_SET_TIME:
> +        return true;
> +    case IMX_SIP_TIME_F_WDOG_START:
> +    case IMX_SIP_TIME_F_WDOG_STOP:
> +    case IMX_SIP_TIME_F_WDOG_SET_ACT:
> +    case IMX_SIP_TIME_F_WDOG_PING:
> +    case IMX_SIP_TIME_F_WDOG_SET_TIMEOUT:
> +    case IMX_SIP_TIME_F_WDOG_GET_STAT:
> +    case IMX_SIP_TIME_F_WDOG_SET_PRETIME:
> +        return true;
> +    default:
> +        printk(XENLOG_WARNING "imx8qm: smc: time: Unknown function id %x\n", function_id);
80 chars limit, move argument list to the next line

> +        return false;
> +    }
> +}
> +
> +static bool imx8qm_smc(struct cpu_user_regs *regs)
> +{
> +    uint32_t service_id = get_user_reg(regs, 0);
In SMCCC naming convention, W0 is called function identifier. Instead you call X1 function_id
which is a bit misleading.

> +    uint32_t function_id = get_user_reg(regs, 1);
> +    struct arm_smccc_res res;
> +
> +    if ( !cpus_have_const_cap(ARM_SMCCC_1_1) )
> +    {
> +        printk_once(XENLOG_WARNING "no SMCCC 1.1 support. Disabling firmware calls\n");
80 chars limit, move string to the next line

> +
> +        return false;
> +    }
> +
> +    /* Only hardware domain may use the SIP calls */
> +    if ( !is_hardware_domain(current->domain) )
> +    {
> +        gprintk(XENLOG_WARNING, "imx8qm: smc: No access\n");
> +        return false;
> +    }
> +
> +    switch ( service_id )
> +    {
> +    case IMX_SIP_GPC: /* Only available on imx8m series */
If we decide to keep iMX8M FIDs, I think adding comments on top of a case would result in a more readable code.

> +        return false;
> +    case IMX_SIP_CPUFREQ: /* Dom0 can't take any informed descision here */
> +        return false;
> +    case IMX_SIP_TIME:
> +        if ( imx8qm_is_sip_time_call_ok(function_id) )
> +            goto allow_call;
> +        return false;
> +    case IMX_SIP_DDR_DVFS: /* Only available on imx8m series */
> +        return false;
> +    case IMX_SIP_SRC: /* Only available on imx8m series */
> +        return false;
> +    case IMX_SIP_GET_SOC_INFO: /* Only available on imx8m series */
> +        return false;
> +    case IMX_SIP_NOC: /* Only available on imx8m series */
> +        return false;
> +    case IMX_SIP_WAKEUP_SRC: /* Xen doesn't have suspend support */
> +        return false;
> +    case IMX_SIP_OTP_WRITE:
> +        /* function_id is the fuse number, no sensible check possible */
> +        goto allow_call;
> +    default:
> +        printk(XENLOG_WARNING "imx8qm: smc: Unknown service id %x\n", service_id);
80 chars limit, move argument list to the next line

> +        return false;
> +    }
> +
> +allow_call:
labels need to be indented with one space

> +    arm_smccc_1_1_smc(service_id,
> +                      function_id,
> +                      get_user_reg(regs, 2),
> +                      get_user_reg(regs, 3),
> +                      get_user_reg(regs, 4),
> +                      get_user_reg(regs, 5),
> +                      get_user_reg(regs, 6),
> +                      get_user_reg(regs, 7),
> +                      &res);
> +
> +    set_user_reg(regs, 0, res.a0);
> +    set_user_reg(regs, 1, res.a1);
> +    set_user_reg(regs, 2, res.a2);
> +    set_user_reg(regs, 3, res.a3);
> +
> +    return true;
> +}
> +
> +PLATFORM_START(imx8qm, "i.MX 8")
Shouldn't it be i.MX 8Q{M,XP} ?

~Michal


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 09:27:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 09:27:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681427.1060182 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raY0y-00060k-8m; Thu, 15 Feb 2024 09:26:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681427.1060182; Thu, 15 Feb 2024 09:26:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raY0y-00060d-6J; Thu, 15 Feb 2024 09:26:44 +0000
Received: by outflank-mailman (input) for mailman id 681427;
 Thu, 15 Feb 2024 09:26:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raY0x-00060X-5i
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 09:26:43 +0000
Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com
 [2a00:1450:4864:20::22b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 538b633d-cbe4-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 10:26:41 +0100 (CET)
Received: by mail-lj1-x22b.google.com with SMTP id
 38308e7fff4ca-2d0e5212559so7444391fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 01:26:41 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f14-20020a5d4dce000000b0033cf4a10359sm1170044wru.61.2024.02.15.01.26.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 01:26:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 538b633d-cbe4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707989200; x=1708594000; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SKGB4E1geMsj83tapJiTWzcJMlLUioOn2ukajXeGsBE=;
        b=c9YL7BLbvWxC02PWHsCZwKrjepOMYaEffeTOMOeKJDt7KCeX53R0qcnBZ9bZ7lIvuZ
         NzmvEf1V3WU8xZMNXIsuPNjHQsDkg8xO3ZXR+B+N9ujie2kXiv8j8Cxr2xJmv6c2vtZe
         00hHc/9Be97rnCA3+2mN1vAZiILcznMUmI9Ox10uHcI2vlKHzGRQ2JFymd3748pS0kib
         nEZOZoHVgOrFOTu9hgLeQMRqTORmdFkb+OyZ++NPAFwX77esPJ0dG5kvciQ2s7Wcp+Bq
         SiQqD60VgSGcBcUGLew8PwE0HrKRr4Ssc6Z+XKtN0WQf/zI5xloe2sdwo5wVjNd0E9Su
         H8IQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707989200; x=1708594000;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SKGB4E1geMsj83tapJiTWzcJMlLUioOn2ukajXeGsBE=;
        b=gR1LP4mYJeUAtvAV51w/+vkC/wb5+y9yMjqONR7lqVCxz1hUfeB2WVi4TaHeT1bpm7
         CEf7HGrgCPOOgnMM7I9LZ2TZVg/MPK8capYCJQkcxEASyamsdzfIEI4TRv1+V6SUd4iZ
         MkGN/6n5CRf8s6k0d9ol8s+KmtpGsamPo84GB59SnPWqcneVbRcWjUlWHEJ0S25Ztyhj
         WQJaXgQWgfiZOPnwp5KIjRoKqRkez9zLC5LHa84Y1ku6S5U6wVUY9FzJvBHRWmZrEmpn
         nyruZ098zDjt+obJyr8z5Xh0IyGq6MjKtM9c7Bt+KbkuaN8bB0p4Lr0Qab8uYQ5C2tr2
         9VnQ==
X-Forwarded-Encrypted: i=1; AJvYcCVmN79SOXTiHupYn0Sv3zVtoMJm2saYksGKHYEqAaL/B4wQUHGGnQwDb3yT3PW7PanaVgNrh+mhzkV+gTXLHg4t1RTm28sFWzMQemYDtq8=
X-Gm-Message-State: AOJu0YwvkiT4oax68daBaW5fAKv06zzhjp1VIVqzTLgHX904tAUg/pen
	P37eCf/7Yh+9pFdL01MvPn8N8BarczGIrYmCQC0AV+CiXxYGn32J7dU7Tor7ZQ==
X-Google-Smtp-Source: AGHT+IGkkU2WGp8QMza03CzMMSRxgbLhfVSFskGhgZqTzLevtx+f8d9TKxgH71j8xfnv8B2609J0zg==
X-Received: by 2002:a2e:8ecf:0:b0:2d0:9cbf:f389 with SMTP id e15-20020a2e8ecf000000b002d09cbff389mr836230ljl.26.1707989200505;
        Thu, 15 Feb 2024 01:26:40 -0800 (PST)
Message-ID: <0923d293-38ed-40ba-8bd8-b37c2b5d22b9@suse.com>
Date: Thu, 15 Feb 2024 10:26:39 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>,
 Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
 <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
 <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
 <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com>
 <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
 <alpine.DEB.2.22.394.2402141600010.1925432@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402141600010.1925432@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.02.2024 01:05, Stefano Stabellini wrote:
> I don't think is a good idea to add further changes to this patch. I
> think we should go ahead with it as-is.

I didn't suggest adding anything right here; there may want/need to be new
prereq-s, though. I'd like to make sure that we don't (significantly)
regress in terms of being able to diagnose programming mistakes.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 09:30:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 09:30:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681432.1060193 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raY4j-0007ik-S5; Thu, 15 Feb 2024 09:30:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681432.1060193; Thu, 15 Feb 2024 09:30:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raY4j-0007id-PB; Thu, 15 Feb 2024 09:30:37 +0000
Received: by outflank-mailman (input) for mailman id 681432;
 Thu, 15 Feb 2024 09:30:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wY2D=JY=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raY4i-0007iX-AK
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 09:30:36 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id df411e21-cbe4-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 10:30:35 +0100 (CET)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2d09cf00214so8429781fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 01:30:35 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 a23-20020a05620a125700b007815c25b32bsm448192qkl.35.2024.02.15.01.30.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 01:30:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df411e21-cbe4-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707989434; x=1708594234; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=cwAnFuF7y1xH9XbRd7r9Y77OMIBU9GS3GWd0lx+S+mg=;
        b=ZDRC4qxEooOOPaLRCRVoBHkXR/DbRiEV7uuIULi6pRH7G1KaqWAUxdOpTO8szndo48
         0EOwUaDepWIH0pdyLjZZ1LbiWsqBZ+5EQnO7Jdq4Cdwk6aVfHRu4ids7O7kKDeAlvxBm
         /qRXBlajVji8OfQwx++CxhmqTCs7/kre+3Y+Y=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707989434; x=1708594234;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=cwAnFuF7y1xH9XbRd7r9Y77OMIBU9GS3GWd0lx+S+mg=;
        b=mr2rNvN53jAIwDZtmjOA3mLP/VTP5RPe0+gd7Quoxg9qyxVgBpWge+esmS3VpJ3/Je
         7U400MJNwnhY9UKmDAv9qFIvpf0mDdJM3VgUcdcGD9jJzuqolSU+G2Dr6JiZNIHIz2wY
         ocENINswPSvWUJ2eHq0Gna0uRPI/F9DJd4GkarqS/+q4xcqwMaeF7i/Zblfom4nLef+/
         81At6DnMWnHZkJxrnCHzLMHwbcObaCj9zloiIPfQ3qLcOM0fZ+D+43QxZo3TBoaPBhu2
         MAN/VP2fFFxt3vCqD8/wOJbmHHBEKbDLBLQcokA0V7/hskVn4nnTWHWfxZ2RNFXPRJhA
         t09Q==
X-Gm-Message-State: AOJu0YzXuQxqRPziHEtnbo0bV0DBv3jubgHcGOh30QEquqlva+yN4zvl
	88j2HE2iWys+4/iWOxUdSxJvemvWP6bNQwOf1sSG8onwOpWchflZVxi1cpQO7oK08OgRslUegbK
	r
X-Google-Smtp-Source: AGHT+IFErRneX9Wd5izZFxOx+B7r/2/HoSPI1fndZR8FtA6QVuKvlO3Nt3VsbR2QUOTZn3REYuzrHQ==
X-Received: by 2002:a2e:b6d1:0:b0:2d0:b6fc:addf with SMTP id m17-20020a2eb6d1000000b002d0b6fcaddfmr810088ljo.3.1707989434304;
        Thu, 15 Feb 2024 01:30:34 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: anthony.perard@citrix.com,
	Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] build/xen: fail to rebuild if Kconfig fails
Date: Thu, 15 Feb 2024 10:30:02 +0100
Message-ID: <20240215093002.23527-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When doing a rebuild with an xen/include/config/auto.conf already present in
the tree, failures from Kconfig are ignored since the target is present:

gmake -C xen install
gmake[1]: Entering directory '/root/src/xen/xen'
tools/kconfig/conf  --syncconfig Kconfig
common/Kconfig:2: syntax error
common/Kconfig:1: invalid statement
gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
  UPD     include/xen/compile.h
 Xen 4.19-unstable
gmake[3]: Nothing to be done for 'all'.
gmake[2]: 'arch/x86/include/asm/asm-offsets.h' is up to date.

Ultimately leading to a successful build despite the Kconfig error.

Fix this by first removing xen/include/config/auto.conf before attempting to
regenerate, and then also make xen/include/config/auto.conf a hard dependency
of the build process (ie: drop the leading '-') and reordering so the .cmd
target is executed before including the configuration file.

This leads to the build properly failing if the config file cannot be
re-generated:

gmake -C xen install
gmake[1]: Entering directory '/root/src/xen/xen'
rm -rf include/config/auto.conf
tools/kconfig/conf  --syncconfig Kconfig
common/Kconfig:2: syntax error
common/Kconfig:1: invalid statement
gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
gmake[1]: *** [Makefile:379: include/config/auto.conf] Error 2
gmake[1]: Leaving directory '/root/src/xen/xen'
gmake: *** [Makefile:143: install-xen] Error 2

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Maybe there's a better way for dealing with all this, my makefile foo is very
limited.
---
 xen/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/Makefile b/xen/Makefile
index 21832d640225..7e6860a58a1d 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -358,10 +358,10 @@ config: tools_fixdep outputmakefile FORCE
 else # !config-build
 
 ifeq ($(need-config),y)
--include include/config/auto.conf
 # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
 # changes are detected.
 -include include/config/auto.conf.cmd
+include include/config/auto.conf
 
 # Allow people to just run `make` as before and not force them to configure
 # Only run defconfig if $(KCONFIG_CONFIG) is missing
@@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
 # This exploits the 'multi-target pattern rule' trick.
 # The syncconfig should be executed only once to make all the targets.
 include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
+	rm -rf include/config/$*.conf
 	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
 
 ifeq ($(CONFIG_DEBUG),y)
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 09:32:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 09:32:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681437.1060203 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raY6L-0008J6-7S; Thu, 15 Feb 2024 09:32:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681437.1060203; Thu, 15 Feb 2024 09:32:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raY6L-0008Iz-4K; Thu, 15 Feb 2024 09:32:17 +0000
Received: by outflank-mailman (input) for mailman id 681437;
 Thu, 15 Feb 2024 09:32:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raY6K-0008Ir-2c
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 09:32:16 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1aabbc99-cbe5-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 10:32:15 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-41102f140b4so5662955e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 01:32:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 j4-20020a05600c1c0400b004120537210esm2340835wms.46.2024.02.15.01.32.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 01:32:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1aabbc99-cbe5-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707989534; x=1708594334; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Xq72nU/U0opIMf+1Ykh4dz/2yLNFr6Im8X+Xd6LpF04=;
        b=Y2N/F513VL80mGr1E+tjHNikWaeDS5PIyHKmA1TGSUNaBdySlRYgl1yE44LtqkWKTB
         f1NZvN4tVmT87AzW65Y0eKsqu/NXn6HPZYRotddSx81psH4IQ5FNU5277voIFNEq7eCa
         kn59NIGD5MEoxrnPkNcRE06Pjvm7OHEOYk4fSFPPhAVWMvCa339PrbQp87iBDpEHuLpP
         zraAsaA01VVuc2SlqKiYEiTUDZzRsuPbsw5LYBenEmSbRAZQ1ExZhcyAh8wXn/IK4jZ5
         fOCBTUIbSn98EAcD1OX9G1eQ0rPU/2lBSC89QjfFXHi26E/nfih7M6tupwYRWxbL2LtN
         +zTg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707989534; x=1708594334;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Xq72nU/U0opIMf+1Ykh4dz/2yLNFr6Im8X+Xd6LpF04=;
        b=WtpkCi/VTZfMknSb32NjkvkSOzAljkmwaeWEnU+wtyLMRGco83i53UK0Ry0JStru3+
         L4t678B9hVcG4tM0ls46Z9wNKnCtbONOL2IAkit8KBWjlgQibPxubA11dmbA+gBB338v
         Mfkh3C6rspGT3nCAKDa1MTjy64I0yGoBMUxVKTP9T52fq7Iwn2QmI0PBE3Iq/j0aOGT4
         Cy98ZcmS3PzupB+Wf2PG86LDmAw6rUdJrGzLPsQFWuTeUsUZMdQ5o26c+3bSi3TFGj8X
         bVnPQyccHX2coqsNZrhV+xOA6eAhzxbRcv/1oIZyZg1HPYpfyi92FbeBuvmUXubOtAB9
         NbyQ==
X-Forwarded-Encrypted: i=1; AJvYcCXBebZck0i09shl6D4W7vGnNPlLZnF1KSZ1rkO/Y1AUhoTZNE8X/nCA/P8i9s97fTfKRRlAKvTe8MHhdbsqCprrInEtSzZ4DKVNZkNPOHY=
X-Gm-Message-State: AOJu0YwW0avE5yNH3MPIf5UfJbMUatQFdXdHECmKy24BaX2z6Iddwg2L
	3XUEIKr7DrNcRJCUohR5+yWDttsGe07LFnIDH0SQwQjiqR4nOdsXDFLUGy1dmQ==
X-Google-Smtp-Source: AGHT+IGx4wtbcvU49SXC7ZsA1MnVjg3ClBXEBMHxdOGRSlvNcvKVK1pVYs0FNuQtSknzyPRB9VUaEQ==
X-Received: by 2002:a05:600c:1c9b:b0:410:cc74:cb9c with SMTP id k27-20020a05600c1c9b00b00410cc74cb9cmr929474wms.25.1707989534701;
        Thu, 15 Feb 2024 01:32:14 -0800 (PST)
Message-ID: <8b4ca5c8-8d4f-41e8-b544-99909298df63@suse.com>
Date: Thu, 15 Feb 2024 10:32:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2] build: Replace `which` with `command -v`
Content-Language: en-US
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240214180039.57527-1-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240214180039.57527-1-anthony.perard@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 14.02.2024 19:00, Anthony PERARD wrote:
> The `which` command is not standard, may not exist on the build host,
> or may not behave as expected by the build system. It is recommended
> to use `command -v` to find out if a command exist and have its path,
> and it's part of a POSIX shell standard (at least, it seems to be
> mandatory since IEEE Std 1003.1-2008, but was optional before).
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Thu Feb 15 09:42:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 09:42:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681448.1060213 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYG7-0001la-3z; Thu, 15 Feb 2024 09:42:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681448.1060213; Thu, 15 Feb 2024 09:42:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYG7-0001lT-0X; Thu, 15 Feb 2024 09:42:23 +0000
Received: by outflank-mailman (input) for mailman id 681448;
 Thu, 15 Feb 2024 09:42:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raYG5-0001lN-Uc
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 09:42:22 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 82ef9500-cbe6-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 10:42:19 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id DB46321E57;
 Thu, 15 Feb 2024 09:42:18 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id AB88B13A82;
 Thu, 15 Feb 2024 09:42:18 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 33ROKHrczWUSHgAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 09:42:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82ef9500-cbe6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707990139; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=Cj6pyE1GOQTLeuJaj/AA8G4xHFwLkvXX3uxItvYzY70=;
	b=ptMv9f9mFBY4nrtkIHDn3aWPUfw3uIsC5MN019rwPuM5rwOhYPmLovY8TX89hVMxB8mw7S
	Ry3mYbFy8Xk57yklsZEk/Ahot6MRr6jBkP8HIkizewfA9dgtj6UeVhdXthdiUmC5ZDwTrU
	YADPOEd7+XDsTIhDET7nbi3bMJTydec=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1707990138; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=Cj6pyE1GOQTLeuJaj/AA8G4xHFwLkvXX3uxItvYzY70=;
	b=B91IJ2GWSeduvClHEx61LZ9CoOiNXNpTcmhZ1A7cB9GbiIDAis6Kqf9PXryuK/NeO/L8Tk
	6EHd4JN8CEHa5bZSOcmOknFWgcKKzAAlq+P+GYu9elm0qKUkgwdDlvx309RBt6NWGbV/ep
	IzGghHxLwMncMa5B3g+7hWf479HQxB8=
Message-ID: <3261b14b-bce1-4aa5-bb61-a25930fca133@suse.com>
Date: Thu, 15 Feb 2024 10:42:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 15/21] tools/xl: support new 9pfs backend xen_9pfsd
Content-Language: en-US
To: xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jason Andryuk <jandryuk@gmail.com>
References: <20240215065541.21067-1-jgross@suse.com>
 <20240215065541.21067-16-jgross@suse.com>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <20240215065541.21067-16-jgross@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------p0I55L4PbTadoXKUmMCzhEap"
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=B91IJ2GW
X-Spamd-Result: default: False [-4.20 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 TO_DN_SOME(0.00)[];
	 HAS_ATTACHMENT(0.00)[];
	 MIME_BASE64_TEXT_BOGUS(1.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 MID_RHS_MATCH_FROM(0.00)[];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: DB46321E57
X-Spam-Level: 
X-Spam-Score: -4.20
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------p0I55L4PbTadoXKUmMCzhEap
Content-Type: multipart/mixed; boundary="------------4sYac0mFmWSHsk0WeNADZmYl";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jason Andryuk <jandryuk@gmail.com>
Message-ID: <3261b14b-bce1-4aa5-bb61-a25930fca133@suse.com>
Subject: Re: [PATCH v6 15/21] tools/xl: support new 9pfs backend xen_9pfsd
References: <20240215065541.21067-1-jgross@suse.com>
 <20240215065541.21067-16-jgross@suse.com>
In-Reply-To: <20240215065541.21067-16-jgross@suse.com>

--------------4sYac0mFmWSHsk0WeNADZmYl
Content-Type: multipart/mixed; boundary="------------rG5Sd0bq0obQ5zm70O0ZQ86d"

--------------rG5Sd0bq0obQ5zm70O0ZQ86d
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMTUuMDIuMjQgMDc6NTUsIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+IEFkZCBzdXBwb3J0
IGZvciB0aGUgbmV3IDlwZnMgYmFja2VuZCAieGVuXzlwZnNkIi4gRm9yIHRoaXMgYmFja2Vu
ZCB0eXBlDQo+IHRoZSB0YWcgZGVmYXVsdHMgdG8gIlhlbiIgYW5kIHRoZSBob3N0IHNpZGUg
cGF0aCB0bw0KPiAiL3Zhci9sb2cveGVuL2d1ZXN0cy88ZG9tLW5hbWU+Ii4NCj4gDQo+IERv
IG1vc3Qgb2YgdGhlIGRlZmF1bHQgc2V0dGluZ3MgaW4gbGlieGwuIFVuZm9ydHVuYXRlbHkg
dGhlIGRlZmF1bHQNCj4gcGF0aCBjYW4ndCBlYXNpbHkgYmUgc2V0IGluIGxpYnhsLCBhcyB0
aGUgZG9tYWluIG5hbWUgaXNuJ3QgYXZhaWxhYmxlDQo+IGluIHRoZSByZWxhdGVkIDlwZnMg
c3BlY2lmaWMgZnVuY3Rpb24uDQo+IA0KPiBTZXR0aW5ncyB0aGUgZGVmYXVsdHMgaW4gbGli
eGwgcmVxdWlyZXMgdG8gbW92ZSB0aGUgc2FuaXR5IGNoZWNraW5nDQo+IG9mIDlwZnMgcGFy
YW1ldGVycyBmcm9tIHhsIHRvIGxpYnhsLCB0b28uDQo+IA0KPiBTaWduZWQtb2ZmLWJ5OiBK
dWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+DQo+IFJldmlld2VkLWJ5OiBKYXNvbiBB
bmRyeXVrIDxqYW5kcnl1a0BnbWFpbC5jb20+DQoNClNvcnJ5IEFudGhvbnksIGp1c3QgZm91
bmQgeW91ciBSLWI6IGluIG15IHNwYW0gZm9sZGVyIDotKA0KDQoNCkp1ZXJnZW4NCg0KPiAt
LS0NCj4gVjI6DQo+IC0gdGVzdCBtYXhfZmlsZXMgYW5kIG1heF9vcGVuX2ZpbGVzLCB0b28g
KEphc29uIEFuZHJ5dWspDQo+IFY0Og0KPiAtIGZpeCBtYW4gcGFnZSB0byB1c2UgdGhlICJ4
ZW5fOXBmc2QiIHR5cGUgZHVlIHRvIGlkbCBsaW1pdGF0aW9uDQo+ICAgIChKYXNvbiBBbmRy
eXVrKQ0KPiAtIHNldCAobW9zdCBvZikgdGhlIGRlZmF1bHRzIGluIGxpYnhsIChBbnRob255
IFBlcmFyZCkNCj4gLS0tDQo+ICAgZG9jcy9tYW4veGwuY2ZnLjUucG9kLmluICAgICAgfCAz
NiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKystLQ0KPiAgIHRvb2xzL2xpYnMv
bGlnaHQvbGlieGxfOXBmcy5jIHwgMTggKysrKysrKysrKysrKysrKysrDQo+ICAgdG9vbHMv
eGwveGxfcGFyc2UuYyAgICAgICAgICAgfCAyMyArKysrKysrKysrKysrKysrKysrLS0tDQo+
ICAgMyBmaWxlcyBjaGFuZ2VkLCA3MiBpbnNlcnRpb25zKCspLCA1IGRlbGV0aW9ucygtKQ0K
PiANCj4gZGlmZiAtLWdpdCBhL2RvY3MvbWFuL3hsLmNmZy41LnBvZC5pbiBiL2RvY3MvbWFu
L3hsLmNmZy41LnBvZC5pbg0KPiBpbmRleCBlYThkNDE3MjdkLi4wMzllMDU3MzE4IDEwMDY0
NA0KPiAtLS0gYS9kb2NzL21hbi94bC5jZmcuNS5wb2QuaW4NCj4gKysrIGIvZG9jcy9tYW4v
eGwuY2ZnLjUucG9kLmluDQo+IEBAIC03NzIsMTAgKzc3MiwxNiBAQCBzZXR0aW5ncywgZnJv
bSB0aGUgZm9sbG93aW5nIGxpc3Q6DQo+ICAgDQo+ICAgPW92ZXIgNA0KPiAgIA0KPiArPWl0
ZW0gQjx0eXBlPVRZUEU+DQo+ICsNCj4gK1RoZSBiYWNrZW5kdHlwZSBmb3IgdGhlIFBWIGRl
dmljZS4gU3VwcG9ydGVkIHZhbHVlcyBhcmUgQjxxZW11PiBhbmQNCj4gK0I8eGVuXzlwZnNk
Pi4gIFRoZSBkZWZhdWx0IGlzIEI8cWVtdT4uDQo+ICsNCj4gICA9aXRlbSBCPHRhZz1TVFJJ
Tkc+DQo+ICAgDQo+ICAgOXBmcyB0YWcgdG8gaWRlbnRpZnkgdGhlIGZpbGVzeXN0ZW0gc2hh
cmUuIFRoZSB0YWcgaXMgbmVlZGVkIG9uIHRoZQ0KPiAtZ3Vlc3Qgc2lkZSB0byBtb3VudCBp
dC4NCj4gK2d1ZXN0IHNpZGUgdG8gbW91bnQgaXQuIEZvciB0aGUgYmFja2VuZHR5cGUgb2Yg
Qjx4ZW5fOXBmc2Q+IHRoZSB0YWcgZGVmYXVsdHMgdG8NCj4gKyJYZW4iLg0KPiAgIA0KPiAg
ID1pdGVtIEI8c2VjdXJpdHlfbW9kZWw9Im5vbmUiPg0KPiAgIA0KPiBAQCAtNzg1LDEyICs3
OTEsMzggQEAgc3F1YXNoIG9yIHJlbWFwKS4NCj4gICANCj4gICA9aXRlbSBCPHBhdGg9U1RS
SU5HPg0KPiAgIA0KPiAtRmlsZXN5c3RlbSBwYXRoIG9uIHRoZSBiYWNrZW5kIHRvIGV4cG9y
dC4NCj4gK0ZpbGVzeXN0ZW0gcGF0aCBvbiB0aGUgYmFja2VuZCB0byBleHBvcnQuIEZvciB0
aGUgYmFja2VuZHR5cGUgb2YgQjx4ZW5fOXBmc2Q+DQo+ICt0aGUgcGF0aCBkZWZhdWx0cyB0
byAiQFhFTl9MT0dfRElSQC9ndWVzdHMvPGd1ZXN0LW5hbWU+Ii4NCj4gICANCj4gICA9aXRl
bSBCPGJhY2tlbmQ9ZG9tYWluLWlkPg0KPiAgIA0KPiAgIFNwZWNpZnkgdGhlIGJhY2tlbmQg
ZG9tYWluIG5hbWUgb3IgaWQsIGRlZmF1bHRzIHRvIGRvbTAuDQo+ICAgDQo+ICs9aXRlbSBC
PG1heC1maWxlcz1OVU1CRVI+DQo+ICsNCj4gK1NwZWNpZnkgdGhlIG1heGltdW0gbnVtYmVy
IG9mIGZpbGVzIGJlbG93IEI8cGF0aD4uIEEgdmFsdWUgb2YgMCAod2hpY2gNCj4gK2lzIHRo
ZSBkZWZhdWx0KSBkb2Vzbid0IGxpbWl0IHRoZSBudW1iZXIgb2YgZmlsZXMuIE9ubHkgdmFs
aWQgZm9yDQo+ICtCPHR5cGU9eGVuXzlwZnNkPi4NCj4gKw0KPiArPWl0ZW0gQjxtYXgtb3Bl
bi1maWxlcz1OVU1CRVI+DQo+ICsNCj4gK1NwZWNpZnkgdGhlIG1heGltdW0gbnVtYmVyIG9m
IGNvbmN1cnJlbnRseSBvcGVuZWQgZmlsZXMgYmVsb3cgQjxwYXRoPi4NCj4gK011bHRpcGxl
IG9wZW5zIG9mIHRoZSBzYW1lIGZpbGUgYXJlIGNvdW50ZWQgaW5kaXZpZHVhbGx5LiBPbmx5
IHZhbGlkIGZvcg0KPiArQjx0eXBlPXhlbl85cGZzZD4sIHdoaWNoIGhhcyBhIGRlZmF1bHQg
b2YgQjxtYXgtb3Blbi1maWxlcz01Pi4NCj4gKw0KPiArPWl0ZW0gQjxtYXgtc3BhY2U9TlVN
QkVSPg0KPiArDQo+ICtTcGVjaWZ5IHRoZSBtYXhpbXVtIHVzZWQgZGlzayBzcGFjZSBpbiBN
aUIgYmVsb3cgQjxwYXRoPi4gQSB2YWx1ZSBvZiAwICh3aGljaA0KPiAraXMgdGhlIGRlZmF1
bHQpIGRvZXNuJ3QgbGltaXQgdGhlIHVzYWJsZSBkaXNrIHNwYWNlLiBPbmx5IHZhbGlkIGZv
cg0KPiArQjx0eXBlPXhlbl85cGZzZD4uDQo+ICsNCj4gKz1pdGVtIEI8YXV0by1kZWxldGU9
Qk9PTEVBTj4NCj4gKw0KPiArV2hlbiBzZXQgdGhlIGJhY2tlbmQgd2lsbCBkZWxldGUgdGhl
IG9sZGVzdCBmaWxlIHdoaWNoIGlzIGN1cnJlbnRseSBub3QNCj4gK29wZW5lZCBieSB0aGUg
Z3Vlc3QgaW4gY2FzZSB0aGUgZGlzayBzcGFjZSBsaW1pdCBzZXQgdmlhIEI8bWF4LXNwYWNl
PiBvciB0aGUNCj4gK2ZpbGUgbGltaXQgc2V0IHZpYSBCPG1heC1maWxlcz4gaXMgYmVpbmcg
cmVhY2hlZC4gT25seSB2YWxpZCBmb3INCj4gK0I8dHlwZT14ZW5fOXBmc2Q+Lg0KPiArDQo+
ICAgPWJhY2sNCj4gICANCj4gICA9aXRlbSBCPHB2Y2FsbHM9WyAiYmFja2VuZD1kb21haW4t
aWQiLCAuLi4gXT4NCj4gZGlmZiAtLWdpdCBhL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfOXBm
cy5jIGIvdG9vbHMvbGlicy9saWdodC9saWJ4bF85cGZzLmMNCj4gaW5kZXggOTAwYzBkNDZh
MC4uZGRlYjRmMjBhNyAxMDA2NDQNCj4gLS0tIGEvdG9vbHMvbGlicy9saWdodC9saWJ4bF85
cGZzLmMNCj4gKysrIGIvdG9vbHMvbGlicy9saWdodC9saWJ4bF85cGZzLmMNCj4gQEAgLTIw
LDYgKzIwLDI0IEBADQo+ICAgc3RhdGljIGludCBsaWJ4bF9fZGV2aWNlX3A5X3NldGRlZmF1
bHQobGlieGxfX2djICpnYywgdWludDMyX3QgZG9taWQsDQo+ICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgbGlieGxfZGV2aWNlX3A5ICpwOSwgYm9vbCBob3Rw
bHVnKQ0KPiAgIHsNCj4gKyAgICBpZiAocDktPnR5cGUgPT0gTElCWExfUDlfVFlQRV9VTktO
T1dOKSB7DQo+ICsgICAgICAgIHA5LT50eXBlID0gTElCWExfUDlfVFlQRV9RRU1VOw0KPiAr
ICAgIH0NCj4gKyAgICBpZiAocDktPnR5cGUgPT0gTElCWExfUDlfVFlQRV9RRU1VICYmDQo+
ICsgICAgICAgIChwOS0+bWF4X2ZpbGVzIHx8IHA5LT5tYXhfb3Blbl9maWxlcyB8fCBwOS0+
bWF4X3NwYWNlIHx8DQo+ICsgICAgICAgICBwOS0+YXV0b19kZWxldGUpKSB7DQo+ICsgICAg
ICAgIExPR0QoRVJST1IsIGRvbWlkLCAiSWxsZWdhbCA5cGZzIHBhcmFtZXRlciBjb21iaW5h
dGlvbiIpOw0KPiArICAgICAgICByZXR1cm4gRVJST1JfSU5WQUw7DQo+ICsgICAgfQ0KPiAr
ICAgIGlmIChwOS0+dHlwZSA9PSBMSUJYTF9QOV9UWVBFX1hFTl85UEZTRCAmJiAhcDktPnRh
Zykgew0KPiArICAgICAgICBwOS0+dGFnID0gbGlieGxfX3N0cmR1cChOT0dDLCAiWGVuIik7
DQo+ICsgICAgfQ0KPiArDQo+ICsgICAgaWYgKCFwOS0+cGF0aCB8fCAhcDktPnNlY3VyaXR5
X21vZGVsIHx8ICFwOS0+dGFnKSB7DQo+ICsgICAgICAgIExPR0QoRVJST1IsIGRvbWlkLCAi
OXBmcyBzcGVjIG1pc3NpbmcgcmVxdWlyZWQgZmllbGQhIik7DQo+ICsgICAgICAgIHJldHVy
biBFUlJPUl9JTlZBTDsNCj4gKyAgICB9DQo+ICsNCj4gICAgICAgcmV0dXJuIGxpYnhsX19y
ZXNvbHZlX2RvbWlkKGdjLCBwOS0+YmFja2VuZF9kb21uYW1lLCAmcDktPmJhY2tlbmRfZG9t
aWQpOw0KPiAgIH0NCj4gICANCj4gZGlmZiAtLWdpdCBhL3Rvb2xzL3hsL3hsX3BhcnNlLmMg
Yi90b29scy94bC94bF9wYXJzZS5jDQo+IGluZGV4IDliMzU4ZjExYjguLjgwZmZlODVmNWUg
MTAwNjQ0DQo+IC0tLSBhL3Rvb2xzL3hsL3hsX3BhcnNlLmMNCj4gKysrIGIvdG9vbHMveGwv
eGxfcGFyc2UuYw0KPiBAQCAtMjIzMyw2ICsyMjMzLDIwIEBAIHZvaWQgcGFyc2VfY29uZmln
X2RhdGEoY29uc3QgY2hhciAqY29uZmlnX3NvdXJjZSwNCj4gICAgICAgICAgICAgICAgICAg
ICAgIHJlcGxhY2Vfc3RyaW5nKCZwOS0+dGFnLCB2YWx1ZSk7DQo+ICAgICAgICAgICAgICAg
ICAgIH0gZWxzZSBpZiAoIXN0cmNtcChrZXksICJiYWNrZW5kIikpIHsNCj4gICAgICAgICAg
ICAgICAgICAgICAgIHJlcGxhY2Vfc3RyaW5nKCZwOS0+YmFja2VuZF9kb21uYW1lLCB2YWx1
ZSk7DQo+ICsgICAgICAgICAgICAgICAgfSBlbHNlIGlmICghc3RyY21wKGtleSwgInR5cGUi
KSkgew0KPiArICAgICAgICAgICAgICAgICAgICBpZiAobGlieGxfcDlfdHlwZV9mcm9tX3N0
cmluZyh2YWx1ZSwgJnA5LT50eXBlKSkgew0KPiArICAgICAgICAgICAgICAgICAgICAgICAg
ZnByaW50ZihzdGRlcnIsICJmYWlsZWQgdG8gcGFyc2UgOXBmcyB0eXBlOiAlc1xuIiwNCj4g
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdmFsdWUpOw0KPiArICAgICAgICAg
ICAgICAgICAgICAgICAgZXhpdCgxKTsNCj4gKyAgICAgICAgICAgICAgICAgICAgfQ0KPiAr
ICAgICAgICAgICAgICAgIH0gZWxzZSBpZiAoIXN0cmNtcChrZXksICJtYXgtZmlsZXMiKSkg
ew0KPiArICAgICAgICAgICAgICAgICAgICBwOS0+bWF4X2ZpbGVzID0gcGFyc2VfdWxvbmco
dmFsdWUpOw0KPiArICAgICAgICAgICAgICAgIH0gZWxzZSBpZiAoIXN0cmNtcChrZXksICJt
YXgtb3Blbi1maWxlcyIpKSB7DQo+ICsgICAgICAgICAgICAgICAgICAgIHA5LT5tYXhfb3Bl
bl9maWxlcyA9IHBhcnNlX3Vsb25nKHZhbHVlKTsNCj4gKyAgICAgICAgICAgICAgICB9IGVs
c2UgaWYgKCFzdHJjbXAoa2V5LCAibWF4LXNwYWNlIikpIHsNCj4gKyAgICAgICAgICAgICAg
ICAgICAgcDktPm1heF9zcGFjZSA9IHBhcnNlX3Vsb25nKHZhbHVlKTsNCj4gKyAgICAgICAg
ICAgICAgICB9IGVsc2UgaWYgKCFzdHJjbXAoa2V5LCAiYXV0by1kZWxldGUiKSkgew0KPiAr
ICAgICAgICAgICAgICAgICAgICBwOS0+YXV0b19kZWxldGUgPSBzdHJ0b3VsKHZhbHVlLCBO
VUxMLCAwKTsNCj4gICAgICAgICAgICAgICAgICAgfSBlbHNlIHsNCj4gICAgICAgICAgICAg
ICAgICAgICAgIGZwcmludGYoc3RkZXJyLCAiVW5rbm93biA5cGZzIHBhcmFtZXRlciAnJXMn
XG4iLCBrZXkpOw0KPiAgICAgICAgICAgICAgICAgICAgICAgZXhpdCgxKTsNCj4gQEAgLTIy
NDMsOSArMjI1NywxMiBAQCB2b2lkIHBhcnNlX2NvbmZpZ19kYXRhKGNvbnN0IGNoYXIgKmNv
bmZpZ19zb3VyY2UsDQo+ICAgDQo+ICAgICAgICAgICAgICAgbGlieGxfc3RyaW5nX2xpc3Rf
ZGlzcG9zZSgmcGFpcnMpOw0KPiAgIA0KPiAtICAgICAgICAgICAgaWYgKCFwOS0+cGF0aCB8
fCAhcDktPnNlY3VyaXR5X21vZGVsIHx8ICFwOS0+dGFnKSB7DQo+IC0gICAgICAgICAgICAg
ICAgZnByaW50ZihzdGRlcnIsICI5cGZzIHNwZWMgbWlzc2luZyByZXF1aXJlZCBmaWVsZCFc
biIpOw0KPiAtICAgICAgICAgICAgICAgIGV4aXQoMSk7DQo+ICsgICAgICAgICAgICBpZiAo
cDktPnR5cGUgPT0gTElCWExfUDlfVFlQRV9YRU5fOVBGU0QgJiYgIXA5LT5wYXRoKSB7DQo+
ICsgICAgICAgICAgICAgICAgY2hhciAqcGF0aDsNCj4gKw0KPiArICAgICAgICAgICAgICAg
IHhhc3ByaW50ZigmcGF0aCwgWEVOX0xPR19ESVIgIi9ndWVzdHMvJXMiLCBjX2luZm8tPm5h
bWUpOw0KPiArICAgICAgICAgICAgICAgIHJlcGxhY2Vfc3RyaW5nKCZwOS0+cGF0aCwgcGF0
aCk7DQo+ICsgICAgICAgICAgICAgICAgZnJlZShwYXRoKTsNCj4gICAgICAgICAgICAgICB9
DQo+ICAgICAgICAgICB9DQo+ICAgICAgIH0NCg0K
--------------rG5Sd0bq0obQ5zm70O0ZQ86d
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------rG5Sd0bq0obQ5zm70O0ZQ86d--

--------------4sYac0mFmWSHsk0WeNADZmYl--

--------------p0I55L4PbTadoXKUmMCzhEap
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXN3HoFAwAAAAAACgkQsN6d1ii/Ey+O
sAgAhLX+c7EPwbtzJEnwnSa2UtkkSZujyC1CXEi7f/2yS6OsH+4M3uLWqnRVdE57NCu5Ppxn0C4P
IsGmWHI1ldZ0hzef5mbSEni9ByapmCVWDjI3knPLCjnJGuLQASX7A7/7AN+UrxY8InbxnPsehpZv
5Z+x/SeyvAxxdnGAo7l8OWbgdkf5X7HlZ8hdqMkdRzpJjpd6TwfQQiUO8837Ui1DZG17TxCFSvgU
vh87da/QutFfcoDhXi13cBhH5btzH/n8+tL8cFJoqxUpVRKJ7Uv0dhYCKyIj8RLf+byMKOnFQdVP
Dzs2MT1iDh47Ivmv/0zlTPti8xo8EQRCmaFqKN/nlg==
=TEMr
-----END PGP SIGNATURE-----

--------------p0I55L4PbTadoXKUmMCzhEap--


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 09:49:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 09:49:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681455.1060224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYN5-0002b9-Vd; Thu, 15 Feb 2024 09:49:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681455.1060224; Thu, 15 Feb 2024 09:49:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYN5-0002b2-RL; Thu, 15 Feb 2024 09:49:35 +0000
Received: by outflank-mailman (input) for mailman id 681455;
 Thu, 15 Feb 2024 09:49:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYN4-0002aw-CJ
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 09:49:34 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 854f354a-cbe7-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 10:49:33 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-3394ca0c874so374814f8f.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 01:49:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n12-20020adff08c000000b0033cf60e268fsm1218247wro.116.2024.02.15.01.49.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 01:49:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 854f354a-cbe7-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707990572; x=1708595372; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IVgEOvZrCtGQjEE1SZTaMaOUll3qd7mZLtDI+zzisLQ=;
        b=ARB3JBLkGlgqLtIx/tWqnxJ5wbtgxv+uKX95xL8CsT+4R2R0jKyrf3276do8efDBvh
         BwizhIghZ2BfgAF4T6k9+RWqXK6+vwEIpR1iIKayOWXC9Z3RrJrc475wyW8Z2v0vdg8C
         GkWJ7VXVsr7rp04/4t1st60AI53yCmr7f8jWzmJGZ89zissRWYQ8P+Zp74hqZKMHecBM
         gCqnZ7CsG2/igV7ijsB/qaknzl9eVZ76cQEAIA+JjhV74YBLhB4/r1Se/LFBGn2AE4o6
         erVKRNRUBTH3O3w0/1yrFIvVCP9a4mpxhr1KNbXE5KXdF6XedM0+ChMpoZ+6U7/IUwp2
         bPnA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707990572; x=1708595372;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IVgEOvZrCtGQjEE1SZTaMaOUll3qd7mZLtDI+zzisLQ=;
        b=BttrbnpCS5xJbpwCPw/lEEO8usSUsok3xhj0JKEl37RsLUbAnot93yC4W2HWaNfo17
         qvR0AyBORWMvqOOs9FOO6O98w+HYxTCWQdfTTw/ebPFs63PowT5ootuH2PoOzqUBDWot
         4s1lsEEYUO5HYkQti2g7p0qsPGh1nOslOwM1NxA1Hpkkz8H73KTnbm06rfg/f3w7AxdB
         dEkY2OyYGr43xnkJ4x7K0nq4yo1/VzArqGuhFAkbWd9JYxod5aIzCQrX4MBFPJPDGjw6
         VwWLtTaefJmtHpq09WKXL++2JksWfn5bUVaiKpIEEbRVrN/og4cchgiP1ktr2D7O7dQN
         K3IQ==
X-Forwarded-Encrypted: i=1; AJvYcCWdUWqg1jO8eDVzijkVxUAlLE8D+o9zMu4Vcl3A56g/BzkOyScVW0fyMxkeoOLbKs5oGrrvH24EjxyCuqdfGdHs+IBBDdjVTQpS11IF7jE=
X-Gm-Message-State: AOJu0YwhxznBsuVqOEl7+pW/sv/RlwxO3ix/YpX8OJvKrt703NU9jVXB
	cdNiqmC0miPnNO280dljS/3Gum1BuEv3hdlpp3DTP0a52AbgDksWJS1roL2nQA==
X-Google-Smtp-Source: AGHT+IFfKANbjMcmyz6TEXKKZZSJIdOoxdNcPrqEzD19b1tjxsq8Zt6GmxV0lox3iv29DgcJp8qKMQ==
X-Received: by 2002:a5d:58cf:0:b0:33b:3b86:b845 with SMTP id o15-20020a5d58cf000000b0033b3b86b845mr758626wrf.58.1707990572575;
        Thu, 15 Feb 2024 01:49:32 -0800 (PST)
Message-ID: <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com>
Date: Thu, 15 Feb 2024 10:49:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: anthony.perard@citrix.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240215093002.23527-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240215093002.23527-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.02.2024 10:30, Roger Pau Monne wrote:
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -358,10 +358,10 @@ config: tools_fixdep outputmakefile FORCE
>  else # !config-build
>  
>  ifeq ($(need-config),y)
> --include include/config/auto.conf
>  # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
>  # changes are detected.
>  -include include/config/auto.conf.cmd
> +include include/config/auto.conf

With the - dropped, ...

> @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
>  # This exploits the 'multi-target pattern rule' trick.
>  # The syncconfig should be executed only once to make all the targets.
>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
> +	rm -rf include/config/$*.conf
>  	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig

... is this really necessary? The error status from the sub-make is ignored
only because of the -, isn't it?

I also don't really follow the need to re-order the include-s above. Their
ordering ought to be benign, as per make's doc stating "If an included
makefile cannot be found in any of these directories it is not an
immediately fatal error; processing of the makefile containing the include
continues." While the description talks about this, I'm afraid I don't
really understand "... the .cmd target is executed before including ...":
What .cmd target are you talking about there? That's merely another
include (and what is included doesn't provide any .cmd target afaics), so
will be dealt with the same as the config/auto.conf one itself.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:07:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:07:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681463.1060232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYeG-0005oZ-C4; Thu, 15 Feb 2024 10:07:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681463.1060232; Thu, 15 Feb 2024 10:07:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYeG-0005oS-9T; Thu, 15 Feb 2024 10:07:20 +0000
Received: by outflank-mailman (input) for mailman id 681463;
 Thu, 15 Feb 2024 10:07:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EUcX=JY=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1raYeF-0005oK-BK
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:07:19 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 00039feb-cbea-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:07:18 +0100 (CET)
Received: from [192.168.1.229] (93-36-216-194.ip62.fastwebnet.it
 [93.36.216.194])
 by support.bugseng.com (Postfix) with ESMTPSA id 4278B4EE0737;
 Thu, 15 Feb 2024 11:07:17 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00039feb-cbea-11ee-8a4d-1f161083a0e0
Message-ID: <d735e980-c5a5-46b1-a511-cc2d666dac23@bugseng.com>
Date: Thu, 15 Feb 2024 11:07:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US, it
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
 <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
 <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
 <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com>
 <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
 <45c19a87-5833-4350-a374-3a70fd3aa9ae@suse.com>
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <45c19a87-5833-4350-a374-3a70fd3aa9ae@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 15/02/24 09:10, Jan Beulich wrote:
> On 14.02.2024 17:11, Federico Serafini wrote:
>> I did some tries with example programs
>> and the assembler error always points to file and line
>> of the most enclosing function that caused the failure.
>> If I am not missing something, using __FILE__ and __LINE__ does not add
>> any information.
>>
>> Therefore, if the new macro is used within the body of other macros,
>> then the resulting assembler error will point to the source of
>> the problem (e.g., the site of a bogus call to put_guest()).
>>
>> In my opinion, converting put_guest() &Co. to inline functions is not
>> convenient: the assembler error will point to the most enclosing
>> function that would be put_unsafe_size(), instead of pointing to the
>> source of the problem.
> 
> The assembler error will point to where the inline function was expanded,
> sure. __FILE__ / __LINE__ ought to point to that inline function (where
> the macro was used) then, though?

This is what I get:

federico@Dell:~$ cat m.c
#define STRINGIFY(arg) #arg
#define STATIC_ASSERT_UNREACHABLE(file, line) \
   asm(".error \"static assertion failed: " file ": " STRINGIFY(line) "\"")

static inline __attribute__((always_inline)) void g(int x) {
   switch(x) {
     case 0:
       STATIC_ASSERT_UNREACHABLE(__FILE__, __LINE__);
   }
}

static inline __attribute__((always_inline)) void f(int x) {
   g(x);
}

int main(void) {
   f(0);
   return 0;
}
federico@Dell:~$ gcc -O3 m.c
m.c: Assembler messages:
m.c:8: Error: static assertion failed: m.c: 8


Note that the linker behaves differently:

federico@Dell:~$ cat m.c
extern void __put_user_bad(void);

static inline __attribute__((always_inline)) void g(int x) {
   switch(x) {
     case 0:
       __put_user_bad();
   }
}

static inline __attribute__((always_inline)) void f(int x) {
   g(x);
}

int main(void) {
   f(0);
   return 0;
}
federico@Dell:~$ gcc -O3 m.c
/usr/bin/ld: /tmp/ccv9KHJD.o: in function `main':
m.c:(.text.startup+0x9): undefined reference to `__put_user_bad'
collect2: error: ld returned 1 exit status

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:11:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:11:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681467.1060243 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYiO-0007Yv-Uf; Thu, 15 Feb 2024 10:11:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681467.1060243; Thu, 15 Feb 2024 10:11:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYiO-0007Yo-RB; Thu, 15 Feb 2024 10:11:36 +0000
Received: by outflank-mailman (input) for mailman id 681467;
 Thu, 15 Feb 2024 10:11:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYiN-0007Yi-0j
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:11:35 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 98a0865d-cbea-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:11:33 +0100 (CET)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-511ac701428so761169e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:11:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 j4-20020a05600c1c0400b004120537210esm2448089wms.46.2024.02.15.02.11.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:11:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 98a0865d-cbea-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707991893; x=1708596693; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=UwzZFR6sa3lUvrnWiTVsUEQMXzcywTGe8DNCW4tzka8=;
        b=EGtBQBSqhxmoFd4bhTZLwhV/oBT9S11gBNjAl8vcpXf/7Zz8GYl+eMMpIjHE3H1dyh
         ovEhY7ULkYO3JJ52kAcyCYf3yFp9ZKQuuClT1Te7A8E0hMz+tZfwUxl96wr2wf+rGQ7l
         6CmBPYgYvXKZyXw6lNk0/k9QRjkOj8vWBVwTnRRFogexmTQe9Fx8D5mdJHQq6bESNJLe
         ESPDTskdBUdGSixvrJV8+JDLfnjSaFOXpwcfoVMgxNl+zejsWF7SmW/miKQdt5RatMV/
         CaAWSyOVaLI2MbfEbeaMQcNl77ERyYwPbLrHWZSvOM6KPvbzHMSyH3d4zBHPj4D/zw8B
         e/fA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707991893; x=1708596693;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=UwzZFR6sa3lUvrnWiTVsUEQMXzcywTGe8DNCW4tzka8=;
        b=L8mfYN1p/TjaTWxvex6+kpkfi4E5BkHcPLrxuZy0F3oVn++UVne4EMegptHXp/gGgK
         kymgwF3E1jFRdCImjbe7W+J6MW+nR2o1CKlnNFiIvjQTCjqufGfSXXid6b2dYuTF3xK5
         1HwaUZnsYRDHSFbEIA60VpTTo+wD54a23338Gx1Xrl3ZUs4ybBHV3wpvzruFelcOtx8u
         30lIHmNRxf6ze9HdF7cO8NLJTKfVQ6iY+ENvCH4sBF5r+xU5FP1M1lgYRS38JpV4NZSA
         wz6UDwlrVKKr54QNARxN72pX3uR52oLWWjxJgpV9rdPyGBR/b6i8EcSExZLShrTrtdWP
         5g2g==
X-Gm-Message-State: AOJu0YyvES9xB9oDhV3Fzg3c790MSGLFhKUv0m2Tp6JEzinwRsgvTeH1
	i0CjlYMavX8mhWnuaXc3XU5v6YeNnsr53oQUpFl8B72epVQSQgD+HNxVW0K1K+6mUHwInF4yus4
	=
X-Google-Smtp-Source: AGHT+IH6B92B0cHZFgY2cjeJdM5Dm805rozF/wMA1iYSrj1ucGizwEwHh4LQOqsPBxOwqNVRtgYA2Q==
X-Received: by 2002:a05:6512:447:b0:511:9e20:5fd with SMTP id y7-20020a056512044700b005119e2005fdmr832251lfk.59.1707991893434;
        Thu, 15 Feb 2024 02:11:33 -0800 (PST)
Message-ID: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Date: Thu, 15 Feb 2024 11:11:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2 00/12] VT-d: SATC handling; ATS: tidying
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On a Sapphire Rapids system I noticed a line in the serial log
indicating a DMAR entry type we're not recognizing. While arranging
for it to be both recognized and respected, I noticed some further
potential for cleaning up.

01: VT-d: correct ATS checking for root complex integrated devices
02: VT-d: tidy error handling of RMRR parsing
03: VT-d: parse ACPI "SoC Integrated Address Translation Cache Reporting Structure"s
04: AMD/IOMMU: add helper to check whether ATS is to be used for a device
05: IOMMU: rename and re-type ats_enabled
06: VT-d: respect ACPI SATC's ATC_REQUIRED flag
07: VT-d: replace find_ats_dev_drhd()
08: VT-d: move ats_device() to the sole file it's used from
09: VT-d: move dev_invalidate_iotlb() to the sole file it's used from
10: VT-d: move {,un}map_vtd_domain_page()
11: VT-d: drop flush_dev_iotlb parameter from IOTLB flush hook
12: PCI/ATS: tidy {en,dis}able_ats_device() a little

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:13:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:13:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681472.1060253 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYkE-00087K-8h; Thu, 15 Feb 2024 10:13:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681472.1060253; Thu, 15 Feb 2024 10:13:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYkE-00087D-6E; Thu, 15 Feb 2024 10:13:30 +0000
Received: by outflank-mailman (input) for mailman id 681472;
 Thu, 15 Feb 2024 10:13:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYkC-000877-Q4
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:13:28 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dbd172f2-cbea-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 11:13:26 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-4121a8635a2so2697005e9.3
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:13:26 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.13.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:13:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dbd172f2-cbea-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992006; x=1708596806; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=kaaCGUPluoRwbjKkcfJEfltfLrIYbRueMH6UyOi+fqA=;
        b=Vd3Xrg6XndVs6UwuGMJ67Hvv+h71ufP30hEAK6md2Wh2UZwJ5e+YCH/Vh6r9sG2Rhp
         Tjx+GMdJB/tfjBDyXHvihrrbjMrQ9kNqdN8pHZBD9EEPrI9YA1THblFP0K2vHO+DX9op
         g9FnP1fJjpxjvZIviNucrvkiWyJo53DWFD2sq4jhs53P2FVEMGWOXWMbHCV5TFoxeYht
         FmPBBMmR/tNIr7eoeFXDP8UrsDUmaFbyF+tY4VjfUDQm7qfmOQZOBycR5d7nH31iOyrM
         OI+EDygncUH0MWjagC9oJGEu5dYh/6n7RK7ipG4X9R8AOHxVa9QBT20WeS/iuXONWnoa
         lGQQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992006; x=1708596806;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=kaaCGUPluoRwbjKkcfJEfltfLrIYbRueMH6UyOi+fqA=;
        b=wIECJdEa3hGjD+qNbjoyCG0ghokd0vjLxdQHsWtkMVsy3oW/DzgH/xxiM6RVjPkix9
         7vys7T0Pdoq7oN1dJT5mRgdUE1Vwx2ku7ji1Ve//OgJlCbIJG6n7iL2evWAJ2Xw/NHPN
         1wsonEHJXhqsdIK3k4gGbxGkMww60fToa0b5rxd15auqTAa4W6A1kA/sq6sbc5S3BoPz
         iJVx5S2HsOOagtnaJfV9L/DkgulQPDHSh+cPIVYC1nkJpGouDgjw6p7PyRUCR1R6dubN
         VU3QHD3v3nLbBmeetP/iF/fb60jroBzEudv/L2dx4d5NqujFUaiyuCQnqGCAmtJfhGOk
         6B+A==
X-Gm-Message-State: AOJu0YyLrzkVAUD5xN9zhLaZZ/uemxqtVvchyN3SY+QeWQ6zwONIw/ot
	GoeOgbQsQ4DzU7Ryl62BUlsHIl10t+EETGu66zYQ+95QTZaP6q/wPVnXMbExKDTnz6hvPGsD/GI
	=
X-Google-Smtp-Source: AGHT+IH1qlDa3iBjsRLyla193sMpUv7aYmnrfSROwwzh0LTnvgsw6UdyGbNpFEs8UsZtldmOTZ0/kQ==
X-Received: by 2002:a05:600c:34c1:b0:411:c25d:703b with SMTP id d1-20020a05600c34c100b00411c25d703bmr1072716wmq.16.1707992006185;
        Thu, 15 Feb 2024 02:13:26 -0800 (PST)
Message-ID: <3686dae7-e005-47b5-9235-14208a68eec5@suse.com>
Date: Thu, 15 Feb 2024 11:13:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 01/12] VT-d: correct ATS checking for root complex
 integrated devices
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Spec version 4.1 says

"The ATSR structures identifies PCI Express Root-Ports supporting
 Address Translation Services (ATS) transactions. Software must enable
 ATS on endpoint devices behind a Root Port only if the Root Port is
 reported as supporting ATS transactions."

Clearly root complex integrated devices aren't "behind root ports",
matching my observation on a SapphireRapids system having an ATS-
capable root complex integrated device. Hence for such devices we
shouldn't try to locate a corresponding ATSR.

Since both pci_find_ext_capability() and pci_find_cap_offset() return
"unsigned int", change "pos" to that type at the same time.

Fixes: 903b93211f56 ("[VTD] laying the ground work for ATS")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ b/xen/drivers/passthrough/vtd/x86/ats.c
@@ -44,7 +44,7 @@ struct acpi_drhd_unit *find_ats_dev_drhd
 int ats_device(const struct pci_dev *pdev, const struct acpi_drhd_unit *drhd)
 {
     struct acpi_drhd_unit *ats_drhd;
-    int pos;
+    unsigned int pos, expfl = 0;
 
     if ( !ats_enabled || !iommu_qinval )
         return 0;
@@ -53,7 +53,12 @@ int ats_device(const struct pci_dev *pde
          !ecap_dev_iotlb(drhd->iommu->ecap) )
         return 0;
 
-    if ( !acpi_find_matched_atsr_unit(pdev) )
+    pos = pci_find_cap_offset(pdev->sbdf, PCI_CAP_ID_EXP);
+    if ( pos )
+        expfl = pci_conf_read16(pdev->sbdf, pos + PCI_EXP_FLAGS);
+
+    if ( MASK_EXTR(expfl, PCI_EXP_FLAGS_TYPE) != PCI_EXP_TYPE_RC_END &&
+         !acpi_find_matched_atsr_unit(pdev) )
         return 0;
 
     ats_drhd = find_ats_dev_drhd(drhd->iommu);



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:14:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:14:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681476.1060263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYko-0000Ac-H5; Thu, 15 Feb 2024 10:14:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681476.1060263; Thu, 15 Feb 2024 10:14:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYko-0000AU-E7; Thu, 15 Feb 2024 10:14:06 +0000
Received: by outflank-mailman (input) for mailman id 681476;
 Thu, 15 Feb 2024 10:14:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYkn-000877-Gm
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:14:05 +0000
Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com
 [2a00:1450:4864:20::433])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f1d22946-cbea-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 11:14:03 +0100 (CET)
Received: by mail-wr1-x433.google.com with SMTP id
 ffacd0b85a97d-33d01faf711so279534f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:14:03 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.14.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:14:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1d22946-cbea-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992043; x=1708596843; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=KU/lqMLMrEbpy7S9jjdmyz27GyrWioKTxk0YxtgDY6A=;
        b=RS3sxTB80kNFYvvxKXiD5fE0JIuIb67cB56i/lx9KLxTbqVdvFtyGwTNBkQDt8uK+B
         5Lf4rVj9wWbuNSK4ZTQwMXO383WJuFh24TabAOtifO5tyW020EW8XMdxJozZiSRDQv24
         Ubays/0ZbEkRUPEmO3A7xuTO4fM2nZuExrj02cJAWYgtxbgZST57sSVZ84H7phEYKKFG
         EZX+SYzAYsfddiVWfgcuq7MD3CRVv+HQ/Du6oaxHwFfppwWLTcNgJDXB/evAFhezaTfx
         lzsA1+nAVvHgSCofOsFNAgd8T4HjV7Zg2iwHtayUdGebmMSIbUUkA6NKTqqxHUYZBttN
         5mvw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992043; x=1708596843;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=KU/lqMLMrEbpy7S9jjdmyz27GyrWioKTxk0YxtgDY6A=;
        b=d0UtahmGns4m50McaFNC12u5tDfnfuCyxfy18uGz1hb6pXlzyE8eGv5HZVFvPcP5zi
         ejSvHDes1aH1rYTbFV/KFXgA2E7ogTZVC0SoJM2pH+N2eu831yW3kDpjEQRG8zC9liE0
         VPgSghg/spFOcxWy47fWcGBawD6OQX+pziYiqHW1xGi6OHMZjCo2o0v6bkkyBSptr1kT
         UMjzKRAJBmnBZZOBFtUCpCmMJgWBkA+HFMYBMO5L96GrPuF+5QYGKH+lBF2L6/zBjy0u
         hCRwzTwVlfKkgGRT70vz8qYUFW2ltuMBCb0J7zSFc3Ccc98Fw1wUq34L4jxNb8ObuaPk
         fgng==
X-Gm-Message-State: AOJu0YzLq+x3ftdxcWLtQOiHn7Z2f6B88k8Fasv//syiDOuPBJXZanVa
	89Q+P9Gc8x8R78Wm7dzgBz2hGnmJo74p+seazMh1zE0+N1ABJF4xDrUDyKKwLBQDWDkR3Gg1H3U
	=
X-Google-Smtp-Source: AGHT+IHKaWfAiFYhRjDVVpSwFB7pRSbD3IcCQS5JuxOtu7EYgKWLGl4fizouEZ0cB440LcqOK4QmQw==
X-Received: by 2002:a5d:5591:0:b0:33c:d139:66fb with SMTP id i17-20020a5d5591000000b0033cd13966fbmr1219944wrv.52.1707992043032;
        Thu, 15 Feb 2024 02:14:03 -0800 (PST)
Message-ID: <97577ebf-3871-47be-97c5-8164701be756@suse.com>
Date: Thu, 15 Feb 2024 11:14:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 02/12] VT-d: tidy error handling of RMRR parsing
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It's acpi_parse_one_rmrr() where the allocation is coming from (by way
of invoking acpi_parse_dev_scope()), or in add_one_user_rmrr()'s case
allocation is even open-coded there, so freeing would better also happen
there. Care needs to be taken to preserve acpi_parse_one_rmrr()'s
ultimate return value.

While fiddling with callers also move scope_devices_free() to .init and
have it use XFREE() instead of open-coding it.

In register_one_rmrr() also have the "ignore" path take the main
function return path.

Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -82,14 +82,13 @@ static int __init acpi_register_rmrr_uni
     return 0;
 }
 
-static void scope_devices_free(struct dmar_scope *scope)
+static void __init scope_devices_free(struct dmar_scope *scope)
 {
     if ( !scope )
         return;
 
     scope->devices_cnt = 0;
-    xfree(scope->devices);
-    scope->devices = NULL;
+    XFREE(scope->devices);
 }
 
 static void __init disable_all_dmar_units(void)
@@ -595,17 +594,13 @@ static int register_one_rmrr(struct acpi
                 " Ignore RMRR [%"PRIx64",%"PRIx64"] as no device"
                 " under its scope is PCI discoverable!\n",
                 rmrru->base_address, rmrru->end_address);
-        scope_devices_free(&rmrru->scope);
-        xfree(rmrru);
-        return 1;
+        ret = 1;
     }
     else if ( rmrru->base_address > rmrru->end_address )
     {
         dprintk(XENLOG_WARNING VTDPREFIX,
                 " RMRR [%"PRIx64",%"PRIx64"] is incorrect!\n",
                 rmrru->base_address, rmrru->end_address);
-        scope_devices_free(&rmrru->scope);
-        xfree(rmrru);
         ret = -EFAULT;
     }
     else
@@ -660,21 +655,20 @@ acpi_parse_one_rmrr(struct acpi_dmar_hea
                                &rmrru->scope, RMRR_TYPE, rmrr->segment);
 
     if ( !ret && (rmrru->scope.devices_cnt != 0) )
-    {
         ret = register_one_rmrr(rmrru);
-        /*
-         * register_one_rmrr() returns greater than 0 when a specified
-         * PCIe device cannot be detected. To prevent VT-d from being
-         * disabled in such cases, reset the return value to 0 here.
-         */
-        if ( ret > 0 )
-            ret = 0;
 
-    }
-    else
+    if ( ret )
+    {
+        scope_devices_free(&rmrru->scope);
         xfree(rmrru);
+    }
 
-    return ret;
+    /*
+     * register_one_rmrr() returns greater than 0 when a specified PCIe
+     * device cannot be detected. To prevent VT-d from being disabled in
+     * such cases, make the return value 0 here.
+     */
+    return ret > 0 ? 0 : ret;
 }
 
 static int __init
@@ -945,9 +939,13 @@ static int __init add_one_user_rmrr(unsi
     rmrr->scope.devices_cnt = dev_count;
 
     if ( register_one_rmrr(rmrr) )
+    {
         printk(XENLOG_ERR VTDPREFIX
                "Could not register RMMR range "ERMRRU_FMT"\n",
                ERMRRU_ARG);
+        scope_devices_free(&rmrr->scope);
+        xfree(rmrr);
+    }
 
     return 1;
 }



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:14:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:14:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681481.1060273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYlG-0000kQ-RK; Thu, 15 Feb 2024 10:14:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681481.1060273; Thu, 15 Feb 2024 10:14:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYlG-0000kJ-Of; Thu, 15 Feb 2024 10:14:34 +0000
Received: by outflank-mailman (input) for mailman id 681481;
 Thu, 15 Feb 2024 10:14:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYlF-0000Sx-JE
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:14:33 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0329edfd-cbeb-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:14:32 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-41205e19660so4893155e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:14:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.14.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:14:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0329edfd-cbeb-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992072; x=1708596872; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=fj8qUorQgdMO+SEeflpHtiiMwgxTjCex7pYcfIyVQuE=;
        b=e8tdgsXF24xURiNWHJri0h8BL3XuKB77vTGJOPmBLlJoW+thPghgJRRx3kEwBLr1R/
         FZmIw03rDpYS7WNPa6hY/CFsC9ioUNb6K6MsF9OQA9GHj2b4HIJ3wEaQHaAGHOGgrpF/
         4DTlysSVws1n7xcZyMoAFvZ/Qoh4/dmv6/pFqfqfYL/e/pNU1OmyVHEBl975sbhIT5Je
         hM3vR6VWI7WHvn+v9KXekTslM0nSpR0gyUJWWjJSkwJ2fkoyGGRbHgGIOUiIcKn/DMx4
         6oYjQ5fPOOIWwr2Bv58uCsvr3EC49j7NSupJVpRx72cHSFn+tM/5xEG+B5bDhTMEPOci
         EihA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992072; x=1708596872;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=fj8qUorQgdMO+SEeflpHtiiMwgxTjCex7pYcfIyVQuE=;
        b=kgbzVEAOoNzsu/7ETPu7ksmEwjD6sQ1TRLgmjF+/zafCkMHhYMxzXJvBGEWYMbyPtr
         btRP0CNSEtEd/MFb5zBQUNThH7vJUdlYJ1tUhWuNMFVtLYiFd8RtmdNiisN3W9QAP2Je
         HYqeONnuk/4PnyPCwVYtSCNy5qk54GZ/hCo6p3R/hIMYdMvGBY0hOShNX7y9DNoI2JJn
         3f7gHkBtX30DjHZxVQIb31aPebdKepNDLX29ZUlBw/PzljvSwJxB/eI8975KQTRYnQSl
         p964f7uKutSrTekG8h1l8wgqZnw2tjcSvYp+MP5dtdkm69k/TC0hO+4wDErNR+TZVA3p
         KvTA==
X-Gm-Message-State: AOJu0YxrzXSvlIV9/TNOE6OEgck+eZZomH/JeK5oczLVmsLHzbx5Ixg0
	fdLtgKrtehIfn8GAAsGGBqANEeptpB/vYM4sAhfwB5GWcOHDWlyMVUx2FURVwDcXtd1s+ATlPP4
	=
X-Google-Smtp-Source: AGHT+IGFGMj7EAeEq5MHaPpPZFlgUlUywi+V2oHpegmyE6iJJ0Ys0Olf4VeGGL4VOtkXN+7+isl0vQ==
X-Received: by 2002:adf:ec82:0:b0:33b:1a39:e501 with SMTP id z2-20020adfec82000000b0033b1a39e501mr874165wrn.28.1707992072274;
        Thu, 15 Feb 2024 02:14:32 -0800 (PST)
Message-ID: <c70b250d-2ec4-4254-89cf-d3241dac0d35@suse.com>
Date: Thu, 15 Feb 2024 11:14:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 03/12] VT-d: parse ACPI "SoC Integrated Address Translation
 Cache Reporting Structure"s
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

This is a prereq to us, in particular, respecting the "ATC required"
flag.

Note that ACPI_SATC_ATC_REQUIRED has its #define put in dmar.h, as we
try to keep actbl*.h in sync what Linux (who in turn inherit from ACPI
CA) has.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Lovely: On the SPR system with the SATC I tried passing "ats" (the
"required" flag is clear there), just to then hit "IOMMU#4: QI dev wait
descriptor taking too long" while setting up Dom0. The 2nd message there
doesn't ever appear, so the request never completes. Not sure whether
that's us doing something wrong or the hardware acting up. In the former
case I'd generally expect an IOMMU fault to be raised, though. FTR same
on 4.18 with just "VT-d: correct ATS checking for root complex
integrated devices" backported there.

Should we check scope entries for appropriate types? (If so, then also
for e.g. ATSR.)
---
v2: Move error case freeing to acpi_parse_one_satc(). Introduce #define
    for the flag bit. Style.

--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -47,6 +47,7 @@ LIST_HEAD_READ_MOSTLY(acpi_drhd_units);
 LIST_HEAD_READ_MOSTLY(acpi_rmrr_units);
 static LIST_HEAD_READ_MOSTLY(acpi_atsr_units);
 static LIST_HEAD_READ_MOSTLY(acpi_rhsa_units);
+static LIST_HEAD_READ_MOSTLY(acpi_satc_units);
 
 static struct acpi_table_header *__read_mostly dmar_table;
 static int __read_mostly dmar_flags;
@@ -750,6 +751,93 @@ acpi_parse_one_rhsa(struct acpi_dmar_hea
     return ret;
 }
 
+static int __init register_one_satc(struct acpi_satc_unit *satcu)
+{
+    bool ignore = false;
+    unsigned int i = 0;
+    int ret = 0;
+
+    /* Skip checking if segment is not accessible yet. */
+    if ( !pci_known_segment(satcu->segment) )
+        i = UINT_MAX;
+
+    for ( ; i < satcu->scope.devices_cnt; i++ )
+    {
+        uint8_t b = PCI_BUS(satcu->scope.devices[i]);
+        uint8_t d = PCI_SLOT(satcu->scope.devices[i]);
+        uint8_t f = PCI_FUNC(satcu->scope.devices[i]);
+
+        if ( !pci_device_detect(satcu->segment, b, d, f) )
+        {
+            dprintk(XENLOG_WARNING VTDPREFIX,
+                    " Non-existent device (%pp) is reported in SATC scope!\n",
+                    &PCI_SBDF(satcu->segment, b, d, f));
+            ignore = true;
+        }
+        else
+        {
+            ignore = false;
+            break;
+        }
+    }
+
+    if ( ignore )
+    {
+        dprintk(XENLOG_WARNING VTDPREFIX,
+                " Ignore SATC for seg %04x as no device under its scope is PCI discoverable\n",
+                satcu->segment);
+        return 1;
+    }
+
+    if ( iommu_verbose )
+        printk(VTDPREFIX " ATC required: %d\n", satcu->atc_required);
+
+    list_add(&satcu->list, &acpi_satc_units);
+
+    return ret;
+}
+
+static int __init
+acpi_parse_one_satc(const struct acpi_dmar_header *header)
+{
+    const struct acpi_dmar_satc *satc =
+        container_of(header, const struct acpi_dmar_satc, header);
+    struct acpi_satc_unit *satcu;
+    const void *dev_scope_start, *dev_scope_end;
+    int ret = acpi_dmar_check_length(header, sizeof(*satc));
+
+    if ( ret )
+        return ret;
+
+    satcu = xzalloc(struct acpi_satc_unit);
+    if ( !satcu )
+        return -ENOMEM;
+
+    satcu->segment = satc->segment;
+    satcu->atc_required = satc->flags & ACPI_SATC_ATC_REQUIRED;
+
+    dev_scope_start = (const void *)(satc + 1);
+    dev_scope_end   = (const void *)satc + header->length;
+    ret = acpi_parse_dev_scope(dev_scope_start, dev_scope_end,
+                               &satcu->scope, SATC_TYPE, satc->segment);
+
+    if ( !ret && satcu->scope.devices_cnt )
+        ret = register_one_satc(satcu);
+
+    if ( ret )
+    {
+        scope_devices_free(&satcu->scope);
+        xfree(satcu);
+    }
+
+    /*
+     * register_one_satc() returns greater than 0 when a specified PCIe
+     * device cannot be detected. To prevent VT-d from being disabled in
+     * such cases, make the return value 0 here.
+     */
+    return ret > 0 ? 0 : ret;
+}
+
 static int __init cf_check acpi_parse_dmar(struct acpi_table_header *table)
 {
     struct acpi_table_dmar *dmar;
@@ -803,6 +891,13 @@ static int __init cf_check acpi_parse_dm
                 printk(VTDPREFIX "found ACPI_DMAR_RHSA:\n");
             ret = acpi_parse_one_rhsa(entry_header);
             break;
+
+        case ACPI_DMAR_TYPE_SATC:
+            if ( iommu_verbose )
+                printk(VTDPREFIX "found ACPI_DMAR_SATC:\n");
+            ret = acpi_parse_one_satc(entry_header);
+            break;
+
         default:
             dprintk(XENLOG_WARNING VTDPREFIX,
                     "Ignore unknown DMAR structure type (%#x)\n",
--- a/xen/drivers/passthrough/vtd/dmar.h
+++ b/xen/drivers/passthrough/vtd/dmar.h
@@ -91,6 +91,16 @@ struct acpi_rhsa_unit {
     u32    proximity_domain;
 };
 
+struct acpi_satc_unit {
+    struct dmar_scope scope;
+    struct list_head list;
+    uint16_t segment;
+    bool atc_required:1;
+};
+
+/* In lieu of a definition in actbl2.h. */
+#define ACPI_SATC_ATC_REQUIRED (1U << 0)
+
 #define for_each_drhd_unit(drhd) \
     list_for_each_entry(drhd, &acpi_drhd_units, list)
 
@@ -106,6 +116,7 @@ struct acpi_atsr_unit *acpi_find_matched
 #define DMAR_TYPE 1
 #define RMRR_TYPE 2
 #define ATSR_TYPE 3
+#define SATC_TYPE 4
 
 #define DMAR_OPERATION_TIMEOUT MILLISECS(1000)
 
--- a/xen/include/acpi/actbl2.h
+++ b/xen/include/acpi/actbl2.h
@@ -345,7 +345,8 @@ enum acpi_dmar_type {
 	ACPI_DMAR_TYPE_RESERVED_MEMORY = 1,
 	ACPI_DMAR_TYPE_ATSR = 2,
 	ACPI_DMAR_HARDWARE_AFFINITY = 3,
-	ACPI_DMAR_TYPE_RESERVED = 4	/* 4 and greater are reserved */
+	ACPI_DMAR_TYPE_SATC = 5,
+	ACPI_DMAR_TYPE_RESERVED = 7	/* 7 and greater are reserved */
 };
 
 /* DMAR Device Scope structure */
@@ -427,6 +428,15 @@ struct acpi_dmar_rhsa {
 	u32 proximity_domain;
 };
 
+/* 5: SOC Integrated Address Translation Cache Reporting Structure */
+
+struct acpi_dmar_satc {
+	struct acpi_dmar_header header;
+	uint8_t flags;
+	uint8_t reserved;
+	uint16_t segment;
+};
+
 /*******************************************************************************
  *
  * HPET - High Precision Event Timer table



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:15:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:15:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681488.1060282 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYlx-0001HP-36; Thu, 15 Feb 2024 10:15:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681488.1060282; Thu, 15 Feb 2024 10:15:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYlx-0001HI-0a; Thu, 15 Feb 2024 10:15:17 +0000
Received: by outflank-mailman (input) for mailman id 681488;
 Thu, 15 Feb 2024 10:15:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYlv-0001H1-GX
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:15:15 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1b66a041-cbeb-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 11:15:13 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-33cf7812777so360866f8f.0
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:15:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.15.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:15:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b66a041-cbeb-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992113; x=1708596913; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=0wmyC+dd48xUA9r3hY+U4fMTTXevckC3qoc0ZYZfRhM=;
        b=USz6MfuGjH/VMGz2zKdeerjYyC7kDh6JwBzQ8044JHn24DXU7ohqZOKuvoza72pRKK
         3chya4b/n7xptNoAKF/z4kUG8xFmOaAlplD4YAfoJKC6BFOCRYJcgbo2o272oSZnhbQr
         qS0gDHtiu/pHHMO+APqHLEKFZH71ItpC5qvE7BpK7B44ZvxwvDW1WRwXMvYAV2KfKBHg
         wpdrlLWDSP9T2xPhFsYyyPCHEmCEt8Q7u5vmInI2WRN5SqUdYImGaZThgJsDXNR5XXLa
         1AuzbQ9cJ6K+PGv57YjikhDCzWqHueERg+mIOI/rWLkKqDOGMCYWuqyby4jEjjoVZf4F
         rsgg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992113; x=1708596913;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=0wmyC+dd48xUA9r3hY+U4fMTTXevckC3qoc0ZYZfRhM=;
        b=dqyULo5DNb3HClvNF64EoSsWJAhpWf32uAShRYag+7JpzdOxdsZ0vEQQC3uPWIPgNM
         CbpElsY7l63JcsVdYu3+oREOB+0flFrEJXDo84BB6mheWTYtAOy3gRCiqELoSQGAKCJ1
         7ovLXb3bUV9K8TEEOcrF2b8Y0qGck4VE6N5jWPTv7Ng0C23nS+Eyi7CIBN1vFGEtDxCq
         gHiKTpEp/Ux+uoNNOZS7hbmyyqoDu1KThtPSvLbDqSaikvEwMSlJQhreu8VHLqXyvzPY
         FnmSqqy6pb+bJQboCNBLaRhozx8wKE2IRLO2YcRmx2jDRnT7iH4CWGwomhGA32z/12be
         bOOQ==
X-Gm-Message-State: AOJu0Yxm4mFGcrrFcNWRFHuNZw7Xnia1V5Fj3CpJfRI5TKxa6/dJuLzj
	LhrvrrguA4xeAfqmYD+62uZtX42CvS6ciqpmQoS++irJde8VArTk48YJLCTfCTOgw81gzgyuL7M
	=
X-Google-Smtp-Source: AGHT+IGyu8WOKpuuNtyzo0ztbULmjaz0cTNntwvaxaHRhmZlM8/A6Elcft8I2B1XFnN9DAfe5SK+/Q==
X-Received: by 2002:a5d:4a4f:0:b0:33b:5087:c913 with SMTP id v15-20020a5d4a4f000000b0033b5087c913mr1055855wrs.54.1707992112857;
        Thu, 15 Feb 2024 02:15:12 -0800 (PST)
Message-ID: <bd89275f-915a-4925-afbb-494133bb6e65@suse.com>
Date: Thu, 15 Feb 2024 11:15:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 04/12] AMD/IOMMU: add helper to check whether ATS is to be
 used for a device
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Paul Durrant <paul@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The same set of conditions is used in three places, requiring to be kept
in sync. Introduce a helper to centralize these checks.

To allow all parameters of the new helper be pointer-to-const,
iommu_has_cap() also needs its 1st parameter to be constified. Beyond
that further "modernize" that function.

Requested-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -404,9 +404,9 @@ static inline void __free_amd_iommu_tabl
     free_xenheap_pages(table, order);
 }
 
-static inline int iommu_has_cap(struct amd_iommu *iommu, uint32_t bit)
+static inline bool iommu_has_cap(const struct amd_iommu *iommu, unsigned int bit)
 {
-    return !!(iommu->cap.header & (1u << bit));
+    return iommu->cap.header & (1u << bit);
 }
 
 /* access device id field from iommu cmd */
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -114,6 +114,16 @@ static bool any_pdev_behind_iommu(const
     return false;
 }
 
+static bool use_ats(
+    const struct pci_dev *pdev,
+    const struct amd_iommu *iommu,
+    const struct ivrs_mappings *ivrs_dev)
+{
+    return !ivrs_dev->block_ats &&
+           iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
+           pci_ats_device(iommu->seg, pdev->bus, pdev->devfn);
+}
+
 static int __must_check amd_iommu_setup_domain_device(
     struct domain *domain, struct amd_iommu *iommu,
     uint8_t devfn, struct pci_dev *pdev)
@@ -185,9 +195,7 @@ static int __must_check amd_iommu_setup_
         dte->ex = ivrs_dev->dte_allow_exclusion;
         dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
 
-        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
-             !ivrs_dev->block_ats &&
-             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
+        if ( use_ats(pdev, iommu, ivrs_dev) )
             dte->i = ats_enabled;
 
         spin_unlock_irqrestore(&iommu->lock, flags);
@@ -248,9 +256,7 @@ static int __must_check amd_iommu_setup_
         ASSERT(dte->sys_mgt == MASK_EXTR(ivrs_dev->device_flags,
                                          ACPI_IVHD_SYSTEM_MGMT));
 
-        if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
-             !ivrs_dev->block_ats &&
-             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) )
+        if ( use_ats(pdev, iommu, ivrs_dev) )
             ASSERT(dte->i == ats_enabled);
 
         spin_unlock_irqrestore(&iommu->lock, flags);
@@ -268,9 +274,7 @@ static int __must_check amd_iommu_setup_
 
     ASSERT(pcidevs_locked());
 
-    if ( pci_ats_device(iommu->seg, bus, pdev->devfn) &&
-         !ivrs_dev->block_ats &&
-         iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
+    if ( use_ats(pdev, iommu, ivrs_dev) &&
          !pci_ats_enabled(iommu->seg, bus, pdev->devfn) )
     {
         if ( devfn == pdev->devfn )



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:15:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:15:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681490.1060293 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYmM-0001lj-CX; Thu, 15 Feb 2024 10:15:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681490.1060293; Thu, 15 Feb 2024 10:15:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYmM-0001lY-7c; Thu, 15 Feb 2024 10:15:42 +0000
Received: by outflank-mailman (input) for mailman id 681490;
 Thu, 15 Feb 2024 10:15:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYmL-0001ZT-3o
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:15:41 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2b716d5d-cbeb-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:15:40 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-339289fead2so377451f8f.3
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:15:40 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.15.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:15:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b716d5d-cbeb-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992140; x=1708596940; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=gocTw3drw4/MldPbi03GFmG0diUgBYkMjWI02M8R94k=;
        b=YoeDS2lZx4qkPuxr6nyw8h4lIMFW2T+tijVEWGdk7iFQA6mMyUmtVw6C5cHH1SnL7E
         M/ov3lqSlZSoA9JzfHIbvDNtmdqQh2eCpXURvjBicmAE+riF52fCH3Ctmuc7SOTlPWUH
         wjrS2+cNs0EHVC0nlKXZvPL9GZcEIqnwbKzChH18xJcrZyVLeZ2LVLwFEq45lIvWU6Zw
         NpI53bICnXZZwn15arEevfsOS2fZ59URfu32x5733VD5LnUpgCTDT8RsFgc86jVNMgoq
         XkknDeTmpzmx4X3OodrL6JMFRb86vSal6m8Uoxso+hHepjRuhcLedxTuD5ns8oFOMkAD
         j3AQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992140; x=1708596940;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=gocTw3drw4/MldPbi03GFmG0diUgBYkMjWI02M8R94k=;
        b=D9TeeT80adyBIGUxbeSLdFXANcUrl3XPzt97YSQpLvQO0VFpNRaclD7pCnPuZblPxQ
         muaOnbY9WW/GYU5G32+gBNVL2bRcaBERLuKsEHlZA9eVgwjSiH3w6mGRIEjqHIy4/aOO
         hu7Yyy+7Q234nK5U8H6bMCNhJttj1oA5c4hcebi9cwnSRzHwufmQDQwgflbROxfF3Q2A
         NVAYL6lAdQUFYnMW+V0WMV4e6FrYAMU9wQQnjqn+oqQ4r7VWVxu7BatWw1GngNA2t0qr
         BddkzBevczdU2Z3jEuOMate56TfzazNOgCQFwiqNsMqDi/SEvF7+fuHwq0JRrI01CsXA
         hleg==
X-Gm-Message-State: AOJu0Yzp0biwbjAB7Two7fF4kFS8yVxX97sv3TnTnmeXD6ZxGDPg4LIi
	KFS4D6dH1z4uzCd/NBHwvgtLFcleien85IsNDi5rPkb2VrwXxrJli4kVwN42rmT+HCjyY/vTYdw
	=
X-Google-Smtp-Source: AGHT+IEoOerVJ4qI6jh0D6P7vaV4hh6KoDawu6j/5/bl736eUS0wdzKYkiDCBkV8iHrfET6mnp+gkQ==
X-Received: by 2002:a5d:660e:0:b0:33c:f9d6:fb20 with SMTP id n14-20020a5d660e000000b0033cf9d6fb20mr1025439wru.45.1707992139790;
        Thu, 15 Feb 2024 02:15:39 -0800 (PST)
Message-ID: <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com>
Date: Thu, 15 Feb 2024 11:15:39 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 05/12] IOMMU: rename and re-type ats_enabled
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Make the variable a tristate, with (as done elsewhere) a negative value
meaning "default". Since all use sites need looking at, also rename it
to match our usual "opt_*" pattern. While touching it, also move it to
.data.ro_after_init.

The only place it retains boolean nature is pci_ats_device(), for now.

In AMD code re-order conditionals to have the config space accesses
after (cheaper) flag checks.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
In domain_context_mapping_one() I'm a little puzzled that translation
type is selected based on only IOMMU and global properties, i.e. not
taking the device itself into account.
---
v2: Re-base over new earlier patches.

--- a/xen/drivers/passthrough/amd/iommu_cmd.c
+++ b/xen/drivers/passthrough/amd/iommu_cmd.c
@@ -282,7 +282,7 @@ void amd_iommu_flush_iotlb(u8 devfn, con
     struct amd_iommu *iommu;
     unsigned int req_id, queueid, maxpend;
 
-    if ( !ats_enabled )
+    if ( opt_ats <= 0 )
         return;
 
     if ( !pci_ats_enabled(pdev->seg, pdev->bus, pdev->devfn) )
@@ -340,7 +340,7 @@ static void _amd_iommu_flush_pages(struc
         flush_command_buffer(iommu, 0);
     }
 
-    if ( ats_enabled )
+    if ( opt_ats > 0 )
     {
         amd_iommu_flush_all_iotlbs(d, daddr, order);
 
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -119,7 +119,7 @@ static bool use_ats(
     const struct amd_iommu *iommu,
     const struct ivrs_mappings *ivrs_dev)
 {
-    return !ivrs_dev->block_ats &&
+    return opt_ats > 0 && !ivrs_dev->block_ats &&
            iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
            pci_ats_device(iommu->seg, pdev->bus, pdev->devfn);
 }
@@ -196,7 +196,7 @@ static int __must_check amd_iommu_setup_
         dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
 
         if ( use_ats(pdev, iommu, ivrs_dev) )
-            dte->i = ats_enabled;
+            dte->i = true;
 
         spin_unlock_irqrestore(&iommu->lock, flags);
 
@@ -257,7 +257,7 @@ static int __must_check amd_iommu_setup_
                                          ACPI_IVHD_SYSTEM_MGMT));
 
         if ( use_ats(pdev, iommu, ivrs_dev) )
-            ASSERT(dte->i == ats_enabled);
+            ASSERT(dte->i);
 
         spin_unlock_irqrestore(&iommu->lock, flags);
 
--- a/xen/drivers/passthrough/ats.c
+++ b/xen/drivers/passthrough/ats.c
@@ -18,8 +18,8 @@
 #include <xen/pci_regs.h>
 #include "ats.h"
 
-bool __read_mostly ats_enabled;
-boolean_param("ats", ats_enabled);
+int8_t __ro_after_init opt_ats = -1;
+boolean_param("ats", opt_ats);
 
 int enable_ats_device(struct pci_dev *pdev, struct list_head *ats_list)
 {
--- a/xen/drivers/passthrough/ats.h
+++ b/xen/drivers/passthrough/ats.h
@@ -22,7 +22,7 @@
 #define ATS_QUEUE_DEPTH_MASK     0x1f
 #define ATS_ENABLE               (1<<15)
 
-extern bool ats_enabled;
+extern int8_t opt_ats;
 
 int enable_ats_device(struct pci_dev *pdev, struct list_head *ats_list);
 void disable_ats_device(struct pci_dev *pdev);
@@ -43,7 +43,7 @@ static inline int pci_ats_enabled(int se
 
 static inline int pci_ats_device(int seg, int bus, int devfn)
 {
-    if ( !ats_enabled )
+    if ( !opt_ats )
         return 0;
 
     return pci_find_ext_capability(PCI_SBDF(seg, bus, devfn),
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1543,7 +1543,7 @@ int domain_context_mapping_one(
         }
 
         context_set_address_root(lctxt, root);
-        if ( ats_enabled && ecap_dev_iotlb(iommu->ecap) )
+        if ( opt_ats > 0 && ecap_dev_iotlb(iommu->ecap) )
             context_set_translation_type(lctxt, CONTEXT_TT_DEV_IOTLB);
         else
             context_set_translation_type(lctxt, CONTEXT_TT_MULTI_LEVEL);
--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ b/xen/drivers/passthrough/vtd/x86/ats.c
@@ -46,7 +46,7 @@ int ats_device(const struct pci_dev *pde
     struct acpi_drhd_unit *ats_drhd;
     unsigned int pos, expfl = 0;
 
-    if ( !ats_enabled || !iommu_qinval )
+    if ( opt_ats <= 0 || !iommu_qinval )
         return 0;
 
     if ( !ecap_queued_inval(drhd->iommu->ecap) ||



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:16:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:16:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681497.1060302 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYmt-0002Nm-Ko; Thu, 15 Feb 2024 10:16:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681497.1060302; Thu, 15 Feb 2024 10:16:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYmt-0002Nf-IF; Thu, 15 Feb 2024 10:16:15 +0000
Received: by outflank-mailman (input) for mailman id 681497;
 Thu, 15 Feb 2024 10:16:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYmr-0001ZT-CG
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:16:13 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3eb48278-cbeb-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:16:12 +0100 (CET)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2d0d95e8133so8328621fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:16:12 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.16.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:16:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3eb48278-cbeb-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992172; x=1708596972; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=pvYCja/S6ezysFDR56QyJhiQ4LakZ7EGAU+IRQcHkow=;
        b=E8F7ztVqfJzJ4XvJ5Ep2pMkHhXV/cb7Po+L6EML++7dr1c0HpG4Jqn+KkDnkQgtR6P
         4aUMYMNfDBn0bgUq2TvpSp/s7vZf+CkOVccOuR4p4KV7cSrh2yH+CBQBmiRa6P273hPF
         Td7yflXf89uoGWsir1x4zEMQe7W5udHa9SpQn6Mw+lL31W/ptNzwZ/3rHqbzk/tdfXLy
         UDA3hm7BshaHpUZ6MOzua0NdpLAWmcHBUxVd9nxu6T7FsuEK1jJii01lEPGKwjSjT8p7
         pyWc+FDR/c8C3jRKyXXT49YMSSt5uNQ5MywfsE6stm/OZpNDjL8m9Jf6u6JU8RuJmljY
         Pu2g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992172; x=1708596972;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=pvYCja/S6ezysFDR56QyJhiQ4LakZ7EGAU+IRQcHkow=;
        b=Qrhfw8gk1TbGXJGEVVnYHX9fei6L7WS/2KmhPyDKa5rhAEVlpjNt9NbJ+B8IFlk+uc
         AJy9Rxaph6v56NLVYGg+t3p9uyDe7aLtU0ObSb2Dgh0CHXhTa9cFmsSVJg7IVUGFCdly
         y0kqATwHsx+8RgNioL1pW0dQ6D6VIGUsSIgNsbGUjTt+LkHGruXDu95Z6B1b8+DcQldi
         2uEDCLBmeifGluEeLa10QLgmHqS+VavdlNa0KDAGNUedqEM1xCnK/wJwZY6ztRxZ4Py7
         rYe12OdGCuFHp4sUER1SivF0z/DJITflIN7+KL91jK2UFfGW3poR3BB+hdt7zJKyZZ+5
         M7+w==
X-Gm-Message-State: AOJu0YyhunNqMy3wITLvUAMzumBu9SnK4WYXFeFFxLcm4oJfcrxVKx8y
	rfPqjlXsSw3xlSSf8h351P3sNDYvPGrapOjmtpGkSuVMl9TnuWY/jqX83SN2EGO4rZ5YHD7ZgHI
	=
X-Google-Smtp-Source: AGHT+IHaNfJ1XoIWuOA4FB8Cfvo2/hwJaB1FIeDk2PiK1bwBIgOwZw0rqaqNJdskZ8Y97NbZ1KnyeA==
X-Received: by 2002:a2e:a171:0:b0:2d0:a90b:e4e3 with SMTP id u17-20020a2ea171000000b002d0a90be4e3mr844303ljl.27.1707992172169;
        Thu, 15 Feb 2024 02:16:12 -0800 (PST)
Message-ID: <e98daa41-c6b6-4f4e-b41d-84006011068d@suse.com>
Date: Thu, 15 Feb 2024 11:16:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 06/12] VT-d: respect ACPI SATC's ATC_REQUIRED flag
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

When the flag is set, permit Dom0 to control the device (no worse than
what we had before and in line with other "best effort" behavior we use
when it comes to Dom0), but suppress passing through to DomU-s unless
ATS can actually be enabled for such devices (and was explicitly enabled
on the command line).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: Re-base over new earlier patches.

--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -225,7 +225,11 @@ exceptions (watchdog NMIs and unexpected
 > Default: `false`
 
 Permits Xen to set up and use PCI Address Translation Services.  This is a
-performance optimisation for PCI Passthrough.
+performance optimisation for PCI Passthrough.  Note that firmware may indicate
+that certain devices need to have ATS enabled for proper operation. For such
+devices ATS will be enabled by default, unless the option is used in its
+negative form.  Such devices will still not be eligible for passing through to
+guests, unless the option is used in its positive form.
 
 **WARNING: Xen cannot currently safely use ATS because of its synchronous wait
 loops for Queued Invalidation completions.**
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -253,6 +253,24 @@ struct acpi_atsr_unit *acpi_find_matched
     return all_ports;
 }
 
+const struct acpi_satc_unit *acpi_find_matched_satc_unit(
+    const struct pci_dev *pdev)
+{
+    const struct acpi_satc_unit *satc;
+
+    list_for_each_entry ( satc, &acpi_satc_units, list )
+    {
+        if ( satc->segment != pdev->seg )
+            continue;
+
+        for ( unsigned int i = 0; i < satc->scope.devices_cnt; ++i )
+            if ( satc->scope.devices[i] == pdev->sbdf.bdf )
+                return satc;
+    }
+
+    return NULL;
+}
+
 struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd)
 {
     struct acpi_rhsa_unit *rhsa;
--- a/xen/drivers/passthrough/vtd/dmar.h
+++ b/xen/drivers/passthrough/vtd/dmar.h
@@ -112,6 +112,8 @@ struct acpi_satc_unit {
 
 struct acpi_drhd_unit *acpi_find_matched_drhd_unit(const struct pci_dev *);
 struct acpi_atsr_unit *acpi_find_matched_atsr_unit(const struct pci_dev *);
+const struct acpi_satc_unit *acpi_find_matched_satc_unit(
+    const struct pci_dev *pdev);
 
 #define DMAR_TYPE 1
 #define RMRR_TYPE 2
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -2364,6 +2364,26 @@ static int cf_check intel_iommu_add_devi
     if ( ret )
         dprintk(XENLOG_ERR VTDPREFIX, "%pd: context mapping failed\n",
                 pdev->domain);
+    else if ( !pdev->broken )
+    {
+        const struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
+        const struct acpi_satc_unit *satc = acpi_find_matched_satc_unit(pdev);
+
+        /*
+         * Prevent the device from getting assigned to an unprivileged domain
+         * when firmware indicates ATS is required, but ATS could not be enabled
+         * or was not explicitly enabled via command line option.
+         */
+        if ( satc && satc->atc_required &&
+             (!drhd || ats_device(pdev, drhd) <= 0 ||
+              !pci_ats_enabled(pdev->seg, pdev->bus, pdev->devfn) ||
+              opt_ats < 0) )
+        {
+            printk(XENLOG_WARNING "ATS: %pp is not eligible for pass-through\n",
+                   &pdev->sbdf);
+            pdev->broken = true;
+        }
+    }
 
     return ret;
 }
@@ -2375,12 +2395,26 @@ static int cf_check intel_iommu_enable_d
 
     pci_vtd_quirk(pdev);
 
-    if ( ret <= 0 )
-        return ret;
+    if ( ret <= 0 ||
+         (ret = enable_ats_device(pdev, &drhd->iommu->ats_devices)) < 0 ||
+         opt_ats < 0 )
+    {
+        const struct acpi_satc_unit *satc = acpi_find_matched_satc_unit(pdev);
+
+        /*
+         * Besides in error cases also prevent the device from getting assigned
+         * to an unprivileged domain when firmware indicates ATS is required,
+         * but ATS use was not explicitly enabled via command line option.
+         */
+        if ( satc && satc->atc_required && !pdev->broken )
+        {
+            printk(XENLOG_WARNING "ATS: %pp is not eligible for pass-through\n",
+                   &pdev->sbdf);
+            pdev->broken = true;
+        }
+    }
 
-    ret = enable_ats_device(pdev, &drhd->iommu->ats_devices);
-
-    return ret >= 0 ? 0 : ret;
+    return ret <= 0 ? ret : 0;
 }
 
 static int cf_check intel_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ b/xen/drivers/passthrough/vtd/x86/ats.c
@@ -45,8 +45,9 @@ int ats_device(const struct pci_dev *pde
 {
     struct acpi_drhd_unit *ats_drhd;
     unsigned int pos, expfl = 0;
+    const struct acpi_satc_unit *satc;
 
-    if ( opt_ats <= 0 || !iommu_qinval )
+    if ( !opt_ats || !iommu_qinval )
         return 0;
 
     if ( !ecap_queued_inval(drhd->iommu->ecap) ||
@@ -61,6 +62,10 @@ int ats_device(const struct pci_dev *pde
          !acpi_find_matched_atsr_unit(pdev) )
         return 0;
 
+    satc = acpi_find_matched_satc_unit(pdev);
+    if ( opt_ats < 0 && (!satc || !satc->atc_required) )
+        return 0;
+
     ats_drhd = find_ats_dev_drhd(drhd->iommu);
     pos = pci_find_ext_capability(pdev->sbdf, PCI_EXT_CAP_ID_ATS);
 



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:16:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:16:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681501.1060313 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYnE-0002sO-TQ; Thu, 15 Feb 2024 10:16:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681501.1060313; Thu, 15 Feb 2024 10:16:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYnE-0002sH-Qn; Thu, 15 Feb 2024 10:16:36 +0000
Received: by outflank-mailman (input) for mailman id 681501;
 Thu, 15 Feb 2024 10:16:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYnD-0001ZT-Rs
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:16:35 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4c127b56-cbeb-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:16:35 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-33b4e6972f6so277487f8f.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:16:35 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.16.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:16:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c127b56-cbeb-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992194; x=1708596994; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=XNeK3bYqgsZfsvAYZefEQLICnz9K9i5e8fWmt9YKjrw=;
        b=FAAFF1AcP2IY3bkknHl7wbK8GpMFJf2q1TRCaw7K1nvQcsoqWESng/m023h/HW+uU2
         dQ0B9JDHjNXoFVCS639eLWlj7v3w4sWTLTzyqLTbQlhHW7r/aUKP8x/uxFFcVMqvtkAL
         faCK0mdlou7aVzYxpGuPLLki3AzcWPLSIbK+JOuSIXaNNX7hdGBKuzaMmrB54kPC7lcQ
         HKGf14iar+ATuyfQJKdxlGlkqpE1XnchjvtjNA5XW8dRWna0D2Zb7E4UxCjSkfLyyDex
         1ZtmsyRwEt9WvsvmMpdqzmRbWMScVEANTB3N2lIEOFW7j62UOHxgSllR43rey7VeYZ+I
         Mo3w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992194; x=1708596994;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=XNeK3bYqgsZfsvAYZefEQLICnz9K9i5e8fWmt9YKjrw=;
        b=tK3WkF4J0zxfoAWVzOG/hpVt/eM51oHjtY2g4/SPxLqx74wcsFpgo631e0kxDiI9UX
         EfRF/JbXpIoVG/d18iYoWLuKRW2fIeMADCWRRLC9eNLdQJP9XH0y8mLHPBdIADcZr66H
         tQS7VgwwdLvpIetrBeJloFJwTH2bCFWPA5/Y6j91Uocnh5enbg4etBxddpWvHPIGaTET
         bS+3+RqRWMB29cozqN3jisgFI+ope9Fcdd1RxwEC/oei0wPprTOAh7dtwX5rh1BYEycJ
         rfsmukQ7gj1Y9QwWQ16DoTdeu2miK1/bxjJJAqBjNQ1OyvN5tnvKrqfZgi3RQUvj6xUZ
         XOFA==
X-Gm-Message-State: AOJu0YzjP0SS6aNUWENzMoAP2YCiETQHMIT15kKwKTTQJ4DOL475wuk3
	zE9mEVVl9Q1NvXLQlmMOMfM8FqyIgWQ6qh0LTOC+hGXEiTgzoe/9qR2D5CWRniK8HU2DuePBeT4
	=
X-Google-Smtp-Source: AGHT+IEBToj/5jt4OtESG6EbViirEv991GlvJtBi+TFX2uxp33Hn+F84evlIFXdE+Q/+B48v8GDU0A==
X-Received: by 2002:a5d:6882:0:b0:33b:6d5e:7645 with SMTP id h2-20020a5d6882000000b0033b6d5e7645mr759086wru.38.1707992194487;
        Thu, 15 Feb 2024 02:16:34 -0800 (PST)
Message-ID: <bbb0e7cd-b5c9-4227-8f56-0d3e159ff7b2@suse.com>
Date: Thu, 15 Feb 2024 11:16:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 07/12] VT-d: replace find_ats_dev_drhd()
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

All callers only care about boolean outcome. For this there's no point
in allocating a duplicate of the respective DRHD structure; a simple
boolean suffices (which eventually may want to become a count, such that
the "any ATS devices assigned state" can also clear again). With that
boolean, remove respective parameters from internal helper functions
right away, as those have access to the flag through another parameter.
(While the corresponding hook can have the parameter dropped too then,
leave this for a separate change, to limit scope here.)

No functional effect intended.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
---
v2: Re-base over new earlier patches.

--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -65,8 +65,6 @@ struct acpi_drhd_unit *ioapic_to_drhd(un
 struct acpi_drhd_unit *hpet_to_drhd(unsigned int hpet_id);
 struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd);
 
-struct acpi_drhd_unit *find_ats_dev_drhd(struct vtd_iommu *iommu);
-
 int ats_device(const struct pci_dev *, const struct acpi_drhd_unit *);
 
 int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -624,8 +624,7 @@ int cf_check vtd_flush_iotlb_reg(
 }
 
 static int __must_check iommu_flush_iotlb_global(struct vtd_iommu *iommu,
-                                                 bool flush_non_present_entry,
-                                                 bool flush_dev_iotlb)
+                                                 bool flush_non_present_entry)
 {
     int status;
 
@@ -633,7 +632,7 @@ static int __must_check iommu_flush_iotl
     vtd_ops_preamble_quirk(iommu);
 
     status = iommu->flush.iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH,
-                                flush_non_present_entry, flush_dev_iotlb);
+                                flush_non_present_entry, iommu->flush_dev_iotlb);
 
     /* undo platform specific errata workarounds */
     vtd_ops_postamble_quirk(iommu);
@@ -642,8 +641,7 @@ static int __must_check iommu_flush_iotl
 }
 
 static int __must_check iommu_flush_iotlb_dsi(struct vtd_iommu *iommu, u16 did,
-                                              bool flush_non_present_entry,
-                                              bool flush_dev_iotlb)
+                                              bool flush_non_present_entry)
 {
     int status;
 
@@ -651,7 +649,7 @@ static int __must_check iommu_flush_iotl
     vtd_ops_preamble_quirk(iommu);
 
     status = iommu->flush.iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH,
-                                flush_non_present_entry, flush_dev_iotlb);
+                                flush_non_present_entry, iommu->flush_dev_iotlb);
 
     /* undo platform specific errata workarounds */
     vtd_ops_postamble_quirk(iommu);
@@ -661,26 +659,23 @@ static int __must_check iommu_flush_iotl
 
 static int __must_check iommu_flush_iotlb_psi(struct vtd_iommu *iommu, u16 did,
                                               u64 addr, unsigned int order,
-                                              bool flush_non_present_entry,
-                                              bool flush_dev_iotlb)
+                                              bool flush_non_present_entry)
 {
     int status;
 
     /* Fallback to domain selective flush if no PSI support */
     if ( !cap_pgsel_inv(iommu->cap) )
-        return iommu_flush_iotlb_dsi(iommu, did, flush_non_present_entry,
-                                     flush_dev_iotlb);
+        return iommu_flush_iotlb_dsi(iommu, did, flush_non_present_entry);
 
     /* Fallback to domain selective flush if size is too big */
     if ( order > cap_max_amask_val(iommu->cap) )
-        return iommu_flush_iotlb_dsi(iommu, did, flush_non_present_entry,
-                                     flush_dev_iotlb);
+        return iommu_flush_iotlb_dsi(iommu, did, flush_non_present_entry);
 
     /* apply platform specific errata workarounds */
     vtd_ops_preamble_quirk(iommu);
 
     status = iommu->flush.iotlb(iommu, did, addr, order, DMA_TLB_PSI_FLUSH,
-                                flush_non_present_entry, flush_dev_iotlb);
+                                flush_non_present_entry, iommu->flush_dev_iotlb);
 
     /* undo platform specific errata workarounds */
     vtd_ops_postamble_quirk(iommu);
@@ -692,7 +687,6 @@ static int __must_check iommu_flush_all(
 {
     struct acpi_drhd_unit *drhd;
     struct vtd_iommu *iommu;
-    bool flush_dev_iotlb;
     int rc = 0;
 
     flush_local(FLUSH_CACHE);
@@ -703,8 +697,7 @@ static int __must_check iommu_flush_all(
 
         iommu = drhd->iommu;
         context_rc = iommu_flush_context_global(iommu, 0);
-        flush_dev_iotlb = !!find_ats_dev_drhd(iommu);
-        iotlb_rc = iommu_flush_iotlb_global(iommu, 0, flush_dev_iotlb);
+        iotlb_rc = iommu_flush_iotlb_global(iommu, 0);
 
         /*
          * The current logic for returns:
@@ -734,7 +727,6 @@ static int __must_check cf_check iommu_f
     struct domain_iommu *hd = dom_iommu(d);
     struct acpi_drhd_unit *drhd;
     struct vtd_iommu *iommu;
-    bool flush_dev_iotlb;
     int iommu_domid;
     int ret = 0;
 
@@ -762,21 +754,18 @@ static int __must_check cf_check iommu_f
         if ( !test_bit(iommu->index, hd->arch.vtd.iommu_bitmap) )
             continue;
 
-        flush_dev_iotlb = !!find_ats_dev_drhd(iommu);
         iommu_domid = get_iommu_did(d->domain_id, iommu, !d->is_dying);
         if ( iommu_domid == -1 )
             continue;
 
         if ( !page_count || (page_count & (page_count - 1)) ||
              dfn_eq(dfn, INVALID_DFN) || !IS_ALIGNED(dfn_x(dfn), page_count) )
-            rc = iommu_flush_iotlb_dsi(iommu, iommu_domid,
-                                       0, flush_dev_iotlb);
+            rc = iommu_flush_iotlb_dsi(iommu, iommu_domid, 0);
         else
             rc = iommu_flush_iotlb_psi(iommu, iommu_domid,
                                        dfn_to_daddr(dfn),
                                        get_order_from_pages(page_count),
-                                       !(flush_flags & IOMMU_FLUSHF_modified),
-                                       flush_dev_iotlb);
+                                       !(flush_flags & IOMMU_FLUSHF_modified));
 
         if ( rc > 0 )
             iommu_flush_write_buffer(iommu);
@@ -1488,7 +1477,6 @@ int domain_context_mapping_one(
     uint16_t seg = iommu->drhd->segment, prev_did = 0;
     struct domain *prev_dom = NULL;
     int rc, ret;
-    bool flush_dev_iotlb;
 
     if ( QUARANTINE_SKIP(domain, pgd_maddr) )
         return 0;
@@ -1637,8 +1625,7 @@ int domain_context_mapping_one(
 
     rc = iommu_flush_context_device(iommu, prev_did, PCI_BDF(bus, devfn),
                                     DMA_CCMD_MASK_NOBIT, !prev_dom);
-    flush_dev_iotlb = !!find_ats_dev_drhd(iommu);
-    ret = iommu_flush_iotlb_dsi(iommu, prev_did, !prev_dom, flush_dev_iotlb);
+    ret = iommu_flush_iotlb_dsi(iommu, prev_did, !prev_dom);
 
     /*
      * The current logic for returns:
@@ -1882,7 +1869,6 @@ int domain_context_unmap_one(
     struct context_entry *context, *context_entries;
     u64 maddr;
     int iommu_domid, rc, ret;
-    bool flush_dev_iotlb;
 
     ASSERT(pcidevs_locked());
     spin_lock(&iommu->lock);
@@ -1908,8 +1894,7 @@ int domain_context_unmap_one(
                                     PCI_BDF(bus, devfn),
                                     DMA_CCMD_MASK_NOBIT, 0);
 
-    flush_dev_iotlb = !!find_ats_dev_drhd(iommu);
-    ret = iommu_flush_iotlb_dsi(iommu, iommu_domid, 0, flush_dev_iotlb);
+    ret = iommu_flush_iotlb_dsi(iommu, iommu_domid, 0);
 
     /*
      * The current logic for returns:
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -484,6 +484,7 @@ struct vtd_iommu {
     spinlock_t register_lock; /* protect iommu register handling */
     u64 root_maddr; /* root entry machine address */
     nodeid_t node;
+    bool flush_dev_iotlb;
     struct msi_desc msi;
     struct acpi_drhd_unit *drhd;
 
--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ b/xen/drivers/passthrough/vtd/x86/ats.c
@@ -28,22 +28,8 @@
 #include "../extern.h"
 #include "../../ats.h"
 
-static LIST_HEAD(ats_dev_drhd_units);
-
-struct acpi_drhd_unit *find_ats_dev_drhd(struct vtd_iommu *iommu)
-{
-    struct acpi_drhd_unit *drhd;
-    list_for_each_entry ( drhd, &ats_dev_drhd_units, list )
-    {
-        if ( drhd->iommu == iommu )
-            return drhd;
-    }
-    return NULL;
-}
-
 int ats_device(const struct pci_dev *pdev, const struct acpi_drhd_unit *drhd)
 {
-    struct acpi_drhd_unit *ats_drhd;
     unsigned int pos, expfl = 0;
     const struct acpi_satc_unit *satc;
 
@@ -66,17 +52,10 @@ int ats_device(const struct pci_dev *pde
     if ( opt_ats < 0 && (!satc || !satc->atc_required) )
         return 0;
 
-    ats_drhd = find_ats_dev_drhd(drhd->iommu);
     pos = pci_find_ext_capability(pdev->sbdf, PCI_EXT_CAP_ID_ATS);
+    if ( pos )
+        drhd->iommu->flush_dev_iotlb = true;
 
-    if ( pos && (ats_drhd == NULL) )
-    {
-        ats_drhd = xmalloc(struct acpi_drhd_unit);
-        if ( !ats_drhd )
-            return -ENOMEM;
-        *ats_drhd = *drhd;
-        list_add_tail(&ats_drhd->list, &ats_dev_drhd_units);
-    }
     return pos;
 }
 



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:17:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:17:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681508.1060323 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYoJ-0003UX-62; Thu, 15 Feb 2024 10:17:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681508.1060323; Thu, 15 Feb 2024 10:17:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYoJ-0003UQ-3E; Thu, 15 Feb 2024 10:17:43 +0000
Received: by outflank-mailman (input) for mailman id 681508;
 Thu, 15 Feb 2024 10:17:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYoH-0003UC-P1
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:17:41 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 72f11edc-cbeb-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:17:40 +0100 (CET)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2d0f7585d89so7928721fa.3
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:17:40 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.17.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:17:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 72f11edc-cbeb-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992260; x=1708597060; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=S2x+PfdcoJsKEcm0haAnCx/jYnp6JxJ2K1CdYjIZgsY=;
        b=Dg+PeYY4g7ZFSrD2W2hUwW5aXV8YhLSoQlPeN8H6B9opyo1Cl3DtFKjDHpfU6Saxzd
         8hkWxWcf7s1zyiep8ZHAj9wvy3ArN16M8xMtgQdISuvo51mImOncpNqx/QXP4UPRbfil
         YSRzOcbP04x36MXUgs5KQyq/gItiQr5LUNTaIri2TRMp3LSKxpsKSjBmR1sSnwvhheSR
         lap1LGUc9cguEJx+i+845aybJpsigz4+Yh5NFya0ie+i78mkUdEBqcZWHQLEYAUz16Rg
         bp8XXwmsUIYExaZXoSkoHYJ+X2wc45Su/SqfUyu7gw2Pl8FbS6otl03nb8eaEfK6My/B
         fijQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992260; x=1708597060;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=S2x+PfdcoJsKEcm0haAnCx/jYnp6JxJ2K1CdYjIZgsY=;
        b=d37m98UOpBrnHA19aTO/2DtI/54axGbNpAryAEvJ4gKz140A46gXDua709ioxlB8SJ
         FjXUdMeCbv10UfULxZ9zjGQ4YLTcYqaQEsov9WR5tq3JLvG+d6DU47d1uOEvCzN86Tt5
         PUFE4s4wlJ87sNsFz4k6z2I4/hGuqSeIlgSyF7wj1OSBU1mfrKYFEYtYdTRiBvmP2faS
         Vtzglhpy0t4sio3rwyYkTiFVxCad0C/rsMB/AYUG2FgROJJYCxOf/bgxhpP5jytITkST
         xCIdTTNvjOWkHik4rbABBwMxPELfSaz4p3k0dj0MXWuGiEgZtOSN5jMQ2AnDHeV6PQwZ
         /3mg==
X-Gm-Message-State: AOJu0YwapNt3GckduHgtvxViUTpU45ySNnXW3lJZ3w+bt80t8tE8BmLB
	wKJGMs0FtAqltrYuGH1ayZOYZMRKld18k3YjRIq+7bvptTWIV7hzrfhPjeSCB13T/YFEFFdeT/Q
	=
X-Google-Smtp-Source: AGHT+IF2ihrYWltASVeLsOU7+3qM+8Hfay8U7bH1prne8Az8dp65xgQ56op0n2UWrhK6Bm6HsDI41w==
X-Received: by 2002:a2e:9e10:0:b0:2d1:10cf:e887 with SMTP id e16-20020a2e9e10000000b002d110cfe887mr1135113ljk.29.1707992259760;
        Thu, 15 Feb 2024 02:17:39 -0800 (PST)
Message-ID: <0a30cf57-93ca-4ae8-a125-48c155f1f5a5@suse.com>
Date: Thu, 15 Feb 2024 11:17:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 08/12] VT-d: move ats_device() to the sole file it's used
 from
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

..., thus allowing it to become static, and thus reducing scope overlap
between it and pci_ats_device(). There's nothing x86-specific about this
function anyway.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>

--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -65,8 +65,6 @@ struct acpi_drhd_unit *ioapic_to_drhd(un
 struct acpi_drhd_unit *hpet_to_drhd(unsigned int hpet_id);
 struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd);
 
-int ats_device(const struct pci_dev *, const struct acpi_drhd_unit *);
-
 int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
                          u64 addr, unsigned int size_order, u64 type);
 
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1455,6 +1455,38 @@ static void __hwdom_init cf_check intel_
     }
 }
 
+static int ats_device(const struct pci_dev *pdev,
+                      const struct acpi_drhd_unit *drhd)
+{
+    unsigned int pos, expfl = 0;
+    const struct acpi_satc_unit *satc;
+
+    if ( !opt_ats || !iommu_qinval )
+        return 0;
+
+    if ( !ecap_queued_inval(drhd->iommu->ecap) ||
+         !ecap_dev_iotlb(drhd->iommu->ecap) )
+        return 0;
+
+    pos = pci_find_cap_offset(pdev->sbdf, PCI_CAP_ID_EXP);
+    if ( pos )
+        expfl = pci_conf_read16(pdev->sbdf, pos + PCI_EXP_FLAGS);
+
+    if ( MASK_EXTR(expfl, PCI_EXP_FLAGS_TYPE) != PCI_EXP_TYPE_RC_END &&
+         !acpi_find_matched_atsr_unit(pdev) )
+        return 0;
+
+    satc = acpi_find_matched_satc_unit(pdev);
+    if ( opt_ats < 0 && (!satc || !satc->atc_required) )
+        return 0;
+
+    pos = pci_find_ext_capability(pdev->sbdf, PCI_EXT_CAP_ID_ATS);
+    if ( pos )
+        drhd->iommu->flush_dev_iotlb = true;
+
+    return pos;
+}
+
 /*
  * This function returns
  * - a negative errno value upon error,
--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ b/xen/drivers/passthrough/vtd/x86/ats.c
@@ -28,37 +28,6 @@
 #include "../extern.h"
 #include "../../ats.h"
 
-int ats_device(const struct pci_dev *pdev, const struct acpi_drhd_unit *drhd)
-{
-    unsigned int pos, expfl = 0;
-    const struct acpi_satc_unit *satc;
-
-    if ( !opt_ats || !iommu_qinval )
-        return 0;
-
-    if ( !ecap_queued_inval(drhd->iommu->ecap) ||
-         !ecap_dev_iotlb(drhd->iommu->ecap) )
-        return 0;
-
-    pos = pci_find_cap_offset(pdev->sbdf, PCI_CAP_ID_EXP);
-    if ( pos )
-        expfl = pci_conf_read16(pdev->sbdf, pos + PCI_EXP_FLAGS);
-
-    if ( MASK_EXTR(expfl, PCI_EXP_FLAGS_TYPE) != PCI_EXP_TYPE_RC_END &&
-         !acpi_find_matched_atsr_unit(pdev) )
-        return 0;
-
-    satc = acpi_find_matched_satc_unit(pdev);
-    if ( opt_ats < 0 && (!satc || !satc->atc_required) )
-        return 0;
-
-    pos = pci_find_ext_capability(pdev->sbdf, PCI_EXT_CAP_ID_ATS);
-    if ( pos )
-        drhd->iommu->flush_dev_iotlb = true;
-
-    return pos;
-}
-
 static bool device_in_domain(const struct vtd_iommu *iommu,
                              const struct pci_dev *pdev, uint16_t did)
 {



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:18:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:18:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681514.1060333 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYop-00044x-Hr; Thu, 15 Feb 2024 10:18:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681514.1060333; Thu, 15 Feb 2024 10:18:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYop-00044q-DT; Thu, 15 Feb 2024 10:18:15 +0000
Received: by outflank-mailman (input) for mailman id 681514;
 Thu, 15 Feb 2024 10:18:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYoo-00043N-E2
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:18:14 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 869c8619-cbeb-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:18:13 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-41211acfcfcso4294975e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:18:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.18.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:18:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 869c8619-cbeb-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992293; x=1708597093; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=HZHY6xKr+5lB7TS1xmNKRfrrhDk5jvTKT8Lq2YpKq9o=;
        b=UrRsqRjpmqJ24SYijvyf46Qbb3svUkWFF4Opwt6JZ4+1Lj6+t0wSKy1ENUUO9M6SG4
         /QY2GTG97GbXlEUmhzyAHZe6g7lTGjnsuTDoYOvxmDPhMw9q2QmdtaIyNVP0wl2/uN5J
         J2b3HNt6/gAFbTBUwv6vJ1yNH7uGToqhmTQhozQIVytcZKKw4QjYuRPRKrjYttaiYkvC
         TsTm4zJHNh7mO9A/5r0Sk3y0+a/ZcRgeLCKe+yCURHnScxuMlQCSvwCUcd9nRbpj8xNF
         jWrH3Od1g2W8fMIlsxsGxJmLry/bOAJh52vgLLzRH1NUVYjwxZwrnHAunfdNZB6rekBR
         jszg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992293; x=1708597093;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=HZHY6xKr+5lB7TS1xmNKRfrrhDk5jvTKT8Lq2YpKq9o=;
        b=RRo97C1moMrKyf9/Xy1ks1ftUhS4kieUweW+5kSg1K3/G1kTJmyWzGFWA9Y43T8CdT
         zeVwkiP4Uvy5ublexZIsBLD95Zcx06JZmHYNTCgTmzTOyoMDEXxUjJIq6g5rDgY/Jxue
         DNQLUEMgykY3+rozwqxS7KbZcgJwjtPjnLMWl2UJkesPYFT6e3WbLVOOtOQ0C2TC195n
         L/ZW4N6L7suW/TeH8K+6k2nUbVzdeDlnQNwve+o3XBK2owe6UIv/tHR9cTMUAGU3j6NO
         EhzA19LXHZWc9wuGoAqYL3svRR5fkmpMXrZNN2Fuu1XVFY9azr/qGMu/fmZLDiJqfSq/
         wOvQ==
X-Gm-Message-State: AOJu0YwvmeMLMJvKxCzaOzntnUF/oOKeLuB68qU3+blxBeOCi6SRHlh3
	wdJGEqBeYxBi8dJriHCQYWEXpOJISCxptWsvA2/o8Uy5RWlLvRRA1pjxb19iOe2gW9hufNAm2U8
	=
X-Google-Smtp-Source: AGHT+IHu/29TZZKNZ6L7qX+8fg//66hhzuIgaQn81jdnlckinsaiscRFqx1rMrhGjUnySpqPZTP6kw==
X-Received: by 2002:a05:600c:35d2:b0:410:8ab6:8c72 with SMTP id r18-20020a05600c35d200b004108ab68c72mr934767wmq.31.1707992292681;
        Thu, 15 Feb 2024 02:18:12 -0800 (PST)
Message-ID: <0af1d658-c43b-46f9-b290-22447ebfd0be@suse.com>
Date: Thu, 15 Feb 2024 11:18:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 09/12] VT-d: move dev_invalidate_iotlb() to the sole file
 it's used from
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

..., thus allowing it and qinval_device_iotlb_sync() to become static.
There's nothing x86-specific about the function anyway. While moving,
adjust types to better match ./CODING_STYLE (albeit use of fixed-width
types for parameters is retained to limit the effective change).

Append a UL to a constant while moving, to please Misra. Also insert
blank lines in the switch(), between non-fall-through case blocks.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
---
v2: Cosmetics.

--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -65,13 +65,6 @@ struct acpi_drhd_unit *ioapic_to_drhd(un
 struct acpi_drhd_unit *hpet_to_drhd(unsigned int hpet_id);
 struct acpi_rhsa_unit *drhd_to_rhsa(const struct acpi_drhd_unit *drhd);
 
-int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
-                         u64 addr, unsigned int size_order, u64 type);
-
-int __must_check qinval_device_iotlb_sync(struct vtd_iommu *iommu,
-                                          struct pci_dev *pdev,
-                                          u16 did, u16 size, u64 addr);
-
 uint64_t alloc_pgtable_maddr(unsigned long npages, nodeid_t node);
 void free_pgtable_maddr(u64 maddr);
 void *map_vtd_domain_page(u64 maddr);
--- a/xen/drivers/passthrough/vtd/qinval.c
+++ b/xen/drivers/passthrough/vtd/qinval.c
@@ -251,8 +251,9 @@ static int __must_check dev_invalidate_s
     return rc;
 }
 
-int qinval_device_iotlb_sync(struct vtd_iommu *iommu, struct pci_dev *pdev,
-                             u16 did, u16 size, u64 addr)
+static int qinval_device_iotlb_sync(struct vtd_iommu *iommu,
+                                    struct pci_dev *pdev, uint16_t did,
+                                    uint16_t size, paddr_t addr)
 {
     unsigned long flags;
     unsigned int index;
@@ -282,6 +283,103 @@ int qinval_device_iotlb_sync(struct vtd_
     return dev_invalidate_sync(iommu, pdev, did);
 }
 
+static bool device_in_domain(const struct vtd_iommu *iommu,
+                             const struct pci_dev *pdev, uint16_t did)
+{
+    struct root_entry *root_entry;
+    struct context_entry *ctxt_entry = NULL;
+    unsigned int tt;
+    bool found = false;
+
+    if ( unlikely(!iommu->root_maddr) )
+    {
+        ASSERT_UNREACHABLE();
+        return false;
+    }
+
+    root_entry = map_vtd_domain_page(iommu->root_maddr);
+    if ( !root_present(root_entry[pdev->bus]) )
+        goto out;
+
+    ctxt_entry = map_vtd_domain_page(root_entry[pdev->bus].val);
+    if ( context_domain_id(ctxt_entry[pdev->devfn]) != did )
+        goto out;
+
+    tt = context_translation_type(ctxt_entry[pdev->devfn]);
+    if ( tt != CONTEXT_TT_DEV_IOTLB )
+        goto out;
+
+    found = true;
+ out:
+    if ( root_entry )
+        unmap_vtd_domain_page(root_entry);
+
+    if ( ctxt_entry )
+        unmap_vtd_domain_page(ctxt_entry);
+
+    return found;
+}
+
+static int dev_invalidate_iotlb(struct vtd_iommu *iommu, uint16_t did,
+                                paddr_t addr, unsigned int size_order,
+                                uint64_t type)
+{
+    struct pci_dev *pdev, *temp;
+    int ret = 0;
+
+    if ( !ecap_dev_iotlb(iommu->ecap) )
+        return ret;
+
+    list_for_each_entry_safe ( pdev, temp, &iommu->ats_devices, ats.list )
+    {
+        bool sbit;
+        int rc = 0;
+
+        switch ( type )
+        {
+        case DMA_TLB_DSI_FLUSH:
+            if ( !device_in_domain(iommu, pdev, did) )
+                break;
+            /* fall through if DSI condition met */
+        case DMA_TLB_GLOBAL_FLUSH:
+            /* invalidate all translations: sbit=1,bit_63=0,bit[62:12]=1 */
+            sbit = 1;
+            addr = (~0UL << PAGE_SHIFT_4K) & 0x7FFFFFFFFFFFFFFFUL;
+            rc = qinval_device_iotlb_sync(iommu, pdev, did, sbit, addr);
+            break;
+
+        case DMA_TLB_PSI_FLUSH:
+            if ( !device_in_domain(iommu, pdev, did) )
+                break;
+
+            /* if size <= 4K, set sbit = 0, else set sbit = 1 */
+            sbit = size_order ? 1 : 0;
+
+            /* clear lower bits */
+            addr &= ~0UL << PAGE_SHIFT_4K;
+
+            /* if sbit == 1, zero out size_order bit and set lower bits to 1 */
+            if ( sbit )
+            {
+                addr &= ~((u64)PAGE_SIZE_4K << (size_order - 1));
+                addr |= (((u64)1 << (size_order - 1)) - 1) << PAGE_SHIFT_4K;
+            }
+
+            rc = qinval_device_iotlb_sync(iommu, pdev, did, sbit, addr);
+            break;
+
+        default:
+            dprintk(XENLOG_WARNING VTDPREFIX, "invalid vt-d flush type\n");
+            return -EOPNOTSUPP;
+        }
+
+        if ( !ret )
+            ret = rc;
+    }
+
+    return ret;
+}
+
 static int __must_check queue_invalidate_iec_sync(struct vtd_iommu *iommu,
                                                   u8 granu, u8 im, u16 iidx)
 {
--- a/xen/drivers/passthrough/vtd/x86/Makefile
+++ b/xen/drivers/passthrough/vtd/x86/Makefile
@@ -1,2 +1 @@
-obj-y += ats.o
 obj-y += vtd.o
--- a/xen/drivers/passthrough/vtd/x86/ats.c
+++ /dev/null
@@ -1,123 +0,0 @@
-/*
- * Copyright (c) 2006, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
- * more details.
- *
- * You should have received a copy of the GNU General Public License along with
- * this program; If not, see <http://www.gnu.org/licenses/>.
- *
- * Author: Allen Kay <allen.m.kay@intel.com>
- */
-
-#include <xen/sched.h>
-#include <xen/iommu.h>
-#include <xen/time.h>
-#include <xen/pci.h>
-#include <xen/pci_regs.h>
-#include <asm/msi.h>
-#include "../iommu.h"
-#include "../dmar.h"
-#include "../vtd.h"
-#include "../extern.h"
-#include "../../ats.h"
-
-static bool device_in_domain(const struct vtd_iommu *iommu,
-                             const struct pci_dev *pdev, uint16_t did)
-{
-    struct root_entry *root_entry;
-    struct context_entry *ctxt_entry = NULL;
-    unsigned int tt;
-    bool found = false;
-
-    if ( unlikely(!iommu->root_maddr) )
-    {
-        ASSERT_UNREACHABLE();
-        return false;
-    }
-
-    root_entry = map_vtd_domain_page(iommu->root_maddr);
-    if ( !root_present(root_entry[pdev->bus]) )
-        goto out;
-
-    ctxt_entry = map_vtd_domain_page(root_entry[pdev->bus].val);
-    if ( context_domain_id(ctxt_entry[pdev->devfn]) != did )
-        goto out;
-
-    tt = context_translation_type(ctxt_entry[pdev->devfn]);
-    if ( tt != CONTEXT_TT_DEV_IOTLB )
-        goto out;
-
-    found = true;
-out:
-    if ( root_entry )
-        unmap_vtd_domain_page(root_entry);
-
-    if ( ctxt_entry )
-        unmap_vtd_domain_page(ctxt_entry);
-
-    return found;
-}
-
-int dev_invalidate_iotlb(struct vtd_iommu *iommu, u16 did,
-    u64 addr, unsigned int size_order, u64 type)
-{
-    struct pci_dev *pdev, *temp;
-    int ret = 0;
-
-    if ( !ecap_dev_iotlb(iommu->ecap) )
-        return ret;
-
-    list_for_each_entry_safe( pdev, temp, &iommu->ats_devices, ats.list )
-    {
-        bool sbit;
-        int rc = 0;
-
-        switch ( type )
-        {
-        case DMA_TLB_DSI_FLUSH:
-            if ( !device_in_domain(iommu, pdev, did) )
-                break;
-            /* fall through if DSI condition met */
-        case DMA_TLB_GLOBAL_FLUSH:
-            /* invalidate all translations: sbit=1,bit_63=0,bit[62:12]=1 */
-            sbit = 1;
-            addr = (~0UL << PAGE_SHIFT_4K) & 0x7FFFFFFFFFFFFFFF;
-            rc = qinval_device_iotlb_sync(iommu, pdev, did, sbit, addr);
-            break;
-        case DMA_TLB_PSI_FLUSH:
-            if ( !device_in_domain(iommu, pdev, did) )
-                break;
-
-            /* if size <= 4K, set sbit = 0, else set sbit = 1 */
-            sbit = size_order ? 1 : 0;
-
-            /* clear lower bits */
-            addr &= ~0UL << PAGE_SHIFT_4K;
-
-            /* if sbit == 1, zero out size_order bit and set lower bits to 1 */
-            if ( sbit )
-            {
-                addr &= ~((u64)PAGE_SIZE_4K << (size_order - 1));
-                addr |= (((u64)1 << (size_order - 1)) - 1) << PAGE_SHIFT_4K;
-            }
-
-            rc = qinval_device_iotlb_sync(iommu, pdev, did, sbit, addr);
-            break;
-        default:
-            dprintk(XENLOG_WARNING VTDPREFIX, "invalid vt-d flush type\n");
-            return -EOPNOTSUPP;
-        }
-
-        if ( !ret )
-            ret = rc;
-    }
-
-    return ret;
-}



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:18:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:18:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681518.1060342 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYpB-0004ax-NT; Thu, 15 Feb 2024 10:18:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681518.1060342; Thu, 15 Feb 2024 10:18:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYpB-0004aq-Ks; Thu, 15 Feb 2024 10:18:37 +0000
Received: by outflank-mailman (input) for mailman id 681518;
 Thu, 15 Feb 2024 10:18:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYpA-0003nv-4z
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:18:36 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9333dd1a-cbeb-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 11:18:34 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-4121b1aad00so2221515e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:18:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.18.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:18:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9333dd1a-cbeb-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992314; x=1708597114; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=lLmg0pi++GhJOs1eVe+G5sBQYtePLzeMwHyGVwgdUjw=;
        b=ZiLqlKtiGeUelRY/XUEBDvqooZJV6xXkOPpklK6R+KtUO3PnhqzDxBOQGNdRykytRZ
         aAY/nqKgZV+qjngS14xYePUeDLGJFaQGm7NxesniwOqJ61Zc63HVECJQgSAcLajeXjph
         Iy4XzhwbAA2AOQX0XE+iBp+x6Bf/eMmUsVdiFPiTIBYxxnhQ5qpUtTesjVE5WxPe4lRe
         2gCMqXN7U02hvL6JxQevA3fiohY5EelVK29pGNpPqZCqWCLz0bwqlj3SMRACqYtGvOqV
         wV5/IttYZ0RY0ejbUHnodIx0bsf2rDXnqowOke5ni2k0s0iLxmpHMteKzZ/nQmJIuSXo
         5ekQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992314; x=1708597114;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=lLmg0pi++GhJOs1eVe+G5sBQYtePLzeMwHyGVwgdUjw=;
        b=gdbOziBznxE2lZhiftc4i4piCBvkhbhijfiPmKLIupnM8X13PxXjWM57Sc2tSFJOmU
         ijqEJzZ5GfN6eD99ne7XNrl5oPHtMA5fbDWFVjpH0HReJvvbHCdyo2XUCwvq/DHLm1wY
         nly7di4gcQHI0YCn8VPySz1cGSrz7PiPsvtkTdRAADr8yZD2qCOk9x73pIHaAvPDW2bC
         tqGLRBdD0gNi2S881/DM3s7tflQV90B3R6UdLUMvYUNjZU2FTzJXIa8W4DWcskwXr6l+
         SZ5CysQ0Ukm0QE2pYE6VlmZlUuavuKwpLubtGl3aPip1eLETbqKveMruTdxp2ZMY/aUh
         QTOQ==
X-Gm-Message-State: AOJu0YxtpAmDiDJMw6L4Yhv2R3+oO+DbWcwYLQgRXj9ehfyze/LK8KC0
	jUy2a+GU7afoUZuyPZ6Rkj2fq5ZECQWqsXa5udhwn+Yui4aFdJpf2HL8Es674bT2i79pT4vuuao
	=
X-Google-Smtp-Source: AGHT+IE2IMDkOamlYVi9ilGu+LX74qvQUe0kA2NJiX3civPx6gq8DwGohV9AMUXH4tiR9vyAQr20cQ==
X-Received: by 2002:a5d:51ce:0:b0:33c:eb37:470e with SMTP id n14-20020a5d51ce000000b0033ceb37470emr985627wrv.44.1707992313772;
        Thu, 15 Feb 2024 02:18:33 -0800 (PST)
Message-ID: <67bfa370-f4be-4f93-aa63-cbca7636a49c@suse.com>
Date: Thu, 15 Feb 2024 11:18:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 10/12] VT-d: move {,un}map_vtd_domain_page()
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

..., thus allowing them to become static. There's nothing x86-specific
about these functions anyway.

Since only the "iommu_inclusive_mapping" parameter declaration would be
left in the file, move that as well. There's nothing VT-d specific about
it (anymore?): "dom0-iommu=map-inclusive" is similarly generic, and
documentation also doesn't say anything.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
---
v2: doc adjustment.

--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1633,7 +1633,7 @@ Specify the timeout of the device IOTLB
 By default, the timeout is 1000 ms. When you see error 'Queue invalidate
 wait descriptor timed out', try increasing this value.
 
-### iommu_inclusive_mapping
+### iommu_inclusive_mapping (x86)
 > `= <boolean>`
 
 **WARNING: This command line option is deprecated, and superseded by
--- a/xen/drivers/passthrough/vtd/Makefile
+++ b/xen/drivers/passthrough/vtd/Makefile
@@ -1,5 +1,3 @@
-obj-$(CONFIG_X86) += x86/
-
 obj-y += iommu.o
 obj-y += dmar.o
 obj-y += utils.o
--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -21,6 +21,7 @@
 #define _VTD_EXTERN_H_
 
 #include "dmar.h"
+#include <xen/domain_page.h>
 #include <xen/keyhandler.h>
 
 #define VTDPREFIX "[VT-D]"
@@ -67,8 +68,6 @@ struct acpi_rhsa_unit *drhd_to_rhsa(cons
 
 uint64_t alloc_pgtable_maddr(unsigned long npages, nodeid_t node);
 void free_pgtable_maddr(u64 maddr);
-void *map_vtd_domain_page(u64 maddr);
-void unmap_vtd_domain_page(const void *va);
 int domain_context_mapping_one(struct domain *domain, struct vtd_iommu *iommu,
                                uint8_t bus, uint8_t devfn,
                                const struct pci_dev *pdev, domid_t domid,
@@ -78,6 +77,16 @@ int domain_context_unmap_one(struct doma
 int cf_check intel_iommu_get_reserved_device_memory(
     iommu_grdm_t *func, void *ctxt);
 
+static inline void *map_vtd_domain_page(paddr_t maddr)
+{
+    return map_domain_page(_mfn(paddr_to_pfn(maddr)));
+}
+
+static inline void unmap_vtd_domain_page(const void *va)
+{
+    unmap_domain_page(va);
+}
+
 unsigned int cf_check io_apic_read_remap_rte(
     unsigned int apic, unsigned int reg);
 void cf_check io_apic_write_remap_rte(
--- a/xen/drivers/passthrough/vtd/x86/Makefile
+++ /dev/null
@@ -1 +0,0 @@
-obj-y += vtd.o
--- a/xen/drivers/passthrough/vtd/x86/vtd.c
+++ /dev/null
@@ -1,48 +0,0 @@
-/*
- * Copyright (c) 2008, Intel Corporation.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
- * more details.
- *
- * You should have received a copy of the GNU General Public License along with
- * this program; If not, see <http://www.gnu.org/licenses/>.
- *
- * Copyright (C) Allen Kay <allen.m.kay@intel.com>
- * Copyright (C) Weidong Han <weidong.han@intel.com>
- */
-
-#include <xen/param.h>
-#include <xen/sched.h>
-#include <xen/softirq.h>
-#include <xen/domain_page.h>
-#include <asm/paging.h>
-#include <xen/iommu.h>
-#include <xen/irq.h>
-#include <xen/numa.h>
-#include <asm/fixmap.h>
-#include "../iommu.h"
-#include "../dmar.h"
-#include "../vtd.h"
-#include "../extern.h"
-
-/*
- * iommu_inclusive_mapping: when set, all memory below 4GB is included in dom0
- * 1:1 iommu mappings except xen and unusable regions.
- */
-boolean_param("iommu_inclusive_mapping", iommu_hwdom_inclusive);
-
-void *map_vtd_domain_page(u64 maddr)
-{
-    return map_domain_page(_mfn(paddr_to_pfn(maddr)));
-}
-
-void unmap_vtd_domain_page(const void *va)
-{
-    unmap_domain_page(va);
-}
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -19,6 +19,7 @@
 #include <xen/paging.h>
 #include <xen/guest_access.h>
 #include <xen/event.h>
+#include <xen/param.h>
 #include <xen/softirq.h>
 #include <xen/vm_event.h>
 #include <xsm/xsm.h>
@@ -36,6 +37,12 @@ bool __initdata iommu_superpages = true;
 
 enum iommu_intremap __read_mostly iommu_intremap = iommu_intremap_full;
 
+/*
+ * iommu_inclusive_mapping: When set, all memory below 4GB is included in dom0
+ * 1:1 iommu mappings except xen and unusable regions.
+ */
+boolean_param("iommu_inclusive_mapping", iommu_hwdom_inclusive);
+
 #ifdef CONFIG_PV
 /* Possible unfiltered LAPIC/MSI messages from untrusted sources? */
 bool __read_mostly untrusted_msi;



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:18:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:18:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681521.1060353 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYpU-00053X-Vt; Thu, 15 Feb 2024 10:18:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681521.1060353; Thu, 15 Feb 2024 10:18:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYpU-00053L-Ss; Thu, 15 Feb 2024 10:18:56 +0000
Received: by outflank-mailman (input) for mailman id 681521;
 Thu, 15 Feb 2024 10:18:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYpT-0003nv-A1
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:18:55 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9e90d39c-cbeb-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 11:18:53 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-40fd72f7125so5788495e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:18:53 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.18.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:18:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e90d39c-cbeb-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992333; x=1708597133; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=9Q/98Bd60cvr5LoSwMJPfl73bySzka3cm2CynrCQTbw=;
        b=OBZAwYdd0Ifor+dYnfJXkpiymKgVy7HAny6JjBEg8G6ehLRaj2Llv5D420fvzrdmDo
         xMwcjn5zs3OcrOtTI9qS8jcUsOQvf/fBC3Gc4mxlpD76zk06AYYHCHHrchvbUsnGXrLM
         U8eGUEn0fDpvUl0PXB70eOoy2DI7nwDg8IriT6/ktpkKh9bYmmIQk879uWFsGi9sQ5uG
         AYuDFmIE4htCpRz+ZLvIZjBPQEZieOb27cRLT7mzbo1S6wio4OYj4rKdkImgkIfjyrF1
         YMMVVbB9y50tTCXkFtdnxjAEMfvIX6o3GpF40xaWz6uGWyI5PQB5iBLvmxkCn8OZ+ZDB
         DgdQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992333; x=1708597133;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=9Q/98Bd60cvr5LoSwMJPfl73bySzka3cm2CynrCQTbw=;
        b=nXMCHCgwQhiPDnmOW31QKj26XtbnHKu1KUeIgawr3o39HDK305YGtAQcRgPcXxBpuy
         1wWfRwfwdIikPwC8HdrQ8auAHXx4SPHBPzrt7sjCydhDp++4KbspiCRE7XU2ZKgLnW19
         Z5GzuNamlpvc59qnhLDCCHmXLom3ccQoZdvVUzlI7a873pXcTIlpeTv6bF6CZKiEDp0p
         MFbBATAOtieuFoOxMETyIdQSR24WSNBp/buTS9N0/QN5eyR29IPfDbPd8b/f83989vMC
         d0z0NWALjoOHxI6PQWU/liQ9sFkNAOLfIe5H9rETZMl3KGvktZo/DglhOXpXb5Wky7mR
         f/ew==
X-Gm-Message-State: AOJu0Ywp/PGOAg3josftSm+LoCffIb1f37hVsDjAPHejKpKLD6uf1+0G
	S7zG5vbsao9VtqYXWRVaqxA7CazJmK3HM5gCTxwjpp+fldDoupmVatttWnXdAO7tgt4P/4tDsXo
	=
X-Google-Smtp-Source: AGHT+IETeTFl4XumyccwXgmsgDQE0YKt+uSSmDRwDjhG+soVHGC3zw23pHuAHdnqMQ5jXb+tByKWvg==
X-Received: by 2002:adf:f682:0:b0:33d:fb3:9021 with SMTP id v2-20020adff682000000b0033d0fb39021mr290463wrp.54.1707992332965;
        Thu, 15 Feb 2024 02:18:52 -0800 (PST)
Message-ID: <d63065bd-517b-4b6a-a554-7991bdabc3c2@suse.com>
Date: Thu, 15 Feb 2024 11:18:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 11/12] VT-d: drop flush_dev_iotlb parameter from IOTLB
 flush hook
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

All call sites pass it using the flag from the IOMMU which they also
pass.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -57,8 +57,7 @@ int __must_check cf_check vtd_flush_cont
     uint8_t function_mask, uint64_t type, bool flush_non_present_entry);
 int __must_check cf_check vtd_flush_iotlb_reg(
     struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
-    unsigned int size_order, uint64_t type, bool flush_non_present_entry,
-    bool flush_dev_iotlb);
+    unsigned int size_order, uint64_t type, bool flush_non_present_entry);
 
 struct vtd_iommu *ioapic_to_iommu(unsigned int apic_id);
 struct vtd_iommu *hpet_to_iommu(unsigned int hpet_id);
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -566,8 +566,7 @@ static int __must_check iommu_flush_cont
 /* return value determine if we need a write buffer flush */
 int cf_check vtd_flush_iotlb_reg(
     struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
-    unsigned int size_order, uint64_t type, bool flush_non_present_entry,
-    bool flush_dev_iotlb)
+    unsigned int size_order, uint64_t type, bool flush_non_present_entry)
 {
     int tlb_offset = ecap_iotlb_offset(iommu->ecap);
     uint64_t val = type | DMA_TLB_IVT;
@@ -632,7 +631,7 @@ static int __must_check iommu_flush_iotl
     vtd_ops_preamble_quirk(iommu);
 
     status = iommu->flush.iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH,
-                                flush_non_present_entry, iommu->flush_dev_iotlb);
+                                flush_non_present_entry);
 
     /* undo platform specific errata workarounds */
     vtd_ops_postamble_quirk(iommu);
@@ -649,7 +648,7 @@ static int __must_check iommu_flush_iotl
     vtd_ops_preamble_quirk(iommu);
 
     status = iommu->flush.iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH,
-                                flush_non_present_entry, iommu->flush_dev_iotlb);
+                                flush_non_present_entry);
 
     /* undo platform specific errata workarounds */
     vtd_ops_postamble_quirk(iommu);
@@ -675,7 +674,7 @@ static int __must_check iommu_flush_iotl
     vtd_ops_preamble_quirk(iommu);
 
     status = iommu->flush.iotlb(iommu, did, addr, order, DMA_TLB_PSI_FLUSH,
-                                flush_non_present_entry, iommu->flush_dev_iotlb);
+                                flush_non_present_entry);
 
     /* undo platform specific errata workarounds */
     vtd_ops_postamble_quirk(iommu);
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -502,8 +502,7 @@ struct vtd_iommu {
                                     bool non_present_entry_flush);
         int __must_check (*iotlb)(struct vtd_iommu *iommu, u16 did, u64 addr,
                                   unsigned int size_order, u64 type,
-                                  bool flush_non_present_entry,
-                                  bool flush_dev_iotlb);
+                                  bool flush_non_present_entry);
     } flush;
 
     struct list_head ats_devices;
--- a/xen/drivers/passthrough/vtd/qinval.c
+++ b/xen/drivers/passthrough/vtd/qinval.c
@@ -452,7 +452,7 @@ static int __must_check cf_check flush_c
 
 static int __must_check cf_check flush_iotlb_qi(
     struct vtd_iommu *iommu, u16 did, u64 addr, unsigned int size_order,
-    u64 type, bool flush_non_present_entry, bool flush_dev_iotlb)
+    u64 type, bool flush_non_present_entry)
 {
     u8 dr = 0, dw = 0;
     int ret = 0, rc;
@@ -478,7 +478,7 @@ static int __must_check cf_check flush_i
     if ( !ret )
         ret = rc;
 
-    if ( flush_dev_iotlb )
+    if ( iommu->flush_dev_iotlb )
     {
         rc = dev_invalidate_iotlb(iommu, did, addr, size_order, type);
         if ( !ret )
@@ -573,8 +573,7 @@ static int cf_check vtd_flush_context_no
 
 static int cf_check vtd_flush_iotlb_noop(
     struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
-    unsigned int size_order, uint64_t type, bool flush_non_present_entry,
-    bool flush_dev_iotlb)
+    unsigned int size_order, uint64_t type, bool flush_non_present_entry)
 {
     WARN();
     return -EIO;



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:19:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:19:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681532.1060363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYqM-00062z-BH; Thu, 15 Feb 2024 10:19:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681532.1060363; Thu, 15 Feb 2024 10:19:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYqM-00062s-8i; Thu, 15 Feb 2024 10:19:50 +0000
Received: by outflank-mailman (input) for mailman id 681532;
 Thu, 15 Feb 2024 10:19:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYqL-00062N-DJ
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:19:49 +0000
Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com
 [2a00:1450:4864:20::429])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id becc0f8a-cbeb-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 11:19:47 +0100 (CET)
Received: by mail-wr1-x429.google.com with SMTP id
 ffacd0b85a97d-33d10936af1so63996f8f.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:19:47 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.19.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:19:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: becc0f8a-cbeb-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992387; x=1708597187; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=n1vU3iIA9oaJ5u3mwUz26PW+j70BjJveVatCei2osX4=;
        b=A7AkwQ4YMd9dftW5esA3vyDOBmmJNEgTY2RgxZs5rdusj6OxC5sZZxqMBu/1T6RCkQ
         DOQOnlj9TELtJGKRIZl6uu17GNCZ3KVhmYQ5AmvA6TStwwhhVhK82WjHk/QvRb6VXh9m
         XqJKV/WzL3JwXvyMQflF+fvMW7Ak68V4e2QBLGq//xlA0Xr6nxxxgUe20Z4GY+rYT8FZ
         IBMWd7iG8vw+YzEXndb357jaBt9PkIxjGz2e7UG5zcOo3xzi6/Te8rU3gT+PnxkTqzt4
         yGPybe31O+456ngkHhXaSXRQdNT2I9ikeFli8BvwhJlYzRw2t7w9ben9ApIX+z20l/Wn
         d4Fg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992387; x=1708597187;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=n1vU3iIA9oaJ5u3mwUz26PW+j70BjJveVatCei2osX4=;
        b=XD7fxLOft60MqoC9ZwYhspOrqBI2n7QYZC82DE4TnxQsqkfhy8w4AwNwvHqVRBDpqp
         SAaHeUY1pDZT/a+BC19+Pm7RfAbZkgFQja+pBt7KnCxGujiCkaEIUxlJT56b/BW1AxcR
         +fEhYWMeip+fxuHB9ksJBiWWY1ghuWcmutKvcHHzuRE3mPos5fhWReFmINfCKvRGGlFM
         WXpSEPOr9oNMGMcEw7PJhdQpfWpXf5gpGQ8LF9p91D8ng7btm+TCVZp0ZMv5C7IAiYZY
         nLr0GRUf9Qiq1VnlHg6Oh85mn2zjzT08kbxZNJ0h06SZHbC4HnXdrJc8FtQKQ7E+q7mN
         HJ9Q==
X-Gm-Message-State: AOJu0YwlIlh2ky1u/9A4gNcQRJiAtCbD03eiBy2lJ+rYRc3oCNTG9sms
	HiEORAsAfpODYuS+zEk9Fp/5J5aory7beKlJnm2fV7rDpjxUX+LLiKBgnLqdBn17QAcm/zmFV4E
	=
X-Google-Smtp-Source: AGHT+IGIfIj8HOKsHO7ITuiOtJqVOoRtnWdu7rLwJYHY0y3h/emNo2LUPfyhohPAdb/ytCC6Nmdofg==
X-Received: by 2002:a5d:644b:0:b0:33b:68c0:8e2f with SMTP id d11-20020a5d644b000000b0033b68c08e2fmr1001767wrw.46.1707992386941;
        Thu, 15 Feb 2024 02:19:46 -0800 (PST)
Message-ID: <edec0260-e338-49aa-a21f-c507ef9e35cb@suse.com>
Date: Thu, 15 Feb 2024 11:19:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 12/12] PCI/ATS: tidy {en,dis}able_ats_device() a little
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Use appropriate types for the control register value as well as the
capability position. Constify a pointer. Use "else" in favor of encoding
the opposite condition of the earlier if().

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/drivers/passthrough/ats.c
+++ b/xen/drivers/passthrough/ats.c
@@ -23,10 +23,9 @@ boolean_param("ats", opt_ats);
 
 int enable_ats_device(struct pci_dev *pdev, struct list_head *ats_list)
 {
-    u32 value;
-    int pos;
+    uint16_t value;
+    unsigned int pos = pci_find_ext_capability(pdev->sbdf, PCI_EXT_CAP_ID_ATS);
 
-    pos = pci_find_ext_capability(pdev->sbdf, PCI_EXT_CAP_ID_ATS);
     BUG_ON(!pos);
 
     if ( iommu_verbose )
@@ -35,7 +34,7 @@ int enable_ats_device(struct pci_dev *pd
     value = pci_conf_read16(pdev->sbdf, pos + ATS_REG_CTL);
     if ( value & ATS_ENABLE )
     {
-        struct pci_dev *other;
+        const struct pci_dev *other;
 
         list_for_each_entry ( other, ats_list, ats.list )
             if ( other == pdev )
@@ -44,8 +43,7 @@ int enable_ats_device(struct pci_dev *pd
                 break;
             }
     }
-
-    if ( !(value & ATS_ENABLE) )
+    else
     {
         value |= ATS_ENABLE;
         pci_conf_write16(pdev->sbdf, pos + ATS_REG_CTL, value);
@@ -69,7 +67,7 @@ int enable_ats_device(struct pci_dev *pd
 
 void disable_ats_device(struct pci_dev *pdev)
 {
-    u32 value;
+    uint16_t value;
 
     BUG_ON(!pdev->ats.cap_pos);
 



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:21:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:21:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681538.1060373 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYrt-0008LP-Lh; Thu, 15 Feb 2024 10:21:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681538.1060373; Thu, 15 Feb 2024 10:21:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYrt-0008LI-Iq; Thu, 15 Feb 2024 10:21:25 +0000
Received: by outflank-mailman (input) for mailman id 681538;
 Thu, 15 Feb 2024 10:21:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raYrs-0008L9-V4
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:21:24 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f83dc56a-cbeb-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:21:23 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-410ca9e851bso5272565e9.3
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:21:23 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bs3-20020a056000070300b0033b45bdb2a1sm1361072wrb.4.2024.02.15.02.21.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:21:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f83dc56a-cbeb-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707992483; x=1708597283; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=N3l8wUaqMtyLaCRIRS8lSh4Ne9ru/GH+EJfrXDmyju4=;
        b=BPSV9rEcj+nPEX5azL/t7m5W9DwWB0o23ugBOrdTxPPDP9GoCley6Xu0a8OibhddSA
         tK/A/fajo9W/u+QdScw/sR66NaqvpgPt6bzRYXHE3JNgADoTb+xbmBfcM/o1E2y2QrdD
         O0WSRLmMJ0O5+ou8G886jQ55DRRPUyA5F7RZdF+OmyfEMPvXAZGmeSV+mV+/KvaTQa/T
         gtz7xGPrYkdJsBOXjVA7pHXgPKVuQALL9eQ3nA8bu5tluL8v6O6eQd6URMEHZIpdumTw
         dJShRwT6MPy2o9r7ZNBWUvNR6/O2TXKHfNUImJ9dItv2NI2WxINSXmnnjndX+SiupaXe
         naVw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992483; x=1708597283;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=N3l8wUaqMtyLaCRIRS8lSh4Ne9ru/GH+EJfrXDmyju4=;
        b=Ubj6tUNWSbtPH1SPvAzaoerBNC4KtGPXj5ZwJEJzPfHeLTD6v+v3hEnK+nKUj6yCRP
         WCBEZ9tyUVoa186URuaw4/L87MGO0w+o/xN1KkssrFFNv6dBVuavLNNYiuFk6c7jB+x0
         JJLQZdFgxIRi0LxUDoWU63v0kPCxHgRLwk0Kf1WOcoUBatlgT/0p2drENiNEUeg43ggU
         bFsYMCgNl3R38w27LFXwDtNxpRzwCoLHKG87nlvTj5NPunEgAlDnF6zAivycMriNbrM9
         tSPamr6/O7/7kaQtChnuhE55dObIbaW4O999AuVz3wX223OsRCLwIT7p2fpbbL4HIGCc
         C6HA==
X-Gm-Message-State: AOJu0YyNw+TQNWfHMWiyT878dl+cCo9xTpiC9wj0lg/8248W4j89G+TX
	/EJ2iffTN5ufubh89CjXNdi2mppFD29otpKo4zRN6DVhWQPQ38M+w7pNl2fEPoek/flFZxBYb/w
	=
X-Google-Smtp-Source: AGHT+IFL2iJPTr6GYhtNeC1yWcFw0ohjZJJ4fYABDUYx2yjcSZqcO8gPjTqvc4RMYB7yxQ3Qph0NEQ==
X-Received: by 2002:a5d:6dad:0:b0:33b:15fc:22bc with SMTP id u13-20020a5d6dad000000b0033b15fc22bcmr1250271wrs.52.1707992483406;
        Thu, 15 Feb 2024 02:21:23 -0800 (PST)
Message-ID: <1696d71e-4925-432f-bca1-126e4f9d61d9@suse.com>
Date: Thu, 15 Feb 2024 11:21:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 05/12] IOMMU: rename and re-type ats_enabled
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kevin Tian <kevin.tian@intel.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Paul Durrant <paul@xen.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <64b028be-2197-4951-ae5b-32f9eabfa84a@suse.com>
 <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <7f11ca06-9bed-443b-9c79-0e62b71a1f96@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.02.2024 11:15, Jan Beulich wrote:
> Make the variable a tristate, with (as done elsewhere) a negative value
> meaning "default". Since all use sites need looking at, also rename it
> to match our usual "opt_*" pattern. While touching it, also move it to
> .data.ro_after_init.
> 
> The only place it retains boolean nature is pci_ats_device(), for now.
> 
> In AMD code re-order conditionals to have the config space accesses
> after (cheaper) flag checks.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

I'm sorry, forgot to Cc Andrew for the AMD IOMMU part of the changes.

Jan

> ---
> In domain_context_mapping_one() I'm a little puzzled that translation
> type is selected based on only IOMMU and global properties, i.e. not
> taking the device itself into account.
> ---
> v2: Re-base over new earlier patches.
> 
> --- a/xen/drivers/passthrough/amd/iommu_cmd.c
> +++ b/xen/drivers/passthrough/amd/iommu_cmd.c
> @@ -282,7 +282,7 @@ void amd_iommu_flush_iotlb(u8 devfn, con
>      struct amd_iommu *iommu;
>      unsigned int req_id, queueid, maxpend;
>  
> -    if ( !ats_enabled )
> +    if ( opt_ats <= 0 )
>          return;
>  
>      if ( !pci_ats_enabled(pdev->seg, pdev->bus, pdev->devfn) )
> @@ -340,7 +340,7 @@ static void _amd_iommu_flush_pages(struc
>          flush_command_buffer(iommu, 0);
>      }
>  
> -    if ( ats_enabled )
> +    if ( opt_ats > 0 )
>      {
>          amd_iommu_flush_all_iotlbs(d, daddr, order);
>  
> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> @@ -119,7 +119,7 @@ static bool use_ats(
>      const struct amd_iommu *iommu,
>      const struct ivrs_mappings *ivrs_dev)
>  {
> -    return !ivrs_dev->block_ats &&
> +    return opt_ats > 0 && !ivrs_dev->block_ats &&
>             iommu_has_cap(iommu, PCI_CAP_IOTLB_SHIFT) &&
>             pci_ats_device(iommu->seg, pdev->bus, pdev->devfn);
>  }
> @@ -196,7 +196,7 @@ static int __must_check amd_iommu_setup_
>          dte->sys_mgt = MASK_EXTR(ivrs_dev->device_flags, ACPI_IVHD_SYSTEM_MGMT);
>  
>          if ( use_ats(pdev, iommu, ivrs_dev) )
> -            dte->i = ats_enabled;
> +            dte->i = true;
>  
>          spin_unlock_irqrestore(&iommu->lock, flags);
>  
> @@ -257,7 +257,7 @@ static int __must_check amd_iommu_setup_
>                                           ACPI_IVHD_SYSTEM_MGMT));
>  
>          if ( use_ats(pdev, iommu, ivrs_dev) )
> -            ASSERT(dte->i == ats_enabled);
> +            ASSERT(dte->i);
>  
>          spin_unlock_irqrestore(&iommu->lock, flags);
>  
> --- a/xen/drivers/passthrough/ats.c
> +++ b/xen/drivers/passthrough/ats.c
> @@ -18,8 +18,8 @@
>  #include <xen/pci_regs.h>
>  #include "ats.h"
>  
> -bool __read_mostly ats_enabled;
> -boolean_param("ats", ats_enabled);
> +int8_t __ro_after_init opt_ats = -1;
> +boolean_param("ats", opt_ats);
>  
>  int enable_ats_device(struct pci_dev *pdev, struct list_head *ats_list)
>  {
> --- a/xen/drivers/passthrough/ats.h
> +++ b/xen/drivers/passthrough/ats.h
> @@ -22,7 +22,7 @@
>  #define ATS_QUEUE_DEPTH_MASK     0x1f
>  #define ATS_ENABLE               (1<<15)
>  
> -extern bool ats_enabled;
> +extern int8_t opt_ats;
>  
>  int enable_ats_device(struct pci_dev *pdev, struct list_head *ats_list);
>  void disable_ats_device(struct pci_dev *pdev);
> @@ -43,7 +43,7 @@ static inline int pci_ats_enabled(int se
>  
>  static inline int pci_ats_device(int seg, int bus, int devfn)
>  {
> -    if ( !ats_enabled )
> +    if ( !opt_ats )
>          return 0;
>  
>      return pci_find_ext_capability(PCI_SBDF(seg, bus, devfn),
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -1543,7 +1543,7 @@ int domain_context_mapping_one(
>          }
>  
>          context_set_address_root(lctxt, root);
> -        if ( ats_enabled && ecap_dev_iotlb(iommu->ecap) )
> +        if ( opt_ats > 0 && ecap_dev_iotlb(iommu->ecap) )
>              context_set_translation_type(lctxt, CONTEXT_TT_DEV_IOTLB);
>          else
>              context_set_translation_type(lctxt, CONTEXT_TT_MULTI_LEVEL);
> --- a/xen/drivers/passthrough/vtd/x86/ats.c
> +++ b/xen/drivers/passthrough/vtd/x86/ats.c
> @@ -46,7 +46,7 @@ int ats_device(const struct pci_dev *pde
>      struct acpi_drhd_unit *ats_drhd;
>      unsigned int pos, expfl = 0;
>  
> -    if ( !ats_enabled || !iommu_qinval )
> +    if ( opt_ats <= 0 || !iommu_qinval )
>          return 0;
>  
>      if ( !ecap_queued_inval(drhd->iommu->ecap) ||
> 



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:29:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:29:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681558.1060383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYz5-0000fN-Cm; Thu, 15 Feb 2024 10:28:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681558.1060383; Thu, 15 Feb 2024 10:28:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raYz5-0000fG-9O; Thu, 15 Feb 2024 10:28:51 +0000
Received: by outflank-mailman (input) for mailman id 681558;
 Thu, 15 Feb 2024 10:28:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wY2D=JY=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raYz3-0000ep-Ul
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:28:49 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0185dcb0-cbed-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:28:48 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-5116ec49081so791126e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:28:48 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 t12-20020ac865cc000000b0042daffe1891sm441068qto.65.2024.02.15.02.28.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 02:28:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0185dcb0-cbed-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707992928; x=1708597728; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=9nK1ys8njevLp372pDNOR9Z5Y0d+20KDJN8o8tshql0=;
        b=K8WjvPl4qCVy0SheKigAsryCBR3Z28y60vNJ0OuK+vcr0iFvJhqHoEt1OmYz0HHwQ7
         rPY4AjckCln5SUrEZm+pE42C9tVZfpeDXSFyr9SIOEmPHpcB3Vs9PJfNMXdvYyKOGgun
         qhKQAyOFAMtEXBSWx9m5zGE3WYieui5VaA7Co=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707992928; x=1708597728;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9nK1ys8njevLp372pDNOR9Z5Y0d+20KDJN8o8tshql0=;
        b=ehQxGu4X1Rjmc7hSFE9JKZLukYZ6Omvd+iVy4hjmZvQr2EBBttpO0B2DNSE5pClOMW
         W9SggMzxdON/GtnVo6pVKDMY4nzZCxaCSqOn2Kx5cjBnqeWRwfYmVtZkF4Sj3Loom0iH
         UpcgBDVV48pUYx3n5vh3IRfarwWAw5N1LGIcmWz6H+uNhKuNcLiXD4YEPK+EvRhZx3PO
         04mC3/NPeiiev74mj2BltGFNhOzktzfoSpxbis7lv5kDgBMyXkM2ZBJnLxd/uegzvo6C
         ASpgdrCOIMML6PwgVVWuIuPoM6c8np9XRFYu8W4Txw/sOObw2FpXrx/zrlXydey+ocmY
         q25g==
X-Forwarded-Encrypted: i=1; AJvYcCVScO6aenn8J+Jl9eRKDe+YuLywx2TqcFxZezsCM+aMYF7qESezQ8spHOrbNfew835ZBReohDW0y6UfkPr9SuIcBXBMZJSNaJ/FoRz4Sp4=
X-Gm-Message-State: AOJu0Yx6qnWve3a1MmwEnXr8SedvtXz41WmuujS66YAd6K8sYaiD3mQ7
	EzHZSC1NqOWPIHKjR1ttUoUgCLiVEfTXT+Fi/mJSRDlKXUu5OmSgtS+153g76gEBcoi8W0jBs0o
	K
X-Google-Smtp-Source: AGHT+IEVYKXD4oOBA5TSTmb48XtQsRPvlYPkKyzuQtJq42Ay4/IRnxQmGTFQyUlSc4cdlpRxZpkt4A==
X-Received: by 2002:a05:6512:1024:b0:512:8881:8f6b with SMTP id r4-20020a056512102400b0051288818f6bmr928428lfr.36.1707992928418;
        Thu, 15 Feb 2024 02:28:48 -0800 (PST)
Date: Thu, 15 Feb 2024 11:28:46 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: anthony.perard@citrix.com, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Message-ID: <Zc3nXpUOlnIHEfsl@macbook>
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com>

On Thu, Feb 15, 2024 at 10:49:31AM +0100, Jan Beulich wrote:
> On 15.02.2024 10:30, Roger Pau Monne wrote:
> > --- a/xen/Makefile
> > +++ b/xen/Makefile
> > @@ -358,10 +358,10 @@ config: tools_fixdep outputmakefile FORCE
> >  else # !config-build
> >  
> >  ifeq ($(need-config),y)
> > --include include/config/auto.conf
> >  # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
> >  # changes are detected.
> >  -include include/config/auto.conf.cmd
> > +include include/config/auto.conf
> 
> With the - dropped, ...
> 
> > @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
> >  # This exploits the 'multi-target pattern rule' trick.
> >  # The syncconfig should be executed only once to make all the targets.
> >  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
> > +	rm -rf include/config/$*.conf
> >  	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
> 
> ... is this really necessary? The error status from the sub-make is ignored
> only because of the -, isn't it?

Without the `rm` the include/config/auto.conf is not removed by
Kconfig on error, so the include will still succeed but use the stale
auto.conf file.

Keep in mind on rebuilds include/config/auto.conf is already present,
so the rule is only executed for the include/config/auto.conf.cmd
target.

> 
> I also don't really follow the need to re-order the include-s above. Their
> ordering ought to be benign, as per make's doc stating "If an included
> makefile cannot be found in any of these directories it is not an
> immediately fatal error; processing of the makefile containing the include
> continues." While the description talks about this, I'm afraid I don't
> really understand "... the .cmd target is executed before including ...":
> What .cmd target are you talking about there?

Without the reordering the include of include/config/auto.conf will
always succeed on rebuilds, because the include is done before
executing the include/config/%.conf.cmd target that does the `rm`.

With the current order the include of include/config/%.conf.cmd that
triggers the re-build of auto.conf happens after having included the
file already.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:32:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:32:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681563.1060392 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZ2z-00037G-Rv; Thu, 15 Feb 2024 10:32:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681563.1060392; Thu, 15 Feb 2024 10:32:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZ2z-000379-Oq; Thu, 15 Feb 2024 10:32:53 +0000
Received: by outflank-mailman (input) for mailman id 681563;
 Thu, 15 Feb 2024 10:32:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raZ2y-00035g-6w
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:32:52 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 91c1d174-cbed-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:32:51 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33aeb088324so361772f8f.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:32:51 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ay14-20020a5d6f0e000000b0033b2799815csm1439651wrb.86.2024.02.15.02.32.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:32:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 91c1d174-cbed-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707993170; x=1708597970; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IOmQG/qM+lNoHDlTkBOz5ABj1XSkHT6x20LcXYNS5k0=;
        b=MoRBV+Lp8DI+lgruiyMGAs4dcsqU1s1Oa1YgSMweQAJt3PkN3aPtpBBh1L9n/5MYk6
         ibL8RYaKSL5LmUsP/CeLv57h4U7FE1tklbEB29CnSanztOEngZEsokSTtwTeh9pD9kTN
         7FKf5jPGrEDH4eQYUgoxDznFguBJqRigMsChVsQvPfx4FlSlXxxafcCZ9CuY1DGl24rZ
         BRZbG6jI7UrQ5Ggn9RRaSzT+NFGR3SQ/r9IUS/OJEUNaMnwB4u8hHBGRr0Du7fMDDXz8
         5nkzGwCNy/s1ZweLOSBhLBqBMKlPEjK+28Je42/LknkUwRHQQrHIgUc4WZjoOyLU4r9N
         rIpQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707993170; x=1708597970;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IOmQG/qM+lNoHDlTkBOz5ABj1XSkHT6x20LcXYNS5k0=;
        b=FAQDDCI455hK8qlpc+hMej21ej7EUrRMs733+lnPh7XAUplUxwXQUpDW9UsU8XQs0f
         J5R3DsOtAzDxDzbbxyoP6V05eWiYZ69+UKSQPOM2ejDX2cW80AmYtHuZDCb6XTTCOIFK
         LM9K3kJ8nzvFpQXuldCJ0h12lczFhwVplH1E4xLTe38amPtxhD7V3Hj0/wXsUlS6oJgu
         f/ZGXXXZ9NHQXAKoNxKlML9pDYIOHOdo3khUAMDKhpy4dm6k8FcJe4/+OYdDRHOev4tN
         d9RIMe0Z0nRNF6HtGBuq1EANV0W8jESXIqxnfiDar/LHudxbQ/NsWLLHjZQ87eFKNLHg
         XYlA==
X-Forwarded-Encrypted: i=1; AJvYcCVLuj0HYyjFHIHjFGdSuHjARo+IYp+nwaQx+q4+q4aYgy1eRr2MmFDwDJMxIGLQ/emppos2cpGjCF8jUenMt/WtKrGirt9KPse4L5CkJ1c=
X-Gm-Message-State: AOJu0YwKJpmUN285w3BJop8g7tbynUuREPcjU62hhauaGnhcerB02DMx
	SQkmliYXc/iBH243BdCLPAE9tGI4FhW7ZIQro+vsusxRL54DjROGLVDaGR5vAYbW/vc5IYEhW9o
	=
X-Google-Smtp-Source: AGHT+IEZ82EnnpRdB3vayI6J21Rmx1snTkTpgtxBxE2JeSKWQIBrH1gn0uQvdtPhyADo29ZlRr/otA==
X-Received: by 2002:a5d:4fc1:0:b0:33c:e392:d98d with SMTP id h1-20020a5d4fc1000000b0033ce392d98dmr989287wrw.58.1707993170424;
        Thu, 15 Feb 2024 02:32:50 -0800 (PST)
Message-ID: <123c446c-e56f-457e-9cf4-58a43a3b069a@suse.com>
Date: Thu, 15 Feb 2024 11:32:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <e28bb23a12fae3c8630f943b469137d367f20022.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
 <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
 <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
 <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com>
 <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
 <45c19a87-5833-4350-a374-3a70fd3aa9ae@suse.com>
 <d735e980-c5a5-46b1-a511-cc2d666dac23@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d735e980-c5a5-46b1-a511-cc2d666dac23@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.02.2024 11:07, Federico Serafini wrote:
> On 15/02/24 09:10, Jan Beulich wrote:
>> On 14.02.2024 17:11, Federico Serafini wrote:
>>> I did some tries with example programs
>>> and the assembler error always points to file and line
>>> of the most enclosing function that caused the failure.
>>> If I am not missing something, using __FILE__ and __LINE__ does not add
>>> any information.
>>>
>>> Therefore, if the new macro is used within the body of other macros,
>>> then the resulting assembler error will point to the source of
>>> the problem (e.g., the site of a bogus call to put_guest()).
>>>
>>> In my opinion, converting put_guest() &Co. to inline functions is not
>>> convenient: the assembler error will point to the most enclosing
>>> function that would be put_unsafe_size(), instead of pointing to the
>>> source of the problem.
>>
>> The assembler error will point to where the inline function was expanded,
>> sure. __FILE__ / __LINE__ ought to point to that inline function (where
>> the macro was used) then, though?
> 
> This is what I get:
> 
> federico@Dell:~$ cat m.c
> #define STRINGIFY(arg) #arg
> #define STATIC_ASSERT_UNREACHABLE(file, line) \
>    asm(".error \"static assertion failed: " file ": " STRINGIFY(line) "\"")

__FILE__ / __LINE__, if to be used, want using here, not at the use
site.

> static inline __attribute__((always_inline)) void g(int x) {
>    switch(x) {
>      case 0:
>        STATIC_ASSERT_UNREACHABLE(__FILE__, __LINE__);
>    }
> }
> 
> static inline __attribute__((always_inline)) void f(int x) {
>    g(x);
> }
> 
> int main(void) {
>    f(0);
>    return 0;
> }
> federico@Dell:~$ gcc -O3 m.c
> m.c: Assembler messages:
> m.c:8: Error: static assertion failed: m.c: 8

That's as expected. There's no mix of macros and inline functions in
your example.

> Note that the linker behaves differently:
> 
> federico@Dell:~$ cat m.c
> extern void __put_user_bad(void);
> 
> static inline __attribute__((always_inline)) void g(int x) {
>    switch(x) {
>      case 0:
>        __put_user_bad();
>    }
> }
> 
> static inline __attribute__((always_inline)) void f(int x) {
>    g(x);
> }
> 
> int main(void) {
>    f(0);
>    return 0;
> }
> federico@Dell:~$ gcc -O3 m.c
> /usr/bin/ld: /tmp/ccv9KHJD.o: in function `main':
> m.c:(.text.startup+0x9): undefined reference to `__put_user_bad'
> collect2: error: ld returned 1 exit status

The important difference is: Here we're told that there was a use of
__put_user_bad, which is easy to grep for, and thus see how the
supplied function / file / line(?) relate to the ultimate problem.

I'm afraid I'm meanwhile confused enough by the various replies
containing results of experimentation that I can't really tell
anymore what case is best. Hence I can only restate my expectation for
an eventual v3: Diagnosing what the issue is, no matter whether the new
macro is used in another macro or in an inline function, should not
become meaningfully more difficult. In how far this is the case wants
clarifying in the description of the change.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:33:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:33:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681564.1060403 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZ36-0003Od-6p; Thu, 15 Feb 2024 10:33:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681564.1060403; Thu, 15 Feb 2024 10:33:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZ36-0003OU-3g; Thu, 15 Feb 2024 10:33:00 +0000
Received: by outflank-mailman (input) for mailman id 681564;
 Thu, 15 Feb 2024 10:32:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XFnF=JY=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1raZ35-00035g-Fo
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:32:59 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 96746083-cbed-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:32:58 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-41222a56492so1503145e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:32:58 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 c8-20020a7bc848000000b00410794ddfc6sm4607107wml.35.2024.02.15.02.32.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 02:32:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 96746083-cbed-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707993178; x=1708597978; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=kS+A7/SknG9vL6W/0a8wtI7CjUWDhZ+jIF8lzDix8/g=;
        b=qCGAEI6NYNMqc5D5WwZuZc0Fc1EWvOnI54QemoPNp7BXZouRWCZGBygy4VCNyXK8je
         mkm9YxIRG/dC1jChczih2xePfEO3jmuexFR0RAwbXGzxzkWCQxweLmLHO+awt/Sb910w
         DSLt0cloeWVnXbsnOXVbuPhrmx1zmxldAGuMY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707993178; x=1708597978;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=kS+A7/SknG9vL6W/0a8wtI7CjUWDhZ+jIF8lzDix8/g=;
        b=o3Xj58F7nFLjkqHI5Wg3jLCt7v/LK7flkwmvXAlihlTtDXKV5K9DWkRvPIiL2Gxu4f
         bqACfexuamctRRRlnhI/9SrBasbxSQASgYEqKKjn7oE/8PfkT1vRv2PXU3m6zvEg0Phe
         SfpF6dfHAGy37DdjN7Ub5BWkn3N4TNBrjOPnWWIrQu2L3AeBy/pkemLGdC8TbCQnW75Q
         aGPq3bzjhst1qFohc8JdCPLsXdO/esePuxaY6K53/oSiFNowiiOZv747Xcb9ru6O8A61
         jGvfvvl3E+oAdz5o/09UiM3jE6SBnyvK6knsCBYgSexgI4RV7Ao7/MsYPNrzN1fqPeDe
         9EZQ==
X-Gm-Message-State: AOJu0YyZkJffOS3DkYXNyOPZye7d29r3DgkRlsd+evkzk5SpEvWicwGJ
	CC+dGzMlId4duCrhCLE8hCmtz63a0T+dC1VJhOn4oJXWViGThRBM2X/Xxv0vgO4=
X-Google-Smtp-Source: AGHT+IHHVG+jpMOoOCmWW+OQV3FaP/bBHz7vX6pGezQa52NTa3pzS9iTtbCaX2KBNkjFCVxIv0a+IA==
X-Received: by 2002:a05:600c:4f96:b0:411:e7d5:cc5 with SMTP id n22-20020a05600c4f9600b00411e7d50cc5mr1091333wmq.4.1707993178339;
        Thu, 15 Feb 2024 02:32:58 -0800 (PST)
Date: Thu, 15 Feb 2024 10:32:57 +0000
From: Anthony PERARD <anthony.perard@citrix.com>
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Message-ID: <23f82e89-d5b5-4b95-bbf9-be0a76a15cc1@perard>
References: <20240215093002.23527-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240215093002.23527-1-roger.pau@citrix.com>

On Thu, Feb 15, 2024 at 10:30:02AM +0100, Roger Pau Monne wrote:
> When doing a rebuild with an xen/include/config/auto.conf already present in
> the tree, failures from Kconfig are ignored since the target is present:
> 
> gmake -C xen install
> gmake[1]: Entering directory '/root/src/xen/xen'
> tools/kconfig/conf  --syncconfig Kconfig
> common/Kconfig:2: syntax error
> common/Kconfig:1: invalid statement
> gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
>   UPD     include/xen/compile.h
>  Xen 4.19-unstable
> gmake[3]: Nothing to be done for 'all'.
> gmake[2]: 'arch/x86/include/asm/asm-offsets.h' is up to date.
> 
> Ultimately leading to a successful build despite the Kconfig error.
> 
> Fix this by first removing xen/include/config/auto.conf before attempting to
> regenerate, and then also make xen/include/config/auto.conf a hard dependency
> of the build process (ie: drop the leading '-') and reordering so the .cmd
> target is executed before including the configuration file.

Could you try to revert commit 8d4c17a90b0a ("xen/build: silence make
warnings about missing auto.conf*") instead? With a much shorter message
like "Don't ignore Kconfig error anymore".

Cheers,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:35:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:35:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681575.1060413 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZ53-0004b7-KC; Thu, 15 Feb 2024 10:35:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681575.1060413; Thu, 15 Feb 2024 10:35:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZ53-0004b0-Fm; Thu, 15 Feb 2024 10:35:01 +0000
Received: by outflank-mailman (input) for mailman id 681575;
 Thu, 15 Feb 2024 10:35:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raZ53-0004Za-10
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:35:01 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id def2eeb9-cbed-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:35:00 +0100 (CET)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2d09bdddfc9so8612081fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:35:00 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f22-20020a7bcd16000000b00411d1ce4f9dsm4644475wmj.34.2024.02.15.02.34.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:34:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: def2eeb9-cbed-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707993300; x=1708598100; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wKsi8CBTAI9tRK9RHuD9QGScWsRlkRXV09da/jZ+0Ho=;
        b=WpOOLWiTwjJgWQyEXwaXj5mSjXaCJICFQ8C5qOZZqr6RvfX2dCFWoF6Qsp9Hl36bDj
         xkuKGcQCn4SA3rQJHhCH/AUk59TWYbi4NwzF18TdjS7jkvtzR5AT06UAljK/3mo/glqz
         jaL0VJHpaNpmKn695x0cfI9hrEF4O3AM57Kz//zHVNyw127T5EVr2kY+V1wsInaH8CiR
         t//TksEkJ4iY2eUhCsx4geFeBnjFhE8YTBEnPYE3xGvJANs3cDNwa0BjWLwYqC5b1PHA
         SQ99BIHxEvtbKS+N2iDUDS3vup2107ULId/k4zZM/QZTKWP3FRJHv4flXlGe3CwKAvIq
         9OuQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707993300; x=1708598100;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wKsi8CBTAI9tRK9RHuD9QGScWsRlkRXV09da/jZ+0Ho=;
        b=Qb4CMCEQhlNPrwy9k+iCMY3Oxb4TGp1wYMqunjTe9JKjdCfWEeJZg7IQJkHULBi7/g
         87BIHi5WOk6xlu3VDWexF2qdyrd3LMGB+kNoqIR+mp3Qt+0//pDBaXWD1oYPhNdV6DGa
         XBfeJcsxC4JsEUSXYqlGCLDZ3uviXpDehgvrMlyIbT2WnwSlWFvBZmtWCMurdw/VL7rE
         xmW/xn17PqASmYmHre1uGzmSNqHKCQ7xIImSawWFbnEvOV/3KtVOq4MJyF0Ind2MJ49T
         cd2wOxQXZPJdJ3MOubGRuunkP+Zob7LQAqkO3j2jepX9BwdqtFBXRC7geA5JNuT0lxjk
         smYQ==
X-Gm-Message-State: AOJu0YzE2pP1Xa9RSjWexsuCqD7jYL4U3yel2ygPJl3e2iu98MTIcUSJ
	j45vOHy5Jd6xO2FlurTJgT2zsr24O3zGlKvSMOkBqe/rGWTJjlfMFbxKu2rLIw==
X-Google-Smtp-Source: AGHT+IGPOwozDd5h0x+SY3lRZfpc3enWyUNrxg0PNommVup7Xy/GhvvHiWsxGxXze83HFTIxM3HiiQ==
X-Received: by 2002:a19:2d16:0:b0:511:aae2:e5e8 with SMTP id k22-20020a192d16000000b00511aae2e5e8mr1059876lfj.52.1707993299863;
        Thu, 15 Feb 2024 02:34:59 -0800 (PST)
Message-ID: <58c754ab-287f-46ac-87ac-6870358d0885@suse.com>
Date: Thu, 15 Feb 2024 11:34:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Content-Language: en-US
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Roger Pau Monne <roger.pau@citrix.com>
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <23f82e89-d5b5-4b95-bbf9-be0a76a15cc1@perard>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <23f82e89-d5b5-4b95-bbf9-be0a76a15cc1@perard>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.02.2024 11:32, Anthony PERARD wrote:
> On Thu, Feb 15, 2024 at 10:30:02AM +0100, Roger Pau Monne wrote:
>> When doing a rebuild with an xen/include/config/auto.conf already present in
>> the tree, failures from Kconfig are ignored since the target is present:
>>
>> gmake -C xen install
>> gmake[1]: Entering directory '/root/src/xen/xen'
>> tools/kconfig/conf  --syncconfig Kconfig
>> common/Kconfig:2: syntax error
>> common/Kconfig:1: invalid statement
>> gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
>>   UPD     include/xen/compile.h
>>  Xen 4.19-unstable
>> gmake[3]: Nothing to be done for 'all'.
>> gmake[2]: 'arch/x86/include/asm/asm-offsets.h' is up to date.
>>
>> Ultimately leading to a successful build despite the Kconfig error.
>>
>> Fix this by first removing xen/include/config/auto.conf before attempting to
>> regenerate, and then also make xen/include/config/auto.conf a hard dependency
>> of the build process (ie: drop the leading '-') and reordering so the .cmd
>> target is executed before including the configuration file.
> 
> Could you try to revert commit 8d4c17a90b0a ("xen/build: silence make
> warnings about missing auto.conf*") instead? With a much shorter message
> like "Don't ignore Kconfig error anymore".

But that'll introduce the problem that was addressed there, won't it?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 10:43:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 10:43:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681582.1060423 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZCt-00077X-Ae; Thu, 15 Feb 2024 10:43:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681582.1060423; Thu, 15 Feb 2024 10:43:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZCt-00077Q-7z; Thu, 15 Feb 2024 10:43:07 +0000
Received: by outflank-mailman (input) for mailman id 681582;
 Thu, 15 Feb 2024 10:43:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raZCr-00077J-JF
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 10:43:05 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ff5c4980-cbee-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 11:43:04 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-41222a56492so1566665e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 02:43:04 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z7-20020a05600c0a0700b00411c3c2fc55sm1564963wmp.45.2024.02.15.02.43.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 02:43:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff5c4980-cbee-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707993784; x=1708598584; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=7MzW1gi9zcwFfvFC1flaKIROTG44GtICm/Pfy/T7M0o=;
        b=RvGoWnpqkcGjmaLoOe3cjZ+RrwwZbR37IGS8FM6wjxkmgFQoftqu/SyyPghzZ+bTQv
         qTDZoSiR0hww6/7YIPJ37+6y4fw+SiOfloAQXI/d7Mj2UGr+/vyTAaoAs9b6zZ1HT0vw
         pSeiedyCcqNfhakI5aGdXf7eUXVV8B4GiG2BTABDG4SR0AD5eAk4chZdFrumoIOnmYvJ
         Ca4LxLhElis4Mpl/MBk2uWKCoO10C1AhLIbmp/GxZYUrZmLeP/QxUrI45yi8WG453Q4M
         DZH3YXXEZy4wyWmmPKI1yWmY2CyXDpLmoYekPLL2veLJLuyhUY+xSBSPp8ai4teJBdF5
         YtOQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707993784; x=1708598584;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=7MzW1gi9zcwFfvFC1flaKIROTG44GtICm/Pfy/T7M0o=;
        b=mAB0wai7zYRVIruFTj6y0BzBVhivvw8aq2fiOwy33WiFU64WfBapVHhsIP6difWYE9
         BOeArvbgyrpfvGcXzwak+oeHpwNQmrTlgqvISisUvKHBQMLmmPRGkKMaRICLh80uAvQp
         2bpsoBQgd9hARFT8wQ9sad/xKga4jdhuICGKGm5X+Mq5AOr6OzjXfVciLAudwJ+/0KXm
         ejfIYkij7ZiVgAayuN61dW4P6mX3VO0igkVpgJzvjwkqhOV1cVh545pI24NC1O4K97II
         qwFBLf1AbONa9c6nP13kKPaA0QZ6Pm/EiL1tzXsCMaarwt2vYUzADYc9m4ookqskHdwh
         Rcwg==
X-Forwarded-Encrypted: i=1; AJvYcCV+6CwHHkuoBl9vR9cvTyeYSLXdcE68wBVDSNHxnW9UyHJlZmT7tJcR77BXT/Gcb6JxyJS79gs2XGAlOJIEwi2IELq0AjLyusTTnMBdI70=
X-Gm-Message-State: AOJu0Ywm8ryWQIAfj12r+f3yLDu3kr5jkjxcwV5jale+fnUUnKn9UAXi
	O8DiqAF1C1l8cH8sSlptwaQez9tW9xJKcUlFjnwBke8ttmxXh+sdzXLj2p4ydQ==
X-Google-Smtp-Source: AGHT+IG5pbETU+SOyPcEkl3ByzeKCTyFBlRScRBABHtLws9FewwdWCQiNfEC2clOnIuuCpJLge+q/A==
X-Received: by 2002:a05:600c:1c18:b0:411:e5c1:9b24 with SMTP id j24-20020a05600c1c1800b00411e5c19b24mr1054702wms.26.1707993783803;
        Thu, 15 Feb 2024 02:43:03 -0800 (PST)
Message-ID: <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com>
Date: Thu, 15 Feb 2024 11:43:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: anthony.perard@citrix.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com> <Zc3nXpUOlnIHEfsl@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zc3nXpUOlnIHEfsl@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.02.2024 11:28, Roger Pau Monné wrote:
> On Thu, Feb 15, 2024 at 10:49:31AM +0100, Jan Beulich wrote:
>> On 15.02.2024 10:30, Roger Pau Monne wrote:
>>> --- a/xen/Makefile
>>> +++ b/xen/Makefile
>>> @@ -358,10 +358,10 @@ config: tools_fixdep outputmakefile FORCE
>>>  else # !config-build
>>>  
>>>  ifeq ($(need-config),y)
>>> --include include/config/auto.conf
>>>  # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
>>>  # changes are detected.
>>>  -include include/config/auto.conf.cmd
>>> +include include/config/auto.conf
>>
>> With the - dropped, ...
>>
>>> @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
>>>  # This exploits the 'multi-target pattern rule' trick.
>>>  # The syncconfig should be executed only once to make all the targets.
>>>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
>>> +	rm -rf include/config/$*.conf
>>>  	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
>>
>> ... is this really necessary? The error status from the sub-make is ignored
>> only because of the -, isn't it?
> 
> Without the `rm` the include/config/auto.conf is not removed by
> Kconfig on error, so the include will still succeed but use the stale
> auto.conf file.
> 
> Keep in mind on rebuilds include/config/auto.conf is already present,
> so the rule is only executed for the include/config/auto.conf.cmd
> target.

But the sub-make ought to return failure, which ought to then stop the
build process?

>> I also don't really follow the need to re-order the include-s above. Their
>> ordering ought to be benign, as per make's doc stating "If an included
>> makefile cannot be found in any of these directories it is not an
>> immediately fatal error; processing of the makefile containing the include
>> continues." While the description talks about this, I'm afraid I don't
>> really understand "... the .cmd target is executed before including ...":
>> What .cmd target are you talking about there?
> 
> Without the reordering the include of include/config/auto.conf will
> always succeed on rebuilds, because the include is done before
> executing the include/config/%.conf.cmd target that does the `rm`.

That's a dual target: It also handles include/config/%.conf. I.e.
because of this ...

> With the current order the include of include/config/%.conf.cmd that
> triggers the re-build of auto.conf happens after having included the
> file already.

... either include would trigger this same rule. IOW I'm afraid I'm still
not seeing what is gained by the re-ordering. I'm also unconvinced that
"triggers" in the sense you use it is actually applicable. Quoting make
doc again: "Once it has finished reading makefiles, make will try to
remake any that are out of date or don’t exist." To me this means that
first all makefile reading will finish, and then whichever included files
need re-making will be re-made.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 11:05:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 11:05:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681590.1060433 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZY9-0003gO-2Z; Thu, 15 Feb 2024 11:05:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681590.1060433; Thu, 15 Feb 2024 11:05:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZY8-0003gH-Vp; Thu, 15 Feb 2024 11:05:04 +0000
Received: by outflank-mailman (input) for mailman id 681590;
 Thu, 15 Feb 2024 11:05:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wY2D=JY=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raZY7-0003gB-J5
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 11:05:03 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 10407673-cbf2-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 12:05:01 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-4122acb7e69so554385e9.3
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 03:05:01 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 9-20020a05600c228900b004101543e843sm4700250wmf.10.2024.02.15.03.05.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 03:05:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 10407673-cbf2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707995100; x=1708599900; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=UddCV46kuNNdWajC1SL/Sy6nZE0RVb+/eMdE9LCfAXE=;
        b=MeDT8cKnzRDPr6vsUTE5ueDpU7eDZd3KSqQnukhlUmCluYfDrn7rEmAQ4nTkmBCb6z
         i780lAa86yu3kwjUj8kWqguhb2S2a0G7FVwH0OXb2KqJL9hdF5J4kF/6RZbc4a+Lw/uF
         lnYbxOYIjrkfb3JiLM9cY4AMAwuQKikNP0Y/Q=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707995100; x=1708599900;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=UddCV46kuNNdWajC1SL/Sy6nZE0RVb+/eMdE9LCfAXE=;
        b=sdTJa9QXZcYz/1iRZJb48qYbZtRBH9dXM6rA0a0FKhwCR2Mebg/QljMtvh6rtH0GES
         mCl3LuWEpGLTLXjJm9AXJnndiWmz87tBaj6mNmTInfzL5HKhfoUzRxJvRVqGGk/4Aus4
         bFjj+pRAhjRY0BX6K/P5SB36VwjzKNL48EQo/xCoWExM3x2e3+BiRmIZN3/HSdEKHxaY
         YMsOzG6008rMAIRNQv3BWQ+Jm2MIaxlQmCfiTu48nqv/2IzRaRND46b0c3hw1HtlpmzW
         F3sTjzW46JyLAbBX8WmKfwaonN9DQM5iHKHCRFZM+4IdxQEiDhwPKU02uEzk1iBdLtX0
         bY6A==
X-Forwarded-Encrypted: i=1; AJvYcCUZWVzl9F/ViH0o6eZm53YI1rek5TcqyJ4coLqCAYhVxu/glzzoT93j6hXoGL1KHuGrGfezswZJi0u76AxFHMxQBy1+tDKsiwFkYEy1nTw=
X-Gm-Message-State: AOJu0Yy2154fWjvErivPZPBzpUvwaqKPTk7SGCrbpWey2ji9bKdnKlYF
	INw1Wgnc18X7mmYCmDse+wcNeBn2uzz4UO3ZXvx8z2zgkx73HRTc6C0zUYaSsfo=
X-Google-Smtp-Source: AGHT+IEmTJUydTlvGo5b0tFvnER9wyNTG0q9bqfEtFmuRvI0BMnaljMMs2DxCxDM5FOu+59whvODRw==
X-Received: by 2002:a05:600c:a386:b0:411:ee70:ed5c with SMTP id hn6-20020a05600ca38600b00411ee70ed5cmr1086443wmb.12.1707995100543;
        Thu, 15 Feb 2024 03:05:00 -0800 (PST)
Date: Thu, 15 Feb 2024 12:04:59 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: anthony.perard@citrix.com, Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Message-ID: <Zc3v20RKMssbaDsl@macbook>
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com>
 <Zc3nXpUOlnIHEfsl@macbook>
 <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com>

On Thu, Feb 15, 2024 at 11:43:02AM +0100, Jan Beulich wrote:
> On 15.02.2024 11:28, Roger Pau Monné wrote:
> > On Thu, Feb 15, 2024 at 10:49:31AM +0100, Jan Beulich wrote:
> >> On 15.02.2024 10:30, Roger Pau Monne wrote:
> >>> --- a/xen/Makefile
> >>> +++ b/xen/Makefile
> >>> @@ -358,10 +358,10 @@ config: tools_fixdep outputmakefile FORCE
> >>>  else # !config-build
> >>>  
> >>>  ifeq ($(need-config),y)
> >>> --include include/config/auto.conf
> >>>  # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
> >>>  # changes are detected.
> >>>  -include include/config/auto.conf.cmd
> >>> +include include/config/auto.conf
> >>
> >> With the - dropped, ...
> >>
> >>> @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
> >>>  # This exploits the 'multi-target pattern rule' trick.
> >>>  # The syncconfig should be executed only once to make all the targets.
> >>>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
> >>> +	rm -rf include/config/$*.conf
> >>>  	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
> >>
> >> ... is this really necessary? The error status from the sub-make is ignored
> >> only because of the -, isn't it?
> > 
> > Without the `rm` the include/config/auto.conf is not removed by
> > Kconfig on error, so the include will still succeed but use the stale
> > auto.conf file.
> > 
> > Keep in mind on rebuilds include/config/auto.conf is already present,
> > so the rule is only executed for the include/config/auto.conf.cmd
> > target.
> 
> But the sub-make ought to return failure, which ought to then stop the
> build process?

For some reason it doesn't, not at least with GNU Make 4.3.

It stops the build if the '-' is dropped from the include of
include/config/auto.conf.cmd.  But that will always fail as
include/config/auto.conf.cmd is never created.

Maybe there's something weird with our makefile, I certainly don't
know that much, but as noted in the commit message,
include/config/auto.conf.cmd failing doesn't cause the build to
stop.

> >> I also don't really follow the need to re-order the include-s above. Their
> >> ordering ought to be benign, as per make's doc stating "If an included
> >> makefile cannot be found in any of these directories it is not an
> >> immediately fatal error; processing of the makefile containing the include
> >> continues." While the description talks about this, I'm afraid I don't
> >> really understand "... the .cmd target is executed before including ...":
> >> What .cmd target are you talking about there?
> > 
> > Without the reordering the include of include/config/auto.conf will
> > always succeed on rebuilds, because the include is done before
> > executing the include/config/%.conf.cmd target that does the `rm`.
> 
> That's a dual target: It also handles include/config/%.conf. I.e.
> because of this ...
> 
> > With the current order the include of include/config/%.conf.cmd that
> > triggers the re-build of auto.conf happens after having included the
> > file already.
> 
> ... either include would trigger this same rule. IOW I'm afraid I'm still
> not seeing what is gained by the re-ordering. I'm also unconvinced that
> "triggers" in the sense you use it is actually applicable. Quoting make
> doc again: "Once it has finished reading makefiles, make will try to
> remake any that are out of date or don’t exist." To me this means that
> first all makefile reading will finish, and then whichever included files
> need re-making will be re-made.

Without the re-ordering the execution is not stopped on failure to
generate include/config/auto.conf:

# gmake -j8 xen clang=y
gmake -C xen install
gmake[1]: Entering directory '/root/src/xen/xen'
rm -rf include/config/auto.conf
tools/kconfig/conf  --syncconfig Kconfig
common/Kconfig:2: syntax error
common/Kconfig:1: invalid statement
gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
gmake[1]: Failed to remake makefile 'include/config/auto.conf'.
  UPD     include/xen/compile.h
 Xen 4.19-unstable
gmake[3]: Nothing to be done for 'all'.
[...]

With the re-ordering:

# gmake -j8 xen clang=y
gmake -C xen install
gmake[1]: Entering directory '/root/src/xen/xen'
rm -rf include/config/auto.conf
tools/kconfig/conf  --syncconfig Kconfig
common/Kconfig:2: syntax error
common/Kconfig:1: invalid statement
gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
gmake[1]: *** [Makefile:379: include/config/auto.conf] Error 2
gmake[1]: Leaving directory '/root/src/xen/xen'
gmake: *** [Makefile:143: install-xen] Error 2
#

So the re-ordering is meaningful.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 11:08:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 11:08:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681596.1060442 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZbi-0004ZC-Kr; Thu, 15 Feb 2024 11:08:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681596.1060442; Thu, 15 Feb 2024 11:08:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raZbi-0004Z5-IJ; Thu, 15 Feb 2024 11:08:46 +0000
Received: by outflank-mailman (input) for mailman id 681596;
 Thu, 15 Feb 2024 11:08:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+jew=JY=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1raZbg-0004Yy-Fb
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 11:08:44 +0000
Received: from fhigh2-smtp.messagingengine.com
 (fhigh2-smtp.messagingengine.com [103.168.172.153])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 93062b4a-cbf2-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 12:08:41 +0100 (CET)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.48])
 by mailfhigh.nyi.internal (Postfix) with ESMTP id E638911400ED;
 Thu, 15 Feb 2024 06:08:39 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute7.internal (MEProxy); Thu, 15 Feb 2024 06:08:39 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
 15 Feb 2024 06:08:38 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 93062b4a-cbf2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1707995319;
	 x=1708081719; bh=X1VMQlH+2KYLHk9ISCkEh8Nsox2VLqcKIH1L8SnH2Yk=; b=
	hp2EvROFT2jjGsfS0PlMIovxx2+MbM8cPNJPy3vieKcBqFyX2pMjXtcdGHDxAZEx
	3UJZFqSqT0VDapDHxBIcuDGp9nGfqAdY9KolHSDx4FjQdrj1NCegSMCbOMpS7+OS
	ibLuuc0u4PT9sVbgjPGgJEqa7zyAZZCeb2HLku13Fx3CLfKbcTroYdRQkLtZ74U+
	g9z6QzGBboWVfF08a2sv8CUb8wIGtdnCdK6Dy+KkKP1UkmzE2//vDyaYtLIWFagw
	suTFReg9nyjJSpumU0O/nhbYICzBiXzuyeNsbNgOkD5Kp6TaAsVaLLXK7BLmTSBa
	e9sRA6LncMvAHdmqd0eIMA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1707995319; x=1708081719; bh=X1VMQlH+2KYLHk9ISCkEh8Nsox2V
	LqcKIH1L8SnH2Yk=; b=e2/g24Um89kE2DWAC0RbaLFIBCNe1nADSXmvs4Qa8ouW
	xKG6QeEfDxn6VAu06+9iWskjrfxgpYwzw1zjmXdkRZ9AV5GoaJmH8Ya6p8J7ldQJ
	pioAjN5yAlAdoaJ1FdKt7XmQYJXOMWJN3Jn8Ks8+rtD2ndbn0jaFLXZuoIwXy4vK
	mY1joe4hccNugQKYKC0xcp93MMTWSVbFIw7B5dAWiTPwl2LrxZU5u9BEPTK3mLXg
	9hfR5yetFV3mLGAS3drR4LnrTrbk8a793vAaeHz8Fw73aCkR3y5a/XUaTedS/QyV
	/lDwfCIb1lr9IUX4n714KYNa2fQYLKiK77S3YUlSWg==
X-ME-Sender: <xms:t_DNZfy132ZxK4b0xEJjuFxyrEX9Z9hWf1YTaIsBkr5RchFDkB6yxQ>
    <xme:t_DNZXQAB1-uwP7X3bjyI6Cb_uWPLzVPZoxhPMx9DxxRHlw_mUI1HQr8Oyc1gVZqY
    -mhO-Q9xI1tLg>
X-ME-Received: <xmr:t_DNZZXBs7rtmK-RVRRQkGOQBIfW5iJLqimwCnAltygt9vlccH-dlWdgY27YvPqo7oBTYkSiyNS4ruelQpfMlKgDGwz7zvNXbg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddtgddvhecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpeffhffvvefukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihcuoehmrghrmhgrrhgvkhesihhnvhhishhisghlvghthh
    hinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeeihfelhfekhefgveejjeet
    vedtgfduveelgeekteehueeufeefudegveejueefhfenucffohhmrghinhepghhithhhuh
    gsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho
    mhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:t_DNZZi2tO4g156Tf37OTFXSb4Tp9lZxMi_5ZYdsMZjABAH7eak_zQ>
    <xmx:t_DNZRATYl_KVOBsll2fHc48Y9fWNehGDVUbU1fcGxA7xgS9OwRADA>
    <xmx:t_DNZSIaia2xphcgqOWdlcojRwpOqebsyAYWcXPfuzaqBzXbUCUbFg>
    <xmx:t_DNZT2S1n8agJ0aYZVq11tlyM_wNuQ6CJ7HBeijvB8L1Gh17Z0h5w>
Feedback-ID: i1568416f:Fastmail
Date: Thu, 15 Feb 2024 12:08:34 +0100
From: Marek Marczykowski <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers
Message-ID: <Zc3wsijDk7J7QgF6@mail-itl>
References: <b591cd2a-2b49-436e-9cf7-788d9064a778@xen.org>
 <4530606b-1b5e-47a4-aa41-e12e9178b06d@suse.com>
 <ZcrlcuNU9y8WymiK@mail-itl>
 <817ceec4-ed12-4360-a6ce-f87732f62251@suse.com>
 <ZcuCTnMpFib4ArHW@mail-itl>
 <b52fa3fa-4e33-4211-af18-f00aaa0b365b@suse.com>
 <ZcuGo_vQhiTAQep0@mail-itl>
 <23bcd506-f843-42f7-a541-0d8db7c319ab@suse.com>
 <Zc10vFQtG40isDHK@mail-itl>
 <6dbb77a9-0a2d-4eb2-a484-d43d007e4182@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="p4AyXtQcOganAOZU"
Content-Disposition: inline
In-Reply-To: <6dbb77a9-0a2d-4eb2-a484-d43d007e4182@suse.com>


--p4AyXtQcOganAOZU
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Thu, 15 Feb 2024 12:08:34 +0100
From: Marek Marczykowski <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH v3 1/8] serial: fake IRQ-regs context in poll handlers

On Thu, Feb 15, 2024 at 09:39:41AM +0100, Jan Beulich wrote:
> On 15.02.2024 03:19, Marek Marczykowski wrote:
> > On Tue, Feb 13, 2024 at 04:44:04PM +0100, Jan Beulich wrote:
> >> On 13.02.2024 16:11, Marek Marczykowski wrote:
> >>> On Tue, Feb 13, 2024 at 04:00:32PM +0100, Jan Beulich wrote:
> >>>> On 13.02.2024 15:53, Marek Marczykowski wrote:
> >>>>> Generally provides sensible output, for example during boot (it is =
using
> >>>>> idle vCPU then, right?).
> >>>>
> >>>> Before Dom0 is started: Yes. With the exception of the phase where PV
> >>>> Dom0's page tables are constructed, albeit in that time window
> >>>> guest_cpu_user_regs() shouldn't yield sensible data either. I can on=
ly
> >>>> say I'm surprised; since I have no way to properly test with an XHCI
> >>>> debug port, I'd have to see about faking something to convince myself
> >>>> (unless you were to supply example output).
> >>>
> >>> Would you like me to test this series with xhci console?
> >>
> >> The behavior shouldn't really be connected to this series. But yes, 'd'
> >> debug key output (just the part for the CPU the key handling was
> >> actually invoked from) with the xhci debug console would be of
> >> interest, for the case where that CPU at that time runs an idle vCPU.
> >=20
> > I managed to press 'd' before dom0 started. Full output at
> > https://gist.github.com/marmarek/a495cd666f4aafed3a5cfcb8393f515b (it's
> > Alder Lake, and smt=3Doff, so CPU numbering is weird).
> > Interestingly, I do _not_ see output for CPU0, where I'd expect the
> > key handler to run... I see all the idle ones, plus one doing memory
> > scrubbing.
>=20
> Which is precisely the problem, just in not exactly the manifestation
> I expected. In dump_execstate() we dump host state only if the
> incoming regs don't indicate guest state. Yet for the idle vCPU they
> (wrongly) do here - see how guest_mode() calculates the delta to what
> guest_cpu_user_regs() returns, i.e. 0 when what guest_cpu_user_regs()
> returned is passed in.
>=20
> Guest state dumping is suppressed for idle vCPU-s. Hence no output
> at all for the CPU where the key processing was actually invoked
> from.
>=20
> > But also, I don't see info about the handling CPU when doing `xl
> > debug-key d`.
>=20
> I'm afraid I'm confused, ...
>=20
> > At one time, with `xl debug-key d` I got this:
> >=20
> > (XEN) *** Dumping CPU6 guest state (d0v7): ***
> > (XEN) ----[ Xen-4.18-unstable  x86_64  debug=3Dy  Tainted:  M     ]----
> > (XEN) CPU:    6
> > (XEN) RIP:    e033:[<ffffffff81e1546a>]
> > (XEN) RFLAGS: 0000000000000286   EM: 0   CONTEXT: pv guest (d0v7)
> > (XEN) rax: 0000000000000023   rbx: 0000000000000005   rcx: ffffffff81e1=
546a
> > (XEN) rdx: 0000000000000000   rsi: 0000000000000000   rdi: 000079147611=
e010
> > (XEN) rbp: ffff88810db53200   rsp: ffffc90041c6bde0   r8:  000000000000=
0000
> > (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 000000000000=
0286
> > (XEN) r12: 0000000000305000   r13: 00007ffc61097f40   r14: ffff88810db5=
3200
> > (XEN) r15: 0000000000000000   cr0: 0000000080050033   cr4: 0000000000b5=
26e0
> > (XEN) cr3: 00000004ae2a7000   cr2: 00000000006d3118
> > (XEN) fsb: 0000791475b8a380   gsb: ffff8881897c0000   gss: 000000000000=
0000
> > (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> > (XEN) Guest stack trace from rsp=3Dffffc90041c6bde0:
> > (XEN)    0000000000000001 0000000000000000 ffffffffc02905a6 00000000000=
00023
> > (XEN)    000079147611e010 0000000000000000 0000000000000000 00000000000=
00000
> > (XEN)    0000000000000000 8064129fc747f100 ffffffffc0291568 00000000003=
05000
> > (XEN)    8064129fc747f100 0000000000000005 ffffffff813f7d4d ffffc90041c=
6bf58
> > (XEN)    ffffc90041c6bf48 0000000000000000 0000000000000000 00000000000=
00000
> > (XEN)    ffffffff81e16158 00000000006d3118 ffffc90041c6bf58 00000000000=
00040
> > (XEN)    ffffffff8132f6bb 0000000000000006 ffffc90041c6bf58 00000000006=
d3118
> > (XEN)    0000000000000255 ffff888102cf8880 ffff888102cf88f0 ffffffff810=
8746f
> > (XEN)    0000000000000000 0000000000000002 ffffc90041c6bf58 ffffc90041c=
6bf58
> > (XEN)    00000000006d3118 0000000000000000 0000000000000006 00000000000=
00000
> > (XEN)    0000000000000000 ffffffff81e1a975 0000000000000000 00000000000=
00000
> > (XEN)    0000000000000000 0000000000000000 ffffffff8200009b 00000000004=
3d9b0
> > (XEN)    000000000043d990 00007ffc61097f90 00007ffc61097fc0 00007ffc610=
99d16
> > (XEN)    00000000006cab40 0000000000000246 0000000000000001 00000000000=
00000
> > (XEN)    0000000000000006 ffffffffffffffda 0000791475f1ed6f 00007ffc610=
97f40
> > (XEN)    0000000000305000 0000000000000005 0000000000000010 0000791475f=
1ed6f
> > (XEN)    0000000000000033 0000000000000246 00007ffc61097ed0 00000000000=
0002b
> > (XEN)     Fault while accessing guest memory.
> > (XEN)=20
> > (XEN) *** Dumping CPU0 host state: ***
> > (XEN) ----[ Xen-4.18-unstable  x86_64  debug=3Dy  Tainted:  M     ]----
> > (XEN) CPU:    0
> > (XEN) RIP:    e008:[<ffff82d04022c07e>] _spin_unlock_irqrestore+0x21/0x=
27
> > (XEN) RFLAGS: 0000000000000206   CONTEXT: hypervisor
> > (XEN) rax: ffff82d0405c1038   rbx: 0000000000000200   rcx: 000000000000=
0008
> > (XEN) rdx: ffff830856d07fff   rsi: ffff8308529d5b28   rdi: ffff8308529d=
5b20
> > (XEN) rbp: ffff830856d07dc8   rsp: ffff830856d07dc0   r8:  000000000000=
0001
> > (XEN) r9:  ffff8308529d5b20   r10: ffff82d0405c13a0   r11: 000000d091e6=
2221
> > (XEN) r12: ffff82d040476898   r13: 0000000000000296   r14: ffff82d04047=
6918
> > (XEN) r15: ffff82cffff04700   cr0: 0000000080050033   cr4: 0000000000b5=
26e0
> > (XEN) cr3: 000000082e7ff000   cr2: ffff888109538618
> > (XEN) fsb: 0000000000000000   gsb: ffff888189600000   gss: 000000000000=
0000
> > (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> > (XEN) Xen code around <ffff82d04022c07e> (_spin_unlock_irqrestore+0x21/=
0x27):
> > (XEN)  fd ff ff 48 09 1c 24 9d <48> 8b 5d f8 c9 c3 0f b7 47 04 66 25 ff=
 0f 66 3d
> > (XEN) Xen stack trace from rsp=3Dffff830856d07dc0:
> > (XEN)    ffff82d0405d0c80 ffff830856d07e08 ffff82d040257c3f 00000000404=
76898
> > (XEN)    ffff82d0405c1280 ffff82d040257bca ffff82d040476898 000000d0911=
fcbc4
> > (XEN)    0000000000000000 ffff830856d07e30 ffff82d04022d55c ffff82d0405=
c1280
> > (XEN)    ffff8308529d5f00 ffff82d0405d0d68 ffff830856d07e70 ffff82d0402=
2de59
> > (XEN)    ffff830856d07ef8 ffff82d0405c7f00 ffffffffffffffff ffff82d0405=
c7f00
> > (XEN)    ffff830856d07fff 0000000000000000 ffff830856d07ea8 ffff82d0402=
2b53e
> > (XEN)    0000000000000000 0000000000007fff ffff82d0405c7f00 ffff82d0405=
c11d0
> > (XEN)    ffff82d0405db2a0 ffff830856d07eb8 ffff82d04022b5d1 ffff830856d=
07ef0
> > (XEN)    ffff82d0402fcd15 ffff82d0402fcc88 ffff8308528cb000 ffff830856d=
07ef8
> > (XEN)    ffff830856ce2000 0000000000000000 ffff830856d07e18 00000000000=
00000
> > (XEN)    0000000000000000 0000000000000000 0000000000000000 ffffffff82c=
1aa40
> > (XEN)    0000000000000000 0000000000000246 0000000000007ff0 00000000000=
00000
> > (XEN)    000000000fd109eb 0000000000000000 ffffffff81e153aa 40000000000=
00000
> > (XEN)    deadbeefdeadf00d deadbeefdeadf00d 0000010000000000 ffffffff81e=
153aa
> > (XEN)    000000000000e033 0000000000000246 ffffffff82c03dd0 00000000000=
0e02b
> > (XEN)    0000000000000000 0000000000000000 0000000000000000 00000000000=
00000
> > (XEN)    0000e01000000000 ffff830856ce1000 0000000000000000 0000000000b=
526e0
> > (XEN)    0000000000000000 0000000000000000 0006030300000000 00000000000=
00000
> > (XEN) Xen call trace:
> > (XEN)    [<ffff82d04022c07e>] R _spin_unlock_irqrestore+0x21/0x27
> > (XEN)    [<ffff82d040257c3f>] F xhci-dbc.c#dbc_uart_poll+0x75/0x17c
> > (XEN)    [<ffff82d04022d55c>] F timer.c#execute_timer+0x45/0x5c
> > (XEN)    [<ffff82d04022de59>] F timer.c#timer_softirq_action+0x71/0x275
> > (XEN)    [<ffff82d04022b53e>] F softirq.c#__do_softirq+0x94/0xbe
> > (XEN)    [<ffff82d04022b5d1>] F do_softirq+0x13/0x15
> > (XEN)    [<ffff82d0402fcd15>] F domain.c#idle_loop+0x8d/0xe6
>=20
> ... this looks to be the issuing CPU. What I don't understand is why we
> are in _spin_unlock_irqrestore() here, called out of dbc_uart_poll().
>=20
> Btw - was any/all of this with or without the series here applied?

This was without this series applied.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--p4AyXtQcOganAOZU
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXN8LIACgkQ24/THMrX
1yxeBggAjG9IN3wpgsYqfMVkmGRZZEiL9QpHQ71F73vwkTOaBVjJcjqBoFpczGTl
UuH/6LTtKrIPqU5SJBTw8h6RXYoNwQdplrL67VjSN6fg6YGqWRKtORQ7VhTr/liM
maa5Z8tW1SAJUjz3rvY81V6Stz+oPUvw0hsz9htBnawFW7nX4HUjfBSLlnfmJ88S
ZGriX2kSK27upHBGRAw/TAEJu3pnItzF5CbWZLvXz9TpSi9go/9WrS9ckPPyeDpk
47EL+wX6JELFmnt+S4eq3LwOTtMCmf+/s1/tLJtoZhX0hlUw3En+Xglu/Ju1sWAz
o+b7PxqqAcmGT0NJe0EWBmrNfNcb2w==
=vsqu
-----END PGP SIGNATURE-----

--p4AyXtQcOganAOZU--


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 11:46:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 11:46:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681611.1060452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaCL-0003Pt-FI; Thu, 15 Feb 2024 11:46:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681611.1060452; Thu, 15 Feb 2024 11:46:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaCL-0003Pm-Ci; Thu, 15 Feb 2024 11:46:37 +0000
Received: by outflank-mailman (input) for mailman id 681611;
 Thu, 15 Feb 2024 11:46:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wY2D=JY=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raaCK-0003Pg-BE
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 11:46:36 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id de2c372f-cbf7-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 12:46:34 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d1025717c7so8217361fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 03:46:34 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 v9-20020ae9e309000000b00785a9820313sm521912qkf.84.2024.02.15.03.46.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 03:46:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de2c372f-cbf7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1707997593; x=1708602393; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Aaq7lSzJZGT0yqG0vRNN27MHOu3dbKZpL/7fwErPFYA=;
        b=ngaj35AKLsbAY2lt2Vft030h7kLRjwEMX8Xbp1kd/iJBkfWCPKwjHiWvCBOSPJjY+i
         xuSo1hT6UPjD2vGXupg+3iNT7fTAUwuIC1mmZzJ5PbOfJ1Yy4FTmWNdab8ys9/HkmiAH
         X3YZa94pyZj61enVpZn5NaLerV3znpzNGcXMw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707997593; x=1708602393;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Aaq7lSzJZGT0yqG0vRNN27MHOu3dbKZpL/7fwErPFYA=;
        b=UZ44n/rbSagUDJIbXBGi6Vf4sGL0rrUCwNtD32BT+truCI3Z7JuqTzBWxkMta4wwV4
         YiGon7tzFU7UAEz4PIfxxZgSgIA+O8GNJogcYm+ytlz+UlVaQxM4VCL/SSkcCUS6j5AK
         j0vcn9JsKkKTnioizA6jmv+riTg8z19EndtByWN6xvPPWOIy8AjzVvv1FB5PCBeCtfrQ
         9XImxavfaO3YaVrNsvn5oMXdhmrgvq29eaeJRvEZx2agfyZSPh0bOZvw7Z9ejoNn4Zlg
         znk9U0c14UzYOTObcGlbxhK5/dlDGz2tOcNMHKDQr5LDG8h2+PBBeAMo7UCRi2GY9P2h
         CzLQ==
X-Gm-Message-State: AOJu0Yy3oSlswUZlfZ4um+7IwFzsjvHcez2tasso7WIoby34//B8gdAN
	lISyi1upCp/X4cISA0m3oFpDKx2SKCAdctkBYOYgHZD3I/Fn3pclab42nvIJskM=
X-Google-Smtp-Source: AGHT+IFpMT84NbS+1BRX+5wb7p+xDlqF5b0dTAnRwAiHGq1SM3xxlpRzCRS7bLRufjGO8kbKm0AflQ==
X-Received: by 2002:a2e:984c:0:b0:2d0:e298:25d4 with SMTP id e12-20020a2e984c000000b002d0e29825d4mr1117664ljj.16.1707997593512;
        Thu, 15 Feb 2024 03:46:33 -0800 (PST)
Date: Thu, 15 Feb 2024 12:46:30 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: xen-devel@lists.xenproject.org,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Message-ID: <Zc35ljv5XYzXGV2P@macbook>
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <23f82e89-d5b5-4b95-bbf9-be0a76a15cc1@perard>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <23f82e89-d5b5-4b95-bbf9-be0a76a15cc1@perard>

On Thu, Feb 15, 2024 at 10:32:57AM +0000, Anthony PERARD wrote:
> On Thu, Feb 15, 2024 at 10:30:02AM +0100, Roger Pau Monne wrote:
> > When doing a rebuild with an xen/include/config/auto.conf already present in
> > the tree, failures from Kconfig are ignored since the target is present:
> > 
> > gmake -C xen install
> > gmake[1]: Entering directory '/root/src/xen/xen'
> > tools/kconfig/conf  --syncconfig Kconfig
> > common/Kconfig:2: syntax error
> > common/Kconfig:1: invalid statement
> > gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
> >   UPD     include/xen/compile.h
> >  Xen 4.19-unstable
> > gmake[3]: Nothing to be done for 'all'.
> > gmake[2]: 'arch/x86/include/asm/asm-offsets.h' is up to date.
> > 
> > Ultimately leading to a successful build despite the Kconfig error.
> > 
> > Fix this by first removing xen/include/config/auto.conf before attempting to
> > regenerate, and then also make xen/include/config/auto.conf a hard dependency
> > of the build process (ie: drop the leading '-') and reordering so the .cmd
> > target is executed before including the configuration file.
> 
> Could you try to revert commit 8d4c17a90b0a ("xen/build: silence make
> warnings about missing auto.conf*") instead? With a much shorter message
> like "Don't ignore Kconfig error anymore".

Yes that seems to solve it also.  I guess the point is that for the
target failure to stop execution the include that triggered it needs
to be non-optional (so not - prefixed).

I'm unsure about the consequences of reverting 8d4c17a90b0a, so would
prefer if you could take care of that.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 11:58:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 11:58:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681617.1060463 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaNu-0005Tg-Gn; Thu, 15 Feb 2024 11:58:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681617.1060463; Thu, 15 Feb 2024 11:58:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaNu-0005TZ-E6; Thu, 15 Feb 2024 11:58:34 +0000
Received: by outflank-mailman (input) for mailman id 681617;
 Thu, 15 Feb 2024 11:58:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raaNt-0005TP-AH; Thu, 15 Feb 2024 11:58:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raaNt-0005Jd-5S; Thu, 15 Feb 2024 11:58:33 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raaNs-0002GB-M3; Thu, 15 Feb 2024 11:58:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raaNs-0008PB-LI; Thu, 15 Feb 2024 11:58:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Aqv0jUA6HN270FCr+D9nQiBbpD9kL9kwoqye3p3ALSo=; b=S9zt/zrf6kT3C8etvhDSdhdhkm
	gieVWoQ7b90SXNtE4TCCP4rEnr9i3JTq579GTThCSZI9d0zHidLTvY6+lco1WgasVCgXr2EHqQEZ4
	HKr4nOz2C3GI+1sUm0q1xlfe1H8EVd6d/rDQ2E0y5NVJuu7Xd8/yKUT/o9DGpbHS91IE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184671-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184671: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=8d3dea210042f54b952b481838c1e7dfc4ec751d
X-Osstest-Versions-That:
    linux=7e90b5c295ec1e47c8ad865429f046970c549a66
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 15 Feb 2024 11:58:32 +0000

flight 184671 linux-linus real [real]
flight 184674 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184671/
http://logs.test-lab.xenproject.org/osstest/logs/184674/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184674-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184659
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184659
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184659
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184659
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184659
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184659
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184659
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184659
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                8d3dea210042f54b952b481838c1e7dfc4ec751d
baseline version:
 linux                7e90b5c295ec1e47c8ad865429f046970c549a66

Last test of basis   184659  2024-02-13 17:43:35 Z    1 days
Testing same since   184671  2024-02-15 00:41:18 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  David Gow <davidgow@google.com>
  David Sterba <dsterba@suse.com>
  Filipe Manana <fdmanana@suse.com>
  Guenter Roeck <linux@roeck-us.net>
  Hu Yadi <hu.yadi@h3c.com>
  Ivan Shapovalov <intelfx@intelfx.name>
  Jiaxun Yang <jiaxun.yang@flygoat.com>
  Johannes Thumshirn <johannes.thumshirn@wdc.com>
  Josef Bacik <josef@toxicpanda.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Mickaël Salaün <mic@digikod.net>
  Shuah Khan <skhan@linuxfoundation.org>
  Thomas Bogendoerfer <tsbogend@alpha.franken.de>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   7e90b5c295ec..8d3dea210042  8d3dea210042f54b952b481838c1e7dfc4ec751d -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 12:02:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 12:02:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681630.1060473 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaRt-0007aK-9u; Thu, 15 Feb 2024 12:02:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681630.1060473; Thu, 15 Feb 2024 12:02:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaRt-0007aD-6v; Thu, 15 Feb 2024 12:02:41 +0000
Received: by outflank-mailman (input) for mailman id 681630;
 Thu, 15 Feb 2024 12:02:39 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1raaRr-0007a5-QH
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 12:02:39 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1raaRq-0005SM-7T; Thu, 15 Feb 2024 12:02:38 +0000
Received: from [15.248.3.91] (helo=[10.24.67.27])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1raaRq-0005L9-00; Thu, 15 Feb 2024 12:02:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=aWcygKsX8KOLDbXVs29E6KNV+EnbpNffkJm7PT0ErVQ=; b=n00gR+Okuq0+MZo5GQGKLrepNO
	LB6/UGTw0TiEDuVgnT07QLCw6wBpkDskZG0xi2B2Vka4T5sPbWEeORNOqWp0a2Y74fikvXSiM7It+
	dc3ERUzdoOWCYdt9SdX5Ts9i4Ahomb7T1T9pqJ49JslMhVwsFjx0h5wedLk8I0G8N2og=;
Message-ID: <ce0f46ff-e7db-4c72-bab0-ea12e496ad8d@xen.org>
Date: Thu, 15 Feb 2024 12:02:35 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/8] keyhandler: drop regs parameter from
 handle_keyregs()
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <ebc330a9-eafa-4858-b5cf-5694c4da911d@suse.com>
 <5258d8a9-a7ef-4342-9b5c-fc0078139bb2@suse.com>
 <0bbe2028-946c-459f-b6cc-419869d22b35@xen.org>
 <bdba33b7-7fc2-4836-8b78-0d56edb30871@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <bdba33b7-7fc2-4836-8b78-0d56edb30871@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 12/02/2024 09:13, Jan Beulich wrote:
> On 08.02.2024 23:09, Julien Grall wrote:
>> On 05/02/2024 13:28, Jan Beulich wrote:
>>> In preparation for further removal of regs parameters, drop it here. In
>>> the two places where it's actually needed, retrieve IRQ context if
>>> available, or else guest context.
>>>
>>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>> ---
>>> As an alternative to the new boolean parameter, I wonder if we couldn't
>>> special-case the idle vCPU case: It's only there where we would not have
>>> proper context retrievable via guest_cpu_user_regs().
>>
>> I am trying to understand the implication. Looking at the code, it seems
>> in the case where we pass NULL, we would expect to call
>> run_in_exception_handler().
> 
> Right, when NULL was passed so far, and when true is passed now, that's
> to indicate to invoke run_in_exception_handler().
> 
>> If I am not mistaken, at least for Arm, regs would not be the same as
>> guest_cpu_user_regs(). So I think your current approach is more correct.
>>
>> Did I miss anything?
> 
> Whether regs are the same isn't overly relevant here. The thing that's
> relevant is whether what would be logged actually makes sense. And
> invoking guest_cpu_user_regs() in idle vCPU context makes no sense.
> Whereas in other contexts its result is good enough to show the present
> state of the CPU; there's no real need in such a case to go through
> run_in_exception_handler().
> 
> The present approach therefore isn't necessarily "more correct", but it
> is closer to prior behavior.
> 
> The corner case that makes me prefer the presently chosen approach is
> when a CPU is in the middle of scheduling, especially considering x86's
> lazy switching (when current != this_cpu(curr_vcpu). The main reason to
> mention the alternative is because iirc Andrew had suggested to move
> more in that direction.

Thanks for the clarification. I don't have any strong preference either 
way. So I would be happy to go with your solution.

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 12:07:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 12:07:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681640.1060483 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaWg-0008DO-Qp; Thu, 15 Feb 2024 12:07:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681640.1060483; Thu, 15 Feb 2024 12:07:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaWg-0008DH-O6; Thu, 15 Feb 2024 12:07:38 +0000
Received: by outflank-mailman (input) for mailman id 681640;
 Thu, 15 Feb 2024 12:07:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=XFnF=JY=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1raaWe-0008DB-UT
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 12:07:36 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cd8f1b8a-cbfa-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 13:07:34 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-412078e983aso6136435e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 04:07:35 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 hn32-20020a05600ca3a000b00410ebcf8180sm4836630wmb.43.2024.02.15.04.07.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 04:07:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd8f1b8a-cbfa-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1707998855; x=1708603655; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=RU5j0fTz/wAQruu0bLSlug1Bo8x/EjzY2X6A0fM5VUc=;
        b=PlQry8xnm/NzBSQpA/ujYEMGqGy9QO1PjxawNUHuf5E1J+vgbSt72iuZjRgcONa3V2
         KKWyAGcPtrV7TtvGPgH9uDCx+d0mReSJig0nQN6ncdgUSo0y3oprzqHGlBFl0Hc72J5z
         cO5KBzYWEFQw5vTa9Zl8/px3AmpkjMm0BZEgM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707998855; x=1708603655;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=RU5j0fTz/wAQruu0bLSlug1Bo8x/EjzY2X6A0fM5VUc=;
        b=JkBm+FLfXSUvgEkIJ/fpzj/oMvUVhU2YBn4J3f71hHvUA3NCGMqh6MCoRP3FYdRIKg
         VnouLs2259lOlHVXYihW08kAwXN1Lu6eFCOvBtb/IwdD8ZOkFiTWF6tT9v9ROvCx8jMI
         Qza6qTW//ROVyHkRzrKzSZjY8zbmH/hRCJns8+QZ6tTgqmNiarS2yVpRomQsy5Iic22W
         6M9MFzg5u7GAmArdFEdfmPVFo4g1EG7knJrFW1BfRrG57iEWTXMtx+Xue1iPnSYW22oI
         kTVjtUD0teSYImCpUB7oSxZzgoL0its1vkk7fC2MSeK+VxDK4vbtH4xSkFGeHVL4NcJa
         9u5Q==
X-Gm-Message-State: AOJu0YxPCmugZ9NZ3AJJkPg1R26z6yFdYDyTccky7r8J0Pb4jz1RfuoJ
	tVdRqNk2icS8VQFsSe7jjmP0Vcu8cv8Chctl9E75FruOE8Uw0+MmO2d1lLsgMC4=
X-Google-Smtp-Source: AGHT+IH7NvhOqe4SC2BghmLoYjvtXb8RBeI8BaxfXR98Q9F7WZnrdK6AtPeCQp5XTXeopeKGKceF3w==
X-Received: by 2002:a05:600c:4f14:b0:411:eb4f:7fb6 with SMTP id l20-20020a05600c4f1400b00411eb4f7fb6mr1215929wmq.8.1707998855111;
        Thu, 15 Feb 2024 04:07:35 -0800 (PST)
Date: Thu, 15 Feb 2024 12:07:34 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: Re: [PATCH v6 15/21] tools/xl: support new 9pfs backend xen_9pfsd
Message-ID: <f4aecdb2-b573-4843-baa0-e6ae6705c54b@perard>
References: <20240215065541.21067-1-jgross@suse.com>
 <20240215065541.21067-16-jgross@suse.com>
 <3261b14b-bce1-4aa5-bb61-a25930fca133@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <3261b14b-bce1-4aa5-bb61-a25930fca133@suse.com>

On Thu, Feb 15, 2024 at 10:42:18AM +0100, Juergen Gross wrote:
> On 15.02.24 07:55, Juergen Gross wrote:
> > Add support for the new 9pfs backend "xen_9pfsd". For this backend type
> > the tag defaults to "Xen" and the host side path to
> > "/var/log/xen/guests/<dom-name>".
> > 
> > Do most of the default settings in libxl. Unfortunately the default
> > path can't easily be set in libxl, as the domain name isn't available
> > in the related 9pfs specific function.
> > 
> > Settings the defaults in libxl requires to move the sanity checking
> > of 9pfs parameters from xl to libxl, too.
> > 
> > Signed-off-by: Juergen Gross <jgross@suse.com>
> > Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
> 
> Sorry Anthony, just found your R-b: in my spam folder :-(

I wonder if my @cloud.com email address would work better, and be less
broken.

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 12:11:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 12:11:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681647.1060492 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaa7-0001Tk-8l; Thu, 15 Feb 2024 12:11:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681647.1060492; Thu, 15 Feb 2024 12:11:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaa7-0001Td-5j; Thu, 15 Feb 2024 12:11:11 +0000
Received: by outflank-mailman (input) for mailman id 681647;
 Thu, 15 Feb 2024 12:11:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raaa5-0001TV-Cc
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 12:11:09 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4bb8ed36-cbfb-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 13:11:06 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-412191ed8aaso3470195e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 04:11:06 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 x17-20020a05600c21d100b00410b98a5c77sm4791449wmj.32.2024.02.15.04.11.05
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 04:11:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4bb8ed36-cbfb-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707999066; x=1708603866; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=BcRMUwPZcmaSPhqmiHsxfNjaMovTXTDT448L2EiX46M=;
        b=XkGXBnH35HMePRoxH3NnH1ohG8OVZvJ2768Dbcf3qSlOdi35EByPtduXWNGdRyPvIG
         GNUPK5dnLFl7mxUrQlD5yHzPG11OZ7h4WLYkqiGjvqY8PN/eAmtCQtYFQbrgbko/DhGS
         G3gJIGTLk2bLS5w+XLx36nuBIC0utvoCuEpzJ4EK4OFBFMVzEHE63bEPmttB7zydobEE
         c2/cHE9r8yihj7Lemd1PTTOkbY/mSMuzUN9KTNiGECsMOOV1wB6rNP6WHKGvItD3zzNj
         5p559AW+F9JjgduNHxmbTt4g/FORRicLepTxEvn2W4u+HHOOMwk/wLqrMzOTfDupZdcO
         dWQA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707999066; x=1708603866;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=BcRMUwPZcmaSPhqmiHsxfNjaMovTXTDT448L2EiX46M=;
        b=J10+soL/z9qlTHxPOOp0glFwPMnQI1yB0xenhaYeUtBPb5B42+hgFqME573caRC97b
         DzXP1a4MCdLnNJ+g53G38r/BxldTLIPDEm0dQRp11wK4PZzNZkPb/LlvN7pMVW6fynDj
         jGP5sU6SNUEM2e/tcDfzuYLmI4jLZvk64oeyQ1qvMgYHwLo79agrCfW9pV76IHfqC4lI
         x0IrTmxAc5yQMgWalS+pYGFUh7hLt49B6sO3rMwV8u+KtA184Mue7MbTr3X9Tkk9VoV7
         DmlWbMq8hLzFc4h8IrfV986BWKZ6OOjqEuti4usbuK0VUiVUyUZjO2tS5nbiPOEqCRKS
         bh0Q==
X-Forwarded-Encrypted: i=1; AJvYcCWlzsCSQ3TTRj+vW0dt7XBtLmScV2jGDJME3N5WA0ShytOTpS47sGTOYnkYnJSPyMCcZVJJDhZYN+hnIkF2XgUPhfM5xhBXMwDOUFJh6q8=
X-Gm-Message-State: AOJu0YxwkMDJjvBHsFV0ucUsUPenDm1ihX7W5NHJ9TBkMr1mgaRVVKRx
	I/7Lnfv8S3lQo480klsYL4k8pCMNZTl70oWUyQBN2Uy9xS62ZZC5Hywy8P90QkqI41m9zhzliJk
	=
X-Google-Smtp-Source: AGHT+IF6Db8sMd+Tb3U9olw/t7dSlP1TGmCvGVen1RoVuwd75gDChg/A5kFyp2AcluXgKh66BpTz7A==
X-Received: by 2002:a05:600c:1c8a:b0:412:1ccd:d770 with SMTP id k10-20020a05600c1c8a00b004121ccdd770mr898337wms.25.1707999065958;
        Thu, 15 Feb 2024 04:11:05 -0800 (PST)
Message-ID: <4a462da5-e826-489f-8b59-40c7e6c6ba68@suse.com>
Date: Thu, 15 Feb 2024 13:11:04 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: anthony.perard@citrix.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com> <Zc3nXpUOlnIHEfsl@macbook>
 <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com> <Zc3v20RKMssbaDsl@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zc3v20RKMssbaDsl@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.02.2024 12:04, Roger Pau Monné wrote:
> On Thu, Feb 15, 2024 at 11:43:02AM +0100, Jan Beulich wrote:
>> On 15.02.2024 11:28, Roger Pau Monné wrote:
>>> Without the reordering the include of include/config/auto.conf will
>>> always succeed on rebuilds, because the include is done before
>>> executing the include/config/%.conf.cmd target that does the `rm`.
>>
>> That's a dual target: It also handles include/config/%.conf. I.e.
>> because of this ...
>>
>>> With the current order the include of include/config/%.conf.cmd that
>>> triggers the re-build of auto.conf happens after having included the
>>> file already.
>>
>> ... either include would trigger this same rule. IOW I'm afraid I'm still
>> not seeing what is gained by the re-ordering. I'm also unconvinced that
>> "triggers" in the sense you use it is actually applicable. Quoting make
>> doc again: "Once it has finished reading makefiles, make will try to
>> remake any that are out of date or don’t exist." To me this means that
>> first all makefile reading will finish, and then whichever included files
>> need re-making will be re-made.
> 
> Without the re-ordering the execution is not stopped on failure to
> generate include/config/auto.conf:
> 
> # gmake -j8 xen clang=y
> gmake -C xen install
> gmake[1]: Entering directory '/root/src/xen/xen'
> rm -rf include/config/auto.conf
> tools/kconfig/conf  --syncconfig Kconfig
> common/Kconfig:2: syntax error
> common/Kconfig:1: invalid statement
> gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
> gmake[1]: Failed to remake makefile 'include/config/auto.conf'.
>   UPD     include/xen/compile.h
>  Xen 4.19-unstable
> gmake[3]: Nothing to be done for 'all'.
> [...]
> 
> With the re-ordering:
> 
> # gmake -j8 xen clang=y
> gmake -C xen install
> gmake[1]: Entering directory '/root/src/xen/xen'
> rm -rf include/config/auto.conf
> tools/kconfig/conf  --syncconfig Kconfig
> common/Kconfig:2: syntax error
> common/Kconfig:1: invalid statement
> gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
> gmake[1]: *** [Makefile:379: include/config/auto.conf] Error 2
> gmake[1]: Leaving directory '/root/src/xen/xen'
> gmake: *** [Makefile:143: install-xen] Error 2
> #
> 
> So the re-ordering is meaningful.

Hmm, I was about to say "no, it isn't" but then remembered to try newer
make. With 3.81 all is working as intended with no re-ordering. I wonder
if this isn't something with make, rather than our makefiles ...

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 12:11:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 12:11:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681651.1060503 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaak-0001yd-G3; Thu, 15 Feb 2024 12:11:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681651.1060503; Thu, 15 Feb 2024 12:11:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raaak-0001yW-Cr; Thu, 15 Feb 2024 12:11:50 +0000
Received: by outflank-mailman (input) for mailman id 681651;
 Thu, 15 Feb 2024 12:11:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1raaaj-0001yQ-1W
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 12:11:49 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1raaai-0005di-Os; Thu, 15 Feb 2024 12:11:48 +0000
Received: from [15.248.3.91] (helo=[10.24.67.27])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1raaai-0005x7-J6; Thu, 15 Feb 2024 12:11:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	References:Cc:To:From:Subject:MIME-Version:Date:Message-ID;
	bh=M42jUP/j5z3T21luWg7PFpdiKZtAZkxU8jMCcx2Pb2w=; b=TrAnQPe9ddP4rtnV0FZFlhzmTq
	F5A4HCYZ4kaKHqG6EnpODkKdTa4KvZaBKzK7fDZ+bUr4lhqqzPFD6B49mJ3/AKLBdqczaImF3V0Hw
	QmMbKvewyrbFa0yamJaDq2yav93Uq/EFJ6eIp8MrIgMiD6i4G8skkuw35PQOD2mAEd4M=;
Message-ID: <d7ba7416-480b-4dbd-8a27-0cc44f589bb9@xen.org>
Date: Thu, 15 Feb 2024 12:11:46 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] docs: add a best practices coding guide
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
To: Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20240208160515.9949-1-jgross@suse.com>
 <e090be7b-1ceb-4dc0-a664-089179236688@xen.org>
In-Reply-To: <e090be7b-1ceb-4dc0-a664-089179236688@xen.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 08/02/2024 16:27, Julien Grall wrote:
> Hi Juergen,
> 
> On 08/02/2024 16:05, Juergen Gross wrote:
>> Today the CODING_STYLE contains a section "Handling unexpected
>> conditions" specific to the hypervisor. This section is kind of
>> misplaced for a coding style. It should rather be part of a "Coding
>> best practices" guide.
>>
>> Add such a guide as docs/process/coding-best-practices.pandoc and
>> move the mentioned section from CODING_STYLE to the new file, while
>> converting the format to pandoc.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> Acked-by: Julien Grall <jgrall@amazon.com>

There was no other feedbacks. So I have committed it.

Cheers,

> 
> Cheers,
> 

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 12:18:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 12:18:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681661.1060512 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raahI-0002uV-5U; Thu, 15 Feb 2024 12:18:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681661.1060512; Thu, 15 Feb 2024 12:18:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raahI-0002uO-31; Thu, 15 Feb 2024 12:18:36 +0000
Received: by outflank-mailman (input) for mailman id 681661;
 Thu, 15 Feb 2024 12:18:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raahG-0002uI-Vp
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 12:18:34 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 564b3dd9-cbfc-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 13:18:33 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-411d715c401so6217285e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 04:18:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ch16-20020a5d5d10000000b0033b8881f31dsm1708886wrb.68.2024.02.15.04.18.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 04:18:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 564b3dd9-cbfc-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1707999513; x=1708604313; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=TcbtW0QQB+zc2LXVhjwVYholr8BwFtWxCTOuZ0Dj2ig=;
        b=ewHYlnnPjXj67KKg8KTrNxBt1TPPZsuDAbV7McvO+q28oyVedyu0UOR+GjcYZHt80R
         jBPwTZTzK1QvH0/B4090b3iOhrQx6f7S7BvFMxUtFCiJa5gzvPvE81eAYhRoOD6YB3vc
         YMJ3QlKLYT2GZUIZunBcTAQrOOl+xrUT9mLwrz+iyfhXo4PO9ME2WMyCbj85XMlTBaBE
         36tDlF1tr/LWWiDyIgt7ClH6laIXSNtxgQoA1gOBoKRDe9sklU0oklLtmtVFtfW5oqMR
         41ILc6BFnl6lh6ZW93ovT2l29p247sY7pqrwLv+1JP/FdqXN9//1FbXbNb5ji0x1a3BH
         FiLQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1707999513; x=1708604313;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=TcbtW0QQB+zc2LXVhjwVYholr8BwFtWxCTOuZ0Dj2ig=;
        b=Dm9h2PC0kKr/C5HfCTQbRvvvd/6DZyHd9nk3MbUiNiJVyOP8ndx2pxMWkhYJSqIVjZ
         udUjxekujGPF+sCpGBT77CLHvTa64OkIJsktP/aMo3YXplLCGyr0SBjumtuBj7D7Tq32
         f5gLE3+eCbgBgAV2BypPUuKjm88MpOI3ygQ4yCbbByn9Jbrms4lPbbywOJhnb7IxgYNO
         Jy+mcgQJqK8mm7b9TWRzNhyYMcEYSpjD1Keykn1YKh20jEol+fEWG3SR5wEVCGj0i1lR
         2Mn8i8C29PLUotxnyEb3JiILLZcfpcq1BvajOMk1RGv9ZLaL4LAPFxM+V8E1cfeF85+/
         2xTw==
X-Forwarded-Encrypted: i=1; AJvYcCXGkJ8M5+skDp0wdmS5bNHeRy1XulsMlafALwOUXumXufH4JsGNtoSWXgTAMhfdh82NBGkqmaVDuttOmYiFJPSXy2R3afqKyUb+vSneDLE=
X-Gm-Message-State: AOJu0YzUjtSezK0pnGOjkid/qzTmXxTqcfob1qmNJ1Bn9W3y190ZCC/h
	S/zMzct1pOKu2RVZOFT5nWa1tCVFQote4xnUe5VvAqExwZH6S81dWNvQuem0dQ==
X-Google-Smtp-Source: AGHT+IEfaY4bCk3TBCOmMkDZaZX1+QDARNBJYeKEas4Jzk6L596oLh1BPzxjRoMCVB48hPJDDHg3yA==
X-Received: by 2002:a5d:64a7:0:b0:33d:ed4:4aad with SMTP id m7-20020a5d64a7000000b0033d0ed44aadmr872114wrp.60.1707999513090;
        Thu, 15 Feb 2024 04:18:33 -0800 (PST)
Message-ID: <1c2c941a-984e-423e-9cfa-6dd103c26e11@suse.com>
Date: Thu, 15 Feb 2024 13:18:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: anthony.perard@citrix.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com> <Zc3nXpUOlnIHEfsl@macbook>
 <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com> <Zc3v20RKMssbaDsl@macbook>
 <4a462da5-e826-489f-8b59-40c7e6c6ba68@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4a462da5-e826-489f-8b59-40c7e6c6ba68@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.02.2024 13:11, Jan Beulich wrote:
> On 15.02.2024 12:04, Roger Pau Monné wrote:
>> On Thu, Feb 15, 2024 at 11:43:02AM +0100, Jan Beulich wrote:
>>> On 15.02.2024 11:28, Roger Pau Monné wrote:
>>>> Without the reordering the include of include/config/auto.conf will
>>>> always succeed on rebuilds, because the include is done before
>>>> executing the include/config/%.conf.cmd target that does the `rm`.
>>>
>>> That's a dual target: It also handles include/config/%.conf. I.e.
>>> because of this ...
>>>
>>>> With the current order the include of include/config/%.conf.cmd that
>>>> triggers the re-build of auto.conf happens after having included the
>>>> file already.
>>>
>>> ... either include would trigger this same rule. IOW I'm afraid I'm still
>>> not seeing what is gained by the re-ordering. I'm also unconvinced that
>>> "triggers" in the sense you use it is actually applicable. Quoting make
>>> doc again: "Once it has finished reading makefiles, make will try to
>>> remake any that are out of date or don’t exist." To me this means that
>>> first all makefile reading will finish, and then whichever included files
>>> need re-making will be re-made.
>>
>> Without the re-ordering the execution is not stopped on failure to
>> generate include/config/auto.conf:
>>
>> # gmake -j8 xen clang=y
>> gmake -C xen install
>> gmake[1]: Entering directory '/root/src/xen/xen'
>> rm -rf include/config/auto.conf
>> tools/kconfig/conf  --syncconfig Kconfig
>> common/Kconfig:2: syntax error
>> common/Kconfig:1: invalid statement
>> gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
>> gmake[1]: Failed to remake makefile 'include/config/auto.conf'.
>>   UPD     include/xen/compile.h
>>  Xen 4.19-unstable
>> gmake[3]: Nothing to be done for 'all'.
>> [...]
>>
>> With the re-ordering:
>>
>> # gmake -j8 xen clang=y
>> gmake -C xen install
>> gmake[1]: Entering directory '/root/src/xen/xen'
>> rm -rf include/config/auto.conf
>> tools/kconfig/conf  --syncconfig Kconfig
>> common/Kconfig:2: syntax error
>> common/Kconfig:1: invalid statement
>> gmake[2]: *** [tools/kconfig/Makefile:73: syncconfig] Error 1
>> gmake[1]: *** [Makefile:379: include/config/auto.conf] Error 2
>> gmake[1]: Leaving directory '/root/src/xen/xen'
>> gmake: *** [Makefile:143: install-xen] Error 2
>> #
>>
>> So the re-ordering is meaningful.
> 
> Hmm, I was about to say "no, it isn't" but then remembered to try newer
> make. With 3.81 all is working as intended with no re-ordering. I wonder
> if this isn't something with make, rather than our makefiles ...

And indeed it's quite strange: The same effect can be had by dropping the
- from the other include (and no re-ordering). Extending the sub-make
invocation to

	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig || { err=$$?; echo "$@: error $$err" >&2; exit $$err; }

I can see that it's the latter of the includes which is reported as $@,
and failure of the sub-make is conveyed only when it's the latter of
the includes which has no -. When really any of the targets being subject
of "include" without - should respect failure.

I'll play with this some more, but considering the overall situation
maybe (perhaps just partly) reverting the commit Anthony pointed out is
then the least bad option, accepting the warnings with old make.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:03:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:03:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681676.1060528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabO3-0002By-Ls; Thu, 15 Feb 2024 13:02:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681676.1060528; Thu, 15 Feb 2024 13:02:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabO3-0002Br-H9; Thu, 15 Feb 2024 13:02:47 +0000
Received: by outflank-mailman (input) for mailman id 681676;
 Thu, 15 Feb 2024 13:02:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rabO2-0002Bl-EC
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:02:46 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 81f265cf-cc02-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:02:43 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-411dd149c97so6329585e9.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 05:02:43 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h17-20020a05600c315100b004122b7a680dsm481815wmo.21.2024.02.15.05.02.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 05:02:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81f265cf-cc02-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708002163; x=1708606963; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8FhGuYE9Pg1G7/QKIdhUEz9mCVcvgJbOBO6LH5n9n0U=;
        b=VNVlfrpdf2jkBahAo/9TdalGIXN4xU3yPuepX3WMyNpB9On78NrwS4IWBXHyXaiAAp
         POc31pFCweDIj5KkpRdvzrtdpjwLJcZRDs5VCrs21lWAcDTUeQ29athAj5ZkCvi6osee
         Iihw+EdsdiYnMucMY7yElfrWSf6vyjeG2UE+pRdO0wJwyoKl+jdw69ClEawkJ5j/zdTF
         JAJLTdrVdPkMNB9njEs1Gkd2S7+vyZrCNZxRKwIjKSj2+nBrraQihL0MNjZBWdV3/eRI
         GY4jS2TDMIR7M+BdQhBtQnogu9KUADgG7ZlT3hmOLzzM2YX9P/OWSix8X1P1LQmuxO5d
         UkzA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708002163; x=1708606963;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8FhGuYE9Pg1G7/QKIdhUEz9mCVcvgJbOBO6LH5n9n0U=;
        b=Gar6gguImhaIFjHo9u/I+4gg0Jrlmx97crbfdRygWUQU3axW+PLW+SCBVjW7Vc6msu
         1Xw6KwtfO5A6+GVEjMP+j+x5rMcdO+Sxi1WsX8bix/wYES9I4UVuuEmv5cdygCUbPqA4
         K7a5uWNeqbSkMcotF1Q+Fna/FfverDw1x/J/Rn2D6JuBjqsug5A/thN/2YexJAlC23Qt
         f3LHYrVhlUQJrsZ8IMZy5xwOqd6ZRvrI6h1Vvq+KFiZNkl7+hdPCFUmioJu3DlRQQs38
         UjAEyqC9NMCNdpacYgUWCEzzPdwj4HN+/kOK/6NXXglpCfDDc1YpTR0HD+IBrkCGs5dx
         FxNA==
X-Forwarded-Encrypted: i=1; AJvYcCVtG+60WRc9G/irnZguXsyJfV+LO1A3VJGwFxDGUJtcKyHfJ6x0i0hKjV1HxUUtvjMlZk9TXibzs6FLYrYe9aGCKPnv6PqbVJJNOqMnQrc=
X-Gm-Message-State: AOJu0YzF1uYorMvL7aAdEJtXuuabp6GxPpgj/gLYhSMWDxWCMz7fo5BG
	8KeJQBGLnuXj/STauXGvUoLaW9ru64qHiLgBgkbgZrILNVym8SgLFD/Inkrl4Q==
X-Google-Smtp-Source: AGHT+IFl2yJn3y/xU7xvdrW0G1tUMl3Yzlxjne8iCZr3QgVDUaaW2PTI1fyr/f13hsaimAHDXUOKDA==
X-Received: by 2002:a05:600c:2203:b0:411:e634:8377 with SMTP id z3-20020a05600c220300b00411e6348377mr1444539wml.38.1708002163308;
        Thu, 15 Feb 2024 05:02:43 -0800 (PST)
Message-ID: <a48cbad6-701d-4077-9044-4205b932a7f3@suse.com>
Date: Thu, 15 Feb 2024 14:02:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Anthony Perard <anthony.perard@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com> <Zc3nXpUOlnIHEfsl@macbook>
 <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com> <Zc3v20RKMssbaDsl@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zc3v20RKMssbaDsl@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.02.2024 12:04, Roger Pau Monné wrote:
> On Thu, Feb 15, 2024 at 11:43:02AM +0100, Jan Beulich wrote:
>> On 15.02.2024 11:28, Roger Pau Monné wrote:
>>> On Thu, Feb 15, 2024 at 10:49:31AM +0100, Jan Beulich wrote:
>>>> On 15.02.2024 10:30, Roger Pau Monne wrote:
>>>>> --- a/xen/Makefile
>>>>> +++ b/xen/Makefile
>>>>> @@ -358,10 +358,10 @@ config: tools_fixdep outputmakefile FORCE
>>>>>  else # !config-build
>>>>>  
>>>>>  ifeq ($(need-config),y)
>>>>> --include include/config/auto.conf
>>>>>  # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
>>>>>  # changes are detected.
>>>>>  -include include/config/auto.conf.cmd
>>>>> +include include/config/auto.conf
>>>>
>>>> With the - dropped, ...
>>>>
>>>>> @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
>>>>>  # This exploits the 'multi-target pattern rule' trick.
>>>>>  # The syncconfig should be executed only once to make all the targets.
>>>>>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
>>>>> +	rm -rf include/config/$*.conf
>>>>>  	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
>>>>
>>>> ... is this really necessary? The error status from the sub-make is ignored
>>>> only because of the -, isn't it?
>>>
>>> Without the `rm` the include/config/auto.conf is not removed by
>>> Kconfig on error, so the include will still succeed but use the stale
>>> auto.conf file.
>>>
>>> Keep in mind on rebuilds include/config/auto.conf is already present,
>>> so the rule is only executed for the include/config/auto.conf.cmd
>>> target.
>>
>> But the sub-make ought to return failure, which ought to then stop the
>> build process?
> 
> For some reason it doesn't, not at least with GNU Make 4.3.
> 
> It stops the build if the '-' is dropped from the include of
> include/config/auto.conf.cmd.  But that will always fail as
> include/config/auto.conf.cmd is never created.
> 
> Maybe there's something weird with our makefile, I certainly don't
> know that much, but as noted in the commit message,
> include/config/auto.conf.cmd failing doesn't cause the build to
> stop.

How about the below as an alternative? I'm not overly happy with the
double ifneq, but I also don't see a good other option.

This way the missing auto.conf is detected slightly later, but this
may well be good enough. Then again I might be overlooking yet
something else that this breaks ...

Jan

--- a/xen/Makefile
+++ b/xen/Makefile
@@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
 # This exploits the 'multi-target pattern rule' trick.
 # The syncconfig should be executed only once to make all the targets.
 include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
+	$(Q)rm -f include/config/$*.conf
 	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
 
 ifeq ($(CONFIG_DEBUG),y)
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -15,7 +15,11 @@ srcdir := $(srctree)/$(src)
 PHONY := __build
 __build:
 
--include $(objtree)/include/config/auto.conf
+ifneq ($(obj),tools)
+ifneq ($(obj),tools/kconfig)
+include $(objtree)/include/config/auto.conf
+endif
+endif
 
 include $(XEN_ROOT)/Config.mk
 include $(srctree)/scripts/Kbuild.include



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:05:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:05:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681679.1060536 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQR-0002jK-WA; Thu, 15 Feb 2024 13:05:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681679.1060536; Thu, 15 Feb 2024 13:05:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQR-0002jD-TR; Thu, 15 Feb 2024 13:05:15 +0000
Received: by outflank-mailman (input) for mailman id 681679;
 Thu, 15 Feb 2024 13:05:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabQQ-0002j3-Qz
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:05:14 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id da937ad5-cc02-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:05:12 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 99B641F8A4;
 Thu, 15 Feb 2024 13:05:11 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 4B599139D0;
 Thu, 15 Feb 2024 13:05:11 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id IPCmEAcMzmVtFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:05:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da937ad5-cc02-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002311; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=Jtnp1ZrZVXiTiMpPlXn2mSwZvvrc43azN9NF1bRjhmk=;
	b=e1t2Ko2Id8/8l+WisNjTgHvAJWf/KGrOtIllfQXTmlfxTwt9bYRRipIyOOMpnLQYJ4YJNQ
	os5dHboI7+g4nUIeonU3eEI7P4y0GjqZdpZzOmAeErei31jzxaaH/YGABDxEEw4Sudfp9S
	I2Jn/flUcRBq4a51cpqLxF6JIPLDDeE=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002311; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=Jtnp1ZrZVXiTiMpPlXn2mSwZvvrc43azN9NF1bRjhmk=;
	b=e1t2Ko2Id8/8l+WisNjTgHvAJWf/KGrOtIllfQXTmlfxTwt9bYRRipIyOOMpnLQYJ4YJNQ
	os5dHboI7+g4nUIeonU3eEI7P4y0GjqZdpZzOmAeErei31jzxaaH/YGABDxEEw4Sudfp9S
	I2Jn/flUcRBq4a51cpqLxF6JIPLDDeE=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH v7 00/21] tools: enable xenstore-stubdom to use 9pfs
Date: Thu, 15 Feb 2024 14:04:48 +0100
Message-Id: <20240215130509.24008-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=e1t2Ko2I
X-Spamd-Result: default: False [1.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,ens-lyon.org];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 1.69
X-Rspamd-Queue-Id: 99B641F8A4
X-Spam-Level: *
X-Spam-Flag: NO
X-Spamd-Bar: +

This series is adding 9pfs support to Xenstore-stubdom, enabling it
to do logging to a dom0 directory.

This is a prerequisite for the final goal to add live update support
to Xenstore-stubdom, as it enables the stubdom to store its state in
a dom0 file.

The 9pfs backend is a new daemon written from scratch. Using a
dedicated 9pfs daemon has several advantages:

- it is using much less resources than a full blown qemu process
- it can serve multiple guests (the idea is to use it for other
  infrastructure domains, like qemu-stubdom or driver domains, too)
- it is designed to support several security enhancements, like
  limiting the number of files for a guest, or limiting the allocated
  file system space
- it doesn't support file links (neither hard nor soft links) or
  referencing parent directories via "..", minimizing the risk that
  a guest can "escape" from its home directory

Note that for now the daemon only contains the minimal needed
functionality to do logging from Xenstore-stubdom. I didn't want to
add all the 9pfs commands and security add-ons in the beginning, in
order to avoid needless efforts in case the idea of the daemon is
being rejected.

Please note that the pending patch for updating the Mini-OS commit
in Config.mk needs to be applied for patch "stubdom: extend xenstore
stubdom configs" and the following ones.

Changes in V7:
- fixed V6 bugs

Changes in V6:
- patch 1 of V5 has been applied
- rebase
- addressed comments

Changes in V5:
- 10 patches have been applied already
- rename source directory to tools/9pfsd
- addressed comments

Changes in V4:
- patch 2 of V3 was applied
- added support of reading directories
- addressed review comments

Changes in V3:
- new patches 1, 23-25
- addressed review comments

Changes in V2:
- support of multiple rings per device
- xenlogd->xen-9pfsd rename
- addressed review comments
- fixed some bugs

Juergen Gross (21):
  tools: add a new xen 9pfs daemon
  tools/xen-9pfsd: connect to frontend
  tools/xen-9pfsd: add transport layer
  tools/xen-9pfsd: add 9pfs response generation support
  tools/xen-9pfsd: add 9pfs version request support
  tools/xen-9pfsd: add 9pfs attach request support
  tools/xen-9pfsd: add 9pfs walk request support
  tools/xen-9pfsd: add 9pfs open request support
  tools/xen-9pfsd: add 9pfs clunk request support
  tools/xen-9pfsd: add 9pfs create request support
  tools/xen-9pfsd: add 9pfs stat request support
  tools/xen-9pfsd: add 9pfs write request support
  tools/xen-9pfsd: add 9pfs read request support
  tools/libs/light: add backend type for 9pfs PV devices
  tools/xl: support new 9pfs backend xen_9pfsd
  stubdom: extend xenstore stubdom configs
  tools: add 9pfs device to xenstore-stubdom
  tools/xenstored: mount 9pfs device in stubdom
  tools/xenstored: add helpers for filename handling
  tools/xenstored: support complete log capabilities in stubdom
  tools/xenstored: have a single do_control_memreport()

 docs/man/xl.cfg.5.pod.in                      |   36 +-
 stubdom/xenstore-minios.cfg                   |    2 +-
 stubdom/xenstorepvh-minios.cfg                |    2 +-
 tools/9pfsd/.gitignore                        |    1 +
 tools/9pfsd/Makefile                          |   38 +
 tools/9pfsd/io.c                              | 1511 +++++++++++++++++
 tools/9pfsd/xen-9pfsd.c                       |  800 +++++++++
 tools/9pfsd/xen-9pfsd.h                       |   99 ++
 tools/Makefile                                |    1 +
 tools/golang/xenlight/helpers.gen.go          |   10 +
 tools/golang/xenlight/types.gen.go            |   12 +
 tools/helpers/init-xenstore-domain.c          |    7 +
 .../Linux/init.d/sysconfig.xencommons.in      |    1 -
 tools/hotplug/Linux/launch-xenstore.in        |    1 +
 tools/include/libxl.h                         |   22 +
 tools/libs/light/libxl_9pfs.c                 |  191 ++-
 tools/libs/light/libxl_create.c               |    4 +-
 tools/libs/light/libxl_dm.c                   |    2 +-
 tools/libs/light/libxl_types.idl              |   11 +
 tools/libs/light/libxl_types_internal.idl     |    1 +
 tools/xenstored/control.c                     |   29 +-
 tools/xenstored/core.c                        |   15 +-
 tools/xenstored/core.h                        |   11 +-
 tools/xenstored/domain.c                      |    2 +
 tools/xenstored/lu_daemon.c                   |    4 +-
 tools/xenstored/minios.c                      |   62 +
 tools/xenstored/posix.c                       |    8 +-
 tools/xl/xl_parse.c                           |   23 +-
 28 files changed, 2864 insertions(+), 42 deletions(-)
 create mode 100644 tools/9pfsd/.gitignore
 create mode 100644 tools/9pfsd/Makefile
 create mode 100644 tools/9pfsd/io.c
 create mode 100644 tools/9pfsd/xen-9pfsd.c
 create mode 100644 tools/9pfsd/xen-9pfsd.h

-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:05:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:05:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681680.1060547 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQW-0002zD-6p; Thu, 15 Feb 2024 13:05:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681680.1060547; Thu, 15 Feb 2024 13:05:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQW-0002z4-40; Thu, 15 Feb 2024 13:05:20 +0000
Received: by outflank-mailman (input) for mailman id 681680;
 Thu, 15 Feb 2024 13:05:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabQU-0002yD-RN
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:05:18 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dda944a4-cc02-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:05:17 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 3A4861F8A4;
 Thu, 15 Feb 2024 13:05:17 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 04E40139D0;
 Thu, 15 Feb 2024 13:05:16 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id BU9iOwwMzmVxFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:05:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dda944a4-cc02-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002317; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=rQPRnnpvHJ9iAPaCFRu4161m+4eTjuy3Ob9j9LfvB8M=;
	b=S5wYaxMBdR42E3tgguDQHkkN9QW8UjTkoOEUy5iNQ4stWpsxcsXr+5f1p7G6rkCZVx11t5
	smqZtFF7L5NlMCB1vmKUI3kP33SVNgqfm3qYOoAoUiolsr7pfZKyToFEl7eSXrFeN1/98V
	1HVoZM8AxAjp/yF/RT6SeMp9kPZQivQ=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002317; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=rQPRnnpvHJ9iAPaCFRu4161m+4eTjuy3Ob9j9LfvB8M=;
	b=S5wYaxMBdR42E3tgguDQHkkN9QW8UjTkoOEUy5iNQ4stWpsxcsXr+5f1p7G6rkCZVx11t5
	smqZtFF7L5NlMCB1vmKUI3kP33SVNgqfm3qYOoAoUiolsr7pfZKyToFEl7eSXrFeN1/98V
	1HVoZM8AxAjp/yF/RT6SeMp9kPZQivQ=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH v7 01/21] tools: add a new xen 9pfs daemon
Date: Thu, 15 Feb 2024 14:04:49 +0100
Message-Id: <20240215130509.24008-2-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=S5wYaxMB
X-Spamd-Result: default: False [1.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[6];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 1.69
X-Rspamd-Queue-Id: 3A4861F8A4
X-Spam-Level: *
X-Spam-Flag: NO
X-Spamd-Bar: +

Add "xen-9pfsd", a new 9pfs daemon meant to support infrastructure
domains (e.g. xenstore-stubdom) to access files in dom0.

For now only add the code needed for starting the daemon and
registering it with Xenstore via a new "libxl/xen-9pfs/state" node by
writing the "running" state to it.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- rename from xenlogd to xen-9pfsd (Andrew Cooper)
- use a backend domain local Xenstore node (Jason Andryuk)
- use "volatile" for stop_me (Andrew Cooper)
V3:
- fix SPDX Identifier (Andrew Cooper)
V4:
- add strerror() test to error logging (Andrew Cooper)
- don't handle "daemon already running" as error (Anthony Perard)
V5:
- rename source directory to tools/9pfsd (Andrew Cooper)
---
 tools/9pfsd/.gitignore  |   1 +
 tools/9pfsd/Makefile    |  38 +++++++++++
 tools/9pfsd/xen-9pfsd.c | 147 ++++++++++++++++++++++++++++++++++++++++
 tools/Makefile          |   1 +
 4 files changed, 187 insertions(+)
 create mode 100644 tools/9pfsd/.gitignore
 create mode 100644 tools/9pfsd/Makefile
 create mode 100644 tools/9pfsd/xen-9pfsd.c

diff --git a/tools/9pfsd/.gitignore b/tools/9pfsd/.gitignore
new file mode 100644
index 0000000000..d0c2d223ef
--- /dev/null
+++ b/tools/9pfsd/.gitignore
@@ -0,0 +1 @@
+/xen-9pfsd
diff --git a/tools/9pfsd/Makefile b/tools/9pfsd/Makefile
new file mode 100644
index 0000000000..089cf5ae24
--- /dev/null
+++ b/tools/9pfsd/Makefile
@@ -0,0 +1,38 @@
+#
+# tools/9pfsd/Makefile
+#
+
+XEN_ROOT = $(CURDIR)/../..
+include $(XEN_ROOT)/tools/Rules.mk
+
+CFLAGS += $(PTHREAD_CFLAGS)
+LDFLAGS += $(PTHREAD_LDFLAGS)
+
+TARGETS := xen-9pfsd
+
+XEN-9PFSD_OBJS = xen-9pfsd.o
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenstore)
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenevtchn)
+$(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
+xen-9pfsd: LDLIBS += $(call xenlibs-ldlibs,store evtchn gnttab)
+
+.PHONY: all
+all: $(TARGETS)
+
+xen-9pfsd: $(XEN-9PFSD_OBJS)
+	$(CC) $(LDFLAGS) -o $@ $(XEN-9PFSD_OBJS) $(LDLIBS) $(APPEND_LDFLAGS)
+
+.PHONY: install
+install: all
+	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
+	for i in $(TARGETS); do $(INSTALL_PROG) $$i $(DESTDIR)$(LIBEXEC_BIN); done
+
+.PHONY: uninstall
+uninstall:
+	for i in $(TARGETS); do rm -f $(DESTDIR)$(LIBEXEC_BIN)/$$i; done
+
+.PHONY: clean
+clean:
+	$(RM) *.o $(TARGETS) $(DEPS_RM)
+
+distclean: clean
diff --git a/tools/9pfsd/xen-9pfsd.c b/tools/9pfsd/xen-9pfsd.c
new file mode 100644
index 0000000000..6939d01574
--- /dev/null
+++ b/tools/9pfsd/xen-9pfsd.c
@@ -0,0 +1,147 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+/*
+ * xen-9pfsd - Xen 9pfs daemon
+ *
+ * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
+ *
+ * Daemon to enable guests to access a directory of the dom0 file system.
+ * Access is made via the 9pfs protocol (xen-9pfsd acts as a PV 9pfs backend).
+ *
+ * Usage: xen-9pfsd
+ *
+ * xen-9pfsd does NOT support writing any links (neither soft links nor hard
+ * links), and it is accepting only canonicalized file paths in order to
+ * avoid the possibility to "escape" from the guest specific directory.
+ *
+ * The backend device string is "xen_9pfs", the tag used for mounting the
+ * 9pfs device is "Xen".
+ *
+ * As an additional security measure the maximum file space used by the guest
+ * can be limited by the backend Xenstore node "max-size" specifying the size
+ * in MBytes. This size includes the size of the root directory of the guest.
+ */
+
+#include <err.h>
+#include <errno.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <syslog.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <xenevtchn.h>
+#include <xengnttab.h>
+#include <xenstore.h>
+
+static volatile bool stop_me;
+static bool daemon_running;
+static struct xs_handle *xs;
+static xengnttab_handle *xg;
+static xenevtchn_handle *xe;
+
+static void handle_stop(int sig)
+{
+    stop_me = true;
+}
+
+static void close_all(void)
+{
+    if ( daemon_running )
+        xs_rm(xs, XBT_NULL, "libxl/xen-9pfs");
+    if ( xe )
+        xenevtchn_close(xe);
+    if ( xg )
+        xengnttab_close(xg);
+    if ( xs )
+        xs_close(xs);
+    closelog();
+}
+
+static void do_err(const char *msg)
+{
+    syslog(LOG_ALERT, "%s, errno = %d, %s", msg, errno, strerror(errno));
+    close_all();
+    exit(1);
+}
+
+static void xen_connect(void)
+{
+    xs_transaction_t t;
+    char *val;
+    unsigned int len;
+
+    xs = xs_open(0);
+    if ( xs == NULL )
+        do_err("xs_open() failed");
+
+    xg = xengnttab_open(NULL, 0);
+    if ( xg == NULL )
+        do_err("xengnttab_open() failed");
+
+    xe = xenevtchn_open(NULL, 0);
+    if ( xe == NULL )
+        do_err("xenevtchn_open() failed");
+
+    while ( true )
+    {
+        t = xs_transaction_start(xs);
+        if ( t == XBT_NULL )
+            do_err("xs_transaction_start() failed");
+
+        val = xs_read(xs, t, "libxl/xen-9pfs/state", &len);
+        if ( val )
+        {
+            free(val);
+            xs_transaction_end(xs, t, true);
+            syslog(LOG_INFO, "daemon already running");
+            close_all();
+            exit(0);
+        }
+
+        if ( !xs_write(xs, t, "libxl/xen-9pfs/state", "running",
+                       strlen("running")) )
+        {
+            xs_transaction_end(xs, t, true);
+            do_err("xs_write() failed writing state");
+        }
+
+        if ( xs_transaction_end(xs, t, false) )
+            break;
+        if ( errno != EAGAIN )
+            do_err("xs_transaction_end() failed");
+    }
+
+    daemon_running = true;
+}
+
+int main(int argc, char *argv[])
+{
+    struct sigaction act = { .sa_handler = handle_stop, };
+    int syslog_mask = LOG_MASK(LOG_WARNING) | LOG_MASK(LOG_ERR) |
+                      LOG_MASK(LOG_CRIT) | LOG_MASK(LOG_ALERT) |
+                      LOG_MASK(LOG_EMERG);
+
+    umask(027);
+    if ( getenv("XEN_9PFSD_VERBOSE") )
+        syslog_mask |= LOG_MASK(LOG_NOTICE) | LOG_MASK(LOG_INFO);
+    openlog("xen-9pfsd", LOG_CONS, LOG_DAEMON);
+    setlogmask(syslog_mask);
+
+    sigemptyset(&act.sa_mask);
+    sigaction(SIGHUP, &act, NULL);
+
+    xen_connect();
+
+    while ( !stop_me )
+    {
+        sleep(60);
+    }
+
+    close_all();
+
+    return 0;
+}
diff --git a/tools/Makefile b/tools/Makefile
index 3a510663a0..7d17211782 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -32,6 +32,7 @@ SUBDIRS-y += xenpmd
 SUBDIRS-$(CONFIG_GOLANG) += golang
 SUBDIRS-y += xl
 SUBDIRS-y += helpers
+SUBDIRS-y += 9pfsd
 SUBDIRS-$(CONFIG_X86) += xenpaging
 SUBDIRS-$(CONFIG_X86) += debugger
 SUBDIRS-$(CONFIG_TESTS) += tests
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:05:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:05:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681681.1060557 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQc-0003Ih-Jo; Thu, 15 Feb 2024 13:05:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681681.1060557; Thu, 15 Feb 2024 13:05:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQc-0003IY-G1; Thu, 15 Feb 2024 13:05:26 +0000
Received: by outflank-mailman (input) for mailman id 681681;
 Thu, 15 Feb 2024 13:05:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabQb-0002j3-HR
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:05:25 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e0e694f9-cc02-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:05:23 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id D11BF21FBC;
 Thu, 15 Feb 2024 13:05:22 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 9C61E139D0;
 Thu, 15 Feb 2024 13:05:22 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id sj/OJBIMzmVzFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:05:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0e694f9-cc02-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002322; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2rcpEbJbgUxiNrBORk+pKJOPpR9mCjteYZqR1AIlbbY=;
	b=C8T/0ZV4tO8ots6sBRqict1+jS/oEW9uW0uVKX2Gza3KmZ7iyEg4IUZLoeELdrbaghnxhE
	tZg4AEERJOzCnEN2+ilRHJ2/ITxjACl0TQoiIv/jF80lHYHFa9leXUilFQLavlgUasISkn
	dWI74THZOzdvBnVuaOanhmfJqTcUCuM=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002322; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=2rcpEbJbgUxiNrBORk+pKJOPpR9mCjteYZqR1AIlbbY=;
	b=C8T/0ZV4tO8ots6sBRqict1+jS/oEW9uW0uVKX2Gza3KmZ7iyEg4IUZLoeELdrbaghnxhE
	tZg4AEERJOzCnEN2+ilRHJ2/ITxjACl0TQoiIv/jF80lHYHFa9leXUilFQLavlgUasISkn
	dWI74THZOzdvBnVuaOanhmfJqTcUCuM=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 02/21] tools/xen-9pfsd: connect to frontend
Date: Thu, 15 Feb 2024 14:04:50 +0100
Message-Id: <20240215130509.24008-3-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-2.10 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: -2.10

Add the code for connecting to frontends to xenlogd.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- support multiple rings per device (Jason Andryuk)
- don't set .revents initially (Jason Andryuk)
- call poll() with infinite timeout (Jason Andryuk)
- take mutex before calling pthread_cond_signal()
V3:
- fix SPDX indentifier (Andrew Cooper)
- better validation of host path (Jason Andryuk)
- don't hard-code dom0 in backend nodes (Jason Andryuk)
- use bool instead of int for some functions' return types
- open root directory (Jason Andryuk)
---
 tools/9pfsd/Makefile    |   2 +-
 tools/9pfsd/io.c        |  45 +++
 tools/9pfsd/xen-9pfsd.c | 653 +++++++++++++++++++++++++++++++++++++++-
 tools/9pfsd/xen-9pfsd.h |  61 ++++
 4 files changed, 757 insertions(+), 4 deletions(-)
 create mode 100644 tools/9pfsd/io.c
 create mode 100644 tools/9pfsd/xen-9pfsd.h

diff --git a/tools/9pfsd/Makefile b/tools/9pfsd/Makefile
index 089cf5ae24..50573121ed 100644
--- a/tools/9pfsd/Makefile
+++ b/tools/9pfsd/Makefile
@@ -10,7 +10,7 @@ LDFLAGS += $(PTHREAD_LDFLAGS)
 
 TARGETS := xen-9pfsd
 
-XEN-9PFSD_OBJS = xen-9pfsd.o
+XEN-9PFSD_OBJS = xen-9pfsd.o io.o
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenstore)
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxenevtchn)
 $(XEN-9PFSD_OBJS): CFLAGS += $(CFLAGS_libxengnttab)
diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
new file mode 100644
index 0000000000..eb7c136e09
--- /dev/null
+++ b/tools/9pfsd/io.c
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+/*
+ * xen-9pfsd - Xen 9pfs daemon
+ *
+ * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
+ *
+ * I/O thread handling.
+ */
+
+#include <stdbool.h>
+#include <string.h>
+#include <syslog.h>
+
+#include "xen-9pfsd.h"
+
+static bool io_work_pending(struct ring *ring)
+{
+    if ( ring->stop_thread )
+        return true;
+    return false;
+}
+
+void *io_thread(void *arg)
+{
+    struct ring *ring = arg;
+
+    while ( !ring->stop_thread )
+    {
+        pthread_mutex_lock(&ring->mutex);
+        if ( !io_work_pending(ring) )
+        {
+            if ( xenevtchn_unmask(xe, ring->evtchn) < 0 )
+                syslog(LOG_WARNING, "xenevtchn_unmask() failed");
+            pthread_cond_wait(&ring->cond, &ring->mutex);
+        }
+        pthread_mutex_unlock(&ring->mutex);
+
+        /* TODO: I/O handling. */
+    }
+
+    ring->thread_active = false;
+
+    return NULL;
+}
diff --git a/tools/9pfsd/xen-9pfsd.c b/tools/9pfsd/xen-9pfsd.c
index 6939d01574..73b6c3a30e 100644
--- a/tools/9pfsd/xen-9pfsd.c
+++ b/tools/9pfsd/xen-9pfsd.c
@@ -24,34 +24,632 @@
 
 #include <err.h>
 #include <errno.h>
+#include <fcntl.h>
+#include <poll.h>
+#include <pthread.h>
 #include <signal.h>
 #include <stdbool.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <sys/mman.h>
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <unistd.h>
-#include <xenevtchn.h>
 #include <xengnttab.h>
 #include <xenstore.h>
 
+#include "xen-9pfsd.h"
+
+/*
+ * List of currently known devices.
+ * The list itself is modified only in the main thread. When a device is being
+ * removed its memory needs to be freed after the I/O thread (if existing)
+ * has stopped.
+ */
+static XEN_TAILQ_HEAD(devhead, device) devs = XEN_TAILQ_HEAD_INITIALIZER(devs);
+
+struct path {
+    char path[100];
+};
+
 static volatile bool stop_me;
 static bool daemon_running;
 static struct xs_handle *xs;
 static xengnttab_handle *xg;
-static xenevtchn_handle *xe;
+static unsigned int now;
+
+xenevtchn_handle *xe;
 
 static void handle_stop(int sig)
 {
     stop_me = true;
 }
 
+static int check_host_path(device *device)
+{
+    struct stat statbuf;
+    char *path, *p;
+    int ret = 1;
+
+    if ( !device->host_path )
+        return 1;
+
+    /* Path must be absolute. */
+    if ( device->host_path[0] != '/' )
+        return 1;
+
+    /* No double "/". */
+    if ( strstr(device->host_path, "//") )
+        return 1;
+
+    /* No trailing "/" (includes refusing to share "/"). */
+    if ( device->host_path[strlen(device->host_path) - 1] == '/' )
+        return 1;
+
+    path = strdup(device->host_path);
+    if ( !path )
+    {
+        syslog(LOG_CRIT, "memory allocation failure!");
+        return 1;
+    }
+
+    for ( p = path; p; )
+    {
+        p = strchr(p + 1, '/');
+        if ( p )
+            *p = 0;
+        if ( !stat(path, &statbuf) )
+        {
+            if ( !(statbuf.st_mode & S_IFDIR) )
+                break;
+            if ( !p )
+            {
+                ret = 0;
+                break;
+            }
+            *p = '/';
+            continue;
+        }
+        if ( mkdir(path, 0777) )
+            break;
+        if ( p )
+            *p = '/';
+    }
+
+    free(path);
+    return ret;
+}
+
+static void construct_frontend_path(device *device, const char *node,
+                                    struct path *p)
+{
+    snprintf(p->path, sizeof(p->path), "/local/domain/%u/device/9pfs/%u/%s",
+             device->domid, device->devid, node);
+}
+
+static void construct_backend_path(device *device, const char *node,
+                                   struct path *p)
+{
+    snprintf(p->path, sizeof(p->path), "backend/xen_9pfs/%u/%u/%s",
+             device->domid, device->devid, node);
+}
+
+static char *read_backend_node(device *device, const char *node)
+{
+    struct path p;
+    char *val;
+    unsigned int len;
+
+    construct_backend_path(device, node, &p);
+    val = xs_read(xs, XBT_NULL, p.path, &len);
+
+    return val;
+}
+
+static unsigned int uint_from_string(char *string, unsigned int def)
+{
+    unsigned long val;
+    char *end;
+
+    if ( !string )
+        return def;
+
+    val = strtoul(string, &end, 10);
+    if ( *end || val > UINT_MAX )
+        val = def;
+    free(string);
+
+    return val;
+}
+
+static unsigned int read_backend_node_uint(device *device, const char *node,
+                                           unsigned int def)
+{
+    return uint_from_string(read_backend_node(device, node), def);
+}
+
+static unsigned int read_frontend_node_uint(device *device, const char *node,
+                                            unsigned int def)
+{
+    struct path p;
+    unsigned int len;
+
+    construct_frontend_path(device, node, &p);
+
+    return uint_from_string(xs_read(xs, XBT_NULL, p.path, &len), def);
+}
+
+static bool write_backend_node(device *device, const char *node,
+                               const char *val)
+{
+    struct path p;
+    unsigned int num_perms;
+    struct xs_permissions *old_perms;
+    struct xs_permissions perms[2] = {
+        { .id = 0, .perms = XS_PERM_NONE },
+        { .id = device->domid, .perms = XS_PERM_READ }
+    };
+
+    construct_backend_path(device, node, &p);
+    if ( !xs_write(xs, XBT_NULL, p.path, val, strlen(val)) )
+    {
+        syslog(LOG_ERR, "error writing bacḱend node \"%s\" for device %u/%u",
+               node, device->domid, device->devid);
+        return false;
+    }
+
+    old_perms = xs_get_permissions(xs, XBT_NULL, p.path, &num_perms);
+    if ( !old_perms )
+    {
+        syslog(LOG_ERR, "error getting permissions for \"%s\"", p.path);
+        return false;
+    }
+
+    perms[0] = old_perms[0];
+    free(old_perms);
+    if ( !xs_set_permissions(xs, XBT_NULL, p.path, perms, 2) )
+    {
+        syslog(LOG_ERR, "error setting permissions for \"%s\"", p.path);
+        return false;
+    }
+
+    return true;
+}
+
+static bool write_backend_node_uint(device *device, const char *node,
+                                   unsigned int val)
+{
+    char str[12];
+
+    snprintf(str, sizeof(str), "%u", val);
+
+    return write_backend_node(device, node, str);
+}
+
+static bool write_backend_state(device *device, enum xenbus_state state)
+{
+    struct path p;
+    char val[2];
+
+    snprintf(val, sizeof(val), "%u", state);
+    construct_backend_path(device, "state", &p);
+    if ( !xs_write(xs, XBT_NULL, p.path, val, 1) )
+    {
+        syslog(LOG_ERR, "error writing backend state %u for device %u/%u",
+               state, device->domid, device->devid);
+        return false;
+    }
+
+    device->backend_state = state;
+
+    return true;
+}
+
+static device *find_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+
+    XEN_TAILQ_FOREACH( device, &devs, list )
+    {
+        if ( domid == device->domid && devid == device->devid )
+            return device;
+    }
+
+    return NULL;
+}
+
+static void free_device(device *device)
+{
+    char token[20];
+    struct path p;
+
+    construct_frontend_path(device, "state", &p);
+    snprintf(token, sizeof(token), "%u/%u", device->domid, device->devid);
+    xs_unwatch(xs, p.path, token);
+
+    if ( device->root_fd >= 0 )
+        close(device->root_fd);
+
+    free(device->host_path);
+    free(device);
+}
+
+static device *new_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+    char token[20];
+    struct path p;
+    char *val;
+
+    device = calloc(1, sizeof(*device));
+    if ( !device )
+    {
+        syslog(LOG_CRIT, "Got no memory for new device %u/%u", domid, devid);
+        return NULL;
+    }
+
+    device->domid = domid;
+    device->devid = devid;
+    device->root_fd = -1;
+
+    construct_frontend_path(device, "state", &p);
+    snprintf(token, sizeof(token), "%u/%u", domid, devid);
+    if ( !xs_watch(xs, p.path, token) )
+    {
+        syslog(LOG_ERR, "Setting up watch for device %u/%u failed",
+               domid, devid);
+        free(device);
+        return NULL;
+    }
+
+    val = read_backend_node(device, "security_model");
+    if ( !val || strcmp(val, "none") )
+    {
+        syslog(LOG_ERR, "Security model \"%s\" for device %u/%u invalid.",
+               val, domid, devid);
+        free(val);
+        goto err;
+    }
+    free(val);
+
+    device->max_space = read_backend_node_uint(device, "max-space", 0);
+    device->max_files = read_backend_node_uint(device, "max-files", 0);
+    device->max_open_files =
+        read_backend_node_uint(device, "max-open-files", 0)
+        ?: MAX_OPEN_FILES_DEFAULT;
+    device->auto_delete = read_backend_node_uint(device, "auto-delete", 0);
+
+    device->host_path = read_backend_node(device, "path");
+    if ( check_host_path(device) )
+    {
+        syslog(LOG_ERR, "Host path \"%s\" for device %u/%u invalid.",
+               device->host_path, domid, devid);
+        goto err;
+    }
+    device->root_fd = open(device->host_path, O_RDONLY | O_DIRECTORY);
+    if ( device->root_fd < 0 )
+        goto err;
+
+    if ( !write_backend_node(device, "versions", "1") )
+        goto err;
+    if ( !write_backend_node_uint(device, "max-rings", MAX_RINGS) )
+        goto err;
+    if ( !write_backend_node_uint(device, "max-ring-page-order",
+                                 MAX_RING_ORDER) )
+        goto err;
+
+    if ( !write_backend_state(device, XenbusStateInitWait) )
+        goto err;
+
+    XEN_TAILQ_INSERT_TAIL(&devs, device, list);
+    syslog(LOG_INFO, "New device %u/%u added", domid, devid);
+
+    return device;
+
+ err:
+    free_device(device);
+    return NULL;
+}
+
+static void disconnect_ring(struct ring *ring)
+{
+    if ( !ring )
+        return;
+
+    if ( ring->thread_active )
+    {
+        ring->stop_thread = true;
+        pthread_cond_signal(&ring->cond);
+        pthread_join(ring->thread, NULL);
+        ring->stop_thread = false;
+    }
+
+    if ( ring->data.in )
+    {
+        xengnttab_unmap(xg, ring->data.in, 1 << ring->ring_order);
+        ring->data.in = NULL;
+    }
+    if ( ring->intf )
+    {
+        xengnttab_unmap(xg, ring->intf, 1 );
+        ring->intf = NULL;
+    }
+
+    if ( ring->evtchn )
+    {
+        xenevtchn_unbind(xe, ring->evtchn);
+        ring->evtchn = 0;
+    }
+
+    pthread_mutex_destroy(&ring->mutex);
+    pthread_cond_destroy(&ring->cond);
+}
+
+static void disconnect_guest(device *device)
+{
+    unsigned int ring_idx;
+
+    for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+    {
+        disconnect_ring(device->ring[ring_idx]);
+        free(device->ring[ring_idx]);
+        device->ring[ring_idx] = NULL;
+    }
+
+    device->num_rings = 0;
+}
+
+static void close_device(device *device, enum xenbus_state state)
+{
+    disconnect_guest(device);
+    write_backend_state(device, state);
+}
+
+static void connect_err(device *device, const char *msg)
+{
+    syslog(LOG_WARNING, "%s", msg);
+    close_device(device, XenbusStateClosed);
+}
+
+static void connect_device(device *device)
+{
+    unsigned int val;
+    unsigned int ring_idx;
+    char node[20];
+    struct ring *ring;
+    xenevtchn_port_or_error_t evtchn;
+
+    val = read_frontend_node_uint(device, "version", 0);
+    if ( val != 1 )
+        return connect_err(device, "frontend specifies illegal version");
+    device->num_rings = read_frontend_node_uint(device, "num-rings", 0);
+    if ( device->num_rings < 1 || device->num_rings > MAX_RINGS )
+        return connect_err(device, "frontend specifies illegal ring number");
+
+    for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+    {
+        ring = calloc(1, sizeof(*ring));
+        if ( !ring )
+            return connect_err(device, "could not allocate ring memory");
+        device->ring[ring_idx] = ring;
+        ring->device = device;
+        pthread_cond_init(&ring->cond, NULL);
+        pthread_mutex_init(&ring->mutex, NULL);
+
+        snprintf(node, sizeof(node), "event-channel-%u", ring_idx);
+        val = read_frontend_node_uint(device, node, 0);
+        if ( val == 0 )
+            return connect_err(device, "frontend specifies illegal evtchn");
+        evtchn = xenevtchn_bind_interdomain(xe, device->domid, val);
+        if ( evtchn < 0 )
+            return connect_err(device, "could not bind to event channel");
+        ring->evtchn = evtchn;
+
+        snprintf(node, sizeof(node), "ring-ref%u", ring_idx);
+        val = read_frontend_node_uint(device, node, 0);
+        if ( val == 0 )
+            return connect_err(device,
+                               "frontend specifies illegal grant for ring");
+        ring->intf = xengnttab_map_grant_ref(xg, device->domid, val,
+                                             PROT_READ | PROT_WRITE);
+        if ( !ring->intf )
+            return connect_err(device, "could not map interface page");
+        ring->ring_order = ring->intf->ring_order;
+        if ( ring->ring_order > MAX_RING_ORDER || ring->ring_order < 1 )
+            return connect_err(device, "frontend specifies illegal ring order");
+        ring->ring_size = XEN_FLEX_RING_SIZE(ring->ring_order);
+        ring->data.in = xengnttab_map_domain_grant_refs(xg,
+                                                        1 << ring->ring_order,
+                                                        device->domid,
+                                                        ring->intf->ref,
+                                                        PROT_READ | PROT_WRITE);
+        if ( !ring->data.in )
+            return connect_err(device, "could not map ring pages");
+        ring->data.out = ring->data.in + ring->ring_size;
+
+        if ( pthread_create(&ring->thread, NULL, io_thread, ring) )
+            return connect_err(device, "could not start I/O thread");
+        ring->thread_active = true;
+    }
+
+    write_backend_state(device, XenbusStateConnected);
+}
+
+static void remove_device(device *device)
+{
+    XEN_TAILQ_REMOVE(&devs, device, list);
+
+    disconnect_guest(device);
+    free_device(device);
+}
+
+static void remove_all_devices(void)
+{
+    device *device, *tmp;
+
+    XEN_TAILQ_FOREACH_SAFE( device, &devs, list, tmp )
+        remove_device(device);
+}
+
+static void frontend_changed(device *device)
+{
+    struct path p;
+    char *state, *end;
+    unsigned int len;
+    unsigned long new_state;
+
+    construct_frontend_path(device, "state", &p);
+    state = xs_read(xs, XBT_NULL, p.path, &len);
+    if ( !state )
+    {
+        close_device(device, XenbusStateClosed);
+        return;
+    }
+
+    new_state = strtoul(state, &end, 10);
+    if ( *end || new_state > XenbusStateReconfigured )
+    {
+        syslog(LOG_WARNING, "unknown state \"%s\" seen for device %u/%u", state,
+               device->domid, device->devid);
+        new_state = XenbusStateUnknown;
+    }
+    free(state);
+
+    if ( new_state == device->frontend_state )
+        return;
+
+    switch ( new_state )
+    {
+    case XenbusStateInitialising:
+        break;
+
+    case XenbusStateInitialised:
+        connect_device(device);
+        break;
+
+    case XenbusStateConnected:
+        break;
+
+    case XenbusStateClosing:
+        close_device(device, XenbusStateClosing);
+        break;
+
+    case XenbusStateClosed:
+        close_device(device, XenbusStateClosed);
+        break;
+
+    default:
+        syslog(LOG_WARNING, "not handled frontend state %lu for device %u/%u",
+               new_state, device->domid, device->devid);
+        break;
+    }
+
+    device->frontend_state = new_state;
+}
+
+static void check_device(unsigned int domid, unsigned int devid)
+{
+    device *device;
+
+    device = find_device(domid, devid);
+    if ( !device )
+    {
+        device = new_device(domid, devid);
+        if ( !device )
+            return;
+    }
+
+    device->last_seen = now;
+}
+
+static void scan_backend(void)
+{
+    char **doms;
+    unsigned int n_doms, dom;
+    char **devices;
+    unsigned int n_devs, dev;
+    char dom_path[24];
+    unsigned long domid, devid;
+    char *end;
+    device *device, *tmp;
+
+    now++;
+
+    doms = xs_directory(xs, XBT_NULL, "backend/xen_9pfs", &n_doms);
+    if ( doms == NULL )
+        return;
+
+    for ( dom = 0; dom < n_doms; dom++ )
+    {
+        errno = 0;
+        domid = strtoul(doms[dom], &end, 10);
+        if ( errno || *end || domid >= DOMID_FIRST_RESERVED )
+            continue;
+
+        snprintf(dom_path, sizeof(dom_path), "backend/xen_9pfs/%lu", domid);
+        devices = xs_directory(xs, XBT_NULL, dom_path, &n_devs);
+
+        for ( dev = 0; dev < n_devs; dev++ )
+        {
+            errno = 0;
+            devid = strtoul(devices[dev], &end, 10);
+            if ( errno || *end || devid > UINT_MAX )
+                continue;
+
+            check_device(domid, devid);
+        }
+
+        free(devices);
+    }
+
+    free(doms);
+
+    XEN_TAILQ_FOREACH_SAFE( device, &devs, list, tmp )
+    {
+        if ( device->last_seen != now )
+            remove_device(device);
+    }
+}
+
+static void handle_watch(char *path, char *token)
+{
+    unsigned int domid, devid;
+    device *device;
+
+    if ( !strcmp(token, "main") )
+    {
+        scan_backend();
+        return;
+    }
+
+    if ( sscanf(token, "%u/%u", &domid, &devid) != 2 )
+    {
+        syslog(LOG_WARNING, "unknown watch event %s %s", path, token);
+        return;
+    }
+
+    device = find_device(domid, devid);
+    if ( !device )
+    {
+        syslog(LOG_WARNING, "watch event for unknown device %u/%u",
+               domid, devid);
+        return;
+    }
+
+    frontend_changed(device);
+}
+
 static void close_all(void)
 {
     if ( daemon_running )
+    {
         xs_rm(xs, XBT_NULL, "libxl/xen-9pfs");
+        xs_unwatch(xs, "backend/xen_9pfs", "main");
+
+        remove_all_devices();
+    }
     if ( xe )
         xenevtchn_close(xe);
     if ( xg )
@@ -68,6 +666,33 @@ static void do_err(const char *msg)
     exit(1);
 }
 
+static void handle_event(void)
+{
+    xenevtchn_port_or_error_t evtchn;
+    device *device;
+    struct ring *ring;
+    unsigned int ring_idx;
+
+    evtchn = xenevtchn_pending(xe);
+    if ( evtchn < 0 )
+        do_err("xenevtchn_pending() failed");
+
+    XEN_TAILQ_FOREACH( device, &devs, list )
+    {
+        for ( ring_idx = 0; ring_idx < device->num_rings; ring_idx++ )
+        {
+            ring = device->ring[ring_idx];
+            if ( ring && ring->evtchn == evtchn )
+            {
+                pthread_mutex_lock(&ring->mutex);
+                pthread_cond_signal(&ring->cond);
+                pthread_mutex_unlock(&ring->mutex);
+                return;
+            }
+        }
+    }
+}
+
 static void xen_connect(void)
 {
     xs_transaction_t t;
@@ -124,6 +749,11 @@ int main(int argc, char *argv[])
     int syslog_mask = LOG_MASK(LOG_WARNING) | LOG_MASK(LOG_ERR) |
                       LOG_MASK(LOG_CRIT) | LOG_MASK(LOG_ALERT) |
                       LOG_MASK(LOG_EMERG);
+    char **watch;
+    struct pollfd p[2] = {
+        { .events = POLLIN },
+        { .events = POLLIN }
+    };
 
     umask(027);
     if ( getenv("XEN_9PFSD_VERBOSE") )
@@ -136,9 +766,26 @@ int main(int argc, char *argv[])
 
     xen_connect();
 
+    if ( !xs_watch(xs, "backend/xen_9pfs", "main") )
+        do_err("xs_watch() in main thread failed");
+    p[0].fd = xs_fileno(xs);
+    p[1].fd = xenevtchn_fd(xe);
+
+    scan_backend();
+
     while ( !stop_me )
     {
-        sleep(60);
+        while ( (p[0].revents & POLLIN) &&
+                (watch = xs_check_watch(xs)) != NULL )
+        {
+            handle_watch(watch[XS_WATCH_PATH], watch[XS_WATCH_TOKEN]);
+            free(watch);
+        }
+
+        if ( p[1].revents & POLLIN )
+            handle_event();
+
+        poll(p, 2, -1);
     }
 
     close_all();
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
new file mode 100644
index 0000000000..ff3562164a
--- /dev/null
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef XEN_9PFSD_H
+#define XEN_9PFSD_H
+
+#include <pthread.h>
+#include <stdbool.h>
+#include <xenevtchn.h>
+#include <xen_list.h>
+#include <xen/xen.h>
+#include <xen/io/xenbus.h>
+#include <xen/io/9pfs.h>
+
+#define MAX_RINGS                4
+#define MAX_RING_ORDER           9
+#define MAX_OPEN_FILES_DEFAULT   5
+
+typedef struct device device;
+
+struct ring {
+    device *device;
+    pthread_t thread;
+    bool thread_active;
+    bool stop_thread;
+    pthread_cond_t cond;
+    pthread_mutex_t mutex;
+
+    evtchn_port_t evtchn;
+    struct xen_9pfs_data_intf *intf;
+    unsigned int ring_order;
+    RING_IDX ring_size;
+    struct xen_9pfs_data data;
+};
+
+struct device {
+    /* Admin data. */
+    XEN_TAILQ_ENTRY(device) list;
+    unsigned int last_seen;    /* Set in scan_backend(). */
+    unsigned int domid;
+    unsigned int devid;
+
+    /* Tool side configuration data. */
+    char *host_path;
+    unsigned int max_space;
+    unsigned int max_files;
+    unsigned int max_open_files;
+    bool auto_delete;
+
+    /* Connection data. */
+    enum xenbus_state backend_state;
+    enum xenbus_state frontend_state;
+    unsigned int num_rings;
+    struct ring *ring[MAX_RINGS];
+    int root_fd;
+};
+
+extern xenevtchn_handle *xe;
+
+void *io_thread(void *arg);
+
+#endif /* XEN_9PFSD_H */
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:05:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:05:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681683.1060567 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQg-0003br-Py; Thu, 15 Feb 2024 13:05:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681683.1060567; Thu, 15 Feb 2024 13:05:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQg-0003bk-Mi; Thu, 15 Feb 2024 13:05:30 +0000
Received: by outflank-mailman (input) for mailman id 681683;
 Thu, 15 Feb 2024 13:05:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabQf-0002yD-HA
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:05:29 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e44470ff-cc02-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:05:28 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 67D451F8A6;
 Thu, 15 Feb 2024 13:05:28 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 386FC139D0;
 Thu, 15 Feb 2024 13:05:28 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id vCNYDBgMzmV6FQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:05:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e44470ff-cc02-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002328; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1Cx2zmy0kYHTdVFW0XAu4VXbYGYjbC4IGhNjDXCFbLc=;
	b=OQSbNEU7Smow+Yd1EQEszr2VrY9Vusv4XxaYHbjA3SzJadL5FLQvcCIAp9k3FceLAWvV1y
	8ARNu+dl5xtlMsuMlSQVspJFWhbnNx+h44zufWFebqnNUVPZdLirUt5Y7aOzxi3t0Ncd2h
	FX0d2znM1/p2wGOY5Yp6Z3JFBKyAgxs=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002328; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=1Cx2zmy0kYHTdVFW0XAu4VXbYGYjbC4IGhNjDXCFbLc=;
	b=OQSbNEU7Smow+Yd1EQEszr2VrY9Vusv4XxaYHbjA3SzJadL5FLQvcCIAp9k3FceLAWvV1y
	8ARNu+dl5xtlMsuMlSQVspJFWhbnNx+h44zufWFebqnNUVPZdLirUt5Y7aOzxi3t0Ncd2h
	FX0d2znM1/p2wGOY5Yp6Z3JFBKyAgxs=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 03/21] tools/xen-9pfsd: add transport layer
Date: Thu, 15 Feb 2024 14:04:51 +0100
Message-Id: <20240215130509.24008-4-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=OQSbNEU7
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [0.49 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 0.49
X-Rspamd-Queue-Id: 67D451F8A6
X-Spam-Flag: NO

Add the transport layer of 9pfs. This is basically the infrastructure
to receive requests from the frontend and to send the related answers
via the rings.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- rename put_request_bytes() (Jason Andryuk)
- rename get_request_bytes() and put_response_bytes() len parameter
  (Jason Andryuk)
- don't unmask event channel if error indicator is set (Jason Andryuk)
---
 tools/9pfsd/io.c        | 143 +++++++++++++++++++++++++++++++++++++++-
 tools/9pfsd/xen-9pfsd.h |  16 +++++
 2 files changed, 156 insertions(+), 3 deletions(-)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index eb7c136e09..4312a62dfe 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -6,39 +6,176 @@
  * Copyright (C) 2024 Juergen Gross <jgross@suse.com>
  *
  * I/O thread handling.
+ *
+ * Only handle one request at a time, pushing out the complete response
+ * before looking for the next request.
  */
 
 #include <stdbool.h>
+#include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <xenctrl.h>           /* For cpu barriers. */
+#include <xen-tools/common-macros.h>
 
 #include "xen-9pfsd.h"
 
+/*
+ * Note that the ring names "in" and "out" are from the frontend's
+ * perspective, so the "in" ring will be used for responses to the frontend,
+ * while the "out" ring is used for requests from the frontend to the
+ * backend.
+ */
+static unsigned int ring_in_free(struct ring *ring)
+{
+    unsigned int queued;
+
+    queued = xen_9pfs_queued(ring->prod_pvt_in, ring->intf->in_cons,
+                             ring->ring_size);
+    xen_rmb();
+
+    return ring->ring_size - queued;
+}
+
+static unsigned int ring_out_data(struct ring *ring)
+{
+    unsigned int queued;
+
+    queued = xen_9pfs_queued(ring->intf->out_prod, ring->cons_pvt_out,
+                             ring->ring_size);
+    xen_rmb();
+
+    return queued;
+}
+
+static unsigned int get_request_bytes(struct ring *ring, unsigned int off,
+                                      unsigned int total_len)
+{
+    unsigned int size;
+    unsigned int out_data = ring_out_data(ring);
+    RING_IDX prod, cons;
+
+    size = min(total_len - off, out_data);
+    prod = xen_9pfs_mask(ring->intf->out_prod, ring->ring_size);
+    cons = xen_9pfs_mask(ring->cons_pvt_out, ring->ring_size);
+    xen_9pfs_read_packet(ring->buffer + off, ring->data.out, size,
+                         prod, &cons, ring->ring_size);
+
+    xen_rmb();           /* Read data out before setting visible consumer. */
+    ring->cons_pvt_out += size;
+    ring->intf->out_cons = ring->cons_pvt_out;
+
+    /* Signal that more space is available now. */
+    xenevtchn_notify(xe, ring->evtchn);
+
+    return size;
+}
+
+static unsigned int put_response_bytes(struct ring *ring, unsigned int off,
+                                       unsigned int total_len)
+{
+    unsigned int size;
+    unsigned int in_data = ring_in_free(ring);
+    RING_IDX prod, cons;
+
+    size = min(total_len - off, in_data);
+    prod = xen_9pfs_mask(ring->prod_pvt_in, ring->ring_size);
+    cons = xen_9pfs_mask(ring->intf->in_cons, ring->ring_size);
+    xen_9pfs_write_packet(ring->data.in, ring->buffer + off, size,
+                          &prod, cons, ring->ring_size);
+
+    xen_wmb();           /* Write data out before setting visible producer. */
+    ring->prod_pvt_in += size;
+    ring->intf->in_prod = ring->prod_pvt_in;
+
+    return size;
+}
+
 static bool io_work_pending(struct ring *ring)
 {
     if ( ring->stop_thread )
         return true;
-    return false;
+    if ( ring->error )
+        return false;
+    return ring->handle_response ? ring_in_free(ring) : ring_out_data(ring);
 }
 
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
+    unsigned int count = 0;
+    struct p9_header hdr;
+    bool in_hdr = true;
+
+    ring->max_size = ring->ring_size;
+    ring->buffer = malloc(ring->max_size);
+    if ( !ring->buffer )
+    {
+        syslog(LOG_CRIT, "memory allocation failure!");
+        return NULL;
+    }
 
     while ( !ring->stop_thread )
     {
         pthread_mutex_lock(&ring->mutex);
         if ( !io_work_pending(ring) )
         {
-            if ( xenevtchn_unmask(xe, ring->evtchn) < 0 )
+            if ( !ring->error && xenevtchn_unmask(xe, ring->evtchn) < 0 )
                 syslog(LOG_WARNING, "xenevtchn_unmask() failed");
             pthread_cond_wait(&ring->cond, &ring->mutex);
         }
         pthread_mutex_unlock(&ring->mutex);
 
-        /* TODO: I/O handling. */
+        if ( ring->stop_thread || ring->error )
+            continue;
+
+        if ( !ring->handle_response )
+        {
+            if ( in_hdr )
+            {
+                count += get_request_bytes(ring, count, sizeof(hdr));
+                if ( count != sizeof(hdr) )
+                    continue;
+                hdr = *(struct p9_header *)ring->buffer;
+                if ( hdr.size > ring->max_size || hdr.size < sizeof(hdr) )
+                {
+                    syslog(LOG_ERR, "%u.%u specified illegal request length %u",
+                           ring->device->domid, ring->device->devid, hdr.size);
+                    ring->error = true;
+                    continue;
+                }
+                in_hdr = false;
+            }
+
+            count += get_request_bytes(ring, count, hdr.size);
+            if ( count < hdr.size )
+                continue;
+
+            /* TODO: handle request (will rewrite hdr.size). */
+
+            ring->handle_response = true;
+            hdr.size = ((struct p9_header *)ring->buffer)->size;
+            count = 0;
+        }
+
+        if ( ring->handle_response )
+        {
+            count += put_response_bytes(ring, count, hdr.size);
+
+            if ( count == hdr.size )
+            {
+                /* Signal presence of response. */
+                xenevtchn_notify(xe, ring->evtchn);
+
+                ring->handle_response = false;
+                in_hdr = true;
+                count = 0;
+            }
+        }
     }
 
+    free(ring->buffer);
+
     ring->thread_active = false;
 
     return NULL;
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index ff3562164a..d587f59a32 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -15,6 +15,12 @@
 #define MAX_RING_ORDER           9
 #define MAX_OPEN_FILES_DEFAULT   5
 
+struct p9_header {
+    uint32_t size;
+    uint8_t cmd;
+    uint16_t tag;
+} __attribute__((packed));
+
 typedef struct device device;
 
 struct ring {
@@ -29,7 +35,17 @@ struct ring {
     struct xen_9pfs_data_intf *intf;
     unsigned int ring_order;
     RING_IDX ring_size;
+
+    /* Transport layer data. */
     struct xen_9pfs_data data;
+    RING_IDX prod_pvt_in;
+    RING_IDX cons_pvt_out;
+
+    /* Request and response handling. */
+    uint32_t max_size;
+    bool error;             /* Protocol error - stop processing. */
+    bool handle_response;   /* Main loop now handling response. */
+    void *buffer;           /* Request/response buffer. */
 };
 
 struct device {
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:05:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:05:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681686.1060577 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQn-0003zP-7G; Thu, 15 Feb 2024 13:05:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681686.1060577; Thu, 15 Feb 2024 13:05:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQn-0003zF-4P; Thu, 15 Feb 2024 13:05:37 +0000
Received: by outflank-mailman (input) for mailman id 681686;
 Thu, 15 Feb 2024 13:05:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabQm-0002j3-A4
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:05:36 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e793b404-cc02-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:05:34 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 1281521FBC;
 Thu, 15 Feb 2024 13:05:34 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id D09C4139D0;
 Thu, 15 Feb 2024 13:05:33 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id ONtNMR0MzmV9FQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:05:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e793b404-cc02-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002334; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=O73f+E3GNfH4W5rlXCuJYVhh6QCuB0n/PfhAGghm1+4=;
	b=Q063CRzUshxKsyPLO69hrUAcvSURwusl0CX0OnRdG1Ei5qcVv6sx+wVfn99iTumv4bQVTq
	vzVquLerBUSriWpgiUkp21Gx50VRzujHIAHNgUTt+Mp+aFnxZ6rXBhANK2DC0yFWXzN3K5
	+Pqc+fUSwZywfZKYEFhAYvpDS96gtQs=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002334; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=O73f+E3GNfH4W5rlXCuJYVhh6QCuB0n/PfhAGghm1+4=;
	b=Q063CRzUshxKsyPLO69hrUAcvSURwusl0CX0OnRdG1Ei5qcVv6sx+wVfn99iTumv4bQVTq
	vzVquLerBUSriWpgiUkp21Gx50VRzujHIAHNgUTt+Mp+aFnxZ6rXBhANK2DC0yFWXzN3K5
	+Pqc+fUSwZywfZKYEFhAYvpDS96gtQs=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 04/21] tools/xen-9pfsd: add 9pfs response generation support
Date: Thu, 15 Feb 2024 14:04:52 +0100
Message-Id: <20240215130509.24008-5-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [1.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: *
X-Spam-Score: 1.90
X-Spam-Flag: NO

Add support for generation a 9pfs protocol response via a format based
approach.

Strings are stored in a per device string buffer and they are
referenced via their offset in this buffer. This allows to avoid
having to dynamically allocate memory for each single string.

As a first user of the response handling add a generic p9_error()
function which will be used to return any error to the client.

Add all format parsing variants in order to avoid additional code churn
later when adding the users of those variants. Prepare a special case
for the "read" case already (format character 'D'): in order to avoid
adding another buffer for read data support doing the read I/O directly
into the response buffer.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- check parameter size limits (Jason Andryuk)
V3:
- use new unaligned access macros (Jason Andryuk)
V4:
- use recursion in fill_buffer() as a preparation for reading dirs
---
 tools/9pfsd/io.c        | 217 +++++++++++++++++++++++++++++++++++++++-
 tools/9pfsd/xen-9pfsd.h |   3 +
 2 files changed, 219 insertions(+), 1 deletion(-)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 4312a62dfe..4a44c70c4d 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -11,6 +11,7 @@
  * before looking for the next request.
  */
 
+#include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
@@ -20,6 +21,16 @@
 
 #include "xen-9pfsd.h"
 
+/* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
+#define P9_CMD_ERROR      107
+
+struct p9_qid {
+    uint8_t type;
+#define QID_TYPE_DIR      0x80
+    uint32_t version;
+    uint64_t path;
+};
+
 /*
  * Note that the ring names "in" and "out" are from the frontend's
  * perspective, so the "in" ring will be used for responses to the frontend,
@@ -100,6 +111,200 @@ static bool io_work_pending(struct ring *ring)
     return ring->handle_response ? ring_in_free(ring) : ring_out_data(ring);
 }
 
+static void fmt_err(const char *fmt)
+{
+    syslog(LOG_CRIT, "illegal format %s passed to fill_buffer()", fmt);
+    exit(1);
+}
+
+/*
+ * Fill buffer with response data.
+ * fmt is a sequence of format characters. Supported characters are:
+ * a: an array (2 bytes number of elements + the following format as elements)
+ *    The number of elements is passed in the first unsigned int parameter, the
+ *    next parameter is a pointer to an array of elements as denoted by the next
+ *    format character.
+ * b: 1 byte unsigned integer
+ * u: 2 byte unsigned integer
+ *    The parameter is a pointer to a uint16_t value
+ * D: Data blob (4 byte length + <length> bytes)
+ *    2 parameters are consumed, first an unsigned int for the length, then a
+ *    pointer to the first uint8_t value.
+ *    No array support.
+ * L: 8 byte unsigned integer
+ *    The parameter is a pointer to a uint64_t value
+ * Q: Qid (struct p9_qid)
+ * S: String (2 byte length + <length> characters)
+ *    The length is obtained via strlen() of the parameter, being a pointer
+ *    to the first character of the string
+ * U: 4 byte unsigned integer
+ *    The parameter is a pointer to a uint32_t value
+ */
+static void fill_buffer_at(void **data, const char *fmt, ...);
+static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
+{
+    const char *f;
+    const void *par;
+    const char *str_val;
+    const struct p9_qid *qid;
+    unsigned int len;
+    unsigned int array_sz = 0;
+    unsigned int elem_sz = 0;
+
+    for ( f = fmt; *f; f++ )
+    {
+        if ( !array_sz )
+            par = va_arg(ap, const void *);
+        else
+        {
+            par += elem_sz;
+            array_sz--;
+        }
+
+        switch ( *f )
+        {
+        case 'a':
+            f++;
+            if ( !*f || array_sz )
+                fmt_err(fmt);
+            array_sz = *(const unsigned int *)par;
+            if ( array_sz > 0xffff )
+            {
+                syslog(LOG_CRIT, "array size %u in fill_buffer()", array_sz);
+                exit(1);
+            }
+            put_unaligned(array_sz, (uint16_t *)*data);
+            *data += sizeof(uint16_t);
+            par = va_arg(ap, const void *);
+            elem_sz = 0;
+            break;
+
+        case 'b':
+            put_unaligned(*(const uint8_t *)par, (uint8_t *)*data);
+            elem_sz = sizeof(uint8_t);
+            *data += sizeof(uint8_t);
+            break;
+
+        case 'u':
+            put_unaligned(*(const uint16_t *)par, (uint16_t *)*data);
+            elem_sz = sizeof(uint16_t);
+            *data += sizeof(uint16_t);
+            break;
+
+        case 'D':
+            if ( array_sz )
+                fmt_err(fmt);
+            len = *(const unsigned int *)par;
+            put_unaligned(len, (uint32_t *)*data);
+            *data += sizeof(uint32_t);
+            par = va_arg(ap, const void *);
+            if ( *data != par )
+                memcpy(*data, par, len);
+            *data += len;
+            break;
+
+        case 'L':
+            put_unaligned(*(const uint64_t *)par, (uint64_t *)*data);
+            elem_sz = sizeof(uint64_t);
+            *data += sizeof(uint64_t);
+            break;
+
+        case 'Q':
+            qid = par;
+            elem_sz = sizeof(*qid);
+            fill_buffer_at(data, "bUL", &qid->type, &qid->version, &qid->path);
+            break;
+
+        case 'S':
+            str_val = par;
+            elem_sz = sizeof(str_val);
+            len = strlen(str_val);
+            if ( len > 0xffff )
+            {
+                syslog(LOG_CRIT, "string length %u in fill_buffer()", len);
+                exit(1);
+            }
+            put_unaligned(len, (uint16_t *)*data);
+            *data += sizeof(uint16_t);
+            memcpy(*data, str_val, len);
+            *data += len;
+            break;
+
+        case 'U':
+            put_unaligned(*(const uint32_t *)par, (uint32_t *)*data);
+            elem_sz = sizeof(uint32_t);
+            *data += sizeof(uint32_t);
+            break;
+
+        default:
+            fmt_err(fmt);
+        }
+
+        if ( array_sz )
+            f--;
+    }
+}
+
+static void fill_buffer_at(void **data, const char *fmt, ...)
+{
+    va_list ap;
+
+    va_start(ap, fmt);
+    vfill_buffer_at(data, fmt, ap);
+    va_end(ap);
+}
+
+static void fill_buffer(struct ring *ring, uint8_t cmd, uint16_t tag,
+                        const char *fmt, ...)
+{
+    struct p9_header *hdr = ring->buffer;
+    void *data = hdr + 1;
+    va_list ap;
+
+    hdr->cmd = cmd;
+    hdr->tag = tag;
+
+    va_start(ap, fmt);
+    vfill_buffer_at(&data, fmt, ap);
+    va_end(ap);
+
+    hdr->size = data - ring->buffer;
+}
+
+static unsigned int add_string(struct ring *ring, const char *str,
+                               unsigned int len)
+{
+    char *tmp;
+    unsigned int ret;
+
+    if ( ring->str_used + len + 1 > ring->str_size )
+    {
+        tmp = realloc(ring->str, ring->str_used + len + 1);
+        if ( !tmp )
+            return ~0;
+        ring->str = tmp;
+        ring->str_size = ring->str_used + len + 1;
+    }
+
+    ret = ring->str_used;
+    memcpy(ring->str + ret, str, len);
+    ring->str_used += len;
+    ring->str[ring->str_used++] = 0;
+
+    return ret;
+}
+
+static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
+{
+    unsigned int erroff;
+
+    strerror_r(err, ring->buffer, ring->ring_size);
+    erroff = add_string(ring, ring->buffer, strlen(ring->buffer));
+    fill_buffer(ring, P9_CMD_ERROR, tag, "SU",
+                erroff != ~0 ? ring->str + erroff : "cannot allocate memory",
+                &err);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -151,7 +356,16 @@ void *io_thread(void *arg)
             if ( count < hdr.size )
                 continue;
 
-            /* TODO: handle request (will rewrite hdr.size). */
+            ring->str_used = 0;
+
+            switch ( hdr.cmd )
+            {
+            default:
+                syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
+                       ring->device->domid, ring->device->devid, hdr.cmd);
+                p9_error(ring, hdr.tag, EOPNOTSUPP);
+                break;
+            }
 
             ring->handle_response = true;
             hdr.size = ((struct p9_header *)ring->buffer)->size;
@@ -174,6 +388,7 @@ void *io_thread(void *arg)
         }
     }
 
+    free(ring->str);
     free(ring->buffer);
 
     ring->thread_active = false;
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index d587f59a32..0cde0d2bb8 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -46,6 +46,9 @@ struct ring {
     bool error;             /* Protocol error - stop processing. */
     bool handle_response;   /* Main loop now handling response. */
     void *buffer;           /* Request/response buffer. */
+    char *str;              /* String work space. */
+    unsigned int str_size;  /* Size of *str. */
+    unsigned int str_used;  /* Currently used size of *str. */
 };
 
 struct device {
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:05:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:05:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681688.1060587 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQs-0004OO-HB; Thu, 15 Feb 2024 13:05:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681688.1060587; Thu, 15 Feb 2024 13:05:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQs-0004OC-DS; Thu, 15 Feb 2024 13:05:42 +0000
Received: by outflank-mailman (input) for mailman id 681688;
 Thu, 15 Feb 2024 13:05:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabQq-0002yD-QN
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:05:40 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id eafd7db8-cc02-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:05:40 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id B6B9222031;
 Thu, 15 Feb 2024 13:05:39 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 7FAD4139D0;
 Thu, 15 Feb 2024 13:05:39 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id reBzHSMMzmWNFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:05:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eafd7db8-cc02-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002339; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Ep5GCYtOAcPi9Z1A4O1nBYAfiV8kAZfwfen9awb2d94=;
	b=SGwNfKu2eT+wcBk3tiUVz3WPOmLp2PjVIuZ2y/oT/YC7y0811+wVrgYlkj6upmuz8i3JGA
	dytxKOyt3gi9MjI3jhRICF7d8tN2/Gv3ZV4EmlBFyz73ypRUWGAo3lLHSy2+IPpsU0BZu9
	/8gOIf2E4rWtVJ4xeSY/cxtXsU3m430=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002339; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Ep5GCYtOAcPi9Z1A4O1nBYAfiV8kAZfwfen9awb2d94=;
	b=SGwNfKu2eT+wcBk3tiUVz3WPOmLp2PjVIuZ2y/oT/YC7y0811+wVrgYlkj6upmuz8i3JGA
	dytxKOyt3gi9MjI3jhRICF7d8tN2/Gv3ZV4EmlBFyz73ypRUWGAo3lLHSy2+IPpsU0BZu9
	/8gOIf2E4rWtVJ4xeSY/cxtXsU3m430=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 05/21] tools/xen-9pfsd: add 9pfs version request support
Date: Thu, 15 Feb 2024 14:04:53 +0100
Message-Id: <20240215130509.24008-6-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: 0.70
X-Spamd-Result: default: False [0.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add the version request of the 9pfs protocol. For the version use the
"9P2000.u" variant, as it is supported by Mini-OS and Linux.

For the request parsing add all format items needed even in future in
order to avoid code churn for those additions later.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V3:
- use unaligned helper macros (Jason Andryuk)
---
 tools/9pfsd/io.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 201 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 4a44c70c4d..839dd1112c 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -22,8 +22,12 @@
 #include "xen-9pfsd.h"
 
 /* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
+#define P9_CMD_VERSION    100
 #define P9_CMD_ERROR      107
 
+#define P9_MIN_MSIZE      2048
+#define P9_VERSION        "9P2000.u"
+
 struct p9_qid {
     uint8_t type;
 #define QID_TYPE_DIR      0x80
@@ -294,6 +298,169 @@ static unsigned int add_string(struct ring *ring, const char *str,
     return ret;
 }
 
+static bool chk_data(struct ring *ring, void *data, unsigned int len)
+{
+    struct p9_header *hdr = ring->buffer;
+
+    if ( data + len <= ring->buffer + hdr->size )
+        return true;
+
+    errno = E2BIG;
+
+    return false;
+}
+
+static bool fill_data_elem(void **par, void **array, unsigned int *array_sz,
+                           unsigned int elem_sz, void *data)
+{
+    if ( *array_sz && !*array )
+    {
+        *array = calloc(*array_sz, elem_sz);
+        if ( !*array )
+            return false;
+        *par = *array;
+    }
+
+    memcpy(*par, data, elem_sz);
+
+    if ( *array_sz )
+    {
+        *par += elem_sz;
+        *array_sz -= 1;
+    }
+
+    return true;
+}
+
+/*
+ * Fill variables with request data.
+ * fmt is a sequence of format characters. Supported characters are:
+ * a: an array (2 bytes number of elements + the following format as elements)
+ *    The number of elements is stored in the first unsigned int parameter, the
+ *    next parameter is a pointer to an array of elements as denoted by the next
+ *    format character. The array is allocated dynamically.
+ * b: 1 byte unsigned integer
+ *    The value is stored in the next parameter with type uint8_t.
+ * D: Data blob (4 byte length + <length> bytes)
+ *    2 parameters are consumed, first an unsigned int for the length, then a
+ *    pointer to the first uint8_t value.
+ *    No array support.
+ * L: 8 byte unsigned integer
+ *    The value is stored in the next parameter with type uint64_t.
+ * S: String (2 byte length + <length> characters)
+ *    The 0-terminated string is stored in device->str + off, off is stored in
+ *    the next parameter with type unsigned int.
+ * U: 4 byte unsigned integer
+ *    The value is stored in the next parameter with type uint32_t.
+ *
+ * Return value: number of filled variables, errno will be set in case of
+ *   error.
+ */
+static int fill_data(struct ring *ring, const char *fmt, ...)
+{
+    struct p9_header *hdr = ring->buffer;
+    void *data = hdr + 1;
+    void *par;
+    unsigned int pars = 0;
+    const char *f;
+    va_list ap;
+    unsigned int len;
+    unsigned int str_off;
+    unsigned int array_sz = 0;
+    void **array = NULL;
+
+    va_start(ap, fmt);
+
+    for ( f = fmt; *f; f++ )
+    {
+        if ( !array_sz )
+            par = va_arg(ap, void *);
+
+        switch ( *f )
+        {
+        case 'a':
+            f++;
+            if ( !*f || array_sz )
+                fmt_err(fmt);
+            if ( !chk_data(ring, data, sizeof(uint16_t)) )
+                return pars;
+            array_sz = get_unaligned((uint16_t *)data);
+            data += sizeof(uint16_t);
+            *(unsigned int *)par = array_sz;
+            array = va_arg(ap, void **);
+            *array = NULL;
+            break;
+
+        case 'b':
+            if ( !chk_data(ring, data, sizeof(uint8_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint8_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint8_t);
+            break;
+
+        case 'D':
+            if ( array_sz )
+                fmt_err(fmt);
+            if ( !chk_data(ring, data, sizeof(uint32_t)) )
+                return pars;
+            len = get_unaligned((uint32_t *)data);
+            data += sizeof(uint32_t);
+            *(unsigned int *)par = len;
+            par = va_arg(ap, void *);
+            if ( !chk_data(ring, data, len) )
+                return pars;
+            memcpy(par, data, len);
+            data += len;
+            break;
+
+        case 'L':
+            if ( !chk_data(ring, data, sizeof(uint64_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint64_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint64_t);
+            break;
+
+        case 'S':
+            if ( !chk_data(ring, data, sizeof(uint16_t)) )
+                return pars;
+            len = get_unaligned((uint16_t *)data);
+            data += sizeof(uint16_t);
+            if ( !chk_data(ring, data, len) )
+                return pars;
+            str_off = add_string(ring, data, len);
+            if ( str_off == ~0 )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(unsigned int),
+                                 &str_off) )
+                return pars;
+            data += len;
+            break;
+
+        case 'U':
+            if ( !chk_data(ring, data, sizeof(uint32_t)) )
+                return pars;
+            if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint32_t),
+                                 data) )
+                return pars;
+            data += sizeof(uint32_t);
+            break;
+
+        default:
+            fmt_err(fmt);
+        }
+
+        if ( array_sz )
+            f--;
+        pars++;
+    }
+
+    return pars;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -305,6 +472,36 @@ static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
                 &err);
 }
 
+static void p9_version(struct ring *ring, struct p9_header *hdr)
+{
+    uint32_t max_size;
+    unsigned int off;
+    char *version;
+    int ret;
+
+    ret = fill_data(ring, "US", &max_size, &off);
+    if ( ret != 2 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    if ( max_size < P9_MIN_MSIZE )
+    {
+        p9_error(ring, hdr->tag, EMSGSIZE);
+        return;
+    }
+
+    if ( max_size < ring->max_size )
+        ring->max_size = max_size;
+
+    version = ring->str + off;
+    if ( strcmp(version, P9_VERSION) )
+        version = "unknown";
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "US", &ring->max_size, version);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -360,6 +557,10 @@ void *io_thread(void *arg)
 
             switch ( hdr.cmd )
             {
+            case P9_CMD_VERSION:
+                p9_version(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:05:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:05:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681689.1060597 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQx-0004o8-OB; Thu, 15 Feb 2024 13:05:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681689.1060597; Thu, 15 Feb 2024 13:05:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabQx-0004nz-Kj; Thu, 15 Feb 2024 13:05:47 +0000
Received: by outflank-mailman (input) for mailman id 681689;
 Thu, 15 Feb 2024 13:05:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabQw-0002yD-Di
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:05:46 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ee5ab304-cc02-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:05:45 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 58A2022031;
 Thu, 15 Feb 2024 13:05:45 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 277AC139D0;
 Thu, 15 Feb 2024 13:05:45 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 0kcxCCkMzmWSFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:05:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ee5ab304-cc02-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002345; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=L9NcHDZsirisD0yciG+rBpMttYLBTYJ34tUyYEKGqh8=;
	b=Fa3l8KpEg+ehmFbS58CxlBh61ofDZoNquqBKccQwvm/ccCfzmOK5QDkMHUYW3+RT58RvmB
	FVejcuUkGK6+yvW12CshYcpvrZRJ1V7IgC9drPnC396TW5sYAKN1wH94WGvx7nnxwyvCBc
	Y6mCtYZR+rPH2YOyCIimXV1E6ZXlSt4=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002345; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=L9NcHDZsirisD0yciG+rBpMttYLBTYJ34tUyYEKGqh8=;
	b=Fa3l8KpEg+ehmFbS58CxlBh61ofDZoNquqBKccQwvm/ccCfzmOK5QDkMHUYW3+RT58RvmB
	FVejcuUkGK6+yvW12CshYcpvrZRJ1V7IgC9drPnC396TW5sYAKN1wH94WGvx7nnxwyvCBc
	Y6mCtYZR+rPH2YOyCIimXV1E6ZXlSt4=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 06/21] tools/xen-9pfsd: add 9pfs attach request support
Date: Thu, 15 Feb 2024 14:04:54 +0100
Message-Id: <20240215130509.24008-7-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [1.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: *
X-Spam-Score: 1.90
X-Spam-Flag: NO

Add the attach request of the 9pfs protocol. This introduces the "fid"
scheme of the 9pfs protocol.

As this will be needed later, use a dedicated memory allocation
function in alloc_fid() and prepare a fid reference count.

For filling the qid data take the approach from the qemu 9pfs backend
implementation.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- make fill_qid() parameter stbuf const (Jason Andryuk)
- free fids after disconnecting guest (Jason Andryuk)
V3:
- only store relative path in fid (Jason Andryuk)
V4:
- store a path directly usable by *at() functions in fid (Jason Andryuk)
V7:
- use EBADF instead of EBADFD (Andrew Cooper)
- use strcpy() instead of strncpy() (Andrew Cooper)
---
 tools/9pfsd/io.c        | 162 ++++++++++++++++++++++++++++++++++++++++
 tools/9pfsd/xen-9pfsd.c |   6 ++
 tools/9pfsd/xen-9pfsd.h |  14 ++++
 3 files changed, 182 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 839dd1112c..4c422b06ac 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -11,11 +11,14 @@
  * before looking for the next request.
  */
 
+#include <assert.h>
 #include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <sys/types.h>
+#include <sys/stat.h>
 #include <xenctrl.h>           /* For cpu barriers. */
 #include <xen-tools/common-macros.h>
 
@@ -23,6 +26,7 @@
 
 /* P9 protocol commands (response is either cmd+1 or P9_CMD_ERROR). */
 #define P9_CMD_VERSION    100
+#define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
 
 #define P9_MIN_MSIZE      2048
@@ -461,6 +465,124 @@ static int fill_data(struct ring *ring, const char *fmt, ...)
     return pars;
 }
 
+static struct p9_fid *find_fid(device *device, unsigned int fid)
+{
+    struct p9_fid *fidp;
+
+    XEN_TAILQ_FOREACH(fidp, &device->fids, list)
+    {
+        if ( fidp->fid == fid )
+            return fidp;
+    }
+
+    return NULL;
+}
+
+static struct p9_fid *alloc_fid_mem(device *device, unsigned int fid,
+                                    const char *path)
+{
+    struct p9_fid *fidp;
+
+    fidp = calloc(sizeof(*fidp) + strlen(path) + 1, 1);
+    if ( !fidp )
+        return NULL;
+
+    fidp->fid = fid;
+    strcpy(fidp->path, path);
+
+    return fidp;
+}
+
+static struct p9_fid *alloc_fid(device *device, unsigned int fid,
+                                const char *path)
+{
+    struct p9_fid *fidp = NULL;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    if ( find_fid(device, fid) )
+    {
+        errno = EBADF;
+        goto out;
+    }
+
+    if ( device->n_fids >= device->max_open_files )
+    {
+        errno = EMFILE;
+        goto out;
+    }
+
+    fidp = alloc_fid_mem(device, fid, path);
+    if ( !fidp )
+        goto out;
+
+    fidp->ref = 1;
+    XEN_TAILQ_INSERT_HEAD(&device->fids, fidp, list);
+    device->n_fids++;
+
+ out:
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    return fidp;
+}
+
+static void free_fid(device *device, struct p9_fid *fidp)
+{
+    if ( !fidp )
+        return;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    fidp->ref--;
+    if ( !fidp->ref )
+    {
+        device->n_fids--;
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        free(fidp);
+    }
+
+    pthread_mutex_unlock(&device->fid_mutex);
+}
+
+void free_fids(device *device)
+{
+    struct p9_fid *fidp;
+
+    while ( (fidp = XEN_TAILQ_FIRST(&device->fids)) != NULL )
+    {
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        free(fidp);
+    }
+}
+
+static const char *relpath_from_path(const char *path)
+{
+    if (!strcmp(path, "/"))
+        return ".";
+
+    return (path[0] == '/') ? path + 1 : path;
+}
+
+static int fill_qid(device *device, const char *path, struct p9_qid *qid,
+                    const struct stat *stbuf)
+{
+    struct stat st;
+
+    if ( !stbuf )
+    {
+        if ( fstatat(device->root_fd, path, &st, 0) )
+            return errno;
+
+        stbuf = &st;
+    }
+
+    qid->type = S_ISDIR(stbuf->st_mode) ? QID_TYPE_DIR : 0;
+    qid->version = stbuf->st_mtime ^ (stbuf->st_size << 8);
+    qid->path = stbuf->st_ino;
+
+    return 0;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -502,6 +624,42 @@ static void p9_version(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "US", &ring->max_size, version);
 }
 
+static void p9_attach(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint32_t dummy_u32;
+    unsigned int dummy_uint;
+    struct p9_qid qid;
+    int ret;
+
+    ret = fill_data(ring, "UUSSU", &fid, &dummy_u32, &dummy_uint, &dummy_uint,
+                    &dummy_u32);
+    if ( ret != 5 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    device->root_fid = alloc_fid(device, fid, relpath_from_path("/"));
+    if ( !device->root_fid )
+    {
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    ret = fill_qid(device, device->root_fid->path, &qid, NULL);
+    if ( ret )
+    {
+        free_fid(device, device->root_fid);
+        device->root_fid = NULL;
+        p9_error(ring, hdr->tag, ret);
+        return;
+    }
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "Q", &qid);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -561,6 +719,10 @@ void *io_thread(void *arg)
                 p9_version(ring, &hdr);
                 break;
 
+            case P9_CMD_ATTACH:
+                p9_attach(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/9pfsd/xen-9pfsd.c b/tools/9pfsd/xen-9pfsd.c
index 73b6c3a30e..b83b896bcc 100644
--- a/tools/9pfsd/xen-9pfsd.c
+++ b/tools/9pfsd/xen-9pfsd.c
@@ -304,6 +304,9 @@ static device *new_device(unsigned int domid, unsigned int devid)
         return NULL;
     }
 
+    pthread_mutex_init(&device->fid_mutex, NULL);
+    XEN_TAILQ_INIT(&device->fids);
+
     val = read_backend_node(device, "security_model");
     if ( !val || strcmp(val, "none") )
     {
@@ -399,6 +402,8 @@ static void disconnect_guest(device *device)
     }
 
     device->num_rings = 0;
+
+    free_fids(device);
 }
 
 static void close_device(device *device, enum xenbus_state state)
@@ -482,6 +487,7 @@ static void remove_device(device *device)
     XEN_TAILQ_REMOVE(&devs, device, list);
 
     disconnect_guest(device);
+    pthread_mutex_destroy(&device->fid_mutex);
     free_device(device);
 }
 
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index 0cde0d2bb8..6c755411a7 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -21,6 +21,13 @@ struct p9_header {
     uint16_t tag;
 } __attribute__((packed));
 
+struct p9_fid {
+    XEN_TAILQ_ENTRY(struct p9_fid) list;
+    unsigned int fid;
+    unsigned int ref;
+    char path[];
+};
+
 typedef struct device device;
 
 struct ring {
@@ -71,10 +78,17 @@ struct device {
     unsigned int num_rings;
     struct ring *ring[MAX_RINGS];
     int root_fd;
+
+    /* File system handling. */
+    pthread_mutex_t fid_mutex;
+    XEN_TAILQ_HEAD(fidhead, struct p9_fid) fids;
+    struct p9_fid *root_fid;
+    unsigned int n_fids;
 };
 
 extern xenevtchn_handle *xe;
 
 void *io_thread(void *arg);
+void free_fids(device *device);
 
 #endif /* XEN_9PFSD_H */
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:05:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:05:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681694.1060607 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabR3-0005Lb-4C; Thu, 15 Feb 2024 13:05:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681694.1060607; Thu, 15 Feb 2024 13:05:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabR3-0005LM-0h; Thu, 15 Feb 2024 13:05:53 +0000
Received: by outflank-mailman (input) for mailman id 681694;
 Thu, 15 Feb 2024 13:05:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabR2-0002yD-0A
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:05:52 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f1a94555-cc02-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:05:51 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id E383121FBC;
 Thu, 15 Feb 2024 13:05:50 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id B4694139D0;
 Thu, 15 Feb 2024 13:05:50 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id GK6mKi4MzmWYFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:05:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1a94555-cc02-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002351; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=MIRj03S7dyhfHn0FgSCrUIpTTvYsmpq2WPHIZTrX2Ro=;
	b=JhZ2inOVsD0KKdqDQliC3jPTwlHBgjDRzQalSNDG3gqKW5Q6R26jUUZVFLQ5pqbXwdiHfK
	BTvqrgD6i1KHjm86lpERE15Q6Fqv9E9BiAOCkqbC9SxDp6ImSFyhyLkpCMOqqOKIdk9wP4
	Dv2pnilOHwFlDa5R6g2MOaW+keCldiM=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002350; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=MIRj03S7dyhfHn0FgSCrUIpTTvYsmpq2WPHIZTrX2Ro=;
	b=WnfwcAXDczwd+79EjnXDrxmXaMvxJ/pktlxFfZXvjcJZGcWg8A7K8r3I2pB2NSJIquVaUw
	rk2b/fIjqiGu860G15Kr8djXS4xmqUVpfsEKQL/1bXfTpiCPE0LV6qXbFg/s8a+Pt7GKLI
	SjTeh+Z1d6r/W/DkWyrTMvyjSkMTJ04=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 07/21] tools/xen-9pfsd: add 9pfs walk request support
Date: Thu, 15 Feb 2024 14:04:55 +0100
Message-Id: <20240215130509.24008-8-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [1.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: *
X-Spam-Score: 1.90
X-Spam-Flag: NO

Add the walk request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- don't allow walking across symbolic links
V6:
- use EBADF instead of EBADFD (Andrew Cooper)
- use strncpy() with strlen() + 1 (Andrew Cooper)
V7:
- undo V6 changes as they were in wrong patch
---
 tools/9pfsd/io.c        | 171 ++++++++++++++++++++++++++++++++++++++++
 tools/9pfsd/xen-9pfsd.h |   1 +
 2 files changed, 172 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 4c422b06ac..731fbd1ad7 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -28,9 +28,11 @@
 #define P9_CMD_VERSION    100
 #define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
+#define P9_CMD_WALK       110
 
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
+#define P9_WALK_MAXELEM   16
 
 struct p9_qid {
     uint8_t type;
@@ -478,6 +480,21 @@ static struct p9_fid *find_fid(device *device, unsigned int fid)
     return NULL;
 }
 
+static struct p9_fid *get_fid_ref(device *device, unsigned int fid)
+{
+    struct p9_fid *fidp;
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    fidp = find_fid(device, fid);
+    if ( fidp )
+        fidp->ref++;
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    return fidp;
+}
+
 static struct p9_fid *alloc_fid_mem(device *device, unsigned int fid,
                                     const char *path)
 {
@@ -576,6 +593,10 @@ static int fill_qid(device *device, const char *path, struct p9_qid *qid,
         stbuf = &st;
     }
 
+    /* Don't allow symbolic links. */
+    if ( S_ISLNK(stbuf->st_mode) )
+        return EMLINK;
+
     qid->type = S_ISDIR(stbuf->st_mode) ? QID_TYPE_DIR : 0;
     qid->version = stbuf->st_mtime ^ (stbuf->st_size << 8);
     qid->path = stbuf->st_ino;
@@ -583,6 +604,20 @@ static int fill_qid(device *device, const char *path, struct p9_qid *qid,
     return 0;
 }
 
+static bool name_ok(const char *str)
+{
+    if ( !*str )
+        return false;
+
+    if ( strchr(str, '/' ) )
+        return false;
+
+    if ( !strcmp(str, "..") || !strcmp(str, ".") )
+        return false;
+
+    return true;
+}
+
 static void p9_error(struct ring *ring, uint16_t tag, uint32_t err)
 {
     unsigned int erroff;
@@ -660,6 +695,138 @@ static void p9_attach(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "Q", &qid);
 }
 
+static void p9_walk(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint32_t newfid;
+    struct p9_fid *fidp = NULL;
+    struct p9_qid *qids = NULL;
+    unsigned int n_names = 0;
+    unsigned int *names = NULL;
+    unsigned int walked = 0;
+    unsigned int i;
+    char *path = NULL;
+    unsigned int path_len;
+    int ret;
+
+    ret = fill_data(ring, "UUaS", &fid, &newfid, &n_names, &names);
+    if ( n_names > P9_WALK_MAXELEM )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        goto out;
+    }
+    if ( ret != 3 + n_names )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        goto out;
+    }
+    if ( fidp->opened )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        goto out;
+    }
+
+    path_len = strlen(fidp->path) + 1;
+    for ( i = 0; i < n_names; i++ )
+    {
+        if ( !name_ok(ring->str + names[i]) )
+        {
+            p9_error(ring, hdr->tag, ENOENT);
+            goto out;
+        }
+        path_len += strlen(ring->str + names[i]) + 1;
+    }
+    path = calloc(path_len + 1, 1);
+    if ( !path )
+    {
+        p9_error(ring, hdr->tag, ENOMEM);
+        goto out;
+    }
+    strcpy(path, fidp->path);
+
+    if ( n_names )
+    {
+        qids = calloc(n_names, sizeof(*qids));
+        if ( !qids )
+        {
+            p9_error(ring, hdr->tag, ENOMEM);
+            goto out;
+        }
+        for ( i = 0; i < n_names; i++ )
+        {
+            strcat(path, "/");
+            strcat(path, ring->str + names[i]);
+            ret = fill_qid(device, path, qids + i, NULL);
+            if ( ret )
+            {
+                if ( !walked )
+                {
+                    p9_error(ring, hdr->tag, errno);
+                    goto out;
+                }
+                break;
+            }
+            walked++;
+        }
+    }
+
+    if ( walked == n_names )
+    {
+        bool ok = false;
+
+        if ( fid == newfid )
+        {
+            struct p9_fid *new_fidp;
+
+            pthread_mutex_lock(&device->fid_mutex);
+
+            if ( fidp->ref != 2 )
+            {
+                errno = EBUSY;
+            }
+            else
+            {
+                new_fidp = alloc_fid_mem(device, fid, path);
+                if ( new_fidp )
+                {
+                    new_fidp->ref = 2;
+                    XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+                    XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+                    free(fidp);
+                    fidp = new_fidp;
+                    ok = true;
+                }
+            }
+
+            pthread_mutex_unlock(&device->fid_mutex);
+        }
+        else
+            ok = alloc_fid(device, newfid, path);
+
+        if ( !ok )
+        {
+            p9_error(ring, hdr->tag, errno);
+            goto out;
+        }
+    }
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "aQ", &walked, qids);
+
+ out:
+    free_fid(device, fidp);
+    free(qids);
+    free(path);
+    free(names);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -723,6 +890,10 @@ void *io_thread(void *arg)
                 p9_attach(ring, &hdr);
                 break;
 
+            case P9_CMD_WALK:
+                p9_walk(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index 6c755411a7..f01fffb0bb 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -25,6 +25,7 @@ struct p9_fid {
     XEN_TAILQ_ENTRY(struct p9_fid) list;
     unsigned int fid;
     unsigned int ref;
+    bool opened;
     char path[];
 };
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:05:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:05:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681696.1060617 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabR9-0005sg-Cy; Thu, 15 Feb 2024 13:05:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681696.1060617; Thu, 15 Feb 2024 13:05:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabR9-0005sV-9E; Thu, 15 Feb 2024 13:05:59 +0000
Received: by outflank-mailman (input) for mailman id 681696;
 Thu, 15 Feb 2024 13:05:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabR7-0002yD-HU
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:05:57 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f503beee-cc02-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:05:57 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 8ACC31F8A4;
 Thu, 15 Feb 2024 13:05:56 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 56694139D0;
 Thu, 15 Feb 2024 13:05:56 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 70V3EzQMzmWeFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:05:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f503beee-cc02-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002356; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=y958eMYICr4NUQbK+9A+PrL+kcGy4gkIM8hHmNn6rfg=;
	b=LtJ0yJpYTTETF9XJaoW+MWfkdp1rZXY63Htg3kAAu2jxKdQTpSSmSsr8vknbjKtDZAQM8S
	45MV7MZjRP36xetkeP7sb/Lg9lyGQpO+dArm5ceglvscuW8+v/b6fAaCKuFlkwOkA4bzD5
	CJjicTMxZ+ZI29F19ZvbZSCXX6CANV0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002356; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=y958eMYICr4NUQbK+9A+PrL+kcGy4gkIM8hHmNn6rfg=;
	b=LtJ0yJpYTTETF9XJaoW+MWfkdp1rZXY63Htg3kAAu2jxKdQTpSSmSsr8vknbjKtDZAQM8S
	45MV7MZjRP36xetkeP7sb/Lg9lyGQpO+dArm5ceglvscuW8+v/b6fAaCKuFlkwOkA4bzD5
	CJjicTMxZ+ZI29F19ZvbZSCXX6CANV0=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 08/21] tools/xen-9pfsd: add 9pfs open request support
Date: Thu, 15 Feb 2024 14:04:56 +0100
Message-Id: <20240215130509.24008-9-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: 0.70
X-Spamd-Result: default: False [0.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add the open request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- don't allow to open symbolic link
V3:
- use openat() (Jason Andryuk)
- use common error handling in p9_open()
---
 tools/9pfsd/io.c        | 137 ++++++++++++++++++++++++++++++++++++++++
 tools/9pfsd/xen-9pfsd.h |   4 ++
 2 files changed, 141 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 731fbd1ad7..5ec780af14 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -19,6 +19,8 @@
 #include <syslog.h>
 #include <sys/types.h>
 #include <sys/stat.h>
+#include <dirent.h>
+#include <fcntl.h>
 #include <xenctrl.h>           /* For cpu barriers. */
 #include <xen-tools/common-macros.h>
 
@@ -29,6 +31,15 @@
 #define P9_CMD_ATTACH     104
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
+#define P9_CMD_OPEN       112
+
+/* P9 protocol open flags. */
+#define P9_OREAD            0   /* read */
+#define P9_OWRITE           1   /* write */
+#define P9_ORDWR            2   /* read and write */
+#define P9_OMODEMASK     0x03
+#define P9_OTRUNC        0x10   /* or'ed in, truncate file first */
+#define P9_OREMOVE       0x40   /* or'ed in, remove file after clunk */
 
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
@@ -827,6 +838,128 @@ static void p9_walk(struct ring *ring, struct p9_header *hdr)
     free(names);
 }
 
+static int open_flags_from_mode(uint8_t mode)
+{
+    int flags;
+
+    switch ( mode & P9_OMODEMASK )
+    {
+    case P9_OREAD:
+        flags = O_RDONLY;
+        break;
+
+    case P9_OWRITE:
+        flags = O_WRONLY;
+        break;
+
+    case P9_ORDWR:
+        flags = O_RDWR;
+        break;
+
+    default:
+        errno = EINVAL;
+        return -1;
+    }
+
+    if ( mode & P9_OTRUNC )
+        flags |= O_TRUNC;
+
+    return flags;
+}
+
+static unsigned int get_iounit(struct ring *ring, struct stat *st)
+{
+    return (ring->max_size - st->st_blksize) & ~(st->st_blksize - 1);
+}
+
+static void p9_open(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint8_t mode;
+    struct p9_fid *fidp;
+    struct stat st;
+    struct p9_qid qid;
+    uint32_t iounit;
+    int flags;
+    int ret;
+
+    ret = fill_data(ring, "Ub", &fid, &mode);
+    if ( ret != 2 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+    if ( mode & ~(P9_OMODEMASK | P9_OTRUNC | P9_OREMOVE) )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+    if ( fidp->opened )
+    {
+        errno = EINVAL;
+        goto err;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        errno = ENOENT;
+        goto err;
+    }
+
+    if ( S_ISLNK(st.st_mode) )
+    {
+        errno = EMLINK;
+        goto err;
+    }
+
+    fidp->isdir = S_ISDIR(st.st_mode);
+    fidp->mode = mode;
+    if ( fidp->isdir )
+    {
+        if ( mode != P9_OREAD )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        fidp->fd = openat(device->root_fd, fidp->path, O_RDONLY);
+        if ( fidp->fd < 0 )
+            goto err;
+        fidp->data = fdopendir(fidp->fd);
+        if ( !fidp->data )
+            goto err;
+    }
+    else
+    {
+        flags = open_flags_from_mode(mode);
+        if ( flags < 0 )
+            goto err;
+
+        fidp->fd = openat(device->root_fd, fidp->path, flags);
+        if ( fidp->fd < 0 )
+            goto err;
+    }
+
+    fill_qid(device, fidp->path, &qid, &st);
+    iounit = get_iounit(ring, &st);
+    fidp->opened = true;
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "QU", &qid, &iounit);
+
+    return;
+
+ err:
+    free_fid(device, fidp);
+    p9_error(ring, hdr->tag, errno);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -894,6 +1027,10 @@ void *io_thread(void *arg)
                 p9_walk(ring, &hdr);
                 break;
 
+            case P9_CMD_OPEN:
+                p9_open(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
diff --git a/tools/9pfsd/xen-9pfsd.h b/tools/9pfsd/xen-9pfsd.h
index f01fffb0bb..757be2da4b 100644
--- a/tools/9pfsd/xen-9pfsd.h
+++ b/tools/9pfsd/xen-9pfsd.h
@@ -25,7 +25,11 @@ struct p9_fid {
     XEN_TAILQ_ENTRY(struct p9_fid) list;
     unsigned int fid;
     unsigned int ref;
+    int fd;
+    uint8_t mode;
     bool opened;
+    bool isdir;
+    void *data;    /* File type specific. */
     char path[];
 };
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:06:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:06:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681700.1060626 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabRH-0006Rt-KW; Thu, 15 Feb 2024 13:06:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681700.1060626; Thu, 15 Feb 2024 13:06:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabRH-0006Rm-Hi; Thu, 15 Feb 2024 13:06:07 +0000
Received: by outflank-mailman (input) for mailman id 681700;
 Thu, 15 Feb 2024 13:06:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabRG-0002j3-0a
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:06 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f97c1002-cc02-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:06:04 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 2D5FA2218B;
 Thu, 15 Feb 2024 13:06:02 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id F0877139D0;
 Thu, 15 Feb 2024 13:06:01 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 1GtCOTkMzmWhFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f97c1002-cc02-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002364; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ZWrUdAn8G/WYw7Z+1jcLbLASeGn0zV6YHdFDB/FVJZ8=;
	b=WgaQ/5YU2tUY56RnuiKXPRFallvy18SVnEjYPNrEX3XLq5N59GKfVqUZp2UCLaf/ZQ/D1D
	4QC9lszG9OQUn3DOzjI0g7ojnSwIWRaF5oxXmdedPi9MGbKQBk/vxF80/aXNEhsPxDfDUt
	ucxdK4xbTHkNCsSqxLMVYuBtBaldQSM=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002362; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=ZWrUdAn8G/WYw7Z+1jcLbLASeGn0zV6YHdFDB/FVJZ8=;
	b=k4LFMBKBObiyWNa/GnSmZtrPLKnlN7hRgRgTxV4Zuy/MZPQH7lMt3nCoFbZ7K2zr49Y+60
	KeaE03RHfdMGe20O0Kve6XD1UAMFm9HkAF0z5vuR/VCGLo0rWNAykq7NxqIfnVOhpu53sF
	q0JAXAwZe1Ra2rjNiarO1I/X/Omfv0E=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 09/21] tools/xen-9pfsd: add 9pfs clunk request support
Date: Thu, 15 Feb 2024 14:04:57 +0100
Message-Id: <20240215130509.24008-10-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [4.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email,citrix.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[11.21%]
X-Spam-Level: ****
X-Spam-Score: 4.90
X-Spam-Flag: NO

Add the clunk request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V3:
- use unlinkat() (Jason Andryuk)
---
 tools/9pfsd/io.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 5ec780af14..6af14e5ee9 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -32,6 +32,7 @@
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
+#define P9_CMD_CLUNK      120
 
 /* P9 protocol open flags. */
 #define P9_OREAD            0   /* read */
@@ -960,6 +961,44 @@ static void p9_open(struct ring *ring, struct p9_header *hdr)
     p9_error(ring, hdr->tag, errno);
 }
 
+static void p9_clunk(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "U", &fid);
+    if ( ret != 1 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( fidp->opened )
+    {
+        fidp->opened = false;
+        free_fid(device, fidp);
+        close(fidp->fd);
+        if ( fidp->mode & P9_OREMOVE )
+            unlinkat(device->root_fd, fidp->path,
+                     fidp->isdir ? AT_REMOVEDIR : 0);
+    }
+
+    /* 2 calls of free_fid(): one for our reference, and one to free it. */
+    free_fid(device, fidp);
+    free_fid(device, fidp);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "");
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1031,6 +1070,10 @@ void *io_thread(void *arg)
                 p9_open(ring, &hdr);
                 break;
 
+            case P9_CMD_CLUNK:
+                p9_clunk(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:08:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:08:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681709.1060636 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabTs-00084O-1r; Thu, 15 Feb 2024 13:08:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681709.1060636; Thu, 15 Feb 2024 13:08:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabTr-00084H-VM; Thu, 15 Feb 2024 13:08:47 +0000
Received: by outflank-mailman (input) for mailman id 681709;
 Thu, 15 Feb 2024 13:08:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kkTL=JY=epam.com=prvs=8775c89ca9=oleksandr_tyshchenko@srs-se1.protection.inumbo.net>)
 id 1rabTq-00083y-8V
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:08:46 +0000
Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com
 [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 57f32638-cc03-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:08:43 +0100 (CET)
Received: from pps.filterd (m0174683.ppops.net [127.0.0.1])
 by mx0b-0039f301.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id
 41FCXna1020881; Thu, 15 Feb 2024 13:08:40 GMT
Received: from eur04-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur04lp2051.outbound.protection.outlook.com [104.47.14.51])
 by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3w9jqt03xy-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Thu, 15 Feb 2024 13:08:40 +0000 (GMT)
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com (2603:10a6:10:ed::15)
 by VI0PR03MB10397.eurprd03.prod.outlook.com (2603:10a6:800:1b9::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Thu, 15 Feb
 2024 13:08:35 +0000
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167]) by DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167%4]) with mapi id 15.20.7292.029; Thu, 15 Feb 2024
 13:08:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57f32638-cc03-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=NKFD7ItsiQH7K0KwjtckM5PkY75Upx59C4EChvSknKpo4X6YsdhIy6ydpiUttT6GfeMSRMOQb5+jFTsuBfDqKCOUu8VL+VacnNpl1TBvR8hLqurQl/ByBm9rexAt+8GhkhU90oNscB25c/0UQh6rRozqFPmvQxkDZlXVebJW14cdpcvyXicfX3HVDk1rfRgfWYWZ+ifOWU6mjdrsswg7x6g8D3Gll810mqvD55SDmx2y7NCYB2QwNkLGyWzRG/JqIDbZlevFxVAxvBeX+A6e0HElzuAv6LUacojyubPFGqlpCdyHQFhKyJgthJ//ZgXU0eMjPwuCTRcr3Rdpr7qvrg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=esFLRZ0NxMYwaOuMwp9NhK/tOhIteumlkqPY4MKo7dg=;
 b=P4krCnyOYxSSGGxfO0QBYAdsIvyNv7vYtMjcoxorx08CzSJ2vsWH62yJdj3sSHzuBMz89Bre+ulY98ZHXo4KhWAgB606YRZCmLG7USReqPQTtVm4+oMUkBGHmliCM/cLz8kdzC0r8913DjgLUGh4a5ycQP7add3vrlaqdQuDe6E5DYY9UROH8MAt5lbRZ5H93R37H382uTndYr8IfKdrUiXlX7XlZjupN20EA+aMCngEzF6U4LIDEBPnoi47uZXcXVoIcYKwE8KsqPhy1IZAOGnWNxDDeewHzXtPd+6foU7y8jpnrhQ/x9Mts9oagKclkAN3VLKtkVXrG0xCCVcFoA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=esFLRZ0NxMYwaOuMwp9NhK/tOhIteumlkqPY4MKo7dg=;
 b=A0+60p3njLNRKE6444HZveonTaiQB6QrAztExHCv2r7K5H3ofUs56psFLwXrvB7F3Y/PnAwtd4IDsdr7nrTYHGjIHigp73jUMaQ84M2HAsubI2rsWGeB4NQihHz7OrccQUI8nxq1BkAazoGeeAMAl9Crx2qyNHXFttZlv95jOWH5qacKAGrQdqIDps3f1KIL5EAPlHZvX459dVm2CwOsnDOQh7Fbwk6fud71E5GpTgqwm+KRMYmYu7pH+CF8i8ZEwCn5KSz+69GAOuqlYlC0vc2Go+9I1yuYHp+j3XESP2stjcFzmGdSbcs67pj/gTmONYHaA0AMclcF4Hm96SriFA==
From: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
To: Anthony PERARD <anthony.perard@citrix.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
        Nick Rosbrook
	<rosbrookn@gmail.com>, Juergen Gross <jgross@suse.com>,
        Viresh Kumar
	<viresh.kumar@linaro.org>,
        Oleksandr Tyshchenko <olekstysh@gmail.com>
Subject: Re: [PATCH V2] libxl: Add "grant_usage" parameter for virtio disk
 devices
Thread-Topic: [PATCH V2] libxl: Add "grant_usage" parameter for virtio disk
 devices
Thread-Index: AQHaWPlfVfBPDqFzI0qC9ZNM13q/+7EIOn0AgAMzrQA=
Date: Thu, 15 Feb 2024 13:08:34 +0000
Message-ID: <49102cc3-9165-450a-9533-b4a2bbc012fd@epam.com>
References: <20240206123814.2308837-1-olekstysh@gmail.com>
 <a12d924d-b570-4795-8ed6-f62f3604c361@perard>
In-Reply-To: <a12d924d-b570-4795-8ed6-f62f3604c361@perard>
Accept-Language: en-US, ru-RU
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DB8PR03MB6108:EE_|VI0PR03MB10397:EE_
x-ms-office365-filtering-correlation-id: 9c179d64-3b39-4374-834b-08dc2e273777
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 HtmNpNkST8WaQeKCUQxvxMFddzcW/SlCJJMl7Hv0Ra0NTmi5+5B6cd1Z79P7VanS/DuPjSvIhUX5OYR4p0Ep2Sq6P7W9qCl6E1Usgdrm62TeuS8nUkgrmIaXOI2Lm1ZFjxtd8S46NB0dMvxn5wtwGp7Kpl3Qbhg9NDyu4CFY42dG28VfGZhHQ8n5aOEi5tVLCwYjyThhGmiFnBPDHPQvFy6AidJOpPYMHNXaRb7V8b1zbPY4MFLkB5yoq+sovreNfZOfMaMox2YiaP5exRpXMYnZvzMh6S60tr596S40XFMq7kxx24Lm+NdbCRu24egrTbvBPPDDxWyWxIjRYxaSTMrj0HF+RuZ+sSU0ArGheHBNqVz8yXi+4BWf3pd22243ilTfsWFAIOujLaW6uW2KNczeABYND+GZ8eGdavWn8KsZQqigW+evU3qfhhddhJBYLZ0fX1XbluOH2NMR++/LMAgI6ZCIGYGZLWE/Yovt/gWdZidStCdhnULAc/VBKYVitrxwBludUpe5HVi4QRs/wmMriQSJ09//VcUohnsN1HhLBltN2LOJtTZ2YCFpIZ+pohltZUfmI+zrd7/T4lFLjhmXcZVXMXob7nZJPczNiSs=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR03MB6108.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(366004)(136003)(346002)(396003)(376002)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(4326008)(38070700009)(6916009)(83380400001)(8676002)(66946007)(26005)(2616005)(41300700001)(76116006)(66556008)(6506007)(53546011)(66446008)(6512007)(66476007)(64756008)(8936002)(6486002)(71200400001)(54906003)(31696002)(966005)(316002)(122000001)(36756003)(38100700002)(86362001)(478600001)(2906002)(31686004)(5660300002);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?blFINDJ6K0g5NmtpRmhwY0pjaWdWUWxjNEhHdVlnUG5lTGJrSzhiejBwa2RF?=
 =?utf-8?B?c2hsUUZETkkrVitwaitIMmRLL3FmbjUwMFI0Y29IYUVUYnhoYitMeHlIajlB?=
 =?utf-8?B?Y25aLzF3NDJMNENFbEZGdWdTSjJMTzI4c2VaZStFL1JieElGR003NWw0a0hq?=
 =?utf-8?B?Mmg1d2tNc211ZDJVMm5ScUcvQWxSUVJGM0NBYzMxUitROG5vWS9Md1R0M0Q0?=
 =?utf-8?B?NVFVRGZOTHVZbVFBZ2t2SmFPNVp1ck1tL243NXlpQnl6ZWxSUEVSVVVrUEIv?=
 =?utf-8?B?VWZ2cGxWV2dhbmgvNW9Ia3RBL1FXak42SnNJQ011UGZXcmE3Zm1ZY2JudE45?=
 =?utf-8?B?LzhqOFVOQ0ZKSUJYZ3JlZjl2NlE0WjRHTFloTENxMEd4QXdqTGtoUy9tUHNp?=
 =?utf-8?B?amdYN0txUkNkTk9wVkkzaUlncTBXWllrNlF2WHlXTm9QVE15WENkRWY1Tk9Q?=
 =?utf-8?B?TjJxVFJuK0hDK3hCcHNEVXNSaUt3emlBYkZoMmVnNjNQa1oyK0wxSWtZSVNI?=
 =?utf-8?B?MFJ3M3MwYlNubXlYS1VqdjVJZjdNMWdsMGhHRkZ3Yk5PbGhkZHBicXBTNDNU?=
 =?utf-8?B?elRubWxlbklJdXBkSlJleTdCb3BVZlJXYVdiaEdIVjZ3SlZBZjFmR2w3L1Ax?=
 =?utf-8?B?d1VxS3ZJOUNva1BaK1krQlVVekpoYXJ2UkhGaU5aYi9ZU2pibHJjU0Z1dTRI?=
 =?utf-8?B?RUh2dUV4N05ybzhPZENVVWhFak9lWVd0N095UWp1Nk1mNmcvTmtaNlhSc0NY?=
 =?utf-8?B?ZXdLaVBDQzBhZEIyT29UYmNibXErYjNqdTNyZTg0alV0QXFDQUxQSWNNREE1?=
 =?utf-8?B?bkFXMVJvSjJXanVJRlpCdXNmZDZpTG4vT1Zsd3M4QzJPWlVNeVI2N0VGUllV?=
 =?utf-8?B?ekxNa0dOV0N4a2tRcW1nbkUvRENDejZoT1grU3JQVVRXNlJxYlQzUGIzNmZ3?=
 =?utf-8?B?SmxmSk9JbnRkQTEwUkdyc3hOWEUrMUxJcGVoTVhwQm1lVE1YY0JxV0V5eTBS?=
 =?utf-8?B?akd4V21ETlI3dDFuTWFVMHpXKzRYa3ZCbGpkZWJMMWthNDFYUlh5YWZmWTI0?=
 =?utf-8?B?VmRRV1c5UjhFNm9YUG8ranpuTEtRTlR6b1NlVkpnaXlETEhMNURJdUpoK0dp?=
 =?utf-8?B?ZWJpTFgzTHp3N3M5VWRNbWtOWmFDRXB5eXMzWndHRkV5NzU1UEVQOWZmZlNI?=
 =?utf-8?B?ZVA1cE0xSnVITFhDbi8wNUg0RmpJN1JBY3hWQ2JJSDBjWFBZMXRDQXFGRlI5?=
 =?utf-8?B?SkptOXN6MWc4ajRQOWhyZDQ2M2dUMSsrYXFJMFdnQ3NWcU9qWDBlcm55U3Ja?=
 =?utf-8?B?WmRiTWtpRkxXTTJlanJZR1RjNkpKbUJsMUhaeFU3eTNOKzlQYTBtODE2aEEr?=
 =?utf-8?B?dHpYNGFQUTgrdll3a2lZK3FNejlIUDJFVkpoUFdEakQxYjhFdjFMWkwvcWNI?=
 =?utf-8?B?QUU3RnhXdlgzb3A5NTdjZUFUdHIrT0ppNjdjb0E4RUo5M0xnYk91S0hpZ3lE?=
 =?utf-8?B?VXROL2Zhblgzdkd0REV0WnUvZUdwc0ZuZG1DQXJOU2JGSkgxcEJKMHAyWjNi?=
 =?utf-8?B?ZksyWlVNNkgzbFZueldwZ3dURE5XSllVc2lLbTkzYmhLQ0phRk5MWEl3VUNY?=
 =?utf-8?B?YnE2L3cyYnErazQvRUdvSmY5VlowUjd4a0c3OFRJTE5vaFdzcExVcEdDek0v?=
 =?utf-8?B?RVZ3NEMyRllodXFMc0NoU0Y3dDl5LzVKTkRpVVM1NFJKV3crT1REM0Q0Vndh?=
 =?utf-8?B?YkwzSWJxRVZEb2VqVEltNTFheVVaSkJoRGwvemlkelNFZW8rNW5yYThueGp0?=
 =?utf-8?B?K1FZNmJlRzhDdzAyamZFeTlYZlp0bW1vTkRMU0pSa2pubE9DOEtFQlBwUUl6?=
 =?utf-8?B?T012OFdDUEhibmtKRGlkQUdnU1VBWGhJNkd1MWVMSjJZSU9xem1sb0VMTjEy?=
 =?utf-8?B?Skh2WTlqcWhBSFN2YWg5RlJqYU5KSThnVTY1YjljTHd1R1c4Q3hGeElXYVA3?=
 =?utf-8?B?dUNxTkUzMDNxQllxYzRacmtXYVpFWVVVL1JhVW83MTQ5WGMxblhnZFFkV0Iv?=
 =?utf-8?B?R0Y2aUJQK3dNTzBRNWV0V0IreEVyUHROa25QMGxUdGJUOGk0NHJ5bGw1RGtS?=
 =?utf-8?B?VEt3NjkweHhQSGphWlE0LzA4VXdhMCtnSkZYNDQrZEJhWXBVSEJaamtIK0JZ?=
 =?utf-8?Q?rSA1U6CDuzuCcxQBvO8eCGo=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C4A710595B067745B1D92C9ED3AD2EAB@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB8PR03MB6108.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c179d64-3b39-4374-834b-08dc2e273777
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Feb 2024 13:08:34.8565
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: S4wyxUVHeBNy7/IPRegjZrvuH6zLN/oQk6XkaIFizzeNUqCa6cWP767QJuy+e2ehlzaz9J3IETKVbi9R0iSXBZxADamkoSuH3kP6yD541ro=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI0PR03MB10397
X-Proofpoint-GUID: TSFpLSy0wYxPIN-3iRpLTwdw37rzNTB-
X-Proofpoint-ORIG-GUID: TSFpLSy0wYxPIN-3iRpLTwdw37rzNTB-
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-15_12,2024-02-14_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0
 adultscore=0 suspectscore=0 clxscore=1015 lowpriorityscore=0
 malwarescore=0 mlxlogscore=999 impostorscore=0 spamscore=0 bulkscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2401310000 definitions=main-2402150105

DQoNCk9uIDEzLjAyLjI0IDE0OjE0LCBBbnRob255IFBFUkFSRCB3cm90ZToNCg0KSGVsbG8gQW50
aG9ueQ0KDQo+IE9uIFR1ZSwgRmViIDA2LCAyMDI0IGF0IDAyOjM4OjE0UE0gKzAyMDAsIE9sZWtz
YW5kciBUeXNoY2hlbmtvIHdyb3RlOg0KPj4gRnJvbTogT2xla3NhbmRyIFR5c2hjaGVua28gPG9s
ZWtzYW5kcl90eXNoY2hlbmtvQGVwYW0uY29tPg0KPj4NCj4+IEFsbG93IGFkbWluaXN0cmF0b3Jz
IHRvIGNvbnRyb2wgd2hldGhlciBYZW4gZ3JhbnQgbWFwcGluZ3MgZm9yDQo+PiB0aGUgdmlydGlv
IGRpc2sgZGV2aWNlcyBzaG91bGQgYmUgdXNlZC4gQnkgZGVmYXVsdCAod2hlbiBuZXcNCj4+IHBh
cmFtZXRlciBpcyBub3Qgc3BlY2lmaWVkKSwgdGhlIGV4aXN0aW5nIGJlaGF2aW9yIGlzIHJldGFp
bmVkDQo+PiAod2UgZW5hYmxlIGdyYW50cyBpZiBiYWNrZW5kLWRvbWlkICE9IDApLg0KPj4NCj4+
IFNpZ25lZC1vZmYtYnk6IE9sZWtzYW5kciBUeXNoY2hlbmtvIDxvbGVrc2FuZHJfdHlzaGNoZW5r
b0BlcGFtLmNvbT4NCj4+IC0tLQ0KPj4gSW4gYWRkaXRpb24gdG8gImxpYnhsOiBhcm06IEFkZCBn
cmFudF91c2FnZSBwYXJhbWV0ZXIgZm9yIHZpcnRpbyBkZXZpY2VzIg0KPj4gaHR0cHM6Ly91cmxk
ZWZlbnNlLmNvbS92My9fX2h0dHBzOi8vZ2l0aHViLmNvbS94ZW4tcHJvamVjdC94ZW4vY29tbWl0
L2MxNDI1NDA2NWZmNDgyNmUzNGY3MTRlMTc5MGVhYjUyMTczNjhjMzhfXzshIUdGXzI5ZGJjUUlV
QlBBITE3MnF0MzB1STdEcHNaY0stSXBadTI1Sm9RU2NQZUY3ZG9fTUQtS0VxSmxvOGdhSC1UVjFQ
X0gyQ1lmc0dJMGpfbDBHZFVQUE80Qmp5VUQyUTg2TGsySWxGNXpDaUZXTUF3JCBbZ2l0aHViWy5d
Y29tXQ0KPj4NCj4+IEkgd29uZGVyLCB3aGV0aGVyIEkgaGFkIHRvIGFsc28gaW5jbHVkZSBhdXRv
Z2VuZXJhdGVkIGNoYW5nZXMgdG86DQo+PiAgIC0gdG9vbHMvbGlicy91dGlsL2xpYnhsdV9kaXNr
X2wuYw0KPj4gICAtIHRvb2xzL2xpYnMvdXRpbC9saWJ4bHVfZGlza19sLmgNCj4gDQo+IFdlbGws
IHRoYXQgY291bGQgYmUgZG9uZSBvbiBjb21taXQuIFRoZSBjaGFuZ2VzIGFyZSBnb2luZyB0byBi
ZSBuZWVkZWQNCj4gdG8gYmUgY29tbWl0dGVkIGFzIHRoZXkgbWF5IG5vdCBiZSByZWdlbmVyYXRl
ZCB0byBpbmNsdWRlIHRoZSBuZXcgZmVhdHVyZQ0KPiBpbiBhIGJ1aWxkLg0KDQpUaGFua3MuIEFz
IFYzIGlzIG5lZWRlZCBhbnl3YXksIEkgd2lsbCBpbmNsdWRlIHRoZW0uDQoNCg0KPiANCj4+IC0t
LQ0KPj4gZGlmZiAtLWdpdCBhL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfZGlzay5jIGIvdG9vbHMv
bGlicy9saWdodC9saWJ4bF9kaXNrLmMNCj4+IGluZGV4IGVhMzYyM2RkNmYuLmVkMDJiNjU1YTMg
MTAwNjQ0DQo+PiAtLS0gYS90b29scy9saWJzL2xpZ2h0L2xpYnhsX2Rpc2suYw0KPj4gKysrIGIv
dG9vbHMvbGlicy9saWdodC9saWJ4bF9kaXNrLmMNCj4+IEBAIC02MjMsNiArNjI4LDE1IEBAIHN0
YXRpYyBpbnQgbGlieGxfX2Rpc2tfZnJvbV94ZW5zdG9yZShsaWJ4bF9fZ2MgKmdjLCBjb25zdCBj
aGFyICpsaWJ4bF9wYXRoLA0KPj4gICAgICAgICAgICAgICBnb3RvIGNsZWFudXA7DQo+PiAgICAg
ICAgICAgfQ0KPj4gICAgICAgICAgIGRpc2stPmlycSA9IHN0cnRvdWwodG1wLCBOVUxMLCAxMCk7
DQo+PiArDQo+PiArICAgICAgICB0bXAgPSBsaWJ4bF9feHNfcmVhZChnYywgWEJUX05VTEwsDQo+
PiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICBHQ1NQUklOVEYoIiVzL2dyYW50X3VzYWdl
IiwgbGlieGxfcGF0aCkpOw0KPj4gKyAgICAgICAgaWYgKCF0bXApIHsNCj4+ICsgICAgICAgICAg
ICBMT0coREVCVUcsICJNaXNzaW5nIHhlbnN0b3JlIG5vZGUgJXMvZ3JhbnRfdXNhZ2UsIHVzaW5n
IGRlZmF1bHQgdmFsdWUiLCBsaWJ4bF9wYXRoKTsNCj4gDQo+IElzIHRoaXMgaW5mb3JtYXRpb24g
dXNlZnVsIGZvciBkZWJ1Z2dpbmc/DQo+IA0KPiBJdCBzaG91bGQgYmUgZWFzeSB0byBmaW5kIG91
dCBpZiB0aGUgZ3JhbnRfdXNhZ2Ugbm9kZSBpcyBwcmVzZW50IG9yIG5vdA0KPiBieSBsb29raW5n
IGF0IHhlbnN0b3JlLCBhbmQgSSBkb24ndCB0aGluayBsaWJ4bCBpcyBnb2luZyB0byBtYWtlIHVz
ZSBvZg0KPiB0aGF0IGluZm9ybWF0aW9uIGFmdGVyIHRoaXMgcG9pbnQsIHNvIEkgZG9uJ3QgdGhp
bmsgdGhhdCdzIGdvaW5nIHRvIGJlDQo+IHZlcnkgdXNlZnVsLg0KDQpJdCBpcyBub3QgdmVyeSB1
c2VmdWwsIHdpbGwgZHJvcCB0aGUgbG9nLg0KDQoNCj4gDQo+PiArICAgICAgICAgICAgbGlieGxf
ZGVmYm9vbF9zZXQoJmRpc2stPmdyYW50X3VzYWdlLA0KPj4gKyAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIGRpc2stPmJhY2tlbmRfZG9taWQgIT0gTElCWExfVE9PTFNUQUNLX0RPTUlEKTsN
Cj4+ICsgICAgICAgIH0gZWxzZQ0KPj4gKyAgICAgICAgICAgIGxpYnhsX2RlZmJvb2xfc2V0KCZk
aXNrLT5ncmFudF91c2FnZSwgc3RydG91bCh0bXAsIE5VTEwsIDApKTsNCj4gDQo+IFBlciBjb2Rp
bmcgc3R5bGUsIGl0J3MgYmV0dGVyIHRvIGhhdmUgYm90aCBzaWRlIG9mIGFuIGlmLi5lbHNlIHRv
IGhhdmUNCj4ge30tYmxvY2sgb3Igbm9uZSBvZiB0aGVtLiBTbyBjb3VsZCB5b3UgYWRkIGEge30g
YmxvY2sgaW4gdGhlIGVsc2UsIG9yDQo+IHJlbW92ZSB0aGUge30gZnJvbSB0aGUgdHJ1ZSBzaWRl
IGlmIHdlIHJlbW92ZSB0aGUgTE9HKCk/DQoNCg0KV2lsbCBkby4NCg0KPiANCj4gVGhhbmtzLA0K
PiA=


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:09:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:09:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681713.1060647 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUA-00008n-E6; Thu, 15 Feb 2024 13:09:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681713.1060647; Thu, 15 Feb 2024 13:09:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUA-00008g-B4; Thu, 15 Feb 2024 13:09:06 +0000
Received: by outflank-mailman (input) for mailman id 681713;
 Thu, 15 Feb 2024 13:09:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabSG-0002j3-64
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:07:08 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1e6ceba8-cc03-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:07:06 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 1069221FBC;
 Thu, 15 Feb 2024 13:07:04 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id C0A7D139D0;
 Thu, 15 Feb 2024 13:07:03 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id gvdYLXcMzmXnFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:07:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1e6ceba8-cc03-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002426; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mx+xsXHmP+kMTvkOl3Dx/uX2x1DBpbeqqzrtIUR5SLs=;
	b=CXaDg2Ov5Npo6W35bdMbfKhQ13AC06etHiojWIjbj3eOLo8wndkIGzHojJ/kJ3iWvUoVow
	F4nct+l1bc5YGv8OwlKb4vttk9asIBbgAubrUd2iE7DYvZHdiKK3sJ6uaTJU3V4xeteWG5
	At0ltHM15EHCGELZM/C5JaDs5VWj74M=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002424; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=mx+xsXHmP+kMTvkOl3Dx/uX2x1DBpbeqqzrtIUR5SLs=;
	b=TZu9bZybVfoUrF4bOMVjZJFqR1YgDa3ugxAjBUJRms6AFO3LpQaZphxTQyvgmy5/hBukCx
	v0FMnG65xSbfFOBQBwxPMTkl6yrouOhuOIquQ4daPmLOx2XJb9bilNHuUiXROBoT90e87k
	IY+cjPrlOQlW3NwcnXN2W+hcRFRHIik=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Julien Grall <julien@xen.org>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v7 20/21] tools/xenstored: support complete log capabilities in stubdom
Date: Thu, 15 Feb 2024 14:05:08 +0100
Message-Id: <20240215130509.24008-21-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=TZu9bZyb
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [0.49 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[xencommons.in:url,suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 0.49
X-Rspamd-Queue-Id: 1069221FBC
X-Spam-Flag: NO

With 9pfs being fully available in Xenstore-stubdom now, there is no
reason to not fully support all logging capabilities in stubdom.

Open the logfile on stubdom only after the 9pfs file system has been
mounted.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- remove now stale comment in sysconfig.xencommons.in (Julien Grall)
---
 .../Linux/init.d/sysconfig.xencommons.in      |  1 -
 tools/hotplug/Linux/launch-xenstore.in        |  1 +
 tools/xenstored/control.c                     | 30 +++++++++----------
 tools/xenstored/minios.c                      |  3 ++
 4 files changed, 19 insertions(+), 16 deletions(-)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index 433e4849af..1bdd830d8a 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -58,7 +58,6 @@ XENSTORED_ARGS=
 ## Default: Not defined, tracing off
 #
 # Log xenstored messages
-# Only evaluated if XENSTORETYPE is "daemon".
 #XENSTORED_TRACE=[yes|on|1]
 
 ## Type: integer
diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in
index e854ca1eb8..da4eeca7c5 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -98,6 +98,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 	[ -z "$XENSTORE_DOMAIN_SIZE" ] && XENSTORE_DOMAIN_SIZE=8
 	XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory $XENSTORE_DOMAIN_SIZE"
 	[ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem $XENSTORE_MAX_DOMAIN_SIZE"
+	[ -z "$XENSTORED_TRACE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS -T xenstored-trace.log"
 
 	echo -n Starting $XENSTORE_DOMAIN_KERNEL...
 	${LIBEXEC_BIN}/init-xenstore-domain $XENSTORE_DOMAIN_ARGS || exit 1
diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index b2f64d674f..dae23a5ac0 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -201,19 +201,6 @@ static int do_control_quota_s(const void *ctx, struct connection *conn,
 	return EINVAL;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_logfile(const void *ctx, struct connection *conn,
 			      const char **vec, int num)
 {
@@ -222,13 +209,26 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 
 	close_log();
 	talloc_free(tracefile);
-	tracefile = talloc_strdup(NULL, vec[0]);
+	tracefile = absolute_filename(NULL, vec[0]);
 	reopen_log();
 
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
 
+#ifdef __MINIOS__
+static int do_control_memreport(const void *ctx, struct connection *conn,
+				const char **vec, int num)
+{
+	if (num)
+		return EINVAL;
+
+	talloc_report_full(NULL, stdout);
+
+	send_ack(conn, XS_CONTROL);
+	return 0;
+}
+#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
@@ -309,10 +309,10 @@ static struct cmd_s cmds[] = {
 		"[-c <cmdline>] [-F] [-t <timeout>] <file>\n"
 		"    Default timeout is 60 seconds.", 5 },
 #endif
+	{ "logfile", do_control_logfile, "<file>" },
 #ifdef __MINIOS__
 	{ "memreport", do_control_memreport, "" },
 #else
-	{ "logfile", do_control_logfile, "<file>" },
 	{ "memreport", do_control_memreport, "[<file>]" },
 #endif
 	{ "print", do_control_print, "<string>" },
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index e70386f8c7..a229954cf4 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -122,6 +122,9 @@ static void mount_thread(void *p)
 	}
 
 	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+
+	/* Start logging if selected. */
+	reopen_log();
 }
 
 void mount_9pfs(void)
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:09:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:09:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681719.1060657 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUL-0000qm-NA; Thu, 15 Feb 2024 13:09:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681719.1060657; Thu, 15 Feb 2024 13:09:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUL-0000qf-KA; Thu, 15 Feb 2024 13:09:17 +0000
Received: by outflank-mailman (input) for mailman id 681719;
 Thu, 15 Feb 2024 13:09:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabRg-0002j3-Pd
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:32 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 09242fa7-cc03-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:06:30 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 3CF961FD63;
 Thu, 15 Feb 2024 13:06:30 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id EE111139D0;
 Thu, 15 Feb 2024 13:06:29 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id yACnOFUMzmW+FQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09242fa7-cc03-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002390; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zgmWzK99RZBrySpVIOBOfTuvIg1qZCvB7PuUVcrLtSw=;
	b=fdQozqubiTxpK61xHdVDLgeuphh8maCl7TpPVAl75xx0+AHbkTNX95JBSvYiGkc0f27OPO
	Lp8lZUU2Mfrt60O9V88gOC6A9F87XibynB1RQo5Y/oC3grjbdUm2XTwRHkXK4ilGttEzuv
	0N+7Ocrg8k/fOWdlJdA14ExCaDqVJR0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002390; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zgmWzK99RZBrySpVIOBOfTuvIg1qZCvB7PuUVcrLtSw=;
	b=fdQozqubiTxpK61xHdVDLgeuphh8maCl7TpPVAl75xx0+AHbkTNX95JBSvYiGkc0f27OPO
	Lp8lZUU2Mfrt60O9V88gOC6A9F87XibynB1RQo5Y/oC3grjbdUm2XTwRHkXK4ilGttEzuv
	0N+7Ocrg8k/fOWdlJdA14ExCaDqVJR0=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	George Dunlap <george.dunlap@cloud.com>
Subject: [PATCH v7 14/21] tools/libs/light: add backend type for 9pfs PV devices
Date: Thu, 15 Feb 2024 14:05:02 +0100
Message-Id: <20240215130509.24008-15-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=fdQozqub
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [0.49 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email,citrix.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,citrix.com,gmail.com,xen.org,cloud.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 0.49
X-Rspamd-Queue-Id: 3CF961FD63
X-Spam-Flag: NO

Make the backend type of 9pfs PV devices configurable. The default is
"qemu" with the related Xenstore backend-side directory being "9pfs".

Add another type "xen_9pfsd" with the related Xenstore backend-side
directory "xen_9pfs".

As additional security features it is possible to specify:
- "max-space" for limiting the maximum space consumed on the filesystem
  in MBs
- "max-files" for limiting the maximum number of files in the
  filesystem
- "max-open-files" for limiting the maximum number of concurrent open
  files

For convenience "auto-delete" is available to let the backend delete the
oldest file of the guest in case otherwise "max-space" or "max-files"
would be violated.

The xen-9pfsd daemon will be started by libxenlight automatically when
the first "xen_9pfs" device is being created.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: George Dunlap <george.dunlap@cloud.com> # Golang bits
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
---
V3:
- regenerate go bindings
V4:
- rename libxl_device_p9_dm_needed() to libxl__device_p9_dm_needed()
  (Anthony Perard)
- reorder span related functions (Anthony Perard)
- add comment for xen9pfsd_spawn() return values (Anthony Perard)
- add LIBXL_HAVE_XEN_9PFS to libxl.h (Anthony Perard)
- use a copy of 'p9' in xen9pfsd_spawn() (Anthony Perard)
V6:
- rebase (Anthony Perard)
- drop callback from struct libxl__aop9_state (Anthony Perard)
---
 tools/golang/xenlight/helpers.gen.go      |  10 ++
 tools/golang/xenlight/types.gen.go        |  12 ++
 tools/include/libxl.h                     |   7 +
 tools/libs/light/libxl_9pfs.c             | 157 +++++++++++++++++++++-
 tools/libs/light/libxl_create.c           |   4 +-
 tools/libs/light/libxl_dm.c               |   2 +-
 tools/libs/light/libxl_types.idl          |  11 ++
 tools/libs/light/libxl_types_internal.idl |   1 +
 8 files changed, 197 insertions(+), 7 deletions(-)

diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 0f8e23773c..8f44397a4e 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -2440,6 +2440,11 @@ x.Tag = C.GoString(xc.tag)
 x.Path = C.GoString(xc.path)
 x.SecurityModel = C.GoString(xc.security_model)
 x.Devid = Devid(xc.devid)
+x.Type = P9Type(xc._type)
+x.MaxSpace = int(xc.max_space)
+x.MaxFiles = int(xc.max_files)
+x.MaxOpenFiles = int(xc.max_open_files)
+x.AutoDelete = bool(xc.auto_delete)
 
  return nil}
 
@@ -2458,6 +2463,11 @@ xc.path = C.CString(x.Path)}
 if x.SecurityModel != "" {
 xc.security_model = C.CString(x.SecurityModel)}
 xc.devid = C.libxl_devid(x.Devid)
+xc._type = C.libxl_p9_type(x.Type)
+xc.max_space = C.int(x.MaxSpace)
+xc.max_files = C.int(x.MaxFiles)
+xc.max_open_files = C.int(x.MaxOpenFiles)
+xc.auto_delete = C.bool(x.AutoDelete)
 
  return nil
  }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 9c8b7b81f6..d31722407a 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -122,6 +122,13 @@ NicTypeVifIoemu NicType = 1
 NicTypeVif NicType = 2
 )
 
+type P9Type int
+const(
+P9TypeUnknown P9Type = 0
+P9TypeQemu P9Type = 1
+P9TypeXen9Pfsd P9Type = 2
+)
+
 type ActionOnShutdown int
 const(
 ActionOnShutdownDestroy ActionOnShutdown = 1
@@ -889,6 +896,11 @@ Tag string
 Path string
 SecurityModel string
 Devid Devid
+Type P9Type
+MaxSpace int
+MaxFiles int
+MaxOpenFiles int
+AutoDelete bool
 }
 
 type DevicePvcallsif struct {
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 46bc774126..9a3e702557 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -615,6 +615,13 @@
  */
 #define LIBXL_HAVE_HVM_PIRQ 1
 
+/*
+ * LIBXL_HAVE_XEN_9PFS indicates the presence of the xen-9pfsd related
+ * fields in libxl_device_p9: type, max_space, max_files, max_open_files and
+ * auto_delete.
+ */
+#define LIBXL_HAVE_XEN_9PFS 1
+
 /*
  * libxl memory management
  *
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 5ab0d3aa21..900c0d46a0 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -33,20 +33,171 @@ static int libxl__set_xenstore_p9(libxl__gc *gc, uint32_t domid,
 
     flexarray_append_pair(front, "tag", p9->tag);
 
+    if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD) {
+        flexarray_append_pair(back, "max-space",
+                              GCSPRINTF("%u", p9->max_space));
+        flexarray_append_pair(back, "max-files",
+                              GCSPRINTF("%u", p9->max_files));
+        flexarray_append_pair(back, "max-open-files",
+                              GCSPRINTF("%u", p9->max_open_files));
+        flexarray_append_pair(back, "auto-delete",
+                              p9->auto_delete ? "1" : "0");
+    }
+
+    return 0;
+}
+
+static int libxl__device_from_p9(libxl__gc *gc, uint32_t domid,
+                                 libxl_device_p9 *type, libxl__device *device)
+{
+    device->backend_devid   = type->devid;
+    device->backend_domid   = type->backend_domid;
+    device->backend_kind    = type->type == LIBXL_P9_TYPE_QEMU
+                              ? LIBXL__DEVICE_KIND_9PFS
+                              : LIBXL__DEVICE_KIND_XEN_9PFS;
+    device->devid           = type->devid;
+    device->domid           = domid;
+    device->kind            = LIBXL__DEVICE_KIND_9PFS;
+
     return 0;
 }
 
-#define libxl__add_p9s NULL
+static int libxl__device_p9_dm_needed(void *e, unsigned domid)
+{
+    libxl_device_p9 *elem = e;
+
+    return elem->type == LIBXL_P9_TYPE_QEMU && elem->backend_domid == domid;
+}
+
+typedef struct libxl__aop9_state libxl__aop9_state;
+
+struct libxl__aop9_state {
+    libxl__spawn_state spawn;
+    libxl__ao_device *aodev;
+    libxl_device_p9 p9;
+    uint32_t domid;
+};
+
+static void xen9pfsd_confirm(libxl__egc *egc, libxl__spawn_state *spawn,
+                             const char *xsdata);
+static void xen9pfsd_failed(libxl__egc *egc, libxl__spawn_state *spawn, int rc);
+static void xen9pfsd_detached(libxl__egc *egc, libxl__spawn_state *spawn);
+static void xen9pfsd_spawn_outcome(libxl__egc *egc, libxl__aop9_state *aop9,
+                                   int rc);
+
+/*
+ * Spawn the xen-9pfsd daemon if needed.
+ * returns:
+ * < 0 if error
+ * 0 if no daemon needs to be spawned
+ * 1 if daemon was spawned
+ */
+static int xen9pfsd_spawn(libxl__egc *egc, uint32_t domid, libxl_device_p9 *p9,
+                         libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    struct libxl__aop9_state *aop9;
+    int rc;
+    char *args[] = { "xen-9pfsd", NULL };
+    char *path = GCSPRINTF("/local/domain/%u/libxl/xen-9pfs",
+                           p9->backend_domid);
+
+    if (p9->type != LIBXL_P9_TYPE_XEN_9PFSD ||
+        libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/state", path)))
+        return 0;
+
+    GCNEW(aop9);
+    aop9->aodev = aodev;
+    libxl_device_p9_copy(CTX, &aop9->p9, p9);
+    aop9->domid = domid;
+
+    aop9->spawn.ao = aodev->ao;
+    aop9->spawn.what = "xen-9pfs daemon";
+    aop9->spawn.xspath = GCSPRINTF("%s/state", path);
+    aop9->spawn.timeout_ms = LIBXL_DEVICE_MODEL_START_TIMEOUT * 1000;
+    aop9->spawn.pidpath = GCSPRINTF("%s/pid", path);
+    aop9->spawn.midproc_cb = libxl__spawn_record_pid;
+    aop9->spawn.confirm_cb = xen9pfsd_confirm;
+    aop9->spawn.failure_cb = xen9pfsd_failed;
+    aop9->spawn.detached_cb = xen9pfsd_detached;
+    rc = libxl__spawn_spawn(egc, &aop9->spawn);
+    if (rc < 0)
+        return rc;
+    if (!rc) {
+        setsid();
+        libxl__exec(gc, -1, -1, -1, LIBEXEC_BIN "/xen-9pfsd", args, NULL);
+    }
+
+    return 1;
+}
+
+static void xen9pfsd_confirm(libxl__egc *egc, libxl__spawn_state *spawn,
+                             const char *xsdata)
+{
+    STATE_AO_GC(spawn->ao);
+
+    if (!xsdata)
+        return;
+
+    if (strcmp(xsdata, "running"))
+        return;
+
+    libxl__spawn_initiate_detach(gc, spawn);
+}
+
+static void xen9pfsd_failed(libxl__egc *egc, libxl__spawn_state *spawn, int rc)
+{
+    libxl__aop9_state *aop9 = CONTAINER_OF(spawn, *aop9, spawn);
+
+    xen9pfsd_spawn_outcome(egc, aop9, rc);
+}
+
+static void xen9pfsd_detached(libxl__egc *egc, libxl__spawn_state *spawn)
+{
+    libxl__aop9_state *aop9 = CONTAINER_OF(spawn, *aop9, spawn);
+
+    xen9pfsd_spawn_outcome(egc, aop9, 0);
+}
+
+static void xen9pfsd_spawn_outcome(libxl__egc *egc, libxl__aop9_state *aop9,
+                                   int rc)
+{
+    aop9->aodev->rc = rc;
+    if (rc)
+        aop9->aodev->callback(egc, aop9->aodev);
+    else
+        libxl__device_add_async(egc, aop9->domid, &libxl__p9_devtype,
+                                &aop9->p9, aop9->aodev);
+}
+
+static void libxl__device_p9_add(libxl__egc *egc, uint32_t domid,
+                                 libxl_device_p9 *p9,
+                                 libxl__ao_device *aodev)
+{
+    int rc;
+
+    rc = xen9pfsd_spawn(egc, domid, p9, aodev);
+    if (rc == 1)
+        return;
+
+    if (rc == 0)
+        libxl__device_add_async(egc, domid, &libxl__p9_devtype, p9, aodev);
+
+    aodev->rc = rc;
+    if (rc)
+        aodev->callback(egc, aodev);
+}
+
 #define libxl_device_p9_list NULL
 #define libxl_device_p9_compare NULL
 
 static LIBXL_DEFINE_UPDATE_DEVID(p9)
-static LIBXL_DEFINE_DEVICE_FROM_TYPE(p9)
+static LIBXL_DEFINE_DEVICES_ADD(p9)
 
 LIBXL_DEFINE_DEVICE_REMOVE(p9)
 
 DEFINE_DEVICE_TYPE_STRUCT(p9, 9PFS, p9s,
-    .skip_attach = 1,
     .set_xenstore_config = (device_set_xenstore_config_fn_t)
                            libxl__set_xenstore_p9,
+    .dm_needed = libxl__device_p9_dm_needed,
 );
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 0008fac607..5546335973 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -1761,9 +1761,6 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
         libxl__device_console_dispose(&console);
     }
 
-    for (i = 0; i < d_config->num_p9s; i++)
-        libxl__device_add(gc, domid, &libxl__p9_devtype, &d_config->p9s[i]);
-
     for (i = 0; i < d_config->num_pvcallsifs; i++)
         libxl__device_add(gc, domid, &libxl__pvcallsif_devtype,
                           &d_config->pvcallsifs[i]);
@@ -1900,6 +1897,7 @@ const libxl__device_type *device_type_tbl[] = {
     &libxl__vdispl_devtype,
     &libxl__vsnd_devtype,
     &libxl__virtio_devtype,
+    &libxl__p9_devtype,
     NULL
 };
 
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 14b593110f..2aaaeb5aa7 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -3761,7 +3761,7 @@ int libxl__need_xenpv_qemu(libxl__gc *gc, libxl_domain_config *d_config)
         goto out;
     }
 
-    if (d_config->num_vfbs > 0 || d_config->num_p9s > 0) {
+    if (d_config->num_vfbs > 0) {
         ret = 1;
         goto out;
     }
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 899ad30969..682092a058 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -150,6 +150,12 @@ libxl_nic_type = Enumeration("nic_type", [
     (2, "VIF"),
     ])
 
+libxl_p9_type = Enumeration("p9_type", [
+    (0, "unknown"),
+    (1, "qemu"),
+    (2, "xen_9pfsd"),
+    ])
+
 libxl_action_on_shutdown = Enumeration("action_on_shutdown", [
     (1, "DESTROY"),
 
@@ -943,6 +949,11 @@ libxl_device_p9 = Struct("device_p9", [
     ("path",             string),
     ("security_model",   string),
     ("devid",            libxl_devid),
+    ("type",             libxl_p9_type),
+    ("max_space",        integer),
+    ("max_files",        integer),
+    ("max_open_files",   integer),
+    ("auto_delete",      bool),
 ])
 
 libxl_device_pvcallsif = Struct("device_pvcallsif", [
diff --git a/tools/libs/light/libxl_types_internal.idl b/tools/libs/light/libxl_types_internal.idl
index 56dccac153..0425e9b6b0 100644
--- a/tools/libs/light/libxl_types_internal.idl
+++ b/tools/libs/light/libxl_types_internal.idl
@@ -35,6 +35,7 @@ libxl__device_kind = Enumeration("device_kind", [
     (17, "VIRTIO_DISK"),
     (18, "VIRTIO"),
     (19, "VBD3"),
+    (20, "XEN_9PFS"),
     ])
 
 libxl__console_backend = Enumeration("console_backend", [
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:09:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:09:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681725.1060667 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUi-0001ge-WC; Thu, 15 Feb 2024 13:09:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681725.1060667; Thu, 15 Feb 2024 13:09:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUi-0001gX-S8; Thu, 15 Feb 2024 13:09:40 +0000
Received: by outflank-mailman (input) for mailman id 681725;
 Thu, 15 Feb 2024 13:09:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabRq-0002yD-9T
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:42 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0fb82da1-cc03-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:06:41 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 5934A1F8A4;
 Thu, 15 Feb 2024 13:06:41 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 2F783139D0;
 Thu, 15 Feb 2024 13:06:41 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id hyQtCmEMzmXUFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0fb82da1-cc03-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002401; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/eJaaQzJFr9iaY84hbFZ6eF6VnkmvJwoY+5FWl4+J1Y=;
	b=jX3jfVxde2pVviIVT8uNiTW6GFTPiKpGpbzzIV/CuUotoF8slE/BAxdldseGn1C/rOUnQA
	VVyuQo93hbpTiLGPf+++ZbsQP3IW+wePe7GtwbpG9jMZK3zwt8IvGfE4+uv7Aud6ZkFhMP
	w8LRbnkiHuT0jiPW0y3m58lIyBnrd6w=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002401; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/eJaaQzJFr9iaY84hbFZ6eF6VnkmvJwoY+5FWl4+J1Y=;
	b=jX3jfVxde2pVviIVT8uNiTW6GFTPiKpGpbzzIV/CuUotoF8slE/BAxdldseGn1C/rOUnQA
	VVyuQo93hbpTiLGPf+++ZbsQP3IW+wePe7GtwbpG9jMZK3zwt8IvGfE4+uv7Aud6ZkFhMP
	w8LRbnkiHuT0jiPW0y3m58lIyBnrd6w=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 16/21] stubdom: extend xenstore stubdom configs
Date: Thu, 15 Feb 2024 14:05:04 +0100
Message-Id: <20240215130509.24008-17-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=jX3jfVxd
X-Spamd-Result: default: False [3.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 TO_DN_SOME(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DWL_DNSWL_BLOCKED(0.00)[suse.com:dkim];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,ens-lyon.org,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-1.00)[87.16%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 3.69
X-Rspamd-Queue-Id: 5934A1F8A4
X-Spam-Level: ***
X-Spam-Flag: NO
X-Spamd-Bar: +++

Extend the config files of the Xenstore stubdoms to include XENBUS
and 9PFRONT items in order to support file based logging.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 stubdom/xenstore-minios.cfg    | 2 +-
 stubdom/xenstorepvh-minios.cfg | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/stubdom/xenstore-minios.cfg b/stubdom/xenstore-minios.cfg
index a41704bb6b..239da519b9 100644
--- a/stubdom/xenstore-minios.cfg
+++ b/stubdom/xenstore-minios.cfg
@@ -3,7 +3,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
diff --git a/stubdom/xenstorepvh-minios.cfg b/stubdom/xenstorepvh-minios.cfg
index 6af51f5753..752b90d7d3 100644
--- a/stubdom/xenstorepvh-minios.cfg
+++ b/stubdom/xenstorepvh-minios.cfg
@@ -4,7 +4,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:09:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:09:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681734.1060688 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUx-0002XC-UT; Thu, 15 Feb 2024 13:09:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681734.1060688; Thu, 15 Feb 2024 13:09:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUx-0002Vm-Of; Thu, 15 Feb 2024 13:09:55 +0000
Received: by outflank-mailman (input) for mailman id 681734;
 Thu, 15 Feb 2024 13:09:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YE+Y=JY=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rabRY-0002j3-58
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:24 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 04105be3-cc03-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:06:22 +0100 (CET)
Received: from nico.bugseng.com (unknown [147.123.100.131])
 by support.bugseng.com (Postfix) with ESMTPSA id EA5CD4EE0737;
 Thu, 15 Feb 2024 14:06:19 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 04105be3-cc03-11ee-98f5-efadbce2ee36
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	Nicola Vetrini <nicola.vetrini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH v2] automation/eclair_analysis: deviate certain macros for Rule 20.12
Date: Thu, 15 Feb 2024 14:06:17 +0100
Message-Id: <7bc72f6ccd858e0405d62d49c32449e3b5abd790.1707996317.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Certain macros are allowed to violate the Rule, since their meaning and
intended use is well-known to all Xen developers.

Variadic macros that rely on the GCC extension for removing a trailing
comma when token pasting the variable argument are similarly
well-understood and therefore allowed.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
Changes in v2:
- Restrict deviation for GENERATE_CASE to vcpreg.c.
- Improve deviation justifications.
---
 .../eclair_analysis/ECLAIR/deviations.ecl     | 20 +++++++++++++++++
 docs/misra/deviations.rst                     | 22 +++++++++++++++++++
 2 files changed, 42 insertions(+)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
index fd32ff8a9cae..04cb41e16a50 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -387,6 +387,26 @@ in assignments."
 {safe, "left_right(^[(,\\[]$,^[),\\]]$)"}
 -doc_end
 
+-doc_begin="Uses of variadic macros that have one of their arguments defined as
+a macro and used within the body for both ordinary parameter expansion and as an
+operand to the # or ## operators have a behavior that is well-understood and
+deliberate."
+-config=MC3R1.R20.12,macros+={deliberate, "variadic()"}
+-doc_end
+
+-doc_begin="Uses of a macro parameter for ordinary expansion and as an operand
+to the # or ## operators within the following macros are deliberate, to provide
+useful diagnostic messages to the user."
+-config=MC3R1.R20.12,macros+={deliberate, "name(ASSERT||BUILD_BUG_ON||BUILD_BUG_ON_ZERO)"}
+-doc_end
+
+-doc_begin="The helper macro GENERATE_CASE may use a macro parameter for ordinary
+expansion and token pasting to improve readability. Only instances where this
+leads to a violation of the Rule are deviated."
+-file_tag+={deliberate_generate_case, "^xen/arch/arm/vcpreg\\.c$"}
+-config=MC3R1.R20.12,macros+={deliberate, "name(GENERATE_CASE)&&loc(file(deliberate_generate_case))"}
+-doc_end
+
 #
 # General
 #
diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
index 123c78e20a01..84da637ef888 100644
--- a/docs/misra/deviations.rst
+++ b/docs/misra/deviations.rst
@@ -330,6 +330,28 @@ Deviations related to MISRA C:2012 Rules:
        (4) as lhs in assignments.
      - Tagged as `safe` for ECLAIR.
 
+   * - R20.12
+     - Variadic macros that use token pasting often employ the gcc extension
+       `ext_paste_comma`, as detailed in `C-language-toolchain.rst`, which is
+       not easily replaceable; macros that in addition perform regular argument
+       expansion on the same argument subject to the # or ## operators violate
+       the Rule if the argument is a macro. 
+     - Tagged as `deliberate` for ECLAIR.
+
+   * - R20.12
+     - Macros that are used for runtime or build-time assertions contain
+       deliberate uses of an argument as both a regular argument and a
+       stringification token, to provide useful diagnostic messages.
+     - Tagged as `deliberate` for ECLAIR.
+
+   * - R20.12
+     - GENERATE_CASE is a local helper macro that allows some selected switch
+       statements to be more compact and readable. As such, the risk of
+       developer confusion in using such macro is deemed negligible. This
+       construct is deviated only in Translation Units that present a violation
+       of the Rule due to uses of this macro.
+     - Tagged as `deliberate` for ECLAIR.
+
 Other deviations:
 -----------------
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:09:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:09:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681731.1060677 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUx-0002O0-Bm; Thu, 15 Feb 2024 13:09:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681731.1060677; Thu, 15 Feb 2024 13:09:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUx-0002Nt-7Y; Thu, 15 Feb 2024 13:09:55 +0000
Received: by outflank-mailman (input) for mailman id 681731;
 Thu, 15 Feb 2024 13:09:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabS8-0002j3-IQ
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:07:00 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 19d40f09-cc03-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:06:58 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 5FFC62218B;
 Thu, 15 Feb 2024 13:06:58 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 17AB9139D0;
 Thu, 15 Feb 2024 13:06:58 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id aBk4BHIMzmXlFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19d40f09-cc03-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002418; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xUq3AE+SsnVI0DjnPSJQwNriZRE6cVmY47vrUqUrQuc=;
	b=Z8GQHXYLcYMI+4d3BxocWdQ8ugWSTSeWig17x3hZy+1S6u9hFchxBjqrSZsr5C9eaiV2GW
	C5/0N5G8dVorAaUWLstuJaFqhJOgtdNX5t6qsPxa6N/TM95D43h3U7zeYWHP+JHj/pcHK2
	u92XgSkHq/3h1qP4xXb37nwmG5WPSxU=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002418; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xUq3AE+SsnVI0DjnPSJQwNriZRE6cVmY47vrUqUrQuc=;
	b=Z8GQHXYLcYMI+4d3BxocWdQ8ugWSTSeWig17x3hZy+1S6u9hFchxBjqrSZsr5C9eaiV2GW
	C5/0N5G8dVorAaUWLstuJaFqhJOgtdNX5t6qsPxa6N/TM95D43h3U7zeYWHP+JHj/pcHK2
	u92XgSkHq/3h1qP4xXb37nwmG5WPSxU=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v7 19/21] tools/xenstored: add helpers for filename handling
Date: Thu, 15 Feb 2024 14:05:07 +0100
Message-Id: <20240215130509.24008-20-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: 0.70
X-Spamd-Result: default: False [0.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Add some helpers for handling filenames which might need different
implementations between stubdom and daemon environments:

- expansion of relative filenames (those are not really defined today,
  just expand them to be relative to /var/lib/xen/xenstore)
- expansion of xenstore_daemon_rundir() (used e.g. for saving the state
  file in case of live update - needs to be unchanged in the daemon
  case, but should result in /var/lib/xen/xenstore for stubdom)

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- make absolute_filename() return a pointer to const (Julien Grall)
---
 tools/xenstored/core.c      | 15 +++++++++++++--
 tools/xenstored/core.h      |  5 ++++-
 tools/xenstored/lu_daemon.c |  4 ++--
 tools/xenstored/minios.c    |  5 +++++
 tools/xenstored/posix.c     |  8 +++++++-
 5 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 48fc787ac1..bada1ad9a2 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -63,7 +63,7 @@ char **orig_argv;
 LIST_HEAD(connections);
 int tracefd = -1;
 bool keep_orphans = false;
-char *tracefile = NULL;
+const char *tracefile = NULL;
 static struct hashtable *nodes;
 unsigned int trace_flags = TRACE_OBJ | TRACE_IO;
 
@@ -137,6 +137,17 @@ void trace_destroy(const void *data, const char *type)
 		trace("obj: DESTROY %s %p\n", type, data);
 }
 
+/*
+ * Return an absolute filename.
+ * In case of a relative filename given as input, prepend XENSTORE_LIB_DIR.
+ */
+const char *absolute_filename(const void *ctx, const char *filename)
+{
+	if (filename[0] != '/')
+		return talloc_asprintf(ctx, XENSTORE_LIB_DIR "/%s", filename);
+	return talloc_strdup(ctx, filename);
+}
+
 void close_log(void)
 {
 	if (tracefd >= 0)
@@ -2759,7 +2770,7 @@ int main(int argc, char *argv[])
 #endif
 
 	if (tracefile)
-		tracefile = talloc_strdup(NULL, tracefile);
+		tracefile = absolute_filename(NULL, tracefile);
 
 #ifndef NO_LIVE_UPDATE
 	/* Read state in case of live update. */
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index fe0ee90581..e58779e88c 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -341,7 +341,7 @@ void close_log(void);
 extern int orig_argc;
 extern char **orig_argv;
 
-extern char *tracefile;
+extern const char *tracefile;
 extern int tracefd;
 
 /* Trace flag values must be kept in sync with trace_switches[] contents. */
@@ -405,6 +405,9 @@ void set_socket_fd(int fd);
 void mount_9pfs(void);
 #endif
 
+const char *xenstore_rundir(void);
+const char *absolute_filename(const void *ctx, const char *filename);
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/lu_daemon.c b/tools/xenstored/lu_daemon.c
index 71bcabadd3..6351111ab0 100644
--- a/tools/xenstored/lu_daemon.c
+++ b/tools/xenstored/lu_daemon.c
@@ -24,7 +24,7 @@ void lu_get_dump_state(struct lu_dump_state *state)
 	state->size = 0;
 
 	state->filename = talloc_asprintf(NULL, "%s/state_dump",
-					  xenstore_daemon_rundir());
+					  xenstore_rundir());
 	if (!state->filename)
 		barf("Allocation failure");
 
@@ -65,7 +65,7 @@ FILE *lu_dump_open(const void *ctx)
 	int fd;
 
 	filename = talloc_asprintf(ctx, "%s/state_dump",
-				   xenstore_daemon_rundir());
+				   xenstore_rundir());
 	if (!filename)
 		return NULL;
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 562a9b4972..e70386f8c7 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -128,3 +128,8 @@ void mount_9pfs(void)
 {
 	create_thread("mount-9pfs", mount_thread, NULL);
 }
+
+const char *xenstore_rundir(void)
+{
+	return XENSTORE_LIB_DIR;
+}
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 496329dfd1..d88c82d972 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -326,9 +326,10 @@ void early_init(bool live_update, bool dofork, const char *pidfile)
 {
 	reopen_log();
 
-	/* Make sure xenstored directory exists. */
+	/* Make sure xenstored directories exist. */
 	/* Errors ignored here, will be reported when we open files */
 	mkdir(xenstore_daemon_rundir(), 0755);
+	mkdir(XENSTORE_LIB_DIR, 0755);
 
 	if (dofork) {
 		openlog("xenstored", 0, LOG_DAEMON);
@@ -406,3 +407,8 @@ void set_socket_fd(int fd)
 {
 	sock = fd;
 }
+
+const char *xenstore_rundir(void)
+{
+	return xenstore_daemon_rundir();
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:09:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:09:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681735.1060694 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUy-0002f2-DJ; Thu, 15 Feb 2024 13:09:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681735.1060694; Thu, 15 Feb 2024 13:09:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUy-0002d4-3Z; Thu, 15 Feb 2024 13:09:56 +0000
Received: by outflank-mailman (input) for mailman id 681735;
 Thu, 15 Feb 2024 13:09:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabRP-0002j3-F1
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:15 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fefd6f6b-cc02-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:06:13 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 5989922031;
 Thu, 15 Feb 2024 13:06:13 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 2B059139D0;
 Thu, 15 Feb 2024 13:06:13 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 9f4nCUUMzmWvFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fefd6f6b-cc02-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002373; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=U8pzJQi1IB2cDnOVCOc5xAPOt+AhORB1buKXKU5naO0=;
	b=J/JBlX4J7n2B15CvskZcQARsA1c7jI2Djn679BAZHKO0wEn52fkCoTxWqYV4Dz08ddNc9Y
	a1FxHqPoO8uzp4E/WnY+OECOsbwxrzwBRvT1jtJAlr971pixwqxQzeVaqQS5yOgh5zOZaZ
	Iy71AX5RPzhc8EflbEjfaqKpOJ/Iqkw=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002373; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=U8pzJQi1IB2cDnOVCOc5xAPOt+AhORB1buKXKU5naO0=;
	b=J/JBlX4J7n2B15CvskZcQARsA1c7jI2Djn679BAZHKO0wEn52fkCoTxWqYV4Dz08ddNc9Y
	a1FxHqPoO8uzp4E/WnY+OECOsbwxrzwBRvT1jtJAlr971pixwqxQzeVaqQS5yOgh5zOZaZ
	Iy71AX5RPzhc8EflbEjfaqKpOJ/Iqkw=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 11/21] tools/xen-9pfsd: add 9pfs stat request support
Date: Thu, 15 Feb 2024 14:04:59 +0100
Message-Id: <20240215130509.24008-12-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [1.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email,citrix.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: *
X-Spam-Score: 1.90
X-Spam-Flag: NO

Add the stat request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V3:
- use fstatat() (Jason Andryuk)
V4:
- add "s" format to fill_buffer() as a preparation for reading dirs
---
 tools/9pfsd/io.c | 102 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 8fa80865b4..8c36106d90 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -34,6 +34,7 @@
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
 #define P9_CMD_CLUNK      120
+#define P9_CMD_STAT       124
 
 /* P9 protocol open flags. */
 #define P9_OREAD            0   /* read */
@@ -60,6 +61,25 @@ struct p9_qid {
     uint64_t path;
 };
 
+struct p9_stat {
+    uint16_t size;
+    uint16_t type;
+    uint32_t dev;
+    struct p9_qid qid;
+    uint32_t mode;
+    uint32_t atime;
+    uint32_t mtime;
+    uint64_t length;
+    const char *name;
+    const char *uid;
+    const char *gid;
+    const char *muid;
+    const char *extension;
+    uint32_t n_uid;
+    uint32_t n_gid;
+    uint32_t n_muid;
+};
+
 /*
  * Note that the ring names "in" and "out" are from the frontend's
  * perspective, so the "in" ring will be used for responses to the frontend,
@@ -166,6 +186,7 @@ static void fmt_err(const char *fmt)
  * S: String (2 byte length + <length> characters)
  *    The length is obtained via strlen() of the parameter, being a pointer
  *    to the first character of the string
+ * s: stat (struct p9_stat)
  * U: 4 byte unsigned integer
  *    The parameter is a pointer to a uint32_t value
  */
@@ -176,6 +197,8 @@ static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
     const void *par;
     const char *str_val;
     const struct p9_qid *qid;
+    const struct p9_stat *stat;
+    uint16_t tlen;
     unsigned int len;
     unsigned int array_sz = 0;
     unsigned int elem_sz = 0;
@@ -259,6 +282,18 @@ static void vfill_buffer_at(void **data, const char *fmt, va_list ap)
             *data += len;
             break;
 
+        case 's':
+            stat = par;
+            elem_sz = sizeof(*stat);
+            tlen = stat->size + sizeof(stat->size);
+            fill_buffer_at(data, "uuuUQUUULSSSSSUUU", &tlen, &stat->size,
+                           &stat->type, &stat->dev, &stat->qid, &stat->mode,
+                           &stat->atime, &stat->mtime, &stat->length,
+                           stat->name, stat->uid, stat->gid, stat->muid,
+                           stat->extension, &stat->n_uid, &stat->n_gid,
+                           &stat->n_muid);
+            break;
+
         case 'U':
             put_unaligned(*(const uint32_t *)par, (uint32_t *)*data);
             elem_sz = sizeof(uint32_t);
@@ -1146,6 +1181,69 @@ static void p9_clunk(struct ring *ring, struct p9_header *hdr)
     fill_buffer(ring, hdr->cmd + 1, hdr->tag, "");
 }
 
+static void fill_p9_stat(device *device, struct p9_stat *p9s, struct stat *st,
+                         const char *name)
+{
+    memset(p9s, 0, sizeof(*p9s));
+    fill_qid(device, NULL, &p9s->qid, st);
+    p9s->mode = st->st_mode & 0777;
+    if ( S_ISDIR(st->st_mode) )
+        p9s->mode |= P9_CREATE_PERM_DIR;
+    p9s->atime = st->st_atime;
+    p9s->mtime = st->st_mtime;
+    p9s->length = st->st_size;
+    p9s->name = name;
+    p9s->uid = "";
+    p9s->gid = "";
+    p9s->muid = "";
+    p9s->extension = "";
+    p9s->n_uid = 0;
+    p9s->n_gid = 0;
+    p9s->n_muid = 0;
+
+    /*
+     * Size of individual fields without the size field, including 5 2-byte
+     * string length fields.
+     */
+    p9s->size = 71 + strlen(p9s->name);
+}
+
+static void p9_stat(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    struct p9_fid *fidp;
+    struct p9_stat p9s;
+    struct stat st;
+    int ret;
+
+    ret = fill_data(ring, "U", &fid);
+    if ( ret != 1 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+    fill_p9_stat(device, &p9s, &st, strrchr(fidp->path, '/') + 1);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "s", &p9s);
+
+ out:
+    free_fid(device, fidp);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1225,6 +1323,10 @@ void *io_thread(void *arg)
                 p9_clunk(ring, &hdr);
                 break;
 
+            case P9_CMD_STAT:
+                p9_stat(ring, &hdr);
+                break;
+
             default:
                 syslog(LOG_DEBUG, "%u.%u sent unhandled command %u\n",
                        ring->device->domid, ring->device->devid, hdr.cmd);
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:09:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:09:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681732.1060683 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUx-0002RA-Ky; Thu, 15 Feb 2024 13:09:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681732.1060683; Thu, 15 Feb 2024 13:09:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUx-0002QB-Fy; Thu, 15 Feb 2024 13:09:55 +0000
Received: by outflank-mailman (input) for mailman id 681732;
 Thu, 15 Feb 2024 13:09:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabRK-0002j3-7x
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:10 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fba55b8b-cc02-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:06:08 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id B87DB21FBC;
 Thu, 15 Feb 2024 13:06:07 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 8A0B9139D0;
 Thu, 15 Feb 2024 13:06:07 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id +n9ZID8MzmWkFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fba55b8b-cc02-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002367; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=IqwLddRteFeWRdtvf3aJAKcFAk5qC0k1ozwKLE3hJvs=;
	b=L71WoaIkbO/kGN6pa6MY7H4p9ml/R5jcHk89fIWMtI2ZSzAhckIvQCvQyCbU5LxLPRdjOq
	zUFS/S3Vr/uO5BPxyHB86L/ZMpcAAQu/ZKyinyvRFdT6p1dRb5Gdu+yaGCOlvPrsZZtSoK
	O1ipBjLnqSOvlYdo+cPcwcdW6BQ41Yk=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002367; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=IqwLddRteFeWRdtvf3aJAKcFAk5qC0k1ozwKLE3hJvs=;
	b=L71WoaIkbO/kGN6pa6MY7H4p9ml/R5jcHk89fIWMtI2ZSzAhckIvQCvQyCbU5LxLPRdjOq
	zUFS/S3Vr/uO5BPxyHB86L/ZMpcAAQu/ZKyinyvRFdT6p1dRb5Gdu+yaGCOlvPrsZZtSoK
	O1ipBjLnqSOvlYdo+cPcwcdW6BQ41Yk=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 10/21] tools/xen-9pfsd: add 9pfs create request support
Date: Thu, 15 Feb 2024 14:04:58 +0100
Message-Id: <20240215130509.24008-11-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [4.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email,citrix.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Level: ****
X-Spam-Score: 4.90
X-Spam-Flag: NO

Add the create request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
V2:
- set permissions correctly (Jason Andryuk)
V3:
- use opendirat() etc. (Jason Andryuk)
- rework error handling a little bit
---
 tools/9pfsd/io.c | 151 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 151 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 6af14e5ee9..8fa80865b4 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -32,6 +32,7 @@
 #define P9_CMD_ERROR      107
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
+#define P9_CMD_CREATE     114
 #define P9_CMD_CLUNK      120
 
 /* P9 protocol open flags. */
@@ -42,6 +43,12 @@
 #define P9_OTRUNC        0x10   /* or'ed in, truncate file first */
 #define P9_OREMOVE       0x40   /* or'ed in, remove file after clunk */
 
+/* P9 protocol create permission masks. */
+#define P9_CREATE_PERM_DIR        0x80000000
+#define P9_CREATE_PERM_NOTSUPP    0x03b00000   /* link, symlink, ... */
+#define P9_CREATE_PERM_DIR_MASK   0777
+#define P9_CREATE_PERM_FILE_MASK  0666
+
 #define P9_MIN_MSIZE      2048
 #define P9_VERSION        "9P2000.u"
 #define P9_WALK_MAXELEM   16
@@ -961,6 +968,146 @@ static void p9_open(struct ring *ring, struct p9_header *hdr)
     p9_error(ring, hdr->tag, errno);
 }
 
+static void p9_create(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    unsigned int name_off;
+    uint32_t perm;
+    uint8_t mode;
+    unsigned int ext_off;
+    struct p9_fid *fidp;
+    struct p9_fid *new_fidp;
+    char *path;
+    struct stat st;
+    struct p9_qid qid;
+    uint32_t iounit;
+    int flags;
+    int ret;
+
+    ret = fill_data(ring, "USUbS", &fid, &name_off, &perm, &mode, &ext_off);
+    if ( ret != 5 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    if ( !name_ok(ring->str + name_off) )
+    {
+        p9_error(ring, hdr->tag, ENOENT);
+        return;
+    }
+
+    if ( perm & P9_CREATE_PERM_NOTSUPP )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || fidp->opened )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, errno);
+        return;
+    }
+
+    path = malloc(strlen(fidp->path) + strlen(ring->str + name_off) + 2);
+    if ( !path )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, ENOMEM);
+        return;
+    }
+    sprintf(path, "%s/%s", fidp->path, ring->str + name_off);
+    new_fidp = alloc_fid_mem(device, fid, path);
+    free(path);
+    if ( !new_fidp )
+    {
+        free_fid(device, fidp);
+        p9_error(ring, hdr->tag, ENOMEM);
+        return;
+    }
+
+    pthread_mutex_lock(&device->fid_mutex);
+
+    new_fidp->ref = fidp->ref;
+
+    if ( perm & P9_CREATE_PERM_DIR )
+    {
+        perm &= P9_CREATE_PERM_DIR_MASK & st.st_mode;
+        if ( mode != P9_OREAD )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        if ( mkdirat(device->root_fd, new_fidp->path, perm) < 0 )
+            goto err;
+
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+        free(fidp);
+        fidp = new_fidp;
+        new_fidp = NULL;
+
+        fidp->fd = openat(device->root_fd, fidp->path, O_RDONLY);
+        if ( fidp->fd < 0 )
+            goto err;
+        fidp->data = fdopendir(fidp->fd);
+        if ( !fidp->data )
+            goto err;
+    }
+    else
+    {
+        flags = open_flags_from_mode(mode);
+        if ( flags < 0 )
+        {
+            errno = EINVAL;
+            goto err;
+        }
+        perm &= P9_CREATE_PERM_FILE_MASK & st.st_mode;
+
+        XEN_TAILQ_REMOVE(&device->fids, fidp, list);
+        XEN_TAILQ_INSERT_HEAD(&device->fids, new_fidp, list);
+        free(fidp);
+        fidp = new_fidp;
+        new_fidp = NULL;
+
+        fidp->fd = openat(device->root_fd, fidp->path, flags | O_CREAT | O_EXCL,
+                          perm);
+        if ( fidp->fd < 0 )
+            goto err;
+    }
+
+    if ( fstatat(device->root_fd, fidp->path, &st, 0) < 0 )
+        goto err;
+
+    fill_qid(device, fidp->path, &qid, &st);
+    iounit = get_iounit(ring, &st);
+    fidp->opened = true;
+    fidp->mode = mode;
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "QU", &qid, &iounit);
+
+    return;
+
+ err:
+    p9_error(ring, hdr->tag, errno);
+
+    pthread_mutex_unlock(&device->fid_mutex);
+
+    free(new_fidp);
+    free_fid(device, fidp);
+}
+
 static void p9_clunk(struct ring *ring, struct p9_header *hdr)
 {
     device *device = ring->device;
@@ -1070,6 +1217,10 @@ void *io_thread(void *arg)
                 p9_open(ring, &hdr);
                 break;
 
+            case P9_CMD_CREATE:
+                p9_create(ring, &hdr);
+                break;
+
             case P9_CMD_CLUNK:
                 p9_clunk(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:09:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:09:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681736.1060704 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUz-0002rc-20; Thu, 15 Feb 2024 13:09:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681736.1060704; Thu, 15 Feb 2024 13:09:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUy-0002pz-LQ; Thu, 15 Feb 2024 13:09:56 +0000
Received: by outflank-mailman (input) for mailman id 681736;
 Thu, 15 Feb 2024 13:09:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabRv-0002yD-Vq
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:47 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 130f66a3-cc03-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:06:47 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id EDA8C2218B;
 Thu, 15 Feb 2024 13:06:46 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id BD91D139D0;
 Thu, 15 Feb 2024 13:06:46 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id YX3WLGYMzmXdFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 130f66a3-cc03-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002407; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=t9xMwpWGoJR7DxJ7ry2LqpTZcYdrlw/DDzPveaCxkX4=;
	b=NWTV1O7+qMrKduemySPAi9pxEk50M8lXxylbztxDGFmIAC2mPVX9cfX8X4FcV0WQZV7yNT
	PptQ0VhwJusKelloetHOj+ZTb+loVj9DrhhfGjVmqsHXvZxslmKw5yo2QctiQNYxAIp2BB
	hSAJPpt6iyCPUmervLJZhEfGCtr0vq8=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002406; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=t9xMwpWGoJR7DxJ7ry2LqpTZcYdrlw/DDzPveaCxkX4=;
	b=mGVNtHN0KEiBdxt0vQFpibpCKnOCj+9fkMUNaoUMoljhy4NNRLLV9f5dhxtCRinqpzcq2n
	S8EH6Wmy1gG0pH2yjv2lEAv1sL49QzJv04v5V2K4HDcL8TYPlvZ9BSwvG/m8gRxSU2sWjd
	7jfmgz/QDs4Ez7QJH0DDc8usqZx3wdA=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v7 17/21] tools: add 9pfs device to xenstore-stubdom
Date: Thu, 15 Feb 2024 14:05:05 +0100
Message-Id: <20240215130509.24008-18-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: ***
X-Spamd-Bar: +++
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=mGVNtHN0
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [3.49 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 3.49
X-Rspamd-Queue-Id: EDA8C2218B
X-Spam-Flag: NO

Add a 9pfs device to Xenstore stubdom in order to allow it to do e.g.
logging into a dom0 file.

Use the following parameters for the new device:

- tag = "Xen"
- type = "xen_9pfsd"
- path = "/var/lib/xen/xenstore"
- security-model = "none"

For now don't limit allowed file space or number of files.

Add a new libxl function for adding it similar to the function for
adding the console device.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V2:
- add security_model parameter to new libxl function (Jason Andryuk)
V4:
- rename function to libxl_device_9pfs_add() (Anthony Perard)
- use a libxl_device_p9 pointer as parameter (Anthony Perard)
---
 tools/helpers/init-xenstore-domain.c |  7 +++++++
 tools/include/libxl.h                | 15 +++++++++++++++
 tools/libs/light/libxl_9pfs.c        | 16 ++++++++++++++++
 3 files changed, 38 insertions(+)

diff --git a/tools/helpers/init-xenstore-domain.c b/tools/helpers/init-xenstore-domain.c
index 140ed610ae..1683438c5c 100644
--- a/tools/helpers/init-xenstore-domain.c
+++ b/tools/helpers/init-xenstore-domain.c
@@ -433,6 +433,12 @@ int main(int argc, char** argv)
     int rv, fd;
     char *maxmem_str = NULL;
     libxl_ctx *ctx;
+    libxl_device_p9 p9 = { .backend_domid = 0,
+                           .tag = "Xen",
+                           .path = XEN_LIB_DIR"/xenstore",
+                           .security_model = "none",
+                           .type = LIBXL_P9_TYPE_XEN_9PFSD,
+    };
 
     while ( (opt = getopt_long(argc, argv, "v", options, NULL)) != -1 )
     {
@@ -543,6 +549,7 @@ int main(int argc, char** argv)
     }
     libxl_console_add_xenstore(ctx, domid, 0, console_evtchn, console_gfn,
                                NULL);
+    libxl_device_9pfs_add(ctx, domid, &p9, NULL);
     libxl_ctx_free(ctx);
 
     fd = creat(XEN_RUN_DIR "/xenstored.pid", 0666);
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 9a3e702557..44a2205d2b 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -583,6 +583,13 @@
  * libxl_console_add_xenstore() in libxl.
  */
 #define LIBXL_HAVE_CONSOLE_ADD_XENSTORE 1
+
+/*
+ * LIBXL_HAVE_P9_ADD_XENSTORE indicates presence of the function
+ * libxl_device_9pfs_add() in libxl.
+ */
+#define LIBXL_HAVE_P9_ADD_XENSTORE 1
+
 /*
  * libxl ABI compatibility
  *
@@ -2074,6 +2081,14 @@ int libxl_console_add_xenstore(libxl_ctx *ctx, uint32_t domid, uint32_t backend,
                                const libxl_asyncop_how *ao_how)
                                LIBXL_EXTERNAL_CALLERS_ONLY;
 
+/* libxl_device_9pfs_add writes the Xenstore entries for a domain's
+ * primary 9pfs device based on domid, and device parameters.
+ * If needed it will start the backend daemon.
+ */
+int libxl_device_9pfs_add(libxl_ctx *ctx, uint32_t domid, libxl_device_p9 *p9,
+                          const libxl_asyncop_how *ao_how)
+                          LIBXL_EXTERNAL_CALLERS_ONLY;
+
 /* May be called with info_r == NULL to check for domain's existence.
  * Returns ERROR_DOMAIN_NOTFOUND if domain does not exist (used to return
  * ERROR_INVAL for this scenario). */
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index ddeb4f20a7..48f894f070 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -206,6 +206,22 @@ static void libxl__device_p9_add(libxl__egc *egc, uint32_t domid,
         aodev->callback(egc, aodev);
 }
 
+int libxl_device_9pfs_add(libxl_ctx *ctx, uint32_t domid, libxl_device_p9 *p9,
+                          const libxl_asyncop_how *ao_how)
+{
+    AO_CREATE(ctx, domid, ao_how);
+    libxl__ao_device *aodev;
+
+    GCNEW(aodev);
+    libxl__prepare_ao_device(ao, aodev);
+    aodev->action = LIBXL__DEVICE_ACTION_ADD;
+    aodev->callback = device_addrm_aocomplete;
+
+    libxl__device_p9_add(egc, domid, p9, aodev);
+
+    return AO_INPROGRESS;
+}
+
 #define libxl_device_p9_list NULL
 #define libxl_device_p9_compare NULL
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:09:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:09:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681737.1060707 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUz-0002z3-Cw; Thu, 15 Feb 2024 13:09:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681737.1060707; Thu, 15 Feb 2024 13:09:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabUz-0002u1-20; Thu, 15 Feb 2024 13:09:57 +0000
Received: by outflank-mailman (input) for mailman id 681737;
 Thu, 15 Feb 2024 13:09:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabSI-0002yD-OM
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:07:10 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 20a66eb0-cc03-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:07:10 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id BBDC821FBC;
 Thu, 15 Feb 2024 13:07:09 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 7B051139EF;
 Thu, 15 Feb 2024 13:07:09 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id f+tTHH0MzmXxFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:07:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20a66eb0-cc03-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002429; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GGQ2M6ZDlC8Q8t5jjlaB/u1rTlT60Df0gemVV+Z+fCo=;
	b=PaWTl29bD7UBwuZ7U9ILETzd1RNJoYjgX1vV3RcGNRiNtH16bn+c97WnFrskzNo/znmat4
	WD6nIcchxCQt8p6QlFbidr/SnoUdlgbJyX7Ivkk0C6MxShazKbwE85bww7RVmFcvl2gTPW
	Rr5PsDTOALvCQhXHKuMk+EnV+YdktHE=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002429; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GGQ2M6ZDlC8Q8t5jjlaB/u1rTlT60Df0gemVV+Z+fCo=;
	b=PaWTl29bD7UBwuZ7U9ILETzd1RNJoYjgX1vV3RcGNRiNtH16bn+c97WnFrskzNo/znmat4
	WD6nIcchxCQt8p6QlFbidr/SnoUdlgbJyX7Ivkk0C6MxShazKbwE85bww7RVmFcvl2gTPW
	Rr5PsDTOALvCQhXHKuMk+EnV+YdktHE=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 21/21] tools/xenstored: have a single do_control_memreport()
Date: Thu, 15 Feb 2024 14:05:09 +0100
Message-Id: <20240215130509.24008-22-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spam-Level: ***
X-Spam-Score: 3.69
X-Spamd-Result: default: False [3.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[6];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.01)[45.67%]
X-Spam-Flag: NO

With 9pfs now available in Xenstore-stubdom, there is no reason to
have distinct do_control_memreport() variants for the daemon and the
stubdom implementations.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 tools/xenstored/control.c | 27 +++++++--------------------
 1 file changed, 7 insertions(+), 20 deletions(-)

diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index dae23a5ac0..9561289179 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -216,23 +216,11 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 	return 0;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
 	FILE *fp;
+	const char *filename;
 	int fd;
 
 	if (num > 1)
@@ -255,8 +243,12 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 			if (!fp)
 				close(fd);
 		}
-	} else
-		fp = fopen(vec[0], "a");
+	} else {
+		filename = absolute_filename(ctx, vec[0]);
+		if (!filename)
+			return ENOMEM;
+		fp = fopen(filename, "a");
+	}
 
 	if (!fp)
 		return EBADF;
@@ -267,7 +259,6 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
-#endif
 
 static int do_control_print(const void *ctx, struct connection *conn,
 			    const char **vec, int num)
@@ -310,11 +301,7 @@ static struct cmd_s cmds[] = {
 		"    Default timeout is 60 seconds.", 5 },
 #endif
 	{ "logfile", do_control_logfile, "<file>" },
-#ifdef __MINIOS__
-	{ "memreport", do_control_memreport, "" },
-#else
 	{ "memreport", do_control_memreport, "[<file>]" },
-#endif
 	{ "print", do_control_print, "<string>" },
 	{ "quota", do_control_quota,
 		"[set <name> <val>|<domid>|max [-r]]" },
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:10:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:10:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681745.1060737 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabV4-000483-2K; Thu, 15 Feb 2024 13:10:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681745.1060737; Thu, 15 Feb 2024 13:10:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabV3-00047q-UP; Thu, 15 Feb 2024 13:10:01 +0000
Received: by outflank-mailman (input) for mailman id 681745;
 Thu, 15 Feb 2024 13:10:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabRU-0002yD-2B
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:20 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0275e794-cc03-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:06:19 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 006A021FBC;
 Thu, 15 Feb 2024 13:06:19 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id B8FD1139D0;
 Thu, 15 Feb 2024 13:06:18 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id xeCvK0oMzmW0FQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0275e794-cc03-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=X2VtlH36Dxgp8+Z8DrH/UT3snGVBYS8nN8oGoKs6b2w=;
	b=KDATXNSOJfA0nBe69MLrQTUtsFGMQkggIkAsB8pNY0Tadg16Z429hgUvGCZ6Po5dorEnWb
	FJqoyL6m5tWPE7k39wAAttK7P8l0eMTkB2ed208Avr7RVdxJGjBXYNqpBBBkjKgdcEGA5J
	a6PkCmn6zwGFdF3UrUmxK5wTuabOLqQ=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=X2VtlH36Dxgp8+Z8DrH/UT3snGVBYS8nN8oGoKs6b2w=;
	b=KDATXNSOJfA0nBe69MLrQTUtsFGMQkggIkAsB8pNY0Tadg16Z429hgUvGCZ6Po5dorEnWb
	FJqoyL6m5tWPE7k39wAAttK7P8l0eMTkB2ed208Avr7RVdxJGjBXYNqpBBBkjKgdcEGA5J
	a6PkCmn6zwGFdF3UrUmxK5wTuabOLqQ=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 12/21] tools/xen-9pfsd: add 9pfs write request support
Date: Thu, 15 Feb 2024 14:05:00 +0100
Message-Id: <20240215130509.24008-13-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: ***
X-Spamd-Bar: +++
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=KDATXNSO
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [3.49 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-0.00)[19.25%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email,citrix.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 3.49
X-Rspamd-Queue-Id: 006A021FBC
X-Spam-Flag: NO

Add the write request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/9pfsd/io.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 8c36106d90..1832c6b06e 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -33,6 +33,7 @@
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
+#define P9_CMD_WRITE      118
 #define P9_CMD_CLUNK      120
 #define P9_CMD_STAT       124
 
@@ -1244,6 +1245,55 @@ static void p9_stat(struct ring *ring, struct p9_header *hdr)
     free_fid(device, fidp);
 }
 
+static void p9_write(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint64_t off;
+    unsigned int len;
+    uint32_t written;
+    void *buf;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "ULD", &fid, &off, &len, ring->buffer);
+    if ( ret != 3 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || !fidp->opened || fidp->isdir )
+    {
+        p9_error(ring, hdr->tag, EBADF);
+        goto out;
+    }
+
+    buf = ring->buffer;
+
+    while ( len != 0 )
+    {
+        ret = pwrite(fidp->fd, buf, len, off);
+        if ( ret < 0 )
+            break;
+        len -= ret;
+        buf += ret;
+        off += ret;
+    }
+
+    written = buf - ring->buffer;
+    if ( written == 0 )
+    {
+        p9_error(ring, hdr->tag, errno);
+        goto out;
+    }
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "U", &written);
+
+ out:
+    free_fid(device, fidp);
+}
+
 void *io_thread(void *arg)
 {
     struct ring *ring = arg;
@@ -1319,6 +1369,10 @@ void *io_thread(void *arg)
                 p9_create(ring, &hdr);
                 break;
 
+            case P9_CMD_WRITE:
+                p9_write(ring, &hdr);
+                break;
+
             case P9_CMD_CLUNK:
                 p9_clunk(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:10:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:10:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681746.1060747 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabV9-0005E6-Fn; Thu, 15 Feb 2024 13:10:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681746.1060747; Thu, 15 Feb 2024 13:10:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabV9-0005DY-AH; Thu, 15 Feb 2024 13:10:07 +0000
Received: by outflank-mailman (input) for mailman id 681746;
 Thu, 15 Feb 2024 13:10:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabRm-0002j3-5u
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:38 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0c6cbca3-cc03-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:06:36 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id D600021FBC;
 Thu, 15 Feb 2024 13:06:35 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 9B76F139D0;
 Thu, 15 Feb 2024 13:06:35 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id MOKXJFsMzmXCFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c6cbca3-cc03-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002395; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=jJ/RTfjmaHP3lqBxIj2S65NW4z+aqqiAB78+0Fnq5Fc=;
	b=hWcRbB4JuTMVGtIaoSBUy3a/+xeZfBrkALT1RXfzlaVEiY4Am8RNmNdF8EyTmMaqoKQh1q
	jMynNj7ZqXSoFAMcH5zDMJtlupoQFCdKcV20v6EaqKHcaPE2nPuaqo/eYypMWHyDHxK5g+
	adFkVn+UTZ8EcqK8s37U/r096xYYasY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002395; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=jJ/RTfjmaHP3lqBxIj2S65NW4z+aqqiAB78+0Fnq5Fc=;
	b=hWcRbB4JuTMVGtIaoSBUy3a/+xeZfBrkALT1RXfzlaVEiY4Am8RNmNdF8EyTmMaqoKQh1q
	jMynNj7ZqXSoFAMcH5zDMJtlupoQFCdKcV20v6EaqKHcaPE2nPuaqo/eYypMWHyDHxK5g+
	adFkVn+UTZ8EcqK8s37U/r096xYYasY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 15/21] tools/xl: support new 9pfs backend xen_9pfsd
Date: Thu, 15 Feb 2024 14:05:03 +0100
Message-Id: <20240215130509.24008-16-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=hWcRbB4J
X-Spamd-Result: default: False [1.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 DWL_DNSWL_BLOCKED(0.00)[suse.com:dkim];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[citrix.com:email,suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 1.69
X-Rspamd-Queue-Id: D600021FBC
X-Spam-Level: *
X-Spam-Flag: NO
X-Spamd-Bar: +

Add support for the new 9pfs backend "xen_9pfsd". For this backend type
the tag defaults to "Xen" and the host side path to
"/var/log/xen/guests/<dom-name>".

Do most of the default settings in libxl. Unfortunately the default
path can't easily be set in libxl, as the domain name isn't available
in the related 9pfs specific function.

Settings the defaults in libxl requires to move the sanity checking
of 9pfs parameters from xl to libxl, too.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- test max_files and max_open_files, too (Jason Andryuk)
V4:
- fix man page to use the "xen_9pfsd" type due to idl limitation
  (Jason Andryuk)
- set (most of) the defaults in libxl (Anthony Perard)
---
 docs/man/xl.cfg.5.pod.in      | 36 +++++++++++++++++++++++++++++++++--
 tools/libs/light/libxl_9pfs.c | 18 ++++++++++++++++++
 tools/xl/xl_parse.c           | 23 +++++++++++++++++++---
 3 files changed, 72 insertions(+), 5 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index ea8d41727d..039e057318 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -772,10 +772,16 @@ settings, from the following list:
 
 =over 4
 
+=item B<type=TYPE>
+
+The backendtype for the PV device. Supported values are B<qemu> and
+B<xen_9pfsd>.  The default is B<qemu>.
+
 =item B<tag=STRING>
 
 9pfs tag to identify the filesystem share. The tag is needed on the
-guest side to mount it.
+guest side to mount it. For the backendtype of B<xen_9pfsd> the tag defaults to
+"Xen".
 
 =item B<security_model="none">
 
@@ -785,12 +791,38 @@ squash or remap).
 
 =item B<path=STRING>
 
-Filesystem path on the backend to export.
+Filesystem path on the backend to export. For the backendtype of B<xen_9pfsd>
+the path defaults to "@XEN_LOG_DIR@/guests/<guest-name>".
 
 =item B<backend=domain-id>
 
 Specify the backend domain name or id, defaults to dom0.
 
+=item B<max-files=NUMBER>
+
+Specify the maximum number of files below B<path>. A value of 0 (which
+is the default) doesn't limit the number of files. Only valid for
+B<type=xen_9pfsd>.
+
+=item B<max-open-files=NUMBER>
+
+Specify the maximum number of concurrently opened files below B<path>.
+Multiple opens of the same file are counted individually. Only valid for
+B<type=xen_9pfsd>, which has a default of B<max-open-files=5>.
+
+=item B<max-space=NUMBER>
+
+Specify the maximum used disk space in MiB below B<path>. A value of 0 (which
+is the default) doesn't limit the usable disk space. Only valid for
+B<type=xen_9pfsd>.
+
+=item B<auto-delete=BOOLEAN>
+
+When set the backend will delete the oldest file which is currently not
+opened by the guest in case the disk space limit set via B<max-space> or the
+file limit set via B<max-files> is being reached. Only valid for
+B<type=xen_9pfsd>.
+
 =back
 
 =item B<pvcalls=[ "backend=domain-id", ... ]>
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 900c0d46a0..ddeb4f20a7 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -20,6 +20,24 @@
 static int libxl__device_p9_setdefault(libxl__gc *gc, uint32_t domid,
                                        libxl_device_p9 *p9, bool hotplug)
 {
+    if (p9->type == LIBXL_P9_TYPE_UNKNOWN) {
+        p9->type = LIBXL_P9_TYPE_QEMU;
+    }
+    if (p9->type == LIBXL_P9_TYPE_QEMU &&
+        (p9->max_files || p9->max_open_files || p9->max_space ||
+         p9->auto_delete)) {
+        LOGD(ERROR, domid, "Illegal 9pfs parameter combination");
+        return ERROR_INVAL;
+    }
+    if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD && !p9->tag) {
+        p9->tag = libxl__strdup(NOGC, "Xen");
+    }
+
+    if (!p9->path || !p9->security_model || !p9->tag) {
+        LOGD(ERROR, domid, "9pfs spec missing required field!");
+        return ERROR_INVAL;
+    }
+
     return libxl__resolve_domid(gc, p9->backend_domname, &p9->backend_domid);
 }
 
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 9b358f11b8..80ffe85f5e 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2233,6 +2233,20 @@ void parse_config_data(const char *config_source,
                     replace_string(&p9->tag, value);
                 } else if (!strcmp(key, "backend")) {
                     replace_string(&p9->backend_domname, value);
+                } else if (!strcmp(key, "type")) {
+                    if (libxl_p9_type_from_string(value, &p9->type)) {
+                        fprintf(stderr, "failed to parse 9pfs type: %s\n",
+                                value);
+                        exit(1);
+                    }
+                } else if (!strcmp(key, "max-files")) {
+                    p9->max_files = parse_ulong(value);
+                } else if (!strcmp(key, "max-open-files")) {
+                    p9->max_open_files = parse_ulong(value);
+                } else if (!strcmp(key, "max-space")) {
+                    p9->max_space = parse_ulong(value);
+                } else if (!strcmp(key, "auto-delete")) {
+                    p9->auto_delete = strtoul(value, NULL, 0);
                 } else {
                     fprintf(stderr, "Unknown 9pfs parameter '%s'\n", key);
                     exit(1);
@@ -2243,9 +2257,12 @@ void parse_config_data(const char *config_source,
 
             libxl_string_list_dispose(&pairs);
 
-            if (!p9->path || !p9->security_model || !p9->tag) {
-                fprintf(stderr, "9pfs spec missing required field!\n");
-                exit(1);
+            if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD && !p9->path) {
+                char *path;
+
+                xasprintf(&path, XEN_LOG_DIR "/guests/%s", c_info->name);
+                replace_string(&p9->path, path);
+                free(path);
             }
         }
     }
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:10:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:10:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681765.1060756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabVb-0007ae-1v; Thu, 15 Feb 2024 13:10:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681765.1060756; Thu, 15 Feb 2024 13:10:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabVa-0007aV-VH; Thu, 15 Feb 2024 13:10:34 +0000
Received: by outflank-mailman (input) for mailman id 681765;
 Thu, 15 Feb 2024 13:10:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabRZ-0002yD-P0
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:25 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 05b67721-cc03-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 14:06:25 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 8E16D22031;
 Thu, 15 Feb 2024 13:06:24 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 5DF61139D0;
 Thu, 15 Feb 2024 13:06:24 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 4UaHFVAMzmW3FQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 05b67721-cc03-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002384; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=87vA+v/yqSIw13sXcaxXTXhvJpR4gCj67AXmDFAMYsw=;
	b=HD/PUmo7iajzRJRmv5GvMx4aXD/9HOxHWCNarSFpPIqNhxjvPnMNSkn+h7FVMT+zzl4Kel
	BNSF9ruffO9492OvZlDnu9So4OiRXtHR1CgyRHs+ennHo7Ai+HHqDp7mfEgQo9YaUMHYOI
	YLrBAFdEQNvYkyTQTmq9K+BFK+8c7ms=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002384; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=87vA+v/yqSIw13sXcaxXTXhvJpR4gCj67AXmDFAMYsw=;
	b=HD/PUmo7iajzRJRmv5GvMx4aXD/9HOxHWCNarSFpPIqNhxjvPnMNSkn+h7FVMT+zzl4Kel
	BNSF9ruffO9492OvZlDnu9So4OiRXtHR1CgyRHs+ennHo7Ai+HHqDp7mfEgQo9YaUMHYOI
	YLrBAFdEQNvYkyTQTmq9K+BFK+8c7ms=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v7 13/21] tools/xen-9pfsd: add 9pfs read request support
Date: Thu, 15 Feb 2024 14:05:01 +0100
Message-Id: <20240215130509.24008-14-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b="HD/PUmo7"
X-Spamd-Result: default: False [4.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 DWL_DNSWL_BLOCKED(0.00)[suse.com:dkim];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email,citrix.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[20.42%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 4.69
X-Rspamd-Queue-Id: 8E16D22031
X-Spam-Level: ****
X-Spam-Flag: NO
X-Spamd-Bar: ++++

Add the read request of the 9pfs protocol.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- make error check more readable (Jason Andryuk)
V4:
- add directory read support
V5:
- rewinddir() if reading a directory and offset is 0 (Jason Andryuk)
---
 tools/9pfsd/io.c | 93 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 93 insertions(+)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index 1832c6b06e..ba2d4a436e 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -33,6 +33,7 @@
 #define P9_CMD_WALK       110
 #define P9_CMD_OPEN       112
 #define P9_CMD_CREATE     114
+#define P9_CMD_READ       116
 #define P9_CMD_WRITE      118
 #define P9_CMD_CLUNK      120
 #define P9_CMD_STAT       124
@@ -1245,6 +1246,94 @@ static void p9_stat(struct ring *ring, struct p9_header *hdr)
     free_fid(device, fidp);
 }
 
+static void p9_read(struct ring *ring, struct p9_header *hdr)
+{
+    device *device = ring->device;
+    uint32_t fid;
+    uint64_t off;
+    unsigned int len;
+    uint32_t count;
+    void *buf;
+    struct p9_fid *fidp;
+    int ret;
+
+    ret = fill_data(ring, "ULU", &fid, &off, &count);
+    if ( ret != 3 )
+    {
+        p9_error(ring, hdr->tag, EINVAL);
+        return;
+    }
+
+    fidp = get_fid_ref(device, fid);
+    if ( !fidp || !fidp->opened )
+    {
+        errno = EBADF;
+        goto err;
+    }
+
+    len = count;
+    buf = ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
+
+    if ( fidp->isdir )
+    {
+        struct dirent *dirent;
+        struct stat st;
+        struct p9_stat p9s;
+
+        if ( off == 0 )
+            rewinddir(fidp->data);
+
+        while ( len != 0 )
+        {
+            errno = 0;
+            dirent = readdir(fidp->data);
+            if ( !dirent )
+            {
+                if ( errno )
+                    goto err;
+                break;
+            }
+            if ( fstatat(fidp->fd, dirent->d_name, &st, 0) < 0 )
+                goto err;
+            fill_p9_stat(device, &p9s, &st, dirent->d_name);
+            if ( p9s.size + sizeof(p9s.size) > len )
+            {
+                seekdir(fidp->data, dirent->d_off);
+                break;
+            }
+            fill_buffer_at(&buf, "s", &p9s);
+            len -= p9s.size + sizeof(p9s.size);
+        }
+    }
+    else
+    {
+        while ( len != 0 )
+        {
+            ret = pread(fidp->fd, buf, len, off);
+            if ( ret <= 0 )
+                break;
+            len -= ret;
+            buf += ret;
+            off += ret;
+        }
+        if ( ret < 0 && len == count )
+            goto err;
+    }
+
+    buf = ring->buffer + sizeof(*hdr) + sizeof(uint32_t);
+    len = count - len;
+    fill_buffer(ring, hdr->cmd + 1, hdr->tag, "D", &len, buf);
+
+ out:
+    free_fid(device, fidp);
+
+    return;
+
+ err:
+    p9_error(ring, hdr->tag, errno);
+    goto out;
+}
+
 static void p9_write(struct ring *ring, struct p9_header *hdr)
 {
     device *device = ring->device;
@@ -1369,6 +1458,10 @@ void *io_thread(void *arg)
                 p9_create(ring, &hdr);
                 break;
 
+            case P9_CMD_READ:
+                p9_read(ring, &hdr);
+                break;
+
             case P9_CMD_WRITE:
                 p9_write(ring, &hdr);
                 break;
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:10:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:10:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681767.1060767 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabVe-0007tg-A3; Thu, 15 Feb 2024 13:10:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681767.1060767; Thu, 15 Feb 2024 13:10:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabVe-0007tZ-6d; Thu, 15 Feb 2024 13:10:38 +0000
Received: by outflank-mailman (input) for mailman id 681767;
 Thu, 15 Feb 2024 13:10:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rabS2-0002j3-Of
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:06:54 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 167399cd-cc03-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:06:53 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id AE2561F8A4;
 Thu, 15 Feb 2024 13:06:52 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 61994139D0;
 Thu, 15 Feb 2024 13:06:52 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id KqoqFmwMzmXgFQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 13:06:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 167399cd-cc03-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002412; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GuKS5JMiiFYV782dbcVdzQvPQKcEtbDic1ZOTvla/ok=;
	b=kMLckWSFMkQtF6U6N8FrxhjleNVeWZgPEz8ruSgoyl1jSP1DgC53AHZ2HLN+hkBQVp6F30
	qTX4MDM4DN0nPD/+frTaYS+7m4+deVLbHPpj3nnOf3GS7Y+AlBs9bF8AsTLS6hwheSJa24
	2vFPHRGzssljphMjQnmavCaJHsf7r8U=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708002412; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GuKS5JMiiFYV782dbcVdzQvPQKcEtbDic1ZOTvla/ok=;
	b=kMLckWSFMkQtF6U6N8FrxhjleNVeWZgPEz8ruSgoyl1jSP1DgC53AHZ2HLN+hkBQVp6F30
	qTX4MDM4DN0nPD/+frTaYS+7m4+deVLbHPpj3nnOf3GS7Y+AlBs9bF8AsTLS6hwheSJa24
	2vFPHRGzssljphMjQnmavCaJHsf7r8U=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v7 18/21] tools/xenstored: mount 9pfs device in stubdom
Date: Thu, 15 Feb 2024 14:05:06 +0100
Message-Id: <20240215130509.24008-19-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240215130509.24008-1-jgross@suse.com>
References: <20240215130509.24008-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=kMLckWSF
X-Spamd-Result: default: False [1.69 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 DWL_DNSWL_BLOCKED(0.00)[suse.com:dkim];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: 1.69
X-Rspamd-Queue-Id: AE2561F8A4
X-Spam-Level: *
X-Spam-Flag: NO
X-Spamd-Bar: +

Mount the 9pfs device in stubdom enabling it to use files.

This has to happen in a worker thread in order to allow the main thread
handling the required Xenstore accesses in parallel.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- add logging in case of errors (Julien Grall)
---
 tools/xenstored/core.h   |  6 +++++
 tools/xenstored/domain.c |  2 ++
 tools/xenstored/minios.c | 54 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+)

diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index f6af086f01..fe0ee90581 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -36,6 +36,8 @@
 #include "list.h"
 #include "hashtable.h"
 
+#define XENSTORE_LIB_DIR	XEN_LIB_DIR "/xenstore"
+
 #ifndef O_CLOEXEC
 #define O_CLOEXEC 0
 /* O_CLOEXEC support is needed for Live Update in the daemon case. */
@@ -399,6 +401,10 @@ void handle_special_fds(void);
 int get_socket_fd(void);
 void set_socket_fd(int fd);
 
+#ifdef __MINIOS__
+void mount_9pfs(void);
+#endif
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index 1a7d5e9756..64c8fd0cc3 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -1236,6 +1236,8 @@ void stubdom_init(void)
 		barf_perror("Failed to initialize stubdom");
 
 	xenevtchn_notify(xce_handle, stubdom->port);
+
+	mount_9pfs();
 #endif
 }
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 22ac8defbd..562a9b4972 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -17,10 +17,20 @@
 */
 #include <sys/types.h>
 #include <sys/mman.h>
+#include <syslog.h>
+#include "talloc.h"
 #include "core.h"
 #include "utils.h"
 #include <xen/grant_table.h>
 #include <mini-os/lib.h>
+#include <mini-os/9pfront.h>
+#include <mini-os/sched.h>
+#include <mini-os/xenbus.h>
+#include <mini-os/xmalloc.h>
+
+#define P9_STATE_PATH	"device/9pfs/0/state"
+
+static void *p9_device;
 
 void finish_daemonize(void)
 {
@@ -74,3 +84,47 @@ int get_socket_fd(void)
 void set_socket_fd(int fd)
 {
 }
+
+static void mount_thread(void *p)
+{
+	xenbus_event_queue events = NULL;
+	char *err;
+	char *dummy;
+
+	err = xenbus_watch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs", &events);
+	if (err) {
+		log("error \"%s\" when setting watch on \"%s\"\n", err,
+		    P9_STATE_PATH);
+		free(err);
+		return;
+	}
+
+	for (;;) {
+		xenbus_wait_for_watch(&events);
+
+		/*
+		 * We only care for existence of the state node.
+		 * State changes are handled in init_9pfront().
+		 */
+		err = xenbus_read(XBT_NIL, P9_STATE_PATH, &dummy);
+		if (!err)
+			break;
+		free(err);
+	}
+
+	free(dummy);
+
+	err = xenbus_unwatch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs");
+	if (err) {
+		log("error \"%s\" when unwatching \"%s\", leaking watch\n",
+		    err, P9_STATE_PATH);
+		free(err);
+	}
+
+	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+}
+
+void mount_9pfs(void)
+{
+	create_thread("mount-9pfs", mount_thread, NULL);
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:41:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:41:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681813.1060777 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabzd-00089X-M0; Thu, 15 Feb 2024 13:41:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681813.1060777; Thu, 15 Feb 2024 13:41:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rabzd-00089Q-I2; Thu, 15 Feb 2024 13:41:37 +0000
Received: by outflank-mailman (input) for mailman id 681813;
 Thu, 15 Feb 2024 13:41:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=G1Z8=JY=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rabzc-00089K-BN
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:41:36 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ee70022f-cc07-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:41:33 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a3da5bf7f9cso61225266b.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 05:41:33 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 n13-20020a170906088d00b00a3d2f55bc2esm546073eje.161.2024.02.15.05.41.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 05:41:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ee70022f-cc07-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708004493; x=1708609293; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=T5SfKdVPXndjEJEfVT6v0PanudGXm19wJIc9RSghH4I=;
        b=CS82GIxtYJzRtOnsfE7euASlDa6YQMFSgl3aW8eGao/PHz+pvENJhZIQl++oqWSXBD
         AFXTWp7xO7wwkZiyUC82WDOGwQ+cdC4BdF0SUtGpDynfW7TgvEOG5C0uZ8DGiwmeTzOz
         bgXbL/VCqcEeEPO9ZPQKkPA7EbPL2dE3V1V4+j0QzVB/hcF/s9CHuRm6oLSlP1qjWiLh
         7PIGGRCYO/b5ybrUThdX0jCp379IRWHuaGdBUxdoyUn3cL7JcZhl4DDgAc1boNn4bH8k
         vKX2j2K25OHqlWEZeLbGdS6gROkSVTQYOjiztJ/OKG6rPYD/yegd8HSLBM3T28ui8EE/
         Y6dw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708004493; x=1708609293;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=T5SfKdVPXndjEJEfVT6v0PanudGXm19wJIc9RSghH4I=;
        b=dTpQbebwKHkV1HHtMpte5S+YBue5iivRSghA60aAhQv9CMmELQm5abPUdTYwq7yiw+
         mv6m0rBvB1hFOIEOU0LKYouDV922g5Kfw5chetyF4k9dizuy0I00mWJM31qpPsS0x5pJ
         vSSN7FAmanFDMqKFJCpsP4EP5eK2U4sQ6+yGTgoxM1frobFXZWrj8YMtE+Svf2iu1m3w
         mZ1zGFdbuNVOTNnXOJljJsAR40DlHBh6byitC9XBOfZKdWvdzpcH4hl51OWkv2paiXgf
         yaGmF1GN+ID9i4Q65xuBU5XBj7PVXpTXgt++O4eA5lWPWRHbTecGgJI55u77TyCRl76/
         IONg==
X-Forwarded-Encrypted: i=1; AJvYcCU23ye6dQl0lnN+n9uXcDlBdXolrBdRLB7XpMf/i+nhSHO+vSW92ps8cHsfg1+Ay9rJmLvaw03NJfURxfUE5WvO5fSxUP1Yb4gQxnKW56o=
X-Gm-Message-State: AOJu0YwbRwx5QEe1J3GGNniruVAJR/5lg9g2iEcR4yZ62yEVA3nDuuKf
	iIWqk8cXgSmUFXXoxgwUFDQgVOwk7PdJxbwQM5nC33j3BVjvaTIe
X-Google-Smtp-Source: AGHT+IHsix9v3lLgCfSz8k3QYchAIfRy3on6asWIKp0wqkHHYXR45obzOeQnN3NfswppfMwqtPa89Q==
X-Received: by 2002:a17:906:855:b0:a3c:d5b2:f53 with SMTP id f21-20020a170906085500b00a3cd5b20f53mr1177819ejd.15.1708004492968;
        Thu, 15 Feb 2024 05:41:32 -0800 (PST)
Message-ID: <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Thu, 15 Feb 2024 14:41:31 +0100
In-Reply-To: <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
	 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "=3Dr" (ret), "+A" (*ptr)=
 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "r" (new) \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "memory" ); \
> > +})
> > +
> > +#define emulate_xchg_1_2(ptr, new, ret, release_barrier,
> > acquire_barrier) \
> > +({ \
> > +=C2=A0=C2=A0=C2=A0 uint32_t *ptr_32b_aligned =3D (uint32_t *)ALIGN_DOW=
N((unsigned
> > long)ptr, 4); \
>=20
> You now appear to assume that this macro is only used with inputs not
> crossing word boundaries. That's okay as long as suitably guaranteed
> at the use sites, but imo wants saying in a comment.
>=20
> > +=C2=A0=C2=A0=C2=A0 uint8_t mask_l =3D ((unsigned long)(ptr) & (0x8 - s=
izeof(*ptr)))
> > * BITS_PER_BYTE; \
>=20
> Why 0x8 (i.e. spanning 64 bits), not 4 (matching the uint32_t use
> above)?
The idea to read 8 bytes was to deal with crossing word boundary. So if
our address is 0x3 and we have to xchg() 2 bytes, what will cross 4
byte boundary. Instead we align add 0x3, so it will become 0x0 and then
just always work with 8 bytes.

>=20
> > +=C2=A0=C2=A0=C2=A0 unsigned long new_ =3D (unsigned long)(new) << mask=
_l; \
> > +=C2=A0=C2=A0=C2=A0 unsigned long ret_; \
> > +=C2=A0=C2=A0=C2=A0 unsigned long rc; \
>=20
> Similarly, why unsigned long here?
sizeof(unsigned long) is 8 bytes and it was chosen as we are working
with lc/sc.d which are working with 8 bytes.

>=20
> I also wonder about the mix of underscore suffixed (or not) variable
> names here.
If the question about ret_, then the same as before size of ret
argument of the macros will be 1 or 2, but {lc/sc}.d expected to work
with 8 bytes.

>=20
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 release_barrier \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "0: lr.d %0, %2\n" \
>=20
> Even here it's an 8-byte access. Even if - didn't check - the insn
> was
> okay to use with just a 4-byte aligned pointer, wouldn't it make
> sense
> then to 8-byte align it, and be consistent throughout this macro wrt
> the base unit acted upon? Alternatively, why not use lr.w here, thus
> reducing possible collisions between multiple CPUs accessing the same
> cache line?
According to the docs:
LR and SC operate on naturally-aligned 64-bit (RV64 only) or 32-bit
words in memory. Misaligned
addresses will generate misaligned address exceptions.

My intention was to deal with 4-byte crossing boundary. so if ptr is 4-
byte aligned then by reading 8-bytes we shouldn't care about boundary
crossing, if I am not missing something.

But your opinion about reduction of collisions makes sense also...

>=20
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "=C2=A0=C2=A0 and=C2=A0 %1,=
 %0, %z4\n" \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "=C2=A0=C2=A0 or=C2=A0=C2=
=A0 %1, %1, %z3\n" \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "=C2=A0=C2=A0 sc.d %1, %1, =
%2\n" \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "=C2=A0=C2=A0 bnez %1, 0b\n=
" \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 acquire_barrier \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "=3D&r" (ret_), "=3D&r" (=
rc), "+A" (*ptr_32b_aligned) \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "rJ" (new_), "rJ" (~mask)=
 \
>=20
> I think that as soon as there are more than 2 or maybe 3 operands,
> legibility is vastly improved by using named asm() operands.
Just to clarify you mean that it would be better to use instead of %0
use names?

>=20
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "memory"); \
>=20
> Nit: Missing blank before closing parenthesis.
>=20
> > +=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 ret =3D (__typeof__(*(ptr)))((ret_ & mask) >> mask_=
l); \
> > +})
>=20
> Why does "ret" need to be a macro argument? If you had only the
> expression here, not the the assigment, ...
>=20
> > +#define __xchg_generic(ptr, new, size, sfx, release_barrier,
> > acquire_barrier) \
> > +({ \
> > +=C2=A0=C2=A0=C2=A0 __typeof__(ptr) ptr__ =3D (ptr); \
>=20
> Is this local variable really needed? Can't you use "ptr" directly
> in the three macro invocations?
>=20
> > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) new__ =3D (new); \
> > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) ret__; \
> > +=C2=A0=C2=A0=C2=A0 switch (size) \
> > +=C2=A0=C2=A0=C2=A0 { \
> > +=C2=A0=C2=A0=C2=A0 case 1: \
> > +=C2=A0=C2=A0=C2=A0 case 2: \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 emulate_xchg_1_2(ptr__, new=
__, ret__, release_barrier,
> > acquire_barrier); \
>=20
> ... this would become
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ret__ =3D emulate_xchg_1_2(ptr=
__, new__, release_barrier,
> acquire_barrier); \
>=20
> But, unlike assumed above, there's no enforcement here that a 2-byte
> quantity won't cross a word, double-word, cache line, or even page
> boundary. That might be okay if then the code would simply crash
> (like
> the AMO insns emitted further down would), but aiui silent
> misbehavior
> would result.
As I mentioned above with 4-byte alignment and then reading and working
with 8-byte then crossing a word or double-word boundary shouldn't be
an issue.

I am not sure that I know how to check that we are crossing cache line
boundary.

Regarding page boundary, if the next page is mapped then all should
work fine, otherwise it will be an exception.

>=20
> Also nit: The switch() higher up is (still/again) missing blanks.
>=20
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 break; \
> > +=C2=A0=C2=A0=C2=A0 case 4: \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __amoswap_generic(ptr__, ne=
w__, ret__,\
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 ".w" sfx,=C2=A0 release_barrier,
> > acquire_barrier); \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 break; \
> > +=C2=A0=C2=A0=C2=A0 case 8: \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __amoswap_generic(ptr__, ne=
w__, ret__,\
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 ".d" sfx,=C2=A0 release_barrier,
> > acquire_barrier); \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 break; \
> > +=C2=A0=C2=A0=C2=A0 default: \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 STATIC_ASSERT_UNREACHABLE()=
; \
> > +=C2=A0=C2=A0=C2=A0 } \
> > +=C2=A0=C2=A0=C2=A0 ret__; \
> > +})
> > +
> > +#define xchg_relaxed(ptr, x) \
> > +({ \
> > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) x_ =3D (x); \
> > +=C2=A0=C2=A0=C2=A0 (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(=
*(ptr)),
> > "", "", ""); \
> > +})
> > +
> > +#define xchg_acquire(ptr, x) \
> > +({ \
> > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) x_ =3D (x); \
> > +=C2=A0=C2=A0=C2=A0 (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(=
*(ptr)), \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 "", "",
> > RISCV_ACQUIRE_BARRIER); \
> > +})
> > +
> > +#define xchg_release(ptr, x) \
> > +({ \
> > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) x_ =3D (x); \
> > +=C2=A0=C2=A0=C2=A0 (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(=
*(ptr)),\
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 "", RISCV_RELEASE_BARRIER,
> > ""); \
> > +})
> > +
> > +#define xchg(ptr,x) \
> > +({ \
> > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) ret__; \
> > +=C2=A0=C2=A0=C2=A0 ret__ =3D (__typeof__(*(ptr))) \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __x=
chg_generic(ptr, (unsigned long)(x),
> > sizeof(*(ptr)), \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 ".aqrl", "", ""); \
>=20
> The .aqrl doesn't look to affect the (emulated) 1- and 2-byte cases.
>=20
> Further, amoswap also exists in release-only and acquire-only forms.
> Why do you prefer explicit barrier insns over those? (Looks to
> similarly apply to the emulation path as well as to the cmpxchg
> machinery then, as both lr and sc also come in all four possible
> acquire/release forms. Perhaps for the emulation path using
> explicit barriers is better, in case the acquire/release forms of
> lr/sc - being used inside the loop - might perform worse.)
As 1- and 2-byte cases are emulated I decided that is not to provide
sfx argument for emulation macros as it will not have to much affect on
emulated types and just consume more performance on acquire and release
version of sc/ld instructions.


> >=20
>=20
> No RISCV_..._BARRIER for use here and ...
>=20
> > +=C2=A0=C2=A0=C2=A0 ret__; \
> > +})
> > +
> > +#define __cmpxchg(ptr, o, n, s) \
> > +({ \
> > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) ret__; \
> > +=C2=A0=C2=A0=C2=A0 ret__ =3D (__typeof__(*(ptr))) \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __c=
mpxchg_generic(ptr, (unsigned long)(o), (unsigned
> > long)(n), \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s, ".rl", "", " fence rw, rw\n"); \
>=20
> ... here? And anyway, wouldn't it make sense to have
>=20
> #define cmpxchg(ptr, o, n) __cmpxchg(ptr, o, n, sizeof(*(ptr))
>=20
> to limit redundancy?
>=20
> Plus wouldn't
>=20
> #define __cmpxchg(ptr, o, n, s) \
> =C2=A0=C2=A0=C2=A0 ((__typeof__(*(ptr))) \
> =C2=A0=C2=A0=C2=A0=C2=A0 __cmpxchg_generic(ptr, (unsigned long)(o), (unsi=
gned long)(n), \
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s, ".rl", "", =
" fence rw, rw\n"))
>=20
> be shorter and thus easier to follow as well? As I notice only now,
> this would apparently apply further up as well.
I understand your point about "#define cmpxchg(ptr, o, n) __cmpxchg(",
but I can't undestand how the definition of __cmxchng should be done
shorter. Could you please clarify that?

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 13:53:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 13:53:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681820.1060786 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1racB3-0003Dn-Q5; Thu, 15 Feb 2024 13:53:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681820.1060786; Thu, 15 Feb 2024 13:53:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1racB3-0003Dg-NP; Thu, 15 Feb 2024 13:53:25 +0000
Received: by outflank-mailman (input) for mailman id 681820;
 Thu, 15 Feb 2024 13:53:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Xu1W=JY=actia.se=john.ernberg@srs-se1.protection.inumbo.net>)
 id 1racB2-0003CF-O3
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 13:53:24 +0000
Received: from mail.actia.se (mail.actia.se [212.181.117.226])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 94ab7efc-cc09-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 14:53:21 +0100 (CET)
Received: from S036ANL.actianordic.se (10.12.31.117) by S035ANL.actianordic.se
 (10.12.31.116) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 15 Feb
 2024 14:53:37 +0100
Received: from S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69]) by
 S036ANL.actianordic.se ([fe80::e13e:1feb:4ea6:ec69%4]) with mapi id
 15.01.2507.035; Thu, 15 Feb 2024 14:53:21 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 94ab7efc-cc09-11ee-98f5-efadbce2ee36
From: John Ernberg <john.ernberg@actia.se>
To: Michal Orzel <michal.orzel@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "Peng
 Fan" <peng.fan@nxp.com>, Jonas Blixt <jonas.blixt@actia.se>
Subject: Re: [PATCH v2 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Topic: [PATCH v2 1/2] xen/arm: Add imx8q{m,x} platform glue
Thread-Index: AQHaX1/Xeq0xXmpbZkiZsYYSHlNyGLELC+MAgABRRAA=
Date: Thu, 15 Feb 2024 13:53:21 +0000
Message-ID: <00df4ae3-8d14-41c3-95f1-65516c48e01c@actia.se>
References: <20240214160644.3418228-1-john.ernberg@actia.se>
 <20240214160644.3418228-2-john.ernberg@actia.se>
 <3fe044c4-5e60-4e11-a99e-38534610640b@amd.com>
In-Reply-To: <3fe044c4-5e60-4e11-a99e-38534610640b@amd.com>
Accept-Language: en-US, sv-SE
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.12.12.30]
x-esetresult: clean, is OK
x-esetid: 37303A2921D7295560776A
Content-Type: text/plain; charset="utf-8"
Content-ID: <616A798E3110B14EAF3D6B37A4C35BB0@actia.se>
Content-Transfer-Encoding: base64
MIME-Version: 1.0

SGkgTWljaGFsLA0KDQpPbiAyLzE1LzI0IDEwOjAyLCBNaWNoYWwgT3J6ZWwgd3JvdGU6DQo+IEhp
LA0KPiANCj4gT24gMTQvMDIvMjAyNCAxNzowNiwgSm9obiBFcm5iZXJnIHdyb3RlOg0KPj4NCj4+
DQo+PiBXaGVuIHVzaW5nIExpbnV4IGZvciBkb20wIHRoZXJlIGFyZSBhIGJ1bmNoIG9mIGRyaXZl
cnMgdGhhdCBuZWVkIHRvIGRvIFNNQw0KPj4gU0lQIGNhbGxzIGludG8gdGhlIGZpcm13YXJlIHRv
IGVuYWJsZSBjZXJ0YWluIGhhcmR3YXJlIGJpdHMgbGlrZSB0aGUNCj4+IHdhdGNoZG9nLg0KPj4N
Cj4+IFByb3ZpZGUgYSBiYXNpYyBwbGF0Zm9ybSBnbHVlIHRoYXQgaW1wbGVtZW50cyB0aGUgbmVl
ZGVkIFNNQyBmb3J3YXJkaW5nLg0KPj4NCj4+IFRoZSBmb3JtYXQgb2YgdGhlc2UgY2FsbHMgYXJl
IGFzIGZvbGxvd3M6DQo+PiAgIC0gcmVnIDA6IHNlcnZpY2UgSUQNCj4+ICAgLSByZWcgMTogZnVu
Y3Rpb24gSUQNCj4+ICAgcmVtYWluaW5nIHJlZ3M6IGFyZ3MNCj4+DQo+PiBGb3Igbm93IHdlIG9u
bHkgYWxsb3cgRG9tMCB0byBtYWtlIHRoZXNlIGNhbGxzIGFzIHRoZXkgYXJlIGFsbCBtYW5hZ2lu
Zw0KPj4gaGFyZHdhcmUuIFRoZXJlIGlzIG5vIHNwZWNpZmljYXRpb24gZm9yIHRoZXNlIFNJUCBj
YWxscywgdGhlIElEcyBhbmQgbmFtZXMNCj4+IGhhdmUgYmVlbiBleHRyYWN0ZWQgZnJvbSB0aGUg
dXBzdHJlYW0gbGludXgga2VybmVsIGFuZCB0aGUgdmVuZG9yIGtlcm5lbC4NCj4+DQo+PiBNb3N0
IG9mIHRoZSBTSVAgY2FsbHMgYXJlIG9ubHkgYXZhaWxhYmxlIGZvciB0aGUgaU1YOE0gc2VyaWVz
IG9mIFNvQ3MsIHNvDQo+PiB0aGV5IGFyZSBlYXN5IHRvIHJlamVjdCBhbmQgdGhleSBuZWVkIHRv
IGJlIHJldmlzaXRlZCB3aGVuIGlNWDhNIHNlcmllcw0KPj4gc3VwcG9ydCBpcyBhZGRlZC4NCj4g
R2l2ZW4gdGhhdCB5b3UgbmFtZWQgeW91ciBkcml2ZXIgKnFtIGFuZCB5b3Ugc2VhcmNoIGZvciBk
dCBjb21wYXRpYmxlICpxbSwgKnF4cCwgZG9lcyBpdCByZWFsbHkNCj4gbWFrZSBzZW5zZSB0byBh
ZGQgU0lQIGNhbGxzIGZvciBpTVg4TT8NCg0KVGhlIGlkZWEgd2FzIHRvIGJlIGV4cGxpY2l0IGFi
b3V0IHdoeSBpdCBtYWtlcyBubyBzZW5zZSB0byBzdXBwb3J0IHRob3NlIA0KU0lQcyBvbiBhIHFt
L3F4cCBpbXBsZW1lbnRhdGlvbi4NCg0KSSBjYW4gdGFrZSBpdCBvdXQgaWYgdGhhdCdzIG5vdCBh
cmd1bWVudCBlbm91Z2ggdG8gaGF2ZSB0aGVtLg0KPiANCj4+DQo+PiAgRnJvbSB0aGUgb3RoZXIg
Y2FsbHMgd2UgY2FuIHJlamVjdCBDUFVGUkVRIGJlY2F1c2UgRG9tMCBjYW5ub3QgbWFrZSBhbg0K
Pj4gaW5mb3JtZWQgZGVjaXNpb24gcmVnYXJkaW5nIENQVSBmcmVxdWVuY3kgc2NhbGluZywgV0FL
RVVQX1NSQyBpcyB0byB3YWtlDQo+PiB1cCBmcm9tIHN1c3BlbmQsIHdoaWNoIFhlbiBkb2Vzbid0
IHN1cHBvcnQgYXQgdGhpcyB0aW1lLg0KPj4NCj4+IFRoaXMgbGVhdmVzIHRoZSBUSU1FIFNJUCBh
bmQgdGhlIE9UUF9XUklURSBTSVAsIHdoaWNoIGZvciBub3cgYXJlIGFsbG93ZWQNCj4+IHRvIERv
bTAuDQo+Pg0KPj4gTk9URTogVGhpcyBjb2RlIGlzIGJhc2VkIG9uIGNvZGUgZm91bmQgaW4gTlhQ
IFhlbiB0cmVlIGxvY2F0ZWQgaGVyZToNCj4+IGh0dHBzOi8vZ2l0aHViLmNvbS9ueHAtaW14L2lt
eC14ZW4vYmxvYi9sZi01LjEwLnlfNC4xMy94ZW4vYXJjaC9hcm0vcGxhdGZvcm1zL2lteDhxbS5j
DQo+Pg0KPj4gU2lnbmVkLW9mZi1ieTogUGVuZyBGYW4gPHBlbmcuZmFuQG54cC5jb20+DQo+PiBb
amVybmJlcmc6IEFkZCBTSVAgY2FsbCBmaWx0ZXJpbmddDQo+PiBTaWduZWQtb2ZmLWJ5OiBKb2hu
IEVybmJlcmcgPGpvaG4uZXJuYmVyZ0BhY3RpYS5zZT4NCj4+DQo+PiAtLS0NCj4+DQo+PiB2MjoN
Cj4+ICAgLSBSZXdvcmQgdGhlIGNvbW1pdCBtZXNzYWdlIHRvIGJlIGEgYml0IGNsZWFyZXINCj4+
ICAgLSBJbmNsdWRlIHRoZSBsaW5rIHByZXZpb3VzbHkgYWRkZWQgYXMgYSBjb250ZXh0IG5vdGUg
dG8gdGhlIGNvbW1pdCBtZXNzYWdlIChKdWxpZW4gR3JhbGwpDQo+PiAgIC0gQWRkIFBlbmdzIHNp
Z25lZCBvZmYgKEp1bGllbiBHcmFsbCwgUGVuZyBGYW4pDQo+PiAgIC0gQWRkIGJhc2ljIFNJUCBj
YWxsIGZpbHRlciAoSnVsaWVuIEdyYWxsKQ0KPj4gICAtIEV4cGFuZCB0aGUgY29tbWl0IG1lc3Nh
Z2UgYSB3aG9sZSBidW5jaCBiZWNhdXNlIG9mIHRoZSBjaGFuZ2VzIHRvIHRoZSBjb2RlDQo+PiAt
LS0NCj4+ICAgeGVuL2FyY2gvYXJtL3BsYXRmb3Jtcy9NYWtlZmlsZSB8ICAgMSArDQo+PiAgIHhl
bi9hcmNoL2FybS9wbGF0Zm9ybXMvaW14OHFtLmMgfCAxNDMgKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysNCj4+ICAgMiBmaWxlcyBjaGFuZ2VkLCAxNDQgaW5zZXJ0aW9ucygrKQ0KPj4g
ICBjcmVhdGUgbW9kZSAxMDA2NDQgeGVuL2FyY2gvYXJtL3BsYXRmb3Jtcy9pbXg4cW0uYw0KPj4N
Cj4+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGxhdGZvcm1zL01ha2VmaWxlIGIveGVuL2Fy
Y2gvYXJtL3BsYXRmb3Jtcy9NYWtlZmlsZQ0KPj4gaW5kZXggODYzMmY0MTE1Zi4uYmVjNmU1NWQx
ZiAxMDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL2FybS9wbGF0Zm9ybXMvTWFrZWZpbGUNCj4+ICsr
KyBiL3hlbi9hcmNoL2FybS9wbGF0Zm9ybXMvTWFrZWZpbGUNCj4+IEBAIC05LDUgKzksNiBAQCBv
YmotJChDT05GSUdfQUxMX1BMQVQpICAgKz0gc3VueGkubw0KPj4gICBvYmotJChDT05GSUdfQUxM
NjRfUExBVCkgKz0gdGh1bmRlcngubw0KPj4gICBvYmotJChDT05GSUdfQUxMNjRfUExBVCkgKz0g
eGdlbmUtc3Rvcm0ubw0KPj4gICBvYmotJChDT05GSUdfQUxMNjRfUExBVCkgKz0gYnJjbS1yYXNw
YmVycnktcGkubw0KPj4gK29iai0kKENPTkZJR19BTEw2NF9QTEFUKSArPSBpbXg4cW0ubw0KPj4g
ICBvYmotJChDT05GSUdfTVBTT0NfUExBVEZPUk0pICArPSB4aWxpbngtenlucW1wLm8NCj4+ICAg
b2JqLSQoQ09ORklHX01QU09DX1BMQVRGT1JNKSAgKz0geGlsaW54LXp5bnFtcC1lZW1pLm8NCj4+
IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vcGxhdGZvcm1zL2lteDhxbS5jIGIveGVuL2FyY2gv
YXJtL3BsYXRmb3Jtcy9pbXg4cW0uYw0KPj4gbmV3IGZpbGUgbW9kZSAxMDA2NDQNCj4+IGluZGV4
IDAwMDAwMDAwMDAuLjQ1MTVjNzU5MzUNCj4+IC0tLSAvZGV2L251bGwNCj4+ICsrKyBiL3hlbi9h
cmNoL2FybS9wbGF0Zm9ybXMvaW14OHFtLmMNCj4+IEBAIC0wLDAgKzEsMTQzIEBADQo+PiArLyog
U1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAtb3ItbGF0ZXIgKi8NCj4+ICsvKg0KPj4g
KyAqIHhlbi9hcmNoL2FybS9wbGF0Zm9ybXMvaW14OHFtLmMNCj4+ICsgKg0KPj4gKyAqIGkuTVgg
OFFNIHNldHVwDQo+PiArICoNCj4+ICsgKiBDb3B5cmlnaHQgKGMpIDIwMTYgRnJlZXNjYWxlIElu
Yy4NCj4+ICsgKiBDb3B5cmlnaHQgMjAxOC0yMDE5IE5YUA0KPj4gKyAqDQo+PiArICoNCj4+ICsg
KiBQZW5nIEZhbiA8cGVuZy5mYW5AbnhwLmNvbT4NCj4+ICsgKi8NCj4+ICsNCj4+ICsjaW5jbHVk
ZSA8eGVuL3NjaGVkLmg+DQo+PiArI2luY2x1ZGUgPGFzbS9wbGF0Zm9ybS5oPg0KPj4gKyNpbmNs
dWRlIDxhc20vc21jY2MuaD4NCj4+ICsNCj4+ICtzdGF0aWMgY29uc3QgY2hhciAqIGNvbnN0IGlt
eDhxbV9kdF9jb21wYXRbXSBfX2luaXRjb25zdCA9DQo+PiArew0KPj4gKyAgICAiZnNsLGlteDhx
bSIsDQo+PiArICAgICJmc2wsaW14OHF4cCIsDQo+PiArICAgIE5VTEwNCj4+ICt9Ow0KPj4gKw0K
Pj4gKyNkZWZpbmUgSU1YX1NJUF9HUEMgICAgICAgICAgMHhDMjAwMDAwMA0KPiBOSVQ6IEEgbWF0
dGVyIG9mIHBlcnNvbmFsIG9waW5pb24gYnV0IGFsbCB0aGUgQXJtNjQgU0lQIHNlcnZpY2VzIHN0
YXJ0cyB3aXRoIDB4YzIwMDAwMDAsDQo+IHNvIHlvdSBjb3VsZCBqdXN0IGRlZmluZSBhIGZ1bmN0
aW9uIGlkIGkuZS4gMSwgMiwgLi4uIGFuZCB1c2UgYSBtYWNybyBzaW1pbGFyIHRvIEVFTUlfRklE
KGZpZCkuDQo+IFRoaXMgaXMganVzdCBhIHN1Z2dlc3Rpb24uDQoNCkFjay4NCj4gDQo+PiArI2Rl
ZmluZSBJTVhfU0lQX0NQVUZSRVEgICAgICAweEMyMDAwMDAxDQo+PiArI2RlZmluZSBJTVhfU0lQ
X1RJTUUgICAgICAgICAweEMyMDAwMDAyDQo+PiArI2RlZmluZSBJTVhfU0lQX0REUl9EVkZTICAg
ICAweEMyMDAwMDA0DQo+PiArI2RlZmluZSBJTVhfU0lQX1NSQyAgICAgICAgICAweEMyMDAwMDA1
DQo+PiArI2RlZmluZSBJTVhfU0lQX0dFVF9TT0NfSU5GTyAweEMyMDAwMDA2DQo+PiArI2RlZmlu
ZSBJTVhfU0lQX05PQyAgICAgICAgICAweEMyMDAwMDA4DQo+PiArI2RlZmluZSBJTVhfU0lQX1dB
S0VVUF9TUkMgICAweEMyMDAwMDA5DQo+PiArI2RlZmluZSBJTVhfU0lQX09UUF9XUklURSAgICAw
eEMyMDAwMDBCDQo+IExvb2tpbmcgYXQgQVRGLCBmb3IgUU0sUVhQIHRoZXJlIGFyZSBhbHNvOg0K
PiBCVUlMRElORk8gMHhDMjAwMDAwMw0KPiBPVFBfUkVBRCAgMHhDMjAwMDAwQQ0KPiBTRVRfVEVN
UCAgMHhDMjAwMDAwQw0KDQpJIGNhbid0IGZpbmQgdGhlbSBpbiB0aGUgY3VycmVudCBMaW51eCBj
b2RlLCBwZXJoYXBzIGRlcHJlY2F0ZWQuDQpJIGNhbiBhZGQgdGhlbSBhbmQgZGVueSB0aGVtIGlm
IGl0IG1ha2VzIHNlbnNlLg0KPiANCj4+ICsNCj4+ICsjZGVmaW5lIElNWF9TSVBfVElNRV9GX1JU
Q19TRVRfVElNRSAgICAgMHgwMA0KPj4gKyNkZWZpbmUgSU1YX1NJUF9USU1FX0ZfV0RPR19TVEFS
VCAgICAgICAweDAxDQo+PiArI2RlZmluZSBJTVhfU0lQX1RJTUVfRl9XRE9HX1NUT1AgICAgICAg
IDB4MDINCj4+ICsjZGVmaW5lIElNWF9TSVBfVElNRV9GX1dET0dfU0VUX0FDVCAgICAgMHgwMw0K
Pj4gKyNkZWZpbmUgSU1YX1NJUF9USU1FX0ZfV0RPR19QSU5HICAgICAgICAweDA0DQo+PiArI2Rl
ZmluZSBJTVhfU0lQX1RJTUVfRl9XRE9HX1NFVF9USU1FT1VUIDB4MDUNCj4+ICsjZGVmaW5lIElN
WF9TSVBfVElNRV9GX1dET0dfR0VUX1NUQVQgICAgMHgwNg0KPj4gKyNkZWZpbmUgSU1YX1NJUF9U
SU1FX0ZfV0RPR19TRVRfUFJFVElNRSAweDA3DQo+PiArDQo+PiArc3RhdGljIGJvb2wgaW14OHFt
X2lzX3NpcF90aW1lX2NhbGxfb2sodWludDMyX3QgZnVuY3Rpb25faWQpDQo+PiArew0KPj4gKyAg
ICBzd2l0Y2ggKCBmdW5jdGlvbl9pZCApDQo+PiArICAgIHsNCj4+ICsgICAgY2FzZSBJTVhfU0lQ
X1RJTUVfRl9SVENfU0VUX1RJTUU6DQo+PiArICAgICAgICByZXR1cm4gdHJ1ZTsNCj4+ICsgICAg
Y2FzZSBJTVhfU0lQX1RJTUVfRl9XRE9HX1NUQVJUOg0KPj4gKyAgICBjYXNlIElNWF9TSVBfVElN
RV9GX1dET0dfU1RPUDoNCj4+ICsgICAgY2FzZSBJTVhfU0lQX1RJTUVfRl9XRE9HX1NFVF9BQ1Q6
DQo+PiArICAgIGNhc2UgSU1YX1NJUF9USU1FX0ZfV0RPR19QSU5HOg0KPj4gKyAgICBjYXNlIElN
WF9TSVBfVElNRV9GX1dET0dfU0VUX1RJTUVPVVQ6DQo+PiArICAgIGNhc2UgSU1YX1NJUF9USU1F
X0ZfV0RPR19HRVRfU1RBVDoNCj4+ICsgICAgY2FzZSBJTVhfU0lQX1RJTUVfRl9XRE9HX1NFVF9Q
UkVUSU1FOg0KPj4gKyAgICAgICAgcmV0dXJuIHRydWU7DQo+PiArICAgIGRlZmF1bHQ6DQo+PiAr
ICAgICAgICBwcmludGsoWEVOTE9HX1dBUk5JTkcgImlteDhxbTogc21jOiB0aW1lOiBVbmtub3du
IGZ1bmN0aW9uIGlkICV4XG4iLCBmdW5jdGlvbl9pZCk7DQo+IDgwIGNoYXJzIGxpbWl0LCBtb3Zl
IGFyZ3VtZW50IGxpc3QgdG8gdGhlIG5leHQgbGluZQ0KDQpBY2suIEkgZm9yZ290IHRvIGNoZWNr
IHRoZSBjb2Rpbmcgc3RhbmRhcmQgb24gbGluZSBsZW5ndGguDQo+IA0KPj4gKyAgICAgICAgcmV0
dXJuIGZhbHNlOw0KPj4gKyAgICB9DQo+PiArfQ0KPj4gKw0KPj4gK3N0YXRpYyBib29sIGlteDhx
bV9zbWMoc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MpDQo+PiArew0KPj4gKyAgICB1aW50MzJf
dCBzZXJ2aWNlX2lkID0gZ2V0X3VzZXJfcmVnKHJlZ3MsIDApOw0KPiBJbiBTTUNDQyBuYW1pbmcg
Y29udmVudGlvbiwgVzAgaXMgY2FsbGVkIGZ1bmN0aW9uIGlkZW50aWZpZXIuIEluc3RlYWQgeW91
IGNhbGwgWDEgZnVuY3Rpb25faWQNCj4gd2hpY2ggaXMgYSBiaXQgbWlzbGVhZGluZy4NCg0KQWNr
LiBXaWxsIHVzZSBmdW5jdGlvbl9pZCBhbmQgc3ViZnVuY3Rpb25faWQgaW5zdGVhZC4NCj4gDQo+
PiArICAgIHVpbnQzMl90IGZ1bmN0aW9uX2lkID0gZ2V0X3VzZXJfcmVnKHJlZ3MsIDEpOw0KPj4g
KyAgICBzdHJ1Y3QgYXJtX3NtY2NjX3JlcyByZXM7DQo+PiArDQo+PiArICAgIGlmICggIWNwdXNf
aGF2ZV9jb25zdF9jYXAoQVJNX1NNQ0NDXzFfMSkgKQ0KPj4gKyAgICB7DQo+PiArICAgICAgICBw
cmludGtfb25jZShYRU5MT0dfV0FSTklORyAibm8gU01DQ0MgMS4xIHN1cHBvcnQuIERpc2FibGlu
ZyBmaXJtd2FyZSBjYWxsc1xuIik7DQo+IDgwIGNoYXJzIGxpbWl0LCBtb3ZlIHN0cmluZyB0byB0
aGUgbmV4dCBsaW5lDQpBY2suDQo+IA0KPj4gKw0KPj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOw0K
Pj4gKyAgICB9DQo+PiArDQo+PiArICAgIC8qIE9ubHkgaGFyZHdhcmUgZG9tYWluIG1heSB1c2Ug
dGhlIFNJUCBjYWxscyAqLw0KPj4gKyAgICBpZiAoICFpc19oYXJkd2FyZV9kb21haW4oY3VycmVu
dC0+ZG9tYWluKSApDQo+PiArICAgIHsNCj4+ICsgICAgICAgIGdwcmludGsoWEVOTE9HX1dBUk5J
TkcsICJpbXg4cW06IHNtYzogTm8gYWNjZXNzXG4iKTsNCj4+ICsgICAgICAgIHJldHVybiBmYWxz
ZTsNCj4+ICsgICAgfQ0KPj4gKw0KPj4gKyAgICBzd2l0Y2ggKCBzZXJ2aWNlX2lkICkNCj4+ICsg
ICAgew0KPj4gKyAgICBjYXNlIElNWF9TSVBfR1BDOiAvKiBPbmx5IGF2YWlsYWJsZSBvbiBpbXg4
bSBzZXJpZXMgKi8NCj4gSWYgd2UgZGVjaWRlIHRvIGtlZXAgaU1YOE0gRklEcywgSSB0aGluayBh
ZGRpbmcgY29tbWVudHMgb24gdG9wIG9mIGEgY2FzZSB3b3VsZCByZXN1bHQgaW4gYSBtb3JlIHJl
YWRhYmxlIGNvZGUuDQo+IA0KPj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOw0KPj4gKyAgICBjYXNl
IElNWF9TSVBfQ1BVRlJFUTogLyogRG9tMCBjYW4ndCB0YWtlIGFueSBpbmZvcm1lZCBkZXNjaXNp
b24gaGVyZSAqLw0KPj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOw0KPj4gKyAgICBjYXNlIElNWF9T
SVBfVElNRToNCj4+ICsgICAgICAgIGlmICggaW14OHFtX2lzX3NpcF90aW1lX2NhbGxfb2soZnVu
Y3Rpb25faWQpICkNCj4+ICsgICAgICAgICAgICBnb3RvIGFsbG93X2NhbGw7DQo+PiArICAgICAg
ICByZXR1cm4gZmFsc2U7DQo+PiArICAgIGNhc2UgSU1YX1NJUF9ERFJfRFZGUzogLyogT25seSBh
dmFpbGFibGUgb24gaW14OG0gc2VyaWVzICovDQo+PiArICAgICAgICByZXR1cm4gZmFsc2U7DQo+
PiArICAgIGNhc2UgSU1YX1NJUF9TUkM6IC8qIE9ubHkgYXZhaWxhYmxlIG9uIGlteDhtIHNlcmll
cyAqLw0KPj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOw0KPj4gKyAgICBjYXNlIElNWF9TSVBfR0VU
X1NPQ19JTkZPOiAvKiBPbmx5IGF2YWlsYWJsZSBvbiBpbXg4bSBzZXJpZXMgKi8NCj4+ICsgICAg
ICAgIHJldHVybiBmYWxzZTsNCj4+ICsgICAgY2FzZSBJTVhfU0lQX05PQzogLyogT25seSBhdmFp
bGFibGUgb24gaW14OG0gc2VyaWVzICovDQo+PiArICAgICAgICByZXR1cm4gZmFsc2U7DQo+PiAr
ICAgIGNhc2UgSU1YX1NJUF9XQUtFVVBfU1JDOiAvKiBYZW4gZG9lc24ndCBoYXZlIHN1c3BlbmQg
c3VwcG9ydCAqLw0KPj4gKyAgICAgICAgcmV0dXJuIGZhbHNlOw0KPj4gKyAgICBjYXNlIElNWF9T
SVBfT1RQX1dSSVRFOg0KPj4gKyAgICAgICAgLyogZnVuY3Rpb25faWQgaXMgdGhlIGZ1c2UgbnVt
YmVyLCBubyBzZW5zaWJsZSBjaGVjayBwb3NzaWJsZSAqLw0KPj4gKyAgICAgICAgZ290byBhbGxv
d19jYWxsOw0KPj4gKyAgICBkZWZhdWx0Og0KPj4gKyAgICAgICAgcHJpbnRrKFhFTkxPR19XQVJO
SU5HICJpbXg4cW06IHNtYzogVW5rbm93biBzZXJ2aWNlIGlkICV4XG4iLCBzZXJ2aWNlX2lkKTsN
Cj4gODAgY2hhcnMgbGltaXQsIG1vdmUgYXJndW1lbnQgbGlzdCB0byB0aGUgbmV4dCBsaW5lDQoN
CkFjaw0KPiANCj4+ICsgICAgICAgIHJldHVybiBmYWxzZTsNCj4+ICsgICAgfQ0KPj4gKw0KPj4g
K2FsbG93X2NhbGw6DQo+IGxhYmVscyBuZWVkIHRvIGJlIGluZGVudGVkIHdpdGggb25lIHNwYWNl
DQpBY2suIE1pc3NlZCB0aGF0IGNsYXVzZSBpbiB0aGUgaW5kZW50YXRpb24gY2hhcHRlci4NCj4g
DQo+PiArICAgIGFybV9zbWNjY18xXzFfc21jKHNlcnZpY2VfaWQsDQo+PiArICAgICAgICAgICAg
ICAgICAgICAgIGZ1bmN0aW9uX2lkLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICBnZXRfdXNl
cl9yZWcocmVncywgMiksDQo+PiArICAgICAgICAgICAgICAgICAgICAgIGdldF91c2VyX3JlZyhy
ZWdzLCAzKSwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgZ2V0X3VzZXJfcmVnKHJlZ3MsIDQp
LA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICBnZXRfdXNlcl9yZWcocmVncywgNSksDQo+PiAr
ICAgICAgICAgICAgICAgICAgICAgIGdldF91c2VyX3JlZyhyZWdzLCA2KSwNCj4+ICsgICAgICAg
ICAgICAgICAgICAgICAgZ2V0X3VzZXJfcmVnKHJlZ3MsIDcpLA0KPj4gKyAgICAgICAgICAgICAg
ICAgICAgICAmcmVzKTsNCj4+ICsNCj4+ICsgICAgc2V0X3VzZXJfcmVnKHJlZ3MsIDAsIHJlcy5h
MCk7DQo+PiArICAgIHNldF91c2VyX3JlZyhyZWdzLCAxLCByZXMuYTEpOw0KPj4gKyAgICBzZXRf
dXNlcl9yZWcocmVncywgMiwgcmVzLmEyKTsNCj4+ICsgICAgc2V0X3VzZXJfcmVnKHJlZ3MsIDMs
IHJlcy5hMyk7DQo+PiArDQo+PiArICAgIHJldHVybiB0cnVlOw0KPj4gK30NCj4+ICsNCj4+ICtQ
TEFURk9STV9TVEFSVChpbXg4cW0sICJpLk1YIDgiKQ0KPiBTaG91bGRuJ3QgaXQgYmUgaS5NWCA4
UXtNLFhQfSA/DQpBY2suDQo+IA0KPiB+TWljaGFsDQoNClRoYW5rcyEgLy8gSm9obiBFcm5iZXJn


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 14:32:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 14:32:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681830.1060797 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1racmD-0002iC-KV; Thu, 15 Feb 2024 14:31:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681830.1060797; Thu, 15 Feb 2024 14:31:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1racmD-0002i5-Gv; Thu, 15 Feb 2024 14:31:49 +0000
Received: by outflank-mailman (input) for mailman id 681830;
 Thu, 15 Feb 2024 14:31:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YE+Y=JY=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1racmC-0002hx-Bn
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 14:31:48 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f0ca4d78-cc0e-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 15:31:43 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 586DF4EE0737;
 Thu, 15 Feb 2024 15:31:43 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0ca4d78-cc0e-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Thu, 15 Feb 2024 15:31:43 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Julien Grall <julien@xen.org>
Cc: Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Xen Devel <xen-devel@lists.xenproject.org>,
 Consulting <consulting@bugseng.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Andrew
 Cooper3 <andrew.cooper3@citrix.com>, Roger Pau <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: Return type of clean_and_invalidate_dcache_va_range
In-Reply-To: <9eb25d90-9f10-44e5-b9aa-32e3f898389a@xen.org>
References: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
 <alpine.DEB.2.22.394.2402091402080.1925432@ubuntu-linux-20-04-desktop>
 <f86baad0-f113-4156-9c10-6910e8c63492@xen.org>
 <02446758-a5a9-4a8a-8ce6-15abdc9ede65@suse.com>
 <99faac70440a68824a17fcaaea55ef48@bugseng.com>
 <09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org>
 <c0b8f2ec-a3bb-45a6-b748-052d55a6fd5c@suse.com>
 <9eb25d90-9f10-44e5-b9aa-32e3f898389a@xen.org>
Message-ID: <2171bcc9eeea6613409e3ee1e9eedbc3@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: multipart/mixed;
 boundary="=_21dddb0de5c10f3729bd45b19a2db9a4"

--=_21dddb0de5c10f3729bd45b19a2db9a4
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII;
 format=flowed

Hi Julien,

On 2024-02-13 18:14, Julien Grall wrote:
> Hi Jan,
> 
> On 13/02/2024 07:13, Jan Beulich wrote:
>> On 12.02.2024 19:38, Julien Grall wrote:
>>> An alternative would be to introduced arch_grant_cache_flush() and 
>>> move
>>> the if/else logic there. Something like:
>>> 
>>> diff --git a/xen/arch/arm/include/asm/page.h
>>> b/xen/arch/arm/include/asm/page.h
>>> index 69f817d1e68a..4a3de49762a1 100644
>>> --- a/xen/arch/arm/include/asm/page.h
>>> +++ b/xen/arch/arm/include/asm/page.h
>>> @@ -281,6 +281,19 @@ static inline void write_pte(lpae_t *p, lpae_t 
>>> pte)
>>>        dsb(sy);
>>>    }
>>> 
>>> +static inline arch_grant_cache_flush(unsigned int op, const void *p,
>>> unsigned long size)
>>> +{
>>> +    unsigned int order = get_order_from_bytes(size);
>>> +
>>> +    if ( (cflush->op & GNTTAB_CACHE_INVAL) && (cflush->op &
>>> GNTTAB_CACHE_CLEAN) )
>>> +        clean_and_invalidate_dcache_va_range(v, cflush->length);
>>> +    else if ( cflush->op & GNTTAB_CACHE_INVAL )
>>> +        invalidate_dcache_va_range(v, cflush->length);
>>> +    else if ( cflush->op & GNTTAB_CACHE_CLEAN )
>>> +        clean_dcache_va_range(v, cflush->length);
>>> +
>>> +    return 0;
>>> +}
>>> 
>>>    /* Flush the dcache for an entire page. */
>>>    void flush_page_to_ram(unsigned long mfn, bool sync_icache);
>>> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
>>> index 424744ad5e1a..647e1522466d 100644
>>> --- a/xen/arch/arm/setup.c
>>> +++ b/xen/arch/arm/setup.c
>>> @@ -735,8 +735,7 @@ void asmlinkage __init start_xen(unsigned long
>>> boot_phys_offset,
>>>                  fdt_paddr);
>>> 
>>>        /* Register Xen's load address as a boot module. */
>>> -    xen_bootmodule = add_boot_module(BOOTMOD_XEN,
>>> -                             virt_to_maddr(_start),
>>> +    xen_bootmodule = add_boot_module(BOOTMOD_XEN, 
>>> virt_to_maddr(_start),
>>>                                 (paddr_t)(uintptr_t)(_end - _start), 
>>> false);
>>>        BUG_ON(!xen_bootmodule);
>>> 
>>> diff --git a/xen/arch/x86/include/asm/flushtlb.h
>>> b/xen/arch/x86/include/asm/flushtlb.h
>>> index bb0ad58db49b..dfe51cddde90 100644
>>> --- a/xen/arch/x86/include/asm/flushtlb.h
>>> +++ b/xen/arch/x86/include/asm/flushtlb.h
>>> @@ -182,23 +182,22 @@ void flush_area_mask(const cpumask_t *mask, 
>>> const
>>> void *va,
>>>    }
>>> 
>>>    static inline void flush_page_to_ram(unsigned long mfn, bool
>>> sync_icache) {}
>>> -static inline int invalidate_dcache_va_range(const void *p,
>>> -                                             unsigned long size)
>>> -{ return -EOPNOTSUPP; }
>>> -static inline int clean_and_invalidate_dcache_va_range(const void 
>>> *p,
>>> -                                                       unsigned long 
>>> size)
>>> +
>>> +unsigned int guest_flush_tlb_flags(const struct domain *d);
>>> +void guest_flush_tlb_mask(const struct domain *d, const cpumask_t 
>>> *mask);
>>> +
>>> +static inline arch_grant_cache_flush(unsigned int op, const void *p,
>>> unsigned long size)
>>>    {
>>> -    unsigned int order = get_order_from_bytes(size);
>>> +    unsigned int order;
>>> +
>>> +    if ( !(cflush->op & GNTTAB_CACHE_CLEAN) )
>>> +        return -EOPNOTSUPP;
>>> +
>>> +    order = get_order_from_bytes(size);
>>>        /* sub-page granularity support needs to be added if necessary 
>>> */
>>>        flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
>>> +
>>>        return 0;
>>>    }
>>> -static inline int clean_dcache_va_range(const void *p, unsigned long 
>>> size)
>>> -{
>>> -    return clean_and_invalidate_dcache_va_range(p, size);
>>> -}
>>> -
>>> -unsigned int guest_flush_tlb_flags(const struct domain *d);
>>> -void guest_flush_tlb_mask(const struct domain *d, const cpumask_t 
>>> *mask);
>>> 
>>>    #endif /* __FLUSHTLB_H__ */
>>> 
>>> I have a slight preference for the latter. I would like to hear the
>>> opinion of the others.
>> 
>> I would prefer this 2nd form, too, assuming the setup.c change wasn't
>> really meant to be there.
> 
> Indeed. I had another previous change I didn't and forgot to remove it.
> 
>> The one thing that doesn't become clear: In
>> the sketch above arch_grant_cache_flush() has no return type, yet has
>> "return 0". This raises a question towards the one that's at the root
>> of this thread: Do you mean the function to have a return value, and
>> if so will it be (sensibly) used?
> 
> Sorry I should have double checked the code before sending it. 
> arch_grant_cache_flush() should return a value. So each arch can decide 
> if they handle a given operation.
> 
> Cheers,

I do like the idea. I applied some of the suggestions to this proof of 
concept patch (attached). Still not compile-tested, since the CI seems a 
bit slow today.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)
--=_21dddb0de5c10f3729bd45b19a2db9a4
Content-Transfer-Encoding: base64
Content-Type: text/x-diff;
 name=cache_helpers.patch
Content-Disposition: attachment;
 filename=cache_helpers.patch;
 size=5715

ZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9pbmNsdWRlL2FzbS9wYWdlLmggYi94ZW4vYXJjaC9h
cm0vaW5jbHVkZS9hc20vcGFnZS5oCmluZGV4IDY5ZjgxN2QxZTY4YS4uNWY5MzU3NjMyMTY0IDEw
MDY0NAotLS0gYS94ZW4vYXJjaC9hcm0vaW5jbHVkZS9hc20vcGFnZS5oCisrKyBiL3hlbi9hcmNo
L2FybS9pbmNsdWRlL2FzbS9wYWdlLmgKQEAgLTE1OSwxMyArMTU5LDEzIEBAIHN0YXRpYyBpbmxp
bmUgc2l6ZV90IHJlYWRfZGNhY2hlX2xpbmVfYnl0ZXModm9pZCkKICAqIGlmICdyYW5nZScgaXMg
bGFyZ2UgZW5vdWdoIHdlIG1pZ2h0IHdhbnQgdG8gdXNlIG1vZGVsLXNwZWNpZmljCiAgKiBmdWxs
LWNhY2hlIGZsdXNoZXMuICovCiAKLXN0YXRpYyBpbmxpbmUgaW50IGludmFsaWRhdGVfZGNhY2hl
X3ZhX3JhbmdlKGNvbnN0IHZvaWQgKnAsIHVuc2lnbmVkIGxvbmcgc2l6ZSkKK3N0YXRpYyBpbmxp
bmUgdm9pZCBpbnZhbGlkYXRlX2RjYWNoZV92YV9yYW5nZShjb25zdCB2b2lkICpwLCB1bnNpZ25l
ZCBsb25nIHNpemUpCiB7CiAgICAgc2l6ZV90IGNhY2hlbGluZV9tYXNrID0gZGNhY2hlX2xpbmVf
Ynl0ZXMgLSAxOwogICAgIHVuc2lnbmVkIGxvbmcgaWR4ID0gMDsKIAogICAgIGlmICggIXNpemUg
KQotICAgICAgICByZXR1cm4gMDsKKyAgICAgICAgcmV0dXJuOwogCiAgICAgLyogUGFzc2luZyBh
IHJlZ2lvbiB0aGF0IHdyYXBzIGFyb3VuZCBpcyBpbGxlZ2FsICovCiAgICAgQVNTRVJUKCgodWlu
dHB0cl90KXAgKyBzaXplIC0gMSkgPj0gKHVpbnRwdHJfdClwKTsKQEAgLTE4OCwxNyArMTg4LDE1
IEBAIHN0YXRpYyBpbmxpbmUgaW50IGludmFsaWRhdGVfZGNhY2hlX3ZhX3JhbmdlKGNvbnN0IHZv
aWQgKnAsIHVuc2lnbmVkIGxvbmcgc2l6ZSkKICAgICAgICAgYXNtIHZvbGF0aWxlIChfX2NsZWFu
X2FuZF9pbnZhbGlkYXRlX2RjYWNoZV9vbmUoMCkgOiA6ICJyIiAocCArIGlkeCkpOwogCiAgICAg
ZHNiKHN5KTsgICAgICAgICAgIC8qIFNvIHdlIGtub3cgdGhlIGZsdXNoZXMgaGFwcGVuIGJlZm9y
ZSBjb250aW51aW5nICovCi0KLSAgICByZXR1cm4gMDsKIH0KIAotc3RhdGljIGlubGluZSBpbnQg
Y2xlYW5fZGNhY2hlX3ZhX3JhbmdlKGNvbnN0IHZvaWQgKnAsIHVuc2lnbmVkIGxvbmcgc2l6ZSkK
K3N0YXRpYyBpbmxpbmUgdm9pZCBjbGVhbl9kY2FjaGVfdmFfcmFuZ2UoY29uc3Qgdm9pZCAqcCwg
dW5zaWduZWQgbG9uZyBzaXplKQogewogICAgIHNpemVfdCBjYWNoZWxpbmVfbWFzayA9IGRjYWNo
ZV9saW5lX2J5dGVzIC0gMTsKICAgICB1bnNpZ25lZCBsb25nIGlkeCA9IDA7CiAKICAgICBpZiAo
ICFzaXplICkKLSAgICAgICAgcmV0dXJuIDA7CisgICAgICAgIHJldHVybjsKIAogICAgIC8qIFBh
c3NpbmcgYSByZWdpb24gdGhhdCB3cmFwcyBhcm91bmQgaXMgaWxsZWdhbCAqLwogICAgIEFTU0VS
VCgoKHVpbnRwdHJfdClwICsgc2l6ZSAtIDEpID49ICh1aW50cHRyX3QpcCk7CkBAIC0yMTEsMTgg
KzIwOSwxNiBAQCBzdGF0aWMgaW5saW5lIGludCBjbGVhbl9kY2FjaGVfdmFfcmFuZ2UoY29uc3Qg
dm9pZCAqcCwgdW5zaWduZWQgbG9uZyBzaXplKQogICAgICAgICAgICAgaWR4ICs9IGRjYWNoZV9s
aW5lX2J5dGVzLCBzaXplIC09IGRjYWNoZV9saW5lX2J5dGVzICkKICAgICAgICAgYXNtIHZvbGF0
aWxlIChfX2NsZWFuX2RjYWNoZV9vbmUoMCkgOiA6ICJyIiAocCArIGlkeCkpOwogICAgIGRzYihz
eSk7ICAgICAgICAgICAvKiBTbyB3ZSBrbm93IHRoZSBmbHVzaGVzIGhhcHBlbiBiZWZvcmUgY29u
dGludWluZyAqLwotICAgIC8qIEFSTSBjYWxsZXJzIGFzc3VtZSB0aGF0IGRjYWNoZV8qIGZ1bmN0
aW9ucyBjYW5ub3QgZmFpbC4gKi8KLSAgICByZXR1cm4gMDsKIH0KIAotc3RhdGljIGlubGluZSBp
bnQgY2xlYW5fYW5kX2ludmFsaWRhdGVfZGNhY2hlX3ZhX3JhbmdlCitzdGF0aWMgaW5saW5lIHZv
aWQgY2xlYW5fYW5kX2ludmFsaWRhdGVfZGNhY2hlX3ZhX3JhbmdlCiAgICAgKGNvbnN0IHZvaWQg
KnAsIHVuc2lnbmVkIGxvbmcgc2l6ZSkKIHsKICAgICBzaXplX3QgY2FjaGVsaW5lX21hc2sgPSBk
Y2FjaGVfbGluZV9ieXRlcyAtIDE7CiAgICAgdW5zaWduZWQgbG9uZyBpZHggPSAwOwogCiAgICAg
aWYgKCAhc2l6ZSApCi0gICAgICAgIHJldHVybiAwOworICAgICAgICByZXR1cm47CiAKICAgICAv
KiBQYXNzaW5nIGEgcmVnaW9uIHRoYXQgd3JhcHMgYXJvdW5kIGlzIGlsbGVnYWwgKi8KICAgICBB
U1NFUlQoKCh1aW50cHRyX3QpcCArIHNpemUgLSAxKSA+PSAodWludHB0cl90KXApOwpAQCAtMjM1
LDggKzIzMSw2IEBAIHN0YXRpYyBpbmxpbmUgaW50IGNsZWFuX2FuZF9pbnZhbGlkYXRlX2RjYWNo
ZV92YV9yYW5nZQogICAgICAgICAgICAgaWR4ICs9IGRjYWNoZV9saW5lX2J5dGVzLCBzaXplIC09
IGRjYWNoZV9saW5lX2J5dGVzICkKICAgICAgICAgYXNtIHZvbGF0aWxlIChfX2NsZWFuX2FuZF9p
bnZhbGlkYXRlX2RjYWNoZV9vbmUoMCkgOiA6ICJyIiAocCArIGlkeCkpOwogICAgIGRzYihzeSk7
ICAgICAgICAgLyogU28gd2Uga25vdyB0aGUgZmx1c2hlcyBoYXBwZW4gYmVmb3JlIGNvbnRpbnVp
bmcgKi8KLSAgICAvKiBBUk0gY2FsbGVycyBhc3N1bWUgdGhhdCBkY2FjaGVfKiBmdW5jdGlvbnMg
Y2Fubm90IGZhaWwuICovCi0gICAgcmV0dXJuIDA7CiB9CiAKIC8qIE1hY3JvcyBmb3IgZmx1c2hp
bmcgYSBzaW5nbGUgc21hbGwgaXRlbS4gIFRoZSBwcmVkaWNhdGUgaXMgYWx3YXlzCkBAIC0yNjYs
NiArMjYwLDIwIEBAIHN0YXRpYyBpbmxpbmUgaW50IGNsZWFuX2FuZF9pbnZhbGlkYXRlX2RjYWNo
ZV92YV9yYW5nZQogICAgICAgICAgICAgOiA6ICJyIiAoX3ApLCAibSIgKCpfcCkpOyAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgXAogfSB3aGlsZSAoMCkKIAorc3RhdGljIGlubGlu
ZSBpbnQgYXJjaF9ncmFudF9jYWNoZV9mbHVzaCh1bnNpZ25lZCBpbnQgb3AsIGNvbnN0IHZvaWQg
KnAsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGxv
bmcgc2l6ZSkKK3sKKyAgICBpZiAoIChvcCAmIEdOVFRBQl9DQUNIRV9JTlZBTCkgJiYgKG9wICYg
R05UVEFCX0NBQ0hFX0NMRUFOKSApCisgICAgICAgIGNsZWFuX2FuZF9pbnZhbGlkYXRlX2RjYWNo
ZV92YV9yYW5nZSh2LCBzaXplKTsKKyAgICBlbHNlIGlmICggb3AgJiBHTlRUQUJfQ0FDSEVfSU5W
QUwgKQorICAgICAgICBpbnZhbGlkYXRlX2RjYWNoZV92YV9yYW5nZSh2LCBzaXplKTsKKyAgICBl
bHNlIGlmICggb3AgJiBHTlRUQUJfQ0FDSEVfQ0xFQU4gKQorICAgICAgICBjbGVhbl9kY2FjaGVf
dmFfcmFuZ2Uodiwgc2l6ZSk7CisKKyAgICAvKiBBUk0gY2FsbGVycyBhc3N1bWUgdGhhdCBkY2Fj
aGVfKiBmdW5jdGlvbnMgY2Fubm90IGZhaWwuICovCisgICAgcmV0dXJuIDA7Cit9CisKIC8qCiAg
KiBXcml0ZSBhIHBhZ2V0YWJsZSBlbnRyeS4KICAqCmRpZmYgLS1naXQgYS94ZW4vYXJjaC94ODYv
aW5jbHVkZS9hc20vZmx1c2h0bGIuaCBiL3hlbi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS9mbHVzaHRs
Yi5oCmluZGV4IGJiMGFkNThkYjQ5Yi4uN2M3MWZlMzc3NzU3IDEwMDY0NAotLS0gYS94ZW4vYXJj
aC94ODYvaW5jbHVkZS9hc20vZmx1c2h0bGIuaAorKysgYi94ZW4vYXJjaC94ODYvaW5jbHVkZS9h
c20vZmx1c2h0bGIuaApAQCAtMTgyLDIxICsxODIsMjEgQEAgdm9pZCBmbHVzaF9hcmVhX21hc2so
Y29uc3QgY3B1bWFza190ICptYXNrLCBjb25zdCB2b2lkICp2YSwKIH0KIAogc3RhdGljIGlubGlu
ZSB2b2lkIGZsdXNoX3BhZ2VfdG9fcmFtKHVuc2lnbmVkIGxvbmcgbWZuLCBib29sIHN5bmNfaWNh
Y2hlKSB7fQotc3RhdGljIGlubGluZSBpbnQgaW52YWxpZGF0ZV9kY2FjaGVfdmFfcmFuZ2UoY29u
c3Qgdm9pZCAqcCwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGxvbmcgc2l6ZSkKLXsgcmV0dXJuIC1FT1BOT1RTVVBQOyB9Ci1zdGF0aWMgaW5s
aW5lIGludCBjbGVhbl9hbmRfaW52YWxpZGF0ZV9kY2FjaGVfdmFfcmFuZ2UoY29uc3Qgdm9pZCAq
cCwKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICB1bnNpZ25lZCBsb25nIHNpemUpCisKK3N0YXRpYyBpbmxpbmUgYXJjaF9ncmFudF9jYWNoZV9m
bHVzaCh1bnNpZ25lZCBpbnQgb3AsIGNvbnN0IHZvaWQgKnAsCisgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBzaXplKQogewotICAgIHVuc2lnbmVkIGlu
dCBvcmRlciA9IGdldF9vcmRlcl9mcm9tX2J5dGVzKHNpemUpOworICAgIHVuc2lnbmVkIGludCBv
cmRlcjsKKworICAgIGlmICggIShjZmx1c2gtPm9wICYgR05UVEFCX0NBQ0hFX0NMRUFOKSApCisg
ICAgICAgIHJldHVybiAtRU9QTk9UU1VQUDsKKworICAgIG9yZGVyID0gZ2V0X29yZGVyX2Zyb21f
Ynl0ZXMoc2l6ZSk7CiAgICAgLyogc3ViLXBhZ2UgZ3JhbnVsYXJpdHkgc3VwcG9ydCBuZWVkcyB0
byBiZSBhZGRlZCBpZiBuZWNlc3NhcnkgKi8KICAgICBmbHVzaF9hcmVhX2xvY2FsKHAsIEZMVVNI
X0NBQ0hFfEZMVVNIX09SREVSKG9yZGVyKSk7CisKICAgICByZXR1cm4gMDsKIH0KLXN0YXRpYyBp
bmxpbmUgaW50IGNsZWFuX2RjYWNoZV92YV9yYW5nZShjb25zdCB2b2lkICpwLCB1bnNpZ25lZCBs
b25nIHNpemUpCi17Ci0gICAgcmV0dXJuIGNsZWFuX2FuZF9pbnZhbGlkYXRlX2RjYWNoZV92YV9y
YW5nZShwLCBzaXplKTsKLX0KIAogdW5zaWduZWQgaW50IGd1ZXN0X2ZsdXNoX3RsYl9mbGFncyhj
b25zdCBzdHJ1Y3QgZG9tYWluICpkKTsKIHZvaWQgZ3Vlc3RfZmx1c2hfdGxiX21hc2soY29uc3Qg
c3RydWN0IGRvbWFpbiAqZCwgY29uc3QgY3B1bWFza190ICptYXNrKTsKZGlmZiAtLWdpdCBhL3hl
bi9jb21tb24vZ3JhbnRfdGFibGUuYyBiL3hlbi9jb21tb24vZ3JhbnRfdGFibGUuYwppbmRleCA1
NzIxZWFiMjI1NjEuLjg2MTVlYTE0NGJiMyAxMDA2NDQKLS0tIGEveGVuL2NvbW1vbi9ncmFudF90
YWJsZS5jCisrKyBiL3hlbi9jb21tb24vZ3JhbnRfdGFibGUuYwpAQCAtMzU3MiwxNCArMzU3Miw3
IEBAIHN0YXRpYyBpbnQgX2NhY2hlX2ZsdXNoKGNvbnN0IGdudHRhYl9jYWNoZV9mbHVzaF90ICpj
Zmx1c2gsIGdyYW50X3JlZl90ICpjdXJfcmVmCiAgICAgdiA9IG1hcF9kb21haW5fcGFnZShtZm4p
OwogICAgIHYgKz0gY2ZsdXNoLT5vZmZzZXQ7CiAKLSAgICBpZiAoIChjZmx1c2gtPm9wICYgR05U
VEFCX0NBQ0hFX0lOVkFMKSAmJiAoY2ZsdXNoLT5vcCAmIEdOVFRBQl9DQUNIRV9DTEVBTikgKQot
ICAgICAgICByZXQgPSBjbGVhbl9hbmRfaW52YWxpZGF0ZV9kY2FjaGVfdmFfcmFuZ2UodiwgY2Zs
dXNoLT5sZW5ndGgpOwotICAgIGVsc2UgaWYgKCBjZmx1c2gtPm9wICYgR05UVEFCX0NBQ0hFX0lO
VkFMICkKLSAgICAgICAgcmV0ID0gaW52YWxpZGF0ZV9kY2FjaGVfdmFfcmFuZ2UodiwgY2ZsdXNo
LT5sZW5ndGgpOwotICAgIGVsc2UgaWYgKCBjZmx1c2gtPm9wICYgR05UVEFCX0NBQ0hFX0NMRUFO
ICkKLSAgICAgICAgcmV0ID0gY2xlYW5fZGNhY2hlX3ZhX3JhbmdlKHYsIGNmbHVzaC0+bGVuZ3Ro
KTsKLSAgICBlbHNlCi0gICAgICAgIHJldCA9IDA7CisgICAgcmV0ID0gYXJjaF9ncmFudF9jYWNo
ZV9mbHVzaChjZmx1c2gtPm9wLCB2LCBjZmx1c2gtPmxlbmd0aCk7CiAKICAgICBpZiAoIGQgIT0g
b3duZXIgKQogICAgIHsK
--=_21dddb0de5c10f3729bd45b19a2db9a4--


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 14:40:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 14:40:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681837.1060807 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1racuF-0004g7-Fr; Thu, 15 Feb 2024 14:40:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681837.1060807; Thu, 15 Feb 2024 14:40:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1racuF-0004g0-CZ; Thu, 15 Feb 2024 14:40:07 +0000
Received: by outflank-mailman (input) for mailman id 681837;
 Thu, 15 Feb 2024 14:40:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DeW9=JY=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1racuE-0004au-G7
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 14:40:06 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1ae3f8d3-cc10-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 15:40:05 +0100 (CET)
Received: from SN7P222CA0007.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:124::17)
 by CH2PR12MB4954.namprd12.prod.outlook.com (2603:10b6:610:63::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.13; Thu, 15 Feb
 2024 14:39:57 +0000
Received: from SN1PEPF0002636C.namprd02.prod.outlook.com
 (2603:10b6:806:124:cafe::77) by SN7P222CA0007.outlook.office365.com
 (2603:10b6:806:124::17) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26 via Frontend
 Transport; Thu, 15 Feb 2024 14:39:56 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SN1PEPF0002636C.mail.protection.outlook.com (10.167.241.137) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 15 Feb 2024 14:39:56 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 15 Feb
 2024 08:39:56 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 15 Feb
 2024 08:39:55 -0600
Received: from XIR-MICHALO-L1.xilinx.com (10.180.168.240) by
 SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35
 via Frontend Transport; Thu, 15 Feb 2024 08:39:54 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ae3f8d3-cc10-11ee-8a4d-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=QjhH/jlyyn7lsM2hFEjAOIX6unOKhMynQ4B46QFz7KXkAfu6Im87S4RDAfDgVQLECefJ3PwN/DYw0q/4CdTIJAFGB62TXmk2cPEtZ4UjMY1kTJhT05wsSguy0kXXpN2o6ITx9qg+lc87ur6OhGgMbfNOYdW4l/t4AQ7toHn4KRPuG7xNdCoU62ne3BnxwxeZmClOh63aU5bYAHXHHhM9fd7yVX4Zjm5hWMT+xG9+pxmQZCtOb4ksZc2BpnCa+AdCRNaRVeQBWcXy97uzchUPIoIs5oToYjRToeTHFCgu3bzGsDQTTpXlhpcOldTgMYulBHD2Rvqi92pdeMkkCvtePQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Pl/4LzIa0JB2UM6kwdo2VC2cy7AYS5i7obHhz//bLcM=;
 b=Jh0uqMVEWT2XJBlg8wx9V9wS8MYk9ODjvxwIB4WYvsWdWhU1aJqXfJqLsEhjyjHU0OTY1KcLLk7IPTepu/ZeF+GAC0H8zSMBwfkphkl3Sk+ijHUMwtTcpmPV98cXlQxDJzWWRwbVUOJK9y7Ag1JTXYFgKxQB9TlBLFQCzIzXW4a7Oinc8/S1u4TRTTpzwZu9m4EDmPR3+g1/CkG7cL4sKGQgEgfgnVGPmMk/aGm2atmxFnsq7N4m8IEcSfcBdFqU/hqQYzWUXbPkyVCEyK0pw2WLjpduF9a0brHsrnEfsCcdB+j8Cx7TFG1Q85ETNrm5EvlpPUg2s03YQWF86Izq8Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Pl/4LzIa0JB2UM6kwdo2VC2cy7AYS5i7obHhz//bLcM=;
 b=eUCrg4TieD9GaxGA3DpGKYaqW9l5K9nU9n9UDVXbbwk5TInbPS/NaN1gR5tsWzpmk86HibynKexHawAhVAFnf80IV6LHMRVS2s7JKbBLyi0MU5hW4W7WkbN7yK9geKzaJeTq/uSGel6xUZj1nNRG9DHu7K/zEjKQUY2zDDmRJsw=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Michal Orzel <michal.orzel@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Michal Orzel <michal.orzel@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH] xen/arm: Make hwdom vUART optional feature
Date: Thu, 15 Feb 2024 15:39:47 +0100
Message-ID: <20240215143947.90073-1-michal.orzel@amd.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF0002636C:EE_|CH2PR12MB4954:EE_
X-MS-Office365-Filtering-Correlation-Id: 461a0367-1a68-4871-5623-08dc2e33fac8
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	P1mNhxAUVHQhaap/eN0S3ZyOWqBJrPoJsZ6cssaCaa3IiWKvd83FOEXWk5kNHSz/r4MFfImCyiurE+WNhvTeuL0LcYJoflNr2DxSIfkY3RnAcKWan5R57nHRl6m8ZYeMi2GNWqlnP8Dh2Tk74SZpC4prcRe/BcFnEuXrADEuJXWI1uUgmHTzB6GelCNpL+D9n1SM9igU89Um0X7CrTMZTTRCc5ghkjpaxYdmoSo8IAUgB0WoyokS/rJUF4RpaeJ6WCuSK/T0bk9GbLuEY4KmXix64Dqj+gjBojP5PluDi9flZcStpfeqnDThFLu6ayd6QVp0IzN5Nxo60MpkVsxFr5ykIooW5xEBS76o7r4gZV/LSbdl3G9sXABcMQQwn2ZY+aaHqMEf1cm3FtMJYXrSZaDvNgkJUphxMiEvj3Osou7xL3JN31TkBeS7oeMTgDjKiEALn6dq2up7M14/td9MaUcdPhpgB2jMZlE74SLtJpJ+4EzxVeH6oI+b3fJ/qD1CMq+r73S8yKhq9jIlbfscHl2gM+ImfCxMI9dNXpn2EsXKQr9hAC0q5awayQaBcobCbhu04rrJg2nqVayMCkHf791ocImCjOKfmDtVhJTK4cY6qk7C81r2SWcbwNDq6f1rrbtehOm0M002f9pO7nDf2EK8SPbBC9CYVsjvFskT6jE=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(39860400002)(396003)(346002)(136003)(230922051799003)(451199024)(1800799012)(82310400011)(64100799003)(186009)(36860700004)(46966006)(40470700004)(5660300002)(44832011)(316002)(54906003)(41300700001)(2906002)(426003)(83380400001)(26005)(1076003)(2616005)(336012)(81166007)(356005)(86362001)(82740400003)(36756003)(6666004)(6916009)(8936002)(70206006)(70586007)(4326008)(8676002)(478600001);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 14:39:56.4785
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 461a0367-1a68-4871-5623-08dc2e33fac8
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF0002636C.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4954

At the moment, the hardware domain vUART is always compiled in. In the
spirit of fine granular configuration, make it optional so that the
feature can be disabled if not needed. This UART is not exposed (e.g.
via device tree) to a domain and is mostly used to support special use
cases like Linux early printk, prints from the decompressor code, etc.

Introduce Kconfig option CONFIG_HWDOM_VUART, enabled by default (to keep
the current behavior) and use it to protect the vUART related code.
Provide stubs for domain_vuart_{init,free}() in case the feature is
disabled. Take the opportunity to add a struct domain forward declaration
to vuart.h, so that the header is self contained.

Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
 xen/arch/arm/Kconfig              |  8 ++++++++
 xen/arch/arm/Makefile             |  2 +-
 xen/arch/arm/include/asm/domain.h |  2 ++
 xen/arch/arm/vuart.h              | 15 +++++++++++++++
 4 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 50e9bfae1ac8..72af329564b7 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -150,6 +150,14 @@ config SBSA_VUART_CONSOLE
 	  Allows a guest to use SBSA Generic UART as a console. The
 	  SBSA Generic UART implements a subset of ARM PL011 UART.
 
+config HWDOM_VUART
+	bool "Emulated UART for hardware domain"
+	default y
+	help
+	  Allows a hardware domain to use a minimalistic UART (single transmit
+	  and status register) which takes information from dtuart. Note that this
+	  UART is not intended to be exposed (e.g. via device-tree) to a domain.
+
 config ARM_SSBD
 	bool "Speculative Store Bypass Disable" if EXPERT
 	depends on HAS_ALTERNATIVE
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 33c677672fe6..7b1350e2ef0a 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -71,7 +71,7 @@ obj-y += vtimer.o
 obj-$(CONFIG_SBSA_VUART_CONSOLE) += vpl011.o
 obj-y += vsmc.o
 obj-y += vpsci.o
-obj-y += vuart.o
+obj-$(CONFIG_HWDOM_VUART) += vuart.o
 
 extra-y += xen.lds
 
diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
index 5fb8cd79c01a..8218afb8626a 100644
--- a/xen/arch/arm/include/asm/domain.h
+++ b/xen/arch/arm/include/asm/domain.h
@@ -91,6 +91,7 @@ struct arch_domain
 
     struct vgic_dist vgic;
 
+#ifdef CONFIG_HWDOM_VUART
     struct vuart {
 #define VUART_BUF_SIZE 128
         char                        *buf;
@@ -98,6 +99,7 @@ struct arch_domain
         const struct vuart_info     *info;
         spinlock_t                  lock;
     } vuart;
+#endif
 
     unsigned int evtchn_irq;
 #ifdef CONFIG_ACPI
diff --git a/xen/arch/arm/vuart.h b/xen/arch/arm/vuart.h
index bd23bd92f631..36658b4a8c7f 100644
--- a/xen/arch/arm/vuart.h
+++ b/xen/arch/arm/vuart.h
@@ -20,9 +20,24 @@
 #ifndef __ARCH_ARM_VUART_H__
 #define __ARCH_ARM_VUART_H__
 
+struct domain;
+
+#ifdef CONFIG_HWDOM_VUART
+
 int domain_vuart_init(struct domain *d);
 void domain_vuart_free(struct domain *d);
 
+#else
+
+static inline int domain_vuart_init(struct domain *d)
+{
+    return 0;
+}
+
+static inline void domain_vuart_free(struct domain *d) {};
+
+#endif /* CONFIG_HWDOM_VUART */
+
 #endif /* __ARCH_ARM_VUART_H__ */
 
 /*
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 15:07:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 15:07:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681846.1060817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1radKW-0007sc-I2; Thu, 15 Feb 2024 15:07:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681846.1060817; Thu, 15 Feb 2024 15:07:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1radKW-0007sV-Eh; Thu, 15 Feb 2024 15:07:16 +0000
Received: by outflank-mailman (input) for mailman id 681846;
 Thu, 15 Feb 2024 15:07:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ebjb=JY=gmail.com=olekstysh@srs-se1.protection.inumbo.net>)
 id 1radKU-0007sO-EW
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 15:07:14 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e540f787-cc13-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 16:07:12 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a34c5ca2537so116943766b.0
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 07:07:11 -0800 (PST)
Received: from EPUAKYIW03DD.. ([91.123.153.74])
 by smtp.gmail.com with ESMTPSA id
 vw3-20020a170907a70300b00a3d99415705sm565868ejc.73.2024.02.15.07.07.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 07:07:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e540f787-cc13-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708009631; x=1708614431; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=5ws6LPjKtIjpFaqXAI/bgM+OTRK7Do4ioHmUfKd4c94=;
        b=lng2qRRWsAj8pbg+/It6MC8V/Jjj07NHoH0aLZbbHgdUN8E/Qtqw8AKR3QHZrdFrnC
         6tO/dayGTkBa7bAmbfSbtN2VCnrY5FFgiU0VPECIPSq/dVqFnS7SSzhE0Njh856tWKTK
         TnQXNgkd4d4Q9QpHso5M3ntwegCvgFXWTR72cGH/ieZbVBIouwKEP1aXcLzfxtmKUwWE
         6eAkaThm6462UKlXiKECCJ2r8OEablrAijPoshEH5CkNIU+mbUDUL7NlbZB5xOax4l37
         92phFQ5s5uGq5vXbSY39aHuh0LP9A7mv5aiWKDM5LWiZGZJ5eiBjRwb1x3yfv+9ISs7a
         EWlA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708009631; x=1708614431;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=5ws6LPjKtIjpFaqXAI/bgM+OTRK7Do4ioHmUfKd4c94=;
        b=eXfHTcC+rfABCouhcDRY5r1HNPkRmk/+Wo+laMQIPF1Q7ZDPtjryHZ6T+xZzj76X/p
         SnX8TTg3WAAkqlfhWykKZTuBF6QK4xsNHD5BqotrkZK9a6SHQpr4zMbvpoM3ww6X/3/l
         0WR0C7/pazkNXLZLhtNE8xvMMNNQwEBegEk+JkQ17H+KD14yqciJyXj8y3BT/ZwK/Grj
         xD7OSEJiOA37nSm3h15v71jx8fGs/F3UDoSzXTM9Q59i6LxYxXpz0OrgqnxebWAvY3+t
         GAGQI9JzDwH7GfGKoJtYKqxRz1sa/yLjxerpNC4mYymm+t1bMvCQsDpqMiUxa7da5qhf
         KXdw==
X-Gm-Message-State: AOJu0YyFegNbKnZiBUtw8F9903ljFWUGoRMviZK7tPSWJ3duDd2KGsZv
	615+T9bmjmQO3A1B9ax3hVEvAppvl5jYLV6IoJ+q5JNzpiNazLzcrzdlkXxe
X-Google-Smtp-Source: AGHT+IExorXP2wMzr3mSfNJI8+mqLSGonJHYGvCsVz+DCdo7EcwCkqU6ApPhrLCVw7dZIPvH2zMzDQ==
X-Received: by 2002:a17:906:aad7:b0:a3c:b23a:9491 with SMTP id kt23-20020a170906aad700b00a3cb23a9491mr1547385ejb.19.1708009630311;
        Thu, 15 Feb 2024 07:07:10 -0800 (PST)
From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Juergen Gross <jgross@suse.com>,
	Viresh Kumar <viresh.kumar@linaro.org>
Subject: [PATCH V3] libxl: Add "grant_usage" parameter for virtio disk devices
Date: Thu, 15 Feb 2024 17:06:44 +0200
Message-Id: <20240215150644.2477378-1-olekstysh@gmail.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Allow administrators to control whether Xen grant mappings for
the virtio disk devices should be used. By default (when new
parameter is not specified), the existing behavior is retained
(we enable grants if backend-domid != 0).

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
In addition to "libxl: arm: Add grant_usage parameter for virtio devices"
https://github.com/xen-project/xen/commit/c14254065ff4826e34f714e1790eab5217368c38

 V2:
  - clarify documentation to match the implementation
  - apply a default value if "grant_usage" is missing the Xenstore
    in libxl__disk_from_xenstore()

 V3:
  - include autogenerated changes to tools/libs/util/libxlu_disk_l.c(h)
  - remove debug log from libxl__disk_from_xenstore(),
    correct coding style
---
 docs/man/xl-disk-configuration.5.pod.in |   24 +
 tools/golang/xenlight/helpers.gen.go    |    6 +
 tools/golang/xenlight/types.gen.go      |    1 +
 tools/include/libxl.h                   |    7 +
 tools/libs/light/libxl_arm.c            |    4 +-
 tools/libs/light/libxl_disk.c           |   13 +
 tools/libs/light/libxl_types.idl        |    1 +
 tools/libs/util/libxlu_disk_l.c         | 1001 ++++++++++++-----------
 tools/libs/util/libxlu_disk_l.h         |    9 +-
 tools/libs/util/libxlu_disk_l.l         |    3 +
 10 files changed, 590 insertions(+), 479 deletions(-)

diff --git a/docs/man/xl-disk-configuration.5.pod.in b/docs/man/xl-disk-configuration.5.pod.in
index cb442bd5b4..986666ebf8 100644
--- a/docs/man/xl-disk-configuration.5.pod.in
+++ b/docs/man/xl-disk-configuration.5.pod.in
@@ -406,6 +406,30 @@ Virtio frontend driver (virtio-blk) to be used. Please note, the virtual
 device (vdev) is not passed to the guest in that case, but it still must be
 specified for the internal purposes.
 
+=item B<grant_usage=BOOLEAN>
+
+=over 4
+
+=item Description
+
+Specifies the usage of Xen grants for accessing guest memory. Only applicable
+to specification "virtio".
+
+=item Supported values
+
+1, 0
+
+=item Mandatory
+
+No
+
+=item Default value
+
+If this option is missing, then the default grant setting will be used,
+i.e. "grant_usage=1" if backend-domid != 0 or "grant_usage=0" otherwise.
+
+=back
+
 =back
 
 =head1 COLO Parameters
diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 0f8e23773c..acdf1c1820 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1885,6 +1885,9 @@ x.ActiveDisk = C.GoString(xc.active_disk)
 x.HiddenDisk = C.GoString(xc.hidden_disk)
 if err := x.Trusted.fromC(&xc.trusted);err != nil {
 return fmt.Errorf("converting field Trusted: %v", err)
+}
+if err := x.GrantUsage.fromC(&xc.grant_usage);err != nil {
+return fmt.Errorf("converting field GrantUsage: %v", err)
 }
 
  return nil}
@@ -1933,6 +1936,9 @@ if x.HiddenDisk != "" {
 xc.hidden_disk = C.CString(x.HiddenDisk)}
 if err := x.Trusted.toC(&xc.trusted); err != nil {
 return fmt.Errorf("converting field Trusted: %v", err)
+}
+if err := x.GrantUsage.toC(&xc.grant_usage); err != nil {
+return fmt.Errorf("converting field GrantUsage: %v", err)
 }
 
  return nil
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 9c8b7b81f6..76b4ed991b 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -741,6 +741,7 @@ ColoExport string
 ActiveDisk string
 HiddenDisk string
 Trusted Defbool
+GrantUsage Defbool
 }
 
 type DeviceNic struct {
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 46bc774126..a370528ba1 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -578,6 +578,13 @@
  */
 #define LIBXL_HAVE_DEVICE_DISK_SPECIFICATION 1
 
+/*
+ * LIBXL_HAVE_DISK_GRANT_USAGE indicates that the libxl_device_disk
+ * has 'grant_usage' field to specify the usage of Xen grants for
+ * the specification 'virtio'.
+ */
+#define LIBXL_HAVE_DISK_GRANT_USAGE 1
+
 /*
  * LIBXL_HAVE_CONSOLE_ADD_XENSTORE indicates presence of the function
  * libxl_console_add_xenstore() in libxl.
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 1539191774..1cb89fa584 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -1372,12 +1372,12 @@ next_resize:
             libxl_device_disk *disk = &d_config->disks[i];
 
             if (disk->specification == LIBXL_DISK_SPECIFICATION_VIRTIO) {
-                if (disk->backend_domid != LIBXL_TOOLSTACK_DOMID)
+                if (libxl_defbool_val(disk->grant_usage))
                     iommu_needed = true;
 
                 FDT( make_virtio_mmio_node(gc, fdt, disk->base, disk->irq,
                                            disk->backend_domid,
-                                           disk->backend_domid != LIBXL_TOOLSTACK_DOMID) );
+                                           libxl_defbool_val(disk->grant_usage)) );
             }
         }
 
diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c
index ee2e4b0bd3..fa7856f28c 100644
--- a/tools/libs/light/libxl_disk.c
+++ b/tools/libs/light/libxl_disk.c
@@ -183,6 +183,9 @@ static int libxl__device_disk_setdefault(libxl__gc *gc, uint32_t domid,
             return ERROR_INVAL;
         }
         disk->transport = LIBXL_DISK_TRANSPORT_MMIO;
+
+        libxl_defbool_setdefault(&disk->grant_usage,
+                                 disk->backend_domid != LIBXL_TOOLSTACK_DOMID);
     }
 
     if (hotplug && disk->specification == LIBXL_DISK_SPECIFICATION_VIRTIO) {
@@ -468,6 +471,8 @@ static void device_disk_add(libxl__egc *egc, uint32_t domid,
             flexarray_append(back, libxl__device_disk_string_of_transport(disk->transport));
             flexarray_append_pair(back, "base", GCSPRINTF("%"PRIu64, disk->base));
             flexarray_append_pair(back, "irq", GCSPRINTF("%u", disk->irq));
+            flexarray_append_pair(back, "grant_usage",
+                                  libxl_defbool_val(disk->grant_usage) ? "1" : "0");
         }
 
         flexarray_append(front, "backend-id");
@@ -662,6 +667,14 @@ static int libxl__disk_from_xenstore(libxl__gc *gc, const char *libxl_path,
             goto cleanup;
         }
         disk->irq = strtoul(tmp, NULL, 10);
+
+        tmp = libxl__xs_read(gc, XBT_NULL,
+                             GCSPRINTF("%s/grant_usage", libxl_path));
+        if (!tmp)
+            libxl_defbool_set(&disk->grant_usage,
+                              disk->backend_domid != LIBXL_TOOLSTACK_DOMID);
+        else
+            libxl_defbool_set(&disk->grant_usage, strtoul(tmp, NULL, 0));
     }
 
     disk->vdev = xs_read(ctx->xsh, XBT_NULL,
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 899ad30969..6d76f25528 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -803,6 +803,7 @@ libxl_device_disk = Struct("device_disk", [
     ("active_disk", string),
     ("hidden_disk", string),
     ("trusted", libxl_defbool),
+    ("grant_usage", libxl_defbool),
     ])
 
 libxl_device_nic = Struct("device_nic", [
diff --git a/tools/libs/util/libxlu_disk_l.c b/tools/libs/util/libxlu_disk_l.c
index 0b59723b71..9213dd957c 100644
--- a/tools/libs/util/libxlu_disk_l.c
+++ b/tools/libs/util/libxlu_disk_l.c
@@ -1,7 +1,10 @@
 #line 1 "libxlu_disk_l.c"
+#line 31 "libxlu_disk_l.l"
 #define _GNU_SOURCE
 
-#line 4 "libxlu_disk_l.c"
+
+
+#line 7 "libxlu_disk_l.c"
 
 #define  YY_INT_ALIGNED short int
 
@@ -549,8 +552,8 @@ static void yynoreturn yy_fatal_error ( const char* msg , yyscan_t yyscanner );
 	yyg->yy_hold_char = *yy_cp; \
 	*yy_cp = '\0'; \
 	yyg->yy_c_buf_p = yy_cp;
-#define YY_NUM_RULES 37
-#define YY_END_OF_BUFFER 38
+#define YY_NUM_RULES 41
+#define YY_END_OF_BUFFER 42
 /* This struct is not used in this scanner,
    but its presence is necessary. */
 struct yy_trans_info
@@ -558,119 +561,127 @@ struct yy_trans_info
 	flex_int32_t yy_verify;
 	flex_int32_t yy_nxt;
 	};
-static const flex_int16_t yy_acclist[594] =
+static const flex_int16_t yy_acclist[634] =
     {   0,
-       36,   36,   38,   34,   35,   37, 8193,   34,   35,   37,
-    16385, 8193,   34,   37,16385,   34,   35,   37,   35,   37,
-       34,   35,   37,   34,   35,   37,   34,   35,   37,   34,
-       35,   37,   34,   35,   37,   34,   35,   37,   34,   35,
-       37,   34,   35,   37,   34,   35,   37,   34,   35,   37,
-       34,   35,   37,   34,   35,   37,   34,   35,   37,   34,
-       35,   37,   34,   35,   37,   34,   35,   37,   36,   37,
-       37,   34,   34, 8193,   34, 8193,   34,16385, 8193,   34,
-     8193,   34,   34, 8225,   34,16417,   34,   34,   34,   34,
-       34,   34,   34,   34,   34,   34,   34,   34,   34,   34,
-
-       34,   34,   34,   34,   34,   34,   34,   34,   34,   34,
-       36, 8193,   34, 8193,   34, 8193, 8225,   34, 8225,   34,
-     8225,   24,   34,   34,   34,   34,   34,   34,   34,   34,
-       34,   34,   34,   34,   34,   34,   34,   34,   34,   34,
-       34,   34,   34,   34,   34,   34,   34, 8225,   34, 8225,
-       34, 8225,   24,   34,   34,   29, 8225,   34,16417,   34,
-       34,   16,   34,   34,   34,   34,   34,   34,   34,   34,
-       34, 8218, 8225,   34,16410,16417,   34,   34,   32, 8225,
-       34,16417,   34, 8217, 8225,   34,16409,16417,   34,   34,
-       34, 8220, 8225,   34,16412,16417,   34,   34,   34,   34,
-
-       34,   29, 8225,   34,   29, 8225,   34,   29,   34,   29,
-     8225,   34,    3,   34,   16,   34,   34,   34,   34,   34,
-       31, 8225,   34,16417,   34,   34,   34, 8218, 8225,   34,
-     8218, 8225,   34, 8218,   34, 8218, 8225,   34,   34,   32,
-     8225,   34,   32, 8225,   34,   32,   34,   32, 8225, 8217,
-     8225,   34, 8217, 8225,   34, 8217,   34, 8217, 8225,   34,
-       34, 8220, 8225,   34, 8220, 8225,   34, 8220,   34, 8220,
-     8225,   34,   34,   11,   34,   34,   29, 8225,   34,   29,
-     8225,   34,   29, 8225,   29,   34,   29,   34,    3,   34,
-       34,   34,   34,   34,   34,   34,   31, 8225,   34,   31,
-
-     8225,   34,   31,   34,   31, 8225,   34,   34,   30, 8225,
-       34,16417, 8218, 8225,   34, 8218, 8225,   34, 8218, 8225,
-     8218,   34, 8218,   34,   34,   32, 8225,   34,   32, 8225,
-       34,   32, 8225,   32,   34,   32, 8217, 8225,   34, 8217,
-     8225,   34, 8217, 8225, 8217,   34, 8217,   34,   34, 8220,
-     8225,   34, 8220, 8225,   34, 8220, 8225, 8220,   34, 8220,
-       34,   34,   11,   24,   11,    7,   34,   34,   34,   34,
-       34,   34,   34,   14,   34,   31, 8225,   34,   31, 8225,
-       34,   31, 8225,   31,   34,   31,    2,   34,   30, 8225,
-       34,   30, 8225,   34,   30,   34,   30, 8225,   17,   34,
-
-       34,   12,   34,   34,   23,   11,   11,   24,    7,   24,
-        7,   34,    8,   34,   34,   34,   34,    6,   34,   14,
-       34,    2,   24,    2,   34,   30, 8225,   34,   30, 8225,
-       34,   30, 8225,   30,   34,   30,   17,   34,   34,   12,
-       24,   12,   34,   27, 8225,   34,16417,   23,   24,   23,
-        7,    7,   24,   34,    8,   24,    8,   34,   34,   34,
-       34,    6,   24,    6,    6,   24,    6,   24,   34,    2,
-        2,   24,   34,   34,   12,   12,   24,   34,   27, 8225,
-       34,   27, 8225,   34,   27,   34,   27, 8225,   23,   24,
-       34,    8,    8,   24,   34,   34,   18,   19,    6,    6,
-
-       24,    6,    6,   34,   34,   15,   34,   34,   27, 8225,
-       34,   27, 8225,   34,   27, 8225,   27,   34,   27,   34,
-       34,   34,   18,   24,   18,   19,   24,   19,    6,    6,
-       34,   34,   15,   34,   34,   21,    9,   20,   18,   18,
-       24,   19,   19,   24,    6,    5,    6,   34,   22,   34,
-       21,   24,   21,    9,   24,    9,   20,   24,   20,    4,
-        6,    5,    6,   34,   22,   24,   22,   34,   21,   21,
-       24,    9,    9,   24,   20,   20,   24,    4,    6,   13,
-       34,   22,   22,   24,   10,   13,   34,   10,   24,   10,
-       10,   10,   24
-
+       40,   40,   42,   38,   39,   41, 8193,   38,   39,   41,
+    16385, 8193,   38,   41,16385,   38,   39,   41,   39,   41,
+       38,   39,   41,   38,   39,   41,   38,   39,   41,   38,
+       39,   41,   38,   39,   41,   38,   39,   41,   38,   39,
+       41,   38,   39,   41,   38,   39,   41,   38,   39,   41,
+       38,   39,   41,   38,   39,   41,   38,   39,   41,   38,
+       39,   41,   38,   39,   41,   38,   39,   41,   38,   39,
+       41,   38,   39,   41,   40,   41,   41,   38,   38, 8193,
+       38, 8193,   38,16385, 8193,   38, 8193,   38,   38, 8229,
+       38,16421,   38,   38,   38,   38,   38,   38,   38,   38,
+
+       38,   38,   38,   38,   38,   38,   38,   38,   38,   38,
+       38,   38,   38,   38,   38,   38,   38,   38,   38,   40,
+     8193,   38, 8193,   38, 8193, 8229,   38, 8229,   38, 8229,
+       28,   38,   38,   38,   38,   38,   38,   38,   38,   38,
+       38,   38,   38,   38,   38,   38,   38,   38,   38,   38,
+       38,   38,   38,   38,   38,   38,   38,   38,   38, 8229,
+       38, 8229,   38, 8229,   28,   38,   38,   33, 8229,   38,
+    16421,   38,   38,   16,   38,   38,   38,   38,   38,   38,
+       38,   38,   38,   38, 8222, 8229,   38,16414,16421,   38,
+       38,   36, 8229,   38,16421,   38, 8221, 8229,   38,16413,
+
+    16421,   38,   38,   38, 8224, 8229,   38,16416,16421,   38,
+       38,   38,   38,   38,   38,   38,   33, 8229,   38,   33,
+     8229,   38,   33,   38,   33, 8229,   38,    3,   38,   16,
+       38,   38,   38,   38,   38,   35, 8229,   38,16421,   38,
+       38,   38,   38, 8222, 8229,   38, 8222, 8229,   38, 8222,
+       38, 8222, 8229,   38,   38,   36, 8229,   38,   36, 8229,
+       38,   36,   38,   36, 8229, 8221, 8229,   38, 8221, 8229,
+       38, 8221,   38, 8221, 8229,   38,   38, 8224, 8229,   38,
+     8224, 8229,   38, 8224,   38, 8224, 8229,   38,   38,   38,
+       38,   11,   38,   38,   33, 8229,   38,   33, 8229,   38,
+
+       33, 8229,   33,   38,   33,   38,    3,   38,   38,   38,
+       38,   38,   38,   38,   35, 8229,   38,   35, 8229,   38,
+       35,   38,   35, 8229,   38,   38,   38,   34, 8229,   38,
+    16421, 8222, 8229,   38, 8222, 8229,   38, 8222, 8229, 8222,
+       38, 8222,   38,   38,   36, 8229,   38,   36, 8229,   38,
+       36, 8229,   36,   38,   36, 8221, 8229,   38, 8221, 8229,
+       38, 8221, 8229, 8221,   38, 8221,   38,   38, 8224, 8229,
+       38, 8224, 8229,   38, 8224, 8229, 8224,   38, 8224,   38,
+       38,   38,   38,   11,   28,   11,    7,   38,   38,   38,
+       38,   38,   38,   38,   14,   38,   35, 8229,   38,   35,
+
+     8229,   38,   35, 8229,   35,   38,   35,    2,   38,   38,
+       34, 8229,   38,   34, 8229,   38,   34,   38,   34, 8229,
+       17,   38,   38,   12,   38,   38,   27,   23,   38,   38,
+       11,   11,   28,    7,   28,    7,   38,    8,   38,   38,
+       38,   38,    6,   38,   14,   38,    2,   28,    2,   38,
+       38,   34, 8229,   38,   34, 8229,   38,   34, 8229,   34,
+       38,   34,   17,   38,   38,   12,   28,   12,   38,   31,
+     8229,   38,16421,   27,   28,   27,   23,   38,   38,    7,
+        7,   28,   38,    8,   28,    8,   38,   38,   38,   38,
+        6,   28,    6,    6,   28,    6,   28,   38,    2,    2,
+
+       28,   38,   38,   38,   12,   12,   28,   38,   31, 8229,
+       38,   31, 8229,   38,   31,   38,   31, 8229,   27,   28,
+       24,   38,   38,    8,    8,   28,   38,   38,   18,   19,
+        6,    6,   28,    6,    6,   38,   38,   38,   15,   38,
+       38,   31, 8229,   38,   31, 8229,   38,   31, 8229,   31,
+       38,   31,   24,   38,   38,   38,   38,   18,   28,   18,
+       19,   28,   19,    6,    6,   38,   38,   38,   15,   38,
+       38,   21,    9,   20,   18,   18,   28,   19,   19,   28,
+        6,    5,    6,   38,   22,   38,   21,   28,   21,    9,
+       28,    9,   20,   28,   20,    4,    6,    5,    6,   38,
+
+       26,   25,   22,   28,   22,   38,   21,   21,   28,    9,
+        9,   28,   20,   20,   28,    4,    6,   13,   38,   26,
+       25,   22,   22,   28,   10,   13,   38,   10,   28,   10,
+       10,   10,   28
     } ;
 
-static const flex_int16_t yy_accept[373] =
+static const flex_int16_t yy_accept[406] =
     {   0,
         1,    1,    1,    2,    3,    4,    7,   12,   16,   19,
        21,   24,   27,   30,   33,   36,   39,   42,   45,   48,
-       51,   54,   57,   60,   63,   66,   69,   71,   72,   73,
-       74,   76,   79,   81,   82,   83,   84,   87,   87,   88,
-       89,   90,   91,   92,   93,   94,   95,   96,   97,   98,
-       99,  100,  101,  102,  103,  104,  105,  106,  107,  108,
-      109,  110,  111,  112,  114,  116,  117,  119,  121,  122,
-      123,  124,  125,  126,  127,  128,  129,  130,  131,  132,
-      133,  134,  135,  136,  137,  138,  139,  140,  141,  142,
-      143,  144,  145,  146,  147,  148,  150,  152,  153,  154,
-
-      155,  156,  160,  161,  162,  164,  165,  166,  167,  168,
-      169,  170,  171,  172,  177,  178,  179,  183,  184,  189,
-      190,  191,  192,  197,  198,  199,  200,  201,  202,  205,
-      208,  210,  212,  213,  215,  217,  218,  219,  220,  221,
-      225,  226,  227,  228,  231,  234,  236,  238,  239,  240,
-      243,  246,  248,  250,  253,  256,  258,  260,  261,  262,
-      265,  268,  270,  272,  273,  274,  275,  276,  277,  280,
-      283,  285,  287,  288,  289,  291,  292,  293,  294,  295,
-      296,  297,  300,  303,  305,  307,  308,  309,  313,  316,
-      319,  321,  323,  324,  325,  326,  329,  332,  334,  336,
-
-      337,  340,  343,  345,  347,  348,  349,  350,  353,  356,
-      358,  360,  361,  362,  363,  365,  366,  367,  368,  369,
-      370,  371,  372,  373,  374,  376,  379,  382,  384,  386,
-      387,  388,  389,  392,  395,  397,  399,  401,  402,  403,
-      404,  405,  406,  407,  409,  411,  412,  413,  414,  415,
-      416,  417,  418,  419,  420,  422,  424,  425,  426,  429,
-      432,  434,  436,  437,  439,  440,  442,  443,  444,  448,
-      450,  451,  452,  454,  455,  457,  458,  459,  460,  461,
-      462,  464,  465,  467,  469,  470,  471,  473,  474,  475,
-      476,  478,  479,  482,  485,  487,  489,  491,  492,  493,
-
-      495,  496,  497,  498,  499,  500,  502,  503,  504,  505,
-      506,  508,  509,  512,  515,  517,  519,  520,  521,  522,
-      523,  525,  526,  528,  529,  530,  531,  532,  533,  535,
-      536,  537,  538,  539,  540,  542,  543,  545,  546,  548,
-      549,  550,  551,  553,  554,  556,  557,  559,  560,  562,
-      564,  565,  567,  568,  569,  570,  572,  573,  575,  576,
-      578,  580,  582,  583,  585,  586,  588,  590,  591,  592,
-      594,  594
+       51,   54,   57,   60,   63,   66,   69,   72,   75,   77,
+       78,   79,   80,   82,   85,   87,   88,   89,   90,   93,
+       93,   94,   95,   96,   97,   98,   99,  100,  101,  102,
+      103,  104,  105,  106,  107,  108,  109,  110,  111,  112,
+      113,  114,  115,  116,  117,  118,  119,  120,  121,  123,
+      125,  126,  128,  130,  131,  132,  133,  134,  135,  136,
+      137,  138,  139,  140,  141,  142,  143,  144,  145,  146,
+      147,  148,  149,  150,  151,  152,  153,  154,  155,  156,
+
+      157,  158,  159,  160,  162,  164,  165,  166,  167,  168,
+      172,  173,  174,  176,  177,  178,  179,  180,  181,  182,
+      183,  184,  185,  190,  191,  192,  196,  197,  202,  203,
+      204,  205,  210,  211,  212,  213,  214,  215,  216,  217,
+      220,  223,  225,  227,  228,  230,  232,  233,  234,  235,
+      236,  240,  241,  242,  243,  244,  247,  250,  252,  254,
+      255,  256,  259,  262,  264,  266,  269,  272,  274,  276,
+      277,  278,  281,  284,  286,  288,  289,  290,  291,  292,
+      293,  294,  295,  298,  301,  303,  305,  306,  307,  309,
+      310,  311,  312,  313,  314,  315,  318,  321,  323,  325,
+
+      326,  327,  328,  332,  335,  338,  340,  342,  343,  344,
+      345,  348,  351,  353,  355,  356,  359,  362,  364,  366,
+      367,  368,  369,  372,  375,  377,  379,  380,  381,  382,
+      383,  384,  386,  387,  388,  389,  390,  391,  392,  393,
+      394,  395,  397,  400,  403,  405,  407,  408,  409,  410,
+      411,  414,  417,  419,  421,  423,  424,  425,  426,  427,
+      428,  430,  431,  432,  434,  436,  437,  438,  439,  440,
+      441,  442,  443,  444,  445,  447,  449,  450,  451,  452,
+      455,  458,  460,  462,  463,  465,  466,  468,  469,  470,
+      474,  476,  477,  479,  480,  481,  483,  484,  486,  487,
+
+      488,  489,  490,  491,  493,  494,  496,  498,  499,  500,
+      502,  503,  504,  505,  506,  508,  509,  512,  515,  517,
+      519,  521,  523,  524,  525,  527,  528,  529,  530,  531,
+      532,  534,  535,  536,  537,  538,  539,  541,  542,  545,
+      548,  550,  552,  553,  555,  556,  557,  558,  560,  561,
+      563,  564,  565,  566,  567,  568,  569,  571,  572,  573,
+      574,  575,  576,  578,  579,  581,  582,  584,  585,  585,
+      586,  587,  589,  590,  592,  593,  595,  596,  598,  600,
+      601,  602,  603,  605,  606,  607,  608,  610,  611,  613,
+      614,  616,  618,  620,  621,  622,  623,  625,  626,  628,
+
+      630,  631,  632,  634,  634
     } ;
 
 static const YY_CHAR yy_ec[256] =
@@ -679,16 +690,16 @@ static const YY_CHAR yy_ec[256] =
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
         1,    2,    1,    1,    1,    1,    1,    1,    1,    1,
-        1,    1,    1,    4,    5,    1,    1,    6,    6,    7,
-        6,    6,    6,    6,    6,    6,    6,    8,    1,    1,
-        9,    1,    1,    1,    1,    1,    1,    1,    1,    1,
+        1,    1,    1,    4,    5,    1,    1,    6,    7,    8,
+        9,    9,    9,    9,    9,    9,    9,   10,    1,    1,
+       11,    1,    1,    1,    1,    1,    1,    1,    1,    1,
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
-        1,    1,    1,    1,    1,    1,   10,   11,   12,   13,
+        1,    1,    1,    1,   12,    1,   13,   14,   15,   16,
 
-       14,   15,   16,   17,   18,   19,   20,   21,   22,   23,
-       24,   25,   26,   27,   28,   29,   30,   31,   32,   33,
-       34,   19,    1,    1,    1,    1,    1,    1,    1,    1,
+       17,   18,   19,   20,   21,   22,   23,   24,   25,   26,
+       27,   28,   29,   30,   31,   32,   33,   34,   35,   36,
+       37,   22,    1,    1,    1,    1,    1,    1,    1,    1,
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
@@ -705,340 +716,362 @@ static const YY_CHAR yy_ec[256] =
         1,    1,    1,    1,    1
     } ;
 
-static const YY_CHAR yy_meta[35] =
+static const YY_CHAR yy_meta[38] =
     {   0,
-        1,    1,    2,    3,    1,    1,    1,    1,    4,    1,
+        1,    1,    2,    3,    1,    1,    1,    1,    1,    1,
+        4,    1,    1,    1,    1,    1,    1,    1,    1,    1,
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
-        1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
-        1,    1,    1,    1
+        1,    1,    1,    1,    1,    1,    1
     } ;
 
-static const flex_int16_t yy_base[443] =
+static const flex_int16_t yy_base[476] =
     {   0,
-        0,    0,  936,  935,  937,  932,   33,   36,  940,  940,
-       45,   63,   31,   42,   51,   52,  925,   33,   65,   67,
-       69,   70,  924,   71,  923,   75,    0,  940,  928,  940,
-       91,   95,    0,    0,  104,  921,  113,    0,   91,   99,
-      114,   92,  115,   80,  100,   48,  119,  121,  122,   74,
-      123,  128,  131,  129,  125,  133,  135,  136,  137,  143,
-      138,  145,    0,  157,    0,    0,  168,    0,    0,  926,
-      140,  146,  165,  159,  152,  164,  155,  168,  171,  176,
-      177,  170,  180,  175,  184,  188,  212,  191,  185,  192,
-      193,  194,  219,  212,  199,  230,    0,    0,  940,  195,
-
-      200,  239,  235,  197,  246,  225,  226,  919,  244,  918,
-      243,  236,  245,  266,  248,  264,  282,  271,  291,  248,
-      270,  254,  300,  279,  296,  302,  288,  303,  311,    0,
-      315,    0,  311,  318,  940,  313,  319,  208,  313,  344,
-      321,  331,  325,  333,    0,  352,    0,  345,  347,  359,
-        0,  361,    0,  368,    0,  370,    0,  322,  366,  379,
-        0,  381,    0,  359,  357,  923,  382,  384,  392,    0,
-        0,    0,    0,  387,  940,  386,  390,  392,  329,  401,
-      397,  409,    0,  417,    0,  399,  412,  426,  429,    0,
-        0,    0,    0,  412,  427,  438,    0,    0,    0,    0,
-
-      440,    0,    0,    0,    0,  436,  405,  447,    0,    0,
-        0,    0,  438,  443,  922,  940,  921,  442,  450,  449,
-      452,  454,  459,  458,  453,  469,    0,    0,    0,    0,
-      920,  470,  481,    0,  483,    0,  484,  481,  919,  368,
-      467,  495,  918,  940,  917,  940,  488,  916,  479,  490,
-      492,  495,  505,  498,  940,  915,  940,  507,  523,    0,
-        0,    0,    0,  940,  503,  864,  940,  846,  532,  836,
-        0,  824,  940,  516,  796,  940,  513,  530,  536,  538,
-      784,  940,  542,  535,  547,  772,  940,  549,  551,  768,
-      940,  502,  562,    0,  564,    0,    0,  562,  764,  940,
-
-      544,  557,  760,  752,  744,  940,  552,  568,  571,  568,
-      581,  577,  588,    0,    0,    0,    0,  589,  580,  591,
-      736,  940,  728,  940,  601,  602,  597,  599,  940,  603,
-      720,  712,  700,  672,  940,  665,  940,  610,  656,  603,
-      648,  607,  629,  940,  627,  940,  625,  940,  624,  940,
-      607,  574,  940,  614,  572,  940,  491,  940,  433,  940,
-      940,  622,  389,  940,  303,  940,  261,  940,  204,  940,
-      940,  635,  639,  642,  646,  650,  654,  658,  662,  666,
-      670,  674,  678,  682,  686,  690,  694,  698,  702,  706,
-      710,  714,  718,  722,  726,  730,  734,  738,  742,  746,
-
-      750,  754,  758,  762,  766,  770,  774,  778,  782,  786,
-      790,  794,  798,  802,  806,  810,  814,  818,  822,  826,
-      830,  834,  838,  842,  846,  850,  854,  858,  862,  866,
-      870,  874,  878,  882,  886,  890,  894,  898,  902,  906,
-      910,  914
+        0,    0, 1008, 1007, 1009, 1004,   36,   39, 1012, 1012,
+       50,   71,   32,   72,   31,   43,   33,   45,   64,  994,
+       65,   67,   54,  993,   73,   77,   83,   84,    0, 1012,
+     1001, 1012,   95,  108,    0,    0,  119,  991,  131,    0,
+      112,   81,   90,  115,  100,  103,  125,  101,  104,  126,
+      127,  990,  133,  134,  139,  145,  154,  135,  147,  140,
+      149,  151,  150,  152,   36,  153,  160,    0,  178,    0,
+        0,  180,    0,    0,  998,  173,  156,  182,  174,  175,
+      180,  181,  183,  186,  190,  191,  185,  192,  196,  198,
+      201,  206,  232,  207,  202,  211,  209,  212,  241,  226,
+
+      215,  225,  216,  249,    0,    0, 1012,  227,  228,  262,
+      258,  243,  265,  246,  264,  988,  271,  987,  248,  269,
+      263,  272,  296,  275,  273,  308,  313,  326,  304,  274,
+      312,  338,  303,  334,  301,  302,  292,  321,  340,  351,
+        0,  355,    0,  343,  357, 1012,  348,  332,  354,  358,
+      388,  362,  360,  361,  367,  392,    0,  398,    0,  389,
+      366,  403,    0,  407,    0,  412,    0,  416,    0,  399,
+      213,  418,    0,  421,    0,  395,  404,  366,  415,  995,
+      333,  425,  430,    0,    0,    0,    0,  424, 1012,  427,
+      429,  432,  422,  444,  437,  451,    0,  457,    0,  453,
+
+      447,  460,  480,  466,    0,    0,    0,    0,  459,  369,
+      468,    0,    0,    0,    0,  484,    0,    0,    0,    0,
+      465,  476,  490,    0,    0,    0,    0,  483,  487,  486,
+      490,  994, 1012,  993,  489,  496,  495,  498,  500,  506,
+      505,  454,  521,    0,    0,    0,    0,  992,  504,  518,
+      533,    0,  535,    0,  527,  532,  991,  536,  540,  537,
+      514,  539,  990, 1012,  989, 1012,  539,  988,  544,  543,
+      546,  548,  561,  552, 1012,  987, 1012,  562,  567,  581,
+        0,    0,    0,    0, 1012,  556,  986, 1012,  976,  592,
+      984,    0, 1012,  574,  983, 1012,  558,  982, 1012,  578,
+
+      583,  594,  596,  981, 1012,  288,  596,  603,  930, 1012,
+      597,  599,  605,  918, 1012,  609,  620,    0,  622,    0,
+        0,  623,  619,  902, 1012,  615,  622,  890,  862,  850,
+     1012,  625,  635,  628,  636,  633,  642,  639,  658,    0,
+        0,    0,    0, 1012,  652,  634,  659,  838, 1012,  834,
+     1012,  646,  660,  663,  661,  666, 1012,  658,  830,  826,
+      818,  810, 1012,  802, 1012,  676,  794,  671,  684,  786,
+      671,  778, 1012,  766, 1012,  738, 1012,  731, 1012,  677,
+      722,  714,  691, 1012,  685,  647, 1012,  565, 1012,  508,
+     1012, 1012,  684, 1012, 1012,  448, 1012,  374, 1012,  167,
+
+     1012,  122, 1012, 1012,  701,  705,  708,  712,  716,  720,
+      724,  728,  732,  736,  740,  744,  748,  752,  756,  760,
+      764,  768,  772,  776,  780,  784,  788,  792,  796,  800,
+      804,  808,  812,  816,  820,  824,  828,  832,  836,  840,
+      844,  848,  852,  856,  860,  864,  868,  872,  876,  880,
+      884,  888,  892,  896,  900,  904,  908,  912,  916,  920,
+      924,  928,  932,  936,  940,  944,  948,  952,  956,  960,
+      964,  968,  972,  976,  980
     } ;
 
-static const flex_int16_t yy_def[443] =
+static const flex_int16_t yy_def[476] =
     {   0,
-      371,    1,  372,  372,  371,  373,  374,  374,  371,  371,
-      375,  375,   12,   12,   12,   12,   12,   12,   12,   12,
-       12,   12,   12,   12,   12,   12,  376,  371,  373,  371,
-      377,  374,  378,  378,  379,   12,  373,  380,   12,   12,
+      404,    1,  405,  405,  404,  406,  407,  407,  404,  404,
+      408,  408,   12,   12,   12,   12,   12,   12,   12,   12,
+       12,   12,   12,   12,   12,   12,   12,   12,  409,  404,
+      406,  404,  410,  407,  411,  411,  412,   12,  406,  413,
+       12,   12,   12,   12,   12,   12,   12,   12,   12,   12,
        12,   12,   12,   12,   12,   12,   12,   12,   12,   12,
+       12,   12,   12,   12,   12,   12,   12,  409,  410,  411,
+      411,  414,  415,  415,  404,   12,   12,   12,   12,   12,
        12,   12,   12,   12,   12,   12,   12,   12,   12,   12,
-       12,   12,  376,  377,  378,  378,  381,  382,  382,  371,
+       12,   12,  412,   12,   12,   12,   12,   12,   12,   12,
+
+       12,   12,   12,  414,  415,  415,  404,   12,   12,  416,
        12,   12,   12,   12,   12,   12,   12,   12,   12,   12,
-       12,   12,   12,   12,   12,   12,  379,   12,   12,   12,
-       12,   12,   12,   12,   12,  381,  382,  382,  371,   12,
-
-       12,  383,   12,   12,   12,   12,   12,   12,   12,   12,
-       12,   12,   12,  384,   87,   87,  385,   12,  386,   12,
-       12,   12,  387,   12,   12,   12,   12,   12,  388,  389,
-      383,  389,   12,   12,  371,   87,   12,   12,   12,  390,
-       12,   12,   12,  391,  392,  384,  392,   87,   87,  393,
-      394,  385,  394,  395,  396,  386,  396,   12,   12,  397,
-      398,  387,  398,   12,   12,  399,   12,   12,  388,  389,
-      389,  400,  400,   12,  371,   87,   87,   87,   12,   12,
-       12,  401,  402,  390,  402,   12,   12,  403,  391,  392,
-      392,  404,  404,   87,   87,  393,  394,  394,  405,  405,
-
-      395,  396,  396,  406,  406,   12,   12,  397,  398,  398,
-      407,  407,   12,   12,  408,  371,  409,   87,   12,   87,
-       87,   87,   12,   87,   12,  401,  402,  402,  410,  410,
-      411,   87,  412,  413,  403,  413,   87,   87,  414,   12,
-       12,  415,  408,  371,  416,  371,   87,  417,   12,   87,
-       87,   87,  418,   87,  371,  419,  371,   87,  412,  413,
-      413,  420,  420,  371,   87,  421,  371,   12,  422,  423,
-      423,  416,  371,   87,  424,  371,   12,   87,   87,   87,
-      425,  371,  425,  425,   87,  419,  371,   87,   87,  421,
-      371,   12,  426,  427,  422,  427,  423,   87,  424,  371,
-
-       12,   87,  428,  429,  425,  371,  425,  425,   87,   87,
-       87,   12,  426,  427,  427,  430,  430,   87,   12,   87,
-      431,  371,  432,  371,  425,  425,   87,   87,  371,   12,
-      433,  434,  435,  431,  371,  432,  371,  425,  425,   87,
-      436,   12,  437,  371,  438,  371,  439,  371,  425,  371,
-       87,  440,  371,   12,  437,  371,  438,  371,  439,  371,
-      371,   87,  440,  371,  441,  371,  442,  371,  442,  371,
-        0,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371
+       12,   12,  417,   93,   93,  418,   12,  419,   12,   12,
+       12,  420,   12,   12,   12,   12,   12,   12,   12,  421,
+      422,  416,  422,   12,   12,  404,   93,   12,   12,   12,
+      423,   12,   12,   12,   12,  424,  425,  417,  425,   93,
+       93,  426,  427,  418,  427,  428,  429,  419,  429,   12,
+       12,  430,  431,  420,  431,   12,   12,   12,   12,  432,
+       12,   12,  421,  422,  422,  433,  433,   12,  404,   93,
+       93,   93,   12,   12,   12,  434,  435,  423,  435,   12,
+
+      406,   12,  436,  424,  425,  425,  437,  437,   93,   93,
+      426,  427,  427,  438,  438,  428,  429,  429,  439,  439,
+       12,   12,  430,  431,  431,  440,  440,   12,   12,   12,
+       12,  441,  404,  442,   93,   12,   93,   93,   93,   12,
+       93,   12,  434,  435,  435,  443,  443,  444,  406,   93,
+      445,  446,  436,  446,   93,   93,  447,   12,   12,  448,
+       12,   12,  441,  404,  449,  404,   93,  450,   12,   93,
+       93,   93,  451,   93,  404,  452,  404,  406,   93,  445,
+      446,  446,  453,  453,  404,   93,  454,  404,   12,  455,
+      456,  456,  404,   12,  449,  404,   93,  457,  404,   12,
+
+       93,   93,   93,  458,  404,  458,  458,   93,  452,  404,
+      406,   93,   93,  454,  404,   12,  459,  460,  455,  460,
+      456,   12,   93,  457,  404,   12,   93,  461,  462,  458,
+      404,  458,  458,   93,  406,   93,   93,   12,  459,  460,
+      460,  463,  463,  404,   93,   12,   93,  464,  404,  465,
+      404,  458,  458,   93,  406,   93,  404,   12,  466,  467,
+      468,  464,  404,  465,  404,  458,  458,   93,  404,  469,
+       12,  470,  404,  471,  404,  472,  404,  458,  404,   93,
+      404,  404,  473,  404,   12,  470,  404,  471,  404,  472,
+      404,  404,   93,  404,  404,  473,  404,  474,  404,  475,
+
+      404,  475,  404,    0,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404
     } ;
 
-static const flex_int16_t yy_nxt[975] =
+static const flex_int16_t yy_nxt[1050] =
     {   0,
-        6,    7,    8,    9,    6,    6,    6,    6,   10,   11,
-       12,   13,   14,   15,   16,   17,   18,   19,   17,   17,
-       17,   17,   20,   17,   21,   22,   23,   24,   25,   17,
-       26,   17,   17,   17,   32,   32,   33,   32,   32,   33,
-       36,   34,   36,   42,   34,   29,   29,   29,   30,   35,
-       50,   36,   37,   38,   43,   44,   39,   36,   80,   45,
-       36,   36,   40,   29,   29,   29,   30,   35,   46,   48,
-       37,   38,   41,   47,   36,   49,   36,   53,   36,   36,
-       36,   56,   58,   36,   36,   55,   83,   61,   51,   36,
-       54,   62,   52,   29,   65,   59,   32,   32,   33,   66,
-
-       36,   36,   71,   34,   29,   29,   29,   30,   36,   36,
-       77,   29,   38,   67,   67,   67,   68,   67,   75,   72,
-       67,   69,   73,   36,   36,   74,   78,   79,   36,   53,
-       36,   36,   36,   87,   36,   76,   84,   36,   36,   85,
-       36,   81,   36,   86,   36,   36,   36,   36,   82,   36,
-       92,   95,   36,  100,   36,   36,   89,   90,   88,   29,
-       65,   36,   91,  101,   36,   66,   90,   93,   36,   94,
-       29,   97,  102,   36,   36,  104,   98,   36,  103,   36,
-       36,  107,  108,  106,   36,   36,   36,  105,   86,   36,
-      109,  110,  111,   36,   36,  114,  112,   36,  117,  119,
-
-       36,   36,   36,   36,   36,  121,   36,  368,   36,   36,
-      120,  113,   29,   29,   29,   30,  118,   36,  134,   29,
-       38,   36,  127,  115,  116,  122,  123,  125,   36,  126,
-      128,  124,   29,   97,   36,   36,  180,  138,   98,  129,
-      129,   67,  130,  129,   36,   36,  129,  132,  133,  135,
-      136,  140,   36,   36,   36,   36,  142,   36,  137,   35,
-       35,  123,   86,   36,  370,  143,  144,  144,   67,  145,
-      144,  148,  158,  144,  147,   35,   35,   90,  119,   36,
-       36,  149,  150,  150,   67,  151,  150,  159,   36,  150,
-      153,  154,  154,   67,  155,  154,  164,   36,  154,  157,
-
-      160,  160,   67,  161,  160,   36,  368,  160,  163,  165,
-      166,   36,   36,   29,  170,  167,  168,   29,  172,  171,
-       36,  175,   36,  173,   35,   35,  176,   36,   36,  177,
-       36,   36,  188,  174,   36,   29,  190,  178,   36,  181,
-       36,  191,  223,  179,  182,  182,   67,  183,  182,  186,
-      206,  182,  185,  187,   29,  192,   35,   35,   35,   35,
-      193,   29,  197,   29,  199,  194,   36,  198,   36,  200,
-       29,  202,   29,  204,  195,   36,  203,   36,  205,  268,
-      207,   29,  209,   29,  211,  214,  213,  210,  218,  212,
-      217,   36,  353,   36,   29,  170,   36,   35,   35,  219,
-
-      171,   35,   35,   35,   35,  224,   36,  231,   36,  225,
-       36,   29,  227,  221,   36,  222,  232,  228,  220,   29,
-      229,   36,  240,   35,   35,  230,  233,  233,   67,  234,
-      233,   29,  190,  233,  236,  237,  348,  191,  238,   35,
-       29,  197,   29,  202,  239,   36,  198,   36,  203,   29,
-      209,  242,   36,   35,  247,  210,  255,  241,  248,   36,
-       35,   35,   36,   35,   35,   35,   35,  253,   36,   35,
-       35,   29,  227,  250,  269,  254,   36,  228,  249,  251,
-      252,   35,  258,   29,  260,   29,  262,  264,   36,  261,
-      265,  263,   35,   35,  346,   35,   35,   70,  271,   35,
-
-       35,   35,   35,   35,   35,  274,   35,   35,  282,   35,
-       35,   36,  277,  278,   35,   35,  283,  284,   35,   35,
-      279,  285,   36,  280,  288,   29,  260,   35,   35,  289,
-      312,  261,  293,  293,   67,  294,  293,  301,  306,  293,
-      296,   35,   35,  298,  303,  306,  304,   35,   35,   35,
-       35,  309,  308,   36,  307,  282,  302,  319,   35,   35,
-       35,   35,   35,  311,   29,  314,   29,  316,   35,   35,
-      315,  282,  317,   35,   35,  344,  310,  364,  325,   35,
-       35,  318,   35,   35,  329,  320,   36,  328,  332,   36,
-       29,  314,   35,   35,  330,  326,  315,  331,  327,  333,
-
-       35,   35,   35,   35,  282,  282,  340,  341,   35,   35,
-       35,   35,   36,  282,   35,   35,   36,  351,   35,   35,
-      362,  339,  365,   36,  338,  366,  342,  361,  360,  354,
-      358,  349,  356,   35,   35,   27,   27,   27,   27,   29,
-       29,   29,   31,   31,   31,   31,   36,   36,   36,   36,
-       63,  353,   63,   63,   64,   64,   64,   64,   66,  350,
-       66,   66,   35,   35,   35,   35,   70,   70,  324,   70,
-       96,   96,   96,   96,   98,  322,   98,   98,  131,  131,
-      131,  131,  146,  146,  146,  146,  152,  152,  152,  152,
-      156,  156,  156,  156,  162,  162,  162,  162,  169,  169,
-
-      169,  169,  171,  348,  171,  171,  184,  184,  184,  184,
-      189,  189,  189,  189,  191,  346,  191,  191,  196,  196,
-      196,  196,  198,  344,  198,  198,  201,  201,  201,  201,
-      203,  337,  203,  203,  208,  208,  208,  208,  210,  335,
-      210,  210,  215,  215,  215,  215,  173,  282,  173,  173,
-      226,  226,  226,  226,  228,  324,  228,  228,  235,  235,
-      235,  235,  193,  322,  193,  193,  200,  276,  200,  200,
-      205,  267,  205,  205,  212,  257,  212,  212,  243,  243,
-      243,  243,  245,  245,  245,  245,  230,  306,  230,  230,
-      256,  256,  256,  256,  259,  259,  259,  259,  261,  300,
-
-      261,  261,  266,  266,  266,  266,  270,  270,  270,  270,
-      272,  272,  272,  272,  275,  275,  275,  275,  281,  281,
-      281,  281,  286,  286,  286,  286,  263,  246,  263,  263,
-      290,  290,  290,  290,  295,  295,  295,  295,  271,  297,
-      271,  271,  299,  299,  299,  299,  305,  305,  305,  305,
-      313,  313,  313,  313,  315,  292,  315,  315,  321,  321,
-      321,  321,  323,  323,  323,  323,  317,  291,  317,  317,
-      334,  334,  334,  334,  336,  336,  336,  336,  343,  343,
-      343,  343,  345,  345,  345,  345,  347,  347,  347,  347,
-      352,  352,  352,  352,  355,  355,  355,  355,  357,  357,
-
-      357,  357,  359,  359,  359,  359,  363,  363,  363,  363,
-      367,  367,  367,  367,  369,  369,  369,  369,  287,  276,
-      273,  216,  267,  257,  246,  244,  216,  141,  139,   99,
-       36,   30,   60,   57,   36,   30,  371,   28,   28,    5,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371
+        6,    7,    8,    9,    6,    6,    6,    6,    6,    6,
+       10,    6,   11,   12,   13,   14,   15,   16,   17,   18,
+       19,   20,   20,   20,   20,   21,   20,   22,   23,   24,
+       25,   26,   27,   28,   20,   20,   20,   34,   34,   35,
+       34,   34,   35,   38,   38,   38,   36,   44,   38,   36,
+       31,   31,   31,   32,   37,   38,   49,   38,   45,   39,
+       40,   31,   52,   50,   41,   53,   38,  102,   59,   51,
+       42,   31,   31,   31,   32,   37,   38,   38,   56,   38,
+       39,   40,   31,   43,   38,   38,   58,   61,   46,   63,
+       54,   57,   47,   38,   55,   38,   38,   31,   70,   66,
+
+       62,   48,   38,   67,   79,   71,   64,   78,   65,   34,
+       34,   35,   38,   38,   85,   38,   38,   56,   36,   31,
+       31,   31,   32,   81,   38,  401,   76,   38,   31,   40,
+       31,   72,   72,   72,   73,   72,   82,   38,   38,   38,
+       72,   74,   72,   77,   80,   38,   38,   38,   89,   86,
+       90,   38,   38,   91,   83,   84,   87,   38,   93,   38,
+       92,   38,   38,   38,   38,   38,   38,   98,   38,  103,
+      403,   94,   38,   95,   96,   96,  109,   99,   97,  100,
+       31,   70,   31,  105,  101,   38,   38,   38,   71,  108,
+      106,  110,   38,   38,   38,   38,  111,   38,   38,  115,
+
+      116,  112,   38,   38,   38,   92,  113,  117,   38,  118,
+       38,  120,  114,   38,   38,  123,  126,  119,   38,   38,
+      128,   38,  121,   38,   38,   38,  130,   38,   38,  129,
+      222,  122,   31,   31,   31,   32,  127,   38,   38,   38,
+       38,   31,   40,   31,  134,  135,  124,  125,  131,  137,
+      132,   31,  105,   38,  136,   38,  133,  138,   38,  106,
+       38,  139,  140,  140,   72,  141,  140,  145,  146,  147,
+       38,  140,  143,  140,  144,   38,   38,   38,  149,  153,
+      151,   38,  148,   38,   38,  154,   38,   37,   37,   37,
+       37,  331,   92,  161,  171,  155,  156,  156,   72,  157,
+
+      156,  160,  180,  332,   38,  156,  159,  156,  162,  162,
+       72,  163,  162,   38,   38,   38,   38,  162,  165,  162,
+       96,  132,  128,  176,   38,   38,  166,  166,   72,  167,
+      166,  170,  178,   38,  179,  166,  169,  166,  172,  172,
+       72,  173,  172,  234,   38,   38,   38,  172,  175,  172,
+      177,  181,   38,   31,  184,   38,  182,   31,  186,  193,
+      189,  185,   37,   37,  190,  187,   38,  191,  188,   38,
+       38,  201,   38,   38,   38,  192,  203,  401,   38,   38,
+       37,   37,  230,  256,   37,  194,  202,  195,  196,  196,
+       72,  197,  196,  200,   31,  205,  210,  196,  199,  196,
+
+       31,  207,  206,   37,   37,   31,  212,   38,  208,   31,
+      214,   38,  209,  213,   31,  217,   38,  215,   31,  219,
+       31,  224,  218,   31,  226,  228,  220,   38,  225,  235,
+      221,  227,   31,  184,   38,  229,   38,   38,  240,  236,
+      185,   37,   37,   37,   37,  231,   37,   37,  241,   38,
+       32,  384,  242,   31,  244,  238,   38,  275,  239,   31,
+      246,  245,  237,  248,  250,   38,   38,  247,   31,  205,
+       31,  212,   38,   37,   37,  257,  206,   38,  213,  249,
+      251,  251,   72,  252,  251,  255,   31,  217,   38,  251,
+      254,  251,   31,  224,  218,   38,  258,  260,   38,   38,
+
+      225,  261,   38,   37,  267,  259,  268,   32,   38,   37,
+       37,  377,   37,   37,   37,   37,  273,  293,   38,   37,
+       37,  262,  270,   31,  244,  274,   38,  269,  271,  272,
+      285,  245,   37,  279,  278,   31,  281,   31,  283,   75,
+      292,   37,   37,  282,  286,  284,   37,   37,   38,  290,
+      289,   38,   38,   37,   37,  294,   38,   37,   37,  297,
+       37,   37,   37,   37,  305,   32,   37,   37,  375,  301,
+       37,   37,   37,   37,  311,  306,  307,  302,  308,  303,
+      300,   37,   37,   31,  281,  313,   38,  312,  323,  322,
+       38,  282,  317,  317,   72,  318,  317,   37,   37,  331,
+
+       32,  317,  320,  317,  328,  326,  329,  334,   37,   37,
+       37,   37,  327,   37,   37,  335,  333,   37,   37,   37,
+      337,   38,   31,  340,   31,  342,  344,   38,  305,  336,
+      341,  346,  343,   37,   37,   38,   37,   37,  305,   32,
+      338,  345,   37,   37,  360,  357,   38,   37,   37,  305,
+      373,   38,  355,  347,  352,  356,   37,   37,  354,  358,
+       31,  340,  359,  305,   32,  353,   37,   37,  341,  361,
+       38,  369,  366,   37,   37,  368,  370,   37,   37,  305,
+       37,   37,  367,   38,  371,   37,   37,  399,  380,  381,
+      382,   37,   37,  393,  397,  398,  385,   38,   37,   37,
+
+      378,   29,   29,   29,   29,   31,   31,   31,   33,   33,
+       33,   33,   38,   38,   38,   38,   68,  395,   68,   68,
+       69,   69,   69,   69,   71,  394,   71,   71,   37,   37,
+       37,   37,   75,   75,  392,   75,  104,  104,  104,  104,
+      106,  391,  106,  106,  142,  142,  142,  142,  158,  158,
+      158,  158,  164,  164,  164,  164,  168,  168,  168,  168,
+      174,  174,  174,  174,  183,  183,  183,  183,  185,  389,
+      185,  185,  198,  198,  198,  198,  204,  204,  204,  204,
+      206,  387,  206,  206,  211,  211,  211,  211,  213,  384,
+      213,  213,  216,  216,  216,  216,  218,  379,  218,  218,
+
+      223,  223,  223,  223,  225,  351,  225,  225,  232,  232,
+      232,  232,  187,  349,  187,  187,  243,  243,  243,  243,
+      245,  377,  245,  245,  253,  253,  253,  253,  208,  375,
+      208,  208,  215,  373,  215,  215,  220,  365,  220,  220,
+      227,  363,  227,  227,  263,  263,  263,  263,  265,  265,
+      265,  265,  247,  305,  247,  247,  276,  276,  276,  276,
+      280,  280,  280,  280,  282,  351,  282,  282,  287,  287,
+      287,  287,  291,  291,  291,  291,  295,  295,  295,  295,
+      298,  298,  298,  298,  304,  304,  304,  304,  309,  309,
+      309,  309,  284,  349,  284,  284,  314,  314,  314,  314,
+
+      319,  319,  319,  319,  292,  299,  292,  292,  324,  324,
+      324,  324,  330,  330,  330,  330,  339,  339,  339,  339,
+      341,  288,  341,  341,  348,  348,  348,  348,  350,  350,
+      350,  350,  343,  277,  343,  343,  362,  362,  362,  362,
+      364,  364,  364,  364,  372,  372,  372,  372,  374,  374,
+      374,  374,  376,  376,  376,  376,  383,  383,  383,  383,
+      386,  386,  386,  386,  388,  388,  388,  388,  390,  390,
+      390,  390,  396,  396,  396,  396,  400,  400,  400,  400,
+      402,  402,  402,  402,  331,  325,  266,  321,  316,  315,
+      310,  299,  296,  233,  288,  277,  266,  264,  233,  152,
+
+      150,  107,   88,   38,   32,   60,   38,   32,  404,   30,
+       30,    5,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404
     } ;
 
-static const flex_int16_t yy_chk[975] =
+static const flex_int16_t yy_chk[1050] =
     {   0,
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
         1,    1,    1,    1,    1,    1,    1,    1,    1,    1,
-        1,    1,    1,    1,    7,    7,    7,    8,    8,    8,
-       13,    7,   18,   13,    8,   11,   11,   11,   11,   11,
-       18,   14,   11,   11,   13,   14,   11,   46,   46,   14,
-       15,   16,   11,   12,   12,   12,   12,   12,   14,   16,
-       12,   12,   12,   15,   19,   16,   20,   20,   21,   22,
-       24,   22,   24,   50,   26,   21,   50,   26,   19,   44,
-       20,   26,   19,   31,   31,   24,   32,   32,   32,   31,
-
-       39,   42,   39,   32,   35,   35,   35,   35,   40,   45,
-       44,   35,   35,   37,   37,   37,   37,   37,   42,   39,
-       37,   37,   40,   41,   43,   41,   45,   45,   47,   47,
-       48,   49,   51,   54,   55,   43,   51,   52,   54,   52,
-       53,   48,   56,   53,   57,   58,   59,   61,   49,   71,
-       59,   61,   60,   71,   62,   72,   56,   62,   55,   64,
-       64,   75,   58,   72,   77,   64,   57,   60,   74,   60,
-       67,   67,   73,   76,   73,   75,   67,   78,   74,   82,
-       79,   78,   79,   77,   84,   80,   81,   76,   80,   83,
-       81,   82,   83,   85,   89,   86,   84,   86,   88,   90,
-
-       88,   90,   91,   92,  100,   92,  104,  369,   95,  101,
-       91,   85,   87,   87,   87,   87,   89,  138,  104,   87,
-       87,   94,  100,   87,   87,   93,   93,   94,   93,   95,
-      101,   93,   96,   96,  106,  107,  138,  107,   96,  102,
-      102,  102,  102,  102,  103,  112,  102,  102,  103,  105,
-      105,  109,  111,  109,  113,  105,  111,  120,  106,  115,
-      115,  122,  113,  122,  367,  112,  114,  114,  114,  114,
-      114,  115,  120,  114,  114,  116,  116,  118,  118,  121,
-      118,  116,  117,  117,  117,  117,  117,  121,  124,  117,
-      117,  119,  119,  119,  119,  119,  124,  127,  119,  119,
-
-      123,  123,  123,  123,  123,  125,  365,  123,  123,  125,
-      126,  126,  128,  129,  129,  127,  128,  131,  131,  129,
-      133,  134,  139,  131,  136,  136,  136,  134,  137,  136,
-      141,  158,  143,  133,  143,  144,  144,  136,  179,  139,
-      142,  144,  179,  137,  140,  140,  140,  140,  140,  141,
-      158,  140,  140,  142,  146,  146,  148,  148,  149,  149,
-      146,  150,  150,  152,  152,  148,  165,  150,  164,  152,
-      154,  154,  156,  156,  149,  159,  154,  240,  156,  240,
-      159,  160,  160,  162,  162,  165,  164,  160,  168,  162,
-      167,  167,  363,  168,  169,  169,  174,  176,  176,  174,
-
-      169,  177,  177,  178,  178,  180,  181,  186,  186,  181,
-      180,  182,  182,  177,  207,  178,  187,  182,  176,  184,
-      184,  187,  207,  194,  194,  184,  188,  188,  188,  188,
-      188,  189,  189,  188,  188,  194,  359,  189,  195,  195,
-      196,  196,  201,  201,  206,  206,  196,  213,  201,  208,
-      208,  214,  214,  218,  218,  208,  225,  213,  219,  219,
-      220,  220,  225,  221,  221,  222,  222,  223,  223,  224,
-      224,  226,  226,  220,  241,  224,  241,  226,  219,  221,
-      222,  232,  232,  233,  233,  235,  235,  237,  249,  233,
-      238,  235,  238,  238,  357,  237,  237,  242,  242,  247,
-
-      247,  250,  250,  251,  251,  247,  252,  252,  253,  254,
-      254,  292,  249,  250,  265,  265,  253,  253,  258,  258,
-      251,  254,  277,  252,  258,  259,  259,  274,  274,  265,
-      292,  259,  269,  269,  269,  269,  269,  277,  284,  269,
-      269,  278,  278,  274,  279,  283,  280,  279,  279,  280,
-      280,  285,  284,  301,  283,  307,  278,  301,  285,  285,
-      288,  288,  289,  289,  293,  293,  295,  295,  302,  302,
-      293,  308,  295,  298,  298,  355,  288,  352,  307,  310,
-      310,  298,  309,  309,  311,  302,  312,  310,  319,  319,
-      313,  313,  311,  311,  312,  308,  313,  318,  309,  320,
-
-      318,  318,  320,  320,  325,  326,  327,  328,  327,  327,
-      328,  328,  330,  338,  340,  340,  342,  340,  351,  351,
-      351,  326,  354,  354,  325,  362,  330,  349,  347,  342,
-      345,  338,  343,  362,  362,  372,  372,  372,  372,  373,
-      373,  373,  374,  374,  374,  374,  375,  375,  375,  375,
-      376,  341,  376,  376,  377,  377,  377,  377,  378,  339,
-      378,  378,  379,  379,  379,  379,  380,  380,  336,  380,
-      381,  381,  381,  381,  382,  334,  382,  382,  383,  383,
-      383,  383,  384,  384,  384,  384,  385,  385,  385,  385,
-      386,  386,  386,  386,  387,  387,  387,  387,  388,  388,
-
-      388,  388,  389,  333,  389,  389,  390,  390,  390,  390,
-      391,  391,  391,  391,  392,  332,  392,  392,  393,  393,
-      393,  393,  394,  331,  394,  394,  395,  395,  395,  395,
-      396,  323,  396,  396,  397,  397,  397,  397,  398,  321,
-      398,  398,  399,  399,  399,  399,  400,  305,  400,  400,
-      401,  401,  401,  401,  402,  304,  402,  402,  403,  403,
-      403,  403,  404,  303,  404,  404,  405,  299,  405,  405,
-      406,  290,  406,  406,  407,  286,  407,  407,  408,  408,
-      408,  408,  409,  409,  409,  409,  410,  281,  410,  410,
-      411,  411,  411,  411,  412,  412,  412,  412,  413,  275,
-
-      413,  413,  414,  414,  414,  414,  415,  415,  415,  415,
-      416,  416,  416,  416,  417,  417,  417,  417,  418,  418,
-      418,  418,  419,  419,  419,  419,  420,  272,  420,  420,
-      421,  421,  421,  421,  422,  422,  422,  422,  423,  270,
-      423,  423,  424,  424,  424,  424,  425,  425,  425,  425,
-      426,  426,  426,  426,  427,  268,  427,  427,  428,  428,
-      428,  428,  429,  429,  429,  429,  430,  266,  430,  430,
-      431,  431,  431,  431,  432,  432,  432,  432,  433,  433,
-      433,  433,  434,  434,  434,  434,  435,  435,  435,  435,
-      436,  436,  436,  436,  437,  437,  437,  437,  438,  438,
-
-      438,  438,  439,  439,  439,  439,  440,  440,  440,  440,
-      441,  441,  441,  441,  442,  442,  442,  442,  256,  248,
-      245,  243,  239,  231,  217,  215,  166,  110,  108,   70,
-       36,   29,   25,   23,   17,    6,    5,    4,    3,  371,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371,  371,  371,  371,  371,  371,  371,
-      371,  371,  371,  371
+        1,    1,    1,    1,    1,    1,    1,    7,    7,    7,
+        8,    8,    8,   15,   13,   17,    7,   13,   65,    8,
+       11,   11,   11,   11,   11,   16,   15,   18,   13,   11,
+       11,   11,   17,   16,   11,   18,   23,   65,   23,   16,
+       11,   12,   12,   12,   12,   12,   19,   21,   21,   22,
+       12,   12,   12,   12,   14,   25,   22,   25,   14,   26,
+       19,   21,   14,   42,   19,   27,   28,   33,   33,   28,
+
+       25,   14,   43,   28,   43,   33,   26,   42,   27,   34,
+       34,   34,   45,   48,   48,   46,   49,   49,   34,   37,
+       37,   37,   37,   45,   41,  402,   41,   44,   37,   37,
+       37,   39,   39,   39,   39,   39,   46,   47,   50,   51,
+       39,   39,   39,   41,   44,   53,   54,   58,   53,   50,
+       54,   55,   60,   55,   47,   47,   51,   56,   57,   59,
+       56,   61,   63,   62,   64,   66,   57,   62,   77,   66,
+      400,   58,   67,   59,   60,   67,   77,   63,   61,   63,
+       69,   69,   72,   72,   64,   76,   79,   80,   69,   76,
+       72,   78,   81,   82,   78,   83,   79,   87,   84,   83,
+
+       84,   80,   85,   86,   88,   85,   81,   86,   89,   87,
+       90,   89,   82,   91,   95,   92,   94,   88,   92,   94,
+       96,   97,   90,   96,   98,  171,   98,  101,  103,   97,
+      171,   91,   93,   93,   93,   93,   95,  102,  100,  108,
+      109,   93,   93,   93,  100,  101,   93,   93,   99,  103,
+       99,  104,  104,   99,  102,  112,   99,  108,  114,  104,
+      119,  109,  110,  110,  110,  110,  110,  112,  113,  113,
+      111,  110,  110,  110,  111,  121,  115,  113,  115,  119,
+      117,  120,  114,  117,  122,  120,  130,  125,  125,  124,
+      124,  306,  122,  125,  130,  121,  123,  123,  123,  123,
+
+      123,  124,  137,  306,  137,  123,  123,  123,  126,  126,
+      126,  126,  126,  135,  136,  133,  129,  126,  126,  126,
+      127,  131,  127,  133,  131,  127,  128,  128,  128,  128,
+      128,  129,  135,  138,  136,  128,  128,  128,  132,  132,
+      132,  132,  132,  181,  148,  181,  134,  132,  132,  132,
+      134,  138,  139,  140,  140,  144,  139,  142,  142,  148,
+      145,  140,  147,  147,  147,  142,  149,  147,  144,  145,
+      150,  153,  153,  154,  152,  147,  155,  398,  178,  155,
+      161,  161,  178,  210,  210,  149,  154,  150,  151,  151,
+      151,  151,  151,  152,  156,  156,  161,  151,  151,  151,
+
+      158,  158,  156,  160,  160,  162,  162,  176,  158,  164,
+      164,  170,  160,  162,  166,  166,  177,  164,  168,  168,
+      172,  172,  166,  174,  174,  176,  168,  179,  172,  182,
+      170,  174,  183,  183,  193,  177,  188,  182,  193,  188,
+      183,  190,  190,  191,  191,  179,  192,  192,  194,  195,
+      201,  396,  195,  196,  196,  191,  194,  242,  192,  198,
+      198,  196,  190,  200,  202,  200,  242,  198,  204,  204,
+      211,  211,  202,  209,  209,  221,  204,  221,  211,  201,
+      203,  203,  203,  203,  203,  209,  216,  216,  222,  203,
+      203,  203,  223,  223,  216,  228,  222,  229,  230,  229,
+
+      223,  230,  231,  235,  235,  228,  236,  249,  236,  237,
+      237,  390,  238,  238,  239,  239,  240,  261,  240,  241,
+      241,  231,  237,  243,  243,  241,  261,  236,  238,  239,
+      255,  243,  250,  250,  249,  251,  251,  253,  253,  260,
+      260,  255,  255,  251,  256,  253,  256,  256,  258,  259,
+      258,  262,  259,  267,  267,  262,  269,  270,  270,  267,
+      271,  271,  272,  272,  273,  278,  274,  274,  388,  270,
+      286,  286,  297,  297,  278,  273,  273,  271,  274,  272,
+      269,  279,  279,  280,  280,  286,  294,  279,  297,  294,
+      300,  280,  290,  290,  290,  290,  290,  301,  301,  307,
+
+      311,  290,  290,  290,  302,  300,  303,  308,  302,  302,
+      303,  303,  301,  312,  312,  311,  307,  308,  308,  313,
+      313,  316,  317,  317,  319,  319,  322,  326,  332,  312,
+      317,  326,  319,  323,  323,  322,  327,  327,  333,  335,
+      316,  323,  334,  334,  346,  337,  346,  336,  336,  352,
+      386,  338,  335,  327,  332,  336,  337,  337,  334,  338,
+      339,  339,  345,  353,  355,  333,  345,  345,  339,  347,
+      358,  355,  352,  347,  347,  354,  356,  354,  354,  366,
+      356,  356,  353,  371,  358,  368,  368,  393,  368,  369,
+      369,  380,  380,  380,  383,  385,  371,  385,  393,  393,
+
+      366,  405,  405,  405,  405,  406,  406,  406,  407,  407,
+      407,  407,  408,  408,  408,  408,  409,  382,  409,  409,
+      410,  410,  410,  410,  411,  381,  411,  411,  412,  412,
+      412,  412,  413,  413,  378,  413,  414,  414,  414,  414,
+      415,  376,  415,  415,  416,  416,  416,  416,  417,  417,
+      417,  417,  418,  418,  418,  418,  419,  419,  419,  419,
+      420,  420,  420,  420,  421,  421,  421,  421,  422,  374,
+      422,  422,  423,  423,  423,  423,  424,  424,  424,  424,
+      425,  372,  425,  425,  426,  426,  426,  426,  427,  370,
+      427,  427,  428,  428,  428,  428,  429,  367,  429,  429,
+
+      430,  430,  430,  430,  431,  364,  431,  431,  432,  432,
+      432,  432,  433,  362,  433,  433,  434,  434,  434,  434,
+      435,  361,  435,  435,  436,  436,  436,  436,  437,  360,
+      437,  437,  438,  359,  438,  438,  439,  350,  439,  439,
+      440,  348,  440,  440,  441,  441,  441,  441,  442,  442,
+      442,  442,  443,  330,  443,  443,  444,  444,  444,  444,
+      445,  445,  445,  445,  446,  329,  446,  446,  447,  447,
+      447,  447,  448,  448,  448,  448,  449,  449,  449,  449,
+      450,  450,  450,  450,  451,  451,  451,  451,  452,  452,
+      452,  452,  453,  328,  453,  453,  454,  454,  454,  454,
+
+      455,  455,  455,  455,  456,  324,  456,  456,  457,  457,
+      457,  457,  458,  458,  458,  458,  459,  459,  459,  459,
+      460,  314,  460,  460,  461,  461,  461,  461,  462,  462,
+      462,  462,  463,  309,  463,  463,  464,  464,  464,  464,
+      465,  465,  465,  465,  466,  466,  466,  466,  467,  467,
+      467,  467,  468,  468,  468,  468,  469,  469,  469,  469,
+      470,  470,  470,  470,  471,  471,  471,  471,  472,  472,
+      472,  472,  473,  473,  473,  473,  474,  474,  474,  474,
+      475,  475,  475,  475,  304,  298,  295,  291,  289,  287,
+      276,  268,  265,  263,  257,  248,  234,  232,  180,  118,
+
+      116,   75,   52,   38,   31,   24,   20,    6,    5,    4,
+        3,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404,  404,
+      404,  404,  404,  404,  404,  404,  404,  404,  404
     } ;
 
 #define YY_TRAILING_MASK 0x2000
@@ -1224,9 +1257,9 @@ static int vdev_and_devtype(DiskParseContext *dpc, char *str) {
 #undef DPC /* needs to be defined differently the actual lexer */
 #define DPC ((DiskParseContext*)yyextra)
 
-#line 1227 "libxlu_disk_l.c"
+#line 1260 "libxlu_disk_l.c"
 
-#line 1229 "libxlu_disk_l.c"
+#line 1262 "libxlu_disk_l.c"
 
 #define INITIAL 0
 #define LEXERR 1
@@ -1508,7 +1541,7 @@ YY_DECL
 #line 188 "libxlu_disk_l.l"
  /*----- the scanner rules which do the parsing -----*/
 
-#line 1511 "libxlu_disk_l.c"
+#line 1544 "libxlu_disk_l.c"
 
 	while ( /*CONSTCOND*/1 )		/* loops until end-of-file is reached */
 		{
@@ -1540,14 +1573,14 @@ yy_match:
 			while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state )
 				{
 				yy_current_state = (int) yy_def[yy_current_state];
-				if ( yy_current_state >= 372 )
+				if ( yy_current_state >= 405 )
 					yy_c = yy_meta[yy_c];
 				}
 			yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c];
 			*yyg->yy_state_ptr++ = yy_current_state;
 			++yy_cp;
 			}
-		while ( yy_current_state != 371 );
+		while ( yy_current_state != 404 );
 
 yy_find_action:
 		yy_current_state = *--yyg->yy_state_ptr;
@@ -1721,34 +1754,54 @@ YY_RULE_SETUP
 #line 219 "libxlu_disk_l.l"
 { STRIP(','); SAVESTRING("hidden-disk", hidden_disk, FROMEQUALS); }
 	YY_BREAK
-/* the target magic parameter, eats the rest of the string */
 case 23:
 YY_RULE_SETUP
-#line 223 "libxlu_disk_l.l"
+#line 221 "libxlu_disk_l.l"
+{ libxl_defbool_set(&DPC->disk->trusted, true); }
+	YY_BREAK
+case 24:
+YY_RULE_SETUP
+#line 222 "libxlu_disk_l.l"
+{ libxl_defbool_set(&DPC->disk->trusted, false); }
+	YY_BREAK
+case 25:
+YY_RULE_SETUP
+#line 224 "libxlu_disk_l.l"
+{ libxl_defbool_set(&DPC->disk->grant_usage, true); }
+	YY_BREAK
+case 26:
+YY_RULE_SETUP
+#line 225 "libxlu_disk_l.l"
+{ libxl_defbool_set(&DPC->disk->grant_usage, false); }
+	YY_BREAK
+/* the target magic parameter, eats the rest of the string */
+case 27:
+YY_RULE_SETUP
+#line 229 "libxlu_disk_l.l"
 { STRIP(','); SAVESTRING("target", pdev_path, FROMEQUALS); }
 	YY_BREAK
 /* unknown parameters */
-case 24:
-/* rule 24 can match eol */
+case 28:
+/* rule 28 can match eol */
 YY_RULE_SETUP
-#line 227 "libxlu_disk_l.l"
+#line 233 "libxlu_disk_l.l"
 { xlu__disk_err(DPC,yytext,"unknown parameter"); }
 	YY_BREAK
 /* deprecated prefixes */
 /* the "/.*" in these patterns ensures that they count as if they
    * matched the whole string, so these patterns take precedence */
-case 25:
+case 29:
 YY_RULE_SETUP
-#line 234 "libxlu_disk_l.l"
+#line 240 "libxlu_disk_l.l"
 {
                     STRIP(':');
                     DPC->had_depr_prefix=1; DEPRECATE("use `[format=]...,'");
                     setformat(DPC, yytext);
                  }
 	YY_BREAK
-case 26:
+case 30:
 YY_RULE_SETUP
-#line 240 "libxlu_disk_l.l"
+#line 246 "libxlu_disk_l.l"
 {
                     char *newscript;
                     STRIP(':');
@@ -1762,65 +1815,65 @@ YY_RULE_SETUP
                     free(newscript);
                 }
 	YY_BREAK
-case 27:
+case 31:
 *yy_cp = yyg->yy_hold_char; /* undo effects of setting up yytext */
 yyg->yy_c_buf_p = yy_cp = yy_bp + 8;
 YY_DO_BEFORE_ACTION; /* set up yytext again */
 YY_RULE_SETUP
-#line 253 "libxlu_disk_l.l"
+#line 259 "libxlu_disk_l.l"
 { DPC->had_depr_prefix=1; DEPRECATE(0); }
 	YY_BREAK
-case 28:
+case 32:
 YY_RULE_SETUP
-#line 254 "libxlu_disk_l.l"
+#line 260 "libxlu_disk_l.l"
 { DPC->had_depr_prefix=1; DEPRECATE(0); }
 	YY_BREAK
-case 29:
+case 33:
 *yy_cp = yyg->yy_hold_char; /* undo effects of setting up yytext */
 yyg->yy_c_buf_p = yy_cp = yy_bp + 4;
 YY_DO_BEFORE_ACTION; /* set up yytext again */
 YY_RULE_SETUP
-#line 255 "libxlu_disk_l.l"
+#line 261 "libxlu_disk_l.l"
 { DPC->had_depr_prefix=1; DEPRECATE(0); }
 	YY_BREAK
-case 30:
+case 34:
 *yy_cp = yyg->yy_hold_char; /* undo effects of setting up yytext */
 yyg->yy_c_buf_p = yy_cp = yy_bp + 6;
 YY_DO_BEFORE_ACTION; /* set up yytext again */
 YY_RULE_SETUP
-#line 256 "libxlu_disk_l.l"
+#line 262 "libxlu_disk_l.l"
 { DPC->had_depr_prefix=1; DEPRECATE(0); }
 	YY_BREAK
-case 31:
+case 35:
 *yy_cp = yyg->yy_hold_char; /* undo effects of setting up yytext */
 yyg->yy_c_buf_p = yy_cp = yy_bp + 5;
 YY_DO_BEFORE_ACTION; /* set up yytext again */
 YY_RULE_SETUP
-#line 257 "libxlu_disk_l.l"
+#line 263 "libxlu_disk_l.l"
 { DPC->had_depr_prefix=1; DEPRECATE(0); }
 	YY_BREAK
-case 32:
+case 36:
 *yy_cp = yyg->yy_hold_char; /* undo effects of setting up yytext */
 yyg->yy_c_buf_p = yy_cp = yy_bp + 4;
 YY_DO_BEFORE_ACTION; /* set up yytext again */
 YY_RULE_SETUP
-#line 258 "libxlu_disk_l.l"
+#line 264 "libxlu_disk_l.l"
 { DPC->had_depr_prefix=1; DEPRECATE(0); }
 	YY_BREAK
-case 33:
-/* rule 33 can match eol */
+case 37:
+/* rule 37 can match eol */
 YY_RULE_SETUP
-#line 260 "libxlu_disk_l.l"
+#line 266 "libxlu_disk_l.l"
 {
 		  xlu__disk_err(DPC,yytext,"unknown deprecated disk prefix");
 		  return 0;
 		}
 	YY_BREAK
 /* positional parameters */
-case 34:
-/* rule 34 can match eol */
+case 38:
+/* rule 38 can match eol */
 YY_RULE_SETUP
-#line 267 "libxlu_disk_l.l"
+#line 273 "libxlu_disk_l.l"
 {
     STRIP(',');
 
@@ -1847,27 +1900,27 @@ YY_RULE_SETUP
     }
 }
 	YY_BREAK
-case 35:
+case 39:
 YY_RULE_SETUP
-#line 293 "libxlu_disk_l.l"
+#line 299 "libxlu_disk_l.l"
 {
     BEGIN(LEXERR);
     yymore();
 }
 	YY_BREAK
-case 36:
+case 40:
 YY_RULE_SETUP
-#line 297 "libxlu_disk_l.l"
+#line 303 "libxlu_disk_l.l"
 {
     xlu__disk_err(DPC,yytext,"bad disk syntax"); return 0;
 }
 	YY_BREAK
-case 37:
+case 41:
 YY_RULE_SETUP
-#line 300 "libxlu_disk_l.l"
+#line 306 "libxlu_disk_l.l"
 YY_FATAL_ERROR( "flex scanner jammed" );
 	YY_BREAK
-#line 1870 "libxlu_disk_l.c"
+#line 1923 "libxlu_disk_l.c"
 			case YY_STATE_EOF(INITIAL):
 			case YY_STATE_EOF(LEXERR):
 				yyterminate();
@@ -2135,7 +2188,7 @@ static int yy_get_next_buffer (yyscan_t yyscanner)
 		while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state )
 			{
 			yy_current_state = (int) yy_def[yy_current_state];
-			if ( yy_current_state >= 372 )
+			if ( yy_current_state >= 405 )
 				yy_c = yy_meta[yy_c];
 			}
 		yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c];
@@ -2159,11 +2212,11 @@ static int yy_get_next_buffer (yyscan_t yyscanner)
 	while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state )
 		{
 		yy_current_state = (int) yy_def[yy_current_state];
-		if ( yy_current_state >= 372 )
+		if ( yy_current_state >= 405 )
 			yy_c = yy_meta[yy_c];
 		}
 	yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c];
-	yy_is_jam = (yy_current_state == 371);
+	yy_is_jam = (yy_current_state == 404);
 	if ( ! yy_is_jam )
 		*yyg->yy_state_ptr++ = yy_current_state;
 
@@ -2972,4 +3025,4 @@ void yyfree (void * ptr , yyscan_t yyscanner)
 
 #define YYTABLES_NAME "yytables"
 
-#line 300 "libxlu_disk_l.l"
+#line 306 "libxlu_disk_l.l"
diff --git a/tools/libs/util/libxlu_disk_l.h b/tools/libs/util/libxlu_disk_l.h
index 509aad67dc..c868422568 100644
--- a/tools/libs/util/libxlu_disk_l.h
+++ b/tools/libs/util/libxlu_disk_l.h
@@ -3,9 +3,12 @@
 #define xlu__disk_yyIN_HEADER 1
 
 #line 5 "libxlu_disk_l.h"
+#line 31 "libxlu_disk_l.l"
 #define _GNU_SOURCE
 
-#line 8 "libxlu_disk_l.h"
+
+
+#line 11 "libxlu_disk_l.h"
 
 #define  YY_INT_ALIGNED short int
 
@@ -694,8 +697,8 @@ extern int yylex (yyscan_t yyscanner);
 #undef yyTABLES_NAME
 #endif
 
-#line 300 "libxlu_disk_l.l"
+#line 306 "libxlu_disk_l.l"
 
-#line 699 "libxlu_disk_l.h"
+#line 702 "libxlu_disk_l.h"
 #undef xlu__disk_yyIN_HEADER
 #endif /* xlu__disk_yyHEADER_H */
diff --git a/tools/libs/util/libxlu_disk_l.l b/tools/libs/util/libxlu_disk_l.l
index 6d53c093a3..f37dd443bd 100644
--- a/tools/libs/util/libxlu_disk_l.l
+++ b/tools/libs/util/libxlu_disk_l.l
@@ -220,6 +220,9 @@ hidden-disk=[^,]*,?	{ STRIP(','); SAVESTRING("hidden-disk", hidden_disk, FROMEQU
 trusted,?		{ libxl_defbool_set(&DPC->disk->trusted, true); }
 untrusted,?		{ libxl_defbool_set(&DPC->disk->trusted, false); }
 
+grant_usage=1,?		{ libxl_defbool_set(&DPC->disk->grant_usage, true); }
+grant_usage=0,?		{ libxl_defbool_set(&DPC->disk->grant_usage, false); }
+
  /* the target magic parameter, eats the rest of the string */
 
 target=.*	{ STRIP(','); SAVESTRING("target", pdev_path, FROMEQUALS); }
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 15:23:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 15:23:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681857.1060828 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rada4-0002WS-3F; Thu, 15 Feb 2024 15:23:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681857.1060828; Thu, 15 Feb 2024 15:23:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rada3-0002WL-UJ; Thu, 15 Feb 2024 15:23:19 +0000
Received: by outflank-mailman (input) for mailman id 681857;
 Thu, 15 Feb 2024 15:23:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YE+Y=JY=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rada2-0002WF-PB
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 15:23:18 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 23cc432a-cc16-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 16:23:15 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 153034EE0737;
 Thu, 15 Feb 2024 16:23:16 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23cc432a-cc16-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Thu, 15 Feb 2024 16:23:16 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Julien Grall <julien@xen.org>
Cc: Jan Beulich <jbeulich@suse.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Xen Devel <xen-devel@lists.xenproject.org>,
 Consulting <consulting@bugseng.com>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Andrew
 Cooper3 <andrew.cooper3@citrix.com>, Roger Pau <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: Return type of clean_and_invalidate_dcache_va_range
In-Reply-To: <2171bcc9eeea6613409e3ee1e9eedbc3@bugseng.com>
References: <e050b096ff39aa857f9c267a8dbb4ef6@bugseng.com>
 <alpine.DEB.2.22.394.2402091402080.1925432@ubuntu-linux-20-04-desktop>
 <f86baad0-f113-4156-9c10-6910e8c63492@xen.org>
 <02446758-a5a9-4a8a-8ce6-15abdc9ede65@suse.com>
 <99faac70440a68824a17fcaaea55ef48@bugseng.com>
 <09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org>
 <c0b8f2ec-a3bb-45a6-b748-052d55a6fd5c@suse.com>
 <9eb25d90-9f10-44e5-b9aa-32e3f898389a@xen.org>
 <2171bcc9eeea6613409e3ee1e9eedbc3@bugseng.com>
Message-ID: <1bb3e139fbc3d8e22f84b9ad22363d1b@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit


> 
> I do like the idea. I applied some of the suggestions to this proof of 
> concept patch (attached). Still not compile-tested, since the CI seems 
> a bit slow today.

arch_grant_cache_flush is missing the return type on x86, sorry.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 15:24:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 15:24:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681860.1060837 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1radbD-00031C-CS; Thu, 15 Feb 2024 15:24:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681860.1060837; Thu, 15 Feb 2024 15:24:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1radbD-000315-8d; Thu, 15 Feb 2024 15:24:31 +0000
Received: by outflank-mailman (input) for mailman id 681860;
 Thu, 15 Feb 2024 15:24:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1radbC-00030z-8I
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 15:24:30 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4e66b770-cc16-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 16:24:27 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 9212B1F37C;
 Thu, 15 Feb 2024 15:24:27 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 65666139D0;
 Thu, 15 Feb 2024 15:24:27 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id SvsjF6sszmUKNAAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 15:24:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e66b770-cc16-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708010667; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=w8YrKr7y5fp/hYmK3TluFyGEjlZFNDerawHnk8q2R7I=;
	b=qfSo7lpGs27P15YkSHy28TPxqlVfMSXrbcvQ13hP4+iy+cCdzDuRRqf7lvV7F0jwTqw9Zf
	xdtXWAlxi0OQJI6Z0+chJSu3xiI1SVCNSVoJO0nBKmKDJGb2WQT1SK/Vacst1weY4UanNQ
	U9ALm1BQr7d9uuGGdwT0F+Ys7p6tdF0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708010667; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=w8YrKr7y5fp/hYmK3TluFyGEjlZFNDerawHnk8q2R7I=;
	b=qfSo7lpGs27P15YkSHy28TPxqlVfMSXrbcvQ13hP4+iy+cCdzDuRRqf7lvV7F0jwTqw9Zf
	xdtXWAlxi0OQJI6Z0+chJSu3xiI1SVCNSVoJO0nBKmKDJGb2WQT1SK/Vacst1weY4UanNQ
	U9ALm1BQr7d9uuGGdwT0F+Ys7p6tdF0=
From: Juergen Gross <jgross@suse.com>
To: torvalds@linux-foundation.org
Cc: linux-kernel@vger.kernel.org,
	xen-devel@lists.xenproject.org,
	sstabellini@kernel.org
Subject: [GIT PULL] xen: branch for v6.8-rc5
Date: Thu, 15 Feb 2024 16:24:26 +0100
Message-Id: <20240215152426.1114-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: 0.70
X-Spamd-Result: default: False [0.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_DN_NONE(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Linus,

Please git pull the following tag:

 git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-6.8a-rc5-tag

xen: branch for v6.8-rc5

It contains the following fixes and simple cleanups:

- A fix using a proper flexible array instead of a one-element array in order to
  avoid array-bounds sanitizer errors.

- A fix adding NULL pointer checks after allocating memory.

- A cleanup using memdup_array_user() instead of open-coding it.

- A fix for a rare race condition in Xen event channel allocation code.

- A small series making struct bus_type instances const.

- A fix of kerneldoc inline comments to match reality.


Thanks.

Juergen

 arch/x86/xen/smp.c                 | 12 ++++++++++++
 drivers/xen/events/events_base.c   |  8 ++++++--
 drivers/xen/gntalloc.c             |  2 +-
 drivers/xen/pcpu.c                 |  2 +-
 drivers/xen/privcmd.c              | 15 +++++----------
 drivers/xen/xen-balloon.c          |  2 +-
 drivers/xen/xenbus/xenbus_client.c | 15 +++++++++------
 include/uapi/xen/gntalloc.h        |  5 ++++-
 8 files changed, 39 insertions(+), 22 deletions(-)

Kees Cook (1):
      xen/gntalloc: Replace UAPI 1-element array

Kunwu Chan (1):
      x86/xen: Add some null pointer checking to smp.c

Markus Elfring (1):
      xen/privcmd: Use memdup_array_user() in alloc_ioreq()

Maximilian Heyne (1):
      xen/events: close evtchn after mapping cleanup

Ricardo B. Marliere (2):
      xen: pcpu: make xen_pcpu_subsys const
      xen: balloon: make balloon_subsys const

SeongJae Park (1):
      xen/xenbus: document will_handle argument for xenbus_watch_path()


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 15:49:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 15:49:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681871.1060846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1radz7-0006IL-5f; Thu, 15 Feb 2024 15:49:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681871.1060846; Thu, 15 Feb 2024 15:49:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1radz7-0006IE-2j; Thu, 15 Feb 2024 15:49:13 +0000
Received: by outflank-mailman (input) for mailman id 681871;
 Thu, 15 Feb 2024 15:49:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1radz5-0006Gh-Sw; Thu, 15 Feb 2024 15:49:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1radz5-0001ii-Mu; Thu, 15 Feb 2024 15:49:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1radz5-0001WA-CU; Thu, 15 Feb 2024 15:49:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1radz5-0008A7-C7; Thu, 15 Feb 2024 15:49:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/Aop4tk5c/ggQ7yOBRW96g0AbhegEW6Vp3QIgSxLq0s=; b=en/M0DPx3EsllwxsTT27ulD5P8
	/sBdqeA2duOwv56SDOQtgVxmLRDkyObHE1YQIYqdKV6YzTQA91D8t2kQZa0DFkywyDJ94KjPOdjSN
	29nMLBlTD4zBZGFtcmrcLdKsz2tWzFrIW0xgnziIsM07WgS9Qg7o3GmrRhg2o8mg2O5A=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184673-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184673: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-dom0pvh-xl-amd:guest-start:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt-qcow2:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm:debian-hvm-install:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=d9bfc2810e1cbe4cd27555bdc48315d36ee91f14
X-Osstest-Versions-That:
    xen=d9bfc2810e1cbe4cd27555bdc48315d36ee91f14
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 15 Feb 2024 15:49:11 +0000

flight 184673 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184673/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-dom0pvh-xl-amd 14 guest-start   fail in 184670 pass in 184673
 test-armhf-armhf-libvirt-qcow2  8 xen-boot                 fail pass in 184670

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check fail in 184670 blocked in 184673
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check fail in 184670 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184670
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 12 debian-hvm-install fail like 184670
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184670
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184670
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184670
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184670
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184670
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184670
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184670
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184670
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184670
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184670
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  d9bfc2810e1cbe4cd27555bdc48315d36ee91f14
baseline version:
 xen                  d9bfc2810e1cbe4cd27555bdc48315d36ee91f14

Last test of basis   184673  2024-02-15 07:18:01 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         fail    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               fail    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 15:49:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 15:49:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681876.1060857 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1radzi-0006q3-FQ; Thu, 15 Feb 2024 15:49:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681876.1060857; Thu, 15 Feb 2024 15:49:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1radzi-0006pw-CF; Thu, 15 Feb 2024 15:49:50 +0000
Received: by outflank-mailman (input) for mailman id 681876;
 Thu, 15 Feb 2024 15:49:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lLkp=JY=arm.com=Luca.Fancellu@srs-se1.protection.inumbo.net>)
 id 1radzh-0006l5-Qw
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 15:49:49 +0000
Received: from EUR02-AM0-obe.outbound.protection.outlook.com
 (mail-am0eur02on20601.outbound.protection.outlook.com
 [2a01:111:f403:2606::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d8319b3d-cc19-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 16:49:47 +0100 (CET)
Received: from DUZP191CA0068.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:4fa::29)
 by DB5PR08MB9998.eurprd08.prod.outlook.com (2603:10a6:10:489::6) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39; Thu, 15 Feb
 2024 15:49:41 +0000
Received: from DU2PEPF00028CFC.eurprd03.prod.outlook.com
 (2603:10a6:10:4fa:cafe::6e) by DUZP191CA0068.outlook.office365.com
 (2603:10a6:10:4fa::29) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26 via Frontend
 Transport; Thu, 15 Feb 2024 15:49:41 +0000
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
 DU2PEPF00028CFC.mail.protection.outlook.com (10.167.242.180) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7292.25 via Frontend Transport; Thu, 15 Feb 2024 15:49:41 +0000
Received: ("Tessian outbound 081d54b8cc4c:v228");
 Thu, 15 Feb 2024 15:49:41 +0000
Received: from 0fdfc400244d.2
 by 64aa7808-outbound-1.mta.getcheckrecipient.com id
 1B7BED59-95BE-4BCA-84CA-642802FD5C86.1; 
 Thu, 15 Feb 2024 15:49:30 +0000
Received: from EUR04-HE1-obe.outbound.protection.outlook.com
 by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 0fdfc400244d.2
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
 Thu, 15 Feb 2024 15:49:30 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com (2603:10a6:20b:8f::22)
 by AS8PR08MB6375.eurprd08.prod.outlook.com (2603:10a6:20b:31b::23)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.29; Thu, 15 Feb
 2024 15:49:27 +0000
Received: from AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::99ee:f28d:84bc:97cb]) by AM6PR08MB3749.eurprd08.prod.outlook.com
 ([fe80::99ee:f28d:84bc:97cb%7]) with mapi id 15.20.7270.036; Thu, 15 Feb 2024
 15:49:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8319b3d-cc19-11ee-98f5-efadbce2ee36
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
 b=UmkV5bZEB3zMhJ2mrLRgAQAwmsoFjzKdieAHydW0oAVzWscut0P0n25vR4+PwEyc16rW8B+ny0IYX/fWn1cDCNH1hVz5g/zEwgegEc9i0Wn2nXMkAxSyfpznTDc/4Czhseu0HZZxd4mpA6RFPnAWH7DM5s88pig48kXXfyxvru0x5yZthTl8v4mBXYWCOYaji7v9HVR6II4RRH7JhdE6wcGk/xRwgGMzoPgDU0eCdf5tsQ4wUrrS/JxcrniT2GeA2Igbq9rhDb1nblGGe1IvmUV7nsxhCvOaQ4MK3/QMsbwy8jx0QbcoLoK7Yu9up3rlwqE64jqYgvPzIGt/T2g5EA==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=8WCZ1M3VC/1KYKTG9M3PBbed7brqmkEF+fhFYWwZpf0=;
 b=i6o20O7YdLX8WTsxuSzJq3IxBPDkppPNocGt3AF5cBZhBAr9gtZ2ZtyXMeT3ZqaaWFcPaTJF/2ZU0ncbuiffFhWbef6STL9jidH/laYpaKaGCR7R3tTMMq4RqwzxewuGjJbA6s5DUFpG//pt10Lm/v88tDtAjwk6ueyd3ILJBb6dec4wCDFSJ82Y5fHbjGP/RBosEe/ato1PA9eFxkJYiFFF0YPgDzpVpEcyTIrTQH0KkU/cvkkkfs50b2v/Evou/Nicfidc66okylI7A5ZV1c4NxwOW/liLhjYdftqYIAFqfwcOVWRkUSazMk9+5qNaUfiS6b09clILaYpJy/e+yQ==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com;
 dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com;
 dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0
 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
 dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8WCZ1M3VC/1KYKTG9M3PBbed7brqmkEF+fhFYWwZpf0=;
 b=2FBp74bUt6YEbHnZM7n2p+rWHjt+klGFDZGV4r8BEWB0w/3MmkFFv1DCe7xmi+7su7uU7XSuKLXRtbfgBxeDedPvXj/CxQSAKYbMRbrmfMyabKQbfpu0B2KpH2ES05ESM2D1kjqcvpqhQb7kuKQSqtyJhOSoJCU06RWvf+1XxAg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
 smtp.mailfrom=arm.com; dkim=pass (signature was verified)
 header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
X-CheckRecipientChecked: true
X-CR-MTA-CID: 3f755adf48f08db8
X-CR-MTA-TID: 64aa7808
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iXVEGdfc6jyC/7NsxXStFl9jY49dFPo/naMFxKWwBdTz5ttamhZh2J0x4MaaFpB3u7apunAPS8d+ZxDBLBjQggPQ0b7OuRvwy7i06miXudOJkP9HsMxkBxyh8URXDySqHGQPz7khIVw++ZLXPpbB+K4PKtL4+4ofekl8wyRWxLm4on3yIHOI8RilmtovwDmHaVvfXSoU96wY0MypXki+W80OMu1yGS62bznpIFhnKgq6cM3NwIq5rLSZDNVcVni7YNMxU1s+JjfAe2adnXqqjba17J2/Z4ZLONxVE0EwM03b40IpugB4g+gMKKUoHn+UQFV1adeMbJkP2VDUMdxofQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=8WCZ1M3VC/1KYKTG9M3PBbed7brqmkEF+fhFYWwZpf0=;
 b=c7zWhiFaLoAimoxhRner2uu3OMN9s0O4bdAboTHcQD0N1y6L9atMiuJ3rF4oPeKVnxxV4FfJV01vw2NJaSAtySZ6snf0/XrFETkOOr5vX2YYq5ylgefvC7253OhLdAXKOcmMJK7ninqAskqegsSNm0XWTH3xdTubgAnt2ksnOQ6wHVn7h/YPOKne5K7Re0rUf13KwEwT4ADR8V0F+xBOxR6G3mVjWJDB6ReMvnNGo0XyJBIC9dhqK/boiHOuA58MORa3P3asDdLLb5m9rNG1r3KhtrboFwLpvjXEBkBr6PxwDfbrlPiG5maz+B3Wh5H0225+ijE5XlYtYZqImm8E4A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
 header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
 s=selector2-armh-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=8WCZ1M3VC/1KYKTG9M3PBbed7brqmkEF+fhFYWwZpf0=;
 b=2FBp74bUt6YEbHnZM7n2p+rWHjt+klGFDZGV4r8BEWB0w/3MmkFFv1DCe7xmi+7su7uU7XSuKLXRtbfgBxeDedPvXj/CxQSAKYbMRbrmfMyabKQbfpu0B2KpH2ES05ESM2D1kjqcvpqhQb7kuKQSqtyJhOSoJCU06RWvf+1XxAg=
From: Luca Fancellu <Luca.Fancellu@arm.com>
To: Michal Orzel <michal.orzel@amd.com>
CC: Xen-devel <xen-devel@lists.xenproject.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<Bertrand.Marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH] xen/arm: Make hwdom vUART optional feature
Thread-Topic: [PATCH] xen/arm: Make hwdom vUART optional feature
Thread-Index: AQHaYBz49ii/8gxOY0aM41gWsHqs/bELjNUA
Date: Thu, 15 Feb 2024 15:49:27 +0000
Message-ID: <44B29A66-D497-439B-B6AA-C6A2CC1115D8@arm.com>
References: <20240215143947.90073-1-michal.orzel@amd.com>
In-Reply-To: <20240215143947.90073-1-michal.orzel@amd.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3774.400.31)
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
x-ms-traffictypediagnostic:
	AM6PR08MB3749:EE_|AS8PR08MB6375:EE_|DU2PEPF00028CFC:EE_|DB5PR08MB9998:EE_
X-MS-Office365-Filtering-Correlation-Id: 41b4d6d2-ca3b-4bce-33e8-08dc2e3db929
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original:
 rE5qTt87AuiQayUVzFSOqyuBp98zGOIZeUzuxZxV39nNCPIaT52z891q46Fic2WdRUZlW6avqUk7zFrCwb4ROqpAU2P66x0KDw1FQBKmQlzMNn8/zEBGNvIZj5AzPlw5EXxuK1iGGi/Ewv1MqNRKxJ+cDCy2wMvjEuJzuijCch/Tk1Xkijh9ZkfNUqbSQ2sJClwZdX3/VUVgZvOMdjz6twGrPVMVjLDt8q58CESYqtMaybHsywKB/pqYTCGXac8T5Gg6p/O5Qz5MGsOJpMOJl9XCLZOd1ymiZMiFXRiXxqDTdqcEhEkuDiP3xsQLcYHwSPTd2JqB1kMQqdzHgwQelXCKjadbdGH7z/8MwAEwk0MO4PLxJRICpyUNUvjSZ9OucsBJH+oL9CducuwV3c5eWF9j1vJ3j1/Qe2lxNc1jX/IbvZdtO2YF9QvxzGVuLDIkAU3sZLfhrCUz0rG99XW53zV2Fz6zyNOIgimvb+m56jRk/SSUPTazJVrN3m3Olufo2Cmm6481HdJiB7K4tVh4Vlr+CkfRS9+RIm/r4gdIxc8pkTaRKR6OjfQy0l/OeWEkPsFAUUjvZUWxGOWwNmQ1qtFyvmV0qoIExMbyhwf4s2ogppZgLkRSK6RerTtOVI9M
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM6PR08MB3749.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(366004)(376002)(136003)(396003)(39860400002)(230922051799003)(186009)(64100799003)(451199024)(1800799012)(33656002)(36756003)(86362001)(4326008)(8936002)(6916009)(8676002)(64756008)(66476007)(66446008)(66556008)(66946007)(76116006)(5660300002)(2906002)(4744005)(83380400001)(38100700002)(122000001)(38070700009)(41300700001)(71200400001)(53546011)(2616005)(316002)(54906003)(26005)(478600001)(6486002)(6512007)(6506007);DIR:OUT;SFP:1101;
Content-Type: text/plain; charset="utf-8"
Content-ID: <5CD5026B82889847840D75352ED950C4@eurprd08.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6375
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU2PEPF00028CFC.eurprd03.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	1d8dd242-23b2-4b9b-46d7-08dc2e3db0a2
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	EhSmtpC5+sUWxeNC72unvJTM6ouyMXkOsAlgwptwT0YgOKW0JlJu4rVWCbcaTxpO2xcDDcela0U6CyoWWV6/r8YgvcynAXiIf0Zce3QExOUxjKl2uAfCSPutZaHkuWsN2ZppKR3A86kGV5TSIN2p8IpfVrogjjYFdT4A+9hVp953p4xU1oyjWC0H7zwD/OLz2nAgydV+t0nu82vNtp/wWbJv5X7o1hi4Td5EBQBz48qar5T2tIQQEnH02qDP8oJ9GYy2waL629e6VWAqb7AQGtMjF1wqWEdW6R5Dmw/Bb4sgXJP/pOKE43jVImYwxm27Wf7cI72Z0UOJP1+whAbdqGrosZrdsCTjc8kIY/MyZgvpPi2eGApWkyQi0yR5h9r1GZ8HH6tK5tBhR5tuU77ATCRObwMA7ELbf+6JC2UvRg6G8J9xzfz2iNn9Vy+SPz/0MNMhfGv0UAtknCPzujJ2d90xHPSrBh/DVCCo/TyRD8zmutZfGrvbVy4a1FN9u1KvSLj4OXIto05J+gNdBTldqWEpMH+sdRhU+kI0hVLrc91kmXY34SZMYAsqjYY6NQONnlj40n17OKT+l7GDcYVUxKeLYTXOPoURBscAsSpIXjnA05+KjxRFjDDJ3SfwP2KiqGQs79fXIXZBNGMQAbyKEu/KMqlXRhN8+4aUjUxPjk8=
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(4636009)(376002)(396003)(346002)(136003)(39860400002)(230922051799003)(1800799012)(451199024)(64100799003)(36860700004)(186009)(82310400011)(40470700004)(46966006)(8676002)(70206006)(70586007)(5660300002)(2906002)(4326008)(6862004)(8936002)(356005)(107886003)(81166007)(33656002)(36756003)(83380400001)(86362001)(82740400003)(53546011)(478600001)(316002)(6506007)(6486002)(6512007)(54906003)(26005)(2616005)(336012)(41300700001);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 15:49:41.3979
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 41b4d6d2-ca3b-4bce-33e8-08dc2e3db929
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU2PEPF00028CFC.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR08MB9998

DQoNCj4gT24gMTUgRmViIDIwMjQsIGF0IDE0OjM5LCBNaWNoYWwgT3J6ZWwgPG1pY2hhbC5vcnpl
bEBhbWQuY29tPiB3cm90ZToNCj4gDQo+IEF0IHRoZSBtb21lbnQsIHRoZSBoYXJkd2FyZSBkb21h
aW4gdlVBUlQgaXMgYWx3YXlzIGNvbXBpbGVkIGluLiBJbiB0aGUNCj4gc3Bpcml0IG9mIGZpbmUg
Z3JhbnVsYXIgY29uZmlndXJhdGlvbiwgbWFrZSBpdCBvcHRpb25hbCBzbyB0aGF0IHRoZQ0KPiBm
ZWF0dXJlIGNhbiBiZSBkaXNhYmxlZCBpZiBub3QgbmVlZGVkLiBUaGlzIFVBUlQgaXMgbm90IGV4
cG9zZWQgKGUuZy4NCj4gdmlhIGRldmljZSB0cmVlKSB0byBhIGRvbWFpbiBhbmQgaXMgbW9zdGx5
IHVzZWQgdG8gc3VwcG9ydCBzcGVjaWFsIHVzZQ0KPiBjYXNlcyBsaWtlIExpbnV4IGVhcmx5IHBy
aW50aywgcHJpbnRzIGZyb20gdGhlIGRlY29tcHJlc3NvciBjb2RlLCBldGMuDQo+IA0KPiBJbnRy
b2R1Y2UgS2NvbmZpZyBvcHRpb24gQ09ORklHX0hXRE9NX1ZVQVJULCBlbmFibGVkIGJ5IGRlZmF1
bHQgKHRvIGtlZXANCj4gdGhlIGN1cnJlbnQgYmVoYXZpb3IpIGFuZCB1c2UgaXQgdG8gcHJvdGVj
dCB0aGUgdlVBUlQgcmVsYXRlZCBjb2RlLg0KPiBQcm92aWRlIHN0dWJzIGZvciBkb21haW5fdnVh
cnRfe2luaXQsZnJlZX0oKSBpbiBjYXNlIHRoZSBmZWF0dXJlIGlzDQo+IGRpc2FibGVkLiBUYWtl
IHRoZSBvcHBvcnR1bml0eSB0byBhZGQgYSBzdHJ1Y3QgZG9tYWluIGZvcndhcmQgZGVjbGFyYXRp
b24NCj4gdG8gdnVhcnQuaCwgc28gdGhhdCB0aGUgaGVhZGVyIGlzIHNlbGYgY29udGFpbmVkLg0K
PiANCj4gU2lnbmVkLW9mZi1ieTogTWljaGFsIE9yemVsIDxtaWNoYWwub3J6ZWxAYW1kLmNvbT4N
Cj4gLS0tDQoNCkhpIE1pY2hhbCwNCg0KTG9va3MgZ29vZCB0byBtZSwgSeKAmXZlIGFsc28gdGVz
dGVkIHdpdGggYW5kIHdpdGhvdXQgdnVhcnQgb24gcWVtdSBhcm02NC4NCg0KUmV2aWV3ZWQtYnk6
IEx1Y2EgRmFuY2VsbHUgPGx1Y2EuZmFuY2VsbHVAYXJtLmNvbT4NClRlc3RlZC1ieTogTHVjYSBG
YW5jZWxsdSA8bHVjYS5mYW5jZWxsdUBhcm0uY29tPg0KDQoNCg==


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 15:54:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 15:54:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681883.1060867 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rae4d-0008Pz-5g; Thu, 15 Feb 2024 15:54:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681883.1060867; Thu, 15 Feb 2024 15:54:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rae4d-0008Ps-1c; Thu, 15 Feb 2024 15:54:55 +0000
Received: by outflank-mailman (input) for mailman id 681883;
 Thu, 15 Feb 2024 15:54:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wY2D=JY=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rae4c-0008Pm-4Q
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 15:54:54 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8da72251-cc1a-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 16:54:51 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-511898b6c9eso1261574e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 07:54:51 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 a15-20020a0ce34f000000b0068c8be959a0sm754837qvm.111.2024.02.15.07.54.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 07:54:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8da72251-cc1a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708012491; x=1708617291; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=LcGsWgY1QzuM4kw98vczSP/1ZmF/SUD7G9dHQBs6D4Q=;
        b=HLIo8YKc+7Pp81VTJZWpEckTj+6vQA71RCTbRyXqmWLLQsCWqLmktUVLqtzYbvk5Gj
         BMXai5mavuHlx4TM1kn9zjgU5mj6Z+q7fwg0yWKzBmMsSjG9C0TBkgiJcIiC7DFMQVvT
         KBlVlws+eXituYmjVH+pHibJBLQJSqWSJh+WA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708012491; x=1708617291;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=LcGsWgY1QzuM4kw98vczSP/1ZmF/SUD7G9dHQBs6D4Q=;
        b=dGE9YNBF9JWijsIECdcsn1Y6lWGtkA2ki6fQYTrwQlqvgUVtTRtf/DA0qG+6kbxd5y
         bGsn379ZElgiVFAGsaZApXpRyZzYWkjsvGkpkO/kMLEuH3U8MNwO75F8KXwmGGFBD3g8
         xCGgy1ZohdbSNhPgTrWMW5guEruvXKHq7ZgSgkfkEAAAxI1dF9L2kKk8udnI0QXyw/+J
         nXvnQ7BdEaGk3XgPtrNhsxHJ+gisyPs/o1HVtwjmxCTMMNGHRPGkHFyWqZcjX3rM/lpC
         2K8T8G17P0BQTU2fXv1AMD74/oInLjloKwmBnnxl0fpKvTOvYtlP5f/xnl9WMmUYCDzq
         JHPQ==
X-Forwarded-Encrypted: i=1; AJvYcCXTTJuWb63stCBK9jS7Kz9YX7VcA9kk+TsG8EVeSrO6GcIKgGMcQWnAfTRYpGvCWnJGmygJEXOHf/PyUF6EepjGIAlthZ0QB1Z6zcN73yI=
X-Gm-Message-State: AOJu0YzhVNTTE8lvYCt4+q/OAwgr4JwxZmnsCAZzzM6GtY6nRneNo+31
	Q2QHT/OPMY6jNRAeKF6u5giCLe3/xB9L696YR7wlugggncDYH134FgItWE7Mjdo=
X-Google-Smtp-Source: AGHT+IGDma+QqjEPUX1iZm7u+lPP20IgU/erE/04BDihlAa6eXITvsVZcUIqVXHh+n4jv//6ve201Q==
X-Received: by 2002:a05:6512:124c:b0:511:85ae:8985 with SMTP id fb12-20020a056512124c00b0051185ae8985mr1944640lfb.53.1708012490824;
        Thu, 15 Feb 2024 07:54:50 -0800 (PST)
Date: Thu, 15 Feb 2024 16:54:48 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v2] x86/vmx: add support for virtualize SPEC_CTRL
Message-ID: <Zc4zyLgM-OUNc6xG@macbook>
References: <20240209114045.97005-1-roger.pau@citrix.com>
 <09cf678e-bdbd-47a9-8c9d-533e7bee9450@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <09cf678e-bdbd-47a9-8c9d-533e7bee9450@suse.com>

On Mon, Feb 12, 2024 at 03:09:01PM +0100, Jan Beulich wrote:
> On 09.02.2024 12:40, Roger Pau Monne wrote:
> > @@ -1378,6 +1379,10 @@ static int construct_vmcs(struct vcpu *v)
> >          rc = vmx_add_msr(v, MSR_PRED_CMD, PRED_CMD_IBPB,
> >                           VMX_MSR_HOST);
> >  
> > +    /* Set any bits we don't allow toggling in the mask field. */
> > +    if ( cpu_has_vmx_virt_spec_ctrl && v->arch.msrs->spec_ctrl.raw )
> > +        __vmwrite(SPEC_CTRL_MASK, v->arch.msrs->spec_ctrl.raw);
> 
> The right side of the conditional isn't strictly necessary here, is it?
> Might it be better to omit it, for clarity?

No strong opinion, my thinking was that skipping the vmwrite would be
better performance wise, but we don't care about performance here
anyway.

> > --- a/xen/arch/x86/hvm/vmx/vmx.c
> > +++ b/xen/arch/x86/hvm/vmx/vmx.c
> > @@ -823,18 +823,29 @@ static void cf_check vmx_cpuid_policy_changed(struct vcpu *v)
> >      {
> >          vmx_clear_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
> >  
> > -        rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
> > -        if ( rc )
> > -            goto out;
> > +        if ( !cpu_has_vmx_virt_spec_ctrl )
> > +        {
> > +            rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
> > +            if ( rc )
> > +                goto out;
> > +        }
> >      }
> >      else
> >      {
> >          vmx_set_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
> >  
> > -        rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
> > -        if ( rc && rc != -ESRCH )
> > -            goto out;
> > -        rc = 0; /* Tolerate -ESRCH */
> > +        /*
> > +         * NB: there's no need to clear the virtualize SPEC_CTRL control, as
> > +         * the MSR intercept takes precedence.  The SPEC_CTRL shadow VMCS field
> > +         * is also not loaded on guest entry/exit if the intercept is set.
> > +         */
> 
> It wasn't so much the shadow field than the mask one that I was concerned
> might be used in some way. The shadow one clearly is used only during
> guest RDMSR/WRMSR processing. To not focus on "shadow", maybe simple say
> "The SPEC_CTRL shadow VMCS fields are also not ..."?

What about:

"The SPEC_CTRL shadow and mask VMCS fields don't take effect if the
intercept is set."

> > +        if ( !cpu_has_vmx_virt_spec_ctrl )
> > +        {
> > +            rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
> > +            if ( rc && rc != -ESRCH )
> > +                goto out;
> > +            rc = 0; /* Tolerate -ESRCH */
> > +        }
> >      }
> >  
> >      /* MSR_PRED_CMD is safe to pass through if the guest knows about it. */
> > @@ -2629,6 +2640,9 @@ static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
> >      switch ( reg )
> >      {
> >      case MSR_SPEC_CTRL:
> > +        if ( cpu_has_vmx_virt_spec_ctrl )
> > +            /* Requires remote VMCS loaded - fetched below. */
> 
> I could see what "fetch" refers to here, but ...
> 
> > +            break;
> >          rc = vmx_read_guest_msr(v, reg, &val);
> >          if ( rc )
> >          {
> > @@ -2652,6 +2666,11 @@ static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
> >      vmx_vmcs_enter(v);
> >      switch ( reg )
> >      {
> > +    case MSR_SPEC_CTRL:
> > +        ASSERT(cpu_has_vmx_virt_spec_ctrl);
> > +        __vmread(SPEC_CTRL_SHADOW, &val);
> > +        break;
> > +
> >      case MSR_IA32_BNDCFGS:
> >          __vmread(GUEST_BNDCFGS, &val);
> >          break;
> > @@ -2678,6 +2697,9 @@ static void cf_check vmx_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
> >      switch ( reg )
> >      {
> >      case MSR_SPEC_CTRL:
> > +        if ( cpu_has_vmx_virt_spec_ctrl )
> > +            /* Requires remote VMCS loaded - fetched below. */
> 
> ... since you also use the word here, I'm not sure it's really
> the VMREAD up there.

That one should be 'set below'.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 16:06:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 16:06:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681893.1060876 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeFM-0002UR-43; Thu, 15 Feb 2024 16:06:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681893.1060876; Thu, 15 Feb 2024 16:06:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeFM-0002UK-1H; Thu, 15 Feb 2024 16:06:00 +0000
Received: by outflank-mailman (input) for mailman id 681893;
 Thu, 15 Feb 2024 16:05:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1raeFK-0002UE-Bz
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 16:05:58 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1raeFJ-0002ao-M3; Thu, 15 Feb 2024 16:05:57 +0000
Received: from [15.248.3.91] (helo=[10.24.67.27])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1raeFJ-00025p-Fz; Thu, 15 Feb 2024 16:05:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=rzCXWsyN+fWS91dgkWDFhBJoHWMF31PZniuXD9BFIx0=; b=yIeFSCUrF2ZiCKUKrD8eJzOPyd
	L20ULzGzdQkIdmF2bhrXlCS9p32xdHoGLfqpc1iMdq/h0BB8/4NYT/O2vn60Zz09hASfcd4ZW3pFq
	mfRaPpPk1t5W8OUAFo6zrcbNFjRl4EL3IagANnh/+LkR0HuoFnMb7qUwaG4YbnOpQnhY=;
Message-ID: <8ea2f0ef-29ab-4a50-a59f-1f01af2f1b2a@xen.org>
Date: Thu, 15 Feb 2024 16:05:55 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Make hwdom vUART optional feature
Content-Language: en-GB
To: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240215143947.90073-1-michal.orzel@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240215143947.90073-1-michal.orzel@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Michal,

On 15/02/2024 14:39, Michal Orzel wrote:
> At the moment, the hardware domain vUART is always compiled in. In the
> spirit of fine granular configuration, make it optional so that the
> feature can be disabled if not needed. This UART is not exposed (e.g.
> via device tree) to a domain and is mostly used to support special use
> cases like Linux early printk, prints from the decompressor code, etc.
> 
> Introduce Kconfig option CONFIG_HWDOM_VUART, enabled by default (to keep
> the current behavior) and use it to protect the vUART related code.
> Provide stubs for domain_vuart_{init,free}() in case the feature is
> disabled. Take the opportunity to add a struct domain forward declaration
> to vuart.h, so that the header is self contained.
> 
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> ---
>   xen/arch/arm/Kconfig              |  8 ++++++++
>   xen/arch/arm/Makefile             |  2 +-
>   xen/arch/arm/include/asm/domain.h |  2 ++
>   xen/arch/arm/vuart.h              | 15 +++++++++++++++
>   4 files changed, 26 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index 50e9bfae1ac8..72af329564b7 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -150,6 +150,14 @@ config SBSA_VUART_CONSOLE
>   	  Allows a guest to use SBSA Generic UART as a console. The
>   	  SBSA Generic UART implements a subset of ARM PL011 UART.
>   
> +config HWDOM_VUART
> +	bool "Emulated UART for hardware domain"
> +	default y
> +	help
> +	  Allows a hardware domain to use a minimalistic UART (single transmit
> +	  and status register) which takes information from dtuart. Note that this
> +	  UART is not intended to be exposed (e.g. via device-tree) to a domain.
> +
>   config ARM_SSBD
>   	bool "Speculative Store Bypass Disable" if EXPERT
>   	depends on HAS_ALTERNATIVE
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 33c677672fe6..7b1350e2ef0a 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -71,7 +71,7 @@ obj-y += vtimer.o
>   obj-$(CONFIG_SBSA_VUART_CONSOLE) += vpl011.o
>   obj-y += vsmc.o
>   obj-y += vpsci.o
> -obj-y += vuart.o
> +obj-$(CONFIG_HWDOM_VUART) += vuart.o
>   
>   extra-y += xen.lds
>   
> diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
> index 5fb8cd79c01a..8218afb8626a 100644
> --- a/xen/arch/arm/include/asm/domain.h
> +++ b/xen/arch/arm/include/asm/domain.h
> @@ -91,6 +91,7 @@ struct arch_domain
>   
>       struct vgic_dist vgic;
>   
> +#ifdef CONFIG_HWDOM_VUART
>       struct vuart {
>   #define VUART_BUF_SIZE 128
>           char                        *buf;
> @@ -98,6 +99,7 @@ struct arch_domain
>           const struct vuart_info     *info;
>           spinlock_t                  lock;
>       } vuart;
> +#endif
>   
>       unsigned int evtchn_irq;
>   #ifdef CONFIG_ACPI
> diff --git a/xen/arch/arm/vuart.h b/xen/arch/arm/vuart.h
> index bd23bd92f631..36658b4a8c7f 100644
> --- a/xen/arch/arm/vuart.h
> +++ b/xen/arch/arm/vuart.h
> @@ -20,9 +20,24 @@
>   #ifndef __ARCH_ARM_VUART_H__
>   #define __ARCH_ARM_VUART_H__
>   
> +struct domain;
> +
> +#ifdef CONFIG_HWDOM_VUART
> +
>   int domain_vuart_init(struct domain *d);
>   void domain_vuart_free(struct domain *d);
>   
> +#else
> +
> +static inline int domain_vuart_init(struct domain *d)
> +{
> +    return 0;

NIT: I was going to query why we return 0 rather than -EOPNOSUPP. But it 
looks like this is because domain_vuart_init() is unconditionally called 
for the hardware domain. This is unusual and would be worth adding a 
comment.

In any case,

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 16:06:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 16:06:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681897.1060887 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeGF-0002yA-CZ; Thu, 15 Feb 2024 16:06:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681897.1060887; Thu, 15 Feb 2024 16:06:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeGF-0002y3-9Q; Thu, 15 Feb 2024 16:06:55 +0000
Received: by outflank-mailman (input) for mailman id 681897;
 Thu, 15 Feb 2024 16:06:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4buu=JY=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1raeGD-0002qL-CQ
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 16:06:53 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3ac2fbdc-cc1c-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 17:06:51 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 9A9D51F8BA;
 Thu, 15 Feb 2024 16:06:49 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 6E86213A82;
 Thu, 15 Feb 2024 16:06:49 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id yRKGGZk2zmUWfwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 15 Feb 2024 16:06:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3ac2fbdc-cc1c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708013210; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=9tVIFMZw6BYNF6n4vpzTLBOVUekmRywRuvET2uc7LI4=;
	b=AVlNXnM6nTZR7uLwobXWnuZFcBw52ZnMGNh9bIkMuKRl3kc1KP+ABbi2qHyPhm7JsSa9Ik
	kgRjSJD4kTD3/IE+Lml22g/IZRQd7utUZztWI3lG24+KEKzQGUfrPT93wcFuwwGy9m9BMF
	xcBdJN8OZkMiAyWLNdIyraR0IBE/T6U=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708013209; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=9tVIFMZw6BYNF6n4vpzTLBOVUekmRywRuvET2uc7LI4=;
	b=VLuCZatLdretGsu17c3GrlP3NhoFTznHp47xBG+Dd/Or7GEAkzHkfYyRe1qCi0bKYgytrT
	NXFpa1moM8C1tlxKIFZ8Uy7/Qn+Y6VhBiDq61YJbxc5JZ+gFv0hEGTVfSlq387WqXN0Kyk
	ezBqmWBLYcgFfvmLGZbM9JmcSj1DbMw=
Message-ID: <4cabd0e1-4010-450e-aacd-80b19b477908@suse.com>
Date: Thu, 15 Feb 2024 17:06:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v7 03/21] tools/xen-9pfsd: add transport layer
Content-Language: en-US
To: xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jason Andryuk <jandryuk@gmail.com>
References: <20240215130509.24008-1-jgross@suse.com>
 <20240215130509.24008-4-jgross@suse.com>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <20240215130509.24008-4-jgross@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------PmAHb0jJzMRpXV2RqLT3So26"
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=VLuCZatL
X-Spamd-Result: default: False [-6.20 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 TO_DN_SOME(0.00)[];
	 HAS_ATTACHMENT(0.00)[];
	 MIME_BASE64_TEXT_BOGUS(1.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 MID_RHS_MATCH_FROM(0.00)[];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email,citrix.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 9A9D51F8BA
X-Spam-Level: 
X-Spam-Score: -6.20
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------PmAHb0jJzMRpXV2RqLT3So26
Content-Type: multipart/mixed; boundary="------------egQ0k5XkWKwBYvKuS8qG1Xl0";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Jason Andryuk <jandryuk@gmail.com>
Message-ID: <4cabd0e1-4010-450e-aacd-80b19b477908@suse.com>
Subject: Re: [PATCH v7 03/21] tools/xen-9pfsd: add transport layer
References: <20240215130509.24008-1-jgross@suse.com>
 <20240215130509.24008-4-jgross@suse.com>
In-Reply-To: <20240215130509.24008-4-jgross@suse.com>

--------------egQ0k5XkWKwBYvKuS8qG1Xl0
Content-Type: multipart/mixed; boundary="------------dXIHiq5xRB4NITX42hSPFSC2"

--------------dXIHiq5xRB4NITX42hSPFSC2
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMTUuMDIuMjQgMTQ6MDQsIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+IEFkZCB0aGUgdHJh
bnNwb3J0IGxheWVyIG9mIDlwZnMuIFRoaXMgaXMgYmFzaWNhbGx5IHRoZSBpbmZyYXN0cnVj
dHVyZQ0KPiB0byByZWNlaXZlIHJlcXVlc3RzIGZyb20gdGhlIGZyb250ZW5kIGFuZCB0byBz
ZW5kIHRoZSByZWxhdGVkIGFuc3dlcnMNCj4gdmlhIHRoZSByaW5ncy4NCj4gDQo+IFNpZ25l
ZC1vZmYtYnk6IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT4NCj4gUmV2aWV3ZWQt
Ynk6IEphc29uIEFuZHJ5dWsgPGphbmRyeXVrQGdtYWlsLmNvbT4NCj4gQWNrZWQtYnk6IEFu
dGhvbnkgUEVSQVJEIDxhbnRob255LnBlcmFyZEBjaXRyaXguY29tPg0KDQpTZWVtcyBDSSBp
cyB1bmhhcHB5IGR1ZSB0byBhIGZhbHNlIHBvc2l0aXZlLg0KDQpUaGUgZm9sbG93aW5nIGRp
ZmYgbmVlZHMgdG8gYmUgZm9sZGVkIGluOg0KDQpkaWZmIC0tZ2l0IGEvdG9vbHMvOXBmc2Qv
aW8uYyBiL3Rvb2xzLzlwZnNkL2lvLmMNCmluZGV4IDQzMTJhNjJkZmUuLjk5NjAxN2E4YmUg
MTAwNjQ0DQotLS0gYS90b29scy85cGZzZC9pby5jDQorKysgYi90b29scy85cGZzZC9pby5j
DQpAQCAtMTA0LDcgKzEwNCw3IEBAIHZvaWQgKmlvX3RocmVhZCh2b2lkICphcmcpDQogIHsN
CiAgICAgIHN0cnVjdCByaW5nICpyaW5nID0gYXJnOw0KICAgICAgdW5zaWduZWQgaW50IGNv
dW50ID0gMDsNCi0gICAgc3RydWN0IHA5X2hlYWRlciBoZHI7DQorICAgIHN0cnVjdCBwOV9o
ZWFkZXIgaGRyID0geyAuc2l6ZSA9IDAgfTsNCiAgICAgIGJvb2wgaW5faGRyID0gdHJ1ZTsN
Cg0KICAgICAgcmluZy0+bWF4X3NpemUgPSByaW5nLT5yaW5nX3NpemU7DQoNCg0KSnVlcmdl
bg0KDQo+IC0tLQ0KPiBWMjoNCj4gLSByZW5hbWUgcHV0X3JlcXVlc3RfYnl0ZXMoKSAoSmFz
b24gQW5kcnl1aykNCj4gLSByZW5hbWUgZ2V0X3JlcXVlc3RfYnl0ZXMoKSBhbmQgcHV0X3Jl
c3BvbnNlX2J5dGVzKCkgbGVuIHBhcmFtZXRlcg0KPiAgICAoSmFzb24gQW5kcnl1aykNCj4g
LSBkb24ndCB1bm1hc2sgZXZlbnQgY2hhbm5lbCBpZiBlcnJvciBpbmRpY2F0b3IgaXMgc2V0
IChKYXNvbiBBbmRyeXVrKQ0KPiAtLS0NCj4gICB0b29scy85cGZzZC9pby5jICAgICAgICB8
IDE0MyArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKystDQo+ICAgdG9v
bHMvOXBmc2QveGVuLTlwZnNkLmggfCAgMTYgKysrKysNCj4gICAyIGZpbGVzIGNoYW5nZWQs
IDE1NiBpbnNlcnRpb25zKCspLCAzIGRlbGV0aW9ucygtKQ0KPiANCj4gZGlmZiAtLWdpdCBh
L3Rvb2xzLzlwZnNkL2lvLmMgYi90b29scy85cGZzZC9pby5jDQo+IGluZGV4IGViN2MxMzZl
MDkuLjQzMTJhNjJkZmUgMTAwNjQ0DQo+IC0tLSBhL3Rvb2xzLzlwZnNkL2lvLmMNCj4gKysr
IGIvdG9vbHMvOXBmc2QvaW8uYw0KPiBAQCAtNiwzOSArNiwxNzYgQEANCj4gICAgKiBDb3B5
cmlnaHQgKEMpIDIwMjQgSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2UuY29tPg0KPiAgICAq
DQo+ICAgICogSS9PIHRocmVhZCBoYW5kbGluZy4NCj4gKyAqDQo+ICsgKiBPbmx5IGhhbmRs
ZSBvbmUgcmVxdWVzdCBhdCBhIHRpbWUsIHB1c2hpbmcgb3V0IHRoZSBjb21wbGV0ZSByZXNw
b25zZQ0KPiArICogYmVmb3JlIGxvb2tpbmcgZm9yIHRoZSBuZXh0IHJlcXVlc3QuDQo+ICAg
ICovDQo+ICAgDQo+ICAgI2luY2x1ZGUgPHN0ZGJvb2wuaD4NCj4gKyNpbmNsdWRlIDxzdGRs
aWIuaD4NCj4gICAjaW5jbHVkZSA8c3RyaW5nLmg+DQo+ICAgI2luY2x1ZGUgPHN5c2xvZy5o
Pg0KPiArI2luY2x1ZGUgPHhlbmN0cmwuaD4gICAgICAgICAgIC8qIEZvciBjcHUgYmFycmll
cnMuICovDQo+ICsjaW5jbHVkZSA8eGVuLXRvb2xzL2NvbW1vbi1tYWNyb3MuaD4NCj4gICAN
Cj4gICAjaW5jbHVkZSAieGVuLTlwZnNkLmgiDQo+ICAgDQo+ICsvKg0KPiArICogTm90ZSB0
aGF0IHRoZSByaW5nIG5hbWVzICJpbiIgYW5kICJvdXQiIGFyZSBmcm9tIHRoZSBmcm9udGVu
ZCdzDQo+ICsgKiBwZXJzcGVjdGl2ZSwgc28gdGhlICJpbiIgcmluZyB3aWxsIGJlIHVzZWQg
Zm9yIHJlc3BvbnNlcyB0byB0aGUgZnJvbnRlbmQsDQo+ICsgKiB3aGlsZSB0aGUgIm91dCIg
cmluZyBpcyB1c2VkIGZvciByZXF1ZXN0cyBmcm9tIHRoZSBmcm9udGVuZCB0byB0aGUNCj4g
KyAqIGJhY2tlbmQuDQo+ICsgKi8NCj4gK3N0YXRpYyB1bnNpZ25lZCBpbnQgcmluZ19pbl9m
cmVlKHN0cnVjdCByaW5nICpyaW5nKQ0KPiArew0KPiArICAgIHVuc2lnbmVkIGludCBxdWV1
ZWQ7DQo+ICsNCj4gKyAgICBxdWV1ZWQgPSB4ZW5fOXBmc19xdWV1ZWQocmluZy0+cHJvZF9w
dnRfaW4sIHJpbmctPmludGYtPmluX2NvbnMsDQo+ICsgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHJpbmctPnJpbmdfc2l6ZSk7DQo+ICsgICAgeGVuX3JtYigpOw0KPiArDQo+ICsg
ICAgcmV0dXJuIHJpbmctPnJpbmdfc2l6ZSAtIHF1ZXVlZDsNCj4gK30NCj4gKw0KPiArc3Rh
dGljIHVuc2lnbmVkIGludCByaW5nX291dF9kYXRhKHN0cnVjdCByaW5nICpyaW5nKQ0KPiAr
ew0KPiArICAgIHVuc2lnbmVkIGludCBxdWV1ZWQ7DQo+ICsNCj4gKyAgICBxdWV1ZWQgPSB4
ZW5fOXBmc19xdWV1ZWQocmluZy0+aW50Zi0+b3V0X3Byb2QsIHJpbmctPmNvbnNfcHZ0X291
dCwNCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcmluZy0+cmluZ19zaXplKTsN
Cj4gKyAgICB4ZW5fcm1iKCk7DQo+ICsNCj4gKyAgICByZXR1cm4gcXVldWVkOw0KPiArfQ0K
PiArDQo+ICtzdGF0aWMgdW5zaWduZWQgaW50IGdldF9yZXF1ZXN0X2J5dGVzKHN0cnVjdCBy
aW5nICpyaW5nLCB1bnNpZ25lZCBpbnQgb2ZmLA0KPiArICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB1bnNpZ25lZCBpbnQgdG90YWxfbGVuKQ0KPiArew0KPiArICAg
IHVuc2lnbmVkIGludCBzaXplOw0KPiArICAgIHVuc2lnbmVkIGludCBvdXRfZGF0YSA9IHJp
bmdfb3V0X2RhdGEocmluZyk7DQo+ICsgICAgUklOR19JRFggcHJvZCwgY29uczsNCj4gKw0K
PiArICAgIHNpemUgPSBtaW4odG90YWxfbGVuIC0gb2ZmLCBvdXRfZGF0YSk7DQo+ICsgICAg
cHJvZCA9IHhlbl85cGZzX21hc2socmluZy0+aW50Zi0+b3V0X3Byb2QsIHJpbmctPnJpbmdf
c2l6ZSk7DQo+ICsgICAgY29ucyA9IHhlbl85cGZzX21hc2socmluZy0+Y29uc19wdnRfb3V0
LCByaW5nLT5yaW5nX3NpemUpOw0KPiArICAgIHhlbl85cGZzX3JlYWRfcGFja2V0KHJpbmct
PmJ1ZmZlciArIG9mZiwgcmluZy0+ZGF0YS5vdXQsIHNpemUsDQo+ICsgICAgICAgICAgICAg
ICAgICAgICAgICAgcHJvZCwgJmNvbnMsIHJpbmctPnJpbmdfc2l6ZSk7DQo+ICsNCj4gKyAg
ICB4ZW5fcm1iKCk7ICAgICAgICAgICAvKiBSZWFkIGRhdGEgb3V0IGJlZm9yZSBzZXR0aW5n
IHZpc2libGUgY29uc3VtZXIuICovDQo+ICsgICAgcmluZy0+Y29uc19wdnRfb3V0ICs9IHNp
emU7DQo+ICsgICAgcmluZy0+aW50Zi0+b3V0X2NvbnMgPSByaW5nLT5jb25zX3B2dF9vdXQ7
DQo+ICsNCj4gKyAgICAvKiBTaWduYWwgdGhhdCBtb3JlIHNwYWNlIGlzIGF2YWlsYWJsZSBu
b3cuICovDQo+ICsgICAgeGVuZXZ0Y2huX25vdGlmeSh4ZSwgcmluZy0+ZXZ0Y2huKTsNCj4g
Kw0KPiArICAgIHJldHVybiBzaXplOw0KPiArfQ0KPiArDQo+ICtzdGF0aWMgdW5zaWduZWQg
aW50IHB1dF9yZXNwb25zZV9ieXRlcyhzdHJ1Y3QgcmluZyAqcmluZywgdW5zaWduZWQgaW50
IG9mZiwNCj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2ln
bmVkIGludCB0b3RhbF9sZW4pDQo+ICt7DQo+ICsgICAgdW5zaWduZWQgaW50IHNpemU7DQo+
ICsgICAgdW5zaWduZWQgaW50IGluX2RhdGEgPSByaW5nX2luX2ZyZWUocmluZyk7DQo+ICsg
ICAgUklOR19JRFggcHJvZCwgY29uczsNCj4gKw0KPiArICAgIHNpemUgPSBtaW4odG90YWxf
bGVuIC0gb2ZmLCBpbl9kYXRhKTsNCj4gKyAgICBwcm9kID0geGVuXzlwZnNfbWFzayhyaW5n
LT5wcm9kX3B2dF9pbiwgcmluZy0+cmluZ19zaXplKTsNCj4gKyAgICBjb25zID0geGVuXzlw
ZnNfbWFzayhyaW5nLT5pbnRmLT5pbl9jb25zLCByaW5nLT5yaW5nX3NpemUpOw0KPiArICAg
IHhlbl85cGZzX3dyaXRlX3BhY2tldChyaW5nLT5kYXRhLmluLCByaW5nLT5idWZmZXIgKyBv
ZmYsIHNpemUsDQo+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICZwcm9kLCBjb25zLCBy
aW5nLT5yaW5nX3NpemUpOw0KPiArDQo+ICsgICAgeGVuX3dtYigpOyAgICAgICAgICAgLyog
V3JpdGUgZGF0YSBvdXQgYmVmb3JlIHNldHRpbmcgdmlzaWJsZSBwcm9kdWNlci4gKi8NCj4g
KyAgICByaW5nLT5wcm9kX3B2dF9pbiArPSBzaXplOw0KPiArICAgIHJpbmctPmludGYtPmlu
X3Byb2QgPSByaW5nLT5wcm9kX3B2dF9pbjsNCj4gKw0KPiArICAgIHJldHVybiBzaXplOw0K
PiArfQ0KPiArDQo+ICAgc3RhdGljIGJvb2wgaW9fd29ya19wZW5kaW5nKHN0cnVjdCByaW5n
ICpyaW5nKQ0KPiAgIHsNCj4gICAgICAgaWYgKCByaW5nLT5zdG9wX3RocmVhZCApDQo+ICAg
ICAgICAgICByZXR1cm4gdHJ1ZTsNCj4gLSAgICByZXR1cm4gZmFsc2U7DQo+ICsgICAgaWYg
KCByaW5nLT5lcnJvciApDQo+ICsgICAgICAgIHJldHVybiBmYWxzZTsNCj4gKyAgICByZXR1
cm4gcmluZy0+aGFuZGxlX3Jlc3BvbnNlID8gcmluZ19pbl9mcmVlKHJpbmcpIDogcmluZ19v
dXRfZGF0YShyaW5nKTsNCj4gICB9DQo+ICAgDQo+ICAgdm9pZCAqaW9fdGhyZWFkKHZvaWQg
KmFyZykNCj4gICB7DQo+ICAgICAgIHN0cnVjdCByaW5nICpyaW5nID0gYXJnOw0KPiArICAg
IHVuc2lnbmVkIGludCBjb3VudCA9IDA7DQo+ICsgICAgc3RydWN0IHA5X2hlYWRlciBoZHI7
DQo+ICsgICAgYm9vbCBpbl9oZHIgPSB0cnVlOw0KPiArDQo+ICsgICAgcmluZy0+bWF4X3Np
emUgPSByaW5nLT5yaW5nX3NpemU7DQo+ICsgICAgcmluZy0+YnVmZmVyID0gbWFsbG9jKHJp
bmctPm1heF9zaXplKTsNCj4gKyAgICBpZiAoICFyaW5nLT5idWZmZXIgKQ0KPiArICAgIHsN
Cj4gKyAgICAgICAgc3lzbG9nKExPR19DUklULCAibWVtb3J5IGFsbG9jYXRpb24gZmFpbHVy
ZSEiKTsNCj4gKyAgICAgICAgcmV0dXJuIE5VTEw7DQo+ICsgICAgfQ0KPiAgIA0KPiAgICAg
ICB3aGlsZSAoICFyaW5nLT5zdG9wX3RocmVhZCApDQo+ICAgICAgIHsNCj4gICAgICAgICAg
IHB0aHJlYWRfbXV0ZXhfbG9jaygmcmluZy0+bXV0ZXgpOw0KPiAgICAgICAgICAgaWYgKCAh
aW9fd29ya19wZW5kaW5nKHJpbmcpICkNCj4gICAgICAgICAgIHsNCj4gLSAgICAgICAgICAg
IGlmICggeGVuZXZ0Y2huX3VubWFzayh4ZSwgcmluZy0+ZXZ0Y2huKSA8IDAgKQ0KPiArICAg
ICAgICAgICAgaWYgKCAhcmluZy0+ZXJyb3IgJiYgeGVuZXZ0Y2huX3VubWFzayh4ZSwgcmlu
Zy0+ZXZ0Y2huKSA8IDAgKQ0KPiAgICAgICAgICAgICAgICAgICBzeXNsb2coTE9HX1dBUk5J
TkcsICJ4ZW5ldnRjaG5fdW5tYXNrKCkgZmFpbGVkIik7DQo+ICAgICAgICAgICAgICAgcHRo
cmVhZF9jb25kX3dhaXQoJnJpbmctPmNvbmQsICZyaW5nLT5tdXRleCk7DQo+ICAgICAgICAg
ICB9DQo+ICAgICAgICAgICBwdGhyZWFkX211dGV4X3VubG9jaygmcmluZy0+bXV0ZXgpOw0K
PiAgIA0KPiAtICAgICAgICAvKiBUT0RPOiBJL08gaGFuZGxpbmcuICovDQo+ICsgICAgICAg
IGlmICggcmluZy0+c3RvcF90aHJlYWQgfHwgcmluZy0+ZXJyb3IgKQ0KPiArICAgICAgICAg
ICAgY29udGludWU7DQo+ICsNCj4gKyAgICAgICAgaWYgKCAhcmluZy0+aGFuZGxlX3Jlc3Bv
bnNlICkNCj4gKyAgICAgICAgew0KPiArICAgICAgICAgICAgaWYgKCBpbl9oZHIgKQ0KPiAr
ICAgICAgICAgICAgew0KPiArICAgICAgICAgICAgICAgIGNvdW50ICs9IGdldF9yZXF1ZXN0
X2J5dGVzKHJpbmcsIGNvdW50LCBzaXplb2YoaGRyKSk7DQo+ICsgICAgICAgICAgICAgICAg
aWYgKCBjb3VudCAhPSBzaXplb2YoaGRyKSApDQo+ICsgICAgICAgICAgICAgICAgICAgIGNv
bnRpbnVlOw0KPiArICAgICAgICAgICAgICAgIGhkciA9ICooc3RydWN0IHA5X2hlYWRlciAq
KXJpbmctPmJ1ZmZlcjsNCj4gKyAgICAgICAgICAgICAgICBpZiAoIGhkci5zaXplID4gcmlu
Zy0+bWF4X3NpemUgfHwgaGRyLnNpemUgPCBzaXplb2YoaGRyKSApDQo+ICsgICAgICAgICAg
ICAgICAgew0KPiArICAgICAgICAgICAgICAgICAgICBzeXNsb2coTE9HX0VSUiwgIiV1LiV1
IHNwZWNpZmllZCBpbGxlZ2FsIHJlcXVlc3QgbGVuZ3RoICV1IiwNCj4gKyAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHJpbmctPmRldmljZS0+ZG9taWQsIHJpbmctPmRldmljZS0+ZGV2
aWQsIGhkci5zaXplKTsNCj4gKyAgICAgICAgICAgICAgICAgICAgcmluZy0+ZXJyb3IgPSB0
cnVlOw0KPiArICAgICAgICAgICAgICAgICAgICBjb250aW51ZTsNCj4gKyAgICAgICAgICAg
ICAgICB9DQo+ICsgICAgICAgICAgICAgICAgaW5faGRyID0gZmFsc2U7DQo+ICsgICAgICAg
ICAgICB9DQo+ICsNCj4gKyAgICAgICAgICAgIGNvdW50ICs9IGdldF9yZXF1ZXN0X2J5dGVz
KHJpbmcsIGNvdW50LCBoZHIuc2l6ZSk7DQo+ICsgICAgICAgICAgICBpZiAoIGNvdW50IDwg
aGRyLnNpemUgKQ0KPiArICAgICAgICAgICAgICAgIGNvbnRpbnVlOw0KPiArDQo+ICsgICAg
ICAgICAgICAvKiBUT0RPOiBoYW5kbGUgcmVxdWVzdCAod2lsbCByZXdyaXRlIGhkci5zaXpl
KS4gKi8NCj4gKw0KPiArICAgICAgICAgICAgcmluZy0+aGFuZGxlX3Jlc3BvbnNlID0gdHJ1
ZTsNCj4gKyAgICAgICAgICAgIGhkci5zaXplID0gKChzdHJ1Y3QgcDlfaGVhZGVyICopcmlu
Zy0+YnVmZmVyKS0+c2l6ZTsNCj4gKyAgICAgICAgICAgIGNvdW50ID0gMDsNCj4gKyAgICAg
ICAgfQ0KPiArDQo+ICsgICAgICAgIGlmICggcmluZy0+aGFuZGxlX3Jlc3BvbnNlICkNCj4g
KyAgICAgICAgew0KPiArICAgICAgICAgICAgY291bnQgKz0gcHV0X3Jlc3BvbnNlX2J5dGVz
KHJpbmcsIGNvdW50LCBoZHIuc2l6ZSk7DQo+ICsNCj4gKyAgICAgICAgICAgIGlmICggY291
bnQgPT0gaGRyLnNpemUgKQ0KPiArICAgICAgICAgICAgew0KPiArICAgICAgICAgICAgICAg
IC8qIFNpZ25hbCBwcmVzZW5jZSBvZiByZXNwb25zZS4gKi8NCj4gKyAgICAgICAgICAgICAg
ICB4ZW5ldnRjaG5fbm90aWZ5KHhlLCByaW5nLT5ldnRjaG4pOw0KPiArDQo+ICsgICAgICAg
ICAgICAgICAgcmluZy0+aGFuZGxlX3Jlc3BvbnNlID0gZmFsc2U7DQo+ICsgICAgICAgICAg
ICAgICAgaW5faGRyID0gdHJ1ZTsNCj4gKyAgICAgICAgICAgICAgICBjb3VudCA9IDA7DQo+
ICsgICAgICAgICAgICB9DQo+ICsgICAgICAgIH0NCj4gICAgICAgfQ0KPiAgIA0KPiArICAg
IGZyZWUocmluZy0+YnVmZmVyKTsNCj4gKw0KPiAgICAgICByaW5nLT50aHJlYWRfYWN0aXZl
ID0gZmFsc2U7DQo+ICAgDQo+ICAgICAgIHJldHVybiBOVUxMOw0KPiBkaWZmIC0tZ2l0IGEv
dG9vbHMvOXBmc2QveGVuLTlwZnNkLmggYi90b29scy85cGZzZC94ZW4tOXBmc2QuaA0KPiBp
bmRleCBmZjM1NjIxNjRhLi5kNTg3ZjU5YTMyIDEwMDY0NA0KPiAtLS0gYS90b29scy85cGZz
ZC94ZW4tOXBmc2QuaA0KPiArKysgYi90b29scy85cGZzZC94ZW4tOXBmc2QuaA0KPiBAQCAt
MTUsNiArMTUsMTIgQEANCj4gICAjZGVmaW5lIE1BWF9SSU5HX09SREVSICAgICAgICAgICA5
DQo+ICAgI2RlZmluZSBNQVhfT1BFTl9GSUxFU19ERUZBVUxUICAgNQ0KPiAgIA0KPiArc3Ry
dWN0IHA5X2hlYWRlciB7DQo+ICsgICAgdWludDMyX3Qgc2l6ZTsNCj4gKyAgICB1aW50OF90
IGNtZDsNCj4gKyAgICB1aW50MTZfdCB0YWc7DQo+ICt9IF9fYXR0cmlidXRlX18oKHBhY2tl
ZCkpOw0KPiArDQo+ICAgdHlwZWRlZiBzdHJ1Y3QgZGV2aWNlIGRldmljZTsNCj4gICANCj4g
ICBzdHJ1Y3QgcmluZyB7DQo+IEBAIC0yOSw3ICszNSwxNyBAQCBzdHJ1Y3QgcmluZyB7DQo+
ICAgICAgIHN0cnVjdCB4ZW5fOXBmc19kYXRhX2ludGYgKmludGY7DQo+ICAgICAgIHVuc2ln
bmVkIGludCByaW5nX29yZGVyOw0KPiAgICAgICBSSU5HX0lEWCByaW5nX3NpemU7DQo+ICsN
Cj4gKyAgICAvKiBUcmFuc3BvcnQgbGF5ZXIgZGF0YS4gKi8NCj4gICAgICAgc3RydWN0IHhl
bl85cGZzX2RhdGEgZGF0YTsNCj4gKyAgICBSSU5HX0lEWCBwcm9kX3B2dF9pbjsNCj4gKyAg
ICBSSU5HX0lEWCBjb25zX3B2dF9vdXQ7DQo+ICsNCj4gKyAgICAvKiBSZXF1ZXN0IGFuZCBy
ZXNwb25zZSBoYW5kbGluZy4gKi8NCj4gKyAgICB1aW50MzJfdCBtYXhfc2l6ZTsNCj4gKyAg
ICBib29sIGVycm9yOyAgICAgICAgICAgICAvKiBQcm90b2NvbCBlcnJvciAtIHN0b3AgcHJv
Y2Vzc2luZy4gKi8NCj4gKyAgICBib29sIGhhbmRsZV9yZXNwb25zZTsgICAvKiBNYWluIGxv
b3Agbm93IGhhbmRsaW5nIHJlc3BvbnNlLiAqLw0KPiArICAgIHZvaWQgKmJ1ZmZlcjsgICAg
ICAgICAgIC8qIFJlcXVlc3QvcmVzcG9uc2UgYnVmZmVyLiAqLw0KPiAgIH07DQo+ICAgDQo+
ICAgc3RydWN0IGRldmljZSB7DQoNCg==
--------------dXIHiq5xRB4NITX42hSPFSC2
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------dXIHiq5xRB4NITX42hSPFSC2--

--------------egQ0k5XkWKwBYvKuS8qG1Xl0--

--------------PmAHb0jJzMRpXV2RqLT3So26
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXONpkFAwAAAAAACgkQsN6d1ii/Ey8X
/Af+NioQfMcKGSd9ISBDmiBMY25xbY86T0VnOV8qAg2bdeahh+0kFmB6d2rLQCZVueSA2FII/ADB
EMCeGNQxjonaHOKK9Dh4sGV+47/vJmgTiq91LealgD1ka6D4nN9i+ii55ZgYhfGENycu2qfK8nv2
LibTrjclqsabc47bWvrC+Qqki2Ez1HzvXg/Q6qXWzXP9MhyyZo3Bx8nhMHG4DODrxtzC+Am1S0EX
1dspRMadJtuNgFqDyuCAnNnXYNeRHWX4grFxYFA5sujDiJHeAvi8P4RkeB+jWRrcerq6+JJuiPmm
dVoHKr40JU8h7+3Take9G56lxTwwufuOHTbVnwY3ng==
=ULUs
-----END PGP SIGNATURE-----

--------------PmAHb0jJzMRpXV2RqLT3So26--


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 16:07:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 16:07:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681901.1060897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeHC-0003YU-Oj; Thu, 15 Feb 2024 16:07:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681901.1060897; Thu, 15 Feb 2024 16:07:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeHC-0003YN-Le; Thu, 15 Feb 2024 16:07:54 +0000
Received: by outflank-mailman (input) for mailman id 681901;
 Thu, 15 Feb 2024 16:07:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raeHC-0003YH-3z
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 16:07:54 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5f82b2bc-cc1c-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 17:07:53 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-411f895c8b6so9038675e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 08:07:53 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 hg27-20020a05600c539b00b0040fc56712e8sm2341350wmb.17.2024.02.15.08.07.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 08:07:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f82b2bc-cc1c-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708013272; x=1708618072; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3CDoS0ibWubzHiJ00+gQSX3l991Mbf6fIF8GroC3PN0=;
        b=VKm8qpCnmL7o9FxWly8XqyE95lfpLeSNh887M66lkfz5qSng+5Ij2OLJp8KrQMf3Q7
         N2ys9xudnvX2Q/MVaebYFdUUL+BHGG4yDsdjwCsbebT/8EOQEPrJu9Lv0M/jodB+9Cew
         nhjKi1gOV43Zew8027nbLqfHDP7zMV752DV3tFk0Lq5PMoK0tAisnf4WY1B6myNBmb5S
         ayVS6MRSZ7GpOrWlSnWf3FCcXl9eOaId/Fx/EHnKOXd9U3ApCRodN85Vk5sEoVYfdKCa
         WF3Upxj2YEbSRMptB7geZOw07hCi0kcqBE5jAbWDmv8XfvE2ybWFqKgKjc+tvmInl58Z
         njiA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708013272; x=1708618072;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3CDoS0ibWubzHiJ00+gQSX3l991Mbf6fIF8GroC3PN0=;
        b=oxkF/kvSbm0QCFi+6CaRh4jX6Z6qbIGVhMfoO2lZyqR4oqew+wemVf/bBm59/Mhvou
         KMeX+bTYxE5rNqHoxHlmFe4Fd2UHPQYk2AV7vDWaJOHvfCZAsG9u0ggooVwcWlBQLFgv
         M1BrZYfbtdJRuU9N5qy5fr/W9Wkn/jnbkVjsYBfo2h903zK1nAHtPtHHVJQlohp6eILP
         EGOkuvQ7T0GImF4yXWyJBf83iLJPnYSTGMMyg/4MoThvs2O1DyWJUKqcButXwnJX4MB0
         dbb2KL6V8KnYeBy0GGfRdOT/f9bdVk06flNzsYvlCNYBwu2u4/HSuRbb6kRwc1uJqnBs
         cW3g==
X-Forwarded-Encrypted: i=1; AJvYcCVI3MKVaVLIa90e2hiqbV0+zfcqFHMtbkvK5Su/2n68P2LX133rT3WUykthr6KcZu6FkbXWg6GYHARM67Dh1tM6o7gCh4SI0pXdfFIEwqs=
X-Gm-Message-State: AOJu0YxpoXtaUyXkktl0oxHrXwtvFOyN568mkpwbyIHvznOMpP1KKMDq
	0oPZ0amE9zCVJIZK7KmmA9PfKSGr8oRW5msR7DTiWjTFsa83XZgpVHmFCdErcA==
X-Google-Smtp-Source: AGHT+IGcZ9P0nWTcyapnKtbaY1mV7T34TaEf7To7XFROE/A7E8/314DXM8uP1ObMDmtpZr7LSomjjg==
X-Received: by 2002:a05:600c:218f:b0:411:f0c2:8877 with SMTP id e15-20020a05600c218f00b00411f0c28877mr1776998wme.38.1708013272485;
        Thu, 15 Feb 2024 08:07:52 -0800 (PST)
Message-ID: <67306bde-9877-4d6d-a964-3ea964256ce4@suse.com>
Date: Thu, 15 Feb 2024 17:07:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/vmx: add support for virtualize SPEC_CTRL
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240209114045.97005-1-roger.pau@citrix.com>
 <09cf678e-bdbd-47a9-8c9d-533e7bee9450@suse.com> <Zc4zyLgM-OUNc6xG@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zc4zyLgM-OUNc6xG@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.02.2024 16:54, Roger Pau Monné wrote:
> On Mon, Feb 12, 2024 at 03:09:01PM +0100, Jan Beulich wrote:
>> On 09.02.2024 12:40, Roger Pau Monne wrote:
>>> --- a/xen/arch/x86/hvm/vmx/vmx.c
>>> +++ b/xen/arch/x86/hvm/vmx/vmx.c
>>> @@ -823,18 +823,29 @@ static void cf_check vmx_cpuid_policy_changed(struct vcpu *v)
>>>      {
>>>          vmx_clear_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
>>>  
>>> -        rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
>>> -        if ( rc )
>>> -            goto out;
>>> +        if ( !cpu_has_vmx_virt_spec_ctrl )
>>> +        {
>>> +            rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
>>> +            if ( rc )
>>> +                goto out;
>>> +        }
>>>      }
>>>      else
>>>      {
>>>          vmx_set_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
>>>  
>>> -        rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
>>> -        if ( rc && rc != -ESRCH )
>>> -            goto out;
>>> -        rc = 0; /* Tolerate -ESRCH */
>>> +        /*
>>> +         * NB: there's no need to clear the virtualize SPEC_CTRL control, as
>>> +         * the MSR intercept takes precedence.  The SPEC_CTRL shadow VMCS field
>>> +         * is also not loaded on guest entry/exit if the intercept is set.
>>> +         */
>>
>> It wasn't so much the shadow field than the mask one that I was concerned
>> might be used in some way. The shadow one clearly is used only during
>> guest RDMSR/WRMSR processing. To not focus on "shadow", maybe simple say
>> "The SPEC_CTRL shadow VMCS fields are also not ..."?
> 
> What about:
> 
> "The SPEC_CTRL shadow and mask VMCS fields don't take effect if the
> intercept is set."

SGTM.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 16:08:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 16:08:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681904.1060907 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeHe-0003zm-0R; Thu, 15 Feb 2024 16:08:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681904.1060907; Thu, 15 Feb 2024 16:08:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeHd-0003zf-Tm; Thu, 15 Feb 2024 16:08:21 +0000
Received: by outflank-mailman (input) for mailman id 681904;
 Thu, 15 Feb 2024 16:08:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wY2D=JY=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raeHd-0003YH-4U
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 16:08:21 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6ffe75e7-cc1c-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 17:08:20 +0100 (CET)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-5128d914604so361252e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 08:08:20 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 g11-20020ac87f4b000000b0042c7f378ed0sm670419qtk.63.2024.02.15.08.08.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 08:08:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6ffe75e7-cc1c-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708013300; x=1708618100; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=TG+wZCK2zcpJnQyGb0Dsyl/46ykMFXe9qhKtUhuqKw4=;
        b=B96oNeFPShRQL5/cmU9kt6Fo+CQatdalzMiLPnFB6P5o3VCALQ11XGycsQ3jeafPwF
         1qAWmpUq1u2cDHLPqbQe316IPrVjL2IE+efvYRDSGawt5PcEWNSSip2PUv7Mxl8S/IvF
         W1n9UIanwXXWgzdMLXimLD82MWdFc6MZp9FJM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708013300; x=1708618100;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=TG+wZCK2zcpJnQyGb0Dsyl/46ykMFXe9qhKtUhuqKw4=;
        b=Ov3Uo1NdqvovDVoluREEzLMDmCamOke3lMSCnJRjI9SJ5BDWa3B02kQyU3JnBRB4Ii
         kLzXRrJbtnm79Na4REwU6PuZWx08HEvhIHxWmczqqNuN3VALhAPuuWkhLdQVmay8fegg
         MjxDZ2Q5gI4H1pdjgfpAsfNzUZ7/HAC7lfOVmHaA4muqKcx+FSpwK/Z/afatXdRGu7Yo
         NTUkyVbWgA+6UhYuKECYAyXkvA+lOFQ0lE4qGx2VCMd4MY6ABqWoh5KIv+UfQ5GfJ9cR
         9CmbtOC0CRtXi1YIEx82eO7oZb6rM5YaMJGftkZQL8QS4/kcBVNmePA1L8OQC0p3W7zx
         Z04g==
X-Forwarded-Encrypted: i=1; AJvYcCXnPj38JZgoHAcQgnacs0lbAmoUOi2ZGH+NhBd6z50mTKhkx4PwVj0UQeAKxKkVZp54t4qYGHAnzozAUP/6+uJSCcuQ5osYhyxi5oM9NAo=
X-Gm-Message-State: AOJu0Yw6ZFl0q022xd9dE2wIk2fYc+Ud7YLeOM9aifw0q+jbcuPh6A3K
	XnVSRKih9prMTR5E9C7FbRT64bAqG/uFixhfMVKE/bplH2oLSHMdw2sni9RC708=
X-Google-Smtp-Source: AGHT+IEIwcFcuIPyyRn6BDl+F45km4t8tUrxrzYocXtVy6fFEFePO7K4LZ6os0kmg+NoKDX0lbboKA==
X-Received: by 2002:ac2:483c:0:b0:511:a40b:e5d0 with SMTP id 28-20020ac2483c000000b00511a40be5d0mr1612694lft.11.1708013300007;
        Thu, 15 Feb 2024 08:08:20 -0800 (PST)
Date: Thu, 15 Feb 2024 17:08:17 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony Perard <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Message-ID: <Zc428VMDoYnPw1zo@macbook>
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com>
 <Zc3nXpUOlnIHEfsl@macbook>
 <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com>
 <Zc3v20RKMssbaDsl@macbook>
 <a48cbad6-701d-4077-9044-4205b932a7f3@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <a48cbad6-701d-4077-9044-4205b932a7f3@suse.com>

On Thu, Feb 15, 2024 at 02:02:41PM +0100, Jan Beulich wrote:
> On 15.02.2024 12:04, Roger Pau Monné wrote:
> > On Thu, Feb 15, 2024 at 11:43:02AM +0100, Jan Beulich wrote:
> >> On 15.02.2024 11:28, Roger Pau Monné wrote:
> >>> On Thu, Feb 15, 2024 at 10:49:31AM +0100, Jan Beulich wrote:
> >>>> On 15.02.2024 10:30, Roger Pau Monne wrote:
> >>>>> --- a/xen/Makefile
> >>>>> +++ b/xen/Makefile
> >>>>> @@ -358,10 +358,10 @@ config: tools_fixdep outputmakefile FORCE
> >>>>>  else # !config-build
> >>>>>  
> >>>>>  ifeq ($(need-config),y)
> >>>>> --include include/config/auto.conf
> >>>>>  # Read in dependencies to all Kconfig* files, make sure to run syncconfig if
> >>>>>  # changes are detected.
> >>>>>  -include include/config/auto.conf.cmd
> >>>>> +include include/config/auto.conf
> >>>>
> >>>> With the - dropped, ...
> >>>>
> >>>>> @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
> >>>>>  # This exploits the 'multi-target pattern rule' trick.
> >>>>>  # The syncconfig should be executed only once to make all the targets.
> >>>>>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
> >>>>> +	rm -rf include/config/$*.conf
> >>>>>  	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
> >>>>
> >>>> ... is this really necessary? The error status from the sub-make is ignored
> >>>> only because of the -, isn't it?
> >>>
> >>> Without the `rm` the include/config/auto.conf is not removed by
> >>> Kconfig on error, so the include will still succeed but use the stale
> >>> auto.conf file.
> >>>
> >>> Keep in mind on rebuilds include/config/auto.conf is already present,
> >>> so the rule is only executed for the include/config/auto.conf.cmd
> >>> target.
> >>
> >> But the sub-make ought to return failure, which ought to then stop the
> >> build process?
> > 
> > For some reason it doesn't, not at least with GNU Make 4.3.
> > 
> > It stops the build if the '-' is dropped from the include of
> > include/config/auto.conf.cmd.  But that will always fail as
> > include/config/auto.conf.cmd is never created.
> > 
> > Maybe there's something weird with our makefile, I certainly don't
> > know that much, but as noted in the commit message,
> > include/config/auto.conf.cmd failing doesn't cause the build to
> > stop.
> 
> How about the below as an alternative? I'm not overly happy with the
> double ifneq, but I also don't see a good other option.

Hm, yes, having the checks against specific paths is IMO not ideal.

> This way the missing auto.conf is detected slightly later, but this
> may well be good enough. Then again I might be overlooking yet
> something else that this breaks ...
> 
> Jan
> 
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
>  # This exploits the 'multi-target pattern rule' trick.
>  # The syncconfig should be executed only once to make all the targets.
>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
> +	$(Q)rm -f include/config/$*.conf
>  	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
>  
>  ifeq ($(CONFIG_DEBUG),y)
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -15,7 +15,11 @@ srcdir := $(srctree)/$(src)
>  PHONY := __build
>  __build:
>  
> --include $(objtree)/include/config/auto.conf
> +ifneq ($(obj),tools)
> +ifneq ($(obj),tools/kconfig)
> +include $(objtree)/include/config/auto.conf
> +endif
> +endif

Trying to understand this, I assume it's to avoid an infinite
dependency loop that generating include/config/auto.conf requires some
tools that are build using xen/Rules.mk?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 16:22:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 16:22:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681923.1060917 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeUv-0007qr-5a; Thu, 15 Feb 2024 16:22:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681923.1060917; Thu, 15 Feb 2024 16:22:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeUv-0007qk-2N; Thu, 15 Feb 2024 16:22:05 +0000
Received: by outflank-mailman (input) for mailman id 681923;
 Thu, 15 Feb 2024 16:22:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raeUt-0007qc-Lr
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 16:22:03 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 59b9d3bc-cc1e-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 17:22:02 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d0a4e8444dso11299361fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 08:22:02 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z21-20020a05600c221500b004101f27737asm5457323wml.29.2024.02.15.08.22.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 08:22:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 59b9d3bc-cc1e-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708014122; x=1708618922; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+ajDoNXNUtH6hm2dr8oDNaxorI1c4VSR0dhnmF/C+Hw=;
        b=YgudTOR8Vg/S8q+o7g2DpkCF59DJgmyN2aSu7N5d2pq3jcMGMw3jJh4bitJ1pvnwyj
         vVOCywN7UWQIOowXWjamHVLbq0eBKzQkBnmuLlAQO4rUYbjd9UAict0MljSm9Ea+mAWM
         ei5muW4PnV06EWsCSRgvEqyPmAfxEqKGJoaLwRpvi5r08KeR1EKkKyYK4hm7H2Q9W3bj
         FQphRjRaEpU5gu3VhQsDNP3ndOrrwehFoe/4vgMMG7xascZtpTYbIR7FmQmzjnrENp9R
         e9Fpi5NJx1hN5Muav3rTSnus2urcKpjkBS8adBosgw9jFHzqdZhLG1lrgM2lVLQnK8Wt
         7mAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708014122; x=1708618922;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+ajDoNXNUtH6hm2dr8oDNaxorI1c4VSR0dhnmF/C+Hw=;
        b=e8FcKwKjvgaDU4odtZULkUMdSIDI+qSN0gmtA8rycPFj7HuU6C+aZxqNv6IeBerKST
         a7w8hjsRFexNBwGVsYT3kwTsPr2xm5xXaXxYNClRsQV+DvCbaH8y8nzIV0YWfHtCYL8V
         +dHk8ta8cAwXEfCJkdH/qfgKQe01PbMMs1poZIW70YmwSbMWlGm52l3l6kCdRJur9mts
         0Yu//kr1oK/PPwfNIl9Ehmaz0L2IPUUjBdYLuoF+fHrANX7xGy7ItU2teVgrMxmKTUln
         NFAer7/iXBdLV48MwJx18oQsGYr19L/QhTI1gsoQnQb1kBmFwC9lqGASVq/IvFIxpuDY
         0klg==
X-Forwarded-Encrypted: i=1; AJvYcCVCRme/e4AMRyIbTtv2NvIOuJr1o0biU7heqnSafDhdQ8vmkSeTAEhXnYSvP3H3r/ud/L5i7s//l6jAE2Gpm0JGHh1SM2Iw+ykm8yaADzk=
X-Gm-Message-State: AOJu0YwHWnY77vhdwudxh70a11PPw/GULuRYDMOStGLbkLxhM1jzw+My
	9V6Qh51nagmm2dtKyuFnlnFN49xQ4Puz10hV7Jba4JexETMqyaAxOjhU+VvMHA==
X-Google-Smtp-Source: AGHT+IHoMiRRAyWbUk53zBO+gf7oS14qLpogDEPsZFVIAGZI6wgY4XGtQZvaAqdZw2NOiOeMY3R3gQ==
X-Received: by 2002:a2e:98c7:0:b0:2d0:b0f4:89ee with SMTP id s7-20020a2e98c7000000b002d0b0f489eemr1556602ljj.47.1708014121739;
        Thu, 15 Feb 2024 08:22:01 -0800 (PST)
Message-ID: <d545cc6c-d213-43da-af31-1768af32aba0@suse.com>
Date: Thu, 15 Feb 2024 17:22:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Anthony Perard <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com> <Zc3nXpUOlnIHEfsl@macbook>
 <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com> <Zc3v20RKMssbaDsl@macbook>
 <a48cbad6-701d-4077-9044-4205b932a7f3@suse.com> <Zc428VMDoYnPw1zo@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zc428VMDoYnPw1zo@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.02.2024 17:08, Roger Pau Monné wrote:
> On Thu, Feb 15, 2024 at 02:02:41PM +0100, Jan Beulich wrote:
>> --- a/xen/Rules.mk
>> +++ b/xen/Rules.mk
>> @@ -15,7 +15,11 @@ srcdir := $(srctree)/$(src)
>>  PHONY := __build
>>  __build:
>>  
>> --include $(objtree)/include/config/auto.conf
>> +ifneq ($(obj),tools)
>> +ifneq ($(obj),tools/kconfig)
>> +include $(objtree)/include/config/auto.conf
>> +endif
>> +endif
> 
> Trying to understand this, I assume it's to avoid an infinite
> dependency loop that generating include/config/auto.conf requires some
> tools that are build using xen/Rules.mk?

The file has dependencies only in xen/Makefile. This is about the
file simply not being there when initially building. Perhaps the
patch description helps that I've written in the meantime:

"Because of using "-include", failure to (re)build auto.conf (with
 auto.conf.cmd produced as a secondary target) won't stop make from
 continuing the build. Arrange for it being possible to drop the - from
 Rules.mk, requiring that the include be skipped for tools-only targets.
 Note that relying on the inclusion in those cases wouldn't be correct
 anyway, as it might be a stale file (yet to be rebuilt) which would be
 included, while during initial build, the file would be absent
 altogether."

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 16:27:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 16:27:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681928.1060926 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeZx-0008RP-Nj; Thu, 15 Feb 2024 16:27:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681928.1060926; Thu, 15 Feb 2024 16:27:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeZx-0008RI-LF; Thu, 15 Feb 2024 16:27:17 +0000
Received: by outflank-mailman (input) for mailman id 681928;
 Thu, 15 Feb 2024 16:27:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raeZw-0008R8-LW; Thu, 15 Feb 2024 16:27:16 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raeZw-000302-Fo; Thu, 15 Feb 2024 16:27:16 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raeZw-0002PG-4V; Thu, 15 Feb 2024 16:27:16 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raeZw-0003eW-41; Thu, 15 Feb 2024 16:27:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=C3eK+FperCSYKCEBx0IL7Ow9uJ6/9sKjm5tzzCVZm5U=; b=G8yyMY/GdD377iAil6usq2Dy3q
	LivdP4nzL3pB/JjA/Y84SxeTnktHvqXj44PnI+A/H7/bLzibIqMPiwvtUk4/tXyJFoPXrmQ43kY4Q
	tft7kyPzzKk2E3S+qEiiTgn3n2oZRk3NgapXhuPXerwsArRoP1O0vE4Hgq5yt3pYg2Ow=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184675-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184675: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=c4d6802d33f962a306405a20f4dccdc287990d07
X-Osstest-Versions-That:
    xen=d9bfc2810e1cbe4cd27555bdc48315d36ee91f14
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 15 Feb 2024 16:27:16 +0000

flight 184675 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184675/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  c4d6802d33f962a306405a20f4dccdc287990d07
baseline version:
 xen                  d9bfc2810e1cbe4cd27555bdc48315d36ee91f14

Last test of basis   184667  2024-02-14 15:00:24 Z    1 days
Testing same since   184675  2024-02-15 13:03:50 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   d9bfc2810e..c4d6802d33  c4d6802d33f962a306405a20f4dccdc287990d07 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 16:38:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 16:38:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681936.1060936 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raekf-00021O-MU; Thu, 15 Feb 2024 16:38:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681936.1060936; Thu, 15 Feb 2024 16:38:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raekf-00021H-Jt; Thu, 15 Feb 2024 16:38:21 +0000
Received: by outflank-mailman (input) for mailman id 681936;
 Thu, 15 Feb 2024 16:38:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=G1Z8=JY=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raeke-00021B-IL
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 16:38:20 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a053a8f4-cc20-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 17:38:19 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d09cf00214so13854821fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 08:38:19 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 k2-20020a2ea282000000b002d0f905ddf9sm350166lja.18.2024.02.15.08.38.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 08:38:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a053a8f4-cc20-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708015099; x=1708619899; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=pFKN2F8CAqIawWvV7bli4CjJ841AKOlmGdXyw9Q8je4=;
        b=hDfXGvvO0SjvnQCvPJo1RgFRONOZOmJP+Np+LFKXZSh+iVG9ojI+TdM0INPQgkYRJc
         ztdxn/9hG3CpwH2kbkq2X2w2eWSMsc5BtUfTgZ7nJej+pf+tgB5w8igaJIzl2bKPLNUs
         cJ30JsSbk5ZJnppon1QkXMYWiA6HmOZxiCx6X5azTYXqXF7e/5Sr+gXtWvPC/aKX32oE
         eH8C9JznlUbOD13U2IHzr0yx4R03/0FOex4k+PKlvhkLuiQgQl2uKuXbYuyvEXICt0wU
         u31ELGKLdeqsoK7L1NzFUiwMLU4xzDDaIuWQqUggHrBcEP9EePSjXERfFzM6IU9LEc7i
         kjtg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708015099; x=1708619899;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pFKN2F8CAqIawWvV7bli4CjJ841AKOlmGdXyw9Q8je4=;
        b=srFSDWH/GogTBiRGeXA+JbSLviGDMH6C9GyYuLJnGCQv3J4NJNH12OaxO3vKh30Mpl
         g2Ehjd9+0U8Lvg+TUW6Qlv3BXKg4qnYr6uSvqC+5+yfzsrurmSOtyV6EZjzyMhpOSGTD
         0nPu8ZNWiPOZP7q0ZYc92RaCEvomF8Ncu22RaLbE3BjirF9kqfS8evzshLnE+cao2azz
         eiwfQph2vTFuxa2B6ikjErXFGCEhVaiL2G/btD4p4K7pZwxR2kmmKoZAATHoHKMAY5ac
         NBahy8KUxqGX/ZazNDqsRQIXBV1SdQo0/SMPTsaaZlbIMi1gvUkT7arZ0OCIKTMTnOMf
         vj0Q==
X-Forwarded-Encrypted: i=1; AJvYcCU3EFqLyy+UWObaakUZzoRK5V46wSRnL5zyRsuJWgF3cUFCt9y1qOcMRGjt1yT9M7/z1vLLXkcWGq1TI7g+huFAO/KvezN9iTukCAiyXAI=
X-Gm-Message-State: AOJu0YxYpGz3L0St/o8VMW2LxWVKmLmto3/NOKOIE02BdgyprtBan8JR
	dJ/18+Y7cY/8casrFLQIo72Z963x0VMbVvVtyeSVTD9hDTUjOYev
X-Google-Smtp-Source: AGHT+IFW6u92havSTdHGYAwHEhf4tPCyJ3yDp5vCqTsXvZ+d8d4YwscSHgF6I2ulIOefBWFfeUXw2w==
X-Received: by 2002:a2e:9053:0:b0:2cf:4d88:1772 with SMTP id n19-20020a2e9053000000b002cf4d881772mr1596166ljg.29.1708015098883;
        Thu, 15 Feb 2024 08:38:18 -0800 (PST)
Message-ID: <095b8031eaaa5324cdae9fee75f9521a795feb46.camel@gmail.com>
Subject: Re: [PATCH v4 25/30] xen/riscv: add minimal stuff to processor.h to
 build full Xen
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Alistair Francis
 <alistair.francis@wdc.com>, Bob Eshleman <bobbyeshleman@gmail.com>, Connor
 Davis <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
Date: Thu, 15 Feb 2024 17:38:17 +0100
In-Reply-To: <6be5102a-624c-463a-9821-c618d110ce7a@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <9efbc232f64b6192cf83f865b8987846fe082720.1707146506.git.oleksii.kurochko@gmail.com>
	 <6be5102a-624c-463a-9821-c618d110ce7a@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Tue, 2024-02-13 at 14:33 +0100, Jan Beulich wrote:
> On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > --- a/xen/arch/riscv/Kconfig
> > +++ b/xen/arch/riscv/Kconfig
> > @@ -45,6 +45,13 @@ config RISCV_ISA_C
> > =C2=A0
> > =C2=A0	=C2=A0 If unsure, say Y.
> > =C2=A0
> > +config TOOLCHAIN_HAS_ZIHINTPAUSE
> > +	bool
> > +	default y
>=20
> Shorter as "def_bool y".
>=20
> > +	depends on !64BIT || $(cc-option,-mabi=3Dlp64 -
> > march=3Drv64ima_zihintpause)
> > +	depends on !32BIT || $(cc-option,-mabi=3Dilp32 -
> > march=3Drv32ima_zihintpause)
>=20
> So for a reason I cannot really see -mabi=3D is indeed required here,
> or else the compiler sees an issue with the D extension. But enabling
> both M and A shouldn't really be needed in this check, as being
> unrelated?
Agree, that M and A could be dropped.

Regarding -mabi my guess is because D extension can be emulated by
compiler, doesn't matter if D is set in -march.  If it is set then
hardware instruction will be used, otherwise emulated instruction will
be used.
And if D extenstion is always present it is need to know which ABI
should be used. If D extenstion has h/w support then -mabi should be
also update to lp64d instead of lp64.

>=20
> > +	depends on LLD_VERSION >=3D 150000 || LD_VERSION >=3D 23600
>=20
> What's the linker dependency here? Depending on the answer I might
> further
> ask why "TOOLCHAIN" when elsewhere we use CC_HAS_ or HAS_CC_ or
> HAS_AS_.
I missed to introduce {L}LLD_VERSION config. It should output from the
command:
  riscv64-linux-gnu-ld --version
>=20
> That said, you may or may not be aware that personally I'm against
> encoding such in Kconfig, and my repeated attempts to get the
> respective
> discussion unstuck have not led anywhere. Therefore if you keep this,
> I'll
> be in trouble whether to actually ack the change as a whole.
Could I ask what is wrong with introduction of such things on KConfig?

Would it be better to put everything in riscv/arch.mk?

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 16:44:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 16:44:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681942.1060947 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeq5-0003th-DK; Thu, 15 Feb 2024 16:43:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681942.1060947; Thu, 15 Feb 2024 16:43:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raeq5-0003ta-9I; Thu, 15 Feb 2024 16:43:57 +0000
Received: by outflank-mailman (input) for mailman id 681942;
 Thu, 15 Feb 2024 16:43:55 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4dJ3=JY=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1raeq3-0003tU-LJ
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 16:43:55 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 67bfbdd2-cc21-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 17:43:54 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-411e5f21c0bso9451975e9.0
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 08:43:54 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 x13-20020a5d60cd000000b0033b17880eacsm2256110wrt.56.2024.02.15.08.43.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 15 Feb 2024 08:43:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 67bfbdd2-cc21-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708015434; x=1708620234; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=uGVNY4C0mPvDCpoPXmQ0H1utfMgnuxF+/tJticYBrzU=;
        b=HUIj95PGOB/PQK42cv9XU/UVxqePKGy9ady4/zhHoPcAcA4emAWjEyUM7vGkXXYfZZ
         /CErfrvBwY1gyrSu1xX3l6u+IOC+y+VweDVF47B/yPPnBu0LDbIBMBwCliQ/x86SZEIq
         1rEBMx5mCMaZdvoBrNqx3lHPmzxEf2yiTL8Jy+QHmhSBvR96lg2lp8Ht9CPgdzy8enSf
         fzZ4cDSY8YC/ofn1/mp7aCeHk8+WHc0DMoGSv0vuJVgSDElpdCJxLyIbJofXzTEDzERB
         QvdzyO1giJhEoWJqrI6Vyn0Jxq79JX4IIHdh1cq36O3yQYDEDOqFHnUgUK8FBmZ+jsa7
         6pzA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708015434; x=1708620234;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=uGVNY4C0mPvDCpoPXmQ0H1utfMgnuxF+/tJticYBrzU=;
        b=q+oHAzpub2RQbN/5Uig7XiVeHh/ZbKWCrOm9G/2y6eok8aLugEEt9AQKakYw13DqeP
         2sC8R9B+yYTcrlqGUYDJCxrrLxX4Sjhx4771dzRML8TIh4YP1FLqxRAuZglOqRb/a34q
         VZXNribLCKzRt3Hzjr7lgZj1NQCxdOhClYYc0UqP+WfYvgxTuVAeSbT1Ri0SUN1dv5UZ
         wwCbtlLLuQHzfGQGbCcvOhTjRSjIRHD9Oo/fXEMjhx77EKUdgabqtZyPEZn8hcTXBulZ
         jC/Buwj7S5pvmIPY/hvQKDoJHnFb4MG2f9MuqoyUijPFjEbgAxChC6sbd1kTmdoKpdUH
         NeQQ==
X-Forwarded-Encrypted: i=1; AJvYcCW6nhuDwBmOYPWZw1JqfkVN8VpC6WSV0QMRfpTWtCr4BUcH/ncRK7ZCMx/hG9cngnEvPXPAiK6OmLmEt7G2hC/bEDivj3UnJqJAVX23K+A=
X-Gm-Message-State: AOJu0YyQn2RCn+ar9vPHtZC4C7r8/ltI3iH8+lAipoByodFlue+LtJX8
	vpiAqVeDAOkANhE46H/M1PY6icEYMlfCLQorSizWQCcF7SL5HKqpeh2DVVC96g==
X-Google-Smtp-Source: AGHT+IF4OKeNbXHmGCWs2pfkxjkmE9Jl7BFlmsliB9Av6MefIMi/hVcCGhta0frWToCOFg7IfkZqfw==
X-Received: by 2002:a05:600c:314a:b0:412:2d3c:9f92 with SMTP id h10-20020a05600c314a00b004122d3c9f92mr715933wmo.13.1708015433792;
        Thu, 15 Feb 2024 08:43:53 -0800 (PST)
Message-ID: <2f1e4d2d-5b33-47ff-912b-c4693744d0e9@suse.com>
Date: Thu, 15 Feb 2024 17:43:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 25/30] xen/riscv: add minimal stuff to processor.h to
 build full Xen
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <9efbc232f64b6192cf83f865b8987846fe082720.1707146506.git.oleksii.kurochko@gmail.com>
 <6be5102a-624c-463a-9821-c618d110ce7a@suse.com>
 <095b8031eaaa5324cdae9fee75f9521a795feb46.camel@gmail.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <095b8031eaaa5324cdae9fee75f9521a795feb46.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 15.02.2024 17:38, Oleksii wrote:
> On Tue, 2024-02-13 at 14:33 +0100, Jan Beulich wrote:
>> On 05.02.2024 16:32, Oleksii Kurochko wrote:
>>> +	depends on LLD_VERSION >= 150000 || LD_VERSION >= 23600
>>
>> What's the linker dependency here? Depending on the answer I might
>> further
>> ask why "TOOLCHAIN" when elsewhere we use CC_HAS_ or HAS_CC_ or
>> HAS_AS_.
> I missed to introduce {L}LLD_VERSION config. It should output from the
> command:
>   riscv64-linux-gnu-ld --version

Doesn't answer my question though where the linker version matters
here.

>> That said, you may or may not be aware that personally I'm against
>> encoding such in Kconfig, and my repeated attempts to get the
>> respective
>> discussion unstuck have not led anywhere. Therefore if you keep this,
>> I'll
>> be in trouble whether to actually ack the change as a whole.
> Could I ask what is wrong with introduction of such things on KConfig?

Just one of several possible pointers:
https://lists.xen.org/archives/html/xen-devel/2022-09/msg01793.html

> Would it be better to put everything in riscv/arch.mk?

Or a mix of both, as per the proposal. Just to be clear, if I say "yes"
to your question, someone else may come along and tell you to turn
around again.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 16:47:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 16:47:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681949.1060956 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raetu-0004SF-RY; Thu, 15 Feb 2024 16:47:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681949.1060956; Thu, 15 Feb 2024 16:47:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raetu-0004S8-OW; Thu, 15 Feb 2024 16:47:54 +0000
Received: by outflank-mailman (input) for mailman id 681949;
 Thu, 15 Feb 2024 16:47:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wY2D=JY=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raett-0004Rj-3z
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 16:47:53 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f49632b7-cc21-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 17:47:50 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d180d6bd32so14188001fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 08:47:51 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 bg38-20020a05600c3ca600b0040f0219c371sm2638382wmb.19.2024.02.15.08.47.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 08:47:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f49632b7-cc21-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708015670; x=1708620470; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=xSzGKUbvTBsdZuAtSzwneLv26whzlKaeewJgB2jp/94=;
        b=hrHcT5XPoMvA3u5OKPUn4l3w3JbwCgr63zEDWE8Uw5l3kFvNkoNX+oDxefefWN7Q/c
         hZ/4L7wzhiAWmRo94DYaZ12mRqWuWU7nu7WasyPXVccyhTtv81oZdXrATKBJ4LHtC4ak
         7qMVIX7SSrUw1PwWMYM/yDJe/OZ6Qrk31L0ro=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708015670; x=1708620470;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=xSzGKUbvTBsdZuAtSzwneLv26whzlKaeewJgB2jp/94=;
        b=WRl70vYutmck7Y0mJIxB4iPTqa2sRGdwvx4N8P1fIv5+IsmnVTMDZBYpnPwcsBdBGw
         2H5YIq0qVLhg+GB+NxVEOcg2nCwugJZfliWbNdGmCT7v6edkukNMgpQOpm/CupiKPuEh
         m4n80NkJdKlTrSUP0DxwS2GxzlOnkWLShEEcKsuHxUFeDhgI4smUpFsf4AZr2pd4Nwgk
         fVyLgMfDndjgXV5CswVALFcTk6IwCZl5EQHXREWOZ9B/ZsP8+JMzwotZeD7zDOC9Czoh
         ZIhL1VKsWYtsgYQ+9G63IMPHkhSJpaQZAeQovUOFf400QD5tXdVhQOG1l0ZyCDp12OWI
         Jvqg==
X-Gm-Message-State: AOJu0Yy2nY1wwfKiUrVVGWdf/UY+J21awnEyZnhxiOL9kBGXaGkBosNK
	M1D4VIgmv6lgoxVnnvLpbX17tffrSHI0yLrVLDEQnP9Bn5xONU+xTjby0ugixpvxR3DwnXLF9ti
	Z
X-Google-Smtp-Source: AGHT+IE3hBMqk8WEqty7E9FUs9rDel5zITU7RK4j3j9/wvow2YcUTc4U6xj9B59iHnWJuA31EWt+Pw==
X-Received: by 2002:a05:651c:1a0c:b0:2d2:a9d:e315 with SMTP id by12-20020a05651c1a0c00b002d20a9de315mr1760900ljb.52.1708015669770;
        Thu, 15 Feb 2024 08:47:49 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3] x86/vmx: add support for virtualize SPEC_CTRL
Date: Thu, 15 Feb 2024 17:46:53 +0100
Message-ID: <20240215164653.27210-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The feature is defined in the tertiary exec control, and is available starting
from Sapphire Rapids and Alder Lake CPUs.

When enabled, two extra VMCS fields are used: SPEC_CTRL mask and shadow.  Bits
set in mask are not allowed to be toggled by the guest (either set or clear)
and the value in the shadow field is the value the guest expects to be in the
SPEC_CTRL register.

By using it the hypervisor can force the value of SPEC_CTRL bits behind the
guest back without having to trap all accesses to SPEC_CTRL, note that no bits
are forced into the guest as part of this patch.  It also allows getting rid of
SPEC_CTRL in the guest MSR load list, since the value in the shadow field will
be loaded by the hardware on vmentry.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v2:
 - Adjust comments.
 - Unconditionally set SPEC_CTRL_MASK in the VMCS if present.

Changes since v1:
 - Expand commit message and code comments.
 - Prefix the output of the VMCS dump with '0x'.
---
 xen/arch/x86/hvm/vmx/vmcs.c             | 10 +++++-
 xen/arch/x86/hvm/vmx/vmx.c              | 41 ++++++++++++++++++++-----
 xen/arch/x86/include/asm/hvm/vmx/vmcs.h |  5 +++
 xen/arch/x86/include/asm/msr.h          |  9 ++++--
 4 files changed, 55 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 9e016634ab5c..1d1e4a8fdd03 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -202,6 +202,7 @@ static void __init vmx_display_features(void)
     P(cpu_has_vmx_tsc_scaling, "TSC Scaling");
     P(cpu_has_vmx_bus_lock_detection, "Bus Lock Detection");
     P(cpu_has_vmx_notify_vm_exiting, "Notify VM Exit");
+    P(cpu_has_vmx_virt_spec_ctrl, "Virtualize SPEC_CTRL");
 #undef P
 
     if ( !printed )
@@ -365,7 +366,7 @@ static int vmx_init_vmcs_config(bool bsp)
 
     if ( _vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS )
     {
-        uint64_t opt = 0;
+        uint64_t opt = TERTIARY_EXEC_VIRT_SPEC_CTRL;
 
         _vmx_tertiary_exec_control = adjust_vmx_controls2(
             "Tertiary Exec Control", 0, opt,
@@ -1378,6 +1379,10 @@ static int construct_vmcs(struct vcpu *v)
         rc = vmx_add_msr(v, MSR_PRED_CMD, PRED_CMD_IBPB,
                          VMX_MSR_HOST);
 
+    /* Set any bits we don't allow toggling in the mask field. */
+    if ( cpu_has_vmx_virt_spec_ctrl )
+        __vmwrite(SPEC_CTRL_MASK, v->arch.msrs->spec_ctrl.raw);
+
  out:
     vmx_vmcs_exit(v);
 
@@ -2086,6 +2091,9 @@ void vmcs_dump_vcpu(struct vcpu *v)
     if ( v->arch.hvm.vmx.secondary_exec_control &
          SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY )
         printk("InterruptStatus = %04x\n", vmr16(GUEST_INTR_STATUS));
+    if ( cpu_has_vmx_virt_spec_ctrl )
+        printk("SPEC_CTRL mask = 0x%016lx  shadow = 0x%016lx\n",
+               vmr(SPEC_CTRL_MASK), vmr(SPEC_CTRL_SHADOW));
 
     printk("*** Host State ***\n");
     printk("RIP = 0x%016lx (%ps)  RSP = 0x%016lx\n",
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 48376cc32751..4ce248fe1276 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -823,18 +823,29 @@ static void cf_check vmx_cpuid_policy_changed(struct vcpu *v)
     {
         vmx_clear_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
 
-        rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
-        if ( rc )
-            goto out;
+        if ( !cpu_has_vmx_virt_spec_ctrl )
+        {
+            rc = vmx_add_guest_msr(v, MSR_SPEC_CTRL, 0);
+            if ( rc )
+                goto out;
+        }
     }
     else
     {
         vmx_set_msr_intercept(v, MSR_SPEC_CTRL, VMX_MSR_RW);
 
-        rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
-        if ( rc && rc != -ESRCH )
-            goto out;
-        rc = 0; /* Tolerate -ESRCH */
+        /*
+         * NB: there's no need to clear the virtualize SPEC_CTRL control, as
+         * the MSR intercept takes precedence.  The SPEC_CTRL shadow and mask
+         * VMCS fields don't take effect if the intercept is set.
+         */
+        if ( !cpu_has_vmx_virt_spec_ctrl )
+        {
+            rc = vmx_del_msr(v, MSR_SPEC_CTRL, VMX_MSR_GUEST);
+            if ( rc && rc != -ESRCH )
+                goto out;
+            rc = 0; /* Tolerate -ESRCH */
+        }
     }
 
     /* MSR_PRED_CMD is safe to pass through if the guest knows about it. */
@@ -2629,6 +2640,9 @@ static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
     switch ( reg )
     {
     case MSR_SPEC_CTRL:
+        if ( cpu_has_vmx_virt_spec_ctrl )
+            /* Requires remote VMCS loaded - fetched below. */
+            break;
         rc = vmx_read_guest_msr(v, reg, &val);
         if ( rc )
         {
@@ -2652,6 +2666,11 @@ static uint64_t cf_check vmx_get_reg(struct vcpu *v, unsigned int reg)
     vmx_vmcs_enter(v);
     switch ( reg )
     {
+    case MSR_SPEC_CTRL:
+        ASSERT(cpu_has_vmx_virt_spec_ctrl);
+        __vmread(SPEC_CTRL_SHADOW, &val);
+        break;
+
     case MSR_IA32_BNDCFGS:
         __vmread(GUEST_BNDCFGS, &val);
         break;
@@ -2678,6 +2697,9 @@ static void cf_check vmx_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
     switch ( reg )
     {
     case MSR_SPEC_CTRL:
+        if ( cpu_has_vmx_virt_spec_ctrl )
+            /* Requires remote VMCS loaded - set below. */
+            break;
         rc = vmx_write_guest_msr(v, reg, val);
         if ( rc )
         {
@@ -2698,6 +2720,11 @@ static void cf_check vmx_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
     vmx_vmcs_enter(v);
     switch ( reg )
     {
+    case MSR_SPEC_CTRL:
+        ASSERT(cpu_has_vmx_virt_spec_ctrl);
+        __vmwrite(SPEC_CTRL_SHADOW, val);
+        break;
+
     case MSR_IA32_BNDCFGS:
         __vmwrite(GUEST_BNDCFGS, val);
         break;
diff --git a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
index a7dd2eeffcad..58140af69153 100644
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -270,6 +270,9 @@ extern u32 vmx_secondary_exec_control;
 #define TERTIARY_EXEC_VIRT_SPEC_CTRL            BIT(7, UL)
 extern uint64_t vmx_tertiary_exec_control;
 
+#define cpu_has_vmx_virt_spec_ctrl \
+     (vmx_tertiary_exec_control & TERTIARY_EXEC_VIRT_SPEC_CTRL)
+
 #define VMX_EPT_EXEC_ONLY_SUPPORTED                         0x00000001
 #define VMX_EPT_WALK_LENGTH_4_SUPPORTED                     0x00000040
 #define VMX_EPT_MEMORY_TYPE_UC                              0x00000100
@@ -436,6 +439,8 @@ enum vmcs_field {
     XSS_EXIT_BITMAP                 = 0x0000202c,
     TSC_MULTIPLIER                  = 0x00002032,
     TERTIARY_VM_EXEC_CONTROL        = 0x00002034,
+    SPEC_CTRL_MASK                  = 0x0000204a,
+    SPEC_CTRL_SHADOW                = 0x0000204c,
     GUEST_PHYSICAL_ADDRESS          = 0x00002400,
     VMCS_LINK_POINTER               = 0x00002800,
     GUEST_IA32_DEBUGCTL             = 0x00002802,
diff --git a/xen/arch/x86/include/asm/msr.h b/xen/arch/x86/include/asm/msr.h
index 1d8ea9f26faa..eed7b36cd992 100644
--- a/xen/arch/x86/include/asm/msr.h
+++ b/xen/arch/x86/include/asm/msr.h
@@ -302,8 +302,13 @@ struct vcpu_msrs
      * For PV guests, this holds the guest kernel value.  It is accessed on
      * every entry/exit path.
      *
-     * For VT-x guests, the guest value is held in the MSR guest load/save
-     * list.
+     * For VT-x guests, the guest value is held in the MSR guest load/save list
+     * if there's no support for virtualized SPEC_CTRL. If virtualized
+     * SPEC_CTRL is enabled the value here signals which bits in SPEC_CTRL the
+     * guest is not able to modify.  Note that the value for those bits used in
+     * Xen context is also used in the guest context.  Setting a bit here
+     * doesn't force such bit to set in the guest context unless also set in
+     * Xen selection of SPEC_CTRL.
      *
      * For SVM, the guest value lives in the VMCB, and hardware saves/restores
      * the host value automatically.  However, guests run with the OR of the
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 16:54:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 16:54:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681955.1060967 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raf0D-0006Uy-Hm; Thu, 15 Feb 2024 16:54:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681955.1060967; Thu, 15 Feb 2024 16:54:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raf0D-0006Ur-DX; Thu, 15 Feb 2024 16:54:25 +0000
Received: by outflank-mailman (input) for mailman id 681955;
 Thu, 15 Feb 2024 16:54:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=G1Z8=JY=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raf0C-0006Ul-6o
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 16:54:24 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dd9d31a6-cc22-11ee-98f5-efadbce2ee36;
 Thu, 15 Feb 2024 17:54:21 +0100 (CET)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2d0d95e8133so13784941fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 08:54:22 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 r16-20020a2eb610000000b002d0a84eb8c3sm355843ljn.73.2024.02.15.08.54.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 08:54:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dd9d31a6-cc22-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708016061; x=1708620861; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=0AuLDYjI6N2/l1F0oTuaUxW/JzdP/wpTtUClGyEDJ0Y=;
        b=GyUbmooWLQSeg/OkLizL+FQATXdIV+xMGT5JDyZbxXumY+0JwlhKVUvdvfbgr7OLEu
         LP6flY55kXW7WZjVoqKTb3b+VRsgkrpCxjKqQ3oOuC5XVG9i1MqgPXqXawzd6wpUiT6i
         DVn7mEcV1UQTI/CMX8rgIHtxbx5j13Qb4l+0HAd442hcKb17C8otbC4+mQBVC5+HFHaX
         0QRr1T8HfjUiThg2AnDYK3/ZgCfS62kYWJTDtd37AI2XFuJ5qEDDSDjsl75mbTP1vMsY
         oHiGb3beuR5yQokc+CiQHT2bzqu2myh0YrcelQtjr+N8So1tn4OhlPT2/5N++UoO+HHY
         KWTg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708016061; x=1708620861;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0AuLDYjI6N2/l1F0oTuaUxW/JzdP/wpTtUClGyEDJ0Y=;
        b=INxy5m1cxdaWCJCA1FEXIaYRvYkg/Se5GT9usKzb2W2but4/VNR/XYwb19MTjoPjis
         wrY0VW+kWykylpcwVqjTCRYnEp1vV7Jv65G1Vc3YDzPlpXFtwhSgUQzIKqBQMzPuYBFE
         /Ddd391nYHREHns/fL0WC1erqEmwuJLx46NNQjZuK7uD7xfJs9b9c8qx9FVdRMmsXp8z
         Q/FwtkdxBKdnH0wKWSLaAH331k5RwTLjjHL8vU5XX2/hJ5l1v5tdA7mdxMvcMlpDPn0q
         Mc+hU8vNdR2LQSDtBW6SD9zRegrCmso/dNDLcj5aypKaselfLu4R5oTqstPGjNUOJI4k
         MWFA==
X-Forwarded-Encrypted: i=1; AJvYcCWQUABJAplLowMi1Xer59lAdf1B1mpvhqEqRUYcGb99f2wIWWSELL4e1FgyiNzxHbAABFSHtTUHXSsn7aI8zmONbh9/bs5GnT8mnMC4v8k=
X-Gm-Message-State: AOJu0YxiE1rlIwuEwLAoj2XVDHvk3Y7W72hTih/ZZpzDWVWZ1uDmX4pj
	tFN6NcV3yImtfFraGNOD4blmjOt6vY6V5lN5W4jFT60AUlGD0n91
X-Google-Smtp-Source: AGHT+IE65womX88jkCVy4Ex0k9LBizKCSsmKGRjUgJ+FsKuN8W4wCGUquGStJuIozslJvkf3VrOF6w==
X-Received: by 2002:a2e:9d97:0:b0:2d0:e1fc:9ca0 with SMTP id c23-20020a2e9d97000000b002d0e1fc9ca0mr1728400ljj.29.1708016061250;
        Thu, 15 Feb 2024 08:54:21 -0800 (PST)
Message-ID: <9fe0221a4c093fb5935ff3a519f2ef7180e569e1.camel@gmail.com>
Subject: Re: [PATCH v8 5/7] xen/asm-generic: introduce generic device.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Date: Thu, 15 Feb 2024 17:54:20 +0100
In-Reply-To: <3171ef24-bcb3-4387-ba32-f1c4d6c7220a@xen.org>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
	 <4ecb5e5f4a1dbf585ac7a9ebe40b5fd8e4709182.1707499278.git.oleksii.kurochko@gmail.com>
	 <319dcf7a-88cd-488e-ae51-6c20c19d8f22@xen.org>
	 <9d4f3093de6f95b09e44bc66fe9a9d4b0009dc43.camel@gmail.com>
	 <3171ef24-bcb3-4387-ba32-f1c4d6c7220a@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

>=20
> On 14/02/2024 09:32, Oleksii wrote:
> > On Tue, 2024-02-13 at 18:09 +0000, Julien Grall wrote:
> > > > +#ifdef CONFIG_HAS_PASSTHROUGH
> > > > +=C2=A0=C2=A0=C2=A0 struct iommu_fwspec *iommu_fwspec; /* per-devic=
e IOMMU
> > > > instance data */
> > > > +#endif
> > > > +};
> > > > +
> > > > +typedef struct device device_t;
> > > > +
> > > > +#ifdef CONFIG_HAS_DEVICE_TREE
> > > > +
> > > > +#include <xen/device_tree.h>
> > > > +
> > > > +#define dev_is_dt(dev)=C2=A0 ((dev)->type =3D=3D DEV_DT)
> > > > +
> > > > +/**
> > > > + *=C2=A0 device_init - Initialize a device
> > > > + *=C2=A0 @dev: device to initialize
> > > > + *=C2=A0 @class: class of the device (serial, network...)
> > > > + *=C2=A0 @data: specific data for initializing the device
> > > > + *
> > > > + *=C2=A0 Return 0 on success.
> > > > + */
> > > > +int device_init(struct dt_device_node *dev, enum device_class
> > > > class,
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 const void *data);
> > > > +
> > > > +/**
> > > > + * device_get_type - Get the type of the device
> > > > + * @dev: device to match
> > > > + *
> > > > + * Return the device type on success or DEVICE_ANY on failure
> > > > + */
> > > > +enum device_class device_get_class(const struct dt_device_node
> > > > *dev);
> > > > +
> > > > +#define DT_DEVICE_START(name_, namestr_, class_)=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > +static const struct device_desc __dev_desc_##name_ __used=C2=A0=C2=
=A0 \
> > > > +__section(".dev.info") =3D {=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 \
> > > > +=C2=A0=C2=A0=C2=A0 .name =3D namestr_,=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > +=C2=A0=C2=A0=C2=A0 .class =3D class_,
> > > > +
> > > > +#define DT_DEVICE_END=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> > > > +};
> > > > +
> > > > +#else /* !CONFIG_HAS_DEVICE_TREE */
> > > > +#define dev_is_dt(dev) ((void)(dev), false)
> > > > +#endif /* CONFIG_HAS_DEVICE_TREE */
> > > > +
> > > > +#define dev_is_pci(dev) ((dev)->type =3D=3D DEV_PCI)
> > > > +
> > > > +struct device_desc {
> > > > +=C2=A0=C2=A0=C2=A0 /* Device name */
> > > > +=C2=A0=C2=A0=C2=A0 const char *name;
> > > > +=C2=A0=C2=A0=C2=A0 /* Device class */
> > > > +=C2=A0=C2=A0=C2=A0 enum device_class class;
> > > > +
> > > > +#ifdef CONFIG_HAS_DEVICE_TREE
> > > > +
> > > > +=C2=A0=C2=A0=C2=A0 /* List of devices supported by this driver */
> > > > +=C2=A0=C2=A0=C2=A0 const struct dt_device_match *dt_match;
> > > > +=C2=A0=C2=A0=C2=A0 /*
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0 * Device initialization.
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0 *
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0 * -EAGAIN is used to indicate that device=
 probing is
> > > > deferred.
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0 */
> > > > +=C2=A0=C2=A0=C2=A0 int (*init)(struct dt_device_node *dev, const v=
oid *data);
> > > > +
> > > > +#endif
> > > > +};
> > > I am not sure I fully understand why "device_desc" is not
> > > protected
> > > by
> > > CONFIG_HAS_DEVICE_TREE. The structure doesn't mean much when the
> > > config
> > > is disabled. Can you clarify?
> > I thought that one day struct device_desc and acpi_device_desc will
> > be
> > "merged", and so decided just to #ifdef only DEVICE_TREE specific
> > fields.
>=20
> It might be possible to merge the two if we were using an union for
> the=20
> ACPI/DT specific part. However the majority of the parsing code needs
> to=20
> differ. So I am not convinced there would be any value to merge the
> two=20
> structures.
In this case, let's have two separate structures.

This is not the current situation, and I don't have a specific example.
It appears that all architectures will use Device Tree or ACPI.
However, does it make sense to keep 'struct device_desc' more generic
to accommodate non-DT or non-ACPI cases?

I am okay with making the following change, but I am just curious if
what I mentioned above makes sense at all:

#ifdef CONFIG_HAS_DEVICE_TREE
struct device_desc {
    /* Device name */
    const char *name;
    /* Device class */
    enum device_class class;

    /* List of devices supported by this driver */
    const struct dt_device_match *dt_match;
    /*
     * Device initialization.
     *
     * -EAGAIN is used to indicate that device probing is deferred.
     */
    int (*init)(struct dt_device_node *dev, const void *data);
};
#endif /* CONFIG_HAS_DEVICE_TREE */=20

>=20
> > Another one reason it is if to protect fully struct device_desc
> > then it
> > would be needed more #ifdef in arm/device.c ( for example,
> > device_init() should be all protected then ) what will require to
> > ifdef
> > all calls of device_init(). As an option device_init can can be
> > defined
> > in case when !CONFIG_HAS_DEVICE_TREE as:
> > =C2=A0=C2=A0=C2=A0 int __init device_init(struct dt_device_node *dev, e=
num
> > device_class
> > =C2=A0=C2=A0=C2=A0 class,
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 const void *data)
> > =C2=A0=C2=A0=C2=A0 {
> > =C2=A0=C2=A0=C2=A0=C2=A0 return -EBADF;
> > =C2=A0=C2=A0=C2=A0 }
> > =C2=A0=C2=A0=C2=A0=20
> > The similar thing will be needed for device_get_class() in Arm's
> > device.c.
>=20
> I agree that in theory device_init() & co should be protected with=20
> CONFIG_HAS_DEVICE_TREE. However, it is not possible to compile Xen on
> Arm without the Device-Tree part today. So I don't view adding the=20
> #ifdef or any extra stub as necessary today.
>=20
> This may be useful in the future though. Note this is not a request
> to=20
> modify the patch more than...
>=20
> >=20
> > Would it be better to ifdef full struct device_desc ?
> .. moving structure within the #ifdef.
Well, I'll update the commit message of the next patch that it is not
possible to compile Xen without CONFIG_HAS_DEVICE_TREE, so
device_init() and Co won't be protected by CONFIG_HAS_DEVICE_TREE.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 17:23:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 17:23:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681963.1060977 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rafSi-00035S-TC; Thu, 15 Feb 2024 17:23:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681963.1060977; Thu, 15 Feb 2024 17:23:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rafSi-00035L-PE; Thu, 15 Feb 2024 17:23:52 +0000
Received: by outflank-mailman (input) for mailman id 681963;
 Thu, 15 Feb 2024 17:23:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wY2D=JY=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rafSh-00035F-Up
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 17:23:51 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fc3b8494-cc26-11ee-8a4d-1f161083a0e0;
 Thu, 15 Feb 2024 18:23:50 +0100 (CET)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2d0a4e8444dso12049891fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 15 Feb 2024 09:23:50 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 q34-20020a05613002a200b007cb869908b0sm263811uac.6.2024.02.15.09.23.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 15 Feb 2024 09:23:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc3b8494-cc26-11ee-8a4d-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708017830; x=1708622630; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=acDTKRqmW5DTP4w8ajARr7nCCLgrlIsgyjdgastbp4A=;
        b=FOBIz7FCmIBRlIhCM//hnHsXo5C1GrVh1NvtkA6awFK5MVccAAply+otNPi/nWC4kT
         y416g9hLTQm204K+pcguqt0CGZpr8i4G5XpdY3oUE5jTbfRkZUyFanil9W8el/32Hf/c
         pL3se800VL6M0Kl+i2B1NzwysEonYnJZMFb1U=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708017830; x=1708622630;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=acDTKRqmW5DTP4w8ajARr7nCCLgrlIsgyjdgastbp4A=;
        b=aZTnMKe4irg/0wp1Mr+fiBDACsZYnYSNiED/YkzSLjf570HryEWuFYTr07UhIrJy0I
         zlvVhLajRQf+emlnfV6kNyfK5g5fRyt7S/Hj/o1rEfGSYoxSXKk5JGwP1vqLvo2Zx65Q
         1j8Wt30UZIqoDWJU1EFUMpu3205r5fb6xiV1LREDnr1ReIdWWjQIT3XC1kgpGFrhmMAz
         bGkaILUch3lhS/y2aI91VeMDcQPRDQZdE+nApm+48QCyh36uMtNRR/5ory5gPGFaa7U5
         SmTXR+HURsG7FPBEwBCGJPSBK2mo1/ak7Rpol2LWooX+xM+KhuMeW05EDbYtY4yi0Syv
         zDGg==
X-Forwarded-Encrypted: i=1; AJvYcCUjuJ0O20PV3OQ/ZAfUwnL113jBi/HLKbTkASs3vTPQsQkh85ld+Xh51wc7mW1dYwyV2Ag/q6BvQMOQRIy0AHrsu8SZYetGA+uw5kZoN2w=
X-Gm-Message-State: AOJu0YxdexqXtDosFD1MzTA+Kwg5NQ8eJ14elP07sAt1mGHnUGn7d+4+
	Ay2mx6pXwMdaYfLkIs2cTLsxkaC3kQcmpepTEh6hw5aSMK0sdz+rWS39Z0LT5oY=
X-Google-Smtp-Source: AGHT+IG6dsv8J0Xi3MirpEW3XwuANimpXQ7WhYEayvmheXVz6n32jzLQWjShh6L0pCUBHNMeEpPUVw==
X-Received: by 2002:a2e:9792:0:b0:2d0:f872:6f16 with SMTP id y18-20020a2e9792000000b002d0f8726f16mr1772152lji.16.1708017830347;
        Thu, 15 Feb 2024 09:23:50 -0800 (PST)
Date: Thu, 15 Feb 2024 18:23:47 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony Perard <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Message-ID: <Zc5Io3dkAlGSt3on@macbook>
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com>
 <Zc3nXpUOlnIHEfsl@macbook>
 <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com>
 <Zc3v20RKMssbaDsl@macbook>
 <a48cbad6-701d-4077-9044-4205b932a7f3@suse.com>
 <Zc428VMDoYnPw1zo@macbook>
 <d545cc6c-d213-43da-af31-1768af32aba0@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <d545cc6c-d213-43da-af31-1768af32aba0@suse.com>

On Thu, Feb 15, 2024 at 05:22:00PM +0100, Jan Beulich wrote:
> On 15.02.2024 17:08, Roger Pau Monné wrote:
> > On Thu, Feb 15, 2024 at 02:02:41PM +0100, Jan Beulich wrote:
> >> --- a/xen/Rules.mk
> >> +++ b/xen/Rules.mk
> >> @@ -15,7 +15,11 @@ srcdir := $(srctree)/$(src)
> >>  PHONY := __build
> >>  __build:
> >>  
> >> --include $(objtree)/include/config/auto.conf
> >> +ifneq ($(obj),tools)
> >> +ifneq ($(obj),tools/kconfig)
> >> +include $(objtree)/include/config/auto.conf
> >> +endif
> >> +endif
> > 
> > Trying to understand this, I assume it's to avoid an infinite
> > dependency loop that generating include/config/auto.conf requires some
> > tools that are build using xen/Rules.mk?
> 
> The file has dependencies only in xen/Makefile. This is about the
> file simply not being there when initially building. Perhaps the
> patch description helps that I've written in the meantime:
> 
> "Because of using "-include", failure to (re)build auto.conf (with
>  auto.conf.cmd produced as a secondary target) won't stop make from
>  continuing the build. Arrange for it being possible to drop the - from
>  Rules.mk, requiring that the include be skipped for tools-only targets.

Wouldn't it be more reliable if we skipped the include for any paths
in $(obj) that start with 'tools', rather than hardcoding 'tools' and
'tools/kconfig'?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 19:50:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 19:50:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681983.1061038 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rahkP-0004cU-MG; Thu, 15 Feb 2024 19:50:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681983.1061038; Thu, 15 Feb 2024 19:50:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rahkP-0004cN-JQ; Thu, 15 Feb 2024 19:50:17 +0000
Received: by outflank-mailman (input) for mailman id 681983;
 Thu, 15 Feb 2024 19:50:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NcNi=JY=kernel.org=pr-tracker-bot@srs-se1.protection.inumbo.net>)
 id 1rahkN-0004cE-PS
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 19:50:15 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6df3310a-cc3b-11ee-8a4e-1f161083a0e0;
 Thu, 15 Feb 2024 20:50:13 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 33C3FCE2800;
 Thu, 15 Feb 2024 19:50:08 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPS id 1EA3CC433A6;
 Thu, 15 Feb 2024 19:50:07 +0000 (UTC)
Received: from aws-us-west-2-korg-oddjob-1.ci.codeaurora.org
 (localhost.localdomain [127.0.0.1])
 by aws-us-west-2-korg-oddjob-1.ci.codeaurora.org (Postfix) with ESMTP id
 078B5DCB6EC; Thu, 15 Feb 2024 19:50:07 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6df3310a-cc3b-11ee-8a4e-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708026607;
	bh=dYG3ZJdb1V/Ktyl5Eml5x37N/Z+k7Tqs1pUPFJ1zkWw=;
	h=Subject:From:In-Reply-To:References:Date:To:Cc:From;
	b=AbynkWgYYGbqxtjxaFZBHd++Mc8Qb0ySMGOmFZ1SpPbX+JcESEe64VJHAU7z+vQQk
	 G8z9bWyxi1QvSZHQG3o2CSwcPlFNFYwArbN26V0aO1S0FeT+fxUicoWfht+HxskhuB
	 VYSFdBT2RIon3xGhAgXjsMawx80pwB5ieIqKVKt3Kd+XrTTi5tgjGwtj4d5zR92Ufv
	 DwGJiu+FXYGU5NCld+z1sePz8Yo5ERhH8lSUwIFKqhjL82HM5zHkylAUTP+yD/psCT
	 pj7l7YZBS906+EBTL4fht9LVAHUZM9WL79pz3f+ydOjlr9zAc/3TSoxQVter8Z7T0L
	 zaRXNu0uVWp5A==
Subject: Re: [GIT PULL] xen: branch for v6.8-rc5
From: pr-tracker-bot@kernel.org
In-Reply-To: <20240215152426.1114-1-jgross@suse.com>
References: <20240215152426.1114-1-jgross@suse.com>
X-PR-Tracked-List-Id: <linux-kernel.vger.kernel.org>
X-PR-Tracked-Message-Id: <20240215152426.1114-1-jgross@suse.com>
X-PR-Tracked-Remote: git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-6.8a-rc5-tag
X-PR-Tracked-Commit-Id: fa765c4b4aed2d64266b694520ecb025c862c5a9
X-PR-Merge-Tree: torvalds/linux.git
X-PR-Merge-Refname: refs/heads/master
X-PR-Merge-Commit-Id: cc9c4f0b3113d513a94bcf489f2fa8cb9cc7c679
Message-Id: <170802660702.17476.18029040592555930591.pr-tracker-bot@kernel.org>
Date: Thu, 15 Feb 2024 19:50:07 +0000
To: Juergen Gross <jgross@suse.com>
Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, sstabellini@kernel.org

The pull request you sent on Thu, 15 Feb 2024 16:24:26 +0100:

> git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git for-linus-6.8a-rc5-tag

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/cc9c4f0b3113d513a94bcf489f2fa8cb9cc7c679

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html


From xen-devel-bounces@lists.xenproject.org Thu Feb 15 20:31:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 20:31:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681989.1061049 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raiNm-0001JN-L2; Thu, 15 Feb 2024 20:30:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681989.1061049; Thu, 15 Feb 2024 20:30:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raiNm-0001JG-IO; Thu, 15 Feb 2024 20:30:58 +0000
Received: by outflank-mailman (input) for mailman id 681989;
 Thu, 15 Feb 2024 20:30:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RjNl=JY=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1raiNk-0001Is-PE
 for xen-devel@lists.xenproject.org; Thu, 15 Feb 2024 20:30:57 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2416::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1d01f6fe-cc41-11ee-8a4e-1f161083a0e0;
 Thu, 15 Feb 2024 21:30:54 +0100 (CET)
Received: from BL1PR13CA0310.namprd13.prod.outlook.com (2603:10b6:208:2c1::15)
 by DM4PR12MB6615.namprd12.prod.outlook.com (2603:10b6:8:8d::9) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7292.23; Thu, 15 Feb 2024 20:30:47 +0000
Received: from BL6PEPF0001AB54.namprd02.prod.outlook.com
 (2603:10b6:208:2c1:cafe::76) by BL1PR13CA0310.outlook.office365.com
 (2603:10b6:208:2c1::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.24 via Frontend
 Transport; Thu, 15 Feb 2024 20:30:47 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL6PEPF0001AB54.mail.protection.outlook.com (10.167.241.6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 15 Feb 2024 20:30:47 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 15 Feb
 2024 14:30:46 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 15 Feb
 2024 14:30:46 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 15 Feb 2024 14:30:44 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d01f6fe-cc41-11ee-8a4e-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LgAnVA6a8zax6KKjXud0daJJ88wBMKOSeTTERHsGnDSZAjzyUMP17XvmQyqS3h0S7E0lYBllMm/BP7ivt+aNngetj/r8qpd49ge2N2GbqeoZn5oxg95pXHvMXqCTKU+FxWUnazbAwEWJNn+kyEynCSuB8A9z+QilQycKva43r7jNRLpU4rFTgV4CBn+s5yYwJjD8rYw6vt4Nc1fazh2J/z+KfiKm7E7b9nw+sGvQzLPgAAz1nkVvLoINSe0/6cYhuaXxQA/yU5wAOAAcVZoj++3KaLXy2op2Rp3721pjuvXedFXEotl0JRpiTslJtbYROLw06Ag9Y2Ax6GiIK686OA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=fsSlVBQLqFxuhTnDvhvSFukqDI9hRqxTMLwpqrqjR8w=;
 b=crCO8d33JEBzXCRy/jN3Og2Ta6bzplObiCJVqRF5LOZ7aE3ODYTTmw102FflKe30JSWgTvX6qIz11fcnp76uyXlXI1hiQHDzk61PSbYmhwTlSsWoAcmNLsdsx+Ui1sWb0DbQ2qAZDJkOJgcg+wggTQqQjfucCbP6oEg0HeW9UK8rtylvXUm8yk/LGtmBpc7h+w47LXZe7UZGw1XAAbNxem/jxyH/7U0FRAntVDQsPvcvZvP/ecw3m7qfYL8WThWQZjMAmzcDgK8WY2VwwSpkxi6HdZydCUANuzJ+gNiLnwtnCusVoCX1a4gvByBmUs5gnY+apLfSQr0y955hS6HauA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fsSlVBQLqFxuhTnDvhvSFukqDI9hRqxTMLwpqrqjR8w=;
 b=ceLgbxsc8zL6yM9zAe10914OgyPXrLQJYVG9EohA600GLPiBI/+F9RYsf7cTZerFkmWXJiLgWt++VDe4c/0hl7E3xxpBsuJZuvvLOZb6ssfgoSgWe/sqNBahteNQhkzAU59a+r3nnL1tqqbBjje2eNW1SUTHUtbuzcVXyoKFR00=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Jan Beulich
	<jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Jun Nakajima
	<jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>, Paul Durrant
	<paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>
Subject: [PATCH v13.1 01/14] vpci: use per-domain PCI lock to protect vpci structure
Date: Thu, 15 Feb 2024 15:30:00 -0500
Message-ID: <20240215203001.1816811-1-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240202213321.1920347-2-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com> <20240202213321.1920347-2-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB54:EE_|DM4PR12MB6615:EE_
X-MS-Office365-Filtering-Correlation-Id: 589e08fb-b7bf-4824-3c99-08dc2e64fe1c
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YdvZCOxnPXmj21A2xBVPCF9Zb72gGUViSqfomsYzihyruFdX7XcnqNe1xixRXgWNzrMLbiwW+yQRago6l6ld8UnbufC+0z8M4EdFjoOcmoiXOEiGjw90ZVv1FiOAgnBOnp+QUkP1SZxHwDH6GyVE+zo8RLx31lIlWtjHrrDojnBufLKf4u/wg7swjX4Ug2ci1kMjoGptPiR046g8STojHBFn7UhIPu0XlgXOhshNIhEo8DsxVECEttoMIBv4ya2sRJ5OZy0T+k534zrShvChoXCyJRGsckAkVhTxNK6mXPd14mSiYVVOTsbhdrnAB5vjcUfIAIkdGCOxVrc1sSxAMqGLvHlRM5OghE5VZRnCZ7MsKwFivjkdshrLpBYDyaPTBGkcL4HEnJvufAhZn6hVEe/AYaPPgRA3qNsz5K1kuQD7UNb6xo4M4wub06onsky6OOZe+8Hl7VR9j1NkN9rIolrSszYq8flPdDMlVCIM5mFdEypJovVkccn6uUGsOgafT08RpBGjcRQNWd9c2FiF4FMYX7KAnD4Alz8h0++u4jwl52/oAsoadEVPWW17AXKBh2I5QT7VAZliAJYzURBQjiJ4/19Gf87BbdIZEbDsxHwuhacV8BQgIE5H8eg0vzJ4hSTPNsJqV8slSmIoSc7mUGJEu7rQWVKIioB5Hp515d8=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(39860400002)(396003)(346002)(136003)(230922051799003)(82310400011)(64100799003)(451199024)(186009)(36860700004)(1800799012)(40470700004)(46966006)(41300700001)(26005)(336012)(30864003)(5660300002)(70586007)(70206006)(426003)(7416002)(2616005)(1076003)(6666004)(44832011)(2906002)(81166007)(36756003)(4326008)(54906003)(8676002)(478600001)(8936002)(356005)(86362001)(316002)(6916009)(83380400001)(82740400003);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 20:30:47.3857
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 589e08fb-b7bf-4824-3c99-08dc2e64fe1c
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB54.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6615

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Use the per-domain PCI read/write lock to protect the presence of the
pci device vpci field. This lock can be used (and in a few cases is used
right away) so that vpci removal can be performed while holding the lock
in write mode. Previously such removal could race with vpci_read for
example.

When taking both d->pci_lock and pdev->vpci->lock, they should be
taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
possible deadlock situations.

1. Per-domain's pci_lock is used to protect pdev->vpci structure
from being removed.

2. Writing the command register and ROM BAR register may trigger
modify_bars to run, which in turn may access multiple pdevs while
checking for the existing BAR's overlap. The overlapping check, if
done under the read lock, requires vpci->lock to be acquired on both
devices being compared, which may produce a deadlock. It is not
possible to upgrade read lock to write lock in such a case. So, in
order to prevent the deadlock, use d->pci_lock in write mode instead.

All other code, which doesn't lead to pdev->vpci destruction and does
not access multiple pdevs at the same time, can still use a
combination of the read lock and pdev->vpci->lock.

3. Drop const qualifier where the new rwlock is used and this is
appropriate.

4. Do not call process_pending_softirqs with any locks held. For that
unlock prior the call and re-acquire the locks after. After
re-acquiring the lock there is no need to check if pdev->vpci exists:
 - in apply_map because of the context it is called (no race condition
   possible)
 - for MSI/MSI-X debug code because it is called at the end of
   pdev->vpci access and no further access to pdev->vpci is made

5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
while accessing pdevs in vpci code.

6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
do not go away. The vPCI functions call several MSI-related functions
which already have existing non-vPCI callers. Change those MSI-related
functions to allow using either pcidevs_lock() or d->pci_lock for
ensuring pdevs do not go away. Holding d->pci_lock in read mode is
sufficient. Note that this pdev protection mechanism does not protect
other state or critical sections. These MSI-related functions already
have other race condition and state protection mechanims (e.g.
d->event_lock and msixtbl RCU), so we deduce that the use of the global
pcidevs_lock() is to ensure that pdevs do not go away.

7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
that pdevs do not go away. The purpose of this wrapper is to aid
readability and document the intent of the pdev protection mechanism.

8. When possible, the existing non-vPCI callers of these MSI-related
functions haven't been switched to use the newly introduced per-domain
pci_lock, and will continue to use the global pcidevs_lock(). This is
done to reduce the risk of the new locking scheme introducing
regressions. Those users will be adjusted in due time. One exception
is where the pcidevs_lock() in allocate_and_map_msi_pirq() is moved to
the caller, physdev_map_pirq(): this instance is switched to
read_lock(&d->pci_lock) right away.

Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
Changes in v13.1:
* move pdev_list_is_read_locked() to pci.h
* use pdev_list_is_read_locked() in more places
* use d directly in pdev_list_is_read_locked()
* wrap pdev_list_is_read_locked() helper in #ifndef NDEBUG, and add
  declaration in the #else case with no implementation
* replace pcidevs_lock() with read_lock(&d->pci_lock) in physdev.c

Changes in v13:
 - hold off adding Roger's R-b tag even though it was provided on v12.2
 - use a wrapper construct to ease readability of odd-looking ASSERTs
 - new placement of ASSERT in __pci_enable_msix(), __pci_enable_msi(),
   and pci_enable_msi(). Rearrange/add pdev NULL check.
 - expand commit description with details about using either
   pcidevs_lock() or d->pci_lock

Changes in v12.2:
 - drop Roger's R-b
 - drop both locks on error paths in vpci_msix_arch_print()
 - add another ASSERT in vpci_msix_arch_print(), to enforce the
   expectation both locks are held before calling vpci_msix_arch_print()
 - move pdev_done label in vpci_dump_msi()
 - update comments in vpci_dump_msi() to say locks (plural)

Changes in v12.1:
 - use read_trylock() in vpci_msix_arch_print()
 - fixup in-code comments (revert double space, use DomXEN) in
   vpci_{read,write}()
 - minor updates in commit message
 - add Roger's R-b

Changes in v12:
 - s/pci_rwlock/pci_lock/ in commit message
 - expand comment about scope of pci_lock in sched.h
 - in vpci_{read,write}, if hwdom is trying to access a device assigned
   to dom_xen, holding hwdom->pci_lock is sufficient (no need to hold
   dom_xen->pci_lock)
 - reintroduce ASSERT in vmx_pi_update_irte()
 - reintroduce ASSERT in __pci_enable_msi{x}()
 - delete note 6. in commit message about removing ASSERTs since we have
   reintroduced them

Changes in v11:
 - Fixed commit message regarding possible spinlocks
 - Removed parameter from allocate_and_map_msi_pirq(), which was added
 in the prev version. Now we are taking pcidevs_lock in
 physdev_map_pirq()
 - Returned ASSERT to pci_enable_msi
 - Fixed case when we took read lock instead of write one
 - Fixed label indentation

Changes in v10:
 - Moved printk pas locked area
 - Returned back ASSERTs
 - Added new parameter to allocate_and_map_msi_pirq() so it knows if
 it should take the global pci lock
 - Added comment about possible improvement in vpci_write
 - Changed ASSERT(rw_is_locked()) to rw_is_write_locked() in
   appropriate places
 - Renamed release_domain_locks() to release_domain_write_locks()
 - moved domain_done label in vpci_dump_msi() to correct place
Changes in v9:
 - extended locked region to protect vpci_remove_device and
   vpci_add_handlers() calls
 - vpci_write() takes lock in the write mode to protect
   potential call to modify_bars()
 - renamed lock releasing function
 - removed ASSERT()s from msi code
 - added trylock in vpci_dump_msi

Changes in v8:
 - changed d->vpci_lock to d->pci_lock
 - introducing d->pci_lock in a separate patch
 - extended locked region in vpci_process_pending
 - removed pcidevs_lockis vpci_dump_msi()
 - removed some changes as they are not needed with
   the new locking scheme
 - added handling for hwdom && dom_xen case
---
 xen/arch/x86/hvm/vmsi.c       | 36 ++++++++++++++++--------
 xen/arch/x86/hvm/vmx/vmx.c    |  2 +-
 xen/arch/x86/irq.c            |  8 +++---
 xen/arch/x86/msi.c            | 20 +++++++++-----
 xen/arch/x86/physdev.c        |  2 ++
 xen/drivers/passthrough/pci.c |  9 +++---
 xen/drivers/vpci/header.c     | 18 ++++++++++++
 xen/drivers/vpci/msi.c        | 30 +++++++++++++++++---
 xen/drivers/vpci/msix.c       | 52 ++++++++++++++++++++++++++++++-----
 xen/drivers/vpci/vpci.c       | 24 ++++++++++++++--
 xen/include/xen/pci.h         | 14 ++++++++++
 xen/include/xen/sched.h       |  3 +-
 12 files changed, 177 insertions(+), 41 deletions(-)

diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index 128f23636279..19c334fb9c29 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -468,7 +468,7 @@ int msixtbl_pt_register(struct domain *d, struct pirq *pirq, uint64_t gtable)
     struct msixtbl_entry *entry, *new_entry;
     int r = -EINVAL;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(d));
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     if ( !msixtbl_initialised(d) )
@@ -538,7 +538,7 @@ void msixtbl_pt_unregister(struct domain *d, struct pirq *pirq)
     struct pci_dev *pdev;
     struct msixtbl_entry *entry;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(d));
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     if ( !msixtbl_initialised(d) )
@@ -684,7 +684,7 @@ static int vpci_msi_update(const struct pci_dev *pdev, uint32_t data,
 {
     unsigned int i;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
 
     if ( (address & MSI_ADDR_BASE_MASK) != MSI_ADDR_HEADER )
     {
@@ -725,8 +725,8 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev)
     int rc;
 
     ASSERT(msi->arch.pirq != INVALID_PIRQ);
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
 
-    pcidevs_lock();
     for ( i = 0; i < msi->vectors && msi->arch.bound; i++ )
     {
         struct xen_domctl_bind_pt_irq unbind = {
@@ -745,7 +745,6 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev)
 
     msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address,
                                        msi->vectors, msi->arch.pirq, msi->mask);
-    pcidevs_unlock();
 }
 
 static int vpci_msi_enable(const struct pci_dev *pdev, unsigned int nr,
@@ -778,15 +777,14 @@ int vpci_msi_arch_enable(struct vpci_msi *msi, const struct pci_dev *pdev,
     int rc;
 
     ASSERT(msi->arch.pirq == INVALID_PIRQ);
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
     rc = vpci_msi_enable(pdev, vectors, 0);
     if ( rc < 0 )
         return rc;
     msi->arch.pirq = rc;
 
-    pcidevs_lock();
     msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address, vectors,
                                        msi->arch.pirq, msi->mask);
-    pcidevs_unlock();
 
     return 0;
 }
@@ -797,8 +795,8 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq,
     unsigned int i;
 
     ASSERT(pirq != INVALID_PIRQ);
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
 
-    pcidevs_lock();
     for ( i = 0; i < nr && bound; i++ )
     {
         struct xen_domctl_bind_pt_irq bind = {
@@ -814,7 +812,6 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq,
     write_lock(&pdev->domain->event_lock);
     unmap_domain_pirq(pdev->domain, pirq);
     write_unlock(&pdev->domain->event_lock);
-    pcidevs_unlock();
 }
 
 void vpci_msi_arch_disable(struct vpci_msi *msi, const struct pci_dev *pdev)
@@ -854,6 +851,7 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
     int rc;
 
     ASSERT(entry->arch.pirq == INVALID_PIRQ);
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
     rc = vpci_msi_enable(pdev, vmsix_entry_nr(pdev->vpci->msix, entry),
                          table_base);
     if ( rc < 0 )
@@ -861,7 +859,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
 
     entry->arch.pirq = rc;
 
-    pcidevs_lock();
     rc = vpci_msi_update(pdev, entry->data, entry->addr, 1, entry->arch.pirq,
                          entry->masked);
     if ( rc )
@@ -869,7 +866,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
         vpci_msi_disable(pdev, entry->arch.pirq, 1, false);
         entry->arch.pirq = INVALID_PIRQ;
     }
-    pcidevs_unlock();
 
     return rc;
 }
@@ -895,6 +891,14 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
 {
     unsigned int i;
 
+    /*
+     * Assert that d->pdev_list doesn't change. pdev_list_is_read_locked() is
+     * not suitable here because we may read_unlock(&pdev->domain->pci_lock)
+     * before returning.
+     */
+    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
+    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));
+
     for ( i = 0; i < msix->max_entries; i++ )
     {
         const struct vpci_msix_entry *entry = &msix->entries[i];
@@ -913,13 +917,23 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
             struct pci_dev *pdev = msix->pdev;
 
             spin_unlock(&msix->pdev->vpci->lock);
+            read_unlock(&pdev->domain->pci_lock);
             process_pending_softirqs();
+
+            if ( !read_trylock(&pdev->domain->pci_lock) )
+                return -EBUSY;
+
             /* NB: we assume that pdev cannot go away for an alive domain. */
             if ( !pdev->vpci || !spin_trylock(&pdev->vpci->lock) )
+            {
+                read_unlock(&pdev->domain->pci_lock);
                 return -EBUSY;
+            }
+
             if ( pdev->vpci->msix != msix )
             {
                 spin_unlock(&pdev->vpci->lock);
+                read_unlock(&pdev->domain->pci_lock);
                 return -EAGAIN;
             }
         }
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 48376cc32751..dd35091516b0 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -413,7 +413,7 @@ static int cf_check vmx_pi_update_irte(const struct vcpu *v,
 
     spin_unlock_irq(&desc->lock);
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(msi_desc->dev->domain));
 
     return iommu_update_ire_from_msi(msi_desc, &msi_desc->msg);
 
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index bbae7751e494..e31144d82fd4 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2164,7 +2164,7 @@ int map_domain_pirq(
         struct pci_dev *pdev;
         unsigned int nr = 0;
 
-        ASSERT(pcidevs_locked());
+        ASSERT(pdev_list_is_read_locked(d));
 
         ret = -ENODEV;
         if ( !cpu_has_apic )
@@ -2321,7 +2321,7 @@ int unmap_domain_pirq(struct domain *d, int pirq)
     if ( (pirq < 0) || (pirq >= d->nr_pirqs) )
         return -EINVAL;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(d));
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     info = pirq_info(d, pirq);
@@ -2886,6 +2886,8 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 {
     int irq, pirq, ret;
 
+    ASSERT(pdev_list_is_read_locked(d));
+
     switch ( type )
     {
     case MAP_PIRQ_TYPE_MSI:
@@ -2915,7 +2917,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 
     msi->irq = irq;
 
-    pcidevs_lock();
     /* Verify or get pirq. */
     write_lock(&d->event_lock);
     pirq = allocate_pirq(d, index, *pirq_p, irq, type, &msi->entry_nr);
@@ -2931,7 +2932,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 
  done:
     write_unlock(&d->event_lock);
-    pcidevs_unlock();
     if ( ret )
     {
         switch ( type )
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index 335c0868a225..e008b6789a28 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -602,7 +602,7 @@ static int msi_capability_init(struct pci_dev *dev,
     unsigned int i, mpos;
     uint16_t control;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(dev->domain));
     pos = pci_find_cap_offset(dev->sbdf, PCI_CAP_ID_MSI);
     if ( !pos )
         return -ENODEV;
@@ -771,7 +771,7 @@ static int msix_capability_init(struct pci_dev *dev,
     if ( !pos )
         return -ENODEV;
 
-    ASSERT(pcidevs_locked());
+    ASSERT(pdev_list_is_read_locked(dev->domain));
 
     control = pci_conf_read16(dev->sbdf, msix_control_reg(pos));
     /*
@@ -988,11 +988,11 @@ static int __pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi,
 {
     struct msi_desc *old_desc;
 
-    ASSERT(pcidevs_locked());
-
     if ( !pdev )
         return -ENODEV;
 
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
+
     old_desc = find_msi_entry(pdev, msi->irq, PCI_CAP_ID_MSI);
     if ( old_desc )
     {
@@ -1043,9 +1043,12 @@ static int __pci_enable_msix(struct pci_dev *pdev, struct msi_info *msi,
 {
     struct msi_desc *old_desc;
 
-    ASSERT(pcidevs_locked());
+    if ( !pdev )
+        return -ENODEV;
+
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
 
-    if ( !pdev || !pdev->msix )
+    if ( !pdev->msix )
         return -ENODEV;
 
     if ( msi->entry_nr >= pdev->msix->nr_entries )
@@ -1154,7 +1157,10 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off)
 int pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi,
                    struct msi_desc **desc)
 {
-    ASSERT(pcidevs_locked());
+    if ( !pdev )
+        return -ENODEV;
+
+    ASSERT(pdev_list_is_read_locked(pdev->domain));
 
     if ( !use_msi )
         return -EPERM;
diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
index 47c4da0af7e1..7efa17cf4c1e 100644
--- a/xen/arch/x86/physdev.c
+++ b/xen/arch/x86/physdev.c
@@ -123,7 +123,9 @@ int physdev_map_pirq(domid_t domid, int type, int *index, int *pirq_p,
 
     case MAP_PIRQ_TYPE_MSI:
     case MAP_PIRQ_TYPE_MULTI_MSI:
+        read_lock(&d->pci_lock);
         ret = allocate_and_map_msi_pirq(d, *index, pirq_p, type, msi);
+        read_unlock(&d->pci_lock);
         break;
 
     default:
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 47c0eee7bdcc..c97dd4504a7a 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -750,7 +750,6 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
         pdev->domain = hardware_domain;
         write_lock(&hardware_domain->pci_lock);
         list_add(&pdev->domain_list, &hardware_domain->pdev_list);
-        write_unlock(&hardware_domain->pci_lock);
 
         /*
          * For devices not discovered by Xen during boot, add vPCI handlers
@@ -759,18 +758,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
         ret = vpci_add_handlers(pdev);
         if ( ret )
         {
-            printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret);
-            write_lock(&hardware_domain->pci_lock);
             list_del(&pdev->domain_list);
             write_unlock(&hardware_domain->pci_lock);
             pdev->domain = NULL;
+            printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret);
             goto out;
         }
+        write_unlock(&hardware_domain->pci_lock);
         ret = iommu_add_device(pdev);
         if ( ret )
         {
-            vpci_remove_device(pdev);
             write_lock(&hardware_domain->pci_lock);
+            vpci_remove_device(pdev);
             list_del(&pdev->domain_list);
             write_unlock(&hardware_domain->pci_lock);
             pdev->domain = NULL;
@@ -1146,7 +1145,9 @@ static void __hwdom_init setup_one_hwdom_device(const struct setup_hwdom *ctxt,
     } while ( devfn != pdev->devfn &&
               PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn) );
 
+    write_lock(&ctxt->d->pci_lock);
     err = vpci_add_handlers(pdev);
+    write_unlock(&ctxt->d->pci_lock);
     if ( err )
         printk(XENLOG_ERR "setup of vPCI for d%d failed: %d\n",
                ctxt->d->domain_id, err);
diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 58195549d50a..8f5850b8cf6d 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -173,6 +173,7 @@ bool vpci_process_pending(struct vcpu *v)
         if ( rc == -ERESTART )
             return true;
 
+        write_lock(&v->domain->pci_lock);
         spin_lock(&v->vpci.pdev->vpci->lock);
         /* Disable memory decoding unconditionally on failure. */
         modify_decoding(v->vpci.pdev,
@@ -191,6 +192,7 @@ bool vpci_process_pending(struct vcpu *v)
              * failure.
              */
             vpci_remove_device(v->vpci.pdev);
+        write_unlock(&v->domain->pci_lock);
     }
 
     return false;
@@ -202,8 +204,20 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
     struct map_data data = { .d = d, .map = true };
     int rc;
 
+    ASSERT(rw_is_write_locked(&d->pci_lock));
+
     while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART )
+    {
+        /*
+         * It's safe to drop and reacquire the lock in this context
+         * without risking pdev disappearing because devices cannot be
+         * removed until the initial domain has been started.
+         */
+        write_unlock(&d->pci_lock);
         process_pending_softirqs();
+        write_lock(&d->pci_lock);
+    }
+
     rangeset_destroy(mem);
     if ( !rc )
         modify_decoding(pdev, cmd, false);
@@ -244,6 +258,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
     unsigned int i;
     int rc;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !mem )
         return -ENOMEM;
 
@@ -524,6 +540,8 @@ static int cf_check init_header(struct pci_dev *pdev)
     int rc;
     bool mask_cap_list = false;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     switch ( pci_conf_read8(pdev->sbdf, PCI_HEADER_TYPE) & 0x7f )
     {
     case PCI_HEADER_TYPE_NORMAL:
diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c
index a253ccbd7db7..dc71938e23f5 100644
--- a/xen/drivers/vpci/msi.c
+++ b/xen/drivers/vpci/msi.c
@@ -263,7 +263,7 @@ REGISTER_VPCI_INIT(init_msi, VPCI_PRIORITY_LOW);
 
 void vpci_dump_msi(void)
 {
-    const struct domain *d;
+    struct domain *d;
 
     rcu_read_lock(&domlist_read_lock);
     for_each_domain ( d )
@@ -275,6 +275,9 @@ void vpci_dump_msi(void)
 
         printk("vPCI MSI/MSI-X d%d\n", d->domain_id);
 
+        if ( !read_trylock(&d->pci_lock) )
+            continue;
+
         for_each_pdev ( d, pdev )
         {
             const struct vpci_msi *msi;
@@ -313,17 +316,36 @@ void vpci_dump_msi(void)
                 {
                     /*
                      * On error vpci_msix_arch_print will always return without
-                     * holding the lock.
+                     * holding the locks.
                      */
                     printk("unable to print all MSI-X entries: %d\n", rc);
-                    process_pending_softirqs();
-                    continue;
+                    goto pdev_done;
                 }
             }
 
+            /*
+             * Unlock locks to process pending softirqs. This is
+             * potentially unsafe, as d->pdev_list can be changed in
+             * meantime.
+             */
             spin_unlock(&pdev->vpci->lock);
+            read_unlock(&d->pci_lock);
+        pdev_done:
             process_pending_softirqs();
+            if ( !read_trylock(&d->pci_lock) )
+            {
+                printk("unable to access other devices for the domain\n");
+                goto domain_done;
+            }
         }
+        read_unlock(&d->pci_lock);
+    domain_done:
+        /*
+         * We need this label at the end of the loop, but some
+         * compilers might not be happy about label at the end of the
+         * compound statement so we adding an empty statement here.
+         */
+        ;
     }
     rcu_read_unlock(&domlist_read_lock);
 }
diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
index d1126a417da9..06371c347b10 100644
--- a/xen/drivers/vpci/msix.c
+++ b/xen/drivers/vpci/msix.c
@@ -147,6 +147,8 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr)
 {
     struct vpci_msix *msix;
 
+    ASSERT(pdev_list_is_read_locked(d));
+
     list_for_each_entry ( msix, &d->arch.hvm.msix_tables, next )
     {
         const struct vpci_bar *bars = msix->pdev->vpci->header.bars;
@@ -163,7 +165,13 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr)
 
 static int cf_check msix_accept(struct vcpu *v, unsigned long addr)
 {
-    return !!msix_find(v->domain, addr);
+    int rc;
+
+    read_lock(&v->domain->pci_lock);
+    rc = !!msix_find(v->domain, addr);
+    read_unlock(&v->domain->pci_lock);
+
+    return rc;
 }
 
 static bool access_allowed(const struct pci_dev *pdev, unsigned long addr,
@@ -358,21 +366,35 @@ static int adjacent_read(const struct domain *d, const struct vpci_msix *msix,
 static int cf_check msix_read(
     struct vcpu *v, unsigned long addr, unsigned int len, unsigned long *data)
 {
-    const struct domain *d = v->domain;
-    struct vpci_msix *msix = msix_find(d, addr);
+    struct domain *d = v->domain;
+    struct vpci_msix *msix;
     const struct vpci_msix_entry *entry;
     unsigned int offset;
 
     *data = ~0UL;
 
+    read_lock(&d->pci_lock);
+
+    msix = msix_find(d, addr);
     if ( !msix )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_RETRY;
+    }
 
     if ( adjacent_handle(msix, addr) )
-        return adjacent_read(d, msix, addr, len, data);
+    {
+        int rc = adjacent_read(d, msix, addr, len, data);
+
+        read_unlock(&d->pci_lock);
+        return rc;
+    }
 
     if ( !access_allowed(msix->pdev, addr, len) )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_OKAY;
+    }
 
     spin_lock(&msix->pdev->vpci->lock);
     entry = get_entry(msix, addr);
@@ -404,6 +426,7 @@ static int cf_check msix_read(
         break;
     }
     spin_unlock(&msix->pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     return X86EMUL_OKAY;
 }
@@ -491,19 +514,33 @@ static int adjacent_write(const struct domain *d, const struct vpci_msix *msix,
 static int cf_check msix_write(
     struct vcpu *v, unsigned long addr, unsigned int len, unsigned long data)
 {
-    const struct domain *d = v->domain;
-    struct vpci_msix *msix = msix_find(d, addr);
+    struct domain *d = v->domain;
+    struct vpci_msix *msix;
     struct vpci_msix_entry *entry;
     unsigned int offset;
 
+    read_lock(&d->pci_lock);
+
+    msix = msix_find(d, addr);
     if ( !msix )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_RETRY;
+    }
 
     if ( adjacent_handle(msix, addr) )
-        return adjacent_write(d, msix, addr, len, data);
+    {
+        int rc = adjacent_write(d, msix, addr, len, data);
+
+        read_unlock(&d->pci_lock);
+        return rc;
+    }
 
     if ( !access_allowed(msix->pdev, addr, len) )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_OKAY;
+    }
 
     spin_lock(&msix->pdev->vpci->lock);
     entry = get_entry(msix, addr);
@@ -579,6 +616,7 @@ static int cf_check msix_write(
         break;
     }
     spin_unlock(&msix->pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     return X86EMUL_OKAY;
 }
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 72ef277c4f8e..475272b173f3 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -42,6 +42,8 @@ extern vpci_register_init_t *const __end_vpci_array[];
 
 void vpci_remove_device(struct pci_dev *pdev)
 {
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !has_vpci(pdev->domain) || !pdev->vpci )
         return;
 
@@ -77,6 +79,8 @@ int vpci_add_handlers(struct pci_dev *pdev)
     const unsigned long *ro_map;
     int rc = 0;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !has_vpci(pdev->domain) )
         return 0;
 
@@ -361,7 +365,7 @@ static uint32_t merge_result(uint32_t data, uint32_t new, unsigned int size,
 
 uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
 {
-    const struct domain *d = current->domain;
+    struct domain *d = current->domain;
     const struct pci_dev *pdev;
     const struct vpci_register *r;
     unsigned int data_offset = 0;
@@ -376,12 +380,18 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
     /*
      * Find the PCI dev matching the address, which for hwdom also requires
      * consulting DomXEN.  Passthrough everything that's not trapped.
+     * If this is hwdom and the device is assigned to DomXEN, acquiring hwdom's
+     * pci_lock is sufficient.
      */
+    read_lock(&d->pci_lock);
     pdev = pci_get_pdev(d, sbdf);
     if ( !pdev && is_hardware_domain(d) )
         pdev = pci_get_pdev(dom_xen, sbdf);
     if ( !pdev || !pdev->vpci )
+    {
+        read_unlock(&d->pci_lock);
         return vpci_read_hw(sbdf, reg, size);
+    }
 
     spin_lock(&pdev->vpci->lock);
 
@@ -428,6 +438,7 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
         ASSERT(data_offset < size);
     }
     spin_unlock(&pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     if ( data_offset < size )
     {
@@ -470,7 +481,7 @@ static void vpci_write_helper(const struct pci_dev *pdev,
 void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
                 uint32_t data)
 {
-    const struct domain *d = current->domain;
+    struct domain *d = current->domain;
     const struct pci_dev *pdev;
     const struct vpci_register *r;
     unsigned int data_offset = 0;
@@ -484,7 +495,13 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
     /*
      * Find the PCI dev matching the address, which for hwdom also requires
      * consulting DomXEN.  Passthrough everything that's not trapped.
+     * If this is hwdom and the device is assigned to DomXEN, acquiring hwdom's
+     * pci_lock is sufficient.
+     *
+     * TODO: We need to take pci_locks in exclusive mode only if we
+     * are modifying BARs, so there is a room for improvement.
      */
+    write_lock(&d->pci_lock);
     pdev = pci_get_pdev(d, sbdf);
     if ( !pdev && is_hardware_domain(d) )
         pdev = pci_get_pdev(dom_xen, sbdf);
@@ -493,6 +510,8 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
         /* Ignore writes to read-only devices, which have no ->vpci. */
         const unsigned long *ro_map = pci_get_ro_map(sbdf.seg);
 
+        write_unlock(&d->pci_lock);
+
         if ( !ro_map || !test_bit(sbdf.bdf, ro_map) )
             vpci_write_hw(sbdf, reg, size, data);
         return;
@@ -534,6 +553,7 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
         ASSERT(data_offset < size);
     }
     spin_unlock(&pdev->vpci->lock);
+    write_unlock(&d->pci_lock);
 
     if ( data_offset < size )
         /* Tailing gap, write the remaining. */
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index aabc5465a7d3..9f31cb84c9f3 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -171,6 +171,20 @@ void pcidevs_lock(void);
 void pcidevs_unlock(void);
 bool __must_check pcidevs_locked(void);
 
+#ifndef NDEBUG
+/*
+ * Check for use in ASSERTs to ensure there will be no changes to the entries
+ * in d->pdev_list (but not the contents of each entry).
+ * This check is not suitable for protecting other state or critical regions.
+ */
+#define pdev_list_is_read_locked(d) ({                             \
+        /* NB: d may be evaluated multiple times, or not at all */ \
+        pcidevs_locked() || (d && rw_is_locked(&d->pci_lock));     \
+    })
+#else
+bool pdev_list_is_read_locked(const struct domain *d);
+#endif
+
 bool pci_known_segment(u16 seg);
 bool pci_device_detect(u16 seg, u8 bus, u8 dev, u8 func);
 int scan_pci_devices(void);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 9da91e0e6244..37f5922f3206 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -462,7 +462,8 @@ struct domain
 #ifdef CONFIG_HAS_PCI
     struct list_head pdev_list;
     /*
-     * pci_lock protects access to pdev_list.
+     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
+     * structure from being removed.
      *
      * Any user *reading* from pdev_list, or from devices stored in pdev_list,
      * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,

base-commit: c4d6802d33f962a306405a20f4dccdc287990d07
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 15 20:50:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 15 Feb 2024 20:50:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.681996.1061058 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raigX-0004Bn-93; Thu, 15 Feb 2024 20:50:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 681996.1061058; Thu, 15 Feb 2024 20:50:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raigX-0004Bg-65; Thu, 15 Feb 2024 20:50:21 +0000
Received: by outflank-mailman (input) for mailman id 681996;
 Thu, 15 Feb 2024 20:50:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raigW-0004BW-Kq; Thu, 15 Feb 2024 20:50:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raigW-00085S-Bk; Thu, 15 Feb 2024 20:50:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1raigV-00050E-VR; Thu, 15 Feb 2024 20:50:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1raigV-0002KS-Ut; Thu, 15 Feb 2024 20:50:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7mjEFDNWdL0TFTmsHgFUU1OMdygdKdZZU+I1tJey6Is=; b=Bn7KcZQHKajo/zAI/vx/FvyLVz
	emhp+MkrNBP+a+9s6+Vbdy98AmgArUsyarUFt5hGPFfiXvSg9a5wIF3gGurbec8ruWFxEfMEA+xII
	lvbTu6yyqenoQ6VaWjFyx1KdhNzv0/13F4FAHRCMbR/2aIsU/MJ0TgFvr2dQ5VILJ6Wk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184678-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184678: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=78398afae10bfb4ab94e8af17b7ed58510a57d96
X-Osstest-Versions-That:
    xen=c4d6802d33f962a306405a20f4dccdc287990d07
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 15 Feb 2024 20:50:19 +0000

flight 184678 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184678/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  78398afae10bfb4ab94e8af17b7ed58510a57d96
baseline version:
 xen                  c4d6802d33f962a306405a20f4dccdc287990d07

Last test of basis   184675  2024-02-15 13:03:50 Z    0 days
Testing same since   184678  2024-02-15 18:02:04 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Juergen Gross <jgross@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   c4d6802d33..78398afae1  78398afae10bfb4ab94e8af17b7ed58510a57d96 -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 01:27:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 01:27:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682013.1061069 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ran0U-0006II-Q8; Fri, 16 Feb 2024 01:27:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682013.1061069; Fri, 16 Feb 2024 01:27:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ran0U-0006IB-LD; Fri, 16 Feb 2024 01:27:14 +0000
Received: by outflank-mailman (input) for mailman id 682013;
 Fri, 16 Feb 2024 01:27:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ran0T-0006I1-Io; Fri, 16 Feb 2024 01:27:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ran0T-0003OY-Ay; Fri, 16 Feb 2024 01:27:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ran0S-00011E-Rq; Fri, 16 Feb 2024 01:27:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1ran0S-0008Lj-RJ; Fri, 16 Feb 2024 01:27:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LKrWRXe3iGUO39zyBtc/kv/XDPC5MKUzf/WG2iUOdDY=; b=EmD5M0hlwM+cbZ0p6BfLCSkyiK
	wL4yS/PROMpRurVmOpOkuLAdHVBSR6BrLpmMJbLxfu5reG66xKISmn1SeCr190566gnwIowqrcWzN
	0jFlTl/N6a8H1/nUJmtixl1rHyF5YvEUiPsTtTx72CO+WjnZBZiyF0Dtt1HyIWfNHJXE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184677-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184677: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:guest-start/debianhvm.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt-raw:guest-start:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=c4d6802d33f962a306405a20f4dccdc287990d07
X-Osstest-Versions-That:
    xen=d9bfc2810e1cbe4cd27555bdc48315d36ee91f14
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 16 Feb 2024 01:27:12 +0000

flight 184677 xen-unstable real [real]
flight 184680 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184677/
http://logs.test-lab.xenproject.org/osstest/logs/184680/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 20 guest-start/debianhvm.repeat fail pass in 184680-retest
 test-armhf-armhf-libvirt-raw 13 guest-start         fail pass in 184680-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check fail in 184680 like 184673
 test-armhf-armhf-libvirt-raw 14 migrate-support-check fail in 184680 never pass
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184670
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184673
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184673
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184673
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184673
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184673
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184673
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184673
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184673
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184673
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184673
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  c4d6802d33f962a306405a20f4dccdc287990d07
baseline version:
 xen                  d9bfc2810e1cbe4cd27555bdc48315d36ee91f14

Last test of basis   184673  2024-02-15 07:18:01 Z    0 days
Testing same since   184677  2024-02-15 16:38:53 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 fail    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   d9bfc2810e..c4d6802d33  c4d6802d33f962a306405a20f4dccdc287990d07 -> master


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 05:56:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 05:56:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682021.1061083 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rarCY-0001Jq-9x; Fri, 16 Feb 2024 05:55:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682021.1061083; Fri, 16 Feb 2024 05:55:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rarCY-0001Jj-6N; Fri, 16 Feb 2024 05:55:58 +0000
Received: by outflank-mailman (input) for mailman id 682021;
 Fri, 16 Feb 2024 05:55:57 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rarCW-0001JZ-VJ; Fri, 16 Feb 2024 05:55:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rarCW-00019x-HA; Fri, 16 Feb 2024 05:55:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rarCV-0004nH-U6; Fri, 16 Feb 2024 05:55:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rarCV-0004yu-Qv; Fri, 16 Feb 2024 05:55:55 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=P5liVFGADLPW0z59/KKRcrFSqul5H0v+efGqE5+F0o8=; b=VDPlnPu6tsNw5MrJjM1lONLhdy
	mQ0cLxDourAd+zb35PX48CXdtzqnd7duGDMpwX/RJOVonvff0bcrrRyLm+W3cF6m+Y/+WBkrq+YAw
	7QYEtqli/UmW/Hah8cZhpSXrd8OmG+bWjjWPUhkRPtb2mUQYLjqoSCJ/YWrXyYsMesaU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184679-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184679: regressions - trouble: broken/fail/pass
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-libvirt-raw:<job status>:broken:regression
    linux-linus:test-amd64-amd64-libvirt-raw:host-install(5):broken:regression
    linux-linus:test-armhf-armhf-xl-credit1:xen-boot:fail:regression
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4f5e5092fdbf5cec6bedc19fbe69cce4f5f08372
X-Osstest-Versions-That:
    linux=8d3dea210042f54b952b481838c1e7dfc4ec751d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 16 Feb 2024 05:55:55 +0000

flight 184679 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184679/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-libvirt-raw    <job status>                 broken
 test-amd64-amd64-libvirt-raw  5 host-install(5)        broken REGR. vs. 184671
 test-armhf-armhf-xl-credit1   8 xen-boot                 fail REGR. vs. 184671

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184671
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184671
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184671
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184671
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184671
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184671
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184671
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184671
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4f5e5092fdbf5cec6bedc19fbe69cce4f5f08372
baseline version:
 linux                8d3dea210042f54b952b481838c1e7dfc4ec751d

Last test of basis   184671  2024-02-15 00:41:18 Z    1 days
Testing same since   184679  2024-02-15 20:11:16 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aaron Conole <aconole@redhat.com>
  Alexander Aring <aahringo@redhat.com>
  Allison Henderson <allison.henderson@oracle.com>
  André Draszik <andre.draszik@linaro.org>
  Arnd Bergmann <arnd@arndb.de>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  Ben Wolsieffer <ben.wolsieffer@hefring.com>
  Breno Leitao <leitao@debian.org>
  Carlos Song <carlos.song@nxp.com>
  Chandan Kumar Rout <chandanx.rout@intel.com> (A Contingent Worker at Intel)
  Christian A. Ehrhardt <lk@c--e.de>
  Christian Lamparter <chunkeey@gmail.com>
  Chuck Lever <chuck.lever@oracle.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Gabay <daniel.gabay@intel.com>
  Dave Ertman <david.m.ertman@intel.com>
  David S. Miller <davem@davemloft.net>
  Dmitry Antipov <dmantipov@yandex.ru>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Emmanuel Grumbach <emmanuel.grumbach@intel.com>
  Eric Dumazet <edumazet@google.com>
  Even Xu <even.xu@intel.com>
  Felix Fietkau <nbd@nbd.name>
  Florian Fainelli <f.fainelli@gmail.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Gavrilov Ilia <Ilia.Gavrilov@infotecs.ru>
  Geliang Tang <geliang.tang@linux.dev>
  Geliang Tang <geliang@kernel.org>
  Guenter Roeck <linux@roeck-us.net>
  Horatiu Vultur <horatiu.vultur@microchip.com>
  Ido Schimmel <idosch@nvidia.com>
  Ivan Vecera <ivecera@redhat.com>
  Jakub Kicinski <kuba@kernel.org>
  Jamal Hadi Salim <jhs@mojatatu.com>
  Jason Gerecke <jason.gerecke@wacom.com>
  Jason Gerecke <killertofu@gmail.com>
  Jiri Kosina <jkosina@suse.com>
  Jiri Pirko <jiri@nvidia.com>
  Johannes Berg <johannes.berg@intel.com>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Juergen Gross <jgross@suse.com>
  Kees Cook <keescook@chromium.org>
  Keqi Wang <wangkeqi_chris@163.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Kunwu Chan <chentao@kylinos.cn>
  Kyle Swenson <kyle.swenson@est.tech>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Maciej Fijalkowski <maciej.fijalkowski@intel.com>
  Magnus Karlsson <magnus.karlsson@intel.com>
  Manuel Fombuena <fombuena@outlook.com>
  Marc Kleine-Budde <mkl@pengutronix.de>
  Mark Brown <broonie@kernel.org>
  Markus Elfring <elfring@users.sourceforge.net>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Maxim Galaganov <max@internet.ru>
  Maxime Jayat <maxime.jayat@mobile-devices.fr>
  Maximilian Heyne <mheyne@amazon.de>
  Michal Simek <michal.simek@amd.com>
  Mika Westerberg <mika.westerberg@linux.intel.com>
  Miri Korenblit <miriam.rachel.korenblit@intel.com>
  Mubin Sayyed <mubin.sayyed@amd.com>
  Naama Meir <naamax.meir@linux.intel.com>
  Niklas Cassel <cassel@kernel.org>
  Nikolay Aleksandrov <razor@blackwall.org>
  Nuno Sa <nuno.sa@analog.com>
  Nícolas F. R. A. Prado <nfraprado@collabora.com>
  Oleksandr Natalenko <oleksandr@natalenko.name>
  Oleksij Rempel <o.rempel@pengutronix.de>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Paolo Abeni <pabeni@redhat.com>
  Parav Pandit <parav@nvidia.com>
  Paul Barker <paul.barker.ct@bp.renesas.com>
  Paul Durrant <paul@xen.org>
  Piyush Mehta <piyush.mehta@amd.com>
  Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
  Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
  Rafal Romanowski <rafal.romanowski@intel.com>
  Ralf Schlatterbeck <rsc@runtux.com>
  Randy Dunlap <rdunlap@infradead.org>
  Ricardo B. Marliere <ricardo@marliere.net>
  Rob Herring <robh@kernel.org>
  Sabrina Dubroca <sd@queasysnail.net>
  Saravana Kannan <saravanak@google.com>
  Sasha Neftin <sasha.neftin@intel.com>
  SeongJae Park <sj@kernel.org>
  Seth Forshee <sforshee@kernel.org>
  Shannon Nelson <shannon.nelson@amd.com>
  Simon Horman <horms@kernel.org>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Subbaraya Sundeep <sbhatta@marvell.com>
  Sujai Buvaneswaran <sujai.buvaneswaran@intel.com>
  Tatsunosuke Tobita <tatsunosuke.tobita@wacom.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vadim Fedorenko <vadfed@meta.com>
  Vaishnav Achath <vaishnav.a@ti.com>
  Victor Nogueira <victor@mojatatu.com>
  Xin Long <lucien.xin@gmail.com>
  Ziqi Zhao <astrajoan@yahoo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 broken  
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-amd64-amd64-libvirt-raw broken
broken-step test-amd64-amd64-libvirt-raw host-install(5)

Not pushing.

(No revision log; it would be 3772 lines long.)


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 08:33:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 08:33:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682047.1061093 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raten-0002H1-PP; Fri, 16 Feb 2024 08:33:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682047.1061093; Fri, 16 Feb 2024 08:33:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raten-0002Gu-Mi; Fri, 16 Feb 2024 08:33:17 +0000
Received: by outflank-mailman (input) for mailman id 682047;
 Fri, 16 Feb 2024 08:33:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AOVM=JZ=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1ratem-0002G2-DR
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 08:33:16 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 060d3ef3-cca6-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 09:33:14 +0100 (CET)
Received: from CYZPR02CA0013.namprd02.prod.outlook.com (2603:10b6:930:a1::11)
 by BN9PR12MB5145.namprd12.prod.outlook.com (2603:10b6:408:136::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.14; Fri, 16 Feb
 2024 08:33:11 +0000
Received: from CY4PEPF0000EE36.namprd05.prod.outlook.com
 (2603:10b6:930:a1:cafe::92) by CYZPR02CA0013.outlook.office365.com
 (2603:10b6:930:a1::11) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.41 via Frontend
 Transport; Fri, 16 Feb 2024 08:33:10 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EE36.mail.protection.outlook.com (10.167.242.42) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Fri, 16 Feb 2024 08:33:09 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 16 Feb
 2024 02:33:09 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 16 Feb 2024 02:33:08 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 060d3ef3-cca6-11ee-8a4f-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=adyCmYYO+RE9FOZSduN54ZuPWa506ZMwlEu2QZgo9UvCgHir5MY5e150zpSihfyFTnJzL+cbJk56N6sFNU5gElRXh/3sLfnxGIsNHY1V5AW65+PPbkqqCok12CZRp2XRH5P93uXg2IMVoENRdOQ2mPOdHAxzoNeHq2+S4Yy0CC4RZeeeiCAHAYYFIxO5gMYDFiqhEd/Hy0Ny5V+1csLSAsewdwWF80OksWvfyjrpnz0bqoIxzy/RRQmC+qbhDo04u57FCNedPmCek1EemV7cwbhN2a96iClwmMrXN2RMADpp/XSsoJ9jvnlA3rB+u6/WhTGQGHHmJXbqzwxe8BeIag==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=UCc/sDE6mMSW7mn4yBfAaAU+QC0ySQvwfs6JiD2mR3A=;
 b=TpIKT0+1vd4m/ui780e8Cfns+ztBbjklus7Ph/Wjzk95Jupsg88xvqX9rachm7+c2gUwqh0teRFMRXbFMEZifO7uEJ/vcCqUdRLzPgNfgQr24C4Z7kU+tLLZW79p2up0UzBD2yej48PcBf/JuG7/ki6WlHCFeAyQSG8lUri7pGmsryM8XWMf5z9Z4BNHHGw2P3RRacjz3MqHaWTvb25VABpuR9HXaaiXotzwGVveFy7P+PLuNE/X9L47ZdIK6p3CRyOcugR9YltBEvLZNxTlwIlq4JRKD68w0DEuyp4x17y2wwEBBDTdqXt8x26UsNMhEoT/o3GOoPTBo4ynES/2aw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UCc/sDE6mMSW7mn4yBfAaAU+QC0ySQvwfs6JiD2mR3A=;
 b=DLUCLrlDOBQLM90uBd0bpPLr4Efz0us8WqYhRtYonjJx9VFjyiLlcB9ZdHKo7JzI+s7mep2QWIR9/Hzq2ANbCKIh1ZOFk5s1IfokctU053uSIg0c4mrYwnKHv3gWSlBwCAX1CTFygXKHv6dN1QByd84tqD7KiLFDwyZSj+7n4vA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <2568567a-d876-4e31-8576-88fca23e4b45@amd.com>
Date: Fri, 16 Feb 2024 09:33:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Make hwdom vUART optional feature
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240215143947.90073-1-michal.orzel@amd.com>
 <8ea2f0ef-29ab-4a50-a59f-1f01af2f1b2a@xen.org>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <8ea2f0ef-29ab-4a50-a59f-1f01af2f1b2a@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE36:EE_|BN9PR12MB5145:EE_
X-MS-Office365-Filtering-Correlation-Id: 67e2e47d-d9c0-4187-14c6-08dc2ec9e844
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XF5rr7QJrFZzKa5FY36GXiJKEqeE6M60AysjQgUENmCgvNoPmRHSHEIsnLZ1zX94cx5gNRdH5Nybu+FaDvcStidTFnKpDIEvbsbKeK/BVNItqIrN74sdyeXPTra52vct6Nx9yi8S+OQXMjszRYomSgvfVrpB0ZtTk51uo4sJ+PymWag2K/GMC/TLb8ZdnFfVpoUWLs4soJ7R72gb2JIUcApmNCcwWLpYJENcYYE0HUqDv+7d4CoD3U3HX2pI00HzNPm1ZclrEX4ig5wfL/kWkz4ZV4IfedU330TzIy1w+GGh6z6VDqQSBQqvSHVIoSPmzuZ6uw10FOMplIPixJyQaCgAPHrJgu3+9o+YyTzimWYX4+xszevx3YA8Li3or/S1FSg/9Egu4YB/YzLSd4cJ84Rk9NUddaz87sMuH9KFUp1RR/tamUDvcFqShdmOmpfCpMU5pVbRspfwF08piq697Pr9X+H29tfuJ4mDrgBhwXoT4DNYj/mpdVe6G8gQ22v0MoS5Jq+KdS1XaXU5uJk7BnTOvy39NHbyX+KBpJCZowtFJQJAdM5P5tQYbeyiX5BKjWeawgIxFXaVt20sY+eAfwX1GZ7SVyVdpaCp4TtvSFo0XxBEb1dsXHLZMmWqT4vES3n1QHF2Yt2V5tFjb5UQqipjxWvJ0RZdFIywSD084eE=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(396003)(346002)(376002)(39860400002)(136003)(230922051799003)(82310400011)(451199024)(1800799012)(64100799003)(186009)(36860700004)(46966006)(40470700004)(83380400001)(44832011)(2906002)(356005)(81166007)(110136005)(82740400003)(8936002)(8676002)(54906003)(16576012)(316002)(53546011)(4326008)(478600001)(70206006)(70586007)(26005)(2616005)(336012)(426003)(5660300002)(41300700001)(86362001)(31696002)(36756003)(31686004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Feb 2024 08:33:09.8910
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 67e2e47d-d9c0-4187-14c6-08dc2ec9e844
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EE36.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5145

Hi Julien,

On 15/02/2024 17:05, Julien Grall wrote:
> 
> 
> Hi Michal,
> 
> On 15/02/2024 14:39, Michal Orzel wrote:
>> At the moment, the hardware domain vUART is always compiled in. In the
>> spirit of fine granular configuration, make it optional so that the
>> feature can be disabled if not needed. This UART is not exposed (e.g.
>> via device tree) to a domain and is mostly used to support special use
>> cases like Linux early printk, prints from the decompressor code, etc.
>>
>> Introduce Kconfig option CONFIG_HWDOM_VUART, enabled by default (to keep
>> the current behavior) and use it to protect the vUART related code.
>> Provide stubs for domain_vuart_{init,free}() in case the feature is
>> disabled. Take the opportunity to add a struct domain forward declaration
>> to vuart.h, so that the header is self contained.
>>
>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>> ---
>>   xen/arch/arm/Kconfig              |  8 ++++++++
>>   xen/arch/arm/Makefile             |  2 +-
>>   xen/arch/arm/include/asm/domain.h |  2 ++
>>   xen/arch/arm/vuart.h              | 15 +++++++++++++++
>>   4 files changed, 26 insertions(+), 1 deletion(-)
>>
>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>> index 50e9bfae1ac8..72af329564b7 100644
>> --- a/xen/arch/arm/Kconfig
>> +++ b/xen/arch/arm/Kconfig
>> @@ -150,6 +150,14 @@ config SBSA_VUART_CONSOLE
>>         Allows a guest to use SBSA Generic UART as a console. The
>>         SBSA Generic UART implements a subset of ARM PL011 UART.
>>
>> +config HWDOM_VUART
>> +     bool "Emulated UART for hardware domain"
>> +     default y
>> +     help
>> +       Allows a hardware domain to use a minimalistic UART (single transmit
>> +       and status register) which takes information from dtuart. Note that this
>> +       UART is not intended to be exposed (e.g. via device-tree) to a domain.
>> +
>>   config ARM_SSBD
>>       bool "Speculative Store Bypass Disable" if EXPERT
>>       depends on HAS_ALTERNATIVE
>> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
>> index 33c677672fe6..7b1350e2ef0a 100644
>> --- a/xen/arch/arm/Makefile
>> +++ b/xen/arch/arm/Makefile
>> @@ -71,7 +71,7 @@ obj-y += vtimer.o
>>   obj-$(CONFIG_SBSA_VUART_CONSOLE) += vpl011.o
>>   obj-y += vsmc.o
>>   obj-y += vpsci.o
>> -obj-y += vuart.o
>> +obj-$(CONFIG_HWDOM_VUART) += vuart.o
>>
>>   extra-y += xen.lds
>>
>> diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
>> index 5fb8cd79c01a..8218afb8626a 100644
>> --- a/xen/arch/arm/include/asm/domain.h
>> +++ b/xen/arch/arm/include/asm/domain.h
>> @@ -91,6 +91,7 @@ struct arch_domain
>>
>>       struct vgic_dist vgic;
>>
>> +#ifdef CONFIG_HWDOM_VUART
>>       struct vuart {
>>   #define VUART_BUF_SIZE 128
>>           char                        *buf;
>> @@ -98,6 +99,7 @@ struct arch_domain
>>           const struct vuart_info     *info;
>>           spinlock_t                  lock;
>>       } vuart;
>> +#endif
>>
>>       unsigned int evtchn_irq;
>>   #ifdef CONFIG_ACPI
>> diff --git a/xen/arch/arm/vuart.h b/xen/arch/arm/vuart.h
>> index bd23bd92f631..36658b4a8c7f 100644
>> --- a/xen/arch/arm/vuart.h
>> +++ b/xen/arch/arm/vuart.h
>> @@ -20,9 +20,24 @@
>>   #ifndef __ARCH_ARM_VUART_H__
>>   #define __ARCH_ARM_VUART_H__
>>
>> +struct domain;
>> +
>> +#ifdef CONFIG_HWDOM_VUART
>> +
>>   int domain_vuart_init(struct domain *d);
>>   void domain_vuart_free(struct domain *d);
>>
>> +#else
>> +
>> +static inline int domain_vuart_init(struct domain *d)
>> +{
>> +    return 0;
> 
> NIT: I was going to query why we return 0 rather than -EOPNOSUPP. But it
> looks like this is because domain_vuart_init() is unconditionally called
> for the hardware domain. This is unusual and would be worth adding a
> comment.
> 
> In any case,
> 
> Acked-by: Julien Grall <jgrall@amazon.com>
Thanks. If you want to add a comment, feel free to do in on commit.

~Michal


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 09:11:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 09:11:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682055.1061103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rauG3-00070B-Km; Fri, 16 Feb 2024 09:11:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682055.1061103; Fri, 16 Feb 2024 09:11:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rauG3-000704-HU; Fri, 16 Feb 2024 09:11:47 +0000
Received: by outflank-mailman (input) for mailman id 682055;
 Fri, 16 Feb 2024 09:11:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=dcAy=JZ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rauG3-0006zy-4C
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 09:11:47 +0000
Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com
 [2a00:1450:4864:20::22b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 685f0bbd-ccab-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 10:11:45 +0100 (CET)
Received: by mail-lj1-x22b.google.com with SMTP id
 38308e7fff4ca-2d109e7bed2so23203341fa.2
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 01:11:45 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 p13-20020a05620a22ad00b0078565ed2bc6sm1336749qkh.124.2024.02.16.01.11.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 01:11:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 685f0bbd-ccab-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708074705; x=1708679505; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=UhNCxLnq2Bh/M+Qg1rjlgZSPcWfjNiUbNdNc86SOifE=;
        b=cpgV194Hv0ED1EfVBvj8lIHvVJeM9w0Ilgc+GPZE40ANDlF99xWdLQTGr8EDmLPFA1
         3/Gd4VfHSuh3p+rdBS+w+40q4ISpAMvKz5ZKaxpG98cHJhZngwKPR7Y71tIwTJSSpEwF
         Wnw5Fu4Jw2LU64LKZPHorCLovoqUGMb/f5IwA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708074705; x=1708679505;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=UhNCxLnq2Bh/M+Qg1rjlgZSPcWfjNiUbNdNc86SOifE=;
        b=FLvIcE/wG/9H5ydUtawoUPOxQal3bAs7XkQIvrWV+XS/o1wU2qwBZnUeUUeUuQLHau
         dpUoWBSVveNhXamEwz5mRTYJ2rkXiqwiGK54WuAobn1YvwS8Swi6d+aqjHU4ObFyRQiJ
         1dHLRI6CvMHk4aVpJzgUnB17BfQui1ESwRrv9FTTgB7R7TYZ/LhXjkmZf/whfVbNbWlR
         bKN/OBIXMk13ekr44H3KdE4q6V48r5HXiu02TdSfDI3bC8GlCnG+yO3y4dSWPA2KiVmI
         WeGaolenndhER0hzHD5/0H7IN9HKHTkCoMGjGNUgU57g4cqIBTUesiBSyFA1eQz/hMky
         KcOg==
X-Gm-Message-State: AOJu0Yw2Q3F1Qonp/1ijChPTQXLalDc2UQO1QQqibp6QtbaIzOOfa77n
	KqTYiMbsL0Fp7NcEObR+0alwE0TsKf4OAG+e2kM8dMo/z1muAln5UWeTKp3D8C8=
X-Google-Smtp-Source: AGHT+IG36kIwA9kRzfLhRc+OotuuzAsRXdzaJfGfrqRkDL2NBMY4AZZUvinOAkKdOKI7V46jQI1wLQ==
X-Received: by 2002:a2e:b707:0:b0:2d2:1c15:dc0b with SMTP id j7-20020a2eb707000000b002d21c15dc0bmr462761ljo.40.1708074705285;
        Fri, 16 Feb 2024 01:11:45 -0800 (PST)
Date: Fri, 16 Feb 2024 10:11:42 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH v2] x86: amend 'n' debug-key output with SMI count
Message-ID: <Zc8mzlzLTJNwBJup@macbook>
References: <549909b7-e34c-4a5c-aa21-9892a1724042@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <549909b7-e34c-4a5c-aa21-9892a1724042@suse.com>

On Wed, Feb 14, 2024 at 11:15:51AM +0100, Jan Beulich wrote:
> ... if available only, of course.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> I don't really like issuing an IPI (and having another cf_check
> function) here, yet then again this is issued only when the debug key
> is actually used, and given how simple the handling function is
> (including that it doesn't use its parameter) it also looks difficult
> to abuse.
> ---
> v2: Actually read each CPU's SMI count in do_nmi_stats().
> 
> --- a/xen/arch/x86/cpu/common.c
> +++ b/xen/arch/x86/cpu/common.c
> @@ -407,9 +407,15 @@ void __init early_cpu_init(bool verbose)
>  		paddr_bits -= (ebx >> 6) & 0x3f;
>  	}
>  
> -	if (!(c->x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON)))
> +	if (!(c->x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON))) {
> +		uint64_t smi_count;
> +
>  		park_offline_cpus = opt_mce;
>  
> +		if (!verbose && !rdmsr_safe(MSR_SMI_COUNT, smi_count))
> +			setup_force_cpu_cap(X86_FEATURE_SMI_COUNT);

Why make it dependent on !verbose?  The call with !verbose is tied to
part of the ucode loading being half-functional (for example
MCU_CONTROL_DIS_MCU_LOAD not being set) but I don't see that as a
signal that SMI count shouldn't be used.

does it need to be part of the early cpu initialization instead of
being in the (later) Intel specific init code part of the
identify_cpu()?

> +	}
> +
>  	initialize_cpu_data(0);
>  }
>  
> --- a/xen/arch/x86/include/asm/cpufeatures.h
> +++ b/xen/arch/x86/include/asm/cpufeatures.h
> @@ -24,7 +24,7 @@ XEN_CPUFEATURE(APERFMPERF,        X86_SY
>  XEN_CPUFEATURE(MFENCE_RDTSC,      X86_SYNTH( 9)) /* MFENCE synchronizes RDTSC */
>  XEN_CPUFEATURE(XEN_SMEP,          X86_SYNTH(10)) /* SMEP gets used by Xen itself */
>  XEN_CPUFEATURE(XEN_SMAP,          X86_SYNTH(11)) /* SMAP gets used by Xen itself */
> -/* Bit 12 unused. */
> +XEN_CPUFEATURE(SMI_COUNT,         X86_SYNTH(12)) /* MSR_SMI_COUNT exists */
>  XEN_CPUFEATURE(IND_THUNK_LFENCE,  X86_SYNTH(13)) /* Use IND_THUNK_LFENCE */
>  XEN_CPUFEATURE(IND_THUNK_JMP,     X86_SYNTH(14)) /* Use IND_THUNK_JMP */
>  XEN_CPUFEATURE(SC_NO_BRANCH_HARDEN, X86_SYNTH(15)) /* (Disable) Conditional branch hardening */
> --- a/xen/arch/x86/include/asm/msr-index.h
> +++ b/xen/arch/x86/include/asm/msr-index.h
> @@ -28,6 +28,8 @@
>  #define  TEST_CTRL_SPLITLOCK_DETECT         (_AC(1, ULL) << 29)
>  #define  TEST_CTRL_SPLITLOCK_DISABLE        (_AC(1, ULL) << 31)
>  
> +#define MSR_SMI_COUNT                       0x00000034
> +
>  #define MSR_INTEL_CORE_THREAD_COUNT         0x00000035
>  #define  MSR_CTC_THREAD_MASK                0x0000ffff
>  #define  MSR_CTC_CORE_MASK                  _AC(0xffff0000, U)
> --- a/xen/arch/x86/nmi.c
> +++ b/xen/arch/x86/nmi.c
> @@ -585,15 +585,34 @@ static void cf_check do_nmi_trigger(unsi
>      self_nmi();
>  }
>  
> +static DEFINE_PER_CPU(unsigned int, smi_count);
> +
> +static void cf_check read_smi_count(void *unused)
> +{
> +    unsigned int dummy;
> +
> +    rdmsr(MSR_SMI_COUNT, this_cpu(smi_count), dummy);
> +}
> +
>  static void cf_check do_nmi_stats(unsigned char key)
>  {
>      const struct vcpu *v;
>      unsigned int cpu;
>      bool pend, mask;
>  
> -    printk("CPU\tNMI\n");
> +    printk("CPU\tNMI%s\n", boot_cpu_has(X86_FEATURE_SMI_COUNT) ? "\tSMI" : "");
> +
> +    if ( boot_cpu_has(X86_FEATURE_SMI_COUNT) )
> +        on_each_cpu(read_smi_count, NULL, 1);
> +
>      for_each_online_cpu ( cpu )
> -        printk("%3u\t%3u\n", cpu, per_cpu(nmi_count, cpu));
> +    {
> +        printk("%3u\t%3u", cpu, per_cpu(nmi_count, cpu));
> +        if ( boot_cpu_has(X86_FEATURE_SMI_COUNT) )
> +            printk("\t%3u\n", per_cpu(smi_count, cpu));
> +        else
> +            printk("\n");
> +    }
>  
>      if ( !hardware_domain || !(v = domain_vcpu(hardware_domain, 0)) )
>          return;

Could you also amend the debug-key help text to mention SMI?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 10:05:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 10:05:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682093.1061129 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rav5Q-0006N2-RZ; Fri, 16 Feb 2024 10:04:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682093.1061129; Fri, 16 Feb 2024 10:04:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rav5Q-0006Mv-OQ; Fri, 16 Feb 2024 10:04:52 +0000
Received: by outflank-mailman (input) for mailman id 682093;
 Fri, 16 Feb 2024 10:04:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x+xa=JZ=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rav5P-0006Mp-6M
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 10:04:51 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d1211488-ccb2-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 11:04:48 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-4125296400cso904935e9.0
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 02:04:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 co3-20020a0560000a0300b0033cf9e35b13sm1796564wrb.72.2024.02.16.02.04.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 16 Feb 2024 02:04:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1211488-ccb2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708077887; x=1708682687; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lVt+Hm/7gTEmh+6OtNrEJ1A9UUR0Y+pgKDaSGK2e/jk=;
        b=abyAyoO643nUvVphHepI9wx5Zt+4B6KjSkEf+5FWSyUJ7Tzn9nQGdcRUvyPfOzhfLt
         N662dp7+wmKjxqasluGi+bio8QSyX+yG4mm8j8QPJZ7C5ud0JQkif2wYGacbcSZb4Uf3
         aPpzXrR4QwVPeFqZxXLmAYy+XFf9tJ0LUZf8yL23EiifnZMGJFAqOUyB2aNkqaEBmEpk
         IFhVM4rJcpypy68++PuT44xyC5OBxmbqC3FOUnZ4oAtg/W4fQnq3tZj3S1cPnaIKeI5q
         8wZQBfuLB0uYvceOfNAZJLx5BFzuoMeq4KBj50P7iAz+fhpQe3L+rsZHcYudvvmx9fH5
         MMYw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708077887; x=1708682687;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=lVt+Hm/7gTEmh+6OtNrEJ1A9UUR0Y+pgKDaSGK2e/jk=;
        b=OeVfz1a0YtH/XzMBui/Hj7Rp0/YJd4YyvB03njrgKAL063Fxjlvs6AfXfLkqHyL9lg
         Gn6p1dAEogWBkchr/+vzQ3NFwtlrkuyu/DFf8GN3W/7EoRfSmH8TtcgiHTx8FhcrHrqL
         9ZBRGetDo0zG7A+sWzpHPeNftsaAJ3zL+UQuC3WOkPLs3NoXy7gLJxfKfcEG6tYSdfv0
         9VxJy/q5oQFyUQs2S2WpqVBI1FKGOmPA0nuNYxN4882Ln4646lpYnT7o3X3vpS5THJ1U
         1J/1OZ6uBExMIVicy7DTnQuK7UF6PxcfydrTRiKrH/vll+d3p9fKBk4f3VWiyJPrz7hc
         tQTw==
X-Forwarded-Encrypted: i=1; AJvYcCXA7ZHrgNkmsQdg5870oFHFZZU0J8gpCYFFyKsgMbnJRIgysxOFq7Vdzg+3UDlAJjQe3Xo4hnptaWThm4us8DRBktetCVg5XIn5XrpFxQI=
X-Gm-Message-State: AOJu0YyRhYRUKKXzbz+qJbeeX7gb9pv+EXrdnmFCaqfgNh4wRCPKt8XV
	oQkYybml9wsw0LyurxbzTUFnuIT8sphEpHIZV2eF87+UG+Lqyalr/6DuCADTYA==
X-Google-Smtp-Source: AGHT+IE1RtzZeRnmQNQ05C0/fYx//I2rkKKD7tKUfXSnr4QqZDsujzG5wQivJXZhKZ/WseW/RHjukg==
X-Received: by 2002:a05:600c:5006:b0:412:1ccd:d7a4 with SMTP id n6-20020a05600c500600b004121ccdd7a4mr2841196wmr.14.1708077887613;
        Fri, 16 Feb 2024 02:04:47 -0800 (PST)
Message-ID: <2fe68b26-87c3-43bf-ba11-f261c81c6373@suse.com>
Date: Fri, 16 Feb 2024 11:04:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Anthony Perard <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com> <Zc3nXpUOlnIHEfsl@macbook>
 <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com> <Zc3v20RKMssbaDsl@macbook>
 <a48cbad6-701d-4077-9044-4205b932a7f3@suse.com> <Zc428VMDoYnPw1zo@macbook>
 <d545cc6c-d213-43da-af31-1768af32aba0@suse.com> <Zc5Io3dkAlGSt3on@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zc5Io3dkAlGSt3on@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.02.2024 18:23, Roger Pau Monné wrote:
> On Thu, Feb 15, 2024 at 05:22:00PM +0100, Jan Beulich wrote:
>> On 15.02.2024 17:08, Roger Pau Monné wrote:
>>> On Thu, Feb 15, 2024 at 02:02:41PM +0100, Jan Beulich wrote:
>>>> --- a/xen/Rules.mk
>>>> +++ b/xen/Rules.mk
>>>> @@ -15,7 +15,11 @@ srcdir := $(srctree)/$(src)
>>>>  PHONY := __build
>>>>  __build:
>>>>  
>>>> --include $(objtree)/include/config/auto.conf
>>>> +ifneq ($(obj),tools)
>>>> +ifneq ($(obj),tools/kconfig)
>>>> +include $(objtree)/include/config/auto.conf
>>>> +endif
>>>> +endif
>>>
>>> Trying to understand this, I assume it's to avoid an infinite
>>> dependency loop that generating include/config/auto.conf requires some
>>> tools that are build using xen/Rules.mk?
>>
>> The file has dependencies only in xen/Makefile. This is about the
>> file simply not being there when initially building. Perhaps the
>> patch description helps that I've written in the meantime:
>>
>> "Because of using "-include", failure to (re)build auto.conf (with
>>  auto.conf.cmd produced as a secondary target) won't stop make from
>>  continuing the build. Arrange for it being possible to drop the - from
>>  Rules.mk, requiring that the include be skipped for tools-only targets.
> 
> Wouldn't it be more reliable if we skipped the include for any paths
> in $(obj) that start with 'tools', rather than hardcoding 'tools' and
> 'tools/kconfig'?

I was first meaning to do so, but the expression would end up more
complex than I'd like (for it needing to be an exact match of "tools"
and a prefix match of "tools/"). Thinking of it,

ifneq ($(obj),tools)
ifneq ($(patsubst tools/%,$(obj)),)

might do (and not be as complex as I first thought, when intending to
put all in a single "if").

Yet then it's not entirely impossible that we might gain a build tool
which is actually to be built after .config was (re)generated, i.e.
in the know of the target configuration. Thoughts?

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 10:06:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 10:06:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682097.1061138 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rav6h-0006rz-50; Fri, 16 Feb 2024 10:06:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682097.1061138; Fri, 16 Feb 2024 10:06:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rav6h-0006rs-20; Fri, 16 Feb 2024 10:06:11 +0000
Received: by outflank-mailman (input) for mailman id 682097;
 Fri, 16 Feb 2024 10:06:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LvWc=JZ=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rav6f-0006rm-IR
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 10:06:09 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fec0d1d1-ccb2-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 11:06:04 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-410ca9e851bso14351175e9.3
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 02:06:04 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 k35-20020a05600c1ca300b00412393ddac2sm1882613wms.6.2024.02.16.02.06.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 02:06:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fec0d1d1-ccb2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708077964; x=1708682764; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=8lWCxDqYfXRaxaQCwA9Hec1KR2+4tL8pkf9Q3DO8XS0=;
        b=jGmVv0NC1e8hnQEedjsvRfDP6vGqRQQjA3OM9F+jKbASvo5/D8Jwi/zlwQbnNTAibb
         SruFvwhpZx91RRKhCMQ+ZnBoVYRxVBohLOX0mLI1HH0sYAixMNdtRNAqjDOT1nncV/8R
         ZsOw5ZwheH10lYjVRXxXDH/8YX61SgKy/vZ0k=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708077964; x=1708682764;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8lWCxDqYfXRaxaQCwA9Hec1KR2+4tL8pkf9Q3DO8XS0=;
        b=LrS3M6poLQgUJDIxCY/xKr0P6hjYybh1BbHxxJwUsW5SVO8xiNXDJFZPlrcVjsYEVg
         ZhLoPlaD9K0ro2s239lUJ6fQqzcrCWr6icN2/mIWBkhVxdLO921YFzoKDF/kSIuQQSFq
         4DaakU0UGqH4x6skrsxgCry04QxU23gJsOnibLIxQkGUqldoovNdsBjwTYngcQyO8Jvb
         CqQhQjuboxfyoN8WlOowzF36l4SN30sxealn/5TbjhiZgqA6E+dqa+tsVhzudR3jJ8gn
         H6s5rAiYB/P3nfSL4GCu5BxH2damSEwIgd5QUafkF4uf27s8KKAiRKN9yoOlPFuh+1CB
         sEoA==
X-Gm-Message-State: AOJu0YxW2TGIylIYcMYJFJnggoz7PnCc2Bsw7Dsz2qVccAskOeUcjLuO
	J/G/mIfcFyqcBktX2MDzag6XYxOi8h5Z5yM9cnKAxANWJsQIHZLkXQGnGZxxnBw=
X-Google-Smtp-Source: AGHT+IGfjGvravXrHcsVPXEsF1dF66g6XrcCb1kO+o4Bd7jg1P6uX1/p8G982uR+h/tsU4504Ga8Zw==
X-Received: by 2002:a05:600c:310e:b0:411:f8e8:16dd with SMTP id g14-20020a05600c310e00b00411f8e816ddmr3272743wmo.0.1708077964201;
        Fri, 16 Feb 2024 02:06:04 -0800 (PST)
Date: Fri, 16 Feb 2024 10:06:03 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Juergen Gross <jgross@suse.com>,
	Viresh Kumar <viresh.kumar@linaro.org>
Subject: Re: [PATCH V3] libxl: Add "grant_usage" parameter for virtio disk
 devices
Message-ID: <657da391-0aab-4e34-8f50-7352b554936f@perard>
References: <20240215150644.2477378-1-olekstysh@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240215150644.2477378-1-olekstysh@gmail.com>

On Thu, Feb 15, 2024 at 05:06:44PM +0200, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> Allow administrators to control whether Xen grant mappings for
> the virtio disk devices should be used. By default (when new
> parameter is not specified), the existing behavior is retained
> (we enable grants if backend-domid != 0).
> 
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
> In addition to "libxl: arm: Add grant_usage parameter for virtio devices"
> https://github.com/xen-project/xen/commit/c14254065ff4826e34f714e1790eab5217368c38
> 
>  V2:
>   - clarify documentation to match the implementation
>   - apply a default value if "grant_usage" is missing the Xenstore
>     in libxl__disk_from_xenstore()
> 
>  V3:
>   - include autogenerated changes to tools/libs/util/libxlu_disk_l.c(h)
>   - remove debug log from libxl__disk_from_xenstore(),
>     correct coding style

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 10:38:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 10:38:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682114.1061148 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ravbP-00037M-LD; Fri, 16 Feb 2024 10:37:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682114.1061148; Fri, 16 Feb 2024 10:37:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ravbP-00037F-IF; Fri, 16 Feb 2024 10:37:55 +0000
Received: by outflank-mailman (input) for mailman id 682114;
 Fri, 16 Feb 2024 10:37:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LvWc=JZ=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1ravbO-000374-EH
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 10:37:54 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6fd6458d-ccb7-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 11:37:52 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33d066f8239so815865f8f.1
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 02:37:52 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 p6-20020adfe606000000b0033cdbe335bcsm1822314wrm.71.2024.02.16.02.37.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 02:37:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fd6458d-ccb7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708079872; x=1708684672; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=dUwJklCk/DnUs+PMaFd7ukwTODbVHMp2YhTBDgA0UUY=;
        b=aOL/SQT0W0dW9hss/bDcMLjm06bkI+p4farZ0gF8l7UC3S0O4D/MIlwL/R0/FifgXw
         L0sCQ7NXc9+/fYBvUKMpUSlmGclAoeWexr24npzJeYEzsLWNCrXFQ3qd/1w+hMBMhwk0
         fD/3644im9G2LUBcXmrrZDh+/GKgaimrX3RCM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708079872; x=1708684672;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dUwJklCk/DnUs+PMaFd7ukwTODbVHMp2YhTBDgA0UUY=;
        b=N6lEfa8x2noiRo3i41uN8FrGmdaX4KjUWlWFl+8mLPjP1jrffV0F6L4Bf28itpbYGT
         eQkGtBaZ5EqPBWoJj1LV/R3wSYXEPt26AOaDSGgjoFrdzp64qRkonDU+2B+eVT8kMj/5
         sfWksjTkGpA7LhIdIkt059R2Q73kOwANkZbRGD0TnBSxMd82OkbrgjvLiyVBqkmijqUP
         AhE4lP2Wy6FDZxTzlTJ1C5r8crBwcFOjlS9oOTwrCrKRlQ+TqMqT4CwCtKqj2uCm3e71
         ZJ6Tyoc8BLP4f4NSSRo7LLBqTRgdWlTU1Sz6Cp7Veb/N3Dbtg5luSMdVTy12CcYvH2+o
         WXEA==
X-Gm-Message-State: AOJu0YyKs37jrU7vcB5ibrMSkByoqB/xSU8++S7KbMc0KhC/xtSLvxwU
	pPojYgEiwfbE0gmojNLQEReLf+bOyri+iDBWuhZWnvC1vhqjFRooQy17RQsNfYY=
X-Google-Smtp-Source: AGHT+IHx1dEAt4yVoJ3kvhj07qaYBSgURra5x50sWKmOOlZcFltTChjoixSw6qWv4/UZErJqGJjgug==
X-Received: by 2002:a5d:4708:0:b0:33c:d1c4:4c3b with SMTP id y8-20020a5d4708000000b0033cd1c44c3bmr3062372wrq.23.1708079871798;
        Fri, 16 Feb 2024 02:37:51 -0800 (PST)
Date: Fri, 16 Feb 2024 10:37:51 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org,
	Marek =?iso-8859-1?Q?Marczykowski-G=F3recki?= <marmarek@invisiblethingslab.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH v2] build: Replace `which` with `command -v`
Message-ID: <a5be85cf-2aba-43d9-9e76-dc5527660d2a@perard>
References: <20240214180039.57527-1-anthony.perard@citrix.com>
 <dca166f1-6b94-46bb-b576-411ef9cce127@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <dca166f1-6b94-46bb-b576-411ef9cce127@citrix.com>

On Wed, Feb 14, 2024 at 06:06:38PM +0000, Andrew Cooper wrote:
> On 14/02/2024 6:00 pm, Anthony PERARD wrote:
> > The `which` command is not standard, may not exist on the build host,
> > or may not behave as expected by the build system. It is recommended
> > to use `command -v` to find out if a command exist and have its path,
> > and it's part of a POSIX shell standard (at least, it seems to be
> > mandatory since IEEE Std 1003.1-2008, but was optional before).
> >
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > Tested-by: Marek Marczykowski-Grecki <marmarek@invisiblethingslab.com>
> 
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> I expect we want fixes tags for c8a8645f1ef (python) and 3b47bcdb6d383
> (figlet). Both my fault it turns out...

Sounds good. I saw this patch as an improvement rather than a bug fix so
I didn't think about fixes tags. It's one less build dependency, so I
guess it's nice to have and backport.

It's common to use `which` to find a path to a binary, and unfortunately
not well known that `command -v` is a more portable alternative. So not
exactly your fault.

It's looks like there's a nice write up about this here:
https://unix.stackexchange.com/questions/85249/why-not-use-which-what-to-use-then/85250#85250

Cheers,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 10:50:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 10:50:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682121.1061159 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ravn3-0005J0-NA; Fri, 16 Feb 2024 10:49:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682121.1061159; Fri, 16 Feb 2024 10:49:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ravn3-0005It-JF; Fri, 16 Feb 2024 10:49:57 +0000
Received: by outflank-mailman (input) for mailman id 682121;
 Fri, 16 Feb 2024 10:49:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ravn2-0005Ih-TL; Fri, 16 Feb 2024 10:49:56 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ravn2-0007hu-JO; Fri, 16 Feb 2024 10:49:56 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ravn2-0000kt-1Q; Fri, 16 Feb 2024 10:49:56 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1ravn2-0003UC-0e; Fri, 16 Feb 2024 10:49:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OdcI15KYFNhH/ddvTGeBe4k07eCp9fbrAQ5DWX2WAMA=; b=EbEasxpY8+99odRy5HrEXTdTYI
	0ipXI8kZEWRLN9PvyHSC2aFPOH28x4coBLIKAutdlU8Yr8jLw/Qg9uLBLp8GVls7QqcyEGeIqRG7n
	iss91yiqna6dv+LB321LnOLs3TByt4nsa9UKiDRdBWyX6QfOqL7sNhy+4NIPBI0CNaqA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184681-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184681: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:guest-start/debian.repeat:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=78398afae10bfb4ab94e8af17b7ed58510a57d96
X-Osstest-Versions-That:
    xen=c4d6802d33f962a306405a20f4dccdc287990d07
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 16 Feb 2024 10:49:56 +0000

flight 184681 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184681/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check fail blocked in 184677
 test-armhf-armhf-libvirt-raw 17 guest-start/debian.repeat fail blocked in 184677
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184677
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184677
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184677
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184677
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184677
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184677
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184677
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184677
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184677
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184677
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184677
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  78398afae10bfb4ab94e8af17b7ed58510a57d96
baseline version:
 xen                  c4d6802d33f962a306405a20f4dccdc287990d07

Last test of basis   184677  2024-02-15 16:38:53 Z    0 days
Testing same since   184681  2024-02-16 01:39:56 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Juergen Gross <jgross@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 fail    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   c4d6802d33..78398afae1  78398afae10bfb4ab94e8af17b7ed58510a57d96 -> master


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 10:50:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 10:50:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682126.1061169 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ravnv-0006e7-0G; Fri, 16 Feb 2024 10:50:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682126.1061169; Fri, 16 Feb 2024 10:50:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ravnu-0006e0-Sg; Fri, 16 Feb 2024 10:50:50 +0000
Received: by outflank-mailman (input) for mailman id 682126;
 Fri, 16 Feb 2024 10:50:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EzDv=JZ=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1ravnu-0006cp-9G
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 10:50:50 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3f112af2-ccb9-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 11:50:49 +0100 (CET)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a3dfb2e03e7so6644266b.3
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 02:50:49 -0800 (PST)
Received: from localhost.localdomain (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ne8-20020a1709077b8800b00a3d7bcfb9a1sm1433960ejc.128.2024.02.16.02.50.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 02:50:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3f112af2-ccb9-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708080649; x=1708685449; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=cD+nRRn5234Rjd+cd7IoO2ec6BbGHAkowqpgkrzy5XA=;
        b=iH5vs59wCTgMByxeGjJ2AqG4Vfj9si4BPsFytm88B3DE3S5hC2niScNgYK/xmyngGT
         kGZAmZMeVlZoz1TM6YjPTMN8nSp294AMZcnSHRumxfAF05bvTRgOU4GSVH8rriBhILVH
         nZM8H7CQ3FU+7Xb4/f/JxTB4rd19OQLJ2/CZQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708080649; x=1708685449;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=cD+nRRn5234Rjd+cd7IoO2ec6BbGHAkowqpgkrzy5XA=;
        b=jaenvzPlb3EU5FdrAEPCq/RQM/Q73shIMRrxhd2qcV+Lx/0UD1Lyu3RCC11HaEdq1y
         NAs1vLaoljHOOSMQ3ER9XfEtpScd8G65+Z94SlCf2JllTypbUI4LprSvMHc8feUJFTnf
         XzbjARS5dkzjof+ZfM6H96c7VBzYmber66wbJh1oWUyiRgNmOwxP87IYf5yTNiEOrdi2
         TMTCOHvDhFjFyRldg0us6mtxrZlkSWE1XT3H96W8a+BqIVgVALc1x0LgGI+5637JblX+
         /DHttZy86wCaRU6y5h28oTwqTFL/wtFMZAlrt+w1a62/U9+/OdDZjuaiHgwLrgVUbQr6
         CYEw==
X-Gm-Message-State: AOJu0YwNEWylU1vVOKo0ekigFOfH7W9zJFW7ZnRsW+1V/JKDLn7xA7Ij
	NB7A6iyNAEHd8ZKCFYPMOfxI0MMenIJ3F+fhTwkTy9W4npJPDMukG+eP362iba4I9GFb4646Kd9
	lpn4isg==
X-Google-Smtp-Source: AGHT+IHfXNgrj41aCXJ+QeCVM4Q1TrEtxbYkdGXnPKuwRqTi4lpf3U5XaH+rziCS1kmzkbp1o53D4g==
X-Received: by 2002:a17:906:ad96:b0:a3d:eb78:c552 with SMTP id la22-20020a170906ad9600b00a3deb78c552mr658238ejb.13.1708080648723;
        Fri, 16 Feb 2024 02:50:48 -0800 (PST)
From: Frediano Ziglio <frediano.ziglio@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: Frediano Ziglio <frediano.ziglio@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3] Reduce assembly code size of exception entry points
Date: Fri, 16 Feb 2024 10:50:28 +0000
Message-Id: <20240216105028.9517-1-frediano.ziglio@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

On many entries we push 8-bytes zero and exception constants are
small so we can just write a single byte saving 3 bytes for
instruction.
With ENDBR64 this reduces the size of many entry points from 32 to
16 bytes (due to alignment).
The push and the mov are overlapping stores either way.  Swapping
between movl and movb will make no difference at all on performance.
Similar code is already used in autogen_stubs.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
--
v3:
- add other missing entries;
- reduce code when TRAP_syscall is used;
- improved commit message.

v2:
- added missing entry points;
- add mention to autogen_stubs code, as suggested.
---
 xen/arch/x86/x86_64/entry.S | 66 ++++++++++++++++++++++---------------
 1 file changed, 40 insertions(+), 26 deletions(-)

diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index ecdd6e5b47..f07e7ed3ac 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -22,6 +22,14 @@
 #endif
 .endm
 
+.macro BUILD_BUG_ON condstr cond:vararg
+        .if \cond
+        .error "Condition \condstr not satisfied"
+        .endif
+.endm
+/* preprocessor macro to make error message more user friendly */
+#define BUILD_BUG_ON(cond) BUILD_BUG_ON #cond cond
+
 #ifdef CONFIG_PV
 /* %rbx: struct vcpu */
 FUNC_LOCAL(switch_to_kernel)
@@ -187,7 +195,8 @@ FUNC_LOCAL(restore_all_guest)
         SPEC_CTRL_EXIT_TO_PV    /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
 
         RESTORE_ALL
-        testw $TRAP_syscall,4(%rsp)
+        BUILD_BUG_ON(TRAP_syscall & 0xff)
+        testb $TRAP_syscall >> 8,4+1(%rsp)
         jz    iret_exit_to_guest
 
         movq  24(%rsp),%r11           # RFLAGS
@@ -254,7 +263,8 @@ FUNC(lstar_enter)
         pushq $FLAT_KERNEL_CS64
         pushq %rcx
         pushq $0
-        movl  $TRAP_syscall, 4(%rsp)
+        BUILD_BUG_ON(TRAP_syscall & 0xff)
+        movb  $TRAP_syscall >> 8, 4+1(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -292,7 +302,8 @@ FUNC(cstar_enter)
         pushq $FLAT_USER_CS32
         pushq %rcx
         pushq $0
-        movl  $TRAP_syscall, 4(%rsp)
+        BUILD_BUG_ON(TRAP_syscall & 0xff)
+        movb  $TRAP_syscall >> 8, 4+1(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -334,7 +345,8 @@ LABEL(sysenter_eflags_saved, 0)
         pushq $3 /* ring 3 null cs */
         pushq $0 /* null rip */
         pushq $0
-        movl  $TRAP_syscall, 4(%rsp)
+        BUILD_BUG_ON(TRAP_syscall & 0xff)
+        movb  $TRAP_syscall >> 8, 4+1(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -389,7 +401,7 @@ FUNC(entry_int80)
         ENDBR64
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         pushq $0
-        movl  $0x80, 4(%rsp)
+        movb  $0x80, 4(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -561,7 +573,8 @@ __UNLIKELY_END(create_bounce_frame_bad_sp)
         /* Rewrite our stack frame and return to guest-OS mode. */
         /* IA32 Ref. Vol. 3: TF, VM, RF and NT flags are cleared on trap. */
         /* Also clear AC: alignment checks shouldn't trigger in kernel mode. */
-        orl   $TRAP_syscall,UREGS_entry_vector+8(%rsp)
+        BUILD_BUG_ON(TRAP_syscall & 0xff)
+        orb   $TRAP_syscall >> 8,UREGS_entry_vector+8+1(%rsp)
         andl  $~(X86_EFLAGS_AC|X86_EFLAGS_VM|X86_EFLAGS_RF|\
                  X86_EFLAGS_NT|X86_EFLAGS_TF),UREGS_eflags+8(%rsp)
         movq  $FLAT_KERNEL_SS,UREGS_ss+8(%rsp)
@@ -653,7 +666,7 @@ END(ret_from_intr)
         .section .init.text, "ax", @progbits
 FUNC(early_page_fault)
         ENDBR64
-        movl  $X86_EXC_PF, 4(%rsp)
+        movb  $X86_EXC_PF, 4(%rsp)
         SAVE_ALL
         movq  %rsp, %rdi
         call  do_early_page_fault
@@ -722,7 +735,7 @@ END(common_interrupt)
 
 FUNC(entry_PF)
         ENDBR64
-        movl  $X86_EXC_PF, 4(%rsp)
+        movb  $X86_EXC_PF, 4(%rsp)
 END(entry_PF)
 /* No special register assumptions. */
 FUNC(handle_exception, 0)
@@ -898,105 +911,106 @@ END(handle_exception)
 FUNC(entry_DE)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_DE, 4(%rsp)
+        /* no need to update exception type, already 0 */
+        BUILD_BUG_ON(X86_EXC_DE)
         jmp   handle_exception
 END(entry_DE)
 
 FUNC(entry_MF)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_MF, 4(%rsp)
+        movb  $X86_EXC_MF, 4(%rsp)
         jmp   handle_exception
 END(entry_MF)
 
 FUNC(entry_XM)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_XM, 4(%rsp)
+        movb  $X86_EXC_XM, 4(%rsp)
         jmp   handle_exception
 END(entry_XM)
 
 FUNC(entry_NM)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_NM, 4(%rsp)
+        movb  $X86_EXC_NM, 4(%rsp)
         jmp   handle_exception
 END(entry_NM)
 
 FUNC(entry_DB)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_DB, 4(%rsp)
+        movb  $X86_EXC_DB, 4(%rsp)
         jmp   handle_ist_exception
 END(entry_DB)
 
 FUNC(entry_BP)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_BP, 4(%rsp)
+        movb  $X86_EXC_BP, 4(%rsp)
         jmp   handle_exception
 END(entry_BP)
 
 FUNC(entry_OF)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_OF, 4(%rsp)
+        movb  $X86_EXC_OF, 4(%rsp)
         jmp   handle_exception
 END(entry_OF)
 
 FUNC(entry_BR)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_BR, 4(%rsp)
+        movb  $X86_EXC_BR, 4(%rsp)
         jmp   handle_exception
 END(entry_BR)
 
 FUNC(entry_UD)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_UD, 4(%rsp)
+        movb  $X86_EXC_UD, 4(%rsp)
         jmp   handle_exception
 END(entry_UD)
 
 FUNC(entry_TS)
         ENDBR64
-        movl  $X86_EXC_TS, 4(%rsp)
+        movb  $X86_EXC_TS, 4(%rsp)
         jmp   handle_exception
 END(entry_TS)
 
 FUNC(entry_NP)
         ENDBR64
-        movl  $X86_EXC_NP, 4(%rsp)
+        movb  $X86_EXC_NP, 4(%rsp)
         jmp   handle_exception
 END(entry_NP)
 
 FUNC(entry_SS)
         ENDBR64
-        movl  $X86_EXC_SS, 4(%rsp)
+        movb  $X86_EXC_SS, 4(%rsp)
         jmp   handle_exception
 END(entry_SS)
 
 FUNC(entry_GP)
         ENDBR64
-        movl  $X86_EXC_GP, 4(%rsp)
+        movb  $X86_EXC_GP, 4(%rsp)
         jmp   handle_exception
 END(entry_GP)
 
 FUNC(entry_AC)
         ENDBR64
-        movl  $X86_EXC_AC, 4(%rsp)
+        movb  $X86_EXC_AC, 4(%rsp)
         jmp   handle_exception
 END(entry_AC)
 
 FUNC(entry_CP)
         ENDBR64
-        movl  $X86_EXC_CP, 4(%rsp)
+        movb  $X86_EXC_CP, 4(%rsp)
         jmp   handle_exception
 END(entry_CP)
 
 FUNC(entry_DF)
         ENDBR64
-        movl  $X86_EXC_DF, 4(%rsp)
+        movb  $X86_EXC_DF, 4(%rsp)
         /* Set AC to reduce chance of further SMAP faults */
         ALTERNATIVE "", stac, X86_FEATURE_XEN_SMAP
         SAVE_ALL
@@ -1022,7 +1036,7 @@ END(entry_DF)
 FUNC(entry_NMI)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_NMI, 4(%rsp)
+        movb  $X86_EXC_NMI, 4(%rsp)
 END(entry_NMI)
 
 FUNC(handle_ist_exception)
@@ -1158,7 +1172,7 @@ END(handle_ist_exception)
 FUNC(entry_MC)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_MC, 4(%rsp)
+        movb  $X86_EXC_MC, 4(%rsp)
         jmp   handle_ist_exception
 END(entry_MC)
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 10:52:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 10:52:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682131.1061179 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ravp4-0007Eu-D8; Fri, 16 Feb 2024 10:52:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682131.1061179; Fri, 16 Feb 2024 10:52:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ravp4-0007En-9X; Fri, 16 Feb 2024 10:52:02 +0000
Received: by outflank-mailman (input) for mailman id 682131;
 Fri, 16 Feb 2024 10:52:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=dcAy=JZ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1ravp3-0007Ef-Hv
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 10:52:01 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 68da02fc-ccb9-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 11:51:59 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-51296ca7f78so449437e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 02:51:59 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 k12-20020ac8074c000000b0042ddde7c415sm489238qth.9.2024.02.16.02.51.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 02:51:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68da02fc-ccb9-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708080719; x=1708685519; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=8n0bPwYe8enx2Q8FWNVK8TZWaqPo1Pct3EbywqUPyps=;
        b=RgQrsGNsSW/KPRXd7AWYyjB3izHN0+SL6AVr4SzegD5goB1b0F6RpCTtGz6Xh40qwZ
         L56A4/JUo/Wh4mtbJmHKZgLf9H/+zTGrUrYc5voaV+97+sBoY/P9lZjQcBhQ96ZG+NOK
         mSe/tYNlzC4gih9EwK6UtMUsNRzMYDdnwCwps=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708080719; x=1708685519;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8n0bPwYe8enx2Q8FWNVK8TZWaqPo1Pct3EbywqUPyps=;
        b=D3WnUFRiKzk87mljcKT8eXQI4K/N8UUxYInsiD7N4119EvmNK9AqotP5bKkUt55fvq
         VEFD4jhXB9W0Utg2uvGuOpe8ajWBNv/Dm5XXhpFiDRp5Z9enp+kGTmapHgtNaukjdlrX
         u5MZrPHm582maBdraD1dUJCyruAGlVR4TY6PFHfL+7zSsbdm27C806DjISGN/O/rKM59
         n11yxsuFA5qOyTlTJsRiZ8KnhBFlW2AXWEtLGSkDY59hV/nwOqwrVszxdWyPzUHe+oeF
         EJCWxD/+qClqvr5hnKMlfjn12XNBNaXW7F3bhA/QXXcX3tbU/+NUiNim/EQnQxae+DqG
         psVA==
X-Forwarded-Encrypted: i=1; AJvYcCWDsiwN4wDG49tReAHlPhatz6+feyD3SLHkYGbtkMIhWuCl1Cs0ggtlUxe0uHPpHsuBuj7gYcCI1SObvaaxcKzHqEC6JIN15WERvtwuhKg=
X-Gm-Message-State: AOJu0YzO0Rj9zpC+Z6xiXveq4N0THH7YaWzD2MsIX/Pq/nSRCPmEbDHg
	IuTefCAgnTELc1Sp2lsuPXBJA1Jn+WfD8C+7ZglxGSJHOOhAFDjysyl/OkX2PjY=
X-Google-Smtp-Source: AGHT+IH4UMmHIlo0Wqq5kJ2+wcYvCon1+GHSAEukwTVFy0gv8BBTYGzAhWz+6Gf5ca8vxM1jNgJzDg==
X-Received: by 2002:a05:6512:b88:b0:512:8de1:2714 with SMTP id b8-20020a0565120b8800b005128de12714mr2306610lfv.24.1708080719062;
        Fri, 16 Feb 2024 02:51:59 -0800 (PST)
Date: Fri, 16 Feb 2024 11:51:56 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Anthony Perard <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Message-ID: <Zc8-TNOMG7ljNCj1@macbook>
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com>
 <Zc3nXpUOlnIHEfsl@macbook>
 <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com>
 <Zc3v20RKMssbaDsl@macbook>
 <a48cbad6-701d-4077-9044-4205b932a7f3@suse.com>
 <Zc428VMDoYnPw1zo@macbook>
 <d545cc6c-d213-43da-af31-1768af32aba0@suse.com>
 <Zc5Io3dkAlGSt3on@macbook>
 <2fe68b26-87c3-43bf-ba11-f261c81c6373@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <2fe68b26-87c3-43bf-ba11-f261c81c6373@suse.com>

On Fri, Feb 16, 2024 at 11:04:46AM +0100, Jan Beulich wrote:
> On 15.02.2024 18:23, Roger Pau Monné wrote:
> > On Thu, Feb 15, 2024 at 05:22:00PM +0100, Jan Beulich wrote:
> >> On 15.02.2024 17:08, Roger Pau Monné wrote:
> >>> On Thu, Feb 15, 2024 at 02:02:41PM +0100, Jan Beulich wrote:
> >>>> --- a/xen/Rules.mk
> >>>> +++ b/xen/Rules.mk
> >>>> @@ -15,7 +15,11 @@ srcdir := $(srctree)/$(src)
> >>>>  PHONY := __build
> >>>>  __build:
> >>>>  
> >>>> --include $(objtree)/include/config/auto.conf
> >>>> +ifneq ($(obj),tools)
> >>>> +ifneq ($(obj),tools/kconfig)
> >>>> +include $(objtree)/include/config/auto.conf
> >>>> +endif
> >>>> +endif
> >>>
> >>> Trying to understand this, I assume it's to avoid an infinite
> >>> dependency loop that generating include/config/auto.conf requires some
> >>> tools that are build using xen/Rules.mk?
> >>
> >> The file has dependencies only in xen/Makefile. This is about the
> >> file simply not being there when initially building. Perhaps the
> >> patch description helps that I've written in the meantime:
> >>
> >> "Because of using "-include", failure to (re)build auto.conf (with
> >>  auto.conf.cmd produced as a secondary target) won't stop make from
> >>  continuing the build. Arrange for it being possible to drop the - from
> >>  Rules.mk, requiring that the include be skipped for tools-only targets.
> > 
> > Wouldn't it be more reliable if we skipped the include for any paths
> > in $(obj) that start with 'tools', rather than hardcoding 'tools' and
> > 'tools/kconfig'?
> 
> I was first meaning to do so, but the expression would end up more
> complex than I'd like (for it needing to be an exact match of "tools"
> and a prefix match of "tools/"). Thinking of it,
> 
> ifneq ($(obj),tools)
> ifneq ($(patsubst tools/%,$(obj)),)
> 
> might do (and not be as complex as I first thought, when intending to
> put all in a single "if").

Would something like the rune below work?

ifneq ($(word 1, $(subst /, ,$(obj))),tools)

That should allow to have a single condition, and should match both
'tools' and 'tools/*'

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 11:03:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 11:03:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682154.1061241 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ravzw-0001Tq-0D; Fri, 16 Feb 2024 11:03:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682154.1061241; Fri, 16 Feb 2024 11:03:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ravzv-0001TZ-Tc; Fri, 16 Feb 2024 11:03:15 +0000
Received: by outflank-mailman (input) for mailman id 682154;
 Fri, 16 Feb 2024 11:03:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=acXU=JZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1ravzu-0001Sd-N2
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 11:03:14 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fa0d7ace-ccba-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 12:03:12 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-51299d29e43so140550e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 03:03:12 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 f28-20020a19381c000000b0051170288d38sm400327lfa.283.2024.02.16.03.03.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 03:03:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa0d7ace-ccba-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708081392; x=1708686192; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=fLS8S8ekIxJIKVao0IBPUqeI/r1LHe8DxXuJUiEO4iI=;
        b=OcL+W/I7uQg3AMCQATNYwiI4VNY/Q8qqxM2K2zorprWpJbV1lfXHVxFk+C31720NZQ
         mwDkqThMy6+WT3pdr8FPnNgiPUIR6Y8vRlMrmo2gr4Q5QA1gp3/KQDBqh7nVhT5jG6LS
         SnyniRYwHZkSXvecS8NMNk82loI0ReOsqsJjdsg+ZjCuRQoUDiJbI96PC+CzIfjVnYUH
         oiZV3CbXWEKX0j/xrwEfhrVxwngvf4BAhk+bVhl0jNSqOgJotL1Lmfr52nAa2yFoh8wO
         txKQbEaIQhPMQPYSBVzNRw6ho/0YWwAaOt1geKPmau3phVH1xpJpQZo97cu16KfAcBk4
         Q1jw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708081392; x=1708686192;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=fLS8S8ekIxJIKVao0IBPUqeI/r1LHe8DxXuJUiEO4iI=;
        b=RHfRMKcHsdn4TG4lWJoUqxu6r9eDqncacenC+MBhCSrAiPURAx563imnXd/ext/3nl
         U/XytDl4bz7WxxnTrBTZl984EtmMvWDSVygTf8NRJchNgXM6UcoLSGnFSsnFWRNqyi66
         Wy6S/UU958FsL8HwU9gJ4Nk0iEsYDlE/diY9/mUN7A1j9+T3LMqn0O4/DM/bj8neHQi1
         h37pGbzugF0tik05JVkuctGQuMK1/NN4e11/qPgPl0LP9xEH808UUn/2kTsnK8zeHlYB
         SioKfFm9kYc/m433bUJ2NqsnvP6HwRn4jNsQV/YWp3I9I5lNxSIrjm+yV1mNjX4nhOuS
         BoSw==
X-Forwarded-Encrypted: i=1; AJvYcCVc9QkzuBDBy6w4xJuR91NrmHd8AOeIAA0fVP5c93u6emeN7R5ZlAWnUzka/oc9j3oN1E3KmcgjRSG/Zfpp3wY/aKgnYiEZ1dg8dTUbPHI=
X-Gm-Message-State: AOJu0YxsTA9vp9F93gHR3HGlDlPg5SFz8sEUMAfw5xvwlunkKm3C/Akh
	vNLuMpI16NCd6kxSv1e3O7ChaFUt+pfl3JVfWCNuGgm7RUrETQtu
X-Google-Smtp-Source: AGHT+IFIALv/eSMii5RtwMGMIpGeh7ok53EHmrDx2uBdkR1Ie+RkP61fhye7sgjVo/975dvA9cI77g==
X-Received: by 2002:a19:2d43:0:b0:511:7b37:1d26 with SMTP id t3-20020a192d43000000b005117b371d26mr3599033lft.13.1708081391896;
        Fri, 16 Feb 2024 03:03:11 -0800 (PST)
Message-ID: <6339742ea1a26c9808b108a1282e521dcf0fc9d0.camel@gmail.com>
Subject: Re: [PATCH v4 26/30] xen/riscv: add minimal stuff to mm.h to build
 full Xen
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Fri, 16 Feb 2024 12:03:10 +0100
In-Reply-To: <6f5e7e21-067e-4323-88bc-f9b5975e95c5@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <369fe9e80d11c5e499810a6819fc1c282ce60628.1707146506.git.oleksii.kurochko@gmail.com>
	 <6f5e7e21-067e-4323-88bc-f9b5975e95c5@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

>=20
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } free;
> > +=C2=A0=C2=A0=C2=A0 } u;
> > +
> > +=C2=A0=C2=A0=C2=A0 union {
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Page is in use, but not =
as a shadow. */
>=20
> I'm also pretty sure I asked before what shadow this comment alludes
> to.
I missed your request about 'shadow' before.

The comment arrived from Arm.

I tried to find out the answer by investigation how 'inuse' is used,
and, unfortunately, I couldn't find an answer what 'shadow' alludes to.

>=20
> > +/*
> > + * Common code requires get_page_type and put_page_type.
> > + * We don't care about typecounts so we just do the minimum to
> > make it
> > + * happy.
> > + */
> > +static inline int get_page_type(struct page_info *page, unsigned
> > long type)
> > +{
> > +=C2=A0=C2=A0=C2=A0 return 1;
> > +}
> > +
> > +static inline void put_page_type(struct page_info *page)
> > +{
> > +}
> > +
> > +static inline void put_page_and_type(struct page_info *page)
> > +{
> > +=C2=A0=C2=A0=C2=A0 put_page_type(page);
> > +=C2=A0=C2=A0=C2=A0 put_page(page);
> > +}
> > +
> > +/*
> > + * RISC-V does not have an M2P, but common code expects a handful
> > of
> > + * M2P-related defines and functions. Provide dummy versions of
> > these.
> > + */
> > +#define INVALID_M2P_ENTRY=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (~=
0UL)
> > +#define SHARED_M2P_ENTRY=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 (~0UL - 1UL)
> > +#define SHARED_M2P(_e)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 ((_e) =3D=3D SHARED_M2P_ENTRY)
> > +
> > +#define set_gpfn_from_mfn(mfn, pfn) do { (void)(mfn), (void)(pfn);
> > } while (0)
> > +#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
> > +
> > +#define PDX_GROUP_SHIFT (16 + 5)
>=20
> Where are these magic numbers coming from? None of the other three
> architectures use literal numbers here, thus making clear what
> values are actually meant. If you can't use suitable constants,
> please add a comment.
This numbers are incorrect for RISC-V, it should be 12 + 9 ( PAGE_SHIFT
+ VPN_BITS ).
I did some comparision of how some macros are defined in PPC and missed
to update that.

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 11:16:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 11:16:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682161.1061250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rawCT-0003Uy-0O; Fri, 16 Feb 2024 11:16:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682161.1061250; Fri, 16 Feb 2024 11:16:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rawCS-0003Ur-U7; Fri, 16 Feb 2024 11:16:12 +0000
Received: by outflank-mailman (input) for mailman id 682161;
 Fri, 16 Feb 2024 11:16:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=acXU=JZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rawCS-0003Ul-5e
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 11:16:12 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ca23784c-ccbc-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 12:16:11 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-5114fa38434so2220582e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 03:16:11 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 d12-20020a19384c000000b005119901e5e2sm587334lfj.53.2024.02.16.03.16.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 03:16:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ca23784c-ccbc-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708082171; x=1708686971; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=qFTJD7ZWQNAYy7lLma5emZeZTHLDT+Unm5MY/VjMcgE=;
        b=USp0LjxVTEkSoKFwRmDTrdUqFhIf857HKNuLyCPiJKDS/qaxvUiKlNAv8+xELckOEq
         zC9mteHCKsbZVNnA9ccfzc7R+mn5rtf7yblHv+EjgTEaQVkyRAN6iTKlaDfauEvTzL3M
         nQDtXpl3LmVxqyXo/2hUdjLDamXRb/q0tMOTA60FZStB5mw9yEQWBt2ncfSM4B2R6R6v
         28GzWY/C8vYFauB3yRFtOBB5DSimgeP8ohGjpXACVvLdHILwnx6ybMmHR6PQtoQJpI5J
         BJazetKaEYlae6VmGc+GvHwFD8rkxofijrPMx5QV3UsBe9Zwig+6IsczXrs90FAjiWfo
         eyZg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708082171; x=1708686971;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=qFTJD7ZWQNAYy7lLma5emZeZTHLDT+Unm5MY/VjMcgE=;
        b=ZAaYMH91bVnfSsZVxey3yAXw6VioEnp1BdlbGMBkrwuYKFfcIAvkE6InzITPdBgPmW
         1eWUVBucwvKGu+onMGPS8IwMdbNnTuuv/Fa0fkTn5sIoaGABFJ6fEztDcke47NOLhkVs
         tW48n7qln7ORLuyPCfR5qRJlfsUUFO1APlCFtIThlR5/IZEnagxL1Fz58xxQWowZkpSL
         kDWMc7qDAvnHy/kwgo5Iw7jBp65RzDKXRurkuM8S2dnYdGrERJBU+iYe0yz6yk6R/nov
         axzIw/zPQJQyE4Cly/uAqDJGIsrDksPW5PyckmT8NkX8lfHvUI2LW0mUgAyCKDZt8yly
         hAsw==
X-Forwarded-Encrypted: i=1; AJvYcCXAnqVOnli2pZMBr358OcMAlt9WD3F3Rl/7MVhOZwPHjiyY+1JWD/kSrn4neD/2/rKixvQ1yRZOBLT2r8d5l13eKAJmOfD1xWbSGvqOgNA=
X-Gm-Message-State: AOJu0Yx0OcJoiHIs2TGD+wI/MhERjM6QWth2DrNSp5i68QDgeeOaLte0
	5EeB1Vp068xmy7GvDcbyTZsA7ptcVQDpZb3jKQS3l1w9NBMgLAk0
X-Google-Smtp-Source: AGHT+IGHBktLvr4FvayL3+anX4AUKAvl3jIyV4Z4LgOi22yWn6IgWj6q+8WFwc5V6OmioQvv35X4Mg==
X-Received: by 2002:a05:6512:3e23:b0:512:8d1f:de39 with SMTP id i35-20020a0565123e2300b005128d1fde39mr3706558lfv.27.1708082170445;
        Fri, 16 Feb 2024 03:16:10 -0800 (PST)
Message-ID: <0bafef389b30251bc9898bb61604aa3efaabe48c.camel@gmail.com>
Subject: Re: [PATCH v4 25/30] xen/riscv: add minimal stuff to processor.h to
 build full Xen
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Alistair Francis
 <alistair.francis@wdc.com>, Bob Eshleman <bobbyeshleman@gmail.com>, Connor
 Davis <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
Date: Fri, 16 Feb 2024 12:16:09 +0100
In-Reply-To: <2f1e4d2d-5b33-47ff-912b-c4693744d0e9@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <9efbc232f64b6192cf83f865b8987846fe082720.1707146506.git.oleksii.kurochko@gmail.com>
	 <6be5102a-624c-463a-9821-c618d110ce7a@suse.com>
	 <095b8031eaaa5324cdae9fee75f9521a795feb46.camel@gmail.com>
	 <2f1e4d2d-5b33-47ff-912b-c4693744d0e9@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Thu, 2024-02-15 at 17:43 +0100, Jan Beulich wrote:
> On 15.02.2024 17:38, Oleksii wrote:
> > On Tue, 2024-02-13 at 14:33 +0100, Jan Beulich wrote:
> > > On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > > > +	depends on LLD_VERSION >=3D 150000 || LD_VERSION >=3D
> > > > 23600
> > >=20
> > > What's the linker dependency here? Depending on the answer I
> > > might
> > > further
> > > ask why "TOOLCHAIN" when elsewhere we use CC_HAS_ or HAS_CC_ or
> > > HAS_AS_.
> > I missed to introduce {L}LLD_VERSION config. It should output from
> > the
> > command:
> > =C2=A0 riscv64-linux-gnu-ld --version
>=20
> Doesn't answer my question though where the linker version matters
> here.
Then I misinterpreted your initial question.
Could you please provide further clarification or rephrase it for
better understanding?

Probably, your question was about why linker dependency is needed here,
then
it is not sufficient to check if a toolchain supports a particular =20
extension without checking if the linker supports that extension  =20
too.
For example, Clang 15 supports Zihintpause but GNU bintutils
2.35.2 does not, leading build errors like so:
   =20
   riscv64-linux-gnu-ld: -march=3Drv64i_zihintpause2p0: Invalid or
   unknown z ISA extension: 'zihintpause'


~ Oleksii


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 11:45:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 11:45:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682168.1061260 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raweM-00081c-36; Fri, 16 Feb 2024 11:45:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682168.1061260; Fri, 16 Feb 2024 11:45:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raweM-00081V-0V; Fri, 16 Feb 2024 11:45:02 +0000
Received: by outflank-mailman (input) for mailman id 682168;
 Fri, 16 Feb 2024 11:45:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=dcAy=JZ=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1raweL-00081P-Dc
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 11:45:01 +0000
Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com
 [2a00:1450:4864:20::436])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d0650744-ccc0-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 12:44:59 +0100 (CET)
Received: by mail-wr1-x436.google.com with SMTP id
 ffacd0b85a97d-3394bec856fso1806640f8f.0
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 03:44:59 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 n22-20020ac86756000000b0042c7a76d267sm1455048qtp.36.2024.02.16.03.44.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 03:44:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d0650744-ccc0-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708083899; x=1708688699; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=EqReswirfy9NFpG8T6pUC1gDvTEd0yiHeelX55cUq6g=;
        b=NXL07pSoAD7TqafC5zJIjBbFkvfSfAjb/tI7ZMm5ud1lq2GEeyVB2h2nb9aPqrhUP9
         kbBrHPhirplt0KD3+6M1+oV7GOcit1j4gGzb7tFdi4DC/N9nM/6yk6lI3jhuFM8WJ+ZL
         ajp3J1cDbHX2Rn1OGCQ9RT9YRS4j7Tul03J+g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708083899; x=1708688699;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=EqReswirfy9NFpG8T6pUC1gDvTEd0yiHeelX55cUq6g=;
        b=G6zCgEFhW9uS/1C1kZ1LepOm6aK0BfVhyJwzWWb29dEwyl6ytF+72UuTrVJ9M5Xd+Q
         SllHNKho95xSvbcKdNq7FWKAw3KdR/kERMi12mzg64YqearFdtUi6q1q8IqJd0ulxuuK
         naAhCXYppSXlR3rveGe9kUJgzxv2Rdr4Q7JPvJCfEFP3RlvE4SX4lEyMEUip0cwBsOuj
         2WL28dF9do/tY6/AIvznTvRKpR1J+ULzB/2zVi494STVEOt8ak4YQZh+8hk2Y6XldPVE
         3YyaoOLpkDLrZ+snrf9Aucs6uByN54uvgmGNd9i9v7dqHFstRWc0Hk0CDx6s/DR+FC5g
         0xIA==
X-Gm-Message-State: AOJu0Yzr3dRD3dIVKK+wkOfn4FFMfrZ/9Z2iGyPYE+p6UyvwydCoylH0
	ysVd3+7CLaJtSe6XCXO7130dQaysdti9iKz/1YQIW3/KXjPNYbLh/AfCpMOtwrM=
X-Google-Smtp-Source: AGHT+IFQn7SHyIzL2g0AB63oqNeJvuP1Csiyus5Qw88UebQaaQ8XfzwM8wd/CENBbmuaId9butHWcQ==
X-Received: by 2002:adf:cd06:0:b0:33c:deef:ac2d with SMTP id w6-20020adfcd06000000b0033cdeefac2dmr6586910wrm.31.1708083899311;
        Fri, 16 Feb 2024 03:44:59 -0800 (PST)
Date: Fri, 16 Feb 2024 12:44:56 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: Re: [PATCH v13.1 01/14] vpci: use per-domain PCI lock to protect
 vpci structure
Message-ID: <Zc9KuCeoOciUdqTN@macbook>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <20240215203001.1816811-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240215203001.1816811-1-stewart.hildebrand@amd.com>

On Thu, Feb 15, 2024 at 03:30:00PM -0500, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Use the per-domain PCI read/write lock to protect the presence of the
> pci device vpci field. This lock can be used (and in a few cases is used
> right away) so that vpci removal can be performed while holding the lock
> in write mode. Previously such removal could race with vpci_read for
> example.
> 
> When taking both d->pci_lock and pdev->vpci->lock, they should be
> taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
> possible deadlock situations.
> 
> 1. Per-domain's pci_lock is used to protect pdev->vpci structure
> from being removed.
> 
> 2. Writing the command register and ROM BAR register may trigger
> modify_bars to run, which in turn may access multiple pdevs while
> checking for the existing BAR's overlap. The overlapping check, if
> done under the read lock, requires vpci->lock to be acquired on both
> devices being compared, which may produce a deadlock. It is not
> possible to upgrade read lock to write lock in such a case. So, in
> order to prevent the deadlock, use d->pci_lock in write mode instead.
> 
> All other code, which doesn't lead to pdev->vpci destruction and does
> not access multiple pdevs at the same time, can still use a
> combination of the read lock and pdev->vpci->lock.
> 
> 3. Drop const qualifier where the new rwlock is used and this is
> appropriate.
> 
> 4. Do not call process_pending_softirqs with any locks held. For that
> unlock prior the call and re-acquire the locks after. After
> re-acquiring the lock there is no need to check if pdev->vpci exists:
>  - in apply_map because of the context it is called (no race condition
>    possible)
>  - for MSI/MSI-X debug code because it is called at the end of
>    pdev->vpci access and no further access to pdev->vpci is made
> 
> 5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
> while accessing pdevs in vpci code.
> 
> 6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
> do not go away. The vPCI functions call several MSI-related functions
> which already have existing non-vPCI callers. Change those MSI-related
> functions to allow using either pcidevs_lock() or d->pci_lock for
> ensuring pdevs do not go away. Holding d->pci_lock in read mode is
> sufficient. Note that this pdev protection mechanism does not protect
> other state or critical sections. These MSI-related functions already
> have other race condition and state protection mechanims (e.g.
> d->event_lock and msixtbl RCU), so we deduce that the use of the global
> pcidevs_lock() is to ensure that pdevs do not go away.
> 
> 7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
> that pdevs do not go away. The purpose of this wrapper is to aid
> readability and document the intent of the pdev protection mechanism.
> 
> 8. When possible, the existing non-vPCI callers of these MSI-related
> functions haven't been switched to use the newly introduced per-domain
> pci_lock, and will continue to use the global pcidevs_lock(). This is
> done to reduce the risk of the new locking scheme introducing
> regressions. Those users will be adjusted in due time. One exception
> is where the pcidevs_lock() in allocate_and_map_msi_pirq() is moved to
> the caller, physdev_map_pirq(): this instance is switched to
> read_lock(&d->pci_lock) right away.
> 
> Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

A couple of questions and the pdev_list_is_read_locked() needs a small
adjustment.

> @@ -895,6 +891,14 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
>  {
>      unsigned int i;
>  
> +    /*
> +     * Assert that d->pdev_list doesn't change. pdev_list_is_read_locked() is
> +     * not suitable here because we may read_unlock(&pdev->domain->pci_lock)
> +     * before returning.

I'm confused by this comment, as I don't see why it matters that the
lock might be lock before returning.  We need to ensure the lock is
taken at the time of the assert, and hence pdev_list_is_read_locked()
can be used.

> +     */
> +    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
> +    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));
> +
>      for ( i = 0; i < msix->max_entries; i++ )
>      {
>          const struct vpci_msix_entry *entry = &msix->entries[i];
> diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
> index aabc5465a7d3..9f31cb84c9f3 100644
> --- a/xen/include/xen/pci.h
> +++ b/xen/include/xen/pci.h
> @@ -171,6 +171,20 @@ void pcidevs_lock(void);
>  void pcidevs_unlock(void);
>  bool __must_check pcidevs_locked(void);
>  
> +#ifndef NDEBUG
> +/*
> + * Check for use in ASSERTs to ensure there will be no changes to the entries
> + * in d->pdev_list (but not the contents of each entry).
> + * This check is not suitable for protecting other state or critical regions.
> + */
> +#define pdev_list_is_read_locked(d) ({                             \
> +        /* NB: d may be evaluated multiple times, or not at all */ \
> +        pcidevs_locked() || (d && rw_is_locked(&d->pci_lock));     \

'd' is missing parentheses here, should be (d).

> +    })
> +#else
> +bool pdev_list_is_read_locked(const struct domain *d);
> +#endif

FWIW, if this is only intended to be used with ASSERT, it might as
well be an ASSERT itself:

ASSERT_PDEV_LIST_IS_READ_LOCKED(d) ...

Don't have a strong opinion, so I'm fine with how it's used, just
noting it might be clearer if it was an ASSERT_ right away.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 12:39:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 12:39:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682193.1061334 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV9-0008Me-Ly; Fri, 16 Feb 2024 12:39:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682193.1061334; Fri, 16 Feb 2024 12:39:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV9-0008LA-GL; Fri, 16 Feb 2024 12:39:35 +0000
Received: by outflank-mailman (input) for mailman id 682193;
 Fri, 16 Feb 2024 12:39:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=acXU=JZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raxV8-0006oJ-4G
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 12:39:34 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6fb2ce04-ccc8-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 13:39:33 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a3de25186fbso65365166b.0
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 04:39:33 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 lj8-20020a170907188800b00a3db1734b92sm1121608ejc.43.2024.02.16.04.39.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 04:39:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fb2ce04-ccc8-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708087173; x=1708691973; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ELu7fx1uXcKKvX0ev7mgwIEsa2oKN1zApZ0Pg4TKtMg=;
        b=F4lUmyvMz8kbw4SheeKxEHi3aPn2hPDs1dxqLCGmvTbk3ZlpAx7gnLjkfZLsYGzZip
         oTB7SIII9/LFQrjD2XnxzuP5ndlSfCouU5VGY30lD3QCSpwhVnQOAhNKIxY4IMSzp8MH
         gn2AZBHWmxrQRqp+vJLupOWdzHiQbN5g3lnaDVjmYeOxGqlH9BbK4Yl/rxyyouH+qiXM
         /xWZp50YOFZrm0d066BhP1nJIMPpSz+yseZnLFEOQRSKYveaHtzelJEP9QILgamhhA3i
         YlLNFO5IM0RAYB7vRP2oTsRBJ/CktKH7KvuV7GxCVsPPH9G9AyQGgQ+BqmrRMzB5LSV/
         YwlA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708087173; x=1708691973;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ELu7fx1uXcKKvX0ev7mgwIEsa2oKN1zApZ0Pg4TKtMg=;
        b=ci6L/IjbJ7sslt/lekD6edSF9BNr7Yn0bODpxWMcxPFnUDCVX1zx9IbHLIDo48654R
         Tfn5ik2xKICNJRfXXvgLCQMiV9IH46Fts/mts0G0BWgyrus0B0ssYNV3BMnShjeAfCWt
         U9ugpP4ZLIKh7HtBhJGq/Z/+7YQ9Cvlxwh6JypvNNDgi2vd+YGzuY+Z7EYMQJfdqLPD7
         v+a7qMoZYyLaHc/kpQ/RKI95oC5cokTGibkHmxiSGTqPChvrjj7Z/Dy6UZP92uk24WTP
         04OSEkuy/jNnt1b+ctGVUYWVlhqfUH4nSgTNvbRRXAIamxWjLYHEG+YzOvkUk6W2AH5S
         Hwyw==
X-Gm-Message-State: AOJu0YwZ9UeC8gpbwo6FTIyyDgdOHqSZlLplXxdBDtdxWZspscMVNL7w
	Q6pAqX7LR5kSzaOL85SAYjtAnxHny17JikhQKBcuSd0UFpUxEV3RNirjmsG5
X-Google-Smtp-Source: AGHT+IEgSArJ2RWktN+cBbDdUCZOEWFMz/h00WcEFOPs7dnjimSk0EQ6RWUaUr/nb0pDUpMPvqyhwQ==
X-Received: by 2002:a17:906:451:b0:a3d:9962:f05f with SMTP id e17-20020a170906045100b00a3d9962f05fmr2952460eja.74.1708087172839;
        Fri, 16 Feb 2024 04:39:32 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Julien Grall <julien@xen.org>,
	Rahul Singh <rahul.singh@arm.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v9 7/7] xen/asm-generic: fold struct devarch into struct dev
Date: Fri, 16 Feb 2024 13:39:19 +0100
Message-ID: <3a5bf394a9d95a28cecac996f6e0decb788c19fd.1708086092.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708086091.git.oleksii.kurochko@gmail.com>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The current patch is a follow-up to the patch titled:
    xen/asm-generic: introduce generic device.h
Also, a prerequisite for this patch is, without which a compilation
error will occur:
    xen/arm: switch Arm to use asm-generic/device.h

The 'struct dev_archdata' is exclusively used within 'struct device',
so it could be merged into 'struct device.'

After the merger, it is necessary to update the 'dev_archdata()'
macros and the comments above 'struct arm_smmu_xen_device' in
drivers/passthrough/arm/smmu.c.
Additionally, it is required to update instances of
"dev->archdata->iommu" to "dev->iommu".

Suggested-by: Julien Grall <julien@xen.org>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
  This patch can be merged with patches 4 and 5 of this patch series.
---
Changes in V9:
 - newly introduced patch.
---
 xen/drivers/passthrough/arm/smmu.c | 12 ++++++------
 xen/include/asm-generic/device.h   |  8 +-------
 2 files changed, 7 insertions(+), 13 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index 32e2ff279b..4a272c8779 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -227,9 +227,9 @@ struct arm_smmu_xen_domain {
 };
 
 /*
- * Xen: Information about each device stored in dev->archdata.iommu
+ * Xen: Information about each device stored in dev->iommu
  *
- * Initially dev->archdata.iommu only stores the iommu_domain (runtime
+ * Initially dev->iommu only stores the iommu_domain (runtime
  * configuration of the SMMU) but, on Xen, we also have to store the
  * iommu_group (list of streamIDs associated to the device).
  *
@@ -242,7 +242,7 @@ struct arm_smmu_xen_device {
 	struct iommu_group *group;
 };
 
-#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->archdata.iommu)
+#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->iommu)
 #define dev_iommu_domain(dev) (dev_archdata(dev)->domain)
 #define dev_iommu_group(dev) (dev_archdata(dev)->group)
 
@@ -2777,9 +2777,9 @@ static int arm_smmu_assign_dev(struct domain *d, u8 devfn,
 
 	xen_domain = dom_iommu(d)->arch.priv;
 
-	if (!dev->archdata.iommu) {
-		dev->archdata.iommu = xzalloc(struct arm_smmu_xen_device);
-		if (!dev->archdata.iommu)
+	if (!dev->iommu) {
+		dev->iommu = xzalloc(struct arm_smmu_xen_device);
+		if (!dev->iommu)
 			return -ENOMEM;
 	}
 
diff --git a/xen/include/asm-generic/device.h b/xen/include/asm-generic/device.h
index f91bb7f771..1acd1ba1d8 100644
--- a/xen/include/asm-generic/device.h
+++ b/xen/include/asm-generic/device.h
@@ -22,12 +22,6 @@ enum device_class
     DEVICE_UNKNOWN,
 };
 
-struct dev_archdata {
-#ifdef CONFIG_HAS_PASSTHROUGH
-    void *iommu;    /* IOMMU private data */
-#endif
-};
-
 /* struct device - The basic device structure */
 struct device
 {
@@ -35,8 +29,8 @@ struct device
 #ifdef CONFIG_HAS_DEVICE_TREE
     struct dt_device_node *of_node; /* Used by drivers imported from Linux */
 #endif
-    struct dev_archdata archdata;
 #ifdef CONFIG_HAS_PASSTHROUGH
+    void *iommu; /* IOMMU private data */;
     struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
 #endif
 };
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 12:39:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 12:39:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682190.1061311 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV6-0007mH-Av; Fri, 16 Feb 2024 12:39:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682190.1061311; Fri, 16 Feb 2024 12:39:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV6-0007m4-76; Fri, 16 Feb 2024 12:39:32 +0000
Received: by outflank-mailman (input) for mailman id 682190;
 Fri, 16 Feb 2024 12:39:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=acXU=JZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raxV4-0006oJ-KC
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 12:39:30 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6d8db062-ccc8-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 13:39:30 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-563d56ee65cso914509a12.2
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 04:39:30 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 lj8-20020a170907188800b00a3db1734b92sm1121608ejc.43.2024.02.16.04.39.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 04:39:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d8db062-ccc8-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708087169; x=1708691969; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=CLqhOGTUK80zlqCeFoI7si1NHAXYOmuSN6ghjuHIAG8=;
        b=DdLycTm+40w/dq3uMaS6vf1l8cze6yLMu35ZdKDSu3lP9DNs9G4bz0IAFXQxeRew3r
         MHQGQx28YIvIR8hHwGWhn810bs41rTGLeIKJg/HNW6tKjveooBM61AGu4687aXD8BcFf
         dAEjNG5CI8OnkGz2Q7mn7OWUV+K+m9xOYVD45eddw2H24cyK6yy06enLxUovCoS6N6v5
         FYI6pVPxRr8siG4+q1uXrEl2wqp2XJVjPM92pexCmluLV0pM2sjfYI0lob8XiLyY3rMx
         aWa8GWR+1Y4fA78WRMBzVew9Q5dRYeZC42+bdH89yZFKF8l5ztLR8vxAFZwD03sD3oJZ
         uN2Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708087169; x=1708691969;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=CLqhOGTUK80zlqCeFoI7si1NHAXYOmuSN6ghjuHIAG8=;
        b=RItpEegOJdVX19oFlOXDVxmPvJ4UgNWcysp6nhfVXqNQdnnQHKnc49e0gLHCjQogiK
         h8x8TRqPCK8shubh4ittU+/glIFw0tVC1XR8F2+YU97eJp32mefM/6+zC1Yv+GecNYDM
         Fjcmu5igm8ACPZ8w1PBcP1CfKxcZu02adlzbFsbnAWqkT5UVvNHs/4/1KqSf99EKirb4
         HBN93MvRl65Khagu3C/7nYjMFtwzn+9wFQVCP3Z9rABsxdZGfKvBn+10HX4xsddZEESq
         uUvOW2bLgtTyOBOpLYJKOQZZzirpehCXhDXCOAF0LsfqtBePI7D8ehxsf9lepcov3H5O
         theQ==
X-Gm-Message-State: AOJu0Yw8ZLkS3bN8HupkOFWUmnEDiKEfZZlqO7+6gMhTPdILYEMzTY3/
	Mahik4YIviaQWkzguYUTXbVwxE3xYhL8e7Ffgoczb7+yX1JhJCy3HqsQ+zEp
X-Google-Smtp-Source: AGHT+IHvLiLfCT0H+vGdVFTX1p2sg25x/p7DSf50cigHt5WcaHswwiFLYJLcD94eG7smSK8KHoT8sg==
X-Received: by 2002:a17:906:f80e:b0:a3d:eec8:6679 with SMTP id kh14-20020a170906f80e00b00a3deec86679mr810735ejb.54.1708087168577;
        Fri, 16 Feb 2024 04:39:28 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v9 4/7] xen/asm-generic: introduce generic device.h
Date: Fri, 16 Feb 2024 13:39:16 +0100
Message-ID: <e6df4fde7de451d089ac680535f44a7206313e15.1708086092.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708086091.git.oleksii.kurochko@gmail.com>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Arm, PPC and RISC-V introduce the same things in asm/device.h, so
generic device.h was introduced.
Arm's device.h was taken as a base with the following changes:
 - #ifdef ACPI related things.
 - Rename #ifdef guards.
 - Add SPDX tag.
 - #ifdef CONFIG_HAS_DEVICE_TREE related things.
 - #ifdef-ing iommu related things with CONFIG_HAS_PASSTHROUGH.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V9:
 - move 'struct device_desc' to #ifdef CONFIG_HAS_DEVICE_TREE as it is expected to
   have this structure only in case of CONFIG_HAS_DEVICE_TREE.
 - rename name_, namestr_, class_ to dev_name, ident and cls in DT_DEVICE_START and ACPI_DEVICE_START
   macros.
 - Add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V8:
 - drop #ifdef HAS_PCI around DEV_PCI in enum device_type to be consistent with DEVICE_PCI_HOSTBRIDGE
 - drop the comment above enum device_type, after DEV_PCI is alaways present in enum device_type, so
   there is no need anymore in comment and DEV_TYPE_MAX.
 - drop #ifdef HAS_PCI around dev_is_pci() as DEV_PCI is alwayds define now.
 - rename args _name, _namestr, _class to name, namestr, class.
 - drop unnessary backslash after .class = class in defines ACPI_DEVICE_START() and DT_DEVICE_START()
 - update the commit message 
---
Changes in V7:
 - keeping DEVICE_PCI_HOSTBRIDGE available for every build based on the reply:
    https://lore.kernel.org/xen-devel/926a5c12-7f02-42ec-92a8-1c82d060c710@xen.org/
 - add comment above enum device_type.h with explanation about DEV_TYPE_MAX.        
 - separate patch  "[PATCH v6 9/9] xen/asm-generic: introduce generic device.h" into 3 patches.
---
Changes in V6:
 - Rebase only.
---
Changes in V5:
  - Removed generated file: xen/include/headers++.chk.new
  - Removed pointless #ifdef CONFIG_HAS_DEVICE_TREE ... #endif for PPC as
    CONFIG_HAS_DEVICE_TREE will be always used for PPC.
---
Changes in V4:
 - Updated the commit message
 - Switched Arm and PPC to asm-generic version of device.h
 - Replaced HAS_PCI with CONFIG_HAS_PCI
 - ifdef-ing iommu filed of dev_archdata struct with CONFIG_HAS_PASSTHROUGH
 - ifdef-ing iommu_fwspec of device struct with CONFIG_HAS_PASSTHROUGH
 - ifdef-ing DT related things with CONFIG_HAS_DEVICE_TREE
 - Updated the commit message ( remove a note with question about
   if device.h should be in asm-generic or not )
 - Replaced DEVICE_IC with DEVICE_INTERRUPT_CONTROLLER
 - Rationalized usage of CONFIG_HAS_* in device.h
 - Fixed indents for ACPI_DEVICE_START and ACPI_DEVICE_END
---
Changes in V3:
 - ifdef device tree related things.
 - update the commit message
---
Changes in V2:
	- take ( as common ) device.h from Arm as PPC and RISC-V use it as a base.
	- #ifdef PCI related things.
	- #ifdef ACPI related things.
	- rename DEVICE_GIC to DEVIC_IC.
	- rename #ifdef guards.
	- switch Arm and PPC to generic device.h
	- add SPDX tag
	- update the commit message
---
 xen/include/asm-generic/device.h | 145 +++++++++++++++++++++++++++++++
 1 file changed, 145 insertions(+)
 create mode 100644 xen/include/asm-generic/device.h

diff --git a/xen/include/asm-generic/device.h b/xen/include/asm-generic/device.h
new file mode 100644
index 0000000000..f91bb7f771
--- /dev/null
+++ b/xen/include/asm-generic/device.h
@@ -0,0 +1,145 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_GENERIC_DEVICE_H__
+#define __ASM_GENERIC_DEVICE_H__
+
+#include <xen/stdbool.h>
+
+enum device_type
+{
+#ifdef CONFIG_HAS_DEVICE_TREE
+    DEV_DT,
+#endif
+    DEV_PCI
+};
+
+enum device_class
+{
+    DEVICE_SERIAL,
+    DEVICE_IOMMU,
+    DEVICE_INTERRUPT_CONTROLLER,
+    DEVICE_PCI_HOSTBRIDGE,
+    /* Use for error */
+    DEVICE_UNKNOWN,
+};
+
+struct dev_archdata {
+#ifdef CONFIG_HAS_PASSTHROUGH
+    void *iommu;    /* IOMMU private data */
+#endif
+};
+
+/* struct device - The basic device structure */
+struct device
+{
+    enum device_type type;
+#ifdef CONFIG_HAS_DEVICE_TREE
+    struct dt_device_node *of_node; /* Used by drivers imported from Linux */
+#endif
+    struct dev_archdata archdata;
+#ifdef CONFIG_HAS_PASSTHROUGH
+    struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
+#endif
+};
+
+typedef struct device device_t;
+
+#ifdef CONFIG_HAS_DEVICE_TREE
+
+#include <xen/device_tree.h>
+
+#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
+
+/**
+ *  device_init - Initialize a device
+ *  @dev: device to initialize
+ *  @class: class of the device (serial, network...)
+ *  @data: specific data for initializing the device
+ *
+ *  Return 0 on success.
+ */
+int device_init(struct dt_device_node *dev, enum device_class class,
+                const void *data);
+
+/**
+ * device_get_type - Get the type of the device
+ * @dev: device to match
+ *
+ * Return the device type on success or DEVICE_ANY on failure
+ */
+enum device_class device_get_class(const struct dt_device_node *dev);
+
+#define DT_DEVICE_START(dev_name, ident, cls)                   \
+static const struct device_desc __dev_desc_##dev_name __used    \
+__section(".dev.info") = {                                      \
+    .name = ident,                                              \
+    .class = cls,
+
+#define DT_DEVICE_END                                           \
+};
+
+struct device_desc {
+    /* Device name */
+    const char *name;
+    /* Device class */
+    enum device_class class;
+
+    /* List of devices supported by this driver */
+    const struct dt_device_match *dt_match;
+    /*
+     * Device initialization.
+     *
+     * -EAGAIN is used to indicate that device probing is deferred.
+     */
+    int (*init)(struct dt_device_node *dev, const void *data);
+};
+
+#else /* !CONFIG_HAS_DEVICE_TREE */
+#define dev_is_dt(dev) ((void)(dev), false)
+#endif /* CONFIG_HAS_DEVICE_TREE */
+
+#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
+
+#ifdef CONFIG_ACPI
+
+struct acpi_device_desc {
+    /* Device name */
+    const char *name;
+    /* Device class */
+    enum device_class class;
+    /* type of device supported by the driver */
+    const int class_type;
+    /* Device initialization */
+    int (*init)(const void *data);
+};
+
+/**
+ *  acpi_device_init - Initialize a device
+ *  @class: class of the device (serial, network...)
+ *  @data: specific data for initializing the device
+ *
+ *  Return 0 on success.
+ */
+int acpi_device_init(enum device_class class,
+                     const void *data, int class_type);
+
+#define ACPI_DEVICE_START(dev_name, ident, cls)                     \
+static const struct acpi_device_desc __dev_desc_##dev_name __used   \
+__section(".adev.info") = {                                         \
+    .name = ident,                                                  \
+    .class = cls,
+
+#define ACPI_DEVICE_END                                             \
+};
+
+#endif /* CONFIG_ACPI */
+
+#endif /* __ASM_GENERIC_DEVICE_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 12:39:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 12:39:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682192.1061326 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV9-00089E-1X; Fri, 16 Feb 2024 12:39:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682192.1061326; Fri, 16 Feb 2024 12:39:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV8-00088H-Qf; Fri, 16 Feb 2024 12:39:34 +0000
Received: by outflank-mailman (input) for mailman id 682192;
 Fri, 16 Feb 2024 12:39:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=acXU=JZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raxV7-0006oJ-3a
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 12:39:33 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6f1d6ee9-ccc8-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 13:39:32 +0100 (CET)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a2a17f3217aso254729366b.2
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 04:39:32 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 lj8-20020a170907188800b00a3db1734b92sm1121608ejc.43.2024.02.16.04.39.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 04:39:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f1d6ee9-ccc8-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708087171; x=1708691971; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=1zM8XpDaj0bwgFuwnlMqun0OZXmeuznnNGSwY8Hidtw=;
        b=k/X+f+qT3NDcd79wTBrBlAKnwTOjXC249j9A3vqWWUR3StHY9kogDBwzrOcol/kWYd
         1I910uUeXJBUMXr7ijlbH0/051FPHS3fKZTwMt6RK+pHFb/rIefknv/zRl5gKyDUdWia
         bHDLKAZ62ymi9JC2AA54CIO5Jfg91MNX+uej19+pb+0E8SvMtXHOQsf50KueKFZI3+rI
         c2Q452PSu/Xc2bkWdSQq9pFMvNrrGU9s89yd4tjOGKF+C0R49A4/JIGchru8BoBysFlz
         j4u36+5sTi4AvflAWRUEuVk/xD8WWa3NyO9/0+36LAJxLi7Ji/cRBhFmpVhhqQ5uwhVI
         Qrhg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708087171; x=1708691971;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=1zM8XpDaj0bwgFuwnlMqun0OZXmeuznnNGSwY8Hidtw=;
        b=absOU9t54rQ2FpriOtqcTw+VSg25hXH7LIQ7gfunFjfao4Yn9n66w5OLEuJpspS/F2
         b021vKBjkGkKDqr2ut858jhyMC9RsNjwFLANgqIdFwGqPKttjec4Cb6dWGf5m8NuCm5w
         upYiU3CvZBh6wqRYBHVg+i8MD7j/7sN3/k1yFmv4WImFySAKKItosP26Rmi9HrtLHFYN
         vlYj95vFQeGAbPFXH5YeKtm9IBP80FXgj8lL1gV1HAQIZyHvSP8tXnoO2kRBErtCw/Jj
         kFz7EGQZWn85zav1wlE65+TJ2l/HXDgdE+8nMVXg4wmoBslmxakXuPTQVl1JipJYEQH3
         P2pw==
X-Gm-Message-State: AOJu0YyNkSm4/x9Vvf5NBcPMtgnG4tlMYLCs0UXvWQ6BGrDRehVL4tUH
	HYzwONoY4ND3bxSEHmHGs+I5slZVUK24RkUpfP8kHoTs4b5/O3jCnJneHFvx
X-Google-Smtp-Source: AGHT+IF+iDaRaUHdatlA0i3c5lu7YparT93bpBVp2gnoxYvBLfVtPC1KMpjFtLaILr0DpwOngQTlgA==
X-Received: by 2002:a17:906:254e:b0:a3d:ab64:7524 with SMTP id j14-20020a170906254e00b00a3dab647524mr2778795ejb.36.1708087171378;
        Fri, 16 Feb 2024 04:39:31 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>
Subject: [PATCH v9 6/7] xen/ppc: switch PPC to use asm-generic/device.h
Date: Fri, 16 Feb 2024 13:39:18 +0100
Message-ID: <222ae66639312712e145bd04d5dd1ebd2c511d43.1708086092.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708086091.git.oleksii.kurochko@gmail.com>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changed in V8:
 - nothing changed. only rebase.
---
Changes in V7:
 - newly introduced patch which is based on the previous version of the patch:
     [PATCH v6 9/9] xen/asm-generic: introduce generic device.h
---
 xen/arch/ppc/include/asm/Makefile |  1 +
 xen/arch/ppc/include/asm/device.h | 53 -------------------------------
 2 files changed, 1 insertion(+), 53 deletions(-)
 delete mode 100644 xen/arch/ppc/include/asm/device.h

diff --git a/xen/arch/ppc/include/asm/Makefile b/xen/arch/ppc/include/asm/Makefile
index 3fd893f3e0..ced02e26ed 100644
--- a/xen/arch/ppc/include/asm/Makefile
+++ b/xen/arch/ppc/include/asm/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
 generic-y += altp2m.h
+generic-y += device.h
 generic-y += div64.h
 generic-y += hardirq.h
 generic-y += hypercall.h
diff --git a/xen/arch/ppc/include/asm/device.h b/xen/arch/ppc/include/asm/device.h
deleted file mode 100644
index 8253e61d51..0000000000
--- a/xen/arch/ppc/include/asm/device.h
+++ /dev/null
@@ -1,53 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-#ifndef __ASM_PPC_DEVICE_H__
-#define __ASM_PPC_DEVICE_H__
-
-enum device_type
-{
-    DEV_DT,
-    DEV_PCI,
-};
-
-struct device {
-    enum device_type type;
-#ifdef CONFIG_HAS_DEVICE_TREE
-    struct dt_device_node *of_node; /* Used by drivers imported from Linux */
-#endif
-};
-
-enum device_class
-{
-    DEVICE_SERIAL,
-    DEVICE_IOMMU,
-    DEVICE_PCI_HOSTBRIDGE,
-    /* Use for error */
-    DEVICE_UNKNOWN,
-};
-
-struct device_desc {
-    /* Device name */
-    const char *name;
-    /* Device class */
-    enum device_class class;
-    /* List of devices supported by this driver */
-    const struct dt_device_match *dt_match;
-    /*
-     * Device initialization.
-     *
-     * -EAGAIN is used to indicate that device probing is deferred.
-     */
-    int (*init)(struct dt_device_node *dev, const void *data);
-};
-
-typedef struct device device_t;
-
-#define DT_DEVICE_START(name_, namestr_, class_)                    \
-static const struct device_desc __dev_desc_##name_ __used           \
-__section(".dev.info") = {                                          \
-    .name = namestr_,                                               \
-    .class = class_,                                                \
-
-#define DT_DEVICE_END                                               \
-};
-
-#endif /* __ASM_PPC_DEVICE_H__ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 12:39:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 12:39:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682191.1061321 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV8-000851-J2; Fri, 16 Feb 2024 12:39:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682191.1061321; Fri, 16 Feb 2024 12:39:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV8-00084t-FO; Fri, 16 Feb 2024 12:39:34 +0000
Received: by outflank-mailman (input) for mailman id 682191;
 Fri, 16 Feb 2024 12:39:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=acXU=JZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raxV7-0006oP-4Q
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 12:39:33 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6e5a3f02-ccc8-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 13:39:31 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a3deb3f563bso51047466b.2
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 04:39:31 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 lj8-20020a170907188800b00a3db1734b92sm1121608ejc.43.2024.02.16.04.39.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 04:39:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e5a3f02-ccc8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708087170; x=1708691970; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YrSY4X90IuSAbrPbKK2oOfiNpaRxGc61KTsJ00MVrAM=;
        b=czg1v7XYdT9o/V/v13CAIKI0vIlr264WjINv5uI5G/UN73bdM/2Xw6UowfSeI6zTDC
         Hkl5hFaWsKxFjOaknAEWAdW4VL1WVUjPywqWXP178LreDJ78VyBQ1slcyOPnA92HMNQ8
         85/Pwye+cQfbZlMbuERXumaIOFUSzFvdkaSkyjGIHdYoMced0A+KEm6ACMoNqGmg7FZu
         x3aXie4189E5VMH6fwmTAy8pZtYIiC45CB3Z4pCd4+EC3j67qG/PoxECBhXycIfjos+r
         ENcjBANvz39WSEovaiTH1/NNgAIcxW7TVb3NNdyiWZt1lxYNeWMBb/TrzQjV+7gr5pEv
         oNag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708087170; x=1708691970;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=YrSY4X90IuSAbrPbKK2oOfiNpaRxGc61KTsJ00MVrAM=;
        b=Cj+ToHO4DdqTtYwer6L5KsxsVMzwwsEgybyLt8OsRxzG/Z+YjnAiaAILfBHlfSX56D
         ScKxuo3bsSXvIdvlc1U5l9DOxI9p0daE9sx54F6KpyIah5Ra1ksx1NXL5POw8s2pezRt
         vURLJojDFHvxPJNOtYFsa90Y+0n1CDjowy7saNAO3XnJIaqS/WNhfV5av56ZvH4m9hjr
         a5A7Jqiwlhh/e0MACPih8bT7gESvvCjTwC7TlinA5XPzj0E1uEW0PhIgwKMlxzNX64Vn
         ieIieyv3eHevsKGTKVtTWIvbTM+rX6DE6sNPKMh8uniDD0qKMKmIjL8I5D5EUW3fpy4z
         u18w==
X-Gm-Message-State: AOJu0Yz8dmeudbG+upMi08okSNWMh02QvY83/D0rvyC6anYe9/d/lpRr
	h7KfN+DTR8Oe6mwic545m2nUKrD7S+malkh7OAxa5/smjiUQIOxpZD2aTAsi
X-Google-Smtp-Source: AGHT+IEshWja8Iu0W37U0QhlRH6bVp5bEopQ6HBO0evdbwZ9+7eDnlQ2tD47VoC+ACQhbgr2qdhHjA==
X-Received: by 2002:a17:906:e28f:b0:a3d:8a86:a8fd with SMTP id gg15-20020a170906e28f00b00a3d8a86a8fdmr867935ejb.30.1708087170060;
        Fri, 16 Feb 2024 04:39:30 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v9 5/7] xen/arm: switch Arm to use asm-generic/device.h
Date: Fri, 16 Feb 2024 13:39:17 +0100
Message-ID: <bd6d5295425b6263e5a952b93c832b229b3fda6b.1708086092.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708086091.git.oleksii.kurochko@gmail.com>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The following changes were done as a result of switching to
asm-generic/device.h:
 * DEVICE_GIC was renamed to DEVICE_INTERRUPT_CONTROLLER according
   to definition of enum device_class in asm-generic/device.h.
 * acpi-related things in Arm code were guarded by #ifdef CONFIG_ACPI
   as struct acpi_device_desc was guarded in asm-generic, also functions
   acpi_device_init() was guarded too as they are using structure
   acpi_device_desc inside.
 * drop arm/include/asm/device.h and update arm/include/asm/Makefile
   to use asm-generic/device.h instead.

As 'struct device_desc' is protected by CONFIG_HAS_DEVICE_TREE,
_sdevice, _edevice, device_init(), and device_get_class should also be
protected.
However, this protection was not implemented because Arm always has
CONFIG_HAS_DEVICE_TREE=y at the moment.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
Changes in V9:
 - protect .adev_info in Arm linker script by #ifdef CONFIG_ACPI.
 - update the commit message. 
 - add Reviewed-by: Julien Grall <jgrall@amazon.com>
---
Changes in V8:
 - update the commit message
---
Changes in V7:
 - newly introduced patch which is based on the previous version of the patch:
     [PATCH v6 9/9] xen/asm-generic: introduce generic device.h
---
 xen/arch/arm/device.c             |   5 ++
 xen/arch/arm/domain_build.c       |   2 +-
 xen/arch/arm/gic-v2.c             |   4 +-
 xen/arch/arm/gic-v3.c             |   6 +-
 xen/arch/arm/gic.c                |   4 +-
 xen/arch/arm/include/asm/Makefile |   1 +
 xen/arch/arm/include/asm/device.h | 124 ------------------------------
 xen/arch/arm/xen.lds.S            |   2 +
 8 files changed, 16 insertions(+), 132 deletions(-)
 delete mode 100644 xen/arch/arm/include/asm/device.h

diff --git a/xen/arch/arm/device.c b/xen/arch/arm/device.c
index 1f631d3274..3e02cff008 100644
--- a/xen/arch/arm/device.c
+++ b/xen/arch/arm/device.c
@@ -16,7 +16,10 @@
 #include <xen/lib.h>
 
 extern const struct device_desc _sdevice[], _edevice[];
+
+#ifdef CONFIG_ACPI
 extern const struct acpi_device_desc _asdevice[], _aedevice[];
+#endif
 
 int __init device_init(struct dt_device_node *dev, enum device_class class,
                        const void *data)
@@ -45,6 +48,7 @@ int __init device_init(struct dt_device_node *dev, enum device_class class,
     return -EBADF;
 }
 
+#ifdef CONFIG_ACPI
 int __init acpi_device_init(enum device_class class, const void *data, int class_type)
 {
     const struct acpi_device_desc *desc;
@@ -61,6 +65,7 @@ int __init acpi_device_init(enum device_class class, const void *data, int class
 
     return -EBADF;
 }
+#endif
 
 enum device_class device_get_class(const struct dt_device_node *dev)
 {
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 46161848dc..085d88671e 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1651,7 +1651,7 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
      * Replace these nodes with our own. Note that the original may be
      * used_by DOMID_XEN so this check comes first.
      */
-    if ( device_get_class(node) == DEVICE_GIC )
+    if ( device_get_class(node) == DEVICE_INTERRUPT_CONTROLLER )
         return make_gic_node(d, kinfo->fdt, node);
     if ( dt_match_node(timer_matches, node) )
         return make_timer_node(kinfo);
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index cf392bfd1c..5d6885e389 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -1366,7 +1366,7 @@ static const struct dt_device_match gicv2_dt_match[] __initconst =
     { /* sentinel */ },
 };
 
-DT_DEVICE_START(gicv2, "GICv2", DEVICE_GIC)
+DT_DEVICE_START(gicv2, "GICv2", DEVICE_INTERRUPT_CONTROLLER)
         .dt_match = gicv2_dt_match,
         .init = gicv2_dt_preinit,
 DT_DEVICE_END
@@ -1381,7 +1381,7 @@ static int __init gicv2_acpi_preinit(const void *data)
     return 0;
 }
 
-ACPI_DEVICE_START(agicv2, "GICv2", DEVICE_GIC)
+ACPI_DEVICE_START(agicv2, "GICv2", DEVICE_INTERRUPT_CONTROLLER)
         .class_type = ACPI_MADT_GIC_VERSION_V2,
         .init = gicv2_acpi_preinit,
 ACPI_DEVICE_END
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index bf0e5c1b75..1cb1360606 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1909,7 +1909,7 @@ static const struct dt_device_match gicv3_dt_match[] __initconst =
     { /* sentinel */ },
 };
 
-DT_DEVICE_START(gicv3, "GICv3", DEVICE_GIC)
+DT_DEVICE_START(gicv3, "GICv3", DEVICE_INTERRUPT_CONTROLLER)
         .dt_match = gicv3_dt_match,
         .init = gicv3_dt_preinit,
 DT_DEVICE_END
@@ -1924,12 +1924,12 @@ static int __init gicv3_acpi_preinit(const void *data)
     return 0;
 }
 
-ACPI_DEVICE_START(agicv3, "GICv3", DEVICE_GIC)
+ACPI_DEVICE_START(agicv3, "GICv3", DEVICE_INTERRUPT_CONTROLLER)
         .class_type = ACPI_MADT_GIC_VERSION_V3,
         .init = gicv3_acpi_preinit,
 ACPI_DEVICE_END
 
-ACPI_DEVICE_START(agicv4, "GICv4", DEVICE_GIC)
+ACPI_DEVICE_START(agicv4, "GICv4", DEVICE_INTERRUPT_CONTROLLER)
         .class_type = ACPI_MADT_GIC_VERSION_V4,
         .init = gicv3_acpi_preinit,
 ACPI_DEVICE_END
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index d922ea67aa..b5a9c8266c 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -234,7 +234,7 @@ static void __init gic_dt_preinit(void)
         if ( !dt_get_parent(node) )
             continue;
 
-        rc = device_init(node, DEVICE_GIC, NULL);
+        rc = device_init(node, DEVICE_INTERRUPT_CONTROLLER, NULL);
         if ( !rc )
         {
             /* NOTE: Only one GIC is supported */
@@ -262,7 +262,7 @@ static void __init gic_acpi_preinit(void)
 
     dist = container_of(header, struct acpi_madt_generic_distributor, header);
 
-    if ( acpi_device_init(DEVICE_GIC, NULL, dist->version) )
+    if ( acpi_device_init(DEVICE_INTERRUPT_CONTROLLER, NULL, dist->version) )
         panic("Unable to find compatible GIC in the ACPI table\n");
 }
 #else
diff --git a/xen/arch/arm/include/asm/Makefile b/xen/arch/arm/include/asm/Makefile
index 505cb49090..4a4036c951 100644
--- a/xen/arch/arm/include/asm/Makefile
+++ b/xen/arch/arm/include/asm/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
 generic-y += altp2m.h
+generic-y += device.h
 generic-y += hardirq.h
 generic-y += iocap.h
 generic-y += paging.h
diff --git a/xen/arch/arm/include/asm/device.h b/xen/arch/arm/include/asm/device.h
deleted file mode 100644
index b5d451e087..0000000000
--- a/xen/arch/arm/include/asm/device.h
+++ /dev/null
@@ -1,124 +0,0 @@
-#ifndef __ASM_ARM_DEVICE_H
-#define __ASM_ARM_DEVICE_H
-
-enum device_type
-{
-    DEV_DT,
-    DEV_PCI,
-};
-
-struct dev_archdata {
-    void *iommu;    /* IOMMU private data */
-};
-
-/* struct device - The basic device structure */
-struct device
-{
-    enum device_type type;
-#ifdef CONFIG_HAS_DEVICE_TREE
-    struct dt_device_node *of_node; /* Used by drivers imported from Linux */
-#endif
-    struct dev_archdata archdata;
-    struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU instance data */
-};
-
-typedef struct device device_t;
-
-#include <xen/device_tree.h>
-
-#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
-#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
-
-enum device_class
-{
-    DEVICE_SERIAL,
-    DEVICE_IOMMU,
-    DEVICE_GIC,
-    DEVICE_PCI_HOSTBRIDGE,
-    /* Use for error */
-    DEVICE_UNKNOWN,
-};
-
-struct device_desc {
-    /* Device name */
-    const char *name;
-    /* Device class */
-    enum device_class class;
-    /* List of devices supported by this driver */
-    const struct dt_device_match *dt_match;
-    /*
-     * Device initialization.
-     *
-     * -EAGAIN is used to indicate that device probing is deferred.
-     */
-    int (*init)(struct dt_device_node *dev, const void *data);
-};
-
-struct acpi_device_desc {
-    /* Device name */
-    const char *name;
-    /* Device class */
-    enum device_class class;
-    /* type of device supported by the driver */
-    const int class_type;
-    /* Device initialization */
-    int (*init)(const void *data);
-};
-
-/**
- *  acpi_device_init - Initialize a device
- *  @class: class of the device (serial, network...)
- *  @data: specific data for initializing the device
- *
- *  Return 0 on success.
- */
-int acpi_device_init(enum device_class class,
-                     const void *data, int class_type);
-
-/**
- *  device_init - Initialize a device
- *  @dev: device to initialize
- *  @class: class of the device (serial, network...)
- *  @data: specific data for initializing the device
- *
- *  Return 0 on success.
- */
-int device_init(struct dt_device_node *dev, enum device_class class,
-                const void *data);
-
-/**
- * device_get_type - Get the type of the device
- * @dev: device to match
- *
- * Return the device type on success or DEVICE_ANY on failure
- */
-enum device_class device_get_class(const struct dt_device_node *dev);
-
-#define DT_DEVICE_START(_name, _namestr, _class)                    \
-static const struct device_desc __dev_desc_##_name __used           \
-__section(".dev.info") = {                                          \
-    .name = _namestr,                                               \
-    .class = _class,                                                \
-
-#define DT_DEVICE_END                                               \
-};
-
-#define ACPI_DEVICE_START(_name, _namestr, _class)                    \
-static const struct acpi_device_desc __dev_desc_##_name __used           \
-__section(".adev.info") = {                       \
-    .name = _namestr,                                               \
-    .class = _class,                                                \
-
-#define ACPI_DEVICE_END                                               \
-};
-
-#endif /* __ASM_ARM_DEVICE_H */
-
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * indent-tabs-mode: nil
- * End:
- */
diff --git a/xen/arch/arm/xen.lds.S b/xen/arch/arm/xen.lds.S
index 470c8f2208..2266c9536f 100644
--- a/xen/arch/arm/xen.lds.S
+++ b/xen/arch/arm/xen.lds.S
@@ -138,12 +138,14 @@ SECTIONS
       _edevice = .;
   } :text
 
+#ifdef CONFIG_ACPI
   . = ALIGN(8);
   .adev.info : {
       _asdevice = .;
       *(.adev.info)
       _aedevice = .;
   } :text
+#endif
 
   . = ALIGN(8);
   .teemediator.info : {
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 12:39:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 12:39:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682188.1061290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV3-0007Hz-OL; Fri, 16 Feb 2024 12:39:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682188.1061290; Fri, 16 Feb 2024 12:39:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV3-0007Hs-La; Fri, 16 Feb 2024 12:39:29 +0000
Received: by outflank-mailman (input) for mailman id 682188;
 Fri, 16 Feb 2024 12:39:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=acXU=JZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raxV2-0006oJ-Rv
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 12:39:28 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6c9484dc-ccc8-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 13:39:28 +0100 (CET)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a3d159220c7so98511866b.2
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 04:39:28 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 lj8-20020a170907188800b00a3db1734b92sm1121608ejc.43.2024.02.16.04.39.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 04:39:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c9484dc-ccc8-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708087167; x=1708691967; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=r9oJ766LbpEcU1AnuyDzWxkmbkR4vj0ev3jWEPdVTr0=;
        b=T6AbBBqzW/PkbkoMPR8SFC82svWHO2/Y+EYjo3vVDMzknTETV7wMcY4Xw2gbxL+9Ew
         XVmA2qKPCd+E0jqkjyg05l0mSVl7+/yt+bZrcfI+viAcelxtgNEHd7qAZjBdn/pYp1bW
         T2NSPhQtOMAr8TuShDeFjxRQ8/f5VcCtIdo1U8Dd2sbYW75v2reB3U5WtYm6uw/sOd6T
         y1XZiOHKkI4s6wgcEcoKTmKCVYrT0RqTM1KCvHaAoIektZVWSirOfXeOxgAeLorIHaZW
         wWwEmMZ02e6Gsi6MUMwRaeaIZfeejzvAP/miCXwI7GJB95dLF8f2hZ/hwn8dCGPz0Dz7
         0KpA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708087167; x=1708691967;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=r9oJ766LbpEcU1AnuyDzWxkmbkR4vj0ev3jWEPdVTr0=;
        b=w7fqXimy4hKSUeecbzJZBNWgqvLN2RK72fxYNlOEgfmp6hANPhwcBWneu0h8u4Zigv
         8ILWsx+JD/09++RaXjmI4CMi8HiFpNuXFdVPlqsA4v9G6gYEJXt2GdMBzq/2bjVgE6j3
         wvsTwoRZlBcbmooaFU2ea98O0+iMArPfGby7gs3hiK2Op+L9D0LXWo0qFP2fh9QBeuzA
         8l6LVvL/FCJcTRgDK02hWosYKBHTZFZtUX3wiaJNFfwzDC6CtLb83cFzXgfDl7U0RQyP
         Y1EAsZYplilOSLYeBR3g3G82arrF3xDLGRT4klo+9GdEzop6V+IUOlUlkOPvdd3F/Igu
         mb7g==
X-Gm-Message-State: AOJu0Yysxa7Uk62yVqWTx9sbzMpFcfbNPPaBf0cPcybKyiQ4RiuG+BsR
	jJrIo4iUJm4R3eJCA3+U+qRLy2bqMZlVoqsLLeMF3ag6RCJrMJSQIM7UBNDw
X-Google-Smtp-Source: AGHT+IFCVznff4oPa0Fh9bza1Xg0r9lv4O6Pdop5CWcmfw+tuMLxmqeMJ9Wsk4s405FP8s0ldCM8Ig==
X-Received: by 2002:a17:906:4ed0:b0:a3d:fb76:cdf5 with SMTP id i16-20020a1709064ed000b00a3dfb76cdf5mr318024ejv.68.1708087167626;
        Fri, 16 Feb 2024 04:39:27 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v9 3/7] xen/asm-generic: ifdef inclusion of <asm/mem_access.h>
Date: Fri, 16 Feb 2024 13:39:15 +0100
Message-ID: <a61ba4ef7345ba1ff5025f48421de69b20e44f45.1708086092.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708086091.git.oleksii.kurochko@gmail.com>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

ifdefing inclusion of <asm/mem_access.h> in <xen/mem_access.h>
allows to avoid generation of empty <asm/mem_access.h> header
for the case when !CONFIG_MEM_ACCESS.

For Arm it was explicitly added inclusion of <asm/mem_access.h> for p2m.c
and traps.c because they require some functions from <asm/mem_access.h> which
aren't available in case of !CONFIG_MEM_ACCESS.

Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
Changes in V9:
 - Acked-by: Julien Grall <jgrall@amazon.com>
---
Changes in V8:
 - Add Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
---
Changes in V7:
 - Nothing changed. Only rebase
---
Changes in V6:
 - Remove the way how CONFIG_MEM_ACCESS is disabled for PPC and RISC-V.
 - Disable the config in ppc64_defconfig and tiny64_defconfig (RISC-V).
---
Changes in V5:
 - Added dependencies for "Config MEM_ACCESS" to be sure that randconfig will not
   turn on the config.
---
Changes in V4:
 - Nothing changed. Only rebase.
---
Changes in V3:
 - Remove unnecessary comment.
---
 xen/arch/arm/p2m.c                      | 1 +
 xen/arch/arm/traps.c                    | 1 +
 xen/arch/ppc/configs/ppc64_defconfig    | 1 +
 xen/arch/ppc/include/asm/mem_access.h   | 5 -----
 xen/arch/riscv/configs/tiny64_defconfig | 1 +
 xen/include/xen/mem_access.h            | 2 ++
 6 files changed, 6 insertions(+), 5 deletions(-)
 delete mode 100644 xen/arch/ppc/include/asm/mem_access.h

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index b991b76ce4..2465c266e9 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -7,6 +7,7 @@
 #include <asm/event.h>
 #include <asm/flushtlb.h>
 #include <asm/guest_walk.h>
+#include <asm/mem_access.h>
 #include <asm/page.h>
 #include <asm/traps.h>
 
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 9c10e8f78c..8ddca643d4 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -35,6 +35,7 @@
 #include <asm/cpufeature.h>
 #include <asm/event.h>
 #include <asm/hsr.h>
+#include <asm/mem_access.h>
 #include <asm/mmio.h>
 #include <asm/regs.h>
 #include <asm/smccc.h>
diff --git a/xen/arch/ppc/configs/ppc64_defconfig b/xen/arch/ppc/configs/ppc64_defconfig
index f7cc075e45..48a053237a 100644
--- a/xen/arch/ppc/configs/ppc64_defconfig
+++ b/xen/arch/ppc/configs/ppc64_defconfig
@@ -6,6 +6,7 @@
 # CONFIG_HYPFS is not set
 # CONFIG_GRANT_TABLE is not set
 # CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
+# CONFIG_MEM_ACCESS is not set
 
 CONFIG_PPC64=y
 CONFIG_DEBUG=y
diff --git a/xen/arch/ppc/include/asm/mem_access.h b/xen/arch/ppc/include/asm/mem_access.h
deleted file mode 100644
index e7986dfdbd..0000000000
--- a/xen/arch/ppc/include/asm/mem_access.h
+++ /dev/null
@@ -1,5 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-#ifndef __ASM_PPC_MEM_ACCESS_H__
-#define __ASM_PPC_MEM_ACCESS_H__
-
-#endif /* __ASM_PPC_MEM_ACCESS_H__ */
diff --git a/xen/arch/riscv/configs/tiny64_defconfig b/xen/arch/riscv/configs/tiny64_defconfig
index 3c9a2ff941..09defe236b 100644
--- a/xen/arch/riscv/configs/tiny64_defconfig
+++ b/xen/arch/riscv/configs/tiny64_defconfig
@@ -6,6 +6,7 @@
 # CONFIG_HYPFS is not set
 # CONFIG_GRANT_TABLE is not set
 # CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
+# CONFIG_MEM_ACCESS is not set
 
 CONFIG_RISCV_64=y
 CONFIG_DEBUG=y
diff --git a/xen/include/xen/mem_access.h b/xen/include/xen/mem_access.h
index 4e4811680d..87d93b31f6 100644
--- a/xen/include/xen/mem_access.h
+++ b/xen/include/xen/mem_access.h
@@ -33,7 +33,9 @@
  */
 struct vm_event_st;
 
+#ifdef CONFIG_MEM_ACCESS
 #include <asm/mem_access.h>
+#endif
 
 /*
  * Additional access types, which are used to further restrict
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 12:39:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 12:39:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682187.1061277 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV2-0006s2-JL; Fri, 16 Feb 2024 12:39:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682187.1061277; Fri, 16 Feb 2024 12:39:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV2-0006qU-Cn; Fri, 16 Feb 2024 12:39:28 +0000
Received: by outflank-mailman (input) for mailman id 682187;
 Fri, 16 Feb 2024 12:39:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=acXU=JZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raxV2-0006oP-40
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 12:39:28 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6b26746a-ccc8-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 13:39:26 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a3d159220c7so98507866b.2
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 04:39:25 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 lj8-20020a170907188800b00a3db1734b92sm1121608ejc.43.2024.02.16.04.39.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 04:39:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b26746a-ccc8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708087164; x=1708691964; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=2suBialpU5BjJzL1fdKyYOY1IMWl7vfTqJIHrpvRUSg=;
        b=kywOSd+l4VGNdqU+kApN/F421auBEquX1WyElNedeKBax9VippHO0XkPqQeSDc/nOa
         AoasIQxYca6GmAqgoHS8IIxwYPhePbpyDstrxQ9rUPMx05STAFrzCDc7rnYouIpkDpI9
         sqaSC3iJXZatqeYGtOgkD+NCB5pweAtpy9872iz6uEo8xPqO1ajGJlYIZIj7pN7/n3F7
         qgJTbspeVi5jct3Rf4Hk0/G0GrHPsgR/fyjS02QiUXMAb74L2gyzUBmFEzgfAqCiBk9c
         umbUmVh8rg4ywEoz+7+GPbzhZoOyu+ekSbP11V5S6kapj0IrOzwfru5F0/8jCD3M6FrT
         cEPA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708087164; x=1708691964;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=2suBialpU5BjJzL1fdKyYOY1IMWl7vfTqJIHrpvRUSg=;
        b=LXdJcDOQntXxfcQsg/V1qhhNlXZI+udy0CdsuAECO2MsrbNfmpLQa1fyRp2ZK4VegZ
         cIevqZrZeYYw3r23l4/GwtN0wRVHGWGCpR7ukEzX78sjnnxgXELZT+3WlXYotajACyQO
         d9BNvJz+IQZipucKu2xjS68KXJlIZzsUM/kl4TG7RQT/u+IEHW1ZdzdFQ8PfJacwJb92
         chjLOvvWOjM4cTjTwBIcikGI3LG2VnsmhCLikOJrYqA3RUrrS0t7Ke+wXMUgun1Wv9t+
         exEWrmEgUSYwiRmGaol16VuQbmvTGQh0Z/XAxyGouzXGTxuMhA3kzLJXYoqWubWfuVAw
         rJGA==
X-Gm-Message-State: AOJu0YxMOYsB5dMCcLZoTNeiDOE89ZLmEKoM05qq7C6TD+rCehq1hGEj
	zpskEcB6rvdqpSTz+QqPwRwciOjogSHs7cKk/Hs71QPcK1XQwZamOnbFooZj
X-Google-Smtp-Source: AGHT+IF+gJdo062UNYdwzzpd3f0cRJp8HoZz6cESOee4FvYvkdZq8KaInyjt7Ehjy6GZCWBEY2tQ+g==
X-Received: by 2002:a17:906:3e09:b0:a3d:1cdf:19eb with SMTP id k9-20020a1709063e0900b00a3d1cdf19ebmr3361862eji.11.1708087164440;
        Fri, 16 Feb 2024 04:39:24 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Michal Orzel <michal.orzel@amd.com>
Subject: [PATCH v9 1/7] automation: introduce fixed randconfig for RISC-V
Date: Fri, 16 Feb 2024 13:39:13 +0100
Message-ID: <d8e330c26394d9415a0d76b2d951bf57b0b6dd4b.1708086092.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708086091.git.oleksii.kurochko@gmail.com>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch introduces the anchor riscv-fixed-randconfig,
which includes all configurations that should be disabled for
randconfig builds.

Suggested-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
The patch were introduced after discussion in a topic:
 https://lore.kernel.org/xen-devel/cover.1701966261.git.oleksii.kurochko@gmail.com/
---
Changes in V9:
 - add '|' symbol after EXTRA_FIXED_RANDCONFIG to make variables in EXTRA_FIXED_RANDCONFIG
   be separated by new line.
 - instead of introduction of new file for riscv-fixed-randconfig, introduce an anchor
   inside build.yaml
---
Changes in V8:
 - Nothing changed. Only rebase
---
Changes in V7:
 - Nothing changed. Only rebase
---
Changes in V6:
 - The patch was introduced in this version of patch series.
---
 automation/gitlab-ci/build.yaml | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 6d2cb18b88..aac29ee13a 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -512,6 +512,14 @@ alpine-3.18-gcc-debug-arm64-boot-cpupools:
       CONFIG_BOOT_TIME_CPUPOOLS=y
 
 # RISC-V 64 cross-build
+.riscv-fixed-randconfig:
+  variables: &riscv-fixed-randconfig
+    EXTRA_FIXED_RANDCONFIG: |
+      CONFIG_COVERAGE=n
+      CONFIG_EXPERT=y
+      CONFIG_GRANT_TABLE=n
+      CONFIG_MEM_ACCESS=n
+
 archlinux-current-gcc-riscv64:
   extends: .gcc-riscv64-cross-build
   variables:
@@ -532,8 +540,7 @@ archlinux-current-gcc-riscv64-randconfig:
     CONTAINER: archlinux:current-riscv64
     KBUILD_DEFCONFIG: tiny64_defconfig
     RANDCONFIG: y
-    EXTRA_FIXED_RANDCONFIG:
-      CONFIG_COVERAGE=n
+    <<: *riscv-fixed-randconfig
 
 archlinux-current-gcc-riscv64-debug-randconfig:
   extends: .gcc-riscv64-cross-build-debug
@@ -541,8 +548,7 @@ archlinux-current-gcc-riscv64-debug-randconfig:
     CONTAINER: archlinux:current-riscv64
     KBUILD_DEFCONFIG: tiny64_defconfig
     RANDCONFIG: y
-    EXTRA_FIXED_RANDCONFIG:
-      CONFIG_COVERAGE=n
+    <<: *riscv-fixed-randconfig
 
 # Power cross-build
 debian-bullseye-gcc-ppc64le:
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 12:39:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 12:39:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682189.1061296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV4-0007K4-2X; Fri, 16 Feb 2024 12:39:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682189.1061296; Fri, 16 Feb 2024 12:39:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV3-0007JU-SG; Fri, 16 Feb 2024 12:39:29 +0000
Received: by outflank-mailman (input) for mailman id 682189;
 Fri, 16 Feb 2024 12:39:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=acXU=JZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raxV3-0006oP-5c
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 12:39:29 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6c0f7a5b-ccc8-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 13:39:27 +0100 (CET)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-563c2b2bddbso1917233a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 04:39:27 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 lj8-20020a170907188800b00a3db1734b92sm1121608ejc.43.2024.02.16.04.39.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 04:39:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6c0f7a5b-ccc8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708087166; x=1708691966; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ThFYlqJK5oTZpY8W/cIg7Lb/8W/p8jm22dHZpYH2e2M=;
        b=g18ZyPL6DZGbI1XbJJ/pOsMnhC1qgil3AsgbM/i+4rUhJaEanKRMTbA8rjSXXtiZcq
         CkHzb7elCmt0CWRy6QgtDBiMF9ygwwg+10iEaY06w3EbzFQT1Fg43suOB3hyykQrD3Yd
         OT8n37upIIdCEbcPXEizjOFE6xJ0A58JHtevqDsBm+ocz88tHrCQaJPr5296Hu7fM9JK
         gHnhNr1Z2qAblPRGcGsRnNem8FnWmKeoOvH43N7MYdmraMQoH5vWWkyyljTKwZTDNu39
         ujYbk/oJOlnt0GRDnDfNwtgFMFfJNHOwiIZ6jsVMU459xqcl6X87G+QqxCwEMSD99Wsq
         YokQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708087166; x=1708691966;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ThFYlqJK5oTZpY8W/cIg7Lb/8W/p8jm22dHZpYH2e2M=;
        b=UvwUxUpFbLj/LrwE8tChBF4fzapf30xVhL4YeICQgcyJZVNO0kNOWnw+ZufYt0cs8f
         6i9mM0w1T1m9iUiVy1kiwv+s5RHiqfdWRgQCh5Ky5CS9w2OdnfiMhJIOoi5AWeHwFFy1
         2dzIAFfDmpI/KruXEpwtSKOjYShiO0kzX0RmuSUCN7jsHgQGQYgkNxGfXVtzt5VcrjHR
         sSpXdS3wBA5xtVkQuTQpw9R5Mf8ShKKJY3WEOnrwCMKU4yqoEp6/KBrJxczvnVhVwXiy
         coSj3nYaB/CCgq1y7PH2wVlOcAxzqfR4AG26IGXIGRt2r8lvyJkYY0QRYbK62bdPcgHu
         /fZw==
X-Gm-Message-State: AOJu0Ywd4nNW5BEXow7QsbJTKOrz2hdDhVEcpWUniau5MufleAFCOVH6
	V8edHqRl/7tTEZeJlcn0veCXWT1E0rsL9HDHQ/kIQGjkxe5DA+zkkITFrYf3
X-Google-Smtp-Source: AGHT+IFgq4auslyaCnWeuNKH+MtZ4S5D0tRHphnqTMD5UeBYjyISJtyIhIEUjTq7mMP+mc7a+GeRGA==
X-Received: by 2002:a17:906:2b09:b0:a3d:6592:a72 with SMTP id a9-20020a1709062b0900b00a3d65920a72mr3093778ejg.8.1708087166161;
        Fri, 16 Feb 2024 04:39:26 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v9 2/7] xen/asm-generic: introduce stub header monitor.h
Date: Fri, 16 Feb 2024 13:39:14 +0100
Message-ID: <296530c13b8400f6872d4afc8cc14e2fcfc7a2d5.1708086092.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708086091.git.oleksii.kurochko@gmail.com>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The header is shared between several archs so it is
moved to asm-generic.

Switch partly Arm and PPC to asm-generic/monitor.h and only
arch_monitor_get_capabilities() left in arch-specific/monitor.h.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
Changes in V9:
 - Add Acked-by: Julien Grall <jgrall@amazon.com> for Arm part
---
Changes in V8:
 - Add Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
---
Changes in V7:
 - Drop definition of arch_monitor_domctl_event for PPC.
 - define arch_monitor_domctl_event in asm-generic/monitor.h.
 - add "define HAS_ARCH_MONITOR_DOMCTL_EVENT" in arm/.../monitor.h as it has arch specific implementation.
---
Changes in V6:
 - Rebase only.
---
Changes in V5:
  - Switched partly Arm and PPC to asm-generic monitor.h only
    arch_monitor_get_capabilities() left in arch-specific/monitor.h.
  - Updated the commit message.
---
Changes in V4:
 - Removed the double blank line.
 - Added Acked-by: Jan Beulich <jbeulich@suse.com>.
 - Update the commit message
---
Changes in V3:
 - Use forward-declaration of struct domain instead of " #include <xen/sched.h> ".
 - Add ' include <xen/errno.h> '
 - Drop PPC's monitor.h.
---
Changes in V2:
	- remove inclusion of "+#include <public/domctl.h>"
	- add "struct xen_domctl_monitor_op;"
	- remove one of SPDX tags.
---
 xen/arch/arm/include/asm/monitor.h | 25 +-----------
 xen/arch/ppc/include/asm/monitor.h | 28 +------------
 xen/arch/ppc/stubs.c               |  8 ----
 xen/include/asm-generic/monitor.h  | 64 ++++++++++++++++++++++++++++++
 4 files changed, 67 insertions(+), 58 deletions(-)
 create mode 100644 xen/include/asm-generic/monitor.h

diff --git a/xen/arch/arm/include/asm/monitor.h b/xen/arch/arm/include/asm/monitor.h
index 7567be66bd..77a3c1a36c 100644
--- a/xen/arch/arm/include/asm/monitor.h
+++ b/xen/arch/arm/include/asm/monitor.h
@@ -25,34 +25,13 @@
 #include <xen/sched.h>
 #include <public/domctl.h>
 
-static inline
-void arch_monitor_allow_userspace(struct domain *d, bool allow_userspace)
-{
-}
+#define HAS_ARCH_MONITOR_DOMCTL_EVENT
 
-static inline
-int arch_monitor_domctl_op(struct domain *d, struct xen_domctl_monitor_op *mop)
-{
-    /* No arch-specific monitor ops on ARM. */
-    return -EOPNOTSUPP;
-}
+#include <asm-generic/monitor.h>
 
 int arch_monitor_domctl_event(struct domain *d,
                               struct xen_domctl_monitor_op *mop);
 
-static inline
-int arch_monitor_init_domain(struct domain *d)
-{
-    /* No arch-specific domain initialization on ARM. */
-    return 0;
-}
-
-static inline
-void arch_monitor_cleanup_domain(struct domain *d)
-{
-    /* No arch-specific domain cleanup on ARM. */
-}
-
 static inline uint32_t arch_monitor_get_capabilities(struct domain *d)
 {
     uint32_t capabilities = 0;
diff --git a/xen/arch/ppc/include/asm/monitor.h b/xen/arch/ppc/include/asm/monitor.h
index e5b0282bf1..89000dacc6 100644
--- a/xen/arch/ppc/include/asm/monitor.h
+++ b/xen/arch/ppc/include/asm/monitor.h
@@ -6,33 +6,7 @@
 #include <public/domctl.h>
 #include <xen/errno.h>
 
-static inline
-void arch_monitor_allow_userspace(struct domain *d, bool allow_userspace)
-{
-}
-
-static inline
-int arch_monitor_domctl_op(struct domain *d, struct xen_domctl_monitor_op *mop)
-{
-    /* No arch-specific monitor ops on PPC. */
-    return -EOPNOTSUPP;
-}
-
-int arch_monitor_domctl_event(struct domain *d,
-                              struct xen_domctl_monitor_op *mop);
-
-static inline
-int arch_monitor_init_domain(struct domain *d)
-{
-    /* No arch-specific domain initialization on PPC. */
-    return 0;
-}
-
-static inline
-void arch_monitor_cleanup_domain(struct domain *d)
-{
-    /* No arch-specific domain cleanup on PPC. */
-}
+#include <asm-generic/monitor.h>
 
 static inline uint32_t arch_monitor_get_capabilities(struct domain *d)
 {
diff --git a/xen/arch/ppc/stubs.c b/xen/arch/ppc/stubs.c
index a96e45626d..da193839bd 100644
--- a/xen/arch/ppc/stubs.c
+++ b/xen/arch/ppc/stubs.c
@@ -95,14 +95,6 @@ void arch_get_domain_info(const struct domain *d,
     BUG_ON("unimplemented");
 }
 
-/* monitor.c */
-
-int arch_monitor_domctl_event(struct domain *d,
-                              struct xen_domctl_monitor_op *mop)
-{
-    BUG_ON("unimplemented");
-}
-
 /* smp.c */
 
 void arch_flush_tlb_mask(const cpumask_t *mask)
diff --git a/xen/include/asm-generic/monitor.h b/xen/include/asm-generic/monitor.h
new file mode 100644
index 0000000000..1ade289099
--- /dev/null
+++ b/xen/include/asm-generic/monitor.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * include/asm-generic/monitor.h
+ *
+ * Arch-specific monitor_op domctl handler.
+ *
+ * Copyright (c) 2015 Tamas K Lengyel (tamas@tklengyel.com)
+ * Copyright (c) 2016, Bitdefender S.R.L.
+ *
+ */
+
+#ifndef __ASM_GENERIC_MONITOR_H__
+#define __ASM_GENERIC_MONITOR_H__
+
+#include <xen/errno.h>
+#include <xen/lib.h>
+
+struct domain;
+struct xen_domctl_monitor_op;
+
+static inline
+void arch_monitor_allow_userspace(struct domain *d, bool allow_userspace)
+{
+}
+
+static inline
+int arch_monitor_domctl_op(struct domain *d, struct xen_domctl_monitor_op *mop)
+{
+    /* No arch-specific monitor ops on GENERIC. */
+    return -EOPNOTSUPP;
+}
+
+#ifndef HAS_ARCH_MONITOR_DOMCTL_EVENT
+static inline
+int arch_monitor_domctl_event(struct domain *d,
+                              struct xen_domctl_monitor_op *mop)
+{
+    BUG_ON("unimplemented");
+}
+#endif
+
+static inline
+int arch_monitor_init_domain(struct domain *d)
+{
+    /* No arch-specific domain initialization on GENERIC. */
+    return 0;
+}
+
+static inline
+void arch_monitor_cleanup_domain(struct domain *d)
+{
+    /* No arch-specific domain cleanup on GENERIC. */
+}
+
+#endif /* __ASM_GENERIC_MONITOR_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: BSD
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 12:39:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 12:39:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682186.1061271 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV2-0006pa-A8; Fri, 16 Feb 2024 12:39:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682186.1061271; Fri, 16 Feb 2024 12:39:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxV2-0006pQ-5O; Fri, 16 Feb 2024 12:39:28 +0000
Received: by outflank-mailman (input) for mailman id 682186;
 Fri, 16 Feb 2024 12:39:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=acXU=JZ=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1raxV0-0006oJ-FQ
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 12:39:26 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6abd6fc5-ccc8-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 13:39:25 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a3d01a9a9a2so184695566b.1
 for <xen-devel@lists.xenproject.org>; Fri, 16 Feb 2024 04:39:25 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 lj8-20020a170907188800b00a3db1734b92sm1121608ejc.43.2024.02.16.04.39.21
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 16 Feb 2024 04:39:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6abd6fc5-ccc8-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708087164; x=1708691964; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=ix0GNJqsQY7P5B/iyQn8vNTQQD332oz4c2RQO4AEM3c=;
        b=FHxtpXhMfDDQRPM5OaH+TxT1r4tIxNHpeAP5dlDfKqBb2IHlusXfQGgZKhpYnzSHMS
         lVs8JJgW1DXRjRlP+QpsqeVMFmHTLPu4IinCUYnH5+3+uzaezD+muNAo8EL6iNGn1gyy
         hDOd1Vfgebfjxix3XBqzbA7wi90I9qrTeXNzzwEHiiKypL0c+eY/gxNBm01DZwuQjsoT
         RNB8fQnfyaHWY+L3K4JdTqlQdkH+1q1Yh7fyPTJzzaU1TJnuCzSCCOPqJK/dRRMWn2Sv
         sm4IQbhdhZBBjryIcdvDRCOqc1BvuBT8hFMG0yViARRXn76mn11IQDC0wb/z2tbZ32Jo
         rq/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708087164; x=1708691964;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=ix0GNJqsQY7P5B/iyQn8vNTQQD332oz4c2RQO4AEM3c=;
        b=UT9KyguENxAsYVDYD2ntITVK5gM9XpRtbuAPyUAjlLIcfBSyBhL3hbumLTAvhXlrfd
         izD3WMySnj0eBOeXYoJftmnCGYJ8SXtEmDKUwmqxzYRjBCdBosD9t7eOIUj37XqzQ74F
         uVypA+FGEvzbV7VoXVKU7NVCRe0VlA+zIKTXMLN11Fmd23Xkwt9mWxzkME1CDmBhbcOX
         ATbxFQf/I7Gv6HTE9IO4bBw3B5qkOJYNxLIEVL+MVXs0Y2Fc3hzZxNSjjGpJkDYEE23j
         CAyCaynuswsj5wQy/fGzaAACpKBOW2HvVBCi8iNpNc8r9pxXMvh6ctp5NelR+T444W+p
         ByIQ==
X-Gm-Message-State: AOJu0YwqMsHKtxwlSkSaqrSrBA7iTjhdogrDfx4Avvz6BWi3PxmXbjkh
	MMRYt7D1iKflPGeRSA4HzCjzgMshm2lA+4WFFHU2VNjaXiIOjBiPpAZwdd2U
X-Google-Smtp-Source: AGHT+IGPCybVo7NkOXXjlMcCpidXhjfdiZDGsbEWe65qPa/TuwavZOpceHWpFx4lMGwHkbrbWL+fFQ==
X-Received: by 2002:a17:906:1718:b0:a3c:e81a:9c35 with SMTP id c24-20020a170906171800b00a3ce81a9c35mr3534774eje.73.1708087163512;
        Fri, 16 Feb 2024 04:39:23 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>,
	Rahul Singh <rahul.singh@arm.com>
Subject: [PATCH v9 0/7]  Introduce generic headers
Date: Fri, 16 Feb 2024 13:39:12 +0100
Message-ID: <cover.1708086091.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Some headers are common between several architectures, so the current patch series
provide them.

Another one reason to have them as generic is a simplification of adding support
necessary to make a complete Xen build as it was/is being done in the patch series [1]
and [2].

Also, instead of providing generic/stub headers, it was used
"#ifdef CONFIG_* #include <asm/*.h> #endif" instead of providing empty headers.

This patch series is a pre-requisite for "Enable build of full Xen for RISC-V" [3].

[1] https://lore.kernel.org/xen-devel/cover.1694543103.git.sanastasio@raptorengineering.com/
[2] https://lore.kernel.org/xen-devel/cover.1692181079.git.oleksii.kurochko@gmail.com/
[3] https://lore.kernel.org/xen-devel/cover.1700761381.git.oleksii.kurochko@gmail.com/
---
Changes in V9:
 - Drop the patch "[PATCH v8 1/7] automation: ensure values in EXTRA_FIXED_RANDCONFIG
   are separated by new line" as better solution was suggested.
 - introduce new follow-up patch for "xen/asm-generic: introduce generic device.h"
   ( it can be merged with patches 4,5 of this patch series ).
 - All other changes please find in revision log to a patch.
---
Changes in V8:
 - Add Acked-by Acked-by: Tamas K Lengyel <tamas@tklengyel.com> for patches:
    [PATCH v7 4/7] xen/asm-generic: ifdef inclusion of <asm/mem_access.h>
    [PATCH v7 3/7] xen/asm-generic: introduce stub header monitor.h
 - Update the commit messages for patches:
    [PATCH v7 5/7] xen/asm-generic: introduce generic device.h
    [PATCH v7 6/7] xen/arm: switch Arm to use asm-generic/device.h
 - Drop #ifdef HAS_PCI in asm-generic/device.h to be consistent with other pci-related
   things in the header.
 - Code styles fixes.    
---
Changes in V7:
 - The following patches were dropped because of rebasing as they were merged
   to staging:
   - [PATCH v6 3/9] xen/asm-generic: introduce generic div64.h header
   - [PATCH v6 6/9] xen/asm-generic: introduce stub header softirq.h
   - [PATCH v6 7/9] xen: ifdef inclusion of <asm/grant_table.h> in <xen/grant_table.h>
   - [PATCH v6 8/9] xen/asm-generic: ifdef inclusion of <asm/mem_access.h>
- [PATCH v6 5/9] xen/asm-generic: introduce stub header numa.h was dropped becaus of
  the patch: "[PATCH v2] NUMA: no need for asm/numa.h when !NUMA"
- Drop definition of arch_monitor_domctl_event for PPC.
- define arch_monitor_domctl_event in asm-generic/monitor.h.
- Add "define HAS_ARCH_MONITOR_DOMCTL_EVENT" in arm/.../monitor.h as it has arch
  specific implementation.
- keeping DEVICE_PCI_HOSTBRIDGE available for every build based on the reply:
    https://lore.kernel.org/xen-devel/926a5c12-7f02-42ec-92a8-1c82d060c710@xen.org/
- add comment above enum device_type.h with explanation about DEV_TYPE_MAX.        
- drop #ifdef HAS_PCI around "(device_get_class(dev) == DEVICE_PCI_HOSTBRIDGE))" in ARM code.
- separate patch  "[PATCH v6 9/9] xen/asm-generic: introduce generic device.h" into 3 patches.
---
Changes in V6:
 - Fix the build script to work properly with EXTRA_FIXED_RANDCONFIG.
 - Introduce separate randconfig yaml with fixed configs for RISC-V.
 - Disable CONFIG_GRANT_TABLE and CONFIG_MEM_ACCESS for PPC and RISC-V.
 - Remove change in Kconfig/common for CONFIG_GRANT_TABLE and CONFIG_MEM_ACCESS.
 - Rebase on top of the latest staging. 
---
Changes in V5:
 - Update the patch series message as patch related to delay.h was merged.
 - Rebase on top of staging because half of the patches of the patch series were
   merged to staging branch.
 - Add A-by for some of the patches.
 - Add "depends on X86 || Arm" for CONFIG_GRANT_TABLE and CONFIG_MEM_ACCESS to be
   sure it won't be turned on by randconfig in CI.
 - Partly switch Arm and PPC to asm-generic/monitor.h.
 - Some other minor changes
---
Changes in V4:
 - Update the cover letter message
 - Add Reviewed-by/Acked-by for patches:
    [PATCH v3 01/14] xen/asm-generic: introduce stub header paging.h
    [PATCH v3 03/14] xen/asm-generic: introduce generic hypercall.h
    [PATCH v3 04/14] xen/asm-generic: introduce generic header iocap.h
    [PATCH v3 05/14] xen/asm-generic: introduce stub header <asm/random.h>
    [PATCH v3 06/14] xen/asm-generic: introduce generic header percpu.h
    [PATCH v3 07/14] xen/asm-generic: introduce generalized hardirq.h
    [PATCH v3 08/14] xen/asm-generic: introduce generic div64.h header
    [PATCH v3 09/14] xen/asm-generic: introduce generic header altp2m.h
    [PATCH v3 10/14] xen/asm-generic: introduce stub header monitor.h
    [PATCH v3 11/14] xen/asm-generic: introduce stub header numa.h
    [PATCH v3 12/14] xen/asm-generic: introduce stub header softirq.h
 - Fix some code style and minor issues.
 - Use asm-generic version of device.h for Arm and PPC.
---
Changes in V3:
 - Update the commit message of the cover letter.
 - Drop the following patch as it can be arch-specific enough:
   * [PATCH v2 09/15] xen/asm-generic: introduce generic header smp.h
 - Drop correspondent arch specific headers and use asm-generic version of
   a header.
 - Back to the patch series patches:
   * xen: ifdef inclusion of <asm/grant_table.h> in <xen/grant_table.h>
   * xen/asm-generic: ifdef inclusion of <asm/mem_access.h>
---
Changes in V2:
 - Update the commit message of the cover letter.
 - Drop the following patches because they are arch-specific or was sent as a separate patch:
   - xen/asm-generic: introduce stub header event.h
	 - xen/asm-generic: introduce stub header spinlock.h
	 - [PATCH v1 03/29] xen/asm-generic: introduce stub header cpufeature.h
	 - [PATCH v1 07/29] xen/asm-generic: introduce stub header guest_atomics.h
	 - [PATCH v1 10/29] xen/asm-generic: introduce stub header iommu.h
	 - [PATCH v1 12/29] xen/asm-generic: introduce stub header pci.h because separate patch was sent [5]
	 - [PATCH v1 14/29] xen/asm-generic: introduce stub header setup.h
	 - [PATCH v1 15/29] xen/asm-generic: introduce stub header xenoprof.h because of [3].
	 - [PATCH v1 16/29] xen/asm-generic: introduce stub header flushtlb.h
	 - [PATCH v1 22/29] xen/asm-generic: introduce stub header delay.h because of [3]
	 - [PATCH v1 23/29] xen/asm-generic: introduce stub header domain.h
	 - [PATCH v1 24/29] xen/asm-generic: introduce stub header guest_access.h
	 - [PATCH v1 25/29] xen/asm-generic: introduce stub header irq.h ( probably not so generic as I expected, I'll back to it if it will be necessary in the future )
	 - [PATCH v1 28/29] xen/asm-generic: introduce stub header p2m.h ( probably not so generic as I expected, I'll back to it if it will be necessary in the future )
 - For the rest of the patches please look at changes for each patch separately.
---

Oleksii Kurochko (7):
  automation: introduce fixed randconfig for RISC-V
  xen/asm-generic: introduce stub header monitor.h
  xen/asm-generic: ifdef inclusion of <asm/mem_access.h>
  xen/asm-generic: introduce generic device.h
  xen/arm: switch Arm to use asm-generic/device.h
  xen/ppc: switch PPC to use asm-generic/device.h
  xen/asm-generic: fold struct devarch into struct dev

 automation/gitlab-ci/build.yaml               |  14 ++-
 xen/arch/arm/device.c                         |   5 +
 xen/arch/arm/domain_build.c                   |   2 +-
 xen/arch/arm/gic-v2.c                         |   4 +-
 xen/arch/arm/gic-v3.c                         |   6 +-
 xen/arch/arm/gic.c                            |   4 +-
 xen/arch/arm/include/asm/Makefile             |   1 +
 xen/arch/arm/include/asm/monitor.h            |  25 +---
 xen/arch/arm/p2m.c                            |   1 +
 xen/arch/arm/traps.c                          |   1 +
 xen/arch/arm/xen.lds.S                        |   2 +
 xen/arch/ppc/configs/ppc64_defconfig          |   1 +
 xen/arch/ppc/include/asm/Makefile             |   1 +
 xen/arch/ppc/include/asm/device.h             |  53 --------
 xen/arch/ppc/include/asm/mem_access.h         |   5 -
 xen/arch/ppc/include/asm/monitor.h            |  28 +----
 xen/arch/ppc/stubs.c                          |   8 --
 xen/arch/riscv/configs/tiny64_defconfig       |   1 +
 xen/drivers/passthrough/arm/smmu.c            |  12 +-
 .../asm => include/asm-generic}/device.h      | 113 ++++++++++--------
 xen/include/asm-generic/monitor.h             |  64 ++++++++++
 xen/include/xen/mem_access.h                  |   2 +
 22 files changed, 170 insertions(+), 183 deletions(-)
 delete mode 100644 xen/arch/ppc/include/asm/device.h
 delete mode 100644 xen/arch/ppc/include/asm/mem_access.h
 rename xen/{arch/arm/include/asm => include/asm-generic}/device.h (70%)
 create mode 100644 xen/include/asm-generic/monitor.h

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 12:48:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 12:48:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682231.1061351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxdr-0003iq-Gt; Fri, 16 Feb 2024 12:48:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682231.1061351; Fri, 16 Feb 2024 12:48:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1raxdr-0003ij-Du; Fri, 16 Feb 2024 12:48:35 +0000
Received: by outflank-mailman (input) for mailman id 682231;
 Fri, 16 Feb 2024 12:48:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=AOVM=JZ=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1raxdq-0003id-Mn
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 12:48:34 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b0abecee-ccc9-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 13:48:32 +0100 (CET)
Received: from CH0PR03CA0397.namprd03.prod.outlook.com (2603:10b6:610:11b::30)
 by PH8PR12MB7349.namprd12.prod.outlook.com (2603:10b6:510:217::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26; Fri, 16 Feb
 2024 12:48:27 +0000
Received: from CH2PEPF00000099.namprd02.prod.outlook.com
 (2603:10b6:610:11b:cafe::70) by CH0PR03CA0397.outlook.office365.com
 (2603:10b6:610:11b::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26 via Frontend
 Transport; Fri, 16 Feb 2024 12:48:27 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CH2PEPF00000099.mail.protection.outlook.com (10.167.244.20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Fri, 16 Feb 2024 12:48:27 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 16 Feb
 2024 06:48:26 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 16 Feb 2024 06:48:26 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b0abecee-ccc9-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KaBPvpiTgTP8UZ7kxZSavi9pcxK3HqNVdhI36DuseF3XGDEUI+OJjReyJmStFsGXMfFjvnNS6qn1dQia7VGnUlWjWaRonptmim7LGSzwvs7d4pnqCL4q3hClFdyQHGZ5hBrCPUZDBcELb1O0/ehwgUlZTLOsqqfnrBpoo89hlPL3iRaXvOjLoz9jT62+B6nkSmNmHF85hcNm942zLQdY04UTfYb/bO0oY3jLINkNMS00ZPrx5NTk2Z3MqsRrqsjujKNJLxd3YfUQteH4Wh0BBAhCEJ07VOb2aNEhNqObO1jpzOEBwXK0xJ39XILCV7u1sZ+yEESCsLBRopTbg8kdFA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=JAxFwtbfnmDfEgXkrnAnZHx5SMdcWKlyFBhUESUvpoM=;
 b=UXGUMW/UIDb2Dd6kXO52BHzJFJMhdFnKyTof3zx1g1sFHek8eyriJZ3d4q29fF7ebn7IWv+qMLfGUiWXIEIeoXRcLuhTHh8TUgYo5rnf/kFMl3T60efu8THEy/A1VrSrtvVDMhDZfr1dR3hoUK+GHvPFd+bZNOlp2WJUjpk6Zmv49PETJ5ihd9MxQi3uIGXhwdVsDz7HeD31//qtAow6DvdQJAnYgzqVrYMjEPPiYutLxZa5McLEzA95UNKR1jEjjrRxN2BQIJftlShRH2UoPUQNCA6GY1G7MBKJ2y0GTGm/IKTXiN0y4ZcC1sZlUx95m9+8G4lkTOPvX0TucIcd1Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=gmail.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JAxFwtbfnmDfEgXkrnAnZHx5SMdcWKlyFBhUESUvpoM=;
 b=tmuEggIr5VnNZJHwsOywUn2TkvFTvl7OZ17Sn/c8komau4Asq84zSqHECPw20+HSiQhebd+Cy1k4yZWvHk0jpokI6wUQEYnBneZRIsXPfHuIobP6OaDpHN5iisWfDbYI4JFeU4OIZsIPwoXeY6iqOCfNH9qLnR4Ovzx/FE5t7qs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <6240bdb3-ab07-49d8-a0ee-30e01cb12554@amd.com>
Date: Fri, 16 Feb 2024 13:48:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 1/7] automation: introduce fixed randconfig for RISC-V
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	<xen-devel@lists.xenproject.org>
CC: Doug Goldstein <cardoe@cardoe.com>, Stefano Stabellini
	<sstabellini@kernel.org>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
 <d8e330c26394d9415a0d76b2d951bf57b0b6dd4b.1708086092.git.oleksii.kurochko@gmail.com>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <d8e330c26394d9415a0d76b2d951bf57b0b6dd4b.1708086092.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CH2PEPF00000099:EE_|PH8PR12MB7349:EE_
X-MS-Office365-Filtering-Correlation-Id: ffa84830-a267-4f6f-d3f0-08dc2eed9226
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	2TjH8dcOr4gbULPhPbmt4S0FEo4d6fPQcJcVdGLlrvi5vuv7jjtDFrQMrcHlMMsWuUvDY+D3+Z6ABzYX0NcLpdwHvQcV6odGZIv+IYhNBDV0yf1F+6l47Jo15wOPjrgFbfPDimbaUJY/NLHg1xM0ODya1EyS+4sN9yOucywSMrMfWCzqfCTDr10CQ0c7vfUrPb/qOA7oAlfC4ILJf+fc3dsObiX/lr61kSaCHBbmfBL0/F+sQXIYMCqMy/xIyugq48J7m/n/jof9EUIs3+hezF6Tz6TQtQvX81S/ug2ZF4YnTUaLYgOlqrtZnhlKnOVvh7v0ypvz9VHLNOlwpsVUxGhe7GBivq74DaRa31VVDGi+00xEbRcWae0IByeQVEWYWP/qxIswfsoxhnZfkQetYjJMOkGCfW0tvx/T2RGRLK9JdGu3epZfoe053TTav+Qo6VQYdH6nx3U2+jsC0hyq9IL5PHWNpZlIV761+Hqwxinlf3cFMyr8kOAWOk5d3kpGoR7B0eh7imreLANDsEoRp3Vuw6fxA4pWxTISL3dxTcQDxxXvoYNAagtDXbWcrmo4wuZojbeaR00Rj/Nveh+uR9wodg2CvuJF+msDLwvP6MbRZNcAmqGz2sd7NmKrW/3CLIlnvnpH7O0r7+0R7QNOFokNorLEmapOO9+BWOQ6C8M=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(396003)(346002)(39860400002)(376002)(230922051799003)(451199024)(36860700004)(82310400011)(64100799003)(1800799012)(186009)(40470700004)(46966006)(31696002)(31686004)(86362001)(5660300002)(8676002)(8936002)(82740400003)(36756003)(2906002)(26005)(44832011)(4744005)(4326008)(2616005)(336012)(81166007)(426003)(356005)(478600001)(83380400001)(70206006)(70586007)(316002)(54906003)(41300700001)(16576012)(53546011)(110136005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Feb 2024 12:48:27.3287
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ffa84830-a267-4f6f-d3f0-08dc2eed9226
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CH2PEPF00000099.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7349

Hi,

On 16/02/2024 13:39, Oleksii Kurochko wrote:
> 
> 
> This patch introduces the anchor riscv-fixed-randconfig,
> which includes all configurations that should be disabled for
> randconfig builds.
> 
> Suggested-by: Stefano Stabellini <sstabellini@kernel.org>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 13:44:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 13:44:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682256.1061361 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rayVQ-0005En-LD; Fri, 16 Feb 2024 13:43:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682256.1061361; Fri, 16 Feb 2024 13:43:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rayVQ-0005Eg-IE; Fri, 16 Feb 2024 13:43:56 +0000
Received: by outflank-mailman (input) for mailman id 682256;
 Fri, 16 Feb 2024 13:43:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rayVP-0005EU-CA; Fri, 16 Feb 2024 13:43:55 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rayVP-0002I4-3g; Fri, 16 Feb 2024 13:43:55 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rayVO-0006En-L0; Fri, 16 Feb 2024 13:43:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rayVO-00039I-KE; Fri, 16 Feb 2024 13:43:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=k7UmV2LpX0JxGr7dBGtAFckW621MpxwrGlbYo0rUtJ4=; b=5ouOF/PY36itkCc71wqD6L5RUo
	rbiUR8R4mlcnUM1nbTq/YKTdjNFEkEstWB5OXH+8OEekpCqDHdIwX+n8w74hqqJwCD0rR3g0VHR+p
	5JSM5lxDSq1CTdohqzQGVg715qyZ/CSpIimL9cs1KITQW33gN+Mr9G9TW4hPkauO80BU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184682-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184682: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=50bca3d45b58a6dd51d3792cc856f4f49fe88bea
X-Osstest-Versions-That:
    libvirt=40ebade891a0d2c08e82dac126822d5ee57f4281
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 16 Feb 2024 13:43:54 +0000

flight 184682 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184682/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184662
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184662
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184662
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              50bca3d45b58a6dd51d3792cc856f4f49fe88bea
baseline version:
 libvirt              40ebade891a0d2c08e82dac126822d5ee57f4281

Last test of basis   184662  2024-02-14 04:20:33 Z    2 days
Testing same since   184682  2024-02-16 04:20:41 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Göran Uddeborg <goeran@uddeborg.se>
  김인수 <simmon@nplob.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   40ebade891..50bca3d45b  50bca3d45b58a6dd51d3792cc856f4f49fe88bea -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 14:41:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 14:41:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682263.1061371 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1razPB-0004H3-QT; Fri, 16 Feb 2024 14:41:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682263.1061371; Fri, 16 Feb 2024 14:41:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1razPB-0004Gw-N1; Fri, 16 Feb 2024 14:41:33 +0000
Received: by outflank-mailman (input) for mailman id 682263;
 Fri, 16 Feb 2024 14:41:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=noRu=JZ=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1razPA-0004Gq-GV
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 14:41:32 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 77feb3ba-ccd9-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 15:41:29 +0100 (CET)
Received: from PR1P264CA0156.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:346::12)
 by SJ1PR12MB6123.namprd12.prod.outlook.com (2603:10b6:a03:45a::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.28; Fri, 16 Feb
 2024 14:41:24 +0000
Received: from SA2PEPF00001504.namprd04.prod.outlook.com
 (2603:10a6:102:346:cafe::ff) by PR1P264CA0156.outlook.office365.com
 (2603:10a6:102:346::12) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.20 via Frontend
 Transport; Fri, 16 Feb 2024 14:41:23 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SA2PEPF00001504.mail.protection.outlook.com (10.167.242.36) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Fri, 16 Feb 2024 14:41:22 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 16 Feb
 2024 08:41:22 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 16 Feb
 2024 08:41:21 -0600
Received: from [172.27.131.30] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Fri, 16 Feb 2024 08:41:19 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 77feb3ba-ccd9-11ee-8a4f-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OQg5BSsMemhu6tbcT2LSqb5IouiXToJerTbdsDAuDa1QAy//5qG30r5aHsUFX6vC0HGW2ao7MrWDOmsZ/wsttDG4IdOLvgFqLuzp9xlFjhIGCy0LIDFDvJ20PfQ+p0Ca4qIyIWcIkYAWOdCKskJT0+S6/ZTgwQONzBHzbR2/D/AUUCSUm5TiXzRTM4i7lDfHni0mqsn1Sh+iNBU/WRHp3RNK05GnV/v4qioW5NsTBTfjRjurf0xswxxhlT2i2PEbSRI/fMZoaPKxVu09XmFQxXtyShe5RzltXJbi4x2fm8kp4N/ZRaw0iXWsWfELu1nkZNfcE6Ba0+Hy5PKpm+/u7g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zVpZ5CiPG75/pzA0IJfoQQfFkGBTRxHvZPoM3ewu2mo=;
 b=NbeFmJCAjBpFHOWduPtL00iO1ptykzeMCGom34quX1e/pAQ7f3HREsKBFWUcbOZzvwOZSHwjwIhepkCllgCLElbYV/GV77GGf070EA0OzUnYJCDAsdPFOdWrAepgysfNYNCtkFQOaL1eZPweUlmYH7vN9SZhIxSROZGu7C2lfrI5Ml7PEnGmLmtG4yhx65Bx0OzrIctUewv/ARajsM620ehFh+PmbvYrrPEXSzsEn6ENwP8Jo+qvoMa8/YyZBfgFYfcRvry0p+XFl6qFRKV2oTq6hu3OZSPeHcg3JdzyxT1+K20Z9wFdfc/LYO2pOIv33lyAPDvVRFTT1btxaIdbnA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=citrix.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zVpZ5CiPG75/pzA0IJfoQQfFkGBTRxHvZPoM3ewu2mo=;
 b=x0LxswawzpPNGvS0/GcIp3LfkGYZOtxzk3QDQv3MLoWHigik6VL8P3h3UgA8AVzKMZhBAfcOlj8RAVFAB7V3D1IsUIGRvJROoBrPMkXppAdsDk55wogcXsJBJWed8K8YcLX+jz74Rf3DnDrxMFKBKMBWS6N0bgCxuTT6R6faCzM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <f1e43851-8a3d-423d-b0da-4d881f8ecf91@amd.com>
Date: Fri, 16 Feb 2024 09:41:19 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13.1 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
CC: <xen-devel@lists.xenproject.org>, Oleksandr Andrushchenko
	<oleksandr_andrushchenko@epam.com>, Jan Beulich <jbeulich@suse.com>, "Andrew
 Cooper" <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian
	<kevin.tian@intel.com>, Paul Durrant <paul@xen.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <20240215203001.1816811-1-stewart.hildebrand@amd.com>
 <Zc9KuCeoOciUdqTN@macbook>
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <Zc9KuCeoOciUdqTN@macbook>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF00001504:EE_|SJ1PR12MB6123:EE_
X-MS-Office365-Filtering-Correlation-Id: 9a60692f-6888-404c-d922-08dc2efd5894
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	SNMgxq8W/y2N44kHvPDevAvS1m+PqkT5woaqTx+DiSI7wt/ychznzz49Ky6gedoq5aZIQr1VDxO0FTfJsIId3d1kIWVsL86/pTxCJcRGZp/xAM2pJkqfbPP+ywv7apCVBPbFwNbbEnwWwaTwkSMXWMyaiUFZYfAiyxvE2sANKRVagB2eBNWZ0QTJMuVfVN1PEHAsX5u37xvILPiwJNRlOpx/VMUVuYGXCTKJBFJhNzAICfBtJ+wgS2lhr5Yrsj6jh2oRgJRzjEwvRGSZv3lO5ZKqlUKn7CQEvJ9MYswHyvjiQYqqfJxr0gsC5NimPhatHPG99BeVIabY5QGXcRqdabSs/a+xOGNHg8RHzHG7uyGTi/7JS6s8X8HIb/U1wFu4NjoUT4fwQ1wzGEvoS11mFzjnh18E5Uh74aYFdRHeQwoCS9VbOt4izAbpn8LwIO+WwuqpKRvjMWUHXg2eOJeDNYJMWlE8OpY9zZoEulcYyzrzy93PHdpddEMCLLL4a0RjyKTKz3j63u0HaJLXralwKTVrQm6ZqhUK0UZfvmpoMjW1cLtCT5iKsZN44nNp6Wbg31T8pCVHFbq9zhIM7dWqEd0+DrwR4rKbBDUvP/yd3Ei3F0RkNa0D1jnfr6SRNU3o7mfeQAeavsPm9+XhZO5pUSNrUqBtxenCYSRH3Sd/zZo=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(396003)(376002)(346002)(136003)(230922051799003)(186009)(82310400011)(451199024)(1800799012)(64100799003)(36860700004)(40470700004)(46966006)(31686004)(53546011)(41300700001)(2616005)(478600001)(70206006)(7416002)(4326008)(5660300002)(44832011)(8676002)(8936002)(2906002)(6916009)(70586007)(86362001)(16576012)(316002)(83380400001)(26005)(31696002)(336012)(36756003)(54906003)(81166007)(426003)(82740400003)(356005);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Feb 2024 14:41:22.6896
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 9a60692f-6888-404c-d922-08dc2efd5894
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF00001504.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6123

On 2/16/24 06:44, Roger Pau Monné wrote:
> On Thu, Feb 15, 2024 at 03:30:00PM -0500, Stewart Hildebrand wrote:
>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>
>> Use the per-domain PCI read/write lock to protect the presence of the
>> pci device vpci field. This lock can be used (and in a few cases is used
>> right away) so that vpci removal can be performed while holding the lock
>> in write mode. Previously such removal could race with vpci_read for
>> example.
>>
>> When taking both d->pci_lock and pdev->vpci->lock, they should be
>> taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
>> possible deadlock situations.
>>
>> 1. Per-domain's pci_lock is used to protect pdev->vpci structure
>> from being removed.
>>
>> 2. Writing the command register and ROM BAR register may trigger
>> modify_bars to run, which in turn may access multiple pdevs while
>> checking for the existing BAR's overlap. The overlapping check, if
>> done under the read lock, requires vpci->lock to be acquired on both
>> devices being compared, which may produce a deadlock. It is not
>> possible to upgrade read lock to write lock in such a case. So, in
>> order to prevent the deadlock, use d->pci_lock in write mode instead.
>>
>> All other code, which doesn't lead to pdev->vpci destruction and does
>> not access multiple pdevs at the same time, can still use a
>> combination of the read lock and pdev->vpci->lock.
>>
>> 3. Drop const qualifier where the new rwlock is used and this is
>> appropriate.
>>
>> 4. Do not call process_pending_softirqs with any locks held. For that
>> unlock prior the call and re-acquire the locks after. After
>> re-acquiring the lock there is no need to check if pdev->vpci exists:
>>  - in apply_map because of the context it is called (no race condition
>>    possible)
>>  - for MSI/MSI-X debug code because it is called at the end of
>>    pdev->vpci access and no further access to pdev->vpci is made
>>
>> 5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
>> while accessing pdevs in vpci code.
>>
>> 6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
>> do not go away. The vPCI functions call several MSI-related functions
>> which already have existing non-vPCI callers. Change those MSI-related
>> functions to allow using either pcidevs_lock() or d->pci_lock for
>> ensuring pdevs do not go away. Holding d->pci_lock in read mode is
>> sufficient. Note that this pdev protection mechanism does not protect
>> other state or critical sections. These MSI-related functions already
>> have other race condition and state protection mechanims (e.g.
>> d->event_lock and msixtbl RCU), so we deduce that the use of the global
>> pcidevs_lock() is to ensure that pdevs do not go away.
>>
>> 7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
>> that pdevs do not go away. The purpose of this wrapper is to aid
>> readability and document the intent of the pdev protection mechanism.
>>
>> 8. When possible, the existing non-vPCI callers of these MSI-related
>> functions haven't been switched to use the newly introduced per-domain
>> pci_lock, and will continue to use the global pcidevs_lock(). This is
>> done to reduce the risk of the new locking scheme introducing
>> regressions. Those users will be adjusted in due time. One exception
>> is where the pcidevs_lock() in allocate_and_map_msi_pirq() is moved to
>> the caller, physdev_map_pirq(): this instance is switched to
>> read_lock(&d->pci_lock) right away.
>>
>> Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
>> Suggested-by: Jan Beulich <jbeulich@suse.com>
>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> 
> A couple of questions and the pdev_list_is_read_locked() needs a small
> adjustment.
> 
>> @@ -895,6 +891,14 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
>>  {
>>      unsigned int i;
>>  
>> +    /*
>> +     * Assert that d->pdev_list doesn't change. pdev_list_is_read_locked() is
>> +     * not suitable here because we may read_unlock(&pdev->domain->pci_lock)
>> +     * before returning.
> 
> I'm confused by this comment, as I don't see why it matters that the
> lock might be lock before returning.  We need to ensure the lock is
> taken at the time of the assert, and hence pdev_list_is_read_locked()
> can be used.

pdev_list_is_read_locked() currently would allow either pcidevs_lock()
or d->pci_lock. If vpci_msix_arch_print() is entered with only
pcidevs_lock() held, we may end up unlocking d->pci_lock when it is
not locked, which would be wrong.

Perhaps the comment could be clarified as:

    /*
     * Assert that d->pdev_list doesn't change. ASSERT_PDEV_LIST_IS_READ_LOCKED
     * is not suitable here because it may allow either pcidevs_lock() or
     * d->pci_lock to be held, but here we rely on d->pci_lock being held, not
     * pcidevs_lock().
     */

> 
>> +     */
>> +    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
>> +    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));
>> +
>>      for ( i = 0; i < msix->max_entries; i++ )
>>      {
>>          const struct vpci_msix_entry *entry = &msix->entries[i];
>> diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
>> index aabc5465a7d3..9f31cb84c9f3 100644
>> --- a/xen/include/xen/pci.h
>> +++ b/xen/include/xen/pci.h
>> @@ -171,6 +171,20 @@ void pcidevs_lock(void);
>>  void pcidevs_unlock(void);
>>  bool __must_check pcidevs_locked(void);
>>  
>> +#ifndef NDEBUG
>> +/*
>> + * Check for use in ASSERTs to ensure there will be no changes to the entries
>> + * in d->pdev_list (but not the contents of each entry).
>> + * This check is not suitable for protecting other state or critical regions.
>> + */
>> +#define pdev_list_is_read_locked(d) ({                             \
>> +        /* NB: d may be evaluated multiple times, or not at all */ \
>> +        pcidevs_locked() || (d && rw_is_locked(&d->pci_lock));     \
> 
> 'd' is missing parentheses here, should be (d).

Thanks for spotting. I'll fix in v13.2.

> 
>> +    })
>> +#else
>> +bool pdev_list_is_read_locked(const struct domain *d);
>> +#endif
> 
> FWIW, if this is only intended to be used with ASSERT, it might as
> well be an ASSERT itself:
> 
> ASSERT_PDEV_LIST_IS_READ_LOCKED(d) ...
> 
> Don't have a strong opinion, so I'm fine with how it's used, just
> noting it might be clearer if it was an ASSERT_ right away.

This would also have the benefit of not relying on dead code elimination
in the #else case. I'll be sending v13.2 anyway, I may as well make the
change.


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 16:31:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 16:31:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682281.1061400 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb180-0000kI-2P; Fri, 16 Feb 2024 16:31:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682281.1061400; Fri, 16 Feb 2024 16:31:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb17z-0000kB-Vz; Fri, 16 Feb 2024 16:31:55 +0000
Received: by outflank-mailman (input) for mailman id 682281;
 Fri, 16 Feb 2024 16:31:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LzRM=JZ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rb17z-0000EA-1k
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 16:31:55 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e456d40a-cce8-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 17:31:53 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id E9C861FB7D;
 Fri, 16 Feb 2024 16:31:52 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id BA14513343;
 Fri, 16 Feb 2024 16:31:52 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 1gzoK/iNz2VEVAAAn2gu4w
 (envelope-from <jgross@suse.com>); Fri, 16 Feb 2024 16:31:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e456d40a-cce8-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v8 2/8] tools/xl: support new 9pfs backend xen_9pfsd
Date: Fri, 16 Feb 2024 17:31:33 +0100
Message-Id: <20240216163139.9093-3-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240216163139.9093-1-jgross@suse.com>
References: <20240216163139.9093-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: E9C861FB7D
X-Spam-Flag: NO

Add support for the new 9pfs backend "xen_9pfsd". For this backend type
the tag defaults to "Xen" and the host side path to
"/var/log/xen/guests/<dom-name>".

Do most of the default settings in libxl. Unfortunately the default
path can't easily be set in libxl, as the domain name isn't available
in the related 9pfs specific function.

Settings the defaults in libxl requires to move the sanity checking
of 9pfs parameters from xl to libxl, too.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- test max_files and max_open_files, too (Jason Andryuk)
V4:
- fix man page to use the "xen_9pfsd" type due to idl limitation
  (Jason Andryuk)
- set (most of) the defaults in libxl (Anthony Perard)
---
 docs/man/xl.cfg.5.pod.in      | 36 +++++++++++++++++++++++++++++++++--
 tools/libs/light/libxl_9pfs.c | 18 ++++++++++++++++++
 tools/xl/xl_parse.c           | 23 +++++++++++++++++++---
 3 files changed, 72 insertions(+), 5 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index ea8d41727d..039e057318 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -772,10 +772,16 @@ settings, from the following list:
 
 =over 4
 
+=item B<type=TYPE>
+
+The backendtype for the PV device. Supported values are B<qemu> and
+B<xen_9pfsd>.  The default is B<qemu>.
+
 =item B<tag=STRING>
 
 9pfs tag to identify the filesystem share. The tag is needed on the
-guest side to mount it.
+guest side to mount it. For the backendtype of B<xen_9pfsd> the tag defaults to
+"Xen".
 
 =item B<security_model="none">
 
@@ -785,12 +791,38 @@ squash or remap).
 
 =item B<path=STRING>
 
-Filesystem path on the backend to export.
+Filesystem path on the backend to export. For the backendtype of B<xen_9pfsd>
+the path defaults to "@XEN_LOG_DIR@/guests/<guest-name>".
 
 =item B<backend=domain-id>
 
 Specify the backend domain name or id, defaults to dom0.
 
+=item B<max-files=NUMBER>
+
+Specify the maximum number of files below B<path>. A value of 0 (which
+is the default) doesn't limit the number of files. Only valid for
+B<type=xen_9pfsd>.
+
+=item B<max-open-files=NUMBER>
+
+Specify the maximum number of concurrently opened files below B<path>.
+Multiple opens of the same file are counted individually. Only valid for
+B<type=xen_9pfsd>, which has a default of B<max-open-files=5>.
+
+=item B<max-space=NUMBER>
+
+Specify the maximum used disk space in MiB below B<path>. A value of 0 (which
+is the default) doesn't limit the usable disk space. Only valid for
+B<type=xen_9pfsd>.
+
+=item B<auto-delete=BOOLEAN>
+
+When set the backend will delete the oldest file which is currently not
+opened by the guest in case the disk space limit set via B<max-space> or the
+file limit set via B<max-files> is being reached. Only valid for
+B<type=xen_9pfsd>.
+
 =back
 
 =item B<pvcalls=[ "backend=domain-id", ... ]>
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 900c0d46a0..ddeb4f20a7 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -20,6 +20,24 @@
 static int libxl__device_p9_setdefault(libxl__gc *gc, uint32_t domid,
                                        libxl_device_p9 *p9, bool hotplug)
 {
+    if (p9->type == LIBXL_P9_TYPE_UNKNOWN) {
+        p9->type = LIBXL_P9_TYPE_QEMU;
+    }
+    if (p9->type == LIBXL_P9_TYPE_QEMU &&
+        (p9->max_files || p9->max_open_files || p9->max_space ||
+         p9->auto_delete)) {
+        LOGD(ERROR, domid, "Illegal 9pfs parameter combination");
+        return ERROR_INVAL;
+    }
+    if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD && !p9->tag) {
+        p9->tag = libxl__strdup(NOGC, "Xen");
+    }
+
+    if (!p9->path || !p9->security_model || !p9->tag) {
+        LOGD(ERROR, domid, "9pfs spec missing required field!");
+        return ERROR_INVAL;
+    }
+
     return libxl__resolve_domid(gc, p9->backend_domname, &p9->backend_domid);
 }
 
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 9b358f11b8..80ffe85f5e 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2233,6 +2233,20 @@ void parse_config_data(const char *config_source,
                     replace_string(&p9->tag, value);
                 } else if (!strcmp(key, "backend")) {
                     replace_string(&p9->backend_domname, value);
+                } else if (!strcmp(key, "type")) {
+                    if (libxl_p9_type_from_string(value, &p9->type)) {
+                        fprintf(stderr, "failed to parse 9pfs type: %s\n",
+                                value);
+                        exit(1);
+                    }
+                } else if (!strcmp(key, "max-files")) {
+                    p9->max_files = parse_ulong(value);
+                } else if (!strcmp(key, "max-open-files")) {
+                    p9->max_open_files = parse_ulong(value);
+                } else if (!strcmp(key, "max-space")) {
+                    p9->max_space = parse_ulong(value);
+                } else if (!strcmp(key, "auto-delete")) {
+                    p9->auto_delete = strtoul(value, NULL, 0);
                 } else {
                     fprintf(stderr, "Unknown 9pfs parameter '%s'\n", key);
                     exit(1);
@@ -2243,9 +2257,12 @@ void parse_config_data(const char *config_source,
 
             libxl_string_list_dispose(&pairs);
 
-            if (!p9->path || !p9->security_model || !p9->tag) {
-                fprintf(stderr, "9pfs spec missing required field!\n");
-                exit(1);
+            if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD && !p9->path) {
+                char *path;
+
+                xasprintf(&path, XEN_LOG_DIR "/guests/%s", c_info->name);
+                replace_string(&p9->path, path);
+                free(path);
             }
         }
     }
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 16:31:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 16:31:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682279.1061380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb17p-0000EN-L3; Fri, 16 Feb 2024 16:31:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682279.1061380; Fri, 16 Feb 2024 16:31:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb17p-0000EG-IZ; Fri, 16 Feb 2024 16:31:45 +0000
Received: by outflank-mailman (input) for mailman id 682279;
 Fri, 16 Feb 2024 16:31:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LzRM=JZ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rb17o-0000EA-RB
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 16:31:44 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ddff5890-cce8-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 17:31:42 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 9FC611FB7A;
 Fri, 16 Feb 2024 16:31:41 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 5C20613343;
 Fri, 16 Feb 2024 16:31:41 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id nAP0FO2Nz2U4VAAAn2gu4w
 (envelope-from <jgross@suse.com>); Fri, 16 Feb 2024 16:31:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ddff5890-cce8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708101101; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=44RbCMtlPx/YsHv8XPhvBu4VsidyaCEOqPGz709Iy7s=;
	b=Ie9I96rP5aIR/69VSDeNJzLbBtwMSr/sjWSDBJNs4QixMmSoqNPXYvSJh/543NdMxQb6Gv
	mYsp+u8s1e4C8+0QPRls6E9zQcknA8hMgiWfrFcdf07K9Mm3tMCJfo/+vmEl+p9qZubiur
	aWJPdxIGs0kI8AQHRVTkCK1kxNtw76E=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708101101; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=44RbCMtlPx/YsHv8XPhvBu4VsidyaCEOqPGz709Iy7s=;
	b=Ie9I96rP5aIR/69VSDeNJzLbBtwMSr/sjWSDBJNs4QixMmSoqNPXYvSJh/543NdMxQb6Gv
	mYsp+u8s1e4C8+0QPRls6E9zQcknA8hMgiWfrFcdf07K9Mm3tMCJfo/+vmEl+p9qZubiur
	aWJPdxIGs0kI8AQHRVTkCK1kxNtw76E=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH v8 0/8] tools: enable xenstore-stubdom to use 9pfs
Date: Fri, 16 Feb 2024 17:31:31 +0100
Message-Id: <20240216163139.9093-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: 0.70
X-Spamd-Result: default: False [0.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,citrix.com,gmail.com,xen.org,ens-lyon.org];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

This series is adding 9pfs support to Xenstore-stubdom, enabling it
to do logging to a dom0 directory.

This is a prerequisite for the final goal to add live update support
to Xenstore-stubdom, as it enables the stubdom to store its state in
a dom0 file.

Reposting the rest series.

Changes in V8:
- patches 1-13 of V7 have been applied

Changes in V7:
- fixed V6 bugs

Changes in V6:
- patch 1 of V5 has been applied
- rebase
- addressed comments

Changes in V5:
- 10 patches have been applied already
- rename source directory to tools/9pfsd
- addressed comments

Changes in V4:
- patch 2 of V3 was applied
- added support of reading directories
- addressed review comments

Changes in V3:
- new patches 1, 23-25
- addressed review comments

Changes in V2:
- support of multiple rings per device
- xenlogd->xen-9pfsd rename
- addressed review comments
- fixed some bugs

Juergen Gross (8):
  tools/libs/light: add backend type for 9pfs PV devices
  tools/xl: support new 9pfs backend xen_9pfsd
  stubdom: extend xenstore stubdom configs
  tools: add 9pfs device to xenstore-stubdom
  tools/xenstored: mount 9pfs device in stubdom
  tools/xenstored: add helpers for filename handling
  tools/xenstored: support complete log capabilities in stubdom
  tools/xenstored: have a single do_control_memreport()

 docs/man/xl.cfg.5.pod.in                      |  36 +++-
 stubdom/xenstore-minios.cfg                   |   2 +-
 stubdom/xenstorepvh-minios.cfg                |   2 +-
 tools/golang/xenlight/helpers.gen.go          |  10 +
 tools/golang/xenlight/types.gen.go            |  12 ++
 tools/helpers/init-xenstore-domain.c          |   7 +
 .../Linux/init.d/sysconfig.xencommons.in      |   1 -
 tools/hotplug/Linux/launch-xenstore.in        |   1 +
 tools/include/libxl.h                         |  22 ++
 tools/libs/light/libxl_9pfs.c                 | 191 +++++++++++++++++-
 tools/libs/light/libxl_create.c               |   4 +-
 tools/libs/light/libxl_dm.c                   |   2 +-
 tools/libs/light/libxl_types.idl              |  11 +
 tools/libs/light/libxl_types_internal.idl     |   1 +
 tools/xenstored/control.c                     |  29 +--
 tools/xenstored/core.c                        |  15 +-
 tools/xenstored/core.h                        |  11 +-
 tools/xenstored/domain.c                      |   2 +
 tools/xenstored/lu_daemon.c                   |   4 +-
 tools/xenstored/minios.c                      |  62 ++++++
 tools/xenstored/posix.c                       |   8 +-
 tools/xl/xl_parse.c                           |  23 ++-
 22 files changed, 414 insertions(+), 42 deletions(-)

-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 16:31:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 16:31:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682280.1061390 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb17u-0000TK-SC; Fri, 16 Feb 2024 16:31:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682280.1061390; Fri, 16 Feb 2024 16:31:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb17u-0000TD-PU; Fri, 16 Feb 2024 16:31:50 +0000
Received: by outflank-mailman (input) for mailman id 682280;
 Fri, 16 Feb 2024 16:31:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LzRM=JZ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rb17t-0000Sx-52
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 16:31:49 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e11b871c-cce8-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 17:31:47 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 54F4F1FB76;
 Fri, 16 Feb 2024 16:31:47 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 1027A13343;
 Fri, 16 Feb 2024 16:31:47 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id WD1FAvONz2U+VAAAn2gu4w
 (envelope-from <jgross@suse.com>); Fri, 16 Feb 2024 16:31:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e11b871c-cce8-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708101107; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zgmWzK99RZBrySpVIOBOfTuvIg1qZCvB7PuUVcrLtSw=;
	b=DmTAkEPkDk3OrdLE6LkLutxX1HgH9G05u70Ov2Rn0lZ7qOPPM+c/hsKuDLjR6vY+tpHFe7
	1byuP5gr+vM339yEONIu63hJzaDpDt/C6omP2IvMfwwT7+GL0BhJrtdzbq3GX/wofVvMVy
	wr0Y2sZsMwDd1uucpxTEXZb/cMtajlY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708101107; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zgmWzK99RZBrySpVIOBOfTuvIg1qZCvB7PuUVcrLtSw=;
	b=DmTAkEPkDk3OrdLE6LkLutxX1HgH9G05u70Ov2Rn0lZ7qOPPM+c/hsKuDLjR6vY+tpHFe7
	1byuP5gr+vM339yEONIu63hJzaDpDt/C6omP2IvMfwwT7+GL0BhJrtdzbq3GX/wofVvMVy
	wr0Y2sZsMwDd1uucpxTEXZb/cMtajlY=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Nick Rosbrook <rosbrookn@gmail.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	George Dunlap <george.dunlap@cloud.com>
Subject: [PATCH v8 1/8] tools/libs/light: add backend type for 9pfs PV devices
Date: Fri, 16 Feb 2024 17:31:32 +0100
Message-Id: <20240216163139.9093-2-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240216163139.9093-1-jgross@suse.com>
References: <20240216163139.9093-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-2.10 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,citrix.com,gmail.com,xen.org,cloud.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: -2.10

Make the backend type of 9pfs PV devices configurable. The default is
"qemu" with the related Xenstore backend-side directory being "9pfs".

Add another type "xen_9pfsd" with the related Xenstore backend-side
directory "xen_9pfs".

As additional security features it is possible to specify:
- "max-space" for limiting the maximum space consumed on the filesystem
  in MBs
- "max-files" for limiting the maximum number of files in the
  filesystem
- "max-open-files" for limiting the maximum number of concurrent open
  files

For convenience "auto-delete" is available to let the backend delete the
oldest file of the guest in case otherwise "max-space" or "max-files"
would be violated.

The xen-9pfsd daemon will be started by libxenlight automatically when
the first "xen_9pfs" device is being created.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: George Dunlap <george.dunlap@cloud.com> # Golang bits
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
---
V3:
- regenerate go bindings
V4:
- rename libxl_device_p9_dm_needed() to libxl__device_p9_dm_needed()
  (Anthony Perard)
- reorder span related functions (Anthony Perard)
- add comment for xen9pfsd_spawn() return values (Anthony Perard)
- add LIBXL_HAVE_XEN_9PFS to libxl.h (Anthony Perard)
- use a copy of 'p9' in xen9pfsd_spawn() (Anthony Perard)
V6:
- rebase (Anthony Perard)
- drop callback from struct libxl__aop9_state (Anthony Perard)
---
 tools/golang/xenlight/helpers.gen.go      |  10 ++
 tools/golang/xenlight/types.gen.go        |  12 ++
 tools/include/libxl.h                     |   7 +
 tools/libs/light/libxl_9pfs.c             | 157 +++++++++++++++++++++-
 tools/libs/light/libxl_create.c           |   4 +-
 tools/libs/light/libxl_dm.c               |   2 +-
 tools/libs/light/libxl_types.idl          |  11 ++
 tools/libs/light/libxl_types_internal.idl |   1 +
 8 files changed, 197 insertions(+), 7 deletions(-)

diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go
index 0f8e23773c..8f44397a4e 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -2440,6 +2440,11 @@ x.Tag = C.GoString(xc.tag)
 x.Path = C.GoString(xc.path)
 x.SecurityModel = C.GoString(xc.security_model)
 x.Devid = Devid(xc.devid)
+x.Type = P9Type(xc._type)
+x.MaxSpace = int(xc.max_space)
+x.MaxFiles = int(xc.max_files)
+x.MaxOpenFiles = int(xc.max_open_files)
+x.AutoDelete = bool(xc.auto_delete)
 
  return nil}
 
@@ -2458,6 +2463,11 @@ xc.path = C.CString(x.Path)}
 if x.SecurityModel != "" {
 xc.security_model = C.CString(x.SecurityModel)}
 xc.devid = C.libxl_devid(x.Devid)
+xc._type = C.libxl_p9_type(x.Type)
+xc.max_space = C.int(x.MaxSpace)
+xc.max_files = C.int(x.MaxFiles)
+xc.max_open_files = C.int(x.MaxOpenFiles)
+xc.auto_delete = C.bool(x.AutoDelete)
 
  return nil
  }
diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go
index 9c8b7b81f6..d31722407a 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -122,6 +122,13 @@ NicTypeVifIoemu NicType = 1
 NicTypeVif NicType = 2
 )
 
+type P9Type int
+const(
+P9TypeUnknown P9Type = 0
+P9TypeQemu P9Type = 1
+P9TypeXen9Pfsd P9Type = 2
+)
+
 type ActionOnShutdown int
 const(
 ActionOnShutdownDestroy ActionOnShutdown = 1
@@ -889,6 +896,11 @@ Tag string
 Path string
 SecurityModel string
 Devid Devid
+Type P9Type
+MaxSpace int
+MaxFiles int
+MaxOpenFiles int
+AutoDelete bool
 }
 
 type DevicePvcallsif struct {
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 46bc774126..9a3e702557 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -615,6 +615,13 @@
  */
 #define LIBXL_HAVE_HVM_PIRQ 1
 
+/*
+ * LIBXL_HAVE_XEN_9PFS indicates the presence of the xen-9pfsd related
+ * fields in libxl_device_p9: type, max_space, max_files, max_open_files and
+ * auto_delete.
+ */
+#define LIBXL_HAVE_XEN_9PFS 1
+
 /*
  * libxl memory management
  *
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index 5ab0d3aa21..900c0d46a0 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -33,20 +33,171 @@ static int libxl__set_xenstore_p9(libxl__gc *gc, uint32_t domid,
 
     flexarray_append_pair(front, "tag", p9->tag);
 
+    if (p9->type == LIBXL_P9_TYPE_XEN_9PFSD) {
+        flexarray_append_pair(back, "max-space",
+                              GCSPRINTF("%u", p9->max_space));
+        flexarray_append_pair(back, "max-files",
+                              GCSPRINTF("%u", p9->max_files));
+        flexarray_append_pair(back, "max-open-files",
+                              GCSPRINTF("%u", p9->max_open_files));
+        flexarray_append_pair(back, "auto-delete",
+                              p9->auto_delete ? "1" : "0");
+    }
+
+    return 0;
+}
+
+static int libxl__device_from_p9(libxl__gc *gc, uint32_t domid,
+                                 libxl_device_p9 *type, libxl__device *device)
+{
+    device->backend_devid   = type->devid;
+    device->backend_domid   = type->backend_domid;
+    device->backend_kind    = type->type == LIBXL_P9_TYPE_QEMU
+                              ? LIBXL__DEVICE_KIND_9PFS
+                              : LIBXL__DEVICE_KIND_XEN_9PFS;
+    device->devid           = type->devid;
+    device->domid           = domid;
+    device->kind            = LIBXL__DEVICE_KIND_9PFS;
+
     return 0;
 }
 
-#define libxl__add_p9s NULL
+static int libxl__device_p9_dm_needed(void *e, unsigned domid)
+{
+    libxl_device_p9 *elem = e;
+
+    return elem->type == LIBXL_P9_TYPE_QEMU && elem->backend_domid == domid;
+}
+
+typedef struct libxl__aop9_state libxl__aop9_state;
+
+struct libxl__aop9_state {
+    libxl__spawn_state spawn;
+    libxl__ao_device *aodev;
+    libxl_device_p9 p9;
+    uint32_t domid;
+};
+
+static void xen9pfsd_confirm(libxl__egc *egc, libxl__spawn_state *spawn,
+                             const char *xsdata);
+static void xen9pfsd_failed(libxl__egc *egc, libxl__spawn_state *spawn, int rc);
+static void xen9pfsd_detached(libxl__egc *egc, libxl__spawn_state *spawn);
+static void xen9pfsd_spawn_outcome(libxl__egc *egc, libxl__aop9_state *aop9,
+                                   int rc);
+
+/*
+ * Spawn the xen-9pfsd daemon if needed.
+ * returns:
+ * < 0 if error
+ * 0 if no daemon needs to be spawned
+ * 1 if daemon was spawned
+ */
+static int xen9pfsd_spawn(libxl__egc *egc, uint32_t domid, libxl_device_p9 *p9,
+                         libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    struct libxl__aop9_state *aop9;
+    int rc;
+    char *args[] = { "xen-9pfsd", NULL };
+    char *path = GCSPRINTF("/local/domain/%u/libxl/xen-9pfs",
+                           p9->backend_domid);
+
+    if (p9->type != LIBXL_P9_TYPE_XEN_9PFSD ||
+        libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/state", path)))
+        return 0;
+
+    GCNEW(aop9);
+    aop9->aodev = aodev;
+    libxl_device_p9_copy(CTX, &aop9->p9, p9);
+    aop9->domid = domid;
+
+    aop9->spawn.ao = aodev->ao;
+    aop9->spawn.what = "xen-9pfs daemon";
+    aop9->spawn.xspath = GCSPRINTF("%s/state", path);
+    aop9->spawn.timeout_ms = LIBXL_DEVICE_MODEL_START_TIMEOUT * 1000;
+    aop9->spawn.pidpath = GCSPRINTF("%s/pid", path);
+    aop9->spawn.midproc_cb = libxl__spawn_record_pid;
+    aop9->spawn.confirm_cb = xen9pfsd_confirm;
+    aop9->spawn.failure_cb = xen9pfsd_failed;
+    aop9->spawn.detached_cb = xen9pfsd_detached;
+    rc = libxl__spawn_spawn(egc, &aop9->spawn);
+    if (rc < 0)
+        return rc;
+    if (!rc) {
+        setsid();
+        libxl__exec(gc, -1, -1, -1, LIBEXEC_BIN "/xen-9pfsd", args, NULL);
+    }
+
+    return 1;
+}
+
+static void xen9pfsd_confirm(libxl__egc *egc, libxl__spawn_state *spawn,
+                             const char *xsdata)
+{
+    STATE_AO_GC(spawn->ao);
+
+    if (!xsdata)
+        return;
+
+    if (strcmp(xsdata, "running"))
+        return;
+
+    libxl__spawn_initiate_detach(gc, spawn);
+}
+
+static void xen9pfsd_failed(libxl__egc *egc, libxl__spawn_state *spawn, int rc)
+{
+    libxl__aop9_state *aop9 = CONTAINER_OF(spawn, *aop9, spawn);
+
+    xen9pfsd_spawn_outcome(egc, aop9, rc);
+}
+
+static void xen9pfsd_detached(libxl__egc *egc, libxl__spawn_state *spawn)
+{
+    libxl__aop9_state *aop9 = CONTAINER_OF(spawn, *aop9, spawn);
+
+    xen9pfsd_spawn_outcome(egc, aop9, 0);
+}
+
+static void xen9pfsd_spawn_outcome(libxl__egc *egc, libxl__aop9_state *aop9,
+                                   int rc)
+{
+    aop9->aodev->rc = rc;
+    if (rc)
+        aop9->aodev->callback(egc, aop9->aodev);
+    else
+        libxl__device_add_async(egc, aop9->domid, &libxl__p9_devtype,
+                                &aop9->p9, aop9->aodev);
+}
+
+static void libxl__device_p9_add(libxl__egc *egc, uint32_t domid,
+                                 libxl_device_p9 *p9,
+                                 libxl__ao_device *aodev)
+{
+    int rc;
+
+    rc = xen9pfsd_spawn(egc, domid, p9, aodev);
+    if (rc == 1)
+        return;
+
+    if (rc == 0)
+        libxl__device_add_async(egc, domid, &libxl__p9_devtype, p9, aodev);
+
+    aodev->rc = rc;
+    if (rc)
+        aodev->callback(egc, aodev);
+}
+
 #define libxl_device_p9_list NULL
 #define libxl_device_p9_compare NULL
 
 static LIBXL_DEFINE_UPDATE_DEVID(p9)
-static LIBXL_DEFINE_DEVICE_FROM_TYPE(p9)
+static LIBXL_DEFINE_DEVICES_ADD(p9)
 
 LIBXL_DEFINE_DEVICE_REMOVE(p9)
 
 DEFINE_DEVICE_TYPE_STRUCT(p9, 9PFS, p9s,
-    .skip_attach = 1,
     .set_xenstore_config = (device_set_xenstore_config_fn_t)
                            libxl__set_xenstore_p9,
+    .dm_needed = libxl__device_p9_dm_needed,
 );
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 0008fac607..5546335973 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -1761,9 +1761,6 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
         libxl__device_console_dispose(&console);
     }
 
-    for (i = 0; i < d_config->num_p9s; i++)
-        libxl__device_add(gc, domid, &libxl__p9_devtype, &d_config->p9s[i]);
-
     for (i = 0; i < d_config->num_pvcallsifs; i++)
         libxl__device_add(gc, domid, &libxl__pvcallsif_devtype,
                           &d_config->pvcallsifs[i]);
@@ -1900,6 +1897,7 @@ const libxl__device_type *device_type_tbl[] = {
     &libxl__vdispl_devtype,
     &libxl__vsnd_devtype,
     &libxl__virtio_devtype,
+    &libxl__p9_devtype,
     NULL
 };
 
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 14b593110f..2aaaeb5aa7 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -3761,7 +3761,7 @@ int libxl__need_xenpv_qemu(libxl__gc *gc, libxl_domain_config *d_config)
         goto out;
     }
 
-    if (d_config->num_vfbs > 0 || d_config->num_p9s > 0) {
+    if (d_config->num_vfbs > 0) {
         ret = 1;
         goto out;
     }
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 899ad30969..682092a058 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -150,6 +150,12 @@ libxl_nic_type = Enumeration("nic_type", [
     (2, "VIF"),
     ])
 
+libxl_p9_type = Enumeration("p9_type", [
+    (0, "unknown"),
+    (1, "qemu"),
+    (2, "xen_9pfsd"),
+    ])
+
 libxl_action_on_shutdown = Enumeration("action_on_shutdown", [
     (1, "DESTROY"),
 
@@ -943,6 +949,11 @@ libxl_device_p9 = Struct("device_p9", [
     ("path",             string),
     ("security_model",   string),
     ("devid",            libxl_devid),
+    ("type",             libxl_p9_type),
+    ("max_space",        integer),
+    ("max_files",        integer),
+    ("max_open_files",   integer),
+    ("auto_delete",      bool),
 ])
 
 libxl_device_pvcallsif = Struct("device_pvcallsif", [
diff --git a/tools/libs/light/libxl_types_internal.idl b/tools/libs/light/libxl_types_internal.idl
index 56dccac153..0425e9b6b0 100644
--- a/tools/libs/light/libxl_types_internal.idl
+++ b/tools/libs/light/libxl_types_internal.idl
@@ -35,6 +35,7 @@ libxl__device_kind = Enumeration("device_kind", [
     (17, "VIRTIO_DISK"),
     (18, "VIRTIO"),
     (19, "VBD3"),
+    (20, "XEN_9PFS"),
     ])
 
 libxl__console_backend = Enumeration("console_backend", [
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 16:32:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 16:32:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682282.1061411 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb186-00013F-De; Fri, 16 Feb 2024 16:32:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682282.1061411; Fri, 16 Feb 2024 16:32:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb186-000138-AU; Fri, 16 Feb 2024 16:32:02 +0000
Received: by outflank-mailman (input) for mailman id 682282;
 Fri, 16 Feb 2024 16:32:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LzRM=JZ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rb184-0000EA-Pj
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 16:32:00 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e7a63c14-cce8-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 17:31:58 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 819B11FB81;
 Fri, 16 Feb 2024 16:31:58 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 580FC13343;
 Fri, 16 Feb 2024 16:31:58 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id /jLzE/6Nz2VHVAAAn2gu4w
 (envelope-from <jgross@suse.com>); Fri, 16 Feb 2024 16:31:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7a63c14-cce8-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v8 3/8] stubdom: extend xenstore stubdom configs
Date: Fri, 16 Feb 2024 17:31:34 +0100
Message-Id: <20240216163139.9093-4-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240216163139.9093-1-jgross@suse.com>
References: <20240216163139.9093-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 819B11FB81
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Extend the config files of the Xenstore stubdoms to include XENBUS
and 9PFRONT items in order to support file based logging.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 stubdom/xenstore-minios.cfg    | 2 +-
 stubdom/xenstorepvh-minios.cfg | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/stubdom/xenstore-minios.cfg b/stubdom/xenstore-minios.cfg
index a41704bb6b..239da519b9 100644
--- a/stubdom/xenstore-minios.cfg
+++ b/stubdom/xenstore-minios.cfg
@@ -3,7 +3,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
diff --git a/stubdom/xenstorepvh-minios.cfg b/stubdom/xenstorepvh-minios.cfg
index 6af51f5753..752b90d7d3 100644
--- a/stubdom/xenstorepvh-minios.cfg
+++ b/stubdom/xenstorepvh-minios.cfg
@@ -4,7 +4,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 16:32:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 16:32:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682283.1061421 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb18A-0001NA-KO; Fri, 16 Feb 2024 16:32:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682283.1061421; Fri, 16 Feb 2024 16:32:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb18A-0001Mu-H9; Fri, 16 Feb 2024 16:32:06 +0000
Received: by outflank-mailman (input) for mailman id 682283;
 Fri, 16 Feb 2024 16:32:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LzRM=JZ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rb18A-0000EA-2z
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 16:32:06 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id eb01f592-cce8-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 17:32:04 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 261C11FD3B;
 Fri, 16 Feb 2024 16:32:04 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id E1A5B13343;
 Fri, 16 Feb 2024 16:32:03 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id I5WaNQOOz2VdVAAAn2gu4w
 (envelope-from <jgross@suse.com>); Fri, 16 Feb 2024 16:32:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb01f592-cce8-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v8 4/8] tools: add 9pfs device to xenstore-stubdom
Date: Fri, 16 Feb 2024 17:31:35 +0100
Message-Id: <20240216163139.9093-5-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240216163139.9093-1-jgross@suse.com>
References: <20240216163139.9093-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 261C11FD3B
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

Add a 9pfs device to Xenstore stubdom in order to allow it to do e.g.
logging into a dom0 file.

Use the following parameters for the new device:

- tag = "Xen"
- type = "xen_9pfsd"
- path = "/var/lib/xen/xenstore"
- security-model = "none"

For now don't limit allowed file space or number of files.

Add a new libxl function for adding it similar to the function for
adding the console device.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
---
V2:
- add security_model parameter to new libxl function (Jason Andryuk)
V4:
- rename function to libxl_device_9pfs_add() (Anthony Perard)
- use a libxl_device_p9 pointer as parameter (Anthony Perard)
V8:
- remove "_XENSTORE" suffix from define (Anthony Perard)
---
 tools/helpers/init-xenstore-domain.c |  7 +++++++
 tools/include/libxl.h                | 15 +++++++++++++++
 tools/libs/light/libxl_9pfs.c        | 16 ++++++++++++++++
 3 files changed, 38 insertions(+)

diff --git a/tools/helpers/init-xenstore-domain.c b/tools/helpers/init-xenstore-domain.c
index 140ed610ae..1683438c5c 100644
--- a/tools/helpers/init-xenstore-domain.c
+++ b/tools/helpers/init-xenstore-domain.c
@@ -433,6 +433,12 @@ int main(int argc, char** argv)
     int rv, fd;
     char *maxmem_str = NULL;
     libxl_ctx *ctx;
+    libxl_device_p9 p9 = { .backend_domid = 0,
+                           .tag = "Xen",
+                           .path = XEN_LIB_DIR"/xenstore",
+                           .security_model = "none",
+                           .type = LIBXL_P9_TYPE_XEN_9PFSD,
+    };
 
     while ( (opt = getopt_long(argc, argv, "v", options, NULL)) != -1 )
     {
@@ -543,6 +549,7 @@ int main(int argc, char** argv)
     }
     libxl_console_add_xenstore(ctx, domid, 0, console_evtchn, console_gfn,
                                NULL);
+    libxl_device_9pfs_add(ctx, domid, &p9, NULL);
     libxl_ctx_free(ctx);
 
     fd = creat(XEN_RUN_DIR "/xenstored.pid", 0666);
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 9a3e702557..804496a9f8 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -583,6 +583,13 @@
  * libxl_console_add_xenstore() in libxl.
  */
 #define LIBXL_HAVE_CONSOLE_ADD_XENSTORE 1
+
+/*
+ * LIBXL_HAVE_P9_ADD indicates presence of the function
+ * libxl_device_9pfs_add() in libxl.
+ */
+#define LIBXL_HAVE_P9_ADD 1
+
 /*
  * libxl ABI compatibility
  *
@@ -2074,6 +2081,14 @@ int libxl_console_add_xenstore(libxl_ctx *ctx, uint32_t domid, uint32_t backend,
                                const libxl_asyncop_how *ao_how)
                                LIBXL_EXTERNAL_CALLERS_ONLY;
 
+/* libxl_device_9pfs_add writes the Xenstore entries for a domain's
+ * primary 9pfs device based on domid, and device parameters.
+ * If needed it will start the backend daemon.
+ */
+int libxl_device_9pfs_add(libxl_ctx *ctx, uint32_t domid, libxl_device_p9 *p9,
+                          const libxl_asyncop_how *ao_how)
+                          LIBXL_EXTERNAL_CALLERS_ONLY;
+
 /* May be called with info_r == NULL to check for domain's existence.
  * Returns ERROR_DOMAIN_NOTFOUND if domain does not exist (used to return
  * ERROR_INVAL for this scenario). */
diff --git a/tools/libs/light/libxl_9pfs.c b/tools/libs/light/libxl_9pfs.c
index ddeb4f20a7..48f894f070 100644
--- a/tools/libs/light/libxl_9pfs.c
+++ b/tools/libs/light/libxl_9pfs.c
@@ -206,6 +206,22 @@ static void libxl__device_p9_add(libxl__egc *egc, uint32_t domid,
         aodev->callback(egc, aodev);
 }
 
+int libxl_device_9pfs_add(libxl_ctx *ctx, uint32_t domid, libxl_device_p9 *p9,
+                          const libxl_asyncop_how *ao_how)
+{
+    AO_CREATE(ctx, domid, ao_how);
+    libxl__ao_device *aodev;
+
+    GCNEW(aodev);
+    libxl__prepare_ao_device(ao, aodev);
+    aodev->action = LIBXL__DEVICE_ACTION_ADD;
+    aodev->callback = device_addrm_aocomplete;
+
+    libxl__device_p9_add(egc, domid, p9, aodev);
+
+    return AO_INPROGRESS;
+}
+
 #define libxl_device_p9_list NULL
 #define libxl_device_p9_compare NULL
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 16:32:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 16:32:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682285.1061431 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb18F-0001q3-Sy; Fri, 16 Feb 2024 16:32:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682285.1061431; Fri, 16 Feb 2024 16:32:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb18F-0001pu-Q3; Fri, 16 Feb 2024 16:32:11 +0000
Received: by outflank-mailman (input) for mailman id 682285;
 Fri, 16 Feb 2024 16:32:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LzRM=JZ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rb18E-0000Sx-TE
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 16:32:10 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ee6e4fe1-cce8-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 17:32:10 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id C43201FB73;
 Fri, 16 Feb 2024 16:32:09 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 896FF13343;
 Fri, 16 Feb 2024 16:32:09 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 9YMgIAmOz2VyVAAAn2gu4w
 (envelope-from <jgross@suse.com>); Fri, 16 Feb 2024 16:32:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ee6e4fe1-cce8-11ee-8a4f-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708101129; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GuKS5JMiiFYV782dbcVdzQvPQKcEtbDic1ZOTvla/ok=;
	b=tRm4VtLfI/GQKkZOpQvVKyI2lfurK4zgGY9gZn6fJpp5hor/nDLqnHzasJimrn1ghGEqcI
	XLoyoiNuonSX5o2MCYwpOb2+byTVo7MfqA21x0INsZMy5YNDwNSy7mfklGDOHA7vGJeHGY
	BeCoZ3IpI5ZQzhHLRi2s76jzBPm7HJg=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708101129; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GuKS5JMiiFYV782dbcVdzQvPQKcEtbDic1ZOTvla/ok=;
	b=tRm4VtLfI/GQKkZOpQvVKyI2lfurK4zgGY9gZn6fJpp5hor/nDLqnHzasJimrn1ghGEqcI
	XLoyoiNuonSX5o2MCYwpOb2+byTVo7MfqA21x0INsZMy5YNDwNSy7mfklGDOHA7vGJeHGY
	BeCoZ3IpI5ZQzhHLRi2s76jzBPm7HJg=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v8 5/8] tools/xenstored: mount 9pfs device in stubdom
Date: Fri, 16 Feb 2024 17:31:36 +0100
Message-Id: <20240216163139.9093-6-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240216163139.9093-1-jgross@suse.com>
References: <20240216163139.9093-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.30
X-Spamd-Result: default: False [-3.30 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 REPLY(-4.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MID_CONTAINS_FROM(1.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Spam-Flag: NO

Mount the 9pfs device in stubdom enabling it to use files.

This has to happen in a worker thread in order to allow the main thread
handling the required Xenstore accesses in parallel.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- add logging in case of errors (Julien Grall)
---
 tools/xenstored/core.h   |  6 +++++
 tools/xenstored/domain.c |  2 ++
 tools/xenstored/minios.c | 54 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+)

diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index f6af086f01..fe0ee90581 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -36,6 +36,8 @@
 #include "list.h"
 #include "hashtable.h"
 
+#define XENSTORE_LIB_DIR	XEN_LIB_DIR "/xenstore"
+
 #ifndef O_CLOEXEC
 #define O_CLOEXEC 0
 /* O_CLOEXEC support is needed for Live Update in the daemon case. */
@@ -399,6 +401,10 @@ void handle_special_fds(void);
 int get_socket_fd(void);
 void set_socket_fd(int fd);
 
+#ifdef __MINIOS__
+void mount_9pfs(void);
+#endif
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index 1a7d5e9756..64c8fd0cc3 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -1236,6 +1236,8 @@ void stubdom_init(void)
 		barf_perror("Failed to initialize stubdom");
 
 	xenevtchn_notify(xce_handle, stubdom->port);
+
+	mount_9pfs();
 #endif
 }
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 22ac8defbd..562a9b4972 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -17,10 +17,20 @@
 */
 #include <sys/types.h>
 #include <sys/mman.h>
+#include <syslog.h>
+#include "talloc.h"
 #include "core.h"
 #include "utils.h"
 #include <xen/grant_table.h>
 #include <mini-os/lib.h>
+#include <mini-os/9pfront.h>
+#include <mini-os/sched.h>
+#include <mini-os/xenbus.h>
+#include <mini-os/xmalloc.h>
+
+#define P9_STATE_PATH	"device/9pfs/0/state"
+
+static void *p9_device;
 
 void finish_daemonize(void)
 {
@@ -74,3 +84,47 @@ int get_socket_fd(void)
 void set_socket_fd(int fd)
 {
 }
+
+static void mount_thread(void *p)
+{
+	xenbus_event_queue events = NULL;
+	char *err;
+	char *dummy;
+
+	err = xenbus_watch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs", &events);
+	if (err) {
+		log("error \"%s\" when setting watch on \"%s\"\n", err,
+		    P9_STATE_PATH);
+		free(err);
+		return;
+	}
+
+	for (;;) {
+		xenbus_wait_for_watch(&events);
+
+		/*
+		 * We only care for existence of the state node.
+		 * State changes are handled in init_9pfront().
+		 */
+		err = xenbus_read(XBT_NIL, P9_STATE_PATH, &dummy);
+		if (!err)
+			break;
+		free(err);
+	}
+
+	free(dummy);
+
+	err = xenbus_unwatch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs");
+	if (err) {
+		log("error \"%s\" when unwatching \"%s\", leaking watch\n",
+		    err, P9_STATE_PATH);
+		free(err);
+	}
+
+	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+}
+
+void mount_9pfs(void)
+{
+	create_thread("mount-9pfs", mount_thread, NULL);
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 16:32:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 16:32:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682290.1061441 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb18N-0002QX-42; Fri, 16 Feb 2024 16:32:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682290.1061441; Fri, 16 Feb 2024 16:32:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb18N-0002QP-0z; Fri, 16 Feb 2024 16:32:19 +0000
Received: by outflank-mailman (input) for mailman id 682290;
 Fri, 16 Feb 2024 16:32:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LzRM=JZ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rb18L-0000EA-Fz
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 16:32:17 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f1c86399-cce8-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 17:32:15 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 7ED0A22286;
 Fri, 16 Feb 2024 16:32:15 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 388DA13343;
 Fri, 16 Feb 2024 16:32:15 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id EQsZDA+Oz2V5VAAAn2gu4w
 (envelope-from <jgross@suse.com>); Fri, 16 Feb 2024 16:32:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1c86399-cce8-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v8 6/8] tools/xenstored: add helpers for filename handling
Date: Fri, 16 Feb 2024 17:31:37 +0100
Message-Id: <20240216163139.9093-7-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240216163139.9093-1-jgross@suse.com>
References: <20240216163139.9093-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: 7ED0A22286
X-Spam-Flag: NO

Add some helpers for handling filenames which might need different
implementations between stubdom and daemon environments:

- expansion of relative filenames (those are not really defined today,
  just expand them to be relative to /var/lib/xen/xenstore)
- expansion of xenstore_daemon_rundir() (used e.g. for saving the state
  file in case of live update - needs to be unchanged in the daemon
  case, but should result in /var/lib/xen/xenstore for stubdom)

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- make absolute_filename() return a pointer to const (Julien Grall)
---
 tools/xenstored/core.c      | 15 +++++++++++++--
 tools/xenstored/core.h      |  5 ++++-
 tools/xenstored/lu_daemon.c |  4 ++--
 tools/xenstored/minios.c    |  5 +++++
 tools/xenstored/posix.c     |  8 +++++++-
 5 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 48fc787ac1..bada1ad9a2 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -63,7 +63,7 @@ char **orig_argv;
 LIST_HEAD(connections);
 int tracefd = -1;
 bool keep_orphans = false;
-char *tracefile = NULL;
+const char *tracefile = NULL;
 static struct hashtable *nodes;
 unsigned int trace_flags = TRACE_OBJ | TRACE_IO;
 
@@ -137,6 +137,17 @@ void trace_destroy(const void *data, const char *type)
 		trace("obj: DESTROY %s %p\n", type, data);
 }
 
+/*
+ * Return an absolute filename.
+ * In case of a relative filename given as input, prepend XENSTORE_LIB_DIR.
+ */
+const char *absolute_filename(const void *ctx, const char *filename)
+{
+	if (filename[0] != '/')
+		return talloc_asprintf(ctx, XENSTORE_LIB_DIR "/%s", filename);
+	return talloc_strdup(ctx, filename);
+}
+
 void close_log(void)
 {
 	if (tracefd >= 0)
@@ -2759,7 +2770,7 @@ int main(int argc, char *argv[])
 #endif
 
 	if (tracefile)
-		tracefile = talloc_strdup(NULL, tracefile);
+		tracefile = absolute_filename(NULL, tracefile);
 
 #ifndef NO_LIVE_UPDATE
 	/* Read state in case of live update. */
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index fe0ee90581..e58779e88c 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -341,7 +341,7 @@ void close_log(void);
 extern int orig_argc;
 extern char **orig_argv;
 
-extern char *tracefile;
+extern const char *tracefile;
 extern int tracefd;
 
 /* Trace flag values must be kept in sync with trace_switches[] contents. */
@@ -405,6 +405,9 @@ void set_socket_fd(int fd);
 void mount_9pfs(void);
 #endif
 
+const char *xenstore_rundir(void);
+const char *absolute_filename(const void *ctx, const char *filename);
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/lu_daemon.c b/tools/xenstored/lu_daemon.c
index 71bcabadd3..6351111ab0 100644
--- a/tools/xenstored/lu_daemon.c
+++ b/tools/xenstored/lu_daemon.c
@@ -24,7 +24,7 @@ void lu_get_dump_state(struct lu_dump_state *state)
 	state->size = 0;
 
 	state->filename = talloc_asprintf(NULL, "%s/state_dump",
-					  xenstore_daemon_rundir());
+					  xenstore_rundir());
 	if (!state->filename)
 		barf("Allocation failure");
 
@@ -65,7 +65,7 @@ FILE *lu_dump_open(const void *ctx)
 	int fd;
 
 	filename = talloc_asprintf(ctx, "%s/state_dump",
-				   xenstore_daemon_rundir());
+				   xenstore_rundir());
 	if (!filename)
 		return NULL;
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 562a9b4972..e70386f8c7 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -128,3 +128,8 @@ void mount_9pfs(void)
 {
 	create_thread("mount-9pfs", mount_thread, NULL);
 }
+
+const char *xenstore_rundir(void)
+{
+	return XENSTORE_LIB_DIR;
+}
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 496329dfd1..d88c82d972 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -326,9 +326,10 @@ void early_init(bool live_update, bool dofork, const char *pidfile)
 {
 	reopen_log();
 
-	/* Make sure xenstored directory exists. */
+	/* Make sure xenstored directories exist. */
 	/* Errors ignored here, will be reported when we open files */
 	mkdir(xenstore_daemon_rundir(), 0755);
+	mkdir(XENSTORE_LIB_DIR, 0755);
 
 	if (dofork) {
 		openlog("xenstored", 0, LOG_DAEMON);
@@ -406,3 +407,8 @@ void set_socket_fd(int fd)
 {
 	sock = fd;
 }
+
+const char *xenstore_rundir(void)
+{
+	return xenstore_daemon_rundir();
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 16:32:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 16:32:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682293.1061451 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb18R-0002rs-E4; Fri, 16 Feb 2024 16:32:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682293.1061451; Fri, 16 Feb 2024 16:32:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb18R-0002rh-8k; Fri, 16 Feb 2024 16:32:23 +0000
Received: by outflank-mailman (input) for mailman id 682293;
 Fri, 16 Feb 2024 16:32:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LzRM=JZ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rb18Q-0000Sx-3t
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 16:32:22 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f524b614-cce8-11ee-8a4f-1f161083a0e0;
 Fri, 16 Feb 2024 17:32:21 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 22FF321241;
 Fri, 16 Feb 2024 16:32:21 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id DA67913343;
 Fri, 16 Feb 2024 16:32:20 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id +pDRMxSOz2V/VAAAn2gu4w
 (envelope-from <jgross@suse.com>); Fri, 16 Feb 2024 16:32:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f524b614-cce8-11ee-8a4f-1f161083a0e0
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Julien Grall <julien@xen.org>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v8 7/8] tools/xenstored: support complete log capabilities in stubdom
Date: Fri, 16 Feb 2024 17:31:38 +0100
Message-Id: <20240216163139.9093-8-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240216163139.9093-1-jgross@suse.com>
References: <20240216163139.9093-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 22FF321241
X-Spam-Level: 
X-Spam-Score: -4.00
X-Spam-Flag: NO

With 9pfs being fully available in Xenstore-stubdom now, there is no
reason to not fully support all logging capabilities in stubdom.

Open the logfile on stubdom only after the 9pfs file system has been
mounted.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V3:
- remove now stale comment in sysconfig.xencommons.in (Julien Grall)
---
 .../Linux/init.d/sysconfig.xencommons.in      |  1 -
 tools/hotplug/Linux/launch-xenstore.in        |  1 +
 tools/xenstored/control.c                     | 30 +++++++++----------
 tools/xenstored/minios.c                      |  3 ++
 4 files changed, 19 insertions(+), 16 deletions(-)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index 433e4849af..1bdd830d8a 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -58,7 +58,6 @@ XENSTORED_ARGS=
 ## Default: Not defined, tracing off
 #
 # Log xenstored messages
-# Only evaluated if XENSTORETYPE is "daemon".
 #XENSTORED_TRACE=[yes|on|1]
 
 ## Type: integer
diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in
index e854ca1eb8..da4eeca7c5 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -98,6 +98,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 	[ -z "$XENSTORE_DOMAIN_SIZE" ] && XENSTORE_DOMAIN_SIZE=8
 	XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory $XENSTORE_DOMAIN_SIZE"
 	[ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem $XENSTORE_MAX_DOMAIN_SIZE"
+	[ -z "$XENSTORED_TRACE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS -T xenstored-trace.log"
 
 	echo -n Starting $XENSTORE_DOMAIN_KERNEL...
 	${LIBEXEC_BIN}/init-xenstore-domain $XENSTORE_DOMAIN_ARGS || exit 1
diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index b2f64d674f..dae23a5ac0 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -201,19 +201,6 @@ static int do_control_quota_s(const void *ctx, struct connection *conn,
 	return EINVAL;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_logfile(const void *ctx, struct connection *conn,
 			      const char **vec, int num)
 {
@@ -222,13 +209,26 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 
 	close_log();
 	talloc_free(tracefile);
-	tracefile = talloc_strdup(NULL, vec[0]);
+	tracefile = absolute_filename(NULL, vec[0]);
 	reopen_log();
 
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
 
+#ifdef __MINIOS__
+static int do_control_memreport(const void *ctx, struct connection *conn,
+				const char **vec, int num)
+{
+	if (num)
+		return EINVAL;
+
+	talloc_report_full(NULL, stdout);
+
+	send_ack(conn, XS_CONTROL);
+	return 0;
+}
+#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
@@ -309,10 +309,10 @@ static struct cmd_s cmds[] = {
 		"[-c <cmdline>] [-F] [-t <timeout>] <file>\n"
 		"    Default timeout is 60 seconds.", 5 },
 #endif
+	{ "logfile", do_control_logfile, "<file>" },
 #ifdef __MINIOS__
 	{ "memreport", do_control_memreport, "" },
 #else
-	{ "logfile", do_control_logfile, "<file>" },
 	{ "memreport", do_control_memreport, "[<file>]" },
 #endif
 	{ "print", do_control_print, "<string>" },
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index e70386f8c7..a229954cf4 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -122,6 +122,9 @@ static void mount_thread(void *p)
 	}
 
 	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+
+	/* Start logging if selected. */
+	reopen_log();
 }
 
 void mount_9pfs(void)
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 16:32:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 16:32:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682296.1061461 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb18X-0003Mn-Q6; Fri, 16 Feb 2024 16:32:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682296.1061461; Fri, 16 Feb 2024 16:32:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb18X-0003Mg-MW; Fri, 16 Feb 2024 16:32:29 +0000
Received: by outflank-mailman (input) for mailman id 682296;
 Fri, 16 Feb 2024 16:32:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=LzRM=JZ=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rb18W-0000EA-JU
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 16:32:28 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f87ca945-cce8-11ee-98f5-efadbce2ee36;
 Fri, 16 Feb 2024 17:32:27 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id BA24821241;
 Fri, 16 Feb 2024 16:32:26 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 8334C13343;
 Fri, 16 Feb 2024 16:32:26 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 8+6YHhqOz2WBVAAAn2gu4w
 (envelope-from <jgross@suse.com>); Fri, 16 Feb 2024 16:32:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f87ca945-cce8-11ee-98f5-efadbce2ee36
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v8 8/8] tools/xenstored: have a single do_control_memreport()
Date: Fri, 16 Feb 2024 17:31:39 +0100
Message-Id: <20240216163139.9093-9-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240216163139.9093-1-jgross@suse.com>
References: <20240216163139.9093-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	none
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.00 / 50.00];
	 REPLY(-4.00)[]
X-Spam-Score: -4.00
X-Rspamd-Queue-Id: BA24821241
X-Spam-Flag: NO

With 9pfs now available in Xenstore-stubdom, there is no reason to
have distinct do_control_memreport() variants for the daemon and the
stubdom implementations.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 tools/xenstored/control.c | 27 +++++++--------------------
 1 file changed, 7 insertions(+), 20 deletions(-)

diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index dae23a5ac0..9561289179 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -216,23 +216,11 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 	return 0;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
 	FILE *fp;
+	const char *filename;
 	int fd;
 
 	if (num > 1)
@@ -255,8 +243,12 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 			if (!fp)
 				close(fd);
 		}
-	} else
-		fp = fopen(vec[0], "a");
+	} else {
+		filename = absolute_filename(ctx, vec[0]);
+		if (!filename)
+			return ENOMEM;
+		fp = fopen(filename, "a");
+	}
 
 	if (!fp)
 		return EBADF;
@@ -267,7 +259,6 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
-#endif
 
 static int do_control_print(const void *ctx, struct connection *conn,
 			    const char **vec, int num)
@@ -310,11 +301,7 @@ static struct cmd_s cmds[] = {
 		"    Default timeout is 60 seconds.", 5 },
 #endif
 	{ "logfile", do_control_logfile, "<file>" },
-#ifdef __MINIOS__
-	{ "memreport", do_control_memreport, "" },
-#else
 	{ "memreport", do_control_memreport, "[<file>]" },
-#endif
 	{ "print", do_control_print, "<string>" },
 	{ "quota", do_control_quota,
 		"[set <name> <val>|<domid>|max [-r]]" },
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Fri Feb 16 17:17:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 17:17:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682331.1061473 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb1py-0001fZ-Fl; Fri, 16 Feb 2024 17:17:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682331.1061473; Fri, 16 Feb 2024 17:17:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb1py-0001fS-DH; Fri, 16 Feb 2024 17:17:22 +0000
Received: by outflank-mailman (input) for mailman id 682331;
 Fri, 16 Feb 2024 17:17:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rb1pw-0001fK-TX
 for xen-devel@lists.xenproject.org; Fri, 16 Feb 2024 17:17:20 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rb1pl-00079I-Mr; Fri, 16 Feb 2024 17:17:09 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rb1pl-0000WF-8e; Fri, 16 Feb 2024 17:17:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=d2PXqay6fCZUfSopUt46LnDOsNwIDSjZ+NJObjpuqHg=; b=PUJwyqPlxl1cxw46Q6lYNEqkv5
	BaOYglLvn/4d3RssWdn0p9110zF35jW8IA1ndbhiBfHVaWOnxcE+gq5gkJTdDlFRjgwQyaeplH6uI
	Bv8mR6eA/236gW0t8vAWPP0s9CNt2d00RGNDU0/+Jj02MeaP6n4PyVn+AJ5iiW79VsHE=;
Message-ID: <f4dfdf1e-9a19-4616-bcbc-1bfa805a35c4@xen.org>
Date: Fri, 16 Feb 2024 17:17:06 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 00/27] Remove the directmap
Content-Language: en-GB
To: Elias El Yandouzi <eliasely@amazon.com>, Jan Beulich <jbeulich@suse.com>
Cc: pdurrant@amazon.com, dwmw@amazon.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Lukasz Hawrylko <lukasz@hawrylko.pl>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 =?UTF-8?Q?Mateusz_M=C3=B3wka?= <mateusz.mowka@intel.com>,
 xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <04d113bf-71ab-4eba-a95f-72318c888d1f@suse.com>
 <217a6edd-96f8-4589-b500-2dc31a1b0bee@amazon.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <217a6edd-96f8-4589-b500-2dc31a1b0bee@amazon.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 05/02/2024 11:11, Elias El Yandouzi wrote:
> Hi Jan,
> 
> On 29/01/2024 08:28, Jan Beulich wrote:
> 
>> On 16.01.2024 20:25, Elias El Yandouzi wrote:
>>> Julien Grall (8):
>>>    xen/vmap: Check the page has been mapped in vm_init_type()
>>>    xen/vmap: Introduce vmap_size() and use it
>>>    xen/arm: fixmap: Rename the fixmap slots to follow the x86 convention
>>
>> Btw, if there was clear indication that e.g. this patch (fully ack-ed
>> afaict) can go in ahead of earlier patches, I probably would have put
>> it in already. Considering it sits half way through the series, I don't
>> want to blindly chance it, though.
>>
> 
> I just forgot to strip off those ack tags. The patch got approved quite 
> a while ago and I thought it would be better to submit it again.

I spoke with Stefano on Matrix and he was fine with keeping the ack. I 
assume that Jan is also ok as he asked if we could commit.

So I have committed the patch.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 17:55:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 17:55:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682339.1061488 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb2QC-0006ta-8g; Fri, 16 Feb 2024 17:54:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682339.1061488; Fri, 16 Feb 2024 17:54:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb2QC-0006tS-6C; Fri, 16 Feb 2024 17:54:48 +0000
Received: by outflank-mailman (input) for mailman id 682339;
 Fri, 16 Feb 2024 17:54:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rb2QB-0006tJ-S2; Fri, 16 Feb 2024 17:54:47 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rb2QB-0007qm-JM; Fri, 16 Feb 2024 17:54:47 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rb2QB-0001VN-33; Fri, 16 Feb 2024 17:54:47 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rb2QB-00029D-2e; Fri, 16 Feb 2024 17:54:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zRqzGQJJBSMOzsz9ip5Ou/xTQm+moHtSzyxRPGo51o0=; b=AkUB9Tj+wab6CiiPbiXmio50uO
	tcaSZaMOi8j+8yzyxF96/odcgQXNzoP4+n7LQCpSoaO3pwtjLbh81wFn2vCLkGfuoTIkd7proHbVj
	ijIBQcKZ9rboFtpMUD8/XCQfYJM4ARDi7WqL8cw/B/DjlVHeJU1Lfbszk7y/xuqceJSo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184683-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184683: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4f5e5092fdbf5cec6bedc19fbe69cce4f5f08372
X-Osstest-Versions-That:
    linux=8d3dea210042f54b952b481838c1e7dfc4ec751d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 16 Feb 2024 17:54:47 +0000

flight 184683 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184683/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184671
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184671
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184671
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184671
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184671
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184671
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184671
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184671
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                4f5e5092fdbf5cec6bedc19fbe69cce4f5f08372
baseline version:
 linux                8d3dea210042f54b952b481838c1e7dfc4ec751d

Last test of basis   184671  2024-02-15 00:41:18 Z    1 days
Testing same since   184679  2024-02-15 20:11:16 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aaron Conole <aconole@redhat.com>
  Alexander Aring <aahringo@redhat.com>
  Allison Henderson <allison.henderson@oracle.com>
  André Draszik <andre.draszik@linaro.org>
  Arnd Bergmann <arnd@arndb.de>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  Ben Wolsieffer <ben.wolsieffer@hefring.com>
  Breno Leitao <leitao@debian.org>
  Carlos Song <carlos.song@nxp.com>
  Chandan Kumar Rout <chandanx.rout@intel.com> (A Contingent Worker at Intel)
  Christian A. Ehrhardt <lk@c--e.de>
  Christian Lamparter <chunkeey@gmail.com>
  Chuck Lever <chuck.lever@oracle.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Gabay <daniel.gabay@intel.com>
  Dave Ertman <david.m.ertman@intel.com>
  David S. Miller <davem@davemloft.net>
  Dmitry Antipov <dmantipov@yandex.ru>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Emmanuel Grumbach <emmanuel.grumbach@intel.com>
  Eric Dumazet <edumazet@google.com>
  Even Xu <even.xu@intel.com>
  Felix Fietkau <nbd@nbd.name>
  Florian Fainelli <f.fainelli@gmail.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Gavrilov Ilia <Ilia.Gavrilov@infotecs.ru>
  Geliang Tang <geliang.tang@linux.dev>
  Geliang Tang <geliang@kernel.org>
  Guenter Roeck <linux@roeck-us.net>
  Horatiu Vultur <horatiu.vultur@microchip.com>
  Ido Schimmel <idosch@nvidia.com>
  Ivan Vecera <ivecera@redhat.com>
  Jakub Kicinski <kuba@kernel.org>
  Jamal Hadi Salim <jhs@mojatatu.com>
  Jason Gerecke <jason.gerecke@wacom.com>
  Jason Gerecke <killertofu@gmail.com>
  Jiri Kosina <jkosina@suse.com>
  Jiri Pirko <jiri@nvidia.com>
  Johannes Berg <johannes.berg@intel.com>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Juergen Gross <jgross@suse.com>
  Kees Cook <keescook@chromium.org>
  Keqi Wang <wangkeqi_chris@163.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Kunwu Chan <chentao@kylinos.cn>
  Kyle Swenson <kyle.swenson@est.tech>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Maciej Fijalkowski <maciej.fijalkowski@intel.com>
  Magnus Karlsson <magnus.karlsson@intel.com>
  Manuel Fombuena <fombuena@outlook.com>
  Marc Kleine-Budde <mkl@pengutronix.de>
  Mark Brown <broonie@kernel.org>
  Markus Elfring <elfring@users.sourceforge.net>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Maxim Galaganov <max@internet.ru>
  Maxime Jayat <maxime.jayat@mobile-devices.fr>
  Maximilian Heyne <mheyne@amazon.de>
  Michal Simek <michal.simek@amd.com>
  Mika Westerberg <mika.westerberg@linux.intel.com>
  Miri Korenblit <miriam.rachel.korenblit@intel.com>
  Mubin Sayyed <mubin.sayyed@amd.com>
  Naama Meir <naamax.meir@linux.intel.com>
  Niklas Cassel <cassel@kernel.org>
  Nikolay Aleksandrov <razor@blackwall.org>
  Nuno Sa <nuno.sa@analog.com>
  Nícolas F. R. A. Prado <nfraprado@collabora.com>
  Oleksandr Natalenko <oleksandr@natalenko.name>
  Oleksij Rempel <o.rempel@pengutronix.de>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Paolo Abeni <pabeni@redhat.com>
  Parav Pandit <parav@nvidia.com>
  Paul Barker <paul.barker.ct@bp.renesas.com>
  Paul Durrant <paul@xen.org>
  Piyush Mehta <piyush.mehta@amd.com>
  Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
  Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
  Rafal Romanowski <rafal.romanowski@intel.com>
  Ralf Schlatterbeck <rsc@runtux.com>
  Randy Dunlap <rdunlap@infradead.org>
  Ricardo B. Marliere <ricardo@marliere.net>
  Rob Herring <robh@kernel.org>
  Sabrina Dubroca <sd@queasysnail.net>
  Saravana Kannan <saravanak@google.com>
  Sasha Neftin <sasha.neftin@intel.com>
  SeongJae Park <sj@kernel.org>
  Seth Forshee <sforshee@kernel.org>
  Shannon Nelson <shannon.nelson@amd.com>
  Simon Horman <horms@kernel.org>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Subbaraya Sundeep <sbhatta@marvell.com>
  Sujai Buvaneswaran <sujai.buvaneswaran@intel.com>
  Tatsunosuke Tobita <tatsunosuke.tobita@wacom.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Vadim Fedorenko <vadfed@meta.com>
  Vaishnav Achath <vaishnav.a@ti.com>
  Victor Nogueira <victor@mojatatu.com>
  Xin Long <lucien.xin@gmail.com>
  Ziqi Zhao <astrajoan@yahoo.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   8d3dea210042..4f5e5092fdbf  4f5e5092fdbf5cec6bedc19fbe69cce4f5f08372 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Fri Feb 16 20:30:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 16 Feb 2024 20:30:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682347.1061500 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb4qH-0006Ln-0e; Fri, 16 Feb 2024 20:29:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682347.1061500; Fri, 16 Feb 2024 20:29:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb4qG-0006Lg-UB; Fri, 16 Feb 2024 20:29:52 +0000
Received: by outflank-mailman (input) for mailman id 682347;
 Fri, 16 Feb 2024 20:29:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rb4qF-0006LW-KY; Fri, 16 Feb 2024 20:29:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rb4qF-0002Yg-9Q; Fri, 16 Feb 2024 20:29:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rb4qE-0007Xq-VR; Fri, 16 Feb 2024 20:29:51 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rb4qE-0004Lt-Uh; Fri, 16 Feb 2024 20:29:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=oPxart4rlBINOteIeN+UJDz6gbHcHBnSqZZBiAryCTg=; b=BIH8zU/owddHvEi+vaw0aE5sfI
	0ODF406vPhCIy8P9Y7x63h/gtOvN4voT9FOxtNVF6T4+wse2ee++zTGZCjw9pZFaeMqwJ+WHHrxQ0
	WlVgE/HvZqbERtvdUisil2Mws0xE9NNFnubpW6SbNjWo81RiuiwbeTx8esO+KnJndz6c=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184685-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184685: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
X-Osstest-Versions-That:
    xen=78398afae10bfb4ab94e8af17b7ed58510a57d96
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 16 Feb 2024 20:29:50 +0000

flight 184685 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184685/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3
baseline version:
 xen                  78398afae10bfb4ab94e8af17b7ed58510a57d96

Last test of basis   184678  2024-02-15 18:02:04 Z    1 days
Testing same since   184685  2024-02-16 18:03:46 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Elias El Yandouzi <eliasely@amazon.com>
  Julien Grall <jgrall@amazon.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   78398afae1..0441c3acc7  0441c3acc7e9e72e984ce49d32e61827894ae4a3 -> smoke


From xen-devel-bounces@lists.xenproject.org Sat Feb 17 00:29:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 17 Feb 2024 00:29:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682359.1061513 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb8aF-0006Ld-CI; Sat, 17 Feb 2024 00:29:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682359.1061513; Sat, 17 Feb 2024 00:29:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rb8aF-0006LW-9j; Sat, 17 Feb 2024 00:29:35 +0000
Received: by outflank-mailman (input) for mailman id 682359;
 Sat, 17 Feb 2024 00:29:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rb8aD-0006LI-63; Sat, 17 Feb 2024 00:29:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rb8aC-0008A3-QS; Sat, 17 Feb 2024 00:29:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rb8aC-0002LH-EW; Sat, 17 Feb 2024 00:29:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rb8aC-0001pb-E5; Sat, 17 Feb 2024 00:29:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=gOLiwIMyXqtyGSF46zRKNjxXS4KD3dzdqhvKf10Wc0s=; b=CfY1DZ65SrxgO0nt57diEgsvA4
	KDVN1mmnU6mHeGvDh3AjeFWn7TXahai4IG5i2t7Gl+iKdK14+xV3uZFdbLqEWzM/UakvS0wSUmO1L
	cbhK68SwiEufJvqXmxTi7tecAo+yUY9SLM2YDKbWHKCwPe0RJDY6tYzEjiTDdNrNK+cE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184684-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184684: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt-raw:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=78398afae10bfb4ab94e8af17b7ed58510a57d96
X-Osstest-Versions-That:
    xen=78398afae10bfb4ab94e8af17b7ed58510a57d96
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 17 Feb 2024 00:29:32 +0000

flight 184684 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184684/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-raw 17 guest-start/debian.repeat fail in 184681 pass in 184684
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184681

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184681
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184681
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184681
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184681
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184681
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184681
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184681
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184681
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184681
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184681
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184681
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184681
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  78398afae10bfb4ab94e8af17b7ed58510a57d96
baseline version:
 xen                  78398afae10bfb4ab94e8af17b7ed58510a57d96

Last test of basis   184684  2024-02-16 10:54:02 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sat Feb 17 03:23:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 17 Feb 2024 03:23:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682371.1061524 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbBI1-0006iy-M2; Sat, 17 Feb 2024 03:22:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682371.1061524; Sat, 17 Feb 2024 03:22:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbBI1-0006ir-I9; Sat, 17 Feb 2024 03:22:57 +0000
Received: by outflank-mailman (input) for mailman id 682371;
 Sat, 17 Feb 2024 03:22:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=1kna=J2=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1rbBI0-0006ij-3e
 for xen-devel@lists.xenproject.org; Sat, 17 Feb 2024 03:22:56 +0000
Received: from wfout6-smtp.messagingengine.com
 (wfout6-smtp.messagingengine.com [64.147.123.149])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d4d93698-cd43-11ee-98f5-efadbce2ee36;
 Sat, 17 Feb 2024 04:22:52 +0100 (CET)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailfout.west.internal (Postfix) with ESMTP id F22F11C0008B;
 Fri, 16 Feb 2024 22:22:49 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute6.internal (MEProxy); Fri, 16 Feb 2024 22:22:50 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 16 Feb 2024 22:22:48 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d4d93698-cd43-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1708140169;
	 x=1708226569; bh=Yf01G1o+UqIKYBUWB3l3I2awerZSH20vNqLMgLnpFvQ=; b=
	SJex7i+3Guapf/FPfjKqWk1wTYsbEushu7ikiT0korY2KlM7lsdrp+CfJyDAQSzV
	NVNO3DCJqhSTbGyf/zOEuG2w+3GSFBin9X2ECG+l6vu9DcKzj6CkbvyptRfZTjd/
	0ithZ2XMUKWPmjS+tCGflCTjsbhkY4uzfQLWNCz4dMFd3U9r1wKRWpqE947F6DdZ
	hrDX1gZUd1YTB5bYkHrP+H25XwZIzkmD3yGyDUgwMslk+LMj9foaxwryfUo2hwvI
	9CHIOXbpgE8B6FJpSmJmEGvKpev3hz5OegidlUGUT0bXfSop6L+MqwSlsQk25zUK
	Qay7AZ78REEGjHycXyy6pw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1708140169; x=1708226569; bh=Yf01G1o+UqIKYBUWB3l3I2awerZS
	H20vNqLMgLnpFvQ=; b=LGriUFRndYn0Gg2wp4qY+gFliYUL2uQMMZgSsk8x05jZ
	g5ktO6+oCLVdCwVGKQpU0aCRT6fCKl8zGu428AN15dgCuNKgoFc49b+sznJFitsM
	9wqt6juyVlff73uZJz8jFwDevfZ6mvya891R/qLoXOMwBpPKcQcvs6/rhbzxGpoq
	AdSCaOSAPV6XQzzLI5f8R5dYy0+1BbcbmvW6DhUlksL3qa9mzrGDs9f/yk70KQAG
	uoBxoYpxGKHbUN7Ya4p0z44FfBGYWLo5evkJYd97IUuj5bgKsrT8oLMLMDitU6tf
	FYzZAebXInWdng+FG8KuWyGLE7QZxWEKBobaqcXCvg==
X-ME-Sender: <xms:iSbQZQcqpsay7yvJxp9_Pm1TEPol9MRujNIGxRjFaowNq8yZ3buGmA>
    <xme:iSbQZSNL7PbLgqa-NLJXxXq5h5P5qFq6_GBSZ__TQxcySQSWlNwV-ZczEtkS3dUXv
    dFITWVf79UvIg>
X-ME-Received: <xmr:iSbQZRjIwQPj1sTMJCdfvGrUpT3gFmoCC-XbuV0r4KwiOgGXX2ptm3laVVvWa1YAlXvj12Gae_Q2sqdie36t0MYFWWHlCa-lQQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdefgdehjecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpeffhffvvefukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeeiledu
    vdekjeeukefhffetgeefiefffeejjedvieejvdfgjeehteduvdekffduieenucffohhmrg
    hinhepkhgvrhhnvghlrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghm
    pehmrghilhhfrhhomhepmhgrrhhmrghrvghksehinhhvihhsihgslhgvthhhihhnghhslh
    grsgdrtghomh
X-ME-Proxy: <xmx:iSbQZV8uSz6GQnw7nVs9Y0WQ_psCsKufUIz9NONY6gZkIvCn1FtBgA>
    <xmx:iSbQZcvcg29hJ2z1lYcNMaqgamj1buP9i7CN71fj0w2ilIlu3NW4VA>
    <xmx:iSbQZcEAjglVTN0iS_frc707OY3RaEnEM8PQ5hLpmulln4kju3Hyjw>
    <xmx:iSbQZWWBcOgr93FKOl_1VGuzQpo5Nv3UqSNdqWdumhBQPVLB-FHoyor9y7I>
Feedback-ID: i1568416f:Fastmail
Date: Sat, 17 Feb 2024 04:22:45 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: qemu-devel@nongnu.org, Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>
Cc: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] i386: load kernel on xen using DMA
Message-ID: <ZdAmhqse3AyEbE82@mail-itl>
References: <20210426034709.595432-1-marmarek@invisiblethingslab.com>
 <87sg1feemf.fsf@linaro.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="Q0+3eHKn/djYPxYa"
Content-Disposition: inline
In-Reply-To: <87sg1feemf.fsf@linaro.org>


--Q0+3eHKn/djYPxYa
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Sat, 17 Feb 2024 04:22:45 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: qemu-devel@nongnu.org, Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>
Cc: Alex =?utf-8?Q?Benn=C3=A9e?= <alex.bennee@linaro.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] i386: load kernel on xen using DMA

On Fri, Jun 18, 2021 at 09:54:14AM +0100, Alex Benn=C3=A9e wrote:
>=20
> Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingslab.com> writes:
>=20
> > Kernel on Xen is loaded via fw_cfg. Previously it used non-DMA version,
> > which loaded the kernel (and initramfs) byte by byte. Change this
> > to DMA, to load in bigger chunks.
> > This change alone reduces load time of a (big) kernel+initramfs from
> > ~10s down to below 1s.
> >
> > This change was suggested initially here:
> > https://lore.kernel.org/xen-devel/20180216204031.000052e9@gmail.com/
> > Apparently this alone is already enough to get massive speedup.
> >
> > Signed-off-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblething=
slab.com>
> > ---
> >  hw/i386/pc.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index 8a84b25a03..14e43d4da4 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -839,7 +839,8 @@ void xen_load_linux(PCMachineState *pcms)
> > =20
> >      assert(MACHINE(pcms)->kernel_filename !=3D NULL);
> > =20
> > -    fw_cfg =3D fw_cfg_init_io(FW_CFG_IO_BASE);
> > +    fw_cfg =3D fw_cfg_init_io_dma(FW_CFG_IO_BASE, FW_CFG_IO_BASE + 4,
> > +                                &address_space_memory);
> >      fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, x86ms->boot_cpus);
> >      rom_set_fw(fw_cfg);
>=20
> Gentle ping. The fix looks perfectly sane to me but I don't have any x86
> Xen HW to test this one. Are the x86 maintainers happy to take this on?

Ping...

>=20
> FWIW:
>=20
> Reviewed-by: Alex Benn=C3=A9e <alex.bennee@linaro.org>
>=20
> --=20
> Alex Benn=C3=A9e
>=20

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--Q0+3eHKn/djYPxYa
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXQJoYACgkQ24/THMrX
1yymZAf8ChLekrtLpigwAq1cK/g9+kUbjLWHksGz666MCdEgd5P+Zy0KSbfa4A/6
PR2754T9zgqf5JCGFkaE9uCKYG3AtQpqXvQO7tLI4GicrIBO1YHYp094OV2F97oT
KTxdR97aYPbKJ78SfRf4phU6lFq1BFFEMs9SsPEYF/g+i1AsP8m+F40AHw7CiK/G
vCznXYoHwIhRFHPbf/lEkBVLHE7L7J12hMXM1Q0j7n3MMGHWMhmN9WL1T4EfSjHf
oGs+2CrWIJ70LW8BbdklCZpEmH692PmuY/br+CiyauL+l4EioPtvZrm3NAlLdKet
509POMwoSg+3YNbl8aRAD1wkx7+baQ==
=IVfk
-----END PGP SIGNATURE-----

--Q0+3eHKn/djYPxYa--


From xen-devel-bounces@lists.xenproject.org Sat Feb 17 06:52:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 17 Feb 2024 06:52:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682377.1061534 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbEYW-0004RG-Tf; Sat, 17 Feb 2024 06:52:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682377.1061534; Sat, 17 Feb 2024 06:52:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbEYW-0004R9-Qj; Sat, 17 Feb 2024 06:52:12 +0000
Received: by outflank-mailman (input) for mailman id 682377;
 Sat, 17 Feb 2024 06:52:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbEYV-0004Qz-1t; Sat, 17 Feb 2024 06:52:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbEYU-0006Mn-P8; Sat, 17 Feb 2024 06:52:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbEYU-0002I3-71; Sat, 17 Feb 2024 06:52:10 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rbEYU-0005af-6J; Sat, 17 Feb 2024 06:52:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3j6mWn0lcKYKc5agMjeUwvj/i0s3c5IXaMtxUZhQ9J4=; b=mOogMD61M9rxpAmOFUn0zZzAmm
	jOIdsvuwYQVwMP+8QBsrNXSUqHcQoRy6glDKqZMmdwWpGoy4KRVlWQMzVvD+MprLCipqvD5LNOuon
	zrNbMP/hjOM57fhR1iIjx9pcF7JNyXUN+5fGehkefRzybObHvfcK7zqpHlxfbZDABn+k=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184686-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184686: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-examine:reboot:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=0f1dd5e91e2ba3990143645faff2bcce2d99778e
X-Osstest-Versions-That:
    linux=4f5e5092fdbf5cec6bedc19fbe69cce4f5f08372
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 17 Feb 2024 06:52:10 +0000

flight 184686 linux-linus real [real]
flight 184690 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184686/
http://logs.test-lab.xenproject.org/osstest/logs/184690/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-examine      8 reboot              fail pass in 184690-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184683
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184683
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184683
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184683
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184683
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184683
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184683
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184683
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                0f1dd5e91e2ba3990143645faff2bcce2d99778e
baseline version:
 linux                4f5e5092fdbf5cec6bedc19fbe69cce4f5f08372

Last test of basis   184683  2024-02-16 05:59:12 Z    1 days
Testing same since   184686  2024-02-16 18:12:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Deucher <alexander.deucher@amd.com>
  Alexandru Gagniuc <alexandru.gagniuc@hp.com>
  Alexey Khoroshilov <khoroshilov@ispras.ru>
  Amadeusz Sławiński <amadeuszx.slawinski@linux.intel.com>
  Arnd Bergmann <arnd@arndb.de>
  Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
  Attila Tőkés <attitokes@gmail.com>
  Aurabindo Pillai <aurabindo.pillai@amd.com>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  bo liu <bo.liu@senarytech.com>
  Casey Schaufler <casey@schaufler-ca.com>
  Cezary Rojewski <cezary.rojewski@intel.com>
  Charles Keepax <ckeepax@opensource.cirrus.com>
  Christian König <christian.koenig@amd.com>
  Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
  Curtis Malainey <cujomalainey@chromium.org>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Danilo Krummrich <dakr@redhat.com>
  Dave Airlie <airlied@redhat.com>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org> # sc7180, sdm845
  Eniac Zhang <eniac-xw.zhang@hp.com>
  Felix Kuehling <Felix.Kuehling@amd.com>
  Gergo Koteles <soyer@irl.hu>
  Hamza Mahfooz <hamza.mahfooz@amd.com>
  Hans de Goede <hdegoede@redhat.com>
  Harshit Mogalapalli <harshit.m.mogalapalli@oracle.com>
  Heiko Stuebner <heiko@sntech.de>
  Jacek Lawrynowicz <jacek.lawrynowicz@linux.intel.com>
  Jani Nikula <jani.nikula@intel.com>
  Jann Horn <jannh@google.com>
  Jean-Loïc Charroud <lagiraudiere+linux@free.fr>
  Jeffrey Hugo <quic_jhugo@quicinc.com>
  Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
  Kent Russell <kent.russell@amd.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lukas Bulwahn <lukas.bulwahn@gmail.com>
  Manasi Navare <navaremanasi@chromium.org>
  Mario Limonciello <mario.limonciello@amd.com>
  Mark Brown <broonie@kernel.org>
  Matthew Auld <matthew.auld@intel.com>
  Maxime Ripard <mripard@redhat.com>
  Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
  Paul Moore <paul@paul-moore.com>
  Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
  Philip Yang <Philip.Yang@amd.com>
  Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
  Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
  Richard Fitzgerald <rf@opensource.cirrus.com>
  Rob Clark <robdclark@chromium.org>
  Rob Clark <robdclark@gmail.com>
  Roman Li <roman.li@amd.com>
  Shuming Fan <shumingf@realtek.com>
  Sohaib Nadeem <sohaib.nadeem@amd.com>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Takashi Iwai <tiwai@suse.de>
  Thomas Hellstrom <thomas.hellstrom@linux.intel.com>
  Thomas Hellström <thomas.hellstrom@linux.intel.com>
  Thong <thong.thai@amd.com>
  Timur Tabi <ttabi@nvidia.com>
  Tom Chung <chiahsuan.chung@amd.com>
  Tomasz Kudela <ramzes005@gmail.com>
  Ville Syrjälä <ville.syrjala@linux.intel.com>
  Zhikai Zhai <zhikai.zhai@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     fail    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   4f5e5092fdbf..0f1dd5e91e2b  0f1dd5e91e2ba3990143645faff2bcce2d99778e -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sat Feb 17 09:11:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 17 Feb 2024 09:11:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682408.1061544 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbGjT-0002gH-BO; Sat, 17 Feb 2024 09:11:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682408.1061544; Sat, 17 Feb 2024 09:11:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbGjT-0002gA-7F; Sat, 17 Feb 2024 09:11:39 +0000
Received: by outflank-mailman (input) for mailman id 682408;
 Sat, 17 Feb 2024 09:11:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ITqj=J2=tls.msk.ru=mjt@srs-se1.protection.inumbo.net>)
 id 1rbGjS-0002g4-00
 for xen-devel@lists.xenproject.org; Sat, 17 Feb 2024 09:11:38 +0000
Received: from isrv.corpit.ru (isrv.corpit.ru [86.62.121.231])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8c969d43-cd74-11ee-8a52-1f161083a0e0;
 Sat, 17 Feb 2024 10:11:37 +0100 (CET)
Received: from tsrv.corpit.ru (tsrv.tls.msk.ru [192.168.177.2])
 by isrv.corpit.ru (Postfix) with ESMTP id 79B124E172;
 Sat, 17 Feb 2024 12:11:47 +0300 (MSK)
Received: from [192.168.177.130] (mjt.wg.tls.msk.ru [192.168.177.130])
 by tsrv.corpit.ru (Postfix) with ESMTP id 5401281CA3;
 Sat, 17 Feb 2024 12:11:34 +0300 (MSK)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8c969d43-cd74-11ee-8a52-1f161083a0e0
Message-ID: <e7943a9b-7b2c-4c5c-8f41-f12cc3e75874@tls.msk.ru>
Date: Sat, 17 Feb 2024 12:11:34 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/pt: fix igd passthrough for pc machine with xen
 accelerator
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Chuck Zmudzinski <brchuckz@aol.com>, qemu-devel@nongnu.org,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 Paolo Bonzini <pbonzini@redhat.com>,
 Richard Henderson <richard.henderson@linaro.org>,
 Eduardo Habkost <eduardo@habkost.net>, "Michael S. Tsirkin"
 <mst@redhat.com>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>,
 Igor Mammedov <imammedo@redhat.com>, xen-devel@lists.xenproject.org,
 qemu-stable@nongnu.org
References: <a304213d26506b066021f803c39b87f6a262ed86.1675820085.git.brchuckz.ref@aol.com>
 <a304213d26506b066021f803c39b87f6a262ed86.1675820085.git.brchuckz@aol.com>
 <986d9eca-5fab-cacb-05c7-b85e4d58665b@msgid.tls.msk.ru>
 <47ed3568-2127-a865-4e4f-ff5902484231@aol.com>
 <2b07603f-6623-9fbf-15df-a86849d9aca3@msgid.tls.msk.ru>
 <alpine.DEB.2.22.394.2305171246020.128889@ubuntu-linux-20-04-desktop>
From: Michael Tokarev <mjt@tls.msk.ru>
Autocrypt: addr=mjt@tls.msk.ru; keydata=
 xsBLBETIiwkBCADh3cFB56BQYPjtMZCfK6PSLR8lw8EB20rsrPeJtd91IoNZlnCjSoxd9Th1
 bLUR8YlpRJ2rjc6O1Bc04VghqUOHgS/tYt8vLjcGWixzdhSLJgPDK3QQZPAvBjMbCt1B6euC
 WuD87Pv5Udlpnzf4aMwxkgfTusx+ynae/o+T5r7tXD+isccbC3SiGhmAPxFyY3zGcFk4+Rxc
 0tP8YY2FWE/baHu+lBDTUN79efWAkHhex1XzVZsV7ZD16rzDbXFK5m6ApvGJWlr5YDEEydTF
 WwmvwBfr4OINVxzEG/ujNiG4fpMf2NsnFGyB9aSbFjXZevB4qWkduYYW+xpK1EryszHtAAYp
 zSBNaWNoYWVsIFRva2FyZXYgPG1qdEB0bHMubXNrLnJ1PsLAlgQTAQoAQAIbAwYLCQgHAwIE
 FQIIAwQWAgMBAh4BAheAAhkBFiEEbuGV0Yhuj/uBDUMkRXzgoIBEZcUFAmBbcjwFCS5e6jMA
 CgkQRXzgoIBEZcUTIQgA1hPsOF82pXxbcJXBMc4zB9OQu4AlnZvERoGyw7I2222QzaN3RFuj
 Fia//mapXzpIQNF08l/AA6cx+CKPeGnXwyZfF9fLa4RfifmdNKME8C00XlqnoJDZBGzq8yMy
 LAKDxl9OQWFcDwDxV+irg5U3fbtNVhvV0kLbS2TyQ0aU5w60ERS2NcyDWplOo7AOzZWChcA4
 UFf78oVdZdCW8YDtU0uQFhA9moNnrePy1HSFqduxnlFHEI+fDj/TiOm2ci48b8SBBJOIJFjl
 SBgH8+SfT9ZqkzhN9vh3YJ49831NwASVm0x1rDHcIwWD32VFZViZ3NjehogRNH9br0PSUYOC
 3s7ATQRX2BjLAQgAnak3m0imYOkv2tO/olULFa686tlwuvl5kL0NWCdGQeXv2uMxy36szcrh
 K1uYhpiQv4r2qNd8BJtYlnYIK16N8GBdkplaDIHcBMbU4t+6bQzEIJIaWoq1hzakmHHngE2a
 pNMnUf/01GFvCRPlv3imkujE/5ILbagjtdyJaHF0wGOSlTnNT4W8j+zPJ/XK0I5EVQwtbmoc
 GY62LKxxz2pID6sPZV4zQVY4JdUQaFvOz1emnBxakkt0cq3Qnnqso1tjiy7vyH9CAwPR/48W
 fpK6dew4Fk+STYtBeixOTfSUS8qRS/wfpUeNa5RnEdTtFQ9IcjpQ/nPrvJJsu9FqwlpjMwAR
 AQABwsBlBBgBCAAPBQJX2BjLAhsMBQkSzAMAAAoJEEV84KCARGXFUKcH/jqKETECkbyPktdP
 cWVqw2ZIsmGxMkIdnZTbPwhORseGXMHadQODayhU9GWfCDdSPkWDWzMamD+qStfl9MhlVT60
 HTbo6wu1W/ogUS70qQPTY9IfsvAj6f8TlSlK0eLMa3s2UxL2oe5FkNs2CnVeRlr4Yqvp/ZQV
 6LXtew4GPRrmplUT/Cre9QIUqR4pxYCQaMoOXQQw3Y0csBwoDYUQujn3slbDJRIweHoppBzT
 rM6ZG5ldWQN3n3d71pVuv80guylX8+TSB8Mvkqwb5I36/NAFKl0CbGbTuQli7SmNiTAKilXc
 Y5Uh9PIrmixt0JrmGVRzke6+11mTjVlio/J5dCM=
In-Reply-To: <alpine.DEB.2.22.394.2305171246020.128889@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

17.05.2023 22:48, Stefano Stabellini :
> On Wed, 17 May 2023, Michael Tokarev wrote:
>> 17.05.2023 12:47, Chuck Zmudzinski wrote:
>>> On 5/17/2023 2:39 AM, Michael Tokarev wrote:
>>>> 08.02.2023 05:03, Chuck Zmudzinski wrote:...
>>>>> Fixes: 998250e97661 ("xen, gfx passthrough: register host bridge
>>>>> specific to passthrough")
>>>>> Signed-off-by: Chuck Zmudzinski <brchuckz@aol.com>
>>>>
>>>> Has this change been forgotten?  Is it not needed anymore?
...
>> It just neeeds to be picked up the usual way as all other qemu changes goes in.

> I am OK with this patch and acked it. However, I think it also needs an
> ack from one if the i386 maintainers, Michal T or Marcel.

Ping again?  It's been quite a while...
@mst, @marcel?

(Just looking at the stable-todo stuff)

/mjt



From xen-devel-bounces@lists.xenproject.org Sat Feb 17 10:09:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 17 Feb 2024 10:09:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682421.1061553 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbHd1-00007a-Fs; Sat, 17 Feb 2024 10:09:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682421.1061553; Sat, 17 Feb 2024 10:09:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbHd1-00007T-D9; Sat, 17 Feb 2024 10:09:03 +0000
Received: by outflank-mailman (input) for mailman id 682421;
 Sat, 17 Feb 2024 10:09:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbHd0-00007H-6H; Sat, 17 Feb 2024 10:09:02 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbHcz-0002Va-Qn; Sat, 17 Feb 2024 10:09:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbHcz-0001w5-C6; Sat, 17 Feb 2024 10:09:01 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rbHcz-0003Ft-B6; Sat, 17 Feb 2024 10:09:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=8pYsFY7HJ4/AvYVSFmmFnF1/QPEpviaxLY85P/1cJHQ=; b=Oz6VHPgN26UWSP4GaIZfmiXcJf
	5QVPVycKS8uBeO8a/okCXD7+EBxWf3GOn7vIU7aF9C7foo2nvrEqUowelpthhiElF887Uw37quseb
	Xgb7RU8FVRmD9lUx+XdVCDqaL4a7vRCZSfV6RmRCPkcQERCl7Ex5RDtvD2sbtTQ669uM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184687-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-6.1 test] 184687: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-6.1:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    linux-6.1:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=8b4118fabd6eb75fed19483b04dab3a036886489
X-Osstest-Versions-That:
    linux=f1bb70486c9c11d7e2d55240d4557f9fc575fbac
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 17 Feb 2024 10:09:01 +0000

flight 184687 linux-6.1 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184687/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 184595

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184595
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184595
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184595
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184595
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184595
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184595
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184595
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184595
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                8b4118fabd6eb75fed19483b04dab3a036886489
baseline version:
 linux                f1bb70486c9c11d7e2d55240d4557f9fc575fbac

Last test of basis   184595  2024-02-05 20:44:35 Z   11 days
Testing same since   184687  2024-02-16 18:13:56 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abhinav Kumar <quic_abhinavk@quicinc.com>
  Ajay Kaher <ajay.kaher@broadcom.com>
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Aring <aahringo@redhat.com>
  Alexander Hansen <alexander.hansen@9elements.com>
  Alexander Tsoy <alexander@tsoy.me>
  Allen Pais <apais@linux.microsoft.com>
  Antoine Tenart <atenart@kernel.org>
  Baokun Li <libaokun1@huawei.com>
  Chao Yu <chao@kernel.org>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Dan Carpenter <dan.carpenter@linaro.org>
  David Howells <dhowells@redhat.com>
  David S. Miller <davem@davemloft.net>
  David Teigland <teigland@redhat.com>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Eric Dumazet <edumazet@google.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Florian Westphal <fw@strlen.de>
  Francesco Dolcini <francesco.dolcini@toradex.com>
  Frank Li <Frank.Li@nxp.com>
  Frederic Weisbecker <frederic@kernel.org>
  Furong Xu <0x1207@gmail.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guanhua Gao <guanhua.gao@nxp.com>
  Guenter Roeck <linux@roeck-us.net>
  Hans de Goede <hdegoede@redhat.com>
  Ivan Vecera <ivecera@redhat.com>
  JackBB Wu <wojackbb@gmail.com>
  Jaegeuk Kim <jaegeuk@kernel.org>
  Jai Luthra <j-luthra@ti.com>
  Jakub Kicinski <kuba@kernel.org>
  Jason Gunthorpe <jgg@nvidia.com>
  Jens Axboe <axboe@kernel.dk>
  Jiri Pirko <jiri@nvidia.com>
  Jiri Wiesner <jwiesner@suse.de>
  Johan Hovold <johan@kernel.org>
  Johannes Berg <johannes.berg@intel.com>
  Jon Hunter <jonathanh@nvidia.com>
  Julian Sikorski <belegdol+github@gmail.com>
  Kelsey Steele <kelseysteele@linux.microsoft.com>
  kernelci.org bot <bot@kernelci.org>
  Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Kuogee Hsieh <quic_khsieh@quicinc.com>
  Leonard Dallmayr <leonard.dallmayr@mailbox.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Loic Prylli <lprylli@netflix.com>
  Martin K. Petersen <martin.petersen@oracle.com>
  Mateusz Jończyk <mat.jonczyk@o2.pl>
  Michael S. Tsirkin <mst@redhat.com>
  Mike Marciniszyn <mike.marciniszyn@intel.com>
  Ming Lei <ming.lei@redhat.com>
  Miquel Raynal <miquel.raynal@bootlin.com>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Paolo Abeni <pabeni@redhat.com>
  Paul E. McKenney <paulmck@kernel.org>
  Paul Menzel <pmenzel@molgen.mpg.de>
  Pavel Machek (CIP) <pavel@denx.de>
  Peter Ujfalusi <peter.ujfalusi@gmail.com>
  Prashanth K <quic_prashk@quicinc.com>
  Prathu Baronia <prathubaronia2011@gmail.com>
  Puliang Lu <puliang.lu@fibocom.com>
  Ron Economos <re@w6rz.net>
  Salvatore Bonaccorso <carnil@debian.org>
  Sasha Levin <sashal@kernel.org>
  Sean Young <sean@mess.org>
  SeongJae Park <sj@kernel.org>
  Sheng Yong <shengyong@oppo.com>
  Shigeru Yoshida <syoshida@redhat.com>
  Shiraz Saleem <shiraz.saleem@intel.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Shyam Prasad N <sprasad@microsoft.com>
  Simon Horman <horms@kernel.org>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Steve French <stfrench@microsoft.com>
  Sven Joachim <svenjoac@gmx.de>
  Takashi Iwai <tiwai@suse.de>
  Tejun Heo <tj@kernel.org>
  Theodore Ts'o <tytso@mit.edu>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Thomas Gleixner <tglx@linutronix.de>
  Tony Lindgren <tony@atomide.com>
  Vinod Koul <vkoul@kernel.org>
  Werner Sembach <wse@tuxedocomputers.com>
  Yann Sionneau <ysionneau@kalrayinc.com>
  Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
  Zhang Rui <rui.zhang@intel.com>
  Zhenyu Wang <zhenyuw@linux.intel.com>
  Zhipeng Lu <alexious@zju.edu.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   f1bb70486c9c..8b4118fabd6e  8b4118fabd6eb75fed19483b04dab3a036886489 -> tested/linux-6.1


From xen-devel-bounces@lists.xenproject.org Sat Feb 17 16:55:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 17 Feb 2024 16:55:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682477.1061568 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbNy6-00007R-Hc; Sat, 17 Feb 2024 16:55:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682477.1061568; Sat, 17 Feb 2024 16:55:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbNy6-00007K-Db; Sat, 17 Feb 2024 16:55:14 +0000
Received: by outflank-mailman (input) for mailman id 682477;
 Sat, 17 Feb 2024 16:55:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbNy5-000078-6S; Sat, 17 Feb 2024 16:55:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbNy5-00021k-2F; Sat, 17 Feb 2024 16:55:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbNy4-0005fD-Mz; Sat, 17 Feb 2024 16:55:12 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rbNy4-0000fM-MR; Sat, 17 Feb 2024 16:55:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SMlkz/c8kMsQio6hHnJIFn/dIuMVMt3x/lsOU6LPYN4=; b=Kq2VNyxCWHF/HcZ+XWZrblGdCV
	9/p4LmEpbVDvBVppA8fXAIItxFMbQklFE4hY12Fpb7+jo+5UCAs8bFS09zJTv7uDhEAnitU0VJkBG
	pyCRCw7X2XBzJk442+wQr2WaxK2oR62a3YetGs4ec+MNMl9Vbhskf6tKNXLD1UUUQ3xc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184688-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184688: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
X-Osstest-Versions-That:
    xen=78398afae10bfb4ab94e8af17b7ed58510a57d96
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 17 Feb 2024 16:55:12 +0000

flight 184688 xen-unstable real [real]
flight 184692 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184688/
http://logs.test-lab.xenproject.org/osstest/logs/184692/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install fail pass in 184692-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop  fail in 184692 like 184684
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184684
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184684
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184684
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184684
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184684
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184684
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184684
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184684
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184684
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184684
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184684
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3
baseline version:
 xen                  78398afae10bfb4ab94e8af17b7ed58510a57d96

Last test of basis   184684  2024-02-16 10:54:02 Z    1 days
Testing same since   184688  2024-02-17 00:40:49 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Elias El Yandouzi <eliasely@amazon.com>
  Julien Grall <jgrall@amazon.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   78398afae1..0441c3acc7  0441c3acc7e9e72e984ce49d32e61827894ae4a3 -> master


From xen-devel-bounces@lists.xenproject.org Sat Feb 17 18:16:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 17 Feb 2024 18:16:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682485.1061578 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbPEW-0000Dw-PU; Sat, 17 Feb 2024 18:16:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682485.1061578; Sat, 17 Feb 2024 18:16:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbPEW-0000Dp-MJ; Sat, 17 Feb 2024 18:16:16 +0000
Received: by outflank-mailman (input) for mailman id 682485;
 Sat, 17 Feb 2024 18:16:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbPEU-0000Df-S6; Sat, 17 Feb 2024 18:16:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbPEU-00059P-Hu; Sat, 17 Feb 2024 18:16:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbPEU-0000Uj-59; Sat, 17 Feb 2024 18:16:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rbPEU-00055P-4m; Sat, 17 Feb 2024 18:16:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ByS0z3ij5ea6s3P3GYp0CKW1oC1E5ClCbciDTWJZOHs=; b=WLujaSm+1PmoYI8N5qNMKuiG4a
	P5ZosXOPQSZKU06R68ZCOWnoYMGt+VvhfF5bjYRmDJlZcm95CHtBRfqAuV6Oxp+Pbqk63ROK404Go
	XVXQnnWyVByxb0+GZWCMdXltrTwN7sfPnx+APurSUfUVRjkYdN4W8wwxyoCN49MCFG/4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184689-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184689: tolerable all pass - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=8d9e1f771834f924b41c4dd93f1fde327aef18ff
X-Osstest-Versions-That:
    libvirt=50bca3d45b58a6dd51d3792cc856f4f49fe88bea
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 17 Feb 2024 18:16:14 +0000

flight 184689 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184689/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184682
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184682
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184682
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              8d9e1f771834f924b41c4dd93f1fde327aef18ff
baseline version:
 libvirt              50bca3d45b58a6dd51d3792cc856f4f49fe88bea

Last test of basis   184682  2024-02-16 04:20:41 Z    1 days
Testing same since   184689  2024-02-17 04:20:47 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Göran Uddeborg <goeran@uddeborg.se>
  Peter Krempa <pkrempa@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   50bca3d45b..8d9e1f7718  8d9e1f771834f924b41c4dd93f1fde327aef18ff -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sat Feb 17 21:01:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 17 Feb 2024 21:01:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682494.1061588 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbRoF-0000oo-9Y; Sat, 17 Feb 2024 21:01:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682494.1061588; Sat, 17 Feb 2024 21:01:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbRoF-0000oh-63; Sat, 17 Feb 2024 21:01:19 +0000
Received: by outflank-mailman (input) for mailman id 682494;
 Sat, 17 Feb 2024 21:01:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DWb+=J2=onlineschubla.de=paul@srs-se1.protection.inumbo.net>)
 id 1rbRoC-0000ob-Up
 for xen-devel@lists.xenproject.org; Sat, 17 Feb 2024 21:01:17 +0000
Received: from mo4-p00-ob.smtp.rzone.de (mo4-p00-ob.smtp.rzone.de
 [81.169.146.162]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ae4cd391-cdd7-11ee-98f5-efadbce2ee36;
 Sat, 17 Feb 2024 22:01:12 +0100 (CET)
Received: from mail.onlineschubla.de by smtp.strato.de (RZmta 49.11.2 AUTH)
 with ESMTPSA id Y5995401HL18yWF
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits))
 (Client did not present a certificate);
 Sat, 17 Feb 2024 22:01:08 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by mail.onlineschubla.de (Postfix) with ESMTP id 7229E20BF4;
 Sat, 17 Feb 2024 22:01:08 +0100 (CET)
Received: from mail.onlineschubla.de ([127.0.0.1])
 by localhost (mail.onlineschubla.de [127.0.0.1]) (amavis, port 10024)
 with ESMTP id UYWYfHpZfOkh; Sat, 17 Feb 2024 22:01:07 +0100 (CET)
Received: from [10.0.0.105] (unknown [10.0.0.105])
 by mail.onlineschubla.de (Postfix) with ESMTPA id 9DD8A202E1;
 Sat, 17 Feb 2024 22:01:07 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ae4cd391-cdd7-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; t=1708203668; cv=none;
    d=strato.com; s=strato-dkim-0002;
    b=NTaJQp2IQUWHCgm4COaB2kgwpdTzzEBTvnnPMGIDhSduk/xZ5ni7RIDoH8UU16lt1D
    lr5YogzyaVYAgq5hrgYaVUJGDGPjmHuhUw04XNdQnNyQGj2Q8Qkk7JXYq63VRUlRNLjE
    rNzZ531kJdWlReeAIV1um5IpNinUIlAGGCiWOZY/IbfHuKl6rF6kMTCrm5HBz/D58Wim
    rNHFrmUA3gTxua1J6QGk5aK4HzVpx8kCW63qDpdRcubdWKN8N53xcXi2hKWDkjmKkTaT
    QGMl7MGPIrOwLUkaM4R6LnjJo7d6zGRkVwG3omBulGcwZwTx+H6lfaXVHx7IjnNfP3nh
    udQQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1708203668;
    s=strato-dkim-0002; d=strato.com;
    h=In-Reply-To:From:References:Cc:To:Subject:Date:Message-ID:Cc:Date:
    From:Subject:Sender;
    bh=X1wJzkstUc4HfiIEHygOoDaGxhWJmdEyuK1qZf2SRw8=;
    b=FrZ1T2Og7enpTuuHcKRLwlML3T5SSuRU6zvOuCZIK+jZNM9eQSHsm+fL3utJRki2JN
    0hKjFFBlNdwrCqp8yGXXvWgzAS7smT7ZEnnNzz1KoRpu1b9Fi0nhFk9Eg7BkbzKMSK2N
    KtKM8e75Dq6BYvtE5IZtDOPUxzsP8irq7UG/Qx59d4/UjBKUWKUswRVpYa36eS5LNW4g
    lMZazTgjkI2t+HeWAQmtMfi5AwKIOqg1Urx/15Ph0nM0xB8+RKgA59AOjLaxCMp/Ie5s
    sYks+wIzn9C+jZfGuQuaxtQXE6imhgYSFGQD1i090sMwfXbTHZam1rDsntBGRz67/6jE
    5LEw==
ARC-Authentication-Results: i=1; strato.com;
    arc=none;
    dkim=none
X-RZG-CLASS-ID: mo00
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1708203668;
    s=strato-dkim-0002; d=onlineschubla.de;
    h=In-Reply-To:From:References:Cc:To:Subject:Date:Message-ID:Cc:Date:
    From:Subject:Sender;
    bh=X1wJzkstUc4HfiIEHygOoDaGxhWJmdEyuK1qZf2SRw8=;
    b=juxdaj/09B1zjuv4GrmdXtpJYR+cmU8Oy2u89+QROqJEkJlWe14w4nLFKBAv1ebscM
    VnGl8KIy8ETeWZOmAxioFG50l3NIJ7UfqvU2VeQb6bGfokG46Db9tR8AZ9gAy1aUdq57
    Ki7EYnUkBjkp/WOy0sDMTiBZPY0jLGnXYGluh/UTpWEkzPsnsc9NxxPwE6OnIVeQe7DA
    ofsOnUQlk1ERttD9S/RjVlR7E8kr0Di4t5ymlk0HUUphD2rOE1UPikW+p2j7+7Jc0dU3
    HJ2rGX/2IxpVuGFblUBLR6+7KFXNPLzMlJ0B0PyC/6oRu3frmWJbvLL0P0pK3h23pk9G
    W5jg==
DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; t=1708203668;
    s=strato-dkim-0003; d=onlineschubla.de;
    h=In-Reply-To:From:References:Cc:To:Subject:Date:Message-ID:Cc:Date:
    From:Subject:Sender;
    bh=X1wJzkstUc4HfiIEHygOoDaGxhWJmdEyuK1qZf2SRw8=;
    b=+OrXcgJCsoi7TemNZf0fBEQEKQwBRip1Oi7jenz8wL8tbzBgMqZiVa0l0DILwEOXmG
    Av63YJGZpfop+BcB7cDg==
X-RZG-AUTH: ":PG0ReWCndfO3rCSML4AvNaDxJ7WJyilEI/NMX3IPsStZLbcZC9mNZx0oO9uVFQeS3s1LjmLQORrCeiSIlV428qMUC/ada3WaVebykAvjCQ=="
X-Virus-Scanned: Debian amavis at onlineschubla.de
Message-ID: <ab6cb863-4aee-4a4e-81c6-b3c0ca8cef13@onlineschubla.de>
Date: Sat, 17 Feb 2024 22:01:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen 4.18rc/ARM64 on Raspberry Pi 4B: Traffic in DomU crashing
 Dom0 when using VLANs
Content-Language: en-US
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>
References: <838ff75f-1352-4d3c-9bc1-c7a57c828231@onlineschubla.de>
 <4cddfd89-f195-4a50-a14d-b86121414e56@onlineschubla.de>
 <f8f39239-ea95-4fc7-9abe-6ca005eb02d2@onlineschubla.de>
 <ZarcWQ6Ugwowy050@mattapan.m5p.com>
 <CA+zSX=Y=BsVEp3o6jRprn5sntWVA0Z6wNXUxtffDN+=fPv_Fzg@mail.gmail.com>
 <D2496DDC-78FA-4CC1-B7FC-EB62E506BC4F@arm.com>
 <1572ed53-494d-4dd0-b470-85ff2efb229b@onlineschubla.de>
 <D24E1CA2-4033-455D-A97A-3D35547FAE76@arm.com>
From: Paul Leiber <paul@onlineschubla.de>
In-Reply-To: <D24E1CA2-4033-455D-A97A-3D35547FAE76@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Am 25.01.2024 um 09:12 schrieb Bertrand Marquis:
> Hi Paul,
> 
> Please keep me in CC so that i can more easily find out that you answered me :-)
> 
>> On 25 Jan 2024, at 00:05, Paul Leiber <paul@onlineschubla.de> wrote:
>>
>> Elliot, Bertrand, George: Thanks for your replies.
>>
>> Am 23.01.2024 um 09:29 schrieb Bertrand Marquis:
>>> Hi,
>>>   will try to explain some of the messages here after but I am not sure of the reason
>>> of the crash so I will just set some pointers...
>>>> On 22 Jan 2024, at 11:46, George Dunlap <george.dunlap@cloud.com> wrote:
>>>>
>>>> On Fri, Jan 19, 2024 at 8:32 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
>>>>>
>>>>> On Sun, Jan 14, 2024 at 10:54:24PM +0100, Paul Leiber wrote:
>>>>>>
>>>>>>>> TL;DR:
>>>>>>>> ------
>>>>>>>>
>>>>>>>> Causing certain web server traffic on a secondary VLAN on Raspberry Pi
>>>>>>>> under vanilla Debian/UEFI in combination with Xen leads to complete
>>>>>>>> system reboot (watchdog triggering for Dom0). Other strange things are
>>>>>>>> happening.
>>>>>>>>

[snip]

>>>>>>
>>>>>> Not knowing if it helps, I added Xen logs from boot until Dom0
>>>>>> crash/reboot below.
>>>>>>
>>>>>> Loading Xen xen ...
>>>>>> Loading Linux 6.1.0-17-arm64 ...
>>>>>> Loading initial ramdisk ...
>>>>>> Using modules provided by bootloader in FDT
>>>>>> Xen 4.19-unstable (c/s Fri Jan 12 11:54:31 2024 +0000
>>>>>> git:1ec3fe1f66-dirty) EFI
>>>>>> loader
>>>>>>   Xen 4.19-unstable
>>>>>> (XEN) Xen version 4.19-unstable (root@lan.onlineschubla.de) (gcc (Debian
>>>>>> 12.2.0-
>>>>>>
>>>>>>
>>>>>>                             14) 12.2.0) debug=y Sun Jan 14 21:46:34 CET 2024
>>>>>> (XEN) Latest ChangeSet: Fri Jan 12 11:54:31 2024 +0000 git:1ec3fe1f66-dirty
>>>>>> (XEN) build-id: babb03cb6107fc46f7d8969142ccd6772a1133c3
>>>>>> (XEN) Console output is synchronous.
>>>>>> (XEN) Processor: 00000000410fd083: "ARM Limited", variant: 0x0, part
>>>>>> 0xd08,rev 0
>>>>>>
>>>>>>
>>>>>>                                 x3
>>>>>> (XEN) 64-bit Execution:
>>>>>> (XEN)   Processor Features: 0000000000002222 0000000000000000
>>>>>> (XEN)     Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
>>>>>> (XEN)     Extensions: FloatingPoint AdvancedSIMD
>>>>>> (XEN)   Debug Features: 0000000010305106 0000000000000000
>>>>>> (XEN)   Auxiliary Features: 0000000000000000 0000000000000000
>>>>>> (XEN)   Memory Model Features: 0000000000001124 0000000000000000
>>>>>> (XEN)   ISA Features:  0000000000010000 0000000000000000
>>>>>> (XEN) 32-bit Execution:
>>>>>> (XEN)   Processor Features: 0000000000000131:0000000000011011
>>>>>> (XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
>>>>>> (XEN)     Extensions: GenericTimer Security
>>>>>> (XEN)   Debug Features: 0000000003010066
>>>>>> (XEN)   Auxiliary Features: 0000000000000000
>>>>>> (XEN)   Memory Model Features: 0000000010201105 0000000040000000
>>>>>> (XEN)                          0000000001260000 0000000002102211
>>>>>> (XEN)   ISA Features: 0000000002101110 0000000013112111 0000000021232042
>>>>>> (XEN)                 0000000001112131 0000000000011142 0000000000010001
>>>>>> (XEN) Using SMC Calling Convention v1.2
>>>>>> (XEN) Using PSCI v1.1
>>>>>> (XEN) ACPI: GICC (acpi_id[0x0000] address[0xff842000] MPIDR[0x0] enabled)
>>>>>> (XEN) ACPI: GICC (acpi_id[0x0001] address[0xff842000] MPIDR[0x1] enabled)
>>>>>> (XEN) ACPI: GICC (acpi_id[0x0002] address[0xff842000] MPIDR[0x2] enabled)
>>>>>> (XEN) ACPI: GICC (acpi_id[0x0003] address[0xff842000] MPIDR[0x3] enabled)
>>>>>> (XEN) 4 CPUs enabled, 4 CPUs total
>>>>>> (XEN) SMP: Allowing 4 CPUs
>>>>>> (XEN) enabled workaround for: ARM erratum 1319537
>>>>>> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 54000 KHz
>>>>>> (XEN) GICv2 initialization:
>>>>>> (XEN)         gic_dist_addr=00000000ff841000
>>>>>> (XEN)         gic_cpu_addr=00000000ff842000
>>>>>> (XEN)         gic_hyp_addr=00000000ff844000
>>>>>> (XEN)         gic_vcpu_addr=00000000ff846000
>>>>>> (XEN)         gic_maintenance_irq=25
>>>>>> (XEN) GICv2: 256 lines, 4 cpus, secure (IID 0200143b).
>>>>>> (XEN) XSM Framework v1.0.1 initialized
>>>>>> (XEN) Initialising XSM SILO mode
>>>>>> (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
>>>>>> (XEN) Initializing Credit2 scheduler
>>>>>> (XEN)  load_precision_shift: 18
>>>>>> (XEN)  load_window_shift: 30
>>>>>> (XEN)  underload_balance_tolerance: 0
>>>>>> (XEN)  overload_balance_tolerance: -3
>>>>>> (XEN)  runqueues arrangement: socket
>>>>>> (XEN)  cap enforcement granularity: 10ms
>>>>>> (XEN) load tracking window length 1073741824 ns
>>>>>> (XEN) Allocated console ring of 32 KiB.
>>>>>> (XEN) CPU0: Guest atomics will try 16 times before pausing the domain
>>>>>> (XEN) Bringing up CPU1
>>>>>> (XEN) CPU1: Guest atomics will try 16 times before pausing the domain
>>>>>> (XEN) CPU 1 booted.
>>>>>> (XEN) Bringing up CPU2
>>>>>> (XEN) CPU2: Guest atomics will try 13 times before pausing the domain
>>>>>> (XEN) CPU 2 booted.
>>>>>> (XEN) Bringing up CPU3
>>>>>> (XEN) CPU3: Guest atomics will try 16 times before pausing the domain
>>>>>> (XEN) Brought up 4 CPUs
>>>>>> (XEN) CPU 3 booted.
>>>>>> (XEN) I/O virtualisation disabled
>>>>>> (XEN) P2M: 44-bit IPA with 44-bit PA and 8-bit VMID
>>>>>> (XEN) P2M: 4 levels with order-0 root, VTCR 0x0000000080043594
>>>>>> (XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
>>>>>> (XEN) Initializing Credit2 scheduler
>>>>>> (XEN)  load_precision_shift: 18
>>>>>> (XEN)  load_window_shift: 30
>>>>>> (XEN)  underload_balance_tolerance: 0
>>>>>> (XEN)  overload_balance_tolerance: -3
>>>>>> (XEN)  runqueues arrangement: socket
>>>>>> (XEN)  cap enforcement granularity: 10ms
>>>>>> (XEN) load tracking window length 1073741824 ns
>>>>>> (XEN) Adding cpu 0 to runqueue 0
>>>>>> (XEN)  First cpu on runqueue, activating
>>>>>> (XEN) Adding cpu 1 to runqueue 0
>>>>>> (XEN) Adding cpu 2 to runqueue 0
>>>>>> (XEN) Adding cpu 3 to runqueue 0
>>>>>> (XEN) alternatives: Patching with alt table 00000a00002ee0b0 ->
>>>>>> 00000a00002ef250
>>>>>> (XEN) CPU2 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>>>>> (XEN) CPU1 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>>>>> (XEN) CPU3 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>>>>> (XEN) CPU0 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>>>>> (XEN) *** LOADING DOMAIN 0 ***
>>>>>> (XEN) Loading d0 kernel from boot module @ 0000000030ef7000
>>>>>> (XEN) Loading ramdisk from boot module @ 000000002ee6d000
>>>>>> (XEN) Allocating 1:1 mappings totalling 1024MB for dom0:
>>>>>> (XEN) BANK[0] 0x00000040000000-0x00000080000000 (1024MB)
>>>>>> (XEN) Grant table range: 0x0000002eceb000-0x0000002ed2b000
>>>>>> (XEN) Allocating PPI 16 for event channel interrupt
>>>>>> (XEN) Loading zImage from 0000000030ef7000 to
>>>>>> 0000000040000000-0000000041f1c7c0
>>>>>> (XEN) Loading d0 initrd from 000000002ee6d000 to
>>>>>> 0x0000000048200000-0x000000004a
>>>>>>
>>>>>>
>>>>>>                                                     288c22
>>>>>> (XEN) Loading d0 DTB to 0x0000000048000000-0x00000000480002c3
>>>>>> (XEN) Initial low memory virq threshold set at 0x4000 pages.
>>>>>> (XEN) Scrubbing Free RAM in background
>>>>>> (XEN) Std. Loglevel: All
>>>>>> (XEN) Guest Loglevel: All
>>>>>> (XEN) ***************************************************
>>>>>> (XEN) WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
>>>>>> (XEN) This option is intended to aid debugging of Xen by ensuring
>>>>>> (XEN) that all output is synchronously delivered on the serial line.
>>>>>> (XEN) However it can introduce SIGNIFICANT latencies and affect
>>>>>> (XEN) timekeeping. It is NOT recommended for production use!
>>>>>> (XEN) ***************************************************
>>>>>> (XEN) 3... 2... 1...
>>>>>> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
>>>>>> (XEN) Freed 376kB init memory.
>>>>>> (XEN) d0v0 Unhandled SMC/HVC: 0x84000050
>>>>>> (XEN) d0v0 Unhandled SMC/HVC: 0x8600ff01
>>> This means Linux is trying to do some kind of smc call which is not handled.
>>> I will highly suggest to investigate here as it could really be some platform
>>> specific calls trying to enable some clocks.
>>>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER4
>>>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER8
>>>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER12
>>>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER16
>>>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER20
>>>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER24
>>>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER28
>>>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>>>>> (XEN) d0v1: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>>>>> (XEN) d0v2: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>>>>> (XEN) d0v3: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>> Those can be ignored. This is Linux writing to some GIC registers we do not
>>> emulate so we warn that those are ignored. Those messages are present on
>>> all arm targets when the log level of Xen is high.
>>>>>> (XEN) d1v0 Unhandled SMC/HVC: 0x84000050
>>>>>> (XEN) d1v0 Unhandled SMC/HVC: 0x8600ff01
>>> Same as upper done on a different core
>>>>>> (XEN) d1v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>> Can be ignored as explained earlier
>>>>>> (XEN) common/grant_table.c:1909:d1v0 Expanding d1 grant table from 1 to
>>>>>> 2 frames
>>> Those are internal debug messages to Xen and can be ignored (they are present
>>> on most arm targets).
>>>>>> (XEN) gnttab_mark_dirty not implemented yet
>>> Can be ignored to
>>>>>> (XEN) d2v0 Unhandled SMC/HVC: 0x84000050
>>>>>> (XEN) d2v0 Unhandled SMC/HVC: 0x8600ff01
>>> Same as upper on a different core
>>>>>> (XEN) d2v0: vGICD: unhandled word write 0x000000
>>> Same as upper on a different core
>>
>> Is this also relevant or not?
> 
> This one is weird and i cannot identify where it could come from as all printk in vGICD
> have something after. Maybe specific to your version of Xen.
> But I would say you can ignore.
> 
>>
>> I couldn't identify this yet. There is a mailing list exchange that could be related:
>>
>> https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg00382.html
>>
>> Although I'm a little bit puzzled because when comparing with the other messages, this log meassage seems to be missing some parts "vGICD: unhandled word write 0x000000" instead of "vGICD: unhandled word write 0x000000xxxxxxx to y"
> 
> Agree, could be that something is lost in the console.
> 
>>
>>>>>> ffffffff to ICACTIVER0
>>>>>> (XEN) common/grant_table.c:1909:d2v0 Expanding d2 grant table from 1 to
>>>>>> 2 frames
>>>>>> (XEN) common/grant_table.c:1909:d2v0 Expanding d2 grant table from 2 to
>>>>>> 3 frames
>>>>>> (XEN) common/grant_table.c:1909:d2v0 Expanding d2 grant table from 3 to
>>>>>> 4 frames
>>>>>> (XEN) Watchdog timer fired for domain 0
>>>>>> (XEN) Hardware Dom0 shutdown: watchdog rebooting machine
>>>>>
>>>>> I'm unsure whose attention to draw to this report.
>>>>>
>>>>> This might be a scheduler issue since the watchdog timer is triggering.
>>>>>
>>>>> This might be an ACPI issue as ACPI is in use here.
>>>>>
>>>>> This might be an ARM Linux kernel issue.
>>>>>
>>>>> In the end this is someone running into trouble with Xen on an ARM
>>>>> device.  Yet despite bringing up the issue hasn't gotten any help...
>>>>
>>>> Hey Elliot,
>>>>
>>>> Thanks for raising the visibility of this.  I'm not familiar with ARM,
>>>> but if I were investigating this I'd try to figure out what the
>>>> "unhandled" error messages are.  "gnttab_mark_dirty not implemented
>>>> yet" looks pretty sus too, and also sounds like it might be something
>>>> ARM-specific.
>>> I tried to explain those and they are not the reason of the problem.
>>>>
>>>> I don't see anything suspicious WRT the scheduler, but a simple way to
>>>> test that would be to set the scheduler to credit1 and see if that
>>>> changes things.
>>> I would definitely suggest to investigate the 2 unhandled SMC/HVC calls
>>> as if one of them is to enable some clock it could explain why the system is
>>> getting stuck at some point (maybe waiting for something to be started).
>>> Other than that i did not see anything that could point to an obvious issue.
>>
>> I'll try to use credit1 scheduler and report results when I've got some more time, just to make sure.
> 
> This is not related to the scheduler. When i said clock i meant linux trying to configure the clock
> frequency or voltage of a specific peripheral.
> 
>>
>>  From what I could find out in ARM documentation, the unhandled calls seem to be the following:
>>
>> 0x84000050 = TRNG_VERSION, returns the implemented TRNG (True Random Number Generator) ABI version [1]
> 
> Could be the origin but then you can ignore i would say, linux might fall back to an other way.

The SMC code 0x84000050 is defined in include/linux/arm-smccc.h as 
constant ARM_SMCCC_TRNG_VERSION. This constant is used in six source 
files [1]. Two source files are smccc.h itself:

include/linux/arm-smccc.h
tools/include/linux/arm-smccc.h).

Three have to do with KVM:

arch/arm64/kvm/trng.c
arch/arm64/kvm/hypercalls.c
tools/testing/selftests/kvm/aarch64/hypercalls.c

One sticks out as not KVM related:

arch/arm64/include/asm/archrandom.h


>> 0x8600ff01 = Call UID Query for Vendor Specific Hypervisor Service, Returns a unique identifier of the service provider [2]
> 
> I would say this one is the problem and you would need to dig in Linux to understand what vendor specific call is being made.
> Linux might expect that this one is done but Xen is ignoring it.
> So Linux could just be waiting for something that was actually not started at all.
> 
>>
>> Does this make sense?
>>
>> What could be next steps?
> 
> I would say understand in Linux who is generating this call and try to disable in the linux
> configuration whatever clock/voltage/performance driver is doing this call.


The SMC code 0x8600ff01 is defined in include/linux/arm-smccc.h as 
constant ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID. This constant is also 
used in six source files [2], 2 of which again are smccc.h itself

include/linux/arm-smccc.h
tools/include/linux/arm-smccc.h

One is KVM documentation:

Documentation/virt/kvm/arm/hypercalls.rst

The three remaining again have to do with KVM:

drivers/firmware/smccc/kvm_guest.c
arch/arm64/kvm/hypercalls.c
tools/testing/selftests/kvm/aarch64/hypercalls.c

I wouldn't expect that anything KVM related could be the cause for the 
Xen watchdog triggering. If this is right, this would leave us with the 
TRNG trace in arch/arm64/include/asm/archrandom.h. Or am I wrong, and 
some KVM component can also be the cause?

Also, what still puzzles me is the fact that the Xen watchdog is only 
triggered when multiple VLANs are in use. Where could the connection to 
networking be?

Best regards,

Paul

[1] 
https://github.com/search?q=repo%3Atorvalds%2Flinux+ARM_SMCCC_TRNG_VERSION&type=code
[2] 
https://github.com/search?q=repo%3Atorvalds/linux%20ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID&type=code




From xen-devel-bounces@lists.xenproject.org Sat Feb 17 21:46:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 17 Feb 2024 21:46:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682501.1061598 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbSVv-0005tX-HG; Sat, 17 Feb 2024 21:46:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682501.1061598; Sat, 17 Feb 2024 21:46:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbSVv-0005tQ-E4; Sat, 17 Feb 2024 21:46:27 +0000
Received: by outflank-mailman (input) for mailman id 682501;
 Sat, 17 Feb 2024 21:46:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DWb+=J2=onlineschubla.de=paul@srs-se1.protection.inumbo.net>)
 id 1rbSVt-0005t7-7g
 for xen-devel@lists.xenproject.org; Sat, 17 Feb 2024 21:46:25 +0000
Received: from mo4-p01-ob.smtp.rzone.de (mo4-p01-ob.smtp.rzone.de
 [81.169.146.166]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fde5d689-cddd-11ee-8a52-1f161083a0e0;
 Sat, 17 Feb 2024 22:46:23 +0100 (CET)
Received: from mail.onlineschubla.de
 by smtp.strato.de (RZmta 49.11.2 DYNA|AUTH)
 with ESMTPSA id Y5995401HLkIyeC
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits))
 (Client did not present a certificate);
 Sat, 17 Feb 2024 22:46:18 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by mail.onlineschubla.de (Postfix) with ESMTP id 677A220BF4;
 Sat, 17 Feb 2024 22:46:18 +0100 (CET)
Received: from mail.onlineschubla.de ([127.0.0.1])
 by localhost (mail.onlineschubla.de [127.0.0.1]) (amavis, port 10024)
 with ESMTP id TISlxg0crh8I; Sat, 17 Feb 2024 22:46:17 +0100 (CET)
Received: from [10.0.0.105] (unknown [10.0.0.105])
 by mail.onlineschubla.de (Postfix) with ESMTPA id 8723B202E1;
 Sat, 17 Feb 2024 22:46:17 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fde5d689-cddd-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; t=1708206379; cv=none;
    d=strato.com; s=strato-dkim-0002;
    b=qArFcFryLORxetwIaa8gzqwOurYgFvcE3CCYJH9FxjSbZ8alLaMCVLARYMmgAJccRY
    yyVqDFzCk0o47kV8cUWx0gEom+qlHG9Z/VID3cYqY4G7CNoHZO8r5HRksMPdPmawwgP/
    +aMfcBon1S/qSG9BRe+RjFR32C1NjE+nye2scerbqVYjvwT9E5uJdIl2uLTAclD1YPq3
    l6rO5eno14ZQL5xPT/Uh0h/CQJxDloNhumywgoF6gf4ClDasnVXwKCk7oBk7h5LeNdtU
    YTOVqFU+9yZj4IjJez8QbzkG8YymEvsF2uBZgj3+iAx2IQca7ltXSq9zNx0nebxCdMiQ
    fglw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1708206379;
    s=strato-dkim-0002; d=strato.com;
    h=In-Reply-To:From:References:Cc:To:Subject:Date:Message-ID:Cc:Date:
    From:Subject:Sender;
    bh=BMA8RwyMKx95emC5xeD/sfznS1HXhwqQhcWTZMNX12E=;
    b=Zc4gr4UEDtMAw+vBQXpeCcb1L6xSQAHU36Z+W4AKJ0Db8q0vkzMjJG169Z4mBf6gde
    iOmBea0ftvGxApILbMyC6/IJ/n5NzJanceiG9QyTqov02AzhuoyAk+ZtePbFO2P11ntO
    QUfeytNE9g1MOOV7nfmAXPY77y3toQ3cG7yRlsNB8AMKJ2QRkIm7eISJIl/h5t8+sY0+
    vVNuS/NG8xa6jJf9ks6xjdC+zZ2xETfCqnNjrtpI7enxIz9+PH3zgi360uo7hq32kWio
    UQ0dcvJ4shIZ9iwpDR1kXcQ8XXdvW6HVI7v4CkVJufIulGDNnQ4irPnxcZpKwm1sg3nM
    0e3A==
ARC-Authentication-Results: i=1; strato.com;
    arc=none;
    dkim=none
X-RZG-CLASS-ID: mo01
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1708206379;
    s=strato-dkim-0002; d=onlineschubla.de;
    h=In-Reply-To:From:References:Cc:To:Subject:Date:Message-ID:Cc:Date:
    From:Subject:Sender;
    bh=BMA8RwyMKx95emC5xeD/sfznS1HXhwqQhcWTZMNX12E=;
    b=UBpwwnV4oor2ctCI9DYlkaloduoX0QYaorpHI8MVmqibj0RanKMCq09WVNuy2GNVRj
    vwy/CEEQ5V0OSoHEa1WDzSZ8uZCYNN9tcKZV8Gs4fhR6Rsacq8IdVSA21VietNFFeiyi
    3vdhiGJBHN5zPueuf+ZoUtD0njgjb+dJrjZ+cHNiqHsuLf3/BiZQ2wpiajOBR6RyMFo6
    ACUWKIp2s23U6bVL9ssxUI6rXiS9Xcsfz9BUa9FT15ScqArsfDTCADDjmDoa5q3dBn+i
    fqLpc+HiBk8GEdbTsWf6148AhtXYpIUL3tfuqLOL//GCRm3MJ44+eeqQsb7ArnJzTCgx
    uqXQ==
DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; t=1708206379;
    s=strato-dkim-0003; d=onlineschubla.de;
    h=In-Reply-To:From:References:Cc:To:Subject:Date:Message-ID:Cc:Date:
    From:Subject:Sender;
    bh=BMA8RwyMKx95emC5xeD/sfznS1HXhwqQhcWTZMNX12E=;
    b=VHtATPv/t6DOkZEK9IWBaDZ6EvbNzzDE/xVETCWPxN2rOQ7dyv9KmYothH9buB7wap
    HkmRAlu1IcPB2au9n7Aw==
X-RZG-AUTH: ":PG0ReWCndfO3rCSML4AvNaDxJ7WJyilEI/NMX3IPsStZLbcZC9mNZx0oO96cCwabxEcBmN8/iSSk"
X-Virus-Scanned: Debian amavis at onlineschubla.de
Message-ID: <4d7dbce9-ef20-4702-8056-9d5ab8152721@onlineschubla.de>
Date: Sat, 17 Feb 2024 22:46:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen 4.18rc/ARM64 on Raspberry Pi 4B: Traffic in DomU crashing
 Dom0 when using VLANs
Content-Language: en-US
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, george.dunlap@cloud.com,
 Julien Grall <julien@xen.org>, Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>
References: <838ff75f-1352-4d3c-9bc1-c7a57c828231@onlineschubla.de>
 <4cddfd89-f195-4a50-a14d-b86121414e56@onlineschubla.de>
 <f8f39239-ea95-4fc7-9abe-6ca005eb02d2@onlineschubla.de>
 <ZarcWQ6Ugwowy050@mattapan.m5p.com>
 <CA+zSX=Y=BsVEp3o6jRprn5sntWVA0Z6wNXUxtffDN+=fPv_Fzg@mail.gmail.com>
From: Paul Leiber <paul@onlineschubla.de>
In-Reply-To: <CA+zSX=Y=BsVEp3o6jRprn5sntWVA0Z6wNXUxtffDN+=fPv_Fzg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Am 22.01.2024 um 11:46 schrieb George Dunlap:
> On Fri, Jan 19, 2024 at 8:32 PM Elliott Mitchell <ehem+xen@m5p.com> wrote:
>>
>> On Sun, Jan 14, 2024 at 10:54:24PM +0100, Paul Leiber wrote:
>>>
>>> Am 22.10.2023 um 07:42 schrieb Paul Leiber:
>>>> Am 13.10.2023 um 20:56 schrieb Paul Leiber:
>>>>> Hi Xen developers list,
>>>>>
>>>>> TL;DR:
>>>>> ------
>>>>>
>>>>> Causing certain web server traffic on a secondary VLAN on Raspberry Pi
>>>>> under vanilla Debian/UEFI in combination with Xen leads to complete
>>>>> system reboot (watchdog triggering for Dom0). Other strange things are
>>>>> happening.
>>>>>
>>>>> Description:
>>>>> ----------
>>>>>
>>>>> I recently set up Xen (self compiled, Version 4.18-rc) on a Raspberry
>>>>> Pi 4B (on vanilla Debian Bookworm, UEFI boot mode). Until some time
>>>>> ago, everything worked well with Dom0, one DomU and one bridge.
>>>>>
>>>>> Then I wanted to actually make use of the virtualization and started
>>>>> to set up a second Debian Bookworm DomU (using xen-create-image) for
>>>>> monitoring my systems with zabbix (a webserver based system monitoring
>>>>> solution). The bridge used for this setup was the device bridging the
>>>>> hardware NIC. I installed zabbix, set it up, and everything went well,
>>>>> I could access the web interface without any problem.
>>>>>
>>>>> Then I set up VLANs (initally using VLAN numbers 1 and 2) to separate
>>>>> network traffic between the DomUs. I made the existing device bridge
>>>>> VLAN 1 (bridge 1) and created a secondary device for bridging VLAN 2
>>>>> (bridge 2). Using only bridge 1 / VLAN 1 everything works well, I can
>>>>> access the zabbix web interface without any noticeable issue. After
>>>>> switching the zabbix DomU to VLAN 2 / bridge 2, everything seemingly
>>>>> keeps on working well, I can ping different devices in my network from
>>>>> the zabbix DomU and vice versa, I can ssh into the machine.
>>>>>
>>>>> However, as soon as I remotely access the zabbix web interface, the
>>>>> complete system (DomUs and Dom0) becomes unresponsive and reboots
>>>>> after some time (usually seconds, sometimes 1-2 minutes). The reboot
>>>>> is reliably reproducable.
>>>>>
>>>>> I didn't see any error message in any log (zabbix, DomU syslog, Dom0
>>>>> syslog) except for the following lines immediately before the system
>>>>> reboots on the Xen serial console:
>>>>>
>>>>> (XEN) Watchdog timer fired for domain 0
>>>>> (XEN) Hardware Dom0 shutdown: watchdog rebooting machine
>>>>>
>>>>> As soon as I change the bridge to bridge 1 (with or without VLAN
>>>>> setup), the web interface is accessible again after booting the zabbix
>>>>> DomU, no reboots.
>>>>>
>>>>> So I assume that causing specific traffic on the virtual NIC when
>>>>> using a VLAN setup with more than one VLAN under Xen makes the Dom0
>>>>> system hard crash. Of course, there might be other causes that I'm not
>>>>> aware of, but to me, this seems to be the most likely explanation
>>>>> right now.
>>>>>
>>>>> What I tried:
>>>>> -------------
>>>>>
>>>>> 1. I changed the VLAN numbers. First to 101, 102, 103 etc. This was
>>>>> when I noticed another strange thing: VLANs with numbers >99 simply
>>>>> don't work on my Raspberry Pi under Debian, with or without Xen. VLAN
>>>>> 99 works, VLAN 100 (or everything else >99 that I tried) doesn't work.
>>>>> If I choose a number >99, the VLAN is not configured, "ip a" doesn't
>>>>> list it. Other Debian systems on x64 architecture don't show this
>>>>> behavior, there, it was no problem to set up VLANs > 99. Therefore,
>>>>> I've changed the VLANs to 10, 20, 30 etc., which worked. But it didn't
>>>>> solve the initial problem of the crashing Dom0 and DomUs.
>>>>>
>>>>> 2. Different bridge options, without noticable effect:
>>>>> bridge_stp off      # dont use STP (spanning tree proto)
>>>>> bridge_waitport 0   # dont wait for port to be available
>>>>> bridge_fd 0         # no forward delay
>>>>>
>>>>> 3. Removing IPv6: No noticable effect.
>>>>>
>>>>> 4. Network traffic analysis: Now, here it becomes _really_ strange. I
>>>>> started tcpdumps on Dom0, and depending on on which interface/bridge
>>>>> traffic was logged, the problem went away, meaning, the DomU was
>>>>> running smoothly for hours, even when accessing the zabbix web
>>>>> interface. Stopping the log makes the system crash (as above, after
>>>>> seconds up to 1-2 minutes) reproducably if I access the zabbix web
>>>>> interface.
>>>>>
>>>>> Logging enabcm6e4ei0 (NIC): no crashes
>>>>> Logging enabcm6e4ei0.10 (VLAN 10): instant crash
>>>>> Logging enabcm6e4ei0.20 (VLAN 20): no crashes
>>>>> Logging xenbr0 (on VLAN 10): instant crash
>>>>> Logging xenbr1 (on VLAN 20): no crashes
>>>>>
>>>>> I am clinging to the thought that there must be a rational explanation
>>>>> for why logging the traffic on certain interfaces/bridges should avoid
>>>>> the crash of the complete system, while logging other
>>>>> interfaces/bridges doesn't. I myself can't think of one.
>>>>>
>>>>> I checked the dumps of enabcm6e4ei0.10 and xenbr0 (where the system
>>>>> crashes) with wireshark, nothing sticks out to me (but I am really no
>>>>> expert in analyzing network traffic). Dumps can be provided.
>>>>>
>>>>> 5. Watchdog: I tried to dig deeper into the cause for the watchdog
>>>>> triggering. However, I didn't find any useful documentation on the web
>>>>> on how the watchdog works or how to enable logging.
>>>>>
>>>>> 6. Eliminating Xen as cause: I booted the Debian system (which in Xen
>>>>> setup would be Dom0) without Xen and set it up to use the VLAN 20
>>>>> bridge (the same that leads to a reboot when using it in the DomU) as
>>>>> primary network interface. Everything seemed to be working, I could
>>>>> download large files from the internet without any problem. Setting up
>>>>> Zabbix on the base Debian system showed that the same setup (VLANs 10
>>>>> and 20, bridges 1 and 2, using bridge 2 as interface for Zabbix)
>>>>> without Xen is working reliably, no reboots. This points to some Xen
>>>>> related component being the root cause, I think.
>>>>>
>>>>> 7. Eliminating Apache as root cause: Reloading the Apache starting
>>>>> page hosted on DomU several times per second didn't lead to a reboot.
>>>>>
>>>>> 8. Recompiling Xen: Independent of which Xen master branch version I
>>>>> was using (all 4.18), the behavior was the same. I didn't get Xen
>>>>> working on ARM64/UEFI in version 4.17.
>>>>>
>>>>> Current situation:(XEN) d3v0 Unhandled SMC/HVC: 0x84000050
>>> (XEN) d3v0 Unhandled SMC/HVC: 0x8600ff01
>>> (XEN) d3v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>> (XEN) common/grant_table.c:1909:d3v0 Expanding d3 grant table from 1 to
>>> 2 frames
>>> (XEN) common/grant_table.c:1909:d3v0 Expanding d3 grant table from 2 to
>>> 3 frames
>>> (XEN) common/grant_table.c:1909:d3v0 Expanding d3 grant table from 3 to
>>> 4 frames
>>>>> ------------------
>>>>>
>>>>> I am out of ideas what to do next. Everything that was recommended to
>>>>> me on xen-users didn't lead to significant insight or solve the problem.
>>>>>
>>>>> I'd appreciate any hints how to troubleshoot this and/or how to
>>>>> proceed otherwise.
>>>>
>>>> O.k., let's try to break that issue down.
>>>>
>>>> Firstly, how can I get more information on why the Xen watchdog
>>>> triggers? Is there documentation? Are there any logs? I couldn't find
>>>> anything useful with my search skills.
>>>>
>>>
>>> After some delay, I have picked up the Raspberry Pi again, built Xen
>>> 4.19-unstable, with the same result: Reboot of the complete system after
>>> the Dom0 watchdog triggering when accessing the Zabbix content on a
>>> webserver on DomU.
>>>
>>> I still would like to find out what's going wrong here, but I have no
>>> idea what to do. I'd appreciate any hint.
>>>
>>> Not knowing if it helps, I added Xen logs from boot until Dom0
>>> crash/reboot below.
>>>
>>> Loading Xen xen ...
>>> Loading Linux 6.1.0-17-arm64 ...
>>> Loading initial ramdisk ...
>>> Using modules provided by bootloader in FDT
>>> Xen 4.19-unstable (c/s Fri Jan 12 11:54:31 2024 +0000
>>> git:1ec3fe1f66-dirty) EFI
>>> loader
>>>    Xen 4.19-unstable
>>> (XEN) Xen version 4.19-unstable (root@lan.onlineschubla.de) (gcc (Debian
>>> 12.2.0-
>>>
>>>
>>>                              14) 12.2.0) debug=y Sun Jan 14 21:46:34 CET 2024
>>> (XEN) Latest ChangeSet: Fri Jan 12 11:54:31 2024 +0000 git:1ec3fe1f66-dirty
>>> (XEN) build-id: babb03cb6107fc46f7d8969142ccd6772a1133c3
>>> (XEN) Console output is synchronous.
>>> (XEN) Processor: 00000000410fd083: "ARM Limited", variant: 0x0, part
>>> 0xd08,rev 0
>>>
>>>
>>>                                  x3
>>> (XEN) 64-bit Execution:
>>> (XEN)   Processor Features: 0000000000002222 0000000000000000
>>> (XEN)     Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
>>> (XEN)     Extensions: FloatingPoint AdvancedSIMD
>>> (XEN)   Debug Features: 0000000010305106 0000000000000000
>>> (XEN)   Auxiliary Features: 0000000000000000 0000000000000000
>>> (XEN)   Memory Model Features: 0000000000001124 0000000000000000
>>> (XEN)   ISA Features:  0000000000010000 0000000000000000
>>> (XEN) 32-bit Execution:
>>> (XEN)   Processor Features: 0000000000000131:0000000000011011
>>> (XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
>>> (XEN)     Extensions: GenericTimer Security
>>> (XEN)   Debug Features: 0000000003010066
>>> (XEN)   Auxiliary Features: 0000000000000000
>>> (XEN)   Memory Model Features: 0000000010201105 0000000040000000
>>> (XEN)                          0000000001260000 0000000002102211
>>> (XEN)   ISA Features: 0000000002101110 0000000013112111 0000000021232042
>>> (XEN)                 0000000001112131 0000000000011142 0000000000010001
>>> (XEN) Using SMC Calling Convention v1.2
>>> (XEN) Using PSCI v1.1
>>> (XEN) ACPI: GICC (acpi_id[0x0000] address[0xff842000] MPIDR[0x0] enabled)
>>> (XEN) ACPI: GICC (acpi_id[0x0001] address[0xff842000] MPIDR[0x1] enabled)
>>> (XEN) ACPI: GICC (acpi_id[0x0002] address[0xff842000] MPIDR[0x2] enabled)
>>> (XEN) ACPI: GICC (acpi_id[0x0003] address[0xff842000] MPIDR[0x3] enabled)
>>> (XEN) 4 CPUs enabled, 4 CPUs total
>>> (XEN) SMP: Allowing 4 CPUs
>>> (XEN) enabled workaround for: ARM erratum 1319537
>>> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 54000 KHz
>>> (XEN) GICv2 initialization:
>>> (XEN)         gic_dist_addr=00000000ff841000
>>> (XEN)         gic_cpu_addr=00000000ff842000
>>> (XEN)         gic_hyp_addr=00000000ff844000
>>> (XEN)         gic_vcpu_addr=00000000ff846000
>>> (XEN)         gic_maintenance_irq=25
>>> (XEN) GICv2: 256 lines, 4 cpus, secure (IID 0200143b).
>>> (XEN) XSM Framework v1.0.1 initialized
>>> (XEN) Initialising XSM SILO mode
>>> (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
>>> (XEN) Initializing Credit2 scheduler
>>> (XEN)  load_precision_shift: 18
>>> (XEN)  load_window_shift: 30
>>> (XEN)  underload_balance_tolerance: 0
>>> (XEN)  overload_balance_tolerance: -3
>>> (XEN)  runqueues arrangement: socket
>>> (XEN)  cap enforcement granularity: 10ms
>>> (XEN) load tracking window length 1073741824 ns
>>> (XEN) Allocated console ring of 32 KiB.
>>> (XEN) CPU0: Guest atomics will try 16 times before pausing the domain
>>> (XEN) Bringing up CPU1
>>> (XEN) CPU1: Guest atomics will try 16 times before pausing the domain
>>> (XEN) CPU 1 booted.
>>> (XEN) Bringing up CPU2
>>> (XEN) CPU2: Guest atomics will try 13 times before pausing the domain
>>> (XEN) CPU 2 booted.
>>> (XEN) Bringing up CPU3
>>> (XEN) CPU3: Guest atomics will try 16 times before pausing the domain
>>> (XEN) Brought up 4 CPUs
>>> (XEN) CPU 3 booted.
>>> (XEN) I/O virtualisation disabled
>>> (XEN) P2M: 44-bit IPA with 44-bit PA and 8-bit VMID
>>> (XEN) P2M: 4 levels with order-0 root, VTCR 0x0000000080043594
>>> (XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
>>> (XEN) Initializing Credit2 scheduler
>>> (XEN)  load_precision_shift: 18
>>> (XEN)  load_window_shift: 30
>>> (XEN)  underload_balance_tolerance: 0
>>> (XEN)  overload_balance_tolerance: -3
>>> (XEN)  runqueues arrangement: socket
>>> (XEN)  cap enforcement granularity: 10ms
>>> (XEN) load tracking window length 1073741824 ns
>>> (XEN) Adding cpu 0 to runqueue 0
>>> (XEN)  First cpu on runqueue, activating
>>> (XEN) Adding cpu 1 to runqueue 0
>>> (XEN) Adding cpu 2 to runqueue 0
>>> (XEN) Adding cpu 3 to runqueue 0
>>> (XEN) alternatives: Patching with alt table 00000a00002ee0b0 ->
>>> 00000a00002ef250
>>> (XEN) CPU2 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>> (XEN) CPU1 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>> (XEN) CPU3 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>> (XEN) CPU0 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>> (XEN) *** LOADING DOMAIN 0 ***
>>> (XEN) Loading d0 kernel from boot module @ 0000000030ef7000
>>> (XEN) Loading ramdisk from boot module @ 000000002ee6d000
>>> (XEN) Allocating 1:1 mappings totalling 1024MB for dom0:
>>> (XEN) BANK[0] 0x00000040000000-0x00000080000000 (1024MB)
>>> (XEN) Grant table range: 0x0000002eceb000-0x0000002ed2b000
>>> (XEN) Allocating PPI 16 for event channel interrupt
>>> (XEN) Loading zImage from 0000000030ef7000 to
>>> 0000000040000000-0000000041f1c7c0
>>> (XEN) Loading d0 initrd from 000000002ee6d000 to
>>> 0x0000000048200000-0x000000004a
>>>
>>>
>>>                                                      288c22
>>> (XEN) Loading d0 DTB to 0x0000000048000000-0x00000000480002c3
>>> (XEN) Initial low memory virq threshold set at 0x4000 pages.
>>> (XEN) Scrubbing Free RAM in background
>>> (XEN) Std. Loglevel: All
>>> (XEN) Guest Loglevel: All
>>> (XEN) ***************************************************
>>> (XEN) WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
>>> (XEN) This option is intended to aid debugging of Xen by ensuring
>>> (XEN) that all output is synchronously delivered on the serial line.
>>> (XEN) However it can introduce SIGNIFICANT latencies and affect
>>> (XEN) timekeeping. It is NOT recommended for production use!
>>> (XEN) ***************************************************
>>> (XEN) 3... 2... 1...
>>> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
>>> (XEN) Freed 376kB init memory.
>>> (XEN) d0v0 Unhandled SMC/HVC: 0x84000050
>>> (XEN) d0v0 Unhandled SMC/HVC: 0x8600ff01
>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER4
>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER8
>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER12
>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER16
>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER20
>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER24
>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER28
>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>> (XEN) d0v1: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>> (XEN) d0v2: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>> (XEN) d0v3: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>> (XEN) d1v0 Unhandled SMC/HVC: 0x84000050
>>> (XEN) d1v0 Unhandled SMC/HVC: 0x8600ff01
>>> (XEN) d1v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>> (XEN) common/grant_table.c:1909:d1v0 Expanding d1 grant table from 1 to
>>> 2 frames
>>> (XEN) gnttab_mark_dirty not implemented yet
>>> (XEN) d2v0 Unhandled SMC/HVC: 0x84000050
>>> (XEN) d2v0 Unhandled SMC/HVC: 0x8600ff01
>>> (XEN) d2v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>> (XEN) common/grant_table.c:1909:d2v0 Expanding d2 grant table from 1 to
>>> 2 frames
>>> (XEN) common/grant_table.c:1909:d2v0 Expanding d2 grant table from 2 to
>>> 3 frames
>>> (XEN) common/grant_table.c:1909:d2v0 Expanding d2 grant table from 3 to
>>> 4 frames
>>> (XEN) Watchdog timer fired for domain 0
>>> (XEN) Hardware Dom0 shutdown: watchdog rebooting machine
>>
>> I'm unsure whose attention to draw to this report.
>>
>> This might be a scheduler issue since the watchdog timer is triggering.
>>
>> This might be an ACPI issue as ACPI is in use here.
>>
>> This might be an ARM Linux kernel issue.
>>
>> In the end this is someone running into trouble with Xen on an ARM
>> device.  Yet despite bringing up the issue hasn't gotten any help...
> 
> Hey Elliot,
> 
> Thanks for raising the visibility of this.  I'm not familiar with ARM,
> but if I were investigating this I'd try to figure out what the
> "unhandled" error messages are.  "gnttab_mark_dirty not implemented
> yet" looks pretty sus too, and also sounds like it might be something
> ARM-specific.
> 
> I don't see anything suspicious WRT the scheduler, but a simple way to
> test that would be to set the scheduler to credit1 and see if that
> changes things.

Just to rule out all possibilities, I did a test with credit(1) 
scheduler, which didn't change anything noticable. The watchdog 
triggered after traffic has been created on the VLAN.

Best regards,

Paul


From xen-devel-bounces@lists.xenproject.org Sat Feb 17 21:51:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 17 Feb 2024 21:51:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682506.1061607 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbSaX-0007ft-5Z; Sat, 17 Feb 2024 21:51:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682506.1061607; Sat, 17 Feb 2024 21:51:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbSaX-0007fm-2P; Sat, 17 Feb 2024 21:51:13 +0000
Received: by outflank-mailman (input) for mailman id 682506;
 Sat, 17 Feb 2024 21:51:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbSaW-0007fZ-21; Sat, 17 Feb 2024 21:51:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbSaV-0000X0-P1; Sat, 17 Feb 2024 21:51:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbSaV-0003kY-AX; Sat, 17 Feb 2024 21:51:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rbSaV-0004q6-A4; Sat, 17 Feb 2024 21:51:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=eYnntM9ym6yEOHCZ5Gfz6RIo1BOoh9z6vcYcEqSTRvE=; b=cHwD7WwFPJu3gO32bIoVXu3pX0
	eSOAJLziY02SNq09seHyD7iVZsK8CDqqU0fSe8vO0siRLreZi0TV342b6nmiBjM+WNyfH/BW2NivM
	YZw4cBd1SX/dYZziDr5L3m7kPLgw3ur04KEWWZSqGcjfzuKLUvYifNqIOgA98Lc7CQm0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184691-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184691: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl-rtds:xen-boot:fail:allowable
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=c1ca10ceffbb289ed02feaf005bc9ee6095b4507
X-Osstest-Versions-That:
    linux=0f1dd5e91e2ba3990143645faff2bcce2d99778e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 17 Feb 2024 21:51:11 +0000

flight 184691 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184691/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds      8 xen-boot                 fail REGR. vs. 184686

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184686
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184686
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184686
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184686
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184686
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184686
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184686
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184686
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                c1ca10ceffbb289ed02feaf005bc9ee6095b4507
baseline version:
 linux                0f1dd5e91e2ba3990143645faff2bcce2d99778e

Last test of basis   184686  2024-02-16 18:12:10 Z    1 days
Testing same since   184691  2024-02-17 06:55:54 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Jones <ajones@ventanamicro.com>
  Andy Shevchenko <andriy.shevchenko@linux.intel.com>
  Anup Patel <anup@brainfault.org>
  Chaitanya Kulkarni <kch@nvidia.com>
  Claudio Imbrenda <imbrenda@linux.ibm.com>
  Damien Le Moal <dlemoal@kernel.org>
  Douglas Anderson <dianders@chromium.org>
  Easwar Hariharan <eahariha@linux.microsoft.com>
  Fangrui Song <maskray@google.com>
  Hannes Reinecke <hare@suse.de>
  Hojin Nam <hj96.nam@samsung.com>
  Ilkka Koskinen <ilkka@os.amperecomputing.com>
  Ilya Dryomov <idryomov@gmail.com>
  Janosch Frank <frankja@linux.ibm.com>
  Jens Axboe <axboe@kernel.dk>
  Kalle Valo <kvalo@kernel.org>
  Keith Busch <kbusch@kernel.org>
  Lee Duncan <lduncan@suse.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Marc Zyngier <maz@kernel.org>
  Mark Brown <broonie@kernel.org>
  Mark Rutland <mark.rutland@arm.com>
  Martin K. Petersen <martin.petersen@oracle.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Mathias Krause <minipli@grsecurity.net>
  Mete Durlu <meted@linux.ibm.com>
  Mike Christie <michael.christie@oracle.com>
  Mingwei Zhang <mizhang@google.com>
  Paolo Bonzini <pbonzini@redhat.com>
  Petr Pavlu <petr.pavlu@suse.com>
  Prasad Pandit <pjp@fedoraproject.org>
  Robin Murphy <robin.murphy@arm.com>
  Sean Christopherson <seanjc@google.com>
  Sebastian Ene <sebastianene@google.com>
  Seongsu Park <sgsu.park@samsung.com>
  Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Sven Schnelle <svens@linux.ibm.com>
  Tejun Heo <tj@kernel.org>
  Thorsten Blum <thorsten.blum@toblux.com>
  Vitaly Kuznetsov <vkuznets@redhat.com>
  Will Deacon <will@kernel.org>
  Xiubo Li <xiubli@redhat.com>
  Zenghui Yu <yuzenghui@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   0f1dd5e91e2b..c1ca10ceffbb  c1ca10ceffbb289ed02feaf005bc9ee6095b4507 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun Feb 18 01:35:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 01:35:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682519.1061619 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbW50-000509-7W; Sun, 18 Feb 2024 01:34:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682519.1061619; Sun, 18 Feb 2024 01:34:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbW50-000502-2D; Sun, 18 Feb 2024 01:34:54 +0000
Received: by outflank-mailman (input) for mailman id 682519;
 Sun, 18 Feb 2024 01:34:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J8zP=J3=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1rbW4y-0004zw-FS
 for xen-devel@lists.xenproject.org; Sun, 18 Feb 2024 01:34:52 +0000
Received: from wfout3-smtp.messagingengine.com
 (wfout3-smtp.messagingengine.com [64.147.123.146])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e6da7471-cdfd-11ee-8a52-1f161083a0e0;
 Sun, 18 Feb 2024 02:34:49 +0100 (CET)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfout.west.internal (Postfix) with ESMTP id E42671C00079;
 Sat, 17 Feb 2024 20:34:45 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Sat, 17 Feb 2024 20:34:46 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat,
 17 Feb 2024 20:34:43 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e6da7471-cdfd-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:message-id:mime-version:reply-to:subject:subject:to:to; s=fm3;
	 t=1708220085; x=1708306485; bh=AU9JacD3M6P4ufXSFjP46LL0rxd7nWVR
	0IwQtMoYP8E=; b=b4lP9LvSM4NAibaLeF+jttCMRv15OqJD6XFnFaDb6wxX/ioj
	pNLLBrs26Ela0fPx8+Urf3BborVGIndJ5TilLOAK9uXMEnrjmjxbK3BItXugqyTE
	ZzaxPDx/C1hh6qbnjRFHpu2x6cIVMsZai8/6dcxRk46YxU/Ve3fVHCN3dw1BKURZ
	5mJUc8eRmIEeYIAoO1mPeE24vA9OuQKszTElD/PnI+bEFkJ9cGNqUVzUwvmiTUVn
	dzDJacsnFI5PXpXkEASHY010KAOja7SvmAT3qxaP7WPpQ8umflTNN6rv3VWo0anM
	YEsM2TOyQrFtp3XFyjNi2VhjWF1knUH+dUt8ng==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:message-id:mime-version:reply-to:subject
	:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
	:x-sasl-enc; s=fm1; t=1708220085; x=1708306485; bh=AU9JacD3M6P4u
	fXSFjP46LL0rxd7nWVR0IwQtMoYP8E=; b=csIL74bWjCqynx+cDDqtjAcwSmCDs
	Ju8UKfLJ5+BV+WJcC5+djITjIQw3oWDcR5tJNVxcxlLr+sWYPr2MYUqFDrVyrVs4
	T6EV5/Jk2+5du4flLuNAvE5NTe841w6VVV5aZ8lUV742idr/WPjx9dvklzGEGeZw
	vQHQvlsYTV3Bw7H+dsway1uzSndFX9InXs/M/SmFB+3yK42/34DfRJPRw93g7HXQ
	jQYDtL1gLP+xYPP6B4v0PC9HFPMKHYa79jhc1XE5Ukcwp6LNXsSW+K85b6Dij12H
	/LOHf8QNP9sS6ekCQ9teQ0R2XkllHpra2LFRBLu68KyvPgW7JpthqqLJg==
X-ME-Sender: <xms:tF7RZVAtqzZcxJ2-Y9Btrr8fChkAg5Dp5L0DvOzElBrBMaEAUslFYQ>
    <xme:tF7RZThHTn-6dCa2iBfoJ3RZqVAEFLoFKZjV48Pm-aIa28aUVYKYILiCUGlI_uk5P
    t4jsPo4Q4kd4w>
X-ME-Received: <xmr:tF7RZQn7ULd36Pg99sNbcgdG4n6pM8kTg8Lk2Ug2nJ9WBHY8SLrxSTuD4TY45iE8YZ75KdSkXos8Z6sIPYfNyDE_4WosVCApVSfpf9n9eyjyeRnqIo0>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdehgdefiecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpefhvfevufffkffogggtgfesthekredtredtjeenucfhrhhomhepofgrrhgvkhcu
    ofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinhhvih
    hsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepleekhfdu
    leetleelleetteevfeefteffkeetteejheelgfegkeelgeehhfdthedvnecuvehluhhsth
    gvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghksehi
    nhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:tF7RZfxkNE7UDBaC8KkaFqHlx4O2SPVudqrrO8c8bEqbzvspWiUNfw>
    <xmx:tF7RZaQrnTta1CQcab4-V8vQGENOSbuMMGA_3ayhz9ZOL-09NN3vTA>
    <xmx:tF7RZSaJN4txQ1RTfOmU7pG2k_e5diXro5VFfRgJjrehjpNafI-HhQ>
    <xmx:tV7RZXQPOCcMy73Q4cytZO7RrPRbdUedlap34Do9UjmuXHp_JwW7skjJrvs>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] ns16550: add Asix AX99100 serial card
Date: Sun, 18 Feb 2024 02:34:29 +0100
Message-ID: <20240218013431.274451-1-marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It's 4-port serial card, each port is presented as a separate function.
According to the specification, it features 256-byte TX FIFO buffer.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
It's a card plugged into a box that can function as yet another gitlab runner.
---
 xen/drivers/char/ns16550.c | 12 ++++++++++++
 xen/include/xen/pci_ids.h  |  2 ++
 2 files changed, 14 insertions(+)

diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c
index 97bf0985344a..00c0da3f373c 100644
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -92,6 +92,7 @@ struct ns16550_config {
         param_exar_xr17v354,
         param_exar_xr17v358,
         param_intel_lpss,
+        param_asix_ax99100,
     } param;
 };
 
@@ -842,6 +843,12 @@ static const struct ns16550_config_param __initconst uart_param[] = {
         .mmio = 1,
         .max_ports = 1,
     },
+    [param_asix_ax99100] = {
+        .reg_width = 1,
+        .lsr_mask = UART_LSR_THRE,
+        .max_ports = 1,
+        .fifo_size = 256,
+    },
 };
 
 static const struct ns16550_config __initconst uart_config[] =
@@ -1170,6 +1177,11 @@ static const struct ns16550_config __initconst uart_config[] =
         .dev_id = 0x7adc,
         .param = param_intel_lpss
     },
+    {
+        .vendor_id = PCI_VENDOR_ID_ASIX,
+        .dev_id = 9100,
+        .param = param_asix_ax99100
+    },
 };
 
 static int __init
diff --git a/xen/include/xen/pci_ids.h b/xen/include/xen/pci_ids.h
index e798477a7e23..2a19f4ab7872 100644
--- a/xen/include/xen/pci_ids.h
+++ b/xen/include/xen/pci_ids.h
@@ -11,3 +11,5 @@
 #define PCI_VENDOR_ID_BROADCOM           0x14e4
 
 #define PCI_VENDOR_ID_INTEL              0x8086
+
+#define PCI_VENDOR_ID_ASIX               0x125b
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Sun Feb 18 03:49:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 03:49:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682526.1061628 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbYBN-0002JT-N2; Sun, 18 Feb 2024 03:49:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682526.1061628; Sun, 18 Feb 2024 03:49:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbYBN-0002JM-JC; Sun, 18 Feb 2024 03:49:37 +0000
Received: by outflank-mailman (input) for mailman id 682526;
 Sun, 18 Feb 2024 03:49:36 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbYBM-0002J9-H1; Sun, 18 Feb 2024 03:49:36 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbYBM-0006PI-6X; Sun, 18 Feb 2024 03:49:36 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbYBL-00086N-Oj; Sun, 18 Feb 2024 03:49:35 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rbYBL-0002aW-Nq; Sun, 18 Feb 2024 03:49:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=0KOqAVCKypJcyolcTMzLh+Fy8y7EeuPxdR4D6vFPgcA=; b=jAEycS9VtpMph2wCuhXzZXXPZN
	HmhesXsUCOFkwJCDEcLCaaSY9dHQPX5oa22NnGWM53BNGo9YlMZZc4VpYNfEbzv77hQC0EbmMhjYy
	mAvmrBQ+1Mz/QODM1qys/dI6g1oDsRVEuOjQJCcGbEagAXHo1bmq3CgjAoUjSsMRiq0o=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184693-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184693: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-vhd:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt-raw:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
X-Osstest-Versions-That:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 18 Feb 2024 03:49:35 +0000

flight 184693 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184693/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install fail in 184688 pass in 184693
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184688
 test-armhf-armhf-xl-vhd      17 guest-start/debian.repeat  fail pass in 184688
 test-armhf-armhf-libvirt-raw 17 guest-start/debian.repeat  fail pass in 184688

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop      fail blocked in 184688
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184688
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184688
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184688
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184688
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184688
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184688
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184688
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184688
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184688
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184688
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184688
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3
baseline version:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3

Last test of basis   184693  2024-02-17 16:57:37 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 fail    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      fail    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun Feb 18 07:47:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 07:47:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682542.1061638 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbbtI-00029X-NY; Sun, 18 Feb 2024 07:47:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682542.1061638; Sun, 18 Feb 2024 07:47:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbbtI-00029Q-Ir; Sun, 18 Feb 2024 07:47:12 +0000
Received: by outflank-mailman (input) for mailman id 682542;
 Sun, 18 Feb 2024 07:47:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbbtH-00029G-P9; Sun, 18 Feb 2024 07:47:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbbtH-00038W-GA; Sun, 18 Feb 2024 07:47:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbbtH-0001xN-1P; Sun, 18 Feb 2024 07:47:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rbbtH-0008BR-0p; Sun, 18 Feb 2024 07:47:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=oANuTKyHPN3MmnkKRo7zbR50i4FGiGoBZwc0YNvjDQg=; b=UcS1WuPSEr5nR7mM1oiZU+BtR6
	nOzZFl/g2I0AE9RxmNL6FEKkei6E4c5n6PJHOfaTf6j+CPHgJ8sKCg60scRnPfDniMpbK9ZeK6x+x
	U811pfYVocQn4NJQ+fOK4cdpHrWbo6cje8erfeQESVoLzYLH1f22/RDfox2d76LS94n4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184694-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184694: trouble: broken/fail/pass
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt-qcow2:<job status>:broken:regression
    linux-linus:test-armhf-armhf-libvirt-qcow2:host-install(5):broken:regression
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=f2667e0c32404a68496891b2d2015825de189b06
X-Osstest-Versions-That:
    linux=c1ca10ceffbb289ed02feaf005bc9ee6095b4507
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 18 Feb 2024 07:47:11 +0000

flight 184694 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184694/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt-qcow2    <job status>                 broken
 test-armhf-armhf-libvirt-qcow2  5 host-install(5)      broken REGR. vs. 184691

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184691
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184691
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184691
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184691
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184691
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184691
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184691
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                f2667e0c32404a68496891b2d2015825de189b06
baseline version:
 linux                c1ca10ceffbb289ed02feaf005bc9ee6095b4507

Last test of basis   184691  2024-02-17 06:55:54 Z    1 days
Testing same since   184694  2024-02-17 22:14:07 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Jiri Slaby (SUSE)" <jirislaby@kernel.org>
  Abel Vesa <abel.vesa@linaro.org>
  Alex Williamson <alex.williamson@redhat.com>
  Arnd Bergmann <arnd@arndb.de>
  Bjorn Helgaas <bhelgaas@google.com>
  Cosmin Tanislav <demonsingur@gmail.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  David Schiller <david.schiller@jku.at>
  Dietmar Eggemann <dietmar.eggemann@arm.com>
  Dimitri Fedrau <dima.fedrau@gmail.com>
  Dinghao Liu <dinghao.liu@zju.edu.cn>
  Emil Kronborg <emil.kronborg@protonmail.com>
  Georgi Djakov <djakov@kernel.org>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Hans de Goede <hdegoede@redhat.com>
  Javier Carrasco <javier.carrasco.cruz@gmail.com>
  Jiri Slaby (SUSE) <jirislaby@kernel.org>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Konrad Dybcio <konrad.dybcio@linaro.org>
  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
  Lee Jones <lee@kernel.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Mario Limonciello <mario.limonciello@amd.com>
  Mark Brown <broonie@kernel.org>
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Matthew Ruffell <matthew.ruffell@canonical.com>
  Mauro Carvalho Chehab <mchehab@kernel.org>
  Mika Westerberg <mika.westerberg@linux.intel.com>
  Mike Tipton <quic_mdtipton@quicinc.com>
  Mohammad Rahimi <rahimi.mhmmd@gmail.com>
  Nuno Sa <nuno.sa@analog.com>
  Paul Barker <paul.barker.ct@bp.renesas.com>
  Paulo Alcantara (Red Hat) <pc@manguebit.com>
  Paulo Alcantara <pc@manguebit.com>
  Pierre Gondois <pierre.gondois@arm.com>
  Randy Dunlap <rdunlap@infradead.org>
  Ronnie Sahlberg <ronniesahlberg@gmail.com>
  Sam Protsenko <semen.protsenko@linaro.org>
  Sanath S <sanath.s@amd.com>
  Saravana Kannan <saravanak@google.com>
  Sasha Levin <sashal@kernel.org>
  Sean Young <sean@mess.org>
  Shyam Prasad N <sprasad@microsoft.com>
  Siddharth Vadapalli <s-vadapalli@ti.com>
  Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Stefan Wahren <wahrenst@gmx.net>
  Steve French <stfrench@microsoft.com>
  Su Yue <glass.su@suse.com>
  Sudeep Holla <sudeep.holla@arm.com>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
  Vincent Guittot <vincent.guittot@linaro.org>
  Xu Yang <xu.yang_2@nxp.com>
  zhili.liu <zhili.liu@ucas.com.cn>
  Zhipeng Lu <alexious@zju.edu.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               broken  
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-armhf-armhf-libvirt-qcow2 broken
broken-step test-armhf-armhf-libvirt-qcow2 host-install(5)

Not pushing.

(No revision log; it would be 1482 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Feb 18 14:24:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 14:24:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682586.1061648 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbi5h-0001vp-Lo; Sun, 18 Feb 2024 14:24:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682586.1061648; Sun, 18 Feb 2024 14:24:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbi5h-0001vW-GK; Sun, 18 Feb 2024 14:24:25 +0000
Received: by outflank-mailman (input) for mailman id 682586;
 Sun, 18 Feb 2024 14:24:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbi5g-0001vM-2U; Sun, 18 Feb 2024 14:24:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbi5f-00030K-Qu; Sun, 18 Feb 2024 14:24:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbi5f-0002Fo-Cg; Sun, 18 Feb 2024 14:24:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rbi5f-0002Fo-CF; Sun, 18 Feb 2024 14:24:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LRQQofZl+ZY6IBW9LBkjM9/F8ZJYJPF7+SUX0U+2ZTM=; b=VhgjKE/YXrjM/7v4MB1O5VQ48Q
	r5YaK4x665ZBCIPb6XRgAY+JBAchEEkN/yH0Z+DrFnqjJVYZpIExhs1wwm4eKLxrruLGp0+2NWSzN
	z8NQD7STV57IP7G/gjV3OriiZ7dyeaKcHrf+rsTKvFRLQ26m3V9lbM6ydSsveb5o2y8o=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184695-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184695: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt-raw:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-vhd:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt-raw:guest-start:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
X-Osstest-Versions-That:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 18 Feb 2024 14:24:23 +0000

flight 184695 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184695/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-raw 17 guest-start/debian.repeat fail in 184693 pass in 184688
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail in 184693 pass in 184695
 test-armhf-armhf-xl-vhd 17 guest-start/debian.repeat fail in 184693 pass in 184695
 test-armhf-armhf-libvirt-raw 13 guest-start                fail pass in 184693

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop fail in 184693 blocked in 184695
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check fail in 184693 like 184688
 test-armhf-armhf-libvirt-raw 14 migrate-support-check fail in 184693 never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install       fail like 184688
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184693
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184693
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184693
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184693
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184693
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184693
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184693
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184693
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184693
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184693
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3
baseline version:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3

Last test of basis   184695  2024-02-18 03:53:36 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 fail    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun Feb 18 17:49:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 17:49:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682598.1061658 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rblI0-0005tg-TR; Sun, 18 Feb 2024 17:49:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682598.1061658; Sun, 18 Feb 2024 17:49:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rblI0-0005tZ-Q3; Sun, 18 Feb 2024 17:49:20 +0000
Received: by outflank-mailman (input) for mailman id 682598;
 Sun, 18 Feb 2024 17:49:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rblHz-0005tT-6E
 for xen-devel@lists.xenproject.org; Sun, 18 Feb 2024 17:49:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rblHy-0006vq-Lu; Sun, 18 Feb 2024 17:49:18 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rblHy-00031V-9y; Sun, 18 Feb 2024 17:49:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=vYkb0NRyLqnTTeGgk4GJ6q5+BOnD+VscqJ5i88p1zGo=; b=nJpbkHTjIEkK+mi5RqG7rbpssP
	EDxpHfoIUP008hcHjtNE5DBK+lY/sfNpzM5DtnLVycxd/1RWKjbnhWIiPLqVWL0d/aiJn+AyIep/p
	afd7Wx/17xrBNP3of5YaUzliD4p5BKMImVLMspInR3Yd8P65HXOw7GzwqJPscqhulYqQ=;
Message-ID: <a5ad8772-01eb-4a84-93e2-c96c3b72fa48@xen.org>
Date: Sun, 18 Feb 2024 17:49:16 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Xen 4.18rc/ARM64 on Raspberry Pi 4B: Traffic in DomU crashing
 Dom0 when using VLANs
To: Paul Leiber <paul@onlineschubla.de>,
 Xen-devel <xen-devel@lists.xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, george.dunlap@cloud.com,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <Bertrand.Marquis@arm.com>
References: <838ff75f-1352-4d3c-9bc1-c7a57c828231@onlineschubla.de>
 <4cddfd89-f195-4a50-a14d-b86121414e56@onlineschubla.de>
 <f8f39239-ea95-4fc7-9abe-6ca005eb02d2@onlineschubla.de>
 <ZarcWQ6Ugwowy050@mattapan.m5p.com>
 <CA+zSX=Y=BsVEp3o6jRprn5sntWVA0Z6wNXUxtffDN+=fPv_Fzg@mail.gmail.com>
 <4d7dbce9-ef20-4702-8056-9d5ab8152721@onlineschubla.de>
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
In-Reply-To: <4d7dbce9-ef20-4702-8056-9d5ab8152721@onlineschubla.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 17/02/2024 21:46, Paul Leiber wrote:
> Am 22.01.2024 um 11:46 schrieb George Dunlap:
>> On Fri, Jan 19, 2024 at 8:32 PM Elliott Mitchell <ehem+xen@m5p.com> 
>> wrote:
>>>
>>> On Sun, Jan 14, 2024 at 10:54:24PM +0100, Paul Leiber wrote:
>>>>
>>>> Am 22.10.2023 um 07:42 schrieb Paul Leiber:
>>>>> Am 13.10.2023 um 20:56 schrieb Paul Leiber:
>>>>>> Hi Xen developers list,
>>>>>>
>>>>>> TL;DR:
>>>>>> ------
>>>>>>
>>>>>> Causing certain web server traffic on a secondary VLAN on 
>>>>>> Raspberry Pi
>>>>>> under vanilla Debian/UEFI in combination with Xen leads to complete
>>>>>> system reboot (watchdog triggering for Dom0). Other strange things 
>>>>>> are
>>>>>> happening.
>>>>>>
>>>>>> Description:
>>>>>> ----------
>>>>>>
>>>>>> I recently set up Xen (self compiled, Version 4.18-rc) on a Raspberry
>>>>>> Pi 4B (on vanilla Debian Bookworm, UEFI boot mode). Until some time
>>>>>> ago, everything worked well with Dom0, one DomU and one bridge.
>>>>>>
>>>>>> Then I wanted to actually make use of the virtualization and started
>>>>>> to set up a second Debian Bookworm DomU (using xen-create-image) for
>>>>>> monitoring my systems with zabbix (a webserver based system 
>>>>>> monitoring
>>>>>> solution). The bridge used for this setup was the device bridging the
>>>>>> hardware NIC. I installed zabbix, set it up, and everything went 
>>>>>> well,
>>>>>> I could access the web interface without any problem.
>>>>>>
>>>>>> Then I set up VLANs (initally using VLAN numbers 1 and 2) to separate
>>>>>> network traffic between the DomUs. I made the existing device bridge
>>>>>> VLAN 1 (bridge 1) and created a secondary device for bridging VLAN 2
>>>>>> (bridge 2). Using only bridge 1 / VLAN 1 everything works well, I can
>>>>>> access the zabbix web interface without any noticeable issue. After
>>>>>> switching the zabbix DomU to VLAN 2 / bridge 2, everything seemingly
>>>>>> keeps on working well, I can ping different devices in my network 
>>>>>> from
>>>>>> the zabbix DomU and vice versa, I can ssh into the machine.
>>>>>>
>>>>>> However, as soon as I remotely access the zabbix web interface, the
>>>>>> complete system (DomUs and Dom0) becomes unresponsive and reboots
>>>>>> after some time (usually seconds, sometimes 1-2 minutes). The reboot
>>>>>> is reliably reproducable.
>>>>>>
>>>>>> I didn't see any error message in any log (zabbix, DomU syslog, Dom0
>>>>>> syslog) except for the following lines immediately before the system
>>>>>> reboots on the Xen serial console:
>>>>>>
>>>>>> (XEN) Watchdog timer fired for domain 0
>>>>>> (XEN) Hardware Dom0 shutdown: watchdog rebooting machine
>>>>>>
>>>>>> As soon as I change the bridge to bridge 1 (with or without VLAN
>>>>>> setup), the web interface is accessible again after booting the 
>>>>>> zabbix
>>>>>> DomU, no reboots.
>>>>>>
>>>>>> So I assume that causing specific traffic on the virtual NIC when
>>>>>> using a VLAN setup with more than one VLAN under Xen makes the Dom0
>>>>>> system hard crash. Of course, there might be other causes that I'm 
>>>>>> not
>>>>>> aware of, but to me, this seems to be the most likely explanation
>>>>>> right now.
>>>>>>
>>>>>> What I tried:
>>>>>> -------------
>>>>>>
>>>>>> 1. I changed the VLAN numbers. First to 101, 102, 103 etc. This was
>>>>>> when I noticed another strange thing: VLANs with numbers >99 simply
>>>>>> don't work on my Raspberry Pi under Debian, with or without Xen. VLAN
>>>>>> 99 works, VLAN 100 (or everything else >99 that I tried) doesn't 
>>>>>> work.
>>>>>> If I choose a number >99, the VLAN is not configured, "ip a" doesn't
>>>>>> list it. Other Debian systems on x64 architecture don't show this
>>>>>> behavior, there, it was no problem to set up VLANs > 99. Therefore,
>>>>>> I've changed the VLANs to 10, 20, 30 etc., which worked. But it 
>>>>>> didn't
>>>>>> solve the initial problem of the crashing Dom0 and DomUs.
>>>>>>
>>>>>> 2. Different bridge options, without noticable effect:
>>>>>> bridge_stp off      # dont use STP (spanning tree proto)
>>>>>> bridge_waitport 0   # dont wait for port to be available
>>>>>> bridge_fd 0         # no forward delay
>>>>>>
>>>>>> 3. Removing IPv6: No noticable effect.
>>>>>>
>>>>>> 4. Network traffic analysis: Now, here it becomes _really_ strange. I
>>>>>> started tcpdumps on Dom0, and depending on on which interface/bridge
>>>>>> traffic was logged, the problem went away, meaning, the DomU was
>>>>>> running smoothly for hours, even when accessing the zabbix web
>>>>>> interface. Stopping the log makes the system crash (as above, after
>>>>>> seconds up to 1-2 minutes) reproducably if I access the zabbix web
>>>>>> interface.
>>>>>>
>>>>>> Logging enabcm6e4ei0 (NIC): no crashes
>>>>>> Logging enabcm6e4ei0.10 (VLAN 10): instant crash
>>>>>> Logging enabcm6e4ei0.20 (VLAN 20): no crashes
>>>>>> Logging xenbr0 (on VLAN 10): instant crash
>>>>>> Logging xenbr1 (on VLAN 20): no crashes
>>>>>>
>>>>>> I am clinging to the thought that there must be a rational 
>>>>>> explanation
>>>>>> for why logging the traffic on certain interfaces/bridges should 
>>>>>> avoid
>>>>>> the crash of the complete system, while logging other
>>>>>> interfaces/bridges doesn't. I myself can't think of one.
>>>>>>
>>>>>> I checked the dumps of enabcm6e4ei0.10 and xenbr0 (where the system
>>>>>> crashes) with wireshark, nothing sticks out to me (but I am really no
>>>>>> expert in analyzing network traffic). Dumps can be provided.
>>>>>>
>>>>>> 5. Watchdog: I tried to dig deeper into the cause for the watchdog
>>>>>> triggering. However, I didn't find any useful documentation on the 
>>>>>> web
>>>>>> on how the watchdog works or how to enable logging.
>>>>>>
>>>>>> 6. Eliminating Xen as cause: I booted the Debian system (which in Xen
>>>>>> setup would be Dom0) without Xen and set it up to use the VLAN 20
>>>>>> bridge (the same that leads to a reboot when using it in the DomU) as
>>>>>> primary network interface. Everything seemed to be working, I could
>>>>>> download large files from the internet without any problem. 
>>>>>> Setting up
>>>>>> Zabbix on the base Debian system showed that the same setup (VLANs 10
>>>>>> and 20, bridges 1 and 2, using bridge 2 as interface for Zabbix)
>>>>>> without Xen is working reliably, no reboots. This points to some Xen
>>>>>> related component being the root cause, I think.
>>>>>>
>>>>>> 7. Eliminating Apache as root cause: Reloading the Apache starting
>>>>>> page hosted on DomU several times per second didn't lead to a reboot.
>>>>>>
>>>>>> 8. Recompiling Xen: Independent of which Xen master branch version I
>>>>>> was using (all 4.18), the behavior was the same. I didn't get Xen
>>>>>> working on ARM64/UEFI in version 4.17.
>>>>>>
>>>>>> Current situation:(XEN) d3v0 Unhandled SMC/HVC: 0x84000050
>>>> (XEN) d3v0 Unhandled SMC/HVC: 0x8600ff01
>>>> (XEN) d3v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>>> (XEN) common/grant_table.c:1909:d3v0 Expanding d3 grant table from 1 to
>>>> 2 frames
>>>> (XEN) common/grant_table.c:1909:d3v0 Expanding d3 grant table from 2 to
>>>> 3 frames
>>>> (XEN) common/grant_table.c:1909:d3v0 Expanding d3 grant table from 3 to
>>>> 4 frames
>>>>>> ------------------
>>>>>>
>>>>>> I am out of ideas what to do next. Everything that was recommended to
>>>>>> me on xen-users didn't lead to significant insight or solve the 
>>>>>> problem.
>>>>>>
>>>>>> I'd appreciate any hints how to troubleshoot this and/or how to
>>>>>> proceed otherwise.
>>>>>
>>>>> O.k., let's try to break that issue down.
>>>>>
>>>>> Firstly, how can I get more information on why the Xen watchdog
>>>>> triggers? Is there documentation? Are there any logs? I couldn't find
>>>>> anything useful with my search skills.
>>>>>
>>>>
>>>> After some delay, I have picked up the Raspberry Pi again, built Xen
>>>> 4.19-unstable, with the same result: Reboot of the complete system 
>>>> after
>>>> the Dom0 watchdog triggering when accessing the Zabbix content on a
>>>> webserver on DomU.
>>>>
>>>> I still would like to find out what's going wrong here, but I have no
>>>> idea what to do. I'd appreciate any hint.
>>>>
>>>> Not knowing if it helps, I added Xen logs from boot until Dom0
>>>> crash/reboot below.
>>>>
>>>> Loading Xen xen ...
>>>> Loading Linux 6.1.0-17-arm64 ...
>>>> Loading initial ramdisk ...
>>>> Using modules provided by bootloader in FDT
>>>> Xen 4.19-unstable (c/s Fri Jan 12 11:54:31 2024 +0000
>>>> git:1ec3fe1f66-dirty) EFI
>>>> loader
>>>>    Xen 4.19-unstable
>>>> (XEN) Xen version 4.19-unstable (root@lan.onlineschubla.de) (gcc 
>>>> (Debian
>>>> 12.2.0-
>>>>
>>>>
>>>>                              14) 12.2.0) debug=y Sun Jan 14 21:46:34 
>>>> CET 2024
>>>> (XEN) Latest ChangeSet: Fri Jan 12 11:54:31 2024 +0000 
>>>> git:1ec3fe1f66-dirty
>>>> (XEN) build-id: babb03cb6107fc46f7d8969142ccd6772a1133c3
>>>> (XEN) Console output is synchronous.
>>>> (XEN) Processor: 00000000410fd083: "ARM Limited", variant: 0x0, part
>>>> 0xd08,rev 0
>>>>
>>>>
>>>>                                  x3
>>>> (XEN) 64-bit Execution:
>>>> (XEN)   Processor Features: 0000000000002222 0000000000000000
>>>> (XEN)     Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
>>>> (XEN)     Extensions: FloatingPoint AdvancedSIMD
>>>> (XEN)   Debug Features: 0000000010305106 0000000000000000
>>>> (XEN)   Auxiliary Features: 0000000000000000 0000000000000000
>>>> (XEN)   Memory Model Features: 0000000000001124 0000000000000000
>>>> (XEN)   ISA Features:  0000000000010000 0000000000000000
>>>> (XEN) 32-bit Execution:
>>>> (XEN)   Processor Features: 0000000000000131:0000000000011011
>>>> (XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
>>>> (XEN)     Extensions: GenericTimer Security
>>>> (XEN)   Debug Features: 0000000003010066
>>>> (XEN)   Auxiliary Features: 0000000000000000
>>>> (XEN)   Memory Model Features: 0000000010201105 0000000040000000
>>>> (XEN)                          0000000001260000 0000000002102211
>>>> (XEN)   ISA Features: 0000000002101110 0000000013112111 
>>>> 0000000021232042
>>>> (XEN)                 0000000001112131 0000000000011142 
>>>> 0000000000010001
>>>> (XEN) Using SMC Calling Convention v1.2
>>>> (XEN) Using PSCI v1.1
>>>> (XEN) ACPI: GICC (acpi_id[0x0000] address[0xff842000] MPIDR[0x0] 
>>>> enabled)
>>>> (XEN) ACPI: GICC (acpi_id[0x0001] address[0xff842000] MPIDR[0x1] 
>>>> enabled)
>>>> (XEN) ACPI: GICC (acpi_id[0x0002] address[0xff842000] MPIDR[0x2] 
>>>> enabled)
>>>> (XEN) ACPI: GICC (acpi_id[0x0003] address[0xff842000] MPIDR[0x3] 
>>>> enabled)
>>>> (XEN) 4 CPUs enabled, 4 CPUs total
>>>> (XEN) SMP: Allowing 4 CPUs
>>>> (XEN) enabled workaround for: ARM erratum 1319537
>>>> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 54000 KHz
>>>> (XEN) GICv2 initialization:
>>>> (XEN)         gic_dist_addr=00000000ff841000
>>>> (XEN)         gic_cpu_addr=00000000ff842000
>>>> (XEN)         gic_hyp_addr=00000000ff844000
>>>> (XEN)         gic_vcpu_addr=00000000ff846000
>>>> (XEN)         gic_maintenance_irq=25
>>>> (XEN) GICv2: 256 lines, 4 cpus, secure (IID 0200143b).
>>>> (XEN) XSM Framework v1.0.1 initialized
>>>> (XEN) Initialising XSM SILO mode
>>>> (XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
>>>> (XEN) Initializing Credit2 scheduler
>>>> (XEN)  load_precision_shift: 18
>>>> (XEN)  load_window_shift: 30
>>>> (XEN)  underload_balance_tolerance: 0
>>>> (XEN)  overload_balance_tolerance: -3
>>>> (XEN)  runqueues arrangement: socket
>>>> (XEN)  cap enforcement granularity: 10ms
>>>> (XEN) load tracking window length 1073741824 ns
>>>> (XEN) Allocated console ring of 32 KiB.
>>>> (XEN) CPU0: Guest atomics will try 16 times before pausing the domain
>>>> (XEN) Bringing up CPU1
>>>> (XEN) CPU1: Guest atomics will try 16 times before pausing the domain
>>>> (XEN) CPU 1 booted.
>>>> (XEN) Bringing up CPU2
>>>> (XEN) CPU2: Guest atomics will try 13 times before pausing the domain
>>>> (XEN) CPU 2 booted.
>>>> (XEN) Bringing up CPU3
>>>> (XEN) CPU3: Guest atomics will try 16 times before pausing the domain
>>>> (XEN) Brought up 4 CPUs
>>>> (XEN) CPU 3 booted.
>>>> (XEN) I/O virtualisation disabled
>>>> (XEN) P2M: 44-bit IPA with 44-bit PA and 8-bit VMID
>>>> (XEN) P2M: 4 levels with order-0 root, VTCR 0x0000000080043594
>>>> (XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
>>>> (XEN) Initializing Credit2 scheduler
>>>> (XEN)  load_precision_shift: 18
>>>> (XEN)  load_window_shift: 30
>>>> (XEN)  underload_balance_tolerance: 0
>>>> (XEN)  overload_balance_tolerance: -3
>>>> (XEN)  runqueues arrangement: socket
>>>> (XEN)  cap enforcement granularity: 10ms
>>>> (XEN) load tracking window length 1073741824 ns
>>>> (XEN) Adding cpu 0 to runqueue 0
>>>> (XEN)  First cpu on runqueue, activating
>>>> (XEN) Adding cpu 1 to runqueue 0
>>>> (XEN) Adding cpu 2 to runqueue 0
>>>> (XEN) Adding cpu 3 to runqueue 0
>>>> (XEN) alternatives: Patching with alt table 00000a00002ee0b0 ->
>>>> 00000a00002ef250
>>>> (XEN) CPU2 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>>> (XEN) CPU1 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>>> (XEN) CPU3 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>>> (XEN) CPU0 will call ARM_SMCCC_ARCH_WORKAROUND_1 on exception entry
>>>> (XEN) *** LOADING DOMAIN 0 ***
>>>> (XEN) Loading d0 kernel from boot module @ 0000000030ef7000
>>>> (XEN) Loading ramdisk from boot module @ 000000002ee6d000
>>>> (XEN) Allocating 1:1 mappings totalling 1024MB for dom0:
>>>> (XEN) BANK[0] 0x00000040000000-0x00000080000000 (1024MB)
>>>> (XEN) Grant table range: 0x0000002eceb000-0x0000002ed2b000
>>>> (XEN) Allocating PPI 16 for event channel interrupt
>>>> (XEN) Loading zImage from 0000000030ef7000 to
>>>> 0000000040000000-0000000041f1c7c0
>>>> (XEN) Loading d0 initrd from 000000002ee6d000 to
>>>> 0x0000000048200000-0x000000004a
>>>>
>>>>
>>>>                                                      288c22
>>>> (XEN) Loading d0 DTB to 0x0000000048000000-0x00000000480002c3
>>>> (XEN) Initial low memory virq threshold set at 0x4000 pages.
>>>> (XEN) Scrubbing Free RAM in background
>>>> (XEN) Std. Loglevel: All
>>>> (XEN) Guest Loglevel: All
>>>> (XEN) ***************************************************
>>>> (XEN) WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
>>>> (XEN) This option is intended to aid debugging of Xen by ensuring
>>>> (XEN) that all output is synchronously delivered on the serial line.
>>>> (XEN) However it can introduce SIGNIFICANT latencies and affect
>>>> (XEN) timekeeping. It is NOT recommended for production use!
>>>> (XEN) ***************************************************
>>>> (XEN) 3... 2... 1...
>>>> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch 
>>>> input)
>>>> (XEN) Freed 376kB init memory.
>>>> (XEN) d0v0 Unhandled SMC/HVC: 0x84000050
>>>> (XEN) d0v0 Unhandled SMC/HVC: 0x8600ff01
>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER4
>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER8
>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER12
>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER16
>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER20
>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER24
>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER28
>>>> (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>>> (XEN) d0v1: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>>> (XEN) d0v2: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>>> (XEN) d0v3: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>>> (XEN) d1v0 Unhandled SMC/HVC: 0x84000050
>>>> (XEN) d1v0 Unhandled SMC/HVC: 0x8600ff01
>>>> (XEN) d1v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>>> (XEN) common/grant_table.c:1909:d1v0 Expanding d1 grant table from 1 to
>>>> 2 frames
>>>> (XEN) gnttab_mark_dirty not implemented yet
>>>> (XEN) d2v0 Unhandled SMC/HVC: 0x84000050
>>>> (XEN) d2v0 Unhandled SMC/HVC: 0x8600ff01
>>>> (XEN) d2v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
>>>> (XEN) common/grant_table.c:1909:d2v0 Expanding d2 grant table from 1 to
>>>> 2 frames
>>>> (XEN) common/grant_table.c:1909:d2v0 Expanding d2 grant table from 2 to
>>>> 3 frames
>>>> (XEN) common/grant_table.c:1909:d2v0 Expanding d2 grant table from 3 to
>>>> 4 frames
>>>> (XEN) Watchdog timer fired for domain 0
>>>> (XEN) Hardware Dom0 shutdown: watchdog rebooting machine
>>>
>>> I'm unsure whose attention to draw to this report.
>>>
>>> This might be a scheduler issue since the watchdog timer is triggering.
>>>
>>> This might be an ACPI issue as ACPI is in use here.
>>>
>>> This might be an ARM Linux kernel issue.
>>>
>>> In the end this is someone running into trouble with Xen on an ARM
>>> device.  Yet despite bringing up the issue hasn't gotten any help...
>>
>> Hey Elliot,
>>
>> Thanks for raising the visibility of this.  I'm not familiar with ARM,
>> but if I were investigating this I'd try to figure out what the
>> "unhandled" error messages are.  "gnttab_mark_dirty not implemented
>> yet" looks pretty sus too, and also sounds like it might be something
>> ARM-specific.
>>
>> I don't see anything suspicious WRT the scheduler, but a simple way to
>> test that would be to set the scheduler to credit1 and see if that
>> changes things.
> 
> Just to rule out all possibilities, I did a test with credit(1) 
> scheduler, which didn't change anything noticable. The watchdog 
> triggered after traffic has been created on the VLAN.

Thanks for trying. Looking at the watchdog code, sadly, it doesn't seem 
to do much logging or even dump some information (e.g. stack trace) when 
it get triggers.

To answer your previous question about the watchdog, it is a daemon that 
is running in the domain (in your case dom0). The only thing it will do 
is every N secs (default 15 seconds), it will tell Xen that it is still 
alive. If the domain has answered after M (default 30 seconds), then the 
domain will be considered as unresponsive and Xen will shutdown it down.

If this is dom0, then the whole system will shutdown. As the others 
hinted, this is likely a sign that your domain deadlock or is overloaded.

Have you tried to disable the watchdog and check the liveness of dom0? I 
don't expect to solve your issue, but it could help to get some further 
information out of the system before rebooting. The service is called 
xen-watchdog.

If dom0 is partially stuck, then I would expect the kernel to start 
printing some RCU stall after a few seconds.

If you have access to the console you can press CTRL + A three times. 
This will switch to Xen console where you can ask to print more 
information. 'h' will give you the list of all the keys supported.

I think, the keys '0' and 'q' would print useful information in your case.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun Feb 18 18:01:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 18:01:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682605.1061667 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rblTQ-0008W1-0i; Sun, 18 Feb 2024 18:01:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682605.1061667; Sun, 18 Feb 2024 18:01:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rblTP-0008Vu-UT; Sun, 18 Feb 2024 18:01:07 +0000
Received: by outflank-mailman (input) for mailman id 682605;
 Sun, 18 Feb 2024 18:01:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rblTO-0008Vo-RF
 for xen-devel@lists.xenproject.org; Sun, 18 Feb 2024 18:01:06 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rblTN-0007EF-KL; Sun, 18 Feb 2024 18:01:05 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rblTN-0003p2-Dd; Sun, 18 Feb 2024 18:01:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=ojsgBToOz6X3UzjRyjj+SvgeG73BehRDNrcOr2Cc43g=; b=m1i9uC3hbSSHLXu9RtBK1AB/EY
	DvMoa4r16TzKT8QqI97K1YrfrbjqJrhomhZkhrDhCF5L+OfMsCnf0A7a0BU0uy+3KNjVu6IauUG1K
	9Enq7I1OY8/8YMr5OAtI54J26KpWjv25i9j5RiOysaq94itCsOOYWq37TntUh4f4fwJE=;
Message-ID: <0374516c-b5fa-4880-a07a-0b788f491e9a@xen.org>
Date: Sun, 18 Feb 2024 18:01:03 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] Argo: don't obtain excess page references
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Christopher Clark <christopher.w.clark@gmail.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <79ed633d-b0bd-4a7d-a0c6-37a034e1ee96@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <79ed633d-b0bd-4a7d-a0c6-37a034e1ee96@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 14/02/2024 10:12, Jan Beulich wrote:
> find_ring_mfn() already holds a page reference when trying to obtain a
> writable type reference. We shouldn't make assumptions on the general
> reference count limit being effectively "infinity". Obtain merely a type
> ref, re-using the general ref by only dropping the previously acquired
> one in the case of an error.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

> ---
> I further question the log-dirty check there: The present P2M type of a
> page doesn't really matter for writing to the page (plus it's stale by
> the time it is looked at). Instead I think every write to such a page
> needs to be accompanied by a call to paging_mark_dirty().

I agree with that.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun Feb 18 18:18:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 18:18:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682611.1061677 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rblkQ-0001pL-ES; Sun, 18 Feb 2024 18:18:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682611.1061677; Sun, 18 Feb 2024 18:18:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rblkQ-0001pE-Bl; Sun, 18 Feb 2024 18:18:42 +0000
Received: by outflank-mailman (input) for mailman id 682611;
 Sun, 18 Feb 2024 18:18:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rblkP-0001op-FL
 for xen-devel@lists.xenproject.org; Sun, 18 Feb 2024 18:18:41 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rblkO-0007Vh-QT; Sun, 18 Feb 2024 18:18:40 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rblkO-0004iO-IJ; Sun, 18 Feb 2024 18:18:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=/EYkFiFKn06M8Qpd5VtyZLGtjiI8a1vMbq4daOrfo0U=; b=UMUuITGAn5DiZ2mSQp3ZX5V/gu
	/CgZ1QeYnhkJqwbDOMFgqXPgUchFcTl0cuSwe2xH9Gx8LGW8Az18aSju/SLgw887RDSov6yBDkzxV
	uopIPxHinkTj2ee9QQskHRV5OvAXMDXdARu5IzMOSB2LzZN0lk6CwkN00UaCJrNRfCaI=;
Message-ID: <f4f49147-0e81-4af7-8f26-a47a2c654990@xen.org>
Date: Sun, 18 Feb 2024 18:18:38 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 16/30] xen/riscv: introduce p2m.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <c2a2e872f8c32d81c3d3f428f0273819dd7df081.1707146506.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <c2a2e872f8c32d81c3d3f428f0273819dd7df081.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 05/02/2024 15:32, Oleksii Kurochko wrote:
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes in V4:
>   - update the comment above p2m_type_t. RISC-V has only 2 free for use bits in PTE, not 4 as Arm.
>   - update the comment after p2m_ram_rw: s/guest/domain/ as this also applies for dom0.
>   - return INVALID_MFN in gfn_to_mfn() instead of mfn(0).
>   - drop PPC changes.
> ---
> Changes in V3:
>   - add SPDX
>   - drop unneeded for now p2m types.
>   - return false in all functions implemented with BUG() inside.
>   - update the commit message
> ---
> Changes in V2:
>   - Nothing changed. Only rebase.
> ---
>   xen/arch/riscv/include/asm/p2m.h | 102 +++++++++++++++++++++++++++++++
>   1 file changed, 102 insertions(+)
>   create mode 100644 xen/arch/riscv/include/asm/p2m.h
> 
> diff --git a/xen/arch/riscv/include/asm/p2m.h b/xen/arch/riscv/include/asm/p2m.h
> new file mode 100644
> index 0000000000..8ad020974f
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/p2m.h
> @@ -0,0 +1,102 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef __ASM_RISCV_P2M_H__
> +#define __ASM_RISCV_P2M_H__
> +
> +#include <asm/page-bits.h>
> +
> +#define paddr_bits PADDR_BITS
> +
> +/*
> + * List of possible type for each page in the p2m entry.
> + * The number of available bit per page in the pte for this purpose is 2 bits.

That's not a lot and I expect you will ran out fairly quickly if you 
decide to store whether...

> + * So it's possible to only have 4 fields. If we run out of value in the
> + * future, it's possible to use higher value for pseudo-type and don't store
> + * them in the p2m entry.
> + */
> +typedef enum {
> +    p2m_invalid = 0,    /* Nothing mapped here */
> +    p2m_ram_rw,         /* Normal read/write domain RAM */

... the RAM is Read-Write. Depend on your P2M implementation, you could 
rely on the HW page-attributes to augment you p2m_type. So effectively, 
your two bits would contain information you can't already store.

Anyway, your approach is ok as your aim is to only build Xen for now. 
BUt this likely want to be re-think once you add the P2M support.

> +} p2m_type_t;
> +
> +#include <xen/p2m-common.h>
> +
> +static inline int get_page_and_type(struct page_info *page,
> +                                    struct domain *domain,
> +                                    unsigned long type)
> +{
> +    BUG_ON("unimplemented");
> +    return -EINVAL;
> +}
> +
> +/* Look up a GFN and take a reference count on the backing page. */
> +typedef unsigned int p2m_query_t;
> +#define P2M_ALLOC    (1u<<0)   /* Populate PoD and paged-out entries */
> +#define P2M_UNSHARE  (1u<<1)   /* Break CoW sharing */

Coding style: I understansd this is what Arm did, but the style is not 
correct. Please add a space before and after <<.

> +
> +static inline struct page_info *get_page_from_gfn(
> +    struct domain *d, unsigned long gfn, p2m_type_t *t, p2m_query_t q)
> +{
> +    BUG_ON("unimplemented");
> +    return NULL;
> +}
> +
> +static inline void memory_type_changed(struct domain *d)
> +{
> +    BUG_ON("unimplemented");
> +}
> +
> +
> +static inline int guest_physmap_mark_populate_on_demand(struct domain *d, unsigned long gfn,
> +                                                        unsigned int order)
> +{
> +    return -EOPNOTSUPP;
> +}
> +
> +static inline int guest_physmap_add_entry(struct domain *d,
> +                            gfn_t gfn,
> +                            mfn_t mfn,
> +                            unsigned long page_order,
> +                            p2m_type_t t)
> +{
> +    BUG_ON("unimplemented");
> +    return -EINVAL;
> +}
> +
> +/* Untyped version for RAM only, for compatibility */
> +static inline int __must_check
> +guest_physmap_add_page(struct domain *d, gfn_t gfn, mfn_t mfn,
> +                       unsigned int page_order)
> +{
> +    return guest_physmap_add_entry(d, gfn, mfn, page_order, p2m_ram_rw);
> +}
> +
> +static inline mfn_t gfn_to_mfn(struct domain *d, gfn_t gfn)
> +{
> +    BUG_ON("unimplemented");
> +    return INVALID_MFN;
> +}
> +
> +static inline bool arch_acquire_resource_check(struct domain *d)
> +{
> +    /*
> +     * The reference counting of foreign entries in set_foreign_p2m_entry()
> +     * is supported on RISCV.
> +     */
> +    return true;

AFAICT, the current implementation of set_foreign_p2m_entry() is a 
BUG_ON(). So I think it would make sense to return 'false' as this 
reflects better the current state.

> +}
> +
> +static inline void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
> +{
> +    /* Not supported on RISCV. */
> +}
> +
> +#endif /* __ASM_RISCV_P2M_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun Feb 18 18:19:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 18:19:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682615.1061688 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbllD-0002SP-Nr; Sun, 18 Feb 2024 18:19:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682615.1061688; Sun, 18 Feb 2024 18:19:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbllD-0002SI-Kc; Sun, 18 Feb 2024 18:19:31 +0000
Received: by outflank-mailman (input) for mailman id 682615;
 Sun, 18 Feb 2024 18:19:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbllC-0002S2-A8; Sun, 18 Feb 2024 18:19:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbllC-0007YB-8Y; Sun, 18 Feb 2024 18:19:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbllB-0007eo-S2; Sun, 18 Feb 2024 18:19:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rbllB-0003O3-Rb; Sun, 18 Feb 2024 18:19:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=chKYPglG3btpYPf3kHIQ3hQYw2llvxmb2cc3CYfTOhw=; b=WBmpLEOA0xiMcpFW6gQ15JbwMg
	cB1vP7RnkhlG3b9REwwrgD2JfirgVeVgwH77mKKiz/7SvcN7lfh2sqWre4S6AnUKtKooQs4ABEuyV
	DNAlqbOFah8qqnGVhyLUWtwSwPNQkH3pbFaydEyJGcU5UGskta0V8VlW97iq+ajHlkJ4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184696-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184696: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=c02197fc9076e7d991c8f6adc11759c5ba52ddc6
X-Osstest-Versions-That:
    linux=c1ca10ceffbb289ed02feaf005bc9ee6095b4507
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 18 Feb 2024 18:19:29 +0000

flight 184696 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184696/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184691
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184691
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184691
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184691
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184691
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184691
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184691
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184691
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                c02197fc9076e7d991c8f6adc11759c5ba52ddc6
baseline version:
 linux                c1ca10ceffbb289ed02feaf005bc9ee6095b4507

Last test of basis   184691  2024-02-17 06:55:54 Z    1 days
Failing since        184694  2024-02-17 22:14:07 Z    0 days    2 attempts
Testing same since   184696  2024-02-18 07:52:17 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Jiri Slaby (SUSE)" <jirislaby@kernel.org>
  Abel Vesa <abel.vesa@linaro.org>
  Alex Williamson <alex.williamson@redhat.com>
  Arnd Bergmann <arnd@arndb.de>
  Bjorn Helgaas <bhelgaas@google.com>
  Cosmin Tanislav <demonsingur@gmail.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  David Engraf <david.engraf@sysgo.com>
  David Schiller <david.schiller@jku.at>
  Dietmar Eggemann <dietmar.eggemann@arm.com>
  Dimitri Fedrau <dima.fedrau@gmail.com>
  Dinghao Liu <dinghao.liu@zju.edu.cn>
  Emil Kronborg <emil.kronborg@protonmail.com>
  Gaurav Batra <gbatra@linux.ibm.com>
  Georgi Djakov <djakov@kernel.org>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Hans de Goede <hdegoede@redhat.com>
  Javier Carrasco <javier.carrasco.cruz@gmail.com>
  Jiangfeng Xiao <xiaojiangfeng@huawei.com>
  Jiri Slaby (SUSE) <jirislaby@kernel.org>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Kent Overstreet <kent.overstreet@linux.dev>
  Konrad Dybcio <konrad.dybcio@linaro.org>
  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
  Lee Jones <lee@kernel.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Mario Limonciello <mario.limonciello@amd.com>
  Mark Brown <broonie@kernel.org>
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Matthew Ruffell <matthew.ruffell@canonical.com>
  Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
  Mauro Carvalho Chehab <mchehab@kernel.org>
  Michael Ellerman <mpe@ellerman.id.au>
  Mika Westerberg <mika.westerberg@linux.intel.com>
  Mike Tipton <quic_mdtipton@quicinc.com>
  Mohammad Rahimi <rahimi.mhmmd@gmail.com>
  Nathan Lynch <nathanl@linux.ibm.com>
  Naveen N Rao <naveen@kernel.org>
  Nuno Sa <nuno.sa@analog.com>
  Paul Barker <paul.barker.ct@bp.renesas.com>
  Paulo Alcantara (Red Hat) <pc@manguebit.com>
  Paulo Alcantara <pc@manguebit.com>
  Pierre Gondois <pierre.gondois@arm.com>
  R Nageswara Sastry <rnsastry@linux.ibm.com>
  Randy Dunlap <rdunlap@infradead.org>
  Ronnie Sahlberg <ronniesahlberg@gmail.com>
  Sam Protsenko <semen.protsenko@linaro.org>
  Sanath S <sanath.s@amd.com>
  Saravana Kannan <saravanak@google.com>
  Sasha Levin <sashal@kernel.org>
  Sean Young <sean@mess.org>
  Shivaprasad G Bhat <sbhat@linux.ibm.com>
  Shrikanth Hegde <sshegde@linux.ibm.com>
  Shyam Prasad N <sprasad@microsoft.com>
  Siddharth Vadapalli <s-vadapalli@ti.com>
  Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Stefan Wahren <wahrenst@gmx.net>
  Steve French <stfrench@microsoft.com>
  Su Yue <glass.su@suse.com>
  Sudeep Holla <sudeep.holla@arm.com>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
  Venkat Rao Bagalkote <venkat88@linux.vnet.ibm.com>
  Vincent Guittot <vincent.guittot@linaro.org>
  Xu Yang <xu.yang_2@nxp.com>
  zhili.liu <zhili.liu@ucas.com.cn>
  Zhipeng Lu <alexious@zju.edu.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   c1ca10ceffbb..c02197fc9076  c02197fc9076e7d991c8f6adc11759c5ba52ddc6 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun Feb 18 18:22:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 18:22:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682620.1061698 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rblns-0003xh-AG; Sun, 18 Feb 2024 18:22:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682620.1061698; Sun, 18 Feb 2024 18:22:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rblns-0003xa-5t; Sun, 18 Feb 2024 18:22:16 +0000
Received: by outflank-mailman (input) for mailman id 682620;
 Sun, 18 Feb 2024 18:22:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rblnr-0003xU-4A
 for xen-devel@lists.xenproject.org; Sun, 18 Feb 2024 18:22:15 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rblnq-0007an-TF; Sun, 18 Feb 2024 18:22:14 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rblnq-00050x-Np; Sun, 18 Feb 2024 18:22:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=cQFI98VV1YZYGzqepX/tZFMS0fNNz4PeN/G23l495II=; b=Ey/JpORooTusyFnG+8pThaCaAI
	ZTmR/OFyDjdcnhK7Ac1Qjszs6pl8q1GMrnCJJflxKq0f0Uvm6DWpRxX1QUQN1PqGnf38bHAgak+wI
	KTtUA2zcNBrcG03rZmjqhynrLxHFrjTDhM8RKKw1BayAp+NxNBSlrH3UldGDHc3hZWxw=;
Message-ID: <ea8d10ef-39cd-49a2-a445-6705b87d50b3@xen.org>
Date: Sun, 18 Feb 2024 18:22:12 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 17/30] xen/riscv: introduce regs.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <13cfdfa01b303d2597124f09c0ec0734859b697d.1707146506.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <13cfdfa01b303d2597124f09c0ec0734859b697d.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 05/02/2024 15:32, Oleksii Kurochko wrote:
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> ------
> Changes in V4:
>   - add Acked-by: Jan Beulich <jbeulich@suse.com>
>   - s/BUG()/BUG_ON("unimplemented")
> ---
> Changes in V3:
>   - update the commit message
>   - add Acked-by: Jan Beulich <jbeulich@suse.com>
>   - remove "include <asm/current.h>" and use a forward declaration instead.
> ---
> Changes in V2:
>   - change xen/lib.h to xen/bug.h
>   - remove unnecessary empty line
> ---
> xen/arch/riscv/include/asm/regs.h | 29 +++++++++++++++++++++++++++++
>   1 file changed, 29 insertions(+)
>   create mode 100644 xen/arch/riscv/include/asm/regs.h
> 
> diff --git a/xen/arch/riscv/include/asm/regs.h b/xen/arch/riscv/include/asm/regs.h
> new file mode 100644
> index 0000000000..c70ea2aa0c
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/regs.h
> @@ -0,0 +1,29 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef __ARM_RISCV_REGS_H__
> +#define __ARM_RISCV_REGS_H__
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <xen/bug.h>
> +
> +#define hyp_mode(r)     (0)

I don't understand where here you return 0 (which should really be 
false) but ...

> +
> +struct cpu_user_regs;
> +
> +static inline bool guest_mode(const struct cpu_user_regs *r)
> +{
> +    BUG_ON("unimplemented");
> +}

... here you return BUG_ON(). But I couldn't find any user of both 
guest_mode() and hyp_mode(). So isn't it a bit prematurate to introduce 
the helpers?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun Feb 18 18:30:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 18:30:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682630.1061707 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rblvh-0005i7-2E; Sun, 18 Feb 2024 18:30:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682630.1061707; Sun, 18 Feb 2024 18:30:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rblvg-0005i0-Vl; Sun, 18 Feb 2024 18:30:20 +0000
Received: by outflank-mailman (input) for mailman id 682630;
 Sun, 18 Feb 2024 18:30:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rblvf-0005hs-Kj
 for xen-devel@lists.xenproject.org; Sun, 18 Feb 2024 18:30:19 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rblve-0007lj-RC; Sun, 18 Feb 2024 18:30:18 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rblve-00058D-JD; Sun, 18 Feb 2024 18:30:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=VIPeXSndfpHhbyM7Ys8RX6Ad5Aos+b5pVNznosgEkSI=; b=6I9LUDzLKFbeCtHfthsQcS59aQ
	+Ru/FgVnTgBHrAUNUlxfIJZXBFPJCGtv4OQrxwYSxvuZK+JZRI4rII9LniBQgrMWfztZh3gbmmHuX
	6H2nA9MCd/ieGJwLEeoKF+EbN3MCJeXy7LcJNo+v4c/c/2eiCQ9QekFPAqK7+BmsoYSY=;
Message-ID: <510d74ca-5cce-48f8-93f6-407cd727cf7b@xen.org>
Date: Sun, 18 Feb 2024 18:30:16 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 07/30] xen/asm-generic: introdure nospec.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>, Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <5889d7a5fa81722472f95cc1448af0be8f359a7d.1707146506.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <5889d7a5fa81722472f95cc1448af0be8f359a7d.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

Title: Typo s/introdure/introduce/

On 05/02/2024 15:32, Oleksii Kurochko wrote:
> The <asm/nospec.h> header is similar between Arm, PPC, and RISC-V,
> so it has been moved to asm-generic.

I am not 100% convinced that moving this header to asm-generic is a good 
idea. At least for Arm, those helpers ought to be non-empty, what about 
RISC-V?

If the answer is they should be non-empty. Then I would consider to keep 
the duplication to make clear that each architecture should take their 
own decision in term of security.

The alternative, is to have a generic implementation that is safe by 
default (if that's even possible).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun Feb 18 19:01:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 19:01:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682637.1061718 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbmPO-0001cZ-Db; Sun, 18 Feb 2024 19:01:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682637.1061718; Sun, 18 Feb 2024 19:01:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbmPO-0001cS-9k; Sun, 18 Feb 2024 19:01:02 +0000
Received: by outflank-mailman (input) for mailman id 682637;
 Sun, 18 Feb 2024 19:01:01 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rbmPM-0001cM-U7
 for xen-devel@lists.xenproject.org; Sun, 18 Feb 2024 19:01:00 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rbmPM-0008Qa-Dc; Sun, 18 Feb 2024 19:01:00 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rbmPM-0006ka-2L; Sun, 18 Feb 2024 19:01:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=Gach0ne8t5TA97xMT6m7LLcX8cXY3fyAnDntvmWBjq0=; b=nbO/VRS4l5I2WuyjTk+Hyp04Lr
	QP6EZZdYPvqeq1P0fnfXsW7qg8Lwdu/ASqVssB5iClwdLIAb8oiqkYSUFYIz5B6csH3RUeZM6hWBS
	k1bu/rZ1SWKesiL1/8syhrmh9XZb3GxJ3qePIR/K41vf9vLv04zgk371o6YV4mX+qKj8=;
Message-ID: <491937fd-2459-416f-944b-0c63f7c12473@xen.org>
Date: Sun, 18 Feb 2024 19:00:58 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 05/02/2024 15:32, Oleksii Kurochko wrote:
> The header was taken from Linux kernl 6.4.0-rc1.
> 
> Addionally, were updated:
> * add emulation of {cmp}xchg for 1/2 byte types

This explaination is a little bit light. IIUC, you are implementing them 
using 32-bit atomic access. Is that correct? If so, please spell it out.

Also, I wonder whether it would be better to try to get rid of the 1/2 
bytes access. Do you know where they are used?

> * replace tabs with spaces
Does this mean you are not planning to backport any Linux fixes?

> * replace __* varialbed with *__

s/varialbed/variable/

> * introduce generic version of xchg_* and cmpxchg_*.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes in V4:
>   - Code style fixes.
>   - enforce in __xchg_*() has the same type for new and *ptr, also "\n"
>     was removed at the end of asm instruction.
>   - dependency from https://lore.kernel.org/xen-devel/cover.1706259490.git.federico.serafini@bugseng.com/
>   - switch from ASSERT_UNREACHABLE to STATIC_ASSERT_UNREACHABLE().
>   - drop xchg32(ptr, x) and xchg64(ptr, x) as they aren't used.
>   - drop cmpxcg{32,64}_{local} as they aren't used.
>   - introduce generic version of xchg_* and cmpxchg_*.
>   - update the commit message.
> ---
> Changes in V3:
>   - update the commit message
>   - add emulation of {cmp}xchg_... for 1 and 2 bytes types
> ---
> Changes in V2:
>   - update the comment at the top of the header.
>   - change xen/lib.h to xen/bug.h.
>   - sort inclusion of headers properly.
> ---
>   xen/arch/riscv/include/asm/cmpxchg.h | 237 +++++++++++++++++++++++++++
>   1 file changed, 237 insertions(+)
>   create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h
> 
> diff --git a/xen/arch/riscv/include/asm/cmpxchg.h b/xen/arch/riscv/include/asm/cmpxchg.h
> new file mode 100644
> index 0000000000..b751a50cbf
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/cmpxchg.h
> @@ -0,0 +1,237 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/* Copyright (C) 2014 Regents of the University of California */
> +
> +#ifndef _ASM_RISCV_CMPXCHG_H
> +#define _ASM_RISCV_CMPXCHG_H
> +
> +#include <xen/compiler.h>
> +#include <xen/lib.h>
> +
> +#include <asm/fence.h>
> +#include <asm/io.h>
> +#include <asm/system.h>
> +
> +#define ALIGN_DOWN(addr, size)  ((addr) & (~((size) - 1)))
> +
> +#define __amoswap_generic(ptr, new, ret, sfx, release_barrier, acquire_barrier) \
> +({ \
> +    asm volatile( \
> +        release_barrier \
> +        " amoswap" sfx " %0, %2, %1\n" \
> +        acquire_barrier \
> +        : "=r" (ret), "+A" (*ptr) \
> +        : "r" (new) \
> +        : "memory" ); \
> +})
> +
> +#define emulate_xchg_1_2(ptr, new, ret, release_barrier, acquire_barrier) \
> +({ \
> +    uint32_t *ptr_32b_aligned = (uint32_t *)ALIGN_DOWN((unsigned long)ptr, 4); \
> +    uint8_t mask_l = ((unsigned long)(ptr) & (0x8 - sizeof(*ptr))) * BITS_PER_BYTE; \
> +    uint8_t mask_size = sizeof(*ptr) * BITS_PER_BYTE; \
> +    uint8_t mask_h = mask_l + mask_size - 1; \
> +    unsigned long mask = GENMASK(mask_h, mask_l); \
> +    unsigned long new_ = (unsigned long)(new) << mask_l; \
> +    unsigned long ret_; \
> +    unsigned long rc; \
> +    \
> +    asm volatile( \
> +        release_barrier \
> +        "0: lr.d %0, %2\n" \

I was going to ask why this is lr.d rather than lr.w. But I see Jan 
already asked. I agree with him that it should probably be a lr.w and ...

> +        "   and  %1, %0, %z4\n" \
> +        "   or   %1, %1, %z3\n" \
> +        "   sc.d %1, %1, %2\n" \

... respectively sc.w. The same applies for cmpxchg.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun Feb 18 19:08:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 19:08:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682643.1061728 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbmVw-0002IH-2u; Sun, 18 Feb 2024 19:07:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682643.1061728; Sun, 18 Feb 2024 19:07:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbmVv-0002IA-Vq; Sun, 18 Feb 2024 19:07:47 +0000
Received: by outflank-mailman (input) for mailman id 682643;
 Sun, 18 Feb 2024 19:07:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rbmVv-0002I2-8K
 for xen-devel@lists.xenproject.org; Sun, 18 Feb 2024 19:07:47 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rbmVu-000064-Ag; Sun, 18 Feb 2024 19:07:46 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rbmVu-00075l-1M; Sun, 18 Feb 2024 19:07:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=16JJ2E5ZzPSV7RUl42eeFeuchlqNnTqERQgrGYqYmTc=; b=kZ3zDdfmg65TVqowh9UXOC3Oyp
	OHNyTw71R+s1Ikx0RGJhZoXM0+3JsIpqkfdB/bZk2asFxiPVCQaVqZboxS5S+N4A8C3JyflCLLsOh
	jqcNyH9vSNfIT5P2TShWGqHYsx/8HH5XwUNyh3ZwuOSJWX9Y6nA0P29l5u5eVgsmjTCk=;
Message-ID: <afa59d63-2310-4081-9fc8-1d8a66ae24fc@xen.org>
Date: Sun, 18 Feb 2024 19:07:43 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 13/30] xen/riscv: introduce io.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <64afc021f680e55d486f8429c02513bc9bf4b9ea.1707146506.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <64afc021f680e55d486f8429c02513bc9bf4b9ea.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 05/02/2024 15:32, Oleksii Kurochko wrote:
> The header taken form Linux 6.4.0-rc1 and is based on
> arch/riscv/include/asm/mmio.h.
> 
> Addionally, to the header was added definions of ioremap_*().

s/definions/definitions/

> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes in V4:
>   - delete inner parentheses in macros.
>   - s/u<N>/uint<N>.
> ---
> Changes in V3:
>   - re-sync with linux kernel
>   - update the commit message
> ---
> Changes in V2:
>   - Nothing changed. Only rebase.
> ---
>   xen/arch/riscv/include/asm/io.h | 142 ++++++++++++++++++++++++++++++++
>   1 file changed, 142 insertions(+)
>   create mode 100644 xen/arch/riscv/include/asm/io.h
> 
> diff --git a/xen/arch/riscv/include/asm/io.h b/xen/arch/riscv/include/asm/io.h
> new file mode 100644
> index 0000000000..1e61a40522
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/io.h
> @@ -0,0 +1,142 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * {read,write}{b,w,l,q} based on arch/arm64/include/asm/io.h
> + *   which was based on arch/arm/include/io.h
> + *
> + * Copyright (C) 1996-2000 Russell King
> + * Copyright (C) 2012 ARM Ltd.
> + * Copyright (C) 2014 Regents of the University of California
> + */
> +
> +
> +#ifndef _ASM_RISCV_IO_H
> +#define _ASM_RISCV_IO_H
> +
> +#include <asm/byteorder.h>
> +
> +/*
> + * The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't
> + * change the properties of memory regions.  This should be fixed by the
> + * upcoming platform spec.
> + */
> +#define ioremap_nocache(addr, size) ioremap(addr, size)
> +#define ioremap_wc(addr, size) ioremap(addr, size)
> +#define ioremap_wt(addr, size) ioremap(addr, size)
> +
> +/* Generic IO read/write.  These perform native-endian accesses. */
> +#define __raw_writeb __raw_writeb
> +static inline void __raw_writeb(uint8_t val, volatile void __iomem *addr)
> +{
> +	asm volatile("sb %0, 0(%1)" : : "r" (val), "r" (addr));
> +}
> +
> +#define __raw_writew __raw_writew
> +static inline void __raw_writew(uint16_t val, volatile void __iomem *addr)
> +{
> +	asm volatile("sh %0, 0(%1)" : : "r" (val), "r" (addr));
> +}
> +
> +#define __raw_writel __raw_writel
> +static inline void __raw_writel(uint32_t val, volatile void __iomem *addr)
> +{
> +	asm volatile("sw %0, 0(%1)" : : "r" (val), "r" (addr));
> +}
> +
> +#ifdef CONFIG_64BIT
> +#define __raw_writeq __raw_writeq
> +static inline void __raw_writeq(u64 val, volatile void __iomem *addr)
> +{
> +	asm volatile("sd %0, 0(%1)" : : "r" (val), "r" (addr));
> +}
> +#endif
> +
> +#define __raw_readb __raw_readb
> +static inline uint8_t __raw_readb(const volatile void __iomem *addr)
> +{
> +	uint8_t val;
> +
> +	asm volatile("lb %0, 0(%1)" : "=r" (val) : "r" (addr));
> +	return val;
> +}
> +
> +#define __raw_readw __raw_readw
> +static inline uint16_t __raw_readw(const volatile void __iomem *addr)
> +{
> +	uint16_t val;
> +
> +	asm volatile("lh %0, 0(%1)" : "=r" (val) : "r" (addr));
> +	return val;
> +}
> +
> +#define __raw_readl __raw_readl
> +static inline uint32_t __raw_readl(const volatile void __iomem *addr)
> +{
> +	uint32_t val;
> +
> +	asm volatile("lw %0, 0(%1)" : "=r" (val) : "r" (addr));
> +	return val;
> +}
> +
> +#ifdef CONFIG_64BIT
> +#define __raw_readq __raw_readq
> +static inline u64 __raw_readq(const volatile void __iomem *addr)
> +{
> +	u64 val;
> +
> +	asm volatile("ld %0, 0(%1)" : "=r" (val) : "r" (addr));
> +	return val;
> +}
> +#endif
> +
> +/*
> + * Unordered I/O memory access primitives.  These are even more relaxed than
> + * the relaxed versions, as they don't even order accesses between successive
> + * operations to the I/O regions.
> + */
> +#define readb_cpu(c)		({ uint8_t  __r = __raw_readb(c); __r; })
> +#define readw_cpu(c)		({ uint16_t __r = le16_to_cpu((__force __le16)__raw_readw(c)); __r; })
> +#define readl_cpu(c)		({ uint32_t __r = le32_to_cpu((__force __le32)__raw_readl(c)); __r; })
> +
> +#define writeb_cpu(v,c)		((void)__raw_writeb(v,c))
> +#define writew_cpu(v,c)		((void)__raw_writew((__force uint16_t)cpu_to_le16(v),c))
> +#define writel_cpu(v,c)		((void)__raw_writel((__force uint32_t)cpu_to_le32(v),c))

NIT: __raw_write*() are already returning void. So I am not sure to 
understand the pointer of the cast. IIUC, this is coming from Linux, are 
you intend to keep the code as-is (including style)? If not, then I 
woudl consider to drop the cast on the three lines above and ...

> +
> +#ifdef CONFIG_64BIT
> +#define readq_cpu(c)		({ u64 __r = le64_to_cpu((__force __le64)__raw_readq(c)); __r; })
> +#define writeq_cpu(v,c)		((void)__raw_writeq((__force u64)cpu_to_le64(v),c))

... here as well.

> +#endif
> +
> +/*
> + * I/O memory access primitives. Reads are ordered relative to any
> + * following Normal memory access. Writes are ordered relative to any prior
> + * Normal memory access.  The memory barriers here are necessary as RISC-V
> + * doesn't define any ordering between the memory space and the I/O space.
> + */
> +#define __io_br()	do {} while (0)
> +#define __io_ar(v)	__asm__ __volatile__ ("fence i,r" : : : "memory");
> +#define __io_bw()	__asm__ __volatile__ ("fence w,o" : : : "memory");
> +#define __io_aw()	do { } while (0)
> +
> +#define readb(c)	({ uint8_t  __v; __io_br(); __v = readb_cpu(c); __io_ar(__v); __v; })
> +#define readw(c)	({ uint16_t __v; __io_br(); __v = readw_cpu(c); __io_ar(__v); __v; })
> +#define readl(c)	({ uint32_t __v; __io_br(); __v = readl_cpu(c); __io_ar(__v); __v; })
> +
> +#define writeb(v,c)	({ __io_bw(); writeb_cpu(v,c); __io_aw(); })
> +#define writew(v,c)	({ __io_bw(); writew_cpu(v,c); __io_aw(); })
> +#define writel(v,c)	({ __io_bw(); writel_cpu(v,c); __io_aw(); })
> +
> +#ifdef CONFIG_64BIT
> +#define readq(c)	({ u64 __v; __io_br(); __v = readq_cpu(c); __io_ar(__v); __v; })
> +#define writeq(v,c)	({ __io_bw(); writeq_cpu((v),(c)); __io_aw(); })
> +#endif
> +
> +#endif /* _ASM_RISCV_IO_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sun Feb 18 19:22:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 18 Feb 2024 19:22:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682649.1061738 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbmkO-0005SN-6t; Sun, 18 Feb 2024 19:22:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682649.1061738; Sun, 18 Feb 2024 19:22:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbmkO-0005SG-40; Sun, 18 Feb 2024 19:22:44 +0000
Received: by outflank-mailman (input) for mailman id 682649;
 Sun, 18 Feb 2024 19:22:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rbmkN-0005SA-5s
 for xen-devel@lists.xenproject.org; Sun, 18 Feb 2024 19:22:43 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rbmkM-0000M4-EO; Sun, 18 Feb 2024 19:22:42 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rbmkM-0007wi-5B; Sun, 18 Feb 2024 19:22:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=tZa49q/QosZ/QmxnsOn3llBAoWIA8D6bzF743wTe0YI=; b=13cOR7PUX180ZBLcOIysUAJRAE
	cQn9LA9BTtdd0VcIslhpWJWtV/WkE9yoTV/Nf1YqbdLs27a4A9VroB6p9/CGs1cTEDz0b4yvxmX/L
	6P62g8rEO/pcwunYwBo2tFXah5htt+A1u6slTkwgf8ll6vccLNq/omEHiF+UNYbwlWr8=;
Message-ID: <26e01c13-364d-474b-8b8e-d1721ff0daf0@xen.org>
Date: Sun, 18 Feb 2024 19:22:40 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 14/30] xen/riscv: introduce atomic.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Bobby Eshleman <bobbyeshleman@gmail.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Connor Davis <connojdavis@gmail.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <6554f2479e19ed3eae6de842ac1568c31d236461.1707146506.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <6554f2479e19ed3eae6de842ac1568c31d236461.1707146506.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 05/02/2024 15:32, Oleksii Kurochko wrote:
> From: Bobby Eshleman <bobbyeshleman@gmail.com>
> 
> Additionally, this patch introduces macros in fence.h,
> which are utilized in atomic.h.
> 
> atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n)
> were updated to use __*xchg_generic().
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

The author is Bobby, but I don't see a Signed-off-by. Did you forgot it?

> ---
> Changes in V4:
>   - do changes related to the updates of [PATCH v3 13/34] xen/riscv: introduce cmpxchg.h
>   - drop casts in read_atomic_size(), write_atomic(), add_sized()
>   - tabs -> spaces
>   - drop #ifdef CONFIG_SMP ... #endif in fence.ha as it is simpler to handle NR_CPUS=1
>     the same as NR_CPUS>1 with accepting less than ideal performance.
> ---
> Changes in V3:
>    - update the commit message
>    - add SPDX for fence.h
>    - code style fixes
>    - Remove /* TODO: ... */ for add_sized macros. It looks correct to me.
>    - re-order the patch
>    - merge to this patch fence.h
> ---
> Changes in V2:
>   - Change an author of commit. I got this header from Bobby's old repo.
> ---
>   xen/arch/riscv/include/asm/atomic.h | 395 ++++++++++++++++++++++++++++
>   xen/arch/riscv/include/asm/fence.h  |   8 +
>   2 files changed, 403 insertions(+)
>   create mode 100644 xen/arch/riscv/include/asm/atomic.h
>   create mode 100644 xen/arch/riscv/include/asm/fence.h
> 
> diff --git a/xen/arch/riscv/include/asm/atomic.h b/xen/arch/riscv/include/asm/atomic.h
> new file mode 100644
> index 0000000000..267d3c0803
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/atomic.h
> @@ -0,0 +1,395 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Taken and modified from Linux.

Which version of Linux? Can you also spell out what are the big changes? 
This would be helpful if we need to re-sync.

> + *
> + * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated to use
> + * __*xchg_generic()
> + *
> + * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
> + * Copyright (C) 2012 Regents of the University of California
> + * Copyright (C) 2017 SiFive
> + * Copyright (C) 2021 Vates SAS
> + */
> +
> +#ifndef _ASM_RISCV_ATOMIC_H
> +#define _ASM_RISCV_ATOMIC_H
> +
> +#include <xen/atomic.h>
> +#include <asm/cmpxchg.h>
> +#include <asm/fence.h>
> +#include <asm/io.h>
> +#include <asm/system.h>
> +
> +void __bad_atomic_size(void);
> +
> +static always_inline void read_atomic_size(const volatile void *p,
> +                                           void *res,
> +                                           unsigned int size)
> +{
> +    switch ( size )
> +    {
> +    case 1: *(uint8_t *)res = readb(p); break;
> +    case 2: *(uint16_t *)res = readw(p); break;
> +    case 4: *(uint32_t *)res = readl(p); break;
> +    case 8: *(uint32_t *)res  = readq(p); break;
> +    default: __bad_atomic_size(); break;
> +    }
> +}
> +
> +#define read_atomic(p) ({                               \
> +    union { typeof(*p) val; char c[0]; } x_;            \
> +    read_atomic_size(p, x_.c, sizeof(*p));              \
> +    x_.val;                                             \
> +})
> +
> +#define write_atomic(p, x)                              \
> +({                                                      \
> +    typeof(*p) x__ = (x);                               \
> +    switch ( sizeof(*p) )                               \
> +    {                                                   \
> +    case 1: writeb((uint8_t)x__,  p); break;            \
> +    case 2: writew((uint16_t)x__, p); break;            \
> +    case 4: writel((uint32_t)x__, p); break;            \
> +    case 8: writeq((uint64_t)x__, p); break;            \
> +    default: __bad_atomic_size(); break;                \
> +    }                                                   \
> +    x__;                                                \
> +})
> +
> +#define add_sized(p, x)                                 \
> +({                                                      \
> +    typeof(*(p)) x__ = (x);                             \
> +    switch ( sizeof(*(p)) )                             \
> +    {                                                   \
> +    case 1: writeb(read_atomic(p) + x__, p); break;     \
> +    case 2: writew(read_atomic(p) + x__, p); break;     \
> +    case 4: writel(read_atomic(p) + x__, p); break;     \
> +    default: __bad_atomic_size(); break;                \
> +    }                                                   \
> +})
> +
> +/*
> + *  __unqual_scalar_typeof(x) - Declare an unqualified scalar type, leaving
> + *               non-scalar types unchanged.
> + *
> + * Prefer C11 _Generic for better compile-times and simpler code. Note: 'char'

Xen is technically built using c99/gnu99. So it is feels a bit odd to 
introduce a C11 feature. I see that _Generic is already used in PPC... 
However, if we decide to add more use of it, then I think this should at 
minimum be documented in docs/misra/C-language-toolchain.rst (the more 
if we plan the macro is moved to common as Jan suggested).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 03:39:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 03:39:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682663.1061748 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbuV8-00067w-W5; Mon, 19 Feb 2024 03:39:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682663.1061748; Mon, 19 Feb 2024 03:39:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbuV8-00067p-RM; Mon, 19 Feb 2024 03:39:30 +0000
Received: by outflank-mailman (input) for mailman id 682663;
 Mon, 19 Feb 2024 03:39:29 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbuV7-00067f-Ok; Mon, 19 Feb 2024 03:39:29 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbuV7-0002YK-Br; Mon, 19 Feb 2024 03:39:29 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rbuV6-00081n-3Y; Mon, 19 Feb 2024 03:39:28 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rbuV6-0002mZ-0n; Mon, 19 Feb 2024 03:39:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zxIi/BxLvtHsXyR2dHLVpxjeyaFyXvsVArin/WgeUX4=; b=dWVAWZ53at9giiypdh2jV7x6M4
	cJ6hcQL8z32Wj8sje51ArVM9RG6ksBOUg0aZcZxx8tv1MUsvVYg/5myfIQi39nx5o22HFpfOhOrwe
	kIeJ1WoEx8HeAWzC25Q+W67Pi2cc+IAEV0Vq558pzQOy/jxrowFVvpMiYKTouq3Ytp0s=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184697-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184697: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-dom0pvh-xl-amd:guest-start:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=6c160f16be5df1f66f6afe186c961ad446d7f94b
X-Osstest-Versions-That:
    linux=c02197fc9076e7d991c8f6adc11759c5ba52ddc6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 19 Feb 2024 03:39:28 +0000

flight 184697 linux-linus real [real]
flight 184698 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184697/
http://logs.test-lab.xenproject.org/osstest/logs/184698/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-dom0pvh-xl-amd 14 guest-start      fail pass in 184698-retest
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 18 guest-localmigrate/x10 fail pass in 184698-retest
 test-armhf-armhf-xl           8 xen-boot            fail pass in 184698-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl         15 migrate-support-check fail in 184698 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 184698 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184696
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184696
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184696
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184696
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184696
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184696
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184696
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184696
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                6c160f16be5df1f66f6afe186c961ad446d7f94b
baseline version:
 linux                c02197fc9076e7d991c8f6adc11759c5ba52ddc6

Last test of basis   184696  2024-02-18 07:52:17 Z    0 days
Testing same since   184697  2024-02-18 18:43:56 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andi Shyti <andi.shyti@kernel.org>
  Andrew Ballance <andrewjballance@gmail.com>
  Arnd Bergmann <arnd@arndb.de>
  Bibo Mao <maobibo@loongson.cn>
  Bryan O'Donoghue <bryan.odonoghue@linaro.org> # qrb5165-rb5
  Dan Carpenter <dan.carpenter@linaro.org>
  Dave Hansen <dave.hansen@linux.intel.com>
  Doug Berger <opendmb@gmail.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Huacai Chen <chenhuacai@loongson.cn>
  Jean Delvare <jdelvare@suse.de>
  Linus Torvalds <torvalds@linux-foundation.org>
  Marc Zyngier <maz@kernel.org>
  Masahiro Yamada <masahiroy@kernel.org>
  Mukesh Kumar Savaliya <quic_msavaliy@quicinc.com>
  Nathan Chancellor <nathan@kernel.org>
  Radek Krejci <radek.krejci@oracle.com>
  Steve Wahl <steve.wahl@hpe.com>
  Thomas Gleixner <tglx@linutronix.de>
  Thorsten Blum <thorsten.blum@toblux.com>
  Viken Dadhaniya <quic_vdadhani@quicinc.com>
  Wolfram Sang <wsa@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             fail    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   c02197fc9076..6c160f16be5d  6c160f16be5df1f66f6afe186c961ad446d7f94b -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 05:07:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 05:07:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682670.1061758 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbvsP-0007cD-7w; Mon, 19 Feb 2024 05:07:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682670.1061758; Mon, 19 Feb 2024 05:07:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbvsP-0007c6-5G; Mon, 19 Feb 2024 05:07:37 +0000
Received: by outflank-mailman (input) for mailman id 682670;
 Mon, 19 Feb 2024 05:07:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qfqo=J4=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rbvsO-0007bx-LZ
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 05:07:36 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c913ff20-cee4-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 06:07:31 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d22fa5c822so14889271fa.2
 for <xen-devel@lists.xenproject.org>; Sun, 18 Feb 2024 21:07:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c913ff20-cee4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708319251; x=1708924051; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=R0sVHfEE5Vx1DRn/FiL+e0VQQd5HXq7/o3UibCZsaxg=;
        b=L1/DHDHCZdq0YXy1WwCe/3o92xBX9+/u2Y+9LtoIiK69+JOTJuv8c4SB6W52XFN09a
         uOHZ4lUC4egF2t0RZStUCUlvv8FjqJ38CffOo/2j4yiZ7nKhYBCAy4KXzD39WaKB5CkP
         I6WCX1Fs71T3kst3LLZU2uaZXp4uP+4alpaCs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708319251; x=1708924051;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=R0sVHfEE5Vx1DRn/FiL+e0VQQd5HXq7/o3UibCZsaxg=;
        b=n5jRgiMLAoqOvCQKhNoV6nwxA+3CPDZrAHmKlgddhmk9zv7SV7nZMyyJeeCeD9D5SP
         j0r5wka6jtyEp2vhVyibbkfTp7l8CRWJAgMUzSn5vbt2ChimODco5iMHf6Rpl1v5L4Ky
         0wAT30DvqviX+NRty288+20p9zeCNMS2pQXUNl+dPnw+86k0tAD9DZMO1zsgb9XaPR7O
         aay0Wl5JD8RH9xKsuHCWPHOQ1HER5F6o9iOrcIuUjDOlmMejm6/KSLDGre3xZGkonmbi
         0U/x+QulvlvHbif4ERW4LNLaK+1xnmR3xuqrKAy12r0Yr8sisfaUyNwC0vurlhcHfmCZ
         VBLw==
X-Gm-Message-State: AOJu0Yyaib/O+hf0M8pfeonY+OrfV+lMZw9RYRdcUY+K71vYPcATWVS1
	4KSObssn1HlKqX5aAsEHys/ugtmzsaaOABR18xqkLijl1zuPd7mo8zQxIOnQ8vQc4g5g0Dejr9t
	5bgTA9DKUuMXeqBwAAKfJYZ2gPLxz6c+JIuecag==
X-Google-Smtp-Source: AGHT+IHlH13+kJvSsOetVen/CjY86nBXV4/cJdqxKMy9qYqB0ZAKy9j6FpxUGEMBfHiAjTSG6KBrVyuX9jyF0VUQ43I=
X-Received: by 2002:a2e:9012:0:b0:2d2:3c1a:a41b with SMTP id
 h18-20020a2e9012000000b002d23c1aa41bmr938865ljg.21.1708319251240; Sun, 18 Feb
 2024 21:07:31 -0800 (PST)
MIME-Version: 1.0
References: <20240214103741.16189-1-roger.pau@citrix.com> <20240214103741.16189-6-roger.pau@citrix.com>
In-Reply-To: <20240214103741.16189-6-roger.pau@citrix.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Mon, 19 Feb 2024 13:07:20 +0800
Message-ID: <CA+zSX=ZZBFnsnc+wtrcB3vC3dMHQS=ZG8zjUdiPeCdFZJ12E4w@mail.gmail.com>
Subject: Re: [PATCH 5/5] mm: add the __must_check attribute to {gfn,mfn}_add()
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 14, 2024 at 7:42=E2=80=AFPM Roger Pau Monne <roger.pau@citrix.c=
om> wrote:
>
> It's not obvious from the function itself whether the incremented value w=
ill be
> stored in the parameter, or returned to the caller.  That has leads to bu=
gs in
> the past as callers assume the incremented value is stored in the paramet=
er.
>
> Add the __must_check attribute to the function to easily spot callers tha=
t
> don't consume the returned value, which signals an error in the caller lo=
gic.
>
> No functional change intended.

Just thinking out loud here... I wonder if "mfn_plus()" would be less
likely to be misunderstood.  Catching this during compile is def
better than catching it w/ Coverity or Eclair, but even better to help
people not make the mistake in the first place.

 -George


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 08:06:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 08:06:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682679.1061767 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbyfJ-0002MA-8s; Mon, 19 Feb 2024 08:06:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682679.1061767; Mon, 19 Feb 2024 08:06:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbyfJ-0002M3-6J; Mon, 19 Feb 2024 08:06:17 +0000
Received: by outflank-mailman (input) for mailman id 682679;
 Mon, 19 Feb 2024 08:06:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rbyfH-0002Lx-Tn
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 08:06:15 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bf405c03-cefd-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 09:06:12 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-40fd72f7125so33469855e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 00:06:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 dn23-20020a05600c655700b0040fafd84095sm10449766wmb.41.2024.02.19.00.06.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 00:06:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf405c03-cefd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708329972; x=1708934772; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=k12lEr+9sd7cL8zwVGAmNia+gVMdZzPb0lgTIHsA8ls=;
        b=LLJ4v3MRotltNlpvmgMy+GyV4PWL/mDoExrA/HhFam5VjICRvZkwLSU5xDgcku1K7L
         9QpIvHKzYplpqmmsyu43JKdiRzZ9A5YQPgocxu7p+H3UI1ILWsr55nsWPZCKp3LOREaL
         zuDnC0V3DevV+hkg0o0/oyPrqfzbaCEE0fcoIMrgI2xitrMI3aXqfrEkxjsXIyJfLN0e
         1Fjifdu1ZgTWVcvTda+yIAE+JIC5mufHGoRJyH7H+6QEeRSCxmWmi3kllVhZdVTzOgKQ
         oNAY9c+Un2uShAsQSnJ4FaVjtO7dI0ytiu0p7uiElpWiNSfah+bH3dqAFWID+XgoFhL+
         k9bw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708329972; x=1708934772;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=k12lEr+9sd7cL8zwVGAmNia+gVMdZzPb0lgTIHsA8ls=;
        b=wKcwFNFMuJZT0pJSVmt5TXt19X4TxJIafjxEjWwARi9Z/aVURXqXHAl919OIcNoQe/
         TXg0ykWnLQ36O19pVaiQM4jx0TaWFtBwyK6JxLJFAte9M7Xj8SAp9uvhidUJAXXlSgFA
         Q/C8P5nMUB/c8Sd7N/oVOpwKYcZSfpYTIlldHMRJ3cH/P8fI4D6e+VJ/ToXVuKHT580q
         ZZXdoCHP0Qw1gtXdfXKqy0JcTItwyCtQbhemv3h5VczAOwLJoi2peEaFqGUZSIlV2ZaP
         DAQ/4ufXkkKWEtinR1o3PYXU7rkfN7Tu+mi3fs5ltPPWfIi3Bal8OgHIk3ivZ5hN0YWA
         s6Dg==
X-Forwarded-Encrypted: i=1; AJvYcCVKEp0RIA/xOkK2IlP1Tw+fr5tf6VZTgECqneNBkAaV8/L2v9e1oINdPBLgs1zQee3+/ox9AeHwB1ROHqf5nXw/afPpNwLKQoV50B/XUDA=
X-Gm-Message-State: AOJu0YypYqcqtFD3YyraAkH10FKZJ7WGEZYGmqQZvGedd28GIuV1F22w
	Lx59XRWnNUL8idYhbl5t+PyCtAJRQf6mmdTfJuw78nUE8bmoHKStMXLKZ3wPTg==
X-Google-Smtp-Source: AGHT+IFL1vAGJHdMrbZuvYW54iJj+EGbU3OLIGgm4eSYQaJ86l9WrCtHKLbO4y0ABgBEuqgEXCeqwQ==
X-Received: by 2002:a05:600c:3ba5:b0:40f:f069:8d13 with SMTP id n37-20020a05600c3ba500b0040ff0698d13mr8136392wms.9.1708329972651;
        Mon, 19 Feb 2024 00:06:12 -0800 (PST)
Message-ID: <5de8f721-461c-4a0e-a11d-63aa7c93a742@suse.com>
Date: Mon, 19 Feb 2024 09:06:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 25/30] xen/riscv: add minimal stuff to processor.h to
 build full Xen
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <9efbc232f64b6192cf83f865b8987846fe082720.1707146506.git.oleksii.kurochko@gmail.com>
 <6be5102a-624c-463a-9821-c618d110ce7a@suse.com>
 <095b8031eaaa5324cdae9fee75f9521a795feb46.camel@gmail.com>
 <2f1e4d2d-5b33-47ff-912b-c4693744d0e9@suse.com>
 <0bafef389b30251bc9898bb61604aa3efaabe48c.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <0bafef389b30251bc9898bb61604aa3efaabe48c.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.02.2024 12:16, Oleksii wrote:
> On Thu, 2024-02-15 at 17:43 +0100, Jan Beulich wrote:
>> On 15.02.2024 17:38, Oleksii wrote:
>>> On Tue, 2024-02-13 at 14:33 +0100, Jan Beulich wrote:
>>>> On 05.02.2024 16:32, Oleksii Kurochko wrote:
>>>>> +	depends on LLD_VERSION >= 150000 || LD_VERSION >=
>>>>> 23600
>>>>
>>>> What's the linker dependency here? Depending on the answer I
>>>> might
>>>> further
>>>> ask why "TOOLCHAIN" when elsewhere we use CC_HAS_ or HAS_CC_ or
>>>> HAS_AS_.
>>> I missed to introduce {L}LLD_VERSION config. It should output from
>>> the
>>> command:
>>>   riscv64-linux-gnu-ld --version
>>
>> Doesn't answer my question though where the linker version matters
>> here.
> Then I misinterpreted your initial question.
> Could you please provide further clarification or rephrase it for
> better understanding?
> 
> Probably, your question was about why linker dependency is needed here,
> then
> it is not sufficient to check if a toolchain supports a particular  
> extension without checking if the linker supports that extension   
> too.
> For example, Clang 15 supports Zihintpause but GNU bintutils
> 2.35.2 does not, leading build errors like so:
>     
>    riscv64-linux-gnu-ld: -march=rv64i_zihintpause2p0: Invalid or
>    unknown z ISA extension: 'zihintpause'

Hmm, that's certainly "interesting" behavior of the RISC-V linker. Yet
isn't the linker capability expected to be tied to that of gas? I would
find it far more natural if a gas dependency existed here. If such a
connection cannot be taken for granted, I'm pretty sure you'd need to
probe both then anyway.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 08:07:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 08:07:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682684.1061778 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbygN-0002tL-LW; Mon, 19 Feb 2024 08:07:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682684.1061778; Mon, 19 Feb 2024 08:07:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbygN-0002tE-IY; Mon, 19 Feb 2024 08:07:23 +0000
Received: by outflank-mailman (input) for mailman id 682684;
 Mon, 19 Feb 2024 08:07:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rbygM-0002t8-EH
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 08:07:22 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e851d994-cefd-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 09:07:21 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-5129e5b8cecso2258253e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 00:07:21 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 dn23-20020a05600c655700b0040fafd84095sm10449766wmb.41.2024.02.19.00.07.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 00:07:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e851d994-cefd-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708330041; x=1708934841; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cbHN8iehOY44UGITfjPuxpNYlsb4ToDDW5BrYCEfMmg=;
        b=AxPuYZr0PY94eW+aPjq3dxSmdnwn7NU2obos+mAuHD38e/Im1619DnzJOS9be7JCAo
         IF1P0imq4QsO0Mez5XS2g2RD2rC0+IS1iOgFRGYb+dLlsn4r6RQZCk0V1FW0oKlj0577
         x5bXAeer8978IyTmLUFDOb7olk01cHYMuGQ/aqP92wS5G1kA2ZJ3Yi903rpU16UVj662
         h5l/N/SQPXEFk9DNJJ4AMIw7RkonjcuYsH+2avhx2hXfCeCcY8IWrKPNw0BQlvFOOPb+
         SxaftNf6Cv7II+b76IIJCUy3icNEOp8k9tV52nPtMxuy/HRKswnLCQtUJ/w85Y54faTR
         YnbA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708330041; x=1708934841;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cbHN8iehOY44UGITfjPuxpNYlsb4ToDDW5BrYCEfMmg=;
        b=JXxiJzWUfHgYt0MJVzQXeS1c+8i7kXOCoq09nWW7wh0PCq7/otEs+nk+D7f7wmZY6H
         v6WouzMT0YrEpWQYqZOO+MSCUDn3Ks/42rS7T50pNUTZrhmvN3AK4rlAowMPfNkVzVKu
         BkjBQL8OGo0gUy6opbWX57N3mrBF38J9+mTT0JCzZDXSReBcdF9FuK1P/tEu12/DhJad
         X0H31aT6n5SkwVtz8czK7x6zpHiSvb7Gcyy4n15pccpJxoq8EuHYYux+g+ElfA6YkxEl
         j/cisiuBqjAw74tRCCHPT2+6fXh7B4J/fB0aJXN9mb0NUuemMI+Qap84JMkzeptkTCW6
         Zaeg==
X-Forwarded-Encrypted: i=1; AJvYcCWZLOdPvRVbsSb6sM5Lawl7kQBJqoGZH/1B7ALfwT3tI9j8JhAiBNpTnVX/ccA3J0ixI5rZj/m2PYj6A5eLP8UAHdIO+dThflmX1ejMrJQ=
X-Gm-Message-State: AOJu0YxGMRNbRrZrfoK/xAhx+kPNFg84cE2/OZ3+NcvwKhVNq5l9sSEg
	nconTody1kKqZg7ozHAV5G0IriCiHM7m8u8PDGSPSbyl/S7K6j8NKal7LT7blg==
X-Google-Smtp-Source: AGHT+IFp6Ls6InEyqnx8mGNRGtMKeRWnj/TuRq9D92L9hV/ztymQXfckf+FeyFe+NNrjwO1QVBClZQ==
X-Received: by 2002:a05:6512:234c:b0:512:a8e1:abd1 with SMTP id p12-20020a056512234c00b00512a8e1abd1mr3805101lfu.23.1708330041029;
        Mon, 19 Feb 2024 00:07:21 -0800 (PST)
Message-ID: <a6ee0e99-0df0-40d0-84df-ab62a8dab8a6@suse.com>
Date: Mon, 19 Feb 2024 09:07:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 26/30] xen/riscv: add minimal stuff to mm.h to build
 full Xen
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <369fe9e80d11c5e499810a6819fc1c282ce60628.1707146506.git.oleksii.kurochko@gmail.com>
 <6f5e7e21-067e-4323-88bc-f9b5975e95c5@suse.com>
 <6339742ea1a26c9808b108a1282e521dcf0fc9d0.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <6339742ea1a26c9808b108a1282e521dcf0fc9d0.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.02.2024 12:03, Oleksii wrote:
>>
>>> +        } free;
>>> +    } u;
>>> +
>>> +    union {
>>> +        /* Page is in use, but not as a shadow. */
>>
>> I'm also pretty sure I asked before what shadow this comment alludes
>> to.
> I missed your request about 'shadow' before.
> 
> The comment arrived from Arm.
> 
> I tried to find out the answer by investigation how 'inuse' is used,
> and, unfortunately, I couldn't find an answer what 'shadow' alludes to.

That's from x86'es shadow paging, where a page can serve as a "shadow" of
a guest page table page.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 08:35:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 08:35:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682693.1061787 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbz7t-0006vS-RB; Mon, 19 Feb 2024 08:35:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682693.1061787; Mon, 19 Feb 2024 08:35:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbz7t-0006vL-Of; Mon, 19 Feb 2024 08:35:49 +0000
Received: by outflank-mailman (input) for mailman id 682693;
 Mon, 19 Feb 2024 08:35:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NiMn=J4=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rbz7s-0006vF-8s
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 08:35:48 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e0cf6369-cf01-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 09:35:47 +0100 (CET)
Received: from [192.168.1.229] (93-36-216-194.ip62.fastwebnet.it
 [93.36.216.194])
 by support.bugseng.com (Postfix) with ESMTPSA id 612194EE0739;
 Mon, 19 Feb 2024 09:35:46 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0cf6369-cf01-11ee-8a52-1f161083a0e0
Message-ID: <56b4a4bb-d3e8-4a21-8c0d-d4ee10c8f9b3@bugseng.com>
Date: Mon, 19 Feb 2024 09:35:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org,
 Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <ec849d3a-4f6a-4afd-a7c4-418906eb909a@suse.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
 <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
 <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
 <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com>
 <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
 <45c19a87-5833-4350-a374-3a70fd3aa9ae@suse.com>
 <d735e980-c5a5-46b1-a511-cc2d666dac23@bugseng.com>
 <123c446c-e56f-457e-9cf4-58a43a3b069a@suse.com>
Content-Language: en-US, it
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <123c446c-e56f-457e-9cf4-58a43a3b069a@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 15/02/24 11:32, Jan Beulich wrote:
> The important difference is: Here we're told that there was a use of
> __put_user_bad, which is easy to grep for, and thus see how the
> supplied function / file / line(?) relate to the ultimate problem.
> 
> I'm afraid I'm meanwhile confused enough by the various replies
> containing results of experimentation that I can't really tell
> anymore what case is best. Hence I can only restate my expectation for
> an eventual v3: Diagnosing what the issue is, no matter whether the new
> macro is used in another macro or in an inline function, should not
> become meaningfully more difficult. In how far this is the case wants
> clarifying in the description of the change.

I think the best thing at the moment is to deviate
__{get,put}_user_bad() for Rule 16.3.
I'll let maintainers further explore the possibility of having a
compile-time assertion based on the assembler error.

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 08:36:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 08:36:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682694.1061797 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbz8B-0007HG-2d; Mon, 19 Feb 2024 08:36:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682694.1061797; Mon, 19 Feb 2024 08:36:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbz8A-0007H3-WF; Mon, 19 Feb 2024 08:36:07 +0000
Received: by outflank-mailman (input) for mailman id 682694;
 Mon, 19 Feb 2024 08:36:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rbz89-0007GX-S3
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 08:36:05 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ea8863a4-cf01-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 09:36:03 +0100 (CET)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-50eac018059so5692809e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 00:36:03 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 u6-20020a05600c19c600b00412572f5175sm7537277wmq.23.2024.02.19.00.36.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 00:36:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea8863a4-cf01-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708331763; x=1708936563; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rq+/CIZ7YDFXLRD5xeoGENi7ZdsPDuO+JPh6bmdQqoM=;
        b=cTbDTmrSHSZAZY7ww8Mn8iEYTvEPp/hvKuvKOrTfaYtRkvd10TAdYak884HkIitGxH
         SAX26BirLKCFH7wtQDJxx3eFhs9YJQHQv3YI5TD2AAoddVD7T+j0K+5L+YGT1Nl7+Oja
         gP6XydNrJry3dMBpOq+uE9ebvqqQGqDmh5s6c80uS4WasU3kYfK0JS5qVt/mbrvYa5eQ
         uWkl/h7fwK349hXFpt2+n908BRxo8VhqXUdd2NcZheCp7RmdMb+GX8UbsxIULlpt8Bot
         8cTxAHR9UIoBcIhaB+2Vlgm7wK1NHTn4nwXYrZQnEAkepg4AprkAROQQXRSjsU9cNKot
         Zk3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708331763; x=1708936563;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rq+/CIZ7YDFXLRD5xeoGENi7ZdsPDuO+JPh6bmdQqoM=;
        b=CMeBqF6Pa04vlf6Mx8Z09Ea6gvviPle8lJNIIb/mF+5EIuMaBugELPDo/RBamaUhNu
         hPNwOd2dCAFF59p4pVRUignJft+oOJnHh+S8W8sz3mev0fNVaePrZLbRk58xyt8XWD98
         uQpYNNfe0Gu/o4+pjKK14it0s7AjdZqQUaGpMrAs+rxhJjjYw/rXT7tXZcmWbeYMpfrI
         9Z9Dob95+ZhUPWUz/QfSpYeCU1NflR29vY7wibIYaB8ccYN2ydLWHv6uC/qTR1CKvBQO
         +b+G+EySypyksMPk3aWHNK1i++sxqw94I8RROSR+ZdKxAdPvmowP7XLFS+v3aA5/nJ/3
         boPw==
X-Forwarded-Encrypted: i=1; AJvYcCWA/T76m3102HbCHB3JRepuRmIa+7X9AuEIg32vYVZFyaUzxqmcqaIcjxChOu1HEW+jF5YJebVVb7LXWARh3u2aptQa+zVOJBRYT3hAslY=
X-Gm-Message-State: AOJu0Ywtm8iqrGnUiPfyeHg7ZwIs9teVJit24knK+Ox6JvMCf5pBYJu3
	j4jabh5/FAcY+gOqCTCl43o2hUpyQS9BeoRERcYwEDqP2ERmNzt5Ia8XVWL0NQ==
X-Google-Smtp-Source: AGHT+IFYvWXQjajVhWSFDIXVBpUl2rR+jfdNqMbZKjm44Ose4degvssMScZLNDO4kAN949ONOK4B/Q==
X-Received: by 2002:a05:6512:519:b0:511:76ff:32bc with SMTP id o25-20020a056512051900b0051176ff32bcmr7287896lfb.2.1708331763164;
        Mon, 19 Feb 2024 00:36:03 -0800 (PST)
Message-ID: <826829ab-28c1-4138-8ecb-32ec6dbb96e6@suse.com>
Date: Mon, 19 Feb 2024 09:36:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build/xen: fail to rebuild if Kconfig fails
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Anthony Perard <anthony.perard@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240215093002.23527-1-roger.pau@citrix.com>
 <a2edb04f-c343-4baf-9f15-d96c4d014f05@suse.com> <Zc3nXpUOlnIHEfsl@macbook>
 <54678829-4bcf-4d83-8134-1ab386f299b6@suse.com> <Zc3v20RKMssbaDsl@macbook>
 <a48cbad6-701d-4077-9044-4205b932a7f3@suse.com> <Zc428VMDoYnPw1zo@macbook>
 <d545cc6c-d213-43da-af31-1768af32aba0@suse.com> <Zc5Io3dkAlGSt3on@macbook>
 <2fe68b26-87c3-43bf-ba11-f261c81c6373@suse.com> <Zc8-TNOMG7ljNCj1@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zc8-TNOMG7ljNCj1@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.02.2024 11:51, Roger Pau Monné wrote:
> On Fri, Feb 16, 2024 at 11:04:46AM +0100, Jan Beulich wrote:
>> On 15.02.2024 18:23, Roger Pau Monné wrote:
>>> On Thu, Feb 15, 2024 at 05:22:00PM +0100, Jan Beulich wrote:
>>>> On 15.02.2024 17:08, Roger Pau Monné wrote:
>>>>> On Thu, Feb 15, 2024 at 02:02:41PM +0100, Jan Beulich wrote:
>>>>>> --- a/xen/Rules.mk
>>>>>> +++ b/xen/Rules.mk
>>>>>> @@ -15,7 +15,11 @@ srcdir := $(srctree)/$(src)
>>>>>>  PHONY := __build
>>>>>>  __build:
>>>>>>  
>>>>>> --include $(objtree)/include/config/auto.conf
>>>>>> +ifneq ($(obj),tools)
>>>>>> +ifneq ($(obj),tools/kconfig)
>>>>>> +include $(objtree)/include/config/auto.conf
>>>>>> +endif
>>>>>> +endif
>>>>>
>>>>> Trying to understand this, I assume it's to avoid an infinite
>>>>> dependency loop that generating include/config/auto.conf requires some
>>>>> tools that are build using xen/Rules.mk?
>>>>
>>>> The file has dependencies only in xen/Makefile. This is about the
>>>> file simply not being there when initially building. Perhaps the
>>>> patch description helps that I've written in the meantime:
>>>>
>>>> "Because of using "-include", failure to (re)build auto.conf (with
>>>>  auto.conf.cmd produced as a secondary target) won't stop make from
>>>>  continuing the build. Arrange for it being possible to drop the - from
>>>>  Rules.mk, requiring that the include be skipped for tools-only targets.
>>>
>>> Wouldn't it be more reliable if we skipped the include for any paths
>>> in $(obj) that start with 'tools', rather than hardcoding 'tools' and
>>> 'tools/kconfig'?
>>
>> I was first meaning to do so, but the expression would end up more
>> complex than I'd like (for it needing to be an exact match of "tools"
>> and a prefix match of "tools/"). Thinking of it,
>>
>> ifneq ($(obj),tools)
>> ifneq ($(patsubst tools/%,$(obj)),)
>>
>> might do (and not be as complex as I first thought, when intending to
>> put all in a single "if").
> 
> Would something like the rune below work?
> 
> ifneq ($(word 1, $(subst /, ,$(obj))),tools)
> 
> That should allow to have a single condition, and should match both
> 'tools' and 'tools/*'

Hmm, yes, that works. $(subst ...) is something I usually try to avoid,
in favor of $(patsubst ...).

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 08:45:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 08:45:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682701.1061808 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbzHC-0000kM-Vy; Mon, 19 Feb 2024 08:45:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682701.1061808; Mon, 19 Feb 2024 08:45:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbzHC-0000kF-S8; Mon, 19 Feb 2024 08:45:26 +0000
Received: by outflank-mailman (input) for mailman id 682701;
 Mon, 19 Feb 2024 08:45:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rbzHA-0000k6-Pe
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 08:45:24 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 37f443ba-cf03-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 09:45:22 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-4125ea5d913so10151385e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 00:45:22 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w9-20020a05600c474900b00412684fbcf3sm1145258wmo.28.2024.02.19.00.45.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 00:45:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 37f443ba-cf03-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708332322; x=1708937122; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OdP1TeRLgjrOgSBURn92vZtVOatpVuwsVdPCFMn3gb0=;
        b=dRivTgH0mHPc1DSNHeQ39PFdp6SHJk07W7bzNihoZakaJvvkWU5kRe5Waq7Cx7mn/X
         I4OcR47VctQBKpRuJHFp0XEUYFyDQbybPg6ThboL8IXpOyeuc+/ls6WEzEa+o4BlAtC8
         EloHI5J2g+l+KdaG0/sS4/uh5V2N23IMI3qKSmkznJm7TpYio92cc2babA9bkNvDesaZ
         xnSSlf5YM+g7EckCeQ3uUdDIHw3L75eGlAjGR194XOBvqqaEcShT2v0dOWjkBYVrZWti
         6EuNy6a9E9qpvRutcEELhI4576txmQoQy7bPW9ViBWlraiixA5bZHtPLjnytSX+4Wozv
         SY/Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708332322; x=1708937122;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OdP1TeRLgjrOgSBURn92vZtVOatpVuwsVdPCFMn3gb0=;
        b=bPWQJICXMl243Z4t9dfCrMR7pzw/36faYrQH1/L6G86raQeBUToLm3+0o9FF9C1/K/
         3gkU2zCvKJFOjfz6LEQnU/ltfi0dgdw5P0Erdfx/wTIMGaIsyd23ealc6ntFqfWmL5uD
         kz3bIT7kUhVg+ZXchzNkDm/GTsnijTY3Dkb/jxu4sAA09FtbxJuF+OOUc2ZuZR5ipPbo
         VgR27FI5icFPbwrstgtMH2mVjk2y+oFyjqbL9HiEnX04Vp9+vryHclE90eLOlmrNxNlx
         69f8UdqbCEpiT1r/LO+V5nfI+w2UZsjvMeGQipzLthE0NnIqzvnTPC6Oe+PYd+iWI5xE
         OPFA==
X-Gm-Message-State: AOJu0YzT0fsrOVFrknibcgwyubkfE/MslG/VZijVuU3zZVYzKQq662XF
	XJeR+1yZRXBt7ZU5VXNrXqJoKizfwvpC4ywgUSMMb4WSn4PtcX/lmm/UxXybQw==
X-Google-Smtp-Source: AGHT+IE01bjpheAN/hG6nFX8IxfRzDIONvgk+KiZUBUvE+zhDI8kikI92lLfk4tkTWFuSOJYw336eQ==
X-Received: by 2002:a05:600c:1ca1:b0:40e:f9df:3531 with SMTP id k33-20020a05600c1ca100b0040ef9df3531mr8124252wms.8.1708332322111;
        Mon, 19 Feb 2024 00:45:22 -0800 (PST)
Message-ID: <ab45f1e5-ed72-40cf-b485-e3682960587d@suse.com>
Date: Mon, 19 Feb 2024 09:45:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86: amend 'n' debug-key output with SMI count
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <549909b7-e34c-4a5c-aa21-9892a1724042@suse.com>
 <Zc8mzlzLTJNwBJup@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zc8mzlzLTJNwBJup@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 16.02.2024 10:11, Roger Pau Monné wrote:
> On Wed, Feb 14, 2024 at 11:15:51AM +0100, Jan Beulich wrote:
>> --- a/xen/arch/x86/cpu/common.c
>> +++ b/xen/arch/x86/cpu/common.c
>> @@ -407,9 +407,15 @@ void __init early_cpu_init(bool verbose)
>>  		paddr_bits -= (ebx >> 6) & 0x3f;
>>  	}
>>  
>> -	if (!(c->x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON)))
>> +	if (!(c->x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON))) {
>> +		uint64_t smi_count;
>> +
>>  		park_offline_cpus = opt_mce;
>>  
>> +		if (!verbose && !rdmsr_safe(MSR_SMI_COUNT, smi_count))
>> +			setup_force_cpu_cap(X86_FEATURE_SMI_COUNT);
> 
> Why make it dependent on !verbose?  The call with !verbose is tied to
> part of the ucode loading being half-functional (for example
> MCU_CONTROL_DIS_MCU_LOAD not being set) but I don't see that as a
> signal that SMI count shouldn't be used.
> 
> does it need to be part of the early cpu initialization instead of
> being in the (later) Intel specific init code part of the
> identify_cpu()?

Yes, the condition was inverted. It could likely also be dropped
altogether; not sure which one's better: On one hand avoiding multiple
setup_force_cpu_cap() seems desirable (albeit not strictly necessary),
while otoh the code would be simpler without.

>> --- a/xen/arch/x86/nmi.c
>> +++ b/xen/arch/x86/nmi.c
>> @@ -585,15 +585,34 @@ static void cf_check do_nmi_trigger(unsi
>>      self_nmi();
>>  }
>>  
>> +static DEFINE_PER_CPU(unsigned int, smi_count);
>> +
>> +static void cf_check read_smi_count(void *unused)
>> +{
>> +    unsigned int dummy;
>> +
>> +    rdmsr(MSR_SMI_COUNT, this_cpu(smi_count), dummy);
>> +}
>> +
>>  static void cf_check do_nmi_stats(unsigned char key)
>>  {
>>      const struct vcpu *v;
>>      unsigned int cpu;
>>      bool pend, mask;
>>  
>> -    printk("CPU\tNMI\n");
>> +    printk("CPU\tNMI%s\n", boot_cpu_has(X86_FEATURE_SMI_COUNT) ? "\tSMI" : "");
>> +
>> +    if ( boot_cpu_has(X86_FEATURE_SMI_COUNT) )
>> +        on_each_cpu(read_smi_count, NULL, 1);
>> +
>>      for_each_online_cpu ( cpu )
>> -        printk("%3u\t%3u\n", cpu, per_cpu(nmi_count, cpu));
>> +    {
>> +        printk("%3u\t%3u", cpu, per_cpu(nmi_count, cpu));
>> +        if ( boot_cpu_has(X86_FEATURE_SMI_COUNT) )
>> +            printk("\t%3u\n", per_cpu(smi_count, cpu));
>> +        else
>> +            printk("\n");
>> +    }
>>  
>>      if ( !hardware_domain || !(v = domain_vcpu(hardware_domain, 0)) )
>>          return;
> 
> Could you also amend the debug-key help text to mention SMI?

Hmm, I had considered that and decided against. I'm uncertain, nevertheless,
so could be talked into amending that help text. Just that I can't make it
"NMI and SMI statistics" as whether SMI data is available is conditional.
Yet "NMI (and maybe SMI) statistics" looks a little clumsy to me ...

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 08:46:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 08:46:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682707.1061818 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbzIV-0001Fl-9B; Mon, 19 Feb 2024 08:46:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682707.1061818; Mon, 19 Feb 2024 08:46:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbzIV-0001Fe-6c; Mon, 19 Feb 2024 08:46:47 +0000
Received: by outflank-mailman (input) for mailman id 682707;
 Mon, 19 Feb 2024 08:46:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+22N=J4=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rbzIU-0001FY-Hq
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 08:46:46 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 68ace331-cf03-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 09:46:44 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-4126a159fa6so843505e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 00:46:44 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 dw5-20020a0560000dc500b0033b278cf5fesm9618991wrb.102.2024.02.19.00.46.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 19 Feb 2024 00:46:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68ace331-cf03-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708332404; x=1708937204; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=/oqXWhZsxZL0QLU80NygwO8q2mduUCkKazGcnk8V+Qk=;
        b=Bygf3Q+cIrX7VvKUNImsBNbENR3FI3ng4wkIbcyZNyHDpStcg+uPTNUj8/XCkkNljm
         IH/QWksCAKS9aSAB1GbipglnkPJVWAOHq9Lpa8gNarTws7o04NCy5JuQg626RWYw257O
         dbbOlFH5C1nfp5YlBe+rEkxDRuJ5rlB2Dz/14=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708332404; x=1708937204;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/oqXWhZsxZL0QLU80NygwO8q2mduUCkKazGcnk8V+Qk=;
        b=sF5OR5ctguE0szhoLGT4RUXL+G1kGGgNK9QWaIFOspu3F5GuMvOP9w12WStVnEsVvR
         0uZE9VYYp1mtsdLMCUm9fDH0yI7PO9WrhF/Nsbxh/LbxbB2YSKJGPNzeal40dTllRYzi
         l1ZZLUgxJQEsdKtQP7tAaSGnnKFW/3HSsedF2mkI5HJJkHNyEX/0iTxt5i1+12/cOOfs
         Gs/0zp7maf6IS8u9/1focqAh7NQJCNeKDgnzKvI7KrVaUB8Wwpzpyts5ctPwD/CKNoFj
         vAxCyyLB4yMYlzm/kS0qXd52+H33HDO3VXblp1xguehxp1irVk7ZqKIOjCtPfNH5gVgb
         Dmbw==
X-Gm-Message-State: AOJu0YziB0vssTW++YFAFol6Pnu+/Asdt7IKd4tA227OnvWVdEOiw73j
	daGOjxvLWSxmNunuQBzmGQFu7rk8mL1dU8w+x0PoKEBlPQr5Hkoh0XyxR8+R81M=
X-Google-Smtp-Source: AGHT+IExeSayAXedpNqAnQ6g+aQ5E10KI0NqNSgNpOXRx7nUAP6uuZ6S7r9+/jZ0EXNLOlApFws+5w==
X-Received: by 2002:a5d:6e8b:0:b0:33d:150a:307f with SMTP id k11-20020a5d6e8b000000b0033d150a307fmr5669853wrz.64.1708332403883;
        Mon, 19 Feb 2024 00:46:43 -0800 (PST)
Date: Mon, 19 Feb 2024 09:46:42 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: Re: [PATCH v13.1 01/14] vpci: use per-domain PCI lock to protect
 vpci structure
Message-ID: <ZdMVctHtAeujcNq5@macbook>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <20240215203001.1816811-1-stewart.hildebrand@amd.com>
 <Zc9KuCeoOciUdqTN@macbook>
 <f1e43851-8a3d-423d-b0da-4d881f8ecf91@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <f1e43851-8a3d-423d-b0da-4d881f8ecf91@amd.com>

On Fri, Feb 16, 2024 at 09:41:19AM -0500, Stewart Hildebrand wrote:
> On 2/16/24 06:44, Roger Pau Monné wrote:
> > On Thu, Feb 15, 2024 at 03:30:00PM -0500, Stewart Hildebrand wrote:
> >> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> >>
> >> Use the per-domain PCI read/write lock to protect the presence of the
> >> pci device vpci field. This lock can be used (and in a few cases is used
> >> right away) so that vpci removal can be performed while holding the lock
> >> in write mode. Previously such removal could race with vpci_read for
> >> example.
> >>
> >> When taking both d->pci_lock and pdev->vpci->lock, they should be
> >> taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
> >> possible deadlock situations.
> >>
> >> 1. Per-domain's pci_lock is used to protect pdev->vpci structure
> >> from being removed.
> >>
> >> 2. Writing the command register and ROM BAR register may trigger
> >> modify_bars to run, which in turn may access multiple pdevs while
> >> checking for the existing BAR's overlap. The overlapping check, if
> >> done under the read lock, requires vpci->lock to be acquired on both
> >> devices being compared, which may produce a deadlock. It is not
> >> possible to upgrade read lock to write lock in such a case. So, in
> >> order to prevent the deadlock, use d->pci_lock in write mode instead.
> >>
> >> All other code, which doesn't lead to pdev->vpci destruction and does
> >> not access multiple pdevs at the same time, can still use a
> >> combination of the read lock and pdev->vpci->lock.
> >>
> >> 3. Drop const qualifier where the new rwlock is used and this is
> >> appropriate.
> >>
> >> 4. Do not call process_pending_softirqs with any locks held. For that
> >> unlock prior the call and re-acquire the locks after. After
> >> re-acquiring the lock there is no need to check if pdev->vpci exists:
> >>  - in apply_map because of the context it is called (no race condition
> >>    possible)
> >>  - for MSI/MSI-X debug code because it is called at the end of
> >>    pdev->vpci access and no further access to pdev->vpci is made
> >>
> >> 5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
> >> while accessing pdevs in vpci code.
> >>
> >> 6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
> >> do not go away. The vPCI functions call several MSI-related functions
> >> which already have existing non-vPCI callers. Change those MSI-related
> >> functions to allow using either pcidevs_lock() or d->pci_lock for
> >> ensuring pdevs do not go away. Holding d->pci_lock in read mode is
> >> sufficient. Note that this pdev protection mechanism does not protect
> >> other state or critical sections. These MSI-related functions already
> >> have other race condition and state protection mechanims (e.g.
> >> d->event_lock and msixtbl RCU), so we deduce that the use of the global
> >> pcidevs_lock() is to ensure that pdevs do not go away.
> >>
> >> 7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
> >> that pdevs do not go away. The purpose of this wrapper is to aid
> >> readability and document the intent of the pdev protection mechanism.
> >>
> >> 8. When possible, the existing non-vPCI callers of these MSI-related
> >> functions haven't been switched to use the newly introduced per-domain
> >> pci_lock, and will continue to use the global pcidevs_lock(). This is
> >> done to reduce the risk of the new locking scheme introducing
> >> regressions. Those users will be adjusted in due time. One exception
> >> is where the pcidevs_lock() in allocate_and_map_msi_pirq() is moved to
> >> the caller, physdev_map_pirq(): this instance is switched to
> >> read_lock(&d->pci_lock) right away.
> >>
> >> Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
> >> Suggested-by: Jan Beulich <jbeulich@suse.com>
> >> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> >> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> >> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> > 
> > A couple of questions and the pdev_list_is_read_locked() needs a small
> > adjustment.
> > 
> >> @@ -895,6 +891,14 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
> >>  {
> >>      unsigned int i;
> >>  
> >> +    /*
> >> +     * Assert that d->pdev_list doesn't change. pdev_list_is_read_locked() is
> >> +     * not suitable here because we may read_unlock(&pdev->domain->pci_lock)
> >> +     * before returning.
> > 
> > I'm confused by this comment, as I don't see why it matters that the
> > lock might be lock before returning.  We need to ensure the lock is
> > taken at the time of the assert, and hence pdev_list_is_read_locked()
> > can be used.
> 
> pdev_list_is_read_locked() currently would allow either pcidevs_lock()
> or d->pci_lock. If vpci_msix_arch_print() is entered with only
> pcidevs_lock() held, we may end up unlocking d->pci_lock when it is
> not locked, which would be wrong.
> 
> Perhaps the comment could be clarified as:
> 
>     /*
>      * Assert that d->pdev_list doesn't change. ASSERT_PDEV_LIST_IS_READ_LOCKED
>      * is not suitable here because it may allow either pcidevs_lock() or
>      * d->pci_lock to be held, but here we rely on d->pci_lock being held, not
>      * pcidevs_lock().
>      */

Yes, this is indeed better.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 08:50:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 08:50:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682711.1061827 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbzMF-000371-Qs; Mon, 19 Feb 2024 08:50:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682711.1061827; Mon, 19 Feb 2024 08:50:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbzMF-00036u-OD; Mon, 19 Feb 2024 08:50:39 +0000
Received: by outflank-mailman (input) for mailman id 682711;
 Mon, 19 Feb 2024 08:50:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rbzMF-00036m-3r
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 08:50:39 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f3f49ec0-cf03-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 09:50:38 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-4126126f4f4so8019355e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 00:50:38 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 k10-20020a7bc40a000000b004101f27737asm10590519wmi.29.2024.02.19.00.50.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 00:50:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f3f49ec0-cf03-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708332637; x=1708937437; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0zywQZWYhohz8TBjhUWm/mnYAf+BD4wOyOCATdZphKw=;
        b=fb3lziAfUqX+SzHhxnpAPG4UK+Lq+VcGrRAdvU8NiQ+DkXvn9abnLpZuyACa8xwBWP
         4mT/fCv4B3dzaMvQ2vT8yoUAzacQwdqmEekQ/mQ4XIBhHQVVXVtJBRtnN8G4Rk8+88si
         +k+abehnaZiT1UBrlS6F4y5NrNJlLWNRHhFFkNPm/iK/k5q14Hi2/kRscb5LuA137GId
         6VLcf0xCfWZ6vYcBXV6Nzq6LdiA+81sAHkR5fK7VqLAzCOdkmNjN+ZkaKn4/FsN6bQmB
         lbRkqzd7p90dKLg3eTkVwynLajSebcbfNTATYqa/qKIpxsKVm7lAk0fQdnozG/tv3q87
         7REA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708332637; x=1708937437;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0zywQZWYhohz8TBjhUWm/mnYAf+BD4wOyOCATdZphKw=;
        b=WuvIXVkA3FuTjuQfEPKFLjP4+xM372002FWT4mCOcm+A5k3EO91ZptB0LQ2s0N7XXC
         Mka3+9BP/0kDUG8P4jMJeDqb7Hd70ZExlfe7F0j9pbBJVUtrErS2KpxMjXDaljHBSkdQ
         N42i8iMEv3//7tIcAkteHjP+Hfzeyvp5Xt+gYgR7Q18IKvMYHKkea4Hmy3FUXAeOKCtu
         LMzQ73JUPXCtc4idinZeX8/lMj5VyiiP+4a11ZG69vbmMGz5mDkTzzBfxK8qrdHk0xIA
         sCxPfmgwYsxfNqd9n77MAQmAEW0e0JcBhvmjdOvQglR5Lhrvfpqo2J3FvEgAmLpm5KR3
         r41Q==
X-Gm-Message-State: AOJu0YyNpp5Apc1GyMEbEng1bL5ac0uWgU5iSDYIK4ADRh/A4pOYUjCH
	kr3LCX2X5F27FeEgpwPN1HCnmuaLnfsqGhW9qgDspGS8Ay9Lm2To4N7+ZHiupA==
X-Google-Smtp-Source: AGHT+IEmMOeDgwSZj3suCYP5BYaJn1fgNrLyKZwvdcVEUtmPcYKi4Jt8ZKX/mDZ1BamIkXU0xYYoyA==
X-Received: by 2002:a05:600c:4fc3:b0:411:a70e:3d20 with SMTP id o3-20020a05600c4fc300b00411a70e3d20mr8674827wmq.1.1708332637641;
        Mon, 19 Feb 2024 00:50:37 -0800 (PST)
Message-ID: <9710b68e-80e2-4cac-9f6e-d52a777fb9a1@suse.com>
Date: Mon, 19 Feb 2024 09:50:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/5] mm: add the __must_check attribute to {gfn,mfn}_add()
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Roger Pau Monne <roger.pau@citrix.com>
References: <20240214103741.16189-1-roger.pau@citrix.com>
 <20240214103741.16189-6-roger.pau@citrix.com>
 <CA+zSX=ZZBFnsnc+wtrcB3vC3dMHQS=ZG8zjUdiPeCdFZJ12E4w@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CA+zSX=ZZBFnsnc+wtrcB3vC3dMHQS=ZG8zjUdiPeCdFZJ12E4w@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 19.02.2024 06:07, George Dunlap wrote:
> On Wed, Feb 14, 2024 at 7:42 PM Roger Pau Monne <roger.pau@citrix.com> wrote:
>>
>> It's not obvious from the function itself whether the incremented value will be
>> stored in the parameter, or returned to the caller.  That has leads to bugs in
>> the past as callers assume the incremented value is stored in the parameter.
>>
>> Add the __must_check attribute to the function to easily spot callers that
>> don't consume the returned value, which signals an error in the caller logic.
>>
>> No functional change intended.
> 
> Just thinking out loud here... I wonder if "mfn_plus()" would be less
> likely to be misunderstood.  Catching this during compile is def
> better than catching it w/ Coverity or Eclair, but even better to help
> people not make the mistake in the first place.

To me while mfn_plus() would indeed be somewhat less likely to cause this
kind of confusion, I don't think that naming would entirely eliminate the
risk.

I was actually considering to add mfn_inc() for the common case of
incrementing by 1, yet I can't see an alternative name avoiding the issue
there.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 08:57:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 08:57:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682719.1061839 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbzTE-0003j6-JL; Mon, 19 Feb 2024 08:57:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682719.1061839; Mon, 19 Feb 2024 08:57:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbzTE-0003iz-Ey; Mon, 19 Feb 2024 08:57:52 +0000
Received: by outflank-mailman (input) for mailman id 682719;
 Mon, 19 Feb 2024 08:57:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rbzTC-0003ib-Uf
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 08:57:50 +0000
Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com
 [2a00:1450:4864:20::42c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f4c197e1-cf04-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 09:57:49 +0100 (CET)
Received: by mail-wr1-x42c.google.com with SMTP id
 ffacd0b85a97d-33d118a181fso1903019f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 00:57:49 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 x11-20020a5d444b000000b0033b7ce8b496sm9769110wrr.108.2024.02.19.00.57.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 00:57:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f4c197e1-cf04-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708333068; x=1708937868; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4yn4H6aPiMGY16QdrMOmZtVckBaVP2SonOO8ojF14Kc=;
        b=UrdygiqG5HIU9z1kSIOhp6l2xSWr0mZb6U2HtJO5ZxuK05EODAJgZuYR6PweCPmK7y
         y6nCduRmrYgyXoupOfEhgxq4KAxwHH5niWHM6QRcNJjoLnRlw5lRCM7S5oVjRehJt1ha
         pI5OJ7N850PKCKeujeoPZnkSUumesGVvOp8efWY1oPxaFHMgxPMKSk30aYoVmwb4sy42
         tLbG8yuiS96knkKPFKk10LHmN2QOD7LxRZl2bGOx4QEXeeRDxEWSCUYgFehD9+FxnXkW
         14VsCYZNbYskASmx6RvnW97+fpEMC946IzjES7tJTJgdr81ArJWTKcuqn2CR5jdO18Wf
         tLxw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708333068; x=1708937868;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=4yn4H6aPiMGY16QdrMOmZtVckBaVP2SonOO8ojF14Kc=;
        b=Xp9Bszqm/j7uKDdVjjdOv++1abaF9YzW9wScrLI5gjeMTDfy6Wq3G5VSPcENXi+z0P
         zKj5pW1p37XDsHxe09kQ2qlXaQPmeknq8HxIob2vi1WA8Rq7aWtvo2Pim2ZL43H8C67C
         +4qQ2PYoopI3P34y5le8XA2RYSu0J4+tLd4y+zhHjB47oIoDrrv/zri8BJv1leXNitBw
         f4rXTOqXf0wl2dk6YhS04FlTyUpk8oavurmene7IQaivaaTWTFqVJ0utGzl172BE7ijM
         VB+9A5CPkVYjCL00SOq5gZZJXBTflXa6UFVmg7uotNU+Harj9k1IrQ9r9pQNOsKgLnJK
         JTTQ==
X-Forwarded-Encrypted: i=1; AJvYcCVEOqCeybw5v4dg57n0cdyAS1+RZVAQAZcjLIBzkWpCBAvkHRFs8VEmjXya7r334kTH+PUULfUFA5e+dfIUqjKkOmFl4WCmknCY9r4GDu8=
X-Gm-Message-State: AOJu0YwbibfNtvs99SE3uOj9IZgNdHxXiveC+MoBJBjt3lXS8yx4GyP/
	ORBUE5+euAryyYmSC4Mj0Z19iWBpZILsmYHvvg359g6uQFENR+oFdoB3nogUuA==
X-Google-Smtp-Source: AGHT+IF8w6LgX6xYXh59rplpnMBlDS2InNxXbob7pjajh4m8/rFu/4FjSZINiW/vryNEAdSwCzrutw==
X-Received: by 2002:a5d:5608:0:b0:33d:513a:c7b2 with SMTP id l8-20020a5d5608000000b0033d513ac7b2mr955667wrv.47.1708333068493;
        Mon, 19 Feb 2024 00:57:48 -0800 (PST)
Message-ID: <7b60cf83-4d08-46ef-a937-911e42fd9e5d@suse.com>
Date: Mon, 19 Feb 2024 09:57:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] ns16550: add Asix AX99100 serial card
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240218013431.274451-1-marmarek@invisiblethingslab.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240218013431.274451-1-marmarek@invisiblethingslab.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 18.02.2024 02:34, Marek Marczykowski-Górecki wrote:
> @@ -1170,6 +1177,11 @@ static const struct ns16550_config __initconst uart_config[] =
>          .dev_id = 0x7adc,
>          .param = param_intel_lpss
>      },
> +    {
> +        .vendor_id = PCI_VENDOR_ID_ASIX,
> +        .dev_id = 9100,

As per Linux this is 0x9100.

> +        .param = param_asix_ax99100
> +    },
>  };
>  
>  static int __init
> diff --git a/xen/include/xen/pci_ids.h b/xen/include/xen/pci_ids.h
> index e798477a7e23..2a19f4ab7872 100644
> --- a/xen/include/xen/pci_ids.h
> +++ b/xen/include/xen/pci_ids.h
> @@ -11,3 +11,5 @@
>  #define PCI_VENDOR_ID_BROADCOM           0x14e4
>  
>  #define PCI_VENDOR_ID_INTEL              0x8086
> +
> +#define PCI_VENDOR_ID_ASIX               0x125b

Please insert such that numeric sorting is retained.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 09:05:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 09:05:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682724.1061848 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbza6-0005fD-8t; Mon, 19 Feb 2024 09:04:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682724.1061848; Mon, 19 Feb 2024 09:04:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbza6-0005f6-5a; Mon, 19 Feb 2024 09:04:58 +0000
Received: by outflank-mailman (input) for mailman id 682724;
 Mon, 19 Feb 2024 09:04:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rbza4-0005f0-EQ
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 09:04:56 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f2e3d09a-cf05-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 10:04:55 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-512b13bf764so862010e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 01:04:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m8-20020a05600c4f4800b00411fb769583sm10793802wmq.27.2024.02.19.01.04.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 01:04:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2e3d09a-cf05-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708333495; x=1708938295; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=QHf+Vm3ybdBP7o7l3iGkjjLO2k//up9DJktTuwI94xM=;
        b=aqo/w9QD+SJBKpw2Jkh/yLZgf0kETT89XVoUDvUvWo1XNVVsujYpX8WOmTyAHXD3pN
         uS53Tp40xRcbFsnTxIheWqo0aa7rPfrOF/1sWHLhP1jaXof/4OiJNo8TUuCX42fcIkH+
         Esbvg2+tLG+vFARFM0gV46IC0ED8A+E3iG2mHxi8p2Konl6hhDEAYLQPOUGFcoeNKvC4
         t0QilX5qZ6mh4Ij6W8Wmtvp5/o5l4RlMowakFq+XKYpBhSd482Ttsndd+qT+yfWXKfan
         BQkLkMq89hYmXu5rvDXvsmf3u3Tb3UnwNCxjY0aD2Q5o8gz4OlFYTQq61QdXp4DsMgyF
         LUuQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708333495; x=1708938295;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=QHf+Vm3ybdBP7o7l3iGkjjLO2k//up9DJktTuwI94xM=;
        b=QvIYKZ8IRx1d2J/P7mtMUyGFM1rli7LIaHDRGsP6i4+395YOSkAPREDDbzzMKoWLLZ
         P4d49VlH1BLzFPJrTyuT6sSO72O5sIf+BwrF5QZrqpOSRq5TQTcY+l/4k+wW8+pk90Db
         Kj4AtK5n4J2pXpus5493RZvCo5Hhk3+APvXxTn0BlZ5KufperTp3WCbL2SFYPl8oxJz2
         Lt2zJc6TT5Qtk0gG++nC52KQ5NeswA6PUA9GJCysyIbDX6joWlYTinclbkxcutPd1f6I
         MTivqQ00q5/zU9f0ngWPqzgGSlX0D2uwvM4iyEUlsX7K1EN/ybsqd1Jvuz3e5KrgQ3bj
         tOHQ==
X-Forwarded-Encrypted: i=1; AJvYcCWd2r0lP9syPgLNjLjZlu/eKonQDEL1Qu35wQqPPH/G2l8dGCwWWiShKmGqD0NYXBdnzmPkS/UDmze18sfOaNM6mRjzi8D6jWDR8S8BgDc=
X-Gm-Message-State: AOJu0YzFPo+7Gw3OCjPfPMrXK1O3MQoBmTuqQoTOTrzqjx2SHzmCg1OE
	8ANA0YDXO+AKkpeK/nX3yROA3gH8eA8qZ+F/NSnVHopZMexbdr5PnNMge0glGg==
X-Google-Smtp-Source: AGHT+IGV0Xc8478b3yoQVK5bYdF55An23GHPjzwZgLvbJP1ZCll3fDS81HijftNJ3+/VnwaErbudtg==
X-Received: by 2002:a05:6512:200e:b0:512:8dad:2918 with SMTP id a14-20020a056512200e00b005128dad2918mr6849934lfb.53.1708333494820;
        Mon, 19 Feb 2024 01:04:54 -0800 (PST)
Message-ID: <d6b6ed42-312b-4b74-9b8e-5e7a04627eee@suse.com>
Date: Mon, 19 Feb 2024 10:04:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] x86/vmx: add support for virtualize SPEC_CTRL
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240215164653.27210-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240215164653.27210-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.02.2024 17:46, Roger Pau Monne wrote:
> The feature is defined in the tertiary exec control, and is available starting
> from Sapphire Rapids and Alder Lake CPUs.
> 
> When enabled, two extra VMCS fields are used: SPEC_CTRL mask and shadow.  Bits
> set in mask are not allowed to be toggled by the guest (either set or clear)
> and the value in the shadow field is the value the guest expects to be in the
> SPEC_CTRL register.
> 
> By using it the hypervisor can force the value of SPEC_CTRL bits behind the
> guest back without having to trap all accesses to SPEC_CTRL, note that no bits
> are forced into the guest as part of this patch.  It also allows getting rid of
> SPEC_CTRL in the guest MSR load list, since the value in the shadow field will
> be loaded by the hardware on vmentry.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon Feb 19 09:19:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 09:19:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682730.1061857 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbznS-0007dT-Du; Mon, 19 Feb 2024 09:18:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682730.1061857; Mon, 19 Feb 2024 09:18:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rbznS-0007dM-B4; Mon, 19 Feb 2024 09:18:46 +0000
Received: by outflank-mailman (input) for mailman id 682730;
 Mon, 19 Feb 2024 09:18:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rbznQ-0007dG-SM
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 09:18:44 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id df48d019-cf07-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 10:18:41 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-41266ef4bc3so3728315e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 01:18:41 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 r17-20020a05600c459100b004124907f43fsm10086350wmo.12.2024.02.19.01.18.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 01:18:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df48d019-cf07-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708334321; x=1708939121; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=iDwWtX23r2V8KSj03hLJqOS1zFTYx+OKrB2vX8bbu0Y=;
        b=UQ/UfG0h47EV4AKVNJ9jreMMnfGAGQThQ/6dsu+32ruwezdeL9FR5s6nuT9mDhIf31
         KLCEEvdAzD2zsZO753r7Fnxg2KpoO1gBrEafGKqkM3t0+DJjoFifOHaCHQzg9QU9H27b
         Sy3WViBDhdQQgnxGpgwWnuxIjwm//LUnXRJ8CUfQwQs4DhJOy3z6WgNh3EHggH0nA8F/
         3CZRUYdxPhvEGVAxhRRiap7cyVsn2wWe7NHJG1b6x3EgVrCMUmsvbtYXar60jVUc2v7Q
         AOsp8kWlrc6O8r3amG39zxlGg2p7as7u9kw4cJ+UNWutRBNiYeBg+RD6upGl+4+j90z+
         2Qdw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708334321; x=1708939121;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=iDwWtX23r2V8KSj03hLJqOS1zFTYx+OKrB2vX8bbu0Y=;
        b=q8dbzN6hLbDxHLKwEyk1ljiC8ECKwmUYeIg6a5EXl9OX8gPz4EUW95NDUXgFHlVSGO
         +NJOdwUq94ShH7yag4eUGKHy51pbZ8YZwHFW99iIBDFqylYEQEct+Ayqb5n9o/+L3TI8
         T2DaHrkFt6NdLsPiEWDMzSYn6/MaGUPZmXl8NIZN1v+CbOcCbXJjH88dznUxY1LdwCyL
         fu85CluAfux7cwYQhhYlhIZBvm3KrTKc9/BLNqNBj9bQTo3pppCjYMaUdof1J2tsMhU9
         obn7q1juM5DD4XF4aw/mXH/LFtV4GCZVe5tH+zSmSh7WWCEOgTiJNciwUTp3BYfGNJJZ
         PaUA==
X-Forwarded-Encrypted: i=1; AJvYcCUCP+UJzYV/N1JPHgPpfEAASJiXoMiKJYZCmum4PQcC/qtUv3PjZAtVv4AXjU8WcGWIg3tdwv0VpgyPgwHAEtM743x4pc59dwdTj4OOx54=
X-Gm-Message-State: AOJu0YwgYWkyloV4yZrICaGBkAOybPRd0dU+tkCtkGnrBhXNXy55q0MB
	s/mz46EPSOaBSdkVtm4W0G0Jt3RNuaqiv2mE0TfLAbMs38XKPPqgwzHjqcUvLg==
X-Google-Smtp-Source: AGHT+IFEHc4IH0UcRO+ZtLpyzJ7eO9bxp8hT5gfZO4XBj6RsD/vDDoadPcjxIAec9YxltcSL9Grvgw==
X-Received: by 2002:a05:600c:35d0:b0:40e:f693:94dc with SMTP id r16-20020a05600c35d000b0040ef69394dcmr8189355wmq.11.1708334320974;
        Mon, 19 Feb 2024 01:18:40 -0800 (PST)
Message-ID: <998a0511-b47b-49c8-8db4-9c2adb3b3d9b@suse.com>
Date: Mon, 19 Feb 2024 10:18:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] Reduce assembly code size of exception entry points
Content-Language: en-US
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240216105028.9517-1-frediano.ziglio@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240216105028.9517-1-frediano.ziglio@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.02.2024 11:50, Frediano Ziglio wrote:
> --- a/xen/arch/x86/x86_64/entry.S
> +++ b/xen/arch/x86/x86_64/entry.S
> @@ -22,6 +22,14 @@
>  #endif
>  .endm
>  
> +.macro BUILD_BUG_ON condstr cond:vararg
> +        .if \cond
> +        .error "Condition \condstr not satisfied"

Maybe

        .error "Condition \"\condstr\" not satisfied"

?

> @@ -187,7 +195,8 @@ FUNC_LOCAL(restore_all_guest)
>          SPEC_CTRL_EXIT_TO_PV    /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
>  
>          RESTORE_ALL
> -        testw $TRAP_syscall,4(%rsp)
> +        BUILD_BUG_ON(TRAP_syscall & 0xff)
> +        testb $TRAP_syscall >> 8,4+1(%rsp)
>          jz    iret_exit_to_guest

Nit: Blank after comma please (and again elsewhere).

Preferably with both adjustments (which I'd be okay making while
committing, so long as you agree specifically on the former)
Reviewed-by: Jan Beulich <jbeulich@suse.com>

That said, especially with this not being an entry point path, and
neither this nor ...

> @@ -254,7 +263,8 @@ FUNC(lstar_enter)
>          pushq $FLAT_KERNEL_CS64
>          pushq %rcx
>          pushq $0
> -        movl  $TRAP_syscall, 4(%rsp)
> +        BUILD_BUG_ON(TRAP_syscall & 0xff)
> +        movb  $TRAP_syscall >> 8, 4+1(%rsp)
>          SAVE_ALL
>  
>          SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
> @@ -292,7 +302,8 @@ FUNC(cstar_enter)
>          pushq $FLAT_USER_CS32
>          pushq %rcx
>          pushq $0
> -        movl  $TRAP_syscall, 4(%rsp)
> +        BUILD_BUG_ON(TRAP_syscall & 0xff)
> +        movb  $TRAP_syscall >> 8, 4+1(%rsp)
>          SAVE_ALL
>  
>          SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
> @@ -334,7 +345,8 @@ LABEL(sysenter_eflags_saved, 0)
>          pushq $3 /* ring 3 null cs */
>          pushq $0 /* null rip */
>          pushq $0
> -        movl  $TRAP_syscall, 4(%rsp)
> +        BUILD_BUG_ON(TRAP_syscall & 0xff)
> +        movb  $TRAP_syscall >> 8, 4+1(%rsp)
>          SAVE_ALL

... any of these being exception entry point paths (and hence none of
these changes being related to the subject), it would likely have been
a good idea to split this into two patches.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 09:49:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 09:49:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682739.1061870 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc0Gu-0003iz-Pa; Mon, 19 Feb 2024 09:49:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682739.1061870; Mon, 19 Feb 2024 09:49:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc0Gu-0003is-MZ; Mon, 19 Feb 2024 09:49:12 +0000
Received: by outflank-mailman (input) for mailman id 682739;
 Mon, 19 Feb 2024 09:49:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rc0Gt-0003ii-It; Mon, 19 Feb 2024 09:49:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rc0Gt-0002tG-5x; Mon, 19 Feb 2024 09:49:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rc0Gs-0001nT-PS; Mon, 19 Feb 2024 09:49:10 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rc0Gs-00050O-Nu; Mon, 19 Feb 2024 09:49:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=DdE2pp9synZWVGWqyyFuQRCrsayBDCMSYYmWtXdxNNs=; b=dSFKxn9ZA1HB/ns7BtsTYV/Ow0
	DV0DAYQL60C+HincyiITZbc9MqSZYLLyzyDJKppFRw7cmtxz8kERt7133x3L37bnUt5oAJlKvYGwa
	ftCoNJsGXWhGmv/r5mI2/3/4pE1H6CuwqtRF3l+nf4saEZ2fj9yEHCDBL1UoE9eYVGp0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184699-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184699: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
X-Osstest-Versions-That:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 19 Feb 2024 09:49:10 +0000

flight 184699 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184699/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184693
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184693
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184695
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184695
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184695
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184695
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184695
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184695
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184695
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184695
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184695
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184695
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3
baseline version:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3

Last test of basis   184699  2024-02-19 01:52:12 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Mon Feb 19 10:03:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 10:03:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682746.1061881 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc0V3-0006b5-Oa; Mon, 19 Feb 2024 10:03:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682746.1061881; Mon, 19 Feb 2024 10:03:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc0V3-0006ay-I8; Mon, 19 Feb 2024 10:03:49 +0000
Received: by outflank-mailman (input) for mailman id 682746;
 Mon, 19 Feb 2024 10:02:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qfqo=J4=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rc0TT-0006Y1-AK
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 10:02:13 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f240241d-cf0d-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 11:02:10 +0100 (CET)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-5129cdae3c6so2400424e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 02:02:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f240241d-cf0d-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708336929; x=1708941729; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=10zLsJ+Dao3HNxLIDSx6uvKwuatf5IQrs6vrSfrZJig=;
        b=gKR+wswEG0Oob7Fd4wD4Ts6ZPWfRTVOQBMra5aNUwbdsQEJ/B5sM9zBlpoMuhIZ+38
         NtF50HQtnelMCGDgk2DuiB6AZ5o28VXf/sJUGv+KxvWp5jZimR95NRe4kW/ETPg1mtyc
         SwMWVWo10Vdq2tT89wb23t/5ff/TNAVDfcK1k=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708336929; x=1708941729;
        h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=10zLsJ+Dao3HNxLIDSx6uvKwuatf5IQrs6vrSfrZJig=;
        b=Tkesjmk8JFA/moh1b1L0R4/hXD8teeyY0dMLqpW8uprm1V5MCEIPd8Jg8eSi9ErprN
         +QDUPSIdqzwSNiY9AETxnMbJqZGrwfjw7wV6eOVcAPJV6XUkUboHrTQ1nKe9FNURqT2I
         OP7/ew/1KeGveYxT7LONPXQrAZU56xDmsQm3bMLl0SkjHuYMFCGxwSruY85rw8PZNe+C
         5yc4fSqEJkxtT2c5pv1JNCmebQ/J2HKjd6SvSmt9nOio5DckHgD54u3C2yIVPBeLJv6y
         I6GRvGEKtLxb/6Cf9TUmFBm1vlzjl3MbomK5SWspAIAj7CG5e7tSWT7n7ZaEckAXyXaN
         HBCQ==
X-Gm-Message-State: AOJu0Yx34XOWU+O8XSPfV61ZOoen4P4suwshmKw45Lh3itkR+/1C+yZP
	PQcmkUFykge3EpUVYZkIK/bgamESoBa2Ejds/JITmbTP0FEvFP8ay5c+cJxTSvYYnBB15ywuHxE
	pYncYpDej30T/5ZxTTn8BfCxZ4T5rHGcOah9DYFAdBAaBWJeR
X-Google-Smtp-Source: AGHT+IHPu9HH78Q3Yg68BWdNTHv6juaYeSLC3kOhSD4HPRn18WAJBVGXQN4ynpJ7ML0Z1DhXZzvzoe3vT0NGnyUSRCA=
X-Received: by 2002:a05:6512:224b:b0:511:621a:5d5f with SMTP id
 i11-20020a056512224b00b00511621a5d5fmr9447491lfu.39.1708336928440; Mon, 19
 Feb 2024 02:02:08 -0800 (PST)
MIME-Version: 1.0
From: George Dunlap <george.dunlap@cloud.com>
Date: Mon, 19 Feb 2024 18:01:54 +0800
Message-ID: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
Subject: Stats on Xen tarball downloads
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: committers@xenproject.org, Kelly Choi <kelly.choi@cloud.com>
Content-Type: multipart/mixed; boundary="000000000000c7064e0611b92fa6"

--000000000000c7064e0611b92fa6
Content-Type: text/plain; charset="UTF-8"

Hey all,

One of the questions we had with respect to changing our release
practice (for instance, making the process more light-weight so that
we could do a point release after every XSA) was, "How many people are
actually using the tarballs?"  I finally got access (again) to
downloads.xenproject.org, and took a look at the logs.  It appears
that we only keep about 2 weeks of logs there.

Short answer: It's pretty clear from looking at the logs that there
are large numbers of automated build systems building various versions
of Xen from tarballs.  It *looks* like there are over 300 people a
week downloading 4.18.0 specifically from various web browsers.

Attached is a report generated by goaccess on the log after filtering
for xen-4.18.0.tar.gz (which includes the .sig), as well as a report
filtering on any Xen release (again including .sig files).

Between 6 and 19 Feb, the xen-4.18.0 tarball was downloaded by 704
"visitors" (unique by IP address, user agent, and date).  There are a
handful of IPs that had more than one "visit', but the vast majority
had only one "visit".  (The "Visitor Hostname and IPs" tab is somewhat
confusing on this point; it lists 800+ visits, but only 366 unique IP
addresses.  I verified independently that there were around 800 unique
IP addresses in the logs, so this may just be a limit as to how much
data is included in the report.)

Of the user agents, only 48 of these visits are classified as
"crawlers"; the vast majority are (in order) Chrome, Firefox, Edge,
Opera, or Safari.  There are a handful (<20) of downloads from
non-browser, non-bot/crawler agents (all wget).

Looking at the *non*-4.18 downloads, nearly all of them have user
agents that make it clear they're part of automated build systems:
user agents like curl and wget, but also "Go-http-client", "libfetch",
and "ansible-http".  There are several references to package managers
as well (xbps, pkgmon, slackrepo).  What is *not* significantly
represented are user-agent strings that look like web browsers; there
are intermittent ones, but not very many.

It's not really clear to me why we'd be getting 300-ish people
downloading the Xen 4.18.0 tarball, 2/3 of which are on Windows.  But
then I'm also not sure why someone would *fake* hundreds of downloads
a week from unique IP addresses; and in particular, if you were going
to fake hundreds of downloads a week, I'm not sure why you'd only fake
the most recent release.

I think we can fairly conclusively conclude that there are regular
users of older versions of the Xen tarballs, as part of automated
build systems which could be disrupted by any significant change to
the way the tarballs worked.

I think we can *tentatively* conclude that there are hundreds of
people per week downloading the most recent release tarball via the
website.  It would be interesting to see if we could determine some
way of trying to evaluate how many of those resulted in a build (e.g.,
by looking at "extfiles" downloads or something).

More conclusions than that (e.g., whether it's worth changing the
tarball layout to make it more automate-able, whether it's worth
investing time making the build-from-tarball experience better, and/or
pointing people to better ways to get Xen) I haven't considered yet.

 -George

--000000000000c7064e0611b92fa6
Content-Type: text/html; charset="US-ASCII"; name="4-18-report.html"
Content-Disposition: attachment; filename="4-18-report.html"
Content-Transfer-Encoding: base64
Content-ID: <f_lssmy78j0>
X-Attachment-Id: f_lssmy78j0

PCFET0NUWVBFIGh0bWw+PGh0bWwgbGFuZz0nZW4nPjxoZWFkPjxtZXRhIGNoYXJzZXQ9J1VURi04
Jz48bWV0YSBuYW1lPSdyZWZlcnJlcicgY29udGVudD0nbm8tcmVmZXJyZXInPjxtZXRhIGh0dHAt
ZXF1aXY9J1gtVUEtQ29tcGF0aWJsZScgY29udGVudD0nSUU9ZWRnZSc+PG1ldGEgbmFtZT0nZ29v
Z2xlJyBjb250ZW50PSdub3RyYW5zbGF0ZSc+PG1ldGEgbmFtZT0ndmlld3BvcnQnIGNvbnRlbnQ9
J3dpZHRoPWRldmljZS13aWR0aCwgaW5pdGlhbC1zY2FsZT0xJz48bWV0YSBuYW1lPSdyb2JvdHMn
IGNvbnRlbnQ9J25vaW5kZXgsIG5vZm9sbG93Jz48bGluayByZWw9J2ljb24nIGhyZWY9J2RhdGE6
aW1hZ2UveC1pY29uO2Jhc2U2NCxBQUFCQUFFQUVCQVFBQUVBQkFBb0FRQUFGZ0FBQUNnQUFBQVFB
QUFBSUFBQUFBRUFCQUFBQUFBQWdBQUFBQUFBQUFBQUFBQUFFQUFBQUFBQUFBREd4c1lBV0ZoWUFC
d2NIQUJmQVA4QS85ZGZBQURYcndBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFJaUlpSWlJaUlpSWpNbFVrUWdBaUlpSWlJaUlpSWlJaUl6SlZKRUlB
QUFJaUlpSWlJaUlpSWlNeVZTUkNBQUlpSWlJaUlpSWlJaUlSRVJFUkVSRVJFUkVSRVJFUkVSRVJJ
aUlpSWlJaUlpSWdBQ1ZWVWlJaUlpSWlJaUlpSWlJaUlBQWxWVklpSWlJaUlpSWlJaUlpSWhFUkVS
RVJFUkVSRVJFUkVSRVJFUkVBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBJyB0eXBlPSdpbWFn
ZS94LWljb24nIC8+PHRpdGxlPlNlcnZlciZuYnNwO1N0YXRpc3RpY3M8L3RpdGxlPjxzdHlsZT5A
Zm9udC1mYWNlIHtmb250LWZhbWlseTogJ2ZhJztzcmM6ICB1cmwoZGF0YTphcHBsaWNhdGlvbi9m
b250LXdvZmY7Y2hhcnNldD11dGYtODtiYXNlNjQsZDA5R1JnQUJBQUFBQUZ4UUFBc0FBQUFBWEFR
QUFRQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUJQVXk4eUFBQUJDQUFBQUdBQUFBQmdEeElQ
YTJOdFlYQUFBQUZvQUFBQ1ZBQUFBbFE5OUFHWFoyRnpjQUFBQTd3QUFBQUlBQUFBQ0FBQUFCQm5i
SGxtQUFBRHhBQUFWQlFBQUZRVWZSVExJMmhsWVdRQUFGZllBQUFBTmdBQUFEWWZpTGVkYUdobFlR
QUFXQkFBQUFBa0FBQUFKQWhVQk1Cb2JYUjRBQUJZTkFBQUFiUUFBQUcwYjlvRGtteHZZMkVBQUZu
b0FBQUEzQUFBQU54THhtR3FiV0Y0Y0FBQVdzUUFBQUFnQUFBQUlBQjRBVmR1WVcxbEFBQmE1QUFB
QVVvQUFBRktJaFdUc25CdmMzUUFBRnd3QUFBQUlBQUFBQ0FBQXdBQUFBTURid0dRQUFVQUFBS1pB
c3dBQUFDUEFwa0N6QUFBQWVzQU13RUpBQUFBQUFBQUFBQUFBQUFBQUFBQUFSQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQVFBQUE4dElEd1AvQUFFQUR3QUJBQUFBQUFRQUFBQUFBQUFBQUFBQUFJQUFBQUFB
QUF3QUFBQU1BQUFBY0FBRUFBd0FBQUJ3QUF3QUJBQUFBSEFBRUFqZ0FBQUNLQUlBQUJnQUtBQUVB
SU9vTDhBZndEZkFSOEJmd0dmQWU4Q0x3TFBBNjhFSHdSUEJHOEUzd1ZQQmE4R2Z3YXZCeDhIandn
UENGOEk3d2x2Q2o4TER3c3ZEQjhNbnd6dkRhOE43dzVQRHo4UGJ4QWZFRjhRanhDdkVNOFJIeElm
RW04U2p4TXZGQzhVVHhUUEZjOFdYeG9QSEo4Yzd4Mi9IZThmZngvdklBOGdYeWFQS004cEx5blBL
Mzh0TC8vZi8vQUFBQUFBQWc2Z3Z3QXZBSjhCSHdFL0FaOEIzd0lmQXI4RHJ3UVBCRThFYndTL0JU
OEZud1ovQnA4SEh3ZC9DQThJWHdqdkNXOEtMd3NQQ3k4TUR3eWZETzhOZnczUERrOFBQdzl2RUE4
UVR4Q1BFSzhRenhFUEVnOFNieEtQRXk4VUh4UlBGTThWenhaUEdnOGNueHp2SGI4ZDd4OXZIKzhn
RHlCUEpvOG92eWt2S2M4cmJ5MHYvOS8vOEFBZi9qRmZrUUF4QUNELzhQL2cvOUQvb1ArQS93RCtN
UDNnL2NEOXNQMXcvU0Q4NFB3Zy9CRDdzUHRnK3ZENnNQb3crY0Q1RVBoUStFRDNjUGNBOXNEMlFQ
WXc5ZUQxQVBUZzlGRDBNUFFROUFEejhQUEE4dUR5b1BLUThnRHhJUEVROEtEdnNPOUE2NkRwSU9q
ZzZDRG9BT2FRNWpEbUlPWHczOURkc04xZzNORGJRTm1nQURBQUVBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFRQUIv
LzhBRHdBQkFBQUFBQUFBQUFBQUFnQUFOemtCQUFBQUFBRUFBQUFBQUFBQUFBQUNBQUEzT1FFQUFB
QUFBUUFBQUFBQUFBQUFBQUlBQURjNUFRQUFBQUFCQUFBQlFBUUFBa0FBRHdBQUV4VVVGak1oTWpZ
OUFUUW1JeUVpQmdBVERRUEFEUk1URGZ4QURSTUNJTUFORXhNTndBMFRFd0FBQUFJQUFQKzNBN2NE
YmdBYkFFSUFBQUUwSnk0Qkp5WWpJZ2NPQVFjR0ZSUVhIZ0VYRmpNeU56NEJOellCRkFZaklpWXZB
UTRCSXlJbkxnRW5KalUwTno0Qk56WXpNaGNlQVJjV0ZSUUdCeGNlQVJVQ2toUVVSaTR2TlRVdUww
VVZGQlFWUlM4dU5UVXZMa1lVRkFFbEt4NFBHd3JFTW5VOVUwbEtiUjhnSUI5dFNrbFRWRWxKYlNB
Z0pTTEVDZ3NCMnpVdkwwVVVGQlFVUlM4dk5UVXVMMFVVRlJVVVJTOHUvbG9lS3dzTHd5TWtJQjl1
U1VsVFZFbEpiaDhnSUI5dVNVbFVQSFV6eEFrYkR3QUFBQU1BQUFBQUJBQURKUUFjQURjQVJ3QUFK
UkVPQVFjT0FRY09BU3NCSWlZbkxnRW5MZ0VuRVJRV015RXlOalVSTkNZaklTSUdGUlFXRng0QkZ4
NEJPd0V5TmpjK0FUYytBVFUzRVJRR0l5RWlKalVSTkRZeklUSVdBN2NKRkFzOWVqd2dUeXdDTEU4
Z1BIbzlDeFFKQ3djRFNnY0xBaEQ4dGdjTExpWTZjamtYU0I0Q0hrZ1hPWEk2SERoSk5pWDh0aVUy
TmlVRFNpVTJXd0czQ2hNSU1HQXlHelUxR3pKZ01BZ1RDdjVKQndzTEJ3SlpDeHdMQnpGVEhpMWJM
Uk02T2hNdFd5MFdVeVFWL1pJbE5qWWxBbTRtTmpZQUFRQUFBQUFFQUFOdUFDb0FBQ1VpSmljQkpp
Y3VBU2NtTlRRM1BnRTNOak15RmhjK0FUTXlGeDRCRnhZVkZBY09BUWNHQndFT0FTTUNBQWNOQmY2
YkFSVVZNQk1VRXhKSE1qTkFTNElpSW9KTFFETXlSeElURkJNeEZCVUMvcHdGRFFjQUJRVUJXQUlW
RlVVdUxqTS9NakZGRWhOYUlpSmFFeEpGTVRJL015NHVSaFVWQXY2cEJRVUFBQUFCQUFBQUdnTzNB
NlVBTGdBQUFSUUdEd0VUSEFFVkZBWWpJaVluSlFVT0FTTWlKalUwTmpVVEp5NEJOVFEyTnlVVFBn
RXpNaFlYRXdVZUFSVUR0d2tHenpFTERBWU1CZjcvL3dBR0N3WU1EQUV4MEFVSkZRc0JINEFFRHdr
S0R3T0JBUjhLRmdKRkNBNEZ5LzdqQXdZREN4RUVBNGFHQXdRUkN3TUdBd0VkeXdVT0NBME1BU29C
QkFnUUVBaisvQ29CREEwQUFnQUFBQm9EdHdPbEFBa0FPQUFBQVRjdkFROEJGd2MzRndFVUJnOEJF
eHdCRlJRR0l5SW1KeVVGRGdFaklpWTFORFkxRXljdUFUVTBOamNsRXo0Qk16SVdGeE1GSGdFVkFv
cXY4bXhzOGE4cTJOZ0JCQWtHenpFTERBWU1CZjcvL3dBR0N3WU1EQUV4MEFVSkZRc0JINEFFRHdr
S0R3T0JBUjhLRmdGNXFpUGIyeU9xOEhGeEFid0lEZ1hML3VNREJnTUxFUVFEaG9ZREJCRUxBd1lE
QVIzTEJRNElEUXdCS2dFRUNCQVFDUDc4S2dFTURRQUFBQUlBQUFBQUF0c0RiZ0FiQURjQUFDVVVC
aU1oSWlZMU5EYytBVGMyTXg0Qk16STJOeklYSGdFWEZoVURGQWNPQVFjR0l5SW5MZ0VuSmpVME56
NEJOell6TWhjZUFSY1dBdHRITXY0WU1rZ0lDQ29sSkRnald6VTBYQ000SkNRcUNBaVNFUkU4S0Nn
dExpZ29PeEVTRWhFN0tDZ3VMU2dvUEJFUmx6NVpXVDQ1T1RwZEhoMGhLQ2doSFI1ZE9qazVBZnN0
S0NnOEVSRVJFVHdvS0MwdUtDZzdFUklTRVRzb0tBQUFBQUFFQUFBQVNRTzNBMjRBRUFBaEFERUFR
UUFBQVJVVUJpTWhJaVk5QVRRMk15RXlGaFVSRlJRR0l5RWlKajBCTkRZeklUSVdGUUVWRkFZaklT
SW1QUUUwTmpNaE1oWVJGUlFHSXlFaUpqMEJORFl6SVRJV0FiY3JIdjdiSGlzckhnRWxIaXNySHY3
Ykhpc3JIZ0VsSGlzQ0FDc2UvdHNlS3lzZUFTVWVLeXNlL3RzZUt5c2VBU1VlS3dGdTNCNHJLeDdj
SGlzckhnRzMzQjRyS3g3Y0hpc3JIdjVKM0I0ckt4N2NIaXNyQVpuY0hpc3JIdHdlS3lzQUNRQUFB
RWtFQUFOdUFBOEFId0F2QUQ4QVR3QmZBRzhBZndDUEFBQWxGUlFHS3dFaUpqMEJORFk3QVRJV0VS
VVVCaXNCSWlZOUFUUTJPd0V5RmdFVkZBWXJBU0ltUFFFME5qc0JNaFlCRlJRR0t3RWlKajBCTkRZ
N0FUSVdBUlVVQmlzQklpWTlBVFEyT3dFeUZnRVZGQVlyQVNJbVBRRTBOanNCTWhZQkZSUUdLd0Vp
SmowQk5EWTdBVElXQVJVVUJpc0JJaVk5QVRRMk93RXlGaEVWRkFZckFTSW1QUUUwTmpzQk1oWUJK
U0VXdHhjZ0lCZTNGaUVoRnJjWElDQVh0eFloQVcwZ0Y3WVhJQ0FYdGhjZy9wTWhGcmNYSUNBWHR4
WWhBVzBnRjdZWElDQVh0aGNnQVc0Z0Y3Y1dJU0VXdHhjZy9wSWdGN1lYSUNBWHRoY2dBVzRnRjdj
V0lTRVd0eGNnSUJlM0ZpRWhGcmNYSU81dUZ5QWdGMjRXSVNFQkRtMFhJQ0FYYlJjZ0lQN0ZiaGNn
SUJkdUZpRWhBak51RnlBZ0YyNFhJQ0QreEcwWElDQVhiUmNnSVA3RmJoY2dJQmR1RmlFaEFqTnVG
eUFnRjI0WElDRCt4RzBYSUNBWGJSY2dJQUVPYmhjZ0lCZHVGeUFnQUFZQUFBQkpCQUFEYmdBUEFC
OEFMd0EvQUU4QVh3QUFKUlVVQmlzQklpWTlBVFEyT3dFeUZoRVZGQVlyQVNJbVBRRTBOanNCTWhZ
QkZSUUdJeUVpSmowQk5EWXpJVElXQVJVVUJpc0JJaVk5QVRRMk93RXlGZ0VWRkFZaklTSW1QUUUw
TmpNaE1oWVJGUlFHSXlFaUpqMEJORFl6SVRJV0FTVWhGcmNYSUNBWHR4WWhJUmEzRnlBZ0Y3Y1dJ
UUxiSUJmOTNCY2dJQmNDSkJjZy9TVWhGcmNYSUNBWHR4WWhBdHNnRi8zY0Z5QWdGd0lrRnlBZ0Yv
M2NGeUFnRndJa0Z5RHViaGNnSUJkdUZpRWhBUTV0RnlBZ0YyMFhJQ0QreFc0WElDQVhiaFloSVFJ
emJoY2dJQmR1RnlBZy9zUnRGeUFnRjIwWElDQUJEbTRYSUNBWGJoY2dJQUFBQVFCRkFGRUR1d0w0
QUNRQUFBRVVCZ2NCRGdFaklpWW5BUzRCTlRRMlB3RStBVE15Rmg4QkFUNEJNeklXSHdFZUFSVUR1
d2dJL2hRSEZRb0xGUWYrNHdnSUNBaE9DQlFMQ2hVSXFBRjJDQlVLQ3hRSVRnZ0lBbk1LRlFmK0ZB
Z0lDQWdCSFFjVkN3b1ZCMDRJQ0FnSXFBRjNDQWdJQ0U0SEZRc0FBQUVBUHdBL0F1WUM1Z0E4QUFB
bEZBWVBBUTRCSXlJbUx3RUhEZ0VqSWlZdkFTNEJOVFEyUHdFbkxnRTFORFkvQVQ0Qk16SVdId0Uz
UGdFek1oWWZBUjRCRlJRR0R3RVhIZ0VWQXVZSkIwNElGQXNMRkFpb3FBY1ZDd29WQjA0SUNBZ0lx
S2dJQ0FnSVRnY1ZDZ3NWQjZpb0NCUUxDeFFJVGdjSkNRZW9xQWNKd3dvVkIwNElDQWdJcUtnSUNB
Z0lUZ2NWQ2dzVkI2aW9DQlFMQ3hRSVRnY0pDUWVvcUFjSkNRZE9DQlFMQ3hRSXFLZ0hGUXNBQUFB
Q0FBQUFBQU51QTdjQVBBQktBQUFCRkFjT0FRY0dJeUluTGdFbkpqVTBOejRCTnpZM05oWVhGZ1lI
RGdFVkZCY2VBUmNXTXpJM1BnRTNOalUwSmljdUFUYytBUmNXRng0QkZ4WVZBUkVVQmlNaUpqVVJO
RFl6TWhZRGJpTWllRkJRV2x0UVVIY2lJd3dMTFNFaEtSazhFaElKR0RjK0Z4ZFFOVFU5UERZMVR4
Y1hQVGdZQ0JJU1BCZ3FJU0F0REF6K2tpc2VIaXNySGg0ckFiZGJVRkIzSWlNakluZFFVRnMwTVRK
YUp5Y2ZFd2tZR0R3U0tudEZQVFUxVUJjWEZ4ZFFOVFU5UlhzcUVqd1lHQWtUSHljbldqSXhOQUcz
L3BJZUt5c2VBVzRlS3lzQUFBQUNBQUFBQUFOdUEyNEFDd0NTQUFBQk5DWWpJZ1lWRkJZek1qWWxG
UlFHRHdFT0FRY2VBUmNlQVJVVUJnY09BU01pSmk4QkRnRUhEZ0VIRGdFckFTSW1Md0V1QVNjSERn
RWpJaVluTGdFbkxnRTFORFkzUGdFM0xnRXZBUzRCUFFFME5qOEJQZ0UzTGdFbkxnRTFORFkzUGdF
ek1oWWZBVDRCTno0Qk56NEJPd0V5Rmg4QkhnRVhOejRCTXpJV0Z4NEJGeDRCRlJRR0J3NEJCeDRC
SHdFZUFSVUNTVlk4UFZWVlBUeFdBU1VKQjJvRkNnY09IeEFDQkFNRENsVVBCQWNFVHd3YURnTUdC
d0lMQ0g4SERBRVFEUm9OVUFNSUF3UUlBeFkyRWdJQ0FnTU9IdzhJREFSb0NBa0pCbXNFQ3djUEho
QURBd01DQzFVUEF3Z0RUdzBhRFFNSEJ3SUxCMzhJREFFUURSb01VUU1IQkFRSEF4YzJFZ0lDQXdJ
T0h3OEhEQVZvQndvQnR6eFdWanc5VlZWN2Z3WU5BUkFPR2d3Vkp4TURDQU1FQndNTldRTUNQZ1lM
QlJvMkdnY0pDZ2RwQlFvR1BRSURBd01WTXhnREJ3UURCd01USnhRT0hBOFBBUXdJZmdjTkFSQU9H
ZzBVSnhNREJ3UUVCZ01PV1FRQ1BRWUxCQnMyR2djSkNnZHFCQW9IUFFNREJBSVZNeGtEQmdRRUJn
TVVKaFFPSEE0UUFnd0hBQUFBQUFZQUFBQUFBeVVEYmdBUEFCOEFMd0E2QUVRQWFRQUFBUkVVQmlz
QklpWTFFVFEyT3dFeUZoY1JGQVlyQVNJbU5SRTBOanNCTWhZWEVSUUdLd0VpSmpVUk5EWTdBVElX
RXhFaEVSUVdNeUV5TmpVQklTY3VBU2NqRGdFSEJSVVVCaXNCRVJRR0l5RWlKalVSSXlJbVBRRTBO
anNCTno0Qk93RXlGaDhCTXpJV0ZRRWxDd2drQ0FzTENDUUlDNUlMQnlVSUNnb0lKUWNMa2dvSUpR
Y0xDd2NsQ0FwSi9nQVFBd0hiQWhEK2dBRUFHd0VIQXJVREJRSUI5d3NJTnpVbS9pVW1OamNIQ3dz
SHNTZ0lMUmUzRmkwSktMQUlDd0lTL3JjSUNnb0lBVWtJQ3dzSS9yY0lDZ29JQVVrSUN3c0kvcmNJ
Q2dvSUFVa0lDd3YrV3dJZC9lTVZGeGNWQW1aREFnUUJBUVFDVlNRSUMvM2pNRVZETHdJZ0N3Z2tD
QXBnRlI0ZUZXQUtDQUFBQWdBVEFFa0RwQU1sQUJVQVBBQUFBUkVVQmlzQk5TTVZJeUltTlJFME5q
RUpBVEFXRlRjSERnRXJBU0ltSndrQkRnRW5JaVl2QVNZMk53RTJNaDhCTlRRMk93RXlGaDBCRng0
QkJ3TWxGZy9iazlzUEZnRUJTQUZKQVg4akF3WURBZ1FHQXY1MC9uVURCd1FEQndJakJRSUZBWnNT
TXhLTEN3aHRDQXQ5QlFJRkFZRCs3ZzhXM053V0R3RVNBUUlCRC83eEFnRW5LZ0lFQWdJQlN2NjJB
Z01CQkFJcUJnOEZBVllQRDNSdkNBc0xDT2xvQlE4R0FBTUFBUCszQTI0RHR3QVRBQndBSmdBQUFS
NEJGUkVVQmlNaElpWTFFVFEyTXlFeUZoY0hGVE11QVM4QkxnRVRFU01pSmowQklSRWhBMGNRRnlB
WC9RQVhJQ0FYQWdBWE54Qk0xd01IQTdJRER0WHVGeUQrU1FMY0F0NFFOeGY5YmhjZ0lCY0RraGNn
RnhBbjF3Z05BN01EQi95WkFra2dGKzc4a2dBREFBQUFBQU51QTI0QUZRQXhBRTBBQUFFUkZBWXJB
U0ltUFFFME5qc0JOVFEyT3dFeUZoVVhOQ2N1QVNjbUl5SUhEZ0VIQmhVVUZ4NEJGeFl6TWpjK0FU
YzJOeFFIRGdFSEJpTWlKeTRCSnlZMU5EYytBVGMyTXpJWEhnRVhGZ0lBQ3dlM0NBb0tDSUFLQ0NV
SEMrNFpHRlU0T1VCQU9UbFVHUmdZR1ZRNU9VQkFPVGhWR0JtQUl5SjNVRkJiVzFCUWR5TWlJaU4z
VUZCYlcxQlFkeUlqQW9EL0FBZ0tDZ2dsQnd2SkNBb0tDTWxBT1RoVkdCa1pHRlU0T1VCQU9UbFVH
UmdZR1ZRNU9VQmJVRkIzSWlNakluZFFVRnRiVUZCM0lpTWpJbmRRVUFBQUFBQUVBQUFBU1FPM0E3
Y0FDd0FYQURFQVVRQUFKVFFtSXlJR0ZSUVdNekkyTnpRbUl5SUdGUlFXTXpJMk54VVVCaU1oSWlZ
OUFUUTJNeUVYSGdFek1qWS9BU0V5RmhVREZnWUhBUTRCSXlJbUp3RXVBVGMrQVRzQkVUUTJPd0V5
RmhVUk16SVdGd0xiRlE4UEZoWVBEeFdURmc4UEZSVVBEeFpKSUJmOHR4Y2dJQmNCQ2swUUtCVVdL
QkJPQVFrWElMb0VCQWovQUFVT0J3WU9CZjhBQ0FVRkJCSUxreFVQa3c4Vmtnd1NCTGNQRlJVUER4
WVdEdzhWRlE4UEZoYVB0eGNnSUJlM0Z5Qk9EeEVSRDA0Z0Z3RkZDaFlJL3dBR0JRVUdBUUFJRmdv
S0RBRUFEeFlXRC84QURBb0FBQU1BQUFBQUEyNERiZ0FZQURRQVVBQUFBUlFHQndVT0FTTWlKaWN1
QVRVUk5EWTNOaklYQlI0QkZUTTBKeTRCSnlZaklnY09BUWNHRlJRWEhnRVhGak15Tno0Qk56WTNG
QWNPQVFjR0l5SW5MZ0VuSmpVME56NEJOell6TWhjZUFSY1dBcVVLQ2Y3SkJBa0ZCUWtFQ1FrSkNR
Z1VDQUUzQ1FwSkdSaFZPRGxBUURrNVZCa1lHQmxVT1RsQVFEazRWUmdaZ0NNaWQxQlFXMXRRVUhj
aklpSWpkMUJRVzF0UVVIY2lJd0czQ2hFRnRnTURBd0lGRVFvQmJnb1JCUVFGdHdRUkNrQTVPRlVZ
R1JrWVZUZzVRRUE1T1ZRWkdCZ1pWRGs1UUZ0UVVIY2lJeU1pZDFCUVcxdFFVSGNpSXlNaWQxQlFB
QUFCQUFBQUFBTnVBMjRBVFFBQUFSRVVCaU1oSWlZbkpqWS9BUzRCSXlJSERnRUhCaFVVRng0QkZ4
WXpNalkzUGdFM01oWWZBUllVQndZSERnRUhCaU1pSnk0Qkp5WTFORGMrQVRjMk16SVhIZ0VYRmhj
M1BnRVhIZ0VWQTI0V0QvOEFDeElGQkFRSVR5aG5PRDAxTlZBWEZ4Y1hVRFUxUFVSNUtnSUhCQVFI
QTA0R0JDQW1KbGN2THpGYlVGQjNJaU1qSW5kUVVGc3FLU2xPSXlNZVNnZ1dDZ29OQXlYL0FBOFdE
UW9LRmdkUEpTa1hGMDgxTmp3OU5UVlFGeGM4TmdNREFRTUNUd1VPQmlVZUhTa0tDeU1pZDFCUVcx
dFBVSGdpSXdrSUh4WVhIRWtJQlFVRUVnc0FBZ0FBQUFBRGJnTnVBRFFBWndBQUFUQVVGUVlIRGdF
SEJpTWlKeTRCSnlZbkJ3NEJJeUltTlJFME5qTWhNaFlWRkFZUEFSNEJNekkyTno0Qk56NEJPd0V5
RmhVVEVSUUdJeUVpSmpVME5qOEJMZ0VqSWdZSERnRUhEZ0VyQVNJbVBRRTJOejRCTnpZek1oY2VB
UmNXRnpjK0FUTXlGaFVEWHhJb0oyOUdSVThxS1NsTUl5UWVTZ1VOQnc4V0ZnOEJBQThWQmdWT0tH
ZzNUSVVvQ2cwSEFna0diZ2dLRHhZUC93QVBGUVVGVHlob04weUZLQXNNQ0FJSUIzRUlDeE1uS0hC
R1JrOHFLU2xOSXlRZVNnWU5CdzhXQVZzREFVdEFQMXNaR1FnSUh4Y1dIVW9GQlJVUEFRQVBGaFlQ
QncwR1RpWXBTMEVRSVJJR0J3c0lBY3IvQUE4V0ZnOEhEUVZQSmloS1FSRWhFUVlIQ3djRVRFQS9X
aG9aQ0FrZkZoY2NTUVVHRlE4QUFBQUlBQUFBU1FRQUEyNEFFQUFnQURBQVFRQlNBR01BZEFDRUFB
QVRGUlFHS3dFaUpqMEJORFk3QVRJV0ZUVVZGQVlyQVNJbVBRRTBOanNCTWhZMUZSUUdLd0VpSmow
Qk5EWTdBVElXQVJVVUJpTWhJaVk5QVRRMk15RXlGaFUxRlJRR0l5RWlKajBCTkRZeklUSVdGVFVW
RkFZaklTSW1QUUUwTmpNaE1oWVZFeEUwSmlNaElnWVZFUlFXTXlFeU5qVVRFUlFHSXlFaUpqVVJO
RFl6SVRJVzJ3c0hKQWdMQ3dna0J3c0xCeVFJQ3dzSUpBY0xDd2NrQ0FzTENDUUhDd0tUQ3dqOTNB
Y0xDd2NDSkFnTEN3ajkzQWNMQ3djQ0pBZ0xDd2o5M0FjTEN3Y0NKQWdMU1FzSC9MWUhDd3NIQTBv
SEMwazJKZnkySlRZMkpRTktKVFlCRWlRSUN3c0lKQWdMQ3dpVEpRY0xDd2NsQndzTGl5VUhDd3NI
SlFjTEMvN1VKQWdMQ3dna0NBc0xDSk1sQndzTEJ5VUhDd3NIa2lVSEN3c0hKUWNMQ3dmK2JnSGJC
d3NMQi80bENBc0xDQUp0L1pNbU5qWW1BbTBtTmpZQUFBQUFBZ0FBQUF3RFlnTnVBQXNBSmdBQUFU
UW1JeUlHRlJRV016STJBUlFHQndFT0FTTWlKaWNCTGdFOUFUUTJPd0V5RmhjQkhnRVZBUUFySGg4
cUtoOGVLd0ppREFuKzV3b2JEdzhiQ2Y1bkZoNHJIdTRlU1JZQm1Ra01BcmNlS3lzZUh5b3EvdFlQ
R3dyKzV3b0xDd29CbVJaSUgrNGVLeDRXL21nTEd3NEFBQUFEQUFBQURBUTlBMjRBQ3dBbUFFUUFB
QUUwSmlNaUJoVVVGak15TmdFVUJnY0JEZ0VqSWlZbkFTNEJQUUUwTmpzQk1oWVhBUjRCRlRNVUJn
Y0JEZ0VqSWlZbkFUNEJOVFFtSndFdUFTTXpNaFlYQVI0QkZRRUFLeDRmS2lvZkhpc0NZZ3dKL3Vj
S0d3OFBHd24rWnhZZUt4N3VIa2tXQVprSkROc0xDdjduQ2hzUEZoc1BBUTBLQ3dzSy9tY1ZTUitB
SDBrVkFaa0tDd0szSGlzckhoOHFLdjdXRHhzSy91Y0tDd3NLQVprV1NCL3VIaXNlRnY1b0N4c09E
eHNLL3VjS0N4TVBBUXdLR3c4T0d3c0JtQlllSGhiK2FBc2JEZ0FJQUFBQVNRUUFBMjRBRUFBaEFE
RUFRZ0JTQUdNQWN3Q0RBQUEzRlJRR0t3RWlKajBCTkRZN0FUSVdGVFVWRkFZckFTSW1QUUUwTmpz
Qk1oWVZOUlVVQmlzQklpWTlBVFEyT3dFeUZnRVZGQVlqSVNJbVBRRTBOak1oTWhZVkFSVVVCaXNC
SWlZOUFUUTJPd0V5RmdFVkZBWWpJU0ltUFFFME5qTWhNaFlWTlJVVUJpTWhJaVk5QVRRMk15RXlG
alVWRkFZaklTSW1QUUUwTmpNaE1oYVNDd2R1QndzTEIyNEhDd3NIYmdjTEN3ZHVCd3NMQjI0SEN3
c0hiZ2NMQTI0TEIvMEFDQXNMQ0FNQUJ3djhrZ3NIYmdjTEN3ZHVCd3NEYmdzSC9RQUlDd3NJQXdB
SEN3c0gvUUFJQ3dzSUF3QUhDd3NIL1FBSUN3c0lBd0FIQzhsdUJ3c0xCMjRIQ3dzSDNHNEhDd3NI
YmdjTEN3ZmJiZ2NMQ3dkdUJ3c0wva0p1QndzTEIyNEhDd3NIQXBKdENBc0xDRzBJQ3d2K1FtNEhD
d3NIYmdjTEN3ZmJiZ2NMQ3dkdUJ3c0wxRzBJQ3dzSWJRZ0xDd0FBQUFRQUFBQUFBMklEWWdBSEFC
d0FJUUF5QUFBL0FTY0hGVE1WTXdFMEppTWlCZ2NCRGdFVkZCWXpNalkzQVQ0Qk5TY1hBU00xQVJR
R0R3RW5OejRCTXpJV0h3RWVBUlhQTklZMFNUMEJLd2NGQXdVQy9zb0NBZ2NHQWdVREFUVUNBaC91
L2lYdUEySU1DVi91WHdvYkRnOGJDb2NKREVrMGhqUTlTUUlTQmdjQ0F2N0tBZ1VDQmdjQ0FnRTJB
Z1VDYnU3K0plNEJwQTRiQ2wvdVhnb01EQXFHQ2hzUEFBQUFBQUlBQUFBQUFra0RiZ0FMQUNnQUFB
RTBKaU1pQmhVVUZqTXlOamNVQmdjRERnRWpJaVluQXk0Qk5UUTNQZ0UzTmpNeUZ4NEJGeFlWQWJk
V1BEMVdWajA4VnBJSUM5QUpKQlFWSkFuUUN3Z1hGMDgyTlQwOE5qVlBGeGNDU1QxVlZUMDhWbFk4
R2pVWC9rWVRGaFlUQWJvWE5SbzlOVFZRRnhjWEYxQTFOVDBBQlFBQUFFa0Q2d051QUFjQUZRQktB
RThBV2dBQUFUY25CeFV6RlRNVEpnWVBBUVlVRnhZMlB3RStBUk1WRkFZaklTSW1OUkUwTmpNaE1o
WVhIZ0VYRmdZUEFRNEJKeTRCSXlFaUJoVVJGQll6SVRJMlBRRTBOajhCUGdFWEhnRVZBeGNCSXpV
QkJ5YzNOaklmQVJZVUJ3SDdRMWRDTmlEOEJBc0V5QVFEQkFzRXlBUUJLbUZFL2lWRllHQkZBZHNS
SWhBRUJRRUJBd01jQkFvRUJ3MEcvaVVtTmpZbUFkc21OUU1ESkFRTEJRVUhONlQrZ0tRQ2ZUU2xO
UkF1RUZZUUVBRVNRMVpDSURjQm5BUUJCTWdFQ3dRRUFRVElCQXYrc0d4RVlXRkVBZHRFWVFjSUFR
Y0ZCQWtESEFRREFnSUNOaWIrSlNZMk5pWklBd2NDSlFRQ0FnSUpCZ0dtcGY2QXBRRTFOYVUwRUJC
WEVDNFBBQUFBQUFJQUFBQkpBNmtEYmdBNkFGQUFBQUVWRkFZaklTSW1OUkUwTmpNaE1oWVhIZ0VY
RmdZUEFRNEJJeUltSXk0Qkl5RWlCaFVSRkJZeklUSTJQUUUwTmo4QlBnRXpNaFlYSGdFVkV3RUdJ
aThCSmpRL0FUWXlId0VCTmpJZkFSWVVCd01sWVVUK0pVVmdZRVVCMnhFaUVBUUZBUUVEQXh3REJ3
TUJBd0VIRFFiK0pTWTJOaVlCMnlZMUF3TWtBd2NEQWdNQ0JRZUUvaTROSmc3MURnNCtEaVlPbGdG
eURTWU9QdzBOQWFPMVJHRmhSQUhiUkdFSENBRUhCUVFKQXh3REF3RUNBalltL2lVbU5qWW1rUU1I
QWlVREF3RUJBZ2tHQVJqK0x3NE85UTRtRGo0T0RwWUJjZzRPUHc0bERnQUFBQUVBQVAvL0F4Y0Ri
Z0FMQUFBSkFRWW1OUkUwTmhjQkZoUURGLzBKRFJNVERRTDNEUUdsL2xvSEN3OERTUThNQ1A1YkNC
VUFBQUFBQWdBQUFBQURiZ051QUE4QUlBQUFBUkVVQmlNaElpWTFFVFEyTXlFeUZnVVJGQVlqSVNJ
bU5SRTBOak1oTWhZVkEyNFdELzdjRHhZV0R3RWtEeGIrQUJZUC90d1BGaFlQQVNRUEZnTkovTndQ
RmhZUEF5UVBGaFlQL053UEZoWVBBeVFQRmhZUEFBQUJBQUFBQUFOdUEyNEFEd0FBQVJFVUJpTWhJ
aVkxRVRRMk15RXlGZ051RmcvODNBOFdGZzhESkE4V0EwbjgzQThXRmc4REpBOFdGZ0FBQUFBQkFH
TUFHZ0tkQTUwQUZRQUFDUUlXRkE4QkJpSW5BU1kwTndFMk1oOEJGaFFIQXAzKzBRRXZDd3RmQ2g0
TC9sZ0xDd0dvQ3g0S1h3c0xBd3YrMFA3UkN4NEtYd3NMQWFnS0hnc0JxQXNMWHdvZUN3QUJBRDRB
R2dKNUE1MEFGUUFBQ1FFR0lpOEJKalEzQ1FFbU5EOEJOaklYQVJZVUJ3SjUvbGdMSGd0ZkNnb0JN
UDdRQ2dwZkN4NExBYWdLQ2dIQy9sZ0xDMThLSGdzQkx3RXdDeDRLWHdzTC9sZ0xIZ29BQUFBQUF3
QUFBQUFEYmdOdUFCQUFRd0JmQUFBbE5UUW1Ld0VpQmgwQkZCWTdBVEkyTlJNMEp5NEJKeVlqSWdZ
SEJoWWZBUjRCTXpJMk56NEJOejRCTXpJV0ZSUUdCdzRCSFFFVUZqc0JNalkxTVRRMk56NEJOUmNV
Qnc0QkJ3WWpJaWN1QVNjbU5UUTNQZ0UzTmpNeUZ4NEJGeFlDQUFzSGJnZ0tDZ2h1Qnd1U0VoTThK
U1VsUm1rbEJBTUdTd0lHQXdRSUFoVVVDUWNhRUJzckdCb2ZRUW9JYmdjTEZ4UWhSdHdqSW5kUVVG
dGJVRkIzSXlJaUkzZFFVRnRiVUZCM0lpT2xiUWdMQ3dodENBc0xDQUdBSnlFaU1RNE5QRDBHRGdR
NUFnSUVBeG9WQmdVSUhSTVhHZ3dPUVNvVkJ3c0xCd29qREJJOVFtNWJVRkIzSWlNakluZFFVRnRi
VUZCM0lpTWpJbmRRVUFBQUFBQURBQUFBQUFOdUEyNEFId0F2QUVzQUFDVTFOQ1lyQVJFMEppc0JJ
Z1lkQVJRV093RVZJeUlHSFFFVUZqTWhNalkxQXpVMEppc0JJZ1lkQVJRV093RXlOZ1VVQnc0QkJ3
WWpJaWN1QVNjbU5UUTNQZ0UzTmpNeUZ4NEJGeFlDU1FvSU53c0h0d2dLQ2dnM053Z0tDZ2dCQUFn
S1NRc0hiZ2dLQ2dodUJ3c0JiaU1pZDFCUVcxdFFVSGNqSWlJamQxQlFXMXRRVUhjaUk2VmJDQW9C
SlFnS0NnaGNCd3UzQ2doYkNBc0xDQUlBV3dnS0NnaGJDQXNMNWx0UVVIY2lJeU1pZDFCUVcxdFFV
SGNpSXlNaWQxQlFBQUFBQUFFQUFBQkpBeVVEYmdBa0FBQUJGUlFHS3dFVkZBWXJBU0ltUFFFaklp
WTlBVFEyT3dFMU5EWTdBVElXSFFFek1oWVZBeVVoRnU0Z0YyNFdJTzRYSUNBWDdpQVdiaGNnN2hZ
aEFoSnRGeUR1RnlBZ0YrNGdGMjBYSU80WElDQVg3aUFYQUFBQkFFMEFBQU5xQTI0QU5RQUFBUjRC
RHdFT0FTOEJGUlFHS3dFaUpqMEJCd1ltTHdFbU5qOEJKeTRCUHdFK0FSOEJOVFEyT3dFeUZoMEJO
ellXSHdFV0JnOEJBMDhhRUE4bER6c2FtQ3NlU1I0cm1CbzdEeVVQRUJxWW1Cb1FEeVVQT3hxWUt4
NUpIaXVZR2pzUEpROFFHcGdCWHc4N0dqOGFFQTlZc0I0ckt4NndXQThRR2o4YU93OVlXQTg3R2o4
YUVBOVlzQjRyS3g2d1dBOFFHajhhT3c5WUFBTUFBQUFBQTI0RGJnQWNBQ3dBUXdBQUFUSVhIZ0VY
RmhVVUJ3NEJCd1lqSWljdUFTY21OVFEzUGdFM05qTVROVFFtS3dFaUJoMEJGQlk3QVRJMkp4TTBK
aWN1QVNzQklnWUhEZ0VWRXhRV093RXlOamNCdDF0UVVIY2lJeU1pZDFCUVcxdFFVSGNqSWlJamQx
QlFXMGtLQ0cwSUN3c0liUWdLQVFvREF3SUhCSDRFQndNREFna01DR2tJQ3dFRGJpTWlkMUJRVzF0
UVVIY2lJeU1pZDFCUVcxdFFVSGNpSS8wM2JRY01EQWR0Q0FzTHpRRmpBd1VDQWdJQ0FnSUZBLzZk
QmdrSkJnQUFBd0FKQUFBRDl3TzNBQThBSmdBOEFBQWxOVFFtS3dFaUJoMEJGQlk3QVRJMkp4TTBK
aWN1QVNzQklnWUhEZ0VWRXhRV093RXlOalVEQVJZVUJ3NEJJeUVpSmljbU5EY0JQZ0V6TWhZWEFr
a0tDRzRJQ2dvSWJnZ0tBUW9DQXdNSEJINEVCd01EQWdrTUNHb0hEQWdCdHdrS0NpSVQvSklUSWdv
S0NRRzNDU01VRkNNSnBXMEhEQXdIYlFnTEM5NEJCZ01HQWdJRUJBSUNCd1ArK3dZSEJ3WUNGdnpi
RVNZUkVSTVRFUkVtRVFNbEVSVVZFUUFBQUFFQVBnQmdBOElDbWdBVkFBQWxCd1lpSndrQkJpSXZB
U1kwTndFMk1oY0JGaFFIQThKZkN4NEwvdEgrMFFzZUMxOEtDZ0dvQ3g0TEFhZ0tDcjVlQ3dzQkwv
N1JDd3RlQ3g4S0FhZ0tDdjVZQ2g4TEFBRUFQZ0JDQThJQ2V3QVZBQUFKQVFZaUp3RW1ORDhCTmpJ
WENRRTJNaDhCRmhRSEE4TCtXQXNlQy81WUNncGZDeDRMQVM4Qkx3c2VDMThLQ2dIcC9sa0xDd0du
Q3g0TFhnc0wvdEVCTHdzTFhnc2VDd0FBQUFBRkFBQUFBQVNTQTI0QUF3QUlBQTRBRXdBWUFBQUJF
U01SQVJFakVUTUJGU0VSTXhFQkVTTVJNemNSSXhFekFXNlRBVzZTa2dKSisyNUpBdHlUazl1U2tn
RzMvdHNCSlFFay9iY0NTZjF1U1FOdS9Oc0NBUDVKQWJmYy9XMENrd0FHQUFEL3Z3UkpBNjRBQ3dB
WEFDTUFwQUQ4QVZRQUFBRTBKaU1pQmhVVUZqTXlOZ1UwSmlNaUJoVVVGak15TmhFMEppTWlCaFVV
RmpNeU5nY1ZGQVlQQVE0QkJ4NEJGeDRCRlJRR0J3NEJJeUltTHdFT0FRY09BUWNPQVNzQklpWXZB
UzRCSndjT0FTTWlKaWN1QVRVME5qYytBVGN1QVM4QkxnRTlBVFEyUHdFK0FUY3VBU2N1QVRVME5q
YytBVE15Rmg4QlBnRTNQZ0UzUGdFN0FUSVdId0VlQVJjM1BnRXpNaFlYSGdFVkZBWUhEZ0VISGdF
ZkFSNEJGUUVWRkFZakRnRUhIZ0VWRkFZSERnRWpJaVluSWdZaklpWWpEZ0VqSWlZbkxnRTFORFkz
TGdFbklpWTlBVFEyTno0Qk55NEJOVEEyTno0Qk16SVdGejRCTXpJV0Z6NEJQd0V5RmhjZUFURVVC
Z2NlQVJjZUFSVVJGUlFHQnc0QkJ4NEJGUlFHQnc0Qkl5SW1KeUlHSXlJbUl3NEJJeUltSnk0Qk5U
UTJOeTRCSnk0QlBRRTBOamMrQVRjdUFUVTBOamMrQVRNeUZoY3lOak15RmpNK0FUOEJNaFlYSGdF
VkZBWUhIZ0VYSGdFVkFnQldQRDFXVmowOFZnRzNLeDRlS3lzZUhpc3JIaDRyS3g0ZUs5d0hCbGdF
Q1FZTUdnNENBZ0lDQ1VjTUJBWUNRZ3NWREFJRkJnSUpCbW9HQ3dFTkN4VUxRd0lHQXdRR0FneEhB
d0VOR1EwR0N3TlhCZ2dJQlZrRENRWU1HZzBDQWdFRENFY05Bd1lEUVFzV0N3SUdCUUlLQm1vR0Nn
RU5DeFlLUXdNR0F3TUdBd3RIQWdJTUdnd0dDZ1JYQmdjQmJrd0pBd2tGQkJrQkFRVkFBZ1l1QkFR
SkJBVUlCQVV1QlFNL0JRSUJHZ1FHQ0FRSVRVMElCQWdHQkJvQkFnVS9Bd1V1QlFRSUJRUUpCQXdh
RGdRQ1FBVUJBUmtFQlFrRENVeE1DUU1KQlFRWkFRRUZRQUlHTGdRRUNRUUZDQVFGTGdVRFB3VUNB
Um9FQmdnRUNFMU5DQVFJQmdRYUFRSUZQd01GTGdVRUNBVUVDUVFNR2c0RUFrQUZBUUVaQkFVSkF3
bE1BYmM4VmxZOFBWVlY2QjRyS3g0ZUt5c0NaeDRzTEI0ZUt5dlNhZ1VMQVE0TEZRc1JJUkFDQmdN
REJnSU1TUUlDTXdVSkJCVXVGUVlJQ0FaWUF3a0dNd0lDQWdJTFJBMERCUU1RSUJFTEdBd05BUW9H
YWdVTEFRME1GUXNSSUJFQ0JnTURCZ0lNU1FJQ013VUpCQlV1RlFZSUNRWlhCQWtGTXdJQ0FnTUtS
UXdEQlFNUkh4RU1Gd3dOQVFvRy9zOVFCZ3NKRGdjSlBnZ0JBZ0VESmp3R0FRRUdQQ1lEQVFJQkJ6
OEpCdzRKQ3daUUJ3b0JDQThIQ0Q4SUF3RUNKanNIQVFFQkFSRWhEZ0lsQXdFRENEOElCdzhJQVFv
SEFrbFFCZ29CQ0E4SENUNElBUUlCQXlVN0JnRUJCanNsQXdFQ0FRYy9DUWNQQ0FFS0JsQUhDZ0VJ
RHdjSVB3Z0JBZ0VDSmpzR0FRRVFJUThCSlFNQkFnRUlQd2dIRHdnQkNnY0FBQUlBQUFCSkJBQUR0
d0FvQUVzQUFBRVZGQVlqSVNJbU5SRTBOak1oTWhZZEFSUUdJeUVpQmhVUkZCWXpJVEkyUFFFME5q
c0JNaFlWRXhFVUJpTWlKaThCQVE0Qkl5SW1Md0V1QVRVME5qY0JKeTRCTlRRMk15RXlGaFVESldG
RS9pVkZZR0JGQVpJSUNnb0kvbTRtTmpZbUFkc21OUXNJSkFnTDJ4WVBCdzBGWmY2TUF3Y0VBd2NE
UVFJRUJBSUJkV1VGQmhZUEFTUVBGZ0dsdDBSaFlVUUIyMFJoQ3dna0NBbzJKdjRsSmpZMkpyY0hD
d3NIQWUzKzNBOFdCZ1ZsL29zREF3TURRUU1IQXdRSEFnRjFaUVVOQnc4V0ZnOEFBQUlBQUFCSkF5
VURiZ0FQQUI4QUFBRWhJZ1lWRVJRV015RXlOalVSTkNZWEVSUUdJeUVpSmpVUk5EWXpJVElXQW9E
K0pTWTJOaVlCMnlZMU5YOWhSUDRsUldCZ1JRSGJSR0VESlRZbS9pVW1OalltQWRzbU5seitKVVJo
WVVRQjIwUmhZUUFEQUNYL3R3UGJBN2NBRWdBd0FHY0FBQVUwSmlNaUpqVTBKaU1pQmhVVUZqTXlO
alVsSVNZbkxnRW5KalUwSnk0Qkp5WWpJZ2NPQVFjR0ZSUUhEZ0VIQmdjaEZBWWpJUlFHSXlJbU5T
RWlKalUyTno0Qk56WTFORGMrQVRjMk55NEJOVFEyTXpJV0ZSUUdCeFlYSGdFWEZoVVVGeDRCRnhZ
WEFna0ZCQ0l3QmdNRUJqc3FCQVgraEFMbUpod2RKZ2tLRFEwMktpazRPQ2txTmcwTkNna21IUndt
QTA0ckh2OEFWanc4VnY4QUhpc2ZJU0ExRVJBUkVUNHRMVGtDQXlBWEZ5QURBamt0TFQ0UkVSQVJO
U0FoSHhJRUJUQWlCQVVGQkNrN0JRU2tMRE16ZGtSRFRSc2dJRGNTRXhNU055QWdHMDFEUkhZek15
d2VLenhXVmp3ckhoc25KM0ZOVFdncEtTcEZHUmtKQlFzR0Z5QWdGd1lMQlFrWkdVVXFLU2xvVFUx
eEp5Y2JBQUVBQVFBQUEyMERiQUJpQUFBQkZ4NEJCdzRCRHdFWEZnWUhEZ0V2QVFjT0FRY2lCaU1p
Smk4QkJ3NEJKeTRCTHdFSEJpWW5MZ0UvQVNjdUFTY21OajhCSnk0Qk56NEJQd0VuSmpZM1BnRWZB
VGMrQVRjMkZoOEJOejRCRng0Qkh3RTNOaFlYSGdFUEFSY2VBUmNXQmdjREVrOElCZ0lERUF0c0h3
TUdDQWdWQzJvY0FoQUxBd1VEQ0E4R1RVMElGZ29MRUFJY2Fnc1ZDQWdHQXg1ckN4QURBZ1VKVGs0
SkJRSURFQXRySGdNR0NBZ1ZDMm9jQWhBTENoWUlUVTBJRlFzTEVBSWNhZ3NWQ0FnR0F4OXNDeEFE
QWdZSUFiZE5DQllLQ3hBQ0hHb0xGUWdJQmdNZWF3c1FBd0VIQms1T0NRVUNBeEFMYXg0REJnZ0lG
UXRxSEFJUUN3b1dDRTFOQ0JVTEN4QUNIR29MRlFnSUJnTWZiQXNQQXdNR0NFOVBDQVlEQXc4TGJC
OERCZ2dJRlF0cUhBSVFDd3NWQ0FBQUFBRUFBd0FBQXlJREpRQWVBQUFCRmdZSEFSRVVCZ2NPQVNN
aUppOEJMZ0UxRVFFdUFUYytBVE1oTWhZWEF5SUVCQWorNWd3S0JBY0VCdzBGa3dVRi91WUlCQVFF
RWd3QzJ3c1NCUU1PQ2hZSS91ZitXQXdTQkFFQ0JRYVNCUTRIQVJZQkdRZ1dDZ29ORFFvQUFBRUFB
QUFBQTI0RGJnQlpBQUFCQnhjM1BnRVhIZ0VWRVJRR0l5RWlKaWNtTmo4Qkp3Y1hIZ0VIRGdFaklT
SW1OUkUwTmpjMkZoOEJOeWNIRGdFaklpWW5MZ0UxRVRRMk15RXlGaGNXQmc4QkZ6Y25MZ0UzUGdF
eklUSVdGUkVVQmdjT0FTTWlKaWNDM2N2TFVnZ1dDZ29ORmcvL0FBc1NCUVFFQ0ZQTHkxSUlCUVVF
RWd2L0FBOFdEUW9LRmdkVHlzcFRCUTBIQkFjRENnMFdEd0VBQ3hJRUJRVUlVc3ZMVXdnRUJBVVND
d0VBRHhZTkNnTUhCQWNOQmdLQ3k4dFNDQVVGQkJJTC93QVBGZzBLQ2hZSFU4cktVd2NXQ2dvTkZn
OEJBQXNTQkFVRkNGTEx5MU1GQlFFQkJSSUxBUUFQRmcwS0NoWUlVc3ZMVWdnV0Nnb05GZy8vQUFz
U0JRRUJCUVVBQUFBR0FBRC90d1JKQTdjQUdnQTJBRUlBWGdCNEFJUUFBQUVPQVFjaklpWTFORGMr
QVRjMk16SVdNekkyTnc0QkZSUVdGd0VVQmlNaElpWTFORGMrQVRjMk16SVdNekkyTXpJWEhnRVhG
aFVCRkFZaklpWTFORFl6TWhZQkZBY09BUWNHSXlJbkxnRW5KalUwTno0Qk56WXpNaGNlQVJjV0JS
UUdLd0V1QVNjK0FUVTBKaWNlQVRNeU5qTXlGeDRCRnhZREZBWWpJaVkxTkRZek1oWUJVeTFPSFV3
clJBRUJEUTRQR3dsU09SUW1FZ0VCR0JZQ1pGUkYvZzFGVkFnSEt5WW5QdzlvVmxWcERqOG5KaXNJ
Qi8yM1ZqMDhWbFk4UFZZQmtoRVNPeWdvTFM0b0tEc1NFUkVTT3lnb0xpMG9LRHNTRVFGSlJDdE1I
VTR0RnhnQ0FSSW1GRHBSQ1J3T0RnMEJBVWxXUEQxV1ZqMDhWZ0czQVNZaUt6QVJJaUZER1Jrd0Jn
Y0tFZ29uU3lEK2xFWk9Ua1l3T2psaUlTRlBUeUVoWWprNk1BTGFQVlpXUFR4V1Z2N29MU2dvUEJF
UkVSRThLQ2d0TGlnb094RVNFaEU3S0NpdU1Dc2lKZ0VnU3ljS0Vnb0hCakFaR1VNaElnRkxQVlpX
UFR4V1ZnQUFBQUFEQUFrQUNRT3VBNjRBSXdCSUFJQUFBQUUwSmk4QkxnRWpJZ1lISGdFVkZBWWpJ
aVluRGdFVkZCWWZBUjRCTXpJMlB3RStBUUUwSmk4QkxnRWpJZ1lQQVE0QkZSUVdId0VlQVRNeU5q
Y3VBVFUwTmpNeUZoYytBVFVCRkFZUEFRNEJJeUltTHdFdUFUVTBOamNuRGdFaklpWXZBUzRCTlRR
MlB3RStBVE15Rmg4QkhnRVZGQVlIRno0Qk16SVdId0VlQVFOQUNBaDNDQlFMREJRSkRoc2dGeFVj
RHdrSkNBaDFDQlFMQ3hRSVZBZ0kvbTRJQ0hVSUZRb0xGQWhVQ0FnSUNIY0hGUXNNRkFrT0d5QVdG
aHdPQ1FvQ0FCb1hWQmM4SVNFOEYzWVhHUnNZTXhjK0lTRTlGM2NYR1JvWFZCYzhJU0U4RjNZWEdC
b1lNaGc5SWlFOUYzY1hHUUVBQ3hRSWR3Y0pDZ2tPSEJZV0lCc09DUlFOQ3hRSWRnZ0lDQWRVQnhR
Qm5nc1VDSFlJQ0FnSFZBZ1RDd3NVQ0hjSUJ3Z0pEeHdWRnlBYkRna1VEZjV0SVR3WFV4Z1lHUmgy
Rnp3aElqNFhNeGdiR1JkM0dEd2hJRDBYVXhjWkdSaDJGejBnSWo0WU1oZ2FHQmgyR0R3QUFBQUFB
d0FBQUVrRGJnTWxBQThBSHdBdkFBQWxGUlFHSXlFaUpqMEJORFl6SVRJV0VSVVVCaU1oSWlZOUFU
UTJNeUV5RmhFVkZBWWpJU0ltUFFFME5qTWhNaFlEYmhZUC9Od1BGaFlQQXlRUEZoWVAvTndQRmhZ
UEF5UVBGaFlQL053UEZoWVBBeVFQRnJkSkR4WVdEMGtQRlJVQkZVa1BGUlVQU1E4V0ZnRVdTUThX
Rmc5SkR4WVdBQW9BQUFCSkE3Y0RiZ0FRQUNBQU1RQkJBRkVBWWdCeUFJTUFsQUNrQUFBbE5UUW1L
d0VpQmgwQkZCWTdBVEkyTlQwQk5DWXJBU0lHSFFFVUZqc0JNallGTlRRbUt3RWlCaDBCRkJZN0FU
STJOUUUxTkNZckFTSUdIUUVVRmpzQk1qWUZOVFFtS3dFaUJoMEJGQlk3QVRJMkJUVTBKaXNCSWdZ
ZEFSUVdPd0V5TmpVQk5UUW1Ld0VpQmgwQkZCWTdBVEkyQlRVMEppc0JJZ1lkQVJRV093RXlOalU5
QVRRbUt3RWlCaDBCRkJZN0FUSTJOVGNSRkFZaklTSW1OUkUwTmpNaE1oWUJKUXNJdHdjTEN3ZTND
QXNMQ0xjSEN3c0h0d2dMQVNRS0NMY0lDZ29JdHdnSy90d0xDTGNIQ3dzSHR3Z0xBU1FLQ0xjSUNn
b0l0d2dLQVNVTENMWUlDd3NJdGdnTC90c0tDTGNJQ2dvSXR3Z0tBU1VMQ0xZSUN3c0l0Z2dMQ3dp
MkNBc0xDTFlJQzBrMkp2MEFKVFkySlFNQUpqYWxiUWdMQ3dodENBc0xDTnR1QndzTEIyNElDZ3JU
YlFnTEN3aHRDQXNMQ0FHMmJnZ0tDZ2h1QndzTDFHNEhDd3NIYmdnS0N0TnRDQXNMQ0cwSUN3c0lB
Ylp1Q0FvS0NHNEhDd3ZVYmdjTEN3ZHVDQW9LQ050dUNBb0tDRzRIQ3dzSHQvMlRKalkySmdKdEpq
WTJBQUFCQUFBQkFBSkpBa2tBRlFBQUFSUUdCd0VPQVNNaUppY0JMZ0UxTkRZeklUSVdGUUpKQmdY
L0FBVU5Cd2dOQmY4QUJRWVdEd0lBRHhVQ0pRZ05CZjhBQlFZR0JRRUFCUTBJRHhVVkR3QUFBQUVB
QUFEYkFra0NKUUFVQUFBQkZBWWpJU0ltTlRRMk53RStBVE15RmhjQkhnRUNTUlVQL2dBUEZnWUZB
UUFGRFFnSERRVUJBQVVHQVFBUEZoWVBCdzRGQVFBRkJnWUYvd0FGRGdBQkFDVUFrZ0Z1QXRzQUZR
QUFBUkVVQmlNaUppY0JMZ0UxTkRZM0FUNEJNeklXRlFGdUZnOEhEUWIvQUFVRkJRVUJBQVlOQnc4
V0FyZitBQThXQmdVQkFBVU9Cd2NOQmdFQUJRVVZEd0FBQUFFQUFBQ1NBVWtDMndBVkFBQUJGQVlI
QVE0Qkl5SW1OUkUwTmpNeUZoY0JIZ0VWQVVrR0JmOEFCUTBIRHhZV0R3Y05CUUVBQlFZQnR3Y09C
ZjhBQlFZV0R3SUFEeFVGQmY4QUJnMEhBQUFBQWdBQUFDVUNTUU5KQUJVQUt3QUFBUlFHQndFT0FT
TWlKaWNCTGdFMU5EWXpJVElXRlRVVUJpTWhJaVkxTkRZM0FUNEJNeklXRndFZUFSVUNTUVlGL3dB
RkRRY0lEUVgvQUFVR0ZnOENBQThWRlEvK0FBOFdCZ1VCQUFVTkNBY05CUUVBQlFZQlNRY05CdjhB
QlFVRkJRRUFCZzBIRHhZV0Q5d1BGaFlQQncwRkFRQUZCZ1lGL3dBRkRRY0FBQUFBQVFBQUFDVUNT
UUZ1QUJVQUFBRVVCZ2NCRGdFaklpWW5BUzRCTlRRMk15RXlGaFVDU1FZRi93QUZEUWNJRFFYL0FB
VUdGZzhDQUE4VkFVa0hEUWIvQUFVRkJRVUJBQVlOQnc4V0ZnOEFBQUFCQUFBQ0FBSkpBMGtBRlFB
QUFSUUdJeUVpSmpVME5qY0JQZ0V6TWhZWEFSNEJGUUpKRlEvK0FBOFdCZ1VCQUFVTkNBY05CUUVB
QlFZQ0pROFdGZzhIRFFVQkFBVUdCZ1gvQUFVTkJ3QUFBQWNBQUFBQUJBQURKUUFMQUJjQUxRQTVB
RVVBVVFCdEFBQVROQ1lqSWdZVkZCWXpNallUTkNZaklnWVZGQll6TWpZWE56WW1KekVtQmc4QkRn
RUhCaFlYRmpZM05pWW5KVFFtSXlJR0ZSUVdNekkyQVRRbUl5SUdGUlFXTXpJMkJUUW1JeUlHRlJR
V016STJGeFFHQnc0Qkl5RWlKaWN1QVRVME56NEJOell6TWhjZUFSY1dGZHNxSHg0ckt4NGZLbTRy
SGg0ckt4NGVLL1U1QkE4UERoc0RPaUkyQ1F3dUxDeFBDd2taSEFGNUt4NGZLaW9mSGl2K2tpc2VI
aXNySGg0ckFRQXJIaDRyS3g0ZUs3Y3BLQVVRQ2Z6ZUNSQUZLQ2tvS1l0ZFhXcHFYVjJMS1NnQkpS
NHJLeDRmS3lzQkh4NHJLeDRmS3l2MDJnOGFCQU1QRDlvREt5TXNUd3NNTGl3alFCUVRIaXNySGg4
ckt3R01IeW9xSHg0ckswOGVLeXNlSHlzcjRVcU1QZ2dKQ1FnOWpVcHBYbDJMS0NrcEtJdGRYbWtB
QUFBQUFnQWwvN2NEMndPM0FCSUFTUUFBQlRRbUl5SW1OVFFtSXlJR0ZSUVdNekkyTlNVVUJpTWhG
QVlqSWlZMUlTSW1OVFkzUGdFM05qVTBOejRCTnpZM0xnRTFORFl6TWhZVkZBWUhGaGNlQVJjV0ZS
UVhIZ0VYRmhjQ0NRVUVJakFHQXdRR095b0VCUUhTS3g3L0FGWThQRmIvQUI0ckh5RWdOUkVRRVJF
K0xTMDVBZ01nRnhjZ0F3STVMUzArRVJFUUVUVWdJUjhTQkFVd0lnUUZCUVFwT3dVRXBCNHJQRlpX
UENzZUd5Y25jVTFOYUNrcEtrVVpHUWtGQ3dZWElDQVhCZ3NGQ1JrWlJTb3BLV2hOVFhFbkp4c0FB
QUFHQUFEL3R3TnVBN2NBRXdBY0FDWUFOd0JIQUZnQUFBRWVBUlVSRkFZaklTSW1OUkUwTmpNaE1o
WVhCeFV6TGdFdkFTNEJFeEVqSWlZOUFTRVJJUUUwTmpNaE1oWWRBUlFHSXlFaUpqMEJCVElXSFFF
VUJpTWhJaVk5QVRRMk13VXlGaDBCRkFZaklTSW1QUUUwTmpNaEEwY1FGeUFYL1FBWElDQVhBZ0FY
TnhCTTF3TUhBN0lERHRYdUZ5RCtTUUxjL2JZTENBR1NDQW9LQ1A1dUNBc0JwUWdLQ2dqK2JnZ0xD
d2dCa2dnS0NnaitiZ2dMQ3dnQmtnTGVFRGNYL1c0WElDQVhBNUlYSUJjUUo5Y0lEUU96QXdmOG1R
SkpJQmZ1L0pJQjdnY0xDd2NsQ0FvS0NDV0FDd2drQ0FvS0NDUUlDNU1LQ0NRSUN3c0lKQWdLQUFB
QUFnQWFBSFVDUWdLdkFDUUFTUUFBSlJRR0R3RU9BU01pSmljQkxnRTFORFkzQVQ0Qk16SVdId0Vl
QVJVVUJnOEJGeDRCRlRNVUJnOEJEZ0VqSWlZbkFTNEJOVFEyTndFK0FUTXlGaDhCSGdFVkZBWVBB
UmNlQVJVQlpnTUNIUU1IQXdRSEF2NzFBZ01EQWdFTEFnY0VBd2NESFFJREF3TGg0UUlEM0FNREhR
SUhCQU1IQS83MkF3TURBd0VLQXdjREJBY0NIUU1EQXdQaDRRTURwUVFIQXh3REF3TURBUW9EQndN
RUJ3SUJDd0lEQXdJZEFnZ0RBd2dDNGVBREJ3TUVCd01jQXdNREF3RUtBd2NEQkFjQ0FRc0NBd01D
SFFJSUF3TUlBdUhnQXdjREFBQUFBZ0FIQUhVQ0x3S3ZBQ1FBU1FBQUFSUUdCd0VPQVNNaUppOEJM
Z0UxTkRZL0FTY3VBVFUwTmo4QlBnRXpNaFlYQVI0QkZUTVVCZ2NCRGdFaklpWXZBUzRCTlRRMlB3
RW5MZ0UxTkRZL0FUNEJNeklXRndFZUFSVUJWQU1EL3ZZREJ3TUVCd0lkQWdRRUF1SGhBZ1FFQWgw
Q0J3UURCd01CQ2dNRDJ3TUMvdlVDQndRREJ3TWNBd01EQStEZ0F3TURBeHdEQndNRUJ3SUJDd0lE
QVpJREJ3UCs5Z01EQXdNY0F3Y0VBd2NENE9FQ0NBTURDQUlkQWdNREF2NzFBZ2NFQXdjRC92WURB
d01ESEFNSEJBTUhBK0RoQWdnREF3Z0NIUUlEQXdMKzlRSUhCQUFBQVFBYUFIVUJaZ0t2QUNRQUFB
RVVCZzhCRng0QkZSUUdEd0VPQVNNaUppY0JMZ0UxTkRZM0FUNEJNeklXSHdFZUFSVUJaZ01DNGVF
Q0F3TUNIUU1IQXdRSEF2NzFBZ01EQWdFTEFnY0VBd2NESFFJREFvQURDQUxoNEFNSEF3UUhBeHdE
QXdNREFRb0RCd01FQndJQkN3SURBd0lkQWdjRUFBQUFBQUVBQndCMUFWUUNyd0FrQUFBQkZBWUhB
UTRCSXlJbUx3RXVBVFUwTmo4Qkp5NEJOVFEyUHdFK0FUTXlGaGNCSGdFVkFWUURBLzcyQXdjREJB
Y0NIUUlFQkFMaDRRSUVCQUlkQWdjRUF3Y0RBUW9EQXdHU0F3Y0QvdllEQXdNREhBTUhCQU1IQStE
aEFnZ0RBd2dDSFFJREF3TCs5UUlIQkFBQUFBQUNBQUFBQUFSSkE3Y0FEd0F1QUFBQkVUUW1JeUVp
QmhVUkZCWXpJVEkyRXhFVUJpTWhGQllWRkFZaklTSW1OVFEyTlNFaUpqVVJORFl6SVRJV0ZRUUFD
d2Y4YlFjTEN3Y0Rrd2NMU1RZbC9za2tGUS8rMnc4VkpQN0pKVFkySlFPVEpUWUJnQUhiQ0FzTENQ
NGxCd3NMQWVMOWt5WTJKRG9QRHhZV0R3ODVKVFltQW0wbU5qWW1BQUFBQUFNQUFBQkpBcElEYmdB
TUFCd0FMQUFBSlRRbUl5SUdGUlFXTXpJMk5UY1JOQ1lqSVNJR0ZSRVVGak1oTWpZVEVSUUdJeUVp
SmpVUk5EWXpJVElXQVc0V0R3OFZGUThQRnRzTEIvNGtCd3NMQndIY0J3dEpOU2IrSkNVMk5pVUIz
Q1kxa2c4V0ZnOFBGUlVQWEFJa0NBc0xDUDNjQ0FzTEFpejlreVkyTmlZQ2JTWTJOZ0FBQUFBQ0FB
QUFBQU51QTI0QUhBQTVBQUFCSWdjT0FRY0dGUlFYSGdFWEZqTXlOejRCTnpZMU5DY3VBU2NtSXdF
VUJ3NEJCd1lqSWljdUFTY21OVFEzUGdFM05qTXhNaGNlQVJjV0FiZEFPVGxVR1JnWUdWUTVPVUJB
T1RoVkdCa1pHRlU0T1VBQnR5TWlkMUJRVzF0UVVIY2pJaUlqZDFCUVcxdFFVSGNpSXdMdUdSaFZP
RGxBUURrNVZCa1lHQmxVT1RsQVFEazRWUmdaL3NsYlVGQjNJaU1qSW5kUVVGdGJVRkIzSWlNaklu
ZFFVQUFBQUFBSUFDWC8yd1BiQTdjQUN3QVhBQ01BTHdBN0FFY0FWQUJoQUFBbEZBWWpJaVkxTkRZ
ek1oWUZGQVlqSWlZMU5EWXpNaFlCRkFZaklpWTFORFl6TWhZQkZBWWpJaVkxTkRZek1oWUJGQVlq
SWlZMU5EWXpNaFlCRkFZaklpWTFORFl6TWhZQkZBWWpJaVkxTkRZek1oWVZCUlFHSXlJbU5UUTJN
eklXRlFFdEt4OGVLeXNlSHlzQkhDc2VIaXNySGg0ci9tNHJIaDhxS2g4ZUt3S3ZLeDRmS3lzZkhp
djkyVFltSlRZMkpTWTJBcHdxSHg0ckt4NGZLdjZUUVMwdFFVRXRMVUVCTDB3ME5rcEtOalJNbWg0
ckt4NGZLaXFVSHlzckh4NHJLd0YwSHlvcUh4NHJLLzdGSGlzckhoOHFLZ0lhSlRZMkpTWTJOdjYr
SHlvcUh4NHJLd0YwTFVGQkxTNUFRQzUyTlV0TE5UVkxTelVBQUFBQUFRQUFBQUFEYmdOdUFCc0FB
QUVVQnc0QkJ3WWpJaWN1QVNjbU5UUTNQZ0UzTmpNeUZ4NEJGeFlEYmlNaWQxQlFXMXRRVUhjaklp
SWpkMUJRVzF0UVVIY2lJd0czVzFCUWR5SWpJeUozVUZCYlcxQlFkeUlqSXlKM1VGQUFBQUFBQWdB
TkFFa0R0d0txQUJVQUpRQUFDUUVHSWk4QkpqUS9BU2NtTkQ4Qk5qSVhBUllVQndFVkZBWWpJU0lt
UFFFME5qTWhNaFlCVHY3MkJnOEZIUVVGNGVFRkJSMEZEd1lCQ2dZR0Fta0xCLzNiQ0FvS0NBSWxC
d3NCaGY3MkJnWWNCZzhHNE9FRkVBVWRCUVgrOVFVUEJ2NzdKUWNMQ3djbENBb0tBQU1BSHdBTEJD
b0RHZ0FWQUNZQVBBQUFKUWNHSWljQkpqUTNBVFl5SHdFV0ZBOEJGeFlVQndFRERnRXZBUzRCTnhN
K0FSOEJIZ0VIQ1FFR0lpOEJKalEvQVNjbU5EOEJOaklYQVJZVUJ3RmhIUVlQQmY3MUJRVUJDd1VQ
QmgwRkJlSGhCUVVCVWRVQ0RRY2tCd2NDMVFJTkJ5UUhCd0lCZVA3MUJROEdIQVlHNE9BR0Jod0dE
d1VCQ3dVRmx4d0dCZ0VLQmc4RkFRc0ZCUjBGRUFYaDRBWVBCZ0ppL1I0SEJ3SUtBZzBIQXVJSENB
SUtBZzRIL296KzlnWUdIQVlQQnVEaEJSQUZIUVVGL3ZVRkR3WUFBQUFFQUFBQUFBSkpBMjRBQ3dB
WEFDTUFXd0FBTnpRbUl5SUdGUlFXTXpJMkVUUW1JeUlHRlJRV016STJCVFFtSXlJR0ZSUVdNekky
TnhRR0J4UUhEZ0VIQmdjT0FSMEJIZ0VWRkFZaklpWTFORFkzRVM0Qk5UUTJNeklXRlJRR0J4RStB
VGMrQVRjdUFUVTBOak15RmhXbElSWVhJQ0FYRmlFaEZoY2dJQmNXSVFGdElCY1dJQ0FXRnlBM0ho
a1pHVWtyS2lWRU5CZ2VRQzB1UUI0WkdSNUFMaTFBSGhnVkxoVlBXQUVaSGtBdExrQnVGaUVoRmhj
Z0lBS3BGeUFnRnhjZ0lESVhJQ0FYRnlBZ0Z4OHlEazB5TVQ0UkVBd1ZJeW9PRHpJZUxrQkFMaDR5
RHdIVURqTWVMVUZCTFI0ekR2N2tDdzhIR1Q5YURqSWZMVUZCTFFBQUFnQTZBRWtDUndNbEFCQUFR
d0FBSlJVVUJpc0JJaVk5QVRRMk93RXlGaFVURkFZSERnRVZNUlFHS3dFaUpqMEJORFkzUGdFMU5D
WWpJZ1lIRGdFSERnRWpJaVl2QVM0Qk56NEJNeklYSGdFWEZoVUJrZzBLaVFrT0RnbUpDZzIxVnlr
WkhnMEtpUWtNVWlVaEhqWWlGQ0VKQ2hvWkJBa0ZCQWNEWGdjREJDK0RWeTR2TGtvWUYrbUpDUTRP
Q1lrS0RRMEtBVmRSVFJjT01Rd0pFaFlLR1RSVEVROGhHeGdsQ2djSEd5QUVCUU1DUndZUkNFeE1F
aEU5S2lveEFBQUFBZ0FBQUFBQzJ3TnVBQWtBSndBQUFSRWhFVDRCTno0Qk5STVJGQWNPQVFjR0J3
NEJJeUltSnlZbkxnRW5KalVSTkRZeklUSVdGUUp1L3dBV1FTSXVXVzAwTllFNE53VUVCd1FFQ0FN
Rk9EZUJOVFVXRHdLU0R4VUJrZ0Z1L1hZTUtCc2thejRCdC81SldrbElhQndkQXdFQ0FnRURIUnhv
U0VsYUFiY1BGaFlQQUFBQUF3QUFBVzRESlFKSkFBOEFId0F3QUFBVEZSUUdLd0VpSmowQk5EWTdB
VElXQlJVVUJpc0JJaVk5QVRRMk93RXlGZ1VWRkFZckFTSW1QUUUwTmpzQk1oWVYyeUFXYmhjZ0lC
ZHVGaUFCSlNBWGJoWWdJQlp1RnlBQkpTRVdiaGNnSUJkdUZpRUNFbTBYSUNBWGJSY2dJQmR0RnlB
Z0YyMFhJQ0FYYlJjZ0lCZHRGeUFnRndBQUFBTUFBQUJKQU5zRGJnQVBBQjhBTHdBQU54VVVCaXNC
SWlZOUFUUTJPd0V5RmhFVkZBWXJBU0ltUFFFME5qc0JNaFlSRlJRR0t3RWlKajBCTkRZN0FUSVcy
eUFXYmhjZ0lCZHVGaUFnRm00WElDQVhiaFlnSUJadUZ5QWdGMjRXSU81dUZ5QWdGMjRXSVNFQkRt
MFhJQ0FYYlJjZ0lBRU9iaGNnSUJkdUZ5QWdBQUFBQUFJQUFBQUFBMjREYmdBY0FEUUFBQUV5Rng0
QkZ4WVZGQWNPQVFjR0l5SW5MZ0VuSmpVME56NEJOell6RXo0Qk5UUW1KeVVtSWdjT0FSVVJGQllY
SGdFek1qWTNBYmRiVUZCM0lpTWpJbmRRVUZ0YlVGQjNJeUlpSTNkUVVGdmJDUW9LQ2Y3SkNCUUlD
UWtKQ1FRSkJRVUpCQU51SXlKM1VGQmJXMUJRZHlJakl5SjNVRkJiVzFCUWR5SWovaWtGRVFvS0VR
UzNCUVFGRVFyK2tnb1JCUUlEQXdNQUFnQUFBQUFEYmdOdUFCOEFMd0FBQVJFMEppTWhJZ1lIQmhZ
ZkFRRUdGQjhCRmpJM0FSY2VBVE15TmpjK0FUVVRFUlFHSXlFaUpqVVJORFl6SVRJV0F0c1ZELzd1
REJJRUJBUUlVdjdQQ3dzNkN4NExBVEZTQlE0SEF3Z0RDZ3lUWVVUOTNFVmdZRVVDSkVSaEFhVUJF
ZzhWREFvS0ZnaFMvczhMSGdzNkN3c0JNVklHQlFJQkJCSU1BU1Q5M0VWZ1lFVUNKRVJoWVFBRkFB
RC90d051QTdjQUNBQWFBQ3NBUEFCTkFBQUJIZ0VYSVJFZUFSY0RJUkVVQmlNaElpWTFFVFEyTXlF
UkZCWVROVFFtSXlFaUJoMEJGQll6SVRJMk5UMEJOQ1lqSVNJR0hRRVVGak1oTWpZMVBRRTBKaU1o
SWdZZEFSUVdNeUV5TmpVRFJ3UUlCUDd5QmdvRkp3RTNJQmY5QUJjZ0lCY0J5U0J5Q2dqK2JnZ0xD
d2dCa2dnS0NnaitiZ2dMQ3dnQmtnZ0tDZ2orYmdnTEN3Z0JrZ2dLQXFjRUN3WUJEZ1FJQlA2NS9h
VVhJQ0FYQTVJWElQN0pGeUQrWENRSUNnb0lKQWdMQ3dpU0pBZ0xDd2drQ0FvS0NKSWxCd3NMQnlV
SUNnb0lBQUFBQUFNQUFBQUFBNUlEYmdBTEFCd0FYQUFBTnpRbUl5SUdGUlFXTXpJMkV4RVVCaXNC
SWlZMUVUUTJPd0V5RmhVaEZBWUhIZ0VWRmdZSEZoUUhEZ0VIRmdZSERnRXJBU0ltSnk0Qkl5NEJO
UkUwTmpjK0FUYytBVGMrQVRjK0FUYytBVE15RmhVVUJnY09BUWN6TWhZVmtoVVBFQlVWRUE4VlhC
WVBwQThXRmcra0R4WUNwQkVPQlFNQkRBMEZCUVFRQ3dNUEVCSTVKa2srY1M0Yk1BOFBGaFVPRUVJ
VEVCME5FUTBGQlE4U0JRNEhZQ0FWQ3dVSUJKOHNRYmNQRlJVUEVCVVZBVFQra3c4V0ZnOEJiUThX
Rmc4V0xoRU5HQVlXSnhFUUloRVFIQW9nTkJNV0Z5QVFDUkFCRlE4QmJnNFZBZ0ZOR0JRa0RSRXdH
Umt5RWdVR2NpRWpOQllLRFE1QkxRQUFBQUFEQUFEL3R3T1NBeVVBREFBY0FGd0FBQk1VQmlNaUpq
VTBOak15RmhVVEVUUW1Ld0VpQmhVUkZCWTdBVEkySlI0QkZSUUdLd0VlQVJjZUFSVVVCaU1pSmlj
dUFTY3VBU2N1QVNjdUFTY3VBVFVSTkRZelBnRTNQZ0U3QVI0QkZ4NEJCeDRCRnhZVUJ4NEJCeFFH
QjVJVkR4QVZGUkFQRlZ3V0Q2UVBGaFlQcEE4V0FvVU9FVUVzbndRSUJRc1ZJR0FIRGdVU0R3VUZE
UkVOSFJBVFFoQU9GUllQRHpBYkxuRStTU1k1RWhBUEF3c1FCQVVGRFF3QkF3VUNiZzhXRmc4UEZS
VVAvdHNCYmc4VkZRLytrZzhWRldRUUxoY3NRZzBPQ1JZMUl5RnhCZ1VTTWhrWk1CRU5JeFFaVFFF
QkZROEJiZzhWQVE4S0R5RUJGaFlUTkNBTEd4QVJJaEFSS0JVR0dBNEFBUUFBQUFBRFhBTnVBRHNB
QUFFaEhnRVZGQWNPQVFjR0l5SW5MZ0VuSmpVME56NEJOell6TWhjZUFSY1dGd2N1QVNNaUJ3NEJC
d1lWRkJjZUFSY1dNekkzUGdFM05qY2pOUUczQVo0REJCNGViMDVOWDF0UVVIY2pJaUlqZDFCUVd5
d3BLVW9nSVIxM0dWWkFPREl4U2hZVkZSWktNVEk0UVM0dE93OFBCUGtCOWhFakZWNVBUM0VnSUNJ
amQxQlFXMXRRVUhjaUl3Z0lIaFVWRzNNWUxCWVdTekl6T1RveU0wc1dGUlFWT2lFaEdKY0FCZ0FB
LzdjRGJnTzNBQk1BSEFBbUFEc0FVQUJnQUFBQkhnRVZFUlFHSXlFaUpqVVJORFl6SVRJV0Z3Y1ZN
eTRCTHdFdUFSTVJJeUltUFFFaEVTRUJQZ0VmQVI0QkR3RVhGZ1lQQVFZbUx3RW1ORGNoRmhRUEFR
NEJMd0V1QVQ4Qkp5WTJQd0UyRmhjRExnRTNFejRCSHdFZUFRY0REZ0VuQTBjUUZ5QVgvUUFYSUNB
WEFnQVhOeEJNMXdNSEE3SUREdFh1RnlEK1NRTGMvZTBGRHdZZEJnSUVhR2dFQWdZZEJnOEZnUU1E
QWt3REE0SUVEd1lkQmdNRmFHZ0ZBd1lkQmc4RTRRY0pBVThDREFja0NBa0NUd0VNQ0FMZUVEY1gv
VzRYSUNBWEE1SVhJQmNRSjljSURRT3pBd2Y4bVFKSklCZnUvSklDQUFZQ0JSVUZEd2FMaXdZUEJC
WUZBZ2FzQlF3RkJRd0ZyQVlDQlJZRUR3YUxpd1lQQlJVRkFnYitUUUVNQ0FIYkJ3a0NCUUlNQi80
bENBZ0JBQUFCQUJML3lRUHVBNThBT2dBQUFSUUhEZ0VIQmlNaUp5NEJKeVkxTkRjK0FUYzJOeFVH
Qnc0QkJ3WVZGQmNlQVJjV016STNQZ0UzTmpVMEp5NEJKeVluTlJZWEhnRVhGaFVEN2ljbmhscGFa
bVphV29Zbkp5RWdjVTFOV1Q4MU5rNFdGeDBkWTBOQ1RFeENRMk1kSFJjV1RqWTFQMWxOVFhFZ0lR
RzNabHBhaGljbkp5ZUdXbHBtWFZOVGdpc3JEWUlOSUNGZk96eENURUpEWXgwZEhSMWpRMEpNUWp3
N1h5RWdEWUlOS3l1Q1UxTmRBQUFDQUFBQUFBTnVBMjRBSEFBNUFBQUJJZ2NPQVFjR0ZSUVhIZ0VY
RmpNeU56NEJOelkxTkNjdUFTY21Jd0VVQnc0QkJ3WWpJaWN1QVNjbU5UUTNQZ0UzTmpNeE1oY2VB
UmNXQWJkTVFrTmpIUjBkSFdORFFreExRME5qSFIwZEhXTkRRMHNCdHlNaWQxQlFXMXRRVUhjaklp
SWpkMUJRVzF0UVVIY2lJd01sSFIxalEwTkxURUpEWXgwZEhSMWpRMEpNUzBORFl4MGQvcEpiVUZC
M0lpTWpJbmRRVUZ0YlVGQjNJaU1qSW5kUVVBQUFBQUFKQUFBQUFBTnVBeVVBQXdBVUFCZ0FIQUFn
QURFQVFnQkdBRW9BQURjVkl6VWxNaFlkQVJRR0t3RWlKajBCTkRZN0FUY1ZJVFVURlNNMUFSVWhO
UU15RmgwQkZBWXJBU0ltUFFFME5qc0JBVElXSFFFVUJpc0JJaVk5QVRRMk93RVhGU00xRXhVaE5j
bkpBWklQRmhZUGtnOFdGZytTWFA0U2dJQURidjViZ0E4V0ZnK1NEeFlXRDVJQmJnOFZGUStTRHhZ
V0Q1SzNnSUQrRXBKSlNVa1ZENUlQRmhZUGtnOFYzRWxKQVNSSlNmMjNTVWtDa3hZUGtnOFdGZytT
RHhiKzJ4WVBrZzhWRlErU0R4WkpTVWtCSkVsSkFBTUFEZiszQklZRHR3QVJBQ1FBVlFBQUFSWVhI
Z0VYRmhjVUJpTWhGQVlqSWlZMUZ6STJOVFFtSXlJbU5UUW1JeUlHRlJRV013RVdGQWNCQmlZdkFT
WTJQd0V1QVRVMk56NEJOelkxTkRjK0FUYzJOeTRCTlRRMk16SVdGUlFHQng0QkZ6YzJGaGNEZWdz
VEV5MGFHaGtzSHY4QVZUMDhWcElFQlFVRUlqQUZCQVFGT3lrQ1BRUUcrOUlGRUFRd0JRRUZhd1lG
SHlFaE5CRVJFQkUrTFMwNUFnTWdGeGNnQWdOS2F4M3ZCZzhGQWRCQU1qRk5IQjBWSGlzOFZsVTla
QVVFQkFVd0lnUUZCUVFwT3dPU0JnOEYvR0VGQVFZM0JnOEZYQWdUQ2hzbkozRk5UV2dwS1NwRkdS
a0pCUXNHRnlBZ0Z3WUxCUXRNTXM4RkFRWUFBQUFFQUEzL3R3U0dBN2NBRWdBbEFEMEFid0FBQlRR
bUl5SW1OVFFtSXlJR0ZSUVdNekkyTlFrQkxnRWpJZ2NPQVFjR0ZSUUhEZ0VIQmdjRkZBWWpJUlFH
SXlJbU5UY2hMZ0VuTnhZWEhnRVhGaGNURnhZVUJ3RUdKaThCSmpZL0FTNEJOVFkzUGdFM05qVTBO
ejRCTnpZM0xnRTFORFl6TWhZVkZBWUhIZ0VYTnpZV0Z3SlNCUVFpTUFVRUJBVTdLUVFGL3MwQjlo
WmtVamdwS2pZTkRRVUZFdzhQRkFNR0xCNy9BRlU5UEZaVkFiRXdRUkUvQ3hNVExSb2FHVEV3QkFi
NzBnVVFCREFGQVFWckJnVWZJU0UwRVJFUUVUNHRMVGtDQXlBWEZ5QUNBMHBySGU4R0R3VVNCQVV3
SWdRRkJRUXBPd1VFQVE4QnNpMUpFeEkzSUNBYk56TXlYQ29wSm1zZUt6eFdWVDFKTm9OT04wQXlN
VTBjSFJVREhEY0dEd1g4WVFVQkJqY0dEd1ZjQ0JNS0d5Y25jVTFOYUNrcEtrVVpHUWtGQ3dZWElD
QVhCZ3NGQzB3eXp3VUJCZ0FBQUFBQ0FBQUFBQVNTQTI0QUJRQUxBQUFsRlNFUk14RUJFeUVSQ1FF
RWt2dHVTUU51a3Z4SkFRQUJTVWxKQTI3ODJ3SkovZ0FCU1FGSy9yWUFBQUFEQUFBQUFBUGJBN2NB
RndBZ0FDa0FBQWtCQmdjT0FRY0dJeUluTGdFbkpqVTBOejRCTnpZekVSY2hGQWNPQVFjR0J4TWhF
VElYSGdFWEZnRzNBVGdlSXlSUEt5d3RXMUJRZHlNaUlpTjNVRkJiYXdHNUNRa2hHUmdlWHY1Slcx
QlFkeUlqQWJyK3lCNFlHQ0lKQ1NNaWQxQlFXMXRRVUhjaUkvNU1BeTRySzFBakl4NEJnUUczSXlK
M1VGQUFBQUFEQUFBQVNRU1NBeVVBR3dBNUFGY0FBQUUwSnk0Qkp5WWpJZ2NPQVFjR0ZSUVhIZ0VY
RmpNeU56NEJOellsTkNjdUFTY21Ld0VXRng0QkZ4WVZGQWNPQVFjR0J6TXlOejRCTnpZM0ZBY09B
UWNHSXlFaUp5NEJKeVkxTkRjK0FUYzJNeUV5Rng0QkZ4WUNraGNYVHpZMVBEMDFOVkFYRnhjWFVE
VTFQVHcxTms4WEZ3RzNGeGRQTmpVODNTRWNHeVlMQ2dvTEpoc2NJZDA4TlRaUEZ4ZEpIQjFrUWtO
TC9rbE1RMEpqSFIwZEhXTkNRMHdCdDB0RFFtUWRIQUczUERZMVR4Y1hGeGRQTlRZOFBUVTFVQmNY
RnhkUU5UVTlQRFkxVHhjWEdTQWdTeWtxTFMwcUtrc2dJQmtYRjFBMU5UMU1Ra05qSFIwZEhXTkRR
a3hMUTBOakhSMGRIV05EUXdBQ0FBQUFTUVNTQXlVQUhnQTZBQUFUTkRjK0FUYzJNeUV5Rng0QkZ4
WVZGQWNPQVFjR0l5RWlKeTRCSnlZMUFUSTNQZ0UzTmpVMEp5NEJKeVlqSWdjT0FRY0dGUlFYSGdF
WEZnQWRIV05DUTB3QnQwdERRbVFkSEJ3ZFpFSkRTLzVKVEVOQ1l4MGRBeVU4TlRaUEZ4Y1hGMDgy
TlR3OU5UVlFGeGNYRjFBMU5RRzNTME5EWXgwZEhSMWpRME5MVEVKRFl4MGRIUjFqUTBKTS90c1hG
MUExTlQwOE5qVlBGeGNYRjA4MU5qdzlOVFZRRnhjQUFBQUFCQUFBLzdjRUFBTzNBQllBS2dBOUFF
a0FBQUV5RmhjZUFSY2xKZ2NPQVFjR0J5YzJOejRCTnpZekJSTVdGeDRCRnhZM0F5WW5MZ0VuSmpV
ME5qY0ZGaGNXQmdjR0J3NEJKeE0yTnpZbUp5WW5KeklXRlJRR0l5SW1OVFEyQWY1Q2d6MURaaUQr
V0MwckswZ2NIQStkSlMwdFpqZzNPUDVWd1JRZkgwd3FLeTJEWEU5UWRDRWhMQ2NEaXlFQkFUczVP
bE5Ea2tqb0dRc01BZzhQSHIxSVpXVklTR1ZsQTdjaUl5ZHRRQllEQ3dzdElpRXI4eTBrSXpFTURl
aitoaWtmSUNnSENBbisvZzR0TFlaV1ZtQk5qenhnVjF0YXFFaElNQ2NnQXdGa0ppc3JWU2twSWdO
bFIwaGxaVWhIWlFBQUF3QUFBQUFEYmdOdUFCQUFJUUE5QUFBQkVUUW1Ld0VpQmhVUkZCWTdBVEky
TlNFUk5DWXJBU0lHRlJFVUZqc0JNalkxTnhRSERnRUhCaU1pSnk0Qkp5WTFORGMrQVRjMk16SVhI
Z0VYRmdHU0NnaVNDQXNMQ0pJSUNnRUFDZ2lTQ0FzTENKSUlDdHdqSW5kUVVGdGJVRkIzSXlJaUkz
ZFFVRnRiVUZCM0lpTUJFZ0ZKQ0FzTENQNjNCd3NMQndGSkNBc0xDUDYzQndzTEI2VmJVRkIzSWlN
akluZFFVRnRiVUZCM0lpTWpJbmRRVUFBQUJBQUFBQUFEYmdOdUFCd0FPQUJKQUZrQUFBRXlGeDRC
RnhZVkZBY09BUWNHSXlJbkxnRW5KalUwTno0Qk56WXpFVEkzUGdFM05qVTBKeTRCSnlZaklnY09B
UWNHRlJRWEhnRVhGamNpSmpVUk5EWTdBVElXRlJFVUJpc0JJeUltTlJFME5qc0JNaFlWRVJRR0l3
RzNXMUJRZHlJakl5SjNVRkJiVzFCUWR5TWlJaU4zVUZCYlFEazRWUmdaR1JoVk9EbEFRRGs1VkJr
WUdCbFVPVGwzQ0FzTENHMElDd3NJYmR3SEN3c0hiZ2dLQ2dnRGJpTWlkMUJRVzF0UVVIY2lJeU1p
ZDFCUVcxdFFVSGNpSS8wU0dCbFVPVGxBUURrNFZSZ1pHUmhWT0RsQVFEazVWQmtZZ0FzSEFVa0lD
d3NJL3JjSEN3c0hBVWtJQ3dzSS9yY0hDd0FBQUFBQ0FCTUFBQVB0QTI0QUF3Qm9BQUFCTnlNSEFR
Y09BU3NCQnpNeUZoY2VBUThCRGdFckFRY09BU3NCSWlZbkxnRS9BU01IRGdFckFTSW1KeTRCUHdF
aklpWW5MZ0UvQVQ0Qk93RTNJeUltSnk0QlB3RStBVHNCTno0Qk93RXlGaGNlQVE4Qk16YytBVHNC
TWhZWEhnRVBBVE15RmhjZUFRY0NOaVdSSlFKSUlBSUpCN29sc2dRSEF3TUNBaUFCQ2dhN0xnSUtC
b0FFQ0FNREFRRXNrUzRDQ2dhQkF3Z0RBZ0lCTExFRkJ3TUNBZ0VnQWdrSHVpV3lCQWNEQXdJQ0lB
RUtCcnN1QWdvSGdBUUhBd01CQVN5UkxnSUtCNEFEQ0FNQ0FnRXNzUVVIQXdJQ0FRRnVrcElCSUlB
R0NKSUVBd1FJQklBR0NMc0dDQVFEQXdrRXNyc0dDQVFEQXdrRXNnUURBd2tFZ0FZSWtnUURBd2tF
Z0FZSXV3WUlCQU1FQ0FTeXV3WUlCQU1FQ0FTeUJBTUVDQVFBQkFBQUFBQURiZ051QUJBQVJBQmhB
SDRBQUFFVkZBWXJBU0ltUFFFME5qc0JNaFlWRXhRR0J3NEJIUUVVQmlzQklpWTlBVFEyTno0Qk5U
UW1JeUlHQnc0QkJ3NEJJeUltTHdFdUFUYytBVE14TWhjZUFSY1dGUU1pQnc0QkJ3WVZGQmNlQVJj
V016STNQZ0UzTmpVMEp5NEJKeVlqQVJRSERnRUhCaU1pSnk0Qkp5WTFORGMrQVRjMk16RXlGeDRC
RnhZQjl3c0hYQWdLQ2doY0J3dVNReDhXR2dzSFhBZ0tQQjhaSGpNY0R4NEtDUlFRQXdjRkF3VUNQ
Z1lDQkNOalFTTWtJemtTRXRKTVFrTmpIUjBkSFdORFFreExRME5qSFIwZEhXTkRRMHNCdHlNaWQx
QlFXMXRRVUhjaklpSWpkMUJRVzF0UVVIY2lJd0VKV3dnTEN3aGJDQW9LQ0FFY1BUb1NEUlFORXdj
TEN3Y25OU3dPREJVVUdTQUlCd1lXRXdRREFRSXZCQThHTnpZTkRTOGZJQ1VCQUIwZFkwTkRTMHhD
UTJNZEhSMGRZME5DVEV0RFEyTWRIZjZTVzFCUWR5SWpJeUozVUZCYlcxQlFkeUlqSXlKM1VGQUFB
QUFBQWdBQS83Y0VBQU8zQUJzQVFnQUFBUkVVQmlNaElpWTFFVFEyTno0Qk56NEJNeklXRng0QkZ4
NEJGUUUrQVRjK0FTOEJMZ0VIRGdFSERnRWpJaVluTGdFbkpnWVBBUVlXRng0QkZ4NEJNekkyTndR
QU5pWDh0aVUyQXdNdFd1QWJWQ1FrVkJ2Z1dpMERBLzYrUTJBaUJnSUZGUVVQQmlKZlF4dFVKQ1JV
RzBOZklnWVBCUlVGQWdZaVlFTWlaRGc1WlNBQ1FmM1JKVFkySlFJdkJBY0RKMHFqRkVCQUZLTktK
d01IQlA2eU1FWWFCQThHSGdZQ0JSbEdNQk5BUUJNeFJSa0ZBZ1llQmc4RUdrWXdHRWxLRndBQUFB
QURBQUQvdHdRQUE3Y0FNQUJIQUd3QUFBRVhGZ1lIQmdjT0FRY0dCdzRCS3dFaUppY21KeTRCSnlZ
bkxnRS9BVDRCRng0QkZ4NEJPd0V5TmpjK0FUYzJGaGNURVM0Qkp5NEJLd0VpQmdjT0FRY1JGQll6
SVRJMk5STVJGQVlqSVNJbU5SRTBOamMyTno0Qk56WTNQZ0U3QVRJV0Z4WVhIZ0VYRmhjZUFSVURT
aGNFQWdVV0lDQTlGeFlESGxBc0FpeFFIZ01XRmpzZklCVUdBZ1VWQlE4R0hsUTlGa2tlQWg1SkZq
OVdIZ1lQQkcwclNNWVdTUjRDSGtrV3hrZ3JDd2NEU2djTFNUWWwvTFlsTmd3TExDNHZWeWduSHg1
UUxBSXNVQjRkSnloWkx5OHJDd3dCclIwR0RnVVJHUmt2RVJJQkdUWTJHUUVSRVM0WUdSQUZEZ1ll
QmdJRkYwRXZFRHM3RURGQ0dBUUNCdjVsQWhNblBKb1FQRHdRbWp3bi9lMEhDd3NIQWhQOTdTVTJO
aVVDRXc4Y0Npa21Ka1FlSGhrWk5qWVpGeDRmUlNZbktBb2NEd0FBQXdBQS83Y0VrZ08zQUFNQURR
QW5BQUEzSVJFaEtRRVJJUlV6TWhZZEFRRVJGQVlqSVJVVUJpTWhJaVkxRVRRMk15RTFORFl6SVRJ
V2tnRzMva2tDU1FFbC9razNKalVCdHpVbS9xUTFKdjNiSlRZMkpRRmNOaVVDSlNZMVNRRWxBYmVU
TlNiSkFlMzkzQ1kyeVNVMk5pVUNKU1kxeVNZMk5nQUFBQUVBQUFBQkFBQ2Q0ajR4WHc4ODlRQUxC
QUFBQUFBQTNldTVtUUFBQUFEZDY3bVpBQUQvdHdTU0E3Y0FBQUFJQUFJQUFBQUFBQUFBQVFBQUE4
RC93QUFBQkpJQUFBQUFCSklBQVFBQUFBQUFBQUFBQUFBQUFBQUFBRzBFQUFBQUFBQUFBQUFBQUFB
Q0FBQUFCQUFBQUFPM0FBQUVBQUFBQkFBQUFBTzNBQUFEdHdBQUF0c0FBQU8zQUFBRUFBQUFCQUFB
QUFRQUFFVURKUUEvQTI0QUFBTnVBQUFESlFBQUE3Y0FFd051QUFBRGJnQUFBN2NBQUFOdUFBQURi
Z0FBQTI0QUFBUUFBQUFEWWdBQUJEMEFBQVFBQUFBRFlnQUFBa2tBQUFRQkFBQUR1d0FBQXlrQUFB
TnVBQUFEYmdBQUF3QUFZd0szQUQ0RGJnQUFBMjRBQUFNbEFBQUR0d0JOQTI0QUFBUUFBQWtFQUFB
K0JBQUFQZ1NTQUFBRVNRQUFCQUFBQUFNbEFBQUVBQUFsQTI0QUFRTWxBQU1EYmdBQUJFa0FBQU8z
QUFrRGJnQUFBN2NBQUFKSkFBQUNTUUFBQVpJQUpRRkpBQUFDU1FBQUFra0FBQUpKQUFBRUFBQUFC
QUFBSlFOdUFBQUNXd0FhQWpjQUJ3R0FBQm9CV3dBSEJFa0FBQUtTQUFBRGJnQUFCQUFBSlFOdUFB
QUR2UUFOQkVrQUh3SkpBQUFDZXdBNkF0c0FBQU1sQUFBQTJ3QUFBMjRBQUFOdUFBQURiZ0FBQTVJ
QUFBT1NBQUFEWEFBQUEyNEFBQVFBQUJJRGJnQUFBMjRBQUFTU0FBMEVrZ0FOQkpJQUFBUGJBQUFF
a2dBQUJKSUFBQVEzQUFBRGJnQUFBMjRBQUFRQUFCTURiZ0FBQkFBQUFBUUFBQUFFa2dBQUFBQUFB
QUFLQUJRQUhnQTZBS0FCQ2dGUUFab0I5Z0pNQXFnRFpBUG1CQ1FFZ0FUeUJjZ0dYQWE0QnZZSGFB
ZmVDRmdJekFsZ0NoQUtVQXE4QzJnTHZBdjhESWdOQUEwY0RWQU5iZzJZRGNRT1RBNjJEdWdQT2cr
ZUQvb1FKQkJRRUlBU1dCTEVFdllUaWhRbUZGd1U0aFdpRmw0V29oZDJGNTRYeEJmc0dCUVlYQmlF
R0t3WlRCbTJHallhcUJzYUcxZ2JsaHZjSENBY2VoMEdIVFlkZEIzY0hsd2V2QjcrSDBJZmhCL1dJ
Q0lna2lFWUlaNGgrQ0tTSXV3alJpT3dKREFrMkNUMkpUNGx3aVllSnBZbThDZHlLQW9vdmlrb0tj
d3FDZ0FCQUFBQWJRRlZBQW9BQUFBQUFBSUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFEZ0N1QUFF
QUFBQUFBQUVBQWdBQUFBRUFBQUFBQUFJQUJ3QXpBQUVBQUFBQUFBTUFBZ0FuQUFFQUFBQUFBQVFB
QWdCSUFBRUFBQUFBQUFVQUN3QUdBQUVBQUFBQUFBWUFBZ0F0QUFFQUFBQUFBQW9BR2dCT0FBTUFB
UVFKQUFFQUJBQUNBQU1BQVFRSkFBSUFEZ0E2QUFNQUFRUUpBQU1BQkFBcEFBTUFBUVFKQUFRQUJB
QktBQU1BQVFRSkFBVUFGZ0FSQUFNQUFRUUpBQVlBQkFBdkFBTUFBUVFKQUFvQU5BQm9abUVBWmdC
aFZtVnljMmx2YmlBeExqQUFWZ0JsQUhJQWN3QnBBRzhBYmdBZ0FERUFMZ0F3Wm1FQVpnQmhabUVB
WmdCaFVtVm5kV3hoY2dCU0FHVUFad0IxQUd3QVlRQnlabUVBWmdCaFJtOXVkQ0JuWlc1bGNtRjBa
V1FnWW5rZ1NXTnZUVzl2Ymk0QVJnQnZBRzRBZEFBZ0FHY0FaUUJ1QUdVQWNnQmhBSFFBWlFCa0FD
QUFZZ0I1QUNBQVNRQmpBRzhBVFFCdkFHOEFiZ0F1QUFBQUF3QUFBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFBQT09KSBmb3JtYXQoJ3dvZmYnKTtmb250LXdlaWdodDogbm9ybWFs
O2ZvbnQtc3R5bGU6IG5vcm1hbDtmb250LWRpc3BsYXk6IGJsb2NrO31bY2xhc3NePSJmYS0iXSwg
W2NsYXNzKj0iIGZhLSJdIHtmb250LWZhbWlseTogJ2ZhJyAhaW1wb3J0YW50O3NwZWFrOiBuZXZl
cjtmb250LXN0eWxlOiBub3JtYWw7Zm9udC13ZWlnaHQ6IG5vcm1hbDtmb250LXZhcmlhbnQ6IG5v
cm1hbDt0ZXh0LXRyYW5zZm9ybTogbm9uZTtsaW5lLWhlaWdodDogMTstd2Via2l0LWZvbnQtc21v
b3RoaW5nOiBhbnRpYWxpYXNlZDstbW96LW9zeC1mb250LXNtb290aGluZzogZ3JheXNjYWxlO30u
ZmEtc3BpbiB7LXdlYmtpdC1hbmltYXRpb246IGZhLXNwaW4gMnMgaW5maW5pdGUgbGluZWFyO2Fu
aW1hdGlvbjogZmEtc3BpbiAycyBpbmZpbml0ZSBsaW5lYXJ9QC13ZWJraXQta2V5ZnJhbWVzICJm
YS1zcGluIiB7MCUgey13ZWJraXQtdHJhbnNmb3JtOiByb3RhdGUoMGRlZyk7dHJhbnNmb3JtOiBy
b3RhdGUoMGRlZyk7fTEwMCUgey13ZWJraXQtdHJhbnNmb3JtOiByb3RhdGUoMzU5ZGVnKTt0cmFu
c2Zvcm06IHJvdGF0ZSgzNTlkZWcpO319QGtleWZyYW1lcyAiZmEtc3BpbiIgezAlIHstd2Via2l0
LXRyYW5zZm9ybTogcm90YXRlKDBkZWcpO3RyYW5zZm9ybTogcm90YXRlKDBkZWcpO30xMDAlIHst
d2Via2l0LXRyYW5zZm9ybTogcm90YXRlKDM1OWRlZyk7dHJhbnNmb3JtOiByb3RhdGUoMzU5ZGVn
KTt9fS5mYS1hc3RlcmlzazpiZWZvcmUge2NvbnRlbnQ6ICJcZjA2OSI7fS5mYS1wbHVzOmJlZm9y
ZSB7Y29udGVudDogIlxmMDY3Ijt9LmZhLXF1ZXN0aW9uOmJlZm9yZSB7Y29udGVudDogIlxmMTI4
Ijt9LmZhLXNlYXJjaDpiZWZvcmUge2NvbnRlbnQ6ICJcZjAwMiI7fS5mYS1lbnZlbG9wZS1vOmJl
Zm9yZSB7Y29udGVudDogIlxmMDAzIjt9LmZhLWhlYXJ0OmJlZm9yZSB7Y29udGVudDogIlxmMDA0
Ijt9LmZhLXN0YXI6YmVmb3JlIHtjb250ZW50OiAiXGYwMDUiO30uZmEtc3Rhci1vOmJlZm9yZSB7
Y29udGVudDogIlxmMDA2Ijt9LmZhLXVzZXI6YmVmb3JlIHtjb250ZW50OiAiXGYwMDciO30uZmEt
dGgtbGFyZ2U6YmVmb3JlIHtjb250ZW50OiAiXGYwMDkiO30uZmEtdGg6YmVmb3JlIHtjb250ZW50
OiAiXGYwMGEiO30uZmEtdGgtbGlzdDpiZWZvcmUge2NvbnRlbnQ6ICJcZjAwYiI7fS5mYS1jaGVj
azpiZWZvcmUge2NvbnRlbnQ6ICJcZjAwYyI7fS5mYS1jbG9zZTpiZWZvcmUge2NvbnRlbnQ6ICJc
ZjAwZCI7fS5mYS1yZW1vdmU6YmVmb3JlIHtjb250ZW50OiAiXGYwMGQiO30uZmEtdGltZXM6YmVm
b3JlIHtjb250ZW50OiAiXGYwMGQiO30uZmEtcG93ZXItb2ZmOmJlZm9yZSB7Y29udGVudDogIlxm
MDExIjt9LmZhLWNvZzpiZWZvcmUge2NvbnRlbnQ6ICJcZjAxMyI7fS5mYS1nZWFyOmJlZm9yZSB7
Y29udGVudDogIlxmMDEzIjt9LmZhLXRyYXNoLW86YmVmb3JlIHtjb250ZW50OiAiXGYwMTQiO30u
ZmEtaG9tZTpiZWZvcmUge2NvbnRlbnQ6ICJcZjAxNSI7fS5mYS1maWxlLW86YmVmb3JlIHtjb250
ZW50OiAiXGYwMTYiO30uZmEtY2xvY2stbzpiZWZvcmUge2NvbnRlbnQ6ICJcZjAxNyI7fS5mYS1k
b3dubG9hZDpiZWZvcmUge2NvbnRlbnQ6ICJcZjAxOSI7fS5mYS1wbGF5LWNpcmNsZS1vOmJlZm9y
ZSB7Y29udGVudDogIlxmMDFkIjt9LmZhLXJlcGVhdDpiZWZvcmUge2NvbnRlbnQ6ICJcZjAxZSI7
fS5mYS1yb3RhdGUtcmlnaHQ6YmVmb3JlIHtjb250ZW50OiAiXGYwMWUiO30uZmEtcmVmcmVzaDpi
ZWZvcmUge2NvbnRlbnQ6ICJcZjAyMSI7fS5mYS1saXN0LWFsdDpiZWZvcmUge2NvbnRlbnQ6ICJc
ZjAyMiI7fS5mYS10YWc6YmVmb3JlIHtjb250ZW50OiAiXGYwMmIiO30uZmEtdGFnczpiZWZvcmUg
e2NvbnRlbnQ6ICJcZjAyYyI7fS5mYS1saXN0OmJlZm9yZSB7Y29udGVudDogIlxmMDNhIjt9LmZh
LXBlbmNpbDpiZWZvcmUge2NvbnRlbnQ6ICJcZjA0MCI7fS5mYS1tYXAtbWFya2VyOmJlZm9yZSB7
Y29udGVudDogIlxmMDQxIjt9LmZhLWVkaXQ6YmVmb3JlIHtjb250ZW50OiAiXGYwNDQiO30uZmEt
cGVuY2lsLXNxdWFyZS1vOmJlZm9yZSB7Y29udGVudDogIlxmMDQ0Ijt9LmZhLWNoZWNrLXNxdWFy
ZS1vOmJlZm9yZSB7Y29udGVudDogIlxmMDQ2Ijt9LmZhLXBsYXk6YmVmb3JlIHtjb250ZW50OiAi
XGYwNGIiO30uZmEtcGF1c2U6YmVmb3JlIHtjb250ZW50OiAiXGYwNGMiO30uZmEtc3RvcDpiZWZv
cmUge2NvbnRlbnQ6ICJcZjA0ZCI7fS5mYS1jaGV2cm9uLWxlZnQ6YmVmb3JlIHtjb250ZW50OiAi
XGYwNTMiO30uZmEtY2hldnJvbi1yaWdodDpiZWZvcmUge2NvbnRlbnQ6ICJcZjA1NCI7fS5mYS1x
dWVzdGlvbi1jaXJjbGU6YmVmb3JlIHtjb250ZW50OiAiXGYwNTkiO30uZmEtaW5mby1jaXJjbGU6
YmVmb3JlIHtjb250ZW50OiAiXGYwNWEiO30uZmEtZXhjbGFtYXRpb24tY2lyY2xlOmJlZm9yZSB7
Y29udGVudDogIlxmMDZhIjt9LmZhLWV4Y2xhbWF0aW9uLXRyaWFuZ2xlOmJlZm9yZSB7Y29udGVu
dDogIlxmMDcxIjt9LmZhLXdhcm5pbmc6YmVmb3JlIHtjb250ZW50OiAiXGYwNzEiO30uZmEtY2hl
dnJvbi11cDpiZWZvcmUge2NvbnRlbnQ6ICJcZjA3NyI7fS5mYS1jaGV2cm9uLWRvd246YmVmb3Jl
IHtjb250ZW50OiAiXGYwNzgiO30uZmEtYmFyLWNoYXJ0OmJlZm9yZSB7Y29udGVudDogIlxmMDgw
Ijt9LmZhLWJhci1jaGFydC1vOmJlZm9yZSB7Y29udGVudDogIlxmMDgwIjt9LmZhLWNvZ3M6YmVm
b3JlIHtjb250ZW50OiAiXGYwODUiO30uZmEtZ2VhcnM6YmVmb3JlIHtjb250ZW50OiAiXGYwODUi
O30uZmEtZXh0ZXJuYWwtbGluazpiZWZvcmUge2NvbnRlbnQ6ICJcZjA4ZSI7fS5mYS1zcXVhcmUt
bzpiZWZvcmUge2NvbnRlbnQ6ICJcZjA5NiI7fS5mYS1iZWxsLW86YmVmb3JlIHtjb250ZW50OiAi
XGYwYTIiO30uZmEtY2VydGlmaWNhdGU6YmVmb3JlIHtjb250ZW50OiAiXGYwYTMiO30uZmEtZmls
dGVyOmJlZm9yZSB7Y29udGVudDogIlxmMGIwIjt9LmZhLWFycm93cy1hbHQ6YmVmb3JlIHtjb250
ZW50OiAiXGYwYjIiO30uZmEtZ3JvdXA6YmVmb3JlIHtjb250ZW50OiAiXGYwYzAiO30uZmEtdXNl
cnM6YmVmb3JlIHtjb250ZW50OiAiXGYwYzAiO30uZmEtY2hhaW46YmVmb3JlIHtjb250ZW50OiAi
XGYwYzEiO30uZmEtbGluazpiZWZvcmUge2NvbnRlbnQ6ICJcZjBjMSI7fS5mYS1iYXJzOmJlZm9y
ZSB7Y29udGVudDogIlxmMGM5Ijt9LmZhLW5hdmljb246YmVmb3JlIHtjb250ZW50OiAiXGYwYzki
O30uZmEtcmVvcmRlcjpiZWZvcmUge2NvbnRlbnQ6ICJcZjBjOSI7fS5mYS10YWJsZTpiZWZvcmUg
e2NvbnRlbnQ6ICJcZjBjZSI7fS5mYS1jYXJldC1kb3duOmJlZm9yZSB7Y29udGVudDogIlxmMGQ3
Ijt9LmZhLWNhcmV0LXVwOmJlZm9yZSB7Y29udGVudDogIlxmMGQ4Ijt9LmZhLWNhcmV0LWxlZnQ6
YmVmb3JlIHtjb250ZW50OiAiXGYwZDkiO30uZmEtY2FyZXQtcmlnaHQ6YmVmb3JlIHtjb250ZW50
OiAiXGYwZGEiO30uZmEtc29ydDpiZWZvcmUge2NvbnRlbnQ6ICJcZjBkYyI7fS5mYS11bnNvcnRl
ZDpiZWZvcmUge2NvbnRlbnQ6ICJcZjBkYyI7fS5mYS1zb3J0LWRlc2M6YmVmb3JlIHtjb250ZW50
OiAiXGYwZGQiO30uZmEtc29ydC1kb3duOmJlZm9yZSB7Y29udGVudDogIlxmMGRkIjt9LmZhLXNv
cnQtYXNjOmJlZm9yZSB7Y29udGVudDogIlxmMGRlIjt9LmZhLXNvcnQtdXA6YmVmb3JlIHtjb250
ZW50OiAiXGYwZGUiO30uZmEtZGFzaGJvYXJkOmJlZm9yZSB7Y29udGVudDogIlxmMGU0Ijt9LmZh
LXRhY2hvbWV0ZXI6YmVmb3JlIHtjb250ZW50OiAiXGYwZTQiO30uZmEtYmVsbDpiZWZvcmUge2Nv
bnRlbnQ6ICJcZjBmMyI7fS5mYS1maWxlLXRleHQtbzpiZWZvcmUge2NvbnRlbnQ6ICJcZjBmNiI7
fS5mYS1hbmdsZS1kb3VibGUtbGVmdDpiZWZvcmUge2NvbnRlbnQ6ICJcZjEwMCI7fS5mYS1hbmds
ZS1kb3VibGUtcmlnaHQ6YmVmb3JlIHtjb250ZW50OiAiXGYxMDEiO30uZmEtYW5nbGUtbGVmdDpi
ZWZvcmUge2NvbnRlbnQ6ICJcZjEwNCI7fS5mYS1hbmdsZS1yaWdodDpiZWZvcmUge2NvbnRlbnQ6
ICJcZjEwNSI7fS5mYS1kZXNrdG9wOmJlZm9yZSB7Y29udGVudDogIlxmMTA4Ijt9LmZhLXRhYmxl
dDpiZWZvcmUge2NvbnRlbnQ6ICJcZjEwYSI7fS5mYS1jaXJjbGUtbzpiZWZvcmUge2NvbnRlbnQ6
ICJcZjEwYyI7fS5mYS1zcGlubmVyOmJlZm9yZSB7Y29udGVudDogIlxmMTEwIjt9LmZhLWNpcmNs
ZTpiZWZvcmUge2NvbnRlbnQ6ICJcZjExMSI7fS5mYS10ZXJtaW5hbDpiZWZvcmUge2NvbnRlbnQ6
ICJcZjEyMCI7fS5mYS1jb2RlOmJlZm9yZSB7Y29udGVudDogIlxmMTIxIjt9LmZhLWNvZGUtZm9y
azpiZWZvcmUge2NvbnRlbnQ6ICJcZjEyNiI7fS5mYS1zaGllbGQ6YmVmb3JlIHtjb250ZW50OiAi
XGYxMzIiO30uZmEtZWxsaXBzaXMtaDpiZWZvcmUge2NvbnRlbnQ6ICJcZjE0MSI7fS5mYS1lbGxp
cHNpcy12OmJlZm9yZSB7Y29udGVudDogIlxmMTQyIjt9LmZhLXBsYXktY2lyY2xlOmJlZm9yZSB7
Y29udGVudDogIlxmMTQ0Ijt9LmZhLWV4dGVybmFsLWxpbmstc3F1YXJlOmJlZm9yZSB7Y29udGVu
dDogIlxmMTRjIjt9LmZhLWZpbGUtdGV4dDpiZWZvcmUge2NvbnRlbnQ6ICJcZjE1YyI7fS5mYS10
aHVtYnMtdXA6YmVmb3JlIHtjb250ZW50OiAiXGYxNjQiO30uZmEtdGh1bWJzLWRvd246YmVmb3Jl
IHtjb250ZW50OiAiXGYxNjUiO30uZmEtZ29vZ2xlOmJlZm9yZSB7Y29udGVudDogIlxmMWEwIjt9
LmZhLWZpbGUtY29kZS1vOmJlZm9yZSB7Y29udGVudDogIlxmMWM5Ijt9LmZhLWNpcmNsZS1vLW5v
dGNoOmJlZm9yZSB7Y29udGVudDogIlxmMWNlIjt9LmZhLWNpcmNsZS10aGluOmJlZm9yZSB7Y29u
dGVudDogIlxmMWRiIjt9LmZhLXNsaWRlcnM6YmVmb3JlIHtjb250ZW50OiAiXGYxZGUiO30uZmEt
YmVsbC1zbGFzaDpiZWZvcmUge2NvbnRlbnQ6ICJcZjFmNiI7fS5mYS1iZWxsLXNsYXNoLW86YmVm
b3JlIHtjb250ZW50OiAiXGYxZjciO30uZmEtYXJlYS1jaGFydDpiZWZvcmUge2NvbnRlbnQ6ICJc
ZjFmZSI7fS5mYS1waWUtY2hhcnQ6YmVmb3JlIHtjb250ZW50OiAiXGYyMDAiO30uZmEtdG9nZ2xl
LW9mZjpiZWZvcmUge2NvbnRlbnQ6ICJcZjIwNCI7fS5mYS10b2dnbGUtb246YmVmb3JlIHtjb250
ZW50OiAiXGYyMDUiO30uZmEtY2hyb21lOmJlZm9yZSB7Y29udGVudDogIlxmMjY4Ijt9LmZhLXBh
dXNlLWNpcmNsZTpiZWZvcmUge2NvbnRlbnQ6ICJcZjI4YiI7fS5mYS1wYXVzZS1jaXJjbGUtbzpi
ZWZvcmUge2NvbnRlbnQ6ICJcZjI4YyI7fS5mYS1oYXNodGFnOmJlZm9yZSB7Y29udGVudDogIlxm
MjkyIjt9LmZhLXF1ZXN0aW9uLWNpcmNsZS1vOmJlZm9yZSB7Y29udGVudDogIlxmMjljIjt9LmZh
LWVudmVsb3BlLW9wZW46YmVmb3JlIHtjb250ZW50OiAiXGYyYjYiO30uZmEtZW52ZWxvcGUtb3Bl
bi1vOmJlZm9yZSB7Y29udGVudDogIlxmMmI3Ijt9LmZhLXdpbmRvdy1yZXN0b3JlOmJlZm9yZSB7
Y29udGVudDogIlxmMmQyIjt9LmZhLW1pbnVzOmJlZm9yZSB7Y29udGVudDogIlxlYTBiIjt9PC9z
dHlsZT48c3R5bGU+LyohKiBCb290c3RyYXAgdjMuMy43IChodHRwOi8vZ2V0Ym9vdHN0cmFwLmNv
bSkqIENvcHlyaWdodCAyMDExLTIwMTYgVHdpdHRlciwgSW5jLiogTGljZW5zZWQgdW5kZXIgTUlU
IChodHRwczovL2dpdGh1Yi5jb20vdHdicy9ib290c3RyYXAvYmxvYi9tYXN0ZXIvTElDRU5TRSkq
Ly8qISBub3JtYWxpemUuY3NzIHYzLjAuMyB8IE1JVCBMaWNlbnNlIHwgZ2l0aHViLmNvbS9uZWNv
bGFzL25vcm1hbGl6ZS5jc3MgKi9odG1se2ZvbnQtZmFtaWx5OnNhbnMtc2VyaWY7LXdlYmtpdC10
ZXh0LXNpemUtYWRqdXN0OjEwMCU7LW1zLXRleHQtc2l6ZS1hZGp1c3Q6MTAwJX1ib2R5e21hcmdp
bjowfWFydGljbGUsYXNpZGUsZGV0YWlscyxmaWdjYXB0aW9uLGZpZ3VyZSxmb290ZXIsaGVhZGVy
LGhncm91cCxtYWluLG1lbnUsbmF2LHNlY3Rpb24sc3VtbWFyeXtkaXNwbGF5OmJsb2NrfWF1ZGlv
LGNhbnZhcyxwcm9ncmVzcyx2aWRlb3tkaXNwbGF5OmlubGluZS1ibG9jazt2ZXJ0aWNhbC1hbGln
bjpiYXNlbGluZX1hdWRpbzpub3QoW2NvbnRyb2xzXSl7ZGlzcGxheTpub25lO2hlaWdodDowfVto
aWRkZW5dLHRlbXBsYXRle2Rpc3BsYXk6bm9uZX1he2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJl
bnR9YTphY3RpdmUsYTpob3ZlcntvdXRsaW5lOjB9YWJiclt0aXRsZV17Ym9yZGVyLWJvdHRvbTox
cHggZG90dGVkfWIsc3Ryb25ne2ZvbnQtd2VpZ2h0OjcwMH1kZm57Zm9udC1zdHlsZTppdGFsaWN9
aDF7bWFyZ2luOi42N2VtIDA7Zm9udC1zaXplOjJlbX1tYXJre2NvbG9yOiMwMDA7YmFja2dyb3Vu
ZDojZmYwfXNtYWxse2ZvbnQtc2l6ZTo4MCV9c3ViLHN1cHtwb3NpdGlvbjpyZWxhdGl2ZTtmb250
LXNpemU6NzUlO2xpbmUtaGVpZ2h0OjA7dmVydGljYWwtYWxpZ246YmFzZWxpbmV9c3Vwe3RvcDot
LjVlbX1zdWJ7Ym90dG9tOi0uMjVlbX1pbWd7Ym9yZGVyOjB9c3ZnOm5vdCg6cm9vdCl7b3ZlcmZs
b3c6aGlkZGVufWZpZ3VyZXttYXJnaW46MWVtIDQwcHh9aHJ7aGVpZ2h0OjA7LXdlYmtpdC1ib3gt
c2l6aW5nOmNvbnRlbnQtYm94Oy1tb3otYm94LXNpemluZzpjb250ZW50LWJveDtib3gtc2l6aW5n
OmNvbnRlbnQtYm94fXByZXtvdmVyZmxvdzphdXRvfWNvZGUsa2JkLHByZSxzYW1we2ZvbnQtZmFt
aWx5Om1vbm9zcGFjZSxtb25vc3BhY2U7Zm9udC1zaXplOjFlbX1idXR0b24saW5wdXQsb3B0Z3Jv
dXAsc2VsZWN0LHRleHRhcmVhe21hcmdpbjowO2ZvbnQ6aW5oZXJpdDtjb2xvcjppbmhlcml0fWJ1
dHRvbntvdmVyZmxvdzp2aXNpYmxlfWJ1dHRvbixzZWxlY3R7dGV4dC10cmFuc2Zvcm06bm9uZX1i
dXR0b24saHRtbCBpbnB1dFt0eXBlPWJ1dHRvbl0saW5wdXRbdHlwZT1yZXNldF0saW5wdXRbdHlw
ZT1zdWJtaXRdey13ZWJraXQtYXBwZWFyYW5jZTpidXR0b247Y3Vyc29yOnBvaW50ZXJ9YnV0dG9u
W2Rpc2FibGVkXSxodG1sIGlucHV0W2Rpc2FibGVkXXtjdXJzb3I6ZGVmYXVsdH1idXR0b246Oi1t
b3otZm9jdXMtaW5uZXIsaW5wdXQ6Oi1tb3otZm9jdXMtaW5uZXJ7cGFkZGluZzowO2JvcmRlcjow
fWlucHV0e2xpbmUtaGVpZ2h0Om5vcm1hbH1pbnB1dFt0eXBlPWNoZWNrYm94XSxpbnB1dFt0eXBl
PXJhZGlvXXstd2Via2l0LWJveC1zaXppbmc6Ym9yZGVyLWJveDstbW96LWJveC1zaXppbmc6Ym9y
ZGVyLWJveDtib3gtc2l6aW5nOmJvcmRlci1ib3g7cGFkZGluZzowfWlucHV0W3R5cGU9bnVtYmVy
XTo6LXdlYmtpdC1pbm5lci1zcGluLWJ1dHRvbixpbnB1dFt0eXBlPW51bWJlcl06Oi13ZWJraXQt
b3V0ZXItc3Bpbi1idXR0b257aGVpZ2h0OmF1dG99aW5wdXRbdHlwZT1zZWFyY2hdey13ZWJraXQt
Ym94LXNpemluZzpjb250ZW50LWJveDstbW96LWJveC1zaXppbmc6Y29udGVudC1ib3g7Ym94LXNp
emluZzpjb250ZW50LWJveDstd2Via2l0LWFwcGVhcmFuY2U6dGV4dGZpZWxkfWlucHV0W3R5cGU9
c2VhcmNoXTo6LXdlYmtpdC1zZWFyY2gtY2FuY2VsLWJ1dHRvbixpbnB1dFt0eXBlPXNlYXJjaF06
Oi13ZWJraXQtc2VhcmNoLWRlY29yYXRpb257LXdlYmtpdC1hcHBlYXJhbmNlOm5vbmV9ZmllbGRz
ZXR7cGFkZGluZzouMzVlbSAuNjI1ZW0gLjc1ZW07bWFyZ2luOjAgMnB4O2JvcmRlcjoxcHggc29s
aWQgc2lsdmVyfWxlZ2VuZHtwYWRkaW5nOjA7Ym9yZGVyOjB9dGV4dGFyZWF7b3ZlcmZsb3c6YXV0
b31vcHRncm91cHtmb250LXdlaWdodDo3MDB9dGFibGV7Ym9yZGVyLXNwYWNpbmc6MDtib3JkZXIt
Y29sbGFwc2U6Y29sbGFwc2V9dGQsdGh7cGFkZGluZzowfS8qISBTb3VyY2U6IGh0dHBzOi8vZ2l0
aHViLmNvbS9oNWJwL2h0bWw1LWJvaWxlcnBsYXRlL2Jsb2IvbWFzdGVyL3NyYy9jc3MvbWFpbi5j
c3MgKi9AbWVkaWEgcHJpbnR7Kiw6YWZ0ZXIsOmJlZm9yZXtjb2xvcjojMDAwIWltcG9ydGFudDt0
ZXh0LXNoYWRvdzpub25lIWltcG9ydGFudDtiYWNrZ3JvdW5kOjAgMCFpbXBvcnRhbnQ7LXdlYmtp
dC1ib3gtc2hhZG93Om5vbmUhaW1wb3J0YW50O2JveC1zaGFkb3c6bm9uZSFpbXBvcnRhbnR9YSxh
OnZpc2l0ZWR7dGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZX1hW2hyZWZdOmFmdGVye2NvbnRlbnQ6
IiAoIiBhdHRyKGhyZWYpICIpIn1hYmJyW3RpdGxlXTphZnRlcntjb250ZW50OiIgKCIgYXR0cih0
aXRsZSkgIikifWFbaHJlZl49ImphdmFzY3JpcHQ6Il06YWZ0ZXIsYVtocmVmXj0iIyJdOmFmdGVy
e2NvbnRlbnQ6IiJ9YmxvY2txdW90ZSxwcmV7Ym9yZGVyOjFweCBzb2xpZCAjOTk5O3BhZ2UtYnJl
YWstaW5zaWRlOmF2b2lkfXRoZWFke2Rpc3BsYXk6dGFibGUtaGVhZGVyLWdyb3VwfWltZyx0cntw
YWdlLWJyZWFrLWluc2lkZTphdm9pZH1pbWd7bWF4LXdpZHRoOjEwMCUhaW1wb3J0YW50fWgyLGgz
LHB7b3JwaGFuczozO3dpZG93czozfWgyLGgze3BhZ2UtYnJlYWstYWZ0ZXI6YXZvaWR9Lm5hdmJh
cntkaXNwbGF5Om5vbmV9LmJ0bj4uY2FyZXQsLmRyb3B1cD4uYnRuPi5jYXJldHtib3JkZXItdG9w
LWNvbG9yOiMwMDAhaW1wb3J0YW50fS5sYWJlbHtib3JkZXI6MXB4IHNvbGlkICMwMDB9LnRhYmxl
e2JvcmRlci1jb2xsYXBzZTpjb2xsYXBzZSFpbXBvcnRhbnR9LnRhYmxlIHRkLC50YWJsZSB0aHti
YWNrZ3JvdW5kLWNvbG9yOiNmZmYhaW1wb3J0YW50fS50YWJsZS1ib3JkZXJlZCB0ZCwudGFibGUt
Ym9yZGVyZWQgdGh7Ym9yZGVyOjFweCBzb2xpZCAjZGRkIWltcG9ydGFudH19QGZvbnQtZmFjZXtm
b250LWZhbWlseTonR2x5cGhpY29ucyBIYWxmbGluZ3MnO3NyYzp1cmwoLi4vZm9udHMvZ2x5cGhp
Y29ucy1oYWxmbGluZ3MtcmVndWxhci5lb3QpO3NyYzp1cmwoLi4vZm9udHMvZ2x5cGhpY29ucy1o
YWxmbGluZ3MtcmVndWxhci5lb3Q/I2llZml4KSBmb3JtYXQoJ2VtYmVkZGVkLW9wZW50eXBlJyks
dXJsKC4uL2ZvbnRzL2dseXBoaWNvbnMtaGFsZmxpbmdzLXJlZ3VsYXIud29mZjIpIGZvcm1hdCgn
d29mZjInKSx1cmwoLi4vZm9udHMvZ2x5cGhpY29ucy1oYWxmbGluZ3MtcmVndWxhci53b2ZmKSBm
b3JtYXQoJ3dvZmYnKSx1cmwoLi4vZm9udHMvZ2x5cGhpY29ucy1oYWxmbGluZ3MtcmVndWxhci50
dGYpIGZvcm1hdCgndHJ1ZXR5cGUnKSx1cmwoLi4vZm9udHMvZ2x5cGhpY29ucy1oYWxmbGluZ3Mt
cmVndWxhci5zdmcjZ2x5cGhpY29uc19oYWxmbGluZ3NyZWd1bGFyKSBmb3JtYXQoJ3N2ZycpfS5n
bHlwaGljb257cG9zaXRpb246cmVsYXRpdmU7dG9wOjFweDtkaXNwbGF5OmlubGluZS1ibG9jaztm
b250LWZhbWlseTonR2x5cGhpY29ucyBIYWxmbGluZ3MnO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQt
d2VpZ2h0OjQwMDtsaW5lLWhlaWdodDoxOy13ZWJraXQtZm9udC1zbW9vdGhpbmc6YW50aWFsaWFz
ZWQ7LW1vei1vc3gtZm9udC1zbW9vdGhpbmc6Z3JheXNjYWxlfS5nbHlwaGljb24tYXN0ZXJpc2s6
YmVmb3Jle2NvbnRlbnQ6IlwwMDJhIn0uZ2x5cGhpY29uLXBsdXM6YmVmb3Jle2NvbnRlbnQ6Ilww
MDJiIn0uZ2x5cGhpY29uLWV1cjpiZWZvcmUsLmdseXBoaWNvbi1ldXJvOmJlZm9yZXtjb250ZW50
OiJcMjBhYyJ9LmdseXBoaWNvbi1taW51czpiZWZvcmV7Y29udGVudDoiXDIyMTIifS5nbHlwaGlj
b24tY2xvdWQ6YmVmb3Jle2NvbnRlbnQ6IlwyNjAxIn0uZ2x5cGhpY29uLWVudmVsb3BlOmJlZm9y
ZXtjb250ZW50OiJcMjcwOSJ9LmdseXBoaWNvbi1wZW5jaWw6YmVmb3Jle2NvbnRlbnQ6IlwyNzBm
In0uZ2x5cGhpY29uLWdsYXNzOmJlZm9yZXtjb250ZW50OiJcZTAwMSJ9LmdseXBoaWNvbi1tdXNp
YzpiZWZvcmV7Y29udGVudDoiXGUwMDIifS5nbHlwaGljb24tc2VhcmNoOmJlZm9yZXtjb250ZW50
OiJcZTAwMyJ9LmdseXBoaWNvbi1oZWFydDpiZWZvcmV7Y29udGVudDoiXGUwMDUifS5nbHlwaGlj
b24tc3RhcjpiZWZvcmV7Y29udGVudDoiXGUwMDYifS5nbHlwaGljb24tc3Rhci1lbXB0eTpiZWZv
cmV7Y29udGVudDoiXGUwMDcifS5nbHlwaGljb24tdXNlcjpiZWZvcmV7Y29udGVudDoiXGUwMDgi
fS5nbHlwaGljb24tZmlsbTpiZWZvcmV7Y29udGVudDoiXGUwMDkifS5nbHlwaGljb24tdGgtbGFy
Z2U6YmVmb3Jle2NvbnRlbnQ6IlxlMDEwIn0uZ2x5cGhpY29uLXRoOmJlZm9yZXtjb250ZW50OiJc
ZTAxMSJ9LmdseXBoaWNvbi10aC1saXN0OmJlZm9yZXtjb250ZW50OiJcZTAxMiJ9LmdseXBoaWNv
bi1vazpiZWZvcmV7Y29udGVudDoiXGUwMTMifS5nbHlwaGljb24tcmVtb3ZlOmJlZm9yZXtjb250
ZW50OiJcZTAxNCJ9LmdseXBoaWNvbi16b29tLWluOmJlZm9yZXtjb250ZW50OiJcZTAxNSJ9Lmds
eXBoaWNvbi16b29tLW91dDpiZWZvcmV7Y29udGVudDoiXGUwMTYifS5nbHlwaGljb24tb2ZmOmJl
Zm9yZXtjb250ZW50OiJcZTAxNyJ9LmdseXBoaWNvbi1zaWduYWw6YmVmb3Jle2NvbnRlbnQ6Ilxl
MDE4In0uZ2x5cGhpY29uLWNvZzpiZWZvcmV7Y29udGVudDoiXGUwMTkifS5nbHlwaGljb24tdHJh
c2g6YmVmb3Jle2NvbnRlbnQ6IlxlMDIwIn0uZ2x5cGhpY29uLWhvbWU6YmVmb3Jle2NvbnRlbnQ6
IlxlMDIxIn0uZ2x5cGhpY29uLWZpbGU6YmVmb3Jle2NvbnRlbnQ6IlxlMDIyIn0uZ2x5cGhpY29u
LXRpbWU6YmVmb3Jle2NvbnRlbnQ6IlxlMDIzIn0uZ2x5cGhpY29uLXJvYWQ6YmVmb3Jle2NvbnRl
bnQ6IlxlMDI0In0uZ2x5cGhpY29uLWRvd25sb2FkLWFsdDpiZWZvcmV7Y29udGVudDoiXGUwMjUi
fS5nbHlwaGljb24tZG93bmxvYWQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDI2In0uZ2x5cGhpY29uLXVw
bG9hZDpiZWZvcmV7Y29udGVudDoiXGUwMjcifS5nbHlwaGljb24taW5ib3g6YmVmb3Jle2NvbnRl
bnQ6IlxlMDI4In0uZ2x5cGhpY29uLXBsYXktY2lyY2xlOmJlZm9yZXtjb250ZW50OiJcZTAyOSJ9
LmdseXBoaWNvbi1yZXBlYXQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDMwIn0uZ2x5cGhpY29uLXJlZnJl
c2g6YmVmb3Jle2NvbnRlbnQ6IlxlMDMxIn0uZ2x5cGhpY29uLWxpc3QtYWx0OmJlZm9yZXtjb250
ZW50OiJcZTAzMiJ9LmdseXBoaWNvbi1sb2NrOmJlZm9yZXtjb250ZW50OiJcZTAzMyJ9LmdseXBo
aWNvbi1mbGFnOmJlZm9yZXtjb250ZW50OiJcZTAzNCJ9LmdseXBoaWNvbi1oZWFkcGhvbmVzOmJl
Zm9yZXtjb250ZW50OiJcZTAzNSJ9LmdseXBoaWNvbi12b2x1bWUtb2ZmOmJlZm9yZXtjb250ZW50
OiJcZTAzNiJ9LmdseXBoaWNvbi12b2x1bWUtZG93bjpiZWZvcmV7Y29udGVudDoiXGUwMzcifS5n
bHlwaGljb24tdm9sdW1lLXVwOmJlZm9yZXtjb250ZW50OiJcZTAzOCJ9LmdseXBoaWNvbi1xcmNv
ZGU6YmVmb3Jle2NvbnRlbnQ6IlxlMDM5In0uZ2x5cGhpY29uLWJhcmNvZGU6YmVmb3Jle2NvbnRl
bnQ6IlxlMDQwIn0uZ2x5cGhpY29uLXRhZzpiZWZvcmV7Y29udGVudDoiXGUwNDEifS5nbHlwaGlj
b24tdGFnczpiZWZvcmV7Y29udGVudDoiXGUwNDIifS5nbHlwaGljb24tYm9vazpiZWZvcmV7Y29u
dGVudDoiXGUwNDMifS5nbHlwaGljb24tYm9va21hcms6YmVmb3Jle2NvbnRlbnQ6IlxlMDQ0In0u
Z2x5cGhpY29uLXByaW50OmJlZm9yZXtjb250ZW50OiJcZTA0NSJ9LmdseXBoaWNvbi1jYW1lcmE6
YmVmb3Jle2NvbnRlbnQ6IlxlMDQ2In0uZ2x5cGhpY29uLWZvbnQ6YmVmb3Jle2NvbnRlbnQ6Ilxl
MDQ3In0uZ2x5cGhpY29uLWJvbGQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDQ4In0uZ2x5cGhpY29uLWl0
YWxpYzpiZWZvcmV7Y29udGVudDoiXGUwNDkifS5nbHlwaGljb24tdGV4dC1oZWlnaHQ6YmVmb3Jl
e2NvbnRlbnQ6IlxlMDUwIn0uZ2x5cGhpY29uLXRleHQtd2lkdGg6YmVmb3Jle2NvbnRlbnQ6Ilxl
MDUxIn0uZ2x5cGhpY29uLWFsaWduLWxlZnQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDUyIn0uZ2x5cGhp
Y29uLWFsaWduLWNlbnRlcjpiZWZvcmV7Y29udGVudDoiXGUwNTMifS5nbHlwaGljb24tYWxpZ24t
cmlnaHQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDU0In0uZ2x5cGhpY29uLWFsaWduLWp1c3RpZnk6YmVm
b3Jle2NvbnRlbnQ6IlxlMDU1In0uZ2x5cGhpY29uLWxpc3Q6YmVmb3Jle2NvbnRlbnQ6IlxlMDU2
In0uZ2x5cGhpY29uLWluZGVudC1sZWZ0OmJlZm9yZXtjb250ZW50OiJcZTA1NyJ9LmdseXBoaWNv
bi1pbmRlbnQtcmlnaHQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDU4In0uZ2x5cGhpY29uLWZhY2V0aW1l
LXZpZGVvOmJlZm9yZXtjb250ZW50OiJcZTA1OSJ9LmdseXBoaWNvbi1waWN0dXJlOmJlZm9yZXtj
b250ZW50OiJcZTA2MCJ9LmdseXBoaWNvbi1tYXAtbWFya2VyOmJlZm9yZXtjb250ZW50OiJcZTA2
MiJ9LmdseXBoaWNvbi1hZGp1c3Q6YmVmb3Jle2NvbnRlbnQ6IlxlMDYzIn0uZ2x5cGhpY29uLXRp
bnQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDY0In0uZ2x5cGhpY29uLWVkaXQ6YmVmb3Jle2NvbnRlbnQ6
IlxlMDY1In0uZ2x5cGhpY29uLXNoYXJlOmJlZm9yZXtjb250ZW50OiJcZTA2NiJ9LmdseXBoaWNv
bi1jaGVjazpiZWZvcmV7Y29udGVudDoiXGUwNjcifS5nbHlwaGljb24tbW92ZTpiZWZvcmV7Y29u
dGVudDoiXGUwNjgifS5nbHlwaGljb24tc3RlcC1iYWNrd2FyZDpiZWZvcmV7Y29udGVudDoiXGUw
NjkifS5nbHlwaGljb24tZmFzdC1iYWNrd2FyZDpiZWZvcmV7Y29udGVudDoiXGUwNzAifS5nbHlw
aGljb24tYmFja3dhcmQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDcxIn0uZ2x5cGhpY29uLXBsYXk6YmVm
b3Jle2NvbnRlbnQ6IlxlMDcyIn0uZ2x5cGhpY29uLXBhdXNlOmJlZm9yZXtjb250ZW50OiJcZTA3
MyJ9LmdseXBoaWNvbi1zdG9wOmJlZm9yZXtjb250ZW50OiJcZTA3NCJ9LmdseXBoaWNvbi1mb3J3
YXJkOmJlZm9yZXtjb250ZW50OiJcZTA3NSJ9LmdseXBoaWNvbi1mYXN0LWZvcndhcmQ6YmVmb3Jl
e2NvbnRlbnQ6IlxlMDc2In0uZ2x5cGhpY29uLXN0ZXAtZm9yd2FyZDpiZWZvcmV7Y29udGVudDoi
XGUwNzcifS5nbHlwaGljb24tZWplY3Q6YmVmb3Jle2NvbnRlbnQ6IlxlMDc4In0uZ2x5cGhpY29u
LWNoZXZyb24tbGVmdDpiZWZvcmV7Y29udGVudDoiXGUwNzkifS5nbHlwaGljb24tY2hldnJvbi1y
aWdodDpiZWZvcmV7Y29udGVudDoiXGUwODAifS5nbHlwaGljb24tcGx1cy1zaWduOmJlZm9yZXtj
b250ZW50OiJcZTA4MSJ9LmdseXBoaWNvbi1taW51cy1zaWduOmJlZm9yZXtjb250ZW50OiJcZTA4
MiJ9LmdseXBoaWNvbi1yZW1vdmUtc2lnbjpiZWZvcmV7Y29udGVudDoiXGUwODMifS5nbHlwaGlj
b24tb2stc2lnbjpiZWZvcmV7Y29udGVudDoiXGUwODQifS5nbHlwaGljb24tcXVlc3Rpb24tc2ln
bjpiZWZvcmV7Y29udGVudDoiXGUwODUifS5nbHlwaGljb24taW5mby1zaWduOmJlZm9yZXtjb250
ZW50OiJcZTA4NiJ9LmdseXBoaWNvbi1zY3JlZW5zaG90OmJlZm9yZXtjb250ZW50OiJcZTA4NyJ9
LmdseXBoaWNvbi1yZW1vdmUtY2lyY2xlOmJlZm9yZXtjb250ZW50OiJcZTA4OCJ9LmdseXBoaWNv
bi1vay1jaXJjbGU6YmVmb3Jle2NvbnRlbnQ6IlxlMDg5In0uZ2x5cGhpY29uLWJhbi1jaXJjbGU6
YmVmb3Jle2NvbnRlbnQ6IlxlMDkwIn0uZ2x5cGhpY29uLWFycm93LWxlZnQ6YmVmb3Jle2NvbnRl
bnQ6IlxlMDkxIn0uZ2x5cGhpY29uLWFycm93LXJpZ2h0OmJlZm9yZXtjb250ZW50OiJcZTA5MiJ9
LmdseXBoaWNvbi1hcnJvdy11cDpiZWZvcmV7Y29udGVudDoiXGUwOTMifS5nbHlwaGljb24tYXJy
b3ctZG93bjpiZWZvcmV7Y29udGVudDoiXGUwOTQifS5nbHlwaGljb24tc2hhcmUtYWx0OmJlZm9y
ZXtjb250ZW50OiJcZTA5NSJ9LmdseXBoaWNvbi1yZXNpemUtZnVsbDpiZWZvcmV7Y29udGVudDoi
XGUwOTYifS5nbHlwaGljb24tcmVzaXplLXNtYWxsOmJlZm9yZXtjb250ZW50OiJcZTA5NyJ9Lmds
eXBoaWNvbi1leGNsYW1hdGlvbi1zaWduOmJlZm9yZXtjb250ZW50OiJcZTEwMSJ9LmdseXBoaWNv
bi1naWZ0OmJlZm9yZXtjb250ZW50OiJcZTEwMiJ9LmdseXBoaWNvbi1sZWFmOmJlZm9yZXtjb250
ZW50OiJcZTEwMyJ9LmdseXBoaWNvbi1maXJlOmJlZm9yZXtjb250ZW50OiJcZTEwNCJ9LmdseXBo
aWNvbi1leWUtb3BlbjpiZWZvcmV7Y29udGVudDoiXGUxMDUifS5nbHlwaGljb24tZXllLWNsb3Nl
OmJlZm9yZXtjb250ZW50OiJcZTEwNiJ9LmdseXBoaWNvbi13YXJuaW5nLXNpZ246YmVmb3Jle2Nv
bnRlbnQ6IlxlMTA3In0uZ2x5cGhpY29uLXBsYW5lOmJlZm9yZXtjb250ZW50OiJcZTEwOCJ9Lmds
eXBoaWNvbi1jYWxlbmRhcjpiZWZvcmV7Y29udGVudDoiXGUxMDkifS5nbHlwaGljb24tcmFuZG9t
OmJlZm9yZXtjb250ZW50OiJcZTExMCJ9LmdseXBoaWNvbi1jb21tZW50OmJlZm9yZXtjb250ZW50
OiJcZTExMSJ9LmdseXBoaWNvbi1tYWduZXQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTEyIn0uZ2x5cGhp
Y29uLWNoZXZyb24tdXA6YmVmb3Jle2NvbnRlbnQ6IlxlMTEzIn0uZ2x5cGhpY29uLWNoZXZyb24t
ZG93bjpiZWZvcmV7Y29udGVudDoiXGUxMTQifS5nbHlwaGljb24tcmV0d2VldDpiZWZvcmV7Y29u
dGVudDoiXGUxMTUifS5nbHlwaGljb24tc2hvcHBpbmctY2FydDpiZWZvcmV7Y29udGVudDoiXGUx
MTYifS5nbHlwaGljb24tZm9sZGVyLWNsb3NlOmJlZm9yZXtjb250ZW50OiJcZTExNyJ9LmdseXBo
aWNvbi1mb2xkZXItb3BlbjpiZWZvcmV7Y29udGVudDoiXGUxMTgifS5nbHlwaGljb24tcmVzaXpl
LXZlcnRpY2FsOmJlZm9yZXtjb250ZW50OiJcZTExOSJ9LmdseXBoaWNvbi1yZXNpemUtaG9yaXpv
bnRhbDpiZWZvcmV7Y29udGVudDoiXGUxMjAifS5nbHlwaGljb24taGRkOmJlZm9yZXtjb250ZW50
OiJcZTEyMSJ9LmdseXBoaWNvbi1idWxsaG9ybjpiZWZvcmV7Y29udGVudDoiXGUxMjIifS5nbHlw
aGljb24tYmVsbDpiZWZvcmV7Y29udGVudDoiXGUxMjMifS5nbHlwaGljb24tY2VydGlmaWNhdGU6
YmVmb3Jle2NvbnRlbnQ6IlxlMTI0In0uZ2x5cGhpY29uLXRodW1icy11cDpiZWZvcmV7Y29udGVu
dDoiXGUxMjUifS5nbHlwaGljb24tdGh1bWJzLWRvd246YmVmb3Jle2NvbnRlbnQ6IlxlMTI2In0u
Z2x5cGhpY29uLWhhbmQtcmlnaHQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTI3In0uZ2x5cGhpY29uLWhh
bmQtbGVmdDpiZWZvcmV7Y29udGVudDoiXGUxMjgifS5nbHlwaGljb24taGFuZC11cDpiZWZvcmV7
Y29udGVudDoiXGUxMjkifS5nbHlwaGljb24taGFuZC1kb3duOmJlZm9yZXtjb250ZW50OiJcZTEz
MCJ9LmdseXBoaWNvbi1jaXJjbGUtYXJyb3ctcmlnaHQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTMxIn0u
Z2x5cGhpY29uLWNpcmNsZS1hcnJvdy1sZWZ0OmJlZm9yZXtjb250ZW50OiJcZTEzMiJ9LmdseXBo
aWNvbi1jaXJjbGUtYXJyb3ctdXA6YmVmb3Jle2NvbnRlbnQ6IlxlMTMzIn0uZ2x5cGhpY29uLWNp
cmNsZS1hcnJvdy1kb3duOmJlZm9yZXtjb250ZW50OiJcZTEzNCJ9LmdseXBoaWNvbi1nbG9iZTpi
ZWZvcmV7Y29udGVudDoiXGUxMzUifS5nbHlwaGljb24td3JlbmNoOmJlZm9yZXtjb250ZW50OiJc
ZTEzNiJ9LmdseXBoaWNvbi10YXNrczpiZWZvcmV7Y29udGVudDoiXGUxMzcifS5nbHlwaGljb24t
ZmlsdGVyOmJlZm9yZXtjb250ZW50OiJcZTEzOCJ9LmdseXBoaWNvbi1icmllZmNhc2U6YmVmb3Jl
e2NvbnRlbnQ6IlxlMTM5In0uZ2x5cGhpY29uLWZ1bGxzY3JlZW46YmVmb3Jle2NvbnRlbnQ6Ilxl
MTQwIn0uZ2x5cGhpY29uLWRhc2hib2FyZDpiZWZvcmV7Y29udGVudDoiXGUxNDEifS5nbHlwaGlj
b24tcGFwZXJjbGlwOmJlZm9yZXtjb250ZW50OiJcZTE0MiJ9LmdseXBoaWNvbi1oZWFydC1lbXB0
eTpiZWZvcmV7Y29udGVudDoiXGUxNDMifS5nbHlwaGljb24tbGluazpiZWZvcmV7Y29udGVudDoi
XGUxNDQifS5nbHlwaGljb24tcGhvbmU6YmVmb3Jle2NvbnRlbnQ6IlxlMTQ1In0uZ2x5cGhpY29u
LXB1c2hwaW46YmVmb3Jle2NvbnRlbnQ6IlxlMTQ2In0uZ2x5cGhpY29uLXVzZDpiZWZvcmV7Y29u
dGVudDoiXGUxNDgifS5nbHlwaGljb24tZ2JwOmJlZm9yZXtjb250ZW50OiJcZTE0OSJ9LmdseXBo
aWNvbi1zb3J0OmJlZm9yZXtjb250ZW50OiJcZTE1MCJ9LmdseXBoaWNvbi1zb3J0LWJ5LWFscGhh
YmV0OmJlZm9yZXtjb250ZW50OiJcZTE1MSJ9LmdseXBoaWNvbi1zb3J0LWJ5LWFscGhhYmV0LWFs
dDpiZWZvcmV7Y29udGVudDoiXGUxNTIifS5nbHlwaGljb24tc29ydC1ieS1vcmRlcjpiZWZvcmV7
Y29udGVudDoiXGUxNTMifS5nbHlwaGljb24tc29ydC1ieS1vcmRlci1hbHQ6YmVmb3Jle2NvbnRl
bnQ6IlxlMTU0In0uZ2x5cGhpY29uLXNvcnQtYnktYXR0cmlidXRlczpiZWZvcmV7Y29udGVudDoi
XGUxNTUifS5nbHlwaGljb24tc29ydC1ieS1hdHRyaWJ1dGVzLWFsdDpiZWZvcmV7Y29udGVudDoi
XGUxNTYifS5nbHlwaGljb24tdW5jaGVja2VkOmJlZm9yZXtjb250ZW50OiJcZTE1NyJ9LmdseXBo
aWNvbi1leHBhbmQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTU4In0uZ2x5cGhpY29uLWNvbGxhcHNlLWRv
d246YmVmb3Jle2NvbnRlbnQ6IlxlMTU5In0uZ2x5cGhpY29uLWNvbGxhcHNlLXVwOmJlZm9yZXtj
b250ZW50OiJcZTE2MCJ9LmdseXBoaWNvbi1sb2ctaW46YmVmb3Jle2NvbnRlbnQ6IlxlMTYxIn0u
Z2x5cGhpY29uLWZsYXNoOmJlZm9yZXtjb250ZW50OiJcZTE2MiJ9LmdseXBoaWNvbi1sb2ctb3V0
OmJlZm9yZXtjb250ZW50OiJcZTE2MyJ9LmdseXBoaWNvbi1uZXctd2luZG93OmJlZm9yZXtjb250
ZW50OiJcZTE2NCJ9LmdseXBoaWNvbi1yZWNvcmQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTY1In0uZ2x5
cGhpY29uLXNhdmU6YmVmb3Jle2NvbnRlbnQ6IlxlMTY2In0uZ2x5cGhpY29uLW9wZW46YmVmb3Jl
e2NvbnRlbnQ6IlxlMTY3In0uZ2x5cGhpY29uLXNhdmVkOmJlZm9yZXtjb250ZW50OiJcZTE2OCJ9
LmdseXBoaWNvbi1pbXBvcnQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTY5In0uZ2x5cGhpY29uLWV4cG9y
dDpiZWZvcmV7Y29udGVudDoiXGUxNzAifS5nbHlwaGljb24tc2VuZDpiZWZvcmV7Y29udGVudDoi
XGUxNzEifS5nbHlwaGljb24tZmxvcHB5LWRpc2s6YmVmb3Jle2NvbnRlbnQ6IlxlMTcyIn0uZ2x5
cGhpY29uLWZsb3BweS1zYXZlZDpiZWZvcmV7Y29udGVudDoiXGUxNzMifS5nbHlwaGljb24tZmxv
cHB5LXJlbW92ZTpiZWZvcmV7Y29udGVudDoiXGUxNzQifS5nbHlwaGljb24tZmxvcHB5LXNhdmU6
YmVmb3Jle2NvbnRlbnQ6IlxlMTc1In0uZ2x5cGhpY29uLWZsb3BweS1vcGVuOmJlZm9yZXtjb250
ZW50OiJcZTE3NiJ9LmdseXBoaWNvbi1jcmVkaXQtY2FyZDpiZWZvcmV7Y29udGVudDoiXGUxNzci
fS5nbHlwaGljb24tdHJhbnNmZXI6YmVmb3Jle2NvbnRlbnQ6IlxlMTc4In0uZ2x5cGhpY29uLWN1
dGxlcnk6YmVmb3Jle2NvbnRlbnQ6IlxlMTc5In0uZ2x5cGhpY29uLWhlYWRlcjpiZWZvcmV7Y29u
dGVudDoiXGUxODAifS5nbHlwaGljb24tY29tcHJlc3NlZDpiZWZvcmV7Y29udGVudDoiXGUxODEi
fS5nbHlwaGljb24tZWFycGhvbmU6YmVmb3Jle2NvbnRlbnQ6IlxlMTgyIn0uZ2x5cGhpY29uLXBo
b25lLWFsdDpiZWZvcmV7Y29udGVudDoiXGUxODMifS5nbHlwaGljb24tdG93ZXI6YmVmb3Jle2Nv
bnRlbnQ6IlxlMTg0In0uZ2x5cGhpY29uLXN0YXRzOmJlZm9yZXtjb250ZW50OiJcZTE4NSJ9Lmds
eXBoaWNvbi1zZC12aWRlbzpiZWZvcmV7Y29udGVudDoiXGUxODYifS5nbHlwaGljb24taGQtdmlk
ZW86YmVmb3Jle2NvbnRlbnQ6IlxlMTg3In0uZ2x5cGhpY29uLXN1YnRpdGxlczpiZWZvcmV7Y29u
dGVudDoiXGUxODgifS5nbHlwaGljb24tc291bmQtc3RlcmVvOmJlZm9yZXtjb250ZW50OiJcZTE4
OSJ9LmdseXBoaWNvbi1zb3VuZC1kb2xieTpiZWZvcmV7Y29udGVudDoiXGUxOTAifS5nbHlwaGlj
b24tc291bmQtNS0xOmJlZm9yZXtjb250ZW50OiJcZTE5MSJ9LmdseXBoaWNvbi1zb3VuZC02LTE6
YmVmb3Jle2NvbnRlbnQ6IlxlMTkyIn0uZ2x5cGhpY29uLXNvdW5kLTctMTpiZWZvcmV7Y29udGVu
dDoiXGUxOTMifS5nbHlwaGljb24tY29weXJpZ2h0LW1hcms6YmVmb3Jle2NvbnRlbnQ6IlxlMTk0
In0uZ2x5cGhpY29uLXJlZ2lzdHJhdGlvbi1tYXJrOmJlZm9yZXtjb250ZW50OiJcZTE5NSJ9Lmds
eXBoaWNvbi1jbG91ZC1kb3dubG9hZDpiZWZvcmV7Y29udGVudDoiXGUxOTcifS5nbHlwaGljb24t
Y2xvdWQtdXBsb2FkOmJlZm9yZXtjb250ZW50OiJcZTE5OCJ9LmdseXBoaWNvbi10cmVlLWNvbmlm
ZXI6YmVmb3Jle2NvbnRlbnQ6IlxlMTk5In0uZ2x5cGhpY29uLXRyZWUtZGVjaWR1b3VzOmJlZm9y
ZXtjb250ZW50OiJcZTIwMCJ9LmdseXBoaWNvbi1jZDpiZWZvcmV7Y29udGVudDoiXGUyMDEifS5n
bHlwaGljb24tc2F2ZS1maWxlOmJlZm9yZXtjb250ZW50OiJcZTIwMiJ9LmdseXBoaWNvbi1vcGVu
LWZpbGU6YmVmb3Jle2NvbnRlbnQ6IlxlMjAzIn0uZ2x5cGhpY29uLWxldmVsLXVwOmJlZm9yZXtj
b250ZW50OiJcZTIwNCJ9LmdseXBoaWNvbi1jb3B5OmJlZm9yZXtjb250ZW50OiJcZTIwNSJ9Lmds
eXBoaWNvbi1wYXN0ZTpiZWZvcmV7Y29udGVudDoiXGUyMDYifS5nbHlwaGljb24tYWxlcnQ6YmVm
b3Jle2NvbnRlbnQ6IlxlMjA5In0uZ2x5cGhpY29uLWVxdWFsaXplcjpiZWZvcmV7Y29udGVudDoi
XGUyMTAifS5nbHlwaGljb24ta2luZzpiZWZvcmV7Y29udGVudDoiXGUyMTEifS5nbHlwaGljb24t
cXVlZW46YmVmb3Jle2NvbnRlbnQ6IlxlMjEyIn0uZ2x5cGhpY29uLXBhd246YmVmb3Jle2NvbnRl
bnQ6IlxlMjEzIn0uZ2x5cGhpY29uLWJpc2hvcDpiZWZvcmV7Y29udGVudDoiXGUyMTQifS5nbHlw
aGljb24ta25pZ2h0OmJlZm9yZXtjb250ZW50OiJcZTIxNSJ9LmdseXBoaWNvbi1iYWJ5LWZvcm11
bGE6YmVmb3Jle2NvbnRlbnQ6IlxlMjE2In0uZ2x5cGhpY29uLXRlbnQ6YmVmb3Jle2NvbnRlbnQ6
IlwyNmZhIn0uZ2x5cGhpY29uLWJsYWNrYm9hcmQ6YmVmb3Jle2NvbnRlbnQ6IlxlMjE4In0uZ2x5
cGhpY29uLWJlZDpiZWZvcmV7Y29udGVudDoiXGUyMTkifS5nbHlwaGljb24tYXBwbGU6YmVmb3Jl
e2NvbnRlbnQ6IlxmOGZmIn0uZ2x5cGhpY29uLWVyYXNlOmJlZm9yZXtjb250ZW50OiJcZTIyMSJ9
LmdseXBoaWNvbi1ob3VyZ2xhc3M6YmVmb3Jle2NvbnRlbnQ6IlwyMzFiIn0uZ2x5cGhpY29uLWxh
bXA6YmVmb3Jle2NvbnRlbnQ6IlxlMjIzIn0uZ2x5cGhpY29uLWR1cGxpY2F0ZTpiZWZvcmV7Y29u
dGVudDoiXGUyMjQifS5nbHlwaGljb24tcGlnZ3ktYmFuazpiZWZvcmV7Y29udGVudDoiXGUyMjUi
fS5nbHlwaGljb24tc2Npc3NvcnM6YmVmb3Jle2NvbnRlbnQ6IlxlMjI2In0uZ2x5cGhpY29uLWJp
dGNvaW46YmVmb3Jle2NvbnRlbnQ6IlxlMjI3In0uZ2x5cGhpY29uLWJ0YzpiZWZvcmV7Y29udGVu
dDoiXGUyMjcifS5nbHlwaGljb24teGJ0OmJlZm9yZXtjb250ZW50OiJcZTIyNyJ9LmdseXBoaWNv
bi15ZW46YmVmb3Jle2NvbnRlbnQ6IlwwMGE1In0uZ2x5cGhpY29uLWpweTpiZWZvcmV7Y29udGVu
dDoiXDAwYTUifS5nbHlwaGljb24tcnVibGU6YmVmb3Jle2NvbnRlbnQ6IlwyMGJkIn0uZ2x5cGhp
Y29uLXJ1YjpiZWZvcmV7Y29udGVudDoiXDIwYmQifS5nbHlwaGljb24tc2NhbGU6YmVmb3Jle2Nv
bnRlbnQ6IlxlMjMwIn0uZ2x5cGhpY29uLWljZS1sb2xseTpiZWZvcmV7Y29udGVudDoiXGUyMzEi
fS5nbHlwaGljb24taWNlLWxvbGx5LXRhc3RlZDpiZWZvcmV7Y29udGVudDoiXGUyMzIifS5nbHlw
aGljb24tZWR1Y2F0aW9uOmJlZm9yZXtjb250ZW50OiJcZTIzMyJ9LmdseXBoaWNvbi1vcHRpb24t
aG9yaXpvbnRhbDpiZWZvcmV7Y29udGVudDoiXGUyMzQifS5nbHlwaGljb24tb3B0aW9uLXZlcnRp
Y2FsOmJlZm9yZXtjb250ZW50OiJcZTIzNSJ9LmdseXBoaWNvbi1tZW51LWhhbWJ1cmdlcjpiZWZv
cmV7Y29udGVudDoiXGUyMzYifS5nbHlwaGljb24tbW9kYWwtd2luZG93OmJlZm9yZXtjb250ZW50
OiJcZTIzNyJ9LmdseXBoaWNvbi1vaWw6YmVmb3Jle2NvbnRlbnQ6IlxlMjM4In0uZ2x5cGhpY29u
LWdyYWluOmJlZm9yZXtjb250ZW50OiJcZTIzOSJ9LmdseXBoaWNvbi1zdW5nbGFzc2VzOmJlZm9y
ZXtjb250ZW50OiJcZTI0MCJ9LmdseXBoaWNvbi10ZXh0LXNpemU6YmVmb3Jle2NvbnRlbnQ6Ilxl
MjQxIn0uZ2x5cGhpY29uLXRleHQtY29sb3I6YmVmb3Jle2NvbnRlbnQ6IlxlMjQyIn0uZ2x5cGhp
Y29uLXRleHQtYmFja2dyb3VuZDpiZWZvcmV7Y29udGVudDoiXGUyNDMifS5nbHlwaGljb24tb2Jq
ZWN0LWFsaWduLXRvcDpiZWZvcmV7Y29udGVudDoiXGUyNDQifS5nbHlwaGljb24tb2JqZWN0LWFs
aWduLWJvdHRvbTpiZWZvcmV7Y29udGVudDoiXGUyNDUifS5nbHlwaGljb24tb2JqZWN0LWFsaWdu
LWhvcml6b250YWw6YmVmb3Jle2NvbnRlbnQ6IlxlMjQ2In0uZ2x5cGhpY29uLW9iamVjdC1hbGln
bi1sZWZ0OmJlZm9yZXtjb250ZW50OiJcZTI0NyJ9LmdseXBoaWNvbi1vYmplY3QtYWxpZ24tdmVy
dGljYWw6YmVmb3Jle2NvbnRlbnQ6IlxlMjQ4In0uZ2x5cGhpY29uLW9iamVjdC1hbGlnbi1yaWdo
dDpiZWZvcmV7Y29udGVudDoiXGUyNDkifS5nbHlwaGljb24tdHJpYW5nbGUtcmlnaHQ6YmVmb3Jl
e2NvbnRlbnQ6IlxlMjUwIn0uZ2x5cGhpY29uLXRyaWFuZ2xlLWxlZnQ6YmVmb3Jle2NvbnRlbnQ6
IlxlMjUxIn0uZ2x5cGhpY29uLXRyaWFuZ2xlLWJvdHRvbTpiZWZvcmV7Y29udGVudDoiXGUyNTIi
fS5nbHlwaGljb24tdHJpYW5nbGUtdG9wOmJlZm9yZXtjb250ZW50OiJcZTI1MyJ9LmdseXBoaWNv
bi1jb25zb2xlOmJlZm9yZXtjb250ZW50OiJcZTI1NCJ9LmdseXBoaWNvbi1zdXBlcnNjcmlwdDpi
ZWZvcmV7Y29udGVudDoiXGUyNTUifS5nbHlwaGljb24tc3Vic2NyaXB0OmJlZm9yZXtjb250ZW50
OiJcZTI1NiJ9LmdseXBoaWNvbi1tZW51LWxlZnQ6YmVmb3Jle2NvbnRlbnQ6IlxlMjU3In0uZ2x5
cGhpY29uLW1lbnUtcmlnaHQ6YmVmb3Jle2NvbnRlbnQ6IlxlMjU4In0uZ2x5cGhpY29uLW1lbnUt
ZG93bjpiZWZvcmV7Y29udGVudDoiXGUyNTkifS5nbHlwaGljb24tbWVudS11cDpiZWZvcmV7Y29u
dGVudDoiXGUyNjAifSp7LXdlYmtpdC1ib3gtc2l6aW5nOmJvcmRlci1ib3g7LW1vei1ib3gtc2l6
aW5nOmJvcmRlci1ib3g7Ym94LXNpemluZzpib3JkZXItYm94fTphZnRlciw6YmVmb3Jley13ZWJr
aXQtYm94LXNpemluZzpib3JkZXItYm94Oy1tb3otYm94LXNpemluZzpib3JkZXItYm94O2JveC1z
aXppbmc6Ym9yZGVyLWJveH1odG1se2ZvbnQtc2l6ZToxMHB4Oy13ZWJraXQtdGFwLWhpZ2hsaWdo
dC1jb2xvcjpyZ2JhKDAsMCwwLDApfWJvZHl7Zm9udC1mYW1pbHk6IkhlbHZldGljYSBOZXVlIixI
ZWx2ZXRpY2EsQXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtsaW5lLWhlaWdodDoxLjQy
ODU3MTQzO2NvbG9yOiMzMzM7YmFja2dyb3VuZC1jb2xvcjojZmZmfWJ1dHRvbixpbnB1dCxzZWxl
Y3QsdGV4dGFyZWF7Zm9udC1mYW1pbHk6aW5oZXJpdDtmb250LXNpemU6aW5oZXJpdDtsaW5lLWhl
aWdodDppbmhlcml0fWF7Y29sb3I6IzMzN2FiNzt0ZXh0LWRlY29yYXRpb246bm9uZX1hOmZvY3Vz
LGE6aG92ZXJ7Y29sb3I6IzIzNTI3Yzt0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lfWE6Zm9jdXN7
b3V0bGluZTo1cHggYXV0byAtd2Via2l0LWZvY3VzLXJpbmctY29sb3I7b3V0bGluZS1vZmZzZXQ6
LTJweH1maWd1cmV7bWFyZ2luOjB9aW1ne3ZlcnRpY2FsLWFsaWduOm1pZGRsZX0uY2Fyb3VzZWwt
aW5uZXI+Lml0ZW0+YT5pbWcsLmNhcm91c2VsLWlubmVyPi5pdGVtPmltZywuaW1nLXJlc3BvbnNp
dmUsLnRodW1ibmFpbCBhPmltZywudGh1bWJuYWlsPmltZ3tkaXNwbGF5OmJsb2NrO21heC13aWR0
aDoxMDAlO2hlaWdodDphdXRvfS5pbWctcm91bmRlZHtib3JkZXItcmFkaXVzOjZweH0uaW1nLXRo
dW1ibmFpbHtkaXNwbGF5OmlubGluZS1ibG9jazttYXgtd2lkdGg6MTAwJTtoZWlnaHQ6YXV0bztw
YWRkaW5nOjRweDtsaW5lLWhlaWdodDoxLjQyODU3MTQzO2JhY2tncm91bmQtY29sb3I6I2ZmZjti
b3JkZXI6MXB4IHNvbGlkICNkZGQ7Ym9yZGVyLXJhZGl1czo0cHg7LXdlYmtpdC10cmFuc2l0aW9u
OmFsbCAuMnMgZWFzZS1pbi1vdXQ7LW8tdHJhbnNpdGlvbjphbGwgLjJzIGVhc2UtaW4tb3V0O3Ry
YW5zaXRpb246YWxsIC4ycyBlYXNlLWluLW91dH0uaW1nLWNpcmNsZXtib3JkZXItcmFkaXVzOjUw
JX1ocnttYXJnaW4tdG9wOjIwcHg7bWFyZ2luLWJvdHRvbToyMHB4O2JvcmRlcjowO2JvcmRlci10
b3A6MXB4IHNvbGlkICNlZWV9LnNyLW9ubHl7cG9zaXRpb246YWJzb2x1dGU7d2lkdGg6MXB4O2hl
aWdodDoxcHg7cGFkZGluZzowO21hcmdpbjotMXB4O292ZXJmbG93OmhpZGRlbjtjbGlwOnJlY3Qo
MCwwLDAsMCk7Ym9yZGVyOjB9LnNyLW9ubHktZm9jdXNhYmxlOmFjdGl2ZSwuc3Itb25seS1mb2N1
c2FibGU6Zm9jdXN7cG9zaXRpb246c3RhdGljO3dpZHRoOmF1dG87aGVpZ2h0OmF1dG87bWFyZ2lu
OjA7b3ZlcmZsb3c6dmlzaWJsZTtjbGlwOmF1dG99W3JvbGU9YnV0dG9uXXtjdXJzb3I6cG9pbnRl
cn0uaDEsLmgyLC5oMywuaDQsLmg1LC5oNixoMSxoMixoMyxoNCxoNSxoNntmb250LWZhbWlseTpp
bmhlcml0O2ZvbnQtd2VpZ2h0OjUwMDtsaW5lLWhlaWdodDoxLjE7Y29sb3I6aW5oZXJpdH0uaDEg
LnNtYWxsLC5oMSBzbWFsbCwuaDIgLnNtYWxsLC5oMiBzbWFsbCwuaDMgLnNtYWxsLC5oMyBzbWFs
bCwuaDQgLnNtYWxsLC5oNCBzbWFsbCwuaDUgLnNtYWxsLC5oNSBzbWFsbCwuaDYgLnNtYWxsLC5o
NiBzbWFsbCxoMSAuc21hbGwsaDEgc21hbGwsaDIgLnNtYWxsLGgyIHNtYWxsLGgzIC5zbWFsbCxo
MyBzbWFsbCxoNCAuc21hbGwsaDQgc21hbGwsaDUgLnNtYWxsLGg1IHNtYWxsLGg2IC5zbWFsbCxo
NiBzbWFsbHtmb250LXdlaWdodDo0MDA7bGluZS1oZWlnaHQ6MTtjb2xvcjojNzc3fS5oMSwuaDIs
LmgzLGgxLGgyLGgze21hcmdpbi10b3A6MjBweDttYXJnaW4tYm90dG9tOjEwcHh9LmgxIC5zbWFs
bCwuaDEgc21hbGwsLmgyIC5zbWFsbCwuaDIgc21hbGwsLmgzIC5zbWFsbCwuaDMgc21hbGwsaDEg
LnNtYWxsLGgxIHNtYWxsLGgyIC5zbWFsbCxoMiBzbWFsbCxoMyAuc21hbGwsaDMgc21hbGx7Zm9u
dC1zaXplOjY1JX0uaDQsLmg1LC5oNixoNCxoNSxoNnttYXJnaW4tdG9wOjEwcHg7bWFyZ2luLWJv
dHRvbToxMHB4fS5oNCAuc21hbGwsLmg0IHNtYWxsLC5oNSAuc21hbGwsLmg1IHNtYWxsLC5oNiAu
c21hbGwsLmg2IHNtYWxsLGg0IC5zbWFsbCxoNCBzbWFsbCxoNSAuc21hbGwsaDUgc21hbGwsaDYg
LnNtYWxsLGg2IHNtYWxse2ZvbnQtc2l6ZTo3NSV9LmgxLGgxe2ZvbnQtc2l6ZTozNnB4fS5oMixo
Mntmb250LXNpemU6MzBweH0uaDMsaDN7Zm9udC1zaXplOjI0cHh9Lmg0LGg0e2ZvbnQtc2l6ZTox
OHB4fS5oNSxoNXtmb250LXNpemU6MTRweH0uaDYsaDZ7Zm9udC1zaXplOjEycHh9cHttYXJnaW46
MCAwIDEwcHh9LmxlYWR7bWFyZ2luLWJvdHRvbToyMHB4O2ZvbnQtc2l6ZToxNnB4O2ZvbnQtd2Vp
Z2h0OjMwMDtsaW5lLWhlaWdodDoxLjR9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5sZWFke2Zv
bnQtc2l6ZToyMXB4fX0uc21hbGwsc21hbGx7Zm9udC1zaXplOjg1JX0ubWFyayxtYXJre3BhZGRp
bmc6LjJlbTtiYWNrZ3JvdW5kLWNvbG9yOiNmY2Y4ZTN9LnRleHQtbGVmdHt0ZXh0LWFsaWduOmxl
ZnR9LnRleHQtcmlnaHR7dGV4dC1hbGlnbjpyaWdodH0udGV4dC1jZW50ZXJ7dGV4dC1hbGlnbjpj
ZW50ZXJ9LnRleHQtanVzdGlmeXt0ZXh0LWFsaWduOmp1c3RpZnl9LnRleHQtbm93cmFwe3doaXRl
LXNwYWNlOm5vd3JhcH0udGV4dC1sb3dlcmNhc2V7dGV4dC10cmFuc2Zvcm06bG93ZXJjYXNlfS50
ZXh0LXVwcGVyY2FzZXt0ZXh0LXRyYW5zZm9ybTp1cHBlcmNhc2V9LnRleHQtY2FwaXRhbGl6ZXt0
ZXh0LXRyYW5zZm9ybTpjYXBpdGFsaXplfS50ZXh0LW11dGVke2NvbG9yOiM3Nzd9LnRleHQtcHJp
bWFyeXtjb2xvcjojMzM3YWI3fWEudGV4dC1wcmltYXJ5OmZvY3VzLGEudGV4dC1wcmltYXJ5Omhv
dmVye2NvbG9yOiMyODYwOTB9LnRleHQtc3VjY2Vzc3tjb2xvcjojM2M3NjNkfWEudGV4dC1zdWNj
ZXNzOmZvY3VzLGEudGV4dC1zdWNjZXNzOmhvdmVye2NvbG9yOiMyYjU0MmN9LnRleHQtaW5mb3tj
b2xvcjojMzE3MDhmfWEudGV4dC1pbmZvOmZvY3VzLGEudGV4dC1pbmZvOmhvdmVye2NvbG9yOiMy
NDUyNjl9LnRleHQtd2FybmluZ3tjb2xvcjojOGE2ZDNifWEudGV4dC13YXJuaW5nOmZvY3VzLGEu
dGV4dC13YXJuaW5nOmhvdmVye2NvbG9yOiM2NjUxMmN9LnRleHQtZGFuZ2Vye2NvbG9yOiNhOTQ0
NDJ9YS50ZXh0LWRhbmdlcjpmb2N1cyxhLnRleHQtZGFuZ2VyOmhvdmVye2NvbG9yOiM4NDM1MzR9
LmJnLXByaW1hcnl7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzMzdhYjd9YS5iZy1wcmlt
YXJ5OmZvY3VzLGEuYmctcHJpbWFyeTpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiMyODYwOTB9LmJn
LXN1Y2Nlc3N7YmFja2dyb3VuZC1jb2xvcjojZGZmMGQ4fWEuYmctc3VjY2Vzczpmb2N1cyxhLmJn
LXN1Y2Nlc3M6aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojYzFlMmIzfS5iZy1pbmZve2JhY2tncm91
bmQtY29sb3I6I2Q5ZWRmN31hLmJnLWluZm86Zm9jdXMsYS5iZy1pbmZvOmhvdmVye2JhY2tncm91
bmQtY29sb3I6I2FmZDllZX0uYmctd2FybmluZ3tiYWNrZ3JvdW5kLWNvbG9yOiNmY2Y4ZTN9YS5i
Zy13YXJuaW5nOmZvY3VzLGEuYmctd2FybmluZzpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNmN2Vj
YjV9LmJnLWRhbmdlcntiYWNrZ3JvdW5kLWNvbG9yOiNmMmRlZGV9YS5iZy1kYW5nZXI6Zm9jdXMs
YS5iZy1kYW5nZXI6aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojZTRiOWI5fS5wYWdlLWhlYWRlcntw
YWRkaW5nLWJvdHRvbTo5cHg7bWFyZ2luOjQwcHggMCAyMHB4O2JvcmRlci1ib3R0b206MXB4IHNv
bGlkICNlZWV9b2wsdWx7bWFyZ2luLXRvcDowO21hcmdpbi1ib3R0b206MTBweH1vbCBvbCxvbCB1
bCx1bCBvbCx1bCB1bHttYXJnaW4tYm90dG9tOjB9Lmxpc3QtdW5zdHlsZWR7cGFkZGluZy1sZWZ0
OjA7bGlzdC1zdHlsZTpub25lfS5saXN0LWlubGluZXtwYWRkaW5nLWxlZnQ6MDttYXJnaW4tbGVm
dDotNXB4O2xpc3Qtc3R5bGU6bm9uZX0ubGlzdC1pbmxpbmU+bGl7ZGlzcGxheTppbmxpbmUtYmxv
Y2s7cGFkZGluZy1yaWdodDo1cHg7cGFkZGluZy1sZWZ0OjVweH1kbHttYXJnaW4tdG9wOjA7bWFy
Z2luLWJvdHRvbToyMHB4fWRkLGR0e2xpbmUtaGVpZ2h0OjEuNDI4NTcxNDN9ZHR7Zm9udC13ZWln
aHQ6NzAwfWRke21hcmdpbi1sZWZ0OjB9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5kbC1ob3Jp
em9udGFsIGR0e2Zsb2F0OmxlZnQ7d2lkdGg6MTYwcHg7b3ZlcmZsb3c6aGlkZGVuO2NsZWFyOmxl
ZnQ7dGV4dC1hbGlnbjpyaWdodDt0ZXh0LW92ZXJmbG93OmVsbGlwc2lzO3doaXRlLXNwYWNlOm5v
d3JhcH0uZGwtaG9yaXpvbnRhbCBkZHttYXJnaW4tbGVmdDoxODBweH19YWJicltkYXRhLW9yaWdp
bmFsLXRpdGxlXSxhYmJyW3RpdGxlXXtjdXJzb3I6aGVscDtib3JkZXItYm90dG9tOjFweCBkb3R0
ZWQgIzc3N30uaW5pdGlhbGlzbXtmb250LXNpemU6OTAlO3RleHQtdHJhbnNmb3JtOnVwcGVyY2Fz
ZX1ibG9ja3F1b3Rle3BhZGRpbmc6MTBweCAyMHB4O21hcmdpbjowIDAgMjBweDtmb250LXNpemU6
MTcuNXB4O2JvcmRlci1sZWZ0OjVweCBzb2xpZCAjZWVlfWJsb2NrcXVvdGUgb2w6bGFzdC1jaGls
ZCxibG9ja3F1b3RlIHA6bGFzdC1jaGlsZCxibG9ja3F1b3RlIHVsOmxhc3QtY2hpbGR7bWFyZ2lu
LWJvdHRvbTowfWJsb2NrcXVvdGUgLnNtYWxsLGJsb2NrcXVvdGUgZm9vdGVyLGJsb2NrcXVvdGUg
c21hbGx7ZGlzcGxheTpibG9jaztmb250LXNpemU6ODAlO2xpbmUtaGVpZ2h0OjEuNDI4NTcxNDM7
Y29sb3I6Izc3N31ibG9ja3F1b3RlIC5zbWFsbDpiZWZvcmUsYmxvY2txdW90ZSBmb290ZXI6YmVm
b3JlLGJsb2NrcXVvdGUgc21hbGw6YmVmb3Jle2NvbnRlbnQ6J1wyMDE0IFwwMEEwJ30uYmxvY2tx
dW90ZS1yZXZlcnNlLGJsb2NrcXVvdGUucHVsbC1yaWdodHtwYWRkaW5nLXJpZ2h0OjE1cHg7cGFk
ZGluZy1sZWZ0OjA7dGV4dC1hbGlnbjpyaWdodDtib3JkZXItcmlnaHQ6NXB4IHNvbGlkICNlZWU7
Ym9yZGVyLWxlZnQ6MH0uYmxvY2txdW90ZS1yZXZlcnNlIC5zbWFsbDpiZWZvcmUsLmJsb2NrcXVv
dGUtcmV2ZXJzZSBmb290ZXI6YmVmb3JlLC5ibG9ja3F1b3RlLXJldmVyc2Ugc21hbGw6YmVmb3Jl
LGJsb2NrcXVvdGUucHVsbC1yaWdodCAuc21hbGw6YmVmb3JlLGJsb2NrcXVvdGUucHVsbC1yaWdo
dCBmb290ZXI6YmVmb3JlLGJsb2NrcXVvdGUucHVsbC1yaWdodCBzbWFsbDpiZWZvcmV7Y29udGVu
dDonJ30uYmxvY2txdW90ZS1yZXZlcnNlIC5zbWFsbDphZnRlciwuYmxvY2txdW90ZS1yZXZlcnNl
IGZvb3RlcjphZnRlciwuYmxvY2txdW90ZS1yZXZlcnNlIHNtYWxsOmFmdGVyLGJsb2NrcXVvdGUu
cHVsbC1yaWdodCAuc21hbGw6YWZ0ZXIsYmxvY2txdW90ZS5wdWxsLXJpZ2h0IGZvb3RlcjphZnRl
cixibG9ja3F1b3RlLnB1bGwtcmlnaHQgc21hbGw6YWZ0ZXJ7Y29udGVudDonXDAwQTAgXDIwMTQn
fWFkZHJlc3N7bWFyZ2luLWJvdHRvbToyMHB4O2ZvbnQtc3R5bGU6bm9ybWFsO2xpbmUtaGVpZ2h0
OjEuNDI4NTcxNDN9Y29kZSxrYmQscHJlLHNhbXB7Zm9udC1mYW1pbHk6TWVubG8sTW9uYWNvLENv
bnNvbGFzLCJDb3VyaWVyIE5ldyIsbW9ub3NwYWNlfWNvZGV7cGFkZGluZzoycHggNHB4O2ZvbnQt
c2l6ZTo5MCU7Y29sb3I6I2M3MjU0ZTtiYWNrZ3JvdW5kLWNvbG9yOiNmOWYyZjQ7Ym9yZGVyLXJh
ZGl1czo0cHh9a2Jke3BhZGRpbmc6MnB4IDRweDtmb250LXNpemU6OTAlO2NvbG9yOiNmZmY7YmFj
a2dyb3VuZC1jb2xvcjojMzMzO2JvcmRlci1yYWRpdXM6M3B4Oy13ZWJraXQtYm94LXNoYWRvdzpp
bnNldCAwIC0xcHggMCByZ2JhKDAsMCwwLC4yNSk7Ym94LXNoYWRvdzppbnNldCAwIC0xcHggMCBy
Z2JhKDAsMCwwLC4yNSl9a2JkIGtiZHtwYWRkaW5nOjA7Zm9udC1zaXplOjEwMCU7Zm9udC13ZWln
aHQ6NzAwOy13ZWJraXQtYm94LXNoYWRvdzpub25lO2JveC1zaGFkb3c6bm9uZX1wcmV7ZGlzcGxh
eTpibG9jaztwYWRkaW5nOjkuNXB4O21hcmdpbjowIDAgMTBweDtmb250LXNpemU6MTNweDtsaW5l
LWhlaWdodDoxLjQyODU3MTQzO2NvbG9yOiMzMzM7d29yZC1icmVhazpicmVhay1hbGw7d29yZC13
cmFwOmJyZWFrLXdvcmQ7YmFja2dyb3VuZC1jb2xvcjojZjVmNWY1O2JvcmRlcjoxcHggc29saWQg
I2NjYztib3JkZXItcmFkaXVzOjRweH1wcmUgY29kZXtwYWRkaW5nOjA7Zm9udC1zaXplOmluaGVy
aXQ7Y29sb3I6aW5oZXJpdDt3aGl0ZS1zcGFjZTpwcmUtd3JhcDtiYWNrZ3JvdW5kLWNvbG9yOnRy
YW5zcGFyZW50O2JvcmRlci1yYWRpdXM6MH0ucHJlLXNjcm9sbGFibGV7bWF4LWhlaWdodDozNDBw
eDtvdmVyZmxvdy15OnNjcm9sbH0uY29udGFpbmVye3BhZGRpbmctcmlnaHQ6MTVweDtwYWRkaW5n
LWxlZnQ6MTVweDttYXJnaW4tcmlnaHQ6YXV0bzttYXJnaW4tbGVmdDphdXRvfUBtZWRpYSAobWlu
LXdpZHRoOjc2OHB4KXsuY29udGFpbmVye3dpZHRoOjc1MHB4fX1AbWVkaWEgKG1pbi13aWR0aDo5
OTJweCl7LmNvbnRhaW5lcnt3aWR0aDo5NzBweH19QG1lZGlhIChtaW4td2lkdGg6MTIwMHB4KXsu
Y29udGFpbmVye3dpZHRoOjExNzBweH19LmNvbnRhaW5lci1mbHVpZHtwYWRkaW5nLXJpZ2h0OjE1
cHg7cGFkZGluZy1sZWZ0OjE1cHg7bWFyZ2luLXJpZ2h0OmF1dG87bWFyZ2luLWxlZnQ6YXV0b30u
cm93e21hcmdpbi1yaWdodDotMTVweDttYXJnaW4tbGVmdDotMTVweH0uY29sLWxnLTEsLmNvbC1s
Zy0xMCwuY29sLWxnLTExLC5jb2wtbGctMTIsLmNvbC1sZy0yLC5jb2wtbGctMywuY29sLWxnLTQs
LmNvbC1sZy01LC5jb2wtbGctNiwuY29sLWxnLTcsLmNvbC1sZy04LC5jb2wtbGctOSwuY29sLW1k
LTEsLmNvbC1tZC0xMCwuY29sLW1kLTExLC5jb2wtbWQtMTIsLmNvbC1tZC0yLC5jb2wtbWQtMywu
Y29sLW1kLTQsLmNvbC1tZC01LC5jb2wtbWQtNiwuY29sLW1kLTcsLmNvbC1tZC04LC5jb2wtbWQt
OSwuY29sLXNtLTEsLmNvbC1zbS0xMCwuY29sLXNtLTExLC5jb2wtc20tMTIsLmNvbC1zbS0yLC5j
b2wtc20tMywuY29sLXNtLTQsLmNvbC1zbS01LC5jb2wtc20tNiwuY29sLXNtLTcsLmNvbC1zbS04
LC5jb2wtc20tOSwuY29sLXhzLTEsLmNvbC14cy0xMCwuY29sLXhzLTExLC5jb2wteHMtMTIsLmNv
bC14cy0yLC5jb2wteHMtMywuY29sLXhzLTQsLmNvbC14cy01LC5jb2wteHMtNiwuY29sLXhzLTcs
LmNvbC14cy04LC5jb2wteHMtOXtwb3NpdGlvbjpyZWxhdGl2ZTttaW4taGVpZ2h0OjFweDtwYWRk
aW5nLXJpZ2h0OjE1cHg7cGFkZGluZy1sZWZ0OjE1cHh9LmNvbC14cy0xLC5jb2wteHMtMTAsLmNv
bC14cy0xMSwuY29sLXhzLTEyLC5jb2wteHMtMiwuY29sLXhzLTMsLmNvbC14cy00LC5jb2wteHMt
NSwuY29sLXhzLTYsLmNvbC14cy03LC5jb2wteHMtOCwuY29sLXhzLTl7ZmxvYXQ6bGVmdH0uY29s
LXhzLTEye3dpZHRoOjEwMCV9LmNvbC14cy0xMXt3aWR0aDo5MS42NjY2NjY2NyV9LmNvbC14cy0x
MHt3aWR0aDo4My4zMzMzMzMzMyV9LmNvbC14cy05e3dpZHRoOjc1JX0uY29sLXhzLTh7d2lkdGg6
NjYuNjY2NjY2NjclfS5jb2wteHMtN3t3aWR0aDo1OC4zMzMzMzMzMyV9LmNvbC14cy02e3dpZHRo
OjUwJX0uY29sLXhzLTV7d2lkdGg6NDEuNjY2NjY2NjclfS5jb2wteHMtNHt3aWR0aDozMy4zMzMz
MzMzMyV9LmNvbC14cy0ze3dpZHRoOjI1JX0uY29sLXhzLTJ7d2lkdGg6MTYuNjY2NjY2NjclfS5j
b2wteHMtMXt3aWR0aDo4LjMzMzMzMzMzJX0uY29sLXhzLXB1bGwtMTJ7cmlnaHQ6MTAwJX0uY29s
LXhzLXB1bGwtMTF7cmlnaHQ6OTEuNjY2NjY2NjclfS5jb2wteHMtcHVsbC0xMHtyaWdodDo4My4z
MzMzMzMzMyV9LmNvbC14cy1wdWxsLTl7cmlnaHQ6NzUlfS5jb2wteHMtcHVsbC04e3JpZ2h0OjY2
LjY2NjY2NjY3JX0uY29sLXhzLXB1bGwtN3tyaWdodDo1OC4zMzMzMzMzMyV9LmNvbC14cy1wdWxs
LTZ7cmlnaHQ6NTAlfS5jb2wteHMtcHVsbC01e3JpZ2h0OjQxLjY2NjY2NjY3JX0uY29sLXhzLXB1
bGwtNHtyaWdodDozMy4zMzMzMzMzMyV9LmNvbC14cy1wdWxsLTN7cmlnaHQ6MjUlfS5jb2wteHMt
cHVsbC0ye3JpZ2h0OjE2LjY2NjY2NjY3JX0uY29sLXhzLXB1bGwtMXtyaWdodDo4LjMzMzMzMzMz
JX0uY29sLXhzLXB1bGwtMHtyaWdodDphdXRvfS5jb2wteHMtcHVzaC0xMntsZWZ0OjEwMCV9LmNv
bC14cy1wdXNoLTExe2xlZnQ6OTEuNjY2NjY2NjclfS5jb2wteHMtcHVzaC0xMHtsZWZ0OjgzLjMz
MzMzMzMzJX0uY29sLXhzLXB1c2gtOXtsZWZ0Ojc1JX0uY29sLXhzLXB1c2gtOHtsZWZ0OjY2LjY2
NjY2NjY3JX0uY29sLXhzLXB1c2gtN3tsZWZ0OjU4LjMzMzMzMzMzJX0uY29sLXhzLXB1c2gtNnts
ZWZ0OjUwJX0uY29sLXhzLXB1c2gtNXtsZWZ0OjQxLjY2NjY2NjY3JX0uY29sLXhzLXB1c2gtNHts
ZWZ0OjMzLjMzMzMzMzMzJX0uY29sLXhzLXB1c2gtM3tsZWZ0OjI1JX0uY29sLXhzLXB1c2gtMnts
ZWZ0OjE2LjY2NjY2NjY3JX0uY29sLXhzLXB1c2gtMXtsZWZ0OjguMzMzMzMzMzMlfS5jb2wteHMt
cHVzaC0we2xlZnQ6YXV0b30uY29sLXhzLW9mZnNldC0xMnttYXJnaW4tbGVmdDoxMDAlfS5jb2wt
eHMtb2Zmc2V0LTExe21hcmdpbi1sZWZ0OjkxLjY2NjY2NjY3JX0uY29sLXhzLW9mZnNldC0xMHtt
YXJnaW4tbGVmdDo4My4zMzMzMzMzMyV9LmNvbC14cy1vZmZzZXQtOXttYXJnaW4tbGVmdDo3NSV9
LmNvbC14cy1vZmZzZXQtOHttYXJnaW4tbGVmdDo2Ni42NjY2NjY2NyV9LmNvbC14cy1vZmZzZXQt
N3ttYXJnaW4tbGVmdDo1OC4zMzMzMzMzMyV9LmNvbC14cy1vZmZzZXQtNnttYXJnaW4tbGVmdDo1
MCV9LmNvbC14cy1vZmZzZXQtNXttYXJnaW4tbGVmdDo0MS42NjY2NjY2NyV9LmNvbC14cy1vZmZz
ZXQtNHttYXJnaW4tbGVmdDozMy4zMzMzMzMzMyV9LmNvbC14cy1vZmZzZXQtM3ttYXJnaW4tbGVm
dDoyNSV9LmNvbC14cy1vZmZzZXQtMnttYXJnaW4tbGVmdDoxNi42NjY2NjY2NyV9LmNvbC14cy1v
ZmZzZXQtMXttYXJnaW4tbGVmdDo4LjMzMzMzMzMzJX0uY29sLXhzLW9mZnNldC0we21hcmdpbi1s
ZWZ0OjB9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5jb2wtc20tMSwuY29sLXNtLTEwLC5jb2wt
c20tMTEsLmNvbC1zbS0xMiwuY29sLXNtLTIsLmNvbC1zbS0zLC5jb2wtc20tNCwuY29sLXNtLTUs
LmNvbC1zbS02LC5jb2wtc20tNywuY29sLXNtLTgsLmNvbC1zbS05e2Zsb2F0OmxlZnR9LmNvbC1z
bS0xMnt3aWR0aDoxMDAlfS5jb2wtc20tMTF7d2lkdGg6OTEuNjY2NjY2NjclfS5jb2wtc20tMTB7
d2lkdGg6ODMuMzMzMzMzMzMlfS5jb2wtc20tOXt3aWR0aDo3NSV9LmNvbC1zbS04e3dpZHRoOjY2
LjY2NjY2NjY3JX0uY29sLXNtLTd7d2lkdGg6NTguMzMzMzMzMzMlfS5jb2wtc20tNnt3aWR0aDo1
MCV9LmNvbC1zbS01e3dpZHRoOjQxLjY2NjY2NjY3JX0uY29sLXNtLTR7d2lkdGg6MzMuMzMzMzMz
MzMlfS5jb2wtc20tM3t3aWR0aDoyNSV9LmNvbC1zbS0ye3dpZHRoOjE2LjY2NjY2NjY3JX0uY29s
LXNtLTF7d2lkdGg6OC4zMzMzMzMzMyV9LmNvbC1zbS1wdWxsLTEye3JpZ2h0OjEwMCV9LmNvbC1z
bS1wdWxsLTExe3JpZ2h0OjkxLjY2NjY2NjY3JX0uY29sLXNtLXB1bGwtMTB7cmlnaHQ6ODMuMzMz
MzMzMzMlfS5jb2wtc20tcHVsbC05e3JpZ2h0Ojc1JX0uY29sLXNtLXB1bGwtOHtyaWdodDo2Ni42
NjY2NjY2NyV9LmNvbC1zbS1wdWxsLTd7cmlnaHQ6NTguMzMzMzMzMzMlfS5jb2wtc20tcHVsbC02
e3JpZ2h0OjUwJX0uY29sLXNtLXB1bGwtNXtyaWdodDo0MS42NjY2NjY2NyV9LmNvbC1zbS1wdWxs
LTR7cmlnaHQ6MzMuMzMzMzMzMzMlfS5jb2wtc20tcHVsbC0ze3JpZ2h0OjI1JX0uY29sLXNtLXB1
bGwtMntyaWdodDoxNi42NjY2NjY2NyV9LmNvbC1zbS1wdWxsLTF7cmlnaHQ6OC4zMzMzMzMzMyV9
LmNvbC1zbS1wdWxsLTB7cmlnaHQ6YXV0b30uY29sLXNtLXB1c2gtMTJ7bGVmdDoxMDAlfS5jb2wt
c20tcHVzaC0xMXtsZWZ0OjkxLjY2NjY2NjY3JX0uY29sLXNtLXB1c2gtMTB7bGVmdDo4My4zMzMz
MzMzMyV9LmNvbC1zbS1wdXNoLTl7bGVmdDo3NSV9LmNvbC1zbS1wdXNoLTh7bGVmdDo2Ni42NjY2
NjY2NyV9LmNvbC1zbS1wdXNoLTd7bGVmdDo1OC4zMzMzMzMzMyV9LmNvbC1zbS1wdXNoLTZ7bGVm
dDo1MCV9LmNvbC1zbS1wdXNoLTV7bGVmdDo0MS42NjY2NjY2NyV9LmNvbC1zbS1wdXNoLTR7bGVm
dDozMy4zMzMzMzMzMyV9LmNvbC1zbS1wdXNoLTN7bGVmdDoyNSV9LmNvbC1zbS1wdXNoLTJ7bGVm
dDoxNi42NjY2NjY2NyV9LmNvbC1zbS1wdXNoLTF7bGVmdDo4LjMzMzMzMzMzJX0uY29sLXNtLXB1
c2gtMHtsZWZ0OmF1dG99LmNvbC1zbS1vZmZzZXQtMTJ7bWFyZ2luLWxlZnQ6MTAwJX0uY29sLXNt
LW9mZnNldC0xMXttYXJnaW4tbGVmdDo5MS42NjY2NjY2NyV9LmNvbC1zbS1vZmZzZXQtMTB7bWFy
Z2luLWxlZnQ6ODMuMzMzMzMzMzMlfS5jb2wtc20tb2Zmc2V0LTl7bWFyZ2luLWxlZnQ6NzUlfS5j
b2wtc20tb2Zmc2V0LTh7bWFyZ2luLWxlZnQ6NjYuNjY2NjY2NjclfS5jb2wtc20tb2Zmc2V0LTd7
bWFyZ2luLWxlZnQ6NTguMzMzMzMzMzMlfS5jb2wtc20tb2Zmc2V0LTZ7bWFyZ2luLWxlZnQ6NTAl
fS5jb2wtc20tb2Zmc2V0LTV7bWFyZ2luLWxlZnQ6NDEuNjY2NjY2NjclfS5jb2wtc20tb2Zmc2V0
LTR7bWFyZ2luLWxlZnQ6MzMuMzMzMzMzMzMlfS5jb2wtc20tb2Zmc2V0LTN7bWFyZ2luLWxlZnQ6
MjUlfS5jb2wtc20tb2Zmc2V0LTJ7bWFyZ2luLWxlZnQ6MTYuNjY2NjY2NjclfS5jb2wtc20tb2Zm
c2V0LTF7bWFyZ2luLWxlZnQ6OC4zMzMzMzMzMyV9LmNvbC1zbS1vZmZzZXQtMHttYXJnaW4tbGVm
dDowfX1AbWVkaWEgKG1pbi13aWR0aDo5OTJweCl7LmNvbC1tZC0xLC5jb2wtbWQtMTAsLmNvbC1t
ZC0xMSwuY29sLW1kLTEyLC5jb2wtbWQtMiwuY29sLW1kLTMsLmNvbC1tZC00LC5jb2wtbWQtNSwu
Y29sLW1kLTYsLmNvbC1tZC03LC5jb2wtbWQtOCwuY29sLW1kLTl7ZmxvYXQ6bGVmdH0uY29sLW1k
LTEye3dpZHRoOjEwMCV9LmNvbC1tZC0xMXt3aWR0aDo5MS42NjY2NjY2NyV9LmNvbC1tZC0xMHt3
aWR0aDo4My4zMzMzMzMzMyV9LmNvbC1tZC05e3dpZHRoOjc1JX0uY29sLW1kLTh7d2lkdGg6NjYu
NjY2NjY2NjclfS5jb2wtbWQtN3t3aWR0aDo1OC4zMzMzMzMzMyV9LmNvbC1tZC02e3dpZHRoOjUw
JX0uY29sLW1kLTV7d2lkdGg6NDEuNjY2NjY2NjclfS5jb2wtbWQtNHt3aWR0aDozMy4zMzMzMzMz
MyV9LmNvbC1tZC0ze3dpZHRoOjI1JX0uY29sLW1kLTJ7d2lkdGg6MTYuNjY2NjY2NjclfS5jb2wt
bWQtMXt3aWR0aDo4LjMzMzMzMzMzJX0uY29sLW1kLXB1bGwtMTJ7cmlnaHQ6MTAwJX0uY29sLW1k
LXB1bGwtMTF7cmlnaHQ6OTEuNjY2NjY2NjclfS5jb2wtbWQtcHVsbC0xMHtyaWdodDo4My4zMzMz
MzMzMyV9LmNvbC1tZC1wdWxsLTl7cmlnaHQ6NzUlfS5jb2wtbWQtcHVsbC04e3JpZ2h0OjY2LjY2
NjY2NjY3JX0uY29sLW1kLXB1bGwtN3tyaWdodDo1OC4zMzMzMzMzMyV9LmNvbC1tZC1wdWxsLTZ7
cmlnaHQ6NTAlfS5jb2wtbWQtcHVsbC01e3JpZ2h0OjQxLjY2NjY2NjY3JX0uY29sLW1kLXB1bGwt
NHtyaWdodDozMy4zMzMzMzMzMyV9LmNvbC1tZC1wdWxsLTN7cmlnaHQ6MjUlfS5jb2wtbWQtcHVs
bC0ye3JpZ2h0OjE2LjY2NjY2NjY3JX0uY29sLW1kLXB1bGwtMXtyaWdodDo4LjMzMzMzMzMzJX0u
Y29sLW1kLXB1bGwtMHtyaWdodDphdXRvfS5jb2wtbWQtcHVzaC0xMntsZWZ0OjEwMCV9LmNvbC1t
ZC1wdXNoLTExe2xlZnQ6OTEuNjY2NjY2NjclfS5jb2wtbWQtcHVzaC0xMHtsZWZ0OjgzLjMzMzMz
MzMzJX0uY29sLW1kLXB1c2gtOXtsZWZ0Ojc1JX0uY29sLW1kLXB1c2gtOHtsZWZ0OjY2LjY2NjY2
NjY3JX0uY29sLW1kLXB1c2gtN3tsZWZ0OjU4LjMzMzMzMzMzJX0uY29sLW1kLXB1c2gtNntsZWZ0
OjUwJX0uY29sLW1kLXB1c2gtNXtsZWZ0OjQxLjY2NjY2NjY3JX0uY29sLW1kLXB1c2gtNHtsZWZ0
OjMzLjMzMzMzMzMzJX0uY29sLW1kLXB1c2gtM3tsZWZ0OjI1JX0uY29sLW1kLXB1c2gtMntsZWZ0
OjE2LjY2NjY2NjY3JX0uY29sLW1kLXB1c2gtMXtsZWZ0OjguMzMzMzMzMzMlfS5jb2wtbWQtcHVz
aC0we2xlZnQ6YXV0b30uY29sLW1kLW9mZnNldC0xMnttYXJnaW4tbGVmdDoxMDAlfS5jb2wtbWQt
b2Zmc2V0LTExe21hcmdpbi1sZWZ0OjkxLjY2NjY2NjY3JX0uY29sLW1kLW9mZnNldC0xMHttYXJn
aW4tbGVmdDo4My4zMzMzMzMzMyV9LmNvbC1tZC1vZmZzZXQtOXttYXJnaW4tbGVmdDo3NSV9LmNv
bC1tZC1vZmZzZXQtOHttYXJnaW4tbGVmdDo2Ni42NjY2NjY2NyV9LmNvbC1tZC1vZmZzZXQtN3tt
YXJnaW4tbGVmdDo1OC4zMzMzMzMzMyV9LmNvbC1tZC1vZmZzZXQtNnttYXJnaW4tbGVmdDo1MCV9
LmNvbC1tZC1vZmZzZXQtNXttYXJnaW4tbGVmdDo0MS42NjY2NjY2NyV9LmNvbC1tZC1vZmZzZXQt
NHttYXJnaW4tbGVmdDozMy4zMzMzMzMzMyV9LmNvbC1tZC1vZmZzZXQtM3ttYXJnaW4tbGVmdDoy
NSV9LmNvbC1tZC1vZmZzZXQtMnttYXJnaW4tbGVmdDoxNi42NjY2NjY2NyV9LmNvbC1tZC1vZmZz
ZXQtMXttYXJnaW4tbGVmdDo4LjMzMzMzMzMzJX0uY29sLW1kLW9mZnNldC0we21hcmdpbi1sZWZ0
OjB9fUBtZWRpYSAobWluLXdpZHRoOjEyMDBweCl7LmNvbC1sZy0xLC5jb2wtbGctMTAsLmNvbC1s
Zy0xMSwuY29sLWxnLTEyLC5jb2wtbGctMiwuY29sLWxnLTMsLmNvbC1sZy00LC5jb2wtbGctNSwu
Y29sLWxnLTYsLmNvbC1sZy03LC5jb2wtbGctOCwuY29sLWxnLTl7ZmxvYXQ6bGVmdH0uY29sLWxn
LTEye3dpZHRoOjEwMCV9LmNvbC1sZy0xMXt3aWR0aDo5MS42NjY2NjY2NyV9LmNvbC1sZy0xMHt3
aWR0aDo4My4zMzMzMzMzMyV9LmNvbC1sZy05e3dpZHRoOjc1JX0uY29sLWxnLTh7d2lkdGg6NjYu
NjY2NjY2NjclfS5jb2wtbGctN3t3aWR0aDo1OC4zMzMzMzMzMyV9LmNvbC1sZy02e3dpZHRoOjUw
JX0uY29sLWxnLTV7d2lkdGg6NDEuNjY2NjY2NjclfS5jb2wtbGctNHt3aWR0aDozMy4zMzMzMzMz
MyV9LmNvbC1sZy0ze3dpZHRoOjI1JX0uY29sLWxnLTJ7d2lkdGg6MTYuNjY2NjY2NjclfS5jb2wt
bGctMXt3aWR0aDo4LjMzMzMzMzMzJX0uY29sLWxnLXB1bGwtMTJ7cmlnaHQ6MTAwJX0uY29sLWxn
LXB1bGwtMTF7cmlnaHQ6OTEuNjY2NjY2NjclfS5jb2wtbGctcHVsbC0xMHtyaWdodDo4My4zMzMz
MzMzMyV9LmNvbC1sZy1wdWxsLTl7cmlnaHQ6NzUlfS5jb2wtbGctcHVsbC04e3JpZ2h0OjY2LjY2
NjY2NjY3JX0uY29sLWxnLXB1bGwtN3tyaWdodDo1OC4zMzMzMzMzMyV9LmNvbC1sZy1wdWxsLTZ7
cmlnaHQ6NTAlfS5jb2wtbGctcHVsbC01e3JpZ2h0OjQxLjY2NjY2NjY3JX0uY29sLWxnLXB1bGwt
NHtyaWdodDozMy4zMzMzMzMzMyV9LmNvbC1sZy1wdWxsLTN7cmlnaHQ6MjUlfS5jb2wtbGctcHVs
bC0ye3JpZ2h0OjE2LjY2NjY2NjY3JX0uY29sLWxnLXB1bGwtMXtyaWdodDo4LjMzMzMzMzMzJX0u
Y29sLWxnLXB1bGwtMHtyaWdodDphdXRvfS5jb2wtbGctcHVzaC0xMntsZWZ0OjEwMCV9LmNvbC1s
Zy1wdXNoLTExe2xlZnQ6OTEuNjY2NjY2NjclfS5jb2wtbGctcHVzaC0xMHtsZWZ0OjgzLjMzMzMz
MzMzJX0uY29sLWxnLXB1c2gtOXtsZWZ0Ojc1JX0uY29sLWxnLXB1c2gtOHtsZWZ0OjY2LjY2NjY2
NjY3JX0uY29sLWxnLXB1c2gtN3tsZWZ0OjU4LjMzMzMzMzMzJX0uY29sLWxnLXB1c2gtNntsZWZ0
OjUwJX0uY29sLWxnLXB1c2gtNXtsZWZ0OjQxLjY2NjY2NjY3JX0uY29sLWxnLXB1c2gtNHtsZWZ0
OjMzLjMzMzMzMzMzJX0uY29sLWxnLXB1c2gtM3tsZWZ0OjI1JX0uY29sLWxnLXB1c2gtMntsZWZ0
OjE2LjY2NjY2NjY3JX0uY29sLWxnLXB1c2gtMXtsZWZ0OjguMzMzMzMzMzMlfS5jb2wtbGctcHVz
aC0we2xlZnQ6YXV0b30uY29sLWxnLW9mZnNldC0xMnttYXJnaW4tbGVmdDoxMDAlfS5jb2wtbGct
b2Zmc2V0LTExe21hcmdpbi1sZWZ0OjkxLjY2NjY2NjY3JX0uY29sLWxnLW9mZnNldC0xMHttYXJn
aW4tbGVmdDo4My4zMzMzMzMzMyV9LmNvbC1sZy1vZmZzZXQtOXttYXJnaW4tbGVmdDo3NSV9LmNv
bC1sZy1vZmZzZXQtOHttYXJnaW4tbGVmdDo2Ni42NjY2NjY2NyV9LmNvbC1sZy1vZmZzZXQtN3tt
YXJnaW4tbGVmdDo1OC4zMzMzMzMzMyV9LmNvbC1sZy1vZmZzZXQtNnttYXJnaW4tbGVmdDo1MCV9
LmNvbC1sZy1vZmZzZXQtNXttYXJnaW4tbGVmdDo0MS42NjY2NjY2NyV9LmNvbC1sZy1vZmZzZXQt
NHttYXJnaW4tbGVmdDozMy4zMzMzMzMzMyV9LmNvbC1sZy1vZmZzZXQtM3ttYXJnaW4tbGVmdDoy
NSV9LmNvbC1sZy1vZmZzZXQtMnttYXJnaW4tbGVmdDoxNi42NjY2NjY2NyV9LmNvbC1sZy1vZmZz
ZXQtMXttYXJnaW4tbGVmdDo4LjMzMzMzMzMzJX0uY29sLWxnLW9mZnNldC0we21hcmdpbi1sZWZ0
OjB9fXRhYmxle2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnR9Y2FwdGlvbntwYWRkaW5nLXRv
cDo4cHg7cGFkZGluZy1ib3R0b206OHB4O2NvbG9yOiM3Nzc7dGV4dC1hbGlnbjpsZWZ0fXRoe3Rl
eHQtYWxpZ246bGVmdH0udGFibGV7d2lkdGg6MTAwJTttYXgtd2lkdGg6MTAwJTttYXJnaW4tYm90
dG9tOjIwcHh9LnRhYmxlPnRib2R5PnRyPnRkLC50YWJsZT50Ym9keT50cj50aCwudGFibGU+dGZv
b3Q+dHI+dGQsLnRhYmxlPnRmb290PnRyPnRoLC50YWJsZT50aGVhZD50cj50ZCwudGFibGU+dGhl
YWQ+dHI+dGh7cGFkZGluZzo4cHg7bGluZS1oZWlnaHQ6MS40Mjg1NzE0Mzt2ZXJ0aWNhbC1hbGln
bjp0b3A7Ym9yZGVyLXRvcDoxcHggc29saWQgI2RkZH0udGFibGU+dGhlYWQ+dHI+dGh7dmVydGlj
YWwtYWxpZ246Ym90dG9tO2JvcmRlci1ib3R0b206MnB4IHNvbGlkICNkZGR9LnRhYmxlPmNhcHRp
b24rdGhlYWQ+dHI6Zmlyc3QtY2hpbGQ+dGQsLnRhYmxlPmNhcHRpb24rdGhlYWQ+dHI6Zmlyc3Qt
Y2hpbGQ+dGgsLnRhYmxlPmNvbGdyb3VwK3RoZWFkPnRyOmZpcnN0LWNoaWxkPnRkLC50YWJsZT5j
b2xncm91cCt0aGVhZD50cjpmaXJzdC1jaGlsZD50aCwudGFibGU+dGhlYWQ6Zmlyc3QtY2hpbGQ+
dHI6Zmlyc3QtY2hpbGQ+dGQsLnRhYmxlPnRoZWFkOmZpcnN0LWNoaWxkPnRyOmZpcnN0LWNoaWxk
PnRoe2JvcmRlci10b3A6MH0udGFibGU+dGJvZHkrdGJvZHl7Ym9yZGVyLXRvcDoycHggc29saWQg
I2RkZH0udGFibGUgLnRhYmxle2JhY2tncm91bmQtY29sb3I6I2ZmZn0udGFibGUtY29uZGVuc2Vk
PnRib2R5PnRyPnRkLC50YWJsZS1jb25kZW5zZWQ+dGJvZHk+dHI+dGgsLnRhYmxlLWNvbmRlbnNl
ZD50Zm9vdD50cj50ZCwudGFibGUtY29uZGVuc2VkPnRmb290PnRyPnRoLC50YWJsZS1jb25kZW5z
ZWQ+dGhlYWQ+dHI+dGQsLnRhYmxlLWNvbmRlbnNlZD50aGVhZD50cj50aHtwYWRkaW5nOjVweH0u
dGFibGUtYm9yZGVyZWR7Ym9yZGVyOjFweCBzb2xpZCAjZGRkfS50YWJsZS1ib3JkZXJlZD50Ym9k
eT50cj50ZCwudGFibGUtYm9yZGVyZWQ+dGJvZHk+dHI+dGgsLnRhYmxlLWJvcmRlcmVkPnRmb290
PnRyPnRkLC50YWJsZS1ib3JkZXJlZD50Zm9vdD50cj50aCwudGFibGUtYm9yZGVyZWQ+dGhlYWQ+
dHI+dGQsLnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRyPnRoe2JvcmRlcjoxcHggc29saWQgI2RkZH0u
dGFibGUtYm9yZGVyZWQ+dGhlYWQ+dHI+dGQsLnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRyPnRoe2Jv
cmRlci1ib3R0b20td2lkdGg6MnB4fS50YWJsZS1zdHJpcGVkPnRib2R5PnRyOm50aC1vZi10eXBl
KG9kZCl7YmFja2dyb3VuZC1jb2xvcjojZjlmOWY5fS50YWJsZS1ob3Zlcj50Ym9keT50cjpob3Zl
cntiYWNrZ3JvdW5kLWNvbG9yOiNmNWY1ZjV9dGFibGUgY29sW2NsYXNzKj1jb2wtXXtwb3NpdGlv
bjpzdGF0aWM7ZGlzcGxheTp0YWJsZS1jb2x1bW47ZmxvYXQ6bm9uZX10YWJsZSB0ZFtjbGFzcyo9
Y29sLV0sdGFibGUgdGhbY2xhc3MqPWNvbC1de3Bvc2l0aW9uOnN0YXRpYztkaXNwbGF5OnRhYmxl
LWNlbGw7ZmxvYXQ6bm9uZX0udGFibGU+dGJvZHk+dHIuYWN0aXZlPnRkLC50YWJsZT50Ym9keT50
ci5hY3RpdmU+dGgsLnRhYmxlPnRib2R5PnRyPnRkLmFjdGl2ZSwudGFibGU+dGJvZHk+dHI+dGgu
YWN0aXZlLC50YWJsZT50Zm9vdD50ci5hY3RpdmU+dGQsLnRhYmxlPnRmb290PnRyLmFjdGl2ZT50
aCwudGFibGU+dGZvb3Q+dHI+dGQuYWN0aXZlLC50YWJsZT50Zm9vdD50cj50aC5hY3RpdmUsLnRh
YmxlPnRoZWFkPnRyLmFjdGl2ZT50ZCwudGFibGU+dGhlYWQ+dHIuYWN0aXZlPnRoLC50YWJsZT50
aGVhZD50cj50ZC5hY3RpdmUsLnRhYmxlPnRoZWFkPnRyPnRoLmFjdGl2ZXtiYWNrZ3JvdW5kLWNv
bG9yOiNmNWY1ZjV9LnRhYmxlLWhvdmVyPnRib2R5PnRyLmFjdGl2ZTpob3Zlcj50ZCwudGFibGUt
aG92ZXI+dGJvZHk+dHIuYWN0aXZlOmhvdmVyPnRoLC50YWJsZS1ob3Zlcj50Ym9keT50cjpob3Zl
cj4uYWN0aXZlLC50YWJsZS1ob3Zlcj50Ym9keT50cj50ZC5hY3RpdmU6aG92ZXIsLnRhYmxlLWhv
dmVyPnRib2R5PnRyPnRoLmFjdGl2ZTpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNlOGU4ZTh9LnRh
YmxlPnRib2R5PnRyLnN1Y2Nlc3M+dGQsLnRhYmxlPnRib2R5PnRyLnN1Y2Nlc3M+dGgsLnRhYmxl
PnRib2R5PnRyPnRkLnN1Y2Nlc3MsLnRhYmxlPnRib2R5PnRyPnRoLnN1Y2Nlc3MsLnRhYmxlPnRm
b290PnRyLnN1Y2Nlc3M+dGQsLnRhYmxlPnRmb290PnRyLnN1Y2Nlc3M+dGgsLnRhYmxlPnRmb290
PnRyPnRkLnN1Y2Nlc3MsLnRhYmxlPnRmb290PnRyPnRoLnN1Y2Nlc3MsLnRhYmxlPnRoZWFkPnRy
LnN1Y2Nlc3M+dGQsLnRhYmxlPnRoZWFkPnRyLnN1Y2Nlc3M+dGgsLnRhYmxlPnRoZWFkPnRyPnRk
LnN1Y2Nlc3MsLnRhYmxlPnRoZWFkPnRyPnRoLnN1Y2Nlc3N7YmFja2dyb3VuZC1jb2xvcjojZGZm
MGQ4fS50YWJsZS1ob3Zlcj50Ym9keT50ci5zdWNjZXNzOmhvdmVyPnRkLC50YWJsZS1ob3Zlcj50
Ym9keT50ci5zdWNjZXNzOmhvdmVyPnRoLC50YWJsZS1ob3Zlcj50Ym9keT50cjpob3Zlcj4uc3Vj
Y2VzcywudGFibGUtaG92ZXI+dGJvZHk+dHI+dGQuc3VjY2Vzczpob3ZlciwudGFibGUtaG92ZXI+
dGJvZHk+dHI+dGguc3VjY2Vzczpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNkMGU5YzZ9LnRhYmxl
PnRib2R5PnRyLmluZm8+dGQsLnRhYmxlPnRib2R5PnRyLmluZm8+dGgsLnRhYmxlPnRib2R5PnRy
PnRkLmluZm8sLnRhYmxlPnRib2R5PnRyPnRoLmluZm8sLnRhYmxlPnRmb290PnRyLmluZm8+dGQs
LnRhYmxlPnRmb290PnRyLmluZm8+dGgsLnRhYmxlPnRmb290PnRyPnRkLmluZm8sLnRhYmxlPnRm
b290PnRyPnRoLmluZm8sLnRhYmxlPnRoZWFkPnRyLmluZm8+dGQsLnRhYmxlPnRoZWFkPnRyLmlu
Zm8+dGgsLnRhYmxlPnRoZWFkPnRyPnRkLmluZm8sLnRhYmxlPnRoZWFkPnRyPnRoLmluZm97YmFj
a2dyb3VuZC1jb2xvcjojZDllZGY3fS50YWJsZS1ob3Zlcj50Ym9keT50ci5pbmZvOmhvdmVyPnRk
LC50YWJsZS1ob3Zlcj50Ym9keT50ci5pbmZvOmhvdmVyPnRoLC50YWJsZS1ob3Zlcj50Ym9keT50
cjpob3Zlcj4uaW5mbywudGFibGUtaG92ZXI+dGJvZHk+dHI+dGQuaW5mbzpob3ZlciwudGFibGUt
aG92ZXI+dGJvZHk+dHI+dGguaW5mbzpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNjNGUzZjN9LnRh
YmxlPnRib2R5PnRyLndhcm5pbmc+dGQsLnRhYmxlPnRib2R5PnRyLndhcm5pbmc+dGgsLnRhYmxl
PnRib2R5PnRyPnRkLndhcm5pbmcsLnRhYmxlPnRib2R5PnRyPnRoLndhcm5pbmcsLnRhYmxlPnRm
b290PnRyLndhcm5pbmc+dGQsLnRhYmxlPnRmb290PnRyLndhcm5pbmc+dGgsLnRhYmxlPnRmb290
PnRyPnRkLndhcm5pbmcsLnRhYmxlPnRmb290PnRyPnRoLndhcm5pbmcsLnRhYmxlPnRoZWFkPnRy
Lndhcm5pbmc+dGQsLnRhYmxlPnRoZWFkPnRyLndhcm5pbmc+dGgsLnRhYmxlPnRoZWFkPnRyPnRk
Lndhcm5pbmcsLnRhYmxlPnRoZWFkPnRyPnRoLndhcm5pbmd7YmFja2dyb3VuZC1jb2xvcjojZmNm
OGUzfS50YWJsZS1ob3Zlcj50Ym9keT50ci53YXJuaW5nOmhvdmVyPnRkLC50YWJsZS1ob3Zlcj50
Ym9keT50ci53YXJuaW5nOmhvdmVyPnRoLC50YWJsZS1ob3Zlcj50Ym9keT50cjpob3Zlcj4ud2Fy
bmluZywudGFibGUtaG92ZXI+dGJvZHk+dHI+dGQud2FybmluZzpob3ZlciwudGFibGUtaG92ZXI+
dGJvZHk+dHI+dGgud2FybmluZzpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNmYWYyY2N9LnRhYmxl
PnRib2R5PnRyLmRhbmdlcj50ZCwudGFibGU+dGJvZHk+dHIuZGFuZ2VyPnRoLC50YWJsZT50Ym9k
eT50cj50ZC5kYW5nZXIsLnRhYmxlPnRib2R5PnRyPnRoLmRhbmdlciwudGFibGU+dGZvb3Q+dHIu
ZGFuZ2VyPnRkLC50YWJsZT50Zm9vdD50ci5kYW5nZXI+dGgsLnRhYmxlPnRmb290PnRyPnRkLmRh
bmdlciwudGFibGU+dGZvb3Q+dHI+dGguZGFuZ2VyLC50YWJsZT50aGVhZD50ci5kYW5nZXI+dGQs
LnRhYmxlPnRoZWFkPnRyLmRhbmdlcj50aCwudGFibGU+dGhlYWQ+dHI+dGQuZGFuZ2VyLC50YWJs
ZT50aGVhZD50cj50aC5kYW5nZXJ7YmFja2dyb3VuZC1jb2xvcjojZjJkZWRlfS50YWJsZS1ob3Zl
cj50Ym9keT50ci5kYW5nZXI6aG92ZXI+dGQsLnRhYmxlLWhvdmVyPnRib2R5PnRyLmRhbmdlcjpo
b3Zlcj50aCwudGFibGUtaG92ZXI+dGJvZHk+dHI6aG92ZXI+LmRhbmdlciwudGFibGUtaG92ZXI+
dGJvZHk+dHI+dGQuZGFuZ2VyOmhvdmVyLC50YWJsZS1ob3Zlcj50Ym9keT50cj50aC5kYW5nZXI6
aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojZWJjY2NjfS50YWJsZS1yZXNwb25zaXZle21pbi1oZWln
aHQ6LjAxJTtvdmVyZmxvdy14OmF1dG99QG1lZGlhIHNjcmVlbiBhbmQgKG1heC13aWR0aDo3Njdw
eCl7LnRhYmxlLXJlc3BvbnNpdmV7d2lkdGg6MTAwJTttYXJnaW4tYm90dG9tOjE1cHg7b3ZlcmZs
b3cteTpoaWRkZW47LW1zLW92ZXJmbG93LXN0eWxlOi1tcy1hdXRvaGlkaW5nLXNjcm9sbGJhcjti
b3JkZXI6MXB4IHNvbGlkICNkZGR9LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxle21hcmdpbi1ib3R0
b206MH0udGFibGUtcmVzcG9uc2l2ZT4udGFibGU+dGJvZHk+dHI+dGQsLnRhYmxlLXJlc3BvbnNp
dmU+LnRhYmxlPnRib2R5PnRyPnRoLC50YWJsZS1yZXNwb25zaXZlPi50YWJsZT50Zm9vdD50cj50
ZCwudGFibGUtcmVzcG9uc2l2ZT4udGFibGU+dGZvb3Q+dHI+dGgsLnRhYmxlLXJlc3BvbnNpdmU+
LnRhYmxlPnRoZWFkPnRyPnRkLC50YWJsZS1yZXNwb25zaXZlPi50YWJsZT50aGVhZD50cj50aHt3
aGl0ZS1zcGFjZTpub3dyYXB9LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVke2JvcmRl
cjowfS50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50Ym9keT50cj50ZDpmaXJzdC1j
aGlsZCwudGFibGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGJvZHk+dHI+dGg6Zmlyc3Qt
Y2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyPnRkOmZpcnN0
LWNoaWxkLC50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50Zm9vdD50cj50aDpmaXJz
dC1jaGlsZCwudGFibGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGhlYWQ+dHI+dGQ6Zmly
c3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRyPnRoOmZp
cnN0LWNoaWxke2JvcmRlci1sZWZ0OjB9LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRib2R5PnRyPnRkOmxhc3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRib2R5PnRyPnRoOmxhc3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRmb290PnRyPnRkOmxhc3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRmb290PnRyPnRoOmxhc3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRoZWFkPnRyPnRkOmxhc3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRoZWFkPnRyPnRoOmxhc3QtY2hpbGR7Ym9yZGVyLXJpZ2h0OjB9LnRhYmxlLXJlc3BvbnNpdmU+
LnRhYmxlLWJvcmRlcmVkPnRib2R5PnRyOmxhc3QtY2hpbGQ+dGQsLnRhYmxlLXJlc3BvbnNpdmU+
LnRhYmxlLWJvcmRlcmVkPnRib2R5PnRyOmxhc3QtY2hpbGQ+dGgsLnRhYmxlLXJlc3BvbnNpdmU+
LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyOmxhc3QtY2hpbGQ+dGQsLnRhYmxlLXJlc3BvbnNpdmU+
LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyOmxhc3QtY2hpbGQ+dGh7Ym9yZGVyLWJvdHRvbTowfX1m
aWVsZHNldHttaW4td2lkdGg6MDtwYWRkaW5nOjA7bWFyZ2luOjA7Ym9yZGVyOjB9bGVnZW5ke2Rp
c3BsYXk6YmxvY2s7d2lkdGg6MTAwJTtwYWRkaW5nOjA7bWFyZ2luLWJvdHRvbToyMHB4O2ZvbnQt
c2l6ZToyMXB4O2xpbmUtaGVpZ2h0OmluaGVyaXQ7Y29sb3I6IzMzMztib3JkZXI6MDtib3JkZXIt
Ym90dG9tOjFweCBzb2xpZCAjZTVlNWU1fWxhYmVse2Rpc3BsYXk6aW5saW5lLWJsb2NrO21heC13
aWR0aDoxMDAlO21hcmdpbi1ib3R0b206NXB4O2ZvbnQtd2VpZ2h0OjcwMH1pbnB1dFt0eXBlPXNl
YXJjaF17LXdlYmtpdC1ib3gtc2l6aW5nOmJvcmRlci1ib3g7LW1vei1ib3gtc2l6aW5nOmJvcmRl
ci1ib3g7Ym94LXNpemluZzpib3JkZXItYm94fWlucHV0W3R5cGU9Y2hlY2tib3hdLGlucHV0W3R5
cGU9cmFkaW9de21hcmdpbjo0cHggMCAwO21hcmdpbi10b3A6MXB4XDk7bGluZS1oZWlnaHQ6bm9y
bWFsfWlucHV0W3R5cGU9ZmlsZV17ZGlzcGxheTpibG9ja31pbnB1dFt0eXBlPXJhbmdlXXtkaXNw
bGF5OmJsb2NrO3dpZHRoOjEwMCV9c2VsZWN0W211bHRpcGxlXSxzZWxlY3Rbc2l6ZV17aGVpZ2h0
OmF1dG99aW5wdXRbdHlwZT1maWxlXTpmb2N1cyxpbnB1dFt0eXBlPWNoZWNrYm94XTpmb2N1cyxp
bnB1dFt0eXBlPXJhZGlvXTpmb2N1c3tvdXRsaW5lOjVweCBhdXRvIC13ZWJraXQtZm9jdXMtcmlu
Zy1jb2xvcjtvdXRsaW5lLW9mZnNldDotMnB4fW91dHB1dHtkaXNwbGF5OmJsb2NrO3BhZGRpbmct
dG9wOjdweDtmb250LXNpemU6MTRweDtsaW5lLWhlaWdodDoxLjQyODU3MTQzO2NvbG9yOiM1NTV9
LmZvcm0tY29udHJvbHtkaXNwbGF5OmJsb2NrO3dpZHRoOjEwMCU7aGVpZ2h0OjM0cHg7cGFkZGlu
Zzo2cHggMTJweDtmb250LXNpemU6MTRweDtsaW5lLWhlaWdodDoxLjQyODU3MTQzO2NvbG9yOiM1
NTU7YmFja2dyb3VuZC1jb2xvcjojZmZmO2JhY2tncm91bmQtaW1hZ2U6bm9uZTtib3JkZXI6MXB4
IHNvbGlkICNjY2M7Ym9yZGVyLXJhZGl1czo0cHg7LXdlYmtpdC1ib3gtc2hhZG93Omluc2V0IDAg
MXB4IDFweCByZ2JhKDAsMCwwLC4wNzUpO2JveC1zaGFkb3c6aW5zZXQgMCAxcHggMXB4IHJnYmEo
MCwwLDAsLjA3NSk7LXdlYmtpdC10cmFuc2l0aW9uOmJvcmRlci1jb2xvciBlYXNlLWluLW91dCAu
MTVzLC13ZWJraXQtYm94LXNoYWRvdyBlYXNlLWluLW91dCAuMTVzOy1vLXRyYW5zaXRpb246Ym9y
ZGVyLWNvbG9yIGVhc2UtaW4tb3V0IC4xNXMsYm94LXNoYWRvdyBlYXNlLWluLW91dCAuMTVzO3Ry
YW5zaXRpb246Ym9yZGVyLWNvbG9yIGVhc2UtaW4tb3V0IC4xNXMsYm94LXNoYWRvdyBlYXNlLWlu
LW91dCAuMTVzfS5mb3JtLWNvbnRyb2w6Zm9jdXN7Ym9yZGVyLWNvbG9yOiM2NmFmZTk7b3V0bGlu
ZTowOy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFweCAxcHggcmdiYSgwLDAsMCwuMDc1KSww
IDAgOHB4IHJnYmEoMTAyLDE3NSwyMzMsLjYpO2JveC1zaGFkb3c6aW5zZXQgMCAxcHggMXB4IHJn
YmEoMCwwLDAsLjA3NSksMCAwIDhweCByZ2JhKDEwMiwxNzUsMjMzLC42KX0uZm9ybS1jb250cm9s
OjotbW96LXBsYWNlaG9sZGVye2NvbG9yOiM5OTk7b3BhY2l0eToxfS5mb3JtLWNvbnRyb2w6LW1z
LWlucHV0LXBsYWNlaG9sZGVye2NvbG9yOiM5OTl9LmZvcm0tY29udHJvbDo6LXdlYmtpdC1pbnB1
dC1wbGFjZWhvbGRlcntjb2xvcjojOTk5fS5mb3JtLWNvbnRyb2w6Oi1tcy1leHBhbmR7YmFja2dy
b3VuZC1jb2xvcjp0cmFuc3BhcmVudDtib3JkZXI6MH0uZm9ybS1jb250cm9sW2Rpc2FibGVkXSwu
Zm9ybS1jb250cm9sW3JlYWRvbmx5XSxmaWVsZHNldFtkaXNhYmxlZF0gLmZvcm0tY29udHJvbHti
YWNrZ3JvdW5kLWNvbG9yOiNlZWU7b3BhY2l0eToxfS5mb3JtLWNvbnRyb2xbZGlzYWJsZWRdLGZp
ZWxkc2V0W2Rpc2FibGVkXSAuZm9ybS1jb250cm9se2N1cnNvcjpub3QtYWxsb3dlZH10ZXh0YXJl
YS5mb3JtLWNvbnRyb2x7aGVpZ2h0OmF1dG99aW5wdXRbdHlwZT1zZWFyY2hdey13ZWJraXQtYXBw
ZWFyYW5jZTpub25lfUBtZWRpYSBzY3JlZW4gYW5kICgtd2Via2l0LW1pbi1kZXZpY2UtcGl4ZWwt
cmF0aW86MCl7aW5wdXRbdHlwZT1kYXRlXS5mb3JtLWNvbnRyb2wsaW5wdXRbdHlwZT10aW1lXS5m
b3JtLWNvbnRyb2wsaW5wdXRbdHlwZT1kYXRldGltZS1sb2NhbF0uZm9ybS1jb250cm9sLGlucHV0
W3R5cGU9bW9udGhdLmZvcm0tY29udHJvbHtsaW5lLWhlaWdodDozNHB4fS5pbnB1dC1ncm91cC1z
bSBpbnB1dFt0eXBlPWRhdGVdLC5pbnB1dC1ncm91cC1zbSBpbnB1dFt0eXBlPXRpbWVdLC5pbnB1
dC1ncm91cC1zbSBpbnB1dFt0eXBlPWRhdGV0aW1lLWxvY2FsXSwuaW5wdXQtZ3JvdXAtc20gaW5w
dXRbdHlwZT1tb250aF0saW5wdXRbdHlwZT1kYXRlXS5pbnB1dC1zbSxpbnB1dFt0eXBlPXRpbWVd
LmlucHV0LXNtLGlucHV0W3R5cGU9ZGF0ZXRpbWUtbG9jYWxdLmlucHV0LXNtLGlucHV0W3R5cGU9
bW9udGhdLmlucHV0LXNte2xpbmUtaGVpZ2h0OjMwcHh9LmlucHV0LWdyb3VwLWxnIGlucHV0W3R5
cGU9ZGF0ZV0sLmlucHV0LWdyb3VwLWxnIGlucHV0W3R5cGU9dGltZV0sLmlucHV0LWdyb3VwLWxn
IGlucHV0W3R5cGU9ZGF0ZXRpbWUtbG9jYWxdLC5pbnB1dC1ncm91cC1sZyBpbnB1dFt0eXBlPW1v
bnRoXSxpbnB1dFt0eXBlPWRhdGVdLmlucHV0LWxnLGlucHV0W3R5cGU9dGltZV0uaW5wdXQtbGcs
aW5wdXRbdHlwZT1kYXRldGltZS1sb2NhbF0uaW5wdXQtbGcsaW5wdXRbdHlwZT1tb250aF0uaW5w
dXQtbGd7bGluZS1oZWlnaHQ6NDZweH19LmZvcm0tZ3JvdXB7bWFyZ2luLWJvdHRvbToxNXB4fS5j
aGVja2JveCwucmFkaW97cG9zaXRpb246cmVsYXRpdmU7ZGlzcGxheTpibG9jazttYXJnaW4tdG9w
OjEwcHg7bWFyZ2luLWJvdHRvbToxMHB4fS5jaGVja2JveCBsYWJlbCwucmFkaW8gbGFiZWx7bWlu
LWhlaWdodDoyMHB4O3BhZGRpbmctbGVmdDoyMHB4O21hcmdpbi1ib3R0b206MDtmb250LXdlaWdo
dDo0MDA7Y3Vyc29yOnBvaW50ZXJ9LmNoZWNrYm94IGlucHV0W3R5cGU9Y2hlY2tib3hdLC5jaGVj
a2JveC1pbmxpbmUgaW5wdXRbdHlwZT1jaGVja2JveF0sLnJhZGlvIGlucHV0W3R5cGU9cmFkaW9d
LC5yYWRpby1pbmxpbmUgaW5wdXRbdHlwZT1yYWRpb117cG9zaXRpb246YWJzb2x1dGU7bWFyZ2lu
LXRvcDo0cHhcOTttYXJnaW4tbGVmdDotMjBweH0uY2hlY2tib3grLmNoZWNrYm94LC5yYWRpbysu
cmFkaW97bWFyZ2luLXRvcDotNXB4fS5jaGVja2JveC1pbmxpbmUsLnJhZGlvLWlubGluZXtwb3Np
dGlvbjpyZWxhdGl2ZTtkaXNwbGF5OmlubGluZS1ibG9jaztwYWRkaW5nLWxlZnQ6MjBweDttYXJn
aW4tYm90dG9tOjA7Zm9udC13ZWlnaHQ6NDAwO3ZlcnRpY2FsLWFsaWduOm1pZGRsZTtjdXJzb3I6
cG9pbnRlcn0uY2hlY2tib3gtaW5saW5lKy5jaGVja2JveC1pbmxpbmUsLnJhZGlvLWlubGluZSsu
cmFkaW8taW5saW5le21hcmdpbi10b3A6MDttYXJnaW4tbGVmdDoxMHB4fWZpZWxkc2V0W2Rpc2Fi
bGVkXSBpbnB1dFt0eXBlPWNoZWNrYm94XSxmaWVsZHNldFtkaXNhYmxlZF0gaW5wdXRbdHlwZT1y
YWRpb10saW5wdXRbdHlwZT1jaGVja2JveF0uZGlzYWJsZWQsaW5wdXRbdHlwZT1jaGVja2JveF1b
ZGlzYWJsZWRdLGlucHV0W3R5cGU9cmFkaW9dLmRpc2FibGVkLGlucHV0W3R5cGU9cmFkaW9dW2Rp
c2FibGVkXXtjdXJzb3I6bm90LWFsbG93ZWR9LmNoZWNrYm94LWlubGluZS5kaXNhYmxlZCwucmFk
aW8taW5saW5lLmRpc2FibGVkLGZpZWxkc2V0W2Rpc2FibGVkXSAuY2hlY2tib3gtaW5saW5lLGZp
ZWxkc2V0W2Rpc2FibGVkXSAucmFkaW8taW5saW5le2N1cnNvcjpub3QtYWxsb3dlZH0uY2hlY2ti
b3guZGlzYWJsZWQgbGFiZWwsLnJhZGlvLmRpc2FibGVkIGxhYmVsLGZpZWxkc2V0W2Rpc2FibGVk
XSAuY2hlY2tib3ggbGFiZWwsZmllbGRzZXRbZGlzYWJsZWRdIC5yYWRpbyBsYWJlbHtjdXJzb3I6
bm90LWFsbG93ZWR9LmZvcm0tY29udHJvbC1zdGF0aWN7bWluLWhlaWdodDozNHB4O3BhZGRpbmct
dG9wOjdweDtwYWRkaW5nLWJvdHRvbTo3cHg7bWFyZ2luLWJvdHRvbTowfS5mb3JtLWNvbnRyb2wt
c3RhdGljLmlucHV0LWxnLC5mb3JtLWNvbnRyb2wtc3RhdGljLmlucHV0LXNte3BhZGRpbmctcmln
aHQ6MDtwYWRkaW5nLWxlZnQ6MH0uaW5wdXQtc217aGVpZ2h0OjMwcHg7cGFkZGluZzo1cHggMTBw
eDtmb250LXNpemU6MTJweDtsaW5lLWhlaWdodDoxLjU7Ym9yZGVyLXJhZGl1czozcHh9c2VsZWN0
LmlucHV0LXNte2hlaWdodDozMHB4O2xpbmUtaGVpZ2h0OjMwcHh9c2VsZWN0W211bHRpcGxlXS5p
bnB1dC1zbSx0ZXh0YXJlYS5pbnB1dC1zbXtoZWlnaHQ6YXV0b30uZm9ybS1ncm91cC1zbSAuZm9y
bS1jb250cm9se2hlaWdodDozMHB4O3BhZGRpbmc6NXB4IDEwcHg7Zm9udC1zaXplOjEycHg7bGlu
ZS1oZWlnaHQ6MS41O2JvcmRlci1yYWRpdXM6M3B4fS5mb3JtLWdyb3VwLXNtIHNlbGVjdC5mb3Jt
LWNvbnRyb2x7aGVpZ2h0OjMwcHg7bGluZS1oZWlnaHQ6MzBweH0uZm9ybS1ncm91cC1zbSBzZWxl
Y3RbbXVsdGlwbGVdLmZvcm0tY29udHJvbCwuZm9ybS1ncm91cC1zbSB0ZXh0YXJlYS5mb3JtLWNv
bnRyb2x7aGVpZ2h0OmF1dG99LmZvcm0tZ3JvdXAtc20gLmZvcm0tY29udHJvbC1zdGF0aWN7aGVp
Z2h0OjMwcHg7bWluLWhlaWdodDozMnB4O3BhZGRpbmc6NnB4IDEwcHg7Zm9udC1zaXplOjEycHg7
bGluZS1oZWlnaHQ6MS41fS5pbnB1dC1sZ3toZWlnaHQ6NDZweDtwYWRkaW5nOjEwcHggMTZweDtm
b250LXNpemU6MThweDtsaW5lLWhlaWdodDoxLjMzMzMzMzM7Ym9yZGVyLXJhZGl1czo2cHh9c2Vs
ZWN0LmlucHV0LWxne2hlaWdodDo0NnB4O2xpbmUtaGVpZ2h0OjQ2cHh9c2VsZWN0W211bHRpcGxl
XS5pbnB1dC1sZyx0ZXh0YXJlYS5pbnB1dC1sZ3toZWlnaHQ6YXV0b30uZm9ybS1ncm91cC1sZyAu
Zm9ybS1jb250cm9se2hlaWdodDo0NnB4O3BhZGRpbmc6MTBweCAxNnB4O2ZvbnQtc2l6ZToxOHB4
O2xpbmUtaGVpZ2h0OjEuMzMzMzMzMztib3JkZXItcmFkaXVzOjZweH0uZm9ybS1ncm91cC1sZyBz
ZWxlY3QuZm9ybS1jb250cm9se2hlaWdodDo0NnB4O2xpbmUtaGVpZ2h0OjQ2cHh9LmZvcm0tZ3Jv
dXAtbGcgc2VsZWN0W211bHRpcGxlXS5mb3JtLWNvbnRyb2wsLmZvcm0tZ3JvdXAtbGcgdGV4dGFy
ZWEuZm9ybS1jb250cm9se2hlaWdodDphdXRvfS5mb3JtLWdyb3VwLWxnIC5mb3JtLWNvbnRyb2wt
c3RhdGlje2hlaWdodDo0NnB4O21pbi1oZWlnaHQ6MzhweDtwYWRkaW5nOjExcHggMTZweDtmb250
LXNpemU6MThweDtsaW5lLWhlaWdodDoxLjMzMzMzMzN9Lmhhcy1mZWVkYmFja3twb3NpdGlvbjpy
ZWxhdGl2ZX0uaGFzLWZlZWRiYWNrIC5mb3JtLWNvbnRyb2x7cGFkZGluZy1yaWdodDo0Mi41cHh9
LmZvcm0tY29udHJvbC1mZWVkYmFja3twb3NpdGlvbjphYnNvbHV0ZTt0b3A6MDtyaWdodDowO3ot
aW5kZXg6MjtkaXNwbGF5OmJsb2NrO3dpZHRoOjM0cHg7aGVpZ2h0OjM0cHg7bGluZS1oZWlnaHQ6
MzRweDt0ZXh0LWFsaWduOmNlbnRlcjtwb2ludGVyLWV2ZW50czpub25lfS5mb3JtLWdyb3VwLWxn
IC5mb3JtLWNvbnRyb2wrLmZvcm0tY29udHJvbC1mZWVkYmFjaywuaW5wdXQtZ3JvdXAtbGcrLmZv
cm0tY29udHJvbC1mZWVkYmFjaywuaW5wdXQtbGcrLmZvcm0tY29udHJvbC1mZWVkYmFja3t3aWR0
aDo0NnB4O2hlaWdodDo0NnB4O2xpbmUtaGVpZ2h0OjQ2cHh9LmZvcm0tZ3JvdXAtc20gLmZvcm0t
Y29udHJvbCsuZm9ybS1jb250cm9sLWZlZWRiYWNrLC5pbnB1dC1ncm91cC1zbSsuZm9ybS1jb250
cm9sLWZlZWRiYWNrLC5pbnB1dC1zbSsuZm9ybS1jb250cm9sLWZlZWRiYWNre3dpZHRoOjMwcHg7
aGVpZ2h0OjMwcHg7bGluZS1oZWlnaHQ6MzBweH0uaGFzLXN1Y2Nlc3MgLmNoZWNrYm94LC5oYXMt
c3VjY2VzcyAuY2hlY2tib3gtaW5saW5lLC5oYXMtc3VjY2VzcyAuY29udHJvbC1sYWJlbCwuaGFz
LXN1Y2Nlc3MgLmhlbHAtYmxvY2ssLmhhcy1zdWNjZXNzIC5yYWRpbywuaGFzLXN1Y2Nlc3MgLnJh
ZGlvLWlubGluZSwuaGFzLXN1Y2Nlc3MuY2hlY2tib3ggbGFiZWwsLmhhcy1zdWNjZXNzLmNoZWNr
Ym94LWlubGluZSBsYWJlbCwuaGFzLXN1Y2Nlc3MucmFkaW8gbGFiZWwsLmhhcy1zdWNjZXNzLnJh
ZGlvLWlubGluZSBsYWJlbHtjb2xvcjojM2M3NjNkfS5oYXMtc3VjY2VzcyAuZm9ybS1jb250cm9s
e2JvcmRlci1jb2xvcjojM2M3NjNkOy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFweCAxcHgg
cmdiYSgwLDAsMCwuMDc1KTtib3gtc2hhZG93Omluc2V0IDAgMXB4IDFweCByZ2JhKDAsMCwwLC4w
NzUpfS5oYXMtc3VjY2VzcyAuZm9ybS1jb250cm9sOmZvY3Vze2JvcmRlci1jb2xvcjojMmI1NDJj
Oy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFweCAxcHggcmdiYSgwLDAsMCwuMDc1KSwwIDAg
NnB4ICM2N2IxNjg7Ym94LXNoYWRvdzppbnNldCAwIDFweCAxcHggcmdiYSgwLDAsMCwuMDc1KSww
IDAgNnB4ICM2N2IxNjh9Lmhhcy1zdWNjZXNzIC5pbnB1dC1ncm91cC1hZGRvbntjb2xvcjojM2M3
NjNkO2JhY2tncm91bmQtY29sb3I6I2RmZjBkODtib3JkZXItY29sb3I6IzNjNzYzZH0uaGFzLXN1
Y2Nlc3MgLmZvcm0tY29udHJvbC1mZWVkYmFja3tjb2xvcjojM2M3NjNkfS5oYXMtd2FybmluZyAu
Y2hlY2tib3gsLmhhcy13YXJuaW5nIC5jaGVja2JveC1pbmxpbmUsLmhhcy13YXJuaW5nIC5jb250
cm9sLWxhYmVsLC5oYXMtd2FybmluZyAuaGVscC1ibG9jaywuaGFzLXdhcm5pbmcgLnJhZGlvLC5o
YXMtd2FybmluZyAucmFkaW8taW5saW5lLC5oYXMtd2FybmluZy5jaGVja2JveCBsYWJlbCwuaGFz
LXdhcm5pbmcuY2hlY2tib3gtaW5saW5lIGxhYmVsLC5oYXMtd2FybmluZy5yYWRpbyBsYWJlbCwu
aGFzLXdhcm5pbmcucmFkaW8taW5saW5lIGxhYmVse2NvbG9yOiM4YTZkM2J9Lmhhcy13YXJuaW5n
IC5mb3JtLWNvbnRyb2x7Ym9yZGVyLWNvbG9yOiM4YTZkM2I7LXdlYmtpdC1ib3gtc2hhZG93Omlu
c2V0IDAgMXB4IDFweCByZ2JhKDAsMCwwLC4wNzUpO2JveC1zaGFkb3c6aW5zZXQgMCAxcHggMXB4
IHJnYmEoMCwwLDAsLjA3NSl9Lmhhcy13YXJuaW5nIC5mb3JtLWNvbnRyb2w6Zm9jdXN7Ym9yZGVy
LWNvbG9yOiM2NjUxMmM7LXdlYmtpdC1ib3gtc2hhZG93Omluc2V0IDAgMXB4IDFweCByZ2JhKDAs
MCwwLC4wNzUpLDAgMCA2cHggI2MwYTE2Yjtib3gtc2hhZG93Omluc2V0IDAgMXB4IDFweCByZ2Jh
KDAsMCwwLC4wNzUpLDAgMCA2cHggI2MwYTE2Yn0uaGFzLXdhcm5pbmcgLmlucHV0LWdyb3VwLWFk
ZG9ue2NvbG9yOiM4YTZkM2I7YmFja2dyb3VuZC1jb2xvcjojZmNmOGUzO2JvcmRlci1jb2xvcjoj
OGE2ZDNifS5oYXMtd2FybmluZyAuZm9ybS1jb250cm9sLWZlZWRiYWNre2NvbG9yOiM4YTZkM2J9
Lmhhcy1lcnJvciAuY2hlY2tib3gsLmhhcy1lcnJvciAuY2hlY2tib3gtaW5saW5lLC5oYXMtZXJy
b3IgLmNvbnRyb2wtbGFiZWwsLmhhcy1lcnJvciAuaGVscC1ibG9jaywuaGFzLWVycm9yIC5yYWRp
bywuaGFzLWVycm9yIC5yYWRpby1pbmxpbmUsLmhhcy1lcnJvci5jaGVja2JveCBsYWJlbCwuaGFz
LWVycm9yLmNoZWNrYm94LWlubGluZSBsYWJlbCwuaGFzLWVycm9yLnJhZGlvIGxhYmVsLC5oYXMt
ZXJyb3IucmFkaW8taW5saW5lIGxhYmVse2NvbG9yOiNhOTQ0NDJ9Lmhhcy1lcnJvciAuZm9ybS1j
b250cm9se2JvcmRlci1jb2xvcjojYTk0NDQyOy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFw
eCAxcHggcmdiYSgwLDAsMCwuMDc1KTtib3gtc2hhZG93Omluc2V0IDAgMXB4IDFweCByZ2JhKDAs
MCwwLC4wNzUpfS5oYXMtZXJyb3IgLmZvcm0tY29udHJvbDpmb2N1c3tib3JkZXItY29sb3I6Izg0
MzUzNDstd2Via2l0LWJveC1zaGFkb3c6aW5zZXQgMCAxcHggMXB4IHJnYmEoMCwwLDAsLjA3NSks
MCAwIDZweCAjY2U4NDgzO2JveC1zaGFkb3c6aW5zZXQgMCAxcHggMXB4IHJnYmEoMCwwLDAsLjA3
NSksMCAwIDZweCAjY2U4NDgzfS5oYXMtZXJyb3IgLmlucHV0LWdyb3VwLWFkZG9ue2NvbG9yOiNh
OTQ0NDI7YmFja2dyb3VuZC1jb2xvcjojZjJkZWRlO2JvcmRlci1jb2xvcjojYTk0NDQyfS5oYXMt
ZXJyb3IgLmZvcm0tY29udHJvbC1mZWVkYmFja3tjb2xvcjojYTk0NDQyfS5oYXMtZmVlZGJhY2sg
bGFiZWx+LmZvcm0tY29udHJvbC1mZWVkYmFja3t0b3A6MjVweH0uaGFzLWZlZWRiYWNrIGxhYmVs
LnNyLW9ubHl+LmZvcm0tY29udHJvbC1mZWVkYmFja3t0b3A6MH0uaGVscC1ibG9ja3tkaXNwbGF5
OmJsb2NrO21hcmdpbi10b3A6NXB4O21hcmdpbi1ib3R0b206MTBweDtjb2xvcjojNzM3MzczfUBt
ZWRpYSAobWluLXdpZHRoOjc2OHB4KXsuZm9ybS1pbmxpbmUgLmZvcm0tZ3JvdXB7ZGlzcGxheTpp
bmxpbmUtYmxvY2s7bWFyZ2luLWJvdHRvbTowO3ZlcnRpY2FsLWFsaWduOm1pZGRsZX0uZm9ybS1p
bmxpbmUgLmZvcm0tY29udHJvbHtkaXNwbGF5OmlubGluZS1ibG9jazt3aWR0aDphdXRvO3ZlcnRp
Y2FsLWFsaWduOm1pZGRsZX0uZm9ybS1pbmxpbmUgLmZvcm0tY29udHJvbC1zdGF0aWN7ZGlzcGxh
eTppbmxpbmUtYmxvY2t9LmZvcm0taW5saW5lIC5pbnB1dC1ncm91cHtkaXNwbGF5OmlubGluZS10
YWJsZTt2ZXJ0aWNhbC1hbGlnbjptaWRkbGV9LmZvcm0taW5saW5lIC5pbnB1dC1ncm91cCAuZm9y
bS1jb250cm9sLC5mb3JtLWlubGluZSAuaW5wdXQtZ3JvdXAgLmlucHV0LWdyb3VwLWFkZG9uLC5m
b3JtLWlubGluZSAuaW5wdXQtZ3JvdXAgLmlucHV0LWdyb3VwLWJ0bnt3aWR0aDphdXRvfS5mb3Jt
LWlubGluZSAuaW5wdXQtZ3JvdXA+LmZvcm0tY29udHJvbHt3aWR0aDoxMDAlfS5mb3JtLWlubGlu
ZSAuY29udHJvbC1sYWJlbHttYXJnaW4tYm90dG9tOjA7dmVydGljYWwtYWxpZ246bWlkZGxlfS5m
b3JtLWlubGluZSAuY2hlY2tib3gsLmZvcm0taW5saW5lIC5yYWRpb3tkaXNwbGF5OmlubGluZS1i
bG9jazttYXJnaW4tdG9wOjA7bWFyZ2luLWJvdHRvbTowO3ZlcnRpY2FsLWFsaWduOm1pZGRsZX0u
Zm9ybS1pbmxpbmUgLmNoZWNrYm94IGxhYmVsLC5mb3JtLWlubGluZSAucmFkaW8gbGFiZWx7cGFk
ZGluZy1sZWZ0OjB9LmZvcm0taW5saW5lIC5jaGVja2JveCBpbnB1dFt0eXBlPWNoZWNrYm94XSwu
Zm9ybS1pbmxpbmUgLnJhZGlvIGlucHV0W3R5cGU9cmFkaW9de3Bvc2l0aW9uOnJlbGF0aXZlO21h
cmdpbi1sZWZ0OjB9LmZvcm0taW5saW5lIC5oYXMtZmVlZGJhY2sgLmZvcm0tY29udHJvbC1mZWVk
YmFja3t0b3A6MH19LmZvcm0taG9yaXpvbnRhbCAuY2hlY2tib3gsLmZvcm0taG9yaXpvbnRhbCAu
Y2hlY2tib3gtaW5saW5lLC5mb3JtLWhvcml6b250YWwgLnJhZGlvLC5mb3JtLWhvcml6b250YWwg
LnJhZGlvLWlubGluZXtwYWRkaW5nLXRvcDo3cHg7bWFyZ2luLXRvcDowO21hcmdpbi1ib3R0b206
MH0uZm9ybS1ob3Jpem9udGFsIC5jaGVja2JveCwuZm9ybS1ob3Jpem9udGFsIC5yYWRpb3ttaW4t
aGVpZ2h0OjI3cHh9LmZvcm0taG9yaXpvbnRhbCAuZm9ybS1ncm91cHttYXJnaW4tcmlnaHQ6LTE1
cHg7bWFyZ2luLWxlZnQ6LTE1cHh9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5mb3JtLWhvcml6
b250YWwgLmNvbnRyb2wtbGFiZWx7cGFkZGluZy10b3A6N3B4O21hcmdpbi1ib3R0b206MDt0ZXh0
LWFsaWduOnJpZ2h0fX0uZm9ybS1ob3Jpem9udGFsIC5oYXMtZmVlZGJhY2sgLmZvcm0tY29udHJv
bC1mZWVkYmFja3tyaWdodDoxNXB4fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsuZm9ybS1ob3Jp
em9udGFsIC5mb3JtLWdyb3VwLWxnIC5jb250cm9sLWxhYmVse3BhZGRpbmctdG9wOjExcHg7Zm9u
dC1zaXplOjE4cHh9fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsuZm9ybS1ob3Jpem9udGFsIC5m
b3JtLWdyb3VwLXNtIC5jb250cm9sLWxhYmVse3BhZGRpbmctdG9wOjZweDtmb250LXNpemU6MTJw
eH19LmJ0bntkaXNwbGF5OmlubGluZS1ibG9jaztwYWRkaW5nOjZweCAxMnB4O21hcmdpbi1ib3R0
b206MDtmb250LXNpemU6MTRweDtmb250LXdlaWdodDo0MDA7bGluZS1oZWlnaHQ6MS40Mjg1NzE0
Mzt0ZXh0LWFsaWduOmNlbnRlcjt3aGl0ZS1zcGFjZTpub3dyYXA7dmVydGljYWwtYWxpZ246bWlk
ZGxlOy1tcy10b3VjaC1hY3Rpb246bWFuaXB1bGF0aW9uO3RvdWNoLWFjdGlvbjptYW5pcHVsYXRp
b247Y3Vyc29yOnBvaW50ZXI7LXdlYmtpdC11c2VyLXNlbGVjdDpub25lOy1tb3otdXNlci1zZWxl
Y3Q6bm9uZTstbXMtdXNlci1zZWxlY3Q6bm9uZTt1c2VyLXNlbGVjdDpub25lO2JhY2tncm91bmQt
aW1hZ2U6bm9uZTtib3JkZXI6MXB4IHNvbGlkIHRyYW5zcGFyZW50O2JvcmRlci1yYWRpdXM6NHB4
fS5idG4uYWN0aXZlLmZvY3VzLC5idG4uYWN0aXZlOmZvY3VzLC5idG4uZm9jdXMsLmJ0bjphY3Rp
dmUuZm9jdXMsLmJ0bjphY3RpdmU6Zm9jdXMsLmJ0bjpmb2N1c3tvdXRsaW5lOjVweCBhdXRvIC13
ZWJraXQtZm9jdXMtcmluZy1jb2xvcjtvdXRsaW5lLW9mZnNldDotMnB4fS5idG4uZm9jdXMsLmJ0
bjpmb2N1cywuYnRuOmhvdmVye2NvbG9yOiMzMzM7dGV4dC1kZWNvcmF0aW9uOm5vbmV9LmJ0bi5h
Y3RpdmUsLmJ0bjphY3RpdmV7YmFja2dyb3VuZC1pbWFnZTpub25lO291dGxpbmU6MDstd2Via2l0
LWJveC1zaGFkb3c6aW5zZXQgMCAzcHggNXB4IHJnYmEoMCwwLDAsLjEyNSk7Ym94LXNoYWRvdzpp
bnNldCAwIDNweCA1cHggcmdiYSgwLDAsMCwuMTI1KX0uYnRuLmRpc2FibGVkLC5idG5bZGlzYWJs
ZWRdLGZpZWxkc2V0W2Rpc2FibGVkXSAuYnRue2N1cnNvcjpub3QtYWxsb3dlZDtmaWx0ZXI6YWxw
aGEob3BhY2l0eT02NSk7LXdlYmtpdC1ib3gtc2hhZG93Om5vbmU7Ym94LXNoYWRvdzpub25lO29w
YWNpdHk6LjY1fWEuYnRuLmRpc2FibGVkLGZpZWxkc2V0W2Rpc2FibGVkXSBhLmJ0bntwb2ludGVy
LWV2ZW50czpub25lfS5idG4tZGVmYXVsdHtjb2xvcjojMzMzO2JhY2tncm91bmQtY29sb3I6I2Zm
Zjtib3JkZXItY29sb3I6I2NjY30uYnRuLWRlZmF1bHQuZm9jdXMsLmJ0bi1kZWZhdWx0OmZvY3Vz
e2NvbG9yOiMzMzM7YmFja2dyb3VuZC1jb2xvcjojZTZlNmU2O2JvcmRlci1jb2xvcjojOGM4Yzhj
fS5idG4tZGVmYXVsdDpob3Zlcntjb2xvcjojMzMzO2JhY2tncm91bmQtY29sb3I6I2U2ZTZlNjti
b3JkZXItY29sb3I6I2FkYWRhZH0uYnRuLWRlZmF1bHQuYWN0aXZlLC5idG4tZGVmYXVsdDphY3Rp
dmUsLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4tZGVmYXVsdHtjb2xvcjojMzMzO2JhY2tncm91
bmQtY29sb3I6I2U2ZTZlNjtib3JkZXItY29sb3I6I2FkYWRhZH0uYnRuLWRlZmF1bHQuYWN0aXZl
LmZvY3VzLC5idG4tZGVmYXVsdC5hY3RpdmU6Zm9jdXMsLmJ0bi1kZWZhdWx0LmFjdGl2ZTpob3Zl
ciwuYnRuLWRlZmF1bHQ6YWN0aXZlLmZvY3VzLC5idG4tZGVmYXVsdDphY3RpdmU6Zm9jdXMsLmJ0
bi1kZWZhdWx0OmFjdGl2ZTpob3Zlciwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi1kZWZhdWx0
LmZvY3VzLC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRuLWRlZmF1bHQ6Zm9jdXMsLm9wZW4+LmRy
b3Bkb3duLXRvZ2dsZS5idG4tZGVmYXVsdDpob3Zlcntjb2xvcjojMzMzO2JhY2tncm91bmQtY29s
b3I6I2Q0ZDRkNDtib3JkZXItY29sb3I6IzhjOGM4Y30uYnRuLWRlZmF1bHQuYWN0aXZlLC5idG4t
ZGVmYXVsdDphY3RpdmUsLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4tZGVmYXVsdHtiYWNrZ3Jv
dW5kLWltYWdlOm5vbmV9LmJ0bi1kZWZhdWx0LmRpc2FibGVkLmZvY3VzLC5idG4tZGVmYXVsdC5k
aXNhYmxlZDpmb2N1cywuYnRuLWRlZmF1bHQuZGlzYWJsZWQ6aG92ZXIsLmJ0bi1kZWZhdWx0W2Rp
c2FibGVkXS5mb2N1cywuYnRuLWRlZmF1bHRbZGlzYWJsZWRdOmZvY3VzLC5idG4tZGVmYXVsdFtk
aXNhYmxlZF06aG92ZXIsZmllbGRzZXRbZGlzYWJsZWRdIC5idG4tZGVmYXVsdC5mb2N1cyxmaWVs
ZHNldFtkaXNhYmxlZF0gLmJ0bi1kZWZhdWx0OmZvY3VzLGZpZWxkc2V0W2Rpc2FibGVkXSAuYnRu
LWRlZmF1bHQ6aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojZmZmO2JvcmRlci1jb2xvcjojY2NjfS5i
dG4tZGVmYXVsdCAuYmFkZ2V7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzMzN9LmJ0bi1w
cmltYXJ5e2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojMzM3YWI3O2JvcmRlci1jb2xvcjoj
MmU2ZGE0fS5idG4tcHJpbWFyeS5mb2N1cywuYnRuLXByaW1hcnk6Zm9jdXN7Y29sb3I6I2ZmZjti
YWNrZ3JvdW5kLWNvbG9yOiMyODYwOTA7Ym9yZGVyLWNvbG9yOiMxMjJiNDB9LmJ0bi1wcmltYXJ5
OmhvdmVye2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojMjg2MDkwO2JvcmRlci1jb2xvcjoj
MjA0ZDc0fS5idG4tcHJpbWFyeS5hY3RpdmUsLmJ0bi1wcmltYXJ5OmFjdGl2ZSwub3Blbj4uZHJv
cGRvd24tdG9nZ2xlLmJ0bi1wcmltYXJ5e2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojMjg2
MDkwO2JvcmRlci1jb2xvcjojMjA0ZDc0fS5idG4tcHJpbWFyeS5hY3RpdmUuZm9jdXMsLmJ0bi1w
cmltYXJ5LmFjdGl2ZTpmb2N1cywuYnRuLXByaW1hcnkuYWN0aXZlOmhvdmVyLC5idG4tcHJpbWFy
eTphY3RpdmUuZm9jdXMsLmJ0bi1wcmltYXJ5OmFjdGl2ZTpmb2N1cywuYnRuLXByaW1hcnk6YWN0
aXZlOmhvdmVyLC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRuLXByaW1hcnkuZm9jdXMsLm9wZW4+
LmRyb3Bkb3duLXRvZ2dsZS5idG4tcHJpbWFyeTpmb2N1cywub3Blbj4uZHJvcGRvd24tdG9nZ2xl
LmJ0bi1wcmltYXJ5OmhvdmVye2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojMjA0ZDc0O2Jv
cmRlci1jb2xvcjojMTIyYjQwfS5idG4tcHJpbWFyeS5hY3RpdmUsLmJ0bi1wcmltYXJ5OmFjdGl2
ZSwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi1wcmltYXJ5e2JhY2tncm91bmQtaW1hZ2U6bm9u
ZX0uYnRuLXByaW1hcnkuZGlzYWJsZWQuZm9jdXMsLmJ0bi1wcmltYXJ5LmRpc2FibGVkOmZvY3Vz
LC5idG4tcHJpbWFyeS5kaXNhYmxlZDpob3ZlciwuYnRuLXByaW1hcnlbZGlzYWJsZWRdLmZvY3Vz
LC5idG4tcHJpbWFyeVtkaXNhYmxlZF06Zm9jdXMsLmJ0bi1wcmltYXJ5W2Rpc2FibGVkXTpob3Zl
cixmaWVsZHNldFtkaXNhYmxlZF0gLmJ0bi1wcmltYXJ5LmZvY3VzLGZpZWxkc2V0W2Rpc2FibGVk
XSAuYnRuLXByaW1hcnk6Zm9jdXMsZmllbGRzZXRbZGlzYWJsZWRdIC5idG4tcHJpbWFyeTpob3Zl
cntiYWNrZ3JvdW5kLWNvbG9yOiMzMzdhYjc7Ym9yZGVyLWNvbG9yOiMyZTZkYTR9LmJ0bi1wcmlt
YXJ5IC5iYWRnZXtjb2xvcjojMzM3YWI3O2JhY2tncm91bmQtY29sb3I6I2ZmZn0uYnRuLXN1Y2Nl
c3N7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiM1Y2I4NWM7Ym9yZGVyLWNvbG9yOiM0Y2Fl
NGN9LmJ0bi1zdWNjZXNzLmZvY3VzLC5idG4tc3VjY2Vzczpmb2N1c3tjb2xvcjojZmZmO2JhY2tn
cm91bmQtY29sb3I6IzQ0OWQ0NDtib3JkZXItY29sb3I6IzI1NTYyNX0uYnRuLXN1Y2Nlc3M6aG92
ZXJ7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiM0NDlkNDQ7Ym9yZGVyLWNvbG9yOiMzOTg0
Mzl9LmJ0bi1zdWNjZXNzLmFjdGl2ZSwuYnRuLXN1Y2Nlc3M6YWN0aXZlLC5vcGVuPi5kcm9wZG93
bi10b2dnbGUuYnRuLXN1Y2Nlc3N7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiM0NDlkNDQ7
Ym9yZGVyLWNvbG9yOiMzOTg0Mzl9LmJ0bi1zdWNjZXNzLmFjdGl2ZS5mb2N1cywuYnRuLXN1Y2Nl
c3MuYWN0aXZlOmZvY3VzLC5idG4tc3VjY2Vzcy5hY3RpdmU6aG92ZXIsLmJ0bi1zdWNjZXNzOmFj
dGl2ZS5mb2N1cywuYnRuLXN1Y2Nlc3M6YWN0aXZlOmZvY3VzLC5idG4tc3VjY2VzczphY3RpdmU6
aG92ZXIsLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4tc3VjY2Vzcy5mb2N1cywub3Blbj4uZHJv
cGRvd24tdG9nZ2xlLmJ0bi1zdWNjZXNzOmZvY3VzLC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRu
LXN1Y2Nlc3M6aG92ZXJ7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzOTg0Mzk7Ym9yZGVy
LWNvbG9yOiMyNTU2MjV9LmJ0bi1zdWNjZXNzLmFjdGl2ZSwuYnRuLXN1Y2Nlc3M6YWN0aXZlLC5v
cGVuPi5kcm9wZG93bi10b2dnbGUuYnRuLXN1Y2Nlc3N7YmFja2dyb3VuZC1pbWFnZTpub25lfS5i
dG4tc3VjY2Vzcy5kaXNhYmxlZC5mb2N1cywuYnRuLXN1Y2Nlc3MuZGlzYWJsZWQ6Zm9jdXMsLmJ0
bi1zdWNjZXNzLmRpc2FibGVkOmhvdmVyLC5idG4tc3VjY2Vzc1tkaXNhYmxlZF0uZm9jdXMsLmJ0
bi1zdWNjZXNzW2Rpc2FibGVkXTpmb2N1cywuYnRuLXN1Y2Nlc3NbZGlzYWJsZWRdOmhvdmVyLGZp
ZWxkc2V0W2Rpc2FibGVkXSAuYnRuLXN1Y2Nlc3MuZm9jdXMsZmllbGRzZXRbZGlzYWJsZWRdIC5i
dG4tc3VjY2Vzczpmb2N1cyxmaWVsZHNldFtkaXNhYmxlZF0gLmJ0bi1zdWNjZXNzOmhvdmVye2Jh
Y2tncm91bmQtY29sb3I6IzVjYjg1Yztib3JkZXItY29sb3I6IzRjYWU0Y30uYnRuLXN1Y2Nlc3Mg
LmJhZGdle2NvbG9yOiM1Y2I4NWM7YmFja2dyb3VuZC1jb2xvcjojZmZmfS5idG4taW5mb3tjb2xv
cjojZmZmO2JhY2tncm91bmQtY29sb3I6IzViYzBkZTtib3JkZXItY29sb3I6IzQ2YjhkYX0uYnRu
LWluZm8uZm9jdXMsLmJ0bi1pbmZvOmZvY3Vze2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjoj
MzFiMGQ1O2JvcmRlci1jb2xvcjojMWI2ZDg1fS5idG4taW5mbzpob3Zlcntjb2xvcjojZmZmO2Jh
Y2tncm91bmQtY29sb3I6IzMxYjBkNTtib3JkZXItY29sb3I6IzI2OWFiY30uYnRuLWluZm8uYWN0
aXZlLC5idG4taW5mbzphY3RpdmUsLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4taW5mb3tjb2xv
cjojZmZmO2JhY2tncm91bmQtY29sb3I6IzMxYjBkNTtib3JkZXItY29sb3I6IzI2OWFiY30uYnRu
LWluZm8uYWN0aXZlLmZvY3VzLC5idG4taW5mby5hY3RpdmU6Zm9jdXMsLmJ0bi1pbmZvLmFjdGl2
ZTpob3ZlciwuYnRuLWluZm86YWN0aXZlLmZvY3VzLC5idG4taW5mbzphY3RpdmU6Zm9jdXMsLmJ0
bi1pbmZvOmFjdGl2ZTpob3Zlciwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi1pbmZvLmZvY3Vz
LC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRuLWluZm86Zm9jdXMsLm9wZW4+LmRyb3Bkb3duLXRv
Z2dsZS5idG4taW5mbzpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6IzI2OWFiYzti
b3JkZXItY29sb3I6IzFiNmQ4NX0uYnRuLWluZm8uYWN0aXZlLC5idG4taW5mbzphY3RpdmUsLm9w
ZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4taW5mb3tiYWNrZ3JvdW5kLWltYWdlOm5vbmV9LmJ0bi1p
bmZvLmRpc2FibGVkLmZvY3VzLC5idG4taW5mby5kaXNhYmxlZDpmb2N1cywuYnRuLWluZm8uZGlz
YWJsZWQ6aG92ZXIsLmJ0bi1pbmZvW2Rpc2FibGVkXS5mb2N1cywuYnRuLWluZm9bZGlzYWJsZWRd
OmZvY3VzLC5idG4taW5mb1tkaXNhYmxlZF06aG92ZXIsZmllbGRzZXRbZGlzYWJsZWRdIC5idG4t
aW5mby5mb2N1cyxmaWVsZHNldFtkaXNhYmxlZF0gLmJ0bi1pbmZvOmZvY3VzLGZpZWxkc2V0W2Rp
c2FibGVkXSAuYnRuLWluZm86aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojNWJjMGRlO2JvcmRlci1j
b2xvcjojNDZiOGRhfS5idG4taW5mbyAuYmFkZ2V7Y29sb3I6IzViYzBkZTtiYWNrZ3JvdW5kLWNv
bG9yOiNmZmZ9LmJ0bi13YXJuaW5ne2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojZjBhZDRl
O2JvcmRlci1jb2xvcjojZWVhMjM2fS5idG4td2FybmluZy5mb2N1cywuYnRuLXdhcm5pbmc6Zm9j
dXN7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiNlYzk3MWY7Ym9yZGVyLWNvbG9yOiM5ODVm
MGR9LmJ0bi13YXJuaW5nOmhvdmVye2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojZWM5NzFm
O2JvcmRlci1jb2xvcjojZDU4NTEyfS5idG4td2FybmluZy5hY3RpdmUsLmJ0bi13YXJuaW5nOmFj
dGl2ZSwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi13YXJuaW5ne2NvbG9yOiNmZmY7YmFja2dy
b3VuZC1jb2xvcjojZWM5NzFmO2JvcmRlci1jb2xvcjojZDU4NTEyfS5idG4td2FybmluZy5hY3Rp
dmUuZm9jdXMsLmJ0bi13YXJuaW5nLmFjdGl2ZTpmb2N1cywuYnRuLXdhcm5pbmcuYWN0aXZlOmhv
dmVyLC5idG4td2FybmluZzphY3RpdmUuZm9jdXMsLmJ0bi13YXJuaW5nOmFjdGl2ZTpmb2N1cywu
YnRuLXdhcm5pbmc6YWN0aXZlOmhvdmVyLC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRuLXdhcm5p
bmcuZm9jdXMsLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4td2FybmluZzpmb2N1cywub3Blbj4u
ZHJvcGRvd24tdG9nZ2xlLmJ0bi13YXJuaW5nOmhvdmVye2NvbG9yOiNmZmY7YmFja2dyb3VuZC1j
b2xvcjojZDU4NTEyO2JvcmRlci1jb2xvcjojOTg1ZjBkfS5idG4td2FybmluZy5hY3RpdmUsLmJ0
bi13YXJuaW5nOmFjdGl2ZSwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi13YXJuaW5ne2JhY2tn
cm91bmQtaW1hZ2U6bm9uZX0uYnRuLXdhcm5pbmcuZGlzYWJsZWQuZm9jdXMsLmJ0bi13YXJuaW5n
LmRpc2FibGVkOmZvY3VzLC5idG4td2FybmluZy5kaXNhYmxlZDpob3ZlciwuYnRuLXdhcm5pbmdb
ZGlzYWJsZWRdLmZvY3VzLC5idG4td2FybmluZ1tkaXNhYmxlZF06Zm9jdXMsLmJ0bi13YXJuaW5n
W2Rpc2FibGVkXTpob3ZlcixmaWVsZHNldFtkaXNhYmxlZF0gLmJ0bi13YXJuaW5nLmZvY3VzLGZp
ZWxkc2V0W2Rpc2FibGVkXSAuYnRuLXdhcm5pbmc6Zm9jdXMsZmllbGRzZXRbZGlzYWJsZWRdIC5i
dG4td2FybmluZzpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNmMGFkNGU7Ym9yZGVyLWNvbG9yOiNl
ZWEyMzZ9LmJ0bi13YXJuaW5nIC5iYWRnZXtjb2xvcjojZjBhZDRlO2JhY2tncm91bmQtY29sb3I6
I2ZmZn0uYnRuLWRhbmdlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6I2Q5NTM0Zjtib3Jk
ZXItY29sb3I6I2Q0M2YzYX0uYnRuLWRhbmdlci5mb2N1cywuYnRuLWRhbmdlcjpmb2N1c3tjb2xv
cjojZmZmO2JhY2tncm91bmQtY29sb3I6I2M5MzAyYztib3JkZXItY29sb3I6Izc2MWMxOX0uYnRu
LWRhbmdlcjpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6I2M5MzAyYztib3JkZXIt
Y29sb3I6I2FjMjkyNX0uYnRuLWRhbmdlci5hY3RpdmUsLmJ0bi1kYW5nZXI6YWN0aXZlLC5vcGVu
Pi5kcm9wZG93bi10b2dnbGUuYnRuLWRhbmdlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6
I2M5MzAyYztib3JkZXItY29sb3I6I2FjMjkyNX0uYnRuLWRhbmdlci5hY3RpdmUuZm9jdXMsLmJ0
bi1kYW5nZXIuYWN0aXZlOmZvY3VzLC5idG4tZGFuZ2VyLmFjdGl2ZTpob3ZlciwuYnRuLWRhbmdl
cjphY3RpdmUuZm9jdXMsLmJ0bi1kYW5nZXI6YWN0aXZlOmZvY3VzLC5idG4tZGFuZ2VyOmFjdGl2
ZTpob3Zlciwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi1kYW5nZXIuZm9jdXMsLm9wZW4+LmRy
b3Bkb3duLXRvZ2dsZS5idG4tZGFuZ2VyOmZvY3VzLC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRu
LWRhbmdlcjpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6I2FjMjkyNTtib3JkZXIt
Y29sb3I6Izc2MWMxOX0uYnRuLWRhbmdlci5hY3RpdmUsLmJ0bi1kYW5nZXI6YWN0aXZlLC5vcGVu
Pi5kcm9wZG93bi10b2dnbGUuYnRuLWRhbmdlcntiYWNrZ3JvdW5kLWltYWdlOm5vbmV9LmJ0bi1k
YW5nZXIuZGlzYWJsZWQuZm9jdXMsLmJ0bi1kYW5nZXIuZGlzYWJsZWQ6Zm9jdXMsLmJ0bi1kYW5n
ZXIuZGlzYWJsZWQ6aG92ZXIsLmJ0bi1kYW5nZXJbZGlzYWJsZWRdLmZvY3VzLC5idG4tZGFuZ2Vy
W2Rpc2FibGVkXTpmb2N1cywuYnRuLWRhbmdlcltkaXNhYmxlZF06aG92ZXIsZmllbGRzZXRbZGlz
YWJsZWRdIC5idG4tZGFuZ2VyLmZvY3VzLGZpZWxkc2V0W2Rpc2FibGVkXSAuYnRuLWRhbmdlcjpm
b2N1cyxmaWVsZHNldFtkaXNhYmxlZF0gLmJ0bi1kYW5nZXI6aG92ZXJ7YmFja2dyb3VuZC1jb2xv
cjojZDk1MzRmO2JvcmRlci1jb2xvcjojZDQzZjNhfS5idG4tZGFuZ2VyIC5iYWRnZXtjb2xvcjoj
ZDk1MzRmO2JhY2tncm91bmQtY29sb3I6I2ZmZn0uYnRuLWxpbmt7Zm9udC13ZWlnaHQ6NDAwO2Nv
bG9yOiMzMzdhYjc7Ym9yZGVyLXJhZGl1czowfS5idG4tbGluaywuYnRuLWxpbmsuYWN0aXZlLC5i
dG4tbGluazphY3RpdmUsLmJ0bi1saW5rW2Rpc2FibGVkXSxmaWVsZHNldFtkaXNhYmxlZF0gLmJ0
bi1saW5re2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7LXdlYmtpdC1ib3gtc2hhZG93Om5v
bmU7Ym94LXNoYWRvdzpub25lfS5idG4tbGluaywuYnRuLWxpbms6YWN0aXZlLC5idG4tbGluazpm
b2N1cywuYnRuLWxpbms6aG92ZXJ7Ym9yZGVyLWNvbG9yOnRyYW5zcGFyZW50fS5idG4tbGluazpm
b2N1cywuYnRuLWxpbms6aG92ZXJ7Y29sb3I6IzIzNTI3Yzt0ZXh0LWRlY29yYXRpb246dW5kZXJs
aW5lO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnR9LmJ0bi1saW5rW2Rpc2FibGVkXTpmb2N1
cywuYnRuLWxpbmtbZGlzYWJsZWRdOmhvdmVyLGZpZWxkc2V0W2Rpc2FibGVkXSAuYnRuLWxpbms6
Zm9jdXMsZmllbGRzZXRbZGlzYWJsZWRdIC5idG4tbGluazpob3Zlcntjb2xvcjojNzc3O3RleHQt
ZGVjb3JhdGlvbjpub25lfS5idG4tZ3JvdXAtbGc+LmJ0biwuYnRuLWxne3BhZGRpbmc6MTBweCAx
NnB4O2ZvbnQtc2l6ZToxOHB4O2xpbmUtaGVpZ2h0OjEuMzMzMzMzMztib3JkZXItcmFkaXVzOjZw
eH0uYnRuLWdyb3VwLXNtPi5idG4sLmJ0bi1zbXtwYWRkaW5nOjVweCAxMHB4O2ZvbnQtc2l6ZTox
MnB4O2xpbmUtaGVpZ2h0OjEuNTtib3JkZXItcmFkaXVzOjNweH0uYnRuLWdyb3VwLXhzPi5idG4s
LmJ0bi14c3twYWRkaW5nOjFweCA1cHg7Zm9udC1zaXplOjEycHg7bGluZS1oZWlnaHQ6MS41O2Jv
cmRlci1yYWRpdXM6M3B4fS5idG4tYmxvY2t7ZGlzcGxheTpibG9jazt3aWR0aDoxMDAlfS5idG4t
YmxvY2srLmJ0bi1ibG9ja3ttYXJnaW4tdG9wOjVweH1pbnB1dFt0eXBlPWJ1dHRvbl0uYnRuLWJs
b2NrLGlucHV0W3R5cGU9cmVzZXRdLmJ0bi1ibG9jayxpbnB1dFt0eXBlPXN1Ym1pdF0uYnRuLWJs
b2Nre3dpZHRoOjEwMCV9LmZhZGV7b3BhY2l0eTowOy13ZWJraXQtdHJhbnNpdGlvbjpvcGFjaXR5
IC4xNXMgbGluZWFyOy1vLXRyYW5zaXRpb246b3BhY2l0eSAuMTVzIGxpbmVhcjt0cmFuc2l0aW9u
Om9wYWNpdHkgLjE1cyBsaW5lYXJ9LmZhZGUuaW57b3BhY2l0eToxfS5jb2xsYXBzZXtkaXNwbGF5
Om5vbmV9LmNvbGxhcHNlLmlue2Rpc3BsYXk6YmxvY2t9dHIuY29sbGFwc2UuaW57ZGlzcGxheTp0
YWJsZS1yb3d9dGJvZHkuY29sbGFwc2UuaW57ZGlzcGxheTp0YWJsZS1yb3ctZ3JvdXB9LmNvbGxh
cHNpbmd7cG9zaXRpb246cmVsYXRpdmU7aGVpZ2h0OjA7b3ZlcmZsb3c6aGlkZGVuOy13ZWJraXQt
dHJhbnNpdGlvbi10aW1pbmctZnVuY3Rpb246ZWFzZTstby10cmFuc2l0aW9uLXRpbWluZy1mdW5j
dGlvbjplYXNlO3RyYW5zaXRpb24tdGltaW5nLWZ1bmN0aW9uOmVhc2U7LXdlYmtpdC10cmFuc2l0
aW9uLWR1cmF0aW9uOi4zNXM7LW8tdHJhbnNpdGlvbi1kdXJhdGlvbjouMzVzO3RyYW5zaXRpb24t
ZHVyYXRpb246LjM1czstd2Via2l0LXRyYW5zaXRpb24tcHJvcGVydHk6aGVpZ2h0LHZpc2liaWxp
dHk7LW8tdHJhbnNpdGlvbi1wcm9wZXJ0eTpoZWlnaHQsdmlzaWJpbGl0eTt0cmFuc2l0aW9uLXBy
b3BlcnR5OmhlaWdodCx2aXNpYmlsaXR5fS5jYXJldHtkaXNwbGF5OmlubGluZS1ibG9jazt3aWR0
aDowO2hlaWdodDowO21hcmdpbi1sZWZ0OjJweDt2ZXJ0aWNhbC1hbGlnbjptaWRkbGU7Ym9yZGVy
LXRvcDo0cHggZGFzaGVkO2JvcmRlci10b3A6NHB4IHNvbGlkXDk7Ym9yZGVyLXJpZ2h0OjRweCBz
b2xpZCB0cmFuc3BhcmVudDtib3JkZXItbGVmdDo0cHggc29saWQgdHJhbnNwYXJlbnR9LmRyb3Bk
b3duLC5kcm9wdXB7cG9zaXRpb246cmVsYXRpdmV9LmRyb3Bkb3duLXRvZ2dsZTpmb2N1c3tvdXRs
aW5lOjB9LmRyb3Bkb3duLW1lbnV7cG9zaXRpb246YWJzb2x1dGU7dG9wOjEwMCU7bGVmdDowO3ot
aW5kZXg6MTAwMDtkaXNwbGF5Om5vbmU7ZmxvYXQ6bGVmdDttaW4td2lkdGg6MTYwcHg7cGFkZGlu
Zzo1cHggMDttYXJnaW46MnB4IDAgMDtmb250LXNpemU6MTRweDt0ZXh0LWFsaWduOmxlZnQ7bGlz
dC1zdHlsZTpub25lO2JhY2tncm91bmQtY29sb3I6I2ZmZjstd2Via2l0LWJhY2tncm91bmQtY2xp
cDpwYWRkaW5nLWJveDtiYWNrZ3JvdW5kLWNsaXA6cGFkZGluZy1ib3g7Ym9yZGVyOjFweCBzb2xp
ZCAjY2NjO2JvcmRlcjoxcHggc29saWQgcmdiYSgwLDAsMCwuMTUpO2JvcmRlci1yYWRpdXM6NHB4
Oy13ZWJraXQtYm94LXNoYWRvdzowIDZweCAxMnB4IHJnYmEoMCwwLDAsLjE3NSk7Ym94LXNoYWRv
dzowIDZweCAxMnB4IHJnYmEoMCwwLDAsLjE3NSl9LmRyb3Bkb3duLW1lbnUucHVsbC1yaWdodHty
aWdodDowO2xlZnQ6YXV0b30uZHJvcGRvd24tbWVudSAuZGl2aWRlcntoZWlnaHQ6MXB4O21hcmdp
bjo5cHggMDtvdmVyZmxvdzpoaWRkZW47YmFja2dyb3VuZC1jb2xvcjojZTVlNWU1fS5kcm9wZG93
bi1tZW51PmxpPmF7ZGlzcGxheTpibG9jaztwYWRkaW5nOjNweCAyMHB4O2NsZWFyOmJvdGg7Zm9u
dC13ZWlnaHQ6NDAwO2xpbmUtaGVpZ2h0OjEuNDI4NTcxNDM7Y29sb3I6IzMzMzt3aGl0ZS1zcGFj
ZTpub3dyYXB9LmRyb3Bkb3duLW1lbnU+bGk+YTpmb2N1cywuZHJvcGRvd24tbWVudT5saT5hOmhv
dmVye2NvbG9yOiMyNjI2MjY7dGV4dC1kZWNvcmF0aW9uOm5vbmU7YmFja2dyb3VuZC1jb2xvcjoj
ZjVmNWY1fS5kcm9wZG93bi1tZW51Pi5hY3RpdmU+YSwuZHJvcGRvd24tbWVudT4uYWN0aXZlPmE6
Zm9jdXMsLmRyb3Bkb3duLW1lbnU+LmFjdGl2ZT5hOmhvdmVye2NvbG9yOiNmZmY7dGV4dC1kZWNv
cmF0aW9uOm5vbmU7YmFja2dyb3VuZC1jb2xvcjojMzM3YWI3O291dGxpbmU6MH0uZHJvcGRvd24t
bWVudT4uZGlzYWJsZWQ+YSwuZHJvcGRvd24tbWVudT4uZGlzYWJsZWQ+YTpmb2N1cywuZHJvcGRv
d24tbWVudT4uZGlzYWJsZWQ+YTpob3Zlcntjb2xvcjojNzc3fS5kcm9wZG93bi1tZW51Pi5kaXNh
YmxlZD5hOmZvY3VzLC5kcm9wZG93bi1tZW51Pi5kaXNhYmxlZD5hOmhvdmVye3RleHQtZGVjb3Jh
dGlvbjpub25lO2N1cnNvcjpub3QtYWxsb3dlZDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50
O2JhY2tncm91bmQtaW1hZ2U6bm9uZTtmaWx0ZXI6cHJvZ2lkOkRYSW1hZ2VUcmFuc2Zvcm0uTWlj
cm9zb2Z0LmdyYWRpZW50KGVuYWJsZWQ9ZmFsc2UpfS5vcGVuPi5kcm9wZG93bi1tZW51e2Rpc3Bs
YXk6YmxvY2t9Lm9wZW4+YXtvdXRsaW5lOjB9LmRyb3Bkb3duLW1lbnUtcmlnaHR7cmlnaHQ6MDts
ZWZ0OmF1dG99LmRyb3Bkb3duLW1lbnUtbGVmdHtyaWdodDphdXRvO2xlZnQ6MH0uZHJvcGRvd24t
aGVhZGVye2Rpc3BsYXk6YmxvY2s7cGFkZGluZzozcHggMjBweDtmb250LXNpemU6MTJweDtsaW5l
LWhlaWdodDoxLjQyODU3MTQzO2NvbG9yOiM3Nzc7d2hpdGUtc3BhY2U6bm93cmFwfS5kcm9wZG93
bi1iYWNrZHJvcHtwb3NpdGlvbjpmaXhlZDt0b3A6MDtyaWdodDowO2JvdHRvbTowO2xlZnQ6MDt6
LWluZGV4Ojk5MH0ucHVsbC1yaWdodD4uZHJvcGRvd24tbWVudXtyaWdodDowO2xlZnQ6YXV0b30u
ZHJvcHVwIC5jYXJldCwubmF2YmFyLWZpeGVkLWJvdHRvbSAuZHJvcGRvd24gLmNhcmV0e2NvbnRl
bnQ6IiI7Ym9yZGVyLXRvcDowO2JvcmRlci1ib3R0b206NHB4IGRhc2hlZDtib3JkZXItYm90dG9t
OjRweCBzb2xpZFw5fS5kcm9wdXAgLmRyb3Bkb3duLW1lbnUsLm5hdmJhci1maXhlZC1ib3R0b20g
LmRyb3Bkb3duIC5kcm9wZG93bi1tZW51e3RvcDphdXRvO2JvdHRvbToxMDAlO21hcmdpbi1ib3R0
b206MnB4fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsubmF2YmFyLXJpZ2h0IC5kcm9wZG93bi1t
ZW51e3JpZ2h0OjA7bGVmdDphdXRvfS5uYXZiYXItcmlnaHQgLmRyb3Bkb3duLW1lbnUtbGVmdHty
aWdodDphdXRvO2xlZnQ6MH19LmJ0bi1ncm91cCwuYnRuLWdyb3VwLXZlcnRpY2Fse3Bvc2l0aW9u
OnJlbGF0aXZlO2Rpc3BsYXk6aW5saW5lLWJsb2NrO3ZlcnRpY2FsLWFsaWduOm1pZGRsZX0uYnRu
LWdyb3VwLXZlcnRpY2FsPi5idG4sLmJ0bi1ncm91cD4uYnRue3Bvc2l0aW9uOnJlbGF0aXZlO2Zs
b2F0OmxlZnR9LmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLmFjdGl2ZSwuYnRuLWdyb3VwLXZlcnRp
Y2FsPi5idG46YWN0aXZlLC5idG4tZ3JvdXAtdmVydGljYWw+LmJ0bjpmb2N1cywuYnRuLWdyb3Vw
LXZlcnRpY2FsPi5idG46aG92ZXIsLmJ0bi1ncm91cD4uYnRuLmFjdGl2ZSwuYnRuLWdyb3VwPi5i
dG46YWN0aXZlLC5idG4tZ3JvdXA+LmJ0bjpmb2N1cywuYnRuLWdyb3VwPi5idG46aG92ZXJ7ei1p
bmRleDoyfS5idG4tZ3JvdXAgLmJ0bisuYnRuLC5idG4tZ3JvdXAgLmJ0bisuYnRuLWdyb3VwLC5i
dG4tZ3JvdXAgLmJ0bi1ncm91cCsuYnRuLC5idG4tZ3JvdXAgLmJ0bi1ncm91cCsuYnRuLWdyb3Vw
e21hcmdpbi1sZWZ0Oi0xcHh9LmJ0bi10b29sYmFye21hcmdpbi1sZWZ0Oi01cHh9LmJ0bi10b29s
YmFyIC5idG4sLmJ0bi10b29sYmFyIC5idG4tZ3JvdXAsLmJ0bi10b29sYmFyIC5pbnB1dC1ncm91
cHtmbG9hdDpsZWZ0fS5idG4tdG9vbGJhcj4uYnRuLC5idG4tdG9vbGJhcj4uYnRuLWdyb3VwLC5i
dG4tdG9vbGJhcj4uaW5wdXQtZ3JvdXB7bWFyZ2luLWxlZnQ6NXB4fS5idG4tZ3JvdXA+LmJ0bjpu
b3QoOmZpcnN0LWNoaWxkKTpub3QoOmxhc3QtY2hpbGQpOm5vdCguZHJvcGRvd24tdG9nZ2xlKXti
b3JkZXItcmFkaXVzOjB9LmJ0bi1ncm91cD4uYnRuOmZpcnN0LWNoaWxke21hcmdpbi1sZWZ0OjB9
LmJ0bi1ncm91cD4uYnRuOmZpcnN0LWNoaWxkOm5vdCg6bGFzdC1jaGlsZCk6bm90KC5kcm9wZG93
bi10b2dnbGUpe2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjA7Ym9yZGVyLWJvdHRvbS1yaWdodC1y
YWRpdXM6MH0uYnRuLWdyb3VwPi5idG46bGFzdC1jaGlsZDpub3QoOmZpcnN0LWNoaWxkKSwuYnRu
LWdyb3VwPi5kcm9wZG93bi10b2dnbGU6bm90KDpmaXJzdC1jaGlsZCl7Ym9yZGVyLXRvcC1sZWZ0
LXJhZGl1czowO2JvcmRlci1ib3R0b20tbGVmdC1yYWRpdXM6MH0uYnRuLWdyb3VwPi5idG4tZ3Jv
dXB7ZmxvYXQ6bGVmdH0uYnRuLWdyb3VwPi5idG4tZ3JvdXA6bm90KDpmaXJzdC1jaGlsZCk6bm90
KDpsYXN0LWNoaWxkKT4uYnRue2JvcmRlci1yYWRpdXM6MH0uYnRuLWdyb3VwPi5idG4tZ3JvdXA6
Zmlyc3QtY2hpbGQ6bm90KDpsYXN0LWNoaWxkKT4uYnRuOmxhc3QtY2hpbGQsLmJ0bi1ncm91cD4u
YnRuLWdyb3VwOmZpcnN0LWNoaWxkOm5vdCg6bGFzdC1jaGlsZCk+LmRyb3Bkb3duLXRvZ2dsZXti
b3JkZXItdG9wLXJpZ2h0LXJhZGl1czowO2JvcmRlci1ib3R0b20tcmlnaHQtcmFkaXVzOjB9LmJ0
bi1ncm91cD4uYnRuLWdyb3VwOmxhc3QtY2hpbGQ6bm90KDpmaXJzdC1jaGlsZCk+LmJ0bjpmaXJz
dC1jaGlsZHtib3JkZXItdG9wLWxlZnQtcmFkaXVzOjA7Ym9yZGVyLWJvdHRvbS1sZWZ0LXJhZGl1
czowfS5idG4tZ3JvdXAgLmRyb3Bkb3duLXRvZ2dsZTphY3RpdmUsLmJ0bi1ncm91cC5vcGVuIC5k
cm9wZG93bi10b2dnbGV7b3V0bGluZTowfS5idG4tZ3JvdXA+LmJ0bisuZHJvcGRvd24tdG9nZ2xl
e3BhZGRpbmctcmlnaHQ6OHB4O3BhZGRpbmctbGVmdDo4cHh9LmJ0bi1ncm91cD4uYnRuLWxnKy5k
cm9wZG93bi10b2dnbGV7cGFkZGluZy1yaWdodDoxMnB4O3BhZGRpbmctbGVmdDoxMnB4fS5idG4t
Z3JvdXAub3BlbiAuZHJvcGRvd24tdG9nZ2xley13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDNw
eCA1cHggcmdiYSgwLDAsMCwuMTI1KTtib3gtc2hhZG93Omluc2V0IDAgM3B4IDVweCByZ2JhKDAs
MCwwLC4xMjUpfS5idG4tZ3JvdXAub3BlbiAuZHJvcGRvd24tdG9nZ2xlLmJ0bi1saW5rey13ZWJr
aXQtYm94LXNoYWRvdzpub25lO2JveC1zaGFkb3c6bm9uZX0uYnRuIC5jYXJldHttYXJnaW4tbGVm
dDowfS5idG4tbGcgLmNhcmV0e2JvcmRlci13aWR0aDo1cHggNXB4IDA7Ym9yZGVyLWJvdHRvbS13
aWR0aDowfS5kcm9wdXAgLmJ0bi1sZyAuY2FyZXR7Ym9yZGVyLXdpZHRoOjAgNXB4IDVweH0uYnRu
LWdyb3VwLXZlcnRpY2FsPi5idG4sLmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLWdyb3VwLC5idG4t
Z3JvdXAtdmVydGljYWw+LmJ0bi1ncm91cD4uYnRue2Rpc3BsYXk6YmxvY2s7ZmxvYXQ6bm9uZTt3
aWR0aDoxMDAlO21heC13aWR0aDoxMDAlfS5idG4tZ3JvdXAtdmVydGljYWw+LmJ0bi1ncm91cD4u
YnRue2Zsb2F0Om5vbmV9LmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuKy5idG4sLmJ0bi1ncm91cC12
ZXJ0aWNhbD4uYnRuKy5idG4tZ3JvdXAsLmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLWdyb3VwKy5i
dG4sLmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLWdyb3VwKy5idG4tZ3JvdXB7bWFyZ2luLXRvcDot
MXB4O21hcmdpbi1sZWZ0OjB9LmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuOm5vdCg6Zmlyc3QtY2hp
bGQpOm5vdCg6bGFzdC1jaGlsZCl7Ym9yZGVyLXJhZGl1czowfS5idG4tZ3JvdXAtdmVydGljYWw+
LmJ0bjpmaXJzdC1jaGlsZDpub3QoOmxhc3QtY2hpbGQpe2JvcmRlci10b3AtbGVmdC1yYWRpdXM6
NHB4O2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjRweDtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1
czowO2JvcmRlci1ib3R0b20tbGVmdC1yYWRpdXM6MH0uYnRuLWdyb3VwLXZlcnRpY2FsPi5idG46
bGFzdC1jaGlsZDpub3QoOmZpcnN0LWNoaWxkKXtib3JkZXItdG9wLWxlZnQtcmFkaXVzOjA7Ym9y
ZGVyLXRvcC1yaWdodC1yYWRpdXM6MDtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1czo0cHg7Ym9y
ZGVyLWJvdHRvbS1sZWZ0LXJhZGl1czo0cHh9LmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLWdyb3Vw
Om5vdCg6Zmlyc3QtY2hpbGQpOm5vdCg6bGFzdC1jaGlsZCk+LmJ0bntib3JkZXItcmFkaXVzOjB9
LmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLWdyb3VwOmZpcnN0LWNoaWxkOm5vdCg6bGFzdC1jaGls
ZCk+LmJ0bjpsYXN0LWNoaWxkLC5idG4tZ3JvdXAtdmVydGljYWw+LmJ0bi1ncm91cDpmaXJzdC1j
aGlsZDpub3QoOmxhc3QtY2hpbGQpPi5kcm9wZG93bi10b2dnbGV7Ym9yZGVyLWJvdHRvbS1yaWdo
dC1yYWRpdXM6MDtib3JkZXItYm90dG9tLWxlZnQtcmFkaXVzOjB9LmJ0bi1ncm91cC12ZXJ0aWNh
bD4uYnRuLWdyb3VwOmxhc3QtY2hpbGQ6bm90KDpmaXJzdC1jaGlsZCk+LmJ0bjpmaXJzdC1jaGls
ZHtib3JkZXItdG9wLWxlZnQtcmFkaXVzOjA7Ym9yZGVyLXRvcC1yaWdodC1yYWRpdXM6MH0uYnRu
LWdyb3VwLWp1c3RpZmllZHtkaXNwbGF5OnRhYmxlO3dpZHRoOjEwMCU7dGFibGUtbGF5b3V0OmZp
eGVkO2JvcmRlci1jb2xsYXBzZTpzZXBhcmF0ZX0uYnRuLWdyb3VwLWp1c3RpZmllZD4uYnRuLC5i
dG4tZ3JvdXAtanVzdGlmaWVkPi5idG4tZ3JvdXB7ZGlzcGxheTp0YWJsZS1jZWxsO2Zsb2F0Om5v
bmU7d2lkdGg6MSV9LmJ0bi1ncm91cC1qdXN0aWZpZWQ+LmJ0bi1ncm91cCAuYnRue3dpZHRoOjEw
MCV9LmJ0bi1ncm91cC1qdXN0aWZpZWQ+LmJ0bi1ncm91cCAuZHJvcGRvd24tbWVudXtsZWZ0OmF1
dG99W2RhdGEtdG9nZ2xlPWJ1dHRvbnNdPi5idG4gaW5wdXRbdHlwZT1jaGVja2JveF0sW2RhdGEt
dG9nZ2xlPWJ1dHRvbnNdPi5idG4gaW5wdXRbdHlwZT1yYWRpb10sW2RhdGEtdG9nZ2xlPWJ1dHRv
bnNdPi5idG4tZ3JvdXA+LmJ0biBpbnB1dFt0eXBlPWNoZWNrYm94XSxbZGF0YS10b2dnbGU9YnV0
dG9uc10+LmJ0bi1ncm91cD4uYnRuIGlucHV0W3R5cGU9cmFkaW9de3Bvc2l0aW9uOmFic29sdXRl
O2NsaXA6cmVjdCgwLDAsMCwwKTtwb2ludGVyLWV2ZW50czpub25lfS5pbnB1dC1ncm91cHtwb3Np
dGlvbjpyZWxhdGl2ZTtkaXNwbGF5OnRhYmxlO2JvcmRlci1jb2xsYXBzZTpzZXBhcmF0ZX0uaW5w
dXQtZ3JvdXBbY2xhc3MqPWNvbC1de2Zsb2F0Om5vbmU7cGFkZGluZy1yaWdodDowO3BhZGRpbmct
bGVmdDowfS5pbnB1dC1ncm91cCAuZm9ybS1jb250cm9se3Bvc2l0aW9uOnJlbGF0aXZlO3otaW5k
ZXg6MjtmbG9hdDpsZWZ0O3dpZHRoOjEwMCU7bWFyZ2luLWJvdHRvbTowfS5pbnB1dC1ncm91cCAu
Zm9ybS1jb250cm9sOmZvY3Vze3otaW5kZXg6M30uaW5wdXQtZ3JvdXAtbGc+LmZvcm0tY29udHJv
bCwuaW5wdXQtZ3JvdXAtbGc+LmlucHV0LWdyb3VwLWFkZG9uLC5pbnB1dC1ncm91cC1sZz4uaW5w
dXQtZ3JvdXAtYnRuPi5idG57aGVpZ2h0OjQ2cHg7cGFkZGluZzoxMHB4IDE2cHg7Zm9udC1zaXpl
OjE4cHg7bGluZS1oZWlnaHQ6MS4zMzMzMzMzO2JvcmRlci1yYWRpdXM6NnB4fXNlbGVjdC5pbnB1
dC1ncm91cC1sZz4uZm9ybS1jb250cm9sLHNlbGVjdC5pbnB1dC1ncm91cC1sZz4uaW5wdXQtZ3Jv
dXAtYWRkb24sc2VsZWN0LmlucHV0LWdyb3VwLWxnPi5pbnB1dC1ncm91cC1idG4+LmJ0bntoZWln
aHQ6NDZweDtsaW5lLWhlaWdodDo0NnB4fXNlbGVjdFttdWx0aXBsZV0uaW5wdXQtZ3JvdXAtbGc+
LmZvcm0tY29udHJvbCxzZWxlY3RbbXVsdGlwbGVdLmlucHV0LWdyb3VwLWxnPi5pbnB1dC1ncm91
cC1hZGRvbixzZWxlY3RbbXVsdGlwbGVdLmlucHV0LWdyb3VwLWxnPi5pbnB1dC1ncm91cC1idG4+
LmJ0bix0ZXh0YXJlYS5pbnB1dC1ncm91cC1sZz4uZm9ybS1jb250cm9sLHRleHRhcmVhLmlucHV0
LWdyb3VwLWxnPi5pbnB1dC1ncm91cC1hZGRvbix0ZXh0YXJlYS5pbnB1dC1ncm91cC1sZz4uaW5w
dXQtZ3JvdXAtYnRuPi5idG57aGVpZ2h0OmF1dG99LmlucHV0LWdyb3VwLXNtPi5mb3JtLWNvbnRy
b2wsLmlucHV0LWdyb3VwLXNtPi5pbnB1dC1ncm91cC1hZGRvbiwuaW5wdXQtZ3JvdXAtc20+Lmlu
cHV0LWdyb3VwLWJ0bj4uYnRue2hlaWdodDozMHB4O3BhZGRpbmc6NXB4IDEwcHg7Zm9udC1zaXpl
OjEycHg7bGluZS1oZWlnaHQ6MS41O2JvcmRlci1yYWRpdXM6M3B4fXNlbGVjdC5pbnB1dC1ncm91
cC1zbT4uZm9ybS1jb250cm9sLHNlbGVjdC5pbnB1dC1ncm91cC1zbT4uaW5wdXQtZ3JvdXAtYWRk
b24sc2VsZWN0LmlucHV0LWdyb3VwLXNtPi5pbnB1dC1ncm91cC1idG4+LmJ0bntoZWlnaHQ6MzBw
eDtsaW5lLWhlaWdodDozMHB4fXNlbGVjdFttdWx0aXBsZV0uaW5wdXQtZ3JvdXAtc20+LmZvcm0t
Y29udHJvbCxzZWxlY3RbbXVsdGlwbGVdLmlucHV0LWdyb3VwLXNtPi5pbnB1dC1ncm91cC1hZGRv
bixzZWxlY3RbbXVsdGlwbGVdLmlucHV0LWdyb3VwLXNtPi5pbnB1dC1ncm91cC1idG4+LmJ0bix0
ZXh0YXJlYS5pbnB1dC1ncm91cC1zbT4uZm9ybS1jb250cm9sLHRleHRhcmVhLmlucHV0LWdyb3Vw
LXNtPi5pbnB1dC1ncm91cC1hZGRvbix0ZXh0YXJlYS5pbnB1dC1ncm91cC1zbT4uaW5wdXQtZ3Jv
dXAtYnRuPi5idG57aGVpZ2h0OmF1dG99LmlucHV0LWdyb3VwIC5mb3JtLWNvbnRyb2wsLmlucHV0
LWdyb3VwLWFkZG9uLC5pbnB1dC1ncm91cC1idG57ZGlzcGxheTp0YWJsZS1jZWxsfS5pbnB1dC1n
cm91cCAuZm9ybS1jb250cm9sOm5vdCg6Zmlyc3QtY2hpbGQpOm5vdCg6bGFzdC1jaGlsZCksLmlu
cHV0LWdyb3VwLWFkZG9uOm5vdCg6Zmlyc3QtY2hpbGQpOm5vdCg6bGFzdC1jaGlsZCksLmlucHV0
LWdyb3VwLWJ0bjpub3QoOmZpcnN0LWNoaWxkKTpub3QoOmxhc3QtY2hpbGQpe2JvcmRlci1yYWRp
dXM6MH0uaW5wdXQtZ3JvdXAtYWRkb24sLmlucHV0LWdyb3VwLWJ0bnt3aWR0aDoxJTt3aGl0ZS1z
cGFjZTpub3dyYXA7dmVydGljYWwtYWxpZ246bWlkZGxlfS5pbnB1dC1ncm91cC1hZGRvbntwYWRk
aW5nOjZweCAxMnB4O2ZvbnQtc2l6ZToxNHB4O2ZvbnQtd2VpZ2h0OjQwMDtsaW5lLWhlaWdodDox
O2NvbG9yOiM1NTU7dGV4dC1hbGlnbjpjZW50ZXI7YmFja2dyb3VuZC1jb2xvcjojZWVlO2JvcmRl
cjoxcHggc29saWQgI2NjYztib3JkZXItcmFkaXVzOjRweH0uaW5wdXQtZ3JvdXAtYWRkb24uaW5w
dXQtc217cGFkZGluZzo1cHggMTBweDtmb250LXNpemU6MTJweDtib3JkZXItcmFkaXVzOjNweH0u
aW5wdXQtZ3JvdXAtYWRkb24uaW5wdXQtbGd7cGFkZGluZzoxMHB4IDE2cHg7Zm9udC1zaXplOjE4
cHg7Ym9yZGVyLXJhZGl1czo2cHh9LmlucHV0LWdyb3VwLWFkZG9uIGlucHV0W3R5cGU9Y2hlY2ti
b3hdLC5pbnB1dC1ncm91cC1hZGRvbiBpbnB1dFt0eXBlPXJhZGlvXXttYXJnaW4tdG9wOjB9Lmlu
cHV0LWdyb3VwIC5mb3JtLWNvbnRyb2w6Zmlyc3QtY2hpbGQsLmlucHV0LWdyb3VwLWFkZG9uOmZp
cnN0LWNoaWxkLC5pbnB1dC1ncm91cC1idG46Zmlyc3QtY2hpbGQ+LmJ0biwuaW5wdXQtZ3JvdXAt
YnRuOmZpcnN0LWNoaWxkPi5idG4tZ3JvdXA+LmJ0biwuaW5wdXQtZ3JvdXAtYnRuOmZpcnN0LWNo
aWxkPi5kcm9wZG93bi10b2dnbGUsLmlucHV0LWdyb3VwLWJ0bjpsYXN0LWNoaWxkPi5idG4tZ3Jv
dXA6bm90KDpsYXN0LWNoaWxkKT4uYnRuLC5pbnB1dC1ncm91cC1idG46bGFzdC1jaGlsZD4uYnRu
Om5vdCg6bGFzdC1jaGlsZCk6bm90KC5kcm9wZG93bi10b2dnbGUpe2JvcmRlci10b3AtcmlnaHQt
cmFkaXVzOjA7Ym9yZGVyLWJvdHRvbS1yaWdodC1yYWRpdXM6MH0uaW5wdXQtZ3JvdXAtYWRkb246
Zmlyc3QtY2hpbGR7Ym9yZGVyLXJpZ2h0OjB9LmlucHV0LWdyb3VwIC5mb3JtLWNvbnRyb2w6bGFz
dC1jaGlsZCwuaW5wdXQtZ3JvdXAtYWRkb246bGFzdC1jaGlsZCwuaW5wdXQtZ3JvdXAtYnRuOmZp
cnN0LWNoaWxkPi5idG4tZ3JvdXA6bm90KDpmaXJzdC1jaGlsZCk+LmJ0biwuaW5wdXQtZ3JvdXAt
YnRuOmZpcnN0LWNoaWxkPi5idG46bm90KDpmaXJzdC1jaGlsZCksLmlucHV0LWdyb3VwLWJ0bjps
YXN0LWNoaWxkPi5idG4sLmlucHV0LWdyb3VwLWJ0bjpsYXN0LWNoaWxkPi5idG4tZ3JvdXA+LmJ0
biwuaW5wdXQtZ3JvdXAtYnRuOmxhc3QtY2hpbGQ+LmRyb3Bkb3duLXRvZ2dsZXtib3JkZXItdG9w
LWxlZnQtcmFkaXVzOjA7Ym9yZGVyLWJvdHRvbS1sZWZ0LXJhZGl1czowfS5pbnB1dC1ncm91cC1h
ZGRvbjpsYXN0LWNoaWxke2JvcmRlci1sZWZ0OjB9LmlucHV0LWdyb3VwLWJ0bntwb3NpdGlvbjpy
ZWxhdGl2ZTtmb250LXNpemU6MDt3aGl0ZS1zcGFjZTpub3dyYXB9LmlucHV0LWdyb3VwLWJ0bj4u
YnRue3Bvc2l0aW9uOnJlbGF0aXZlfS5pbnB1dC1ncm91cC1idG4+LmJ0bisuYnRue21hcmdpbi1s
ZWZ0Oi0xcHh9LmlucHV0LWdyb3VwLWJ0bj4uYnRuOmFjdGl2ZSwuaW5wdXQtZ3JvdXAtYnRuPi5i
dG46Zm9jdXMsLmlucHV0LWdyb3VwLWJ0bj4uYnRuOmhvdmVye3otaW5kZXg6Mn0uaW5wdXQtZ3Jv
dXAtYnRuOmZpcnN0LWNoaWxkPi5idG4sLmlucHV0LWdyb3VwLWJ0bjpmaXJzdC1jaGlsZD4uYnRu
LWdyb3Vwe21hcmdpbi1yaWdodDotMXB4fS5pbnB1dC1ncm91cC1idG46bGFzdC1jaGlsZD4uYnRu
LC5pbnB1dC1ncm91cC1idG46bGFzdC1jaGlsZD4uYnRuLWdyb3Vwe3otaW5kZXg6MjttYXJnaW4t
bGVmdDotMXB4fS5uYXZ7cGFkZGluZy1sZWZ0OjA7bWFyZ2luLWJvdHRvbTowO2xpc3Qtc3R5bGU6
bm9uZX0ubmF2Pmxpe3Bvc2l0aW9uOnJlbGF0aXZlO2Rpc3BsYXk6YmxvY2t9Lm5hdj5saT5he3Bv
c2l0aW9uOnJlbGF0aXZlO2Rpc3BsYXk6YmxvY2s7cGFkZGluZzoxMHB4IDE1cHh9Lm5hdj5saT5h
OmZvY3VzLC5uYXY+bGk+YTpob3Zlcnt0ZXh0LWRlY29yYXRpb246bm9uZTtiYWNrZ3JvdW5kLWNv
bG9yOiNlZWV9Lm5hdj5saS5kaXNhYmxlZD5he2NvbG9yOiM3Nzd9Lm5hdj5saS5kaXNhYmxlZD5h
OmZvY3VzLC5uYXY+bGkuZGlzYWJsZWQ+YTpob3Zlcntjb2xvcjojNzc3O3RleHQtZGVjb3JhdGlv
bjpub25lO2N1cnNvcjpub3QtYWxsb3dlZDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50fS5u
YXYgLm9wZW4+YSwubmF2IC5vcGVuPmE6Zm9jdXMsLm5hdiAub3Blbj5hOmhvdmVye2JhY2tncm91
bmQtY29sb3I6I2VlZTtib3JkZXItY29sb3I6IzMzN2FiN30ubmF2IC5uYXYtZGl2aWRlcntoZWln
aHQ6MXB4O21hcmdpbjo5cHggMDtvdmVyZmxvdzpoaWRkZW47YmFja2dyb3VuZC1jb2xvcjojZTVl
NWU1fS5uYXY+bGk+YT5pbWd7bWF4LXdpZHRoOm5vbmV9Lm5hdi10YWJze2JvcmRlci1ib3R0b206
MXB4IHNvbGlkICNkZGR9Lm5hdi10YWJzPmxpe2Zsb2F0OmxlZnQ7bWFyZ2luLWJvdHRvbTotMXB4
fS5uYXYtdGFicz5saT5he21hcmdpbi1yaWdodDoycHg7bGluZS1oZWlnaHQ6MS40Mjg1NzE0Mzti
b3JkZXI6MXB4IHNvbGlkIHRyYW5zcGFyZW50O2JvcmRlci1yYWRpdXM6NHB4IDRweCAwIDB9Lm5h
di10YWJzPmxpPmE6aG92ZXJ7Ym9yZGVyLWNvbG9yOiNlZWUgI2VlZSAjZGRkfS5uYXYtdGFicz5s
aS5hY3RpdmU+YSwubmF2LXRhYnM+bGkuYWN0aXZlPmE6Zm9jdXMsLm5hdi10YWJzPmxpLmFjdGl2
ZT5hOmhvdmVye2NvbG9yOiM1NTU7Y3Vyc29yOmRlZmF1bHQ7YmFja2dyb3VuZC1jb2xvcjojZmZm
O2JvcmRlcjoxcHggc29saWQgI2RkZDtib3JkZXItYm90dG9tLWNvbG9yOnRyYW5zcGFyZW50fS5u
YXYtdGFicy5uYXYtanVzdGlmaWVke3dpZHRoOjEwMCU7Ym9yZGVyLWJvdHRvbTowfS5uYXYtdGFi
cy5uYXYtanVzdGlmaWVkPmxpe2Zsb2F0Om5vbmV9Lm5hdi10YWJzLm5hdi1qdXN0aWZpZWQ+bGk+
YXttYXJnaW4tYm90dG9tOjVweDt0ZXh0LWFsaWduOmNlbnRlcn0ubmF2LXRhYnMubmF2LWp1c3Rp
ZmllZD4uZHJvcGRvd24gLmRyb3Bkb3duLW1lbnV7dG9wOmF1dG87bGVmdDphdXRvfUBtZWRpYSAo
bWluLXdpZHRoOjc2OHB4KXsubmF2LXRhYnMubmF2LWp1c3RpZmllZD5saXtkaXNwbGF5OnRhYmxl
LWNlbGw7d2lkdGg6MSV9Lm5hdi10YWJzLm5hdi1qdXN0aWZpZWQ+bGk+YXttYXJnaW4tYm90dG9t
OjB9fS5uYXYtdGFicy5uYXYtanVzdGlmaWVkPmxpPmF7bWFyZ2luLXJpZ2h0OjA7Ym9yZGVyLXJh
ZGl1czo0cHh9Lm5hdi10YWJzLm5hdi1qdXN0aWZpZWQ+LmFjdGl2ZT5hLC5uYXYtdGFicy5uYXYt
anVzdGlmaWVkPi5hY3RpdmU+YTpmb2N1cywubmF2LXRhYnMubmF2LWp1c3RpZmllZD4uYWN0aXZl
PmE6aG92ZXJ7Ym9yZGVyOjFweCBzb2xpZCAjZGRkfUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsu
bmF2LXRhYnMubmF2LWp1c3RpZmllZD5saT5he2JvcmRlci1ib3R0b206MXB4IHNvbGlkICNkZGQ7
Ym9yZGVyLXJhZGl1czo0cHggNHB4IDAgMH0ubmF2LXRhYnMubmF2LWp1c3RpZmllZD4uYWN0aXZl
PmEsLm5hdi10YWJzLm5hdi1qdXN0aWZpZWQ+LmFjdGl2ZT5hOmZvY3VzLC5uYXYtdGFicy5uYXYt
anVzdGlmaWVkPi5hY3RpdmU+YTpob3Zlcntib3JkZXItYm90dG9tLWNvbG9yOiNmZmZ9fS5uYXYt
cGlsbHM+bGl7ZmxvYXQ6bGVmdH0ubmF2LXBpbGxzPmxpPmF7Ym9yZGVyLXJhZGl1czo0cHh9Lm5h
di1waWxscz5saStsaXttYXJnaW4tbGVmdDoycHh9Lm5hdi1waWxscz5saS5hY3RpdmU+YSwubmF2
LXBpbGxzPmxpLmFjdGl2ZT5hOmZvY3VzLC5uYXYtcGlsbHM+bGkuYWN0aXZlPmE6aG92ZXJ7Y29s
b3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzMzdhYjd9Lm5hdi1zdGFja2VkPmxpe2Zsb2F0Om5v
bmV9Lm5hdi1zdGFja2VkPmxpK2xpe21hcmdpbi10b3A6MnB4O21hcmdpbi1sZWZ0OjB9Lm5hdi1q
dXN0aWZpZWR7d2lkdGg6MTAwJX0ubmF2LWp1c3RpZmllZD5saXtmbG9hdDpub25lfS5uYXYtanVz
dGlmaWVkPmxpPmF7bWFyZ2luLWJvdHRvbTo1cHg7dGV4dC1hbGlnbjpjZW50ZXJ9Lm5hdi1qdXN0
aWZpZWQ+LmRyb3Bkb3duIC5kcm9wZG93bi1tZW51e3RvcDphdXRvO2xlZnQ6YXV0b31AbWVkaWEg
KG1pbi13aWR0aDo3NjhweCl7Lm5hdi1qdXN0aWZpZWQ+bGl7ZGlzcGxheTp0YWJsZS1jZWxsO3dp
ZHRoOjElfS5uYXYtanVzdGlmaWVkPmxpPmF7bWFyZ2luLWJvdHRvbTowfX0ubmF2LXRhYnMtanVz
dGlmaWVke2JvcmRlci1ib3R0b206MH0ubmF2LXRhYnMtanVzdGlmaWVkPmxpPmF7bWFyZ2luLXJp
Z2h0OjA7Ym9yZGVyLXJhZGl1czo0cHh9Lm5hdi10YWJzLWp1c3RpZmllZD4uYWN0aXZlPmEsLm5h
di10YWJzLWp1c3RpZmllZD4uYWN0aXZlPmE6Zm9jdXMsLm5hdi10YWJzLWp1c3RpZmllZD4uYWN0
aXZlPmE6aG92ZXJ7Ym9yZGVyOjFweCBzb2xpZCAjZGRkfUBtZWRpYSAobWluLXdpZHRoOjc2OHB4
KXsubmF2LXRhYnMtanVzdGlmaWVkPmxpPmF7Ym9yZGVyLWJvdHRvbToxcHggc29saWQgI2RkZDti
b3JkZXItcmFkaXVzOjRweCA0cHggMCAwfS5uYXYtdGFicy1qdXN0aWZpZWQ+LmFjdGl2ZT5hLC5u
YXYtdGFicy1qdXN0aWZpZWQ+LmFjdGl2ZT5hOmZvY3VzLC5uYXYtdGFicy1qdXN0aWZpZWQ+LmFj
dGl2ZT5hOmhvdmVye2JvcmRlci1ib3R0b20tY29sb3I6I2ZmZn19LnRhYi1jb250ZW50Pi50YWIt
cGFuZXtkaXNwbGF5Om5vbmV9LnRhYi1jb250ZW50Pi5hY3RpdmV7ZGlzcGxheTpibG9ja30ubmF2
LXRhYnMgLmRyb3Bkb3duLW1lbnV7bWFyZ2luLXRvcDotMXB4O2JvcmRlci10b3AtbGVmdC1yYWRp
dXM6MDtib3JkZXItdG9wLXJpZ2h0LXJhZGl1czowfS5uYXZiYXJ7cG9zaXRpb246cmVsYXRpdmU7
bWluLWhlaWdodDo1MHB4O21hcmdpbi1ib3R0b206MjBweDtib3JkZXI6MXB4IHNvbGlkIHRyYW5z
cGFyZW50fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsubmF2YmFye2JvcmRlci1yYWRpdXM6NHB4
fX1AbWVkaWEgKG1pbi13aWR0aDo3NjhweCl7Lm5hdmJhci1oZWFkZXJ7ZmxvYXQ6bGVmdH19Lm5h
dmJhci1jb2xsYXBzZXtwYWRkaW5nLXJpZ2h0OjE1cHg7cGFkZGluZy1sZWZ0OjE1cHg7b3ZlcmZs
b3cteDp2aXNpYmxlOy13ZWJraXQtb3ZlcmZsb3ctc2Nyb2xsaW5nOnRvdWNoO2JvcmRlci10b3A6
MXB4IHNvbGlkIHRyYW5zcGFyZW50Oy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFweCAwIHJn
YmEoMjU1LDI1NSwyNTUsLjEpO2JveC1zaGFkb3c6aW5zZXQgMCAxcHggMCByZ2JhKDI1NSwyNTUs
MjU1LC4xKX0ubmF2YmFyLWNvbGxhcHNlLmlue292ZXJmbG93LXk6YXV0b31AbWVkaWEgKG1pbi13
aWR0aDo3NjhweCl7Lm5hdmJhci1jb2xsYXBzZXt3aWR0aDphdXRvO2JvcmRlci10b3A6MDstd2Vi
a2l0LWJveC1zaGFkb3c6bm9uZTtib3gtc2hhZG93Om5vbmV9Lm5hdmJhci1jb2xsYXBzZS5jb2xs
YXBzZXtkaXNwbGF5OmJsb2NrIWltcG9ydGFudDtoZWlnaHQ6YXV0byFpbXBvcnRhbnQ7cGFkZGlu
Zy1ib3R0b206MDtvdmVyZmxvdzp2aXNpYmxlIWltcG9ydGFudH0ubmF2YmFyLWNvbGxhcHNlLmlu
e292ZXJmbG93LXk6dmlzaWJsZX0ubmF2YmFyLWZpeGVkLWJvdHRvbSAubmF2YmFyLWNvbGxhcHNl
LC5uYXZiYXItZml4ZWQtdG9wIC5uYXZiYXItY29sbGFwc2UsLm5hdmJhci1zdGF0aWMtdG9wIC5u
YXZiYXItY29sbGFwc2V7cGFkZGluZy1yaWdodDowO3BhZGRpbmctbGVmdDowfX0ubmF2YmFyLWZp
eGVkLWJvdHRvbSAubmF2YmFyLWNvbGxhcHNlLC5uYXZiYXItZml4ZWQtdG9wIC5uYXZiYXItY29s
bGFwc2V7bWF4LWhlaWdodDozNDBweH1AbWVkaWEgKG1heC1kZXZpY2Utd2lkdGg6NDgwcHgpIGFu
ZCAob3JpZW50YXRpb246bGFuZHNjYXBlKXsubmF2YmFyLWZpeGVkLWJvdHRvbSAubmF2YmFyLWNv
bGxhcHNlLC5uYXZiYXItZml4ZWQtdG9wIC5uYXZiYXItY29sbGFwc2V7bWF4LWhlaWdodDoyMDBw
eH19LmNvbnRhaW5lci1mbHVpZD4ubmF2YmFyLWNvbGxhcHNlLC5jb250YWluZXItZmx1aWQ+Lm5h
dmJhci1oZWFkZXIsLmNvbnRhaW5lcj4ubmF2YmFyLWNvbGxhcHNlLC5jb250YWluZXI+Lm5hdmJh
ci1oZWFkZXJ7bWFyZ2luLXJpZ2h0Oi0xNXB4O21hcmdpbi1sZWZ0Oi0xNXB4fUBtZWRpYSAobWlu
LXdpZHRoOjc2OHB4KXsuY29udGFpbmVyLWZsdWlkPi5uYXZiYXItY29sbGFwc2UsLmNvbnRhaW5l
ci1mbHVpZD4ubmF2YmFyLWhlYWRlciwuY29udGFpbmVyPi5uYXZiYXItY29sbGFwc2UsLmNvbnRh
aW5lcj4ubmF2YmFyLWhlYWRlcnttYXJnaW4tcmlnaHQ6MDttYXJnaW4tbGVmdDowfX0ubmF2YmFy
LXN0YXRpYy10b3B7ei1pbmRleDoxMDAwO2JvcmRlci13aWR0aDowIDAgMXB4fUBtZWRpYSAobWlu
LXdpZHRoOjc2OHB4KXsubmF2YmFyLXN0YXRpYy10b3B7Ym9yZGVyLXJhZGl1czowfX0ubmF2YmFy
LWZpeGVkLWJvdHRvbSwubmF2YmFyLWZpeGVkLXRvcHtwb3NpdGlvbjpmaXhlZDtyaWdodDowO2xl
ZnQ6MDt6LWluZGV4OjEwMzB9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5uYXZiYXItZml4ZWQt
Ym90dG9tLC5uYXZiYXItZml4ZWQtdG9we2JvcmRlci1yYWRpdXM6MH19Lm5hdmJhci1maXhlZC10
b3B7dG9wOjA7Ym9yZGVyLXdpZHRoOjAgMCAxcHh9Lm5hdmJhci1maXhlZC1ib3R0b217Ym90dG9t
OjA7bWFyZ2luLWJvdHRvbTowO2JvcmRlci13aWR0aDoxcHggMCAwfS5uYXZiYXItYnJhbmR7Zmxv
YXQ6bGVmdDtoZWlnaHQ6NTBweDtwYWRkaW5nOjE1cHggMTVweDtmb250LXNpemU6MThweDtsaW5l
LWhlaWdodDoyMHB4fS5uYXZiYXItYnJhbmQ6Zm9jdXMsLm5hdmJhci1icmFuZDpob3Zlcnt0ZXh0
LWRlY29yYXRpb246bm9uZX0ubmF2YmFyLWJyYW5kPmltZ3tkaXNwbGF5OmJsb2NrfUBtZWRpYSAo
bWluLXdpZHRoOjc2OHB4KXsubmF2YmFyPi5jb250YWluZXIgLm5hdmJhci1icmFuZCwubmF2YmFy
Pi5jb250YWluZXItZmx1aWQgLm5hdmJhci1icmFuZHttYXJnaW4tbGVmdDotMTVweH19Lm5hdmJh
ci10b2dnbGV7cG9zaXRpb246cmVsYXRpdmU7ZmxvYXQ6cmlnaHQ7cGFkZGluZzo5cHggMTBweDtt
YXJnaW4tdG9wOjhweDttYXJnaW4tcmlnaHQ6MTVweDttYXJnaW4tYm90dG9tOjhweDtiYWNrZ3Jv
dW5kLWNvbG9yOnRyYW5zcGFyZW50O2JhY2tncm91bmQtaW1hZ2U6bm9uZTtib3JkZXI6MXB4IHNv
bGlkIHRyYW5zcGFyZW50O2JvcmRlci1yYWRpdXM6NHB4fS5uYXZiYXItdG9nZ2xlOmZvY3Vze291
dGxpbmU6MH0ubmF2YmFyLXRvZ2dsZSAuaWNvbi1iYXJ7ZGlzcGxheTpibG9jazt3aWR0aDoyMnB4
O2hlaWdodDoycHg7Ym9yZGVyLXJhZGl1czoxcHh9Lm5hdmJhci10b2dnbGUgLmljb24tYmFyKy5p
Y29uLWJhcnttYXJnaW4tdG9wOjRweH1AbWVkaWEgKG1pbi13aWR0aDo3NjhweCl7Lm5hdmJhci10
b2dnbGV7ZGlzcGxheTpub25lfX0ubmF2YmFyLW5hdnttYXJnaW46Ny41cHggLTE1cHh9Lm5hdmJh
ci1uYXY+bGk+YXtwYWRkaW5nLXRvcDoxMHB4O3BhZGRpbmctYm90dG9tOjEwcHg7bGluZS1oZWln
aHQ6MjBweH1AbWVkaWEgKG1heC13aWR0aDo3NjdweCl7Lm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bk
b3duLW1lbnV7cG9zaXRpb246c3RhdGljO2Zsb2F0Om5vbmU7d2lkdGg6YXV0bzttYXJnaW4tdG9w
OjA7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtib3JkZXI6MDstd2Via2l0LWJveC1zaGFk
b3c6bm9uZTtib3gtc2hhZG93Om5vbmV9Lm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bkb3duLW1lbnUg
LmRyb3Bkb3duLWhlYWRlciwubmF2YmFyLW5hdiAub3BlbiAuZHJvcGRvd24tbWVudT5saT5he3Bh
ZGRpbmc6NXB4IDE1cHggNXB4IDI1cHh9Lm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bkb3duLW1lbnU+
bGk+YXtsaW5lLWhlaWdodDoyMHB4fS5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1tZW51Pmxp
PmE6Zm9jdXMsLm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bkb3duLW1lbnU+bGk+YTpob3ZlcntiYWNr
Z3JvdW5kLWltYWdlOm5vbmV9fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsubmF2YmFyLW5hdntm
bG9hdDpsZWZ0O21hcmdpbjowfS5uYXZiYXItbmF2Pmxpe2Zsb2F0OmxlZnR9Lm5hdmJhci1uYXY+
bGk+YXtwYWRkaW5nLXRvcDoxNXB4O3BhZGRpbmctYm90dG9tOjE1cHh9fS5uYXZiYXItZm9ybXtw
YWRkaW5nOjEwcHggMTVweDttYXJnaW4tdG9wOjhweDttYXJnaW4tcmlnaHQ6LTE1cHg7bWFyZ2lu
LWJvdHRvbTo4cHg7bWFyZ2luLWxlZnQ6LTE1cHg7Ym9yZGVyLXRvcDoxcHggc29saWQgdHJhbnNw
YXJlbnQ7Ym9yZGVyLWJvdHRvbToxcHggc29saWQgdHJhbnNwYXJlbnQ7LXdlYmtpdC1ib3gtc2hh
ZG93Omluc2V0IDAgMXB4IDAgcmdiYSgyNTUsMjU1LDI1NSwuMSksMCAxcHggMCByZ2JhKDI1NSwy
NTUsMjU1LC4xKTtib3gtc2hhZG93Omluc2V0IDAgMXB4IDAgcmdiYSgyNTUsMjU1LDI1NSwuMSks
MCAxcHggMCByZ2JhKDI1NSwyNTUsMjU1LC4xKX1AbWVkaWEgKG1pbi13aWR0aDo3NjhweCl7Lm5h
dmJhci1mb3JtIC5mb3JtLWdyb3Vwe2Rpc3BsYXk6aW5saW5lLWJsb2NrO21hcmdpbi1ib3R0b206
MDt2ZXJ0aWNhbC1hbGlnbjptaWRkbGV9Lm5hdmJhci1mb3JtIC5mb3JtLWNvbnRyb2x7ZGlzcGxh
eTppbmxpbmUtYmxvY2s7d2lkdGg6YXV0bzt2ZXJ0aWNhbC1hbGlnbjptaWRkbGV9Lm5hdmJhci1m
b3JtIC5mb3JtLWNvbnRyb2wtc3RhdGlje2Rpc3BsYXk6aW5saW5lLWJsb2NrfS5uYXZiYXItZm9y
bSAuaW5wdXQtZ3JvdXB7ZGlzcGxheTppbmxpbmUtdGFibGU7dmVydGljYWwtYWxpZ246bWlkZGxl
fS5uYXZiYXItZm9ybSAuaW5wdXQtZ3JvdXAgLmZvcm0tY29udHJvbCwubmF2YmFyLWZvcm0gLmlu
cHV0LWdyb3VwIC5pbnB1dC1ncm91cC1hZGRvbiwubmF2YmFyLWZvcm0gLmlucHV0LWdyb3VwIC5p
bnB1dC1ncm91cC1idG57d2lkdGg6YXV0b30ubmF2YmFyLWZvcm0gLmlucHV0LWdyb3VwPi5mb3Jt
LWNvbnRyb2x7d2lkdGg6MTAwJX0ubmF2YmFyLWZvcm0gLmNvbnRyb2wtbGFiZWx7bWFyZ2luLWJv
dHRvbTowO3ZlcnRpY2FsLWFsaWduOm1pZGRsZX0ubmF2YmFyLWZvcm0gLmNoZWNrYm94LC5uYXZi
YXItZm9ybSAucmFkaW97ZGlzcGxheTppbmxpbmUtYmxvY2s7bWFyZ2luLXRvcDowO21hcmdpbi1i
b3R0b206MDt2ZXJ0aWNhbC1hbGlnbjptaWRkbGV9Lm5hdmJhci1mb3JtIC5jaGVja2JveCBsYWJl
bCwubmF2YmFyLWZvcm0gLnJhZGlvIGxhYmVse3BhZGRpbmctbGVmdDowfS5uYXZiYXItZm9ybSAu
Y2hlY2tib3ggaW5wdXRbdHlwZT1jaGVja2JveF0sLm5hdmJhci1mb3JtIC5yYWRpbyBpbnB1dFt0
eXBlPXJhZGlvXXtwb3NpdGlvbjpyZWxhdGl2ZTttYXJnaW4tbGVmdDowfS5uYXZiYXItZm9ybSAu
aGFzLWZlZWRiYWNrIC5mb3JtLWNvbnRyb2wtZmVlZGJhY2t7dG9wOjB9fUBtZWRpYSAobWF4LXdp
ZHRoOjc2N3B4KXsubmF2YmFyLWZvcm0gLmZvcm0tZ3JvdXB7bWFyZ2luLWJvdHRvbTo1cHh9Lm5h
dmJhci1mb3JtIC5mb3JtLWdyb3VwOmxhc3QtY2hpbGR7bWFyZ2luLWJvdHRvbTowfX1AbWVkaWEg
KG1pbi13aWR0aDo3NjhweCl7Lm5hdmJhci1mb3Jte3dpZHRoOmF1dG87cGFkZGluZy10b3A6MDtw
YWRkaW5nLWJvdHRvbTowO21hcmdpbi1yaWdodDowO21hcmdpbi1sZWZ0OjA7Ym9yZGVyOjA7LXdl
YmtpdC1ib3gtc2hhZG93Om5vbmU7Ym94LXNoYWRvdzpub25lfX0ubmF2YmFyLW5hdj5saT4uZHJv
cGRvd24tbWVudXttYXJnaW4tdG9wOjA7Ym9yZGVyLXRvcC1sZWZ0LXJhZGl1czowO2JvcmRlci10
b3AtcmlnaHQtcmFkaXVzOjB9Lm5hdmJhci1maXhlZC1ib3R0b20gLm5hdmJhci1uYXY+bGk+LmRy
b3Bkb3duLW1lbnV7bWFyZ2luLWJvdHRvbTowO2JvcmRlci10b3AtbGVmdC1yYWRpdXM6NHB4O2Jv
cmRlci10b3AtcmlnaHQtcmFkaXVzOjRweDtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1czowO2Jv
cmRlci1ib3R0b20tbGVmdC1yYWRpdXM6MH0ubmF2YmFyLWJ0bnttYXJnaW4tdG9wOjhweDttYXJn
aW4tYm90dG9tOjhweH0ubmF2YmFyLWJ0bi5idG4tc217bWFyZ2luLXRvcDoxMHB4O21hcmdpbi1i
b3R0b206MTBweH0ubmF2YmFyLWJ0bi5idG4teHN7bWFyZ2luLXRvcDoxNHB4O21hcmdpbi1ib3R0
b206MTRweH0ubmF2YmFyLXRleHR7bWFyZ2luLXRvcDoxNXB4O21hcmdpbi1ib3R0b206MTVweH1A
bWVkaWEgKG1pbi13aWR0aDo3NjhweCl7Lm5hdmJhci10ZXh0e2Zsb2F0OmxlZnQ7bWFyZ2luLXJp
Z2h0OjE1cHg7bWFyZ2luLWxlZnQ6MTVweH19QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5uYXZi
YXItbGVmdHtmbG9hdDpsZWZ0IWltcG9ydGFudH0ubmF2YmFyLXJpZ2h0e2Zsb2F0OnJpZ2h0IWlt
cG9ydGFudDttYXJnaW4tcmlnaHQ6LTE1cHh9Lm5hdmJhci1yaWdodH4ubmF2YmFyLXJpZ2h0e21h
cmdpbi1yaWdodDowfX0ubmF2YmFyLWRlZmF1bHR7YmFja2dyb3VuZC1jb2xvcjojZjhmOGY4O2Jv
cmRlci1jb2xvcjojZTdlN2U3fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLWJyYW5ke2NvbG9yOiM3
Nzd9Lm5hdmJhci1kZWZhdWx0IC5uYXZiYXItYnJhbmQ6Zm9jdXMsLm5hdmJhci1kZWZhdWx0IC5u
YXZiYXItYnJhbmQ6aG92ZXJ7Y29sb3I6IzVlNWU1ZTtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFy
ZW50fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLXRleHR7Y29sb3I6Izc3N30ubmF2YmFyLWRlZmF1
bHQgLm5hdmJhci1uYXY+bGk+YXtjb2xvcjojNzc3fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLW5h
dj5saT5hOmZvY3VzLC5uYXZiYXItZGVmYXVsdCAubmF2YmFyLW5hdj5saT5hOmhvdmVye2NvbG9y
OiMzMzM7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudH0ubmF2YmFyLWRlZmF1bHQgLm5hdmJh
ci1uYXY+LmFjdGl2ZT5hLC5uYXZiYXItZGVmYXVsdCAubmF2YmFyLW5hdj4uYWN0aXZlPmE6Zm9j
dXMsLm5hdmJhci1kZWZhdWx0IC5uYXZiYXItbmF2Pi5hY3RpdmU+YTpob3Zlcntjb2xvcjojNTU1
O2JhY2tncm91bmQtY29sb3I6I2U3ZTdlN30ubmF2YmFyLWRlZmF1bHQgLm5hdmJhci1uYXY+LmRp
c2FibGVkPmEsLm5hdmJhci1kZWZhdWx0IC5uYXZiYXItbmF2Pi5kaXNhYmxlZD5hOmZvY3VzLC5u
YXZiYXItZGVmYXVsdCAubmF2YmFyLW5hdj4uZGlzYWJsZWQ+YTpob3Zlcntjb2xvcjojY2NjO2Jh
Y2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnR9Lm5hdmJhci1kZWZhdWx0IC5uYXZiYXItdG9nZ2xl
e2JvcmRlci1jb2xvcjojZGRkfS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLXRvZ2dsZTpmb2N1cywu
bmF2YmFyLWRlZmF1bHQgLm5hdmJhci10b2dnbGU6aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojZGRk
fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLXRvZ2dsZSAuaWNvbi1iYXJ7YmFja2dyb3VuZC1jb2xv
cjojODg4fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLWNvbGxhcHNlLC5uYXZiYXItZGVmYXVsdCAu
bmF2YmFyLWZvcm17Ym9yZGVyLWNvbG9yOiNlN2U3ZTd9Lm5hdmJhci1kZWZhdWx0IC5uYXZiYXIt
bmF2Pi5vcGVuPmEsLm5hdmJhci1kZWZhdWx0IC5uYXZiYXItbmF2Pi5vcGVuPmE6Zm9jdXMsLm5h
dmJhci1kZWZhdWx0IC5uYXZiYXItbmF2Pi5vcGVuPmE6aG92ZXJ7Y29sb3I6IzU1NTtiYWNrZ3Jv
dW5kLWNvbG9yOiNlN2U3ZTd9QG1lZGlhIChtYXgtd2lkdGg6NzY3cHgpey5uYXZiYXItZGVmYXVs
dCAubmF2YmFyLW5hdiAub3BlbiAuZHJvcGRvd24tbWVudT5saT5he2NvbG9yOiM3Nzd9Lm5hdmJh
ci1kZWZhdWx0IC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1tZW51PmxpPmE6Zm9jdXMsLm5h
dmJhci1kZWZhdWx0IC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1tZW51PmxpPmE6aG92ZXJ7
Y29sb3I6IzMzMztiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50fS5uYXZiYXItZGVmYXVsdCAu
bmF2YmFyLW5hdiAub3BlbiAuZHJvcGRvd24tbWVudT4uYWN0aXZlPmEsLm5hdmJhci1kZWZhdWx0
IC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1tZW51Pi5hY3RpdmU+YTpmb2N1cywubmF2YmFy
LWRlZmF1bHQgLm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bkb3duLW1lbnU+LmFjdGl2ZT5hOmhvdmVy
e2NvbG9yOiM1NTU7YmFja2dyb3VuZC1jb2xvcjojZTdlN2U3fS5uYXZiYXItZGVmYXVsdCAubmF2
YmFyLW5hdiAub3BlbiAuZHJvcGRvd24tbWVudT4uZGlzYWJsZWQ+YSwubmF2YmFyLWRlZmF1bHQg
Lm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bkb3duLW1lbnU+LmRpc2FibGVkPmE6Zm9jdXMsLm5hdmJh
ci1kZWZhdWx0IC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1tZW51Pi5kaXNhYmxlZD5hOmhv
dmVye2NvbG9yOiNjY2M7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudH19Lm5hdmJhci1kZWZh
dWx0IC5uYXZiYXItbGlua3tjb2xvcjojNzc3fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLWxpbms6
aG92ZXJ7Y29sb3I6IzMzM30ubmF2YmFyLWRlZmF1bHQgLmJ0bi1saW5re2NvbG9yOiM3Nzd9Lm5h
dmJhci1kZWZhdWx0IC5idG4tbGluazpmb2N1cywubmF2YmFyLWRlZmF1bHQgLmJ0bi1saW5rOmhv
dmVye2NvbG9yOiMzMzN9Lm5hdmJhci1kZWZhdWx0IC5idG4tbGlua1tkaXNhYmxlZF06Zm9jdXMs
Lm5hdmJhci1kZWZhdWx0IC5idG4tbGlua1tkaXNhYmxlZF06aG92ZXIsZmllbGRzZXRbZGlzYWJs
ZWRdIC5uYXZiYXItZGVmYXVsdCAuYnRuLWxpbms6Zm9jdXMsZmllbGRzZXRbZGlzYWJsZWRdIC5u
YXZiYXItZGVmYXVsdCAuYnRuLWxpbms6aG92ZXJ7Y29sb3I6I2NjY30ubmF2YmFyLWludmVyc2V7
YmFja2dyb3VuZC1jb2xvcjojMjIyO2JvcmRlci1jb2xvcjojMDgwODA4fS5uYXZiYXItaW52ZXJz
ZSAubmF2YmFyLWJyYW5ke2NvbG9yOiM5ZDlkOWR9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItYnJh
bmQ6Zm9jdXMsLm5hdmJhci1pbnZlcnNlIC5uYXZiYXItYnJhbmQ6aG92ZXJ7Y29sb3I6I2ZmZjti
YWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50fS5uYXZiYXItaW52ZXJzZSAubmF2YmFyLXRleHR7
Y29sb3I6IzlkOWQ5ZH0ubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXY+bGk+YXtjb2xvcjojOWQ5
ZDlkfS5uYXZiYXItaW52ZXJzZSAubmF2YmFyLW5hdj5saT5hOmZvY3VzLC5uYXZiYXItaW52ZXJz
ZSAubmF2YmFyLW5hdj5saT5hOmhvdmVye2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjp0cmFu
c3BhcmVudH0ubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXY+LmFjdGl2ZT5hLC5uYXZiYXItaW52
ZXJzZSAubmF2YmFyLW5hdj4uYWN0aXZlPmE6Zm9jdXMsLm5hdmJhci1pbnZlcnNlIC5uYXZiYXIt
bmF2Pi5hY3RpdmU+YTpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6IzA4MDgwOH0u
bmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXY+LmRpc2FibGVkPmEsLm5hdmJhci1pbnZlcnNlIC5u
YXZiYXItbmF2Pi5kaXNhYmxlZD5hOmZvY3VzLC5uYXZiYXItaW52ZXJzZSAubmF2YmFyLW5hdj4u
ZGlzYWJsZWQ+YTpob3Zlcntjb2xvcjojNDQ0O2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnR9
Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItdG9nZ2xle2JvcmRlci1jb2xvcjojMzMzfS5uYXZiYXIt
aW52ZXJzZSAubmF2YmFyLXRvZ2dsZTpmb2N1cywubmF2YmFyLWludmVyc2UgLm5hdmJhci10b2dn
bGU6aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojMzMzfS5uYXZiYXItaW52ZXJzZSAubmF2YmFyLXRv
Z2dsZSAuaWNvbi1iYXJ7YmFja2dyb3VuZC1jb2xvcjojZmZmfS5uYXZiYXItaW52ZXJzZSAubmF2
YmFyLWNvbGxhcHNlLC5uYXZiYXItaW52ZXJzZSAubmF2YmFyLWZvcm17Ym9yZGVyLWNvbG9yOiMx
MDEwMTB9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbmF2Pi5vcGVuPmEsLm5hdmJhci1pbnZlcnNl
IC5uYXZiYXItbmF2Pi5vcGVuPmE6Zm9jdXMsLm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbmF2Pi5v
cGVuPmE6aG92ZXJ7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMwODA4MDh9QG1lZGlhICht
YXgtd2lkdGg6NzY3cHgpey5uYXZiYXItaW52ZXJzZSAubmF2YmFyLW5hdiAub3BlbiAuZHJvcGRv
d24tbWVudT4uZHJvcGRvd24taGVhZGVye2JvcmRlci1jb2xvcjojMDgwODA4fS5uYXZiYXItaW52
ZXJzZSAubmF2YmFyLW5hdiAub3BlbiAuZHJvcGRvd24tbWVudSAuZGl2aWRlcntiYWNrZ3JvdW5k
LWNvbG9yOiMwODA4MDh9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93
bi1tZW51PmxpPmF7Y29sb3I6IzlkOWQ5ZH0ubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXYgLm9w
ZW4gLmRyb3Bkb3duLW1lbnU+bGk+YTpmb2N1cywubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXYg
Lm9wZW4gLmRyb3Bkb3duLW1lbnU+bGk+YTpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29s
b3I6dHJhbnNwYXJlbnR9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93
bi1tZW51Pi5hY3RpdmU+YSwubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bk
b3duLW1lbnU+LmFjdGl2ZT5hOmZvY3VzLC5uYXZiYXItaW52ZXJzZSAubmF2YmFyLW5hdiAub3Bl
biAuZHJvcGRvd24tbWVudT4uYWN0aXZlPmE6aG92ZXJ7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNv
bG9yOiMwODA4MDh9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1t
ZW51Pi5kaXNhYmxlZD5hLC5uYXZiYXItaW52ZXJzZSAubmF2YmFyLW5hdiAub3BlbiAuZHJvcGRv
d24tbWVudT4uZGlzYWJsZWQ+YTpmb2N1cywubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXYgLm9w
ZW4gLmRyb3Bkb3duLW1lbnU+LmRpc2FibGVkPmE6aG92ZXJ7Y29sb3I6IzQ0NDtiYWNrZ3JvdW5k
LWNvbG9yOnRyYW5zcGFyZW50fX0ubmF2YmFyLWludmVyc2UgLm5hdmJhci1saW5re2NvbG9yOiM5
ZDlkOWR9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbGluazpob3Zlcntjb2xvcjojZmZmfS5uYXZi
YXItaW52ZXJzZSAuYnRuLWxpbmt7Y29sb3I6IzlkOWQ5ZH0ubmF2YmFyLWludmVyc2UgLmJ0bi1s
aW5rOmZvY3VzLC5uYXZiYXItaW52ZXJzZSAuYnRuLWxpbms6aG92ZXJ7Y29sb3I6I2ZmZn0ubmF2
YmFyLWludmVyc2UgLmJ0bi1saW5rW2Rpc2FibGVkXTpmb2N1cywubmF2YmFyLWludmVyc2UgLmJ0
bi1saW5rW2Rpc2FibGVkXTpob3ZlcixmaWVsZHNldFtkaXNhYmxlZF0gLm5hdmJhci1pbnZlcnNl
IC5idG4tbGluazpmb2N1cyxmaWVsZHNldFtkaXNhYmxlZF0gLm5hdmJhci1pbnZlcnNlIC5idG4t
bGluazpob3Zlcntjb2xvcjojNDQ0fS5icmVhZGNydW1ie3BhZGRpbmc6OHB4IDE1cHg7bWFyZ2lu
LWJvdHRvbToyMHB4O2xpc3Qtc3R5bGU6bm9uZTtiYWNrZ3JvdW5kLWNvbG9yOiNmNWY1ZjU7Ym9y
ZGVyLXJhZGl1czo0cHh9LmJyZWFkY3J1bWI+bGl7ZGlzcGxheTppbmxpbmUtYmxvY2t9LmJyZWFk
Y3J1bWI+bGkrbGk6YmVmb3Jle3BhZGRpbmc6MCA1cHg7Y29sb3I6I2NjYztjb250ZW50OiIvXDAw
YTAifS5icmVhZGNydW1iPi5hY3RpdmV7Y29sb3I6Izc3N30ucGFnaW5hdGlvbntkaXNwbGF5Omlu
bGluZS1ibG9jaztwYWRkaW5nLWxlZnQ6MDttYXJnaW46MjBweCAwO2JvcmRlci1yYWRpdXM6NHB4
fS5wYWdpbmF0aW9uPmxpe2Rpc3BsYXk6aW5saW5lfS5wYWdpbmF0aW9uPmxpPmEsLnBhZ2luYXRp
b24+bGk+c3Bhbntwb3NpdGlvbjpyZWxhdGl2ZTtmbG9hdDpsZWZ0O3BhZGRpbmc6NnB4IDEycHg7
bWFyZ2luLWxlZnQ6LTFweDtsaW5lLWhlaWdodDoxLjQyODU3MTQzO2NvbG9yOiMzMzdhYjc7dGV4
dC1kZWNvcmF0aW9uOm5vbmU7YmFja2dyb3VuZC1jb2xvcjojZmZmO2JvcmRlcjoxcHggc29saWQg
I2RkZH0ucGFnaW5hdGlvbj5saTpmaXJzdC1jaGlsZD5hLC5wYWdpbmF0aW9uPmxpOmZpcnN0LWNo
aWxkPnNwYW57bWFyZ2luLWxlZnQ6MDtib3JkZXItdG9wLWxlZnQtcmFkaXVzOjRweDtib3JkZXIt
Ym90dG9tLWxlZnQtcmFkaXVzOjRweH0ucGFnaW5hdGlvbj5saTpsYXN0LWNoaWxkPmEsLnBhZ2lu
YXRpb24+bGk6bGFzdC1jaGlsZD5zcGFue2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjRweDtib3Jk
ZXItYm90dG9tLXJpZ2h0LXJhZGl1czo0cHh9LnBhZ2luYXRpb24+bGk+YTpmb2N1cywucGFnaW5h
dGlvbj5saT5hOmhvdmVyLC5wYWdpbmF0aW9uPmxpPnNwYW46Zm9jdXMsLnBhZ2luYXRpb24+bGk+
c3Bhbjpob3Zlcnt6LWluZGV4OjI7Y29sb3I6IzIzNTI3YztiYWNrZ3JvdW5kLWNvbG9yOiNlZWU7
Ym9yZGVyLWNvbG9yOiNkZGR9LnBhZ2luYXRpb24+LmFjdGl2ZT5hLC5wYWdpbmF0aW9uPi5hY3Rp
dmU+YTpmb2N1cywucGFnaW5hdGlvbj4uYWN0aXZlPmE6aG92ZXIsLnBhZ2luYXRpb24+LmFjdGl2
ZT5zcGFuLC5wYWdpbmF0aW9uPi5hY3RpdmU+c3Bhbjpmb2N1cywucGFnaW5hdGlvbj4uYWN0aXZl
PnNwYW46aG92ZXJ7ei1pbmRleDozO2NvbG9yOiNmZmY7Y3Vyc29yOmRlZmF1bHQ7YmFja2dyb3Vu
ZC1jb2xvcjojMzM3YWI3O2JvcmRlci1jb2xvcjojMzM3YWI3fS5wYWdpbmF0aW9uPi5kaXNhYmxl
ZD5hLC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5hOmZvY3VzLC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5h
OmhvdmVyLC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5zcGFuLC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5z
cGFuOmZvY3VzLC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5zcGFuOmhvdmVye2NvbG9yOiM3Nzc7Y3Vy
c29yOm5vdC1hbGxvd2VkO2JhY2tncm91bmQtY29sb3I6I2ZmZjtib3JkZXItY29sb3I6I2RkZH0u
cGFnaW5hdGlvbi1sZz5saT5hLC5wYWdpbmF0aW9uLWxnPmxpPnNwYW57cGFkZGluZzoxMHB4IDE2
cHg7Zm9udC1zaXplOjE4cHg7bGluZS1oZWlnaHQ6MS4zMzMzMzMzfS5wYWdpbmF0aW9uLWxnPmxp
OmZpcnN0LWNoaWxkPmEsLnBhZ2luYXRpb24tbGc+bGk6Zmlyc3QtY2hpbGQ+c3Bhbntib3JkZXIt
dG9wLWxlZnQtcmFkaXVzOjZweDtib3JkZXItYm90dG9tLWxlZnQtcmFkaXVzOjZweH0ucGFnaW5h
dGlvbi1sZz5saTpsYXN0LWNoaWxkPmEsLnBhZ2luYXRpb24tbGc+bGk6bGFzdC1jaGlsZD5zcGFu
e2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjZweDtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1czo2
cHh9LnBhZ2luYXRpb24tc20+bGk+YSwucGFnaW5hdGlvbi1zbT5saT5zcGFue3BhZGRpbmc6NXB4
IDEwcHg7Zm9udC1zaXplOjEycHg7bGluZS1oZWlnaHQ6MS41fS5wYWdpbmF0aW9uLXNtPmxpOmZp
cnN0LWNoaWxkPmEsLnBhZ2luYXRpb24tc20+bGk6Zmlyc3QtY2hpbGQ+c3Bhbntib3JkZXItdG9w
LWxlZnQtcmFkaXVzOjNweDtib3JkZXItYm90dG9tLWxlZnQtcmFkaXVzOjNweH0ucGFnaW5hdGlv
bi1zbT5saTpsYXN0LWNoaWxkPmEsLnBhZ2luYXRpb24tc20+bGk6bGFzdC1jaGlsZD5zcGFue2Jv
cmRlci10b3AtcmlnaHQtcmFkaXVzOjNweDtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1czozcHh9
LnBhZ2Vye3BhZGRpbmctbGVmdDowO21hcmdpbjoyMHB4IDA7dGV4dC1hbGlnbjpjZW50ZXI7bGlz
dC1zdHlsZTpub25lfS5wYWdlciBsaXtkaXNwbGF5OmlubGluZX0ucGFnZXIgbGk+YSwucGFnZXIg
bGk+c3BhbntkaXNwbGF5OmlubGluZS1ibG9jaztwYWRkaW5nOjVweCAxNHB4O2JhY2tncm91bmQt
Y29sb3I6I2ZmZjtib3JkZXI6MXB4IHNvbGlkICNkZGQ7Ym9yZGVyLXJhZGl1czoxNXB4fS5wYWdl
ciBsaT5hOmZvY3VzLC5wYWdlciBsaT5hOmhvdmVye3RleHQtZGVjb3JhdGlvbjpub25lO2JhY2tn
cm91bmQtY29sb3I6I2VlZX0ucGFnZXIgLm5leHQ+YSwucGFnZXIgLm5leHQ+c3BhbntmbG9hdDpy
aWdodH0ucGFnZXIgLnByZXZpb3VzPmEsLnBhZ2VyIC5wcmV2aW91cz5zcGFue2Zsb2F0OmxlZnR9
LnBhZ2VyIC5kaXNhYmxlZD5hLC5wYWdlciAuZGlzYWJsZWQ+YTpmb2N1cywucGFnZXIgLmRpc2Fi
bGVkPmE6aG92ZXIsLnBhZ2VyIC5kaXNhYmxlZD5zcGFue2NvbG9yOiM3Nzc7Y3Vyc29yOm5vdC1h
bGxvd2VkO2JhY2tncm91bmQtY29sb3I6I2ZmZn0ubGFiZWx7ZGlzcGxheTppbmxpbmU7cGFkZGlu
ZzouMmVtIC42ZW0gLjNlbTtmb250LXNpemU6NzUlO2ZvbnQtd2VpZ2h0OjcwMDtsaW5lLWhlaWdo
dDoxO2NvbG9yOiNmZmY7dGV4dC1hbGlnbjpjZW50ZXI7d2hpdGUtc3BhY2U6bm93cmFwO3ZlcnRp
Y2FsLWFsaWduOmJhc2VsaW5lO2JvcmRlci1yYWRpdXM6LjI1ZW19YS5sYWJlbDpmb2N1cyxhLmxh
YmVsOmhvdmVye2NvbG9yOiNmZmY7dGV4dC1kZWNvcmF0aW9uOm5vbmU7Y3Vyc29yOnBvaW50ZXJ9
LmxhYmVsOmVtcHR5e2Rpc3BsYXk6bm9uZX0uYnRuIC5sYWJlbHtwb3NpdGlvbjpyZWxhdGl2ZTt0
b3A6LTFweH0ubGFiZWwtZGVmYXVsdHtiYWNrZ3JvdW5kLWNvbG9yOiM3Nzd9LmxhYmVsLWRlZmF1
bHRbaHJlZl06Zm9jdXMsLmxhYmVsLWRlZmF1bHRbaHJlZl06aG92ZXJ7YmFja2dyb3VuZC1jb2xv
cjojNWU1ZTVlfS5sYWJlbC1wcmltYXJ5e2JhY2tncm91bmQtY29sb3I6IzMzN2FiN30ubGFiZWwt
cHJpbWFyeVtocmVmXTpmb2N1cywubGFiZWwtcHJpbWFyeVtocmVmXTpob3ZlcntiYWNrZ3JvdW5k
LWNvbG9yOiMyODYwOTB9LmxhYmVsLXN1Y2Nlc3N7YmFja2dyb3VuZC1jb2xvcjojNWNiODVjfS5s
YWJlbC1zdWNjZXNzW2hyZWZdOmZvY3VzLC5sYWJlbC1zdWNjZXNzW2hyZWZdOmhvdmVye2JhY2tn
cm91bmQtY29sb3I6IzQ0OWQ0NH0ubGFiZWwtaW5mb3tiYWNrZ3JvdW5kLWNvbG9yOiM1YmMwZGV9
LmxhYmVsLWluZm9baHJlZl06Zm9jdXMsLmxhYmVsLWluZm9baHJlZl06aG92ZXJ7YmFja2dyb3Vu
ZC1jb2xvcjojMzFiMGQ1fS5sYWJlbC13YXJuaW5ne2JhY2tncm91bmQtY29sb3I6I2YwYWQ0ZX0u
bGFiZWwtd2FybmluZ1tocmVmXTpmb2N1cywubGFiZWwtd2FybmluZ1tocmVmXTpob3ZlcntiYWNr
Z3JvdW5kLWNvbG9yOiNlYzk3MWZ9LmxhYmVsLWRhbmdlcntiYWNrZ3JvdW5kLWNvbG9yOiNkOTUz
NGZ9LmxhYmVsLWRhbmdlcltocmVmXTpmb2N1cywubGFiZWwtZGFuZ2VyW2hyZWZdOmhvdmVye2Jh
Y2tncm91bmQtY29sb3I6I2M5MzAyY30uYmFkZ2V7ZGlzcGxheTppbmxpbmUtYmxvY2s7bWluLXdp
ZHRoOjEwcHg7cGFkZGluZzozcHggN3B4O2ZvbnQtc2l6ZToxMnB4O2ZvbnQtd2VpZ2h0OjcwMDts
aW5lLWhlaWdodDoxO2NvbG9yOiNmZmY7dGV4dC1hbGlnbjpjZW50ZXI7d2hpdGUtc3BhY2U6bm93
cmFwO3ZlcnRpY2FsLWFsaWduOm1pZGRsZTtiYWNrZ3JvdW5kLWNvbG9yOiM3Nzc7Ym9yZGVyLXJh
ZGl1czoxMHB4fS5iYWRnZTplbXB0eXtkaXNwbGF5Om5vbmV9LmJ0biAuYmFkZ2V7cG9zaXRpb246
cmVsYXRpdmU7dG9wOi0xcHh9LmJ0bi1ncm91cC14cz4uYnRuIC5iYWRnZSwuYnRuLXhzIC5iYWRn
ZXt0b3A6MDtwYWRkaW5nOjFweCA1cHh9YS5iYWRnZTpmb2N1cyxhLmJhZGdlOmhvdmVye2NvbG9y
OiNmZmY7dGV4dC1kZWNvcmF0aW9uOm5vbmU7Y3Vyc29yOnBvaW50ZXJ9Lmxpc3QtZ3JvdXAtaXRl
bS5hY3RpdmU+LmJhZGdlLC5uYXYtcGlsbHM+LmFjdGl2ZT5hPi5iYWRnZXtjb2xvcjojMzM3YWI3
O2JhY2tncm91bmQtY29sb3I6I2ZmZn0ubGlzdC1ncm91cC1pdGVtPi5iYWRnZXtmbG9hdDpyaWdo
dH0ubGlzdC1ncm91cC1pdGVtPi5iYWRnZSsuYmFkZ2V7bWFyZ2luLXJpZ2h0OjVweH0ubmF2LXBp
bGxzPmxpPmE+LmJhZGdle21hcmdpbi1sZWZ0OjNweH0uanVtYm90cm9ue3BhZGRpbmctdG9wOjMw
cHg7cGFkZGluZy1ib3R0b206MzBweDttYXJnaW4tYm90dG9tOjMwcHg7Y29sb3I6aW5oZXJpdDti
YWNrZ3JvdW5kLWNvbG9yOiNlZWV9Lmp1bWJvdHJvbiAuaDEsLmp1bWJvdHJvbiBoMXtjb2xvcjpp
bmhlcml0fS5qdW1ib3Ryb24gcHttYXJnaW4tYm90dG9tOjE1cHg7Zm9udC1zaXplOjIxcHg7Zm9u
dC13ZWlnaHQ6MjAwfS5qdW1ib3Ryb24+aHJ7Ym9yZGVyLXRvcC1jb2xvcjojZDVkNWQ1fS5jb250
YWluZXIgLmp1bWJvdHJvbiwuY29udGFpbmVyLWZsdWlkIC5qdW1ib3Ryb257cGFkZGluZy1yaWdo
dDoxNXB4O3BhZGRpbmctbGVmdDoxNXB4O2JvcmRlci1yYWRpdXM6NnB4fS5qdW1ib3Ryb24gLmNv
bnRhaW5lcnttYXgtd2lkdGg6MTAwJX1AbWVkaWEgc2NyZWVuIGFuZCAobWluLXdpZHRoOjc2OHB4
KXsuanVtYm90cm9ue3BhZGRpbmctdG9wOjQ4cHg7cGFkZGluZy1ib3R0b206NDhweH0uY29udGFp
bmVyIC5qdW1ib3Ryb24sLmNvbnRhaW5lci1mbHVpZCAuanVtYm90cm9ue3BhZGRpbmctcmlnaHQ6
NjBweDtwYWRkaW5nLWxlZnQ6NjBweH0uanVtYm90cm9uIC5oMSwuanVtYm90cm9uIGgxe2ZvbnQt
c2l6ZTo2M3B4fX0udGh1bWJuYWlse2Rpc3BsYXk6YmxvY2s7cGFkZGluZzo0cHg7bWFyZ2luLWJv
dHRvbToyMHB4O2xpbmUtaGVpZ2h0OjEuNDI4NTcxNDM7YmFja2dyb3VuZC1jb2xvcjojZmZmO2Jv
cmRlcjoxcHggc29saWQgI2RkZDtib3JkZXItcmFkaXVzOjRweDstd2Via2l0LXRyYW5zaXRpb246
Ym9yZGVyIC4ycyBlYXNlLWluLW91dDstby10cmFuc2l0aW9uOmJvcmRlciAuMnMgZWFzZS1pbi1v
dXQ7dHJhbnNpdGlvbjpib3JkZXIgLjJzIGVhc2UtaW4tb3V0fS50aHVtYm5haWwgYT5pbWcsLnRo
dW1ibmFpbD5pbWd7bWFyZ2luLXJpZ2h0OmF1dG87bWFyZ2luLWxlZnQ6YXV0b31hLnRodW1ibmFp
bC5hY3RpdmUsYS50aHVtYm5haWw6Zm9jdXMsYS50aHVtYm5haWw6aG92ZXJ7Ym9yZGVyLWNvbG9y
OiMzMzdhYjd9LnRodW1ibmFpbCAuY2FwdGlvbntwYWRkaW5nOjlweDtjb2xvcjojMzMzfS5hbGVy
dHtwYWRkaW5nOjE1cHg7bWFyZ2luLWJvdHRvbToyMHB4O2JvcmRlcjoxcHggc29saWQgdHJhbnNw
YXJlbnQ7Ym9yZGVyLXJhZGl1czo0cHh9LmFsZXJ0IGg0e21hcmdpbi10b3A6MDtjb2xvcjppbmhl
cml0fS5hbGVydCAuYWxlcnQtbGlua3tmb250LXdlaWdodDo3MDB9LmFsZXJ0PnAsLmFsZXJ0PnVs
e21hcmdpbi1ib3R0b206MH0uYWxlcnQ+cCtwe21hcmdpbi10b3A6NXB4fS5hbGVydC1kaXNtaXNz
YWJsZSwuYWxlcnQtZGlzbWlzc2libGV7cGFkZGluZy1yaWdodDozNXB4fS5hbGVydC1kaXNtaXNz
YWJsZSAuY2xvc2UsLmFsZXJ0LWRpc21pc3NpYmxlIC5jbG9zZXtwb3NpdGlvbjpyZWxhdGl2ZTt0
b3A6LTJweDtyaWdodDotMjFweDtjb2xvcjppbmhlcml0fS5hbGVydC1zdWNjZXNze2NvbG9yOiMz
Yzc2M2Q7YmFja2dyb3VuZC1jb2xvcjojZGZmMGQ4O2JvcmRlci1jb2xvcjojZDZlOWM2fS5hbGVy
dC1zdWNjZXNzIGhye2JvcmRlci10b3AtY29sb3I6I2M5ZTJiM30uYWxlcnQtc3VjY2VzcyAuYWxl
cnQtbGlua3tjb2xvcjojMmI1NDJjfS5hbGVydC1pbmZve2NvbG9yOiMzMTcwOGY7YmFja2dyb3Vu
ZC1jb2xvcjojZDllZGY3O2JvcmRlci1jb2xvcjojYmNlOGYxfS5hbGVydC1pbmZvIGhye2JvcmRl
ci10b3AtY29sb3I6I2E2ZTFlY30uYWxlcnQtaW5mbyAuYWxlcnQtbGlua3tjb2xvcjojMjQ1MjY5
fS5hbGVydC13YXJuaW5ne2NvbG9yOiM4YTZkM2I7YmFja2dyb3VuZC1jb2xvcjojZmNmOGUzO2Jv
cmRlci1jb2xvcjojZmFlYmNjfS5hbGVydC13YXJuaW5nIGhye2JvcmRlci10b3AtY29sb3I6I2Y3
ZTFiNX0uYWxlcnQtd2FybmluZyAuYWxlcnQtbGlua3tjb2xvcjojNjY1MTJjfS5hbGVydC1kYW5n
ZXJ7Y29sb3I6I2E5NDQ0MjtiYWNrZ3JvdW5kLWNvbG9yOiNmMmRlZGU7Ym9yZGVyLWNvbG9yOiNl
YmNjZDF9LmFsZXJ0LWRhbmdlciBocntib3JkZXItdG9wLWNvbG9yOiNlNGI5YzB9LmFsZXJ0LWRh
bmdlciAuYWxlcnQtbGlua3tjb2xvcjojODQzNTM0fUAtd2Via2l0LWtleWZyYW1lcyBwcm9ncmVz
cy1iYXItc3RyaXBlc3tmcm9te2JhY2tncm91bmQtcG9zaXRpb246NDBweCAwfXRve2JhY2tncm91
bmQtcG9zaXRpb246MCAwfX1ALW8ta2V5ZnJhbWVzIHByb2dyZXNzLWJhci1zdHJpcGVze2Zyb217
YmFja2dyb3VuZC1wb3NpdGlvbjo0MHB4IDB9dG97YmFja2dyb3VuZC1wb3NpdGlvbjowIDB9fUBr
ZXlmcmFtZXMgcHJvZ3Jlc3MtYmFyLXN0cmlwZXN7ZnJvbXtiYWNrZ3JvdW5kLXBvc2l0aW9uOjQw
cHggMH10b3tiYWNrZ3JvdW5kLXBvc2l0aW9uOjAgMH19LnByb2dyZXNze2hlaWdodDoyMHB4O21h
cmdpbi1ib3R0b206MjBweDtvdmVyZmxvdzpoaWRkZW47YmFja2dyb3VuZC1jb2xvcjojZjVmNWY1
O2JvcmRlci1yYWRpdXM6NHB4Oy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFweCAycHggcmdi
YSgwLDAsMCwuMSk7Ym94LXNoYWRvdzppbnNldCAwIDFweCAycHggcmdiYSgwLDAsMCwuMSl9LnBy
b2dyZXNzLWJhcntmbG9hdDpsZWZ0O3dpZHRoOjA7aGVpZ2h0OjEwMCU7Zm9udC1zaXplOjEycHg7
bGluZS1oZWlnaHQ6MjBweDtjb2xvcjojZmZmO3RleHQtYWxpZ246Y2VudGVyO2JhY2tncm91bmQt
Y29sb3I6IzMzN2FiNzstd2Via2l0LWJveC1zaGFkb3c6aW5zZXQgMCAtMXB4IDAgcmdiYSgwLDAs
MCwuMTUpO2JveC1zaGFkb3c6aW5zZXQgMCAtMXB4IDAgcmdiYSgwLDAsMCwuMTUpOy13ZWJraXQt
dHJhbnNpdGlvbjp3aWR0aCAuNnMgZWFzZTstby10cmFuc2l0aW9uOndpZHRoIC42cyBlYXNlO3Ry
YW5zaXRpb246d2lkdGggLjZzIGVhc2V9LnByb2dyZXNzLWJhci1zdHJpcGVkLC5wcm9ncmVzcy1z
dHJpcGVkIC5wcm9ncmVzcy1iYXJ7YmFja2dyb3VuZC1pbWFnZTotd2Via2l0LWxpbmVhci1ncmFk
aWVudCg0NWRlZyxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgMjUlLHRyYW5zcGFyZW50IDI1JSx0cmFu
c3BhcmVudCA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4x
NSkgNzUlLHRyYW5zcGFyZW50IDc1JSx0cmFuc3BhcmVudCk7YmFja2dyb3VuZC1pbWFnZTotby1s
aW5lYXItZ3JhZGllbnQoNDVkZWcscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDI1JSx0cmFuc3BhcmVu
dCAyNSUsdHJhbnNwYXJlbnQgNTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA1MCUscmdiYSgyNTUs
MjU1LDI1NSwuMTUpIDc1JSx0cmFuc3BhcmVudCA3NSUsdHJhbnNwYXJlbnQpO2JhY2tncm91bmQt
aW1hZ2U6bGluZWFyLWdyYWRpZW50KDQ1ZGVnLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSAyNSUsdHJh
bnNwYXJlbnQgMjUlLHRyYW5zcGFyZW50IDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgNTAlLHJn
YmEoMjU1LDI1NSwyNTUsLjE1KSA3NSUsdHJhbnNwYXJlbnQgNzUlLHRyYW5zcGFyZW50KTstd2Vi
a2l0LWJhY2tncm91bmQtc2l6ZTo0MHB4IDQwcHg7YmFja2dyb3VuZC1zaXplOjQwcHggNDBweH0u
cHJvZ3Jlc3MtYmFyLmFjdGl2ZSwucHJvZ3Jlc3MuYWN0aXZlIC5wcm9ncmVzcy1iYXJ7LXdlYmtp
dC1hbmltYXRpb246cHJvZ3Jlc3MtYmFyLXN0cmlwZXMgMnMgbGluZWFyIGluZmluaXRlOy1vLWFu
aW1hdGlvbjpwcm9ncmVzcy1iYXItc3RyaXBlcyAycyBsaW5lYXIgaW5maW5pdGU7YW5pbWF0aW9u
OnByb2dyZXNzLWJhci1zdHJpcGVzIDJzIGxpbmVhciBpbmZpbml0ZX0ucHJvZ3Jlc3MtYmFyLXN1
Y2Nlc3N7YmFja2dyb3VuZC1jb2xvcjojNWNiODVjfS5wcm9ncmVzcy1zdHJpcGVkIC5wcm9ncmVz
cy1iYXItc3VjY2Vzc3tiYWNrZ3JvdW5kLWltYWdlOi13ZWJraXQtbGluZWFyLWdyYWRpZW50KDQ1
ZGVnLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSAyNSUsdHJhbnNwYXJlbnQgMjUlLHRyYW5zcGFyZW50
IDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgNTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA3NSUs
dHJhbnNwYXJlbnQgNzUlLHRyYW5zcGFyZW50KTtiYWNrZ3JvdW5kLWltYWdlOi1vLWxpbmVhci1n
cmFkaWVudCg0NWRlZyxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgMjUlLHRyYW5zcGFyZW50IDI1JSx0
cmFuc3BhcmVudCA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDUwJSxyZ2JhKDI1NSwyNTUsMjU1
LC4xNSkgNzUlLHRyYW5zcGFyZW50IDc1JSx0cmFuc3BhcmVudCk7YmFja2dyb3VuZC1pbWFnZTps
aW5lYXItZ3JhZGllbnQoNDVkZWcscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDI1JSx0cmFuc3BhcmVu
dCAyNSUsdHJhbnNwYXJlbnQgNTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA1MCUscmdiYSgyNTUs
MjU1LDI1NSwuMTUpIDc1JSx0cmFuc3BhcmVudCA3NSUsdHJhbnNwYXJlbnQpfS5wcm9ncmVzcy1i
YXItaW5mb3tiYWNrZ3JvdW5kLWNvbG9yOiM1YmMwZGV9LnByb2dyZXNzLXN0cmlwZWQgLnByb2dy
ZXNzLWJhci1pbmZve2JhY2tncm91bmQtaW1hZ2U6LXdlYmtpdC1saW5lYXItZ3JhZGllbnQoNDVk
ZWcscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDI1JSx0cmFuc3BhcmVudCAyNSUsdHJhbnNwYXJlbnQg
NTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDc1JSx0
cmFuc3BhcmVudCA3NSUsdHJhbnNwYXJlbnQpO2JhY2tncm91bmQtaW1hZ2U6LW8tbGluZWFyLWdy
YWRpZW50KDQ1ZGVnLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSAyNSUsdHJhbnNwYXJlbnQgMjUlLHRy
YW5zcGFyZW50IDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgNTAlLHJnYmEoMjU1LDI1NSwyNTUs
LjE1KSA3NSUsdHJhbnNwYXJlbnQgNzUlLHRyYW5zcGFyZW50KTtiYWNrZ3JvdW5kLWltYWdlOmxp
bmVhci1ncmFkaWVudCg0NWRlZyxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgMjUlLHRyYW5zcGFyZW50
IDI1JSx0cmFuc3BhcmVudCA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDUwJSxyZ2JhKDI1NSwy
NTUsMjU1LC4xNSkgNzUlLHRyYW5zcGFyZW50IDc1JSx0cmFuc3BhcmVudCl9LnByb2dyZXNzLWJh
ci13YXJuaW5ne2JhY2tncm91bmQtY29sb3I6I2YwYWQ0ZX0ucHJvZ3Jlc3Mtc3RyaXBlZCAucHJv
Z3Jlc3MtYmFyLXdhcm5pbmd7YmFja2dyb3VuZC1pbWFnZTotd2Via2l0LWxpbmVhci1ncmFkaWVu
dCg0NWRlZyxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgMjUlLHRyYW5zcGFyZW50IDI1JSx0cmFuc3Bh
cmVudCA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkg
NzUlLHRyYW5zcGFyZW50IDc1JSx0cmFuc3BhcmVudCk7YmFja2dyb3VuZC1pbWFnZTotby1saW5l
YXItZ3JhZGllbnQoNDVkZWcscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDI1JSx0cmFuc3BhcmVudCAy
NSUsdHJhbnNwYXJlbnQgNTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA1MCUscmdiYSgyNTUsMjU1
LDI1NSwuMTUpIDc1JSx0cmFuc3BhcmVudCA3NSUsdHJhbnNwYXJlbnQpO2JhY2tncm91bmQtaW1h
Z2U6bGluZWFyLWdyYWRpZW50KDQ1ZGVnLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSAyNSUsdHJhbnNw
YXJlbnQgMjUlLHRyYW5zcGFyZW50IDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgNTAlLHJnYmEo
MjU1LDI1NSwyNTUsLjE1KSA3NSUsdHJhbnNwYXJlbnQgNzUlLHRyYW5zcGFyZW50KX0ucHJvZ3Jl
c3MtYmFyLWRhbmdlcntiYWNrZ3JvdW5kLWNvbG9yOiNkOTUzNGZ9LnByb2dyZXNzLXN0cmlwZWQg
LnByb2dyZXNzLWJhci1kYW5nZXJ7YmFja2dyb3VuZC1pbWFnZTotd2Via2l0LWxpbmVhci1ncmFk
aWVudCg0NWRlZyxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgMjUlLHRyYW5zcGFyZW50IDI1JSx0cmFu
c3BhcmVudCA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4x
NSkgNzUlLHRyYW5zcGFyZW50IDc1JSx0cmFuc3BhcmVudCk7YmFja2dyb3VuZC1pbWFnZTotby1s
aW5lYXItZ3JhZGllbnQoNDVkZWcscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDI1JSx0cmFuc3BhcmVu
dCAyNSUsdHJhbnNwYXJlbnQgNTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA1MCUscmdiYSgyNTUs
MjU1LDI1NSwuMTUpIDc1JSx0cmFuc3BhcmVudCA3NSUsdHJhbnNwYXJlbnQpO2JhY2tncm91bmQt
aW1hZ2U6bGluZWFyLWdyYWRpZW50KDQ1ZGVnLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSAyNSUsdHJh
bnNwYXJlbnQgMjUlLHRyYW5zcGFyZW50IDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgNTAlLHJn
YmEoMjU1LDI1NSwyNTUsLjE1KSA3NSUsdHJhbnNwYXJlbnQgNzUlLHRyYW5zcGFyZW50KX0ubWVk
aWF7bWFyZ2luLXRvcDoxNXB4fS5tZWRpYTpmaXJzdC1jaGlsZHttYXJnaW4tdG9wOjB9Lm1lZGlh
LC5tZWRpYS1ib2R5e292ZXJmbG93OmhpZGRlbjt6b29tOjF9Lm1lZGlhLWJvZHl7d2lkdGg6MTAw
MDBweH0ubWVkaWEtb2JqZWN0e2Rpc3BsYXk6YmxvY2t9Lm1lZGlhLW9iamVjdC5pbWctdGh1bWJu
YWlse21heC13aWR0aDpub25lfS5tZWRpYS1yaWdodCwubWVkaWE+LnB1bGwtcmlnaHR7cGFkZGlu
Zy1sZWZ0OjEwcHh9Lm1lZGlhLWxlZnQsLm1lZGlhPi5wdWxsLWxlZnR7cGFkZGluZy1yaWdodDox
MHB4fS5tZWRpYS1ib2R5LC5tZWRpYS1sZWZ0LC5tZWRpYS1yaWdodHtkaXNwbGF5OnRhYmxlLWNl
bGw7dmVydGljYWwtYWxpZ246dG9wfS5tZWRpYS1taWRkbGV7dmVydGljYWwtYWxpZ246bWlkZGxl
fS5tZWRpYS1ib3R0b217dmVydGljYWwtYWxpZ246Ym90dG9tfS5tZWRpYS1oZWFkaW5ne21hcmdp
bi10b3A6MDttYXJnaW4tYm90dG9tOjVweH0ubWVkaWEtbGlzdHtwYWRkaW5nLWxlZnQ6MDtsaXN0
LXN0eWxlOm5vbmV9Lmxpc3QtZ3JvdXB7cGFkZGluZy1sZWZ0OjA7bWFyZ2luLWJvdHRvbToyMHB4
fS5saXN0LWdyb3VwLWl0ZW17cG9zaXRpb246cmVsYXRpdmU7ZGlzcGxheTpibG9jaztwYWRkaW5n
OjEwcHggMTVweDttYXJnaW4tYm90dG9tOi0xcHg7YmFja2dyb3VuZC1jb2xvcjojZmZmO2JvcmRl
cjoxcHggc29saWQgI2RkZH0ubGlzdC1ncm91cC1pdGVtOmZpcnN0LWNoaWxke2JvcmRlci10b3At
bGVmdC1yYWRpdXM6NHB4O2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjRweH0ubGlzdC1ncm91cC1p
dGVtOmxhc3QtY2hpbGR7bWFyZ2luLWJvdHRvbTowO2JvcmRlci1ib3R0b20tcmlnaHQtcmFkaXVz
OjRweDtib3JkZXItYm90dG9tLWxlZnQtcmFkaXVzOjRweH1hLmxpc3QtZ3JvdXAtaXRlbSxidXR0
b24ubGlzdC1ncm91cC1pdGVte2NvbG9yOiM1NTV9YS5saXN0LWdyb3VwLWl0ZW0gLmxpc3QtZ3Jv
dXAtaXRlbS1oZWFkaW5nLGJ1dHRvbi5saXN0LWdyb3VwLWl0ZW0gLmxpc3QtZ3JvdXAtaXRlbS1o
ZWFkaW5ne2NvbG9yOiMzMzN9YS5saXN0LWdyb3VwLWl0ZW06Zm9jdXMsYS5saXN0LWdyb3VwLWl0
ZW06aG92ZXIsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbTpmb2N1cyxidXR0b24ubGlzdC1ncm91cC1p
dGVtOmhvdmVye2NvbG9yOiM1NTU7dGV4dC1kZWNvcmF0aW9uOm5vbmU7YmFja2dyb3VuZC1jb2xv
cjojZjVmNWY1fWJ1dHRvbi5saXN0LWdyb3VwLWl0ZW17d2lkdGg6MTAwJTt0ZXh0LWFsaWduOmxl
ZnR9Lmxpc3QtZ3JvdXAtaXRlbS5kaXNhYmxlZCwubGlzdC1ncm91cC1pdGVtLmRpc2FibGVkOmZv
Y3VzLC5saXN0LWdyb3VwLWl0ZW0uZGlzYWJsZWQ6aG92ZXJ7Y29sb3I6Izc3NztjdXJzb3I6bm90
LWFsbG93ZWQ7YmFja2dyb3VuZC1jb2xvcjojZWVlfS5saXN0LWdyb3VwLWl0ZW0uZGlzYWJsZWQg
Lmxpc3QtZ3JvdXAtaXRlbS1oZWFkaW5nLC5saXN0LWdyb3VwLWl0ZW0uZGlzYWJsZWQ6Zm9jdXMg
Lmxpc3QtZ3JvdXAtaXRlbS1oZWFkaW5nLC5saXN0LWdyb3VwLWl0ZW0uZGlzYWJsZWQ6aG92ZXIg
Lmxpc3QtZ3JvdXAtaXRlbS1oZWFkaW5ne2NvbG9yOmluaGVyaXR9Lmxpc3QtZ3JvdXAtaXRlbS5k
aXNhYmxlZCAubGlzdC1ncm91cC1pdGVtLXRleHQsLmxpc3QtZ3JvdXAtaXRlbS5kaXNhYmxlZDpm
b2N1cyAubGlzdC1ncm91cC1pdGVtLXRleHQsLmxpc3QtZ3JvdXAtaXRlbS5kaXNhYmxlZDpob3Zl
ciAubGlzdC1ncm91cC1pdGVtLXRleHR7Y29sb3I6Izc3N30ubGlzdC1ncm91cC1pdGVtLmFjdGl2
ZSwubGlzdC1ncm91cC1pdGVtLmFjdGl2ZTpmb2N1cywubGlzdC1ncm91cC1pdGVtLmFjdGl2ZTpo
b3Zlcnt6LWluZGV4OjI7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzMzdhYjc7Ym9yZGVy
LWNvbG9yOiMzMzdhYjd9Lmxpc3QtZ3JvdXAtaXRlbS5hY3RpdmUgLmxpc3QtZ3JvdXAtaXRlbS1o
ZWFkaW5nLC5saXN0LWdyb3VwLWl0ZW0uYWN0aXZlIC5saXN0LWdyb3VwLWl0ZW0taGVhZGluZz4u
c21hbGwsLmxpc3QtZ3JvdXAtaXRlbS5hY3RpdmUgLmxpc3QtZ3JvdXAtaXRlbS1oZWFkaW5nPnNt
YWxsLC5saXN0LWdyb3VwLWl0ZW0uYWN0aXZlOmZvY3VzIC5saXN0LWdyb3VwLWl0ZW0taGVhZGlu
ZywubGlzdC1ncm91cC1pdGVtLmFjdGl2ZTpmb2N1cyAubGlzdC1ncm91cC1pdGVtLWhlYWRpbmc+
LnNtYWxsLC5saXN0LWdyb3VwLWl0ZW0uYWN0aXZlOmZvY3VzIC5saXN0LWdyb3VwLWl0ZW0taGVh
ZGluZz5zbWFsbCwubGlzdC1ncm91cC1pdGVtLmFjdGl2ZTpob3ZlciAubGlzdC1ncm91cC1pdGVt
LWhlYWRpbmcsLmxpc3QtZ3JvdXAtaXRlbS5hY3RpdmU6aG92ZXIgLmxpc3QtZ3JvdXAtaXRlbS1o
ZWFkaW5nPi5zbWFsbCwubGlzdC1ncm91cC1pdGVtLmFjdGl2ZTpob3ZlciAubGlzdC1ncm91cC1p
dGVtLWhlYWRpbmc+c21hbGx7Y29sb3I6aW5oZXJpdH0ubGlzdC1ncm91cC1pdGVtLmFjdGl2ZSAu
bGlzdC1ncm91cC1pdGVtLXRleHQsLmxpc3QtZ3JvdXAtaXRlbS5hY3RpdmU6Zm9jdXMgLmxpc3Qt
Z3JvdXAtaXRlbS10ZXh0LC5saXN0LWdyb3VwLWl0ZW0uYWN0aXZlOmhvdmVyIC5saXN0LWdyb3Vw
LWl0ZW0tdGV4dHtjb2xvcjojYzdkZGVmfS5saXN0LWdyb3VwLWl0ZW0tc3VjY2Vzc3tjb2xvcjoj
M2M3NjNkO2JhY2tncm91bmQtY29sb3I6I2RmZjBkOH1hLmxpc3QtZ3JvdXAtaXRlbS1zdWNjZXNz
LGJ1dHRvbi5saXN0LWdyb3VwLWl0ZW0tc3VjY2Vzc3tjb2xvcjojM2M3NjNkfWEubGlzdC1ncm91
cC1pdGVtLXN1Y2Nlc3MgLmxpc3QtZ3JvdXAtaXRlbS1oZWFkaW5nLGJ1dHRvbi5saXN0LWdyb3Vw
LWl0ZW0tc3VjY2VzcyAubGlzdC1ncm91cC1pdGVtLWhlYWRpbmd7Y29sb3I6aW5oZXJpdH1hLmxp
c3QtZ3JvdXAtaXRlbS1zdWNjZXNzOmZvY3VzLGEubGlzdC1ncm91cC1pdGVtLXN1Y2Nlc3M6aG92
ZXIsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS1zdWNjZXNzOmZvY3VzLGJ1dHRvbi5saXN0LWdyb3Vw
LWl0ZW0tc3VjY2Vzczpob3Zlcntjb2xvcjojM2M3NjNkO2JhY2tncm91bmQtY29sb3I6I2QwZTlj
Nn1hLmxpc3QtZ3JvdXAtaXRlbS1zdWNjZXNzLmFjdGl2ZSxhLmxpc3QtZ3JvdXAtaXRlbS1zdWNj
ZXNzLmFjdGl2ZTpmb2N1cyxhLmxpc3QtZ3JvdXAtaXRlbS1zdWNjZXNzLmFjdGl2ZTpob3Zlcixi
dXR0b24ubGlzdC1ncm91cC1pdGVtLXN1Y2Nlc3MuYWN0aXZlLGJ1dHRvbi5saXN0LWdyb3VwLWl0
ZW0tc3VjY2Vzcy5hY3RpdmU6Zm9jdXMsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS1zdWNjZXNzLmFj
dGl2ZTpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6IzNjNzYzZDtib3JkZXItY29s
b3I6IzNjNzYzZH0ubGlzdC1ncm91cC1pdGVtLWluZm97Y29sb3I6IzMxNzA4ZjtiYWNrZ3JvdW5k
LWNvbG9yOiNkOWVkZjd9YS5saXN0LWdyb3VwLWl0ZW0taW5mbyxidXR0b24ubGlzdC1ncm91cC1p
dGVtLWluZm97Y29sb3I6IzMxNzA4Zn1hLmxpc3QtZ3JvdXAtaXRlbS1pbmZvIC5saXN0LWdyb3Vw
LWl0ZW0taGVhZGluZyxidXR0b24ubGlzdC1ncm91cC1pdGVtLWluZm8gLmxpc3QtZ3JvdXAtaXRl
bS1oZWFkaW5ne2NvbG9yOmluaGVyaXR9YS5saXN0LWdyb3VwLWl0ZW0taW5mbzpmb2N1cyxhLmxp
c3QtZ3JvdXAtaXRlbS1pbmZvOmhvdmVyLGJ1dHRvbi5saXN0LWdyb3VwLWl0ZW0taW5mbzpmb2N1
cyxidXR0b24ubGlzdC1ncm91cC1pdGVtLWluZm86aG92ZXJ7Y29sb3I6IzMxNzA4ZjtiYWNrZ3Jv
dW5kLWNvbG9yOiNjNGUzZjN9YS5saXN0LWdyb3VwLWl0ZW0taW5mby5hY3RpdmUsYS5saXN0LWdy
b3VwLWl0ZW0taW5mby5hY3RpdmU6Zm9jdXMsYS5saXN0LWdyb3VwLWl0ZW0taW5mby5hY3RpdmU6
aG92ZXIsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS1pbmZvLmFjdGl2ZSxidXR0b24ubGlzdC1ncm91
cC1pdGVtLWluZm8uYWN0aXZlOmZvY3VzLGJ1dHRvbi5saXN0LWdyb3VwLWl0ZW0taW5mby5hY3Rp
dmU6aG92ZXJ7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzMTcwOGY7Ym9yZGVyLWNvbG9y
OiMzMTcwOGZ9Lmxpc3QtZ3JvdXAtaXRlbS13YXJuaW5ne2NvbG9yOiM4YTZkM2I7YmFja2dyb3Vu
ZC1jb2xvcjojZmNmOGUzfWEubGlzdC1ncm91cC1pdGVtLXdhcm5pbmcsYnV0dG9uLmxpc3QtZ3Jv
dXAtaXRlbS13YXJuaW5ne2NvbG9yOiM4YTZkM2J9YS5saXN0LWdyb3VwLWl0ZW0td2FybmluZyAu
bGlzdC1ncm91cC1pdGVtLWhlYWRpbmcsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS13YXJuaW5nIC5s
aXN0LWdyb3VwLWl0ZW0taGVhZGluZ3tjb2xvcjppbmhlcml0fWEubGlzdC1ncm91cC1pdGVtLXdh
cm5pbmc6Zm9jdXMsYS5saXN0LWdyb3VwLWl0ZW0td2FybmluZzpob3ZlcixidXR0b24ubGlzdC1n
cm91cC1pdGVtLXdhcm5pbmc6Zm9jdXMsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS13YXJuaW5nOmhv
dmVye2NvbG9yOiM4YTZkM2I7YmFja2dyb3VuZC1jb2xvcjojZmFmMmNjfWEubGlzdC1ncm91cC1p
dGVtLXdhcm5pbmcuYWN0aXZlLGEubGlzdC1ncm91cC1pdGVtLXdhcm5pbmcuYWN0aXZlOmZvY3Vz
LGEubGlzdC1ncm91cC1pdGVtLXdhcm5pbmcuYWN0aXZlOmhvdmVyLGJ1dHRvbi5saXN0LWdyb3Vw
LWl0ZW0td2FybmluZy5hY3RpdmUsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS13YXJuaW5nLmFjdGl2
ZTpmb2N1cyxidXR0b24ubGlzdC1ncm91cC1pdGVtLXdhcm5pbmcuYWN0aXZlOmhvdmVye2NvbG9y
OiNmZmY7YmFja2dyb3VuZC1jb2xvcjojOGE2ZDNiO2JvcmRlci1jb2xvcjojOGE2ZDNifS5saXN0
LWdyb3VwLWl0ZW0tZGFuZ2Vye2NvbG9yOiNhOTQ0NDI7YmFja2dyb3VuZC1jb2xvcjojZjJkZWRl
fWEubGlzdC1ncm91cC1pdGVtLWRhbmdlcixidXR0b24ubGlzdC1ncm91cC1pdGVtLWRhbmdlcntj
b2xvcjojYTk0NDQyfWEubGlzdC1ncm91cC1pdGVtLWRhbmdlciAubGlzdC1ncm91cC1pdGVtLWhl
YWRpbmcsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS1kYW5nZXIgLmxpc3QtZ3JvdXAtaXRlbS1oZWFk
aW5ne2NvbG9yOmluaGVyaXR9YS5saXN0LWdyb3VwLWl0ZW0tZGFuZ2VyOmZvY3VzLGEubGlzdC1n
cm91cC1pdGVtLWRhbmdlcjpob3ZlcixidXR0b24ubGlzdC1ncm91cC1pdGVtLWRhbmdlcjpmb2N1
cyxidXR0b24ubGlzdC1ncm91cC1pdGVtLWRhbmdlcjpob3Zlcntjb2xvcjojYTk0NDQyO2JhY2tn
cm91bmQtY29sb3I6I2ViY2NjY31hLmxpc3QtZ3JvdXAtaXRlbS1kYW5nZXIuYWN0aXZlLGEubGlz
dC1ncm91cC1pdGVtLWRhbmdlci5hY3RpdmU6Zm9jdXMsYS5saXN0LWdyb3VwLWl0ZW0tZGFuZ2Vy
LmFjdGl2ZTpob3ZlcixidXR0b24ubGlzdC1ncm91cC1pdGVtLWRhbmdlci5hY3RpdmUsYnV0dG9u
Lmxpc3QtZ3JvdXAtaXRlbS1kYW5nZXIuYWN0aXZlOmZvY3VzLGJ1dHRvbi5saXN0LWdyb3VwLWl0
ZW0tZGFuZ2VyLmFjdGl2ZTpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6I2E5NDQ0
Mjtib3JkZXItY29sb3I6I2E5NDQ0Mn0ubGlzdC1ncm91cC1pdGVtLWhlYWRpbmd7bWFyZ2luLXRv
cDowO21hcmdpbi1ib3R0b206NXB4fS5saXN0LWdyb3VwLWl0ZW0tdGV4dHttYXJnaW4tYm90dG9t
OjA7bGluZS1oZWlnaHQ6MS4zfS5wYW5lbHttYXJnaW4tYm90dG9tOjIwcHg7YmFja2dyb3VuZC1j
b2xvcjojZmZmO2JvcmRlcjoxcHggc29saWQgdHJhbnNwYXJlbnQ7Ym9yZGVyLXJhZGl1czo0cHg7
LXdlYmtpdC1ib3gtc2hhZG93OjAgMXB4IDFweCByZ2JhKDAsMCwwLC4wNSk7Ym94LXNoYWRvdzow
IDFweCAxcHggcmdiYSgwLDAsMCwuMDUpfS5wYW5lbC1ib2R5e3BhZGRpbmc6MTVweH0ucGFuZWwt
aGVhZGluZ3twYWRkaW5nOjEwcHggMTVweDtib3JkZXItYm90dG9tOjFweCBzb2xpZCB0cmFuc3Bh
cmVudDtib3JkZXItdG9wLWxlZnQtcmFkaXVzOjNweDtib3JkZXItdG9wLXJpZ2h0LXJhZGl1czoz
cHh9LnBhbmVsLWhlYWRpbmc+LmRyb3Bkb3duIC5kcm9wZG93bi10b2dnbGV7Y29sb3I6aW5oZXJp
dH0ucGFuZWwtdGl0bGV7bWFyZ2luLXRvcDowO21hcmdpbi1ib3R0b206MDtmb250LXNpemU6MTZw
eDtjb2xvcjppbmhlcml0fS5wYW5lbC10aXRsZT4uc21hbGwsLnBhbmVsLXRpdGxlPi5zbWFsbD5h
LC5wYW5lbC10aXRsZT5hLC5wYW5lbC10aXRsZT5zbWFsbCwucGFuZWwtdGl0bGU+c21hbGw+YXtj
b2xvcjppbmhlcml0fS5wYW5lbC1mb290ZXJ7cGFkZGluZzoxMHB4IDE1cHg7YmFja2dyb3VuZC1j
b2xvcjojZjVmNWY1O2JvcmRlci10b3A6MXB4IHNvbGlkICNkZGQ7Ym9yZGVyLWJvdHRvbS1yaWdo
dC1yYWRpdXM6M3B4O2JvcmRlci1ib3R0b20tbGVmdC1yYWRpdXM6M3B4fS5wYW5lbD4ubGlzdC1n
cm91cCwucGFuZWw+LnBhbmVsLWNvbGxhcHNlPi5saXN0LWdyb3Vwe21hcmdpbi1ib3R0b206MH0u
cGFuZWw+Lmxpc3QtZ3JvdXAgLmxpc3QtZ3JvdXAtaXRlbSwucGFuZWw+LnBhbmVsLWNvbGxhcHNl
Pi5saXN0LWdyb3VwIC5saXN0LWdyb3VwLWl0ZW17Ym9yZGVyLXdpZHRoOjFweCAwO2JvcmRlci1y
YWRpdXM6MH0ucGFuZWw+Lmxpc3QtZ3JvdXA6Zmlyc3QtY2hpbGQgLmxpc3QtZ3JvdXAtaXRlbTpm
aXJzdC1jaGlsZCwucGFuZWw+LnBhbmVsLWNvbGxhcHNlPi5saXN0LWdyb3VwOmZpcnN0LWNoaWxk
IC5saXN0LWdyb3VwLWl0ZW06Zmlyc3QtY2hpbGR7Ym9yZGVyLXRvcDowO2JvcmRlci10b3AtbGVm
dC1yYWRpdXM6M3B4O2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjNweH0ucGFuZWw+Lmxpc3QtZ3Jv
dXA6bGFzdC1jaGlsZCAubGlzdC1ncm91cC1pdGVtOmxhc3QtY2hpbGQsLnBhbmVsPi5wYW5lbC1j
b2xsYXBzZT4ubGlzdC1ncm91cDpsYXN0LWNoaWxkIC5saXN0LWdyb3VwLWl0ZW06bGFzdC1jaGls
ZHtib3JkZXItYm90dG9tOjA7Ym9yZGVyLWJvdHRvbS1yaWdodC1yYWRpdXM6M3B4O2JvcmRlci1i
b3R0b20tbGVmdC1yYWRpdXM6M3B4fS5wYW5lbD4ucGFuZWwtaGVhZGluZysucGFuZWwtY29sbGFw
c2U+Lmxpc3QtZ3JvdXAgLmxpc3QtZ3JvdXAtaXRlbTpmaXJzdC1jaGlsZHtib3JkZXItdG9wLWxl
ZnQtcmFkaXVzOjA7Ym9yZGVyLXRvcC1yaWdodC1yYWRpdXM6MH0ucGFuZWwtaGVhZGluZysubGlz
dC1ncm91cCAubGlzdC1ncm91cC1pdGVtOmZpcnN0LWNoaWxke2JvcmRlci10b3Atd2lkdGg6MH0u
bGlzdC1ncm91cCsucGFuZWwtZm9vdGVye2JvcmRlci10b3Atd2lkdGg6MH0ucGFuZWw+LnBhbmVs
LWNvbGxhcHNlPi50YWJsZSwucGFuZWw+LnRhYmxlLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZT4u
dGFibGV7bWFyZ2luLWJvdHRvbTowfS5wYW5lbD4ucGFuZWwtY29sbGFwc2U+LnRhYmxlIGNhcHRp
b24sLnBhbmVsPi50YWJsZSBjYXB0aW9uLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZT4udGFibGUg
Y2FwdGlvbntwYWRkaW5nLXJpZ2h0OjE1cHg7cGFkZGluZy1sZWZ0OjE1cHh9LnBhbmVsPi50YWJs
ZS1yZXNwb25zaXZlOmZpcnN0LWNoaWxkPi50YWJsZTpmaXJzdC1jaGlsZCwucGFuZWw+LnRhYmxl
OmZpcnN0LWNoaWxke2JvcmRlci10b3AtbGVmdC1yYWRpdXM6M3B4O2JvcmRlci10b3AtcmlnaHQt
cmFkaXVzOjNweH0ucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU6Zmlyc3QtY2hpbGQ+LnRhYmxlOmZp
cnN0LWNoaWxkPnRib2R5OmZpcnN0LWNoaWxkPnRyOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGUt
cmVzcG9uc2l2ZTpmaXJzdC1jaGlsZD4udGFibGU6Zmlyc3QtY2hpbGQ+dGhlYWQ6Zmlyc3QtY2hp
bGQ+dHI6Zmlyc3QtY2hpbGQsLnBhbmVsPi50YWJsZTpmaXJzdC1jaGlsZD50Ym9keTpmaXJzdC1j
aGlsZD50cjpmaXJzdC1jaGlsZCwucGFuZWw+LnRhYmxlOmZpcnN0LWNoaWxkPnRoZWFkOmZpcnN0
LWNoaWxkPnRyOmZpcnN0LWNoaWxke2JvcmRlci10b3AtbGVmdC1yYWRpdXM6M3B4O2JvcmRlci10
b3AtcmlnaHQtcmFkaXVzOjNweH0ucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU6Zmlyc3QtY2hpbGQ+
LnRhYmxlOmZpcnN0LWNoaWxkPnRib2R5OmZpcnN0LWNoaWxkPnRyOmZpcnN0LWNoaWxkIHRkOmZp
cnN0LWNoaWxkLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZTpmaXJzdC1jaGlsZD4udGFibGU6Zmly
c3QtY2hpbGQ+dGJvZHk6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGg6Zmlyc3QtY2hpbGQs
LnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmZpcnN0LWNoaWxkPi50YWJsZTpmaXJzdC1jaGlsZD50
aGVhZDpmaXJzdC1jaGlsZD50cjpmaXJzdC1jaGlsZCB0ZDpmaXJzdC1jaGlsZCwucGFuZWw+LnRh
YmxlLXJlc3BvbnNpdmU6Zmlyc3QtY2hpbGQ+LnRhYmxlOmZpcnN0LWNoaWxkPnRoZWFkOmZpcnN0
LWNoaWxkPnRyOmZpcnN0LWNoaWxkIHRoOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGU6Zmlyc3Qt
Y2hpbGQ+dGJvZHk6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGQ6Zmlyc3QtY2hpbGQsLnBh
bmVsPi50YWJsZTpmaXJzdC1jaGlsZD50Ym9keTpmaXJzdC1jaGlsZD50cjpmaXJzdC1jaGlsZCB0
aDpmaXJzdC1jaGlsZCwucGFuZWw+LnRhYmxlOmZpcnN0LWNoaWxkPnRoZWFkOmZpcnN0LWNoaWxk
PnRyOmZpcnN0LWNoaWxkIHRkOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGU6Zmlyc3QtY2hpbGQ+
dGhlYWQ6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGg6Zmlyc3QtY2hpbGR7Ym9yZGVyLXRv
cC1sZWZ0LXJhZGl1czozcHh9LnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmZpcnN0LWNoaWxkPi50
YWJsZTpmaXJzdC1jaGlsZD50Ym9keTpmaXJzdC1jaGlsZD50cjpmaXJzdC1jaGlsZCB0ZDpsYXN0
LWNoaWxkLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZTpmaXJzdC1jaGlsZD4udGFibGU6Zmlyc3Qt
Y2hpbGQ+dGJvZHk6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGg6bGFzdC1jaGlsZCwucGFu
ZWw+LnRhYmxlLXJlc3BvbnNpdmU6Zmlyc3QtY2hpbGQ+LnRhYmxlOmZpcnN0LWNoaWxkPnRoZWFk
OmZpcnN0LWNoaWxkPnRyOmZpcnN0LWNoaWxkIHRkOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1y
ZXNwb25zaXZlOmZpcnN0LWNoaWxkPi50YWJsZTpmaXJzdC1jaGlsZD50aGVhZDpmaXJzdC1jaGls
ZD50cjpmaXJzdC1jaGlsZCB0aDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGU6Zmlyc3QtY2hpbGQ+
dGJvZHk6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGQ6bGFzdC1jaGlsZCwucGFuZWw+LnRh
YmxlOmZpcnN0LWNoaWxkPnRib2R5OmZpcnN0LWNoaWxkPnRyOmZpcnN0LWNoaWxkIHRoOmxhc3Qt
Y2hpbGQsLnBhbmVsPi50YWJsZTpmaXJzdC1jaGlsZD50aGVhZDpmaXJzdC1jaGlsZD50cjpmaXJz
dC1jaGlsZCB0ZDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGU6Zmlyc3QtY2hpbGQ+dGhlYWQ6Zmly
c3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGg6bGFzdC1jaGlsZHtib3JkZXItdG9wLXJpZ2h0LXJh
ZGl1czozcHh9LnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3Qt
Y2hpbGQsLnBhbmVsPi50YWJsZTpsYXN0LWNoaWxke2JvcmRlci1ib3R0b20tcmlnaHQtcmFkaXVz
OjNweDtib3JkZXItYm90dG9tLWxlZnQtcmFkaXVzOjNweH0ucGFuZWw+LnRhYmxlLXJlc3BvbnNp
dmU6bGFzdC1jaGlsZD4udGFibGU6bGFzdC1jaGlsZD50Ym9keTpsYXN0LWNoaWxkPnRyOmxhc3Qt
Y2hpbGQsLnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3QtY2hp
bGQ+dGZvb3Q6bGFzdC1jaGlsZD50cjpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGU6bGFzdC1jaGls
ZD50Ym9keTpsYXN0LWNoaWxkPnRyOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZTpsYXN0LWNoaWxk
PnRmb290Omxhc3QtY2hpbGQ+dHI6bGFzdC1jaGlsZHtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1
czozcHg7Ym9yZGVyLWJvdHRvbS1sZWZ0LXJhZGl1czozcHh9LnBhbmVsPi50YWJsZS1yZXNwb25z
aXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3QtY2hpbGQ+dGJvZHk6bGFzdC1jaGlsZD50cjpsYXN0
LWNoaWxkIHRkOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZTpsYXN0LWNoaWxk
Pi50YWJsZTpsYXN0LWNoaWxkPnRib2R5Omxhc3QtY2hpbGQ+dHI6bGFzdC1jaGlsZCB0aDpmaXJz
dC1jaGlsZCwucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU6bGFzdC1jaGlsZD4udGFibGU6bGFzdC1j
aGlsZD50Zm9vdDpsYXN0LWNoaWxkPnRyOmxhc3QtY2hpbGQgdGQ6Zmlyc3QtY2hpbGQsLnBhbmVs
Pi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3QtY2hpbGQ+dGZvb3Q6bGFz
dC1jaGlsZD50cjpsYXN0LWNoaWxkIHRoOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGU6bGFzdC1j
aGlsZD50Ym9keTpsYXN0LWNoaWxkPnRyOmxhc3QtY2hpbGQgdGQ6Zmlyc3QtY2hpbGQsLnBhbmVs
Pi50YWJsZTpsYXN0LWNoaWxkPnRib2R5Omxhc3QtY2hpbGQ+dHI6bGFzdC1jaGlsZCB0aDpmaXJz
dC1jaGlsZCwucGFuZWw+LnRhYmxlOmxhc3QtY2hpbGQ+dGZvb3Q6bGFzdC1jaGlsZD50cjpsYXN0
LWNoaWxkIHRkOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGU6bGFzdC1jaGlsZD50Zm9vdDpsYXN0
LWNoaWxkPnRyOmxhc3QtY2hpbGQgdGg6Zmlyc3QtY2hpbGR7Ym9yZGVyLWJvdHRvbS1sZWZ0LXJh
ZGl1czozcHh9LnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3Qt
Y2hpbGQ+dGJvZHk6bGFzdC1jaGlsZD50cjpsYXN0LWNoaWxkIHRkOmxhc3QtY2hpbGQsLnBhbmVs
Pi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3QtY2hpbGQ+dGJvZHk6bGFz
dC1jaGlsZD50cjpsYXN0LWNoaWxkIHRoOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1yZXNwb25z
aXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3QtY2hpbGQ+dGZvb3Q6bGFzdC1jaGlsZD50cjpsYXN0
LWNoaWxkIHRkOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+
LnRhYmxlOmxhc3QtY2hpbGQ+dGZvb3Q6bGFzdC1jaGlsZD50cjpsYXN0LWNoaWxkIHRoOmxhc3Qt
Y2hpbGQsLnBhbmVsPi50YWJsZTpsYXN0LWNoaWxkPnRib2R5Omxhc3QtY2hpbGQ+dHI6bGFzdC1j
aGlsZCB0ZDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGU6bGFzdC1jaGlsZD50Ym9keTpsYXN0LWNo
aWxkPnRyOmxhc3QtY2hpbGQgdGg6bGFzdC1jaGlsZCwucGFuZWw+LnRhYmxlOmxhc3QtY2hpbGQ+
dGZvb3Q6bGFzdC1jaGlsZD50cjpsYXN0LWNoaWxkIHRkOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJs
ZTpsYXN0LWNoaWxkPnRmb290Omxhc3QtY2hpbGQ+dHI6bGFzdC1jaGlsZCB0aDpsYXN0LWNoaWxk
e2JvcmRlci1ib3R0b20tcmlnaHQtcmFkaXVzOjNweH0ucGFuZWw+LnBhbmVsLWJvZHkrLnRhYmxl
LC5wYW5lbD4ucGFuZWwtYm9keSsudGFibGUtcmVzcG9uc2l2ZSwucGFuZWw+LnRhYmxlKy5wYW5l
bC1ib2R5LC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZSsucGFuZWwtYm9keXtib3JkZXItdG9wOjFw
eCBzb2xpZCAjZGRkfS5wYW5lbD4udGFibGU+dGJvZHk6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hp
bGQgdGQsLnBhbmVsPi50YWJsZT50Ym9keTpmaXJzdC1jaGlsZD50cjpmaXJzdC1jaGlsZCB0aHti
b3JkZXItdG9wOjB9LnBhbmVsPi50YWJsZS1ib3JkZXJlZCwucGFuZWw+LnRhYmxlLXJlc3BvbnNp
dmU+LnRhYmxlLWJvcmRlcmVke2JvcmRlcjowfS5wYW5lbD4udGFibGUtYm9yZGVyZWQ+dGJvZHk+
dHI+dGQ6Zmlyc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1ib3JkZXJlZD50Ym9keT50cj50aDpmaXJz
dC1jaGlsZCwucGFuZWw+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyPnRkOmZpcnN0LWNoaWxkLC5w
YW5lbD4udGFibGUtYm9yZGVyZWQ+dGZvb3Q+dHI+dGg6Zmlyc3QtY2hpbGQsLnBhbmVsPi50YWJs
ZS1ib3JkZXJlZD50aGVhZD50cj50ZDpmaXJzdC1jaGlsZCwucGFuZWw+LnRhYmxlLWJvcmRlcmVk
PnRoZWFkPnRyPnRoOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZT4udGFibGUt
Ym9yZGVyZWQ+dGJvZHk+dHI+dGQ6Zmlyc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1yZXNwb25zaXZl
Pi50YWJsZS1ib3JkZXJlZD50Ym9keT50cj50aDpmaXJzdC1jaGlsZCwucGFuZWw+LnRhYmxlLXJl
c3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyPnRkOmZpcnN0LWNoaWxkLC5wYW5lbD4u
dGFibGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGZvb3Q+dHI+dGg6Zmlyc3QtY2hpbGQs
LnBhbmVsPi50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50aGVhZD50cj50ZDpmaXJz
dC1jaGlsZCwucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRy
PnRoOmZpcnN0LWNoaWxke2JvcmRlci1sZWZ0OjB9LnBhbmVsPi50YWJsZS1ib3JkZXJlZD50Ym9k
eT50cj50ZDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGUtYm9yZGVyZWQ+dGJvZHk+dHI+dGg6bGFz
dC1jaGlsZCwucGFuZWw+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyPnRkOmxhc3QtY2hpbGQsLnBh
bmVsPi50YWJsZS1ib3JkZXJlZD50Zm9vdD50cj50aDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGUt
Ym9yZGVyZWQ+dGhlYWQ+dHI+dGQ6bGFzdC1jaGlsZCwucGFuZWw+LnRhYmxlLWJvcmRlcmVkPnRo
ZWFkPnRyPnRoOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3Jk
ZXJlZD50Ym9keT50cj50ZDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZT4udGFi
bGUtYm9yZGVyZWQ+dGJvZHk+dHI+dGg6bGFzdC1jaGlsZCwucGFuZWw+LnRhYmxlLXJlc3BvbnNp
dmU+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyPnRkOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1y
ZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50Zm9vdD50cj50aDpsYXN0LWNoaWxkLC5wYW5lbD4u
dGFibGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGhlYWQ+dHI+dGQ6bGFzdC1jaGlsZCwu
cGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRyPnRoOmxhc3Qt
Y2hpbGR7Ym9yZGVyLXJpZ2h0OjB9LnBhbmVsPi50YWJsZS1ib3JkZXJlZD50Ym9keT50cjpmaXJz
dC1jaGlsZD50ZCwucGFuZWw+LnRhYmxlLWJvcmRlcmVkPnRib2R5PnRyOmZpcnN0LWNoaWxkPnRo
LC5wYW5lbD4udGFibGUtYm9yZGVyZWQ+dGhlYWQ+dHI6Zmlyc3QtY2hpbGQ+dGQsLnBhbmVsPi50
YWJsZS1ib3JkZXJlZD50aGVhZD50cjpmaXJzdC1jaGlsZD50aCwucGFuZWw+LnRhYmxlLXJlc3Bv
bnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRib2R5PnRyOmZpcnN0LWNoaWxkPnRkLC5wYW5lbD4udGFi
bGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGJvZHk+dHI6Zmlyc3QtY2hpbGQ+dGgsLnBh
bmVsPi50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50aGVhZD50cjpmaXJzdC1jaGls
ZD50ZCwucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRyOmZp
cnN0LWNoaWxkPnRoe2JvcmRlci1ib3R0b206MH0ucGFuZWw+LnRhYmxlLWJvcmRlcmVkPnRib2R5
PnRyOmxhc3QtY2hpbGQ+dGQsLnBhbmVsPi50YWJsZS1ib3JkZXJlZD50Ym9keT50cjpsYXN0LWNo
aWxkPnRoLC5wYW5lbD4udGFibGUtYm9yZGVyZWQ+dGZvb3Q+dHI6bGFzdC1jaGlsZD50ZCwucGFu
ZWw+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyOmxhc3QtY2hpbGQ+dGgsLnBhbmVsPi50YWJsZS1y
ZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50Ym9keT50cjpsYXN0LWNoaWxkPnRkLC5wYW5lbD4u
dGFibGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGJvZHk+dHI6bGFzdC1jaGlsZD50aCwu
cGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyOmxhc3QtY2hp
bGQ+dGQsLnBhbmVsPi50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50Zm9vdD50cjps
YXN0LWNoaWxkPnRoe2JvcmRlci1ib3R0b206MH0ucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmV7bWFy
Z2luLWJvdHRvbTowO2JvcmRlcjowfS5wYW5lbC1ncm91cHttYXJnaW4tYm90dG9tOjIwcHh9LnBh
bmVsLWdyb3VwIC5wYW5lbHttYXJnaW4tYm90dG9tOjA7Ym9yZGVyLXJhZGl1czo0cHh9LnBhbmVs
LWdyb3VwIC5wYW5lbCsucGFuZWx7bWFyZ2luLXRvcDo1cHh9LnBhbmVsLWdyb3VwIC5wYW5lbC1o
ZWFkaW5ne2JvcmRlci1ib3R0b206MH0ucGFuZWwtZ3JvdXAgLnBhbmVsLWhlYWRpbmcrLnBhbmVs
LWNvbGxhcHNlPi5saXN0LWdyb3VwLC5wYW5lbC1ncm91cCAucGFuZWwtaGVhZGluZysucGFuZWwt
Y29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9yZGVyLXRvcDoxcHggc29saWQgI2RkZH0ucGFuZWwtZ3Jv
dXAgLnBhbmVsLWZvb3Rlcntib3JkZXItdG9wOjB9LnBhbmVsLWdyb3VwIC5wYW5lbC1mb290ZXIr
LnBhbmVsLWNvbGxhcHNlIC5wYW5lbC1ib2R5e2JvcmRlci1ib3R0b206MXB4IHNvbGlkICNkZGR9
LnBhbmVsLWRlZmF1bHR7Ym9yZGVyLWNvbG9yOiNkZGR9LnBhbmVsLWRlZmF1bHQ+LnBhbmVsLWhl
YWRpbmd7Y29sb3I6IzMzMztiYWNrZ3JvdW5kLWNvbG9yOiNmNWY1ZjU7Ym9yZGVyLWNvbG9yOiNk
ZGR9LnBhbmVsLWRlZmF1bHQ+LnBhbmVsLWhlYWRpbmcrLnBhbmVsLWNvbGxhcHNlPi5wYW5lbC1i
b2R5e2JvcmRlci10b3AtY29sb3I6I2RkZH0ucGFuZWwtZGVmYXVsdD4ucGFuZWwtaGVhZGluZyAu
YmFkZ2V7Y29sb3I6I2Y1ZjVmNTtiYWNrZ3JvdW5kLWNvbG9yOiMzMzN9LnBhbmVsLWRlZmF1bHQ+
LnBhbmVsLWZvb3RlcisucGFuZWwtY29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9yZGVyLWJvdHRvbS1j
b2xvcjojZGRkfS5wYW5lbC1wcmltYXJ5e2JvcmRlci1jb2xvcjojMzM3YWI3fS5wYW5lbC1wcmlt
YXJ5Pi5wYW5lbC1oZWFkaW5ne2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojMzM3YWI3O2Jv
cmRlci1jb2xvcjojMzM3YWI3fS5wYW5lbC1wcmltYXJ5Pi5wYW5lbC1oZWFkaW5nKy5wYW5lbC1j
b2xsYXBzZT4ucGFuZWwtYm9keXtib3JkZXItdG9wLWNvbG9yOiMzMzdhYjd9LnBhbmVsLXByaW1h
cnk+LnBhbmVsLWhlYWRpbmcgLmJhZGdle2NvbG9yOiMzMzdhYjc7YmFja2dyb3VuZC1jb2xvcjoj
ZmZmfS5wYW5lbC1wcmltYXJ5Pi5wYW5lbC1mb290ZXIrLnBhbmVsLWNvbGxhcHNlPi5wYW5lbC1i
b2R5e2JvcmRlci1ib3R0b20tY29sb3I6IzMzN2FiN30ucGFuZWwtc3VjY2Vzc3tib3JkZXItY29s
b3I6I2Q2ZTljNn0ucGFuZWwtc3VjY2Vzcz4ucGFuZWwtaGVhZGluZ3tjb2xvcjojM2M3NjNkO2Jh
Y2tncm91bmQtY29sb3I6I2RmZjBkODtib3JkZXItY29sb3I6I2Q2ZTljNn0ucGFuZWwtc3VjY2Vz
cz4ucGFuZWwtaGVhZGluZysucGFuZWwtY29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9yZGVyLXRvcC1j
b2xvcjojZDZlOWM2fS5wYW5lbC1zdWNjZXNzPi5wYW5lbC1oZWFkaW5nIC5iYWRnZXtjb2xvcjoj
ZGZmMGQ4O2JhY2tncm91bmQtY29sb3I6IzNjNzYzZH0ucGFuZWwtc3VjY2Vzcz4ucGFuZWwtZm9v
dGVyKy5wYW5lbC1jb2xsYXBzZT4ucGFuZWwtYm9keXtib3JkZXItYm90dG9tLWNvbG9yOiNkNmU5
YzZ9LnBhbmVsLWluZm97Ym9yZGVyLWNvbG9yOiNiY2U4ZjF9LnBhbmVsLWluZm8+LnBhbmVsLWhl
YWRpbmd7Y29sb3I6IzMxNzA4ZjtiYWNrZ3JvdW5kLWNvbG9yOiNkOWVkZjc7Ym9yZGVyLWNvbG9y
OiNiY2U4ZjF9LnBhbmVsLWluZm8+LnBhbmVsLWhlYWRpbmcrLnBhbmVsLWNvbGxhcHNlPi5wYW5l
bC1ib2R5e2JvcmRlci10b3AtY29sb3I6I2JjZThmMX0ucGFuZWwtaW5mbz4ucGFuZWwtaGVhZGlu
ZyAuYmFkZ2V7Y29sb3I6I2Q5ZWRmNztiYWNrZ3JvdW5kLWNvbG9yOiMzMTcwOGZ9LnBhbmVsLWlu
Zm8+LnBhbmVsLWZvb3RlcisucGFuZWwtY29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9yZGVyLWJvdHRv
bS1jb2xvcjojYmNlOGYxfS5wYW5lbC13YXJuaW5ne2JvcmRlci1jb2xvcjojZmFlYmNjfS5wYW5l
bC13YXJuaW5nPi5wYW5lbC1oZWFkaW5ne2NvbG9yOiM4YTZkM2I7YmFja2dyb3VuZC1jb2xvcjoj
ZmNmOGUzO2JvcmRlci1jb2xvcjojZmFlYmNjfS5wYW5lbC13YXJuaW5nPi5wYW5lbC1oZWFkaW5n
Ky5wYW5lbC1jb2xsYXBzZT4ucGFuZWwtYm9keXtib3JkZXItdG9wLWNvbG9yOiNmYWViY2N9LnBh
bmVsLXdhcm5pbmc+LnBhbmVsLWhlYWRpbmcgLmJhZGdle2NvbG9yOiNmY2Y4ZTM7YmFja2dyb3Vu
ZC1jb2xvcjojOGE2ZDNifS5wYW5lbC13YXJuaW5nPi5wYW5lbC1mb290ZXIrLnBhbmVsLWNvbGxh
cHNlPi5wYW5lbC1ib2R5e2JvcmRlci1ib3R0b20tY29sb3I6I2ZhZWJjY30ucGFuZWwtZGFuZ2Vy
e2JvcmRlci1jb2xvcjojZWJjY2QxfS5wYW5lbC1kYW5nZXI+LnBhbmVsLWhlYWRpbmd7Y29sb3I6
I2E5NDQ0MjtiYWNrZ3JvdW5kLWNvbG9yOiNmMmRlZGU7Ym9yZGVyLWNvbG9yOiNlYmNjZDF9LnBh
bmVsLWRhbmdlcj4ucGFuZWwtaGVhZGluZysucGFuZWwtY29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9y
ZGVyLXRvcC1jb2xvcjojZWJjY2QxfS5wYW5lbC1kYW5nZXI+LnBhbmVsLWhlYWRpbmcgLmJhZGdl
e2NvbG9yOiNmMmRlZGU7YmFja2dyb3VuZC1jb2xvcjojYTk0NDQyfS5wYW5lbC1kYW5nZXI+LnBh
bmVsLWZvb3RlcisucGFuZWwtY29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9yZGVyLWJvdHRvbS1jb2xv
cjojZWJjY2QxfS5lbWJlZC1yZXNwb25zaXZle3Bvc2l0aW9uOnJlbGF0aXZlO2Rpc3BsYXk6Ymxv
Y2s7aGVpZ2h0OjA7cGFkZGluZzowO292ZXJmbG93OmhpZGRlbn0uZW1iZWQtcmVzcG9uc2l2ZSAu
ZW1iZWQtcmVzcG9uc2l2ZS1pdGVtLC5lbWJlZC1yZXNwb25zaXZlIGVtYmVkLC5lbWJlZC1yZXNw
b25zaXZlIGlmcmFtZSwuZW1iZWQtcmVzcG9uc2l2ZSBvYmplY3QsLmVtYmVkLXJlc3BvbnNpdmUg
dmlkZW97cG9zaXRpb246YWJzb2x1dGU7dG9wOjA7Ym90dG9tOjA7bGVmdDowO3dpZHRoOjEwMCU7
aGVpZ2h0OjEwMCU7Ym9yZGVyOjB9LmVtYmVkLXJlc3BvbnNpdmUtMTZieTl7cGFkZGluZy1ib3R0
b206NTYuMjUlfS5lbWJlZC1yZXNwb25zaXZlLTRieTN7cGFkZGluZy1ib3R0b206NzUlfS53ZWxs
e21pbi1oZWlnaHQ6MjBweDtwYWRkaW5nOjE5cHg7bWFyZ2luLWJvdHRvbToyMHB4O2JhY2tncm91
bmQtY29sb3I6I2Y1ZjVmNTtib3JkZXI6MXB4IHNvbGlkICNlM2UzZTM7Ym9yZGVyLXJhZGl1czo0
cHg7LXdlYmtpdC1ib3gtc2hhZG93Omluc2V0IDAgMXB4IDFweCByZ2JhKDAsMCwwLC4wNSk7Ym94
LXNoYWRvdzppbnNldCAwIDFweCAxcHggcmdiYSgwLDAsMCwuMDUpfS53ZWxsIGJsb2NrcXVvdGV7
Ym9yZGVyLWNvbG9yOiNkZGQ7Ym9yZGVyLWNvbG9yOnJnYmEoMCwwLDAsLjE1KX0ud2VsbC1sZ3tw
YWRkaW5nOjI0cHg7Ym9yZGVyLXJhZGl1czo2cHh9LndlbGwtc217cGFkZGluZzo5cHg7Ym9yZGVy
LXJhZGl1czozcHh9LmNsb3Nle2Zsb2F0OnJpZ2h0O2ZvbnQtc2l6ZToyMXB4O2ZvbnQtd2VpZ2h0
OjcwMDtsaW5lLWhlaWdodDoxO2NvbG9yOiMwMDA7dGV4dC1zaGFkb3c6MCAxcHggMCAjZmZmO2Zp
bHRlcjphbHBoYShvcGFjaXR5PTIwKTtvcGFjaXR5Oi4yfS5jbG9zZTpmb2N1cywuY2xvc2U6aG92
ZXJ7Y29sb3I6IzAwMDt0ZXh0LWRlY29yYXRpb246bm9uZTtjdXJzb3I6cG9pbnRlcjtmaWx0ZXI6
YWxwaGEob3BhY2l0eT01MCk7b3BhY2l0eTouNX1idXR0b24uY2xvc2V7LXdlYmtpdC1hcHBlYXJh
bmNlOm5vbmU7cGFkZGluZzowO2N1cnNvcjpwb2ludGVyO2JhY2tncm91bmQ6MCAwO2JvcmRlcjow
fS5tb2RhbC1vcGVue292ZXJmbG93OmhpZGRlbn0ubW9kYWx7cG9zaXRpb246Zml4ZWQ7dG9wOjA7
cmlnaHQ6MDtib3R0b206MDtsZWZ0OjA7ei1pbmRleDoxMDUwO2Rpc3BsYXk6bm9uZTtvdmVyZmxv
dzpoaWRkZW47LXdlYmtpdC1vdmVyZmxvdy1zY3JvbGxpbmc6dG91Y2g7b3V0bGluZTowfS5tb2Rh
bC5mYWRlIC5tb2RhbC1kaWFsb2d7LXdlYmtpdC10cmFuc2l0aW9uOi13ZWJraXQtdHJhbnNmb3Jt
IC4zcyBlYXNlLW91dDstby10cmFuc2l0aW9uOi1vLXRyYW5zZm9ybSAuM3MgZWFzZS1vdXQ7dHJh
bnNpdGlvbjp0cmFuc2Zvcm0gLjNzIGVhc2Utb3V0Oy13ZWJraXQtdHJhbnNmb3JtOnRyYW5zbGF0
ZSgwLC0yNSUpOy1tcy10cmFuc2Zvcm06dHJhbnNsYXRlKDAsLTI1JSk7LW8tdHJhbnNmb3JtOnRy
YW5zbGF0ZSgwLC0yNSUpO3RyYW5zZm9ybTp0cmFuc2xhdGUoMCwtMjUlKX0ubW9kYWwuaW4gLm1v
ZGFsLWRpYWxvZ3std2Via2l0LXRyYW5zZm9ybTp0cmFuc2xhdGUoMCwwKTstbXMtdHJhbnNmb3Jt
OnRyYW5zbGF0ZSgwLDApOy1vLXRyYW5zZm9ybTp0cmFuc2xhdGUoMCwwKTt0cmFuc2Zvcm06dHJh
bnNsYXRlKDAsMCl9Lm1vZGFsLW9wZW4gLm1vZGFse292ZXJmbG93LXg6aGlkZGVuO292ZXJmbG93
LXk6YXV0b30ubW9kYWwtZGlhbG9ne3Bvc2l0aW9uOnJlbGF0aXZlO3dpZHRoOmF1dG87bWFyZ2lu
OjEwcHh9Lm1vZGFsLWNvbnRlbnR7cG9zaXRpb246cmVsYXRpdmU7YmFja2dyb3VuZC1jb2xvcjoj
ZmZmOy13ZWJraXQtYmFja2dyb3VuZC1jbGlwOnBhZGRpbmctYm94O2JhY2tncm91bmQtY2xpcDpw
YWRkaW5nLWJveDtib3JkZXI6MXB4IHNvbGlkICM5OTk7Ym9yZGVyOjFweCBzb2xpZCByZ2JhKDAs
MCwwLC4yKTtib3JkZXItcmFkaXVzOjZweDtvdXRsaW5lOjA7LXdlYmtpdC1ib3gtc2hhZG93OjAg
M3B4IDlweCByZ2JhKDAsMCwwLC41KTtib3gtc2hhZG93OjAgM3B4IDlweCByZ2JhKDAsMCwwLC41
KX0ubW9kYWwtYmFja2Ryb3B7cG9zaXRpb246Zml4ZWQ7dG9wOjA7cmlnaHQ6MDtib3R0b206MDts
ZWZ0OjA7ei1pbmRleDoxMDQwO2JhY2tncm91bmQtY29sb3I6IzAwMH0ubW9kYWwtYmFja2Ryb3Au
ZmFkZXtmaWx0ZXI6YWxwaGEob3BhY2l0eT0wKTtvcGFjaXR5OjB9Lm1vZGFsLWJhY2tkcm9wLmlu
e2ZpbHRlcjphbHBoYShvcGFjaXR5PTUwKTtvcGFjaXR5Oi41fS5tb2RhbC1oZWFkZXJ7cGFkZGlu
ZzoxNXB4O2JvcmRlci1ib3R0b206MXB4IHNvbGlkICNlNWU1ZTV9Lm1vZGFsLWhlYWRlciAuY2xv
c2V7bWFyZ2luLXRvcDotMnB4fS5tb2RhbC10aXRsZXttYXJnaW46MDtsaW5lLWhlaWdodDoxLjQy
ODU3MTQzfS5tb2RhbC1ib2R5e3Bvc2l0aW9uOnJlbGF0aXZlO3BhZGRpbmc6MTVweH0ubW9kYWwt
Zm9vdGVye3BhZGRpbmc6MTVweDt0ZXh0LWFsaWduOnJpZ2h0O2JvcmRlci10b3A6MXB4IHNvbGlk
ICNlNWU1ZTV9Lm1vZGFsLWZvb3RlciAuYnRuKy5idG57bWFyZ2luLWJvdHRvbTowO21hcmdpbi1s
ZWZ0OjVweH0ubW9kYWwtZm9vdGVyIC5idG4tZ3JvdXAgLmJ0bisuYnRue21hcmdpbi1sZWZ0Oi0x
cHh9Lm1vZGFsLWZvb3RlciAuYnRuLWJsb2NrKy5idG4tYmxvY2t7bWFyZ2luLWxlZnQ6MH0ubW9k
YWwtc2Nyb2xsYmFyLW1lYXN1cmV7cG9zaXRpb246YWJzb2x1dGU7dG9wOi05OTk5cHg7d2lkdGg6
NTBweDtoZWlnaHQ6NTBweDtvdmVyZmxvdzpzY3JvbGx9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgp
ey5tb2RhbC1kaWFsb2d7d2lkdGg6NjAwcHg7bWFyZ2luOjMwcHggYXV0b30ubW9kYWwtY29udGVu
dHstd2Via2l0LWJveC1zaGFkb3c6MCA1cHggMTVweCByZ2JhKDAsMCwwLC41KTtib3gtc2hhZG93
OjAgNXB4IDE1cHggcmdiYSgwLDAsMCwuNSl9Lm1vZGFsLXNte3dpZHRoOjMwMHB4fX1AbWVkaWEg
KG1pbi13aWR0aDo5OTJweCl7Lm1vZGFsLWxne3dpZHRoOjkwMHB4fX0udG9vbHRpcHtwb3NpdGlv
bjphYnNvbHV0ZTt6LWluZGV4OjEwNzA7ZGlzcGxheTpibG9jaztmb250LWZhbWlseToiSGVsdmV0
aWNhIE5ldWUiLEhlbHZldGljYSxBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxMnB4O2ZvbnQt
c3R5bGU6bm9ybWFsO2ZvbnQtd2VpZ2h0OjQwMDtsaW5lLWhlaWdodDoxLjQyODU3MTQzO3RleHQt
YWxpZ246bGVmdDt0ZXh0LWFsaWduOnN0YXJ0O3RleHQtZGVjb3JhdGlvbjpub25lO3RleHQtc2hh
ZG93Om5vbmU7dGV4dC10cmFuc2Zvcm06bm9uZTtsZXR0ZXItc3BhY2luZzpub3JtYWw7d29yZC1i
cmVhazpub3JtYWw7d29yZC1zcGFjaW5nOm5vcm1hbDt3b3JkLXdyYXA6bm9ybWFsO3doaXRlLXNw
YWNlOm5vcm1hbDtmaWx0ZXI6YWxwaGEob3BhY2l0eT0wKTtvcGFjaXR5OjA7bGluZS1icmVhazph
dXRvfS50b29sdGlwLmlue2ZpbHRlcjphbHBoYShvcGFjaXR5PTkwKTtvcGFjaXR5Oi45fS50b29s
dGlwLnRvcHtwYWRkaW5nOjVweCAwO21hcmdpbi10b3A6LTNweH0udG9vbHRpcC5yaWdodHtwYWRk
aW5nOjAgNXB4O21hcmdpbi1sZWZ0OjNweH0udG9vbHRpcC5ib3R0b217cGFkZGluZzo1cHggMDtt
YXJnaW4tdG9wOjNweH0udG9vbHRpcC5sZWZ0e3BhZGRpbmc6MCA1cHg7bWFyZ2luLWxlZnQ6LTNw
eH0udG9vbHRpcC1pbm5lcnttYXgtd2lkdGg6MjAwcHg7cGFkZGluZzozcHggOHB4O2NvbG9yOiNm
ZmY7dGV4dC1hbGlnbjpjZW50ZXI7YmFja2dyb3VuZC1jb2xvcjojMDAwO2JvcmRlci1yYWRpdXM6
NHB4fS50b29sdGlwLWFycm93e3Bvc2l0aW9uOmFic29sdXRlO3dpZHRoOjA7aGVpZ2h0OjA7Ym9y
ZGVyLWNvbG9yOnRyYW5zcGFyZW50O2JvcmRlci1zdHlsZTpzb2xpZH0udG9vbHRpcC50b3AgLnRv
b2x0aXAtYXJyb3d7Ym90dG9tOjA7bGVmdDo1MCU7bWFyZ2luLWxlZnQ6LTVweDtib3JkZXItd2lk
dGg6NXB4IDVweCAwO2JvcmRlci10b3AtY29sb3I6IzAwMH0udG9vbHRpcC50b3AtbGVmdCAudG9v
bHRpcC1hcnJvd3tyaWdodDo1cHg7Ym90dG9tOjA7bWFyZ2luLWJvdHRvbTotNXB4O2JvcmRlci13
aWR0aDo1cHggNXB4IDA7Ym9yZGVyLXRvcC1jb2xvcjojMDAwfS50b29sdGlwLnRvcC1yaWdodCAu
dG9vbHRpcC1hcnJvd3tib3R0b206MDtsZWZ0OjVweDttYXJnaW4tYm90dG9tOi01cHg7Ym9yZGVy
LXdpZHRoOjVweCA1cHggMDtib3JkZXItdG9wLWNvbG9yOiMwMDB9LnRvb2x0aXAucmlnaHQgLnRv
b2x0aXAtYXJyb3d7dG9wOjUwJTtsZWZ0OjA7bWFyZ2luLXRvcDotNXB4O2JvcmRlci13aWR0aDo1
cHggNXB4IDVweCAwO2JvcmRlci1yaWdodC1jb2xvcjojMDAwfS50b29sdGlwLmxlZnQgLnRvb2x0
aXAtYXJyb3d7dG9wOjUwJTtyaWdodDowO21hcmdpbi10b3A6LTVweDtib3JkZXItd2lkdGg6NXB4
IDAgNXB4IDVweDtib3JkZXItbGVmdC1jb2xvcjojMDAwfS50b29sdGlwLmJvdHRvbSAudG9vbHRp
cC1hcnJvd3t0b3A6MDtsZWZ0OjUwJTttYXJnaW4tbGVmdDotNXB4O2JvcmRlci13aWR0aDowIDVw
eCA1cHg7Ym9yZGVyLWJvdHRvbS1jb2xvcjojMDAwfS50b29sdGlwLmJvdHRvbS1sZWZ0IC50b29s
dGlwLWFycm93e3RvcDowO3JpZ2h0OjVweDttYXJnaW4tdG9wOi01cHg7Ym9yZGVyLXdpZHRoOjAg
NXB4IDVweDtib3JkZXItYm90dG9tLWNvbG9yOiMwMDB9LnRvb2x0aXAuYm90dG9tLXJpZ2h0IC50
b29sdGlwLWFycm93e3RvcDowO2xlZnQ6NXB4O21hcmdpbi10b3A6LTVweDtib3JkZXItd2lkdGg6
MCA1cHggNXB4O2JvcmRlci1ib3R0b20tY29sb3I6IzAwMH0ucG9wb3Zlcntwb3NpdGlvbjphYnNv
bHV0ZTt0b3A6MDtsZWZ0OjA7ei1pbmRleDoxMDYwO2Rpc3BsYXk6bm9uZTttYXgtd2lkdGg6Mjc2
cHg7cGFkZGluZzoxcHg7Zm9udC1mYW1pbHk6IkhlbHZldGljYSBOZXVlIixIZWx2ZXRpY2EsQXJp
YWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtmb250LXN0eWxlOm5vcm1hbDtmb250LXdlaWdo
dDo0MDA7bGluZS1oZWlnaHQ6MS40Mjg1NzE0Mzt0ZXh0LWFsaWduOmxlZnQ7dGV4dC1hbGlnbjpz
dGFydDt0ZXh0LWRlY29yYXRpb246bm9uZTt0ZXh0LXNoYWRvdzpub25lO3RleHQtdHJhbnNmb3Jt
Om5vbmU7bGV0dGVyLXNwYWNpbmc6bm9ybWFsO3dvcmQtYnJlYWs6bm9ybWFsO3dvcmQtc3BhY2lu
Zzpub3JtYWw7d29yZC13cmFwOm5vcm1hbDt3aGl0ZS1zcGFjZTpub3JtYWw7YmFja2dyb3VuZC1j
b2xvcjojZmZmOy13ZWJraXQtYmFja2dyb3VuZC1jbGlwOnBhZGRpbmctYm94O2JhY2tncm91bmQt
Y2xpcDpwYWRkaW5nLWJveDtib3JkZXI6MXB4IHNvbGlkICNjY2M7Ym9yZGVyOjFweCBzb2xpZCBy
Z2JhKDAsMCwwLC4yKTtib3JkZXItcmFkaXVzOjZweDstd2Via2l0LWJveC1zaGFkb3c6MCA1cHgg
MTBweCByZ2JhKDAsMCwwLC4yKTtib3gtc2hhZG93OjAgNXB4IDEwcHggcmdiYSgwLDAsMCwuMik7
bGluZS1icmVhazphdXRvfS5wb3BvdmVyLnRvcHttYXJnaW4tdG9wOi0xMHB4fS5wb3BvdmVyLnJp
Z2h0e21hcmdpbi1sZWZ0OjEwcHh9LnBvcG92ZXIuYm90dG9te21hcmdpbi10b3A6MTBweH0ucG9w
b3Zlci5sZWZ0e21hcmdpbi1sZWZ0Oi0xMHB4fS5wb3BvdmVyLXRpdGxle3BhZGRpbmc6OHB4IDE0
cHg7bWFyZ2luOjA7Zm9udC1zaXplOjE0cHg7YmFja2dyb3VuZC1jb2xvcjojZjdmN2Y3O2JvcmRl
ci1ib3R0b206MXB4IHNvbGlkICNlYmViZWI7Ym9yZGVyLXJhZGl1czo1cHggNXB4IDAgMH0ucG9w
b3Zlci1jb250ZW50e3BhZGRpbmc6OXB4IDE0cHh9LnBvcG92ZXI+LmFycm93LC5wb3BvdmVyPi5h
cnJvdzphZnRlcntwb3NpdGlvbjphYnNvbHV0ZTtkaXNwbGF5OmJsb2NrO3dpZHRoOjA7aGVpZ2h0
OjA7Ym9yZGVyLWNvbG9yOnRyYW5zcGFyZW50O2JvcmRlci1zdHlsZTpzb2xpZH0ucG9wb3Zlcj4u
YXJyb3d7Ym9yZGVyLXdpZHRoOjExcHh9LnBvcG92ZXI+LmFycm93OmFmdGVye2NvbnRlbnQ6IiI7
Ym9yZGVyLXdpZHRoOjEwcHh9LnBvcG92ZXIudG9wPi5hcnJvd3tib3R0b206LTExcHg7bGVmdDo1
MCU7bWFyZ2luLWxlZnQ6LTExcHg7Ym9yZGVyLXRvcC1jb2xvcjojOTk5O2JvcmRlci10b3AtY29s
b3I6cmdiYSgwLDAsMCwuMjUpO2JvcmRlci1ib3R0b20td2lkdGg6MH0ucG9wb3Zlci50b3A+LmFy
cm93OmFmdGVye2JvdHRvbToxcHg7bWFyZ2luLWxlZnQ6LTEwcHg7Y29udGVudDoiICI7Ym9yZGVy
LXRvcC1jb2xvcjojZmZmO2JvcmRlci1ib3R0b20td2lkdGg6MH0ucG9wb3Zlci5yaWdodD4uYXJy
b3d7dG9wOjUwJTtsZWZ0Oi0xMXB4O21hcmdpbi10b3A6LTExcHg7Ym9yZGVyLXJpZ2h0LWNvbG9y
OiM5OTk7Ym9yZGVyLXJpZ2h0LWNvbG9yOnJnYmEoMCwwLDAsLjI1KTtib3JkZXItbGVmdC13aWR0
aDowfS5wb3BvdmVyLnJpZ2h0Pi5hcnJvdzphZnRlcntib3R0b206LTEwcHg7bGVmdDoxcHg7Y29u
dGVudDoiICI7Ym9yZGVyLXJpZ2h0LWNvbG9yOiNmZmY7Ym9yZGVyLWxlZnQtd2lkdGg6MH0ucG9w
b3Zlci5ib3R0b20+LmFycm93e3RvcDotMTFweDtsZWZ0OjUwJTttYXJnaW4tbGVmdDotMTFweDti
b3JkZXItdG9wLXdpZHRoOjA7Ym9yZGVyLWJvdHRvbS1jb2xvcjojOTk5O2JvcmRlci1ib3R0b20t
Y29sb3I6cmdiYSgwLDAsMCwuMjUpfS5wb3BvdmVyLmJvdHRvbT4uYXJyb3c6YWZ0ZXJ7dG9wOjFw
eDttYXJnaW4tbGVmdDotMTBweDtjb250ZW50OiIgIjtib3JkZXItdG9wLXdpZHRoOjA7Ym9yZGVy
LWJvdHRvbS1jb2xvcjojZmZmfS5wb3BvdmVyLmxlZnQ+LmFycm93e3RvcDo1MCU7cmlnaHQ6LTEx
cHg7bWFyZ2luLXRvcDotMTFweDtib3JkZXItcmlnaHQtd2lkdGg6MDtib3JkZXItbGVmdC1jb2xv
cjojOTk5O2JvcmRlci1sZWZ0LWNvbG9yOnJnYmEoMCwwLDAsLjI1KX0ucG9wb3Zlci5sZWZ0Pi5h
cnJvdzphZnRlcntyaWdodDoxcHg7Ym90dG9tOi0xMHB4O2NvbnRlbnQ6IiAiO2JvcmRlci1yaWdo
dC13aWR0aDowO2JvcmRlci1sZWZ0LWNvbG9yOiNmZmZ9LmNhcm91c2Vse3Bvc2l0aW9uOnJlbGF0
aXZlfS5jYXJvdXNlbC1pbm5lcntwb3NpdGlvbjpyZWxhdGl2ZTt3aWR0aDoxMDAlO292ZXJmbG93
OmhpZGRlbn0uY2Fyb3VzZWwtaW5uZXI+Lml0ZW17cG9zaXRpb246cmVsYXRpdmU7ZGlzcGxheTpu
b25lOy13ZWJraXQtdHJhbnNpdGlvbjouNnMgZWFzZS1pbi1vdXQgbGVmdDstby10cmFuc2l0aW9u
Oi42cyBlYXNlLWluLW91dCBsZWZ0O3RyYW5zaXRpb246LjZzIGVhc2UtaW4tb3V0IGxlZnR9LmNh
cm91c2VsLWlubmVyPi5pdGVtPmE+aW1nLC5jYXJvdXNlbC1pbm5lcj4uaXRlbT5pbWd7bGluZS1o
ZWlnaHQ6MX1AbWVkaWEgYWxsIGFuZCAodHJhbnNmb3JtLTNkKSwoLXdlYmtpdC10cmFuc2Zvcm0t
M2Qpey5jYXJvdXNlbC1pbm5lcj4uaXRlbXstd2Via2l0LXRyYW5zaXRpb246LXdlYmtpdC10cmFu
c2Zvcm0gLjZzIGVhc2UtaW4tb3V0Oy1vLXRyYW5zaXRpb246LW8tdHJhbnNmb3JtIC42cyBlYXNl
LWluLW91dDt0cmFuc2l0aW9uOnRyYW5zZm9ybSAuNnMgZWFzZS1pbi1vdXQ7LXdlYmtpdC1iYWNr
ZmFjZS12aXNpYmlsaXR5OmhpZGRlbjtiYWNrZmFjZS12aXNpYmlsaXR5OmhpZGRlbjstd2Via2l0
LXBlcnNwZWN0aXZlOjEwMDBweDtwZXJzcGVjdGl2ZToxMDAwcHh9LmNhcm91c2VsLWlubmVyPi5p
dGVtLmFjdGl2ZS5yaWdodCwuY2Fyb3VzZWwtaW5uZXI+Lml0ZW0ubmV4dHtsZWZ0OjA7LXdlYmtp
dC10cmFuc2Zvcm06dHJhbnNsYXRlM2QoMTAwJSwwLDApO3RyYW5zZm9ybTp0cmFuc2xhdGUzZCgx
MDAlLDAsMCl9LmNhcm91c2VsLWlubmVyPi5pdGVtLmFjdGl2ZS5sZWZ0LC5jYXJvdXNlbC1pbm5l
cj4uaXRlbS5wcmV2e2xlZnQ6MDstd2Via2l0LXRyYW5zZm9ybTp0cmFuc2xhdGUzZCgtMTAwJSww
LDApO3RyYW5zZm9ybTp0cmFuc2xhdGUzZCgtMTAwJSwwLDApfS5jYXJvdXNlbC1pbm5lcj4uaXRl
bS5hY3RpdmUsLmNhcm91c2VsLWlubmVyPi5pdGVtLm5leHQubGVmdCwuY2Fyb3VzZWwtaW5uZXI+
Lml0ZW0ucHJldi5yaWdodHtsZWZ0OjA7LXdlYmtpdC10cmFuc2Zvcm06dHJhbnNsYXRlM2QoMCww
LDApO3RyYW5zZm9ybTp0cmFuc2xhdGUzZCgwLDAsMCl9fS5jYXJvdXNlbC1pbm5lcj4uYWN0aXZl
LC5jYXJvdXNlbC1pbm5lcj4ubmV4dCwuY2Fyb3VzZWwtaW5uZXI+LnByZXZ7ZGlzcGxheTpibG9j
a30uY2Fyb3VzZWwtaW5uZXI+LmFjdGl2ZXtsZWZ0OjB9LmNhcm91c2VsLWlubmVyPi5uZXh0LC5j
YXJvdXNlbC1pbm5lcj4ucHJldntwb3NpdGlvbjphYnNvbHV0ZTt0b3A6MDt3aWR0aDoxMDAlfS5j
YXJvdXNlbC1pbm5lcj4ubmV4dHtsZWZ0OjEwMCV9LmNhcm91c2VsLWlubmVyPi5wcmV2e2xlZnQ6
LTEwMCV9LmNhcm91c2VsLWlubmVyPi5uZXh0LmxlZnQsLmNhcm91c2VsLWlubmVyPi5wcmV2LnJp
Z2h0e2xlZnQ6MH0uY2Fyb3VzZWwtaW5uZXI+LmFjdGl2ZS5sZWZ0e2xlZnQ6LTEwMCV9LmNhcm91
c2VsLWlubmVyPi5hY3RpdmUucmlnaHR7bGVmdDoxMDAlfS5jYXJvdXNlbC1jb250cm9se3Bvc2l0
aW9uOmFic29sdXRlO3RvcDowO2JvdHRvbTowO2xlZnQ6MDt3aWR0aDoxNSU7Zm9udC1zaXplOjIw
cHg7Y29sb3I6I2ZmZjt0ZXh0LWFsaWduOmNlbnRlcjt0ZXh0LXNoYWRvdzowIDFweCAycHggcmdi
YSgwLDAsMCwuNik7YmFja2dyb3VuZC1jb2xvcjpyZ2JhKDAsMCwwLDApO2ZpbHRlcjphbHBoYShv
cGFjaXR5PTUwKTtvcGFjaXR5Oi41fS5jYXJvdXNlbC1jb250cm9sLmxlZnR7YmFja2dyb3VuZC1p
bWFnZTotd2Via2l0LWxpbmVhci1ncmFkaWVudChsZWZ0LHJnYmEoMCwwLDAsLjUpIDAscmdiYSgw
LDAsMCwuMDAwMSkgMTAwJSk7YmFja2dyb3VuZC1pbWFnZTotby1saW5lYXItZ3JhZGllbnQobGVm
dCxyZ2JhKDAsMCwwLC41KSAwLHJnYmEoMCwwLDAsLjAwMDEpIDEwMCUpO2JhY2tncm91bmQtaW1h
Z2U6LXdlYmtpdC1ncmFkaWVudChsaW5lYXIsbGVmdCB0b3AscmlnaHQgdG9wLGZyb20ocmdiYSgw
LDAsMCwuNSkpLHRvKHJnYmEoMCwwLDAsLjAwMDEpKSk7YmFja2dyb3VuZC1pbWFnZTpsaW5lYXIt
Z3JhZGllbnQodG8gcmlnaHQscmdiYSgwLDAsMCwuNSkgMCxyZ2JhKDAsMCwwLC4wMDAxKSAxMDAl
KTtmaWx0ZXI6cHJvZ2lkOkRYSW1hZ2VUcmFuc2Zvcm0uTWljcm9zb2Z0LmdyYWRpZW50KHN0YXJ0
Q29sb3JzdHI9JyM4MDAwMDAwMCcsIGVuZENvbG9yc3RyPScjMDAwMDAwMDAnLCBHcmFkaWVudFR5
cGU9MSk7YmFja2dyb3VuZC1yZXBlYXQ6cmVwZWF0LXh9LmNhcm91c2VsLWNvbnRyb2wucmlnaHR7
cmlnaHQ6MDtsZWZ0OmF1dG87YmFja2dyb3VuZC1pbWFnZTotd2Via2l0LWxpbmVhci1ncmFkaWVu
dChsZWZ0LHJnYmEoMCwwLDAsLjAwMDEpIDAscmdiYSgwLDAsMCwuNSkgMTAwJSk7YmFja2dyb3Vu
ZC1pbWFnZTotby1saW5lYXItZ3JhZGllbnQobGVmdCxyZ2JhKDAsMCwwLC4wMDAxKSAwLHJnYmEo
MCwwLDAsLjUpIDEwMCUpO2JhY2tncm91bmQtaW1hZ2U6LXdlYmtpdC1ncmFkaWVudChsaW5lYXIs
bGVmdCB0b3AscmlnaHQgdG9wLGZyb20ocmdiYSgwLDAsMCwuMDAwMSkpLHRvKHJnYmEoMCwwLDAs
LjUpKSk7YmFja2dyb3VuZC1pbWFnZTpsaW5lYXItZ3JhZGllbnQodG8gcmlnaHQscmdiYSgwLDAs
MCwuMDAwMSkgMCxyZ2JhKDAsMCwwLC41KSAxMDAlKTtmaWx0ZXI6cHJvZ2lkOkRYSW1hZ2VUcmFu
c2Zvcm0uTWljcm9zb2Z0LmdyYWRpZW50KHN0YXJ0Q29sb3JzdHI9JyMwMDAwMDAwMCcsIGVuZENv
bG9yc3RyPScjODAwMDAwMDAnLCBHcmFkaWVudFR5cGU9MSk7YmFja2dyb3VuZC1yZXBlYXQ6cmVw
ZWF0LXh9LmNhcm91c2VsLWNvbnRyb2w6Zm9jdXMsLmNhcm91c2VsLWNvbnRyb2w6aG92ZXJ7Y29s
b3I6I2ZmZjt0ZXh0LWRlY29yYXRpb246bm9uZTtmaWx0ZXI6YWxwaGEob3BhY2l0eT05MCk7b3V0
bGluZTowO29wYWNpdHk6Ljl9LmNhcm91c2VsLWNvbnRyb2wgLmdseXBoaWNvbi1jaGV2cm9uLWxl
ZnQsLmNhcm91c2VsLWNvbnRyb2wgLmdseXBoaWNvbi1jaGV2cm9uLXJpZ2h0LC5jYXJvdXNlbC1j
b250cm9sIC5pY29uLW5leHQsLmNhcm91c2VsLWNvbnRyb2wgLmljb24tcHJldntwb3NpdGlvbjph
YnNvbHV0ZTt0b3A6NTAlO3otaW5kZXg6NTtkaXNwbGF5OmlubGluZS1ibG9jazttYXJnaW4tdG9w
Oi0xMHB4fS5jYXJvdXNlbC1jb250cm9sIC5nbHlwaGljb24tY2hldnJvbi1sZWZ0LC5jYXJvdXNl
bC1jb250cm9sIC5pY29uLXByZXZ7bGVmdDo1MCU7bWFyZ2luLWxlZnQ6LTEwcHh9LmNhcm91c2Vs
LWNvbnRyb2wgLmdseXBoaWNvbi1jaGV2cm9uLXJpZ2h0LC5jYXJvdXNlbC1jb250cm9sIC5pY29u
LW5leHR7cmlnaHQ6NTAlO21hcmdpbi1yaWdodDotMTBweH0uY2Fyb3VzZWwtY29udHJvbCAuaWNv
bi1uZXh0LC5jYXJvdXNlbC1jb250cm9sIC5pY29uLXByZXZ7d2lkdGg6MjBweDtoZWlnaHQ6MjBw
eDtmb250LWZhbWlseTpzZXJpZjtsaW5lLWhlaWdodDoxfS5jYXJvdXNlbC1jb250cm9sIC5pY29u
LXByZXY6YmVmb3Jle2NvbnRlbnQ6J1wyMDM5J30uY2Fyb3VzZWwtY29udHJvbCAuaWNvbi1uZXh0
OmJlZm9yZXtjb250ZW50OidcMjAzYSd9LmNhcm91c2VsLWluZGljYXRvcnN7cG9zaXRpb246YWJz
b2x1dGU7Ym90dG9tOjEwcHg7bGVmdDo1MCU7ei1pbmRleDoxNTt3aWR0aDo2MCU7cGFkZGluZy1s
ZWZ0OjA7bWFyZ2luLWxlZnQ6LTMwJTt0ZXh0LWFsaWduOmNlbnRlcjtsaXN0LXN0eWxlOm5vbmV9
LmNhcm91c2VsLWluZGljYXRvcnMgbGl7ZGlzcGxheTppbmxpbmUtYmxvY2s7d2lkdGg6MTBweDto
ZWlnaHQ6MTBweDttYXJnaW46MXB4O3RleHQtaW5kZW50Oi05OTlweDtjdXJzb3I6cG9pbnRlcjti
YWNrZ3JvdW5kLWNvbG9yOiMwMDBcOTtiYWNrZ3JvdW5kLWNvbG9yOnJnYmEoMCwwLDAsMCk7Ym9y
ZGVyOjFweCBzb2xpZCAjZmZmO2JvcmRlci1yYWRpdXM6MTBweH0uY2Fyb3VzZWwtaW5kaWNhdG9y
cyAuYWN0aXZle3dpZHRoOjEycHg7aGVpZ2h0OjEycHg7bWFyZ2luOjA7YmFja2dyb3VuZC1jb2xv
cjojZmZmfS5jYXJvdXNlbC1jYXB0aW9ue3Bvc2l0aW9uOmFic29sdXRlO3JpZ2h0OjE1JTtib3R0
b206MjBweDtsZWZ0OjE1JTt6LWluZGV4OjEwO3BhZGRpbmctdG9wOjIwcHg7cGFkZGluZy1ib3R0
b206MjBweDtjb2xvcjojZmZmO3RleHQtYWxpZ246Y2VudGVyO3RleHQtc2hhZG93OjAgMXB4IDJw
eCByZ2JhKDAsMCwwLC42KX0uY2Fyb3VzZWwtY2FwdGlvbiAuYnRue3RleHQtc2hhZG93Om5vbmV9
QG1lZGlhIHNjcmVlbiBhbmQgKG1pbi13aWR0aDo3NjhweCl7LmNhcm91c2VsLWNvbnRyb2wgLmds
eXBoaWNvbi1jaGV2cm9uLWxlZnQsLmNhcm91c2VsLWNvbnRyb2wgLmdseXBoaWNvbi1jaGV2cm9u
LXJpZ2h0LC5jYXJvdXNlbC1jb250cm9sIC5pY29uLW5leHQsLmNhcm91c2VsLWNvbnRyb2wgLmlj
b24tcHJldnt3aWR0aDozMHB4O2hlaWdodDozMHB4O21hcmdpbi10b3A6LTEwcHg7Zm9udC1zaXpl
OjMwcHh9LmNhcm91c2VsLWNvbnRyb2wgLmdseXBoaWNvbi1jaGV2cm9uLWxlZnQsLmNhcm91c2Vs
LWNvbnRyb2wgLmljb24tcHJldnttYXJnaW4tbGVmdDotMTBweH0uY2Fyb3VzZWwtY29udHJvbCAu
Z2x5cGhpY29uLWNoZXZyb24tcmlnaHQsLmNhcm91c2VsLWNvbnRyb2wgLmljb24tbmV4dHttYXJn
aW4tcmlnaHQ6LTEwcHh9LmNhcm91c2VsLWNhcHRpb257cmlnaHQ6MjAlO2xlZnQ6MjAlO3BhZGRp
bmctYm90dG9tOjMwcHh9LmNhcm91c2VsLWluZGljYXRvcnN7Ym90dG9tOjIwcHh9fS5idG4tZ3Jv
dXAtdmVydGljYWw+LmJ0bi1ncm91cDphZnRlciwuYnRuLWdyb3VwLXZlcnRpY2FsPi5idG4tZ3Jv
dXA6YmVmb3JlLC5idG4tdG9vbGJhcjphZnRlciwuYnRuLXRvb2xiYXI6YmVmb3JlLC5jbGVhcmZp
eDphZnRlciwuY2xlYXJmaXg6YmVmb3JlLC5jb250YWluZXItZmx1aWQ6YWZ0ZXIsLmNvbnRhaW5l
ci1mbHVpZDpiZWZvcmUsLmNvbnRhaW5lcjphZnRlciwuY29udGFpbmVyOmJlZm9yZSwuZGwtaG9y
aXpvbnRhbCBkZDphZnRlciwuZGwtaG9yaXpvbnRhbCBkZDpiZWZvcmUsLmZvcm0taG9yaXpvbnRh
bCAuZm9ybS1ncm91cDphZnRlciwuZm9ybS1ob3Jpem9udGFsIC5mb3JtLWdyb3VwOmJlZm9yZSwu
bW9kYWwtZm9vdGVyOmFmdGVyLC5tb2RhbC1mb290ZXI6YmVmb3JlLC5tb2RhbC1oZWFkZXI6YWZ0
ZXIsLm1vZGFsLWhlYWRlcjpiZWZvcmUsLm5hdjphZnRlciwubmF2OmJlZm9yZSwubmF2YmFyLWNv
bGxhcHNlOmFmdGVyLC5uYXZiYXItY29sbGFwc2U6YmVmb3JlLC5uYXZiYXItaGVhZGVyOmFmdGVy
LC5uYXZiYXItaGVhZGVyOmJlZm9yZSwubmF2YmFyOmFmdGVyLC5uYXZiYXI6YmVmb3JlLC5wYWdl
cjphZnRlciwucGFnZXI6YmVmb3JlLC5wYW5lbC1ib2R5OmFmdGVyLC5wYW5lbC1ib2R5OmJlZm9y
ZSwucm93OmFmdGVyLC5yb3c6YmVmb3Jle2Rpc3BsYXk6dGFibGU7Y29udGVudDoiICJ9LmJ0bi1n
cm91cC12ZXJ0aWNhbD4uYnRuLWdyb3VwOmFmdGVyLC5idG4tdG9vbGJhcjphZnRlciwuY2xlYXJm
aXg6YWZ0ZXIsLmNvbnRhaW5lci1mbHVpZDphZnRlciwuY29udGFpbmVyOmFmdGVyLC5kbC1ob3Jp
em9udGFsIGRkOmFmdGVyLC5mb3JtLWhvcml6b250YWwgLmZvcm0tZ3JvdXA6YWZ0ZXIsLm1vZGFs
LWZvb3RlcjphZnRlciwubW9kYWwtaGVhZGVyOmFmdGVyLC5uYXY6YWZ0ZXIsLm5hdmJhci1jb2xs
YXBzZTphZnRlciwubmF2YmFyLWhlYWRlcjphZnRlciwubmF2YmFyOmFmdGVyLC5wYWdlcjphZnRl
ciwucGFuZWwtYm9keTphZnRlciwucm93OmFmdGVye2NsZWFyOmJvdGh9LmNlbnRlci1ibG9ja3tk
aXNwbGF5OmJsb2NrO21hcmdpbi1yaWdodDphdXRvO21hcmdpbi1sZWZ0OmF1dG99LnB1bGwtcmln
aHR7ZmxvYXQ6cmlnaHQhaW1wb3J0YW50fS5wdWxsLWxlZnR7ZmxvYXQ6bGVmdCFpbXBvcnRhbnR9
LmhpZGV7ZGlzcGxheTpub25lIWltcG9ydGFudH0uc2hvd3tkaXNwbGF5OmJsb2NrIWltcG9ydGFu
dH0uaW52aXNpYmxle3Zpc2liaWxpdHk6aGlkZGVufS50ZXh0LWhpZGV7Zm9udDowLzAgYTtjb2xv
cjp0cmFuc3BhcmVudDt0ZXh0LXNoYWRvdzpub25lO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJl
bnQ7Ym9yZGVyOjB9LmhpZGRlbntkaXNwbGF5Om5vbmUhaW1wb3J0YW50fS5hZmZpeHtwb3NpdGlv
bjpmaXhlZH1ALW1zLXZpZXdwb3J0e3dpZHRoOmRldmljZS13aWR0aH0udmlzaWJsZS1sZywudmlz
aWJsZS1tZCwudmlzaWJsZS1zbSwudmlzaWJsZS14c3tkaXNwbGF5Om5vbmUhaW1wb3J0YW50fS52
aXNpYmxlLWxnLWJsb2NrLC52aXNpYmxlLWxnLWlubGluZSwudmlzaWJsZS1sZy1pbmxpbmUtYmxv
Y2ssLnZpc2libGUtbWQtYmxvY2ssLnZpc2libGUtbWQtaW5saW5lLC52aXNpYmxlLW1kLWlubGlu
ZS1ibG9jaywudmlzaWJsZS1zbS1ibG9jaywudmlzaWJsZS1zbS1pbmxpbmUsLnZpc2libGUtc20t
aW5saW5lLWJsb2NrLC52aXNpYmxlLXhzLWJsb2NrLC52aXNpYmxlLXhzLWlubGluZSwudmlzaWJs
ZS14cy1pbmxpbmUtYmxvY2t7ZGlzcGxheTpub25lIWltcG9ydGFudH1AbWVkaWEgKG1heC13aWR0
aDo3NjdweCl7LnZpc2libGUteHN7ZGlzcGxheTpibG9jayFpbXBvcnRhbnR9dGFibGUudmlzaWJs
ZS14c3tkaXNwbGF5OnRhYmxlIWltcG9ydGFudH10ci52aXNpYmxlLXhze2Rpc3BsYXk6dGFibGUt
cm93IWltcG9ydGFudH10ZC52aXNpYmxlLXhzLHRoLnZpc2libGUteHN7ZGlzcGxheTp0YWJsZS1j
ZWxsIWltcG9ydGFudH19QG1lZGlhIChtYXgtd2lkdGg6NzY3cHgpey52aXNpYmxlLXhzLWJsb2Nr
e2Rpc3BsYXk6YmxvY2shaW1wb3J0YW50fX1AbWVkaWEgKG1heC13aWR0aDo3NjdweCl7LnZpc2li
bGUteHMtaW5saW5le2Rpc3BsYXk6aW5saW5lIWltcG9ydGFudH19QG1lZGlhIChtYXgtd2lkdGg6
NzY3cHgpey52aXNpYmxlLXhzLWlubGluZS1ibG9ja3tkaXNwbGF5OmlubGluZS1ibG9jayFpbXBv
cnRhbnR9fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KSBhbmQgKG1heC13aWR0aDo5OTFweCl7LnZp
c2libGUtc217ZGlzcGxheTpibG9jayFpbXBvcnRhbnR9dGFibGUudmlzaWJsZS1zbXtkaXNwbGF5
OnRhYmxlIWltcG9ydGFudH10ci52aXNpYmxlLXNte2Rpc3BsYXk6dGFibGUtcm93IWltcG9ydGFu
dH10ZC52aXNpYmxlLXNtLHRoLnZpc2libGUtc217ZGlzcGxheTp0YWJsZS1jZWxsIWltcG9ydGFu
dH19QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpIGFuZCAobWF4LXdpZHRoOjk5MXB4KXsudmlzaWJs
ZS1zbS1ibG9ja3tkaXNwbGF5OmJsb2NrIWltcG9ydGFudH19QG1lZGlhIChtaW4td2lkdGg6NzY4
cHgpIGFuZCAobWF4LXdpZHRoOjk5MXB4KXsudmlzaWJsZS1zbS1pbmxpbmV7ZGlzcGxheTppbmxp
bmUhaW1wb3J0YW50fX1AbWVkaWEgKG1pbi13aWR0aDo3NjhweCkgYW5kIChtYXgtd2lkdGg6OTkx
cHgpey52aXNpYmxlLXNtLWlubGluZS1ibG9ja3tkaXNwbGF5OmlubGluZS1ibG9jayFpbXBvcnRh
bnR9fUBtZWRpYSAobWluLXdpZHRoOjk5MnB4KSBhbmQgKG1heC13aWR0aDoxMTk5cHgpey52aXNp
YmxlLW1ke2Rpc3BsYXk6YmxvY2shaW1wb3J0YW50fXRhYmxlLnZpc2libGUtbWR7ZGlzcGxheTp0
YWJsZSFpbXBvcnRhbnR9dHIudmlzaWJsZS1tZHtkaXNwbGF5OnRhYmxlLXJvdyFpbXBvcnRhbnR9
dGQudmlzaWJsZS1tZCx0aC52aXNpYmxlLW1ke2Rpc3BsYXk6dGFibGUtY2VsbCFpbXBvcnRhbnR9
fUBtZWRpYSAobWluLXdpZHRoOjk5MnB4KSBhbmQgKG1heC13aWR0aDoxMTk5cHgpey52aXNpYmxl
LW1kLWJsb2Nre2Rpc3BsYXk6YmxvY2shaW1wb3J0YW50fX1AbWVkaWEgKG1pbi13aWR0aDo5OTJw
eCkgYW5kIChtYXgtd2lkdGg6MTE5OXB4KXsudmlzaWJsZS1tZC1pbmxpbmV7ZGlzcGxheTppbmxp
bmUhaW1wb3J0YW50fX1AbWVkaWEgKG1pbi13aWR0aDo5OTJweCkgYW5kIChtYXgtd2lkdGg6MTE5
OXB4KXsudmlzaWJsZS1tZC1pbmxpbmUtYmxvY2t7ZGlzcGxheTppbmxpbmUtYmxvY2shaW1wb3J0
YW50fX1AbWVkaWEgKG1pbi13aWR0aDoxMjAwcHgpey52aXNpYmxlLWxne2Rpc3BsYXk6YmxvY2sh
aW1wb3J0YW50fXRhYmxlLnZpc2libGUtbGd7ZGlzcGxheTp0YWJsZSFpbXBvcnRhbnR9dHIudmlz
aWJsZS1sZ3tkaXNwbGF5OnRhYmxlLXJvdyFpbXBvcnRhbnR9dGQudmlzaWJsZS1sZyx0aC52aXNp
YmxlLWxne2Rpc3BsYXk6dGFibGUtY2VsbCFpbXBvcnRhbnR9fUBtZWRpYSAobWluLXdpZHRoOjEy
MDBweCl7LnZpc2libGUtbGctYmxvY2t7ZGlzcGxheTpibG9jayFpbXBvcnRhbnR9fUBtZWRpYSAo
bWluLXdpZHRoOjEyMDBweCl7LnZpc2libGUtbGctaW5saW5le2Rpc3BsYXk6aW5saW5lIWltcG9y
dGFudH19QG1lZGlhIChtaW4td2lkdGg6MTIwMHB4KXsudmlzaWJsZS1sZy1pbmxpbmUtYmxvY2t7
ZGlzcGxheTppbmxpbmUtYmxvY2shaW1wb3J0YW50fX1AbWVkaWEgKG1heC13aWR0aDo3NjdweCl7
LmhpZGRlbi14c3tkaXNwbGF5Om5vbmUhaW1wb3J0YW50fX1AbWVkaWEgKG1pbi13aWR0aDo3Njhw
eCkgYW5kIChtYXgtd2lkdGg6OTkxcHgpey5oaWRkZW4tc217ZGlzcGxheTpub25lIWltcG9ydGFu
dH19QG1lZGlhIChtaW4td2lkdGg6OTkycHgpIGFuZCAobWF4LXdpZHRoOjExOTlweCl7LmhpZGRl
bi1tZHtkaXNwbGF5Om5vbmUhaW1wb3J0YW50fX1AbWVkaWEgKG1pbi13aWR0aDoxMjAwcHgpey5o
aWRkZW4tbGd7ZGlzcGxheTpub25lIWltcG9ydGFudH19LnZpc2libGUtcHJpbnR7ZGlzcGxheTpu
b25lIWltcG9ydGFudH1AbWVkaWEgcHJpbnR7LnZpc2libGUtcHJpbnR7ZGlzcGxheTpibG9jayFp
bXBvcnRhbnR9dGFibGUudmlzaWJsZS1wcmludHtkaXNwbGF5OnRhYmxlIWltcG9ydGFudH10ci52
aXNpYmxlLXByaW50e2Rpc3BsYXk6dGFibGUtcm93IWltcG9ydGFudH10ZC52aXNpYmxlLXByaW50
LHRoLnZpc2libGUtcHJpbnR7ZGlzcGxheTp0YWJsZS1jZWxsIWltcG9ydGFudH19LnZpc2libGUt
cHJpbnQtYmxvY2t7ZGlzcGxheTpub25lIWltcG9ydGFudH1AbWVkaWEgcHJpbnR7LnZpc2libGUt
cHJpbnQtYmxvY2t7ZGlzcGxheTpibG9jayFpbXBvcnRhbnR9fS52aXNpYmxlLXByaW50LWlubGlu
ZXtkaXNwbGF5Om5vbmUhaW1wb3J0YW50fUBtZWRpYSBwcmludHsudmlzaWJsZS1wcmludC1pbmxp
bmV7ZGlzcGxheTppbmxpbmUhaW1wb3J0YW50fX0udmlzaWJsZS1wcmludC1pbmxpbmUtYmxvY2t7
ZGlzcGxheTpub25lIWltcG9ydGFudH1AbWVkaWEgcHJpbnR7LnZpc2libGUtcHJpbnQtaW5saW5l
LWJsb2Nre2Rpc3BsYXk6aW5saW5lLWJsb2NrIWltcG9ydGFudH19QG1lZGlhIHByaW50ey5oaWRk
ZW4tcHJpbnR7ZGlzcGxheTpub25lIWltcG9ydGFudH19LyojIHNvdXJjZU1hcHBpbmdVUkw9Ym9v
dHN0cmFwLm1pbi5jc3MubWFwICovPC9zdHlsZT48c3R5bGU+LyogR0xPQkFMICovaHRtbCxib2R5
IHtiYWNrZ3JvdW5kOiAjZjBmMGYwO292ZXJmbG93LXg6IGhpZGRlbjt9aDEge2ZvbnQtd2VpZ2h0
OiBib2xkO2xldHRlci1zcGFjaW5nOiAtM3B4O31oMyB7Zm9udC1zaXplOiAyMXB4O2xldHRlci1z
cGFjaW5nOiAtMXB4O30uaC1kYXNoYm9hcmQge3RleHQtdHJhbnNmb3JtOiBsb3dlcmNhc2U7fS5w
YWdlLWhlYWRlciB7Ym9yZGVyLWJvdHRvbTogMXB4IHNvbGlkIHJnYmEoMCwgMCwgMCwgMC4xNSk7
bWFyZ2luOiAyNXB4IDAgMjBweDtwb3NpdGlvbjogcmVsYXRpdmU7fS5wYWdlLWhlYWRlciBoMSB7
bWFyZ2luOiAwO30ucGFnaW5hdGlvbiB7bWFyZ2luOiA1cHggMDt9LmNsaWNrYWJsZSwuZXhwYW5k
YWJsZT50ZCB7Y3Vyc29yOiBwb2ludGVyO30uc3Bpbm5lciB7Y29sb3I6ICM5OTk7bGVmdDogNTAl
O3Bvc2l0aW9uOiBhYnNvbHV0ZTt0b3A6IDUwJTt9LnBvd2VyZWQge2JvdHRvbTogMTkwcHg7Y29s
b3I6ICM5RTlFOUU7Zm9udC1zaXplOiBzbWFsbGVyO3Bvc2l0aW9uOiBhYnNvbHV0ZTtyaWdodDog
MjBweDt0cmFuc2Zvcm0tb3JpZ2luOiAxMDAlIDA7dHJhbnNmb3JtOiByb3RhdGUoLTkwZGVnKTt9
LnBvd2VyZWQgYSB7Y29sb3I6ICM2MzYzNjM7fS5wb3dlcmVkIHNwYW4ge2NvbG9yOiAjMDA3YmMz
O30uZHJvcGRvd24taGVhZGVyIHtjb2xvcjogIzAwN2JjMztwYWRkaW5nOiAzcHggMjVweDt0ZXh0
LXRyYW5zZm9ybTogdXBwZXJjYXNlO30uZ2hlYWRlciB7bGV0dGVyLXNwYWNpbmc6IC0xcHg7dGV4
dC10cmFuc2Zvcm06IHVwcGVyY2FzZTt9aDUuZ2hlYWRlciB7bGV0dGVyLXNwYWNpbmc6IDA7fS5w
YW5lbC1oZWFkZXIgaDQuZ2hlYWRlciB7bWFyZ2luLXRvcDogMjBweDt9LnBhbmVsLWhlYWRlciAu
Z2hlYWRlciBzbWFsbCB7Zm9udC1zaXplOiA2OSU7fS8qIE5BVklHQVRJT04gKi9uYXYgey13ZWJr
aXQtdHJhbnNpdGlvbjogbGVmdCAuMnM7YmFja2dyb3VuZDogIzFDMUMxQztib3JkZXItcmlnaHQ6
IDNweCBzb2xpZCAjNWJjMGRlO2hlaWdodDogMTAwJTtsZWZ0OiAtMjM2cHg7b3ZlcmZsb3c6IGhp
ZGRlbjtwb3NpdGlvbjogZml4ZWQ7dG9wOiAwO3RyYW5zaXRpb246IGxlZnQgLjJzO3dpZHRoOiAz
MDBweDt6LWluZGV4OiAyO31uYXYgLm5hdi1saXN0IHtib3R0b206IDA7bGVmdDogMDtvdmVyZmxv
dy15OiBzY3JvbGw7cG9zaXRpb246IGFic29sdXRlO3JpZ2h0OiAtMTdweDt0b3A6IDA7fW5hdiBo
ZWFkZXIge21hcmdpbjogNDBweCAyMHB4IDMwcHg7fW5hdiBoZWFkZXIgYSB7Y29sb3I6IHJnYmEo
MjQwLDI0MCwyNDAsLjcpO2ZvbnQtc2l6ZTogMi43ZW07Zm9udC13ZWlnaHQ6IDMwMDt0ZXh0LXRy
YW5zZm9ybTogdXBwZXJjYXNlO31uYXYgaGVhZGVyIGE6aG92ZXIge2NvbG9yOiAjZWVlO31uYXYu
YWN0aXZlIHtkaXNwbGF5OiBibG9jayAhaW1wb3J0YW50O2xlZnQ6IDA7b3BhY2l0eTogLjk3O31u
YXY6aG92ZXIgfiAjY29udGVudCB7b3BhY2l0eTogLjM7fW5hdi5hY3RpdmUgLm5hdi1iYXJzLG5h
di5hY3RpdmUgLm5hdi1nZWFycyxuYXYuYWN0aXZlIC5uYXYtd3Mtc3RhdHVzIHtvcGFjaXR5OiAw
O31uYXYgLm5hdi1iYXJzLG5hdiAubmF2LWdlYXJzLG5hdiAubmF2LXdzLXN0YXR1cyB7LXdlYmtp
dC10cmFuc2l0aW9uOiBvcGFjaXR5IC4ycztjb2xvcjogIzlFOUU5RTtjdXJzb3I6IHBvaW50ZXI7
ZmxvYXQ6IHJpZ2h0O2ZvbnQtc2l6ZTogMzZweDtoZWlnaHQ6IDMycHg7bGVmdDogMTNweDtsaW5l
LWhlaWdodDogMzJweDtwb3NpdGlvbjogZml4ZWQ7dGV4dC1hbGlnbjogY2VudGVyO3RvcDogMzBw
eDt0cmFuc2l0aW9uOiBvcGFjaXR5IC4yczt3aWR0aDogMzJweDt9bmF2IC5uYXYtZ2VhcnMge3Rv
cDogMTAwcHg7b3BhY2l0eTogMC42O31uYXYgLm5hdi13cy1zdGF0dXMsLm5hdi13cy1zdGF0dXMu
bWluaSB7Y29sb3I6ICM2QTZBNkE7Y3Vyc29yOiBoZWxwO2Rpc3BsYXk6IG5vbmU7Zm9udC1zaXpl
OiAxMnB4O31uYXYgLm5hdi13cy1zdGF0dXMge2xlZnQ6IDI1cHg7dG9wOiAxMjVweDt9Lm5hdi13
cy1zdGF0dXMubWluaSB7dG9wOiAxNHB4O2xlZnQ6IDUwcHg7cG9zaXRpb246IGFic29sdXRlO30u
bmF2LXdzLXN0YXR1cy5jb25uZWN0ZWQge2NvbG9yOiAjNURCNTZBO31uYXYgbGkge3Bvc2l0aW9u
OiByZWxhdGl2ZTt9bmF2IGxpIC50b2dnbGUtcGFuZWwge2N1cnNvcjogcG9pbnRlcjtvcGFjaXR5
OiAwO3BhZGRpbmc6IDlweCAyMHB4O3Bvc2l0aW9uOiBhYnNvbHV0ZTtyaWdodDogMDt0b3A6IDA7
dHJhbnNpdGlvbjogYWxsIC4yczt2aXNpYmlsaXR5OiBoaWRkZW47fW5hdiBsaSAudG9nZ2xlLXBh
bmVsIGkge2NvbG9yOiByZ2JhKDIwMCwyMDAsMjAwLC41KTtvcGFjaXR5OiAwO31uYXYgbGkgLnRv
Z2dsZS1wYW5lbC5hY3RpdmUgaSB7Y29sb3I6ICNlZWU7b3BhY2l0eTogMTt9bmF2LmFjdGl2ZSBs
aSAudG9nZ2xlLXBhbmVsIHt2aXNpYmlsaXR5OiB2aXNpYmxlO29wYWNpdHk6IDE7fW5hdi5hY3Rp
dmUgbGk6aG92ZXIgLnRvZ2dsZS1wYW5lbCBpIHtvcGFjaXR5OiAxO31uYXYgbGkgYSB7Ym9yZGVy
LWxlZnQ6IDNweCBzb2xpZCB0cmFuc3BhcmVudDtjb2xvcjogcmdiYSgyMDAsMjAwLDIwMCwuNSk7
ZGlzcGxheTogYmxvY2s7Zm9udC1zaXplOiBzbWFsbGVyO21heC13aWR0aDogMjM1cHg7b3BhY2l0
eTogMDtvdmVyZmxvdzogaGlkZGVuO3BhZGRpbmc6IDlweCAyMHB4O3RleHQtb3ZlcmZsb3c6IGVs
bGlwc2lzO3RleHQtdHJhbnNmb3JtOiB1cHBlcmNhc2U7dHJhbnNpdGlvbjogb3BhY2l0eSAuMnM7
d2hpdGUtc3BhY2U6IG5vd3JhcDt9bmF2LmFjdGl2ZSBsaSBhIHttYXgtd2lkdGg6IDkwJTtvcGFj
aXR5OiAxO31uYXYgbGk6aG92ZXIgYSxuYXYgbGkuYWN0aXZlIGEge2JhY2tncm91bmQ6IHJnYmEo
MCwwLDAsLjEpO2JvcmRlci1jb2xvcjogIzVCQzBERTtjb2xvcjogI2VlZTt9bmF2IHVsIHtwYWRk
aW5nLWxlZnQ6IDA7bGlzdC1zdHlsZTogbm9uZTt9LyogTmF2aWdhdGlvbiAgLS0gSWNvbiAqL25h
diBhLG5hdiBhOmhvdmVyIHt0ZXh0LWRlY29yYXRpb246IG5vbmU7fW5hdiBoMyB7Y29sb3I6ICNG
RkYgIWltcG9ydGFudDtmb250LXNpemU6IG1lZGl1bTtmb250LXdlaWdodDogYm9sZDttYXJnaW46
IDIwcHggMjVweCAxMHB4O3RleHQtdHJhbnNmb3JtOiB1cHBlcmNhc2U7fS8qIENPTlRBSU5FUiAq
L0BtZWRpYSBzY3JlZW4gYW5kIChtYXgtd2lkdGg6IDc2N3B4KSB7LnJvdy1vZmZjYW52YXMgey13
ZWJraXQtdHJhbnNpdGlvbjogYWxsIC4yNXMgZWFzZS1vdXQ7LW8tdHJhbnNpdGlvbjogYWxsIC4y
NXMgZWFzZS1vdXQ7cG9zaXRpb246IHJlbGF0aXZlO3RyYW5zaXRpb246IGFsbCAuMjVzIGVhc2Ut
b3V0O30ucm93LW9mZmNhbnZhcy1yaWdodCB7cmlnaHQ6IDA7fS5yb3ctb2ZmY2FudmFzLWxlZnQg
e2xlZnQ6IDA7fS5yb3ctb2ZmY2FudmFzLXJpZ2h0LnNpZGViYXItb2ZmY2FudmFzIHtyaWdodDog
LTUwJTt9LnJvdy1vZmZjYW52YXMtbGVmdC5zaWRlYmFyLW9mZmNhbnZhcyB7bGVmdDogLTUwJTt9
LnJvdy1vZmZjYW52YXMtcmlnaHQuYWN0aXZlIHtyaWdodDogNTAlO30ucm93LW9mZmNhbnZhcy1s
ZWZ0LmFjdGl2ZSB7bGVmdDogNTAlO30uc2lkZWJhci1vZmZjYW52YXMge3Bvc2l0aW9uOiBhYnNv
bHV0ZTt0b3A6IDA7d2lkdGg6IDUwJTt9O31AbWVkaWEgKG1pbi13aWR0aDogNzY4cHgpIHsuY29u
dGFpbmVyIHt3aWR0aDogNzUwcHg7fTt9QG1lZGlhIChtYXgtd2lkdGg6IDQ4MHB4KSB7LndyYXAt
Z2VuZXJhbCBoNSwud3JhcC1wYW5lbCBoNSB7d2hpdGUtc3BhY2U6IG5vd3JhcDtvdmVyZmxvdzog
aGlkZGVuO3RleHQtb3ZlcmZsb3c6IGVsbGlwc2lzO30ud3JhcC1nZW5lcmFsIGg1IHt3aWR0aDog
MTAwJX0ud3JhcC1wYW5lbCBoNSB7d2lkdGg6IDcwJX19LmNvbnRhaW5lci1mbHVpZCB7bWFyZ2lu
LWxlZnQ6IDc1cHg7fUBtZWRpYSAobWluLXdpZHRoOiAxMTIwcHgpIHsuY29udGFpbmVyIHt3aWR0
aDogOTcwcHg7fTt9QG1lZGlhIChtaW4td2lkdGg6IDEzMjBweCkgey5jb250YWluZXIge3dpZHRo
OiAxMTcwcHg7fTt9QG1lZGlhIChtYXgtd2lkdGg6IDk5MnB4KSB7LmNvbnRhaW5lci1mbHVpZCB7
bWFyZ2luLWxlZnQ6IGF1dG87fTt9QG1lZGlhIChtYXgtd2lkdGg6IDc2OHB4KSB7LmNvbnRhaW5l
ci1mbHVpZCB7cGFkZGluZy1sZWZ0OiA1cHg7cGFkZGluZy1yaWdodDogNXB4O30ucGFnZS1oZWFk
ZXIge3BhZGRpbmc6IDAgMTBweDt9fS8qIFBBTkVMIFNUWUxFUyAqLy53cmFwLXBhbmVsIC5wYW5l
bC1oZWFkZXIge3Bvc2l0aW9uOiByZWxhdGl2ZTt9ZGl2LndyYXAtcGFuZWwgPiBkaXYge2JhY2tn
cm91bmQ6ICNGRkY7bWFyZ2luLXRvcDogMTBweDtwYWRkaW5nOiAwIDEwcHg7Ym9yZGVyLXRvcDog
MXB4IHNvbGlkIHJnYmEoMCwgMCwgMCwgMC4xNSk7fS8qIFBBTkVMIFRBQkxFUyAqLy53cmFwLXBh
bmVsIHRhYmxlLnRhYmxlLWJvcmRlcmxlc3MgdGJvZHkgdHIgdGQsLndyYXAtcGFuZWwgdGFibGUu
dGFibGUtYm9yZGVybGVzcyB0Ym9keSB0ciB0aCwud3JhcC1wYW5lbCB0YWJsZS50YWJsZS1ib3Jk
ZXJsZXNzIHRoZWFkIHRyIHRoIHtib3JkZXI6IG5vbmU7fS53cmFwLXBhbmVsIHRhYmxlIHRoZWFk
IHRyIHRoIHt0ZXh0LWFsaWduOiByaWdodDtib3JkZXItYm90dG9tLXdpZHRoOiAxcHg7fS53cmFw
LXBhbmVsIHRhYmxlIC5zdHJpbmcsLndyYXAtcGFuZWwgdGFibGUgLmRhdGUge3RleHQtYWxpZ246
IGxlZnQ7fS53cmFwLXBhbmVsIHRhYmxlIC5wZXJjZW50IHtjb2xvcjogIzg5ODk4OTt9LndyYXAt
cGFuZWwgdGFibGUgdGQsLndyYXAtcGFuZWwgdGFibGUgdGgge3doaXRlLXNwYWNlOiBub3dyYXA7
b3ZlcmZsb3c6IGhpZGRlbjt9LndyYXAtcGFuZWwgdGFibGUgdGguc29ydGFibGUge2N1cnNvcjog
cG9pbnRlcjt9LndyYXAtcGFuZWwgdGFibGUudGFibGUtYm9yZGVybGVzcyB0aGVhZD50ci50aGVh
ZC1jb2xzIHRoIHtmb250LXNpemU6IDc4JTt0ZXh0LXRyYW5zZm9ybTogdXBwZXJjYXNlO30ud3Jh
cC1wYW5lbCB0YWJsZSAuY2VsbC1obCB7cGFkZGluZzogMnB4IDNweDtjb2xvcjogI0ZGRjtib3Jk
ZXItcmFkaXVzOiA1cHg7ZGlzcGxheTogYmxvY2s7dGV4dC1hbGlnbjogY2VudGVyO30ud3JhcC1w
YW5lbCB0YWJsZSAuc3Bhbi1obCB7cGFkZGluZzogMnB4IDNweDtib3JkZXItcmFkaXVzOiAzcHg7
Y29sb3I6ICMwMDA7fS53cmFwLXBhbmVsIHRhYmxlIC5zcGFuLWhsLmc1IHtiYWNrZ3JvdW5kOiAj
ZTllY2VmO30ud3JhcC1wYW5lbCB0YWJsZSAuY2VsbC1obC5iMSB7YmFja2dyb3VuZDogIzdGNjY5
RDt9LndyYXAtcGFuZWwgdGFibGUgLmNlbGwtaGwuYjIge2JhY2tncm91bmQ6ICNCQTk0RDE7fS53
cmFwLXBhbmVsIHRhYmxlIC5jZWxsLWhsLmIzIHtiYWNrZ3JvdW5kOiAjREVCQUNFO30ud3JhcC1w
YW5lbCB0YWJsZSAuY2VsbC1obC5kMSB7YmFja2dyb3VuZDogIzlkOWQ5ZDM4O30ud3JhcC1wYW5l
bCB0YWJsZSAuY2VsbC1obC5kMiB7YmFja2dyb3VuZDogIzlkOWQ5ZDYxO30ud3JhcC1wYW5lbCB0
YWJsZSAuY2VsbC1obC5kMyB7YmFja2dyb3VuZDogIzlkOWQ5ZDljO30ud3JhcC1wYW5lbCB0YWJs
ZSAuY2VsbC1obC5kNCB7YmFja2dyb3VuZDogIzlkOWQ5ZDt9LndyYXAtcGFuZWwgdGFibGUgLnNw
YW4taGwubGdybiB7YmFja2dyb3VuZDogI2U2ZjRlYTtjb2xvcjogIzEzNzMzMzt9LndyYXAtcGFu
ZWwgdGFibGUgLnNwYW4taGwubHllbCB7YmFja2dyb3VuZDogI2ZmZjNjZDtjb2xvcjogI2QzOGEx
MDt9LndyYXAtcGFuZWwgdGFibGUgLnNwYW4taGwubHJlZCB7YmFja2dyb3VuZDogI2ZjZThlNjtj
b2xvcjogI2M1MjIxZjt9LndyYXAtcGFuZWwgdGFibGUgLnNwYW4taGwubGJsdSB7YmFja2dyb3Vu
ZDogI2NmZTJmZjtjb2xvcjogIzA1MmM2NTt9LndyYXAtcGFuZWwgdGFibGUgLnNwYW4taGwubHBy
cCB7YmFja2dyb3VuZDogI2NkYzdmZjtjb2xvcjogIzM0MzE1MDt9LyogdGhlYWQgbWV0YSAqLy53
cmFwLXBhbmVsIHRhYmxlIHRoZWFkPnRyLnRoZWFkLW1pbiB0aC5tZXRhLWxhYmVsLC53cmFwLXBh
bmVsIHRhYmxlIHRoZWFkPnRyLnRoZWFkLWF2ZyB0aC5tZXRhLWxhYmVsLC53cmFwLXBhbmVsIHRh
YmxlIHRmb290PnRyLnRmb290LXRvdGFscyB0aC5tZXRhLWxhYmVsLC53cmFwLXBhbmVsIHRhYmxl
IHRoZWFkPnRyLnRoZWFkLW1heCB0aC5tZXRhLWxhYmVsIHtmb250LXdlaWdodDogYm9sZDt0ZXh0
LXRyYW5zZm9ybTogdXBwZXJjYXNlO30ud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtbWluLC53cmFw
LXBhbmVsIHRhYmxlIC50aGVhZC1hdmcsLndyYXAtcGFuZWwgdGFibGUgLnRoZWFkLW1heCB7YmFj
a2dyb3VuZDogI0Y4RjhGODt9LndyYXAtcGFuZWwgdGFibGUgLnRoZWFkLWF2ZyB7Ym9yZGVyLWJv
dHRvbTogMnB4IHNvbGlkICMwMDA7fS53cmFwLXBhbmVsIHRhYmxlIHRoZWFkPnRyLnRoZWFkLW1p
biB0aCwud3JhcC1wYW5lbCB0YWJsZSB0aGVhZD50ci50aGVhZC1hdmcgdGgsLndyYXAtcGFuZWwg
dGFibGUgdGhlYWQ+dHIudGhlYWQtbWF4IHRoIHtmb250LXNpemU6IHNtYWxsZXI7Zm9udC13ZWln
aHQ6IG5vcm1hbDtwYWRkaW5nOiAzcHggOHB4IDNweCA4cHg7dGV4dC10cmFuc2Zvcm06IGluaGVy
aXQ7fS53cmFwLXBhbmVsIHRhYmxlIC50aGVhZC1taW4gdGgsLndyYXAtcGFuZWwgdGFibGUgLnRo
ZWFkLWF2ZyB0aCwud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtbWF4IHRoIHtwYWRkaW5nOiAzcHg7
fS53cmFwLXBhbmVsIHRhYmxlIHRmb290PnRyPnRoIHtib3JkZXItdG9wOiAxcHggZG90dGVkICMw
MDA7cGFkZGluZzogOHB4O30vKiB0aGVhZCBkYXRhICovLndyYXAtcGFuZWwgdGFibGUgdGJvZHku
dGJvZHktZGF0YSB0ciB0ZCB7Ym9yZGVyLXJpZ2h0OiAxcHggc29saWQgI0YxRjFGMTtmb250LXNp
emU6IHNtYWxsZXI7fS53cmFwLXBhbmVsIHRhYmxlIHRib2R5LnRib2R5LWRhdGEgdGQ6bGFzdC1j
aGlsZCB7Ym9yZGVyLXJpZ2h0OiBub25lO30ud3JhcC1wYW5lbCB0YWJsZSB0Ym9keS50Ym9keS1k
YXRhIHRkLnJvdy1pZHgge2ZvbnQtd2VpZ2h0OiA3MDA7fS53cmFwLXBhbmVsIHRhYmxlPnRoZWFk
PnRyLnRoZWFkLWNvbHMge2JvcmRlci1ib3R0b206IDJweCBzb2xpZCAjMjIyO30ud3JhcC1wYW5l
bCB0YWJsZSB0Ym9keS50Ym9keS1kYXRhIHRyLnNoYWRlZCB7YmFja2dyb3VuZC1jb2xvcjogI0Y3
RjdGNzt9LndyYXAtcGFuZWwgdGFibGUgdGJvZHkudGJvZHktZGF0YSB0ci4ge2JhY2tncm91bmQt
Y29sb3I6ICNGN0Y3Rjc7fS53cmFwLXBhbmVsIHRhYmxlIHRib2R5LnRib2R5LWRhdGEgdHIuY2hp
bGQgdGQ6bnRoLWNoaWxkKDEpLC53cmFwLXBhbmVsIHRhYmxlIHRib2R5LnRib2R5LWRhdGEgdHIu
Y2hpbGQgdGQ6bnRoLWNoaWxkKDIpIHtib3JkZXItcmlnaHQ6IG5vbmU7fS53cmFwLXBhbmVsIHRh
YmxlLnRhYmxlLWhvdmVyPnRib2R5PnRyOmhvdmVyIHtiYWNrZ3JvdW5kLWNvbG9yOiAjRUVFO30u
d3JhcC1wYW5lbCAucm93Om50aC1jaGlsZCgyKTpub3QoLnRhYmxlLXdyYXBwZXIpPi5jb2wtbWQt
MTIge3BhZGRpbmc6IDA7fS8qIEdFTkVSQUwgKi8ud3JhcC1nZW5lcmFsIHtwb3NpdGlvbjogcmVs
YXRpdmU7fS5yZXBvcnQtdGl0bGUge2JhY2tncm91bmQ6ICNGRkY7Ym9yZGVyLXJhZGl1czogNHB4
O2JvdHRvbTogLTEwcHg7Y29sb3I6ICM5RTlFOUU7Zm9udC1zaXplOiBzbWFsbDtwYWRkaW5nOiAw
IDEwcHg7cG9zaXRpb246IGFic29sdXRlO3JpZ2h0OiAwO3otaW5kZXg6IDE7fS5wYW5lbC1wbG90
LXdyYXAge3Bvc2l0aW9uOiBhYnNvbHV0ZTtyaWdodDogMDt0b3A6IDE4cHg7fS5jb2wtdGl0bGUg
e2ZvbnQtc2l6ZTogODUlO292ZXJmbG93OiBoaWRkZW47dGV4dC1vdmVyZmxvdzogZWxsaXBzaXM7
dGV4dC1zaGFkb3c6IDFweCAxcHggMCAjRkZGO3doaXRlLXNwYWNlOiBub3dyYXA7d2lkdGg6IDEw
MCU7fS5ncmlkLW1vZHVsZSB7YmFja2dyb3VuZDogI0ZGRjtjb2xvcjogcmdiKDM2LCAzNiwgMzYp
O2ZvbnQtd2VpZ2h0OiBub3JtYWw7bWFyZ2luLXRvcDogNXB4O3BhZGRpbmc6IDdweDt9LmdyaWQt
bW9kdWxlIGgzIHtmb250LXNpemU6IDI1cHg7bWFyZ2luOiAwO292ZXJmbG93OiBoaWRkZW47dGV4
dC1vdmVyZmxvdzogZWxsaXBzaXM7d2hpdGUtc3BhY2U6IG5vd3JhcDt3aWR0aDogMTAwJTt9Lmdy
aWQtbW9kdWxlLmJsYWNrIHtib3JkZXItdG9wOiA0cHggc29saWQgIzBGMTIxNDt9LmdyaWQtbW9k
dWxlLmdyYXkge2JvcmRlci10b3A6IDRweCBzb2xpZCAjOUU5RTlFO30uZ3JpZC1tb2R1bGUucmVk
IHtib3JkZXItdG9wOiA0cHggc29saWQgI0ZGMzAzRTt9LmdyaWQtbW9kdWxlLmJsdWV7Ym9yZGVy
LXRvcDogNHB4IHNvbGlkICMwMEQ0RTE7fS5ncmlkLW1vZHVsZS5ncmVlbiB7Ym9yZGVyLXRvcDog
NHB4IHNvbGlkICMyMjlmNzU7fUBtZWRpYSAobWF4LXdpZHRoOiA3NjdweCkgey5wYW5lbC1wbG90
LXdyYXAge3RvcDogMTBweDt9LnBvd2VyZWQge2JvdHRvbTogMTBweDtsZWZ0OiAyNXB4O3RyYW5z
Zm9ybTogaW5pdGlhbDt9fS8qIENIQVJUUyAqLy5jaGFydC13cmFwIHttYXJnaW4tYm90dG9tOiAx
NXB4O3Bvc2l0aW9uOiByZWxhdGl2ZTt9c3ZnIHtiYWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3BhcmVu
dDtkaXNwbGF5OiBibG9jazt9LmF4aXMgcGF0aCB7ZmlsbDogdHJhbnNwYXJlbnQ7c3Ryb2tlOiBi
bGFjaztzaGFwZS1yZW5kZXJpbmc6IGNyaXNwRWRnZXM7c3Ryb2tlLXdpZHRoOiAxO30uZ3JpZC55
IC50aWNrIGxpbmUsLmdyaWQueCAudGljayBsaW5lIHtzaGFwZS1yZW5kZXJpbmc6IGNyaXNwRWRn
ZXM7c3Ryb2tlOiAjOTk5O3N0cm9rZS1kYXNoYXJyYXk6IDMgMztzdHJva2Utd2lkdGg6IDE7fS5h
eGlzLnggLnRpY2sgbGluZSwuYXhpcy55MCAudGljayBsaW5lLC5heGlzLnkxIC50aWNrIGxpbmUs
LmdyaWQueSAudGljazpmaXJzdC1jaGlsZCBsaW5lIHtzdHJva2U6IGJsYWNrO3N0cm9rZS13aWR0
aDogMTtzaGFwZS1yZW5kZXJpbmc6IGNyaXNwRWRnZXM7fS5iYXJzIHJlY3QuYmFyIHtzaGFwZS1y
ZW5kZXJpbmc6IGNyaXNwRWRnZXM7fS5yZWN0cyByZWN0IHtmaWxsOiB0cmFuc3BhcmVudDt9LmFy
ZWEge29wYWNpdHk6IDAuMjt9LnBvaW50cyB7c3Ryb2tlOiB0cmFuc3BhcmVudDt9bGluZS5pbmRp
Y2F0b3Ige2ZpbGw6IHRyYW5zcGFyZW50O3BvaW50ZXItZXZlbnRzOiBub25lO3NoYXBlLXJlbmRl
cmluZzogY3Jpc3BFZGdlcztzdHJva2U6ICM5OTk7c3Ryb2tlLXdpZHRoOiAxO2Rpc3BsYXk6IG5v
bmU7fS5hcmVhMCwuYmFycy55MCAuYmFyLC5wb2ludHMueTAscmVjdC5sZWdlbmQueTAge2ZpbGw6
ICM0NDdGQjM7fS5hcmVhMSwuYmFycy55MSAuYmFyLC5wb2ludHMueTEscmVjdC5sZWdlbmQueTEg
e2ZpbGw6ICNGRjY4NTQ7fS5saW5lMCwubGluZTEge2ZpbGw6IHRyYW5zcGFyZW50O3N0cm9rZS13
aWR0aDogMTt9LmxpbmUwIHtzdHJva2U6ICMwMDdCQzM7fS5saW5lMSB7c3Ryb2tlOiAjRkYzMDNF
O30uYXhpcyB0ZXh0LC5heGlzLWxhYmVsLHRleHQubGVnZW5kIHtmb250OiAxMHB4IHNhbnMtc2Vy
aWY7fS5heGlzLWxhYmVsLnkwLC5heGlzLWxhYmVsLnkxIHt0ZXh0LWFuY2hvcjogZW5kO31yZWN0
LmxlZ2VuZCB7aGVpZ2h0OiAxMHB4O3dpZHRoOiAxMHB4O30ubGVnZW5kIHtjdXJzb3I6IHBvaW50
ZXI7fS53cmFwLXRleHQgdGV4dCB7dGV4dC1hbmNob3I6IHN0YXJ0IWltcG9ydGFudDt9LyogQ0hB
UlQgVE9PTFRJUCAqLy5jaGFydC10b29sdGlwLXdyYXAge2xlZnQ6IDA7cG9pbnRlci1ldmVudHM6
IG5vbmU7cG9zaXRpb246IGFic29sdXRlO3RvcDogMTBweDt6LWluZGV4OiAxMDt9LmNoYXJ0LXRv
b2x0aXAgey1tb3otYm94LXNoYWRvdzogN3B4IDdweCAxMnB4IC05cHggIzc3Nzc3Nzstd2Via2l0
LWJveC1zaGFkb3c6IDdweCA3cHggMTJweCAtOXB4ICM3Nzc3Nzc7YmFja2dyb3VuZC1jb2xvcjog
I2ZmZjtib3JkZXItY29sbGFwc2U6IGNvbGxhcHNlO2JvcmRlci1zcGFjaW5nOiAwO2JveC1zaGFk
b3c6IDdweCA3cHggMTJweCAtOXB4ICM3Nzc3Nzc7ZW1wdHktY2VsbHM6IHNob3c7b3BhY2l0eTog
MC45O30uY2hhcnQtdG9vbHRpcCB0ciB7Ym9yZGVyOiAxcHggc29saWQgI0NDQzt9LmNoYXJ0LXRv
b2x0aXAgdGgge2JhY2tncm91bmQtY29sb3I6ICNhYWE7Y29sb3I6ICNGRkY7Zm9udC1zaXplOiAx
NHB4O21heC13aWR0aDogMzgwcHg7b3ZlcmZsb3c6IGhpZGRlbjtwYWRkaW5nOiAycHggNXB4O3Rl
eHQtYWxpZ246IGxlZnQ7dGV4dC1vdmVyZmxvdzogZWxsaXBzaXM7d2hpdGUtc3BhY2U6IG5vd3Jh
cDt9LmNoYXJ0LXRvb2x0aXAgdGQge2JvcmRlci1sZWZ0OiAxcHggZG90dGVkICM5OTk7Zm9udC1z
aXplOiAxM3B4O3BhZGRpbmc6IDNweCA2cHg7fS5jaGFydC10b29sdGlwIHRkID4gc3BhbiB7ZGlz
cGxheTogaW5saW5lLWJsb2NrO2hlaWdodDogMTBweDttYXJnaW4tcmlnaHQ6IDZweDt3aWR0aDog
MTBweDt9LmNoYXJ0LXRvb2x0aXAgdGQudmFsdWUge3RleHQtYWxpZ246IHJpZ2h0O30uY2hhcnQt
dG9vbHRpcCAuYmx1ZSB7YmFja2dyb3VuZC1jb2xvcjogIzAwN0JDMzt9LmNoYXJ0LXRvb2x0aXAg
LnJlZCB7YmFja2dyb3VuZC1jb2xvcjogI0ZGMzAzRTt9LyogREFSSyBUSEVNRSAqLy5kYXJrIGgx
IHtjb2xvcjogcmdiYSgyNTUsIDI1NSwgMjU1LCAwLjYpO30uZGFyayBoMywuZGFyayBoNCwuZGFy
ayBoNSB7Y29sb3I6IHJnYmEoMjU1LDI1NSwyNTUsMC40KTt9LmRhcmsgLnRhYmxlLXJlc3BvbnNp
dmUge2JvcmRlcjogbm9uZTt9LmRhcmsgLndyYXAtcGFuZWwgPiBkaXYgPiB0YWJsZSB7Y29sb3I6
ICNEMkQyRDI7fS5kYXJrIC53cmFwLXBhbmVsIHRhYmxlIHRib2R5LnRib2R5LWRhdGEgdHIgdGQg
e2JvcmRlci1yaWdodDogbm9uZTt9LmRhcmsgLndyYXAtcGFuZWwgdGFibGUudGFibGUtaG92ZXI+
dGJvZHkudGJvZHktZGF0YT50cjpob3ZlciB7YmFja2dyb3VuZC1jb2xvcjogcmdiYSgyNTUsIDI1
NSwgMjU1LCAwLjA4KSAhaW1wb3J0YW50O30uZGFyayAuY29sLXRpdGxlIHtjb2xvcjogIzllOWU5
ZTt0ZXh0LXNoYWRvdzpub25lO30uZGFyayAuZ3JpZC1tb2R1bGUgaDMge2NvbG9yOiAjRkZGO30u
ZGFyayAuZHJvcGRvd24tbWVudT5saT5hIHtjb2xvcjogI0ZGRjt9LmRhcmsgZGl2LndyYXAtcGFu
ZWwgPiBkaXYge2NvbG9yOiAjRUVFO21hcmdpbi10b3A6IDEwcHg7cGFkZGluZzogMCAxMHB4O2Jv
cmRlci10b3A6IDFweCBzb2xpZCByZ2JhKDI1NSwgMjU1LCAyNTUsIDAuMTUpO30uZGFyayAud3Jh
cC1wYW5lbCB0YWJsZSAuY2VsbC1obC5kMSB7YmFja2dyb3VuZDogIzE2MTYxNjt9LmRhcmsgLndy
YXAtcGFuZWwgdGFibGUgLmNlbGwtaGwuZDIge2JhY2tncm91bmQ6ICMzYzNjM2M7fS5kYXJrIC53
cmFwLXBhbmVsIHRhYmxlIC5jZWxsLWhsLmQzIHtiYWNrZ3JvdW5kOiAjNWE1YTVhO30uZGFyayAu
d3JhcC1wYW5lbCB0YWJsZSAuY2VsbC1obC5kNCB7YmFja2dyb3VuZDogIzdlN2U3ZTt9LyogREFS
SyBCTFVFIFRIRU1FICovaHRtbC5kYXJrLmJsdWUsLmRhcmsuYmx1ZSBib2R5IHtiYWNrZ3JvdW5k
OiAjMjUyQjMwO30uZGFyay5ibHVlIC5jb250YWluZXIge2JhY2tncm91bmQ6ICMyNTJCMzA7fS5k
YXJrLmJsdWUgLnBhZ2UtaGVhZGVyIHtib3JkZXItYm90dG9tOiAxcHggc29saWQgIzNCNDQ0Qzt9
LmRhcmsuYmx1ZSAubGFiZWwtaW5mbyB7YmFja2dyb3VuZC1jb2xvcjogIzI1MkIzMDt9LmRhcmsu
Ymx1ZSBuYXYge2JvcmRlci1yaWdodDogMXB4IHNvbGlkICMxODFCMUY7YmFja2dyb3VuZDogIzFG
MjMyODt9LmRhcmsuYmx1ZSBkaXYud3JhcC1wYW5lbCA+IGRpdiB7YmFja2dyb3VuZDogIzFGMjMy
ODt9LmRhcmsuYmx1ZSAud3JhcC1wYW5lbCB0YWJsZSB0Zm9vdD50cj50aCB7Ym9yZGVyLXRvcDog
MXB4IGRvdHRlZCAjOTk5O30uZGFyay5ibHVlIC53cmFwLXBhbmVsIHRhYmxlIC50aGVhZC1taW4s
LmRhcmsuYmx1ZSAud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtYXZnLC5kYXJrLmJsdWUgLndyYXAt
cGFuZWwgdGFibGUgLnRoZWFkLW1heCB7YmFja2dyb3VuZDogIzFmMjMyODt9LmRhcmsuYmx1ZSAu
d3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtYXZnIHtib3JkZXItYm90dG9tOiAycHggc29saWQgIzk5
OTt9LmRhcmsuYmx1ZSAud3JhcC1wYW5lbCB0YWJsZT50aGVhZD50ci50aGVhZC1jb2xzIHtib3Jk
ZXItYm90dG9tOiAycHggc29saWQgIzk5OTt9LmRhcmsuYmx1ZSAud3JhcC1wYW5lbCB0YWJsZSB0
Ym9keS50Ym9keS1kYXRhIHRyLnNoYWRlZCB7YmFja2dyb3VuZC1jb2xvcjogIzE4MUIxRjt9LmRh
cmsuYmx1ZSAuZ3JheSB7Ym9yZGVyLXRvcDogNHB4IHNvbGlkICMzQjQ0NEM7fS5kYXJrLmJsdWUg
LmdyaWQtbW9kdWxlIHtiYWNrZ3JvdW5kOiAjMUYyMzI4O30uZGFyay5ibHVlIC5idG4tZGVmYXVs
dCB7Y29sb3I6ICM5RTlFOUU7YmFja2dyb3VuZC1jb2xvcjogIzFGMjMyODtib3JkZXItY29sb3I6
ICMzQjQ0NEM7fS5kYXJrLmJsdWUgLmJ0bi1kZWZhdWx0OmFjdGl2ZSwuZGFyay5ibHVlIC5idG4t
ZGVmYXVsdDpob3ZlciwuZGFyay5ibHVlIC5idG4tZGVmYXVsdC5hY3RpdmUsLmRhcmsuYmx1ZSAu
b3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi1kZWZhdWx0IHtjb2xvcjogIzNCNDQ0QztiYWNrZ3Jv
dW5kLWNvbG9yOiAjMUYyMzI4O2JvcmRlci1jb2xvcjogIzBGMTIxNDt9LmRhcmsuYmx1ZSAucGFn
aW5hdGlvbj4uZGlzYWJsZWQ+YSwuZGFyay5ibHVlIC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5hOmhv
dmVyLC5kYXJrLmJsdWUgLnBhZ2luYXRpb24+LmRpc2FibGVkPmE6Zm9jdXMge2NvbG9yOiAjNzc3
O30uZGFyay5ibHVlIC5wYWdpbmF0aW9uPmxpPmEge2JhY2tncm91bmQtY29sb3I6ICMxRjIzMjg7
Ym9yZGVyOiAxcHggc29saWQgIzNCNDQ0Qzt9LmRhcmsuYmx1ZSAucGFnaW5hdGlvbj5saT5hOmhv
dmVyLC5kYXJrLmJsdWUgLnBhZ2luYXRpb24+bGk+YTphY3RpdmUsLmRhcmsuYmx1ZSAucGFnaW5h
dGlvbj5saT5hOmZvY3VzIHtjb2xvcjogIzAzNzBCMDtiYWNrZ3JvdW5kLWNvbG9yOiAjMUYyMzI4
O2JvcmRlci1jb2xvcjogIzNCNDQ0Qzt9LmRhcmsuYmx1ZSAuZHJvcGRvd24tbWVudT5saT5hOmhv
dmVyLC5kYXJrLmJsdWUgLmRyb3Bkb3duLW1lbnU+bGk+YTpmb2N1cyB7Y29sb3I6ICNGRkY7YmFj
a2dyb3VuZC1jb2xvcjogIzNCNDQ0Qzt9LmRhcmsuYmx1ZSAuZHJvcGRvd24tbWVudSB7YmFja2dy
b3VuZC1jb2xvcjogIzI1MkIzMDt9LmRhcmsuYmx1ZTo6LXdlYmtpdC1zY3JvbGxiYXItdHJhY2ss
LmRhcmsuYmx1ZSAudGFibGUtcmVzcG9uc2l2ZTo6LXdlYmtpdC1zY3JvbGxiYXItdHJhY2sgey13
ZWJraXQtYm94LXNoYWRvdzogaW5zZXQgMCAwIDZweCByZ2JhKDAsMCwwLDAuMyk7YmFja2dyb3Vu
ZC1jb2xvcjogIzlFOUU5RTt9LmRhcmsuYmx1ZTo6LXdlYmtpdC1zY3JvbGxiYXIsLmRhcmsuYmx1
ZSAudGFibGUtcmVzcG9uc2l2ZTo6LXdlYmtpdC1zY3JvbGxiYXIge3dpZHRoOiAxMHB4O2hlaWdo
dDogMTBweDtiYWNrZ3JvdW5kLWNvbG9yOiAjOUU5RTlFO30uZGFyay5ibHVlOjotd2Via2l0LXNj
cm9sbGJhci10aHVtYiwuZGFyay5ibHVlIC50YWJsZS1yZXNwb25zaXZlOjotd2Via2l0LXNjcm9s
bGJhci10aHVtYiB7YmFja2dyb3VuZC1jb2xvcjogIzNCNDQ0Qzt9LmRhcmsuYmx1ZSAuY2hhcnQt
dG9vbHRpcCB7YmFja2dyb3VuZC1jb2xvcjogIzI1MkIzMDt9LmRhcmsuYmx1ZSAucmVwb3J0LXRp
dGxlIHtiYWNrZ3JvdW5kOiAjMUYyMzI4O30vKiBEQVJLIEdSRVkgVEhFTUUgKi9odG1sLmRhcmsu
Z3JheSwuZGFyay5ncmF5IGJvZHkge2JhY2tncm91bmQ6ICMyMTIxMjE7fS5kYXJrLmdyYXkgLmNv
bnRhaW5lciB7YmFja2dyb3VuZDogIzIxMjEyMTt9LmRhcmsuZ3JheSAucGFnZS1oZWFkZXIge2Jv
cmRlci1ib3R0b206IDFweCBzb2xpZCAjMzAzMDMwO30uZGFyay5ncmF5IC5sYWJlbC1pbmZvIHti
YWNrZ3JvdW5kLWNvbG9yOiAjMzAzMDMwO30uZGFyay5ncmF5IG5hdiB7Ym9yZGVyLXJpZ2h0OiAx
cHggc29saWQgIzM2MzczNztiYWNrZ3JvdW5kOiAjMUMxQzFDO30uZGFyay5ncmF5IGRpdi53cmFw
LXBhbmVsID4gZGl2IHtiYWNrZ3JvdW5kOiAjMUMxQzFDO30uZGFyay5ncmF5IC53cmFwLXBhbmVs
IHRhYmxlIHRmb290PnRyPnRoIHtib3JkZXItdG9wOiAxcHggZG90dGVkICM5OTk7fS5kYXJrLmdy
YXkgLndyYXAtcGFuZWwgdGFibGUgLnRoZWFkLW1pbiwuZGFyay5ncmF5IC53cmFwLXBhbmVsIHRh
YmxlIC50aGVhZC1hdmcsLmRhcmsuZ3JheSAud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtbWF4IHti
YWNrZ3JvdW5kOiAjMWMxYzFjO30uZGFyay5ncmF5IC53cmFwLXBhbmVsIHRhYmxlIC50aGVhZC1h
dmcge2JvcmRlci1ib3R0b206IDJweCBzb2xpZCAjOTk5O30uZGFyay5ncmF5IC53cmFwLXBhbmVs
IHRhYmxlPnRoZWFkPnRyLnRoZWFkLWNvbHMge2JvcmRlci1ib3R0b206IDJweCBzb2xpZCAjOTk5
O30uZGFyay5ncmF5IC53cmFwLXBhbmVsIHRhYmxlIHRib2R5LnRib2R5LWRhdGEgdHIuc2hhZGVk
IHtiYWNrZ3JvdW5kLWNvbG9yOiByZ2JhKDQ4LCA0OCwgNDgsIDAuNDgpO30uZGFyay5ncmF5IC5n
cmF5IHtib3JkZXItdG9wOiA0cHggc29saWQgIzMwMzAzMDt9LmRhcmsuZ3JheSAuZ3JpZC1tb2R1
bGUge2JhY2tncm91bmQ6ICMxQzFDMUM7fS5kYXJrLmdyYXkgLmJ0bi1kZWZhdWx0IHtjb2xvcjog
IzlFOUU5RTtiYWNrZ3JvdW5kLWNvbG9yOiAjMjEyMTIxO2JvcmRlci1jb2xvcjogIzMwMzAzMDt9
LmRhcmsuZ3JheSAuYnRuLWRlZmF1bHQ6YWN0aXZlLC5kYXJrLmdyYXkgLmJ0bi1kZWZhdWx0Omhv
dmVyLC5kYXJrLmdyYXkgLmJ0bi1kZWZhdWx0LmFjdGl2ZSwuZGFyay5ncmF5IC5vcGVuPi5kcm9w
ZG93bi10b2dnbGUuYnRuLWRlZmF1bHQge2NvbG9yOiAjMzYzNzM3O2JhY2tncm91bmQtY29sb3I6
ICMxQzFDMUM7Ym9yZGVyLWNvbG9yOiAjMEYxMjE0O30uZGFyay5ncmF5IC5wYWdpbmF0aW9uPi5k
aXNhYmxlZD5hLC5kYXJrLmdyYXkgLnBhZ2luYXRpb24+LmRpc2FibGVkPmE6aG92ZXIsLmRhcmsu
Z3JheSAucGFnaW5hdGlvbj4uZGlzYWJsZWQ+YTpmb2N1cyB7Y29sb3I6ICM3Nzc7fS5kYXJrLmdy
YXkgLnBhZ2luYXRpb24+bGk+YSB7YmFja2dyb3VuZC1jb2xvcjogIzIxMjEyMTtib3JkZXI6IDFw
eCBzb2xpZCAjMzAzMDMwO30uZGFyay5ncmF5IC5wYWdpbmF0aW9uPmxpPmE6aG92ZXIsLmRhcmsu
Z3JheSAucGFnaW5hdGlvbj5saT5hOmFjdGl2ZSwuZGFyay5ncmF5IC5wYWdpbmF0aW9uPmxpPmE6
Zm9jdXMge2NvbG9yOiAjMDM3MEIwO2JhY2tncm91bmQtY29sb3I6ICMyMTIxMjE7Ym9yZGVyLWNv
bG9yOiAjMzAzMDMwO30uZGFyay5ncmF5IC5kcm9wZG93bi1tZW51PmxpPmEge2NvbG9yOiAjRkZG
O30uZGFyay5ncmF5IC5kcm9wZG93bi1tZW51PmxpPmE6aG92ZXIsLmRhcmsuZ3JheSAuZHJvcGRv
d24tbWVudT5saT5hOmZvY3VzIHtjb2xvcjogI0ZGRjtiYWNrZ3JvdW5kLWNvbG9yOiAjMzAzMDMw
O30uZGFyay5ncmF5IC5kcm9wZG93bi1tZW51IHtiYWNrZ3JvdW5kLWNvbG9yOiAjMjEyMTIxO30u
ZGFyay5ncmF5Ojotd2Via2l0LXNjcm9sbGJhci10cmFjaywuZGFyay5ncmF5IC50YWJsZS1yZXNw
b25zaXZlOjotd2Via2l0LXNjcm9sbGJhci10cmFjayB7LXdlYmtpdC1ib3gtc2hhZG93OiBpbnNl
dCAwIDAgNnB4IHJnYmEoMCwwLDAsMC4zKTtiYWNrZ3JvdW5kLWNvbG9yOiAjOUU5RTlFO30uZGFy
ay5ncmF5Ojotd2Via2l0LXNjcm9sbGJhciwuZGFyay5ncmF5IC50YWJsZS1yZXNwb25zaXZlOjot
d2Via2l0LXNjcm9sbGJhciB7d2lkdGg6IDEwcHg7aGVpZ2h0OiAxMHB4O2JhY2tncm91bmQtY29s
b3I6ICM5RTlFOUU7fS5kYXJrLmdyYXk6Oi13ZWJraXQtc2Nyb2xsYmFyLXRodW1iLC5kYXJrLmdy
YXkgLnRhYmxlLXJlc3BvbnNpdmU6Oi13ZWJraXQtc2Nyb2xsYmFyLXRodW1iIHtiYWNrZ3JvdW5k
LWNvbG9yOiAjMzAzMDMwO30uZGFyay5ncmF5IC5jaGFydC10b29sdGlwIHtiYWNrZ3JvdW5kLWNv
bG9yOiAjMzAzMDMwO30uZGFyay5ncmF5IC5yZXBvcnQtdGl0bGUge2JhY2tncm91bmQ6ICMzMDMw
MzA7fS8qIERBUksgQ0hBUlRTICovLmRhcmsgLmFyZWEge29wYWNpdHk6IDAuMTt9LmRhcmsgLmxp
bmUwLC5kYXJrIC5saW5lMSB7c3Ryb2tlLXdpZHRoOiAyO30uZGFyayAuYXJlYTAsLmRhcmsgLmJh
cnMueTAgLmJhciwuZGFyayByZWN0LmxlZ2VuZC55MCB7ZmlsbDogIzAwN0JDMzt9LmRhcmsgLmFy
ZWExLC5kYXJrIC5iYXJzLnkxIC5iYXIsLmRhcmsgLnBvaW50cy55MSwuZGFyayByZWN0LmxlZ2Vu
ZC55MSB7ZmlsbDogI0ZGMzAzRTt9LmRhcmsgLnBvaW50cy55MCB7ZmlsbDogIzAwRDRFMTt9LmRh
cmsgLmxpbmUwIHtzdHJva2U6ICMwMDdCQzM7fS5kYXJrIC5saW5lMSB7c3Ryb2tlOiAjRkYzMDNF
O30uZGFyayAuZ3JpZC55IC50aWNrIGxpbmUsLmRhcmsgLmdyaWQueCAudGljayBsaW5lIHtzdHJv
a2U6ICM0NDQ3NEI7c3Ryb2tlLWRhc2hhcnJheTogMSAxO30uZGFyayAuYXhpcyB0ZXh0LC5kYXJr
IC5heGlzLWxhYmVsLC5kYXJrIHRleHQubGVnZW5kIHtmaWxsOiAjOUU5RTlFO30uZGFyayAuYXhp
cyBwYXRoIHtzdHJva2U6ICM5OTk5OTk7fS5kYXJrIC5heGlzLnggLnRpY2sgbGluZSwuZGFyayAu
YXhpcy55MCAudGljayBsaW5lLC5kYXJrIC5heGlzLnkxIC50aWNrIGxpbmUsLmRhcmsgLmdyaWQu
eSAudGljazpmaXJzdC1jaGlsZCBsaW5lIHtzdHJva2U6ICMzQjQ0NEM7fS5kYXJrIC5jaGFydC10
b29sdGlwIHRoIHtiYWNrZ3JvdW5kLWNvbG9yOiAjMWMxYzFjO30uZGFyayAuY2hhcnQtdG9vbHRp
cCB0ciB7Ym9yZGVyOiAxcHggc29saWQgIzM2MzczNzt9LyogREFSSyBQVVJQTEUgVEhFTUUgKi9o
dG1sLmRhcmsucHVycGxlLC5kYXJrLnB1cnBsZSBib2R5IHtiYWNrZ3JvdW5kOiAjMWUxZTJmO30u
ZGFyay5wdXJwbGUgLmNvbnRhaW5lciB7YmFja2dyb3VuZDogIzFlMWUyZjt9LmRhcmsucHVycGxl
IC5wYWdlLWhlYWRlciB7Ym9yZGVyLWJvdHRvbTogMXB4IHNvbGlkICMyYjM1NTM7fS5kYXJrLnB1
cnBsZSAubGFiZWwtaW5mbyB7YmFja2dyb3VuZC1jb2xvcjogIzE4MTgyMzt9LmRhcmsucHVycGxl
IG5hdiB7Ym9yZGVyLXJpZ2h0OiAxcHggc29saWQgI2UxNGVjYTtiYWNrZ3JvdW5kOiAjMTgxODIz
O30uZGFyay5wdXJwbGUgZGl2LndyYXAtcGFuZWwgPiBkaXYge2JhY2tncm91bmQ6ICMyNzI5M2Q7
Ym9yZGVyLXRvcDogMXB4IHNvbGlkICMyYjM1NTM7fS5kYXJrLnB1cnBsZSAud3JhcC1wYW5lbCB0
YWJsZSB0Ym9keS50Ym9keS1kYXRhIHRyLnNoYWRlZCB7YmFja2dyb3VuZC1jb2xvcjogIzFlMWUy
Zjt9LmRhcmsucHVycGxlIC53cmFwLXBhbmVsIHRhYmxlIHRmb290PnRyPnRoIHtib3JkZXItdG9w
OiAxcHggZG90dGVkICM5OTk7fS5kYXJrLnB1cnBsZSAud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQt
bWluLC5kYXJrLnB1cnBsZSAud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtYXZnLC5kYXJrLnB1cnBs
ZSAud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtbWF4IHtiYWNrZ3JvdW5kOiAjMjcyOTNkO30uZGFy
ay5wdXJwbGUgLndyYXAtcGFuZWwgdGFibGUgLnRoZWFkLWF2ZyB7Ym9yZGVyLWJvdHRvbTogMnB4
IHNvbGlkICM5OTk7fS5kYXJrLnB1cnBsZSAud3JhcC1wYW5lbCB0YWJsZT50aGVhZD50ci50aGVh
ZC1jb2xzIHtib3JkZXItYm90dG9tOiAycHggc29saWQgIzk5OTt9LmRhcmsucHVycGxlIC5ncmF5
IHtib3JkZXItdG9wOiA0cHggc29saWQgIzJiMzU1Mzt9LmRhcmsucHVycGxlIC5yZWQge2JvcmRl
ci10b3A6IDRweCBzb2xpZCAjZmQ1ZDkzO30uZGFyay5wdXJwbGUgLmdyZWVuIHtib3JkZXItdG9w
OiA0cHggc29saWQgIzAwZjJjMzt9LmRhcmsucHVycGxlIC5ibHVlIHtib3JkZXItdG9wOiA0cHgg
c29saWQgIzFmOGVmMTt9LmRhcmsucHVycGxlIGgzLCAuZGFyay5wdXJwbGUgaDQsIC5kYXJrLnB1
cnBsZSBoNSB7Y29sb3I6ICM5YTlhOWE7fS5kYXJrLnB1cnBsZSAuZ3JpZC1tb2R1bGUge2JhY2tn
cm91bmQ6ICMyNzI5M2Q7fS5kYXJrLnB1cnBsZSAuZ3JpZC1tb2R1bGUgaDMge2NvbG9yOiAjRkZG
O30uZGFyay5wdXJwbGUgLmJ0bi1kZWZhdWx0IHtjb2xvcjogIzlFOUU5RTtiYWNrZ3JvdW5kLWNv
bG9yOiAjMWUxZTJmO2JvcmRlci1jb2xvcjogIzJiMzU1Mzt9LmRhcmsucHVycGxlIC5idG4tZGVm
YXVsdDphY3RpdmUsLmRhcmsucHVycGxlIC5idG4tZGVmYXVsdDpob3ZlciwuZGFyay5wdXJwbGUg
LmJ0bi1kZWZhdWx0LmFjdGl2ZSwuZGFyay5wdXJwbGUgLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5i
dG4tZGVmYXVsdCB7Y29sb3I6ICM1OTU5NWY7YmFja2dyb3VuZC1jb2xvcjogIzFlMWUyZjtib3Jk
ZXItY29sb3I6ICMyYjM1NTM7fS5kYXJrLnB1cnBsZSAucGFnaW5hdGlvbj4uZGlzYWJsZWQ+YSwu
ZGFyay5wdXJwbGUgLnBhZ2luYXRpb24+LmRpc2FibGVkPmE6aG92ZXIsLmRhcmsucHVycGxlIC5w
YWdpbmF0aW9uPi5kaXNhYmxlZD5hOmZvY3VzIHtjb2xvcjogIzc3Nzt9LmRhcmsucHVycGxlIC5w
YWdpbmF0aW9uPmxpPmEge2JhY2tncm91bmQtY29sb3I6ICMxZTFlMmY7Ym9yZGVyOiAxcHggc29s
aWQgIzNCNDQ0Qzt9LmRhcmsucHVycGxlIC5wYWdpbmF0aW9uPmxpPmE6aG92ZXIsLmRhcmsucHVy
cGxlIC5wYWdpbmF0aW9uPmxpPmE6YWN0aXZlLC5kYXJrLnB1cnBsZSAucGFnaW5hdGlvbj5saT5h
OmZvY3VzIHtjb2xvcjogIzAzNzBCMDtiYWNrZ3JvdW5kLWNvbG9yOiAjMTgxODIzO30uZGFyay5w
dXJwbGUgLmRyb3Bkb3duLW1lbnU+bGk+YTpob3ZlciwuZGFyay5wdXJwbGUgLmRyb3Bkb3duLW1l
bnU+bGk+YTpmb2N1cyB7Y29sb3I6ICNGRkY7YmFja2dyb3VuZC1jb2xvcjogIzE4MTgyMzt9LmRh
cmsucHVycGxlIC5kcm9wZG93bi1tZW51IHtiYWNrZ3JvdW5kLWNvbG9yOiAjMWUxZTJmO30uZGFy
ay5wdXJwbGU6Oi13ZWJraXQtc2Nyb2xsYmFyLXRyYWNrLC5kYXJrLnB1cnBsZSAudGFibGUtcmVz
cG9uc2l2ZTo6LXdlYmtpdC1zY3JvbGxiYXItdHJhY2sgey13ZWJraXQtYm94LXNoYWRvdzogaW5z
ZXQgMCAwIDZweCByZ2JhKDAsMCwwLDAuMyk7YmFja2dyb3VuZC1jb2xvcjogIzlFOUU5RTt9LmRh
cmsucHVycGxlOjotd2Via2l0LXNjcm9sbGJhciwuZGFyay5wdXJwbGUgLnRhYmxlLXJlc3BvbnNp
dmU6Oi13ZWJraXQtc2Nyb2xsYmFyIHt3aWR0aDogMTBweDtoZWlnaHQ6IDEwcHg7YmFja2dyb3Vu
ZC1jb2xvcjogIzlFOUU5RTt9LmRhcmsucHVycGxlOjotd2Via2l0LXNjcm9sbGJhci10aHVtYiwu
ZGFyay5wdXJwbGUgLnRhYmxlLXJlc3BvbnNpdmU6Oi13ZWJraXQtc2Nyb2xsYmFyLXRodW1iIHti
YWNrZ3JvdW5kLWNvbG9yOiAjMWUxZTJmO30uZGFyay5wdXJwbGUgLmNoYXJ0LXRvb2x0aXAge2Jh
Y2tncm91bmQtY29sb3I6ICMxODE4MjM7fS5kYXJrLnB1cnBsZSAucmVwb3J0LXRpdGxlIHtiYWNr
Z3JvdW5kOiAjMTgxODIzO30uZGFyay5wdXJwbGUgLmFyZWEwLC5kYXJrLnB1cnBsZSAuYmFycy55
MCAuYmFyLC5kYXJrLnB1cnBsZSByZWN0LmxlZ2VuZC55MCB7ZmlsbDogIzAwN0JDMzt9LmRhcmsu
cHVycGxlIC5hcmVhMSwuZGFyay5wdXJwbGUgLmJhcnMueTEgLmJhciwuZGFyay5wdXJwbGUgLnBv
aW50cy55MSwuZGFyay5wdXJwbGUgcmVjdC5sZWdlbmQueTEge2ZpbGw6ICNkMDQ4YjY7fS5kYXJr
LnB1cnBsZSAucG9pbnRzLnkwIHtmaWxsOiAjMDBENEUxO30uZGFyay5wdXJwbGUgLmxpbmUwIHtz
dHJva2U6ICMwMDdCQzM7fS5kYXJrLnB1cnBsZSAubGluZTEge3N0cm9rZTogI2QwNDhiNjt9PC9z
dHlsZT48L2hlYWQ+PGJvZHk+PG5hdiBjbGFzcz0naGlkZGVuLXhzIGhpZGRlbi1zbSBoaWRlJz48
L25hdj48aSBjbGFzcz0nc3Bpbm5lciBmYSBmYS1jaXJjbGUtby1ub3RjaCBmYS1zcGluIGZhLTN4
IGZhLWZ3Jz48L2k+PGRpdiBjbGFzcz0nY29udGFpbmVyIGhpZGUnPjxkaXYgY2xhc3M9J3dyYXAt
aGVhZGVyJz48ZGl2IGNsYXNzPSdyb3cgcm93LW9mZmNhbnZhcyByb3ctb2ZmY2FudmFzLXJpZ2h0
Jz48ZGl2IGNsYXNzPSdjb2wtbWQtMTInPjxkaXYgY2xhc3M9J3BhZ2UtaGVhZGVyIGNsZWFyZml4
Jz48ZGl2IGNsYXNzPSdwdWxsLXJpZ2h0Jz48aDQ+PHNwYW4gY2xhc3M9J2xhYmVsIGxhYmVsLWlu
Zm8nIHN0eWxlPSdkaXNwbGF5OmJsb2NrJz48c3BhbiBjbGFzcz0naGlkZGVuLXhzJz5MYXN0IFVw
ZGF0ZWQ6IDwvc3Bhbj48c3BhbiBjbGFzcz0nbGFzdC11cGRhdGVkJz4yMDI0LTAyLTE5IDE0OjU4
OjQ5ICswODAwPC9zcGFuPjwvc3Bhbj48L2g0PjwvZGl2PjxoMSBjbGFzcz0naC1kYXNoYm9hcmQn
PjxzcGFuIGNsYXNzPSdoaWRkZW4teHMgaGlkZGVuLXNtJz48aSBjbGFzcz0nZmEgZmEtdGFjaG9t
ZXRlcic+PC9pPiBEYXNoYm9hcmQ8L3NwYW4+PHNwYW4gY2xhc3M9J3Zpc2libGUteHMgdmlzaWJs
ZS1zbSc+PGkgY2xhc3M9J2ZhIGZhLWJhcnMgbmF2LW1pbmliYXJzJz48L2k+PGkgY2xhc3M9J2Zh
IGZhLWNpcmNsZSBuYXYtd3Mtc3RhdHVzIG1pbmknPjwvaT48L3NwYW4+PC9oMT48ZGl2IGNsYXNz
PSdyZXBvcnQtdGl0bGUnPjwvZGl2PjwvZGl2PjxkaXYgY2xhc3M9J3dyYXAtZ2VuZXJhbCc+PC9k
aXY+PC9kaXY+PC9kaXY+PC9kaXY+PGRpdiBjbGFzcz0nd3JhcC1wYW5lbHMnPjwvZGl2PjwvZGl2
PjwhLS0gVFBMIEdlbmVyYWwgIC0tPjxzY3JpcHQgaWQ9InRwbC1nZW5lcmFsIiB0eXBlPSJ0ZXh0
L3RlbXBsYXRlIj48aDQgY2xhc3M9ImhpZGRlbi14cyBnaGVhZGVyIj57e2hlYWR9fTxzcGFuIGNs
YXNzPSJwdWxsLXJpZ2h0Ij57eyNmcm9tfX08c3BhbiBjbGFzcz0iZnJvbSI+e3tmcm9tfX08L3Nw
YW4+e3svZnJvbX19e3sjdG99fSAmIzgyMTI7IDxzcGFuIGNsYXNzPSJ0byI+e3t0b319PC9zcGFu
Pnt7L3RvfX08L3NwYW4+PC9oND48aDUgY2xhc3M9InZpc2libGUteHMgaGlkZGVuLXNtIGhpZGRl
bi1tZCBoaWRkZW4tbGcgZ2hlYWRlciI+e3toZWFkfX0mbmJzcDsmbmJzcDt7eyNmcm9tfX08c3Bh
biBjbGFzcz0iZnJvbSI+e3tmcm9tfX08L3NwYW4+e3svZnJvbX19e3sjdG99fSAmIzgyMTI7IDxz
cGFuIGNsYXNzPSJ0byI+e3t0b319PC9zcGFuPnt7L3RvfX08L2g1PjxkaXYgY2xhc3M9IndyYXAt
Z2VuZXJhbC1pdGVtcyI+PC9kaXY+PC9zY3JpcHQ+PCEtLSBUUEwgR2VuZXJhbCBJdGVtcyAtLT48
c2NyaXB0IGlkPSJ0cGwtZ2VuZXJhbC1pdGVtcyIgdHlwZT0idGV4dC90ZW1wbGF0ZSI+PGRpdiBj
bGFzcz0iY29sLW1kLTIiPjxkaXYgY2xhc3M9ImdyaWQtbW9kdWxlIHt7I2NsYXNzTmFtZX19e3tj
bGFzc05hbWV9fXt7L2NsYXNzTmFtZX19e3teY2xhc3NOYW1lfX1ncmF5e3svY2xhc3NOYW1lfX0i
PjxkaXYgY2xhc3M9ImNvbC10aXRsZSI+PGkgY2xhc3M9ImZhIGZhLWJhci1jaGFydCI+PC9pPiB7
eyNsYWJlbH19e3tsYWJlbH19e3svbGFiZWx9fTwvZGl2PjxoMyBpZD0ie3tpZH19IiBzdHlsZT0i
cGFkZGluZy10b3A6IDA7Ij57e3ZhbHVlfX08L2gzPjwvZGl2PjwvZGl2Pjwvc2NyaXB0PjwhLS0g
VFBMIFBhbmVsIFRhYmxlIC0tPjxzY3JpcHQgaWQ9InRwbC10YWJsZS1yb3ciIHR5cGU9InRleHQv
dGVtcGxhdGUiPnt7I3Jvd3N9fTx0ciBjbGFzcz0ie3sjY2xhc3NOYW1lfX17e2NsYXNzTmFtZX19
e3svY2xhc3NOYW1lfX0ge3sjaGFzU3ViSXRlbXN9fXt7I2l0ZW1zfX1leHBhbmRhYmxle3svaXRl
bXN9fXt7L2hhc1N1Ykl0ZW1zfX0iIHt7I2lkeH19ZGF0YS1waWQ9Int7aWR4fX0ie3svaWR4fX0g
ZGF0YS1wYW5lbD0ie3twYW5lbH19IiB7eyNrZXl9fWRhdGEta2V5PSJ7e2tleX19Int7L2tleX19
Pnt7I2hhc1N1Ykl0ZW1zfX08dGQgY2xhc3M9InJvdy1leHBhbmRhYmxlIHRleHQtY2VudGVyIHt7
I2l0ZW1zfX1jbGlja2FibGV7ey9pdGVtc319Ij57eyNpdGVtc319PGkgY2xhc3M9ImZhIHt7I2V4
cGFuZGVkfX1mYS1jYXJldC1kb3due3svZXhwYW5kZWR9fXt7XmV4cGFuZGVkfX1mYSBmYS1jYXJl
dC1yaWdodHt7L2V4cGFuZGVkfX0iPjwvaT57ey9pdGVtc319e3teaXRlbXN9fTxpPjwvaT57ey9p
dGVtc319PC90ZD57ey9oYXNTdWJJdGVtc319PHRkIGNsYXNzPSJyb3ctaWR4IHRleHQtcmlnaHQi
Pnt7I2lkeH19e3tpZHh9fXt7L2lkeH19PC90ZD57eyNjZWxsc319PHRkIGNsYXNzPSJ7e2NsYXNz
TmFtZX19IiB7eyNjb2xzcGFufX1jb2xzcGFuPSJ7e2NvbHNwYW59fSJ7ey9jb2xzcGFufX0+PHNw
YW4gY2xhc3M9InZhbHVlIj57e3t2YWx1ZX19fTwvc3Bhbj57eyNwZXJjZW50fX08c3BhbiBjbGFz
cz0icGVyY2VudCI+ICh7e3BlcmNlbnR9fSk8L3NwYW4+e3svcGVyY2VudH19PC90ZD57ey9jZWxs
c319PC90cj57ey9yb3dzfX08L3NjcmlwdD48IS0tIFRQTCBQYW5lbCBUYWJsZSBNZXRhIC0tPjxz
Y3JpcHQgaWQ9InRwbC10YWJsZS1yb3ctbWV0YSIgdHlwZT0idGV4dC90ZW1wbGF0ZSI+e3sjcm93
fX17eyNoYXNTdWJJdGVtc319PHRoIGNsYXNzPSIiPjwvdGg+e3svaGFzU3ViSXRlbXN9fTx0aCBj
bGFzcz0ibWV0YS1sYWJlbCI+e3trZXl9fS48L3RoPnt7I2NlbGxzfX08dGggY2xhc3M9Int7Y2xh
c3NOYW1lfX0iIHt7I2NvbHNwYW59fWNvbHNwYW49Int7Y29sc3Bhbn19Int7L2NvbHNwYW59fT57
eyN2YWx1ZX19PHNwYW4+e3t2YWx1ZX19IHt7I3BlcmNlbnR9fTxzcGFuIGNsYXNzPSJ0ZXh0LW11
dGVkIj4oe3twZXJjZW50fX0lKTwvc3Bhbj57ey9wZXJjZW50fX08L3NwYW4+e3svdmFsdWV9fXt7
XnZhbHVlfX08c3BhbiBjbGFzcz0idGV4dC1tdXRlZCI+Jm1kYXNoOzwvc3Bhbj57ey92YWx1ZX19
PC90aD57ey9jZWxsc319e3svcm93fX08L3NjcmlwdD48IS0tIFRQTCBQYW5lbCBUYWJsZSBUb3Rh
bHMgLS0+PHNjcmlwdCBpZD0idHBsLXRhYmxlLXJvdy10b3RhbHMiIHR5cGU9InRleHQvdGVtcGxh
dGUiPnt7I3Jvd319e3sjaGFzU3ViSXRlbXN9fTx0ZCBjbGFzcz0iIj48L3RkPnt7L2hhc1N1Ykl0
ZW1zfX08dGQgY2xhc3M9Im1ldGEtbGFiZWwiPlRvdC48L3RkPnt7I2NlbGxzfX08dGQgY2xhc3M9
Int7Y2xhc3NOYW1lfX0iIHt7I2NvbHNwYW59fWNvbHNwYW49Int7Y29sc3Bhbn19Int7L2NvbHNw
YW59fT57eyNsYWJlbH19PHNwYW4+e3tsYWJlbH19PC9zcGFuPiZuYnNwO3t7L2xhYmVsfX08c3Bh
bj57e3ZhbHVlfX08L3NwYW4+PC90ZD57ey9jZWxsc319e3svcm93fX08L3NjcmlwdD48IS0tIFRQ
TCBUYWJsZSB0aGVhZCAtLT48c2NyaXB0IGlkPSJ0cGwtdGFibGUtdGhlYWQiIHR5cGU9InRleHQv
dGVtcGxhdGUiPnt7I2hhc1N1Ykl0ZW1zfX08dGg+PC90aD57ey9oYXNTdWJJdGVtc319PHRoPiM8
L3RoPnt7I2l0ZW1zfX08dGggY2xhc3M9Int7ZGF0YVR5cGV9fSB7eyNrZXl9fXNvcnRhYmxle3sv
a2V5fX0iIGRhdGEta2V5PSJ7e2tleX19IiB7eyNzb3J0fX1kYXRhLW9yZGVyPSJ7eyNhc2N9fWFz
Y3t7L2FzY319e3teYXNjfX1kZXNje3svYXNjfX0ie3svc29ydH19Pnt7bGFiZWx9fSA8aSBjbGFz
cz0iZmEgZmEte3tec29ydH19c29ydHt7L3NvcnR9fXt7I3NvcnR9fXt7I2FzY319Y2FyZXQtdXB7
ey9hc2N9fXt7XmFzY319Y2FyZXQtZG93bnt7L2FzY319e3svc29ydH19Ij48L2k+PC90aD57ey9p
dGVtc319PC9zY3JpcHQ+PCEtLSBUUEwgUGFuZWwgT3B0aW9ucyBEcm9wRG93biAtLT48c2NyaXB0
IGlkPSJ0cGwtcGFuZWwtb3B0cyIgdHlwZT0idGV4dC90ZW1wbGF0ZSI+PGxpIGNsYXNzPSJkcm9w
ZG93bi1oZWFkZXIiPnt7IGxhYmVscy5jaGFydF9vcHRzIH19PC9saT48bGk+PGEgaHJlZj0iamF2
YXNjcmlwdDp2b2lkKDApOyIgZGF0YS1wYW5lbD0ie3tpZH19IiBkYXRhLWNoYXJ0PSJ7e3Nob3dD
aGFydH19Ij48aSBjbGFzcz0iZmEgZmEte3sjc2hvd0NoYXJ0fX1jaGVjay17ey9zaG93Q2hhcnR9
fXNxdWFyZS1vIj48L2k+IHt7bGFiZWxzLmNoYXJ0fX08L2E+PC9saT57eyNwbG90Lmxlbmd0aH19
PGxpIGNsYXNzPSJkcm9wZG93bi1oZWFkZXIiPnt7IGxhYmVscy50eXBlIH19PC9saT48bGk+PGEg
aHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgZGF0YS1wYW5lbD0ie3tpZH19IiBkYXRhLWNoYXJ0
LXR5cGU9ImFyZWEtc3BsaW5lIj48aSBjbGFzcz0iZmEgZmEtY2lyY2xle3teYXJlYS1zcGxpbmV9
fS1ve3svYXJlYS1zcGxpbmV9fSI+PC9pPiB7e2xhYmVscy5hcmVhX3NwbGluZX19PC9hPjwvbGk+
PGxpPjxhIGhyZWY9ImphdmFzY3JpcHQ6dm9pZCgwKTsiIGRhdGEtcGFuZWw9Int7aWR9fSIgZGF0
YS1jaGFydC10eXBlPSJiYXIiPjxpIGNsYXNzPSJmYSBmYS1jaXJjbGV7e15iYXJ9fS1ve3svYmFy
fX0iPjwvaT4ge3tsYWJlbHMuYmFyfX08L2E+PC9saT48bGkgY2xhc3M9ImRyb3Bkb3duLWhlYWRl
ciI+e3tsYWJlbHMucGxvdF9tZXRyaWN9fTwvbGk+e3sjcGxvdH19PGxpPjxhIGhyZWY9ImphdmFz
Y3JpcHQ6dm9pZCgwKTsiIGRhdGEtcGFuZWw9Int7aWR9fSIgZGF0YS1wbG90PSJ7e2NsYXNzTmFt
ZX19IiBjbGFzcz0icGFuZWwtcGxvdC17e2NsYXNzTmFtZX19Ij48aSBjbGFzcz0iZmEgZmEtY2ly
Y2xle3tec2VsZWN0ZWR9fS1ve3svc2VsZWN0ZWR9fSI+PC9pPiB7e2xhYmVsfX08L2E+PC9saT57
ey9wbG90fX17ey9wbG90Lmxlbmd0aH19PGxpIGNsYXNzPSJkcm9wZG93bi1oZWFkZXIiPnt7bGFi
ZWxzLnRhYmxlX2NvbHVtbnN9fTwvbGk+e3sjaXRlbXN9fTxsaT48YSBocmVmPSJqYXZhc2NyaXB0
OnZvaWQoMCk7IiBkYXRhLXBhbmVsPSJ7e2lkfX0iIGRhdGEtbWV0cmljPSJ7e2tleX19Ij48aSBj
bGFzcz0iZmEgZmEte3teaGlkZX19Y2hlY2ste3svaGlkZX19c3F1YXJlLW8iPjwvaT4ge3tsYWJl
bH19PC9hPjwvbGk+e3svaXRlbXN9fTwvc2NyaXB0PjwhLS0gVFBMIFRhYmxlIGNvbGdyb3VwIC0t
PjxzY3JpcHQgaWQ9InRwbC10YWJsZS1jb2xncm91cCIgdHlwZT0idGV4dC90ZW1wbGF0ZSI+e3sj
aGFzU3ViSXRlbXN9fTxjb2wgc3R5bGU9IndpZHRoOiAyJTsiPiA8IS0tIHJpZ2h0LWNhcmV0IC0t
Pnt7L2hhc1N1Ykl0ZW1zfX08Y29sIHN0eWxlPSJ3aWR0aDogMyU7Ij4gPCEtLSByb3cgIyAtLT57
eyNpdGVtc319PGNvbCBzdHlsZT0id2lkdGg6e3tjb2xXaWR0aH19Ij57ey9pdGVtc319PC9zY3Jp
cHQ+PCEtLSBUUEwgUGFuZWwgLS0+PHNjcmlwdCBpZD0idHBsLXBhbmVsIiB0eXBlPSJ0ZXh0L3Rl
bXBsYXRlIj48ZGl2IGNsYXNzPSJyb3ciPjxkaXYgY2xhc3M9ImNvbC1tZC0xMiI+PGRpdiBjbGFz
cz0iZm9ybS1ncm91cCBjbGVhcmZpeCBwYW5lbC1oZWFkZXIiPjxoNCBjbGFzcz0icHVsbC1sZWZ0
IGhpZGRlbi14cyBnaGVhZGVyIiBpZD0ie3tpZH19Ij57e2hlYWR9fTxicj48c21hbGw+e3tkZXNj
fX08L3NtYWxsPjwvaDQ+PGg1IGNsYXNzPSJwdWxsLWxlZnQgdmlzaWJsZS14cyBoaWRkZW4tc20g
aGlkZGVuLW1kIGhpZGRlbi1sZyBnaGVhZGVyIiBpZD0ie3tpZH19Ij57e2hlYWR9fTxicj48c21h
bGw+e3tkZXNjfX08L3NtYWxsPjwvaDU+PGRpdiBjbGFzcz0icGFuZWwtcGxvdC13cmFwIj48ZGl2
IGNsYXNzPSJkcm9wZG93biI+PGJ1dHRvbiBjbGFzcz0iYnRuIGJ0bi1kZWZhdWx0IGJ0bi1zbSBk
cm9wZG93bi10b2dnbGUiIHR5cGU9ImJ1dHRvbiIgZGF0YS10b2dnbGU9ImRyb3Bkb3duIiBhcmlh
LWhhc3BvcHVwPSJ0cnVlIiBhcmlhLWV4cGFuZGVkPSJ0cnVlIiBkYXRhLXBhbmVsPSJ7e2lkfX0i
PjxpIGNsYXNzPSJmYSBmYS1nZWFyIj48L2k+IHt7bGFiZWxzLnBhbmVsX29wdHN9fSA8c3BhbiBj
bGFzcz0iZmEgZmEtY2FyZXQtZG93biI+PC9zcGFuPjwvYnV0dG9uPjx1bCBjbGFzcz0iZHJvcGRv
d24tbWVudSBkcm9wZG93bi1tZW51LXJpZ2h0IHBhbmVsLW9wdHMte3tpZH19Ij48L3VsPjwvZGl2
PjwvZGl2PjwvZGl2PjwvZGl2PjwvZGl2Pnt7I2NoYXJ0fX08ZGl2IGNsYXNzPSJyb3ciPjxkaXYg
Y2xhc3M9ImNvbC1tZC0xMiI+PGRpdiBpZD0iY2hhcnQte3tpZH19IiBjbGFzcz0iY2hhcnQtd3Jh
cCI+PC9kaXY+PC9kaXY+PC9kaXY+e3svY2hhcnR9fXt7I3RhYmxlfX08ZGl2IGNsYXNzPSJyb3cg
Y2xlYXJmaXggdGFibGUtd3JhcHBlciB7eyNhdXRvSGlkZVRhYmxlc319aGlkZGVuLXhze3svYXV0
b0hpZGVUYWJsZXN9fSI+PGRpdiBjbGFzcz0iY29sLW1kLTEyIj48ZGl2IGNsYXNzPSJ0YWJsZS1y
ZXNwb25zaXZlIj48dGFibGUgZGF0YS1wYW5lbD0ie3tpZH19IiBjbGFzcz0idGFibGUgdGFibGUt
Ym9yZGVybGVzcyB0YWJsZS1ob3ZlciB0YWJsZS17e2lkfX0iPjxjb2xncm91cD48L2NvbGdyb3Vw
Pjx0aGVhZCA+PHRyIGNsYXNzPSJ0aGVhZC1jb2xzIj48L3RyPjx0ciBjbGFzcz0idGhlYWQtbWlu
Ij48L3RyPjx0ciBjbGFzcz0idGhlYWQtbWF4Ij48L3RyPjx0ciBjbGFzcz0idGhlYWQtYXZnIj48
L3RyPjwvdGhlYWQ+PHRib2R5IGNsYXNzPSJ0Ym9keS1kYXRhIj48L3Rib2R5Pjx0Zm9vdCBjbGFz
cz0idGZvb3QtdG90YWxzIj48dHIgY2xhc3M9InRmb290LXRvdGFscyI+PC90cj48L3Rmb290Pjwv
dGFibGU+PC9kaXY+PHVsIGNsYXNzPSJwYWdpbmF0aW9uIHBhZ2luYXRpb24tc20gcHVsbC1sZWZ0
Ij48bGkgY2xhc3M9ImRpc2FibGVkIj48YSBjbGFzcz0icGFuZWwtZmlyc3QiIGhyZWY9ImphdmFz
Y3JpcHQ6dm9pZCgwKTsiIGFyaWEtbGFiZWw9Int7bGFiZWxzLmZpcnN0fX0iIGRhdGEtcGFuZWw9
Int7aWR9fSIgdGl0bGU9Int7bGFiZWxzLmZpcnN0fX0iPjxpIGNsYXNzPSJmYSBmYS1hbmdsZS1k
b3VibGUtbGVmdCI+PC9pPjwvYT48L2xpPjxsaSBjbGFzcz0iZGlzYWJsZWQiPjxhIGNsYXNzPSJw
YW5lbC1wcmV2IiBocmVmPSJqYXZhc2NyaXB0OnZvaWQoMCk7IiBhcmlhLWxhYmVsPSJ7e2xhYmVs
cy5wcmV2aW91c319IiBkYXRhLXBhbmVsPSJ7e2lkfX0iIHRpdGxlPSJ7e2xhYmVscy5wcmV2aW91
c319Ij48aSBjbGFzcz0iZmEgZmEtYW5nbGUtbGVmdCI+PC9pPjwvYT48L2xpPjxsaT48YSBjbGFz
cz0icGFuZWwtbmV4dCIgaHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgYXJpYS1sYWJlbD0ie3ts
YWJlbHMubmV4dH19IiBkYXRhLXBhbmVsPSJ7e2lkfX0iIHRpdGxlPSJ7e2xhYmVscy5uZXh0fX0i
PjxpIGNsYXNzPSJmYSBmYS1hbmdsZS1yaWdodCI+PC9pPjwvYT48L2xpPjxsaT48YSBjbGFzcz0i
cGFuZWwtbGFzdCIgaHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgYXJpYS1sYWJlbD0ie3tsYWJl
bHMubGFzdH19IiBkYXRhLXBhbmVsPSJ7e2lkfX0iIHRpdGxlPSJ7e2xhYmVscy5sYXN0fX0iPjxp
IGNsYXNzPSJmYSBmYS1hbmdsZS1kb3VibGUtcmlnaHQiPjwvaT48L2E+PC9saT48L3VsPjwvZGl2
PjwvZGl2Pnt7L3RhYmxlfX08L3NjcmlwdD48IS0tIFRQTCBOYXYgQmFyIHdyYXBwZXIgLS0+PHNj
cmlwdCBpZD0idHBsLW5hdi13cmFwIiB0eXBlPSJ0ZXh0L3RlbXBsYXRlIj48ZGl2IGNsYXNzPSJu
YXYtbGlzdCI+PC9kaXY+PGRpdiBjbGFzcz0ibmF2LWJhcnMgZmEgZmEtYmFycyI+PC9kaXY+PGRp
diBjbGFzcz0ibmF2LWdlYXJzIGZhIGZhLWNvZyI+PC9kaXY+PGRpdiBjbGFzcz0ibmF2LXdzLXN0
YXR1cyBmYSBmYS1jaXJjbGUiPjwvZGl2PjxkaXYgY2xhc3M9InBvd2VyZWQgaGlkZGVuLXhzIGhp
ZGRlbi1zbSI+YnkgPGEgaHJlZj0iaHR0cHM6Ly9nb2FjY2Vzcy5pby8iPkdvQWNjZXNzPC9hPiA8
c3Bhbj52e3t2ZXJzaW9ufX08L3NwYW4+IGFuZCA8YSBocmVmPSJodHRwczovL2d3c29ja2V0Lmlv
LyI+R1dTb2NrZXQ8L2E+PC9kaXY+PC9zY3JpcHQ+PCEtLSBUUEwgTmF2IEJhciBpdGVtcyAtLT48
c2NyaXB0IGlkPSJ0cGwtbmF2LW1lbnUiIHR5cGU9InRleHQvdGVtcGxhdGUiPjxoMz57e2xhYmVs
cy5wYW5lbHN9fTwvaDM+PHVsPjxsaSB7eyNvdmVyYWxsX2N1cnJlbnR9fWNsYXNzPSJhY3RpdmUi
e3svb3ZlcmFsbF9jdXJyZW50fX0+PGEgaHJlZj0iIyI+PGkgY2xhc3M9ImZhIGZhLWJhci1jaGFy
dCI+PC9pPiB7e2xhYmVscy50aGVhZH19PC9hPjxzcGFuIHRpdGxlPSJ7e2xhYmVscy50b2dnbGVf
cGFuZWx9fSIgY2xhc3M9InRvZ2dsZS1wYW5lbCB7eyNvdmVyYWxsX2hpZGRlbn19YWN0aXZle3sv
b3ZlcmFsbF9oaWRkZW59fSIgZGF0YS1wYW5lbD0iZ2VuZXJhbCI+PGkgY2xhc3M9ImZhIGZhLWNp
cmNsZS1vIj48L2k+PC9zcGFuPjwvbGk+e3sjbmF2fX08bGkge3sjY3VycmVudH19Y2xhc3M9ImFj
dGl2ZSJ7ey9jdXJyZW50fX0+PGEgaHJlZj0iI3t7a2V5fX0iPjxpIGNsYXNzPSJmYSBmYS17e2lj
b259fSI+PC9pPiB7e2hlYWR9fTwvYT48c3BhbiB0aXRsZT0ie3tsYWJlbHMudG9nZ2xlX3BhbmVs
fX0iIGNsYXNzPSJ0b2dnbGUtcGFuZWwge3sjaGlkZGVufX1hY3RpdmV7ey9oaWRkZW59fSIgZGF0
YS1wYW5lbD0ie3trZXl9fSI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZS1vIj48L2k+PC9zcGFuPjwv
bGk+e3svbmF2fX08L3VsPjwvc2NyaXB0PjwhLS0gVFBMIE5hdiBCYXIgb3B0aW9ucyAtLT48c2Ny
aXB0IGlkPSJ0cGwtbmF2LW9wdHMiIHR5cGU9InRleHQvdGVtcGxhdGUiPjxoMz48aSBjbGFzcz0i
ZmEgZmEtaGFzaHRhZyI+PC9pPiB7e2xhYmVscy50aGVtZX19PC9oMz48dWw+PGxpIHt7I2RhcmtH
cmF5fX1jbGFzcz0iYWN0aXZlInt7L2RhcmtHcmF5fX0+PGEgaHJlZj0iamF2YXNjcmlwdDp2b2lk
KDApOyIgY2xhc3M9InRoZW1lLWRhcmstZ3JheSI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7XmRh
cmtHcmF5fX0tb3t7L2RhcmtHcmF5fX0iPjwvaT4ge3tsYWJlbHMuZGFya19ncmF5fX08L2E+PC9s
aT48bGkge3sjYnJpZ2h0fX1jbGFzcz0iYWN0aXZlInt7L2JyaWdodH19PjxhIGhyZWY9ImphdmFz
Y3JpcHQ6dm9pZCgwKTsiIGNsYXNzPSJ0aGVtZS1icmlnaHQiPjxpIGNsYXNzPSJmYSBmYS1jaXJj
bGV7e15icmlnaHR9fS1ve3svYnJpZ2h0fX0iPjwvaT4ge3tsYWJlbHMuYnJpZ2h0fX08L2E+PC9s
aT48bGkge3sjZGFya0JsdWV9fWNsYXNzPSJhY3RpdmUie3svZGFya0JsdWV9fT48YSBocmVmPSJq
YXZhc2NyaXB0OnZvaWQoMCk7IiBjbGFzcz0idGhlbWUtZGFyay1ibHVlIj48aSBjbGFzcz0iZmEg
ZmEtY2lyY2xle3teZGFya0JsdWV9fS1ve3svZGFya0JsdWV9fSI+PC9pPiB7e2xhYmVscy5kYXJr
X2JsdWV9fTwvYT48L2xpPjxsaSB7eyNkYXJrUHVycGxlfX1jbGFzcz0iYWN0aXZlInt7L2RhcmtQ
dXJwbGV9fT48YSBocmVmPSJqYXZhc2NyaXB0OnZvaWQoMCk7IiBjbGFzcz0idGhlbWUtZGFyay1w
dXJwbGUiPjxpIGNsYXNzPSJmYSBmYS1jaXJjbGV7e15kYXJrUHVycGxlfX0tb3t7L2RhcmtQdXJw
bGV9fSI+PC9pPiB7e2xhYmVscy5kYXJrX3B1cnBsZX19PC9hPjwvbGk+PC91bD48aDM+PGkgY2xh
c3M9ImZhIGZhLWxpc3QtYWx0Ij48L2k+IHt7bGFiZWxzLnBhbmVsc319PC9oMz48dWwgY2xhc3M9
InBlcnBhZ2Utd3JhcCI+PGxpIGNsYXNzPSJkcm9wZG93bi1oZWFkZXIiPjxpIGNsYXNzPSJmYSBm
YS1saXN0Ij48L2k+IHt7bGFiZWxzLml0ZW1zX3Blcl9wYWdlfX08L2xpPjxsaSB7eyNwZXJQYWdl
M319Y2xhc3M9ImFjdGl2ZSJ7ey9wZXJQYWdlM319PjxhIGhyZWY9ImphdmFzY3JpcHQ6dm9pZCgw
KTsiIGRhdGEtcGVycGFnZT0iMyI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7XnBlclBhZ2UzfX0t
b3t7L3BlclBhZ2UzfX0iPjwvaT4gMzwvYT48L2xpPjxsaSB7eyNwZXJQYWdlNX19Y2xhc3M9ImFj
dGl2ZSJ7ey9wZXJQYWdlNX19PjxhIGhyZWY9ImphdmFzY3JpcHQ6dm9pZCgwKTsiIGRhdGEtcGVy
cGFnZT0iNSI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7XnBlclBhZ2U1fX0tb3t7L3BlclBhZ2U1
fX0iPjwvaT4gNTwvYT48L2xpPjxsaSB7eyNwZXJQYWdlN319Y2xhc3M9ImFjdGl2ZSJ7ey9wZXJQ
YWdlN319PjxhIGhyZWY9ImphdmFzY3JpcHQ6dm9pZCgwKTsiIGRhdGEtcGVycGFnZT0iNyI+PGkg
Y2xhc3M9ImZhIGZhLWNpcmNsZXt7XnBlclBhZ2U3fX0tb3t7L3BlclBhZ2U3fX0iPjwvaT4gNzwv
YT48L2xpPjxsaSB7eyNwZXJQYWdlMTB9fWNsYXNzPSJhY3RpdmUie3svcGVyUGFnZTEwfX0+PGEg
aHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgZGF0YS1wZXJwYWdlPSIxMCI+PGkgY2xhc3M9ImZh
IGZhLWNpcmNsZXt7XnBlclBhZ2UxMH19LW97ey9wZXJQYWdlMTB9fSI+PC9pPiAxMDwvYT48L2xp
PjxsaSB7eyNwZXJQYWdlMTV9fWNsYXNzPSJhY3RpdmUie3svcGVyUGFnZTE1fX0+PGEgaHJlZj0i
amF2YXNjcmlwdDp2b2lkKDApOyIgZGF0YS1wZXJwYWdlPSIxNSI+PGkgY2xhc3M9ImZhIGZhLWNp
cmNsZXt7XnBlclBhZ2UxNX19LW97ey9wZXJQYWdlMTV9fSI+PC9pPiAxNTwvYT48L2xpPjxsaSB7
eyNwZXJQYWdlMjB9fWNsYXNzPSJhY3RpdmUie3svcGVyUGFnZTIwfX0+PGEgaHJlZj0iamF2YXNj
cmlwdDp2b2lkKDApOyIgZGF0YS1wZXJwYWdlPSIyMCI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7
XnBlclBhZ2UyMH19LW97ey9wZXJQYWdlMjB9fSI+PC9pPiAyMDwvYT48L2xpPjxsaSB7eyNwZXJQ
YWdlMjR9fWNsYXNzPSJhY3RpdmUie3svcGVyUGFnZTI0fX0+PGEgaHJlZj0iamF2YXNjcmlwdDp2
b2lkKDApOyIgZGF0YS1wZXJwYWdlPSIyNCI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7XnBlclBh
Z2UyNH19LW97ey9wZXJQYWdlMjR9fSI+PC9pPiAyNDwvYT48L2xpPjxsaSBjbGFzcz0iZHJvcGRv
d24taGVhZGVyIj48aSBjbGFzcz0iZmEgZmEtdGFibGUiPjwvaT4ge3tsYWJlbHMudGFibGVzfX08
L2xpPjxsaSB7eyNzaG93VGFibGVzfX1jbGFzcz0iYWN0aXZlInt7L3Nob3dUYWJsZXN9fT48YSBo
cmVmPSJqYXZhc2NyaXB0OnZvaWQoMCk7IiBkYXRhLXNob3ctdGFibGVzPSIxIj48aSBjbGFzcz0i
ZmEgZmEte3sjc2hvd1RhYmxlc319Y2hlY2ste3svc2hvd1RhYmxlc319c3F1YXJlLW8iPjwvaT4g
e3tsYWJlbHMuZGlzcGxheV90YWJsZXN9fTwvYT48L2xpPjxsaSB7eyNhdXRvSGlkZVRhYmxlc319
Y2xhc3M9ImFjdGl2ZSJ7ey9hdXRvSGlkZVRhYmxlc319PjxhIGhyZWY9ImphdmFzY3JpcHQ6dm9p
ZCgwKTsiIGRhdGEtYXV0b2hpZGUtdGFibGVzPSIxIiB0aXRsZT0ie3tsYWJlbHMuYWhfc21hbGxf
dGl0bGV9fSI+PGkgY2xhc3M9ImZhIGZhLXt7I2F1dG9IaWRlVGFibGVzfX1jaGVjay17ey9hdXRv
SGlkZVRhYmxlc319c3F1YXJlLW8iPjwvaT4ge3tsYWJlbHMuYWhfc21hbGx9fTwvYT48L2xpPjwv
dWw+PGgzPjxpIGNsYXNzPSJmYSBmYS10aC1sYXJnZSI+PC9pPiB7e2xhYmVscy5sYXlvdXR9fTwv
aDM+PHVsPjxsaSB7eyNob3Jpem9udGFsfX1jbGFzcz0iYWN0aXZlInt7L2hvcml6b250YWx9fT48
YSBocmVmPSJqYXZhc2NyaXB0OnZvaWQoMCk7IiBjbGFzcz0ibGF5b3V0LWhvcml6b250YWwiPjxp
IGNsYXNzPSJmYSBmYS1jaXJjbGV7e15ob3Jpem9udGFsfX0tb3t7L2hvcml6b250YWx9fSI+PC9p
PiB7e2xhYmVscy5ob3Jpem9udGFsfX08L2E+PC9saT48bGkge3sjd2lkZX19Y2xhc3M9ImFjdGl2
ZSJ7ey93aWRlfX0+PGEgaHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgY2xhc3M9ImxheW91dC13
aWRlIj48aSBjbGFzcz0iZmEgZmEtY2lyY2xle3ted2lkZX19LW97ey93aWRlfX0iPjwvaT4ge3ts
YWJlbHMud2lkZX19PC9hPjwvbGk+PGxpIHt7I3ZlcnRpY2FsfX1jbGFzcz0iYWN0aXZlInt7L3Zl
cnRpY2FsfX0+PGEgaHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgY2xhc3M9ImxheW91dC12ZXJ0
aWNhbCI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7XnZlcnRpY2FsfX0tb3t7L3ZlcnRpY2FsfX0i
PjwvaT4ge3tsYWJlbHMudmVydGljYWx9fTwvYT48L2xpPjwvdWw+PGgzPjxpIGNsYXNzPSJmYSBm
YS1jb2ciPjwvaT4ge3tsYWJlbHMuZmlsZV9vcHRzfX08L2gzPjx1bD48bGk+PGEgaHJlZj0iamF2
YXNjcmlwdDp2b2lkKDApOyIgY2xhc3M9ImV4cG9ydC1qc29uIj48aSBjbGFzcz0iZmEgZmEtY29k
ZSI+PC9pPiB7e2xhYmVscy5leHBvcnRfanNvbn19PC9hPjwvbGk+PC91bD48L3NjcmlwdD48IS0t
IFRQTCBDaGFydCB0b29sdGlwIC0tPjxzY3JpcHQgaWQ9InRwbC1jaGFydC10b29sdGlwIiB0eXBl
PSJ0ZXh0L3RlbXBsYXRlIj48dGFibGUgY2xhc3M9ImNoYXJ0LXRvb2x0aXAiPjx0Ym9keT48dHI+
PHRoIGNvbHNwYW49IjIiPnt7ZGF0YS4wfX08L3RoPjwvdHI+PHRyPjx0ZCBjbGFzcz0ibmFtZSI+
PHNwYW4gY2xhc3M9ImJsdWUiPjwvc3Bhbj5oaXRzPC90ZD48dGQgY2xhc3M9InZhbHVlIj57e2Rh
dGEuMX19PC90ZD48L3RyPnt7I2RhdGEuMn19PHRyPjx0ZCBjbGFzcz0ibmFtZSI+PHNwYW4gY2xh
c3M9InJlZCI+PC9zcGFuPnZpc2l0b3JzPC90ZD48dGQgY2xhc3M9InZhbHVlIj57e2RhdGEuMn19
PC90ZD48L3RyPnt7L2RhdGEuMn19PC90Ym9keT48L3RhYmxlPjwvc2NyaXB0PjxzY3JpcHQgdHlw
ZT0ndGV4dC9qYXZhc2NyaXB0Jz52YXIganNvbl9pMThuPXsidGhlbWUiOiAiVGhlbWUiLCJkYXJr
X2dyYXkiOiAiRGFyayBHcmF5IiwiYnJpZ2h0IjogIkJyaWdodCIsImRhcmtfYmx1ZSI6ICJEYXJr
IEJsdWUiLCJkYXJrX3B1cnBsZSI6ICJEYXJrIFB1cnBsZSIsInBhbmVscyI6ICJQYW5lbHMiLCJp
dGVtc19wZXJfcGFnZSI6ICJJdGVtcyBwZXIgUGFnZSIsInRhYmxlcyI6ICJUYWJsZXMiLCJkaXNw
bGF5X3RhYmxlcyI6ICJEaXNwbGF5IFRhYmxlcyIsImFoX3NtYWxsIjogIkF1dG8tSGlkZSBvbiBT
bWFsbCBEZXZpY2VzIiwiYWhfc21hbGxfdGl0bGUiOiAiQXV0b21hdGljYWxseSBoaWRlIHRhYmxl
cyBvbiBzbWFsbCBzY3JlZW4gZGV2aWNlcyIsInRvZ2dsZV9wYW5lbCI6ICJUb2dnbGUgUGFuZWwi
LCJsYXlvdXQiOiAiTGF5b3V0IiwiaG9yaXpvbnRhbCI6ICJIb3Jpem9udGFsIiwidmVydGljYWwi
OiAiVmVydGljYWwiLCJ3aWRlIjogIldpZGVTY3JlZW4iLCJmaWxlX29wdHMiOiAiRmlsZSBPcHRp
b25zIiwiZXhwb3J0X2pzb24iOiAiRXhwb3J0IGFzIEpTT04iLCJwYW5lbF9vcHRzIjogIlBhbmVs
IE9wdGlvbnMiLCJwcmV2aW91cyI6ICJQcmV2aW91cyIsIm5leHQiOiAiTmV4dCIsImZpcnN0Ijog
IkZpcnN0IiwibGFzdCI6ICJMYXN0IiwiY2hhcnRfb3B0cyI6ICJDaGFydCBPcHRpb25zIiwiY2hh
cnQiOiAiQ2hhcnQiLCJ0eXBlIjogIlR5cGUiLCJhcmVhX3NwbGluZSI6ICJBcmVhIFNwbGluZSIs
ImJhciI6ICJCYXIiLCJwbG90X21ldHJpYyI6ICJQbG90IE1ldHJpYyIsInRhYmxlX2NvbHVtbnMi
OiAiVGFibGUgQ29sdW1ucyIsInRoZWFkIjogIk92ZXJhbGwgQW5hbHl6ZWQgUmVxdWVzdHMiLCJ2
ZXJzaW9uIjogIjEuOS4xIix9O3ZhciBodG1sX3ByZWZzPXt9O3ZhciB1c2VyX2ludGVyZmFjZT17
ImdlbmVyYWwiOiB7ImhlYWQiOiAiT3ZlcmFsbCBBbmFseXplZCBSZXF1ZXN0cyIsImRlc2MiOiAi
IiwiaXRlbXMiOiB7InRvdGFsX3JlcXVlc3RzIjogeyJjbGFzc05hbWUiOiAiYmxhY2siLCJkYXRh
VHlwZSI6ICJudW1lcmljIiwibGFiZWwiOiAiVG90YWwgUmVxdWVzdHMifSwidmFsaWRfcmVxdWVz
dHMiOiB7ImNsYXNzTmFtZSI6ICJncmVlbiIsImRhdGFUeXBlIjogIm51bWVyaWMiLCJsYWJlbCI6
ICJWYWxpZCBSZXF1ZXN0cyJ9LCJmYWlsZWRfcmVxdWVzdHMiOiB7ImNsYXNzTmFtZSI6ICJyZWQi
LCJkYXRhVHlwZSI6ICJudW1lcmljIiwibGFiZWwiOiAiRmFpbGVkIFJlcXVlc3RzIn0sImdlbmVy
YXRpb25fdGltZSI6IHsiY2xhc3NOYW1lIjogImdyYXkiLCJkYXRhVHlwZSI6ICJzZWNzIiwibGFi
ZWwiOiAiTG9nIFBhcnNpbmcgVGltZSJ9LCJ1bmlxdWVfdmlzaXRvcnMiOiB7ImNsYXNzTmFtZSI6
ICJibHVlIiwiZGF0YVR5cGUiOiAibnVtZXJpYyIsImxhYmVsIjogIlVuaXF1ZSBWaXNpdG9ycyJ9
LCJ1bmlxdWVfZmlsZXMiOiB7ImRhdGFUeXBlIjogIm51bWVyaWMiLCJsYWJlbCI6ICJSZXF1ZXN0
ZWQgRmlsZXMifSwiZXhjbHVkZWRfaGl0cyI6IHsiZGF0YVR5cGUiOiAibnVtZXJpYyIsImxhYmVs
IjogIkV4Y2wuIElQIEhpdHMifSwidW5pcXVlX3JlZmVycmVycyI6IHsiZGF0YVR5cGUiOiAibnVt
ZXJpYyIsImxhYmVsIjogIlJlZmVycmVycyJ9LCJ1bmlxdWVfbm90X2ZvdW5kIjogeyJkYXRhVHlw
ZSI6ICJudW1lcmljIiwibGFiZWwiOiAiTm90IEZvdW5kIn0sInVuaXF1ZV9zdGF0aWNfZmlsZXMi
OiB7ImRhdGFUeXBlIjogIm51bWVyaWMiLCJsYWJlbCI6ICJTdGF0aWMgRmlsZXMifSwibG9nX3Np
emUiOiB7ImRhdGFUeXBlIjogImJ5dGVzIiwibGFiZWwiOiAiTG9nIFNpemUifSwiYmFuZHdpZHRo
IjogeyJkYXRhVHlwZSI6ICJieXRlcyIsImxhYmVsIjogIlR4LiBBbW91bnQifX19LCJ2aXNpdG9y
cyI6IHsiaGVhZCI6ICJVbmlxdWUgdmlzaXRvcnMgcGVyIGRheSAtIEluY2x1ZGluZyBzcGlkZXJz
IiwiZGVzYyI6ICJIaXRzIGhhdmluZyB0aGUgc2FtZSBJUCwgZGF0ZSBhbmQgYWdlbnQgYXJlIGEg
dW5pcXVlIHZpc2l0LiIsImlkIjogInZpc2l0b3JzIiwidGFibGUiOiAxLCJzb3J0IjogeyJmaWVs
ZCI6ICJkYXRhIiwib3JkZXIiOiAiREVTQyJ9LCJwbG90IjogW3siY2xhc3NOYW1lIjogImhpdHMt
dmlzaXRvcnMiLCJsYWJlbCI6ICJIaXRzL1Zpc2l0b3JzIiwiY2hhcnRUeXBlIjogImFyZWEtc3Bs
aW5lIiwiY2hhcnRSZXZlcnNlIjogMSwicmVkcmF3T25FeHBhbmQiOiAxLCJkMyI6IHsieTAiOiB7
ImtleSI6ICJoaXRzIiwibGFiZWwiOiAiSGl0cyJ9LCJ5MSI6IHsia2V5IjogInZpc2l0b3JzIiwi
bGFiZWwiOiAiVmlzaXRvcnMifX19LHsiY2xhc3NOYW1lIjogImJhbmR3aWR0aCIsImxhYmVsIjog
IlR4LiBBbW91bnQiLCJjaGFydFR5cGUiOiAiYXJlYS1zcGxpbmUiLCJjaGFydFJldmVyc2UiOiAx
LCJyZWRyYXdPbkV4cGFuZCI6IDEsImQzIjogeyJ5MCI6IHsia2V5IjogImJ5dGVzIiwibGFiZWwi
OiAiVHguIEFtb3VudCIsImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVtcyI6IFt7ImNvbFdpZHRo
IjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmljIiwia2V5IjogImhp
dHMiLCJsYWJlbCI6ICJIaXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50Iiwi
ZGF0YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxhYmVsIjogIlZpc2l0b3Jz
In0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAiYnl0ZXMi
LCJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJjbGFzc05hbWUiOiAidHJ1
bmMiLCJjb2xXaWR0aCI6ICIxMDAlIiwibWV0YSI6ICJ1bmlxdWUiLCJtZXRhVHlwZSI6ICJudW1l
cmljIiwibWV0YUxhYmVsIjogIlRvdGFsIiwiZGF0YVR5cGUiOiAiZGF0ZSIsImhscmVnZXgiOiAi
e1wiXigxXFxcXGR7Mn18MXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBsYmx1
Jz4kMTwvYj4kMlwiLFwiXigyXFxcXGR7Mn18Mnh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0n
c3Bhbi1obCBsZ3JuJz4kMTwvYj4kMlwiLFwiXigzXFxcXGR7Mn18M3h4KShcXFxccy4qKSRcIjog
XCI8YiBjbGFzcz0nc3Bhbi1obCBscHJwJz4kMTwvYj4kMlwiLFwiXig0XFxcXGR7Mn18NHh4KShc
XFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBseWVsJz4kMTwvYj4kMlwiLFwiXig1XFxc
XGR7Mn18NXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBscmVkJz4kMTwvYj4k
MlwiLFwiXihBU1xcXFxkKylcIjogXCI8Yj4kMTwvYj5cIixcIl4oXFxcXGQrOilcIjogXCI8Yj4k
MTwvYj5cIixcIihcXFxcZCspfCg6XFxcXGQrKXwoOlxcXFxkKzpcXFxcZCspXCI6IFwiJDE8Yj4k
MjwvYj5cIixcIl4oW0EtWl17Mn0pKFxcXFxzLiokKVwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGc1
Jz4kMTwvYj4kMlwifSIsImtleSI6ICJkYXRhIiwibGFiZWwiOiAiRGF0YSJ9XX0sInJlcXVlc3Rz
IjogeyJoZWFkIjogIlJlcXVlc3RlZCBGaWxlcyAoVVJMcykiLCJkZXNjIjogIlRvcCByZXF1ZXN0
cyBzb3J0ZWQgYnkgaGl0cyBbLCBhdmd0cywgY3VtdHMsIG1heHRzLCBtdGhkLCBwcm90b10iLCJp
ZCI6ICJyZXF1ZXN0cyIsInRhYmxlIjogMSwic29ydCI6IHsiZmllbGQiOiAiaGl0cyIsIm9yZGVy
IjogIkRFU0MifSwicGxvdCI6IFt7ImNsYXNzTmFtZSI6ICJoaXRzLXZpc2l0b3JzIiwibGFiZWwi
OiAiSGl0cy9WaXNpdG9ycyIsImNoYXJ0VHlwZSI6ICJiYXIiLCJjaGFydFJldmVyc2UiOiAwLCJy
ZWRyYXdPbkV4cGFuZCI6IDAsImQzIjogeyJ4IjogeyJrZXkiOiBbIm1ldGhvZCIsICJkYXRhIiwg
InByb3RvY29sIl19LCJ5MCI6IHsia2V5IjogImhpdHMiLCJsYWJlbCI6ICJIaXRzIn0sInkxIjog
eyJrZXkiOiAidmlzaXRvcnMiLCJsYWJlbCI6ICJWaXNpdG9ycyJ9fX0seyJjbGFzc05hbWUiOiAi
YmFuZHdpZHRoIiwibGFiZWwiOiAiVHguIEFtb3VudCIsImNoYXJ0VHlwZSI6ICJiYXIiLCJjaGFy
dFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDAsImQzIjogeyJ4IjogeyJrZXkiOiBbIm1l
dGhvZCIsICJwcm90b2NvbCIsICJkYXRhIl19LCJ5MCI6IHsia2V5IjogImJ5dGVzIiwibGFiZWwi
OiAiVHguIEFtb3VudCIsImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVtcyI6IFt7ImNvbFdpZHRo
IjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmljIiwia2V5IjogImhp
dHMiLCJsYWJlbCI6ICJIaXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50Iiwi
ZGF0YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxhYmVsIjogIlZpc2l0b3Jz
In0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAiYnl0ZXMi
LCJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJjb2xXaWR0aCI6ICI2JSIs
ImRhdGFUeXBlIjogInN0cmluZyIsImhscmVnZXgiOiAie1wiKFxcXFxiW0EtWl17M31cXFxcYilc
IjogXCI8YiBjbGFzcz0nY2VsbC1obCBiMSc+JDE8L2I+XCIsXCIoXFxcXGJbQS1aXXs0fVxcXFxi
KVwiOiBcIjxiIGNsYXNzPSdjZWxsLWhsIGIyJz4kMTwvYj5cIixcIihcXFxcYltBLVpdezUsfVxc
XFxiKVwiOiBcIjxiIGNsYXNzPSdjZWxsLWhsIGIzJz4kMTwvYj5cIn0iLCJrZXkiOiAibWV0aG9k
IiwibGFiZWwiOiAiTWV0aG9kIn0seyJjb2xXaWR0aCI6ICI3JSIsImRhdGFUeXBlIjogInN0cmlu
ZyIsImhscmVnZXgiOiAie1wiKFxcXFxiSFRUUC8xLjBcXFxcYilcIjogXCI8YiBjbGFzcz0nY2Vs
bC1obCBkMSc+JDE8L2I+XCIsXCIoXFxcXGJIVFRQLzEuMVxcXFxiKVwiOiBcIjxiIGNsYXNzPSdj
ZWxsLWhsIGQyJz4kMTwvYj5cIixcIihcXFxcYkhUVFAvMlxcXFxiKVwiOiBcIjxiIGNsYXNzPSdj
ZWxsLWhsIGQzJz4kMTwvYj5cIixcIihcXFxcYkhUVFAvM1xcXFxiKVwiOiBcIjxiIGNsYXNzPSdj
ZWxsLWhsIGQ0Jz4kMTwvYj5cIn0iLCJrZXkiOiAicHJvdG9jb2wiLCJsYWJlbCI6ICJQcm90b2Nv
bCJ9LHsiY2xhc3NOYW1lIjogInRydW5jIiwiY29sV2lkdGgiOiAiMTAwJSIsIm1ldGEiOiAidW5p
cXVlIiwibWV0YVR5cGUiOiAibnVtZXJpYyIsIm1ldGFMYWJlbCI6ICJUb3RhbCIsImRhdGFUeXBl
IjogInN0cmluZyIsImhscmVnZXgiOiAie1wiXigxXFxcXGR7Mn18MXh4KShcXFxccy4qKSRcIjog
XCI8YiBjbGFzcz0nc3Bhbi1obCBsYmx1Jz4kMTwvYj4kMlwiLFwiXigyXFxcXGR7Mn18Mnh4KShc
XFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBsZ3JuJz4kMTwvYj4kMlwiLFwiXigzXFxc
XGR7Mn18M3h4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBscHJwJz4kMTwvYj4k
MlwiLFwiXig0XFxcXGR7Mn18NHh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBs
eWVsJz4kMTwvYj4kMlwiLFwiXig1XFxcXGR7Mn18NXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFz
cz0nc3Bhbi1obCBscmVkJz4kMTwvYj4kMlwiLFwiXihBU1xcXFxkKylcIjogXCI8Yj4kMTwvYj5c
IixcIl4oXFxcXGQrOilcIjogXCI8Yj4kMTwvYj5cIixcIihcXFxcZCspfCg6XFxcXGQrKXwoOlxc
XFxkKzpcXFxcZCspXCI6IFwiJDE8Yj4kMjwvYj5cIixcIl4oW0EtWl17Mn0pKFxcXFxzLiokKVwi
OiBcIjxiIGNsYXNzPSdzcGFuLWhsIGc1Jz4kMTwvYj4kMlwifSIsImtleSI6ICJkYXRhIiwibGFi
ZWwiOiAiRGF0YSJ9XX0sInN0YXRpY19yZXF1ZXN0cyI6IHsiaGVhZCI6ICJTdGF0aWMgUmVxdWVz
dHMiLCJkZXNjIjogIlRvcCBzdGF0aWMgcmVxdWVzdHMgc29ydGVkIGJ5IGhpdHMgWywgYXZndHMs
IGN1bXRzLCBtYXh0cywgbXRoZCwgcHJvdG9dIiwiaWQiOiAic3RhdGljX3JlcXVlc3RzIiwidGFi
bGUiOiAxLCJzb3J0IjogeyJmaWVsZCI6ICJoaXRzIiwib3JkZXIiOiAiREVTQyJ9LCJwbG90Ijog
W3siY2xhc3NOYW1lIjogImhpdHMtdmlzaXRvcnMiLCJsYWJlbCI6ICJIaXRzL1Zpc2l0b3JzIiwi
Y2hhcnRUeXBlIjogImJhciIsImNoYXJ0UmV2ZXJzZSI6IDAsInJlZHJhd09uRXhwYW5kIjogMCwi
ZDMiOiB7IngiOiB7ImtleSI6IFsibWV0aG9kIiwgImRhdGEiLCAicHJvdG9jb2wiXX0sInkwIjog
eyJrZXkiOiAiaGl0cyIsImxhYmVsIjogIkhpdHMifSwieTEiOiB7ImtleSI6ICJ2aXNpdG9ycyIs
ImxhYmVsIjogIlZpc2l0b3JzIn19fSx7ImNsYXNzTmFtZSI6ICJiYW5kd2lkdGgiLCJsYWJlbCI6
ICJUeC4gQW1vdW50IiwiY2hhcnRUeXBlIjogImJhciIsImNoYXJ0UmV2ZXJzZSI6IDAsInJlZHJh
d09uRXhwYW5kIjogMCwiZDMiOiB7IngiOiB7ImtleSI6IFsibWV0aG9kIiwgInByb3RvY29sIiwg
ImRhdGEiXX0sInkwIjogeyJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50IiwiZm9y
bWF0IjogImJ5dGVzIn19fV0sIml0ZW1zIjogW3siY29sV2lkdGgiOiAiMTIlIiwibWV0YSI6ICJj
b3VudCIsImRhdGFUeXBlIjogIm51bWVyaWMiLCJrZXkiOiAiaGl0cyIsImxhYmVsIjogIkhpdHMi
fSx7ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmlj
Iiwia2V5IjogInZpc2l0b3JzIiwibGFiZWwiOiAiVmlzaXRvcnMifSx7ImNvbFdpZHRoIjogIjEy
JSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJieXRlcyIsImtleSI6ICJieXRlcyIsImxh
YmVsIjogIlR4LiBBbW91bnQifSx7ImNvbFdpZHRoIjogIjYlIiwiZGF0YVR5cGUiOiAic3RyaW5n
IiwiaGxyZWdleCI6ICJ7XCIoXFxcXGJbQS1aXXszfVxcXFxiKVwiOiBcIjxiIGNsYXNzPSdjZWxs
LWhsIGIxJz4kMTwvYj5cIixcIihcXFxcYltBLVpdezR9XFxcXGIpXCI6IFwiPGIgY2xhc3M9J2Nl
bGwtaGwgYjInPiQxPC9iPlwiLFwiKFxcXFxiW0EtWl17NSx9XFxcXGIpXCI6IFwiPGIgY2xhc3M9
J2NlbGwtaGwgYjMnPiQxPC9iPlwifSIsImtleSI6ICJtZXRob2QiLCJsYWJlbCI6ICJNZXRob2Qi
fSx7ImNvbFdpZHRoIjogIjclIiwiZGF0YVR5cGUiOiAic3RyaW5nIiwiaGxyZWdleCI6ICJ7XCIo
XFxcXGJIVFRQLzEuMFxcXFxiKVwiOiBcIjxiIGNsYXNzPSdjZWxsLWhsIGQxJz4kMTwvYj5cIixc
IihcXFxcYkhUVFAvMS4xXFxcXGIpXCI6IFwiPGIgY2xhc3M9J2NlbGwtaGwgZDInPiQxPC9iPlwi
LFwiKFxcXFxiSFRUUC8yXFxcXGIpXCI6IFwiPGIgY2xhc3M9J2NlbGwtaGwgZDMnPiQxPC9iPlwi
LFwiKFxcXFxiSFRUUC8zXFxcXGIpXCI6IFwiPGIgY2xhc3M9J2NlbGwtaGwgZDQnPiQxPC9iPlwi
fSIsImtleSI6ICJwcm90b2NvbCIsImxhYmVsIjogIlByb3RvY29sIn0seyJjbGFzc05hbWUiOiAi
dHJ1bmMiLCJjb2xXaWR0aCI6ICIxMDAlIiwibWV0YSI6ICJ1bmlxdWUiLCJtZXRhVHlwZSI6ICJu
dW1lcmljIiwibWV0YUxhYmVsIjogIlRvdGFsIiwiZGF0YVR5cGUiOiAic3RyaW5nIiwiaGxyZWdl
eCI6ICJ7XCJeKDFcXFxcZHsyfXwxeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhs
IGxibHUnPiQxPC9iPiQyXCIsXCJeKDJcXFxcZHsyfXwyeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNs
YXNzPSdzcGFuLWhsIGxncm4nPiQxPC9iPiQyXCIsXCJeKDNcXFxcZHsyfXwzeHgpKFxcXFxzLiop
JFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxwcnAnPiQxPC9iPiQyXCIsXCJeKDRcXFxcZHsyfXw0
eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGx5ZWwnPiQxPC9iPiQyXCIsXCJe
KDVcXFxcZHsyfXw1eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxyZWQnPiQx
PC9iPiQyXCIsXCJeKEFTXFxcXGQrKVwiOiBcIjxiPiQxPC9iPlwiLFwiXihcXFxcZCs6KVwiOiBc
IjxiPiQxPC9iPlwiLFwiKFxcXFxkKyl8KDpcXFxcZCspfCg6XFxcXGQrOlxcXFxkKylcIjogXCIk
MTxiPiQyPC9iPlwiLFwiXihbQS1aXXsyfSkoXFxcXHMuKiQpXCI6IFwiPGIgY2xhc3M9J3NwYW4t
aGwgZzUnPiQxPC9iPiQyXCJ9Iiwia2V5IjogImRhdGEiLCJsYWJlbCI6ICJEYXRhIn1dfSwibm90
X2ZvdW5kIjogeyJoZWFkIjogIk5vdCBGb3VuZCBVUkxzICg0MDRzKSIsImRlc2MiOiAiVG9wIG5v
dCBmb3VuZCBVUkxzIHNvcnRlZCBieSBoaXRzIFssIGF2Z3RzLCBjdW10cywgbWF4dHMsIG10aGQs
IHByb3RvXSIsImlkIjogIm5vdF9mb3VuZCIsInRhYmxlIjogMSwic29ydCI6IHsiZmllbGQiOiAi
aGl0cyIsIm9yZGVyIjogIkRFU0MifSwicGxvdCI6IFt7ImNsYXNzTmFtZSI6ICJoaXRzLXZpc2l0
b3JzIiwibGFiZWwiOiAiSGl0cy9WaXNpdG9ycyIsImNoYXJ0VHlwZSI6ICJiYXIiLCJjaGFydFJl
dmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDAsImQzIjogeyJ4IjogeyJrZXkiOiBbIm1ldGhv
ZCIsICJkYXRhIiwgInByb3RvY29sIl19LCJ5MCI6IHsia2V5IjogImhpdHMiLCJsYWJlbCI6ICJI
aXRzIn0sInkxIjogeyJrZXkiOiAidmlzaXRvcnMiLCJsYWJlbCI6ICJWaXNpdG9ycyJ9fX0seyJj
bGFzc05hbWUiOiAiYmFuZHdpZHRoIiwibGFiZWwiOiAiVHguIEFtb3VudCIsImNoYXJ0VHlwZSI6
ICJiYXIiLCJjaGFydFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDAsImQzIjogeyJ4Ijog
eyJrZXkiOiBbIm1ldGhvZCIsICJwcm90b2NvbCIsICJkYXRhIl19LCJ5MCI6IHsia2V5IjogImJ5
dGVzIiwibGFiZWwiOiAiVHguIEFtb3VudCIsImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVtcyI6
IFt7ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmlj
Iiwia2V5IjogImhpdHMiLCJsYWJlbCI6ICJIaXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRh
IjogImNvdW50IiwiZGF0YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxhYmVs
IjogIlZpc2l0b3JzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5
cGUiOiAiYnl0ZXMiLCJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJjb2xX
aWR0aCI6ICI2JSIsImRhdGFUeXBlIjogInN0cmluZyIsImhscmVnZXgiOiAie1wiKFxcXFxiW0Et
Wl17M31cXFxcYilcIjogXCI8YiBjbGFzcz0nY2VsbC1obCBiMSc+JDE8L2I+XCIsXCIoXFxcXGJb
QS1aXXs0fVxcXFxiKVwiOiBcIjxiIGNsYXNzPSdjZWxsLWhsIGIyJz4kMTwvYj5cIixcIihcXFxc
YltBLVpdezUsfVxcXFxiKVwiOiBcIjxiIGNsYXNzPSdjZWxsLWhsIGIzJz4kMTwvYj5cIn0iLCJr
ZXkiOiAibWV0aG9kIiwibGFiZWwiOiAiTWV0aG9kIn0seyJjb2xXaWR0aCI6ICI3JSIsImRhdGFU
eXBlIjogInN0cmluZyIsImhscmVnZXgiOiAie1wiKFxcXFxiSFRUUC8xLjBcXFxcYilcIjogXCI8
YiBjbGFzcz0nY2VsbC1obCBkMSc+JDE8L2I+XCIsXCIoXFxcXGJIVFRQLzEuMVxcXFxiKVwiOiBc
IjxiIGNsYXNzPSdjZWxsLWhsIGQyJz4kMTwvYj5cIixcIihcXFxcYkhUVFAvMlxcXFxiKVwiOiBc
IjxiIGNsYXNzPSdjZWxsLWhsIGQzJz4kMTwvYj5cIixcIihcXFxcYkhUVFAvM1xcXFxiKVwiOiBc
IjxiIGNsYXNzPSdjZWxsLWhsIGQ0Jz4kMTwvYj5cIn0iLCJrZXkiOiAicHJvdG9jb2wiLCJsYWJl
bCI6ICJQcm90b2NvbCJ9LHsiY2xhc3NOYW1lIjogInRydW5jIiwiY29sV2lkdGgiOiAiMTAwJSIs
Im1ldGEiOiAidW5pcXVlIiwibWV0YVR5cGUiOiAibnVtZXJpYyIsIm1ldGFMYWJlbCI6ICJUb3Rh
bCIsImRhdGFUeXBlIjogInN0cmluZyIsImhscmVnZXgiOiAie1wiXigxXFxcXGR7Mn18MXh4KShc
XFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBsYmx1Jz4kMTwvYj4kMlwiLFwiXigyXFxc
XGR7Mn18Mnh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBsZ3JuJz4kMTwvYj4k
MlwiLFwiXigzXFxcXGR7Mn18M3h4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBs
cHJwJz4kMTwvYj4kMlwiLFwiXig0XFxcXGR7Mn18NHh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFz
cz0nc3Bhbi1obCBseWVsJz4kMTwvYj4kMlwiLFwiXig1XFxcXGR7Mn18NXh4KShcXFxccy4qKSRc
IjogXCI8YiBjbGFzcz0nc3Bhbi1obCBscmVkJz4kMTwvYj4kMlwiLFwiXihBU1xcXFxkKylcIjog
XCI8Yj4kMTwvYj5cIixcIl4oXFxcXGQrOilcIjogXCI8Yj4kMTwvYj5cIixcIihcXFxcZCspfCg6
XFxcXGQrKXwoOlxcXFxkKzpcXFxcZCspXCI6IFwiJDE8Yj4kMjwvYj5cIixcIl4oW0EtWl17Mn0p
KFxcXFxzLiokKVwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGc1Jz4kMTwvYj4kMlwifSIsImtleSI6
ICJkYXRhIiwibGFiZWwiOiAiRGF0YSJ9XX0sImhvc3RzIjogeyJoZWFkIjogIlZpc2l0b3IgSG9z
dG5hbWVzIGFuZCBJUHMiLCJkZXNjIjogIlRvcCB2aXNpdG9yIGhvc3RzIHNvcnRlZCBieSBoaXRz
IFssIGF2Z3RzLCBjdW10cywgbWF4dHNdIiwiaWQiOiAiaG9zdHMiLCJ0YWJsZSI6IDEsInNvcnQi
OiB7ImZpZWxkIjogImhpdHMiLCJvcmRlciI6ICJERVNDIn0sInBsb3QiOiBbeyJjbGFzc05hbWUi
OiAiaGl0cy12aXNpdG9ycyIsImxhYmVsIjogIkhpdHMvVmlzaXRvcnMiLCJjaGFydFR5cGUiOiAi
YmFyIiwiY2hhcnRSZXZlcnNlIjogMCwicmVkcmF3T25FeHBhbmQiOiAwLCJkMyI6IHsieTAiOiB7
ImtleSI6ICJoaXRzIiwibGFiZWwiOiAiSGl0cyJ9LCJ5MSI6IHsia2V5IjogInZpc2l0b3JzIiwi
bGFiZWwiOiAiVmlzaXRvcnMifX19LHsiY2xhc3NOYW1lIjogImJhbmR3aWR0aCIsImxhYmVsIjog
IlR4LiBBbW91bnQiLCJjaGFydFR5cGUiOiAiYmFyIiwiY2hhcnRSZXZlcnNlIjogMCwicmVkcmF3
T25FeHBhbmQiOiAwLCJkMyI6IHsieTAiOiB7ImtleSI6ICJieXRlcyIsImxhYmVsIjogIlR4LiBB
bW91bnQiLCJmb3JtYXQiOiAiYnl0ZXMifX19XSwiaXRlbXMiOiBbeyJjb2xXaWR0aCI6ICIxMiUi
LCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJoaXRzIiwibGFi
ZWwiOiAiSGl0cyJ9LHsiY29sV2lkdGgiOiAiMTIlIiwibWV0YSI6ICJjb3VudCIsImRhdGFUeXBl
IjogIm51bWVyaWMiLCJrZXkiOiAidmlzaXRvcnMiLCJsYWJlbCI6ICJWaXNpdG9ycyJ9LHsiY29s
V2lkdGgiOiAiMTIlIiwibWV0YSI6ICJjb3VudCIsImRhdGFUeXBlIjogImJ5dGVzIiwia2V5Ijog
ImJ5dGVzIiwibGFiZWwiOiAiVHguIEFtb3VudCJ9LHsiY2xhc3NOYW1lIjogInRydW5jIiwiY29s
V2lkdGgiOiAiMTAwJSIsIm1ldGEiOiAidW5pcXVlIiwibWV0YVR5cGUiOiAibnVtZXJpYyIsIm1l
dGFMYWJlbCI6ICJUb3RhbCIsImRhdGFUeXBlIjogInN0cmluZyIsImhscmVnZXgiOiAie1wiXigx
XFxcXGR7Mn18MXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBsYmx1Jz4kMTwv
Yj4kMlwiLFwiXigyXFxcXGR7Mn18Mnh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1o
bCBsZ3JuJz4kMTwvYj4kMlwiLFwiXigzXFxcXGR7Mn18M3h4KShcXFxccy4qKSRcIjogXCI8YiBj
bGFzcz0nc3Bhbi1obCBscHJwJz4kMTwvYj4kMlwiLFwiXig0XFxcXGR7Mn18NHh4KShcXFxccy4q
KSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBseWVsJz4kMTwvYj4kMlwiLFwiXig1XFxcXGR7Mn18
NXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBscmVkJz4kMTwvYj4kMlwiLFwi
XihBU1xcXFxkKylcIjogXCI8Yj4kMTwvYj5cIixcIl4oXFxcXGQrOilcIjogXCI8Yj4kMTwvYj5c
IixcIihcXFxcZCspfCg6XFxcXGQrKXwoOlxcXFxkKzpcXFxcZCspXCI6IFwiJDE8Yj4kMjwvYj5c
IixcIl4oW0EtWl17Mn0pKFxcXFxzLiokKVwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGc1Jz4kMTwv
Yj4kMlwifSIsImtleSI6ICJkYXRhIiwibGFiZWwiOiAiRGF0YSJ9XX0sIm9zIjogeyJoZWFkIjog
Ik9wZXJhdGluZyBTeXN0ZW1zIiwiZGVzYyI6ICJUb3AgT3BlcmF0aW5nIFN5c3RlbXMgc29ydGVk
IGJ5IGhpdHMgWywgYXZndHMsIGN1bXRzLCBtYXh0c10iLCJpZCI6ICJvcyIsInRhYmxlIjogMSwi
c29ydCI6IHsiZmllbGQiOiAiaGl0cyIsIm9yZGVyIjogIkRFU0MifSwicGxvdCI6IFt7ImNsYXNz
TmFtZSI6ICJoaXRzLXZpc2l0b3JzIiwibGFiZWwiOiAiSGl0cy9WaXNpdG9ycyIsImNoYXJ0VHlw
ZSI6ICJiYXIiLCJjaGFydFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDEsImQzIjogeyJ5
MCI6IHsia2V5IjogImhpdHMiLCJsYWJlbCI6ICJIaXRzIn0sInkxIjogeyJrZXkiOiAidmlzaXRv
cnMiLCJsYWJlbCI6ICJWaXNpdG9ycyJ9fX0seyJjbGFzc05hbWUiOiAiYmFuZHdpZHRoIiwibGFi
ZWwiOiAiVHguIEFtb3VudCIsImNoYXJ0VHlwZSI6ICJiYXIiLCJjaGFydFJldmVyc2UiOiAwLCJy
ZWRyYXdPbkV4cGFuZCI6IDEsImQzIjogeyJ5MCI6IHsia2V5IjogImJ5dGVzIiwibGFiZWwiOiAi
VHguIEFtb3VudCIsImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVtcyI6IFt7ImNvbFdpZHRoIjog
IjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmljIiwia2V5IjogImhpdHMi
LCJsYWJlbCI6ICJIaXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0
YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxhYmVsIjogIlZpc2l0b3JzIn0s
eyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAiYnl0ZXMiLCJr
ZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJjbGFzc05hbWUiOiAidHJ1bmMi
LCJjb2xXaWR0aCI6ICIxMDAlIiwibWV0YSI6ICJ1bmlxdWUiLCJtZXRhVHlwZSI6ICJudW1lcmlj
IiwibWV0YUxhYmVsIjogIlRvdGFsIiwiZGF0YVR5cGUiOiAic3RyaW5nIiwiaGxyZWdleCI6ICJ7
XCJeKDFcXFxcZHsyfXwxeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxibHUn
PiQxPC9iPiQyXCIsXCJeKDJcXFxcZHsyfXwyeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdz
cGFuLWhsIGxncm4nPiQxPC9iPiQyXCIsXCJeKDNcXFxcZHsyfXwzeHgpKFxcXFxzLiopJFwiOiBc
IjxiIGNsYXNzPSdzcGFuLWhsIGxwcnAnPiQxPC9iPiQyXCIsXCJeKDRcXFxcZHsyfXw0eHgpKFxc
XFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGx5ZWwnPiQxPC9iPiQyXCIsXCJeKDVcXFxc
ZHsyfXw1eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxyZWQnPiQxPC9iPiQy
XCIsXCJeKEFTXFxcXGQrKVwiOiBcIjxiPiQxPC9iPlwiLFwiXihcXFxcZCs6KVwiOiBcIjxiPiQx
PC9iPlwiLFwiKFxcXFxkKyl8KDpcXFxcZCspfCg6XFxcXGQrOlxcXFxkKylcIjogXCIkMTxiPiQy
PC9iPlwiLFwiXihbQS1aXXsyfSkoXFxcXHMuKiQpXCI6IFwiPGIgY2xhc3M9J3NwYW4taGwgZzUn
PiQxPC9iPiQyXCJ9Iiwia2V5IjogImRhdGEiLCJsYWJlbCI6ICJEYXRhIn1dfSwiYnJvd3NlcnMi
OiB7ImhlYWQiOiAiQnJvd3NlcnMiLCJkZXNjIjogIlRvcCBCcm93c2VycyBzb3J0ZWQgYnkgaGl0
cyBbLCBhdmd0cywgY3VtdHMsIG1heHRzXSIsImlkIjogImJyb3dzZXJzIiwidGFibGUiOiAxLCJz
b3J0IjogeyJmaWVsZCI6ICJoaXRzIiwib3JkZXIiOiAiREVTQyJ9LCJwbG90IjogW3siY2xhc3NO
YW1lIjogImhpdHMtdmlzaXRvcnMiLCJsYWJlbCI6ICJIaXRzL1Zpc2l0b3JzIiwiY2hhcnRUeXBl
IjogImJhciIsImNoYXJ0UmV2ZXJzZSI6IDAsInJlZHJhd09uRXhwYW5kIjogMSwiZDMiOiB7Inkw
IjogeyJrZXkiOiAiaGl0cyIsImxhYmVsIjogIkhpdHMifSwieTEiOiB7ImtleSI6ICJ2aXNpdG9y
cyIsImxhYmVsIjogIlZpc2l0b3JzIn19fSx7ImNsYXNzTmFtZSI6ICJiYW5kd2lkdGgiLCJsYWJl
bCI6ICJUeC4gQW1vdW50IiwiY2hhcnRUeXBlIjogImJhciIsImNoYXJ0UmV2ZXJzZSI6IDAsInJl
ZHJhd09uRXhwYW5kIjogMSwiZDMiOiB7InkwIjogeyJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJU
eC4gQW1vdW50IiwiZm9ybWF0IjogImJ5dGVzIn19fV0sIml0ZW1zIjogW3siY29sV2lkdGgiOiAi
MTIlIiwibWV0YSI6ICJjb3VudCIsImRhdGFUeXBlIjogIm51bWVyaWMiLCJrZXkiOiAiaGl0cyIs
ImxhYmVsIjogIkhpdHMifSx7ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRh
VHlwZSI6ICJudW1lcmljIiwia2V5IjogInZpc2l0b3JzIiwibGFiZWwiOiAiVmlzaXRvcnMifSx7
ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJieXRlcyIsImtl
eSI6ICJieXRlcyIsImxhYmVsIjogIlR4LiBBbW91bnQifSx7ImNsYXNzTmFtZSI6ICJ0cnVuYyIs
ImNvbFdpZHRoIjogIjEwMCUiLCJtZXRhIjogInVuaXF1ZSIsIm1ldGFUeXBlIjogIm51bWVyaWMi
LCJtZXRhTGFiZWwiOiAiVG90YWwiLCJkYXRhVHlwZSI6ICJzdHJpbmciLCJobHJlZ2V4IjogIntc
Il4oMVxcXFxkezJ9fDF4eCkoXFxcXHMuKikkXCI6IFwiPGIgY2xhc3M9J3NwYW4taGwgbGJsdSc+
JDE8L2I+JDJcIixcIl4oMlxcXFxkezJ9fDJ4eCkoXFxcXHMuKikkXCI6IFwiPGIgY2xhc3M9J3Nw
YW4taGwgbGdybic+JDE8L2I+JDJcIixcIl4oM1xcXFxkezJ9fDN4eCkoXFxcXHMuKikkXCI6IFwi
PGIgY2xhc3M9J3NwYW4taGwgbHBycCc+JDE8L2I+JDJcIixcIl4oNFxcXFxkezJ9fDR4eCkoXFxc
XHMuKikkXCI6IFwiPGIgY2xhc3M9J3NwYW4taGwgbHllbCc+JDE8L2I+JDJcIixcIl4oNVxcXFxk
ezJ9fDV4eCkoXFxcXHMuKikkXCI6IFwiPGIgY2xhc3M9J3NwYW4taGwgbHJlZCc+JDE8L2I+JDJc
IixcIl4oQVNcXFxcZCspXCI6IFwiPGI+JDE8L2I+XCIsXCJeKFxcXFxkKzopXCI6IFwiPGI+JDE8
L2I+XCIsXCIoXFxcXGQrKXwoOlxcXFxkKyl8KDpcXFxcZCs6XFxcXGQrKVwiOiBcIiQxPGI+JDI8
L2I+XCIsXCJeKFtBLVpdezJ9KShcXFxccy4qJClcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBnNSc+
JDE8L2I+JDJcIn0iLCJrZXkiOiAiZGF0YSIsImxhYmVsIjogIkRhdGEifV19LCJ2aXNpdF90aW1l
IjogeyJoZWFkIjogIlRpbWUgRGlzdHJpYnV0aW9uIiwiZGVzYyI6ICJEYXRhIHNvcnRlZCBieSBo
b3VyIFssIGF2Z3RzLCBjdW10cywgbWF4dHNdIiwiaWQiOiAidmlzaXRfdGltZSIsInRhYmxlIjog
MSwic29ydCI6IHsiZmllbGQiOiAiZGF0YSIsIm9yZGVyIjogIkFTQyJ9LCJwbG90IjogW3siY2xh
c3NOYW1lIjogImhpdHMtdmlzaXRvcnMiLCJsYWJlbCI6ICJIaXRzL1Zpc2l0b3JzIiwiY2hhcnRU
eXBlIjogImFyZWEtc3BsaW5lIiwiY2hhcnRSZXZlcnNlIjogMCwicmVkcmF3T25FeHBhbmQiOiAx
LCJkMyI6IHsieTAiOiB7ImtleSI6ICJoaXRzIiwibGFiZWwiOiAiSGl0cyJ9LCJ5MSI6IHsia2V5
IjogInZpc2l0b3JzIiwibGFiZWwiOiAiVmlzaXRvcnMifX19LHsiY2xhc3NOYW1lIjogImJhbmR3
aWR0aCIsImxhYmVsIjogIlR4LiBBbW91bnQiLCJjaGFydFR5cGUiOiAiYXJlYS1zcGxpbmUiLCJj
aGFydFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDEsImQzIjogeyJ5MCI6IHsia2V5Ijog
ImJ5dGVzIiwibGFiZWwiOiAiVHguIEFtb3VudCIsImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVt
cyI6IFt7ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1l
cmljIiwia2V5IjogImhpdHMiLCJsYWJlbCI6ICJIaXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJt
ZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxh
YmVsIjogIlZpc2l0b3JzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0
YVR5cGUiOiAiYnl0ZXMiLCJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJj
bGFzc05hbWUiOiAidHJ1bmMiLCJjb2xXaWR0aCI6ICIxMDAlIiwibWV0YSI6ICJ1bmlxdWUiLCJt
ZXRhVHlwZSI6ICJudW1lcmljIiwibWV0YUxhYmVsIjogIlRvdGFsIiwiZGF0YVR5cGUiOiAic3Ry
aW5nIiwiaGxyZWdleCI6ICJ7XCJeKDFcXFxcZHsyfXwxeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNs
YXNzPSdzcGFuLWhsIGxibHUnPiQxPC9iPiQyXCIsXCJeKDJcXFxcZHsyfXwyeHgpKFxcXFxzLiop
JFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxncm4nPiQxPC9iPiQyXCIsXCJeKDNcXFxcZHsyfXwz
eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxwcnAnPiQxPC9iPiQyXCIsXCJe
KDRcXFxcZHsyfXw0eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGx5ZWwnPiQx
PC9iPiQyXCIsXCJeKDVcXFxcZHsyfXw1eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFu
LWhsIGxyZWQnPiQxPC9iPiQyXCIsXCJeKEFTXFxcXGQrKVwiOiBcIjxiPiQxPC9iPlwiLFwiXihc
XFxcZCs6KVwiOiBcIjxiPiQxPC9iPlwiLFwiKFxcXFxkKyl8KDpcXFxcZCspfCg6XFxcXGQrOlxc
XFxkKylcIjogXCIkMTxiPiQyPC9iPlwiLFwiXihbQS1aXXsyfSkoXFxcXHMuKiQpXCI6IFwiPGIg
Y2xhc3M9J3NwYW4taGwgZzUnPiQxPC9iPiQyXCJ9Iiwia2V5IjogImRhdGEiLCJsYWJlbCI6ICJE
YXRhIn1dfSwicmVmZXJyaW5nX3NpdGVzIjogeyJoZWFkIjogIlJlZmVycmluZyBTaXRlcyIsImRl
c2MiOiAiVG9wIFJlZmVycmluZyBTaXRlcyBzb3J0ZWQgYnkgaGl0cyBbLCBhdmd0cywgY3VtdHMs
IG1heHRzXSIsImlkIjogInJlZmVycmluZ19zaXRlcyIsInRhYmxlIjogMSwic29ydCI6IHsiZmll
bGQiOiAiaGl0cyIsIm9yZGVyIjogIkRFU0MifSwicGxvdCI6IFt7ImNsYXNzTmFtZSI6ICJoaXRz
LXZpc2l0b3JzIiwibGFiZWwiOiAiSGl0cy9WaXNpdG9ycyIsImNoYXJ0VHlwZSI6ICJiYXIiLCJj
aGFydFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDAsImQzIjogeyJ5MCI6IHsia2V5Ijog
ImhpdHMiLCJsYWJlbCI6ICJIaXRzIn0sInkxIjogeyJrZXkiOiAidmlzaXRvcnMiLCJsYWJlbCI6
ICJWaXNpdG9ycyJ9fX0seyJjbGFzc05hbWUiOiAiYmFuZHdpZHRoIiwibGFiZWwiOiAiVHguIEFt
b3VudCIsImNoYXJ0VHlwZSI6ICJiYXIiLCJjaGFydFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFu
ZCI6IDAsImQzIjogeyJ5MCI6IHsia2V5IjogImJ5dGVzIiwibGFiZWwiOiAiVHguIEFtb3VudCIs
ImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVtcyI6IFt7ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEi
OiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmljIiwia2V5IjogImhpdHMiLCJsYWJlbCI6ICJI
aXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAibnVt
ZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxhYmVsIjogIlZpc2l0b3JzIn0seyJjb2xXaWR0aCI6
ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAiYnl0ZXMiLCJrZXkiOiAiYnl0ZXMi
LCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJjbGFzc05hbWUiOiAidHJ1bmMiLCJjb2xXaWR0aCI6
ICIxMDAlIiwibWV0YSI6ICJ1bmlxdWUiLCJtZXRhVHlwZSI6ICJudW1lcmljIiwibWV0YUxhYmVs
IjogIlRvdGFsIiwiZGF0YVR5cGUiOiAic3RyaW5nIiwiaGxyZWdleCI6ICJ7XCJeKDFcXFxcZHsy
fXwxeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxibHUnPiQxPC9iPiQyXCIs
XCJeKDJcXFxcZHsyfXwyeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxncm4n
PiQxPC9iPiQyXCIsXCJeKDNcXFxcZHsyfXwzeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdz
cGFuLWhsIGxwcnAnPiQxPC9iPiQyXCIsXCJeKDRcXFxcZHsyfXw0eHgpKFxcXFxzLiopJFwiOiBc
IjxiIGNsYXNzPSdzcGFuLWhsIGx5ZWwnPiQxPC9iPiQyXCIsXCJeKDVcXFxcZHsyfXw1eHgpKFxc
XFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxyZWQnPiQxPC9iPiQyXCIsXCJeKEFTXFxc
XGQrKVwiOiBcIjxiPiQxPC9iPlwiLFwiXihcXFxcZCs6KVwiOiBcIjxiPiQxPC9iPlwiLFwiKFxc
XFxkKyl8KDpcXFxcZCspfCg6XFxcXGQrOlxcXFxkKylcIjogXCIkMTxiPiQyPC9iPlwiLFwiXihb
QS1aXXsyfSkoXFxcXHMuKiQpXCI6IFwiPGIgY2xhc3M9J3NwYW4taGwgZzUnPiQxPC9iPiQyXCJ9
Iiwia2V5IjogImRhdGEiLCJsYWJlbCI6ICJEYXRhIn1dfSwic3RhdHVzX2NvZGVzIjogeyJoZWFk
IjogIkhUVFAgU3RhdHVzIENvZGVzIiwiZGVzYyI6ICJUb3AgSFRUUCBTdGF0dXMgQ29kZXMgc29y
dGVkIGJ5IGhpdHMgWywgYXZndHMsIGN1bXRzLCBtYXh0c10iLCJpZCI6ICJzdGF0dXNfY29kZXMi
LCJ0YWJsZSI6IDEsInNvcnQiOiB7ImZpZWxkIjogImhpdHMiLCJvcmRlciI6ICJERVNDIn0sInBs
b3QiOiBbeyJjbGFzc05hbWUiOiAiaGl0cy12aXNpdG9ycyIsImxhYmVsIjogIkhpdHMvVmlzaXRv
cnMiLCJjaGFydFR5cGUiOiAiYmFyIiwiY2hhcnRSZXZlcnNlIjogMCwicmVkcmF3T25FeHBhbmQi
OiAxLCJkMyI6IHsieTAiOiB7ImtleSI6ICJoaXRzIiwibGFiZWwiOiAiSGl0cyJ9LCJ5MSI6IHsi
a2V5IjogInZpc2l0b3JzIiwibGFiZWwiOiAiVmlzaXRvcnMifX19LHsiY2xhc3NOYW1lIjogImJh
bmR3aWR0aCIsImxhYmVsIjogIlR4LiBBbW91bnQiLCJjaGFydFR5cGUiOiAiYmFyIiwiY2hhcnRS
ZXZlcnNlIjogMCwicmVkcmF3T25FeHBhbmQiOiAxLCJkMyI6IHsieTAiOiB7ImtleSI6ICJieXRl
cyIsImxhYmVsIjogIlR4LiBBbW91bnQiLCJmb3JtYXQiOiAiYnl0ZXMifX19XSwiaXRlbXMiOiBb
eyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAibnVtZXJpYyIs
ImtleSI6ICJoaXRzIiwibGFiZWwiOiAiSGl0cyJ9LHsiY29sV2lkdGgiOiAiMTIlIiwibWV0YSI6
ICJjb3VudCIsImRhdGFUeXBlIjogIm51bWVyaWMiLCJrZXkiOiAidmlzaXRvcnMiLCJsYWJlbCI6
ICJWaXNpdG9ycyJ9LHsiY29sV2lkdGgiOiAiMTIlIiwibWV0YSI6ICJjb3VudCIsImRhdGFUeXBl
IjogImJ5dGVzIiwia2V5IjogImJ5dGVzIiwibGFiZWwiOiAiVHguIEFtb3VudCJ9LHsiY2xhc3NO
YW1lIjogInRydW5jIiwiY29sV2lkdGgiOiAiMTAwJSIsIm1ldGEiOiAidW5pcXVlIiwibWV0YVR5
cGUiOiAibnVtZXJpYyIsIm1ldGFMYWJlbCI6ICJUb3RhbCIsImRhdGFUeXBlIjogInN0cmluZyIs
ImhscmVnZXgiOiAie1wiXigxXFxcXGR7Mn18MXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0n
c3Bhbi1obCBsYmx1Jz4kMTwvYj4kMlwiLFwiXigyXFxcXGR7Mn18Mnh4KShcXFxccy4qKSRcIjog
XCI8YiBjbGFzcz0nc3Bhbi1obCBsZ3JuJz4kMTwvYj4kMlwiLFwiXigzXFxcXGR7Mn18M3h4KShc
XFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBscHJwJz4kMTwvYj4kMlwiLFwiXig0XFxc
XGR7Mn18NHh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBseWVsJz4kMTwvYj4k
MlwiLFwiXig1XFxcXGR7Mn18NXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBs
cmVkJz4kMTwvYj4kMlwiLFwiXihBU1xcXFxkKylcIjogXCI8Yj4kMTwvYj5cIixcIl4oXFxcXGQr
OilcIjogXCI8Yj4kMTwvYj5cIixcIihcXFxcZCspfCg6XFxcXGQrKXwoOlxcXFxkKzpcXFxcZCsp
XCI6IFwiJDE8Yj4kMjwvYj5cIixcIl4oW0EtWl17Mn0pKFxcXFxzLiokKVwiOiBcIjxiIGNsYXNz
PSdzcGFuLWhsIGc1Jz4kMTwvYj4kMlwifSIsImtleSI6ICJkYXRhIiwibGFiZWwiOiAiRGF0YSJ9
XX0sfTwvc2NyaXB0PjxzY3JpcHQgdHlwZT0ndGV4dC9qYXZhc2NyaXB0Jz52YXIganNvbl9kYXRh
PXsiZ2VuZXJhbCI6IHsic3RhcnRfZGF0ZSI6ICIwNi9GZWIvMjAyNCIsImVuZF9kYXRlIjogIjE5
L0ZlYi8yMDI0IiwiZGF0ZV90aW1lIjogIjIwMjQtMDItMTkgMTQ6NTg6NDkgKzA4MDAiLCJ0b3Rh
bF9yZXF1ZXN0cyI6IDE1MzMsInZhbGlkX3JlcXVlc3RzIjogMTUzMywiZmFpbGVkX3JlcXVlc3Rz
IjogMCwiZ2VuZXJhdGlvbl90aW1lIjogMSwidW5pcXVlX3Zpc2l0b3JzIjogODM0LCJ1bmlxdWVf
ZmlsZXMiOiAwLCJleGNsdWRlZF9oaXRzIjogMCwidW5pcXVlX3JlZmVycmVycyI6IDAsInVuaXF1
ZV9ub3RfZm91bmQiOiAxLCJ1bmlxdWVfc3RhdGljX2ZpbGVzIjogMiwibG9nX3NpemUiOiAzODE4
ODMsImJhbmR3aWR0aCI6IDMyMDc2MDkzNTAyLCJsb2dfcGF0aCI6IFsiNC0xOC5sb2ciXX0sInZp
c2l0b3JzIjogeyJtZXRhZGF0YSI6IHsiYnl0ZXMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDMyMDc2
MDkzNTAyfSwiYXZnIjogeyJ2YWx1ZSI6IDIyOTExNDk1NjgsInBlcmNlbnQiOiAiMDcuMTQifSwi
bWF4IjogeyJ2YWx1ZSI6IDM1ODg2NzI1MjEsInBlcmNlbnQiOiAiMTEuMTkifSwibWluIjogeyJ2
YWx1ZSI6IDQxNTcxMDkxMiwicGVyY2VudCI6ICIwMS4zMCJ9fSwidmlzaXRvcnMiOiB7InRvdGFs
IjogeyJ2YWx1ZSI6IDgzNH0sImF2ZyI6IHsidmFsdWUiOiA1OSwicGVyY2VudCI6ICIwNy4wNyJ9
LCJtYXgiOiB7InZhbHVlIjogOTEsInBlcmNlbnQiOiAiMTAuOTEifSwibWluIjogeyJ2YWx1ZSI6
IDgsInBlcmNlbnQiOiAiMDAuOTYifX0sImhpdHMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDE1MzN9
LCJhdmciOiB7InZhbHVlIjogMTA5LCJwZXJjZW50IjogIjA3LjExIn0sIm1heCI6IHsidmFsdWUi
OiAyMDIsInBlcmNlbnQiOiAiMTMuMTgifSwibWluIjogeyJ2YWx1ZSI6IDEwLCJwZXJjZW50Ijog
IjAwLjY1In19LCJkYXRhIjogeyJ0b3RhbCI6IHsJInZhbHVlIjogMTR9fX0sImRhdGEiOiBbeyJo
aXRzIjogeyJjb3VudCI6IDEwLCJwZXJjZW50IjogIjAwLjY1In0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDgsInBlcmNlbnQiOiAiMDAuOTYifSwiYnl0ZXMiOiB7ImNvdW50IjogNDE1NzEwOTEyLCJw
ZXJjZW50IjogIjAxLjMwIn0sImRhdGEiOiAiMjAyNDAyMTkifSx7ImhpdHMiOiB7ImNvdW50Ijog
NzYsInBlcmNlbnQiOiAiMDQuOTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNDgsInBlcmNlbnQi
OiAiMDUuNzYifSwiYnl0ZXMiOiB7ImNvdW50IjogMTg5OTc2NDg1MywicGVyY2VudCI6ICIwNS45
MiJ9LCJkYXRhIjogIjIwMjQwMjE4In0seyJoaXRzIjogeyJjb3VudCI6IDExNiwicGVyY2VudCI6
ICIwNy41NyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA2NywicGVyY2VudCI6ICIwOC4wMyJ9LCJi
eXRlcyI6IHsiY291bnQiOiAyMjY1NDA2NzYyLCJwZXJjZW50IjogIjA3LjA2In0sImRhdGEiOiAi
MjAyNDAyMTcifSx7ImhpdHMiOiB7ImNvdW50IjogMjAyLCJwZXJjZW50IjogIjEzLjE4In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDgyLCJwZXJjZW50IjogIjA5LjgzIn0sImJ5dGVzIjogeyJjb3Vu
dCI6IDMyNzMxNjk0MDIsInBlcmNlbnQiOiAiMTAuMjAifSwiZGF0YSI6ICIyMDI0MDIxNiJ9LHsi
aGl0cyI6IHsiY291bnQiOiAxMTEsInBlcmNlbnQiOiAiMDcuMjQifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogNzUsInBlcmNlbnQiOiAiMDguOTkifSwiYnl0ZXMiOiB7ImNvdW50IjogMzEyNzcxNTY0
NiwicGVyY2VudCI6ICIwOS43NSJ9LCJkYXRhIjogIjIwMjQwMjE1In0seyJoaXRzIjogeyJjb3Vu
dCI6IDE2NiwicGVyY2VudCI6ICIxMC44MyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA4NSwicGVy
Y2VudCI6ICIxMC4xOSJ9LCJieXRlcyI6IHsiY291bnQiOiAzNTg4NjcyNTIxLCJwZXJjZW50Ijog
IjExLjE5In0sImRhdGEiOiAiMjAyNDAyMTQifSx7ImhpdHMiOiB7ImNvdW50IjogMTM0LCJwZXJj
ZW50IjogIjA4Ljc0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDkxLCJwZXJjZW50IjogIjEwLjkx
In0sImJ5dGVzIjogeyJjb3VudCI6IDM1NTQ1ODA0MDUsInBlcmNlbnQiOiAiMTEuMDgifSwiZGF0
YSI6ICIyMDI0MDIxMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAxMDIsInBlcmNlbnQiOiAiMDYuNjUi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogNTIsInBlcmNlbnQiOiAiMDYuMjQifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTgzMDQ3NTcxOCwicGVyY2VudCI6ICIwNS43MSJ9LCJkYXRhIjogIjIwMjQwMjEy
In0seyJoaXRzIjogeyJjb3VudCI6IDEzOCwicGVyY2VudCI6ICIwOS4wMCJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiA0NCwicGVyY2VudCI6ICIwNS4yOCJ9LCJieXRlcyI6IHsiY291bnQiOiAxODM5
MDgzNzMxLCJwZXJjZW50IjogIjA1LjczIn0sImRhdGEiOiAiMjAyNDAyMTEifSx7ImhpdHMiOiB7
ImNvdW50IjogODgsInBlcmNlbnQiOiAiMDUuNzQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNTUs
InBlcmNlbnQiOiAiMDYuNTkifSwiYnl0ZXMiOiB7ImNvdW50IjogMjE1MDk2MTQ2NywicGVyY2Vu
dCI6ICIwNi43MSJ9LCJkYXRhIjogIjIwMjQwMjEwIn0seyJoaXRzIjogeyJjb3VudCI6IDkyLCJw
ZXJjZW50IjogIjA2LjAwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDYzLCJwZXJjZW50IjogIjA3
LjU1In0sImJ5dGVzIjogeyJjb3VudCI6IDI0MDEzNTU2NjIsInBlcmNlbnQiOiAiMDcuNDkifSwi
ZGF0YSI6ICIyMDI0MDIwOSJ9LHsiaGl0cyI6IHsiY291bnQiOiA5NiwicGVyY2VudCI6ICIwNi4y
NiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA1NiwicGVyY2VudCI6ICIwNi43MSJ9LCJieXRlcyI6
IHsiY291bnQiOiAyMTM1NTU4MjgwLCJwZXJjZW50IjogIjA2LjY2In0sImRhdGEiOiAiMjAyNDAy
MDgifSx7ImhpdHMiOiB7ImNvdW50IjogODcsInBlcmNlbnQiOiAiMDUuNjgifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogNTcsInBlcmNlbnQiOiAiMDYuODMifSwiYnl0ZXMiOiB7ImNvdW50IjogMTk0
MDAxMzE3OSwicGVyY2VudCI6ICIwNi4wNSJ9LCJkYXRhIjogIjIwMjQwMjA3In0seyJoaXRzIjog
eyJjb3VudCI6IDExNSwicGVyY2VudCI6ICIwNy41MCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA1
MSwicGVyY2VudCI6ICIwNi4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNjUzNjI0OTY0LCJwZXJj
ZW50IjogIjA1LjE2In0sImRhdGEiOiAiMjAyNDAyMDYifV19LCJyZXF1ZXN0cyI6IHsibWV0YWRh
dGEiOiB7ImJ5dGVzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiAwfSwiYXZnIjogeyJ2YWx1ZSI6IDAs
InBlcmNlbnQiOiAiMDAuMDAifSwibWF4IjogeyJ2YWx1ZSI6IDAsInBlcmNlbnQiOiAiMDAuMDAi
fSwibWluIjogeyJ2YWx1ZSI6IDAsInBlcmNlbnQiOiAiMDAuMDAifX0sInZpc2l0b3JzIjogeyJ0
b3RhbCI6IHsidmFsdWUiOiAwfSwiYXZnIjogeyJ2YWx1ZSI6IDAsInBlcmNlbnQiOiAiMDAuMDAi
fSwibWF4IjogeyJ2YWx1ZSI6IDAsInBlcmNlbnQiOiAiMDAuMDAifSwibWluIjogeyJ2YWx1ZSI6
IDAsInBlcmNlbnQiOiAiMDAuMDAifX0sImhpdHMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDB9LCJh
dmciOiB7InZhbHVlIjogMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJtYXgiOiB7InZhbHVlIjogMCwi
cGVyY2VudCI6ICIwMC4wMCJ9LCJtaW4iOiB7InZhbHVlIjogMCwicGVyY2VudCI6ICIwMC4wMCJ9
fSwiZGF0YSI6IHsidG90YWwiOiB7CSJ2YWx1ZSI6IDB9fX0sImRhdGEiOiBbXX0sInN0YXRpY19y
ZXF1ZXN0cyI6IHsibWV0YWRhdGEiOiB7ImJ5dGVzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiAzMjA3
NjA4MjA1MH0sImF2ZyI6IHsidmFsdWUiOiAxNjAzODA0MDU3NiwicGVyY2VudCI6ICI1MC4wMCJ9
LCJtYXgiOiB7InZhbHVlIjogMzIwNzQxNjQ4NzYsInBlcmNlbnQiOiAiOTkuOTkifSwibWluIjog
eyJ2YWx1ZSI6IDE5MTcxNzQsInBlcmNlbnQiOiAiMDAuMDEifX0sInZpc2l0b3JzIjogeyJ0b3Rh
bCI6IHsidmFsdWUiOiAxMDUxfSwiYXZnIjogeyJ2YWx1ZSI6IDUyNSwicGVyY2VudCI6ICI0OS45
NSJ9LCJtYXgiOiB7InZhbHVlIjogNzA0LCJwZXJjZW50IjogIjY2Ljk4In0sIm1pbiI6IHsidmFs
dWUiOiAzNDcsInBlcmNlbnQiOiAiMzMuMDIifX0sImhpdHMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6
IDE1MzF9LCJhdmciOiB7InZhbHVlIjogNzY1LCJwZXJjZW50IjogIjQ5Ljk3In0sIm1heCI6IHsi
dmFsdWUiOiAxMTcyLCJwZXJjZW50IjogIjc2LjU1In0sIm1pbiI6IHsidmFsdWUiOiAzNTksInBl
cmNlbnQiOiAiMjMuNDUifX0sImRhdGEiOiB7InRvdGFsIjogewkidmFsdWUiOiAyfX19LCJkYXRh
IjogW3siaGl0cyI6IHsiY291bnQiOiAxMTcyLCJwZXJjZW50IjogIjc2LjQ1In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDcwNCwicGVyY2VudCI6ICI4NC40MSJ9LCJieXRlcyI6IHsiY291bnQiOiAz
MjA3NDE2NDg3NiwicGVyY2VudCI6ICI5OS45OSJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wi
OiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE4LjBcL3hlbi00LjE4LjAu
dGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6IDM1OSwicGVyY2VudCI6ICIyMy40MiJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAzNDcsInBlcmNlbnQiOiAiNDEuNjEifSwiYnl0ZXMiOiB7ImNvdW50
IjogMTkxNzE3NCwicGVyY2VudCI6ICIwMC4wMSJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wi
OiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE4LjBcL3hlbi00LjE4LjAu
dGFyLmd6LnNpZyJ9XX0sIm5vdF9mb3VuZCI6IHsibWV0YWRhdGEiOiB7ImJ5dGVzIjogeyJ0b3Rh
bCI6IHsidmFsdWUiOiAxMTQ1Mn0sImF2ZyI6IHsidmFsdWUiOiAxMTQ1MiwicGVyY2VudCI6ICIx
MDAuMDAifSwibWF4IjogeyJ2YWx1ZSI6IDExNDUyLCJwZXJjZW50IjogIjEwMC4wMCJ9LCJtaW4i
OiB7InZhbHVlIjogMTE0NTIsInBlcmNlbnQiOiAiMTAwLjAwIn19LCJ2aXNpdG9ycyI6IHsidG90
YWwiOiB7InZhbHVlIjogMH0sImF2ZyI6IHsidmFsdWUiOiAwLCJwZXJjZW50IjogIjAwLjAwIn0s
Im1heCI6IHsidmFsdWUiOiAwLCJwZXJjZW50IjogIjAwLjAwIn0sIm1pbiI6IHsidmFsdWUiOiAw
LCJwZXJjZW50IjogIjAwLjAwIn19LCJoaXRzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiAyfSwiYXZn
IjogeyJ2YWx1ZSI6IDIsInBlcmNlbnQiOiAiMTAwLjAwIn0sIm1heCI6IHsidmFsdWUiOiAyLCJw
ZXJjZW50IjogIjEwMC4wMCJ9LCJtaW4iOiB7InZhbHVlIjogMiwicGVyY2VudCI6ICIxMDAuMDAi
fX0sImRhdGEiOiB7InRvdGFsIjogewkidmFsdWUiOiAxfX19LCJkYXRhIjogW3siaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDAsInBl
cmNlbnQiOiAiMDAuMDAifSwiYnl0ZXMiOiB7ImNvdW50IjogMTE0NTIsInBlcmNlbnQiOiAiMDAu
MDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3Jl
bGVhc2VcL3hlblwvNC4xOC4wXC94ZW4tNC4xOC4wLnRhci5neiwifV19LCJob3N0cyI6IHsibWV0
YWRhdGEiOiB7ImJ5dGVzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiAzMjA3NjA5MzUwMn0sImF2ZyI6
IHsidmFsdWUiOiA0MDA0NTA2MCwicGVyY2VudCI6ICIwMC4xMiJ9LCJtYXgiOiB7InZhbHVlIjog
MjU2MDk2OTMwLCJwZXJjZW50IjogIjAwLjgwIn0sIm1pbiI6IHsidmFsdWUiOiA2NDUsInBlcmNl
bnQiOiAiMDAuMDAifX0sInZpc2l0b3JzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiA4MzR9LCJhdmci
OiB7InZhbHVlIjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJtYXgiOiB7InZhbHVlIjogNiwicGVy
Y2VudCI6ICIwMC43MiJ9LCJtaW4iOiB7InZhbHVlIjogMSwicGVyY2VudCI6ICIwMC4xMiJ9fSwi
aGl0cyI6IHsidG90YWwiOiB7InZhbHVlIjogMTUzM30sImF2ZyI6IHsidmFsdWUiOiAxLCJwZXJj
ZW50IjogIjAwLjA3In0sIm1heCI6IHsidmFsdWUiOiA1MCwicGVyY2VudCI6ICIwMy4yNiJ9LCJt
aW4iOiB7InZhbHVlIjogMSwicGVyY2VudCI6ICIwMC4wNyJ9fSwiZGF0YSI6IHsidG90YWwiOiB7
CSJ2YWx1ZSI6IDgwMX19fSwiZGF0YSI6IFt7ImhpdHMiOiB7ImNvdW50IjogNTAsInBlcmNlbnQi
OiAiMDMuMjYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1NzI0MTcyMSwicGVyY2VudCI6ICIwMC4xOCJ9LCJkYXRhIjogIjEw
Mi4zOC4zMC4yNDkifSx7ImhpdHMiOiB7ImNvdW50IjogMjksInBlcmNlbnQiOiAiMDEuODkifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291
bnQiOiA1NDQyMjMxNiwicGVyY2VudCI6ICIwMC4xNyJ9LCJkYXRhIjogIjExNy4xNTQuODguMTI3
In0seyJoaXRzIjogeyJjb3VudCI6IDI2LCJwZXJjZW50IjogIjAxLjcwIn0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNjkwNTU3
NDcsInBlcmNlbnQiOiAiMDAuMjIifSwiZGF0YSI6ICIxMjUuMTY4LjI1My40MSJ9LHsiaGl0cyI6
IHsiY291bnQiOiAyNCwicGVyY2VudCI6ICIwMS41NyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDEwNDY5MDgwNywicGVyY2Vu
dCI6ICIwMC4zMyJ9LCJkYXRhIjogIjE3OS42Ni42Mi4zNyJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
MCwicGVyY2VudCI6ICIwMS4zMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDE0NjQ3NjIyOSwicGVyY2VudCI6ICIwMC40NiJ9
LCJkYXRhIjogIjE0OS4xMDIuMjUyLjExIn0seyJoaXRzIjogeyJjb3VudCI6IDIwLCJwZXJjZW50
IjogIjAxLjMwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNjI5OTA5MjksInBlcmNlbnQiOiAiMDAuMjAifSwiZGF0YSI6ICI0
MS4yMzAuMjE2LjI0NiJ9LHsiaGl0cyI6IHsiY291bnQiOiAxNiwicGVyY2VudCI6ICIwMS4wNCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDg3MjQyOTQ0LCJwZXJjZW50IjogIjAwLjI3In0sImRhdGEiOiAiMjE3LjE3OC4xOTQu
MTkifSx7ImhpdHMiOiB7ImNvdW50IjogMTYsInBlcmNlbnQiOiAiMDEuMDQifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA2NDYy
Nzk0OCwicGVyY2VudCI6ICIwMC4yMCJ9LCJkYXRhIjogIjM3LjE3NC4yNDAuMjQ2In0seyJoaXRz
IjogeyJjb3VudCI6IDE1LCJwZXJjZW50IjogIjAwLjk4In0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNjYyNjIxNiwicGVyY2Vu
dCI6ICIwMC4wMiJ9LCJkYXRhIjogIjEwNS4xMDIuNTcuMTc4In0seyJoaXRzIjogeyJjb3VudCI6
IDE1LCJwZXJjZW50IjogIjAwLjk4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMjU2MDk2OTMwLCJwZXJjZW50IjogIjAwLjgw
In0sImRhdGEiOiAiMTA0LjI4LjE5Mi45OSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxNSwicGVyY2Vu
dCI6ICIwMC45OCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0s
ImJ5dGVzIjogeyJjb3VudCI6IDgzNTQ0MTE1LCJwZXJjZW50IjogIjAwLjI2In0sImRhdGEiOiAi
NDkuNDMuMjYuODYifSx7ImhpdHMiOiB7ImNvdW50IjogMTMsInBlcmNlbnQiOiAiMDAuODUifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4yNCJ9LCJieXRlcyI6IHsiY291
bnQiOiA3NDczMDg0NCwicGVyY2VudCI6ICIwMC4yMyJ9LCJkYXRhIjogIjg5LjIxNi45NS4xMjAi
fSx7ImhpdHMiOiB7ImNvdW50IjogMTMsInBlcmNlbnQiOiAiMDAuODUifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA2OTg0OTQ2
MSwicGVyY2VudCI6ICIwMC4yMiJ9LCJkYXRhIjogIjEwNy41LjIxOC4xMTYifSx7ImhpdHMiOiB7
ImNvdW50IjogMTIsInBlcmNlbnQiOiAiMDAuNzgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA3NDk3MzI2MCwicGVyY2VudCI6
ICIwMC4yMyJ9LCJkYXRhIjogIjEwNy4yMTEuMjE2LjE3NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAx
MSwicGVyY2VudCI6ICIwMC43MiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDg3NDIxODEyLCJwZXJjZW50IjogIjAwLjI3In0s
ImRhdGEiOiAiMTcxLjEwNS4xNzcuMTM2In0seyJoaXRzIjogeyJjb3VudCI6IDExLCJwZXJjZW50
IjogIjAwLjcyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNjMxMDg5MjEsInBlcmNlbnQiOiAiMDAuMjAifSwiZGF0YSI6ICIx
MDUuMTU1LjI0NC40NiJ9LHsiaGl0cyI6IHsiY291bnQiOiAxMSwicGVyY2VudCI6ICIwMC43MiJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDM4MzExNTY5LCJwZXJjZW50IjogIjAwLjEyIn0sImRhdGEiOiAiMzcuMjYuODEuNTQi
fSx7ImhpdHMiOiB7ImNvdW50IjogMTAsInBlcmNlbnQiOiAiMDAuNjUifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNDgxMDI0
MSwicGVyY2VudCI6ICIwMC4wNSJ9LCJkYXRhIjogIjQ2LjE4Mi41Mi44MyJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAxMCwicGVyY2VudCI6ICIwMC42NSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDM5ODEwNTQsInBlcmNlbnQiOiAi
MDAuMDEifSwiZGF0YSI6ICIxOTYuNjQuNTguOTkifSx7ImhpdHMiOiB7ImNvdW50IjogOSwicGVy
Y2VudCI6ICIwMC41OSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEy
In0sImJ5dGVzIjogeyJjb3VudCI6IDI2NzY4MzAsInBlcmNlbnQiOiAiMDAuMDEifSwiZGF0YSI6
ICIxNDYuNzAuMTQ1Ljk0In0seyJoaXRzIjogeyJjb3VudCI6IDgsInBlcmNlbnQiOiAiMDAuNTIi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogNiwicGVyY2VudCI6ICIwMC43MiJ9LCJieXRlcyI6IHsi
Y291bnQiOiAyMjE0MjMxMzcsInBlcmNlbnQiOiAiMDAuNjkifSwiZGF0YSI6ICIxOTQuMTUzLjg4
LjEzNSJ9LHsiaGl0cyI6IHsiY291bnQiOiA3LCJwZXJjZW50IjogIjAwLjQ2In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNzMx
ODY5MCwicGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjogIjE4OS41MS45Ni4yMjMifSx7ImhpdHMi
OiB7ImNvdW50IjogNiwicGVyY2VudCI6ICIwMC4zOSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJjb3VudCI6IDY4OTU4NDY2LCJwZXJjZW50
IjogIjAwLjIxIn0sImRhdGEiOiAiMTMxLjE1MC4yMTIuMTIzIn0seyJoaXRzIjogeyJjb3VudCI6
IDUsInBlcmNlbnQiOiAiMDAuMzMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4yNCJ9LCJieXRlcyI6IHsiY291bnQiOiA2NDU5NTY5NSwicGVyY2VudCI6ICIwMC4yMCJ9
LCJkYXRhIjogIjc5LjExNy4yNy4xMzIifSx7ImhpdHMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6
ICIwMC4zMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDU5MzAzNjQ0LCJwZXJjZW50IjogIjAwLjE4In0sImRhdGEiOiAiNDIu
MTA0LjE0MS4xNDAifSx7ImhpdHMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4zMyJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDE1NTg5MzU4OCwicGVyY2VudCI6ICIwMC40OSJ9LCJkYXRhIjogIjYyLjE2My4xODEuNDMi
fSx7ImhpdHMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4zMyJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDk4Mjc5NjIz
LCJwZXJjZW50IjogIjAwLjMxIn0sImRhdGEiOiAiMTM2LjIyNi41MS4xMTcifSx7ImhpdHMiOiB7
ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4yNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU1NDQzODUwLCJwZXJjZW50Ijog
IjAwLjE3In0sImRhdGEiOiAiNzguMTM5LjkwLjExMCJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJw
ZXJjZW50IjogIjAwLjI2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTg1ODE4NTUsInBlcmNlbnQiOiAiMDAuMTgifSwiZGF0
YSI6ICIxOTYuNzUuMTguMTE4In0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAu
MjYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4yNCJ9LCJieXRlcyI6
IHsiY291bnQiOiAyMDc4NDk1NzYsInBlcmNlbnQiOiAiMDAuNjUifSwiZGF0YSI6ICI1Mi4xNjcu
MTQ0LjE4MSJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjI2In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50Ijog
MTEwODExNTY3LCJwZXJjZW50IjogIjAwLjM1In0sImRhdGEiOiAiMTE4LjIxMC4yMTAuNSJ9LHsi
aGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjI2In0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50IjogNTgyNDA2NjUsInBl
cmNlbnQiOiAiMDAuMTgifSwiZGF0YSI6ICI0Ny40Mi4xNy4xMTAifSx7ImhpdHMiOiB7ImNvdW50
IjogNCwicGVyY2VudCI6ICIwMC4yNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU1NTcxNTIzLCJwZXJjZW50IjogIjAwLjE3
In0sImRhdGEiOiAiMTA5LjE1Ny45MC42In0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQi
OiAiMDAuMjYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1ODQ5Nzg2MywicGVyY2VudCI6ICIwMC4xOCJ9LCJkYXRhIjogIjE5
NC4yMzAuMTQ4LjEzMSJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjI2In0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNv
dW50IjogNTk5MjY4NTgsInBlcmNlbnQiOiAiMDAuMTkifSwiZGF0YSI6ICI3MC41MS4yMzguNjQi
fSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4yNiJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU1MTk5NTIw
LCJwZXJjZW50IjogIjAwLjE3In0sImRhdGEiOiAiODYuMTI3LjI1NC42MiJ9LHsiaGl0cyI6IHsi
Y291bnQiOiA0LCJwZXJjZW50IjogIjAwLjI2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTQyOTYwMzEsInBlcmNlbnQiOiAi
MDAuMTcifSwiZGF0YSI6ICIyLjUyLjMxLjIwNCJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJj
ZW50IjogIjAwLjI2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTM2NzI0MzQsInBlcmNlbnQiOiAiMDAuMTcifSwiZGF0YSI6
ICIyMDkuMTg4LjM1LjI0MiJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjI2
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTYxMTQwMTksInBlcmNlbnQiOiAiMDAuMDUifSwiZGF0YSI6ICIxNDkuMTQwLjE1
Ni4xMTQifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4yNiJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU1
NzA3ODQ5LCJwZXJjZW50IjogIjAwLjE3In0sImRhdGEiOiAiOTQuMzQuMTk5Ljc1In0seyJoaXRz
IjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMjYifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAyODQ5MTcsInBlcmNlbnQi
OiAiMDAuMDAifSwiZGF0YSI6ICIxMTcuMjMzLjE2My4yIn0seyJoaXRzIjogeyJjb3VudCI6IDQs
InBlcmNlbnQiOiAiMDAuMjYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA5NDY3MTk1LCJwZXJjZW50IjogIjAwLjAzIn0sImRh
dGEiOiAiMTg3LjE5LjIyMi4yMTAifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIw
MC4yNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVz
IjogeyJjb3VudCI6IDYzNzA2MTAzLCJwZXJjZW50IjogIjAwLjIwIn0sImRhdGEiOiAiOTQuNDMu
ODkuNzQifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4yNiJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU1
MjM4MjI5LCJwZXJjZW50IjogIjAwLjE3In0sImRhdGEiOiAiMzcuNDcuMTcxLjE0MSJ9LHsiaGl0
cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50IjogNjE2ODAyMTksInBlcmNl
bnQiOiAiMDAuMTkifSwiZGF0YSI6ICI4Ny4xMzAuMTEzLjEwNiJ9LHsiaGl0cyI6IHsiY291bnQi
OiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNjAxMDA2NzQsInBlcmNlbnQiOiAiMDAuMTki
fSwiZGF0YSI6ICI2Ni4xNzcuOTcuMTg4In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQi
OiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4yNCJ9LCJi
eXRlcyI6IHsiY291bnQiOiAxMzg3MjI2MTcsInBlcmNlbnQiOiAiMDAuNDMifSwiZGF0YSI6ICI2
OS4xMjcuMjI3LjE1MyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNv
dW50IjogMTA3NzQyNDQwLCJwZXJjZW50IjogIjAwLjM0In0sImRhdGEiOiAiMTA5LjI0Mi4yMjQu
MTAxIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAzMDYx
MDQyMywicGVyY2VudCI6ICIwMC4xMCJ9LCJkYXRhIjogIjE4NS42NS4xMzUuMTg4In0seyJoaXRz
IjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MjEyNTgsInBlcmNl
bnQiOiAiMDAuMzIifSwiZGF0YSI6ICI5OC4xODguMTAyLjg5In0seyJoaXRzIjogeyJjb3VudCI6
IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2OTUyNCwicGVyY2VudCI6ICIwMC4xNiJ9
LCJkYXRhIjogIjE5My40OC40NS4yMTIifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6
ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDE3MDI0NDA0LCJwZXJjZW50IjogIjAwLjA1In0sImRhdGEiOiAiNS4x
MjIuNzAuMjYifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJjb3VudCI6
IDQ4NTc0MTcxLCJwZXJjZW50IjogIjAwLjE1In0sImRhdGEiOiAiOTAuMjE3LjI3LjU0In0seyJo
aXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMiwicGVyY2VudCI6ICIwMC4yNCJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDU4NDc3MCwicGVy
Y2VudCI6ICIwMC4wMyJ9LCJkYXRhIjogIjE0LjEzOS4xMjIuMTM5In0seyJoaXRzIjogeyJjb3Vu
dCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA5MjAxMjI3LCJwZXJjZW50IjogIjAwLjAz
In0sImRhdGEiOiAiMTk2LjExOS44Ni45NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50
IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogODI4MTUxNzgsInBlcmNlbnQiOiAiMDAuMjYifSwiZGF0YSI6ICIx
ODUuMTc5LjE0Mi4yMjkifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDk5MDA3NDksInBlcmNlbnQiOiAiMDAuMDMifSwiZGF0YSI6ICI4Ny4xODIuMTE0LjE4
MiJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTgzNzkx
NjUsInBlcmNlbnQiOiAiMDAuMTgifSwiZGF0YSI6ICIzNy4xNjkuMTc0LjI0NyJ9LHsiaGl0cyI6
IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTAzOTI1OTYzLCJwZXJjZW50
IjogIjAwLjMyIn0sImRhdGEiOiAiODMuMjA0LjQzLjU3In0seyJoaXRzIjogeyJjb3VudCI6IDMs
InBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIw
MC4zNiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzQ4MSwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRh
IjogIjM1LjE2MC4yNy4yMjEifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4y
MCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjog
eyJjb3VudCI6IDEwMzkyNTk2MywicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjc1LjcwLjIx
OS45NyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTQ0
ODc3ODcsInBlcmNlbnQiOiAiMDAuMTcifSwiZGF0YSI6ICIyMTcuMjM1LjIyNy4yMTAifSx7Imhp
dHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDE2MDYzOTAsInBlcmNl
bnQiOiAiMDAuMDEifSwiZGF0YSI6ICIxMzYuMjMuMjEuODYifSx7ImhpdHMiOiB7ImNvdW50Ijog
MywicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDEwMzkyMTI1OCwicGVyY2VudCI6ICIwMC4zMiJ9
LCJkYXRhIjogIjEwMy44OC4xMDEuMjA3In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQi
OiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1MTIzMjMzNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjQ5
LjM3LjMyLjIwMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50
IjogMTA0MzIyNDM3LCJwZXJjZW50IjogIjAwLjMzIn0sImRhdGEiOiAiMTczLjE1LjE2NC44MSJ9
LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50IjogMTA4MDM5NDcs
InBlcmNlbnQiOiAiMDAuMDMifSwiZGF0YSI6ICI4OC45Ny40Ni4yMjMifSx7ImhpdHMiOiB7ImNv
dW50IjogMywicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDc2NzMyOTQxLCJwZXJjZW50IjogIjAw
LjI0In0sImRhdGEiOiAiMTY1LjIyNS4xMDQuMTMzIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBl
cmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4x
MiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNTkwODM2MiwicGVyY2VudCI6ICIwMC4wNSJ9LCJkYXRh
IjogIjE3OC4yMDcuOS4yMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIw
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTAzOTI1MzgzLCJwZXJjZW50IjogIjAwLjMyIn0sImRhdGEiOiAiNzguMTI2LjIx
NS45NyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50IjogMTU1
ODg3MTgyLCJwZXJjZW50IjogIjAwLjQ5In0sImRhdGEiOiAiNDAuNzcuMTY3LjI1In0seyJoaXRz
IjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAzODYxMTQ3LCJwZXJjZW50
IjogIjAwLjAxIn0sImRhdGEiOiAiNjcuNDQuMjA4LjE2NiJ9LHsiaGl0cyI6IHsiY291bnQiOiAz
LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTcwNjU4NTgsInBlcmNlbnQiOiAiMDAuMTgifSwi
ZGF0YSI6ICI0NS4xMzAuMjAwLjExNyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50Ijog
IjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0
ZXMiOiB7ImNvdW50IjogMTAzOTI1MzgzLCJwZXJjZW50IjogIjAwLjMyIn0sImRhdGEiOiAiNS4z
OS4yMzMuNjEifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6
IDE1NTg4MTgxNywicGVyY2VudCI6ICIwMC40OSJ9LCJkYXRhIjogIjUyLjE2Ny4xNDQuMjMzIn0s
eyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNTU4ODE4MTcs
InBlcmNlbnQiOiAiMDAuNDkifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjIyNiJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTgzNTU3NzMsInBlcmNlbnQiOiAi
MDAuMTgifSwiZGF0YSI6ICIxMDguMzUuMjQuNDgifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVy
Y2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEy
In0sImJ5dGVzIjogeyJjb3VudCI6IDcyODQzMzA3LCJwZXJjZW50IjogIjAwLjIzIn0sImRhdGEi
OiAiNDUuMTAuMTUyLjIyOSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIw
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogMjAzNzE1NDIsInBlcmNlbnQiOiAiMDAuMDYifSwiZGF0YSI6ICIxODQuMTYwLjc0
LjE1MSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTY4
NDA2NDksInBlcmNlbnQiOiAiMDAuMTgifSwiZGF0YSI6ICIxNTcuMjMxLjcuMTEzIn0seyJoaXRz
IjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMjQ1OCwicGVyY2VudCI6
ICIwMC4wMCJ9LCJkYXRhIjogIjE4MC4yNDQuMTY0LjE5MCJ9LHsiaGl0cyI6IHsiY291bnQiOiAz
LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTc5ODA3NDUsInBlcmNlbnQiOiAiMDAuMTgifSwi
ZGF0YSI6ICI3Ny4yMDUuMTE3LjMifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIw
MC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVz
IjogeyJjb3VudCI6IDEwMzkyNTk2MywicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjc2Ljgu
MTQ3Ljc1In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4yNCJ9LCJieXRlcyI6IHsiY291bnQiOiA4
MzU3ODA3LCJwZXJjZW50IjogIjAwLjAzIn0sImRhdGEiOiAiMTAzLjIzMS4yMzYuMiJ9LHsiaGl0
cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5Njg5NDQsInBlcmNl
bnQiOiAiMDAuMTYifSwiZGF0YSI6ICIzNS4xMzEuMTgwLjM0In0seyJoaXRzIjogeyJjb3VudCI6
IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA0MDc0MjgyMSwicGVyY2VudCI6ICIwMC4xMyJ9
LCJkYXRhIjogIjM2LjExMi4xNjUuNDgifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6
ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5
dGVzIjogeyJjb3VudCI6IDEwMzkzMDc0MywicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjUy
LjE2Ny4xNDQuMTQ1In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291
bnQiOiA1NzA2NzE1MCwicGVyY2VudCI6ICIwMC4xOCJ9LCJkYXRhIjogIjE1Ni4xNDYuNjAuMTMz
In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MzA2
NjgsInBlcmNlbnQiOiAiMDAuMzIifSwiZGF0YSI6ICIxMTMuMTU4LjE5Ny44NiJ9LHsiaGl0cyI6
IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTM1ODQ2OTksInBlcmNlbnQi
OiAiMDAuMDQifSwiZGF0YSI6ICI1OC4xNjEuNjUuMTkifSx7ImhpdHMiOiB7ImNvdW50IjogMywi
cGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU5MTgxNzIyLCJwZXJjZW50IjogIjAwLjE4In0sImRh
dGEiOiAiNTAuNDYuMzguNDAifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4y
MCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjog
eyJjb3VudCI6IDgwODI0MzE4LCJwZXJjZW50IjogIjAwLjI1In0sImRhdGEiOiAiNDkuMzYuMTgu
MjMzIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA4MzQx
ODE4OSwicGVyY2VudCI6ICIwMC4yNiJ9LCJkYXRhIjogIjkyLjE4NC4xMDIuOTcifSx7ImhpdHMi
OiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDQyNDExMTIyLCJwZXJjZW50
IjogIjAwLjEzIn0sImRhdGEiOiAiMTY2LjE5NC4xNDcuMzYifSx7ImhpdHMiOiB7ImNvdW50Ijog
MywicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDQzOTQ1ODI3LCJwZXJjZW50IjogIjAwLjE0In0s
ImRhdGEiOiAiOTUuOTEuMjQ4LjE5MCJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50Ijog
IjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0
ZXMiOiB7ImNvdW50IjogNTE5Njg5NDQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIyMTYu
MjUyLjY2Ljk1In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQi
OiAxMDM5MjEyNTgsInBlcmNlbnQiOiAiMDAuMzIifSwiZGF0YSI6ICI3NC44MS4yMDYuOTcifSx7
ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDQzODcxODc1LCJw
ZXJjZW50IjogIjAwLjE0In0sImRhdGEiOiAiNjMuMjUxLjEyMi4xMTIifSx7ImhpdHMiOiB7ImNv
dW50IjogMywicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUyMDcyMDQ3LCJwZXJjZW50IjogIjAw
LjE2In0sImRhdGEiOiAiMTA3Ljc3LjIyNy45NyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJj
ZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTc1ODE0MCwicGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjog
IjEwMy4yMTEuNDAuMTc3In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsi
Y291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjIuMjcuMzEuMTE0
In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MjQ3
MzgsInBlcmNlbnQiOiAiMDAuMzIifSwiZGF0YSI6ICIxODguMTcyLjEyMS4xNTQifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4MzAzLCJwZXJjZW50
IjogIjAwLjE2In0sImRhdGEiOiAiMTU1LjE5MC4xNy41In0seyJoaXRzIjogeyJjb3VudCI6IDIs
InBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJk
YXRhIjogIjEzMS4xOTYuMTc4LjU4In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRl
cyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjEzOC40
OC4zMy4xMDgifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6
IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTQ3LjEyNC45NS4xNzYifSx7
ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDQ4MTMwMjEwLCJw
ZXJjZW50IjogIjAwLjE1In0sImRhdGEiOiAiNjYuMjI2LjEwMi4yNTEifSx7ImhpdHMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAw
LjE2In0sImRhdGEiOiAiMTAyLjE2NC45Ny4yNSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6
ICIxMzcuNjMuMjE4LjIwOSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEz
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTgyOTAwOTMsInBlcmNlbnQiOiAiMDAuMDYifSwiZGF0YSI6ICI5Mi43My4yNi4x
MjkifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4
Mjk5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiNzMuMTY1LjE5Mi4yOCJ9LHsiaGl0cyI6
IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQi
OiAiMDAuMTYifSwiZGF0YSI6ICI4Ni4zLjE2MC40NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJw
ZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0
YSI6ICIxMi4xNDYuMTUxLjk4In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6
IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjEzMC4yMjEu
MTQ1LjUifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUx
OTY4Mjk5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTk5LjIxMi42Ny4wIn0seyJoaXRz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2Vu
dCI6ICIwMC4xNiJ9LCJkYXRhIjogIjE3OC4yNC4yMzMuMTU0In0seyJoaXRzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9
LCJkYXRhIjogIjkxLjM2LjU0LjIyMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50Ijog
IjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0
ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI0NS4x
OC4zMS4xMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50Ijog
NTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI0Ni4xMTIuMC44NCJ9LHsiaGl0
cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTYsInBlcmNl
bnQiOiAiMDAuMTYifSwiZGF0YSI6ICI1MC4yNDIuMjYuMTkzIn0seyJoaXRzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9
LCJkYXRhIjogIjcxLjcxLjg1Ljg0In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRl
cyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjc2LjE0
OS4xMy4yNTMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJjb3VudCI6
IDEwMzkyNDczOCwicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjg2LjEyMC4yMDguMTQ3In0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwi
cGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjE4Ny4xMy4xNzMuMTc3In0seyJoaXRzIjogeyJj
b3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA5MzU0NTc2OSwicGVyY2VudCI6ICIw
MC4yOSJ9LCJkYXRhIjogIjE4Ny4xOTAuMTk5LjIzMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJw
ZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0
YSI6ICI3OC45Mi4xMTQuMTg0In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6
IHsiY291bnQiOiAzNDk4NDAsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICI4Mi4xNDIuMTUy
LjE5NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5
NjgyOTksInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI2Mi4xMi4xMjkuMTM5In0seyJoaXRz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2Vu
dCI6ICIwMC4xNiJ9LCJkYXRhIjogIjc5LjExNi41OC45OSJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNjczODQwNDMsInBlcmNlbnQiOiAiMDAuMjEifSwi
ZGF0YSI6ICIxNzIuODMuNDcuMTU2In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRl
cyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjc1LjEz
NS4xNTAuODcifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6
IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiNzMuMTU1LjIxMy4xMzgifSx7
ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJw
ZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTc2LjE3OS43NC4yNDYifSx7ImhpdHMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJwZXJjZW50IjogIjAw
LjE2In0sImRhdGEiOiAiMTg4LjE5MC4yNDUuMjE0In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBl
cmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4y
NCJ9LCJieXRlcyI6IHsiY291bnQiOiA5MzE0MjUzNCwicGVyY2VudCI6ICIwMC4yOSJ9LCJkYXRh
IjogIjc5LjExNi44LjE5NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEz
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogNTU2NDk2NTMsInBlcmNlbnQiOiAiMDAuMTcifSwiZGF0YSI6ICIzNy4xNTIuMTMx
LjI0OSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTAz
OTI0Nzg4LCJwZXJjZW50IjogIjAwLjMyIn0sImRhdGEiOiAiNDAuNzcuMTY3LjEifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDQ0OTgyMDUsInBlcmNlbnQi
OiAiMDAuMDEifSwiZGF0YSI6ICI4Ni4yMDUuMTk2LjEyMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTI2MDE1NzQsInBlcmNlbnQiOiAiMDAuMDQifSwi
ZGF0YSI6ICIxOTMuMTcwLjIyOC4xMzYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDIzOTY0Nzg4LCJwZXJjZW50IjogIjAwLjA3In0sImRhdGEiOiAiMjEy
LjE3MC4yOC4yMjUifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3Vu
dCI6IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiNzcuMTMzLjI0OS43NSJ9
LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQs
InBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIxOTIuMTM1LjE1MS4xNTEifSx7ImhpdHMiOiB7
ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50Ijog
IjAwLjE2In0sImRhdGEiOiAiMTk0Ljg0LjI1LjU0In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBl
cmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4x
MiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRh
IjogIjE4My45My4xNzQuNDYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4x
MyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjog
eyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiNzIuMjAxLjgw
LjIwNyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTAz
OTI0Nzg4LCJwZXJjZW50IjogIjAwLjMyIn0sImRhdGEiOiAiNTIuMTY3LjE0NC4yMzUifSx7Imhp
dHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJwZXJj
ZW50IjogIjAwLjE2In0sImRhdGEiOiAiNjIuMTQzLjIxMy43NSJ9LHsiaGl0cyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTIyOTA1NzMsInBlcmNlbnQiOiAiMDAuMTYi
fSwiZGF0YSI6ICI4NS4xMC40OS4yNTQifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiODIu
MjAzLjcyLjMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJjb3VudCI6
IDEwMzkyNDczOCwicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjk5LjcyLjE5Mi4xNDIifSx7
ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJw
ZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTQ1LjIyNC42Ny4zMiJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAu
MTYifSwiZGF0YSI6ICIxMDQuMjguNDAuMTQwIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9
LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjog
IjE5MC4xNTMuODYuMjE3In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsi
Y291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjg3Ljg4LjE3My4x
NTIifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDEwMzkx
OTQ1MywicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjQxLjIxNi4yMDIuMTgwIn0seyJoaXRz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA0Njc4OTYxNCwicGVyY2Vu
dCI6ICIwMC4xNSJ9LCJkYXRhIjogIjQ1LjE3Ny40My4yNCJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTYifSwi
ZGF0YSI6ICI4Mi4xNTQuMTYxLjI0NSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50Ijog
IjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0
ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIxMDku
MjQ3LjEwMC41MiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50
IjogMzEzMjg0NzYsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0YSI6ICI5MC44NC4yMzcuMTIzIn0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwi
cGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjkzLjIwNS4xMzcuODMifSx7ImhpdHMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJwZXJjZW50IjogIjAw
LjE2In0sImRhdGEiOiAiODkuOTUuMjE1LjExOSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6
ICI5MS40OC4xMjAuMTMxIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsi
Y291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjExNC4yMDMuMTEw
LjY3In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2
MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjYwLjI0My40Ni4zMiJ9LHsiaGl0cyI6
IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQi
OiAiMDAuMTYifSwiZGF0YSI6ICIxMzQuOTYuMTkxLjI1NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYifSwi
ZGF0YSI6ICI3Mi42NS41OS4yMDIifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIw
MC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVz
IjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTg1LjIw
My4yMTkuMjU0In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQi
OiA5MDEyNDI5LCJwZXJjZW50IjogIjAwLjAzIn0sImRhdGEiOiAiMjEyLjI0LjEzMi42NiJ9LHsi
aGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMjg3NjkzOTAsInBl
cmNlbnQiOiAiMDAuMDkifSwiZGF0YSI6ICI5Mi4xODcuMTk0LjcxIn0seyJoaXRzIjogeyJjb3Vu
dCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4x
NiJ9LCJkYXRhIjogIjYwLjIyNi4xNjUuNTYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2Vu
dCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0s
ImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAi
MTc5LjIxNy4yNDYuNjEifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDQ5OTA1MjUsInBlcmNlbnQiOiAiMDAuMDIifSwiZGF0YSI6ICIxMDkuMjMzLjk1Ljg4
In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5
NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjE0OS4xMjAuNzAuMTgyIn0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6
ICIwMC4xNiJ9LCJkYXRhIjogIjE0OS4xOTkuODAuMTI4In0seyJoaXRzIjogeyJjb3VudCI6IDIs
InBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MjAwMzMsInBlcmNlbnQiOiAiMDAuMzIifSwi
ZGF0YSI6ICI2Ny4xNzYuMjAzLjIwMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50Ijog
IjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0
ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI3NC4x
MzEuMTc0LjEyMiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50
IjogMTAzOTIwMDMzLCJwZXJjZW50IjogIjAwLjMyIn0sImRhdGEiOiAiNzguMzIuMTUwLjEyMyJ9
LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQs
InBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIxNTYuMzQuMTg5LjQxIn0seyJoaXRzIjogeyJj
b3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIw
MC4xNiJ9LCJkYXRhIjogIjIyMC44Ny45Ny41MyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6
ICI5MS4xMDIuMTgwLjE5MSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEz
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI4Ni45MC4zOS4y
MjYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4
Mjk5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTA5LjI0OC4zNy4xMjUifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50
IjogIjAwLjE2In0sImRhdGEiOiAiMTA5LjE1OS4xMi4xNjAifSx7ImhpdHMiOiB7ImNvdW50Ijog
MiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjE2In0s
ImRhdGEiOiAiOTkuMTA2LjU0LjU4In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRl
cyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjEwOC42
Ny4yMS4xNjYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6
IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTg1LjE5Mi4xNi41OCJ9LHsi
aGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTEwODY0MjQsInBl
cmNlbnQiOiAiMDAuMDMifSwiZGF0YSI6ICI0Ni4xMTQuMTM4LjMxIn0seyJoaXRzIjogeyJjb3Vu
dCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAyODQ4NjIwMiwicGVyY2VudCI6ICIwMC4w
OSJ9LCJkYXRhIjogIjE5NC44Ny4zOS4xNTYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2Vu
dCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0s
ImJ5dGVzIjogeyJjb3VudCI6IDEwMzkyNDc4OCwicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjog
IjUyLjE2Ny4xNDQuMjM2In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsi
Y291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjkzLjE5OC4yOS4y
MTcifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4
Mjk5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTc2LjEwMC40My4xMiJ9LHsiaGl0cyI6
IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQi
OiAiMDAuMTYifSwiZGF0YSI6ICIxNzcuMTg1LjMwLjE0MiJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTYifSwi
ZGF0YSI6ICI5MC4xNTYuNTYuMjI4In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRl
cyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjY4LjEz
My40NC42In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1
Mjk2MDk5NywicGVyY2VudCI6ICIwMC4xNyJ9LCJkYXRhIjogIjE0Ni43MC4xODcuNDkifSx7Imhp
dHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDY1MDIyMzcsInBlcmNl
bnQiOiAiMDAuMDIifSwiZGF0YSI6ICI4MC45NC4yNTAuMTc1In0seyJoaXRzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xNiJ9
LCJkYXRhIjogIjE4OC40Ny4xMTUuNTMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDI3OTQwMTMwLCJwZXJjZW50IjogIjAwLjA5In0sImRhdGEiOiAiMTg3
LjE4OS4xNTUuNjIifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3Vu
dCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTczLjIzOS4yMTcuMTU1
In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAx
NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjg5LjExNC4yNTUuMTYxIn0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MTk0NTMsInBlcmNlbnQi
OiAiMDAuMzIifSwiZGF0YSI6ICI4MS41My4xNTUuMjA0In0seyJoaXRzIjogeyJjb3VudCI6IDIs
InBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAyOTA5NDUwLCJwZXJjZW50IjogIjAwLjAxIn0sImRh
dGEiOiAiMTg1LjE5Ny4xOTQuNDMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIw
MC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVz
IjogeyJjb3VudCI6IDEzNTM4NDIzLCJwZXJjZW50IjogIjAwLjA0In0sImRhdGEiOiAiMjIxLjE5
NC4xNzEuMjI4In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQi
OiA0OTIzNDYzLCJwZXJjZW50IjogIjAwLjAyIn0sImRhdGEiOiAiODcuMTYxLjMwLjIyMyJ9LHsi
aGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBl
cmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIyNC44LjEwOC4zMCJ9LHsiaGl0cyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTYi
fSwiZGF0YSI6ICIxODUuMTczLjIwMS4xODMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2Vu
dCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0s
ImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAi
NjYuMTk4LjIyMi4yMDMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDEwMzkyMDAzMywicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjg2LjI5Ljk4Ljci
fSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDEwMzkyNDc4
OCwicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjIwNy40Ni4xMy4xMDIifSx7ImhpdHMiOiB7
ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJwZXJjZW50Ijog
IjAwLjE2In0sImRhdGEiOiAiMTU2LjU3LjE2My40MSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJw
ZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTk2NzI1MzQsInBlcmNlbnQiOiAiMDAuMDYifSwiZGF0
YSI6ICIxODUuNTIuMTQxLjIyIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6
IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjIxMi4yNi43
OC42In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2
MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjIwMi4xNzguMTI0LjEyNiJ9LHsiaGl0
cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNl
bnQiOiAiMDAuMTYifSwiZGF0YSI6ICIxMDEuMTg1LjE3MS4xMTEifSx7ImhpdHMiOiB7ImNvdW50
IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDY1NzUsInBlcmNlbnQiOiAiMDAuMDAifSwi
ZGF0YSI6ICIxODEuMjI0LjExMS41MCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50Ijog
IjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0
ZXMiOiB7ImNvdW50IjogMjkzNzEyMTQsInBlcmNlbnQiOiAiMDAuMDkifSwiZGF0YSI6ICI5MC4x
MzkuMTAxLjMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJjb3VudCI6
IDIxNTIyODIsInBlcmNlbnQiOiAiMDAuMDEifSwiZGF0YSI6ICIxOTYuMTIuMjE4LjE1MCJ9LHsi
aGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTcwNTMyMDgsInBl
cmNlbnQiOiAiMDAuMTgifSwiZGF0YSI6ICI1OC44NC42MC4yMjIifSx7ImhpdHMiOiB7ImNvdW50
IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDQzNTM5ODI3LCJwZXJjZW50IjogIjAwLjE0
In0sImRhdGEiOiAiMzEuNTMuMTI5LjExNiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50
IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI4
OS4yNDMuNjAuNzMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3Vu
dCI6IDUxOTY4Mjk5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTg1LjE4Ny45OS41MCJ9
LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5ODkwNzAs
InBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIxMzYuMjI2LjE5LjE5MyJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5ODQ4NjEsInBlcmNlbnQiOiAi
MDAuMTYifSwiZGF0YSI6ICI5NS4xNi42Mi4xNCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogMTY5MDkyNTQsInBlcmNlbnQiOiAiMDAuMDUifSwiZGF0YSI6
ICIxNjUuOTguMTM3LjY2In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsi
Y291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjE3Ni4xMTQuMTI4
LjEwNiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5
NjMwMTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI3My4xMy4yMjQuMTA3In0seyJoaXRz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2Vu
dCI6ICIwMC4xNiJ9LCJkYXRhIjogIjE5NC42MC4yNDIuNTgifSx7ImhpdHMiOiB7ImNvdW50Ijog
MiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjE2In0s
ImRhdGEiOiAiMTg1LjE5MC4yNDEuMjEwIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQi
OiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJi
eXRlcyI6IHsiY291bnQiOiAzMDYzNzA2MCwicGVyY2VudCI6ICIwMC4xMCJ9LCJkYXRhIjogIjE5
NC4zOS40NC4yMjgifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3Vu
dCI6IDE3NDEwNzEsInBlcmNlbnQiOiAiMDAuMDEifSwiZGF0YSI6ICIxMTQuNzkuNDYuMjMzIn0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMzk5Mzk1LCJw
ZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiMTE1LjE3OC4yMzguMTM2In0seyJoaXRzIjogeyJj
b3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MjQ3ODgsInBlcmNlbnQiOiAi
MDAuMzIifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjIxNiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJw
ZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0
YSI6ICIxNjIuMjIxLjIzMC4zNCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMi
OiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI5OC4xMTMu
MjAwLjI0MyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50Ijog
MTE3MTIyMTAsInBlcmNlbnQiOiAiMDAuMDQifSwiZGF0YSI6ICI2Ny4yMjMuMy4yMyJ9LHsiaGl0
cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNl
bnQiOiAiMDAuMTYifSwiZGF0YSI6ICI4Ny4yMC4xMjMuNDMifSx7ImhpdHMiOiB7ImNvdW50Ijog
MiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJwZXJjZW50IjogIjAwLjE2In0s
ImRhdGEiOiAiNzMuMjA3LjgwLjExMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50Ijog
IjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0
ZXMiOiB7ImNvdW50IjogMjcwMzA0NiwicGVyY2VudCI6ICIwMC4wMSJ9LCJkYXRhIjogIjE3Ny4y
MjguOTcuMjAwIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQi
OiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjIxNi4xOTcuMTU3LjU1In0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwi
cGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjQ1LjgwLjEzOC4yMjAifSx7ImhpdHMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50IjogIjAw
LjE2In0sImRhdGEiOiAiODAuMjkuMTI2LjIzOSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNzQxNTU2OCwicGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjog
IjEzNi4yMzIuMS4xNjYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJj
b3VudCI6IDEwMzkyNDY5MywicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjIxNy4xNDAuMTA2
LjQ5In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2
MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjQ1LjEzNC4yMTMuMTk3In0seyJoaXRz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2Vu
dCI6ICIwMC4xNiJ9LCJkYXRhIjogIjExMS4yMjMuMTg0LjE2MiJ9LHsiaGl0cyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTYi
fSwiZGF0YSI6ICI0MS4xMTEuMTQ0LjIyMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50
IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNDMzNDg2NiwicGVyY2VudCI6ICIwMC4wMSJ9LCJkYXRhIjogIjc2
LjM0LjExOS4xMzUifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3Vu
dCI6IDE4MjcyNDc4LCJwZXJjZW50IjogIjAwLjA2In0sImRhdGEiOiAiNDkuMjI4LjEwMS4xMTMi
fSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5
LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMzcuNi4xMjcuNTAifSx7ImhpdHMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDkyNDE2ODIsInBlcmNlbnQiOiAiMDAu
MDMifSwiZGF0YSI6ICIzNy43Ni4xNi4xODkifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2Vu
dCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0s
ImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAi
MTkxLjYuMTIuMzYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3Vu
dCI6IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTk5LjI3LjI1My4yMjMi
fSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0
LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiODguMTYwLjgwLjQ3In0seyJoaXRzIjogeyJj
b3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMTg2MCwicGVyY2VudCI6ICIwMC4w
MCJ9LCJkYXRhIjogIjE4OS4yMzguODkuMjUxIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9
LCJieXRlcyI6IHsiY291bnQiOiA3MDMyODkwLCJwZXJjZW50IjogIjAwLjAyIn0sImRhdGEiOiAi
MTI4LjkwLjQzLjI3In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291
bnQiOiAxMDM5MjQ3ODgsInBlcmNlbnQiOiAiMDAuMzIifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjIz
OCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1
OTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIyLjExNy4zMy40MiJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMjQ5ODc3OTIsInBlcmNlbnQiOiAi
MDAuMDgifSwiZGF0YSI6ICI4My4zNi4xOTYuMTUifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0
In0sImJ5dGVzIjogeyJjb3VudCI6IDEwMzkyNDczOCwicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRh
IjogIjE5NC4xODcuODcuMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEz
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIyLjIyOC4xMzEu
MTE3In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2
MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjQ1LjExLjIwMi44In0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6
ICIwMC4xNiJ9LCJkYXRhIjogIjkxLjE3My4xMDMuMTk5In0seyJoaXRzIjogeyJjb3VudCI6IDIs
InBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJk
YXRhIjogIjEwMy4xMjEuNjguMTYxIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRl
cyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjEwNC4y
OC4xMzUuODAifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6
IDM5Mjg2MjUwLCJwZXJjZW50IjogIjAwLjEyIn0sImRhdGEiOiAiODUuMTc0LjE5Mi4xNzEifSx7
ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJw
ZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMjA4LjE4Ni4yNTUuNjAifSx7ImhpdHMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDQ0Mzk1ODksInBlcmNlbnQiOiAiMDAu
MDEifSwiZGF0YSI6ICIxNTQuMTYuMTkyLjE4NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6
ICIxMzcuMTAxLjIzNS45NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEz
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI3My4xMjkuMTgu
MTczIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2
MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjc1LjExNS43MS4yIn0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MjQ3ODgsInBlcmNlbnQi
OiAiMDAuMzIifSwiZGF0YSI6ICI0MC43Ny4xNjcuMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJw
ZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMjE2NTU5MTgsInBlcmNlbnQiOiAiMDAuMDcifSwiZGF0
YSI6ICI2Mi4xODEuNTEuNDQifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4x
MyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjog
eyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiOTMuMzQuOTIu
MTU1In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM2
NDE0OTgsInBlcmNlbnQiOiAiMDAuMzIifSwiZGF0YSI6ICI3OC44MC44MC4xOTcifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50
IjogIjAwLjE2In0sImRhdGEiOiAiMTIxLjc1LjExMC44MyJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYifSwi
ZGF0YSI6ICIxOTIuNDIuMTE2LjIwMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50Ijog
IjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0
ZXMiOiB7ImNvdW50IjogNTk1NTIyNzcsInBlcmNlbnQiOiAiMDAuMTkifSwiZGF0YSI6ICI3NS44
Mi4xOTMuNyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50Ijog
MTE4NjAsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICIxNzkuNDguMjQ4LjIyIn0seyJoaXRz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2Vu
dCI6ICIwMC4xNiJ9LCJkYXRhIjogIjg5LjAuMjA4LjE1OCJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTAzOTI0Nzg4LCJwZXJjZW50IjogIjAwLjMyIn0s
ImRhdGEiOiAiNTIuMTY3LjE0NC4xNzYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDE3MjMyOCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjE5Ny4y
MzEuMjM5LjE4In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQi
OiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjIuMjEwLjUyLjIxNSJ9LHsi
aGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMjE1OTM3MzgsInBl
cmNlbnQiOiAiMDAuMDcifSwiZGF0YSI6ICIyMTMuMTM0LjE4Mi4yNTMifSx7ImhpdHMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUyNDI2NDg1LCJwZXJjZW50IjogIjAw
LjE2In0sImRhdGEiOiAiODQuMTkuODYuOTEifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2Vu
dCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0s
ImJ5dGVzIjogeyJjb3VudCI6IDExNjgxMTU0LCJwZXJjZW50IjogIjAwLjA0In0sImRhdGEiOiAi
OTEuMTY5LjE3Ni4xMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNv
dW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI4Ni40MS41Ni4xODEi
fSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDE4NzM2NDIs
InBlcmNlbnQiOiAiMDAuMDEifSwiZGF0YSI6ICIyMTIuNDcuMTM1LjI1MCJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAi
MDAuMTYifSwiZGF0YSI6ICI3OS4xNTUuNTIuNzgifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEy
In0sImJ5dGVzIjogeyJjb3VudCI6IDEyMzUxNjkwLCJwZXJjZW50IjogIjAwLjA0In0sImRhdGEi
OiAiMTY1LjczLjEyMS4yMTgifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4x
MyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjog
eyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiOTQuMjM3LjQz
LjY3In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNjgy
MDA5MCwicGVyY2VudCI6ICIwMC4wNSJ9LCJkYXRhIjogIjc4LjE4Ni4xODQuMTE3In0seyJoaXRz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MjQ3ODgsInBlcmNl
bnQiOiAiMDAuMzIifSwiZGF0YSI6ICI0MC43Ny4xNjcuNzcifSx7ImhpdHMiOiB7ImNvdW50Ijog
MiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjE2In0s
ImRhdGEiOiAiODUuMjE2LjE5LjU3In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRl
cyI6IHsiY291bnQiOiA4NTg5Mjg4LCJwZXJjZW50IjogIjAwLjAzIn0sImRhdGEiOiAiMjEzLjE5
Mi4yMTQuODAifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6
IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTA4LjE2LjUzLjE4MiJ9LHsi
aGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMzgwOTE4MDUsInBl
cmNlbnQiOiAiMDAuMTIifSwiZGF0YSI6ICI3NC4yMTkuMjMwLjIyNiJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAu
MTYifSwiZGF0YSI6ICI0NS4xNzcuNzguMjA4In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9
LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjog
IjE0Ni43MC4xNzkuMjgifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDEwMzkyNDc4OCwicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjQwLjc3LjE2Ny4y
OCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50IjogOTQ4NzA5
MCwicGVyY2VudCI6ICIwMC4wMyJ9LCJkYXRhIjogIjc1LjIxMi4xMzIuMTAwIn0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA2MTMxNTQ5LCJwZXJjZW50Ijog
IjAwLjAyIn0sImRhdGEiOiAiMTgxLjkuMTU1LjYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEy
In0sImJ5dGVzIjogeyJjb3VudCI6IDc1MTY1NzEsInBlcmNlbnQiOiAiMDAuMDIifSwiZGF0YSI6
ICIxMTkuOC4xNzcuMTA4In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsi
Y291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjEuMjM5LjE4NS4x
MzQifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4
Mjk5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiODkuMjQ3LjE3MC4xMTgifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50
IjogIjAwLjE2In0sImRhdGEiOiAiMjMuMjI2Ljg2LjE2NyJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTIyOTA5NTUsInBlcmNlbnQiOiAiMDAuMTYifSwi
ZGF0YSI6ICIxNTguMTQwLjE3Ni4xNzIifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTk0
LjM5LjIxOC4xNiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50
IjogODA0NzI1MywicGVyY2VudCI6ICIwMC4wMyJ9LCJkYXRhIjogIjgwLjEyMC4xOTYuOTcifSx7
ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJw
ZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiNDEuOTAuMTE2LjE0In0seyJoaXRzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwicGVyY2VudCI6ICIwMC4x
NiJ9LCJkYXRhIjogIjkxLjE2MC45LjgzIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1OTMwLCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiMzcuMTU3
LjIzMS4xMzEifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6
IDUxOTYyMzcxLCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiODUuMTYzLjIzMy4yNTAifSx7
ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYyMzY5LCJw
ZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiOTIuMzMuMjM2LjQ5In0seyJoaXRzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwicGVyY2VudCI6ICIwMC4x
NiJ9LCJkYXRhIjogIjIwOS4xMjIuMTM3LjExNSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjIzNjksInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6
ICI2NC4xMjQuMTczLjE5MCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogNTk1NSwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjIwNy40Ni4xMy4xMTYi
fSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYyMzY5
LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMTg1LjEwNy40NC45NyJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjIzNjksInBlcmNlbnQiOiAi
MDAuMTYifSwiZGF0YSI6ICI4NS4xNDUuMTkxLjg4In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4x
MiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRh
IjogIjIxNy4xMC4yMzAuNSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogNTkzMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjM4LjM0LjY5LjEwMSJ9
LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjIzNjks
InBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIyMTkuNzguMjE5LjE0NSJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjIzNjksInBlcmNlbnQiOiAi
MDAuMTYifSwiZGF0YSI6ICIxMTEuMTA4LjExMS4xMzQifSx7ImhpdHMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYyMzY5LCJwZXJjZW50IjogIjAwLjE2In0sImRh
dGEiOiAiMTg2LjExOS45Ny4yNTUifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVz
IjogeyJjb3VudCI6IDIwMDg0MDQsInBlcmNlbnQiOiAiMDAuMDEifSwiZGF0YSI6ICIyMDguNzQu
MjMyLjE1NyJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50Ijog
NTE5NjIzNjksInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIxNzcuNTUuMjI1Ljg2In0seyJo
aXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwicGVy
Y2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjE3Mi41Ni4xNjguMTI3In0seyJoaXRzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA2NTA1MjA0LCJwZXJjZW50IjogIjAwLjAy
In0sImRhdGEiOiAiOTYuMjQzLjE3LjE4In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJi
eXRlcyI6IHsiY291bnQiOiAxNDYwNDAyMCwicGVyY2VudCI6ICIwMC4wNSJ9LCJkYXRhIjogIjE3
Mi4yMjYuMTUuNjAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3Vu
dCI6IDUxOTYyMzY5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiNzAuMTMwLjEyMi40MCJ9
LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjIzNjks
InBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIxMjkuNDEuNDYuNyJ9LHsiaGl0cyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTkzMCwicGVyY2VudCI6ICIwMC4wMCJ9
LCJkYXRhIjogIjE2OC4xNjcuMjQ3LjEyIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjU4
Ljg0LjYxLjI4In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQi
OiA1MTk2MjM2OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjc2Ljk3LjIzOC4xNzkifSx7
ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYyMzY5LCJw
ZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiMS4xNDUuMjUzLjIwMyJ9LHsiaGl0cyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTkzMCwicGVyY2VudCI6ICIwMC4wMCJ9
LCJkYXRhIjogIjIyMC4yNDUuMjExLjI0NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNTE5NjIzNjksInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIy
MTMuMTIyLjE2My4xNDYifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDU5MzAsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICIxODUuMjAzLjE2LjY2In0s
eyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1ODQ2LCJwZXJj
ZW50IjogIjAwLjAwIn0sImRhdGEiOiAiMTg1LjE5MS4xNzEuMTEifSx7ImhpdHMiOiB7ImNvdW50
IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYyMzY5LCJwZXJjZW50IjogIjAwLjE2
In0sImRhdGEiOiAiOTkuMTMwLjI4LjEyOCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNTkzMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjc4LjIw
OS42Ni45MiJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50Ijog
NTkzMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjE4Ni4yMzIuMTA4LjYifSx7ImhpdHMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDE1NDM5MjA0LCJwZXJjZW50
IjogIjAwLjA1In0sImRhdGEiOiAiMTg1LjE4MS4xMzcuMTAwIn0seyJoaXRzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAyMzQ5NzkyNCwicGVyY2VudCI6ICIwMC4wNyJ9
LCJkYXRhIjogIjE2My4xNzIuODIuMTYzIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjgy
LjQ1Ljk2LjIzNiJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50
IjogNDI2OTEwNTMsInBlcmNlbnQiOiAiMDAuMTMifSwiZGF0YSI6ICI4MC4xNDYuMjQ1LjE2MiJ9
LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjIzNjks
InBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIxOTAuMTAxLjE4Ni41NSJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogOTk3NDE2NCwicGVyY2VudCI6ICIw
MC4wMyJ9LCJkYXRhIjogIjEyMi4xNzYuNDYuMTkyIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4x
MiJ9LCJieXRlcyI6IHsiY291bnQiOiA1OTMwLCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAi
MTU3LjE0My4xNzYuNjAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDE3NjE4NzI0LCJwZXJjZW50IjogIjAwLjA1In0sImRhdGEiOiAiMTQ1LjQwLjEyOC4x
ODkifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU5MzAs
InBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICIxNzcuMTQ5LjE0MS4yMDIifSx7ImhpdHMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDEyNTY4NCwicGVyY2VudCI6ICIw
MC4wMCJ9LCJkYXRhIjogIjEwOS4xMjYuMTY4LjI0MSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjIzNjksInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0
YSI6ICI3Ny4xNzQuMTczLjM4In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6
IHsiY291bnQiOiA1MTgwMTcxNSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjg1LjU4LjIz
MS41NyJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTk1
NSwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjIwNy40Ni4xMy4xNCJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjIzNjksInBlcmNlbnQiOiAi
MDAuMTYifSwiZGF0YSI6ICI3NS4xNTEuNTkuMTI1In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4x
MiJ9LCJieXRlcyI6IHsiY291bnQiOiAzMjk2ODIyOSwicGVyY2VudCI6ICIwMC4xMCJ9LCJkYXRh
IjogIjE4OC4yNS4xNjQuOTIifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4w
NyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjog
eyJjb3VudCI6IDUxOTYyMzY5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiNTEuMTkwLjE0
LjIwMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5
NjIzNzEsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIxMzYuMjI2Ljg3LjE0In0seyJoaXRz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwicGVyY2Vu
dCI6ICIwMC4xNiJ9LCJkYXRhIjogIjIwNS4yMTUuMjIyLjE0NiJ9LHsiaGl0cyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTkzMiwicGVyY2VudCI6ICIwMC4wMCJ9LCJk
YXRhIjogIjE2NS4yMjUuMTI0LjIzMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0
ZXMiOiB7ImNvdW50IjogNTkzMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjEwNC4yOC4x
MzAuNSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5
NjIzNjksInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI4LjQ2LjIwNS4yMDQifSx7ImhpdHMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU5NTUsInBlcmNlbnQiOiAi
MDAuMDAifSwiZGF0YSI6ICIyMDcuNDYuMTMuMTcifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEy
In0sImJ5dGVzIjogeyJjb3VudCI6IDIwODYxNDM0LCJwZXJjZW50IjogIjAwLjA3In0sImRhdGEi
OiAiMTkyLjk1LjY2LjQifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDU5MzAsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICI4OC4yMzYuMTg5LjI5In0s
eyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwi
cGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjEyOS4xODYuMTkyLjI1MCJ9XX0sIm9zIjogeyJt
ZXRhZGF0YSI6IHsiYnl0ZXMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDMyMDc2MDkzNTAyfSwiYXZn
IjogeyJ2YWx1ZSI6IDEwMzQ3MTI3MDQsInBlcmNlbnQiOiAiMDMuMjMifSwibWF4IjogeyJ2YWx1
ZSI6IDE5NTg3NTY4OTc4LCJwZXJjZW50IjogIjYxLjA3In0sIm1pbiI6IHsidmFsdWUiOiA1MTk4
LCJwZXJjZW50IjogIjAwLjAwIn19LCJ2aXNpdG9ycyI6IHsidG90YWwiOiB7InZhbHVlIjogODM0
fSwiYXZnIjogeyJ2YWx1ZSI6IDI2LCJwZXJjZW50IjogIjAzLjEyIn0sIm1heCI6IHsidmFsdWUi
OiA0OTgsInBlcmNlbnQiOiAiNTkuNzEifSwibWluIjogeyJ2YWx1ZSI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifX0sImhpdHMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDE1MzN9LCJhdmciOiB7InZhbHVl
IjogNDksInBlcmNlbnQiOiAiMDMuMjAifSwibWF4IjogeyJ2YWx1ZSI6IDEwMDEsInBlcmNlbnQi
OiAiNjUuMzAifSwibWluIjogeyJ2YWx1ZSI6IDEsInBlcmNlbnQiOiAiMDAuMDcifX0sImRhdGEi
OiB7InRvdGFsIjogewkidmFsdWUiOiAzMX19fSwiZGF0YSI6IFt7ImhpdHMiOiB7ImNvdW50Ijog
MTAwNSwicGVyY2VudCI6ICI2NS41NiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA1MDIsInBlcmNl
bnQiOiAiNjAuMTkifSwiYnl0ZXMiOiB7ImNvdW50IjogMTk2OTM2NDU5OTgsInBlcmNlbnQiOiAi
NjEuNDAifSwiZGF0YSI6ICJXaW5kb3dzIiwiaXRlbXMiOiBbeyJoaXRzIjogeyJjb3VudCI6IDEw
MDEsInBlcmNlbnQiOiAiNjUuMzAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNDk4LCJwZXJjZW50
IjogIjU5LjcxIn0sImJ5dGVzIjogeyJjb3VudCI6IDE5NTg3NTY4OTc4LCJwZXJjZW50IjogIjYx
LjA3In0sImRhdGEiOiAiV2luZG93cyAxMCJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50
IjogIjAwLjI2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuNDgifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMTA2MDc3MDIwLCJwZXJjZW50IjogIjAwLjMzIn0sImRhdGEiOiAi
V2luZG93cyA3In1dfSx7ImhpdHMiOiB7ImNvdW50IjogMTc1LCJwZXJjZW50IjogIjExLjQyIn0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDEzMiwicGVyY2VudCI6ICIxNS44MyJ9LCJieXRlcyI6IHsi
Y291bnQiOiA1Mjc4MjUzMzMxLCJwZXJjZW50IjogIjE2LjQ2In0sImRhdGEiOiAiTGludXgiLCJp
dGVtcyI6IFt7ImhpdHMiOiB7ImNvdW50IjogMTM5LCJwZXJjZW50IjogIjA5LjA3In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEwNCwicGVyY2VudCI6ICIxMi40NyJ9LCJieXRlcyI6IHsiY291bnQi
OiA0MDI2MjA4Nzc5LCJwZXJjZW50IjogIjEyLjU1In0sImRhdGEiOiAiTGludXgifSx7ImhpdHMi
OiB7ImNvdW50IjogMzAsInBlcmNlbnQiOiAiMDEuOTYifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MjIsInBlcmNlbnQiOiAiMDIuNjQifSwiYnl0ZXMiOiB7ImNvdW50IjogOTQwMjQ5MDk5LCJwZXJj
ZW50IjogIjAyLjkzIn0sImRhdGEiOiAiVWJ1bnR1In0seyJoaXRzIjogeyJjb3VudCI6IDYsInBl
cmNlbnQiOiAiMDAuMzkifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNiwicGVyY2VudCI6ICIwMC43
MiJ9LCJieXRlcyI6IHsiY291bnQiOiAzMTE3OTU0NTMsInBlcmNlbnQiOiAiMDAuOTcifSwiZGF0
YSI6ICJsaW51eC1nbnUifV19LHsiaGl0cyI6IHsiY291bnQiOiAxNjIsInBlcmNlbnQiOiAiMTAu
NTcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNjEsInBlcmNlbnQiOiAiMDcuMzEifSwiYnl0ZXMi
OiB7ImNvdW50IjogMTQyNjE3MTM4NCwicGVyY2VudCI6ICIwNC40NSJ9LCJkYXRhIjogIkFuZHJv
aWQiLCJpdGVtcyI6IFt7ImhpdHMiOiB7ImNvdW50IjogOTEsInBlcmNlbnQiOiAiMDUuOTQifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMzQsInBlcmNlbnQiOiAiMDQuMDgifSwiYnl0ZXMiOiB7ImNv
dW50IjogMTA2Njk1MjgwNCwicGVyY2VudCI6ICIwMy4zMyJ9LCJkYXRhIjogIkFuZHJvaWQgMTAi
fSx7ImhpdHMiOiB7ImNvdW50IjogMjMsInBlcmNlbnQiOiAiMDEuNTAifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMTAsInBlcmNlbnQiOiAiMDEuMjAifSwiYnl0ZXMiOiB7ImNvdW50IjogMTY2Mzg4
MjE3LCJwZXJjZW50IjogIjAwLjUyIn0sImRhdGEiOiAiQW5kcm9pZCAxMyJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAxNCwicGVyY2VudCI6ICIwMC45MSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA1LCJw
ZXJjZW50IjogIjAwLjYwIn0sImJ5dGVzIjogeyJjb3VudCI6IDEyMTYyMTM3NSwicGVyY2VudCI6
ICIwMC4zOCJ9LCJkYXRhIjogIkFuZHJvaWQgMTEifSx7ImhpdHMiOiB7ImNvdW50IjogMTEsInBl
cmNlbnQiOiAiMDAuNzIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4z
NiJ9LCJieXRlcyI6IHsiY291bnQiOiAzMDgxNDA1LCJwZXJjZW50IjogIjAwLjAxIn0sImRhdGEi
OiAiTWFyc2htYWxsb3cgNi4wIn0seyJoaXRzIjogeyJjb3VudCI6IDEwLCJwZXJjZW50IjogIjAw
LjY1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMi
OiB7ImNvdW50IjogMTQ4MTAyNDEsInBlcmNlbnQiOiAiMDAuMDUifSwiZGF0YSI6ICJMb2xsaXBv
cCA1LjEifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4yNiJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJjb3VudCI6IDMx
NDA0NjYsInBlcmNlbnQiOiAiMDAuMDEifSwiZGF0YSI6ICJOb3VnYXQgNy4xIn0seyJoaXRzIjog
eyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzAyNDQwNCwicGVyY2VudCI6
ICIwMC4wNSJ9LCJkYXRhIjogIk1hcnNobWFsbG93IDYuMC4xIn0seyJoaXRzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4yNCJ9LCJieXRlcyI6IHsiY291bnQiOiAzODQ2MTQsInBlcmNlbnQiOiAiMDAuMDAifSwi
ZGF0YSI6ICJPcmVvIDguMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEz
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTE1NzIsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICJOb3VnYXQgNy4wIn0s
eyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAzMjc1MDM1Niwi
cGVyY2VudCI6ICIwMC4xMCJ9LCJkYXRhIjogIkFuZHJvaWQgMTQifSx7ImhpdHMiOiB7ImNvdW50
IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU5MzAsInBlcmNlbnQiOiAiMDAuMDAifSwi
ZGF0YSI6ICJBbmRyb2lkIEFQSSAzNCJ9XX0seyJoaXRzIjogeyJjb3VudCI6IDk0LCJwZXJjZW50
IjogIjA2LjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDY4LCJwZXJjZW50IjogIjA4LjE1In0s
ImJ5dGVzIjogeyJjb3VudCI6IDI2MDM4MTQzMTYsInBlcmNlbnQiOiAiMDguMTIifSwiZGF0YSI6
ICJtYWNPUyIsIml0ZW1zIjogW3siaGl0cyI6IHsiY291bnQiOiA4OCwicGVyY2VudCI6ICIwNS43
NCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA2NSwicGVyY2VudCI6ICIwNy43OSJ9LCJieXRlcyI6
IHsiY291bnQiOiAyNDQ3OTE0ODU2LCJwZXJjZW50IjogIjA3LjYzIn0sImRhdGEiOiAibWFjT1Mg
MTAuMTUgQ2F0YWxpbmEifSx7ImhpdHMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4zMyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJj
b3VudCI6IDE1NTg5NDI2MiwicGVyY2VudCI6ICIwMC40OSJ9LCJkYXRhIjogIm1hY09TIDEwLjEz
IEhpZ2ggU2llcnJhIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291
bnQiOiA1MTk4LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiT1MgWCAxMC4xMCBZb3NlbWl0
ZSJ9XX0seyJoaXRzIjogeyJjb3VudCI6IDY3LCJwZXJjZW50IjogIjA0LjM3In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDQ4LCJwZXJjZW50IjogIjA1Ljc2In0sImJ5dGVzIjogeyJjb3VudCI6IDI0
MTUzNDkyNDAsInBlcmNlbnQiOiAiMDcuNTMifSwiZGF0YSI6ICJDcmF3bGVycyIsIml0ZW1zIjog
W3siaGl0cyI6IHsiY291bnQiOiA2MSwicGVyY2VudCI6ICIwMy45OCJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiA0MywicGVyY2VudCI6ICIwNS4xNiJ9LCJieXRlcyI6IHsiY291bnQiOiAyNDE0OTUz
NzAyLCJwZXJjZW50IjogIjA3LjUzIn0sImRhdGEiOiAiYmluZ2JvdFwvMi4wIn0seyJoaXRzIjog
eyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywi
cGVyY2VudCI6ICIwMC4zNiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzQ4MSwicGVyY2VudCI6ICIw
MC4wMCJ9LCJkYXRhIjogIlNlbWFudGljU2Nob2xhckJvdCJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTE1NzIsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0
YSI6ICJTZW1ydXNoQm90XC83fmJsIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRl
cyI6IHsiY291bnQiOiAzNjY0ODUsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICJBaHJlZnNC
b3RcLzcuMCJ9XX0seyJoaXRzIjogeyJjb3VudCI6IDEyLCJwZXJjZW50IjogIjAwLjc4In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDksInBlcmNlbnQiOiAiMDEuMDgifSwiYnl0ZXMiOiB7ImNvdW50
IjogNDMyNTMyODMzLCJwZXJjZW50IjogIjAxLjM1In0sImRhdGEiOiAiVW5rbm93biIsIml0ZW1z
IjogW3siaGl0cyI6IHsiY291bnQiOiAxMiwicGVyY2VudCI6ICIwMC43OCJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiA5LCJwZXJjZW50IjogIjAxLjA4In0sImJ5dGVzIjogeyJjb3VudCI6IDQzMjUz
MjgzMywicGVyY2VudCI6ICIwMS4zNSJ9LCJkYXRhIjogIlVua25vd24ifV19LHsiaGl0cyI6IHsi
Y291bnQiOiAxMSwicGVyY2VudCI6ICIwMC43MiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxMCwi
cGVyY2VudCI6ICIwMS4yMCJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDA2MzAxMzYsInBlcmNlbnQi
OiAiMDAuMzEifSwiZGF0YSI6ICJpT1MiLCJpdGVtcyI6IFt7ImhpdHMiOiB7ImNvdW50IjogNCwi
cGVyY2VudCI6ICIwMC4yNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAw
LjM2In0sImJ5dGVzIjogeyJjb3VudCI6IDYzOTkxOTgyLCJwZXJjZW50IjogIjAwLjIwIn0sImRh
dGEiOiAiaVBob25lIE9TIDE3LjMuMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50Ijog
IjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMzYifSwiYnl0
ZXMiOiB7ImNvdW50IjogMTc2NTQ4MTIsInBlcmNlbnQiOiAiMDAuMDYifSwiZGF0YSI6ICJpUGhv
bmUgT1MgMTcuMi4xIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291
bnQiOiAxMTU0OTk2LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiaVBob25lIE9TIDE3LjAu
MSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMzIxODM5
NiwicGVyY2VudCI6ICIwMC4wMSJ9LCJkYXRhIjogImlQYWQgT1MgMTcuMy4xNy4zLjEifSx7Imhp
dHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU5MzAsInBlcmNlbnQi
OiAiMDAuMDAifSwiZGF0YSI6ICJpUGhvbmUgT1MgMTcuMS4xIn0seyJoaXRzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNDYwNDAyMCwicGVyY2VudCI6ICIwMC4wNSJ9
LCJkYXRhIjogImlQaG9uZSBPUyAxNy40In1dfSx7ImhpdHMiOiB7ImNvdW50IjogNywicGVyY2Vu
dCI6ICIwMC40NiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjQ4In0s
ImJ5dGVzIjogeyJjb3VudCI6IDEyNTY5NjI2NCwicGVyY2VudCI6ICIwMC4zOSJ9LCJkYXRhIjog
IkNocm9tZSBPUyIsIml0ZW1zIjogW3siaGl0cyI6IHsiY291bnQiOiA3LCJwZXJjZW50IjogIjAw
LjQ2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuNDgifSwiYnl0ZXMi
OiB7ImNvdW50IjogMTI1Njk2MjY0LCJwZXJjZW50IjogIjAwLjM5In0sImRhdGEiOiAiQ3JPUyJ9
XX1dfSwiYnJvd3NlcnMiOiB7Im1ldGFkYXRhIjogeyJieXRlcyI6IHsidG90YWwiOiB7InZhbHVl
IjogMzIwNzYwOTM1MDJ9LCJhdmciOiB7InZhbHVlIjogNDI3NjgxMjQ4LCJwZXJjZW50IjogIjAx
LjMzIn0sIm1heCI6IHsidmFsdWUiOiAxMTEwNDc2MjQ3NSwicGVyY2VudCI6ICIzNC42MiJ9LCJt
aW4iOiB7InZhbHVlIjogNTE5OCwicGVyY2VudCI6ICIwMC4wMCJ9fSwidmlzaXRvcnMiOiB7InRv
dGFsIjogeyJ2YWx1ZSI6IDgzM30sImF2ZyI6IHsidmFsdWUiOiAxMSwicGVyY2VudCI6ICIwMS4z
MiJ9LCJtYXgiOiB7InZhbHVlIjogMjc2LCJwZXJjZW50IjogIjMzLjEzIn0sIm1pbiI6IHsidmFs
dWUiOiAxLCJwZXJjZW50IjogIjAwLjEyIn19LCJoaXRzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiAx
NTMzfSwiYXZnIjogeyJ2YWx1ZSI6IDIwLCJwZXJjZW50IjogIjAxLjMwIn0sIm1heCI6IHsidmFs
dWUiOiA1OTQsInBlcmNlbnQiOiAiMzguNzUifSwibWluIjogeyJ2YWx1ZSI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifX0sImRhdGEiOiB7InRvdGFsIjogewkidmFsdWUiOiA3NX19fSwiZGF0YSI6IFt7
ImhpdHMiOiB7ImNvdW50IjogNzE2LCJwZXJjZW50IjogIjQ2LjcxIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDMzMywicGVyY2VudCI6ICIzOS45MyJ9LCJieXRlcyI6IHsiY291bnQiOiAxMzEyNTQ2
OTY0MywicGVyY2VudCI6ICI0MC45MiJ9LCJkYXRhIjogIkNocm9tZSIsIml0ZW1zIjogW3siaGl0
cyI6IHsiY291bnQiOiA1OTQsInBlcmNlbnQiOiAiMzguNzUifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMjc2LCJwZXJjZW50IjogIjMzLjA5In0sImJ5dGVzIjogeyJjb3VudCI6IDExMTA0NzYyNDc1
LCJwZXJjZW50IjogIjM0LjYyIn0sImRhdGEiOiAiQ2hyb21lXC8xMjEuMC4wLjAifSx7ImhpdHMi
OiB7ImNvdW50IjogNDAsInBlcmNlbnQiOiAiMDIuNjEifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MjMsInBlcmNlbnQiOiAiMDIuNzYifSwiYnl0ZXMiOiB7ImNvdW50IjogODg4NzY1OTE2LCJwZXJj
ZW50IjogIjAyLjc3In0sImRhdGEiOiAiQ2hyb21lXC8xMjAuMC4wLjAifSx7ImhpdHMiOiB7ImNv
dW50IjogMjksInBlcmNlbnQiOiAiMDEuODkifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1NDQyMjMxNiwicGVyY2VudCI6ICIw
MC4xNyJ9LCJkYXRhIjogIkNocm9tZVwvNjcuMC4zMzk2Ljk5In0seyJoaXRzIjogeyJjb3VudCI6
IDgsInBlcmNlbnQiOiAiMDAuNTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6
ICIwMC40OCJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzIwMDE3NzEsInBlcmNlbnQiOiAiMDAuNTQi
fSwiZGF0YSI6ICJDaHJvbWVcLzExOS4wLjAuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiA3LCJwZXJj
ZW50IjogIjAwLjQ2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMzYi
fSwiYnl0ZXMiOiB7ImNvdW50IjogMTI1Njk2MjY0LCJwZXJjZW50IjogIjAwLjM5In0sImRhdGEi
OiAiQ2hyb21lXC8xMjMuMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogNiwicGVyY2VudCI6ICIw
MC4zOSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjQ4In0sImJ5dGVz
IjogeyJjb3VudCI6IDE1Nzc2MTk3NCwicGVyY2VudCI6ICIwMC40OSJ9LCJkYXRhIjogIkNocm9t
ZVwvMTA5LjAuMC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMjYifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4yNCJ9LCJieXRlcyI6IHsiY291
bnQiOiAxMDM5MjY2MDgsInBlcmNlbnQiOiAiMDAuMzIifSwiZGF0YSI6ICJDaHJvbWVcLzEyMi4w
LjAuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNDM4
NzE4NzUsInBlcmNlbnQiOiAiMDAuMTQifSwiZGF0YSI6ICJDaHJvbWVcLzExMC4wLjAuMCJ9LHsi
aGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTcwMjQ0MDQsInBl
cmNlbnQiOiAiMDAuMDUifSwiZGF0YSI6ICJDaHJvbWVcLzEwNi4wLjAuMCJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBl
cmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50IjogMTk2MTAxMywicGVyY2VudCI6ICIw
MC4wMSJ9LCJkYXRhIjogIkNocm9tZVwvMTIxLjAuNjE2Ny4xNjQifSx7ImhpdHMiOiB7ImNvdW50
IjogMywicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU3MDY1ODU4LCJwZXJjZW50IjogIjAwLjE4
In0sImRhdGEiOiAiQ2hyb21lXC8xMTUuMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0
In0sImJ5dGVzIjogeyJjb3VudCI6IDU3MDQ2MDE5LCJwZXJjZW50IjogIjAwLjE4In0sImRhdGEi
OiAiQ2hyb21lXC8xMTcuMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIw
MC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVz
IjogeyJjb3VudCI6IDE3MjMyOCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIkNocm9tZVwv
ODcuMC40MjgwLjE0MSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNv
dW50IjogMzg0NjE0LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiQ2hyb21lXC84Ny4wLjQy
ODAuODgifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUx
OTYyMzY5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiQ2hyb21lXC85MC4wLjQ0MzAuMjEy
In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjM2
OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIkNocm9tZVwvOTEuMC40NDcyLjEwNiJ9LHsi
aGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMjc3OTY1NjQsInBl
cmNlbnQiOiAiMDAuMDkifSwiZGF0YSI6ICJDaHJvbWVcLzExNi4wLjU4NDUuMTQwIn0seyJoaXRz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwicGVyY2Vu
dCI6ICIwMC4xNiJ9LCJkYXRhIjogIkhlYWRsZXNzQ2hyb21lXC8xMjAuMC4wLjAifSx7ImhpdHMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYyMzY5LCJwZXJjZW50
IjogIjAwLjE2In0sImRhdGEiOiAiQ2hyb21lXC8xMDguMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50
IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU5MzAsInBlcmNlbnQiOiAiMDAuMDAifSwi
ZGF0YSI6ICJDaHJvbWVcLzEyMS4wLjYxNjcuMTQ0In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4x
MiJ9LCJieXRlcyI6IHsiY291bnQiOiAxOTkzMTYsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6
ICJDaHJvbWVcLzkwLjAuNDQzMC4yMTAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDUxOTYyMzY5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiQ2hy
b21lXC8xMTIuMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDUxOTYyMzY5LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiQ2hyb21lXC8xMTMu
MC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDgz
MDE4NCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIkNocm9tZVwvMTE4LjAuMC4wIn1dfSx7
ImhpdHMiOiB7ImNvdW50IjogMzY0LCJwZXJjZW50IjogIjIzLjc0In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDIyMiwicGVyY2VudCI6ICIyNi42MiJ9LCJieXRlcyI6IHsiY291bnQiOiA4MTM2MjY5
MDIwLCJwZXJjZW50IjogIjI1LjM3In0sImRhdGEiOiAiRmlyZWZveCIsIml0ZW1zIjogW3siaGl0
cyI6IHsiY291bnQiOiAyODgsInBlcmNlbnQiOiAiMTguNzkifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMTY1LCJwZXJjZW50IjogIjE5Ljc4In0sImJ5dGVzIjogeyJjb3VudCI6IDYyNjM2MTg0OTIs
InBlcmNlbnQiOiAiMTkuNTMifSwiZGF0YSI6ICJGaXJlZm94XC8xMjIuMCJ9LHsiaGl0cyI6IHsi
Y291bnQiOiA0NSwicGVyY2VudCI6ICIwMi45NCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzNCwi
cGVyY2VudCI6ICIwNC4wOCJ9LCJieXRlcyI6IHsiY291bnQiOiAxMjE5NzA5NTc1LCJwZXJjZW50
IjogIjAzLjgwIn0sImRhdGEiOiAiRmlyZWZveFwvMTE1LjAifSx7ImhpdHMiOiB7ImNvdW50Ijog
MTAsInBlcmNlbnQiOiAiMDAuNjUifSwidmlzaXRvcnMiOiB7ImNvdW50IjogOCwicGVyY2VudCI6
ICIwMC45NiJ9LCJieXRlcyI6IHsiY291bnQiOiAxOTYwMjkwNDcsInBlcmNlbnQiOiAiMDAuNjEi
fSwiZGF0YSI6ICJGaXJlZm94XC8xMjEuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiA4LCJwZXJjZW50
IjogIjAwLjUyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuNjAifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMTMxODE2NDE4LCJwZXJjZW50IjogIjAwLjQxIn0sImRhdGEiOiAi
RmlyZWZveFwvMTIzLjAifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4yNiJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjM2In0sImJ5dGVzIjogeyJj
b3VudCI6IDEwMzkzMTg5MywicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIkZpcmVmb3hcLzEx
OS4wIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4yNCJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5
MjUzODMsInBlcmNlbnQiOiAiMDAuMzIifSwiZGF0YSI6ICJGaXJlZm94XC8xMTguMCJ9LHsiaGl0
cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNl
bnQiOiAiMDAuMTYifSwiZGF0YSI6ICJGaXJlZm94XC8xMDIuMCJ9LHsiaGl0cyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMzEzMjg0NzYsInBlcmNlbnQiOiAiMDAuMTAi
fSwiZGF0YSI6ICJGaXJlZm94XC8xMjQuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMzM5MzU1MDcsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICJG
aXJlZm94XC8xMDcuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNv
dW50IjogNTkzMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIkZpcmVmb3hcLzExNi4wIn1d
fSx7ImhpdHMiOiB7ImNvdW50IjogMjM2LCJwZXJjZW50IjogIjE1LjM5In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEzNywicGVyY2VudCI6ICIxNi40MyJ9LCJieXRlcyI6IHsiY291bnQiOiA1Mzkz
NTQzNDM1LCJwZXJjZW50IjogIjE2LjgxIn0sImRhdGEiOiAiRWRnZSIsIml0ZW1zIjogW3siaGl0
cyI6IHsiY291bnQiOiAyMTQsInBlcmNlbnQiOiAiMTMuOTYifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMTIyLCJwZXJjZW50IjogIjE0LjYzIn0sImJ5dGVzIjogeyJjb3VudCI6IDQ5OTM2ODkzOTAs
InBlcmNlbnQiOiAiMTUuNTcifSwiZGF0YSI6ICJFZGdcLzEyMS4wLjAuMCJ9LHsiaGl0cyI6IHsi
Y291bnQiOiA3LCJwZXJjZW50IjogIjAwLjQ2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDUsInBl
cmNlbnQiOiAiMDAuNjAifSwiYnl0ZXMiOiB7ImNvdW50IjogMjIwMjAxODExLCJwZXJjZW50Ijog
IjAwLjY5In0sImRhdGEiOiAiRWRnXC8xMjIuMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogNSwi
cGVyY2VudCI6ICIwMC4zMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjI0In0sImJ5dGVzIjogeyJjb3VudCI6IDI1NjA2MDQ1LCJwZXJjZW50IjogIjAwLjA4In0sImRh
dGEiOiAiRWRnQVwvMTIxLjAuMC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRl
cyI6IHsiY291bnQiOiAyODQ4NjIwMiwicGVyY2VudCI6ICIwMC4wOSJ9LCJkYXRhIjogIkVkZ1wv
OTIuMC45MDIuNjcifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3Vu
dCI6IDI0OTg3NzkyLCJwZXJjZW50IjogIjAwLjA4In0sImRhdGEiOiAiRWRnXC8xMjMuMC4wLjAi
fSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJjb3VudCI6IDYyNzc2OTcz
LCJwZXJjZW50IjogIjAwLjIwIn0sImRhdGEiOiAiRWRnXC8xMjAuMC4wLjAifSx7ImhpdHMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU5MzAsInBlcmNlbnQiOiAiMDAu
MDAifSwiZGF0YSI6ICJFZGdcLzk3LjAuMTA3Mi42OSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MTIifSwiYnl0ZXMiOiB7ImNvdW50IjogMzE5Mzk0NDQsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0
YSI6ICJFZGdcLzExMy4wLjE3NzQuNTAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDU1Njk0NDQsInBlcmNlbnQiOiAiMDAuMDIifSwiZGF0YSI6ICJFZGdc
LzEyMS4wLjIyNzcuMTA1In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDci
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsi
Y291bnQiOiAyODA0MDQsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICJFZGdcLzExMi4wLjE3
MjIuNTgifV19LHsiaGl0cyI6IHsiY291bnQiOiA2NywicGVyY2VudCI6ICIwNC4zNyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiA0OCwicGVyY2VudCI6ICIwNS43NiJ9LCJieXRlcyI6IHsiY291bnQi
OiAyNDE1MzQ5MjQwLCJwZXJjZW50IjogIjA3LjUzIn0sImRhdGEiOiAiQ3Jhd2xlcnMiLCJpdGVt
cyI6IFt7ImhpdHMiOiB7ImNvdW50IjogNjEsInBlcmNlbnQiOiAiMDMuOTgifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogNDMsInBlcmNlbnQiOiAiMDUuMTYifSwiYnl0ZXMiOiB7ImNvdW50IjogMjQx
NDk1MzcwMiwicGVyY2VudCI6ICIwNy41MyJ9LCJkYXRhIjogImJpbmdib3RcLzIuMCJ9LHsiaGl0
cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDMsInBlcmNlbnQiOiAiMDAuMzYifSwiYnl0ZXMiOiB7ImNvdW50IjogMTc0ODEsInBlcmNlbnQi
OiAiMDAuMDAifSwiZGF0YSI6ICJTZW1hbnRpY1NjaG9sYXJCb3QifSx7ImhpdHMiOiB7ImNvdW50
IjogMiwicGVyY2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDExNTcyLCJwZXJjZW50IjogIjAwLjAwIn0s
ImRhdGEiOiAiU2VtcnVzaEJvdFwvN35ibCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMzY2NDg1LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiQWhy
ZWZzQm90XC83LjAifV19LHsiaGl0cyI6IHsiY291bnQiOiA1NCwicGVyY2VudCI6ICIwMy41MiJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyOCwicGVyY2VudCI6ICIwMy4zNiJ9LCJieXRlcyI6IHsi
Y291bnQiOiA4NjY2MTE3OTgsInBlcmNlbnQiOiAiMDIuNzAifSwiZGF0YSI6ICJPcGVyYSIsIml0
ZW1zIjogW3siaGl0cyI6IHsiY291bnQiOiAyMSwicGVyY2VudCI6ICIwMS4zNyJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAxMywicGVyY2VudCI6ICIwMS41NiJ9LCJieXRlcyI6IHsiY291bnQiOiA1
NDAyMDExMzMsInBlcmNlbnQiOiAiMDEuNjgifSwiZGF0YSI6ICJPcGVyYVwvMTA2LjAuMC4wIn0s
eyJoaXRzIjogeyJjb3VudCI6IDEwLCJwZXJjZW50IjogIjAwLjY1In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50IjogMzA3NTQ3NSwi
cGVyY2VudCI6ICIwMC4wMSJ9LCJkYXRhIjogIk9wZXJhXC83Mi42LjM3NjcuNjk5MzYifSx7Imhp
dHMiOiB7ImNvdW50IjogOSwicGVyY2VudCI6ICIwMC41OSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjI0In0sImJ5dGVzIjogeyJjb3VudCI6IDcxOTM2OTcwLCJwZXJj
ZW50IjogIjAwLjIyIn0sImRhdGEiOiAiT3BlcmFcLzc5LjMuNDE5NS43NjY3NCJ9LHsiaGl0cyI6
IHsiY291bnQiOiA3LCJwZXJjZW50IjogIjAwLjQ2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDYs
InBlcmNlbnQiOiAiMDAuNzIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTk2MjgzNTI1LCJwZXJjZW50
IjogIjAwLjYxIn0sImRhdGEiOiAiT3BlcmFcLzEwNy4wLjAuMCJ9LHsiaGl0cyI6IHsiY291bnQi
OiA0LCJwZXJjZW50IjogIjAwLjI2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQi
OiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50IjogMzE0MDQ2NiwicGVyY2VudCI6ICIwMC4wMSJ9
LCJkYXRhIjogIk9wZXJhXC83OS42LjQxOTUuNzcxNjgifSx7ImhpdHMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYyMzY5LCJwZXJjZW50IjogIjAwLjE2In0sImRh
dGEiOiAiT3BlcmFcLzkxLjAuNDUxNi4yMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNTkzMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIk9wZXJh
XC8xMDYuMC4wLjAgKEVkaXRpb24gc3RkLTEpIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9
LCJieXRlcyI6IHsiY291bnQiOiA1OTMwLCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiT3Bl
cmFcLzEwNi4wLjAuMCAoRWRpdGlvbiBZeCkifV19LHsiaGl0cyI6IHsiY291bnQiOiA0NywicGVy
Y2VudCI6ICIwMy4wNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzNywicGVyY2VudCI6ICIwNC40
NCJ9LCJieXRlcyI6IHsiY291bnQiOiA5NDg5NDM3MTgsInBlcmNlbnQiOiAiMDIuOTYifSwiZGF0
YSI6ICJTYWZhcmkiLCJpdGVtcyI6IFt7ImhpdHMiOiB7ImNvdW50IjogMzMsInBlcmNlbnQiOiAi
MDIuMTUifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMjUsInBlcmNlbnQiOiAiMDMuMDAifSwiYnl0
ZXMiOiB7ImNvdW50IjogODQ4Mjk2ODEyLCJwZXJjZW50IjogIjAyLjY0In0sImRhdGEiOiAiU2Fm
YXJpXC82MDUuMS4xNSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxMSwicGVyY2VudCI6ICIwMC43MiJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxMCwicGVyY2VudCI6ICIwMS4yMCJ9LCJieXRlcyI6IHsi
Y291bnQiOiAxMDA2MzAxMzYsInBlcmNlbnQiOiAiMDAuMzEifSwiZGF0YSI6ICJTYWZhcmlcLzYw
NC4xIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMTU3
MiwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIlNhZmFyaVwvNTM3LjM2In0seyJoaXRzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk4LCJwZXJjZW50IjogIjAw
LjAwIn0sImRhdGEiOiAiU2FmYXJpXC82MDAuMS4yNSJ9XX0seyJoaXRzIjogeyJjb3VudCI6IDMw
LCJwZXJjZW50IjogIjAxLjk2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDE3LCJwZXJjZW50Ijog
IjAyLjA0In0sImJ5dGVzIjogeyJjb3VudCI6IDgxMTEwMjEyMSwicGVyY2VudCI6ICIwMi41MyJ9
LCJkYXRhIjogIk90aGVycyIsIml0ZW1zIjogW3siaGl0cyI6IHsiY291bnQiOiAxMCwicGVyY2Vu
dCI6ICIwMC42NSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0s
ImJ5dGVzIjogeyJjb3VudCI6IDE0ODEwMjQxLCJwZXJjZW50IjogIjAwLjA1In0sImRhdGEiOiAi
SFVBV0VJIn0seyJoaXRzIjogeyJjb3VudCI6IDcsInBlcmNlbnQiOiAiMDAuNDYifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC42MCJ9LCJieXRlcyI6IHsiY291bnQiOiAy
NTk4MTgzMzQsInBlcmNlbnQiOiAiMDAuODEifSwiZGF0YSI6ICJXZ2V0XC8xLjIxLjMifSx7Imhp
dHMiOiB7ImNvdW50IjogNiwicGVyY2VudCI6ICIwMC4zOSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiA2LCJwZXJjZW50IjogIjAwLjcyIn0sImJ5dGVzIjogeyJjb3VudCI6IDMxMTc5NTQ1MywicGVy
Y2VudCI6ICIwMC45NyJ9LCJkYXRhIjogIldnZXRcLzEuMjAuMyJ9LHsiaGl0cyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTYi
fSwiZGF0YSI6ICJTZWFNb25rZXlcLzIuNTMuMTEifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4xMyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEy
In0sImJ5dGVzIjogeyJjb3VudCI6IDU1NzgwMDY5LCJwZXJjZW50IjogIjAwLjE3In0sImRhdGEi
OiAiV2dldFwvMS4yMS40In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTMi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4yNCJ9LCJieXRlcyI6IHsi
Y291bnQiOiA2NDk3MjExNywicGVyY2VudCI6ICIwMC4yMCJ9LCJkYXRhIjogImN1cmxcLzguNC4w
In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjMx
MywicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogImN1cmxcLzguNi4wIn1dfSx7ImhpdHMiOiB7
ImNvdW50IjogMTcsInBlcmNlbnQiOiAiMDEuMTEifSwidmlzaXRvcnMiOiB7ImNvdW50IjogOSwi
cGVyY2VudCI6ICIwMS4wOCJ9LCJieXRlcyI6IHsiY291bnQiOiAzNzY2NTIyNDUsInBlcmNlbnQi
OiAiMDEuMTcifSwiZGF0YSI6ICJZYW5kZXguQnJvd3MiLCJpdGVtcyI6IFt7ImhpdHMiOiB7ImNv
dW50IjogNiwicGVyY2VudCI6ICIwMC4zOSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0LCJwZXJj
ZW50IjogIjAwLjQ4In0sImJ5dGVzIjogeyJjb3VudCI6IDE1MDkzNjIyOSwicGVyY2VudCI6ICIw
MC40NyJ9LCJkYXRhIjogIllhQnJvd3NlclwvMjQuMS4wLjAifSx7ImhpdHMiOiB7ImNvdW50Ijog
NCwicGVyY2VudCI6ICIwMC4yNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU1NDQzODUwLCJwZXJjZW50IjogIjAwLjE3In0s
ImRhdGEiOiAiWWFCcm93c2VyXC8yNC4xLjIuOTIuMDEifSx7ImhpdHMiOiB7ImNvdW50IjogMywi
cGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjEyIn0sImJ5dGVzIjogeyJjb3VudCI6IDU3MDY3MTUwLCJwZXJjZW50IjogIjAwLjE4In0sImRh
dGEiOiAiWWFCcm93c2VyXC8yMy4xMS4xLjcxNCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6
ICJZYUJyb3dzZXJcLzIzLjExLjAuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMTIifSwiYnl0
ZXMiOiB7ImNvdW50IjogOTI3NDM0OCwicGVyY2VudCI6ICIwMC4wMyJ9LCJkYXRhIjogIllhQnJv
d3NlclwvMjQuMS4yLjkyLjAwIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6
IHsiY291bnQiOiA1MTk2MjM2OSwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIllhQnJvd3Nl
clwvMjMuMTEuMC4yNDg5In1dfSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4w
NyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjEyIn0sImJ5dGVzIjog
eyJjb3VudCI6IDIxNDYzNTIsInBlcmNlbnQiOiAiMDAuMDEifSwiZGF0YSI6ICJNU0lFIiwiaXRl
bXMiOiBbeyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiAyMTQ2
MzUyLCJwZXJjZW50IjogIjAwLjAxIn0sImRhdGEiOiAiTVNJRVwvMTEuMCJ9XX0seyJoaXRzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJieXRlcyI6IHsiY291bnQiOiA1OTMwLCJwZXJjZW50IjogIjAw
LjAwIn0sImRhdGEiOiAiVW5rbm93biIsIml0ZW1zIjogW3siaGl0cyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MTIifSwiYnl0ZXMiOiB7ImNvdW50IjogNTkzMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjog
IlVua25vd24ifV19XX0sInZpc2l0X3RpbWUiOiB7Im1ldGFkYXRhIjogeyJieXRlcyI6IHsidG90
YWwiOiB7InZhbHVlIjogMzIwNzYwOTM1MDJ9LCJhdmciOiB7InZhbHVlIjogMTMzNjUwMzkzNiwi
cGVyY2VudCI6ICIwNC4xNyJ9LCJtYXgiOiB7InZhbHVlIjogMjM2NzkzMzA2MCwicGVyY2VudCI6
ICIwNy4zOCJ9LCJtaW4iOiB7InZhbHVlIjogNTgxOTQyNDA1LCJwZXJjZW50IjogIjAxLjgxIn19
LCJ2aXNpdG9ycyI6IHsidG90YWwiOiB7InZhbHVlIjogODQyfSwiYXZnIjogeyJ2YWx1ZSI6IDM1
LCJwZXJjZW50IjogIjA0LjE2In0sIm1heCI6IHsidmFsdWUiOiA1NCwicGVyY2VudCI6ICIwNi40
MSJ9LCJtaW4iOiB7InZhbHVlIjogMTUsInBlcmNlbnQiOiAiMDEuNzgifX0sImhpdHMiOiB7InRv
dGFsIjogeyJ2YWx1ZSI6IDE1MzN9LCJhdmciOiB7InZhbHVlIjogNjMsInBlcmNlbnQiOiAiMDQu
MTEifSwibWF4IjogeyJ2YWx1ZSI6IDE1MiwicGVyY2VudCI6ICIwOS45MiJ9LCJtaW4iOiB7InZh
bHVlIjogMjcsInBlcmNlbnQiOiAiMDEuNzYifX0sImRhdGEiOiB7InRvdGFsIjogewkidmFsdWUi
OiAyNH19fSwiZGF0YSI6IFt7ImhpdHMiOiB7ImNvdW50IjogMzgsInBlcmNlbnQiOiAiMDIuNDgi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMjcsInBlcmNlbnQiOiAiMDMuMjQifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTE4NDk5NTEwNSwicGVyY2VudCI6ICIwMy42OSJ9LCJkYXRhIjogIjAwIn0seyJo
aXRzIjogeyJjb3VudCI6IDI3LCJwZXJjZW50IjogIjAxLjc2In0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDIyLCJwZXJjZW50IjogIjAyLjY0In0sImJ5dGVzIjogeyJjb3VudCI6IDg1NzY2NjUzNCwi
cGVyY2VudCI6ICIwMi42NyJ9LCJkYXRhIjogIjAxIn0seyJoaXRzIjogeyJjb3VudCI6IDMwLCJw
ZXJjZW50IjogIjAxLjk2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIxLCJwZXJjZW50IjogIjAy
LjUyIn0sImJ5dGVzIjogeyJjb3VudCI6IDc1OTQ2NjA5NSwicGVyY2VudCI6ICIwMi4zNyJ9LCJk
YXRhIjogIjAyIn0seyJoaXRzIjogeyJjb3VudCI6IDQyLCJwZXJjZW50IjogIjAyLjc0In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDIwLCJwZXJjZW50IjogIjAyLjQwIn0sImJ5dGVzIjogeyJjb3Vu
dCI6IDk1NTAyNjg5MCwicGVyY2VudCI6ICIwMi45OCJ9LCJkYXRhIjogIjAzIn0seyJoaXRzIjog
eyJjb3VudCI6IDM2LCJwZXJjZW50IjogIjAyLjM1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDE1
LCJwZXJjZW50IjogIjAxLjgwIn0sImJ5dGVzIjogeyJjb3VudCI6IDU4MTk0MjQwNSwicGVyY2Vu
dCI6ICIwMS44MSJ9LCJkYXRhIjogIjA0In0seyJoaXRzIjogeyJjb3VudCI6IDQwLCJwZXJjZW50
IjogIjAyLjYxIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDI3LCJwZXJjZW50IjogIjAzLjI0In0s
ImJ5dGVzIjogeyJjb3VudCI6IDEwNzA4MzQ0NTksInBlcmNlbnQiOiAiMDMuMzQifSwiZGF0YSI6
ICIwNSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzNSwicGVyY2VudCI6ICIwMi4yOCJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAyMiwicGVyY2VudCI6ICIwMi42NCJ9LCJieXRlcyI6IHsiY291bnQiOiA4
NDU2NTQ3NDUsInBlcmNlbnQiOiAiMDIuNjQifSwiZGF0YSI6ICIwNiJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyOCwicGVyY2VudCI6ICIwMS44MyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyMSwicGVy
Y2VudCI6ICIwMi41MiJ9LCJieXRlcyI6IHsiY291bnQiOiA4MjMzODE5MjMsInBlcmNlbnQiOiAi
MDIuNTcifSwiZGF0YSI6ICIwNyJ9LHsiaGl0cyI6IHsiY291bnQiOiA3OCwicGVyY2VudCI6ICIw
NS4wOSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0MSwicGVyY2VudCI6ICIwNC45MiJ9LCJieXRl
cyI6IHsiY291bnQiOiAxMjU5NDA2NzM0LCJwZXJjZW50IjogIjAzLjkzIn0sImRhdGEiOiAiMDgi
fSx7ImhpdHMiOiB7ImNvdW50IjogMTQ0LCJwZXJjZW50IjogIjA5LjM5In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDUyLCJwZXJjZW50IjogIjA2LjI0In0sImJ5dGVzIjogeyJjb3VudCI6IDE5Mjc3
NDM4MTAsInBlcmNlbnQiOiAiMDYuMDEifSwiZGF0YSI6ICIwOSJ9LHsiaGl0cyI6IHsiY291bnQi
OiA3MSwicGVyY2VudCI6ICIwNC42MyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzNCwicGVyY2Vu
dCI6ICIwNC4wOCJ9LCJieXRlcyI6IHsiY291bnQiOiAxMzE4Njg4ODQ2LCJwZXJjZW50IjogIjA0
LjExIn0sImRhdGEiOiAiMTAifSx7ImhpdHMiOiB7ImNvdW50IjogMTUyLCJwZXJjZW50IjogIjA5
LjkyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDUyLCJwZXJjZW50IjogIjA2LjI0In0sImJ5dGVz
IjogeyJjb3VudCI6IDIzNjc5MzMwNjAsInBlcmNlbnQiOiAiMDcuMzgifSwiZGF0YSI6ICIxMSJ9
LHsiaGl0cyI6IHsiY291bnQiOiA3MSwicGVyY2VudCI6ICIwNC42MyJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiA0NiwicGVyY2VudCI6ICIwNS41MiJ9LCJieXRlcyI6IHsiY291bnQiOiAxOTI2ODM2
OTMzLCJwZXJjZW50IjogIjA2LjAxIn0sImRhdGEiOiAiMTIifSx7ImhpdHMiOiB7ImNvdW50Ijog
NTQsInBlcmNlbnQiOiAiMDMuNTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNDAsInBlcmNlbnQi
OiAiMDQuODAifSwiYnl0ZXMiOiB7ImNvdW50IjogMTM5NDgyNzIzOCwicGVyY2VudCI6ICIwNC4z
NSJ9LCJkYXRhIjogIjEzIn0seyJoaXRzIjogeyJjb3VudCI6IDEwNiwicGVyY2VudCI6ICIwNi45
MSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA1NCwicGVyY2VudCI6ICIwNi40NyJ9LCJieXRlcyI6
IHsiY291bnQiOiAxNjczMTk3MTQyLCJwZXJjZW50IjogIjA1LjIyIn0sImRhdGEiOiAiMTQifSx7
ImhpdHMiOiB7ImNvdW50IjogODAsInBlcmNlbnQiOiAiMDUuMjIifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogNDMsInBlcmNlbnQiOiAiMDUuMTYifSwiYnl0ZXMiOiB7ImNvdW50IjogMTcwMzM0NjIy
NSwicGVyY2VudCI6ICIwNS4zMSJ9LCJkYXRhIjogIjE1In0seyJoaXRzIjogeyJjb3VudCI6IDY0
LCJwZXJjZW50IjogIjA0LjE3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQxLCJwZXJjZW50Ijog
IjA0LjkyIn0sImJ5dGVzIjogeyJjb3VudCI6IDE2NTQ5ODA4ODEsInBlcmNlbnQiOiAiMDUuMTYi
fSwiZGF0YSI6ICIxNiJ9LHsiaGl0cyI6IHsiY291bnQiOiA2OSwicGVyY2VudCI6ICIwNC41MCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0NSwicGVyY2VudCI6ICIwNS40MCJ9LCJieXRlcyI6IHsi
Y291bnQiOiAxNjQ4MTkxMTIwLCJwZXJjZW50IjogIjA1LjE0In0sImRhdGEiOiAiMTcifSx7Imhp
dHMiOiB7ImNvdW50IjogNjgsInBlcmNlbnQiOiAiMDQuNDQifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogNDAsInBlcmNlbnQiOiAiMDQuODAifSwiYnl0ZXMiOiB7ImNvdW50IjogMTcxNjAxMTkxMywi
cGVyY2VudCI6ICIwNS4zNSJ9LCJkYXRhIjogIjE4In0seyJoaXRzIjogeyJjb3VudCI6IDEwNCwi
cGVyY2VudCI6ICIwNi43OCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0OCwicGVyY2VudCI6ICIw
NS43NiJ9LCJieXRlcyI6IHsiY291bnQiOiAxODIzNjQ0MTI5LCJwZXJjZW50IjogIjA1LjY5In0s
ImRhdGEiOiAiMTkifSx7ImhpdHMiOiB7ImNvdW50IjogNTUsInBlcmNlbnQiOiAiMDMuNTkifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMzksInBlcmNlbnQiOiAiMDQuNjgifSwiYnl0ZXMiOiB7ImNv
dW50IjogMTU5NjI5ODA2NCwicGVyY2VudCI6ICIwNC45OCJ9LCJkYXRhIjogIjIwIn0seyJoaXRz
IjogeyJjb3VudCI6IDQ3LCJwZXJjZW50IjogIjAzLjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDMwLCJwZXJjZW50IjogIjAzLjYwIn0sImJ5dGVzIjogeyJjb3VudCI6IDk1NzM2Mzc1NCwicGVy
Y2VudCI6ICIwMi45OCJ9LCJkYXRhIjogIjIxIn0seyJoaXRzIjogeyJjb3VudCI6IDU5LCJwZXJj
ZW50IjogIjAzLjg1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDM1LCJwZXJjZW50IjogIjA0LjIw
In0sImJ5dGVzIjogeyJjb3VudCI6IDExMjMwMDE5OTYsInBlcmNlbnQiOiAiMDMuNTAifSwiZGF0
YSI6ICIyMiJ9LHsiaGl0cyI6IHsiY291bnQiOiAzNSwicGVyY2VudCI6ICIwMi4yOCJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAyNywicGVyY2VudCI6ICIwMy4yNCJ9LCJieXRlcyI6IHsiY291bnQi
OiA5MDU2NTM1MDEsInBlcmNlbnQiOiAiMDIuODIifSwiZGF0YSI6ICIyMyJ9XX0sInJlZmVycmlu
Z19zaXRlcyI6IHsibWV0YWRhdGEiOiB7ImJ5dGVzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiAyODYz
OTM1Njk1OH0sImF2ZyI6IHsidmFsdWUiOiA3MTU5ODM5MjMyLCJwZXJjZW50IjogIjI1LjAwIn0s
Im1heCI6IHsidmFsdWUiOiAyODU4NDUxMzQxMSwicGVyY2VudCI6ICI5OS44MSJ9LCJtaW4iOiB7
InZhbHVlIjogNTcyNiwicGVyY2VudCI6ICIwMC4wMCJ9fSwidmlzaXRvcnMiOiB7InRvdGFsIjog
eyJ2YWx1ZSI6IDc2NH0sImF2ZyI6IHsidmFsdWUiOiAxOTEsInBlcmNlbnQiOiAiMjUuMDAifSwi
bWF4IjogeyJ2YWx1ZSI6IDc1NywicGVyY2VudCI6ICI5OS4wOCJ9LCJtaW4iOiB7InZhbHVlIjog
MiwicGVyY2VudCI6ICIwMC4yNiJ9fSwiaGl0cyI6IHsidG90YWwiOiB7InZhbHVlIjogMTQwMn0s
ImF2ZyI6IHsidmFsdWUiOiAzNTAsInBlcmNlbnQiOiAiMjQuOTYifSwibWF4IjogeyJ2YWx1ZSI6
IDEzOTIsInBlcmNlbnQiOiAiOTkuMjkifSwibWluIjogeyJ2YWx1ZSI6IDEsInBlcmNlbnQiOiAi
MDAuMDcifX0sImRhdGEiOiB7InRvdGFsIjogewkidmFsdWUiOiA0fX19LCJkYXRhIjogW3siaGl0
cyI6IHsiY291bnQiOiAxMzkyLCJwZXJjZW50IjogIjkwLjgwIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDc1NywicGVyY2VudCI6ICI5MC43NyJ9LCJieXRlcyI6IHsiY291bnQiOiAyODU4NDUxMzQx
MSwicGVyY2VudCI6ICI4OS4xMSJ9LCJkYXRhIjogInhlbnByb2plY3Qub3JnIn0seyJoaXRzIjog
eyJjb3VudCI6IDcsInBlcmNlbnQiOiAiMDAuNDYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNSwi
cGVyY2VudCI6ICIwMC42MCJ9LCJieXRlcyI6IHsiY291bnQiOiA1NDQ1MzIwNywicGVyY2VudCI6
ICIwMC4xNyJ9LCJkYXRhIjogImRvd25sb2Fkcy54ZW5wcm9qZWN0Lm9yZyJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBl
cmNlbnQiOiAiMDAuMjQifSwiYnl0ZXMiOiB7ImNvdW50IjogMzg0NjE0LCJwZXJjZW50IjogIjAw
LjAwIn0sImRhdGEiOiAiYmFpZHUuY29tIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1NzI2LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiMTcyLjI1
NS41MS4xMTkifV19LCJzdGF0dXNfY29kZXMiOiB7Im1ldGFkYXRhIjogeyJieXRlcyI6IHsidG90
YWwiOiB7InZhbHVlIjogMzIwNzYwOTM1MDJ9LCJhdmciOiB7InZhbHVlIjogMTA2OTIwMzE0ODgs
InBlcmNlbnQiOiAiMzMuMzMifSwibWF4IjogeyJ2YWx1ZSI6IDI5NzA0MzA4NTU1LCJwZXJjZW50
IjogIjkyLjYxIn0sIm1pbiI6IHsidmFsdWUiOiAxMTQ1MiwicGVyY2VudCI6ICIwMC4wMCJ9fSwi
dmlzaXRvcnMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDg5OX0sImF2ZyI6IHsidmFsdWUiOiAyOTks
InBlcmNlbnQiOiAiMzMuMjYifSwibWF4IjogeyJ2YWx1ZSI6IDgzMCwicGVyY2VudCI6ICI5Mi4z
MiJ9LCJtaW4iOiB7InZhbHVlIjogNjksInBlcmNlbnQiOiAiMDcuNjgifX0sImhpdHMiOiB7InRv
dGFsIjogeyJ2YWx1ZSI6IDE1MzN9LCJhdmciOiB7InZhbHVlIjogNTExLCJwZXJjZW50IjogIjMz
LjMzIn0sIm1heCI6IHsidmFsdWUiOiAxMTM0LCJwZXJjZW50IjogIjczLjk3In0sIm1pbiI6IHsi
dmFsdWUiOiAyLCJwZXJjZW50IjogIjAwLjEzIn19LCJkYXRhIjogeyJ0b3RhbCI6IHsJInZhbHVl
IjogM319fSwiZGF0YSI6IFt7ImhpdHMiOiB7ImNvdW50IjogMTUzMSwicGVyY2VudCI6ICI5OS44
NyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA4OTksInBlcmNlbnQiOiAiMTA3Ljc5In0sImJ5dGVz
IjogeyJjb3VudCI6IDMyMDc2MDgyMDUwLCJwZXJjZW50IjogIjEwMC4wMCJ9LCJkYXRhIjogIjJ4
eCBTdWNjZXNzIiwiaXRlbXMiOiBbeyJoaXRzIjogeyJjb3VudCI6IDExMzQsInBlcmNlbnQiOiAi
NzMuOTcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogODMwLCJwZXJjZW50IjogIjk5LjUyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDI5NzA0MzA4NTU1LCJwZXJjZW50IjogIjkyLjYxIn0sImRhdGEiOiAi
MjAwIC0gT0s6IFRoZSByZXF1ZXN0IHNlbnQgYnkgdGhlIGNsaWVudCB3YXMgc3VjY2Vzc2Z1bCJ9
LHsiaGl0cyI6IHsiY291bnQiOiAzOTcsInBlcmNlbnQiOiAiMjUuOTAifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogNjksInBlcmNlbnQiOiAiMDguMjcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjM3MTc3
MzQ5NSwicGVyY2VudCI6ICIwNy4zOSJ9LCJkYXRhIjogIjIwNiAtIFBhcnRpYWwgQ29udGVudDog
VGhlIHBhcnRpYWwgR0VUIGhhcyBiZWVuIHN1Y2Nlc3NmdWwifV19LHsiaGl0cyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDAsInBlcmNlbnQi
OiAiMDAuMDAifSwiYnl0ZXMiOiB7ImNvdW50IjogMTE0NTIsInBlcmNlbnQiOiAiMDAuMDAifSwi
ZGF0YSI6ICI0eHggQ2xpZW50IEVycm9ycyIsIml0ZW1zIjogW3siaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjEzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDAsInBlcmNlbnQiOiAi
MDAuMDAifSwiYnl0ZXMiOiB7ImNvdW50IjogMTE0NTIsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0
YSI6ICI0MDQgLSBOb3QgRm91bmQ6IFJlcXVlc3RlZCByZXNvdXJjZSBjb3VsZCBub3QgYmUgZm91
bmQifV19XX19PC9zY3JpcHQ+PHNjcmlwdD4vKiBodHRwczovL2QzanMub3JnIHY3LjguNCBDb3B5
cmlnaHQgMjAxMC0yMDIzIE1pa2UgQm9zdG9jayAqLyFmdW5jdGlvbih0LG4peyJvYmplY3QiPT10
eXBlb2YgZXhwb3J0cyYmInVuZGVmaW5lZCIhPXR5cGVvZiBtb2R1bGU/bihleHBvcnRzKToiZnVu
Y3Rpb24iPT10eXBlb2YgZGVmaW5lJiZkZWZpbmUuYW1kP2RlZmluZShbImV4cG9ydHMiXSxuKTpu
KCh0PSJ1bmRlZmluZWQiIT10eXBlb2YgZ2xvYmFsVGhpcz9nbG9iYWxUaGlzOnR8fHNlbGYpLmQz
PXQuZDN8fHt9KX0odGhpcywoZnVuY3Rpb24odCl7InVzZSBzdHJpY3QiO2Z1bmN0aW9uIG4odCxu
KXtyZXR1cm4gbnVsbD09dHx8bnVsbD09bj9OYU46dDxuPy0xOnQ+bj8xOnQ+PW4/MDpOYU59ZnVu
Y3Rpb24gZSh0LG4pe3JldHVybiBudWxsPT10fHxudWxsPT1uP05hTjpuPHQ/LTE6bj50PzE6bj49
dD8wOk5hTn1mdW5jdGlvbiByKHQpe2xldCByLG8sYTtmdW5jdGlvbiB1KHQsbixlPTAsaT10Lmxl
bmd0aCl7aWYoZTxpKXtpZigwIT09cihuLG4pKXJldHVybiBpO2Rve2NvbnN0IHI9ZStpPj4+MTtv
KHRbcl0sbik8MD9lPXIrMTppPXJ9d2hpbGUoZTxpKX1yZXR1cm4gZX1yZXR1cm4gMiE9PXQubGVu
Z3RoPyhyPW4sbz0oZSxyKT0+bih0KGUpLHIpLGE9KG4sZSk9PnQobiktZSk6KHI9dD09PW58fHQ9
PT1lP3Q6aSxvPXQsYT10KSx7bGVmdDp1LGNlbnRlcjpmdW5jdGlvbih0LG4sZT0wLHI9dC5sZW5n
dGgpe2NvbnN0IGk9dSh0LG4sZSxyLTEpO3JldHVybiBpPmUmJmEodFtpLTFdLG4pPi1hKHRbaV0s
bik/aS0xOml9LHJpZ2h0OmZ1bmN0aW9uKHQsbixlPTAsaT10Lmxlbmd0aCl7aWYoZTxpKXtpZigw
IT09cihuLG4pKXJldHVybiBpO2Rve2NvbnN0IHI9ZStpPj4+MTtvKHRbcl0sbik8PTA/ZT1yKzE6
aT1yfXdoaWxlKGU8aSl9cmV0dXJuIGV9fX1mdW5jdGlvbiBpKCl7cmV0dXJuIDB9ZnVuY3Rpb24g
byh0KXtyZXR1cm4gbnVsbD09PXQ/TmFOOit0fWZ1bmN0aW9uKmEodCxuKXtpZih2b2lkIDA9PT1u
KWZvcihsZXQgbiBvZiB0KW51bGwhPW4mJihuPStuKT49biYmKHlpZWxkIG4pO2Vsc2V7bGV0IGU9
LTE7Zm9yKGxldCByIG9mIHQpbnVsbCE9KHI9bihyLCsrZSx0KSkmJihyPStyKT49ciYmKHlpZWxk
IHIpfX1jb25zdCB1PXIobiksYz11LnJpZ2h0LGY9dS5sZWZ0LHM9cihvKS5jZW50ZXI7dmFyIGw9
Yztjb25zdCBoPXAodiksZD1wKChmdW5jdGlvbih0KXtjb25zdCBuPXYodCk7cmV0dXJuKHQsZSxy
LGksbyk9PntuKHQsZSwocjw8PTIpKzAsKGk8PD0yKSswLG88PD0yKSxuKHQsZSxyKzEsaSsxLG8p
LG4odCxlLHIrMixpKzIsbyksbih0LGUsciszLGkrMyxvKX19KSk7ZnVuY3Rpb24gcCh0KXtyZXR1
cm4gZnVuY3Rpb24obixlLHI9ZSl7aWYoISgoZT0rZSk+PTApKXRocm93IG5ldyBSYW5nZUVycm9y
KCJpbnZhbGlkIHJ4Iik7aWYoISgocj0rcik+PTApKXRocm93IG5ldyBSYW5nZUVycm9yKCJpbnZh
bGlkIHJ5Iik7bGV0e2RhdGE6aSx3aWR0aDpvLGhlaWdodDphfT1uO2lmKCEoKG89TWF0aC5mbG9v
cihvKSk+PTApKXRocm93IG5ldyBSYW5nZUVycm9yKCJpbnZhbGlkIHdpZHRoIik7aWYoISgoYT1N
YXRoLmZsb29yKHZvaWQgMCE9PWE/YTppLmxlbmd0aC9vKSk+PTApKXRocm93IG5ldyBSYW5nZUVy
cm9yKCJpbnZhbGlkIGhlaWdodCIpO2lmKCFvfHwhYXx8IWUmJiFyKXJldHVybiBuO2NvbnN0IHU9
ZSYmdChlKSxjPXImJnQociksZj1pLnNsaWNlKCk7cmV0dXJuIHUmJmM/KGcodSxmLGksbyxhKSxn
KHUsaSxmLG8sYSksZyh1LGYsaSxvLGEpLHkoYyxpLGYsbyxhKSx5KGMsZixpLG8sYSkseShjLGks
ZixvLGEpKTp1PyhnKHUsaSxmLG8sYSksZyh1LGYsaSxvLGEpLGcodSxpLGYsbyxhKSk6YyYmKHko
YyxpLGYsbyxhKSx5KGMsZixpLG8sYSkseShjLGksZixvLGEpKSxufX1mdW5jdGlvbiBnKHQsbixl
LHIsaSl7Zm9yKGxldCBvPTAsYT1yKmk7bzxhOyl0KG4sZSxvLG8rPXIsMSl9ZnVuY3Rpb24geSh0
LG4sZSxyLGkpe2ZvcihsZXQgbz0wLGE9cippO288cjsrK28pdChuLGUsbyxvK2Escil9ZnVuY3Rp
b24gdih0KXtjb25zdCBuPU1hdGguZmxvb3IodCk7aWYobj09PXQpcmV0dXJuIGZ1bmN0aW9uKHQp
e2NvbnN0IG49Mip0KzE7cmV0dXJuKGUscixpLG8sYSk9PntpZighKChvLT1hKT49aSkpcmV0dXJu
O2xldCB1PXQqcltpXTtjb25zdCBjPWEqdDtmb3IobGV0IHQ9aSxuPWkrYzt0PG47dCs9YSl1Kz1y
W01hdGgubWluKG8sdCldO2ZvcihsZXQgdD1pLGY9bzt0PD1mO3QrPWEpdSs9cltNYXRoLm1pbihv
LHQrYyldLGVbdF09dS9uLHUtPXJbTWF0aC5tYXgoaSx0LWMpXX19KHQpO2NvbnN0IGU9dC1uLHI9
Mip0KzE7cmV0dXJuKHQsaSxvLGEsdSk9PntpZighKChhLT11KT49bykpcmV0dXJuO2xldCBjPW4q
aVtvXTtjb25zdCBmPXUqbixzPWYrdTtmb3IobGV0IHQ9byxuPW8rZjt0PG47dCs9dSljKz1pW01h
dGgubWluKGEsdCldO2ZvcihsZXQgbj1vLGw9YTtuPD1sO24rPXUpYys9aVtNYXRoLm1pbihhLG4r
ZildLHRbbl09KGMrZSooaVtNYXRoLm1heChvLG4tcyldK2lbTWF0aC5taW4oYSxuK3MpXSkpL3Is
Yy09aVtNYXRoLm1heChvLG4tZildfX1mdW5jdGlvbiBfKHQsbil7bGV0IGU9MDtpZih2b2lkIDA9
PT1uKWZvcihsZXQgbiBvZiB0KW51bGwhPW4mJihuPStuKT49biYmKytlO2Vsc2V7bGV0IHI9LTE7
Zm9yKGxldCBpIG9mIHQpbnVsbCE9KGk9bihpLCsrcix0KSkmJihpPStpKT49aSYmKytlfXJldHVy
biBlfWZ1bmN0aW9uIGIodCl7cmV0dXJuIDB8dC5sZW5ndGh9ZnVuY3Rpb24gbSh0KXtyZXR1cm4h
KHQ+MCl9ZnVuY3Rpb24geCh0KXtyZXR1cm4ib2JqZWN0IiE9dHlwZW9mIHR8fCJsZW5ndGgiaW4g
dD90OkFycmF5LmZyb20odCl9ZnVuY3Rpb24gdyh0LG4pe2xldCBlLHI9MCxpPTAsbz0wO2lmKHZv
aWQgMD09PW4pZm9yKGxldCBuIG9mIHQpbnVsbCE9biYmKG49K24pPj1uJiYoZT1uLWksaSs9ZS8r
K3Isbys9ZSoobi1pKSk7ZWxzZXtsZXQgYT0tMTtmb3IobGV0IHUgb2YgdCludWxsIT0odT1uKHUs
KythLHQpKSYmKHU9K3UpPj11JiYoZT11LWksaSs9ZS8rK3Isbys9ZSoodS1pKSl9aWYocj4xKXJl
dHVybiBvLyhyLTEpfWZ1bmN0aW9uIE0odCxuKXtjb25zdCBlPXcodCxuKTtyZXR1cm4gZT9NYXRo
LnNxcnQoZSk6ZX1mdW5jdGlvbiBUKHQsbil7bGV0IGUscjtpZih2b2lkIDA9PT1uKWZvcihjb25z
dCBuIG9mIHQpbnVsbCE9biYmKHZvaWQgMD09PWU/bj49biYmKGU9cj1uKTooZT5uJiYoZT1uKSxy
PG4mJihyPW4pKSk7ZWxzZXtsZXQgaT0tMTtmb3IobGV0IG8gb2YgdCludWxsIT0obz1uKG8sKytp
LHQpKSYmKHZvaWQgMD09PWU/bz49byYmKGU9cj1vKTooZT5vJiYoZT1vKSxyPG8mJihyPW8pKSl9
cmV0dXJuW2Uscl19Y2xhc3MgQXtjb25zdHJ1Y3Rvcigpe3RoaXMuX3BhcnRpYWxzPW5ldyBGbG9h
dDY0QXJyYXkoMzIpLHRoaXMuX249MH1hZGQodCl7Y29uc3Qgbj10aGlzLl9wYXJ0aWFscztsZXQg
ZT0wO2ZvcihsZXQgcj0wO3I8dGhpcy5fbiYmcjwzMjtyKyspe2NvbnN0IGk9bltyXSxvPXQraSxh
PU1hdGguYWJzKHQpPE1hdGguYWJzKGkpP3QtKG8taSk6aS0oby10KTthJiYobltlKytdPWEpLHQ9
b31yZXR1cm4gbltlXT10LHRoaXMuX249ZSsxLHRoaXN9dmFsdWVPZigpe2NvbnN0IHQ9dGhpcy5f
cGFydGlhbHM7bGV0IG4sZSxyLGk9dGhpcy5fbixvPTA7aWYoaT4wKXtmb3Iobz10Wy0taV07aT4w
JiYobj1vLGU9dFstLWldLG89bitlLHI9ZS0oby1uKSwhcik7KTtpPjAmJihyPDAmJnRbaS0xXTww
fHxyPjAmJnRbaS0xXT4wKSYmKGU9MipyLG49bytlLGU9PW4tbyYmKG89bikpfXJldHVybiBvfX1j
bGFzcyBJbnRlcm5NYXAgZXh0ZW5kcyBNYXB7Y29uc3RydWN0b3IodCxuPWspe2lmKHN1cGVyKCks
T2JqZWN0LmRlZmluZVByb3BlcnRpZXModGhpcyx7X2ludGVybjp7dmFsdWU6bmV3IE1hcH0sX2tl
eTp7dmFsdWU6bn19KSxudWxsIT10KWZvcihjb25zdFtuLGVdb2YgdCl0aGlzLnNldChuLGUpfWdl
dCh0KXtyZXR1cm4gc3VwZXIuZ2V0KFModGhpcyx0KSl9aGFzKHQpe3JldHVybiBzdXBlci5oYXMo
Uyh0aGlzLHQpKX1zZXQodCxuKXtyZXR1cm4gc3VwZXIuc2V0KEUodGhpcyx0KSxuKX1kZWxldGUo
dCl7cmV0dXJuIHN1cGVyLmRlbGV0ZShOKHRoaXMsdCkpfX1jbGFzcyBJbnRlcm5TZXQgZXh0ZW5k
cyBTZXR7Y29uc3RydWN0b3IodCxuPWspe2lmKHN1cGVyKCksT2JqZWN0LmRlZmluZVByb3BlcnRp
ZXModGhpcyx7X2ludGVybjp7dmFsdWU6bmV3IE1hcH0sX2tleTp7dmFsdWU6bn19KSxudWxsIT10
KWZvcihjb25zdCBuIG9mIHQpdGhpcy5hZGQobil9aGFzKHQpe3JldHVybiBzdXBlci5oYXMoUyh0
aGlzLHQpKX1hZGQodCl7cmV0dXJuIHN1cGVyLmFkZChFKHRoaXMsdCkpfWRlbGV0ZSh0KXtyZXR1
cm4gc3VwZXIuZGVsZXRlKE4odGhpcyx0KSl9fWZ1bmN0aW9uIFMoe19pbnRlcm46dCxfa2V5Om59
LGUpe2NvbnN0IHI9bihlKTtyZXR1cm4gdC5oYXMocik/dC5nZXQocik6ZX1mdW5jdGlvbiBFKHtf
aW50ZXJuOnQsX2tleTpufSxlKXtjb25zdCByPW4oZSk7cmV0dXJuIHQuaGFzKHIpP3QuZ2V0KHIp
Oih0LnNldChyLGUpLGUpfWZ1bmN0aW9uIE4oe19pbnRlcm46dCxfa2V5Om59LGUpe2NvbnN0IHI9
bihlKTtyZXR1cm4gdC5oYXMocikmJihlPXQuZ2V0KHIpLHQuZGVsZXRlKHIpKSxlfWZ1bmN0aW9u
IGsodCl7cmV0dXJuIG51bGwhPT10JiYib2JqZWN0Ij09dHlwZW9mIHQ/dC52YWx1ZU9mKCk6dH1m
dW5jdGlvbiBDKHQpe3JldHVybiB0fWZ1bmN0aW9uIFAodCwuLi5uKXtyZXR1cm4gcSh0LEMsQyxu
KX1mdW5jdGlvbiB6KHQsLi4ubil7cmV0dXJuIHEodCxBcnJheS5mcm9tLEMsbil9ZnVuY3Rpb24g
JCh0LG4pe2ZvcihsZXQgZT0xLHI9bi5sZW5ndGg7ZTxyOysrZSl0PXQuZmxhdE1hcCgodD0+dC5w
b3AoKS5tYXAoKChbbixlXSk9PlsuLi50LG4sZV0pKSkpO3JldHVybiB0fWZ1bmN0aW9uIEQodCxu
LC4uLmUpe3JldHVybiBxKHQsQyxuLGUpfWZ1bmN0aW9uIFIodCxuLC4uLmUpe3JldHVybiBxKHQs
QXJyYXkuZnJvbSxuLGUpfWZ1bmN0aW9uIEYodCl7aWYoMSE9PXQubGVuZ3RoKXRocm93IG5ldyBF
cnJvcigiZHVwbGljYXRlIGtleSIpO3JldHVybiB0WzBdfWZ1bmN0aW9uIHEodCxuLGUscil7cmV0
dXJuIGZ1bmN0aW9uIHQoaSxvKXtpZihvPj1yLmxlbmd0aClyZXR1cm4gZShpKTtjb25zdCBhPW5l
dyBJbnRlcm5NYXAsdT1yW28rK107bGV0IGM9LTE7Zm9yKGNvbnN0IHQgb2YgaSl7Y29uc3Qgbj11
KHQsKytjLGkpLGU9YS5nZXQobik7ZT9lLnB1c2godCk6YS5zZXQobixbdF0pfWZvcihjb25zdFtu
LGVdb2YgYSlhLnNldChuLHQoZSxvKSk7cmV0dXJuIG4oYSl9KHQsMCl9ZnVuY3Rpb24gVSh0LG4p
e3JldHVybiBBcnJheS5mcm9tKG4sKG49PnRbbl0pKX1mdW5jdGlvbiBJKHQsLi4ubil7aWYoImZ1
bmN0aW9uIiE9dHlwZW9mIHRbU3ltYm9sLml0ZXJhdG9yXSl0aHJvdyBuZXcgVHlwZUVycm9yKCJ2
YWx1ZXMgaXMgbm90IGl0ZXJhYmxlIik7dD1BcnJheS5mcm9tKHQpO2xldFtlXT1uO2lmKGUmJjIh
PT1lLmxlbmd0aHx8bi5sZW5ndGg+MSl7Y29uc3Qgcj1VaW50MzJBcnJheS5mcm9tKHQsKCh0LG4p
PT5uKSk7cmV0dXJuIG4ubGVuZ3RoPjE/KG49bi5tYXAoKG49PnQubWFwKG4pKSksci5zb3J0KCgo
dCxlKT0+e2Zvcihjb25zdCByIG9mIG4pe2NvbnN0IG49QihyW3RdLHJbZV0pO2lmKG4pcmV0dXJu
IG59fSkpKTooZT10Lm1hcChlKSxyLnNvcnQoKCh0LG4pPT5CKGVbdF0sZVtuXSkpKSksVSh0LHIp
fXJldHVybiB0LnNvcnQoTyhlKSl9ZnVuY3Rpb24gTyh0PW4pe2lmKHQ9PT1uKXJldHVybiBCO2lm
KCJmdW5jdGlvbiIhPXR5cGVvZiB0KXRocm93IG5ldyBUeXBlRXJyb3IoImNvbXBhcmUgaXMgbm90
IGEgZnVuY3Rpb24iKTtyZXR1cm4obixlKT0+e2NvbnN0IHI9dChuLGUpO3JldHVybiByfHwwPT09
cj9yOigwPT09dChlLGUpKS0oMD09PXQobixuKSl9fWZ1bmN0aW9uIEIodCxuKXtyZXR1cm4obnVs
bD09dHx8ISh0Pj10KSktKG51bGw9PW58fCEobj49bikpfHwodDxuPy0xOnQ+bj8xOjApfXZhciBZ
PUFycmF5LnByb3RvdHlwZS5zbGljZTtmdW5jdGlvbiBMKHQpe3JldHVybigpPT50fWNvbnN0IGo9
TWF0aC5zcXJ0KDUwKSxIPU1hdGguc3FydCgxMCksWD1NYXRoLnNxcnQoMik7ZnVuY3Rpb24gRyh0
LG4sZSl7Y29uc3Qgcj0obi10KS9NYXRoLm1heCgwLGUpLGk9TWF0aC5mbG9vcihNYXRoLmxvZzEw
KHIpKSxvPXIvTWF0aC5wb3coMTAsaSksYT1vPj1qPzEwOm8+PUg/NTpvPj1YPzI6MTtsZXQgdSxj
LGY7cmV0dXJuIGk8MD8oZj1NYXRoLnBvdygxMCwtaSkvYSx1PU1hdGgucm91bmQodCpmKSxjPU1h
dGgucm91bmQobipmKSx1L2Y8dCYmKyt1LGMvZj5uJiYtLWMsZj0tZik6KGY9TWF0aC5wb3coMTAs
aSkqYSx1PU1hdGgucm91bmQodC9mKSxjPU1hdGgucm91bmQobi9mKSx1KmY8dCYmKyt1LGMqZj5u
JiYtLWMpLGM8dSYmLjU8PWUmJmU8Mj9HKHQsbiwyKmUpOlt1LGMsZl19ZnVuY3Rpb24gVih0LG4s
ZSl7aWYoISgoZT0rZSk+MCkpcmV0dXJuW107aWYoKHQ9K3QpPT09KG49K24pKXJldHVyblt0XTtj
b25zdCByPW48dCxbaSxvLGFdPXI/RyhuLHQsZSk6Ryh0LG4sZSk7aWYoIShvPj1pKSlyZXR1cm5b
XTtjb25zdCB1PW8taSsxLGM9bmV3IEFycmF5KHUpO2lmKHIpaWYoYTwwKWZvcihsZXQgdD0wO3Q8
dTsrK3QpY1t0XT0oby10KS8tYTtlbHNlIGZvcihsZXQgdD0wO3Q8dTsrK3QpY1t0XT0oby10KSph
O2Vsc2UgaWYoYTwwKWZvcihsZXQgdD0wO3Q8dTsrK3QpY1t0XT0oaSt0KS8tYTtlbHNlIGZvcihs
ZXQgdD0wO3Q8dTsrK3QpY1t0XT0oaSt0KSphO3JldHVybiBjfWZ1bmN0aW9uIFcodCxuLGUpe3Jl
dHVybiBHKHQ9K3Qsbj0rbixlPStlKVsyXX1mdW5jdGlvbiBaKHQsbixlKXtlPStlO2NvbnN0IHI9
KG49K24pPCh0PSt0KSxpPXI/VyhuLHQsZSk6Vyh0LG4sZSk7cmV0dXJuKHI/LTE6MSkqKGk8MD8x
Ly1pOmkpfWZ1bmN0aW9uIEsodCxuLGUpe2xldCByO2Zvcig7Oyl7Y29uc3QgaT1XKHQsbixlKTtp
ZihpPT09cnx8MD09PWl8fCFpc0Zpbml0ZShpKSlyZXR1cm5bdCxuXTtpPjA/KHQ9TWF0aC5mbG9v
cih0L2kpKmksbj1NYXRoLmNlaWwobi9pKSppKTppPDAmJih0PU1hdGguY2VpbCh0KmkpL2ksbj1N
YXRoLmZsb29yKG4qaSkvaSkscj1pfX1mdW5jdGlvbiBRKHQpe3JldHVybiBNYXRoLm1heCgxLE1h
dGguY2VpbChNYXRoLmxvZyhfKHQpKS9NYXRoLkxOMikrMSl9ZnVuY3Rpb24gSigpe3ZhciB0PUMs
bj1ULGU9UTtmdW5jdGlvbiByKHIpe0FycmF5LmlzQXJyYXkocil8fChyPUFycmF5LmZyb20ocikp
O3ZhciBpLG8sYSx1PXIubGVuZ3RoLGM9bmV3IEFycmF5KHUpO2ZvcihpPTA7aTx1OysraSljW2ld
PXQocltpXSxpLHIpO3ZhciBmPW4oYykscz1mWzBdLGg9ZlsxXSxkPWUoYyxzLGgpO2lmKCFBcnJh
eS5pc0FycmF5KGQpKXtjb25zdCB0PWgsZT0rZDtpZihuPT09VCYmKFtzLGhdPUsocyxoLGUpKSwo
ZD1WKHMsaCxlKSlbMF08PXMmJihhPVcocyxoLGUpKSxkW2QubGVuZ3RoLTFdPj1oKWlmKHQ+PWgm
Jm49PT1UKXtjb25zdCB0PVcocyxoLGUpO2lzRmluaXRlKHQpJiYodD4wP2g9KE1hdGguZmxvb3Io
aC90KSsxKSp0OnQ8MCYmKGg9KE1hdGguY2VpbChoKi10KSsxKS8tdCkpfWVsc2UgZC5wb3AoKX1m
b3IodmFyIHA9ZC5sZW5ndGgsZz0wLHk9cDtkW2ddPD1zOykrK2c7Zm9yKDtkW3ktMV0+aDspLS15
OyhnfHx5PHApJiYoZD1kLnNsaWNlKGcseSkscD15LWcpO3ZhciB2LF89bmV3IEFycmF5KHArMSk7
Zm9yKGk9MDtpPD1wOysraSkodj1fW2ldPVtdKS54MD1pPjA/ZFtpLTFdOnMsdi54MT1pPHA/ZFtp
XTpoO2lmKGlzRmluaXRlKGEpKXtpZihhPjApZm9yKGk9MDtpPHU7KytpKW51bGwhPShvPWNbaV0p
JiZzPD1vJiZvPD1oJiZfW01hdGgubWluKHAsTWF0aC5mbG9vcigoby1zKS9hKSldLnB1c2gocltp
XSk7ZWxzZSBpZihhPDApZm9yKGk9MDtpPHU7KytpKWlmKG51bGwhPShvPWNbaV0pJiZzPD1vJiZv
PD1oKXtjb25zdCB0PU1hdGguZmxvb3IoKHMtbykqYSk7X1tNYXRoLm1pbihwLHQrKGRbdF08PW8p
KV0ucHVzaChyW2ldKX19ZWxzZSBmb3IoaT0wO2k8dTsrK2kpbnVsbCE9KG89Y1tpXSkmJnM8PW8m
Jm88PWgmJl9bbChkLG8sMCxwKV0ucHVzaChyW2ldKTtyZXR1cm4gX31yZXR1cm4gci52YWx1ZT1m
dW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD0iZnVuY3Rpb24iPT10eXBlb2Yg
bj9uOkwobikscik6dH0sci5kb21haW49ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5n
dGg/KG49ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpMKFt0WzBdLHRbMV1dKSxyKTpufSxyLnRocmVz
aG9sZHM9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGU9ImZ1bmN0aW9uIj09
dHlwZW9mIHQ/dDpMKEFycmF5LmlzQXJyYXkodCk/WS5jYWxsKHQpOnQpLHIpOmV9LHJ9ZnVuY3Rp
b24gdHQodCxuKXtsZXQgZTtpZih2b2lkIDA9PT1uKWZvcihjb25zdCBuIG9mIHQpbnVsbCE9biYm
KGU8bnx8dm9pZCAwPT09ZSYmbj49bikmJihlPW4pO2Vsc2V7bGV0IHI9LTE7Zm9yKGxldCBpIG9m
IHQpbnVsbCE9KGk9bihpLCsrcix0KSkmJihlPGl8fHZvaWQgMD09PWUmJmk+PWkpJiYoZT1pKX1y
ZXR1cm4gZX1mdW5jdGlvbiBudCh0LG4pe2xldCBlLHI9LTEsaT0tMTtpZih2b2lkIDA9PT1uKWZv
cihjb25zdCBuIG9mIHQpKytpLG51bGwhPW4mJihlPG58fHZvaWQgMD09PWUmJm4+PW4pJiYoZT1u
LHI9aSk7ZWxzZSBmb3IobGV0IG8gb2YgdCludWxsIT0obz1uKG8sKytpLHQpKSYmKGU8b3x8dm9p
ZCAwPT09ZSYmbz49bykmJihlPW8scj1pKTtyZXR1cm4gcn1mdW5jdGlvbiBldCh0LG4pe2xldCBl
O2lmKHZvaWQgMD09PW4pZm9yKGNvbnN0IG4gb2YgdCludWxsIT1uJiYoZT5ufHx2b2lkIDA9PT1l
JiZuPj1uKSYmKGU9bik7ZWxzZXtsZXQgcj0tMTtmb3IobGV0IGkgb2YgdCludWxsIT0oaT1uKGks
KytyLHQpKSYmKGU+aXx8dm9pZCAwPT09ZSYmaT49aSkmJihlPWkpfXJldHVybiBlfWZ1bmN0aW9u
IHJ0KHQsbil7bGV0IGUscj0tMSxpPS0xO2lmKHZvaWQgMD09PW4pZm9yKGNvbnN0IG4gb2YgdCkr
K2ksbnVsbCE9biYmKGU+bnx8dm9pZCAwPT09ZSYmbj49bikmJihlPW4scj1pKTtlbHNlIGZvcihs
ZXQgbyBvZiB0KW51bGwhPShvPW4obywrK2ksdCkpJiYoZT5vfHx2b2lkIDA9PT1lJiZvPj1vKSYm
KGU9byxyPWkpO3JldHVybiByfWZ1bmN0aW9uIGl0KHQsbixlPTAscj0xLzAsaSl7aWYobj1NYXRo
LmZsb29yKG4pLGU9TWF0aC5mbG9vcihNYXRoLm1heCgwLGUpKSxyPU1hdGguZmxvb3IoTWF0aC5t
aW4odC5sZW5ndGgtMSxyKSksIShlPD1uJiZuPD1yKSlyZXR1cm4gdDtmb3IoaT12b2lkIDA9PT1p
P0I6TyhpKTtyPmU7KXtpZihyLWU+NjAwKXtjb25zdCBvPXItZSsxLGE9bi1lKzEsdT1NYXRoLmxv
ZyhvKSxjPS41Kk1hdGguZXhwKDIqdS8zKSxmPS41Kk1hdGguc3FydCh1KmMqKG8tYykvbykqKGEt
by8yPDA/LTE6MSk7aXQodCxuLE1hdGgubWF4KGUsTWF0aC5mbG9vcihuLWEqYy9vK2YpKSxNYXRo
Lm1pbihyLE1hdGguZmxvb3Iobisoby1hKSpjL28rZikpLGkpfWNvbnN0IG89dFtuXTtsZXQgYT1l
LHU9cjtmb3Iob3QodCxlLG4pLGkodFtyXSxvKT4wJiZvdCh0LGUscik7YTx1Oyl7Zm9yKG90KHQs
YSx1KSwrK2EsLS11O2kodFthXSxvKTwwOykrK2E7Zm9yKDtpKHRbdV0sbyk+MDspLS11fTA9PT1p
KHRbZV0sbyk/b3QodCxlLHUpOigrK3Usb3QodCx1LHIpKSx1PD1uJiYoZT11KzEpLG48PXUmJihy
PXUtMSl9cmV0dXJuIHR9ZnVuY3Rpb24gb3QodCxuLGUpe2NvbnN0IHI9dFtuXTt0W25dPXRbZV0s
dFtlXT1yfWZ1bmN0aW9uIGF0KHQsZT1uKXtsZXQgcixpPSExO2lmKDE9PT1lLmxlbmd0aCl7bGV0
IG87Zm9yKGNvbnN0IGEgb2YgdCl7Y29uc3QgdD1lKGEpOyhpP24odCxvKT4wOjA9PT1uKHQsdCkp
JiYocj1hLG89dCxpPSEwKX19ZWxzZSBmb3IoY29uc3QgbiBvZiB0KShpP2UobixyKT4wOjA9PT1l
KG4sbikpJiYocj1uLGk9ITApO3JldHVybiByfWZ1bmN0aW9uIHV0KHQsbixlKXtpZigocj0odD1G
bG9hdDY0QXJyYXkuZnJvbShhKHQsZSkpKS5sZW5ndGgpJiYhaXNOYU4obj0rbikpe2lmKG48PTB8
fHI8MilyZXR1cm4gZXQodCk7aWYobj49MSlyZXR1cm4gdHQodCk7dmFyIHIsaT0oci0xKSpuLG89
TWF0aC5mbG9vcihpKSx1PXR0KGl0KHQsbykuc3ViYXJyYXkoMCxvKzEpKTtyZXR1cm4gdSsoZXQo
dC5zdWJhcnJheShvKzEpKS11KSooaS1vKX19ZnVuY3Rpb24gY3QodCxuLGU9byl7aWYoKHI9dC5s
ZW5ndGgpJiYhaXNOYU4obj0rbikpe2lmKG48PTB8fHI8MilyZXR1cm4rZSh0WzBdLDAsdCk7aWYo
bj49MSlyZXR1cm4rZSh0W3ItMV0sci0xLHQpO3ZhciByLGk9KHItMSkqbixhPU1hdGguZmxvb3Io
aSksdT0rZSh0W2FdLGEsdCk7cmV0dXJuIHUrKCtlKHRbYSsxXSxhKzEsdCktdSkqKGktYSl9fWZ1
bmN0aW9uIGZ0KHQsbixlKXtpZigocj0odD1GbG9hdDY0QXJyYXkuZnJvbShhKHQsZSkpKS5sZW5n
dGgpJiYhaXNOYU4obj0rbikpe2lmKG48PTB8fHI8MilyZXR1cm4gcnQodCk7aWYobj49MSlyZXR1
cm4gbnQodCk7dmFyIHIsaT1NYXRoLmZsb29yKChyLTEpKm4pLG89aXQoVWludDMyQXJyYXkuZnJv
bSh0LCgodCxuKT0+bikpLGksMCxyLTEsKChuLGUpPT5CKHRbbl0sdFtlXSkpKTtyZXR1cm4gYXQo
by5zdWJhcnJheSgwLGkrMSksKG49PnRbbl0pKX19ZnVuY3Rpb24gc3QodCl7cmV0dXJuIEFycmF5
LmZyb20oZnVuY3Rpb24qKHQpe2Zvcihjb25zdCBuIG9mIHQpeWllbGQqbn0odCkpfWZ1bmN0aW9u
IGx0KHQsbil7cmV0dXJuW3Qsbl19ZnVuY3Rpb24gaHQodCxuLGUpe3Q9K3Qsbj0rbixlPShpPWFy
Z3VtZW50cy5sZW5ndGgpPDI/KG49dCx0PTAsMSk6aTwzPzE6K2U7Zm9yKHZhciByPS0xLGk9MHxN
YXRoLm1heCgwLE1hdGguY2VpbCgobi10KS9lKSksbz1uZXcgQXJyYXkoaSk7KytyPGk7KW9bcl09
dCtyKmU7cmV0dXJuIG99ZnVuY3Rpb24gZHQodCxlPW4pe2lmKDE9PT1lLmxlbmd0aClyZXR1cm4g
cnQodCxlKTtsZXQgcixpPS0xLG89LTE7Zm9yKGNvbnN0IG4gb2YgdCkrK28sKGk8MD8wPT09ZShu
LG4pOmUobixyKTwwKSYmKHI9bixpPW8pO3JldHVybiBpfXZhciBwdD1ndChNYXRoLnJhbmRvbSk7
ZnVuY3Rpb24gZ3QodCl7cmV0dXJuIGZ1bmN0aW9uKG4sZT0wLHI9bi5sZW5ndGgpe2xldCBpPXIt
KGU9K2UpO2Zvcig7aTspe2NvbnN0IHI9dCgpKmktLXwwLG89bltpK2VdO25baStlXT1uW3IrZV0s
bltyK2VdPW99cmV0dXJuIG59fWZ1bmN0aW9uIHl0KHQpe2lmKCEoaT10Lmxlbmd0aCkpcmV0dXJu
W107Zm9yKHZhciBuPS0xLGU9ZXQodCx2dCkscj1uZXcgQXJyYXkoZSk7KytuPGU7KWZvcih2YXIg
aSxvPS0xLGE9cltuXT1uZXcgQXJyYXkoaSk7KytvPGk7KWFbb109dFtvXVtuXTtyZXR1cm4gcn1m
dW5jdGlvbiB2dCh0KXtyZXR1cm4gdC5sZW5ndGh9ZnVuY3Rpb24gX3QodCl7cmV0dXJuIHQgaW5z
dGFuY2VvZiBJbnRlcm5TZXQ/dDpuZXcgSW50ZXJuU2V0KHQpfWZ1bmN0aW9uIGJ0KHQsbil7Y29u
c3QgZT10W1N5bWJvbC5pdGVyYXRvcl0oKSxyPW5ldyBTZXQ7Zm9yKGNvbnN0IHQgb2Ygbil7Y29u
c3Qgbj1tdCh0KTtpZihyLmhhcyhuKSljb250aW51ZTtsZXQgaSxvO2Zvcig7KHt2YWx1ZTppLGRv
bmU6b309ZS5uZXh0KCkpOyl7aWYobylyZXR1cm4hMTtjb25zdCB0PW10KGkpO2lmKHIuYWRkKHQp
LE9iamVjdC5pcyhuLHQpKWJyZWFrfX1yZXR1cm4hMH1mdW5jdGlvbiBtdCh0KXtyZXR1cm4gbnVs
bCE9PXQmJiJvYmplY3QiPT10eXBlb2YgdD90LnZhbHVlT2YoKTp0fWZ1bmN0aW9uIHh0KHQpe3Jl
dHVybiB0fXZhciB3dD0xLE10PTIsVHQ9MyxBdD00LFN0PTFlLTY7ZnVuY3Rpb24gRXQodCl7cmV0
dXJuInRyYW5zbGF0ZSgiK3QrIiwwKSJ9ZnVuY3Rpb24gTnQodCl7cmV0dXJuInRyYW5zbGF0ZSgw
LCIrdCsiKSJ9ZnVuY3Rpb24ga3QodCl7cmV0dXJuIG49Pit0KG4pfWZ1bmN0aW9uIEN0KHQsbil7
cmV0dXJuIG49TWF0aC5tYXgoMCx0LmJhbmR3aWR0aCgpLTIqbikvMix0LnJvdW5kKCkmJihuPU1h
dGgucm91bmQobikpLGU9Pit0KGUpK259ZnVuY3Rpb24gUHQoKXtyZXR1cm4hdGhpcy5fX2F4aXN9
ZnVuY3Rpb24genQodCxuKXt2YXIgZT1bXSxyPW51bGwsaT1udWxsLG89NixhPTYsdT0zLGM9InVu
ZGVmaW5lZCIhPXR5cGVvZiB3aW5kb3cmJndpbmRvdy5kZXZpY2VQaXhlbFJhdGlvPjE/MDouNSxm
PXQ9PT13dHx8dD09PUF0Py0xOjEscz10PT09QXR8fHQ9PT1NdD8ieCI6InkiLGw9dD09PXd0fHx0
PT09VHQ/RXQ6TnQ7ZnVuY3Rpb24gaChoKXt2YXIgZD1udWxsPT1yP24udGlja3M/bi50aWNrcy5h
cHBseShuLGUpOm4uZG9tYWluKCk6cixwPW51bGw9PWk/bi50aWNrRm9ybWF0P24udGlja0Zvcm1h
dC5hcHBseShuLGUpOnh0OmksZz1NYXRoLm1heChvLDApK3UseT1uLnJhbmdlKCksdj0reVswXStj
LF89K3lbeS5sZW5ndGgtMV0rYyxiPShuLmJhbmR3aWR0aD9DdDprdCkobi5jb3B5KCksYyksbT1o
LnNlbGVjdGlvbj9oLnNlbGVjdGlvbigpOmgseD1tLnNlbGVjdEFsbCgiLmRvbWFpbiIpLmRhdGEo
W251bGxdKSx3PW0uc2VsZWN0QWxsKCIudGljayIpLmRhdGEoZCxuKS5vcmRlcigpLE09dy5leGl0
KCksVD13LmVudGVyKCkuYXBwZW5kKCJnIikuYXR0cigiY2xhc3MiLCJ0aWNrIiksQT13LnNlbGVj
dCgibGluZSIpLFM9dy5zZWxlY3QoInRleHQiKTt4PXgubWVyZ2UoeC5lbnRlcigpLmluc2VydCgi
cGF0aCIsIi50aWNrIikuYXR0cigiY2xhc3MiLCJkb21haW4iKS5hdHRyKCJzdHJva2UiLCJjdXJy
ZW50Q29sb3IiKSksdz13Lm1lcmdlKFQpLEE9QS5tZXJnZShULmFwcGVuZCgibGluZSIpLmF0dHIo
InN0cm9rZSIsImN1cnJlbnRDb2xvciIpLmF0dHIocysiMiIsZipvKSksUz1TLm1lcmdlKFQuYXBw
ZW5kKCJ0ZXh0IikuYXR0cigiZmlsbCIsImN1cnJlbnRDb2xvciIpLmF0dHIocyxmKmcpLmF0dHIo
ImR5Iix0PT09d3Q/IjBlbSI6dD09PVR0PyIwLjcxZW0iOiIwLjMyZW0iKSksaCE9PW0mJih4PXgu
dHJhbnNpdGlvbihoKSx3PXcudHJhbnNpdGlvbihoKSxBPUEudHJhbnNpdGlvbihoKSxTPVMudHJh
bnNpdGlvbihoKSxNPU0udHJhbnNpdGlvbihoKS5hdHRyKCJvcGFjaXR5IixTdCkuYXR0cigidHJh
bnNmb3JtIiwoZnVuY3Rpb24odCl7cmV0dXJuIGlzRmluaXRlKHQ9Yih0KSk/bCh0K2MpOnRoaXMu
Z2V0QXR0cmlidXRlKCJ0cmFuc2Zvcm0iKX0pKSxULmF0dHIoIm9wYWNpdHkiLFN0KS5hdHRyKCJ0
cmFuc2Zvcm0iLChmdW5jdGlvbih0KXt2YXIgbj10aGlzLnBhcmVudE5vZGUuX19heGlzO3JldHVy
biBsKChuJiZpc0Zpbml0ZShuPW4odCkpP246Yih0KSkrYyl9KSkpLE0ucmVtb3ZlKCkseC5hdHRy
KCJkIix0PT09QXR8fHQ9PT1NdD9hPyJNIitmKmErIiwiK3YrIkgiK2MrIlYiK18rIkgiK2YqYToi
TSIrYysiLCIrdisiViIrXzphPyJNIit2KyIsIitmKmErIlYiK2MrIkgiK18rIlYiK2YqYToiTSIr
disiLCIrYysiSCIrXyksdy5hdHRyKCJvcGFjaXR5IiwxKS5hdHRyKCJ0cmFuc2Zvcm0iLChmdW5j
dGlvbih0KXtyZXR1cm4gbChiKHQpK2MpfSkpLEEuYXR0cihzKyIyIixmKm8pLFMuYXR0cihzLGYq
ZykudGV4dChwKSxtLmZpbHRlcihQdCkuYXR0cigiZmlsbCIsIm5vbmUiKS5hdHRyKCJmb250LXNp
emUiLDEwKS5hdHRyKCJmb250LWZhbWlseSIsInNhbnMtc2VyaWYiKS5hdHRyKCJ0ZXh0LWFuY2hv
ciIsdD09PU10PyJzdGFydCI6dD09PUF0PyJlbmQiOiJtaWRkbGUiKSxtLmVhY2goKGZ1bmN0aW9u
KCl7dGhpcy5fX2F4aXM9Yn0pKX1yZXR1cm4gaC5zY2FsZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJn
dW1lbnRzLmxlbmd0aD8obj10LGgpOm59LGgudGlja3M9ZnVuY3Rpb24oKXtyZXR1cm4gZT1BcnJh
eS5mcm9tKGFyZ3VtZW50cyksaH0saC50aWNrQXJndW1lbnRzPWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyhlPW51bGw9PXQ/W106QXJyYXkuZnJvbSh0KSxoKTplLnNsaWNlKCl9
LGgudGlja1ZhbHVlcz1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj1udWxs
PT10P251bGw6QXJyYXkuZnJvbSh0KSxoKTpyJiZyLnNsaWNlKCl9LGgudGlja0Zvcm1hdD1mdW5j
dGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oaT10LGgpOml9LGgudGlja1NpemU9ZnVu
Y3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG89YT0rdCxoKTpvfSxoLnRpY2tTaXpl
SW5uZXI9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG89K3QsaCk6b30saC50
aWNrU2l6ZU91dGVyPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhhPSt0LGgp
OmF9LGgudGlja1BhZGRpbmc9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHU9
K3QsaCk6dX0saC5vZmZzZXQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGM9
K3QsaCk6Y30saH12YXIgJHQ9e3ZhbHVlOigpPT57fX07ZnVuY3Rpb24gRHQoKXtmb3IodmFyIHQs
bj0wLGU9YXJndW1lbnRzLmxlbmd0aCxyPXt9O248ZTsrK24pe2lmKCEodD1hcmd1bWVudHNbbl0r
IiIpfHx0IGluIHJ8fC9bXHMuXS8udGVzdCh0KSl0aHJvdyBuZXcgRXJyb3IoImlsbGVnYWwgdHlw
ZTogIit0KTtyW3RdPVtdfXJldHVybiBuZXcgUnQocil9ZnVuY3Rpb24gUnQodCl7dGhpcy5fPXR9
ZnVuY3Rpb24gRnQodCxuKXtmb3IodmFyIGUscj0wLGk9dC5sZW5ndGg7cjxpOysrcilpZigoZT10
W3JdKS5uYW1lPT09bilyZXR1cm4gZS52YWx1ZX1mdW5jdGlvbiBxdCh0LG4sZSl7Zm9yKHZhciBy
PTAsaT10Lmxlbmd0aDtyPGk7KytyKWlmKHRbcl0ubmFtZT09PW4pe3Rbcl09JHQsdD10LnNsaWNl
KDAscikuY29uY2F0KHQuc2xpY2UocisxKSk7YnJlYWt9cmV0dXJuIG51bGwhPWUmJnQucHVzaCh7
bmFtZTpuLHZhbHVlOmV9KSx0fVJ0LnByb3RvdHlwZT1EdC5wcm90b3R5cGU9e2NvbnN0cnVjdG9y
OlJ0LG9uOmZ1bmN0aW9uKHQsbil7dmFyIGUscixpPXRoaXMuXyxvPShyPWksKHQrIiIpLnRyaW0o
KS5zcGxpdCgvXnxccysvKS5tYXAoKGZ1bmN0aW9uKHQpe3ZhciBuPSIiLGU9dC5pbmRleE9mKCIu
Iik7aWYoZT49MCYmKG49dC5zbGljZShlKzEpLHQ9dC5zbGljZSgwLGUpKSx0JiYhci5oYXNPd25Q
cm9wZXJ0eSh0KSl0aHJvdyBuZXcgRXJyb3IoInVua25vd24gdHlwZTogIit0KTtyZXR1cm57dHlw
ZTp0LG5hbWU6bn19KSkpLGE9LTEsdT1vLmxlbmd0aDtpZighKGFyZ3VtZW50cy5sZW5ndGg8Mikp
e2lmKG51bGwhPW4mJiJmdW5jdGlvbiIhPXR5cGVvZiBuKXRocm93IG5ldyBFcnJvcigiaW52YWxp
ZCBjYWxsYmFjazogIituKTtmb3IoOysrYTx1OylpZihlPSh0PW9bYV0pLnR5cGUpaVtlXT1xdChp
W2VdLHQubmFtZSxuKTtlbHNlIGlmKG51bGw9PW4pZm9yKGUgaW4gaSlpW2VdPXF0KGlbZV0sdC5u
YW1lLG51bGwpO3JldHVybiB0aGlzfWZvcig7KythPHU7KWlmKChlPSh0PW9bYV0pLnR5cGUpJiYo
ZT1GdChpW2VdLHQubmFtZSkpKXJldHVybiBlfSxjb3B5OmZ1bmN0aW9uKCl7dmFyIHQ9e30sbj10
aGlzLl87Zm9yKHZhciBlIGluIG4pdFtlXT1uW2VdLnNsaWNlKCk7cmV0dXJuIG5ldyBSdCh0KX0s
Y2FsbDpmdW5jdGlvbih0LG4pe2lmKChlPWFyZ3VtZW50cy5sZW5ndGgtMik+MClmb3IodmFyIGUs
cixpPW5ldyBBcnJheShlKSxvPTA7bzxlOysrbylpW29dPWFyZ3VtZW50c1tvKzJdO2lmKCF0aGlz
Ll8uaGFzT3duUHJvcGVydHkodCkpdGhyb3cgbmV3IEVycm9yKCJ1bmtub3duIHR5cGU6ICIrdCk7
Zm9yKG89MCxlPShyPXRoaXMuX1t0XSkubGVuZ3RoO288ZTsrK28pcltvXS52YWx1ZS5hcHBseShu
LGkpfSxhcHBseTpmdW5jdGlvbih0LG4sZSl7aWYoIXRoaXMuXy5oYXNPd25Qcm9wZXJ0eSh0KSl0
aHJvdyBuZXcgRXJyb3IoInVua25vd24gdHlwZTogIit0KTtmb3IodmFyIHI9dGhpcy5fW3RdLGk9
MCxvPXIubGVuZ3RoO2k8bzsrK2kpcltpXS52YWx1ZS5hcHBseShuLGUpfX07dmFyIFV0PSJodHRw
Oi8vd3d3LnczLm9yZy8xOTk5L3hodG1sIixJdD17c3ZnOiJodHRwOi8vd3d3LnczLm9yZy8yMDAw
L3N2ZyIseGh0bWw6VXQseGxpbms6Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiLHhtbDoi
aHR0cDovL3d3dy53My5vcmcvWE1MLzE5OTgvbmFtZXNwYWNlIix4bWxuczoiaHR0cDovL3d3dy53
My5vcmcvMjAwMC94bWxucy8ifTtmdW5jdGlvbiBPdCh0KXt2YXIgbj10Kz0iIixlPW4uaW5kZXhP
ZigiOiIpO3JldHVybiBlPj0wJiYieG1sbnMiIT09KG49dC5zbGljZSgwLGUpKSYmKHQ9dC5zbGlj
ZShlKzEpKSxJdC5oYXNPd25Qcm9wZXJ0eShuKT97c3BhY2U6SXRbbl0sbG9jYWw6dH06dH1mdW5j
dGlvbiBCdCh0KXtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgbj10aGlzLm93bmVyRG9jdW1lbnQsZT10
aGlzLm5hbWVzcGFjZVVSSTtyZXR1cm4gZT09PVV0JiZuLmRvY3VtZW50RWxlbWVudC5uYW1lc3Bh
Y2VVUkk9PT1VdD9uLmNyZWF0ZUVsZW1lbnQodCk6bi5jcmVhdGVFbGVtZW50TlMoZSx0KX19ZnVu
Y3Rpb24gWXQodCl7cmV0dXJuIGZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMub3duZXJEb2N1bWVudC5j
cmVhdGVFbGVtZW50TlModC5zcGFjZSx0LmxvY2FsKX19ZnVuY3Rpb24gTHQodCl7dmFyIG49T3Qo
dCk7cmV0dXJuKG4ubG9jYWw/WXQ6QnQpKG4pfWZ1bmN0aW9uIGp0KCl7fWZ1bmN0aW9uIEh0KHQp
e3JldHVybiBudWxsPT10P2p0OmZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMucXVlcnlTZWxlY3Rvcih0
KX19ZnVuY3Rpb24gWHQodCl7cmV0dXJuIG51bGw9PXQ/W106QXJyYXkuaXNBcnJheSh0KT90OkFy
cmF5LmZyb20odCl9ZnVuY3Rpb24gR3QoKXtyZXR1cm5bXX1mdW5jdGlvbiBWdCh0KXtyZXR1cm4g
bnVsbD09dD9HdDpmdW5jdGlvbigpe3JldHVybiB0aGlzLnF1ZXJ5U2VsZWN0b3JBbGwodCl9fWZ1
bmN0aW9uIFd0KHQpe3JldHVybiBmdW5jdGlvbigpe3JldHVybiB0aGlzLm1hdGNoZXModCl9fWZ1
bmN0aW9uIFp0KHQpe3JldHVybiBmdW5jdGlvbihuKXtyZXR1cm4gbi5tYXRjaGVzKHQpfX12YXIg
S3Q9QXJyYXkucHJvdG90eXBlLmZpbmQ7ZnVuY3Rpb24gUXQoKXtyZXR1cm4gdGhpcy5maXJzdEVs
ZW1lbnRDaGlsZH12YXIgSnQ9QXJyYXkucHJvdG90eXBlLmZpbHRlcjtmdW5jdGlvbiB0bigpe3Jl
dHVybiBBcnJheS5mcm9tKHRoaXMuY2hpbGRyZW4pfWZ1bmN0aW9uIG5uKHQpe3JldHVybiBuZXcg
QXJyYXkodC5sZW5ndGgpfWZ1bmN0aW9uIGVuKHQsbil7dGhpcy5vd25lckRvY3VtZW50PXQub3du
ZXJEb2N1bWVudCx0aGlzLm5hbWVzcGFjZVVSST10Lm5hbWVzcGFjZVVSSSx0aGlzLl9uZXh0PW51
bGwsdGhpcy5fcGFyZW50PXQsdGhpcy5fX2RhdGFfXz1ufWZ1bmN0aW9uIHJuKHQsbixlLHIsaSxv
KXtmb3IodmFyIGEsdT0wLGM9bi5sZW5ndGgsZj1vLmxlbmd0aDt1PGY7Kyt1KShhPW5bdV0pPyhh
Ll9fZGF0YV9fPW9bdV0sclt1XT1hKTplW3VdPW5ldyBlbih0LG9bdV0pO2Zvcig7dTxjOysrdSko
YT1uW3VdKSYmKGlbdV09YSl9ZnVuY3Rpb24gb24odCxuLGUscixpLG8sYSl7dmFyIHUsYyxmLHM9
bmV3IE1hcCxsPW4ubGVuZ3RoLGg9by5sZW5ndGgsZD1uZXcgQXJyYXkobCk7Zm9yKHU9MDt1PGw7
Kyt1KShjPW5bdV0pJiYoZFt1XT1mPWEuY2FsbChjLGMuX19kYXRhX18sdSxuKSsiIixzLmhhcyhm
KT9pW3VdPWM6cy5zZXQoZixjKSk7Zm9yKHU9MDt1PGg7Kyt1KWY9YS5jYWxsKHQsb1t1XSx1LG8p
KyIiLChjPXMuZ2V0KGYpKT8oclt1XT1jLGMuX19kYXRhX189b1t1XSxzLmRlbGV0ZShmKSk6ZVt1
XT1uZXcgZW4odCxvW3VdKTtmb3IodT0wO3U8bDsrK3UpKGM9blt1XSkmJnMuZ2V0KGRbdV0pPT09
YyYmKGlbdV09Yyl9ZnVuY3Rpb24gYW4odCl7cmV0dXJuIHQuX19kYXRhX199ZnVuY3Rpb24gdW4o
dCl7cmV0dXJuIm9iamVjdCI9PXR5cGVvZiB0JiYibGVuZ3RoImluIHQ/dDpBcnJheS5mcm9tKHQp
fWZ1bmN0aW9uIGNuKHQsbil7cmV0dXJuIHQ8bj8tMTp0Pm4/MTp0Pj1uPzA6TmFOfWZ1bmN0aW9u
IGZuKHQpe3JldHVybiBmdW5jdGlvbigpe3RoaXMucmVtb3ZlQXR0cmlidXRlKHQpfX1mdW5jdGlv
biBzbih0KXtyZXR1cm4gZnVuY3Rpb24oKXt0aGlzLnJlbW92ZUF0dHJpYnV0ZU5TKHQuc3BhY2Us
dC5sb2NhbCl9fWZ1bmN0aW9uIGxuKHQsbil7cmV0dXJuIGZ1bmN0aW9uKCl7dGhpcy5zZXRBdHRy
aWJ1dGUodCxuKX19ZnVuY3Rpb24gaG4odCxuKXtyZXR1cm4gZnVuY3Rpb24oKXt0aGlzLnNldEF0
dHJpYnV0ZU5TKHQuc3BhY2UsdC5sb2NhbCxuKX19ZnVuY3Rpb24gZG4odCxuKXtyZXR1cm4gZnVu
Y3Rpb24oKXt2YXIgZT1uLmFwcGx5KHRoaXMsYXJndW1lbnRzKTtudWxsPT1lP3RoaXMucmVtb3Zl
QXR0cmlidXRlKHQpOnRoaXMuc2V0QXR0cmlidXRlKHQsZSl9fWZ1bmN0aW9uIHBuKHQsbil7cmV0
dXJuIGZ1bmN0aW9uKCl7dmFyIGU9bi5hcHBseSh0aGlzLGFyZ3VtZW50cyk7bnVsbD09ZT90aGlz
LnJlbW92ZUF0dHJpYnV0ZU5TKHQuc3BhY2UsdC5sb2NhbCk6dGhpcy5zZXRBdHRyaWJ1dGVOUyh0
LnNwYWNlLHQubG9jYWwsZSl9fWZ1bmN0aW9uIGduKHQpe3JldHVybiB0Lm93bmVyRG9jdW1lbnQm
JnQub3duZXJEb2N1bWVudC5kZWZhdWx0Vmlld3x8dC5kb2N1bWVudCYmdHx8dC5kZWZhdWx0Vmll
d31mdW5jdGlvbiB5bih0KXtyZXR1cm4gZnVuY3Rpb24oKXt0aGlzLnN0eWxlLnJlbW92ZVByb3Bl
cnR5KHQpfX1mdW5jdGlvbiB2bih0LG4sZSl7cmV0dXJuIGZ1bmN0aW9uKCl7dGhpcy5zdHlsZS5z
ZXRQcm9wZXJ0eSh0LG4sZSl9fWZ1bmN0aW9uIF9uKHQsbixlKXtyZXR1cm4gZnVuY3Rpb24oKXt2
YXIgcj1uLmFwcGx5KHRoaXMsYXJndW1lbnRzKTtudWxsPT1yP3RoaXMuc3R5bGUucmVtb3ZlUHJv
cGVydHkodCk6dGhpcy5zdHlsZS5zZXRQcm9wZXJ0eSh0LHIsZSl9fWZ1bmN0aW9uIGJuKHQsbil7
cmV0dXJuIHQuc3R5bGUuZ2V0UHJvcGVydHlWYWx1ZShuKXx8Z24odCkuZ2V0Q29tcHV0ZWRTdHls
ZSh0LG51bGwpLmdldFByb3BlcnR5VmFsdWUobil9ZnVuY3Rpb24gbW4odCl7cmV0dXJuIGZ1bmN0
aW9uKCl7ZGVsZXRlIHRoaXNbdF19fWZ1bmN0aW9uIHhuKHQsbil7cmV0dXJuIGZ1bmN0aW9uKCl7
dGhpc1t0XT1ufX1mdW5jdGlvbiB3bih0LG4pe3JldHVybiBmdW5jdGlvbigpe3ZhciBlPW4uYXBw
bHkodGhpcyxhcmd1bWVudHMpO251bGw9PWU/ZGVsZXRlIHRoaXNbdF06dGhpc1t0XT1lfX1mdW5j
dGlvbiBNbih0KXtyZXR1cm4gdC50cmltKCkuc3BsaXQoL158XHMrLyl9ZnVuY3Rpb24gVG4odCl7
cmV0dXJuIHQuY2xhc3NMaXN0fHxuZXcgQW4odCl9ZnVuY3Rpb24gQW4odCl7dGhpcy5fbm9kZT10
LHRoaXMuX25hbWVzPU1uKHQuZ2V0QXR0cmlidXRlKCJjbGFzcyIpfHwiIil9ZnVuY3Rpb24gU24o
dCxuKXtmb3IodmFyIGU9VG4odCkscj0tMSxpPW4ubGVuZ3RoOysrcjxpOyllLmFkZChuW3JdKX1m
dW5jdGlvbiBFbih0LG4pe2Zvcih2YXIgZT1Ubih0KSxyPS0xLGk9bi5sZW5ndGg7KytyPGk7KWUu
cmVtb3ZlKG5bcl0pfWZ1bmN0aW9uIE5uKHQpe3JldHVybiBmdW5jdGlvbigpe1NuKHRoaXMsdCl9
fWZ1bmN0aW9uIGtuKHQpe3JldHVybiBmdW5jdGlvbigpe0VuKHRoaXMsdCl9fWZ1bmN0aW9uIENu
KHQsbil7cmV0dXJuIGZ1bmN0aW9uKCl7KG4uYXBwbHkodGhpcyxhcmd1bWVudHMpP1NuOkVuKSh0
aGlzLHQpfX1mdW5jdGlvbiBQbigpe3RoaXMudGV4dENvbnRlbnQ9IiJ9ZnVuY3Rpb24gem4odCl7
cmV0dXJuIGZ1bmN0aW9uKCl7dGhpcy50ZXh0Q29udGVudD10fX1mdW5jdGlvbiAkbih0KXtyZXR1
cm4gZnVuY3Rpb24oKXt2YXIgbj10LmFwcGx5KHRoaXMsYXJndW1lbnRzKTt0aGlzLnRleHRDb250
ZW50PW51bGw9PW4/IiI6bn19ZnVuY3Rpb24gRG4oKXt0aGlzLmlubmVySFRNTD0iIn1mdW5jdGlv
biBSbih0KXtyZXR1cm4gZnVuY3Rpb24oKXt0aGlzLmlubmVySFRNTD10fX1mdW5jdGlvbiBGbih0
KXtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgbj10LmFwcGx5KHRoaXMsYXJndW1lbnRzKTt0aGlzLmlu
bmVySFRNTD1udWxsPT1uPyIiOm59fWZ1bmN0aW9uIHFuKCl7dGhpcy5uZXh0U2libGluZyYmdGhp
cy5wYXJlbnROb2RlLmFwcGVuZENoaWxkKHRoaXMpfWZ1bmN0aW9uIFVuKCl7dGhpcy5wcmV2aW91
c1NpYmxpbmcmJnRoaXMucGFyZW50Tm9kZS5pbnNlcnRCZWZvcmUodGhpcyx0aGlzLnBhcmVudE5v
ZGUuZmlyc3RDaGlsZCl9ZnVuY3Rpb24gSW4oKXtyZXR1cm4gbnVsbH1mdW5jdGlvbiBPbigpe3Zh
ciB0PXRoaXMucGFyZW50Tm9kZTt0JiZ0LnJlbW92ZUNoaWxkKHRoaXMpfWZ1bmN0aW9uIEJuKCl7
dmFyIHQ9dGhpcy5jbG9uZU5vZGUoITEpLG49dGhpcy5wYXJlbnROb2RlO3JldHVybiBuP24uaW5z
ZXJ0QmVmb3JlKHQsdGhpcy5uZXh0U2libGluZyk6dH1mdW5jdGlvbiBZbigpe3ZhciB0PXRoaXMu
Y2xvbmVOb2RlKCEwKSxuPXRoaXMucGFyZW50Tm9kZTtyZXR1cm4gbj9uLmluc2VydEJlZm9yZSh0
LHRoaXMubmV4dFNpYmxpbmcpOnR9ZnVuY3Rpb24gTG4odCl7cmV0dXJuIGZ1bmN0aW9uKCl7dmFy
IG49dGhpcy5fX29uO2lmKG4pe2Zvcih2YXIgZSxyPTAsaT0tMSxvPW4ubGVuZ3RoO3I8bzsrK3Ip
ZT1uW3JdLHQudHlwZSYmZS50eXBlIT09dC50eXBlfHxlLm5hbWUhPT10Lm5hbWU/blsrK2ldPWU6
dGhpcy5yZW1vdmVFdmVudExpc3RlbmVyKGUudHlwZSxlLmxpc3RlbmVyLGUub3B0aW9ucyk7Kytp
P24ubGVuZ3RoPWk6ZGVsZXRlIHRoaXMuX19vbn19fWZ1bmN0aW9uIGpuKHQsbixlKXtyZXR1cm4g
ZnVuY3Rpb24oKXt2YXIgcixpPXRoaXMuX19vbixvPWZ1bmN0aW9uKHQpe3JldHVybiBmdW5jdGlv
bihuKXt0LmNhbGwodGhpcyxuLHRoaXMuX19kYXRhX18pfX0obik7aWYoaSlmb3IodmFyIGE9MCx1
PWkubGVuZ3RoO2E8dTsrK2EpaWYoKHI9aVthXSkudHlwZT09PXQudHlwZSYmci5uYW1lPT09dC5u
YW1lKXJldHVybiB0aGlzLnJlbW92ZUV2ZW50TGlzdGVuZXIoci50eXBlLHIubGlzdGVuZXIsci5v
cHRpb25zKSx0aGlzLmFkZEV2ZW50TGlzdGVuZXIoci50eXBlLHIubGlzdGVuZXI9byxyLm9wdGlv
bnM9ZSksdm9pZChyLnZhbHVlPW4pO3RoaXMuYWRkRXZlbnRMaXN0ZW5lcih0LnR5cGUsbyxlKSxy
PXt0eXBlOnQudHlwZSxuYW1lOnQubmFtZSx2YWx1ZTpuLGxpc3RlbmVyOm8sb3B0aW9uczplfSxp
P2kucHVzaChyKTp0aGlzLl9fb249W3JdfX1mdW5jdGlvbiBIbih0LG4sZSl7dmFyIHI9Z24odCks
aT1yLkN1c3RvbUV2ZW50OyJmdW5jdGlvbiI9PXR5cGVvZiBpP2k9bmV3IGkobixlKTooaT1yLmRv
Y3VtZW50LmNyZWF0ZUV2ZW50KCJFdmVudCIpLGU/KGkuaW5pdEV2ZW50KG4sZS5idWJibGVzLGUu
Y2FuY2VsYWJsZSksaS5kZXRhaWw9ZS5kZXRhaWwpOmkuaW5pdEV2ZW50KG4sITEsITEpKSx0LmRp
c3BhdGNoRXZlbnQoaSl9ZnVuY3Rpb24gWG4odCxuKXtyZXR1cm4gZnVuY3Rpb24oKXtyZXR1cm4g
SG4odGhpcyx0LG4pfX1mdW5jdGlvbiBHbih0LG4pe3JldHVybiBmdW5jdGlvbigpe3JldHVybiBI
bih0aGlzLHQsbi5hcHBseSh0aGlzLGFyZ3VtZW50cykpfX1lbi5wcm90b3R5cGU9e2NvbnN0cnVj
dG9yOmVuLGFwcGVuZENoaWxkOmZ1bmN0aW9uKHQpe3JldHVybiB0aGlzLl9wYXJlbnQuaW5zZXJ0
QmVmb3JlKHQsdGhpcy5fbmV4dCl9LGluc2VydEJlZm9yZTpmdW5jdGlvbih0LG4pe3JldHVybiB0
aGlzLl9wYXJlbnQuaW5zZXJ0QmVmb3JlKHQsbil9LHF1ZXJ5U2VsZWN0b3I6ZnVuY3Rpb24odCl7
cmV0dXJuIHRoaXMuX3BhcmVudC5xdWVyeVNlbGVjdG9yKHQpfSxxdWVyeVNlbGVjdG9yQWxsOmZ1
bmN0aW9uKHQpe3JldHVybiB0aGlzLl9wYXJlbnQucXVlcnlTZWxlY3RvckFsbCh0KX19LEFuLnBy
b3RvdHlwZT17YWRkOmZ1bmN0aW9uKHQpe3RoaXMuX25hbWVzLmluZGV4T2YodCk8MCYmKHRoaXMu
X25hbWVzLnB1c2godCksdGhpcy5fbm9kZS5zZXRBdHRyaWJ1dGUoImNsYXNzIix0aGlzLl9uYW1l
cy5qb2luKCIgIikpKX0scmVtb3ZlOmZ1bmN0aW9uKHQpe3ZhciBuPXRoaXMuX25hbWVzLmluZGV4
T2YodCk7bj49MCYmKHRoaXMuX25hbWVzLnNwbGljZShuLDEpLHRoaXMuX25vZGUuc2V0QXR0cmli
dXRlKCJjbGFzcyIsdGhpcy5fbmFtZXMuam9pbigiICIpKSl9LGNvbnRhaW5zOmZ1bmN0aW9uKHQp
e3JldHVybiB0aGlzLl9uYW1lcy5pbmRleE9mKHQpPj0wfX07dmFyIFZuPVtudWxsXTtmdW5jdGlv
biBXbih0LG4pe3RoaXMuX2dyb3Vwcz10LHRoaXMuX3BhcmVudHM9bn1mdW5jdGlvbiBabigpe3Jl
dHVybiBuZXcgV24oW1tkb2N1bWVudC5kb2N1bWVudEVsZW1lbnRdXSxWbil9ZnVuY3Rpb24gS24o
dCl7cmV0dXJuInN0cmluZyI9PXR5cGVvZiB0P25ldyBXbihbW2RvY3VtZW50LnF1ZXJ5U2VsZWN0
b3IodCldXSxbZG9jdW1lbnQuZG9jdW1lbnRFbGVtZW50XSk6bmV3IFduKFtbdF1dLFZuKX1Xbi5w
cm90b3R5cGU9Wm4ucHJvdG90eXBlPXtjb25zdHJ1Y3RvcjpXbixzZWxlY3Q6ZnVuY3Rpb24odCl7
ImZ1bmN0aW9uIiE9dHlwZW9mIHQmJih0PUh0KHQpKTtmb3IodmFyIG49dGhpcy5fZ3JvdXBzLGU9
bi5sZW5ndGgscj1uZXcgQXJyYXkoZSksaT0wO2k8ZTsrK2kpZm9yKHZhciBvLGEsdT1uW2ldLGM9
dS5sZW5ndGgsZj1yW2ldPW5ldyBBcnJheShjKSxzPTA7czxjOysrcykobz11W3NdKSYmKGE9dC5j
YWxsKG8sby5fX2RhdGFfXyxzLHUpKSYmKCJfX2RhdGFfXyJpbiBvJiYoYS5fX2RhdGFfXz1vLl9f
ZGF0YV9fKSxmW3NdPWEpO3JldHVybiBuZXcgV24ocix0aGlzLl9wYXJlbnRzKX0sc2VsZWN0QWxs
OmZ1bmN0aW9uKHQpe3Q9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/ZnVuY3Rpb24odCl7cmV0dXJuIGZ1
bmN0aW9uKCl7cmV0dXJuIFh0KHQuYXBwbHkodGhpcyxhcmd1bWVudHMpKX19KHQpOlZ0KHQpO2Zv
cih2YXIgbj10aGlzLl9ncm91cHMsZT1uLmxlbmd0aCxyPVtdLGk9W10sbz0wO288ZTsrK28pZm9y
KHZhciBhLHU9bltvXSxjPXUubGVuZ3RoLGY9MDtmPGM7KytmKShhPXVbZl0pJiYoci5wdXNoKHQu
Y2FsbChhLGEuX19kYXRhX18sZix1KSksaS5wdXNoKGEpKTtyZXR1cm4gbmV3IFduKHIsaSl9LHNl
bGVjdENoaWxkOmZ1bmN0aW9uKHQpe3JldHVybiB0aGlzLnNlbGVjdChudWxsPT10P1F0OmZ1bmN0
aW9uKHQpe3JldHVybiBmdW5jdGlvbigpe3JldHVybiBLdC5jYWxsKHRoaXMuY2hpbGRyZW4sdCl9
fSgiZnVuY3Rpb24iPT10eXBlb2YgdD90Olp0KHQpKSl9LHNlbGVjdENoaWxkcmVuOmZ1bmN0aW9u
KHQpe3JldHVybiB0aGlzLnNlbGVjdEFsbChudWxsPT10P3RuOmZ1bmN0aW9uKHQpe3JldHVybiBm
dW5jdGlvbigpe3JldHVybiBKdC5jYWxsKHRoaXMuY2hpbGRyZW4sdCl9fSgiZnVuY3Rpb24iPT10
eXBlb2YgdD90Olp0KHQpKSl9LGZpbHRlcjpmdW5jdGlvbih0KXsiZnVuY3Rpb24iIT10eXBlb2Yg
dCYmKHQ9V3QodCkpO2Zvcih2YXIgbj10aGlzLl9ncm91cHMsZT1uLmxlbmd0aCxyPW5ldyBBcnJh
eShlKSxpPTA7aTxlOysraSlmb3IodmFyIG8sYT1uW2ldLHU9YS5sZW5ndGgsYz1yW2ldPVtdLGY9
MDtmPHU7KytmKShvPWFbZl0pJiZ0LmNhbGwobyxvLl9fZGF0YV9fLGYsYSkmJmMucHVzaChvKTty
ZXR1cm4gbmV3IFduKHIsdGhpcy5fcGFyZW50cyl9LGRhdGE6ZnVuY3Rpb24odCxuKXtpZighYXJn
dW1lbnRzLmxlbmd0aClyZXR1cm4gQXJyYXkuZnJvbSh0aGlzLGFuKTt2YXIgZT1uP29uOnJuLHI9
dGhpcy5fcGFyZW50cyxpPXRoaXMuX2dyb3VwczsiZnVuY3Rpb24iIT10eXBlb2YgdCYmKHQ9ZnVu
Y3Rpb24odCl7cmV0dXJuIGZ1bmN0aW9uKCl7cmV0dXJuIHR9fSh0KSk7Zm9yKHZhciBvPWkubGVu
Z3RoLGE9bmV3IEFycmF5KG8pLHU9bmV3IEFycmF5KG8pLGM9bmV3IEFycmF5KG8pLGY9MDtmPG87
KytmKXt2YXIgcz1yW2ZdLGw9aVtmXSxoPWwubGVuZ3RoLGQ9dW4odC5jYWxsKHMscyYmcy5fX2Rh
dGFfXyxmLHIpKSxwPWQubGVuZ3RoLGc9dVtmXT1uZXcgQXJyYXkocCkseT1hW2ZdPW5ldyBBcnJh
eShwKTtlKHMsbCxnLHksY1tmXT1uZXcgQXJyYXkoaCksZCxuKTtmb3IodmFyIHYsXyxiPTAsbT0w
O2I8cDsrK2IpaWYodj1nW2JdKXtmb3IoYj49bSYmKG09YisxKTshKF89eVttXSkmJisrbTxwOyk7
di5fbmV4dD1ffHxudWxsfX1yZXR1cm4oYT1uZXcgV24oYSxyKSkuX2VudGVyPXUsYS5fZXhpdD1j
LGF9LGVudGVyOmZ1bmN0aW9uKCl7cmV0dXJuIG5ldyBXbih0aGlzLl9lbnRlcnx8dGhpcy5fZ3Jv
dXBzLm1hcChubiksdGhpcy5fcGFyZW50cyl9LGV4aXQ6ZnVuY3Rpb24oKXtyZXR1cm4gbmV3IFdu
KHRoaXMuX2V4aXR8fHRoaXMuX2dyb3Vwcy5tYXAobm4pLHRoaXMuX3BhcmVudHMpfSxqb2luOmZ1
bmN0aW9uKHQsbixlKXt2YXIgcj10aGlzLmVudGVyKCksaT10aGlzLG89dGhpcy5leGl0KCk7cmV0
dXJuImZ1bmN0aW9uIj09dHlwZW9mIHQ/KHI9dChyKSkmJihyPXIuc2VsZWN0aW9uKCkpOnI9ci5h
cHBlbmQodCsiIiksbnVsbCE9biYmKGk9bihpKSkmJihpPWkuc2VsZWN0aW9uKCkpLG51bGw9PWU/
by5yZW1vdmUoKTplKG8pLHImJmk/ci5tZXJnZShpKS5vcmRlcigpOml9LG1lcmdlOmZ1bmN0aW9u
KHQpe2Zvcih2YXIgbj10LnNlbGVjdGlvbj90LnNlbGVjdGlvbigpOnQsZT10aGlzLl9ncm91cHMs
cj1uLl9ncm91cHMsaT1lLmxlbmd0aCxvPXIubGVuZ3RoLGE9TWF0aC5taW4oaSxvKSx1PW5ldyBB
cnJheShpKSxjPTA7YzxhOysrYylmb3IodmFyIGYscz1lW2NdLGw9cltjXSxoPXMubGVuZ3RoLGQ9
dVtjXT1uZXcgQXJyYXkoaCkscD0wO3A8aDsrK3ApKGY9c1twXXx8bFtwXSkmJihkW3BdPWYpO2Zv
cig7YzxpOysrYyl1W2NdPWVbY107cmV0dXJuIG5ldyBXbih1LHRoaXMuX3BhcmVudHMpfSxzZWxl
Y3Rpb246ZnVuY3Rpb24oKXtyZXR1cm4gdGhpc30sb3JkZXI6ZnVuY3Rpb24oKXtmb3IodmFyIHQ9
dGhpcy5fZ3JvdXBzLG49LTEsZT10Lmxlbmd0aDsrK248ZTspZm9yKHZhciByLGk9dFtuXSxvPWku
bGVuZ3RoLTEsYT1pW29dOy0tbz49MDspKHI9aVtvXSkmJihhJiY0XnIuY29tcGFyZURvY3VtZW50
UG9zaXRpb24oYSkmJmEucGFyZW50Tm9kZS5pbnNlcnRCZWZvcmUocixhKSxhPXIpO3JldHVybiB0
aGlzfSxzb3J0OmZ1bmN0aW9uKHQpe2Z1bmN0aW9uIG4obixlKXtyZXR1cm4gbiYmZT90KG4uX19k
YXRhX18sZS5fX2RhdGFfXyk6IW4tIWV9dHx8KHQ9Y24pO2Zvcih2YXIgZT10aGlzLl9ncm91cHMs
cj1lLmxlbmd0aCxpPW5ldyBBcnJheShyKSxvPTA7bzxyOysrbyl7Zm9yKHZhciBhLHU9ZVtvXSxj
PXUubGVuZ3RoLGY9aVtvXT1uZXcgQXJyYXkoYykscz0wO3M8YzsrK3MpKGE9dVtzXSkmJihmW3Nd
PWEpO2Yuc29ydChuKX1yZXR1cm4gbmV3IFduKGksdGhpcy5fcGFyZW50cykub3JkZXIoKX0sY2Fs
bDpmdW5jdGlvbigpe3ZhciB0PWFyZ3VtZW50c1swXTtyZXR1cm4gYXJndW1lbnRzWzBdPXRoaXMs
dC5hcHBseShudWxsLGFyZ3VtZW50cyksdGhpc30sbm9kZXM6ZnVuY3Rpb24oKXtyZXR1cm4gQXJy
YXkuZnJvbSh0aGlzKX0sbm9kZTpmdW5jdGlvbigpe2Zvcih2YXIgdD10aGlzLl9ncm91cHMsbj0w
LGU9dC5sZW5ndGg7bjxlOysrbilmb3IodmFyIHI9dFtuXSxpPTAsbz1yLmxlbmd0aDtpPG87Kytp
KXt2YXIgYT1yW2ldO2lmKGEpcmV0dXJuIGF9cmV0dXJuIG51bGx9LHNpemU6ZnVuY3Rpb24oKXts
ZXQgdD0wO2Zvcihjb25zdCBuIG9mIHRoaXMpKyt0O3JldHVybiB0fSxlbXB0eTpmdW5jdGlvbigp
e3JldHVybiF0aGlzLm5vZGUoKX0sZWFjaDpmdW5jdGlvbih0KXtmb3IodmFyIG49dGhpcy5fZ3Jv
dXBzLGU9MCxyPW4ubGVuZ3RoO2U8cjsrK2UpZm9yKHZhciBpLG89bltlXSxhPTAsdT1vLmxlbmd0
aDthPHU7KythKShpPW9bYV0pJiZ0LmNhbGwoaSxpLl9fZGF0YV9fLGEsbyk7cmV0dXJuIHRoaXN9
LGF0dHI6ZnVuY3Rpb24odCxuKXt2YXIgZT1PdCh0KTtpZihhcmd1bWVudHMubGVuZ3RoPDIpe3Zh
ciByPXRoaXMubm9kZSgpO3JldHVybiBlLmxvY2FsP3IuZ2V0QXR0cmlidXRlTlMoZS5zcGFjZSxl
LmxvY2FsKTpyLmdldEF0dHJpYnV0ZShlKX1yZXR1cm4gdGhpcy5lYWNoKChudWxsPT1uP2UubG9j
YWw/c246Zm46ImZ1bmN0aW9uIj09dHlwZW9mIG4/ZS5sb2NhbD9wbjpkbjplLmxvY2FsP2huOmxu
KShlLG4pKX0sc3R5bGU6ZnVuY3Rpb24odCxuLGUpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPjE/
dGhpcy5lYWNoKChudWxsPT1uP3luOiJmdW5jdGlvbiI9PXR5cGVvZiBuP19uOnZuKSh0LG4sbnVs
bD09ZT8iIjplKSk6Ym4odGhpcy5ub2RlKCksdCl9LHByb3BlcnR5OmZ1bmN0aW9uKHQsbil7cmV0
dXJuIGFyZ3VtZW50cy5sZW5ndGg+MT90aGlzLmVhY2goKG51bGw9PW4/bW46ImZ1bmN0aW9uIj09
dHlwZW9mIG4/d246eG4pKHQsbikpOnRoaXMubm9kZSgpW3RdfSxjbGFzc2VkOmZ1bmN0aW9uKHQs
bil7dmFyIGU9TW4odCsiIik7aWYoYXJndW1lbnRzLmxlbmd0aDwyKXtmb3IodmFyIHI9VG4odGhp
cy5ub2RlKCkpLGk9LTEsbz1lLmxlbmd0aDsrK2k8bzspaWYoIXIuY29udGFpbnMoZVtpXSkpcmV0
dXJuITE7cmV0dXJuITB9cmV0dXJuIHRoaXMuZWFjaCgoImZ1bmN0aW9uIj09dHlwZW9mIG4/Q246
bj9ObjprbikoZSxuKSl9LHRleHQ6ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
dGhpcy5lYWNoKG51bGw9PXQ/UG46KCJmdW5jdGlvbiI9PXR5cGVvZiB0PyRuOnpuKSh0KSk6dGhp
cy5ub2RlKCkudGV4dENvbnRlbnR9LGh0bWw6ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/dGhpcy5lYWNoKG51bGw9PXQ/RG46KCJmdW5jdGlvbiI9PXR5cGVvZiB0P0ZuOlJuKSh0
KSk6dGhpcy5ub2RlKCkuaW5uZXJIVE1MfSxyYWlzZTpmdW5jdGlvbigpe3JldHVybiB0aGlzLmVh
Y2gocW4pfSxsb3dlcjpmdW5jdGlvbigpe3JldHVybiB0aGlzLmVhY2goVW4pfSxhcHBlbmQ6ZnVu
Y3Rpb24odCl7dmFyIG49ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpMdCh0KTtyZXR1cm4gdGhpcy5z
ZWxlY3QoKGZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuYXBwZW5kQ2hpbGQobi5hcHBseSh0aGlzLGFy
Z3VtZW50cykpfSkpfSxpbnNlcnQ6ZnVuY3Rpb24odCxuKXt2YXIgZT0iZnVuY3Rpb24iPT10eXBl
b2YgdD90Okx0KHQpLHI9bnVsbD09bj9JbjoiZnVuY3Rpb24iPT10eXBlb2Ygbj9uOkh0KG4pO3Jl
dHVybiB0aGlzLnNlbGVjdCgoZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5pbnNlcnRCZWZvcmUoZS5h
cHBseSh0aGlzLGFyZ3VtZW50cyksci5hcHBseSh0aGlzLGFyZ3VtZW50cyl8fG51bGwpfSkpfSxy
ZW1vdmU6ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5lYWNoKE9uKX0sY2xvbmU6ZnVuY3Rpb24odCl7
cmV0dXJuIHRoaXMuc2VsZWN0KHQ/WW46Qm4pfSxkYXR1bTpmdW5jdGlvbih0KXtyZXR1cm4gYXJn
dW1lbnRzLmxlbmd0aD90aGlzLnByb3BlcnR5KCJfX2RhdGFfXyIsdCk6dGhpcy5ub2RlKCkuX19k
YXRhX199LG9uOmZ1bmN0aW9uKHQsbixlKXt2YXIgcixpLG89ZnVuY3Rpb24odCl7cmV0dXJuIHQu
dHJpbSgpLnNwbGl0KC9efFxzKy8pLm1hcCgoZnVuY3Rpb24odCl7dmFyIG49IiIsZT10LmluZGV4
T2YoIi4iKTtyZXR1cm4gZT49MCYmKG49dC5zbGljZShlKzEpLHQ9dC5zbGljZSgwLGUpKSx7dHlw
ZTp0LG5hbWU6bn19KSl9KHQrIiIpLGE9by5sZW5ndGg7aWYoIShhcmd1bWVudHMubGVuZ3RoPDIp
KXtmb3IodT1uP2puOkxuLHI9MDtyPGE7KytyKXRoaXMuZWFjaCh1KG9bcl0sbixlKSk7cmV0dXJu
IHRoaXN9dmFyIHU9dGhpcy5ub2RlKCkuX19vbjtpZih1KWZvcih2YXIgYyxmPTAscz11Lmxlbmd0
aDtmPHM7KytmKWZvcihyPTAsYz11W2ZdO3I8YTsrK3IpaWYoKGk9b1tyXSkudHlwZT09PWMudHlw
ZSYmaS5uYW1lPT09Yy5uYW1lKXJldHVybiBjLnZhbHVlfSxkaXNwYXRjaDpmdW5jdGlvbih0LG4p
e3JldHVybiB0aGlzLmVhY2goKCJmdW5jdGlvbiI9PXR5cGVvZiBuP0duOlhuKSh0LG4pKX0sW1N5
bWJvbC5pdGVyYXRvcl06ZnVuY3Rpb24qKCl7Zm9yKHZhciB0PXRoaXMuX2dyb3VwcyxuPTAsZT10
Lmxlbmd0aDtuPGU7KytuKWZvcih2YXIgcixpPXRbbl0sbz0wLGE9aS5sZW5ndGg7bzxhOysrbyko
cj1pW29dKSYmKHlpZWxkIHIpfX07dmFyIFFuPTA7ZnVuY3Rpb24gSm4oKXtyZXR1cm4gbmV3IHRl
fWZ1bmN0aW9uIHRlKCl7dGhpcy5fPSJAIisoKytRbikudG9TdHJpbmcoMzYpfWZ1bmN0aW9uIG5l
KHQpe2xldCBuO2Zvcig7bj10LnNvdXJjZUV2ZW50Oyl0PW47cmV0dXJuIHR9ZnVuY3Rpb24gZWUo
dCxuKXtpZih0PW5lKHQpLHZvaWQgMD09PW4mJihuPXQuY3VycmVudFRhcmdldCksbil7dmFyIGU9
bi5vd25lclNWR0VsZW1lbnR8fG47aWYoZS5jcmVhdGVTVkdQb2ludCl7dmFyIHI9ZS5jcmVhdGVT
VkdQb2ludCgpO3JldHVybiByLng9dC5jbGllbnRYLHIueT10LmNsaWVudFksWyhyPXIubWF0cml4
VHJhbnNmb3JtKG4uZ2V0U2NyZWVuQ1RNKCkuaW52ZXJzZSgpKSkueCxyLnldfWlmKG4uZ2V0Qm91
bmRpbmdDbGllbnRSZWN0KXt2YXIgaT1uLmdldEJvdW5kaW5nQ2xpZW50UmVjdCgpO3JldHVyblt0
LmNsaWVudFgtaS5sZWZ0LW4uY2xpZW50TGVmdCx0LmNsaWVudFktaS50b3Atbi5jbGllbnRUb3Bd
fX1yZXR1cm5bdC5wYWdlWCx0LnBhZ2VZXX10ZS5wcm90b3R5cGU9Sm4ucHJvdG90eXBlPXtjb25z
dHJ1Y3Rvcjp0ZSxnZXQ6ZnVuY3Rpb24odCl7Zm9yKHZhciBuPXRoaXMuXzshKG4gaW4gdCk7KWlm
KCEodD10LnBhcmVudE5vZGUpKXJldHVybjtyZXR1cm4gdFtuXX0sc2V0OmZ1bmN0aW9uKHQsbil7
cmV0dXJuIHRbdGhpcy5fXT1ufSxyZW1vdmU6ZnVuY3Rpb24odCl7cmV0dXJuIHRoaXMuXyBpbiB0
JiZkZWxldGUgdFt0aGlzLl9dfSx0b1N0cmluZzpmdW5jdGlvbigpe3JldHVybiB0aGlzLl99fTtj
b25zdCByZT17cGFzc2l2ZTohMX0saWU9e2NhcHR1cmU6ITAscGFzc2l2ZTohMX07ZnVuY3Rpb24g
b2UodCl7dC5zdG9wSW1tZWRpYXRlUHJvcGFnYXRpb24oKX1mdW5jdGlvbiBhZSh0KXt0LnByZXZl
bnREZWZhdWx0KCksdC5zdG9wSW1tZWRpYXRlUHJvcGFnYXRpb24oKX1mdW5jdGlvbiB1ZSh0KXt2
YXIgbj10LmRvY3VtZW50LmRvY3VtZW50RWxlbWVudCxlPUtuKHQpLm9uKCJkcmFnc3RhcnQuZHJh
ZyIsYWUsaWUpOyJvbnNlbGVjdHN0YXJ0ImluIG4/ZS5vbigic2VsZWN0c3RhcnQuZHJhZyIsYWUs
aWUpOihuLl9fbm9zZWxlY3Q9bi5zdHlsZS5Nb3pVc2VyU2VsZWN0LG4uc3R5bGUuTW96VXNlclNl
bGVjdD0ibm9uZSIpfWZ1bmN0aW9uIGNlKHQsbil7dmFyIGU9dC5kb2N1bWVudC5kb2N1bWVudEVs
ZW1lbnQscj1Lbih0KS5vbigiZHJhZ3N0YXJ0LmRyYWciLG51bGwpO24mJihyLm9uKCJjbGljay5k
cmFnIixhZSxpZSksc2V0VGltZW91dCgoZnVuY3Rpb24oKXtyLm9uKCJjbGljay5kcmFnIixudWxs
KX0pLDApKSwib25zZWxlY3RzdGFydCJpbiBlP3Iub24oInNlbGVjdHN0YXJ0LmRyYWciLG51bGwp
OihlLnN0eWxlLk1velVzZXJTZWxlY3Q9ZS5fX25vc2VsZWN0LGRlbGV0ZSBlLl9fbm9zZWxlY3Qp
fXZhciBmZT10PT4oKT0+dDtmdW5jdGlvbiBzZSh0LHtzb3VyY2VFdmVudDpuLHN1YmplY3Q6ZSx0
YXJnZXQ6cixpZGVudGlmaWVyOmksYWN0aXZlOm8seDphLHk6dSxkeDpjLGR5OmYsZGlzcGF0Y2g6
c30pe09iamVjdC5kZWZpbmVQcm9wZXJ0aWVzKHRoaXMse3R5cGU6e3ZhbHVlOnQsZW51bWVyYWJs
ZTohMCxjb25maWd1cmFibGU6ITB9LHNvdXJjZUV2ZW50Ont2YWx1ZTpuLGVudW1lcmFibGU6ITAs
Y29uZmlndXJhYmxlOiEwfSxzdWJqZWN0Ont2YWx1ZTplLGVudW1lcmFibGU6ITAsY29uZmlndXJh
YmxlOiEwfSx0YXJnZXQ6e3ZhbHVlOnIsZW51bWVyYWJsZTohMCxjb25maWd1cmFibGU6ITB9LGlk
ZW50aWZpZXI6e3ZhbHVlOmksZW51bWVyYWJsZTohMCxjb25maWd1cmFibGU6ITB9LGFjdGl2ZTp7
dmFsdWU6byxlbnVtZXJhYmxlOiEwLGNvbmZpZ3VyYWJsZTohMH0seDp7dmFsdWU6YSxlbnVtZXJh
YmxlOiEwLGNvbmZpZ3VyYWJsZTohMH0seTp7dmFsdWU6dSxlbnVtZXJhYmxlOiEwLGNvbmZpZ3Vy
YWJsZTohMH0sZHg6e3ZhbHVlOmMsZW51bWVyYWJsZTohMCxjb25maWd1cmFibGU6ITB9LGR5Ont2
YWx1ZTpmLGVudW1lcmFibGU6ITAsY29uZmlndXJhYmxlOiEwfSxfOnt2YWx1ZTpzfX0pfWZ1bmN0
aW9uIGxlKHQpe3JldHVybiF0LmN0cmxLZXkmJiF0LmJ1dHRvbn1mdW5jdGlvbiBoZSgpe3JldHVy
biB0aGlzLnBhcmVudE5vZGV9ZnVuY3Rpb24gZGUodCxuKXtyZXR1cm4gbnVsbD09bj97eDp0Lngs
eTp0Lnl9Om59ZnVuY3Rpb24gcGUoKXtyZXR1cm4gbmF2aWdhdG9yLm1heFRvdWNoUG9pbnRzfHwi
b250b3VjaHN0YXJ0ImluIHRoaXN9ZnVuY3Rpb24gZ2UodCxuLGUpe3QucHJvdG90eXBlPW4ucHJv
dG90eXBlPWUsZS5jb25zdHJ1Y3Rvcj10fWZ1bmN0aW9uIHllKHQsbil7dmFyIGU9T2JqZWN0LmNy
ZWF0ZSh0LnByb3RvdHlwZSk7Zm9yKHZhciByIGluIG4pZVtyXT1uW3JdO3JldHVybiBlfWZ1bmN0
aW9uIHZlKCl7fXNlLnByb3RvdHlwZS5vbj1mdW5jdGlvbigpe3ZhciB0PXRoaXMuXy5vbi5hcHBs
eSh0aGlzLl8sYXJndW1lbnRzKTtyZXR1cm4gdD09PXRoaXMuXz90aGlzOnR9O3ZhciBfZT0uNyxi
ZT0xL19lLG1lPSJcXHMqKFsrLV0/XFxkKylcXHMqIix4ZT0iXFxzKihbKy1dPyg/OlxcZCpcXC4p
P1xcZCsoPzpbZUVdWystXT9cXGQrKT8pXFxzKiIsd2U9IlxccyooWystXT8oPzpcXGQqXFwuKT9c
XGQrKD86W2VFXVsrLV0/XFxkKyk/KSVcXHMqIixNZT0vXiMoWzAtOWEtZl17Myw4fSkkLyxUZT1u
ZXcgUmVnRXhwKGBecmdiXFwoJHttZX0sJHttZX0sJHttZX1cXCkkYCksQWU9bmV3IFJlZ0V4cChg
XnJnYlxcKCR7d2V9LCR7d2V9LCR7d2V9XFwpJGApLFNlPW5ldyBSZWdFeHAoYF5yZ2JhXFwoJHtt
ZX0sJHttZX0sJHttZX0sJHt4ZX1cXCkkYCksRWU9bmV3IFJlZ0V4cChgXnJnYmFcXCgke3dlfSwk
e3dlfSwke3dlfSwke3hlfVxcKSRgKSxOZT1uZXcgUmVnRXhwKGBeaHNsXFwoJHt4ZX0sJHt3ZX0s
JHt3ZX1cXCkkYCksa2U9bmV3IFJlZ0V4cChgXmhzbGFcXCgke3hlfSwke3dlfSwke3dlfSwke3hl
fVxcKSRgKSxDZT17YWxpY2VibHVlOjE1NzkyMzgzLGFudGlxdWV3aGl0ZToxNjQ0NDM3NSxhcXVh
OjY1NTM1LGFxdWFtYXJpbmU6ODM4ODU2NCxhenVyZToxNTc5NDE3NSxiZWlnZToxNjExOTI2MCxi
aXNxdWU6MTY3NzAyNDQsYmxhY2s6MCxibGFuY2hlZGFsbW9uZDoxNjc3MjA0NSxibHVlOjI1NSxi
bHVldmlvbGV0OjkwNTUyMDIsYnJvd246MTA4MjQyMzQsYnVybHl3b29kOjE0NTk2MjMxLGNhZGV0
Ymx1ZTo2MjY2NTI4LGNoYXJ0cmV1c2U6ODM4ODM1MixjaG9jb2xhdGU6MTM3ODk0NzAsY29yYWw6
MTY3NDQyNzIsY29ybmZsb3dlcmJsdWU6NjU5MTk4MSxjb3Juc2lsazoxNjc3NTM4OCxjcmltc29u
OjE0NDIzMTAwLGN5YW46NjU1MzUsZGFya2JsdWU6MTM5LGRhcmtjeWFuOjM1NzIzLGRhcmtnb2xk
ZW5yb2Q6MTIwOTI5MzksZGFya2dyYXk6MTExMTkwMTcsZGFya2dyZWVuOjI1NjAwLGRhcmtncmV5
OjExMTE5MDE3LGRhcmtraGFraToxMjQzMzI1OSxkYXJrbWFnZW50YTo5MTA5NjQzLGRhcmtvbGl2
ZWdyZWVuOjU1OTc5OTksZGFya29yYW5nZToxNjc0NzUyMCxkYXJrb3JjaGlkOjEwMDQwMDEyLGRh
cmtyZWQ6OTEwOTUwNCxkYXJrc2FsbW9uOjE1MzA4NDEwLGRhcmtzZWFncmVlbjo5NDE5OTE5LGRh
cmtzbGF0ZWJsdWU6NDczNDM0NyxkYXJrc2xhdGVncmF5OjMxMDA0OTUsZGFya3NsYXRlZ3JleToz
MTAwNDk1LGRhcmt0dXJxdW9pc2U6NTI5NDUsZGFya3Zpb2xldDo5Njk5NTM5LGRlZXBwaW5rOjE2
NzE2OTQ3LGRlZXBza3libHVlOjQ5MTUxLGRpbWdyYXk6NjkwODI2NSxkaW1ncmV5OjY5MDgyNjUs
ZG9kZ2VyYmx1ZToyMDAzMTk5LGZpcmVicmljazoxMTY3NDE0NixmbG9yYWx3aGl0ZToxNjc3NTky
MCxmb3Jlc3RncmVlbjoyMjYzODQyLGZ1Y2hzaWE6MTY3MTE5MzUsZ2FpbnNib3JvOjE0NDc0NDYw
LGdob3N0d2hpdGU6MTYzMTY2NzEsZ29sZDoxNjc2NjcyMCxnb2xkZW5yb2Q6MTQzMjkxMjAsZ3Jh
eTo4NDIxNTA0LGdyZWVuOjMyNzY4LGdyZWVueWVsbG93OjExNDAzMDU1LGdyZXk6ODQyMTUwNCxo
b25leWRldzoxNTc5NDE2MCxob3RwaW5rOjE2NzM4NzQwLGluZGlhbnJlZDoxMzQ1ODUyNCxpbmRp
Z286NDkxNTMzMCxpdm9yeToxNjc3NzIwMCxraGFraToxNTc4NzY2MCxsYXZlbmRlcjoxNTEzMjQx
MCxsYXZlbmRlcmJsdXNoOjE2NzczMzY1LGxhd25ncmVlbjo4MTkwOTc2LGxlbW9uY2hpZmZvbjox
Njc3NTg4NSxsaWdodGJsdWU6MTEzOTMyNTQsbGlnaHRjb3JhbDoxNTc2MTUzNixsaWdodGN5YW46
MTQ3NDU1OTksbGlnaHRnb2xkZW5yb2R5ZWxsb3c6MTY0NDgyMTAsbGlnaHRncmF5OjEzODgyMzIz
LGxpZ2h0Z3JlZW46OTQ5ODI1NixsaWdodGdyZXk6MTM4ODIzMjMsbGlnaHRwaW5rOjE2NzU4NDY1
LGxpZ2h0c2FsbW9uOjE2NzUyNzYyLGxpZ2h0c2VhZ3JlZW46MjE0Mjg5MCxsaWdodHNreWJsdWU6
ODkwMDM0NixsaWdodHNsYXRlZ3JheTo3ODMzNzUzLGxpZ2h0c2xhdGVncmV5Ojc4MzM3NTMsbGln
aHRzdGVlbGJsdWU6MTE1ODQ3MzQsbGlnaHR5ZWxsb3c6MTY3NzcxODQsbGltZTo2NTI4MCxsaW1l
Z3JlZW46MzMyOTMzMCxsaW5lbjoxNjQ0NTY3MCxtYWdlbnRhOjE2NzExOTM1LG1hcm9vbjo4Mzg4
NjA4LG1lZGl1bWFxdWFtYXJpbmU6NjczNzMyMixtZWRpdW1ibHVlOjIwNSxtZWRpdW1vcmNoaWQ6
MTIyMTE2NjcsbWVkaXVtcHVycGxlOjk2NjI2ODMsbWVkaXVtc2VhZ3JlZW46Mzk3ODA5NyxtZWRp
dW1zbGF0ZWJsdWU6ODA4Nzc5MCxtZWRpdW1zcHJpbmdncmVlbjo2NDE1NCxtZWRpdW10dXJxdW9p
c2U6NDc3MjMwMCxtZWRpdW12aW9sZXRyZWQ6MTMwNDcxNzMsbWlkbmlnaHRibHVlOjE2NDQ5MTIs
bWludGNyZWFtOjE2MTIxODUwLG1pc3R5cm9zZToxNjc3MDI3Myxtb2NjYXNpbjoxNjc3MDIyOSxu
YXZham93aGl0ZToxNjc2ODY4NSxuYXZ5OjEyOCxvbGRsYWNlOjE2NjQzNTU4LG9saXZlOjg0MjEz
NzYsb2xpdmVkcmFiOjcwNDg3Mzksb3JhbmdlOjE2NzUzOTIwLG9yYW5nZXJlZDoxNjcyOTM0NCxv
cmNoaWQ6MTQzMTU3MzQscGFsZWdvbGRlbnJvZDoxNTY1NzEzMCxwYWxlZ3JlZW46MTAwMjU4ODAs
cGFsZXR1cnF1b2lzZToxMTUyOTk2NixwYWxldmlvbGV0cmVkOjE0MzgxMjAzLHBhcGF5YXdoaXA6
MTY3NzMwNzcscGVhY2hwdWZmOjE2NzY3NjczLHBlcnU6MTM0Njg5OTEscGluazoxNjc2MTAzNSxw
bHVtOjE0NTI0NjM3LHBvd2RlcmJsdWU6MTE1OTE5MTAscHVycGxlOjgzODg3MzYscmViZWNjYXB1
cnBsZTo2Njk3ODgxLHJlZDoxNjcxMTY4MCxyb3N5YnJvd246MTIzNTc1MTkscm95YWxibHVlOjQy
ODY5NDUsc2FkZGxlYnJvd246OTEyNzE4NyxzYWxtb246MTY0MTY4ODIsc2FuZHlicm93bjoxNjAz
Mjg2NCxzZWFncmVlbjozMDUwMzI3LHNlYXNoZWxsOjE2Nzc0NjM4LHNpZW5uYToxMDUwNjc5Nyxz
aWx2ZXI6MTI2MzIyNTYsc2t5Ymx1ZTo4OTAwMzMxLHNsYXRlYmx1ZTo2OTcwMDYxLHNsYXRlZ3Jh
eTo3MzcyOTQ0LHNsYXRlZ3JleTo3MzcyOTQ0LHNub3c6MTY3NzU5MzAsc3ByaW5nZ3JlZW46NjU0
MDcsc3RlZWxibHVlOjQ2MjA5ODAsdGFuOjEzODA4NzgwLHRlYWw6MzI4OTYsdGhpc3RsZToxNDIw
NDg4OCx0b21hdG86MTY3MzcwOTUsdHVycXVvaXNlOjQyNTE4NTYsdmlvbGV0OjE1NjMxMDg2LHdo
ZWF0OjE2MTEzMzMxLHdoaXRlOjE2Nzc3MjE1LHdoaXRlc21va2U6MTYxMTkyODUseWVsbG93OjE2
Nzc2OTYwLHllbGxvd2dyZWVuOjEwMTQ1MDc0fTtmdW5jdGlvbiBQZSgpe3JldHVybiB0aGlzLnJn
YigpLmZvcm1hdEhleCgpfWZ1bmN0aW9uIHplKCl7cmV0dXJuIHRoaXMucmdiKCkuZm9ybWF0Umdi
KCl9ZnVuY3Rpb24gJGUodCl7dmFyIG4sZTtyZXR1cm4gdD0odCsiIikudHJpbSgpLnRvTG93ZXJD
YXNlKCksKG49TWUuZXhlYyh0KSk/KGU9blsxXS5sZW5ndGgsbj1wYXJzZUludChuWzFdLDE2KSw2
PT09ZT9EZShuKTozPT09ZT9uZXcgVWUobj4+OCYxNXxuPj40JjI0MCxuPj40JjE1fDI0MCZuLCgx
NSZuKTw8NHwxNSZuLDEpOjg9PT1lP1JlKG4+PjI0JjI1NSxuPj4xNiYyNTUsbj4+OCYyNTUsKDI1
NSZuKS8yNTUpOjQ9PT1lP1JlKG4+PjEyJjE1fG4+PjgmMjQwLG4+PjgmMTV8bj4+NCYyNDAsbj4+
NCYxNXwyNDAmbiwoKDE1Jm4pPDw0fDE1Jm4pLzI1NSk6bnVsbCk6KG49VGUuZXhlYyh0KSk/bmV3
IFVlKG5bMV0sblsyXSxuWzNdLDEpOihuPUFlLmV4ZWModCkpP25ldyBVZSgyNTUqblsxXS8xMDAs
MjU1Km5bMl0vMTAwLDI1NSpuWzNdLzEwMCwxKToobj1TZS5leGVjKHQpKT9SZShuWzFdLG5bMl0s
blszXSxuWzRdKToobj1FZS5leGVjKHQpKT9SZSgyNTUqblsxXS8xMDAsMjU1Km5bMl0vMTAwLDI1
NSpuWzNdLzEwMCxuWzRdKToobj1OZS5leGVjKHQpKT9qZShuWzFdLG5bMl0vMTAwLG5bM10vMTAw
LDEpOihuPWtlLmV4ZWModCkpP2plKG5bMV0sblsyXS8xMDAsblszXS8xMDAsbls0XSk6Q2UuaGFz
T3duUHJvcGVydHkodCk/RGUoQ2VbdF0pOiJ0cmFuc3BhcmVudCI9PT10P25ldyBVZShOYU4sTmFO
LE5hTiwwKTpudWxsfWZ1bmN0aW9uIERlKHQpe3JldHVybiBuZXcgVWUodD4+MTYmMjU1LHQ+Pjgm
MjU1LDI1NSZ0LDEpfWZ1bmN0aW9uIFJlKHQsbixlLHIpe3JldHVybiByPD0wJiYodD1uPWU9TmFO
KSxuZXcgVWUodCxuLGUscil9ZnVuY3Rpb24gRmUodCl7cmV0dXJuIHQgaW5zdGFuY2VvZiB2ZXx8
KHQ9JGUodCkpLHQ/bmV3IFVlKCh0PXQucmdiKCkpLnIsdC5nLHQuYix0Lm9wYWNpdHkpOm5ldyBV
ZX1mdW5jdGlvbiBxZSh0LG4sZSxyKXtyZXR1cm4gMT09PWFyZ3VtZW50cy5sZW5ndGg/RmUodCk6
bmV3IFVlKHQsbixlLG51bGw9PXI/MTpyKX1mdW5jdGlvbiBVZSh0LG4sZSxyKXt0aGlzLnI9K3Qs
dGhpcy5nPStuLHRoaXMuYj0rZSx0aGlzLm9wYWNpdHk9K3J9ZnVuY3Rpb24gSWUoKXtyZXR1cm5g
IyR7TGUodGhpcy5yKX0ke0xlKHRoaXMuZyl9JHtMZSh0aGlzLmIpfWB9ZnVuY3Rpb24gT2UoKXtj
b25zdCB0PUJlKHRoaXMub3BhY2l0eSk7cmV0dXJuYCR7MT09PXQ/InJnYigiOiJyZ2JhKCJ9JHtZ
ZSh0aGlzLnIpfSwgJHtZZSh0aGlzLmcpfSwgJHtZZSh0aGlzLmIpfSR7MT09PXQ/IikiOmAsICR7
dH0pYH1gfWZ1bmN0aW9uIEJlKHQpe3JldHVybiBpc05hTih0KT8xOk1hdGgubWF4KDAsTWF0aC5t
aW4oMSx0KSl9ZnVuY3Rpb24gWWUodCl7cmV0dXJuIE1hdGgubWF4KDAsTWF0aC5taW4oMjU1LE1h
dGgucm91bmQodCl8fDApKX1mdW5jdGlvbiBMZSh0KXtyZXR1cm4oKHQ9WWUodCkpPDE2PyIwIjoi
IikrdC50b1N0cmluZygxNil9ZnVuY3Rpb24gamUodCxuLGUscil7cmV0dXJuIHI8PTA/dD1uPWU9
TmFOOmU8PTB8fGU+PTE/dD1uPU5hTjpuPD0wJiYodD1OYU4pLG5ldyBHZSh0LG4sZSxyKX1mdW5j
dGlvbiBIZSh0KXtpZih0IGluc3RhbmNlb2YgR2UpcmV0dXJuIG5ldyBHZSh0LmgsdC5zLHQubCx0
Lm9wYWNpdHkpO2lmKHQgaW5zdGFuY2VvZiB2ZXx8KHQ9JGUodCkpLCF0KXJldHVybiBuZXcgR2U7
aWYodCBpbnN0YW5jZW9mIEdlKXJldHVybiB0O3ZhciBuPSh0PXQucmdiKCkpLnIvMjU1LGU9dC5n
LzI1NSxyPXQuYi8yNTUsaT1NYXRoLm1pbihuLGUsciksbz1NYXRoLm1heChuLGUsciksYT1OYU4s
dT1vLWksYz0obytpKS8yO3JldHVybiB1PyhhPW49PT1vPyhlLXIpL3UrNiooZTxyKTplPT09bz8o
ci1uKS91KzI6KG4tZSkvdSs0LHUvPWM8LjU/bytpOjItby1pLGEqPTYwKTp1PWM+MCYmYzwxPzA6
YSxuZXcgR2UoYSx1LGMsdC5vcGFjaXR5KX1mdW5jdGlvbiBYZSh0LG4sZSxyKXtyZXR1cm4gMT09
PWFyZ3VtZW50cy5sZW5ndGg/SGUodCk6bmV3IEdlKHQsbixlLG51bGw9PXI/MTpyKX1mdW5jdGlv
biBHZSh0LG4sZSxyKXt0aGlzLmg9K3QsdGhpcy5zPStuLHRoaXMubD0rZSx0aGlzLm9wYWNpdHk9
K3J9ZnVuY3Rpb24gVmUodCl7cmV0dXJuKHQ9KHR8fDApJTM2MCk8MD90KzM2MDp0fWZ1bmN0aW9u
IFdlKHQpe3JldHVybiBNYXRoLm1heCgwLE1hdGgubWluKDEsdHx8MCkpfWZ1bmN0aW9uIFplKHQs
bixlKXtyZXR1cm4gMjU1Kih0PDYwP24rKGUtbikqdC82MDp0PDE4MD9lOnQ8MjQwP24rKGUtbikq
KDI0MC10KS82MDpuKX1nZSh2ZSwkZSx7Y29weSh0KXtyZXR1cm4gT2JqZWN0LmFzc2lnbihuZXcg
dGhpcy5jb25zdHJ1Y3Rvcix0aGlzLHQpfSxkaXNwbGF5YWJsZSgpe3JldHVybiB0aGlzLnJnYigp
LmRpc3BsYXlhYmxlKCl9LGhleDpQZSxmb3JtYXRIZXg6UGUsZm9ybWF0SGV4ODpmdW5jdGlvbigp
e3JldHVybiB0aGlzLnJnYigpLmZvcm1hdEhleDgoKX0sZm9ybWF0SHNsOmZ1bmN0aW9uKCl7cmV0
dXJuIEhlKHRoaXMpLmZvcm1hdEhzbCgpfSxmb3JtYXRSZ2I6emUsdG9TdHJpbmc6emV9KSxnZShV
ZSxxZSx5ZSh2ZSx7YnJpZ2h0ZXIodCl7cmV0dXJuIHQ9bnVsbD09dD9iZTpNYXRoLnBvdyhiZSx0
KSxuZXcgVWUodGhpcy5yKnQsdGhpcy5nKnQsdGhpcy5iKnQsdGhpcy5vcGFjaXR5KX0sZGFya2Vy
KHQpe3JldHVybiB0PW51bGw9PXQ/X2U6TWF0aC5wb3coX2UsdCksbmV3IFVlKHRoaXMucip0LHRo
aXMuZyp0LHRoaXMuYip0LHRoaXMub3BhY2l0eSl9LHJnYigpe3JldHVybiB0aGlzfSxjbGFtcCgp
e3JldHVybiBuZXcgVWUoWWUodGhpcy5yKSxZZSh0aGlzLmcpLFllKHRoaXMuYiksQmUodGhpcy5v
cGFjaXR5KSl9LGRpc3BsYXlhYmxlKCl7cmV0dXJuLS41PD10aGlzLnImJnRoaXMucjwyNTUuNSYm
LS41PD10aGlzLmcmJnRoaXMuZzwyNTUuNSYmLS41PD10aGlzLmImJnRoaXMuYjwyNTUuNSYmMDw9
dGhpcy5vcGFjaXR5JiZ0aGlzLm9wYWNpdHk8PTF9LGhleDpJZSxmb3JtYXRIZXg6SWUsZm9ybWF0
SGV4ODpmdW5jdGlvbigpe3JldHVybmAjJHtMZSh0aGlzLnIpfSR7TGUodGhpcy5nKX0ke0xlKHRo
aXMuYil9JHtMZSgyNTUqKGlzTmFOKHRoaXMub3BhY2l0eSk/MTp0aGlzLm9wYWNpdHkpKX1gfSxm
b3JtYXRSZ2I6T2UsdG9TdHJpbmc6T2V9KSksZ2UoR2UsWGUseWUodmUse2JyaWdodGVyKHQpe3Jl
dHVybiB0PW51bGw9PXQ/YmU6TWF0aC5wb3coYmUsdCksbmV3IEdlKHRoaXMuaCx0aGlzLnMsdGhp
cy5sKnQsdGhpcy5vcGFjaXR5KX0sZGFya2VyKHQpe3JldHVybiB0PW51bGw9PXQ/X2U6TWF0aC5w
b3coX2UsdCksbmV3IEdlKHRoaXMuaCx0aGlzLnMsdGhpcy5sKnQsdGhpcy5vcGFjaXR5KX0scmdi
KCl7dmFyIHQ9dGhpcy5oJTM2MCszNjAqKHRoaXMuaDwwKSxuPWlzTmFOKHQpfHxpc05hTih0aGlz
LnMpPzA6dGhpcy5zLGU9dGhpcy5sLHI9ZSsoZTwuNT9lOjEtZSkqbixpPTIqZS1yO3JldHVybiBu
ZXcgVWUoWmUodD49MjQwP3QtMjQwOnQrMTIwLGksciksWmUodCxpLHIpLFplKHQ8MTIwP3QrMjQw
OnQtMTIwLGksciksdGhpcy5vcGFjaXR5KX0sY2xhbXAoKXtyZXR1cm4gbmV3IEdlKFZlKHRoaXMu
aCksV2UodGhpcy5zKSxXZSh0aGlzLmwpLEJlKHRoaXMub3BhY2l0eSkpfSxkaXNwbGF5YWJsZSgp
e3JldHVybigwPD10aGlzLnMmJnRoaXMuczw9MXx8aXNOYU4odGhpcy5zKSkmJjA8PXRoaXMubCYm
dGhpcy5sPD0xJiYwPD10aGlzLm9wYWNpdHkmJnRoaXMub3BhY2l0eTw9MX0sZm9ybWF0SHNsKCl7
Y29uc3QgdD1CZSh0aGlzLm9wYWNpdHkpO3JldHVybmAkezE9PT10PyJoc2woIjoiaHNsYSgifSR7
VmUodGhpcy5oKX0sICR7MTAwKldlKHRoaXMucyl9JSwgJHsxMDAqV2UodGhpcy5sKX0lJHsxPT09
dD8iKSI6YCwgJHt0fSlgfWB9fSkpO2NvbnN0IEtlPU1hdGguUEkvMTgwLFFlPTE4MC9NYXRoLlBJ
LEplPS45NjQyMix0cj0xLG5yPS44MjUyMSxlcj00LzI5LHJyPTYvMjksaXI9Mypycipycixvcj1y
cipycipycjtmdW5jdGlvbiBhcih0KXtpZih0IGluc3RhbmNlb2YgY3IpcmV0dXJuIG5ldyBjcih0
LmwsdC5hLHQuYix0Lm9wYWNpdHkpO2lmKHQgaW5zdGFuY2VvZiBncilyZXR1cm4geXIodCk7dCBp
bnN0YW5jZW9mIFVlfHwodD1GZSh0KSk7dmFyIG4sZSxyPWhyKHQuciksaT1ocih0LmcpLG89aHIo
dC5iKSxhPWZyKCguMjIyNTA0NSpyKy43MTY4Nzg2KmkrLjA2MDYxNjkqbykvdHIpO3JldHVybiBy
PT09aSYmaT09PW8/bj1lPWE6KG49ZnIoKC40MzYwNzQ3KnIrLjM4NTA2NDkqaSsuMTQzMDgwNCpv
KS9KZSksZT1mcigoLjAxMzkzMjIqcisuMDk3MTA0NSppKy43MTQxNzMzKm8pL25yKSksbmV3IGNy
KDExNiphLTE2LDUwMCoobi1hKSwyMDAqKGEtZSksdC5vcGFjaXR5KX1mdW5jdGlvbiB1cih0LG4s
ZSxyKXtyZXR1cm4gMT09PWFyZ3VtZW50cy5sZW5ndGg/YXIodCk6bmV3IGNyKHQsbixlLG51bGw9
PXI/MTpyKX1mdW5jdGlvbiBjcih0LG4sZSxyKXt0aGlzLmw9K3QsdGhpcy5hPStuLHRoaXMuYj0r
ZSx0aGlzLm9wYWNpdHk9K3J9ZnVuY3Rpb24gZnIodCl7cmV0dXJuIHQ+b3I/TWF0aC5wb3codCwx
LzMpOnQvaXIrZXJ9ZnVuY3Rpb24gc3IodCl7cmV0dXJuIHQ+cnI/dCp0KnQ6aXIqKHQtZXIpfWZ1
bmN0aW9uIGxyKHQpe3JldHVybiAyNTUqKHQ8PS4wMDMxMzA4PzEyLjkyKnQ6MS4wNTUqTWF0aC5w
b3codCwxLzIuNCktLjA1NSl9ZnVuY3Rpb24gaHIodCl7cmV0dXJuKHQvPTI1NSk8PS4wNDA0NT90
LzEyLjkyOk1hdGgucG93KCh0Ky4wNTUpLzEuMDU1LDIuNCl9ZnVuY3Rpb24gZHIodCl7aWYodCBp
bnN0YW5jZW9mIGdyKXJldHVybiBuZXcgZ3IodC5oLHQuYyx0LmwsdC5vcGFjaXR5KTtpZih0IGlu
c3RhbmNlb2YgY3J8fCh0PWFyKHQpKSwwPT09dC5hJiYwPT09dC5iKXJldHVybiBuZXcgZ3IoTmFO
LDA8dC5sJiZ0Lmw8MTAwPzA6TmFOLHQubCx0Lm9wYWNpdHkpO3ZhciBuPU1hdGguYXRhbjIodC5i
LHQuYSkqUWU7cmV0dXJuIG5ldyBncihuPDA/biszNjA6bixNYXRoLnNxcnQodC5hKnQuYSt0LmIq
dC5iKSx0LmwsdC5vcGFjaXR5KX1mdW5jdGlvbiBwcih0LG4sZSxyKXtyZXR1cm4gMT09PWFyZ3Vt
ZW50cy5sZW5ndGg/ZHIodCk6bmV3IGdyKHQsbixlLG51bGw9PXI/MTpyKX1mdW5jdGlvbiBncih0
LG4sZSxyKXt0aGlzLmg9K3QsdGhpcy5jPStuLHRoaXMubD0rZSx0aGlzLm9wYWNpdHk9K3J9ZnVu
Y3Rpb24geXIodCl7aWYoaXNOYU4odC5oKSlyZXR1cm4gbmV3IGNyKHQubCwwLDAsdC5vcGFjaXR5
KTt2YXIgbj10LmgqS2U7cmV0dXJuIG5ldyBjcih0LmwsTWF0aC5jb3MobikqdC5jLE1hdGguc2lu
KG4pKnQuYyx0Lm9wYWNpdHkpfWdlKGNyLHVyLHllKHZlLHticmlnaHRlcih0KXtyZXR1cm4gbmV3
IGNyKHRoaXMubCsxOCoobnVsbD09dD8xOnQpLHRoaXMuYSx0aGlzLmIsdGhpcy5vcGFjaXR5KX0s
ZGFya2VyKHQpe3JldHVybiBuZXcgY3IodGhpcy5sLTE4KihudWxsPT10PzE6dCksdGhpcy5hLHRo
aXMuYix0aGlzLm9wYWNpdHkpfSxyZ2IoKXt2YXIgdD0odGhpcy5sKzE2KS8xMTYsbj1pc05hTih0
aGlzLmEpP3Q6dCt0aGlzLmEvNTAwLGU9aXNOYU4odGhpcy5iKT90OnQtdGhpcy5iLzIwMDtyZXR1
cm4gbmV3IFVlKGxyKDMuMTMzODU2MSoobj1KZSpzcihuKSktMS42MTY4NjY3Kih0PXRyKnNyKHQp
KS0uNDkwNjE0NiooZT1ucipzcihlKSkpLGxyKC0uOTc4NzY4NCpuKzEuOTE2MTQxNSp0Ky4wMzM0
NTQqZSksbHIoLjA3MTk0NTMqbi0uMjI4OTkxNCp0KzEuNDA1MjQyNyplKSx0aGlzLm9wYWNpdHkp
fX0pKSxnZShncixwcix5ZSh2ZSx7YnJpZ2h0ZXIodCl7cmV0dXJuIG5ldyBncih0aGlzLmgsdGhp
cy5jLHRoaXMubCsxOCoobnVsbD09dD8xOnQpLHRoaXMub3BhY2l0eSl9LGRhcmtlcih0KXtyZXR1
cm4gbmV3IGdyKHRoaXMuaCx0aGlzLmMsdGhpcy5sLTE4KihudWxsPT10PzE6dCksdGhpcy5vcGFj
aXR5KX0scmdiKCl7cmV0dXJuIHlyKHRoaXMpLnJnYigpfX0pKTt2YXIgdnI9LS4xNDg2MSxfcj0x
Ljc4Mjc3LGJyPS0uMjkyMjcsbXI9LS45MDY0OSx4cj0xLjk3Mjk0LHdyPXhyKm1yLE1yPXhyKl9y
LFRyPV9yKmJyLW1yKnZyO2Z1bmN0aW9uIEFyKHQsbixlLHIpe3JldHVybiAxPT09YXJndW1lbnRz
Lmxlbmd0aD9mdW5jdGlvbih0KXtpZih0IGluc3RhbmNlb2YgU3IpcmV0dXJuIG5ldyBTcih0Lmgs
dC5zLHQubCx0Lm9wYWNpdHkpO3QgaW5zdGFuY2VvZiBVZXx8KHQ9RmUodCkpO3ZhciBuPXQuci8y
NTUsZT10LmcvMjU1LHI9dC5iLzI1NSxpPShUcipyK3dyKm4tTXIqZSkvKFRyK3dyLU1yKSxvPXIt
aSxhPSh4ciooZS1pKS1icipvKS9tcix1PU1hdGguc3FydChhKmErbypvKS8oeHIqaSooMS1pKSks
Yz11P01hdGguYXRhbjIoYSxvKSpRZS0xMjA6TmFOO3JldHVybiBuZXcgU3IoYzwwP2MrMzYwOmMs
dSxpLHQub3BhY2l0eSl9KHQpOm5ldyBTcih0LG4sZSxudWxsPT1yPzE6cil9ZnVuY3Rpb24gU3Io
dCxuLGUscil7dGhpcy5oPSt0LHRoaXMucz0rbix0aGlzLmw9K2UsdGhpcy5vcGFjaXR5PStyfWZ1
bmN0aW9uIEVyKHQsbixlLHIsaSl7dmFyIG89dCp0LGE9byp0O3JldHVybigoMS0zKnQrMypvLWEp
Km4rKDQtNipvKzMqYSkqZSsoMSszKnQrMypvLTMqYSkqcithKmkpLzZ9ZnVuY3Rpb24gTnIodCl7
dmFyIG49dC5sZW5ndGgtMTtyZXR1cm4gZnVuY3Rpb24oZSl7dmFyIHI9ZTw9MD9lPTA6ZT49MT8o
ZT0xLG4tMSk6TWF0aC5mbG9vcihlKm4pLGk9dFtyXSxvPXRbcisxXSxhPXI+MD90W3ItMV06Mipp
LW8sdT1yPG4tMT90W3IrMl06MipvLWk7cmV0dXJuIEVyKChlLXIvbikqbixhLGksbyx1KX19ZnVu
Y3Rpb24ga3IodCl7dmFyIG49dC5sZW5ndGg7cmV0dXJuIGZ1bmN0aW9uKGUpe3ZhciByPU1hdGgu
Zmxvb3IoKChlJT0xKTwwPysrZTplKSpuKSxpPXRbKHIrbi0xKSVuXSxvPXRbciVuXSxhPXRbKHIr
MSklbl0sdT10WyhyKzIpJW5dO3JldHVybiBFcigoZS1yL24pKm4saSxvLGEsdSl9fWdlKFNyLEFy
LHllKHZlLHticmlnaHRlcih0KXtyZXR1cm4gdD1udWxsPT10P2JlOk1hdGgucG93KGJlLHQpLG5l
dyBTcih0aGlzLmgsdGhpcy5zLHRoaXMubCp0LHRoaXMub3BhY2l0eSl9LGRhcmtlcih0KXtyZXR1
cm4gdD1udWxsPT10P19lOk1hdGgucG93KF9lLHQpLG5ldyBTcih0aGlzLmgsdGhpcy5zLHRoaXMu
bCp0LHRoaXMub3BhY2l0eSl9LHJnYigpe3ZhciB0PWlzTmFOKHRoaXMuaCk/MDoodGhpcy5oKzEy
MCkqS2Usbj0rdGhpcy5sLGU9aXNOYU4odGhpcy5zKT8wOnRoaXMucypuKigxLW4pLHI9TWF0aC5j
b3ModCksaT1NYXRoLnNpbih0KTtyZXR1cm4gbmV3IFVlKDI1NSoobitlKih2cipyK19yKmkpKSwy
NTUqKG4rZSooYnIqcittcippKSksMjU1KihuK2UqKHhyKnIpKSx0aGlzLm9wYWNpdHkpfX0pKTt2
YXIgQ3I9dD0+KCk9PnQ7ZnVuY3Rpb24gUHIodCxuKXtyZXR1cm4gZnVuY3Rpb24oZSl7cmV0dXJu
IHQrZSpufX1mdW5jdGlvbiB6cih0LG4pe3ZhciBlPW4tdDtyZXR1cm4gZT9Qcih0LGU+MTgwfHxl
PC0xODA/ZS0zNjAqTWF0aC5yb3VuZChlLzM2MCk6ZSk6Q3IoaXNOYU4odCk/bjp0KX1mdW5jdGlv
biAkcih0KXtyZXR1cm4gMT09KHQ9K3QpP0RyOmZ1bmN0aW9uKG4sZSl7cmV0dXJuIGUtbj9mdW5j
dGlvbih0LG4sZSl7cmV0dXJuIHQ9TWF0aC5wb3codCxlKSxuPU1hdGgucG93KG4sZSktdCxlPTEv
ZSxmdW5jdGlvbihyKXtyZXR1cm4gTWF0aC5wb3codCtyKm4sZSl9fShuLGUsdCk6Q3IoaXNOYU4o
bik/ZTpuKX19ZnVuY3Rpb24gRHIodCxuKXt2YXIgZT1uLXQ7cmV0dXJuIGU/UHIodCxlKTpDcihp
c05hTih0KT9uOnQpfXZhciBScj1mdW5jdGlvbiB0KG4pe3ZhciBlPSRyKG4pO2Z1bmN0aW9uIHIo
dCxuKXt2YXIgcj1lKCh0PXFlKHQpKS5yLChuPXFlKG4pKS5yKSxpPWUodC5nLG4uZyksbz1lKHQu
YixuLmIpLGE9RHIodC5vcGFjaXR5LG4ub3BhY2l0eSk7cmV0dXJuIGZ1bmN0aW9uKG4pe3JldHVy
biB0LnI9cihuKSx0Lmc9aShuKSx0LmI9byhuKSx0Lm9wYWNpdHk9YShuKSx0KyIifX1yZXR1cm4g
ci5nYW1tYT10LHJ9KDEpO2Z1bmN0aW9uIEZyKHQpe3JldHVybiBmdW5jdGlvbihuKXt2YXIgZSxy
LGk9bi5sZW5ndGgsbz1uZXcgQXJyYXkoaSksYT1uZXcgQXJyYXkoaSksdT1uZXcgQXJyYXkoaSk7
Zm9yKGU9MDtlPGk7KytlKXI9cWUobltlXSksb1tlXT1yLnJ8fDAsYVtlXT1yLmd8fDAsdVtlXT1y
LmJ8fDA7cmV0dXJuIG89dChvKSxhPXQoYSksdT10KHUpLHIub3BhY2l0eT0xLGZ1bmN0aW9uKHQp
e3JldHVybiByLnI9byh0KSxyLmc9YSh0KSxyLmI9dSh0KSxyKyIifX19dmFyIHFyPUZyKE5yKSxV
cj1Gcihrcik7ZnVuY3Rpb24gSXIodCxuKXtufHwobj1bXSk7dmFyIGUscj10P01hdGgubWluKG4u
bGVuZ3RoLHQubGVuZ3RoKTowLGk9bi5zbGljZSgpO3JldHVybiBmdW5jdGlvbihvKXtmb3IoZT0w
O2U8cjsrK2UpaVtlXT10W2VdKigxLW8pK25bZV0qbztyZXR1cm4gaX19ZnVuY3Rpb24gT3IodCl7
cmV0dXJuIEFycmF5QnVmZmVyLmlzVmlldyh0KSYmISh0IGluc3RhbmNlb2YgRGF0YVZpZXcpfWZ1
bmN0aW9uIEJyKHQsbil7dmFyIGUscj1uP24ubGVuZ3RoOjAsaT10P01hdGgubWluKHIsdC5sZW5n
dGgpOjAsbz1uZXcgQXJyYXkoaSksYT1uZXcgQXJyYXkocik7Zm9yKGU9MDtlPGk7KytlKW9bZV09
VnIodFtlXSxuW2VdKTtmb3IoO2U8cjsrK2UpYVtlXT1uW2VdO3JldHVybiBmdW5jdGlvbih0KXtm
b3IoZT0wO2U8aTsrK2UpYVtlXT1vW2VdKHQpO3JldHVybiBhfX1mdW5jdGlvbiBZcih0LG4pe3Zh
ciBlPW5ldyBEYXRlO3JldHVybiB0PSt0LG49K24sZnVuY3Rpb24ocil7cmV0dXJuIGUuc2V0VGlt
ZSh0KigxLXIpK24qciksZX19ZnVuY3Rpb24gTHIodCxuKXtyZXR1cm4gdD0rdCxuPStuLGZ1bmN0
aW9uKGUpe3JldHVybiB0KigxLWUpK24qZX19ZnVuY3Rpb24ganIodCxuKXt2YXIgZSxyPXt9LGk9
e307Zm9yKGUgaW4gbnVsbCE9PXQmJiJvYmplY3QiPT10eXBlb2YgdHx8KHQ9e30pLG51bGwhPT1u
JiYib2JqZWN0Ij09dHlwZW9mIG58fChuPXt9KSxuKWUgaW4gdD9yW2VdPVZyKHRbZV0sbltlXSk6
aVtlXT1uW2VdO3JldHVybiBmdW5jdGlvbih0KXtmb3IoZSBpbiByKWlbZV09cltlXSh0KTtyZXR1
cm4gaX19dmFyIEhyPS9bLStdPyg/OlxkK1wuP1xkKnxcLj9cZCspKD86W2VFXVstK10/XGQrKT8v
ZyxYcj1uZXcgUmVnRXhwKEhyLnNvdXJjZSwiZyIpO2Z1bmN0aW9uIEdyKHQsbil7dmFyIGUscixp
LG89SHIubGFzdEluZGV4PVhyLmxhc3RJbmRleD0wLGE9LTEsdT1bXSxjPVtdO2Zvcih0Kz0iIixu
Kz0iIjsoZT1Ici5leGVjKHQpKSYmKHI9WHIuZXhlYyhuKSk7KShpPXIuaW5kZXgpPm8mJihpPW4u
c2xpY2UobyxpKSx1W2FdP3VbYV0rPWk6dVsrK2FdPWkpLChlPWVbMF0pPT09KHI9clswXSk/dVth
XT91W2FdKz1yOnVbKythXT1yOih1WysrYV09bnVsbCxjLnB1c2goe2k6YSx4OkxyKGUscil9KSks
bz1Yci5sYXN0SW5kZXg7cmV0dXJuIG88bi5sZW5ndGgmJihpPW4uc2xpY2UobyksdVthXT91W2Fd
Kz1pOnVbKythXT1pKSx1Lmxlbmd0aDwyP2NbMF0/ZnVuY3Rpb24odCl7cmV0dXJuIGZ1bmN0aW9u
KG4pe3JldHVybiB0KG4pKyIifX0oY1swXS54KTpmdW5jdGlvbih0KXtyZXR1cm4gZnVuY3Rpb24o
KXtyZXR1cm4gdH19KG4pOihuPWMubGVuZ3RoLGZ1bmN0aW9uKHQpe2Zvcih2YXIgZSxyPTA7cjxu
Oysrcil1WyhlPWNbcl0pLmldPWUueCh0KTtyZXR1cm4gdS5qb2luKCIiKX0pfWZ1bmN0aW9uIFZy
KHQsbil7dmFyIGUscj10eXBlb2YgbjtyZXR1cm4gbnVsbD09bnx8ImJvb2xlYW4iPT09cj9Dcihu
KTooIm51bWJlciI9PT1yP0xyOiJzdHJpbmciPT09cj8oZT0kZShuKSk/KG49ZSxScik6R3I6biBp
bnN0YW5jZW9mICRlP1JyOm4gaW5zdGFuY2VvZiBEYXRlP1lyOk9yKG4pP0lyOkFycmF5LmlzQXJy
YXkobik/QnI6ImZ1bmN0aW9uIiE9dHlwZW9mIG4udmFsdWVPZiYmImZ1bmN0aW9uIiE9dHlwZW9m
IG4udG9TdHJpbmd8fGlzTmFOKG4pP2pyOkxyKSh0LG4pfWZ1bmN0aW9uIFdyKHQsbil7cmV0dXJu
IHQ9K3Qsbj0rbixmdW5jdGlvbihlKXtyZXR1cm4gTWF0aC5yb3VuZCh0KigxLWUpK24qZSl9fXZh
ciBacixLcj0xODAvTWF0aC5QSSxRcj17dHJhbnNsYXRlWDowLHRyYW5zbGF0ZVk6MCxyb3RhdGU6
MCxza2V3WDowLHNjYWxlWDoxLHNjYWxlWToxfTtmdW5jdGlvbiBKcih0LG4sZSxyLGksbyl7dmFy
IGEsdSxjO3JldHVybihhPU1hdGguc3FydCh0KnQrbipuKSkmJih0Lz1hLG4vPWEpLChjPXQqZStu
KnIpJiYoZS09dCpjLHItPW4qYyksKHU9TWF0aC5zcXJ0KGUqZStyKnIpKSYmKGUvPXUsci89dSxj
Lz11KSx0KnI8biplJiYodD0tdCxuPS1uLGM9LWMsYT0tYSkse3RyYW5zbGF0ZVg6aSx0cmFuc2xh
dGVZOm8scm90YXRlOk1hdGguYXRhbjIobix0KSpLcixza2V3WDpNYXRoLmF0YW4oYykqS3Isc2Nh
bGVYOmEsc2NhbGVZOnV9fWZ1bmN0aW9uIHRpKHQsbixlLHIpe2Z1bmN0aW9uIGkodCl7cmV0dXJu
IHQubGVuZ3RoP3QucG9wKCkrIiAiOiIifXJldHVybiBmdW5jdGlvbihvLGEpe3ZhciB1PVtdLGM9
W107cmV0dXJuIG89dChvKSxhPXQoYSksZnVuY3Rpb24odCxyLGksbyxhLHUpe2lmKHQhPT1pfHxy
IT09byl7dmFyIGM9YS5wdXNoKCJ0cmFuc2xhdGUoIixudWxsLG4sbnVsbCxlKTt1LnB1c2goe2k6
Yy00LHg6THIodCxpKX0se2k6Yy0yLHg6THIocixvKX0pfWVsc2UoaXx8bykmJmEucHVzaCgidHJh
bnNsYXRlKCIraStuK28rZSl9KG8udHJhbnNsYXRlWCxvLnRyYW5zbGF0ZVksYS50cmFuc2xhdGVY
LGEudHJhbnNsYXRlWSx1LGMpLGZ1bmN0aW9uKHQsbixlLG8pe3QhPT1uPyh0LW4+MTgwP24rPTM2
MDpuLXQ+MTgwJiYodCs9MzYwKSxvLnB1c2goe2k6ZS5wdXNoKGkoZSkrInJvdGF0ZSgiLG51bGws
ciktMix4OkxyKHQsbil9KSk6biYmZS5wdXNoKGkoZSkrInJvdGF0ZSgiK24rcil9KG8ucm90YXRl
LGEucm90YXRlLHUsYyksZnVuY3Rpb24odCxuLGUsbyl7dCE9PW4/by5wdXNoKHtpOmUucHVzaChp
KGUpKyJza2V3WCgiLG51bGwsciktMix4OkxyKHQsbil9KTpuJiZlLnB1c2goaShlKSsic2tld1go
IituK3IpfShvLnNrZXdYLGEuc2tld1gsdSxjKSxmdW5jdGlvbih0LG4sZSxyLG8sYSl7aWYodCE9
PWV8fG4hPT1yKXt2YXIgdT1vLnB1c2goaShvKSsic2NhbGUoIixudWxsLCIsIixudWxsLCIpIik7
YS5wdXNoKHtpOnUtNCx4OkxyKHQsZSl9LHtpOnUtMix4OkxyKG4scil9KX1lbHNlIDE9PT1lJiYx
PT09cnx8by5wdXNoKGkobykrInNjYWxlKCIrZSsiLCIrcisiKSIpfShvLnNjYWxlWCxvLnNjYWxl
WSxhLnNjYWxlWCxhLnNjYWxlWSx1LGMpLG89YT1udWxsLGZ1bmN0aW9uKHQpe2Zvcih2YXIgbixl
PS0xLHI9Yy5sZW5ndGg7KytlPHI7KXVbKG49Y1tlXSkuaV09bi54KHQpO3JldHVybiB1LmpvaW4o
IiIpfX19dmFyIG5pPXRpKChmdW5jdGlvbih0KXtjb25zdCBuPW5ldygiZnVuY3Rpb24iPT10eXBl
b2YgRE9NTWF0cml4P0RPTU1hdHJpeDpXZWJLaXRDU1NNYXRyaXgpKHQrIiIpO3JldHVybiBuLmlz
SWRlbnRpdHk/UXI6SnIobi5hLG4uYixuLmMsbi5kLG4uZSxuLmYpfSksInB4LCAiLCJweCkiLCJk
ZWcpIiksZWk9dGkoKGZ1bmN0aW9uKHQpe3JldHVybiBudWxsPT10P1FyOihacnx8KFpyPWRvY3Vt
ZW50LmNyZWF0ZUVsZW1lbnROUygiaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciLCJnIikpLFpy
LnNldEF0dHJpYnV0ZSgidHJhbnNmb3JtIix0KSwodD1aci50cmFuc2Zvcm0uYmFzZVZhbC5jb25z
b2xpZGF0ZSgpKT9KcigodD10Lm1hdHJpeCkuYSx0LmIsdC5jLHQuZCx0LmUsdC5mKTpRcil9KSwi
LCAiLCIpIiwiKSIpO2Z1bmN0aW9uIHJpKHQpe3JldHVybigodD1NYXRoLmV4cCh0KSkrMS90KS8y
fXZhciBpaT1mdW5jdGlvbiB0KG4sZSxyKXtmdW5jdGlvbiBpKHQsaSl7dmFyIG8sYSx1PXRbMF0s
Yz10WzFdLGY9dFsyXSxzPWlbMF0sbD1pWzFdLGg9aVsyXSxkPXMtdSxwPWwtYyxnPWQqZCtwKnA7
aWYoZzwxZS0xMilhPU1hdGgubG9nKGgvZikvbixvPWZ1bmN0aW9uKHQpe3JldHVyblt1K3QqZCxj
K3QqcCxmKk1hdGguZXhwKG4qdCphKV19O2Vsc2V7dmFyIHk9TWF0aC5zcXJ0KGcpLHY9KGgqaC1m
KmYrcipnKS8oMipmKmUqeSksXz0oaCpoLWYqZi1yKmcpLygyKmgqZSp5KSxiPU1hdGgubG9nKE1h
dGguc3FydCh2KnYrMSktdiksbT1NYXRoLmxvZyhNYXRoLnNxcnQoXypfKzEpLV8pO2E9KG0tYikv
bixvPWZ1bmN0aW9uKHQpe3ZhciByPXQqYSxpPXJpKGIpLG89Zi8oZSp5KSooaSpmdW5jdGlvbih0
KXtyZXR1cm4oKHQ9TWF0aC5leHAoMip0KSktMSkvKHQrMSl9KG4qcitiKS1mdW5jdGlvbih0KXty
ZXR1cm4oKHQ9TWF0aC5leHAodCkpLTEvdCkvMn0oYikpO3JldHVyblt1K28qZCxjK28qcCxmKmkv
cmkobipyK2IpXX19cmV0dXJuIG8uZHVyYXRpb249MWUzKmEqbi9NYXRoLlNRUlQyLG99cmV0dXJu
IGkucmhvPWZ1bmN0aW9uKG4pe3ZhciBlPU1hdGgubWF4KC4wMDEsK24pLHI9ZSplO3JldHVybiB0
KGUscixyKnIpfSxpfShNYXRoLlNRUlQyLDIsNCk7ZnVuY3Rpb24gb2kodCl7cmV0dXJuIGZ1bmN0
aW9uKG4sZSl7dmFyIHI9dCgobj1YZShuKSkuaCwoZT1YZShlKSkuaCksaT1EcihuLnMsZS5zKSxv
PURyKG4ubCxlLmwpLGE9RHIobi5vcGFjaXR5LGUub3BhY2l0eSk7cmV0dXJuIGZ1bmN0aW9uKHQp
e3JldHVybiBuLmg9cih0KSxuLnM9aSh0KSxuLmw9byh0KSxuLm9wYWNpdHk9YSh0KSxuKyIifX19
dmFyIGFpPW9pKHpyKSx1aT1vaShEcik7ZnVuY3Rpb24gY2kodCl7cmV0dXJuIGZ1bmN0aW9uKG4s
ZSl7dmFyIHI9dCgobj1wcihuKSkuaCwoZT1wcihlKSkuaCksaT1EcihuLmMsZS5jKSxvPURyKG4u
bCxlLmwpLGE9RHIobi5vcGFjaXR5LGUub3BhY2l0eSk7cmV0dXJuIGZ1bmN0aW9uKHQpe3JldHVy
biBuLmg9cih0KSxuLmM9aSh0KSxuLmw9byh0KSxuLm9wYWNpdHk9YSh0KSxuKyIifX19dmFyIGZp
PWNpKHpyKSxzaT1jaShEcik7ZnVuY3Rpb24gbGkodCl7cmV0dXJuIGZ1bmN0aW9uIG4oZSl7ZnVu
Y3Rpb24gcihuLHIpe3ZhciBpPXQoKG49QXIobikpLmgsKHI9QXIocikpLmgpLG89RHIobi5zLHIu
cyksYT1EcihuLmwsci5sKSx1PURyKG4ub3BhY2l0eSxyLm9wYWNpdHkpO3JldHVybiBmdW5jdGlv
bih0KXtyZXR1cm4gbi5oPWkodCksbi5zPW8odCksbi5sPWEoTWF0aC5wb3codCxlKSksbi5vcGFj
aXR5PXUodCksbisiIn19cmV0dXJuIGU9K2Usci5nYW1tYT1uLHJ9KDEpfXZhciBoaT1saSh6ciks
ZGk9bGkoRHIpO2Z1bmN0aW9uIHBpKHQsbil7dm9pZCAwPT09biYmKG49dCx0PVZyKTtmb3IodmFy
IGU9MCxyPW4ubGVuZ3RoLTEsaT1uWzBdLG89bmV3IEFycmF5KHI8MD8wOnIpO2U8cjspb1tlXT10
KGksaT1uWysrZV0pO3JldHVybiBmdW5jdGlvbih0KXt2YXIgbj1NYXRoLm1heCgwLE1hdGgubWlu
KHItMSxNYXRoLmZsb29yKHQqPXIpKSk7cmV0dXJuIG9bbl0odC1uKX19dmFyIGdpLHlpLHZpPTAs
X2k9MCxiaT0wLG1pPTFlMyx4aT0wLHdpPTAsTWk9MCxUaT0ib2JqZWN0Ij09dHlwZW9mIHBlcmZv
cm1hbmNlJiZwZXJmb3JtYW5jZS5ub3c/cGVyZm9ybWFuY2U6RGF0ZSxBaT0ib2JqZWN0Ij09dHlw
ZW9mIHdpbmRvdyYmd2luZG93LnJlcXVlc3RBbmltYXRpb25GcmFtZT93aW5kb3cucmVxdWVzdEFu
aW1hdGlvbkZyYW1lLmJpbmQod2luZG93KTpmdW5jdGlvbih0KXtzZXRUaW1lb3V0KHQsMTcpfTtm
dW5jdGlvbiBTaSgpe3JldHVybiB3aXx8KEFpKEVpKSx3aT1UaS5ub3coKStNaSl9ZnVuY3Rpb24g
RWkoKXt3aT0wfWZ1bmN0aW9uIE5pKCl7dGhpcy5fY2FsbD10aGlzLl90aW1lPXRoaXMuX25leHQ9
bnVsbH1mdW5jdGlvbiBraSh0LG4sZSl7dmFyIHI9bmV3IE5pO3JldHVybiByLnJlc3RhcnQodCxu
LGUpLHJ9ZnVuY3Rpb24gQ2koKXtTaSgpLCsrdmk7Zm9yKHZhciB0LG49Z2k7bjspKHQ9d2ktbi5f
dGltZSk+PTAmJm4uX2NhbGwuY2FsbCh2b2lkIDAsdCksbj1uLl9uZXh0Oy0tdml9ZnVuY3Rpb24g
UGkoKXt3aT0oeGk9VGkubm93KCkpK01pLHZpPV9pPTA7dHJ5e0NpKCl9ZmluYWxseXt2aT0wLGZ1
bmN0aW9uKCl7dmFyIHQsbixlPWdpLHI9MS8wO2Zvcig7ZTspZS5fY2FsbD8ocj5lLl90aW1lJiYo
cj1lLl90aW1lKSx0PWUsZT1lLl9uZXh0KToobj1lLl9uZXh0LGUuX25leHQ9bnVsbCxlPXQ/dC5f
bmV4dD1uOmdpPW4pO3lpPXQsJGkocil9KCksd2k9MH19ZnVuY3Rpb24gemkoKXt2YXIgdD1UaS5u
b3coKSxuPXQteGk7bj5taSYmKE1pLT1uLHhpPXQpfWZ1bmN0aW9uICRpKHQpe3ZpfHwoX2kmJihf
aT1jbGVhclRpbWVvdXQoX2kpKSx0LXdpPjI0Pyh0PDEvMCYmKF9pPXNldFRpbWVvdXQoUGksdC1U
aS5ub3coKS1NaSkpLGJpJiYoYmk9Y2xlYXJJbnRlcnZhbChiaSkpKTooYml8fCh4aT1UaS5ub3co
KSxiaT1zZXRJbnRlcnZhbCh6aSxtaSkpLHZpPTEsQWkoUGkpKSl9ZnVuY3Rpb24gRGkodCxuLGUp
e3ZhciByPW5ldyBOaTtyZXR1cm4gbj1udWxsPT1uPzA6K24sci5yZXN0YXJ0KChlPT57ci5zdG9w
KCksdChlK24pfSksbixlKSxyfU5pLnByb3RvdHlwZT1raS5wcm90b3R5cGU9e2NvbnN0cnVjdG9y
Ok5pLHJlc3RhcnQ6ZnVuY3Rpb24odCxuLGUpe2lmKCJmdW5jdGlvbiIhPXR5cGVvZiB0KXRocm93
IG5ldyBUeXBlRXJyb3IoImNhbGxiYWNrIGlzIG5vdCBhIGZ1bmN0aW9uIik7ZT0obnVsbD09ZT9T
aSgpOitlKSsobnVsbD09bj8wOituKSx0aGlzLl9uZXh0fHx5aT09PXRoaXN8fCh5aT95aS5fbmV4
dD10aGlzOmdpPXRoaXMseWk9dGhpcyksdGhpcy5fY2FsbD10LHRoaXMuX3RpbWU9ZSwkaSgpfSxz
dG9wOmZ1bmN0aW9uKCl7dGhpcy5fY2FsbCYmKHRoaXMuX2NhbGw9bnVsbCx0aGlzLl90aW1lPTEv
MCwkaSgpKX19O3ZhciBSaT1EdCgic3RhcnQiLCJlbmQiLCJjYW5jZWwiLCJpbnRlcnJ1cHQiKSxG
aT1bXSxxaT0wLFVpPTEsSWk9MixPaT0zLEJpPTQsWWk9NSxMaT02O2Z1bmN0aW9uIGppKHQsbixl
LHIsaSxvKXt2YXIgYT10Ll9fdHJhbnNpdGlvbjtpZihhKXtpZihlIGluIGEpcmV0dXJufWVsc2Ug
dC5fX3RyYW5zaXRpb249e307IWZ1bmN0aW9uKHQsbixlKXt2YXIgcixpPXQuX190cmFuc2l0aW9u
O2Z1bmN0aW9uIG8odCl7ZS5zdGF0ZT1VaSxlLnRpbWVyLnJlc3RhcnQoYSxlLmRlbGF5LGUudGlt
ZSksZS5kZWxheTw9dCYmYSh0LWUuZGVsYXkpfWZ1bmN0aW9uIGEobyl7dmFyIGYscyxsLGg7aWYo
ZS5zdGF0ZSE9PVVpKXJldHVybiBjKCk7Zm9yKGYgaW4gaSlpZigoaD1pW2ZdKS5uYW1lPT09ZS5u
YW1lKXtpZihoLnN0YXRlPT09T2kpcmV0dXJuIERpKGEpO2guc3RhdGU9PT1CaT8oaC5zdGF0ZT1M
aSxoLnRpbWVyLnN0b3AoKSxoLm9uLmNhbGwoImludGVycnVwdCIsdCx0Ll9fZGF0YV9fLGguaW5k
ZXgsaC5ncm91cCksZGVsZXRlIGlbZl0pOitmPG4mJihoLnN0YXRlPUxpLGgudGltZXIuc3RvcCgp
LGgub24uY2FsbCgiY2FuY2VsIix0LHQuX19kYXRhX18saC5pbmRleCxoLmdyb3VwKSxkZWxldGUg
aVtmXSl9aWYoRGkoKGZ1bmN0aW9uKCl7ZS5zdGF0ZT09PU9pJiYoZS5zdGF0ZT1CaSxlLnRpbWVy
LnJlc3RhcnQodSxlLmRlbGF5LGUudGltZSksdShvKSl9KSksZS5zdGF0ZT1JaSxlLm9uLmNhbGwo
InN0YXJ0Iix0LHQuX19kYXRhX18sZS5pbmRleCxlLmdyb3VwKSxlLnN0YXRlPT09SWkpe2Zvcihl
LnN0YXRlPU9pLHI9bmV3IEFycmF5KGw9ZS50d2Vlbi5sZW5ndGgpLGY9MCxzPS0xO2Y8bDsrK2Yp
KGg9ZS50d2VlbltmXS52YWx1ZS5jYWxsKHQsdC5fX2RhdGFfXyxlLmluZGV4LGUuZ3JvdXApKSYm
KHJbKytzXT1oKTtyLmxlbmd0aD1zKzF9fWZ1bmN0aW9uIHUobil7Zm9yKHZhciBpPW48ZS5kdXJh
dGlvbj9lLmVhc2UuY2FsbChudWxsLG4vZS5kdXJhdGlvbik6KGUudGltZXIucmVzdGFydChjKSxl
LnN0YXRlPVlpLDEpLG89LTEsYT1yLmxlbmd0aDsrK288YTspcltvXS5jYWxsKHQsaSk7ZS5zdGF0
ZT09PVlpJiYoZS5vbi5jYWxsKCJlbmQiLHQsdC5fX2RhdGFfXyxlLmluZGV4LGUuZ3JvdXApLGMo
KSl9ZnVuY3Rpb24gYygpe2Zvcih2YXIgciBpbiBlLnN0YXRlPUxpLGUudGltZXIuc3RvcCgpLGRl
bGV0ZSBpW25dLGkpcmV0dXJuO2RlbGV0ZSB0Ll9fdHJhbnNpdGlvbn1pW25dPWUsZS50aW1lcj1r
aShvLDAsZS50aW1lKX0odCxlLHtuYW1lOm4saW5kZXg6cixncm91cDppLG9uOlJpLHR3ZWVuOkZp
LHRpbWU6by50aW1lLGRlbGF5Om8uZGVsYXksZHVyYXRpb246by5kdXJhdGlvbixlYXNlOm8uZWFz
ZSx0aW1lcjpudWxsLHN0YXRlOnFpfSl9ZnVuY3Rpb24gSGkodCxuKXt2YXIgZT1HaSh0LG4pO2lm
KGUuc3RhdGU+cWkpdGhyb3cgbmV3IEVycm9yKCJ0b28gbGF0ZTsgYWxyZWFkeSBzY2hlZHVsZWQi
KTtyZXR1cm4gZX1mdW5jdGlvbiBYaSh0LG4pe3ZhciBlPUdpKHQsbik7aWYoZS5zdGF0ZT5PaSl0
aHJvdyBuZXcgRXJyb3IoInRvbyBsYXRlOyBhbHJlYWR5IHJ1bm5pbmciKTtyZXR1cm4gZX1mdW5j
dGlvbiBHaSh0LG4pe3ZhciBlPXQuX190cmFuc2l0aW9uO2lmKCFlfHwhKGU9ZVtuXSkpdGhyb3cg
bmV3IEVycm9yKCJ0cmFuc2l0aW9uIG5vdCBmb3VuZCIpO3JldHVybiBlfWZ1bmN0aW9uIFZpKHQs
bil7dmFyIGUscixpLG89dC5fX3RyYW5zaXRpb24sYT0hMDtpZihvKXtmb3IoaSBpbiBuPW51bGw9
PW4/bnVsbDpuKyIiLG8pKGU9b1tpXSkubmFtZT09PW4/KHI9ZS5zdGF0ZT5JaSYmZS5zdGF0ZTxZ
aSxlLnN0YXRlPUxpLGUudGltZXIuc3RvcCgpLGUub24uY2FsbChyPyJpbnRlcnJ1cHQiOiJjYW5j
ZWwiLHQsdC5fX2RhdGFfXyxlLmluZGV4LGUuZ3JvdXApLGRlbGV0ZSBvW2ldKTphPSExO2EmJmRl
bGV0ZSB0Ll9fdHJhbnNpdGlvbn19ZnVuY3Rpb24gV2kodCxuKXt2YXIgZSxyO3JldHVybiBmdW5j
dGlvbigpe3ZhciBpPVhpKHRoaXMsdCksbz1pLnR3ZWVuO2lmKG8hPT1lKWZvcih2YXIgYT0wLHU9
KHI9ZT1vKS5sZW5ndGg7YTx1OysrYSlpZihyW2FdLm5hbWU9PT1uKXsocj1yLnNsaWNlKCkpLnNw
bGljZShhLDEpO2JyZWFrfWkudHdlZW49cn19ZnVuY3Rpb24gWmkodCxuLGUpe3ZhciByLGk7aWYo
ImZ1bmN0aW9uIiE9dHlwZW9mIGUpdGhyb3cgbmV3IEVycm9yO3JldHVybiBmdW5jdGlvbigpe3Zh
ciBvPVhpKHRoaXMsdCksYT1vLnR3ZWVuO2lmKGEhPT1yKXtpPShyPWEpLnNsaWNlKCk7Zm9yKHZh
ciB1PXtuYW1lOm4sdmFsdWU6ZX0sYz0wLGY9aS5sZW5ndGg7YzxmOysrYylpZihpW2NdLm5hbWU9
PT1uKXtpW2NdPXU7YnJlYWt9Yz09PWYmJmkucHVzaCh1KX1vLnR3ZWVuPWl9fWZ1bmN0aW9uIEtp
KHQsbixlKXt2YXIgcj10Ll9pZDtyZXR1cm4gdC5lYWNoKChmdW5jdGlvbigpe3ZhciB0PVhpKHRo
aXMscik7KHQudmFsdWV8fCh0LnZhbHVlPXt9KSlbbl09ZS5hcHBseSh0aGlzLGFyZ3VtZW50cyl9
KSksZnVuY3Rpb24odCl7cmV0dXJuIEdpKHQscikudmFsdWVbbl19fWZ1bmN0aW9uIFFpKHQsbil7
dmFyIGU7cmV0dXJuKCJudW1iZXIiPT10eXBlb2Ygbj9McjpuIGluc3RhbmNlb2YgJGU/UnI6KGU9
JGUobikpPyhuPWUsUnIpOkdyKSh0LG4pfWZ1bmN0aW9uIEppKHQpe3JldHVybiBmdW5jdGlvbigp
e3RoaXMucmVtb3ZlQXR0cmlidXRlKHQpfX1mdW5jdGlvbiB0byh0KXtyZXR1cm4gZnVuY3Rpb24o
KXt0aGlzLnJlbW92ZUF0dHJpYnV0ZU5TKHQuc3BhY2UsdC5sb2NhbCl9fWZ1bmN0aW9uIG5vKHQs
bixlKXt2YXIgcixpLG89ZSsiIjtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgYT10aGlzLmdldEF0dHJp
YnV0ZSh0KTtyZXR1cm4gYT09PW8/bnVsbDphPT09cj9pOmk9bihyPWEsZSl9fWZ1bmN0aW9uIGVv
KHQsbixlKXt2YXIgcixpLG89ZSsiIjtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgYT10aGlzLmdldEF0
dHJpYnV0ZU5TKHQuc3BhY2UsdC5sb2NhbCk7cmV0dXJuIGE9PT1vP251bGw6YT09PXI/aTppPW4o
cj1hLGUpfX1mdW5jdGlvbiBybyh0LG4sZSl7dmFyIHIsaSxvO3JldHVybiBmdW5jdGlvbigpe3Zh
ciBhLHUsYz1lKHRoaXMpO2lmKG51bGwhPWMpcmV0dXJuKGE9dGhpcy5nZXRBdHRyaWJ1dGUodCkp
PT09KHU9YysiIik/bnVsbDphPT09ciYmdT09PWk/bzooaT11LG89bihyPWEsYykpO3RoaXMucmVt
b3ZlQXR0cmlidXRlKHQpfX1mdW5jdGlvbiBpbyh0LG4sZSl7dmFyIHIsaSxvO3JldHVybiBmdW5j
dGlvbigpe3ZhciBhLHUsYz1lKHRoaXMpO2lmKG51bGwhPWMpcmV0dXJuKGE9dGhpcy5nZXRBdHRy
aWJ1dGVOUyh0LnNwYWNlLHQubG9jYWwpKT09PSh1PWMrIiIpP251bGw6YT09PXImJnU9PT1pP286
KGk9dSxvPW4ocj1hLGMpKTt0aGlzLnJlbW92ZUF0dHJpYnV0ZU5TKHQuc3BhY2UsdC5sb2NhbCl9
fWZ1bmN0aW9uIG9vKHQsbil7dmFyIGUscjtmdW5jdGlvbiBpKCl7dmFyIGk9bi5hcHBseSh0aGlz
LGFyZ3VtZW50cyk7cmV0dXJuIGkhPT1yJiYoZT0ocj1pKSYmZnVuY3Rpb24odCxuKXtyZXR1cm4g
ZnVuY3Rpb24oZSl7dGhpcy5zZXRBdHRyaWJ1dGVOUyh0LnNwYWNlLHQubG9jYWwsbi5jYWxsKHRo
aXMsZSkpfX0odCxpKSksZX1yZXR1cm4gaS5fdmFsdWU9bixpfWZ1bmN0aW9uIGFvKHQsbil7dmFy
IGUscjtmdW5jdGlvbiBpKCl7dmFyIGk9bi5hcHBseSh0aGlzLGFyZ3VtZW50cyk7cmV0dXJuIGkh
PT1yJiYoZT0ocj1pKSYmZnVuY3Rpb24odCxuKXtyZXR1cm4gZnVuY3Rpb24oZSl7dGhpcy5zZXRB
dHRyaWJ1dGUodCxuLmNhbGwodGhpcyxlKSl9fSh0LGkpKSxlfXJldHVybiBpLl92YWx1ZT1uLGl9
ZnVuY3Rpb24gdW8odCxuKXtyZXR1cm4gZnVuY3Rpb24oKXtIaSh0aGlzLHQpLmRlbGF5PStuLmFw
cGx5KHRoaXMsYXJndW1lbnRzKX19ZnVuY3Rpb24gY28odCxuKXtyZXR1cm4gbj0rbixmdW5jdGlv
bigpe0hpKHRoaXMsdCkuZGVsYXk9bn19ZnVuY3Rpb24gZm8odCxuKXtyZXR1cm4gZnVuY3Rpb24o
KXtYaSh0aGlzLHQpLmR1cmF0aW9uPStuLmFwcGx5KHRoaXMsYXJndW1lbnRzKX19ZnVuY3Rpb24g
c28odCxuKXtyZXR1cm4gbj0rbixmdW5jdGlvbigpe1hpKHRoaXMsdCkuZHVyYXRpb249bn19dmFy
IGxvPVpuLnByb3RvdHlwZS5jb25zdHJ1Y3RvcjtmdW5jdGlvbiBobyh0KXtyZXR1cm4gZnVuY3Rp
b24oKXt0aGlzLnN0eWxlLnJlbW92ZVByb3BlcnR5KHQpfX12YXIgcG89MDtmdW5jdGlvbiBnbyh0
LG4sZSxyKXt0aGlzLl9ncm91cHM9dCx0aGlzLl9wYXJlbnRzPW4sdGhpcy5fbmFtZT1lLHRoaXMu
X2lkPXJ9ZnVuY3Rpb24geW8odCl7cmV0dXJuIFpuKCkudHJhbnNpdGlvbih0KX1mdW5jdGlvbiB2
bygpe3JldHVybisrcG99dmFyIF9vPVpuLnByb3RvdHlwZTtnby5wcm90b3R5cGU9eW8ucHJvdG90
eXBlPXtjb25zdHJ1Y3RvcjpnbyxzZWxlY3Q6ZnVuY3Rpb24odCl7dmFyIG49dGhpcy5fbmFtZSxl
PXRoaXMuX2lkOyJmdW5jdGlvbiIhPXR5cGVvZiB0JiYodD1IdCh0KSk7Zm9yKHZhciByPXRoaXMu
X2dyb3VwcyxpPXIubGVuZ3RoLG89bmV3IEFycmF5KGkpLGE9MDthPGk7KythKWZvcih2YXIgdSxj
LGY9clthXSxzPWYubGVuZ3RoLGw9b1thXT1uZXcgQXJyYXkocyksaD0wO2g8czsrK2gpKHU9Zlto
XSkmJihjPXQuY2FsbCh1LHUuX19kYXRhX18saCxmKSkmJigiX19kYXRhX18iaW4gdSYmKGMuX19k
YXRhX189dS5fX2RhdGFfXyksbFtoXT1jLGppKGxbaF0sbixlLGgsbCxHaSh1LGUpKSk7cmV0dXJu
IG5ldyBnbyhvLHRoaXMuX3BhcmVudHMsbixlKX0sc2VsZWN0QWxsOmZ1bmN0aW9uKHQpe3ZhciBu
PXRoaXMuX25hbWUsZT10aGlzLl9pZDsiZnVuY3Rpb24iIT10eXBlb2YgdCYmKHQ9VnQodCkpO2Zv
cih2YXIgcj10aGlzLl9ncm91cHMsaT1yLmxlbmd0aCxvPVtdLGE9W10sdT0wO3U8aTsrK3UpZm9y
KHZhciBjLGY9clt1XSxzPWYubGVuZ3RoLGw9MDtsPHM7KytsKWlmKGM9ZltsXSl7Zm9yKHZhciBo
LGQ9dC5jYWxsKGMsYy5fX2RhdGFfXyxsLGYpLHA9R2koYyxlKSxnPTAseT1kLmxlbmd0aDtnPHk7
KytnKShoPWRbZ10pJiZqaShoLG4sZSxnLGQscCk7by5wdXNoKGQpLGEucHVzaChjKX1yZXR1cm4g
bmV3IGdvKG8sYSxuLGUpfSxzZWxlY3RDaGlsZDpfby5zZWxlY3RDaGlsZCxzZWxlY3RDaGlsZHJl
bjpfby5zZWxlY3RDaGlsZHJlbixmaWx0ZXI6ZnVuY3Rpb24odCl7ImZ1bmN0aW9uIiE9dHlwZW9m
IHQmJih0PVd0KHQpKTtmb3IodmFyIG49dGhpcy5fZ3JvdXBzLGU9bi5sZW5ndGgscj1uZXcgQXJy
YXkoZSksaT0wO2k8ZTsrK2kpZm9yKHZhciBvLGE9bltpXSx1PWEubGVuZ3RoLGM9cltpXT1bXSxm
PTA7Zjx1OysrZikobz1hW2ZdKSYmdC5jYWxsKG8sby5fX2RhdGFfXyxmLGEpJiZjLnB1c2gobyk7
cmV0dXJuIG5ldyBnbyhyLHRoaXMuX3BhcmVudHMsdGhpcy5fbmFtZSx0aGlzLl9pZCl9LG1lcmdl
OmZ1bmN0aW9uKHQpe2lmKHQuX2lkIT09dGhpcy5faWQpdGhyb3cgbmV3IEVycm9yO2Zvcih2YXIg
bj10aGlzLl9ncm91cHMsZT10Ll9ncm91cHMscj1uLmxlbmd0aCxpPWUubGVuZ3RoLG89TWF0aC5t
aW4ocixpKSxhPW5ldyBBcnJheShyKSx1PTA7dTxvOysrdSlmb3IodmFyIGMsZj1uW3VdLHM9ZVt1
XSxsPWYubGVuZ3RoLGg9YVt1XT1uZXcgQXJyYXkobCksZD0wO2Q8bDsrK2QpKGM9ZltkXXx8c1tk
XSkmJihoW2RdPWMpO2Zvcig7dTxyOysrdSlhW3VdPW5bdV07cmV0dXJuIG5ldyBnbyhhLHRoaXMu
X3BhcmVudHMsdGhpcy5fbmFtZSx0aGlzLl9pZCl9LHNlbGVjdGlvbjpmdW5jdGlvbigpe3JldHVy
biBuZXcgbG8odGhpcy5fZ3JvdXBzLHRoaXMuX3BhcmVudHMpfSx0cmFuc2l0aW9uOmZ1bmN0aW9u
KCl7Zm9yKHZhciB0PXRoaXMuX25hbWUsbj10aGlzLl9pZCxlPXZvKCkscj10aGlzLl9ncm91cHMs
aT1yLmxlbmd0aCxvPTA7bzxpOysrbylmb3IodmFyIGEsdT1yW29dLGM9dS5sZW5ndGgsZj0wO2Y8
YzsrK2YpaWYoYT11W2ZdKXt2YXIgcz1HaShhLG4pO2ppKGEsdCxlLGYsdSx7dGltZTpzLnRpbWUr
cy5kZWxheStzLmR1cmF0aW9uLGRlbGF5OjAsZHVyYXRpb246cy5kdXJhdGlvbixlYXNlOnMuZWFz
ZX0pfXJldHVybiBuZXcgZ28ocix0aGlzLl9wYXJlbnRzLHQsZSl9LGNhbGw6X28uY2FsbCxub2Rl
czpfby5ub2Rlcyxub2RlOl9vLm5vZGUsc2l6ZTpfby5zaXplLGVtcHR5Ol9vLmVtcHR5LGVhY2g6
X28uZWFjaCxvbjpmdW5jdGlvbih0LG4pe3ZhciBlPXRoaXMuX2lkO3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPDI/R2kodGhpcy5ub2RlKCksZSkub24ub24odCk6dGhpcy5lYWNoKGZ1bmN0aW9uKHQs
bixlKXt2YXIgcixpLG89ZnVuY3Rpb24odCl7cmV0dXJuKHQrIiIpLnRyaW0oKS5zcGxpdCgvXnxc
cysvKS5ldmVyeSgoZnVuY3Rpb24odCl7dmFyIG49dC5pbmRleE9mKCIuIik7cmV0dXJuIG4+PTAm
Jih0PXQuc2xpY2UoMCxuKSksIXR8fCJzdGFydCI9PT10fSkpfShuKT9IaTpYaTtyZXR1cm4gZnVu
Y3Rpb24oKXt2YXIgYT1vKHRoaXMsdCksdT1hLm9uO3UhPT1yJiYoaT0ocj11KS5jb3B5KCkpLm9u
KG4sZSksYS5vbj1pfX0oZSx0LG4pKX0sYXR0cjpmdW5jdGlvbih0LG4pe3ZhciBlPU90KHQpLHI9
InRyYW5zZm9ybSI9PT1lP2VpOlFpO3JldHVybiB0aGlzLmF0dHJUd2Vlbih0LCJmdW5jdGlvbiI9
PXR5cGVvZiBuPyhlLmxvY2FsP2lvOnJvKShlLHIsS2kodGhpcywiYXR0ci4iK3QsbikpOm51bGw9
PW4/KGUubG9jYWw/dG86SmkpKGUpOihlLmxvY2FsP2VvOm5vKShlLHIsbikpfSxhdHRyVHdlZW46
ZnVuY3Rpb24odCxuKXt2YXIgZT0iYXR0ci4iK3Q7aWYoYXJndW1lbnRzLmxlbmd0aDwyKXJldHVy
bihlPXRoaXMudHdlZW4oZSkpJiZlLl92YWx1ZTtpZihudWxsPT1uKXJldHVybiB0aGlzLnR3ZWVu
KGUsbnVsbCk7aWYoImZ1bmN0aW9uIiE9dHlwZW9mIG4pdGhyb3cgbmV3IEVycm9yO3ZhciByPU90
KHQpO3JldHVybiB0aGlzLnR3ZWVuKGUsKHIubG9jYWw/b286YW8pKHIsbikpfSxzdHlsZTpmdW5j
dGlvbih0LG4sZSl7dmFyIHI9InRyYW5zZm9ybSI9PSh0Kz0iIik/bmk6UWk7cmV0dXJuIG51bGw9
PW4/dGhpcy5zdHlsZVR3ZWVuKHQsZnVuY3Rpb24odCxuKXt2YXIgZSxyLGk7cmV0dXJuIGZ1bmN0
aW9uKCl7dmFyIG89Ym4odGhpcyx0KSxhPSh0aGlzLnN0eWxlLnJlbW92ZVByb3BlcnR5KHQpLGJu
KHRoaXMsdCkpO3JldHVybiBvPT09YT9udWxsOm89PT1lJiZhPT09cj9pOmk9bihlPW8scj1hKX19
KHQscikpLm9uKCJlbmQuc3R5bGUuIit0LGhvKHQpKToiZnVuY3Rpb24iPT10eXBlb2Ygbj90aGlz
LnN0eWxlVHdlZW4odCxmdW5jdGlvbih0LG4sZSl7dmFyIHIsaSxvO3JldHVybiBmdW5jdGlvbigp
e3ZhciBhPWJuKHRoaXMsdCksdT1lKHRoaXMpLGM9dSsiIjtyZXR1cm4gbnVsbD09dSYmKHRoaXMu
c3R5bGUucmVtb3ZlUHJvcGVydHkodCksYz11PWJuKHRoaXMsdCkpLGE9PT1jP251bGw6YT09PXIm
JmM9PT1pP286KGk9YyxvPW4ocj1hLHUpKX19KHQscixLaSh0aGlzLCJzdHlsZS4iK3QsbikpKS5l
YWNoKGZ1bmN0aW9uKHQsbil7dmFyIGUscixpLG8sYT0ic3R5bGUuIituLHU9ImVuZC4iK2E7cmV0
dXJuIGZ1bmN0aW9uKCl7dmFyIGM9WGkodGhpcyx0KSxmPWMub24scz1udWxsPT1jLnZhbHVlW2Fd
P298fChvPWhvKG4pKTp2b2lkIDA7Zj09PWUmJmk9PT1zfHwocj0oZT1mKS5jb3B5KCkpLm9uKHUs
aT1zKSxjLm9uPXJ9fSh0aGlzLl9pZCx0KSk6dGhpcy5zdHlsZVR3ZWVuKHQsZnVuY3Rpb24odCxu
LGUpe3ZhciByLGksbz1lKyIiO3JldHVybiBmdW5jdGlvbigpe3ZhciBhPWJuKHRoaXMsdCk7cmV0
dXJuIGE9PT1vP251bGw6YT09PXI/aTppPW4ocj1hLGUpfX0odCxyLG4pLGUpLm9uKCJlbmQuc3R5
bGUuIit0LG51bGwpfSxzdHlsZVR3ZWVuOmZ1bmN0aW9uKHQsbixlKXt2YXIgcj0ic3R5bGUuIiso
dCs9IiIpO2lmKGFyZ3VtZW50cy5sZW5ndGg8MilyZXR1cm4ocj10aGlzLnR3ZWVuKHIpKSYmci5f
dmFsdWU7aWYobnVsbD09bilyZXR1cm4gdGhpcy50d2VlbihyLG51bGwpO2lmKCJmdW5jdGlvbiIh
PXR5cGVvZiBuKXRocm93IG5ldyBFcnJvcjtyZXR1cm4gdGhpcy50d2VlbihyLGZ1bmN0aW9uKHQs
bixlKXt2YXIgcixpO2Z1bmN0aW9uIG8oKXt2YXIgbz1uLmFwcGx5KHRoaXMsYXJndW1lbnRzKTty
ZXR1cm4gbyE9PWkmJihyPShpPW8pJiZmdW5jdGlvbih0LG4sZSl7cmV0dXJuIGZ1bmN0aW9uKHIp
e3RoaXMuc3R5bGUuc2V0UHJvcGVydHkodCxuLmNhbGwodGhpcyxyKSxlKX19KHQsbyxlKSkscn1y
ZXR1cm4gby5fdmFsdWU9bixvfSh0LG4sbnVsbD09ZT8iIjplKSl9LHRleHQ6ZnVuY3Rpb24odCl7
cmV0dXJuIHRoaXMudHdlZW4oInRleHQiLCJmdW5jdGlvbiI9PXR5cGVvZiB0P2Z1bmN0aW9uKHQp
e3JldHVybiBmdW5jdGlvbigpe3ZhciBuPXQodGhpcyk7dGhpcy50ZXh0Q29udGVudD1udWxsPT1u
PyIiOm59fShLaSh0aGlzLCJ0ZXh0Iix0KSk6ZnVuY3Rpb24odCl7cmV0dXJuIGZ1bmN0aW9uKCl7
dGhpcy50ZXh0Q29udGVudD10fX0obnVsbD09dD8iIjp0KyIiKSl9LHRleHRUd2VlbjpmdW5jdGlv
bih0KXt2YXIgbj0idGV4dCI7aWYoYXJndW1lbnRzLmxlbmd0aDwxKXJldHVybihuPXRoaXMudHdl
ZW4obikpJiZuLl92YWx1ZTtpZihudWxsPT10KXJldHVybiB0aGlzLnR3ZWVuKG4sbnVsbCk7aWYo
ImZ1bmN0aW9uIiE9dHlwZW9mIHQpdGhyb3cgbmV3IEVycm9yO3JldHVybiB0aGlzLnR3ZWVuKG4s
ZnVuY3Rpb24odCl7dmFyIG4sZTtmdW5jdGlvbiByKCl7dmFyIHI9dC5hcHBseSh0aGlzLGFyZ3Vt
ZW50cyk7cmV0dXJuIHIhPT1lJiYobj0oZT1yKSYmZnVuY3Rpb24odCl7cmV0dXJuIGZ1bmN0aW9u
KG4pe3RoaXMudGV4dENvbnRlbnQ9dC5jYWxsKHRoaXMsbil9fShyKSksbn1yZXR1cm4gci5fdmFs
dWU9dCxyfSh0KSl9LHJlbW92ZTpmdW5jdGlvbigpe3JldHVybiB0aGlzLm9uKCJlbmQucmVtb3Zl
IixmdW5jdGlvbih0KXtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgbj10aGlzLnBhcmVudE5vZGU7Zm9y
KHZhciBlIGluIHRoaXMuX190cmFuc2l0aW9uKWlmKCtlIT09dClyZXR1cm47biYmbi5yZW1vdmVD
aGlsZCh0aGlzKX19KHRoaXMuX2lkKSl9LHR3ZWVuOmZ1bmN0aW9uKHQsbil7dmFyIGU9dGhpcy5f
aWQ7aWYodCs9IiIsYXJndW1lbnRzLmxlbmd0aDwyKXtmb3IodmFyIHIsaT1HaSh0aGlzLm5vZGUo
KSxlKS50d2VlbixvPTAsYT1pLmxlbmd0aDtvPGE7KytvKWlmKChyPWlbb10pLm5hbWU9PT10KXJl
dHVybiByLnZhbHVlO3JldHVybiBudWxsfXJldHVybiB0aGlzLmVhY2goKG51bGw9PW4/V2k6Wmkp
KGUsdCxuKSl9LGRlbGF5OmZ1bmN0aW9uKHQpe3ZhciBuPXRoaXMuX2lkO3JldHVybiBhcmd1bWVu
dHMubGVuZ3RoP3RoaXMuZWFjaCgoImZ1bmN0aW9uIj09dHlwZW9mIHQ/dW86Y28pKG4sdCkpOkdp
KHRoaXMubm9kZSgpLG4pLmRlbGF5fSxkdXJhdGlvbjpmdW5jdGlvbih0KXt2YXIgbj10aGlzLl9p
ZDtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD90aGlzLmVhY2goKCJmdW5jdGlvbiI9PXR5cGVvZiB0
P2ZvOnNvKShuLHQpKTpHaSh0aGlzLm5vZGUoKSxuKS5kdXJhdGlvbn0sZWFzZTpmdW5jdGlvbih0
KXt2YXIgbj10aGlzLl9pZDtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD90aGlzLmVhY2goZnVuY3Rp
b24odCxuKXtpZigiZnVuY3Rpb24iIT10eXBlb2Ygbil0aHJvdyBuZXcgRXJyb3I7cmV0dXJuIGZ1
bmN0aW9uKCl7WGkodGhpcyx0KS5lYXNlPW59fShuLHQpKTpHaSh0aGlzLm5vZGUoKSxuKS5lYXNl
fSxlYXNlVmFyeWluZzpmdW5jdGlvbih0KXtpZigiZnVuY3Rpb24iIT10eXBlb2YgdCl0aHJvdyBu
ZXcgRXJyb3I7cmV0dXJuIHRoaXMuZWFjaChmdW5jdGlvbih0LG4pe3JldHVybiBmdW5jdGlvbigp
e3ZhciBlPW4uYXBwbHkodGhpcyxhcmd1bWVudHMpO2lmKCJmdW5jdGlvbiIhPXR5cGVvZiBlKXRo
cm93IG5ldyBFcnJvcjtYaSh0aGlzLHQpLmVhc2U9ZX19KHRoaXMuX2lkLHQpKX0sZW5kOmZ1bmN0
aW9uKCl7dmFyIHQsbixlPXRoaXMscj1lLl9pZCxpPWUuc2l6ZSgpO3JldHVybiBuZXcgUHJvbWlz
ZSgoZnVuY3Rpb24obyxhKXt2YXIgdT17dmFsdWU6YX0sYz17dmFsdWU6ZnVuY3Rpb24oKXswPT0t
LWkmJm8oKX19O2UuZWFjaCgoZnVuY3Rpb24oKXt2YXIgZT1YaSh0aGlzLHIpLGk9ZS5vbjtpIT09
dCYmKChuPSh0PWkpLmNvcHkoKSkuXy5jYW5jZWwucHVzaCh1KSxuLl8uaW50ZXJydXB0LnB1c2go
dSksbi5fLmVuZC5wdXNoKGMpKSxlLm9uPW59KSksMD09PWkmJm8oKX0pKX0sW1N5bWJvbC5pdGVy
YXRvcl06X29bU3ltYm9sLml0ZXJhdG9yXX07ZnVuY3Rpb24gYm8odCl7cmV0dXJuKCh0Kj0yKTw9
MT90KnQ6LS10KigyLXQpKzEpLzJ9ZnVuY3Rpb24gbW8odCl7cmV0dXJuKCh0Kj0yKTw9MT90KnQq
dDoodC09MikqdCp0KzIpLzJ9dmFyIHhvPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0KXtyZXR1
cm4gTWF0aC5wb3codCxuKX1yZXR1cm4gbj0rbixlLmV4cG9uZW50PXQsZX0oMyksd289ZnVuY3Rp
b24gdChuKXtmdW5jdGlvbiBlKHQpe3JldHVybiAxLU1hdGgucG93KDEtdCxuKX1yZXR1cm4gbj0r
bixlLmV4cG9uZW50PXQsZX0oMyksTW89ZnVuY3Rpb24gdChuKXtmdW5jdGlvbiBlKHQpe3JldHVy
bigodCo9Mik8PTE/TWF0aC5wb3codCxuKToyLU1hdGgucG93KDItdCxuKSkvMn1yZXR1cm4gbj0r
bixlLmV4cG9uZW50PXQsZX0oMyksVG89TWF0aC5QSSxBbz1Uby8yO2Z1bmN0aW9uIFNvKHQpe3Jl
dHVybigxLU1hdGguY29zKFRvKnQpKS8yfWZ1bmN0aW9uIEVvKHQpe3JldHVybiAxLjAwMDk3NzUx
NzEwNjU0OTQqKE1hdGgucG93KDIsLTEwKnQpLS4wMDA5NzY1NjI1KX1mdW5jdGlvbiBObyh0KXty
ZXR1cm4oKHQqPTIpPD0xP0VvKDEtdCk6Mi1Fbyh0LTEpKS8yfWZ1bmN0aW9uIGtvKHQpe3JldHVy
bigodCo9Mik8PTE/MS1NYXRoLnNxcnQoMS10KnQpOk1hdGguc3FydCgxLSh0LT0yKSp0KSsxKS8y
fXZhciBDbz00LzExLFBvPTYvMTEsem89OC8xMSwkbz0zLzQsRG89OS8xMSxSbz0xMC8xMSxGbz0x
NS8xNixxbz0yMS8yMixVbz02My82NCxJbz0xL0NvL0NvO2Z1bmN0aW9uIE9vKHQpe3JldHVybih0
PSt0KTxDbz9Jbyp0KnQ6dDx6bz9JbyoodC09UG8pKnQrJG86dDxSbz9JbyoodC09RG8pKnQrRm86
SW8qKHQtPXFvKSp0K1VvfXZhciBCbz0xLjcwMTU4LFlvPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24g
ZSh0KXtyZXR1cm4odD0rdCkqdCoobioodC0xKSt0KX1yZXR1cm4gbj0rbixlLm92ZXJzaG9vdD10
LGV9KEJvKSxMbz1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCl7cmV0dXJuLS10KnQqKCh0KzEp
Km4rdCkrMX1yZXR1cm4gbj0rbixlLm92ZXJzaG9vdD10LGV9KEJvKSxqbz1mdW5jdGlvbiB0KG4p
e2Z1bmN0aW9uIGUodCl7cmV0dXJuKCh0Kj0yKTwxP3QqdCooKG4rMSkqdC1uKToodC09MikqdCoo
KG4rMSkqdCtuKSsyKS8yfXJldHVybiBuPStuLGUub3ZlcnNob290PXQsZX0oQm8pLEhvPTIqTWF0
aC5QSSxYbz1mdW5jdGlvbiB0KG4sZSl7dmFyIHI9TWF0aC5hc2luKDEvKG49TWF0aC5tYXgoMSxu
KSkpKihlLz1Ibyk7ZnVuY3Rpb24gaSh0KXtyZXR1cm4gbipFbygtIC0tdCkqTWF0aC5zaW4oKHIt
dCkvZSl9cmV0dXJuIGkuYW1wbGl0dWRlPWZ1bmN0aW9uKG4pe3JldHVybiB0KG4sZSpIbyl9LGku
cGVyaW9kPWZ1bmN0aW9uKGUpe3JldHVybiB0KG4sZSl9LGl9KDEsLjMpLEdvPWZ1bmN0aW9uIHQo
bixlKXt2YXIgcj1NYXRoLmFzaW4oMS8obj1NYXRoLm1heCgxLG4pKSkqKGUvPUhvKTtmdW5jdGlv
biBpKHQpe3JldHVybiAxLW4qRW8odD0rdCkqTWF0aC5zaW4oKHQrcikvZSl9cmV0dXJuIGkuYW1w
bGl0dWRlPWZ1bmN0aW9uKG4pe3JldHVybiB0KG4sZSpIbyl9LGkucGVyaW9kPWZ1bmN0aW9uKGUp
e3JldHVybiB0KG4sZSl9LGl9KDEsLjMpLFZvPWZ1bmN0aW9uIHQobixlKXt2YXIgcj1NYXRoLmFz
aW4oMS8obj1NYXRoLm1heCgxLG4pKSkqKGUvPUhvKTtmdW5jdGlvbiBpKHQpe3JldHVybigodD0y
KnQtMSk8MD9uKkVvKC10KSpNYXRoLnNpbigoci10KS9lKToyLW4qRW8odCkqTWF0aC5zaW4oKHIr
dCkvZSkpLzJ9cmV0dXJuIGkuYW1wbGl0dWRlPWZ1bmN0aW9uKG4pe3JldHVybiB0KG4sZSpIbyl9
LGkucGVyaW9kPWZ1bmN0aW9uKGUpe3JldHVybiB0KG4sZSl9LGl9KDEsLjMpLFdvPXt0aW1lOm51
bGwsZGVsYXk6MCxkdXJhdGlvbjoyNTAsZWFzZTptb307ZnVuY3Rpb24gWm8odCxuKXtmb3IodmFy
IGU7IShlPXQuX190cmFuc2l0aW9uKXx8IShlPWVbbl0pOylpZighKHQ9dC5wYXJlbnROb2RlKSl0
aHJvdyBuZXcgRXJyb3IoYHRyYW5zaXRpb24gJHtufSBub3QgZm91bmRgKTtyZXR1cm4gZX1abi5w
cm90b3R5cGUuaW50ZXJydXB0PWZ1bmN0aW9uKHQpe3JldHVybiB0aGlzLmVhY2goKGZ1bmN0aW9u
KCl7VmkodGhpcyx0KX0pKX0sWm4ucHJvdG90eXBlLnRyYW5zaXRpb249ZnVuY3Rpb24odCl7dmFy
IG4sZTt0IGluc3RhbmNlb2YgZ28/KG49dC5faWQsdD10Ll9uYW1lKToobj12bygpLChlPVdvKS50
aW1lPVNpKCksdD1udWxsPT10P251bGw6dCsiIik7Zm9yKHZhciByPXRoaXMuX2dyb3VwcyxpPXIu
bGVuZ3RoLG89MDtvPGk7KytvKWZvcih2YXIgYSx1PXJbb10sYz11Lmxlbmd0aCxmPTA7ZjxjOysr
ZikoYT11W2ZdKSYmamkoYSx0LG4sZix1LGV8fFpvKGEsbikpO3JldHVybiBuZXcgZ28ocix0aGlz
Ll9wYXJlbnRzLHQsbil9O3ZhciBLbz1bbnVsbF07dmFyIFFvPXQ9PigpPT50O2Z1bmN0aW9uIEpv
KHQse3NvdXJjZUV2ZW50Om4sdGFyZ2V0OmUsc2VsZWN0aW9uOnIsbW9kZTppLGRpc3BhdGNoOm99
KXtPYmplY3QuZGVmaW5lUHJvcGVydGllcyh0aGlzLHt0eXBlOnt2YWx1ZTp0LGVudW1lcmFibGU6
ITAsY29uZmlndXJhYmxlOiEwfSxzb3VyY2VFdmVudDp7dmFsdWU6bixlbnVtZXJhYmxlOiEwLGNv
bmZpZ3VyYWJsZTohMH0sdGFyZ2V0Ont2YWx1ZTplLGVudW1lcmFibGU6ITAsY29uZmlndXJhYmxl
OiEwfSxzZWxlY3Rpb246e3ZhbHVlOnIsZW51bWVyYWJsZTohMCxjb25maWd1cmFibGU6ITB9LG1v
ZGU6e3ZhbHVlOmksZW51bWVyYWJsZTohMCxjb25maWd1cmFibGU6ITB9LF86e3ZhbHVlOm99fSl9
ZnVuY3Rpb24gdGEodCl7dC5wcmV2ZW50RGVmYXVsdCgpLHQuc3RvcEltbWVkaWF0ZVByb3BhZ2F0
aW9uKCl9dmFyIG5hPXtuYW1lOiJkcmFnIn0sZWE9e25hbWU6InNwYWNlIn0scmE9e25hbWU6Imhh
bmRsZSJ9LGlhPXtuYW1lOiJjZW50ZXIifTtjb25zdHthYnM6b2EsbWF4OmFhLG1pbjp1YX09TWF0
aDtmdW5jdGlvbiBjYSh0KXtyZXR1cm5bK3RbMF0sK3RbMV1dfWZ1bmN0aW9uIGZhKHQpe3JldHVy
bltjYSh0WzBdKSxjYSh0WzFdKV19dmFyIHNhPXtuYW1lOiJ4IixoYW5kbGVzOlsidyIsImUiXS5t
YXAoX2EpLGlucHV0OmZ1bmN0aW9uKHQsbil7cmV0dXJuIG51bGw9PXQ/bnVsbDpbWyt0WzBdLG5b
MF1bMV1dLFsrdFsxXSxuWzFdWzFdXV19LG91dHB1dDpmdW5jdGlvbih0KXtyZXR1cm4gdCYmW3Rb
MF1bMF0sdFsxXVswXV19fSxsYT17bmFtZToieSIsaGFuZGxlczpbIm4iLCJzIl0ubWFwKF9hKSxp
bnB1dDpmdW5jdGlvbih0LG4pe3JldHVybiBudWxsPT10P251bGw6W1tuWzBdWzBdLCt0WzBdXSxb
blsxXVswXSwrdFsxXV1dfSxvdXRwdXQ6ZnVuY3Rpb24odCl7cmV0dXJuIHQmJlt0WzBdWzFdLHRb
MV1bMV1dfX0saGE9e25hbWU6Inh5IixoYW5kbGVzOlsibiIsInciLCJlIiwicyIsIm53IiwibmUi
LCJzdyIsInNlIl0ubWFwKF9hKSxpbnB1dDpmdW5jdGlvbih0KXtyZXR1cm4gbnVsbD09dD9udWxs
OmZhKHQpfSxvdXRwdXQ6ZnVuY3Rpb24odCl7cmV0dXJuIHR9fSxkYT17b3ZlcmxheToiY3Jvc3No
YWlyIixzZWxlY3Rpb246Im1vdmUiLG46Im5zLXJlc2l6ZSIsZToiZXctcmVzaXplIixzOiJucy1y
ZXNpemUiLHc6ImV3LXJlc2l6ZSIsbnc6Im53c2UtcmVzaXplIixuZToibmVzdy1yZXNpemUiLHNl
OiJud3NlLXJlc2l6ZSIsc3c6Im5lc3ctcmVzaXplIn0scGE9e2U6InciLHc6ImUiLG53OiJuZSIs
bmU6Im53IixzZToic3ciLHN3OiJzZSJ9LGdhPXtuOiJzIixzOiJuIixudzoic3ciLG5lOiJzZSIs
c2U6Im5lIixzdzoibncifSx5YT17b3ZlcmxheToxLHNlbGVjdGlvbjoxLG46bnVsbCxlOjEsczpu
dWxsLHc6LTEsbnc6LTEsbmU6MSxzZToxLHN3Oi0xfSx2YT17b3ZlcmxheToxLHNlbGVjdGlvbjox
LG46LTEsZTpudWxsLHM6MSx3Om51bGwsbnc6LTEsbmU6LTEsc2U6MSxzdzoxfTtmdW5jdGlvbiBf
YSh0KXtyZXR1cm57dHlwZTp0fX1mdW5jdGlvbiBiYSh0KXtyZXR1cm4hdC5jdHJsS2V5JiYhdC5i
dXR0b259ZnVuY3Rpb24gbWEoKXt2YXIgdD10aGlzLm93bmVyU1ZHRWxlbWVudHx8dGhpcztyZXR1
cm4gdC5oYXNBdHRyaWJ1dGUoInZpZXdCb3giKT9bWyh0PXQudmlld0JveC5iYXNlVmFsKS54LHQu
eV0sW3QueCt0LndpZHRoLHQueSt0LmhlaWdodF1dOltbMCwwXSxbdC53aWR0aC5iYXNlVmFsLnZh
bHVlLHQuaGVpZ2h0LmJhc2VWYWwudmFsdWVdXX1mdW5jdGlvbiB4YSgpe3JldHVybiBuYXZpZ2F0
b3IubWF4VG91Y2hQb2ludHN8fCJvbnRvdWNoc3RhcnQiaW4gdGhpc31mdW5jdGlvbiB3YSh0KXtm
b3IoOyF0Ll9fYnJ1c2g7KWlmKCEodD10LnBhcmVudE5vZGUpKXJldHVybjtyZXR1cm4gdC5fX2Jy
dXNofWZ1bmN0aW9uIE1hKHQpe3ZhciBuLGU9bWEscj1iYSxpPXhhLG89ITAsYT1EdCgic3RhcnQi
LCJicnVzaCIsImVuZCIpLHU9NjtmdW5jdGlvbiBjKG4pe3ZhciBlPW4ucHJvcGVydHkoIl9fYnJ1
c2giLGcpLnNlbGVjdEFsbCgiLm92ZXJsYXkiKS5kYXRhKFtfYSgib3ZlcmxheSIpXSk7ZS5lbnRl
cigpLmFwcGVuZCgicmVjdCIpLmF0dHIoImNsYXNzIiwib3ZlcmxheSIpLmF0dHIoInBvaW50ZXIt
ZXZlbnRzIiwiYWxsIikuYXR0cigiY3Vyc29yIixkYS5vdmVybGF5KS5tZXJnZShlKS5lYWNoKChm
dW5jdGlvbigpe3ZhciB0PXdhKHRoaXMpLmV4dGVudDtLbih0aGlzKS5hdHRyKCJ4Iix0WzBdWzBd
KS5hdHRyKCJ5Iix0WzBdWzFdKS5hdHRyKCJ3aWR0aCIsdFsxXVswXS10WzBdWzBdKS5hdHRyKCJo
ZWlnaHQiLHRbMV1bMV0tdFswXVsxXSl9KSksbi5zZWxlY3RBbGwoIi5zZWxlY3Rpb24iKS5kYXRh
KFtfYSgic2VsZWN0aW9uIildKS5lbnRlcigpLmFwcGVuZCgicmVjdCIpLmF0dHIoImNsYXNzIiwi
c2VsZWN0aW9uIikuYXR0cigiY3Vyc29yIixkYS5zZWxlY3Rpb24pLmF0dHIoImZpbGwiLCIjNzc3
IikuYXR0cigiZmlsbC1vcGFjaXR5IiwuMykuYXR0cigic3Ryb2tlIiwiI2ZmZiIpLmF0dHIoInNo
YXBlLXJlbmRlcmluZyIsImNyaXNwRWRnZXMiKTt2YXIgcj1uLnNlbGVjdEFsbCgiLmhhbmRsZSIp
LmRhdGEodC5oYW5kbGVzLChmdW5jdGlvbih0KXtyZXR1cm4gdC50eXBlfSkpO3IuZXhpdCgpLnJl
bW92ZSgpLHIuZW50ZXIoKS5hcHBlbmQoInJlY3QiKS5hdHRyKCJjbGFzcyIsKGZ1bmN0aW9uKHQp
e3JldHVybiJoYW5kbGUgaGFuZGxlLS0iK3QudHlwZX0pKS5hdHRyKCJjdXJzb3IiLChmdW5jdGlv
bih0KXtyZXR1cm4gZGFbdC50eXBlXX0pKSxuLmVhY2goZikuYXR0cigiZmlsbCIsIm5vbmUiKS5h
dHRyKCJwb2ludGVyLWV2ZW50cyIsImFsbCIpLm9uKCJtb3VzZWRvd24uYnJ1c2giLGgpLmZpbHRl
cihpKS5vbigidG91Y2hzdGFydC5icnVzaCIsaCkub24oInRvdWNobW92ZS5icnVzaCIsZCkub24o
InRvdWNoZW5kLmJydXNoIHRvdWNoY2FuY2VsLmJydXNoIixwKS5zdHlsZSgidG91Y2gtYWN0aW9u
Iiwibm9uZSIpLnN0eWxlKCItd2Via2l0LXRhcC1oaWdobGlnaHQtY29sb3IiLCJyZ2JhKDAsMCww
LDApIil9ZnVuY3Rpb24gZigpe3ZhciB0PUtuKHRoaXMpLG49d2EodGhpcykuc2VsZWN0aW9uO24/
KHQuc2VsZWN0QWxsKCIuc2VsZWN0aW9uIikuc3R5bGUoImRpc3BsYXkiLG51bGwpLmF0dHIoIngi
LG5bMF1bMF0pLmF0dHIoInkiLG5bMF1bMV0pLmF0dHIoIndpZHRoIixuWzFdWzBdLW5bMF1bMF0p
LmF0dHIoImhlaWdodCIsblsxXVsxXS1uWzBdWzFdKSx0LnNlbGVjdEFsbCgiLmhhbmRsZSIpLnN0
eWxlKCJkaXNwbGF5IixudWxsKS5hdHRyKCJ4IiwoZnVuY3Rpb24odCl7cmV0dXJuImUiPT09dC50
eXBlW3QudHlwZS5sZW5ndGgtMV0/blsxXVswXS11LzI6blswXVswXS11LzJ9KSkuYXR0cigieSIs
KGZ1bmN0aW9uKHQpe3JldHVybiJzIj09PXQudHlwZVswXT9uWzFdWzFdLXUvMjpuWzBdWzFdLXUv
Mn0pKS5hdHRyKCJ3aWR0aCIsKGZ1bmN0aW9uKHQpe3JldHVybiJuIj09PXQudHlwZXx8InMiPT09
dC50eXBlP25bMV1bMF0tblswXVswXSt1OnV9KSkuYXR0cigiaGVpZ2h0IiwoZnVuY3Rpb24odCl7
cmV0dXJuImUiPT09dC50eXBlfHwidyI9PT10LnR5cGU/blsxXVsxXS1uWzBdWzFdK3U6dX0pKSk6
dC5zZWxlY3RBbGwoIi5zZWxlY3Rpb24sLmhhbmRsZSIpLnN0eWxlKCJkaXNwbGF5Iiwibm9uZSIp
LmF0dHIoIngiLG51bGwpLmF0dHIoInkiLG51bGwpLmF0dHIoIndpZHRoIixudWxsKS5hdHRyKCJo
ZWlnaHQiLG51bGwpfWZ1bmN0aW9uIHModCxuLGUpe3ZhciByPXQuX19icnVzaC5lbWl0dGVyO3Jl
dHVybiFyfHxlJiZyLmNsZWFuP25ldyBsKHQsbixlKTpyfWZ1bmN0aW9uIGwodCxuLGUpe3RoaXMu
dGhhdD10LHRoaXMuYXJncz1uLHRoaXMuc3RhdGU9dC5fX2JydXNoLHRoaXMuYWN0aXZlPTAsdGhp
cy5jbGVhbj1lfWZ1bmN0aW9uIGgoZSl7aWYoKCFufHxlLnRvdWNoZXMpJiZyLmFwcGx5KHRoaXMs
YXJndW1lbnRzKSl7dmFyIGksYSx1LGMsbCxoLGQscCxnLHksdixfPXRoaXMsYj1lLnRhcmdldC5f
X2RhdGFfXy50eXBlLG09InNlbGVjdGlvbiI9PT0obyYmZS5tZXRhS2V5P2I9Im92ZXJsYXkiOmIp
P25hOm8mJmUuYWx0S2V5P2lhOnJhLHg9dD09PWxhP251bGw6eWFbYl0sdz10PT09c2E/bnVsbDp2
YVtiXSxNPXdhKF8pLFQ9TS5leHRlbnQsQT1NLnNlbGVjdGlvbixTPVRbMF1bMF0sRT1UWzBdWzFd
LE49VFsxXVswXSxrPVRbMV1bMV0sQz0wLFA9MCx6PXgmJncmJm8mJmUuc2hpZnRLZXksJD1BcnJh
eS5mcm9tKGUudG91Y2hlc3x8W2VdLCh0PT57Y29uc3Qgbj10LmlkZW50aWZpZXI7cmV0dXJuKHQ9
ZWUodCxfKSkucG9pbnQwPXQuc2xpY2UoKSx0LmlkZW50aWZpZXI9bix0fSkpO1ZpKF8pO3ZhciBE
PXMoXyxhcmd1bWVudHMsITApLmJlZm9yZXN0YXJ0KCk7aWYoIm92ZXJsYXkiPT09Yil7QSYmKGc9
ITApO2NvbnN0IG49WyRbMF0sJFsxXXx8JFswXV07TS5zZWxlY3Rpb249QT1bW2k9dD09PWxhP1M6
dWEoblswXVswXSxuWzFdWzBdKSx1PXQ9PT1zYT9FOnVhKG5bMF1bMV0sblsxXVsxXSldLFtsPXQ9
PT1sYT9OOmFhKG5bMF1bMF0sblsxXVswXSksZD10PT09c2E/azphYShuWzBdWzFdLG5bMV1bMV0p
XV0sJC5sZW5ndGg+MSYmSShlKX1lbHNlIGk9QVswXVswXSx1PUFbMF1bMV0sbD1BWzFdWzBdLGQ9
QVsxXVsxXTthPWksYz11LGg9bCxwPWQ7dmFyIFI9S24oXykuYXR0cigicG9pbnRlci1ldmVudHMi
LCJub25lIiksRj1SLnNlbGVjdEFsbCgiLm92ZXJsYXkiKS5hdHRyKCJjdXJzb3IiLGRhW2JdKTtp
ZihlLnRvdWNoZXMpRC5tb3ZlZD1VLEQuZW5kZWQ9TztlbHNle3ZhciBxPUtuKGUudmlldykub24o
Im1vdXNlbW92ZS5icnVzaCIsVSwhMCkub24oIm1vdXNldXAuYnJ1c2giLE8sITApO28mJnEub24o
ImtleWRvd24uYnJ1c2giLChmdW5jdGlvbih0KXtzd2l0Y2godC5rZXlDb2RlKXtjYXNlIDE2Ono9
eCYmdzticmVhaztjYXNlIDE4Om09PT1yYSYmKHgmJihsPWgtQyp4LGk9YStDKngpLHcmJihkPXAt
UCp3LHU9YytQKncpLG09aWEsSSh0KSk7YnJlYWs7Y2FzZSAzMjptIT09cmEmJm0hPT1pYXx8KHg8
MD9sPWgtQzp4PjAmJihpPWEtQyksdzwwP2Q9cC1QOnc+MCYmKHU9Yy1QKSxtPWVhLEYuYXR0cigi
Y3Vyc29yIixkYS5zZWxlY3Rpb24pLEkodCkpO2JyZWFrO2RlZmF1bHQ6cmV0dXJufXRhKHQpfSks
ITApLm9uKCJrZXl1cC5icnVzaCIsKGZ1bmN0aW9uKHQpe3N3aXRjaCh0LmtleUNvZGUpe2Nhc2Ug
MTY6eiYmKHk9dj16PSExLEkodCkpO2JyZWFrO2Nhc2UgMTg6bT09PWlhJiYoeDwwP2w9aDp4PjAm
JihpPWEpLHc8MD9kPXA6dz4wJiYodT1jKSxtPXJhLEkodCkpO2JyZWFrO2Nhc2UgMzI6bT09PWVh
JiYodC5hbHRLZXk/KHgmJihsPWgtQyp4LGk9YStDKngpLHcmJihkPXAtUCp3LHU9YytQKncpLG09
aWEpOih4PDA/bD1oOng+MCYmKGk9YSksdzwwP2Q9cDp3PjAmJih1PWMpLG09cmEpLEYuYXR0cigi
Y3Vyc29yIixkYVtiXSksSSh0KSk7YnJlYWs7ZGVmYXVsdDpyZXR1cm59dGEodCl9KSwhMCksdWUo
ZS52aWV3KX1mLmNhbGwoXyksRC5zdGFydChlLG0ubmFtZSl9ZnVuY3Rpb24gVSh0KXtmb3IoY29u
c3QgbiBvZiB0LmNoYW5nZWRUb3VjaGVzfHxbdF0pZm9yKGNvbnN0IHQgb2YgJCl0LmlkZW50aWZp
ZXI9PT1uLmlkZW50aWZpZXImJih0LmN1cj1lZShuLF8pKTtpZih6JiYheSYmIXYmJjE9PT0kLmxl
bmd0aCl7Y29uc3QgdD0kWzBdO29hKHQuY3VyWzBdLXRbMF0pPm9hKHQuY3VyWzFdLXRbMV0pP3Y9
ITA6eT0hMH1mb3IoY29uc3QgdCBvZiAkKXQuY3VyJiYodFswXT10LmN1clswXSx0WzFdPXQuY3Vy
WzFdKTtnPSEwLHRhKHQpLEkodCl9ZnVuY3Rpb24gSSh0KXtjb25zdCBuPSRbMF0sZT1uLnBvaW50
MDt2YXIgcjtzd2l0Y2goQz1uWzBdLWVbMF0sUD1uWzFdLWVbMV0sbSl7Y2FzZSBlYTpjYXNlIG5h
OngmJihDPWFhKFMtaSx1YShOLWwsQykpLGE9aStDLGg9bCtDKSx3JiYoUD1hYShFLXUsdWEoay1k
LFApKSxjPXUrUCxwPWQrUCk7YnJlYWs7Y2FzZSByYTokWzFdPyh4JiYoYT1hYShTLHVhKE4sJFsw
XVswXSkpLGg9YWEoUyx1YShOLCRbMV1bMF0pKSx4PTEpLHcmJihjPWFhKEUsdWEoaywkWzBdWzFd
KSkscD1hYShFLHVhKGssJFsxXVsxXSkpLHc9MSkpOih4PDA/KEM9YWEoUy1pLHVhKE4taSxDKSks
YT1pK0MsaD1sKTp4PjAmJihDPWFhKFMtbCx1YShOLWwsQykpLGE9aSxoPWwrQyksdzwwPyhQPWFh
KEUtdSx1YShrLXUsUCkpLGM9dStQLHA9ZCk6dz4wJiYoUD1hYShFLWQsdWEoay1kLFApKSxjPXUs
cD1kK1ApKTticmVhaztjYXNlIGlhOngmJihhPWFhKFMsdWEoTixpLUMqeCkpLGg9YWEoUyx1YShO
LGwrQyp4KSkpLHcmJihjPWFhKEUsdWEoayx1LVAqdykpLHA9YWEoRSx1YShrLGQrUCp3KSkpfWg8
YSYmKHgqPS0xLHI9aSxpPWwsbD1yLHI9YSxhPWgsaD1yLGIgaW4gcGEmJkYuYXR0cigiY3Vyc29y
IixkYVtiPXBhW2JdXSkpLHA8YyYmKHcqPS0xLHI9dSx1PWQsZD1yLHI9YyxjPXAscD1yLGIgaW4g
Z2EmJkYuYXR0cigiY3Vyc29yIixkYVtiPWdhW2JdXSkpLE0uc2VsZWN0aW9uJiYoQT1NLnNlbGVj
dGlvbikseSYmKGE9QVswXVswXSxoPUFbMV1bMF0pLHYmJihjPUFbMF1bMV0scD1BWzFdWzFdKSxB
WzBdWzBdPT09YSYmQVswXVsxXT09PWMmJkFbMV1bMF09PT1oJiZBWzFdWzFdPT09cHx8KE0uc2Vs
ZWN0aW9uPVtbYSxjXSxbaCxwXV0sZi5jYWxsKF8pLEQuYnJ1c2godCxtLm5hbWUpKX1mdW5jdGlv
biBPKHQpe2lmKGZ1bmN0aW9uKHQpe3Quc3RvcEltbWVkaWF0ZVByb3BhZ2F0aW9uKCl9KHQpLHQu
dG91Y2hlcyl7aWYodC50b3VjaGVzLmxlbmd0aClyZXR1cm47biYmY2xlYXJUaW1lb3V0KG4pLG49
c2V0VGltZW91dCgoZnVuY3Rpb24oKXtuPW51bGx9KSw1MDApfWVsc2UgY2UodC52aWV3LGcpLHEu
b24oImtleWRvd24uYnJ1c2gga2V5dXAuYnJ1c2ggbW91c2Vtb3ZlLmJydXNoIG1vdXNldXAuYnJ1
c2giLG51bGwpO1IuYXR0cigicG9pbnRlci1ldmVudHMiLCJhbGwiKSxGLmF0dHIoImN1cnNvciIs
ZGEub3ZlcmxheSksTS5zZWxlY3Rpb24mJihBPU0uc2VsZWN0aW9uKSxmdW5jdGlvbih0KXtyZXR1
cm4gdFswXVswXT09PXRbMV1bMF18fHRbMF1bMV09PT10WzFdWzFdfShBKSYmKE0uc2VsZWN0aW9u
PW51bGwsZi5jYWxsKF8pKSxELmVuZCh0LG0ubmFtZSl9fWZ1bmN0aW9uIGQodCl7cyh0aGlzLGFy
Z3VtZW50cykubW92ZWQodCl9ZnVuY3Rpb24gcCh0KXtzKHRoaXMsYXJndW1lbnRzKS5lbmRlZCh0
KX1mdW5jdGlvbiBnKCl7dmFyIG49dGhpcy5fX2JydXNofHx7c2VsZWN0aW9uOm51bGx9O3JldHVy
biBuLmV4dGVudD1mYShlLmFwcGx5KHRoaXMsYXJndW1lbnRzKSksbi5kaW09dCxufXJldHVybiBj
Lm1vdmU9ZnVuY3Rpb24obixlLHIpe24udHdlZW4/bi5vbigic3RhcnQuYnJ1c2giLChmdW5jdGlv
bih0KXtzKHRoaXMsYXJndW1lbnRzKS5iZWZvcmVzdGFydCgpLnN0YXJ0KHQpfSkpLm9uKCJpbnRl
cnJ1cHQuYnJ1c2ggZW5kLmJydXNoIiwoZnVuY3Rpb24odCl7cyh0aGlzLGFyZ3VtZW50cykuZW5k
KHQpfSkpLnR3ZWVuKCJicnVzaCIsKGZ1bmN0aW9uKCl7dmFyIG49dGhpcyxyPW4uX19icnVzaCxp
PXMobixhcmd1bWVudHMpLG89ci5zZWxlY3Rpb24sYT10LmlucHV0KCJmdW5jdGlvbiI9PXR5cGVv
ZiBlP2UuYXBwbHkodGhpcyxhcmd1bWVudHMpOmUsci5leHRlbnQpLHU9VnIobyxhKTtmdW5jdGlv
biBjKHQpe3Iuc2VsZWN0aW9uPTE9PT10JiZudWxsPT09YT9udWxsOnUodCksZi5jYWxsKG4pLGku
YnJ1c2goKX1yZXR1cm4gbnVsbCE9PW8mJm51bGwhPT1hP2M6YygxKX0pKTpuLmVhY2goKGZ1bmN0
aW9uKCl7dmFyIG49dGhpcyxpPWFyZ3VtZW50cyxvPW4uX19icnVzaCxhPXQuaW5wdXQoImZ1bmN0
aW9uIj09dHlwZW9mIGU/ZS5hcHBseShuLGkpOmUsby5leHRlbnQpLHU9cyhuLGkpLmJlZm9yZXN0
YXJ0KCk7Vmkobiksby5zZWxlY3Rpb249bnVsbD09PWE/bnVsbDphLGYuY2FsbChuKSx1LnN0YXJ0
KHIpLmJydXNoKHIpLmVuZChyKX0pKX0sYy5jbGVhcj1mdW5jdGlvbih0LG4pe2MubW92ZSh0LG51
bGwsbil9LGwucHJvdG90eXBlPXtiZWZvcmVzdGFydDpmdW5jdGlvbigpe3JldHVybiAxPT0rK3Ro
aXMuYWN0aXZlJiYodGhpcy5zdGF0ZS5lbWl0dGVyPXRoaXMsdGhpcy5zdGFydGluZz0hMCksdGhp
c30sc3RhcnQ6ZnVuY3Rpb24odCxuKXtyZXR1cm4gdGhpcy5zdGFydGluZz8odGhpcy5zdGFydGlu
Zz0hMSx0aGlzLmVtaXQoInN0YXJ0Iix0LG4pKTp0aGlzLmVtaXQoImJydXNoIix0KSx0aGlzfSxi
cnVzaDpmdW5jdGlvbih0LG4pe3JldHVybiB0aGlzLmVtaXQoImJydXNoIix0LG4pLHRoaXN9LGVu
ZDpmdW5jdGlvbih0LG4pe3JldHVybiAwPT0tLXRoaXMuYWN0aXZlJiYoZGVsZXRlIHRoaXMuc3Rh
dGUuZW1pdHRlcix0aGlzLmVtaXQoImVuZCIsdCxuKSksdGhpc30sZW1pdDpmdW5jdGlvbihuLGUs
cil7dmFyIGk9S24odGhpcy50aGF0KS5kYXR1bSgpO2EuY2FsbChuLHRoaXMudGhhdCxuZXcgSm8o
bix7c291cmNlRXZlbnQ6ZSx0YXJnZXQ6YyxzZWxlY3Rpb246dC5vdXRwdXQodGhpcy5zdGF0ZS5z
ZWxlY3Rpb24pLG1vZGU6cixkaXNwYXRjaDphfSksaSl9fSxjLmV4dGVudD1mdW5jdGlvbih0KXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oZT0iZnVuY3Rpb24iPT10eXBlb2YgdD90OlFvKGZhKHQp
KSxjKTplfSxjLmZpbHRlcj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj0i
ZnVuY3Rpb24iPT10eXBlb2YgdD90OlFvKCEhdCksYyk6cn0sYy50b3VjaGFibGU9ZnVuY3Rpb24o
dCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGk9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpRbygh
IXQpLGMpOml9LGMuaGFuZGxlU2l6ZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0
aD8odT0rdCxjKTp1fSxjLmtleU1vZGlmaWVycz1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD8obz0hIXQsYyk6b30sYy5vbj1mdW5jdGlvbigpe3ZhciB0PWEub24uYXBwbHkoYSxh
cmd1bWVudHMpO3JldHVybiB0PT09YT9jOnR9LGN9dmFyIFRhPU1hdGguYWJzLEFhPU1hdGguY29z
LFNhPU1hdGguc2luLEVhPU1hdGguUEksTmE9RWEvMixrYT0yKkVhLENhPU1hdGgubWF4LFBhPTFl
LTEyO2Z1bmN0aW9uIHphKHQsbil7cmV0dXJuIEFycmF5LmZyb20oe2xlbmd0aDpuLXR9LCgobixl
KT0+dCtlKSl9ZnVuY3Rpb24gJGEodCxuKXt2YXIgZT0wLHI9bnVsbCxpPW51bGwsbz1udWxsO2Z1
bmN0aW9uIGEoYSl7dmFyIHUsYz1hLmxlbmd0aCxmPW5ldyBBcnJheShjKSxzPXphKDAsYyksbD1u
ZXcgQXJyYXkoYypjKSxoPW5ldyBBcnJheShjKSxkPTA7YT1GbG9hdDY0QXJyYXkuZnJvbSh7bGVu
Z3RoOmMqY30sbj8odCxuKT0+YVtuJWNdW24vY3wwXToodCxuKT0+YVtuL2N8MF1bbiVjXSk7Zm9y
KGxldCBuPTA7bjxjOysrbil7bGV0IGU9MDtmb3IobGV0IHI9MDtyPGM7KytyKWUrPWFbbipjK3Jd
K3QqYVtyKmMrbl07ZCs9ZltuXT1lfXU9KGQ9Q2EoMCxrYS1lKmMpL2QpP2U6a2EvYzt7bGV0IG49
MDtyJiZzLnNvcnQoKCh0LG4pPT5yKGZbdF0sZltuXSkpKTtmb3IoY29uc3QgZSBvZiBzKXtjb25z
dCByPW47aWYodCl7Y29uc3QgdD16YSgxK35jLGMpLmZpbHRlcigodD0+dDwwP2FbfnQqYytlXTph
W2UqYyt0XSkpO2kmJnQuc29ydCgoKHQsbik9PmkodDwwPy1hW350KmMrZV06YVtlKmMrdF0sbjww
Py1hW35uKmMrZV06YVtlKmMrbl0pKSk7Zm9yKGNvbnN0IHIgb2YgdClpZihyPDApeyhsW35yKmMr
ZV18fChsW35yKmMrZV09e3NvdXJjZTpudWxsLHRhcmdldDpudWxsfSkpLnRhcmdldD17aW5kZXg6
ZSxzdGFydEFuZ2xlOm4sZW5kQW5nbGU6bis9YVt+cipjK2VdKmQsdmFsdWU6YVt+cipjK2VdfX1l
bHNleyhsW2UqYytyXXx8KGxbZSpjK3JdPXtzb3VyY2U6bnVsbCx0YXJnZXQ6bnVsbH0pKS5zb3Vy
Y2U9e2luZGV4OmUsc3RhcnRBbmdsZTpuLGVuZEFuZ2xlOm4rPWFbZSpjK3JdKmQsdmFsdWU6YVtl
KmMrcl19fWhbZV09e2luZGV4OmUsc3RhcnRBbmdsZTpyLGVuZEFuZ2xlOm4sdmFsdWU6ZltlXX19
ZWxzZXtjb25zdCB0PXphKDAsYykuZmlsdGVyKCh0PT5hW2UqYyt0XXx8YVt0KmMrZV0pKTtpJiZ0
LnNvcnQoKCh0LG4pPT5pKGFbZSpjK3RdLGFbZSpjK25dKSkpO2Zvcihjb25zdCByIG9mIHQpe2xl
dCB0O2lmKGU8cj8odD1sW2UqYytyXXx8KGxbZSpjK3JdPXtzb3VyY2U6bnVsbCx0YXJnZXQ6bnVs
bH0pLHQuc291cmNlPXtpbmRleDplLHN0YXJ0QW5nbGU6bixlbmRBbmdsZTpuKz1hW2UqYytyXSpk
LHZhbHVlOmFbZSpjK3JdfSk6KHQ9bFtyKmMrZV18fChsW3IqYytlXT17c291cmNlOm51bGwsdGFy
Z2V0Om51bGx9KSx0LnRhcmdldD17aW5kZXg6ZSxzdGFydEFuZ2xlOm4sZW5kQW5nbGU6bis9YVtl
KmMrcl0qZCx2YWx1ZTphW2UqYytyXX0sZT09PXImJih0LnNvdXJjZT10LnRhcmdldCkpLHQuc291
cmNlJiZ0LnRhcmdldCYmdC5zb3VyY2UudmFsdWU8dC50YXJnZXQudmFsdWUpe2NvbnN0IG49dC5z
b3VyY2U7dC5zb3VyY2U9dC50YXJnZXQsdC50YXJnZXQ9bn19aFtlXT17aW5kZXg6ZSxzdGFydEFu
Z2xlOnIsZW5kQW5nbGU6bix2YWx1ZTpmW2VdfX1uKz11fX1yZXR1cm4obD1PYmplY3QudmFsdWVz
KGwpKS5ncm91cHM9aCxvP2wuc29ydChvKTpsfXJldHVybiBhLnBhZEFuZ2xlPWZ1bmN0aW9uKHQp
e3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlPUNhKDAsdCksYSk6ZX0sYS5zb3J0R3JvdXBzPWZ1
bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhyPXQsYSk6cn0sYS5zb3J0U3ViZ3Jv
dXBzPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhpPXQsYSk6aX0sYS5zb3J0
Q2hvcmRzPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhudWxsPT10P289bnVs
bDoobj10LG89ZnVuY3Rpb24odCxlKXtyZXR1cm4gbih0LnNvdXJjZS52YWx1ZSt0LnRhcmdldC52
YWx1ZSxlLnNvdXJjZS52YWx1ZStlLnRhcmdldC52YWx1ZSl9KS5fPXQsYSk6byYmby5fO3ZhciBu
fSxhfWNvbnN0IERhPU1hdGguUEksUmE9MipEYSxGYT0xZS02LHFhPVJhLUZhO2Z1bmN0aW9uIFVh
KHQpe3RoaXMuXys9dFswXTtmb3IobGV0IG49MSxlPXQubGVuZ3RoO248ZTsrK24pdGhpcy5fKz1h
cmd1bWVudHNbbl0rdFtuXX1sZXQgSWE9Y2xhc3N7Y29uc3RydWN0b3IodCl7dGhpcy5feDA9dGhp
cy5feTA9dGhpcy5feDE9dGhpcy5feTE9bnVsbCx0aGlzLl89IiIsdGhpcy5fYXBwZW5kPW51bGw9
PXQ/VWE6ZnVuY3Rpb24odCl7bGV0IG49TWF0aC5mbG9vcih0KTtpZighKG4+PTApKXRocm93IG5l
dyBFcnJvcihgaW52YWxpZCBkaWdpdHM6ICR7dH1gKTtpZihuPjE1KXJldHVybiBVYTtjb25zdCBl
PTEwKipuO3JldHVybiBmdW5jdGlvbih0KXt0aGlzLl8rPXRbMF07Zm9yKGxldCBuPTEscj10Lmxl
bmd0aDtuPHI7KytuKXRoaXMuXys9TWF0aC5yb3VuZChhcmd1bWVudHNbbl0qZSkvZSt0W25dfX0o
dCl9bW92ZVRvKHQsbil7dGhpcy5fYXBwZW5kYE0ke3RoaXMuX3gwPXRoaXMuX3gxPSt0fSwke3Ro
aXMuX3kwPXRoaXMuX3kxPStufWB9Y2xvc2VQYXRoKCl7bnVsbCE9PXRoaXMuX3gxJiYodGhpcy5f
eDE9dGhpcy5feDAsdGhpcy5feTE9dGhpcy5feTAsdGhpcy5fYXBwZW5kYFpgKX1saW5lVG8odCxu
KXt0aGlzLl9hcHBlbmRgTCR7dGhpcy5feDE9K3R9LCR7dGhpcy5feTE9K259YH1xdWFkcmF0aWND
dXJ2ZVRvKHQsbixlLHIpe3RoaXMuX2FwcGVuZGBRJHsrdH0sJHsrbn0sJHt0aGlzLl94MT0rZX0s
JHt0aGlzLl95MT0rcn1gfWJlemllckN1cnZlVG8odCxuLGUscixpLG8pe3RoaXMuX2FwcGVuZGBD
JHsrdH0sJHsrbn0sJHsrZX0sJHsrcn0sJHt0aGlzLl94MT0raX0sJHt0aGlzLl95MT0rb31gfWFy
Y1RvKHQsbixlLHIsaSl7aWYodD0rdCxuPStuLGU9K2Uscj0rciwoaT0raSk8MCl0aHJvdyBuZXcg
RXJyb3IoYG5lZ2F0aXZlIHJhZGl1czogJHtpfWApO2xldCBvPXRoaXMuX3gxLGE9dGhpcy5feTEs
dT1lLXQsYz1yLW4sZj1vLXQscz1hLW4sbD1mKmYrcypzO2lmKG51bGw9PT10aGlzLl94MSl0aGlz
Ll9hcHBlbmRgTSR7dGhpcy5feDE9dH0sJHt0aGlzLl95MT1ufWA7ZWxzZSBpZihsPkZhKWlmKE1h
dGguYWJzKHMqdS1jKmYpPkZhJiZpKXtsZXQgaD1lLW8sZD1yLWEscD11KnUrYypjLGc9aCpoK2Qq
ZCx5PU1hdGguc3FydChwKSx2PU1hdGguc3FydChsKSxfPWkqTWF0aC50YW4oKERhLU1hdGguYWNv
cygocCtsLWcpLygyKnkqdikpKS8yKSxiPV8vdixtPV8veTtNYXRoLmFicyhiLTEpPkZhJiZ0aGlz
Ll9hcHBlbmRgTCR7dCtiKmZ9LCR7bitiKnN9YCx0aGlzLl9hcHBlbmRgQSR7aX0sJHtpfSwwLDAs
JHsrKHMqaD5mKmQpfSwke3RoaXMuX3gxPXQrbSp1fSwke3RoaXMuX3kxPW4rbSpjfWB9ZWxzZSB0
aGlzLl9hcHBlbmRgTCR7dGhpcy5feDE9dH0sJHt0aGlzLl95MT1ufWA7ZWxzZTt9YXJjKHQsbixl
LHIsaSxvKXtpZih0PSt0LG49K24sbz0hIW8sKGU9K2UpPDApdGhyb3cgbmV3IEVycm9yKGBuZWdh
dGl2ZSByYWRpdXM6ICR7ZX1gKTtsZXQgYT1lKk1hdGguY29zKHIpLHU9ZSpNYXRoLnNpbihyKSxj
PXQrYSxmPW4rdSxzPTFebyxsPW8/ci1pOmktcjtudWxsPT09dGhpcy5feDE/dGhpcy5fYXBwZW5k
YE0ke2N9LCR7Zn1gOihNYXRoLmFicyh0aGlzLl94MS1jKT5GYXx8TWF0aC5hYnModGhpcy5feTEt
Zik+RmEpJiZ0aGlzLl9hcHBlbmRgTCR7Y30sJHtmfWAsZSYmKGw8MCYmKGw9bCVSYStSYSksbD5x
YT90aGlzLl9hcHBlbmRgQSR7ZX0sJHtlfSwwLDEsJHtzfSwke3QtYX0sJHtuLXV9QSR7ZX0sJHtl
fSwwLDEsJHtzfSwke3RoaXMuX3gxPWN9LCR7dGhpcy5feTE9Zn1gOmw+RmEmJnRoaXMuX2FwcGVu
ZGBBJHtlfSwke2V9LDAsJHsrKGw+PURhKX0sJHtzfSwke3RoaXMuX3gxPXQrZSpNYXRoLmNvcyhp
KX0sJHt0aGlzLl95MT1uK2UqTWF0aC5zaW4oaSl9YCl9cmVjdCh0LG4sZSxyKXt0aGlzLl9hcHBl
bmRgTSR7dGhpcy5feDA9dGhpcy5feDE9K3R9LCR7dGhpcy5feTA9dGhpcy5feTE9K259aCR7ZT0r
ZX12JHsrcn1oJHstZX1aYH10b1N0cmluZygpe3JldHVybiB0aGlzLl99fTtmdW5jdGlvbiBPYSgp
e3JldHVybiBuZXcgSWF9T2EucHJvdG90eXBlPUlhLnByb3RvdHlwZTt2YXIgQmE9QXJyYXkucHJv
dG90eXBlLnNsaWNlO2Z1bmN0aW9uIFlhKHQpe3JldHVybiBmdW5jdGlvbigpe3JldHVybiB0fX1m
dW5jdGlvbiBMYSh0KXtyZXR1cm4gdC5zb3VyY2V9ZnVuY3Rpb24gamEodCl7cmV0dXJuIHQudGFy
Z2V0fWZ1bmN0aW9uIEhhKHQpe3JldHVybiB0LnJhZGl1c31mdW5jdGlvbiBYYSh0KXtyZXR1cm4g
dC5zdGFydEFuZ2xlfWZ1bmN0aW9uIEdhKHQpe3JldHVybiB0LmVuZEFuZ2xlfWZ1bmN0aW9uIFZh
KCl7cmV0dXJuIDB9ZnVuY3Rpb24gV2EoKXtyZXR1cm4gMTB9ZnVuY3Rpb24gWmEodCl7dmFyIG49
TGEsZT1qYSxyPUhhLGk9SGEsbz1YYSxhPUdhLHU9VmEsYz1udWxsO2Z1bmN0aW9uIGYoKXt2YXIg
ZixzPW4uYXBwbHkodGhpcyxhcmd1bWVudHMpLGw9ZS5hcHBseSh0aGlzLGFyZ3VtZW50cyksaD11
LmFwcGx5KHRoaXMsYXJndW1lbnRzKS8yLGQ9QmEuY2FsbChhcmd1bWVudHMpLHA9K3IuYXBwbHko
dGhpcywoZFswXT1zLGQpKSxnPW8uYXBwbHkodGhpcyxkKS1OYSx5PWEuYXBwbHkodGhpcyxkKS1O
YSx2PStpLmFwcGx5KHRoaXMsKGRbMF09bCxkKSksXz1vLmFwcGx5KHRoaXMsZCktTmEsYj1hLmFw
cGx5KHRoaXMsZCktTmE7aWYoY3x8KGM9Zj1PYSgpKSxoPlBhJiYoVGEoeS1nKT4yKmgrUGE/eT5n
PyhnKz1oLHktPWgpOihnLT1oLHkrPWgpOmc9eT0oZyt5KS8yLFRhKGItXyk+MipoK1BhP2I+Xz8o
Xys9aCxiLT1oKTooXy09aCxiKz1oKTpfPWI9KF8rYikvMiksYy5tb3ZlVG8ocCpBYShnKSxwKlNh
KGcpKSxjLmFyYygwLDAscCxnLHkpLGchPT1ffHx5IT09YilpZih0KXt2YXIgbT12LSt0LmFwcGx5
KHRoaXMsYXJndW1lbnRzKSx4PShfK2IpLzI7Yy5xdWFkcmF0aWNDdXJ2ZVRvKDAsMCxtKkFhKF8p
LG0qU2EoXykpLGMubGluZVRvKHYqQWEoeCksdipTYSh4KSksYy5saW5lVG8obSpBYShiKSxtKlNh
KGIpKX1lbHNlIGMucXVhZHJhdGljQ3VydmVUbygwLDAsdipBYShfKSx2KlNhKF8pKSxjLmFyYygw
LDAsdixfLGIpO2lmKGMucXVhZHJhdGljQ3VydmVUbygwLDAscCpBYShnKSxwKlNhKGcpKSxjLmNs
b3NlUGF0aCgpLGYpcmV0dXJuIGM9bnVsbCxmKyIifHxudWxsfXJldHVybiB0JiYoZi5oZWFkUmFk
aXVzPWZ1bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh0PSJmdW5jdGlvbiI9PXR5
cGVvZiBuP246WWEoK24pLGYpOnR9KSxmLnJhZGl1cz1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1l
bnRzLmxlbmd0aD8ocj1pPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6WWEoK3QpLGYpOnJ9LGYuc291
cmNlUmFkaXVzPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhyPSJmdW5jdGlv
biI9PXR5cGVvZiB0P3Q6WWEoK3QpLGYpOnJ9LGYudGFyZ2V0UmFkaXVzPWZ1bmN0aW9uKHQpe3Jl
dHVybiBhcmd1bWVudHMubGVuZ3RoPyhpPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6WWEoK3QpLGYp
Oml9LGYuc3RhcnRBbmdsZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obz0i
ZnVuY3Rpb24iPT10eXBlb2YgdD90OllhKCt0KSxmKTpvfSxmLmVuZEFuZ2xlPWZ1bmN0aW9uKHQp
e3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhhPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6WWEoK3Qp
LGYpOmF9LGYucGFkQW5nbGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHU9
ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpZYSgrdCksZik6dX0sZi5zb3VyY2U9ZnVuY3Rpb24odCl7
cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49dCxmKTpufSxmLnRhcmdldD1mdW5jdGlvbih0KXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oZT10LGYpOmV9LGYuY29udGV4dD1mdW5jdGlvbih0KXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYz1udWxsPT10P251bGw6dCxmKTpjfSxmfXZhciBLYT1B
cnJheS5wcm90b3R5cGUuc2xpY2U7ZnVuY3Rpb24gUWEodCxuKXtyZXR1cm4gdC1ufXZhciBKYT10
PT4oKT0+dDtmdW5jdGlvbiB0dSh0LG4pe2Zvcih2YXIgZSxyPS0xLGk9bi5sZW5ndGg7KytyPGk7
KWlmKGU9bnUodCxuW3JdKSlyZXR1cm4gZTtyZXR1cm4gMH1mdW5jdGlvbiBudSh0LG4pe2Zvcih2
YXIgZT1uWzBdLHI9blsxXSxpPS0xLG89MCxhPXQubGVuZ3RoLHU9YS0xO288YTt1PW8rKyl7dmFy
IGM9dFtvXSxmPWNbMF0scz1jWzFdLGw9dFt1XSxoPWxbMF0sZD1sWzFdO2lmKGV1KGMsbCxuKSly
ZXR1cm4gMDtzPnIhPWQ+ciYmZTwoaC1mKSooci1zKS8oZC1zKStmJiYoaT0taSl9cmV0dXJuIGl9
ZnVuY3Rpb24gZXUodCxuLGUpe3ZhciByLGksbyxhO3JldHVybiBmdW5jdGlvbih0LG4sZSl7cmV0
dXJuKG5bMF0tdFswXSkqKGVbMV0tdFsxXSk9PShlWzBdLXRbMF0pKihuWzFdLXRbMV0pfSh0LG4s
ZSkmJihpPXRbcj0rKHRbMF09PT1uWzBdKV0sbz1lW3JdLGE9bltyXSxpPD1vJiZvPD1hfHxhPD1v
JiZvPD1pKX1mdW5jdGlvbiBydSgpe312YXIgaXU9W1tdLFtbWzEsMS41XSxbLjUsMV1dXSxbW1sx
LjUsMV0sWzEsMS41XV1dLFtbWzEuNSwxXSxbLjUsMV1dXSxbW1sxLC41XSxbMS41LDFdXV0sW1tb
MSwxLjVdLFsuNSwxXV0sW1sxLC41XSxbMS41LDFdXV0sW1tbMSwuNV0sWzEsMS41XV1dLFtbWzEs
LjVdLFsuNSwxXV1dLFtbWy41LDFdLFsxLC41XV1dLFtbWzEsMS41XSxbMSwuNV1dXSxbW1suNSwx
XSxbMSwuNV1dLFtbMS41LDFdLFsxLDEuNV1dXSxbW1sxLjUsMV0sWzEsLjVdXV0sW1tbLjUsMV0s
WzEuNSwxXV1dLFtbWzEsMS41XSxbMS41LDFdXV0sW1tbLjUsMV0sWzEsMS41XV1dLFtdXTtmdW5j
dGlvbiBvdSgpe3ZhciB0PTEsbj0xLGU9USxyPXU7ZnVuY3Rpb24gaSh0KXt2YXIgbj1lKHQpO2lm
KEFycmF5LmlzQXJyYXkobikpbj1uLnNsaWNlKCkuc29ydChRYSk7ZWxzZXtjb25zdCBlPVQodCxh
dSk7Zm9yKG49ViguLi5LKGVbMF0sZVsxXSxuKSxuKTtuW24ubGVuZ3RoLTFdPj1lWzFdOyluLnBv
cCgpO2Zvcig7blsxXTxlWzBdOyluLnNoaWZ0KCl9cmV0dXJuIG4ubWFwKChuPT5vKHQsbikpKX1m
dW5jdGlvbiBvKGUsaSl7Y29uc3Qgbz1udWxsPT1pP05hTjoraTtpZihpc05hTihvKSl0aHJvdyBu
ZXcgRXJyb3IoYGludmFsaWQgdmFsdWU6ICR7aX1gKTt2YXIgdT1bXSxjPVtdO3JldHVybiBmdW5j
dGlvbihlLHIsaSl7dmFyIG8sdSxjLGYscyxsLGg9bmV3IEFycmF5LGQ9bmV3IEFycmF5O289dT0t
MSxmPXV1KGVbMF0sciksaXVbZjw8MV0uZm9yRWFjaChwKTtmb3IoOysrbzx0LTE7KWM9ZixmPXV1
KGVbbysxXSxyKSxpdVtjfGY8PDFdLmZvckVhY2gocCk7aXVbZjw8MF0uZm9yRWFjaChwKTtmb3Io
OysrdTxuLTE7KXtmb3Iobz0tMSxmPXV1KGVbdSp0K3RdLHIpLHM9dXUoZVt1KnRdLHIpLGl1W2Y8
PDF8czw8Ml0uZm9yRWFjaChwKTsrK288dC0xOyljPWYsZj11dShlW3UqdCt0K28rMV0sciksbD1z
LHM9dXUoZVt1KnQrbysxXSxyKSxpdVtjfGY8PDF8czw8MnxsPDwzXS5mb3JFYWNoKHApO2l1W2Z8
czw8M10uZm9yRWFjaChwKX1vPS0xLHM9ZVt1KnRdPj1yLGl1W3M8PDJdLmZvckVhY2gocCk7Zm9y
KDsrK288dC0xOylsPXMscz11dShlW3UqdCtvKzFdLHIpLGl1W3M8PDJ8bDw8M10uZm9yRWFjaChw
KTtmdW5jdGlvbiBwKHQpe3ZhciBuLGUscj1bdFswXVswXStvLHRbMF1bMV0rdV0sYz1bdFsxXVsw
XStvLHRbMV1bMV0rdV0sZj1hKHIpLHM9YShjKTsobj1kW2ZdKT8oZT1oW3NdKT8oZGVsZXRlIGRb
bi5lbmRdLGRlbGV0ZSBoW2Uuc3RhcnRdLG49PT1lPyhuLnJpbmcucHVzaChjKSxpKG4ucmluZykp
Omhbbi5zdGFydF09ZFtlLmVuZF09e3N0YXJ0Om4uc3RhcnQsZW5kOmUuZW5kLHJpbmc6bi5yaW5n
LmNvbmNhdChlLnJpbmcpfSk6KGRlbGV0ZSBkW24uZW5kXSxuLnJpbmcucHVzaChjKSxkW24uZW5k
PXNdPW4pOihuPWhbc10pPyhlPWRbZl0pPyhkZWxldGUgaFtuLnN0YXJ0XSxkZWxldGUgZFtlLmVu
ZF0sbj09PWU/KG4ucmluZy5wdXNoKGMpLGkobi5yaW5nKSk6aFtlLnN0YXJ0XT1kW24uZW5kXT17
c3RhcnQ6ZS5zdGFydCxlbmQ6bi5lbmQscmluZzplLnJpbmcuY29uY2F0KG4ucmluZyl9KTooZGVs
ZXRlIGhbbi5zdGFydF0sbi5yaW5nLnVuc2hpZnQociksaFtuLnN0YXJ0PWZdPW4pOmhbZl09ZFtz
XT17c3RhcnQ6ZixlbmQ6cyxyaW5nOltyLGNdfX1pdVtzPDwzXS5mb3JFYWNoKHApfShlLG8sKGZ1
bmN0aW9uKHQpe3IodCxlLG8pLGZ1bmN0aW9uKHQpe2Zvcih2YXIgbj0wLGU9dC5sZW5ndGgscj10
W2UtMV1bMV0qdFswXVswXS10W2UtMV1bMF0qdFswXVsxXTsrK248ZTspcis9dFtuLTFdWzFdKnRb
bl1bMF0tdFtuLTFdWzBdKnRbbl1bMV07cmV0dXJuIHJ9KHQpPjA/dS5wdXNoKFt0XSk6Yy5wdXNo
KHQpfSkpLGMuZm9yRWFjaCgoZnVuY3Rpb24odCl7Zm9yKHZhciBuLGU9MCxyPXUubGVuZ3RoO2U8
cjsrK2UpaWYoLTEhPT10dSgobj11W2VdKVswXSx0KSlyZXR1cm4gdm9pZCBuLnB1c2godCl9KSks
e3R5cGU6Ik11bHRpUG9seWdvbiIsdmFsdWU6aSxjb29yZGluYXRlczp1fX1mdW5jdGlvbiBhKG4p
e3JldHVybiAyKm5bMF0rblsxXSoodCsxKSo0fWZ1bmN0aW9uIHUoZSxyLGkpe2UuZm9yRWFjaCgo
ZnVuY3Rpb24oZSl7dmFyIG89ZVswXSxhPWVbMV0sdT0wfG8sYz0wfGEsZj1jdShyW2MqdCt1XSk7
bz4wJiZvPHQmJnU9PT1vJiYoZVswXT1mdShvLGN1KHJbYyp0K3UtMV0pLGYsaSkpLGE+MCYmYTxu
JiZjPT09YSYmKGVbMV09ZnUoYSxjdShyWyhjLTEpKnQrdV0pLGYsaSkpfSkpfXJldHVybiBpLmNv
bnRvdXI9byxpLnNpemU9ZnVuY3Rpb24oZSl7aWYoIWFyZ3VtZW50cy5sZW5ndGgpcmV0dXJuW3Qs
bl07dmFyIHI9TWF0aC5mbG9vcihlWzBdKSxvPU1hdGguZmxvb3IoZVsxXSk7aWYoIShyPj0wJiZv
Pj0wKSl0aHJvdyBuZXcgRXJyb3IoImludmFsaWQgc2l6ZSIpO3JldHVybiB0PXIsbj1vLGl9LGku
dGhyZXNob2xkcz1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oZT0iZnVuY3Rp
b24iPT10eXBlb2YgdD90OkFycmF5LmlzQXJyYXkodCk/SmEoS2EuY2FsbCh0KSk6SmEodCksaSk6
ZX0saS5zbW9vdGg9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9dD91OnJ1
LGkpOnI9PT11fSxpfWZ1bmN0aW9uIGF1KHQpe3JldHVybiBpc0Zpbml0ZSh0KT90Ok5hTn1mdW5j
dGlvbiB1dSh0LG4pe3JldHVybiBudWxsIT10JiYrdD49bn1mdW5jdGlvbiBjdSh0KXtyZXR1cm4g
bnVsbD09dHx8aXNOYU4odD0rdCk/LTEvMDp0fWZ1bmN0aW9uIGZ1KHQsbixlLHIpe2NvbnN0IGk9
ci1uLG89ZS1uLGE9aXNGaW5pdGUoaSl8fGlzRmluaXRlKG8pP2kvbzpNYXRoLnNpZ24oaSkvTWF0
aC5zaWduKG8pO3JldHVybiBpc05hTihhKT90OnQrYS0uNX1mdW5jdGlvbiBzdSh0KXtyZXR1cm4g
dFswXX1mdW5jdGlvbiBsdSh0KXtyZXR1cm4gdFsxXX1mdW5jdGlvbiBodSgpe3JldHVybiAxfWNv
bnN0IGR1PTEzNDIxNzcyOSxwdT0zMzMwNjY5MDczODc1NDcwNmUtMzI7ZnVuY3Rpb24gZ3UodCxu
LGUscixpKXtsZXQgbyxhLHUsYyxmPW5bMF0scz1yWzBdLGw9MCxoPTA7cz5mPT1zPi1mPyhvPWYs
Zj1uWysrbF0pOihvPXMscz1yWysraF0pO2xldCBkPTA7aWYobDx0JiZoPGUpZm9yKHM+Zj09cz4t
Zj8oYT1mK28sdT1vLShhLWYpLGY9blsrK2xdKTooYT1zK28sdT1vLShhLXMpLHM9clsrK2hdKSxv
PWEsMCE9PXUmJihpW2QrK109dSk7bDx0JiZoPGU7KXM+Zj09cz4tZj8oYT1vK2YsYz1hLW8sdT1v
LShhLWMpKyhmLWMpLGY9blsrK2xdKTooYT1vK3MsYz1hLW8sdT1vLShhLWMpKyhzLWMpLHM9clsr
K2hdKSxvPWEsMCE9PXUmJihpW2QrK109dSk7Zm9yKDtsPHQ7KWE9bytmLGM9YS1vLHU9by0oYS1j
KSsoZi1jKSxmPW5bKytsXSxvPWEsMCE9PXUmJihpW2QrK109dSk7Zm9yKDtoPGU7KWE9bytzLGM9
YS1vLHU9by0oYS1jKSsocy1jKSxzPXJbKytoXSxvPWEsMCE9PXUmJihpW2QrK109dSk7cmV0dXJu
IDA9PT1vJiYwIT09ZHx8KGlbZCsrXT1vKSxkfWZ1bmN0aW9uIHl1KHQpe3JldHVybiBuZXcgRmxv
YXQ2NEFycmF5KHQpfWNvbnN0IHZ1PTIyMjA0NDYwNDkyNTAzMTQ2ZS0zMixfdT0xMTA5MzM1NjQ3
OTY3MDQ4N2UtNDcsYnU9eXUoNCksbXU9eXUoOCkseHU9eXUoMTIpLHd1PXl1KDE2KSxNdT15dSg0
KTtmdW5jdGlvbiBUdSh0LG4sZSxyLGksbyl7Y29uc3QgYT0obi1vKSooZS1pKSx1PSh0LWkpKihy
LW8pLGM9YS11O2lmKDA9PT1hfHwwPT09dXx8YT4wIT11PjApcmV0dXJuIGM7Y29uc3QgZj1NYXRo
LmFicyhhK3UpO3JldHVybiBNYXRoLmFicyhjKT49MzMzMDY2OTA3Mzg3NTQ3MTZlLTMyKmY/Yzot
ZnVuY3Rpb24odCxuLGUscixpLG8sYSl7bGV0IHUsYyxmLHMsbCxoLGQscCxnLHksdixfLGIsbSx4
LHcsTSxUO2NvbnN0IEE9dC1pLFM9ZS1pLEU9bi1vLE49ci1vO209QSpOLGg9ZHUqQSxkPWgtKGgt
QSkscD1BLWQsaD1kdSpOLGc9aC0oaC1OKSx5PU4tZyx4PXAqeS0obS1kKmctcCpnLWQqeSksdz1F
KlMsaD1kdSpFLGQ9aC0oaC1FKSxwPUUtZCxoPWR1KlMsZz1oLShoLVMpLHk9Uy1nLE09cCp5LSh3
LWQqZy1wKmctZCp5KSx2PXgtTSxsPXgtdixidVswXT14LSh2K2wpKyhsLU0pLF89bSt2LGw9Xy1t
LGI9bS0oXy1sKSsodi1sKSx2PWItdyxsPWItdixidVsxXT1iLSh2K2wpKyhsLXcpLFQ9Xyt2LGw9
VC1fLGJ1WzJdPV8tKFQtbCkrKHYtbCksYnVbM109VDtsZXQgaz1mdW5jdGlvbih0LG4pe2xldCBl
PW5bMF07Zm9yKGxldCByPTE7cjx0O3IrKyllKz1uW3JdO3JldHVybiBlfSg0LGJ1KSxDPXZ1KmE7
aWYoaz49Q3x8LWs+PUMpcmV0dXJuIGs7aWYobD10LUEsdT10LShBK2wpKyhsLWkpLGw9ZS1TLGY9
ZS0oUytsKSsobC1pKSxsPW4tRSxjPW4tKEUrbCkrKGwtbyksbD1yLU4scz1yLShOK2wpKyhsLW8p
LDA9PT11JiYwPT09YyYmMD09PWYmJjA9PT1zKXJldHVybiBrO2lmKEM9X3UqYStwdSpNYXRoLmFi
cyhrKSxrKz1BKnMrTip1LShFKmYrUypjKSxrPj1DfHwtaz49QylyZXR1cm4gazttPXUqTixoPWR1
KnUsZD1oLShoLXUpLHA9dS1kLGg9ZHUqTixnPWgtKGgtTikseT1OLWcseD1wKnktKG0tZCpnLXAq
Zy1kKnkpLHc9YypTLGg9ZHUqYyxkPWgtKGgtYykscD1jLWQsaD1kdSpTLGc9aC0oaC1TKSx5PVMt
ZyxNPXAqeS0ody1kKmctcCpnLWQqeSksdj14LU0sbD14LXYsTXVbMF09eC0oditsKSsobC1NKSxf
PW0rdixsPV8tbSxiPW0tKF8tbCkrKHYtbCksdj1iLXcsbD1iLXYsTXVbMV09Yi0oditsKSsobC13
KSxUPV8rdixsPVQtXyxNdVsyXT1fLShULWwpKyh2LWwpLE11WzNdPVQ7Y29uc3QgUD1ndSg0LGJ1
LDQsTXUsbXUpO209QSpzLGg9ZHUqQSxkPWgtKGgtQSkscD1BLWQsaD1kdSpzLGc9aC0oaC1zKSx5
PXMtZyx4PXAqeS0obS1kKmctcCpnLWQqeSksdz1FKmYsaD1kdSpFLGQ9aC0oaC1FKSxwPUUtZCxo
PWR1KmYsZz1oLShoLWYpLHk9Zi1nLE09cCp5LSh3LWQqZy1wKmctZCp5KSx2PXgtTSxsPXgtdixN
dVswXT14LSh2K2wpKyhsLU0pLF89bSt2LGw9Xy1tLGI9bS0oXy1sKSsodi1sKSx2PWItdyxsPWIt
dixNdVsxXT1iLSh2K2wpKyhsLXcpLFQ9Xyt2LGw9VC1fLE11WzJdPV8tKFQtbCkrKHYtbCksTXVb
M109VDtjb25zdCB6PWd1KFAsbXUsNCxNdSx4dSk7bT11KnMsaD1kdSp1LGQ9aC0oaC11KSxwPXUt
ZCxoPWR1KnMsZz1oLShoLXMpLHk9cy1nLHg9cCp5LShtLWQqZy1wKmctZCp5KSx3PWMqZixoPWR1
KmMsZD1oLShoLWMpLHA9Yy1kLGg9ZHUqZixnPWgtKGgtZikseT1mLWcsTT1wKnktKHctZCpnLXAq
Zy1kKnkpLHY9eC1NLGw9eC12LE11WzBdPXgtKHYrbCkrKGwtTSksXz1tK3YsbD1fLW0sYj1tLShf
LWwpKyh2LWwpLHY9Yi13LGw9Yi12LE11WzFdPWItKHYrbCkrKGwtdyksVD1fK3YsbD1ULV8sTXVb
Ml09Xy0oVC1sKSsodi1sKSxNdVszXT1UO2NvbnN0ICQ9Z3Uoeix4dSw0LE11LHd1KTtyZXR1cm4g
d3VbJC0xXX0odCxuLGUscixpLG8sZil9Y29uc3QgQXU9TWF0aC5wb3coMiwtNTIpLFN1PW5ldyBV
aW50MzJBcnJheSg1MTIpO2NsYXNzIEV1e3N0YXRpYyBmcm9tKHQsbj0kdSxlPUR1KXtjb25zdCBy
PXQubGVuZ3RoLGk9bmV3IEZsb2F0NjRBcnJheSgyKnIpO2ZvcihsZXQgbz0wO288cjtvKyspe2Nv
bnN0IHI9dFtvXTtpWzIqb109bihyKSxpWzIqbysxXT1lKHIpfXJldHVybiBuZXcgRXUoaSl9Y29u
c3RydWN0b3IodCl7Y29uc3Qgbj10Lmxlbmd0aD4+MTtpZihuPjAmJiJudW1iZXIiIT10eXBlb2Yg
dFswXSl0aHJvdyBuZXcgRXJyb3IoIkV4cGVjdGVkIGNvb3JkcyB0byBjb250YWluIG51bWJlcnMu
Iik7dGhpcy5jb29yZHM9dDtjb25zdCBlPU1hdGgubWF4KDIqbi01LDApO3RoaXMuX3RyaWFuZ2xl
cz1uZXcgVWludDMyQXJyYXkoMyplKSx0aGlzLl9oYWxmZWRnZXM9bmV3IEludDMyQXJyYXkoMypl
KSx0aGlzLl9oYXNoU2l6ZT1NYXRoLmNlaWwoTWF0aC5zcXJ0KG4pKSx0aGlzLl9odWxsUHJldj1u
ZXcgVWludDMyQXJyYXkobiksdGhpcy5faHVsbE5leHQ9bmV3IFVpbnQzMkFycmF5KG4pLHRoaXMu
X2h1bGxUcmk9bmV3IFVpbnQzMkFycmF5KG4pLHRoaXMuX2h1bGxIYXNoPW5ldyBJbnQzMkFycmF5
KHRoaXMuX2hhc2hTaXplKS5maWxsKC0xKSx0aGlzLl9pZHM9bmV3IFVpbnQzMkFycmF5KG4pLHRo
aXMuX2Rpc3RzPW5ldyBGbG9hdDY0QXJyYXkobiksdGhpcy51cGRhdGUoKX11cGRhdGUoKXtjb25z
dHtjb29yZHM6dCxfaHVsbFByZXY6bixfaHVsbE5leHQ6ZSxfaHVsbFRyaTpyLF9odWxsSGFzaDpp
fT10aGlzLG89dC5sZW5ndGg+PjE7bGV0IGE9MS8wLHU9MS8wLGM9LTEvMCxmPS0xLzA7Zm9yKGxl
dCBuPTA7bjxvO24rKyl7Y29uc3QgZT10WzIqbl0scj10WzIqbisxXTtlPGEmJihhPWUpLHI8dSYm
KHU9ciksZT5jJiYoYz1lKSxyPmYmJihmPXIpLHRoaXMuX2lkc1tuXT1ufWNvbnN0IHM9KGErYykv
MixsPSh1K2YpLzI7bGV0IGgsZCxwLGc9MS8wO2ZvcihsZXQgbj0wO248bztuKyspe2NvbnN0IGU9
TnUocyxsLHRbMipuXSx0WzIqbisxXSk7ZTxnJiYoaD1uLGc9ZSl9Y29uc3QgeT10WzIqaF0sdj10
WzIqaCsxXTtnPTEvMDtmb3IobGV0IG49MDtuPG87bisrKXtpZihuPT09aCljb250aW51ZTtjb25z
dCBlPU51KHksdix0WzIqbl0sdFsyKm4rMV0pO2U8ZyYmZT4wJiYoZD1uLGc9ZSl9bGV0IF89dFsy
KmRdLGI9dFsyKmQrMV0sbT0xLzA7Zm9yKGxldCBuPTA7bjxvO24rKyl7aWYobj09PWh8fG49PT1k
KWNvbnRpbnVlO2NvbnN0IGU9Q3UoeSx2LF8sYix0WzIqbl0sdFsyKm4rMV0pO2U8bSYmKHA9bixt
PWUpfWxldCB4PXRbMipwXSx3PXRbMipwKzFdO2lmKG09PT0xLzApe2ZvcihsZXQgbj0wO248bztu
KyspdGhpcy5fZGlzdHNbbl09dFsyKm5dLXRbMF18fHRbMipuKzFdLXRbMV07UHUodGhpcy5faWRz
LHRoaXMuX2Rpc3RzLDAsby0xKTtjb25zdCBuPW5ldyBVaW50MzJBcnJheShvKTtsZXQgZT0wO2Zv
cihsZXQgdD0wLHI9LTEvMDt0PG87dCsrKXtjb25zdCBpPXRoaXMuX2lkc1t0XTt0aGlzLl9kaXN0
c1tpXT5yJiYobltlKytdPWkscj10aGlzLl9kaXN0c1tpXSl9cmV0dXJuIHRoaXMuaHVsbD1uLnN1
YmFycmF5KDAsZSksdGhpcy50cmlhbmdsZXM9bmV3IFVpbnQzMkFycmF5KDApLHZvaWQodGhpcy5o
YWxmZWRnZXM9bmV3IFVpbnQzMkFycmF5KDApKX1pZihUdSh5LHYsXyxiLHgsdyk8MCl7Y29uc3Qg
dD1kLG49XyxlPWI7ZD1wLF89eCxiPXcscD10LHg9bix3PWV9Y29uc3QgTT1mdW5jdGlvbih0LG4s
ZSxyLGksbyl7Y29uc3QgYT1lLXQsdT1yLW4sYz1pLXQsZj1vLW4scz1hKmErdSp1LGw9YypjK2Yq
ZixoPS41LyhhKmYtdSpjKSxkPXQrKGYqcy11KmwpKmgscD1uKyhhKmwtYypzKSpoO3JldHVybnt4
OmQseTpwfX0oeSx2LF8sYix4LHcpO3RoaXMuX2N4PU0ueCx0aGlzLl9jeT1NLnk7Zm9yKGxldCBu
PTA7bjxvO24rKyl0aGlzLl9kaXN0c1tuXT1OdSh0WzIqbl0sdFsyKm4rMV0sTS54LE0ueSk7UHUo
dGhpcy5faWRzLHRoaXMuX2Rpc3RzLDAsby0xKSx0aGlzLl9odWxsU3RhcnQ9aDtsZXQgVD0zO2Vb
aF09bltwXT1kLGVbZF09bltoXT1wLGVbcF09bltkXT1oLHJbaF09MCxyW2RdPTEscltwXT0yLGku
ZmlsbCgtMSksaVt0aGlzLl9oYXNoS2V5KHksdildPWgsaVt0aGlzLl9oYXNoS2V5KF8sYildPWQs
aVt0aGlzLl9oYXNoS2V5KHgsdyldPXAsdGhpcy50cmlhbmdsZXNMZW49MCx0aGlzLl9hZGRUcmlh
bmdsZShoLGQscCwtMSwtMSwtMSk7Zm9yKGxldCBvLGEsdT0wO3U8dGhpcy5faWRzLmxlbmd0aDt1
Kyspe2NvbnN0IGM9dGhpcy5faWRzW3VdLGY9dFsyKmNdLHM9dFsyKmMrMV07aWYodT4wJiZNYXRo
LmFicyhmLW8pPD1BdSYmTWF0aC5hYnMocy1hKTw9QXUpY29udGludWU7aWYobz1mLGE9cyxjPT09
aHx8Yz09PWR8fGM9PT1wKWNvbnRpbnVlO2xldCBsPTA7Zm9yKGxldCB0PTAsbj10aGlzLl9oYXNo
S2V5KGYscyk7dDx0aGlzLl9oYXNoU2l6ZSYmKGw9aVsobit0KSV0aGlzLl9oYXNoU2l6ZV0sLTE9
PT1sfHxsPT09ZVtsXSk7dCsrKTtsPW5bbF07bGV0IGcseT1sO2Zvcig7Zz1lW3ldLFR1KGYscyx0
WzIqeV0sdFsyKnkrMV0sdFsyKmddLHRbMipnKzFdKT49MDspaWYoeT1nLHk9PT1sKXt5PS0xO2Jy
ZWFrfWlmKC0xPT09eSljb250aW51ZTtsZXQgdj10aGlzLl9hZGRUcmlhbmdsZSh5LGMsZVt5XSwt
MSwtMSxyW3ldKTtyW2NdPXRoaXMuX2xlZ2FsaXplKHYrMiksclt5XT12LFQrKztsZXQgXz1lW3ld
O2Zvcig7Zz1lW19dLFR1KGYscyx0WzIqX10sdFsyKl8rMV0sdFsyKmddLHRbMipnKzFdKTwwOyl2
PXRoaXMuX2FkZFRyaWFuZ2xlKF8sYyxnLHJbY10sLTEscltfXSkscltjXT10aGlzLl9sZWdhbGl6
ZSh2KzIpLGVbX109XyxULS0sXz1nO2lmKHk9PT1sKWZvcig7Zz1uW3ldLFR1KGYscyx0WzIqZ10s
dFsyKmcrMV0sdFsyKnldLHRbMip5KzFdKTwwOyl2PXRoaXMuX2FkZFRyaWFuZ2xlKGcsYyx5LC0x
LHJbeV0scltnXSksdGhpcy5fbGVnYWxpemUodisyKSxyW2ddPXYsZVt5XT15LFQtLSx5PWc7dGhp
cy5faHVsbFN0YXJ0PW5bY109eSxlW3ldPW5bX109YyxlW2NdPV8saVt0aGlzLl9oYXNoS2V5KGYs
cyldPWMsaVt0aGlzLl9oYXNoS2V5KHRbMip5XSx0WzIqeSsxXSldPXl9dGhpcy5odWxsPW5ldyBV
aW50MzJBcnJheShUKTtmb3IobGV0IHQ9MCxuPXRoaXMuX2h1bGxTdGFydDt0PFQ7dCsrKXRoaXMu
aHVsbFt0XT1uLG49ZVtuXTt0aGlzLnRyaWFuZ2xlcz10aGlzLl90cmlhbmdsZXMuc3ViYXJyYXko
MCx0aGlzLnRyaWFuZ2xlc0xlbiksdGhpcy5oYWxmZWRnZXM9dGhpcy5faGFsZmVkZ2VzLnN1YmFy
cmF5KDAsdGhpcy50cmlhbmdsZXNMZW4pfV9oYXNoS2V5KHQsbil7cmV0dXJuIE1hdGguZmxvb3Io
ZnVuY3Rpb24odCxuKXtjb25zdCBlPXQvKE1hdGguYWJzKHQpK01hdGguYWJzKG4pKTtyZXR1cm4o
bj4wPzMtZToxK2UpLzR9KHQtdGhpcy5fY3gsbi10aGlzLl9jeSkqdGhpcy5faGFzaFNpemUpJXRo
aXMuX2hhc2hTaXplfV9sZWdhbGl6ZSh0KXtjb25zdHtfdHJpYW5nbGVzOm4sX2hhbGZlZGdlczpl
LGNvb3JkczpyfT10aGlzO2xldCBpPTAsbz0wO2Zvcig7Oyl7Y29uc3QgYT1lW3RdLHU9dC10JTM7
aWYobz11Kyh0KzIpJTMsLTE9PT1hKXtpZigwPT09aSlicmVhazt0PVN1Wy0taV07Y29udGludWV9
Y29uc3QgYz1hLWElMyxmPXUrKHQrMSklMyxzPWMrKGErMiklMyxsPW5bb10saD1uW3RdLGQ9bltm
XSxwPW5bc107aWYoa3UoclsyKmxdLHJbMipsKzFdLHJbMipoXSxyWzIqaCsxXSxyWzIqZF0sclsy
KmQrMV0sclsyKnBdLHJbMipwKzFdKSl7blt0XT1wLG5bYV09bDtjb25zdCByPWVbc107aWYoLTE9
PT1yKXtsZXQgbj10aGlzLl9odWxsU3RhcnQ7ZG97aWYodGhpcy5faHVsbFRyaVtuXT09PXMpe3Ro
aXMuX2h1bGxUcmlbbl09dDticmVha31uPXRoaXMuX2h1bGxQcmV2W25dfXdoaWxlKG4hPT10aGlz
Ll9odWxsU3RhcnQpfXRoaXMuX2xpbmsodCxyKSx0aGlzLl9saW5rKGEsZVtvXSksdGhpcy5fbGlu
ayhvLHMpO2NvbnN0IHU9YysoYSsxKSUzO2k8U3UubGVuZ3RoJiYoU3VbaSsrXT11KX1lbHNle2lm
KDA9PT1pKWJyZWFrO3Q9U3VbLS1pXX19cmV0dXJuIG99X2xpbmsodCxuKXt0aGlzLl9oYWxmZWRn
ZXNbdF09biwtMSE9PW4mJih0aGlzLl9oYWxmZWRnZXNbbl09dCl9X2FkZFRyaWFuZ2xlKHQsbixl
LHIsaSxvKXtjb25zdCBhPXRoaXMudHJpYW5nbGVzTGVuO3JldHVybiB0aGlzLl90cmlhbmdsZXNb
YV09dCx0aGlzLl90cmlhbmdsZXNbYSsxXT1uLHRoaXMuX3RyaWFuZ2xlc1thKzJdPWUsdGhpcy5f
bGluayhhLHIpLHRoaXMuX2xpbmsoYSsxLGkpLHRoaXMuX2xpbmsoYSsyLG8pLHRoaXMudHJpYW5n
bGVzTGVuKz0zLGF9fWZ1bmN0aW9uIE51KHQsbixlLHIpe2NvbnN0IGk9dC1lLG89bi1yO3JldHVy
biBpKmkrbypvfWZ1bmN0aW9uIGt1KHQsbixlLHIsaSxvLGEsdSl7Y29uc3QgYz10LWEsZj1uLXUs
cz1lLWEsbD1yLXUsaD1pLWEsZD1vLXUscD1zKnMrbCpsLGc9aCpoK2QqZDtyZXR1cm4gYyoobCpn
LXAqZCktZioocypnLXAqaCkrKGMqYytmKmYpKihzKmQtbCpoKTwwfWZ1bmN0aW9uIEN1KHQsbixl
LHIsaSxvKXtjb25zdCBhPWUtdCx1PXItbixjPWktdCxmPW8tbixzPWEqYSt1KnUsbD1jKmMrZipm
LGg9LjUvKGEqZi11KmMpLGQ9KGYqcy11KmwpKmgscD0oYSpsLWMqcykqaDtyZXR1cm4gZCpkK3Aq
cH1mdW5jdGlvbiBQdSh0LG4sZSxyKXtpZihyLWU8PTIwKWZvcihsZXQgaT1lKzE7aTw9cjtpKysp
e2NvbnN0IHI9dFtpXSxvPW5bcl07bGV0IGE9aS0xO2Zvcig7YT49ZSYmblt0W2FdXT5vOyl0W2Er
MV09dFthLS1dO3RbYSsxXT1yfWVsc2V7bGV0IGk9ZSsxLG89cjt6dSh0LGUrcj4+MSxpKSxuW3Rb
ZV1dPm5bdFtyXV0mJnp1KHQsZSxyKSxuW3RbaV1dPm5bdFtyXV0mJnp1KHQsaSxyKSxuW3RbZV1d
Pm5bdFtpXV0mJnp1KHQsZSxpKTtjb25zdCBhPXRbaV0sdT1uW2FdO2Zvcig7Oyl7ZG97aSsrfXdo
aWxlKG5bdFtpXV08dSk7ZG97by0tfXdoaWxlKG5bdFtvXV0+dSk7aWYobzxpKWJyZWFrO3p1KHQs
aSxvKX10W2UrMV09dFtvXSx0W29dPWEsci1pKzE+PW8tZT8oUHUodCxuLGksciksUHUodCxuLGUs
by0xKSk6KFB1KHQsbixlLG8tMSksUHUodCxuLGkscikpfX1mdW5jdGlvbiB6dSh0LG4sZSl7Y29u
c3Qgcj10W25dO3Rbbl09dFtlXSx0W2VdPXJ9ZnVuY3Rpb24gJHUodCl7cmV0dXJuIHRbMF19ZnVu
Y3Rpb24gRHUodCl7cmV0dXJuIHRbMV19Y29uc3QgUnU9MWUtNjtjbGFzcyBGdXtjb25zdHJ1Y3Rv
cigpe3RoaXMuX3gwPXRoaXMuX3kwPXRoaXMuX3gxPXRoaXMuX3kxPW51bGwsdGhpcy5fPSIifW1v
dmVUbyh0LG4pe3RoaXMuXys9YE0ke3RoaXMuX3gwPXRoaXMuX3gxPSt0fSwke3RoaXMuX3kwPXRo
aXMuX3kxPStufWB9Y2xvc2VQYXRoKCl7bnVsbCE9PXRoaXMuX3gxJiYodGhpcy5feDE9dGhpcy5f
eDAsdGhpcy5feTE9dGhpcy5feTAsdGhpcy5fKz0iWiIpfWxpbmVUbyh0LG4pe3RoaXMuXys9YEwk
e3RoaXMuX3gxPSt0fSwke3RoaXMuX3kxPStufWB9YXJjKHQsbixlKXtjb25zdCByPSh0PSt0KSso
ZT0rZSksaT1uPStuO2lmKGU8MCl0aHJvdyBuZXcgRXJyb3IoIm5lZ2F0aXZlIHJhZGl1cyIpO251
bGw9PT10aGlzLl94MT90aGlzLl8rPWBNJHtyfSwke2l9YDooTWF0aC5hYnModGhpcy5feDEtcik+
UnV8fE1hdGguYWJzKHRoaXMuX3kxLWkpPlJ1KSYmKHRoaXMuXys9IkwiK3IrIiwiK2kpLGUmJih0
aGlzLl8rPWBBJHtlfSwke2V9LDAsMSwxLCR7dC1lfSwke259QSR7ZX0sJHtlfSwwLDEsMSwke3Ro
aXMuX3gxPXJ9LCR7dGhpcy5feTE9aX1gKX1yZWN0KHQsbixlLHIpe3RoaXMuXys9YE0ke3RoaXMu
X3gwPXRoaXMuX3gxPSt0fSwke3RoaXMuX3kwPXRoaXMuX3kxPStufWgkeytlfXYkeytyfWgkey1l
fVpgfXZhbHVlKCl7cmV0dXJuIHRoaXMuX3x8bnVsbH19Y2xhc3MgcXV7Y29uc3RydWN0b3IoKXt0
aGlzLl89W119bW92ZVRvKHQsbil7dGhpcy5fLnB1c2goW3Qsbl0pfWNsb3NlUGF0aCgpe3RoaXMu
Xy5wdXNoKHRoaXMuX1swXS5zbGljZSgpKX1saW5lVG8odCxuKXt0aGlzLl8ucHVzaChbdCxuXSl9
dmFsdWUoKXtyZXR1cm4gdGhpcy5fLmxlbmd0aD90aGlzLl86bnVsbH19Y2xhc3MgVXV7Y29uc3Ry
dWN0b3IodCxbbixlLHIsaV09WzAsMCw5NjAsNTAwXSl7aWYoISgocj0rcik+PShuPStuKSYmKGk9
K2kpPj0oZT0rZSkpKXRocm93IG5ldyBFcnJvcigiaW52YWxpZCBib3VuZHMiKTt0aGlzLmRlbGF1
bmF5PXQsdGhpcy5fY2lyY3VtY2VudGVycz1uZXcgRmxvYXQ2NEFycmF5KDIqdC5wb2ludHMubGVu
Z3RoKSx0aGlzLnZlY3RvcnM9bmV3IEZsb2F0NjRBcnJheSgyKnQucG9pbnRzLmxlbmd0aCksdGhp
cy54bWF4PXIsdGhpcy54bWluPW4sdGhpcy55bWF4PWksdGhpcy55bWluPWUsdGhpcy5faW5pdCgp
fXVwZGF0ZSgpe3JldHVybiB0aGlzLmRlbGF1bmF5LnVwZGF0ZSgpLHRoaXMuX2luaXQoKSx0aGlz
fV9pbml0KCl7Y29uc3R7ZGVsYXVuYXk6e3BvaW50czp0LGh1bGw6bix0cmlhbmdsZXM6ZX0sdmVj
dG9yczpyfT10aGlzO2xldCBpLG87Y29uc3QgYT10aGlzLmNpcmN1bWNlbnRlcnM9dGhpcy5fY2ly
Y3VtY2VudGVycy5zdWJhcnJheSgwLGUubGVuZ3RoLzMqMik7Zm9yKGxldCByLHUsYz0wLGY9MCxz
PWUubGVuZ3RoO2M8cztjKz0zLGYrPTIpe2NvbnN0IHM9MiplW2NdLGw9MiplW2MrMV0saD0yKmVb
YysyXSxkPXRbc10scD10W3MrMV0sZz10W2xdLHk9dFtsKzFdLHY9dFtoXSxfPXRbaCsxXSxiPWct
ZCxtPXktcCx4PXYtZCx3PV8tcCxNPTIqKGIqdy1tKngpO2lmKE1hdGguYWJzKE0pPDFlLTkpe2lm
KHZvaWQgMD09PWkpe2k9bz0wO2Zvcihjb25zdCBlIG9mIG4paSs9dFsyKmVdLG8rPXRbMiplKzFd
O2kvPW4ubGVuZ3RoLG8vPW4ubGVuZ3RofWNvbnN0IGU9MWU5Kk1hdGguc2lnbigoaS1kKSp3LShv
LXApKngpO3I9KGQrdikvMi1lKncsdT0ocCtfKS8yK2UqeH1lbHNle2NvbnN0IHQ9MS9NLG49Yipi
K20qbSxlPXgqeCt3Knc7cj1kKyh3Km4tbSplKSp0LHU9cCsoYiplLXgqbikqdH1hW2ZdPXIsYVtm
KzFdPXV9bGV0IHUsYyxmLHM9bltuLmxlbmd0aC0xXSxsPTQqcyxoPXRbMipzXSxkPXRbMipzKzFd
O3IuZmlsbCgwKTtmb3IobGV0IGU9MDtlPG4ubGVuZ3RoOysrZSlzPW5bZV0sdT1sLGM9aCxmPWQs
bD00KnMsaD10WzIqc10sZD10WzIqcysxXSxyW3UrMl09cltsXT1mLWQsclt1KzNdPXJbbCsxXT1o
LWN9cmVuZGVyKHQpe2NvbnN0IG49bnVsbD09dD90PW5ldyBGdTp2b2lkIDAse2RlbGF1bmF5Onto
YWxmZWRnZXM6ZSxpbmVkZ2VzOnIsaHVsbDppfSxjaXJjdW1jZW50ZXJzOm8sdmVjdG9yczphfT10
aGlzO2lmKGkubGVuZ3RoPD0xKXJldHVybiBudWxsO2ZvcihsZXQgbj0wLHI9ZS5sZW5ndGg7bjxy
Oysrbil7Y29uc3Qgcj1lW25dO2lmKHI8biljb250aW51ZTtjb25zdCBpPTIqTWF0aC5mbG9vcihu
LzMpLGE9MipNYXRoLmZsb29yKHIvMyksdT1vW2ldLGM9b1tpKzFdLGY9b1thXSxzPW9bYSsxXTt0
aGlzLl9yZW5kZXJTZWdtZW50KHUsYyxmLHMsdCl9bGV0IHUsYz1pW2kubGVuZ3RoLTFdO2Zvcihs
ZXQgbj0wO248aS5sZW5ndGg7KytuKXt1PWMsYz1pW25dO2NvbnN0IGU9MipNYXRoLmZsb29yKHJb
Y10vMyksZj1vW2VdLHM9b1tlKzFdLGw9NCp1LGg9dGhpcy5fcHJvamVjdChmLHMsYVtsKzJdLGFb
bCszXSk7aCYmdGhpcy5fcmVuZGVyU2VnbWVudChmLHMsaFswXSxoWzFdLHQpfXJldHVybiBuJiZu
LnZhbHVlKCl9cmVuZGVyQm91bmRzKHQpe2NvbnN0IG49bnVsbD09dD90PW5ldyBGdTp2b2lkIDA7
cmV0dXJuIHQucmVjdCh0aGlzLnhtaW4sdGhpcy55bWluLHRoaXMueG1heC10aGlzLnhtaW4sdGhp
cy55bWF4LXRoaXMueW1pbiksbiYmbi52YWx1ZSgpfXJlbmRlckNlbGwodCxuKXtjb25zdCBlPW51
bGw9PW4/bj1uZXcgRnU6dm9pZCAwLHI9dGhpcy5fY2xpcCh0KTtpZihudWxsPT09cnx8IXIubGVu
Z3RoKXJldHVybjtuLm1vdmVUbyhyWzBdLHJbMV0pO2xldCBpPXIubGVuZ3RoO2Zvcig7clswXT09
PXJbaS0yXSYmclsxXT09PXJbaS0xXSYmaT4xOylpLT0yO2ZvcihsZXQgdD0yO3Q8aTt0Kz0yKXJb
dF09PT1yW3QtMl0mJnJbdCsxXT09PXJbdC0xXXx8bi5saW5lVG8oclt0XSxyW3QrMV0pO3JldHVy
biBuLmNsb3NlUGF0aCgpLGUmJmUudmFsdWUoKX0qY2VsbFBvbHlnb25zKCl7Y29uc3R7ZGVsYXVu
YXk6e3BvaW50czp0fX09dGhpcztmb3IobGV0IG49MCxlPXQubGVuZ3RoLzI7bjxlOysrbil7Y29u
c3QgdD10aGlzLmNlbGxQb2x5Z29uKG4pO3QmJih0LmluZGV4PW4seWllbGQgdCl9fWNlbGxQb2x5
Z29uKHQpe2NvbnN0IG49bmV3IHF1O3JldHVybiB0aGlzLnJlbmRlckNlbGwodCxuKSxuLnZhbHVl
KCl9X3JlbmRlclNlZ21lbnQodCxuLGUscixpKXtsZXQgbztjb25zdCBhPXRoaXMuX3JlZ2lvbmNv
ZGUodCxuKSx1PXRoaXMuX3JlZ2lvbmNvZGUoZSxyKTswPT09YSYmMD09PXU/KGkubW92ZVRvKHQs
biksaS5saW5lVG8oZSxyKSk6KG89dGhpcy5fY2xpcFNlZ21lbnQodCxuLGUscixhLHUpKSYmKGku
bW92ZVRvKG9bMF0sb1sxXSksaS5saW5lVG8ob1syXSxvWzNdKSl9Y29udGFpbnModCxuLGUpe3Jl
dHVybihuPStuKT09biYmKGU9K2UpPT1lJiZ0aGlzLmRlbGF1bmF5Ll9zdGVwKHQsbixlKT09PXR9
Km5laWdoYm9ycyh0KXtjb25zdCBuPXRoaXMuX2NsaXAodCk7aWYobilmb3IoY29uc3QgZSBvZiB0
aGlzLmRlbGF1bmF5Lm5laWdoYm9ycyh0KSl7Y29uc3QgdD10aGlzLl9jbGlwKGUpO2lmKHQpdDpm
b3IobGV0IHI9MCxpPW4ubGVuZ3RoO3I8aTtyKz0yKWZvcihsZXQgbz0wLGE9dC5sZW5ndGg7bzxh
O28rPTIpaWYobltyXT09PXRbb10mJm5bcisxXT09PXRbbysxXSYmblsocisyKSVpXT09PXRbKG8r
YS0yKSVhXSYmblsociszKSVpXT09PXRbKG8rYS0xKSVhXSl7eWllbGQgZTticmVhayB0fX19X2Nl
bGwodCl7Y29uc3R7Y2lyY3VtY2VudGVyczpuLGRlbGF1bmF5OntpbmVkZ2VzOmUsaGFsZmVkZ2Vz
OnIsdHJpYW5nbGVzOml9fT10aGlzLG89ZVt0XTtpZigtMT09PW8pcmV0dXJuIG51bGw7Y29uc3Qg
YT1bXTtsZXQgdT1vO2Rve2NvbnN0IGU9TWF0aC5mbG9vcih1LzMpO2lmKGEucHVzaChuWzIqZV0s
blsyKmUrMV0pLHU9dSUzPT0yP3UtMjp1KzEsaVt1XSE9PXQpYnJlYWs7dT1yW3VdfXdoaWxlKHUh
PT1vJiYtMSE9PXUpO3JldHVybiBhfV9jbGlwKHQpe2lmKDA9PT10JiYxPT09dGhpcy5kZWxhdW5h
eS5odWxsLmxlbmd0aClyZXR1cm5bdGhpcy54bWF4LHRoaXMueW1pbix0aGlzLnhtYXgsdGhpcy55
bWF4LHRoaXMueG1pbix0aGlzLnltYXgsdGhpcy54bWluLHRoaXMueW1pbl07Y29uc3Qgbj10aGlz
Ll9jZWxsKHQpO2lmKG51bGw9PT1uKXJldHVybiBudWxsO2NvbnN0e3ZlY3RvcnM6ZX09dGhpcyxy
PTQqdDtyZXR1cm4gdGhpcy5fc2ltcGxpZnkoZVtyXXx8ZVtyKzFdP3RoaXMuX2NsaXBJbmZpbml0
ZSh0LG4sZVtyXSxlW3IrMV0sZVtyKzJdLGVbciszXSk6dGhpcy5fY2xpcEZpbml0ZSh0LG4pKX1f
Y2xpcEZpbml0ZSh0LG4pe2NvbnN0IGU9bi5sZW5ndGg7bGV0IHIsaSxvLGEsdT1udWxsLGM9bltl
LTJdLGY9bltlLTFdLHM9dGhpcy5fcmVnaW9uY29kZShjLGYpLGw9MDtmb3IobGV0IGg9MDtoPGU7
aCs9MilpZihyPWMsaT1mLGM9bltoXSxmPW5baCsxXSxvPXMscz10aGlzLl9yZWdpb25jb2RlKGMs
ZiksMD09PW8mJjA9PT1zKWE9bCxsPTAsdT91LnB1c2goYyxmKTp1PVtjLGZdO2Vsc2V7bGV0IG4s
ZSxoLGQscDtpZigwPT09byl7aWYobnVsbD09PShuPXRoaXMuX2NsaXBTZWdtZW50KHIsaSxjLGYs
byxzKSkpY29udGludWU7W2UsaCxkLHBdPW59ZWxzZXtpZihudWxsPT09KG49dGhpcy5fY2xpcFNl
Z21lbnQoYyxmLHIsaSxzLG8pKSljb250aW51ZTtbZCxwLGUsaF09bixhPWwsbD10aGlzLl9lZGdl
Y29kZShlLGgpLGEmJmwmJnRoaXMuX2VkZ2UodCxhLGwsdSx1Lmxlbmd0aCksdT91LnB1c2goZSxo
KTp1PVtlLGhdfWE9bCxsPXRoaXMuX2VkZ2Vjb2RlKGQscCksYSYmbCYmdGhpcy5fZWRnZSh0LGEs
bCx1LHUubGVuZ3RoKSx1P3UucHVzaChkLHApOnU9W2QscF19aWYodSlhPWwsbD10aGlzLl9lZGdl
Y29kZSh1WzBdLHVbMV0pLGEmJmwmJnRoaXMuX2VkZ2UodCxhLGwsdSx1Lmxlbmd0aCk7ZWxzZSBp
Zih0aGlzLmNvbnRhaW5zKHQsKHRoaXMueG1pbit0aGlzLnhtYXgpLzIsKHRoaXMueW1pbit0aGlz
LnltYXgpLzIpKXJldHVyblt0aGlzLnhtYXgsdGhpcy55bWluLHRoaXMueG1heCx0aGlzLnltYXgs
dGhpcy54bWluLHRoaXMueW1heCx0aGlzLnhtaW4sdGhpcy55bWluXTtyZXR1cm4gdX1fY2xpcFNl
Z21lbnQodCxuLGUscixpLG8pe2NvbnN0IGE9aTxvO2ZvcihhJiYoW3QsbixlLHIsaSxvXT1bZSxy
LHQsbixvLGldKTs7KXtpZigwPT09aSYmMD09PW8pcmV0dXJuIGE/W2Uscix0LG5dOlt0LG4sZSxy
XTtpZihpJm8pcmV0dXJuIG51bGw7bGV0IHUsYyxmPWl8fG87OCZmPyh1PXQrKGUtdCkqKHRoaXMu
eW1heC1uKS8oci1uKSxjPXRoaXMueW1heCk6NCZmPyh1PXQrKGUtdCkqKHRoaXMueW1pbi1uKS8o
ci1uKSxjPXRoaXMueW1pbik6MiZmPyhjPW4rKHItbikqKHRoaXMueG1heC10KS8oZS10KSx1PXRo
aXMueG1heCk6KGM9bisoci1uKSoodGhpcy54bWluLXQpLyhlLXQpLHU9dGhpcy54bWluKSxpPyh0
PXUsbj1jLGk9dGhpcy5fcmVnaW9uY29kZSh0LG4pKTooZT11LHI9YyxvPXRoaXMuX3JlZ2lvbmNv
ZGUoZSxyKSl9fV9jbGlwSW5maW5pdGUodCxuLGUscixpLG8pe2xldCBhLHU9QXJyYXkuZnJvbShu
KTtpZigoYT10aGlzLl9wcm9qZWN0KHVbMF0sdVsxXSxlLHIpKSYmdS51bnNoaWZ0KGFbMF0sYVsx
XSksKGE9dGhpcy5fcHJvamVjdCh1W3UubGVuZ3RoLTJdLHVbdS5sZW5ndGgtMV0saSxvKSkmJnUu
cHVzaChhWzBdLGFbMV0pLHU9dGhpcy5fY2xpcEZpbml0ZSh0LHUpKWZvcihsZXQgbixlPTAscj11
Lmxlbmd0aCxpPXRoaXMuX2VkZ2Vjb2RlKHVbci0yXSx1W3ItMV0pO2U8cjtlKz0yKW49aSxpPXRo
aXMuX2VkZ2Vjb2RlKHVbZV0sdVtlKzFdKSxuJiZpJiYoZT10aGlzLl9lZGdlKHQsbixpLHUsZSks
cj11Lmxlbmd0aCk7ZWxzZSB0aGlzLmNvbnRhaW5zKHQsKHRoaXMueG1pbit0aGlzLnhtYXgpLzIs
KHRoaXMueW1pbit0aGlzLnltYXgpLzIpJiYodT1bdGhpcy54bWluLHRoaXMueW1pbix0aGlzLnht
YXgsdGhpcy55bWluLHRoaXMueG1heCx0aGlzLnltYXgsdGhpcy54bWluLHRoaXMueW1heF0pO3Jl
dHVybiB1fV9lZGdlKHQsbixlLHIsaSl7Zm9yKDtuIT09ZTspe2xldCBlLG87c3dpdGNoKG4pe2Nh
c2UgNTpuPTQ7Y29udGludWU7Y2FzZSA0Om49NixlPXRoaXMueG1heCxvPXRoaXMueW1pbjticmVh
aztjYXNlIDY6bj0yO2NvbnRpbnVlO2Nhc2UgMjpuPTEwLGU9dGhpcy54bWF4LG89dGhpcy55bWF4
O2JyZWFrO2Nhc2UgMTA6bj04O2NvbnRpbnVlO2Nhc2UgODpuPTksZT10aGlzLnhtaW4sbz10aGlz
LnltYXg7YnJlYWs7Y2FzZSA5Om49MTtjb250aW51ZTtjYXNlIDE6bj01LGU9dGhpcy54bWluLG89
dGhpcy55bWlufXJbaV09PT1lJiZyW2krMV09PT1vfHwhdGhpcy5jb250YWlucyh0LGUsbyl8fChy
LnNwbGljZShpLDAsZSxvKSxpKz0yKX1yZXR1cm4gaX1fcHJvamVjdCh0LG4sZSxyKXtsZXQgaSxv
LGEsdT0xLzA7aWYocjwwKXtpZihuPD10aGlzLnltaW4pcmV0dXJuIG51bGw7KGk9KHRoaXMueW1p
bi1uKS9yKTx1JiYoYT10aGlzLnltaW4sbz10Kyh1PWkpKmUpfWVsc2UgaWYocj4wKXtpZihuPj10
aGlzLnltYXgpcmV0dXJuIG51bGw7KGk9KHRoaXMueW1heC1uKS9yKTx1JiYoYT10aGlzLnltYXgs
bz10Kyh1PWkpKmUpfWlmKGU+MCl7aWYodD49dGhpcy54bWF4KXJldHVybiBudWxsOyhpPSh0aGlz
LnhtYXgtdCkvZSk8dSYmKG89dGhpcy54bWF4LGE9bisodT1pKSpyKX1lbHNlIGlmKGU8MCl7aWYo
dDw9dGhpcy54bWluKXJldHVybiBudWxsOyhpPSh0aGlzLnhtaW4tdCkvZSk8dSYmKG89dGhpcy54
bWluLGE9bisodT1pKSpyKX1yZXR1cm5bbyxhXX1fZWRnZWNvZGUodCxuKXtyZXR1cm4odD09PXRo
aXMueG1pbj8xOnQ9PT10aGlzLnhtYXg/MjowKXwobj09PXRoaXMueW1pbj80Om49PT10aGlzLnlt
YXg/ODowKX1fcmVnaW9uY29kZSh0LG4pe3JldHVybih0PHRoaXMueG1pbj8xOnQ+dGhpcy54bWF4
PzI6MCl8KG48dGhpcy55bWluPzQ6bj50aGlzLnltYXg/ODowKX1fc2ltcGxpZnkodCl7aWYodCYm
dC5sZW5ndGg+NCl7Zm9yKGxldCBuPTA7bjx0Lmxlbmd0aDtuKz0yKXtjb25zdCBlPShuKzIpJXQu
bGVuZ3RoLHI9KG4rNCkldC5sZW5ndGg7KHRbbl09PT10W2VdJiZ0W2VdPT09dFtyXXx8dFtuKzFd
PT09dFtlKzFdJiZ0W2UrMV09PT10W3IrMV0pJiYodC5zcGxpY2UoZSwyKSxuLT0yKX10Lmxlbmd0
aHx8KHQ9bnVsbCl9cmV0dXJuIHR9fWNvbnN0IEl1PTIqTWF0aC5QSSxPdT1NYXRoLnBvdztmdW5j
dGlvbiBCdSh0KXtyZXR1cm4gdFswXX1mdW5jdGlvbiBZdSh0KXtyZXR1cm4gdFsxXX1mdW5jdGlv
biBMdSh0LG4sZSl7cmV0dXJuW3QrTWF0aC5zaW4odCtuKSplLG4rTWF0aC5jb3ModC1uKSplXX1j
bGFzcyBqdXtzdGF0aWMgZnJvbSh0LG49QnUsZT1ZdSxyKXtyZXR1cm4gbmV3IGp1KCJsZW5ndGgi
aW4gdD9mdW5jdGlvbih0LG4sZSxyKXtjb25zdCBpPXQubGVuZ3RoLG89bmV3IEZsb2F0NjRBcnJh
eSgyKmkpO2ZvcihsZXQgYT0wO2E8aTsrK2Epe2NvbnN0IGk9dFthXTtvWzIqYV09bi5jYWxsKHIs
aSxhLHQpLG9bMiphKzFdPWUuY2FsbChyLGksYSx0KX1yZXR1cm4gb30odCxuLGUscik6RmxvYXQ2
NEFycmF5LmZyb20oZnVuY3Rpb24qKHQsbixlLHIpe2xldCBpPTA7Zm9yKGNvbnN0IG8gb2YgdCl5
aWVsZCBuLmNhbGwocixvLGksdCkseWllbGQgZS5jYWxsKHIsbyxpLHQpLCsraX0odCxuLGUscikp
KX1jb25zdHJ1Y3Rvcih0KXt0aGlzLl9kZWxhdW5hdG9yPW5ldyBFdSh0KSx0aGlzLmluZWRnZXM9
bmV3IEludDMyQXJyYXkodC5sZW5ndGgvMiksdGhpcy5faHVsbEluZGV4PW5ldyBJbnQzMkFycmF5
KHQubGVuZ3RoLzIpLHRoaXMucG9pbnRzPXRoaXMuX2RlbGF1bmF0b3IuY29vcmRzLHRoaXMuX2lu
aXQoKX11cGRhdGUoKXtyZXR1cm4gdGhpcy5fZGVsYXVuYXRvci51cGRhdGUoKSx0aGlzLl9pbml0
KCksdGhpc31faW5pdCgpe2NvbnN0IHQ9dGhpcy5fZGVsYXVuYXRvcixuPXRoaXMucG9pbnRzO2lm
KHQuaHVsbCYmdC5odWxsLmxlbmd0aD4yJiZmdW5jdGlvbih0KXtjb25zdHt0cmlhbmdsZXM6bixj
b29yZHM6ZX09dDtmb3IobGV0IHQ9MDt0PG4ubGVuZ3RoO3QrPTMpe2NvbnN0IHI9MipuW3RdLGk9
MipuW3QrMV0sbz0yKm5bdCsyXTtpZigoZVtvXS1lW3JdKSooZVtpKzFdLWVbcisxXSktKGVbaV0t
ZVtyXSkqKGVbbysxXS1lW3IrMV0pPjFlLTEwKXJldHVybiExfXJldHVybiEwfSh0KSl7dGhpcy5j
b2xsaW5lYXI9SW50MzJBcnJheS5mcm9tKHtsZW5ndGg6bi5sZW5ndGgvMn0sKCh0LG4pPT5uKSku
c29ydCgoKHQsZSk9Pm5bMip0XS1uWzIqZV18fG5bMip0KzFdLW5bMiplKzFdKSk7Y29uc3QgdD10
aGlzLmNvbGxpbmVhclswXSxlPXRoaXMuY29sbGluZWFyW3RoaXMuY29sbGluZWFyLmxlbmd0aC0x
XSxyPVtuWzIqdF0sblsyKnQrMV0sblsyKmVdLG5bMiplKzFdXSxpPTFlLTgqTWF0aC5oeXBvdChy
WzNdLXJbMV0sclsyXS1yWzBdKTtmb3IobGV0IHQ9MCxlPW4ubGVuZ3RoLzI7dDxlOysrdCl7Y29u
c3QgZT1MdShuWzIqdF0sblsyKnQrMV0saSk7blsyKnRdPWVbMF0sblsyKnQrMV09ZVsxXX10aGlz
Ll9kZWxhdW5hdG9yPW5ldyBFdShuKX1lbHNlIGRlbGV0ZSB0aGlzLmNvbGxpbmVhcjtjb25zdCBl
PXRoaXMuaGFsZmVkZ2VzPXRoaXMuX2RlbGF1bmF0b3IuaGFsZmVkZ2VzLHI9dGhpcy5odWxsPXRo
aXMuX2RlbGF1bmF0b3IuaHVsbCxpPXRoaXMudHJpYW5nbGVzPXRoaXMuX2RlbGF1bmF0b3IudHJp
YW5nbGVzLG89dGhpcy5pbmVkZ2VzLmZpbGwoLTEpLGE9dGhpcy5faHVsbEluZGV4LmZpbGwoLTEp
O2ZvcihsZXQgdD0wLG49ZS5sZW5ndGg7dDxuOysrdCl7Y29uc3Qgbj1pW3QlMz09Mj90LTI6dCsx
XTstMSE9PWVbdF0mJi0xIT09b1tuXXx8KG9bbl09dCl9Zm9yKGxldCB0PTAsbj1yLmxlbmd0aDt0
PG47Kyt0KWFbclt0XV09dDtyLmxlbmd0aDw9MiYmci5sZW5ndGg+MCYmKHRoaXMudHJpYW5nbGVz
PW5ldyBJbnQzMkFycmF5KDMpLmZpbGwoLTEpLHRoaXMuaGFsZmVkZ2VzPW5ldyBJbnQzMkFycmF5
KDMpLmZpbGwoLTEpLHRoaXMudHJpYW5nbGVzWzBdPXJbMF0sb1tyWzBdXT0xLDI9PT1yLmxlbmd0
aCYmKG9bclsxXV09MCx0aGlzLnRyaWFuZ2xlc1sxXT1yWzFdLHRoaXMudHJpYW5nbGVzWzJdPXJb
MV0pKX12b3Jvbm9pKHQpe3JldHVybiBuZXcgVXUodGhpcyx0KX0qbmVpZ2hib3JzKHQpe2NvbnN0
e2luZWRnZXM6bixodWxsOmUsX2h1bGxJbmRleDpyLGhhbGZlZGdlczppLHRyaWFuZ2xlczpvLGNv
bGxpbmVhcjphfT10aGlzO2lmKGEpe2NvbnN0IG49YS5pbmRleE9mKHQpO3JldHVybiBuPjAmJih5
aWVsZCBhW24tMV0pLHZvaWQobjxhLmxlbmd0aC0xJiYoeWllbGQgYVtuKzFdKSl9Y29uc3QgdT1u
W3RdO2lmKC0xPT09dSlyZXR1cm47bGV0IGM9dSxmPS0xO2Rve2lmKHlpZWxkIGY9b1tjXSxjPWMl
Mz09Mj9jLTI6YysxLG9bY10hPT10KXJldHVybjtpZihjPWlbY10sLTE9PT1jKXtjb25zdCBuPWVb
KHJbdF0rMSklZS5sZW5ndGhdO3JldHVybiB2b2lkKG4hPT1mJiYoeWllbGQgbikpfX13aGlsZShj
IT09dSl9ZmluZCh0LG4sZT0wKXtpZigodD0rdCkhPXR8fChuPStuKSE9bilyZXR1cm4tMTtjb25z
dCByPWU7bGV0IGk7Zm9yKDsoaT10aGlzLl9zdGVwKGUsdCxuKSk+PTAmJmkhPT1lJiZpIT09cjsp
ZT1pO3JldHVybiBpfV9zdGVwKHQsbixlKXtjb25zdHtpbmVkZ2VzOnIsaHVsbDppLF9odWxsSW5k
ZXg6byxoYWxmZWRnZXM6YSx0cmlhbmdsZXM6dSxwb2ludHM6Y309dGhpcztpZigtMT09PXJbdF18
fCFjLmxlbmd0aClyZXR1cm4odCsxKSUoYy5sZW5ndGg+PjEpO2xldCBmPXQscz1PdShuLWNbMip0
XSwyKStPdShlLWNbMip0KzFdLDIpO2NvbnN0IGw9clt0XTtsZXQgaD1sO2Rve2xldCByPXVbaF07
Y29uc3QgbD1PdShuLWNbMipyXSwyKStPdShlLWNbMipyKzFdLDIpO2lmKGw8cyYmKHM9bCxmPXIp
LGg9aCUzPT0yP2gtMjpoKzEsdVtoXSE9PXQpYnJlYWs7aWYoaD1hW2hdLC0xPT09aCl7aWYoaD1p
WyhvW3RdKzEpJWkubGVuZ3RoXSxoIT09ciYmT3Uobi1jWzIqaF0sMikrT3UoZS1jWzIqaCsxXSwy
KTxzKXJldHVybiBoO2JyZWFrfX13aGlsZShoIT09bCk7cmV0dXJuIGZ9cmVuZGVyKHQpe2NvbnN0
IG49bnVsbD09dD90PW5ldyBGdTp2b2lkIDAse3BvaW50czplLGhhbGZlZGdlczpyLHRyaWFuZ2xl
czppfT10aGlzO2ZvcihsZXQgbj0wLG89ci5sZW5ndGg7bjxvOysrbil7Y29uc3Qgbz1yW25dO2lm
KG88biljb250aW51ZTtjb25zdCBhPTIqaVtuXSx1PTIqaVtvXTt0Lm1vdmVUbyhlW2FdLGVbYSsx
XSksdC5saW5lVG8oZVt1XSxlW3UrMV0pfXJldHVybiB0aGlzLnJlbmRlckh1bGwodCksbiYmbi52
YWx1ZSgpfXJlbmRlclBvaW50cyh0LG4pe3ZvaWQgMCE9PW58fHQmJiJmdW5jdGlvbiI9PXR5cGVv
ZiB0Lm1vdmVUb3x8KG49dCx0PW51bGwpLG49bnVsbD09bj8yOituO2NvbnN0IGU9bnVsbD09dD90
PW5ldyBGdTp2b2lkIDAse3BvaW50czpyfT10aGlzO2ZvcihsZXQgZT0wLGk9ci5sZW5ndGg7ZTxp
O2UrPTIpe2NvbnN0IGk9cltlXSxvPXJbZSsxXTt0Lm1vdmVUbyhpK24sbyksdC5hcmMoaSxvLG4s
MCxJdSl9cmV0dXJuIGUmJmUudmFsdWUoKX1yZW5kZXJIdWxsKHQpe2NvbnN0IG49bnVsbD09dD90
PW5ldyBGdTp2b2lkIDAse2h1bGw6ZSxwb2ludHM6cn09dGhpcyxpPTIqZVswXSxvPWUubGVuZ3Ro
O3QubW92ZVRvKHJbaV0scltpKzFdKTtmb3IobGV0IG49MTtuPG87KytuKXtjb25zdCBpPTIqZVtu
XTt0LmxpbmVUbyhyW2ldLHJbaSsxXSl9cmV0dXJuIHQuY2xvc2VQYXRoKCksbiYmbi52YWx1ZSgp
fWh1bGxQb2x5Z29uKCl7Y29uc3QgdD1uZXcgcXU7cmV0dXJuIHRoaXMucmVuZGVySHVsbCh0KSx0
LnZhbHVlKCl9cmVuZGVyVHJpYW5nbGUodCxuKXtjb25zdCBlPW51bGw9PW4/bj1uZXcgRnU6dm9p
ZCAwLHtwb2ludHM6cix0cmlhbmdsZXM6aX09dGhpcyxvPTIqaVt0Kj0zXSxhPTIqaVt0KzFdLHU9
MippW3QrMl07cmV0dXJuIG4ubW92ZVRvKHJbb10scltvKzFdKSxuLmxpbmVUbyhyW2FdLHJbYSsx
XSksbi5saW5lVG8oclt1XSxyW3UrMV0pLG4uY2xvc2VQYXRoKCksZSYmZS52YWx1ZSgpfSp0cmlh
bmdsZVBvbHlnb25zKCl7Y29uc3R7dHJpYW5nbGVzOnR9PXRoaXM7Zm9yKGxldCBuPTAsZT10Lmxl
bmd0aC8zO248ZTsrK24peWllbGQgdGhpcy50cmlhbmdsZVBvbHlnb24obil9dHJpYW5nbGVQb2x5
Z29uKHQpe2NvbnN0IG49bmV3IHF1O3JldHVybiB0aGlzLnJlbmRlclRyaWFuZ2xlKHQsbiksbi52
YWx1ZSgpfX12YXIgSHU9e30sWHU9e30sR3U9MzQsVnU9MTAsV3U9MTM7ZnVuY3Rpb24gWnUodCl7
cmV0dXJuIG5ldyBGdW5jdGlvbigiZCIsInJldHVybiB7Iit0Lm1hcCgoZnVuY3Rpb24odCxuKXty
ZXR1cm4gSlNPTi5zdHJpbmdpZnkodCkrIjogZFsiK24rJ10gfHwgIiInfSkpLmpvaW4oIiwiKSsi
fSIpfWZ1bmN0aW9uIEt1KHQpe3ZhciBuPU9iamVjdC5jcmVhdGUobnVsbCksZT1bXTtyZXR1cm4g
dC5mb3JFYWNoKChmdW5jdGlvbih0KXtmb3IodmFyIHIgaW4gdClyIGluIG58fGUucHVzaChuW3Jd
PXIpfSkpLGV9ZnVuY3Rpb24gUXUodCxuKXt2YXIgZT10KyIiLHI9ZS5sZW5ndGg7cmV0dXJuIHI8
bj9uZXcgQXJyYXkobi1yKzEpLmpvaW4oMCkrZTplfWZ1bmN0aW9uIEp1KHQpe3ZhciBuLGU9dC5n
ZXRVVENIb3VycygpLHI9dC5nZXRVVENNaW51dGVzKCksaT10LmdldFVUQ1NlY29uZHMoKSxvPXQu
Z2V0VVRDTWlsbGlzZWNvbmRzKCk7cmV0dXJuIGlzTmFOKHQpPyJJbnZhbGlkIERhdGUiOigobj10
LmdldFVUQ0Z1bGxZZWFyKCkpPDA/Ii0iK1F1KC1uLDYpOm4+OTk5OT8iKyIrUXUobiw2KTpRdShu
LDQpKSsiLSIrUXUodC5nZXRVVENNb250aCgpKzEsMikrIi0iK1F1KHQuZ2V0VVRDRGF0ZSgpLDIp
KyhvPyJUIitRdShlLDIpKyI6IitRdShyLDIpKyI6IitRdShpLDIpKyIuIitRdShvLDMpKyJaIjpp
PyJUIitRdShlLDIpKyI6IitRdShyLDIpKyI6IitRdShpLDIpKyJaIjpyfHxlPyJUIitRdShlLDIp
KyI6IitRdShyLDIpKyJaIjoiIil9ZnVuY3Rpb24gdGModCl7dmFyIG49bmV3IFJlZ0V4cCgnWyIn
K3QrIlxuXHJdIiksZT10LmNoYXJDb2RlQXQoMCk7ZnVuY3Rpb24gcih0LG4pe3ZhciByLGk9W10s
bz10Lmxlbmd0aCxhPTAsdT0wLGM9bzw9MCxmPSExO2Z1bmN0aW9uIHMoKXtpZihjKXJldHVybiBY
dTtpZihmKXJldHVybiBmPSExLEh1O3ZhciBuLHIsaT1hO2lmKHQuY2hhckNvZGVBdChpKT09PUd1
KXtmb3IoO2ErKzxvJiZ0LmNoYXJDb2RlQXQoYSkhPT1HdXx8dC5jaGFyQ29kZUF0KCsrYSk9PT1H
dTspO3JldHVybihuPWEpPj1vP2M9ITA6KHI9dC5jaGFyQ29kZUF0KGErKykpPT09VnU/Zj0hMDpy
PT09V3UmJihmPSEwLHQuY2hhckNvZGVBdChhKT09PVZ1JiYrK2EpLHQuc2xpY2UoaSsxLG4tMSku
cmVwbGFjZSgvIiIvZywnIicpfWZvcig7YTxvOyl7aWYoKHI9dC5jaGFyQ29kZUF0KG49YSsrKSk9
PT1WdSlmPSEwO2Vsc2UgaWYocj09PVd1KWY9ITAsdC5jaGFyQ29kZUF0KGEpPT09VnUmJisrYTtl
bHNlIGlmKHIhPT1lKWNvbnRpbnVlO3JldHVybiB0LnNsaWNlKGksbil9cmV0dXJuIGM9ITAsdC5z
bGljZShpLG8pfWZvcih0LmNoYXJDb2RlQXQoby0xKT09PVZ1JiYtLW8sdC5jaGFyQ29kZUF0KG8t
MSk9PT1XdSYmLS1vOyhyPXMoKSkhPT1YdTspe2Zvcih2YXIgbD1bXTtyIT09SHUmJnIhPT1YdTsp
bC5wdXNoKHIpLHI9cygpO24mJm51bGw9PShsPW4obCx1KyspKXx8aS5wdXNoKGwpfXJldHVybiBp
fWZ1bmN0aW9uIGkobixlKXtyZXR1cm4gbi5tYXAoKGZ1bmN0aW9uKG4pe3JldHVybiBlLm1hcCgo
ZnVuY3Rpb24odCl7cmV0dXJuIGEoblt0XSl9KSkuam9pbih0KX0pKX1mdW5jdGlvbiBvKG4pe3Jl
dHVybiBuLm1hcChhKS5qb2luKHQpfWZ1bmN0aW9uIGEodCl7cmV0dXJuIG51bGw9PXQ/IiI6dCBp
bnN0YW5jZW9mIERhdGU/SnUodCk6bi50ZXN0KHQrPSIiKT8nIicrdC5yZXBsYWNlKC8iL2csJyIi
JykrJyInOnR9cmV0dXJue3BhcnNlOmZ1bmN0aW9uKHQsbil7dmFyIGUsaSxvPXIodCwoZnVuY3Rp
b24odCxyKXtpZihlKXJldHVybiBlKHQsci0xKTtpPXQsZT1uP2Z1bmN0aW9uKHQsbil7dmFyIGU9
WnUodCk7cmV0dXJuIGZ1bmN0aW9uKHIsaSl7cmV0dXJuIG4oZShyKSxpLHQpfX0odCxuKTpadSh0
KX0pKTtyZXR1cm4gby5jb2x1bW5zPWl8fFtdLG99LHBhcnNlUm93czpyLGZvcm1hdDpmdW5jdGlv
bihuLGUpe3JldHVybiBudWxsPT1lJiYoZT1LdShuKSksW2UubWFwKGEpLmpvaW4odCldLmNvbmNh
dChpKG4sZSkpLmpvaW4oIlxuIil9LGZvcm1hdEJvZHk6ZnVuY3Rpb24odCxuKXtyZXR1cm4gbnVs
bD09biYmKG49S3UodCkpLGkodCxuKS5qb2luKCJcbiIpfSxmb3JtYXRSb3dzOmZ1bmN0aW9uKHQp
e3JldHVybiB0Lm1hcChvKS5qb2luKCJcbiIpfSxmb3JtYXRSb3c6byxmb3JtYXRWYWx1ZTphfX12
YXIgbmM9dGMoIiwiKSxlYz1uYy5wYXJzZSxyYz1uYy5wYXJzZVJvd3MsaWM9bmMuZm9ybWF0LG9j
PW5jLmZvcm1hdEJvZHksYWM9bmMuZm9ybWF0Um93cyx1Yz1uYy5mb3JtYXRSb3csY2M9bmMuZm9y
bWF0VmFsdWUsZmM9dGMoIlx0Iiksc2M9ZmMucGFyc2UsbGM9ZmMucGFyc2VSb3dzLGhjPWZjLmZv
cm1hdCxkYz1mYy5mb3JtYXRCb2R5LHBjPWZjLmZvcm1hdFJvd3MsZ2M9ZmMuZm9ybWF0Um93LHlj
PWZjLmZvcm1hdFZhbHVlO2NvbnN0IHZjPW5ldyBEYXRlKCIyMDE5LTAxLTAxVDAwOjAwIikuZ2V0
SG91cnMoKXx8bmV3IERhdGUoIjIwMTktMDctMDFUMDA6MDAiKS5nZXRIb3VycygpO2Z1bmN0aW9u
IF9jKHQpe2lmKCF0Lm9rKXRocm93IG5ldyBFcnJvcih0LnN0YXR1cysiICIrdC5zdGF0dXNUZXh0
KTtyZXR1cm4gdC5ibG9iKCl9ZnVuY3Rpb24gYmModCl7aWYoIXQub2spdGhyb3cgbmV3IEVycm9y
KHQuc3RhdHVzKyIgIit0LnN0YXR1c1RleHQpO3JldHVybiB0LmFycmF5QnVmZmVyKCl9ZnVuY3Rp
b24gbWModCl7aWYoIXQub2spdGhyb3cgbmV3IEVycm9yKHQuc3RhdHVzKyIgIit0LnN0YXR1c1Rl
eHQpO3JldHVybiB0LnRleHQoKX1mdW5jdGlvbiB4Yyh0LG4pe3JldHVybiBmZXRjaCh0LG4pLnRo
ZW4obWMpfWZ1bmN0aW9uIHdjKHQpe3JldHVybiBmdW5jdGlvbihuLGUscil7cmV0dXJuIDI9PT1h
cmd1bWVudHMubGVuZ3RoJiYiZnVuY3Rpb24iPT10eXBlb2YgZSYmKHI9ZSxlPXZvaWQgMCkseGMo
bixlKS50aGVuKChmdW5jdGlvbihuKXtyZXR1cm4gdChuLHIpfSkpfX12YXIgTWM9d2MoZWMpLFRj
PXdjKHNjKTtmdW5jdGlvbiBBYyh0KXtpZighdC5vayl0aHJvdyBuZXcgRXJyb3IodC5zdGF0dXMr
IiAiK3Quc3RhdHVzVGV4dCk7aWYoMjA0IT09dC5zdGF0dXMmJjIwNSE9PXQuc3RhdHVzKXJldHVy
biB0Lmpzb24oKX1mdW5jdGlvbiBTYyh0KXtyZXR1cm4obixlKT0+eGMobixlKS50aGVuKChuPT4o
bmV3IERPTVBhcnNlcikucGFyc2VGcm9tU3RyaW5nKG4sdCkpKX12YXIgRWM9U2MoImFwcGxpY2F0
aW9uL3htbCIpLE5jPVNjKCJ0ZXh0L2h0bWwiKSxrYz1TYygiaW1hZ2Uvc3ZnK3htbCIpO2Z1bmN0
aW9uIENjKHQsbixlLHIpe2lmKGlzTmFOKG4pfHxpc05hTihlKSlyZXR1cm4gdDt2YXIgaSxvLGEs
dSxjLGYscyxsLGgsZD10Ll9yb290LHA9e2RhdGE6cn0sZz10Ll94MCx5PXQuX3kwLHY9dC5feDEs
Xz10Ll95MTtpZighZClyZXR1cm4gdC5fcm9vdD1wLHQ7Zm9yKDtkLmxlbmd0aDspaWYoKGY9bj49
KG89KGcrdikvMikpP2c9bzp2PW8sKHM9ZT49KGE9KHkrXykvMikpP3k9YTpfPWEsaT1kLCEoZD1k
W2w9czw8MXxmXSkpcmV0dXJuIGlbbF09cCx0O2lmKHU9K3QuX3guY2FsbChudWxsLGQuZGF0YSks
Yz0rdC5feS5jYWxsKG51bGwsZC5kYXRhKSxuPT09dSYmZT09PWMpcmV0dXJuIHAubmV4dD1kLGk/
aVtsXT1wOnQuX3Jvb3Q9cCx0O2Rve2k9aT9pW2xdPW5ldyBBcnJheSg0KTp0Ll9yb290PW5ldyBB
cnJheSg0KSwoZj1uPj0obz0oZyt2KS8yKSk/Zz1vOnY9bywocz1lPj0oYT0oeStfKS8yKSk/eT1h
Ol89YX13aGlsZSgobD1zPDwxfGYpPT0oaD0oYz49YSk8PDF8dT49bykpO3JldHVybiBpW2hdPWQs
aVtsXT1wLHR9ZnVuY3Rpb24gUGModCxuLGUscixpKXt0aGlzLm5vZGU9dCx0aGlzLngwPW4sdGhp
cy55MD1lLHRoaXMueDE9cix0aGlzLnkxPWl9ZnVuY3Rpb24gemModCl7cmV0dXJuIHRbMF19ZnVu
Y3Rpb24gJGModCl7cmV0dXJuIHRbMV19ZnVuY3Rpb24gRGModCxuLGUpe3ZhciByPW5ldyBSYyhu
dWxsPT1uP3pjOm4sbnVsbD09ZT8kYzplLE5hTixOYU4sTmFOLE5hTik7cmV0dXJuIG51bGw9PXQ/
cjpyLmFkZEFsbCh0KX1mdW5jdGlvbiBSYyh0LG4sZSxyLGksbyl7dGhpcy5feD10LHRoaXMuX3k9
bix0aGlzLl94MD1lLHRoaXMuX3kwPXIsdGhpcy5feDE9aSx0aGlzLl95MT1vLHRoaXMuX3Jvb3Q9
dm9pZCAwfWZ1bmN0aW9uIEZjKHQpe2Zvcih2YXIgbj17ZGF0YTp0LmRhdGF9LGU9bjt0PXQubmV4
dDspZT1lLm5leHQ9e2RhdGE6dC5kYXRhfTtyZXR1cm4gbn12YXIgcWM9RGMucHJvdG90eXBlPVJj
LnByb3RvdHlwZTtmdW5jdGlvbiBVYyh0KXtyZXR1cm4gZnVuY3Rpb24oKXtyZXR1cm4gdH19ZnVu
Y3Rpb24gSWModCl7cmV0dXJuIDFlLTYqKHQoKS0uNSl9ZnVuY3Rpb24gT2ModCl7cmV0dXJuIHQu
eCt0LnZ4fWZ1bmN0aW9uIEJjKHQpe3JldHVybiB0LnkrdC52eX1mdW5jdGlvbiBZYyh0KXtyZXR1
cm4gdC5pbmRleH1mdW5jdGlvbiBMYyh0LG4pe3ZhciBlPXQuZ2V0KG4pO2lmKCFlKXRocm93IG5l
dyBFcnJvcigibm9kZSBub3QgZm91bmQ6ICIrbik7cmV0dXJuIGV9cWMuY29weT1mdW5jdGlvbigp
e3ZhciB0LG4sZT1uZXcgUmModGhpcy5feCx0aGlzLl95LHRoaXMuX3gwLHRoaXMuX3kwLHRoaXMu
X3gxLHRoaXMuX3kxKSxyPXRoaXMuX3Jvb3Q7aWYoIXIpcmV0dXJuIGU7aWYoIXIubGVuZ3RoKXJl
dHVybiBlLl9yb290PUZjKHIpLGU7Zm9yKHQ9W3tzb3VyY2U6cix0YXJnZXQ6ZS5fcm9vdD1uZXcg
QXJyYXkoNCl9XTtyPXQucG9wKCk7KWZvcih2YXIgaT0wO2k8NDsrK2kpKG49ci5zb3VyY2VbaV0p
JiYobi5sZW5ndGg/dC5wdXNoKHtzb3VyY2U6bix0YXJnZXQ6ci50YXJnZXRbaV09bmV3IEFycmF5
KDQpfSk6ci50YXJnZXRbaV09RmMobikpO3JldHVybiBlfSxxYy5hZGQ9ZnVuY3Rpb24odCl7Y29u
c3Qgbj0rdGhpcy5feC5jYWxsKG51bGwsdCksZT0rdGhpcy5feS5jYWxsKG51bGwsdCk7cmV0dXJu
IENjKHRoaXMuY292ZXIobixlKSxuLGUsdCl9LHFjLmFkZEFsbD1mdW5jdGlvbih0KXt2YXIgbixl
LHIsaSxvPXQubGVuZ3RoLGE9bmV3IEFycmF5KG8pLHU9bmV3IEFycmF5KG8pLGM9MS8wLGY9MS8w
LHM9LTEvMCxsPS0xLzA7Zm9yKGU9MDtlPG87KytlKWlzTmFOKHI9K3RoaXMuX3guY2FsbChudWxs
LG49dFtlXSkpfHxpc05hTihpPSt0aGlzLl95LmNhbGwobnVsbCxuKSl8fChhW2VdPXIsdVtlXT1p
LHI8YyYmKGM9cikscj5zJiYocz1yKSxpPGYmJihmPWkpLGk+bCYmKGw9aSkpO2lmKGM+c3x8Zj5s
KXJldHVybiB0aGlzO2Zvcih0aGlzLmNvdmVyKGMsZikuY292ZXIocyxsKSxlPTA7ZTxvOysrZSlD
Yyh0aGlzLGFbZV0sdVtlXSx0W2VdKTtyZXR1cm4gdGhpc30scWMuY292ZXI9ZnVuY3Rpb24odCxu
KXtpZihpc05hTih0PSt0KXx8aXNOYU4obj0rbikpcmV0dXJuIHRoaXM7dmFyIGU9dGhpcy5feDAs
cj10aGlzLl95MCxpPXRoaXMuX3gxLG89dGhpcy5feTE7aWYoaXNOYU4oZSkpaT0oZT1NYXRoLmZs
b29yKHQpKSsxLG89KHI9TWF0aC5mbG9vcihuKSkrMTtlbHNle2Zvcih2YXIgYSx1LGM9aS1lfHwx
LGY9dGhpcy5fcm9vdDtlPnR8fHQ+PWl8fHI+bnx8bj49bzspc3dpdGNoKHU9KG48cik8PDF8dDxl
LChhPW5ldyBBcnJheSg0KSlbdV09ZixmPWEsYyo9Mix1KXtjYXNlIDA6aT1lK2Msbz1yK2M7YnJl
YWs7Y2FzZSAxOmU9aS1jLG89citjO2JyZWFrO2Nhc2UgMjppPWUrYyxyPW8tYzticmVhaztjYXNl
IDM6ZT1pLWMscj1vLWN9dGhpcy5fcm9vdCYmdGhpcy5fcm9vdC5sZW5ndGgmJih0aGlzLl9yb290
PWYpfXJldHVybiB0aGlzLl94MD1lLHRoaXMuX3kwPXIsdGhpcy5feDE9aSx0aGlzLl95MT1vLHRo
aXN9LHFjLmRhdGE9ZnVuY3Rpb24oKXt2YXIgdD1bXTtyZXR1cm4gdGhpcy52aXNpdCgoZnVuY3Rp
b24obil7aWYoIW4ubGVuZ3RoKWRve3QucHVzaChuLmRhdGEpfXdoaWxlKG49bi5uZXh0KX0pKSx0
fSxxYy5leHRlbnQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/dGhpcy5jb3Zl
cigrdFswXVswXSwrdFswXVsxXSkuY292ZXIoK3RbMV1bMF0sK3RbMV1bMV0pOmlzTmFOKHRoaXMu
X3gwKT92b2lkIDA6W1t0aGlzLl94MCx0aGlzLl95MF0sW3RoaXMuX3gxLHRoaXMuX3kxXV19LHFj
LmZpbmQ9ZnVuY3Rpb24odCxuLGUpe3ZhciByLGksbyxhLHUsYyxmLHM9dGhpcy5feDAsbD10aGlz
Ll95MCxoPXRoaXMuX3gxLGQ9dGhpcy5feTEscD1bXSxnPXRoaXMuX3Jvb3Q7Zm9yKGcmJnAucHVz
aChuZXcgUGMoZyxzLGwsaCxkKSksbnVsbD09ZT9lPTEvMDoocz10LWUsbD1uLWUsaD10K2UsZD1u
K2UsZSo9ZSk7Yz1wLnBvcCgpOylpZighKCEoZz1jLm5vZGUpfHwoaT1jLngwKT5ofHwobz1jLnkw
KT5kfHwoYT1jLngxKTxzfHwodT1jLnkxKTxsKSlpZihnLmxlbmd0aCl7dmFyIHk9KGkrYSkvMix2
PShvK3UpLzI7cC5wdXNoKG5ldyBQYyhnWzNdLHksdixhLHUpLG5ldyBQYyhnWzJdLGksdix5LHUp
LG5ldyBQYyhnWzFdLHksbyxhLHYpLG5ldyBQYyhnWzBdLGksbyx5LHYpKSwoZj0obj49dik8PDF8
dD49eSkmJihjPXBbcC5sZW5ndGgtMV0scFtwLmxlbmd0aC0xXT1wW3AubGVuZ3RoLTEtZl0scFtw
Lmxlbmd0aC0xLWZdPWMpfWVsc2V7dmFyIF89dC0rdGhpcy5feC5jYWxsKG51bGwsZy5kYXRhKSxi
PW4tK3RoaXMuX3kuY2FsbChudWxsLGcuZGF0YSksbT1fKl8rYipiO2lmKG08ZSl7dmFyIHg9TWF0
aC5zcXJ0KGU9bSk7cz10LXgsbD1uLXgsaD10K3gsZD1uK3gscj1nLmRhdGF9fXJldHVybiByfSxx
Yy5yZW1vdmU9ZnVuY3Rpb24odCl7aWYoaXNOYU4obz0rdGhpcy5feC5jYWxsKG51bGwsdCkpfHxp
c05hTihhPSt0aGlzLl95LmNhbGwobnVsbCx0KSkpcmV0dXJuIHRoaXM7dmFyIG4sZSxyLGksbyxh
LHUsYyxmLHMsbCxoLGQ9dGhpcy5fcm9vdCxwPXRoaXMuX3gwLGc9dGhpcy5feTAseT10aGlzLl94
MSx2PXRoaXMuX3kxO2lmKCFkKXJldHVybiB0aGlzO2lmKGQubGVuZ3RoKWZvcig7Oyl7aWYoKGY9
bz49KHU9KHAreSkvMikpP3A9dTp5PXUsKHM9YT49KGM9KGcrdikvMikpP2c9Yzp2PWMsbj1kLCEo
ZD1kW2w9czw8MXxmXSkpcmV0dXJuIHRoaXM7aWYoIWQubGVuZ3RoKWJyZWFrOyhuW2wrMSYzXXx8
bltsKzImM118fG5bbCszJjNdKSYmKGU9bixoPWwpfWZvcig7ZC5kYXRhIT09dDspaWYocj1kLCEo
ZD1kLm5leHQpKXJldHVybiB0aGlzO3JldHVybihpPWQubmV4dCkmJmRlbGV0ZSBkLm5leHQscj8o
aT9yLm5leHQ9aTpkZWxldGUgci5uZXh0LHRoaXMpOm4/KGk/bltsXT1pOmRlbGV0ZSBuW2xdLChk
PW5bMF18fG5bMV18fG5bMl18fG5bM10pJiZkPT09KG5bM118fG5bMl18fG5bMV18fG5bMF0pJiYh
ZC5sZW5ndGgmJihlP2VbaF09ZDp0aGlzLl9yb290PWQpLHRoaXMpOih0aGlzLl9yb290PWksdGhp
cyl9LHFjLnJlbW92ZUFsbD1mdW5jdGlvbih0KXtmb3IodmFyIG49MCxlPXQubGVuZ3RoO248ZTsr
K24pdGhpcy5yZW1vdmUodFtuXSk7cmV0dXJuIHRoaXN9LHFjLnJvb3Q9ZnVuY3Rpb24oKXtyZXR1
cm4gdGhpcy5fcm9vdH0scWMuc2l6ZT1mdW5jdGlvbigpe3ZhciB0PTA7cmV0dXJuIHRoaXMudmlz
aXQoKGZ1bmN0aW9uKG4pe2lmKCFuLmxlbmd0aClkb3srK3R9d2hpbGUobj1uLm5leHQpfSkpLHR9
LHFjLnZpc2l0PWZ1bmN0aW9uKHQpe3ZhciBuLGUscixpLG8sYSx1PVtdLGM9dGhpcy5fcm9vdDtm
b3IoYyYmdS5wdXNoKG5ldyBQYyhjLHRoaXMuX3gwLHRoaXMuX3kwLHRoaXMuX3gxLHRoaXMuX3kx
KSk7bj11LnBvcCgpOylpZighdChjPW4ubm9kZSxyPW4ueDAsaT1uLnkwLG89bi54MSxhPW4ueTEp
JiZjLmxlbmd0aCl7dmFyIGY9KHIrbykvMixzPShpK2EpLzI7KGU9Y1szXSkmJnUucHVzaChuZXcg
UGMoZSxmLHMsbyxhKSksKGU9Y1syXSkmJnUucHVzaChuZXcgUGMoZSxyLHMsZixhKSksKGU9Y1sx
XSkmJnUucHVzaChuZXcgUGMoZSxmLGksbyxzKSksKGU9Y1swXSkmJnUucHVzaChuZXcgUGMoZSxy
LGksZixzKSl9cmV0dXJuIHRoaXN9LHFjLnZpc2l0QWZ0ZXI9ZnVuY3Rpb24odCl7dmFyIG4sZT1b
XSxyPVtdO2Zvcih0aGlzLl9yb290JiZlLnB1c2gobmV3IFBjKHRoaXMuX3Jvb3QsdGhpcy5feDAs
dGhpcy5feTAsdGhpcy5feDEsdGhpcy5feTEpKTtuPWUucG9wKCk7KXt2YXIgaT1uLm5vZGU7aWYo
aS5sZW5ndGgpe3ZhciBvLGE9bi54MCx1PW4ueTAsYz1uLngxLGY9bi55MSxzPShhK2MpLzIsbD0o
dStmKS8yOyhvPWlbMF0pJiZlLnB1c2gobmV3IFBjKG8sYSx1LHMsbCkpLChvPWlbMV0pJiZlLnB1
c2gobmV3IFBjKG8scyx1LGMsbCkpLChvPWlbMl0pJiZlLnB1c2gobmV3IFBjKG8sYSxsLHMsZikp
LChvPWlbM10pJiZlLnB1c2gobmV3IFBjKG8scyxsLGMsZikpfXIucHVzaChuKX1mb3IoO249ci5w
b3AoKTspdChuLm5vZGUsbi54MCxuLnkwLG4ueDEsbi55MSk7cmV0dXJuIHRoaXN9LHFjLng9ZnVu
Y3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHRoaXMuX3g9dCx0aGlzKTp0aGlzLl94
fSxxYy55PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh0aGlzLl95PXQsdGhp
cyk6dGhpcy5feX07Y29uc3QgamM9MTY2NDUyNSxIYz0xMDEzOTA0MjIzLFhjPTQyOTQ5NjcyOTY7
ZnVuY3Rpb24gR2ModCl7cmV0dXJuIHQueH1mdW5jdGlvbiBWYyh0KXtyZXR1cm4gdC55fXZhciBX
Yz1NYXRoLlBJKigzLU1hdGguc3FydCg1KSk7ZnVuY3Rpb24gWmModCxuKXtpZigoZT0odD1uP3Qu
dG9FeHBvbmVudGlhbChuLTEpOnQudG9FeHBvbmVudGlhbCgpKS5pbmRleE9mKCJlIikpPDApcmV0
dXJuIG51bGw7dmFyIGUscj10LnNsaWNlKDAsZSk7cmV0dXJuW3IubGVuZ3RoPjE/clswXStyLnNs
aWNlKDIpOnIsK3Quc2xpY2UoZSsxKV19ZnVuY3Rpb24gS2ModCl7cmV0dXJuKHQ9WmMoTWF0aC5h
YnModCkpKT90WzFdOk5hTn12YXIgUWMsSmM9L14oPzooLik/KFs8Pj1eXSkpPyhbK1wtKCBdKT8o
WyQjXSk/KDApPyhcZCspPygsKT8oXC5cZCspPyh+KT8oW2EteiVdKT8kL2k7ZnVuY3Rpb24gdGYo
dCl7aWYoIShuPUpjLmV4ZWModCkpKXRocm93IG5ldyBFcnJvcigiaW52YWxpZCBmb3JtYXQ6ICIr
dCk7dmFyIG47cmV0dXJuIG5ldyBuZih7ZmlsbDpuWzFdLGFsaWduOm5bMl0sc2lnbjpuWzNdLHN5
bWJvbDpuWzRdLHplcm86bls1XSx3aWR0aDpuWzZdLGNvbW1hOm5bN10scHJlY2lzaW9uOm5bOF0m
Jm5bOF0uc2xpY2UoMSksdHJpbTpuWzldLHR5cGU6blsxMF19KX1mdW5jdGlvbiBuZih0KXt0aGlz
LmZpbGw9dm9pZCAwPT09dC5maWxsPyIgIjp0LmZpbGwrIiIsdGhpcy5hbGlnbj12b2lkIDA9PT10
LmFsaWduPyI+Ijp0LmFsaWduKyIiLHRoaXMuc2lnbj12b2lkIDA9PT10LnNpZ24/Ii0iOnQuc2ln
bisiIix0aGlzLnN5bWJvbD12b2lkIDA9PT10LnN5bWJvbD8iIjp0LnN5bWJvbCsiIix0aGlzLnpl
cm89ISF0Lnplcm8sdGhpcy53aWR0aD12b2lkIDA9PT10LndpZHRoP3ZvaWQgMDordC53aWR0aCx0
aGlzLmNvbW1hPSEhdC5jb21tYSx0aGlzLnByZWNpc2lvbj12b2lkIDA9PT10LnByZWNpc2lvbj92
b2lkIDA6K3QucHJlY2lzaW9uLHRoaXMudHJpbT0hIXQudHJpbSx0aGlzLnR5cGU9dm9pZCAwPT09
dC50eXBlPyIiOnQudHlwZSsiIn1mdW5jdGlvbiBlZih0LG4pe3ZhciBlPVpjKHQsbik7aWYoIWUp
cmV0dXJuIHQrIiI7dmFyIHI9ZVswXSxpPWVbMV07cmV0dXJuIGk8MD8iMC4iK25ldyBBcnJheSgt
aSkuam9pbigiMCIpK3I6ci5sZW5ndGg+aSsxP3Iuc2xpY2UoMCxpKzEpKyIuIityLnNsaWNlKGkr
MSk6cituZXcgQXJyYXkoaS1yLmxlbmd0aCsyKS5qb2luKCIwIil9dGYucHJvdG90eXBlPW5mLnBy
b3RvdHlwZSxuZi5wcm90b3R5cGUudG9TdHJpbmc9ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5maWxs
K3RoaXMuYWxpZ24rdGhpcy5zaWduK3RoaXMuc3ltYm9sKyh0aGlzLnplcm8/IjAiOiIiKSsodm9p
ZCAwPT09dGhpcy53aWR0aD8iIjpNYXRoLm1heCgxLDB8dGhpcy53aWR0aCkpKyh0aGlzLmNvbW1h
PyIsIjoiIikrKHZvaWQgMD09PXRoaXMucHJlY2lzaW9uPyIiOiIuIitNYXRoLm1heCgwLDB8dGhp
cy5wcmVjaXNpb24pKSsodGhpcy50cmltPyJ+IjoiIikrdGhpcy50eXBlfTt2YXIgcmY9eyIlIjoo
dCxuKT0+KDEwMCp0KS50b0ZpeGVkKG4pLGI6dD0+TWF0aC5yb3VuZCh0KS50b1N0cmluZygyKSxj
OnQ9PnQrIiIsZDpmdW5jdGlvbih0KXtyZXR1cm4gTWF0aC5hYnModD1NYXRoLnJvdW5kKHQpKT49
MWUyMT90LnRvTG9jYWxlU3RyaW5nKCJlbiIpLnJlcGxhY2UoLywvZywiIik6dC50b1N0cmluZygx
MCl9LGU6KHQsbik9PnQudG9FeHBvbmVudGlhbChuKSxmOih0LG4pPT50LnRvRml4ZWQobiksZzoo
dCxuKT0+dC50b1ByZWNpc2lvbihuKSxvOnQ9Pk1hdGgucm91bmQodCkudG9TdHJpbmcoOCkscDoo
dCxuKT0+ZWYoMTAwKnQsbikscjplZixzOmZ1bmN0aW9uKHQsbil7dmFyIGU9WmModCxuKTtpZigh
ZSlyZXR1cm4gdCsiIjt2YXIgcj1lWzBdLGk9ZVsxXSxvPWktKFFjPTMqTWF0aC5tYXgoLTgsTWF0
aC5taW4oOCxNYXRoLmZsb29yKGkvMykpKSkrMSxhPXIubGVuZ3RoO3JldHVybiBvPT09YT9yOm8+
YT9yK25ldyBBcnJheShvLWErMSkuam9pbigiMCIpOm8+MD9yLnNsaWNlKDAsbykrIi4iK3Iuc2xp
Y2Uobyk6IjAuIituZXcgQXJyYXkoMS1vKS5qb2luKCIwIikrWmModCxNYXRoLm1heCgwLG4rby0x
KSlbMF19LFg6dD0+TWF0aC5yb3VuZCh0KS50b1N0cmluZygxNikudG9VcHBlckNhc2UoKSx4OnQ9
Pk1hdGgucm91bmQodCkudG9TdHJpbmcoMTYpfTtmdW5jdGlvbiBvZih0KXtyZXR1cm4gdH12YXIg
YWYsdWY9QXJyYXkucHJvdG90eXBlLm1hcCxjZj1bInkiLCJ6IiwiYSIsImYiLCJwIiwibiIsIsK1
IiwibSIsIiIsImsiLCJNIiwiRyIsIlQiLCJQIiwiRSIsIloiLCJZIl07ZnVuY3Rpb24gZmYodCl7
dmFyIG4sZSxyPXZvaWQgMD09PXQuZ3JvdXBpbmd8fHZvaWQgMD09PXQudGhvdXNhbmRzP29mOihu
PXVmLmNhbGwodC5ncm91cGluZyxOdW1iZXIpLGU9dC50aG91c2FuZHMrIiIsZnVuY3Rpb24odCxy
KXtmb3IodmFyIGk9dC5sZW5ndGgsbz1bXSxhPTAsdT1uWzBdLGM9MDtpPjAmJnU+MCYmKGMrdSsx
PnImJih1PU1hdGgubWF4KDEsci1jKSksby5wdXNoKHQuc3Vic3RyaW5nKGktPXUsaSt1KSksISgo
Yys9dSsxKT5yKSk7KXU9blthPShhKzEpJW4ubGVuZ3RoXTtyZXR1cm4gby5yZXZlcnNlKCkuam9p
bihlKX0pLGk9dm9pZCAwPT09dC5jdXJyZW5jeT8iIjp0LmN1cnJlbmN5WzBdKyIiLG89dm9pZCAw
PT09dC5jdXJyZW5jeT8iIjp0LmN1cnJlbmN5WzFdKyIiLGE9dm9pZCAwPT09dC5kZWNpbWFsPyIu
Ijp0LmRlY2ltYWwrIiIsdT12b2lkIDA9PT10Lm51bWVyYWxzP29mOmZ1bmN0aW9uKHQpe3JldHVy
biBmdW5jdGlvbihuKXtyZXR1cm4gbi5yZXBsYWNlKC9bMC05XS9nLChmdW5jdGlvbihuKXtyZXR1
cm4gdFsrbl19KSl9fSh1Zi5jYWxsKHQubnVtZXJhbHMsU3RyaW5nKSksYz12b2lkIDA9PT10LnBl
cmNlbnQ/IiUiOnQucGVyY2VudCsiIixmPXZvaWQgMD09PXQubWludXM/IuKIkiI6dC5taW51cysi
IixzPXZvaWQgMD09PXQubmFuPyJOYU4iOnQubmFuKyIiO2Z1bmN0aW9uIGwodCl7dmFyIG49KHQ9
dGYodCkpLmZpbGwsZT10LmFsaWduLGw9dC5zaWduLGg9dC5zeW1ib2wsZD10Lnplcm8scD10Lndp
ZHRoLGc9dC5jb21tYSx5PXQucHJlY2lzaW9uLHY9dC50cmltLF89dC50eXBlOyJuIj09PV8/KGc9
ITAsXz0iZyIpOnJmW19dfHwodm9pZCAwPT09eSYmKHk9MTIpLHY9ITAsXz0iZyIpLChkfHwiMCI9
PT1uJiYiPSI9PT1lKSYmKGQ9ITAsbj0iMCIsZT0iPSIpO3ZhciBiPSIkIj09PWg/aToiIyI9PT1o
JiYvW2JveFhdLy50ZXN0KF8pPyIwIitfLnRvTG93ZXJDYXNlKCk6IiIsbT0iJCI9PT1oP286L1sl
cF0vLnRlc3QoXyk/YzoiIix4PXJmW19dLHc9L1tkZWZncHJzJV0vLnRlc3QoXyk7ZnVuY3Rpb24g
TSh0KXt2YXIgaSxvLGMsaD1iLE09bTtpZigiYyI9PT1fKU09eCh0KStNLHQ9IiI7ZWxzZXt2YXIg
VD0odD0rdCk8MHx8MS90PDA7aWYodD1pc05hTih0KT9zOngoTWF0aC5hYnModCkseSksdiYmKHQ9
ZnVuY3Rpb24odCl7dDpmb3IodmFyIG4sZT10Lmxlbmd0aCxyPTEsaT0tMTtyPGU7KytyKXN3aXRj
aCh0W3JdKXtjYXNlIi4iOmk9bj1yO2JyZWFrO2Nhc2UiMCI6MD09PWkmJihpPXIpLG49cjticmVh
aztkZWZhdWx0OmlmKCErdFtyXSlicmVhayB0O2k+MCYmKGk9MCl9cmV0dXJuIGk+MD90LnNsaWNl
KDAsaSkrdC5zbGljZShuKzEpOnR9KHQpKSxUJiYwPT0rdCYmIisiIT09bCYmKFQ9ITEpLGg9KFQ/
IigiPT09bD9sOmY6Ii0iPT09bHx8IigiPT09bD8iIjpsKStoLE09KCJzIj09PV8/Y2ZbOCtRYy8z
XToiIikrTSsoVCYmIigiPT09bD8iKSI6IiIpLHcpZm9yKGk9LTEsbz10Lmxlbmd0aDsrK2k8bzsp
aWYoNDg+KGM9dC5jaGFyQ29kZUF0KGkpKXx8Yz41Nyl7TT0oNDY9PT1jP2ErdC5zbGljZShpKzEp
OnQuc2xpY2UoaSkpK00sdD10LnNsaWNlKDAsaSk7YnJlYWt9fWcmJiFkJiYodD1yKHQsMS8wKSk7
dmFyIEE9aC5sZW5ndGgrdC5sZW5ndGgrTS5sZW5ndGgsUz1BPHA/bmV3IEFycmF5KHAtQSsxKS5q
b2luKG4pOiIiO3N3aXRjaChnJiZkJiYodD1yKFMrdCxTLmxlbmd0aD9wLU0ubGVuZ3RoOjEvMCks
Uz0iIiksZSl7Y2FzZSI8Ijp0PWgrdCtNK1M7YnJlYWs7Y2FzZSI9Ijp0PWgrUyt0K007YnJlYWs7
Y2FzZSJeIjp0PVMuc2xpY2UoMCxBPVMubGVuZ3RoPj4xKStoK3QrTStTLnNsaWNlKEEpO2JyZWFr
O2RlZmF1bHQ6dD1TK2grdCtNfXJldHVybiB1KHQpfXJldHVybiB5PXZvaWQgMD09PXk/NjovW2dw
cnNdLy50ZXN0KF8pP01hdGgubWF4KDEsTWF0aC5taW4oMjEseSkpOk1hdGgubWF4KDAsTWF0aC5t
aW4oMjAseSkpLE0udG9TdHJpbmc9ZnVuY3Rpb24oKXtyZXR1cm4gdCsiIn0sTX1yZXR1cm57Zm9y
bWF0OmwsZm9ybWF0UHJlZml4OmZ1bmN0aW9uKHQsbil7dmFyIGU9bCgoKHQ9dGYodCkpLnR5cGU9
ImYiLHQpKSxyPTMqTWF0aC5tYXgoLTgsTWF0aC5taW4oOCxNYXRoLmZsb29yKEtjKG4pLzMpKSks
aT1NYXRoLnBvdygxMCwtciksbz1jZls4K3IvM107cmV0dXJuIGZ1bmN0aW9uKHQpe3JldHVybiBl
KGkqdCkrb319fX1mdW5jdGlvbiBzZihuKXtyZXR1cm4gYWY9ZmYobiksdC5mb3JtYXQ9YWYuZm9y
bWF0LHQuZm9ybWF0UHJlZml4PWFmLmZvcm1hdFByZWZpeCxhZn1mdW5jdGlvbiBsZih0KXtyZXR1
cm4gTWF0aC5tYXgoMCwtS2MoTWF0aC5hYnModCkpKX1mdW5jdGlvbiBoZih0LG4pe3JldHVybiBN
YXRoLm1heCgwLDMqTWF0aC5tYXgoLTgsTWF0aC5taW4oOCxNYXRoLmZsb29yKEtjKG4pLzMpKSkt
S2MoTWF0aC5hYnModCkpKX1mdW5jdGlvbiBkZih0LG4pe3JldHVybiB0PU1hdGguYWJzKHQpLG49
TWF0aC5hYnMobiktdCxNYXRoLm1heCgwLEtjKG4pLUtjKHQpKSsxfXQuZm9ybWF0PXZvaWQgMCx0
LmZvcm1hdFByZWZpeD12b2lkIDAsc2Yoe3Rob3VzYW5kczoiLCIsZ3JvdXBpbmc6WzNdLGN1cnJl
bmN5OlsiJCIsIiJdfSk7dmFyIHBmPTFlLTYsZ2Y9MWUtMTIseWY9TWF0aC5QSSx2Zj15Zi8yLF9m
PXlmLzQsYmY9Mip5ZixtZj0xODAveWYseGY9eWYvMTgwLHdmPU1hdGguYWJzLE1mPU1hdGguYXRh
bixUZj1NYXRoLmF0YW4yLEFmPU1hdGguY29zLFNmPU1hdGguY2VpbCxFZj1NYXRoLmV4cCxOZj1N
YXRoLmh5cG90LGtmPU1hdGgubG9nLENmPU1hdGgucG93LFBmPU1hdGguc2luLHpmPU1hdGguc2ln
bnx8ZnVuY3Rpb24odCl7cmV0dXJuIHQ+MD8xOnQ8MD8tMTowfSwkZj1NYXRoLnNxcnQsRGY9TWF0
aC50YW47ZnVuY3Rpb24gUmYodCl7cmV0dXJuIHQ+MT8wOnQ8LTE/eWY6TWF0aC5hY29zKHQpfWZ1
bmN0aW9uIEZmKHQpe3JldHVybiB0PjE/dmY6dDwtMT8tdmY6TWF0aC5hc2luKHQpfWZ1bmN0aW9u
IHFmKHQpe3JldHVybih0PVBmKHQvMikpKnR9ZnVuY3Rpb24gVWYoKXt9ZnVuY3Rpb24gSWYodCxu
KXt0JiZCZi5oYXNPd25Qcm9wZXJ0eSh0LnR5cGUpJiZCZlt0LnR5cGVdKHQsbil9dmFyIE9mPXtG
ZWF0dXJlOmZ1bmN0aW9uKHQsbil7SWYodC5nZW9tZXRyeSxuKX0sRmVhdHVyZUNvbGxlY3Rpb246
ZnVuY3Rpb24odCxuKXtmb3IodmFyIGU9dC5mZWF0dXJlcyxyPS0xLGk9ZS5sZW5ndGg7KytyPGk7
KUlmKGVbcl0uZ2VvbWV0cnksbil9fSxCZj17U3BoZXJlOmZ1bmN0aW9uKHQsbil7bi5zcGhlcmUo
KX0sUG9pbnQ6ZnVuY3Rpb24odCxuKXt0PXQuY29vcmRpbmF0ZXMsbi5wb2ludCh0WzBdLHRbMV0s
dFsyXSl9LE11bHRpUG9pbnQ6ZnVuY3Rpb24odCxuKXtmb3IodmFyIGU9dC5jb29yZGluYXRlcyxy
PS0xLGk9ZS5sZW5ndGg7KytyPGk7KXQ9ZVtyXSxuLnBvaW50KHRbMF0sdFsxXSx0WzJdKX0sTGlu
ZVN0cmluZzpmdW5jdGlvbih0LG4pe1lmKHQuY29vcmRpbmF0ZXMsbiwwKX0sTXVsdGlMaW5lU3Ry
aW5nOmZ1bmN0aW9uKHQsbil7Zm9yKHZhciBlPXQuY29vcmRpbmF0ZXMscj0tMSxpPWUubGVuZ3Ro
OysrcjxpOylZZihlW3JdLG4sMCl9LFBvbHlnb246ZnVuY3Rpb24odCxuKXtMZih0LmNvb3JkaW5h
dGVzLG4pfSxNdWx0aVBvbHlnb246ZnVuY3Rpb24odCxuKXtmb3IodmFyIGU9dC5jb29yZGluYXRl
cyxyPS0xLGk9ZS5sZW5ndGg7KytyPGk7KUxmKGVbcl0sbil9LEdlb21ldHJ5Q29sbGVjdGlvbjpm
dW5jdGlvbih0LG4pe2Zvcih2YXIgZT10Lmdlb21ldHJpZXMscj0tMSxpPWUubGVuZ3RoOysrcjxp
OylJZihlW3JdLG4pfX07ZnVuY3Rpb24gWWYodCxuLGUpe3ZhciByLGk9LTEsbz10Lmxlbmd0aC1l
O2ZvcihuLmxpbmVTdGFydCgpOysraTxvOylyPXRbaV0sbi5wb2ludChyWzBdLHJbMV0sclsyXSk7
bi5saW5lRW5kKCl9ZnVuY3Rpb24gTGYodCxuKXt2YXIgZT0tMSxyPXQubGVuZ3RoO2ZvcihuLnBv
bHlnb25TdGFydCgpOysrZTxyOylZZih0W2VdLG4sMSk7bi5wb2x5Z29uRW5kKCl9ZnVuY3Rpb24g
amYodCxuKXt0JiZPZi5oYXNPd25Qcm9wZXJ0eSh0LnR5cGUpP09mW3QudHlwZV0odCxuKTpJZih0
LG4pfXZhciBIZixYZixHZixWZixXZixaZixLZixRZixKZix0cyxucyxlcyxycyxpcyxvcyxhcyx1
cz1uZXcgQSxjcz1uZXcgQSxmcz17cG9pbnQ6VWYsbGluZVN0YXJ0OlVmLGxpbmVFbmQ6VWYscG9s
eWdvblN0YXJ0OmZ1bmN0aW9uKCl7dXM9bmV3IEEsZnMubGluZVN0YXJ0PXNzLGZzLmxpbmVFbmQ9
bHN9LHBvbHlnb25FbmQ6ZnVuY3Rpb24oKXt2YXIgdD0rdXM7Y3MuYWRkKHQ8MD9iZit0OnQpLHRo
aXMubGluZVN0YXJ0PXRoaXMubGluZUVuZD10aGlzLnBvaW50PVVmfSxzcGhlcmU6ZnVuY3Rpb24o
KXtjcy5hZGQoYmYpfX07ZnVuY3Rpb24gc3MoKXtmcy5wb2ludD1oc31mdW5jdGlvbiBscygpe2Rz
KEhmLFhmKX1mdW5jdGlvbiBocyh0LG4pe2ZzLnBvaW50PWRzLEhmPXQsWGY9bixHZj10Kj14ZixW
Zj1BZihuPShuKj14ZikvMitfZiksV2Y9UGYobil9ZnVuY3Rpb24gZHModCxuKXt2YXIgZT0odCo9
eGYpLUdmLHI9ZT49MD8xOi0xLGk9ciplLG89QWYobj0obio9eGYpLzIrX2YpLGE9UGYobiksdT1X
ZiphLGM9VmYqbyt1KkFmKGkpLGY9dSpyKlBmKGkpO3VzLmFkZChUZihmLGMpKSxHZj10LFZmPW8s
V2Y9YX1mdW5jdGlvbiBwcyh0KXtyZXR1cm5bVGYodFsxXSx0WzBdKSxGZih0WzJdKV19ZnVuY3Rp
b24gZ3ModCl7dmFyIG49dFswXSxlPXRbMV0scj1BZihlKTtyZXR1cm5bcipBZihuKSxyKlBmKG4p
LFBmKGUpXX1mdW5jdGlvbiB5cyh0LG4pe3JldHVybiB0WzBdKm5bMF0rdFsxXSpuWzFdK3RbMl0q
blsyXX1mdW5jdGlvbiB2cyh0LG4pe3JldHVyblt0WzFdKm5bMl0tdFsyXSpuWzFdLHRbMl0qblsw
XS10WzBdKm5bMl0sdFswXSpuWzFdLXRbMV0qblswXV19ZnVuY3Rpb24gX3ModCxuKXt0WzBdKz1u
WzBdLHRbMV0rPW5bMV0sdFsyXSs9blsyXX1mdW5jdGlvbiBicyh0LG4pe3JldHVyblt0WzBdKm4s
dFsxXSpuLHRbMl0qbl19ZnVuY3Rpb24gbXModCl7dmFyIG49JGYodFswXSp0WzBdK3RbMV0qdFsx
XSt0WzJdKnRbMl0pO3RbMF0vPW4sdFsxXS89bix0WzJdLz1ufXZhciB4cyx3cyxNcyxUcyxBcyxT
cyxFcyxOcyxrcyxDcyxQcyx6cywkcyxEcyxScyxGcyxxcz17cG9pbnQ6VXMsbGluZVN0YXJ0Ok9z
LGxpbmVFbmQ6QnMscG9seWdvblN0YXJ0OmZ1bmN0aW9uKCl7cXMucG9pbnQ9WXMscXMubGluZVN0
YXJ0PUxzLHFzLmxpbmVFbmQ9anMsaXM9bmV3IEEsZnMucG9seWdvblN0YXJ0KCl9LHBvbHlnb25F
bmQ6ZnVuY3Rpb24oKXtmcy5wb2x5Z29uRW5kKCkscXMucG9pbnQ9VXMscXMubGluZVN0YXJ0PU9z
LHFzLmxpbmVFbmQ9QnMsdXM8MD8oWmY9LShRZj0xODApLEtmPS0oSmY9OTApKTppcz5wZj9KZj05
MDppczwtcGYmJihLZj0tOTApLGFzWzBdPVpmLGFzWzFdPVFmfSxzcGhlcmU6ZnVuY3Rpb24oKXta
Zj0tKFFmPTE4MCksS2Y9LShKZj05MCl9fTtmdW5jdGlvbiBVcyh0LG4pe29zLnB1c2goYXM9W1pm
PXQsUWY9dF0pLG48S2YmJihLZj1uKSxuPkpmJiYoSmY9bil9ZnVuY3Rpb24gSXModCxuKXt2YXIg
ZT1ncyhbdCp4ZixuKnhmXSk7aWYocnMpe3ZhciByPXZzKHJzLGUpLGk9dnMoW3JbMV0sLXJbMF0s
MF0scik7bXMoaSksaT1wcyhpKTt2YXIgbyxhPXQtdHMsdT1hPjA/MTotMSxjPWlbMF0qbWYqdSxm
PXdmKGEpPjE4MDtmXih1KnRzPGMmJmM8dSp0KT8obz1pWzFdKm1mKT5KZiYmKEpmPW8pOmZeKHUq
dHM8KGM9KGMrMzYwKSUzNjAtMTgwKSYmYzx1KnQpPyhvPS1pWzFdKm1mKTxLZiYmKEtmPW8pOihu
PEtmJiYoS2Y9biksbj5KZiYmKEpmPW4pKSxmP3Q8dHM/SHMoWmYsdCk+SHMoWmYsUWYpJiYoUWY9
dCk6SHModCxRZik+SHMoWmYsUWYpJiYoWmY9dCk6UWY+PVpmPyh0PFpmJiYoWmY9dCksdD5RZiYm
KFFmPXQpKTp0PnRzP0hzKFpmLHQpPkhzKFpmLFFmKSYmKFFmPXQpOkhzKHQsUWYpPkhzKFpmLFFm
KSYmKFpmPXQpfWVsc2Ugb3MucHVzaChhcz1bWmY9dCxRZj10XSk7bjxLZiYmKEtmPW4pLG4+SmYm
JihKZj1uKSxycz1lLHRzPXR9ZnVuY3Rpb24gT3MoKXtxcy5wb2ludD1Jc31mdW5jdGlvbiBCcygp
e2FzWzBdPVpmLGFzWzFdPVFmLHFzLnBvaW50PVVzLHJzPW51bGx9ZnVuY3Rpb24gWXModCxuKXtp
Zihycyl7dmFyIGU9dC10cztpcy5hZGQod2YoZSk+MTgwP2UrKGU+MD8zNjA6LTM2MCk6ZSl9ZWxz
ZSBucz10LGVzPW47ZnMucG9pbnQodCxuKSxJcyh0LG4pfWZ1bmN0aW9uIExzKCl7ZnMubGluZVN0
YXJ0KCl9ZnVuY3Rpb24ganMoKXtZcyhucyxlcyksZnMubGluZUVuZCgpLHdmKGlzKT5wZiYmKFpm
PS0oUWY9MTgwKSksYXNbMF09WmYsYXNbMV09UWYscnM9bnVsbH1mdW5jdGlvbiBIcyh0LG4pe3Jl
dHVybihuLT10KTwwP24rMzYwOm59ZnVuY3Rpb24gWHModCxuKXtyZXR1cm4gdFswXS1uWzBdfWZ1
bmN0aW9uIEdzKHQsbil7cmV0dXJuIHRbMF08PXRbMV0/dFswXTw9biYmbjw9dFsxXTpuPHRbMF18
fHRbMV08bn12YXIgVnM9e3NwaGVyZTpVZixwb2ludDpXcyxsaW5lU3RhcnQ6S3MsbGluZUVuZDp0
bCxwb2x5Z29uU3RhcnQ6ZnVuY3Rpb24oKXtWcy5saW5lU3RhcnQ9bmwsVnMubGluZUVuZD1lbH0s
cG9seWdvbkVuZDpmdW5jdGlvbigpe1ZzLmxpbmVTdGFydD1LcyxWcy5saW5lRW5kPXRsfX07ZnVu
Y3Rpb24gV3ModCxuKXt0Kj14Zjt2YXIgZT1BZihuKj14Zik7WnMoZSpBZih0KSxlKlBmKHQpLFBm
KG4pKX1mdW5jdGlvbiBacyh0LG4sZSl7Kyt4cyxNcys9KHQtTXMpL3hzLFRzKz0obi1UcykveHMs
QXMrPShlLUFzKS94c31mdW5jdGlvbiBLcygpe1ZzLnBvaW50PVFzfWZ1bmN0aW9uIFFzKHQsbil7
dCo9eGY7dmFyIGU9QWYobio9eGYpO0RzPWUqQWYodCksUnM9ZSpQZih0KSxGcz1QZihuKSxWcy5w
b2ludD1KcyxacyhEcyxScyxGcyl9ZnVuY3Rpb24gSnModCxuKXt0Kj14Zjt2YXIgZT1BZihuKj14
Zikscj1lKkFmKHQpLGk9ZSpQZih0KSxvPVBmKG4pLGE9VGYoJGYoKGE9UnMqby1GcyppKSphKyhh
PUZzKnItRHMqbykqYSsoYT1EcyppLVJzKnIpKmEpLERzKnIrUnMqaStGcypvKTt3cys9YSxTcys9
YSooRHMrKERzPXIpKSxFcys9YSooUnMrKFJzPWkpKSxOcys9YSooRnMrKEZzPW8pKSxacyhEcyxS
cyxGcyl9ZnVuY3Rpb24gdGwoKXtWcy5wb2ludD1Xc31mdW5jdGlvbiBubCgpe1ZzLnBvaW50PXJs
fWZ1bmN0aW9uIGVsKCl7aWwoenMsJHMpLFZzLnBvaW50PVdzfWZ1bmN0aW9uIHJsKHQsbil7enM9
dCwkcz1uLHQqPXhmLG4qPXhmLFZzLnBvaW50PWlsO3ZhciBlPUFmKG4pO0RzPWUqQWYodCksUnM9
ZSpQZih0KSxGcz1QZihuKSxacyhEcyxScyxGcyl9ZnVuY3Rpb24gaWwodCxuKXt0Kj14Zjt2YXIg
ZT1BZihuKj14Zikscj1lKkFmKHQpLGk9ZSpQZih0KSxvPVBmKG4pLGE9UnMqby1GcyppLHU9RnMq
ci1EcypvLGM9RHMqaS1ScypyLGY9TmYoYSx1LGMpLHM9RmYoZiksbD1mJiYtcy9mO2tzLmFkZChs
KmEpLENzLmFkZChsKnUpLFBzLmFkZChsKmMpLHdzKz1zLFNzKz1zKihEcysoRHM9cikpLEVzKz1z
KihScysoUnM9aSkpLE5zKz1zKihGcysoRnM9bykpLFpzKERzLFJzLEZzKX1mdW5jdGlvbiBvbCh0
KXtyZXR1cm4gZnVuY3Rpb24oKXtyZXR1cm4gdH19ZnVuY3Rpb24gYWwodCxuKXtmdW5jdGlvbiBl
KGUscil7cmV0dXJuIGU9dChlLHIpLG4oZVswXSxlWzFdKX1yZXR1cm4gdC5pbnZlcnQmJm4uaW52
ZXJ0JiYoZS5pbnZlcnQ9ZnVuY3Rpb24oZSxyKXtyZXR1cm4oZT1uLmludmVydChlLHIpKSYmdC5p
bnZlcnQoZVswXSxlWzFdKX0pLGV9ZnVuY3Rpb24gdWwodCxuKXtyZXR1cm4gd2YodCk+eWYmJih0
LT1NYXRoLnJvdW5kKHQvYmYpKmJmKSxbdCxuXX1mdW5jdGlvbiBjbCh0LG4sZSl7cmV0dXJuKHQl
PWJmKT9ufHxlP2FsKHNsKHQpLGxsKG4sZSkpOnNsKHQpOm58fGU/bGwobixlKTp1bH1mdW5jdGlv
biBmbCh0KXtyZXR1cm4gZnVuY3Rpb24obixlKXtyZXR1cm4gd2Yobis9dCk+eWYmJihuLT1NYXRo
LnJvdW5kKG4vYmYpKmJmKSxbbixlXX19ZnVuY3Rpb24gc2wodCl7dmFyIG49ZmwodCk7cmV0dXJu
IG4uaW52ZXJ0PWZsKC10KSxufWZ1bmN0aW9uIGxsKHQsbil7dmFyIGU9QWYodCkscj1QZih0KSxp
PUFmKG4pLG89UGYobik7ZnVuY3Rpb24gYSh0LG4pe3ZhciBhPUFmKG4pLHU9QWYodCkqYSxjPVBm
KHQpKmEsZj1QZihuKSxzPWYqZSt1KnI7cmV0dXJuW1RmKGMqaS1zKm8sdSplLWYqciksRmYocypp
K2MqbyldfXJldHVybiBhLmludmVydD1mdW5jdGlvbih0LG4pe3ZhciBhPUFmKG4pLHU9QWYodCkq
YSxjPVBmKHQpKmEsZj1QZihuKSxzPWYqaS1jKm87cmV0dXJuW1RmKGMqaStmKm8sdSplK3Mqciks
RmYocyplLXUqcildfSxhfWZ1bmN0aW9uIGhsKHQpe2Z1bmN0aW9uIG4obil7cmV0dXJuKG49dChu
WzBdKnhmLG5bMV0qeGYpKVswXSo9bWYsblsxXSo9bWYsbn1yZXR1cm4gdD1jbCh0WzBdKnhmLHRb
MV0qeGYsdC5sZW5ndGg+Mj90WzJdKnhmOjApLG4uaW52ZXJ0PWZ1bmN0aW9uKG4pe3JldHVybihu
PXQuaW52ZXJ0KG5bMF0qeGYsblsxXSp4ZikpWzBdKj1tZixuWzFdKj1tZixufSxufWZ1bmN0aW9u
IGRsKHQsbixlLHIsaSxvKXtpZihlKXt2YXIgYT1BZihuKSx1PVBmKG4pLGM9ciplO251bGw9PWk/
KGk9bityKmJmLG89bi1jLzIpOihpPXBsKGEsaSksbz1wbChhLG8pLChyPjA/aTxvOmk+bykmJihp
Kz1yKmJmKSk7Zm9yKHZhciBmLHM9aTtyPjA/cz5vOnM8bztzLT1jKWY9cHMoW2EsLXUqQWYocyks
LXUqUGYocyldKSx0LnBvaW50KGZbMF0sZlsxXSl9fWZ1bmN0aW9uIHBsKHQsbil7KG49Z3Mobikp
WzBdLT10LG1zKG4pO3ZhciBlPVJmKC1uWzFdKTtyZXR1cm4oKC1uWzJdPDA/LWU6ZSkrYmYtcGYp
JWJmfWZ1bmN0aW9uIGdsKCl7dmFyIHQsbj1bXTtyZXR1cm57cG9pbnQ6ZnVuY3Rpb24obixlLHIp
e3QucHVzaChbbixlLHJdKX0sbGluZVN0YXJ0OmZ1bmN0aW9uKCl7bi5wdXNoKHQ9W10pfSxsaW5l
RW5kOlVmLHJlam9pbjpmdW5jdGlvbigpe24ubGVuZ3RoPjEmJm4ucHVzaChuLnBvcCgpLmNvbmNh
dChuLnNoaWZ0KCkpKX0scmVzdWx0OmZ1bmN0aW9uKCl7dmFyIGU9bjtyZXR1cm4gbj1bXSx0PW51
bGwsZX19fWZ1bmN0aW9uIHlsKHQsbil7cmV0dXJuIHdmKHRbMF0tblswXSk8cGYmJndmKHRbMV0t
blsxXSk8cGZ9ZnVuY3Rpb24gdmwodCxuLGUscil7dGhpcy54PXQsdGhpcy56PW4sdGhpcy5vPWUs
dGhpcy5lPXIsdGhpcy52PSExLHRoaXMubj10aGlzLnA9bnVsbH1mdW5jdGlvbiBfbCh0LG4sZSxy
LGkpe3ZhciBvLGEsdT1bXSxjPVtdO2lmKHQuZm9yRWFjaCgoZnVuY3Rpb24odCl7aWYoISgobj10
Lmxlbmd0aC0xKTw9MCkpe3ZhciBuLGUscj10WzBdLGE9dFtuXTtpZih5bChyLGEpKXtpZighclsy
XSYmIWFbMl0pe2ZvcihpLmxpbmVTdGFydCgpLG89MDtvPG47KytvKWkucG9pbnQoKHI9dFtvXSlb
MF0sclsxXSk7cmV0dXJuIHZvaWQgaS5saW5lRW5kKCl9YVswXSs9MipwZn11LnB1c2goZT1uZXcg
dmwocix0LG51bGwsITApKSxjLnB1c2goZS5vPW5ldyB2bChyLG51bGwsZSwhMSkpLHUucHVzaChl
PW5ldyB2bChhLHQsbnVsbCwhMSkpLGMucHVzaChlLm89bmV3IHZsKGEsbnVsbCxlLCEwKSl9fSkp
LHUubGVuZ3RoKXtmb3IoYy5zb3J0KG4pLGJsKHUpLGJsKGMpLG89MCxhPWMubGVuZ3RoO288YTsr
K28pY1tvXS5lPWU9IWU7Zm9yKHZhciBmLHMsbD11WzBdOzspe2Zvcih2YXIgaD1sLGQ9ITA7aC52
OylpZigoaD1oLm4pPT09bClyZXR1cm47Zj1oLnosaS5saW5lU3RhcnQoKTtkb3tpZihoLnY9aC5v
LnY9ITAsaC5lKXtpZihkKWZvcihvPTAsYT1mLmxlbmd0aDtvPGE7KytvKWkucG9pbnQoKHM9Zltv
XSlbMF0sc1sxXSk7ZWxzZSByKGgueCxoLm4ueCwxLGkpO2g9aC5ufWVsc2V7aWYoZClmb3IoZj1o
LnAueixvPWYubGVuZ3RoLTE7bz49MDstLW8paS5wb2ludCgocz1mW29dKVswXSxzWzFdKTtlbHNl
IHIoaC54LGgucC54LC0xLGkpO2g9aC5wfWY9KGg9aC5vKS56LGQ9IWR9d2hpbGUoIWgudik7aS5s
aW5lRW5kKCl9fX1mdW5jdGlvbiBibCh0KXtpZihuPXQubGVuZ3RoKXtmb3IodmFyIG4sZSxyPTAs
aT10WzBdOysrcjxuOylpLm49ZT10W3JdLGUucD1pLGk9ZTtpLm49ZT10WzBdLGUucD1pfX1mdW5j
dGlvbiBtbCh0KXtyZXR1cm4gd2YodFswXSk8PXlmP3RbMF06emYodFswXSkqKCh3Zih0WzBdKSt5
ZiklYmYteWYpfWZ1bmN0aW9uIHhsKHQsbil7dmFyIGU9bWwobikscj1uWzFdLGk9UGYociksbz1b
UGYoZSksLUFmKGUpLDBdLGE9MCx1PTAsYz1uZXcgQTsxPT09aT9yPXZmK3BmOi0xPT09aSYmKHI9
LXZmLXBmKTtmb3IodmFyIGY9MCxzPXQubGVuZ3RoO2Y8czsrK2YpaWYoaD0obD10W2ZdKS5sZW5n
dGgpZm9yKHZhciBsLGgsZD1sW2gtMV0scD1tbChkKSxnPWRbMV0vMitfZix5PVBmKGcpLHY9QWYo
ZyksXz0wO188aDsrK18scD1tLHk9dyx2PU0sZD1iKXt2YXIgYj1sW19dLG09bWwoYikseD1iWzFd
LzIrX2Ysdz1QZih4KSxNPUFmKHgpLFQ9bS1wLFM9VD49MD8xOi0xLEU9UypULE49RT55ZixrPXkq
dztpZihjLmFkZChUZihrKlMqUGYoRSksdipNK2sqQWYoRSkpKSxhKz1OP1QrUypiZjpULE5ecD49
ZV5tPj1lKXt2YXIgQz12cyhncyhkKSxncyhiKSk7bXMoQyk7dmFyIFA9dnMobyxDKTttcyhQKTt2
YXIgej0oTl5UPj0wPy0xOjEpKkZmKFBbMl0pOyhyPnp8fHI9PT16JiYoQ1swXXx8Q1sxXSkpJiYo
dSs9Tl5UPj0wPzE6LTEpfX1yZXR1cm4oYTwtcGZ8fGE8cGYmJmM8LWdmKV4xJnV9ZnVuY3Rpb24g
d2wodCxuLGUscil7cmV0dXJuIGZ1bmN0aW9uKGkpe3ZhciBvLGEsdSxjPW4oaSksZj1nbCgpLHM9
bihmKSxsPSExLGg9e3BvaW50OmQsbGluZVN0YXJ0OmcsbGluZUVuZDp5LHBvbHlnb25TdGFydDpm
dW5jdGlvbigpe2gucG9pbnQ9dixoLmxpbmVTdGFydD1fLGgubGluZUVuZD1iLGE9W10sbz1bXX0s
cG9seWdvbkVuZDpmdW5jdGlvbigpe2gucG9pbnQ9ZCxoLmxpbmVTdGFydD1nLGgubGluZUVuZD15
LGE9c3QoYSk7dmFyIHQ9eGwobyxyKTthLmxlbmd0aD8obHx8KGkucG9seWdvblN0YXJ0KCksbD0h
MCksX2woYSxUbCx0LGUsaSkpOnQmJihsfHwoaS5wb2x5Z29uU3RhcnQoKSxsPSEwKSxpLmxpbmVT
dGFydCgpLGUobnVsbCxudWxsLDEsaSksaS5saW5lRW5kKCkpLGwmJihpLnBvbHlnb25FbmQoKSxs
PSExKSxhPW89bnVsbH0sc3BoZXJlOmZ1bmN0aW9uKCl7aS5wb2x5Z29uU3RhcnQoKSxpLmxpbmVT
dGFydCgpLGUobnVsbCxudWxsLDEsaSksaS5saW5lRW5kKCksaS5wb2x5Z29uRW5kKCl9fTtmdW5j
dGlvbiBkKG4sZSl7dChuLGUpJiZpLnBvaW50KG4sZSl9ZnVuY3Rpb24gcCh0LG4pe2MucG9pbnQo
dCxuKX1mdW5jdGlvbiBnKCl7aC5wb2ludD1wLGMubGluZVN0YXJ0KCl9ZnVuY3Rpb24geSgpe2gu
cG9pbnQ9ZCxjLmxpbmVFbmQoKX1mdW5jdGlvbiB2KHQsbil7dS5wdXNoKFt0LG5dKSxzLnBvaW50
KHQsbil9ZnVuY3Rpb24gXygpe3MubGluZVN0YXJ0KCksdT1bXX1mdW5jdGlvbiBiKCl7dih1WzBd
WzBdLHVbMF1bMV0pLHMubGluZUVuZCgpO3ZhciB0LG4sZSxyLGM9cy5jbGVhbigpLGg9Zi5yZXN1
bHQoKSxkPWgubGVuZ3RoO2lmKHUucG9wKCksby5wdXNoKHUpLHU9bnVsbCxkKWlmKDEmYyl7aWYo
KG49KGU9aFswXSkubGVuZ3RoLTEpPjApe2ZvcihsfHwoaS5wb2x5Z29uU3RhcnQoKSxsPSEwKSxp
LmxpbmVTdGFydCgpLHQ9MDt0PG47Kyt0KWkucG9pbnQoKHI9ZVt0XSlbMF0sclsxXSk7aS5saW5l
RW5kKCl9fWVsc2UgZD4xJiYyJmMmJmgucHVzaChoLnBvcCgpLmNvbmNhdChoLnNoaWZ0KCkpKSxh
LnB1c2goaC5maWx0ZXIoTWwpKX1yZXR1cm4gaH19ZnVuY3Rpb24gTWwodCl7cmV0dXJuIHQubGVu
Z3RoPjF9ZnVuY3Rpb24gVGwodCxuKXtyZXR1cm4oKHQ9dC54KVswXTwwP3RbMV0tdmYtcGY6dmYt
dFsxXSktKChuPW4ueClbMF08MD9uWzFdLXZmLXBmOnZmLW5bMV0pfXVsLmludmVydD11bDt2YXIg
QWw9d2woKGZ1bmN0aW9uKCl7cmV0dXJuITB9KSwoZnVuY3Rpb24odCl7dmFyIG4sZT1OYU4scj1O
YU4saT1OYU47cmV0dXJue2xpbmVTdGFydDpmdW5jdGlvbigpe3QubGluZVN0YXJ0KCksbj0xfSxw
b2ludDpmdW5jdGlvbihvLGEpe3ZhciB1PW8+MD95ZjoteWYsYz13ZihvLWUpO3dmKGMteWYpPHBm
Pyh0LnBvaW50KGUscj0ocithKS8yPjA/dmY6LXZmKSx0LnBvaW50KGksciksdC5saW5lRW5kKCks
dC5saW5lU3RhcnQoKSx0LnBvaW50KHUsciksdC5wb2ludChvLHIpLG49MCk6aSE9PXUmJmM+PXlm
JiYod2YoZS1pKTxwZiYmKGUtPWkqcGYpLHdmKG8tdSk8cGYmJihvLT11KnBmKSxyPWZ1bmN0aW9u
KHQsbixlLHIpe3ZhciBpLG8sYT1QZih0LWUpO3JldHVybiB3ZihhKT5wZj9NZigoUGYobikqKG89
QWYocikpKlBmKGUpLVBmKHIpKihpPUFmKG4pKSpQZih0KSkvKGkqbyphKSk6KG4rcikvMn0oZSxy
LG8sYSksdC5wb2ludChpLHIpLHQubGluZUVuZCgpLHQubGluZVN0YXJ0KCksdC5wb2ludCh1LHIp
LG49MCksdC5wb2ludChlPW8scj1hKSxpPXV9LGxpbmVFbmQ6ZnVuY3Rpb24oKXt0LmxpbmVFbmQo
KSxlPXI9TmFOfSxjbGVhbjpmdW5jdGlvbigpe3JldHVybiAyLW59fX0pLChmdW5jdGlvbih0LG4s
ZSxyKXt2YXIgaTtpZihudWxsPT10KWk9ZSp2ZixyLnBvaW50KC15ZixpKSxyLnBvaW50KDAsaSks
ci5wb2ludCh5ZixpKSxyLnBvaW50KHlmLDApLHIucG9pbnQoeWYsLWkpLHIucG9pbnQoMCwtaSks
ci5wb2ludCgteWYsLWkpLHIucG9pbnQoLXlmLDApLHIucG9pbnQoLXlmLGkpO2Vsc2UgaWYod2Yo
dFswXS1uWzBdKT5wZil7dmFyIG89dFswXTxuWzBdP3lmOi15ZjtpPWUqby8yLHIucG9pbnQoLW8s
aSksci5wb2ludCgwLGkpLHIucG9pbnQobyxpKX1lbHNlIHIucG9pbnQoblswXSxuWzFdKX0pLFst
eWYsLXZmXSk7ZnVuY3Rpb24gU2wodCl7dmFyIG49QWYodCksZT02KnhmLHI9bj4wLGk9d2Yobik+
cGY7ZnVuY3Rpb24gbyh0LGUpe3JldHVybiBBZih0KSpBZihlKT5ufWZ1bmN0aW9uIGEodCxlLHIp
e3ZhciBpPVsxLDAsMF0sbz12cyhncyh0KSxncyhlKSksYT15cyhvLG8pLHU9b1swXSxjPWEtdSp1
O2lmKCFjKXJldHVybiFyJiZ0O3ZhciBmPW4qYS9jLHM9LW4qdS9jLGw9dnMoaSxvKSxoPWJzKGks
Zik7X3MoaCxicyhvLHMpKTt2YXIgZD1sLHA9eXMoaCxkKSxnPXlzKGQsZCkseT1wKnAtZyooeXMo
aCxoKS0xKTtpZighKHk8MCkpe3ZhciB2PSRmKHkpLF89YnMoZCwoLXAtdikvZyk7aWYoX3MoXyxo
KSxfPXBzKF8pLCFyKXJldHVybiBfO3ZhciBiLG09dFswXSx4PWVbMF0sdz10WzFdLE09ZVsxXTt4
PG0mJihiPW0sbT14LHg9Yik7dmFyIFQ9eC1tLEE9d2YoVC15Zik8cGY7aWYoIUEmJk08dyYmKGI9
dyx3PU0sTT1iKSxBfHxUPHBmP0E/dytNPjBeX1sxXTwod2YoX1swXS1tKTxwZj93Ok0pOnc8PV9b
MV0mJl9bMV08PU06VD55Zl4obTw9X1swXSYmX1swXTw9eCkpe3ZhciBTPWJzKGQsKC1wK3YpL2cp
O3JldHVybiBfcyhTLGgpLFtfLHBzKFMpXX19fWZ1bmN0aW9uIHUobixlKXt2YXIgaT1yP3Q6eWYt
dCxvPTA7cmV0dXJuIG48LWk/b3w9MTpuPmkmJihvfD0yKSxlPC1pP298PTQ6ZT5pJiYob3w9OCks
b31yZXR1cm4gd2wobywoZnVuY3Rpb24odCl7dmFyIG4sZSxjLGYscztyZXR1cm57bGluZVN0YXJ0
OmZ1bmN0aW9uKCl7Zj1jPSExLHM9MX0scG9pbnQ6ZnVuY3Rpb24obCxoKXt2YXIgZCxwPVtsLGhd
LGc9byhsLGgpLHk9cj9nPzA6dShsLGgpOmc/dShsKyhsPDA/eWY6LXlmKSxoKTowO2lmKCFuJiYo
Zj1jPWcpJiZ0LmxpbmVTdGFydCgpLGchPT1jJiYoIShkPWEobixwKSl8fHlsKG4sZCl8fHlsKHAs
ZCkpJiYocFsyXT0xKSxnIT09YylzPTAsZz8odC5saW5lU3RhcnQoKSxkPWEocCxuKSx0LnBvaW50
KGRbMF0sZFsxXSkpOihkPWEobixwKSx0LnBvaW50KGRbMF0sZFsxXSwyKSx0LmxpbmVFbmQoKSks
bj1kO2Vsc2UgaWYoaSYmbiYmcl5nKXt2YXIgdjt5JmV8fCEodj1hKHAsbiwhMCkpfHwocz0wLHI/
KHQubGluZVN0YXJ0KCksdC5wb2ludCh2WzBdWzBdLHZbMF1bMV0pLHQucG9pbnQodlsxXVswXSx2
WzFdWzFdKSx0LmxpbmVFbmQoKSk6KHQucG9pbnQodlsxXVswXSx2WzFdWzFdKSx0LmxpbmVFbmQo
KSx0LmxpbmVTdGFydCgpLHQucG9pbnQodlswXVswXSx2WzBdWzFdLDMpKSl9IWd8fG4mJnlsKG4s
cCl8fHQucG9pbnQocFswXSxwWzFdKSxuPXAsYz1nLGU9eX0sbGluZUVuZDpmdW5jdGlvbigpe2Mm
JnQubGluZUVuZCgpLG49bnVsbH0sY2xlYW46ZnVuY3Rpb24oKXtyZXR1cm4gc3woZiYmYyk8PDF9
fX0pLChmdW5jdGlvbihuLHIsaSxvKXtkbChvLHQsZSxpLG4scil9KSxyP1swLC10XTpbLXlmLHQt
eWZdKX12YXIgRWwsTmwsa2wsQ2wsUGw9MWU5LHpsPS1QbDtmdW5jdGlvbiAkbCh0LG4sZSxyKXtm
dW5jdGlvbiBpKGksbyl7cmV0dXJuIHQ8PWkmJmk8PWUmJm48PW8mJm88PXJ9ZnVuY3Rpb24gbyhp
LG8sdSxmKXt2YXIgcz0wLGw9MDtpZihudWxsPT1pfHwocz1hKGksdSkpIT09KGw9YShvLHUpKXx8
YyhpLG8pPDBedT4wKWRve2YucG9pbnQoMD09PXN8fDM9PT1zP3Q6ZSxzPjE/cjpuKX13aGlsZSgo
cz0ocyt1KzQpJTQpIT09bCk7ZWxzZSBmLnBvaW50KG9bMF0sb1sxXSl9ZnVuY3Rpb24gYShyLGkp
e3JldHVybiB3ZihyWzBdLXQpPHBmP2k+MD8wOjM6d2YoclswXS1lKTxwZj9pPjA/MjoxOndmKHJb
MV0tbik8cGY/aT4wPzE6MDppPjA/MzoyfWZ1bmN0aW9uIHUodCxuKXtyZXR1cm4gYyh0Lngsbi54
KX1mdW5jdGlvbiBjKHQsbil7dmFyIGU9YSh0LDEpLHI9YShuLDEpO3JldHVybiBlIT09cj9lLXI6
MD09PWU/blsxXS10WzFdOjE9PT1lP3RbMF0tblswXToyPT09ZT90WzFdLW5bMV06blswXS10WzBd
fXJldHVybiBmdW5jdGlvbihhKXt2YXIgYyxmLHMsbCxoLGQscCxnLHksdixfLGI9YSxtPWdsKCks
eD17cG9pbnQ6dyxsaW5lU3RhcnQ6ZnVuY3Rpb24oKXt4LnBvaW50PU0sZiYmZi5wdXNoKHM9W10p
O3Y9ITAseT0hMSxwPWc9TmFOfSxsaW5lRW5kOmZ1bmN0aW9uKCl7YyYmKE0obCxoKSxkJiZ5JiZt
LnJlam9pbigpLGMucHVzaChtLnJlc3VsdCgpKSk7eC5wb2ludD13LHkmJmIubGluZUVuZCgpfSxw
b2x5Z29uU3RhcnQ6ZnVuY3Rpb24oKXtiPW0sYz1bXSxmPVtdLF89ITB9LHBvbHlnb25FbmQ6ZnVu
Y3Rpb24oKXt2YXIgbj1mdW5jdGlvbigpe2Zvcih2YXIgbj0wLGU9MCxpPWYubGVuZ3RoO2U8aTsr
K2UpZm9yKHZhciBvLGEsdT1mW2VdLGM9MSxzPXUubGVuZ3RoLGw9dVswXSxoPWxbMF0sZD1sWzFd
O2M8czsrK2Mpbz1oLGE9ZCxoPShsPXVbY10pWzBdLGQ9bFsxXSxhPD1yP2Q+ciYmKGgtbykqKHIt
YSk+KGQtYSkqKHQtbykmJisrbjpkPD1yJiYoaC1vKSooci1hKTwoZC1hKSoodC1vKSYmLS1uO3Jl
dHVybiBufSgpLGU9XyYmbixpPShjPXN0KGMpKS5sZW5ndGg7KGV8fGkpJiYoYS5wb2x5Z29uU3Rh
cnQoKSxlJiYoYS5saW5lU3RhcnQoKSxvKG51bGwsbnVsbCwxLGEpLGEubGluZUVuZCgpKSxpJiZf
bChjLHUsbixvLGEpLGEucG9seWdvbkVuZCgpKTtiPWEsYz1mPXM9bnVsbH19O2Z1bmN0aW9uIHco
dCxuKXtpKHQsbikmJmIucG9pbnQodCxuKX1mdW5jdGlvbiBNKG8sYSl7dmFyIHU9aShvLGEpO2lm
KGYmJnMucHVzaChbbyxhXSksdilsPW8saD1hLGQ9dSx2PSExLHUmJihiLmxpbmVTdGFydCgpLGIu
cG9pbnQobyxhKSk7ZWxzZSBpZih1JiZ5KWIucG9pbnQobyxhKTtlbHNle3ZhciBjPVtwPU1hdGgu
bWF4KHpsLE1hdGgubWluKFBsLHApKSxnPU1hdGgubWF4KHpsLE1hdGgubWluKFBsLGcpKV0sbT1b
bz1NYXRoLm1heCh6bCxNYXRoLm1pbihQbCxvKSksYT1NYXRoLm1heCh6bCxNYXRoLm1pbihQbCxh
KSldOyFmdW5jdGlvbih0LG4sZSxyLGksbyl7dmFyIGEsdT10WzBdLGM9dFsxXSxmPTAscz0xLGw9
blswXS11LGg9blsxXS1jO2lmKGE9ZS11LGx8fCEoYT4wKSl7aWYoYS89bCxsPDApe2lmKGE8Zily
ZXR1cm47YTxzJiYocz1hKX1lbHNlIGlmKGw+MCl7aWYoYT5zKXJldHVybjthPmYmJihmPWEpfWlm
KGE9aS11LGx8fCEoYTwwKSl7aWYoYS89bCxsPDApe2lmKGE+cylyZXR1cm47YT5mJiYoZj1hKX1l
bHNlIGlmKGw+MCl7aWYoYTxmKXJldHVybjthPHMmJihzPWEpfWlmKGE9ci1jLGh8fCEoYT4wKSl7
aWYoYS89aCxoPDApe2lmKGE8ZilyZXR1cm47YTxzJiYocz1hKX1lbHNlIGlmKGg+MCl7aWYoYT5z
KXJldHVybjthPmYmJihmPWEpfWlmKGE9by1jLGh8fCEoYTwwKSl7aWYoYS89aCxoPDApe2lmKGE+
cylyZXR1cm47YT5mJiYoZj1hKX1lbHNlIGlmKGg+MCl7aWYoYTxmKXJldHVybjthPHMmJihzPWEp
fXJldHVybiBmPjAmJih0WzBdPXUrZipsLHRbMV09YytmKmgpLHM8MSYmKG5bMF09dStzKmwsblsx
XT1jK3MqaCksITB9fX19fShjLG0sdCxuLGUscik/dSYmKGIubGluZVN0YXJ0KCksYi5wb2ludChv
LGEpLF89ITEpOih5fHwoYi5saW5lU3RhcnQoKSxiLnBvaW50KGNbMF0sY1sxXSkpLGIucG9pbnQo
bVswXSxtWzFdKSx1fHxiLmxpbmVFbmQoKSxfPSExKX1wPW8sZz1hLHk9dX1yZXR1cm4geH19dmFy
IERsPXtzcGhlcmU6VWYscG9pbnQ6VWYsbGluZVN0YXJ0OmZ1bmN0aW9uKCl7RGwucG9pbnQ9Rmws
RGwubGluZUVuZD1SbH0sbGluZUVuZDpVZixwb2x5Z29uU3RhcnQ6VWYscG9seWdvbkVuZDpVZn07
ZnVuY3Rpb24gUmwoKXtEbC5wb2ludD1EbC5saW5lRW5kPVVmfWZ1bmN0aW9uIEZsKHQsbil7Tmw9
dCo9eGYsa2w9UGYobio9eGYpLENsPUFmKG4pLERsLnBvaW50PXFsfWZ1bmN0aW9uIHFsKHQsbil7
dCo9eGY7dmFyIGU9UGYobio9eGYpLHI9QWYobiksaT13Zih0LU5sKSxvPUFmKGkpLGE9cipQZihp
KSx1PUNsKmUta2wqcipvLGM9a2wqZStDbCpyKm87RWwuYWRkKFRmKCRmKGEqYSt1KnUpLGMpKSxO
bD10LGtsPWUsQ2w9cn1mdW5jdGlvbiBVbCh0KXtyZXR1cm4gRWw9bmV3IEEsamYodCxEbCksK0Vs
fXZhciBJbD1bbnVsbCxudWxsXSxPbD17dHlwZToiTGluZVN0cmluZyIsY29vcmRpbmF0ZXM6SWx9
O2Z1bmN0aW9uIEJsKHQsbil7cmV0dXJuIElsWzBdPXQsSWxbMV09bixVbChPbCl9dmFyIFlsPXtG
ZWF0dXJlOmZ1bmN0aW9uKHQsbil7cmV0dXJuIGpsKHQuZ2VvbWV0cnksbil9LEZlYXR1cmVDb2xs
ZWN0aW9uOmZ1bmN0aW9uKHQsbil7Zm9yKHZhciBlPXQuZmVhdHVyZXMscj0tMSxpPWUubGVuZ3Ro
OysrcjxpOylpZihqbChlW3JdLmdlb21ldHJ5LG4pKXJldHVybiEwO3JldHVybiExfX0sTGw9e1Nw
aGVyZTpmdW5jdGlvbigpe3JldHVybiEwfSxQb2ludDpmdW5jdGlvbih0LG4pe3JldHVybiBIbCh0
LmNvb3JkaW5hdGVzLG4pfSxNdWx0aVBvaW50OmZ1bmN0aW9uKHQsbil7Zm9yKHZhciBlPXQuY29v
cmRpbmF0ZXMscj0tMSxpPWUubGVuZ3RoOysrcjxpOylpZihIbChlW3JdLG4pKXJldHVybiEwO3Jl
dHVybiExfSxMaW5lU3RyaW5nOmZ1bmN0aW9uKHQsbil7cmV0dXJuIFhsKHQuY29vcmRpbmF0ZXMs
bil9LE11bHRpTGluZVN0cmluZzpmdW5jdGlvbih0LG4pe2Zvcih2YXIgZT10LmNvb3JkaW5hdGVz
LHI9LTEsaT1lLmxlbmd0aDsrK3I8aTspaWYoWGwoZVtyXSxuKSlyZXR1cm4hMDtyZXR1cm4hMX0s
UG9seWdvbjpmdW5jdGlvbih0LG4pe3JldHVybiBHbCh0LmNvb3JkaW5hdGVzLG4pfSxNdWx0aVBv
bHlnb246ZnVuY3Rpb24odCxuKXtmb3IodmFyIGU9dC5jb29yZGluYXRlcyxyPS0xLGk9ZS5sZW5n
dGg7KytyPGk7KWlmKEdsKGVbcl0sbikpcmV0dXJuITA7cmV0dXJuITF9LEdlb21ldHJ5Q29sbGVj
dGlvbjpmdW5jdGlvbih0LG4pe2Zvcih2YXIgZT10Lmdlb21ldHJpZXMscj0tMSxpPWUubGVuZ3Ro
OysrcjxpOylpZihqbChlW3JdLG4pKXJldHVybiEwO3JldHVybiExfX07ZnVuY3Rpb24gamwodCxu
KXtyZXR1cm4hKCF0fHwhTGwuaGFzT3duUHJvcGVydHkodC50eXBlKSkmJkxsW3QudHlwZV0odCxu
KX1mdW5jdGlvbiBIbCh0LG4pe3JldHVybiAwPT09QmwodCxuKX1mdW5jdGlvbiBYbCh0LG4pe2Zv
cih2YXIgZSxyLGksbz0wLGE9dC5sZW5ndGg7bzxhO28rKyl7aWYoMD09PShyPUJsKHRbb10sbikp
KXJldHVybiEwO2lmKG8+MCYmKGk9QmwodFtvXSx0W28tMV0pKT4wJiZlPD1pJiZyPD1pJiYoZSty
LWkpKigxLU1hdGgucG93KChlLXIpL2ksMikpPGdmKmkpcmV0dXJuITA7ZT1yfXJldHVybiExfWZ1
bmN0aW9uIEdsKHQsbil7cmV0dXJuISF4bCh0Lm1hcChWbCksV2wobikpfWZ1bmN0aW9uIFZsKHQp
e3JldHVybih0PXQubWFwKFdsKSkucG9wKCksdH1mdW5jdGlvbiBXbCh0KXtyZXR1cm5bdFswXSp4
Zix0WzFdKnhmXX1mdW5jdGlvbiBabCh0LG4sZSl7dmFyIHI9aHQodCxuLXBmLGUpLmNvbmNhdChu
KTtyZXR1cm4gZnVuY3Rpb24odCl7cmV0dXJuIHIubWFwKChmdW5jdGlvbihuKXtyZXR1cm5bdCxu
XX0pKX19ZnVuY3Rpb24gS2wodCxuLGUpe3ZhciByPWh0KHQsbi1wZixlKS5jb25jYXQobik7cmV0
dXJuIGZ1bmN0aW9uKHQpe3JldHVybiByLm1hcCgoZnVuY3Rpb24obil7cmV0dXJuW24sdF19KSl9
fWZ1bmN0aW9uIFFsKCl7dmFyIHQsbixlLHIsaSxvLGEsdSxjLGYscyxsLGg9MTAsZD1oLHA9OTAs
Zz0zNjAseT0yLjU7ZnVuY3Rpb24gdigpe3JldHVybnt0eXBlOiJNdWx0aUxpbmVTdHJpbmciLGNv
b3JkaW5hdGVzOl8oKX19ZnVuY3Rpb24gXygpe3JldHVybiBodChTZihyL3ApKnAsZSxwKS5tYXAo
cykuY29uY2F0KGh0KFNmKHUvZykqZyxhLGcpLm1hcChsKSkuY29uY2F0KGh0KFNmKG4vaCkqaCx0
LGgpLmZpbHRlcigoZnVuY3Rpb24odCl7cmV0dXJuIHdmKHQlcCk+cGZ9KSkubWFwKGMpKS5jb25j
YXQoaHQoU2Yoby9kKSpkLGksZCkuZmlsdGVyKChmdW5jdGlvbih0KXtyZXR1cm4gd2YodCVnKT5w
Zn0pKS5tYXAoZikpfXJldHVybiB2LmxpbmVzPWZ1bmN0aW9uKCl7cmV0dXJuIF8oKS5tYXAoKGZ1
bmN0aW9uKHQpe3JldHVybnt0eXBlOiJMaW5lU3RyaW5nIixjb29yZGluYXRlczp0fX0pKX0sdi5v
dXRsaW5lPWZ1bmN0aW9uKCl7cmV0dXJue3R5cGU6IlBvbHlnb24iLGNvb3JkaW5hdGVzOltzKHIp
LmNvbmNhdChsKGEpLnNsaWNlKDEpLHMoZSkucmV2ZXJzZSgpLnNsaWNlKDEpLGwodSkucmV2ZXJz
ZSgpLnNsaWNlKDEpKV19fSx2LmV4dGVudD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxl
bmd0aD92LmV4dGVudE1ham9yKHQpLmV4dGVudE1pbm9yKHQpOnYuZXh0ZW50TWlub3IoKX0sdi5l
eHRlbnRNYWpvcj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj0rdFswXVsw
XSxlPSt0WzFdWzBdLHU9K3RbMF1bMV0sYT0rdFsxXVsxXSxyPmUmJih0PXIscj1lLGU9dCksdT5h
JiYodD11LHU9YSxhPXQpLHYucHJlY2lzaW9uKHkpKTpbW3IsdV0sW2UsYV1dfSx2LmV4dGVudE1p
bm9yPWZ1bmN0aW9uKGUpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhuPStlWzBdWzBdLHQ9K2Vb
MV1bMF0sbz0rZVswXVsxXSxpPStlWzFdWzFdLG4+dCYmKGU9bixuPXQsdD1lKSxvPmkmJihlPW8s
bz1pLGk9ZSksdi5wcmVjaXNpb24oeSkpOltbbixvXSxbdCxpXV19LHYuc3RlcD1mdW5jdGlvbih0
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD92LnN0ZXBNYWpvcih0KS5zdGVwTWlub3IodCk6di5z
dGVwTWlub3IoKX0sdi5zdGVwTWFqb3I9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5n
dGg/KHA9K3RbMF0sZz0rdFsxXSx2KTpbcCxnXX0sdi5zdGVwTWlub3I9ZnVuY3Rpb24odCl7cmV0
dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGg9K3RbMF0sZD0rdFsxXSx2KTpbaCxkXX0sdi5wcmVjaXNp
b249ZnVuY3Rpb24oaCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHk9K2gsYz1abChvLGksOTAp
LGY9S2wobix0LHkpLHM9WmwodSxhLDkwKSxsPUtsKHIsZSx5KSx2KTp5fSx2LmV4dGVudE1ham9y
KFtbLTE4MCwtOTArcGZdLFsxODAsOTAtcGZdXSkuZXh0ZW50TWlub3IoW1stMTgwLC04MC1wZl0s
WzE4MCw4MCtwZl1dKX12YXIgSmwsdGgsbmgsZWgscmg9dD0+dCxpaD1uZXcgQSxvaD1uZXcgQSxh
aD17cG9pbnQ6VWYsbGluZVN0YXJ0OlVmLGxpbmVFbmQ6VWYscG9seWdvblN0YXJ0OmZ1bmN0aW9u
KCl7YWgubGluZVN0YXJ0PXVoLGFoLmxpbmVFbmQ9c2h9LHBvbHlnb25FbmQ6ZnVuY3Rpb24oKXth
aC5saW5lU3RhcnQ9YWgubGluZUVuZD1haC5wb2ludD1VZixpaC5hZGQod2Yob2gpKSxvaD1uZXcg
QX0scmVzdWx0OmZ1bmN0aW9uKCl7dmFyIHQ9aWgvMjtyZXR1cm4gaWg9bmV3IEEsdH19O2Z1bmN0
aW9uIHVoKCl7YWgucG9pbnQ9Y2h9ZnVuY3Rpb24gY2godCxuKXthaC5wb2ludD1maCxKbD1uaD10
LHRoPWVoPW59ZnVuY3Rpb24gZmgodCxuKXtvaC5hZGQoZWgqdC1uaCpuKSxuaD10LGVoPW59ZnVu
Y3Rpb24gc2goKXtmaChKbCx0aCl9dmFyIGxoPWFoLGhoPTEvMCxkaD1oaCxwaD0taGgsZ2g9cGgs
eWg9e3BvaW50OmZ1bmN0aW9uKHQsbil7dDxoaCYmKGhoPXQpO3Q+cGgmJihwaD10KTtuPGRoJiYo
ZGg9bik7bj5naCYmKGdoPW4pfSxsaW5lU3RhcnQ6VWYsbGluZUVuZDpVZixwb2x5Z29uU3RhcnQ6
VWYscG9seWdvbkVuZDpVZixyZXN1bHQ6ZnVuY3Rpb24oKXt2YXIgdD1bW2hoLGRoXSxbcGgsZ2hd
XTtyZXR1cm4gcGg9Z2g9LShkaD1oaD0xLzApLHR9fTt2YXIgdmgsX2gsYmgsbWgseGg9eWgsd2g9
MCxNaD0wLFRoPTAsQWg9MCxTaD0wLEVoPTAsTmg9MCxraD0wLENoPTAsUGg9e3BvaW50OnpoLGxp
bmVTdGFydDokaCxsaW5lRW5kOkZoLHBvbHlnb25TdGFydDpmdW5jdGlvbigpe1BoLmxpbmVTdGFy
dD1xaCxQaC5saW5lRW5kPVVofSxwb2x5Z29uRW5kOmZ1bmN0aW9uKCl7UGgucG9pbnQ9emgsUGgu
bGluZVN0YXJ0PSRoLFBoLmxpbmVFbmQ9Rmh9LHJlc3VsdDpmdW5jdGlvbigpe3ZhciB0PUNoP1tO
aC9DaCxraC9DaF06RWg/W0FoL0VoLFNoL0VoXTpUaD9bd2gvVGgsTWgvVGhdOltOYU4sTmFOXTty
ZXR1cm4gd2g9TWg9VGg9QWg9U2g9RWg9Tmg9a2g9Q2g9MCx0fX07ZnVuY3Rpb24gemgodCxuKXt3
aCs9dCxNaCs9biwrK1RofWZ1bmN0aW9uICRoKCl7UGgucG9pbnQ9RGh9ZnVuY3Rpb24gRGgodCxu
KXtQaC5wb2ludD1SaCx6aChiaD10LG1oPW4pfWZ1bmN0aW9uIFJoKHQsbil7dmFyIGU9dC1iaCxy
PW4tbWgsaT0kZihlKmUrcipyKTtBaCs9aSooYmgrdCkvMixTaCs9aSoobWgrbikvMixFaCs9aSx6
aChiaD10LG1oPW4pfWZ1bmN0aW9uIEZoKCl7UGgucG9pbnQ9emh9ZnVuY3Rpb24gcWgoKXtQaC5w
b2ludD1JaH1mdW5jdGlvbiBVaCgpe09oKHZoLF9oKX1mdW5jdGlvbiBJaCh0LG4pe1BoLnBvaW50
PU9oLHpoKHZoPWJoPXQsX2g9bWg9bil9ZnVuY3Rpb24gT2godCxuKXt2YXIgZT10LWJoLHI9bi1t
aCxpPSRmKGUqZStyKnIpO0FoKz1pKihiaCt0KS8yLFNoKz1pKihtaCtuKS8yLEVoKz1pLE5oKz0o
aT1taCp0LWJoKm4pKihiaCt0KSxraCs9aSoobWgrbiksQ2grPTMqaSx6aChiaD10LG1oPW4pfXZh
ciBCaD1QaDtmdW5jdGlvbiBZaCh0KXt0aGlzLl9jb250ZXh0PXR9WWgucHJvdG90eXBlPXtfcmFk
aXVzOjQuNSxwb2ludFJhZGl1czpmdW5jdGlvbih0KXtyZXR1cm4gdGhpcy5fcmFkaXVzPXQsdGhp
c30scG9seWdvblN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5fbGluZT0wfSxwb2x5Z29uRW5kOmZ1bmN0
aW9uKCl7dGhpcy5fbGluZT1OYU59LGxpbmVTdGFydDpmdW5jdGlvbigpe3RoaXMuX3BvaW50PTB9
LGxpbmVFbmQ6ZnVuY3Rpb24oKXswPT09dGhpcy5fbGluZSYmdGhpcy5fY29udGV4dC5jbG9zZVBh
dGgoKSx0aGlzLl9wb2ludD1OYU59LHBvaW50OmZ1bmN0aW9uKHQsbil7c3dpdGNoKHRoaXMuX3Bv
aW50KXtjYXNlIDA6dGhpcy5fY29udGV4dC5tb3ZlVG8odCxuKSx0aGlzLl9wb2ludD0xO2JyZWFr
O2Nhc2UgMTp0aGlzLl9jb250ZXh0LmxpbmVUbyh0LG4pO2JyZWFrO2RlZmF1bHQ6dGhpcy5fY29u
dGV4dC5tb3ZlVG8odCt0aGlzLl9yYWRpdXMsbiksdGhpcy5fY29udGV4dC5hcmModCxuLHRoaXMu
X3JhZGl1cywwLGJmKX19LHJlc3VsdDpVZn07dmFyIExoLGpoLEhoLFhoLEdoLFZoPW5ldyBBLFdo
PXtwb2ludDpVZixsaW5lU3RhcnQ6ZnVuY3Rpb24oKXtXaC5wb2ludD1aaH0sbGluZUVuZDpmdW5j
dGlvbigpe0xoJiZLaChqaCxIaCksV2gucG9pbnQ9VWZ9LHBvbHlnb25TdGFydDpmdW5jdGlvbigp
e0xoPSEwfSxwb2x5Z29uRW5kOmZ1bmN0aW9uKCl7TGg9bnVsbH0scmVzdWx0OmZ1bmN0aW9uKCl7
dmFyIHQ9K1ZoO3JldHVybiBWaD1uZXcgQSx0fX07ZnVuY3Rpb24gWmgodCxuKXtXaC5wb2ludD1L
aCxqaD1YaD10LEhoPUdoPW59ZnVuY3Rpb24gS2godCxuKXtYaC09dCxHaC09bixWaC5hZGQoJGYo
WGgqWGgrR2gqR2gpKSxYaD10LEdoPW59dmFyIFFoPVdoO2xldCBKaCx0ZCxuZCxlZDtjbGFzcyBy
ZHtjb25zdHJ1Y3Rvcih0KXt0aGlzLl9hcHBlbmQ9bnVsbD09dD9pZDpmdW5jdGlvbih0KXtjb25z
dCBuPU1hdGguZmxvb3IodCk7aWYoIShuPj0wKSl0aHJvdyBuZXcgUmFuZ2VFcnJvcihgaW52YWxp
ZCBkaWdpdHM6ICR7dH1gKTtpZihuPjE1KXJldHVybiBpZDtpZihuIT09Smgpe2NvbnN0IHQ9MTAq
Km47Smg9bix0ZD1mdW5jdGlvbihuKXtsZXQgZT0xO3RoaXMuXys9blswXTtmb3IoY29uc3Qgcj1u
Lmxlbmd0aDtlPHI7KytlKXRoaXMuXys9TWF0aC5yb3VuZChhcmd1bWVudHNbZV0qdCkvdCtuW2Vd
fX1yZXR1cm4gdGR9KHQpLHRoaXMuX3JhZGl1cz00LjUsdGhpcy5fPSIifXBvaW50UmFkaXVzKHQp
e3JldHVybiB0aGlzLl9yYWRpdXM9K3QsdGhpc31wb2x5Z29uU3RhcnQoKXt0aGlzLl9saW5lPTB9
cG9seWdvbkVuZCgpe3RoaXMuX2xpbmU9TmFOfWxpbmVTdGFydCgpe3RoaXMuX3BvaW50PTB9bGlu
ZUVuZCgpezA9PT10aGlzLl9saW5lJiYodGhpcy5fKz0iWiIpLHRoaXMuX3BvaW50PU5hTn1wb2lu
dCh0LG4pe3N3aXRjaCh0aGlzLl9wb2ludCl7Y2FzZSAwOnRoaXMuX2FwcGVuZGBNJHt0fSwke259
YCx0aGlzLl9wb2ludD0xO2JyZWFrO2Nhc2UgMTp0aGlzLl9hcHBlbmRgTCR7dH0sJHtufWA7YnJl
YWs7ZGVmYXVsdDppZih0aGlzLl9hcHBlbmRgTSR7dH0sJHtufWAsdGhpcy5fcmFkaXVzIT09bmR8
fHRoaXMuX2FwcGVuZCE9PXRkKXtjb25zdCB0PXRoaXMuX3JhZGl1cyxuPXRoaXMuXzt0aGlzLl89
IiIsdGhpcy5fYXBwZW5kYG0wLCR7dH1hJHt0fSwke3R9IDAgMSwxIDAsJHstMip0fWEke3R9LCR7
dH0gMCAxLDEgMCwkezIqdH16YCxuZD10LHRkPXRoaXMuX2FwcGVuZCxlZD10aGlzLl8sdGhpcy5f
PW59dGhpcy5fKz1lZH19cmVzdWx0KCl7Y29uc3QgdD10aGlzLl87cmV0dXJuIHRoaXMuXz0iIix0
Lmxlbmd0aD90Om51bGx9fWZ1bmN0aW9uIGlkKHQpe2xldCBuPTE7dGhpcy5fKz10WzBdO2Zvcihj
b25zdCBlPXQubGVuZ3RoO248ZTsrK24pdGhpcy5fKz1hcmd1bWVudHNbbl0rdFtuXX1mdW5jdGlv
biBvZCh0KXtyZXR1cm4gZnVuY3Rpb24obil7dmFyIGU9bmV3IGFkO2Zvcih2YXIgciBpbiB0KWVb
cl09dFtyXTtyZXR1cm4gZS5zdHJlYW09bixlfX1mdW5jdGlvbiBhZCgpe31mdW5jdGlvbiB1ZCh0
LG4sZSl7dmFyIHI9dC5jbGlwRXh0ZW50JiZ0LmNsaXBFeHRlbnQoKTtyZXR1cm4gdC5zY2FsZSgx
NTApLnRyYW5zbGF0ZShbMCwwXSksbnVsbCE9ciYmdC5jbGlwRXh0ZW50KG51bGwpLGpmKGUsdC5z
dHJlYW0oeGgpKSxuKHhoLnJlc3VsdCgpKSxudWxsIT1yJiZ0LmNsaXBFeHRlbnQociksdH1mdW5j
dGlvbiBjZCh0LG4sZSl7cmV0dXJuIHVkKHQsKGZ1bmN0aW9uKGUpe3ZhciByPW5bMV1bMF0tblsw
XVswXSxpPW5bMV1bMV0tblswXVsxXSxvPU1hdGgubWluKHIvKGVbMV1bMF0tZVswXVswXSksaS8o
ZVsxXVsxXS1lWzBdWzFdKSksYT0rblswXVswXSsoci1vKihlWzFdWzBdK2VbMF1bMF0pKS8yLHU9
K25bMF1bMV0rKGktbyooZVsxXVsxXStlWzBdWzFdKSkvMjt0LnNjYWxlKDE1MCpvKS50cmFuc2xh
dGUoW2EsdV0pfSksZSl9ZnVuY3Rpb24gZmQodCxuLGUpe3JldHVybiBjZCh0LFtbMCwwXSxuXSxl
KX1mdW5jdGlvbiBzZCh0LG4sZSl7cmV0dXJuIHVkKHQsKGZ1bmN0aW9uKGUpe3ZhciByPStuLGk9
ci8oZVsxXVswXS1lWzBdWzBdKSxvPShyLWkqKGVbMV1bMF0rZVswXVswXSkpLzIsYT0taSplWzBd
WzFdO3Quc2NhbGUoMTUwKmkpLnRyYW5zbGF0ZShbbyxhXSl9KSxlKX1mdW5jdGlvbiBsZCh0LG4s
ZSl7cmV0dXJuIHVkKHQsKGZ1bmN0aW9uKGUpe3ZhciByPStuLGk9ci8oZVsxXVsxXS1lWzBdWzFd
KSxvPS1pKmVbMF1bMF0sYT0oci1pKihlWzFdWzFdK2VbMF1bMV0pKS8yO3Quc2NhbGUoMTUwKmkp
LnRyYW5zbGF0ZShbbyxhXSl9KSxlKX1hZC5wcm90b3R5cGU9e2NvbnN0cnVjdG9yOmFkLHBvaW50
OmZ1bmN0aW9uKHQsbil7dGhpcy5zdHJlYW0ucG9pbnQodCxuKX0sc3BoZXJlOmZ1bmN0aW9uKCl7
dGhpcy5zdHJlYW0uc3BoZXJlKCl9LGxpbmVTdGFydDpmdW5jdGlvbigpe3RoaXMuc3RyZWFtLmxp
bmVTdGFydCgpfSxsaW5lRW5kOmZ1bmN0aW9uKCl7dGhpcy5zdHJlYW0ubGluZUVuZCgpfSxwb2x5
Z29uU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLnN0cmVhbS5wb2x5Z29uU3RhcnQoKX0scG9seWdvbkVu
ZDpmdW5jdGlvbigpe3RoaXMuc3RyZWFtLnBvbHlnb25FbmQoKX19O3ZhciBoZD0xNixkZD1BZigz
MCp4Zik7ZnVuY3Rpb24gcGQodCxuKXtyZXR1cm4rbj9mdW5jdGlvbih0LG4pe2Z1bmN0aW9uIGUo
cixpLG8sYSx1LGMsZixzLGwsaCxkLHAsZyx5KXt2YXIgdj1mLXIsXz1zLWksYj12KnYrXypfO2lm
KGI+NCpuJiZnLS0pe3ZhciBtPWEraCx4PXUrZCx3PWMrcCxNPSRmKG0qbSt4Kngrdyp3KSxUPUZm
KHcvPU0pLEE9d2Yod2YodyktMSk8cGZ8fHdmKG8tbCk8cGY/KG8rbCkvMjpUZih4LG0pLFM9dChB
LFQpLEU9U1swXSxOPVNbMV0saz1FLXIsQz1OLWksUD1fKmstdipDOyhQKlAvYj5ufHx3Zigodipr
K18qQykvYi0uNSk+LjN8fGEqaCt1KmQrYypwPGRkKSYmKGUocixpLG8sYSx1LGMsRSxOLEEsbS89
TSx4Lz1NLHcsZyx5KSx5LnBvaW50KEUsTiksZShFLE4sQSxtLHgsdyxmLHMsbCxoLGQscCxnLHkp
KX19cmV0dXJuIGZ1bmN0aW9uKG4pe3ZhciByLGksbyxhLHUsYyxmLHMsbCxoLGQscCxnPXtwb2lu
dDp5LGxpbmVTdGFydDp2LGxpbmVFbmQ6Yixwb2x5Z29uU3RhcnQ6ZnVuY3Rpb24oKXtuLnBvbHln
b25TdGFydCgpLGcubGluZVN0YXJ0PW19LHBvbHlnb25FbmQ6ZnVuY3Rpb24oKXtuLnBvbHlnb25F
bmQoKSxnLmxpbmVTdGFydD12fX07ZnVuY3Rpb24geShlLHIpe2U9dChlLHIpLG4ucG9pbnQoZVsw
XSxlWzFdKX1mdW5jdGlvbiB2KCl7cz1OYU4sZy5wb2ludD1fLG4ubGluZVN0YXJ0KCl9ZnVuY3Rp
b24gXyhyLGkpe3ZhciBvPWdzKFtyLGldKSxhPXQocixpKTtlKHMsbCxmLGgsZCxwLHM9YVswXSxs
PWFbMV0sZj1yLGg9b1swXSxkPW9bMV0scD1vWzJdLGhkLG4pLG4ucG9pbnQocyxsKX1mdW5jdGlv
biBiKCl7Zy5wb2ludD15LG4ubGluZUVuZCgpfWZ1bmN0aW9uIG0oKXt2KCksZy5wb2ludD14LGcu
bGluZUVuZD13fWZ1bmN0aW9uIHgodCxuKXtfKHI9dCxuKSxpPXMsbz1sLGE9aCx1PWQsYz1wLGcu
cG9pbnQ9X31mdW5jdGlvbiB3KCl7ZShzLGwsZixoLGQscCxpLG8scixhLHUsYyxoZCxuKSxnLmxp
bmVFbmQ9YixiKCl9cmV0dXJuIGd9fSh0LG4pOmZ1bmN0aW9uKHQpe3JldHVybiBvZCh7cG9pbnQ6
ZnVuY3Rpb24obixlKXtuPXQobixlKSx0aGlzLnN0cmVhbS5wb2ludChuWzBdLG5bMV0pfX0pfSh0
KX12YXIgZ2Q9b2Qoe3BvaW50OmZ1bmN0aW9uKHQsbil7dGhpcy5zdHJlYW0ucG9pbnQodCp4Zixu
KnhmKX19KTtmdW5jdGlvbiB5ZCh0LG4sZSxyLGksbyl7aWYoIW8pcmV0dXJuIGZ1bmN0aW9uKHQs
bixlLHIsaSl7ZnVuY3Rpb24gbyhvLGEpe3JldHVybltuK3QqKG8qPXIpLGUtdCooYSo9aSldfXJl
dHVybiBvLmludmVydD1mdW5jdGlvbihvLGEpe3JldHVyblsoby1uKS90KnIsKGUtYSkvdCppXX0s
b30odCxuLGUscixpKTt2YXIgYT1BZihvKSx1PVBmKG8pLGM9YSp0LGY9dSp0LHM9YS90LGw9dS90
LGg9KHUqZS1hKm4pL3QsZD0odSpuK2EqZSkvdDtmdW5jdGlvbiBwKHQsbyl7cmV0dXJuW2MqKHQq
PXIpLWYqKG8qPWkpK24sZS1mKnQtYypvXX1yZXR1cm4gcC5pbnZlcnQ9ZnVuY3Rpb24odCxuKXty
ZXR1cm5bcioocyp0LWwqbitoKSxpKihkLWwqdC1zKm4pXX0scH1mdW5jdGlvbiB2ZCh0KXtyZXR1
cm4gX2QoKGZ1bmN0aW9uKCl7cmV0dXJuIHR9KSkoKX1mdW5jdGlvbiBfZCh0KXt2YXIgbixlLHIs
aSxvLGEsdSxjLGYscyxsPTE1MCxoPTQ4MCxkPTI1MCxwPTAsZz0wLHk9MCx2PTAsXz0wLGI9MCxt
PTEseD0xLHc9bnVsbCxNPUFsLFQ9bnVsbCxBPXJoLFM9LjU7ZnVuY3Rpb24gRSh0KXtyZXR1cm4g
Yyh0WzBdKnhmLHRbMV0qeGYpfWZ1bmN0aW9uIE4odCl7cmV0dXJuKHQ9Yy5pbnZlcnQodFswXSx0
WzFdKSkmJlt0WzBdKm1mLHRbMV0qbWZdfWZ1bmN0aW9uIGsoKXt2YXIgdD15ZChsLDAsMCxtLHgs
YikuYXBwbHkobnVsbCxuKHAsZykpLHI9eWQobCxoLXRbMF0sZC10WzFdLG0seCxiKTtyZXR1cm4g
ZT1jbCh5LHYsXyksdT1hbChuLHIpLGM9YWwoZSx1KSxhPXBkKHUsUyksQygpfWZ1bmN0aW9uIEMo
KXtyZXR1cm4gZj1zPW51bGwsRX1yZXR1cm4gRS5zdHJlYW09ZnVuY3Rpb24odCl7cmV0dXJuIGYm
JnM9PT10P2Y6Zj1nZChmdW5jdGlvbih0KXtyZXR1cm4gb2Qoe3BvaW50OmZ1bmN0aW9uKG4sZSl7
dmFyIHI9dChuLGUpO3JldHVybiB0aGlzLnN0cmVhbS5wb2ludChyWzBdLHJbMV0pfX0pfShlKShN
KGEoQShzPXQpKSkpKX0sRS5wcmVjbGlwPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPyhNPXQsdz12b2lkIDAsQygpKTpNfSxFLnBvc3RjbGlwPWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyhBPXQsVD1yPWk9bz1udWxsLEMoKSk6QX0sRS5jbGlwQW5nbGU9ZnVu
Y3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KE09K3Q/U2wodz10KnhmKToodz1udWxs
LEFsKSxDKCkpOncqbWZ9LEUuY2xpcEV4dGVudD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD8oQT1udWxsPT10PyhUPXI9aT1vPW51bGwscmgpOiRsKFQ9K3RbMF1bMF0scj0rdFsw
XVsxXSxpPSt0WzFdWzBdLG89K3RbMV1bMV0pLEMoKSk6bnVsbD09VD9udWxsOltbVCxyXSxbaSxv
XV19LEUuc2NhbGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGw9K3Qsaygp
KTpsfSxFLnRyYW5zbGF0ZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oaD0r
dFswXSxkPSt0WzFdLGsoKSk6W2gsZF19LEUuY2VudGVyPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1
bWVudHMubGVuZ3RoPyhwPXRbMF0lMzYwKnhmLGc9dFsxXSUzNjAqeGYsaygpKTpbcCptZixnKm1m
XX0sRS5yb3RhdGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHk9dFswXSUz
NjAqeGYsdj10WzFdJTM2MCp4ZixfPXQubGVuZ3RoPjI/dFsyXSUzNjAqeGY6MCxrKCkpOlt5Km1m
LHYqbWYsXyptZl19LEUuYW5nbGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KGI9dCUzNjAqeGYsaygpKTpiKm1mfSxFLnJlZmxlY3RYPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1
bWVudHMubGVuZ3RoPyhtPXQ/LTE6MSxrKCkpOm08MH0sRS5yZWZsZWN0WT1mdW5jdGlvbih0KXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oeD10Py0xOjEsaygpKTp4PDB9LEUucHJlY2lzaW9uPWZ1
bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhhPXBkKHUsUz10KnQpLEMoKSk6JGYo
Uyl9LEUuZml0RXh0ZW50PWZ1bmN0aW9uKHQsbil7cmV0dXJuIGNkKEUsdCxuKX0sRS5maXRTaXpl
PWZ1bmN0aW9uKHQsbil7cmV0dXJuIGZkKEUsdCxuKX0sRS5maXRXaWR0aD1mdW5jdGlvbih0LG4p
e3JldHVybiBzZChFLHQsbil9LEUuZml0SGVpZ2h0PWZ1bmN0aW9uKHQsbil7cmV0dXJuIGxkKEUs
dCxuKX0sZnVuY3Rpb24oKXtyZXR1cm4gbj10LmFwcGx5KHRoaXMsYXJndW1lbnRzKSxFLmludmVy
dD1uLmludmVydCYmTixrKCl9fWZ1bmN0aW9uIGJkKHQpe3ZhciBuPTAsZT15Zi8zLHI9X2QodCks
aT1yKG4sZSk7cmV0dXJuIGkucGFyYWxsZWxzPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoP3Iobj10WzBdKnhmLGU9dFsxXSp4Zik6W24qbWYsZSptZl19LGl9ZnVuY3Rpb24gbWQo
dCxuKXt2YXIgZT1QZih0KSxyPShlK1BmKG4pKS8yO2lmKHdmKHIpPHBmKXJldHVybiBmdW5jdGlv
bih0KXt2YXIgbj1BZih0KTtmdW5jdGlvbiBlKHQsZSl7cmV0dXJuW3QqbixQZihlKS9uXX1yZXR1
cm4gZS5pbnZlcnQ9ZnVuY3Rpb24odCxlKXtyZXR1cm5bdC9uLEZmKGUqbildfSxlfSh0KTt2YXIg
aT0xK2UqKDIqci1lKSxvPSRmKGkpL3I7ZnVuY3Rpb24gYSh0LG4pe3ZhciBlPSRmKGktMipyKlBm
KG4pKS9yO3JldHVybltlKlBmKHQqPXIpLG8tZSpBZih0KV19cmV0dXJuIGEuaW52ZXJ0PWZ1bmN0
aW9uKHQsbil7dmFyIGU9by1uLGE9VGYodCx3ZihlKSkqemYoZSk7cmV0dXJuIGUqcjwwJiYoYS09
eWYqemYodCkqemYoZSkpLFthL3IsRmYoKGktKHQqdCtlKmUpKnIqcikvKDIqcikpXX0sYX1mdW5j
dGlvbiB4ZCgpe3JldHVybiBiZChtZCkuc2NhbGUoMTU1LjQyNCkuY2VudGVyKFswLDMzLjY0NDJd
KX1mdW5jdGlvbiB3ZCgpe3JldHVybiB4ZCgpLnBhcmFsbGVscyhbMjkuNSw0NS41XSkuc2NhbGUo
MTA3MCkudHJhbnNsYXRlKFs0ODAsMjUwXSkucm90YXRlKFs5NiwwXSkuY2VudGVyKFstLjYsMzgu
N10pfWZ1bmN0aW9uIE1kKHQpe3JldHVybiBmdW5jdGlvbihuLGUpe3ZhciByPUFmKG4pLGk9QWYo
ZSksbz10KHIqaSk7cmV0dXJuIG89PT0xLzA/WzIsMF06W28qaSpQZihuKSxvKlBmKGUpXX19ZnVu
Y3Rpb24gVGQodCl7cmV0dXJuIGZ1bmN0aW9uKG4sZSl7dmFyIHI9JGYobipuK2UqZSksaT10KHIp
LG89UGYoaSksYT1BZihpKTtyZXR1cm5bVGYobipvLHIqYSksRmYociYmZSpvL3IpXX19dmFyIEFk
PU1kKChmdW5jdGlvbih0KXtyZXR1cm4gJGYoMi8oMSt0KSl9KSk7QWQuaW52ZXJ0PVRkKChmdW5j
dGlvbih0KXtyZXR1cm4gMipGZih0LzIpfSkpO3ZhciBTZD1NZCgoZnVuY3Rpb24odCl7cmV0dXJu
KHQ9UmYodCkpJiZ0L1BmKHQpfSkpO2Z1bmN0aW9uIEVkKHQsbil7cmV0dXJuW3Qsa2YoRGYoKHZm
K24pLzIpKV19ZnVuY3Rpb24gTmQodCl7dmFyIG4sZSxyLGk9dmQodCksbz1pLmNlbnRlcixhPWku
c2NhbGUsdT1pLnRyYW5zbGF0ZSxjPWkuY2xpcEV4dGVudCxmPW51bGw7ZnVuY3Rpb24gcygpe3Zh
ciBvPXlmKmEoKSx1PWkoaGwoaS5yb3RhdGUoKSkuaW52ZXJ0KFswLDBdKSk7cmV0dXJuIGMobnVs
bD09Zj9bW3VbMF0tbyx1WzFdLW9dLFt1WzBdK28sdVsxXStvXV06dD09PUVkP1tbTWF0aC5tYXgo
dVswXS1vLGYpLG5dLFtNYXRoLm1pbih1WzBdK28sZSkscl1dOltbZixNYXRoLm1heCh1WzFdLW8s
bildLFtlLE1hdGgubWluKHVbMV0rbyxyKV1dKX1yZXR1cm4gaS5zY2FsZT1mdW5jdGlvbih0KXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYSh0KSxzKCkpOmEoKX0saS50cmFuc2xhdGU9ZnVuY3Rp
b24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHUodCkscygpKTp1KCl9LGkuY2VudGVyPWZ1
bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhvKHQpLHMoKSk6bygpfSxpLmNsaXBF
eHRlbnQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG51bGw9PXQ/Zj1uPWU9
cj1udWxsOihmPSt0WzBdWzBdLG49K3RbMF1bMV0sZT0rdFsxXVswXSxyPSt0WzFdWzFdKSxzKCkp
Om51bGw9PWY/bnVsbDpbW2Ysbl0sW2Uscl1dfSxzKCl9ZnVuY3Rpb24ga2QodCl7cmV0dXJuIERm
KCh2Zit0KS8yKX1mdW5jdGlvbiBDZCh0LG4pe3ZhciBlPUFmKHQpLHI9dD09PW4/UGYodCk6a2Yo
ZS9BZihuKSkva2Yoa2Qobikva2QodCkpLGk9ZSpDZihrZCh0KSxyKS9yO2lmKCFyKXJldHVybiBF
ZDtmdW5jdGlvbiBvKHQsbil7aT4wP248LXZmK3BmJiYobj0tdmYrcGYpOm4+dmYtcGYmJihuPXZm
LXBmKTt2YXIgZT1pL0NmKGtkKG4pLHIpO3JldHVybltlKlBmKHIqdCksaS1lKkFmKHIqdCldfXJl
dHVybiBvLmludmVydD1mdW5jdGlvbih0LG4pe3ZhciBlPWktbixvPXpmKHIpKiRmKHQqdCtlKmUp
LGE9VGYodCx3ZihlKSkqemYoZSk7cmV0dXJuIGUqcjwwJiYoYS09eWYqemYodCkqemYoZSkpLFth
L3IsMipNZihDZihpL28sMS9yKSktdmZdfSxvfWZ1bmN0aW9uIFBkKHQsbil7cmV0dXJuW3Qsbl19
ZnVuY3Rpb24gemQodCxuKXt2YXIgZT1BZih0KSxyPXQ9PT1uP1BmKHQpOihlLUFmKG4pKS8obi10
KSxpPWUvcit0O2lmKHdmKHIpPHBmKXJldHVybiBQZDtmdW5jdGlvbiBvKHQsbil7dmFyIGU9aS1u
LG89cip0O3JldHVybltlKlBmKG8pLGktZSpBZihvKV19cmV0dXJuIG8uaW52ZXJ0PWZ1bmN0aW9u
KHQsbil7dmFyIGU9aS1uLG89VGYodCx3ZihlKSkqemYoZSk7cmV0dXJuIGUqcjwwJiYoby09eWYq
emYodCkqemYoZSkpLFtvL3IsaS16ZihyKSokZih0KnQrZSplKV19LG99U2QuaW52ZXJ0PVRkKChm
dW5jdGlvbih0KXtyZXR1cm4gdH0pKSxFZC5pbnZlcnQ9ZnVuY3Rpb24odCxuKXtyZXR1cm5bdCwy
Kk1mKEVmKG4pKS12Zl19LFBkLmludmVydD1QZDt2YXIgJGQ9MS4zNDAyNjQsRGQ9LS4wODExMDYs
UmQ9ODkzZS02LEZkPS4wMDM3OTYscWQ9JGYoMykvMjtmdW5jdGlvbiBVZCh0LG4pe3ZhciBlPUZm
KHFkKlBmKG4pKSxyPWUqZSxpPXIqcipyO3JldHVyblt0KkFmKGUpLyhxZCooJGQrMypEZCpyK2kq
KDcqUmQrOSpGZCpyKSkpLGUqKCRkK0RkKnIraSooUmQrRmQqcikpXX1mdW5jdGlvbiBJZCh0LG4p
e3ZhciBlPUFmKG4pLHI9QWYodCkqZTtyZXR1cm5bZSpQZih0KS9yLFBmKG4pL3JdfWZ1bmN0aW9u
IE9kKHQsbil7dmFyIGU9bipuLHI9ZSplO3JldHVyblt0KiguODcwNy0uMTMxOTc5KmUrcioocioo
LjAwMzk3MSplLS4wMDE1MjkqciktLjAxMzc5MSkpLG4qKDEuMDA3MjI2K2UqKC4wMTUwODUrcioo
LjAyODg3NCplLS4wNDQ0NzUtLjAwNTkxNipyKSkpXX1mdW5jdGlvbiBCZCh0LG4pe3JldHVybltB
ZihuKSpQZih0KSxQZihuKV19ZnVuY3Rpb24gWWQodCxuKXt2YXIgZT1BZihuKSxyPTErQWYodCkq
ZTtyZXR1cm5bZSpQZih0KS9yLFBmKG4pL3JdfWZ1bmN0aW9uIExkKHQsbil7cmV0dXJuW2tmKERm
KCh2ZituKS8yKSksLXRdfWZ1bmN0aW9uIGpkKHQsbil7cmV0dXJuIHQucGFyZW50PT09bi5wYXJl
bnQ/MToyfWZ1bmN0aW9uIEhkKHQsbil7cmV0dXJuIHQrbi54fWZ1bmN0aW9uIFhkKHQsbil7cmV0
dXJuIE1hdGgubWF4KHQsbi55KX1mdW5jdGlvbiBHZCh0KXt2YXIgbj0wLGU9dC5jaGlsZHJlbixy
PWUmJmUubGVuZ3RoO2lmKHIpZm9yKDstLXI+PTA7KW4rPWVbcl0udmFsdWU7ZWxzZSBuPTE7dC52
YWx1ZT1ufWZ1bmN0aW9uIFZkKHQsbil7dCBpbnN0YW5jZW9mIE1hcD8odD1bdm9pZCAwLHRdLHZv
aWQgMD09PW4mJihuPVpkKSk6dm9pZCAwPT09biYmKG49V2QpO2Zvcih2YXIgZSxyLGksbyxhLHU9
bmV3IEpkKHQpLGM9W3VdO2U9Yy5wb3AoKTspaWYoKGk9bihlLmRhdGEpKSYmKGE9KGk9QXJyYXku
ZnJvbShpKSkubGVuZ3RoKSlmb3IoZS5jaGlsZHJlbj1pLG89YS0xO28+PTA7LS1vKWMucHVzaChy
PWlbb109bmV3IEpkKGlbb10pKSxyLnBhcmVudD1lLHIuZGVwdGg9ZS5kZXB0aCsxO3JldHVybiB1
LmVhY2hCZWZvcmUoUWQpfWZ1bmN0aW9uIFdkKHQpe3JldHVybiB0LmNoaWxkcmVufWZ1bmN0aW9u
IFpkKHQpe3JldHVybiBBcnJheS5pc0FycmF5KHQpP3RbMV06bnVsbH1mdW5jdGlvbiBLZCh0KXt2
b2lkIDAhPT10LmRhdGEudmFsdWUmJih0LnZhbHVlPXQuZGF0YS52YWx1ZSksdC5kYXRhPXQuZGF0
YS5kYXRhfWZ1bmN0aW9uIFFkKHQpe3ZhciBuPTA7ZG97dC5oZWlnaHQ9bn13aGlsZSgodD10LnBh
cmVudCkmJnQuaGVpZ2h0PCsrbil9ZnVuY3Rpb24gSmQodCl7dGhpcy5kYXRhPXQsdGhpcy5kZXB0
aD10aGlzLmhlaWdodD0wLHRoaXMucGFyZW50PW51bGx9ZnVuY3Rpb24gdHAodCl7cmV0dXJuIG51
bGw9PXQ/bnVsbDpucCh0KX1mdW5jdGlvbiBucCh0KXtpZigiZnVuY3Rpb24iIT10eXBlb2YgdCl0
aHJvdyBuZXcgRXJyb3I7cmV0dXJuIHR9ZnVuY3Rpb24gZXAoKXtyZXR1cm4gMH1mdW5jdGlvbiBy
cCh0KXtyZXR1cm4gZnVuY3Rpb24oKXtyZXR1cm4gdH19VWQuaW52ZXJ0PWZ1bmN0aW9uKHQsbil7
Zm9yKHZhciBlLHI9bixpPXIqcixvPWkqaSppLGE9MDthPDEyJiYobz0oaT0oci09ZT0ociooJGQr
RGQqaStvKihSZCtGZCppKSktbikvKCRkKzMqRGQqaStvKig3KlJkKzkqRmQqaSkpKSpyKSppKmks
ISh3ZihlKTxnZikpOysrYSk7cmV0dXJuW3FkKnQqKCRkKzMqRGQqaStvKig3KlJkKzkqRmQqaSkp
L0FmKHIpLEZmKFBmKHIpL3FkKV19LElkLmludmVydD1UZChNZiksT2QuaW52ZXJ0PWZ1bmN0aW9u
KHQsbil7dmFyIGUscj1uLGk9MjU7ZG97dmFyIG89cipyLGE9bypvO3ItPWU9KHIqKDEuMDA3MjI2
K28qKC4wMTUwODUrYSooLjAyODg3NCpvLS4wNDQ0NzUtLjAwNTkxNiphKSkpLW4pLygxLjAwNzIy
NitvKiguMDQ1MjU1K2EqKC4yNTk4NjYqby0uMzExMzI1LS4wMDU5MTYqMTEqYSkpKX13aGlsZSh3
ZihlKT5wZiYmLS1pPjApO3JldHVyblt0LyguODcwNysobz1yKnIpKihvKihvKm8qbyooLjAwMzk3
MS0uMDAxNTI5Km8pLS4wMTM3OTEpLS4xMzE5NzkpKSxyXX0sQmQuaW52ZXJ0PVRkKEZmKSxZZC5p
bnZlcnQ9VGQoKGZ1bmN0aW9uKHQpe3JldHVybiAyKk1mKHQpfSkpLExkLmludmVydD1mdW5jdGlv
bih0LG4pe3JldHVyblstbiwyKk1mKEVmKHQpKS12Zl19LEpkLnByb3RvdHlwZT1WZC5wcm90b3R5
cGU9e2NvbnN0cnVjdG9yOkpkLGNvdW50OmZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuZWFjaEFmdGVy
KEdkKX0sZWFjaDpmdW5jdGlvbih0LG4pe2xldCBlPS0xO2Zvcihjb25zdCByIG9mIHRoaXMpdC5j
YWxsKG4sciwrK2UsdGhpcyk7cmV0dXJuIHRoaXN9LGVhY2hBZnRlcjpmdW5jdGlvbih0LG4pe2Zv
cih2YXIgZSxyLGksbz10aGlzLGE9W29dLHU9W10sYz0tMTtvPWEucG9wKCk7KWlmKHUucHVzaChv
KSxlPW8uY2hpbGRyZW4pZm9yKHI9MCxpPWUubGVuZ3RoO3I8aTsrK3IpYS5wdXNoKGVbcl0pO2Zv
cig7bz11LnBvcCgpOyl0LmNhbGwobixvLCsrYyx0aGlzKTtyZXR1cm4gdGhpc30sZWFjaEJlZm9y
ZTpmdW5jdGlvbih0LG4pe2Zvcih2YXIgZSxyLGk9dGhpcyxvPVtpXSxhPS0xO2k9by5wb3AoKTsp
aWYodC5jYWxsKG4saSwrK2EsdGhpcyksZT1pLmNoaWxkcmVuKWZvcihyPWUubGVuZ3RoLTE7cj49
MDstLXIpby5wdXNoKGVbcl0pO3JldHVybiB0aGlzfSxmaW5kOmZ1bmN0aW9uKHQsbil7bGV0IGU9
LTE7Zm9yKGNvbnN0IHIgb2YgdGhpcylpZih0LmNhbGwobixyLCsrZSx0aGlzKSlyZXR1cm4gcn0s
c3VtOmZ1bmN0aW9uKHQpe3JldHVybiB0aGlzLmVhY2hBZnRlcigoZnVuY3Rpb24obil7Zm9yKHZh
ciBlPSt0KG4uZGF0YSl8fDAscj1uLmNoaWxkcmVuLGk9ciYmci5sZW5ndGg7LS1pPj0wOyllKz1y
W2ldLnZhbHVlO24udmFsdWU9ZX0pKX0sc29ydDpmdW5jdGlvbih0KXtyZXR1cm4gdGhpcy5lYWNo
QmVmb3JlKChmdW5jdGlvbihuKXtuLmNoaWxkcmVuJiZuLmNoaWxkcmVuLnNvcnQodCl9KSl9LHBh
dGg6ZnVuY3Rpb24odCl7Zm9yKHZhciBuPXRoaXMsZT1mdW5jdGlvbih0LG4pe2lmKHQ9PT1uKXJl
dHVybiB0O3ZhciBlPXQuYW5jZXN0b3JzKCkscj1uLmFuY2VzdG9ycygpLGk9bnVsbDt0PWUucG9w
KCksbj1yLnBvcCgpO2Zvcig7dD09PW47KWk9dCx0PWUucG9wKCksbj1yLnBvcCgpO3JldHVybiBp
fShuLHQpLHI9W25dO24hPT1lOyluPW4ucGFyZW50LHIucHVzaChuKTtmb3IodmFyIGk9ci5sZW5n
dGg7dCE9PWU7KXIuc3BsaWNlKGksMCx0KSx0PXQucGFyZW50O3JldHVybiByfSxhbmNlc3RvcnM6
ZnVuY3Rpb24oKXtmb3IodmFyIHQ9dGhpcyxuPVt0XTt0PXQucGFyZW50OyluLnB1c2godCk7cmV0
dXJuIG59LGRlc2NlbmRhbnRzOmZ1bmN0aW9uKCl7cmV0dXJuIEFycmF5LmZyb20odGhpcyl9LGxl
YXZlczpmdW5jdGlvbigpe3ZhciB0PVtdO3JldHVybiB0aGlzLmVhY2hCZWZvcmUoKGZ1bmN0aW9u
KG4pe24uY2hpbGRyZW58fHQucHVzaChuKX0pKSx0fSxsaW5rczpmdW5jdGlvbigpe3ZhciB0PXRo
aXMsbj1bXTtyZXR1cm4gdC5lYWNoKChmdW5jdGlvbihlKXtlIT09dCYmbi5wdXNoKHtzb3VyY2U6
ZS5wYXJlbnQsdGFyZ2V0OmV9KX0pKSxufSxjb3B5OmZ1bmN0aW9uKCl7cmV0dXJuIFZkKHRoaXMp
LmVhY2hCZWZvcmUoS2QpfSxbU3ltYm9sLml0ZXJhdG9yXTpmdW5jdGlvbiooKXt2YXIgdCxuLGUs
cixpPXRoaXMsbz1baV07ZG97Zm9yKHQ9by5yZXZlcnNlKCksbz1bXTtpPXQucG9wKCk7KWlmKHlp
ZWxkIGksbj1pLmNoaWxkcmVuKWZvcihlPTAscj1uLmxlbmd0aDtlPHI7KytlKW8ucHVzaChuW2Vd
KX13aGlsZShvLmxlbmd0aCl9fTtjb25zdCBpcD0xNjY0NTI1LG9wPTEwMTM5MDQyMjMsYXA9NDI5
NDk2NzI5NjtmdW5jdGlvbiB1cCgpe2xldCB0PTE7cmV0dXJuKCk9Pih0PShpcCp0K29wKSVhcCkv
YXB9ZnVuY3Rpb24gY3AodCxuKXtmb3IodmFyIGUscixpPTAsbz0odD1mdW5jdGlvbih0LG4pe2xl
dCBlLHIsaT10Lmxlbmd0aDtmb3IoO2k7KXI9bigpKmktLXwwLGU9dFtpXSx0W2ldPXRbcl0sdFty
XT1lO3JldHVybiB0fShBcnJheS5mcm9tKHQpLG4pKS5sZW5ndGgsYT1bXTtpPG87KWU9dFtpXSxy
JiZscChyLGUpPysraToocj1kcChhPWZwKGEsZSkpLGk9MCk7cmV0dXJuIHJ9ZnVuY3Rpb24gZnAo
dCxuKXt2YXIgZSxyO2lmKGhwKG4sdCkpcmV0dXJuW25dO2ZvcihlPTA7ZTx0Lmxlbmd0aDsrK2Up
aWYoc3Aobix0W2VdKSYmaHAocHAodFtlXSxuKSx0KSlyZXR1cm5bdFtlXSxuXTtmb3IoZT0wO2U8
dC5sZW5ndGgtMTsrK2UpZm9yKHI9ZSsxO3I8dC5sZW5ndGg7KytyKWlmKHNwKHBwKHRbZV0sdFty
XSksbikmJnNwKHBwKHRbZV0sbiksdFtyXSkmJnNwKHBwKHRbcl0sbiksdFtlXSkmJmhwKGdwKHRb
ZV0sdFtyXSxuKSx0KSlyZXR1cm5bdFtlXSx0W3JdLG5dO3Rocm93IG5ldyBFcnJvcn1mdW5jdGlv
biBzcCh0LG4pe3ZhciBlPXQuci1uLnIscj1uLngtdC54LGk9bi55LXQueTtyZXR1cm4gZTwwfHxl
KmU8cipyK2kqaX1mdW5jdGlvbiBscCh0LG4pe3ZhciBlPXQuci1uLnIrMWUtOSpNYXRoLm1heCh0
LnIsbi5yLDEpLHI9bi54LXQueCxpPW4ueS10Lnk7cmV0dXJuIGU+MCYmZSplPnIqcitpKml9ZnVu
Y3Rpb24gaHAodCxuKXtmb3IodmFyIGU9MDtlPG4ubGVuZ3RoOysrZSlpZighbHAodCxuW2VdKSly
ZXR1cm4hMTtyZXR1cm4hMH1mdW5jdGlvbiBkcCh0KXtzd2l0Y2godC5sZW5ndGgpe2Nhc2UgMTpy
ZXR1cm4gZnVuY3Rpb24odCl7cmV0dXJue3g6dC54LHk6dC55LHI6dC5yfX0odFswXSk7Y2FzZSAy
OnJldHVybiBwcCh0WzBdLHRbMV0pO2Nhc2UgMzpyZXR1cm4gZ3AodFswXSx0WzFdLHRbMl0pfX1m
dW5jdGlvbiBwcCh0LG4pe3ZhciBlPXQueCxyPXQueSxpPXQucixvPW4ueCxhPW4ueSx1PW4ucixj
PW8tZSxmPWEtcixzPXUtaSxsPU1hdGguc3FydChjKmMrZipmKTtyZXR1cm57eDooZStvK2MvbCpz
KS8yLHk6KHIrYStmL2wqcykvMixyOihsK2krdSkvMn19ZnVuY3Rpb24gZ3AodCxuLGUpe3ZhciBy
PXQueCxpPXQueSxvPXQucixhPW4ueCx1PW4ueSxjPW4ucixmPWUueCxzPWUueSxsPWUucixoPXIt
YSxkPXItZixwPWktdSxnPWktcyx5PWMtbyx2PWwtbyxfPXIqcitpKmktbypvLGI9Xy1hKmEtdSp1
K2MqYyxtPV8tZipmLXMqcytsKmwseD1kKnAtaCpnLHc9KHAqbS1nKmIpLygyKngpLXIsTT0oZyp5
LXAqdikveCxUPShkKmItaCptKS8oMip4KS1pLEE9KGgqdi1kKnkpL3gsUz1NKk0rQSpBLTEsRT0y
KihvK3cqTStUKkEpLE49dyp3K1QqVC1vKm8saz0tKE1hdGguYWJzKFMpPjFlLTY/KEUrTWF0aC5z
cXJ0KEUqRS00KlMqTikpLygyKlMpOk4vRSk7cmV0dXJue3g6cit3K00qayx5OmkrVCtBKmsscjpr
fX1mdW5jdGlvbiB5cCh0LG4sZSl7dmFyIHIsaSxvLGEsdT10Lngtbi54LGM9dC55LW4ueSxmPXUq
dStjKmM7Zj8oaT1uLnIrZS5yLGkqPWksYT10LnIrZS5yLGk+KGEqPWEpPyhyPShmK2EtaSkvKDIq
Ziksbz1NYXRoLnNxcnQoTWF0aC5tYXgoMCxhL2YtcipyKSksZS54PXQueC1yKnUtbypjLGUueT10
LnktcipjK28qdSk6KHI9KGYraS1hKS8oMipmKSxvPU1hdGguc3FydChNYXRoLm1heCgwLGkvZi1y
KnIpKSxlLng9bi54K3IqdS1vKmMsZS55PW4ueStyKmMrbyp1KSk6KGUueD1uLngrZS5yLGUueT1u
LnkpfWZ1bmN0aW9uIHZwKHQsbil7dmFyIGU9dC5yK24uci0xZS02LHI9bi54LXQueCxpPW4ueS10
Lnk7cmV0dXJuIGU+MCYmZSplPnIqcitpKml9ZnVuY3Rpb24gX3AodCl7dmFyIG49dC5fLGU9dC5u
ZXh0Ll8scj1uLnIrZS5yLGk9KG4ueCplLnIrZS54Km4ucikvcixvPShuLnkqZS5yK2UueSpuLnIp
L3I7cmV0dXJuIGkqaStvKm99ZnVuY3Rpb24gYnAodCl7dGhpcy5fPXQsdGhpcy5uZXh0PW51bGws
dGhpcy5wcmV2aW91cz1udWxsfWZ1bmN0aW9uIG1wKHQsbil7aWYoIShvPSh0PWZ1bmN0aW9uKHQp
e3JldHVybiJvYmplY3QiPT10eXBlb2YgdCYmImxlbmd0aCJpbiB0P3Q6QXJyYXkuZnJvbSh0KX0o
dCkpLmxlbmd0aCkpcmV0dXJuIDA7dmFyIGUscixpLG8sYSx1LGMsZixzLGwsaDtpZigoZT10WzBd
KS54PTAsZS55PTAsIShvPjEpKXJldHVybiBlLnI7aWYocj10WzFdLGUueD0tci5yLHIueD1lLnIs
ci55PTAsIShvPjIpKXJldHVybiBlLnIrci5yO3lwKHIsZSxpPXRbMl0pLGU9bmV3IGJwKGUpLHI9
bmV3IGJwKHIpLGk9bmV3IGJwKGkpLGUubmV4dD1pLnByZXZpb3VzPXIsci5uZXh0PWUucHJldmlv
dXM9aSxpLm5leHQ9ci5wcmV2aW91cz1lO3Q6Zm9yKGM9MztjPG87KytjKXt5cChlLl8sci5fLGk9
dFtjXSksaT1uZXcgYnAoaSksZj1yLm5leHQscz1lLnByZXZpb3VzLGw9ci5fLnIsaD1lLl8ucjtk
b3tpZihsPD1oKXtpZih2cChmLl8saS5fKSl7cj1mLGUubmV4dD1yLHIucHJldmlvdXM9ZSwtLWM7
Y29udGludWUgdH1sKz1mLl8ucixmPWYubmV4dH1lbHNle2lmKHZwKHMuXyxpLl8pKXsoZT1zKS5u
ZXh0PXIsci5wcmV2aW91cz1lLC0tYztjb250aW51ZSB0fWgrPXMuXy5yLHM9cy5wcmV2aW91c319
d2hpbGUoZiE9PXMubmV4dCk7Zm9yKGkucHJldmlvdXM9ZSxpLm5leHQ9cixlLm5leHQ9ci5wcmV2
aW91cz1yPWksYT1fcChlKTsoaT1pLm5leHQpIT09cjspKHU9X3AoaSkpPGEmJihlPWksYT11KTty
PWUubmV4dH1mb3IoZT1bci5fXSxpPXI7KGk9aS5uZXh0KSE9PXI7KWUucHVzaChpLl8pO2Zvcihp
PWNwKGUsbiksYz0wO2M8bzsrK2MpKGU9dFtjXSkueC09aS54LGUueS09aS55O3JldHVybiBpLnJ9
ZnVuY3Rpb24geHAodCl7cmV0dXJuIE1hdGguc3FydCh0LnZhbHVlKX1mdW5jdGlvbiB3cCh0KXty
ZXR1cm4gZnVuY3Rpb24obil7bi5jaGlsZHJlbnx8KG4ucj1NYXRoLm1heCgwLCt0KG4pfHwwKSl9
fWZ1bmN0aW9uIE1wKHQsbixlKXtyZXR1cm4gZnVuY3Rpb24ocil7aWYoaT1yLmNoaWxkcmVuKXt2
YXIgaSxvLGEsdT1pLmxlbmd0aCxjPXQocikqbnx8MDtpZihjKWZvcihvPTA7bzx1OysrbylpW29d
LnIrPWM7aWYoYT1tcChpLGUpLGMpZm9yKG89MDtvPHU7KytvKWlbb10uci09YztyLnI9YStjfX19
ZnVuY3Rpb24gVHAodCl7cmV0dXJuIGZ1bmN0aW9uKG4pe3ZhciBlPW4ucGFyZW50O24ucio9dCxl
JiYobi54PWUueCt0Km4ueCxuLnk9ZS55K3Qqbi55KX19ZnVuY3Rpb24gQXAodCl7dC54MD1NYXRo
LnJvdW5kKHQueDApLHQueTA9TWF0aC5yb3VuZCh0LnkwKSx0LngxPU1hdGgucm91bmQodC54MSks
dC55MT1NYXRoLnJvdW5kKHQueTEpfWZ1bmN0aW9uIFNwKHQsbixlLHIsaSl7Zm9yKHZhciBvLGE9
dC5jaGlsZHJlbix1PS0xLGM9YS5sZW5ndGgsZj10LnZhbHVlJiYoci1uKS90LnZhbHVlOysrdTxj
Oykobz1hW3VdKS55MD1lLG8ueTE9aSxvLngwPW4sby54MT1uKz1vLnZhbHVlKmZ9dmFyIEVwPXtk
ZXB0aDotMX0sTnA9e30sa3A9e307ZnVuY3Rpb24gQ3AodCl7cmV0dXJuIHQuaWR9ZnVuY3Rpb24g
UHAodCl7cmV0dXJuIHQucGFyZW50SWR9ZnVuY3Rpb24genAodCl7bGV0IG49dC5sZW5ndGg7aWYo
bjwyKXJldHVybiIiO2Zvcig7LS1uPjEmJiEkcCh0LG4pOyk7cmV0dXJuIHQuc2xpY2UoMCxuKX1m
dW5jdGlvbiAkcCh0LG4pe2lmKCIvIj09PXRbbl0pe2xldCBlPTA7Zm9yKDtuPjAmJiJcXCI9PT10
Wy0tbl07KSsrZTtpZigwPT0oMSZlKSlyZXR1cm4hMH1yZXR1cm4hMX1mdW5jdGlvbiBEcCh0LG4p
e3JldHVybiB0LnBhcmVudD09PW4ucGFyZW50PzE6Mn1mdW5jdGlvbiBScCh0KXt2YXIgbj10LmNo
aWxkcmVuO3JldHVybiBuP25bMF06dC50fWZ1bmN0aW9uIEZwKHQpe3ZhciBuPXQuY2hpbGRyZW47
cmV0dXJuIG4/bltuLmxlbmd0aC0xXTp0LnR9ZnVuY3Rpb24gcXAodCxuLGUpe3ZhciByPWUvKG4u
aS10LmkpO24uYy09cixuLnMrPWUsdC5jKz1yLG4ueis9ZSxuLm0rPWV9ZnVuY3Rpb24gVXAodCxu
LGUpe3JldHVybiB0LmEucGFyZW50PT09bi5wYXJlbnQ/dC5hOmV9ZnVuY3Rpb24gSXAodCxuKXt0
aGlzLl89dCx0aGlzLnBhcmVudD1udWxsLHRoaXMuY2hpbGRyZW49bnVsbCx0aGlzLkE9bnVsbCx0
aGlzLmE9dGhpcyx0aGlzLno9MCx0aGlzLm09MCx0aGlzLmM9MCx0aGlzLnM9MCx0aGlzLnQ9bnVs
bCx0aGlzLmk9bn1mdW5jdGlvbiBPcCh0LG4sZSxyLGkpe2Zvcih2YXIgbyxhPXQuY2hpbGRyZW4s
dT0tMSxjPWEubGVuZ3RoLGY9dC52YWx1ZSYmKGktZSkvdC52YWx1ZTsrK3U8YzspKG89YVt1XSku
eDA9bixvLngxPXIsby55MD1lLG8ueTE9ZSs9by52YWx1ZSpmfUlwLnByb3RvdHlwZT1PYmplY3Qu
Y3JlYXRlKEpkLnByb3RvdHlwZSk7dmFyIEJwPSgxK01hdGguc3FydCg1KSkvMjtmdW5jdGlvbiBZ
cCh0LG4sZSxyLGksbyl7Zm9yKHZhciBhLHUsYyxmLHMsbCxoLGQscCxnLHksdj1bXSxfPW4uY2hp
bGRyZW4sYj0wLG09MCx4PV8ubGVuZ3RoLHc9bi52YWx1ZTtiPHg7KXtjPWktZSxmPW8tcjtkb3tz
PV9bbSsrXS52YWx1ZX13aGlsZSghcyYmbTx4KTtmb3IobD1oPXMseT1zKnMqKGc9TWF0aC5tYXgo
Zi9jLGMvZikvKHcqdCkpLHA9TWF0aC5tYXgoaC95LHkvbCk7bTx4OysrbSl7aWYocys9dT1fW21d
LnZhbHVlLHU8bCYmKGw9dSksdT5oJiYoaD11KSx5PXMqcypnLChkPU1hdGgubWF4KGgveSx5L2wp
KT5wKXtzLT11O2JyZWFrfXA9ZH12LnB1c2goYT17dmFsdWU6cyxkaWNlOmM8ZixjaGlsZHJlbjpf
LnNsaWNlKGIsbSl9KSxhLmRpY2U/U3AoYSxlLHIsaSx3P3IrPWYqcy93Om8pOk9wKGEsZSxyLHc/
ZSs9YypzL3c6aSxvKSx3LT1zLGI9bX1yZXR1cm4gdn12YXIgTHA9ZnVuY3Rpb24gdChuKXtmdW5j
dGlvbiBlKHQsZSxyLGksbyl7WXAobix0LGUscixpLG8pfXJldHVybiBlLnJhdGlvPWZ1bmN0aW9u
KG4pe3JldHVybiB0KChuPStuKT4xP246MSl9LGV9KEJwKTt2YXIganA9ZnVuY3Rpb24gdChuKXtm
dW5jdGlvbiBlKHQsZSxyLGksbyl7aWYoKGE9dC5fc3F1YXJpZnkpJiZhLnJhdGlvPT09bilmb3Io
dmFyIGEsdSxjLGYscyxsPS0xLGg9YS5sZW5ndGgsZD10LnZhbHVlOysrbDxoOyl7Zm9yKGM9KHU9
YVtsXSkuY2hpbGRyZW4sZj11LnZhbHVlPTAscz1jLmxlbmd0aDtmPHM7KytmKXUudmFsdWUrPWNb
Zl0udmFsdWU7dS5kaWNlP1NwKHUsZSxyLGksZD9yKz0oby1yKSp1LnZhbHVlL2Q6byk6T3AodSxl
LHIsZD9lKz0oaS1lKSp1LnZhbHVlL2Q6aSxvKSxkLT11LnZhbHVlfWVsc2UgdC5fc3F1YXJpZnk9
YT1ZcChuLHQsZSxyLGksbyksYS5yYXRpbz1ufXJldHVybiBlLnJhdGlvPWZ1bmN0aW9uKG4pe3Jl
dHVybiB0KChuPStuKT4xP246MSl9LGV9KEJwKTtmdW5jdGlvbiBIcCh0LG4sZSl7cmV0dXJuKG5b
MF0tdFswXSkqKGVbMV0tdFsxXSktKG5bMV0tdFsxXSkqKGVbMF0tdFswXSl9ZnVuY3Rpb24gWHAo
dCxuKXtyZXR1cm4gdFswXS1uWzBdfHx0WzFdLW5bMV19ZnVuY3Rpb24gR3AodCl7Y29uc3Qgbj10
Lmxlbmd0aCxlPVswLDFdO2xldCByLGk9Mjtmb3Iocj0yO3I8bjsrK3Ipe2Zvcig7aT4xJiZIcCh0
W2VbaS0yXV0sdFtlW2ktMV1dLHRbcl0pPD0wOyktLWk7ZVtpKytdPXJ9cmV0dXJuIGUuc2xpY2Uo
MCxpKX12YXIgVnA9TWF0aC5yYW5kb20sV3A9ZnVuY3Rpb24gdChuKXtmdW5jdGlvbiBlKHQsZSl7
cmV0dXJuIHQ9bnVsbD09dD8wOit0LGU9bnVsbD09ZT8xOitlLDE9PT1hcmd1bWVudHMubGVuZ3Ro
PyhlPXQsdD0wKTplLT10LGZ1bmN0aW9uKCl7cmV0dXJuIG4oKSplK3R9fXJldHVybiBlLnNvdXJj
ZT10LGV9KFZwKSxacD1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCxlKXtyZXR1cm4gYXJndW1l
bnRzLmxlbmd0aDwyJiYoZT10LHQ9MCksdD1NYXRoLmZsb29yKHQpLGU9TWF0aC5mbG9vcihlKS10
LGZ1bmN0aW9uKCl7cmV0dXJuIE1hdGguZmxvb3IobigpKmUrdCl9fXJldHVybiBlLnNvdXJjZT10
LGV9KFZwKSxLcD1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCxlKXt2YXIgcixpO3JldHVybiB0
PW51bGw9PXQ/MDordCxlPW51bGw9PWU/MTorZSxmdW5jdGlvbigpe3ZhciBvO2lmKG51bGwhPXIp
bz1yLHI9bnVsbDtlbHNlIGRve3I9MipuKCktMSxvPTIqbigpLTEsaT1yKnIrbypvfXdoaWxlKCFp
fHxpPjEpO3JldHVybiB0K2UqbypNYXRoLnNxcnQoLTIqTWF0aC5sb2coaSkvaSl9fXJldHVybiBl
LnNvdXJjZT10LGV9KFZwKSxRcD1mdW5jdGlvbiB0KG4pe3ZhciBlPUtwLnNvdXJjZShuKTtmdW5j
dGlvbiByKCl7dmFyIHQ9ZS5hcHBseSh0aGlzLGFyZ3VtZW50cyk7cmV0dXJuIGZ1bmN0aW9uKCl7
cmV0dXJuIE1hdGguZXhwKHQoKSl9fXJldHVybiByLnNvdXJjZT10LHJ9KFZwKSxKcD1mdW5jdGlv
biB0KG4pe2Z1bmN0aW9uIGUodCl7cmV0dXJuKHQ9K3QpPD0wPygpPT4wOmZ1bmN0aW9uKCl7Zm9y
KHZhciBlPTAscj10O3I+MTstLXIpZSs9bigpO3JldHVybiBlK3IqbigpfX1yZXR1cm4gZS5zb3Vy
Y2U9dCxlfShWcCksdGc9ZnVuY3Rpb24gdChuKXt2YXIgZT1KcC5zb3VyY2Uobik7ZnVuY3Rpb24g
cih0KXtpZigwPT0odD0rdCkpcmV0dXJuIG47dmFyIHI9ZSh0KTtyZXR1cm4gZnVuY3Rpb24oKXty
ZXR1cm4gcigpL3R9fXJldHVybiByLnNvdXJjZT10LHJ9KFZwKSxuZz1mdW5jdGlvbiB0KG4pe2Z1
bmN0aW9uIGUodCl7cmV0dXJuIGZ1bmN0aW9uKCl7cmV0dXJuLU1hdGgubG9nMXAoLW4oKSkvdH19
cmV0dXJuIGUuc291cmNlPXQsZX0oVnApLGVnPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0KXtp
ZigodD0rdCk8MCl0aHJvdyBuZXcgUmFuZ2VFcnJvcigiaW52YWxpZCBhbHBoYSIpO3JldHVybiB0
PTEvLXQsZnVuY3Rpb24oKXtyZXR1cm4gTWF0aC5wb3coMS1uKCksdCl9fXJldHVybiBlLnNvdXJj
ZT10LGV9KFZwKSxyZz1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCl7aWYoKHQ9K3QpPDB8fHQ+
MSl0aHJvdyBuZXcgUmFuZ2VFcnJvcigiaW52YWxpZCBwIik7cmV0dXJuIGZ1bmN0aW9uKCl7cmV0
dXJuIE1hdGguZmxvb3IobigpK3QpfX1yZXR1cm4gZS5zb3VyY2U9dCxlfShWcCksaWc9ZnVuY3Rp
b24gdChuKXtmdW5jdGlvbiBlKHQpe2lmKCh0PSt0KTwwfHx0PjEpdGhyb3cgbmV3IFJhbmdlRXJy
b3IoImludmFsaWQgcCIpO3JldHVybiAwPT09dD8oKT0+MS8wOjE9PT10PygpPT4xOih0PU1hdGgu
bG9nMXAoLXQpLGZ1bmN0aW9uKCl7cmV0dXJuIDErTWF0aC5mbG9vcihNYXRoLmxvZzFwKC1uKCkp
L3QpfSl9cmV0dXJuIGUuc291cmNlPXQsZX0oVnApLG9nPWZ1bmN0aW9uIHQobil7dmFyIGU9S3Au
c291cmNlKG4pKCk7ZnVuY3Rpb24gcih0LHIpe2lmKCh0PSt0KTwwKXRocm93IG5ldyBSYW5nZUVy
cm9yKCJpbnZhbGlkIGsiKTtpZigwPT09dClyZXR1cm4oKT0+MDtpZihyPW51bGw9PXI/MTorciwx
PT09dClyZXR1cm4oKT0+LU1hdGgubG9nMXAoLW4oKSkqcjt2YXIgaT0odDwxP3QrMTp0KS0xLzMs
bz0xLygzKk1hdGguc3FydChpKSksYT10PDE/KCk9Pk1hdGgucG93KG4oKSwxL3QpOigpPT4xO3Jl
dHVybiBmdW5jdGlvbigpe2Rve2Rve3ZhciB0PWUoKSx1PTErbyp0fXdoaWxlKHU8PTApO3UqPXUq
dTt2YXIgYz0xLW4oKX13aGlsZShjPj0xLS4wMzMxKnQqdCp0KnQmJk1hdGgubG9nKGMpPj0uNSp0
KnQraSooMS11K01hdGgubG9nKHUpKSk7cmV0dXJuIGkqdSphKCkqcn19cmV0dXJuIHIuc291cmNl
PXQscn0oVnApLGFnPWZ1bmN0aW9uIHQobil7dmFyIGU9b2cuc291cmNlKG4pO2Z1bmN0aW9uIHIo
dCxuKXt2YXIgcj1lKHQpLGk9ZShuKTtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgdD1yKCk7cmV0dXJu
IDA9PT10PzA6dC8odCtpKCkpfX1yZXR1cm4gci5zb3VyY2U9dCxyfShWcCksdWc9ZnVuY3Rpb24g
dChuKXt2YXIgZT1pZy5zb3VyY2Uobikscj1hZy5zb3VyY2Uobik7ZnVuY3Rpb24gaSh0LG4pe3Jl
dHVybiB0PSt0LChuPStuKT49MT8oKT0+dDpuPD0wPygpPT4wOmZ1bmN0aW9uKCl7Zm9yKHZhciBp
PTAsbz10LGE9bjtvKmE+MTYmJm8qKDEtYSk+MTY7KXt2YXIgdT1NYXRoLmZsb29yKChvKzEpKmEp
LGM9cih1LG8tdSsxKSgpO2M8PWE/KGkrPXUsby09dSxhPShhLWMpLygxLWMpKToobz11LTEsYS89
Yyl9Zm9yKHZhciBmPWE8LjUscz1lKGY/YToxLWEpLGw9cygpLGg9MDtsPD1vOysraClsKz1zKCk7
cmV0dXJuIGkrKGY/aDpvLWgpfX1yZXR1cm4gaS5zb3VyY2U9dCxpfShWcCksY2c9ZnVuY3Rpb24g
dChuKXtmdW5jdGlvbiBlKHQsZSxyKXt2YXIgaTtyZXR1cm4gMD09KHQ9K3QpP2k9dD0+LU1hdGgu
bG9nKHQpOih0PTEvdCxpPW49Pk1hdGgucG93KG4sdCkpLGU9bnVsbD09ZT8wOitlLHI9bnVsbD09
cj8xOityLGZ1bmN0aW9uKCl7cmV0dXJuIGUrcippKC1NYXRoLmxvZzFwKC1uKCkpKX19cmV0dXJu
IGUuc291cmNlPXQsZX0oVnApLGZnPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0LGUpe3JldHVy
biB0PW51bGw9PXQ/MDordCxlPW51bGw9PWU/MTorZSxmdW5jdGlvbigpe3JldHVybiB0K2UqTWF0
aC50YW4oTWF0aC5QSSpuKCkpfX1yZXR1cm4gZS5zb3VyY2U9dCxlfShWcCksc2c9ZnVuY3Rpb24g
dChuKXtmdW5jdGlvbiBlKHQsZSl7cmV0dXJuIHQ9bnVsbD09dD8wOit0LGU9bnVsbD09ZT8xOitl
LGZ1bmN0aW9uKCl7dmFyIHI9bigpO3JldHVybiB0K2UqTWF0aC5sb2coci8oMS1yKSl9fXJldHVy
biBlLnNvdXJjZT10LGV9KFZwKSxsZz1mdW5jdGlvbiB0KG4pe3ZhciBlPW9nLnNvdXJjZShuKSxy
PXVnLnNvdXJjZShuKTtmdW5jdGlvbiBpKHQpe3JldHVybiBmdW5jdGlvbigpe2Zvcih2YXIgaT0w
LG89dDtvPjE2Oyl7dmFyIGE9TWF0aC5mbG9vciguODc1Km8pLHU9ZShhKSgpO2lmKHU+bylyZXR1
cm4gaStyKGEtMSxvL3UpKCk7aSs9YSxvLT11fWZvcih2YXIgYz0tTWF0aC5sb2cxcCgtbigpKSxm
PTA7Yzw9bzsrK2YpYy09TWF0aC5sb2cxcCgtbigpKTtyZXR1cm4gaStmfX1yZXR1cm4gaS5zb3Vy
Y2U9dCxpfShWcCk7Y29uc3QgaGc9MS80Mjk0OTY3Mjk2O2Z1bmN0aW9uIGRnKHQsbil7c3dpdGNo
KGFyZ3VtZW50cy5sZW5ndGgpe2Nhc2UgMDpicmVhaztjYXNlIDE6dGhpcy5yYW5nZSh0KTticmVh
aztkZWZhdWx0OnRoaXMucmFuZ2UobikuZG9tYWluKHQpfXJldHVybiB0aGlzfWZ1bmN0aW9uIHBn
KHQsbil7c3dpdGNoKGFyZ3VtZW50cy5sZW5ndGgpe2Nhc2UgMDpicmVhaztjYXNlIDE6ImZ1bmN0
aW9uIj09dHlwZW9mIHQ/dGhpcy5pbnRlcnBvbGF0b3IodCk6dGhpcy5yYW5nZSh0KTticmVhaztk
ZWZhdWx0OnRoaXMuZG9tYWluKHQpLCJmdW5jdGlvbiI9PXR5cGVvZiBuP3RoaXMuaW50ZXJwb2xh
dG9yKG4pOnRoaXMucmFuZ2Uobil9cmV0dXJuIHRoaXN9Y29uc3QgZ2c9U3ltYm9sKCJpbXBsaWNp
dCIpO2Z1bmN0aW9uIHlnKCl7dmFyIHQ9bmV3IEludGVybk1hcCxuPVtdLGU9W10scj1nZztmdW5j
dGlvbiBpKGkpe2xldCBvPXQuZ2V0KGkpO2lmKHZvaWQgMD09PW8pe2lmKHIhPT1nZylyZXR1cm4g
cjt0LnNldChpLG89bi5wdXNoKGkpLTEpfXJldHVybiBlW28lZS5sZW5ndGhdfXJldHVybiBpLmRv
bWFpbj1mdW5jdGlvbihlKXtpZighYXJndW1lbnRzLmxlbmd0aClyZXR1cm4gbi5zbGljZSgpO249
W10sdD1uZXcgSW50ZXJuTWFwO2Zvcihjb25zdCByIG9mIGUpdC5oYXMocil8fHQuc2V0KHIsbi5w
dXNoKHIpLTEpO3JldHVybiBpfSxpLnJhbmdlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPyhlPUFycmF5LmZyb20odCksaSk6ZS5zbGljZSgpfSxpLnVua25vd249ZnVuY3Rpb24o
dCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9dCxpKTpyfSxpLmNvcHk9ZnVuY3Rpb24oKXty
ZXR1cm4geWcobixlKS51bmtub3duKHIpfSxkZy5hcHBseShpLGFyZ3VtZW50cyksaX1mdW5jdGlv
biB2Zygpe3ZhciB0LG4sZT15ZygpLnVua25vd24odm9pZCAwKSxyPWUuZG9tYWluLGk9ZS5yYW5n
ZSxvPTAsYT0xLHU9ITEsYz0wLGY9MCxzPS41O2Z1bmN0aW9uIGwoKXt2YXIgZT1yKCkubGVuZ3Ro
LGw9YTxvLGg9bD9hOm8sZD1sP286YTt0PShkLWgpL01hdGgubWF4KDEsZS1jKzIqZiksdSYmKHQ9
TWF0aC5mbG9vcih0KSksaCs9KGQtaC10KihlLWMpKSpzLG49dCooMS1jKSx1JiYoaD1NYXRoLnJv
dW5kKGgpLG49TWF0aC5yb3VuZChuKSk7dmFyIHA9aHQoZSkubWFwKChmdW5jdGlvbihuKXtyZXR1
cm4gaCt0Km59KSk7cmV0dXJuIGkobD9wLnJldmVyc2UoKTpwKX1yZXR1cm4gZGVsZXRlIGUudW5r
bm93bixlLmRvbWFpbj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocih0KSxs
KCkpOnIoKX0sZS5yYW5nZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oW28s
YV09dCxvPStvLGE9K2EsbCgpKTpbbyxhXX0sZS5yYW5nZVJvdW5kPWZ1bmN0aW9uKHQpe3JldHVy
bltvLGFdPXQsbz0rbyxhPSthLHU9ITAsbCgpfSxlLmJhbmR3aWR0aD1mdW5jdGlvbigpe3JldHVy
biBufSxlLnN0ZXA9ZnVuY3Rpb24oKXtyZXR1cm4gdH0sZS5yb3VuZD1mdW5jdGlvbih0KXtyZXR1
cm4gYXJndW1lbnRzLmxlbmd0aD8odT0hIXQsbCgpKTp1fSxlLnBhZGRpbmc9ZnVuY3Rpb24odCl7
cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGM9TWF0aC5taW4oMSxmPSt0KSxsKCkpOmN9LGUucGFk
ZGluZ0lubmVyPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhjPU1hdGgubWlu
KDEsdCksbCgpKTpjfSxlLnBhZGRpbmdPdXRlcj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD8oZj0rdCxsKCkpOmZ9LGUuYWxpZ249ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50
cy5sZW5ndGg/KHM9TWF0aC5tYXgoMCxNYXRoLm1pbigxLHQpKSxsKCkpOnN9LGUuY29weT1mdW5j
dGlvbigpe3JldHVybiB2ZyhyKCksW28sYV0pLnJvdW5kKHUpLnBhZGRpbmdJbm5lcihjKS5wYWRk
aW5nT3V0ZXIoZikuYWxpZ24ocyl9LGRnLmFwcGx5KGwoKSxhcmd1bWVudHMpfWZ1bmN0aW9uIF9n
KHQpe3ZhciBuPXQuY29weTtyZXR1cm4gdC5wYWRkaW5nPXQucGFkZGluZ091dGVyLGRlbGV0ZSB0
LnBhZGRpbmdJbm5lcixkZWxldGUgdC5wYWRkaW5nT3V0ZXIsdC5jb3B5PWZ1bmN0aW9uKCl7cmV0
dXJuIF9nKG4oKSl9LHR9ZnVuY3Rpb24gYmcodCl7cmV0dXJuK3R9dmFyIG1nPVswLDFdO2Z1bmN0
aW9uIHhnKHQpe3JldHVybiB0fWZ1bmN0aW9uIHdnKHQsbil7cmV0dXJuKG4tPXQ9K3QpP2Z1bmN0
aW9uKGUpe3JldHVybihlLXQpL259OmZ1bmN0aW9uKHQpe3JldHVybiBmdW5jdGlvbigpe3JldHVy
biB0fX0oaXNOYU4obik/TmFOOi41KX1mdW5jdGlvbiBNZyh0LG4sZSl7dmFyIHI9dFswXSxpPXRb
MV0sbz1uWzBdLGE9blsxXTtyZXR1cm4gaTxyPyhyPXdnKGksciksbz1lKGEsbykpOihyPXdnKHIs
aSksbz1lKG8sYSkpLGZ1bmN0aW9uKHQpe3JldHVybiBvKHIodCkpfX1mdW5jdGlvbiBUZyh0LG4s
ZSl7dmFyIHI9TWF0aC5taW4odC5sZW5ndGgsbi5sZW5ndGgpLTEsaT1uZXcgQXJyYXkociksbz1u
ZXcgQXJyYXkociksYT0tMTtmb3IodFtyXTx0WzBdJiYodD10LnNsaWNlKCkucmV2ZXJzZSgpLG49
bi5zbGljZSgpLnJldmVyc2UoKSk7KythPHI7KWlbYV09d2codFthXSx0W2ErMV0pLG9bYV09ZShu
W2FdLG5bYSsxXSk7cmV0dXJuIGZ1bmN0aW9uKG4pe3ZhciBlPWwodCxuLDEsciktMTtyZXR1cm4g
b1tlXShpW2VdKG4pKX19ZnVuY3Rpb24gQWcodCxuKXtyZXR1cm4gbi5kb21haW4odC5kb21haW4o
KSkucmFuZ2UodC5yYW5nZSgpKS5pbnRlcnBvbGF0ZSh0LmludGVycG9sYXRlKCkpLmNsYW1wKHQu
Y2xhbXAoKSkudW5rbm93bih0LnVua25vd24oKSl9ZnVuY3Rpb24gU2coKXt2YXIgdCxuLGUscixp
LG8sYT1tZyx1PW1nLGM9VnIsZj14ZztmdW5jdGlvbiBzKCl7dmFyIHQ9TWF0aC5taW4oYS5sZW5n
dGgsdS5sZW5ndGgpO3JldHVybiBmIT09eGcmJihmPWZ1bmN0aW9uKHQsbil7dmFyIGU7cmV0dXJu
IHQ+biYmKGU9dCx0PW4sbj1lKSxmdW5jdGlvbihlKXtyZXR1cm4gTWF0aC5tYXgodCxNYXRoLm1p
bihuLGUpKX19KGFbMF0sYVt0LTFdKSkscj10PjI/VGc6TWcsaT1vPW51bGwsbH1mdW5jdGlvbiBs
KG4pe3JldHVybiBudWxsPT1ufHxpc05hTihuPStuKT9lOihpfHwoaT1yKGEubWFwKHQpLHUsYykp
KSh0KGYobikpKX1yZXR1cm4gbC5pbnZlcnQ9ZnVuY3Rpb24oZSl7cmV0dXJuIGYobigob3x8KG89
cih1LGEubWFwKHQpLExyKSkpKGUpKSl9LGwuZG9tYWluPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1
bWVudHMubGVuZ3RoPyhhPUFycmF5LmZyb20odCxiZykscygpKTphLnNsaWNlKCl9LGwucmFuZ2U9
ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHU9QXJyYXkuZnJvbSh0KSxzKCkp
OnUuc2xpY2UoKX0sbC5yYW5nZVJvdW5kPWZ1bmN0aW9uKHQpe3JldHVybiB1PUFycmF5LmZyb20o
dCksYz1XcixzKCl9LGwuY2xhbXA9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KGY9ISF0fHx4ZyxzKCkpOmYhPT14Z30sbC5pbnRlcnBvbGF0ZT1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8oYz10LHMoKSk6Y30sbC51bmtub3duPWZ1bmN0aW9uKHQpe3JldHVy
biBhcmd1bWVudHMubGVuZ3RoPyhlPXQsbCk6ZX0sZnVuY3Rpb24oZSxyKXtyZXR1cm4gdD1lLG49
cixzKCl9fWZ1bmN0aW9uIEVnKCl7cmV0dXJuIFNnKCkoeGcseGcpfWZ1bmN0aW9uIE5nKG4sZSxy
LGkpe3ZhciBvLGE9WihuLGUscik7c3dpdGNoKChpPXRmKG51bGw9PWk/IixmIjppKSkudHlwZSl7
Y2FzZSJzIjp2YXIgdT1NYXRoLm1heChNYXRoLmFicyhuKSxNYXRoLmFicyhlKSk7cmV0dXJuIG51
bGwhPWkucHJlY2lzaW9ufHxpc05hTihvPWhmKGEsdSkpfHwoaS5wcmVjaXNpb249byksdC5mb3Jt
YXRQcmVmaXgoaSx1KTtjYXNlIiI6Y2FzZSJlIjpjYXNlImciOmNhc2UicCI6Y2FzZSJyIjpudWxs
IT1pLnByZWNpc2lvbnx8aXNOYU4obz1kZihhLE1hdGgubWF4KE1hdGguYWJzKG4pLE1hdGguYWJz
KGUpKSkpfHwoaS5wcmVjaXNpb249by0oImUiPT09aS50eXBlKSk7YnJlYWs7Y2FzZSJmIjpjYXNl
IiUiOm51bGwhPWkucHJlY2lzaW9ufHxpc05hTihvPWxmKGEpKXx8KGkucHJlY2lzaW9uPW8tMioo
IiUiPT09aS50eXBlKSl9cmV0dXJuIHQuZm9ybWF0KGkpfWZ1bmN0aW9uIGtnKHQpe3ZhciBuPXQu
ZG9tYWluO3JldHVybiB0LnRpY2tzPWZ1bmN0aW9uKHQpe3ZhciBlPW4oKTtyZXR1cm4gVihlWzBd
LGVbZS5sZW5ndGgtMV0sbnVsbD09dD8xMDp0KX0sdC50aWNrRm9ybWF0PWZ1bmN0aW9uKHQsZSl7
dmFyIHI9bigpO3JldHVybiBOZyhyWzBdLHJbci5sZW5ndGgtMV0sbnVsbD09dD8xMDp0LGUpfSx0
Lm5pY2U9ZnVuY3Rpb24oZSl7bnVsbD09ZSYmKGU9MTApO3ZhciByLGksbz1uKCksYT0wLHU9by5s
ZW5ndGgtMSxjPW9bYV0sZj1vW3VdLHM9MTA7Zm9yKGY8YyYmKGk9YyxjPWYsZj1pLGk9YSxhPXUs
dT1pKTtzLS0gPjA7KXtpZigoaT1XKGMsZixlKSk9PT1yKXJldHVybiBvW2FdPWMsb1t1XT1mLG4o
byk7aWYoaT4wKWM9TWF0aC5mbG9vcihjL2kpKmksZj1NYXRoLmNlaWwoZi9pKSppO2Vsc2V7aWYo
IShpPDApKWJyZWFrO2M9TWF0aC5jZWlsKGMqaSkvaSxmPU1hdGguZmxvb3IoZippKS9pfXI9aX1y
ZXR1cm4gdH0sdH1mdW5jdGlvbiBDZyh0LG4pe3ZhciBlLHI9MCxpPSh0PXQuc2xpY2UoKSkubGVu
Z3RoLTEsbz10W3JdLGE9dFtpXTtyZXR1cm4gYTxvJiYoZT1yLHI9aSxpPWUsZT1vLG89YSxhPWUp
LHRbcl09bi5mbG9vcihvKSx0W2ldPW4uY2VpbChhKSx0fWZ1bmN0aW9uIFBnKHQpe3JldHVybiBN
YXRoLmxvZyh0KX1mdW5jdGlvbiB6Zyh0KXtyZXR1cm4gTWF0aC5leHAodCl9ZnVuY3Rpb24gJGco
dCl7cmV0dXJuLU1hdGgubG9nKC10KX1mdW5jdGlvbiBEZyh0KXtyZXR1cm4tTWF0aC5leHAoLXQp
fWZ1bmN0aW9uIFJnKHQpe3JldHVybiBpc0Zpbml0ZSh0KT8rKCIxZSIrdCk6dDwwPzA6dH1mdW5j
dGlvbiBGZyh0KXtyZXR1cm4obixlKT0+LXQoLW4sZSl9ZnVuY3Rpb24gcWcobil7Y29uc3QgZT1u
KFBnLHpnKSxyPWUuZG9tYWluO2xldCBpLG8sYT0xMDtmdW5jdGlvbiB1KCl7cmV0dXJuIGk9ZnVu
Y3Rpb24odCl7cmV0dXJuIHQ9PT1NYXRoLkU/TWF0aC5sb2c6MTA9PT10JiZNYXRoLmxvZzEwfHwy
PT09dCYmTWF0aC5sb2cyfHwodD1NYXRoLmxvZyh0KSxuPT5NYXRoLmxvZyhuKS90KX0oYSksbz1m
dW5jdGlvbih0KXtyZXR1cm4gMTA9PT10P1JnOnQ9PT1NYXRoLkU/TWF0aC5leHA6bj0+TWF0aC5w
b3codCxuKX0oYSkscigpWzBdPDA/KGk9RmcoaSksbz1GZyhvKSxuKCRnLERnKSk6bihQZyx6Zyks
ZX1yZXR1cm4gZS5iYXNlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhhPSt0
LHUoKSk6YX0sZS5kb21haW49ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHIo
dCksdSgpKTpyKCl9LGUudGlja3M9dD0+e2NvbnN0IG49cigpO2xldCBlPW5bMF0sdT1uW24ubGVu
Z3RoLTFdO2NvbnN0IGM9dTxlO2MmJihbZSx1XT1bdSxlXSk7bGV0IGYscyxsPWkoZSksaD1pKHUp
O2NvbnN0IGQ9bnVsbD09dD8xMDordDtsZXQgcD1bXTtpZighKGElMSkmJmgtbDxkKXtpZihsPU1h
dGguZmxvb3IobCksaD1NYXRoLmNlaWwoaCksZT4wKXtmb3IoO2w8PWg7KytsKWZvcihmPTE7Zjxh
OysrZilpZihzPWw8MD9mL28oLWwpOmYqbyhsKSwhKHM8ZSkpe2lmKHM+dSlicmVhaztwLnB1c2go
cyl9fWVsc2UgZm9yKDtsPD1oOysrbClmb3IoZj1hLTE7Zj49MTstLWYpaWYocz1sPjA/Zi9vKC1s
KTpmKm8obCksIShzPGUpKXtpZihzPnUpYnJlYWs7cC5wdXNoKHMpfTIqcC5sZW5ndGg8ZCYmKHA9
VihlLHUsZCkpfWVsc2UgcD1WKGwsaCxNYXRoLm1pbihoLWwsZCkpLm1hcChvKTtyZXR1cm4gYz9w
LnJldmVyc2UoKTpwfSxlLnRpY2tGb3JtYXQ9KG4scik9PntpZihudWxsPT1uJiYobj0xMCksbnVs
bD09ciYmKHI9MTA9PT1hPyJzIjoiLCIpLCJmdW5jdGlvbiIhPXR5cGVvZiByJiYoYSUxfHxudWxs
IT0ocj10ZihyKSkucHJlY2lzaW9ufHwoci50cmltPSEwKSxyPXQuZm9ybWF0KHIpKSxuPT09MS8w
KXJldHVybiByO2NvbnN0IHU9TWF0aC5tYXgoMSxhKm4vZS50aWNrcygpLmxlbmd0aCk7cmV0dXJu
IHQ9PntsZXQgbj10L28oTWF0aC5yb3VuZChpKHQpKSk7cmV0dXJuIG4qYTxhLS41JiYobio9YSks
bjw9dT9yKHQpOiIifX0sZS5uaWNlPSgpPT5yKENnKHIoKSx7Zmxvb3I6dD0+byhNYXRoLmZsb29y
KGkodCkpKSxjZWlsOnQ9Pm8oTWF0aC5jZWlsKGkodCkpKX0pKSxlfWZ1bmN0aW9uIFVnKHQpe3Jl
dHVybiBmdW5jdGlvbihuKXtyZXR1cm4gTWF0aC5zaWduKG4pKk1hdGgubG9nMXAoTWF0aC5hYnMo
bi90KSl9fWZ1bmN0aW9uIElnKHQpe3JldHVybiBmdW5jdGlvbihuKXtyZXR1cm4gTWF0aC5zaWdu
KG4pKk1hdGguZXhwbTEoTWF0aC5hYnMobikpKnR9fWZ1bmN0aW9uIE9nKHQpe3ZhciBuPTEsZT10
KFVnKG4pLElnKG4pKTtyZXR1cm4gZS5jb25zdGFudD1mdW5jdGlvbihlKXtyZXR1cm4gYXJndW1l
bnRzLmxlbmd0aD90KFVnKG49K2UpLElnKG4pKTpufSxrZyhlKX1mdW5jdGlvbiBCZyh0KXtyZXR1
cm4gZnVuY3Rpb24obil7cmV0dXJuIG48MD8tTWF0aC5wb3coLW4sdCk6TWF0aC5wb3cobix0KX19
ZnVuY3Rpb24gWWcodCl7cmV0dXJuIHQ8MD8tTWF0aC5zcXJ0KC10KTpNYXRoLnNxcnQodCl9ZnVu
Y3Rpb24gTGcodCl7cmV0dXJuIHQ8MD8tdCp0OnQqdH1mdW5jdGlvbiBqZyh0KXt2YXIgbj10KHhn
LHhnKSxlPTE7cmV0dXJuIG4uZXhwb25lbnQ9ZnVuY3Rpb24obil7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/MT09PShlPStuKT90KHhnLHhnKTouNT09PWU/dChZZyxMZyk6dChCZyhlKSxCZygxL2Up
KTplfSxrZyhuKX1mdW5jdGlvbiBIZygpe3ZhciB0PWpnKFNnKCkpO3JldHVybiB0LmNvcHk9ZnVu
Y3Rpb24oKXtyZXR1cm4gQWcodCxIZygpKS5leHBvbmVudCh0LmV4cG9uZW50KCkpfSxkZy5hcHBs
eSh0LGFyZ3VtZW50cyksdH1mdW5jdGlvbiBYZyh0KXtyZXR1cm4gTWF0aC5zaWduKHQpKnQqdH1j
b25zdCBHZz1uZXcgRGF0ZSxWZz1uZXcgRGF0ZTtmdW5jdGlvbiBXZyh0LG4sZSxyKXtmdW5jdGlv
biBpKG4pe3JldHVybiB0KG49MD09PWFyZ3VtZW50cy5sZW5ndGg/bmV3IERhdGU6bmV3IERhdGUo
K24pKSxufXJldHVybiBpLmZsb29yPW49Pih0KG49bmV3IERhdGUoK24pKSxuKSxpLmNlaWw9ZT0+
KHQoZT1uZXcgRGF0ZShlLTEpKSxuKGUsMSksdChlKSxlKSxpLnJvdW5kPXQ9Pntjb25zdCBuPWko
dCksZT1pLmNlaWwodCk7cmV0dXJuIHQtbjxlLXQ/bjplfSxpLm9mZnNldD0odCxlKT0+KG4odD1u
ZXcgRGF0ZSgrdCksbnVsbD09ZT8xOk1hdGguZmxvb3IoZSkpLHQpLGkucmFuZ2U9KGUscixvKT0+
e2NvbnN0IGE9W107aWYoZT1pLmNlaWwoZSksbz1udWxsPT1vPzE6TWF0aC5mbG9vcihvKSwhKGU8
ciYmbz4wKSlyZXR1cm4gYTtsZXQgdTtkb3thLnB1c2godT1uZXcgRGF0ZSgrZSkpLG4oZSxvKSx0
KGUpfXdoaWxlKHU8ZSYmZTxyKTtyZXR1cm4gYX0saS5maWx0ZXI9ZT0+V2coKG49PntpZihuPj1u
KWZvcig7dChuKSwhZShuKTspbi5zZXRUaW1lKG4tMSl9KSwoKHQscik9PntpZih0Pj10KWlmKHI8
MClmb3IoOysrcjw9MDspZm9yKDtuKHQsLTEpLCFlKHQpOyk7ZWxzZSBmb3IoOy0tcj49MDspZm9y
KDtuKHQsMSksIWUodCk7KTt9KSksZSYmKGkuY291bnQ9KG4scik9PihHZy5zZXRUaW1lKCtuKSxW
Zy5zZXRUaW1lKCtyKSx0KEdnKSx0KFZnKSxNYXRoLmZsb29yKGUoR2csVmcpKSksaS5ldmVyeT10
PT4odD1NYXRoLmZsb29yKHQpLGlzRmluaXRlKHQpJiZ0PjA/dD4xP2kuZmlsdGVyKHI/bj0+cihu
KSV0PT0wOm49PmkuY291bnQoMCxuKSV0PT0wKTppOm51bGwpKSxpfWNvbnN0IFpnPVdnKCgoKT0+
e30pLCgodCxuKT0+e3Quc2V0VGltZSgrdCtuKX0pLCgodCxuKT0+bi10KSk7WmcuZXZlcnk9dD0+
KHQ9TWF0aC5mbG9vcih0KSxpc0Zpbml0ZSh0KSYmdD4wP3Q+MT9XZygobj0+e24uc2V0VGltZShN
YXRoLmZsb29yKG4vdCkqdCl9KSwoKG4sZSk9PntuLnNldFRpbWUoK24rZSp0KX0pLCgobixlKT0+
KGUtbikvdCkpOlpnOm51bGwpO2NvbnN0IEtnPVpnLnJhbmdlLFFnPTFlMyxKZz02ZTQsdHk9NjAq
Smcsbnk9MjQqdHksZXk9NypueSxyeT0zMCpueSxpeT0zNjUqbnksb3k9V2coKHQ9Pnt0LnNldFRp
bWUodC10LmdldE1pbGxpc2Vjb25kcygpKX0pLCgodCxuKT0+e3Quc2V0VGltZSgrdCtuKlFnKX0p
LCgodCxuKT0+KG4tdCkvUWcpLCh0PT50LmdldFVUQ1NlY29uZHMoKSkpLGF5PW95LnJhbmdlLHV5
PVdnKCh0PT57dC5zZXRUaW1lKHQtdC5nZXRNaWxsaXNlY29uZHMoKS10LmdldFNlY29uZHMoKSpR
Zyl9KSwoKHQsbik9Pnt0LnNldFRpbWUoK3QrbipKZyl9KSwoKHQsbik9PihuLXQpL0pnKSwodD0+
dC5nZXRNaW51dGVzKCkpKSxjeT11eS5yYW5nZSxmeT1XZygodD0+e3Quc2V0VVRDU2Vjb25kcygw
LDApfSksKCh0LG4pPT57dC5zZXRUaW1lKCt0K24qSmcpfSksKCh0LG4pPT4obi10KS9KZyksKHQ9
PnQuZ2V0VVRDTWludXRlcygpKSksc3k9ZnkucmFuZ2UsbHk9V2coKHQ9Pnt0LnNldFRpbWUodC10
LmdldE1pbGxpc2Vjb25kcygpLXQuZ2V0U2Vjb25kcygpKlFnLXQuZ2V0TWludXRlcygpKkpnKX0p
LCgodCxuKT0+e3Quc2V0VGltZSgrdCtuKnR5KX0pLCgodCxuKT0+KG4tdCkvdHkpLCh0PT50Lmdl
dEhvdXJzKCkpKSxoeT1seS5yYW5nZSxkeT1XZygodD0+e3Quc2V0VVRDTWludXRlcygwLDAsMCl9
KSwoKHQsbik9Pnt0LnNldFRpbWUoK3Qrbip0eSl9KSwoKHQsbik9PihuLXQpL3R5KSwodD0+dC5n
ZXRVVENIb3VycygpKSkscHk9ZHkucmFuZ2UsZ3k9V2coKHQ9PnQuc2V0SG91cnMoMCwwLDAsMCkp
LCgodCxuKT0+dC5zZXREYXRlKHQuZ2V0RGF0ZSgpK24pKSwoKHQsbik9PihuLXQtKG4uZ2V0VGlt
ZXpvbmVPZmZzZXQoKS10LmdldFRpbWV6b25lT2Zmc2V0KCkpKkpnKS9ueSksKHQ9PnQuZ2V0RGF0
ZSgpLTEpKSx5eT1neS5yYW5nZSx2eT1XZygodD0+e3Quc2V0VVRDSG91cnMoMCwwLDAsMCl9KSwo
KHQsbik9Pnt0LnNldFVUQ0RhdGUodC5nZXRVVENEYXRlKCkrbil9KSwoKHQsbik9PihuLXQpL255
KSwodD0+dC5nZXRVVENEYXRlKCktMSkpLF95PXZ5LnJhbmdlLGJ5PVdnKCh0PT57dC5zZXRVVENI
b3VycygwLDAsMCwwKX0pLCgodCxuKT0+e3Quc2V0VVRDRGF0ZSh0LmdldFVUQ0RhdGUoKStuKX0p
LCgodCxuKT0+KG4tdCkvbnkpLCh0PT5NYXRoLmZsb29yKHQvbnkpKSksbXk9YnkucmFuZ2U7ZnVu
Y3Rpb24geHkodCl7cmV0dXJuIFdnKChuPT57bi5zZXREYXRlKG4uZ2V0RGF0ZSgpLShuLmdldERh
eSgpKzctdCklNyksbi5zZXRIb3VycygwLDAsMCwwKX0pLCgodCxuKT0+e3Quc2V0RGF0ZSh0Lmdl
dERhdGUoKSs3Km4pfSksKCh0LG4pPT4obi10LShuLmdldFRpbWV6b25lT2Zmc2V0KCktdC5nZXRU
aW1lem9uZU9mZnNldCgpKSpKZykvZXkpKX1jb25zdCB3eT14eSgwKSxNeT14eSgxKSxUeT14eSgy
KSxBeT14eSgzKSxTeT14eSg0KSxFeT14eSg1KSxOeT14eSg2KSxreT13eS5yYW5nZSxDeT1NeS5y
YW5nZSxQeT1UeS5yYW5nZSx6eT1BeS5yYW5nZSwkeT1TeS5yYW5nZSxEeT1FeS5yYW5nZSxSeT1O
eS5yYW5nZTtmdW5jdGlvbiBGeSh0KXtyZXR1cm4gV2coKG49PntuLnNldFVUQ0RhdGUobi5nZXRV
VENEYXRlKCktKG4uZ2V0VVRDRGF5KCkrNy10KSU3KSxuLnNldFVUQ0hvdXJzKDAsMCwwLDApfSks
KCh0LG4pPT57dC5zZXRVVENEYXRlKHQuZ2V0VVRDRGF0ZSgpKzcqbil9KSwoKHQsbik9PihuLXQp
L2V5KSl9Y29uc3QgcXk9RnkoMCksVXk9RnkoMSksSXk9RnkoMiksT3k9RnkoMyksQnk9RnkoNCks
WXk9RnkoNSksTHk9RnkoNiksank9cXkucmFuZ2UsSHk9VXkucmFuZ2UsWHk9SXkucmFuZ2UsR3k9
T3kucmFuZ2UsVnk9QnkucmFuZ2UsV3k9WXkucmFuZ2UsWnk9THkucmFuZ2UsS3k9V2coKHQ9Pnt0
LnNldERhdGUoMSksdC5zZXRIb3VycygwLDAsMCwwKX0pLCgodCxuKT0+e3Quc2V0TW9udGgodC5n
ZXRNb250aCgpK24pfSksKCh0LG4pPT5uLmdldE1vbnRoKCktdC5nZXRNb250aCgpKzEyKihuLmdl
dEZ1bGxZZWFyKCktdC5nZXRGdWxsWWVhcigpKSksKHQ9PnQuZ2V0TW9udGgoKSkpLFF5PUt5LnJh
bmdlLEp5PVdnKCh0PT57dC5zZXRVVENEYXRlKDEpLHQuc2V0VVRDSG91cnMoMCwwLDAsMCl9KSwo
KHQsbik9Pnt0LnNldFVUQ01vbnRoKHQuZ2V0VVRDTW9udGgoKStuKX0pLCgodCxuKT0+bi5nZXRV
VENNb250aCgpLXQuZ2V0VVRDTW9udGgoKSsxMioobi5nZXRVVENGdWxsWWVhcigpLXQuZ2V0VVRD
RnVsbFllYXIoKSkpLCh0PT50LmdldFVUQ01vbnRoKCkpKSx0dj1KeS5yYW5nZSxudj1XZygodD0+
e3Quc2V0TW9udGgoMCwxKSx0LnNldEhvdXJzKDAsMCwwLDApfSksKCh0LG4pPT57dC5zZXRGdWxs
WWVhcih0LmdldEZ1bGxZZWFyKCkrbil9KSwoKHQsbik9Pm4uZ2V0RnVsbFllYXIoKS10LmdldEZ1
bGxZZWFyKCkpLCh0PT50LmdldEZ1bGxZZWFyKCkpKTtudi5ldmVyeT10PT5pc0Zpbml0ZSh0PU1h
dGguZmxvb3IodCkpJiZ0PjA/V2coKG49PntuLnNldEZ1bGxZZWFyKE1hdGguZmxvb3Iobi5nZXRG
dWxsWWVhcigpL3QpKnQpLG4uc2V0TW9udGgoMCwxKSxuLnNldEhvdXJzKDAsMCwwLDApfSksKChu
LGUpPT57bi5zZXRGdWxsWWVhcihuLmdldEZ1bGxZZWFyKCkrZSp0KX0pKTpudWxsO2NvbnN0IGV2
PW52LnJhbmdlLHJ2PVdnKCh0PT57dC5zZXRVVENNb250aCgwLDEpLHQuc2V0VVRDSG91cnMoMCww
LDAsMCl9KSwoKHQsbik9Pnt0LnNldFVUQ0Z1bGxZZWFyKHQuZ2V0VVRDRnVsbFllYXIoKStuKX0p
LCgodCxuKT0+bi5nZXRVVENGdWxsWWVhcigpLXQuZ2V0VVRDRnVsbFllYXIoKSksKHQ9PnQuZ2V0
VVRDRnVsbFllYXIoKSkpO3J2LmV2ZXJ5PXQ9PmlzRmluaXRlKHQ9TWF0aC5mbG9vcih0KSkmJnQ+
MD9XZygobj0+e24uc2V0VVRDRnVsbFllYXIoTWF0aC5mbG9vcihuLmdldFVUQ0Z1bGxZZWFyKCkv
dCkqdCksbi5zZXRVVENNb250aCgwLDEpLG4uc2V0VVRDSG91cnMoMCwwLDAsMCl9KSwoKG4sZSk9
PntuLnNldFVUQ0Z1bGxZZWFyKG4uZ2V0VVRDRnVsbFllYXIoKStlKnQpfSkpOm51bGw7Y29uc3Qg
aXY9cnYucmFuZ2U7ZnVuY3Rpb24gb3YodCxuLGUsaSxvLGEpe2NvbnN0IHU9W1tveSwxLFFnXSxb
b3ksNSw1ZTNdLFtveSwxNSwxNWUzXSxbb3ksMzAsM2U0XSxbYSwxLEpnXSxbYSw1LDUqSmddLFth
LDE1LDE1KkpnXSxbYSwzMCwzMCpKZ10sW28sMSx0eV0sW28sMywzKnR5XSxbbyw2LDYqdHldLFtv
LDEyLDEyKnR5XSxbaSwxLG55XSxbaSwyLDIqbnldLFtlLDEsZXldLFtuLDEscnldLFtuLDMsMypy
eV0sW3QsMSxpeV1dO2Z1bmN0aW9uIGMobixlLGkpe2NvbnN0IG89TWF0aC5hYnMoZS1uKS9pLGE9
cigoKFssLHRdKT0+dCkpLnJpZ2h0KHUsbyk7aWYoYT09PXUubGVuZ3RoKXJldHVybiB0LmV2ZXJ5
KFoobi9peSxlL2l5LGkpKTtpZigwPT09YSlyZXR1cm4gWmcuZXZlcnkoTWF0aC5tYXgoWihuLGUs
aSksMSkpO2NvbnN0W2MsZl09dVtvL3VbYS0xXVsyXTx1W2FdWzJdL28/YS0xOmFdO3JldHVybiBj
LmV2ZXJ5KGYpfXJldHVybltmdW5jdGlvbih0LG4sZSl7Y29uc3Qgcj1uPHQ7ciYmKFt0LG5dPVtu
LHRdKTtjb25zdCBpPWUmJiJmdW5jdGlvbiI9PXR5cGVvZiBlLnJhbmdlP2U6Yyh0LG4sZSksbz1p
P2kucmFuZ2UodCwrbisxKTpbXTtyZXR1cm4gcj9vLnJldmVyc2UoKTpvfSxjXX1jb25zdFthdix1
dl09b3YocnYsSnkscXksYnksZHksZnkpLFtjdixmdl09b3YobnYsS3ksd3ksZ3ksbHksdXkpO2Z1
bmN0aW9uIHN2KHQpe2lmKDA8PXQueSYmdC55PDEwMCl7dmFyIG49bmV3IERhdGUoLTEsdC5tLHQu
ZCx0LkgsdC5NLHQuUyx0LkwpO3JldHVybiBuLnNldEZ1bGxZZWFyKHQueSksbn1yZXR1cm4gbmV3
IERhdGUodC55LHQubSx0LmQsdC5ILHQuTSx0LlMsdC5MKX1mdW5jdGlvbiBsdih0KXtpZigwPD10
LnkmJnQueTwxMDApe3ZhciBuPW5ldyBEYXRlKERhdGUuVVRDKC0xLHQubSx0LmQsdC5ILHQuTSx0
LlMsdC5MKSk7cmV0dXJuIG4uc2V0VVRDRnVsbFllYXIodC55KSxufXJldHVybiBuZXcgRGF0ZShE
YXRlLlVUQyh0LnksdC5tLHQuZCx0LkgsdC5NLHQuUyx0LkwpKX1mdW5jdGlvbiBodih0LG4sZSl7
cmV0dXJue3k6dCxtOm4sZDplLEg6MCxNOjAsUzowLEw6MH19ZnVuY3Rpb24gZHYodCl7dmFyIG49
dC5kYXRlVGltZSxlPXQuZGF0ZSxyPXQudGltZSxpPXQucGVyaW9kcyxvPXQuZGF5cyxhPXQuc2hv
cnREYXlzLHU9dC5tb250aHMsYz10LnNob3J0TW9udGhzLGY9eHYoaSkscz13dihpKSxsPXh2KG8p
LGg9d3YobyksZD14dihhKSxwPXd2KGEpLGc9eHYodSkseT13dih1KSx2PXh2KGMpLF89d3YoYyks
Yj17YTpmdW5jdGlvbih0KXtyZXR1cm4gYVt0LmdldERheSgpXX0sQTpmdW5jdGlvbih0KXtyZXR1
cm4gb1t0LmdldERheSgpXX0sYjpmdW5jdGlvbih0KXtyZXR1cm4gY1t0LmdldE1vbnRoKCldfSxC
OmZ1bmN0aW9uKHQpe3JldHVybiB1W3QuZ2V0TW9udGgoKV19LGM6bnVsbCxkOkx2LGU6THYsZjpW
dixnOm9fLEc6dV8sSDpqdixJOkh2LGo6WHYsTDpHdixtOld2LE06WnYscDpmdW5jdGlvbih0KXty
ZXR1cm4gaVsrKHQuZ2V0SG91cnMoKT49MTIpXX0scTpmdW5jdGlvbih0KXtyZXR1cm4gMSt+fih0
LmdldE1vbnRoKCkvMyl9LFE6Q18sczpQXyxTOkt2LHU6UXYsVTpKdixWOm5fLHc6ZV8sVzpyXyx4
Om51bGwsWDpudWxsLHk6aV8sWTphXyxaOmNfLCIlIjprX30sbT17YTpmdW5jdGlvbih0KXtyZXR1
cm4gYVt0LmdldFVUQ0RheSgpXX0sQTpmdW5jdGlvbih0KXtyZXR1cm4gb1t0LmdldFVUQ0RheSgp
XX0sYjpmdW5jdGlvbih0KXtyZXR1cm4gY1t0LmdldFVUQ01vbnRoKCldfSxCOmZ1bmN0aW9uKHQp
e3JldHVybiB1W3QuZ2V0VVRDTW9udGgoKV19LGM6bnVsbCxkOmZfLGU6Zl8sZjpwXyxnOkFfLEc6
RV8sSDpzXyxJOmxfLGo6aF8sTDpkXyxtOmdfLE06eV8scDpmdW5jdGlvbih0KXtyZXR1cm4gaVsr
KHQuZ2V0VVRDSG91cnMoKT49MTIpXX0scTpmdW5jdGlvbih0KXtyZXR1cm4gMSt+fih0LmdldFVU
Q01vbnRoKCkvMyl9LFE6Q18sczpQXyxTOnZfLHU6X18sVTpiXyxWOnhfLHc6d18sVzpNXyx4Om51
bGwsWDpudWxsLHk6VF8sWTpTXyxaOk5fLCIlIjprX30seD17YTpmdW5jdGlvbih0LG4sZSl7dmFy
IHI9ZC5leGVjKG4uc2xpY2UoZSkpO3JldHVybiByPyh0Lnc9cC5nZXQoclswXS50b0xvd2VyQ2Fz
ZSgpKSxlK3JbMF0ubGVuZ3RoKTotMX0sQTpmdW5jdGlvbih0LG4sZSl7dmFyIHI9bC5leGVjKG4u
c2xpY2UoZSkpO3JldHVybiByPyh0Lnc9aC5nZXQoclswXS50b0xvd2VyQ2FzZSgpKSxlK3JbMF0u
bGVuZ3RoKTotMX0sYjpmdW5jdGlvbih0LG4sZSl7dmFyIHI9di5leGVjKG4uc2xpY2UoZSkpO3Jl
dHVybiByPyh0Lm09Xy5nZXQoclswXS50b0xvd2VyQ2FzZSgpKSxlK3JbMF0ubGVuZ3RoKTotMX0s
QjpmdW5jdGlvbih0LG4sZSl7dmFyIHI9Zy5leGVjKG4uc2xpY2UoZSkpO3JldHVybiByPyh0Lm09
eS5nZXQoclswXS50b0xvd2VyQ2FzZSgpKSxlK3JbMF0ubGVuZ3RoKTotMX0sYzpmdW5jdGlvbih0
LGUscil7cmV0dXJuIFQodCxuLGUscil9LGQ6JHYsZTokdixmOkl2LGc6a3YsRzpOdixIOlJ2LEk6
UnYsajpEdixMOlV2LG06enYsTTpGdixwOmZ1bmN0aW9uKHQsbixlKXt2YXIgcj1mLmV4ZWMobi5z
bGljZShlKSk7cmV0dXJuIHI/KHQucD1zLmdldChyWzBdLnRvTG93ZXJDYXNlKCkpLGUrclswXS5s
ZW5ndGgpOi0xfSxxOlB2LFE6QnYsczpZdixTOnF2LHU6VHYsVTpBdixWOlN2LHc6TXYsVzpFdix4
OmZ1bmN0aW9uKHQsbixyKXtyZXR1cm4gVCh0LGUsbixyKX0sWDpmdW5jdGlvbih0LG4sZSl7cmV0
dXJuIFQodCxyLG4sZSl9LHk6a3YsWTpOdixaOkN2LCIlIjpPdn07ZnVuY3Rpb24gdyh0LG4pe3Jl
dHVybiBmdW5jdGlvbihlKXt2YXIgcixpLG8sYT1bXSx1PS0xLGM9MCxmPXQubGVuZ3RoO2Zvcihl
IGluc3RhbmNlb2YgRGF0ZXx8KGU9bmV3IERhdGUoK2UpKTsrK3U8ZjspMzc9PT10LmNoYXJDb2Rl
QXQodSkmJihhLnB1c2godC5zbGljZShjLHUpKSxudWxsIT0oaT1ndltyPXQuY2hhckF0KCsrdSld
KT9yPXQuY2hhckF0KCsrdSk6aT0iZSI9PT1yPyIgIjoiMCIsKG89bltyXSkmJihyPW8oZSxpKSks
YS5wdXNoKHIpLGM9dSsxKTtyZXR1cm4gYS5wdXNoKHQuc2xpY2UoYyx1KSksYS5qb2luKCIiKX19
ZnVuY3Rpb24gTSh0LG4pe3JldHVybiBmdW5jdGlvbihlKXt2YXIgcixpLG89aHYoMTkwMCx2b2lk
IDAsMSk7aWYoVChvLHQsZSs9IiIsMCkhPWUubGVuZ3RoKXJldHVybiBudWxsO2lmKCJRImluIG8p
cmV0dXJuIG5ldyBEYXRlKG8uUSk7aWYoInMiaW4gbylyZXR1cm4gbmV3IERhdGUoMWUzKm8ucyso
IkwiaW4gbz9vLkw6MCkpO2lmKG4mJiEoIloiaW4gbykmJihvLlo9MCksInAiaW4gbyYmKG8uSD1v
LkglMTIrMTIqby5wKSx2b2lkIDA9PT1vLm0mJihvLm09InEiaW4gbz9vLnE6MCksIlYiaW4gbyl7
aWYoby5WPDF8fG8uVj41MylyZXR1cm4gbnVsbDsidyJpbiBvfHwoby53PTEpLCJaImluIG8/KGk9
KHI9bHYoaHYoby55LDAsMSkpKS5nZXRVVENEYXkoKSxyPWk+NHx8MD09PWk/VXkuY2VpbChyKTpV
eShyKSxyPXZ5Lm9mZnNldChyLDcqKG8uVi0xKSksby55PXIuZ2V0VVRDRnVsbFllYXIoKSxvLm09
ci5nZXRVVENNb250aCgpLG8uZD1yLmdldFVUQ0RhdGUoKSsoby53KzYpJTcpOihpPShyPXN2KGh2
KG8ueSwwLDEpKSkuZ2V0RGF5KCkscj1pPjR8fDA9PT1pP015LmNlaWwocik6TXkocikscj1neS5v
ZmZzZXQociw3KihvLlYtMSkpLG8ueT1yLmdldEZ1bGxZZWFyKCksby5tPXIuZ2V0TW9udGgoKSxv
LmQ9ci5nZXREYXRlKCkrKG8udys2KSU3KX1lbHNlKCJXImluIG98fCJVImluIG8pJiYoInciaW4g
b3x8KG8udz0idSJpbiBvP28udSU3OiJXImluIG8/MTowKSxpPSJaImluIG8/bHYoaHYoby55LDAs
MSkpLmdldFVUQ0RheSgpOnN2KGh2KG8ueSwwLDEpKS5nZXREYXkoKSxvLm09MCxvLmQ9IlciaW4g
bz8oby53KzYpJTcrNypvLlctKGkrNSklNzpvLncrNypvLlUtKGkrNiklNyk7cmV0dXJuIloiaW4g
bz8oby5IKz1vLlovMTAwfDAsby5NKz1vLlolMTAwLGx2KG8pKTpzdihvKX19ZnVuY3Rpb24gVCh0
LG4sZSxyKXtmb3IodmFyIGksbyxhPTAsdT1uLmxlbmd0aCxjPWUubGVuZ3RoO2E8dTspe2lmKHI+
PWMpcmV0dXJuLTE7aWYoMzc9PT0oaT1uLmNoYXJDb2RlQXQoYSsrKSkpe2lmKGk9bi5jaGFyQXQo
YSsrKSwhKG89eFtpIGluIGd2P24uY2hhckF0KGErKyk6aV0pfHwocj1vKHQsZSxyKSk8MClyZXR1
cm4tMX1lbHNlIGlmKGkhPWUuY2hhckNvZGVBdChyKyspKXJldHVybi0xfXJldHVybiByfXJldHVy
biBiLng9dyhlLGIpLGIuWD13KHIsYiksYi5jPXcobixiKSxtLng9dyhlLG0pLG0uWD13KHIsbSks
bS5jPXcobixtKSx7Zm9ybWF0OmZ1bmN0aW9uKHQpe3ZhciBuPXcodCs9IiIsYik7cmV0dXJuIG4u
dG9TdHJpbmc9ZnVuY3Rpb24oKXtyZXR1cm4gdH0sbn0scGFyc2U6ZnVuY3Rpb24odCl7dmFyIG49
TSh0Kz0iIiwhMSk7cmV0dXJuIG4udG9TdHJpbmc9ZnVuY3Rpb24oKXtyZXR1cm4gdH0sbn0sdXRj
Rm9ybWF0OmZ1bmN0aW9uKHQpe3ZhciBuPXcodCs9IiIsbSk7cmV0dXJuIG4udG9TdHJpbmc9ZnVu
Y3Rpb24oKXtyZXR1cm4gdH0sbn0sdXRjUGFyc2U6ZnVuY3Rpb24odCl7dmFyIG49TSh0Kz0iIiwh
MCk7cmV0dXJuIG4udG9TdHJpbmc9ZnVuY3Rpb24oKXtyZXR1cm4gdH0sbn19fXZhciBwdixndj17
Ii0iOiIiLF86IiAiLDA6IjAifSx5dj0vXlxzKlxkKy8sdnY9L14lLyxfdj0vW1xcXiQqKz98W1xd
KCkue31dL2c7ZnVuY3Rpb24gYnYodCxuLGUpe3ZhciByPXQ8MD8iLSI6IiIsaT0ocj8tdDp0KSsi
IixvPWkubGVuZ3RoO3JldHVybiByKyhvPGU/bmV3IEFycmF5KGUtbysxKS5qb2luKG4pK2k6aSl9
ZnVuY3Rpb24gbXYodCl7cmV0dXJuIHQucmVwbGFjZShfdiwiXFwkJiIpfWZ1bmN0aW9uIHh2KHQp
e3JldHVybiBuZXcgUmVnRXhwKCJeKD86Iit0Lm1hcChtdikuam9pbigifCIpKyIpIiwiaSIpfWZ1
bmN0aW9uIHd2KHQpe3JldHVybiBuZXcgTWFwKHQubWFwKCgodCxuKT0+W3QudG9Mb3dlckNhc2Uo
KSxuXSkpKX1mdW5jdGlvbiBNdih0LG4sZSl7dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUsZSsxKSk7
cmV0dXJuIHI/KHQudz0rclswXSxlK3JbMF0ubGVuZ3RoKTotMX1mdW5jdGlvbiBUdih0LG4sZSl7
dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUsZSsxKSk7cmV0dXJuIHI/KHQudT0rclswXSxlK3JbMF0u
bGVuZ3RoKTotMX1mdW5jdGlvbiBBdih0LG4sZSl7dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUsZSsy
KSk7cmV0dXJuIHI/KHQuVT0rclswXSxlK3JbMF0ubGVuZ3RoKTotMX1mdW5jdGlvbiBTdih0LG4s
ZSl7dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUsZSsyKSk7cmV0dXJuIHI/KHQuVj0rclswXSxlK3Jb
MF0ubGVuZ3RoKTotMX1mdW5jdGlvbiBFdih0LG4sZSl7dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUs
ZSsyKSk7cmV0dXJuIHI/KHQuVz0rclswXSxlK3JbMF0ubGVuZ3RoKTotMX1mdW5jdGlvbiBOdih0
LG4sZSl7dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUsZSs0KSk7cmV0dXJuIHI/KHQueT0rclswXSxl
K3JbMF0ubGVuZ3RoKTotMX1mdW5jdGlvbiBrdih0LG4sZSl7dmFyIHI9eXYuZXhlYyhuLnNsaWNl
KGUsZSsyKSk7cmV0dXJuIHI/KHQueT0rclswXSsoK3JbMF0+Njg/MTkwMDoyZTMpLGUrclswXS5s
ZW5ndGgpOi0xfWZ1bmN0aW9uIEN2KHQsbixlKXt2YXIgcj0vXihaKXwoWystXVxkXGQpKD86Oj8o
XGRcZCkpPy8uZXhlYyhuLnNsaWNlKGUsZSs2KSk7cmV0dXJuIHI/KHQuWj1yWzFdPzA6LShyWzJd
KyhyWzNdfHwiMDAiKSksZStyWzBdLmxlbmd0aCk6LTF9ZnVuY3Rpb24gUHYodCxuLGUpe3ZhciBy
PXl2LmV4ZWMobi5zbGljZShlLGUrMSkpO3JldHVybiByPyh0LnE9MypyWzBdLTMsZStyWzBdLmxl
bmd0aCk6LTF9ZnVuY3Rpb24genYodCxuLGUpe3ZhciByPXl2LmV4ZWMobi5zbGljZShlLGUrMikp
O3JldHVybiByPyh0Lm09clswXS0xLGUrclswXS5sZW5ndGgpOi0xfWZ1bmN0aW9uICR2KHQsbixl
KXt2YXIgcj15di5leGVjKG4uc2xpY2UoZSxlKzIpKTtyZXR1cm4gcj8odC5kPStyWzBdLGUrclsw
XS5sZW5ndGgpOi0xfWZ1bmN0aW9uIER2KHQsbixlKXt2YXIgcj15di5leGVjKG4uc2xpY2UoZSxl
KzMpKTtyZXR1cm4gcj8odC5tPTAsdC5kPStyWzBdLGUrclswXS5sZW5ndGgpOi0xfWZ1bmN0aW9u
IFJ2KHQsbixlKXt2YXIgcj15di5leGVjKG4uc2xpY2UoZSxlKzIpKTtyZXR1cm4gcj8odC5IPSty
WzBdLGUrclswXS5sZW5ndGgpOi0xfWZ1bmN0aW9uIEZ2KHQsbixlKXt2YXIgcj15di5leGVjKG4u
c2xpY2UoZSxlKzIpKTtyZXR1cm4gcj8odC5NPStyWzBdLGUrclswXS5sZW5ndGgpOi0xfWZ1bmN0
aW9uIHF2KHQsbixlKXt2YXIgcj15di5leGVjKG4uc2xpY2UoZSxlKzIpKTtyZXR1cm4gcj8odC5T
PStyWzBdLGUrclswXS5sZW5ndGgpOi0xfWZ1bmN0aW9uIFV2KHQsbixlKXt2YXIgcj15di5leGVj
KG4uc2xpY2UoZSxlKzMpKTtyZXR1cm4gcj8odC5MPStyWzBdLGUrclswXS5sZW5ndGgpOi0xfWZ1
bmN0aW9uIEl2KHQsbixlKXt2YXIgcj15di5leGVjKG4uc2xpY2UoZSxlKzYpKTtyZXR1cm4gcj8o
dC5MPU1hdGguZmxvb3IoclswXS8xZTMpLGUrclswXS5sZW5ndGgpOi0xfWZ1bmN0aW9uIE92KHQs
bixlKXt2YXIgcj12di5leGVjKG4uc2xpY2UoZSxlKzEpKTtyZXR1cm4gcj9lK3JbMF0ubGVuZ3Ro
Oi0xfWZ1bmN0aW9uIEJ2KHQsbixlKXt2YXIgcj15di5leGVjKG4uc2xpY2UoZSkpO3JldHVybiBy
Pyh0LlE9K3JbMF0sZStyWzBdLmxlbmd0aCk6LTF9ZnVuY3Rpb24gWXYodCxuLGUpe3ZhciByPXl2
LmV4ZWMobi5zbGljZShlKSk7cmV0dXJuIHI/KHQucz0rclswXSxlK3JbMF0ubGVuZ3RoKTotMX1m
dW5jdGlvbiBMdih0LG4pe3JldHVybiBidih0LmdldERhdGUoKSxuLDIpfWZ1bmN0aW9uIGp2KHQs
bil7cmV0dXJuIGJ2KHQuZ2V0SG91cnMoKSxuLDIpfWZ1bmN0aW9uIEh2KHQsbil7cmV0dXJuIGJ2
KHQuZ2V0SG91cnMoKSUxMnx8MTIsbiwyKX1mdW5jdGlvbiBYdih0LG4pe3JldHVybiBidigxK2d5
LmNvdW50KG52KHQpLHQpLG4sMyl9ZnVuY3Rpb24gR3YodCxuKXtyZXR1cm4gYnYodC5nZXRNaWxs
aXNlY29uZHMoKSxuLDMpfWZ1bmN0aW9uIFZ2KHQsbil7cmV0dXJuIEd2KHQsbikrIjAwMCJ9ZnVu
Y3Rpb24gV3YodCxuKXtyZXR1cm4gYnYodC5nZXRNb250aCgpKzEsbiwyKX1mdW5jdGlvbiBadih0
LG4pe3JldHVybiBidih0LmdldE1pbnV0ZXMoKSxuLDIpfWZ1bmN0aW9uIEt2KHQsbil7cmV0dXJu
IGJ2KHQuZ2V0U2Vjb25kcygpLG4sMil9ZnVuY3Rpb24gUXYodCl7dmFyIG49dC5nZXREYXkoKTty
ZXR1cm4gMD09PW4/NzpufWZ1bmN0aW9uIEp2KHQsbil7cmV0dXJuIGJ2KHd5LmNvdW50KG52KHQp
LTEsdCksbiwyKX1mdW5jdGlvbiB0Xyh0KXt2YXIgbj10LmdldERheSgpO3JldHVybiBuPj00fHww
PT09bj9TeSh0KTpTeS5jZWlsKHQpfWZ1bmN0aW9uIG5fKHQsbil7cmV0dXJuIHQ9dF8odCksYnYo
U3kuY291bnQobnYodCksdCkrKDQ9PT1udih0KS5nZXREYXkoKSksbiwyKX1mdW5jdGlvbiBlXyh0
KXtyZXR1cm4gdC5nZXREYXkoKX1mdW5jdGlvbiByXyh0LG4pe3JldHVybiBidihNeS5jb3VudChu
dih0KS0xLHQpLG4sMil9ZnVuY3Rpb24gaV8odCxuKXtyZXR1cm4gYnYodC5nZXRGdWxsWWVhcigp
JTEwMCxuLDIpfWZ1bmN0aW9uIG9fKHQsbil7cmV0dXJuIGJ2KCh0PXRfKHQpKS5nZXRGdWxsWWVh
cigpJTEwMCxuLDIpfWZ1bmN0aW9uIGFfKHQsbil7cmV0dXJuIGJ2KHQuZ2V0RnVsbFllYXIoKSUx
ZTQsbiw0KX1mdW5jdGlvbiB1Xyh0LG4pe3ZhciBlPXQuZ2V0RGF5KCk7cmV0dXJuIGJ2KCh0PWU+
PTR8fDA9PT1lP1N5KHQpOlN5LmNlaWwodCkpLmdldEZ1bGxZZWFyKCklMWU0LG4sNCl9ZnVuY3Rp
b24gY18odCl7dmFyIG49dC5nZXRUaW1lem9uZU9mZnNldCgpO3JldHVybihuPjA/Ii0iOihuKj0t
MSwiKyIpKStidihuLzYwfDAsIjAiLDIpK2J2KG4lNjAsIjAiLDIpfWZ1bmN0aW9uIGZfKHQsbil7
cmV0dXJuIGJ2KHQuZ2V0VVRDRGF0ZSgpLG4sMil9ZnVuY3Rpb24gc18odCxuKXtyZXR1cm4gYnYo
dC5nZXRVVENIb3VycygpLG4sMil9ZnVuY3Rpb24gbF8odCxuKXtyZXR1cm4gYnYodC5nZXRVVENI
b3VycygpJTEyfHwxMixuLDIpfWZ1bmN0aW9uIGhfKHQsbil7cmV0dXJuIGJ2KDErdnkuY291bnQo
cnYodCksdCksbiwzKX1mdW5jdGlvbiBkXyh0LG4pe3JldHVybiBidih0LmdldFVUQ01pbGxpc2Vj
b25kcygpLG4sMyl9ZnVuY3Rpb24gcF8odCxuKXtyZXR1cm4gZF8odCxuKSsiMDAwIn1mdW5jdGlv
biBnXyh0LG4pe3JldHVybiBidih0LmdldFVUQ01vbnRoKCkrMSxuLDIpfWZ1bmN0aW9uIHlfKHQs
bil7cmV0dXJuIGJ2KHQuZ2V0VVRDTWludXRlcygpLG4sMil9ZnVuY3Rpb24gdl8odCxuKXtyZXR1
cm4gYnYodC5nZXRVVENTZWNvbmRzKCksbiwyKX1mdW5jdGlvbiBfXyh0KXt2YXIgbj10LmdldFVU
Q0RheSgpO3JldHVybiAwPT09bj83Om59ZnVuY3Rpb24gYl8odCxuKXtyZXR1cm4gYnYocXkuY291
bnQocnYodCktMSx0KSxuLDIpfWZ1bmN0aW9uIG1fKHQpe3ZhciBuPXQuZ2V0VVRDRGF5KCk7cmV0
dXJuIG4+PTR8fDA9PT1uP0J5KHQpOkJ5LmNlaWwodCl9ZnVuY3Rpb24geF8odCxuKXtyZXR1cm4g
dD1tXyh0KSxidihCeS5jb3VudChydih0KSx0KSsoND09PXJ2KHQpLmdldFVUQ0RheSgpKSxuLDIp
fWZ1bmN0aW9uIHdfKHQpe3JldHVybiB0LmdldFVUQ0RheSgpfWZ1bmN0aW9uIE1fKHQsbil7cmV0
dXJuIGJ2KFV5LmNvdW50KHJ2KHQpLTEsdCksbiwyKX1mdW5jdGlvbiBUXyh0LG4pe3JldHVybiBi
dih0LmdldFVUQ0Z1bGxZZWFyKCklMTAwLG4sMil9ZnVuY3Rpb24gQV8odCxuKXtyZXR1cm4gYnYo
KHQ9bV8odCkpLmdldFVUQ0Z1bGxZZWFyKCklMTAwLG4sMil9ZnVuY3Rpb24gU18odCxuKXtyZXR1
cm4gYnYodC5nZXRVVENGdWxsWWVhcigpJTFlNCxuLDQpfWZ1bmN0aW9uIEVfKHQsbil7dmFyIGU9
dC5nZXRVVENEYXkoKTtyZXR1cm4gYnYoKHQ9ZT49NHx8MD09PWU/QnkodCk6QnkuY2VpbCh0KSku
Z2V0VVRDRnVsbFllYXIoKSUxZTQsbiw0KX1mdW5jdGlvbiBOXygpe3JldHVybiIrMDAwMCJ9ZnVu
Y3Rpb24ga18oKXtyZXR1cm4iJSJ9ZnVuY3Rpb24gQ18odCl7cmV0dXJuK3R9ZnVuY3Rpb24gUF8o
dCl7cmV0dXJuIE1hdGguZmxvb3IoK3QvMWUzKX1mdW5jdGlvbiB6XyhuKXtyZXR1cm4gcHY9ZHYo
biksdC50aW1lRm9ybWF0PXB2LmZvcm1hdCx0LnRpbWVQYXJzZT1wdi5wYXJzZSx0LnV0Y0Zvcm1h
dD1wdi51dGNGb3JtYXQsdC51dGNQYXJzZT1wdi51dGNQYXJzZSxwdn10LnRpbWVGb3JtYXQ9dm9p
ZCAwLHQudGltZVBhcnNlPXZvaWQgMCx0LnV0Y0Zvcm1hdD12b2lkIDAsdC51dGNQYXJzZT12b2lk
IDAsel8oe2RhdGVUaW1lOiIleCwgJVgiLGRhdGU6IiUtbS8lLWQvJVkiLHRpbWU6IiUtSTolTTol
UyAlcCIscGVyaW9kczpbIkFNIiwiUE0iXSxkYXlzOlsiU3VuZGF5IiwiTW9uZGF5IiwiVHVlc2Rh
eSIsIldlZG5lc2RheSIsIlRodXJzZGF5IiwiRnJpZGF5IiwiU2F0dXJkYXkiXSxzaG9ydERheXM6
WyJTdW4iLCJNb24iLCJUdWUiLCJXZWQiLCJUaHUiLCJGcmkiLCJTYXQiXSxtb250aHM6WyJKYW51
YXJ5IiwiRmVicnVhcnkiLCJNYXJjaCIsIkFwcmlsIiwiTWF5IiwiSnVuZSIsIkp1bHkiLCJBdWd1
c3QiLCJTZXB0ZW1iZXIiLCJPY3RvYmVyIiwiTm92ZW1iZXIiLCJEZWNlbWJlciJdLHNob3J0TW9u
dGhzOlsiSmFuIiwiRmViIiwiTWFyIiwiQXByIiwiTWF5IiwiSnVuIiwiSnVsIiwiQXVnIiwiU2Vw
IiwiT2N0IiwiTm92IiwiRGVjIl19KTt2YXIgJF89IiVZLSVtLSVkVCVIOiVNOiVTLiVMWiI7dmFy
IERfPURhdGUucHJvdG90eXBlLnRvSVNPU3RyaW5nP2Z1bmN0aW9uKHQpe3JldHVybiB0LnRvSVNP
U3RyaW5nKCl9OnQudXRjRm9ybWF0KCRfKSxSXz1EXzt2YXIgRl89K25ldyBEYXRlKCIyMDAwLTAx
LTAxVDAwOjAwOjAwLjAwMFoiKT9mdW5jdGlvbih0KXt2YXIgbj1uZXcgRGF0ZSh0KTtyZXR1cm4g
aXNOYU4obik/bnVsbDpufTp0LnV0Y1BhcnNlKCRfKSxxXz1GXztmdW5jdGlvbiBVXyh0KXtyZXR1
cm4gbmV3IERhdGUodCl9ZnVuY3Rpb24gSV8odCl7cmV0dXJuIHQgaW5zdGFuY2VvZiBEYXRlPyt0
OituZXcgRGF0ZSgrdCl9ZnVuY3Rpb24gT18odCxuLGUscixpLG8sYSx1LGMsZil7dmFyIHM9RWco
KSxsPXMuaW52ZXJ0LGg9cy5kb21haW4sZD1mKCIuJUwiKSxwPWYoIjolUyIpLGc9ZigiJUk6JU0i
KSx5PWYoIiVJICVwIiksdj1mKCIlYSAlZCIpLF89ZigiJWIgJWQiKSxiPWYoIiVCIiksbT1mKCIl
WSIpO2Z1bmN0aW9uIHgodCl7cmV0dXJuKGModCk8dD9kOnUodCk8dD9wOmEodCk8dD9nOm8odCk8
dD95OnIodCk8dD9pKHQpPHQ/djpfOmUodCk8dD9iOm0pKHQpfXJldHVybiBzLmludmVydD1mdW5j
dGlvbih0KXtyZXR1cm4gbmV3IERhdGUobCh0KSl9LHMuZG9tYWluPWZ1bmN0aW9uKHQpe3JldHVy
biBhcmd1bWVudHMubGVuZ3RoP2goQXJyYXkuZnJvbSh0LElfKSk6aCgpLm1hcChVXyl9LHMudGlj
a3M9ZnVuY3Rpb24obil7dmFyIGU9aCgpO3JldHVybiB0KGVbMF0sZVtlLmxlbmd0aC0xXSxudWxs
PT1uPzEwOm4pfSxzLnRpY2tGb3JtYXQ9ZnVuY3Rpb24odCxuKXtyZXR1cm4gbnVsbD09bj94OmYo
bil9LHMubmljZT1mdW5jdGlvbih0KXt2YXIgZT1oKCk7cmV0dXJuIHQmJiJmdW5jdGlvbiI9PXR5
cGVvZiB0LnJhbmdlfHwodD1uKGVbMF0sZVtlLmxlbmd0aC0xXSxudWxsPT10PzEwOnQpKSx0P2go
Q2coZSx0KSk6c30scy5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIEFnKHMsT18odCxuLGUscixpLG8s
YSx1LGMsZikpfSxzfWZ1bmN0aW9uIEJfKCl7dmFyIHQsbixlLHIsaSxvPTAsYT0xLHU9eGcsYz0h
MTtmdW5jdGlvbiBmKG4pe3JldHVybiBudWxsPT1ufHxpc05hTihuPStuKT9pOnUoMD09PWU/LjU6
KG49KHIobiktdCkqZSxjP01hdGgubWF4KDAsTWF0aC5taW4oMSxuKSk6bikpfWZ1bmN0aW9uIHMo
dCl7cmV0dXJuIGZ1bmN0aW9uKG4pe3ZhciBlLHI7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KFtl
LHJdPW4sdT10KGUsciksZik6W3UoMCksdSgxKV19fXJldHVybiBmLmRvbWFpbj1mdW5jdGlvbihp
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oW28sYV09aSx0PXIobz0rbyksbj1yKGE9K2EpLGU9
dD09PW4/MDoxLyhuLXQpLGYpOltvLGFdfSxmLmNsYW1wPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1
bWVudHMubGVuZ3RoPyhjPSEhdCxmKTpjfSxmLmludGVycG9sYXRvcj1mdW5jdGlvbih0KXtyZXR1
cm4gYXJndW1lbnRzLmxlbmd0aD8odT10LGYpOnV9LGYucmFuZ2U9cyhWciksZi5yYW5nZVJvdW5k
PXMoV3IpLGYudW5rbm93bj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oaT10
LGYpOml9LGZ1bmN0aW9uKGkpe3JldHVybiByPWksdD1pKG8pLG49aShhKSxlPXQ9PT1uPzA6MS8o
bi10KSxmfX1mdW5jdGlvbiBZXyh0LG4pe3JldHVybiBuLmRvbWFpbih0LmRvbWFpbigpKS5pbnRl
cnBvbGF0b3IodC5pbnRlcnBvbGF0b3IoKSkuY2xhbXAodC5jbGFtcCgpKS51bmtub3duKHQudW5r
bm93bigpKX1mdW5jdGlvbiBMXygpe3ZhciB0PWpnKEJfKCkpO3JldHVybiB0LmNvcHk9ZnVuY3Rp
b24oKXtyZXR1cm4gWV8odCxMXygpKS5leHBvbmVudCh0LmV4cG9uZW50KCkpfSxwZy5hcHBseSh0
LGFyZ3VtZW50cyl9ZnVuY3Rpb24gal8oKXt2YXIgdCxuLGUscixpLG8sYSx1PTAsYz0uNSxmPTEs
cz0xLGw9eGcsaD0hMTtmdW5jdGlvbiBkKHQpe3JldHVybiBpc05hTih0PSt0KT9hOih0PS41Kygo
dD0rbyh0KSktbikqKHMqdDxzKm4/cjppKSxsKGg/TWF0aC5tYXgoMCxNYXRoLm1pbigxLHQpKTp0
KSl9ZnVuY3Rpb24gcCh0KXtyZXR1cm4gZnVuY3Rpb24obil7dmFyIGUscixpO3JldHVybiBhcmd1
bWVudHMubGVuZ3RoPyhbZSxyLGldPW4sbD1waSh0LFtlLHIsaV0pLGQpOltsKDApLGwoLjUpLGwo
MSldfX1yZXR1cm4gZC5kb21haW49ZnVuY3Rpb24oYSl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KFt1LGMsZl09YSx0PW8odT0rdSksbj1vKGM9K2MpLGU9byhmPStmKSxyPXQ9PT1uPzA6LjUvKG4t
dCksaT1uPT09ZT8wOi41LyhlLW4pLHM9bjx0Py0xOjEsZCk6W3UsYyxmXX0sZC5jbGFtcD1mdW5j
dGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oaD0hIXQsZCk6aH0sZC5pbnRlcnBvbGF0
b3I9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGw9dCxkKTpsfSxkLnJhbmdl
PXAoVnIpLGQucmFuZ2VSb3VuZD1wKFdyKSxkLnVua25vd249ZnVuY3Rpb24odCl7cmV0dXJuIGFy
Z3VtZW50cy5sZW5ndGg/KGE9dCxkKTphfSxmdW5jdGlvbihhKXtyZXR1cm4gbz1hLHQ9YSh1KSxu
PWEoYyksZT1hKGYpLHI9dD09PW4/MDouNS8obi10KSxpPW49PT1lPzA6LjUvKGUtbikscz1uPHQ/
LTE6MSxkfX1mdW5jdGlvbiBIXygpe3ZhciB0PWpnKGpfKCkpO3JldHVybiB0LmNvcHk9ZnVuY3Rp
b24oKXtyZXR1cm4gWV8odCxIXygpKS5leHBvbmVudCh0LmV4cG9uZW50KCkpfSxwZy5hcHBseSh0
LGFyZ3VtZW50cyl9ZnVuY3Rpb24gWF8odCl7Zm9yKHZhciBuPXQubGVuZ3RoLzZ8MCxlPW5ldyBB
cnJheShuKSxyPTA7cjxuOyllW3JdPSIjIit0LnNsaWNlKDYqciw2Kisrcik7cmV0dXJuIGV9dmFy
IEdfPVhfKCIxZjc3YjRmZjdmMGUyY2EwMmNkNjI3Mjg5NDY3YmQ4YzU2NGJlMzc3YzI3ZjdmN2Zi
Y2JkMjIxN2JlY2YiKSxWXz1YXygiN2ZjOTdmYmVhZWQ0ZmRjMDg2ZmZmZjk5Mzg2Y2IwZjAwMjdm
YmY1YjE3NjY2NjY2IiksV189WF8oIjFiOWU3N2Q5NWYwMjc1NzBiM2U3Mjk4YTY2YTYxZWU2YWIw
MmE2NzYxZDY2NjY2NiIpLFpfPVhfKCJhNmNlZTMxZjc4YjRiMmRmOGEzM2EwMmNmYjlhOTllMzFh
MWNmZGJmNmZmZjdmMDBjYWIyZDY2YTNkOWFmZmZmOTliMTU5MjgiKSxLXz1YXygiZmJiNGFlYjNj
ZGUzY2NlYmM1ZGVjYmU0ZmVkOWE2ZmZmZmNjZTVkOGJkZmRkYWVjZjJmMmYyIiksUV89WF8oImIz
ZTJjZGZkY2RhY2NiZDVlOGY0Y2FlNGU2ZjVjOWZmZjJhZWYxZTJjY2NjY2NjYyIpLEpfPVhfKCJl
NDFhMWMzNzdlYjg0ZGFmNGE5ODRlYTNmZjdmMDBmZmZmMzNhNjU2MjhmNzgxYmY5OTk5OTkiKSx0
Yj1YXygiNjZjMmE1ZmM4ZDYyOGRhMGNiZTc4YWMzYTZkODU0ZmZkOTJmZTVjNDk0YjNiM2IzIiks
bmI9WF8oIjhkZDNjN2ZmZmZiM2JlYmFkYWZiODA3MjgwYjFkM2ZkYjQ2MmIzZGU2OWZjY2RlNWQ5
ZDlkOWJjODBiZGNjZWJjNWZmZWQ2ZiIpLGViPVhfKCI0ZTc5YTdmMjhlMmNlMTU3NTk3NmI3YjI1
OWExNGZlZGM5NDlhZjdhYTFmZjlkYTc5Yzc1NWZiYWIwYWIiKSxyYj10PT5xcih0W3QubGVuZ3Ro
LTFdKSxpYj1uZXcgQXJyYXkoMykuY29uY2F0KCJkOGIzNjVmNWY1ZjU1YWI0YWMiLCJhNjYxMWFk
ZmMyN2Q4MGNkYzEwMTg1NzEiLCJhNjYxMWFkZmMyN2RmNWY1ZjU4MGNkYzEwMTg1NzEiLCI4YzUx
MGFkOGIzNjVmNmU4YzNjN2VhZTU1YWI0YWMwMTY2NWUiLCI4YzUxMGFkOGIzNjVmNmU4YzNmNWY1
ZjVjN2VhZTU1YWI0YWMwMTY2NWUiLCI4YzUxMGFiZjgxMmRkZmMyN2RmNmU4YzNjN2VhZTU4MGNk
YzEzNTk3OGYwMTY2NWUiLCI4YzUxMGFiZjgxMmRkZmMyN2RmNmU4YzNmNWY1ZjVjN2VhZTU4MGNk
YzEzNTk3OGYwMTY2NWUiLCI1NDMwMDU4YzUxMGFiZjgxMmRkZmMyN2RmNmU4YzNjN2VhZTU4MGNk
YzEzNTk3OGYwMTY2NWUwMDNjMzAiLCI1NDMwMDU4YzUxMGFiZjgxMmRkZmMyN2RmNmU4YzNmNWY1
ZjVjN2VhZTU4MGNkYzEzNTk3OGYwMTY2NWUwMDNjMzAiKS5tYXAoWF8pLG9iPXJiKGliKSxhYj1u
ZXcgQXJyYXkoMykuY29uY2F0KCJhZjhkYzNmN2Y3Zjc3ZmJmN2IiLCI3YjMyOTRjMmE1Y2ZhNmRi
YTAwMDg4MzciLCI3YjMyOTRjMmE1Y2ZmN2Y3ZjdhNmRiYTAwMDg4MzciLCI3NjJhODNhZjhkYzNl
N2Q0ZThkOWYwZDM3ZmJmN2IxYjc4MzciLCI3NjJhODNhZjhkYzNlN2Q0ZThmN2Y3ZjdkOWYwZDM3
ZmJmN2IxYjc4MzciLCI3NjJhODM5OTcwYWJjMmE1Y2ZlN2Q0ZThkOWYwZDNhNmRiYTA1YWFlNjEx
Yjc4MzciLCI3NjJhODM5OTcwYWJjMmE1Y2ZlN2Q0ZThmN2Y3ZjdkOWYwZDNhNmRiYTA1YWFlNjEx
Yjc4MzciLCI0MDAwNGI3NjJhODM5OTcwYWJjMmE1Y2ZlN2Q0ZThkOWYwZDNhNmRiYTA1YWFlNjEx
Yjc4MzcwMDQ0MWIiLCI0MDAwNGI3NjJhODM5OTcwYWJjMmE1Y2ZlN2Q0ZThmN2Y3ZjdkOWYwZDNh
NmRiYTA1YWFlNjExYjc4MzcwMDQ0MWIiKS5tYXAoWF8pLHViPXJiKGFiKSxjYj1uZXcgQXJyYXko
MykuY29uY2F0KCJlOWEzYzlmN2Y3ZjdhMWQ3NmEiLCJkMDFjOGJmMWI2ZGFiOGUxODY0ZGFjMjYi
LCJkMDFjOGJmMWI2ZGFmN2Y3ZjdiOGUxODY0ZGFjMjYiLCJjNTFiN2RlOWEzYzlmZGUwZWZlNmY1
ZDBhMWQ3NmE0ZDkyMjEiLCJjNTFiN2RlOWEzYzlmZGUwZWZmN2Y3ZjdlNmY1ZDBhMWQ3NmE0ZDky
MjEiLCJjNTFiN2RkZTc3YWVmMWI2ZGFmZGUwZWZlNmY1ZDBiOGUxODY3ZmJjNDE0ZDkyMjEiLCJj
NTFiN2RkZTc3YWVmMWI2ZGFmZGUwZWZmN2Y3ZjdlNmY1ZDBiOGUxODY3ZmJjNDE0ZDkyMjEiLCI4
ZTAxNTJjNTFiN2RkZTc3YWVmMWI2ZGFmZGUwZWZlNmY1ZDBiOGUxODY3ZmJjNDE0ZDkyMjEyNzY0
MTkiLCI4ZTAxNTJjNTFiN2RkZTc3YWVmMWI2ZGFmZGUwZWZmN2Y3ZjdlNmY1ZDBiOGUxODY3ZmJj
NDE0ZDkyMjEyNzY0MTkiKS5tYXAoWF8pLGZiPXJiKGNiKSxzYj1uZXcgQXJyYXkoMykuY29uY2F0
KCI5OThlYzNmN2Y3ZjdmMWEzNDAiLCI1ZTNjOTliMmFiZDJmZGI4NjNlNjYxMDEiLCI1ZTNjOTli
MmFiZDJmN2Y3ZjdmZGI4NjNlNjYxMDEiLCI1NDI3ODg5OThlYzNkOGRhZWJmZWUwYjZmMWEzNDBi
MzU4MDYiLCI1NDI3ODg5OThlYzNkOGRhZWJmN2Y3ZjdmZWUwYjZmMWEzNDBiMzU4MDYiLCI1NDI3
ODg4MDczYWNiMmFiZDJkOGRhZWJmZWUwYjZmZGI4NjNlMDgyMTRiMzU4MDYiLCI1NDI3ODg4MDcz
YWNiMmFiZDJkOGRhZWJmN2Y3ZjdmZWUwYjZmZGI4NjNlMDgyMTRiMzU4MDYiLCIyZDAwNGI1NDI3
ODg4MDczYWNiMmFiZDJkOGRhZWJmZWUwYjZmZGI4NjNlMDgyMTRiMzU4MDY3ZjNiMDgiLCIyZDAw
NGI1NDI3ODg4MDczYWNiMmFiZDJkOGRhZWJmN2Y3ZjdmZWUwYjZmZGI4NjNlMDgyMTRiMzU4MDY3
ZjNiMDgiKS5tYXAoWF8pLGxiPXJiKHNiKSxoYj1uZXcgQXJyYXkoMykuY29uY2F0KCJlZjhhNjJm
N2Y3Zjc2N2E5Y2YiLCJjYTAwMjBmNGE1ODI5MmM1ZGUwNTcxYjAiLCJjYTAwMjBmNGE1ODJmN2Y3
Zjc5MmM1ZGUwNTcxYjAiLCJiMjE4MmJlZjhhNjJmZGRiYzdkMWU1ZjA2N2E5Y2YyMTY2YWMiLCJi
MjE4MmJlZjhhNjJmZGRiYzdmN2Y3ZjdkMWU1ZjA2N2E5Y2YyMTY2YWMiLCJiMjE4MmJkNjYwNGRm
NGE1ODJmZGRiYzdkMWU1ZjA5MmM1ZGU0MzkzYzMyMTY2YWMiLCJiMjE4MmJkNjYwNGRmNGE1ODJm
ZGRiYzdmN2Y3ZjdkMWU1ZjA5MmM1ZGU0MzkzYzMyMTY2YWMiLCI2NzAwMWZiMjE4MmJkNjYwNGRm
NGE1ODJmZGRiYzdkMWU1ZjA5MmM1ZGU0MzkzYzMyMTY2YWMwNTMwNjEiLCI2NzAwMWZiMjE4MmJk
NjYwNGRmNGE1ODJmZGRiYzdmN2Y3ZjdkMWU1ZjA5MmM1ZGU0MzkzYzMyMTY2YWMwNTMwNjEiKS5t
YXAoWF8pLGRiPXJiKGhiKSxwYj1uZXcgQXJyYXkoMykuY29uY2F0KCJlZjhhNjJmZmZmZmY5OTk5
OTkiLCJjYTAwMjBmNGE1ODJiYWJhYmE0MDQwNDAiLCJjYTAwMjBmNGE1ODJmZmZmZmZiYWJhYmE0
MDQwNDAiLCJiMjE4MmJlZjhhNjJmZGRiYzdlMGUwZTA5OTk5OTk0ZDRkNGQiLCJiMjE4MmJlZjhh
NjJmZGRiYzdmZmZmZmZlMGUwZTA5OTk5OTk0ZDRkNGQiLCJiMjE4MmJkNjYwNGRmNGE1ODJmZGRi
YzdlMGUwZTBiYWJhYmE4Nzg3ODc0ZDRkNGQiLCJiMjE4MmJkNjYwNGRmNGE1ODJmZGRiYzdmZmZm
ZmZlMGUwZTBiYWJhYmE4Nzg3ODc0ZDRkNGQiLCI2NzAwMWZiMjE4MmJkNjYwNGRmNGE1ODJmZGRi
YzdlMGUwZTBiYWJhYmE4Nzg3ODc0ZDRkNGQxYTFhMWEiLCI2NzAwMWZiMjE4MmJkNjYwNGRmNGE1
ODJmZGRiYzdmZmZmZmZlMGUwZTBiYWJhYmE4Nzg3ODc0ZDRkNGQxYTFhMWEiKS5tYXAoWF8pLGdi
PXJiKHBiKSx5Yj1uZXcgQXJyYXkoMykuY29uY2F0KCJmYzhkNTlmZmZmYmY5MWJmZGIiLCJkNzE5
MWNmZGFlNjFhYmQ5ZTkyYzdiYjYiLCJkNzE5MWNmZGFlNjFmZmZmYmZhYmQ5ZTkyYzdiYjYiLCJk
NzMwMjdmYzhkNTlmZWUwOTBlMGYzZjg5MWJmZGI0NTc1YjQiLCJkNzMwMjdmYzhkNTlmZWUwOTBm
ZmZmYmZlMGYzZjg5MWJmZGI0NTc1YjQiLCJkNzMwMjdmNDZkNDNmZGFlNjFmZWUwOTBlMGYzZjhh
YmQ5ZTk3NGFkZDE0NTc1YjQiLCJkNzMwMjdmNDZkNDNmZGFlNjFmZWUwOTBmZmZmYmZlMGYzZjhh
YmQ5ZTk3NGFkZDE0NTc1YjQiLCJhNTAwMjZkNzMwMjdmNDZkNDNmZGFlNjFmZWUwOTBlMGYzZjhh
YmQ5ZTk3NGFkZDE0NTc1YjQzMTM2OTUiLCJhNTAwMjZkNzMwMjdmNDZkNDNmZGFlNjFmZWUwOTBm
ZmZmYmZlMGYzZjhhYmQ5ZTk3NGFkZDE0NTc1YjQzMTM2OTUiKS5tYXAoWF8pLHZiPXJiKHliKSxf
Yj1uZXcgQXJyYXkoMykuY29uY2F0KCJmYzhkNTlmZmZmYmY5MWNmNjAiLCJkNzE5MWNmZGFlNjFh
NmQ5NmExYTk2NDEiLCJkNzE5MWNmZGFlNjFmZmZmYmZhNmQ5NmExYTk2NDEiLCJkNzMwMjdmYzhk
NTlmZWUwOGJkOWVmOGI5MWNmNjAxYTk4NTAiLCJkNzMwMjdmYzhkNTlmZWUwOGJmZmZmYmZkOWVm
OGI5MWNmNjAxYTk4NTAiLCJkNzMwMjdmNDZkNDNmZGFlNjFmZWUwOGJkOWVmOGJhNmQ5NmE2NmJk
NjMxYTk4NTAiLCJkNzMwMjdmNDZkNDNmZGFlNjFmZWUwOGJmZmZmYmZkOWVmOGJhNmQ5NmE2NmJk
NjMxYTk4NTAiLCJhNTAwMjZkNzMwMjdmNDZkNDNmZGFlNjFmZWUwOGJkOWVmOGJhNmQ5NmE2NmJk
NjMxYTk4NTAwMDY4MzciLCJhNTAwMjZkNzMwMjdmNDZkNDNmZGFlNjFmZWUwOGJmZmZmYmZkOWVm
OGJhNmQ5NmE2NmJkNjMxYTk4NTAwMDY4MzciKS5tYXAoWF8pLGJiPXJiKF9iKSxtYj1uZXcgQXJy
YXkoMykuY29uY2F0KCJmYzhkNTlmZmZmYmY5OWQ1OTQiLCJkNzE5MWNmZGFlNjFhYmRkYTQyYjgz
YmEiLCJkNzE5MWNmZGFlNjFmZmZmYmZhYmRkYTQyYjgzYmEiLCJkNTNlNGZmYzhkNTlmZWUwOGJl
NmY1OTg5OWQ1OTQzMjg4YmQiLCJkNTNlNGZmYzhkNTlmZWUwOGJmZmZmYmZlNmY1OTg5OWQ1OTQz
Mjg4YmQiLCJkNTNlNGZmNDZkNDNmZGFlNjFmZWUwOGJlNmY1OThhYmRkYTQ2NmMyYTUzMjg4YmQi
LCJkNTNlNGZmNDZkNDNmZGFlNjFmZWUwOGJmZmZmYmZlNmY1OThhYmRkYTQ2NmMyYTUzMjg4YmQi
LCI5ZTAxNDJkNTNlNGZmNDZkNDNmZGFlNjFmZWUwOGJlNmY1OThhYmRkYTQ2NmMyYTUzMjg4YmQ1
ZTRmYTIiLCI5ZTAxNDJkNTNlNGZmNDZkNDNmZGFlNjFmZWUwOGJmZmZmYmZlNmY1OThhYmRkYTQ2
NmMyYTUzMjg4YmQ1ZTRmYTIiKS5tYXAoWF8pLHhiPXJiKG1iKSx3Yj1uZXcgQXJyYXkoMykuY29u
Y2F0KCJlNWY1Zjk5OWQ4YzkyY2EyNWYiLCJlZGY4ZmJiMmUyZTI2NmMyYTQyMzhiNDUiLCJlZGY4
ZmJiMmUyZTI2NmMyYTQyY2EyNWYwMDZkMmMiLCJlZGY4ZmJjY2VjZTY5OWQ4Yzk2NmMyYTQyY2Ey
NWYwMDZkMmMiLCJlZGY4ZmJjY2VjZTY5OWQ4Yzk2NmMyYTQ0MWFlNzYyMzhiNDUwMDU4MjQiLCJm
N2ZjZmRlNWY1ZjljY2VjZTY5OWQ4Yzk2NmMyYTQ0MWFlNzYyMzhiNDUwMDU4MjQiLCJmN2ZjZmRl
NWY1ZjljY2VjZTY5OWQ4Yzk2NmMyYTQ0MWFlNzYyMzhiNDUwMDZkMmMwMDQ0MWIiKS5tYXAoWF8p
LE1iPXJiKHdiKSxUYj1uZXcgQXJyYXkoMykuY29uY2F0KCJlMGVjZjQ5ZWJjZGE4ODU2YTciLCJl
ZGY4ZmJiM2NkZTM4Yzk2YzY4ODQxOWQiLCJlZGY4ZmJiM2NkZTM4Yzk2YzY4ODU2YTc4MTBmN2Mi
LCJlZGY4ZmJiZmQzZTY5ZWJjZGE4Yzk2YzY4ODU2YTc4MTBmN2MiLCJlZGY4ZmJiZmQzZTY5ZWJj
ZGE4Yzk2YzY4YzZiYjE4ODQxOWQ2ZTAxNmIiLCJmN2ZjZmRlMGVjZjRiZmQzZTY5ZWJjZGE4Yzk2
YzY4YzZiYjE4ODQxOWQ2ZTAxNmIiLCJmN2ZjZmRlMGVjZjRiZmQzZTY5ZWJjZGE4Yzk2YzY4YzZi
YjE4ODQxOWQ4MTBmN2M0ZDAwNGIiKS5tYXAoWF8pLEFiPXJiKFRiKSxTYj1uZXcgQXJyYXkoMyku
Y29uY2F0KCJlMGYzZGJhOGRkYjU0M2EyY2EiLCJmMGY5ZThiYWU0YmM3YmNjYzQyYjhjYmUiLCJm
MGY5ZThiYWU0YmM3YmNjYzQ0M2EyY2EwODY4YWMiLCJmMGY5ZThjY2ViYzVhOGRkYjU3YmNjYzQ0
M2EyY2EwODY4YWMiLCJmMGY5ZThjY2ViYzVhOGRkYjU3YmNjYzQ0ZWIzZDMyYjhjYmUwODU4OWUi
LCJmN2ZjZjBlMGYzZGJjY2ViYzVhOGRkYjU3YmNjYzQ0ZWIzZDMyYjhjYmUwODU4OWUiLCJmN2Zj
ZjBlMGYzZGJjY2ViYzVhOGRkYjU3YmNjYzQ0ZWIzZDMyYjhjYmUwODY4YWMwODQwODEiKS5tYXAo
WF8pLEViPXJiKFNiKSxOYj1uZXcgQXJyYXkoMykuY29uY2F0KCJmZWU4YzhmZGJiODRlMzRhMzMi
LCJmZWYwZDlmZGNjOGFmYzhkNTlkNzMwMWYiLCJmZWYwZDlmZGNjOGFmYzhkNTllMzRhMzNiMzAw
MDAiLCJmZWYwZDlmZGQ0OWVmZGJiODRmYzhkNTllMzRhMzNiMzAwMDAiLCJmZWYwZDlmZGQ0OWVm
ZGJiODRmYzhkNTllZjY1NDhkNzMwMWY5OTAwMDAiLCJmZmY3ZWNmZWU4YzhmZGQ0OWVmZGJiODRm
YzhkNTllZjY1NDhkNzMwMWY5OTAwMDAiLCJmZmY3ZWNmZWU4YzhmZGQ0OWVmZGJiODRmYzhkNTll
ZjY1NDhkNzMwMWZiMzAwMDA3ZjAwMDAiKS5tYXAoWF8pLGtiPXJiKE5iKSxDYj1uZXcgQXJyYXko
MykuY29uY2F0KCJlY2UyZjBhNmJkZGIxYzkwOTkiLCJmNmVmZjdiZGM5ZTE2N2E5Y2YwMjgxOGEi
LCJmNmVmZjdiZGM5ZTE2N2E5Y2YxYzkwOTkwMTZjNTkiLCJmNmVmZjdkMGQxZTZhNmJkZGI2N2E5
Y2YxYzkwOTkwMTZjNTkiLCJmNmVmZjdkMGQxZTZhNmJkZGI2N2E5Y2YzNjkwYzAwMjgxOGEwMTY0
NTAiLCJmZmY3ZmJlY2UyZjBkMGQxZTZhNmJkZGI2N2E5Y2YzNjkwYzAwMjgxOGEwMTY0NTAiLCJm
ZmY3ZmJlY2UyZjBkMGQxZTZhNmJkZGI2N2E5Y2YzNjkwYzAwMjgxOGEwMTZjNTkwMTQ2MzYiKS5t
YXAoWF8pLFBiPXJiKENiKSx6Yj1uZXcgQXJyYXkoMykuY29uY2F0KCJlY2U3ZjJhNmJkZGIyYjhj
YmUiLCJmMWVlZjZiZGM5ZTE3NGE5Y2YwNTcwYjAiLCJmMWVlZjZiZGM5ZTE3NGE5Y2YyYjhjYmUw
NDVhOGQiLCJmMWVlZjZkMGQxZTZhNmJkZGI3NGE5Y2YyYjhjYmUwNDVhOGQiLCJmMWVlZjZkMGQx
ZTZhNmJkZGI3NGE5Y2YzNjkwYzAwNTcwYjAwMzRlN2IiLCJmZmY3ZmJlY2U3ZjJkMGQxZTZhNmJk
ZGI3NGE5Y2YzNjkwYzAwNTcwYjAwMzRlN2IiLCJmZmY3ZmJlY2U3ZjJkMGQxZTZhNmJkZGI3NGE5
Y2YzNjkwYzAwNTcwYjAwNDVhOGQwMjM4NTgiKS5tYXAoWF8pLCRiPXJiKHpiKSxEYj1uZXcgQXJy
YXkoMykuY29uY2F0KCJlN2UxZWZjOTk0YzdkZDFjNzciLCJmMWVlZjZkN2I1ZDhkZjY1YjBjZTEy
NTYiLCJmMWVlZjZkN2I1ZDhkZjY1YjBkZDFjNzc5ODAwNDMiLCJmMWVlZjZkNGI5ZGFjOTk0Yzdk
ZjY1YjBkZDFjNzc5ODAwNDMiLCJmMWVlZjZkNGI5ZGFjOTk0YzdkZjY1YjBlNzI5OGFjZTEyNTY5
MTAwM2YiLCJmN2Y0ZjllN2UxZWZkNGI5ZGFjOTk0YzdkZjY1YjBlNzI5OGFjZTEyNTY5MTAwM2Yi
LCJmN2Y0ZjllN2UxZWZkNGI5ZGFjOTk0YzdkZjY1YjBlNzI5OGFjZTEyNTY5ODAwNDM2NzAwMWYi
KS5tYXAoWF8pLFJiPXJiKERiKSxGYj1uZXcgQXJyYXkoMykuY29uY2F0KCJmZGUwZGRmYTlmYjVj
NTFiOGEiLCJmZWViZTJmYmI0YjlmNzY4YTFhZTAxN2UiLCJmZWViZTJmYmI0YjlmNzY4YTFjNTFi
OGE3YTAxNzciLCJmZWViZTJmY2M1YzBmYTlmYjVmNzY4YTFjNTFiOGE3YTAxNzciLCJmZWViZTJm
Y2M1YzBmYTlmYjVmNzY4YTFkZDM0OTdhZTAxN2U3YTAxNzciLCJmZmY3ZjNmZGUwZGRmY2M1YzBm
YTlmYjVmNzY4YTFkZDM0OTdhZTAxN2U3YTAxNzciLCJmZmY3ZjNmZGUwZGRmY2M1YzBmYTlmYjVm
NzY4YTFkZDM0OTdhZTAxN2U3YTAxNzc0OTAwNmEiKS5tYXAoWF8pLHFiPXJiKEZiKSxVYj1uZXcg
QXJyYXkoMykuY29uY2F0KCJlZGY4YjE3ZmNkYmIyYzdmYjgiLCJmZmZmY2NhMWRhYjQ0MWI2YzQy
MjVlYTgiLCJmZmZmY2NhMWRhYjQ0MWI2YzQyYzdmYjgyNTM0OTQiLCJmZmZmY2NjN2U5YjQ3ZmNk
YmI0MWI2YzQyYzdmYjgyNTM0OTQiLCJmZmZmY2NjN2U5YjQ3ZmNkYmI0MWI2YzQxZDkxYzAyMjVl
YTgwYzJjODQiLCJmZmZmZDllZGY4YjFjN2U5YjQ3ZmNkYmI0MWI2YzQxZDkxYzAyMjVlYTgwYzJj
ODQiLCJmZmZmZDllZGY4YjFjN2U5YjQ3ZmNkYmI0MWI2YzQxZDkxYzAyMjVlYTgyNTM0OTQwODFk
NTgiKS5tYXAoWF8pLEliPXJiKFViKSxPYj1uZXcgQXJyYXkoMykuY29uY2F0KCJmN2ZjYjlhZGRk
OGUzMWEzNTQiLCJmZmZmY2NjMmU2OTk3OGM2NzkyMzg0NDMiLCJmZmZmY2NjMmU2OTk3OGM2Nzkz
MWEzNTQwMDY4MzciLCJmZmZmY2NkOWYwYTNhZGRkOGU3OGM2NzkzMWEzNTQwMDY4MzciLCJmZmZm
Y2NkOWYwYTNhZGRkOGU3OGM2Nzk0MWFiNWQyMzg0NDMwMDVhMzIiLCJmZmZmZTVmN2ZjYjlkOWYw
YTNhZGRkOGU3OGM2Nzk0MWFiNWQyMzg0NDMwMDVhMzIiLCJmZmZmZTVmN2ZjYjlkOWYwYTNhZGRk
OGU3OGM2Nzk0MWFiNWQyMzg0NDMwMDY4MzcwMDQ1MjkiKS5tYXAoWF8pLEJiPXJiKE9iKSxZYj1u
ZXcgQXJyYXkoMykuY29uY2F0KCJmZmY3YmNmZWM0NGZkOTVmMGUiLCJmZmZmZDRmZWQ5OGVmZTk5
MjljYzRjMDIiLCJmZmZmZDRmZWQ5OGVmZTk5MjlkOTVmMGU5OTM0MDQiLCJmZmZmZDRmZWUzOTFm
ZWM0NGZmZTk5MjlkOTVmMGU5OTM0MDQiLCJmZmZmZDRmZWUzOTFmZWM0NGZmZTk5MjllYzcwMTRj
YzRjMDI4YzJkMDQiLCJmZmZmZTVmZmY3YmNmZWUzOTFmZWM0NGZmZTk5MjllYzcwMTRjYzRjMDI4
YzJkMDQiLCJmZmZmZTVmZmY3YmNmZWUzOTFmZWM0NGZmZTk5MjllYzcwMTRjYzRjMDI5OTM0MDQ2
NjI1MDYiKS5tYXAoWF8pLExiPXJiKFliKSxqYj1uZXcgQXJyYXkoMykuY29uY2F0KCJmZmVkYTBm
ZWIyNGNmMDNiMjAiLCJmZmZmYjJmZWNjNWNmZDhkM2NlMzFhMWMiLCJmZmZmYjJmZWNjNWNmZDhk
M2NmMDNiMjBiZDAwMjYiLCJmZmZmYjJmZWQ5NzZmZWIyNGNmZDhkM2NmMDNiMjBiZDAwMjYiLCJm
ZmZmYjJmZWQ5NzZmZWIyNGNmZDhkM2NmYzRlMmFlMzFhMWNiMTAwMjYiLCJmZmZmY2NmZmVkYTBm
ZWQ5NzZmZWIyNGNmZDhkM2NmYzRlMmFlMzFhMWNiMTAwMjYiLCJmZmZmY2NmZmVkYTBmZWQ5NzZm
ZWIyNGNmZDhkM2NmYzRlMmFlMzFhMWNiZDAwMjY4MDAwMjYiKS5tYXAoWF8pLEhiPXJiKGpiKSxY
Yj1uZXcgQXJyYXkoMykuY29uY2F0KCJkZWViZjc5ZWNhZTEzMTgyYmQiLCJlZmYzZmZiZGQ3ZTc2
YmFlZDYyMTcxYjUiLCJlZmYzZmZiZGQ3ZTc2YmFlZDYzMTgyYmQwODUxOWMiLCJlZmYzZmZjNmRi
ZWY5ZWNhZTE2YmFlZDYzMTgyYmQwODUxOWMiLCJlZmYzZmZjNmRiZWY5ZWNhZTE2YmFlZDY0Mjky
YzYyMTcxYjUwODQ1OTQiLCJmN2ZiZmZkZWViZjdjNmRiZWY5ZWNhZTE2YmFlZDY0MjkyYzYyMTcx
YjUwODQ1OTQiLCJmN2ZiZmZkZWViZjdjNmRiZWY5ZWNhZTE2YmFlZDY0MjkyYzYyMTcxYjUwODUx
OWMwODMwNmIiKS5tYXAoWF8pLEdiPXJiKFhiKSxWYj1uZXcgQXJyYXkoMykuY29uY2F0KCJlNWY1
ZTBhMWQ5OWIzMWEzNTQiLCJlZGY4ZTliYWU0YjM3NGM0NzYyMzhiNDUiLCJlZGY4ZTliYWU0YjM3
NGM0NzYzMWEzNTQwMDZkMmMiLCJlZGY4ZTljN2U5YzBhMWQ5OWI3NGM0NzYzMWEzNTQwMDZkMmMi
LCJlZGY4ZTljN2U5YzBhMWQ5OWI3NGM0NzY0MWFiNWQyMzhiNDUwMDVhMzIiLCJmN2ZjZjVlNWY1
ZTBjN2U5YzBhMWQ5OWI3NGM0NzY0MWFiNWQyMzhiNDUwMDVhMzIiLCJmN2ZjZjVlNWY1ZTBjN2U5
YzBhMWQ5OWI3NGM0NzY0MWFiNWQyMzhiNDUwMDZkMmMwMDQ0MWIiKS5tYXAoWF8pLFdiPXJiKFZi
KSxaYj1uZXcgQXJyYXkoMykuY29uY2F0KCJmMGYwZjBiZGJkYmQ2MzYzNjMiLCJmN2Y3ZjdjY2Nj
Y2M5Njk2OTY1MjUyNTIiLCJmN2Y3ZjdjY2NjY2M5Njk2OTY2MzYzNjMyNTI1MjUiLCJmN2Y3Zjdk
OWQ5ZDliZGJkYmQ5Njk2OTY2MzYzNjMyNTI1MjUiLCJmN2Y3ZjdkOWQ5ZDliZGJkYmQ5Njk2OTY3
MzczNzM1MjUyNTIyNTI1MjUiLCJmZmZmZmZmMGYwZjBkOWQ5ZDliZGJkYmQ5Njk2OTY3MzczNzM1
MjUyNTIyNTI1MjUiLCJmZmZmZmZmMGYwZjBkOWQ5ZDliZGJkYmQ5Njk2OTY3MzczNzM1MjUyNTIy
NTI1MjUwMDAwMDAiKS5tYXAoWF8pLEtiPXJiKFpiKSxRYj1uZXcgQXJyYXkoMykuY29uY2F0KCJl
ZmVkZjViY2JkZGM3NTZiYjEiLCJmMmYwZjdjYmM5ZTI5ZTlhYzg2YTUxYTMiLCJmMmYwZjdjYmM5
ZTI5ZTlhYzg3NTZiYjE1NDI3OGYiLCJmMmYwZjdkYWRhZWJiY2JkZGM5ZTlhYzg3NTZiYjE1NDI3
OGYiLCJmMmYwZjdkYWRhZWJiY2JkZGM5ZTlhYzg4MDdkYmE2YTUxYTM0YTE0ODYiLCJmY2ZiZmRl
ZmVkZjVkYWRhZWJiY2JkZGM5ZTlhYzg4MDdkYmE2YTUxYTM0YTE0ODYiLCJmY2ZiZmRlZmVkZjVk
YWRhZWJiY2JkZGM5ZTlhYzg4MDdkYmE2YTUxYTM1NDI3OGYzZjAwN2QiKS5tYXAoWF8pLEpiPXJi
KFFiKSx0bT1uZXcgQXJyYXkoMykuY29uY2F0KCJmZWUwZDJmYzkyNzJkZTJkMjYiLCJmZWU1ZDlm
Y2FlOTFmYjZhNGFjYjE4MWQiLCJmZWU1ZDlmY2FlOTFmYjZhNGFkZTJkMjZhNTBmMTUiLCJmZWU1
ZDlmY2JiYTFmYzkyNzJmYjZhNGFkZTJkMjZhNTBmMTUiLCJmZWU1ZDlmY2JiYTFmYzkyNzJmYjZh
NGFlZjNiMmNjYjE4MWQ5OTAwMGQiLCJmZmY1ZjBmZWUwZDJmY2JiYTFmYzkyNzJmYjZhNGFlZjNi
MmNjYjE4MWQ5OTAwMGQiLCJmZmY1ZjBmZWUwZDJmY2JiYTFmYzkyNzJmYjZhNGFlZjNiMmNjYjE4
MWRhNTBmMTU2NzAwMGQiKS5tYXAoWF8pLG5tPXJiKHRtKSxlbT1uZXcgQXJyYXkoMykuY29uY2F0
KCJmZWU2Y2VmZGFlNmJlNjU1MGQiLCJmZWVkZGVmZGJlODVmZDhkM2NkOTQ3MDEiLCJmZWVkZGVm
ZGJlODVmZDhkM2NlNjU1MGRhNjM2MDMiLCJmZWVkZGVmZGQwYTJmZGFlNmJmZDhkM2NlNjU1MGRh
NjM2MDMiLCJmZWVkZGVmZGQwYTJmZGFlNmJmZDhkM2NmMTY5MTNkOTQ4MDE4YzJkMDQiLCJmZmY1
ZWJmZWU2Y2VmZGQwYTJmZGFlNmJmZDhkM2NmMTY5MTNkOTQ4MDE4YzJkMDQiLCJmZmY1ZWJmZWU2
Y2VmZGQwYTJmZGFlNmJmZDhkM2NmMTY5MTNkOTQ4MDFhNjM2MDM3ZjI3MDQiKS5tYXAoWF8pLHJt
PXJiKGVtKTt2YXIgaW09ZGkoQXIoMzAwLC41LDApLEFyKC0yNDAsLjUsMSkpLG9tPWRpKEFyKC0x
MDAsLjc1LC4zNSksQXIoODAsMS41LC44KSksYW09ZGkoQXIoMjYwLC43NSwuMzUpLEFyKDgwLDEu
NSwuOCkpLHVtPUFyKCk7dmFyIGNtPXFlKCksZm09TWF0aC5QSS8zLHNtPTIqTWF0aC5QSS8zO2Z1
bmN0aW9uIGxtKHQpe3ZhciBuPXQubGVuZ3RoO3JldHVybiBmdW5jdGlvbihlKXtyZXR1cm4gdFtN
YXRoLm1heCgwLE1hdGgubWluKG4tMSxNYXRoLmZsb29yKGUqbikpKV19fXZhciBobT1sbShYXygi
NDQwMTU0NDQwMjU2NDUwNDU3NDUwNTU5NDYwNzVhNDYwODVjNDYwYTVkNDYwYjVlNDcwZDYwNDcw
ZTYxNDcxMDYzNDcxMTY0NDcxMzY1NDgxNDY3NDgxNjY4NDgxNzY5NDgxODZhNDgxYTZjNDgxYjZk
NDgxYzZlNDgxZDZmNDgxZjcwNDgyMDcxNDgyMTczNDgyMzc0NDgyNDc1NDgyNTc2NDgyNjc3NDgy
ODc4NDgyOTc5NDcyYTdhNDcyYzdhNDcyZDdiNDcyZTdjNDcyZjdkNDYzMDdlNDYzMjdlNDYzMzdm
NDYzNDgwNDUzNTgxNDUzNzgxNDUzODgyNDQzOTgzNDQzYTgzNDQzYjg0NDMzZDg0NDMzZTg1NDIz
Zjg1NDI0MDg2NDI0MTg2NDE0Mjg3NDE0NDg3NDA0NTg4NDA0Njg4M2Y0Nzg4M2Y0ODg5M2U0OTg5
M2U0YTg5M2U0YzhhM2Q0ZDhhM2Q0ZThhM2M0ZjhhM2M1MDhiM2I1MThiM2I1MjhiM2E1MzhiM2E1
NDhjMzk1NThjMzk1NjhjMzg1ODhjMzg1OThjMzc1YThjMzc1YjhkMzY1YzhkMzY1ZDhkMzU1ZThk
MzU1ZjhkMzQ2MDhkMzQ2MThkMzM2MjhkMzM2MzhkMzI2NDhlMzI2NThlMzE2NjhlMzE2NzhlMzE2
ODhlMzA2OThlMzA2YThlMmY2YjhlMmY2YzhlMmU2ZDhlMmU2ZThlMmU2ZjhlMmQ3MDhlMmQ3MThl
MmM3MThlMmM3MjhlMmM3MzhlMmI3NDhlMmI3NThlMmE3NjhlMmE3NzhlMmE3ODhlMjk3OThlMjk3
YThlMjk3YjhlMjg3YzhlMjg3ZDhlMjc3ZThlMjc3ZjhlMjc4MDhlMjY4MThlMjY4MjhlMjY4Mjhl
MjU4MzhlMjU4NDhlMjU4NThlMjQ4NjhlMjQ4NzhlMjM4ODhlMjM4OThlMjM4YThkMjI4YjhkMjI4
YzhkMjI4ZDhkMjE4ZThkMjE4ZjhkMjE5MDhkMjE5MThjMjA5MjhjMjA5MjhjMjA5MzhjMWY5NDhj
MWY5NThiMWY5NjhiMWY5NzhiMWY5ODhiMWY5OThhMWY5YThhMWU5YjhhMWU5Yzg5MWU5ZDg5MWY5
ZTg5MWY5Zjg4MWZhMDg4MWZhMTg4MWZhMTg3MWZhMjg3MjBhMzg2MjBhNDg2MjFhNTg1MjFhNjg1
MjJhNzg1MjJhODg0MjNhOTgzMjRhYTgzMjVhYjgyMjVhYzgyMjZhZDgxMjdhZDgxMjhhZTgwMjlh
ZjdmMmFiMDdmMmNiMTdlMmRiMjdkMmViMzdjMmZiNDdjMzFiNTdiMzJiNjdhMzRiNjc5MzViNzc5
MzdiODc4MzhiOTc3M2FiYTc2M2JiYjc1M2RiYzc0M2ZiYzczNDBiZDcyNDJiZTcxNDRiZjcwNDZj
MDZmNDhjMTZlNGFjMTZkNGNjMjZjNGVjMzZiNTBjNDZhNTJjNTY5NTRjNTY4NTZjNjY3NThjNzY1
NWFjODY0NWNjODYzNWVjOTYyNjBjYTYwNjNjYjVmNjVjYjVlNjdjYzVjNjljZDViNmNjZDVhNmVj
ZTU4NzBjZjU3NzNkMDU2NzVkMDU0NzdkMTUzN2FkMTUxN2NkMjUwN2ZkMzRlODFkMzRkODRkNDRi
ODZkNTQ5ODlkNTQ4OGJkNjQ2OGVkNjQ1OTBkNzQzOTNkNzQxOTVkODQwOThkODNlOWJkOTNjOWRk
OTNiYTBkYTM5YTJkYTM3YTVkYjM2YThkYjM0YWFkYzMyYWRkYzMwYjBkZDJmYjJkZDJkYjVkZTJi
YjhkZTI5YmFkZTI4YmRkZjI2YzBkZjI1YzJkZjIzYzVlMDIxYzhlMDIwY2FlMTFmY2RlMTFkZDBl
MTFjZDJlMjFiZDVlMjFhZDhlMjE5ZGFlMzE5ZGRlMzE4ZGZlMzE4ZTJlNDE4ZTVlNDE5ZTdlNDE5
ZWFlNTFhZWNlNTFiZWZlNTFjZjFlNTFkZjRlNjFlZjZlNjIwZjhlNjIxZmJlNzIzZmRlNzI1Iikp
LGRtPWxtKFhfKCIwMDAwMDQwMTAwMDUwMTAxMDYwMTAxMDgwMjAxMDkwMjAyMGIwMjAyMGQwMzAz
MGYwMzAzMTIwNDA0MTQwNTA0MTYwNjA1MTgwNjA1MWEwNzA2MWMwODA3MWUwOTA3MjAwYTA4MjIw
YjA5MjQwYzA5MjYwZDBhMjkwZTBiMmIxMDBiMmQxMTBjMmYxMjBkMzExMzBkMzQxNDBlMzYxNTBl
MzgxNjBmM2IxODBmM2QxOTEwM2YxYTEwNDIxYzEwNDQxZDExNDcxZTExNDkyMDExNGIyMTExNGUy
MjExNTAyNDEyNTMyNTEyNTUyNzEyNTgyOTExNWEyYTExNWMyYzExNWYyZDExNjEyZjExNjMzMTEx
NjUzMzEwNjczNDEwNjkzNjEwNmIzODEwNmMzOTBmNmUzYjBmNzAzZDBmNzEzZjBmNzI0MDBmNzQ0
MjBmNzU0NDBmNzY0NTEwNzc0NzEwNzg0OTEwNzg0YTEwNzk0YzExN2E0ZTExN2I0ZjEyN2I1MTEy
N2M1MjEzN2M1NDEzN2Q1NjE0N2Q1NzE1N2U1OTE1N2U1YTE2N2U1YzE2N2Y1ZDE3N2Y1ZjE4N2Y2
MDE4ODA2MjE5ODA2NDFhODA2NTFhODA2NzFiODA2ODFjODE2YTFjODE2YjFkODE2ZDFkODE2ZTFl
ODE3MDFmODE3MjFmODE3MzIwODE3NTIxODE3NjIxODE3ODIyODE3OTIyODI3YjIzODI3YzIzODI3
ZTI0ODI4MDI1ODI4MTI1ODE4MzI2ODE4NDI2ODE4NjI3ODE4ODI3ODE4OTI4ODE4YjI5ODE4YzI5
ODE4ZTJhODE5MDJhODE5MTJiODE5MzJiODA5NDJjODA5NjJjODA5ODJkODA5OTJkODA5YjJlN2Y5
YzJlN2Y5ZTJmN2ZhMDJmN2ZhMTMwN2VhMzMwN2VhNTMxN2VhNjMxN2RhODMyN2RhYTMzN2RhYjMz
N2NhZDM0N2NhZTM0N2JiMDM1N2JiMjM1N2JiMzM2N2FiNTM2N2FiNzM3NzliODM3NzliYTM4Nzhi
YzM5NzhiZDM5NzdiZjNhNzdjMDNhNzZjMjNiNzVjNDNjNzVjNTNjNzRjNzNkNzNjODNlNzNjYTNl
NzJjYzNmNzFjZDQwNzFjZjQwNzBkMDQxNmZkMjQyNmZkMzQzNmVkNTQ0NmRkNjQ1NmNkODQ1NmNk
OTQ2NmJkYjQ3NmFkYzQ4NjlkZTQ5NjhkZjRhNjhlMDRjNjdlMjRkNjZlMzRlNjVlNDRmNjRlNTUw
NjRlNzUyNjNlODUzNjJlOTU0NjJlYTU2NjFlYjU3NjBlYzU4NjBlZDVhNWZlZTViNWVlZjVkNWVm
MDVmNWVmMTYwNWRmMjYyNWRmMjY0NWNmMzY1NWNmNDY3NWNmNDY5NWNmNTZiNWNmNjZjNWNmNjZl
NWNmNzcwNWNmNzcyNWNmODc0NWNmODc2NWNmOTc4NWRmOTc5NWRmOTdiNWRmYTdkNWVmYTdmNWVm
YTgxNWZmYjgzNWZmYjg1NjBmYjg3NjFmYzg5NjFmYzhhNjJmYzhjNjNmYzhlNjRmYzkwNjVmZDky
NjZmZDk0NjdmZDk2NjhmZDk4NjlmZDlhNmFmZDliNmJmZTlkNmNmZTlmNmRmZWExNmVmZWEzNmZm
ZWE1NzFmZWE3NzJmZWE5NzNmZWFhNzRmZWFjNzZmZWFlNzdmZWIwNzhmZWIyN2FmZWI0N2JmZWI2
N2NmZWI3N2VmZWI5N2ZmZWJiODFmZWJkODJmZWJmODRmZWMxODVmZWMyODdmZWM0ODhmZWM2OGFm
ZWM4OGNmZWNhOGRmZWNjOGZmZWNkOTBmZWNmOTJmZWQxOTRmZWQzOTVmZWQ1OTdmZWQ3OTlmZWQ4
OWFmZGRhOWNmZGRjOWVmZGRlYTBmZGUwYTFmZGUyYTNmZGUzYTVmZGU1YTdmZGU3YTlmZGU5YWFm
ZGViYWNmY2VjYWVmY2VlYjBmY2YwYjJmY2YyYjRmY2Y0YjZmY2Y2YjhmY2Y3YjlmY2Y5YmJmY2Zi
YmRmY2ZkYmYiKSkscG09bG0oWF8oIjAwMDAwNDAxMDAwNTAxMDEwNjAxMDEwODAyMDEwYTAyMDIw
YzAyMDIwZTAzMDIxMDA0MDMxMjA0MDMxNDA1MDQxNzA2MDQxOTA3MDUxYjA4MDUxZDA5MDYxZjBh
MDcyMjBiMDcyNDBjMDgyNjBkMDgyOTBlMDkyYjEwMDkyZDExMGEzMDEyMGEzMjE0MGIzNDE1MGIz
NzE2MGIzOTE4MGMzYzE5MGMzZTFiMGM0MTFjMGM0MzFlMGM0NTFmMGM0ODIxMGM0YTIzMGM0YzI0
MGM0ZjI2MGM1MTI4MGI1MzI5MGI1NTJiMGI1NzJkMGI1OTJmMGE1YjMxMGE1YzMyMGE1ZTM0MGE1
ZjM2MDk2MTM4MDk2MjM5MDk2MzNiMDk2NDNkMDk2NTNlMDk2NjQwMGE2NzQyMGE2ODQ0MGE2ODQ1
MGE2OTQ3MGI2YTQ5MGI2YTRhMGM2YjRjMGM2YjRkMGQ2YzRmMGQ2YzUxMGU2YzUyMGU2ZDU0MGY2
ZDU1MGY2ZDU3MTA2ZTU5MTA2ZTVhMTE2ZTVjMTI2ZTVkMTI2ZTVmMTM2ZTYxMTM2ZTYyMTQ2ZTY0
MTU2ZTY1MTU2ZTY3MTY2ZTY5MTY2ZTZhMTc2ZTZjMTg2ZTZkMTg2ZTZmMTk2ZTcxMTk2ZTcyMWE2
ZTc0MWE2ZTc1MWI2ZTc3MWM2ZDc4MWM2ZDdhMWQ2ZDdjMWQ2ZDdkMWU2ZDdmMWU2YzgwMWY2Yzgy
MjA2Yzg0MjA2Yjg1MjE2Yjg3MjE2Yjg4MjI2YThhMjI2YThjMjM2OThkMjM2OThmMjQ2OTkwMjU2
ODkyMjU2ODkzMjY2Nzk1MjY2Nzk3Mjc2Njk4Mjc2NjlhMjg2NTliMjk2NDlkMjk2NDlmMmE2M2Ew
MmE2M2EyMmI2MmEzMmM2MWE1MmM2MGE2MmQ2MGE4MmU1ZmE5MmU1ZWFiMmY1ZWFkMzA1ZGFlMzA1
Y2IwMzE1YmIxMzI1YWIzMzI1YWI0MzM1OWI2MzQ1OGI3MzU1N2I5MzU1NmJhMzY1NWJjMzc1NGJk
Mzg1M2JmMzk1MmMwM2E1MWMxM2E1MGMzM2I0ZmM0M2M0ZWM2M2Q0ZGM3M2U0Y2M4M2Y0YmNhNDA0
YWNiNDE0OWNjNDI0OGNlNDM0N2NmNDQ0NmQwNDU0NWQyNDY0NGQzNDc0M2Q0NDg0MmQ1NGE0MWQ3
NGIzZmQ4NGMzZWQ5NGQzZGRhNGUzY2RiNTAzYmRkNTEzYWRlNTIzOGRmNTMzN2UwNTUzNmUxNTYz
NWUyNTczNGUzNTkzM2U0NWEzMWU1NWMzMGU2NWQyZmU3NWUyZWU4NjAyZGU5NjEyYmVhNjMyYWVi
NjQyOWViNjYyOGVjNjcyNmVkNjkyNWVlNmEyNGVmNmMyM2VmNmUyMWYwNmYyMGYxNzExZmYxNzMx
ZGYyNzQxY2YzNzYxYmYzNzgxOWY0NzkxOGY1N2IxN2Y1N2QxNWY2N2UxNGY2ODAxM2Y3ODIxMmY3
ODQxMGY4ODUwZmY4ODcwZWY4ODkwY2Y5OGIwYmY5OGMwYWY5OGUwOWZhOTAwOGZhOTIwN2ZhOTQw
N2ZiOTYwNmZiOTcwNmZiOTkwNmZiOWIwNmZiOWQwN2ZjOWYwN2ZjYTEwOGZjYTMwOWZjYTUwYWZj
YTYwY2ZjYTgwZGZjYWEwZmZjYWMxMWZjYWUxMmZjYjAxNGZjYjIxNmZjYjQxOGZiYjYxYWZiYjgx
ZGZiYmExZmZiYmMyMWZiYmUyM2ZhYzAyNmZhYzIyOGZhYzQyYWZhYzYyZGY5YzcyZmY5YzkzMmY5
Y2IzNWY4Y2QzN2Y4Y2YzYWY3ZDEzZGY3ZDM0MGY2ZDU0M2Y2ZDc0NmY1ZDk0OWY1ZGI0Y2Y0ZGQ0
ZmY0ZGY1M2Y0ZTE1NmYzZTM1YWYzZTU1ZGYyZTY2MWYyZTg2NWYyZWE2OWYxZWM2ZGYxZWQ3MWYx
ZWY3NWYxZjE3OWYyZjI3ZGYyZjQ4MmYzZjU4NmYzZjY4YWY0Zjg4ZWY1Zjk5MmY2ZmE5NmY4ZmI5
YWY5ZmM5ZGZhZmRhMWZjZmZhNCIpKSxnbT1sbShYXygiMGQwODg3MTAwNzg4MTMwNzg5MTYwNzhh
MTkwNjhjMWIwNjhkMWQwNjhlMjAwNjhmMjIwNjkwMjQwNjkxMjYwNTkxMjgwNTkyMmEwNTkzMmMw
NTk0MmUwNTk1MmYwNTk2MzEwNTk3MzMwNTk3MzUwNDk4MzcwNDk5MzgwNDlhM2EwNDlhM2MwNDli
M2UwNDljM2YwNDljNDEwNDlkNDMwMzllNDQwMzllNDYwMzlmNDgwMzlmNDkwM2EwNGIwM2ExNGMw
MmExNGUwMmEyNTAwMmEyNTEwMmEzNTMwMmEzNTUwMmE0NTYwMWE0NTgwMWE0NTkwMWE1NWIwMWE1
NWMwMWE2NWUwMWE2NjAwMWE2NjEwMGE3NjMwMGE3NjQwMGE3NjYwMGE3NjcwMGE4NjkwMGE4NmEw
MGE4NmMwMGE4NmUwMGE4NmYwMGE4NzEwMGE4NzIwMWE4NzQwMWE4NzUwMWE4NzcwMWE4NzgwMWE4
N2EwMmE4N2IwMmE4N2QwM2E4N2UwM2E4ODAwNGE4ODEwNGE3ODMwNWE3ODQwNWE3ODYwNmE2ODcw
N2E2ODgwOGE2OGEwOWE1OGIwYWE1OGQwYmE1OGUwY2E0OGYwZGE0OTEwZWEzOTIwZmEzOTQxMGEy
OTUxMWExOTYxM2ExOTgxNGEwOTkxNTlmOWExNjlmOWMxNzllOWQxODlkOWUxOTlkYTAxYTljYTEx
YjliYTIxZDlhYTMxZTlhYTUxZjk5YTYyMDk4YTcyMTk3YTgyMjk2YWEyMzk1YWIyNDk0YWMyNjk0
YWQyNzkzYWUyODkyYjAyOTkxYjEyYTkwYjIyYjhmYjMyYzhlYjQyZThkYjUyZjhjYjYzMDhiYjcz
MThhYjgzMjg5YmEzMzg4YmIzNDg4YmMzNTg3YmQzNzg2YmUzODg1YmYzOTg0YzAzYTgzYzEzYjgy
YzIzYzgxYzMzZDgwYzQzZTdmYzU0MDdlYzY0MTdkYzc0MjdjYzg0MzdiYzk0NDdhY2E0NTdhY2I0
Njc5Y2M0Nzc4Y2M0OTc3Y2Q0YTc2Y2U0Yjc1Y2Y0Yzc0ZDA0ZDczZDE0ZTcyZDI0ZjcxZDM1MTcx
ZDQ1MjcwZDU1MzZmZDU1NDZlZDY1NTZkZDc1NjZjZDg1NzZiZDk1ODZhZGE1YTZhZGE1YjY5ZGI1
YzY4ZGM1ZDY3ZGQ1ZTY2ZGU1ZjY1ZGU2MTY0ZGY2MjYzZTA2MzYzZTE2NDYyZTI2NTYxZTI2NjYw
ZTM2ODVmZTQ2OTVlZTU2YTVkZTU2YjVkZTY2YzVjZTc2ZTViZTc2ZjVhZTg3MDU5ZTk3MTU4ZTk3
MjU3ZWE3NDU3ZWI3NTU2ZWI3NjU1ZWM3NzU0ZWQ3OTUzZWQ3YTUyZWU3YjUxZWY3YzUxZWY3ZTUw
ZjA3ZjRmZjA4MDRlZjE4MTRkZjE4MzRjZjI4NDRiZjM4NTRiZjM4NzRhZjQ4ODQ5ZjQ4OTQ4ZjU4
YjQ3ZjU4YzQ2ZjY4ZDQ1ZjY4ZjQ0Zjc5MDQ0Zjc5MTQzZjc5MzQyZjg5NDQxZjg5NTQwZjk5NzNm
Zjk5ODNlZjk5YTNlZmE5YjNkZmE5YzNjZmE5ZTNiZmI5ZjNhZmJhMTM5ZmJhMjM4ZmNhMzM4ZmNh
NTM3ZmNhNjM2ZmNhODM1ZmNhOTM0ZmRhYjMzZmRhYzMzZmRhZTMyZmRhZjMxZmRiMTMwZmRiMjJm
ZmRiNDJmZmRiNTJlZmViNzJkZmViODJjZmViYTJjZmViYjJiZmViZDJhZmViZTJhZmVjMDI5ZmRj
MjI5ZmRjMzI4ZmRjNTI3ZmRjNjI3ZmRjODI3ZmRjYTI2ZmRjYjI2ZmNjZDI1ZmNjZTI1ZmNkMDI1
ZmNkMjI1ZmJkMzI0ZmJkNTI0ZmJkNzI0ZmFkODI0ZmFkYTI0ZjlkYzI0ZjlkZDI1ZjhkZjI1Zjhl
MTI1ZjdlMjI1ZjdlNDI1ZjZlNjI2ZjZlODI2ZjVlOTI2ZjVlYjI3ZjRlZDI3ZjNlZTI3ZjNmMDI3
ZjJmMjI3ZjFmNDI2ZjFmNTI1ZjBmNzI0ZjBmOTIxIikpO2Z1bmN0aW9uIHltKHQpe3JldHVybiBm
dW5jdGlvbigpe3JldHVybiB0fX1jb25zdCB2bT1NYXRoLmFicyxfbT1NYXRoLmF0YW4yLGJtPU1h
dGguY29zLG1tPU1hdGgubWF4LHhtPU1hdGgubWluLHdtPU1hdGguc2luLE1tPU1hdGguc3FydCxU
bT0xZS0xMixBbT1NYXRoLlBJLFNtPUFtLzIsRW09MipBbTtmdW5jdGlvbiBObSh0KXtyZXR1cm4g
dD49MT9TbTp0PD0tMT8tU206TWF0aC5hc2luKHQpfWZ1bmN0aW9uIGttKHQpe2xldCBuPTM7cmV0
dXJuIHQuZGlnaXRzPWZ1bmN0aW9uKGUpe2lmKCFhcmd1bWVudHMubGVuZ3RoKXJldHVybiBuO2lm
KG51bGw9PWUpbj1udWxsO2Vsc2V7Y29uc3QgdD1NYXRoLmZsb29yKGUpO2lmKCEodD49MCkpdGhy
b3cgbmV3IFJhbmdlRXJyb3IoYGludmFsaWQgZGlnaXRzOiAke2V9YCk7bj10fXJldHVybiB0fSwo
KT0+bmV3IElhKG4pfWZ1bmN0aW9uIENtKHQpe3JldHVybiB0LmlubmVyUmFkaXVzfWZ1bmN0aW9u
IFBtKHQpe3JldHVybiB0Lm91dGVyUmFkaXVzfWZ1bmN0aW9uIHptKHQpe3JldHVybiB0LnN0YXJ0
QW5nbGV9ZnVuY3Rpb24gJG0odCl7cmV0dXJuIHQuZW5kQW5nbGV9ZnVuY3Rpb24gRG0odCl7cmV0
dXJuIHQmJnQucGFkQW5nbGV9ZnVuY3Rpb24gUm0odCxuLGUscixpLG8sYSl7dmFyIHU9dC1lLGM9
bi1yLGY9KGE/bzotbykvTW0odSp1K2MqYykscz1mKmMsbD0tZip1LGg9dCtzLGQ9bitsLHA9ZStz
LGc9citsLHk9KGgrcCkvMix2PShkK2cpLzIsXz1wLWgsYj1nLWQsbT1fKl8rYipiLHg9aS1vLHc9
aCpnLXAqZCxNPShiPDA/LTE6MSkqTW0obW0oMCx4KngqbS13KncpKSxUPSh3KmItXypNKS9tLEE9
KC13Kl8tYipNKS9tLFM9KHcqYitfKk0pL20sRT0oLXcqXytiKk0pL20sTj1ULXksaz1BLXYsQz1T
LXksUD1FLXY7cmV0dXJuIE4qTitrKms+QypDK1AqUCYmKFQ9UyxBPUUpLHtjeDpULGN5OkEseDAx
Oi1zLHkwMTotbCx4MTE6VCooaS94LTEpLHkxMTpBKihpL3gtMSl9fXZhciBGbT1BcnJheS5wcm90
b3R5cGUuc2xpY2U7ZnVuY3Rpb24gcW0odCl7cmV0dXJuIm9iamVjdCI9PXR5cGVvZiB0JiYibGVu
Z3RoImluIHQ/dDpBcnJheS5mcm9tKHQpfWZ1bmN0aW9uIFVtKHQpe3RoaXMuX2NvbnRleHQ9dH1m
dW5jdGlvbiBJbSh0KXtyZXR1cm4gbmV3IFVtKHQpfWZ1bmN0aW9uIE9tKHQpe3JldHVybiB0WzBd
fWZ1bmN0aW9uIEJtKHQpe3JldHVybiB0WzFdfWZ1bmN0aW9uIFltKHQsbil7dmFyIGU9eW0oITAp
LHI9bnVsbCxpPUltLG89bnVsbCxhPWttKHUpO2Z1bmN0aW9uIHUodSl7dmFyIGMsZixzLGw9KHU9
cW0odSkpLmxlbmd0aCxoPSExO2ZvcihudWxsPT1yJiYobz1pKHM9YSgpKSksYz0wO2M8PWw7Kytj
KSEoYzxsJiZlKGY9dVtjXSxjLHUpKT09PWgmJigoaD0haCk/by5saW5lU3RhcnQoKTpvLmxpbmVF
bmQoKSksaCYmby5wb2ludCgrdChmLGMsdSksK24oZixjLHUpKTtpZihzKXJldHVybiBvPW51bGws
cysiInx8bnVsbH1yZXR1cm4gdD0iZnVuY3Rpb24iPT10eXBlb2YgdD90OnZvaWQgMD09PXQ/T206
eW0odCksbj0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOnZvaWQgMD09PW4/Qm06eW0obiksdS54PWZ1
bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh0PSJmdW5jdGlvbiI9PXR5cGVvZiBu
P246eW0oK24pLHUpOnR9LHUueT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8o
bj0iZnVuY3Rpb24iPT10eXBlb2YgdD90OnltKCt0KSx1KTpufSx1LmRlZmluZWQ9ZnVuY3Rpb24o
dCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGU9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp5bSgh
IXQpLHUpOmV9LHUuY3VydmU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGk9
dCxudWxsIT1yJiYobz1pKHIpKSx1KTppfSx1LmNvbnRleHQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFy
Z3VtZW50cy5sZW5ndGg/KG51bGw9PXQ/cj1vPW51bGw6bz1pKHI9dCksdSk6cn0sdX1mdW5jdGlv
biBMbSh0LG4sZSl7dmFyIHI9bnVsbCxpPXltKCEwKSxvPW51bGwsYT1JbSx1PW51bGwsYz1rbShm
KTtmdW5jdGlvbiBmKGYpe3ZhciBzLGwsaCxkLHAsZz0oZj1xbShmKSkubGVuZ3RoLHk9ITEsdj1u
ZXcgQXJyYXkoZyksXz1uZXcgQXJyYXkoZyk7Zm9yKG51bGw9PW8mJih1PWEocD1jKCkpKSxzPTA7
czw9ZzsrK3Mpe2lmKCEoczxnJiZpKGQ9ZltzXSxzLGYpKT09PXkpaWYoeT0heSlsPXMsdS5hcmVh
U3RhcnQoKSx1LmxpbmVTdGFydCgpO2Vsc2V7Zm9yKHUubGluZUVuZCgpLHUubGluZVN0YXJ0KCks
aD1zLTE7aD49bDstLWgpdS5wb2ludCh2W2hdLF9baF0pO3UubGluZUVuZCgpLHUuYXJlYUVuZCgp
fXkmJih2W3NdPSt0KGQscyxmKSxfW3NdPStuKGQscyxmKSx1LnBvaW50KHI/K3IoZCxzLGYpOnZb
c10sZT8rZShkLHMsZik6X1tzXSkpfWlmKHApcmV0dXJuIHU9bnVsbCxwKyIifHxudWxsfWZ1bmN0
aW9uIHMoKXtyZXR1cm4gWW0oKS5kZWZpbmVkKGkpLmN1cnZlKGEpLmNvbnRleHQobyl9cmV0dXJu
IHQ9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp2b2lkIDA9PT10P09tOnltKCt0KSxuPSJmdW5jdGlv
biI9PXR5cGVvZiBuP246eW0odm9pZCAwPT09bj8wOituKSxlPSJmdW5jdGlvbiI9PXR5cGVvZiBl
P2U6dm9pZCAwPT09ZT9CbTp5bSgrZSksZi54PWZ1bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPyh0PSJmdW5jdGlvbiI9PXR5cGVvZiBuP246eW0oK24pLHI9bnVsbCxmKTp0fSxmLngw
PWZ1bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh0PSJmdW5jdGlvbiI9PXR5cGVv
ZiBuP246eW0oK24pLGYpOnR9LGYueDE9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5n
dGg/KHI9bnVsbD09dD9udWxsOiJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6eW0oK3QpLGYpOnJ9LGYu
eT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obj0iZnVuY3Rpb24iPT10eXBl
b2YgdD90OnltKCt0KSxlPW51bGwsZik6bn0sZi55MD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1l
bnRzLmxlbmd0aD8obj0iZnVuY3Rpb24iPT10eXBlb2YgdD90OnltKCt0KSxmKTpufSxmLnkxPWZ1
bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlPW51bGw9PXQ/bnVsbDoiZnVuY3Rp
b24iPT10eXBlb2YgdD90OnltKCt0KSxmKTplfSxmLmxpbmVYMD1mLmxpbmVZMD1mdW5jdGlvbigp
e3JldHVybiBzKCkueCh0KS55KG4pfSxmLmxpbmVZMT1mdW5jdGlvbigpe3JldHVybiBzKCkueCh0
KS55KGUpfSxmLmxpbmVYMT1mdW5jdGlvbigpe3JldHVybiBzKCkueChyKS55KG4pfSxmLmRlZmlu
ZWQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGk9ImZ1bmN0aW9uIj09dHlw
ZW9mIHQ/dDp5bSghIXQpLGYpOml9LGYuY3VydmU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50
cy5sZW5ndGg/KGE9dCxudWxsIT1vJiYodT1hKG8pKSxmKTphfSxmLmNvbnRleHQ9ZnVuY3Rpb24o
dCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG51bGw9PXQ/bz11PW51bGw6dT1hKG89dCksZik6
b30sZn1mdW5jdGlvbiBqbSh0LG4pe3JldHVybiBuPHQ/LTE6bj50PzE6bj49dD8wOk5hTn1mdW5j
dGlvbiBIbSh0KXtyZXR1cm4gdH1VbS5wcm90b3R5cGU9e2FyZWFTdGFydDpmdW5jdGlvbigpe3Ro
aXMuX2xpbmU9MH0sYXJlYUVuZDpmdW5jdGlvbigpe3RoaXMuX2xpbmU9TmFOfSxsaW5lU3RhcnQ6
ZnVuY3Rpb24oKXt0aGlzLl9wb2ludD0wfSxsaW5lRW5kOmZ1bmN0aW9uKCl7KHRoaXMuX2xpbmV8
fDAhPT10aGlzLl9saW5lJiYxPT09dGhpcy5fcG9pbnQpJiZ0aGlzLl9jb250ZXh0LmNsb3NlUGF0
aCgpLHRoaXMuX2xpbmU9MS10aGlzLl9saW5lfSxwb2ludDpmdW5jdGlvbih0LG4pe3N3aXRjaCh0
PSt0LG49K24sdGhpcy5fcG9pbnQpe2Nhc2UgMDp0aGlzLl9wb2ludD0xLHRoaXMuX2xpbmU/dGhp
cy5fY29udGV4dC5saW5lVG8odCxuKTp0aGlzLl9jb250ZXh0Lm1vdmVUbyh0LG4pO2JyZWFrO2Nh
c2UgMTp0aGlzLl9wb2ludD0yO2RlZmF1bHQ6dGhpcy5fY29udGV4dC5saW5lVG8odCxuKX19fTt2
YXIgWG09Vm0oSW0pO2Z1bmN0aW9uIEdtKHQpe3RoaXMuX2N1cnZlPXR9ZnVuY3Rpb24gVm0odCl7
ZnVuY3Rpb24gbihuKXtyZXR1cm4gbmV3IEdtKHQobikpfXJldHVybiBuLl9jdXJ2ZT10LG59ZnVu
Y3Rpb24gV20odCl7dmFyIG49dC5jdXJ2ZTtyZXR1cm4gdC5hbmdsZT10LngsZGVsZXRlIHQueCx0
LnJhZGl1cz10LnksZGVsZXRlIHQueSx0LmN1cnZlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVu
dHMubGVuZ3RoP24oVm0odCkpOm4oKS5fY3VydmV9LHR9ZnVuY3Rpb24gWm0oKXtyZXR1cm4gV20o
WW0oKS5jdXJ2ZShYbSkpfWZ1bmN0aW9uIEttKCl7dmFyIHQ9TG0oKS5jdXJ2ZShYbSksbj10LmN1
cnZlLGU9dC5saW5lWDAscj10LmxpbmVYMSxpPXQubGluZVkwLG89dC5saW5lWTE7cmV0dXJuIHQu
YW5nbGU9dC54LGRlbGV0ZSB0LngsdC5zdGFydEFuZ2xlPXQueDAsZGVsZXRlIHQueDAsdC5lbmRB
bmdsZT10LngxLGRlbGV0ZSB0LngxLHQucmFkaXVzPXQueSxkZWxldGUgdC55LHQuaW5uZXJSYWRp
dXM9dC55MCxkZWxldGUgdC55MCx0Lm91dGVyUmFkaXVzPXQueTEsZGVsZXRlIHQueTEsdC5saW5l
U3RhcnRBbmdsZT1mdW5jdGlvbigpe3JldHVybiBXbShlKCkpfSxkZWxldGUgdC5saW5lWDAsdC5s
aW5lRW5kQW5nbGU9ZnVuY3Rpb24oKXtyZXR1cm4gV20ocigpKX0sZGVsZXRlIHQubGluZVgxLHQu
bGluZUlubmVyUmFkaXVzPWZ1bmN0aW9uKCl7cmV0dXJuIFdtKGkoKSl9LGRlbGV0ZSB0LmxpbmVZ
MCx0LmxpbmVPdXRlclJhZGl1cz1mdW5jdGlvbigpe3JldHVybiBXbShvKCkpfSxkZWxldGUgdC5s
aW5lWTEsdC5jdXJ2ZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD9uKFZtKHQp
KTpuKCkuX2N1cnZlfSx0fWZ1bmN0aW9uIFFtKHQsbil7cmV0dXJuWyhuPStuKSpNYXRoLmNvcyh0
LT1NYXRoLlBJLzIpLG4qTWF0aC5zaW4odCldfUdtLnByb3RvdHlwZT17YXJlYVN0YXJ0OmZ1bmN0
aW9uKCl7dGhpcy5fY3VydmUuYXJlYVN0YXJ0KCl9LGFyZWFFbmQ6ZnVuY3Rpb24oKXt0aGlzLl9j
dXJ2ZS5hcmVhRW5kKCl9LGxpbmVTdGFydDpmdW5jdGlvbigpe3RoaXMuX2N1cnZlLmxpbmVTdGFy
dCgpfSxsaW5lRW5kOmZ1bmN0aW9uKCl7dGhpcy5fY3VydmUubGluZUVuZCgpfSxwb2ludDpmdW5j
dGlvbih0LG4pe3RoaXMuX2N1cnZlLnBvaW50KG4qTWF0aC5zaW4odCksbiotTWF0aC5jb3ModCkp
fX07Y2xhc3MgSm17Y29uc3RydWN0b3IodCxuKXt0aGlzLl9jb250ZXh0PXQsdGhpcy5feD1ufWFy
ZWFTdGFydCgpe3RoaXMuX2xpbmU9MH1hcmVhRW5kKCl7dGhpcy5fbGluZT1OYU59bGluZVN0YXJ0
KCl7dGhpcy5fcG9pbnQ9MH1saW5lRW5kKCl7KHRoaXMuX2xpbmV8fDAhPT10aGlzLl9saW5lJiYx
PT09dGhpcy5fcG9pbnQpJiZ0aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgpLHRoaXMuX2xpbmU9MS10
aGlzLl9saW5lfXBvaW50KHQsbil7c3dpdGNoKHQ9K3Qsbj0rbix0aGlzLl9wb2ludCl7Y2FzZSAw
OnRoaXMuX3BvaW50PTEsdGhpcy5fbGluZT90aGlzLl9jb250ZXh0LmxpbmVUbyh0LG4pOnRoaXMu
X2NvbnRleHQubW92ZVRvKHQsbik7YnJlYWs7Y2FzZSAxOnRoaXMuX3BvaW50PTI7ZGVmYXVsdDp0
aGlzLl94P3RoaXMuX2NvbnRleHQuYmV6aWVyQ3VydmVUbyh0aGlzLl94MD0odGhpcy5feDArdCkv
Mix0aGlzLl95MCx0aGlzLl94MCxuLHQsbik6dGhpcy5fY29udGV4dC5iZXppZXJDdXJ2ZVRvKHRo
aXMuX3gwLHRoaXMuX3kwPSh0aGlzLl95MCtuKS8yLHQsdGhpcy5feTAsdCxuKX10aGlzLl94MD10
LHRoaXMuX3kwPW59fWNsYXNzIHR4e2NvbnN0cnVjdG9yKHQpe3RoaXMuX2NvbnRleHQ9dH1saW5l
U3RhcnQoKXt0aGlzLl9wb2ludD0wfWxpbmVFbmQoKXt9cG9pbnQodCxuKXtpZih0PSt0LG49K24s
MD09PXRoaXMuX3BvaW50KXRoaXMuX3BvaW50PTE7ZWxzZXtjb25zdCBlPVFtKHRoaXMuX3gwLHRo
aXMuX3kwKSxyPVFtKHRoaXMuX3gwLHRoaXMuX3kwPSh0aGlzLl95MCtuKS8yKSxpPVFtKHQsdGhp
cy5feTApLG89UW0odCxuKTt0aGlzLl9jb250ZXh0Lm1vdmVUbyguLi5lKSx0aGlzLl9jb250ZXh0
LmJlemllckN1cnZlVG8oLi4uciwuLi5pLC4uLm8pfXRoaXMuX3gwPXQsdGhpcy5feTA9bn19ZnVu
Y3Rpb24gbngodCl7cmV0dXJuIG5ldyBKbSh0LCEwKX1mdW5jdGlvbiBleCh0KXtyZXR1cm4gbmV3
IEptKHQsITEpfWZ1bmN0aW9uIHJ4KHQpe3JldHVybiBuZXcgdHgodCl9ZnVuY3Rpb24gaXgodCl7
cmV0dXJuIHQuc291cmNlfWZ1bmN0aW9uIG94KHQpe3JldHVybiB0LnRhcmdldH1mdW5jdGlvbiBh
eCh0KXtsZXQgbj1peCxlPW94LHI9T20saT1CbSxvPW51bGwsYT1udWxsLHU9a20oYyk7ZnVuY3Rp
b24gYygpe2xldCBjO2NvbnN0IGY9Rm0uY2FsbChhcmd1bWVudHMpLHM9bi5hcHBseSh0aGlzLGYp
LGw9ZS5hcHBseSh0aGlzLGYpO2lmKG51bGw9PW8mJihhPXQoYz11KCkpKSxhLmxpbmVTdGFydCgp
LGZbMF09cyxhLnBvaW50KCtyLmFwcGx5KHRoaXMsZiksK2kuYXBwbHkodGhpcyxmKSksZlswXT1s
LGEucG9pbnQoK3IuYXBwbHkodGhpcyxmKSwraS5hcHBseSh0aGlzLGYpKSxhLmxpbmVFbmQoKSxj
KXJldHVybiBhPW51bGwsYysiInx8bnVsbH1yZXR1cm4gYy5zb3VyY2U9ZnVuY3Rpb24odCl7cmV0
dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49dCxjKTpufSxjLnRhcmdldD1mdW5jdGlvbih0KXtyZXR1
cm4gYXJndW1lbnRzLmxlbmd0aD8oZT10LGMpOmV9LGMueD1mdW5jdGlvbih0KXtyZXR1cm4gYXJn
dW1lbnRzLmxlbmd0aD8ocj0iZnVuY3Rpb24iPT10eXBlb2YgdD90OnltKCt0KSxjKTpyfSxjLnk9
ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGk9ImZ1bmN0aW9uIj09dHlwZW9m
IHQ/dDp5bSgrdCksYyk6aX0sYy5jb250ZXh0PWZ1bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPyhudWxsPT1uP289YT1udWxsOmE9dChvPW4pLGMpOm99LGN9Y29uc3QgdXg9TW0oMyk7
dmFyIGN4PXtkcmF3KHQsbil7Y29uc3QgZT0uNTk0MzYqTW0obit4bShuLzI4LC43NSkpLHI9ZS8y
LGk9cip1eDt0Lm1vdmVUbygwLGUpLHQubGluZVRvKDAsLWUpLHQubW92ZVRvKC1pLC1yKSx0Lmxp
bmVUbyhpLHIpLHQubW92ZVRvKC1pLHIpLHQubGluZVRvKGksLXIpfX0sZng9e2RyYXcodCxuKXtj
b25zdCBlPU1tKG4vQW0pO3QubW92ZVRvKGUsMCksdC5hcmMoMCwwLGUsMCxFbSl9fSxzeD17ZHJh
dyh0LG4pe2NvbnN0IGU9TW0obi81KS8yO3QubW92ZVRvKC0zKmUsLWUpLHQubGluZVRvKC1lLC1l
KSx0LmxpbmVUbygtZSwtMyplKSx0LmxpbmVUbyhlLC0zKmUpLHQubGluZVRvKGUsLWUpLHQubGlu
ZVRvKDMqZSwtZSksdC5saW5lVG8oMyplLGUpLHQubGluZVRvKGUsZSksdC5saW5lVG8oZSwzKmUp
LHQubGluZVRvKC1lLDMqZSksdC5saW5lVG8oLWUsZSksdC5saW5lVG8oLTMqZSxlKSx0LmNsb3Nl
UGF0aCgpfX07Y29uc3QgbHg9TW0oMS8zKSxoeD0yKmx4O3ZhciBkeD17ZHJhdyh0LG4pe2NvbnN0
IGU9TW0obi9oeCkscj1lKmx4O3QubW92ZVRvKDAsLWUpLHQubGluZVRvKHIsMCksdC5saW5lVG8o
MCxlKSx0LmxpbmVUbygtciwwKSx0LmNsb3NlUGF0aCgpfX0scHg9e2RyYXcodCxuKXtjb25zdCBl
PS42MjYyNSpNbShuKTt0Lm1vdmVUbygwLC1lKSx0LmxpbmVUbyhlLDApLHQubGluZVRvKDAsZSks
dC5saW5lVG8oLWUsMCksdC5jbG9zZVBhdGgoKX19LGd4PXtkcmF3KHQsbil7Y29uc3QgZT0uODc1
NTkqTW0obi14bShuLzcsMikpO3QubW92ZVRvKC1lLDApLHQubGluZVRvKGUsMCksdC5tb3ZlVG8o
MCxlKSx0LmxpbmVUbygwLC1lKX19LHl4PXtkcmF3KHQsbil7Y29uc3QgZT1NbShuKSxyPS1lLzI7
dC5yZWN0KHIscixlLGUpfX0sdng9e2RyYXcodCxuKXtjb25zdCBlPS40NDMxKk1tKG4pO3QubW92
ZVRvKGUsZSksdC5saW5lVG8oZSwtZSksdC5saW5lVG8oLWUsLWUpLHQubGluZVRvKC1lLGUpLHQu
Y2xvc2VQYXRoKCl9fTtjb25zdCBfeD13bShBbS8xMCkvd20oNypBbS8xMCksYng9d20oRW0vMTAp
Kl94LG14PS1ibShFbS8xMCkqX3g7dmFyIHh4PXtkcmF3KHQsbil7Y29uc3QgZT1NbSguODkwODEz
MDkxNTI5Mjg1MipuKSxyPWJ4KmUsaT1teCplO3QubW92ZVRvKDAsLWUpLHQubGluZVRvKHIsaSk7
Zm9yKGxldCBuPTE7bjw1Oysrbil7Y29uc3Qgbz1FbSpuLzUsYT1ibShvKSx1PXdtKG8pO3QubGlu
ZVRvKHUqZSwtYSplKSx0LmxpbmVUbyhhKnItdSppLHUqcithKmkpfXQuY2xvc2VQYXRoKCl9fTtj
b25zdCB3eD1NbSgzKTt2YXIgTXg9e2RyYXcodCxuKXtjb25zdCBlPS1NbShuLygzKnd4KSk7dC5t
b3ZlVG8oMCwyKmUpLHQubGluZVRvKC13eCplLC1lKSx0LmxpbmVUbyh3eCplLC1lKSx0LmNsb3Nl
UGF0aCgpfX07Y29uc3QgVHg9TW0oMyk7dmFyIEF4PXtkcmF3KHQsbil7Y29uc3QgZT0uNjgyNCpN
bShuKSxyPWUvMixpPWUqVHgvMjt0Lm1vdmVUbygwLC1lKSx0LmxpbmVUbyhpLHIpLHQubGluZVRv
KC1pLHIpLHQuY2xvc2VQYXRoKCl9fTtjb25zdCBTeD0tLjUsRXg9TW0oMykvMixOeD0xL01tKDEy
KSxreD0zKihOeC8yKzEpO3ZhciBDeD17ZHJhdyh0LG4pe2NvbnN0IGU9TW0obi9reCkscj1lLzIs
aT1lKk54LG89cixhPWUqTngrZSx1PS1vLGM9YTt0Lm1vdmVUbyhyLGkpLHQubGluZVRvKG8sYSks
dC5saW5lVG8odSxjKSx0LmxpbmVUbyhTeCpyLUV4KmksRXgqcitTeCppKSx0LmxpbmVUbyhTeCpv
LUV4KmEsRXgqbytTeCphKSx0LmxpbmVUbyhTeCp1LUV4KmMsRXgqdStTeCpjKSx0LmxpbmVUbyhT
eCpyK0V4KmksU3gqaS1FeCpyKSx0LmxpbmVUbyhTeCpvK0V4KmEsU3gqYS1FeCpvKSx0LmxpbmVU
byhTeCp1K0V4KmMsU3gqYy1FeCp1KSx0LmNsb3NlUGF0aCgpfX0sUHg9e2RyYXcodCxuKXtjb25z
dCBlPS42MTg5Kk1tKG4teG0obi82LDEuNykpO3QubW92ZVRvKC1lLC1lKSx0LmxpbmVUbyhlLGUp
LHQubW92ZVRvKC1lLGUpLHQubGluZVRvKGUsLWUpfX07Y29uc3Qgeng9W2Z4LHN4LGR4LHl4LHh4
LE14LEN4XSwkeD1bZngsZ3gsUHgsQXgsY3gsdngscHhdO2Z1bmN0aW9uIER4KCl7fWZ1bmN0aW9u
IFJ4KHQsbixlKXt0Ll9jb250ZXh0LmJlemllckN1cnZlVG8oKDIqdC5feDArdC5feDEpLzMsKDIq
dC5feTArdC5feTEpLzMsKHQuX3gwKzIqdC5feDEpLzMsKHQuX3kwKzIqdC5feTEpLzMsKHQuX3gw
KzQqdC5feDErbikvNiwodC5feTArNCp0Ll95MStlKS82KX1mdW5jdGlvbiBGeCh0KXt0aGlzLl9j
b250ZXh0PXR9ZnVuY3Rpb24gcXgodCl7dGhpcy5fY29udGV4dD10fWZ1bmN0aW9uIFV4KHQpe3Ro
aXMuX2NvbnRleHQ9dH1mdW5jdGlvbiBJeCh0LG4pe3RoaXMuX2Jhc2lzPW5ldyBGeCh0KSx0aGlz
Ll9iZXRhPW59RngucHJvdG90eXBlPXthcmVhU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLl9saW5lPTB9
LGFyZWFFbmQ6ZnVuY3Rpb24oKXt0aGlzLl9saW5lPU5hTn0sbGluZVN0YXJ0OmZ1bmN0aW9uKCl7
dGhpcy5feDA9dGhpcy5feDE9dGhpcy5feTA9dGhpcy5feTE9TmFOLHRoaXMuX3BvaW50PTB9LGxp
bmVFbmQ6ZnVuY3Rpb24oKXtzd2l0Y2godGhpcy5fcG9pbnQpe2Nhc2UgMzpSeCh0aGlzLHRoaXMu
X3gxLHRoaXMuX3kxKTtjYXNlIDI6dGhpcy5fY29udGV4dC5saW5lVG8odGhpcy5feDEsdGhpcy5f
eTEpfSh0aGlzLl9saW5lfHwwIT09dGhpcy5fbGluZSYmMT09PXRoaXMuX3BvaW50KSYmdGhpcy5f
Y29udGV4dC5jbG9zZVBhdGgoKSx0aGlzLl9saW5lPTEtdGhpcy5fbGluZX0scG9pbnQ6ZnVuY3Rp
b24odCxuKXtzd2l0Y2godD0rdCxuPStuLHRoaXMuX3BvaW50KXtjYXNlIDA6dGhpcy5fcG9pbnQ9
MSx0aGlzLl9saW5lP3RoaXMuX2NvbnRleHQubGluZVRvKHQsbik6dGhpcy5fY29udGV4dC5tb3Zl
VG8odCxuKTticmVhaztjYXNlIDE6dGhpcy5fcG9pbnQ9MjticmVhaztjYXNlIDI6dGhpcy5fcG9p
bnQ9Myx0aGlzLl9jb250ZXh0LmxpbmVUbygoNSp0aGlzLl94MCt0aGlzLl94MSkvNiwoNSp0aGlz
Ll95MCt0aGlzLl95MSkvNik7ZGVmYXVsdDpSeCh0aGlzLHQsbil9dGhpcy5feDA9dGhpcy5feDEs
dGhpcy5feDE9dCx0aGlzLl95MD10aGlzLl95MSx0aGlzLl95MT1ufX0scXgucHJvdG90eXBlPXth
cmVhU3RhcnQ6RHgsYXJlYUVuZDpEeCxsaW5lU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLl94MD10aGlz
Ll94MT10aGlzLl94Mj10aGlzLl94Mz10aGlzLl94ND10aGlzLl95MD10aGlzLl95MT10aGlzLl95
Mj10aGlzLl95Mz10aGlzLl95ND1OYU4sdGhpcy5fcG9pbnQ9MH0sbGluZUVuZDpmdW5jdGlvbigp
e3N3aXRjaCh0aGlzLl9wb2ludCl7Y2FzZSAxOnRoaXMuX2NvbnRleHQubW92ZVRvKHRoaXMuX3gy
LHRoaXMuX3kyKSx0aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgpO2JyZWFrO2Nhc2UgMjp0aGlzLl9j
b250ZXh0Lm1vdmVUbygodGhpcy5feDIrMip0aGlzLl94MykvMywodGhpcy5feTIrMip0aGlzLl95
MykvMyksdGhpcy5fY29udGV4dC5saW5lVG8oKHRoaXMuX3gzKzIqdGhpcy5feDIpLzMsKHRoaXMu
X3kzKzIqdGhpcy5feTIpLzMpLHRoaXMuX2NvbnRleHQuY2xvc2VQYXRoKCk7YnJlYWs7Y2FzZSAz
OnRoaXMucG9pbnQodGhpcy5feDIsdGhpcy5feTIpLHRoaXMucG9pbnQodGhpcy5feDMsdGhpcy5f
eTMpLHRoaXMucG9pbnQodGhpcy5feDQsdGhpcy5feTQpfX0scG9pbnQ6ZnVuY3Rpb24odCxuKXtz
d2l0Y2godD0rdCxuPStuLHRoaXMuX3BvaW50KXtjYXNlIDA6dGhpcy5fcG9pbnQ9MSx0aGlzLl94
Mj10LHRoaXMuX3kyPW47YnJlYWs7Y2FzZSAxOnRoaXMuX3BvaW50PTIsdGhpcy5feDM9dCx0aGlz
Ll95Mz1uO2JyZWFrO2Nhc2UgMjp0aGlzLl9wb2ludD0zLHRoaXMuX3g0PXQsdGhpcy5feTQ9bix0
aGlzLl9jb250ZXh0Lm1vdmVUbygodGhpcy5feDArNCp0aGlzLl94MSt0KS82LCh0aGlzLl95MCs0
KnRoaXMuX3kxK24pLzYpO2JyZWFrO2RlZmF1bHQ6UngodGhpcyx0LG4pfXRoaXMuX3gwPXRoaXMu
X3gxLHRoaXMuX3gxPXQsdGhpcy5feTA9dGhpcy5feTEsdGhpcy5feTE9bn19LFV4LnByb3RvdHlw
ZT17YXJlYVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5fbGluZT0wfSxhcmVhRW5kOmZ1bmN0aW9uKCl7
dGhpcy5fbGluZT1OYU59LGxpbmVTdGFydDpmdW5jdGlvbigpe3RoaXMuX3gwPXRoaXMuX3gxPXRo
aXMuX3kwPXRoaXMuX3kxPU5hTix0aGlzLl9wb2ludD0wfSxsaW5lRW5kOmZ1bmN0aW9uKCl7KHRo
aXMuX2xpbmV8fDAhPT10aGlzLl9saW5lJiYzPT09dGhpcy5fcG9pbnQpJiZ0aGlzLl9jb250ZXh0
LmNsb3NlUGF0aCgpLHRoaXMuX2xpbmU9MS10aGlzLl9saW5lfSxwb2ludDpmdW5jdGlvbih0LG4p
e3N3aXRjaCh0PSt0LG49K24sdGhpcy5fcG9pbnQpe2Nhc2UgMDp0aGlzLl9wb2ludD0xO2JyZWFr
O2Nhc2UgMTp0aGlzLl9wb2ludD0yO2JyZWFrO2Nhc2UgMjp0aGlzLl9wb2ludD0zO3ZhciBlPSh0
aGlzLl94MCs0KnRoaXMuX3gxK3QpLzYscj0odGhpcy5feTArNCp0aGlzLl95MStuKS82O3RoaXMu
X2xpbmU/dGhpcy5fY29udGV4dC5saW5lVG8oZSxyKTp0aGlzLl9jb250ZXh0Lm1vdmVUbyhlLHIp
O2JyZWFrO2Nhc2UgMzp0aGlzLl9wb2ludD00O2RlZmF1bHQ6UngodGhpcyx0LG4pfXRoaXMuX3gw
PXRoaXMuX3gxLHRoaXMuX3gxPXQsdGhpcy5feTA9dGhpcy5feTEsdGhpcy5feTE9bn19LEl4LnBy
b3RvdHlwZT17bGluZVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5feD1bXSx0aGlzLl95PVtdLHRoaXMu
X2Jhc2lzLmxpbmVTdGFydCgpfSxsaW5lRW5kOmZ1bmN0aW9uKCl7dmFyIHQ9dGhpcy5feCxuPXRo
aXMuX3ksZT10Lmxlbmd0aC0xO2lmKGU+MClmb3IodmFyIHIsaT10WzBdLG89blswXSxhPXRbZV0t
aSx1PW5bZV0tbyxjPS0xOysrYzw9ZTspcj1jL2UsdGhpcy5fYmFzaXMucG9pbnQodGhpcy5fYmV0
YSp0W2NdKygxLXRoaXMuX2JldGEpKihpK3IqYSksdGhpcy5fYmV0YSpuW2NdKygxLXRoaXMuX2Jl
dGEpKihvK3IqdSkpO3RoaXMuX3g9dGhpcy5feT1udWxsLHRoaXMuX2Jhc2lzLmxpbmVFbmQoKX0s
cG9pbnQ6ZnVuY3Rpb24odCxuKXt0aGlzLl94LnB1c2goK3QpLHRoaXMuX3kucHVzaCgrbil9fTt2
YXIgT3g9ZnVuY3Rpb24gdChuKXtmdW5jdGlvbiBlKHQpe3JldHVybiAxPT09bj9uZXcgRngodCk6
bmV3IEl4KHQsbil9cmV0dXJuIGUuYmV0YT1mdW5jdGlvbihuKXtyZXR1cm4gdCgrbil9LGV9KC44
NSk7ZnVuY3Rpb24gQngodCxuLGUpe3QuX2NvbnRleHQuYmV6aWVyQ3VydmVUbyh0Ll94MSt0Ll9r
Kih0Ll94Mi10Ll94MCksdC5feTErdC5fayoodC5feTItdC5feTApLHQuX3gyK3QuX2sqKHQuX3gx
LW4pLHQuX3kyK3QuX2sqKHQuX3kxLWUpLHQuX3gyLHQuX3kyKX1mdW5jdGlvbiBZeCh0LG4pe3Ro
aXMuX2NvbnRleHQ9dCx0aGlzLl9rPSgxLW4pLzZ9WXgucHJvdG90eXBlPXthcmVhU3RhcnQ6ZnVu
Y3Rpb24oKXt0aGlzLl9saW5lPTB9LGFyZWFFbmQ6ZnVuY3Rpb24oKXt0aGlzLl9saW5lPU5hTn0s
bGluZVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5feDA9dGhpcy5feDE9dGhpcy5feDI9dGhpcy5feTA9
dGhpcy5feTE9dGhpcy5feTI9TmFOLHRoaXMuX3BvaW50PTB9LGxpbmVFbmQ6ZnVuY3Rpb24oKXtz
d2l0Y2godGhpcy5fcG9pbnQpe2Nhc2UgMjp0aGlzLl9jb250ZXh0LmxpbmVUbyh0aGlzLl94Mix0
aGlzLl95Mik7YnJlYWs7Y2FzZSAzOkJ4KHRoaXMsdGhpcy5feDEsdGhpcy5feTEpfSh0aGlzLl9s
aW5lfHwwIT09dGhpcy5fbGluZSYmMT09PXRoaXMuX3BvaW50KSYmdGhpcy5fY29udGV4dC5jbG9z
ZVBhdGgoKSx0aGlzLl9saW5lPTEtdGhpcy5fbGluZX0scG9pbnQ6ZnVuY3Rpb24odCxuKXtzd2l0
Y2godD0rdCxuPStuLHRoaXMuX3BvaW50KXtjYXNlIDA6dGhpcy5fcG9pbnQ9MSx0aGlzLl9saW5l
P3RoaXMuX2NvbnRleHQubGluZVRvKHQsbik6dGhpcy5fY29udGV4dC5tb3ZlVG8odCxuKTticmVh
aztjYXNlIDE6dGhpcy5fcG9pbnQ9Mix0aGlzLl94MT10LHRoaXMuX3kxPW47YnJlYWs7Y2FzZSAy
OnRoaXMuX3BvaW50PTM7ZGVmYXVsdDpCeCh0aGlzLHQsbil9dGhpcy5feDA9dGhpcy5feDEsdGhp
cy5feDE9dGhpcy5feDIsdGhpcy5feDI9dCx0aGlzLl95MD10aGlzLl95MSx0aGlzLl95MT10aGlz
Ll95Mix0aGlzLl95Mj1ufX07dmFyIEx4PWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0KXtyZXR1
cm4gbmV3IFl4KHQsbil9cmV0dXJuIGUudGVuc2lvbj1mdW5jdGlvbihuKXtyZXR1cm4gdCgrbil9
LGV9KDApO2Z1bmN0aW9uIGp4KHQsbil7dGhpcy5fY29udGV4dD10LHRoaXMuX2s9KDEtbikvNn1q
eC5wcm90b3R5cGU9e2FyZWFTdGFydDpEeCxhcmVhRW5kOkR4LGxpbmVTdGFydDpmdW5jdGlvbigp
e3RoaXMuX3gwPXRoaXMuX3gxPXRoaXMuX3gyPXRoaXMuX3gzPXRoaXMuX3g0PXRoaXMuX3g1PXRo
aXMuX3kwPXRoaXMuX3kxPXRoaXMuX3kyPXRoaXMuX3kzPXRoaXMuX3k0PXRoaXMuX3k1PU5hTix0
aGlzLl9wb2ludD0wfSxsaW5lRW5kOmZ1bmN0aW9uKCl7c3dpdGNoKHRoaXMuX3BvaW50KXtjYXNl
IDE6dGhpcy5fY29udGV4dC5tb3ZlVG8odGhpcy5feDMsdGhpcy5feTMpLHRoaXMuX2NvbnRleHQu
Y2xvc2VQYXRoKCk7YnJlYWs7Y2FzZSAyOnRoaXMuX2NvbnRleHQubGluZVRvKHRoaXMuX3gzLHRo
aXMuX3kzKSx0aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgpO2JyZWFrO2Nhc2UgMzp0aGlzLnBvaW50
KHRoaXMuX3gzLHRoaXMuX3kzKSx0aGlzLnBvaW50KHRoaXMuX3g0LHRoaXMuX3k0KSx0aGlzLnBv
aW50KHRoaXMuX3g1LHRoaXMuX3k1KX19LHBvaW50OmZ1bmN0aW9uKHQsbil7c3dpdGNoKHQ9K3Qs
bj0rbix0aGlzLl9wb2ludCl7Y2FzZSAwOnRoaXMuX3BvaW50PTEsdGhpcy5feDM9dCx0aGlzLl95
Mz1uO2JyZWFrO2Nhc2UgMTp0aGlzLl9wb2ludD0yLHRoaXMuX2NvbnRleHQubW92ZVRvKHRoaXMu
X3g0PXQsdGhpcy5feTQ9bik7YnJlYWs7Y2FzZSAyOnRoaXMuX3BvaW50PTMsdGhpcy5feDU9dCx0
aGlzLl95NT1uO2JyZWFrO2RlZmF1bHQ6QngodGhpcyx0LG4pfXRoaXMuX3gwPXRoaXMuX3gxLHRo
aXMuX3gxPXRoaXMuX3gyLHRoaXMuX3gyPXQsdGhpcy5feTA9dGhpcy5feTEsdGhpcy5feTE9dGhp
cy5feTIsdGhpcy5feTI9bn19O3ZhciBIeD1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCl7cmV0
dXJuIG5ldyBqeCh0LG4pfXJldHVybiBlLnRlbnNpb249ZnVuY3Rpb24obil7cmV0dXJuIHQoK24p
fSxlfSgwKTtmdW5jdGlvbiBYeCh0LG4pe3RoaXMuX2NvbnRleHQ9dCx0aGlzLl9rPSgxLW4pLzZ9
WHgucHJvdG90eXBlPXthcmVhU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLl9saW5lPTB9LGFyZWFFbmQ6
ZnVuY3Rpb24oKXt0aGlzLl9saW5lPU5hTn0sbGluZVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5feDA9
dGhpcy5feDE9dGhpcy5feDI9dGhpcy5feTA9dGhpcy5feTE9dGhpcy5feTI9TmFOLHRoaXMuX3Bv
aW50PTB9LGxpbmVFbmQ6ZnVuY3Rpb24oKXsodGhpcy5fbGluZXx8MCE9PXRoaXMuX2xpbmUmJjM9
PT10aGlzLl9wb2ludCkmJnRoaXMuX2NvbnRleHQuY2xvc2VQYXRoKCksdGhpcy5fbGluZT0xLXRo
aXMuX2xpbmV9LHBvaW50OmZ1bmN0aW9uKHQsbil7c3dpdGNoKHQ9K3Qsbj0rbix0aGlzLl9wb2lu
dCl7Y2FzZSAwOnRoaXMuX3BvaW50PTE7YnJlYWs7Y2FzZSAxOnRoaXMuX3BvaW50PTI7YnJlYWs7
Y2FzZSAyOnRoaXMuX3BvaW50PTMsdGhpcy5fbGluZT90aGlzLl9jb250ZXh0LmxpbmVUbyh0aGlz
Ll94Mix0aGlzLl95Mik6dGhpcy5fY29udGV4dC5tb3ZlVG8odGhpcy5feDIsdGhpcy5feTIpO2Jy
ZWFrO2Nhc2UgMzp0aGlzLl9wb2ludD00O2RlZmF1bHQ6QngodGhpcyx0LG4pfXRoaXMuX3gwPXRo
aXMuX3gxLHRoaXMuX3gxPXRoaXMuX3gyLHRoaXMuX3gyPXQsdGhpcy5feTA9dGhpcy5feTEsdGhp
cy5feTE9dGhpcy5feTIsdGhpcy5feTI9bn19O3ZhciBHeD1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9u
IGUodCl7cmV0dXJuIG5ldyBYeCh0LG4pfXJldHVybiBlLnRlbnNpb249ZnVuY3Rpb24obil7cmV0
dXJuIHQoK24pfSxlfSgwKTtmdW5jdGlvbiBWeCh0LG4sZSl7dmFyIHI9dC5feDEsaT10Ll95MSxv
PXQuX3gyLGE9dC5feTI7aWYodC5fbDAxX2E+VG0pe3ZhciB1PTIqdC5fbDAxXzJhKzMqdC5fbDAx
X2EqdC5fbDEyX2ErdC5fbDEyXzJhLGM9Myp0Ll9sMDFfYSoodC5fbDAxX2ErdC5fbDEyX2EpO3I9
KHIqdS10Ll94MCp0Ll9sMTJfMmErdC5feDIqdC5fbDAxXzJhKS9jLGk9KGkqdS10Ll95MCp0Ll9s
MTJfMmErdC5feTIqdC5fbDAxXzJhKS9jfWlmKHQuX2wyM19hPlRtKXt2YXIgZj0yKnQuX2wyM18y
YSszKnQuX2wyM19hKnQuX2wxMl9hK3QuX2wxMl8yYSxzPTMqdC5fbDIzX2EqKHQuX2wyM19hK3Qu
X2wxMl9hKTtvPShvKmYrdC5feDEqdC5fbDIzXzJhLW4qdC5fbDEyXzJhKS9zLGE9KGEqZit0Ll95
MSp0Ll9sMjNfMmEtZSp0Ll9sMTJfMmEpL3N9dC5fY29udGV4dC5iZXppZXJDdXJ2ZVRvKHIsaSxv
LGEsdC5feDIsdC5feTIpfWZ1bmN0aW9uIFd4KHQsbil7dGhpcy5fY29udGV4dD10LHRoaXMuX2Fs
cGhhPW59V3gucHJvdG90eXBlPXthcmVhU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLl9saW5lPTB9LGFy
ZWFFbmQ6ZnVuY3Rpb24oKXt0aGlzLl9saW5lPU5hTn0sbGluZVN0YXJ0OmZ1bmN0aW9uKCl7dGhp
cy5feDA9dGhpcy5feDE9dGhpcy5feDI9dGhpcy5feTA9dGhpcy5feTE9dGhpcy5feTI9TmFOLHRo
aXMuX2wwMV9hPXRoaXMuX2wxMl9hPXRoaXMuX2wyM19hPXRoaXMuX2wwMV8yYT10aGlzLl9sMTJf
MmE9dGhpcy5fbDIzXzJhPXRoaXMuX3BvaW50PTB9LGxpbmVFbmQ6ZnVuY3Rpb24oKXtzd2l0Y2go
dGhpcy5fcG9pbnQpe2Nhc2UgMjp0aGlzLl9jb250ZXh0LmxpbmVUbyh0aGlzLl94Mix0aGlzLl95
Mik7YnJlYWs7Y2FzZSAzOnRoaXMucG9pbnQodGhpcy5feDIsdGhpcy5feTIpfSh0aGlzLl9saW5l
fHwwIT09dGhpcy5fbGluZSYmMT09PXRoaXMuX3BvaW50KSYmdGhpcy5fY29udGV4dC5jbG9zZVBh
dGgoKSx0aGlzLl9saW5lPTEtdGhpcy5fbGluZX0scG9pbnQ6ZnVuY3Rpb24odCxuKXtpZih0PSt0
LG49K24sdGhpcy5fcG9pbnQpe3ZhciBlPXRoaXMuX3gyLXQscj10aGlzLl95Mi1uO3RoaXMuX2wy
M19hPU1hdGguc3FydCh0aGlzLl9sMjNfMmE9TWF0aC5wb3coZSplK3Iqcix0aGlzLl9hbHBoYSkp
fXN3aXRjaCh0aGlzLl9wb2ludCl7Y2FzZSAwOnRoaXMuX3BvaW50PTEsdGhpcy5fbGluZT90aGlz
Ll9jb250ZXh0LmxpbmVUbyh0LG4pOnRoaXMuX2NvbnRleHQubW92ZVRvKHQsbik7YnJlYWs7Y2Fz
ZSAxOnRoaXMuX3BvaW50PTI7YnJlYWs7Y2FzZSAyOnRoaXMuX3BvaW50PTM7ZGVmYXVsdDpWeCh0
aGlzLHQsbil9dGhpcy5fbDAxX2E9dGhpcy5fbDEyX2EsdGhpcy5fbDEyX2E9dGhpcy5fbDIzX2Es
dGhpcy5fbDAxXzJhPXRoaXMuX2wxMl8yYSx0aGlzLl9sMTJfMmE9dGhpcy5fbDIzXzJhLHRoaXMu
X3gwPXRoaXMuX3gxLHRoaXMuX3gxPXRoaXMuX3gyLHRoaXMuX3gyPXQsdGhpcy5feTA9dGhpcy5f
eTEsdGhpcy5feTE9dGhpcy5feTIsdGhpcy5feTI9bn19O3ZhciBaeD1mdW5jdGlvbiB0KG4pe2Z1
bmN0aW9uIGUodCl7cmV0dXJuIG4/bmV3IFd4KHQsbik6bmV3IFl4KHQsMCl9cmV0dXJuIGUuYWxw
aGE9ZnVuY3Rpb24obil7cmV0dXJuIHQoK24pfSxlfSguNSk7ZnVuY3Rpb24gS3godCxuKXt0aGlz
Ll9jb250ZXh0PXQsdGhpcy5fYWxwaGE9bn1LeC5wcm90b3R5cGU9e2FyZWFTdGFydDpEeCxhcmVh
RW5kOkR4LGxpbmVTdGFydDpmdW5jdGlvbigpe3RoaXMuX3gwPXRoaXMuX3gxPXRoaXMuX3gyPXRo
aXMuX3gzPXRoaXMuX3g0PXRoaXMuX3g1PXRoaXMuX3kwPXRoaXMuX3kxPXRoaXMuX3kyPXRoaXMu
X3kzPXRoaXMuX3k0PXRoaXMuX3k1PU5hTix0aGlzLl9sMDFfYT10aGlzLl9sMTJfYT10aGlzLl9s
MjNfYT10aGlzLl9sMDFfMmE9dGhpcy5fbDEyXzJhPXRoaXMuX2wyM18yYT10aGlzLl9wb2ludD0w
fSxsaW5lRW5kOmZ1bmN0aW9uKCl7c3dpdGNoKHRoaXMuX3BvaW50KXtjYXNlIDE6dGhpcy5fY29u
dGV4dC5tb3ZlVG8odGhpcy5feDMsdGhpcy5feTMpLHRoaXMuX2NvbnRleHQuY2xvc2VQYXRoKCk7
YnJlYWs7Y2FzZSAyOnRoaXMuX2NvbnRleHQubGluZVRvKHRoaXMuX3gzLHRoaXMuX3kzKSx0aGlz
Ll9jb250ZXh0LmNsb3NlUGF0aCgpO2JyZWFrO2Nhc2UgMzp0aGlzLnBvaW50KHRoaXMuX3gzLHRo
aXMuX3kzKSx0aGlzLnBvaW50KHRoaXMuX3g0LHRoaXMuX3k0KSx0aGlzLnBvaW50KHRoaXMuX3g1
LHRoaXMuX3k1KX19LHBvaW50OmZ1bmN0aW9uKHQsbil7aWYodD0rdCxuPStuLHRoaXMuX3BvaW50
KXt2YXIgZT10aGlzLl94Mi10LHI9dGhpcy5feTItbjt0aGlzLl9sMjNfYT1NYXRoLnNxcnQodGhp
cy5fbDIzXzJhPU1hdGgucG93KGUqZStyKnIsdGhpcy5fYWxwaGEpKX1zd2l0Y2godGhpcy5fcG9p
bnQpe2Nhc2UgMDp0aGlzLl9wb2ludD0xLHRoaXMuX3gzPXQsdGhpcy5feTM9bjticmVhaztjYXNl
IDE6dGhpcy5fcG9pbnQ9Mix0aGlzLl9jb250ZXh0Lm1vdmVUbyh0aGlzLl94ND10LHRoaXMuX3k0
PW4pO2JyZWFrO2Nhc2UgMjp0aGlzLl9wb2ludD0zLHRoaXMuX3g1PXQsdGhpcy5feTU9bjticmVh
aztkZWZhdWx0OlZ4KHRoaXMsdCxuKX10aGlzLl9sMDFfYT10aGlzLl9sMTJfYSx0aGlzLl9sMTJf
YT10aGlzLl9sMjNfYSx0aGlzLl9sMDFfMmE9dGhpcy5fbDEyXzJhLHRoaXMuX2wxMl8yYT10aGlz
Ll9sMjNfMmEsdGhpcy5feDA9dGhpcy5feDEsdGhpcy5feDE9dGhpcy5feDIsdGhpcy5feDI9dCx0
aGlzLl95MD10aGlzLl95MSx0aGlzLl95MT10aGlzLl95Mix0aGlzLl95Mj1ufX07dmFyIFF4PWZ1
bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0KXtyZXR1cm4gbj9uZXcgS3godCxuKTpuZXcgangodCww
KX1yZXR1cm4gZS5hbHBoYT1mdW5jdGlvbihuKXtyZXR1cm4gdCgrbil9LGV9KC41KTtmdW5jdGlv
biBKeCh0LG4pe3RoaXMuX2NvbnRleHQ9dCx0aGlzLl9hbHBoYT1ufUp4LnByb3RvdHlwZT17YXJl
YVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5fbGluZT0wfSxhcmVhRW5kOmZ1bmN0aW9uKCl7dGhpcy5f
bGluZT1OYU59LGxpbmVTdGFydDpmdW5jdGlvbigpe3RoaXMuX3gwPXRoaXMuX3gxPXRoaXMuX3gy
PXRoaXMuX3kwPXRoaXMuX3kxPXRoaXMuX3kyPU5hTix0aGlzLl9sMDFfYT10aGlzLl9sMTJfYT10
aGlzLl9sMjNfYT10aGlzLl9sMDFfMmE9dGhpcy5fbDEyXzJhPXRoaXMuX2wyM18yYT10aGlzLl9w
b2ludD0wfSxsaW5lRW5kOmZ1bmN0aW9uKCl7KHRoaXMuX2xpbmV8fDAhPT10aGlzLl9saW5lJiYz
PT09dGhpcy5fcG9pbnQpJiZ0aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgpLHRoaXMuX2xpbmU9MS10
aGlzLl9saW5lfSxwb2ludDpmdW5jdGlvbih0LG4pe2lmKHQ9K3Qsbj0rbix0aGlzLl9wb2ludCl7
dmFyIGU9dGhpcy5feDItdCxyPXRoaXMuX3kyLW47dGhpcy5fbDIzX2E9TWF0aC5zcXJ0KHRoaXMu
X2wyM18yYT1NYXRoLnBvdyhlKmUrcipyLHRoaXMuX2FscGhhKSl9c3dpdGNoKHRoaXMuX3BvaW50
KXtjYXNlIDA6dGhpcy5fcG9pbnQ9MTticmVhaztjYXNlIDE6dGhpcy5fcG9pbnQ9MjticmVhaztj
YXNlIDI6dGhpcy5fcG9pbnQ9Myx0aGlzLl9saW5lP3RoaXMuX2NvbnRleHQubGluZVRvKHRoaXMu
X3gyLHRoaXMuX3kyKTp0aGlzLl9jb250ZXh0Lm1vdmVUbyh0aGlzLl94Mix0aGlzLl95Mik7YnJl
YWs7Y2FzZSAzOnRoaXMuX3BvaW50PTQ7ZGVmYXVsdDpWeCh0aGlzLHQsbil9dGhpcy5fbDAxX2E9
dGhpcy5fbDEyX2EsdGhpcy5fbDEyX2E9dGhpcy5fbDIzX2EsdGhpcy5fbDAxXzJhPXRoaXMuX2wx
Ml8yYSx0aGlzLl9sMTJfMmE9dGhpcy5fbDIzXzJhLHRoaXMuX3gwPXRoaXMuX3gxLHRoaXMuX3gx
PXRoaXMuX3gyLHRoaXMuX3gyPXQsdGhpcy5feTA9dGhpcy5feTEsdGhpcy5feTE9dGhpcy5feTIs
dGhpcy5feTI9bn19O3ZhciB0dz1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCl7cmV0dXJuIG4/
bmV3IEp4KHQsbik6bmV3IFh4KHQsMCl9cmV0dXJuIGUuYWxwaGE9ZnVuY3Rpb24obil7cmV0dXJu
IHQoK24pfSxlfSguNSk7ZnVuY3Rpb24gbncodCl7dGhpcy5fY29udGV4dD10fWZ1bmN0aW9uIGV3
KHQpe3JldHVybiB0PDA/LTE6MX1mdW5jdGlvbiBydyh0LG4sZSl7dmFyIHI9dC5feDEtdC5feDAs
aT1uLXQuX3gxLG89KHQuX3kxLXQuX3kwKS8ocnx8aTwwJiYtMCksYT0oZS10Ll95MSkvKGl8fHI8
MCYmLTApLHU9KG8qaSthKnIpLyhyK2kpO3JldHVybihldyhvKStldyhhKSkqTWF0aC5taW4oTWF0
aC5hYnMobyksTWF0aC5hYnMoYSksLjUqTWF0aC5hYnModSkpfHwwfWZ1bmN0aW9uIGl3KHQsbil7
dmFyIGU9dC5feDEtdC5feDA7cmV0dXJuIGU/KDMqKHQuX3kxLXQuX3kwKS9lLW4pLzI6bn1mdW5j
dGlvbiBvdyh0LG4sZSl7dmFyIHI9dC5feDAsaT10Ll95MCxvPXQuX3gxLGE9dC5feTEsdT0oby1y
KS8zO3QuX2NvbnRleHQuYmV6aWVyQ3VydmVUbyhyK3UsaSt1Km4sby11LGEtdSplLG8sYSl9ZnVu
Y3Rpb24gYXcodCl7dGhpcy5fY29udGV4dD10fWZ1bmN0aW9uIHV3KHQpe3RoaXMuX2NvbnRleHQ9
bmV3IGN3KHQpfWZ1bmN0aW9uIGN3KHQpe3RoaXMuX2NvbnRleHQ9dH1mdW5jdGlvbiBmdyh0KXt0
aGlzLl9jb250ZXh0PXR9ZnVuY3Rpb24gc3codCl7dmFyIG4sZSxyPXQubGVuZ3RoLTEsaT1uZXcg
QXJyYXkociksbz1uZXcgQXJyYXkociksYT1uZXcgQXJyYXkocik7Zm9yKGlbMF09MCxvWzBdPTIs
YVswXT10WzBdKzIqdFsxXSxuPTE7bjxyLTE7KytuKWlbbl09MSxvW25dPTQsYVtuXT00KnRbbl0r
Mip0W24rMV07Zm9yKGlbci0xXT0yLG9bci0xXT03LGFbci0xXT04KnRbci0xXSt0W3JdLG49MTtu
PHI7KytuKWU9aVtuXS9vW24tMV0sb1tuXS09ZSxhW25dLT1lKmFbbi0xXTtmb3IoaVtyLTFdPWFb
ci0xXS9vW3ItMV0sbj1yLTI7bj49MDstLW4paVtuXT0oYVtuXS1pW24rMV0pL29bbl07Zm9yKG9b
ci0xXT0odFtyXStpW3ItMV0pLzIsbj0wO248ci0xOysrbilvW25dPTIqdFtuKzFdLWlbbisxXTty
ZXR1cm5baSxvXX1mdW5jdGlvbiBsdyh0LG4pe3RoaXMuX2NvbnRleHQ9dCx0aGlzLl90PW59ZnVu
Y3Rpb24gaHcodCxuKXtpZigoaT10Lmxlbmd0aCk+MSlmb3IodmFyIGUscixpLG89MSxhPXRbblsw
XV0sdT1hLmxlbmd0aDtvPGk7KytvKWZvcihyPWEsYT10W25bb11dLGU9MDtlPHU7KytlKWFbZV1b
MV0rPWFbZV1bMF09aXNOYU4ocltlXVsxXSk/cltlXVswXTpyW2VdWzFdfWZ1bmN0aW9uIGR3KHQp
e2Zvcih2YXIgbj10Lmxlbmd0aCxlPW5ldyBBcnJheShuKTstLW4+PTA7KWVbbl09bjtyZXR1cm4g
ZX1mdW5jdGlvbiBwdyh0LG4pe3JldHVybiB0W25dfWZ1bmN0aW9uIGd3KHQpe2NvbnN0IG49W107
cmV0dXJuIG4ua2V5PXQsbn1mdW5jdGlvbiB5dyh0KXt2YXIgbj10Lm1hcCh2dyk7cmV0dXJuIGR3
KHQpLnNvcnQoKGZ1bmN0aW9uKHQsZSl7cmV0dXJuIG5bdF0tbltlXX0pKX1mdW5jdGlvbiB2dyh0
KXtmb3IodmFyIG4sZT0tMSxyPTAsaT10Lmxlbmd0aCxvPS0xLzA7KytlPGk7KShuPSt0W2VdWzFd
KT5vJiYobz1uLHI9ZSk7cmV0dXJuIHJ9ZnVuY3Rpb24gX3codCl7dmFyIG49dC5tYXAoYncpO3Jl
dHVybiBkdyh0KS5zb3J0KChmdW5jdGlvbih0LGUpe3JldHVybiBuW3RdLW5bZV19KSl9ZnVuY3Rp
b24gYncodCl7Zm9yKHZhciBuLGU9MCxyPS0xLGk9dC5sZW5ndGg7KytyPGk7KShuPSt0W3JdWzFd
KSYmKGUrPW4pO3JldHVybiBlfW53LnByb3RvdHlwZT17YXJlYVN0YXJ0OkR4LGFyZWFFbmQ6RHgs
bGluZVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5fcG9pbnQ9MH0sbGluZUVuZDpmdW5jdGlvbigpe3Ro
aXMuX3BvaW50JiZ0aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgpfSxwb2ludDpmdW5jdGlvbih0LG4p
e3Q9K3Qsbj0rbix0aGlzLl9wb2ludD90aGlzLl9jb250ZXh0LmxpbmVUbyh0LG4pOih0aGlzLl9w
b2ludD0xLHRoaXMuX2NvbnRleHQubW92ZVRvKHQsbikpfX0sYXcucHJvdG90eXBlPXthcmVhU3Rh
cnQ6ZnVuY3Rpb24oKXt0aGlzLl9saW5lPTB9LGFyZWFFbmQ6ZnVuY3Rpb24oKXt0aGlzLl9saW5l
PU5hTn0sbGluZVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5feDA9dGhpcy5feDE9dGhpcy5feTA9dGhp
cy5feTE9dGhpcy5fdDA9TmFOLHRoaXMuX3BvaW50PTB9LGxpbmVFbmQ6ZnVuY3Rpb24oKXtzd2l0
Y2godGhpcy5fcG9pbnQpe2Nhc2UgMjp0aGlzLl9jb250ZXh0LmxpbmVUbyh0aGlzLl94MSx0aGlz
Ll95MSk7YnJlYWs7Y2FzZSAzOm93KHRoaXMsdGhpcy5fdDAsaXcodGhpcyx0aGlzLl90MCkpfSh0
aGlzLl9saW5lfHwwIT09dGhpcy5fbGluZSYmMT09PXRoaXMuX3BvaW50KSYmdGhpcy5fY29udGV4
dC5jbG9zZVBhdGgoKSx0aGlzLl9saW5lPTEtdGhpcy5fbGluZX0scG9pbnQ6ZnVuY3Rpb24odCxu
KXt2YXIgZT1OYU47aWYobj0rbiwodD0rdCkhPT10aGlzLl94MXx8biE9PXRoaXMuX3kxKXtzd2l0
Y2godGhpcy5fcG9pbnQpe2Nhc2UgMDp0aGlzLl9wb2ludD0xLHRoaXMuX2xpbmU/dGhpcy5fY29u
dGV4dC5saW5lVG8odCxuKTp0aGlzLl9jb250ZXh0Lm1vdmVUbyh0LG4pO2JyZWFrO2Nhc2UgMTp0
aGlzLl9wb2ludD0yO2JyZWFrO2Nhc2UgMjp0aGlzLl9wb2ludD0zLG93KHRoaXMsaXcodGhpcyxl
PXJ3KHRoaXMsdCxuKSksZSk7YnJlYWs7ZGVmYXVsdDpvdyh0aGlzLHRoaXMuX3QwLGU9cncodGhp
cyx0LG4pKX10aGlzLl94MD10aGlzLl94MSx0aGlzLl94MT10LHRoaXMuX3kwPXRoaXMuX3kxLHRo
aXMuX3kxPW4sdGhpcy5fdDA9ZX19fSwodXcucHJvdG90eXBlPU9iamVjdC5jcmVhdGUoYXcucHJv
dG90eXBlKSkucG9pbnQ9ZnVuY3Rpb24odCxuKXthdy5wcm90b3R5cGUucG9pbnQuY2FsbCh0aGlz
LG4sdCl9LGN3LnByb3RvdHlwZT17bW92ZVRvOmZ1bmN0aW9uKHQsbil7dGhpcy5fY29udGV4dC5t
b3ZlVG8obix0KX0sY2xvc2VQYXRoOmZ1bmN0aW9uKCl7dGhpcy5fY29udGV4dC5jbG9zZVBhdGgo
KX0sbGluZVRvOmZ1bmN0aW9uKHQsbil7dGhpcy5fY29udGV4dC5saW5lVG8obix0KX0sYmV6aWVy
Q3VydmVUbzpmdW5jdGlvbih0LG4sZSxyLGksbyl7dGhpcy5fY29udGV4dC5iZXppZXJDdXJ2ZVRv
KG4sdCxyLGUsbyxpKX19LGZ3LnByb3RvdHlwZT17YXJlYVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5f
bGluZT0wfSxhcmVhRW5kOmZ1bmN0aW9uKCl7dGhpcy5fbGluZT1OYU59LGxpbmVTdGFydDpmdW5j
dGlvbigpe3RoaXMuX3g9W10sdGhpcy5feT1bXX0sbGluZUVuZDpmdW5jdGlvbigpe3ZhciB0PXRo
aXMuX3gsbj10aGlzLl95LGU9dC5sZW5ndGg7aWYoZSlpZih0aGlzLl9saW5lP3RoaXMuX2NvbnRl
eHQubGluZVRvKHRbMF0sblswXSk6dGhpcy5fY29udGV4dC5tb3ZlVG8odFswXSxuWzBdKSwyPT09
ZSl0aGlzLl9jb250ZXh0LmxpbmVUbyh0WzFdLG5bMV0pO2Vsc2UgZm9yKHZhciByPXN3KHQpLGk9
c3cobiksbz0wLGE9MTthPGU7KytvLCsrYSl0aGlzLl9jb250ZXh0LmJlemllckN1cnZlVG8oclsw
XVtvXSxpWzBdW29dLHJbMV1bb10saVsxXVtvXSx0W2FdLG5bYV0pOyh0aGlzLl9saW5lfHwwIT09
dGhpcy5fbGluZSYmMT09PWUpJiZ0aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgpLHRoaXMuX2xpbmU9
MS10aGlzLl9saW5lLHRoaXMuX3g9dGhpcy5feT1udWxsfSxwb2ludDpmdW5jdGlvbih0LG4pe3Ro
aXMuX3gucHVzaCgrdCksdGhpcy5feS5wdXNoKCtuKX19LGx3LnByb3RvdHlwZT17YXJlYVN0YXJ0
OmZ1bmN0aW9uKCl7dGhpcy5fbGluZT0wfSxhcmVhRW5kOmZ1bmN0aW9uKCl7dGhpcy5fbGluZT1O
YU59LGxpbmVTdGFydDpmdW5jdGlvbigpe3RoaXMuX3g9dGhpcy5feT1OYU4sdGhpcy5fcG9pbnQ9
MH0sbGluZUVuZDpmdW5jdGlvbigpezA8dGhpcy5fdCYmdGhpcy5fdDwxJiYyPT09dGhpcy5fcG9p
bnQmJnRoaXMuX2NvbnRleHQubGluZVRvKHRoaXMuX3gsdGhpcy5feSksKHRoaXMuX2xpbmV8fDAh
PT10aGlzLl9saW5lJiYxPT09dGhpcy5fcG9pbnQpJiZ0aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgp
LHRoaXMuX2xpbmU+PTAmJih0aGlzLl90PTEtdGhpcy5fdCx0aGlzLl9saW5lPTEtdGhpcy5fbGlu
ZSl9LHBvaW50OmZ1bmN0aW9uKHQsbil7c3dpdGNoKHQ9K3Qsbj0rbix0aGlzLl9wb2ludCl7Y2Fz
ZSAwOnRoaXMuX3BvaW50PTEsdGhpcy5fbGluZT90aGlzLl9jb250ZXh0LmxpbmVUbyh0LG4pOnRo
aXMuX2NvbnRleHQubW92ZVRvKHQsbik7YnJlYWs7Y2FzZSAxOnRoaXMuX3BvaW50PTI7ZGVmYXVs
dDppZih0aGlzLl90PD0wKXRoaXMuX2NvbnRleHQubGluZVRvKHRoaXMuX3gsbiksdGhpcy5fY29u
dGV4dC5saW5lVG8odCxuKTtlbHNle3ZhciBlPXRoaXMuX3gqKDEtdGhpcy5fdCkrdCp0aGlzLl90
O3RoaXMuX2NvbnRleHQubGluZVRvKGUsdGhpcy5feSksdGhpcy5fY29udGV4dC5saW5lVG8oZSxu
KX19dGhpcy5feD10LHRoaXMuX3k9bn19O3ZhciBtdz10PT4oKT0+dDtmdW5jdGlvbiB4dyh0LHtz
b3VyY2VFdmVudDpuLHRhcmdldDplLHRyYW5zZm9ybTpyLGRpc3BhdGNoOml9KXtPYmplY3QuZGVm
aW5lUHJvcGVydGllcyh0aGlzLHt0eXBlOnt2YWx1ZTp0LGVudW1lcmFibGU6ITAsY29uZmlndXJh
YmxlOiEwfSxzb3VyY2VFdmVudDp7dmFsdWU6bixlbnVtZXJhYmxlOiEwLGNvbmZpZ3VyYWJsZToh
MH0sdGFyZ2V0Ont2YWx1ZTplLGVudW1lcmFibGU6ITAsY29uZmlndXJhYmxlOiEwfSx0cmFuc2Zv
cm06e3ZhbHVlOnIsZW51bWVyYWJsZTohMCxjb25maWd1cmFibGU6ITB9LF86e3ZhbHVlOml9fSl9
ZnVuY3Rpb24gd3codCxuLGUpe3RoaXMuaz10LHRoaXMueD1uLHRoaXMueT1lfXd3LnByb3RvdHlw
ZT17Y29uc3RydWN0b3I6d3csc2NhbGU6ZnVuY3Rpb24odCl7cmV0dXJuIDE9PT10P3RoaXM6bmV3
IHd3KHRoaXMuayp0LHRoaXMueCx0aGlzLnkpfSx0cmFuc2xhdGU6ZnVuY3Rpb24odCxuKXtyZXR1
cm4gMD09PXQmMD09PW4/dGhpczpuZXcgd3codGhpcy5rLHRoaXMueCt0aGlzLmsqdCx0aGlzLnkr
dGhpcy5rKm4pfSxhcHBseTpmdW5jdGlvbih0KXtyZXR1cm5bdFswXSp0aGlzLmsrdGhpcy54LHRb
MV0qdGhpcy5rK3RoaXMueV19LGFwcGx5WDpmdW5jdGlvbih0KXtyZXR1cm4gdCp0aGlzLmsrdGhp
cy54fSxhcHBseVk6ZnVuY3Rpb24odCl7cmV0dXJuIHQqdGhpcy5rK3RoaXMueX0saW52ZXJ0OmZ1
bmN0aW9uKHQpe3JldHVyblsodFswXS10aGlzLngpL3RoaXMuaywodFsxXS10aGlzLnkpL3RoaXMu
a119LGludmVydFg6ZnVuY3Rpb24odCl7cmV0dXJuKHQtdGhpcy54KS90aGlzLmt9LGludmVydFk6
ZnVuY3Rpb24odCl7cmV0dXJuKHQtdGhpcy55KS90aGlzLmt9LHJlc2NhbGVYOmZ1bmN0aW9uKHQp
e3JldHVybiB0LmNvcHkoKS5kb21haW4odC5yYW5nZSgpLm1hcCh0aGlzLmludmVydFgsdGhpcyku
bWFwKHQuaW52ZXJ0LHQpKX0scmVzY2FsZVk6ZnVuY3Rpb24odCl7cmV0dXJuIHQuY29weSgpLmRv
bWFpbih0LnJhbmdlKCkubWFwKHRoaXMuaW52ZXJ0WSx0aGlzKS5tYXAodC5pbnZlcnQsdCkpfSx0
b1N0cmluZzpmdW5jdGlvbigpe3JldHVybiJ0cmFuc2xhdGUoIit0aGlzLngrIiwiK3RoaXMueSsi
KSBzY2FsZSgiK3RoaXMuaysiKSJ9fTt2YXIgTXc9bmV3IHd3KDEsMCwwKTtmdW5jdGlvbiBUdyh0
KXtmb3IoOyF0Ll9fem9vbTspaWYoISh0PXQucGFyZW50Tm9kZSkpcmV0dXJuIE13O3JldHVybiB0
Ll9fem9vbX1mdW5jdGlvbiBBdyh0KXt0LnN0b3BJbW1lZGlhdGVQcm9wYWdhdGlvbigpfWZ1bmN0
aW9uIFN3KHQpe3QucHJldmVudERlZmF1bHQoKSx0LnN0b3BJbW1lZGlhdGVQcm9wYWdhdGlvbigp
fWZ1bmN0aW9uIEV3KHQpe3JldHVybiEodC5jdHJsS2V5JiYid2hlZWwiIT09dC50eXBlfHx0LmJ1
dHRvbil9ZnVuY3Rpb24gTncoKXt2YXIgdD10aGlzO3JldHVybiB0IGluc3RhbmNlb2YgU1ZHRWxl
bWVudD8odD10Lm93bmVyU1ZHRWxlbWVudHx8dCkuaGFzQXR0cmlidXRlKCJ2aWV3Qm94Iik/W1so
dD10LnZpZXdCb3guYmFzZVZhbCkueCx0LnldLFt0LngrdC53aWR0aCx0LnkrdC5oZWlnaHRdXTpb
WzAsMF0sW3Qud2lkdGguYmFzZVZhbC52YWx1ZSx0LmhlaWdodC5iYXNlVmFsLnZhbHVlXV06W1sw
LDBdLFt0LmNsaWVudFdpZHRoLHQuY2xpZW50SGVpZ2h0XV19ZnVuY3Rpb24ga3coKXtyZXR1cm4g
dGhpcy5fX3pvb218fE13fWZ1bmN0aW9uIEN3KHQpe3JldHVybi10LmRlbHRhWSooMT09PXQuZGVs
dGFNb2RlPy4wNTp0LmRlbHRhTW9kZT8xOi4wMDIpKih0LmN0cmxLZXk/MTA6MSl9ZnVuY3Rpb24g
UHcoKXtyZXR1cm4gbmF2aWdhdG9yLm1heFRvdWNoUG9pbnRzfHwib250b3VjaHN0YXJ0ImluIHRo
aXN9ZnVuY3Rpb24gencodCxuLGUpe3ZhciByPXQuaW52ZXJ0WChuWzBdWzBdKS1lWzBdWzBdLGk9
dC5pbnZlcnRYKG5bMV1bMF0pLWVbMV1bMF0sbz10LmludmVydFkoblswXVsxXSktZVswXVsxXSxh
PXQuaW52ZXJ0WShuWzFdWzFdKS1lWzFdWzFdO3JldHVybiB0LnRyYW5zbGF0ZShpPnI/KHIraSkv
MjpNYXRoLm1pbigwLHIpfHxNYXRoLm1heCgwLGkpLGE+bz8obythKS8yOk1hdGgubWluKDAsbyl8
fE1hdGgubWF4KDAsYSkpfVR3LnByb3RvdHlwZT13dy5wcm90b3R5cGUsdC5BZGRlcj1BLHQuRGVs
YXVuYXk9anUsdC5Gb3JtYXRTcGVjaWZpZXI9bmYsdC5JbnRlcm5NYXA9SW50ZXJuTWFwLHQuSW50
ZXJuU2V0PUludGVyblNldCx0Lk5vZGU9SmQsdC5QYXRoPUlhLHQuVm9yb25vaT1VdSx0Llpvb21U
cmFuc2Zvcm09d3csdC5hY3RpdmU9ZnVuY3Rpb24odCxuKXt2YXIgZSxyLGk9dC5fX3RyYW5zaXRp
b247aWYoaSlmb3IociBpbiBuPW51bGw9PW4/bnVsbDpuKyIiLGkpaWYoKGU9aVtyXSkuc3RhdGU+
VWkmJmUubmFtZT09PW4pcmV0dXJuIG5ldyBnbyhbW3RdXSxLbyxuLCtyKTtyZXR1cm4gbnVsbH0s
dC5hcmM9ZnVuY3Rpb24oKXt2YXIgdD1DbSxuPVBtLGU9eW0oMCkscj1udWxsLGk9em0sbz0kbSxh
PURtLHU9bnVsbCxjPWttKGYpO2Z1bmN0aW9uIGYoKXt2YXIgZixzLGw9K3QuYXBwbHkodGhpcyxh
cmd1bWVudHMpLGg9K24uYXBwbHkodGhpcyxhcmd1bWVudHMpLGQ9aS5hcHBseSh0aGlzLGFyZ3Vt
ZW50cyktU20scD1vLmFwcGx5KHRoaXMsYXJndW1lbnRzKS1TbSxnPXZtKHAtZCkseT1wPmQ7aWYo
dXx8KHU9Zj1jKCkpLGg8bCYmKHM9aCxoPWwsbD1zKSxoPlRtKWlmKGc+RW0tVG0pdS5tb3ZlVG8o
aCpibShkKSxoKndtKGQpKSx1LmFyYygwLDAsaCxkLHAsIXkpLGw+VG0mJih1Lm1vdmVUbyhsKmJt
KHApLGwqd20ocCkpLHUuYXJjKDAsMCxsLHAsZCx5KSk7ZWxzZXt2YXIgdixfLGI9ZCxtPXAseD1k
LHc9cCxNPWcsVD1nLEE9YS5hcHBseSh0aGlzLGFyZ3VtZW50cykvMixTPUE+VG0mJihyPytyLmFw
cGx5KHRoaXMsYXJndW1lbnRzKTpNbShsKmwraCpoKSksRT14bSh2bShoLWwpLzIsK2UuYXBwbHko
dGhpcyxhcmd1bWVudHMpKSxOPUUsaz1FO2lmKFM+VG0pe3ZhciBDPU5tKFMvbCp3bShBKSksUD1O
bShTL2gqd20oQSkpOyhNLT0yKkMpPlRtPyh4Kz1DKj15PzE6LTEsdy09Qyk6KE09MCx4PXc9KGQr
cCkvMiksKFQtPTIqUCk+VG0/KGIrPVAqPXk/MTotMSxtLT1QKTooVD0wLGI9bT0oZCtwKS8yKX12
YXIgej1oKmJtKGIpLCQ9aCp3bShiKSxEPWwqYm0odyksUj1sKndtKHcpO2lmKEU+VG0pe3ZhciBG
LHE9aCpibShtKSxVPWgqd20obSksST1sKmJtKHgpLE89bCp3bSh4KTtpZihnPEFtKWlmKEY9ZnVu
Y3Rpb24odCxuLGUscixpLG8sYSx1KXt2YXIgYz1lLXQsZj1yLW4scz1hLWksbD11LW8saD1sKmMt
cypmO2lmKCEoaCpoPFRtKSlyZXR1cm5bdCsoaD0ocyoobi1vKS1sKih0LWkpKS9oKSpjLG4raCpm
XX0oeiwkLEksTyxxLFUsRCxSKSl7dmFyIEI9ei1GWzBdLFk9JC1GWzFdLEw9cS1GWzBdLGo9VS1G
WzFdLEg9MS93bShmdW5jdGlvbih0KXtyZXR1cm4gdD4xPzA6dDwtMT9BbTpNYXRoLmFjb3ModCl9
KChCKkwrWSpqKS8oTW0oQipCK1kqWSkqTW0oTCpMK2oqaikpKS8yKSxYPU1tKEZbMF0qRlswXStG
WzFdKkZbMV0pO049eG0oRSwobC1YKS8oSC0xKSksaz14bShFLChoLVgpLyhIKzEpKX1lbHNlIE49
az0wfVQ+VG0/az5UbT8odj1SbShJLE8seiwkLGgsayx5KSxfPVJtKHEsVSxELFIsaCxrLHkpLHUu
bW92ZVRvKHYuY3grdi54MDEsdi5jeSt2LnkwMSksazxFP3UuYXJjKHYuY3gsdi5jeSxrLF9tKHYu
eTAxLHYueDAxKSxfbShfLnkwMSxfLngwMSksIXkpOih1LmFyYyh2LmN4LHYuY3ksayxfbSh2Lnkw
MSx2LngwMSksX20odi55MTEsdi54MTEpLCF5KSx1LmFyYygwLDAsaCxfbSh2LmN5K3YueTExLHYu
Y3grdi54MTEpLF9tKF8uY3krXy55MTEsXy5jeCtfLngxMSksIXkpLHUuYXJjKF8uY3gsXy5jeSxr
LF9tKF8ueTExLF8ueDExKSxfbShfLnkwMSxfLngwMSksIXkpKSk6KHUubW92ZVRvKHosJCksdS5h
cmMoMCwwLGgsYixtLCF5KSk6dS5tb3ZlVG8oeiwkKSxsPlRtJiZNPlRtP04+VG0/KHY9Um0oRCxS
LHEsVSxsLC1OLHkpLF89Um0oeiwkLEksTyxsLC1OLHkpLHUubGluZVRvKHYuY3grdi54MDEsdi5j
eSt2LnkwMSksTjxFP3UuYXJjKHYuY3gsdi5jeSxOLF9tKHYueTAxLHYueDAxKSxfbShfLnkwMSxf
LngwMSksIXkpOih1LmFyYyh2LmN4LHYuY3ksTixfbSh2LnkwMSx2LngwMSksX20odi55MTEsdi54
MTEpLCF5KSx1LmFyYygwLDAsbCxfbSh2LmN5K3YueTExLHYuY3grdi54MTEpLF9tKF8uY3krXy55
MTEsXy5jeCtfLngxMSkseSksdS5hcmMoXy5jeCxfLmN5LE4sX20oXy55MTEsXy54MTEpLF9tKF8u
eTAxLF8ueDAxKSwheSkpKTp1LmFyYygwLDAsbCx3LHgseSk6dS5saW5lVG8oRCxSKX1lbHNlIHUu
bW92ZVRvKDAsMCk7aWYodS5jbG9zZVBhdGgoKSxmKXJldHVybiB1PW51bGwsZisiInx8bnVsbH1y
ZXR1cm4gZi5jZW50cm9pZD1mdW5jdGlvbigpe3ZhciBlPSgrdC5hcHBseSh0aGlzLGFyZ3VtZW50
cykrICtuLmFwcGx5KHRoaXMsYXJndW1lbnRzKSkvMixyPSgraS5hcHBseSh0aGlzLGFyZ3VtZW50
cykrICtvLmFwcGx5KHRoaXMsYXJndW1lbnRzKSkvMi1BbS8yO3JldHVybltibShyKSplLHdtKHIp
KmVdfSxmLmlubmVyUmFkaXVzPWZ1bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh0
PSJmdW5jdGlvbiI9PXR5cGVvZiBuP246eW0oK24pLGYpOnR9LGYub3V0ZXJSYWRpdXM9ZnVuY3Rp
b24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp5
bSgrdCksZik6bn0sZi5jb3JuZXJSYWRpdXM9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/KGU9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp5bSgrdCksZik6ZX0sZi5wYWRSYWRpdXM9
ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9bnVsbD09dD9udWxsOiJmdW5j
dGlvbiI9PXR5cGVvZiB0P3Q6eW0oK3QpLGYpOnJ9LGYuc3RhcnRBbmdsZT1mdW5jdGlvbih0KXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oaT0iZnVuY3Rpb24iPT10eXBlb2YgdD90OnltKCt0KSxm
KTppfSxmLmVuZEFuZ2xlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhvPSJm
dW5jdGlvbiI9PXR5cGVvZiB0P3Q6eW0oK3QpLGYpOm99LGYucGFkQW5nbGU9ZnVuY3Rpb24odCl7
cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGE9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp5bSgrdCks
Zik6YX0sZi5jb250ZXh0PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh1PW51
bGw9PXQ/bnVsbDp0LGYpOnV9LGZ9LHQuYXJlYT1MbSx0LmFyZWFSYWRpYWw9S20sdC5hc2NlbmRp
bmc9bix0LmF1dG9UeXBlPWZ1bmN0aW9uKHQpe2Zvcih2YXIgbiBpbiB0KXt2YXIgZSxyLGk9dFtu
XS50cmltKCk7aWYoaSlpZigidHJ1ZSI9PT1pKWk9ITA7ZWxzZSBpZigiZmFsc2UiPT09aSlpPSEx
O2Vsc2UgaWYoIk5hTiI9PT1pKWk9TmFOO2Vsc2UgaWYoaXNOYU4oZT0raSkpe2lmKCEocj1pLm1h
dGNoKC9eKFstK11cZHsyfSk/XGR7NH0oLVxkezJ9KC1cZHsyfSk/KT8oVFxkezJ9OlxkezJ9KDpc
ZHsyfShcLlxkezN9KT8pPyhafFstK11cZHsyfTpcZHsyfSk/KT8kLykpKWNvbnRpbnVlO3ZjJiZy
WzRdJiYhcls3XSYmKGk9aS5yZXBsYWNlKC8tL2csIi8iKS5yZXBsYWNlKC9ULywiICIpKSxpPW5l
dyBEYXRlKGkpfWVsc2UgaT1lO2Vsc2UgaT1udWxsO3Rbbl09aX1yZXR1cm4gdH0sdC5heGlzQm90
dG9tPWZ1bmN0aW9uKHQpe3JldHVybiB6dChUdCx0KX0sdC5heGlzTGVmdD1mdW5jdGlvbih0KXty
ZXR1cm4genQoQXQsdCl9LHQuYXhpc1JpZ2h0PWZ1bmN0aW9uKHQpe3JldHVybiB6dChNdCx0KX0s
dC5heGlzVG9wPWZ1bmN0aW9uKHQpe3JldHVybiB6dCh3dCx0KX0sdC5iaW49Six0LmJpc2VjdD1s
LHQuYmlzZWN0Q2VudGVyPXMsdC5iaXNlY3RMZWZ0PWYsdC5iaXNlY3RSaWdodD1jLHQuYmlzZWN0
b3I9cix0LmJsb2I9ZnVuY3Rpb24odCxuKXtyZXR1cm4gZmV0Y2godCxuKS50aGVuKF9jKX0sdC5i
bHVyPWZ1bmN0aW9uKHQsbil7aWYoISgobj0rbik+PTApKXRocm93IG5ldyBSYW5nZUVycm9yKCJp
bnZhbGlkIHIiKTtsZXQgZT10Lmxlbmd0aDtpZighKChlPU1hdGguZmxvb3IoZSkpPj0wKSl0aHJv
dyBuZXcgUmFuZ2VFcnJvcigiaW52YWxpZCBsZW5ndGgiKTtpZighZXx8IW4pcmV0dXJuIHQ7Y29u
c3Qgcj12KG4pLGk9dC5zbGljZSgpO3JldHVybiByKHQsaSwwLGUsMSkscihpLHQsMCxlLDEpLHIo
dCxpLDAsZSwxKSx0fSx0LmJsdXIyPWgsdC5ibHVySW1hZ2U9ZCx0LmJydXNoPWZ1bmN0aW9uKCl7
cmV0dXJuIE1hKGhhKX0sdC5icnVzaFNlbGVjdGlvbj1mdW5jdGlvbih0KXt2YXIgbj10Ll9fYnJ1
c2g7cmV0dXJuIG4/bi5kaW0ub3V0cHV0KG4uc2VsZWN0aW9uKTpudWxsfSx0LmJydXNoWD1mdW5j
dGlvbigpe3JldHVybiBNYShzYSl9LHQuYnJ1c2hZPWZ1bmN0aW9uKCl7cmV0dXJuIE1hKGxhKX0s
dC5idWZmZXI9ZnVuY3Rpb24odCxuKXtyZXR1cm4gZmV0Y2godCxuKS50aGVuKGJjKX0sdC5jaG9y
ZD1mdW5jdGlvbigpe3JldHVybiAkYSghMSwhMSl9LHQuY2hvcmREaXJlY3RlZD1mdW5jdGlvbigp
e3JldHVybiAkYSghMCwhMSl9LHQuY2hvcmRUcmFuc3Bvc2U9ZnVuY3Rpb24oKXtyZXR1cm4gJGEo
ITEsITApfSx0LmNsdXN0ZXI9ZnVuY3Rpb24oKXt2YXIgdD1qZCxuPTEsZT0xLHI9ITE7ZnVuY3Rp
b24gaShpKXt2YXIgbyxhPTA7aS5lYWNoQWZ0ZXIoKGZ1bmN0aW9uKG4pe3ZhciBlPW4uY2hpbGRy
ZW47ZT8obi54PWZ1bmN0aW9uKHQpe3JldHVybiB0LnJlZHVjZShIZCwwKS90Lmxlbmd0aH0oZSks
bi55PWZ1bmN0aW9uKHQpe3JldHVybiAxK3QucmVkdWNlKFhkLDApfShlKSk6KG4ueD1vP2ErPXQo
bixvKTowLG4ueT0wLG89bil9KSk7dmFyIHU9ZnVuY3Rpb24odCl7Zm9yKHZhciBuO249dC5jaGls
ZHJlbjspdD1uWzBdO3JldHVybiB0fShpKSxjPWZ1bmN0aW9uKHQpe2Zvcih2YXIgbjtuPXQuY2hp
bGRyZW47KXQ9bltuLmxlbmd0aC0xXTtyZXR1cm4gdH0oaSksZj11LngtdCh1LGMpLzIscz1jLngr
dChjLHUpLzI7cmV0dXJuIGkuZWFjaEFmdGVyKHI/ZnVuY3Rpb24odCl7dC54PSh0LngtaS54KSpu
LHQueT0oaS55LXQueSkqZX06ZnVuY3Rpb24odCl7dC54PSh0LngtZikvKHMtZikqbix0Lnk9KDEt
KGkueT90LnkvaS55OjEpKSplfSl9cmV0dXJuIGkuc2VwYXJhdGlvbj1mdW5jdGlvbihuKXtyZXR1
cm4gYXJndW1lbnRzLmxlbmd0aD8odD1uLGkpOnR9LGkuc2l6ZT1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8ocj0hMSxuPSt0WzBdLGU9K3RbMV0saSk6cj9udWxsOltuLGVdfSxp
Lm5vZGVTaXplPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhyPSEwLG49K3Rb
MF0sZT0rdFsxXSxpKTpyP1tuLGVdOm51bGx9LGl9LHQuY29sb3I9JGUsdC5jb250b3VyRGVuc2l0
eT1mdW5jdGlvbigpe3ZhciB0PXN1LG49bHUsZT1odSxyPTk2MCxpPTUwMCxvPTIwLGE9Mix1PTMq
byxjPXIrMip1Pj5hLGY9aSsyKnU+PmEscz1KYSgyMCk7ZnVuY3Rpb24gbChyKXt2YXIgaT1uZXcg
RmxvYXQzMkFycmF5KGMqZikscz1NYXRoLnBvdygyLC1hKSxsPS0xO2Zvcihjb25zdCBvIG9mIHIp
e3ZhciBkPSh0KG8sKytsLHIpK3UpKnMscD0obihvLGwscikrdSkqcyxnPStlKG8sbCxyKTtpZihn
JiZkPj0wJiZkPGMmJnA+PTAmJnA8Zil7dmFyIHk9TWF0aC5mbG9vcihkKSx2PU1hdGguZmxvb3Io
cCksXz1kLXktLjUsYj1wLXYtLjU7aVt5K3YqY10rPSgxLV8pKigxLWIpKmcsaVt5KzErdipjXSs9
XyooMS1iKSpnLGlbeSsxKyh2KzEpKmNdKz1fKmIqZyxpW3krKHYrMSkqY10rPSgxLV8pKmIqZ319
cmV0dXJuIGgoe2RhdGE6aSx3aWR0aDpjLGhlaWdodDpmfSxvKnMpLGl9ZnVuY3Rpb24gZCh0KXt2
YXIgbj1sKHQpLGU9cyhuKSxyPU1hdGgucG93KDIsMiphKTtyZXR1cm4gQXJyYXkuaXNBcnJheShl
KXx8KGU9VihOdW1iZXIuTUlOX1ZBTFVFLHR0KG4pL3IsZSkpLG91KCkuc2l6ZShbYyxmXSkudGhy
ZXNob2xkcyhlLm1hcCgodD0+dCpyKSkpKG4pLm1hcCgoKHQsbik9Pih0LnZhbHVlPStlW25dLHAo
dCkpKSl9ZnVuY3Rpb24gcCh0KXtyZXR1cm4gdC5jb29yZGluYXRlcy5mb3JFYWNoKGcpLHR9ZnVu
Y3Rpb24gZyh0KXt0LmZvckVhY2goeSl9ZnVuY3Rpb24geSh0KXt0LmZvckVhY2godil9ZnVuY3Rp
b24gdih0KXt0WzBdPXRbMF0qTWF0aC5wb3coMixhKS11LHRbMV09dFsxXSpNYXRoLnBvdygyLGEp
LXV9ZnVuY3Rpb24gXygpe3JldHVybiBjPXIrMioodT0zKm8pPj5hLGY9aSsyKnU+PmEsZH1yZXR1
cm4gZC5jb250b3Vycz1mdW5jdGlvbih0KXt2YXIgbj1sKHQpLGU9b3UoKS5zaXplKFtjLGZdKSxy
PU1hdGgucG93KDIsMiphKSxpPXQ9Pnt0PSt0O3ZhciBpPXAoZS5jb250b3VyKG4sdCpyKSk7cmV0
dXJuIGkudmFsdWU9dCxpfTtyZXR1cm4gT2JqZWN0LmRlZmluZVByb3BlcnR5KGksIm1heCIse2dl
dDooKT0+dHQobikvcn0pLGl9LGQueD1mdW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0
aD8odD0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOkphKCtuKSxkKTp0fSxkLnk9ZnVuY3Rpb24odCl7
cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpKYSgrdCks
ZCk6bn0sZC53ZWlnaHQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGU9ImZ1
bmN0aW9uIj09dHlwZW9mIHQ/dDpKYSgrdCksZCk6ZX0sZC5zaXplPWZ1bmN0aW9uKHQpe2lmKCFh
cmd1bWVudHMubGVuZ3RoKXJldHVybltyLGldO3ZhciBuPSt0WzBdLGU9K3RbMV07aWYoIShuPj0w
JiZlPj0wKSl0aHJvdyBuZXcgRXJyb3IoImludmFsaWQgc2l6ZSIpO3JldHVybiByPW4saT1lLF8o
KX0sZC5jZWxsU2l6ZT1mdW5jdGlvbih0KXtpZighYXJndW1lbnRzLmxlbmd0aClyZXR1cm4gMTw8
YTtpZighKCh0PSt0KT49MSkpdGhyb3cgbmV3IEVycm9yKCJpbnZhbGlkIGNlbGwgc2l6ZSIpO3Jl
dHVybiBhPU1hdGguZmxvb3IoTWF0aC5sb2codCkvTWF0aC5MTjIpLF8oKX0sZC50aHJlc2hvbGRz
PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhzPSJmdW5jdGlvbiI9PXR5cGVv
ZiB0P3Q6QXJyYXkuaXNBcnJheSh0KT9KYShLYS5jYWxsKHQpKTpKYSh0KSxkKTpzfSxkLmJhbmR3
aWR0aD1mdW5jdGlvbih0KXtpZighYXJndW1lbnRzLmxlbmd0aClyZXR1cm4gTWF0aC5zcXJ0KG8q
KG8rMSkpO2lmKCEoKHQ9K3QpPj0wKSl0aHJvdyBuZXcgRXJyb3IoImludmFsaWQgYmFuZHdpZHRo
Iik7cmV0dXJuIG89KE1hdGguc3FydCg0KnQqdCsxKS0xKS8yLF8oKX0sZH0sdC5jb250b3Vycz1v
dSx0LmNvdW50PV8sdC5jcmVhdGU9ZnVuY3Rpb24odCl7cmV0dXJuIEtuKEx0KHQpLmNhbGwoZG9j
dW1lbnQuZG9jdW1lbnRFbGVtZW50KSl9LHQuY3JlYXRvcj1MdCx0LmNyb3NzPWZ1bmN0aW9uKC4u
LnQpe2NvbnN0IG49ImZ1bmN0aW9uIj09dHlwZW9mIHRbdC5sZW5ndGgtMV0mJmZ1bmN0aW9uKHQp
e3JldHVybiBuPT50KC4uLm4pfSh0LnBvcCgpKSxlPSh0PXQubWFwKHgpKS5tYXAoYikscj10Lmxl
bmd0aC0xLGk9bmV3IEFycmF5KHIrMSkuZmlsbCgwKSxvPVtdO2lmKHI8MHx8ZS5zb21lKG0pKXJl
dHVybiBvO2Zvcig7Oyl7by5wdXNoKGkubWFwKCgobixlKT0+dFtlXVtuXSkpKTtsZXQgYT1yO2Zv
cig7KytpW2FdPT09ZVthXTspe2lmKDA9PT1hKXJldHVybiBuP28ubWFwKG4pOm87aVthLS1dPTB9
fX0sdC5jc3Y9TWMsdC5jc3ZGb3JtYXQ9aWMsdC5jc3ZGb3JtYXRCb2R5PW9jLHQuY3N2Rm9ybWF0
Um93PXVjLHQuY3N2Rm9ybWF0Um93cz1hYyx0LmNzdkZvcm1hdFZhbHVlPWNjLHQuY3N2UGFyc2U9
ZWMsdC5jc3ZQYXJzZVJvd3M9cmMsdC5jdWJlaGVsaXg9QXIsdC5jdW1zdW09ZnVuY3Rpb24odCxu
KXt2YXIgZT0wLHI9MDtyZXR1cm4gRmxvYXQ2NEFycmF5LmZyb20odCx2b2lkIDA9PT1uP3Q9PmUr
PSt0fHwwOmk9PmUrPStuKGkscisrLHQpfHwwKX0sdC5jdXJ2ZUJhc2lzPWZ1bmN0aW9uKHQpe3Jl
dHVybiBuZXcgRngodCl9LHQuY3VydmVCYXNpc0Nsb3NlZD1mdW5jdGlvbih0KXtyZXR1cm4gbmV3
IHF4KHQpfSx0LmN1cnZlQmFzaXNPcGVuPWZ1bmN0aW9uKHQpe3JldHVybiBuZXcgVXgodCl9LHQu
Y3VydmVCdW1wWD1ueCx0LmN1cnZlQnVtcFk9ZXgsdC5jdXJ2ZUJ1bmRsZT1PeCx0LmN1cnZlQ2Fy
ZGluYWw9THgsdC5jdXJ2ZUNhcmRpbmFsQ2xvc2VkPUh4LHQuY3VydmVDYXJkaW5hbE9wZW49R3gs
dC5jdXJ2ZUNhdG11bGxSb209WngsdC5jdXJ2ZUNhdG11bGxSb21DbG9zZWQ9UXgsdC5jdXJ2ZUNh
dG11bGxSb21PcGVuPXR3LHQuY3VydmVMaW5lYXI9SW0sdC5jdXJ2ZUxpbmVhckNsb3NlZD1mdW5j
dGlvbih0KXtyZXR1cm4gbmV3IG53KHQpfSx0LmN1cnZlTW9ub3RvbmVYPWZ1bmN0aW9uKHQpe3Jl
dHVybiBuZXcgYXcodCl9LHQuY3VydmVNb25vdG9uZVk9ZnVuY3Rpb24odCl7cmV0dXJuIG5ldyB1
dyh0KX0sdC5jdXJ2ZU5hdHVyYWw9ZnVuY3Rpb24odCl7cmV0dXJuIG5ldyBmdyh0KX0sdC5jdXJ2
ZVN0ZXA9ZnVuY3Rpb24odCl7cmV0dXJuIG5ldyBsdyh0LC41KX0sdC5jdXJ2ZVN0ZXBBZnRlcj1m
dW5jdGlvbih0KXtyZXR1cm4gbmV3IGx3KHQsMSl9LHQuY3VydmVTdGVwQmVmb3JlPWZ1bmN0aW9u
KHQpe3JldHVybiBuZXcgbHcodCwwKX0sdC5kZXNjZW5kaW5nPWUsdC5kZXZpYXRpb249TSx0LmRp
ZmZlcmVuY2U9ZnVuY3Rpb24odCwuLi5uKXt0PW5ldyBJbnRlcm5TZXQodCk7Zm9yKGNvbnN0IGUg
b2Ygbilmb3IoY29uc3QgbiBvZiBlKXQuZGVsZXRlKG4pO3JldHVybiB0fSx0LmRpc2pvaW50PWZ1
bmN0aW9uKHQsbil7Y29uc3QgZT1uW1N5bWJvbC5pdGVyYXRvcl0oKSxyPW5ldyBJbnRlcm5TZXQ7
Zm9yKGNvbnN0IG4gb2YgdCl7aWYoci5oYXMobikpcmV0dXJuITE7bGV0IHQsaTtmb3IoOyh7dmFs
dWU6dCxkb25lOml9PWUubmV4dCgpKSYmIWk7KXtpZihPYmplY3QuaXMobix0KSlyZXR1cm4hMTty
LmFkZCh0KX19cmV0dXJuITB9LHQuZGlzcGF0Y2g9RHQsdC5kcmFnPWZ1bmN0aW9uKCl7dmFyIHQs
bixlLHIsaT1sZSxvPWhlLGE9ZGUsdT1wZSxjPXt9LGY9RHQoInN0YXJ0IiwiZHJhZyIsImVuZCIp
LHM9MCxsPTA7ZnVuY3Rpb24gaCh0KXt0Lm9uKCJtb3VzZWRvd24uZHJhZyIsZCkuZmlsdGVyKHUp
Lm9uKCJ0b3VjaHN0YXJ0LmRyYWciLHkpLm9uKCJ0b3VjaG1vdmUuZHJhZyIsdixyZSkub24oInRv
dWNoZW5kLmRyYWcgdG91Y2hjYW5jZWwuZHJhZyIsXykuc3R5bGUoInRvdWNoLWFjdGlvbiIsIm5v
bmUiKS5zdHlsZSgiLXdlYmtpdC10YXAtaGlnaGxpZ2h0LWNvbG9yIiwicmdiYSgwLDAsMCwwKSIp
fWZ1bmN0aW9uIGQoYSx1KXtpZighciYmaS5jYWxsKHRoaXMsYSx1KSl7dmFyIGM9Yih0aGlzLG8u
Y2FsbCh0aGlzLGEsdSksYSx1LCJtb3VzZSIpO2MmJihLbihhLnZpZXcpLm9uKCJtb3VzZW1vdmUu
ZHJhZyIscCxpZSkub24oIm1vdXNldXAuZHJhZyIsZyxpZSksdWUoYS52aWV3KSxvZShhKSxlPSEx
LHQ9YS5jbGllbnRYLG49YS5jbGllbnRZLGMoInN0YXJ0IixhKSl9fWZ1bmN0aW9uIHAocil7aWYo
YWUociksIWUpe3ZhciBpPXIuY2xpZW50WC10LG89ci5jbGllbnRZLW47ZT1pKmkrbypvPmx9Yy5t
b3VzZSgiZHJhZyIscil9ZnVuY3Rpb24gZyh0KXtLbih0LnZpZXcpLm9uKCJtb3VzZW1vdmUuZHJh
ZyBtb3VzZXVwLmRyYWciLG51bGwpLGNlKHQudmlldyxlKSxhZSh0KSxjLm1vdXNlKCJlbmQiLHQp
fWZ1bmN0aW9uIHkodCxuKXtpZihpLmNhbGwodGhpcyx0LG4pKXt2YXIgZSxyLGE9dC5jaGFuZ2Vk
VG91Y2hlcyx1PW8uY2FsbCh0aGlzLHQsbiksYz1hLmxlbmd0aDtmb3IoZT0wO2U8YzsrK2UpKHI9
Yih0aGlzLHUsdCxuLGFbZV0uaWRlbnRpZmllcixhW2VdKSkmJihvZSh0KSxyKCJzdGFydCIsdCxh
W2VdKSl9fWZ1bmN0aW9uIHYodCl7dmFyIG4sZSxyPXQuY2hhbmdlZFRvdWNoZXMsaT1yLmxlbmd0
aDtmb3Iobj0wO248aTsrK24pKGU9Y1tyW25dLmlkZW50aWZpZXJdKSYmKGFlKHQpLGUoImRyYWci
LHQscltuXSkpfWZ1bmN0aW9uIF8odCl7dmFyIG4sZSxpPXQuY2hhbmdlZFRvdWNoZXMsbz1pLmxl
bmd0aDtmb3IociYmY2xlYXJUaW1lb3V0KHIpLHI9c2V0VGltZW91dCgoZnVuY3Rpb24oKXtyPW51
bGx9KSw1MDApLG49MDtuPG87KytuKShlPWNbaVtuXS5pZGVudGlmaWVyXSkmJihvZSh0KSxlKCJl
bmQiLHQsaVtuXSkpfWZ1bmN0aW9uIGIodCxuLGUscixpLG8pe3ZhciB1LGwsZCxwPWYuY29weSgp
LGc9ZWUob3x8ZSxuKTtpZihudWxsIT0oZD1hLmNhbGwodCxuZXcgc2UoImJlZm9yZXN0YXJ0Iix7
c291cmNlRXZlbnQ6ZSx0YXJnZXQ6aCxpZGVudGlmaWVyOmksYWN0aXZlOnMseDpnWzBdLHk6Z1sx
XSxkeDowLGR5OjAsZGlzcGF0Y2g6cH0pLHIpKSlyZXR1cm4gdT1kLngtZ1swXXx8MCxsPWQueS1n
WzFdfHwwLGZ1bmN0aW9uIGUobyxhLGYpe3ZhciB5LHY9Zztzd2l0Y2gobyl7Y2FzZSJzdGFydCI6
Y1tpXT1lLHk9cysrO2JyZWFrO2Nhc2UiZW5kIjpkZWxldGUgY1tpXSwtLXM7Y2FzZSJkcmFnIjpn
PWVlKGZ8fGEsbikseT1zfXAuY2FsbChvLHQsbmV3IHNlKG8se3NvdXJjZUV2ZW50OmEsc3ViamVj
dDpkLHRhcmdldDpoLGlkZW50aWZpZXI6aSxhY3RpdmU6eSx4OmdbMF0rdSx5OmdbMV0rbCxkeDpn
WzBdLXZbMF0sZHk6Z1sxXS12WzFdLGRpc3BhdGNoOnB9KSxyKX19cmV0dXJuIGguZmlsdGVyPWZ1
bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhpPSJmdW5jdGlvbiI9PXR5cGVvZiB0
P3Q6ZmUoISF0KSxoKTppfSxoLmNvbnRhaW5lcj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD8obz0iZnVuY3Rpb24iPT10eXBlb2YgdD90OmZlKHQpLGgpOm99LGguc3ViamVjdD1m
dW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYT0iZnVuY3Rpb24iPT10eXBlb2Yg
dD90OmZlKHQpLGgpOmF9LGgudG91Y2hhYmxlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPyh1PSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6ZmUoISF0KSxoKTp1fSxoLm9uPWZ1bmN0
aW9uKCl7dmFyIHQ9Zi5vbi5hcHBseShmLGFyZ3VtZW50cyk7cmV0dXJuIHQ9PT1mP2g6dH0saC5j
bGlja0Rpc3RhbmNlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhsPSh0PSt0
KSp0LGgpOk1hdGguc3FydChsKX0saH0sdC5kcmFnRGlzYWJsZT11ZSx0LmRyYWdFbmFibGU9Y2Us
dC5kc3Y9ZnVuY3Rpb24odCxuLGUscil7Mz09PWFyZ3VtZW50cy5sZW5ndGgmJiJmdW5jdGlvbiI9
PXR5cGVvZiBlJiYocj1lLGU9dm9pZCAwKTt2YXIgaT10Yyh0KTtyZXR1cm4geGMobixlKS50aGVu
KChmdW5jdGlvbih0KXtyZXR1cm4gaS5wYXJzZSh0LHIpfSkpfSx0LmRzdkZvcm1hdD10Yyx0LmVh
c2VCYWNrPWpvLHQuZWFzZUJhY2tJbj1Zbyx0LmVhc2VCYWNrSW5PdXQ9am8sdC5lYXNlQmFja091
dD1Mbyx0LmVhc2VCb3VuY2U9T28sdC5lYXNlQm91bmNlSW49ZnVuY3Rpb24odCl7cmV0dXJuIDEt
T28oMS10KX0sdC5lYXNlQm91bmNlSW5PdXQ9ZnVuY3Rpb24odCl7cmV0dXJuKCh0Kj0yKTw9MT8x
LU9vKDEtdCk6T28odC0xKSsxKS8yfSx0LmVhc2VCb3VuY2VPdXQ9T28sdC5lYXNlQ2lyY2xlPWtv
LHQuZWFzZUNpcmNsZUluPWZ1bmN0aW9uKHQpe3JldHVybiAxLU1hdGguc3FydCgxLXQqdCl9LHQu
ZWFzZUNpcmNsZUluT3V0PWtvLHQuZWFzZUNpcmNsZU91dD1mdW5jdGlvbih0KXtyZXR1cm4gTWF0
aC5zcXJ0KDEtIC0tdCp0KX0sdC5lYXNlQ3ViaWM9bW8sdC5lYXNlQ3ViaWNJbj1mdW5jdGlvbih0
KXtyZXR1cm4gdCp0KnR9LHQuZWFzZUN1YmljSW5PdXQ9bW8sdC5lYXNlQ3ViaWNPdXQ9ZnVuY3Rp
b24odCl7cmV0dXJuLS10KnQqdCsxfSx0LmVhc2VFbGFzdGljPUdvLHQuZWFzZUVsYXN0aWNJbj1Y
byx0LmVhc2VFbGFzdGljSW5PdXQ9Vm8sdC5lYXNlRWxhc3RpY091dD1Hbyx0LmVhc2VFeHA9Tm8s
dC5lYXNlRXhwSW49ZnVuY3Rpb24odCl7cmV0dXJuIEVvKDEtK3QpfSx0LmVhc2VFeHBJbk91dD1O
byx0LmVhc2VFeHBPdXQ9ZnVuY3Rpb24odCl7cmV0dXJuIDEtRW8odCl9LHQuZWFzZUxpbmVhcj10
PT4rdCx0LmVhc2VQb2x5PU1vLHQuZWFzZVBvbHlJbj14byx0LmVhc2VQb2x5SW5PdXQ9TW8sdC5l
YXNlUG9seU91dD13byx0LmVhc2VRdWFkPWJvLHQuZWFzZVF1YWRJbj1mdW5jdGlvbih0KXtyZXR1
cm4gdCp0fSx0LmVhc2VRdWFkSW5PdXQ9Ym8sdC5lYXNlUXVhZE91dD1mdW5jdGlvbih0KXtyZXR1
cm4gdCooMi10KX0sdC5lYXNlU2luPVNvLHQuZWFzZVNpbkluPWZ1bmN0aW9uKHQpe3JldHVybiAx
PT0rdD8xOjEtTWF0aC5jb3ModCpBbyl9LHQuZWFzZVNpbkluT3V0PVNvLHQuZWFzZVNpbk91dD1m
dW5jdGlvbih0KXtyZXR1cm4gTWF0aC5zaW4odCpBbyl9LHQuZXZlcnk9ZnVuY3Rpb24odCxuKXtp
ZigiZnVuY3Rpb24iIT10eXBlb2Ygbil0aHJvdyBuZXcgVHlwZUVycm9yKCJ0ZXN0IGlzIG5vdCBh
IGZ1bmN0aW9uIik7bGV0IGU9LTE7Zm9yKGNvbnN0IHIgb2YgdClpZighbihyLCsrZSx0KSlyZXR1
cm4hMTtyZXR1cm4hMH0sdC5leHRlbnQ9VCx0LmZjdW1zdW09ZnVuY3Rpb24odCxuKXtjb25zdCBl
PW5ldyBBO2xldCByPS0xO3JldHVybiBGbG9hdDY0QXJyYXkuZnJvbSh0LHZvaWQgMD09PW4/dD0+
ZS5hZGQoK3R8fDApOmk9PmUuYWRkKCtuKGksKytyLHQpfHwwKSl9LHQuZmlsdGVyPWZ1bmN0aW9u
KHQsbil7aWYoImZ1bmN0aW9uIiE9dHlwZW9mIG4pdGhyb3cgbmV3IFR5cGVFcnJvcigidGVzdCBp
cyBub3QgYSBmdW5jdGlvbiIpO2NvbnN0IGU9W107bGV0IHI9LTE7Zm9yKGNvbnN0IGkgb2YgdClu
KGksKytyLHQpJiZlLnB1c2goaSk7cmV0dXJuIGV9LHQuZmxhdEdyb3VwPWZ1bmN0aW9uKHQsLi4u
bil7cmV0dXJuICQoeih0LC4uLm4pLG4pfSx0LmZsYXRSb2xsdXA9ZnVuY3Rpb24odCxuLC4uLmUp
e3JldHVybiAkKFIodCxuLC4uLmUpLGUpfSx0LmZvcmNlQ2VudGVyPWZ1bmN0aW9uKHQsbil7dmFy
IGUscj0xO2Z1bmN0aW9uIGkoKXt2YXIgaSxvLGE9ZS5sZW5ndGgsdT0wLGM9MDtmb3IoaT0wO2k8
YTsrK2kpdSs9KG89ZVtpXSkueCxjKz1vLnk7Zm9yKHU9KHUvYS10KSpyLGM9KGMvYS1uKSpyLGk9
MDtpPGE7KytpKShvPWVbaV0pLngtPXUsby55LT1jfXJldHVybiBudWxsPT10JiYodD0wKSxudWxs
PT1uJiYobj0wKSxpLmluaXRpYWxpemU9ZnVuY3Rpb24odCl7ZT10fSxpLng9ZnVuY3Rpb24obil7
cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHQ9K24saSk6dH0saS55PWZ1bmN0aW9uKHQpe3JldHVy
biBhcmd1bWVudHMubGVuZ3RoPyhuPSt0LGkpOm59LGkuc3RyZW5ndGg9ZnVuY3Rpb24odCl7cmV0
dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9K3QsaSk6cn0saX0sdC5mb3JjZUNvbGxpZGU9ZnVuY3Rp
b24odCl7dmFyIG4sZSxyLGk9MSxvPTE7ZnVuY3Rpb24gYSgpe2Zvcih2YXIgdCxhLGMsZixzLGws
aCxkPW4ubGVuZ3RoLHA9MDtwPG87KytwKWZvcihhPURjKG4sT2MsQmMpLnZpc2l0QWZ0ZXIodSks
dD0wO3Q8ZDsrK3QpYz1uW3RdLGw9ZVtjLmluZGV4XSxoPWwqbCxmPWMueCtjLnZ4LHM9Yy55K2Mu
dnksYS52aXNpdChnKTtmdW5jdGlvbiBnKHQsbixlLG8sYSl7dmFyIHU9dC5kYXRhLGQ9dC5yLHA9
bCtkO2lmKCF1KXJldHVybiBuPmYrcHx8bzxmLXB8fGU+cytwfHxhPHMtcDtpZih1LmluZGV4PmMu
aW5kZXgpe3ZhciBnPWYtdS54LXUudngseT1zLXUueS11LnZ5LHY9ZypnK3kqeTt2PHAqcCYmKDA9
PT1nJiYodis9KGc9SWMocikpKmcpLDA9PT15JiYodis9KHk9SWMocikpKnkpLHY9KHAtKHY9TWF0
aC5zcXJ0KHYpKSkvdippLGMudngrPShnKj12KSoocD0oZCo9ZCkvKGgrZCkpLGMudnkrPSh5Kj12
KSpwLHUudngtPWcqKHA9MS1wKSx1LnZ5LT15KnApfX19ZnVuY3Rpb24gdSh0KXtpZih0LmRhdGEp
cmV0dXJuIHQucj1lW3QuZGF0YS5pbmRleF07Zm9yKHZhciBuPXQucj0wO248NDsrK24pdFtuXSYm
dFtuXS5yPnQuciYmKHQucj10W25dLnIpfWZ1bmN0aW9uIGMoKXtpZihuKXt2YXIgcixpLG89bi5s
ZW5ndGg7Zm9yKGU9bmV3IEFycmF5KG8pLHI9MDtyPG87KytyKWk9bltyXSxlW2kuaW5kZXhdPSt0
KGkscixuKX19cmV0dXJuImZ1bmN0aW9uIiE9dHlwZW9mIHQmJih0PVVjKG51bGw9PXQ/MTordCkp
LGEuaW5pdGlhbGl6ZT1mdW5jdGlvbih0LGUpe249dCxyPWUsYygpfSxhLml0ZXJhdGlvbnM9ZnVu
Y3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG89K3QsYSk6b30sYS5zdHJlbmd0aD1m
dW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oaT0rdCxhKTppfSxhLnJhZGl1cz1m
dW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD0iZnVuY3Rpb24iPT10eXBlb2Yg
bj9uOlVjKCtuKSxjKCksYSk6dH0sYX0sdC5mb3JjZUxpbms9ZnVuY3Rpb24odCl7dmFyIG4sZSxy
LGksbyxhLHU9WWMsYz1mdW5jdGlvbih0KXtyZXR1cm4gMS9NYXRoLm1pbihpW3Quc291cmNlLmlu
ZGV4XSxpW3QudGFyZ2V0LmluZGV4XSl9LGY9VWMoMzApLHM9MTtmdW5jdGlvbiBsKHIpe2Zvcih2
YXIgaT0wLHU9dC5sZW5ndGg7aTxzOysraSlmb3IodmFyIGMsZixsLGgsZCxwLGcseT0wO3k8dTsr
K3kpZj0oYz10W3ldKS5zb3VyY2UsaD0obD1jLnRhcmdldCkueCtsLnZ4LWYueC1mLnZ4fHxJYyhh
KSxkPWwueStsLnZ5LWYueS1mLnZ5fHxJYyhhKSxoKj1wPSgocD1NYXRoLnNxcnQoaCpoK2QqZCkp
LWVbeV0pL3AqcipuW3ldLGQqPXAsbC52eC09aCooZz1vW3ldKSxsLnZ5LT1kKmcsZi52eCs9aCoo
Zz0xLWcpLGYudnkrPWQqZ31mdW5jdGlvbiBoKCl7aWYocil7dmFyIGEsYyxmPXIubGVuZ3RoLHM9
dC5sZW5ndGgsbD1uZXcgTWFwKHIubWFwKCgodCxuKT0+W3UodCxuLHIpLHRdKSkpO2ZvcihhPTAs
aT1uZXcgQXJyYXkoZik7YTxzOysrYSkoYz10W2FdKS5pbmRleD1hLCJvYmplY3QiIT10eXBlb2Yg
Yy5zb3VyY2UmJihjLnNvdXJjZT1MYyhsLGMuc291cmNlKSksIm9iamVjdCIhPXR5cGVvZiBjLnRh
cmdldCYmKGMudGFyZ2V0PUxjKGwsYy50YXJnZXQpKSxpW2Muc291cmNlLmluZGV4XT0oaVtjLnNv
dXJjZS5pbmRleF18fDApKzEsaVtjLnRhcmdldC5pbmRleF09KGlbYy50YXJnZXQuaW5kZXhdfHww
KSsxO2ZvcihhPTAsbz1uZXcgQXJyYXkocyk7YTxzOysrYSljPXRbYV0sb1thXT1pW2Muc291cmNl
LmluZGV4XS8oaVtjLnNvdXJjZS5pbmRleF0raVtjLnRhcmdldC5pbmRleF0pO249bmV3IEFycmF5
KHMpLGQoKSxlPW5ldyBBcnJheShzKSxwKCl9fWZ1bmN0aW9uIGQoKXtpZihyKWZvcih2YXIgZT0w
LGk9dC5sZW5ndGg7ZTxpOysrZSluW2VdPStjKHRbZV0sZSx0KX1mdW5jdGlvbiBwKCl7aWYocilm
b3IodmFyIG49MCxpPXQubGVuZ3RoO248aTsrK24pZVtuXT0rZih0W25dLG4sdCl9cmV0dXJuIG51
bGw9PXQmJih0PVtdKSxsLmluaXRpYWxpemU9ZnVuY3Rpb24odCxuKXtyPXQsYT1uLGgoKX0sbC5s
aW5rcz1mdW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD1uLGgoKSxsKTp0fSxs
LmlkPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh1PXQsbCk6dX0sbC5pdGVy
YXRpb25zPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhzPSt0LGwpOnN9LGwu
c3RyZW5ndGg9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGM9ImZ1bmN0aW9u
Ij09dHlwZW9mIHQ/dDpVYygrdCksZCgpLGwpOmN9LGwuZGlzdGFuY2U9ZnVuY3Rpb24odCl7cmV0
dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGY9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpVYygrdCkscCgp
LGwpOmZ9LGx9LHQuZm9yY2VNYW55Qm9keT1mdW5jdGlvbigpe3ZhciB0LG4sZSxyLGksbz1VYygt
MzApLGE9MSx1PTEvMCxjPS44MTtmdW5jdGlvbiBmKGUpe3ZhciBpLG89dC5sZW5ndGgsYT1EYyh0
LEdjLFZjKS52aXNpdEFmdGVyKGwpO2ZvcihyPWUsaT0wO2k8bzsrK2kpbj10W2ldLGEudmlzaXQo
aCl9ZnVuY3Rpb24gcygpe2lmKHQpe3ZhciBuLGUscj10Lmxlbmd0aDtmb3IoaT1uZXcgQXJyYXko
ciksbj0wO248cjsrK24pZT10W25dLGlbZS5pbmRleF09K28oZSxuLHQpfX1mdW5jdGlvbiBsKHQp
e3ZhciBuLGUscixvLGEsdT0wLGM9MDtpZih0Lmxlbmd0aCl7Zm9yKHI9bz1hPTA7YTw0OysrYSko
bj10W2FdKSYmKGU9TWF0aC5hYnMobi52YWx1ZSkpJiYodSs9bi52YWx1ZSxjKz1lLHIrPWUqbi54
LG8rPWUqbi55KTt0Lng9ci9jLHQueT1vL2N9ZWxzZXsobj10KS54PW4uZGF0YS54LG4ueT1uLmRh
dGEueTtkb3t1Kz1pW24uZGF0YS5pbmRleF19d2hpbGUobj1uLm5leHQpfXQudmFsdWU9dX1mdW5j
dGlvbiBoKHQsbyxmLHMpe2lmKCF0LnZhbHVlKXJldHVybiEwO3ZhciBsPXQueC1uLngsaD10Lnkt
bi55LGQ9cy1vLHA9bCpsK2gqaDtpZihkKmQvYzxwKXJldHVybiBwPHUmJigwPT09bCYmKHArPShs
PUljKGUpKSpsKSwwPT09aCYmKHArPShoPUljKGUpKSpoKSxwPGEmJihwPU1hdGguc3FydChhKnAp
KSxuLnZ4Kz1sKnQudmFsdWUqci9wLG4udnkrPWgqdC52YWx1ZSpyL3ApLCEwO2lmKCEodC5sZW5n
dGh8fHA+PXUpKXsodC5kYXRhIT09bnx8dC5uZXh0KSYmKDA9PT1sJiYocCs9KGw9SWMoZSkpKmwp
LDA9PT1oJiYocCs9KGg9SWMoZSkpKmgpLHA8YSYmKHA9TWF0aC5zcXJ0KGEqcCkpKTtkb3t0LmRh
dGEhPT1uJiYoZD1pW3QuZGF0YS5pbmRleF0qci9wLG4udngrPWwqZCxuLnZ5Kz1oKmQpfXdoaWxl
KHQ9dC5uZXh0KX19cmV0dXJuIGYuaW5pdGlhbGl6ZT1mdW5jdGlvbihuLHIpe3Q9bixlPXIscygp
fSxmLnN0cmVuZ3RoPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhvPSJmdW5j
dGlvbiI9PXR5cGVvZiB0P3Q6VWMoK3QpLHMoKSxmKTpvfSxmLmRpc3RhbmNlTWluPWZ1bmN0aW9u
KHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhhPXQqdCxmKTpNYXRoLnNxcnQoYSl9LGYuZGlz
dGFuY2VNYXg9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHU9dCp0LGYpOk1h
dGguc3FydCh1KX0sZi50aGV0YT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8o
Yz10KnQsZik6TWF0aC5zcXJ0KGMpfSxmfSx0LmZvcmNlUmFkaWFsPWZ1bmN0aW9uKHQsbixlKXt2
YXIgcixpLG8sYT1VYyguMSk7ZnVuY3Rpb24gdSh0KXtmb3IodmFyIGE9MCx1PXIubGVuZ3RoO2E8
dTsrK2Epe3ZhciBjPXJbYV0sZj1jLngtbnx8MWUtNixzPWMueS1lfHwxZS02LGw9TWF0aC5zcXJ0
KGYqZitzKnMpLGg9KG9bYV0tbCkqaVthXSp0L2w7Yy52eCs9ZipoLGMudnkrPXMqaH19ZnVuY3Rp
b24gYygpe2lmKHIpe3ZhciBuLGU9ci5sZW5ndGg7Zm9yKGk9bmV3IEFycmF5KGUpLG89bmV3IEFy
cmF5KGUpLG49MDtuPGU7KytuKW9bbl09K3QocltuXSxuLHIpLGlbbl09aXNOYU4ob1tuXSk/MDor
YShyW25dLG4scil9fXJldHVybiJmdW5jdGlvbiIhPXR5cGVvZiB0JiYodD1VYygrdCkpLG51bGw9
PW4mJihuPTApLG51bGw9PWUmJihlPTApLHUuaW5pdGlhbGl6ZT1mdW5jdGlvbih0KXtyPXQsYygp
fSx1LnN0cmVuZ3RoPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhhPSJmdW5j
dGlvbiI9PXR5cGVvZiB0P3Q6VWMoK3QpLGMoKSx1KTphfSx1LnJhZGl1cz1mdW5jdGlvbihuKXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOlVjKCtuKSxj
KCksdSk6dH0sdS54PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhuPSt0LHUp
Om59LHUueT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oZT0rdCx1KTplfSx1
fSx0LmZvcmNlU2ltdWxhdGlvbj1mdW5jdGlvbih0KXt2YXIgbixlPTEscj0uMDAxLGk9MS1NYXRo
LnBvdyhyLDEvMzAwKSxvPTAsYT0uNix1PW5ldyBNYXAsYz1raShsKSxmPUR0KCJ0aWNrIiwiZW5k
Iikscz1mdW5jdGlvbigpe2xldCB0PTE7cmV0dXJuKCk9Pih0PShqYyp0K0hjKSVYYykvWGN9KCk7
ZnVuY3Rpb24gbCgpe2goKSxmLmNhbGwoInRpY2siLG4pLGU8ciYmKGMuc3RvcCgpLGYuY2FsbCgi
ZW5kIixuKSl9ZnVuY3Rpb24gaChyKXt2YXIgYyxmLHM9dC5sZW5ndGg7dm9pZCAwPT09ciYmKHI9
MSk7Zm9yKHZhciBsPTA7bDxyOysrbClmb3IoZSs9KG8tZSkqaSx1LmZvckVhY2goKGZ1bmN0aW9u
KHQpe3QoZSl9KSksYz0wO2M8czsrK2MpbnVsbD09KGY9dFtjXSkuZng/Zi54Kz1mLnZ4Kj1hOihm
Lng9Zi5meCxmLnZ4PTApLG51bGw9PWYuZnk/Zi55Kz1mLnZ5Kj1hOihmLnk9Zi5meSxmLnZ5PTAp
O3JldHVybiBufWZ1bmN0aW9uIGQoKXtmb3IodmFyIG4sZT0wLHI9dC5sZW5ndGg7ZTxyOysrZSl7
aWYoKG49dFtlXSkuaW5kZXg9ZSxudWxsIT1uLmZ4JiYobi54PW4uZngpLG51bGwhPW4uZnkmJihu
Lnk9bi5meSksaXNOYU4obi54KXx8aXNOYU4obi55KSl7dmFyIGk9MTAqTWF0aC5zcXJ0KC41K2Up
LG89ZSpXYztuLng9aSpNYXRoLmNvcyhvKSxuLnk9aSpNYXRoLnNpbihvKX0oaXNOYU4obi52eCl8
fGlzTmFOKG4udnkpKSYmKG4udng9bi52eT0wKX19ZnVuY3Rpb24gcChuKXtyZXR1cm4gbi5pbml0
aWFsaXplJiZuLmluaXRpYWxpemUodCxzKSxufXJldHVybiBudWxsPT10JiYodD1bXSksZCgpLG49
e3RpY2s6aCxyZXN0YXJ0OmZ1bmN0aW9uKCl7cmV0dXJuIGMucmVzdGFydChsKSxufSxzdG9wOmZ1
bmN0aW9uKCl7cmV0dXJuIGMuc3RvcCgpLG59LG5vZGVzOmZ1bmN0aW9uKGUpe3JldHVybiBhcmd1
bWVudHMubGVuZ3RoPyh0PWUsZCgpLHUuZm9yRWFjaChwKSxuKTp0fSxhbHBoYTpmdW5jdGlvbih0
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oZT0rdCxuKTplfSxhbHBoYU1pbjpmdW5jdGlvbih0
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj0rdCxuKTpyfSxhbHBoYURlY2F5OmZ1bmN0aW9u
KHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhpPSt0LG4pOitpfSxhbHBoYVRhcmdldDpmdW5j
dGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obz0rdCxuKTpvfSx2ZWxvY2l0eURlY2F5
OmZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhhPTEtdCxuKToxLWF9LHJhbmRv
bVNvdXJjZTpmdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocz10LHUuZm9yRWFj
aChwKSxuKTpzfSxmb3JjZTpmdW5jdGlvbih0LGUpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPjE/
KG51bGw9PWU/dS5kZWxldGUodCk6dS5zZXQodCxwKGUpKSxuKTp1LmdldCh0KX0sZmluZDpmdW5j
dGlvbihuLGUscil7dmFyIGksbyxhLHUsYyxmPTAscz10Lmxlbmd0aDtmb3IobnVsbD09cj9yPTEv
MDpyKj1yLGY9MDtmPHM7KytmKShhPShpPW4tKHU9dFtmXSkueCkqaSsobz1lLXUueSkqbyk8ciYm
KGM9dSxyPWEpO3JldHVybiBjfSxvbjpmdW5jdGlvbih0LGUpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPjE/KGYub24odCxlKSxuKTpmLm9uKHQpfX19LHQuZm9yY2VYPWZ1bmN0aW9uKHQpe3ZhciBu
LGUscixpPVVjKC4xKTtmdW5jdGlvbiBvKHQpe2Zvcih2YXIgaSxvPTAsYT1uLmxlbmd0aDtvPGE7
KytvKShpPW5bb10pLnZ4Kz0ocltvXS1pLngpKmVbb10qdH1mdW5jdGlvbiBhKCl7aWYobil7dmFy
IG8sYT1uLmxlbmd0aDtmb3IoZT1uZXcgQXJyYXkoYSkscj1uZXcgQXJyYXkoYSksbz0wO288YTsr
K28pZVtvXT1pc05hTihyW29dPSt0KG5bb10sbyxuKSk/MDoraShuW29dLG8sbil9fXJldHVybiJm
dW5jdGlvbiIhPXR5cGVvZiB0JiYodD1VYyhudWxsPT10PzA6K3QpKSxvLmluaXRpYWxpemU9ZnVu
Y3Rpb24odCl7bj10LGEoKX0sby5zdHJlbmd0aD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD8oaT0iZnVuY3Rpb24iPT10eXBlb2YgdD90OlVjKCt0KSxhKCksbyk6aX0sby54PWZ1
bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh0PSJmdW5jdGlvbiI9PXR5cGVvZiBu
P246VWMoK24pLGEoKSxvKTp0fSxvfSx0LmZvcmNlWT1mdW5jdGlvbih0KXt2YXIgbixlLHIsaT1V
YyguMSk7ZnVuY3Rpb24gbyh0KXtmb3IodmFyIGksbz0wLGE9bi5sZW5ndGg7bzxhOysrbykoaT1u
W29dKS52eSs9KHJbb10taS55KSplW29dKnR9ZnVuY3Rpb24gYSgpe2lmKG4pe3ZhciBvLGE9bi5s
ZW5ndGg7Zm9yKGU9bmV3IEFycmF5KGEpLHI9bmV3IEFycmF5KGEpLG89MDtvPGE7KytvKWVbb109
aXNOYU4ocltvXT0rdChuW29dLG8sbikpPzA6K2kobltvXSxvLG4pfX1yZXR1cm4iZnVuY3Rpb24i
IT10eXBlb2YgdCYmKHQ9VWMobnVsbD09dD8wOit0KSksby5pbml0aWFsaXplPWZ1bmN0aW9uKHQp
e249dCxhKCl9LG8uc3RyZW5ndGg9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KGk9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpVYygrdCksYSgpLG8pOml9LG8ueT1mdW5jdGlvbihu
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOlVjKCtu
KSxhKCksbyk6dH0sb30sdC5mb3JtYXREZWZhdWx0TG9jYWxlPXNmLHQuZm9ybWF0TG9jYWxlPWZm
LHQuZm9ybWF0U3BlY2lmaWVyPXRmLHQuZnN1bT1mdW5jdGlvbih0LG4pe2NvbnN0IGU9bmV3IEE7
aWYodm9pZCAwPT09bilmb3IobGV0IG4gb2YgdCkobj0rbikmJmUuYWRkKG4pO2Vsc2V7bGV0IHI9
LTE7Zm9yKGxldCBpIG9mIHQpKGk9K24oaSwrK3IsdCkpJiZlLmFkZChpKX1yZXR1cm4rZX0sdC5n
ZW9BbGJlcnM9d2QsdC5nZW9BbGJlcnNVc2E9ZnVuY3Rpb24oKXt2YXIgdCxuLGUscixpLG8sYT13
ZCgpLHU9eGQoKS5yb3RhdGUoWzE1NCwwXSkuY2VudGVyKFstMiw1OC41XSkucGFyYWxsZWxzKFs1
NSw2NV0pLGM9eGQoKS5yb3RhdGUoWzE1NywwXSkuY2VudGVyKFstMywxOS45XSkucGFyYWxsZWxz
KFs4LDE4XSksZj17cG9pbnQ6ZnVuY3Rpb24odCxuKXtvPVt0LG5dfX07ZnVuY3Rpb24gcyh0KXt2
YXIgbj10WzBdLGE9dFsxXTtyZXR1cm4gbz1udWxsLGUucG9pbnQobixhKSxvfHwoci5wb2ludChu
LGEpLG8pfHwoaS5wb2ludChuLGEpLG8pfWZ1bmN0aW9uIGwoKXtyZXR1cm4gdD1uPW51bGwsc31y
ZXR1cm4gcy5pbnZlcnQ9ZnVuY3Rpb24odCl7dmFyIG49YS5zY2FsZSgpLGU9YS50cmFuc2xhdGUo
KSxyPSh0WzBdLWVbMF0pL24saT0odFsxXS1lWzFdKS9uO3JldHVybihpPj0uMTImJmk8LjIzNCYm
cj49LS40MjUmJnI8LS4yMTQ/dTppPj0uMTY2JiZpPC4yMzQmJnI+PS0uMjE0JiZyPC0uMTE1P2M6
YSkuaW52ZXJ0KHQpfSxzLnN0cmVhbT1mdW5jdGlvbihlKXtyZXR1cm4gdCYmbj09PWU/dDoocj1b
YS5zdHJlYW0obj1lKSx1LnN0cmVhbShlKSxjLnN0cmVhbShlKV0saT1yLmxlbmd0aCx0PXtwb2lu
dDpmdW5jdGlvbih0LG4pe2Zvcih2YXIgZT0tMTsrK2U8aTspcltlXS5wb2ludCh0LG4pfSxzcGhl
cmU6ZnVuY3Rpb24oKXtmb3IodmFyIHQ9LTE7Kyt0PGk7KXJbdF0uc3BoZXJlKCl9LGxpbmVTdGFy
dDpmdW5jdGlvbigpe2Zvcih2YXIgdD0tMTsrK3Q8aTspclt0XS5saW5lU3RhcnQoKX0sbGluZUVu
ZDpmdW5jdGlvbigpe2Zvcih2YXIgdD0tMTsrK3Q8aTspclt0XS5saW5lRW5kKCl9LHBvbHlnb25T
dGFydDpmdW5jdGlvbigpe2Zvcih2YXIgdD0tMTsrK3Q8aTspclt0XS5wb2x5Z29uU3RhcnQoKX0s
cG9seWdvbkVuZDpmdW5jdGlvbigpe2Zvcih2YXIgdD0tMTsrK3Q8aTspclt0XS5wb2x5Z29uRW5k
KCl9fSk7dmFyIHIsaX0scy5wcmVjaXNpb249ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/KGEucHJlY2lzaW9uKHQpLHUucHJlY2lzaW9uKHQpLGMucHJlY2lzaW9uKHQpLGwoKSk6
YS5wcmVjaXNpb24oKX0scy5zY2FsZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0
aD8oYS5zY2FsZSh0KSx1LnNjYWxlKC4zNSp0KSxjLnNjYWxlKHQpLHMudHJhbnNsYXRlKGEudHJh
bnNsYXRlKCkpKTphLnNjYWxlKCl9LHMudHJhbnNsYXRlPWZ1bmN0aW9uKHQpe2lmKCFhcmd1bWVu
dHMubGVuZ3RoKXJldHVybiBhLnRyYW5zbGF0ZSgpO3ZhciBuPWEuc2NhbGUoKSxvPSt0WzBdLHM9
K3RbMV07cmV0dXJuIGU9YS50cmFuc2xhdGUodCkuY2xpcEV4dGVudChbW28tLjQ1NSpuLHMtLjIz
OCpuXSxbbysuNDU1Km4scysuMjM4Km5dXSkuc3RyZWFtKGYpLHI9dS50cmFuc2xhdGUoW28tLjMw
NypuLHMrLjIwMSpuXSkuY2xpcEV4dGVudChbW28tLjQyNSpuK3BmLHMrLjEyKm4rcGZdLFtvLS4y
MTQqbi1wZixzKy4yMzQqbi1wZl1dKS5zdHJlYW0oZiksaT1jLnRyYW5zbGF0ZShbby0uMjA1Km4s
cysuMjEyKm5dKS5jbGlwRXh0ZW50KFtbby0uMjE0Km4rcGYscysuMTY2Km4rcGZdLFtvLS4xMTUq
bi1wZixzKy4yMzQqbi1wZl1dKS5zdHJlYW0oZiksbCgpfSxzLmZpdEV4dGVudD1mdW5jdGlvbih0
LG4pe3JldHVybiBjZChzLHQsbil9LHMuZml0U2l6ZT1mdW5jdGlvbih0LG4pe3JldHVybiBmZChz
LHQsbil9LHMuZml0V2lkdGg9ZnVuY3Rpb24odCxuKXtyZXR1cm4gc2Qocyx0LG4pfSxzLmZpdEhl
aWdodD1mdW5jdGlvbih0LG4pe3JldHVybiBsZChzLHQsbil9LHMuc2NhbGUoMTA3MCl9LHQuZ2Vv
QXJlYT1mdW5jdGlvbih0KXtyZXR1cm4gY3M9bmV3IEEsamYodCxmcyksMipjc30sdC5nZW9Bemlt
dXRoYWxFcXVhbEFyZWE9ZnVuY3Rpb24oKXtyZXR1cm4gdmQoQWQpLnNjYWxlKDEyNC43NSkuY2xp
cEFuZ2xlKDE3OS45OTkpfSx0Lmdlb0F6aW11dGhhbEVxdWFsQXJlYVJhdz1BZCx0Lmdlb0F6aW11
dGhhbEVxdWlkaXN0YW50PWZ1bmN0aW9uKCl7cmV0dXJuIHZkKFNkKS5zY2FsZSg3OS40MTg4KS5j
bGlwQW5nbGUoMTc5Ljk5OSl9LHQuZ2VvQXppbXV0aGFsRXF1aWRpc3RhbnRSYXc9U2QsdC5nZW9C
b3VuZHM9ZnVuY3Rpb24odCl7dmFyIG4sZSxyLGksbyxhLHU7aWYoSmY9UWY9LShaZj1LZj0xLzAp
LG9zPVtdLGpmKHQscXMpLGU9b3MubGVuZ3RoKXtmb3Iob3Muc29ydChYcyksbj0xLG89W3I9b3Nb
MF1dO248ZTsrK24pR3MociwoaT1vc1tuXSlbMF0pfHxHcyhyLGlbMV0pPyhIcyhyWzBdLGlbMV0p
PkhzKHJbMF0sclsxXSkmJihyWzFdPWlbMV0pLEhzKGlbMF0sclsxXSk+SHMoclswXSxyWzFdKSYm
KHJbMF09aVswXSkpOm8ucHVzaChyPWkpO2ZvcihhPS0xLzAsbj0wLHI9b1tlPW8ubGVuZ3RoLTFd
O248PWU7cj1pLCsrbilpPW9bbl0sKHU9SHMoclsxXSxpWzBdKSk+YSYmKGE9dSxaZj1pWzBdLFFm
PXJbMV0pfXJldHVybiBvcz1hcz1udWxsLFpmPT09MS8wfHxLZj09PTEvMD9bW05hTixOYU5dLFtO
YU4sTmFOXV06W1taZixLZl0sW1FmLEpmXV19LHQuZ2VvQ2VudHJvaWQ9ZnVuY3Rpb24odCl7eHM9
d3M9TXM9VHM9QXM9U3M9RXM9TnM9MCxrcz1uZXcgQSxDcz1uZXcgQSxQcz1uZXcgQSxqZih0LFZz
KTt2YXIgbj0ra3MsZT0rQ3Mscj0rUHMsaT1OZihuLGUscik7cmV0dXJuIGk8Z2YmJihuPVNzLGU9
RXMscj1Ocyx3czxwZiYmKG49TXMsZT1UcyxyPUFzKSwoaT1OZihuLGUscikpPGdmKT9bTmFOLE5h
Tl06W1RmKGUsbikqbWYsRmYoci9pKSptZl19LHQuZ2VvQ2lyY2xlPWZ1bmN0aW9uKCl7dmFyIHQs
bixlPW9sKFswLDBdKSxyPW9sKDkwKSxpPW9sKDYpLG89e3BvaW50OmZ1bmN0aW9uKGUscil7dC5w
dXNoKGU9bihlLHIpKSxlWzBdKj1tZixlWzFdKj1tZn19O2Z1bmN0aW9uIGEoKXt2YXIgYT1lLmFw
cGx5KHRoaXMsYXJndW1lbnRzKSx1PXIuYXBwbHkodGhpcyxhcmd1bWVudHMpKnhmLGM9aS5hcHBs
eSh0aGlzLGFyZ3VtZW50cykqeGY7cmV0dXJuIHQ9W10sbj1jbCgtYVswXSp4ZiwtYVsxXSp4Ziww
KS5pbnZlcnQsZGwobyx1LGMsMSksYT17dHlwZToiUG9seWdvbiIsY29vcmRpbmF0ZXM6W3RdfSx0
PW49bnVsbCxhfXJldHVybiBhLmNlbnRlcj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxl
bmd0aD8oZT0iZnVuY3Rpb24iPT10eXBlb2YgdD90Om9sKFsrdFswXSwrdFsxXV0pLGEpOmV9LGEu
cmFkaXVzPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhyPSJmdW5jdGlvbiI9
PXR5cGVvZiB0P3Q6b2woK3QpLGEpOnJ9LGEucHJlY2lzaW9uPWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyhpPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6b2woK3QpLGEpOml9LGF9
LHQuZ2VvQ2xpcEFudGltZXJpZGlhbj1BbCx0Lmdlb0NsaXBDaXJjbGU9U2wsdC5nZW9DbGlwRXh0
ZW50PWZ1bmN0aW9uKCl7dmFyIHQsbixlLHI9MCxpPTAsbz05NjAsYT01MDA7cmV0dXJuIGU9e3N0
cmVhbTpmdW5jdGlvbihlKXtyZXR1cm4gdCYmbj09PWU/dDp0PSRsKHIsaSxvLGEpKG49ZSl9LGV4
dGVudDpmdW5jdGlvbih1KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj0rdVswXVswXSxpPSt1
WzBdWzFdLG89K3VbMV1bMF0sYT0rdVsxXVsxXSx0PW49bnVsbCxlKTpbW3IsaV0sW28sYV1dfX19
LHQuZ2VvQ2xpcFJlY3RhbmdsZT0kbCx0Lmdlb0NvbmljQ29uZm9ybWFsPWZ1bmN0aW9uKCl7cmV0
dXJuIGJkKENkKS5zY2FsZSgxMDkuNSkucGFyYWxsZWxzKFszMCwzMF0pfSx0Lmdlb0NvbmljQ29u
Zm9ybWFsUmF3PUNkLHQuZ2VvQ29uaWNFcXVhbEFyZWE9eGQsdC5nZW9Db25pY0VxdWFsQXJlYVJh
dz1tZCx0Lmdlb0NvbmljRXF1aWRpc3RhbnQ9ZnVuY3Rpb24oKXtyZXR1cm4gYmQoemQpLnNjYWxl
KDEzMS4xNTQpLmNlbnRlcihbMCwxMy45Mzg5XSl9LHQuZ2VvQ29uaWNFcXVpZGlzdGFudFJhdz16
ZCx0Lmdlb0NvbnRhaW5zPWZ1bmN0aW9uKHQsbil7cmV0dXJuKHQmJllsLmhhc093blByb3BlcnR5
KHQudHlwZSk/WWxbdC50eXBlXTpqbCkodCxuKX0sdC5nZW9EaXN0YW5jZT1CbCx0Lmdlb0VxdWFs
RWFydGg9ZnVuY3Rpb24oKXtyZXR1cm4gdmQoVWQpLnNjYWxlKDE3Ny4xNTgpfSx0Lmdlb0VxdWFs
RWFydGhSYXc9VWQsdC5nZW9FcXVpcmVjdGFuZ3VsYXI9ZnVuY3Rpb24oKXtyZXR1cm4gdmQoUGQp
LnNjYWxlKDE1Mi42Myl9LHQuZ2VvRXF1aXJlY3Rhbmd1bGFyUmF3PVBkLHQuZ2VvR25vbW9uaWM9
ZnVuY3Rpb24oKXtyZXR1cm4gdmQoSWQpLnNjYWxlKDE0NC4wNDkpLmNsaXBBbmdsZSg2MCl9LHQu
Z2VvR25vbW9uaWNSYXc9SWQsdC5nZW9HcmF0aWN1bGU9UWwsdC5nZW9HcmF0aWN1bGUxMD1mdW5j
dGlvbigpe3JldHVybiBRbCgpKCl9LHQuZ2VvSWRlbnRpdHk9ZnVuY3Rpb24oKXt2YXIgdCxuLGUs
cixpLG8sYSx1PTEsYz0wLGY9MCxzPTEsbD0xLGg9MCxkPW51bGwscD0xLGc9MSx5PW9kKHtwb2lu
dDpmdW5jdGlvbih0LG4pe3ZhciBlPWIoW3Qsbl0pO3RoaXMuc3RyZWFtLnBvaW50KGVbMF0sZVsx
XSl9fSksdj1yaDtmdW5jdGlvbiBfKCl7cmV0dXJuIHA9dSpzLGc9dSpsLG89YT1udWxsLGJ9ZnVu
Y3Rpb24gYihlKXt2YXIgcj1lWzBdKnAsaT1lWzFdKmc7aWYoaCl7dmFyIG89aSp0LXIqbjtyPXIq
dCtpKm4saT1vfXJldHVybltyK2MsaStmXX1yZXR1cm4gYi5pbnZlcnQ9ZnVuY3Rpb24oZSl7dmFy
IHI9ZVswXS1jLGk9ZVsxXS1mO2lmKGgpe3ZhciBvPWkqdCtyKm47cj1yKnQtaSpuLGk9b31yZXR1
cm5bci9wLGkvZ119LGIuc3RyZWFtPWZ1bmN0aW9uKHQpe3JldHVybiBvJiZhPT09dD9vOm89eSh2
KGE9dCkpfSxiLnBvc3RjbGlwPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh2
PXQsZD1lPXI9aT1udWxsLF8oKSk6dn0sYi5jbGlwRXh0ZW50PWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyh2PW51bGw9PXQ/KGQ9ZT1yPWk9bnVsbCxyaCk6JGwoZD0rdFswXVsw
XSxlPSt0WzBdWzFdLHI9K3RbMV1bMF0saT0rdFsxXVsxXSksXygpKTpudWxsPT1kP251bGw6W1tk
LGVdLFtyLGldXX0sYi5zY2FsZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8o
dT0rdCxfKCkpOnV9LGIudHJhbnNsYXRlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPyhjPSt0WzBdLGY9K3RbMV0sXygpKTpbYyxmXX0sYi5hbmdsZT1mdW5jdGlvbihlKXtyZXR1
cm4gYXJndW1lbnRzLmxlbmd0aD8obj1QZihoPWUlMzYwKnhmKSx0PUFmKGgpLF8oKSk6aCptZn0s
Yi5yZWZsZWN0WD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocz10Py0xOjEs
XygpKTpzPDB9LGIucmVmbGVjdFk9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KGw9dD8tMToxLF8oKSk6bDwwfSxiLmZpdEV4dGVudD1mdW5jdGlvbih0LG4pe3JldHVybiBjZChi
LHQsbil9LGIuZml0U2l6ZT1mdW5jdGlvbih0LG4pe3JldHVybiBmZChiLHQsbil9LGIuZml0V2lk
dGg9ZnVuY3Rpb24odCxuKXtyZXR1cm4gc2QoYix0LG4pfSxiLmZpdEhlaWdodD1mdW5jdGlvbih0
LG4pe3JldHVybiBsZChiLHQsbil9LGJ9LHQuZ2VvSW50ZXJwb2xhdGU9ZnVuY3Rpb24odCxuKXt2
YXIgZT10WzBdKnhmLHI9dFsxXSp4ZixpPW5bMF0qeGYsbz1uWzFdKnhmLGE9QWYociksdT1QZihy
KSxjPUFmKG8pLGY9UGYobykscz1hKkFmKGUpLGw9YSpQZihlKSxoPWMqQWYoaSksZD1jKlBmKGkp
LHA9MipGZigkZihxZihvLXIpK2EqYypxZihpLWUpKSksZz1QZihwKSx5PXA/ZnVuY3Rpb24odCl7
dmFyIG49UGYodCo9cCkvZyxlPVBmKHAtdCkvZyxyPWUqcytuKmgsaT1lKmwrbipkLG89ZSp1K24q
ZjtyZXR1cm5bVGYoaSxyKSptZixUZihvLCRmKHIqcitpKmkpKSptZl19OmZ1bmN0aW9uKCl7cmV0
dXJuW2UqbWYsciptZl19O3JldHVybiB5LmRpc3RhbmNlPXAseX0sdC5nZW9MZW5ndGg9VWwsdC5n
ZW9NZXJjYXRvcj1mdW5jdGlvbigpe3JldHVybiBOZChFZCkuc2NhbGUoOTYxL2JmKX0sdC5nZW9N
ZXJjYXRvclJhdz1FZCx0Lmdlb05hdHVyYWxFYXJ0aDE9ZnVuY3Rpb24oKXtyZXR1cm4gdmQoT2Qp
LnNjYWxlKDE3NS4yOTUpfSx0Lmdlb05hdHVyYWxFYXJ0aDFSYXc9T2QsdC5nZW9PcnRob2dyYXBo
aWM9ZnVuY3Rpb24oKXtyZXR1cm4gdmQoQmQpLnNjYWxlKDI0OS41KS5jbGlwQW5nbGUoOTArcGYp
fSx0Lmdlb09ydGhvZ3JhcGhpY1Jhdz1CZCx0Lmdlb1BhdGg9ZnVuY3Rpb24odCxuKXtsZXQgZSxy
LGk9MyxvPTQuNTtmdW5jdGlvbiBhKHQpe3JldHVybiB0JiYoImZ1bmN0aW9uIj09dHlwZW9mIG8m
JnIucG9pbnRSYWRpdXMoK28uYXBwbHkodGhpcyxhcmd1bWVudHMpKSxqZih0LGUocikpKSxyLnJl
c3VsdCgpfXJldHVybiBhLmFyZWE9ZnVuY3Rpb24odCl7cmV0dXJuIGpmKHQsZShsaCkpLGxoLnJl
c3VsdCgpfSxhLm1lYXN1cmU9ZnVuY3Rpb24odCl7cmV0dXJuIGpmKHQsZShRaCkpLFFoLnJlc3Vs
dCgpfSxhLmJvdW5kcz1mdW5jdGlvbih0KXtyZXR1cm4gamYodCxlKHhoKSkseGgucmVzdWx0KCl9
LGEuY2VudHJvaWQ9ZnVuY3Rpb24odCl7cmV0dXJuIGpmKHQsZShCaCkpLEJoLnJlc3VsdCgpfSxh
LnByb2plY3Rpb249ZnVuY3Rpb24obil7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGU9bnVsbD09
bj8odD1udWxsLHJoKToodD1uKS5zdHJlYW0sYSk6dH0sYS5jb250ZXh0PWZ1bmN0aW9uKHQpe3Jl
dHVybiBhcmd1bWVudHMubGVuZ3RoPyhyPW51bGw9PXQ/KG49bnVsbCxuZXcgcmQoaSkpOm5ldyBZ
aChuPXQpLCJmdW5jdGlvbiIhPXR5cGVvZiBvJiZyLnBvaW50UmFkaXVzKG8pLGEpOm59LGEucG9p
bnRSYWRpdXM9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG89ImZ1bmN0aW9u
Ij09dHlwZW9mIHQ/dDooci5wb2ludFJhZGl1cygrdCksK3QpLGEpOm99LGEuZGlnaXRzPWZ1bmN0
aW9uKHQpe2lmKCFhcmd1bWVudHMubGVuZ3RoKXJldHVybiBpO2lmKG51bGw9PXQpaT1udWxsO2Vs
c2V7Y29uc3Qgbj1NYXRoLmZsb29yKHQpO2lmKCEobj49MCkpdGhyb3cgbmV3IFJhbmdlRXJyb3Io
YGludmFsaWQgZGlnaXRzOiAke3R9YCk7aT1ufXJldHVybiBudWxsPT09biYmKHI9bmV3IHJkKGkp
KSxhfSxhLnByb2plY3Rpb24odCkuZGlnaXRzKGkpLmNvbnRleHQobil9LHQuZ2VvUHJvamVjdGlv
bj12ZCx0Lmdlb1Byb2plY3Rpb25NdXRhdG9yPV9kLHQuZ2VvUm90YXRpb249aGwsdC5nZW9TdGVy
ZW9ncmFwaGljPWZ1bmN0aW9uKCl7cmV0dXJuIHZkKFlkKS5zY2FsZSgyNTApLmNsaXBBbmdsZSgx
NDIpfSx0Lmdlb1N0ZXJlb2dyYXBoaWNSYXc9WWQsdC5nZW9TdHJlYW09amYsdC5nZW9UcmFuc2Zv
cm09ZnVuY3Rpb24odCl7cmV0dXJue3N0cmVhbTpvZCh0KX19LHQuZ2VvVHJhbnN2ZXJzZU1lcmNh
dG9yPWZ1bmN0aW9uKCl7dmFyIHQ9TmQoTGQpLG49dC5jZW50ZXIsZT10LnJvdGF0ZTtyZXR1cm4g
dC5jZW50ZXI9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/bihbLXRbMV0sdFsw
XV0pOlsodD1uKCkpWzFdLC10WzBdXX0sdC5yb3RhdGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3Vt
ZW50cy5sZW5ndGg/ZShbdFswXSx0WzFdLHQubGVuZ3RoPjI/dFsyXSs5MDo5MF0pOlsodD1lKCkp
WzBdLHRbMV0sdFsyXS05MF19LGUoWzAsMCw5MF0pLnNjYWxlKDE1OS4xNTUpfSx0Lmdlb1RyYW5z
dmVyc2VNZXJjYXRvclJhdz1MZCx0LmdyYXk9ZnVuY3Rpb24odCxuKXtyZXR1cm4gbmV3IGNyKHQs
MCwwLG51bGw9PW4/MTpuKX0sdC5ncmVhdGVzdD1hdCx0LmdyZWF0ZXN0SW5kZXg9ZnVuY3Rpb24o
dCxlPW4pe2lmKDE9PT1lLmxlbmd0aClyZXR1cm4gbnQodCxlKTtsZXQgcixpPS0xLG89LTE7Zm9y
KGNvbnN0IG4gb2YgdCkrK28sKGk8MD8wPT09ZShuLG4pOmUobixyKT4wKSYmKHI9bixpPW8pO3Jl
dHVybiBpfSx0Lmdyb3VwPVAsdC5ncm91cFNvcnQ9ZnVuY3Rpb24odCxlLHIpe3JldHVybigyIT09
ZS5sZW5ndGg/SShEKHQsZSxyKSwoKFt0LGVdLFtyLGldKT0+bihlLGkpfHxuKHQscikpKTpJKFAo
dCxyKSwoKFt0LHJdLFtpLG9dKT0+ZShyLG8pfHxuKHQsaSkpKSkubWFwKCgoW3RdKT0+dCkpfSx0
Lmdyb3Vwcz16LHQuaGNsPXByLHQuaGllcmFyY2h5PVZkLHQuaGlzdG9ncmFtPUosdC5oc2w9WGUs
dC5odG1sPU5jLHQuaW1hZ2U9ZnVuY3Rpb24odCxuKXtyZXR1cm4gbmV3IFByb21pc2UoKGZ1bmN0
aW9uKGUscil7dmFyIGk9bmV3IEltYWdlO2Zvcih2YXIgbyBpbiBuKWlbb109bltvXTtpLm9uZXJy
b3I9cixpLm9ubG9hZD1mdW5jdGlvbigpe2UoaSl9LGkuc3JjPXR9KSl9LHQuaW5kZXg9ZnVuY3Rp
b24odCwuLi5uKXtyZXR1cm4gcSh0LEMsRixuKX0sdC5pbmRleGVzPWZ1bmN0aW9uKHQsLi4ubil7
cmV0dXJuIHEodCxBcnJheS5mcm9tLEYsbil9LHQuaW50ZXJwb2xhdGU9VnIsdC5pbnRlcnBvbGF0
ZUFycmF5PWZ1bmN0aW9uKHQsbil7cmV0dXJuKE9yKG4pP0lyOkJyKSh0LG4pfSx0LmludGVycG9s
YXRlQmFzaXM9TnIsdC5pbnRlcnBvbGF0ZUJhc2lzQ2xvc2VkPWtyLHQuaW50ZXJwb2xhdGVCbHVl
cz1HYix0LmludGVycG9sYXRlQnJCRz1vYix0LmludGVycG9sYXRlQnVHbj1NYix0LmludGVycG9s
YXRlQnVQdT1BYix0LmludGVycG9sYXRlQ2l2aWRpcz1mdW5jdGlvbih0KXtyZXR1cm4gdD1NYXRo
Lm1heCgwLE1hdGgubWluKDEsdCkpLCJyZ2IoIitNYXRoLm1heCgwLE1hdGgubWluKDI1NSxNYXRo
LnJvdW5kKC00LjU0LXQqKDM1LjM0LXQqKDIzODEuNzMtdCooNjQwMi43LXQqKDcwMjQuNzItMjcx
MC41Nyp0KSkpKSkpKSsiLCAiK01hdGgubWF4KDAsTWF0aC5taW4oMjU1LE1hdGgucm91bmQoMzIu
NDkrdCooMTcwLjczK3QqKDUyLjgyLXQqKDEzMS40Ni10KigxNzYuNTgtNjcuMzcqdCkpKSkpKSkr
IiwgIitNYXRoLm1heCgwLE1hdGgubWluKDI1NSxNYXRoLnJvdW5kKDgxLjI0K3QqKDQ0Mi4zNi10
KigyNDgyLjQzLXQqKDYxNjcuMjQtdCooNjYxNC45NC0yNDc1LjY3KnQpKSkpKSkpKyIpIn0sdC5p
bnRlcnBvbGF0ZUNvb2w9YW0sdC5pbnRlcnBvbGF0ZUN1YmVoZWxpeD1oaSx0LmludGVycG9sYXRl
Q3ViZWhlbGl4RGVmYXVsdD1pbSx0LmludGVycG9sYXRlQ3ViZWhlbGl4TG9uZz1kaSx0LmludGVy
cG9sYXRlRGF0ZT1Zcix0LmludGVycG9sYXRlRGlzY3JldGU9ZnVuY3Rpb24odCl7dmFyIG49dC5s
ZW5ndGg7cmV0dXJuIGZ1bmN0aW9uKGUpe3JldHVybiB0W01hdGgubWF4KDAsTWF0aC5taW4obi0x
LE1hdGguZmxvb3IoZSpuKSkpXX19LHQuaW50ZXJwb2xhdGVHbkJ1PUViLHQuaW50ZXJwb2xhdGVH
cmVlbnM9V2IsdC5pbnRlcnBvbGF0ZUdyZXlzPUtiLHQuaW50ZXJwb2xhdGVIY2w9ZmksdC5pbnRl
cnBvbGF0ZUhjbExvbmc9c2ksdC5pbnRlcnBvbGF0ZUhzbD1haSx0LmludGVycG9sYXRlSHNsTG9u
Zz11aSx0LmludGVycG9sYXRlSHVlPWZ1bmN0aW9uKHQsbil7dmFyIGU9enIoK3QsK24pO3JldHVy
biBmdW5jdGlvbih0KXt2YXIgbj1lKHQpO3JldHVybiBuLTM2MCpNYXRoLmZsb29yKG4vMzYwKX19
LHQuaW50ZXJwb2xhdGVJbmZlcm5vPXBtLHQuaW50ZXJwb2xhdGVMYWI9ZnVuY3Rpb24odCxuKXt2
YXIgZT1EcigodD11cih0KSkubCwobj11cihuKSkubCkscj1Ecih0LmEsbi5hKSxpPURyKHQuYixu
LmIpLG89RHIodC5vcGFjaXR5LG4ub3BhY2l0eSk7cmV0dXJuIGZ1bmN0aW9uKG4pe3JldHVybiB0
Lmw9ZShuKSx0LmE9cihuKSx0LmI9aShuKSx0Lm9wYWNpdHk9byhuKSx0KyIifX0sdC5pbnRlcnBv
bGF0ZU1hZ21hPWRtLHQuaW50ZXJwb2xhdGVOdW1iZXI9THIsdC5pbnRlcnBvbGF0ZU51bWJlckFy
cmF5PUlyLHQuaW50ZXJwb2xhdGVPYmplY3Q9anIsdC5pbnRlcnBvbGF0ZU9yUmQ9a2IsdC5pbnRl
cnBvbGF0ZU9yYW5nZXM9cm0sdC5pbnRlcnBvbGF0ZVBSR249dWIsdC5pbnRlcnBvbGF0ZVBpWUc9
ZmIsdC5pbnRlcnBvbGF0ZVBsYXNtYT1nbSx0LmludGVycG9sYXRlUHVCdT0kYix0LmludGVycG9s
YXRlUHVCdUduPVBiLHQuaW50ZXJwb2xhdGVQdU9yPWxiLHQuaW50ZXJwb2xhdGVQdVJkPVJiLHQu
aW50ZXJwb2xhdGVQdXJwbGVzPUpiLHQuaW50ZXJwb2xhdGVSYWluYm93PWZ1bmN0aW9uKHQpeyh0
PDB8fHQ+MSkmJih0LT1NYXRoLmZsb29yKHQpKTt2YXIgbj1NYXRoLmFicyh0LS41KTtyZXR1cm4g
dW0uaD0zNjAqdC0xMDAsdW0ucz0xLjUtMS41Km4sdW0ubD0uOC0uOSpuLHVtKyIifSx0LmludGVy
cG9sYXRlUmRCdT1kYix0LmludGVycG9sYXRlUmRHeT1nYix0LmludGVycG9sYXRlUmRQdT1xYix0
LmludGVycG9sYXRlUmRZbEJ1PXZiLHQuaW50ZXJwb2xhdGVSZFlsR249YmIsdC5pbnRlcnBvbGF0
ZVJlZHM9bm0sdC5pbnRlcnBvbGF0ZVJnYj1Scix0LmludGVycG9sYXRlUmdiQmFzaXM9cXIsdC5p
bnRlcnBvbGF0ZVJnYkJhc2lzQ2xvc2VkPVVyLHQuaW50ZXJwb2xhdGVSb3VuZD1Xcix0LmludGVy
cG9sYXRlU2luZWJvdz1mdW5jdGlvbih0KXt2YXIgbjtyZXR1cm4gdD0oLjUtdCkqTWF0aC5QSSxj
bS5yPTI1NSoobj1NYXRoLnNpbih0KSkqbixjbS5nPTI1NSoobj1NYXRoLnNpbih0K2ZtKSkqbixj
bS5iPTI1NSoobj1NYXRoLnNpbih0K3NtKSkqbixjbSsiIn0sdC5pbnRlcnBvbGF0ZVNwZWN0cmFs
PXhiLHQuaW50ZXJwb2xhdGVTdHJpbmc9R3IsdC5pbnRlcnBvbGF0ZVRyYW5zZm9ybUNzcz1uaSx0
LmludGVycG9sYXRlVHJhbnNmb3JtU3ZnPWVpLHQuaW50ZXJwb2xhdGVUdXJibz1mdW5jdGlvbih0
KXtyZXR1cm4gdD1NYXRoLm1heCgwLE1hdGgubWluKDEsdCkpLCJyZ2IoIitNYXRoLm1heCgwLE1h
dGgubWluKDI1NSxNYXRoLnJvdW5kKDM0LjYxK3QqKDExNzIuMzMtdCooMTA3OTMuNTYtdCooMzMz
MDAuMTItdCooMzgzOTQuNDktMTQ4MjUuMDUqdCkpKSkpKSkrIiwgIitNYXRoLm1heCgwLE1hdGgu
bWluKDI1NSxNYXRoLnJvdW5kKDIzLjMxK3QqKDU1Ny4zMyt0KigxMjI1LjMzLXQqKDM1NzQuOTYt
dCooMTA3My43Nys3MDcuNTYqdCkpKSkpKSkrIiwgIitNYXRoLm1heCgwLE1hdGgubWluKDI1NSxN
YXRoLnJvdW5kKDI3LjIrdCooMzIxMS4xLXQqKDE1MzI3Ljk3LXQqKDI3ODE0LXQqKDIyNTY5LjE4
LTY4MzguNjYqdCkpKSkpKSkrIikifSx0LmludGVycG9sYXRlVmlyaWRpcz1obSx0LmludGVycG9s
YXRlV2FybT1vbSx0LmludGVycG9sYXRlWWxHbj1CYix0LmludGVycG9sYXRlWWxHbkJ1PUliLHQu
aW50ZXJwb2xhdGVZbE9yQnI9TGIsdC5pbnRlcnBvbGF0ZVlsT3JSZD1IYix0LmludGVycG9sYXRl
Wm9vbT1paSx0LmludGVycnVwdD1WaSx0LmludGVyc2VjdGlvbj1mdW5jdGlvbih0LC4uLm4pe3Q9
bmV3IEludGVyblNldCh0KSxuPW4ubWFwKF90KTt0OmZvcihjb25zdCBlIG9mIHQpZm9yKGNvbnN0
IHIgb2YgbilpZighci5oYXMoZSkpe3QuZGVsZXRlKGUpO2NvbnRpbnVlIHR9cmV0dXJuIHR9LHQu
aW50ZXJ2YWw9ZnVuY3Rpb24odCxuLGUpe3ZhciByPW5ldyBOaSxpPW47cmV0dXJuIG51bGw9PW4/
KHIucmVzdGFydCh0LG4sZSkscik6KHIuX3Jlc3RhcnQ9ci5yZXN0YXJ0LHIucmVzdGFydD1mdW5j
dGlvbih0LG4sZSl7bj0rbixlPW51bGw9PWU/U2koKTorZSxyLl9yZXN0YXJ0KChmdW5jdGlvbiBv
KGEpe2ErPWksci5fcmVzdGFydChvLGkrPW4sZSksdChhKX0pLG4sZSl9LHIucmVzdGFydCh0LG4s
ZSkscil9LHQuaXNvRm9ybWF0PVJfLHQuaXNvUGFyc2U9cV8sdC5qc29uPWZ1bmN0aW9uKHQsbil7
cmV0dXJuIGZldGNoKHQsbikudGhlbihBYyl9LHQubGFiPXVyLHQubGNoPWZ1bmN0aW9uKHQsbixl
LHIpe3JldHVybiAxPT09YXJndW1lbnRzLmxlbmd0aD9kcih0KTpuZXcgZ3IoZSxuLHQsbnVsbD09
cj8xOnIpfSx0LmxlYXN0PWZ1bmN0aW9uKHQsZT1uKXtsZXQgcixpPSExO2lmKDE9PT1lLmxlbmd0
aCl7bGV0IG87Zm9yKGNvbnN0IGEgb2YgdCl7Y29uc3QgdD1lKGEpOyhpP24odCxvKTwwOjA9PT1u
KHQsdCkpJiYocj1hLG89dCxpPSEwKX19ZWxzZSBmb3IoY29uc3QgbiBvZiB0KShpP2UobixyKTww
OjA9PT1lKG4sbikpJiYocj1uLGk9ITApO3JldHVybiByfSx0LmxlYXN0SW5kZXg9ZHQsdC5saW5l
PVltLHQubGluZVJhZGlhbD1abSx0Lmxpbms9YXgsdC5saW5rSG9yaXpvbnRhbD1mdW5jdGlvbigp
e3JldHVybiBheChueCl9LHQubGlua1JhZGlhbD1mdW5jdGlvbigpe2NvbnN0IHQ9YXgocngpO3Jl
dHVybiB0LmFuZ2xlPXQueCxkZWxldGUgdC54LHQucmFkaXVzPXQueSxkZWxldGUgdC55LHR9LHQu
bGlua1ZlcnRpY2FsPWZ1bmN0aW9uKCl7cmV0dXJuIGF4KGV4KX0sdC5sb2NhbD1Kbix0Lm1hcD1m
dW5jdGlvbih0LG4pe2lmKCJmdW5jdGlvbiIhPXR5cGVvZiB0W1N5bWJvbC5pdGVyYXRvcl0pdGhy
b3cgbmV3IFR5cGVFcnJvcigidmFsdWVzIGlzIG5vdCBpdGVyYWJsZSIpO2lmKCJmdW5jdGlvbiIh
PXR5cGVvZiBuKXRocm93IG5ldyBUeXBlRXJyb3IoIm1hcHBlciBpcyBub3QgYSBmdW5jdGlvbiIp
O3JldHVybiBBcnJheS5mcm9tKHQsKChlLHIpPT5uKGUscix0KSkpfSx0Lm1hdGNoZXI9V3QsdC5t
YXg9dHQsdC5tYXhJbmRleD1udCx0Lm1lYW49ZnVuY3Rpb24odCxuKXtsZXQgZT0wLHI9MDtpZih2
b2lkIDA9PT1uKWZvcihsZXQgbiBvZiB0KW51bGwhPW4mJihuPStuKT49biYmKCsrZSxyKz1uKTtl
bHNle2xldCBpPS0xO2ZvcihsZXQgbyBvZiB0KW51bGwhPShvPW4obywrK2ksdCkpJiYobz0rbyk+
PW8mJigrK2Uscis9byl9aWYoZSlyZXR1cm4gci9lfSx0Lm1lZGlhbj1mdW5jdGlvbih0LG4pe3Jl
dHVybiB1dCh0LC41LG4pfSx0Lm1lZGlhbkluZGV4PWZ1bmN0aW9uKHQsbil7cmV0dXJuIGZ0KHQs
LjUsbil9LHQubWVyZ2U9c3QsdC5taW49ZXQsdC5taW5JbmRleD1ydCx0Lm1vZGU9ZnVuY3Rpb24o
dCxuKXtjb25zdCBlPW5ldyBJbnRlcm5NYXA7aWYodm9pZCAwPT09bilmb3IobGV0IG4gb2YgdClu
dWxsIT1uJiZuPj1uJiZlLnNldChuLChlLmdldChuKXx8MCkrMSk7ZWxzZXtsZXQgcj0tMTtmb3Io
bGV0IGkgb2YgdCludWxsIT0oaT1uKGksKytyLHQpKSYmaT49aSYmZS5zZXQoaSwoZS5nZXQoaSl8
fDApKzEpfWxldCByLGk9MDtmb3IoY29uc3RbdCxuXW9mIGUpbj5pJiYoaT1uLHI9dCk7cmV0dXJu
IHJ9LHQubmFtZXNwYWNlPU90LHQubmFtZXNwYWNlcz1JdCx0Lm5pY2U9Syx0Lm5vdz1TaSx0LnBh
Y2s9ZnVuY3Rpb24oKXt2YXIgdD1udWxsLG49MSxlPTEscj1lcDtmdW5jdGlvbiBpKGkpe2NvbnN0
IG89dXAoKTtyZXR1cm4gaS54PW4vMixpLnk9ZS8yLHQ/aS5lYWNoQmVmb3JlKHdwKHQpKS5lYWNo
QWZ0ZXIoTXAociwuNSxvKSkuZWFjaEJlZm9yZShUcCgxKSk6aS5lYWNoQmVmb3JlKHdwKHhwKSku
ZWFjaEFmdGVyKE1wKGVwLDEsbykpLmVhY2hBZnRlcihNcChyLGkuci9NYXRoLm1pbihuLGUpLG8p
KS5lYWNoQmVmb3JlKFRwKE1hdGgubWluKG4sZSkvKDIqaS5yKSkpLGl9cmV0dXJuIGkucmFkaXVz
PWZ1bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh0PXRwKG4pLGkpOnR9LGkuc2l6
ZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obj0rdFswXSxlPSt0WzFdLGkp
OltuLGVdfSxpLnBhZGRpbmc9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9
ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpycCgrdCksaSk6cn0saX0sdC5wYWNrRW5jbG9zZT1mdW5j
dGlvbih0KXtyZXR1cm4gY3AodCx1cCgpKX0sdC5wYWNrU2libGluZ3M9ZnVuY3Rpb24odCl7cmV0
dXJuIG1wKHQsdXAoKSksdH0sdC5wYWlycz1mdW5jdGlvbih0LG49bHQpe2NvbnN0IGU9W107bGV0
IHIsaT0hMTtmb3IoY29uc3QgbyBvZiB0KWkmJmUucHVzaChuKHIsbykpLHI9byxpPSEwO3JldHVy
biBlfSx0LnBhcnRpdGlvbj1mdW5jdGlvbigpe3ZhciB0PTEsbj0xLGU9MCxyPSExO2Z1bmN0aW9u
IGkoaSl7dmFyIG89aS5oZWlnaHQrMTtyZXR1cm4gaS54MD1pLnkwPWUsaS54MT10LGkueTE9bi9v
LGkuZWFjaEJlZm9yZShmdW5jdGlvbih0LG4pe3JldHVybiBmdW5jdGlvbihyKXtyLmNoaWxkcmVu
JiZTcChyLHIueDAsdCooci5kZXB0aCsxKS9uLHIueDEsdCooci5kZXB0aCsyKS9uKTt2YXIgaT1y
LngwLG89ci55MCxhPXIueDEtZSx1PXIueTEtZTthPGkmJihpPWE9KGkrYSkvMiksdTxvJiYobz11
PShvK3UpLzIpLHIueDA9aSxyLnkwPW8sci54MT1hLHIueTE9dX19KG4sbykpLHImJmkuZWFjaEJl
Zm9yZShBcCksaX1yZXR1cm4gaS5yb3VuZD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxl
bmd0aD8ocj0hIXQsaSk6cn0saS5zaXplPWZ1bmN0aW9uKGUpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPyh0PStlWzBdLG49K2VbMV0saSk6W3Qsbl19LGkucGFkZGluZz1mdW5jdGlvbih0KXtyZXR1
cm4gYXJndW1lbnRzLmxlbmd0aD8oZT0rdCxpKTplfSxpfSx0LnBhdGg9T2EsdC5wYXRoUm91bmQ9
ZnVuY3Rpb24odD0zKXtyZXR1cm4gbmV3IElhKCt0KX0sdC5wZXJtdXRlPVUsdC5waWU9ZnVuY3Rp
b24oKXt2YXIgdD1IbSxuPWptLGU9bnVsbCxyPXltKDApLGk9eW0oRW0pLG89eW0oMCk7ZnVuY3Rp
b24gYShhKXt2YXIgdSxjLGYscyxsLGg9KGE9cW0oYSkpLmxlbmd0aCxkPTAscD1uZXcgQXJyYXko
aCksZz1uZXcgQXJyYXkoaCkseT0rci5hcHBseSh0aGlzLGFyZ3VtZW50cyksdj1NYXRoLm1pbihF
bSxNYXRoLm1heCgtRW0saS5hcHBseSh0aGlzLGFyZ3VtZW50cykteSkpLF89TWF0aC5taW4oTWF0
aC5hYnModikvaCxvLmFwcGx5KHRoaXMsYXJndW1lbnRzKSksYj1fKih2PDA/LTE6MSk7Zm9yKHU9
MDt1PGg7Kyt1KShsPWdbcFt1XT11XT0rdChhW3VdLHUsYSkpPjAmJihkKz1sKTtmb3IobnVsbCE9
bj9wLnNvcnQoKGZ1bmN0aW9uKHQsZSl7cmV0dXJuIG4oZ1t0XSxnW2VdKX0pKTpudWxsIT1lJiZw
LnNvcnQoKGZ1bmN0aW9uKHQsbil7cmV0dXJuIGUoYVt0XSxhW25dKX0pKSx1PTAsZj1kPyh2LWgq
YikvZDowO3U8aDsrK3UseT1zKWM9cFt1XSxzPXkrKChsPWdbY10pPjA/bCpmOjApK2IsZ1tjXT17
ZGF0YTphW2NdLGluZGV4OnUsdmFsdWU6bCxzdGFydEFuZ2xlOnksZW5kQW5nbGU6cyxwYWRBbmds
ZTpffTtyZXR1cm4gZ31yZXR1cm4gYS52YWx1ZT1mdW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD8odD0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOnltKCtuKSxhKTp0fSxhLnNvcnRWYWx1
ZXM9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49dCxlPW51bGwsYSk6bn0s
YS5zb3J0PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlPXQsbj1udWxsLGEp
OmV9LGEuc3RhcnRBbmdsZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj0i
ZnVuY3Rpb24iPT10eXBlb2YgdD90OnltKCt0KSxhKTpyfSxhLmVuZEFuZ2xlPWZ1bmN0aW9uKHQp
e3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhpPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6eW0oK3Qp
LGEpOml9LGEucGFkQW5nbGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG89
ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp5bSgrdCksYSk6b30sYX0sdC5waWVjZXdpc2U9cGksdC5w
b2ludFJhZGlhbD1RbSx0LnBvaW50ZXI9ZWUsdC5wb2ludGVycz1mdW5jdGlvbih0LG4pe3JldHVy
biB0LnRhcmdldCYmKHQ9bmUodCksdm9pZCAwPT09biYmKG49dC5jdXJyZW50VGFyZ2V0KSx0PXQu
dG91Y2hlc3x8W3RdKSxBcnJheS5mcm9tKHQsKHQ9PmVlKHQsbikpKX0sdC5wb2x5Z29uQXJlYT1m
dW5jdGlvbih0KXtmb3IodmFyIG4sZT0tMSxyPXQubGVuZ3RoLGk9dFtyLTFdLG89MDsrK2U8cjsp
bj1pLGk9dFtlXSxvKz1uWzFdKmlbMF0tblswXSppWzFdO3JldHVybiBvLzJ9LHQucG9seWdvbkNl
bnRyb2lkPWZ1bmN0aW9uKHQpe2Zvcih2YXIgbixlLHI9LTEsaT10Lmxlbmd0aCxvPTAsYT0wLHU9
dFtpLTFdLGM9MDsrK3I8aTspbj11LHU9dFtyXSxjKz1lPW5bMF0qdVsxXS11WzBdKm5bMV0sbys9
KG5bMF0rdVswXSkqZSxhKz0oblsxXSt1WzFdKSplO3JldHVybltvLyhjKj0zKSxhL2NdfSx0LnBv
bHlnb25Db250YWlucz1mdW5jdGlvbih0LG4pe2Zvcih2YXIgZSxyLGk9dC5sZW5ndGgsbz10W2kt
MV0sYT1uWzBdLHU9blsxXSxjPW9bMF0sZj1vWzFdLHM9ITEsbD0wO2w8aTsrK2wpZT0obz10W2xd
KVswXSwocj1vWzFdKT51IT1mPnUmJmE8KGMtZSkqKHUtcikvKGYtcikrZSYmKHM9IXMpLGM9ZSxm
PXI7cmV0dXJuIHN9LHQucG9seWdvbkh1bGw9ZnVuY3Rpb24odCl7aWYoKGU9dC5sZW5ndGgpPDMp
cmV0dXJuIG51bGw7dmFyIG4sZSxyPW5ldyBBcnJheShlKSxpPW5ldyBBcnJheShlKTtmb3Iobj0w
O248ZTsrK24pcltuXT1bK3Rbbl1bMF0sK3Rbbl1bMV0sbl07Zm9yKHIuc29ydChYcCksbj0wO248
ZTsrK24paVtuXT1bcltuXVswXSwtcltuXVsxXV07dmFyIG89R3AociksYT1HcChpKSx1PWFbMF09
PT1vWzBdLGM9YVthLmxlbmd0aC0xXT09PW9bby5sZW5ndGgtMV0sZj1bXTtmb3Iobj1vLmxlbmd0
aC0xO24+PTA7LS1uKWYucHVzaCh0W3Jbb1tuXV1bMl1dKTtmb3Iobj0rdTtuPGEubGVuZ3RoLWM7
KytuKWYucHVzaCh0W3JbYVtuXV1bMl1dKTtyZXR1cm4gZn0sdC5wb2x5Z29uTGVuZ3RoPWZ1bmN0
aW9uKHQpe2Zvcih2YXIgbixlLHI9LTEsaT10Lmxlbmd0aCxvPXRbaS0xXSxhPW9bMF0sdT1vWzFd
LGM9MDsrK3I8aTspbj1hLGU9dSxuLT1hPShvPXRbcl0pWzBdLGUtPXU9b1sxXSxjKz1NYXRoLmh5
cG90KG4sZSk7cmV0dXJuIGN9LHQucHJlY2lzaW9uRml4ZWQ9bGYsdC5wcmVjaXNpb25QcmVmaXg9
aGYsdC5wcmVjaXNpb25Sb3VuZD1kZix0LnF1YWR0cmVlPURjLHQucXVhbnRpbGU9dXQsdC5xdWFu
dGlsZUluZGV4PWZ0LHQucXVhbnRpbGVTb3J0ZWQ9Y3QsdC5xdWFudGl6ZT1mdW5jdGlvbih0LG4p
e2Zvcih2YXIgZT1uZXcgQXJyYXkobikscj0wO3I8bjsrK3IpZVtyXT10KHIvKG4tMSkpO3JldHVy
biBlfSx0LnF1aWNrc2VsZWN0PWl0LHQucmFkaWFsQXJlYT1LbSx0LnJhZGlhbExpbmU9Wm0sdC5y
YW5kb21CYXRlcz10Zyx0LnJhbmRvbUJlcm5vdWxsaT1yZyx0LnJhbmRvbUJldGE9YWcsdC5yYW5k
b21CaW5vbWlhbD11Zyx0LnJhbmRvbUNhdWNoeT1mZyx0LnJhbmRvbUV4cG9uZW50aWFsPW5nLHQu
cmFuZG9tR2FtbWE9b2csdC5yYW5kb21HZW9tZXRyaWM9aWcsdC5yYW5kb21JbnQ9WnAsdC5yYW5k
b21JcndpbkhhbGw9SnAsdC5yYW5kb21MY2c9ZnVuY3Rpb24odD1NYXRoLnJhbmRvbSgpKXtsZXQg
bj0wfCgwPD10JiZ0PDE/dC9oZzpNYXRoLmFicyh0KSk7cmV0dXJuKCk9PihuPTE2NjQ1MjUqbisx
MDEzOTA0MjIzfDAsaGcqKG4+Pj4wKSl9LHQucmFuZG9tTG9nTm9ybWFsPVFwLHQucmFuZG9tTG9n
aXN0aWM9c2csdC5yYW5kb21Ob3JtYWw9S3AsdC5yYW5kb21QYXJldG89ZWcsdC5yYW5kb21Qb2lz
c29uPWxnLHQucmFuZG9tVW5pZm9ybT1XcCx0LnJhbmRvbVdlaWJ1bGw9Y2csdC5yYW5nZT1odCx0
LnJhbms9ZnVuY3Rpb24odCxlPW4pe2lmKCJmdW5jdGlvbiIhPXR5cGVvZiB0W1N5bWJvbC5pdGVy
YXRvcl0pdGhyb3cgbmV3IFR5cGVFcnJvcigidmFsdWVzIGlzIG5vdCBpdGVyYWJsZSIpO2xldCBy
PUFycmF5LmZyb20odCk7Y29uc3QgaT1uZXcgRmxvYXQ2NEFycmF5KHIubGVuZ3RoKTsyIT09ZS5s
ZW5ndGgmJihyPXIubWFwKGUpLGU9bik7Y29uc3Qgbz0odCxuKT0+ZShyW3RdLHJbbl0pO2xldCBh
LHU7cmV0dXJuKHQ9VWludDMyQXJyYXkuZnJvbShyLCgodCxuKT0+bikpKS5zb3J0KGU9PT1uPyh0
LG4pPT5CKHJbdF0scltuXSk6TyhvKSksdC5mb3JFYWNoKCgodCxuKT0+e2NvbnN0IGU9byh0LHZv
aWQgMD09PWE/dDphKTtlPj0wPygodm9pZCAwPT09YXx8ZT4wKSYmKGE9dCx1PW4pLGlbdF09dSk6
aVt0XT1OYU59KSksaX0sdC5yZWR1Y2U9ZnVuY3Rpb24odCxuLGUpe2lmKCJmdW5jdGlvbiIhPXR5
cGVvZiBuKXRocm93IG5ldyBUeXBlRXJyb3IoInJlZHVjZXIgaXMgbm90IGEgZnVuY3Rpb24iKTtj
b25zdCByPXRbU3ltYm9sLml0ZXJhdG9yXSgpO2xldCBpLG8sYT0tMTtpZihhcmd1bWVudHMubGVu
Z3RoPDMpe2lmKCh7ZG9uZTppLHZhbHVlOmV9PXIubmV4dCgpKSxpKXJldHVybjsrK2F9Zm9yKDso
e2RvbmU6aSx2YWx1ZTpvfT1yLm5leHQoKSksIWk7KWU9bihlLG8sKythLHQpO3JldHVybiBlfSx0
LnJldmVyc2U9ZnVuY3Rpb24odCl7aWYoImZ1bmN0aW9uIiE9dHlwZW9mIHRbU3ltYm9sLml0ZXJh
dG9yXSl0aHJvdyBuZXcgVHlwZUVycm9yKCJ2YWx1ZXMgaXMgbm90IGl0ZXJhYmxlIik7cmV0dXJu
IEFycmF5LmZyb20odCkucmV2ZXJzZSgpfSx0LnJnYj1xZSx0LnJpYmJvbj1mdW5jdGlvbigpe3Jl
dHVybiBaYSgpfSx0LnJpYmJvbkFycm93PWZ1bmN0aW9uKCl7cmV0dXJuIFphKFdhKX0sdC5yb2xs
dXA9RCx0LnJvbGx1cHM9Uix0LnNjYWxlQmFuZD12Zyx0LnNjYWxlRGl2ZXJnaW5nPWZ1bmN0aW9u
IHQoKXt2YXIgbj1rZyhqXygpKHhnKSk7cmV0dXJuIG4uY29weT1mdW5jdGlvbigpe3JldHVybiBZ
XyhuLHQoKSl9LHBnLmFwcGx5KG4sYXJndW1lbnRzKX0sdC5zY2FsZURpdmVyZ2luZ0xvZz1mdW5j
dGlvbiB0KCl7dmFyIG49cWcoal8oKSkuZG9tYWluKFsuMSwxLDEwXSk7cmV0dXJuIG4uY29weT1m
dW5jdGlvbigpe3JldHVybiBZXyhuLHQoKSkuYmFzZShuLmJhc2UoKSl9LHBnLmFwcGx5KG4sYXJn
dW1lbnRzKX0sdC5zY2FsZURpdmVyZ2luZ1Bvdz1IXyx0LnNjYWxlRGl2ZXJnaW5nU3FydD1mdW5j
dGlvbigpe3JldHVybiBIXy5hcHBseShudWxsLGFyZ3VtZW50cykuZXhwb25lbnQoLjUpfSx0LnNj
YWxlRGl2ZXJnaW5nU3ltbG9nPWZ1bmN0aW9uIHQoKXt2YXIgbj1PZyhqXygpKTtyZXR1cm4gbi5j
b3B5PWZ1bmN0aW9uKCl7cmV0dXJuIFlfKG4sdCgpKS5jb25zdGFudChuLmNvbnN0YW50KCkpfSxw
Zy5hcHBseShuLGFyZ3VtZW50cyl9LHQuc2NhbGVJZGVudGl0eT1mdW5jdGlvbiB0KG4pe3ZhciBl
O2Z1bmN0aW9uIHIodCl7cmV0dXJuIG51bGw9PXR8fGlzTmFOKHQ9K3QpP2U6dH1yZXR1cm4gci5p
bnZlcnQ9cixyLmRvbWFpbj1yLnJhbmdlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPyhuPUFycmF5LmZyb20odCxiZykscik6bi5zbGljZSgpfSxyLnVua25vd249ZnVuY3Rpb24o
dCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGU9dCxyKTplfSxyLmNvcHk9ZnVuY3Rpb24oKXty
ZXR1cm4gdChuKS51bmtub3duKGUpfSxuPWFyZ3VtZW50cy5sZW5ndGg/QXJyYXkuZnJvbShuLGJn
KTpbMCwxXSxrZyhyKX0sdC5zY2FsZUltcGxpY2l0PWdnLHQuc2NhbGVMaW5lYXI9ZnVuY3Rpb24g
dCgpe3ZhciBuPUVnKCk7cmV0dXJuIG4uY29weT1mdW5jdGlvbigpe3JldHVybiBBZyhuLHQoKSl9
LGRnLmFwcGx5KG4sYXJndW1lbnRzKSxrZyhuKX0sdC5zY2FsZUxvZz1mdW5jdGlvbiB0KCl7Y29u
c3Qgbj1xZyhTZygpKS5kb21haW4oWzEsMTBdKTtyZXR1cm4gbi5jb3B5PSgpPT5BZyhuLHQoKSku
YmFzZShuLmJhc2UoKSksZGcuYXBwbHkobixhcmd1bWVudHMpLG59LHQuc2NhbGVPcmRpbmFsPXln
LHQuc2NhbGVQb2ludD1mdW5jdGlvbigpe3JldHVybiBfZyh2Zy5hcHBseShudWxsLGFyZ3VtZW50
cykucGFkZGluZ0lubmVyKDEpKX0sdC5zY2FsZVBvdz1IZyx0LnNjYWxlUXVhbnRpbGU9ZnVuY3Rp
b24gdCgpe3ZhciBlLHI9W10saT1bXSxvPVtdO2Z1bmN0aW9uIGEoKXt2YXIgdD0wLG49TWF0aC5t
YXgoMSxpLmxlbmd0aCk7Zm9yKG89bmV3IEFycmF5KG4tMSk7Kyt0PG47KW9bdC0xXT1jdChyLHQv
bik7cmV0dXJuIHV9ZnVuY3Rpb24gdSh0KXtyZXR1cm4gbnVsbD09dHx8aXNOYU4odD0rdCk/ZTpp
W2wobyx0KV19cmV0dXJuIHUuaW52ZXJ0RXh0ZW50PWZ1bmN0aW9uKHQpe3ZhciBuPWkuaW5kZXhP
Zih0KTtyZXR1cm4gbjwwP1tOYU4sTmFOXTpbbj4wP29bbi0xXTpyWzBdLG48by5sZW5ndGg/b1tu
XTpyW3IubGVuZ3RoLTFdXX0sdS5kb21haW49ZnVuY3Rpb24odCl7aWYoIWFyZ3VtZW50cy5sZW5n
dGgpcmV0dXJuIHIuc2xpY2UoKTtyPVtdO2ZvcihsZXQgbiBvZiB0KW51bGw9PW58fGlzTmFOKG49
K24pfHxyLnB1c2gobik7cmV0dXJuIHIuc29ydChuKSxhKCl9LHUucmFuZ2U9ZnVuY3Rpb24odCl7
cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGk9QXJyYXkuZnJvbSh0KSxhKCkpOmkuc2xpY2UoKX0s
dS51bmtub3duPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlPXQsdSk6ZX0s
dS5xdWFudGlsZXM9ZnVuY3Rpb24oKXtyZXR1cm4gby5zbGljZSgpfSx1LmNvcHk9ZnVuY3Rpb24o
KXtyZXR1cm4gdCgpLmRvbWFpbihyKS5yYW5nZShpKS51bmtub3duKGUpfSxkZy5hcHBseSh1LGFy
Z3VtZW50cyl9LHQuc2NhbGVRdWFudGl6ZT1mdW5jdGlvbiB0KCl7dmFyIG4sZT0wLHI9MSxpPTEs
bz1bLjVdLGE9WzAsMV07ZnVuY3Rpb24gdSh0KXtyZXR1cm4gbnVsbCE9dCYmdDw9dD9hW2wobyx0
LDAsaSldOm59ZnVuY3Rpb24gYygpe3ZhciB0PS0xO2ZvcihvPW5ldyBBcnJheShpKTsrK3Q8aTsp
b1t0XT0oKHQrMSkqci0odC1pKSplKS8oaSsxKTtyZXR1cm4gdX1yZXR1cm4gdS5kb21haW49ZnVu
Y3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KFtlLHJdPXQsZT0rZSxyPStyLGMoKSk6
W2Uscl19LHUucmFuZ2U9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGk9KGE9
QXJyYXkuZnJvbSh0KSkubGVuZ3RoLTEsYygpKTphLnNsaWNlKCl9LHUuaW52ZXJ0RXh0ZW50PWZ1
bmN0aW9uKHQpe3ZhciBuPWEuaW5kZXhPZih0KTtyZXR1cm4gbjwwP1tOYU4sTmFOXTpuPDE/W2Us
b1swXV06bj49aT9bb1tpLTFdLHJdOltvW24tMV0sb1tuXV19LHUudW5rbm93bj1mdW5jdGlvbih0
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obj10LHUpOnV9LHUudGhyZXNob2xkcz1mdW5jdGlv
bigpe3JldHVybiBvLnNsaWNlKCl9LHUuY29weT1mdW5jdGlvbigpe3JldHVybiB0KCkuZG9tYWlu
KFtlLHJdKS5yYW5nZShhKS51bmtub3duKG4pfSxkZy5hcHBseShrZyh1KSxhcmd1bWVudHMpfSx0
LnNjYWxlUmFkaWFsPWZ1bmN0aW9uIHQoKXt2YXIgbixlPUVnKCkscj1bMCwxXSxpPSExO2Z1bmN0
aW9uIG8odCl7dmFyIHI9ZnVuY3Rpb24odCl7cmV0dXJuIE1hdGguc2lnbih0KSpNYXRoLnNxcnQo
TWF0aC5hYnModCkpfShlKHQpKTtyZXR1cm4gaXNOYU4ocik/bjppP01hdGgucm91bmQocik6cn1y
ZXR1cm4gby5pbnZlcnQ9ZnVuY3Rpb24odCl7cmV0dXJuIGUuaW52ZXJ0KFhnKHQpKX0sby5kb21h
aW49ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGUuZG9tYWluKHQpLG8pOmUu
ZG9tYWluKCl9LG8ucmFuZ2U9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGUu
cmFuZ2UoKHI9QXJyYXkuZnJvbSh0LGJnKSkubWFwKFhnKSksbyk6ci5zbGljZSgpfSxvLnJhbmdl
Um91bmQ9ZnVuY3Rpb24odCl7cmV0dXJuIG8ucmFuZ2UodCkucm91bmQoITApfSxvLnJvdW5kPWZ1
bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhpPSEhdCxvKTppfSxvLmNsYW1wPWZ1
bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlLmNsYW1wKHQpLG8pOmUuY2xhbXAo
KX0sby51bmtub3duPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhuPXQsbyk6
bn0sby5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIHQoZS5kb21haW4oKSxyKS5yb3VuZChpKS5jbGFt
cChlLmNsYW1wKCkpLnVua25vd24obil9LGRnLmFwcGx5KG8sYXJndW1lbnRzKSxrZyhvKX0sdC5z
Y2FsZVNlcXVlbnRpYWw9ZnVuY3Rpb24gdCgpe3ZhciBuPWtnKEJfKCkoeGcpKTtyZXR1cm4gbi5j
b3B5PWZ1bmN0aW9uKCl7cmV0dXJuIFlfKG4sdCgpKX0scGcuYXBwbHkobixhcmd1bWVudHMpfSx0
LnNjYWxlU2VxdWVudGlhbExvZz1mdW5jdGlvbiB0KCl7dmFyIG49cWcoQl8oKSkuZG9tYWluKFsx
LDEwXSk7cmV0dXJuIG4uY29weT1mdW5jdGlvbigpe3JldHVybiBZXyhuLHQoKSkuYmFzZShuLmJh
c2UoKSl9LHBnLmFwcGx5KG4sYXJndW1lbnRzKX0sdC5zY2FsZVNlcXVlbnRpYWxQb3c9TF8sdC5z
Y2FsZVNlcXVlbnRpYWxRdWFudGlsZT1mdW5jdGlvbiB0KCl7dmFyIGU9W10scj14ZztmdW5jdGlv
biBpKHQpe2lmKG51bGwhPXQmJiFpc05hTih0PSt0KSlyZXR1cm4gcigobChlLHQsMSktMSkvKGUu
bGVuZ3RoLTEpKX1yZXR1cm4gaS5kb21haW49ZnVuY3Rpb24odCl7aWYoIWFyZ3VtZW50cy5sZW5n
dGgpcmV0dXJuIGUuc2xpY2UoKTtlPVtdO2ZvcihsZXQgbiBvZiB0KW51bGw9PW58fGlzTmFOKG49
K24pfHxlLnB1c2gobik7cmV0dXJuIGUuc29ydChuKSxpfSxpLmludGVycG9sYXRvcj1mdW5jdGlv
bih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj10LGkpOnJ9LGkucmFuZ2U9ZnVuY3Rpb24o
KXtyZXR1cm4gZS5tYXAoKCh0LG4pPT5yKG4vKGUubGVuZ3RoLTEpKSkpfSxpLnF1YW50aWxlcz1m
dW5jdGlvbih0KXtyZXR1cm4gQXJyYXkuZnJvbSh7bGVuZ3RoOnQrMX0sKChuLHIpPT51dChlLHIv
dCkpKX0saS5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIHQocikuZG9tYWluKGUpfSxwZy5hcHBseShp
LGFyZ3VtZW50cyl9LHQuc2NhbGVTZXF1ZW50aWFsU3FydD1mdW5jdGlvbigpe3JldHVybiBMXy5h
cHBseShudWxsLGFyZ3VtZW50cykuZXhwb25lbnQoLjUpfSx0LnNjYWxlU2VxdWVudGlhbFN5bWxv
Zz1mdW5jdGlvbiB0KCl7dmFyIG49T2coQl8oKSk7cmV0dXJuIG4uY29weT1mdW5jdGlvbigpe3Jl
dHVybiBZXyhuLHQoKSkuY29uc3RhbnQobi5jb25zdGFudCgpKX0scGcuYXBwbHkobixhcmd1bWVu
dHMpfSx0LnNjYWxlU3FydD1mdW5jdGlvbigpe3JldHVybiBIZy5hcHBseShudWxsLGFyZ3VtZW50
cykuZXhwb25lbnQoLjUpfSx0LnNjYWxlU3ltbG9nPWZ1bmN0aW9uIHQoKXt2YXIgbj1PZyhTZygp
KTtyZXR1cm4gbi5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIEFnKG4sdCgpKS5jb25zdGFudChuLmNv
bnN0YW50KCkpfSxkZy5hcHBseShuLGFyZ3VtZW50cyl9LHQuc2NhbGVUaHJlc2hvbGQ9ZnVuY3Rp
b24gdCgpe3ZhciBuLGU9Wy41XSxyPVswLDFdLGk9MTtmdW5jdGlvbiBvKHQpe3JldHVybiBudWxs
IT10JiZ0PD10P3JbbChlLHQsMCxpKV06bn1yZXR1cm4gby5kb21haW49ZnVuY3Rpb24odCl7cmV0
dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGU9QXJyYXkuZnJvbSh0KSxpPU1hdGgubWluKGUubGVuZ3Ro
LHIubGVuZ3RoLTEpLG8pOmUuc2xpY2UoKX0sby5yYW5nZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJn
dW1lbnRzLmxlbmd0aD8ocj1BcnJheS5mcm9tKHQpLGk9TWF0aC5taW4oZS5sZW5ndGgsci5sZW5n
dGgtMSksbyk6ci5zbGljZSgpfSxvLmludmVydEV4dGVudD1mdW5jdGlvbih0KXt2YXIgbj1yLmlu
ZGV4T2YodCk7cmV0dXJuW2Vbbi0xXSxlW25dXX0sby51bmtub3duPWZ1bmN0aW9uKHQpe3JldHVy
biBhcmd1bWVudHMubGVuZ3RoPyhuPXQsbyk6bn0sby5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIHQo
KS5kb21haW4oZSkucmFuZ2UocikudW5rbm93bihuKX0sZGcuYXBwbHkobyxhcmd1bWVudHMpfSx0
LnNjYWxlVGltZT1mdW5jdGlvbigpe3JldHVybiBkZy5hcHBseShPXyhjdixmdixudixLeSx3eSxn
eSxseSx1eSxveSx0LnRpbWVGb3JtYXQpLmRvbWFpbihbbmV3IERhdGUoMmUzLDAsMSksbmV3IERh
dGUoMmUzLDAsMildKSxhcmd1bWVudHMpfSx0LnNjYWxlVXRjPWZ1bmN0aW9uKCl7cmV0dXJuIGRn
LmFwcGx5KE9fKGF2LHV2LHJ2LEp5LHF5LHZ5LGR5LGZ5LG95LHQudXRjRm9ybWF0KS5kb21haW4o
W0RhdGUuVVRDKDJlMywwLDEpLERhdGUuVVRDKDJlMywwLDIpXSksYXJndW1lbnRzKX0sdC5zY2Fu
PWZ1bmN0aW9uKHQsbil7Y29uc3QgZT1kdCh0LG4pO3JldHVybiBlPDA/dm9pZCAwOmV9LHQuc2No
ZW1lQWNjZW50PVZfLHQuc2NoZW1lQmx1ZXM9WGIsdC5zY2hlbWVCckJHPWliLHQuc2NoZW1lQnVH
bj13Yix0LnNjaGVtZUJ1UHU9VGIsdC5zY2hlbWVDYXRlZ29yeTEwPUdfLHQuc2NoZW1lRGFyazI9
V18sdC5zY2hlbWVHbkJ1PVNiLHQuc2NoZW1lR3JlZW5zPVZiLHQuc2NoZW1lR3JleXM9WmIsdC5z
Y2hlbWVPclJkPU5iLHQuc2NoZW1lT3Jhbmdlcz1lbSx0LnNjaGVtZVBSR249YWIsdC5zY2hlbWVQ
YWlyZWQ9Wl8sdC5zY2hlbWVQYXN0ZWwxPUtfLHQuc2NoZW1lUGFzdGVsMj1RXyx0LnNjaGVtZVBp
WUc9Y2IsdC5zY2hlbWVQdUJ1PXpiLHQuc2NoZW1lUHVCdUduPUNiLHQuc2NoZW1lUHVPcj1zYix0
LnNjaGVtZVB1UmQ9RGIsdC5zY2hlbWVQdXJwbGVzPVFiLHQuc2NoZW1lUmRCdT1oYix0LnNjaGVt
ZVJkR3k9cGIsdC5zY2hlbWVSZFB1PUZiLHQuc2NoZW1lUmRZbEJ1PXliLHQuc2NoZW1lUmRZbEdu
PV9iLHQuc2NoZW1lUmVkcz10bSx0LnNjaGVtZVNldDE9Sl8sdC5zY2hlbWVTZXQyPXRiLHQuc2No
ZW1lU2V0Mz1uYix0LnNjaGVtZVNwZWN0cmFsPW1iLHQuc2NoZW1lVGFibGVhdTEwPWViLHQuc2No
ZW1lWWxHbj1PYix0LnNjaGVtZVlsR25CdT1VYix0LnNjaGVtZVlsT3JCcj1ZYix0LnNjaGVtZVls
T3JSZD1qYix0LnNlbGVjdD1Lbix0LnNlbGVjdEFsbD1mdW5jdGlvbih0KXtyZXR1cm4ic3RyaW5n
Ij09dHlwZW9mIHQ/bmV3IFduKFtkb2N1bWVudC5xdWVyeVNlbGVjdG9yQWxsKHQpXSxbZG9jdW1l
bnQuZG9jdW1lbnRFbGVtZW50XSk6bmV3IFduKFtYdCh0KV0sVm4pfSx0LnNlbGVjdGlvbj1abix0
LnNlbGVjdG9yPUh0LHQuc2VsZWN0b3JBbGw9VnQsdC5zaHVmZmxlPXB0LHQuc2h1ZmZsZXI9Z3Qs
dC5zb21lPWZ1bmN0aW9uKHQsbil7aWYoImZ1bmN0aW9uIiE9dHlwZW9mIG4pdGhyb3cgbmV3IFR5
cGVFcnJvcigidGVzdCBpcyBub3QgYSBmdW5jdGlvbiIpO2xldCBlPS0xO2Zvcihjb25zdCByIG9m
IHQpaWYobihyLCsrZSx0KSlyZXR1cm4hMDtyZXR1cm4hMX0sdC5zb3J0PUksdC5zdGFjaz1mdW5j
dGlvbigpe3ZhciB0PXltKFtdKSxuPWR3LGU9aHcscj1wdztmdW5jdGlvbiBpKGkpe3ZhciBvLGEs
dT1BcnJheS5mcm9tKHQuYXBwbHkodGhpcyxhcmd1bWVudHMpLGd3KSxjPXUubGVuZ3RoLGY9LTE7
Zm9yKGNvbnN0IHQgb2YgaSlmb3Iobz0wLCsrZjtvPGM7KytvKSh1W29dW2ZdPVswLCtyKHQsdVtv
XS5rZXksZixpKV0pLmRhdGE9dDtmb3Iobz0wLGE9cW0obih1KSk7bzxjOysrbyl1W2Fbb11dLmlu
ZGV4PW87cmV0dXJuIGUodSxhKSx1fXJldHVybiBpLmtleXM9ZnVuY3Rpb24obil7cmV0dXJuIGFy
Z3VtZW50cy5sZW5ndGg/KHQ9ImZ1bmN0aW9uIj09dHlwZW9mIG4/bjp5bShBcnJheS5mcm9tKG4p
KSxpKTp0fSxpLnZhbHVlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhyPSJm
dW5jdGlvbiI9PXR5cGVvZiB0P3Q6eW0oK3QpLGkpOnJ9LGkub3JkZXI9ZnVuY3Rpb24odCl7cmV0
dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49bnVsbD09dD9kdzoiZnVuY3Rpb24iPT10eXBlb2YgdD90
OnltKEFycmF5LmZyb20odCkpLGkpOm59LGkub2Zmc2V0PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1
bWVudHMubGVuZ3RoPyhlPW51bGw9PXQ/aHc6dCxpKTplfSxpfSx0LnN0YWNrT2Zmc2V0RGl2ZXJn
aW5nPWZ1bmN0aW9uKHQsbil7aWYoKHU9dC5sZW5ndGgpPjApZm9yKHZhciBlLHIsaSxvLGEsdSxj
PTAsZj10W25bMF1dLmxlbmd0aDtjPGY7KytjKWZvcihvPWE9MCxlPTA7ZTx1OysrZSkoaT0ocj10
W25bZV1dW2NdKVsxXS1yWzBdKT4wPyhyWzBdPW8sclsxXT1vKz1pKTppPDA/KHJbMV09YSxyWzBd
PWErPWkpOihyWzBdPTAsclsxXT1pKX0sdC5zdGFja09mZnNldEV4cGFuZD1mdW5jdGlvbih0LG4p
e2lmKChyPXQubGVuZ3RoKT4wKXtmb3IodmFyIGUscixpLG89MCxhPXRbMF0ubGVuZ3RoO288YTsr
K28pe2ZvcihpPWU9MDtlPHI7KytlKWkrPXRbZV1bb11bMV18fDA7aWYoaSlmb3IoZT0wO2U8cjsr
K2UpdFtlXVtvXVsxXS89aX1odyh0LG4pfX0sdC5zdGFja09mZnNldE5vbmU9aHcsdC5zdGFja09m
ZnNldFNpbGhvdWV0dGU9ZnVuY3Rpb24odCxuKXtpZigoZT10Lmxlbmd0aCk+MCl7Zm9yKHZhciBl
LHI9MCxpPXRbblswXV0sbz1pLmxlbmd0aDtyPG87KytyKXtmb3IodmFyIGE9MCx1PTA7YTxlOysr
YSl1Kz10W2FdW3JdWzFdfHwwO2lbcl1bMV0rPWlbcl1bMF09LXUvMn1odyh0LG4pfX0sdC5zdGFj
a09mZnNldFdpZ2dsZT1mdW5jdGlvbih0LG4pe2lmKChpPXQubGVuZ3RoKT4wJiYocj0oZT10W25b
MF1dKS5sZW5ndGgpPjApe2Zvcih2YXIgZSxyLGksbz0wLGE9MTthPHI7KythKXtmb3IodmFyIHU9
MCxjPTAsZj0wO3U8aTsrK3Upe2Zvcih2YXIgcz10W25bdV1dLGw9c1thXVsxXXx8MCxoPShsLShz
W2EtMV1bMV18fDApKS8yLGQ9MDtkPHU7KytkKXt2YXIgcD10W25bZF1dO2grPShwW2FdWzFdfHww
KS0ocFthLTFdWzFdfHwwKX1jKz1sLGYrPWgqbH1lW2EtMV1bMV0rPWVbYS0xXVswXT1vLGMmJihv
LT1mL2MpfWVbYS0xXVsxXSs9ZVthLTFdWzBdPW8saHcodCxuKX19LHQuc3RhY2tPcmRlckFwcGVh
cmFuY2U9eXcsdC5zdGFja09yZGVyQXNjZW5kaW5nPV93LHQuc3RhY2tPcmRlckRlc2NlbmRpbmc9
ZnVuY3Rpb24odCl7cmV0dXJuIF93KHQpLnJldmVyc2UoKX0sdC5zdGFja09yZGVySW5zaWRlT3V0
PWZ1bmN0aW9uKHQpe3ZhciBuLGUscj10Lmxlbmd0aCxpPXQubWFwKGJ3KSxvPXl3KHQpLGE9MCx1
PTAsYz1bXSxmPVtdO2ZvcihuPTA7bjxyOysrbillPW9bbl0sYTx1PyhhKz1pW2VdLGMucHVzaChl
KSk6KHUrPWlbZV0sZi5wdXNoKGUpKTtyZXR1cm4gZi5yZXZlcnNlKCkuY29uY2F0KGMpfSx0LnN0
YWNrT3JkZXJOb25lPWR3LHQuc3RhY2tPcmRlclJldmVyc2U9ZnVuY3Rpb24odCl7cmV0dXJuIGR3
KHQpLnJldmVyc2UoKX0sdC5zdHJhdGlmeT1mdW5jdGlvbigpe3ZhciB0LG49Q3AsZT1QcDtmdW5j
dGlvbiByKHIpe3ZhciBpLG8sYSx1LGMsZixzLGwsaD1BcnJheS5mcm9tKHIpLGQ9bixwPWUsZz1u
ZXcgTWFwO2lmKG51bGwhPXQpe2NvbnN0IG49aC5tYXAoKChuLGUpPT5mdW5jdGlvbih0KXt0PWAk
e3R9YDtsZXQgbj10Lmxlbmd0aDskcCh0LG4tMSkmJiEkcCh0LG4tMikmJih0PXQuc2xpY2UoMCwt
MSkpO3JldHVybiIvIj09PXRbMF0/dDpgLyR7dH1gfSh0KG4sZSxyKSkpKSxlPW4ubWFwKHpwKSxp
PW5ldyBTZXQobikuYWRkKCIiKTtmb3IoY29uc3QgdCBvZiBlKWkuaGFzKHQpfHwoaS5hZGQodCks
bi5wdXNoKHQpLGUucHVzaCh6cCh0KSksaC5wdXNoKGtwKSk7ZD0odCxlKT0+bltlXSxwPSh0LG4p
PT5lW25dfWZvcihhPTAsaT1oLmxlbmd0aDthPGk7KythKW89aFthXSxmPWhbYV09bmV3IEpkKG8p
LG51bGwhPShzPWQobyxhLHIpKSYmKHMrPSIiKSYmKGw9Zi5pZD1zLGcuc2V0KGwsZy5oYXMobCk/
TnA6ZikpLG51bGwhPShzPXAobyxhLHIpKSYmKHMrPSIiKSYmKGYucGFyZW50PXMpO2ZvcihhPTA7
YTxpOysrYSlpZihzPShmPWhbYV0pLnBhcmVudCl7aWYoIShjPWcuZ2V0KHMpKSl0aHJvdyBuZXcg
RXJyb3IoIm1pc3Npbmc6ICIrcyk7aWYoYz09PU5wKXRocm93IG5ldyBFcnJvcigiYW1iaWd1b3Vz
OiAiK3MpO2MuY2hpbGRyZW4/Yy5jaGlsZHJlbi5wdXNoKGYpOmMuY2hpbGRyZW49W2ZdLGYucGFy
ZW50PWN9ZWxzZXtpZih1KXRocm93IG5ldyBFcnJvcigibXVsdGlwbGUgcm9vdHMiKTt1PWZ9aWYo
IXUpdGhyb3cgbmV3IEVycm9yKCJubyByb290Iik7aWYobnVsbCE9dCl7Zm9yKDt1LmRhdGE9PT1r
cCYmMT09PXUuY2hpbGRyZW4ubGVuZ3RoOyl1PXUuY2hpbGRyZW5bMF0sLS1pO2ZvcihsZXQgdD1o
Lmxlbmd0aC0xO3Q+PTAmJihmPWhbdF0pLmRhdGE9PT1rcDstLXQpZi5kYXRhPW51bGx9aWYodS5w
YXJlbnQ9RXAsdS5lYWNoQmVmb3JlKChmdW5jdGlvbih0KXt0LmRlcHRoPXQucGFyZW50LmRlcHRo
KzEsLS1pfSkpLmVhY2hCZWZvcmUoUWQpLHUucGFyZW50PW51bGwsaT4wKXRocm93IG5ldyBFcnJv
cigiY3ljbGUiKTtyZXR1cm4gdX1yZXR1cm4gci5pZD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1l
bnRzLmxlbmd0aD8obj10cCh0KSxyKTpufSxyLnBhcmVudElkPWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyhlPXRwKHQpLHIpOmV9LHIucGF0aD1mdW5jdGlvbihuKXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8odD10cChuKSxyKTp0fSxyfSx0LnN0eWxlPWJuLHQuc3Vic2V0PWZ1
bmN0aW9uKHQsbil7cmV0dXJuIGJ0KG4sdCl9LHQuc3VtPWZ1bmN0aW9uKHQsbil7bGV0IGU9MDtp
Zih2b2lkIDA9PT1uKWZvcihsZXQgbiBvZiB0KShuPStuKSYmKGUrPW4pO2Vsc2V7bGV0IHI9LTE7
Zm9yKGxldCBpIG9mIHQpKGk9K24oaSwrK3IsdCkpJiYoZSs9aSl9cmV0dXJuIGV9LHQuc3VwZXJz
ZXQ9YnQsdC5zdmc9a2MsdC5zeW1ib2w9ZnVuY3Rpb24odCxuKXtsZXQgZT1udWxsLHI9a20oaSk7
ZnVuY3Rpb24gaSgpe2xldCBpO2lmKGV8fChlPWk9cigpKSx0LmFwcGx5KHRoaXMsYXJndW1lbnRz
KS5kcmF3KGUsK24uYXBwbHkodGhpcyxhcmd1bWVudHMpKSxpKXJldHVybiBlPW51bGwsaSsiInx8
bnVsbH1yZXR1cm4gdD0iZnVuY3Rpb24iPT10eXBlb2YgdD90OnltKHR8fGZ4KSxuPSJmdW5jdGlv
biI9PXR5cGVvZiBuP246eW0odm9pZCAwPT09bj82NDorbiksaS50eXBlPWZ1bmN0aW9uKG4pe3Jl
dHVybiBhcmd1bWVudHMubGVuZ3RoPyh0PSJmdW5jdGlvbiI9PXR5cGVvZiBuP246eW0obiksaSk6
dH0saS5zaXplPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhuPSJmdW5jdGlv
biI9PXR5cGVvZiB0P3Q6eW0oK3QpLGkpOm59LGkuY29udGV4dD1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8oZT1udWxsPT10P251bGw6dCxpKTplfSxpfSx0LnN5bWJvbEFzdGVy
aXNrPWN4LHQuc3ltYm9sQ2lyY2xlPWZ4LHQuc3ltYm9sQ3Jvc3M9c3gsdC5zeW1ib2xEaWFtb25k
PWR4LHQuc3ltYm9sRGlhbW9uZDI9cHgsdC5zeW1ib2xQbHVzPWd4LHQuc3ltYm9sU3F1YXJlPXl4
LHQuc3ltYm9sU3F1YXJlMj12eCx0LnN5bWJvbFN0YXI9eHgsdC5zeW1ib2xUaW1lcz1QeCx0LnN5
bWJvbFRyaWFuZ2xlPU14LHQuc3ltYm9sVHJpYW5nbGUyPUF4LHQuc3ltYm9sV3llPUN4LHQuc3lt
Ym9sWD1QeCx0LnN5bWJvbHM9engsdC5zeW1ib2xzRmlsbD16eCx0LnN5bWJvbHNTdHJva2U9JHgs
dC50ZXh0PXhjLHQudGhyZXNob2xkRnJlZWRtYW5EaWFjb25pcz1mdW5jdGlvbih0LG4sZSl7Y29u
c3Qgcj1fKHQpLGk9dXQodCwuNzUpLXV0KHQsLjI1KTtyZXR1cm4gciYmaT9NYXRoLmNlaWwoKGUt
bikvKDIqaSpNYXRoLnBvdyhyLC0xLzMpKSk6MX0sdC50aHJlc2hvbGRTY290dD1mdW5jdGlvbih0
LG4sZSl7Y29uc3Qgcj1fKHQpLGk9TSh0KTtyZXR1cm4gciYmaT9NYXRoLmNlaWwoKGUtbikqTWF0
aC5jYnJ0KHIpLygzLjQ5KmkpKToxfSx0LnRocmVzaG9sZFN0dXJnZXM9USx0LnRpY2tGb3JtYXQ9
TmcsdC50aWNrSW5jcmVtZW50PVcsdC50aWNrU3RlcD1aLHQudGlja3M9Vix0LnRpbWVEYXk9Z3ks
dC50aW1lRGF5cz15eSx0LnRpbWVGb3JtYXREZWZhdWx0TG9jYWxlPXpfLHQudGltZUZvcm1hdExv
Y2FsZT1kdix0LnRpbWVGcmlkYXk9RXksdC50aW1lRnJpZGF5cz1EeSx0LnRpbWVIb3VyPWx5LHQu
dGltZUhvdXJzPWh5LHQudGltZUludGVydmFsPVdnLHQudGltZU1pbGxpc2Vjb25kPVpnLHQudGlt
ZU1pbGxpc2Vjb25kcz1LZyx0LnRpbWVNaW51dGU9dXksdC50aW1lTWludXRlcz1jeSx0LnRpbWVN
b25kYXk9TXksdC50aW1lTW9uZGF5cz1DeSx0LnRpbWVNb250aD1LeSx0LnRpbWVNb250aHM9UXks
dC50aW1lU2F0dXJkYXk9TnksdC50aW1lU2F0dXJkYXlzPVJ5LHQudGltZVNlY29uZD1veSx0LnRp
bWVTZWNvbmRzPWF5LHQudGltZVN1bmRheT13eSx0LnRpbWVTdW5kYXlzPWt5LHQudGltZVRodXJz
ZGF5PVN5LHQudGltZVRodXJzZGF5cz0keSx0LnRpbWVUaWNrSW50ZXJ2YWw9ZnYsdC50aW1lVGlj
a3M9Y3YsdC50aW1lVHVlc2RheT1UeSx0LnRpbWVUdWVzZGF5cz1QeSx0LnRpbWVXZWRuZXNkYXk9
QXksdC50aW1lV2VkbmVzZGF5cz16eSx0LnRpbWVXZWVrPXd5LHQudGltZVdlZWtzPWt5LHQudGlt
ZVllYXI9bnYsdC50aW1lWWVhcnM9ZXYsdC50aW1lb3V0PURpLHQudGltZXI9a2ksdC50aW1lckZs
dXNoPUNpLHQudHJhbnNpdGlvbj15byx0LnRyYW5zcG9zZT15dCx0LnRyZWU9ZnVuY3Rpb24oKXt2
YXIgdD1EcCxuPTEsZT0xLHI9bnVsbDtmdW5jdGlvbiBpKGkpe3ZhciBjPWZ1bmN0aW9uKHQpe2Zv
cih2YXIgbixlLHIsaSxvLGE9bmV3IElwKHQsMCksdT1bYV07bj11LnBvcCgpOylpZihyPW4uXy5j
aGlsZHJlbilmb3Iobi5jaGlsZHJlbj1uZXcgQXJyYXkobz1yLmxlbmd0aCksaT1vLTE7aT49MDst
LWkpdS5wdXNoKGU9bi5jaGlsZHJlbltpXT1uZXcgSXAocltpXSxpKSksZS5wYXJlbnQ9bjtyZXR1
cm4oYS5wYXJlbnQ9bmV3IElwKG51bGwsMCkpLmNoaWxkcmVuPVthXSxhfShpKTtpZihjLmVhY2hB
ZnRlcihvKSxjLnBhcmVudC5tPS1jLnosYy5lYWNoQmVmb3JlKGEpLHIpaS5lYWNoQmVmb3JlKHUp
O2Vsc2V7dmFyIGY9aSxzPWksbD1pO2kuZWFjaEJlZm9yZSgoZnVuY3Rpb24odCl7dC54PGYueCYm
KGY9dCksdC54PnMueCYmKHM9dCksdC5kZXB0aD5sLmRlcHRoJiYobD10KX0pKTt2YXIgaD1mPT09
cz8xOnQoZixzKS8yLGQ9aC1mLngscD1uLyhzLngraCtkKSxnPWUvKGwuZGVwdGh8fDEpO2kuZWFj
aEJlZm9yZSgoZnVuY3Rpb24odCl7dC54PSh0LngrZCkqcCx0Lnk9dC5kZXB0aCpnfSkpfXJldHVy
biBpfWZ1bmN0aW9uIG8obil7dmFyIGU9bi5jaGlsZHJlbixyPW4ucGFyZW50LmNoaWxkcmVuLGk9
bi5pP3Jbbi5pLTFdOm51bGw7aWYoZSl7IWZ1bmN0aW9uKHQpe2Zvcih2YXIgbixlPTAscj0wLGk9
dC5jaGlsZHJlbixvPWkubGVuZ3RoOy0tbz49MDspKG49aVtvXSkueis9ZSxuLm0rPWUsZSs9bi5z
KyhyKz1uLmMpfShuKTt2YXIgbz0oZVswXS56K2VbZS5sZW5ndGgtMV0ueikvMjtpPyhuLno9aS56
K3Qobi5fLGkuXyksbi5tPW4uei1vKTpuLno9b31lbHNlIGkmJihuLno9aS56K3Qobi5fLGkuXykp
O24ucGFyZW50LkE9ZnVuY3Rpb24obixlLHIpe2lmKGUpe2Zvcih2YXIgaSxvPW4sYT1uLHU9ZSxj
PW8ucGFyZW50LmNoaWxkcmVuWzBdLGY9by5tLHM9YS5tLGw9dS5tLGg9Yy5tO3U9RnAodSksbz1S
cChvKSx1JiZvOyljPVJwKGMpLChhPUZwKGEpKS5hPW4sKGk9dS56K2wtby56LWYrdCh1Ll8sby5f
KSk+MCYmKHFwKFVwKHUsbixyKSxuLGkpLGYrPWkscys9aSksbCs9dS5tLGYrPW8ubSxoKz1jLm0s
cys9YS5tO3UmJiFGcChhKSYmKGEudD11LGEubSs9bC1zKSxvJiYhUnAoYykmJihjLnQ9byxjLm0r
PWYtaCxyPW4pfXJldHVybiByfShuLGksbi5wYXJlbnQuQXx8clswXSl9ZnVuY3Rpb24gYSh0KXt0
Ll8ueD10LnordC5wYXJlbnQubSx0Lm0rPXQucGFyZW50Lm19ZnVuY3Rpb24gdSh0KXt0LngqPW4s
dC55PXQuZGVwdGgqZX1yZXR1cm4gaS5zZXBhcmF0aW9uPWZ1bmN0aW9uKG4pe3JldHVybiBhcmd1
bWVudHMubGVuZ3RoPyh0PW4saSk6dH0saS5zaXplPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVu
dHMubGVuZ3RoPyhyPSExLG49K3RbMF0sZT0rdFsxXSxpKTpyP251bGw6W24sZV19LGkubm9kZVNp
emU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9ITAsbj0rdFswXSxlPSt0
WzFdLGkpOnI/W24sZV06bnVsbH0saX0sdC50cmVlbWFwPWZ1bmN0aW9uKCl7dmFyIHQ9THAsbj0h
MSxlPTEscj0xLGk9WzBdLG89ZXAsYT1lcCx1PWVwLGM9ZXAsZj1lcDtmdW5jdGlvbiBzKHQpe3Jl
dHVybiB0LngwPXQueTA9MCx0LngxPWUsdC55MT1yLHQuZWFjaEJlZm9yZShsKSxpPVswXSxuJiZ0
LmVhY2hCZWZvcmUoQXApLHR9ZnVuY3Rpb24gbChuKXt2YXIgZT1pW24uZGVwdGhdLHI9bi54MCtl
LHM9bi55MCtlLGw9bi54MS1lLGg9bi55MS1lO2w8ciYmKHI9bD0ocitsKS8yKSxoPHMmJihzPWg9
KHMraCkvMiksbi54MD1yLG4ueTA9cyxuLngxPWwsbi55MT1oLG4uY2hpbGRyZW4mJihlPWlbbi5k
ZXB0aCsxXT1vKG4pLzIscis9ZihuKS1lLHMrPWEobiktZSwobC09dShuKS1lKTxyJiYocj1sPShy
K2wpLzIpLChoLT1jKG4pLWUpPHMmJihzPWg9KHMraCkvMiksdChuLHIscyxsLGgpKX1yZXR1cm4g
cy5yb3VuZD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obj0hIXQscyk6bn0s
cy5zaXplPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlPSt0WzBdLHI9K3Rb
MV0scyk6W2Uscl19LHMudGlsZT1mdW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8o
dD1ucChuKSxzKTp0fSxzLnBhZGRpbmc9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5n
dGg/cy5wYWRkaW5nSW5uZXIodCkucGFkZGluZ091dGVyKHQpOnMucGFkZGluZ0lubmVyKCl9LHMu
cGFkZGluZ0lubmVyPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhvPSJmdW5j
dGlvbiI9PXR5cGVvZiB0P3Q6cnAoK3QpLHMpOm99LHMucGFkZGluZ091dGVyPWZ1bmN0aW9uKHQp
e3JldHVybiBhcmd1bWVudHMubGVuZ3RoP3MucGFkZGluZ1RvcCh0KS5wYWRkaW5nUmlnaHQodCku
cGFkZGluZ0JvdHRvbSh0KS5wYWRkaW5nTGVmdCh0KTpzLnBhZGRpbmdUb3AoKX0scy5wYWRkaW5n
VG9wPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhhPSJmdW5jdGlvbiI9PXR5
cGVvZiB0P3Q6cnAoK3QpLHMpOmF9LHMucGFkZGluZ1JpZ2h0PWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyh1PSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6cnAoK3QpLHMpOnV9LHMu
cGFkZGluZ0JvdHRvbT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYz0iZnVu
Y3Rpb24iPT10eXBlb2YgdD90OnJwKCt0KSxzKTpjfSxzLnBhZGRpbmdMZWZ0PWZ1bmN0aW9uKHQp
e3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhmPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6cnAoK3Qp
LHMpOmZ9LHN9LHQudHJlZW1hcEJpbmFyeT1mdW5jdGlvbih0LG4sZSxyLGkpe3ZhciBvLGEsdT10
LmNoaWxkcmVuLGM9dS5sZW5ndGgsZj1uZXcgQXJyYXkoYysxKTtmb3IoZlswXT1hPW89MDtvPGM7
KytvKWZbbysxXT1hKz11W29dLnZhbHVlOyFmdW5jdGlvbiB0KG4sZSxyLGksbyxhLGMpe2lmKG4+
PWUtMSl7dmFyIHM9dVtuXTtyZXR1cm4gcy54MD1pLHMueTA9byxzLngxPWEsdm9pZChzLnkxPWMp
fXZhciBsPWZbbl0saD1yLzIrbCxkPW4rMSxwPWUtMTtmb3IoO2Q8cDspe3ZhciBnPWQrcD4+PjE7
ZltnXTxoP2Q9ZysxOnA9Z31oLWZbZC0xXTxmW2RdLWgmJm4rMTxkJiYtLWQ7dmFyIHk9ZltkXS1s
LHY9ci15O2lmKGEtaT5jLW8pe3ZhciBfPXI/KGkqdithKnkpL3I6YTt0KG4sZCx5LGksbyxfLGMp
LHQoZCxlLHYsXyxvLGEsYyl9ZWxzZXt2YXIgYj1yPyhvKnYrYyp5KS9yOmM7dChuLGQseSxpLG8s
YSxiKSx0KGQsZSx2LGksYixhLGMpfX0oMCxjLHQudmFsdWUsbixlLHIsaSl9LHQudHJlZW1hcERp
Y2U9U3AsdC50cmVlbWFwUmVzcXVhcmlmeT1qcCx0LnRyZWVtYXBTbGljZT1PcCx0LnRyZWVtYXBT
bGljZURpY2U9ZnVuY3Rpb24odCxuLGUscixpKXsoMSZ0LmRlcHRoP09wOlNwKSh0LG4sZSxyLGkp
fSx0LnRyZWVtYXBTcXVhcmlmeT1McCx0LnRzdj1UYyx0LnRzdkZvcm1hdD1oYyx0LnRzdkZvcm1h
dEJvZHk9ZGMsdC50c3ZGb3JtYXRSb3c9Z2MsdC50c3ZGb3JtYXRSb3dzPXBjLHQudHN2Rm9ybWF0
VmFsdWU9eWMsdC50c3ZQYXJzZT1zYyx0LnRzdlBhcnNlUm93cz1sYyx0LnVuaW9uPWZ1bmN0aW9u
KC4uLnQpe2NvbnN0IG49bmV3IEludGVyblNldDtmb3IoY29uc3QgZSBvZiB0KWZvcihjb25zdCB0
IG9mIGUpbi5hZGQodCk7cmV0dXJuIG59LHQudW5peERheT1ieSx0LnVuaXhEYXlzPW15LHQudXRj
RGF5PXZ5LHQudXRjRGF5cz1feSx0LnV0Y0ZyaWRheT1ZeSx0LnV0Y0ZyaWRheXM9V3ksdC51dGNI
b3VyPWR5LHQudXRjSG91cnM9cHksdC51dGNNaWxsaXNlY29uZD1aZyx0LnV0Y01pbGxpc2Vjb25k
cz1LZyx0LnV0Y01pbnV0ZT1meSx0LnV0Y01pbnV0ZXM9c3ksdC51dGNNb25kYXk9VXksdC51dGNN
b25kYXlzPUh5LHQudXRjTW9udGg9SnksdC51dGNNb250aHM9dHYsdC51dGNTYXR1cmRheT1MeSx0
LnV0Y1NhdHVyZGF5cz1aeSx0LnV0Y1NlY29uZD1veSx0LnV0Y1NlY29uZHM9YXksdC51dGNTdW5k
YXk9cXksdC51dGNTdW5kYXlzPWp5LHQudXRjVGh1cnNkYXk9QnksdC51dGNUaHVyc2RheXM9Vnks
dC51dGNUaWNrSW50ZXJ2YWw9dXYsdC51dGNUaWNrcz1hdix0LnV0Y1R1ZXNkYXk9SXksdC51dGNU
dWVzZGF5cz1YeSx0LnV0Y1dlZG5lc2RheT1PeSx0LnV0Y1dlZG5lc2RheXM9R3ksdC51dGNXZWVr
PXF5LHQudXRjV2Vla3M9anksdC51dGNZZWFyPXJ2LHQudXRjWWVhcnM9aXYsdC52YXJpYW5jZT13
LHQudmVyc2lvbj0iNy44LjQiLHQud2luZG93PWduLHQueG1sPUVjLHQuemlwPWZ1bmN0aW9uKCl7
cmV0dXJuIHl0KGFyZ3VtZW50cyl9LHQuem9vbT1mdW5jdGlvbigpe3ZhciB0LG4sZSxyPUV3LGk9
Tncsbz16dyxhPUN3LHU9UHcsYz1bMCwxLzBdLGY9W1stMS8wLC0xLzBdLFsxLzAsMS8wXV0scz0y
NTAsbD1paSxoPUR0KCJzdGFydCIsInpvb20iLCJlbmQiKSxkPTUwMCxwPTE1MCxnPTAseT0xMDtm
dW5jdGlvbiB2KHQpe3QucHJvcGVydHkoIl9fem9vbSIsa3cpLm9uKCJ3aGVlbC56b29tIixULHtw
YXNzaXZlOiExfSkub24oIm1vdXNlZG93bi56b29tIixBKS5vbigiZGJsY2xpY2suem9vbSIsUyku
ZmlsdGVyKHUpLm9uKCJ0b3VjaHN0YXJ0Lnpvb20iLEUpLm9uKCJ0b3VjaG1vdmUuem9vbSIsTiku
b24oInRvdWNoZW5kLnpvb20gdG91Y2hjYW5jZWwuem9vbSIsaykuc3R5bGUoIi13ZWJraXQtdGFw
LWhpZ2hsaWdodC1jb2xvciIsInJnYmEoMCwwLDAsMCkiKX1mdW5jdGlvbiBfKHQsbil7cmV0dXJu
KG49TWF0aC5tYXgoY1swXSxNYXRoLm1pbihjWzFdLG4pKSk9PT10Lms/dDpuZXcgd3cobix0Lngs
dC55KX1mdW5jdGlvbiBiKHQsbixlKXt2YXIgcj1uWzBdLWVbMF0qdC5rLGk9blsxXS1lWzFdKnQu
aztyZXR1cm4gcj09PXQueCYmaT09PXQueT90Om5ldyB3dyh0LmsscixpKX1mdW5jdGlvbiBtKHQp
e3JldHVyblsoK3RbMF1bMF0rICt0WzFdWzBdKS8yLCgrdFswXVsxXSsgK3RbMV1bMV0pLzJdfWZ1
bmN0aW9uIHgodCxuLGUscil7dC5vbigic3RhcnQuem9vbSIsKGZ1bmN0aW9uKCl7dyh0aGlzLGFy
Z3VtZW50cykuZXZlbnQocikuc3RhcnQoKX0pKS5vbigiaW50ZXJydXB0Lnpvb20gZW5kLnpvb20i
LChmdW5jdGlvbigpe3codGhpcyxhcmd1bWVudHMpLmV2ZW50KHIpLmVuZCgpfSkpLnR3ZWVuKCJ6
b29tIiwoZnVuY3Rpb24oKXt2YXIgdD10aGlzLG89YXJndW1lbnRzLGE9dyh0LG8pLmV2ZW50KHIp
LHU9aS5hcHBseSh0LG8pLGM9bnVsbD09ZT9tKHUpOiJmdW5jdGlvbiI9PXR5cGVvZiBlP2UuYXBw
bHkodCxvKTplLGY9TWF0aC5tYXgodVsxXVswXS11WzBdWzBdLHVbMV1bMV0tdVswXVsxXSkscz10
Ll9fem9vbSxoPSJmdW5jdGlvbiI9PXR5cGVvZiBuP24uYXBwbHkodCxvKTpuLGQ9bChzLmludmVy
dChjKS5jb25jYXQoZi9zLmspLGguaW52ZXJ0KGMpLmNvbmNhdChmL2guaykpO3JldHVybiBmdW5j
dGlvbih0KXtpZigxPT09dCl0PWg7ZWxzZXt2YXIgbj1kKHQpLGU9Zi9uWzJdO3Q9bmV3IHd3KGUs
Y1swXS1uWzBdKmUsY1sxXS1uWzFdKmUpfWEuem9vbShudWxsLHQpfX0pKX1mdW5jdGlvbiB3KHQs
bixlKXtyZXR1cm4hZSYmdC5fX3pvb21pbmd8fG5ldyBNKHQsbil9ZnVuY3Rpb24gTSh0LG4pe3Ro
aXMudGhhdD10LHRoaXMuYXJncz1uLHRoaXMuYWN0aXZlPTAsdGhpcy5zb3VyY2VFdmVudD1udWxs
LHRoaXMuZXh0ZW50PWkuYXBwbHkodCxuKSx0aGlzLnRhcHM9MH1mdW5jdGlvbiBUKHQsLi4ubil7
aWYoci5hcHBseSh0aGlzLGFyZ3VtZW50cykpe3ZhciBlPXcodGhpcyxuKS5ldmVudCh0KSxpPXRo
aXMuX196b29tLHU9TWF0aC5tYXgoY1swXSxNYXRoLm1pbihjWzFdLGkuaypNYXRoLnBvdygyLGEu
YXBwbHkodGhpcyxhcmd1bWVudHMpKSkpLHM9ZWUodCk7aWYoZS53aGVlbCllLm1vdXNlWzBdWzBd
PT09c1swXSYmZS5tb3VzZVswXVsxXT09PXNbMV18fChlLm1vdXNlWzFdPWkuaW52ZXJ0KGUubW91
c2VbMF09cykpLGNsZWFyVGltZW91dChlLndoZWVsKTtlbHNle2lmKGkuaz09PXUpcmV0dXJuO2Uu
bW91c2U9W3MsaS5pbnZlcnQocyldLFZpKHRoaXMpLGUuc3RhcnQoKX1Tdyh0KSxlLndoZWVsPXNl
dFRpbWVvdXQoKGZ1bmN0aW9uKCl7ZS53aGVlbD1udWxsLGUuZW5kKCl9KSxwKSxlLnpvb20oIm1v
dXNlIixvKGIoXyhpLHUpLGUubW91c2VbMF0sZS5tb3VzZVsxXSksZS5leHRlbnQsZikpfX1mdW5j
dGlvbiBBKHQsLi4ubil7aWYoIWUmJnIuYXBwbHkodGhpcyxhcmd1bWVudHMpKXt2YXIgaT10LmN1
cnJlbnRUYXJnZXQsYT13KHRoaXMsbiwhMCkuZXZlbnQodCksdT1Lbih0LnZpZXcpLm9uKCJtb3Vz
ZW1vdmUuem9vbSIsKGZ1bmN0aW9uKHQpe2lmKFN3KHQpLCFhLm1vdmVkKXt2YXIgbj10LmNsaWVu
dFgtcyxlPXQuY2xpZW50WS1sO2EubW92ZWQ9bipuK2UqZT5nfWEuZXZlbnQodCkuem9vbSgibW91
c2UiLG8oYihhLnRoYXQuX196b29tLGEubW91c2VbMF09ZWUodCxpKSxhLm1vdXNlWzFdKSxhLmV4
dGVudCxmKSl9KSwhMCkub24oIm1vdXNldXAuem9vbSIsKGZ1bmN0aW9uKHQpe3Uub24oIm1vdXNl
bW92ZS56b29tIG1vdXNldXAuem9vbSIsbnVsbCksY2UodC52aWV3LGEubW92ZWQpLFN3KHQpLGEu
ZXZlbnQodCkuZW5kKCl9KSwhMCksYz1lZSh0LGkpLHM9dC5jbGllbnRYLGw9dC5jbGllbnRZO3Vl
KHQudmlldyksQXcodCksYS5tb3VzZT1bYyx0aGlzLl9fem9vbS5pbnZlcnQoYyldLFZpKHRoaXMp
LGEuc3RhcnQoKX19ZnVuY3Rpb24gUyh0LC4uLm4pe2lmKHIuYXBwbHkodGhpcyxhcmd1bWVudHMp
KXt2YXIgZT10aGlzLl9fem9vbSxhPWVlKHQuY2hhbmdlZFRvdWNoZXM/dC5jaGFuZ2VkVG91Y2hl
c1swXTp0LHRoaXMpLHU9ZS5pbnZlcnQoYSksYz1lLmsqKHQuc2hpZnRLZXk/LjU6MiksbD1vKGIo
XyhlLGMpLGEsdSksaS5hcHBseSh0aGlzLG4pLGYpO1N3KHQpLHM+MD9Lbih0aGlzKS50cmFuc2l0
aW9uKCkuZHVyYXRpb24ocykuY2FsbCh4LGwsYSx0KTpLbih0aGlzKS5jYWxsKHYudHJhbnNmb3Jt
LGwsYSx0KX19ZnVuY3Rpb24gRShlLC4uLmkpe2lmKHIuYXBwbHkodGhpcyxhcmd1bWVudHMpKXt2
YXIgbyxhLHUsYyxmPWUudG91Y2hlcyxzPWYubGVuZ3RoLGw9dyh0aGlzLGksZS5jaGFuZ2VkVG91
Y2hlcy5sZW5ndGg9PT1zKS5ldmVudChlKTtmb3IoQXcoZSksYT0wO2E8czsrK2EpYz1bYz1lZSh1
PWZbYV0sdGhpcyksdGhpcy5fX3pvb20uaW52ZXJ0KGMpLHUuaWRlbnRpZmllcl0sbC50b3VjaDA/
bC50b3VjaDF8fGwudG91Y2gwWzJdPT09Y1syXXx8KGwudG91Y2gxPWMsbC50YXBzPTApOihsLnRv
dWNoMD1jLG89ITAsbC50YXBzPTErISF0KTt0JiYodD1jbGVhclRpbWVvdXQodCkpLG8mJihsLnRh
cHM8MiYmKG49Y1swXSx0PXNldFRpbWVvdXQoKGZ1bmN0aW9uKCl7dD1udWxsfSksZCkpLFZpKHRo
aXMpLGwuc3RhcnQoKSl9fWZ1bmN0aW9uIE4odCwuLi5uKXtpZih0aGlzLl9fem9vbWluZyl7dmFy
IGUscixpLGEsdT13KHRoaXMsbikuZXZlbnQodCksYz10LmNoYW5nZWRUb3VjaGVzLHM9Yy5sZW5n
dGg7Zm9yKFN3KHQpLGU9MDtlPHM7KytlKWk9ZWUocj1jW2VdLHRoaXMpLHUudG91Y2gwJiZ1LnRv
dWNoMFsyXT09PXIuaWRlbnRpZmllcj91LnRvdWNoMFswXT1pOnUudG91Y2gxJiZ1LnRvdWNoMVsy
XT09PXIuaWRlbnRpZmllciYmKHUudG91Y2gxWzBdPWkpO2lmKHI9dS50aGF0Ll9fem9vbSx1LnRv
dWNoMSl7dmFyIGw9dS50b3VjaDBbMF0saD11LnRvdWNoMFsxXSxkPXUudG91Y2gxWzBdLHA9dS50
b3VjaDFbMV0sZz0oZz1kWzBdLWxbMF0pKmcrKGc9ZFsxXS1sWzFdKSpnLHk9KHk9cFswXS1oWzBd
KSp5Kyh5PXBbMV0taFsxXSkqeTtyPV8ocixNYXRoLnNxcnQoZy95KSksaT1bKGxbMF0rZFswXSkv
MiwobFsxXStkWzFdKS8yXSxhPVsoaFswXStwWzBdKS8yLChoWzFdK3BbMV0pLzJdfWVsc2V7aWYo
IXUudG91Y2gwKXJldHVybjtpPXUudG91Y2gwWzBdLGE9dS50b3VjaDBbMV19dS56b29tKCJ0b3Vj
aCIsbyhiKHIsaSxhKSx1LmV4dGVudCxmKSl9fWZ1bmN0aW9uIGsodCwuLi5yKXtpZih0aGlzLl9f
em9vbWluZyl7dmFyIGksbyxhPXcodGhpcyxyKS5ldmVudCh0KSx1PXQuY2hhbmdlZFRvdWNoZXMs
Yz11Lmxlbmd0aDtmb3IoQXcodCksZSYmY2xlYXJUaW1lb3V0KGUpLGU9c2V0VGltZW91dCgoZnVu
Y3Rpb24oKXtlPW51bGx9KSxkKSxpPTA7aTxjOysraSlvPXVbaV0sYS50b3VjaDAmJmEudG91Y2gw
WzJdPT09by5pZGVudGlmaWVyP2RlbGV0ZSBhLnRvdWNoMDphLnRvdWNoMSYmYS50b3VjaDFbMl09
PT1vLmlkZW50aWZpZXImJmRlbGV0ZSBhLnRvdWNoMTtpZihhLnRvdWNoMSYmIWEudG91Y2gwJiYo
YS50b3VjaDA9YS50b3VjaDEsZGVsZXRlIGEudG91Y2gxKSxhLnRvdWNoMClhLnRvdWNoMFsxXT10
aGlzLl9fem9vbS5pbnZlcnQoYS50b3VjaDBbMF0pO2Vsc2UgaWYoYS5lbmQoKSwyPT09YS50YXBz
JiYobz1lZShvLHRoaXMpLE1hdGguaHlwb3QoblswXS1vWzBdLG5bMV0tb1sxXSk8eSkpe3ZhciBm
PUtuKHRoaXMpLm9uKCJkYmxjbGljay56b29tIik7ZiYmZi5hcHBseSh0aGlzLGFyZ3VtZW50cyl9
fX1yZXR1cm4gdi50cmFuc2Zvcm09ZnVuY3Rpb24odCxuLGUscil7dmFyIGk9dC5zZWxlY3Rpb24/
dC5zZWxlY3Rpb24oKTp0O2kucHJvcGVydHkoIl9fem9vbSIsa3cpLHQhPT1pP3godCxuLGUscik6
aS5pbnRlcnJ1cHQoKS5lYWNoKChmdW5jdGlvbigpe3codGhpcyxhcmd1bWVudHMpLmV2ZW50KHIp
LnN0YXJ0KCkuem9vbShudWxsLCJmdW5jdGlvbiI9PXR5cGVvZiBuP24uYXBwbHkodGhpcyxhcmd1
bWVudHMpOm4pLmVuZCgpfSkpfSx2LnNjYWxlQnk9ZnVuY3Rpb24odCxuLGUscil7di5zY2FsZVRv
KHQsKGZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuX196b29tLmsqKCJmdW5jdGlvbiI9PXR5cGVvZiBu
P24uYXBwbHkodGhpcyxhcmd1bWVudHMpOm4pfSksZSxyKX0sdi5zY2FsZVRvPWZ1bmN0aW9uKHQs
bixlLHIpe3YudHJhbnNmb3JtKHQsKGZ1bmN0aW9uKCl7dmFyIHQ9aS5hcHBseSh0aGlzLGFyZ3Vt
ZW50cykscj10aGlzLl9fem9vbSxhPW51bGw9PWU/bSh0KToiZnVuY3Rpb24iPT10eXBlb2YgZT9l
LmFwcGx5KHRoaXMsYXJndW1lbnRzKTplLHU9ci5pbnZlcnQoYSksYz0iZnVuY3Rpb24iPT10eXBl
b2Ygbj9uLmFwcGx5KHRoaXMsYXJndW1lbnRzKTpuO3JldHVybiBvKGIoXyhyLGMpLGEsdSksdCxm
KX0pLGUscil9LHYudHJhbnNsYXRlQnk9ZnVuY3Rpb24odCxuLGUscil7di50cmFuc2Zvcm0odCwo
ZnVuY3Rpb24oKXtyZXR1cm4gbyh0aGlzLl9fem9vbS50cmFuc2xhdGUoImZ1bmN0aW9uIj09dHlw
ZW9mIG4/bi5hcHBseSh0aGlzLGFyZ3VtZW50cyk6biwiZnVuY3Rpb24iPT10eXBlb2YgZT9lLmFw
cGx5KHRoaXMsYXJndW1lbnRzKTplKSxpLmFwcGx5KHRoaXMsYXJndW1lbnRzKSxmKX0pLG51bGws
cil9LHYudHJhbnNsYXRlVG89ZnVuY3Rpb24odCxuLGUscixhKXt2LnRyYW5zZm9ybSh0LChmdW5j
dGlvbigpe3ZhciB0PWkuYXBwbHkodGhpcyxhcmd1bWVudHMpLGE9dGhpcy5fX3pvb20sdT1udWxs
PT1yP20odCk6ImZ1bmN0aW9uIj09dHlwZW9mIHI/ci5hcHBseSh0aGlzLGFyZ3VtZW50cyk6cjty
ZXR1cm4gbyhNdy50cmFuc2xhdGUodVswXSx1WzFdKS5zY2FsZShhLmspLnRyYW5zbGF0ZSgiZnVu
Y3Rpb24iPT10eXBlb2Ygbj8tbi5hcHBseSh0aGlzLGFyZ3VtZW50cyk6LW4sImZ1bmN0aW9uIj09
dHlwZW9mIGU/LWUuYXBwbHkodGhpcyxhcmd1bWVudHMpOi1lKSx0LGYpfSkscixhKX0sTS5wcm90
b3R5cGU9e2V2ZW50OmZ1bmN0aW9uKHQpe3JldHVybiB0JiYodGhpcy5zb3VyY2VFdmVudD10KSx0
aGlzfSxzdGFydDpmdW5jdGlvbigpe3JldHVybiAxPT0rK3RoaXMuYWN0aXZlJiYodGhpcy50aGF0
Ll9fem9vbWluZz10aGlzLHRoaXMuZW1pdCgic3RhcnQiKSksdGhpc30sem9vbTpmdW5jdGlvbih0
LG4pe3JldHVybiB0aGlzLm1vdXNlJiYibW91c2UiIT09dCYmKHRoaXMubW91c2VbMV09bi5pbnZl
cnQodGhpcy5tb3VzZVswXSkpLHRoaXMudG91Y2gwJiYidG91Y2giIT09dCYmKHRoaXMudG91Y2gw
WzFdPW4uaW52ZXJ0KHRoaXMudG91Y2gwWzBdKSksdGhpcy50b3VjaDEmJiJ0b3VjaCIhPT10JiYo
dGhpcy50b3VjaDFbMV09bi5pbnZlcnQodGhpcy50b3VjaDFbMF0pKSx0aGlzLnRoYXQuX196b29t
PW4sdGhpcy5lbWl0KCJ6b29tIiksdGhpc30sZW5kOmZ1bmN0aW9uKCl7cmV0dXJuIDA9PS0tdGhp
cy5hY3RpdmUmJihkZWxldGUgdGhpcy50aGF0Ll9fem9vbWluZyx0aGlzLmVtaXQoImVuZCIpKSx0
aGlzfSxlbWl0OmZ1bmN0aW9uKHQpe3ZhciBuPUtuKHRoaXMudGhhdCkuZGF0dW0oKTtoLmNhbGwo
dCx0aGlzLnRoYXQsbmV3IHh3KHQse3NvdXJjZUV2ZW50OnRoaXMuc291cmNlRXZlbnQsdGFyZ2V0
OnYsdHlwZTp0LHRyYW5zZm9ybTp0aGlzLnRoYXQuX196b29tLGRpc3BhdGNoOmh9KSxuKX19LHYu
d2hlZWxEZWx0YT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYT0iZnVuY3Rp
b24iPT10eXBlb2YgdD90Om13KCt0KSx2KTphfSx2LmZpbHRlcj1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8ocj0iZnVuY3Rpb24iPT10eXBlb2YgdD90Om13KCEhdCksdik6cn0s
di50b3VjaGFibGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHU9ImZ1bmN0
aW9uIj09dHlwZW9mIHQ/dDptdyghIXQpLHYpOnV9LHYuZXh0ZW50PWZ1bmN0aW9uKHQpe3JldHVy
biBhcmd1bWVudHMubGVuZ3RoPyhpPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6bXcoW1srdFswXVsw
XSwrdFswXVsxXV0sWyt0WzFdWzBdLCt0WzFdWzFdXV0pLHYpOml9LHYuc2NhbGVFeHRlbnQ9ZnVu
Y3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGNbMF09K3RbMF0sY1sxXT0rdFsxXSx2
KTpbY1swXSxjWzFdXX0sdi50cmFuc2xhdGVFeHRlbnQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3Vt
ZW50cy5sZW5ndGg/KGZbMF1bMF09K3RbMF1bMF0sZlsxXVswXT0rdFsxXVswXSxmWzBdWzFdPSt0
WzBdWzFdLGZbMV1bMV09K3RbMV1bMV0sdik6W1tmWzBdWzBdLGZbMF1bMV1dLFtmWzFdWzBdLGZb
MV1bMV1dXX0sdi5jb25zdHJhaW49ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KG89dCx2KTpvfSx2LmR1cmF0aW9uPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3Ro
PyhzPSt0LHYpOnN9LHYuaW50ZXJwb2xhdGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/KGw9dCx2KTpsfSx2Lm9uPWZ1bmN0aW9uKCl7dmFyIHQ9aC5vbi5hcHBseShoLGFyZ3Vt
ZW50cyk7cmV0dXJuIHQ9PT1oP3Y6dH0sdi5jbGlja0Rpc3RhbmNlPWZ1bmN0aW9uKHQpe3JldHVy
biBhcmd1bWVudHMubGVuZ3RoPyhnPSh0PSt0KSp0LHYpOk1hdGguc3FydChnKX0sdi50YXBEaXN0
YW5jZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oeT0rdCx2KTp5fSx2fSx0
Lnpvb21JZGVudGl0eT1Ndyx0Lnpvb21UcmFuc2Zvcm09VHd9KSk7PC9zY3JpcHQ+PHNjcmlwdD4v
KioqIEBwcmVzZXJ2ZSBDb3B5cmlnaHQgMjAxMiBUd2l0dGVyLCBJbmMuKiBAbGljZW5zZSBodHRw
Oi8vd3d3LmFwYWNoZS5vcmcvbGljZW5zZXMvTElDRU5TRS0yLjAudHh0Ki92YXIgSG9nYW49e307
IWZ1bmN0aW9uKHQpe2Z1bmN0aW9uIG4odCxuLGUpe3ZhciBpO3JldHVybiBuJiYib2JqZWN0Ij09
dHlwZW9mIG4mJih2b2lkIDAhPT1uW3RdP2k9blt0XTplJiZuLmdldCYmImZ1bmN0aW9uIj09dHlw
ZW9mIG4uZ2V0JiYoaT1uLmdldCh0KSkpLGl9ZnVuY3Rpb24gZSh0LG4sZSxpLHIscyl7ZnVuY3Rp
b24gYSgpe31mdW5jdGlvbiBvKCl7fWEucHJvdG90eXBlPXQsby5wcm90b3R5cGU9dC5zdWJzO3Zh
ciB1LGM9bmV3IGE7Yy5zdWJzPW5ldyBvLGMuc3Vic1RleHQ9e30sYy5idWY9IiIsaT1pfHx7fSxj
LnN0YWNrU3Vicz1pLGMuc3Vic1RleHQ9cztmb3IodSBpbiBuKWlbdV18fChpW3VdPW5bdV0pO2Zv
cih1IGluIGkpYy5zdWJzW3VdPWlbdV07cj1yfHx7fSxjLnN0YWNrUGFydGlhbHM9cjtmb3IodSBp
biBlKXJbdV18fChyW3VdPWVbdV0pO2Zvcih1IGluIHIpYy5wYXJ0aWFsc1t1XT1yW3VdO3JldHVy
biBjfWZ1bmN0aW9uIGkodCl7cmV0dXJuIFN0cmluZyhudWxsPT09dHx8dm9pZCAwPT09dD8iIjp0
KX1mdW5jdGlvbiByKHQpe3JldHVybiB0PWkodCksbC50ZXN0KHQpP3QucmVwbGFjZShzLCImYW1w
OyIpLnJlcGxhY2UoYSwiJmx0OyIpLnJlcGxhY2UobywiJmd0OyIpLnJlcGxhY2UodSwiJiMzOTsi
KS5yZXBsYWNlKGMsIiZxdW90OyIpOnR9dC5UZW1wbGF0ZT1mdW5jdGlvbih0LG4sZSxpKXt0PXR8
fHt9LHRoaXMucj10LmNvZGV8fHRoaXMucix0aGlzLmM9ZSx0aGlzLm9wdGlvbnM9aXx8e30sdGhp
cy50ZXh0PW58fCIiLHRoaXMucGFydGlhbHM9dC5wYXJ0aWFsc3x8e30sdGhpcy5zdWJzPXQuc3Vi
c3x8e30sdGhpcy5idWY9IiJ9LHQuVGVtcGxhdGUucHJvdG90eXBlPXtyOmZ1bmN0aW9uKCl7cmV0
dXJuIiJ9LHY6cix0OmkscmVuZGVyOmZ1bmN0aW9uKHQsbixlKXtyZXR1cm4gdGhpcy5yaShbdF0s
bnx8e30sZSl9LHJpOmZ1bmN0aW9uKHQsbixlKXtyZXR1cm4gdGhpcy5yKHQsbixlKX0sZXA6ZnVu
Y3Rpb24odCxuKXt2YXIgaT10aGlzLnBhcnRpYWxzW3RdLHI9bltpLm5hbWVdO2lmKGkuaW5zdGFu
Y2UmJmkuYmFzZT09cilyZXR1cm4gaS5pbnN0YW5jZTtpZigic3RyaW5nIj09dHlwZW9mIHIpe2lm
KCF0aGlzLmMpdGhyb3cgbmV3IEVycm9yKCJObyBjb21waWxlciBhdmFpbGFibGUuIik7cj10aGlz
LmMuY29tcGlsZShyLHRoaXMub3B0aW9ucyl9aWYoIXIpcmV0dXJuIG51bGw7aWYodGhpcy5wYXJ0
aWFsc1t0XS5iYXNlPXIsaS5zdWJzKXtuLnN0YWNrVGV4dHx8KG4uc3RhY2tUZXh0PXt9KTtmb3Io
a2V5IGluIGkuc3VicyluLnN0YWNrVGV4dFtrZXldfHwobi5zdGFja1RleHRba2V5XT12b2lkIDAh
PT10aGlzLmFjdGl2ZVN1YiYmbi5zdGFja1RleHRbdGhpcy5hY3RpdmVTdWJdP24uc3RhY2tUZXh0
W3RoaXMuYWN0aXZlU3ViXTp0aGlzLnRleHQpO3I9ZShyLGkuc3VicyxpLnBhcnRpYWxzLHRoaXMu
c3RhY2tTdWJzLHRoaXMuc3RhY2tQYXJ0aWFscyxuLnN0YWNrVGV4dCl9cmV0dXJuIHRoaXMucGFy
dGlhbHNbdF0uaW5zdGFuY2U9cixyfSxycDpmdW5jdGlvbih0LG4sZSxpKXt2YXIgcj10aGlzLmVw
KHQsZSk7cmV0dXJuIHI/ci5yaShuLGUsaSk6IiJ9LHJzOmZ1bmN0aW9uKHQsbixlKXt2YXIgaT10
W3QubGVuZ3RoLTFdO2lmKCFmKGkpKXJldHVybiB2b2lkIGUodCxuLHRoaXMpO2Zvcih2YXIgcj0w
O3I8aS5sZW5ndGg7cisrKXQucHVzaChpW3JdKSxlKHQsbix0aGlzKSx0LnBvcCgpfSxzOmZ1bmN0
aW9uKHQsbixlLGkscixzLGEpe3ZhciBvO3JldHVybiBmKHQpJiYwPT09dC5sZW5ndGg/ITE6KCJm
dW5jdGlvbiI9PXR5cGVvZiB0JiYodD10aGlzLm1zKHQsbixlLGkscixzLGEpKSxvPSEhdCwhaSYm
byYmbiYmbi5wdXNoKCJvYmplY3QiPT10eXBlb2YgdD90Om5bbi5sZW5ndGgtMV0pLG8pfSxkOmZ1
bmN0aW9uKHQsZSxpLHIpe3ZhciBzLGE9dC5zcGxpdCgiLiIpLG89dGhpcy5mKGFbMF0sZSxpLHIp
LHU9dGhpcy5vcHRpb25zLm1vZGVsR2V0LGM9bnVsbDtpZigiLiI9PT10JiZmKGVbZS5sZW5ndGgt
Ml0pKW89ZVtlLmxlbmd0aC0xXTtlbHNlIGZvcih2YXIgbD0xO2w8YS5sZW5ndGg7bCsrKXM9bihh
W2xdLG8sdSksdm9pZCAwIT09cz8oYz1vLG89cyk6bz0iIjtyZXR1cm4gciYmIW8/ITE6KHJ8fCJm
dW5jdGlvbiIhPXR5cGVvZiBvfHwoZS5wdXNoKGMpLG89dGhpcy5tdihvLGUsaSksZS5wb3AoKSks
byl9LGY6ZnVuY3Rpb24odCxlLGkscil7Zm9yKHZhciBzPSExLGE9bnVsbCxvPSExLHU9dGhpcy5v
cHRpb25zLm1vZGVsR2V0LGM9ZS5sZW5ndGgtMTtjPj0wO2MtLSlpZihhPWVbY10scz1uKHQsYSx1
KSx2b2lkIDAhPT1zKXtvPSEwO2JyZWFrfXJldHVybiBvPyhyfHwiZnVuY3Rpb24iIT10eXBlb2Yg
c3x8KHM9dGhpcy5tdihzLGUsaSkpLHMpOnI/ITE6IiJ9LGxzOmZ1bmN0aW9uKHQsbixlLHIscyl7
dmFyIGE9dGhpcy5vcHRpb25zLmRlbGltaXRlcnM7cmV0dXJuIHRoaXMub3B0aW9ucy5kZWxpbWl0
ZXJzPXMsdGhpcy5iKHRoaXMuY3QoaSh0LmNhbGwobixyKSksbixlKSksdGhpcy5vcHRpb25zLmRl
bGltaXRlcnM9YSwhMX0sY3Q6ZnVuY3Rpb24odCxuLGUpe2lmKHRoaXMub3B0aW9ucy5kaXNhYmxl
TGFtYmRhKXRocm93IG5ldyBFcnJvcigiTGFtYmRhIGZlYXR1cmVzIGRpc2FibGVkLiIpO3JldHVy
biB0aGlzLmMuY29tcGlsZSh0LHRoaXMub3B0aW9ucykucmVuZGVyKG4sZSl9LGI6ZnVuY3Rpb24o
dCl7dGhpcy5idWYrPXR9LGZsOmZ1bmN0aW9uKCl7dmFyIHQ9dGhpcy5idWY7cmV0dXJuIHRoaXMu
YnVmPSIiLHR9LG1zOmZ1bmN0aW9uKHQsbixlLGkscixzLGEpe3ZhciBvLHU9bltuLmxlbmd0aC0x
XSxjPXQuY2FsbCh1KTtyZXR1cm4iZnVuY3Rpb24iPT10eXBlb2YgYz9pPyEwOihvPXRoaXMuYWN0
aXZlU3ViJiZ0aGlzLnN1YnNUZXh0JiZ0aGlzLnN1YnNUZXh0W3RoaXMuYWN0aXZlU3ViXT90aGlz
LnN1YnNUZXh0W3RoaXMuYWN0aXZlU3ViXTp0aGlzLnRleHQsdGhpcy5scyhjLHUsZSxvLnN1YnN0
cmluZyhyLHMpLGEpKTpjfSxtdjpmdW5jdGlvbih0LG4sZSl7dmFyIHI9bltuLmxlbmd0aC0xXSxz
PXQuY2FsbChyKTtyZXR1cm4iZnVuY3Rpb24iPT10eXBlb2Ygcz90aGlzLmN0KGkocy5jYWxsKHIp
KSxyLGUpOnN9LHN1YjpmdW5jdGlvbih0LG4sZSxpKXt2YXIgcj10aGlzLnN1YnNbdF07ciYmKHRo
aXMuYWN0aXZlU3ViPXQscihuLGUsdGhpcyxpKSx0aGlzLmFjdGl2ZVN1Yj0hMSl9fTt2YXIgcz0v
Ji9nLGE9LzwvZyxvPS8+L2csdT0vXCcvZyxjPS9cIi9nLGw9L1smPD5cIlwnXS8sZj1BcnJheS5p
c0FycmF5fHxmdW5jdGlvbih0KXtyZXR1cm4iW29iamVjdCBBcnJheV0iPT09T2JqZWN0LnByb3Rv
dHlwZS50b1N0cmluZy5jYWxsKHQpfX0oInVuZGVmaW5lZCIhPXR5cGVvZiBleHBvcnRzP2V4cG9y
dHM6SG9nYW4pLGZ1bmN0aW9uKHQpe2Z1bmN0aW9uIG4odCl7In0iPT09dC5uLnN1YnN0cih0Lm4u
bGVuZ3RoLTEpJiYodC5uPXQubi5zdWJzdHJpbmcoMCx0Lm4ubGVuZ3RoLTEpKX1mdW5jdGlvbiBl
KHQpe3JldHVybiB0LnRyaW0/dC50cmltKCk6dC5yZXBsYWNlKC9eXHMqfFxzKiQvZywiIil9ZnVu
Y3Rpb24gaSh0LG4sZSl7aWYobi5jaGFyQXQoZSkhPXQuY2hhckF0KDApKXJldHVybiExO2Zvcih2
YXIgaT0xLHI9dC5sZW5ndGg7cj5pO2krKylpZihuLmNoYXJBdChlK2kpIT10LmNoYXJBdChpKSly
ZXR1cm4hMTtyZXR1cm4hMH1mdW5jdGlvbiByKG4sZSxpLG8pe3ZhciB1PVtdLGM9bnVsbCxsPW51
bGwsZj1udWxsO2ZvcihsPWlbaS5sZW5ndGgtMV07bi5sZW5ndGg+MDspe2lmKGY9bi5zaGlmdCgp
LGwmJiI8Ij09bC50YWcmJiEoZi50YWcgaW4gaykpdGhyb3cgbmV3IEVycm9yKCJJbGxlZ2FsIGNv
bnRlbnQgaW4gPCBzdXBlciB0YWcuIik7aWYodC50YWdzW2YudGFnXTw9dC50YWdzLiR8fHMoZixv
KSlpLnB1c2goZiksZi5ub2Rlcz1yKG4sZi50YWcsaSxvKTtlbHNle2lmKCIvIj09Zi50YWcpe2lm
KDA9PT1pLmxlbmd0aCl0aHJvdyBuZXcgRXJyb3IoIkNsb3NpbmcgdGFnIHdpdGhvdXQgb3BlbmVy
OiAvIitmLm4pO2lmKGM9aS5wb3AoKSxmLm4hPWMubiYmIWEoZi5uLGMubixvKSl0aHJvdyBuZXcg
RXJyb3IoIk5lc3RpbmcgZXJyb3I6ICIrYy5uKyIgdnMuICIrZi5uKTtyZXR1cm4gYy5lbmQ9Zi5p
LHV9IlxuIj09Zi50YWcmJihmLmxhc3Q9MD09bi5sZW5ndGh8fCJcbiI9PW5bMF0udGFnKX11LnB1
c2goZil9aWYoaS5sZW5ndGg+MCl0aHJvdyBuZXcgRXJyb3IoIm1pc3NpbmcgY2xvc2luZyB0YWc6
ICIraS5wb3AoKS5uKTtyZXR1cm4gdX1mdW5jdGlvbiBzKHQsbil7Zm9yKHZhciBlPTAsaT1uLmxl
bmd0aDtpPmU7ZSsrKWlmKG5bZV0ubz09dC5uKXJldHVybiB0LnRhZz0iIyIsITB9ZnVuY3Rpb24g
YSh0LG4sZSl7Zm9yKHZhciBpPTAscj1lLmxlbmd0aDtyPmk7aSsrKWlmKGVbaV0uYz09dCYmZVtp
XS5vPT1uKXJldHVybiEwfWZ1bmN0aW9uIG8odCl7dmFyIG49W107Zm9yKHZhciBlIGluIHQpbi5w
dXNoKCciJytjKGUpKyciOiBmdW5jdGlvbihjLHAsdCxpKSB7Jyt0W2VdKyJ9Iik7cmV0dXJuInsg
IituLmpvaW4oIiwiKSsiIH0ifWZ1bmN0aW9uIHUodCl7dmFyIG49W107Zm9yKHZhciBlIGluIHQu
cGFydGlhbHMpbi5wdXNoKCciJytjKGUpKyciOntuYW1lOiInK2ModC5wYXJ0aWFsc1tlXS5uYW1l
KSsnIiwgJyt1KHQucGFydGlhbHNbZV0pKyJ9Iik7cmV0dXJuInBhcnRpYWxzOiB7IituLmpvaW4o
IiwiKSsifSwgc3ViczogIitvKHQuc3Vicyl9ZnVuY3Rpb24gYyh0KXtyZXR1cm4gdC5yZXBsYWNl
KG0sIlxcXFwiKS5yZXBsYWNlKHYsJ1xcIicpLnJlcGxhY2UoYiwiXFxuIikucmVwbGFjZShkLCJc
XHIiKS5yZXBsYWNlKHgsIlxcdTIwMjgiKS5yZXBsYWNlKHcsIlxcdTIwMjkiKX1mdW5jdGlvbiBs
KHQpe3JldHVybn50LmluZGV4T2YoIi4iKT8iZCI6ImYifWZ1bmN0aW9uIGYodCxuKXt2YXIgZT0i
PCIrKG4ucHJlZml4fHwiIiksaT1lK3Qubit5Kys7cmV0dXJuIG4ucGFydGlhbHNbaV09e25hbWU6
dC5uLHBhcnRpYWxzOnt9fSxuLmNvZGUrPSd0LmIodC5ycCgiJytjKGkpKyciLGMscCwiJysodC5p
bmRlbnR8fCIiKSsnIikpOycsaX1mdW5jdGlvbiBoKHQsbil7bi5jb2RlKz0idC5iKHQudCh0LiIr
bCh0Lm4pKycoIicrYyh0Lm4pKyciLGMscCwwKSkpOyd9ZnVuY3Rpb24gcCh0KXtyZXR1cm4idC5i
KCIrdCsiKTsifXZhciBnPS9cUy8sdj0vXCIvZyxiPS9cbi9nLGQ9L1xyL2csbT0vXFwvZyx4PS9c
dTIwMjgvLHc9L1x1MjAyOS87dC50YWdzPXsiIyI6MSwiXiI6MiwiPCI6MywkOjQsIi8iOjUsIiEi
OjYsIj4iOjcsIj0iOjgsX3Y6OSwieyI6MTAsIiYiOjExLF90OjEyfSx0LnNjYW49ZnVuY3Rpb24o
cixzKXtmdW5jdGlvbiBhKCl7bS5sZW5ndGg+MCYmKHgucHVzaCh7dGFnOiJfdCIsdGV4dDpuZXcg
U3RyaW5nKG0pfSksbT0iIil9ZnVuY3Rpb24gbygpe2Zvcih2YXIgbj0hMCxlPXk7ZTx4Lmxlbmd0
aDtlKyspaWYobj10LnRhZ3NbeFtlXS50YWddPHQudGFncy5fdnx8Il90Ij09eFtlXS50YWcmJm51
bGw9PT14W2VdLnRleHQubWF0Y2goZyksIW4pcmV0dXJuITE7cmV0dXJuIG59ZnVuY3Rpb24gdSh0
LG4pe2lmKGEoKSx0JiZvKCkpZm9yKHZhciBlLGk9eTtpPHgubGVuZ3RoO2krKyl4W2ldLnRleHQm
JigoZT14W2krMV0pJiYiPiI9PWUudGFnJiYoZS5pbmRlbnQ9eFtpXS50ZXh0LnRvU3RyaW5nKCkp
LHguc3BsaWNlKGksMSkpO2Vsc2Ugbnx8eC5wdXNoKHt0YWc6IlxuIn0pO3c9ITEseT14Lmxlbmd0
aH1mdW5jdGlvbiBjKHQsbil7dmFyIGk9Ij0iK1Mscj10LmluZGV4T2YoaSxuKSxzPWUodC5zdWJz
dHJpbmcodC5pbmRleE9mKCI9IixuKSsxLHIpKS5zcGxpdCgiICIpO3JldHVybiBUPXNbMF0sUz1z
W3MubGVuZ3RoLTFdLHIraS5sZW5ndGgtMX12YXIgbD1yLmxlbmd0aCxmPTAsaD0xLHA9Mix2PWYs
Yj1udWxsLGQ9bnVsbCxtPSIiLHg9W10sdz0hMSxrPTAseT0wLFQ9Int7IixTPSJ9fSI7Zm9yKHMm
JihzPXMuc3BsaXQoIiAiKSxUPXNbMF0sUz1zWzFdKSxrPTA7bD5rO2srKyl2PT1mP2koVCxyLGsp
PygtLWssYSgpLHY9aCk6IlxuIj09ci5jaGFyQXQoayk/dSh3KTptKz1yLmNoYXJBdChrKTp2PT1o
PyhrKz1ULmxlbmd0aC0xLGQ9dC50YWdzW3IuY2hhckF0KGsrMSldLGI9ZD9yLmNoYXJBdChrKzEp
OiJfdiIsIj0iPT1iPyhrPWMocixrKSx2PWYpOihkJiZrKyssdj1wKSx3PWspOmkoUyxyLGspPyh4
LnB1c2goe3RhZzpiLG46ZShtKSxvdGFnOlQsY3RhZzpTLGk6Ii8iPT1iP3ctVC5sZW5ndGg6aytT
Lmxlbmd0aH0pLG09IiIsays9Uy5sZW5ndGgtMSx2PWYsInsiPT1iJiYoIn19Ij09Uz9rKys6bih4
W3gubGVuZ3RoLTFdKSkpOm0rPXIuY2hhckF0KGspO3JldHVybiB1KHcsITApLHh9O3ZhciBrPXtf
dDohMCwiXG4iOiEwLCQ6ITAsIi8iOiEwfTt0LnN0cmluZ2lmeT1mdW5jdGlvbihuKXtyZXR1cm4i
e2NvZGU6IGZ1bmN0aW9uIChjLHAsaSkgeyAiK3Qud3JhcE1haW4obi5jb2RlKSsiIH0sIit1KG4p
KyJ9In07dmFyIHk9MDt0LmdlbmVyYXRlPWZ1bmN0aW9uKG4sZSxpKXt5PTA7dmFyIHI9e2NvZGU6
IiIsc3Viczp7fSxwYXJ0aWFsczp7fX07cmV0dXJuIHQud2FsayhuLHIpLGkuYXNTdHJpbmc/dGhp
cy5zdHJpbmdpZnkocixlLGkpOnRoaXMubWFrZVRlbXBsYXRlKHIsZSxpKX0sdC53cmFwTWFpbj1m
dW5jdGlvbih0KXtyZXR1cm4ndmFyIHQ9dGhpczt0LmIoaT1pfHwiIik7Jyt0KyJyZXR1cm4gdC5m
bCgpOyJ9LHQudGVtcGxhdGU9dC5UZW1wbGF0ZSx0Lm1ha2VUZW1wbGF0ZT1mdW5jdGlvbih0LG4s
ZSl7dmFyIGk9dGhpcy5tYWtlUGFydGlhbHModCk7cmV0dXJuIGkuY29kZT1uZXcgRnVuY3Rpb24o
ImMiLCJwIiwiaSIsdGhpcy53cmFwTWFpbih0LmNvZGUpKSxuZXcgdGhpcy50ZW1wbGF0ZShpLG4s
dGhpcyxlKX0sdC5tYWtlUGFydGlhbHM9ZnVuY3Rpb24odCl7dmFyIG4sZT17c3Viczp7fSxwYXJ0
aWFsczp0LnBhcnRpYWxzLG5hbWU6dC5uYW1lfTtmb3IobiBpbiBlLnBhcnRpYWxzKWUucGFydGlh
bHNbbl09dGhpcy5tYWtlUGFydGlhbHMoZS5wYXJ0aWFsc1tuXSk7Zm9yKG4gaW4gdC5zdWJzKWUu
c3Vic1tuXT1uZXcgRnVuY3Rpb24oImMiLCJwIiwidCIsImkiLHQuc3Vic1tuXSk7cmV0dXJuIGV9
LHQuY29kZWdlbj17IiMiOmZ1bmN0aW9uKG4sZSl7ZS5jb2RlKz0iaWYodC5zKHQuIitsKG4ubikr
JygiJytjKG4ubikrJyIsYyxwLDEpLGMscCwwLCcrbi5pKyIsIituLmVuZCsnLCInK24ub3RhZysi
ICIrbi5jdGFnKyciKSl7dC5ycyhjLHAsZnVuY3Rpb24oYyxwLHQpeycsdC53YWxrKG4ubm9kZXMs
ZSksZS5jb2RlKz0ifSk7Yy5wb3AoKTt9In0sIl4iOmZ1bmN0aW9uKG4sZSl7ZS5jb2RlKz0iaWYo
IXQucyh0LiIrbChuLm4pKycoIicrYyhuLm4pKyciLGMscCwxKSxjLHAsMSwwLDAsIiIpKXsnLHQu
d2FsayhuLm5vZGVzLGUpLGUuY29kZSs9In07In0sIj4iOmYsIjwiOmZ1bmN0aW9uKG4sZSl7dmFy
IGk9e3BhcnRpYWxzOnt9LGNvZGU6IiIsc3Viczp7fSxpblBhcnRpYWw6ITB9O3Qud2FsayhuLm5v
ZGVzLGkpO3ZhciByPWUucGFydGlhbHNbZihuLGUpXTtyLnN1YnM9aS5zdWJzLHIucGFydGlhbHM9
aS5wYXJ0aWFsc30sJDpmdW5jdGlvbihuLGUpe3ZhciBpPXtzdWJzOnt9LGNvZGU6IiIscGFydGlh
bHM6ZS5wYXJ0aWFscyxwcmVmaXg6bi5ufTt0LndhbGsobi5ub2RlcyxpKSxlLnN1YnNbbi5uXT1p
LmNvZGUsZS5pblBhcnRpYWx8fChlLmNvZGUrPSd0LnN1YigiJytjKG4ubikrJyIsYyxwLGkpOycp
fSwiXG4iOmZ1bmN0aW9uKHQsbil7bi5jb2RlKz1wKCciXFxuIicrKHQubGFzdD8iIjoiICsgaSIp
KX0sX3Y6ZnVuY3Rpb24odCxuKXtuLmNvZGUrPSJ0LmIodC52KHQuIitsKHQubikrJygiJytjKHQu
bikrJyIsYyxwLDApKSk7J30sX3Q6ZnVuY3Rpb24odCxuKXtuLmNvZGUrPXAoJyInK2ModC50ZXh0
KSsnIicpfSwieyI6aCwiJiI6aH0sdC53YWxrPWZ1bmN0aW9uKG4sZSl7Zm9yKHZhciBpLHI9MCxz
PW4ubGVuZ3RoO3M+cjtyKyspaT10LmNvZGVnZW5bbltyXS50YWddLGkmJmkobltyXSxlKTtyZXR1
cm4gZX0sdC5wYXJzZT1mdW5jdGlvbih0LG4sZSl7cmV0dXJuIGU9ZXx8e30scih0LCIiLFtdLGUu
c2VjdGlvblRhZ3N8fFtdKX0sdC5jYWNoZT17fSx0LmNhY2hlS2V5PWZ1bmN0aW9uKHQsbil7cmV0
dXJuW3QsISFuLmFzU3RyaW5nLCEhbi5kaXNhYmxlTGFtYmRhLG4uZGVsaW1pdGVycywhIW4ubW9k
ZWxHZXRdLmpvaW4oInx8Iil9LHQuY29tcGlsZT1mdW5jdGlvbihuLGUpe2U9ZXx8e307dmFyIGk9
dC5jYWNoZUtleShuLGUpLHI9dGhpcy5jYWNoZVtpXTtpZihyKXt2YXIgcz1yLnBhcnRpYWxzO2Zv
cih2YXIgYSBpbiBzKWRlbGV0ZSBzW2FdLmluc3RhbmNlO3JldHVybiByfXJldHVybiByPXRoaXMu
Z2VuZXJhdGUodGhpcy5wYXJzZSh0aGlzLnNjYW4obixlLmRlbGltaXRlcnMpLG4sZSksbixlKSx0
aGlzLmNhY2hlW2ldPXJ9fSgidW5kZWZpbmVkIiE9dHlwZW9mIGV4cG9ydHM/ZXhwb3J0czpIb2dh
bik7PC9zY3JpcHQ+PHNjcmlwdD4vKmpzaGludCBzdWI6dHJ1ZSovKGZ1bmN0aW9uICgpIHsndXNl
IHN0cmljdCc7ZnVuY3Rpb24gJChzZWxlY3Rvcikge3JldHVybiBkb2N1bWVudC5xdWVyeVNlbGVj
dG9yKHNlbGVjdG9yKTt9ZnVuY3Rpb24gJCQoc2VsZWN0b3IsIGNhbGxiYWNrKSB7dmFyIGVsZW1z
ID0gZG9jdW1lbnQucXVlcnlTZWxlY3RvckFsbChzZWxlY3Rvcik7Zm9yICh2YXIgaSA9IDA7IGkg
PCBlbGVtcy5sZW5ndGg7ICsraSkge2lmIChjYWxsYmFjayAmJiB0eXBlb2YgY2FsbGJhY2sgPT0g
J2Z1bmN0aW9uJyljYWxsYmFjay5jYWxsKHRoaXMsIGVsZW1zW2ldKTt9fXZhciBkZWJvdW5jZSA9
IGZ1bmN0aW9uIChmdW5jLCB3YWl0LCBub3cpIHt2YXIgdGltZW91dDtyZXR1cm4gZnVuY3Rpb24g
ZGVib3VuY2VkICgpIHt2YXIgdGhhdCA9IHRoaXMsIGFyZ3MgPSBhcmd1bWVudHM7ZnVuY3Rpb24g
ZGVsYXllZCgpIHtpZiAoIW5vdylmdW5jLmFwcGx5KHRoYXQsIGFyZ3MpO3RpbWVvdXQgPSBudWxs
O31pZiAodGltZW91dCkge2NsZWFyVGltZW91dCh0aW1lb3V0KTt9IGVsc2UgaWYgKG5vdykge2Z1
bmMuYXBwbHkob2JqLCBhcmdzKTt9dGltZW91dCA9IHNldFRpbWVvdXQoZGVsYXllZCwgd2FpdCB8
fCAyNTApO307fTt3aW5kb3cuR29BY2Nlc3MgPSB3aW5kb3cuR29BY2Nlc3MgfHwge2luaXRpYWxp
emU6IGZ1bmN0aW9uIChvcHRpb25zKSB7dGhpcy5vcHRzID0gb3B0aW9uczt2YXIgY3cgPSBNYXRo
Lm1heChkb2N1bWVudC5kb2N1bWVudEVsZW1lbnQuY2xpZW50V2lkdGggfHwgMCwgd2luZG93Lmlu
bmVyV2lkdGggfHwgMCk7dGhpcy5BcHBTdGF0ZSAgPSB7fTt0aGlzLkFwcFRwbHMgICA9IHt9O3Ro
aXMuQXBwQ2hhcnRzID0ge307dGhpcy5BcHBVSURhdGEgPSAodGhpcy5vcHRzIHx8IHt9KS51aURh
dGEgfHwge307dGhpcy5BcHBEYXRhICAgPSAodGhpcy5vcHRzIHx8IHt9KS5wYW5lbERhdGEgfHwg
e307dGhpcy5BcHBXU0Nvbm4gPSAodGhpcy5vcHRzIHx8IHt9KS53c0Nvbm5lY3Rpb24gfHwge307
dGhpcy5pMThuID0gKHRoaXMub3B0cyB8fCB7fSkuaTE4biB8fCB7fTt0aGlzLkFwcFByZWZzICA9
IHsnYXV0b0hpZGVUYWJsZXMnOiB0cnVlLCdsYXlvdXQnOiBjdyA+IDI1NjAgPyAnd2lkZScgOiAn
aG9yaXpvbnRhbCcsJ3BlclBhZ2UnOiA3LCd0aGVtZSc6ICh3aW5kb3cubWF0Y2hNZWRpYSAmJiB3
aW5kb3cubWF0Y2hNZWRpYSgnKHByZWZlcnMtY29sb3Itc2NoZW1lOiBkYXJrKScpLm1hdGNoZXMp
ID8gJ2RhcmtQdXJwbGUnIDogJ2JyaWdodCcsJ2hpZGRlblBhbmVscyc6IFtdLH07dGhpcy5BcHBQ
cmVmcyA9IEdvQWNjZXNzLlV0aWwubWVyZ2UodGhpcy5BcHBQcmVmcywgdGhpcy5vcHRzLnByZWZz
KTt0aGlzLndzRGVsYXkgICAgPSB0aGlzLmN1cnJEZWxheSA9IDFFMzt0aGlzLm1heERlbGF5ICAg
PSAyMEUzO3RoaXMucmV0cmllcyAgICA9IDA7dGhpcy5tYXhSZXRyaWVzID0gMjA7aWYgKEdvQWNj
ZXNzLlV0aWwuaGFzTG9jYWxTdG9yYWdlKCkpIHt2YXIgbHMgPSBKU09OLnBhcnNlKGxvY2FsU3Rv
cmFnZS5nZXRJdGVtKCdBcHBQcmVmcycpKTt0aGlzLkFwcFByZWZzID0gR29BY2Nlc3MuVXRpbC5t
ZXJnZSh0aGlzLkFwcFByZWZzLCBscyk7fWlmIChPYmplY3Qua2V5cyh0aGlzLkFwcFdTQ29ubiku
bGVuZ3RoKXRoaXMuc2V0V2ViU29ja2V0KHRoaXMuQXBwV1NDb25uKTt9LGdldFBhbmVsVUk6IGZ1
bmN0aW9uIChwYW5lbCkge3JldHVybiBwYW5lbCA/IHRoaXMuQXBwVUlEYXRhW3BhbmVsXSA6IHRo
aXMuQXBwVUlEYXRhO30sZ2V0UHJlZnM6IGZ1bmN0aW9uIChwYW5lbCkge3JldHVybiBwYW5lbCA/
IHRoaXMuQXBwUHJlZnNbcGFuZWxdIDogdGhpcy5BcHBQcmVmczt9LHNldFByZWZzOiBmdW5jdGlv
biAoKSB7aWYgKEdvQWNjZXNzLlV0aWwuaGFzTG9jYWxTdG9yYWdlKCkpIHtsb2NhbFN0b3JhZ2Uu
c2V0SXRlbSgnQXBwUHJlZnMnLCBKU09OLnN0cmluZ2lmeShHb0FjY2Vzcy5nZXRQcmVmcygpKSk7
fX0sZ2V0UGFuZWxEYXRhOiBmdW5jdGlvbiAocGFuZWwpIHtyZXR1cm4gcGFuZWwgPyB0aGlzLkFw
cERhdGFbcGFuZWxdIDogdGhpcy5BcHBEYXRhO30scmVjb25uZWN0OiBmdW5jdGlvbiAod3NDb25u
KSB7aWYgKHRoaXMucmV0cmllcyA+PSB0aGlzLm1heFJldHJpZXMpcmV0dXJuIHdpbmRvdy5jbGVh
clRpbWVvdXQodGhpcy53c1RpbWVyKTt0aGlzLnJldHJpZXMrKztpZiAodGhpcy5jdXJyRGVsYXkg
PCB0aGlzLm1heERlbGF5KXRoaXMuY3VyckRlbGF5ICo9IDI7dGhpcy5zZXRXZWJTb2NrZXQod3ND
b25uKTt9LGJ1aWxkV1NVUkk6IGZ1bmN0aW9uICh3c0Nvbm4pIHt2YXIgdXJsID0gbnVsbDtpZiAo
IXdzQ29ubi51cmwgfHwgIXdzQ29ubi5wb3J0KXJldHVybiBudWxsO3VybCA9IC9ed3NzPzpcL1wv
L2kudGVzdCh3c0Nvbm4udXJsKSA/IHdzQ29ubi51cmwgOiB3aW5kb3cubG9jYXRpb24ucHJvdG9j
b2wgPT09ICJodHRwczoiID8gJ3dzczovLycgKyB3c0Nvbm4udXJsIDogJ3dzOi8vJyArIHdzQ29u
bi51cmw7cmV0dXJuIG5ldyBVUkwodXJsKS5wcm90b2NvbCArICcvLycgKyBuZXcgVVJMKHVybCku
aG9zdG5hbWUgKyAnOicgKyB3c0Nvbm4ucG9ydCArIG5ldyBVUkwodXJsKS5wYXRobmFtZTt9LHNl
dFdlYlNvY2tldDogZnVuY3Rpb24gKHdzQ29ubikge3ZhciBob3N0ID0gbnVsbCwgcGluZ0lkID0g
bnVsbCwgdXJpID0gbnVsbCwgZGVmVVJJID0gbnVsbCwgc3RyID0gbnVsbDtkZWZVUkkgPSB3aW5k
b3cubG9jYXRpb24uaG9zdG5hbWUgPyB3aW5kb3cubG9jYXRpb24uaG9zdG5hbWUgKyAnOicgKyB3
c0Nvbm4ucG9ydCA6ICJsb2NhbGhvc3QiICsgJzonICsgd3NDb25uLnBvcnQ7dXJpID0gd3NDb25u
LnVybCAmJiAvXih3c3M/OlwvXC8pP1teXC9dKzpbMC05XXsxLDV9Ly50ZXN0KHdzQ29ubi51cmwp
ID8gd3NDb25uLnVybCA6IHRoaXMuYnVpbGRXU1VSSSh3c0Nvbm4pO3N0ciA9IHVyaSB8fCBkZWZV
Ukk7c3RyID0gIS9ed3NzPzpcL1wvL2kudGVzdChzdHIpID8gKHdpbmRvdy5sb2NhdGlvbi5wcm90
b2NvbCA9PT0gImh0dHBzOiIgPyAnd3NzOi8vJyA6ICd3czovLycpICsgc3RyIDogc3RyO3ZhciBz
b2NrZXQgPSBuZXcgV2ViU29ja2V0KHN0cik7c29ja2V0Lm9ub3BlbiA9IGZ1bmN0aW9uIChldmVu
dCkge3RoaXMuY3VyckRlbGF5ID0gdGhpcy53c0RlbGF5O3RoaXMucmV0cmllcyA9IDA7aWYgKHdz
Q29ubi5waW5nX2ludGVydmFsKXBpbmdJZCA9IHNldEludGVydmFsKCgpID0+IHsgc29ja2V0LnNl
bmQoJ3BpbmcnKTsgfSwgd3NDb25uLnBpbmdfaW50ZXJ2YWwgKiAxRTMpO0dvQWNjZXNzLk5hdi5X
U09wZW4oc3RyKTt9LmJpbmQodGhpcyk7c29ja2V0Lm9ubWVzc2FnZSA9IGZ1bmN0aW9uIChldmVu
dCkge3RoaXMuQXBwU3RhdGVbJ3VwZGF0ZWQnXSA9IHRydWU7dGhpcy5BcHBEYXRhID0gSlNPTi5w
YXJzZShldmVudC5kYXRhKTt0aGlzLkFwcC5yZW5kZXJEYXRhKCk7fS5iaW5kKHRoaXMpO3NvY2tl
dC5vbmNsb3NlID0gZnVuY3Rpb24gKGV2ZW50KSB7R29BY2Nlc3MuTmF2LldTQ2xvc2UoKTt3aW5k
b3cuY2xlYXJJbnRlcnZhbChwaW5nSWQpO3NvY2tldCA9IG51bGw7dGhpcy53c1RpbWVyID0gc2V0
VGltZW91dCgoKSA9PiB7IHRoaXMucmVjb25uZWN0KHdzQ29ubik7IH0sIHRoaXMuY3VyckRlbGF5
KTt9LmJpbmQodGhpcyk7fSx9O0dvQWNjZXNzLlV0aWwgPSB7bW9udGhzOiBbIkphbiIsICJGZWIi
LCAiTWFyIiwgIkFwciIsICJNYXkiLCAiSnVuIiwgIkp1bCIsIkF1ZyIsICJTZXAiLCAiT2N0Iiwg
Ik5vdiIsICJEZWMiXSxtZXJnZTogZnVuY3Rpb24gKG8sIG4pIHt2YXIgb2JqID0ge30sIGkgPSAw
LCBpbCA9IGFyZ3VtZW50cy5sZW5ndGgsIGtleTtmb3IgKDsgaSA8IGlsOyBpKyspIHtmb3IgKGtl
eSBpbiBhcmd1bWVudHNbaV0pIHtpZiAoYXJndW1lbnRzW2ldLmhhc093blByb3BlcnR5KGtleSkp
IHtvYmpba2V5XSA9IGFyZ3VtZW50c1tpXVtrZXldO319fXJldHVybiBvYmo7fSxoYXNoQ29kZTog
ZnVuY3Rpb24gKHMpIHtyZXR1cm4gKHMuc3BsaXQoJycpLnJlZHVjZShmdW5jdGlvbiAoYSwgYikg
e2EgPSAoKGEgPDwgNSkgLSBhKSArIGIuY2hhckNvZGVBdCgwKTtyZXR1cm4gYSZhO30sIDApID4+
PiAwKS50b1N0cmluZygxNik7fSxmb3JtYXRCeXRlczogZnVuY3Rpb24gKGJ5dGVzLCBkZWNpbWFs
cywgbnVtT25seSkge2lmIChieXRlcyA9PSAwKXJldHVybiBudW1Pbmx5ID8gMCA6ICcwIEJ5dGUn
O3ZhciBrID0gMTAyNDt2YXIgZG0gPSBkZWNpbWFscyArIDEgfHwgMjt2YXIgc2l6ZXMgPSBbJ0In
LCAnS2lCJywgJ01pQicsICdHaUInLCAnVGlCJywgJ1BpQiddO3ZhciBpID0gTWF0aC5mbG9vcihN
YXRoLmxvZyhieXRlcykgLyBNYXRoLmxvZyhrKSk7cmV0dXJuIHBhcnNlRmxvYXQoKGJ5dGVzIC8g
TWF0aC5wb3coaywgaSkpLnRvRml4ZWQoZG0pKSArIChudW1Pbmx5ID8gJycgOiAoJyAnICsgc2l6
ZXNbaV0pKTt9LGlzTnVtZXJpYzogZnVuY3Rpb24gKG4pIHtyZXR1cm4gIWlzTmFOKHBhcnNlRmxv
YXQobikpICYmIGlzRmluaXRlKG4pO30sdXRpbWUyc3RyOiBmdW5jdGlvbiAodXNlYykge2lmICh1
c2VjID49IDg2NEU4KXJldHVybiAoKHVzZWMpIC8gODY0RTgpLnRvRml4ZWQoMikgKyAnIGQnO2Vs
c2UgaWYgKHVzZWMgPj0gMzZFOClyZXR1cm4gKCh1c2VjKSAvIDM2RTgpLnRvRml4ZWQoMikgKyAn
IGgnO2Vsc2UgaWYgKHVzZWMgPj0gNkU3KXJldHVybiAoKHVzZWMpIC8gNkU3KS50b0ZpeGVkKDIp
ICsgJyBtJztlbHNlIGlmICh1c2VjID49IDFFNilyZXR1cm4gKCh1c2VjKSAvIDFFNikudG9GaXhl
ZCgyKSArICcgcyc7ZWxzZSBpZiAodXNlYyA+PSAxRTMpcmV0dXJuICgodXNlYykgLyAxRTMpLnRv
Rml4ZWQoMikgKyAnIG1zJztyZXR1cm4gKHVzZWMpLnRvRml4ZWQoMikgKyAnIHVzJzt9LGZvcm1h
dERhdGU6IGZ1bmN0aW9uIChzdHIpIHt2YXIgeSA9IHN0ci5zdWJzdHIoMCw0KSwgbSA9IHN0ci5z
dWJzdHIoNCwyKSAtIDEsIGQgPSBzdHIuc3Vic3RyKDYsMiksaCA9IHN0ci5zdWJzdHIoOCwyKSB8
fCAwLCBpID0gc3RyLnN1YnN0cigxMCwgMikgIHx8IDAsIHMgPSBzdHIuc3Vic3RyKDEyLCAyKSB8
fCAwO3ZhciBkYXRlID0gbmV3IERhdGUoeSxtLGQsaCxpLHMpO3ZhciBvdXQgPSAoJzAnICsgZGF0
ZS5nZXREYXRlKCkpLnNsaWNlKC0yKSArICcvJyArIHRoaXMubW9udGhzW2RhdGUuZ2V0TW9udGgo
KV0gKyAnLycgKyBkYXRlLmdldEZ1bGxZZWFyKCk7MTAgPD0gc3RyLmxlbmd0aCAmJiAob3V0ICs9
ICI6IiArIGgpOzEyIDw9IHN0ci5sZW5ndGggJiYgKG91dCArPSAiOiIgKyBpKTsxNCA8PSBzdHIu
bGVuZ3RoICYmIChvdXQgKz0gIjoiICsgcyk7cmV0dXJuIG91dDt9LHNob3J0TnVtOiAgZnVuY3Rp
b24gKG4pIHtpZiAobiA8IDFlMykgcmV0dXJuIG47aWYgKG4gPj0gMWUzICYmIG4gPCAxZTYpIHJl
dHVybiArKG4gLyAxZTMpLnRvRml4ZWQoMSkgKyAiSyI7aWYgKG4gPj0gMWU2ICYmIG4gPCAxZTkp
IHJldHVybiArKG4gLyAxZTYpLnRvRml4ZWQoMSkgKyAiTSI7aWYgKG4gPj0gMWU5ICYmIG4gPCAx
ZTEyKSByZXR1cm4gKyhuIC8gMWU5KS50b0ZpeGVkKDEpICsgIkIiO2lmIChuID49IDFlMTIpIHJl
dHVybiArKG4gLyAxZTEyKS50b0ZpeGVkKDEpICsgIlQiO30sZm10VmFsdWU6IGZ1bmN0aW9uICh2
YWx1ZSwgZGF0YVR5cGUsIGRlY2ltYWxzLCBzaG9ydGVuLCBobHJlZ2V4LCBobHZhbHVlKSB7dmFy
IHZhbCA9IDA7aWYgKCFkYXRhVHlwZSl2YWwgPSB2YWx1ZTtzd2l0Y2ggKGRhdGFUeXBlKSB7Y2Fz
ZSAndXRpbWUnOnZhbCA9IHRoaXMudXRpbWUyc3RyKCt2YWx1ZSk7YnJlYWs7Y2FzZSAnZGF0ZSc6
dmFsID0gdGhpcy5mb3JtYXREYXRlKHZhbHVlKTticmVhaztjYXNlICdudW1lcmljJzppZiAodGhp
cy5pc051bWVyaWModmFsdWUpKXZhbCA9IHNob3J0ZW4gPyB0aGlzLnNob3J0TnVtKHZhbHVlKSA6
ICgrdmFsdWUpLnRvTG9jYWxlU3RyaW5nKCk7YnJlYWs7Y2FzZSAnYnl0ZXMnOnZhbCA9IHRoaXMu
Zm9ybWF0Qnl0ZXModmFsdWUsIGRlY2ltYWxzKTticmVhaztjYXNlICdwZXJjZW50Jzp2YWwgPSB2
YWx1ZS5yZXBsYWNlKCcsJywgJy4nKSArICclJzticmVhaztjYXNlICd0aW1lJzppZiAodGhpcy5p
c051bWVyaWModmFsdWUpKXZhbCA9IHZhbHVlLnRvTG9jYWxlU3RyaW5nKCk7YnJlYWs7Y2FzZSAn
c2Vjcyc6dmFyIHQgPSBuZXcgRGF0ZShudWxsKTt0LnNldFNlY29uZHModmFsdWUpO3ZhbCA9IHQu
dG9JU09TdHJpbmcoKS5zdWJzdHIoMTEsIDgpO2JyZWFrO2RlZmF1bHQ6dmFsID0gdmFsdWU7fWlm
IChobHJlZ2V4KSB7bGV0IG8gPSBKU09OLnBhcnNlKGhscmVnZXgpLCB0bXAgPSAnJztmb3IgKHZh
ciB4IGluIG8pIHtpZiAoIXZhbCkgY29udGludWU7dG1wID0gdmFsLnJlcGxhY2UobmV3IFJlZ0V4
cCh4LCAnZ2knKSwgb1t4XSk7aWYgKHRtcCAhPSB2YWwpIHt2YWwgPSB0bXA7YnJlYWs7fXZhbCA9
IHRtcDt9fXJldHVybiB2YWx1ZSA9PSAwID8gU3RyaW5nKHZhbCkgOiAodmFsID09PSB1bmRlZmlu
ZWQgPyAn4oCUJyA6IHZhbCk7fSxpc1BhbmVsSGlkZGVuOiBmdW5jdGlvbiAocGFuZWwpIHtyZXR1
cm4gR29BY2Nlc3MuQXBwUHJlZnMuaGlkZGVuUGFuZWxzLmluY2x1ZGVzKHBhbmVsKTt9LGlzUGFu
ZWxWYWxpZDogZnVuY3Rpb24gKHBhbmVsKSB7dmFyIGRhdGEgPSBHb0FjY2Vzcy5nZXRQYW5lbERh
dGEoKSwgdWkgPSBHb0FjY2Vzcy5nZXRQYW5lbFVJKCk7cmV0dXJuICghdWkuaGFzT3duUHJvcGVy
dHkocGFuZWwpIHx8ICFkYXRhLmhhc093blByb3BlcnR5KHBhbmVsKSB8fCAhdWlbcGFuZWxdLmlk
KTt9LGdldENvdW50OiBmdW5jdGlvbiAoaXRlbSkge2lmICh0aGlzLmlzT2JqZWN0KGl0ZW0pICYm
ICdjb3VudCcgaW4gaXRlbSlyZXR1cm4gaXRlbS5jb3VudDtyZXR1cm4gaXRlbTt9LGdldFBlcmNl
bnQ6IGZ1bmN0aW9uIChpdGVtKSB7aWYgKHRoaXMuaXNPYmplY3QoaXRlbSkgJiYgJ3BlcmNlbnQn
IGluIGl0ZW0pcmV0dXJuIHRoaXMuZm10VmFsdWUoaXRlbS5wZXJjZW50LCAncGVyY2VudCcpO3Jl
dHVybiBudWxsO30saXNPYmplY3Q6IGZ1bmN0aW9uIChvKSB7cmV0dXJuIG8gPT09IE9iamVjdChv
KTt9LHNldFByb3A6IGZ1bmN0aW9uIChvLCBzLCB2KSB7dmFyIHNjaGVtYSA9IG87dmFyIGEgPSBz
LnNwbGl0KCcuJyk7Zm9yICh2YXIgaSA9IDAsIG4gPSBhLmxlbmd0aDsgaSA8IG4tMTsgKytpKSB7
dmFyIGsgPSBhW2ldO2lmICghc2NoZW1hW2tdKXNjaGVtYVtrXSA9IHt9O3NjaGVtYSA9IHNjaGVt
YVtrXTt9c2NoZW1hW2Fbbi0xXV0gPSB2O30sZ2V0UHJvcDogZnVuY3Rpb24gKG8sIHMpIHtzID0g
cy5yZXBsYWNlKC9cWyhcdyspXF0vZywgJy4kMScpO3MgPSBzLnJlcGxhY2UoL15cLi8sICcnKTt2
YXIgYSA9IHMuc3BsaXQoJy4nKTtmb3IgKHZhciBpID0gMCwgbiA9IGEubGVuZ3RoOyBpIDwgbjsg
KytpKSB7dmFyIGsgPSBhW2ldO2lmICh0aGlzLmlzT2JqZWN0KG8pICYmIGsgaW4gbykge28gPSBv
W2tdO30gZWxzZSB7cmV0dXJuO319cmV0dXJuIG87fSxoYXNMb2NhbFN0b3JhZ2U6IGZ1bmN0aW9u
ICgpIHt0cnkge2xvY2FsU3RvcmFnZS5zZXRJdGVtKCd0ZXN0JywgJ3Rlc3QnKTtsb2NhbFN0b3Jh
Z2UucmVtb3ZlSXRlbSgndGVzdCcpO3JldHVybiB0cnVlO30gY2F0Y2goZSkge3JldHVybiBmYWxz
ZTt9fSxpc1dpdGhpblZpZXdQb3J0OiBmdW5jdGlvbiAoZWwpIHt2YXIgZWxlbVRvcCA9IGVsLmdl
dEJvdW5kaW5nQ2xpZW50UmVjdCgpLnRvcDt2YXIgZWxlbUJvdHRvbSA9IGVsLmdldEJvdW5kaW5n
Q2xpZW50UmVjdCgpLmJvdHRvbTtyZXR1cm4gZWxlbVRvcCA8IHdpbmRvdy5pbm5lckhlaWdodCAm
JiBlbGVtQm90dG9tID49IDA7fSx0b2dnbGVQYW5lbDogZnVuY3Rpb24ocGFuZWwpIHt2YXIgaW5k
ZXggPSBHb0FjY2Vzcy5BcHBQcmVmcy5oaWRkZW5QYW5lbHMuaW5kZXhPZihwYW5lbCk7aWYgKGlu
ZGV4ID09IC0xKSB7R29BY2Nlc3MuQXBwUHJlZnMuaGlkZGVuUGFuZWxzLnB1c2gocGFuZWwpO30g
ZWxzZSB7R29BY2Nlc3MuQXBwUHJlZnMuaGlkZGVuUGFuZWxzLnNwbGljZShpbmRleCwgMSk7fUdv
QWNjZXNzLnNldFByZWZzKCk7ZGVsZXRlIEdvQWNjZXNzLkFwcENoYXJ0c1twYW5lbF07R29BY2Nl
c3MuT3ZlcmFsbFN0YXRzLmluaXRpYWxpemUoKTtHb0FjY2Vzcy5QYW5lbHMuaW5pdGlhbGl6ZSgp
O0dvQWNjZXNzLkNoYXJ0cy5pbml0aWFsaXplKCk7R29BY2Nlc3MuVGFibGVzLmluaXRpYWxpemUo
KTt9LH07R29BY2Nlc3MuT3ZlcmFsbFN0YXRzID0ge3RvdGFsX3JlcXVlc3RzOiAwLHJlbmRlckJv
eDogZnVuY3Rpb24gKGRhdGEsIHVpLCByb3csIHgsIGlkeCkge3ZhciB3cmFwID0gJCgnLndyYXAt
Z2VuZXJhbC1pdGVtcycpO2lmIChpZHggJSA2ID09IDApIHtyb3cgPSBkb2N1bWVudC5jcmVhdGVF
bGVtZW50KCdkaXYnKTtyb3cuc2V0QXR0cmlidXRlKCdjbGFzcycsICdyb3cnKTt3cmFwLmFwcGVu
ZENoaWxkKHJvdyk7fXZhciBib3ggPSBkb2N1bWVudC5jcmVhdGVFbGVtZW50KCdkaXYnKTtib3gu
aW5uZXJIVE1MID0gR29BY2Nlc3MuQXBwVHBscy5HZW5lcmFsLml0ZW1zLnJlbmRlcih7J2lkJzog
eCwnY2xhc3NOYW1lJzogdWkuaXRlbXNbeF0uY2xhc3NOYW1lLCdsYWJlbCc6IHVpLml0ZW1zW3hd
LmxhYmVsLCd2YWx1ZSc6IEdvQWNjZXNzLlV0aWwuZm10VmFsdWUoZGF0YVt4XSwgdWkuaXRlbXNb
eF0uZGF0YVR5cGUpLH0pO3Jvdy5hcHBlbmRDaGlsZChib3gpO3JldHVybiByb3c7fSxyZW5kZXJE
YXRhOiBmdW5jdGlvbiAoZGF0YSwgdWkpIHt2YXIgaWR4ID0gMCwgcm93ID0gbnVsbDskKCcubGFz
dC11cGRhdGVkJykuaW5uZXJIVE1MID0gZGF0YS5kYXRlX3RpbWU7JCgnLndyYXAtZ2VuZXJhbCcp
LmlubmVySFRNTCA9ICcnO2lmIChHb0FjY2Vzcy5VdGlsLmlzUGFuZWxIaWRkZW4oJ2dlbmVyYWwn
KSlyZXR1cm4gZmFsc2U7JCgnLndyYXAtZ2VuZXJhbCcpLmlubmVySFRNTCA9IEdvQWNjZXNzLkFw
cFRwbHMuR2VuZXJhbC53cmFwLnJlbmRlcihHb0FjY2Vzcy5VdGlsLm1lcmdlKHVpLCB7J2Zyb20n
OiBkYXRhLnN0YXJ0X2RhdGUsJ3RvJzogZGF0YS5lbmRfZGF0ZSx9KSk7Zm9yICh2YXIgeCBpbiBk
YXRhKSB7aWYgKCFkYXRhLmhhc093blByb3BlcnR5KHgpIHx8ICF1aS5pdGVtcy5oYXNPd25Qcm9w
ZXJ0eSh4KSljb250aW51ZTtyb3cgPSB0aGlzLnJlbmRlckJveChkYXRhLCB1aSwgcm93LCB4LCBp
ZHgpO2lkeCsrO319LGluaXRpYWxpemU6IGZ1bmN0aW9uICgpIHt2YXIgdWkgPSBHb0FjY2Vzcy5n
ZXRQYW5lbFVJKCdnZW5lcmFsJyk7dmFyIGRhdGEgPSBHb0FjY2Vzcy5nZXRQYW5lbERhdGEoJ2dl
bmVyYWwnKTt0aGlzLnRvdGFsX3JlcXVlc3RzID0gZGF0YS50b3RhbF9yZXF1ZXN0czt0aGlzLnJl
bmRlckRhdGEoZGF0YSwgdWkpO319O0dvQWNjZXNzLk5hdiA9IHtldmVudHM6IGZ1bmN0aW9uICgp
IHskKCcubmF2LWJhcnMnKS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHtlLnN0b3BQcm9wYWdhdGlv
bigpO3RoaXMucmVuZGVyTWVudShlKTt9LmJpbmQodGhpcyk7JCgnLm5hdi1nZWFycycpLm9uY2xp
Y2sgPSBmdW5jdGlvbiAoZSkge2Uuc3RvcFByb3BhZ2F0aW9uKCk7dGhpcy5yZW5kZXJPcHRzKGUp
O30uYmluZCh0aGlzKTskKCcubmF2LW1pbmliYXJzJykub25jbGljayA9IGZ1bmN0aW9uIChlKSB7
ZS5zdG9wUHJvcGFnYXRpb24oKTt0aGlzLnJlbmRlck9wdHMoZSk7fS5iaW5kKHRoaXMpOyQoJ2Jv
ZHknKS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHskKCduYXYnKS5jbGFzc0xpc3QucmVtb3ZlKCdh
Y3RpdmUnKTt9LmJpbmQodGhpcyk7JCQoJy5leHBvcnQtanNvbicsIGZ1bmN0aW9uIChpdGVtKSB7
aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt0aGlzLmRvd25sb2FkSlNPTihlKTt9LmJpbmQo
dGhpcyk7fS5iaW5kKHRoaXMpKTskJCgnLnRoZW1lLWJyaWdodCcsIGZ1bmN0aW9uIChpdGVtKSB7
aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt0aGlzLnNldFRoZW1lKCdicmlnaHQnKTt9LmJp
bmQodGhpcyk7fS5iaW5kKHRoaXMpKTskJCgnLnRoZW1lLWRhcmstYmx1ZScsIGZ1bmN0aW9uIChp
dGVtKSB7aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt0aGlzLnNldFRoZW1lKCdkYXJrQmx1
ZScpO30uYmluZCh0aGlzKTt9LmJpbmQodGhpcykpOyQkKCcudGhlbWUtZGFyay1ncmF5JywgZnVu
Y3Rpb24gKGl0ZW0pIHtpdGVtLm9uY2xpY2sgPSBmdW5jdGlvbiAoZSkge3RoaXMuc2V0VGhlbWUo
J2RhcmtHcmF5Jyk7fS5iaW5kKHRoaXMpO30uYmluZCh0aGlzKSk7JCQoJy50aGVtZS1kYXJrLXB1
cnBsZScsIGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt0aGlz
LnNldFRoZW1lKCdkYXJrUHVycGxlJyk7fS5iaW5kKHRoaXMpO30uYmluZCh0aGlzKSk7JCQoJy5s
YXlvdXQtaG9yaXpvbnRhbCcsIGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5vbmNsaWNrID0gZnVuY3Rp
b24gKGUpIHt0aGlzLnNldExheW91dCgnaG9yaXpvbnRhbCcpO30uYmluZCh0aGlzKTt9LmJpbmQo
dGhpcykpOyQkKCcubGF5b3V0LXZlcnRpY2FsJywgZnVuY3Rpb24gKGl0ZW0pIHtpdGVtLm9uY2xp
Y2sgPSBmdW5jdGlvbiAoZSkge3RoaXMuc2V0TGF5b3V0KCd2ZXJ0aWNhbCcpO30uYmluZCh0aGlz
KTt9LmJpbmQodGhpcykpOyQkKCcubGF5b3V0LXdpZGUnLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0u
b25jbGljayA9IGZ1bmN0aW9uIChlKSB7dGhpcy5zZXRMYXlvdXQoJ3dpZGUnKTt9LmJpbmQodGhp
cyk7fS5iaW5kKHRoaXMpKTskJCgnW2RhdGEtcGVycGFnZV0nLCBmdW5jdGlvbiAoaXRlbSkge2l0
ZW0ub25jbGljayA9IGZ1bmN0aW9uIChlKSB7dGhpcy5zZXRQZXJQYWdlKGUpO30uYmluZCh0aGlz
KTt9LmJpbmQodGhpcykpOyQkKCdbZGF0YS1zaG93LXRhYmxlc10nLCBmdW5jdGlvbiAoaXRlbSkg
e2l0ZW0ub25jbGljayA9IGZ1bmN0aW9uIChlKSB7dGhpcy50b2dnbGVUYWJsZXMoKTt9LmJpbmQo
dGhpcyk7fS5iaW5kKHRoaXMpKTskJCgnW2RhdGEtYXV0b2hpZGUtdGFibGVzXScsIGZ1bmN0aW9u
IChpdGVtKSB7aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt0aGlzLnRvZ2dsZUF1dG9IaWRl
VGFibGVzKCk7fS5iaW5kKHRoaXMpO30uYmluZCh0aGlzKSk7JCQoJy50b2dnbGUtcGFuZWwnLCBm
dW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9IGZ1bmN0aW9uIChlKSB7ZS5zdG9wUHJvcGFn
YXRpb24oKTt2YXIgcGFuZWwgPSBlLmN1cnJlbnRUYXJnZXQuZ2V0QXR0cmlidXRlKCdkYXRhLXBh
bmVsJyk7R29BY2Nlc3MuVXRpbC50b2dnbGVQYW5lbChwYW5lbCk7aXRlbS5jbGFzc0xpc3QudG9n
Z2xlKCdhY3RpdmUnKTt9LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTt9LGRvd25sb2FkSlNPTjog
ZnVuY3Rpb24gKGUpIHt2YXIgdGFyZyA9IGUuY3VycmVudFRhcmdldDt2YXIgZGF0YSA9ICJ0ZXh0
L2pzb247Y2hhcnNldD11dGYtOCwiICsgZW5jb2RlVVJJQ29tcG9uZW50KEpTT04uc3RyaW5naWZ5
KEdvQWNjZXNzLmdldFBhbmVsRGF0YSgpKSk7dGFyZy5ocmVmID0gJ2RhdGE6JyArIGRhdGE7dGFy
Zy5kb3dubG9hZCA9ICdnb2FjY2Vzcy0nICsgKCtuZXcgRGF0ZSgpKSArICcuanNvbic7fSxzZXRM
YXlvdXQ6IGZ1bmN0aW9uIChsYXlvdXQpIHtpZiAoKCdob3Jpem9udGFsJyA9PSBsYXlvdXQgfHwg
J3dpZGUnID09IGxheW91dCkgJiYgJCgnLmNvbnRhaW5lcicpKSB7JCgnLmNvbnRhaW5lcicpLmNs
YXNzTGlzdC5hZGQoJ2NvbnRhaW5lci1mbHVpZCcpOyQoJy5jb250YWluZXInKS5jbGFzc0xpc3Qu
cmVtb3ZlKCdjb250YWluZXInKTt9IGVsc2UgaWYgKCd2ZXJ0aWNhbCcgPT0gbGF5b3V0ICYmICQo
Jy5jb250YWluZXItZmx1aWQnKSkgeyQoJy5jb250YWluZXItZmx1aWQnKS5jbGFzc0xpc3QuYWRk
KCdjb250YWluZXInKTskKCcuY29udGFpbmVyJykuY2xhc3NMaXN0LnJlbW92ZSgnY29udGFpbmVy
LWZsdWlkJyk7fUdvQWNjZXNzLkFwcFByZWZzWydsYXlvdXQnXSA9IGxheW91dDtHb0FjY2Vzcy5z
ZXRQcmVmcygpO0dvQWNjZXNzLlBhbmVscy5pbml0aWFsaXplKCk7R29BY2Nlc3MuQ2hhcnRzLmlu
aXRpYWxpemUoKTtHb0FjY2Vzcy5UYWJsZXMuaW5pdGlhbGl6ZSgpO30sdG9nZ2xlQXV0b0hpZGVU
YWJsZXM6IGZ1bmN0aW9uIChlKSB7dmFyIGF1dG9IaWRlVGFibGVzID0gR29BY2Nlc3MuVGFibGVz
LmF1dG9IaWRlVGFibGVzKCk7JCQoJy50YWJsZS13cmFwcGVyJywgZnVuY3Rpb24gKGl0ZW0pIHtp
ZiAoYXV0b0hpZGVUYWJsZXMpIHtpdGVtLmNsYXNzTGlzdC5yZW1vdmUoJ2hpZGRlbi14cycpO30g
ZWxzZSB7aXRlbS5jbGFzc0xpc3QuYWRkKCdoaWRkZW4teHMnKTt9fS5iaW5kKHRoaXMpKTtHb0Fj
Y2Vzcy5BcHBQcmVmc1snYXV0b0hpZGVUYWJsZXMnXSA9ICFhdXRvSGlkZVRhYmxlcztHb0FjY2Vz
cy5zZXRQcmVmcygpO30sdG9nZ2xlVGFibGVzOiBmdW5jdGlvbiAoKSB7dmFyIHVpID0gR29BY2Nl
c3MuZ2V0UGFuZWxVSSgpO3ZhciBzaG93VGFibGVzID0gR29BY2Nlc3MuVGFibGVzLnNob3dUYWJs
ZXMoKTtPYmplY3Qua2V5cyh1aSkuZm9yRWFjaChmdW5jdGlvbiAocGFuZWwsIGlkeCkge2lmICgh
R29BY2Nlc3MuVXRpbC5pc1BhbmVsVmFsaWQocGFuZWwpIHx8IEdvQWNjZXNzLlV0aWwuaXNQYW5l
bEhpZGRlbihwYW5lbCkpdWlbcGFuZWxdWyd0YWJsZSddID0gIXNob3dUYWJsZXM7fS5iaW5kKHRo
aXMpKTtHb0FjY2Vzcy5BcHBQcmVmc1snc2hvd1RhYmxlcyddID0gIXNob3dUYWJsZXM7R29BY2Nl
c3Muc2V0UHJlZnMoKTtHb0FjY2Vzcy5QYW5lbHMuaW5pdGlhbGl6ZSgpO0dvQWNjZXNzLkNoYXJ0
cy5pbml0aWFsaXplKCk7R29BY2Nlc3MuVGFibGVzLmluaXRpYWxpemUoKTt9LHNldFRoZW1lOiBm
dW5jdGlvbiAodGhlbWUpIHtpZiAoIXRoZW1lKXJldHVybjskKCdodG1sJykuY2xhc3NOYW1lID0g
Jyc7c3dpdGNoKHRoZW1lKSB7Y2FzZSAnZGFya0dyYXknOiQoJ2h0bWwnKS5jbGFzc0xpc3QuYWRk
KCdkYXJrJyk7JCgnaHRtbCcpLmNsYXNzTGlzdC5hZGQoJ2dyYXknKTticmVhaztjYXNlICdkYXJr
Qmx1ZSc6JCgnaHRtbCcpLmNsYXNzTGlzdC5hZGQoJ2RhcmsnKTskKCdodG1sJykuY2xhc3NMaXN0
LmFkZCgnYmx1ZScpO2JyZWFrO2Nhc2UgJ2RhcmtQdXJwbGUnOiQoJ2h0bWwnKS5jbGFzc0xpc3Qu
YWRkKCdkYXJrJyk7JCgnaHRtbCcpLmNsYXNzTGlzdC5hZGQoJ3B1cnBsZScpO2JyZWFrO31Hb0Fj
Y2Vzcy5BcHBQcmVmc1sndGhlbWUnXSA9IHRoZW1lO0dvQWNjZXNzLnNldFByZWZzKCk7fSxnZXRJ
Y29uOiBmdW5jdGlvbiAoa2V5KSB7c3dpdGNoKGtleSkge2Nhc2UgJ3Zpc2l0b3JzJyAgICAgICAg
OiByZXR1cm4gJ3VzZXJzJztjYXNlICdyZXF1ZXN0cycgICAgICAgIDogcmV0dXJuICdmaWxlJztj
YXNlICdzdGF0aWNfcmVxdWVzdHMnIDogcmV0dXJuICdmaWxlLXRleHQnO2Nhc2UgJ25vdF9mb3Vu
ZCcgICAgICAgOiByZXR1cm4gJ2ZpbGUtbyc7Y2FzZSAnaG9zdHMnICAgICAgICAgICA6IHJldHVy
biAndXNlcic7Y2FzZSAnb3MnICAgICAgICAgICAgICA6IHJldHVybiAnZGVza3RvcCc7Y2FzZSAn
YnJvd3NlcnMnICAgICAgICA6IHJldHVybiAnY2hyb21lJztjYXNlICd2aXNpdF90aW1lJyAgICAg
IDogcmV0dXJuICdjbG9jay1vJztjYXNlICd2aG9zdHMnICAgICAgICAgIDogcmV0dXJuICd0aC1s
aXN0JztjYXNlICdyZWZlcnJlcnMnICAgICAgIDogcmV0dXJuICdleHRlcm5hbC1saW5rJztjYXNl
ICdyZWZlcnJpbmdfc2l0ZXMnIDogcmV0dXJuICdleHRlcm5hbC1saW5rJztjYXNlICdrZXlwaHJh
c2VzJyAgICAgIDogcmV0dXJuICdnb29nbGUnO2Nhc2UgJ3N0YXR1c19jb2RlcycgICAgOiByZXR1
cm4gJ3dhcm5pbmcnO2Nhc2UgJ3JlbW90ZV91c2VyJyAgICAgOiByZXR1cm4gJ3VzZXJzJztjYXNl
ICdnZW9sb2NhdGlvbicgICAgIDogcmV0dXJuICdtYXAtbWFya2VyJztjYXNlICdhc24nICAgICAg
ICAgICAgIDogcmV0dXJuICdtYXAtbWFya2VyJztjYXNlICdtaW1lX3R5cGUnICAgICAgIDogcmV0
dXJuICdmaWxlLW8nO2Nhc2UgJ3Rsc190eXBlJyAgICAgICAgOiByZXR1cm4gJ3dhcm5pbmcnO2Rl
ZmF1bHQgICAgICAgICAgICAgICAgOiByZXR1cm4gJ3BpZS1jaGFydCc7fX0sZ2V0SXRlbXM6IGZ1
bmN0aW9uICgpIHt2YXIgdWkgPSBHb0FjY2Vzcy5nZXRQYW5lbFVJKCksIG1lbnUgPSBbXTtmb3Ig
KHZhciBwYW5lbCBpbiB1aSkge2lmIChHb0FjY2Vzcy5VdGlsLmlzUGFuZWxWYWxpZChwYW5lbCkp
Y29udGludWU7bWVudS5wdXNoKHsnY3VycmVudCc6IHdpbmRvdy5sb2NhdGlvbi5oYXNoLnN1YnN0
cigxKSA9PSBwYW5lbCwnaGVhZCc6IHVpW3BhbmVsXS5oZWFkLCdrZXknOiBwYW5lbCwnaWNvbic6
IHRoaXMuZ2V0SWNvbihwYW5lbCksJ2hpZGRlbic6IEdvQWNjZXNzLlV0aWwuaXNQYW5lbEhpZGRl
bihwYW5lbCl9KTt9cmV0dXJuIG1lbnU7fSxzZXRQZXJQYWdlOiBmdW5jdGlvbiAoZSkge0dvQWNj
ZXNzLkFwcFByZWZzWydwZXJQYWdlJ10gPSArZS5jdXJyZW50VGFyZ2V0LmdldEF0dHJpYnV0ZSgn
ZGF0YS1wZXJwYWdlJyk7R29BY2Nlc3MuQXBwLnJlbmRlckRhdGEoKTtHb0FjY2Vzcy5zZXRQcmVm
cygpO0dvQWNjZXNzLlRhYmxlcy5pbml0aWFsaXplKCk7fSxnZXRUaGVtZTogZnVuY3Rpb24gKCkg
e3JldHVybiBHb0FjY2Vzcy5BcHBQcmVmcy50aGVtZSB8fCAnZGFya0dyYXknO30sZ2V0TGF5b3V0
OiBmdW5jdGlvbiAoKSB7cmV0dXJuIEdvQWNjZXNzLkFwcFByZWZzLmxheW91dCB8fCAnaG9yaXpv
bnRhbCc7fSxnZXRQZXJQYWdlOiBmdW5jdGlvbiAoKSB7cmV0dXJuIEdvQWNjZXNzLkFwcFByZWZz
LnBlclBhZ2UgfHwgNzt9LHJlbmRlck9wdHM6IGZ1bmN0aW9uICgpIHt2YXIgbyA9IHt9O29bdGhp
cy5nZXRMYXlvdXQoKV0gPSB0cnVlO29bdGhpcy5nZXRUaGVtZSgpXSA9IHRydWU7b1sncGVyUGFn
ZScgKyB0aGlzLmdldFBlclBhZ2UoKV0gPSB0cnVlO29bJ2F1dG9IaWRlVGFibGVzJ10gPSBHb0Fj
Y2Vzcy5UYWJsZXMuYXV0b0hpZGVUYWJsZXMoKTtvWydzaG93VGFibGVzJ10gPSBHb0FjY2Vzcy5U
YWJsZXMuc2hvd1RhYmxlcygpO29bJ2xhYmVscyddID0gR29BY2Nlc3MuaTE4bjskKCcubmF2LWxp
c3QnKS5pbm5lckhUTUwgPSBHb0FjY2Vzcy5BcHBUcGxzLk5hdi5vcHRzLnJlbmRlcihvKTtyZXF1
ZXN0QW5pbWF0aW9uRnJhbWUoZnVuY3Rpb24gKCkgeyQoJ25hdicpLmNsYXNzTGlzdC50b2dnbGUo
J2FjdGl2ZScpO30pO3RoaXMuZXZlbnRzKCk7fSxyZW5kZXJNZW51OiBmdW5jdGlvbiAoZSkgeyQo
Jy5uYXYtbGlzdCcpLmlubmVySFRNTCA9IEdvQWNjZXNzLkFwcFRwbHMuTmF2Lm1lbnUucmVuZGVy
KHsnbmF2JzogdGhpcy5nZXRJdGVtcygpLCdvdmVyYWxsX2N1cnJlbnQnOiB3aW5kb3cubG9jYXRp
b24uaGFzaC5zdWJzdHIoMSkgPT0gJycsJ292ZXJhbGxfaGlkZGVuJzogR29BY2Nlc3MuVXRpbC5p
c1BhbmVsSGlkZGVuKCdnZW5lcmFsJyksJ2xhYmVscyc6IEdvQWNjZXNzLmkxOG4sfSk7cmVxdWVz
dEFuaW1hdGlvbkZyYW1lKGZ1bmN0aW9uICgpIHskKCduYXYnKS5jbGFzc0xpc3QudG9nZ2xlKCdh
Y3RpdmUnKTt9KTt0aGlzLmV2ZW50cygpO30sV1NTdGF0dXM6IGZ1bmN0aW9uICgpIHtpZiAoT2Jq
ZWN0LmtleXMoR29BY2Nlc3MuQXBwV1NDb25uKS5sZW5ndGgpJCQoJy5uYXYtd3Mtc3RhdHVzJywg
ZnVuY3Rpb24gKGl0ZW0pIHsgaXRlbS5zdHlsZS5kaXNwbGF5ID0gJ2Jsb2NrJzsgfSk7fSxXU0Ns
b3NlOiBmdW5jdGlvbiAoKSB7JCQoJy5uYXYtd3Mtc3RhdHVzJywgZnVuY3Rpb24gKGl0ZW0pIHtp
dGVtLmNsYXNzTGlzdC5yZW1vdmUoJ2Nvbm5lY3RlZCcpO2l0ZW0uc2V0QXR0cmlidXRlKCd0aXRs
ZScsICdEaXNjb25uZWN0ZWQnKTt9KTt9LFdTT3BlbjogZnVuY3Rpb24gKHN0cikgeyQkKCcubmF2
LXdzLXN0YXR1cycsIGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5jbGFzc0xpc3QuYWRkKCdjb25uZWN0
ZWQnKTtpdGVtLnNldEF0dHJpYnV0ZSgndGl0bGUnLCAnQ29ubmVjdGVkIHRvICcgKyBzdHIpO30p
O30scmVuZGVyV3JhcDogZnVuY3Rpb24gKG5hdikgeyQoJ25hdicpLmlubmVySFRNTCA9IEdvQWNj
ZXNzLkFwcFRwbHMuTmF2LndyYXAucmVuZGVyKEdvQWNjZXNzLmkxOG4pO30saW5pdGlhbGl6ZTog
ZnVuY3Rpb24gKCkge3RoaXMuc2V0VGhlbWUoR29BY2Nlc3MuQXBwUHJlZnMudGhlbWUpO3RoaXMu
cmVuZGVyV3JhcCgpO3RoaXMuV1NTdGF0dXMoKTt0aGlzLmV2ZW50cygpO319O0dvQWNjZXNzLlBh
bmVscyA9IHtldmVudHM6IGZ1bmN0aW9uICgpIHskJCgnW2RhdGEtdG9nZ2xlPWRyb3Bkb3duXScs
IGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt0aGlzLm9wZW5P
cHRzKGUuY3VycmVudFRhcmdldCk7fS5iaW5kKHRoaXMpO2l0ZW0ub25ibHVyID0gZnVuY3Rpb24g
KGUpIHt0aGlzLmNsb3NlT3B0cyhlKTt9LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTskJCgnW2Rh
dGEtcGxvdF0nLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9IGZ1bmN0aW9uIChlKSB7
R29BY2Nlc3MuQ2hhcnRzLnJlZHJhd0NoYXJ0KGUuY3VycmVudFRhcmdldCk7fS5iaW5kKHRoaXMp
O30uYmluZCh0aGlzKSk7JCQoJ1tkYXRhLWNoYXJ0XScsIGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5v
bmNsaWNrID0gZnVuY3Rpb24gKGUpIHtHb0FjY2Vzcy5DaGFydHMudG9nZ2xlQ2hhcnQoZS5jdXJy
ZW50VGFyZ2V0KTt9LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTskJCgnW2RhdGEtY2hhcnQtdHlw
ZV0nLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9IGZ1bmN0aW9uIChlKSB7R29BY2Nl
c3MuQ2hhcnRzLnNldENoYXJ0VHlwZShlLmN1cnJlbnRUYXJnZXQpO30uYmluZCh0aGlzKTt9LmJp
bmQodGhpcykpOyQkKCdbZGF0YS1tZXRyaWNdJywgZnVuY3Rpb24gKGl0ZW0pIHtpdGVtLm9uY2xp
Y2sgPSBmdW5jdGlvbiAoZSkge0dvQWNjZXNzLlRhYmxlcy50b2dnbGVDb2x1bW4oZS5jdXJyZW50
VGFyZ2V0KTt9LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTt9LG9wZW5PcHRzOiBmdW5jdGlvbiAo
dGFyZykge3ZhciBwYW5lbCA9IHRhcmcuZ2V0QXR0cmlidXRlKCdkYXRhLXBhbmVsJyk7dGFyZy5w
YXJlbnRFbGVtZW50LmNsYXNzTGlzdC50b2dnbGUoJ29wZW4nKTt0aGlzLnJlbmRlck9wdHMocGFu
ZWwpO30sY2xvc2VPcHRzOiBmdW5jdGlvbiAoZSkge2UuY3VycmVudFRhcmdldC5wYXJlbnRFbGVt
ZW50LmNsYXNzTGlzdC5yZW1vdmUoJ29wZW4nKTtpZiAoZS5yZWxhdGVkVGFyZ2V0ICYmIGUucmVs
YXRlZFRhcmdldC5nZXRBdHRyaWJ1dGUoJ2RhdGEtdG9nZ2xlJykgIT09ICdkcm9wZG93bicpZS5y
ZWxhdGVkVGFyZ2V0LmNsaWNrKCk7fSxzZXRQbG90U2VsZWN0aW9uOiBmdW5jdGlvbiAodWksIHBy
ZWZzKSB7dmFyIGNoYXJ0VHlwZSA9ICgocHJlZnMgfHwge30pLnBsb3QgfHwge30pLmNoYXJ0VHlw
ZSB8fCB1aS5wbG90WzBdLmNoYXJ0VHlwZTt2YXIgbWV0cmljID0gKChwcmVmcyB8fCB7fSkucGxv
dCB8fCB7fSkubWV0cmljIHx8IHVpLnBsb3RbMF0uY2xhc3NOYW1lO3VpW2NoYXJ0VHlwZV0gPSB0
cnVlO2ZvciAodmFyIGkgPSAwLCBsZW4gPSB1aS5wbG90Lmxlbmd0aDsgaSA8IGxlbjsgKytpKWlm
ICh1aS5wbG90W2ldLmNsYXNzTmFtZSA9PSBtZXRyaWMpdWkucGxvdFtpXVsnc2VsZWN0ZWQnXSA9
IHRydWU7fSxzZXRDb2xTZWxlY3Rpb246IGZ1bmN0aW9uIChpdGVtcywgcHJlZnMpIHt2YXIgY29s
dW1ucyA9IChwcmVmcyB8fCB7fSkuY29sdW1ucyB8fCB7fTtmb3IgKHZhciBpID0gMCwgbGVuID0g
aXRlbXMubGVuZ3RoOyBpIDwgbGVuOyArK2kpaWYgKChpdGVtc1tpXS5rZXkgaW4gY29sdW1ucykg
JiYgY29sdW1uc1tpdGVtc1tpXS5rZXldWydoaWRlJ10paXRlbXNbaV1bJ2hpZGUnXSA9IHRydWU7
fSxzZXRDaGFydFNlbGVjdGlvbjogZnVuY3Rpb24gKHVpLCBwcmVmcykge3VpWydzaG93Q2hhcnQn
XSA9IHByZWZzICYmICgnY2hhcnQnIGluIHByZWZzKSA/IHByZWZzLmNoYXJ0IDogdHJ1ZTt9LHNl
dE9wdHM6IGZ1bmN0aW9uIChwYW5lbCkge3ZhciB1aSA9IEpTT04ucGFyc2UoSlNPTi5zdHJpbmdp
ZnkoR29BY2Nlc3MuZ2V0UGFuZWxVSShwYW5lbCkpKSwgcHJlZnMgPSBHb0FjY2Vzcy5nZXRQcmVm
cyhwYW5lbCk7dGhpcy5zZXRDaGFydFNlbGVjdGlvbih1aSwgcHJlZnMpO3RoaXMuc2V0UGxvdFNl
bGVjdGlvbih1aSwgcHJlZnMpO3RoaXMuc2V0Q29sU2VsZWN0aW9uKHVpLml0ZW1zLCBwcmVmcyk7
cmV0dXJuIEdvQWNjZXNzLlV0aWwubWVyZ2UodWksIHsnbGFiZWxzJzogR29BY2Nlc3MuaTE4bn0p
O30scmVuZGVyT3B0czogZnVuY3Rpb24gKHBhbmVsKSB7JCgnLnBhbmVsLW9wdHMtJyArIHBhbmVs
KS5pbm5lckhUTUwgPSBHb0FjY2Vzcy5BcHBUcGxzLlBhbmVscy5vcHRzLnJlbmRlcih0aGlzLnNl
dE9wdHMocGFuZWwpKTt0aGlzLmV2ZW50cygpO30sZW5hYmxlUHJldjogZnVuY3Rpb24gKHBhbmVs
KSB7dmFyICRwYWdpbmF0aW9uID0gJCgnI3BhbmVsLScgKyBwYW5lbCArICcgLnBhZ2luYXRpb24g
YS5wYW5lbC1wcmV2Jyk7aWYgKCRwYWdpbmF0aW9uKSRwYWdpbmF0aW9uLnBhcmVudE5vZGUuY2xh
c3NMaXN0LnJlbW92ZSgnZGlzYWJsZWQnKTt9LGRpc2FibGVQcmV2OiBmdW5jdGlvbiAocGFuZWwp
IHt2YXIgJHBhZ2luYXRpb24gPSAkKCcjcGFuZWwtJyArIHBhbmVsICsgJyAucGFnaW5hdGlvbiBh
LnBhbmVsLXByZXYnKTtpZiAoJHBhZ2luYXRpb24pJHBhZ2luYXRpb24ucGFyZW50Tm9kZS5jbGFz
c0xpc3QuYWRkKCdkaXNhYmxlZCcpO30sZW5hYmxlTmV4dDogZnVuY3Rpb24gKHBhbmVsKSB7dmFy
ICRwYWdpbmF0aW9uID0gJCgnI3BhbmVsLScgKyBwYW5lbCArICcgLnBhZ2luYXRpb24gYS5wYW5l
bC1uZXh0Jyk7aWYgKCRwYWdpbmF0aW9uKSRwYWdpbmF0aW9uLnBhcmVudE5vZGUuY2xhc3NMaXN0
LnJlbW92ZSgnZGlzYWJsZWQnKTt9LGRpc2FibGVOZXh0OiBmdW5jdGlvbiAocGFuZWwpIHt2YXIg
JHBhZ2luYXRpb24gPSAkKCcjcGFuZWwtJyArIHBhbmVsICsgJyAucGFnaW5hdGlvbiBhLnBhbmVs
LW5leHQnKTtpZiAoJHBhZ2luYXRpb24pJHBhZ2luYXRpb24ucGFyZW50Tm9kZS5jbGFzc0xpc3Qu
YWRkKCdkaXNhYmxlZCcpO30sZW5hYmxlRmlyc3Q6IGZ1bmN0aW9uIChwYW5lbCkge3ZhciAkcGFn
aW5hdGlvbiA9ICQoJyNwYW5lbC0nICsgcGFuZWwgKyAnIC5wYWdpbmF0aW9uIGEucGFuZWwtZmly
c3QnKTtpZiAoJHBhZ2luYXRpb24pJHBhZ2luYXRpb24ucGFyZW50Tm9kZS5jbGFzc0xpc3QucmVt
b3ZlKCdkaXNhYmxlZCcpO30sZGlzYWJsZUZpcnN0OiBmdW5jdGlvbiAocGFuZWwpIHt2YXIgJHBh
Z2luYXRpb24gPSAkKCcjcGFuZWwtJyArIHBhbmVsICsgJyAucGFnaW5hdGlvbiBhLnBhbmVsLWZp
cnN0Jyk7aWYgKCRwYWdpbmF0aW9uKSRwYWdpbmF0aW9uLnBhcmVudE5vZGUuY2xhc3NMaXN0LmFk
ZCgnZGlzYWJsZWQnKTt9LGVuYWJsZUxhc3Q6IGZ1bmN0aW9uIChwYW5lbCkge3ZhciAkcGFnaW5h
dGlvbiA9ICQoJyNwYW5lbC0nICsgcGFuZWwgKyAnIC5wYWdpbmF0aW9uIGEucGFuZWwtbGFzdCcp
O2lmICgkcGFnaW5hdGlvbikkcGFnaW5hdGlvbi5wYXJlbnROb2RlLmNsYXNzTGlzdC5yZW1vdmUo
J2Rpc2FibGVkJyk7fSxkaXNhYmxlTGFzdDogZnVuY3Rpb24gKHBhbmVsKSB7dmFyICRwYWdpbmF0
aW9uID0gJCgnI3BhbmVsLScgKyBwYW5lbCArICcgLnBhZ2luYXRpb24gYS5wYW5lbC1sYXN0Jyk7
aWYgKCRwYWdpbmF0aW9uKSRwYWdpbmF0aW9uLnBhcmVudE5vZGUuY2xhc3NMaXN0LmFkZCgnZGlz
YWJsZWQnKTt9LGVuYWJsZVBhZ2luYXRpb246IGZ1bmN0aW9uIChwYW5lbCkge3RoaXMuZW5hYmxl
UHJldihwYW5lbCk7dGhpcy5lbmFibGVOZXh0KHBhbmVsKTt0aGlzLmVuYWJsZUZpcnN0KHBhbmVs
KTt0aGlzLmVuYWJsZUxhc3QocGFuZWwpO30sZGlzYWJsZVBhZ2luYXRpb246IGZ1bmN0aW9uIChw
YW5lbCkge3RoaXMuZGlzYWJsZVByZXYocGFuZWwpO3RoaXMuZGlzYWJsZU5leHQocGFuZWwpO3Ro
aXMuZGlzYWJsZUZpcnN0KHBhbmVsKTt0aGlzLmRpc2FibGVMYXN0KHBhbmVsKTt9LGhhc1N1Ykl0
ZW1zOiBmdW5jdGlvbiAodWksIGRhdGEpIHtmb3IgKHZhciBpID0gMCwgbGVuID0gZGF0YS5sZW5n
dGg7IGkgPCBsZW47ICsraSkge2lmICghZGF0YVtpXS5pdGVtcylyZXR1cm4gKHVpWydoYXNTdWJJ
dGVtcyddID0gZmFsc2UpO2lmIChkYXRhW2ldLml0ZW1zLmxlbmd0aCkge3JldHVybiAodWlbJ2hh
c1N1Ykl0ZW1zJ10gPSB0cnVlKTt9fXJldHVybiBmYWxzZTt9LHNldENvbXB1dGVkRGF0YTogZnVu
Y3Rpb24gKHBhbmVsLCB1aSwgZGF0YSkge3RoaXMuaGFzU3ViSXRlbXModWksIGRhdGEuZGF0YSk7
R29BY2Nlc3MuQ2hhcnRzLmhhc0NoYXJ0KHBhbmVsLCB1aSk7R29BY2Nlc3MuVGFibGVzLmhhc1Rh
YmxlKHVpKTt9LHJlbmRlclBhbmVsOiBmdW5jdGlvbiAocGFuZWwsIHVpLCBjb2wpIHt2YXIgZGF0
YSA9IEdvQWNjZXNzLmdldFBhbmVsRGF0YShwYW5lbCk7dGhpcy5zZXRDb21wdXRlZERhdGEocGFu
ZWwsIHVpLCBkYXRhKTt2YXIgYm94ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnZGl2Jyk7Ym94
LmlkID0gJ3BhbmVsLScgKyBwYW5lbDtib3guaW5uZXJIVE1MID0gR29BY2Nlc3MuQXBwVHBscy5Q
YW5lbHMud3JhcC5yZW5kZXIoR29BY2Nlc3MuVXRpbC5tZXJnZSh1aSwgeydsYWJlbHMnOiBHb0Fj
Y2Vzcy5pMThufSkpO2NvbC5hcHBlbmRDaGlsZChib3gpO2lmIChkYXRhLmRhdGEubGVuZ3RoIDw9
IEdvQWNjZXNzLmdldFByZWZzKCkucGVyUGFnZSl0aGlzLmRpc2FibGVQYWdpbmF0aW9uKHBhbmVs
KTtHb0FjY2Vzcy5UYWJsZXMucmVuZGVyVGhlYWQocGFuZWwsIHVpKTtyZXR1cm4gY29sO30sY3Jl
YXRlQ29sOiBmdW5jdGlvbiAocm93KSB7dmFyIGxheW91dCA9IEdvQWNjZXNzLkFwcFByZWZzWyds
YXlvdXQnXTt2YXIgcGVyUm93ID0gJ2hvcml6b250YWwnID09IGxheW91dCA/IDYgOiAnd2lkZScg
PT0gbGF5b3V0ID8gMyA6IDEyO3ZhciBjb2wgPSBkb2N1bWVudC5jcmVhdGVFbGVtZW50KCdkaXYn
KTtjb2wuc2V0QXR0cmlidXRlKCdjbGFzcycsICdjb2wtbWQtJyArIHBlclJvdyArICcgd3JhcC1w
YW5lbCcpO3Jvdy5hcHBlbmRDaGlsZChjb2wpO3JldHVybiBjb2w7fSxjcmVhdGVSb3c6IGZ1bmN0
aW9uIChyb3csIGlkeCkge3ZhciB3cmFwID0gJCgnLndyYXAtcGFuZWxzJyk7dmFyIGxheW91dCA9
IEdvQWNjZXNzLkFwcFByZWZzWydsYXlvdXQnXTt2YXIgZXZlcnkgPSAnaG9yaXpvbnRhbCcgPT0g
bGF5b3V0ID8gMiA6ICd3aWRlJyA9PSBsYXlvdXQgPyA0IDogMTtpZiAoaWR4ICUgZXZlcnkgPT0g
MCkge3JvdyA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2RpdicpO3Jvdy5zZXRBdHRyaWJ1dGUo
J2NsYXNzJywgJ3JvdycgKyAoZXZlcnkgPT0gMiB8fCBldmVyeSA9PSA0ID8gJyBlcXVhbCcgOiAn
JykpO3dyYXAuYXBwZW5kQ2hpbGQocm93KTt9cmV0dXJuIHJvdzt9LHJlc2V0UGFuZWw6IGZ1bmN0
aW9uIChwYW5lbCkge3ZhciB1aSA9IEdvQWNjZXNzLmdldFBhbmVsVUkoKTt2YXIgZWxlID0gJCgn
I3BhbmVsLScgKyBwYW5lbCk7aWYgKEdvQWNjZXNzLlV0aWwuaXNQYW5lbFZhbGlkKHBhbmVsKSB8
fCBHb0FjY2Vzcy5VdGlsLmlzUGFuZWxIaWRkZW4ocGFuZWwpKXJldHVybiBmYWxzZTt2YXIgY29s
ID0gZWxlLnBhcmVudE5vZGU7Y29sLnJlbW92ZUNoaWxkKGVsZSk7dGhpcy5yZW5kZXJQYW5lbChw
YW5lbCwgdWlbcGFuZWxdLCBjb2wpO3RoaXMuZXZlbnRzKCk7fSxyZW5kZXJQYW5lbHM6IGZ1bmN0
aW9uICgpIHt2YXIgdWkgPSBHb0FjY2Vzcy5nZXRQYW5lbFVJKCksIGlkeCA9IDAsIHJvdyA9IG51
bGwsIGNvbCA9IG51bGw7JCgnLndyYXAtcGFuZWxzJykuaW5uZXJIVE1MID0gJyc7Zm9yICh2YXIg
cGFuZWwgaW4gdWkpIHtpZiAoR29BY2Nlc3MuVXRpbC5pc1BhbmVsVmFsaWQocGFuZWwpIHx8IEdv
QWNjZXNzLlV0aWwuaXNQYW5lbEhpZGRlbihwYW5lbCkpY29udGludWU7cm93ID0gdGhpcy5jcmVh
dGVSb3cocm93LCBpZHgrKyk7Y29sID0gdGhpcy5jcmVhdGVDb2wocm93KTt0aGlzLnJlbmRlclBh
bmVsKHBhbmVsLCB1aVtwYW5lbF0sIGNvbCk7fX0saW5pdGlhbGl6ZTogZnVuY3Rpb24gKCkge3Ro
aXMucmVuZGVyUGFuZWxzKCk7dGhpcy5ldmVudHMoKTt9fTtHb0FjY2Vzcy5DaGFydHMgPSB7aXRl
cjogZnVuY3Rpb24gKGNhbGxiYWNrKSB7T2JqZWN0LmtleXMoR29BY2Nlc3MuQXBwQ2hhcnRzKS5m
b3JFYWNoKGZ1bmN0aW9uIChwYW5lbCkge2lmICghR29BY2Nlc3MuVXRpbC5pc1dpdGhpblZpZXdQ
b3J0KCQoJyNwYW5lbC0nICsgcGFuZWwpKSlyZXR1cm47aWYgKGNhbGxiYWNrICYmIHR5cGVvZiBj
YWxsYmFjayA9PT0gJ2Z1bmN0aW9uJyljYWxsYmFjay5jYWxsKHRoaXMsIEdvQWNjZXNzLkFwcENo
YXJ0c1twYW5lbF0sIHBhbmVsKTt9KTt9LGdldE1ldHJpY0tleXM6IGZ1bmN0aW9uIChwYW5lbCwg
a2V5KSB7cmV0dXJuIEdvQWNjZXNzLmdldFBhbmVsVUkocGFuZWwpWydpdGVtcyddLm1hcChmdW5j
dGlvbiAoYSkgeyByZXR1cm4gYVtrZXldOyB9KTt9LGdldFBhbmVsRGF0YTogZnVuY3Rpb24gKHBh
bmVsLCBkYXRhKSB7dmFyIHBsb3QgPSBHb0FjY2Vzcy5VdGlsLmdldFByb3AoR29BY2Nlc3MuQXBw
U3RhdGUsIHBhbmVsICsgJy5wbG90Jyk7ZGF0YSA9IGRhdGEgfHwgdGhpcy5wcm9jZXNzQ2hhcnRE
YXRhKEdvQWNjZXNzLmdldFBhbmVsRGF0YShwYW5lbCkuZGF0YSk7cmV0dXJuIHBsb3QuY2hhcnRS
ZXZlcnNlID8gZGF0YS5yZXZlcnNlKCkgOiBkYXRhO30sZHJhd1Bsb3Q6IGZ1bmN0aW9uIChwYW5l
bCwgcGxvdFVJLCBkYXRhKSB7dmFyIGNoYXJ0ID0gdGhpcy5nZXRDaGFydChwYW5lbCwgcGxvdFVJ
LCBkYXRhKTtpZiAoIWNoYXJ0KXJldHVybjt0aGlzLnJlbmRlckNoYXJ0KHBhbmVsLCBjaGFydCwg
ZGF0YSk7R29BY2Nlc3MuQXBwQ2hhcnRzW3BhbmVsXSA9IG51bGw7R29BY2Nlc3MuQXBwQ2hhcnRz
W3BhbmVsXSA9IGNoYXJ0O30sc2V0Q2hhcnRUeXBlOiBmdW5jdGlvbiAodGFyZykge3ZhciBwYW5l
bCA9IHRhcmcuZ2V0QXR0cmlidXRlKCdkYXRhLXBhbmVsJyk7dmFyIHR5cGUgPSB0YXJnLmdldEF0
dHJpYnV0ZSgnZGF0YS1jaGFydC10eXBlJyk7R29BY2Nlc3MuVXRpbC5zZXRQcm9wKEdvQWNjZXNz
LkFwcFByZWZzLCBwYW5lbCArICcucGxvdC5jaGFydFR5cGUnLCB0eXBlKTtHb0FjY2Vzcy5zZXRQ
cmVmcygpO3ZhciBwbG90VUkgPSBHb0FjY2Vzcy5VdGlsLmdldFByb3AoR29BY2Nlc3MuQXBwU3Rh
dGUsIHBhbmVsICsgJy5wbG90Jyk7dGhpcy5kcmF3UGxvdChwYW5lbCwgcGxvdFVJLCB0aGlzLmdl
dFBhbmVsRGF0YShwYW5lbCkpO30sdG9nZ2xlQ2hhcnQ6IGZ1bmN0aW9uICh0YXJnKSB7dmFyIHBh
bmVsID0gdGFyZy5nZXRBdHRyaWJ1dGUoJ2RhdGEtcGFuZWwnKTt2YXIgcHJlZnMgPSBHb0FjY2Vz
cy5nZXRQcmVmcyhwYW5lbCksY2hhcnQgPSBwcmVmcyAmJiAoJ2NoYXJ0JyBpbiBwcmVmcykgPyBw
cmVmcy5jaGFydCA6IHRydWU7R29BY2Nlc3MuVXRpbC5zZXRQcm9wKEdvQWNjZXNzLkFwcFByZWZz
LCBwYW5lbCArICcuY2hhcnQnLCAhY2hhcnQpO0dvQWNjZXNzLnNldFByZWZzKCk7R29BY2Nlc3Mu
UGFuZWxzLnJlc2V0UGFuZWwocGFuZWwpO0dvQWNjZXNzLkNoYXJ0cy5yZXNldENoYXJ0KHBhbmVs
KTtHb0FjY2Vzcy5UYWJsZXMucmVuZGVyRnVsbFRhYmxlKHBhbmVsKTt9LGhhc0NoYXJ0OiBmdW5j
dGlvbiAocGFuZWwsIHVpKSB7dmFyIHByZWZzID0gR29BY2Nlc3MuZ2V0UHJlZnMocGFuZWwpLGNo
YXJ0ID0gcHJlZnMgJiYgKCdjaGFydCcgaW4gcHJlZnMpID8gcHJlZnMuY2hhcnQgOiB0cnVlO3Vp
WydjaGFydCddID0gdWkucGxvdC5sZW5ndGggJiYgY2hhcnQgJiYgY2hhcnQ7fSxyZWRyYXdDaGFy
dDogZnVuY3Rpb24gKHRhcmcpIHt2YXIgcGxvdCA9IHRhcmcuZ2V0QXR0cmlidXRlKCdkYXRhLXBs
b3QnKTt2YXIgcGFuZWwgPSB0YXJnLmdldEF0dHJpYnV0ZSgnZGF0YS1wYW5lbCcpO3ZhciB1aSA9
IEdvQWNjZXNzLmdldFBhbmVsVUkocGFuZWwpO3ZhciBwbG90VUkgPSB1aS5wbG90O0dvQWNjZXNz
LlV0aWwuc2V0UHJvcChHb0FjY2Vzcy5BcHBQcmVmcywgcGFuZWwgKyAnLnBsb3QubWV0cmljJywg
cGxvdCk7R29BY2Nlc3Muc2V0UHJlZnMoKTtmb3IgKHZhciB4IGluIHBsb3RVSSkge2lmICghcGxv
dFVJLmhhc093blByb3BlcnR5KHgpIHx8IHBsb3RVSVt4XS5jbGFzc05hbWUgIT0gcGxvdCljb250
aW51ZTtHb0FjY2Vzcy5VdGlsLnNldFByb3AoR29BY2Nlc3MuQXBwU3RhdGUsIHBhbmVsICsgJy5w
bG90JywgcGxvdFVJW3hdKTt0aGlzLmRyYXdQbG90KHBhbmVsLCBwbG90VUlbeF0sIHRoaXMuZ2V0
UGFuZWxEYXRhKHBhbmVsKSk7YnJlYWs7fX0sZXh0cmFjdENvdW50OiBmdW5jdGlvbiAoaXRlbSkg
e3ZhciBvID0ge307Zm9yICh2YXIgcHJvcCBpbiBpdGVtKW9bcHJvcF0gPSBHb0FjY2Vzcy5VdGls
LmdldENvdW50KGl0ZW1bcHJvcF0pO3JldHVybiBvO30scHJvY2Vzc0NoYXJ0RGF0YTogZnVuY3Rp
b24gKGRhdGEpIHt2YXIgb3V0ID0gW107Zm9yICh2YXIgaSA9IDA7IGkgPCBkYXRhLmxlbmd0aDsg
KytpKW91dC5wdXNoKHRoaXMuZXh0cmFjdENvdW50KGRhdGFbaV0pKTtyZXR1cm4gb3V0O30sZmlu
ZFVJSXRlbTogZnVuY3Rpb24gKHBhbmVsLCBrZXkpIHt2YXIgaXRlbXMgPSBHb0FjY2Vzcy5nZXRQ
YW5lbFVJKHBhbmVsKS5pdGVtcztmb3IgKHZhciBpID0gMDsgaSA8IGl0ZW1zLmxlbmd0aDsgKytp
KSB7aWYgKGl0ZW1zW2ldLmtleSA9PSBrZXkpcmV0dXJuIGl0ZW1zW2ldO31yZXR1cm4gbnVsbDt9
LGdldFhLZXk6IGZ1bmN0aW9uIChkYXR1bSwga2V5KSB7dmFyIGFyciA9IFtdO2lmICh0eXBlb2Yg
a2V5ID09PSAnc3RyaW5nJylyZXR1cm4gZGF0dW1ba2V5XTtmb3IgKHZhciBwcm9wIGluIGtleSlh
cnIucHVzaChkYXR1bVtrZXlbcHJvcF1dKTtyZXR1cm4gYXJyLmpvaW4oJyAnKTt9LGdldEFyZWFT
cGxpbmU6IGZ1bmN0aW9uIChwYW5lbCwgcGxvdFVJLCBkYXRhKSB7dmFyIGR1YWxZYXhpcyA9IHBs
b3RVSVsnZDMnXVsneTEnXTt2YXIgY2hhcnQgPSBBcmVhQ2hhcnQoZHVhbFlheGlzKS5sYWJlbHMo
e3kwOiBwbG90VUlbJ2QzJ11bJ3kwJ10ubGFiZWwseTE6IGR1YWxZYXhpcyA/IHBsb3RVSVsnZDMn
XVsneTEnXS5sYWJlbCA6ICcnfSkueChmdW5jdGlvbiAoZCkge2lmICgoKChwbG90VUkgfHwge30p
LmQzIHx8IHt9KS54IHx8IHt9KS5rZXkpcmV0dXJuIHRoaXMuZ2V0WEtleShkLCBwbG90VUlbJ2Qz
J11bJ3gnXVsna2V5J10pO3JldHVybiBkLmRhdGE7fS5iaW5kKHRoaXMpKS55MChmdW5jdGlvbiAo
ZCkge3JldHVybiArZFtwbG90VUlbJ2QzJ11bJ3kwJ11bJ2tleSddXTt9KS53aWR0aCgkKCIjY2hh
cnQtIiArIHBhbmVsKS5nZXRCb3VuZGluZ0NsaWVudFJlY3QoKS53aWR0aCkuaGVpZ2h0KDE3NSku
Zm9ybWF0KHt4OiAodGhpcy5maW5kVUlJdGVtKHBhbmVsLCAnZGF0YScpIHx8IHt9KS5kYXRhVHlw
ZSB8fCBudWxsLHkwOiAoKHBsb3RVSS5kMyB8fCB7fSkueTAgfHwge30pLmZvcm1hdCx5MTogKChw
bG90VUkuZDMgfHwge30pLnkxIHx8IHt9KS5mb3JtYXQsfSkub3B0cyhwbG90VUkpO2R1YWxZYXhp
cyAmJiBjaGFydC55MShmdW5jdGlvbiAoZCkge3JldHVybiArZFtwbG90VUlbJ2QzJ11bJ3kxJ11b
J2tleSddXTt9KTtyZXR1cm4gY2hhcnQ7fSxnZXRWQmFyOiBmdW5jdGlvbiAocGFuZWwsIHBsb3RV
SSwgZGF0YSkge3ZhciBkdWFsWWF4aXMgPSBwbG90VUlbJ2QzJ11bJ3kxJ107dmFyIGNoYXJ0ID0g
QmFyQ2hhcnQoZHVhbFlheGlzKS5sYWJlbHMoe3kwOiBwbG90VUlbJ2QzJ11bJ3kwJ10ubGFiZWws
eTE6IGR1YWxZYXhpcyA/IHBsb3RVSVsnZDMnXVsneTEnXS5sYWJlbCA6ICcnfSkueChmdW5jdGlv
biAoZCkge2lmICgoKChwbG90VUkgfHwge30pLmQzIHx8IHt9KS54IHx8IHt9KS5rZXkpcmV0dXJu
IHRoaXMuZ2V0WEtleShkLCBwbG90VUlbJ2QzJ11bJ3gnXVsna2V5J10pO3JldHVybiBkLmRhdGE7
fS5iaW5kKHRoaXMpKS55MChmdW5jdGlvbiAoZCkge3JldHVybiArZFtwbG90VUlbJ2QzJ11bJ3kw
J11bJ2tleSddXTt9KS53aWR0aCgkKCIjY2hhcnQtIiArIHBhbmVsKS5nZXRCb3VuZGluZ0NsaWVu
dFJlY3QoKS53aWR0aCkuaGVpZ2h0KDE3NSkuZm9ybWF0KHt4OiAodGhpcy5maW5kVUlJdGVtKHBh
bmVsLCAnZGF0YScpIHx8IHt9KS5kYXRhVHlwZSB8fCBudWxsLHkwOiAoKHBsb3RVSS5kMyB8fCB7
fSkueTAgfHwge30pLmZvcm1hdCx5MTogKChwbG90VUkuZDMgfHwge30pLnkxIHx8IHt9KS5mb3Jt
YXQsfSkub3B0cyhwbG90VUkpO2R1YWxZYXhpcyAmJiBjaGFydC55MShmdW5jdGlvbiAoZCkge3Jl
dHVybiArZFtwbG90VUlbJ2QzJ11bJ3kxJ11bJ2tleSddXTt9KTtyZXR1cm4gY2hhcnQ7fSxnZXRD
aGFydFR5cGU6IGZ1bmN0aW9uIChwYW5lbCkge3ZhciB1aSA9IEdvQWNjZXNzLmdldFBhbmVsVUko
cGFuZWwpO2lmICghdWkuY2hhcnQpcmV0dXJuICcnO3JldHVybiBHb0FjY2Vzcy5VdGlsLmdldFBy
b3AoR29BY2Nlc3MuZ2V0UHJlZnMoKSwgcGFuZWwgKyAnLnBsb3QuY2hhcnRUeXBlJykgfHwgdWku
cGxvdFswXS5jaGFydFR5cGU7fSxnZXRQbG90VUk6IGZ1bmN0aW9uIChwYW5lbCwgdWkpIHt2YXIg
bWV0cmljID0gR29BY2Nlc3MuVXRpbC5nZXRQcm9wKEdvQWNjZXNzLmdldFByZWZzKCksIHBhbmVs
ICsgJy5wbG90Lm1ldHJpYycpO2lmICghbWV0cmljKXJldHVybiB1aS5wbG90WzBdO3JldHVybiB1
aS5wbG90LmZpbHRlcihmdW5jdGlvbiAodikge3JldHVybiB2LmNsYXNzTmFtZSA9PSBtZXRyaWM7
fSlbMF07fSxnZXRDaGFydDogZnVuY3Rpb24gKHBhbmVsLCBwbG90VUksIGRhdGEpIHt2YXIgY2hh
cnQgPSBudWxsO3N3aXRjaCAodGhpcy5nZXRDaGFydFR5cGUocGFuZWwpKSB7Y2FzZSAnYXJlYS1z
cGxpbmUnOmNoYXJ0ID0gdGhpcy5nZXRBcmVhU3BsaW5lKHBhbmVsLCBwbG90VUksIGRhdGEpO2Jy
ZWFrO2Nhc2UgJ2Jhcic6Y2hhcnQgPSB0aGlzLmdldFZCYXIocGFuZWwsIHBsb3RVSSwgZGF0YSk7
YnJlYWs7fXJldHVybiBjaGFydDt9LHJlbmRlckNoYXJ0OiBmdW5jdGlvbiAocGFuZWwsIGNoYXJ0
LCBkYXRhKSB7ZDMuc2VsZWN0KCcjY2hhcnQtJyArIHBhbmVsICsgJz4uY2hhcnQtdG9vbHRpcC13
cmFwJykucmVtb3ZlKCk7ZDMuc2VsZWN0KCcjY2hhcnQtJyArIHBhbmVsKS5zZWxlY3QoJ3N2Zycp
LnJlbW92ZSgpO2QzLnNlbGVjdCgiI2NoYXJ0LSIgKyBwYW5lbCkuZGF0dW0oZGF0YSkuY2FsbChj
aGFydCkuYXBwZW5kKCJkaXYiKS5hdHRyKCJjbGFzcyIsICJjaGFydC10b29sdGlwLXdyYXAiKTt9
LGFkZENoYXJ0OiBmdW5jdGlvbiAocGFuZWwsIHVpKSB7dmFyIHBsb3RVSSA9IG51bGwsIGNoYXJ0
ID0gbnVsbDtpZiAoIXVpLnBsb3QgfHwgIXVpLnBsb3QubGVuZ3RoKXJldHVybjtwbG90VUkgPSB0
aGlzLmdldFBsb3RVSShwYW5lbCwgdWkpO0dvQWNjZXNzLlV0aWwuc2V0UHJvcChHb0FjY2Vzcy5B
cHBTdGF0ZSwgcGFuZWwgKyAnLnBsb3QnLCBwbG90VUkpO3ZhciBkYXRhID0gdGhpcy5nZXRQYW5l
bERhdGEocGFuZWwpO2lmICghKGNoYXJ0ID0gdGhpcy5nZXRDaGFydChwYW5lbCwgcGxvdFVJLCBk
YXRhKSkpcmV0dXJuO3RoaXMucmVuZGVyQ2hhcnQocGFuZWwsIGNoYXJ0LCBkYXRhKTtHb0FjY2Vz
cy5BcHBDaGFydHNbcGFuZWxdID0gY2hhcnQ7fSxyZW5kZXJDaGFydHM6IGZ1bmN0aW9uICh1aSkg
e2ZvciAodmFyIHBhbmVsIGluIHVpKSB7aWYgKEdvQWNjZXNzLlV0aWwuaXNQYW5lbFZhbGlkKHBh
bmVsKSB8fCBHb0FjY2Vzcy5VdGlsLmlzUGFuZWxIaWRkZW4ocGFuZWwpKWNvbnRpbnVlO3RoaXMu
YWRkQ2hhcnQocGFuZWwsIHVpW3BhbmVsXSk7fX0scmVzZXRDaGFydDogZnVuY3Rpb24gKHBhbmVs
KSB7dmFyIHVpID0ge307aWYgKEdvQWNjZXNzLlV0aWwuaXNQYW5lbFZhbGlkKHBhbmVsKSB8fCBH
b0FjY2Vzcy5VdGlsLmlzUGFuZWxIaWRkZW4ocGFuZWwpKXJldHVybiBmYWxzZTt1aSA9IEdvQWNj
ZXNzLmdldFBhbmVsVUkocGFuZWwpO3RoaXMuYWRkQ2hhcnQocGFuZWwsIHVpKTt9LHJlbG9hZENo
YXJ0OiBmdW5jdGlvbiAoY2hhcnQsIHBhbmVsKSB7dmFyIHN1Ykl0ZW1zID0gR29BY2Nlc3MuVGFi
bGVzLmdldFN1Ykl0ZW1zRGF0YShwYW5lbCk7dmFyIGRhdGEgPSAoc3ViSXRlbXMubGVuZ3RoID8g
c3ViSXRlbXMgOiBHb0FjY2Vzcy5nZXRQYW5lbERhdGEocGFuZWwpLmRhdGEpLnNsaWNlKDApO2Qz
LnNlbGVjdCgiI2NoYXJ0LSIgKyBwYW5lbCkuZGF0dW0odGhpcy5wcm9jZXNzQ2hhcnREYXRhKHRo
aXMuZ2V0UGFuZWxEYXRhKHBhbmVsLCBkYXRhKSkpLmNhbGwoY2hhcnQud2lkdGgoJCgiI2NoYXJ0
LSIgKyBwYW5lbCkub2Zmc2V0V2lkdGgpKS5hcHBlbmQoImRpdiIpLmF0dHIoImNsYXNzIiwgImNo
YXJ0LXRvb2x0aXAtd3JhcCIpO30scmVsb2FkQ2hhcnRzOiBmdW5jdGlvbiAoKSB7dGhpcy5pdGVy
KGZ1bmN0aW9uIChjaGFydCwgcGFuZWwpIHt0aGlzLnJlbG9hZENoYXJ0KGNoYXJ0LCBwYW5lbCk7
fS5iaW5kKHRoaXMpKTtHb0FjY2Vzcy5BcHBTdGF0ZS51cGRhdGVkID0gZmFsc2U7fSxyZWRyYXdD
aGFydHM6IGZ1bmN0aW9uICgpIHt0aGlzLml0ZXIoZnVuY3Rpb24gKGNoYXJ0LCBwYW5lbCkge2Qz
LnNlbGVjdCgiI2NoYXJ0LSIgKyBwYW5lbCkuY2FsbChjaGFydC53aWR0aCgkKCIjY2hhcnQtIiAr
IHBhbmVsKS5vZmZzZXRXaWR0aCkpO30pO30saW5pdGlhbGl6ZTogZnVuY3Rpb24gKCkge3RoaXMu
cmVuZGVyQ2hhcnRzKEdvQWNjZXNzLmdldFBhbmVsVUkoKSk7ZDMuc2VsZWN0KHdpbmRvdykub24o
J3Njcm9sbC5jaGFydHMnLCBkZWJvdW5jZShmdW5jdGlvbiAoKSB7dGhpcy5yZWxvYWRDaGFydHMo
KTt9LCAyNTAsIGZhbHNlKS5iaW5kKHRoaXMpKS5vbigncmVzaXplLmNoYXJ0cycsIGZ1bmN0aW9u
ICgpIHt0aGlzLnJlZHJhd0NoYXJ0cygpO30uYmluZCh0aGlzKSk7fX07R29BY2Nlc3MuVGFibGVz
ID0ge2NoYXJ0RGF0YToge30sZXZlbnRzOiBmdW5jdGlvbiAoKSB7JCQoJy5wYW5lbC1uZXh0Jywg
ZnVuY3Rpb24gKGl0ZW0pIHtpdGVtLm9uY2xpY2sgPSBmdW5jdGlvbiAoZSkge3ZhciBwYW5lbCA9
IGUuY3VycmVudFRhcmdldC5nZXRBdHRyaWJ1dGUoJ2RhdGEtcGFuZWwnKTt0aGlzLnJlbmRlclRh
YmxlKHBhbmVsLCB0aGlzLm5leHRQYWdlKHBhbmVsKSk7fS5iaW5kKHRoaXMpO30uYmluZCh0aGlz
KSk7JCQoJy5wYW5lbC1wcmV2JywgZnVuY3Rpb24gKGl0ZW0pIHtpdGVtLm9uY2xpY2sgPSBmdW5j
dGlvbiAoZSkge3ZhciBwYW5lbCA9IGUuY3VycmVudFRhcmdldC5nZXRBdHRyaWJ1dGUoJ2RhdGEt
cGFuZWwnKTt0aGlzLnJlbmRlclRhYmxlKHBhbmVsLCB0aGlzLnByZXZQYWdlKHBhbmVsKSk7fS5i
aW5kKHRoaXMpO30uYmluZCh0aGlzKSk7JCQoJy5wYW5lbC1maXJzdCcsIGZ1bmN0aW9uIChpdGVt
KSB7aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt2YXIgcGFuZWwgPSBlLmN1cnJlbnRUYXJn
ZXQuZ2V0QXR0cmlidXRlKCdkYXRhLXBhbmVsJyk7dGhpcy5yZW5kZXJUYWJsZShwYW5lbCwgIkZJ
UlNUX1BBR0UiKTt9LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTskJCgnLnBhbmVsLWxhc3QnLCBm
dW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9IGZ1bmN0aW9uIChlKSB7dmFyIHBhbmVsID0g
ZS5jdXJyZW50VGFyZ2V0LmdldEF0dHJpYnV0ZSgnZGF0YS1wYW5lbCcpO3RoaXMucmVuZGVyVGFi
bGUocGFuZWwsICJMQVNUX1BBR0UiKTt9LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTskJCgnLmV4
cGFuZGFibGU+dGQnLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9IGZ1bmN0aW9uIChl
KSB7aWYgKCF3aW5kb3cuZ2V0U2VsZWN0aW9uKCkudG9TdHJpbmcoKSl0aGlzLnRvZ2dsZVJvdyhl
LmN1cnJlbnRUYXJnZXQpO30uYmluZCh0aGlzKTt9LmJpbmQodGhpcykpOyQkKCcucm93LWV4cGFu
ZGFibGUuY2xpY2thYmxlJywgZnVuY3Rpb24gKGl0ZW0pIHtpdGVtLm9uY2xpY2sgPSBmdW5jdGlv
biAoZSkge3RoaXMudG9nZ2xlUm93KGUuY3VycmVudFRhcmdldCk7fS5iaW5kKHRoaXMpO30uYmlu
ZCh0aGlzKSk7JCQoJy5zb3J0YWJsZScsIGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5vbmNsaWNrID0g
ZnVuY3Rpb24gKGUpIHt0aGlzLnNvcnRDb2x1bW4oZS5jdXJyZW50VGFyZ2V0KTt9LmJpbmQodGhp
cyk7fS5iaW5kKHRoaXMpKTt9LHRvZ2dsZUNvbHVtbjogZnVuY3Rpb24gKHRhcmcpIHt2YXIgcGFu
ZWwgPSB0YXJnLmdldEF0dHJpYnV0ZSgnZGF0YS1wYW5lbCcpO3ZhciBtZXRyaWMgPSB0YXJnLmdl
dEF0dHJpYnV0ZSgnZGF0YS1tZXRyaWMnKTt2YXIgY29sdW1ucyA9IChHb0FjY2Vzcy5nZXRQcmVm
cyhwYW5lbCkgfHwge30pLmNvbHVtbnMgfHwge307aWYgKG1ldHJpYyBpbiBjb2x1bW5zKSB7ZGVs
ZXRlIGNvbHVtbnNbbWV0cmljXTt9IGVsc2Uge0dvQWNjZXNzLlV0aWwuc2V0UHJvcChjb2x1bW5z
LCBtZXRyaWMgKyAnLmhpZGUnLCB0cnVlKTt9R29BY2Nlc3MuVXRpbC5zZXRQcm9wKEdvQWNjZXNz
LkFwcFByZWZzLCBwYW5lbCArICcuY29sdW1ucycsIGNvbHVtbnMpO0dvQWNjZXNzLnNldFByZWZz
KCk7R29BY2Nlc3MuVGFibGVzLnJlbmRlclRoZWFkKHBhbmVsLCBHb0FjY2Vzcy5nZXRQYW5lbFVJ
KHBhbmVsKSk7R29BY2Nlc3MuVGFibGVzLnJlbmRlckZ1bGxUYWJsZShwYW5lbCk7fSxzb3J0Q29s
dW1uOiBmdW5jdGlvbiAoZWxlKSB7dmFyIGZpZWxkID0gZWxlLmdldEF0dHJpYnV0ZSgnZGF0YS1r
ZXknKTt2YXIgb3JkZXIgPSBlbGUuZ2V0QXR0cmlidXRlKCdkYXRhLW9yZGVyJyk7dmFyIHBhbmVs
ID0gZWxlLnBhcmVudEVsZW1lbnQucGFyZW50RWxlbWVudC5wYXJlbnRFbGVtZW50LmdldEF0dHJp
YnV0ZSgnZGF0YS1wYW5lbCcpO29yZGVyID0gb3JkZXIgPyAnYXNjJyA9PSBvcmRlciA/ICdkZXNj
JyA6ICdhc2MnIDogJ2FzYyc7R29BY2Nlc3MuQXBwLnNvcnREYXRhKHBhbmVsLCBmaWVsZCwgb3Jk
ZXIpO0dvQWNjZXNzLlV0aWwuc2V0UHJvcChHb0FjY2Vzcy5BcHBTdGF0ZSwgcGFuZWwgKyAnLnNv
cnQnLCB7J2ZpZWxkJzogZmllbGQsJ29yZGVyJzogb3JkZXIsfSk7dGhpcy5yZW5kZXJUaGVhZChw
YW5lbCwgR29BY2Nlc3MuZ2V0UGFuZWxVSShwYW5lbCkpO3RoaXMucmVuZGVyVGFibGUocGFuZWws
IHRoaXMuZ2V0Q3VyUGFnZShwYW5lbCkpO0dvQWNjZXNzLkNoYXJ0cy5yZWxvYWRDaGFydChHb0Fj
Y2Vzcy5BcHBDaGFydHNbcGFuZWxdLCBwYW5lbCk7fSxnZXREYXRhQnlLZXk6IGZ1bmN0aW9uIChw
YW5lbCwga2V5KSB7dmFyIGRhdGEgPSBHb0FjY2Vzcy5nZXRQYW5lbERhdGEocGFuZWwpLmRhdGE7
Zm9yICh2YXIgaSA9IDAsIG4gPSBkYXRhLmxlbmd0aDsgaSA8IG47ICsraSkge2lmIChHb0FjY2Vz
cy5VdGlsLmhhc2hDb2RlKGRhdGFbaV0uZGF0YSkgPT0ga2V5KXJldHVybiBkYXRhW2ldO31yZXR1
cm4gbnVsbDt9LGdldFN1Ykl0ZW1zRGF0YTogZnVuY3Rpb24gKHBhbmVsKSB7dmFyIG91dCA9IFtd
LCBpdGVtcyA9IHRoaXMuY2hhcnREYXRhW3BhbmVsXTtmb3IgKHZhciB4IGluIGl0ZW1zKSB7aWYg
KCFpdGVtcy5oYXNPd25Qcm9wZXJ0eSh4KSljb250aW51ZTtvdXQgPSBvdXQuY29uY2F0KGl0ZW1z
W3hdKTt9cmV0dXJuIG91dDt9LGFkZENoYXJ0RGF0YTogZnVuY3Rpb24gKHBhbmVsLCBrZXkpIHt2
YXIgZGF0YSA9IHRoaXMuZ2V0RGF0YUJ5S2V5KHBhbmVsLCBrZXkpO3ZhciBwYXRoID0gcGFuZWwg
KyAnLicgKyBrZXk7aWYgKCFkYXRhIHx8ICFkYXRhLml0ZW1zKXJldHVybiBbXTtHb0FjY2Vzcy5V
dGlsLnNldFByb3AodGhpcy5jaGFydERhdGEsIHBhdGgsIGRhdGEuaXRlbXMpO3JldHVybiB0aGlz
LmdldFN1Ykl0ZW1zRGF0YShwYW5lbCk7fSxyZW1vdmVDaGFydERhdGE6IGZ1bmN0aW9uIChwYW5l
bCwga2V5KSB7aWYgKEdvQWNjZXNzLlV0aWwuZ2V0UHJvcCh0aGlzLmNoYXJ0RGF0YSwgcGFuZWwg
KyAnLicgKyBrZXkpKWRlbGV0ZSB0aGlzLmNoYXJ0RGF0YVtwYW5lbF1ba2V5XTtpZiAoIXRoaXMu
Y2hhcnREYXRhW3BhbmVsXSB8fCBPYmplY3Qua2V5cyh0aGlzLmNoYXJ0RGF0YVtwYW5lbF0pLmxl
bmd0aCA9PSAwKXJldHVybiBHb0FjY2Vzcy5nZXRQYW5lbERhdGEocGFuZWwpLmRhdGE7cmV0dXJu
IHRoaXMuZ2V0U3ViSXRlbXNEYXRhKHBhbmVsKTt9LGlzRXhwYW5kZWQ6IGZ1bmN0aW9uIChwYW5l
bCwga2V5KSB7dmFyIHBhdGggPSBwYW5lbCArICcuZXhwYW5kZWQuJyArIGtleTtyZXR1cm4gR29B
Y2Nlc3MuVXRpbC5nZXRQcm9wKEdvQWNjZXNzLkFwcFN0YXRlLCBwYXRoKTt9LHRvZ2dsZUV4cGFu
ZGVkOiBmdW5jdGlvbiAocGFuZWwsIGtleSkge3ZhciBwYXRoID0gcGFuZWwgKyAnLmV4cGFuZGVk
LicgKyBrZXksIHJldCA9IHRydWU7aWYgKHRoaXMuaXNFeHBhbmRlZChwYW5lbCwga2V5KSkge2Rl
bGV0ZSBHb0FjY2Vzcy5BcHBTdGF0ZVtwYW5lbF1bJ2V4cGFuZGVkJ11ba2V5XTt9IGVsc2Uge0dv
QWNjZXNzLlV0aWwuc2V0UHJvcChHb0FjY2Vzcy5BcHBTdGF0ZSwgcGF0aCwgdHJ1ZSksIHJldCA9
IGZhbHNlO31yZXR1cm4gcmV0O30sdG9nZ2xlUm93OiBmdW5jdGlvbiAoZWxlKSB7dmFyIGhpZGUg
PSBmYWxzZSwgZGF0YSA9IFtdO3ZhciByb3cgPSBlbGUucGFyZW50Tm9kZTt2YXIgcGFuZWwgPSBy
b3cuZ2V0QXR0cmlidXRlKCdkYXRhLXBhbmVsJyksIGtleSA9IHJvdy5nZXRBdHRyaWJ1dGUoJ2Rh
dGEta2V5Jyk7dmFyIHBsb3RVSSA9IEdvQWNjZXNzLkFwcENoYXJ0c1twYW5lbF0ub3B0cygpO2hp
ZGUgPSB0aGlzLnRvZ2dsZUV4cGFuZGVkKHBhbmVsLCBrZXkpO3RoaXMucmVuZGVyVGFibGUocGFu
ZWwsIHRoaXMuZ2V0Q3VyUGFnZShwYW5lbCkpO2lmICghcGxvdFVJLnJlZHJhd09uRXhwYW5kKXJl
dHVybjtpZiAoIWhpZGUpIHtkYXRhID0gR29BY2Nlc3MuQ2hhcnRzLnByb2Nlc3NDaGFydERhdGEo
dGhpcy5hZGRDaGFydERhdGEocGFuZWwsIGtleSkpO30gZWxzZSB7ZGF0YSA9IEdvQWNjZXNzLkNo
YXJ0cy5wcm9jZXNzQ2hhcnREYXRhKHRoaXMucmVtb3ZlQ2hhcnREYXRhKHBhbmVsLCBrZXkpKTt9
R29BY2Nlc3MuQ2hhcnRzLmRyYXdQbG90KHBhbmVsLCBwbG90VUksIGRhdGEpO30sZ2V0Q3VyUGFn
ZTogZnVuY3Rpb24gKHBhbmVsKSB7cmV0dXJuIEdvQWNjZXNzLlV0aWwuZ2V0UHJvcChHb0FjY2Vz
cy5BcHBTdGF0ZSwgcGFuZWwgKyAnLmN1clBhZ2UnKSB8fCAwO30scGFnZU9mZlNldDogZnVuY3Rp
b24gKHBhbmVsKSB7cmV0dXJuICgodGhpcy5nZXRDdXJQYWdlKHBhbmVsKSAtIDEpICogR29BY2Nl
c3MuZ2V0UHJlZnMoKS5wZXJQYWdlKTt9LGdldFRvdGFsUGFnZXM6IGZ1bmN0aW9uIChkYXRhSXRl
bXMpIHtyZXR1cm4gTWF0aC5jZWlsKGRhdGFJdGVtcy5sZW5ndGggLyBHb0FjY2Vzcy5nZXRQcmVm
cygpLnBlclBhZ2UpO30sZ2V0UGFnZTogZnVuY3Rpb24gKHBhbmVsLCBkYXRhSXRlbXMsIHBhZ2Up
IHt2YXIgdG90YWxQYWdlcyA9IHRoaXMuZ2V0VG90YWxQYWdlcyhkYXRhSXRlbXMpO2lmIChwYWdl
IDwgMSlwYWdlID0gMTtpZiAocGFnZSA+IHRvdGFsUGFnZXMpcGFnZSA9IHRvdGFsUGFnZXM7R29B
Y2Nlc3MuVXRpbC5zZXRQcm9wKEdvQWNjZXNzLkFwcFN0YXRlLCBwYW5lbCArICcuY3VyUGFnZScs
IHBhZ2UpO3ZhciBzdGFydCA9IHRoaXMucGFnZU9mZlNldChwYW5lbCk7dmFyIGVuZCA9IHN0YXJ0
ICsgR29BY2Nlc3MuZ2V0UHJlZnMoKS5wZXJQYWdlO3JldHVybiBkYXRhSXRlbXMuc2xpY2Uoc3Rh
cnQsIGVuZCk7fSxwcmV2UGFnZTogZnVuY3Rpb24gKHBhbmVsKSB7cmV0dXJuIHRoaXMuZ2V0Q3Vy
UGFnZShwYW5lbCkgLSAxO30sbmV4dFBhZ2U6IGZ1bmN0aW9uIChwYW5lbCkge3JldHVybiB0aGlz
LmdldEN1clBhZ2UocGFuZWwpICsgMTt9LGdldE1ldGFDZWxsOiBmdW5jdGlvbiAodWksIG8sIGtl
eSkge3ZhciB2YWwgPSAgbyAmJiAoa2V5IGluIG8pICYmIG9ba2V5XS52YWx1ZSA/IG9ba2V5XS52
YWx1ZSA6IG51bGw7dmFyIHBlcmMgPSBvICYmICAoa2V5IGluIG8pICYmIG9ba2V5XS5wZXJjZW50
ID8gb1trZXldLnBlcmNlbnQgOiBudWxsO3ZhciB2dHlwZSA9IHVpLm1ldGFUeXBlIHx8IHVpLmRh
dGFUeXBlO3ZhciBjbGFzc05hbWUgPSB1aS5jbGFzc05hbWUgfHwgJyc7Y2xhc3NOYW1lICs9ICFb
J3N0cmluZyddLmluY2x1ZGVzKHVpLmRhdGFUeXBlKSA/ICd0ZXh0LXJpZ2h0JyA6ICcnO3JldHVy
biB7J2NsYXNzTmFtZSc6IGNsYXNzTmFtZSwndmFsdWUnICAgIDogdmFsID8gR29BY2Nlc3MuVXRp
bC5mbXRWYWx1ZSh2YWwsIHZ0eXBlKSA6IG51bGwsJ3BlcmNlbnQnICA6IHBlcmMsJ3RpdGxlJyAg
ICA6IHVpLm1ldGEsJ2xhYmVsJyAgICA6IHVpLm1ldGFMYWJlbCB8fCBudWxsLH07fSxoaWRlQ29s
dW1uOiBmdW5jdGlvbiAocGFuZWwsIGNvbCkge3ZhciBjb2x1bW5zID0gKEdvQWNjZXNzLmdldFBy
ZWZzKHBhbmVsKSB8fCB7fSkuY29sdW1ucyB8fCB7fTtyZXR1cm4gKChjb2wgaW4gY29sdW1ucykg
JiYgY29sdW1uc1tjb2xdWydoaWRlJ10pO30sc2hvd1RhYmxlczogZnVuY3Rpb24gKCkge3JldHVy
biAoJ3Nob3dUYWJsZXMnIGluIEdvQWNjZXNzLmdldFByZWZzKCkpID8gR29BY2Nlc3MuZ2V0UHJl
ZnMoKS5zaG93VGFibGVzIDogdHJ1ZTt9LGF1dG9IaWRlVGFibGVzOiBmdW5jdGlvbiAoKSB7cmV0
dXJuICgnYXV0b0hpZGVUYWJsZXMnIGluIEdvQWNjZXNzLmdldFByZWZzKCkpID8gR29BY2Nlc3Mu
Z2V0UHJlZnMoKS5hdXRvSGlkZVRhYmxlcyA6IHRydWU7fSxoYXNUYWJsZTogZnVuY3Rpb24gKHVp
KSB7dWlbJ3RhYmxlJ10gPSB0aGlzLnNob3dUYWJsZXMoKTt1aVsnYXV0b0hpZGVUYWJsZXMnXSA9
IHRoaXMuYXV0b0hpZGVUYWJsZXMoKTt9LGdldE1ldGFSb3dzOiBmdW5jdGlvbiAocGFuZWwsIHVp
LCBrZXkpIHt2YXIgY2VsbHMgPSBbXSwgdWlJdGVtcyA9IHVpLml0ZW1zO3ZhciBkYXRhID0gR29B
Y2Nlc3MuZ2V0UGFuZWxEYXRhKHBhbmVsKS5tZXRhZGF0YTtmb3IgKHZhciBpID0gMDsgaSA8IHVp
SXRlbXMubGVuZ3RoOyArK2kpIHt2YXIgaXRlbSA9IHVpSXRlbXNbaV07aWYgKHRoaXMuaGlkZUNv
bHVtbihwYW5lbCwgaXRlbS5rZXkpKWNvbnRpbnVlO2NlbGxzLnB1c2godGhpcy5nZXRNZXRhQ2Vs
bChpdGVtLCBkYXRhW2l0ZW0ua2V5XSwga2V5KSk7fXJldHVybiBbeydoYXNTdWJJdGVtcyc6IHVp
Lmhhc1N1Ykl0ZW1zLCdjZWxscyc6IGNlbGxzLCdrZXknIDoga2V5LnN1YnN0cmluZygwLCAzKSx9
XTt9LHJlbmRlck1ldGFSb3c6IGZ1bmN0aW9uIChwYW5lbCwgbWV0YXJvd3MsIGNsYXNzTmFtZSkg
e3ZhciB0YWJsZSA9ICQoJy50YWJsZS0nICsgcGFuZWwgKyAnIHRyLicgKyBjbGFzc05hbWUpO2lm
ICghdGFibGUpcmV0dXJuO3RhYmxlLmlubmVySFRNTCA9IEdvQWNjZXNzLkFwcFRwbHMuVGFibGVz
Lm1ldGEucmVuZGVyKHtyb3c6IG1ldGFyb3dzfSk7fSxpdGVyVUlJdGVtczogZnVuY3Rpb24gKHBh
bmVsLCB1aUl0ZW1zLCBkYXRhSXRlbXMsIGNhbGxiYWNrKSB7dmFyIG91dCA9IFtdO2ZvciAodmFy
IGkgPSAwOyBpIDwgdWlJdGVtcy5sZW5ndGg7ICsraSkge3ZhciB1aUl0ZW0gPSB1aUl0ZW1zW2ld
O2lmICh0aGlzLmhpZGVDb2x1bW4ocGFuZWwsIHVpSXRlbS5rZXkpKWNvbnRpbnVlO3ZhciBkYXRh
SXRlbSA9IGRhdGFJdGVtc1t1aUl0ZW0ua2V5XTtpZiAoY2FsbGJhY2sgJiYgdHlwZW9mIGNhbGxi
YWNrID09ICdmdW5jdGlvbicpIHt2YXIgcmV0ID0gY2FsbGJhY2suY2FsbCh0aGlzLCBwYW5lbCwg
dWlJdGVtLCBkYXRhSXRlbSk7aWYgKHJldCkgb3V0LnB1c2gocmV0KTt9fXJldHVybiBvdXQ7fSxn
ZXRPYmplY3RDZWxsOiBmdW5jdGlvbiAocGFuZWwsIHVpLCB2YWx1ZSkge3ZhciBjbGFzc05hbWUg
PSB1aS5jbGFzc05hbWUgfHwgJyc7Y2xhc3NOYW1lICs9ICFbJ3N0cmluZyddLmluY2x1ZGVzKHVp
LmRhdGFUeXBlKSA/ICd0ZXh0LXJpZ2h0JyA6ICcnO3JldHVybiB7J2NsYXNzTmFtZSc6IGNsYXNz
TmFtZSwncGVyY2VudCc6IEdvQWNjZXNzLlV0aWwuZ2V0UGVyY2VudCh2YWx1ZSksJ3ZhbHVlJzog
R29BY2Nlc3MuVXRpbC5mbXRWYWx1ZShHb0FjY2Vzcy5VdGlsLmdldENvdW50KHZhbHVlKSwgdWku
ZGF0YVR5cGUsIG51bGwsIG51bGwsIHVpLmhscmVnZXgsIHVpLmhsdmFsdWUsIHVpLmhsaWR4KX07
fSxyZW5kZXJSb3c6IGZ1bmN0aW9uIChwYW5lbCwgY2FsbGJhY2ssIHVpLCBkYXRhSXRlbSwgaWR4
LCBzdWJJdGVtLCBwYXJlbnRJZCwgZXhwYW5kZWQpIHt2YXIgc2hhZGVQYXJlbnQgPSAoKCFzdWJJ
dGVtICYmIGlkeCAlIDIgIT0gMCkgPyAnc2hhZGVkJyA6ICcnKTt2YXIgc2hhZGVDaGlsZCA9ICgo
cGFyZW50SWQgJSAyICE9IDApID8gJ3NoYWRlZCcgOiAnJyk7cmV0dXJuIHsncGFuZWwnICAgICAg
IDogcGFuZWwsJ2lkeCcgICAgICAgICA6ICFzdWJJdGVtICYmIChTdHJpbmcoKGlkeCArIDEpICsg
dGhpcy5wYWdlT2ZmU2V0KHBhbmVsKSkpLCdrZXknICAgICAgICAgOiAhc3ViSXRlbSA/IEdvQWNj
ZXNzLlV0aWwuaGFzaENvZGUoZGF0YUl0ZW0uZGF0YSkgOiAnJywnZXhwYW5kZWQnICAgIDogIXN1
Ykl0ZW0gJiYgZXhwYW5kZWQsJ3BhcmVudElkJyAgICA6IHN1Ykl0ZW0gPyBTdHJpbmcocGFyZW50
SWQpIDogJycsJ2NsYXNzTmFtZScgICA6IHN1Ykl0ZW0gPyAnY2hpbGQgJyArIHNoYWRlQ2hpbGQg
OiAncGFyZW50ICcgKyBzaGFkZVBhcmVudCwnaGFzU3ViSXRlbXMnIDogdWkuaGFzU3ViSXRlbXMs
J2l0ZW1zJyAgICAgICA6IGRhdGFJdGVtLml0ZW1zID8gZGF0YUl0ZW0uaXRlbXMubGVuZ3RoIDog
MCwnY2VsbHMnICAgICAgIDogY2FsbGJhY2suY2FsbCh0aGlzKSx9O30scmVuZGVyUm93czogZnVu
Y3Rpb24gKHJvd3MsIHBhbmVsLCB1aSwgZGF0YUl0ZW1zLCBzdWJJdGVtLCBwYXJlbnRJZCkge3N1
Ykl0ZW0gPSBzdWJJdGVtIHx8IGZhbHNlO2lmIChkYXRhSXRlbXMubGVuZ3RoID09IDAgJiYgdWku
aXRlbXMubGVuZ3RoKSB7cm93cy5wdXNoKHtjZWxsczogW3tjbGFzc05hbWU6ICd0ZXh0LWNlbnRl
cicsY29sc3BhbjogdWkuaXRlbXMubGVuZ3RoICsgMSx2YWx1ZTogJ05vIGRhdGEgb24gdGhpcyBw
YW5lbC4nfV19KTt9dmFyIGNlbGxjYiA9IG51bGw7Zm9yICh2YXIgaSA9IDA7IGkgPCBkYXRhSXRl
bXMubGVuZ3RoOyArK2kpIHt2YXIgZGF0YUl0ZW0gPSBkYXRhSXRlbXNbaV0sIGRhdGEgPSBudWxs
LCBleHBhbmRlZCA9IGZhbHNlO3N3aXRjaCh0eXBlb2YgZGF0YUl0ZW0pIHtjYXNlICdzdHJpbmcn
OmRhdGEgPSBkYXRhSXRlbTtjZWxsY2IgPSBmdW5jdGlvbiAoKSB7cmV0dXJuIHsnY29sc3Bhbic6
IHVpLml0ZW1zLmxlbmd0aCwndmFsdWUnOiBkYXRhfTt9O2JyZWFrO2RlZmF1bHQ6ZGF0YSA9IGRh
dGFJdGVtLmRhdGE7Y2VsbGNiID0gdGhpcy5pdGVyVUlJdGVtcy5iaW5kKHRoaXMsIHBhbmVsLCB1
aS5pdGVtcywgZGF0YUl0ZW0sIHRoaXMuZ2V0T2JqZWN0Q2VsbC5iaW5kKHRoaXMpKTt9ZXhwYW5k
ZWQgPSB0aGlzLmlzRXhwYW5kZWQocGFuZWwsIEdvQWNjZXNzLlV0aWwuaGFzaENvZGUoZGF0YSkp
O3Jvd3MucHVzaCh0aGlzLnJlbmRlclJvdyhwYW5lbCwgY2VsbGNiLCB1aSwgZGF0YUl0ZW0sIGks
IHN1Ykl0ZW0sIHBhcmVudElkLCBleHBhbmRlZCkpO2lmIChkYXRhSXRlbS5pdGVtcyAmJiBkYXRh
SXRlbS5pdGVtcy5sZW5ndGggJiYgZXhwYW5kZWQpIHt0aGlzLnJlbmRlclJvd3Mocm93cywgcGFu
ZWwsIHVpLCBkYXRhSXRlbS5pdGVtcywgdHJ1ZSwgaSwgZXhwYW5kZWQpO319fSxyZW5kZXJEYXRh
Um93czogZnVuY3Rpb24gKHBhbmVsLCB1aSwgZGF0YUl0ZW1zLCBwYWdlKSB7dmFyIHRhYmxlID0g
JCgnLnRhYmxlLScgKyBwYW5lbCArICcgdGJvZHkudGJvZHktZGF0YScpO2lmICghdGFibGUpcmV0
dXJuO2RhdGFJdGVtcyA9IHRoaXMuZ2V0UGFnZShwYW5lbCwgZGF0YUl0ZW1zLCBwYWdlKTt2YXIg
cm93cyA9IFtdO3RoaXMucmVuZGVyUm93cyhyb3dzLCBwYW5lbCwgdWksIGRhdGFJdGVtcyk7aWYg
KHJvd3MubGVuZ3RoID09IDApcmV0dXJuO3RhYmxlLmlubmVySFRNTCA9IEdvQWNjZXNzLkFwcFRw
bHMuVGFibGVzLmRhdGEucmVuZGVyKHtyb3dzOiByb3dzfSk7fSx0b2dnbGVQYWdpbmF0aW9uOiBm
dW5jdGlvbiAocGFuZWwsIHBhZ2UsIGRhdGFJdGVtcykge0dvQWNjZXNzLlBhbmVscy5lbmFibGVQ
YWdpbmF0aW9uKHBhbmVsKTtpZiAocGFnZSA+PSB0aGlzLmdldFRvdGFsUGFnZXMoZGF0YUl0ZW1z
KSkge0dvQWNjZXNzLlBhbmVscy5kaXNhYmxlTmV4dChwYW5lbCk7R29BY2Nlc3MuUGFuZWxzLmRp
c2FibGVMYXN0KHBhbmVsKTt9aWYgKHBhZ2UgPD0gMSkge0dvQWNjZXNzLlBhbmVscy5kaXNhYmxl
UHJldihwYW5lbCk7R29BY2Nlc3MuUGFuZWxzLmRpc2FibGVGaXJzdChwYW5lbCk7fX0scmVuZGVy
VGFibGU6IGZ1bmN0aW9uIChwYW5lbCwgcGFnZSkge3ZhciBkYXRhSXRlbXMgPSBHb0FjY2Vzcy5n
ZXRQYW5lbERhdGEocGFuZWwpLmRhdGE7dmFyIHVpID0gR29BY2Nlc3MuZ2V0UGFuZWxVSShwYW5l
bCk7aWYgKHBhZ2UgPT09ICJMQVNUX1BBR0UiKSB7cGFnZSA9IHRoaXMuZ2V0VG90YWxQYWdlcyhk
YXRhSXRlbXMpO30gZWxzZSBpZiAocGFnZSA9PT0gIkZJUlNUX1BBR0UiKSB7cGFnZSA9IDE7fXRo
aXMudG9nZ2xlUGFnaW5hdGlvbihwYW5lbCwgcGFnZSwgZGF0YUl0ZW1zKTt0aGlzLnJlbmRlckRh
dGFSb3dzKHBhbmVsLCB1aSwgZGF0YUl0ZW1zLCBwYWdlKTt0aGlzLmV2ZW50cygpO30scmVuZGVy
RnVsbFRhYmxlOiBmdW5jdGlvbiAocGFuZWwpIHt2YXIgdWkgPSBHb0FjY2Vzcy5nZXRQYW5lbFVJ
KHBhbmVsKSwgcGFnZSA9IDA7dmFyIGRhdGEgPSBHb0FjY2Vzcy5nZXRQYW5lbERhdGEocGFuZWwp
O2lmIChkYXRhLmhhc093blByb3BlcnR5KCdtZXRhZGF0YScpKSB7dGhpcy5yZW5kZXJNZXRhUm93
KHBhbmVsLCB0aGlzLmdldE1ldGFSb3dzKHBhbmVsLCB1aSwgJ21pbicpLCAndGhlYWQtbWluJyk7
dGhpcy5yZW5kZXJNZXRhUm93KHBhbmVsLCB0aGlzLmdldE1ldGFSb3dzKHBhbmVsLCB1aSwgJ21h
eCcpLCAndGhlYWQtbWF4Jyk7dGhpcy5yZW5kZXJNZXRhUm93KHBhbmVsLCB0aGlzLmdldE1ldGFS
b3dzKHBhbmVsLCB1aSwgJ2F2ZycpLCAndGhlYWQtYXZnJyk7fWlmIChkYXRhLmhhc093blByb3Bl
cnR5KCdkYXRhJykpIHtwYWdlID0gdGhpcy5nZXRDdXJQYWdlKHBhbmVsKTt0aGlzLnRvZ2dsZVBh
Z2luYXRpb24ocGFuZWwsIHBhZ2UsIGRhdGEuZGF0YSk7dGhpcy5yZW5kZXJEYXRhUm93cyhwYW5l
bCwgdWksIGRhdGEuZGF0YSwgcGFnZSk7fWlmIChkYXRhLmhhc093blByb3BlcnR5KCdtZXRhZGF0
YScpKSB7dGhpcy5yZW5kZXJNZXRhUm93KHBhbmVsLCB0aGlzLmdldE1ldGFSb3dzKHBhbmVsLCB1
aSwgJ3RvdGFsJyksICd0Zm9vdC10b3RhbHMnKTt9fSxyZW5kZXJUYWJsZXM6IGZ1bmN0aW9uIChm
b3JjZSkge3ZhciB1aSA9IEdvQWNjZXNzLmdldFBhbmVsVUkoKTtmb3IgKHZhciBwYW5lbCBpbiB1
aSkge2lmIChHb0FjY2Vzcy5VdGlsLmlzUGFuZWxWYWxpZChwYW5lbCkgfHwgR29BY2Nlc3MuVXRp
bC5pc1BhbmVsSGlkZGVuKHBhbmVsKSB8fCAhdGhpcy5zaG93VGFibGVzKCkpY29udGludWU7aWYg
KGZvcmNlIHx8IEdvQWNjZXNzLlV0aWwuaXNXaXRoaW5WaWV3UG9ydCgkKCcjcGFuZWwtJyArIHBh
bmVsKSkpdGhpcy5yZW5kZXJGdWxsVGFibGUocGFuZWwpO319LHNvcnQyVHBsOiBmdW5jdGlvbiAo
cGFuZWwsIHVpKSB7dmFyIHVpQ2xvbmUgPSBKU09OLnBhcnNlKEpTT04uc3RyaW5naWZ5KHVpKSks
IG91dCA9IFtdO3ZhciBzb3J0ID0gR29BY2Nlc3MuVXRpbC5nZXRQcm9wKEdvQWNjZXNzLkFwcFN0
YXRlLCBwYW5lbCArICcuc29ydCcpO2ZvciAodmFyIGkgPSAwLCBsZW4gPSB1aUNsb25lLml0ZW1z
Lmxlbmd0aDsgaSA8IGxlbjsgKytpKSB7dmFyIGl0ZW0gPSB1aUNsb25lLml0ZW1zW2ldO2lmICh0
aGlzLmhpZGVDb2x1bW4ocGFuZWwsIGl0ZW0ua2V5KSljb250aW51ZTtpdGVtWydzb3J0J10gPSBm
YWxzZTtpZiAoaXRlbS5rZXkgPT0gc29ydC5maWVsZCAmJiBzb3J0Lm9yZGVyKSB7aXRlbVsnc29y
dCddID0gdHJ1ZTtpdGVtW3NvcnQub3JkZXIudG9Mb3dlckNhc2UoKV0gPSB0cnVlO31vdXQucHVz
aChpdGVtKTt9dWlDbG9uZS5pdGVtcyA9IG91dDtyZXR1cm4gdWlDbG9uZTt9LHJlbmRlclRoZWFk
OiBmdW5jdGlvbiAocGFuZWwsIHVpKSB7dmFyICR0aGVhZCA9ICQoJy50YWJsZS0nICsgcGFuZWwg
KyAnPnRoZWFkPnRyLnRoZWFkLWNvbHMnKSwkY29sZ3JvdXAgPSAkKCcudGFibGUtJyArIHBhbmVs
ICsgJz5jb2xncm91cCcpO2lmICgkdGhlYWQgJiYgJGNvbGdyb3VwICYmIHRoaXMuc2hvd1RhYmxl
cygpKSB7dWkgPSB0aGlzLnNvcnQyVHBsKHBhbmVsLCB1aSk7JHRoZWFkLmlubmVySFRNTCA9IEdv
QWNjZXNzLkFwcFRwbHMuVGFibGVzLmhlYWQucmVuZGVyKHVpKTskY29sZ3JvdXAuaW5uZXJIVE1M
ID0gR29BY2Nlc3MuQXBwVHBscy5UYWJsZXMuY29sZ3JvdXAucmVuZGVyKHVpKTt9fSxyZWxvYWRU
YWJsZXM6IGZ1bmN0aW9uICgpIHt0aGlzLnJlbmRlclRhYmxlcyhmYWxzZSk7dGhpcy5ldmVudHMo
KTt9LGluaXRpYWxpemU6IGZ1bmN0aW9uICgpIHt0aGlzLnJlbmRlclRhYmxlcyh0cnVlKTt0aGlz
LmV2ZW50cygpO2QzLnNlbGVjdCh3aW5kb3cpLm9uKCdzY3JvbGwudGFibGVzJywgZGVib3VuY2Uo
ZnVuY3Rpb24gKCkge3RoaXMucmVsb2FkVGFibGVzKCk7fSwgMjUwLCBmYWxzZSkuYmluZCh0aGlz
KSk7fSx9O0dvQWNjZXNzLkFwcCA9IHtoYXNGb2N1czogdHJ1ZSx0cGw6IGZ1bmN0aW9uICh0cGwp
IHtyZXR1cm4gSG9nYW4uY29tcGlsZSh0cGwpO30sc2V0VHBsczogZnVuY3Rpb24gKCkge0dvQWNj
ZXNzLkFwcFRwbHMgPSB7J05hdic6IHsnd3JhcCc6IHRoaXMudHBsKCQoJyN0cGwtbmF2LXdyYXAn
KS5pbm5lckhUTUwpLCdtZW51JzogdGhpcy50cGwoJCgnI3RwbC1uYXYtbWVudScpLmlubmVySFRN
TCksJ29wdHMnOiB0aGlzLnRwbCgkKCcjdHBsLW5hdi1vcHRzJykuaW5uZXJIVE1MKSx9LCdQYW5l
bHMnOiB7J3dyYXAnOiB0aGlzLnRwbCgkKCcjdHBsLXBhbmVsJykuaW5uZXJIVE1MKSwnb3B0cyc6
IHRoaXMudHBsKCQoJyN0cGwtcGFuZWwtb3B0cycpLmlubmVySFRNTCksfSwnR2VuZXJhbCc6IHsn
d3JhcCc6IHRoaXMudHBsKCQoJyN0cGwtZ2VuZXJhbCcpLmlubmVySFRNTCksJ2l0ZW1zJzogdGhp
cy50cGwoJCgnI3RwbC1nZW5lcmFsLWl0ZW1zJykuaW5uZXJIVE1MKSx9LCdUYWJsZXMnOiB7J2Nv
bGdyb3VwJzogdGhpcy50cGwoJCgnI3RwbC10YWJsZS1jb2xncm91cCcpLmlubmVySFRNTCksJ2hl
YWQnOiB0aGlzLnRwbCgkKCcjdHBsLXRhYmxlLXRoZWFkJykuaW5uZXJIVE1MKSwnbWV0YSc6IHRo
aXMudHBsKCQoJyN0cGwtdGFibGUtcm93LW1ldGEnKS5pbm5lckhUTUwpLCd0b3RhbHMnOiB0aGlz
LnRwbCgkKCcjdHBsLXRhYmxlLXJvdy10b3RhbHMnKS5pbm5lckhUTUwpLCdkYXRhJzogdGhpcy50
cGwoJCgnI3RwbC10YWJsZS1yb3cnKS5pbm5lckhUTUwpLH0sfTt9LHNvcnRGaWVsZDogZnVuY3Rp
b24gKG8sIGZpZWxkKSB7dmFyIGYgPSBvW2ZpZWxkXTtpZiAoR29BY2Nlc3MuVXRpbC5pc09iamVj
dChmKSAmJiAoZiAhPT0gbnVsbCkpZiA9IG9bZmllbGRdLmNvdW50O3JldHVybiBmO30sc29ydERh
dGE6IGZ1bmN0aW9uIChwYW5lbCwgZmllbGQsIG9yZGVyKSB7dmFyIHBhbmVsRGF0YSA9IEdvQWNj
ZXNzLmdldFBhbmVsRGF0YShwYW5lbCkuZGF0YTt2YXIgc29ydEFycmF5ID0gZnVuY3Rpb24oYXJy
KSB7YXJyLnNvcnQoZnVuY3Rpb24gKGEsIGIpIHthID0gdGhpcy5zb3J0RmllbGQoYSwgZmllbGQp
O2IgPSB0aGlzLnNvcnRGaWVsZChiLCBmaWVsZCk7aWYgKHR5cGVvZiBhID09PSAnc3RyaW5nJyAm
JiB0eXBlb2YgYiA9PT0gJ3N0cmluZycpcmV0dXJuICdhc2MnID09IG9yZGVyID8gYS5sb2NhbGVD
b21wYXJlKGIpIDogYi5sb2NhbGVDb21wYXJlKGEpO3JldHVybiAgJ2FzYycgPT0gb3JkZXIgPyBh
IC0gYiA6IGIgLSBhO30uYmluZCh0aGlzKSk7fS5iaW5kKHRoaXMpO3NvcnRBcnJheShwYW5lbERh
dGEpO3BhbmVsRGF0YS5mb3JFYWNoKGZ1bmN0aW9uKGl0ZW0pIHtpZiAoaXRlbS5pdGVtcykge3Nv
cnRBcnJheShpdGVtLml0ZW1zKTt9fSk7fSxzZXRJbml0U29ydDogZnVuY3Rpb24gKCkge3ZhciB1
aSA9IEdvQWNjZXNzLmdldFBhbmVsVUkoKTtmb3IgKHZhciBwYW5lbCBpbiB1aSkge2lmIChHb0Fj
Y2Vzcy5VdGlsLmlzUGFuZWxWYWxpZChwYW5lbCkpY29udGludWU7R29BY2Nlc3MuVXRpbC5zZXRQ
cm9wKEdvQWNjZXNzLkFwcFN0YXRlLCBwYW5lbCArICcuc29ydCcsIHVpW3BhbmVsXS5zb3J0KTt9
fSx2ZXJpZnlTb3J0OiBmdW5jdGlvbiAoKSB7dmFyIHVpID0gR29BY2Nlc3MuZ2V0UGFuZWxVSSgp
O2ZvciAodmFyIHBhbmVsIGluIHVpKSB7aWYgKEdvQWNjZXNzLlV0aWwuaXNQYW5lbFZhbGlkKHBh
bmVsKSB8fCBHb0FjY2Vzcy5VdGlsLmlzUGFuZWxIaWRkZW4ocGFuZWwpKWNvbnRpbnVlO3ZhciBz
b3J0ID0gR29BY2Nlc3MuVXRpbC5nZXRQcm9wKEdvQWNjZXNzLkFwcFN0YXRlLCBwYW5lbCArICcu
c29ydCcpO2lmIChKU09OLnN0cmluZ2lmeShzb3J0KSA9PT0gSlNPTi5zdHJpbmdpZnkodWlbcGFu
ZWxdLnNvcnQpKWNvbnRpbnVlO3RoaXMuc29ydERhdGEocGFuZWwsIHNvcnQuZmllbGQsIHNvcnQu
b3JkZXIpO319LGluaXREb206IGZ1bmN0aW9uICgpIHskKCduYXYnKS5jbGFzc0xpc3QucmVtb3Zl
KCdoaWRlJyk7JCgnLmNvbnRhaW5lcicpLmNsYXNzTGlzdC5yZW1vdmUoJ2hpZGUnKTskKCcuc3Bp
bm5lcicpLmNsYXNzTGlzdC5hZGQoJ2hpZGUnKTtpZiAoR29BY2Nlc3MuQXBwUHJlZnNbJ2xheW91
dCddID09ICdob3Jpem9udGFsJyB8fCBHb0FjY2Vzcy5BcHBQcmVmc1snbGF5b3V0J10gPT0gJ3dp
ZGUnKSB7JCgnLmNvbnRhaW5lcicpLmNsYXNzTGlzdC5hZGQoJ2NvbnRhaW5lci1mbHVpZCcpOyQo
Jy5jb250YWluZXItZmx1aWQnKS5jbGFzc0xpc3QucmVtb3ZlKCdjb250YWluZXInKTt9fSxyZW5k
ZXJEYXRhOiBmdW5jdGlvbiAoKSB7aWYgKCF0aGlzLmhhc0ZvY3VzKXJldHVybjtpZiAoR29BY2Nl
c3MuT3ZlcmFsbFN0YXRzLnRvdGFsX3JlcXVlc3RzID09IDAgJiYgR29BY2Nlc3MuT3ZlcmFsbFN0
YXRzLnRvdGFsX3JlcXVlc3RzICE9IEdvQWNjZXNzLkFwcERhdGEuZ2VuZXJhbC50b3RhbF9yZXF1
ZXN0cylHb0FjY2Vzcy5QYW5lbHMuaW5pdGlhbGl6ZSgpO0dvQWNjZXNzLk92ZXJhbGxTdGF0cy50
b3RhbF9yZXF1ZXN0cyA9IEdvQWNjZXNzLkFwcERhdGEuZ2VuZXJhbC50b3RhbF9yZXF1ZXN0czt0
aGlzLnZlcmlmeVNvcnQoKTtHb0FjY2Vzcy5PdmVyYWxsU3RhdHMuaW5pdGlhbGl6ZSgpO2lmICgh
R29BY2Nlc3MuQXBwU3RhdGUudXBkYXRlZClyZXR1cm47R29BY2Nlc3MuQ2hhcnRzLnJlbG9hZENo
YXJ0cygpO0dvQWNjZXNzLlRhYmxlcy5yZWxvYWRUYWJsZXMoKTt9LGluaXRpYWxpemU6IGZ1bmN0
aW9uICgpIHt0aGlzLnNldEluaXRTb3J0KCk7dGhpcy5zZXRUcGxzKCk7R29BY2Nlc3MuTmF2Lmlu
aXRpYWxpemUoKTt0aGlzLmluaXREb20oKTtHb0FjY2Vzcy5PdmVyYWxsU3RhdHMuaW5pdGlhbGl6
ZSgpO0dvQWNjZXNzLlBhbmVscy5pbml0aWFsaXplKCk7R29BY2Nlc3MuQ2hhcnRzLmluaXRpYWxp
emUoKTtHb0FjY2Vzcy5UYWJsZXMuaW5pdGlhbGl6ZSgpO30sfTtkb2N1bWVudC5hZGRFdmVudExp
c3RlbmVyKCd2aXNpYmlsaXR5Y2hhbmdlJywgZnVuY3Rpb24gKCkge2lmIChkb2N1bWVudC52aXNp
YmlsaXR5U3RhdGUgPT09ICdoaWRkZW4nKUdvQWNjZXNzLkFwcC5oYXNGb2N1cyA9IGZhbHNlO2lm
IChkb2N1bWVudC52aXNpYmlsaXR5U3RhdGUgPT09ICd2aXNpYmxlJykge3ZhciBoYXNGb2N1cyA9
IEdvQWNjZXNzLkFwcC5oYXNGb2N1cztHb0FjY2Vzcy5BcHAuaGFzRm9jdXMgPSB0cnVlO2hhc0Zv
Y3VzIHx8IEdvQWNjZXNzLkFwcC5yZW5kZXJEYXRhKCk7fX0pO3dpbmRvdy5vbmxvYWQgPSBmdW5j
dGlvbiAoKSB7R29BY2Nlc3MuaW5pdGlhbGl6ZSh7J2kxOG4nOiB3aW5kb3cuanNvbl9pMThuLCd1
aURhdGEnOiB3aW5kb3cudXNlcl9pbnRlcmZhY2UsJ3BhbmVsRGF0YSc6IHdpbmRvdy5qc29uX2Rh
dGEsJ3dzQ29ubmVjdGlvbic6IHdpbmRvdy5jb25uZWN0aW9uIHx8IG51bGwsJ3ByZWZzJzogd2lu
ZG93Lmh0bWxfcHJlZnMgfHwge30sfSk7R29BY2Nlc3MuQXBwLmluaXRpYWxpemUoKTt9O30oKSk7
PC9zY3JpcHQ+PHNjcmlwdD4vKioqICAgIF9fX19fXyAgICAgIF9fXyogICAvIF9fX18vX19fICAv
ICAgfCBfX19fX19fX19fX19fICBfX19fX19fX19fKiAgLyAvIF9fLyBfXyBcLyAvfCB8LyBfX18v
IF9fXy8gXyBcLyBfX18vIF9fXy8qIC8gL18vIC8gL18vIC8gX19fIC8gL19fLyAvX18vICBfXyhf
XyAgfF9fICApKiBcX19fXy9cX19fXy9fLyAgfF9cX19fL1xfX18vXF9fXy9fX19fL19fX18vKiog
VGhlIE1JVCBMaWNlbnNlIChNSVQpKiBDb3B5cmlnaHQgKGMpIDIwMDktMjAxOCBHZXJhcmRvIE9y
ZWxsYW5hIDxoZWxsbyBAIGdvYWNjZXNzLmlvPiovJ3VzZSBzdHJpY3QnO2Z1bmN0aW9uIHRydW5j
YXRlKHRleHQsIHdpZHRoKSB7dGV4dC5lYWNoKGZ1bmN0aW9uICgpIHt2YXIgcGFyZW50ID0gdGhp
cy5wYXJlbnROb2RlLCAkZDNwYXJlbnQgPSBkMy5zZWxlY3QocGFyZW50KTt2YXIgZ3cgPSAkZDNw
YXJlbnQubm9kZSgpLmdldEJCb3goKTt2YXIgeCA9IChNYXRoLm1pbihndy53aWR0aCwgd2lkdGgp
IC8gMikgKiAtMTtpZiAoJ3N2ZycgPT0gcGFyZW50Lm5vZGVOYW1lKSB7JGQzcGFyZW50LmF0dHIo
J3dpZHRoJywgd2lkdGgpLmF0dHIoJ3gnLCB4KTt9ZWxzZSB7JGQzcGFyZW50Lmluc2VydCgnc3Zn
JywgZnVuY3Rpb24gKCkge3JldHVybiB0aGlzO30uYmluZCh0aGlzKSkuYXR0cignY2xhc3MnLCAn
d3JhcC10ZXh0JykuYXR0cignd2lkdGgnLCB3aWR0aCkuYXR0cigneCcsIHgpLmFwcGVuZChmdW5j
dGlvbiAoKSB7cmV0dXJuIHRoaXM7fS5iaW5kKHRoaXMpKTt9fSk7fWZ1bmN0aW9uIEFyZWFDaGFy
dChkdWFsWWF4aXMpIHt2YXIgb3B0cyA9IHt9O3ZhciBtYXJnaW4gPSB7dG9wICAgIDogMjAscmln
aHQgIDogNTAsYm90dG9tIDogNDAsbGVmdCAgIDogNTAsfSxoZWlnaHQgPSAxNzAsblRpY2tzID0g
MTAscGFkZGluZyA9IDEwLHdpZHRoID0gNzYwO3ZhciBsYWJlbHMgPSB7IHg6ICdVbm5hbWVkJywg
eTA6ICdVbm5hbWVkJywgeTE6ICdVbm5hbWVkJyB9O3ZhciBmb3JtYXQgPSB7IHg6IG51bGwsIHkw
OiBudWxsLCB5MTogbnVsbH07dmFyIHhWYWx1ZSA9IGZ1bmN0aW9uIChkKSB7cmV0dXJuIGRbMF07
fSx5VmFsdWUwID0gZnVuY3Rpb24gKGQpIHtyZXR1cm4gZFsxXTt9LHlWYWx1ZTEgPSBmdW5jdGlv
biAoZCkge3JldHVybiBkWzJdO307dmFyIHhTY2FsZSA9IGQzLnNjYWxlQmFuZCgpO3ZhciB5U2Nh
bGUwID0gZDMuc2NhbGVMaW5lYXIoKS5uaWNlKCk7dmFyIHlTY2FsZTEgPSBkMy5zY2FsZUxpbmVh
cigpLm5pY2UoKTt2YXIgeEF4aXMgPSBkMy5heGlzQm90dG9tKHhTY2FsZSkudGlja0Zvcm1hdChm
dW5jdGlvbihkKSB7aWYgKGZvcm1hdC54KXJldHVybiBHb0FjY2Vzcy5VdGlsLmZtdFZhbHVlKGQs
IGZvcm1hdC54KTtyZXR1cm4gZDt9KTt2YXIgeUF4aXMwID0gZDMuYXhpc0xlZnQoeVNjYWxlMCku
dGlja0Zvcm1hdChmdW5jdGlvbihkKSB7cmV0dXJuIGQzLmZvcm1hdCgnLjJzJykoZCk7fSk7dmFy
IHlBeGlzMSA9IGQzLmF4aXNSaWdodCh5U2NhbGUxKS50aWNrRm9ybWF0KGZ1bmN0aW9uKGQpIHtp
ZiAoZm9ybWF0LnkxKXJldHVybiBHb0FjY2Vzcy5VdGlsLmZtdFZhbHVlKGQsIGZvcm1hdC55MSk7
cmV0dXJuIGQzLmZvcm1hdCgnLjJzJykoZCk7fSk7dmFyIHhHcmlkID0gZDMuYXhpc0JvdHRvbSh4
U2NhbGUpO3ZhciB5R3JpZCA9IGQzLmF4aXNMZWZ0KHlTY2FsZTApO3ZhciBhcmVhMCA9IGQzLmFy
ZWEoKS5jdXJ2ZShkMy5jdXJ2ZU1vbm90b25lWCkueChYKS55MChoZWlnaHQpLnkxKFkwKTt2YXIg
YXJlYTEgPSBkMy5hcmVhKCkuY3VydmUoZDMuY3VydmVNb25vdG9uZVgpLngoWCkueTAoWTEpLnkx
KGhlaWdodCk7dmFyIGxpbmUwID0gZDMubGluZSgpLmN1cnZlKGQzLmN1cnZlTW9ub3RvbmVYKS54
KFgpLnkoWTApO3ZhciBsaW5lMSA9IGQzLmxpbmUoKS5jdXJ2ZShkMy5jdXJ2ZU1vbm90b25lWCku
eChYKS55KFkxKTtmdW5jdGlvbiBYKGQpIHtyZXR1cm4gKHhTY2FsZShkWzBdKSArIHhTY2FsZS5i
YW5kd2lkdGgoKSAvIDIpO31mdW5jdGlvbiBZMChkKSB7cmV0dXJuIHlTY2FsZTAoZFsxXSk7fWZ1
bmN0aW9uIFkxKGQpIHtyZXR1cm4geVNjYWxlMShkWzJdKTt9ZnVuY3Rpb24gaW5uZXJXKCkge3Jl
dHVybiB3aWR0aCAtIG1hcmdpbi5sZWZ0IC0gbWFyZ2luLnJpZ2h0O31mdW5jdGlvbiBpbm5lckgo
KSB7cmV0dXJuIGhlaWdodCAtIG1hcmdpbi50b3AgLSBtYXJnaW4uYm90dG9tO31mdW5jdGlvbiBn
ZXRYVGlja3MoZGF0YSkge2NvbnN0IGRvbWFpbiA9IHhTY2FsZS5kb21haW4oKTtpZiAoZGF0YS5s
ZW5ndGggPCBuVGlja3MpcmV0dXJuIGRvbWFpbjtyZXR1cm4gZDMucmFuZ2UoMCwgblRpY2tzKS5t
YXAoZnVuY3Rpb24oaSkge2NvbnN0IGluZGV4ID0gTWF0aC5mbG9vcihpICogKGRvbWFpbi5sZW5n
dGggLSAxKSAvIChuVGlja3MgLSAxKSk7aWYgKGluZGV4ID49IDAgJiYgaW5kZXggPCBkb21haW4u
bGVuZ3RoKXJldHVybiBkb21haW5baW5kZXhdO3JldHVybiBudWxsO30pO31mdW5jdGlvbiBnZXRZ
VGlja3Moc2NhbGUpIHt2YXIgZG9tYWluID0gc2NhbGUuZG9tYWluKCk7cmV0dXJuIGQzLnJhbmdl
KGRvbWFpblswXSwgZG9tYWluWzFdLCBNYXRoLmNlaWwoZG9tYWluWzFdIC8gblRpY2tzKSk7fWZ1
bmN0aW9uIG1hcERhdGEoZGF0YSkge3ZhciBfZGF0dW0gPSBmdW5jdGlvbiAoZCwgaSkge3ZhciBk
YXR1bSA9IFt4VmFsdWUuY2FsbChkYXRhLCBkLCBpKSwgeVZhbHVlMC5jYWxsKGRhdGEsIGQsIGkp
XTtkdWFsWWF4aXMgJiYgZGF0dW0ucHVzaCh5VmFsdWUxLmNhbGwoZGF0YSwgZCwgaSkpO3JldHVy
biBkYXR1bTt9O3JldHVybiBkYXRhLm1hcChmdW5jdGlvbiAoZCwgaSkge3JldHVybiBfZGF0dW0o
ZCwgaSk7fSk7fWZ1bmN0aW9uIHVwZGF0ZVNjYWxlcyhkYXRhKSB7eFNjYWxlLmRvbWFpbihkYXRh
Lm1hcChmdW5jdGlvbiAoZCkge3JldHVybiBkWzBdO30pKS5yYW5nZShbMCwgaW5uZXJXKCldKTt5
U2NhbGUwLmRvbWFpbihbMCwgZDMubWF4KGRhdGEsIGZ1bmN0aW9uIChkKSB7cmV0dXJuIGRbMV07
fSldKS5yYW5nZShbaW5uZXJIKCksIDBdKTtkdWFsWWF4aXMgJiYgeVNjYWxlMS5kb21haW4oWzAs
IGQzLm1heChkYXRhLCBmdW5jdGlvbiAoZCkge3JldHVybiBkWzJdO30pXSkucmFuZ2UoW2lubmVy
SCgpLCAwXSk7fWZ1bmN0aW9uIHRvZ2dsZU9wYWNpdHkoZWxlLCBvcCkge2QzLnNlbGVjdChlbGUu
cGFyZW50Tm9kZSkuc2VsZWN0QWxsKCcuJyArIChlbGUuZ2V0QXR0cmlidXRlKCdkYXRhLXlheGlz
JykgPT0gJ3kwJyA/ICd5MScgOiAneTAnKSkuYXR0cignc3R5bGUnLCBvcCk7fWZ1bmN0aW9uIHNl
dExlZ2VuZExhYmVscyhzdmcpIHt2YXIgcmVjdCA9IHN2Zy5zZWxlY3RBbGwoJ3JlY3QubGVnZW5k
LnkwJykuZGF0YShbbnVsbF0pO3ZhciByZWN0RW50ZXIgPSByZWN0LmVudGVyKCkuYXBwZW5kKCdy
ZWN0JykuYXR0cignY2xhc3MnLCAnbGVnZW5kIHkwJykuYXR0cignZGF0YS15YXhpcycsICd5MCcp
Lm9uKCdtb3VzZW1vdmUnLCBmdW5jdGlvbihkLCBpKSB7dG9nZ2xlT3BhY2l0eSh0aGlzLCAnb3Bh
Y2l0eTowLjEnKTt9KS5vbignbW91c2VsZWF2ZScsIGZ1bmN0aW9uKGQsIGkpIHt0b2dnbGVPcGFj
aXR5KHRoaXMsIG51bGwpO30pLmF0dHIoJ3knLCAoaGVpZ2h0IC0gMTUpKTtyZWN0RW50ZXIubWVy
Z2UocmVjdCkuYXR0cigneCcsICh3aWR0aCAvIDIpIC0gMTAwKTt2YXIgdGV4dCA9IHN2Zy5zZWxl
Y3RBbGwoJ3RleHQubGVnZW5kLnkwJykuZGF0YShbbnVsbF0pO3ZhciB0ZXh0RW50ZXIgPSB0ZXh0
LmVudGVyKCkuYXBwZW5kKCd0ZXh0JykuYXR0cignY2xhc3MnLCAnbGVnZW5kIHkwJykuYXR0cign
ZGF0YS15YXhpcycsICd5MCcpLm9uKCdtb3VzZW1vdmUnLCBmdW5jdGlvbihkLCBpKSB7dG9nZ2xl
T3BhY2l0eSh0aGlzLCAnb3BhY2l0eTowLjEnKTt9KS5vbignbW91c2VsZWF2ZScsIGZ1bmN0aW9u
KGQsIGkpIHt0b2dnbGVPcGFjaXR5KHRoaXMsIG51bGwpO30pLmF0dHIoJ3knLCAoaGVpZ2h0IC0g
NikpO3RleHRFbnRlci5tZXJnZSh0ZXh0KS5hdHRyKCd4JywgKHdpZHRoIC8gMikgLSA4NSkudGV4
dChsYWJlbHMueTApO2lmICghZHVhbFlheGlzKXJldHVybjtyZWN0ID0gc3ZnLnNlbGVjdEFsbCgn
cmVjdC5sZWdlbmQueTEnKS5kYXRhKFtudWxsXSk7dmFyIHJlY3RFbnRlciA9IHJlY3QuZW50ZXIo
KS5hcHBlbmQoJ3JlY3QnKS5hdHRyKCdjbGFzcycsICdsZWdlbmQgeTEnKS5hdHRyKCdkYXRhLXlh
eGlzJywgJ3kxJykub24oJ21vdXNlbW92ZScsIGZ1bmN0aW9uKGQsIGkpIHt0b2dnbGVPcGFjaXR5
KHRoaXMsICdvcGFjaXR5OjAuMScpO30pLm9uKCdtb3VzZWxlYXZlJywgZnVuY3Rpb24oZCwgaSkg
e3RvZ2dsZU9wYWNpdHkodGhpcywgbnVsbCk7fSkuYXR0cigneScsIChoZWlnaHQgLSAxNSkpO3Jl
Y3RFbnRlci5tZXJnZShyZWN0KS5hdHRyKCd4JywgKHdpZHRoIC8gMikpO3RleHQgPSBzdmcuc2Vs
ZWN0QWxsKCd0ZXh0LmxlZ2VuZC55MScpLmRhdGEoW251bGxdKTt2YXIgdGV4dEVudGVyID0gdGV4
dC5lbnRlcigpLmFwcGVuZCgndGV4dCcpLmF0dHIoJ2NsYXNzJywgJ2xlZ2VuZCB5MScpLmF0dHIo
J2RhdGEteWF4aXMnLCAneTEnKS5vbignbW91c2Vtb3ZlJywgZnVuY3Rpb24oZCwgaSkge3RvZ2ds
ZU9wYWNpdHkodGhpcywgJ29wYWNpdHk6MC4xJyk7fSkub24oJ21vdXNlbGVhdmUnLCBmdW5jdGlv
bihkLCBpKSB7dG9nZ2xlT3BhY2l0eSh0aGlzLCBudWxsKTt9KS5hdHRyKCd5JywgKGhlaWdodCAt
IDYpKTt0ZXh0RW50ZXIubWVyZ2UodGV4dCkuYXR0cigneCcsICh3aWR0aCAvIDIpICsgMTUpLnRl
eHQobGFiZWxzLnkxKTt9ZnVuY3Rpb24gc2V0QXhpc0xhYmVscyhzdmcpIHtzdmcuc2VsZWN0QWxs
KCd0ZXh0LmF4aXMtbGFiZWwueTAnKS5kYXRhKFtudWxsXSkuZW50ZXIoKS5hcHBlbmQoJ3RleHQn
KS5hdHRyKCdjbGFzcycsICdheGlzLWxhYmVsIHkwJykuYXR0cigneScsIDEwKS5hdHRyKCd4Jywg
NTMpLnRleHQobGFiZWxzLnkwKTtpZiAoIWR1YWxZYXhpcykgcmV0dXJuO3ZhciB0RW50ZXIgPSBz
dmcuc2VsZWN0QWxsKCd0ZXh0LmF4aXMtbGFiZWwueTEnKS5kYXRhKFtudWxsXSkuZW50ZXIoKS5h
cHBlbmQoJ3RleHQnKS5hdHRyKCdjbGFzcycsICdheGlzLWxhYmVsIHkxJykuYXR0cigneScsIDEw
KS50ZXh0KGxhYmVscy55MSk7ZHVhbFlheGlzICYmIHRFbnRlci5hdHRyKCd4Jywgd2lkdGggLSAy
NSk7fWZ1bmN0aW9uIGNyZWF0ZVNrZWxldG9uKHN2Zykge2NvbnN0IGcgPSBzdmcuYXBwZW5kKCdn
Jyk7Zy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFzcycsICdsaW5lIGxpbmUwIHkwJyk7ZHVhbFlheGlz
ICYmIGcuYXBwZW5kKCdnJykuYXR0cignY2xhc3MnLCAnbGluZSBsaW5lMSB5MScpO2cuYXBwZW5k
KCdnJykuYXR0cignY2xhc3MnLCAnYXJlYSBhcmVhMCB5MCcpO2R1YWxZYXhpcyAmJiBnLmFwcGVu
ZCgnZycpLmF0dHIoJ2NsYXNzJywgJ2FyZWEgYXJlYTEgeTEnKTtnLmFwcGVuZCgnZycpLmF0dHIo
J2NsYXNzJywgJ3BvaW50cyB5MCcpO2R1YWxZYXhpcyAmJiBnLmFwcGVuZCgnZycpLmF0dHIoJ2Ns
YXNzJywgJ3BvaW50cyB5MScpO2cuYXBwZW5kKCdnJykuYXR0cignY2xhc3MnLCAneCBncmlkJyk7
Zy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFzcycsICd5IGdyaWQnKTtnLmFwcGVuZCgnZycpLmF0dHIo
J2NsYXNzJywgJ3ggYXhpcycpO2cuYXBwZW5kKCdnJykuYXR0cignY2xhc3MnLCAneTAgYXhpcycp
O2R1YWxZYXhpcyAmJiBnLmFwcGVuZCgnZycpLmF0dHIoJ2NsYXNzJywgJ3kxIGF4aXMnKTtnLmFw
cGVuZCgnZycpLmF0dHIoJ2NsYXNzJywgJ3JlY3RzJyk7c2V0QXhpc0xhYmVscyhzdmcpO3NldExl
Z2VuZExhYmVscyhzdmcpO2cuYXBwZW5kKCdsaW5lJykuYXR0cigneTInLCBpbm5lckgoKSkuYXR0
cigneTEnLCAwKS5hdHRyKCdjbGFzcycsICdpbmRpY2F0b3InKTt9ZnVuY3Rpb24gcGF0aExlbihk
KSB7cmV0dXJuIGQubm9kZSgpLmdldFRvdGFsTGVuZ3RoKCk7fWZ1bmN0aW9uIGFkZExpbmUoZywg
ZGF0YSwgbGluZSwgY05hbWUpIHt2YXIgcGF0aCA9IGcuc2VsZWN0KCdnLicgKyBjTmFtZSkuc2Vs
ZWN0QWxsKCdwYXRoLicgKyBjTmFtZSkuZGF0YShbZGF0YV0pO3ZhciBwYXRoRW50ZXIgPSBwYXRo
LmVudGVyKCkuYXBwZW5kKCdzdmc6cGF0aCcpLmF0dHIoJ2QnLCBsaW5lKS5hdHRyKCdjbGFzcycs
IGNOYW1lKS5hdHRyKCdzdHJva2UtZGFzaGFycmF5JywgZnVuY3Rpb24oZCkge3ZhciBwbCA9IHBh
dGhMZW4oZDMuc2VsZWN0KHRoaXMpKTtyZXR1cm4gcGwgKyAnICcgKyBwbDt9KS5hdHRyKCdzdHJv
a2UtZGFzaG9mZnNldCcsIGZ1bmN0aW9uKGQpIHtyZXR1cm4gcGF0aExlbihkMy5zZWxlY3QodGhp
cykpO30pO3BhdGhFbnRlci5tZXJnZShwYXRoKS5hdHRyKCdkJywgbGluZSkudHJhbnNpdGlvbigp
LmF0dHIoJ3N0cm9rZS1kYXNoYXJyYXknLCBmdW5jdGlvbihkKSB7dmFyIHBsID0gcGF0aExlbihk
My5zZWxlY3QodGhpcykpO3JldHVybiBwbCArICcgJyArIHBsO30pLmR1cmF0aW9uKDIwMDApLmF0
dHIoJ3N0cm9rZS1kYXNob2Zmc2V0JywgMCk7cGF0aC5leGl0KCkucmVtb3ZlKCk7fWZ1bmN0aW9u
IGFkZEFyZWEoZywgZGF0YSwgY2IsIGNOYW1lKSB7dmFyIGFyZWEgPSBnLnNlbGVjdCgnZy4nICsg
Y05hbWUpLnNlbGVjdEFsbCgncGF0aC4nICsgY05hbWUpLmRhdGEoW2RhdGFdKTt2YXIgYXJlYUVu
dGVyID0gYXJlYS5lbnRlcigpLmFwcGVuZCgnc3ZnOnBhdGgnKS5hdHRyKCdjbGFzcycsIGNOYW1l
KTthcmVhRW50ZXIubWVyZ2UoYXJlYSkuYXR0cignZCcsIGNiKTthcmVhLmV4aXQoKS5yZW1vdmUo
KTt9ZnVuY3Rpb24gYWRkQXJlYUxpbmVzKGcsIGRhdGEpIHthZGRBcmVhKGcsIGRhdGEsIGFyZWEw
LnkwKHlTY2FsZTAucmFuZ2UoKVswXSksICdhcmVhMCcpO2FkZExpbmUoZywgZGF0YSwgbGluZTAs
ICdsaW5lMCcpO2FkZEFyZWEoZywgZGF0YSwgYXJlYTEueTEoeVNjYWxlMS5yYW5nZSgpWzBdKSwg
J2FyZWExJyk7YWRkTGluZShnLCBkYXRhLCBsaW5lMSwgJ2xpbmUxJyk7fWZ1bmN0aW9uIGFkZFBv
aW50cyhnLCBkYXRhKSB7dmFyIHJhZGl1cyA9IGRhdGEubGVuZ3RoID4gMTAwID8gMSA6IDIuNTt2
YXIgcG9pbnRzID0gZy5zZWxlY3QoJ2cucG9pbnRzLnkwJykuc2VsZWN0QWxsKCdjaXJjbGUucG9p
bnQnKS5kYXRhKGRhdGEpO3ZhciBwb2ludHNFbnRlciA9IHBvaW50cy5lbnRlcigpLmFwcGVuZCgn
c3ZnOmNpcmNsZScpLmF0dHIoJ3InLCByYWRpdXMpLmF0dHIoJ2NsYXNzJywgJ3BvaW50Jyk7cG9p
bnRzRW50ZXIubWVyZ2UocG9pbnRzKS5hdHRyKCdjeCcsIGZ1bmN0aW9uKGQpIHtyZXR1cm4gKHhT
Y2FsZShkWzBdKSArIHhTY2FsZS5iYW5kd2lkdGgoKSAvIDIpO30pLmF0dHIoJ2N5JywgZnVuY3Rp
b24oZCkge3JldHVybiB5U2NhbGUwKGRbMV0pO30pO3BvaW50cy5leGl0KCkucmVtb3ZlKCk7aWYg
KCFkdWFsWWF4aXMpcmV0dXJuO3BvaW50cyA9IGcuc2VsZWN0KCdnLnBvaW50cy55MScpLnNlbGVj
dEFsbCgnY2lyY2xlLnBvaW50JykuZGF0YShkYXRhKTtwb2ludHNFbnRlciA9IHBvaW50cy5lbnRl
cigpLmFwcGVuZCgnc3ZnOmNpcmNsZScpLmF0dHIoJ3InLCByYWRpdXMpLmF0dHIoJ2NsYXNzJywg
J3BvaW50Jyk7cG9pbnRzRW50ZXIubWVyZ2UocG9pbnRzKS5hdHRyKCdjeCcsIGZ1bmN0aW9uKGQp
IHtyZXR1cm4gKHhTY2FsZShkWzBdKSArIHhTY2FsZS5iYW5kd2lkdGgoKSAvIDIpO30pLmF0dHIo
J2N5JywgZnVuY3Rpb24oZCkge3JldHVybiB5U2NhbGUxKGRbMl0pO30pO3BvaW50cy5leGl0KCku
cmVtb3ZlKCk7fWZ1bmN0aW9uIGFkZEF4aXMoZywgZGF0YSkge3ZhciB4VGlja3MgPSBnZXRYVGlj
a3MoZGF0YSk7dmFyIHRpY2tEaXN0YW5jZSA9IHhUaWNrcy5sZW5ndGggPiAxID8gKHhTY2FsZSh4
VGlja3NbMV0pIC0geFNjYWxlKHhUaWNrc1swXSkpIDogaW5uZXJXKCk7dmFyIGxhYmVsVyA9IHRp
Y2tEaXN0YW5jZSAtIHBhZGRpbmc7Zy5zZWxlY3QoJy54LmF4aXMnKS5hdHRyKCd0cmFuc2Zvcm0n
LCAndHJhbnNsYXRlKDAsJyArIHlTY2FsZTAucmFuZ2UoKVswXSArICcpJykuY2FsbCh4QXhpcy50
aWNrVmFsdWVzKHhUaWNrcykpLnNlbGVjdEFsbCgiLnRpY2sgdGV4dCIpLmNhbGwodHJ1bmNhdGUs
IGxhYmVsVyA+IDAgPyBsYWJlbFcgOiBpbm5lclcoKSk7Zy5zZWxlY3QoJy55MC5heGlzJykuY2Fs
bCh5QXhpczAudGlja1ZhbHVlcyhnZXRZVGlja3MoeVNjYWxlMCkpKTtpZiAoIWR1YWxZYXhpcyly
ZXR1cm47Zy5zZWxlY3QoJy55MS5heGlzJykuYXR0cigndHJhbnNmb3JtJywgJ3RyYW5zbGF0ZSgn
ICsgaW5uZXJXKCkgKyAnLCAwKScpLmNhbGwoeUF4aXMxLnRpY2tWYWx1ZXMoZ2V0WVRpY2tzKHlT
Y2FsZTEpKSk7fWZ1bmN0aW9uIGFkZEdyaWQoZywgZGF0YSkge2cuc2VsZWN0KCcueC5ncmlkJyku
YXR0cigndHJhbnNmb3JtJywgJ3RyYW5zbGF0ZSgwLCcgKyB5U2NhbGUwLnJhbmdlKClbMF0gKyAn
KScpLmNhbGwoeEdyaWQudGlja1ZhbHVlcyhnZXRYVGlja3MoZGF0YSkpLnRpY2tTaXplKC1pbm5l
ckgoKSwgMCwgMCkudGlja1NpemVPdXRlcigwKS50aWNrRm9ybWF0KCcnKSk7Zy5zZWxlY3QoJy55
LmdyaWQnKS5jYWxsKHlHcmlkLnRpY2tWYWx1ZXMoZ2V0WVRpY2tzKHlTY2FsZTApKS50aWNrU2l6
ZSgtaW5uZXJXKCksIDApLnRpY2tTaXplT3V0ZXIoMCkudGlja0Zvcm1hdCgnJykpO31mdW5jdGlv
biBmb3JtYXRUb29sdGlwKGRhdGEpIHt2YXIgZCA9IGRhdGEuc2xpY2UoMCk7ZFswXSA9IChmb3Jt
YXQueCkgPyBHb0FjY2Vzcy5VdGlsLmZtdFZhbHVlKGRbMF0sIGZvcm1hdC54KSA6IGRbMF07ZFsx
XSA9IChmb3JtYXQueTApID8gR29BY2Nlc3MuVXRpbC5mbXRWYWx1ZShkWzFdLCBmb3JtYXQueTAp
IDogZDMuZm9ybWF0KCcsJykoZFsxXSk7ZHVhbFlheGlzICYmIChkWzJdID0gKGZvcm1hdC55MSkg
PyBHb0FjY2Vzcy5VdGlsLmZtdFZhbHVlKGRbMl0sIGZvcm1hdC55MSkgOiBkMy5mb3JtYXQoJywn
KShkWzJdKSk7dmFyIHRlbXBsYXRlID0gZDMuc2VsZWN0KCcjdHBsLWNoYXJ0LXRvb2x0aXAnKS5o
dG1sKCk7cmV0dXJuIEhvZ2FuLmNvbXBpbGUodGVtcGxhdGUpLnJlbmRlcih7J2RhdGEnOiBkfSk7
fWZ1bmN0aW9uIG1vdXNlb3ZlcihldmVudCwgc2VsZWN0aW9uLCBkYXRhKSB7dmFyIHRvb2x0aXAg
PSBzZWxlY3Rpb24uc2VsZWN0KCcuY2hhcnQtdG9vbHRpcC13cmFwJyk7dG9vbHRpcC5odG1sKGZv
cm1hdFRvb2x0aXAoZGF0YSkpLnN0eWxlKCdsZWZ0JywgWChkYXRhKSArICdweCcpLnN0eWxlKCd0
b3AnLCAgKGQzLnBvaW50ZXIoZXZlbnQpWzFdICsgMTApICsgJ3B4Jykuc3R5bGUoJ2Rpc3BsYXkn
LCAnYmxvY2snKTtzZWxlY3Rpb24uc2VsZWN0KCdsaW5lLmluZGljYXRvcicpLnN0eWxlKCdkaXNw
bGF5JywgJ2Jsb2NrJykuYXR0cigndHJhbnNmb3JtJywgJ3RyYW5zbGF0ZSgnICsgWChkYXRhKSAr
ICcsJyArIDAgKyAnKScpO31mdW5jdGlvbiBtb3VzZW91dChzZWxlY3Rpb24sIGcpIHt2YXIgdG9v
bHRpcCA9IHNlbGVjdGlvbi5zZWxlY3QoJy5jaGFydC10b29sdGlwLXdyYXAnKTt0b29sdGlwLnN0
eWxlKCdkaXNwbGF5JywgJ25vbmUnKTtnLnNlbGVjdCgnbGluZS5pbmRpY2F0b3InKS5zdHlsZSgn
ZGlzcGxheScsICdub25lJyk7fWZ1bmN0aW9uIGFkZFJlY3RzKHNlbGVjdGlvbiwgZywgZGF0YSkg
e3ZhciB3ID0gKGlubmVyVygpIC8gZGF0YS5sZW5ndGgpO3ZhciByZWN0cyA9IGcuc2VsZWN0KCdn
LnJlY3RzJykuc2VsZWN0QWxsKCdyZWN0JykuZGF0YShkYXRhKTt2YXIgcmVjdHNFbnRlciA9IHJl
Y3RzLmVudGVyKCkuYXBwZW5kKCdzdmc6cmVjdCcpLmF0dHIoJ2hlaWdodCcsIGlubmVySCgpKS5h
dHRyKCdjbGFzcycsICdwb2ludCcpO3JlY3RzRW50ZXIubWVyZ2UocmVjdHMpLmF0dHIoJ3dpZHRo
JywgdykuYXR0cigneCcsIGZ1bmN0aW9uKGQsIGkpIHtyZXR1cm4gKHcgKiBpKTt9KS5hdHRyKCd5
JywgMCkub24oJ21vdXNlbW92ZScsIGZ1bmN0aW9uKGV2ZW50KSB7bW91c2VvdmVyKGV2ZW50LCBz
ZWxlY3Rpb24sIGQzLnNlbGVjdCh0aGlzKS5kYXR1bSgpKTt9KS5vbignbW91c2VsZWF2ZScsIGZ1
bmN0aW9uKGV2ZW50KSB7bW91c2VvdXQoc2VsZWN0aW9uLCBnKTt9KTtyZWN0cy5leGl0KCkucmVt
b3ZlKCk7fWZ1bmN0aW9uIGNoYXJ0KHNlbGVjdGlvbikge3NlbGVjdGlvbi5lYWNoKGZ1bmN0aW9u
IChkYXRhKSB7ZGF0YSA9IG1hcERhdGEoZGF0YSk7dXBkYXRlU2NhbGVzKGRhdGEpO2xldCBzdmcg
PSBkMy5zZWxlY3QodGhpcykuc2VsZWN0KCdzdmcnKTtpZiAoc3ZnLmVtcHR5KCkpIHtzdmcgPSBk
My5zZWxlY3QodGhpcykuYXBwZW5kKCdzdmcnKS5hdHRyKCd3aWR0aCcsIHdpZHRoKS5hdHRyKCdo
ZWlnaHQnLCBoZWlnaHQpO2NyZWF0ZVNrZWxldG9uKHN2Zyk7fXZhciBnID0gc3ZnLnNlbGVjdCgn
ZycpLmF0dHIoJ3RyYW5zZm9ybScsICd0cmFuc2xhdGUoJyArIG1hcmdpbi5sZWZ0ICsgJywnICsg
bWFyZ2luLnRvcCArICcpJyk7YWRkR3JpZChnLCBkYXRhKTthZGRBcmVhTGluZXMoZywgZGF0YSk7
YWRkUG9pbnRzKGcsIGRhdGEpO2FkZEF4aXMoZywgZGF0YSk7YWRkUmVjdHMoc2VsZWN0aW9uLCBn
LCBkYXRhKTt9KTt9Y2hhcnQub3B0cyA9IGZ1bmN0aW9uIChfKSB7aWYgKCFhcmd1bWVudHMubGVu
Z3RoKSByZXR1cm4gb3B0cztvcHRzID0gXztyZXR1cm4gY2hhcnQ7fTtjaGFydC5mb3JtYXQgPSBm
dW5jdGlvbiAoXykge2lmICghYXJndW1lbnRzLmxlbmd0aCkgcmV0dXJuIGZvcm1hdDtmb3JtYXQg
PSBfO3JldHVybiBjaGFydDt9O2NoYXJ0LmxhYmVscyA9IGZ1bmN0aW9uIChfKSB7aWYgKCFhcmd1
bWVudHMubGVuZ3RoKSByZXR1cm4gbGFiZWxzO2xhYmVscyA9IF87cmV0dXJuIGNoYXJ0O307Y2hh
cnQubWFyZ2luID0gZnVuY3Rpb24gKF8pIHtpZiAoIWFyZ3VtZW50cy5sZW5ndGgpIHJldHVybiBt
YXJnaW47bWFyZ2luID0gXztyZXR1cm4gY2hhcnQ7fTtjaGFydC53aWR0aCA9IGZ1bmN0aW9uIChf
KSB7aWYgKCFhcmd1bWVudHMubGVuZ3RoKSByZXR1cm4gd2lkdGg7d2lkdGggPSBfO3JldHVybiBj
aGFydDt9O2NoYXJ0LmhlaWdodCA9IGZ1bmN0aW9uIChfKSB7aWYgKCFhcmd1bWVudHMubGVuZ3Ro
KSByZXR1cm4gaGVpZ2h0O2hlaWdodCA9IF87cmV0dXJuIGNoYXJ0O307Y2hhcnQueCA9IGZ1bmN0
aW9uIChfKSB7aWYgKCFhcmd1bWVudHMubGVuZ3RoKSByZXR1cm4geFZhbHVlO3hWYWx1ZSA9IF87
cmV0dXJuIGNoYXJ0O307Y2hhcnQueTAgPSBmdW5jdGlvbiAoXykge2lmICghYXJndW1lbnRzLmxl
bmd0aCkgcmV0dXJuIHlWYWx1ZTA7eVZhbHVlMCA9IF87cmV0dXJuIGNoYXJ0O307Y2hhcnQueTEg
PSBmdW5jdGlvbiAoXykge2lmICghYXJndW1lbnRzLmxlbmd0aCkgcmV0dXJuIHlWYWx1ZTE7eVZh
bHVlMSA9IF87cmV0dXJuIGNoYXJ0O307cmV0dXJuIGNoYXJ0O31mdW5jdGlvbiBCYXJDaGFydChk
dWFsWWF4aXMpIHt2YXIgb3B0cyA9IHt9O3ZhciBtYXJnaW4gPSB7dG9wICAgIDogMjAscmlnaHQg
IDogNTAsYm90dG9tIDogNDAsbGVmdCAgIDogNTAsfSxoZWlnaHQgPSAxNzAsblRpY2tzID0gMTAs
cGFkZGluZyA9IDEwLHdpZHRoID0gNzYwO3ZhciBsYWJlbHMgPSB7IHg6ICdVbm5hbWVkJywgeTA6
ICdVbm5hbWVkJywgeTE6ICdVbm5hbWVkJyB9O3ZhciBmb3JtYXQgPSB7IHg6IG51bGwsIHkwOiBu
dWxsLCB5MTogbnVsbH07dmFyIHhWYWx1ZSA9IGZ1bmN0aW9uIChkKSB7cmV0dXJuIGRbMF07fSx5
VmFsdWUwID0gZnVuY3Rpb24gKGQpIHtyZXR1cm4gZFsxXTt9LHlWYWx1ZTEgPSBmdW5jdGlvbiAo
ZCkge3JldHVybiBkWzJdO307dmFyIHhTY2FsZSA9IGQzLnNjYWxlQmFuZCgpLnBhZGRpbmdJbm5l
cigwLjEpLnBhZGRpbmdPdXRlcigwLjEpO3ZhciB5U2NhbGUwID0gZDMuc2NhbGVMaW5lYXIoKS5u
aWNlKCk7dmFyIHlTY2FsZTEgPSBkMy5zY2FsZUxpbmVhcigpLm5pY2UoKTt2YXIgeEF4aXMgPSBk
My5heGlzQm90dG9tKHhTY2FsZSkudGlja0Zvcm1hdChmdW5jdGlvbiAoZCkge2lmIChmb3JtYXQu
eClyZXR1cm4gR29BY2Nlc3MuVXRpbC5mbXRWYWx1ZShkLCBmb3JtYXQueCk7cmV0dXJuIGQ7fSk7
dmFyIHlBeGlzMCA9IGQzLmF4aXNMZWZ0KHlTY2FsZTApLnRpY2tGb3JtYXQoZnVuY3Rpb24gKGQp
IHtyZXR1cm4gZDMuZm9ybWF0KCcuMnMnKShkKTt9KTt2YXIgeUF4aXMxID0gZDMuYXhpc1JpZ2h0
KHlTY2FsZTEpLnRpY2tGb3JtYXQoZnVuY3Rpb24gKGQpIHtpZiAoZm9ybWF0LnkxKXJldHVybiBH
b0FjY2Vzcy5VdGlsLmZtdFZhbHVlKGQsIGZvcm1hdC55MSk7cmV0dXJuIGQzLmZvcm1hdCgnLjJz
JykoZCk7fSk7dmFyIHhHcmlkID0gZDMuYXhpc0JvdHRvbSh4U2NhbGUpO3ZhciB5R3JpZCA9IGQz
LmF4aXNMZWZ0KHlTY2FsZTApO2Z1bmN0aW9uIGlubmVyVygpIHtyZXR1cm4gd2lkdGggLSBtYXJn
aW4ubGVmdCAtIG1hcmdpbi5yaWdodDt9ZnVuY3Rpb24gaW5uZXJIKCkge3JldHVybiBoZWlnaHQg
LSBtYXJnaW4udG9wIC0gbWFyZ2luLmJvdHRvbTt9ZnVuY3Rpb24gZ2V0WFRpY2tzKGRhdGEpIHtj
b25zdCBkb21haW4gPSB4U2NhbGUuZG9tYWluKCk7aWYgKGRhdGEubGVuZ3RoIDwgblRpY2tzKXJl
dHVybiBkb21haW47cmV0dXJuIGQzLnJhbmdlKDAsIG5UaWNrcykubWFwKGZ1bmN0aW9uKGkpIHtj
b25zdCBpbmRleCA9IE1hdGguZmxvb3IoaSAqIChkb21haW4ubGVuZ3RoIC0gMSkgLyAoblRpY2tz
IC0gMSkpO2lmIChpbmRleCA+PSAwICYmIGluZGV4IDwgZG9tYWluLmxlbmd0aClyZXR1cm4gZG9t
YWluW2luZGV4XTtyZXR1cm4gbnVsbDt9KTt9ZnVuY3Rpb24gZ2V0WVRpY2tzKHNjYWxlKSB7dmFy
IGRvbWFpbiA9IHNjYWxlLmRvbWFpbigpO3JldHVybiBkMy5yYW5nZShkb21haW5bMF0sIGRvbWFp
blsxXSwgTWF0aC5jZWlsKGRvbWFpblsxXSAvIG5UaWNrcykpO31mdW5jdGlvbiBYKGQpIHtyZXR1
cm4gKHhTY2FsZShkWzBdKSArIHhTY2FsZS5iYW5kd2lkdGgoKSAvIDIpO31mdW5jdGlvbiBtYXBE
YXRhKGRhdGEpIHt2YXIgX2RhdHVtID0gZnVuY3Rpb24gKGQsIGkpIHt2YXIgZGF0dW0gPSBbeFZh
bHVlLmNhbGwoZGF0YSwgZCwgaSksIHlWYWx1ZTAuY2FsbChkYXRhLCBkLCBpKV07ZHVhbFlheGlz
ICYmIGRhdHVtLnB1c2goeVZhbHVlMS5jYWxsKGRhdGEsIGQsIGkpKTtyZXR1cm4gZGF0dW07fTty
ZXR1cm4gZGF0YS5tYXAoZnVuY3Rpb24gKGQsIGkpIHtyZXR1cm4gX2RhdHVtKGQsIGkpO30pO31m
dW5jdGlvbiB1cGRhdGVTY2FsZXMoZGF0YSkge3hTY2FsZS5kb21haW4oZGF0YS5tYXAoZnVuY3Rp
b24gKGQpIHtyZXR1cm4gZFswXTt9KSkucmFuZ2UoWzAsIGlubmVyVygpXSk7eVNjYWxlMC5kb21h
aW4oWzAsIGQzLm1heChkYXRhLCBmdW5jdGlvbiAoZCkge3JldHVybiBkWzFdO30pXSkucmFuZ2Uo
W2lubmVySCgpLCAwXSk7ZHVhbFlheGlzICYmIHlTY2FsZTEuZG9tYWluKFswLCBkMy5tYXgoZGF0
YSwgZnVuY3Rpb24gKGQpIHtyZXR1cm4gZFsyXTt9KSArIDAuMV0pLnJhbmdlKFtpbm5lckgoKSwg
MF0pO31mdW5jdGlvbiB0b2dnbGVPcGFjaXR5KGVsZSwgb3ApIHtkMy5zZWxlY3QoZWxlLnBhcmVu
dE5vZGUpLnNlbGVjdEFsbCgnLicgKyAoZWxlLmdldEF0dHJpYnV0ZSgnZGF0YS15YXhpcycpID09
ICd5MCcgPyAneTEnIDogJ3kwJykpLmF0dHIoJ3N0eWxlJywgb3ApO31mdW5jdGlvbiBzZXRMZWdl
bmRMYWJlbHMoc3ZnKSB7dmFyIHJlY3QgPSBzdmcuc2VsZWN0QWxsKCdyZWN0LmxlZ2VuZC55MCcp
LmRhdGEoW251bGxdKTt2YXIgcmVjdEVudGVyID0gcmVjdC5lbnRlcigpLmFwcGVuZCgncmVjdCcp
LmF0dHIoJ2NsYXNzJywgJ2xlZ2VuZCB5MCcpLmF0dHIoJ2RhdGEteWF4aXMnLCAneTAnKS5vbign
bW91c2Vtb3ZlJywgZnVuY3Rpb24oZCwgaSkge3RvZ2dsZU9wYWNpdHkodGhpcywgJ29wYWNpdHk6
MC4xJyk7fSkub24oJ21vdXNlbGVhdmUnLCBmdW5jdGlvbihkLCBpKSB7dG9nZ2xlT3BhY2l0eSh0
aGlzLCBudWxsKTt9KS5hdHRyKCd5JywgKGhlaWdodCAtIDE1KSk7cmVjdEVudGVyLm1lcmdlKHJl
Y3QpLmF0dHIoJ3gnLCAod2lkdGggLyAyKSAtIDEwMCk7dmFyIHRleHQgPSBzdmcuc2VsZWN0QWxs
KCd0ZXh0LmxlZ2VuZC55MCcpLmRhdGEoW251bGxdKTt2YXIgdGV4dEVudGVyID0gdGV4dC5lbnRl
cigpLmFwcGVuZCgndGV4dCcpLmF0dHIoJ2NsYXNzJywgJ2xlZ2VuZCB5MCcpLmF0dHIoJ2RhdGEt
eWF4aXMnLCAneTAnKS5vbignbW91c2Vtb3ZlJywgZnVuY3Rpb24oZCwgaSkge3RvZ2dsZU9wYWNp
dHkodGhpcywgJ29wYWNpdHk6MC4xJyk7fSkub24oJ21vdXNlbGVhdmUnLCBmdW5jdGlvbihkLCBp
KSB7dG9nZ2xlT3BhY2l0eSh0aGlzLCBudWxsKTt9KS5hdHRyKCd5JywgKGhlaWdodCAtIDYpKTt0
ZXh0RW50ZXIubWVyZ2UodGV4dCkuYXR0cigneCcsICh3aWR0aCAvIDIpIC0gODUpLnRleHQobGFi
ZWxzLnkwKTtpZiAoIWR1YWxZYXhpcylyZXR1cm47cmVjdCA9IHN2Zy5zZWxlY3RBbGwoJ3JlY3Qu
bGVnZW5kLnkxJykuZGF0YShbbnVsbF0pO3ZhciByZWN0RW50ZXIgPSByZWN0LmVudGVyKCkuYXBw
ZW5kKCdyZWN0JykuYXR0cignY2xhc3MnLCAnbGVnZW5kIHkxJykuYXR0cignZGF0YS15YXhpcycs
ICd5MScpLm9uKCdtb3VzZW1vdmUnLCBmdW5jdGlvbihkLCBpKSB7dG9nZ2xlT3BhY2l0eSh0aGlz
LCAnb3BhY2l0eTowLjEnKTt9KS5vbignbW91c2VsZWF2ZScsIGZ1bmN0aW9uKGQsIGkpIHt0b2dn
bGVPcGFjaXR5KHRoaXMsIG51bGwpO30pLmF0dHIoJ3knLCAoaGVpZ2h0IC0gMTUpKTtyZWN0RW50
ZXIubWVyZ2UocmVjdCkuYXR0cigneCcsICh3aWR0aCAvIDIpKTt0ZXh0ID0gc3ZnLnNlbGVjdEFs
bCgndGV4dC5sZWdlbmQueTEnKS5kYXRhKFtudWxsXSk7dmFyIHRleHRFbnRlciA9IHRleHQuZW50
ZXIoKS5hcHBlbmQoJ3RleHQnKS5hdHRyKCdjbGFzcycsICdsZWdlbmQgeTEnKS5hdHRyKCdkYXRh
LXlheGlzJywgJ3kxJykub24oJ21vdXNlbW92ZScsIGZ1bmN0aW9uKGQsIGkpIHt0b2dnbGVPcGFj
aXR5KHRoaXMsICdvcGFjaXR5OjAuMScpO30pLm9uKCdtb3VzZWxlYXZlJywgZnVuY3Rpb24oZCwg
aSkge3RvZ2dsZU9wYWNpdHkodGhpcywgbnVsbCk7fSkuYXR0cigneScsIChoZWlnaHQgLSA2KSk7
dGV4dEVudGVyLm1lcmdlKHRleHQpLmF0dHIoJ3gnLCAod2lkdGggLyAyKSArIDE1KS50ZXh0KGxh
YmVscy55MSk7fWZ1bmN0aW9uIHNldEF4aXNMYWJlbHMoc3ZnKSB7c3ZnLnNlbGVjdEFsbCgndGV4
dC5heGlzLWxhYmVsLnkwJykuZGF0YShbbnVsbF0pLmVudGVyKCkuYXBwZW5kKCd0ZXh0JykuYXR0
cignY2xhc3MnLCAnYXhpcy1sYWJlbCB5MCcpLmF0dHIoJ3knLCAxMCkuYXR0cigneCcsIDUzKS50
ZXh0KGxhYmVscy55MCk7aWYgKCFkdWFsWWF4aXMpIHJldHVybjt2YXIgdEVudGVyID0gc3ZnLnNl
bGVjdEFsbCgndGV4dC5heGlzLWxhYmVsLnkxJykuZGF0YShbbnVsbF0pLmVudGVyKCkuYXBwZW5k
KCd0ZXh0JykuYXR0cignY2xhc3MnLCAnYXhpcy1sYWJlbCB5MScpLmF0dHIoJ3knLCAxMCkudGV4
dChsYWJlbHMueTEpO2R1YWxZYXhpcyAmJiB0RW50ZXIuYXR0cigneCcsIHdpZHRoIC0gMjUpO31m
dW5jdGlvbiBjcmVhdGVTa2VsZXRvbihzdmcpIHtjb25zdCBnID0gc3ZnLmFwcGVuZCgnZycpO2cu
YXBwZW5kKCdnJykuYXR0cignY2xhc3MnLCAneCBncmlkJyk7Zy5hcHBlbmQoJ2cnKS5hdHRyKCdj
bGFzcycsICd5IGdyaWQnKTtnLmFwcGVuZCgnZycpLmF0dHIoJ2NsYXNzJywgJ3ggYXhpcycpO2cu
YXBwZW5kKCdnJykuYXR0cignY2xhc3MnLCAneTAgYXhpcycpO2R1YWxZYXhpcyAmJiBnLmFwcGVu
ZCgnZycpLmF0dHIoJ2NsYXNzJywgJ3kxIGF4aXMnKTtnLmFwcGVuZCgnZycpLmF0dHIoJ2NsYXNz
JywgJ2JhcnMgeTAnKTtkdWFsWWF4aXMgJiYgZy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFzcycsICdi
YXJzIHkxJyk7Zy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFzcycsICdyZWN0cycpO3NldEF4aXNMYWJl
bHMoc3ZnKTtzZXRMZWdlbmRMYWJlbHMoc3ZnKTtnLmFwcGVuZCgnbGluZScpLmF0dHIoJ3kyJywg
aW5uZXJIKCkpLmF0dHIoJ3kxJywgMCkuYXR0cignY2xhc3MnLCAnaW5kaWNhdG9yJyk7fWZ1bmN0
aW9uIGFkZEJhcnMoZywgZGF0YSkge3ZhciBiYXJzID0gZy5zZWxlY3QoJ2cuYmFycy55MCcpLnNl
bGVjdEFsbCgncmVjdC5iYXInKS5kYXRhKGRhdGEpO3ZhciBlbnRlciA9IGJhcnMuZW50ZXIoKS5h
cHBlbmQoJ3N2ZzpyZWN0JykuYXR0cignY2xhc3MnLCAnYmFyJykuYXR0cignaGVpZ2h0JywgMCku
YXR0cignd2lkdGgnLCBmdW5jdGlvbiAoZCwgaSkgeyByZXR1cm4geFNjYWxlLmJhbmR3aWR0aCgp
IC8gMjsgfSkuYXR0cigneCcsIGZ1bmN0aW9uIChkLCBpKSB7IHJldHVybiB4U2NhbGUoZFswXSk7
IH0pLmF0dHIoJ3knLCBmdW5jdGlvbiAoZCwgaSkgeyByZXR1cm4gaW5uZXJIKCk7IH0pO2JhcnMu
bWVyZ2UoZW50ZXIpLmF0dHIoJ3dpZHRoJywgeFNjYWxlLmJhbmR3aWR0aCgpIC8gMikuYXR0cign
eCcsIGZ1bmN0aW9uIChkKSB7IHJldHVybiB4U2NhbGUoZFswXSk7IH0pLnRyYW5zaXRpb24oKS5k
ZWxheShmdW5jdGlvbiAoZCwgaSkgeyByZXR1cm4gaSAvIGRhdGEubGVuZ3RoICogMTAwMDsgfSku
ZHVyYXRpb24oNTAwKS5hdHRyKCdoZWlnaHQnLCBmdW5jdGlvbiAoZCwgaSkgeyByZXR1cm4gaW5u
ZXJIKCkgLSB5U2NhbGUwKGRbMV0pOyB9KS5hdHRyKCd5JywgZnVuY3Rpb24gKGQsIGkpIHsgcmV0
dXJuIHlTY2FsZTAoZFsxXSk7IH0pO2JhcnMuZXhpdCgpLnJlbW92ZSgpO2lmICghZHVhbFlheGlz
KXJldHVybjtiYXJzID0gZy5zZWxlY3QoJ2cuYmFycy55MScpLnNlbGVjdEFsbCgncmVjdC5iYXIn
KS5kYXRhKGRhdGEpO2VudGVyID0gYmFycy5lbnRlcigpLmFwcGVuZCgnc3ZnOnJlY3QnKS5hdHRy
KCdjbGFzcycsICdiYXInKS5hdHRyKCdoZWlnaHQnLCAwKS5hdHRyKCd3aWR0aCcsIGZ1bmN0aW9u
IChkLCBpKSB7IHJldHVybiB4U2NhbGUuYmFuZHdpZHRoKCkgLyAyOyB9KS5hdHRyKCd4JywgZnVu
Y3Rpb24gKGQpIHsgcmV0dXJuICh4U2NhbGUoZFswXSkgKyB4U2NhbGUuYmFuZHdpZHRoKCkgLyAy
KTsgfSkuYXR0cigneScsIGZ1bmN0aW9uIChkLCBpKSB7IHJldHVybiBpbm5lckgoKTsgfSk7YmFy
cy5tZXJnZShlbnRlcikuYXR0cignd2lkdGgnLCB4U2NhbGUuYmFuZHdpZHRoKCkgLyAyKS5hdHRy
KCd4JywgZnVuY3Rpb24gKGQpIHsgcmV0dXJuICh4U2NhbGUoZFswXSkgKyB4U2NhbGUuYmFuZHdp
ZHRoKCkgLyAyKTsgfSkudHJhbnNpdGlvbigpLmRlbGF5KGZ1bmN0aW9uIChkLCBpKSB7IHJldHVy
biBpIC8gZGF0YS5sZW5ndGggKiAxMDAwOyB9KS5kdXJhdGlvbig1MDApLmF0dHIoJ2hlaWdodCcs
IGZ1bmN0aW9uIChkLCBpKSB7IHJldHVybiBpbm5lckgoKSAtIHlTY2FsZTEoZFsyXSk7IH0pLmF0
dHIoJ3knLCBmdW5jdGlvbiAoZCwgaSkgeyByZXR1cm4geVNjYWxlMShkWzJdKTsgfSk7YmFycy5l
eGl0KCkucmVtb3ZlKCk7fWZ1bmN0aW9uIGFkZEF4aXMoZywgZGF0YSkge3ZhciB4VGlja3MgPSBn
ZXRYVGlja3MoZGF0YSk7dmFyIHRpY2tEaXN0YW5jZSA9IHhUaWNrcy5sZW5ndGggPiAxID8gKHhT
Y2FsZSh4VGlja3NbMV0pIC0geFNjYWxlKHhUaWNrc1swXSkpIDogaW5uZXJXKCk7dmFyIGxhYmVs
VyA9IHRpY2tEaXN0YW5jZSAtIHBhZGRpbmc7Zy5zZWxlY3QoJy54LmF4aXMnKS5hdHRyKCd0cmFu
c2Zvcm0nLCAndHJhbnNsYXRlKDAsJyArIHlTY2FsZTAucmFuZ2UoKVswXSArICcpJykuY2FsbCh4
QXhpcy50aWNrVmFsdWVzKHhUaWNrcykpLnNlbGVjdEFsbCgiLnRpY2sgdGV4dCIpLmNhbGwodHJ1
bmNhdGUsIGxhYmVsVyA+IDAgPyBsYWJlbFcgOiBpbm5lclcoKSk7Zy5zZWxlY3QoJy55MC5heGlz
JykuY2FsbCh5QXhpczAudGlja1ZhbHVlcyhnZXRZVGlja3MoeVNjYWxlMCkpKTtpZiAoIWR1YWxZ
YXhpcylyZXR1cm47Zy5zZWxlY3QoJy55MS5heGlzJykuYXR0cigndHJhbnNmb3JtJywgJ3RyYW5z
bGF0ZSgnICsgaW5uZXJXKCkgKyAnLCAwKScpLmNhbGwoeUF4aXMxLnRpY2tWYWx1ZXMoZ2V0WVRp
Y2tzKHlTY2FsZTEpKSk7fWZ1bmN0aW9uIGFkZEdyaWQoZywgZGF0YSkge2cuc2VsZWN0KCcueC5n
cmlkJykuYXR0cigndHJhbnNmb3JtJywgJ3RyYW5zbGF0ZSgwLCcgKyB5U2NhbGUwLnJhbmdlKClb
MF0gKyAnKScpLmNhbGwoeEdyaWQudGlja1ZhbHVlcyhnZXRYVGlja3MoZGF0YSkpLnRpY2tTaXpl
KC1pbm5lckgoKSwgMCwgMCkudGlja1NpemVPdXRlcigwKS50aWNrRm9ybWF0KCcnKSk7Zy5zZWxl
Y3QoJy55LmdyaWQnKS5jYWxsKHlHcmlkLnRpY2tWYWx1ZXMoZ2V0WVRpY2tzKHlTY2FsZTApKS50
aWNrU2l6ZSgtaW5uZXJXKCksIDApLnRpY2tTaXplT3V0ZXIoMCkudGlja0Zvcm1hdCgnJykpO31m
dW5jdGlvbiBmb3JtYXRUb29sdGlwKGRhdGEpIHt2YXIgZCA9IGRhdGEuc2xpY2UoMCk7ZFswXSA9
IChmb3JtYXQueCkgPyBHb0FjY2Vzcy5VdGlsLmZtdFZhbHVlKGRbMF0sIGZvcm1hdC54KSA6IGRb
MF07ZFsxXSA9IChmb3JtYXQueTApID8gR29BY2Nlc3MuVXRpbC5mbXRWYWx1ZShkWzFdLCBmb3Jt
YXQueTApIDogZDMuZm9ybWF0KCcsJykoZFsxXSk7ZHVhbFlheGlzICYmIChkWzJdID0gKGZvcm1h
dC55MSkgPyBHb0FjY2Vzcy5VdGlsLmZtdFZhbHVlKGRbMl0sIGZvcm1hdC55MSkgOiBkMy5mb3Jt
YXQoJywnKShkWzJdKSk7dmFyIHRlbXBsYXRlID0gZDMuc2VsZWN0KCcjdHBsLWNoYXJ0LXRvb2x0
aXAnKS5odG1sKCk7cmV0dXJuIEhvZ2FuLmNvbXBpbGUodGVtcGxhdGUpLnJlbmRlcih7J2RhdGEn
OiBkfSk7fWZ1bmN0aW9uIG1vdXNlb3ZlcihldmVudCwgc2VsZWN0aW9uLCBkYXRhKSB7dmFyIHRv
b2x0aXAgPSBzZWxlY3Rpb24uc2VsZWN0KCcuY2hhcnQtdG9vbHRpcC13cmFwJyk7dG9vbHRpcC5o
dG1sKGZvcm1hdFRvb2x0aXAoZGF0YSkpLnN0eWxlKCdsZWZ0JywgWChkYXRhKSArICdweCcpLnN0
eWxlKCd0b3AnLCAgKGQzLnBvaW50ZXIoZXZlbnQpWzFdICsgMTApICsgJ3B4Jykuc3R5bGUoJ2Rp
c3BsYXknLCAnYmxvY2snKTtzZWxlY3Rpb24uc2VsZWN0KCdsaW5lLmluZGljYXRvcicpLnN0eWxl
KCdkaXNwbGF5JywgJ2Jsb2NrJykuYXR0cigndHJhbnNmb3JtJywgJ3RyYW5zbGF0ZSgnICsgWChk
YXRhKSArICcsJyArIDAgKyAnKScpO31mdW5jdGlvbiBtb3VzZW91dChzZWxlY3Rpb24sIGcpIHt2
YXIgdG9vbHRpcCA9IHNlbGVjdGlvbi5zZWxlY3QoJy5jaGFydC10b29sdGlwLXdyYXAnKTt0b29s
dGlwLnN0eWxlKCdkaXNwbGF5JywgJ25vbmUnKTtnLnNlbGVjdCgnbGluZS5pbmRpY2F0b3InKS5z
dHlsZSgnZGlzcGxheScsICdub25lJyk7fWZ1bmN0aW9uIGFkZFJlY3RzKHNlbGVjdGlvbiwgZywg
ZGF0YSkge3ZhciB3ID0gKGlubmVyVygpIC8gZGF0YS5sZW5ndGgpO3ZhciByZWN0cyA9IGcuc2Vs
ZWN0KCdnLnJlY3RzJykuc2VsZWN0QWxsKCdyZWN0JykuZGF0YShkYXRhKTt2YXIgcmVjdHNFbnRl
ciA9IHJlY3RzLmVudGVyKCkuYXBwZW5kKCdzdmc6cmVjdCcpLmF0dHIoJ2hlaWdodCcsIGlubmVy
SCgpKS5hdHRyKCdjbGFzcycsICdwb2ludCcpO3JlY3RzRW50ZXIubWVyZ2UocmVjdHMpLmF0dHIo
J3dpZHRoJywgdykuYXR0cigneCcsIGZ1bmN0aW9uKGQsIGkpIHtyZXR1cm4gKHcgKiBpKTt9KS5h
dHRyKCd5JywgMCkub24oJ21vdXNlbW92ZScsIGZ1bmN0aW9uKGV2ZW50KSB7bW91c2VvdmVyKGV2
ZW50LCBzZWxlY3Rpb24sIGQzLnNlbGVjdCh0aGlzKS5kYXR1bSgpKTt9KS5vbignbW91c2VsZWF2
ZScsIGZ1bmN0aW9uKGV2ZW50KSB7bW91c2VvdXQoc2VsZWN0aW9uLCBnKTt9KTtyZWN0cy5leGl0
KCkucmVtb3ZlKCk7fWZ1bmN0aW9uIGNoYXJ0KHNlbGVjdGlvbikge3NlbGVjdGlvbi5lYWNoKGZ1
bmN0aW9uIChkYXRhKSB7ZGF0YSA9IG1hcERhdGEoZGF0YSk7dXBkYXRlU2NhbGVzKGRhdGEpO2xl
dCBzdmcgPSBkMy5zZWxlY3QodGhpcykuc2VsZWN0KCdzdmcnKTtpZiAoc3ZnLmVtcHR5KCkpIHtz
dmcgPSBkMy5zZWxlY3QodGhpcykuYXBwZW5kKCdzdmcnKS5hdHRyKCd3aWR0aCcsIHdpZHRoKS5h
dHRyKCdoZWlnaHQnLCBoZWlnaHQpO2NyZWF0ZVNrZWxldG9uKHN2Zyk7fXZhciBnID0gc3ZnLnNl
bGVjdCgnZycpLmF0dHIoJ3RyYW5zZm9ybScsICd0cmFuc2xhdGUoJyArIG1hcmdpbi5sZWZ0ICsg
JywnICsgbWFyZ2luLnRvcCArICcpJyk7YWRkR3JpZChnLCBkYXRhKTthZGRBeGlzKGcsIGRhdGEp
O2FkZEJhcnMoZywgZGF0YSk7YWRkUmVjdHMoc2VsZWN0aW9uLCBnLCBkYXRhKTt9KTt9Y2hhcnQu
b3B0cyA9IGZ1bmN0aW9uIChfKSB7aWYgKCFhcmd1bWVudHMubGVuZ3RoKSByZXR1cm4gb3B0cztv
cHRzID0gXztyZXR1cm4gY2hhcnQ7fTtjaGFydC5mb3JtYXQgPSBmdW5jdGlvbiAoXykge2lmICgh
YXJndW1lbnRzLmxlbmd0aCkgcmV0dXJuIGZvcm1hdDtmb3JtYXQgPSBfO3JldHVybiBjaGFydDt9
O2NoYXJ0LmxhYmVscyA9IGZ1bmN0aW9uIChfKSB7aWYgKCFhcmd1bWVudHMubGVuZ3RoKSByZXR1
cm4gbGFiZWxzO2xhYmVscyA9IF87cmV0dXJuIGNoYXJ0O307Y2hhcnQud2lkdGggPSBmdW5jdGlv
biAoXykge2lmICghYXJndW1lbnRzLmxlbmd0aCkgcmV0dXJuIHdpZHRoO3dpZHRoID0gXztyZXR1
cm4gY2hhcnQ7fTtjaGFydC5oZWlnaHQgPSBmdW5jdGlvbiAoXykge2lmICghYXJndW1lbnRzLmxl
bmd0aCkgcmV0dXJuIGhlaWdodDtoZWlnaHQgPSBfO3JldHVybiBjaGFydDt9O2NoYXJ0LnggPSBm
dW5jdGlvbiAoXykge2lmICghYXJndW1lbnRzLmxlbmd0aCkgcmV0dXJuIHhWYWx1ZTt4VmFsdWUg
PSBfO3JldHVybiBjaGFydDt9O2NoYXJ0LnkwID0gZnVuY3Rpb24gKF8pIHtpZiAoIWFyZ3VtZW50
cy5sZW5ndGgpIHJldHVybiB5VmFsdWUwO3lWYWx1ZTAgPSBfO3JldHVybiBjaGFydDt9O2NoYXJ0
LnkxID0gZnVuY3Rpb24gKF8pIHtpZiAoIWFyZ3VtZW50cy5sZW5ndGgpIHJldHVybiB5VmFsdWUx
O3lWYWx1ZTEgPSBfO3JldHVybiBjaGFydDt9O3JldHVybiBjaGFydDt9PC9zY3JpcHQ+PC9ib2R5
PjwvaHRtbD4=
--000000000000c7064e0611b92fa6
Content-Type: text/html; charset="US-ASCII"; name="report.html"
Content-Disposition: attachment; filename="report.html"
Content-Transfer-Encoding: base64
Content-ID: <f_lssroi4w1>
X-Attachment-Id: f_lssroi4w1

PCFET0NUWVBFIGh0bWw+PGh0bWwgbGFuZz0nZW4nPjxoZWFkPjxtZXRhIGNoYXJzZXQ9J1VURi04
Jz48bWV0YSBuYW1lPSdyZWZlcnJlcicgY29udGVudD0nbm8tcmVmZXJyZXInPjxtZXRhIGh0dHAt
ZXF1aXY9J1gtVUEtQ29tcGF0aWJsZScgY29udGVudD0nSUU9ZWRnZSc+PG1ldGEgbmFtZT0nZ29v
Z2xlJyBjb250ZW50PSdub3RyYW5zbGF0ZSc+PG1ldGEgbmFtZT0ndmlld3BvcnQnIGNvbnRlbnQ9
J3dpZHRoPWRldmljZS13aWR0aCwgaW5pdGlhbC1zY2FsZT0xJz48bWV0YSBuYW1lPSdyb2JvdHMn
IGNvbnRlbnQ9J25vaW5kZXgsIG5vZm9sbG93Jz48bGluayByZWw9J2ljb24nIGhyZWY9J2RhdGE6
aW1hZ2UveC1pY29uO2Jhc2U2NCxBQUFCQUFFQUVCQVFBQUVBQkFBb0FRQUFGZ0FBQUNnQUFBQVFB
QUFBSUFBQUFBRUFCQUFBQUFBQWdBQUFBQUFBQUFBQUFBQUFFQUFBQUFBQUFBREd4c1lBV0ZoWUFC
d2NIQUJmQVA4QS85ZGZBQURYcndBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFJaUlpSWlJaUlpSWpNbFVrUWdBaUlpSWlJaUlpSWlJaUl6SlZKRUlB
QUFJaUlpSWlJaUlpSWlNeVZTUkNBQUlpSWlJaUlpSWlJaUlSRVJFUkVSRVJFUkVSRVJFUkVSRVJJ
aUlpSWlJaUlpSWdBQ1ZWVWlJaUlpSWlJaUlpSWlJaUlBQWxWVklpSWlJaUlpSWlJaUlpSWhFUkVS
RVJFUkVSRVJFUkVSRVJFUkVBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBJyB0eXBlPSdpbWFn
ZS94LWljb24nIC8+PHRpdGxlPlNlcnZlciZuYnNwO1N0YXRpc3RpY3M8L3RpdGxlPjxzdHlsZT5A
Zm9udC1mYWNlIHtmb250LWZhbWlseTogJ2ZhJztzcmM6ICB1cmwoZGF0YTphcHBsaWNhdGlvbi9m
b250LXdvZmY7Y2hhcnNldD11dGYtODtiYXNlNjQsZDA5R1JnQUJBQUFBQUZ4UUFBc0FBQUFBWEFR
QUFRQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUJQVXk4eUFBQUJDQUFBQUdBQUFBQmdEeElQ
YTJOdFlYQUFBQUZvQUFBQ1ZBQUFBbFE5OUFHWFoyRnpjQUFBQTd3QUFBQUlBQUFBQ0FBQUFCQm5i
SGxtQUFBRHhBQUFWQlFBQUZRVWZSVExJMmhsWVdRQUFGZllBQUFBTmdBQUFEWWZpTGVkYUdobFlR
QUFXQkFBQUFBa0FBQUFKQWhVQk1Cb2JYUjRBQUJZTkFBQUFiUUFBQUcwYjlvRGtteHZZMkVBQUZu
b0FBQUEzQUFBQU54THhtR3FiV0Y0Y0FBQVdzUUFBQUFnQUFBQUlBQjRBVmR1WVcxbEFBQmE1QUFB
QVVvQUFBRktJaFdUc25CdmMzUUFBRnd3QUFBQUlBQUFBQ0FBQXdBQUFBTURid0dRQUFVQUFBS1pB
c3dBQUFDUEFwa0N6QUFBQWVzQU13RUpBQUFBQUFBQUFBQUFBQUFBQUFBQUFSQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQVFBQUE4dElEd1AvQUFFQUR3QUJBQUFBQUFRQUFBQUFBQUFBQUFBQUFJQUFBQUFB
QUF3QUFBQU1BQUFBY0FBRUFBd0FBQUJ3QUF3QUJBQUFBSEFBRUFqZ0FBQUNLQUlBQUJnQUtBQUVB
SU9vTDhBZndEZkFSOEJmd0dmQWU4Q0x3TFBBNjhFSHdSUEJHOEUzd1ZQQmE4R2Z3YXZCeDhIandn
UENGOEk3d2x2Q2o4TER3c3ZEQjhNbnd6dkRhOE43dzVQRHo4UGJ4QWZFRjhRanhDdkVNOFJIeElm
RW04U2p4TXZGQzhVVHhUUEZjOFdYeG9QSEo4Yzd4Mi9IZThmZngvdklBOGdYeWFQS004cEx5blBL
Mzh0TC8vZi8vQUFBQUFBQWc2Z3Z3QXZBSjhCSHdFL0FaOEIzd0lmQXI4RHJ3UVBCRThFYndTL0JU
OEZud1ovQnA4SEh3ZC9DQThJWHdqdkNXOEtMd3NQQ3k4TUR3eWZETzhOZnczUERrOFBQdzl2RUE4
UVR4Q1BFSzhRenhFUEVnOFNieEtQRXk4VUh4UlBGTThWenhaUEdnOGNueHp2SGI4ZDd4OXZIKzhn
RHlCUEpvOG92eWt2S2M4cmJ5MHYvOS8vOEFBZi9qRmZrUUF4QUNELzhQL2cvOUQvb1ArQS93RCtN
UDNnL2NEOXNQMXcvU0Q4NFB3Zy9CRDdzUHRnK3ZENnNQb3crY0Q1RVBoUStFRDNjUGNBOXNEMlFQ
WXc5ZUQxQVBUZzlGRDBNUFFROUFEejhQUEE4dUR5b1BLUThnRHhJUEVROEtEdnNPOUE2NkRwSU9q
ZzZDRG9BT2FRNWpEbUlPWHczOURkc04xZzNORGJRTm1nQURBQUVBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFRQUIv
LzhBRHdBQkFBQUFBQUFBQUFBQUFnQUFOemtCQUFBQUFBRUFBQUFBQUFBQUFBQUNBQUEzT1FFQUFB
QUFBUUFBQUFBQUFBQUFBQUlBQURjNUFRQUFBQUFCQUFBQlFBUUFBa0FBRHdBQUV4VVVGak1oTWpZ
OUFUUW1JeUVpQmdBVERRUEFEUk1URGZ4QURSTUNJTUFORXhNTndBMFRFd0FBQUFJQUFQKzNBN2NE
YmdBYkFFSUFBQUUwSnk0Qkp5WWpJZ2NPQVFjR0ZSUVhIZ0VYRmpNeU56NEJOellCRkFZaklpWXZB
UTRCSXlJbkxnRW5KalUwTno0Qk56WXpNaGNlQVJjV0ZSUUdCeGNlQVJVQ2toUVVSaTR2TlRVdUww
VVZGQlFWUlM4dU5UVXZMa1lVRkFFbEt4NFBHd3JFTW5VOVUwbEtiUjhnSUI5dFNrbFRWRWxKYlNB
Z0pTTEVDZ3NCMnpVdkwwVVVGQlFVUlM4dk5UVXVMMFVVRlJVVVJTOHUvbG9lS3dzTHd5TWtJQjl1
U1VsVFZFbEpiaDhnSUI5dVNVbFVQSFV6eEFrYkR3QUFBQU1BQUFBQUJBQURKUUFjQURjQVJ3QUFK
UkVPQVFjT0FRY09BU3NCSWlZbkxnRW5MZ0VuRVJRV015RXlOalVSTkNZaklTSUdGUlFXRng0QkZ4
NEJPd0V5TmpjK0FUYytBVFUzRVJRR0l5RWlKalVSTkRZeklUSVdBN2NKRkFzOWVqd2dUeXdDTEU4
Z1BIbzlDeFFKQ3djRFNnY0xBaEQ4dGdjTExpWTZjamtYU0I0Q0hrZ1hPWEk2SERoSk5pWDh0aVUy
TmlVRFNpVTJXd0czQ2hNSU1HQXlHelUxR3pKZ01BZ1RDdjVKQndzTEJ3SlpDeHdMQnpGVEhpMWJM
Uk02T2hNdFd5MFdVeVFWL1pJbE5qWWxBbTRtTmpZQUFRQUFBQUFFQUFOdUFDb0FBQ1VpSmljQkpp
Y3VBU2NtTlRRM1BnRTNOak15RmhjK0FUTXlGeDRCRnhZVkZBY09BUWNHQndFT0FTTUNBQWNOQmY2
YkFSVVZNQk1VRXhKSE1qTkFTNElpSW9KTFFETXlSeElURkJNeEZCVUMvcHdGRFFjQUJRVUJXQUlW
RlVVdUxqTS9NakZGRWhOYUlpSmFFeEpGTVRJL015NHVSaFVWQXY2cEJRVUFBQUFCQUFBQUdnTzNB
NlVBTGdBQUFSUUdEd0VUSEFFVkZBWWpJaVluSlFVT0FTTWlKalUwTmpVVEp5NEJOVFEyTnlVVFBn
RXpNaFlYRXdVZUFSVUR0d2tHenpFTERBWU1CZjcvL3dBR0N3WU1EQUV4MEFVSkZRc0JINEFFRHdr
S0R3T0JBUjhLRmdKRkNBNEZ5LzdqQXdZREN4RUVBNGFHQXdRUkN3TUdBd0VkeXdVT0NBME1BU29C
QkFnUUVBaisvQ29CREEwQUFnQUFBQm9EdHdPbEFBa0FPQUFBQVRjdkFROEJGd2MzRndFVUJnOEJF
eHdCRlJRR0l5SW1KeVVGRGdFaklpWTFORFkxRXljdUFUVTBOamNsRXo0Qk16SVdGeE1GSGdFVkFv
cXY4bXhzOGE4cTJOZ0JCQWtHenpFTERBWU1CZjcvL3dBR0N3WU1EQUV4MEFVSkZRc0JINEFFRHdr
S0R3T0JBUjhLRmdGNXFpUGIyeU9xOEhGeEFid0lEZ1hML3VNREJnTUxFUVFEaG9ZREJCRUxBd1lE
QVIzTEJRNElEUXdCS2dFRUNCQVFDUDc4S2dFTURRQUFBQUlBQUFBQUF0c0RiZ0FiQURjQUFDVVVC
aU1oSWlZMU5EYytBVGMyTXg0Qk16STJOeklYSGdFWEZoVURGQWNPQVFjR0l5SW5MZ0VuSmpVME56
NEJOell6TWhjZUFSY1dBdHRITXY0WU1rZ0lDQ29sSkRnald6VTBYQ000SkNRcUNBaVNFUkU4S0Nn
dExpZ29PeEVTRWhFN0tDZ3VMU2dvUEJFUmx6NVpXVDQ1T1RwZEhoMGhLQ2doSFI1ZE9qazVBZnN0
S0NnOEVSRVJFVHdvS0MwdUtDZzdFUklTRVRzb0tBQUFBQUFFQUFBQVNRTzNBMjRBRUFBaEFERUFR
UUFBQVJVVUJpTWhJaVk5QVRRMk15RXlGaFVSRlJRR0l5RWlKajBCTkRZeklUSVdGUUVWRkFZaklT
SW1QUUUwTmpNaE1oWVJGUlFHSXlFaUpqMEJORFl6SVRJV0FiY3JIdjdiSGlzckhnRWxIaXNySHY3
Ykhpc3JIZ0VsSGlzQ0FDc2UvdHNlS3lzZUFTVWVLeXNlL3RzZUt5c2VBU1VlS3dGdTNCNHJLeDdj
SGlzckhnRzMzQjRyS3g3Y0hpc3JIdjVKM0I0ckt4N2NIaXNyQVpuY0hpc3JIdHdlS3lzQUNRQUFB
RWtFQUFOdUFBOEFId0F2QUQ4QVR3QmZBRzhBZndDUEFBQWxGUlFHS3dFaUpqMEJORFk3QVRJV0VS
VVVCaXNCSWlZOUFUUTJPd0V5RmdFVkZBWXJBU0ltUFFFME5qc0JNaFlCRlJRR0t3RWlKajBCTkRZ
N0FUSVdBUlVVQmlzQklpWTlBVFEyT3dFeUZnRVZGQVlyQVNJbVBRRTBOanNCTWhZQkZSUUdLd0Vp
SmowQk5EWTdBVElXQVJVVUJpc0JJaVk5QVRRMk93RXlGaEVWRkFZckFTSW1QUUUwTmpzQk1oWUJK
U0VXdHhjZ0lCZTNGaUVoRnJjWElDQVh0eFloQVcwZ0Y3WVhJQ0FYdGhjZy9wTWhGcmNYSUNBWHR4
WWhBVzBnRjdZWElDQVh0aGNnQVc0Z0Y3Y1dJU0VXdHhjZy9wSWdGN1lYSUNBWHRoY2dBVzRnRjdj
V0lTRVd0eGNnSUJlM0ZpRWhGcmNYSU81dUZ5QWdGMjRXSVNFQkRtMFhJQ0FYYlJjZ0lQN0ZiaGNn
SUJkdUZpRWhBak51RnlBZ0YyNFhJQ0QreEcwWElDQVhiUmNnSVA3RmJoY2dJQmR1RmlFaEFqTnVG
eUFnRjI0WElDRCt4RzBYSUNBWGJSY2dJQUVPYmhjZ0lCZHVGeUFnQUFZQUFBQkpCQUFEYmdBUEFC
OEFMd0EvQUU4QVh3QUFKUlVVQmlzQklpWTlBVFEyT3dFeUZoRVZGQVlyQVNJbVBRRTBOanNCTWhZ
QkZSUUdJeUVpSmowQk5EWXpJVElXQVJVVUJpc0JJaVk5QVRRMk93RXlGZ0VWRkFZaklTSW1QUUUw
TmpNaE1oWVJGUlFHSXlFaUpqMEJORFl6SVRJV0FTVWhGcmNYSUNBWHR4WWhJUmEzRnlBZ0Y3Y1dJ
UUxiSUJmOTNCY2dJQmNDSkJjZy9TVWhGcmNYSUNBWHR4WWhBdHNnRi8zY0Z5QWdGd0lrRnlBZ0Yv
M2NGeUFnRndJa0Z5RHViaGNnSUJkdUZpRWhBUTV0RnlBZ0YyMFhJQ0QreFc0WElDQVhiaFloSVFJ
emJoY2dJQmR1RnlBZy9zUnRGeUFnRjIwWElDQUJEbTRYSUNBWGJoY2dJQUFBQVFCRkFGRUR1d0w0
QUNRQUFBRVVCZ2NCRGdFaklpWW5BUzRCTlRRMlB3RStBVE15Rmg4QkFUNEJNeklXSHdFZUFSVUR1
d2dJL2hRSEZRb0xGUWYrNHdnSUNBaE9DQlFMQ2hVSXFBRjJDQlVLQ3hRSVRnZ0lBbk1LRlFmK0ZB
Z0lDQWdCSFFjVkN3b1ZCMDRJQ0FnSXFBRjNDQWdJQ0U0SEZRc0FBQUVBUHdBL0F1WUM1Z0E4QUFB
bEZBWVBBUTRCSXlJbUx3RUhEZ0VqSWlZdkFTNEJOVFEyUHdFbkxnRTFORFkvQVQ0Qk16SVdId0Uz
UGdFek1oWWZBUjRCRlJRR0R3RVhIZ0VWQXVZSkIwNElGQXNMRkFpb3FBY1ZDd29WQjA0SUNBZ0lx
S2dJQ0FnSVRnY1ZDZ3NWQjZpb0NCUUxDeFFJVGdjSkNRZW9xQWNKd3dvVkIwNElDQWdJcUtnSUNB
Z0lUZ2NWQ2dzVkI2aW9DQlFMQ3hRSVRnY0pDUWVvcUFjSkNRZE9DQlFMQ3hRSXFLZ0hGUXNBQUFB
Q0FBQUFBQU51QTdjQVBBQktBQUFCRkFjT0FRY0dJeUluTGdFbkpqVTBOejRCTnpZM05oWVhGZ1lI
RGdFVkZCY2VBUmNXTXpJM1BnRTNOalUwSmljdUFUYytBUmNXRng0QkZ4WVZBUkVVQmlNaUpqVVJO
RFl6TWhZRGJpTWllRkJRV2x0UVVIY2lJd3dMTFNFaEtSazhFaElKR0RjK0Z4ZFFOVFU5UERZMVR4
Y1hQVGdZQ0JJU1BCZ3FJU0F0REF6K2tpc2VIaXNySGg0ckFiZGJVRkIzSWlNakluZFFVRnMwTVRK
YUp5Y2ZFd2tZR0R3U0tudEZQVFUxVUJjWEZ4ZFFOVFU5UlhzcUVqd1lHQWtUSHljbldqSXhOQUcz
L3BJZUt5c2VBVzRlS3lzQUFBQUNBQUFBQUFOdUEyNEFDd0NTQUFBQk5DWWpJZ1lWRkJZek1qWWxG
UlFHRHdFT0FRY2VBUmNlQVJVVUJnY09BU01pSmk4QkRnRUhEZ0VIRGdFckFTSW1Md0V1QVNjSERn
RWpJaVluTGdFbkxnRTFORFkzUGdFM0xnRXZBUzRCUFFFME5qOEJQZ0UzTGdFbkxnRTFORFkzUGdF
ek1oWWZBVDRCTno0Qk56NEJPd0V5Rmg4QkhnRVhOejRCTXpJV0Z4NEJGeDRCRlJRR0J3NEJCeDRC
SHdFZUFSVUNTVlk4UFZWVlBUeFdBU1VKQjJvRkNnY09IeEFDQkFNRENsVVBCQWNFVHd3YURnTUdC
d0lMQ0g4SERBRVFEUm9OVUFNSUF3UUlBeFkyRWdJQ0FnTU9IdzhJREFSb0NBa0pCbXNFQ3djUEho
QURBd01DQzFVUEF3Z0RUdzBhRFFNSEJ3SUxCMzhJREFFUURSb01VUU1IQkFRSEF4YzJFZ0lDQXdJ
T0h3OEhEQVZvQndvQnR6eFdWanc5VlZWN2Z3WU5BUkFPR2d3Vkp4TURDQU1FQndNTldRTUNQZ1lM
QlJvMkdnY0pDZ2RwQlFvR1BRSURBd01WTXhnREJ3UURCd01USnhRT0hBOFBBUXdJZmdjTkFSQU9H
ZzBVSnhNREJ3UUVCZ01PV1FRQ1BRWUxCQnMyR2djSkNnZHFCQW9IUFFNREJBSVZNeGtEQmdRRUJn
TVVKaFFPSEE0UUFnd0hBQUFBQUFZQUFBQUFBeVVEYmdBUEFCOEFMd0E2QUVRQWFRQUFBUkVVQmlz
QklpWTFFVFEyT3dFeUZoY1JGQVlyQVNJbU5SRTBOanNCTWhZWEVSUUdLd0VpSmpVUk5EWTdBVElX
RXhFaEVSUVdNeUV5TmpVQklTY3VBU2NqRGdFSEJSVVVCaXNCRVJRR0l5RWlKalVSSXlJbVBRRTBO
anNCTno0Qk93RXlGaDhCTXpJV0ZRRWxDd2drQ0FzTENDUUlDNUlMQnlVSUNnb0lKUWNMa2dvSUpR
Y0xDd2NsQ0FwSi9nQVFBd0hiQWhEK2dBRUFHd0VIQXJVREJRSUI5d3NJTnpVbS9pVW1OamNIQ3dz
SHNTZ0lMUmUzRmkwSktMQUlDd0lTL3JjSUNnb0lBVWtJQ3dzSS9yY0lDZ29JQVVrSUN3c0kvcmNJ
Q2dvSUFVa0lDd3YrV3dJZC9lTVZGeGNWQW1aREFnUUJBUVFDVlNRSUMvM2pNRVZETHdJZ0N3Z2tD
QXBnRlI0ZUZXQUtDQUFBQWdBVEFFa0RwQU1sQUJVQVBBQUFBUkVVQmlzQk5TTVZJeUltTlJFME5q
RUpBVEFXRlRjSERnRXJBU0ltSndrQkRnRW5JaVl2QVNZMk53RTJNaDhCTlRRMk93RXlGaDBCRng0
QkJ3TWxGZy9iazlzUEZnRUJTQUZKQVg4akF3WURBZ1FHQXY1MC9uVURCd1FEQndJakJRSUZBWnNT
TXhLTEN3aHRDQXQ5QlFJRkFZRCs3ZzhXM053V0R3RVNBUUlCRC83eEFnRW5LZ0lFQWdJQlN2NjJB
Z01CQkFJcUJnOEZBVllQRDNSdkNBc0xDT2xvQlE4R0FBTUFBUCszQTI0RHR3QVRBQndBSmdBQUFS
NEJGUkVVQmlNaElpWTFFVFEyTXlFeUZoY0hGVE11QVM4QkxnRVRFU01pSmowQklSRWhBMGNRRnlB
WC9RQVhJQ0FYQWdBWE54Qk0xd01IQTdJRER0WHVGeUQrU1FMY0F0NFFOeGY5YmhjZ0lCY0RraGNn
RnhBbjF3Z05BN01EQi95WkFra2dGKzc4a2dBREFBQUFBQU51QTI0QUZRQXhBRTBBQUFFUkZBWXJB
U0ltUFFFME5qc0JOVFEyT3dFeUZoVVhOQ2N1QVNjbUl5SUhEZ0VIQmhVVUZ4NEJGeFl6TWpjK0FU
YzJOeFFIRGdFSEJpTWlKeTRCSnlZMU5EYytBVGMyTXpJWEhnRVhGZ0lBQ3dlM0NBb0tDSUFLQ0NV
SEMrNFpHRlU0T1VCQU9UbFVHUmdZR1ZRNU9VQkFPVGhWR0JtQUl5SjNVRkJiVzFCUWR5TWlJaU4z
VUZCYlcxQlFkeUlqQW9EL0FBZ0tDZ2dsQnd2SkNBb0tDTWxBT1RoVkdCa1pHRlU0T1VCQU9UbFVH
UmdZR1ZRNU9VQmJVRkIzSWlNakluZFFVRnRiVUZCM0lpTWpJbmRRVUFBQUFBQUVBQUFBU1FPM0E3
Y0FDd0FYQURFQVVRQUFKVFFtSXlJR0ZSUVdNekkyTnpRbUl5SUdGUlFXTXpJMk54VVVCaU1oSWlZ
OUFUUTJNeUVYSGdFek1qWS9BU0V5RmhVREZnWUhBUTRCSXlJbUp3RXVBVGMrQVRzQkVUUTJPd0V5
RmhVUk16SVdGd0xiRlE4UEZoWVBEeFdURmc4UEZSVVBEeFpKSUJmOHR4Y2dJQmNCQ2swUUtCVVdL
QkJPQVFrWElMb0VCQWovQUFVT0J3WU9CZjhBQ0FVRkJCSUxreFVQa3c4Vmtnd1NCTGNQRlJVUER4
WVdEdzhWRlE4UEZoYVB0eGNnSUJlM0Z5Qk9EeEVSRDA0Z0Z3RkZDaFlJL3dBR0JRVUdBUUFJRmdv
S0RBRUFEeFlXRC84QURBb0FBQU1BQUFBQUEyNERiZ0FZQURRQVVBQUFBUlFHQndVT0FTTWlKaWN1
QVRVUk5EWTNOaklYQlI0QkZUTTBKeTRCSnlZaklnY09BUWNHRlJRWEhnRVhGak15Tno0Qk56WTNG
QWNPQVFjR0l5SW5MZ0VuSmpVME56NEJOell6TWhjZUFSY1dBcVVLQ2Y3SkJBa0ZCUWtFQ1FrSkNR
Z1VDQUUzQ1FwSkdSaFZPRGxBUURrNVZCa1lHQmxVT1RsQVFEazRWUmdaZ0NNaWQxQlFXMXRRVUhj
aklpSWpkMUJRVzF0UVVIY2lJd0czQ2hFRnRnTURBd0lGRVFvQmJnb1JCUVFGdHdRUkNrQTVPRlVZ
R1JrWVZUZzVRRUE1T1ZRWkdCZ1pWRGs1UUZ0UVVIY2lJeU1pZDFCUVcxdFFVSGNpSXlNaWQxQlFB
QUFCQUFBQUFBTnVBMjRBVFFBQUFSRVVCaU1oSWlZbkpqWS9BUzRCSXlJSERnRUhCaFVVRng0QkZ4
WXpNalkzUGdFM01oWWZBUllVQndZSERnRUhCaU1pSnk0Qkp5WTFORGMrQVRjMk16SVhIZ0VYRmhj
M1BnRVhIZ0VWQTI0V0QvOEFDeElGQkFRSVR5aG5PRDAxTlZBWEZ4Y1hVRFUxUFVSNUtnSUhCQVFI
QTA0R0JDQW1KbGN2THpGYlVGQjNJaU1qSW5kUVVGc3FLU2xPSXlNZVNnZ1dDZ29OQXlYL0FBOFdE
UW9LRmdkUEpTa1hGMDgxTmp3OU5UVlFGeGM4TmdNREFRTUNUd1VPQmlVZUhTa0tDeU1pZDFCUVcx
dFBVSGdpSXdrSUh4WVhIRWtJQlFVRUVnc0FBZ0FBQUFBRGJnTnVBRFFBWndBQUFUQVVGUVlIRGdF
SEJpTWlKeTRCSnlZbkJ3NEJJeUltTlJFME5qTWhNaFlWRkFZUEFSNEJNekkyTno0Qk56NEJPd0V5
RmhVVEVSUUdJeUVpSmpVME5qOEJMZ0VqSWdZSERnRUhEZ0VyQVNJbVBRRTJOejRCTnpZek1oY2VB
UmNXRnpjK0FUTXlGaFVEWHhJb0oyOUdSVThxS1NsTUl5UWVTZ1VOQnc4V0ZnOEJBQThWQmdWT0tH
ZzNUSVVvQ2cwSEFna0diZ2dLRHhZUC93QVBGUVVGVHlob04weUZLQXNNQ0FJSUIzRUlDeE1uS0hC
R1JrOHFLU2xOSXlRZVNnWU5CdzhXQVZzREFVdEFQMXNaR1FnSUh4Y1dIVW9GQlJVUEFRQVBGaFlQ
QncwR1RpWXBTMEVRSVJJR0J3c0lBY3IvQUE4V0ZnOEhEUVZQSmloS1FSRWhFUVlIQ3djRVRFQS9X
aG9aQ0FrZkZoY2NTUVVHRlE4QUFBQUlBQUFBU1FRQUEyNEFFQUFnQURBQVFRQlNBR01BZEFDRUFB
QVRGUlFHS3dFaUpqMEJORFk3QVRJV0ZUVVZGQVlyQVNJbVBRRTBOanNCTWhZMUZSUUdLd0VpSmow
Qk5EWTdBVElXQVJVVUJpTWhJaVk5QVRRMk15RXlGaFUxRlJRR0l5RWlKajBCTkRZeklUSVdGVFVW
RkFZaklTSW1QUUUwTmpNaE1oWVZFeEUwSmlNaElnWVZFUlFXTXlFeU5qVVRFUlFHSXlFaUpqVVJO
RFl6SVRJVzJ3c0hKQWdMQ3dna0J3c0xCeVFJQ3dzSUpBY0xDd2NrQ0FzTENDUUhDd0tUQ3dqOTNB
Y0xDd2NDSkFnTEN3ajkzQWNMQ3djQ0pBZ0xDd2o5M0FjTEN3Y0NKQWdMU1FzSC9MWUhDd3NIQTBv
SEMwazJKZnkySlRZMkpRTktKVFlCRWlRSUN3c0lKQWdMQ3dpVEpRY0xDd2NsQndzTGl5VUhDd3NI
SlFjTEMvN1VKQWdMQ3dna0NBc0xDSk1sQndzTEJ5VUhDd3NIa2lVSEN3c0hKUWNMQ3dmK2JnSGJC
d3NMQi80bENBc0xDQUp0L1pNbU5qWW1BbTBtTmpZQUFBQUFBZ0FBQUF3RFlnTnVBQXNBSmdBQUFU
UW1JeUlHRlJRV016STJBUlFHQndFT0FTTWlKaWNCTGdFOUFUUTJPd0V5RmhjQkhnRVZBUUFySGg4
cUtoOGVLd0ppREFuKzV3b2JEdzhiQ2Y1bkZoNHJIdTRlU1JZQm1Ra01BcmNlS3lzZUh5b3EvdFlQ
R3dyKzV3b0xDd29CbVJaSUgrNGVLeDRXL21nTEd3NEFBQUFEQUFBQURBUTlBMjRBQ3dBbUFFUUFB
QUUwSmlNaUJoVVVGak15TmdFVUJnY0JEZ0VqSWlZbkFTNEJQUUUwTmpzQk1oWVhBUjRCRlRNVUJn
Y0JEZ0VqSWlZbkFUNEJOVFFtSndFdUFTTXpNaFlYQVI0QkZRRUFLeDRmS2lvZkhpc0NZZ3dKL3Vj
S0d3OFBHd24rWnhZZUt4N3VIa2tXQVprSkROc0xDdjduQ2hzUEZoc1BBUTBLQ3dzSy9tY1ZTUitB
SDBrVkFaa0tDd0szSGlzckhoOHFLdjdXRHhzSy91Y0tDd3NLQVprV1NCL3VIaXNlRnY1b0N4c09E
eHNLL3VjS0N4TVBBUXdLR3c4T0d3c0JtQlllSGhiK2FBc2JEZ0FJQUFBQVNRUUFBMjRBRUFBaEFE
RUFRZ0JTQUdNQWN3Q0RBQUEzRlJRR0t3RWlKajBCTkRZN0FUSVdGVFVWRkFZckFTSW1QUUUwTmpz
Qk1oWVZOUlVVQmlzQklpWTlBVFEyT3dFeUZnRVZGQVlqSVNJbVBRRTBOak1oTWhZVkFSVVVCaXNC
SWlZOUFUUTJPd0V5RmdFVkZBWWpJU0ltUFFFME5qTWhNaFlWTlJVVUJpTWhJaVk5QVRRMk15RXlG
alVWRkFZaklTSW1QUUUwTmpNaE1oYVNDd2R1QndzTEIyNEhDd3NIYmdjTEN3ZHVCd3NMQjI0SEN3
c0hiZ2NMQTI0TEIvMEFDQXNMQ0FNQUJ3djhrZ3NIYmdjTEN3ZHVCd3NEYmdzSC9RQUlDd3NJQXdB
SEN3c0gvUUFJQ3dzSUF3QUhDd3NIL1FBSUN3c0lBd0FIQzhsdUJ3c0xCMjRIQ3dzSDNHNEhDd3NI
YmdjTEN3ZmJiZ2NMQ3dkdUJ3c0wva0p1QndzTEIyNEhDd3NIQXBKdENBc0xDRzBJQ3d2K1FtNEhD
d3NIYmdjTEN3ZmJiZ2NMQ3dkdUJ3c0wxRzBJQ3dzSWJRZ0xDd0FBQUFRQUFBQUFBMklEWWdBSEFC
d0FJUUF5QUFBL0FTY0hGVE1WTXdFMEppTWlCZ2NCRGdFVkZCWXpNalkzQVQ0Qk5TY1hBU00xQVJR
R0R3RW5OejRCTXpJV0h3RWVBUlhQTklZMFNUMEJLd2NGQXdVQy9zb0NBZ2NHQWdVREFUVUNBaC91
L2lYdUEySU1DVi91WHdvYkRnOGJDb2NKREVrMGhqUTlTUUlTQmdjQ0F2N0tBZ1VDQmdjQ0FnRTJB
Z1VDYnU3K0plNEJwQTRiQ2wvdVhnb01EQXFHQ2hzUEFBQUFBQUlBQUFBQUFra0RiZ0FMQUNnQUFB
RTBKaU1pQmhVVUZqTXlOamNVQmdjRERnRWpJaVluQXk0Qk5UUTNQZ0UzTmpNeUZ4NEJGeFlWQWJk
V1BEMVdWajA4VnBJSUM5QUpKQlFWSkFuUUN3Z1hGMDgyTlQwOE5qVlBGeGNDU1QxVlZUMDhWbFk4
R2pVWC9rWVRGaFlUQWJvWE5SbzlOVFZRRnhjWEYxQTFOVDBBQlFBQUFFa0Q2d051QUFjQUZRQktB
RThBV2dBQUFUY25CeFV6RlRNVEpnWVBBUVlVRnhZMlB3RStBUk1WRkFZaklTSW1OUkUwTmpNaE1o
WVhIZ0VYRmdZUEFRNEJKeTRCSXlFaUJoVVJGQll6SVRJMlBRRTBOajhCUGdFWEhnRVZBeGNCSXpV
QkJ5YzNOaklmQVJZVUJ3SDdRMWRDTmlEOEJBc0V5QVFEQkFzRXlBUUJLbUZFL2lWRllHQkZBZHNS
SWhBRUJRRUJBd01jQkFvRUJ3MEcvaVVtTmpZbUFkc21OUU1ESkFRTEJRVUhONlQrZ0tRQ2ZUU2xO
UkF1RUZZUUVBRVNRMVpDSURjQm5BUUJCTWdFQ3dRRUFRVElCQXYrc0d4RVlXRkVBZHRFWVFjSUFR
Y0ZCQWtESEFRREFnSUNOaWIrSlNZMk5pWklBd2NDSlFRQ0FnSUpCZ0dtcGY2QXBRRTFOYVUwRUJC
WEVDNFBBQUFBQUFJQUFBQkpBNmtEYmdBNkFGQUFBQUVWRkFZaklTSW1OUkUwTmpNaE1oWVhIZ0VY
RmdZUEFRNEJJeUltSXk0Qkl5RWlCaFVSRkJZeklUSTJQUUUwTmo4QlBnRXpNaFlYSGdFVkV3RUdJ
aThCSmpRL0FUWXlId0VCTmpJZkFSWVVCd01sWVVUK0pVVmdZRVVCMnhFaUVBUUZBUUVEQXh3REJ3
TUJBd0VIRFFiK0pTWTJOaVlCMnlZMUF3TWtBd2NEQWdNQ0JRZUUvaTROSmc3MURnNCtEaVlPbGdG
eURTWU9QdzBOQWFPMVJHRmhSQUhiUkdFSENBRUhCUVFKQXh3REF3RUNBalltL2lVbU5qWW1rUU1I
QWlVREF3RUJBZ2tHQVJqK0x3NE85UTRtRGo0T0RwWUJjZzRPUHc0bERnQUFBQUVBQVAvL0F4Y0Ri
Z0FMQUFBSkFRWW1OUkUwTmhjQkZoUURGLzBKRFJNVERRTDNEUUdsL2xvSEN3OERTUThNQ1A1YkNC
VUFBQUFBQWdBQUFBQURiZ051QUE4QUlBQUFBUkVVQmlNaElpWTFFVFEyTXlFeUZnVVJGQVlqSVNJ
bU5SRTBOak1oTWhZVkEyNFdELzdjRHhZV0R3RWtEeGIrQUJZUC90d1BGaFlQQVNRUEZnTkovTndQ
RmhZUEF5UVBGaFlQL053UEZoWVBBeVFQRmhZUEFBQUJBQUFBQUFOdUEyNEFEd0FBQVJFVUJpTWhJ
aVkxRVRRMk15RXlGZ051RmcvODNBOFdGZzhESkE4V0EwbjgzQThXRmc4REpBOFdGZ0FBQUFBQkFH
TUFHZ0tkQTUwQUZRQUFDUUlXRkE4QkJpSW5BU1kwTndFMk1oOEJGaFFIQXAzKzBRRXZDd3RmQ2g0
TC9sZ0xDd0dvQ3g0S1h3c0xBd3YrMFA3UkN4NEtYd3NMQWFnS0hnc0JxQXNMWHdvZUN3QUJBRDRB
R2dKNUE1MEFGUUFBQ1FFR0lpOEJKalEzQ1FFbU5EOEJOaklYQVJZVUJ3SjUvbGdMSGd0ZkNnb0JN
UDdRQ2dwZkN4NExBYWdLQ2dIQy9sZ0xDMThLSGdzQkx3RXdDeDRLWHdzTC9sZ0xIZ29BQUFBQUF3
QUFBQUFEYmdOdUFCQUFRd0JmQUFBbE5UUW1Ld0VpQmgwQkZCWTdBVEkyTlJNMEp5NEJKeVlqSWdZ
SEJoWWZBUjRCTXpJMk56NEJOejRCTXpJV0ZSUUdCdzRCSFFFVUZqc0JNalkxTVRRMk56NEJOUmNV
Qnc0QkJ3WWpJaWN1QVNjbU5UUTNQZ0UzTmpNeUZ4NEJGeFlDQUFzSGJnZ0tDZ2h1Qnd1U0VoTThK
U1VsUm1rbEJBTUdTd0lHQXdRSUFoVVVDUWNhRUJzckdCb2ZRUW9JYmdjTEZ4UWhSdHdqSW5kUVVG
dGJVRkIzSXlJaUkzZFFVRnRiVUZCM0lpT2xiUWdMQ3dodENBc0xDQUdBSnlFaU1RNE5QRDBHRGdR
NUFnSUVBeG9WQmdVSUhSTVhHZ3dPUVNvVkJ3c0xCd29qREJJOVFtNWJVRkIzSWlNakluZFFVRnRi
VUZCM0lpTWpJbmRRVUFBQUFBQURBQUFBQUFOdUEyNEFId0F2QUVzQUFDVTFOQ1lyQVJFMEppc0JJ
Z1lkQVJRV093RVZJeUlHSFFFVUZqTWhNalkxQXpVMEppc0JJZ1lkQVJRV093RXlOZ1VVQnc0QkJ3
WWpJaWN1QVNjbU5UUTNQZ0UzTmpNeUZ4NEJGeFlDU1FvSU53c0h0d2dLQ2dnM053Z0tDZ2dCQUFn
S1NRc0hiZ2dLQ2dodUJ3c0JiaU1pZDFCUVcxdFFVSGNqSWlJamQxQlFXMXRRVUhjaUk2VmJDQW9C
SlFnS0NnaGNCd3UzQ2doYkNBc0xDQUlBV3dnS0NnaGJDQXNMNWx0UVVIY2lJeU1pZDFCUVcxdFFV
SGNpSXlNaWQxQlFBQUFBQUFFQUFBQkpBeVVEYmdBa0FBQUJGUlFHS3dFVkZBWXJBU0ltUFFFaklp
WTlBVFEyT3dFMU5EWTdBVElXSFFFek1oWVZBeVVoRnU0Z0YyNFdJTzRYSUNBWDdpQVdiaGNnN2hZ
aEFoSnRGeUR1RnlBZ0YrNGdGMjBYSU80WElDQVg3aUFYQUFBQkFFMEFBQU5xQTI0QU5RQUFBUjRC
RHdFT0FTOEJGUlFHS3dFaUpqMEJCd1ltTHdFbU5qOEJKeTRCUHdFK0FSOEJOVFEyT3dFeUZoMEJO
ellXSHdFV0JnOEJBMDhhRUE4bER6c2FtQ3NlU1I0cm1CbzdEeVVQRUJxWW1Cb1FEeVVQT3hxWUt4
NUpIaXVZR2pzUEpROFFHcGdCWHc4N0dqOGFFQTlZc0I0ckt4NndXQThRR2o4YU93OVlXQTg3R2o4
YUVBOVlzQjRyS3g2d1dBOFFHajhhT3c5WUFBTUFBQUFBQTI0RGJnQWNBQ3dBUXdBQUFUSVhIZ0VY
RmhVVUJ3NEJCd1lqSWljdUFTY21OVFEzUGdFM05qTVROVFFtS3dFaUJoMEJGQlk3QVRJMkp4TTBK
aWN1QVNzQklnWUhEZ0VWRXhRV093RXlOamNCdDF0UVVIY2lJeU1pZDFCUVcxdFFVSGNqSWlJamQx
QlFXMGtLQ0cwSUN3c0liUWdLQVFvREF3SUhCSDRFQndNREFna01DR2tJQ3dFRGJpTWlkMUJRVzF0
UVVIY2lJeU1pZDFCUVcxdFFVSGNpSS8wM2JRY01EQWR0Q0FzTHpRRmpBd1VDQWdJQ0FnSUZBLzZk
QmdrSkJnQUFBd0FKQUFBRDl3TzNBQThBSmdBOEFBQWxOVFFtS3dFaUJoMEJGQlk3QVRJMkp4TTBK
aWN1QVNzQklnWUhEZ0VWRXhRV093RXlOalVEQVJZVUJ3NEJJeUVpSmljbU5EY0JQZ0V6TWhZWEFr
a0tDRzRJQ2dvSWJnZ0tBUW9DQXdNSEJINEVCd01EQWdrTUNHb0hEQWdCdHdrS0NpSVQvSklUSWdv
S0NRRzNDU01VRkNNSnBXMEhEQXdIYlFnTEM5NEJCZ01HQWdJRUJBSUNCd1ArK3dZSEJ3WUNGdnpi
RVNZUkVSTVRFUkVtRVFNbEVSVVZFUUFBQUFFQVBnQmdBOElDbWdBVkFBQWxCd1lpSndrQkJpSXZB
U1kwTndFMk1oY0JGaFFIQThKZkN4NEwvdEgrMFFzZUMxOEtDZ0dvQ3g0TEFhZ0tDcjVlQ3dzQkwv
N1JDd3RlQ3g4S0FhZ0tDdjVZQ2g4TEFBRUFQZ0JDQThJQ2V3QVZBQUFKQVFZaUp3RW1ORDhCTmpJ
WENRRTJNaDhCRmhRSEE4TCtXQXNlQy81WUNncGZDeDRMQVM4Qkx3c2VDMThLQ2dIcC9sa0xDd0du
Q3g0TFhnc0wvdEVCTHdzTFhnc2VDd0FBQUFBRkFBQUFBQVNTQTI0QUF3QUlBQTRBRXdBWUFBQUJF
U01SQVJFakVUTUJGU0VSTXhFQkVTTVJNemNSSXhFekFXNlRBVzZTa2dKSisyNUpBdHlUazl1U2tn
RzMvdHNCSlFFay9iY0NTZjF1U1FOdS9Oc0NBUDVKQWJmYy9XMENrd0FHQUFEL3Z3UkpBNjRBQ3dB
WEFDTUFwQUQ4QVZRQUFBRTBKaU1pQmhVVUZqTXlOZ1UwSmlNaUJoVVVGak15TmhFMEppTWlCaFVV
RmpNeU5nY1ZGQVlQQVE0QkJ4NEJGeDRCRlJRR0J3NEJJeUltTHdFT0FRY09BUWNPQVNzQklpWXZB
UzRCSndjT0FTTWlKaWN1QVRVME5qYytBVGN1QVM4QkxnRTlBVFEyUHdFK0FUY3VBU2N1QVRVME5q
YytBVE15Rmg4QlBnRTNQZ0UzUGdFN0FUSVdId0VlQVJjM1BnRXpNaFlYSGdFVkZBWUhEZ0VISGdF
ZkFSNEJGUUVWRkFZakRnRUhIZ0VWRkFZSERnRWpJaVluSWdZaklpWWpEZ0VqSWlZbkxnRTFORFkz
TGdFbklpWTlBVFEyTno0Qk55NEJOVEEyTno0Qk16SVdGejRCTXpJV0Z6NEJQd0V5RmhjZUFURVVC
Z2NlQVJjZUFSVVJGUlFHQnc0QkJ4NEJGUlFHQnc0Qkl5SW1KeUlHSXlJbUl3NEJJeUltSnk0Qk5U
UTJOeTRCSnk0QlBRRTBOamMrQVRjdUFUVTBOamMrQVRNeUZoY3lOak15RmpNK0FUOEJNaFlYSGdF
VkZBWUhIZ0VYSGdFVkFnQldQRDFXVmowOFZnRzNLeDRlS3lzZUhpc3JIaDRyS3g0ZUs5d0hCbGdF
Q1FZTUdnNENBZ0lDQ1VjTUJBWUNRZ3NWREFJRkJnSUpCbW9HQ3dFTkN4VUxRd0lHQXdRR0FneEhB
d0VOR1EwR0N3TlhCZ2dJQlZrRENRWU1HZzBDQWdFRENFY05Bd1lEUVFzV0N3SUdCUUlLQm1vR0Nn
RU5DeFlLUXdNR0F3TUdBd3RIQWdJTUdnd0dDZ1JYQmdjQmJrd0pBd2tGQkJrQkFRVkFBZ1l1QkFR
SkJBVUlCQVV1QlFNL0JRSUJHZ1FHQ0FRSVRVMElCQWdHQkJvQkFnVS9Bd1V1QlFRSUJRUUpCQXdh
RGdRQ1FBVUJBUmtFQlFrRENVeE1DUU1KQlFRWkFRRUZRQUlHTGdRRUNRUUZDQVFGTGdVRFB3VUNB
Um9FQmdnRUNFMU5DQVFJQmdRYUFRSUZQd01GTGdVRUNBVUVDUVFNR2c0RUFrQUZBUUVaQkFVSkF3
bE1BYmM4VmxZOFBWVlY2QjRyS3g0ZUt5c0NaeDRzTEI0ZUt5dlNhZ1VMQVE0TEZRc1JJUkFDQmdN
REJnSU1TUUlDTXdVSkJCVXVGUVlJQ0FaWUF3a0dNd0lDQWdJTFJBMERCUU1RSUJFTEdBd05BUW9H
YWdVTEFRME1GUXNSSUJFQ0JnTURCZ0lNU1FJQ013VUpCQlV1RlFZSUNRWlhCQWtGTXdJQ0FnTUtS
UXdEQlFNUkh4RU1Gd3dOQVFvRy9zOVFCZ3NKRGdjSlBnZ0JBZ0VESmp3R0FRRUdQQ1lEQVFJQkJ6
OEpCdzRKQ3daUUJ3b0JDQThIQ0Q4SUF3RUNKanNIQVFFQkFSRWhEZ0lsQXdFRENEOElCdzhJQVFv
SEFrbFFCZ29CQ0E4SENUNElBUUlCQXlVN0JnRUJCanNsQXdFQ0FRYy9DUWNQQ0FFS0JsQUhDZ0VJ
RHdjSVB3Z0JBZ0VDSmpzR0FRRVFJUThCSlFNQkFnRUlQd2dIRHdnQkNnY0FBQUlBQUFCSkJBQUR0
d0FvQUVzQUFBRVZGQVlqSVNJbU5SRTBOak1oTWhZZEFSUUdJeUVpQmhVUkZCWXpJVEkyUFFFME5q
c0JNaFlWRXhFVUJpTWlKaThCQVE0Qkl5SW1Md0V1QVRVME5qY0JKeTRCTlRRMk15RXlGaFVESldG
RS9pVkZZR0JGQVpJSUNnb0kvbTRtTmpZbUFkc21OUXNJSkFnTDJ4WVBCdzBGWmY2TUF3Y0VBd2NE
UVFJRUJBSUJkV1VGQmhZUEFTUVBGZ0dsdDBSaFlVUUIyMFJoQ3dna0NBbzJKdjRsSmpZMkpyY0hD
d3NIQWUzKzNBOFdCZ1ZsL29zREF3TURRUU1IQXdRSEFnRjFaUVVOQnc4V0ZnOEFBQUlBQUFCSkF5
VURiZ0FQQUI4QUFBRWhJZ1lWRVJRV015RXlOalVSTkNZWEVSUUdJeUVpSmpVUk5EWXpJVElXQW9E
K0pTWTJOaVlCMnlZMU5YOWhSUDRsUldCZ1JRSGJSR0VESlRZbS9pVW1OalltQWRzbU5seitKVVJo
WVVRQjIwUmhZUUFEQUNYL3R3UGJBN2NBRWdBd0FHY0FBQVUwSmlNaUpqVTBKaU1pQmhVVUZqTXlO
alVsSVNZbkxnRW5KalUwSnk0Qkp5WWpJZ2NPQVFjR0ZSUUhEZ0VIQmdjaEZBWWpJUlFHSXlJbU5T
RWlKalUyTno0Qk56WTFORGMrQVRjMk55NEJOVFEyTXpJV0ZSUUdCeFlYSGdFWEZoVVVGeDRCRnhZ
WEFna0ZCQ0l3QmdNRUJqc3FCQVgraEFMbUpod2RKZ2tLRFEwMktpazRPQ2txTmcwTkNna21IUndt
QTA0ckh2OEFWanc4VnY4QUhpc2ZJU0ExRVJBUkVUNHRMVGtDQXlBWEZ5QURBamt0TFQ0UkVSQVJO
U0FoSHhJRUJUQWlCQVVGQkNrN0JRU2tMRE16ZGtSRFRSc2dJRGNTRXhNU055QWdHMDFEUkhZek15
d2VLenhXVmp3ckhoc25KM0ZOVFdncEtTcEZHUmtKQlFzR0Z5QWdGd1lMQlFrWkdVVXFLU2xvVFUx
eEp5Y2JBQUVBQVFBQUEyMERiQUJpQUFBQkZ4NEJCdzRCRHdFWEZnWUhEZ0V2QVFjT0FRY2lCaU1p
Smk4QkJ3NEJKeTRCTHdFSEJpWW5MZ0UvQVNjdUFTY21OajhCSnk0Qk56NEJQd0VuSmpZM1BnRWZB
VGMrQVRjMkZoOEJOejRCRng0Qkh3RTNOaFlYSGdFUEFSY2VBUmNXQmdjREVrOElCZ0lERUF0c0h3
TUdDQWdWQzJvY0FoQUxBd1VEQ0E4R1RVMElGZ29MRUFJY2Fnc1ZDQWdHQXg1ckN4QURBZ1VKVGs0
SkJRSURFQXRySGdNR0NBZ1ZDMm9jQWhBTENoWUlUVTBJRlFzTEVBSWNhZ3NWQ0FnR0F4OXNDeEFE
QWdZSUFiZE5DQllLQ3hBQ0hHb0xGUWdJQmdNZWF3c1FBd0VIQms1T0NRVUNBeEFMYXg0REJnZ0lG
UXRxSEFJUUN3b1dDRTFOQ0JVTEN4QUNIR29MRlFnSUJnTWZiQXNQQXdNR0NFOVBDQVlEQXc4TGJC
OERCZ2dJRlF0cUhBSVFDd3NWQ0FBQUFBRUFBd0FBQXlJREpRQWVBQUFCRmdZSEFSRVVCZ2NPQVNN
aUppOEJMZ0UxRVFFdUFUYytBVE1oTWhZWEF5SUVCQWorNWd3S0JBY0VCdzBGa3dVRi91WUlCQVFF
RWd3QzJ3c1NCUU1PQ2hZSS91ZitXQXdTQkFFQ0JRYVNCUTRIQVJZQkdRZ1dDZ29ORFFvQUFBRUFB
QUFBQTI0RGJnQlpBQUFCQnhjM1BnRVhIZ0VWRVJRR0l5RWlKaWNtTmo4Qkp3Y1hIZ0VIRGdFaklT
SW1OUkUwTmpjMkZoOEJOeWNIRGdFaklpWW5MZ0UxRVRRMk15RXlGaGNXQmc4QkZ6Y25MZ0UzUGdF
eklUSVdGUkVVQmdjT0FTTWlKaWNDM2N2TFVnZ1dDZ29ORmcvL0FBc1NCUVFFQ0ZQTHkxSUlCUVVF
RWd2L0FBOFdEUW9LRmdkVHlzcFRCUTBIQkFjRENnMFdEd0VBQ3hJRUJRVUlVc3ZMVXdnRUJBVVND
d0VBRHhZTkNnTUhCQWNOQmdLQ3k4dFNDQVVGQkJJTC93QVBGZzBLQ2hZSFU4cktVd2NXQ2dvTkZn
OEJBQXNTQkFVRkNGTEx5MU1GQlFFQkJSSUxBUUFQRmcwS0NoWUlVc3ZMVWdnV0Nnb05GZy8vQUFz
U0JRRUJCUVVBQUFBR0FBRC90d1JKQTdjQUdnQTJBRUlBWGdCNEFJUUFBQUVPQVFjaklpWTFORGMr
QVRjMk16SVdNekkyTnc0QkZSUVdGd0VVQmlNaElpWTFORGMrQVRjMk16SVdNekkyTXpJWEhnRVhG
aFVCRkFZaklpWTFORFl6TWhZQkZBY09BUWNHSXlJbkxnRW5KalUwTno0Qk56WXpNaGNlQVJjV0JS
UUdLd0V1QVNjK0FUVTBKaWNlQVRNeU5qTXlGeDRCRnhZREZBWWpJaVkxTkRZek1oWUJVeTFPSFV3
clJBRUJEUTRQR3dsU09SUW1FZ0VCR0JZQ1pGUkYvZzFGVkFnSEt5WW5QdzlvVmxWcERqOG5KaXNJ
Qi8yM1ZqMDhWbFk4UFZZQmtoRVNPeWdvTFM0b0tEc1NFUkVTT3lnb0xpMG9LRHNTRVFGSlJDdE1I
VTR0RnhnQ0FSSW1GRHBSQ1J3T0RnMEJBVWxXUEQxV1ZqMDhWZ0czQVNZaUt6QVJJaUZER1Jrd0Jn
Y0tFZ29uU3lEK2xFWk9Ua1l3T2psaUlTRlBUeUVoWWprNk1BTGFQVlpXUFR4V1Z2N29MU2dvUEJF
UkVSRThLQ2d0TGlnb094RVNFaEU3S0NpdU1Dc2lKZ0VnU3ljS0Vnb0hCakFaR1VNaElnRkxQVlpX
UFR4V1ZnQUFBQUFEQUFrQUNRT3VBNjRBSXdCSUFJQUFBQUUwSmk4QkxnRWpJZ1lISGdFVkZBWWpJ
aVluRGdFVkZCWWZBUjRCTXpJMlB3RStBUUUwSmk4QkxnRWpJZ1lQQVE0QkZSUVdId0VlQVRNeU5q
Y3VBVFUwTmpNeUZoYytBVFVCRkFZUEFRNEJJeUltTHdFdUFUVTBOamNuRGdFaklpWXZBUzRCTlRR
MlB3RStBVE15Rmg4QkhnRVZGQVlIRno0Qk16SVdId0VlQVFOQUNBaDNDQlFMREJRSkRoc2dGeFVj
RHdrSkNBaDFDQlFMQ3hRSVZBZ0kvbTRJQ0hVSUZRb0xGQWhVQ0FnSUNIY0hGUXNNRkFrT0d5QVdG
aHdPQ1FvQ0FCb1hWQmM4SVNFOEYzWVhHUnNZTXhjK0lTRTlGM2NYR1JvWFZCYzhJU0U4RjNZWEdC
b1lNaGc5SWlFOUYzY1hHUUVBQ3hRSWR3Y0pDZ2tPSEJZV0lCc09DUlFOQ3hRSWRnZ0lDQWRVQnhR
Qm5nc1VDSFlJQ0FnSFZBZ1RDd3NVQ0hjSUJ3Z0pEeHdWRnlBYkRna1VEZjV0SVR3WFV4Z1lHUmgy
Rnp3aElqNFhNeGdiR1JkM0dEd2hJRDBYVXhjWkdSaDJGejBnSWo0WU1oZ2FHQmgyR0R3QUFBQUFB
d0FBQUVrRGJnTWxBQThBSHdBdkFBQWxGUlFHSXlFaUpqMEJORFl6SVRJV0VSVVVCaU1oSWlZOUFU
UTJNeUV5RmhFVkZBWWpJU0ltUFFFME5qTWhNaFlEYmhZUC9Od1BGaFlQQXlRUEZoWVAvTndQRmhZ
UEF5UVBGaFlQL053UEZoWVBBeVFQRnJkSkR4WVdEMGtQRlJVQkZVa1BGUlVQU1E4V0ZnRVdTUThX
Rmc5SkR4WVdBQW9BQUFCSkE3Y0RiZ0FRQUNBQU1RQkJBRkVBWWdCeUFJTUFsQUNrQUFBbE5UUW1L
d0VpQmgwQkZCWTdBVEkyTlQwQk5DWXJBU0lHSFFFVUZqc0JNallGTlRRbUt3RWlCaDBCRkJZN0FU
STJOUUUxTkNZckFTSUdIUUVVRmpzQk1qWUZOVFFtS3dFaUJoMEJGQlk3QVRJMkJUVTBKaXNCSWdZ
ZEFSUVdPd0V5TmpVQk5UUW1Ld0VpQmgwQkZCWTdBVEkyQlRVMEppc0JJZ1lkQVJRV093RXlOalU5
QVRRbUt3RWlCaDBCRkJZN0FUSTJOVGNSRkFZaklTSW1OUkUwTmpNaE1oWUJKUXNJdHdjTEN3ZTND
QXNMQ0xjSEN3c0h0d2dMQVNRS0NMY0lDZ29JdHdnSy90d0xDTGNIQ3dzSHR3Z0xBU1FLQ0xjSUNn
b0l0d2dLQVNVTENMWUlDd3NJdGdnTC90c0tDTGNJQ2dvSXR3Z0tBU1VMQ0xZSUN3c0l0Z2dMQ3dp
MkNBc0xDTFlJQzBrMkp2MEFKVFkySlFNQUpqYWxiUWdMQ3dodENBc0xDTnR1QndzTEIyNElDZ3JU
YlFnTEN3aHRDQXNMQ0FHMmJnZ0tDZ2h1QndzTDFHNEhDd3NIYmdnS0N0TnRDQXNMQ0cwSUN3c0lB
Ylp1Q0FvS0NHNEhDd3ZVYmdjTEN3ZHVDQW9LQ050dUNBb0tDRzRIQ3dzSHQvMlRKalkySmdKdEpq
WTJBQUFCQUFBQkFBSkpBa2tBRlFBQUFSUUdCd0VPQVNNaUppY0JMZ0UxTkRZeklUSVdGUUpKQmdY
L0FBVU5Cd2dOQmY4QUJRWVdEd0lBRHhVQ0pRZ05CZjhBQlFZR0JRRUFCUTBJRHhVVkR3QUFBQUVB
QUFEYkFra0NKUUFVQUFBQkZBWWpJU0ltTlRRMk53RStBVE15RmhjQkhnRUNTUlVQL2dBUEZnWUZB
UUFGRFFnSERRVUJBQVVHQVFBUEZoWVBCdzRGQVFBRkJnWUYvd0FGRGdBQkFDVUFrZ0Z1QXRzQUZR
QUFBUkVVQmlNaUppY0JMZ0UxTkRZM0FUNEJNeklXRlFGdUZnOEhEUWIvQUFVRkJRVUJBQVlOQnc4
V0FyZitBQThXQmdVQkFBVU9Cd2NOQmdFQUJRVVZEd0FBQUFFQUFBQ1NBVWtDMndBVkFBQUJGQVlI
QVE0Qkl5SW1OUkUwTmpNeUZoY0JIZ0VWQVVrR0JmOEFCUTBIRHhZV0R3Y05CUUVBQlFZQnR3Y09C
ZjhBQlFZV0R3SUFEeFVGQmY4QUJnMEhBQUFBQWdBQUFDVUNTUU5KQUJVQUt3QUFBUlFHQndFT0FT
TWlKaWNCTGdFMU5EWXpJVElXRlRVVUJpTWhJaVkxTkRZM0FUNEJNeklXRndFZUFSVUNTUVlGL3dB
RkRRY0lEUVgvQUFVR0ZnOENBQThWRlEvK0FBOFdCZ1VCQUFVTkNBY05CUUVBQlFZQlNRY05CdjhB
QlFVRkJRRUFCZzBIRHhZV0Q5d1BGaFlQQncwRkFRQUZCZ1lGL3dBRkRRY0FBQUFBQVFBQUFDVUNT
UUZ1QUJVQUFBRVVCZ2NCRGdFaklpWW5BUzRCTlRRMk15RXlGaFVDU1FZRi93QUZEUWNJRFFYL0FB
VUdGZzhDQUE4VkFVa0hEUWIvQUFVRkJRVUJBQVlOQnc4V0ZnOEFBQUFCQUFBQ0FBSkpBMGtBRlFB
QUFSUUdJeUVpSmpVME5qY0JQZ0V6TWhZWEFSNEJGUUpKRlEvK0FBOFdCZ1VCQUFVTkNBY05CUUVB
QlFZQ0pROFdGZzhIRFFVQkFBVUdCZ1gvQUFVTkJ3QUFBQWNBQUFBQUJBQURKUUFMQUJjQUxRQTVB
RVVBVVFCdEFBQVROQ1lqSWdZVkZCWXpNallUTkNZaklnWVZGQll6TWpZWE56WW1KekVtQmc4QkRn
RUhCaFlYRmpZM05pWW5KVFFtSXlJR0ZSUVdNekkyQVRRbUl5SUdGUlFXTXpJMkJUUW1JeUlHRlJR
V016STJGeFFHQnc0Qkl5RWlKaWN1QVRVME56NEJOell6TWhjZUFSY1dGZHNxSHg0ckt4NGZLbTRy
SGg0ckt4NGVLL1U1QkE4UERoc0RPaUkyQ1F3dUxDeFBDd2taSEFGNUt4NGZLaW9mSGl2K2tpc2VI
aXNySGg0ckFRQXJIaDRyS3g0ZUs3Y3BLQVVRQ2Z6ZUNSQUZLQ2tvS1l0ZFhXcHFYVjJMS1NnQkpS
NHJLeDRmS3lzQkh4NHJLeDRmS3l2MDJnOGFCQU1QRDlvREt5TXNUd3NNTGl3alFCUVRIaXNySGg4
ckt3R01IeW9xSHg0ckswOGVLeXNlSHlzcjRVcU1QZ2dKQ1FnOWpVcHBYbDJMS0NrcEtJdGRYbWtB
QUFBQUFnQWwvN2NEMndPM0FCSUFTUUFBQlRRbUl5SW1OVFFtSXlJR0ZSUVdNekkyTlNVVUJpTWhG
QVlqSWlZMUlTSW1OVFkzUGdFM05qVTBOejRCTnpZM0xnRTFORFl6TWhZVkZBWUhGaGNlQVJjV0ZS
UVhIZ0VYRmhjQ0NRVUVJakFHQXdRR095b0VCUUhTS3g3L0FGWThQRmIvQUI0ckh5RWdOUkVRRVJF
K0xTMDVBZ01nRnhjZ0F3STVMUzArRVJFUUVUVWdJUjhTQkFVd0lnUUZCUVFwT3dVRXBCNHJQRlpX
UENzZUd5Y25jVTFOYUNrcEtrVVpHUWtGQ3dZWElDQVhCZ3NGQ1JrWlJTb3BLV2hOVFhFbkp4c0FB
QUFHQUFEL3R3TnVBN2NBRXdBY0FDWUFOd0JIQUZnQUFBRWVBUlVSRkFZaklTSW1OUkUwTmpNaE1o
WVhCeFV6TGdFdkFTNEJFeEVqSWlZOUFTRVJJUUUwTmpNaE1oWWRBUlFHSXlFaUpqMEJCVElXSFFF
VUJpTWhJaVk5QVRRMk13VXlGaDBCRkFZaklTSW1QUUUwTmpNaEEwY1FGeUFYL1FBWElDQVhBZ0FY
TnhCTTF3TUhBN0lERHRYdUZ5RCtTUUxjL2JZTENBR1NDQW9LQ1A1dUNBc0JwUWdLQ2dqK2JnZ0xD
d2dCa2dnS0NnaitiZ2dMQ3dnQmtnTGVFRGNYL1c0WElDQVhBNUlYSUJjUUo5Y0lEUU96QXdmOG1R
SkpJQmZ1L0pJQjdnY0xDd2NsQ0FvS0NDV0FDd2drQ0FvS0NDUUlDNU1LQ0NRSUN3c0lKQWdLQUFB
QUFnQWFBSFVDUWdLdkFDUUFTUUFBSlJRR0R3RU9BU01pSmljQkxnRTFORFkzQVQ0Qk16SVdId0Vl
QVJVVUJnOEJGeDRCRlRNVUJnOEJEZ0VqSWlZbkFTNEJOVFEyTndFK0FUTXlGaDhCSGdFVkZBWVBB
UmNlQVJVQlpnTUNIUU1IQXdRSEF2NzFBZ01EQWdFTEFnY0VBd2NESFFJREF3TGg0UUlEM0FNREhR
SUhCQU1IQS83MkF3TURBd0VLQXdjREJBY0NIUU1EQXdQaDRRTURwUVFIQXh3REF3TURBUW9EQndN
RUJ3SUJDd0lEQXdJZEFnZ0RBd2dDNGVBREJ3TUVCd01jQXdNREF3RUtBd2NEQkFjQ0FRc0NBd01D
SFFJSUF3TUlBdUhnQXdjREFBQUFBZ0FIQUhVQ0x3S3ZBQ1FBU1FBQUFSUUdCd0VPQVNNaUppOEJM
Z0UxTkRZL0FTY3VBVFUwTmo4QlBnRXpNaFlYQVI0QkZUTVVCZ2NCRGdFaklpWXZBUzRCTlRRMlB3
RW5MZ0UxTkRZL0FUNEJNeklXRndFZUFSVUJWQU1EL3ZZREJ3TUVCd0lkQWdRRUF1SGhBZ1FFQWgw
Q0J3UURCd01CQ2dNRDJ3TUMvdlVDQndRREJ3TWNBd01EQStEZ0F3TURBeHdEQndNRUJ3SUJDd0lE
QVpJREJ3UCs5Z01EQXdNY0F3Y0VBd2NENE9FQ0NBTURDQUlkQWdNREF2NzFBZ2NFQXdjRC92WURB
d01ESEFNSEJBTUhBK0RoQWdnREF3Z0NIUUlEQXdMKzlRSUhCQUFBQVFBYUFIVUJaZ0t2QUNRQUFB
RVVCZzhCRng0QkZSUUdEd0VPQVNNaUppY0JMZ0UxTkRZM0FUNEJNeklXSHdFZUFSVUJaZ01DNGVF
Q0F3TUNIUU1IQXdRSEF2NzFBZ01EQWdFTEFnY0VBd2NESFFJREFvQURDQUxoNEFNSEF3UUhBeHdE
QXdNREFRb0RCd01FQndJQkN3SURBd0lkQWdjRUFBQUFBQUVBQndCMUFWUUNyd0FrQUFBQkZBWUhB
UTRCSXlJbUx3RXVBVFUwTmo4Qkp5NEJOVFEyUHdFK0FUTXlGaGNCSGdFVkFWUURBLzcyQXdjREJB
Y0NIUUlFQkFMaDRRSUVCQUlkQWdjRUF3Y0RBUW9EQXdHU0F3Y0QvdllEQXdNREhBTUhCQU1IQStE
aEFnZ0RBd2dDSFFJREF3TCs5UUlIQkFBQUFBQUNBQUFBQUFSSkE3Y0FEd0F1QUFBQkVUUW1JeUVp
QmhVUkZCWXpJVEkyRXhFVUJpTWhGQllWRkFZaklTSW1OVFEyTlNFaUpqVVJORFl6SVRJV0ZRUUFD
d2Y4YlFjTEN3Y0Rrd2NMU1RZbC9za2tGUS8rMnc4VkpQN0pKVFkySlFPVEpUWUJnQUhiQ0FzTENQ
NGxCd3NMQWVMOWt5WTJKRG9QRHhZV0R3ODVKVFltQW0wbU5qWW1BQUFBQUFNQUFBQkpBcElEYmdB
TUFCd0FMQUFBSlRRbUl5SUdGUlFXTXpJMk5UY1JOQ1lqSVNJR0ZSRVVGak1oTWpZVEVSUUdJeUVp
SmpVUk5EWXpJVElXQVc0V0R3OFZGUThQRnRzTEIvNGtCd3NMQndIY0J3dEpOU2IrSkNVMk5pVUIz
Q1kxa2c4V0ZnOFBGUlVQWEFJa0NBc0xDUDNjQ0FzTEFpejlreVkyTmlZQ2JTWTJOZ0FBQUFBQ0FB
QUFBQU51QTI0QUhBQTVBQUFCSWdjT0FRY0dGUlFYSGdFWEZqTXlOejRCTnpZMU5DY3VBU2NtSXdF
VUJ3NEJCd1lqSWljdUFTY21OVFEzUGdFM05qTXhNaGNlQVJjV0FiZEFPVGxVR1JnWUdWUTVPVUJB
T1RoVkdCa1pHRlU0T1VBQnR5TWlkMUJRVzF0UVVIY2pJaUlqZDFCUVcxdFFVSGNpSXdMdUdSaFZP
RGxBUURrNVZCa1lHQmxVT1RsQVFEazRWUmdaL3NsYlVGQjNJaU1qSW5kUVVGdGJVRkIzSWlNaklu
ZFFVQUFBQUFBSUFDWC8yd1BiQTdjQUN3QVhBQ01BTHdBN0FFY0FWQUJoQUFBbEZBWWpJaVkxTkRZ
ek1oWUZGQVlqSWlZMU5EWXpNaFlCRkFZaklpWTFORFl6TWhZQkZBWWpJaVkxTkRZek1oWUJGQVlq
SWlZMU5EWXpNaFlCRkFZaklpWTFORFl6TWhZQkZBWWpJaVkxTkRZek1oWVZCUlFHSXlJbU5UUTJN
eklXRlFFdEt4OGVLeXNlSHlzQkhDc2VIaXNySGg0ci9tNHJIaDhxS2g4ZUt3S3ZLeDRmS3lzZkhp
djkyVFltSlRZMkpTWTJBcHdxSHg0ckt4NGZLdjZUUVMwdFFVRXRMVUVCTDB3ME5rcEtOalJNbWg0
ckt4NGZLaXFVSHlzckh4NHJLd0YwSHlvcUh4NHJLLzdGSGlzckhoOHFLZ0lhSlRZMkpTWTJOdjYr
SHlvcUh4NHJLd0YwTFVGQkxTNUFRQzUyTlV0TE5UVkxTelVBQUFBQUFRQUFBQUFEYmdOdUFCc0FB
QUVVQnc0QkJ3WWpJaWN1QVNjbU5UUTNQZ0UzTmpNeUZ4NEJGeFlEYmlNaWQxQlFXMXRRVUhjaklp
SWpkMUJRVzF0UVVIY2lJd0czVzFCUWR5SWpJeUozVUZCYlcxQlFkeUlqSXlKM1VGQUFBQUFBQWdB
TkFFa0R0d0txQUJVQUpRQUFDUUVHSWk4QkpqUS9BU2NtTkQ4Qk5qSVhBUllVQndFVkZBWWpJU0lt
UFFFME5qTWhNaFlCVHY3MkJnOEZIUVVGNGVFRkJSMEZEd1lCQ2dZR0Fta0xCLzNiQ0FvS0NBSWxC
d3NCaGY3MkJnWWNCZzhHNE9FRkVBVWRCUVgrOVFVUEJ2NzdKUWNMQ3djbENBb0tBQU1BSHdBTEJD
b0RHZ0FWQUNZQVBBQUFKUWNHSWljQkpqUTNBVFl5SHdFV0ZBOEJGeFlVQndFRERnRXZBUzRCTnhN
K0FSOEJIZ0VIQ1FFR0lpOEJKalEvQVNjbU5EOEJOaklYQVJZVUJ3RmhIUVlQQmY3MUJRVUJDd1VQ
QmgwRkJlSGhCUVVCVWRVQ0RRY2tCd2NDMVFJTkJ5UUhCd0lCZVA3MUJROEdIQVlHNE9BR0Jod0dE
d1VCQ3dVRmx4d0dCZ0VLQmc4RkFRc0ZCUjBGRUFYaDRBWVBCZ0ppL1I0SEJ3SUtBZzBIQXVJSENB
SUtBZzRIL296KzlnWUdIQVlQQnVEaEJSQUZIUVVGL3ZVRkR3WUFBQUFFQUFBQUFBSkpBMjRBQ3dB
WEFDTUFXd0FBTnpRbUl5SUdGUlFXTXpJMkVUUW1JeUlHRlJRV016STJCVFFtSXlJR0ZSUVdNekky
TnhRR0J4UUhEZ0VIQmdjT0FSMEJIZ0VWRkFZaklpWTFORFkzRVM0Qk5UUTJNeklXRlJRR0J4RStB
VGMrQVRjdUFUVTBOak15RmhXbElSWVhJQ0FYRmlFaEZoY2dJQmNXSVFGdElCY1dJQ0FXRnlBM0ho
a1pHVWtyS2lWRU5CZ2VRQzB1UUI0WkdSNUFMaTFBSGhnVkxoVlBXQUVaSGtBdExrQnVGaUVoRmhj
Z0lBS3BGeUFnRnhjZ0lESVhJQ0FYRnlBZ0Z4OHlEazB5TVQ0UkVBd1ZJeW9PRHpJZUxrQkFMaDR5
RHdIVURqTWVMVUZCTFI0ekR2N2tDdzhIR1Q5YURqSWZMVUZCTFFBQUFnQTZBRWtDUndNbEFCQUFR
d0FBSlJVVUJpc0JJaVk5QVRRMk93RXlGaFVURkFZSERnRVZNUlFHS3dFaUpqMEJORFkzUGdFMU5D
WWpJZ1lIRGdFSERnRWpJaVl2QVM0Qk56NEJNeklYSGdFWEZoVUJrZzBLaVFrT0RnbUpDZzIxVnlr
WkhnMEtpUWtNVWlVaEhqWWlGQ0VKQ2hvWkJBa0ZCQWNEWGdjREJDK0RWeTR2TGtvWUYrbUpDUTRP
Q1lrS0RRMEtBVmRSVFJjT01Rd0pFaFlLR1RSVEVROGhHeGdsQ2djSEd5QUVCUU1DUndZUkNFeE1F
aEU5S2lveEFBQUFBZ0FBQUFBQzJ3TnVBQWtBSndBQUFSRWhFVDRCTno0Qk5STVJGQWNPQVFjR0J3
NEJJeUltSnlZbkxnRW5KalVSTkRZeklUSVdGUUp1L3dBV1FTSXVXVzAwTllFNE53VUVCd1FFQ0FN
Rk9EZUJOVFVXRHdLU0R4VUJrZ0Z1L1hZTUtCc2thejRCdC81SldrbElhQndkQXdFQ0FnRURIUnhv
U0VsYUFiY1BGaFlQQUFBQUF3QUFBVzRESlFKSkFBOEFId0F3QUFBVEZSUUdLd0VpSmowQk5EWTdB
VElXQlJVVUJpc0JJaVk5QVRRMk93RXlGZ1VWRkFZckFTSW1QUUUwTmpzQk1oWVYyeUFXYmhjZ0lC
ZHVGaUFCSlNBWGJoWWdJQlp1RnlBQkpTRVdiaGNnSUJkdUZpRUNFbTBYSUNBWGJSY2dJQmR0RnlB
Z0YyMFhJQ0FYYlJjZ0lCZHRGeUFnRndBQUFBTUFBQUJKQU5zRGJnQVBBQjhBTHdBQU54VVVCaXNC
SWlZOUFUUTJPd0V5RmhFVkZBWXJBU0ltUFFFME5qc0JNaFlSRlJRR0t3RWlKajBCTkRZN0FUSVcy
eUFXYmhjZ0lCZHVGaUFnRm00WElDQVhiaFlnSUJadUZ5QWdGMjRXSU81dUZ5QWdGMjRXSVNFQkRt
MFhJQ0FYYlJjZ0lBRU9iaGNnSUJkdUZ5QWdBQUFBQUFJQUFBQUFBMjREYmdBY0FEUUFBQUV5Rng0
QkZ4WVZGQWNPQVFjR0l5SW5MZ0VuSmpVME56NEJOell6RXo0Qk5UUW1KeVVtSWdjT0FSVVJGQllY
SGdFek1qWTNBYmRiVUZCM0lpTWpJbmRRVUZ0YlVGQjNJeUlpSTNkUVVGdmJDUW9LQ2Y3SkNCUUlD
UWtKQ1FRSkJRVUpCQU51SXlKM1VGQmJXMUJRZHlJakl5SjNVRkJiVzFCUWR5SWovaWtGRVFvS0VR
UzNCUVFGRVFyK2tnb1JCUUlEQXdNQUFnQUFBQUFEYmdOdUFCOEFMd0FBQVJFMEppTWhJZ1lIQmhZ
ZkFRRUdGQjhCRmpJM0FSY2VBVE15TmpjK0FUVVRFUlFHSXlFaUpqVVJORFl6SVRJV0F0c1ZELzd1
REJJRUJBUUlVdjdQQ3dzNkN4NExBVEZTQlE0SEF3Z0RDZ3lUWVVUOTNFVmdZRVVDSkVSaEFhVUJF
ZzhWREFvS0ZnaFMvczhMSGdzNkN3c0JNVklHQlFJQkJCSU1BU1Q5M0VWZ1lFVUNKRVJoWVFBRkFB
RC90d051QTdjQUNBQWFBQ3NBUEFCTkFBQUJIZ0VYSVJFZUFSY0RJUkVVQmlNaElpWTFFVFEyTXlF
UkZCWVROVFFtSXlFaUJoMEJGQll6SVRJMk5UMEJOQ1lqSVNJR0hRRVVGak1oTWpZMVBRRTBKaU1o
SWdZZEFSUVdNeUV5TmpVRFJ3UUlCUDd5QmdvRkp3RTNJQmY5QUJjZ0lCY0J5U0J5Q2dqK2JnZ0xD
d2dCa2dnS0NnaitiZ2dMQ3dnQmtnZ0tDZ2orYmdnTEN3Z0JrZ2dLQXFjRUN3WUJEZ1FJQlA2NS9h
VVhJQ0FYQTVJWElQN0pGeUQrWENRSUNnb0lKQWdMQ3dpU0pBZ0xDd2drQ0FvS0NKSWxCd3NMQnlV
SUNnb0lBQUFBQUFNQUFBQUFBNUlEYmdBTEFCd0FYQUFBTnpRbUl5SUdGUlFXTXpJMkV4RVVCaXNC
SWlZMUVUUTJPd0V5RmhVaEZBWUhIZ0VWRmdZSEZoUUhEZ0VIRmdZSERnRXJBU0ltSnk0Qkl5NEJO
UkUwTmpjK0FUYytBVGMrQVRjK0FUYytBVE15RmhVVUJnY09BUWN6TWhZVmtoVVBFQlVWRUE4VlhC
WVBwQThXRmcra0R4WUNwQkVPQlFNQkRBMEZCUVFRQ3dNUEVCSTVKa2srY1M0Yk1BOFBGaFVPRUVJ
VEVCME5FUTBGQlE4U0JRNEhZQ0FWQ3dVSUJKOHNRYmNQRlJVUEVCVVZBVFQra3c4V0ZnOEJiUThX
Rmc4V0xoRU5HQVlXSnhFUUloRVFIQW9nTkJNV0Z5QVFDUkFCRlE4QmJnNFZBZ0ZOR0JRa0RSRXdH
Umt5RWdVR2NpRWpOQllLRFE1QkxRQUFBQUFEQUFEL3R3T1NBeVVBREFBY0FGd0FBQk1VQmlNaUpq
VTBOak15RmhVVEVUUW1Ld0VpQmhVUkZCWTdBVEkySlI0QkZSUUdLd0VlQVJjZUFSVVVCaU1pSmlj
dUFTY3VBU2N1QVNjdUFTY3VBVFVSTkRZelBnRTNQZ0U3QVI0QkZ4NEJCeDRCRnhZVUJ4NEJCeFFH
QjVJVkR4QVZGUkFQRlZ3V0Q2UVBGaFlQcEE4V0FvVU9FVUVzbndRSUJRc1ZJR0FIRGdVU0R3VUZE
UkVOSFJBVFFoQU9GUllQRHpBYkxuRStTU1k1RWhBUEF3c1FCQVVGRFF3QkF3VUNiZzhXRmc4UEZS
VVAvdHNCYmc4VkZRLytrZzhWRldRUUxoY3NRZzBPQ1JZMUl5RnhCZ1VTTWhrWk1CRU5JeFFaVFFF
QkZROEJiZzhWQVE4S0R5RUJGaFlUTkNBTEd4QVJJaEFSS0JVR0dBNEFBUUFBQUFBRFhBTnVBRHNB
QUFFaEhnRVZGQWNPQVFjR0l5SW5MZ0VuSmpVME56NEJOell6TWhjZUFSY1dGd2N1QVNNaUJ3NEJC
d1lWRkJjZUFSY1dNekkzUGdFM05qY2pOUUczQVo0REJCNGViMDVOWDF0UVVIY2pJaUlqZDFCUVd5
d3BLVW9nSVIxM0dWWkFPREl4U2hZVkZSWktNVEk0UVM0dE93OFBCUGtCOWhFakZWNVBUM0VnSUNJ
amQxQlFXMXRRVUhjaUl3Z0lIaFVWRzNNWUxCWVdTekl6T1RveU0wc1dGUlFWT2lFaEdKY0FCZ0FB
LzdjRGJnTzNBQk1BSEFBbUFEc0FVQUJnQUFBQkhnRVZFUlFHSXlFaUpqVVJORFl6SVRJV0Z3Y1ZN
eTRCTHdFdUFSTVJJeUltUFFFaEVTRUJQZ0VmQVI0QkR3RVhGZ1lQQVFZbUx3RW1ORGNoRmhRUEFR
NEJMd0V1QVQ4Qkp5WTJQd0UyRmhjRExnRTNFejRCSHdFZUFRY0REZ0VuQTBjUUZ5QVgvUUFYSUNB
WEFnQVhOeEJNMXdNSEE3SUREdFh1RnlEK1NRTGMvZTBGRHdZZEJnSUVhR2dFQWdZZEJnOEZnUU1E
QWt3REE0SUVEd1lkQmdNRmFHZ0ZBd1lkQmc4RTRRY0pBVThDREFja0NBa0NUd0VNQ0FMZUVEY1gv
VzRYSUNBWEE1SVhJQmNRSjljSURRT3pBd2Y4bVFKSklCZnUvSklDQUFZQ0JSVUZEd2FMaXdZUEJC
WUZBZ2FzQlF3RkJRd0ZyQVlDQlJZRUR3YUxpd1lQQlJVRkFnYitUUUVNQ0FIYkJ3a0NCUUlNQi80
bENBZ0JBQUFCQUJML3lRUHVBNThBT2dBQUFSUUhEZ0VIQmlNaUp5NEJKeVkxTkRjK0FUYzJOeFVH
Qnc0QkJ3WVZGQmNlQVJjV016STNQZ0UzTmpVMEp5NEJKeVluTlJZWEhnRVhGaFVEN2ljbmhscGFa
bVphV29Zbkp5RWdjVTFOV1Q4MU5rNFdGeDBkWTBOQ1RFeENRMk1kSFJjV1RqWTFQMWxOVFhFZ0lR
RzNabHBhaGljbkp5ZUdXbHBtWFZOVGdpc3JEWUlOSUNGZk96eENURUpEWXgwZEhSMWpRMEpNUWp3
N1h5RWdEWUlOS3l1Q1UxTmRBQUFDQUFBQUFBTnVBMjRBSEFBNUFBQUJJZ2NPQVFjR0ZSUVhIZ0VY
RmpNeU56NEJOelkxTkNjdUFTY21Jd0VVQnc0QkJ3WWpJaWN1QVNjbU5UUTNQZ0UzTmpNeE1oY2VB
UmNXQWJkTVFrTmpIUjBkSFdORFFreExRME5qSFIwZEhXTkRRMHNCdHlNaWQxQlFXMXRRVUhjaklp
SWpkMUJRVzF0UVVIY2lJd01sSFIxalEwTkxURUpEWXgwZEhSMWpRMEpNUzBORFl4MGQvcEpiVUZC
M0lpTWpJbmRRVUZ0YlVGQjNJaU1qSW5kUVVBQUFBQUFKQUFBQUFBTnVBeVVBQXdBVUFCZ0FIQUFn
QURFQVFnQkdBRW9BQURjVkl6VWxNaFlkQVJRR0t3RWlKajBCTkRZN0FUY1ZJVFVURlNNMUFSVWhO
UU15RmgwQkZBWXJBU0ltUFFFME5qc0JBVElXSFFFVUJpc0JJaVk5QVRRMk93RVhGU00xRXhVaE5j
bkpBWklQRmhZUGtnOFdGZytTWFA0U2dJQURidjViZ0E4V0ZnK1NEeFlXRDVJQmJnOFZGUStTRHhZ
V0Q1SzNnSUQrRXBKSlNVa1ZENUlQRmhZUGtnOFYzRWxKQVNSSlNmMjNTVWtDa3hZUGtnOFdGZytT
RHhiKzJ4WVBrZzhWRlErU0R4WkpTVWtCSkVsSkFBTUFEZiszQklZRHR3QVJBQ1FBVlFBQUFSWVhI
Z0VYRmhjVUJpTWhGQVlqSWlZMUZ6STJOVFFtSXlJbU5UUW1JeUlHRlJRV013RVdGQWNCQmlZdkFT
WTJQd0V1QVRVMk56NEJOelkxTkRjK0FUYzJOeTRCTlRRMk16SVdGUlFHQng0QkZ6YzJGaGNEZWdz
VEV5MGFHaGtzSHY4QVZUMDhWcElFQlFVRUlqQUZCQVFGT3lrQ1BRUUcrOUlGRUFRd0JRRUZhd1lG
SHlFaE5CRVJFQkUrTFMwNUFnTWdGeGNnQWdOS2F4M3ZCZzhGQWRCQU1qRk5IQjBWSGlzOFZsVTla
QVVFQkFVd0lnUUZCUVFwT3dPU0JnOEYvR0VGQVFZM0JnOEZYQWdUQ2hzbkozRk5UV2dwS1NwRkdS
a0pCUXNHRnlBZ0Z3WUxCUXRNTXM4RkFRWUFBQUFFQUEzL3R3U0dBN2NBRWdBbEFEMEFid0FBQlRR
bUl5SW1OVFFtSXlJR0ZSUVdNekkyTlFrQkxnRWpJZ2NPQVFjR0ZSUUhEZ0VIQmdjRkZBWWpJUlFH
SXlJbU5UY2hMZ0VuTnhZWEhnRVhGaGNURnhZVUJ3RUdKaThCSmpZL0FTNEJOVFkzUGdFM05qVTBO
ejRCTnpZM0xnRTFORFl6TWhZVkZBWUhIZ0VYTnpZV0Z3SlNCUVFpTUFVRUJBVTdLUVFGL3MwQjlo
WmtVamdwS2pZTkRRVUZFdzhQRkFNR0xCNy9BRlU5UEZaVkFiRXdRUkUvQ3hNVExSb2FHVEV3QkFi
NzBnVVFCREFGQVFWckJnVWZJU0UwRVJFUUVUNHRMVGtDQXlBWEZ5QUNBMHBySGU4R0R3VVNCQVV3
SWdRRkJRUXBPd1VFQVE4QnNpMUpFeEkzSUNBYk56TXlYQ29wSm1zZUt6eFdWVDFKTm9OT04wQXlN
VTBjSFJVREhEY0dEd1g4WVFVQkJqY0dEd1ZjQ0JNS0d5Y25jVTFOYUNrcEtrVVpHUWtGQ3dZWElD
QVhCZ3NGQzB3eXp3VUJCZ0FBQUFBQ0FBQUFBQVNTQTI0QUJRQUxBQUFsRlNFUk14RUJFeUVSQ1FF
RWt2dHVTUU51a3Z4SkFRQUJTVWxKQTI3ODJ3SkovZ0FCU1FGSy9yWUFBQUFEQUFBQUFBUGJBN2NB
RndBZ0FDa0FBQWtCQmdjT0FRY0dJeUluTGdFbkpqVTBOejRCTnpZekVSY2hGQWNPQVFjR0J4TWhF
VElYSGdFWEZnRzNBVGdlSXlSUEt5d3RXMUJRZHlNaUlpTjNVRkJiYXdHNUNRa2hHUmdlWHY1Slcx
QlFkeUlqQWJyK3lCNFlHQ0lKQ1NNaWQxQlFXMXRRVUhjaUkvNU1BeTRySzFBakl4NEJnUUczSXlK
M1VGQUFBQUFEQUFBQVNRU1NBeVVBR3dBNUFGY0FBQUUwSnk0Qkp5WWpJZ2NPQVFjR0ZSUVhIZ0VY
RmpNeU56NEJOellsTkNjdUFTY21Ld0VXRng0QkZ4WVZGQWNPQVFjR0J6TXlOejRCTnpZM0ZBY09B
UWNHSXlFaUp5NEJKeVkxTkRjK0FUYzJNeUV5Rng0QkZ4WUNraGNYVHpZMVBEMDFOVkFYRnhjWFVE
VTFQVHcxTms4WEZ3RzNGeGRQTmpVODNTRWNHeVlMQ2dvTEpoc2NJZDA4TlRaUEZ4ZEpIQjFrUWtO
TC9rbE1RMEpqSFIwZEhXTkNRMHdCdDB0RFFtUWRIQUczUERZMVR4Y1hGeGRQTlRZOFBUVTFVQmNY
RnhkUU5UVTlQRFkxVHhjWEdTQWdTeWtxTFMwcUtrc2dJQmtYRjFBMU5UMU1Ra05qSFIwZEhXTkRR
a3hMUTBOakhSMGRIV05EUXdBQ0FBQUFTUVNTQXlVQUhnQTZBQUFUTkRjK0FUYzJNeUV5Rng0QkZ4
WVZGQWNPQVFjR0l5RWlKeTRCSnlZMUFUSTNQZ0UzTmpVMEp5NEJKeVlqSWdjT0FRY0dGUlFYSGdF
WEZnQWRIV05DUTB3QnQwdERRbVFkSEJ3ZFpFSkRTLzVKVEVOQ1l4MGRBeVU4TlRaUEZ4Y1hGMDgy
TlR3OU5UVlFGeGNYRjFBMU5RRzNTME5EWXgwZEhSMWpRME5MVEVKRFl4MGRIUjFqUTBKTS90c1hG
MUExTlQwOE5qVlBGeGNYRjA4MU5qdzlOVFZRRnhjQUFBQUFCQUFBLzdjRUFBTzNBQllBS2dBOUFF
a0FBQUV5RmhjZUFSY2xKZ2NPQVFjR0J5YzJOejRCTnpZekJSTVdGeDRCRnhZM0F5WW5MZ0VuSmpV
ME5qY0ZGaGNXQmdjR0J3NEJKeE0yTnpZbUp5WW5KeklXRlJRR0l5SW1OVFEyQWY1Q2d6MURaaUQr
V0MwckswZ2NIQStkSlMwdFpqZzNPUDVWd1JRZkgwd3FLeTJEWEU5UWRDRWhMQ2NEaXlFQkFUczVP
bE5Ea2tqb0dRc01BZzhQSHIxSVpXVklTR1ZsQTdjaUl5ZHRRQllEQ3dzdElpRXI4eTBrSXpFTURl
aitoaWtmSUNnSENBbisvZzR0TFlaV1ZtQk5qenhnVjF0YXFFaElNQ2NnQXdGa0ppc3JWU2twSWdO
bFIwaGxaVWhIWlFBQUF3QUFBQUFEYmdOdUFCQUFJUUE5QUFBQkVUUW1Ld0VpQmhVUkZCWTdBVEky
TlNFUk5DWXJBU0lHRlJFVUZqc0JNalkxTnhRSERnRUhCaU1pSnk0Qkp5WTFORGMrQVRjMk16SVhI
Z0VYRmdHU0NnaVNDQXNMQ0pJSUNnRUFDZ2lTQ0FzTENKSUlDdHdqSW5kUVVGdGJVRkIzSXlJaUkz
ZFFVRnRiVUZCM0lpTUJFZ0ZKQ0FzTENQNjNCd3NMQndGSkNBc0xDUDYzQndzTEI2VmJVRkIzSWlN
akluZFFVRnRiVUZCM0lpTWpJbmRRVUFBQUJBQUFBQUFEYmdOdUFCd0FPQUJKQUZrQUFBRXlGeDRC
RnhZVkZBY09BUWNHSXlJbkxnRW5KalUwTno0Qk56WXpFVEkzUGdFM05qVTBKeTRCSnlZaklnY09B
UWNHRlJRWEhnRVhGamNpSmpVUk5EWTdBVElXRlJFVUJpc0JJeUltTlJFME5qc0JNaFlWRVJRR0l3
RzNXMUJRZHlJakl5SjNVRkJiVzFCUWR5TWlJaU4zVUZCYlFEazRWUmdaR1JoVk9EbEFRRGs1VkJr
WUdCbFVPVGwzQ0FzTENHMElDd3NJYmR3SEN3c0hiZ2dLQ2dnRGJpTWlkMUJRVzF0UVVIY2lJeU1p
ZDFCUVcxdFFVSGNpSS8wU0dCbFVPVGxBUURrNFZSZ1pHUmhWT0RsQVFEazVWQmtZZ0FzSEFVa0lD
d3NJL3JjSEN3c0hBVWtJQ3dzSS9yY0hDd0FBQUFBQ0FCTUFBQVB0QTI0QUF3Qm9BQUFCTnlNSEFR
Y09BU3NCQnpNeUZoY2VBUThCRGdFckFRY09BU3NCSWlZbkxnRS9BU01IRGdFckFTSW1KeTRCUHdF
aklpWW5MZ0UvQVQ0Qk93RTNJeUltSnk0QlB3RStBVHNCTno0Qk93RXlGaGNlQVE4Qk16YytBVHNC
TWhZWEhnRVBBVE15RmhjZUFRY0NOaVdSSlFKSUlBSUpCN29sc2dRSEF3TUNBaUFCQ2dhN0xnSUtC
b0FFQ0FNREFRRXNrUzRDQ2dhQkF3Z0RBZ0lCTExFRkJ3TUNBZ0VnQWdrSHVpV3lCQWNEQXdJQ0lB
RUtCcnN1QWdvSGdBUUhBd01CQVN5UkxnSUtCNEFEQ0FNQ0FnRXNzUVVIQXdJQ0FRRnVrcElCSUlB
R0NKSUVBd1FJQklBR0NMc0dDQVFEQXdrRXNyc0dDQVFEQXdrRXNnUURBd2tFZ0FZSWtnUURBd2tF
Z0FZSXV3WUlCQU1FQ0FTeXV3WUlCQU1FQ0FTeUJBTUVDQVFBQkFBQUFBQURiZ051QUJBQVJBQmhB
SDRBQUFFVkZBWXJBU0ltUFFFME5qc0JNaFlWRXhRR0J3NEJIUUVVQmlzQklpWTlBVFEyTno0Qk5U
UW1JeUlHQnc0QkJ3NEJJeUltTHdFdUFUYytBVE14TWhjZUFSY1dGUU1pQnc0QkJ3WVZGQmNlQVJj
V016STNQZ0UzTmpVMEp5NEJKeVlqQVJRSERnRUhCaU1pSnk0Qkp5WTFORGMrQVRjMk16RXlGeDRC
RnhZQjl3c0hYQWdLQ2doY0J3dVNReDhXR2dzSFhBZ0tQQjhaSGpNY0R4NEtDUlFRQXdjRkF3VUNQ
Z1lDQkNOalFTTWtJemtTRXRKTVFrTmpIUjBkSFdORFFreExRME5qSFIwZEhXTkRRMHNCdHlNaWQx
QlFXMXRRVUhjaklpSWpkMUJRVzF0UVVIY2lJd0VKV3dnTEN3aGJDQW9LQ0FFY1BUb1NEUlFORXdj
TEN3Y25OU3dPREJVVUdTQUlCd1lXRXdRREFRSXZCQThHTnpZTkRTOGZJQ1VCQUIwZFkwTkRTMHhD
UTJNZEhSMGRZME5DVEV0RFEyTWRIZjZTVzFCUWR5SWpJeUozVUZCYlcxQlFkeUlqSXlKM1VGQUFB
QUFBQWdBQS83Y0VBQU8zQUJzQVFnQUFBUkVVQmlNaElpWTFFVFEyTno0Qk56NEJNeklXRng0QkZ4
NEJGUUUrQVRjK0FTOEJMZ0VIRGdFSERnRWpJaVluTGdFbkpnWVBBUVlXRng0QkZ4NEJNekkyTndR
QU5pWDh0aVUyQXdNdFd1QWJWQ1FrVkJ2Z1dpMERBLzYrUTJBaUJnSUZGUVVQQmlKZlF4dFVKQ1JV
RzBOZklnWVBCUlVGQWdZaVlFTWlaRGc1WlNBQ1FmM1JKVFkySlFJdkJBY0RKMHFqRkVCQUZLTktK
d01IQlA2eU1FWWFCQThHSGdZQ0JSbEdNQk5BUUJNeFJSa0ZBZ1llQmc4RUdrWXdHRWxLRndBQUFB
QURBQUQvdHdRQUE3Y0FNQUJIQUd3QUFBRVhGZ1lIQmdjT0FRY0dCdzRCS3dFaUppY21KeTRCSnlZ
bkxnRS9BVDRCRng0QkZ4NEJPd0V5TmpjK0FUYzJGaGNURVM0Qkp5NEJLd0VpQmdjT0FRY1JGQll6
SVRJMk5STVJGQVlqSVNJbU5SRTBOamMyTno0Qk56WTNQZ0U3QVRJV0Z4WVhIZ0VYRmhjZUFSVURT
aGNFQWdVV0lDQTlGeFlESGxBc0FpeFFIZ01XRmpzZklCVUdBZ1VWQlE4R0hsUTlGa2tlQWg1SkZq
OVdIZ1lQQkcwclNNWVdTUjRDSGtrV3hrZ3JDd2NEU2djTFNUWWwvTFlsTmd3TExDNHZWeWduSHg1
UUxBSXNVQjRkSnloWkx5OHJDd3dCclIwR0RnVVJHUmt2RVJJQkdUWTJHUUVSRVM0WUdSQUZEZ1ll
QmdJRkYwRXZFRHM3RURGQ0dBUUNCdjVsQWhNblBKb1FQRHdRbWp3bi9lMEhDd3NIQWhQOTdTVTJO
aVVDRXc4Y0Npa21Ka1FlSGhrWk5qWVpGeDRmUlNZbktBb2NEd0FBQXdBQS83Y0VrZ08zQUFNQURR
QW5BQUEzSVJFaEtRRVJJUlV6TWhZZEFRRVJGQVlqSVJVVUJpTWhJaVkxRVRRMk15RTFORFl6SVRJ
V2tnRzMva2tDU1FFbC9razNKalVCdHpVbS9xUTFKdjNiSlRZMkpRRmNOaVVDSlNZMVNRRWxBYmVU
TlNiSkFlMzkzQ1kyeVNVMk5pVUNKU1kxeVNZMk5nQUFBQUVBQUFBQkFBQ2Q0ajR4WHc4ODlRQUxC
QUFBQUFBQTNldTVtUUFBQUFEZDY3bVpBQUQvdHdTU0E3Y0FBQUFJQUFJQUFBQUFBQUFBQVFBQUE4
RC93QUFBQkpJQUFBQUFCSklBQVFBQUFBQUFBQUFBQUFBQUFBQUFBRzBFQUFBQUFBQUFBQUFBQUFB
Q0FBQUFCQUFBQUFPM0FBQUVBQUFBQkFBQUFBTzNBQUFEdHdBQUF0c0FBQU8zQUFBRUFBQUFCQUFB
QUFRQUFFVURKUUEvQTI0QUFBTnVBQUFESlFBQUE3Y0FFd051QUFBRGJnQUFBN2NBQUFOdUFBQURi
Z0FBQTI0QUFBUUFBQUFEWWdBQUJEMEFBQVFBQUFBRFlnQUFBa2tBQUFRQkFBQUR1d0FBQXlrQUFB
TnVBQUFEYmdBQUF3QUFZd0szQUQ0RGJnQUFBMjRBQUFNbEFBQUR0d0JOQTI0QUFBUUFBQWtFQUFB
K0JBQUFQZ1NTQUFBRVNRQUFCQUFBQUFNbEFBQUVBQUFsQTI0QUFRTWxBQU1EYmdBQUJFa0FBQU8z
QUFrRGJnQUFBN2NBQUFKSkFBQUNTUUFBQVpJQUpRRkpBQUFDU1FBQUFra0FBQUpKQUFBRUFBQUFC
QUFBSlFOdUFBQUNXd0FhQWpjQUJ3R0FBQm9CV3dBSEJFa0FBQUtTQUFBRGJnQUFCQUFBSlFOdUFB
QUR2UUFOQkVrQUh3SkpBQUFDZXdBNkF0c0FBQU1sQUFBQTJ3QUFBMjRBQUFOdUFBQURiZ0FBQTVJ
QUFBT1NBQUFEWEFBQUEyNEFBQVFBQUJJRGJnQUFBMjRBQUFTU0FBMEVrZ0FOQkpJQUFBUGJBQUFF
a2dBQUJKSUFBQVEzQUFBRGJnQUFBMjRBQUFRQUFCTURiZ0FBQkFBQUFBUUFBQUFFa2dBQUFBQUFB
QUFLQUJRQUhnQTZBS0FCQ2dGUUFab0I5Z0pNQXFnRFpBUG1CQ1FFZ0FUeUJjZ0dYQWE0QnZZSGFB
ZmVDRmdJekFsZ0NoQUtVQXE4QzJnTHZBdjhESWdOQUEwY0RWQU5iZzJZRGNRT1RBNjJEdWdQT2cr
ZUQvb1FKQkJRRUlBU1dCTEVFdllUaWhRbUZGd1U0aFdpRmw0V29oZDJGNTRYeEJmc0dCUVlYQmlF
R0t3WlRCbTJHallhcUJzYUcxZ2JsaHZjSENBY2VoMEdIVFlkZEIzY0hsd2V2QjcrSDBJZmhCL1dJ
Q0lna2lFWUlaNGgrQ0tTSXV3alJpT3dKREFrMkNUMkpUNGx3aVllSnBZbThDZHlLQW9vdmlrb0tj
d3FDZ0FCQUFBQWJRRlZBQW9BQUFBQUFBSUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFEZ0N1QUFF
QUFBQUFBQUVBQWdBQUFBRUFBQUFBQUFJQUJ3QXpBQUVBQUFBQUFBTUFBZ0FuQUFFQUFBQUFBQVFB
QWdCSUFBRUFBQUFBQUFVQUN3QUdBQUVBQUFBQUFBWUFBZ0F0QUFFQUFBQUFBQW9BR2dCT0FBTUFB
UVFKQUFFQUJBQUNBQU1BQVFRSkFBSUFEZ0E2QUFNQUFRUUpBQU1BQkFBcEFBTUFBUVFKQUFRQUJB
QktBQU1BQVFRSkFBVUFGZ0FSQUFNQUFRUUpBQVlBQkFBdkFBTUFBUVFKQUFvQU5BQm9abUVBWmdC
aFZtVnljMmx2YmlBeExqQUFWZ0JsQUhJQWN3QnBBRzhBYmdBZ0FERUFMZ0F3Wm1FQVpnQmhabUVB
WmdCaFVtVm5kV3hoY2dCU0FHVUFad0IxQUd3QVlRQnlabUVBWmdCaFJtOXVkQ0JuWlc1bGNtRjBa
V1FnWW5rZ1NXTnZUVzl2Ymk0QVJnQnZBRzRBZEFBZ0FHY0FaUUJ1QUdVQWNnQmhBSFFBWlFCa0FD
QUFZZ0I1QUNBQVNRQmpBRzhBVFFCdkFHOEFiZ0F1QUFBQUF3QUFBQUFBQUFBQUFBQUFBQUFBQUFB
QUFBQUFBQUFBQUFBQUFBQUFBQT09KSBmb3JtYXQoJ3dvZmYnKTtmb250LXdlaWdodDogbm9ybWFs
O2ZvbnQtc3R5bGU6IG5vcm1hbDtmb250LWRpc3BsYXk6IGJsb2NrO31bY2xhc3NePSJmYS0iXSwg
W2NsYXNzKj0iIGZhLSJdIHtmb250LWZhbWlseTogJ2ZhJyAhaW1wb3J0YW50O3NwZWFrOiBuZXZl
cjtmb250LXN0eWxlOiBub3JtYWw7Zm9udC13ZWlnaHQ6IG5vcm1hbDtmb250LXZhcmlhbnQ6IG5v
cm1hbDt0ZXh0LXRyYW5zZm9ybTogbm9uZTtsaW5lLWhlaWdodDogMTstd2Via2l0LWZvbnQtc21v
b3RoaW5nOiBhbnRpYWxpYXNlZDstbW96LW9zeC1mb250LXNtb290aGluZzogZ3JheXNjYWxlO30u
ZmEtc3BpbiB7LXdlYmtpdC1hbmltYXRpb246IGZhLXNwaW4gMnMgaW5maW5pdGUgbGluZWFyO2Fu
aW1hdGlvbjogZmEtc3BpbiAycyBpbmZpbml0ZSBsaW5lYXJ9QC13ZWJraXQta2V5ZnJhbWVzICJm
YS1zcGluIiB7MCUgey13ZWJraXQtdHJhbnNmb3JtOiByb3RhdGUoMGRlZyk7dHJhbnNmb3JtOiBy
b3RhdGUoMGRlZyk7fTEwMCUgey13ZWJraXQtdHJhbnNmb3JtOiByb3RhdGUoMzU5ZGVnKTt0cmFu
c2Zvcm06IHJvdGF0ZSgzNTlkZWcpO319QGtleWZyYW1lcyAiZmEtc3BpbiIgezAlIHstd2Via2l0
LXRyYW5zZm9ybTogcm90YXRlKDBkZWcpO3RyYW5zZm9ybTogcm90YXRlKDBkZWcpO30xMDAlIHst
d2Via2l0LXRyYW5zZm9ybTogcm90YXRlKDM1OWRlZyk7dHJhbnNmb3JtOiByb3RhdGUoMzU5ZGVn
KTt9fS5mYS1hc3RlcmlzazpiZWZvcmUge2NvbnRlbnQ6ICJcZjA2OSI7fS5mYS1wbHVzOmJlZm9y
ZSB7Y29udGVudDogIlxmMDY3Ijt9LmZhLXF1ZXN0aW9uOmJlZm9yZSB7Y29udGVudDogIlxmMTI4
Ijt9LmZhLXNlYXJjaDpiZWZvcmUge2NvbnRlbnQ6ICJcZjAwMiI7fS5mYS1lbnZlbG9wZS1vOmJl
Zm9yZSB7Y29udGVudDogIlxmMDAzIjt9LmZhLWhlYXJ0OmJlZm9yZSB7Y29udGVudDogIlxmMDA0
Ijt9LmZhLXN0YXI6YmVmb3JlIHtjb250ZW50OiAiXGYwMDUiO30uZmEtc3Rhci1vOmJlZm9yZSB7
Y29udGVudDogIlxmMDA2Ijt9LmZhLXVzZXI6YmVmb3JlIHtjb250ZW50OiAiXGYwMDciO30uZmEt
dGgtbGFyZ2U6YmVmb3JlIHtjb250ZW50OiAiXGYwMDkiO30uZmEtdGg6YmVmb3JlIHtjb250ZW50
OiAiXGYwMGEiO30uZmEtdGgtbGlzdDpiZWZvcmUge2NvbnRlbnQ6ICJcZjAwYiI7fS5mYS1jaGVj
azpiZWZvcmUge2NvbnRlbnQ6ICJcZjAwYyI7fS5mYS1jbG9zZTpiZWZvcmUge2NvbnRlbnQ6ICJc
ZjAwZCI7fS5mYS1yZW1vdmU6YmVmb3JlIHtjb250ZW50OiAiXGYwMGQiO30uZmEtdGltZXM6YmVm
b3JlIHtjb250ZW50OiAiXGYwMGQiO30uZmEtcG93ZXItb2ZmOmJlZm9yZSB7Y29udGVudDogIlxm
MDExIjt9LmZhLWNvZzpiZWZvcmUge2NvbnRlbnQ6ICJcZjAxMyI7fS5mYS1nZWFyOmJlZm9yZSB7
Y29udGVudDogIlxmMDEzIjt9LmZhLXRyYXNoLW86YmVmb3JlIHtjb250ZW50OiAiXGYwMTQiO30u
ZmEtaG9tZTpiZWZvcmUge2NvbnRlbnQ6ICJcZjAxNSI7fS5mYS1maWxlLW86YmVmb3JlIHtjb250
ZW50OiAiXGYwMTYiO30uZmEtY2xvY2stbzpiZWZvcmUge2NvbnRlbnQ6ICJcZjAxNyI7fS5mYS1k
b3dubG9hZDpiZWZvcmUge2NvbnRlbnQ6ICJcZjAxOSI7fS5mYS1wbGF5LWNpcmNsZS1vOmJlZm9y
ZSB7Y29udGVudDogIlxmMDFkIjt9LmZhLXJlcGVhdDpiZWZvcmUge2NvbnRlbnQ6ICJcZjAxZSI7
fS5mYS1yb3RhdGUtcmlnaHQ6YmVmb3JlIHtjb250ZW50OiAiXGYwMWUiO30uZmEtcmVmcmVzaDpi
ZWZvcmUge2NvbnRlbnQ6ICJcZjAyMSI7fS5mYS1saXN0LWFsdDpiZWZvcmUge2NvbnRlbnQ6ICJc
ZjAyMiI7fS5mYS10YWc6YmVmb3JlIHtjb250ZW50OiAiXGYwMmIiO30uZmEtdGFnczpiZWZvcmUg
e2NvbnRlbnQ6ICJcZjAyYyI7fS5mYS1saXN0OmJlZm9yZSB7Y29udGVudDogIlxmMDNhIjt9LmZh
LXBlbmNpbDpiZWZvcmUge2NvbnRlbnQ6ICJcZjA0MCI7fS5mYS1tYXAtbWFya2VyOmJlZm9yZSB7
Y29udGVudDogIlxmMDQxIjt9LmZhLWVkaXQ6YmVmb3JlIHtjb250ZW50OiAiXGYwNDQiO30uZmEt
cGVuY2lsLXNxdWFyZS1vOmJlZm9yZSB7Y29udGVudDogIlxmMDQ0Ijt9LmZhLWNoZWNrLXNxdWFy
ZS1vOmJlZm9yZSB7Y29udGVudDogIlxmMDQ2Ijt9LmZhLXBsYXk6YmVmb3JlIHtjb250ZW50OiAi
XGYwNGIiO30uZmEtcGF1c2U6YmVmb3JlIHtjb250ZW50OiAiXGYwNGMiO30uZmEtc3RvcDpiZWZv
cmUge2NvbnRlbnQ6ICJcZjA0ZCI7fS5mYS1jaGV2cm9uLWxlZnQ6YmVmb3JlIHtjb250ZW50OiAi
XGYwNTMiO30uZmEtY2hldnJvbi1yaWdodDpiZWZvcmUge2NvbnRlbnQ6ICJcZjA1NCI7fS5mYS1x
dWVzdGlvbi1jaXJjbGU6YmVmb3JlIHtjb250ZW50OiAiXGYwNTkiO30uZmEtaW5mby1jaXJjbGU6
YmVmb3JlIHtjb250ZW50OiAiXGYwNWEiO30uZmEtZXhjbGFtYXRpb24tY2lyY2xlOmJlZm9yZSB7
Y29udGVudDogIlxmMDZhIjt9LmZhLWV4Y2xhbWF0aW9uLXRyaWFuZ2xlOmJlZm9yZSB7Y29udGVu
dDogIlxmMDcxIjt9LmZhLXdhcm5pbmc6YmVmb3JlIHtjb250ZW50OiAiXGYwNzEiO30uZmEtY2hl
dnJvbi11cDpiZWZvcmUge2NvbnRlbnQ6ICJcZjA3NyI7fS5mYS1jaGV2cm9uLWRvd246YmVmb3Jl
IHtjb250ZW50OiAiXGYwNzgiO30uZmEtYmFyLWNoYXJ0OmJlZm9yZSB7Y29udGVudDogIlxmMDgw
Ijt9LmZhLWJhci1jaGFydC1vOmJlZm9yZSB7Y29udGVudDogIlxmMDgwIjt9LmZhLWNvZ3M6YmVm
b3JlIHtjb250ZW50OiAiXGYwODUiO30uZmEtZ2VhcnM6YmVmb3JlIHtjb250ZW50OiAiXGYwODUi
O30uZmEtZXh0ZXJuYWwtbGluazpiZWZvcmUge2NvbnRlbnQ6ICJcZjA4ZSI7fS5mYS1zcXVhcmUt
bzpiZWZvcmUge2NvbnRlbnQ6ICJcZjA5NiI7fS5mYS1iZWxsLW86YmVmb3JlIHtjb250ZW50OiAi
XGYwYTIiO30uZmEtY2VydGlmaWNhdGU6YmVmb3JlIHtjb250ZW50OiAiXGYwYTMiO30uZmEtZmls
dGVyOmJlZm9yZSB7Y29udGVudDogIlxmMGIwIjt9LmZhLWFycm93cy1hbHQ6YmVmb3JlIHtjb250
ZW50OiAiXGYwYjIiO30uZmEtZ3JvdXA6YmVmb3JlIHtjb250ZW50OiAiXGYwYzAiO30uZmEtdXNl
cnM6YmVmb3JlIHtjb250ZW50OiAiXGYwYzAiO30uZmEtY2hhaW46YmVmb3JlIHtjb250ZW50OiAi
XGYwYzEiO30uZmEtbGluazpiZWZvcmUge2NvbnRlbnQ6ICJcZjBjMSI7fS5mYS1iYXJzOmJlZm9y
ZSB7Y29udGVudDogIlxmMGM5Ijt9LmZhLW5hdmljb246YmVmb3JlIHtjb250ZW50OiAiXGYwYzki
O30uZmEtcmVvcmRlcjpiZWZvcmUge2NvbnRlbnQ6ICJcZjBjOSI7fS5mYS10YWJsZTpiZWZvcmUg
e2NvbnRlbnQ6ICJcZjBjZSI7fS5mYS1jYXJldC1kb3duOmJlZm9yZSB7Y29udGVudDogIlxmMGQ3
Ijt9LmZhLWNhcmV0LXVwOmJlZm9yZSB7Y29udGVudDogIlxmMGQ4Ijt9LmZhLWNhcmV0LWxlZnQ6
YmVmb3JlIHtjb250ZW50OiAiXGYwZDkiO30uZmEtY2FyZXQtcmlnaHQ6YmVmb3JlIHtjb250ZW50
OiAiXGYwZGEiO30uZmEtc29ydDpiZWZvcmUge2NvbnRlbnQ6ICJcZjBkYyI7fS5mYS11bnNvcnRl
ZDpiZWZvcmUge2NvbnRlbnQ6ICJcZjBkYyI7fS5mYS1zb3J0LWRlc2M6YmVmb3JlIHtjb250ZW50
OiAiXGYwZGQiO30uZmEtc29ydC1kb3duOmJlZm9yZSB7Y29udGVudDogIlxmMGRkIjt9LmZhLXNv
cnQtYXNjOmJlZm9yZSB7Y29udGVudDogIlxmMGRlIjt9LmZhLXNvcnQtdXA6YmVmb3JlIHtjb250
ZW50OiAiXGYwZGUiO30uZmEtZGFzaGJvYXJkOmJlZm9yZSB7Y29udGVudDogIlxmMGU0Ijt9LmZh
LXRhY2hvbWV0ZXI6YmVmb3JlIHtjb250ZW50OiAiXGYwZTQiO30uZmEtYmVsbDpiZWZvcmUge2Nv
bnRlbnQ6ICJcZjBmMyI7fS5mYS1maWxlLXRleHQtbzpiZWZvcmUge2NvbnRlbnQ6ICJcZjBmNiI7
fS5mYS1hbmdsZS1kb3VibGUtbGVmdDpiZWZvcmUge2NvbnRlbnQ6ICJcZjEwMCI7fS5mYS1hbmds
ZS1kb3VibGUtcmlnaHQ6YmVmb3JlIHtjb250ZW50OiAiXGYxMDEiO30uZmEtYW5nbGUtbGVmdDpi
ZWZvcmUge2NvbnRlbnQ6ICJcZjEwNCI7fS5mYS1hbmdsZS1yaWdodDpiZWZvcmUge2NvbnRlbnQ6
ICJcZjEwNSI7fS5mYS1kZXNrdG9wOmJlZm9yZSB7Y29udGVudDogIlxmMTA4Ijt9LmZhLXRhYmxl
dDpiZWZvcmUge2NvbnRlbnQ6ICJcZjEwYSI7fS5mYS1jaXJjbGUtbzpiZWZvcmUge2NvbnRlbnQ6
ICJcZjEwYyI7fS5mYS1zcGlubmVyOmJlZm9yZSB7Y29udGVudDogIlxmMTEwIjt9LmZhLWNpcmNs
ZTpiZWZvcmUge2NvbnRlbnQ6ICJcZjExMSI7fS5mYS10ZXJtaW5hbDpiZWZvcmUge2NvbnRlbnQ6
ICJcZjEyMCI7fS5mYS1jb2RlOmJlZm9yZSB7Y29udGVudDogIlxmMTIxIjt9LmZhLWNvZGUtZm9y
azpiZWZvcmUge2NvbnRlbnQ6ICJcZjEyNiI7fS5mYS1zaGllbGQ6YmVmb3JlIHtjb250ZW50OiAi
XGYxMzIiO30uZmEtZWxsaXBzaXMtaDpiZWZvcmUge2NvbnRlbnQ6ICJcZjE0MSI7fS5mYS1lbGxp
cHNpcy12OmJlZm9yZSB7Y29udGVudDogIlxmMTQyIjt9LmZhLXBsYXktY2lyY2xlOmJlZm9yZSB7
Y29udGVudDogIlxmMTQ0Ijt9LmZhLWV4dGVybmFsLWxpbmstc3F1YXJlOmJlZm9yZSB7Y29udGVu
dDogIlxmMTRjIjt9LmZhLWZpbGUtdGV4dDpiZWZvcmUge2NvbnRlbnQ6ICJcZjE1YyI7fS5mYS10
aHVtYnMtdXA6YmVmb3JlIHtjb250ZW50OiAiXGYxNjQiO30uZmEtdGh1bWJzLWRvd246YmVmb3Jl
IHtjb250ZW50OiAiXGYxNjUiO30uZmEtZ29vZ2xlOmJlZm9yZSB7Y29udGVudDogIlxmMWEwIjt9
LmZhLWZpbGUtY29kZS1vOmJlZm9yZSB7Y29udGVudDogIlxmMWM5Ijt9LmZhLWNpcmNsZS1vLW5v
dGNoOmJlZm9yZSB7Y29udGVudDogIlxmMWNlIjt9LmZhLWNpcmNsZS10aGluOmJlZm9yZSB7Y29u
dGVudDogIlxmMWRiIjt9LmZhLXNsaWRlcnM6YmVmb3JlIHtjb250ZW50OiAiXGYxZGUiO30uZmEt
YmVsbC1zbGFzaDpiZWZvcmUge2NvbnRlbnQ6ICJcZjFmNiI7fS5mYS1iZWxsLXNsYXNoLW86YmVm
b3JlIHtjb250ZW50OiAiXGYxZjciO30uZmEtYXJlYS1jaGFydDpiZWZvcmUge2NvbnRlbnQ6ICJc
ZjFmZSI7fS5mYS1waWUtY2hhcnQ6YmVmb3JlIHtjb250ZW50OiAiXGYyMDAiO30uZmEtdG9nZ2xl
LW9mZjpiZWZvcmUge2NvbnRlbnQ6ICJcZjIwNCI7fS5mYS10b2dnbGUtb246YmVmb3JlIHtjb250
ZW50OiAiXGYyMDUiO30uZmEtY2hyb21lOmJlZm9yZSB7Y29udGVudDogIlxmMjY4Ijt9LmZhLXBh
dXNlLWNpcmNsZTpiZWZvcmUge2NvbnRlbnQ6ICJcZjI4YiI7fS5mYS1wYXVzZS1jaXJjbGUtbzpi
ZWZvcmUge2NvbnRlbnQ6ICJcZjI4YyI7fS5mYS1oYXNodGFnOmJlZm9yZSB7Y29udGVudDogIlxm
MjkyIjt9LmZhLXF1ZXN0aW9uLWNpcmNsZS1vOmJlZm9yZSB7Y29udGVudDogIlxmMjljIjt9LmZh
LWVudmVsb3BlLW9wZW46YmVmb3JlIHtjb250ZW50OiAiXGYyYjYiO30uZmEtZW52ZWxvcGUtb3Bl
bi1vOmJlZm9yZSB7Y29udGVudDogIlxmMmI3Ijt9LmZhLXdpbmRvdy1yZXN0b3JlOmJlZm9yZSB7
Y29udGVudDogIlxmMmQyIjt9LmZhLW1pbnVzOmJlZm9yZSB7Y29udGVudDogIlxlYTBiIjt9PC9z
dHlsZT48c3R5bGU+LyohKiBCb290c3RyYXAgdjMuMy43IChodHRwOi8vZ2V0Ym9vdHN0cmFwLmNv
bSkqIENvcHlyaWdodCAyMDExLTIwMTYgVHdpdHRlciwgSW5jLiogTGljZW5zZWQgdW5kZXIgTUlU
IChodHRwczovL2dpdGh1Yi5jb20vdHdicy9ib290c3RyYXAvYmxvYi9tYXN0ZXIvTElDRU5TRSkq
Ly8qISBub3JtYWxpemUuY3NzIHYzLjAuMyB8IE1JVCBMaWNlbnNlIHwgZ2l0aHViLmNvbS9uZWNv
bGFzL25vcm1hbGl6ZS5jc3MgKi9odG1se2ZvbnQtZmFtaWx5OnNhbnMtc2VyaWY7LXdlYmtpdC10
ZXh0LXNpemUtYWRqdXN0OjEwMCU7LW1zLXRleHQtc2l6ZS1hZGp1c3Q6MTAwJX1ib2R5e21hcmdp
bjowfWFydGljbGUsYXNpZGUsZGV0YWlscyxmaWdjYXB0aW9uLGZpZ3VyZSxmb290ZXIsaGVhZGVy
LGhncm91cCxtYWluLG1lbnUsbmF2LHNlY3Rpb24sc3VtbWFyeXtkaXNwbGF5OmJsb2NrfWF1ZGlv
LGNhbnZhcyxwcm9ncmVzcyx2aWRlb3tkaXNwbGF5OmlubGluZS1ibG9jazt2ZXJ0aWNhbC1hbGln
bjpiYXNlbGluZX1hdWRpbzpub3QoW2NvbnRyb2xzXSl7ZGlzcGxheTpub25lO2hlaWdodDowfVto
aWRkZW5dLHRlbXBsYXRle2Rpc3BsYXk6bm9uZX1he2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJl
bnR9YTphY3RpdmUsYTpob3ZlcntvdXRsaW5lOjB9YWJiclt0aXRsZV17Ym9yZGVyLWJvdHRvbTox
cHggZG90dGVkfWIsc3Ryb25ne2ZvbnQtd2VpZ2h0OjcwMH1kZm57Zm9udC1zdHlsZTppdGFsaWN9
aDF7bWFyZ2luOi42N2VtIDA7Zm9udC1zaXplOjJlbX1tYXJre2NvbG9yOiMwMDA7YmFja2dyb3Vu
ZDojZmYwfXNtYWxse2ZvbnQtc2l6ZTo4MCV9c3ViLHN1cHtwb3NpdGlvbjpyZWxhdGl2ZTtmb250
LXNpemU6NzUlO2xpbmUtaGVpZ2h0OjA7dmVydGljYWwtYWxpZ246YmFzZWxpbmV9c3Vwe3RvcDot
LjVlbX1zdWJ7Ym90dG9tOi0uMjVlbX1pbWd7Ym9yZGVyOjB9c3ZnOm5vdCg6cm9vdCl7b3ZlcmZs
b3c6aGlkZGVufWZpZ3VyZXttYXJnaW46MWVtIDQwcHh9aHJ7aGVpZ2h0OjA7LXdlYmtpdC1ib3gt
c2l6aW5nOmNvbnRlbnQtYm94Oy1tb3otYm94LXNpemluZzpjb250ZW50LWJveDtib3gtc2l6aW5n
OmNvbnRlbnQtYm94fXByZXtvdmVyZmxvdzphdXRvfWNvZGUsa2JkLHByZSxzYW1we2ZvbnQtZmFt
aWx5Om1vbm9zcGFjZSxtb25vc3BhY2U7Zm9udC1zaXplOjFlbX1idXR0b24saW5wdXQsb3B0Z3Jv
dXAsc2VsZWN0LHRleHRhcmVhe21hcmdpbjowO2ZvbnQ6aW5oZXJpdDtjb2xvcjppbmhlcml0fWJ1
dHRvbntvdmVyZmxvdzp2aXNpYmxlfWJ1dHRvbixzZWxlY3R7dGV4dC10cmFuc2Zvcm06bm9uZX1i
dXR0b24saHRtbCBpbnB1dFt0eXBlPWJ1dHRvbl0saW5wdXRbdHlwZT1yZXNldF0saW5wdXRbdHlw
ZT1zdWJtaXRdey13ZWJraXQtYXBwZWFyYW5jZTpidXR0b247Y3Vyc29yOnBvaW50ZXJ9YnV0dG9u
W2Rpc2FibGVkXSxodG1sIGlucHV0W2Rpc2FibGVkXXtjdXJzb3I6ZGVmYXVsdH1idXR0b246Oi1t
b3otZm9jdXMtaW5uZXIsaW5wdXQ6Oi1tb3otZm9jdXMtaW5uZXJ7cGFkZGluZzowO2JvcmRlcjow
fWlucHV0e2xpbmUtaGVpZ2h0Om5vcm1hbH1pbnB1dFt0eXBlPWNoZWNrYm94XSxpbnB1dFt0eXBl
PXJhZGlvXXstd2Via2l0LWJveC1zaXppbmc6Ym9yZGVyLWJveDstbW96LWJveC1zaXppbmc6Ym9y
ZGVyLWJveDtib3gtc2l6aW5nOmJvcmRlci1ib3g7cGFkZGluZzowfWlucHV0W3R5cGU9bnVtYmVy
XTo6LXdlYmtpdC1pbm5lci1zcGluLWJ1dHRvbixpbnB1dFt0eXBlPW51bWJlcl06Oi13ZWJraXQt
b3V0ZXItc3Bpbi1idXR0b257aGVpZ2h0OmF1dG99aW5wdXRbdHlwZT1zZWFyY2hdey13ZWJraXQt
Ym94LXNpemluZzpjb250ZW50LWJveDstbW96LWJveC1zaXppbmc6Y29udGVudC1ib3g7Ym94LXNp
emluZzpjb250ZW50LWJveDstd2Via2l0LWFwcGVhcmFuY2U6dGV4dGZpZWxkfWlucHV0W3R5cGU9
c2VhcmNoXTo6LXdlYmtpdC1zZWFyY2gtY2FuY2VsLWJ1dHRvbixpbnB1dFt0eXBlPXNlYXJjaF06
Oi13ZWJraXQtc2VhcmNoLWRlY29yYXRpb257LXdlYmtpdC1hcHBlYXJhbmNlOm5vbmV9ZmllbGRz
ZXR7cGFkZGluZzouMzVlbSAuNjI1ZW0gLjc1ZW07bWFyZ2luOjAgMnB4O2JvcmRlcjoxcHggc29s
aWQgc2lsdmVyfWxlZ2VuZHtwYWRkaW5nOjA7Ym9yZGVyOjB9dGV4dGFyZWF7b3ZlcmZsb3c6YXV0
b31vcHRncm91cHtmb250LXdlaWdodDo3MDB9dGFibGV7Ym9yZGVyLXNwYWNpbmc6MDtib3JkZXIt
Y29sbGFwc2U6Y29sbGFwc2V9dGQsdGh7cGFkZGluZzowfS8qISBTb3VyY2U6IGh0dHBzOi8vZ2l0
aHViLmNvbS9oNWJwL2h0bWw1LWJvaWxlcnBsYXRlL2Jsb2IvbWFzdGVyL3NyYy9jc3MvbWFpbi5j
c3MgKi9AbWVkaWEgcHJpbnR7Kiw6YWZ0ZXIsOmJlZm9yZXtjb2xvcjojMDAwIWltcG9ydGFudDt0
ZXh0LXNoYWRvdzpub25lIWltcG9ydGFudDtiYWNrZ3JvdW5kOjAgMCFpbXBvcnRhbnQ7LXdlYmtp
dC1ib3gtc2hhZG93Om5vbmUhaW1wb3J0YW50O2JveC1zaGFkb3c6bm9uZSFpbXBvcnRhbnR9YSxh
OnZpc2l0ZWR7dGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZX1hW2hyZWZdOmFmdGVye2NvbnRlbnQ6
IiAoIiBhdHRyKGhyZWYpICIpIn1hYmJyW3RpdGxlXTphZnRlcntjb250ZW50OiIgKCIgYXR0cih0
aXRsZSkgIikifWFbaHJlZl49ImphdmFzY3JpcHQ6Il06YWZ0ZXIsYVtocmVmXj0iIyJdOmFmdGVy
e2NvbnRlbnQ6IiJ9YmxvY2txdW90ZSxwcmV7Ym9yZGVyOjFweCBzb2xpZCAjOTk5O3BhZ2UtYnJl
YWstaW5zaWRlOmF2b2lkfXRoZWFke2Rpc3BsYXk6dGFibGUtaGVhZGVyLWdyb3VwfWltZyx0cntw
YWdlLWJyZWFrLWluc2lkZTphdm9pZH1pbWd7bWF4LXdpZHRoOjEwMCUhaW1wb3J0YW50fWgyLGgz
LHB7b3JwaGFuczozO3dpZG93czozfWgyLGgze3BhZ2UtYnJlYWstYWZ0ZXI6YXZvaWR9Lm5hdmJh
cntkaXNwbGF5Om5vbmV9LmJ0bj4uY2FyZXQsLmRyb3B1cD4uYnRuPi5jYXJldHtib3JkZXItdG9w
LWNvbG9yOiMwMDAhaW1wb3J0YW50fS5sYWJlbHtib3JkZXI6MXB4IHNvbGlkICMwMDB9LnRhYmxl
e2JvcmRlci1jb2xsYXBzZTpjb2xsYXBzZSFpbXBvcnRhbnR9LnRhYmxlIHRkLC50YWJsZSB0aHti
YWNrZ3JvdW5kLWNvbG9yOiNmZmYhaW1wb3J0YW50fS50YWJsZS1ib3JkZXJlZCB0ZCwudGFibGUt
Ym9yZGVyZWQgdGh7Ym9yZGVyOjFweCBzb2xpZCAjZGRkIWltcG9ydGFudH19QGZvbnQtZmFjZXtm
b250LWZhbWlseTonR2x5cGhpY29ucyBIYWxmbGluZ3MnO3NyYzp1cmwoLi4vZm9udHMvZ2x5cGhp
Y29ucy1oYWxmbGluZ3MtcmVndWxhci5lb3QpO3NyYzp1cmwoLi4vZm9udHMvZ2x5cGhpY29ucy1o
YWxmbGluZ3MtcmVndWxhci5lb3Q/I2llZml4KSBmb3JtYXQoJ2VtYmVkZGVkLW9wZW50eXBlJyks
dXJsKC4uL2ZvbnRzL2dseXBoaWNvbnMtaGFsZmxpbmdzLXJlZ3VsYXIud29mZjIpIGZvcm1hdCgn
d29mZjInKSx1cmwoLi4vZm9udHMvZ2x5cGhpY29ucy1oYWxmbGluZ3MtcmVndWxhci53b2ZmKSBm
b3JtYXQoJ3dvZmYnKSx1cmwoLi4vZm9udHMvZ2x5cGhpY29ucy1oYWxmbGluZ3MtcmVndWxhci50
dGYpIGZvcm1hdCgndHJ1ZXR5cGUnKSx1cmwoLi4vZm9udHMvZ2x5cGhpY29ucy1oYWxmbGluZ3Mt
cmVndWxhci5zdmcjZ2x5cGhpY29uc19oYWxmbGluZ3NyZWd1bGFyKSBmb3JtYXQoJ3N2ZycpfS5n
bHlwaGljb257cG9zaXRpb246cmVsYXRpdmU7dG9wOjFweDtkaXNwbGF5OmlubGluZS1ibG9jaztm
b250LWZhbWlseTonR2x5cGhpY29ucyBIYWxmbGluZ3MnO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQt
d2VpZ2h0OjQwMDtsaW5lLWhlaWdodDoxOy13ZWJraXQtZm9udC1zbW9vdGhpbmc6YW50aWFsaWFz
ZWQ7LW1vei1vc3gtZm9udC1zbW9vdGhpbmc6Z3JheXNjYWxlfS5nbHlwaGljb24tYXN0ZXJpc2s6
YmVmb3Jle2NvbnRlbnQ6IlwwMDJhIn0uZ2x5cGhpY29uLXBsdXM6YmVmb3Jle2NvbnRlbnQ6Ilww
MDJiIn0uZ2x5cGhpY29uLWV1cjpiZWZvcmUsLmdseXBoaWNvbi1ldXJvOmJlZm9yZXtjb250ZW50
OiJcMjBhYyJ9LmdseXBoaWNvbi1taW51czpiZWZvcmV7Y29udGVudDoiXDIyMTIifS5nbHlwaGlj
b24tY2xvdWQ6YmVmb3Jle2NvbnRlbnQ6IlwyNjAxIn0uZ2x5cGhpY29uLWVudmVsb3BlOmJlZm9y
ZXtjb250ZW50OiJcMjcwOSJ9LmdseXBoaWNvbi1wZW5jaWw6YmVmb3Jle2NvbnRlbnQ6IlwyNzBm
In0uZ2x5cGhpY29uLWdsYXNzOmJlZm9yZXtjb250ZW50OiJcZTAwMSJ9LmdseXBoaWNvbi1tdXNp
YzpiZWZvcmV7Y29udGVudDoiXGUwMDIifS5nbHlwaGljb24tc2VhcmNoOmJlZm9yZXtjb250ZW50
OiJcZTAwMyJ9LmdseXBoaWNvbi1oZWFydDpiZWZvcmV7Y29udGVudDoiXGUwMDUifS5nbHlwaGlj
b24tc3RhcjpiZWZvcmV7Y29udGVudDoiXGUwMDYifS5nbHlwaGljb24tc3Rhci1lbXB0eTpiZWZv
cmV7Y29udGVudDoiXGUwMDcifS5nbHlwaGljb24tdXNlcjpiZWZvcmV7Y29udGVudDoiXGUwMDgi
fS5nbHlwaGljb24tZmlsbTpiZWZvcmV7Y29udGVudDoiXGUwMDkifS5nbHlwaGljb24tdGgtbGFy
Z2U6YmVmb3Jle2NvbnRlbnQ6IlxlMDEwIn0uZ2x5cGhpY29uLXRoOmJlZm9yZXtjb250ZW50OiJc
ZTAxMSJ9LmdseXBoaWNvbi10aC1saXN0OmJlZm9yZXtjb250ZW50OiJcZTAxMiJ9LmdseXBoaWNv
bi1vazpiZWZvcmV7Y29udGVudDoiXGUwMTMifS5nbHlwaGljb24tcmVtb3ZlOmJlZm9yZXtjb250
ZW50OiJcZTAxNCJ9LmdseXBoaWNvbi16b29tLWluOmJlZm9yZXtjb250ZW50OiJcZTAxNSJ9Lmds
eXBoaWNvbi16b29tLW91dDpiZWZvcmV7Y29udGVudDoiXGUwMTYifS5nbHlwaGljb24tb2ZmOmJl
Zm9yZXtjb250ZW50OiJcZTAxNyJ9LmdseXBoaWNvbi1zaWduYWw6YmVmb3Jle2NvbnRlbnQ6Ilxl
MDE4In0uZ2x5cGhpY29uLWNvZzpiZWZvcmV7Y29udGVudDoiXGUwMTkifS5nbHlwaGljb24tdHJh
c2g6YmVmb3Jle2NvbnRlbnQ6IlxlMDIwIn0uZ2x5cGhpY29uLWhvbWU6YmVmb3Jle2NvbnRlbnQ6
IlxlMDIxIn0uZ2x5cGhpY29uLWZpbGU6YmVmb3Jle2NvbnRlbnQ6IlxlMDIyIn0uZ2x5cGhpY29u
LXRpbWU6YmVmb3Jle2NvbnRlbnQ6IlxlMDIzIn0uZ2x5cGhpY29uLXJvYWQ6YmVmb3Jle2NvbnRl
bnQ6IlxlMDI0In0uZ2x5cGhpY29uLWRvd25sb2FkLWFsdDpiZWZvcmV7Y29udGVudDoiXGUwMjUi
fS5nbHlwaGljb24tZG93bmxvYWQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDI2In0uZ2x5cGhpY29uLXVw
bG9hZDpiZWZvcmV7Y29udGVudDoiXGUwMjcifS5nbHlwaGljb24taW5ib3g6YmVmb3Jle2NvbnRl
bnQ6IlxlMDI4In0uZ2x5cGhpY29uLXBsYXktY2lyY2xlOmJlZm9yZXtjb250ZW50OiJcZTAyOSJ9
LmdseXBoaWNvbi1yZXBlYXQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDMwIn0uZ2x5cGhpY29uLXJlZnJl
c2g6YmVmb3Jle2NvbnRlbnQ6IlxlMDMxIn0uZ2x5cGhpY29uLWxpc3QtYWx0OmJlZm9yZXtjb250
ZW50OiJcZTAzMiJ9LmdseXBoaWNvbi1sb2NrOmJlZm9yZXtjb250ZW50OiJcZTAzMyJ9LmdseXBo
aWNvbi1mbGFnOmJlZm9yZXtjb250ZW50OiJcZTAzNCJ9LmdseXBoaWNvbi1oZWFkcGhvbmVzOmJl
Zm9yZXtjb250ZW50OiJcZTAzNSJ9LmdseXBoaWNvbi12b2x1bWUtb2ZmOmJlZm9yZXtjb250ZW50
OiJcZTAzNiJ9LmdseXBoaWNvbi12b2x1bWUtZG93bjpiZWZvcmV7Y29udGVudDoiXGUwMzcifS5n
bHlwaGljb24tdm9sdW1lLXVwOmJlZm9yZXtjb250ZW50OiJcZTAzOCJ9LmdseXBoaWNvbi1xcmNv
ZGU6YmVmb3Jle2NvbnRlbnQ6IlxlMDM5In0uZ2x5cGhpY29uLWJhcmNvZGU6YmVmb3Jle2NvbnRl
bnQ6IlxlMDQwIn0uZ2x5cGhpY29uLXRhZzpiZWZvcmV7Y29udGVudDoiXGUwNDEifS5nbHlwaGlj
b24tdGFnczpiZWZvcmV7Y29udGVudDoiXGUwNDIifS5nbHlwaGljb24tYm9vazpiZWZvcmV7Y29u
dGVudDoiXGUwNDMifS5nbHlwaGljb24tYm9va21hcms6YmVmb3Jle2NvbnRlbnQ6IlxlMDQ0In0u
Z2x5cGhpY29uLXByaW50OmJlZm9yZXtjb250ZW50OiJcZTA0NSJ9LmdseXBoaWNvbi1jYW1lcmE6
YmVmb3Jle2NvbnRlbnQ6IlxlMDQ2In0uZ2x5cGhpY29uLWZvbnQ6YmVmb3Jle2NvbnRlbnQ6Ilxl
MDQ3In0uZ2x5cGhpY29uLWJvbGQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDQ4In0uZ2x5cGhpY29uLWl0
YWxpYzpiZWZvcmV7Y29udGVudDoiXGUwNDkifS5nbHlwaGljb24tdGV4dC1oZWlnaHQ6YmVmb3Jl
e2NvbnRlbnQ6IlxlMDUwIn0uZ2x5cGhpY29uLXRleHQtd2lkdGg6YmVmb3Jle2NvbnRlbnQ6Ilxl
MDUxIn0uZ2x5cGhpY29uLWFsaWduLWxlZnQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDUyIn0uZ2x5cGhp
Y29uLWFsaWduLWNlbnRlcjpiZWZvcmV7Y29udGVudDoiXGUwNTMifS5nbHlwaGljb24tYWxpZ24t
cmlnaHQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDU0In0uZ2x5cGhpY29uLWFsaWduLWp1c3RpZnk6YmVm
b3Jle2NvbnRlbnQ6IlxlMDU1In0uZ2x5cGhpY29uLWxpc3Q6YmVmb3Jle2NvbnRlbnQ6IlxlMDU2
In0uZ2x5cGhpY29uLWluZGVudC1sZWZ0OmJlZm9yZXtjb250ZW50OiJcZTA1NyJ9LmdseXBoaWNv
bi1pbmRlbnQtcmlnaHQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDU4In0uZ2x5cGhpY29uLWZhY2V0aW1l
LXZpZGVvOmJlZm9yZXtjb250ZW50OiJcZTA1OSJ9LmdseXBoaWNvbi1waWN0dXJlOmJlZm9yZXtj
b250ZW50OiJcZTA2MCJ9LmdseXBoaWNvbi1tYXAtbWFya2VyOmJlZm9yZXtjb250ZW50OiJcZTA2
MiJ9LmdseXBoaWNvbi1hZGp1c3Q6YmVmb3Jle2NvbnRlbnQ6IlxlMDYzIn0uZ2x5cGhpY29uLXRp
bnQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDY0In0uZ2x5cGhpY29uLWVkaXQ6YmVmb3Jle2NvbnRlbnQ6
IlxlMDY1In0uZ2x5cGhpY29uLXNoYXJlOmJlZm9yZXtjb250ZW50OiJcZTA2NiJ9LmdseXBoaWNv
bi1jaGVjazpiZWZvcmV7Y29udGVudDoiXGUwNjcifS5nbHlwaGljb24tbW92ZTpiZWZvcmV7Y29u
dGVudDoiXGUwNjgifS5nbHlwaGljb24tc3RlcC1iYWNrd2FyZDpiZWZvcmV7Y29udGVudDoiXGUw
NjkifS5nbHlwaGljb24tZmFzdC1iYWNrd2FyZDpiZWZvcmV7Y29udGVudDoiXGUwNzAifS5nbHlw
aGljb24tYmFja3dhcmQ6YmVmb3Jle2NvbnRlbnQ6IlxlMDcxIn0uZ2x5cGhpY29uLXBsYXk6YmVm
b3Jle2NvbnRlbnQ6IlxlMDcyIn0uZ2x5cGhpY29uLXBhdXNlOmJlZm9yZXtjb250ZW50OiJcZTA3
MyJ9LmdseXBoaWNvbi1zdG9wOmJlZm9yZXtjb250ZW50OiJcZTA3NCJ9LmdseXBoaWNvbi1mb3J3
YXJkOmJlZm9yZXtjb250ZW50OiJcZTA3NSJ9LmdseXBoaWNvbi1mYXN0LWZvcndhcmQ6YmVmb3Jl
e2NvbnRlbnQ6IlxlMDc2In0uZ2x5cGhpY29uLXN0ZXAtZm9yd2FyZDpiZWZvcmV7Y29udGVudDoi
XGUwNzcifS5nbHlwaGljb24tZWplY3Q6YmVmb3Jle2NvbnRlbnQ6IlxlMDc4In0uZ2x5cGhpY29u
LWNoZXZyb24tbGVmdDpiZWZvcmV7Y29udGVudDoiXGUwNzkifS5nbHlwaGljb24tY2hldnJvbi1y
aWdodDpiZWZvcmV7Y29udGVudDoiXGUwODAifS5nbHlwaGljb24tcGx1cy1zaWduOmJlZm9yZXtj
b250ZW50OiJcZTA4MSJ9LmdseXBoaWNvbi1taW51cy1zaWduOmJlZm9yZXtjb250ZW50OiJcZTA4
MiJ9LmdseXBoaWNvbi1yZW1vdmUtc2lnbjpiZWZvcmV7Y29udGVudDoiXGUwODMifS5nbHlwaGlj
b24tb2stc2lnbjpiZWZvcmV7Y29udGVudDoiXGUwODQifS5nbHlwaGljb24tcXVlc3Rpb24tc2ln
bjpiZWZvcmV7Y29udGVudDoiXGUwODUifS5nbHlwaGljb24taW5mby1zaWduOmJlZm9yZXtjb250
ZW50OiJcZTA4NiJ9LmdseXBoaWNvbi1zY3JlZW5zaG90OmJlZm9yZXtjb250ZW50OiJcZTA4NyJ9
LmdseXBoaWNvbi1yZW1vdmUtY2lyY2xlOmJlZm9yZXtjb250ZW50OiJcZTA4OCJ9LmdseXBoaWNv
bi1vay1jaXJjbGU6YmVmb3Jle2NvbnRlbnQ6IlxlMDg5In0uZ2x5cGhpY29uLWJhbi1jaXJjbGU6
YmVmb3Jle2NvbnRlbnQ6IlxlMDkwIn0uZ2x5cGhpY29uLWFycm93LWxlZnQ6YmVmb3Jle2NvbnRl
bnQ6IlxlMDkxIn0uZ2x5cGhpY29uLWFycm93LXJpZ2h0OmJlZm9yZXtjb250ZW50OiJcZTA5MiJ9
LmdseXBoaWNvbi1hcnJvdy11cDpiZWZvcmV7Y29udGVudDoiXGUwOTMifS5nbHlwaGljb24tYXJy
b3ctZG93bjpiZWZvcmV7Y29udGVudDoiXGUwOTQifS5nbHlwaGljb24tc2hhcmUtYWx0OmJlZm9y
ZXtjb250ZW50OiJcZTA5NSJ9LmdseXBoaWNvbi1yZXNpemUtZnVsbDpiZWZvcmV7Y29udGVudDoi
XGUwOTYifS5nbHlwaGljb24tcmVzaXplLXNtYWxsOmJlZm9yZXtjb250ZW50OiJcZTA5NyJ9Lmds
eXBoaWNvbi1leGNsYW1hdGlvbi1zaWduOmJlZm9yZXtjb250ZW50OiJcZTEwMSJ9LmdseXBoaWNv
bi1naWZ0OmJlZm9yZXtjb250ZW50OiJcZTEwMiJ9LmdseXBoaWNvbi1sZWFmOmJlZm9yZXtjb250
ZW50OiJcZTEwMyJ9LmdseXBoaWNvbi1maXJlOmJlZm9yZXtjb250ZW50OiJcZTEwNCJ9LmdseXBo
aWNvbi1leWUtb3BlbjpiZWZvcmV7Y29udGVudDoiXGUxMDUifS5nbHlwaGljb24tZXllLWNsb3Nl
OmJlZm9yZXtjb250ZW50OiJcZTEwNiJ9LmdseXBoaWNvbi13YXJuaW5nLXNpZ246YmVmb3Jle2Nv
bnRlbnQ6IlxlMTA3In0uZ2x5cGhpY29uLXBsYW5lOmJlZm9yZXtjb250ZW50OiJcZTEwOCJ9Lmds
eXBoaWNvbi1jYWxlbmRhcjpiZWZvcmV7Y29udGVudDoiXGUxMDkifS5nbHlwaGljb24tcmFuZG9t
OmJlZm9yZXtjb250ZW50OiJcZTExMCJ9LmdseXBoaWNvbi1jb21tZW50OmJlZm9yZXtjb250ZW50
OiJcZTExMSJ9LmdseXBoaWNvbi1tYWduZXQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTEyIn0uZ2x5cGhp
Y29uLWNoZXZyb24tdXA6YmVmb3Jle2NvbnRlbnQ6IlxlMTEzIn0uZ2x5cGhpY29uLWNoZXZyb24t
ZG93bjpiZWZvcmV7Y29udGVudDoiXGUxMTQifS5nbHlwaGljb24tcmV0d2VldDpiZWZvcmV7Y29u
dGVudDoiXGUxMTUifS5nbHlwaGljb24tc2hvcHBpbmctY2FydDpiZWZvcmV7Y29udGVudDoiXGUx
MTYifS5nbHlwaGljb24tZm9sZGVyLWNsb3NlOmJlZm9yZXtjb250ZW50OiJcZTExNyJ9LmdseXBo
aWNvbi1mb2xkZXItb3BlbjpiZWZvcmV7Y29udGVudDoiXGUxMTgifS5nbHlwaGljb24tcmVzaXpl
LXZlcnRpY2FsOmJlZm9yZXtjb250ZW50OiJcZTExOSJ9LmdseXBoaWNvbi1yZXNpemUtaG9yaXpv
bnRhbDpiZWZvcmV7Y29udGVudDoiXGUxMjAifS5nbHlwaGljb24taGRkOmJlZm9yZXtjb250ZW50
OiJcZTEyMSJ9LmdseXBoaWNvbi1idWxsaG9ybjpiZWZvcmV7Y29udGVudDoiXGUxMjIifS5nbHlw
aGljb24tYmVsbDpiZWZvcmV7Y29udGVudDoiXGUxMjMifS5nbHlwaGljb24tY2VydGlmaWNhdGU6
YmVmb3Jle2NvbnRlbnQ6IlxlMTI0In0uZ2x5cGhpY29uLXRodW1icy11cDpiZWZvcmV7Y29udGVu
dDoiXGUxMjUifS5nbHlwaGljb24tdGh1bWJzLWRvd246YmVmb3Jle2NvbnRlbnQ6IlxlMTI2In0u
Z2x5cGhpY29uLWhhbmQtcmlnaHQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTI3In0uZ2x5cGhpY29uLWhh
bmQtbGVmdDpiZWZvcmV7Y29udGVudDoiXGUxMjgifS5nbHlwaGljb24taGFuZC11cDpiZWZvcmV7
Y29udGVudDoiXGUxMjkifS5nbHlwaGljb24taGFuZC1kb3duOmJlZm9yZXtjb250ZW50OiJcZTEz
MCJ9LmdseXBoaWNvbi1jaXJjbGUtYXJyb3ctcmlnaHQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTMxIn0u
Z2x5cGhpY29uLWNpcmNsZS1hcnJvdy1sZWZ0OmJlZm9yZXtjb250ZW50OiJcZTEzMiJ9LmdseXBo
aWNvbi1jaXJjbGUtYXJyb3ctdXA6YmVmb3Jle2NvbnRlbnQ6IlxlMTMzIn0uZ2x5cGhpY29uLWNp
cmNsZS1hcnJvdy1kb3duOmJlZm9yZXtjb250ZW50OiJcZTEzNCJ9LmdseXBoaWNvbi1nbG9iZTpi
ZWZvcmV7Y29udGVudDoiXGUxMzUifS5nbHlwaGljb24td3JlbmNoOmJlZm9yZXtjb250ZW50OiJc
ZTEzNiJ9LmdseXBoaWNvbi10YXNrczpiZWZvcmV7Y29udGVudDoiXGUxMzcifS5nbHlwaGljb24t
ZmlsdGVyOmJlZm9yZXtjb250ZW50OiJcZTEzOCJ9LmdseXBoaWNvbi1icmllZmNhc2U6YmVmb3Jl
e2NvbnRlbnQ6IlxlMTM5In0uZ2x5cGhpY29uLWZ1bGxzY3JlZW46YmVmb3Jle2NvbnRlbnQ6Ilxl
MTQwIn0uZ2x5cGhpY29uLWRhc2hib2FyZDpiZWZvcmV7Y29udGVudDoiXGUxNDEifS5nbHlwaGlj
b24tcGFwZXJjbGlwOmJlZm9yZXtjb250ZW50OiJcZTE0MiJ9LmdseXBoaWNvbi1oZWFydC1lbXB0
eTpiZWZvcmV7Y29udGVudDoiXGUxNDMifS5nbHlwaGljb24tbGluazpiZWZvcmV7Y29udGVudDoi
XGUxNDQifS5nbHlwaGljb24tcGhvbmU6YmVmb3Jle2NvbnRlbnQ6IlxlMTQ1In0uZ2x5cGhpY29u
LXB1c2hwaW46YmVmb3Jle2NvbnRlbnQ6IlxlMTQ2In0uZ2x5cGhpY29uLXVzZDpiZWZvcmV7Y29u
dGVudDoiXGUxNDgifS5nbHlwaGljb24tZ2JwOmJlZm9yZXtjb250ZW50OiJcZTE0OSJ9LmdseXBo
aWNvbi1zb3J0OmJlZm9yZXtjb250ZW50OiJcZTE1MCJ9LmdseXBoaWNvbi1zb3J0LWJ5LWFscGhh
YmV0OmJlZm9yZXtjb250ZW50OiJcZTE1MSJ9LmdseXBoaWNvbi1zb3J0LWJ5LWFscGhhYmV0LWFs
dDpiZWZvcmV7Y29udGVudDoiXGUxNTIifS5nbHlwaGljb24tc29ydC1ieS1vcmRlcjpiZWZvcmV7
Y29udGVudDoiXGUxNTMifS5nbHlwaGljb24tc29ydC1ieS1vcmRlci1hbHQ6YmVmb3Jle2NvbnRl
bnQ6IlxlMTU0In0uZ2x5cGhpY29uLXNvcnQtYnktYXR0cmlidXRlczpiZWZvcmV7Y29udGVudDoi
XGUxNTUifS5nbHlwaGljb24tc29ydC1ieS1hdHRyaWJ1dGVzLWFsdDpiZWZvcmV7Y29udGVudDoi
XGUxNTYifS5nbHlwaGljb24tdW5jaGVja2VkOmJlZm9yZXtjb250ZW50OiJcZTE1NyJ9LmdseXBo
aWNvbi1leHBhbmQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTU4In0uZ2x5cGhpY29uLWNvbGxhcHNlLWRv
d246YmVmb3Jle2NvbnRlbnQ6IlxlMTU5In0uZ2x5cGhpY29uLWNvbGxhcHNlLXVwOmJlZm9yZXtj
b250ZW50OiJcZTE2MCJ9LmdseXBoaWNvbi1sb2ctaW46YmVmb3Jle2NvbnRlbnQ6IlxlMTYxIn0u
Z2x5cGhpY29uLWZsYXNoOmJlZm9yZXtjb250ZW50OiJcZTE2MiJ9LmdseXBoaWNvbi1sb2ctb3V0
OmJlZm9yZXtjb250ZW50OiJcZTE2MyJ9LmdseXBoaWNvbi1uZXctd2luZG93OmJlZm9yZXtjb250
ZW50OiJcZTE2NCJ9LmdseXBoaWNvbi1yZWNvcmQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTY1In0uZ2x5
cGhpY29uLXNhdmU6YmVmb3Jle2NvbnRlbnQ6IlxlMTY2In0uZ2x5cGhpY29uLW9wZW46YmVmb3Jl
e2NvbnRlbnQ6IlxlMTY3In0uZ2x5cGhpY29uLXNhdmVkOmJlZm9yZXtjb250ZW50OiJcZTE2OCJ9
LmdseXBoaWNvbi1pbXBvcnQ6YmVmb3Jle2NvbnRlbnQ6IlxlMTY5In0uZ2x5cGhpY29uLWV4cG9y
dDpiZWZvcmV7Y29udGVudDoiXGUxNzAifS5nbHlwaGljb24tc2VuZDpiZWZvcmV7Y29udGVudDoi
XGUxNzEifS5nbHlwaGljb24tZmxvcHB5LWRpc2s6YmVmb3Jle2NvbnRlbnQ6IlxlMTcyIn0uZ2x5
cGhpY29uLWZsb3BweS1zYXZlZDpiZWZvcmV7Y29udGVudDoiXGUxNzMifS5nbHlwaGljb24tZmxv
cHB5LXJlbW92ZTpiZWZvcmV7Y29udGVudDoiXGUxNzQifS5nbHlwaGljb24tZmxvcHB5LXNhdmU6
YmVmb3Jle2NvbnRlbnQ6IlxlMTc1In0uZ2x5cGhpY29uLWZsb3BweS1vcGVuOmJlZm9yZXtjb250
ZW50OiJcZTE3NiJ9LmdseXBoaWNvbi1jcmVkaXQtY2FyZDpiZWZvcmV7Y29udGVudDoiXGUxNzci
fS5nbHlwaGljb24tdHJhbnNmZXI6YmVmb3Jle2NvbnRlbnQ6IlxlMTc4In0uZ2x5cGhpY29uLWN1
dGxlcnk6YmVmb3Jle2NvbnRlbnQ6IlxlMTc5In0uZ2x5cGhpY29uLWhlYWRlcjpiZWZvcmV7Y29u
dGVudDoiXGUxODAifS5nbHlwaGljb24tY29tcHJlc3NlZDpiZWZvcmV7Y29udGVudDoiXGUxODEi
fS5nbHlwaGljb24tZWFycGhvbmU6YmVmb3Jle2NvbnRlbnQ6IlxlMTgyIn0uZ2x5cGhpY29uLXBo
b25lLWFsdDpiZWZvcmV7Y29udGVudDoiXGUxODMifS5nbHlwaGljb24tdG93ZXI6YmVmb3Jle2Nv
bnRlbnQ6IlxlMTg0In0uZ2x5cGhpY29uLXN0YXRzOmJlZm9yZXtjb250ZW50OiJcZTE4NSJ9Lmds
eXBoaWNvbi1zZC12aWRlbzpiZWZvcmV7Y29udGVudDoiXGUxODYifS5nbHlwaGljb24taGQtdmlk
ZW86YmVmb3Jle2NvbnRlbnQ6IlxlMTg3In0uZ2x5cGhpY29uLXN1YnRpdGxlczpiZWZvcmV7Y29u
dGVudDoiXGUxODgifS5nbHlwaGljb24tc291bmQtc3RlcmVvOmJlZm9yZXtjb250ZW50OiJcZTE4
OSJ9LmdseXBoaWNvbi1zb3VuZC1kb2xieTpiZWZvcmV7Y29udGVudDoiXGUxOTAifS5nbHlwaGlj
b24tc291bmQtNS0xOmJlZm9yZXtjb250ZW50OiJcZTE5MSJ9LmdseXBoaWNvbi1zb3VuZC02LTE6
YmVmb3Jle2NvbnRlbnQ6IlxlMTkyIn0uZ2x5cGhpY29uLXNvdW5kLTctMTpiZWZvcmV7Y29udGVu
dDoiXGUxOTMifS5nbHlwaGljb24tY29weXJpZ2h0LW1hcms6YmVmb3Jle2NvbnRlbnQ6IlxlMTk0
In0uZ2x5cGhpY29uLXJlZ2lzdHJhdGlvbi1tYXJrOmJlZm9yZXtjb250ZW50OiJcZTE5NSJ9Lmds
eXBoaWNvbi1jbG91ZC1kb3dubG9hZDpiZWZvcmV7Y29udGVudDoiXGUxOTcifS5nbHlwaGljb24t
Y2xvdWQtdXBsb2FkOmJlZm9yZXtjb250ZW50OiJcZTE5OCJ9LmdseXBoaWNvbi10cmVlLWNvbmlm
ZXI6YmVmb3Jle2NvbnRlbnQ6IlxlMTk5In0uZ2x5cGhpY29uLXRyZWUtZGVjaWR1b3VzOmJlZm9y
ZXtjb250ZW50OiJcZTIwMCJ9LmdseXBoaWNvbi1jZDpiZWZvcmV7Y29udGVudDoiXGUyMDEifS5n
bHlwaGljb24tc2F2ZS1maWxlOmJlZm9yZXtjb250ZW50OiJcZTIwMiJ9LmdseXBoaWNvbi1vcGVu
LWZpbGU6YmVmb3Jle2NvbnRlbnQ6IlxlMjAzIn0uZ2x5cGhpY29uLWxldmVsLXVwOmJlZm9yZXtj
b250ZW50OiJcZTIwNCJ9LmdseXBoaWNvbi1jb3B5OmJlZm9yZXtjb250ZW50OiJcZTIwNSJ9Lmds
eXBoaWNvbi1wYXN0ZTpiZWZvcmV7Y29udGVudDoiXGUyMDYifS5nbHlwaGljb24tYWxlcnQ6YmVm
b3Jle2NvbnRlbnQ6IlxlMjA5In0uZ2x5cGhpY29uLWVxdWFsaXplcjpiZWZvcmV7Y29udGVudDoi
XGUyMTAifS5nbHlwaGljb24ta2luZzpiZWZvcmV7Y29udGVudDoiXGUyMTEifS5nbHlwaGljb24t
cXVlZW46YmVmb3Jle2NvbnRlbnQ6IlxlMjEyIn0uZ2x5cGhpY29uLXBhd246YmVmb3Jle2NvbnRl
bnQ6IlxlMjEzIn0uZ2x5cGhpY29uLWJpc2hvcDpiZWZvcmV7Y29udGVudDoiXGUyMTQifS5nbHlw
aGljb24ta25pZ2h0OmJlZm9yZXtjb250ZW50OiJcZTIxNSJ9LmdseXBoaWNvbi1iYWJ5LWZvcm11
bGE6YmVmb3Jle2NvbnRlbnQ6IlxlMjE2In0uZ2x5cGhpY29uLXRlbnQ6YmVmb3Jle2NvbnRlbnQ6
IlwyNmZhIn0uZ2x5cGhpY29uLWJsYWNrYm9hcmQ6YmVmb3Jle2NvbnRlbnQ6IlxlMjE4In0uZ2x5
cGhpY29uLWJlZDpiZWZvcmV7Y29udGVudDoiXGUyMTkifS5nbHlwaGljb24tYXBwbGU6YmVmb3Jl
e2NvbnRlbnQ6IlxmOGZmIn0uZ2x5cGhpY29uLWVyYXNlOmJlZm9yZXtjb250ZW50OiJcZTIyMSJ9
LmdseXBoaWNvbi1ob3VyZ2xhc3M6YmVmb3Jle2NvbnRlbnQ6IlwyMzFiIn0uZ2x5cGhpY29uLWxh
bXA6YmVmb3Jle2NvbnRlbnQ6IlxlMjIzIn0uZ2x5cGhpY29uLWR1cGxpY2F0ZTpiZWZvcmV7Y29u
dGVudDoiXGUyMjQifS5nbHlwaGljb24tcGlnZ3ktYmFuazpiZWZvcmV7Y29udGVudDoiXGUyMjUi
fS5nbHlwaGljb24tc2Npc3NvcnM6YmVmb3Jle2NvbnRlbnQ6IlxlMjI2In0uZ2x5cGhpY29uLWJp
dGNvaW46YmVmb3Jle2NvbnRlbnQ6IlxlMjI3In0uZ2x5cGhpY29uLWJ0YzpiZWZvcmV7Y29udGVu
dDoiXGUyMjcifS5nbHlwaGljb24teGJ0OmJlZm9yZXtjb250ZW50OiJcZTIyNyJ9LmdseXBoaWNv
bi15ZW46YmVmb3Jle2NvbnRlbnQ6IlwwMGE1In0uZ2x5cGhpY29uLWpweTpiZWZvcmV7Y29udGVu
dDoiXDAwYTUifS5nbHlwaGljb24tcnVibGU6YmVmb3Jle2NvbnRlbnQ6IlwyMGJkIn0uZ2x5cGhp
Y29uLXJ1YjpiZWZvcmV7Y29udGVudDoiXDIwYmQifS5nbHlwaGljb24tc2NhbGU6YmVmb3Jle2Nv
bnRlbnQ6IlxlMjMwIn0uZ2x5cGhpY29uLWljZS1sb2xseTpiZWZvcmV7Y29udGVudDoiXGUyMzEi
fS5nbHlwaGljb24taWNlLWxvbGx5LXRhc3RlZDpiZWZvcmV7Y29udGVudDoiXGUyMzIifS5nbHlw
aGljb24tZWR1Y2F0aW9uOmJlZm9yZXtjb250ZW50OiJcZTIzMyJ9LmdseXBoaWNvbi1vcHRpb24t
aG9yaXpvbnRhbDpiZWZvcmV7Y29udGVudDoiXGUyMzQifS5nbHlwaGljb24tb3B0aW9uLXZlcnRp
Y2FsOmJlZm9yZXtjb250ZW50OiJcZTIzNSJ9LmdseXBoaWNvbi1tZW51LWhhbWJ1cmdlcjpiZWZv
cmV7Y29udGVudDoiXGUyMzYifS5nbHlwaGljb24tbW9kYWwtd2luZG93OmJlZm9yZXtjb250ZW50
OiJcZTIzNyJ9LmdseXBoaWNvbi1vaWw6YmVmb3Jle2NvbnRlbnQ6IlxlMjM4In0uZ2x5cGhpY29u
LWdyYWluOmJlZm9yZXtjb250ZW50OiJcZTIzOSJ9LmdseXBoaWNvbi1zdW5nbGFzc2VzOmJlZm9y
ZXtjb250ZW50OiJcZTI0MCJ9LmdseXBoaWNvbi10ZXh0LXNpemU6YmVmb3Jle2NvbnRlbnQ6Ilxl
MjQxIn0uZ2x5cGhpY29uLXRleHQtY29sb3I6YmVmb3Jle2NvbnRlbnQ6IlxlMjQyIn0uZ2x5cGhp
Y29uLXRleHQtYmFja2dyb3VuZDpiZWZvcmV7Y29udGVudDoiXGUyNDMifS5nbHlwaGljb24tb2Jq
ZWN0LWFsaWduLXRvcDpiZWZvcmV7Y29udGVudDoiXGUyNDQifS5nbHlwaGljb24tb2JqZWN0LWFs
aWduLWJvdHRvbTpiZWZvcmV7Y29udGVudDoiXGUyNDUifS5nbHlwaGljb24tb2JqZWN0LWFsaWdu
LWhvcml6b250YWw6YmVmb3Jle2NvbnRlbnQ6IlxlMjQ2In0uZ2x5cGhpY29uLW9iamVjdC1hbGln
bi1sZWZ0OmJlZm9yZXtjb250ZW50OiJcZTI0NyJ9LmdseXBoaWNvbi1vYmplY3QtYWxpZ24tdmVy
dGljYWw6YmVmb3Jle2NvbnRlbnQ6IlxlMjQ4In0uZ2x5cGhpY29uLW9iamVjdC1hbGlnbi1yaWdo
dDpiZWZvcmV7Y29udGVudDoiXGUyNDkifS5nbHlwaGljb24tdHJpYW5nbGUtcmlnaHQ6YmVmb3Jl
e2NvbnRlbnQ6IlxlMjUwIn0uZ2x5cGhpY29uLXRyaWFuZ2xlLWxlZnQ6YmVmb3Jle2NvbnRlbnQ6
IlxlMjUxIn0uZ2x5cGhpY29uLXRyaWFuZ2xlLWJvdHRvbTpiZWZvcmV7Y29udGVudDoiXGUyNTIi
fS5nbHlwaGljb24tdHJpYW5nbGUtdG9wOmJlZm9yZXtjb250ZW50OiJcZTI1MyJ9LmdseXBoaWNv
bi1jb25zb2xlOmJlZm9yZXtjb250ZW50OiJcZTI1NCJ9LmdseXBoaWNvbi1zdXBlcnNjcmlwdDpi
ZWZvcmV7Y29udGVudDoiXGUyNTUifS5nbHlwaGljb24tc3Vic2NyaXB0OmJlZm9yZXtjb250ZW50
OiJcZTI1NiJ9LmdseXBoaWNvbi1tZW51LWxlZnQ6YmVmb3Jle2NvbnRlbnQ6IlxlMjU3In0uZ2x5
cGhpY29uLW1lbnUtcmlnaHQ6YmVmb3Jle2NvbnRlbnQ6IlxlMjU4In0uZ2x5cGhpY29uLW1lbnUt
ZG93bjpiZWZvcmV7Y29udGVudDoiXGUyNTkifS5nbHlwaGljb24tbWVudS11cDpiZWZvcmV7Y29u
dGVudDoiXGUyNjAifSp7LXdlYmtpdC1ib3gtc2l6aW5nOmJvcmRlci1ib3g7LW1vei1ib3gtc2l6
aW5nOmJvcmRlci1ib3g7Ym94LXNpemluZzpib3JkZXItYm94fTphZnRlciw6YmVmb3Jley13ZWJr
aXQtYm94LXNpemluZzpib3JkZXItYm94Oy1tb3otYm94LXNpemluZzpib3JkZXItYm94O2JveC1z
aXppbmc6Ym9yZGVyLWJveH1odG1se2ZvbnQtc2l6ZToxMHB4Oy13ZWJraXQtdGFwLWhpZ2hsaWdo
dC1jb2xvcjpyZ2JhKDAsMCwwLDApfWJvZHl7Zm9udC1mYW1pbHk6IkhlbHZldGljYSBOZXVlIixI
ZWx2ZXRpY2EsQXJpYWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtsaW5lLWhlaWdodDoxLjQy
ODU3MTQzO2NvbG9yOiMzMzM7YmFja2dyb3VuZC1jb2xvcjojZmZmfWJ1dHRvbixpbnB1dCxzZWxl
Y3QsdGV4dGFyZWF7Zm9udC1mYW1pbHk6aW5oZXJpdDtmb250LXNpemU6aW5oZXJpdDtsaW5lLWhl
aWdodDppbmhlcml0fWF7Y29sb3I6IzMzN2FiNzt0ZXh0LWRlY29yYXRpb246bm9uZX1hOmZvY3Vz
LGE6aG92ZXJ7Y29sb3I6IzIzNTI3Yzt0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lfWE6Zm9jdXN7
b3V0bGluZTo1cHggYXV0byAtd2Via2l0LWZvY3VzLXJpbmctY29sb3I7b3V0bGluZS1vZmZzZXQ6
LTJweH1maWd1cmV7bWFyZ2luOjB9aW1ne3ZlcnRpY2FsLWFsaWduOm1pZGRsZX0uY2Fyb3VzZWwt
aW5uZXI+Lml0ZW0+YT5pbWcsLmNhcm91c2VsLWlubmVyPi5pdGVtPmltZywuaW1nLXJlc3BvbnNp
dmUsLnRodW1ibmFpbCBhPmltZywudGh1bWJuYWlsPmltZ3tkaXNwbGF5OmJsb2NrO21heC13aWR0
aDoxMDAlO2hlaWdodDphdXRvfS5pbWctcm91bmRlZHtib3JkZXItcmFkaXVzOjZweH0uaW1nLXRo
dW1ibmFpbHtkaXNwbGF5OmlubGluZS1ibG9jazttYXgtd2lkdGg6MTAwJTtoZWlnaHQ6YXV0bztw
YWRkaW5nOjRweDtsaW5lLWhlaWdodDoxLjQyODU3MTQzO2JhY2tncm91bmQtY29sb3I6I2ZmZjti
b3JkZXI6MXB4IHNvbGlkICNkZGQ7Ym9yZGVyLXJhZGl1czo0cHg7LXdlYmtpdC10cmFuc2l0aW9u
OmFsbCAuMnMgZWFzZS1pbi1vdXQ7LW8tdHJhbnNpdGlvbjphbGwgLjJzIGVhc2UtaW4tb3V0O3Ry
YW5zaXRpb246YWxsIC4ycyBlYXNlLWluLW91dH0uaW1nLWNpcmNsZXtib3JkZXItcmFkaXVzOjUw
JX1ocnttYXJnaW4tdG9wOjIwcHg7bWFyZ2luLWJvdHRvbToyMHB4O2JvcmRlcjowO2JvcmRlci10
b3A6MXB4IHNvbGlkICNlZWV9LnNyLW9ubHl7cG9zaXRpb246YWJzb2x1dGU7d2lkdGg6MXB4O2hl
aWdodDoxcHg7cGFkZGluZzowO21hcmdpbjotMXB4O292ZXJmbG93OmhpZGRlbjtjbGlwOnJlY3Qo
MCwwLDAsMCk7Ym9yZGVyOjB9LnNyLW9ubHktZm9jdXNhYmxlOmFjdGl2ZSwuc3Itb25seS1mb2N1
c2FibGU6Zm9jdXN7cG9zaXRpb246c3RhdGljO3dpZHRoOmF1dG87aGVpZ2h0OmF1dG87bWFyZ2lu
OjA7b3ZlcmZsb3c6dmlzaWJsZTtjbGlwOmF1dG99W3JvbGU9YnV0dG9uXXtjdXJzb3I6cG9pbnRl
cn0uaDEsLmgyLC5oMywuaDQsLmg1LC5oNixoMSxoMixoMyxoNCxoNSxoNntmb250LWZhbWlseTpp
bmhlcml0O2ZvbnQtd2VpZ2h0OjUwMDtsaW5lLWhlaWdodDoxLjE7Y29sb3I6aW5oZXJpdH0uaDEg
LnNtYWxsLC5oMSBzbWFsbCwuaDIgLnNtYWxsLC5oMiBzbWFsbCwuaDMgLnNtYWxsLC5oMyBzbWFs
bCwuaDQgLnNtYWxsLC5oNCBzbWFsbCwuaDUgLnNtYWxsLC5oNSBzbWFsbCwuaDYgLnNtYWxsLC5o
NiBzbWFsbCxoMSAuc21hbGwsaDEgc21hbGwsaDIgLnNtYWxsLGgyIHNtYWxsLGgzIC5zbWFsbCxo
MyBzbWFsbCxoNCAuc21hbGwsaDQgc21hbGwsaDUgLnNtYWxsLGg1IHNtYWxsLGg2IC5zbWFsbCxo
NiBzbWFsbHtmb250LXdlaWdodDo0MDA7bGluZS1oZWlnaHQ6MTtjb2xvcjojNzc3fS5oMSwuaDIs
LmgzLGgxLGgyLGgze21hcmdpbi10b3A6MjBweDttYXJnaW4tYm90dG9tOjEwcHh9LmgxIC5zbWFs
bCwuaDEgc21hbGwsLmgyIC5zbWFsbCwuaDIgc21hbGwsLmgzIC5zbWFsbCwuaDMgc21hbGwsaDEg
LnNtYWxsLGgxIHNtYWxsLGgyIC5zbWFsbCxoMiBzbWFsbCxoMyAuc21hbGwsaDMgc21hbGx7Zm9u
dC1zaXplOjY1JX0uaDQsLmg1LC5oNixoNCxoNSxoNnttYXJnaW4tdG9wOjEwcHg7bWFyZ2luLWJv
dHRvbToxMHB4fS5oNCAuc21hbGwsLmg0IHNtYWxsLC5oNSAuc21hbGwsLmg1IHNtYWxsLC5oNiAu
c21hbGwsLmg2IHNtYWxsLGg0IC5zbWFsbCxoNCBzbWFsbCxoNSAuc21hbGwsaDUgc21hbGwsaDYg
LnNtYWxsLGg2IHNtYWxse2ZvbnQtc2l6ZTo3NSV9LmgxLGgxe2ZvbnQtc2l6ZTozNnB4fS5oMixo
Mntmb250LXNpemU6MzBweH0uaDMsaDN7Zm9udC1zaXplOjI0cHh9Lmg0LGg0e2ZvbnQtc2l6ZTox
OHB4fS5oNSxoNXtmb250LXNpemU6MTRweH0uaDYsaDZ7Zm9udC1zaXplOjEycHh9cHttYXJnaW46
MCAwIDEwcHh9LmxlYWR7bWFyZ2luLWJvdHRvbToyMHB4O2ZvbnQtc2l6ZToxNnB4O2ZvbnQtd2Vp
Z2h0OjMwMDtsaW5lLWhlaWdodDoxLjR9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5sZWFke2Zv
bnQtc2l6ZToyMXB4fX0uc21hbGwsc21hbGx7Zm9udC1zaXplOjg1JX0ubWFyayxtYXJre3BhZGRp
bmc6LjJlbTtiYWNrZ3JvdW5kLWNvbG9yOiNmY2Y4ZTN9LnRleHQtbGVmdHt0ZXh0LWFsaWduOmxl
ZnR9LnRleHQtcmlnaHR7dGV4dC1hbGlnbjpyaWdodH0udGV4dC1jZW50ZXJ7dGV4dC1hbGlnbjpj
ZW50ZXJ9LnRleHQtanVzdGlmeXt0ZXh0LWFsaWduOmp1c3RpZnl9LnRleHQtbm93cmFwe3doaXRl
LXNwYWNlOm5vd3JhcH0udGV4dC1sb3dlcmNhc2V7dGV4dC10cmFuc2Zvcm06bG93ZXJjYXNlfS50
ZXh0LXVwcGVyY2FzZXt0ZXh0LXRyYW5zZm9ybTp1cHBlcmNhc2V9LnRleHQtY2FwaXRhbGl6ZXt0
ZXh0LXRyYW5zZm9ybTpjYXBpdGFsaXplfS50ZXh0LW11dGVke2NvbG9yOiM3Nzd9LnRleHQtcHJp
bWFyeXtjb2xvcjojMzM3YWI3fWEudGV4dC1wcmltYXJ5OmZvY3VzLGEudGV4dC1wcmltYXJ5Omhv
dmVye2NvbG9yOiMyODYwOTB9LnRleHQtc3VjY2Vzc3tjb2xvcjojM2M3NjNkfWEudGV4dC1zdWNj
ZXNzOmZvY3VzLGEudGV4dC1zdWNjZXNzOmhvdmVye2NvbG9yOiMyYjU0MmN9LnRleHQtaW5mb3tj
b2xvcjojMzE3MDhmfWEudGV4dC1pbmZvOmZvY3VzLGEudGV4dC1pbmZvOmhvdmVye2NvbG9yOiMy
NDUyNjl9LnRleHQtd2FybmluZ3tjb2xvcjojOGE2ZDNifWEudGV4dC13YXJuaW5nOmZvY3VzLGEu
dGV4dC13YXJuaW5nOmhvdmVye2NvbG9yOiM2NjUxMmN9LnRleHQtZGFuZ2Vye2NvbG9yOiNhOTQ0
NDJ9YS50ZXh0LWRhbmdlcjpmb2N1cyxhLnRleHQtZGFuZ2VyOmhvdmVye2NvbG9yOiM4NDM1MzR9
LmJnLXByaW1hcnl7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzMzdhYjd9YS5iZy1wcmlt
YXJ5OmZvY3VzLGEuYmctcHJpbWFyeTpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiMyODYwOTB9LmJn
LXN1Y2Nlc3N7YmFja2dyb3VuZC1jb2xvcjojZGZmMGQ4fWEuYmctc3VjY2Vzczpmb2N1cyxhLmJn
LXN1Y2Nlc3M6aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojYzFlMmIzfS5iZy1pbmZve2JhY2tncm91
bmQtY29sb3I6I2Q5ZWRmN31hLmJnLWluZm86Zm9jdXMsYS5iZy1pbmZvOmhvdmVye2JhY2tncm91
bmQtY29sb3I6I2FmZDllZX0uYmctd2FybmluZ3tiYWNrZ3JvdW5kLWNvbG9yOiNmY2Y4ZTN9YS5i
Zy13YXJuaW5nOmZvY3VzLGEuYmctd2FybmluZzpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNmN2Vj
YjV9LmJnLWRhbmdlcntiYWNrZ3JvdW5kLWNvbG9yOiNmMmRlZGV9YS5iZy1kYW5nZXI6Zm9jdXMs
YS5iZy1kYW5nZXI6aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojZTRiOWI5fS5wYWdlLWhlYWRlcntw
YWRkaW5nLWJvdHRvbTo5cHg7bWFyZ2luOjQwcHggMCAyMHB4O2JvcmRlci1ib3R0b206MXB4IHNv
bGlkICNlZWV9b2wsdWx7bWFyZ2luLXRvcDowO21hcmdpbi1ib3R0b206MTBweH1vbCBvbCxvbCB1
bCx1bCBvbCx1bCB1bHttYXJnaW4tYm90dG9tOjB9Lmxpc3QtdW5zdHlsZWR7cGFkZGluZy1sZWZ0
OjA7bGlzdC1zdHlsZTpub25lfS5saXN0LWlubGluZXtwYWRkaW5nLWxlZnQ6MDttYXJnaW4tbGVm
dDotNXB4O2xpc3Qtc3R5bGU6bm9uZX0ubGlzdC1pbmxpbmU+bGl7ZGlzcGxheTppbmxpbmUtYmxv
Y2s7cGFkZGluZy1yaWdodDo1cHg7cGFkZGluZy1sZWZ0OjVweH1kbHttYXJnaW4tdG9wOjA7bWFy
Z2luLWJvdHRvbToyMHB4fWRkLGR0e2xpbmUtaGVpZ2h0OjEuNDI4NTcxNDN9ZHR7Zm9udC13ZWln
aHQ6NzAwfWRke21hcmdpbi1sZWZ0OjB9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5kbC1ob3Jp
em9udGFsIGR0e2Zsb2F0OmxlZnQ7d2lkdGg6MTYwcHg7b3ZlcmZsb3c6aGlkZGVuO2NsZWFyOmxl
ZnQ7dGV4dC1hbGlnbjpyaWdodDt0ZXh0LW92ZXJmbG93OmVsbGlwc2lzO3doaXRlLXNwYWNlOm5v
d3JhcH0uZGwtaG9yaXpvbnRhbCBkZHttYXJnaW4tbGVmdDoxODBweH19YWJicltkYXRhLW9yaWdp
bmFsLXRpdGxlXSxhYmJyW3RpdGxlXXtjdXJzb3I6aGVscDtib3JkZXItYm90dG9tOjFweCBkb3R0
ZWQgIzc3N30uaW5pdGlhbGlzbXtmb250LXNpemU6OTAlO3RleHQtdHJhbnNmb3JtOnVwcGVyY2Fz
ZX1ibG9ja3F1b3Rle3BhZGRpbmc6MTBweCAyMHB4O21hcmdpbjowIDAgMjBweDtmb250LXNpemU6
MTcuNXB4O2JvcmRlci1sZWZ0OjVweCBzb2xpZCAjZWVlfWJsb2NrcXVvdGUgb2w6bGFzdC1jaGls
ZCxibG9ja3F1b3RlIHA6bGFzdC1jaGlsZCxibG9ja3F1b3RlIHVsOmxhc3QtY2hpbGR7bWFyZ2lu
LWJvdHRvbTowfWJsb2NrcXVvdGUgLnNtYWxsLGJsb2NrcXVvdGUgZm9vdGVyLGJsb2NrcXVvdGUg
c21hbGx7ZGlzcGxheTpibG9jaztmb250LXNpemU6ODAlO2xpbmUtaGVpZ2h0OjEuNDI4NTcxNDM7
Y29sb3I6Izc3N31ibG9ja3F1b3RlIC5zbWFsbDpiZWZvcmUsYmxvY2txdW90ZSBmb290ZXI6YmVm
b3JlLGJsb2NrcXVvdGUgc21hbGw6YmVmb3Jle2NvbnRlbnQ6J1wyMDE0IFwwMEEwJ30uYmxvY2tx
dW90ZS1yZXZlcnNlLGJsb2NrcXVvdGUucHVsbC1yaWdodHtwYWRkaW5nLXJpZ2h0OjE1cHg7cGFk
ZGluZy1sZWZ0OjA7dGV4dC1hbGlnbjpyaWdodDtib3JkZXItcmlnaHQ6NXB4IHNvbGlkICNlZWU7
Ym9yZGVyLWxlZnQ6MH0uYmxvY2txdW90ZS1yZXZlcnNlIC5zbWFsbDpiZWZvcmUsLmJsb2NrcXVv
dGUtcmV2ZXJzZSBmb290ZXI6YmVmb3JlLC5ibG9ja3F1b3RlLXJldmVyc2Ugc21hbGw6YmVmb3Jl
LGJsb2NrcXVvdGUucHVsbC1yaWdodCAuc21hbGw6YmVmb3JlLGJsb2NrcXVvdGUucHVsbC1yaWdo
dCBmb290ZXI6YmVmb3JlLGJsb2NrcXVvdGUucHVsbC1yaWdodCBzbWFsbDpiZWZvcmV7Y29udGVu
dDonJ30uYmxvY2txdW90ZS1yZXZlcnNlIC5zbWFsbDphZnRlciwuYmxvY2txdW90ZS1yZXZlcnNl
IGZvb3RlcjphZnRlciwuYmxvY2txdW90ZS1yZXZlcnNlIHNtYWxsOmFmdGVyLGJsb2NrcXVvdGUu
cHVsbC1yaWdodCAuc21hbGw6YWZ0ZXIsYmxvY2txdW90ZS5wdWxsLXJpZ2h0IGZvb3RlcjphZnRl
cixibG9ja3F1b3RlLnB1bGwtcmlnaHQgc21hbGw6YWZ0ZXJ7Y29udGVudDonXDAwQTAgXDIwMTQn
fWFkZHJlc3N7bWFyZ2luLWJvdHRvbToyMHB4O2ZvbnQtc3R5bGU6bm9ybWFsO2xpbmUtaGVpZ2h0
OjEuNDI4NTcxNDN9Y29kZSxrYmQscHJlLHNhbXB7Zm9udC1mYW1pbHk6TWVubG8sTW9uYWNvLENv
bnNvbGFzLCJDb3VyaWVyIE5ldyIsbW9ub3NwYWNlfWNvZGV7cGFkZGluZzoycHggNHB4O2ZvbnQt
c2l6ZTo5MCU7Y29sb3I6I2M3MjU0ZTtiYWNrZ3JvdW5kLWNvbG9yOiNmOWYyZjQ7Ym9yZGVyLXJh
ZGl1czo0cHh9a2Jke3BhZGRpbmc6MnB4IDRweDtmb250LXNpemU6OTAlO2NvbG9yOiNmZmY7YmFj
a2dyb3VuZC1jb2xvcjojMzMzO2JvcmRlci1yYWRpdXM6M3B4Oy13ZWJraXQtYm94LXNoYWRvdzpp
bnNldCAwIC0xcHggMCByZ2JhKDAsMCwwLC4yNSk7Ym94LXNoYWRvdzppbnNldCAwIC0xcHggMCBy
Z2JhKDAsMCwwLC4yNSl9a2JkIGtiZHtwYWRkaW5nOjA7Zm9udC1zaXplOjEwMCU7Zm9udC13ZWln
aHQ6NzAwOy13ZWJraXQtYm94LXNoYWRvdzpub25lO2JveC1zaGFkb3c6bm9uZX1wcmV7ZGlzcGxh
eTpibG9jaztwYWRkaW5nOjkuNXB4O21hcmdpbjowIDAgMTBweDtmb250LXNpemU6MTNweDtsaW5l
LWhlaWdodDoxLjQyODU3MTQzO2NvbG9yOiMzMzM7d29yZC1icmVhazpicmVhay1hbGw7d29yZC13
cmFwOmJyZWFrLXdvcmQ7YmFja2dyb3VuZC1jb2xvcjojZjVmNWY1O2JvcmRlcjoxcHggc29saWQg
I2NjYztib3JkZXItcmFkaXVzOjRweH1wcmUgY29kZXtwYWRkaW5nOjA7Zm9udC1zaXplOmluaGVy
aXQ7Y29sb3I6aW5oZXJpdDt3aGl0ZS1zcGFjZTpwcmUtd3JhcDtiYWNrZ3JvdW5kLWNvbG9yOnRy
YW5zcGFyZW50O2JvcmRlci1yYWRpdXM6MH0ucHJlLXNjcm9sbGFibGV7bWF4LWhlaWdodDozNDBw
eDtvdmVyZmxvdy15OnNjcm9sbH0uY29udGFpbmVye3BhZGRpbmctcmlnaHQ6MTVweDtwYWRkaW5n
LWxlZnQ6MTVweDttYXJnaW4tcmlnaHQ6YXV0bzttYXJnaW4tbGVmdDphdXRvfUBtZWRpYSAobWlu
LXdpZHRoOjc2OHB4KXsuY29udGFpbmVye3dpZHRoOjc1MHB4fX1AbWVkaWEgKG1pbi13aWR0aDo5
OTJweCl7LmNvbnRhaW5lcnt3aWR0aDo5NzBweH19QG1lZGlhIChtaW4td2lkdGg6MTIwMHB4KXsu
Y29udGFpbmVye3dpZHRoOjExNzBweH19LmNvbnRhaW5lci1mbHVpZHtwYWRkaW5nLXJpZ2h0OjE1
cHg7cGFkZGluZy1sZWZ0OjE1cHg7bWFyZ2luLXJpZ2h0OmF1dG87bWFyZ2luLWxlZnQ6YXV0b30u
cm93e21hcmdpbi1yaWdodDotMTVweDttYXJnaW4tbGVmdDotMTVweH0uY29sLWxnLTEsLmNvbC1s
Zy0xMCwuY29sLWxnLTExLC5jb2wtbGctMTIsLmNvbC1sZy0yLC5jb2wtbGctMywuY29sLWxnLTQs
LmNvbC1sZy01LC5jb2wtbGctNiwuY29sLWxnLTcsLmNvbC1sZy04LC5jb2wtbGctOSwuY29sLW1k
LTEsLmNvbC1tZC0xMCwuY29sLW1kLTExLC5jb2wtbWQtMTIsLmNvbC1tZC0yLC5jb2wtbWQtMywu
Y29sLW1kLTQsLmNvbC1tZC01LC5jb2wtbWQtNiwuY29sLW1kLTcsLmNvbC1tZC04LC5jb2wtbWQt
OSwuY29sLXNtLTEsLmNvbC1zbS0xMCwuY29sLXNtLTExLC5jb2wtc20tMTIsLmNvbC1zbS0yLC5j
b2wtc20tMywuY29sLXNtLTQsLmNvbC1zbS01LC5jb2wtc20tNiwuY29sLXNtLTcsLmNvbC1zbS04
LC5jb2wtc20tOSwuY29sLXhzLTEsLmNvbC14cy0xMCwuY29sLXhzLTExLC5jb2wteHMtMTIsLmNv
bC14cy0yLC5jb2wteHMtMywuY29sLXhzLTQsLmNvbC14cy01LC5jb2wteHMtNiwuY29sLXhzLTcs
LmNvbC14cy04LC5jb2wteHMtOXtwb3NpdGlvbjpyZWxhdGl2ZTttaW4taGVpZ2h0OjFweDtwYWRk
aW5nLXJpZ2h0OjE1cHg7cGFkZGluZy1sZWZ0OjE1cHh9LmNvbC14cy0xLC5jb2wteHMtMTAsLmNv
bC14cy0xMSwuY29sLXhzLTEyLC5jb2wteHMtMiwuY29sLXhzLTMsLmNvbC14cy00LC5jb2wteHMt
NSwuY29sLXhzLTYsLmNvbC14cy03LC5jb2wteHMtOCwuY29sLXhzLTl7ZmxvYXQ6bGVmdH0uY29s
LXhzLTEye3dpZHRoOjEwMCV9LmNvbC14cy0xMXt3aWR0aDo5MS42NjY2NjY2NyV9LmNvbC14cy0x
MHt3aWR0aDo4My4zMzMzMzMzMyV9LmNvbC14cy05e3dpZHRoOjc1JX0uY29sLXhzLTh7d2lkdGg6
NjYuNjY2NjY2NjclfS5jb2wteHMtN3t3aWR0aDo1OC4zMzMzMzMzMyV9LmNvbC14cy02e3dpZHRo
OjUwJX0uY29sLXhzLTV7d2lkdGg6NDEuNjY2NjY2NjclfS5jb2wteHMtNHt3aWR0aDozMy4zMzMz
MzMzMyV9LmNvbC14cy0ze3dpZHRoOjI1JX0uY29sLXhzLTJ7d2lkdGg6MTYuNjY2NjY2NjclfS5j
b2wteHMtMXt3aWR0aDo4LjMzMzMzMzMzJX0uY29sLXhzLXB1bGwtMTJ7cmlnaHQ6MTAwJX0uY29s
LXhzLXB1bGwtMTF7cmlnaHQ6OTEuNjY2NjY2NjclfS5jb2wteHMtcHVsbC0xMHtyaWdodDo4My4z
MzMzMzMzMyV9LmNvbC14cy1wdWxsLTl7cmlnaHQ6NzUlfS5jb2wteHMtcHVsbC04e3JpZ2h0OjY2
LjY2NjY2NjY3JX0uY29sLXhzLXB1bGwtN3tyaWdodDo1OC4zMzMzMzMzMyV9LmNvbC14cy1wdWxs
LTZ7cmlnaHQ6NTAlfS5jb2wteHMtcHVsbC01e3JpZ2h0OjQxLjY2NjY2NjY3JX0uY29sLXhzLXB1
bGwtNHtyaWdodDozMy4zMzMzMzMzMyV9LmNvbC14cy1wdWxsLTN7cmlnaHQ6MjUlfS5jb2wteHMt
cHVsbC0ye3JpZ2h0OjE2LjY2NjY2NjY3JX0uY29sLXhzLXB1bGwtMXtyaWdodDo4LjMzMzMzMzMz
JX0uY29sLXhzLXB1bGwtMHtyaWdodDphdXRvfS5jb2wteHMtcHVzaC0xMntsZWZ0OjEwMCV9LmNv
bC14cy1wdXNoLTExe2xlZnQ6OTEuNjY2NjY2NjclfS5jb2wteHMtcHVzaC0xMHtsZWZ0OjgzLjMz
MzMzMzMzJX0uY29sLXhzLXB1c2gtOXtsZWZ0Ojc1JX0uY29sLXhzLXB1c2gtOHtsZWZ0OjY2LjY2
NjY2NjY3JX0uY29sLXhzLXB1c2gtN3tsZWZ0OjU4LjMzMzMzMzMzJX0uY29sLXhzLXB1c2gtNnts
ZWZ0OjUwJX0uY29sLXhzLXB1c2gtNXtsZWZ0OjQxLjY2NjY2NjY3JX0uY29sLXhzLXB1c2gtNHts
ZWZ0OjMzLjMzMzMzMzMzJX0uY29sLXhzLXB1c2gtM3tsZWZ0OjI1JX0uY29sLXhzLXB1c2gtMnts
ZWZ0OjE2LjY2NjY2NjY3JX0uY29sLXhzLXB1c2gtMXtsZWZ0OjguMzMzMzMzMzMlfS5jb2wteHMt
cHVzaC0we2xlZnQ6YXV0b30uY29sLXhzLW9mZnNldC0xMnttYXJnaW4tbGVmdDoxMDAlfS5jb2wt
eHMtb2Zmc2V0LTExe21hcmdpbi1sZWZ0OjkxLjY2NjY2NjY3JX0uY29sLXhzLW9mZnNldC0xMHtt
YXJnaW4tbGVmdDo4My4zMzMzMzMzMyV9LmNvbC14cy1vZmZzZXQtOXttYXJnaW4tbGVmdDo3NSV9
LmNvbC14cy1vZmZzZXQtOHttYXJnaW4tbGVmdDo2Ni42NjY2NjY2NyV9LmNvbC14cy1vZmZzZXQt
N3ttYXJnaW4tbGVmdDo1OC4zMzMzMzMzMyV9LmNvbC14cy1vZmZzZXQtNnttYXJnaW4tbGVmdDo1
MCV9LmNvbC14cy1vZmZzZXQtNXttYXJnaW4tbGVmdDo0MS42NjY2NjY2NyV9LmNvbC14cy1vZmZz
ZXQtNHttYXJnaW4tbGVmdDozMy4zMzMzMzMzMyV9LmNvbC14cy1vZmZzZXQtM3ttYXJnaW4tbGVm
dDoyNSV9LmNvbC14cy1vZmZzZXQtMnttYXJnaW4tbGVmdDoxNi42NjY2NjY2NyV9LmNvbC14cy1v
ZmZzZXQtMXttYXJnaW4tbGVmdDo4LjMzMzMzMzMzJX0uY29sLXhzLW9mZnNldC0we21hcmdpbi1s
ZWZ0OjB9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5jb2wtc20tMSwuY29sLXNtLTEwLC5jb2wt
c20tMTEsLmNvbC1zbS0xMiwuY29sLXNtLTIsLmNvbC1zbS0zLC5jb2wtc20tNCwuY29sLXNtLTUs
LmNvbC1zbS02LC5jb2wtc20tNywuY29sLXNtLTgsLmNvbC1zbS05e2Zsb2F0OmxlZnR9LmNvbC1z
bS0xMnt3aWR0aDoxMDAlfS5jb2wtc20tMTF7d2lkdGg6OTEuNjY2NjY2NjclfS5jb2wtc20tMTB7
d2lkdGg6ODMuMzMzMzMzMzMlfS5jb2wtc20tOXt3aWR0aDo3NSV9LmNvbC1zbS04e3dpZHRoOjY2
LjY2NjY2NjY3JX0uY29sLXNtLTd7d2lkdGg6NTguMzMzMzMzMzMlfS5jb2wtc20tNnt3aWR0aDo1
MCV9LmNvbC1zbS01e3dpZHRoOjQxLjY2NjY2NjY3JX0uY29sLXNtLTR7d2lkdGg6MzMuMzMzMzMz
MzMlfS5jb2wtc20tM3t3aWR0aDoyNSV9LmNvbC1zbS0ye3dpZHRoOjE2LjY2NjY2NjY3JX0uY29s
LXNtLTF7d2lkdGg6OC4zMzMzMzMzMyV9LmNvbC1zbS1wdWxsLTEye3JpZ2h0OjEwMCV9LmNvbC1z
bS1wdWxsLTExe3JpZ2h0OjkxLjY2NjY2NjY3JX0uY29sLXNtLXB1bGwtMTB7cmlnaHQ6ODMuMzMz
MzMzMzMlfS5jb2wtc20tcHVsbC05e3JpZ2h0Ojc1JX0uY29sLXNtLXB1bGwtOHtyaWdodDo2Ni42
NjY2NjY2NyV9LmNvbC1zbS1wdWxsLTd7cmlnaHQ6NTguMzMzMzMzMzMlfS5jb2wtc20tcHVsbC02
e3JpZ2h0OjUwJX0uY29sLXNtLXB1bGwtNXtyaWdodDo0MS42NjY2NjY2NyV9LmNvbC1zbS1wdWxs
LTR7cmlnaHQ6MzMuMzMzMzMzMzMlfS5jb2wtc20tcHVsbC0ze3JpZ2h0OjI1JX0uY29sLXNtLXB1
bGwtMntyaWdodDoxNi42NjY2NjY2NyV9LmNvbC1zbS1wdWxsLTF7cmlnaHQ6OC4zMzMzMzMzMyV9
LmNvbC1zbS1wdWxsLTB7cmlnaHQ6YXV0b30uY29sLXNtLXB1c2gtMTJ7bGVmdDoxMDAlfS5jb2wt
c20tcHVzaC0xMXtsZWZ0OjkxLjY2NjY2NjY3JX0uY29sLXNtLXB1c2gtMTB7bGVmdDo4My4zMzMz
MzMzMyV9LmNvbC1zbS1wdXNoLTl7bGVmdDo3NSV9LmNvbC1zbS1wdXNoLTh7bGVmdDo2Ni42NjY2
NjY2NyV9LmNvbC1zbS1wdXNoLTd7bGVmdDo1OC4zMzMzMzMzMyV9LmNvbC1zbS1wdXNoLTZ7bGVm
dDo1MCV9LmNvbC1zbS1wdXNoLTV7bGVmdDo0MS42NjY2NjY2NyV9LmNvbC1zbS1wdXNoLTR7bGVm
dDozMy4zMzMzMzMzMyV9LmNvbC1zbS1wdXNoLTN7bGVmdDoyNSV9LmNvbC1zbS1wdXNoLTJ7bGVm
dDoxNi42NjY2NjY2NyV9LmNvbC1zbS1wdXNoLTF7bGVmdDo4LjMzMzMzMzMzJX0uY29sLXNtLXB1
c2gtMHtsZWZ0OmF1dG99LmNvbC1zbS1vZmZzZXQtMTJ7bWFyZ2luLWxlZnQ6MTAwJX0uY29sLXNt
LW9mZnNldC0xMXttYXJnaW4tbGVmdDo5MS42NjY2NjY2NyV9LmNvbC1zbS1vZmZzZXQtMTB7bWFy
Z2luLWxlZnQ6ODMuMzMzMzMzMzMlfS5jb2wtc20tb2Zmc2V0LTl7bWFyZ2luLWxlZnQ6NzUlfS5j
b2wtc20tb2Zmc2V0LTh7bWFyZ2luLWxlZnQ6NjYuNjY2NjY2NjclfS5jb2wtc20tb2Zmc2V0LTd7
bWFyZ2luLWxlZnQ6NTguMzMzMzMzMzMlfS5jb2wtc20tb2Zmc2V0LTZ7bWFyZ2luLWxlZnQ6NTAl
fS5jb2wtc20tb2Zmc2V0LTV7bWFyZ2luLWxlZnQ6NDEuNjY2NjY2NjclfS5jb2wtc20tb2Zmc2V0
LTR7bWFyZ2luLWxlZnQ6MzMuMzMzMzMzMzMlfS5jb2wtc20tb2Zmc2V0LTN7bWFyZ2luLWxlZnQ6
MjUlfS5jb2wtc20tb2Zmc2V0LTJ7bWFyZ2luLWxlZnQ6MTYuNjY2NjY2NjclfS5jb2wtc20tb2Zm
c2V0LTF7bWFyZ2luLWxlZnQ6OC4zMzMzMzMzMyV9LmNvbC1zbS1vZmZzZXQtMHttYXJnaW4tbGVm
dDowfX1AbWVkaWEgKG1pbi13aWR0aDo5OTJweCl7LmNvbC1tZC0xLC5jb2wtbWQtMTAsLmNvbC1t
ZC0xMSwuY29sLW1kLTEyLC5jb2wtbWQtMiwuY29sLW1kLTMsLmNvbC1tZC00LC5jb2wtbWQtNSwu
Y29sLW1kLTYsLmNvbC1tZC03LC5jb2wtbWQtOCwuY29sLW1kLTl7ZmxvYXQ6bGVmdH0uY29sLW1k
LTEye3dpZHRoOjEwMCV9LmNvbC1tZC0xMXt3aWR0aDo5MS42NjY2NjY2NyV9LmNvbC1tZC0xMHt3
aWR0aDo4My4zMzMzMzMzMyV9LmNvbC1tZC05e3dpZHRoOjc1JX0uY29sLW1kLTh7d2lkdGg6NjYu
NjY2NjY2NjclfS5jb2wtbWQtN3t3aWR0aDo1OC4zMzMzMzMzMyV9LmNvbC1tZC02e3dpZHRoOjUw
JX0uY29sLW1kLTV7d2lkdGg6NDEuNjY2NjY2NjclfS5jb2wtbWQtNHt3aWR0aDozMy4zMzMzMzMz
MyV9LmNvbC1tZC0ze3dpZHRoOjI1JX0uY29sLW1kLTJ7d2lkdGg6MTYuNjY2NjY2NjclfS5jb2wt
bWQtMXt3aWR0aDo4LjMzMzMzMzMzJX0uY29sLW1kLXB1bGwtMTJ7cmlnaHQ6MTAwJX0uY29sLW1k
LXB1bGwtMTF7cmlnaHQ6OTEuNjY2NjY2NjclfS5jb2wtbWQtcHVsbC0xMHtyaWdodDo4My4zMzMz
MzMzMyV9LmNvbC1tZC1wdWxsLTl7cmlnaHQ6NzUlfS5jb2wtbWQtcHVsbC04e3JpZ2h0OjY2LjY2
NjY2NjY3JX0uY29sLW1kLXB1bGwtN3tyaWdodDo1OC4zMzMzMzMzMyV9LmNvbC1tZC1wdWxsLTZ7
cmlnaHQ6NTAlfS5jb2wtbWQtcHVsbC01e3JpZ2h0OjQxLjY2NjY2NjY3JX0uY29sLW1kLXB1bGwt
NHtyaWdodDozMy4zMzMzMzMzMyV9LmNvbC1tZC1wdWxsLTN7cmlnaHQ6MjUlfS5jb2wtbWQtcHVs
bC0ye3JpZ2h0OjE2LjY2NjY2NjY3JX0uY29sLW1kLXB1bGwtMXtyaWdodDo4LjMzMzMzMzMzJX0u
Y29sLW1kLXB1bGwtMHtyaWdodDphdXRvfS5jb2wtbWQtcHVzaC0xMntsZWZ0OjEwMCV9LmNvbC1t
ZC1wdXNoLTExe2xlZnQ6OTEuNjY2NjY2NjclfS5jb2wtbWQtcHVzaC0xMHtsZWZ0OjgzLjMzMzMz
MzMzJX0uY29sLW1kLXB1c2gtOXtsZWZ0Ojc1JX0uY29sLW1kLXB1c2gtOHtsZWZ0OjY2LjY2NjY2
NjY3JX0uY29sLW1kLXB1c2gtN3tsZWZ0OjU4LjMzMzMzMzMzJX0uY29sLW1kLXB1c2gtNntsZWZ0
OjUwJX0uY29sLW1kLXB1c2gtNXtsZWZ0OjQxLjY2NjY2NjY3JX0uY29sLW1kLXB1c2gtNHtsZWZ0
OjMzLjMzMzMzMzMzJX0uY29sLW1kLXB1c2gtM3tsZWZ0OjI1JX0uY29sLW1kLXB1c2gtMntsZWZ0
OjE2LjY2NjY2NjY3JX0uY29sLW1kLXB1c2gtMXtsZWZ0OjguMzMzMzMzMzMlfS5jb2wtbWQtcHVz
aC0we2xlZnQ6YXV0b30uY29sLW1kLW9mZnNldC0xMnttYXJnaW4tbGVmdDoxMDAlfS5jb2wtbWQt
b2Zmc2V0LTExe21hcmdpbi1sZWZ0OjkxLjY2NjY2NjY3JX0uY29sLW1kLW9mZnNldC0xMHttYXJn
aW4tbGVmdDo4My4zMzMzMzMzMyV9LmNvbC1tZC1vZmZzZXQtOXttYXJnaW4tbGVmdDo3NSV9LmNv
bC1tZC1vZmZzZXQtOHttYXJnaW4tbGVmdDo2Ni42NjY2NjY2NyV9LmNvbC1tZC1vZmZzZXQtN3tt
YXJnaW4tbGVmdDo1OC4zMzMzMzMzMyV9LmNvbC1tZC1vZmZzZXQtNnttYXJnaW4tbGVmdDo1MCV9
LmNvbC1tZC1vZmZzZXQtNXttYXJnaW4tbGVmdDo0MS42NjY2NjY2NyV9LmNvbC1tZC1vZmZzZXQt
NHttYXJnaW4tbGVmdDozMy4zMzMzMzMzMyV9LmNvbC1tZC1vZmZzZXQtM3ttYXJnaW4tbGVmdDoy
NSV9LmNvbC1tZC1vZmZzZXQtMnttYXJnaW4tbGVmdDoxNi42NjY2NjY2NyV9LmNvbC1tZC1vZmZz
ZXQtMXttYXJnaW4tbGVmdDo4LjMzMzMzMzMzJX0uY29sLW1kLW9mZnNldC0we21hcmdpbi1sZWZ0
OjB9fUBtZWRpYSAobWluLXdpZHRoOjEyMDBweCl7LmNvbC1sZy0xLC5jb2wtbGctMTAsLmNvbC1s
Zy0xMSwuY29sLWxnLTEyLC5jb2wtbGctMiwuY29sLWxnLTMsLmNvbC1sZy00LC5jb2wtbGctNSwu
Y29sLWxnLTYsLmNvbC1sZy03LC5jb2wtbGctOCwuY29sLWxnLTl7ZmxvYXQ6bGVmdH0uY29sLWxn
LTEye3dpZHRoOjEwMCV9LmNvbC1sZy0xMXt3aWR0aDo5MS42NjY2NjY2NyV9LmNvbC1sZy0xMHt3
aWR0aDo4My4zMzMzMzMzMyV9LmNvbC1sZy05e3dpZHRoOjc1JX0uY29sLWxnLTh7d2lkdGg6NjYu
NjY2NjY2NjclfS5jb2wtbGctN3t3aWR0aDo1OC4zMzMzMzMzMyV9LmNvbC1sZy02e3dpZHRoOjUw
JX0uY29sLWxnLTV7d2lkdGg6NDEuNjY2NjY2NjclfS5jb2wtbGctNHt3aWR0aDozMy4zMzMzMzMz
MyV9LmNvbC1sZy0ze3dpZHRoOjI1JX0uY29sLWxnLTJ7d2lkdGg6MTYuNjY2NjY2NjclfS5jb2wt
bGctMXt3aWR0aDo4LjMzMzMzMzMzJX0uY29sLWxnLXB1bGwtMTJ7cmlnaHQ6MTAwJX0uY29sLWxn
LXB1bGwtMTF7cmlnaHQ6OTEuNjY2NjY2NjclfS5jb2wtbGctcHVsbC0xMHtyaWdodDo4My4zMzMz
MzMzMyV9LmNvbC1sZy1wdWxsLTl7cmlnaHQ6NzUlfS5jb2wtbGctcHVsbC04e3JpZ2h0OjY2LjY2
NjY2NjY3JX0uY29sLWxnLXB1bGwtN3tyaWdodDo1OC4zMzMzMzMzMyV9LmNvbC1sZy1wdWxsLTZ7
cmlnaHQ6NTAlfS5jb2wtbGctcHVsbC01e3JpZ2h0OjQxLjY2NjY2NjY3JX0uY29sLWxnLXB1bGwt
NHtyaWdodDozMy4zMzMzMzMzMyV9LmNvbC1sZy1wdWxsLTN7cmlnaHQ6MjUlfS5jb2wtbGctcHVs
bC0ye3JpZ2h0OjE2LjY2NjY2NjY3JX0uY29sLWxnLXB1bGwtMXtyaWdodDo4LjMzMzMzMzMzJX0u
Y29sLWxnLXB1bGwtMHtyaWdodDphdXRvfS5jb2wtbGctcHVzaC0xMntsZWZ0OjEwMCV9LmNvbC1s
Zy1wdXNoLTExe2xlZnQ6OTEuNjY2NjY2NjclfS5jb2wtbGctcHVzaC0xMHtsZWZ0OjgzLjMzMzMz
MzMzJX0uY29sLWxnLXB1c2gtOXtsZWZ0Ojc1JX0uY29sLWxnLXB1c2gtOHtsZWZ0OjY2LjY2NjY2
NjY3JX0uY29sLWxnLXB1c2gtN3tsZWZ0OjU4LjMzMzMzMzMzJX0uY29sLWxnLXB1c2gtNntsZWZ0
OjUwJX0uY29sLWxnLXB1c2gtNXtsZWZ0OjQxLjY2NjY2NjY3JX0uY29sLWxnLXB1c2gtNHtsZWZ0
OjMzLjMzMzMzMzMzJX0uY29sLWxnLXB1c2gtM3tsZWZ0OjI1JX0uY29sLWxnLXB1c2gtMntsZWZ0
OjE2LjY2NjY2NjY3JX0uY29sLWxnLXB1c2gtMXtsZWZ0OjguMzMzMzMzMzMlfS5jb2wtbGctcHVz
aC0we2xlZnQ6YXV0b30uY29sLWxnLW9mZnNldC0xMnttYXJnaW4tbGVmdDoxMDAlfS5jb2wtbGct
b2Zmc2V0LTExe21hcmdpbi1sZWZ0OjkxLjY2NjY2NjY3JX0uY29sLWxnLW9mZnNldC0xMHttYXJn
aW4tbGVmdDo4My4zMzMzMzMzMyV9LmNvbC1sZy1vZmZzZXQtOXttYXJnaW4tbGVmdDo3NSV9LmNv
bC1sZy1vZmZzZXQtOHttYXJnaW4tbGVmdDo2Ni42NjY2NjY2NyV9LmNvbC1sZy1vZmZzZXQtN3tt
YXJnaW4tbGVmdDo1OC4zMzMzMzMzMyV9LmNvbC1sZy1vZmZzZXQtNnttYXJnaW4tbGVmdDo1MCV9
LmNvbC1sZy1vZmZzZXQtNXttYXJnaW4tbGVmdDo0MS42NjY2NjY2NyV9LmNvbC1sZy1vZmZzZXQt
NHttYXJnaW4tbGVmdDozMy4zMzMzMzMzMyV9LmNvbC1sZy1vZmZzZXQtM3ttYXJnaW4tbGVmdDoy
NSV9LmNvbC1sZy1vZmZzZXQtMnttYXJnaW4tbGVmdDoxNi42NjY2NjY2NyV9LmNvbC1sZy1vZmZz
ZXQtMXttYXJnaW4tbGVmdDo4LjMzMzMzMzMzJX0uY29sLWxnLW9mZnNldC0we21hcmdpbi1sZWZ0
OjB9fXRhYmxle2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnR9Y2FwdGlvbntwYWRkaW5nLXRv
cDo4cHg7cGFkZGluZy1ib3R0b206OHB4O2NvbG9yOiM3Nzc7dGV4dC1hbGlnbjpsZWZ0fXRoe3Rl
eHQtYWxpZ246bGVmdH0udGFibGV7d2lkdGg6MTAwJTttYXgtd2lkdGg6MTAwJTttYXJnaW4tYm90
dG9tOjIwcHh9LnRhYmxlPnRib2R5PnRyPnRkLC50YWJsZT50Ym9keT50cj50aCwudGFibGU+dGZv
b3Q+dHI+dGQsLnRhYmxlPnRmb290PnRyPnRoLC50YWJsZT50aGVhZD50cj50ZCwudGFibGU+dGhl
YWQ+dHI+dGh7cGFkZGluZzo4cHg7bGluZS1oZWlnaHQ6MS40Mjg1NzE0Mzt2ZXJ0aWNhbC1hbGln
bjp0b3A7Ym9yZGVyLXRvcDoxcHggc29saWQgI2RkZH0udGFibGU+dGhlYWQ+dHI+dGh7dmVydGlj
YWwtYWxpZ246Ym90dG9tO2JvcmRlci1ib3R0b206MnB4IHNvbGlkICNkZGR9LnRhYmxlPmNhcHRp
b24rdGhlYWQ+dHI6Zmlyc3QtY2hpbGQ+dGQsLnRhYmxlPmNhcHRpb24rdGhlYWQ+dHI6Zmlyc3Qt
Y2hpbGQ+dGgsLnRhYmxlPmNvbGdyb3VwK3RoZWFkPnRyOmZpcnN0LWNoaWxkPnRkLC50YWJsZT5j
b2xncm91cCt0aGVhZD50cjpmaXJzdC1jaGlsZD50aCwudGFibGU+dGhlYWQ6Zmlyc3QtY2hpbGQ+
dHI6Zmlyc3QtY2hpbGQ+dGQsLnRhYmxlPnRoZWFkOmZpcnN0LWNoaWxkPnRyOmZpcnN0LWNoaWxk
PnRoe2JvcmRlci10b3A6MH0udGFibGU+dGJvZHkrdGJvZHl7Ym9yZGVyLXRvcDoycHggc29saWQg
I2RkZH0udGFibGUgLnRhYmxle2JhY2tncm91bmQtY29sb3I6I2ZmZn0udGFibGUtY29uZGVuc2Vk
PnRib2R5PnRyPnRkLC50YWJsZS1jb25kZW5zZWQ+dGJvZHk+dHI+dGgsLnRhYmxlLWNvbmRlbnNl
ZD50Zm9vdD50cj50ZCwudGFibGUtY29uZGVuc2VkPnRmb290PnRyPnRoLC50YWJsZS1jb25kZW5z
ZWQ+dGhlYWQ+dHI+dGQsLnRhYmxlLWNvbmRlbnNlZD50aGVhZD50cj50aHtwYWRkaW5nOjVweH0u
dGFibGUtYm9yZGVyZWR7Ym9yZGVyOjFweCBzb2xpZCAjZGRkfS50YWJsZS1ib3JkZXJlZD50Ym9k
eT50cj50ZCwudGFibGUtYm9yZGVyZWQ+dGJvZHk+dHI+dGgsLnRhYmxlLWJvcmRlcmVkPnRmb290
PnRyPnRkLC50YWJsZS1ib3JkZXJlZD50Zm9vdD50cj50aCwudGFibGUtYm9yZGVyZWQ+dGhlYWQ+
dHI+dGQsLnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRyPnRoe2JvcmRlcjoxcHggc29saWQgI2RkZH0u
dGFibGUtYm9yZGVyZWQ+dGhlYWQ+dHI+dGQsLnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRyPnRoe2Jv
cmRlci1ib3R0b20td2lkdGg6MnB4fS50YWJsZS1zdHJpcGVkPnRib2R5PnRyOm50aC1vZi10eXBl
KG9kZCl7YmFja2dyb3VuZC1jb2xvcjojZjlmOWY5fS50YWJsZS1ob3Zlcj50Ym9keT50cjpob3Zl
cntiYWNrZ3JvdW5kLWNvbG9yOiNmNWY1ZjV9dGFibGUgY29sW2NsYXNzKj1jb2wtXXtwb3NpdGlv
bjpzdGF0aWM7ZGlzcGxheTp0YWJsZS1jb2x1bW47ZmxvYXQ6bm9uZX10YWJsZSB0ZFtjbGFzcyo9
Y29sLV0sdGFibGUgdGhbY2xhc3MqPWNvbC1de3Bvc2l0aW9uOnN0YXRpYztkaXNwbGF5OnRhYmxl
LWNlbGw7ZmxvYXQ6bm9uZX0udGFibGU+dGJvZHk+dHIuYWN0aXZlPnRkLC50YWJsZT50Ym9keT50
ci5hY3RpdmU+dGgsLnRhYmxlPnRib2R5PnRyPnRkLmFjdGl2ZSwudGFibGU+dGJvZHk+dHI+dGgu
YWN0aXZlLC50YWJsZT50Zm9vdD50ci5hY3RpdmU+dGQsLnRhYmxlPnRmb290PnRyLmFjdGl2ZT50
aCwudGFibGU+dGZvb3Q+dHI+dGQuYWN0aXZlLC50YWJsZT50Zm9vdD50cj50aC5hY3RpdmUsLnRh
YmxlPnRoZWFkPnRyLmFjdGl2ZT50ZCwudGFibGU+dGhlYWQ+dHIuYWN0aXZlPnRoLC50YWJsZT50
aGVhZD50cj50ZC5hY3RpdmUsLnRhYmxlPnRoZWFkPnRyPnRoLmFjdGl2ZXtiYWNrZ3JvdW5kLWNv
bG9yOiNmNWY1ZjV9LnRhYmxlLWhvdmVyPnRib2R5PnRyLmFjdGl2ZTpob3Zlcj50ZCwudGFibGUt
aG92ZXI+dGJvZHk+dHIuYWN0aXZlOmhvdmVyPnRoLC50YWJsZS1ob3Zlcj50Ym9keT50cjpob3Zl
cj4uYWN0aXZlLC50YWJsZS1ob3Zlcj50Ym9keT50cj50ZC5hY3RpdmU6aG92ZXIsLnRhYmxlLWhv
dmVyPnRib2R5PnRyPnRoLmFjdGl2ZTpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNlOGU4ZTh9LnRh
YmxlPnRib2R5PnRyLnN1Y2Nlc3M+dGQsLnRhYmxlPnRib2R5PnRyLnN1Y2Nlc3M+dGgsLnRhYmxl
PnRib2R5PnRyPnRkLnN1Y2Nlc3MsLnRhYmxlPnRib2R5PnRyPnRoLnN1Y2Nlc3MsLnRhYmxlPnRm
b290PnRyLnN1Y2Nlc3M+dGQsLnRhYmxlPnRmb290PnRyLnN1Y2Nlc3M+dGgsLnRhYmxlPnRmb290
PnRyPnRkLnN1Y2Nlc3MsLnRhYmxlPnRmb290PnRyPnRoLnN1Y2Nlc3MsLnRhYmxlPnRoZWFkPnRy
LnN1Y2Nlc3M+dGQsLnRhYmxlPnRoZWFkPnRyLnN1Y2Nlc3M+dGgsLnRhYmxlPnRoZWFkPnRyPnRk
LnN1Y2Nlc3MsLnRhYmxlPnRoZWFkPnRyPnRoLnN1Y2Nlc3N7YmFja2dyb3VuZC1jb2xvcjojZGZm
MGQ4fS50YWJsZS1ob3Zlcj50Ym9keT50ci5zdWNjZXNzOmhvdmVyPnRkLC50YWJsZS1ob3Zlcj50
Ym9keT50ci5zdWNjZXNzOmhvdmVyPnRoLC50YWJsZS1ob3Zlcj50Ym9keT50cjpob3Zlcj4uc3Vj
Y2VzcywudGFibGUtaG92ZXI+dGJvZHk+dHI+dGQuc3VjY2Vzczpob3ZlciwudGFibGUtaG92ZXI+
dGJvZHk+dHI+dGguc3VjY2Vzczpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNkMGU5YzZ9LnRhYmxl
PnRib2R5PnRyLmluZm8+dGQsLnRhYmxlPnRib2R5PnRyLmluZm8+dGgsLnRhYmxlPnRib2R5PnRy
PnRkLmluZm8sLnRhYmxlPnRib2R5PnRyPnRoLmluZm8sLnRhYmxlPnRmb290PnRyLmluZm8+dGQs
LnRhYmxlPnRmb290PnRyLmluZm8+dGgsLnRhYmxlPnRmb290PnRyPnRkLmluZm8sLnRhYmxlPnRm
b290PnRyPnRoLmluZm8sLnRhYmxlPnRoZWFkPnRyLmluZm8+dGQsLnRhYmxlPnRoZWFkPnRyLmlu
Zm8+dGgsLnRhYmxlPnRoZWFkPnRyPnRkLmluZm8sLnRhYmxlPnRoZWFkPnRyPnRoLmluZm97YmFj
a2dyb3VuZC1jb2xvcjojZDllZGY3fS50YWJsZS1ob3Zlcj50Ym9keT50ci5pbmZvOmhvdmVyPnRk
LC50YWJsZS1ob3Zlcj50Ym9keT50ci5pbmZvOmhvdmVyPnRoLC50YWJsZS1ob3Zlcj50Ym9keT50
cjpob3Zlcj4uaW5mbywudGFibGUtaG92ZXI+dGJvZHk+dHI+dGQuaW5mbzpob3ZlciwudGFibGUt
aG92ZXI+dGJvZHk+dHI+dGguaW5mbzpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNjNGUzZjN9LnRh
YmxlPnRib2R5PnRyLndhcm5pbmc+dGQsLnRhYmxlPnRib2R5PnRyLndhcm5pbmc+dGgsLnRhYmxl
PnRib2R5PnRyPnRkLndhcm5pbmcsLnRhYmxlPnRib2R5PnRyPnRoLndhcm5pbmcsLnRhYmxlPnRm
b290PnRyLndhcm5pbmc+dGQsLnRhYmxlPnRmb290PnRyLndhcm5pbmc+dGgsLnRhYmxlPnRmb290
PnRyPnRkLndhcm5pbmcsLnRhYmxlPnRmb290PnRyPnRoLndhcm5pbmcsLnRhYmxlPnRoZWFkPnRy
Lndhcm5pbmc+dGQsLnRhYmxlPnRoZWFkPnRyLndhcm5pbmc+dGgsLnRhYmxlPnRoZWFkPnRyPnRk
Lndhcm5pbmcsLnRhYmxlPnRoZWFkPnRyPnRoLndhcm5pbmd7YmFja2dyb3VuZC1jb2xvcjojZmNm
OGUzfS50YWJsZS1ob3Zlcj50Ym9keT50ci53YXJuaW5nOmhvdmVyPnRkLC50YWJsZS1ob3Zlcj50
Ym9keT50ci53YXJuaW5nOmhvdmVyPnRoLC50YWJsZS1ob3Zlcj50Ym9keT50cjpob3Zlcj4ud2Fy
bmluZywudGFibGUtaG92ZXI+dGJvZHk+dHI+dGQud2FybmluZzpob3ZlciwudGFibGUtaG92ZXI+
dGJvZHk+dHI+dGgud2FybmluZzpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNmYWYyY2N9LnRhYmxl
PnRib2R5PnRyLmRhbmdlcj50ZCwudGFibGU+dGJvZHk+dHIuZGFuZ2VyPnRoLC50YWJsZT50Ym9k
eT50cj50ZC5kYW5nZXIsLnRhYmxlPnRib2R5PnRyPnRoLmRhbmdlciwudGFibGU+dGZvb3Q+dHIu
ZGFuZ2VyPnRkLC50YWJsZT50Zm9vdD50ci5kYW5nZXI+dGgsLnRhYmxlPnRmb290PnRyPnRkLmRh
bmdlciwudGFibGU+dGZvb3Q+dHI+dGguZGFuZ2VyLC50YWJsZT50aGVhZD50ci5kYW5nZXI+dGQs
LnRhYmxlPnRoZWFkPnRyLmRhbmdlcj50aCwudGFibGU+dGhlYWQ+dHI+dGQuZGFuZ2VyLC50YWJs
ZT50aGVhZD50cj50aC5kYW5nZXJ7YmFja2dyb3VuZC1jb2xvcjojZjJkZWRlfS50YWJsZS1ob3Zl
cj50Ym9keT50ci5kYW5nZXI6aG92ZXI+dGQsLnRhYmxlLWhvdmVyPnRib2R5PnRyLmRhbmdlcjpo
b3Zlcj50aCwudGFibGUtaG92ZXI+dGJvZHk+dHI6aG92ZXI+LmRhbmdlciwudGFibGUtaG92ZXI+
dGJvZHk+dHI+dGQuZGFuZ2VyOmhvdmVyLC50YWJsZS1ob3Zlcj50Ym9keT50cj50aC5kYW5nZXI6
aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojZWJjY2NjfS50YWJsZS1yZXNwb25zaXZle21pbi1oZWln
aHQ6LjAxJTtvdmVyZmxvdy14OmF1dG99QG1lZGlhIHNjcmVlbiBhbmQgKG1heC13aWR0aDo3Njdw
eCl7LnRhYmxlLXJlc3BvbnNpdmV7d2lkdGg6MTAwJTttYXJnaW4tYm90dG9tOjE1cHg7b3ZlcmZs
b3cteTpoaWRkZW47LW1zLW92ZXJmbG93LXN0eWxlOi1tcy1hdXRvaGlkaW5nLXNjcm9sbGJhcjti
b3JkZXI6MXB4IHNvbGlkICNkZGR9LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxle21hcmdpbi1ib3R0
b206MH0udGFibGUtcmVzcG9uc2l2ZT4udGFibGU+dGJvZHk+dHI+dGQsLnRhYmxlLXJlc3BvbnNp
dmU+LnRhYmxlPnRib2R5PnRyPnRoLC50YWJsZS1yZXNwb25zaXZlPi50YWJsZT50Zm9vdD50cj50
ZCwudGFibGUtcmVzcG9uc2l2ZT4udGFibGU+dGZvb3Q+dHI+dGgsLnRhYmxlLXJlc3BvbnNpdmU+
LnRhYmxlPnRoZWFkPnRyPnRkLC50YWJsZS1yZXNwb25zaXZlPi50YWJsZT50aGVhZD50cj50aHt3
aGl0ZS1zcGFjZTpub3dyYXB9LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVke2JvcmRl
cjowfS50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50Ym9keT50cj50ZDpmaXJzdC1j
aGlsZCwudGFibGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGJvZHk+dHI+dGg6Zmlyc3Qt
Y2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyPnRkOmZpcnN0
LWNoaWxkLC50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50Zm9vdD50cj50aDpmaXJz
dC1jaGlsZCwudGFibGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGhlYWQ+dHI+dGQ6Zmly
c3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRyPnRoOmZp
cnN0LWNoaWxke2JvcmRlci1sZWZ0OjB9LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRib2R5PnRyPnRkOmxhc3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRib2R5PnRyPnRoOmxhc3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRmb290PnRyPnRkOmxhc3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRmb290PnRyPnRoOmxhc3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRoZWFkPnRyPnRkOmxhc3QtY2hpbGQsLnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVk
PnRoZWFkPnRyPnRoOmxhc3QtY2hpbGR7Ym9yZGVyLXJpZ2h0OjB9LnRhYmxlLXJlc3BvbnNpdmU+
LnRhYmxlLWJvcmRlcmVkPnRib2R5PnRyOmxhc3QtY2hpbGQ+dGQsLnRhYmxlLXJlc3BvbnNpdmU+
LnRhYmxlLWJvcmRlcmVkPnRib2R5PnRyOmxhc3QtY2hpbGQ+dGgsLnRhYmxlLXJlc3BvbnNpdmU+
LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyOmxhc3QtY2hpbGQ+dGQsLnRhYmxlLXJlc3BvbnNpdmU+
LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyOmxhc3QtY2hpbGQ+dGh7Ym9yZGVyLWJvdHRvbTowfX1m
aWVsZHNldHttaW4td2lkdGg6MDtwYWRkaW5nOjA7bWFyZ2luOjA7Ym9yZGVyOjB9bGVnZW5ke2Rp
c3BsYXk6YmxvY2s7d2lkdGg6MTAwJTtwYWRkaW5nOjA7bWFyZ2luLWJvdHRvbToyMHB4O2ZvbnQt
c2l6ZToyMXB4O2xpbmUtaGVpZ2h0OmluaGVyaXQ7Y29sb3I6IzMzMztib3JkZXI6MDtib3JkZXIt
Ym90dG9tOjFweCBzb2xpZCAjZTVlNWU1fWxhYmVse2Rpc3BsYXk6aW5saW5lLWJsb2NrO21heC13
aWR0aDoxMDAlO21hcmdpbi1ib3R0b206NXB4O2ZvbnQtd2VpZ2h0OjcwMH1pbnB1dFt0eXBlPXNl
YXJjaF17LXdlYmtpdC1ib3gtc2l6aW5nOmJvcmRlci1ib3g7LW1vei1ib3gtc2l6aW5nOmJvcmRl
ci1ib3g7Ym94LXNpemluZzpib3JkZXItYm94fWlucHV0W3R5cGU9Y2hlY2tib3hdLGlucHV0W3R5
cGU9cmFkaW9de21hcmdpbjo0cHggMCAwO21hcmdpbi10b3A6MXB4XDk7bGluZS1oZWlnaHQ6bm9y
bWFsfWlucHV0W3R5cGU9ZmlsZV17ZGlzcGxheTpibG9ja31pbnB1dFt0eXBlPXJhbmdlXXtkaXNw
bGF5OmJsb2NrO3dpZHRoOjEwMCV9c2VsZWN0W211bHRpcGxlXSxzZWxlY3Rbc2l6ZV17aGVpZ2h0
OmF1dG99aW5wdXRbdHlwZT1maWxlXTpmb2N1cyxpbnB1dFt0eXBlPWNoZWNrYm94XTpmb2N1cyxp
bnB1dFt0eXBlPXJhZGlvXTpmb2N1c3tvdXRsaW5lOjVweCBhdXRvIC13ZWJraXQtZm9jdXMtcmlu
Zy1jb2xvcjtvdXRsaW5lLW9mZnNldDotMnB4fW91dHB1dHtkaXNwbGF5OmJsb2NrO3BhZGRpbmct
dG9wOjdweDtmb250LXNpemU6MTRweDtsaW5lLWhlaWdodDoxLjQyODU3MTQzO2NvbG9yOiM1NTV9
LmZvcm0tY29udHJvbHtkaXNwbGF5OmJsb2NrO3dpZHRoOjEwMCU7aGVpZ2h0OjM0cHg7cGFkZGlu
Zzo2cHggMTJweDtmb250LXNpemU6MTRweDtsaW5lLWhlaWdodDoxLjQyODU3MTQzO2NvbG9yOiM1
NTU7YmFja2dyb3VuZC1jb2xvcjojZmZmO2JhY2tncm91bmQtaW1hZ2U6bm9uZTtib3JkZXI6MXB4
IHNvbGlkICNjY2M7Ym9yZGVyLXJhZGl1czo0cHg7LXdlYmtpdC1ib3gtc2hhZG93Omluc2V0IDAg
MXB4IDFweCByZ2JhKDAsMCwwLC4wNzUpO2JveC1zaGFkb3c6aW5zZXQgMCAxcHggMXB4IHJnYmEo
MCwwLDAsLjA3NSk7LXdlYmtpdC10cmFuc2l0aW9uOmJvcmRlci1jb2xvciBlYXNlLWluLW91dCAu
MTVzLC13ZWJraXQtYm94LXNoYWRvdyBlYXNlLWluLW91dCAuMTVzOy1vLXRyYW5zaXRpb246Ym9y
ZGVyLWNvbG9yIGVhc2UtaW4tb3V0IC4xNXMsYm94LXNoYWRvdyBlYXNlLWluLW91dCAuMTVzO3Ry
YW5zaXRpb246Ym9yZGVyLWNvbG9yIGVhc2UtaW4tb3V0IC4xNXMsYm94LXNoYWRvdyBlYXNlLWlu
LW91dCAuMTVzfS5mb3JtLWNvbnRyb2w6Zm9jdXN7Ym9yZGVyLWNvbG9yOiM2NmFmZTk7b3V0bGlu
ZTowOy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFweCAxcHggcmdiYSgwLDAsMCwuMDc1KSww
IDAgOHB4IHJnYmEoMTAyLDE3NSwyMzMsLjYpO2JveC1zaGFkb3c6aW5zZXQgMCAxcHggMXB4IHJn
YmEoMCwwLDAsLjA3NSksMCAwIDhweCByZ2JhKDEwMiwxNzUsMjMzLC42KX0uZm9ybS1jb250cm9s
OjotbW96LXBsYWNlaG9sZGVye2NvbG9yOiM5OTk7b3BhY2l0eToxfS5mb3JtLWNvbnRyb2w6LW1z
LWlucHV0LXBsYWNlaG9sZGVye2NvbG9yOiM5OTl9LmZvcm0tY29udHJvbDo6LXdlYmtpdC1pbnB1
dC1wbGFjZWhvbGRlcntjb2xvcjojOTk5fS5mb3JtLWNvbnRyb2w6Oi1tcy1leHBhbmR7YmFja2dy
b3VuZC1jb2xvcjp0cmFuc3BhcmVudDtib3JkZXI6MH0uZm9ybS1jb250cm9sW2Rpc2FibGVkXSwu
Zm9ybS1jb250cm9sW3JlYWRvbmx5XSxmaWVsZHNldFtkaXNhYmxlZF0gLmZvcm0tY29udHJvbHti
YWNrZ3JvdW5kLWNvbG9yOiNlZWU7b3BhY2l0eToxfS5mb3JtLWNvbnRyb2xbZGlzYWJsZWRdLGZp
ZWxkc2V0W2Rpc2FibGVkXSAuZm9ybS1jb250cm9se2N1cnNvcjpub3QtYWxsb3dlZH10ZXh0YXJl
YS5mb3JtLWNvbnRyb2x7aGVpZ2h0OmF1dG99aW5wdXRbdHlwZT1zZWFyY2hdey13ZWJraXQtYXBw
ZWFyYW5jZTpub25lfUBtZWRpYSBzY3JlZW4gYW5kICgtd2Via2l0LW1pbi1kZXZpY2UtcGl4ZWwt
cmF0aW86MCl7aW5wdXRbdHlwZT1kYXRlXS5mb3JtLWNvbnRyb2wsaW5wdXRbdHlwZT10aW1lXS5m
b3JtLWNvbnRyb2wsaW5wdXRbdHlwZT1kYXRldGltZS1sb2NhbF0uZm9ybS1jb250cm9sLGlucHV0
W3R5cGU9bW9udGhdLmZvcm0tY29udHJvbHtsaW5lLWhlaWdodDozNHB4fS5pbnB1dC1ncm91cC1z
bSBpbnB1dFt0eXBlPWRhdGVdLC5pbnB1dC1ncm91cC1zbSBpbnB1dFt0eXBlPXRpbWVdLC5pbnB1
dC1ncm91cC1zbSBpbnB1dFt0eXBlPWRhdGV0aW1lLWxvY2FsXSwuaW5wdXQtZ3JvdXAtc20gaW5w
dXRbdHlwZT1tb250aF0saW5wdXRbdHlwZT1kYXRlXS5pbnB1dC1zbSxpbnB1dFt0eXBlPXRpbWVd
LmlucHV0LXNtLGlucHV0W3R5cGU9ZGF0ZXRpbWUtbG9jYWxdLmlucHV0LXNtLGlucHV0W3R5cGU9
bW9udGhdLmlucHV0LXNte2xpbmUtaGVpZ2h0OjMwcHh9LmlucHV0LWdyb3VwLWxnIGlucHV0W3R5
cGU9ZGF0ZV0sLmlucHV0LWdyb3VwLWxnIGlucHV0W3R5cGU9dGltZV0sLmlucHV0LWdyb3VwLWxn
IGlucHV0W3R5cGU9ZGF0ZXRpbWUtbG9jYWxdLC5pbnB1dC1ncm91cC1sZyBpbnB1dFt0eXBlPW1v
bnRoXSxpbnB1dFt0eXBlPWRhdGVdLmlucHV0LWxnLGlucHV0W3R5cGU9dGltZV0uaW5wdXQtbGcs
aW5wdXRbdHlwZT1kYXRldGltZS1sb2NhbF0uaW5wdXQtbGcsaW5wdXRbdHlwZT1tb250aF0uaW5w
dXQtbGd7bGluZS1oZWlnaHQ6NDZweH19LmZvcm0tZ3JvdXB7bWFyZ2luLWJvdHRvbToxNXB4fS5j
aGVja2JveCwucmFkaW97cG9zaXRpb246cmVsYXRpdmU7ZGlzcGxheTpibG9jazttYXJnaW4tdG9w
OjEwcHg7bWFyZ2luLWJvdHRvbToxMHB4fS5jaGVja2JveCBsYWJlbCwucmFkaW8gbGFiZWx7bWlu
LWhlaWdodDoyMHB4O3BhZGRpbmctbGVmdDoyMHB4O21hcmdpbi1ib3R0b206MDtmb250LXdlaWdo
dDo0MDA7Y3Vyc29yOnBvaW50ZXJ9LmNoZWNrYm94IGlucHV0W3R5cGU9Y2hlY2tib3hdLC5jaGVj
a2JveC1pbmxpbmUgaW5wdXRbdHlwZT1jaGVja2JveF0sLnJhZGlvIGlucHV0W3R5cGU9cmFkaW9d
LC5yYWRpby1pbmxpbmUgaW5wdXRbdHlwZT1yYWRpb117cG9zaXRpb246YWJzb2x1dGU7bWFyZ2lu
LXRvcDo0cHhcOTttYXJnaW4tbGVmdDotMjBweH0uY2hlY2tib3grLmNoZWNrYm94LC5yYWRpbysu
cmFkaW97bWFyZ2luLXRvcDotNXB4fS5jaGVja2JveC1pbmxpbmUsLnJhZGlvLWlubGluZXtwb3Np
dGlvbjpyZWxhdGl2ZTtkaXNwbGF5OmlubGluZS1ibG9jaztwYWRkaW5nLWxlZnQ6MjBweDttYXJn
aW4tYm90dG9tOjA7Zm9udC13ZWlnaHQ6NDAwO3ZlcnRpY2FsLWFsaWduOm1pZGRsZTtjdXJzb3I6
cG9pbnRlcn0uY2hlY2tib3gtaW5saW5lKy5jaGVja2JveC1pbmxpbmUsLnJhZGlvLWlubGluZSsu
cmFkaW8taW5saW5le21hcmdpbi10b3A6MDttYXJnaW4tbGVmdDoxMHB4fWZpZWxkc2V0W2Rpc2Fi
bGVkXSBpbnB1dFt0eXBlPWNoZWNrYm94XSxmaWVsZHNldFtkaXNhYmxlZF0gaW5wdXRbdHlwZT1y
YWRpb10saW5wdXRbdHlwZT1jaGVja2JveF0uZGlzYWJsZWQsaW5wdXRbdHlwZT1jaGVja2JveF1b
ZGlzYWJsZWRdLGlucHV0W3R5cGU9cmFkaW9dLmRpc2FibGVkLGlucHV0W3R5cGU9cmFkaW9dW2Rp
c2FibGVkXXtjdXJzb3I6bm90LWFsbG93ZWR9LmNoZWNrYm94LWlubGluZS5kaXNhYmxlZCwucmFk
aW8taW5saW5lLmRpc2FibGVkLGZpZWxkc2V0W2Rpc2FibGVkXSAuY2hlY2tib3gtaW5saW5lLGZp
ZWxkc2V0W2Rpc2FibGVkXSAucmFkaW8taW5saW5le2N1cnNvcjpub3QtYWxsb3dlZH0uY2hlY2ti
b3guZGlzYWJsZWQgbGFiZWwsLnJhZGlvLmRpc2FibGVkIGxhYmVsLGZpZWxkc2V0W2Rpc2FibGVk
XSAuY2hlY2tib3ggbGFiZWwsZmllbGRzZXRbZGlzYWJsZWRdIC5yYWRpbyBsYWJlbHtjdXJzb3I6
bm90LWFsbG93ZWR9LmZvcm0tY29udHJvbC1zdGF0aWN7bWluLWhlaWdodDozNHB4O3BhZGRpbmct
dG9wOjdweDtwYWRkaW5nLWJvdHRvbTo3cHg7bWFyZ2luLWJvdHRvbTowfS5mb3JtLWNvbnRyb2wt
c3RhdGljLmlucHV0LWxnLC5mb3JtLWNvbnRyb2wtc3RhdGljLmlucHV0LXNte3BhZGRpbmctcmln
aHQ6MDtwYWRkaW5nLWxlZnQ6MH0uaW5wdXQtc217aGVpZ2h0OjMwcHg7cGFkZGluZzo1cHggMTBw
eDtmb250LXNpemU6MTJweDtsaW5lLWhlaWdodDoxLjU7Ym9yZGVyLXJhZGl1czozcHh9c2VsZWN0
LmlucHV0LXNte2hlaWdodDozMHB4O2xpbmUtaGVpZ2h0OjMwcHh9c2VsZWN0W211bHRpcGxlXS5p
bnB1dC1zbSx0ZXh0YXJlYS5pbnB1dC1zbXtoZWlnaHQ6YXV0b30uZm9ybS1ncm91cC1zbSAuZm9y
bS1jb250cm9se2hlaWdodDozMHB4O3BhZGRpbmc6NXB4IDEwcHg7Zm9udC1zaXplOjEycHg7bGlu
ZS1oZWlnaHQ6MS41O2JvcmRlci1yYWRpdXM6M3B4fS5mb3JtLWdyb3VwLXNtIHNlbGVjdC5mb3Jt
LWNvbnRyb2x7aGVpZ2h0OjMwcHg7bGluZS1oZWlnaHQ6MzBweH0uZm9ybS1ncm91cC1zbSBzZWxl
Y3RbbXVsdGlwbGVdLmZvcm0tY29udHJvbCwuZm9ybS1ncm91cC1zbSB0ZXh0YXJlYS5mb3JtLWNv
bnRyb2x7aGVpZ2h0OmF1dG99LmZvcm0tZ3JvdXAtc20gLmZvcm0tY29udHJvbC1zdGF0aWN7aGVp
Z2h0OjMwcHg7bWluLWhlaWdodDozMnB4O3BhZGRpbmc6NnB4IDEwcHg7Zm9udC1zaXplOjEycHg7
bGluZS1oZWlnaHQ6MS41fS5pbnB1dC1sZ3toZWlnaHQ6NDZweDtwYWRkaW5nOjEwcHggMTZweDtm
b250LXNpemU6MThweDtsaW5lLWhlaWdodDoxLjMzMzMzMzM7Ym9yZGVyLXJhZGl1czo2cHh9c2Vs
ZWN0LmlucHV0LWxne2hlaWdodDo0NnB4O2xpbmUtaGVpZ2h0OjQ2cHh9c2VsZWN0W211bHRpcGxl
XS5pbnB1dC1sZyx0ZXh0YXJlYS5pbnB1dC1sZ3toZWlnaHQ6YXV0b30uZm9ybS1ncm91cC1sZyAu
Zm9ybS1jb250cm9se2hlaWdodDo0NnB4O3BhZGRpbmc6MTBweCAxNnB4O2ZvbnQtc2l6ZToxOHB4
O2xpbmUtaGVpZ2h0OjEuMzMzMzMzMztib3JkZXItcmFkaXVzOjZweH0uZm9ybS1ncm91cC1sZyBz
ZWxlY3QuZm9ybS1jb250cm9se2hlaWdodDo0NnB4O2xpbmUtaGVpZ2h0OjQ2cHh9LmZvcm0tZ3Jv
dXAtbGcgc2VsZWN0W211bHRpcGxlXS5mb3JtLWNvbnRyb2wsLmZvcm0tZ3JvdXAtbGcgdGV4dGFy
ZWEuZm9ybS1jb250cm9se2hlaWdodDphdXRvfS5mb3JtLWdyb3VwLWxnIC5mb3JtLWNvbnRyb2wt
c3RhdGlje2hlaWdodDo0NnB4O21pbi1oZWlnaHQ6MzhweDtwYWRkaW5nOjExcHggMTZweDtmb250
LXNpemU6MThweDtsaW5lLWhlaWdodDoxLjMzMzMzMzN9Lmhhcy1mZWVkYmFja3twb3NpdGlvbjpy
ZWxhdGl2ZX0uaGFzLWZlZWRiYWNrIC5mb3JtLWNvbnRyb2x7cGFkZGluZy1yaWdodDo0Mi41cHh9
LmZvcm0tY29udHJvbC1mZWVkYmFja3twb3NpdGlvbjphYnNvbHV0ZTt0b3A6MDtyaWdodDowO3ot
aW5kZXg6MjtkaXNwbGF5OmJsb2NrO3dpZHRoOjM0cHg7aGVpZ2h0OjM0cHg7bGluZS1oZWlnaHQ6
MzRweDt0ZXh0LWFsaWduOmNlbnRlcjtwb2ludGVyLWV2ZW50czpub25lfS5mb3JtLWdyb3VwLWxn
IC5mb3JtLWNvbnRyb2wrLmZvcm0tY29udHJvbC1mZWVkYmFjaywuaW5wdXQtZ3JvdXAtbGcrLmZv
cm0tY29udHJvbC1mZWVkYmFjaywuaW5wdXQtbGcrLmZvcm0tY29udHJvbC1mZWVkYmFja3t3aWR0
aDo0NnB4O2hlaWdodDo0NnB4O2xpbmUtaGVpZ2h0OjQ2cHh9LmZvcm0tZ3JvdXAtc20gLmZvcm0t
Y29udHJvbCsuZm9ybS1jb250cm9sLWZlZWRiYWNrLC5pbnB1dC1ncm91cC1zbSsuZm9ybS1jb250
cm9sLWZlZWRiYWNrLC5pbnB1dC1zbSsuZm9ybS1jb250cm9sLWZlZWRiYWNre3dpZHRoOjMwcHg7
aGVpZ2h0OjMwcHg7bGluZS1oZWlnaHQ6MzBweH0uaGFzLXN1Y2Nlc3MgLmNoZWNrYm94LC5oYXMt
c3VjY2VzcyAuY2hlY2tib3gtaW5saW5lLC5oYXMtc3VjY2VzcyAuY29udHJvbC1sYWJlbCwuaGFz
LXN1Y2Nlc3MgLmhlbHAtYmxvY2ssLmhhcy1zdWNjZXNzIC5yYWRpbywuaGFzLXN1Y2Nlc3MgLnJh
ZGlvLWlubGluZSwuaGFzLXN1Y2Nlc3MuY2hlY2tib3ggbGFiZWwsLmhhcy1zdWNjZXNzLmNoZWNr
Ym94LWlubGluZSBsYWJlbCwuaGFzLXN1Y2Nlc3MucmFkaW8gbGFiZWwsLmhhcy1zdWNjZXNzLnJh
ZGlvLWlubGluZSBsYWJlbHtjb2xvcjojM2M3NjNkfS5oYXMtc3VjY2VzcyAuZm9ybS1jb250cm9s
e2JvcmRlci1jb2xvcjojM2M3NjNkOy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFweCAxcHgg
cmdiYSgwLDAsMCwuMDc1KTtib3gtc2hhZG93Omluc2V0IDAgMXB4IDFweCByZ2JhKDAsMCwwLC4w
NzUpfS5oYXMtc3VjY2VzcyAuZm9ybS1jb250cm9sOmZvY3Vze2JvcmRlci1jb2xvcjojMmI1NDJj
Oy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFweCAxcHggcmdiYSgwLDAsMCwuMDc1KSwwIDAg
NnB4ICM2N2IxNjg7Ym94LXNoYWRvdzppbnNldCAwIDFweCAxcHggcmdiYSgwLDAsMCwuMDc1KSww
IDAgNnB4ICM2N2IxNjh9Lmhhcy1zdWNjZXNzIC5pbnB1dC1ncm91cC1hZGRvbntjb2xvcjojM2M3
NjNkO2JhY2tncm91bmQtY29sb3I6I2RmZjBkODtib3JkZXItY29sb3I6IzNjNzYzZH0uaGFzLXN1
Y2Nlc3MgLmZvcm0tY29udHJvbC1mZWVkYmFja3tjb2xvcjojM2M3NjNkfS5oYXMtd2FybmluZyAu
Y2hlY2tib3gsLmhhcy13YXJuaW5nIC5jaGVja2JveC1pbmxpbmUsLmhhcy13YXJuaW5nIC5jb250
cm9sLWxhYmVsLC5oYXMtd2FybmluZyAuaGVscC1ibG9jaywuaGFzLXdhcm5pbmcgLnJhZGlvLC5o
YXMtd2FybmluZyAucmFkaW8taW5saW5lLC5oYXMtd2FybmluZy5jaGVja2JveCBsYWJlbCwuaGFz
LXdhcm5pbmcuY2hlY2tib3gtaW5saW5lIGxhYmVsLC5oYXMtd2FybmluZy5yYWRpbyBsYWJlbCwu
aGFzLXdhcm5pbmcucmFkaW8taW5saW5lIGxhYmVse2NvbG9yOiM4YTZkM2J9Lmhhcy13YXJuaW5n
IC5mb3JtLWNvbnRyb2x7Ym9yZGVyLWNvbG9yOiM4YTZkM2I7LXdlYmtpdC1ib3gtc2hhZG93Omlu
c2V0IDAgMXB4IDFweCByZ2JhKDAsMCwwLC4wNzUpO2JveC1zaGFkb3c6aW5zZXQgMCAxcHggMXB4
IHJnYmEoMCwwLDAsLjA3NSl9Lmhhcy13YXJuaW5nIC5mb3JtLWNvbnRyb2w6Zm9jdXN7Ym9yZGVy
LWNvbG9yOiM2NjUxMmM7LXdlYmtpdC1ib3gtc2hhZG93Omluc2V0IDAgMXB4IDFweCByZ2JhKDAs
MCwwLC4wNzUpLDAgMCA2cHggI2MwYTE2Yjtib3gtc2hhZG93Omluc2V0IDAgMXB4IDFweCByZ2Jh
KDAsMCwwLC4wNzUpLDAgMCA2cHggI2MwYTE2Yn0uaGFzLXdhcm5pbmcgLmlucHV0LWdyb3VwLWFk
ZG9ue2NvbG9yOiM4YTZkM2I7YmFja2dyb3VuZC1jb2xvcjojZmNmOGUzO2JvcmRlci1jb2xvcjoj
OGE2ZDNifS5oYXMtd2FybmluZyAuZm9ybS1jb250cm9sLWZlZWRiYWNre2NvbG9yOiM4YTZkM2J9
Lmhhcy1lcnJvciAuY2hlY2tib3gsLmhhcy1lcnJvciAuY2hlY2tib3gtaW5saW5lLC5oYXMtZXJy
b3IgLmNvbnRyb2wtbGFiZWwsLmhhcy1lcnJvciAuaGVscC1ibG9jaywuaGFzLWVycm9yIC5yYWRp
bywuaGFzLWVycm9yIC5yYWRpby1pbmxpbmUsLmhhcy1lcnJvci5jaGVja2JveCBsYWJlbCwuaGFz
LWVycm9yLmNoZWNrYm94LWlubGluZSBsYWJlbCwuaGFzLWVycm9yLnJhZGlvIGxhYmVsLC5oYXMt
ZXJyb3IucmFkaW8taW5saW5lIGxhYmVse2NvbG9yOiNhOTQ0NDJ9Lmhhcy1lcnJvciAuZm9ybS1j
b250cm9se2JvcmRlci1jb2xvcjojYTk0NDQyOy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFw
eCAxcHggcmdiYSgwLDAsMCwuMDc1KTtib3gtc2hhZG93Omluc2V0IDAgMXB4IDFweCByZ2JhKDAs
MCwwLC4wNzUpfS5oYXMtZXJyb3IgLmZvcm0tY29udHJvbDpmb2N1c3tib3JkZXItY29sb3I6Izg0
MzUzNDstd2Via2l0LWJveC1zaGFkb3c6aW5zZXQgMCAxcHggMXB4IHJnYmEoMCwwLDAsLjA3NSks
MCAwIDZweCAjY2U4NDgzO2JveC1zaGFkb3c6aW5zZXQgMCAxcHggMXB4IHJnYmEoMCwwLDAsLjA3
NSksMCAwIDZweCAjY2U4NDgzfS5oYXMtZXJyb3IgLmlucHV0LWdyb3VwLWFkZG9ue2NvbG9yOiNh
OTQ0NDI7YmFja2dyb3VuZC1jb2xvcjojZjJkZWRlO2JvcmRlci1jb2xvcjojYTk0NDQyfS5oYXMt
ZXJyb3IgLmZvcm0tY29udHJvbC1mZWVkYmFja3tjb2xvcjojYTk0NDQyfS5oYXMtZmVlZGJhY2sg
bGFiZWx+LmZvcm0tY29udHJvbC1mZWVkYmFja3t0b3A6MjVweH0uaGFzLWZlZWRiYWNrIGxhYmVs
LnNyLW9ubHl+LmZvcm0tY29udHJvbC1mZWVkYmFja3t0b3A6MH0uaGVscC1ibG9ja3tkaXNwbGF5
OmJsb2NrO21hcmdpbi10b3A6NXB4O21hcmdpbi1ib3R0b206MTBweDtjb2xvcjojNzM3MzczfUBt
ZWRpYSAobWluLXdpZHRoOjc2OHB4KXsuZm9ybS1pbmxpbmUgLmZvcm0tZ3JvdXB7ZGlzcGxheTpp
bmxpbmUtYmxvY2s7bWFyZ2luLWJvdHRvbTowO3ZlcnRpY2FsLWFsaWduOm1pZGRsZX0uZm9ybS1p
bmxpbmUgLmZvcm0tY29udHJvbHtkaXNwbGF5OmlubGluZS1ibG9jazt3aWR0aDphdXRvO3ZlcnRp
Y2FsLWFsaWduOm1pZGRsZX0uZm9ybS1pbmxpbmUgLmZvcm0tY29udHJvbC1zdGF0aWN7ZGlzcGxh
eTppbmxpbmUtYmxvY2t9LmZvcm0taW5saW5lIC5pbnB1dC1ncm91cHtkaXNwbGF5OmlubGluZS10
YWJsZTt2ZXJ0aWNhbC1hbGlnbjptaWRkbGV9LmZvcm0taW5saW5lIC5pbnB1dC1ncm91cCAuZm9y
bS1jb250cm9sLC5mb3JtLWlubGluZSAuaW5wdXQtZ3JvdXAgLmlucHV0LWdyb3VwLWFkZG9uLC5m
b3JtLWlubGluZSAuaW5wdXQtZ3JvdXAgLmlucHV0LWdyb3VwLWJ0bnt3aWR0aDphdXRvfS5mb3Jt
LWlubGluZSAuaW5wdXQtZ3JvdXA+LmZvcm0tY29udHJvbHt3aWR0aDoxMDAlfS5mb3JtLWlubGlu
ZSAuY29udHJvbC1sYWJlbHttYXJnaW4tYm90dG9tOjA7dmVydGljYWwtYWxpZ246bWlkZGxlfS5m
b3JtLWlubGluZSAuY2hlY2tib3gsLmZvcm0taW5saW5lIC5yYWRpb3tkaXNwbGF5OmlubGluZS1i
bG9jazttYXJnaW4tdG9wOjA7bWFyZ2luLWJvdHRvbTowO3ZlcnRpY2FsLWFsaWduOm1pZGRsZX0u
Zm9ybS1pbmxpbmUgLmNoZWNrYm94IGxhYmVsLC5mb3JtLWlubGluZSAucmFkaW8gbGFiZWx7cGFk
ZGluZy1sZWZ0OjB9LmZvcm0taW5saW5lIC5jaGVja2JveCBpbnB1dFt0eXBlPWNoZWNrYm94XSwu
Zm9ybS1pbmxpbmUgLnJhZGlvIGlucHV0W3R5cGU9cmFkaW9de3Bvc2l0aW9uOnJlbGF0aXZlO21h
cmdpbi1sZWZ0OjB9LmZvcm0taW5saW5lIC5oYXMtZmVlZGJhY2sgLmZvcm0tY29udHJvbC1mZWVk
YmFja3t0b3A6MH19LmZvcm0taG9yaXpvbnRhbCAuY2hlY2tib3gsLmZvcm0taG9yaXpvbnRhbCAu
Y2hlY2tib3gtaW5saW5lLC5mb3JtLWhvcml6b250YWwgLnJhZGlvLC5mb3JtLWhvcml6b250YWwg
LnJhZGlvLWlubGluZXtwYWRkaW5nLXRvcDo3cHg7bWFyZ2luLXRvcDowO21hcmdpbi1ib3R0b206
MH0uZm9ybS1ob3Jpem9udGFsIC5jaGVja2JveCwuZm9ybS1ob3Jpem9udGFsIC5yYWRpb3ttaW4t
aGVpZ2h0OjI3cHh9LmZvcm0taG9yaXpvbnRhbCAuZm9ybS1ncm91cHttYXJnaW4tcmlnaHQ6LTE1
cHg7bWFyZ2luLWxlZnQ6LTE1cHh9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5mb3JtLWhvcml6
b250YWwgLmNvbnRyb2wtbGFiZWx7cGFkZGluZy10b3A6N3B4O21hcmdpbi1ib3R0b206MDt0ZXh0
LWFsaWduOnJpZ2h0fX0uZm9ybS1ob3Jpem9udGFsIC5oYXMtZmVlZGJhY2sgLmZvcm0tY29udHJv
bC1mZWVkYmFja3tyaWdodDoxNXB4fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsuZm9ybS1ob3Jp
em9udGFsIC5mb3JtLWdyb3VwLWxnIC5jb250cm9sLWxhYmVse3BhZGRpbmctdG9wOjExcHg7Zm9u
dC1zaXplOjE4cHh9fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsuZm9ybS1ob3Jpem9udGFsIC5m
b3JtLWdyb3VwLXNtIC5jb250cm9sLWxhYmVse3BhZGRpbmctdG9wOjZweDtmb250LXNpemU6MTJw
eH19LmJ0bntkaXNwbGF5OmlubGluZS1ibG9jaztwYWRkaW5nOjZweCAxMnB4O21hcmdpbi1ib3R0
b206MDtmb250LXNpemU6MTRweDtmb250LXdlaWdodDo0MDA7bGluZS1oZWlnaHQ6MS40Mjg1NzE0
Mzt0ZXh0LWFsaWduOmNlbnRlcjt3aGl0ZS1zcGFjZTpub3dyYXA7dmVydGljYWwtYWxpZ246bWlk
ZGxlOy1tcy10b3VjaC1hY3Rpb246bWFuaXB1bGF0aW9uO3RvdWNoLWFjdGlvbjptYW5pcHVsYXRp
b247Y3Vyc29yOnBvaW50ZXI7LXdlYmtpdC11c2VyLXNlbGVjdDpub25lOy1tb3otdXNlci1zZWxl
Y3Q6bm9uZTstbXMtdXNlci1zZWxlY3Q6bm9uZTt1c2VyLXNlbGVjdDpub25lO2JhY2tncm91bmQt
aW1hZ2U6bm9uZTtib3JkZXI6MXB4IHNvbGlkIHRyYW5zcGFyZW50O2JvcmRlci1yYWRpdXM6NHB4
fS5idG4uYWN0aXZlLmZvY3VzLC5idG4uYWN0aXZlOmZvY3VzLC5idG4uZm9jdXMsLmJ0bjphY3Rp
dmUuZm9jdXMsLmJ0bjphY3RpdmU6Zm9jdXMsLmJ0bjpmb2N1c3tvdXRsaW5lOjVweCBhdXRvIC13
ZWJraXQtZm9jdXMtcmluZy1jb2xvcjtvdXRsaW5lLW9mZnNldDotMnB4fS5idG4uZm9jdXMsLmJ0
bjpmb2N1cywuYnRuOmhvdmVye2NvbG9yOiMzMzM7dGV4dC1kZWNvcmF0aW9uOm5vbmV9LmJ0bi5h
Y3RpdmUsLmJ0bjphY3RpdmV7YmFja2dyb3VuZC1pbWFnZTpub25lO291dGxpbmU6MDstd2Via2l0
LWJveC1zaGFkb3c6aW5zZXQgMCAzcHggNXB4IHJnYmEoMCwwLDAsLjEyNSk7Ym94LXNoYWRvdzpp
bnNldCAwIDNweCA1cHggcmdiYSgwLDAsMCwuMTI1KX0uYnRuLmRpc2FibGVkLC5idG5bZGlzYWJs
ZWRdLGZpZWxkc2V0W2Rpc2FibGVkXSAuYnRue2N1cnNvcjpub3QtYWxsb3dlZDtmaWx0ZXI6YWxw
aGEob3BhY2l0eT02NSk7LXdlYmtpdC1ib3gtc2hhZG93Om5vbmU7Ym94LXNoYWRvdzpub25lO29w
YWNpdHk6LjY1fWEuYnRuLmRpc2FibGVkLGZpZWxkc2V0W2Rpc2FibGVkXSBhLmJ0bntwb2ludGVy
LWV2ZW50czpub25lfS5idG4tZGVmYXVsdHtjb2xvcjojMzMzO2JhY2tncm91bmQtY29sb3I6I2Zm
Zjtib3JkZXItY29sb3I6I2NjY30uYnRuLWRlZmF1bHQuZm9jdXMsLmJ0bi1kZWZhdWx0OmZvY3Vz
e2NvbG9yOiMzMzM7YmFja2dyb3VuZC1jb2xvcjojZTZlNmU2O2JvcmRlci1jb2xvcjojOGM4Yzhj
fS5idG4tZGVmYXVsdDpob3Zlcntjb2xvcjojMzMzO2JhY2tncm91bmQtY29sb3I6I2U2ZTZlNjti
b3JkZXItY29sb3I6I2FkYWRhZH0uYnRuLWRlZmF1bHQuYWN0aXZlLC5idG4tZGVmYXVsdDphY3Rp
dmUsLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4tZGVmYXVsdHtjb2xvcjojMzMzO2JhY2tncm91
bmQtY29sb3I6I2U2ZTZlNjtib3JkZXItY29sb3I6I2FkYWRhZH0uYnRuLWRlZmF1bHQuYWN0aXZl
LmZvY3VzLC5idG4tZGVmYXVsdC5hY3RpdmU6Zm9jdXMsLmJ0bi1kZWZhdWx0LmFjdGl2ZTpob3Zl
ciwuYnRuLWRlZmF1bHQ6YWN0aXZlLmZvY3VzLC5idG4tZGVmYXVsdDphY3RpdmU6Zm9jdXMsLmJ0
bi1kZWZhdWx0OmFjdGl2ZTpob3Zlciwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi1kZWZhdWx0
LmZvY3VzLC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRuLWRlZmF1bHQ6Zm9jdXMsLm9wZW4+LmRy
b3Bkb3duLXRvZ2dsZS5idG4tZGVmYXVsdDpob3Zlcntjb2xvcjojMzMzO2JhY2tncm91bmQtY29s
b3I6I2Q0ZDRkNDtib3JkZXItY29sb3I6IzhjOGM4Y30uYnRuLWRlZmF1bHQuYWN0aXZlLC5idG4t
ZGVmYXVsdDphY3RpdmUsLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4tZGVmYXVsdHtiYWNrZ3Jv
dW5kLWltYWdlOm5vbmV9LmJ0bi1kZWZhdWx0LmRpc2FibGVkLmZvY3VzLC5idG4tZGVmYXVsdC5k
aXNhYmxlZDpmb2N1cywuYnRuLWRlZmF1bHQuZGlzYWJsZWQ6aG92ZXIsLmJ0bi1kZWZhdWx0W2Rp
c2FibGVkXS5mb2N1cywuYnRuLWRlZmF1bHRbZGlzYWJsZWRdOmZvY3VzLC5idG4tZGVmYXVsdFtk
aXNhYmxlZF06aG92ZXIsZmllbGRzZXRbZGlzYWJsZWRdIC5idG4tZGVmYXVsdC5mb2N1cyxmaWVs
ZHNldFtkaXNhYmxlZF0gLmJ0bi1kZWZhdWx0OmZvY3VzLGZpZWxkc2V0W2Rpc2FibGVkXSAuYnRu
LWRlZmF1bHQ6aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojZmZmO2JvcmRlci1jb2xvcjojY2NjfS5i
dG4tZGVmYXVsdCAuYmFkZ2V7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzMzN9LmJ0bi1w
cmltYXJ5e2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojMzM3YWI3O2JvcmRlci1jb2xvcjoj
MmU2ZGE0fS5idG4tcHJpbWFyeS5mb2N1cywuYnRuLXByaW1hcnk6Zm9jdXN7Y29sb3I6I2ZmZjti
YWNrZ3JvdW5kLWNvbG9yOiMyODYwOTA7Ym9yZGVyLWNvbG9yOiMxMjJiNDB9LmJ0bi1wcmltYXJ5
OmhvdmVye2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojMjg2MDkwO2JvcmRlci1jb2xvcjoj
MjA0ZDc0fS5idG4tcHJpbWFyeS5hY3RpdmUsLmJ0bi1wcmltYXJ5OmFjdGl2ZSwub3Blbj4uZHJv
cGRvd24tdG9nZ2xlLmJ0bi1wcmltYXJ5e2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojMjg2
MDkwO2JvcmRlci1jb2xvcjojMjA0ZDc0fS5idG4tcHJpbWFyeS5hY3RpdmUuZm9jdXMsLmJ0bi1w
cmltYXJ5LmFjdGl2ZTpmb2N1cywuYnRuLXByaW1hcnkuYWN0aXZlOmhvdmVyLC5idG4tcHJpbWFy
eTphY3RpdmUuZm9jdXMsLmJ0bi1wcmltYXJ5OmFjdGl2ZTpmb2N1cywuYnRuLXByaW1hcnk6YWN0
aXZlOmhvdmVyLC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRuLXByaW1hcnkuZm9jdXMsLm9wZW4+
LmRyb3Bkb3duLXRvZ2dsZS5idG4tcHJpbWFyeTpmb2N1cywub3Blbj4uZHJvcGRvd24tdG9nZ2xl
LmJ0bi1wcmltYXJ5OmhvdmVye2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojMjA0ZDc0O2Jv
cmRlci1jb2xvcjojMTIyYjQwfS5idG4tcHJpbWFyeS5hY3RpdmUsLmJ0bi1wcmltYXJ5OmFjdGl2
ZSwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi1wcmltYXJ5e2JhY2tncm91bmQtaW1hZ2U6bm9u
ZX0uYnRuLXByaW1hcnkuZGlzYWJsZWQuZm9jdXMsLmJ0bi1wcmltYXJ5LmRpc2FibGVkOmZvY3Vz
LC5idG4tcHJpbWFyeS5kaXNhYmxlZDpob3ZlciwuYnRuLXByaW1hcnlbZGlzYWJsZWRdLmZvY3Vz
LC5idG4tcHJpbWFyeVtkaXNhYmxlZF06Zm9jdXMsLmJ0bi1wcmltYXJ5W2Rpc2FibGVkXTpob3Zl
cixmaWVsZHNldFtkaXNhYmxlZF0gLmJ0bi1wcmltYXJ5LmZvY3VzLGZpZWxkc2V0W2Rpc2FibGVk
XSAuYnRuLXByaW1hcnk6Zm9jdXMsZmllbGRzZXRbZGlzYWJsZWRdIC5idG4tcHJpbWFyeTpob3Zl
cntiYWNrZ3JvdW5kLWNvbG9yOiMzMzdhYjc7Ym9yZGVyLWNvbG9yOiMyZTZkYTR9LmJ0bi1wcmlt
YXJ5IC5iYWRnZXtjb2xvcjojMzM3YWI3O2JhY2tncm91bmQtY29sb3I6I2ZmZn0uYnRuLXN1Y2Nl
c3N7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiM1Y2I4NWM7Ym9yZGVyLWNvbG9yOiM0Y2Fl
NGN9LmJ0bi1zdWNjZXNzLmZvY3VzLC5idG4tc3VjY2Vzczpmb2N1c3tjb2xvcjojZmZmO2JhY2tn
cm91bmQtY29sb3I6IzQ0OWQ0NDtib3JkZXItY29sb3I6IzI1NTYyNX0uYnRuLXN1Y2Nlc3M6aG92
ZXJ7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiM0NDlkNDQ7Ym9yZGVyLWNvbG9yOiMzOTg0
Mzl9LmJ0bi1zdWNjZXNzLmFjdGl2ZSwuYnRuLXN1Y2Nlc3M6YWN0aXZlLC5vcGVuPi5kcm9wZG93
bi10b2dnbGUuYnRuLXN1Y2Nlc3N7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiM0NDlkNDQ7
Ym9yZGVyLWNvbG9yOiMzOTg0Mzl9LmJ0bi1zdWNjZXNzLmFjdGl2ZS5mb2N1cywuYnRuLXN1Y2Nl
c3MuYWN0aXZlOmZvY3VzLC5idG4tc3VjY2Vzcy5hY3RpdmU6aG92ZXIsLmJ0bi1zdWNjZXNzOmFj
dGl2ZS5mb2N1cywuYnRuLXN1Y2Nlc3M6YWN0aXZlOmZvY3VzLC5idG4tc3VjY2VzczphY3RpdmU6
aG92ZXIsLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4tc3VjY2Vzcy5mb2N1cywub3Blbj4uZHJv
cGRvd24tdG9nZ2xlLmJ0bi1zdWNjZXNzOmZvY3VzLC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRu
LXN1Y2Nlc3M6aG92ZXJ7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzOTg0Mzk7Ym9yZGVy
LWNvbG9yOiMyNTU2MjV9LmJ0bi1zdWNjZXNzLmFjdGl2ZSwuYnRuLXN1Y2Nlc3M6YWN0aXZlLC5v
cGVuPi5kcm9wZG93bi10b2dnbGUuYnRuLXN1Y2Nlc3N7YmFja2dyb3VuZC1pbWFnZTpub25lfS5i
dG4tc3VjY2Vzcy5kaXNhYmxlZC5mb2N1cywuYnRuLXN1Y2Nlc3MuZGlzYWJsZWQ6Zm9jdXMsLmJ0
bi1zdWNjZXNzLmRpc2FibGVkOmhvdmVyLC5idG4tc3VjY2Vzc1tkaXNhYmxlZF0uZm9jdXMsLmJ0
bi1zdWNjZXNzW2Rpc2FibGVkXTpmb2N1cywuYnRuLXN1Y2Nlc3NbZGlzYWJsZWRdOmhvdmVyLGZp
ZWxkc2V0W2Rpc2FibGVkXSAuYnRuLXN1Y2Nlc3MuZm9jdXMsZmllbGRzZXRbZGlzYWJsZWRdIC5i
dG4tc3VjY2Vzczpmb2N1cyxmaWVsZHNldFtkaXNhYmxlZF0gLmJ0bi1zdWNjZXNzOmhvdmVye2Jh
Y2tncm91bmQtY29sb3I6IzVjYjg1Yztib3JkZXItY29sb3I6IzRjYWU0Y30uYnRuLXN1Y2Nlc3Mg
LmJhZGdle2NvbG9yOiM1Y2I4NWM7YmFja2dyb3VuZC1jb2xvcjojZmZmfS5idG4taW5mb3tjb2xv
cjojZmZmO2JhY2tncm91bmQtY29sb3I6IzViYzBkZTtib3JkZXItY29sb3I6IzQ2YjhkYX0uYnRu
LWluZm8uZm9jdXMsLmJ0bi1pbmZvOmZvY3Vze2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjoj
MzFiMGQ1O2JvcmRlci1jb2xvcjojMWI2ZDg1fS5idG4taW5mbzpob3Zlcntjb2xvcjojZmZmO2Jh
Y2tncm91bmQtY29sb3I6IzMxYjBkNTtib3JkZXItY29sb3I6IzI2OWFiY30uYnRuLWluZm8uYWN0
aXZlLC5idG4taW5mbzphY3RpdmUsLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4taW5mb3tjb2xv
cjojZmZmO2JhY2tncm91bmQtY29sb3I6IzMxYjBkNTtib3JkZXItY29sb3I6IzI2OWFiY30uYnRu
LWluZm8uYWN0aXZlLmZvY3VzLC5idG4taW5mby5hY3RpdmU6Zm9jdXMsLmJ0bi1pbmZvLmFjdGl2
ZTpob3ZlciwuYnRuLWluZm86YWN0aXZlLmZvY3VzLC5idG4taW5mbzphY3RpdmU6Zm9jdXMsLmJ0
bi1pbmZvOmFjdGl2ZTpob3Zlciwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi1pbmZvLmZvY3Vz
LC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRuLWluZm86Zm9jdXMsLm9wZW4+LmRyb3Bkb3duLXRv
Z2dsZS5idG4taW5mbzpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6IzI2OWFiYzti
b3JkZXItY29sb3I6IzFiNmQ4NX0uYnRuLWluZm8uYWN0aXZlLC5idG4taW5mbzphY3RpdmUsLm9w
ZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4taW5mb3tiYWNrZ3JvdW5kLWltYWdlOm5vbmV9LmJ0bi1p
bmZvLmRpc2FibGVkLmZvY3VzLC5idG4taW5mby5kaXNhYmxlZDpmb2N1cywuYnRuLWluZm8uZGlz
YWJsZWQ6aG92ZXIsLmJ0bi1pbmZvW2Rpc2FibGVkXS5mb2N1cywuYnRuLWluZm9bZGlzYWJsZWRd
OmZvY3VzLC5idG4taW5mb1tkaXNhYmxlZF06aG92ZXIsZmllbGRzZXRbZGlzYWJsZWRdIC5idG4t
aW5mby5mb2N1cyxmaWVsZHNldFtkaXNhYmxlZF0gLmJ0bi1pbmZvOmZvY3VzLGZpZWxkc2V0W2Rp
c2FibGVkXSAuYnRuLWluZm86aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojNWJjMGRlO2JvcmRlci1j
b2xvcjojNDZiOGRhfS5idG4taW5mbyAuYmFkZ2V7Y29sb3I6IzViYzBkZTtiYWNrZ3JvdW5kLWNv
bG9yOiNmZmZ9LmJ0bi13YXJuaW5ne2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojZjBhZDRl
O2JvcmRlci1jb2xvcjojZWVhMjM2fS5idG4td2FybmluZy5mb2N1cywuYnRuLXdhcm5pbmc6Zm9j
dXN7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiNlYzk3MWY7Ym9yZGVyLWNvbG9yOiM5ODVm
MGR9LmJ0bi13YXJuaW5nOmhvdmVye2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojZWM5NzFm
O2JvcmRlci1jb2xvcjojZDU4NTEyfS5idG4td2FybmluZy5hY3RpdmUsLmJ0bi13YXJuaW5nOmFj
dGl2ZSwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi13YXJuaW5ne2NvbG9yOiNmZmY7YmFja2dy
b3VuZC1jb2xvcjojZWM5NzFmO2JvcmRlci1jb2xvcjojZDU4NTEyfS5idG4td2FybmluZy5hY3Rp
dmUuZm9jdXMsLmJ0bi13YXJuaW5nLmFjdGl2ZTpmb2N1cywuYnRuLXdhcm5pbmcuYWN0aXZlOmhv
dmVyLC5idG4td2FybmluZzphY3RpdmUuZm9jdXMsLmJ0bi13YXJuaW5nOmFjdGl2ZTpmb2N1cywu
YnRuLXdhcm5pbmc6YWN0aXZlOmhvdmVyLC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRuLXdhcm5p
bmcuZm9jdXMsLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5idG4td2FybmluZzpmb2N1cywub3Blbj4u
ZHJvcGRvd24tdG9nZ2xlLmJ0bi13YXJuaW5nOmhvdmVye2NvbG9yOiNmZmY7YmFja2dyb3VuZC1j
b2xvcjojZDU4NTEyO2JvcmRlci1jb2xvcjojOTg1ZjBkfS5idG4td2FybmluZy5hY3RpdmUsLmJ0
bi13YXJuaW5nOmFjdGl2ZSwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi13YXJuaW5ne2JhY2tn
cm91bmQtaW1hZ2U6bm9uZX0uYnRuLXdhcm5pbmcuZGlzYWJsZWQuZm9jdXMsLmJ0bi13YXJuaW5n
LmRpc2FibGVkOmZvY3VzLC5idG4td2FybmluZy5kaXNhYmxlZDpob3ZlciwuYnRuLXdhcm5pbmdb
ZGlzYWJsZWRdLmZvY3VzLC5idG4td2FybmluZ1tkaXNhYmxlZF06Zm9jdXMsLmJ0bi13YXJuaW5n
W2Rpc2FibGVkXTpob3ZlcixmaWVsZHNldFtkaXNhYmxlZF0gLmJ0bi13YXJuaW5nLmZvY3VzLGZp
ZWxkc2V0W2Rpc2FibGVkXSAuYnRuLXdhcm5pbmc6Zm9jdXMsZmllbGRzZXRbZGlzYWJsZWRdIC5i
dG4td2FybmluZzpob3ZlcntiYWNrZ3JvdW5kLWNvbG9yOiNmMGFkNGU7Ym9yZGVyLWNvbG9yOiNl
ZWEyMzZ9LmJ0bi13YXJuaW5nIC5iYWRnZXtjb2xvcjojZjBhZDRlO2JhY2tncm91bmQtY29sb3I6
I2ZmZn0uYnRuLWRhbmdlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6I2Q5NTM0Zjtib3Jk
ZXItY29sb3I6I2Q0M2YzYX0uYnRuLWRhbmdlci5mb2N1cywuYnRuLWRhbmdlcjpmb2N1c3tjb2xv
cjojZmZmO2JhY2tncm91bmQtY29sb3I6I2M5MzAyYztib3JkZXItY29sb3I6Izc2MWMxOX0uYnRu
LWRhbmdlcjpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6I2M5MzAyYztib3JkZXIt
Y29sb3I6I2FjMjkyNX0uYnRuLWRhbmdlci5hY3RpdmUsLmJ0bi1kYW5nZXI6YWN0aXZlLC5vcGVu
Pi5kcm9wZG93bi10b2dnbGUuYnRuLWRhbmdlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6
I2M5MzAyYztib3JkZXItY29sb3I6I2FjMjkyNX0uYnRuLWRhbmdlci5hY3RpdmUuZm9jdXMsLmJ0
bi1kYW5nZXIuYWN0aXZlOmZvY3VzLC5idG4tZGFuZ2VyLmFjdGl2ZTpob3ZlciwuYnRuLWRhbmdl
cjphY3RpdmUuZm9jdXMsLmJ0bi1kYW5nZXI6YWN0aXZlOmZvY3VzLC5idG4tZGFuZ2VyOmFjdGl2
ZTpob3Zlciwub3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi1kYW5nZXIuZm9jdXMsLm9wZW4+LmRy
b3Bkb3duLXRvZ2dsZS5idG4tZGFuZ2VyOmZvY3VzLC5vcGVuPi5kcm9wZG93bi10b2dnbGUuYnRu
LWRhbmdlcjpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6I2FjMjkyNTtib3JkZXIt
Y29sb3I6Izc2MWMxOX0uYnRuLWRhbmdlci5hY3RpdmUsLmJ0bi1kYW5nZXI6YWN0aXZlLC5vcGVu
Pi5kcm9wZG93bi10b2dnbGUuYnRuLWRhbmdlcntiYWNrZ3JvdW5kLWltYWdlOm5vbmV9LmJ0bi1k
YW5nZXIuZGlzYWJsZWQuZm9jdXMsLmJ0bi1kYW5nZXIuZGlzYWJsZWQ6Zm9jdXMsLmJ0bi1kYW5n
ZXIuZGlzYWJsZWQ6aG92ZXIsLmJ0bi1kYW5nZXJbZGlzYWJsZWRdLmZvY3VzLC5idG4tZGFuZ2Vy
W2Rpc2FibGVkXTpmb2N1cywuYnRuLWRhbmdlcltkaXNhYmxlZF06aG92ZXIsZmllbGRzZXRbZGlz
YWJsZWRdIC5idG4tZGFuZ2VyLmZvY3VzLGZpZWxkc2V0W2Rpc2FibGVkXSAuYnRuLWRhbmdlcjpm
b2N1cyxmaWVsZHNldFtkaXNhYmxlZF0gLmJ0bi1kYW5nZXI6aG92ZXJ7YmFja2dyb3VuZC1jb2xv
cjojZDk1MzRmO2JvcmRlci1jb2xvcjojZDQzZjNhfS5idG4tZGFuZ2VyIC5iYWRnZXtjb2xvcjoj
ZDk1MzRmO2JhY2tncm91bmQtY29sb3I6I2ZmZn0uYnRuLWxpbmt7Zm9udC13ZWlnaHQ6NDAwO2Nv
bG9yOiMzMzdhYjc7Ym9yZGVyLXJhZGl1czowfS5idG4tbGluaywuYnRuLWxpbmsuYWN0aXZlLC5i
dG4tbGluazphY3RpdmUsLmJ0bi1saW5rW2Rpc2FibGVkXSxmaWVsZHNldFtkaXNhYmxlZF0gLmJ0
bi1saW5re2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7LXdlYmtpdC1ib3gtc2hhZG93Om5v
bmU7Ym94LXNoYWRvdzpub25lfS5idG4tbGluaywuYnRuLWxpbms6YWN0aXZlLC5idG4tbGluazpm
b2N1cywuYnRuLWxpbms6aG92ZXJ7Ym9yZGVyLWNvbG9yOnRyYW5zcGFyZW50fS5idG4tbGluazpm
b2N1cywuYnRuLWxpbms6aG92ZXJ7Y29sb3I6IzIzNTI3Yzt0ZXh0LWRlY29yYXRpb246dW5kZXJs
aW5lO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnR9LmJ0bi1saW5rW2Rpc2FibGVkXTpmb2N1
cywuYnRuLWxpbmtbZGlzYWJsZWRdOmhvdmVyLGZpZWxkc2V0W2Rpc2FibGVkXSAuYnRuLWxpbms6
Zm9jdXMsZmllbGRzZXRbZGlzYWJsZWRdIC5idG4tbGluazpob3Zlcntjb2xvcjojNzc3O3RleHQt
ZGVjb3JhdGlvbjpub25lfS5idG4tZ3JvdXAtbGc+LmJ0biwuYnRuLWxne3BhZGRpbmc6MTBweCAx
NnB4O2ZvbnQtc2l6ZToxOHB4O2xpbmUtaGVpZ2h0OjEuMzMzMzMzMztib3JkZXItcmFkaXVzOjZw
eH0uYnRuLWdyb3VwLXNtPi5idG4sLmJ0bi1zbXtwYWRkaW5nOjVweCAxMHB4O2ZvbnQtc2l6ZTox
MnB4O2xpbmUtaGVpZ2h0OjEuNTtib3JkZXItcmFkaXVzOjNweH0uYnRuLWdyb3VwLXhzPi5idG4s
LmJ0bi14c3twYWRkaW5nOjFweCA1cHg7Zm9udC1zaXplOjEycHg7bGluZS1oZWlnaHQ6MS41O2Jv
cmRlci1yYWRpdXM6M3B4fS5idG4tYmxvY2t7ZGlzcGxheTpibG9jazt3aWR0aDoxMDAlfS5idG4t
YmxvY2srLmJ0bi1ibG9ja3ttYXJnaW4tdG9wOjVweH1pbnB1dFt0eXBlPWJ1dHRvbl0uYnRuLWJs
b2NrLGlucHV0W3R5cGU9cmVzZXRdLmJ0bi1ibG9jayxpbnB1dFt0eXBlPXN1Ym1pdF0uYnRuLWJs
b2Nre3dpZHRoOjEwMCV9LmZhZGV7b3BhY2l0eTowOy13ZWJraXQtdHJhbnNpdGlvbjpvcGFjaXR5
IC4xNXMgbGluZWFyOy1vLXRyYW5zaXRpb246b3BhY2l0eSAuMTVzIGxpbmVhcjt0cmFuc2l0aW9u
Om9wYWNpdHkgLjE1cyBsaW5lYXJ9LmZhZGUuaW57b3BhY2l0eToxfS5jb2xsYXBzZXtkaXNwbGF5
Om5vbmV9LmNvbGxhcHNlLmlue2Rpc3BsYXk6YmxvY2t9dHIuY29sbGFwc2UuaW57ZGlzcGxheTp0
YWJsZS1yb3d9dGJvZHkuY29sbGFwc2UuaW57ZGlzcGxheTp0YWJsZS1yb3ctZ3JvdXB9LmNvbGxh
cHNpbmd7cG9zaXRpb246cmVsYXRpdmU7aGVpZ2h0OjA7b3ZlcmZsb3c6aGlkZGVuOy13ZWJraXQt
dHJhbnNpdGlvbi10aW1pbmctZnVuY3Rpb246ZWFzZTstby10cmFuc2l0aW9uLXRpbWluZy1mdW5j
dGlvbjplYXNlO3RyYW5zaXRpb24tdGltaW5nLWZ1bmN0aW9uOmVhc2U7LXdlYmtpdC10cmFuc2l0
aW9uLWR1cmF0aW9uOi4zNXM7LW8tdHJhbnNpdGlvbi1kdXJhdGlvbjouMzVzO3RyYW5zaXRpb24t
ZHVyYXRpb246LjM1czstd2Via2l0LXRyYW5zaXRpb24tcHJvcGVydHk6aGVpZ2h0LHZpc2liaWxp
dHk7LW8tdHJhbnNpdGlvbi1wcm9wZXJ0eTpoZWlnaHQsdmlzaWJpbGl0eTt0cmFuc2l0aW9uLXBy
b3BlcnR5OmhlaWdodCx2aXNpYmlsaXR5fS5jYXJldHtkaXNwbGF5OmlubGluZS1ibG9jazt3aWR0
aDowO2hlaWdodDowO21hcmdpbi1sZWZ0OjJweDt2ZXJ0aWNhbC1hbGlnbjptaWRkbGU7Ym9yZGVy
LXRvcDo0cHggZGFzaGVkO2JvcmRlci10b3A6NHB4IHNvbGlkXDk7Ym9yZGVyLXJpZ2h0OjRweCBz
b2xpZCB0cmFuc3BhcmVudDtib3JkZXItbGVmdDo0cHggc29saWQgdHJhbnNwYXJlbnR9LmRyb3Bk
b3duLC5kcm9wdXB7cG9zaXRpb246cmVsYXRpdmV9LmRyb3Bkb3duLXRvZ2dsZTpmb2N1c3tvdXRs
aW5lOjB9LmRyb3Bkb3duLW1lbnV7cG9zaXRpb246YWJzb2x1dGU7dG9wOjEwMCU7bGVmdDowO3ot
aW5kZXg6MTAwMDtkaXNwbGF5Om5vbmU7ZmxvYXQ6bGVmdDttaW4td2lkdGg6MTYwcHg7cGFkZGlu
Zzo1cHggMDttYXJnaW46MnB4IDAgMDtmb250LXNpemU6MTRweDt0ZXh0LWFsaWduOmxlZnQ7bGlz
dC1zdHlsZTpub25lO2JhY2tncm91bmQtY29sb3I6I2ZmZjstd2Via2l0LWJhY2tncm91bmQtY2xp
cDpwYWRkaW5nLWJveDtiYWNrZ3JvdW5kLWNsaXA6cGFkZGluZy1ib3g7Ym9yZGVyOjFweCBzb2xp
ZCAjY2NjO2JvcmRlcjoxcHggc29saWQgcmdiYSgwLDAsMCwuMTUpO2JvcmRlci1yYWRpdXM6NHB4
Oy13ZWJraXQtYm94LXNoYWRvdzowIDZweCAxMnB4IHJnYmEoMCwwLDAsLjE3NSk7Ym94LXNoYWRv
dzowIDZweCAxMnB4IHJnYmEoMCwwLDAsLjE3NSl9LmRyb3Bkb3duLW1lbnUucHVsbC1yaWdodHty
aWdodDowO2xlZnQ6YXV0b30uZHJvcGRvd24tbWVudSAuZGl2aWRlcntoZWlnaHQ6MXB4O21hcmdp
bjo5cHggMDtvdmVyZmxvdzpoaWRkZW47YmFja2dyb3VuZC1jb2xvcjojZTVlNWU1fS5kcm9wZG93
bi1tZW51PmxpPmF7ZGlzcGxheTpibG9jaztwYWRkaW5nOjNweCAyMHB4O2NsZWFyOmJvdGg7Zm9u
dC13ZWlnaHQ6NDAwO2xpbmUtaGVpZ2h0OjEuNDI4NTcxNDM7Y29sb3I6IzMzMzt3aGl0ZS1zcGFj
ZTpub3dyYXB9LmRyb3Bkb3duLW1lbnU+bGk+YTpmb2N1cywuZHJvcGRvd24tbWVudT5saT5hOmhv
dmVye2NvbG9yOiMyNjI2MjY7dGV4dC1kZWNvcmF0aW9uOm5vbmU7YmFja2dyb3VuZC1jb2xvcjoj
ZjVmNWY1fS5kcm9wZG93bi1tZW51Pi5hY3RpdmU+YSwuZHJvcGRvd24tbWVudT4uYWN0aXZlPmE6
Zm9jdXMsLmRyb3Bkb3duLW1lbnU+LmFjdGl2ZT5hOmhvdmVye2NvbG9yOiNmZmY7dGV4dC1kZWNv
cmF0aW9uOm5vbmU7YmFja2dyb3VuZC1jb2xvcjojMzM3YWI3O291dGxpbmU6MH0uZHJvcGRvd24t
bWVudT4uZGlzYWJsZWQ+YSwuZHJvcGRvd24tbWVudT4uZGlzYWJsZWQ+YTpmb2N1cywuZHJvcGRv
d24tbWVudT4uZGlzYWJsZWQ+YTpob3Zlcntjb2xvcjojNzc3fS5kcm9wZG93bi1tZW51Pi5kaXNh
YmxlZD5hOmZvY3VzLC5kcm9wZG93bi1tZW51Pi5kaXNhYmxlZD5hOmhvdmVye3RleHQtZGVjb3Jh
dGlvbjpub25lO2N1cnNvcjpub3QtYWxsb3dlZDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50
O2JhY2tncm91bmQtaW1hZ2U6bm9uZTtmaWx0ZXI6cHJvZ2lkOkRYSW1hZ2VUcmFuc2Zvcm0uTWlj
cm9zb2Z0LmdyYWRpZW50KGVuYWJsZWQ9ZmFsc2UpfS5vcGVuPi5kcm9wZG93bi1tZW51e2Rpc3Bs
YXk6YmxvY2t9Lm9wZW4+YXtvdXRsaW5lOjB9LmRyb3Bkb3duLW1lbnUtcmlnaHR7cmlnaHQ6MDts
ZWZ0OmF1dG99LmRyb3Bkb3duLW1lbnUtbGVmdHtyaWdodDphdXRvO2xlZnQ6MH0uZHJvcGRvd24t
aGVhZGVye2Rpc3BsYXk6YmxvY2s7cGFkZGluZzozcHggMjBweDtmb250LXNpemU6MTJweDtsaW5l
LWhlaWdodDoxLjQyODU3MTQzO2NvbG9yOiM3Nzc7d2hpdGUtc3BhY2U6bm93cmFwfS5kcm9wZG93
bi1iYWNrZHJvcHtwb3NpdGlvbjpmaXhlZDt0b3A6MDtyaWdodDowO2JvdHRvbTowO2xlZnQ6MDt6
LWluZGV4Ojk5MH0ucHVsbC1yaWdodD4uZHJvcGRvd24tbWVudXtyaWdodDowO2xlZnQ6YXV0b30u
ZHJvcHVwIC5jYXJldCwubmF2YmFyLWZpeGVkLWJvdHRvbSAuZHJvcGRvd24gLmNhcmV0e2NvbnRl
bnQ6IiI7Ym9yZGVyLXRvcDowO2JvcmRlci1ib3R0b206NHB4IGRhc2hlZDtib3JkZXItYm90dG9t
OjRweCBzb2xpZFw5fS5kcm9wdXAgLmRyb3Bkb3duLW1lbnUsLm5hdmJhci1maXhlZC1ib3R0b20g
LmRyb3Bkb3duIC5kcm9wZG93bi1tZW51e3RvcDphdXRvO2JvdHRvbToxMDAlO21hcmdpbi1ib3R0
b206MnB4fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsubmF2YmFyLXJpZ2h0IC5kcm9wZG93bi1t
ZW51e3JpZ2h0OjA7bGVmdDphdXRvfS5uYXZiYXItcmlnaHQgLmRyb3Bkb3duLW1lbnUtbGVmdHty
aWdodDphdXRvO2xlZnQ6MH19LmJ0bi1ncm91cCwuYnRuLWdyb3VwLXZlcnRpY2Fse3Bvc2l0aW9u
OnJlbGF0aXZlO2Rpc3BsYXk6aW5saW5lLWJsb2NrO3ZlcnRpY2FsLWFsaWduOm1pZGRsZX0uYnRu
LWdyb3VwLXZlcnRpY2FsPi5idG4sLmJ0bi1ncm91cD4uYnRue3Bvc2l0aW9uOnJlbGF0aXZlO2Zs
b2F0OmxlZnR9LmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLmFjdGl2ZSwuYnRuLWdyb3VwLXZlcnRp
Y2FsPi5idG46YWN0aXZlLC5idG4tZ3JvdXAtdmVydGljYWw+LmJ0bjpmb2N1cywuYnRuLWdyb3Vw
LXZlcnRpY2FsPi5idG46aG92ZXIsLmJ0bi1ncm91cD4uYnRuLmFjdGl2ZSwuYnRuLWdyb3VwPi5i
dG46YWN0aXZlLC5idG4tZ3JvdXA+LmJ0bjpmb2N1cywuYnRuLWdyb3VwPi5idG46aG92ZXJ7ei1p
bmRleDoyfS5idG4tZ3JvdXAgLmJ0bisuYnRuLC5idG4tZ3JvdXAgLmJ0bisuYnRuLWdyb3VwLC5i
dG4tZ3JvdXAgLmJ0bi1ncm91cCsuYnRuLC5idG4tZ3JvdXAgLmJ0bi1ncm91cCsuYnRuLWdyb3Vw
e21hcmdpbi1sZWZ0Oi0xcHh9LmJ0bi10b29sYmFye21hcmdpbi1sZWZ0Oi01cHh9LmJ0bi10b29s
YmFyIC5idG4sLmJ0bi10b29sYmFyIC5idG4tZ3JvdXAsLmJ0bi10b29sYmFyIC5pbnB1dC1ncm91
cHtmbG9hdDpsZWZ0fS5idG4tdG9vbGJhcj4uYnRuLC5idG4tdG9vbGJhcj4uYnRuLWdyb3VwLC5i
dG4tdG9vbGJhcj4uaW5wdXQtZ3JvdXB7bWFyZ2luLWxlZnQ6NXB4fS5idG4tZ3JvdXA+LmJ0bjpu
b3QoOmZpcnN0LWNoaWxkKTpub3QoOmxhc3QtY2hpbGQpOm5vdCguZHJvcGRvd24tdG9nZ2xlKXti
b3JkZXItcmFkaXVzOjB9LmJ0bi1ncm91cD4uYnRuOmZpcnN0LWNoaWxke21hcmdpbi1sZWZ0OjB9
LmJ0bi1ncm91cD4uYnRuOmZpcnN0LWNoaWxkOm5vdCg6bGFzdC1jaGlsZCk6bm90KC5kcm9wZG93
bi10b2dnbGUpe2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjA7Ym9yZGVyLWJvdHRvbS1yaWdodC1y
YWRpdXM6MH0uYnRuLWdyb3VwPi5idG46bGFzdC1jaGlsZDpub3QoOmZpcnN0LWNoaWxkKSwuYnRu
LWdyb3VwPi5kcm9wZG93bi10b2dnbGU6bm90KDpmaXJzdC1jaGlsZCl7Ym9yZGVyLXRvcC1sZWZ0
LXJhZGl1czowO2JvcmRlci1ib3R0b20tbGVmdC1yYWRpdXM6MH0uYnRuLWdyb3VwPi5idG4tZ3Jv
dXB7ZmxvYXQ6bGVmdH0uYnRuLWdyb3VwPi5idG4tZ3JvdXA6bm90KDpmaXJzdC1jaGlsZCk6bm90
KDpsYXN0LWNoaWxkKT4uYnRue2JvcmRlci1yYWRpdXM6MH0uYnRuLWdyb3VwPi5idG4tZ3JvdXA6
Zmlyc3QtY2hpbGQ6bm90KDpsYXN0LWNoaWxkKT4uYnRuOmxhc3QtY2hpbGQsLmJ0bi1ncm91cD4u
YnRuLWdyb3VwOmZpcnN0LWNoaWxkOm5vdCg6bGFzdC1jaGlsZCk+LmRyb3Bkb3duLXRvZ2dsZXti
b3JkZXItdG9wLXJpZ2h0LXJhZGl1czowO2JvcmRlci1ib3R0b20tcmlnaHQtcmFkaXVzOjB9LmJ0
bi1ncm91cD4uYnRuLWdyb3VwOmxhc3QtY2hpbGQ6bm90KDpmaXJzdC1jaGlsZCk+LmJ0bjpmaXJz
dC1jaGlsZHtib3JkZXItdG9wLWxlZnQtcmFkaXVzOjA7Ym9yZGVyLWJvdHRvbS1sZWZ0LXJhZGl1
czowfS5idG4tZ3JvdXAgLmRyb3Bkb3duLXRvZ2dsZTphY3RpdmUsLmJ0bi1ncm91cC5vcGVuIC5k
cm9wZG93bi10b2dnbGV7b3V0bGluZTowfS5idG4tZ3JvdXA+LmJ0bisuZHJvcGRvd24tdG9nZ2xl
e3BhZGRpbmctcmlnaHQ6OHB4O3BhZGRpbmctbGVmdDo4cHh9LmJ0bi1ncm91cD4uYnRuLWxnKy5k
cm9wZG93bi10b2dnbGV7cGFkZGluZy1yaWdodDoxMnB4O3BhZGRpbmctbGVmdDoxMnB4fS5idG4t
Z3JvdXAub3BlbiAuZHJvcGRvd24tdG9nZ2xley13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDNw
eCA1cHggcmdiYSgwLDAsMCwuMTI1KTtib3gtc2hhZG93Omluc2V0IDAgM3B4IDVweCByZ2JhKDAs
MCwwLC4xMjUpfS5idG4tZ3JvdXAub3BlbiAuZHJvcGRvd24tdG9nZ2xlLmJ0bi1saW5rey13ZWJr
aXQtYm94LXNoYWRvdzpub25lO2JveC1zaGFkb3c6bm9uZX0uYnRuIC5jYXJldHttYXJnaW4tbGVm
dDowfS5idG4tbGcgLmNhcmV0e2JvcmRlci13aWR0aDo1cHggNXB4IDA7Ym9yZGVyLWJvdHRvbS13
aWR0aDowfS5kcm9wdXAgLmJ0bi1sZyAuY2FyZXR7Ym9yZGVyLXdpZHRoOjAgNXB4IDVweH0uYnRu
LWdyb3VwLXZlcnRpY2FsPi5idG4sLmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLWdyb3VwLC5idG4t
Z3JvdXAtdmVydGljYWw+LmJ0bi1ncm91cD4uYnRue2Rpc3BsYXk6YmxvY2s7ZmxvYXQ6bm9uZTt3
aWR0aDoxMDAlO21heC13aWR0aDoxMDAlfS5idG4tZ3JvdXAtdmVydGljYWw+LmJ0bi1ncm91cD4u
YnRue2Zsb2F0Om5vbmV9LmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuKy5idG4sLmJ0bi1ncm91cC12
ZXJ0aWNhbD4uYnRuKy5idG4tZ3JvdXAsLmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLWdyb3VwKy5i
dG4sLmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLWdyb3VwKy5idG4tZ3JvdXB7bWFyZ2luLXRvcDot
MXB4O21hcmdpbi1sZWZ0OjB9LmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuOm5vdCg6Zmlyc3QtY2hp
bGQpOm5vdCg6bGFzdC1jaGlsZCl7Ym9yZGVyLXJhZGl1czowfS5idG4tZ3JvdXAtdmVydGljYWw+
LmJ0bjpmaXJzdC1jaGlsZDpub3QoOmxhc3QtY2hpbGQpe2JvcmRlci10b3AtbGVmdC1yYWRpdXM6
NHB4O2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjRweDtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1
czowO2JvcmRlci1ib3R0b20tbGVmdC1yYWRpdXM6MH0uYnRuLWdyb3VwLXZlcnRpY2FsPi5idG46
bGFzdC1jaGlsZDpub3QoOmZpcnN0LWNoaWxkKXtib3JkZXItdG9wLWxlZnQtcmFkaXVzOjA7Ym9y
ZGVyLXRvcC1yaWdodC1yYWRpdXM6MDtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1czo0cHg7Ym9y
ZGVyLWJvdHRvbS1sZWZ0LXJhZGl1czo0cHh9LmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLWdyb3Vw
Om5vdCg6Zmlyc3QtY2hpbGQpOm5vdCg6bGFzdC1jaGlsZCk+LmJ0bntib3JkZXItcmFkaXVzOjB9
LmJ0bi1ncm91cC12ZXJ0aWNhbD4uYnRuLWdyb3VwOmZpcnN0LWNoaWxkOm5vdCg6bGFzdC1jaGls
ZCk+LmJ0bjpsYXN0LWNoaWxkLC5idG4tZ3JvdXAtdmVydGljYWw+LmJ0bi1ncm91cDpmaXJzdC1j
aGlsZDpub3QoOmxhc3QtY2hpbGQpPi5kcm9wZG93bi10b2dnbGV7Ym9yZGVyLWJvdHRvbS1yaWdo
dC1yYWRpdXM6MDtib3JkZXItYm90dG9tLWxlZnQtcmFkaXVzOjB9LmJ0bi1ncm91cC12ZXJ0aWNh
bD4uYnRuLWdyb3VwOmxhc3QtY2hpbGQ6bm90KDpmaXJzdC1jaGlsZCk+LmJ0bjpmaXJzdC1jaGls
ZHtib3JkZXItdG9wLWxlZnQtcmFkaXVzOjA7Ym9yZGVyLXRvcC1yaWdodC1yYWRpdXM6MH0uYnRu
LWdyb3VwLWp1c3RpZmllZHtkaXNwbGF5OnRhYmxlO3dpZHRoOjEwMCU7dGFibGUtbGF5b3V0OmZp
eGVkO2JvcmRlci1jb2xsYXBzZTpzZXBhcmF0ZX0uYnRuLWdyb3VwLWp1c3RpZmllZD4uYnRuLC5i
dG4tZ3JvdXAtanVzdGlmaWVkPi5idG4tZ3JvdXB7ZGlzcGxheTp0YWJsZS1jZWxsO2Zsb2F0Om5v
bmU7d2lkdGg6MSV9LmJ0bi1ncm91cC1qdXN0aWZpZWQ+LmJ0bi1ncm91cCAuYnRue3dpZHRoOjEw
MCV9LmJ0bi1ncm91cC1qdXN0aWZpZWQ+LmJ0bi1ncm91cCAuZHJvcGRvd24tbWVudXtsZWZ0OmF1
dG99W2RhdGEtdG9nZ2xlPWJ1dHRvbnNdPi5idG4gaW5wdXRbdHlwZT1jaGVja2JveF0sW2RhdGEt
dG9nZ2xlPWJ1dHRvbnNdPi5idG4gaW5wdXRbdHlwZT1yYWRpb10sW2RhdGEtdG9nZ2xlPWJ1dHRv
bnNdPi5idG4tZ3JvdXA+LmJ0biBpbnB1dFt0eXBlPWNoZWNrYm94XSxbZGF0YS10b2dnbGU9YnV0
dG9uc10+LmJ0bi1ncm91cD4uYnRuIGlucHV0W3R5cGU9cmFkaW9de3Bvc2l0aW9uOmFic29sdXRl
O2NsaXA6cmVjdCgwLDAsMCwwKTtwb2ludGVyLWV2ZW50czpub25lfS5pbnB1dC1ncm91cHtwb3Np
dGlvbjpyZWxhdGl2ZTtkaXNwbGF5OnRhYmxlO2JvcmRlci1jb2xsYXBzZTpzZXBhcmF0ZX0uaW5w
dXQtZ3JvdXBbY2xhc3MqPWNvbC1de2Zsb2F0Om5vbmU7cGFkZGluZy1yaWdodDowO3BhZGRpbmct
bGVmdDowfS5pbnB1dC1ncm91cCAuZm9ybS1jb250cm9se3Bvc2l0aW9uOnJlbGF0aXZlO3otaW5k
ZXg6MjtmbG9hdDpsZWZ0O3dpZHRoOjEwMCU7bWFyZ2luLWJvdHRvbTowfS5pbnB1dC1ncm91cCAu
Zm9ybS1jb250cm9sOmZvY3Vze3otaW5kZXg6M30uaW5wdXQtZ3JvdXAtbGc+LmZvcm0tY29udHJv
bCwuaW5wdXQtZ3JvdXAtbGc+LmlucHV0LWdyb3VwLWFkZG9uLC5pbnB1dC1ncm91cC1sZz4uaW5w
dXQtZ3JvdXAtYnRuPi5idG57aGVpZ2h0OjQ2cHg7cGFkZGluZzoxMHB4IDE2cHg7Zm9udC1zaXpl
OjE4cHg7bGluZS1oZWlnaHQ6MS4zMzMzMzMzO2JvcmRlci1yYWRpdXM6NnB4fXNlbGVjdC5pbnB1
dC1ncm91cC1sZz4uZm9ybS1jb250cm9sLHNlbGVjdC5pbnB1dC1ncm91cC1sZz4uaW5wdXQtZ3Jv
dXAtYWRkb24sc2VsZWN0LmlucHV0LWdyb3VwLWxnPi5pbnB1dC1ncm91cC1idG4+LmJ0bntoZWln
aHQ6NDZweDtsaW5lLWhlaWdodDo0NnB4fXNlbGVjdFttdWx0aXBsZV0uaW5wdXQtZ3JvdXAtbGc+
LmZvcm0tY29udHJvbCxzZWxlY3RbbXVsdGlwbGVdLmlucHV0LWdyb3VwLWxnPi5pbnB1dC1ncm91
cC1hZGRvbixzZWxlY3RbbXVsdGlwbGVdLmlucHV0LWdyb3VwLWxnPi5pbnB1dC1ncm91cC1idG4+
LmJ0bix0ZXh0YXJlYS5pbnB1dC1ncm91cC1sZz4uZm9ybS1jb250cm9sLHRleHRhcmVhLmlucHV0
LWdyb3VwLWxnPi5pbnB1dC1ncm91cC1hZGRvbix0ZXh0YXJlYS5pbnB1dC1ncm91cC1sZz4uaW5w
dXQtZ3JvdXAtYnRuPi5idG57aGVpZ2h0OmF1dG99LmlucHV0LWdyb3VwLXNtPi5mb3JtLWNvbnRy
b2wsLmlucHV0LWdyb3VwLXNtPi5pbnB1dC1ncm91cC1hZGRvbiwuaW5wdXQtZ3JvdXAtc20+Lmlu
cHV0LWdyb3VwLWJ0bj4uYnRue2hlaWdodDozMHB4O3BhZGRpbmc6NXB4IDEwcHg7Zm9udC1zaXpl
OjEycHg7bGluZS1oZWlnaHQ6MS41O2JvcmRlci1yYWRpdXM6M3B4fXNlbGVjdC5pbnB1dC1ncm91
cC1zbT4uZm9ybS1jb250cm9sLHNlbGVjdC5pbnB1dC1ncm91cC1zbT4uaW5wdXQtZ3JvdXAtYWRk
b24sc2VsZWN0LmlucHV0LWdyb3VwLXNtPi5pbnB1dC1ncm91cC1idG4+LmJ0bntoZWlnaHQ6MzBw
eDtsaW5lLWhlaWdodDozMHB4fXNlbGVjdFttdWx0aXBsZV0uaW5wdXQtZ3JvdXAtc20+LmZvcm0t
Y29udHJvbCxzZWxlY3RbbXVsdGlwbGVdLmlucHV0LWdyb3VwLXNtPi5pbnB1dC1ncm91cC1hZGRv
bixzZWxlY3RbbXVsdGlwbGVdLmlucHV0LWdyb3VwLXNtPi5pbnB1dC1ncm91cC1idG4+LmJ0bix0
ZXh0YXJlYS5pbnB1dC1ncm91cC1zbT4uZm9ybS1jb250cm9sLHRleHRhcmVhLmlucHV0LWdyb3Vw
LXNtPi5pbnB1dC1ncm91cC1hZGRvbix0ZXh0YXJlYS5pbnB1dC1ncm91cC1zbT4uaW5wdXQtZ3Jv
dXAtYnRuPi5idG57aGVpZ2h0OmF1dG99LmlucHV0LWdyb3VwIC5mb3JtLWNvbnRyb2wsLmlucHV0
LWdyb3VwLWFkZG9uLC5pbnB1dC1ncm91cC1idG57ZGlzcGxheTp0YWJsZS1jZWxsfS5pbnB1dC1n
cm91cCAuZm9ybS1jb250cm9sOm5vdCg6Zmlyc3QtY2hpbGQpOm5vdCg6bGFzdC1jaGlsZCksLmlu
cHV0LWdyb3VwLWFkZG9uOm5vdCg6Zmlyc3QtY2hpbGQpOm5vdCg6bGFzdC1jaGlsZCksLmlucHV0
LWdyb3VwLWJ0bjpub3QoOmZpcnN0LWNoaWxkKTpub3QoOmxhc3QtY2hpbGQpe2JvcmRlci1yYWRp
dXM6MH0uaW5wdXQtZ3JvdXAtYWRkb24sLmlucHV0LWdyb3VwLWJ0bnt3aWR0aDoxJTt3aGl0ZS1z
cGFjZTpub3dyYXA7dmVydGljYWwtYWxpZ246bWlkZGxlfS5pbnB1dC1ncm91cC1hZGRvbntwYWRk
aW5nOjZweCAxMnB4O2ZvbnQtc2l6ZToxNHB4O2ZvbnQtd2VpZ2h0OjQwMDtsaW5lLWhlaWdodDox
O2NvbG9yOiM1NTU7dGV4dC1hbGlnbjpjZW50ZXI7YmFja2dyb3VuZC1jb2xvcjojZWVlO2JvcmRl
cjoxcHggc29saWQgI2NjYztib3JkZXItcmFkaXVzOjRweH0uaW5wdXQtZ3JvdXAtYWRkb24uaW5w
dXQtc217cGFkZGluZzo1cHggMTBweDtmb250LXNpemU6MTJweDtib3JkZXItcmFkaXVzOjNweH0u
aW5wdXQtZ3JvdXAtYWRkb24uaW5wdXQtbGd7cGFkZGluZzoxMHB4IDE2cHg7Zm9udC1zaXplOjE4
cHg7Ym9yZGVyLXJhZGl1czo2cHh9LmlucHV0LWdyb3VwLWFkZG9uIGlucHV0W3R5cGU9Y2hlY2ti
b3hdLC5pbnB1dC1ncm91cC1hZGRvbiBpbnB1dFt0eXBlPXJhZGlvXXttYXJnaW4tdG9wOjB9Lmlu
cHV0LWdyb3VwIC5mb3JtLWNvbnRyb2w6Zmlyc3QtY2hpbGQsLmlucHV0LWdyb3VwLWFkZG9uOmZp
cnN0LWNoaWxkLC5pbnB1dC1ncm91cC1idG46Zmlyc3QtY2hpbGQ+LmJ0biwuaW5wdXQtZ3JvdXAt
YnRuOmZpcnN0LWNoaWxkPi5idG4tZ3JvdXA+LmJ0biwuaW5wdXQtZ3JvdXAtYnRuOmZpcnN0LWNo
aWxkPi5kcm9wZG93bi10b2dnbGUsLmlucHV0LWdyb3VwLWJ0bjpsYXN0LWNoaWxkPi5idG4tZ3Jv
dXA6bm90KDpsYXN0LWNoaWxkKT4uYnRuLC5pbnB1dC1ncm91cC1idG46bGFzdC1jaGlsZD4uYnRu
Om5vdCg6bGFzdC1jaGlsZCk6bm90KC5kcm9wZG93bi10b2dnbGUpe2JvcmRlci10b3AtcmlnaHQt
cmFkaXVzOjA7Ym9yZGVyLWJvdHRvbS1yaWdodC1yYWRpdXM6MH0uaW5wdXQtZ3JvdXAtYWRkb246
Zmlyc3QtY2hpbGR7Ym9yZGVyLXJpZ2h0OjB9LmlucHV0LWdyb3VwIC5mb3JtLWNvbnRyb2w6bGFz
dC1jaGlsZCwuaW5wdXQtZ3JvdXAtYWRkb246bGFzdC1jaGlsZCwuaW5wdXQtZ3JvdXAtYnRuOmZp
cnN0LWNoaWxkPi5idG4tZ3JvdXA6bm90KDpmaXJzdC1jaGlsZCk+LmJ0biwuaW5wdXQtZ3JvdXAt
YnRuOmZpcnN0LWNoaWxkPi5idG46bm90KDpmaXJzdC1jaGlsZCksLmlucHV0LWdyb3VwLWJ0bjps
YXN0LWNoaWxkPi5idG4sLmlucHV0LWdyb3VwLWJ0bjpsYXN0LWNoaWxkPi5idG4tZ3JvdXA+LmJ0
biwuaW5wdXQtZ3JvdXAtYnRuOmxhc3QtY2hpbGQ+LmRyb3Bkb3duLXRvZ2dsZXtib3JkZXItdG9w
LWxlZnQtcmFkaXVzOjA7Ym9yZGVyLWJvdHRvbS1sZWZ0LXJhZGl1czowfS5pbnB1dC1ncm91cC1h
ZGRvbjpsYXN0LWNoaWxke2JvcmRlci1sZWZ0OjB9LmlucHV0LWdyb3VwLWJ0bntwb3NpdGlvbjpy
ZWxhdGl2ZTtmb250LXNpemU6MDt3aGl0ZS1zcGFjZTpub3dyYXB9LmlucHV0LWdyb3VwLWJ0bj4u
YnRue3Bvc2l0aW9uOnJlbGF0aXZlfS5pbnB1dC1ncm91cC1idG4+LmJ0bisuYnRue21hcmdpbi1s
ZWZ0Oi0xcHh9LmlucHV0LWdyb3VwLWJ0bj4uYnRuOmFjdGl2ZSwuaW5wdXQtZ3JvdXAtYnRuPi5i
dG46Zm9jdXMsLmlucHV0LWdyb3VwLWJ0bj4uYnRuOmhvdmVye3otaW5kZXg6Mn0uaW5wdXQtZ3Jv
dXAtYnRuOmZpcnN0LWNoaWxkPi5idG4sLmlucHV0LWdyb3VwLWJ0bjpmaXJzdC1jaGlsZD4uYnRu
LWdyb3Vwe21hcmdpbi1yaWdodDotMXB4fS5pbnB1dC1ncm91cC1idG46bGFzdC1jaGlsZD4uYnRu
LC5pbnB1dC1ncm91cC1idG46bGFzdC1jaGlsZD4uYnRuLWdyb3Vwe3otaW5kZXg6MjttYXJnaW4t
bGVmdDotMXB4fS5uYXZ7cGFkZGluZy1sZWZ0OjA7bWFyZ2luLWJvdHRvbTowO2xpc3Qtc3R5bGU6
bm9uZX0ubmF2Pmxpe3Bvc2l0aW9uOnJlbGF0aXZlO2Rpc3BsYXk6YmxvY2t9Lm5hdj5saT5he3Bv
c2l0aW9uOnJlbGF0aXZlO2Rpc3BsYXk6YmxvY2s7cGFkZGluZzoxMHB4IDE1cHh9Lm5hdj5saT5h
OmZvY3VzLC5uYXY+bGk+YTpob3Zlcnt0ZXh0LWRlY29yYXRpb246bm9uZTtiYWNrZ3JvdW5kLWNv
bG9yOiNlZWV9Lm5hdj5saS5kaXNhYmxlZD5he2NvbG9yOiM3Nzd9Lm5hdj5saS5kaXNhYmxlZD5h
OmZvY3VzLC5uYXY+bGkuZGlzYWJsZWQ+YTpob3Zlcntjb2xvcjojNzc3O3RleHQtZGVjb3JhdGlv
bjpub25lO2N1cnNvcjpub3QtYWxsb3dlZDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50fS5u
YXYgLm9wZW4+YSwubmF2IC5vcGVuPmE6Zm9jdXMsLm5hdiAub3Blbj5hOmhvdmVye2JhY2tncm91
bmQtY29sb3I6I2VlZTtib3JkZXItY29sb3I6IzMzN2FiN30ubmF2IC5uYXYtZGl2aWRlcntoZWln
aHQ6MXB4O21hcmdpbjo5cHggMDtvdmVyZmxvdzpoaWRkZW47YmFja2dyb3VuZC1jb2xvcjojZTVl
NWU1fS5uYXY+bGk+YT5pbWd7bWF4LXdpZHRoOm5vbmV9Lm5hdi10YWJze2JvcmRlci1ib3R0b206
MXB4IHNvbGlkICNkZGR9Lm5hdi10YWJzPmxpe2Zsb2F0OmxlZnQ7bWFyZ2luLWJvdHRvbTotMXB4
fS5uYXYtdGFicz5saT5he21hcmdpbi1yaWdodDoycHg7bGluZS1oZWlnaHQ6MS40Mjg1NzE0Mzti
b3JkZXI6MXB4IHNvbGlkIHRyYW5zcGFyZW50O2JvcmRlci1yYWRpdXM6NHB4IDRweCAwIDB9Lm5h
di10YWJzPmxpPmE6aG92ZXJ7Ym9yZGVyLWNvbG9yOiNlZWUgI2VlZSAjZGRkfS5uYXYtdGFicz5s
aS5hY3RpdmU+YSwubmF2LXRhYnM+bGkuYWN0aXZlPmE6Zm9jdXMsLm5hdi10YWJzPmxpLmFjdGl2
ZT5hOmhvdmVye2NvbG9yOiM1NTU7Y3Vyc29yOmRlZmF1bHQ7YmFja2dyb3VuZC1jb2xvcjojZmZm
O2JvcmRlcjoxcHggc29saWQgI2RkZDtib3JkZXItYm90dG9tLWNvbG9yOnRyYW5zcGFyZW50fS5u
YXYtdGFicy5uYXYtanVzdGlmaWVke3dpZHRoOjEwMCU7Ym9yZGVyLWJvdHRvbTowfS5uYXYtdGFi
cy5uYXYtanVzdGlmaWVkPmxpe2Zsb2F0Om5vbmV9Lm5hdi10YWJzLm5hdi1qdXN0aWZpZWQ+bGk+
YXttYXJnaW4tYm90dG9tOjVweDt0ZXh0LWFsaWduOmNlbnRlcn0ubmF2LXRhYnMubmF2LWp1c3Rp
ZmllZD4uZHJvcGRvd24gLmRyb3Bkb3duLW1lbnV7dG9wOmF1dG87bGVmdDphdXRvfUBtZWRpYSAo
bWluLXdpZHRoOjc2OHB4KXsubmF2LXRhYnMubmF2LWp1c3RpZmllZD5saXtkaXNwbGF5OnRhYmxl
LWNlbGw7d2lkdGg6MSV9Lm5hdi10YWJzLm5hdi1qdXN0aWZpZWQ+bGk+YXttYXJnaW4tYm90dG9t
OjB9fS5uYXYtdGFicy5uYXYtanVzdGlmaWVkPmxpPmF7bWFyZ2luLXJpZ2h0OjA7Ym9yZGVyLXJh
ZGl1czo0cHh9Lm5hdi10YWJzLm5hdi1qdXN0aWZpZWQ+LmFjdGl2ZT5hLC5uYXYtdGFicy5uYXYt
anVzdGlmaWVkPi5hY3RpdmU+YTpmb2N1cywubmF2LXRhYnMubmF2LWp1c3RpZmllZD4uYWN0aXZl
PmE6aG92ZXJ7Ym9yZGVyOjFweCBzb2xpZCAjZGRkfUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsu
bmF2LXRhYnMubmF2LWp1c3RpZmllZD5saT5he2JvcmRlci1ib3R0b206MXB4IHNvbGlkICNkZGQ7
Ym9yZGVyLXJhZGl1czo0cHggNHB4IDAgMH0ubmF2LXRhYnMubmF2LWp1c3RpZmllZD4uYWN0aXZl
PmEsLm5hdi10YWJzLm5hdi1qdXN0aWZpZWQ+LmFjdGl2ZT5hOmZvY3VzLC5uYXYtdGFicy5uYXYt
anVzdGlmaWVkPi5hY3RpdmU+YTpob3Zlcntib3JkZXItYm90dG9tLWNvbG9yOiNmZmZ9fS5uYXYt
cGlsbHM+bGl7ZmxvYXQ6bGVmdH0ubmF2LXBpbGxzPmxpPmF7Ym9yZGVyLXJhZGl1czo0cHh9Lm5h
di1waWxscz5saStsaXttYXJnaW4tbGVmdDoycHh9Lm5hdi1waWxscz5saS5hY3RpdmU+YSwubmF2
LXBpbGxzPmxpLmFjdGl2ZT5hOmZvY3VzLC5uYXYtcGlsbHM+bGkuYWN0aXZlPmE6aG92ZXJ7Y29s
b3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzMzdhYjd9Lm5hdi1zdGFja2VkPmxpe2Zsb2F0Om5v
bmV9Lm5hdi1zdGFja2VkPmxpK2xpe21hcmdpbi10b3A6MnB4O21hcmdpbi1sZWZ0OjB9Lm5hdi1q
dXN0aWZpZWR7d2lkdGg6MTAwJX0ubmF2LWp1c3RpZmllZD5saXtmbG9hdDpub25lfS5uYXYtanVz
dGlmaWVkPmxpPmF7bWFyZ2luLWJvdHRvbTo1cHg7dGV4dC1hbGlnbjpjZW50ZXJ9Lm5hdi1qdXN0
aWZpZWQ+LmRyb3Bkb3duIC5kcm9wZG93bi1tZW51e3RvcDphdXRvO2xlZnQ6YXV0b31AbWVkaWEg
KG1pbi13aWR0aDo3NjhweCl7Lm5hdi1qdXN0aWZpZWQ+bGl7ZGlzcGxheTp0YWJsZS1jZWxsO3dp
ZHRoOjElfS5uYXYtanVzdGlmaWVkPmxpPmF7bWFyZ2luLWJvdHRvbTowfX0ubmF2LXRhYnMtanVz
dGlmaWVke2JvcmRlci1ib3R0b206MH0ubmF2LXRhYnMtanVzdGlmaWVkPmxpPmF7bWFyZ2luLXJp
Z2h0OjA7Ym9yZGVyLXJhZGl1czo0cHh9Lm5hdi10YWJzLWp1c3RpZmllZD4uYWN0aXZlPmEsLm5h
di10YWJzLWp1c3RpZmllZD4uYWN0aXZlPmE6Zm9jdXMsLm5hdi10YWJzLWp1c3RpZmllZD4uYWN0
aXZlPmE6aG92ZXJ7Ym9yZGVyOjFweCBzb2xpZCAjZGRkfUBtZWRpYSAobWluLXdpZHRoOjc2OHB4
KXsubmF2LXRhYnMtanVzdGlmaWVkPmxpPmF7Ym9yZGVyLWJvdHRvbToxcHggc29saWQgI2RkZDti
b3JkZXItcmFkaXVzOjRweCA0cHggMCAwfS5uYXYtdGFicy1qdXN0aWZpZWQ+LmFjdGl2ZT5hLC5u
YXYtdGFicy1qdXN0aWZpZWQ+LmFjdGl2ZT5hOmZvY3VzLC5uYXYtdGFicy1qdXN0aWZpZWQ+LmFj
dGl2ZT5hOmhvdmVye2JvcmRlci1ib3R0b20tY29sb3I6I2ZmZn19LnRhYi1jb250ZW50Pi50YWIt
cGFuZXtkaXNwbGF5Om5vbmV9LnRhYi1jb250ZW50Pi5hY3RpdmV7ZGlzcGxheTpibG9ja30ubmF2
LXRhYnMgLmRyb3Bkb3duLW1lbnV7bWFyZ2luLXRvcDotMXB4O2JvcmRlci10b3AtbGVmdC1yYWRp
dXM6MDtib3JkZXItdG9wLXJpZ2h0LXJhZGl1czowfS5uYXZiYXJ7cG9zaXRpb246cmVsYXRpdmU7
bWluLWhlaWdodDo1MHB4O21hcmdpbi1ib3R0b206MjBweDtib3JkZXI6MXB4IHNvbGlkIHRyYW5z
cGFyZW50fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsubmF2YmFye2JvcmRlci1yYWRpdXM6NHB4
fX1AbWVkaWEgKG1pbi13aWR0aDo3NjhweCl7Lm5hdmJhci1oZWFkZXJ7ZmxvYXQ6bGVmdH19Lm5h
dmJhci1jb2xsYXBzZXtwYWRkaW5nLXJpZ2h0OjE1cHg7cGFkZGluZy1sZWZ0OjE1cHg7b3ZlcmZs
b3cteDp2aXNpYmxlOy13ZWJraXQtb3ZlcmZsb3ctc2Nyb2xsaW5nOnRvdWNoO2JvcmRlci10b3A6
MXB4IHNvbGlkIHRyYW5zcGFyZW50Oy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFweCAwIHJn
YmEoMjU1LDI1NSwyNTUsLjEpO2JveC1zaGFkb3c6aW5zZXQgMCAxcHggMCByZ2JhKDI1NSwyNTUs
MjU1LC4xKX0ubmF2YmFyLWNvbGxhcHNlLmlue292ZXJmbG93LXk6YXV0b31AbWVkaWEgKG1pbi13
aWR0aDo3NjhweCl7Lm5hdmJhci1jb2xsYXBzZXt3aWR0aDphdXRvO2JvcmRlci10b3A6MDstd2Vi
a2l0LWJveC1zaGFkb3c6bm9uZTtib3gtc2hhZG93Om5vbmV9Lm5hdmJhci1jb2xsYXBzZS5jb2xs
YXBzZXtkaXNwbGF5OmJsb2NrIWltcG9ydGFudDtoZWlnaHQ6YXV0byFpbXBvcnRhbnQ7cGFkZGlu
Zy1ib3R0b206MDtvdmVyZmxvdzp2aXNpYmxlIWltcG9ydGFudH0ubmF2YmFyLWNvbGxhcHNlLmlu
e292ZXJmbG93LXk6dmlzaWJsZX0ubmF2YmFyLWZpeGVkLWJvdHRvbSAubmF2YmFyLWNvbGxhcHNl
LC5uYXZiYXItZml4ZWQtdG9wIC5uYXZiYXItY29sbGFwc2UsLm5hdmJhci1zdGF0aWMtdG9wIC5u
YXZiYXItY29sbGFwc2V7cGFkZGluZy1yaWdodDowO3BhZGRpbmctbGVmdDowfX0ubmF2YmFyLWZp
eGVkLWJvdHRvbSAubmF2YmFyLWNvbGxhcHNlLC5uYXZiYXItZml4ZWQtdG9wIC5uYXZiYXItY29s
bGFwc2V7bWF4LWhlaWdodDozNDBweH1AbWVkaWEgKG1heC1kZXZpY2Utd2lkdGg6NDgwcHgpIGFu
ZCAob3JpZW50YXRpb246bGFuZHNjYXBlKXsubmF2YmFyLWZpeGVkLWJvdHRvbSAubmF2YmFyLWNv
bGxhcHNlLC5uYXZiYXItZml4ZWQtdG9wIC5uYXZiYXItY29sbGFwc2V7bWF4LWhlaWdodDoyMDBw
eH19LmNvbnRhaW5lci1mbHVpZD4ubmF2YmFyLWNvbGxhcHNlLC5jb250YWluZXItZmx1aWQ+Lm5h
dmJhci1oZWFkZXIsLmNvbnRhaW5lcj4ubmF2YmFyLWNvbGxhcHNlLC5jb250YWluZXI+Lm5hdmJh
ci1oZWFkZXJ7bWFyZ2luLXJpZ2h0Oi0xNXB4O21hcmdpbi1sZWZ0Oi0xNXB4fUBtZWRpYSAobWlu
LXdpZHRoOjc2OHB4KXsuY29udGFpbmVyLWZsdWlkPi5uYXZiYXItY29sbGFwc2UsLmNvbnRhaW5l
ci1mbHVpZD4ubmF2YmFyLWhlYWRlciwuY29udGFpbmVyPi5uYXZiYXItY29sbGFwc2UsLmNvbnRh
aW5lcj4ubmF2YmFyLWhlYWRlcnttYXJnaW4tcmlnaHQ6MDttYXJnaW4tbGVmdDowfX0ubmF2YmFy
LXN0YXRpYy10b3B7ei1pbmRleDoxMDAwO2JvcmRlci13aWR0aDowIDAgMXB4fUBtZWRpYSAobWlu
LXdpZHRoOjc2OHB4KXsubmF2YmFyLXN0YXRpYy10b3B7Ym9yZGVyLXJhZGl1czowfX0ubmF2YmFy
LWZpeGVkLWJvdHRvbSwubmF2YmFyLWZpeGVkLXRvcHtwb3NpdGlvbjpmaXhlZDtyaWdodDowO2xl
ZnQ6MDt6LWluZGV4OjEwMzB9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5uYXZiYXItZml4ZWQt
Ym90dG9tLC5uYXZiYXItZml4ZWQtdG9we2JvcmRlci1yYWRpdXM6MH19Lm5hdmJhci1maXhlZC10
b3B7dG9wOjA7Ym9yZGVyLXdpZHRoOjAgMCAxcHh9Lm5hdmJhci1maXhlZC1ib3R0b217Ym90dG9t
OjA7bWFyZ2luLWJvdHRvbTowO2JvcmRlci13aWR0aDoxcHggMCAwfS5uYXZiYXItYnJhbmR7Zmxv
YXQ6bGVmdDtoZWlnaHQ6NTBweDtwYWRkaW5nOjE1cHggMTVweDtmb250LXNpemU6MThweDtsaW5l
LWhlaWdodDoyMHB4fS5uYXZiYXItYnJhbmQ6Zm9jdXMsLm5hdmJhci1icmFuZDpob3Zlcnt0ZXh0
LWRlY29yYXRpb246bm9uZX0ubmF2YmFyLWJyYW5kPmltZ3tkaXNwbGF5OmJsb2NrfUBtZWRpYSAo
bWluLXdpZHRoOjc2OHB4KXsubmF2YmFyPi5jb250YWluZXIgLm5hdmJhci1icmFuZCwubmF2YmFy
Pi5jb250YWluZXItZmx1aWQgLm5hdmJhci1icmFuZHttYXJnaW4tbGVmdDotMTVweH19Lm5hdmJh
ci10b2dnbGV7cG9zaXRpb246cmVsYXRpdmU7ZmxvYXQ6cmlnaHQ7cGFkZGluZzo5cHggMTBweDtt
YXJnaW4tdG9wOjhweDttYXJnaW4tcmlnaHQ6MTVweDttYXJnaW4tYm90dG9tOjhweDtiYWNrZ3Jv
dW5kLWNvbG9yOnRyYW5zcGFyZW50O2JhY2tncm91bmQtaW1hZ2U6bm9uZTtib3JkZXI6MXB4IHNv
bGlkIHRyYW5zcGFyZW50O2JvcmRlci1yYWRpdXM6NHB4fS5uYXZiYXItdG9nZ2xlOmZvY3Vze291
dGxpbmU6MH0ubmF2YmFyLXRvZ2dsZSAuaWNvbi1iYXJ7ZGlzcGxheTpibG9jazt3aWR0aDoyMnB4
O2hlaWdodDoycHg7Ym9yZGVyLXJhZGl1czoxcHh9Lm5hdmJhci10b2dnbGUgLmljb24tYmFyKy5p
Y29uLWJhcnttYXJnaW4tdG9wOjRweH1AbWVkaWEgKG1pbi13aWR0aDo3NjhweCl7Lm5hdmJhci10
b2dnbGV7ZGlzcGxheTpub25lfX0ubmF2YmFyLW5hdnttYXJnaW46Ny41cHggLTE1cHh9Lm5hdmJh
ci1uYXY+bGk+YXtwYWRkaW5nLXRvcDoxMHB4O3BhZGRpbmctYm90dG9tOjEwcHg7bGluZS1oZWln
aHQ6MjBweH1AbWVkaWEgKG1heC13aWR0aDo3NjdweCl7Lm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bk
b3duLW1lbnV7cG9zaXRpb246c3RhdGljO2Zsb2F0Om5vbmU7d2lkdGg6YXV0bzttYXJnaW4tdG9w
OjA7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtib3JkZXI6MDstd2Via2l0LWJveC1zaGFk
b3c6bm9uZTtib3gtc2hhZG93Om5vbmV9Lm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bkb3duLW1lbnUg
LmRyb3Bkb3duLWhlYWRlciwubmF2YmFyLW5hdiAub3BlbiAuZHJvcGRvd24tbWVudT5saT5he3Bh
ZGRpbmc6NXB4IDE1cHggNXB4IDI1cHh9Lm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bkb3duLW1lbnU+
bGk+YXtsaW5lLWhlaWdodDoyMHB4fS5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1tZW51Pmxp
PmE6Zm9jdXMsLm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bkb3duLW1lbnU+bGk+YTpob3ZlcntiYWNr
Z3JvdW5kLWltYWdlOm5vbmV9fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KXsubmF2YmFyLW5hdntm
bG9hdDpsZWZ0O21hcmdpbjowfS5uYXZiYXItbmF2Pmxpe2Zsb2F0OmxlZnR9Lm5hdmJhci1uYXY+
bGk+YXtwYWRkaW5nLXRvcDoxNXB4O3BhZGRpbmctYm90dG9tOjE1cHh9fS5uYXZiYXItZm9ybXtw
YWRkaW5nOjEwcHggMTVweDttYXJnaW4tdG9wOjhweDttYXJnaW4tcmlnaHQ6LTE1cHg7bWFyZ2lu
LWJvdHRvbTo4cHg7bWFyZ2luLWxlZnQ6LTE1cHg7Ym9yZGVyLXRvcDoxcHggc29saWQgdHJhbnNw
YXJlbnQ7Ym9yZGVyLWJvdHRvbToxcHggc29saWQgdHJhbnNwYXJlbnQ7LXdlYmtpdC1ib3gtc2hh
ZG93Omluc2V0IDAgMXB4IDAgcmdiYSgyNTUsMjU1LDI1NSwuMSksMCAxcHggMCByZ2JhKDI1NSwy
NTUsMjU1LC4xKTtib3gtc2hhZG93Omluc2V0IDAgMXB4IDAgcmdiYSgyNTUsMjU1LDI1NSwuMSks
MCAxcHggMCByZ2JhKDI1NSwyNTUsMjU1LC4xKX1AbWVkaWEgKG1pbi13aWR0aDo3NjhweCl7Lm5h
dmJhci1mb3JtIC5mb3JtLWdyb3Vwe2Rpc3BsYXk6aW5saW5lLWJsb2NrO21hcmdpbi1ib3R0b206
MDt2ZXJ0aWNhbC1hbGlnbjptaWRkbGV9Lm5hdmJhci1mb3JtIC5mb3JtLWNvbnRyb2x7ZGlzcGxh
eTppbmxpbmUtYmxvY2s7d2lkdGg6YXV0bzt2ZXJ0aWNhbC1hbGlnbjptaWRkbGV9Lm5hdmJhci1m
b3JtIC5mb3JtLWNvbnRyb2wtc3RhdGlje2Rpc3BsYXk6aW5saW5lLWJsb2NrfS5uYXZiYXItZm9y
bSAuaW5wdXQtZ3JvdXB7ZGlzcGxheTppbmxpbmUtdGFibGU7dmVydGljYWwtYWxpZ246bWlkZGxl
fS5uYXZiYXItZm9ybSAuaW5wdXQtZ3JvdXAgLmZvcm0tY29udHJvbCwubmF2YmFyLWZvcm0gLmlu
cHV0LWdyb3VwIC5pbnB1dC1ncm91cC1hZGRvbiwubmF2YmFyLWZvcm0gLmlucHV0LWdyb3VwIC5p
bnB1dC1ncm91cC1idG57d2lkdGg6YXV0b30ubmF2YmFyLWZvcm0gLmlucHV0LWdyb3VwPi5mb3Jt
LWNvbnRyb2x7d2lkdGg6MTAwJX0ubmF2YmFyLWZvcm0gLmNvbnRyb2wtbGFiZWx7bWFyZ2luLWJv
dHRvbTowO3ZlcnRpY2FsLWFsaWduOm1pZGRsZX0ubmF2YmFyLWZvcm0gLmNoZWNrYm94LC5uYXZi
YXItZm9ybSAucmFkaW97ZGlzcGxheTppbmxpbmUtYmxvY2s7bWFyZ2luLXRvcDowO21hcmdpbi1i
b3R0b206MDt2ZXJ0aWNhbC1hbGlnbjptaWRkbGV9Lm5hdmJhci1mb3JtIC5jaGVja2JveCBsYWJl
bCwubmF2YmFyLWZvcm0gLnJhZGlvIGxhYmVse3BhZGRpbmctbGVmdDowfS5uYXZiYXItZm9ybSAu
Y2hlY2tib3ggaW5wdXRbdHlwZT1jaGVja2JveF0sLm5hdmJhci1mb3JtIC5yYWRpbyBpbnB1dFt0
eXBlPXJhZGlvXXtwb3NpdGlvbjpyZWxhdGl2ZTttYXJnaW4tbGVmdDowfS5uYXZiYXItZm9ybSAu
aGFzLWZlZWRiYWNrIC5mb3JtLWNvbnRyb2wtZmVlZGJhY2t7dG9wOjB9fUBtZWRpYSAobWF4LXdp
ZHRoOjc2N3B4KXsubmF2YmFyLWZvcm0gLmZvcm0tZ3JvdXB7bWFyZ2luLWJvdHRvbTo1cHh9Lm5h
dmJhci1mb3JtIC5mb3JtLWdyb3VwOmxhc3QtY2hpbGR7bWFyZ2luLWJvdHRvbTowfX1AbWVkaWEg
KG1pbi13aWR0aDo3NjhweCl7Lm5hdmJhci1mb3Jte3dpZHRoOmF1dG87cGFkZGluZy10b3A6MDtw
YWRkaW5nLWJvdHRvbTowO21hcmdpbi1yaWdodDowO21hcmdpbi1sZWZ0OjA7Ym9yZGVyOjA7LXdl
YmtpdC1ib3gtc2hhZG93Om5vbmU7Ym94LXNoYWRvdzpub25lfX0ubmF2YmFyLW5hdj5saT4uZHJv
cGRvd24tbWVudXttYXJnaW4tdG9wOjA7Ym9yZGVyLXRvcC1sZWZ0LXJhZGl1czowO2JvcmRlci10
b3AtcmlnaHQtcmFkaXVzOjB9Lm5hdmJhci1maXhlZC1ib3R0b20gLm5hdmJhci1uYXY+bGk+LmRy
b3Bkb3duLW1lbnV7bWFyZ2luLWJvdHRvbTowO2JvcmRlci10b3AtbGVmdC1yYWRpdXM6NHB4O2Jv
cmRlci10b3AtcmlnaHQtcmFkaXVzOjRweDtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1czowO2Jv
cmRlci1ib3R0b20tbGVmdC1yYWRpdXM6MH0ubmF2YmFyLWJ0bnttYXJnaW4tdG9wOjhweDttYXJn
aW4tYm90dG9tOjhweH0ubmF2YmFyLWJ0bi5idG4tc217bWFyZ2luLXRvcDoxMHB4O21hcmdpbi1i
b3R0b206MTBweH0ubmF2YmFyLWJ0bi5idG4teHN7bWFyZ2luLXRvcDoxNHB4O21hcmdpbi1ib3R0
b206MTRweH0ubmF2YmFyLXRleHR7bWFyZ2luLXRvcDoxNXB4O21hcmdpbi1ib3R0b206MTVweH1A
bWVkaWEgKG1pbi13aWR0aDo3NjhweCl7Lm5hdmJhci10ZXh0e2Zsb2F0OmxlZnQ7bWFyZ2luLXJp
Z2h0OjE1cHg7bWFyZ2luLWxlZnQ6MTVweH19QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpey5uYXZi
YXItbGVmdHtmbG9hdDpsZWZ0IWltcG9ydGFudH0ubmF2YmFyLXJpZ2h0e2Zsb2F0OnJpZ2h0IWlt
cG9ydGFudDttYXJnaW4tcmlnaHQ6LTE1cHh9Lm5hdmJhci1yaWdodH4ubmF2YmFyLXJpZ2h0e21h
cmdpbi1yaWdodDowfX0ubmF2YmFyLWRlZmF1bHR7YmFja2dyb3VuZC1jb2xvcjojZjhmOGY4O2Jv
cmRlci1jb2xvcjojZTdlN2U3fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLWJyYW5ke2NvbG9yOiM3
Nzd9Lm5hdmJhci1kZWZhdWx0IC5uYXZiYXItYnJhbmQ6Zm9jdXMsLm5hdmJhci1kZWZhdWx0IC5u
YXZiYXItYnJhbmQ6aG92ZXJ7Y29sb3I6IzVlNWU1ZTtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFy
ZW50fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLXRleHR7Y29sb3I6Izc3N30ubmF2YmFyLWRlZmF1
bHQgLm5hdmJhci1uYXY+bGk+YXtjb2xvcjojNzc3fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLW5h
dj5saT5hOmZvY3VzLC5uYXZiYXItZGVmYXVsdCAubmF2YmFyLW5hdj5saT5hOmhvdmVye2NvbG9y
OiMzMzM7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudH0ubmF2YmFyLWRlZmF1bHQgLm5hdmJh
ci1uYXY+LmFjdGl2ZT5hLC5uYXZiYXItZGVmYXVsdCAubmF2YmFyLW5hdj4uYWN0aXZlPmE6Zm9j
dXMsLm5hdmJhci1kZWZhdWx0IC5uYXZiYXItbmF2Pi5hY3RpdmU+YTpob3Zlcntjb2xvcjojNTU1
O2JhY2tncm91bmQtY29sb3I6I2U3ZTdlN30ubmF2YmFyLWRlZmF1bHQgLm5hdmJhci1uYXY+LmRp
c2FibGVkPmEsLm5hdmJhci1kZWZhdWx0IC5uYXZiYXItbmF2Pi5kaXNhYmxlZD5hOmZvY3VzLC5u
YXZiYXItZGVmYXVsdCAubmF2YmFyLW5hdj4uZGlzYWJsZWQ+YTpob3Zlcntjb2xvcjojY2NjO2Jh
Y2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnR9Lm5hdmJhci1kZWZhdWx0IC5uYXZiYXItdG9nZ2xl
e2JvcmRlci1jb2xvcjojZGRkfS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLXRvZ2dsZTpmb2N1cywu
bmF2YmFyLWRlZmF1bHQgLm5hdmJhci10b2dnbGU6aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojZGRk
fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLXRvZ2dsZSAuaWNvbi1iYXJ7YmFja2dyb3VuZC1jb2xv
cjojODg4fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLWNvbGxhcHNlLC5uYXZiYXItZGVmYXVsdCAu
bmF2YmFyLWZvcm17Ym9yZGVyLWNvbG9yOiNlN2U3ZTd9Lm5hdmJhci1kZWZhdWx0IC5uYXZiYXIt
bmF2Pi5vcGVuPmEsLm5hdmJhci1kZWZhdWx0IC5uYXZiYXItbmF2Pi5vcGVuPmE6Zm9jdXMsLm5h
dmJhci1kZWZhdWx0IC5uYXZiYXItbmF2Pi5vcGVuPmE6aG92ZXJ7Y29sb3I6IzU1NTtiYWNrZ3Jv
dW5kLWNvbG9yOiNlN2U3ZTd9QG1lZGlhIChtYXgtd2lkdGg6NzY3cHgpey5uYXZiYXItZGVmYXVs
dCAubmF2YmFyLW5hdiAub3BlbiAuZHJvcGRvd24tbWVudT5saT5he2NvbG9yOiM3Nzd9Lm5hdmJh
ci1kZWZhdWx0IC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1tZW51PmxpPmE6Zm9jdXMsLm5h
dmJhci1kZWZhdWx0IC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1tZW51PmxpPmE6aG92ZXJ7
Y29sb3I6IzMzMztiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50fS5uYXZiYXItZGVmYXVsdCAu
bmF2YmFyLW5hdiAub3BlbiAuZHJvcGRvd24tbWVudT4uYWN0aXZlPmEsLm5hdmJhci1kZWZhdWx0
IC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1tZW51Pi5hY3RpdmU+YTpmb2N1cywubmF2YmFy
LWRlZmF1bHQgLm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bkb3duLW1lbnU+LmFjdGl2ZT5hOmhvdmVy
e2NvbG9yOiM1NTU7YmFja2dyb3VuZC1jb2xvcjojZTdlN2U3fS5uYXZiYXItZGVmYXVsdCAubmF2
YmFyLW5hdiAub3BlbiAuZHJvcGRvd24tbWVudT4uZGlzYWJsZWQ+YSwubmF2YmFyLWRlZmF1bHQg
Lm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bkb3duLW1lbnU+LmRpc2FibGVkPmE6Zm9jdXMsLm5hdmJh
ci1kZWZhdWx0IC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1tZW51Pi5kaXNhYmxlZD5hOmhv
dmVye2NvbG9yOiNjY2M7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudH19Lm5hdmJhci1kZWZh
dWx0IC5uYXZiYXItbGlua3tjb2xvcjojNzc3fS5uYXZiYXItZGVmYXVsdCAubmF2YmFyLWxpbms6
aG92ZXJ7Y29sb3I6IzMzM30ubmF2YmFyLWRlZmF1bHQgLmJ0bi1saW5re2NvbG9yOiM3Nzd9Lm5h
dmJhci1kZWZhdWx0IC5idG4tbGluazpmb2N1cywubmF2YmFyLWRlZmF1bHQgLmJ0bi1saW5rOmhv
dmVye2NvbG9yOiMzMzN9Lm5hdmJhci1kZWZhdWx0IC5idG4tbGlua1tkaXNhYmxlZF06Zm9jdXMs
Lm5hdmJhci1kZWZhdWx0IC5idG4tbGlua1tkaXNhYmxlZF06aG92ZXIsZmllbGRzZXRbZGlzYWJs
ZWRdIC5uYXZiYXItZGVmYXVsdCAuYnRuLWxpbms6Zm9jdXMsZmllbGRzZXRbZGlzYWJsZWRdIC5u
YXZiYXItZGVmYXVsdCAuYnRuLWxpbms6aG92ZXJ7Y29sb3I6I2NjY30ubmF2YmFyLWludmVyc2V7
YmFja2dyb3VuZC1jb2xvcjojMjIyO2JvcmRlci1jb2xvcjojMDgwODA4fS5uYXZiYXItaW52ZXJz
ZSAubmF2YmFyLWJyYW5ke2NvbG9yOiM5ZDlkOWR9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItYnJh
bmQ6Zm9jdXMsLm5hdmJhci1pbnZlcnNlIC5uYXZiYXItYnJhbmQ6aG92ZXJ7Y29sb3I6I2ZmZjti
YWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50fS5uYXZiYXItaW52ZXJzZSAubmF2YmFyLXRleHR7
Y29sb3I6IzlkOWQ5ZH0ubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXY+bGk+YXtjb2xvcjojOWQ5
ZDlkfS5uYXZiYXItaW52ZXJzZSAubmF2YmFyLW5hdj5saT5hOmZvY3VzLC5uYXZiYXItaW52ZXJz
ZSAubmF2YmFyLW5hdj5saT5hOmhvdmVye2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjp0cmFu
c3BhcmVudH0ubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXY+LmFjdGl2ZT5hLC5uYXZiYXItaW52
ZXJzZSAubmF2YmFyLW5hdj4uYWN0aXZlPmE6Zm9jdXMsLm5hdmJhci1pbnZlcnNlIC5uYXZiYXIt
bmF2Pi5hY3RpdmU+YTpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6IzA4MDgwOH0u
bmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXY+LmRpc2FibGVkPmEsLm5hdmJhci1pbnZlcnNlIC5u
YXZiYXItbmF2Pi5kaXNhYmxlZD5hOmZvY3VzLC5uYXZiYXItaW52ZXJzZSAubmF2YmFyLW5hdj4u
ZGlzYWJsZWQ+YTpob3Zlcntjb2xvcjojNDQ0O2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnR9
Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItdG9nZ2xle2JvcmRlci1jb2xvcjojMzMzfS5uYXZiYXIt
aW52ZXJzZSAubmF2YmFyLXRvZ2dsZTpmb2N1cywubmF2YmFyLWludmVyc2UgLm5hdmJhci10b2dn
bGU6aG92ZXJ7YmFja2dyb3VuZC1jb2xvcjojMzMzfS5uYXZiYXItaW52ZXJzZSAubmF2YmFyLXRv
Z2dsZSAuaWNvbi1iYXJ7YmFja2dyb3VuZC1jb2xvcjojZmZmfS5uYXZiYXItaW52ZXJzZSAubmF2
YmFyLWNvbGxhcHNlLC5uYXZiYXItaW52ZXJzZSAubmF2YmFyLWZvcm17Ym9yZGVyLWNvbG9yOiMx
MDEwMTB9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbmF2Pi5vcGVuPmEsLm5hdmJhci1pbnZlcnNl
IC5uYXZiYXItbmF2Pi5vcGVuPmE6Zm9jdXMsLm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbmF2Pi5v
cGVuPmE6aG92ZXJ7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMwODA4MDh9QG1lZGlhICht
YXgtd2lkdGg6NzY3cHgpey5uYXZiYXItaW52ZXJzZSAubmF2YmFyLW5hdiAub3BlbiAuZHJvcGRv
d24tbWVudT4uZHJvcGRvd24taGVhZGVye2JvcmRlci1jb2xvcjojMDgwODA4fS5uYXZiYXItaW52
ZXJzZSAubmF2YmFyLW5hdiAub3BlbiAuZHJvcGRvd24tbWVudSAuZGl2aWRlcntiYWNrZ3JvdW5k
LWNvbG9yOiMwODA4MDh9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93
bi1tZW51PmxpPmF7Y29sb3I6IzlkOWQ5ZH0ubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXYgLm9w
ZW4gLmRyb3Bkb3duLW1lbnU+bGk+YTpmb2N1cywubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXYg
Lm9wZW4gLmRyb3Bkb3duLW1lbnU+bGk+YTpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29s
b3I6dHJhbnNwYXJlbnR9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93
bi1tZW51Pi5hY3RpdmU+YSwubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXYgLm9wZW4gLmRyb3Bk
b3duLW1lbnU+LmFjdGl2ZT5hOmZvY3VzLC5uYXZiYXItaW52ZXJzZSAubmF2YmFyLW5hdiAub3Bl
biAuZHJvcGRvd24tbWVudT4uYWN0aXZlPmE6aG92ZXJ7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNv
bG9yOiMwODA4MDh9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbmF2IC5vcGVuIC5kcm9wZG93bi1t
ZW51Pi5kaXNhYmxlZD5hLC5uYXZiYXItaW52ZXJzZSAubmF2YmFyLW5hdiAub3BlbiAuZHJvcGRv
d24tbWVudT4uZGlzYWJsZWQ+YTpmb2N1cywubmF2YmFyLWludmVyc2UgLm5hdmJhci1uYXYgLm9w
ZW4gLmRyb3Bkb3duLW1lbnU+LmRpc2FibGVkPmE6aG92ZXJ7Y29sb3I6IzQ0NDtiYWNrZ3JvdW5k
LWNvbG9yOnRyYW5zcGFyZW50fX0ubmF2YmFyLWludmVyc2UgLm5hdmJhci1saW5re2NvbG9yOiM5
ZDlkOWR9Lm5hdmJhci1pbnZlcnNlIC5uYXZiYXItbGluazpob3Zlcntjb2xvcjojZmZmfS5uYXZi
YXItaW52ZXJzZSAuYnRuLWxpbmt7Y29sb3I6IzlkOWQ5ZH0ubmF2YmFyLWludmVyc2UgLmJ0bi1s
aW5rOmZvY3VzLC5uYXZiYXItaW52ZXJzZSAuYnRuLWxpbms6aG92ZXJ7Y29sb3I6I2ZmZn0ubmF2
YmFyLWludmVyc2UgLmJ0bi1saW5rW2Rpc2FibGVkXTpmb2N1cywubmF2YmFyLWludmVyc2UgLmJ0
bi1saW5rW2Rpc2FibGVkXTpob3ZlcixmaWVsZHNldFtkaXNhYmxlZF0gLm5hdmJhci1pbnZlcnNl
IC5idG4tbGluazpmb2N1cyxmaWVsZHNldFtkaXNhYmxlZF0gLm5hdmJhci1pbnZlcnNlIC5idG4t
bGluazpob3Zlcntjb2xvcjojNDQ0fS5icmVhZGNydW1ie3BhZGRpbmc6OHB4IDE1cHg7bWFyZ2lu
LWJvdHRvbToyMHB4O2xpc3Qtc3R5bGU6bm9uZTtiYWNrZ3JvdW5kLWNvbG9yOiNmNWY1ZjU7Ym9y
ZGVyLXJhZGl1czo0cHh9LmJyZWFkY3J1bWI+bGl7ZGlzcGxheTppbmxpbmUtYmxvY2t9LmJyZWFk
Y3J1bWI+bGkrbGk6YmVmb3Jle3BhZGRpbmc6MCA1cHg7Y29sb3I6I2NjYztjb250ZW50OiIvXDAw
YTAifS5icmVhZGNydW1iPi5hY3RpdmV7Y29sb3I6Izc3N30ucGFnaW5hdGlvbntkaXNwbGF5Omlu
bGluZS1ibG9jaztwYWRkaW5nLWxlZnQ6MDttYXJnaW46MjBweCAwO2JvcmRlci1yYWRpdXM6NHB4
fS5wYWdpbmF0aW9uPmxpe2Rpc3BsYXk6aW5saW5lfS5wYWdpbmF0aW9uPmxpPmEsLnBhZ2luYXRp
b24+bGk+c3Bhbntwb3NpdGlvbjpyZWxhdGl2ZTtmbG9hdDpsZWZ0O3BhZGRpbmc6NnB4IDEycHg7
bWFyZ2luLWxlZnQ6LTFweDtsaW5lLWhlaWdodDoxLjQyODU3MTQzO2NvbG9yOiMzMzdhYjc7dGV4
dC1kZWNvcmF0aW9uOm5vbmU7YmFja2dyb3VuZC1jb2xvcjojZmZmO2JvcmRlcjoxcHggc29saWQg
I2RkZH0ucGFnaW5hdGlvbj5saTpmaXJzdC1jaGlsZD5hLC5wYWdpbmF0aW9uPmxpOmZpcnN0LWNo
aWxkPnNwYW57bWFyZ2luLWxlZnQ6MDtib3JkZXItdG9wLWxlZnQtcmFkaXVzOjRweDtib3JkZXIt
Ym90dG9tLWxlZnQtcmFkaXVzOjRweH0ucGFnaW5hdGlvbj5saTpsYXN0LWNoaWxkPmEsLnBhZ2lu
YXRpb24+bGk6bGFzdC1jaGlsZD5zcGFue2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjRweDtib3Jk
ZXItYm90dG9tLXJpZ2h0LXJhZGl1czo0cHh9LnBhZ2luYXRpb24+bGk+YTpmb2N1cywucGFnaW5h
dGlvbj5saT5hOmhvdmVyLC5wYWdpbmF0aW9uPmxpPnNwYW46Zm9jdXMsLnBhZ2luYXRpb24+bGk+
c3Bhbjpob3Zlcnt6LWluZGV4OjI7Y29sb3I6IzIzNTI3YztiYWNrZ3JvdW5kLWNvbG9yOiNlZWU7
Ym9yZGVyLWNvbG9yOiNkZGR9LnBhZ2luYXRpb24+LmFjdGl2ZT5hLC5wYWdpbmF0aW9uPi5hY3Rp
dmU+YTpmb2N1cywucGFnaW5hdGlvbj4uYWN0aXZlPmE6aG92ZXIsLnBhZ2luYXRpb24+LmFjdGl2
ZT5zcGFuLC5wYWdpbmF0aW9uPi5hY3RpdmU+c3Bhbjpmb2N1cywucGFnaW5hdGlvbj4uYWN0aXZl
PnNwYW46aG92ZXJ7ei1pbmRleDozO2NvbG9yOiNmZmY7Y3Vyc29yOmRlZmF1bHQ7YmFja2dyb3Vu
ZC1jb2xvcjojMzM3YWI3O2JvcmRlci1jb2xvcjojMzM3YWI3fS5wYWdpbmF0aW9uPi5kaXNhYmxl
ZD5hLC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5hOmZvY3VzLC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5h
OmhvdmVyLC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5zcGFuLC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5z
cGFuOmZvY3VzLC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5zcGFuOmhvdmVye2NvbG9yOiM3Nzc7Y3Vy
c29yOm5vdC1hbGxvd2VkO2JhY2tncm91bmQtY29sb3I6I2ZmZjtib3JkZXItY29sb3I6I2RkZH0u
cGFnaW5hdGlvbi1sZz5saT5hLC5wYWdpbmF0aW9uLWxnPmxpPnNwYW57cGFkZGluZzoxMHB4IDE2
cHg7Zm9udC1zaXplOjE4cHg7bGluZS1oZWlnaHQ6MS4zMzMzMzMzfS5wYWdpbmF0aW9uLWxnPmxp
OmZpcnN0LWNoaWxkPmEsLnBhZ2luYXRpb24tbGc+bGk6Zmlyc3QtY2hpbGQ+c3Bhbntib3JkZXIt
dG9wLWxlZnQtcmFkaXVzOjZweDtib3JkZXItYm90dG9tLWxlZnQtcmFkaXVzOjZweH0ucGFnaW5h
dGlvbi1sZz5saTpsYXN0LWNoaWxkPmEsLnBhZ2luYXRpb24tbGc+bGk6bGFzdC1jaGlsZD5zcGFu
e2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjZweDtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1czo2
cHh9LnBhZ2luYXRpb24tc20+bGk+YSwucGFnaW5hdGlvbi1zbT5saT5zcGFue3BhZGRpbmc6NXB4
IDEwcHg7Zm9udC1zaXplOjEycHg7bGluZS1oZWlnaHQ6MS41fS5wYWdpbmF0aW9uLXNtPmxpOmZp
cnN0LWNoaWxkPmEsLnBhZ2luYXRpb24tc20+bGk6Zmlyc3QtY2hpbGQ+c3Bhbntib3JkZXItdG9w
LWxlZnQtcmFkaXVzOjNweDtib3JkZXItYm90dG9tLWxlZnQtcmFkaXVzOjNweH0ucGFnaW5hdGlv
bi1zbT5saTpsYXN0LWNoaWxkPmEsLnBhZ2luYXRpb24tc20+bGk6bGFzdC1jaGlsZD5zcGFue2Jv
cmRlci10b3AtcmlnaHQtcmFkaXVzOjNweDtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1czozcHh9
LnBhZ2Vye3BhZGRpbmctbGVmdDowO21hcmdpbjoyMHB4IDA7dGV4dC1hbGlnbjpjZW50ZXI7bGlz
dC1zdHlsZTpub25lfS5wYWdlciBsaXtkaXNwbGF5OmlubGluZX0ucGFnZXIgbGk+YSwucGFnZXIg
bGk+c3BhbntkaXNwbGF5OmlubGluZS1ibG9jaztwYWRkaW5nOjVweCAxNHB4O2JhY2tncm91bmQt
Y29sb3I6I2ZmZjtib3JkZXI6MXB4IHNvbGlkICNkZGQ7Ym9yZGVyLXJhZGl1czoxNXB4fS5wYWdl
ciBsaT5hOmZvY3VzLC5wYWdlciBsaT5hOmhvdmVye3RleHQtZGVjb3JhdGlvbjpub25lO2JhY2tn
cm91bmQtY29sb3I6I2VlZX0ucGFnZXIgLm5leHQ+YSwucGFnZXIgLm5leHQ+c3BhbntmbG9hdDpy
aWdodH0ucGFnZXIgLnByZXZpb3VzPmEsLnBhZ2VyIC5wcmV2aW91cz5zcGFue2Zsb2F0OmxlZnR9
LnBhZ2VyIC5kaXNhYmxlZD5hLC5wYWdlciAuZGlzYWJsZWQ+YTpmb2N1cywucGFnZXIgLmRpc2Fi
bGVkPmE6aG92ZXIsLnBhZ2VyIC5kaXNhYmxlZD5zcGFue2NvbG9yOiM3Nzc7Y3Vyc29yOm5vdC1h
bGxvd2VkO2JhY2tncm91bmQtY29sb3I6I2ZmZn0ubGFiZWx7ZGlzcGxheTppbmxpbmU7cGFkZGlu
ZzouMmVtIC42ZW0gLjNlbTtmb250LXNpemU6NzUlO2ZvbnQtd2VpZ2h0OjcwMDtsaW5lLWhlaWdo
dDoxO2NvbG9yOiNmZmY7dGV4dC1hbGlnbjpjZW50ZXI7d2hpdGUtc3BhY2U6bm93cmFwO3ZlcnRp
Y2FsLWFsaWduOmJhc2VsaW5lO2JvcmRlci1yYWRpdXM6LjI1ZW19YS5sYWJlbDpmb2N1cyxhLmxh
YmVsOmhvdmVye2NvbG9yOiNmZmY7dGV4dC1kZWNvcmF0aW9uOm5vbmU7Y3Vyc29yOnBvaW50ZXJ9
LmxhYmVsOmVtcHR5e2Rpc3BsYXk6bm9uZX0uYnRuIC5sYWJlbHtwb3NpdGlvbjpyZWxhdGl2ZTt0
b3A6LTFweH0ubGFiZWwtZGVmYXVsdHtiYWNrZ3JvdW5kLWNvbG9yOiM3Nzd9LmxhYmVsLWRlZmF1
bHRbaHJlZl06Zm9jdXMsLmxhYmVsLWRlZmF1bHRbaHJlZl06aG92ZXJ7YmFja2dyb3VuZC1jb2xv
cjojNWU1ZTVlfS5sYWJlbC1wcmltYXJ5e2JhY2tncm91bmQtY29sb3I6IzMzN2FiN30ubGFiZWwt
cHJpbWFyeVtocmVmXTpmb2N1cywubGFiZWwtcHJpbWFyeVtocmVmXTpob3ZlcntiYWNrZ3JvdW5k
LWNvbG9yOiMyODYwOTB9LmxhYmVsLXN1Y2Nlc3N7YmFja2dyb3VuZC1jb2xvcjojNWNiODVjfS5s
YWJlbC1zdWNjZXNzW2hyZWZdOmZvY3VzLC5sYWJlbC1zdWNjZXNzW2hyZWZdOmhvdmVye2JhY2tn
cm91bmQtY29sb3I6IzQ0OWQ0NH0ubGFiZWwtaW5mb3tiYWNrZ3JvdW5kLWNvbG9yOiM1YmMwZGV9
LmxhYmVsLWluZm9baHJlZl06Zm9jdXMsLmxhYmVsLWluZm9baHJlZl06aG92ZXJ7YmFja2dyb3Vu
ZC1jb2xvcjojMzFiMGQ1fS5sYWJlbC13YXJuaW5ne2JhY2tncm91bmQtY29sb3I6I2YwYWQ0ZX0u
bGFiZWwtd2FybmluZ1tocmVmXTpmb2N1cywubGFiZWwtd2FybmluZ1tocmVmXTpob3ZlcntiYWNr
Z3JvdW5kLWNvbG9yOiNlYzk3MWZ9LmxhYmVsLWRhbmdlcntiYWNrZ3JvdW5kLWNvbG9yOiNkOTUz
NGZ9LmxhYmVsLWRhbmdlcltocmVmXTpmb2N1cywubGFiZWwtZGFuZ2VyW2hyZWZdOmhvdmVye2Jh
Y2tncm91bmQtY29sb3I6I2M5MzAyY30uYmFkZ2V7ZGlzcGxheTppbmxpbmUtYmxvY2s7bWluLXdp
ZHRoOjEwcHg7cGFkZGluZzozcHggN3B4O2ZvbnQtc2l6ZToxMnB4O2ZvbnQtd2VpZ2h0OjcwMDts
aW5lLWhlaWdodDoxO2NvbG9yOiNmZmY7dGV4dC1hbGlnbjpjZW50ZXI7d2hpdGUtc3BhY2U6bm93
cmFwO3ZlcnRpY2FsLWFsaWduOm1pZGRsZTtiYWNrZ3JvdW5kLWNvbG9yOiM3Nzc7Ym9yZGVyLXJh
ZGl1czoxMHB4fS5iYWRnZTplbXB0eXtkaXNwbGF5Om5vbmV9LmJ0biAuYmFkZ2V7cG9zaXRpb246
cmVsYXRpdmU7dG9wOi0xcHh9LmJ0bi1ncm91cC14cz4uYnRuIC5iYWRnZSwuYnRuLXhzIC5iYWRn
ZXt0b3A6MDtwYWRkaW5nOjFweCA1cHh9YS5iYWRnZTpmb2N1cyxhLmJhZGdlOmhvdmVye2NvbG9y
OiNmZmY7dGV4dC1kZWNvcmF0aW9uOm5vbmU7Y3Vyc29yOnBvaW50ZXJ9Lmxpc3QtZ3JvdXAtaXRl
bS5hY3RpdmU+LmJhZGdlLC5uYXYtcGlsbHM+LmFjdGl2ZT5hPi5iYWRnZXtjb2xvcjojMzM3YWI3
O2JhY2tncm91bmQtY29sb3I6I2ZmZn0ubGlzdC1ncm91cC1pdGVtPi5iYWRnZXtmbG9hdDpyaWdo
dH0ubGlzdC1ncm91cC1pdGVtPi5iYWRnZSsuYmFkZ2V7bWFyZ2luLXJpZ2h0OjVweH0ubmF2LXBp
bGxzPmxpPmE+LmJhZGdle21hcmdpbi1sZWZ0OjNweH0uanVtYm90cm9ue3BhZGRpbmctdG9wOjMw
cHg7cGFkZGluZy1ib3R0b206MzBweDttYXJnaW4tYm90dG9tOjMwcHg7Y29sb3I6aW5oZXJpdDti
YWNrZ3JvdW5kLWNvbG9yOiNlZWV9Lmp1bWJvdHJvbiAuaDEsLmp1bWJvdHJvbiBoMXtjb2xvcjpp
bmhlcml0fS5qdW1ib3Ryb24gcHttYXJnaW4tYm90dG9tOjE1cHg7Zm9udC1zaXplOjIxcHg7Zm9u
dC13ZWlnaHQ6MjAwfS5qdW1ib3Ryb24+aHJ7Ym9yZGVyLXRvcC1jb2xvcjojZDVkNWQ1fS5jb250
YWluZXIgLmp1bWJvdHJvbiwuY29udGFpbmVyLWZsdWlkIC5qdW1ib3Ryb257cGFkZGluZy1yaWdo
dDoxNXB4O3BhZGRpbmctbGVmdDoxNXB4O2JvcmRlci1yYWRpdXM6NnB4fS5qdW1ib3Ryb24gLmNv
bnRhaW5lcnttYXgtd2lkdGg6MTAwJX1AbWVkaWEgc2NyZWVuIGFuZCAobWluLXdpZHRoOjc2OHB4
KXsuanVtYm90cm9ue3BhZGRpbmctdG9wOjQ4cHg7cGFkZGluZy1ib3R0b206NDhweH0uY29udGFp
bmVyIC5qdW1ib3Ryb24sLmNvbnRhaW5lci1mbHVpZCAuanVtYm90cm9ue3BhZGRpbmctcmlnaHQ6
NjBweDtwYWRkaW5nLWxlZnQ6NjBweH0uanVtYm90cm9uIC5oMSwuanVtYm90cm9uIGgxe2ZvbnQt
c2l6ZTo2M3B4fX0udGh1bWJuYWlse2Rpc3BsYXk6YmxvY2s7cGFkZGluZzo0cHg7bWFyZ2luLWJv
dHRvbToyMHB4O2xpbmUtaGVpZ2h0OjEuNDI4NTcxNDM7YmFja2dyb3VuZC1jb2xvcjojZmZmO2Jv
cmRlcjoxcHggc29saWQgI2RkZDtib3JkZXItcmFkaXVzOjRweDstd2Via2l0LXRyYW5zaXRpb246
Ym9yZGVyIC4ycyBlYXNlLWluLW91dDstby10cmFuc2l0aW9uOmJvcmRlciAuMnMgZWFzZS1pbi1v
dXQ7dHJhbnNpdGlvbjpib3JkZXIgLjJzIGVhc2UtaW4tb3V0fS50aHVtYm5haWwgYT5pbWcsLnRo
dW1ibmFpbD5pbWd7bWFyZ2luLXJpZ2h0OmF1dG87bWFyZ2luLWxlZnQ6YXV0b31hLnRodW1ibmFp
bC5hY3RpdmUsYS50aHVtYm5haWw6Zm9jdXMsYS50aHVtYm5haWw6aG92ZXJ7Ym9yZGVyLWNvbG9y
OiMzMzdhYjd9LnRodW1ibmFpbCAuY2FwdGlvbntwYWRkaW5nOjlweDtjb2xvcjojMzMzfS5hbGVy
dHtwYWRkaW5nOjE1cHg7bWFyZ2luLWJvdHRvbToyMHB4O2JvcmRlcjoxcHggc29saWQgdHJhbnNw
YXJlbnQ7Ym9yZGVyLXJhZGl1czo0cHh9LmFsZXJ0IGg0e21hcmdpbi10b3A6MDtjb2xvcjppbmhl
cml0fS5hbGVydCAuYWxlcnQtbGlua3tmb250LXdlaWdodDo3MDB9LmFsZXJ0PnAsLmFsZXJ0PnVs
e21hcmdpbi1ib3R0b206MH0uYWxlcnQ+cCtwe21hcmdpbi10b3A6NXB4fS5hbGVydC1kaXNtaXNz
YWJsZSwuYWxlcnQtZGlzbWlzc2libGV7cGFkZGluZy1yaWdodDozNXB4fS5hbGVydC1kaXNtaXNz
YWJsZSAuY2xvc2UsLmFsZXJ0LWRpc21pc3NpYmxlIC5jbG9zZXtwb3NpdGlvbjpyZWxhdGl2ZTt0
b3A6LTJweDtyaWdodDotMjFweDtjb2xvcjppbmhlcml0fS5hbGVydC1zdWNjZXNze2NvbG9yOiMz
Yzc2M2Q7YmFja2dyb3VuZC1jb2xvcjojZGZmMGQ4O2JvcmRlci1jb2xvcjojZDZlOWM2fS5hbGVy
dC1zdWNjZXNzIGhye2JvcmRlci10b3AtY29sb3I6I2M5ZTJiM30uYWxlcnQtc3VjY2VzcyAuYWxl
cnQtbGlua3tjb2xvcjojMmI1NDJjfS5hbGVydC1pbmZve2NvbG9yOiMzMTcwOGY7YmFja2dyb3Vu
ZC1jb2xvcjojZDllZGY3O2JvcmRlci1jb2xvcjojYmNlOGYxfS5hbGVydC1pbmZvIGhye2JvcmRl
ci10b3AtY29sb3I6I2E2ZTFlY30uYWxlcnQtaW5mbyAuYWxlcnQtbGlua3tjb2xvcjojMjQ1MjY5
fS5hbGVydC13YXJuaW5ne2NvbG9yOiM4YTZkM2I7YmFja2dyb3VuZC1jb2xvcjojZmNmOGUzO2Jv
cmRlci1jb2xvcjojZmFlYmNjfS5hbGVydC13YXJuaW5nIGhye2JvcmRlci10b3AtY29sb3I6I2Y3
ZTFiNX0uYWxlcnQtd2FybmluZyAuYWxlcnQtbGlua3tjb2xvcjojNjY1MTJjfS5hbGVydC1kYW5n
ZXJ7Y29sb3I6I2E5NDQ0MjtiYWNrZ3JvdW5kLWNvbG9yOiNmMmRlZGU7Ym9yZGVyLWNvbG9yOiNl
YmNjZDF9LmFsZXJ0LWRhbmdlciBocntib3JkZXItdG9wLWNvbG9yOiNlNGI5YzB9LmFsZXJ0LWRh
bmdlciAuYWxlcnQtbGlua3tjb2xvcjojODQzNTM0fUAtd2Via2l0LWtleWZyYW1lcyBwcm9ncmVz
cy1iYXItc3RyaXBlc3tmcm9te2JhY2tncm91bmQtcG9zaXRpb246NDBweCAwfXRve2JhY2tncm91
bmQtcG9zaXRpb246MCAwfX1ALW8ta2V5ZnJhbWVzIHByb2dyZXNzLWJhci1zdHJpcGVze2Zyb217
YmFja2dyb3VuZC1wb3NpdGlvbjo0MHB4IDB9dG97YmFja2dyb3VuZC1wb3NpdGlvbjowIDB9fUBr
ZXlmcmFtZXMgcHJvZ3Jlc3MtYmFyLXN0cmlwZXN7ZnJvbXtiYWNrZ3JvdW5kLXBvc2l0aW9uOjQw
cHggMH10b3tiYWNrZ3JvdW5kLXBvc2l0aW9uOjAgMH19LnByb2dyZXNze2hlaWdodDoyMHB4O21h
cmdpbi1ib3R0b206MjBweDtvdmVyZmxvdzpoaWRkZW47YmFja2dyb3VuZC1jb2xvcjojZjVmNWY1
O2JvcmRlci1yYWRpdXM6NHB4Oy13ZWJraXQtYm94LXNoYWRvdzppbnNldCAwIDFweCAycHggcmdi
YSgwLDAsMCwuMSk7Ym94LXNoYWRvdzppbnNldCAwIDFweCAycHggcmdiYSgwLDAsMCwuMSl9LnBy
b2dyZXNzLWJhcntmbG9hdDpsZWZ0O3dpZHRoOjA7aGVpZ2h0OjEwMCU7Zm9udC1zaXplOjEycHg7
bGluZS1oZWlnaHQ6MjBweDtjb2xvcjojZmZmO3RleHQtYWxpZ246Y2VudGVyO2JhY2tncm91bmQt
Y29sb3I6IzMzN2FiNzstd2Via2l0LWJveC1zaGFkb3c6aW5zZXQgMCAtMXB4IDAgcmdiYSgwLDAs
MCwuMTUpO2JveC1zaGFkb3c6aW5zZXQgMCAtMXB4IDAgcmdiYSgwLDAsMCwuMTUpOy13ZWJraXQt
dHJhbnNpdGlvbjp3aWR0aCAuNnMgZWFzZTstby10cmFuc2l0aW9uOndpZHRoIC42cyBlYXNlO3Ry
YW5zaXRpb246d2lkdGggLjZzIGVhc2V9LnByb2dyZXNzLWJhci1zdHJpcGVkLC5wcm9ncmVzcy1z
dHJpcGVkIC5wcm9ncmVzcy1iYXJ7YmFja2dyb3VuZC1pbWFnZTotd2Via2l0LWxpbmVhci1ncmFk
aWVudCg0NWRlZyxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgMjUlLHRyYW5zcGFyZW50IDI1JSx0cmFu
c3BhcmVudCA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4x
NSkgNzUlLHRyYW5zcGFyZW50IDc1JSx0cmFuc3BhcmVudCk7YmFja2dyb3VuZC1pbWFnZTotby1s
aW5lYXItZ3JhZGllbnQoNDVkZWcscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDI1JSx0cmFuc3BhcmVu
dCAyNSUsdHJhbnNwYXJlbnQgNTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA1MCUscmdiYSgyNTUs
MjU1LDI1NSwuMTUpIDc1JSx0cmFuc3BhcmVudCA3NSUsdHJhbnNwYXJlbnQpO2JhY2tncm91bmQt
aW1hZ2U6bGluZWFyLWdyYWRpZW50KDQ1ZGVnLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSAyNSUsdHJh
bnNwYXJlbnQgMjUlLHRyYW5zcGFyZW50IDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgNTAlLHJn
YmEoMjU1LDI1NSwyNTUsLjE1KSA3NSUsdHJhbnNwYXJlbnQgNzUlLHRyYW5zcGFyZW50KTstd2Vi
a2l0LWJhY2tncm91bmQtc2l6ZTo0MHB4IDQwcHg7YmFja2dyb3VuZC1zaXplOjQwcHggNDBweH0u
cHJvZ3Jlc3MtYmFyLmFjdGl2ZSwucHJvZ3Jlc3MuYWN0aXZlIC5wcm9ncmVzcy1iYXJ7LXdlYmtp
dC1hbmltYXRpb246cHJvZ3Jlc3MtYmFyLXN0cmlwZXMgMnMgbGluZWFyIGluZmluaXRlOy1vLWFu
aW1hdGlvbjpwcm9ncmVzcy1iYXItc3RyaXBlcyAycyBsaW5lYXIgaW5maW5pdGU7YW5pbWF0aW9u
OnByb2dyZXNzLWJhci1zdHJpcGVzIDJzIGxpbmVhciBpbmZpbml0ZX0ucHJvZ3Jlc3MtYmFyLXN1
Y2Nlc3N7YmFja2dyb3VuZC1jb2xvcjojNWNiODVjfS5wcm9ncmVzcy1zdHJpcGVkIC5wcm9ncmVz
cy1iYXItc3VjY2Vzc3tiYWNrZ3JvdW5kLWltYWdlOi13ZWJraXQtbGluZWFyLWdyYWRpZW50KDQ1
ZGVnLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSAyNSUsdHJhbnNwYXJlbnQgMjUlLHRyYW5zcGFyZW50
IDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgNTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA3NSUs
dHJhbnNwYXJlbnQgNzUlLHRyYW5zcGFyZW50KTtiYWNrZ3JvdW5kLWltYWdlOi1vLWxpbmVhci1n
cmFkaWVudCg0NWRlZyxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgMjUlLHRyYW5zcGFyZW50IDI1JSx0
cmFuc3BhcmVudCA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDUwJSxyZ2JhKDI1NSwyNTUsMjU1
LC4xNSkgNzUlLHRyYW5zcGFyZW50IDc1JSx0cmFuc3BhcmVudCk7YmFja2dyb3VuZC1pbWFnZTps
aW5lYXItZ3JhZGllbnQoNDVkZWcscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDI1JSx0cmFuc3BhcmVu
dCAyNSUsdHJhbnNwYXJlbnQgNTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA1MCUscmdiYSgyNTUs
MjU1LDI1NSwuMTUpIDc1JSx0cmFuc3BhcmVudCA3NSUsdHJhbnNwYXJlbnQpfS5wcm9ncmVzcy1i
YXItaW5mb3tiYWNrZ3JvdW5kLWNvbG9yOiM1YmMwZGV9LnByb2dyZXNzLXN0cmlwZWQgLnByb2dy
ZXNzLWJhci1pbmZve2JhY2tncm91bmQtaW1hZ2U6LXdlYmtpdC1saW5lYXItZ3JhZGllbnQoNDVk
ZWcscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDI1JSx0cmFuc3BhcmVudCAyNSUsdHJhbnNwYXJlbnQg
NTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDc1JSx0
cmFuc3BhcmVudCA3NSUsdHJhbnNwYXJlbnQpO2JhY2tncm91bmQtaW1hZ2U6LW8tbGluZWFyLWdy
YWRpZW50KDQ1ZGVnLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSAyNSUsdHJhbnNwYXJlbnQgMjUlLHRy
YW5zcGFyZW50IDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgNTAlLHJnYmEoMjU1LDI1NSwyNTUs
LjE1KSA3NSUsdHJhbnNwYXJlbnQgNzUlLHRyYW5zcGFyZW50KTtiYWNrZ3JvdW5kLWltYWdlOmxp
bmVhci1ncmFkaWVudCg0NWRlZyxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgMjUlLHRyYW5zcGFyZW50
IDI1JSx0cmFuc3BhcmVudCA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDUwJSxyZ2JhKDI1NSwy
NTUsMjU1LC4xNSkgNzUlLHRyYW5zcGFyZW50IDc1JSx0cmFuc3BhcmVudCl9LnByb2dyZXNzLWJh
ci13YXJuaW5ne2JhY2tncm91bmQtY29sb3I6I2YwYWQ0ZX0ucHJvZ3Jlc3Mtc3RyaXBlZCAucHJv
Z3Jlc3MtYmFyLXdhcm5pbmd7YmFja2dyb3VuZC1pbWFnZTotd2Via2l0LWxpbmVhci1ncmFkaWVu
dCg0NWRlZyxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgMjUlLHRyYW5zcGFyZW50IDI1JSx0cmFuc3Bh
cmVudCA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkg
NzUlLHRyYW5zcGFyZW50IDc1JSx0cmFuc3BhcmVudCk7YmFja2dyb3VuZC1pbWFnZTotby1saW5l
YXItZ3JhZGllbnQoNDVkZWcscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDI1JSx0cmFuc3BhcmVudCAy
NSUsdHJhbnNwYXJlbnQgNTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA1MCUscmdiYSgyNTUsMjU1
LDI1NSwuMTUpIDc1JSx0cmFuc3BhcmVudCA3NSUsdHJhbnNwYXJlbnQpO2JhY2tncm91bmQtaW1h
Z2U6bGluZWFyLWdyYWRpZW50KDQ1ZGVnLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSAyNSUsdHJhbnNw
YXJlbnQgMjUlLHRyYW5zcGFyZW50IDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgNTAlLHJnYmEo
MjU1LDI1NSwyNTUsLjE1KSA3NSUsdHJhbnNwYXJlbnQgNzUlLHRyYW5zcGFyZW50KX0ucHJvZ3Jl
c3MtYmFyLWRhbmdlcntiYWNrZ3JvdW5kLWNvbG9yOiNkOTUzNGZ9LnByb2dyZXNzLXN0cmlwZWQg
LnByb2dyZXNzLWJhci1kYW5nZXJ7YmFja2dyb3VuZC1pbWFnZTotd2Via2l0LWxpbmVhci1ncmFk
aWVudCg0NWRlZyxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgMjUlLHRyYW5zcGFyZW50IDI1JSx0cmFu
c3BhcmVudCA1MCUscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4x
NSkgNzUlLHRyYW5zcGFyZW50IDc1JSx0cmFuc3BhcmVudCk7YmFja2dyb3VuZC1pbWFnZTotby1s
aW5lYXItZ3JhZGllbnQoNDVkZWcscmdiYSgyNTUsMjU1LDI1NSwuMTUpIDI1JSx0cmFuc3BhcmVu
dCAyNSUsdHJhbnNwYXJlbnQgNTAlLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSA1MCUscmdiYSgyNTUs
MjU1LDI1NSwuMTUpIDc1JSx0cmFuc3BhcmVudCA3NSUsdHJhbnNwYXJlbnQpO2JhY2tncm91bmQt
aW1hZ2U6bGluZWFyLWdyYWRpZW50KDQ1ZGVnLHJnYmEoMjU1LDI1NSwyNTUsLjE1KSAyNSUsdHJh
bnNwYXJlbnQgMjUlLHRyYW5zcGFyZW50IDUwJSxyZ2JhKDI1NSwyNTUsMjU1LC4xNSkgNTAlLHJn
YmEoMjU1LDI1NSwyNTUsLjE1KSA3NSUsdHJhbnNwYXJlbnQgNzUlLHRyYW5zcGFyZW50KX0ubWVk
aWF7bWFyZ2luLXRvcDoxNXB4fS5tZWRpYTpmaXJzdC1jaGlsZHttYXJnaW4tdG9wOjB9Lm1lZGlh
LC5tZWRpYS1ib2R5e292ZXJmbG93OmhpZGRlbjt6b29tOjF9Lm1lZGlhLWJvZHl7d2lkdGg6MTAw
MDBweH0ubWVkaWEtb2JqZWN0e2Rpc3BsYXk6YmxvY2t9Lm1lZGlhLW9iamVjdC5pbWctdGh1bWJu
YWlse21heC13aWR0aDpub25lfS5tZWRpYS1yaWdodCwubWVkaWE+LnB1bGwtcmlnaHR7cGFkZGlu
Zy1sZWZ0OjEwcHh9Lm1lZGlhLWxlZnQsLm1lZGlhPi5wdWxsLWxlZnR7cGFkZGluZy1yaWdodDox
MHB4fS5tZWRpYS1ib2R5LC5tZWRpYS1sZWZ0LC5tZWRpYS1yaWdodHtkaXNwbGF5OnRhYmxlLWNl
bGw7dmVydGljYWwtYWxpZ246dG9wfS5tZWRpYS1taWRkbGV7dmVydGljYWwtYWxpZ246bWlkZGxl
fS5tZWRpYS1ib3R0b217dmVydGljYWwtYWxpZ246Ym90dG9tfS5tZWRpYS1oZWFkaW5ne21hcmdp
bi10b3A6MDttYXJnaW4tYm90dG9tOjVweH0ubWVkaWEtbGlzdHtwYWRkaW5nLWxlZnQ6MDtsaXN0
LXN0eWxlOm5vbmV9Lmxpc3QtZ3JvdXB7cGFkZGluZy1sZWZ0OjA7bWFyZ2luLWJvdHRvbToyMHB4
fS5saXN0LWdyb3VwLWl0ZW17cG9zaXRpb246cmVsYXRpdmU7ZGlzcGxheTpibG9jaztwYWRkaW5n
OjEwcHggMTVweDttYXJnaW4tYm90dG9tOi0xcHg7YmFja2dyb3VuZC1jb2xvcjojZmZmO2JvcmRl
cjoxcHggc29saWQgI2RkZH0ubGlzdC1ncm91cC1pdGVtOmZpcnN0LWNoaWxke2JvcmRlci10b3At
bGVmdC1yYWRpdXM6NHB4O2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjRweH0ubGlzdC1ncm91cC1p
dGVtOmxhc3QtY2hpbGR7bWFyZ2luLWJvdHRvbTowO2JvcmRlci1ib3R0b20tcmlnaHQtcmFkaXVz
OjRweDtib3JkZXItYm90dG9tLWxlZnQtcmFkaXVzOjRweH1hLmxpc3QtZ3JvdXAtaXRlbSxidXR0
b24ubGlzdC1ncm91cC1pdGVte2NvbG9yOiM1NTV9YS5saXN0LWdyb3VwLWl0ZW0gLmxpc3QtZ3Jv
dXAtaXRlbS1oZWFkaW5nLGJ1dHRvbi5saXN0LWdyb3VwLWl0ZW0gLmxpc3QtZ3JvdXAtaXRlbS1o
ZWFkaW5ne2NvbG9yOiMzMzN9YS5saXN0LWdyb3VwLWl0ZW06Zm9jdXMsYS5saXN0LWdyb3VwLWl0
ZW06aG92ZXIsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbTpmb2N1cyxidXR0b24ubGlzdC1ncm91cC1p
dGVtOmhvdmVye2NvbG9yOiM1NTU7dGV4dC1kZWNvcmF0aW9uOm5vbmU7YmFja2dyb3VuZC1jb2xv
cjojZjVmNWY1fWJ1dHRvbi5saXN0LWdyb3VwLWl0ZW17d2lkdGg6MTAwJTt0ZXh0LWFsaWduOmxl
ZnR9Lmxpc3QtZ3JvdXAtaXRlbS5kaXNhYmxlZCwubGlzdC1ncm91cC1pdGVtLmRpc2FibGVkOmZv
Y3VzLC5saXN0LWdyb3VwLWl0ZW0uZGlzYWJsZWQ6aG92ZXJ7Y29sb3I6Izc3NztjdXJzb3I6bm90
LWFsbG93ZWQ7YmFja2dyb3VuZC1jb2xvcjojZWVlfS5saXN0LWdyb3VwLWl0ZW0uZGlzYWJsZWQg
Lmxpc3QtZ3JvdXAtaXRlbS1oZWFkaW5nLC5saXN0LWdyb3VwLWl0ZW0uZGlzYWJsZWQ6Zm9jdXMg
Lmxpc3QtZ3JvdXAtaXRlbS1oZWFkaW5nLC5saXN0LWdyb3VwLWl0ZW0uZGlzYWJsZWQ6aG92ZXIg
Lmxpc3QtZ3JvdXAtaXRlbS1oZWFkaW5ne2NvbG9yOmluaGVyaXR9Lmxpc3QtZ3JvdXAtaXRlbS5k
aXNhYmxlZCAubGlzdC1ncm91cC1pdGVtLXRleHQsLmxpc3QtZ3JvdXAtaXRlbS5kaXNhYmxlZDpm
b2N1cyAubGlzdC1ncm91cC1pdGVtLXRleHQsLmxpc3QtZ3JvdXAtaXRlbS5kaXNhYmxlZDpob3Zl
ciAubGlzdC1ncm91cC1pdGVtLXRleHR7Y29sb3I6Izc3N30ubGlzdC1ncm91cC1pdGVtLmFjdGl2
ZSwubGlzdC1ncm91cC1pdGVtLmFjdGl2ZTpmb2N1cywubGlzdC1ncm91cC1pdGVtLmFjdGl2ZTpo
b3Zlcnt6LWluZGV4OjI7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzMzdhYjc7Ym9yZGVy
LWNvbG9yOiMzMzdhYjd9Lmxpc3QtZ3JvdXAtaXRlbS5hY3RpdmUgLmxpc3QtZ3JvdXAtaXRlbS1o
ZWFkaW5nLC5saXN0LWdyb3VwLWl0ZW0uYWN0aXZlIC5saXN0LWdyb3VwLWl0ZW0taGVhZGluZz4u
c21hbGwsLmxpc3QtZ3JvdXAtaXRlbS5hY3RpdmUgLmxpc3QtZ3JvdXAtaXRlbS1oZWFkaW5nPnNt
YWxsLC5saXN0LWdyb3VwLWl0ZW0uYWN0aXZlOmZvY3VzIC5saXN0LWdyb3VwLWl0ZW0taGVhZGlu
ZywubGlzdC1ncm91cC1pdGVtLmFjdGl2ZTpmb2N1cyAubGlzdC1ncm91cC1pdGVtLWhlYWRpbmc+
LnNtYWxsLC5saXN0LWdyb3VwLWl0ZW0uYWN0aXZlOmZvY3VzIC5saXN0LWdyb3VwLWl0ZW0taGVh
ZGluZz5zbWFsbCwubGlzdC1ncm91cC1pdGVtLmFjdGl2ZTpob3ZlciAubGlzdC1ncm91cC1pdGVt
LWhlYWRpbmcsLmxpc3QtZ3JvdXAtaXRlbS5hY3RpdmU6aG92ZXIgLmxpc3QtZ3JvdXAtaXRlbS1o
ZWFkaW5nPi5zbWFsbCwubGlzdC1ncm91cC1pdGVtLmFjdGl2ZTpob3ZlciAubGlzdC1ncm91cC1p
dGVtLWhlYWRpbmc+c21hbGx7Y29sb3I6aW5oZXJpdH0ubGlzdC1ncm91cC1pdGVtLmFjdGl2ZSAu
bGlzdC1ncm91cC1pdGVtLXRleHQsLmxpc3QtZ3JvdXAtaXRlbS5hY3RpdmU6Zm9jdXMgLmxpc3Qt
Z3JvdXAtaXRlbS10ZXh0LC5saXN0LWdyb3VwLWl0ZW0uYWN0aXZlOmhvdmVyIC5saXN0LWdyb3Vw
LWl0ZW0tdGV4dHtjb2xvcjojYzdkZGVmfS5saXN0LWdyb3VwLWl0ZW0tc3VjY2Vzc3tjb2xvcjoj
M2M3NjNkO2JhY2tncm91bmQtY29sb3I6I2RmZjBkOH1hLmxpc3QtZ3JvdXAtaXRlbS1zdWNjZXNz
LGJ1dHRvbi5saXN0LWdyb3VwLWl0ZW0tc3VjY2Vzc3tjb2xvcjojM2M3NjNkfWEubGlzdC1ncm91
cC1pdGVtLXN1Y2Nlc3MgLmxpc3QtZ3JvdXAtaXRlbS1oZWFkaW5nLGJ1dHRvbi5saXN0LWdyb3Vw
LWl0ZW0tc3VjY2VzcyAubGlzdC1ncm91cC1pdGVtLWhlYWRpbmd7Y29sb3I6aW5oZXJpdH1hLmxp
c3QtZ3JvdXAtaXRlbS1zdWNjZXNzOmZvY3VzLGEubGlzdC1ncm91cC1pdGVtLXN1Y2Nlc3M6aG92
ZXIsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS1zdWNjZXNzOmZvY3VzLGJ1dHRvbi5saXN0LWdyb3Vw
LWl0ZW0tc3VjY2Vzczpob3Zlcntjb2xvcjojM2M3NjNkO2JhY2tncm91bmQtY29sb3I6I2QwZTlj
Nn1hLmxpc3QtZ3JvdXAtaXRlbS1zdWNjZXNzLmFjdGl2ZSxhLmxpc3QtZ3JvdXAtaXRlbS1zdWNj
ZXNzLmFjdGl2ZTpmb2N1cyxhLmxpc3QtZ3JvdXAtaXRlbS1zdWNjZXNzLmFjdGl2ZTpob3Zlcixi
dXR0b24ubGlzdC1ncm91cC1pdGVtLXN1Y2Nlc3MuYWN0aXZlLGJ1dHRvbi5saXN0LWdyb3VwLWl0
ZW0tc3VjY2Vzcy5hY3RpdmU6Zm9jdXMsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS1zdWNjZXNzLmFj
dGl2ZTpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6IzNjNzYzZDtib3JkZXItY29s
b3I6IzNjNzYzZH0ubGlzdC1ncm91cC1pdGVtLWluZm97Y29sb3I6IzMxNzA4ZjtiYWNrZ3JvdW5k
LWNvbG9yOiNkOWVkZjd9YS5saXN0LWdyb3VwLWl0ZW0taW5mbyxidXR0b24ubGlzdC1ncm91cC1p
dGVtLWluZm97Y29sb3I6IzMxNzA4Zn1hLmxpc3QtZ3JvdXAtaXRlbS1pbmZvIC5saXN0LWdyb3Vw
LWl0ZW0taGVhZGluZyxidXR0b24ubGlzdC1ncm91cC1pdGVtLWluZm8gLmxpc3QtZ3JvdXAtaXRl
bS1oZWFkaW5ne2NvbG9yOmluaGVyaXR9YS5saXN0LWdyb3VwLWl0ZW0taW5mbzpmb2N1cyxhLmxp
c3QtZ3JvdXAtaXRlbS1pbmZvOmhvdmVyLGJ1dHRvbi5saXN0LWdyb3VwLWl0ZW0taW5mbzpmb2N1
cyxidXR0b24ubGlzdC1ncm91cC1pdGVtLWluZm86aG92ZXJ7Y29sb3I6IzMxNzA4ZjtiYWNrZ3Jv
dW5kLWNvbG9yOiNjNGUzZjN9YS5saXN0LWdyb3VwLWl0ZW0taW5mby5hY3RpdmUsYS5saXN0LWdy
b3VwLWl0ZW0taW5mby5hY3RpdmU6Zm9jdXMsYS5saXN0LWdyb3VwLWl0ZW0taW5mby5hY3RpdmU6
aG92ZXIsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS1pbmZvLmFjdGl2ZSxidXR0b24ubGlzdC1ncm91
cC1pdGVtLWluZm8uYWN0aXZlOmZvY3VzLGJ1dHRvbi5saXN0LWdyb3VwLWl0ZW0taW5mby5hY3Rp
dmU6aG92ZXJ7Y29sb3I6I2ZmZjtiYWNrZ3JvdW5kLWNvbG9yOiMzMTcwOGY7Ym9yZGVyLWNvbG9y
OiMzMTcwOGZ9Lmxpc3QtZ3JvdXAtaXRlbS13YXJuaW5ne2NvbG9yOiM4YTZkM2I7YmFja2dyb3Vu
ZC1jb2xvcjojZmNmOGUzfWEubGlzdC1ncm91cC1pdGVtLXdhcm5pbmcsYnV0dG9uLmxpc3QtZ3Jv
dXAtaXRlbS13YXJuaW5ne2NvbG9yOiM4YTZkM2J9YS5saXN0LWdyb3VwLWl0ZW0td2FybmluZyAu
bGlzdC1ncm91cC1pdGVtLWhlYWRpbmcsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS13YXJuaW5nIC5s
aXN0LWdyb3VwLWl0ZW0taGVhZGluZ3tjb2xvcjppbmhlcml0fWEubGlzdC1ncm91cC1pdGVtLXdh
cm5pbmc6Zm9jdXMsYS5saXN0LWdyb3VwLWl0ZW0td2FybmluZzpob3ZlcixidXR0b24ubGlzdC1n
cm91cC1pdGVtLXdhcm5pbmc6Zm9jdXMsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS13YXJuaW5nOmhv
dmVye2NvbG9yOiM4YTZkM2I7YmFja2dyb3VuZC1jb2xvcjojZmFmMmNjfWEubGlzdC1ncm91cC1p
dGVtLXdhcm5pbmcuYWN0aXZlLGEubGlzdC1ncm91cC1pdGVtLXdhcm5pbmcuYWN0aXZlOmZvY3Vz
LGEubGlzdC1ncm91cC1pdGVtLXdhcm5pbmcuYWN0aXZlOmhvdmVyLGJ1dHRvbi5saXN0LWdyb3Vw
LWl0ZW0td2FybmluZy5hY3RpdmUsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS13YXJuaW5nLmFjdGl2
ZTpmb2N1cyxidXR0b24ubGlzdC1ncm91cC1pdGVtLXdhcm5pbmcuYWN0aXZlOmhvdmVye2NvbG9y
OiNmZmY7YmFja2dyb3VuZC1jb2xvcjojOGE2ZDNiO2JvcmRlci1jb2xvcjojOGE2ZDNifS5saXN0
LWdyb3VwLWl0ZW0tZGFuZ2Vye2NvbG9yOiNhOTQ0NDI7YmFja2dyb3VuZC1jb2xvcjojZjJkZWRl
fWEubGlzdC1ncm91cC1pdGVtLWRhbmdlcixidXR0b24ubGlzdC1ncm91cC1pdGVtLWRhbmdlcntj
b2xvcjojYTk0NDQyfWEubGlzdC1ncm91cC1pdGVtLWRhbmdlciAubGlzdC1ncm91cC1pdGVtLWhl
YWRpbmcsYnV0dG9uLmxpc3QtZ3JvdXAtaXRlbS1kYW5nZXIgLmxpc3QtZ3JvdXAtaXRlbS1oZWFk
aW5ne2NvbG9yOmluaGVyaXR9YS5saXN0LWdyb3VwLWl0ZW0tZGFuZ2VyOmZvY3VzLGEubGlzdC1n
cm91cC1pdGVtLWRhbmdlcjpob3ZlcixidXR0b24ubGlzdC1ncm91cC1pdGVtLWRhbmdlcjpmb2N1
cyxidXR0b24ubGlzdC1ncm91cC1pdGVtLWRhbmdlcjpob3Zlcntjb2xvcjojYTk0NDQyO2JhY2tn
cm91bmQtY29sb3I6I2ViY2NjY31hLmxpc3QtZ3JvdXAtaXRlbS1kYW5nZXIuYWN0aXZlLGEubGlz
dC1ncm91cC1pdGVtLWRhbmdlci5hY3RpdmU6Zm9jdXMsYS5saXN0LWdyb3VwLWl0ZW0tZGFuZ2Vy
LmFjdGl2ZTpob3ZlcixidXR0b24ubGlzdC1ncm91cC1pdGVtLWRhbmdlci5hY3RpdmUsYnV0dG9u
Lmxpc3QtZ3JvdXAtaXRlbS1kYW5nZXIuYWN0aXZlOmZvY3VzLGJ1dHRvbi5saXN0LWdyb3VwLWl0
ZW0tZGFuZ2VyLmFjdGl2ZTpob3Zlcntjb2xvcjojZmZmO2JhY2tncm91bmQtY29sb3I6I2E5NDQ0
Mjtib3JkZXItY29sb3I6I2E5NDQ0Mn0ubGlzdC1ncm91cC1pdGVtLWhlYWRpbmd7bWFyZ2luLXRv
cDowO21hcmdpbi1ib3R0b206NXB4fS5saXN0LWdyb3VwLWl0ZW0tdGV4dHttYXJnaW4tYm90dG9t
OjA7bGluZS1oZWlnaHQ6MS4zfS5wYW5lbHttYXJnaW4tYm90dG9tOjIwcHg7YmFja2dyb3VuZC1j
b2xvcjojZmZmO2JvcmRlcjoxcHggc29saWQgdHJhbnNwYXJlbnQ7Ym9yZGVyLXJhZGl1czo0cHg7
LXdlYmtpdC1ib3gtc2hhZG93OjAgMXB4IDFweCByZ2JhKDAsMCwwLC4wNSk7Ym94LXNoYWRvdzow
IDFweCAxcHggcmdiYSgwLDAsMCwuMDUpfS5wYW5lbC1ib2R5e3BhZGRpbmc6MTVweH0ucGFuZWwt
aGVhZGluZ3twYWRkaW5nOjEwcHggMTVweDtib3JkZXItYm90dG9tOjFweCBzb2xpZCB0cmFuc3Bh
cmVudDtib3JkZXItdG9wLWxlZnQtcmFkaXVzOjNweDtib3JkZXItdG9wLXJpZ2h0LXJhZGl1czoz
cHh9LnBhbmVsLWhlYWRpbmc+LmRyb3Bkb3duIC5kcm9wZG93bi10b2dnbGV7Y29sb3I6aW5oZXJp
dH0ucGFuZWwtdGl0bGV7bWFyZ2luLXRvcDowO21hcmdpbi1ib3R0b206MDtmb250LXNpemU6MTZw
eDtjb2xvcjppbmhlcml0fS5wYW5lbC10aXRsZT4uc21hbGwsLnBhbmVsLXRpdGxlPi5zbWFsbD5h
LC5wYW5lbC10aXRsZT5hLC5wYW5lbC10aXRsZT5zbWFsbCwucGFuZWwtdGl0bGU+c21hbGw+YXtj
b2xvcjppbmhlcml0fS5wYW5lbC1mb290ZXJ7cGFkZGluZzoxMHB4IDE1cHg7YmFja2dyb3VuZC1j
b2xvcjojZjVmNWY1O2JvcmRlci10b3A6MXB4IHNvbGlkICNkZGQ7Ym9yZGVyLWJvdHRvbS1yaWdo
dC1yYWRpdXM6M3B4O2JvcmRlci1ib3R0b20tbGVmdC1yYWRpdXM6M3B4fS5wYW5lbD4ubGlzdC1n
cm91cCwucGFuZWw+LnBhbmVsLWNvbGxhcHNlPi5saXN0LWdyb3Vwe21hcmdpbi1ib3R0b206MH0u
cGFuZWw+Lmxpc3QtZ3JvdXAgLmxpc3QtZ3JvdXAtaXRlbSwucGFuZWw+LnBhbmVsLWNvbGxhcHNl
Pi5saXN0LWdyb3VwIC5saXN0LWdyb3VwLWl0ZW17Ym9yZGVyLXdpZHRoOjFweCAwO2JvcmRlci1y
YWRpdXM6MH0ucGFuZWw+Lmxpc3QtZ3JvdXA6Zmlyc3QtY2hpbGQgLmxpc3QtZ3JvdXAtaXRlbTpm
aXJzdC1jaGlsZCwucGFuZWw+LnBhbmVsLWNvbGxhcHNlPi5saXN0LWdyb3VwOmZpcnN0LWNoaWxk
IC5saXN0LWdyb3VwLWl0ZW06Zmlyc3QtY2hpbGR7Ym9yZGVyLXRvcDowO2JvcmRlci10b3AtbGVm
dC1yYWRpdXM6M3B4O2JvcmRlci10b3AtcmlnaHQtcmFkaXVzOjNweH0ucGFuZWw+Lmxpc3QtZ3Jv
dXA6bGFzdC1jaGlsZCAubGlzdC1ncm91cC1pdGVtOmxhc3QtY2hpbGQsLnBhbmVsPi5wYW5lbC1j
b2xsYXBzZT4ubGlzdC1ncm91cDpsYXN0LWNoaWxkIC5saXN0LWdyb3VwLWl0ZW06bGFzdC1jaGls
ZHtib3JkZXItYm90dG9tOjA7Ym9yZGVyLWJvdHRvbS1yaWdodC1yYWRpdXM6M3B4O2JvcmRlci1i
b3R0b20tbGVmdC1yYWRpdXM6M3B4fS5wYW5lbD4ucGFuZWwtaGVhZGluZysucGFuZWwtY29sbGFw
c2U+Lmxpc3QtZ3JvdXAgLmxpc3QtZ3JvdXAtaXRlbTpmaXJzdC1jaGlsZHtib3JkZXItdG9wLWxl
ZnQtcmFkaXVzOjA7Ym9yZGVyLXRvcC1yaWdodC1yYWRpdXM6MH0ucGFuZWwtaGVhZGluZysubGlz
dC1ncm91cCAubGlzdC1ncm91cC1pdGVtOmZpcnN0LWNoaWxke2JvcmRlci10b3Atd2lkdGg6MH0u
bGlzdC1ncm91cCsucGFuZWwtZm9vdGVye2JvcmRlci10b3Atd2lkdGg6MH0ucGFuZWw+LnBhbmVs
LWNvbGxhcHNlPi50YWJsZSwucGFuZWw+LnRhYmxlLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZT4u
dGFibGV7bWFyZ2luLWJvdHRvbTowfS5wYW5lbD4ucGFuZWwtY29sbGFwc2U+LnRhYmxlIGNhcHRp
b24sLnBhbmVsPi50YWJsZSBjYXB0aW9uLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZT4udGFibGUg
Y2FwdGlvbntwYWRkaW5nLXJpZ2h0OjE1cHg7cGFkZGluZy1sZWZ0OjE1cHh9LnBhbmVsPi50YWJs
ZS1yZXNwb25zaXZlOmZpcnN0LWNoaWxkPi50YWJsZTpmaXJzdC1jaGlsZCwucGFuZWw+LnRhYmxl
OmZpcnN0LWNoaWxke2JvcmRlci10b3AtbGVmdC1yYWRpdXM6M3B4O2JvcmRlci10b3AtcmlnaHQt
cmFkaXVzOjNweH0ucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU6Zmlyc3QtY2hpbGQ+LnRhYmxlOmZp
cnN0LWNoaWxkPnRib2R5OmZpcnN0LWNoaWxkPnRyOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGUt
cmVzcG9uc2l2ZTpmaXJzdC1jaGlsZD4udGFibGU6Zmlyc3QtY2hpbGQ+dGhlYWQ6Zmlyc3QtY2hp
bGQ+dHI6Zmlyc3QtY2hpbGQsLnBhbmVsPi50YWJsZTpmaXJzdC1jaGlsZD50Ym9keTpmaXJzdC1j
aGlsZD50cjpmaXJzdC1jaGlsZCwucGFuZWw+LnRhYmxlOmZpcnN0LWNoaWxkPnRoZWFkOmZpcnN0
LWNoaWxkPnRyOmZpcnN0LWNoaWxke2JvcmRlci10b3AtbGVmdC1yYWRpdXM6M3B4O2JvcmRlci10
b3AtcmlnaHQtcmFkaXVzOjNweH0ucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU6Zmlyc3QtY2hpbGQ+
LnRhYmxlOmZpcnN0LWNoaWxkPnRib2R5OmZpcnN0LWNoaWxkPnRyOmZpcnN0LWNoaWxkIHRkOmZp
cnN0LWNoaWxkLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZTpmaXJzdC1jaGlsZD4udGFibGU6Zmly
c3QtY2hpbGQ+dGJvZHk6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGg6Zmlyc3QtY2hpbGQs
LnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmZpcnN0LWNoaWxkPi50YWJsZTpmaXJzdC1jaGlsZD50
aGVhZDpmaXJzdC1jaGlsZD50cjpmaXJzdC1jaGlsZCB0ZDpmaXJzdC1jaGlsZCwucGFuZWw+LnRh
YmxlLXJlc3BvbnNpdmU6Zmlyc3QtY2hpbGQ+LnRhYmxlOmZpcnN0LWNoaWxkPnRoZWFkOmZpcnN0
LWNoaWxkPnRyOmZpcnN0LWNoaWxkIHRoOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGU6Zmlyc3Qt
Y2hpbGQ+dGJvZHk6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGQ6Zmlyc3QtY2hpbGQsLnBh
bmVsPi50YWJsZTpmaXJzdC1jaGlsZD50Ym9keTpmaXJzdC1jaGlsZD50cjpmaXJzdC1jaGlsZCB0
aDpmaXJzdC1jaGlsZCwucGFuZWw+LnRhYmxlOmZpcnN0LWNoaWxkPnRoZWFkOmZpcnN0LWNoaWxk
PnRyOmZpcnN0LWNoaWxkIHRkOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGU6Zmlyc3QtY2hpbGQ+
dGhlYWQ6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGg6Zmlyc3QtY2hpbGR7Ym9yZGVyLXRv
cC1sZWZ0LXJhZGl1czozcHh9LnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmZpcnN0LWNoaWxkPi50
YWJsZTpmaXJzdC1jaGlsZD50Ym9keTpmaXJzdC1jaGlsZD50cjpmaXJzdC1jaGlsZCB0ZDpsYXN0
LWNoaWxkLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZTpmaXJzdC1jaGlsZD4udGFibGU6Zmlyc3Qt
Y2hpbGQ+dGJvZHk6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGg6bGFzdC1jaGlsZCwucGFu
ZWw+LnRhYmxlLXJlc3BvbnNpdmU6Zmlyc3QtY2hpbGQ+LnRhYmxlOmZpcnN0LWNoaWxkPnRoZWFk
OmZpcnN0LWNoaWxkPnRyOmZpcnN0LWNoaWxkIHRkOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1y
ZXNwb25zaXZlOmZpcnN0LWNoaWxkPi50YWJsZTpmaXJzdC1jaGlsZD50aGVhZDpmaXJzdC1jaGls
ZD50cjpmaXJzdC1jaGlsZCB0aDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGU6Zmlyc3QtY2hpbGQ+
dGJvZHk6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGQ6bGFzdC1jaGlsZCwucGFuZWw+LnRh
YmxlOmZpcnN0LWNoaWxkPnRib2R5OmZpcnN0LWNoaWxkPnRyOmZpcnN0LWNoaWxkIHRoOmxhc3Qt
Y2hpbGQsLnBhbmVsPi50YWJsZTpmaXJzdC1jaGlsZD50aGVhZDpmaXJzdC1jaGlsZD50cjpmaXJz
dC1jaGlsZCB0ZDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGU6Zmlyc3QtY2hpbGQ+dGhlYWQ6Zmly
c3QtY2hpbGQ+dHI6Zmlyc3QtY2hpbGQgdGg6bGFzdC1jaGlsZHtib3JkZXItdG9wLXJpZ2h0LXJh
ZGl1czozcHh9LnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3Qt
Y2hpbGQsLnBhbmVsPi50YWJsZTpsYXN0LWNoaWxke2JvcmRlci1ib3R0b20tcmlnaHQtcmFkaXVz
OjNweDtib3JkZXItYm90dG9tLWxlZnQtcmFkaXVzOjNweH0ucGFuZWw+LnRhYmxlLXJlc3BvbnNp
dmU6bGFzdC1jaGlsZD4udGFibGU6bGFzdC1jaGlsZD50Ym9keTpsYXN0LWNoaWxkPnRyOmxhc3Qt
Y2hpbGQsLnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3QtY2hp
bGQ+dGZvb3Q6bGFzdC1jaGlsZD50cjpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGU6bGFzdC1jaGls
ZD50Ym9keTpsYXN0LWNoaWxkPnRyOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZTpsYXN0LWNoaWxk
PnRmb290Omxhc3QtY2hpbGQ+dHI6bGFzdC1jaGlsZHtib3JkZXItYm90dG9tLXJpZ2h0LXJhZGl1
czozcHg7Ym9yZGVyLWJvdHRvbS1sZWZ0LXJhZGl1czozcHh9LnBhbmVsPi50YWJsZS1yZXNwb25z
aXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3QtY2hpbGQ+dGJvZHk6bGFzdC1jaGlsZD50cjpsYXN0
LWNoaWxkIHRkOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZTpsYXN0LWNoaWxk
Pi50YWJsZTpsYXN0LWNoaWxkPnRib2R5Omxhc3QtY2hpbGQ+dHI6bGFzdC1jaGlsZCB0aDpmaXJz
dC1jaGlsZCwucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU6bGFzdC1jaGlsZD4udGFibGU6bGFzdC1j
aGlsZD50Zm9vdDpsYXN0LWNoaWxkPnRyOmxhc3QtY2hpbGQgdGQ6Zmlyc3QtY2hpbGQsLnBhbmVs
Pi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3QtY2hpbGQ+dGZvb3Q6bGFz
dC1jaGlsZD50cjpsYXN0LWNoaWxkIHRoOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGU6bGFzdC1j
aGlsZD50Ym9keTpsYXN0LWNoaWxkPnRyOmxhc3QtY2hpbGQgdGQ6Zmlyc3QtY2hpbGQsLnBhbmVs
Pi50YWJsZTpsYXN0LWNoaWxkPnRib2R5Omxhc3QtY2hpbGQ+dHI6bGFzdC1jaGlsZCB0aDpmaXJz
dC1jaGlsZCwucGFuZWw+LnRhYmxlOmxhc3QtY2hpbGQ+dGZvb3Q6bGFzdC1jaGlsZD50cjpsYXN0
LWNoaWxkIHRkOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGU6bGFzdC1jaGlsZD50Zm9vdDpsYXN0
LWNoaWxkPnRyOmxhc3QtY2hpbGQgdGg6Zmlyc3QtY2hpbGR7Ym9yZGVyLWJvdHRvbS1sZWZ0LXJh
ZGl1czozcHh9LnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3Qt
Y2hpbGQ+dGJvZHk6bGFzdC1jaGlsZD50cjpsYXN0LWNoaWxkIHRkOmxhc3QtY2hpbGQsLnBhbmVs
Pi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3QtY2hpbGQ+dGJvZHk6bGFz
dC1jaGlsZD50cjpsYXN0LWNoaWxkIHRoOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1yZXNwb25z
aXZlOmxhc3QtY2hpbGQ+LnRhYmxlOmxhc3QtY2hpbGQ+dGZvb3Q6bGFzdC1jaGlsZD50cjpsYXN0
LWNoaWxkIHRkOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1yZXNwb25zaXZlOmxhc3QtY2hpbGQ+
LnRhYmxlOmxhc3QtY2hpbGQ+dGZvb3Q6bGFzdC1jaGlsZD50cjpsYXN0LWNoaWxkIHRoOmxhc3Qt
Y2hpbGQsLnBhbmVsPi50YWJsZTpsYXN0LWNoaWxkPnRib2R5Omxhc3QtY2hpbGQ+dHI6bGFzdC1j
aGlsZCB0ZDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGU6bGFzdC1jaGlsZD50Ym9keTpsYXN0LWNo
aWxkPnRyOmxhc3QtY2hpbGQgdGg6bGFzdC1jaGlsZCwucGFuZWw+LnRhYmxlOmxhc3QtY2hpbGQ+
dGZvb3Q6bGFzdC1jaGlsZD50cjpsYXN0LWNoaWxkIHRkOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJs
ZTpsYXN0LWNoaWxkPnRmb290Omxhc3QtY2hpbGQ+dHI6bGFzdC1jaGlsZCB0aDpsYXN0LWNoaWxk
e2JvcmRlci1ib3R0b20tcmlnaHQtcmFkaXVzOjNweH0ucGFuZWw+LnBhbmVsLWJvZHkrLnRhYmxl
LC5wYW5lbD4ucGFuZWwtYm9keSsudGFibGUtcmVzcG9uc2l2ZSwucGFuZWw+LnRhYmxlKy5wYW5l
bC1ib2R5LC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZSsucGFuZWwtYm9keXtib3JkZXItdG9wOjFw
eCBzb2xpZCAjZGRkfS5wYW5lbD4udGFibGU+dGJvZHk6Zmlyc3QtY2hpbGQ+dHI6Zmlyc3QtY2hp
bGQgdGQsLnBhbmVsPi50YWJsZT50Ym9keTpmaXJzdC1jaGlsZD50cjpmaXJzdC1jaGlsZCB0aHti
b3JkZXItdG9wOjB9LnBhbmVsPi50YWJsZS1ib3JkZXJlZCwucGFuZWw+LnRhYmxlLXJlc3BvbnNp
dmU+LnRhYmxlLWJvcmRlcmVke2JvcmRlcjowfS5wYW5lbD4udGFibGUtYm9yZGVyZWQ+dGJvZHk+
dHI+dGQ6Zmlyc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1ib3JkZXJlZD50Ym9keT50cj50aDpmaXJz
dC1jaGlsZCwucGFuZWw+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyPnRkOmZpcnN0LWNoaWxkLC5w
YW5lbD4udGFibGUtYm9yZGVyZWQ+dGZvb3Q+dHI+dGg6Zmlyc3QtY2hpbGQsLnBhbmVsPi50YWJs
ZS1ib3JkZXJlZD50aGVhZD50cj50ZDpmaXJzdC1jaGlsZCwucGFuZWw+LnRhYmxlLWJvcmRlcmVk
PnRoZWFkPnRyPnRoOmZpcnN0LWNoaWxkLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZT4udGFibGUt
Ym9yZGVyZWQ+dGJvZHk+dHI+dGQ6Zmlyc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1yZXNwb25zaXZl
Pi50YWJsZS1ib3JkZXJlZD50Ym9keT50cj50aDpmaXJzdC1jaGlsZCwucGFuZWw+LnRhYmxlLXJl
c3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyPnRkOmZpcnN0LWNoaWxkLC5wYW5lbD4u
dGFibGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGZvb3Q+dHI+dGg6Zmlyc3QtY2hpbGQs
LnBhbmVsPi50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50aGVhZD50cj50ZDpmaXJz
dC1jaGlsZCwucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRy
PnRoOmZpcnN0LWNoaWxke2JvcmRlci1sZWZ0OjB9LnBhbmVsPi50YWJsZS1ib3JkZXJlZD50Ym9k
eT50cj50ZDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGUtYm9yZGVyZWQ+dGJvZHk+dHI+dGg6bGFz
dC1jaGlsZCwucGFuZWw+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyPnRkOmxhc3QtY2hpbGQsLnBh
bmVsPi50YWJsZS1ib3JkZXJlZD50Zm9vdD50cj50aDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGUt
Ym9yZGVyZWQ+dGhlYWQ+dHI+dGQ6bGFzdC1jaGlsZCwucGFuZWw+LnRhYmxlLWJvcmRlcmVkPnRo
ZWFkPnRyPnRoOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3Jk
ZXJlZD50Ym9keT50cj50ZDpsYXN0LWNoaWxkLC5wYW5lbD4udGFibGUtcmVzcG9uc2l2ZT4udGFi
bGUtYm9yZGVyZWQ+dGJvZHk+dHI+dGg6bGFzdC1jaGlsZCwucGFuZWw+LnRhYmxlLXJlc3BvbnNp
dmU+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyPnRkOmxhc3QtY2hpbGQsLnBhbmVsPi50YWJsZS1y
ZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50Zm9vdD50cj50aDpsYXN0LWNoaWxkLC5wYW5lbD4u
dGFibGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGhlYWQ+dHI+dGQ6bGFzdC1jaGlsZCwu
cGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRyPnRoOmxhc3Qt
Y2hpbGR7Ym9yZGVyLXJpZ2h0OjB9LnBhbmVsPi50YWJsZS1ib3JkZXJlZD50Ym9keT50cjpmaXJz
dC1jaGlsZD50ZCwucGFuZWw+LnRhYmxlLWJvcmRlcmVkPnRib2R5PnRyOmZpcnN0LWNoaWxkPnRo
LC5wYW5lbD4udGFibGUtYm9yZGVyZWQ+dGhlYWQ+dHI6Zmlyc3QtY2hpbGQ+dGQsLnBhbmVsPi50
YWJsZS1ib3JkZXJlZD50aGVhZD50cjpmaXJzdC1jaGlsZD50aCwucGFuZWw+LnRhYmxlLXJlc3Bv
bnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRib2R5PnRyOmZpcnN0LWNoaWxkPnRkLC5wYW5lbD4udGFi
bGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGJvZHk+dHI6Zmlyc3QtY2hpbGQ+dGgsLnBh
bmVsPi50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50aGVhZD50cjpmaXJzdC1jaGls
ZD50ZCwucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRoZWFkPnRyOmZp
cnN0LWNoaWxkPnRoe2JvcmRlci1ib3R0b206MH0ucGFuZWw+LnRhYmxlLWJvcmRlcmVkPnRib2R5
PnRyOmxhc3QtY2hpbGQ+dGQsLnBhbmVsPi50YWJsZS1ib3JkZXJlZD50Ym9keT50cjpsYXN0LWNo
aWxkPnRoLC5wYW5lbD4udGFibGUtYm9yZGVyZWQ+dGZvb3Q+dHI6bGFzdC1jaGlsZD50ZCwucGFu
ZWw+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyOmxhc3QtY2hpbGQ+dGgsLnBhbmVsPi50YWJsZS1y
ZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50Ym9keT50cjpsYXN0LWNoaWxkPnRkLC5wYW5lbD4u
dGFibGUtcmVzcG9uc2l2ZT4udGFibGUtYm9yZGVyZWQ+dGJvZHk+dHI6bGFzdC1jaGlsZD50aCwu
cGFuZWw+LnRhYmxlLXJlc3BvbnNpdmU+LnRhYmxlLWJvcmRlcmVkPnRmb290PnRyOmxhc3QtY2hp
bGQ+dGQsLnBhbmVsPi50YWJsZS1yZXNwb25zaXZlPi50YWJsZS1ib3JkZXJlZD50Zm9vdD50cjps
YXN0LWNoaWxkPnRoe2JvcmRlci1ib3R0b206MH0ucGFuZWw+LnRhYmxlLXJlc3BvbnNpdmV7bWFy
Z2luLWJvdHRvbTowO2JvcmRlcjowfS5wYW5lbC1ncm91cHttYXJnaW4tYm90dG9tOjIwcHh9LnBh
bmVsLWdyb3VwIC5wYW5lbHttYXJnaW4tYm90dG9tOjA7Ym9yZGVyLXJhZGl1czo0cHh9LnBhbmVs
LWdyb3VwIC5wYW5lbCsucGFuZWx7bWFyZ2luLXRvcDo1cHh9LnBhbmVsLWdyb3VwIC5wYW5lbC1o
ZWFkaW5ne2JvcmRlci1ib3R0b206MH0ucGFuZWwtZ3JvdXAgLnBhbmVsLWhlYWRpbmcrLnBhbmVs
LWNvbGxhcHNlPi5saXN0LWdyb3VwLC5wYW5lbC1ncm91cCAucGFuZWwtaGVhZGluZysucGFuZWwt
Y29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9yZGVyLXRvcDoxcHggc29saWQgI2RkZH0ucGFuZWwtZ3Jv
dXAgLnBhbmVsLWZvb3Rlcntib3JkZXItdG9wOjB9LnBhbmVsLWdyb3VwIC5wYW5lbC1mb290ZXIr
LnBhbmVsLWNvbGxhcHNlIC5wYW5lbC1ib2R5e2JvcmRlci1ib3R0b206MXB4IHNvbGlkICNkZGR9
LnBhbmVsLWRlZmF1bHR7Ym9yZGVyLWNvbG9yOiNkZGR9LnBhbmVsLWRlZmF1bHQ+LnBhbmVsLWhl
YWRpbmd7Y29sb3I6IzMzMztiYWNrZ3JvdW5kLWNvbG9yOiNmNWY1ZjU7Ym9yZGVyLWNvbG9yOiNk
ZGR9LnBhbmVsLWRlZmF1bHQ+LnBhbmVsLWhlYWRpbmcrLnBhbmVsLWNvbGxhcHNlPi5wYW5lbC1i
b2R5e2JvcmRlci10b3AtY29sb3I6I2RkZH0ucGFuZWwtZGVmYXVsdD4ucGFuZWwtaGVhZGluZyAu
YmFkZ2V7Y29sb3I6I2Y1ZjVmNTtiYWNrZ3JvdW5kLWNvbG9yOiMzMzN9LnBhbmVsLWRlZmF1bHQ+
LnBhbmVsLWZvb3RlcisucGFuZWwtY29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9yZGVyLWJvdHRvbS1j
b2xvcjojZGRkfS5wYW5lbC1wcmltYXJ5e2JvcmRlci1jb2xvcjojMzM3YWI3fS5wYW5lbC1wcmlt
YXJ5Pi5wYW5lbC1oZWFkaW5ne2NvbG9yOiNmZmY7YmFja2dyb3VuZC1jb2xvcjojMzM3YWI3O2Jv
cmRlci1jb2xvcjojMzM3YWI3fS5wYW5lbC1wcmltYXJ5Pi5wYW5lbC1oZWFkaW5nKy5wYW5lbC1j
b2xsYXBzZT4ucGFuZWwtYm9keXtib3JkZXItdG9wLWNvbG9yOiMzMzdhYjd9LnBhbmVsLXByaW1h
cnk+LnBhbmVsLWhlYWRpbmcgLmJhZGdle2NvbG9yOiMzMzdhYjc7YmFja2dyb3VuZC1jb2xvcjoj
ZmZmfS5wYW5lbC1wcmltYXJ5Pi5wYW5lbC1mb290ZXIrLnBhbmVsLWNvbGxhcHNlPi5wYW5lbC1i
b2R5e2JvcmRlci1ib3R0b20tY29sb3I6IzMzN2FiN30ucGFuZWwtc3VjY2Vzc3tib3JkZXItY29s
b3I6I2Q2ZTljNn0ucGFuZWwtc3VjY2Vzcz4ucGFuZWwtaGVhZGluZ3tjb2xvcjojM2M3NjNkO2Jh
Y2tncm91bmQtY29sb3I6I2RmZjBkODtib3JkZXItY29sb3I6I2Q2ZTljNn0ucGFuZWwtc3VjY2Vz
cz4ucGFuZWwtaGVhZGluZysucGFuZWwtY29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9yZGVyLXRvcC1j
b2xvcjojZDZlOWM2fS5wYW5lbC1zdWNjZXNzPi5wYW5lbC1oZWFkaW5nIC5iYWRnZXtjb2xvcjoj
ZGZmMGQ4O2JhY2tncm91bmQtY29sb3I6IzNjNzYzZH0ucGFuZWwtc3VjY2Vzcz4ucGFuZWwtZm9v
dGVyKy5wYW5lbC1jb2xsYXBzZT4ucGFuZWwtYm9keXtib3JkZXItYm90dG9tLWNvbG9yOiNkNmU5
YzZ9LnBhbmVsLWluZm97Ym9yZGVyLWNvbG9yOiNiY2U4ZjF9LnBhbmVsLWluZm8+LnBhbmVsLWhl
YWRpbmd7Y29sb3I6IzMxNzA4ZjtiYWNrZ3JvdW5kLWNvbG9yOiNkOWVkZjc7Ym9yZGVyLWNvbG9y
OiNiY2U4ZjF9LnBhbmVsLWluZm8+LnBhbmVsLWhlYWRpbmcrLnBhbmVsLWNvbGxhcHNlPi5wYW5l
bC1ib2R5e2JvcmRlci10b3AtY29sb3I6I2JjZThmMX0ucGFuZWwtaW5mbz4ucGFuZWwtaGVhZGlu
ZyAuYmFkZ2V7Y29sb3I6I2Q5ZWRmNztiYWNrZ3JvdW5kLWNvbG9yOiMzMTcwOGZ9LnBhbmVsLWlu
Zm8+LnBhbmVsLWZvb3RlcisucGFuZWwtY29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9yZGVyLWJvdHRv
bS1jb2xvcjojYmNlOGYxfS5wYW5lbC13YXJuaW5ne2JvcmRlci1jb2xvcjojZmFlYmNjfS5wYW5l
bC13YXJuaW5nPi5wYW5lbC1oZWFkaW5ne2NvbG9yOiM4YTZkM2I7YmFja2dyb3VuZC1jb2xvcjoj
ZmNmOGUzO2JvcmRlci1jb2xvcjojZmFlYmNjfS5wYW5lbC13YXJuaW5nPi5wYW5lbC1oZWFkaW5n
Ky5wYW5lbC1jb2xsYXBzZT4ucGFuZWwtYm9keXtib3JkZXItdG9wLWNvbG9yOiNmYWViY2N9LnBh
bmVsLXdhcm5pbmc+LnBhbmVsLWhlYWRpbmcgLmJhZGdle2NvbG9yOiNmY2Y4ZTM7YmFja2dyb3Vu
ZC1jb2xvcjojOGE2ZDNifS5wYW5lbC13YXJuaW5nPi5wYW5lbC1mb290ZXIrLnBhbmVsLWNvbGxh
cHNlPi5wYW5lbC1ib2R5e2JvcmRlci1ib3R0b20tY29sb3I6I2ZhZWJjY30ucGFuZWwtZGFuZ2Vy
e2JvcmRlci1jb2xvcjojZWJjY2QxfS5wYW5lbC1kYW5nZXI+LnBhbmVsLWhlYWRpbmd7Y29sb3I6
I2E5NDQ0MjtiYWNrZ3JvdW5kLWNvbG9yOiNmMmRlZGU7Ym9yZGVyLWNvbG9yOiNlYmNjZDF9LnBh
bmVsLWRhbmdlcj4ucGFuZWwtaGVhZGluZysucGFuZWwtY29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9y
ZGVyLXRvcC1jb2xvcjojZWJjY2QxfS5wYW5lbC1kYW5nZXI+LnBhbmVsLWhlYWRpbmcgLmJhZGdl
e2NvbG9yOiNmMmRlZGU7YmFja2dyb3VuZC1jb2xvcjojYTk0NDQyfS5wYW5lbC1kYW5nZXI+LnBh
bmVsLWZvb3RlcisucGFuZWwtY29sbGFwc2U+LnBhbmVsLWJvZHl7Ym9yZGVyLWJvdHRvbS1jb2xv
cjojZWJjY2QxfS5lbWJlZC1yZXNwb25zaXZle3Bvc2l0aW9uOnJlbGF0aXZlO2Rpc3BsYXk6Ymxv
Y2s7aGVpZ2h0OjA7cGFkZGluZzowO292ZXJmbG93OmhpZGRlbn0uZW1iZWQtcmVzcG9uc2l2ZSAu
ZW1iZWQtcmVzcG9uc2l2ZS1pdGVtLC5lbWJlZC1yZXNwb25zaXZlIGVtYmVkLC5lbWJlZC1yZXNw
b25zaXZlIGlmcmFtZSwuZW1iZWQtcmVzcG9uc2l2ZSBvYmplY3QsLmVtYmVkLXJlc3BvbnNpdmUg
dmlkZW97cG9zaXRpb246YWJzb2x1dGU7dG9wOjA7Ym90dG9tOjA7bGVmdDowO3dpZHRoOjEwMCU7
aGVpZ2h0OjEwMCU7Ym9yZGVyOjB9LmVtYmVkLXJlc3BvbnNpdmUtMTZieTl7cGFkZGluZy1ib3R0
b206NTYuMjUlfS5lbWJlZC1yZXNwb25zaXZlLTRieTN7cGFkZGluZy1ib3R0b206NzUlfS53ZWxs
e21pbi1oZWlnaHQ6MjBweDtwYWRkaW5nOjE5cHg7bWFyZ2luLWJvdHRvbToyMHB4O2JhY2tncm91
bmQtY29sb3I6I2Y1ZjVmNTtib3JkZXI6MXB4IHNvbGlkICNlM2UzZTM7Ym9yZGVyLXJhZGl1czo0
cHg7LXdlYmtpdC1ib3gtc2hhZG93Omluc2V0IDAgMXB4IDFweCByZ2JhKDAsMCwwLC4wNSk7Ym94
LXNoYWRvdzppbnNldCAwIDFweCAxcHggcmdiYSgwLDAsMCwuMDUpfS53ZWxsIGJsb2NrcXVvdGV7
Ym9yZGVyLWNvbG9yOiNkZGQ7Ym9yZGVyLWNvbG9yOnJnYmEoMCwwLDAsLjE1KX0ud2VsbC1sZ3tw
YWRkaW5nOjI0cHg7Ym9yZGVyLXJhZGl1czo2cHh9LndlbGwtc217cGFkZGluZzo5cHg7Ym9yZGVy
LXJhZGl1czozcHh9LmNsb3Nle2Zsb2F0OnJpZ2h0O2ZvbnQtc2l6ZToyMXB4O2ZvbnQtd2VpZ2h0
OjcwMDtsaW5lLWhlaWdodDoxO2NvbG9yOiMwMDA7dGV4dC1zaGFkb3c6MCAxcHggMCAjZmZmO2Zp
bHRlcjphbHBoYShvcGFjaXR5PTIwKTtvcGFjaXR5Oi4yfS5jbG9zZTpmb2N1cywuY2xvc2U6aG92
ZXJ7Y29sb3I6IzAwMDt0ZXh0LWRlY29yYXRpb246bm9uZTtjdXJzb3I6cG9pbnRlcjtmaWx0ZXI6
YWxwaGEob3BhY2l0eT01MCk7b3BhY2l0eTouNX1idXR0b24uY2xvc2V7LXdlYmtpdC1hcHBlYXJh
bmNlOm5vbmU7cGFkZGluZzowO2N1cnNvcjpwb2ludGVyO2JhY2tncm91bmQ6MCAwO2JvcmRlcjow
fS5tb2RhbC1vcGVue292ZXJmbG93OmhpZGRlbn0ubW9kYWx7cG9zaXRpb246Zml4ZWQ7dG9wOjA7
cmlnaHQ6MDtib3R0b206MDtsZWZ0OjA7ei1pbmRleDoxMDUwO2Rpc3BsYXk6bm9uZTtvdmVyZmxv
dzpoaWRkZW47LXdlYmtpdC1vdmVyZmxvdy1zY3JvbGxpbmc6dG91Y2g7b3V0bGluZTowfS5tb2Rh
bC5mYWRlIC5tb2RhbC1kaWFsb2d7LXdlYmtpdC10cmFuc2l0aW9uOi13ZWJraXQtdHJhbnNmb3Jt
IC4zcyBlYXNlLW91dDstby10cmFuc2l0aW9uOi1vLXRyYW5zZm9ybSAuM3MgZWFzZS1vdXQ7dHJh
bnNpdGlvbjp0cmFuc2Zvcm0gLjNzIGVhc2Utb3V0Oy13ZWJraXQtdHJhbnNmb3JtOnRyYW5zbGF0
ZSgwLC0yNSUpOy1tcy10cmFuc2Zvcm06dHJhbnNsYXRlKDAsLTI1JSk7LW8tdHJhbnNmb3JtOnRy
YW5zbGF0ZSgwLC0yNSUpO3RyYW5zZm9ybTp0cmFuc2xhdGUoMCwtMjUlKX0ubW9kYWwuaW4gLm1v
ZGFsLWRpYWxvZ3std2Via2l0LXRyYW5zZm9ybTp0cmFuc2xhdGUoMCwwKTstbXMtdHJhbnNmb3Jt
OnRyYW5zbGF0ZSgwLDApOy1vLXRyYW5zZm9ybTp0cmFuc2xhdGUoMCwwKTt0cmFuc2Zvcm06dHJh
bnNsYXRlKDAsMCl9Lm1vZGFsLW9wZW4gLm1vZGFse292ZXJmbG93LXg6aGlkZGVuO292ZXJmbG93
LXk6YXV0b30ubW9kYWwtZGlhbG9ne3Bvc2l0aW9uOnJlbGF0aXZlO3dpZHRoOmF1dG87bWFyZ2lu
OjEwcHh9Lm1vZGFsLWNvbnRlbnR7cG9zaXRpb246cmVsYXRpdmU7YmFja2dyb3VuZC1jb2xvcjoj
ZmZmOy13ZWJraXQtYmFja2dyb3VuZC1jbGlwOnBhZGRpbmctYm94O2JhY2tncm91bmQtY2xpcDpw
YWRkaW5nLWJveDtib3JkZXI6MXB4IHNvbGlkICM5OTk7Ym9yZGVyOjFweCBzb2xpZCByZ2JhKDAs
MCwwLC4yKTtib3JkZXItcmFkaXVzOjZweDtvdXRsaW5lOjA7LXdlYmtpdC1ib3gtc2hhZG93OjAg
M3B4IDlweCByZ2JhKDAsMCwwLC41KTtib3gtc2hhZG93OjAgM3B4IDlweCByZ2JhKDAsMCwwLC41
KX0ubW9kYWwtYmFja2Ryb3B7cG9zaXRpb246Zml4ZWQ7dG9wOjA7cmlnaHQ6MDtib3R0b206MDts
ZWZ0OjA7ei1pbmRleDoxMDQwO2JhY2tncm91bmQtY29sb3I6IzAwMH0ubW9kYWwtYmFja2Ryb3Au
ZmFkZXtmaWx0ZXI6YWxwaGEob3BhY2l0eT0wKTtvcGFjaXR5OjB9Lm1vZGFsLWJhY2tkcm9wLmlu
e2ZpbHRlcjphbHBoYShvcGFjaXR5PTUwKTtvcGFjaXR5Oi41fS5tb2RhbC1oZWFkZXJ7cGFkZGlu
ZzoxNXB4O2JvcmRlci1ib3R0b206MXB4IHNvbGlkICNlNWU1ZTV9Lm1vZGFsLWhlYWRlciAuY2xv
c2V7bWFyZ2luLXRvcDotMnB4fS5tb2RhbC10aXRsZXttYXJnaW46MDtsaW5lLWhlaWdodDoxLjQy
ODU3MTQzfS5tb2RhbC1ib2R5e3Bvc2l0aW9uOnJlbGF0aXZlO3BhZGRpbmc6MTVweH0ubW9kYWwt
Zm9vdGVye3BhZGRpbmc6MTVweDt0ZXh0LWFsaWduOnJpZ2h0O2JvcmRlci10b3A6MXB4IHNvbGlk
ICNlNWU1ZTV9Lm1vZGFsLWZvb3RlciAuYnRuKy5idG57bWFyZ2luLWJvdHRvbTowO21hcmdpbi1s
ZWZ0OjVweH0ubW9kYWwtZm9vdGVyIC5idG4tZ3JvdXAgLmJ0bisuYnRue21hcmdpbi1sZWZ0Oi0x
cHh9Lm1vZGFsLWZvb3RlciAuYnRuLWJsb2NrKy5idG4tYmxvY2t7bWFyZ2luLWxlZnQ6MH0ubW9k
YWwtc2Nyb2xsYmFyLW1lYXN1cmV7cG9zaXRpb246YWJzb2x1dGU7dG9wOi05OTk5cHg7d2lkdGg6
NTBweDtoZWlnaHQ6NTBweDtvdmVyZmxvdzpzY3JvbGx9QG1lZGlhIChtaW4td2lkdGg6NzY4cHgp
ey5tb2RhbC1kaWFsb2d7d2lkdGg6NjAwcHg7bWFyZ2luOjMwcHggYXV0b30ubW9kYWwtY29udGVu
dHstd2Via2l0LWJveC1zaGFkb3c6MCA1cHggMTVweCByZ2JhKDAsMCwwLC41KTtib3gtc2hhZG93
OjAgNXB4IDE1cHggcmdiYSgwLDAsMCwuNSl9Lm1vZGFsLXNte3dpZHRoOjMwMHB4fX1AbWVkaWEg
KG1pbi13aWR0aDo5OTJweCl7Lm1vZGFsLWxne3dpZHRoOjkwMHB4fX0udG9vbHRpcHtwb3NpdGlv
bjphYnNvbHV0ZTt6LWluZGV4OjEwNzA7ZGlzcGxheTpibG9jaztmb250LWZhbWlseToiSGVsdmV0
aWNhIE5ldWUiLEhlbHZldGljYSxBcmlhbCxzYW5zLXNlcmlmO2ZvbnQtc2l6ZToxMnB4O2ZvbnQt
c3R5bGU6bm9ybWFsO2ZvbnQtd2VpZ2h0OjQwMDtsaW5lLWhlaWdodDoxLjQyODU3MTQzO3RleHQt
YWxpZ246bGVmdDt0ZXh0LWFsaWduOnN0YXJ0O3RleHQtZGVjb3JhdGlvbjpub25lO3RleHQtc2hh
ZG93Om5vbmU7dGV4dC10cmFuc2Zvcm06bm9uZTtsZXR0ZXItc3BhY2luZzpub3JtYWw7d29yZC1i
cmVhazpub3JtYWw7d29yZC1zcGFjaW5nOm5vcm1hbDt3b3JkLXdyYXA6bm9ybWFsO3doaXRlLXNw
YWNlOm5vcm1hbDtmaWx0ZXI6YWxwaGEob3BhY2l0eT0wKTtvcGFjaXR5OjA7bGluZS1icmVhazph
dXRvfS50b29sdGlwLmlue2ZpbHRlcjphbHBoYShvcGFjaXR5PTkwKTtvcGFjaXR5Oi45fS50b29s
dGlwLnRvcHtwYWRkaW5nOjVweCAwO21hcmdpbi10b3A6LTNweH0udG9vbHRpcC5yaWdodHtwYWRk
aW5nOjAgNXB4O21hcmdpbi1sZWZ0OjNweH0udG9vbHRpcC5ib3R0b217cGFkZGluZzo1cHggMDtt
YXJnaW4tdG9wOjNweH0udG9vbHRpcC5sZWZ0e3BhZGRpbmc6MCA1cHg7bWFyZ2luLWxlZnQ6LTNw
eH0udG9vbHRpcC1pbm5lcnttYXgtd2lkdGg6MjAwcHg7cGFkZGluZzozcHggOHB4O2NvbG9yOiNm
ZmY7dGV4dC1hbGlnbjpjZW50ZXI7YmFja2dyb3VuZC1jb2xvcjojMDAwO2JvcmRlci1yYWRpdXM6
NHB4fS50b29sdGlwLWFycm93e3Bvc2l0aW9uOmFic29sdXRlO3dpZHRoOjA7aGVpZ2h0OjA7Ym9y
ZGVyLWNvbG9yOnRyYW5zcGFyZW50O2JvcmRlci1zdHlsZTpzb2xpZH0udG9vbHRpcC50b3AgLnRv
b2x0aXAtYXJyb3d7Ym90dG9tOjA7bGVmdDo1MCU7bWFyZ2luLWxlZnQ6LTVweDtib3JkZXItd2lk
dGg6NXB4IDVweCAwO2JvcmRlci10b3AtY29sb3I6IzAwMH0udG9vbHRpcC50b3AtbGVmdCAudG9v
bHRpcC1hcnJvd3tyaWdodDo1cHg7Ym90dG9tOjA7bWFyZ2luLWJvdHRvbTotNXB4O2JvcmRlci13
aWR0aDo1cHggNXB4IDA7Ym9yZGVyLXRvcC1jb2xvcjojMDAwfS50b29sdGlwLnRvcC1yaWdodCAu
dG9vbHRpcC1hcnJvd3tib3R0b206MDtsZWZ0OjVweDttYXJnaW4tYm90dG9tOi01cHg7Ym9yZGVy
LXdpZHRoOjVweCA1cHggMDtib3JkZXItdG9wLWNvbG9yOiMwMDB9LnRvb2x0aXAucmlnaHQgLnRv
b2x0aXAtYXJyb3d7dG9wOjUwJTtsZWZ0OjA7bWFyZ2luLXRvcDotNXB4O2JvcmRlci13aWR0aDo1
cHggNXB4IDVweCAwO2JvcmRlci1yaWdodC1jb2xvcjojMDAwfS50b29sdGlwLmxlZnQgLnRvb2x0
aXAtYXJyb3d7dG9wOjUwJTtyaWdodDowO21hcmdpbi10b3A6LTVweDtib3JkZXItd2lkdGg6NXB4
IDAgNXB4IDVweDtib3JkZXItbGVmdC1jb2xvcjojMDAwfS50b29sdGlwLmJvdHRvbSAudG9vbHRp
cC1hcnJvd3t0b3A6MDtsZWZ0OjUwJTttYXJnaW4tbGVmdDotNXB4O2JvcmRlci13aWR0aDowIDVw
eCA1cHg7Ym9yZGVyLWJvdHRvbS1jb2xvcjojMDAwfS50b29sdGlwLmJvdHRvbS1sZWZ0IC50b29s
dGlwLWFycm93e3RvcDowO3JpZ2h0OjVweDttYXJnaW4tdG9wOi01cHg7Ym9yZGVyLXdpZHRoOjAg
NXB4IDVweDtib3JkZXItYm90dG9tLWNvbG9yOiMwMDB9LnRvb2x0aXAuYm90dG9tLXJpZ2h0IC50
b29sdGlwLWFycm93e3RvcDowO2xlZnQ6NXB4O21hcmdpbi10b3A6LTVweDtib3JkZXItd2lkdGg6
MCA1cHggNXB4O2JvcmRlci1ib3R0b20tY29sb3I6IzAwMH0ucG9wb3Zlcntwb3NpdGlvbjphYnNv
bHV0ZTt0b3A6MDtsZWZ0OjA7ei1pbmRleDoxMDYwO2Rpc3BsYXk6bm9uZTttYXgtd2lkdGg6Mjc2
cHg7cGFkZGluZzoxcHg7Zm9udC1mYW1pbHk6IkhlbHZldGljYSBOZXVlIixIZWx2ZXRpY2EsQXJp
YWwsc2Fucy1zZXJpZjtmb250LXNpemU6MTRweDtmb250LXN0eWxlOm5vcm1hbDtmb250LXdlaWdo
dDo0MDA7bGluZS1oZWlnaHQ6MS40Mjg1NzE0Mzt0ZXh0LWFsaWduOmxlZnQ7dGV4dC1hbGlnbjpz
dGFydDt0ZXh0LWRlY29yYXRpb246bm9uZTt0ZXh0LXNoYWRvdzpub25lO3RleHQtdHJhbnNmb3Jt
Om5vbmU7bGV0dGVyLXNwYWNpbmc6bm9ybWFsO3dvcmQtYnJlYWs6bm9ybWFsO3dvcmQtc3BhY2lu
Zzpub3JtYWw7d29yZC13cmFwOm5vcm1hbDt3aGl0ZS1zcGFjZTpub3JtYWw7YmFja2dyb3VuZC1j
b2xvcjojZmZmOy13ZWJraXQtYmFja2dyb3VuZC1jbGlwOnBhZGRpbmctYm94O2JhY2tncm91bmQt
Y2xpcDpwYWRkaW5nLWJveDtib3JkZXI6MXB4IHNvbGlkICNjY2M7Ym9yZGVyOjFweCBzb2xpZCBy
Z2JhKDAsMCwwLC4yKTtib3JkZXItcmFkaXVzOjZweDstd2Via2l0LWJveC1zaGFkb3c6MCA1cHgg
MTBweCByZ2JhKDAsMCwwLC4yKTtib3gtc2hhZG93OjAgNXB4IDEwcHggcmdiYSgwLDAsMCwuMik7
bGluZS1icmVhazphdXRvfS5wb3BvdmVyLnRvcHttYXJnaW4tdG9wOi0xMHB4fS5wb3BvdmVyLnJp
Z2h0e21hcmdpbi1sZWZ0OjEwcHh9LnBvcG92ZXIuYm90dG9te21hcmdpbi10b3A6MTBweH0ucG9w
b3Zlci5sZWZ0e21hcmdpbi1sZWZ0Oi0xMHB4fS5wb3BvdmVyLXRpdGxle3BhZGRpbmc6OHB4IDE0
cHg7bWFyZ2luOjA7Zm9udC1zaXplOjE0cHg7YmFja2dyb3VuZC1jb2xvcjojZjdmN2Y3O2JvcmRl
ci1ib3R0b206MXB4IHNvbGlkICNlYmViZWI7Ym9yZGVyLXJhZGl1czo1cHggNXB4IDAgMH0ucG9w
b3Zlci1jb250ZW50e3BhZGRpbmc6OXB4IDE0cHh9LnBvcG92ZXI+LmFycm93LC5wb3BvdmVyPi5h
cnJvdzphZnRlcntwb3NpdGlvbjphYnNvbHV0ZTtkaXNwbGF5OmJsb2NrO3dpZHRoOjA7aGVpZ2h0
OjA7Ym9yZGVyLWNvbG9yOnRyYW5zcGFyZW50O2JvcmRlci1zdHlsZTpzb2xpZH0ucG9wb3Zlcj4u
YXJyb3d7Ym9yZGVyLXdpZHRoOjExcHh9LnBvcG92ZXI+LmFycm93OmFmdGVye2NvbnRlbnQ6IiI7
Ym9yZGVyLXdpZHRoOjEwcHh9LnBvcG92ZXIudG9wPi5hcnJvd3tib3R0b206LTExcHg7bGVmdDo1
MCU7bWFyZ2luLWxlZnQ6LTExcHg7Ym9yZGVyLXRvcC1jb2xvcjojOTk5O2JvcmRlci10b3AtY29s
b3I6cmdiYSgwLDAsMCwuMjUpO2JvcmRlci1ib3R0b20td2lkdGg6MH0ucG9wb3Zlci50b3A+LmFy
cm93OmFmdGVye2JvdHRvbToxcHg7bWFyZ2luLWxlZnQ6LTEwcHg7Y29udGVudDoiICI7Ym9yZGVy
LXRvcC1jb2xvcjojZmZmO2JvcmRlci1ib3R0b20td2lkdGg6MH0ucG9wb3Zlci5yaWdodD4uYXJy
b3d7dG9wOjUwJTtsZWZ0Oi0xMXB4O21hcmdpbi10b3A6LTExcHg7Ym9yZGVyLXJpZ2h0LWNvbG9y
OiM5OTk7Ym9yZGVyLXJpZ2h0LWNvbG9yOnJnYmEoMCwwLDAsLjI1KTtib3JkZXItbGVmdC13aWR0
aDowfS5wb3BvdmVyLnJpZ2h0Pi5hcnJvdzphZnRlcntib3R0b206LTEwcHg7bGVmdDoxcHg7Y29u
dGVudDoiICI7Ym9yZGVyLXJpZ2h0LWNvbG9yOiNmZmY7Ym9yZGVyLWxlZnQtd2lkdGg6MH0ucG9w
b3Zlci5ib3R0b20+LmFycm93e3RvcDotMTFweDtsZWZ0OjUwJTttYXJnaW4tbGVmdDotMTFweDti
b3JkZXItdG9wLXdpZHRoOjA7Ym9yZGVyLWJvdHRvbS1jb2xvcjojOTk5O2JvcmRlci1ib3R0b20t
Y29sb3I6cmdiYSgwLDAsMCwuMjUpfS5wb3BvdmVyLmJvdHRvbT4uYXJyb3c6YWZ0ZXJ7dG9wOjFw
eDttYXJnaW4tbGVmdDotMTBweDtjb250ZW50OiIgIjtib3JkZXItdG9wLXdpZHRoOjA7Ym9yZGVy
LWJvdHRvbS1jb2xvcjojZmZmfS5wb3BvdmVyLmxlZnQ+LmFycm93e3RvcDo1MCU7cmlnaHQ6LTEx
cHg7bWFyZ2luLXRvcDotMTFweDtib3JkZXItcmlnaHQtd2lkdGg6MDtib3JkZXItbGVmdC1jb2xv
cjojOTk5O2JvcmRlci1sZWZ0LWNvbG9yOnJnYmEoMCwwLDAsLjI1KX0ucG9wb3Zlci5sZWZ0Pi5h
cnJvdzphZnRlcntyaWdodDoxcHg7Ym90dG9tOi0xMHB4O2NvbnRlbnQ6IiAiO2JvcmRlci1yaWdo
dC13aWR0aDowO2JvcmRlci1sZWZ0LWNvbG9yOiNmZmZ9LmNhcm91c2Vse3Bvc2l0aW9uOnJlbGF0
aXZlfS5jYXJvdXNlbC1pbm5lcntwb3NpdGlvbjpyZWxhdGl2ZTt3aWR0aDoxMDAlO292ZXJmbG93
OmhpZGRlbn0uY2Fyb3VzZWwtaW5uZXI+Lml0ZW17cG9zaXRpb246cmVsYXRpdmU7ZGlzcGxheTpu
b25lOy13ZWJraXQtdHJhbnNpdGlvbjouNnMgZWFzZS1pbi1vdXQgbGVmdDstby10cmFuc2l0aW9u
Oi42cyBlYXNlLWluLW91dCBsZWZ0O3RyYW5zaXRpb246LjZzIGVhc2UtaW4tb3V0IGxlZnR9LmNh
cm91c2VsLWlubmVyPi5pdGVtPmE+aW1nLC5jYXJvdXNlbC1pbm5lcj4uaXRlbT5pbWd7bGluZS1o
ZWlnaHQ6MX1AbWVkaWEgYWxsIGFuZCAodHJhbnNmb3JtLTNkKSwoLXdlYmtpdC10cmFuc2Zvcm0t
M2Qpey5jYXJvdXNlbC1pbm5lcj4uaXRlbXstd2Via2l0LXRyYW5zaXRpb246LXdlYmtpdC10cmFu
c2Zvcm0gLjZzIGVhc2UtaW4tb3V0Oy1vLXRyYW5zaXRpb246LW8tdHJhbnNmb3JtIC42cyBlYXNl
LWluLW91dDt0cmFuc2l0aW9uOnRyYW5zZm9ybSAuNnMgZWFzZS1pbi1vdXQ7LXdlYmtpdC1iYWNr
ZmFjZS12aXNpYmlsaXR5OmhpZGRlbjtiYWNrZmFjZS12aXNpYmlsaXR5OmhpZGRlbjstd2Via2l0
LXBlcnNwZWN0aXZlOjEwMDBweDtwZXJzcGVjdGl2ZToxMDAwcHh9LmNhcm91c2VsLWlubmVyPi5p
dGVtLmFjdGl2ZS5yaWdodCwuY2Fyb3VzZWwtaW5uZXI+Lml0ZW0ubmV4dHtsZWZ0OjA7LXdlYmtp
dC10cmFuc2Zvcm06dHJhbnNsYXRlM2QoMTAwJSwwLDApO3RyYW5zZm9ybTp0cmFuc2xhdGUzZCgx
MDAlLDAsMCl9LmNhcm91c2VsLWlubmVyPi5pdGVtLmFjdGl2ZS5sZWZ0LC5jYXJvdXNlbC1pbm5l
cj4uaXRlbS5wcmV2e2xlZnQ6MDstd2Via2l0LXRyYW5zZm9ybTp0cmFuc2xhdGUzZCgtMTAwJSww
LDApO3RyYW5zZm9ybTp0cmFuc2xhdGUzZCgtMTAwJSwwLDApfS5jYXJvdXNlbC1pbm5lcj4uaXRl
bS5hY3RpdmUsLmNhcm91c2VsLWlubmVyPi5pdGVtLm5leHQubGVmdCwuY2Fyb3VzZWwtaW5uZXI+
Lml0ZW0ucHJldi5yaWdodHtsZWZ0OjA7LXdlYmtpdC10cmFuc2Zvcm06dHJhbnNsYXRlM2QoMCww
LDApO3RyYW5zZm9ybTp0cmFuc2xhdGUzZCgwLDAsMCl9fS5jYXJvdXNlbC1pbm5lcj4uYWN0aXZl
LC5jYXJvdXNlbC1pbm5lcj4ubmV4dCwuY2Fyb3VzZWwtaW5uZXI+LnByZXZ7ZGlzcGxheTpibG9j
a30uY2Fyb3VzZWwtaW5uZXI+LmFjdGl2ZXtsZWZ0OjB9LmNhcm91c2VsLWlubmVyPi5uZXh0LC5j
YXJvdXNlbC1pbm5lcj4ucHJldntwb3NpdGlvbjphYnNvbHV0ZTt0b3A6MDt3aWR0aDoxMDAlfS5j
YXJvdXNlbC1pbm5lcj4ubmV4dHtsZWZ0OjEwMCV9LmNhcm91c2VsLWlubmVyPi5wcmV2e2xlZnQ6
LTEwMCV9LmNhcm91c2VsLWlubmVyPi5uZXh0LmxlZnQsLmNhcm91c2VsLWlubmVyPi5wcmV2LnJp
Z2h0e2xlZnQ6MH0uY2Fyb3VzZWwtaW5uZXI+LmFjdGl2ZS5sZWZ0e2xlZnQ6LTEwMCV9LmNhcm91
c2VsLWlubmVyPi5hY3RpdmUucmlnaHR7bGVmdDoxMDAlfS5jYXJvdXNlbC1jb250cm9se3Bvc2l0
aW9uOmFic29sdXRlO3RvcDowO2JvdHRvbTowO2xlZnQ6MDt3aWR0aDoxNSU7Zm9udC1zaXplOjIw
cHg7Y29sb3I6I2ZmZjt0ZXh0LWFsaWduOmNlbnRlcjt0ZXh0LXNoYWRvdzowIDFweCAycHggcmdi
YSgwLDAsMCwuNik7YmFja2dyb3VuZC1jb2xvcjpyZ2JhKDAsMCwwLDApO2ZpbHRlcjphbHBoYShv
cGFjaXR5PTUwKTtvcGFjaXR5Oi41fS5jYXJvdXNlbC1jb250cm9sLmxlZnR7YmFja2dyb3VuZC1p
bWFnZTotd2Via2l0LWxpbmVhci1ncmFkaWVudChsZWZ0LHJnYmEoMCwwLDAsLjUpIDAscmdiYSgw
LDAsMCwuMDAwMSkgMTAwJSk7YmFja2dyb3VuZC1pbWFnZTotby1saW5lYXItZ3JhZGllbnQobGVm
dCxyZ2JhKDAsMCwwLC41KSAwLHJnYmEoMCwwLDAsLjAwMDEpIDEwMCUpO2JhY2tncm91bmQtaW1h
Z2U6LXdlYmtpdC1ncmFkaWVudChsaW5lYXIsbGVmdCB0b3AscmlnaHQgdG9wLGZyb20ocmdiYSgw
LDAsMCwuNSkpLHRvKHJnYmEoMCwwLDAsLjAwMDEpKSk7YmFja2dyb3VuZC1pbWFnZTpsaW5lYXIt
Z3JhZGllbnQodG8gcmlnaHQscmdiYSgwLDAsMCwuNSkgMCxyZ2JhKDAsMCwwLC4wMDAxKSAxMDAl
KTtmaWx0ZXI6cHJvZ2lkOkRYSW1hZ2VUcmFuc2Zvcm0uTWljcm9zb2Z0LmdyYWRpZW50KHN0YXJ0
Q29sb3JzdHI9JyM4MDAwMDAwMCcsIGVuZENvbG9yc3RyPScjMDAwMDAwMDAnLCBHcmFkaWVudFR5
cGU9MSk7YmFja2dyb3VuZC1yZXBlYXQ6cmVwZWF0LXh9LmNhcm91c2VsLWNvbnRyb2wucmlnaHR7
cmlnaHQ6MDtsZWZ0OmF1dG87YmFja2dyb3VuZC1pbWFnZTotd2Via2l0LWxpbmVhci1ncmFkaWVu
dChsZWZ0LHJnYmEoMCwwLDAsLjAwMDEpIDAscmdiYSgwLDAsMCwuNSkgMTAwJSk7YmFja2dyb3Vu
ZC1pbWFnZTotby1saW5lYXItZ3JhZGllbnQobGVmdCxyZ2JhKDAsMCwwLC4wMDAxKSAwLHJnYmEo
MCwwLDAsLjUpIDEwMCUpO2JhY2tncm91bmQtaW1hZ2U6LXdlYmtpdC1ncmFkaWVudChsaW5lYXIs
bGVmdCB0b3AscmlnaHQgdG9wLGZyb20ocmdiYSgwLDAsMCwuMDAwMSkpLHRvKHJnYmEoMCwwLDAs
LjUpKSk7YmFja2dyb3VuZC1pbWFnZTpsaW5lYXItZ3JhZGllbnQodG8gcmlnaHQscmdiYSgwLDAs
MCwuMDAwMSkgMCxyZ2JhKDAsMCwwLC41KSAxMDAlKTtmaWx0ZXI6cHJvZ2lkOkRYSW1hZ2VUcmFu
c2Zvcm0uTWljcm9zb2Z0LmdyYWRpZW50KHN0YXJ0Q29sb3JzdHI9JyMwMDAwMDAwMCcsIGVuZENv
bG9yc3RyPScjODAwMDAwMDAnLCBHcmFkaWVudFR5cGU9MSk7YmFja2dyb3VuZC1yZXBlYXQ6cmVw
ZWF0LXh9LmNhcm91c2VsLWNvbnRyb2w6Zm9jdXMsLmNhcm91c2VsLWNvbnRyb2w6aG92ZXJ7Y29s
b3I6I2ZmZjt0ZXh0LWRlY29yYXRpb246bm9uZTtmaWx0ZXI6YWxwaGEob3BhY2l0eT05MCk7b3V0
bGluZTowO29wYWNpdHk6Ljl9LmNhcm91c2VsLWNvbnRyb2wgLmdseXBoaWNvbi1jaGV2cm9uLWxl
ZnQsLmNhcm91c2VsLWNvbnRyb2wgLmdseXBoaWNvbi1jaGV2cm9uLXJpZ2h0LC5jYXJvdXNlbC1j
b250cm9sIC5pY29uLW5leHQsLmNhcm91c2VsLWNvbnRyb2wgLmljb24tcHJldntwb3NpdGlvbjph
YnNvbHV0ZTt0b3A6NTAlO3otaW5kZXg6NTtkaXNwbGF5OmlubGluZS1ibG9jazttYXJnaW4tdG9w
Oi0xMHB4fS5jYXJvdXNlbC1jb250cm9sIC5nbHlwaGljb24tY2hldnJvbi1sZWZ0LC5jYXJvdXNl
bC1jb250cm9sIC5pY29uLXByZXZ7bGVmdDo1MCU7bWFyZ2luLWxlZnQ6LTEwcHh9LmNhcm91c2Vs
LWNvbnRyb2wgLmdseXBoaWNvbi1jaGV2cm9uLXJpZ2h0LC5jYXJvdXNlbC1jb250cm9sIC5pY29u
LW5leHR7cmlnaHQ6NTAlO21hcmdpbi1yaWdodDotMTBweH0uY2Fyb3VzZWwtY29udHJvbCAuaWNv
bi1uZXh0LC5jYXJvdXNlbC1jb250cm9sIC5pY29uLXByZXZ7d2lkdGg6MjBweDtoZWlnaHQ6MjBw
eDtmb250LWZhbWlseTpzZXJpZjtsaW5lLWhlaWdodDoxfS5jYXJvdXNlbC1jb250cm9sIC5pY29u
LXByZXY6YmVmb3Jle2NvbnRlbnQ6J1wyMDM5J30uY2Fyb3VzZWwtY29udHJvbCAuaWNvbi1uZXh0
OmJlZm9yZXtjb250ZW50OidcMjAzYSd9LmNhcm91c2VsLWluZGljYXRvcnN7cG9zaXRpb246YWJz
b2x1dGU7Ym90dG9tOjEwcHg7bGVmdDo1MCU7ei1pbmRleDoxNTt3aWR0aDo2MCU7cGFkZGluZy1s
ZWZ0OjA7bWFyZ2luLWxlZnQ6LTMwJTt0ZXh0LWFsaWduOmNlbnRlcjtsaXN0LXN0eWxlOm5vbmV9
LmNhcm91c2VsLWluZGljYXRvcnMgbGl7ZGlzcGxheTppbmxpbmUtYmxvY2s7d2lkdGg6MTBweDto
ZWlnaHQ6MTBweDttYXJnaW46MXB4O3RleHQtaW5kZW50Oi05OTlweDtjdXJzb3I6cG9pbnRlcjti
YWNrZ3JvdW5kLWNvbG9yOiMwMDBcOTtiYWNrZ3JvdW5kLWNvbG9yOnJnYmEoMCwwLDAsMCk7Ym9y
ZGVyOjFweCBzb2xpZCAjZmZmO2JvcmRlci1yYWRpdXM6MTBweH0uY2Fyb3VzZWwtaW5kaWNhdG9y
cyAuYWN0aXZle3dpZHRoOjEycHg7aGVpZ2h0OjEycHg7bWFyZ2luOjA7YmFja2dyb3VuZC1jb2xv
cjojZmZmfS5jYXJvdXNlbC1jYXB0aW9ue3Bvc2l0aW9uOmFic29sdXRlO3JpZ2h0OjE1JTtib3R0
b206MjBweDtsZWZ0OjE1JTt6LWluZGV4OjEwO3BhZGRpbmctdG9wOjIwcHg7cGFkZGluZy1ib3R0
b206MjBweDtjb2xvcjojZmZmO3RleHQtYWxpZ246Y2VudGVyO3RleHQtc2hhZG93OjAgMXB4IDJw
eCByZ2JhKDAsMCwwLC42KX0uY2Fyb3VzZWwtY2FwdGlvbiAuYnRue3RleHQtc2hhZG93Om5vbmV9
QG1lZGlhIHNjcmVlbiBhbmQgKG1pbi13aWR0aDo3NjhweCl7LmNhcm91c2VsLWNvbnRyb2wgLmds
eXBoaWNvbi1jaGV2cm9uLWxlZnQsLmNhcm91c2VsLWNvbnRyb2wgLmdseXBoaWNvbi1jaGV2cm9u
LXJpZ2h0LC5jYXJvdXNlbC1jb250cm9sIC5pY29uLW5leHQsLmNhcm91c2VsLWNvbnRyb2wgLmlj
b24tcHJldnt3aWR0aDozMHB4O2hlaWdodDozMHB4O21hcmdpbi10b3A6LTEwcHg7Zm9udC1zaXpl
OjMwcHh9LmNhcm91c2VsLWNvbnRyb2wgLmdseXBoaWNvbi1jaGV2cm9uLWxlZnQsLmNhcm91c2Vs
LWNvbnRyb2wgLmljb24tcHJldnttYXJnaW4tbGVmdDotMTBweH0uY2Fyb3VzZWwtY29udHJvbCAu
Z2x5cGhpY29uLWNoZXZyb24tcmlnaHQsLmNhcm91c2VsLWNvbnRyb2wgLmljb24tbmV4dHttYXJn
aW4tcmlnaHQ6LTEwcHh9LmNhcm91c2VsLWNhcHRpb257cmlnaHQ6MjAlO2xlZnQ6MjAlO3BhZGRp
bmctYm90dG9tOjMwcHh9LmNhcm91c2VsLWluZGljYXRvcnN7Ym90dG9tOjIwcHh9fS5idG4tZ3Jv
dXAtdmVydGljYWw+LmJ0bi1ncm91cDphZnRlciwuYnRuLWdyb3VwLXZlcnRpY2FsPi5idG4tZ3Jv
dXA6YmVmb3JlLC5idG4tdG9vbGJhcjphZnRlciwuYnRuLXRvb2xiYXI6YmVmb3JlLC5jbGVhcmZp
eDphZnRlciwuY2xlYXJmaXg6YmVmb3JlLC5jb250YWluZXItZmx1aWQ6YWZ0ZXIsLmNvbnRhaW5l
ci1mbHVpZDpiZWZvcmUsLmNvbnRhaW5lcjphZnRlciwuY29udGFpbmVyOmJlZm9yZSwuZGwtaG9y
aXpvbnRhbCBkZDphZnRlciwuZGwtaG9yaXpvbnRhbCBkZDpiZWZvcmUsLmZvcm0taG9yaXpvbnRh
bCAuZm9ybS1ncm91cDphZnRlciwuZm9ybS1ob3Jpem9udGFsIC5mb3JtLWdyb3VwOmJlZm9yZSwu
bW9kYWwtZm9vdGVyOmFmdGVyLC5tb2RhbC1mb290ZXI6YmVmb3JlLC5tb2RhbC1oZWFkZXI6YWZ0
ZXIsLm1vZGFsLWhlYWRlcjpiZWZvcmUsLm5hdjphZnRlciwubmF2OmJlZm9yZSwubmF2YmFyLWNv
bGxhcHNlOmFmdGVyLC5uYXZiYXItY29sbGFwc2U6YmVmb3JlLC5uYXZiYXItaGVhZGVyOmFmdGVy
LC5uYXZiYXItaGVhZGVyOmJlZm9yZSwubmF2YmFyOmFmdGVyLC5uYXZiYXI6YmVmb3JlLC5wYWdl
cjphZnRlciwucGFnZXI6YmVmb3JlLC5wYW5lbC1ib2R5OmFmdGVyLC5wYW5lbC1ib2R5OmJlZm9y
ZSwucm93OmFmdGVyLC5yb3c6YmVmb3Jle2Rpc3BsYXk6dGFibGU7Y29udGVudDoiICJ9LmJ0bi1n
cm91cC12ZXJ0aWNhbD4uYnRuLWdyb3VwOmFmdGVyLC5idG4tdG9vbGJhcjphZnRlciwuY2xlYXJm
aXg6YWZ0ZXIsLmNvbnRhaW5lci1mbHVpZDphZnRlciwuY29udGFpbmVyOmFmdGVyLC5kbC1ob3Jp
em9udGFsIGRkOmFmdGVyLC5mb3JtLWhvcml6b250YWwgLmZvcm0tZ3JvdXA6YWZ0ZXIsLm1vZGFs
LWZvb3RlcjphZnRlciwubW9kYWwtaGVhZGVyOmFmdGVyLC5uYXY6YWZ0ZXIsLm5hdmJhci1jb2xs
YXBzZTphZnRlciwubmF2YmFyLWhlYWRlcjphZnRlciwubmF2YmFyOmFmdGVyLC5wYWdlcjphZnRl
ciwucGFuZWwtYm9keTphZnRlciwucm93OmFmdGVye2NsZWFyOmJvdGh9LmNlbnRlci1ibG9ja3tk
aXNwbGF5OmJsb2NrO21hcmdpbi1yaWdodDphdXRvO21hcmdpbi1sZWZ0OmF1dG99LnB1bGwtcmln
aHR7ZmxvYXQ6cmlnaHQhaW1wb3J0YW50fS5wdWxsLWxlZnR7ZmxvYXQ6bGVmdCFpbXBvcnRhbnR9
LmhpZGV7ZGlzcGxheTpub25lIWltcG9ydGFudH0uc2hvd3tkaXNwbGF5OmJsb2NrIWltcG9ydGFu
dH0uaW52aXNpYmxle3Zpc2liaWxpdHk6aGlkZGVufS50ZXh0LWhpZGV7Zm9udDowLzAgYTtjb2xv
cjp0cmFuc3BhcmVudDt0ZXh0LXNoYWRvdzpub25lO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJl
bnQ7Ym9yZGVyOjB9LmhpZGRlbntkaXNwbGF5Om5vbmUhaW1wb3J0YW50fS5hZmZpeHtwb3NpdGlv
bjpmaXhlZH1ALW1zLXZpZXdwb3J0e3dpZHRoOmRldmljZS13aWR0aH0udmlzaWJsZS1sZywudmlz
aWJsZS1tZCwudmlzaWJsZS1zbSwudmlzaWJsZS14c3tkaXNwbGF5Om5vbmUhaW1wb3J0YW50fS52
aXNpYmxlLWxnLWJsb2NrLC52aXNpYmxlLWxnLWlubGluZSwudmlzaWJsZS1sZy1pbmxpbmUtYmxv
Y2ssLnZpc2libGUtbWQtYmxvY2ssLnZpc2libGUtbWQtaW5saW5lLC52aXNpYmxlLW1kLWlubGlu
ZS1ibG9jaywudmlzaWJsZS1zbS1ibG9jaywudmlzaWJsZS1zbS1pbmxpbmUsLnZpc2libGUtc20t
aW5saW5lLWJsb2NrLC52aXNpYmxlLXhzLWJsb2NrLC52aXNpYmxlLXhzLWlubGluZSwudmlzaWJs
ZS14cy1pbmxpbmUtYmxvY2t7ZGlzcGxheTpub25lIWltcG9ydGFudH1AbWVkaWEgKG1heC13aWR0
aDo3NjdweCl7LnZpc2libGUteHN7ZGlzcGxheTpibG9jayFpbXBvcnRhbnR9dGFibGUudmlzaWJs
ZS14c3tkaXNwbGF5OnRhYmxlIWltcG9ydGFudH10ci52aXNpYmxlLXhze2Rpc3BsYXk6dGFibGUt
cm93IWltcG9ydGFudH10ZC52aXNpYmxlLXhzLHRoLnZpc2libGUteHN7ZGlzcGxheTp0YWJsZS1j
ZWxsIWltcG9ydGFudH19QG1lZGlhIChtYXgtd2lkdGg6NzY3cHgpey52aXNpYmxlLXhzLWJsb2Nr
e2Rpc3BsYXk6YmxvY2shaW1wb3J0YW50fX1AbWVkaWEgKG1heC13aWR0aDo3NjdweCl7LnZpc2li
bGUteHMtaW5saW5le2Rpc3BsYXk6aW5saW5lIWltcG9ydGFudH19QG1lZGlhIChtYXgtd2lkdGg6
NzY3cHgpey52aXNpYmxlLXhzLWlubGluZS1ibG9ja3tkaXNwbGF5OmlubGluZS1ibG9jayFpbXBv
cnRhbnR9fUBtZWRpYSAobWluLXdpZHRoOjc2OHB4KSBhbmQgKG1heC13aWR0aDo5OTFweCl7LnZp
c2libGUtc217ZGlzcGxheTpibG9jayFpbXBvcnRhbnR9dGFibGUudmlzaWJsZS1zbXtkaXNwbGF5
OnRhYmxlIWltcG9ydGFudH10ci52aXNpYmxlLXNte2Rpc3BsYXk6dGFibGUtcm93IWltcG9ydGFu
dH10ZC52aXNpYmxlLXNtLHRoLnZpc2libGUtc217ZGlzcGxheTp0YWJsZS1jZWxsIWltcG9ydGFu
dH19QG1lZGlhIChtaW4td2lkdGg6NzY4cHgpIGFuZCAobWF4LXdpZHRoOjk5MXB4KXsudmlzaWJs
ZS1zbS1ibG9ja3tkaXNwbGF5OmJsb2NrIWltcG9ydGFudH19QG1lZGlhIChtaW4td2lkdGg6NzY4
cHgpIGFuZCAobWF4LXdpZHRoOjk5MXB4KXsudmlzaWJsZS1zbS1pbmxpbmV7ZGlzcGxheTppbmxp
bmUhaW1wb3J0YW50fX1AbWVkaWEgKG1pbi13aWR0aDo3NjhweCkgYW5kIChtYXgtd2lkdGg6OTkx
cHgpey52aXNpYmxlLXNtLWlubGluZS1ibG9ja3tkaXNwbGF5OmlubGluZS1ibG9jayFpbXBvcnRh
bnR9fUBtZWRpYSAobWluLXdpZHRoOjk5MnB4KSBhbmQgKG1heC13aWR0aDoxMTk5cHgpey52aXNp
YmxlLW1ke2Rpc3BsYXk6YmxvY2shaW1wb3J0YW50fXRhYmxlLnZpc2libGUtbWR7ZGlzcGxheTp0
YWJsZSFpbXBvcnRhbnR9dHIudmlzaWJsZS1tZHtkaXNwbGF5OnRhYmxlLXJvdyFpbXBvcnRhbnR9
dGQudmlzaWJsZS1tZCx0aC52aXNpYmxlLW1ke2Rpc3BsYXk6dGFibGUtY2VsbCFpbXBvcnRhbnR9
fUBtZWRpYSAobWluLXdpZHRoOjk5MnB4KSBhbmQgKG1heC13aWR0aDoxMTk5cHgpey52aXNpYmxl
LW1kLWJsb2Nre2Rpc3BsYXk6YmxvY2shaW1wb3J0YW50fX1AbWVkaWEgKG1pbi13aWR0aDo5OTJw
eCkgYW5kIChtYXgtd2lkdGg6MTE5OXB4KXsudmlzaWJsZS1tZC1pbmxpbmV7ZGlzcGxheTppbmxp
bmUhaW1wb3J0YW50fX1AbWVkaWEgKG1pbi13aWR0aDo5OTJweCkgYW5kIChtYXgtd2lkdGg6MTE5
OXB4KXsudmlzaWJsZS1tZC1pbmxpbmUtYmxvY2t7ZGlzcGxheTppbmxpbmUtYmxvY2shaW1wb3J0
YW50fX1AbWVkaWEgKG1pbi13aWR0aDoxMjAwcHgpey52aXNpYmxlLWxne2Rpc3BsYXk6YmxvY2sh
aW1wb3J0YW50fXRhYmxlLnZpc2libGUtbGd7ZGlzcGxheTp0YWJsZSFpbXBvcnRhbnR9dHIudmlz
aWJsZS1sZ3tkaXNwbGF5OnRhYmxlLXJvdyFpbXBvcnRhbnR9dGQudmlzaWJsZS1sZyx0aC52aXNp
YmxlLWxne2Rpc3BsYXk6dGFibGUtY2VsbCFpbXBvcnRhbnR9fUBtZWRpYSAobWluLXdpZHRoOjEy
MDBweCl7LnZpc2libGUtbGctYmxvY2t7ZGlzcGxheTpibG9jayFpbXBvcnRhbnR9fUBtZWRpYSAo
bWluLXdpZHRoOjEyMDBweCl7LnZpc2libGUtbGctaW5saW5le2Rpc3BsYXk6aW5saW5lIWltcG9y
dGFudH19QG1lZGlhIChtaW4td2lkdGg6MTIwMHB4KXsudmlzaWJsZS1sZy1pbmxpbmUtYmxvY2t7
ZGlzcGxheTppbmxpbmUtYmxvY2shaW1wb3J0YW50fX1AbWVkaWEgKG1heC13aWR0aDo3NjdweCl7
LmhpZGRlbi14c3tkaXNwbGF5Om5vbmUhaW1wb3J0YW50fX1AbWVkaWEgKG1pbi13aWR0aDo3Njhw
eCkgYW5kIChtYXgtd2lkdGg6OTkxcHgpey5oaWRkZW4tc217ZGlzcGxheTpub25lIWltcG9ydGFu
dH19QG1lZGlhIChtaW4td2lkdGg6OTkycHgpIGFuZCAobWF4LXdpZHRoOjExOTlweCl7LmhpZGRl
bi1tZHtkaXNwbGF5Om5vbmUhaW1wb3J0YW50fX1AbWVkaWEgKG1pbi13aWR0aDoxMjAwcHgpey5o
aWRkZW4tbGd7ZGlzcGxheTpub25lIWltcG9ydGFudH19LnZpc2libGUtcHJpbnR7ZGlzcGxheTpu
b25lIWltcG9ydGFudH1AbWVkaWEgcHJpbnR7LnZpc2libGUtcHJpbnR7ZGlzcGxheTpibG9jayFp
bXBvcnRhbnR9dGFibGUudmlzaWJsZS1wcmludHtkaXNwbGF5OnRhYmxlIWltcG9ydGFudH10ci52
aXNpYmxlLXByaW50e2Rpc3BsYXk6dGFibGUtcm93IWltcG9ydGFudH10ZC52aXNpYmxlLXByaW50
LHRoLnZpc2libGUtcHJpbnR7ZGlzcGxheTp0YWJsZS1jZWxsIWltcG9ydGFudH19LnZpc2libGUt
cHJpbnQtYmxvY2t7ZGlzcGxheTpub25lIWltcG9ydGFudH1AbWVkaWEgcHJpbnR7LnZpc2libGUt
cHJpbnQtYmxvY2t7ZGlzcGxheTpibG9jayFpbXBvcnRhbnR9fS52aXNpYmxlLXByaW50LWlubGlu
ZXtkaXNwbGF5Om5vbmUhaW1wb3J0YW50fUBtZWRpYSBwcmludHsudmlzaWJsZS1wcmludC1pbmxp
bmV7ZGlzcGxheTppbmxpbmUhaW1wb3J0YW50fX0udmlzaWJsZS1wcmludC1pbmxpbmUtYmxvY2t7
ZGlzcGxheTpub25lIWltcG9ydGFudH1AbWVkaWEgcHJpbnR7LnZpc2libGUtcHJpbnQtaW5saW5l
LWJsb2Nre2Rpc3BsYXk6aW5saW5lLWJsb2NrIWltcG9ydGFudH19QG1lZGlhIHByaW50ey5oaWRk
ZW4tcHJpbnR7ZGlzcGxheTpub25lIWltcG9ydGFudH19LyojIHNvdXJjZU1hcHBpbmdVUkw9Ym9v
dHN0cmFwLm1pbi5jc3MubWFwICovPC9zdHlsZT48c3R5bGU+LyogR0xPQkFMICovaHRtbCxib2R5
IHtiYWNrZ3JvdW5kOiAjZjBmMGYwO292ZXJmbG93LXg6IGhpZGRlbjt9aDEge2ZvbnQtd2VpZ2h0
OiBib2xkO2xldHRlci1zcGFjaW5nOiAtM3B4O31oMyB7Zm9udC1zaXplOiAyMXB4O2xldHRlci1z
cGFjaW5nOiAtMXB4O30uaC1kYXNoYm9hcmQge3RleHQtdHJhbnNmb3JtOiBsb3dlcmNhc2U7fS5w
YWdlLWhlYWRlciB7Ym9yZGVyLWJvdHRvbTogMXB4IHNvbGlkIHJnYmEoMCwgMCwgMCwgMC4xNSk7
bWFyZ2luOiAyNXB4IDAgMjBweDtwb3NpdGlvbjogcmVsYXRpdmU7fS5wYWdlLWhlYWRlciBoMSB7
bWFyZ2luOiAwO30ucGFnaW5hdGlvbiB7bWFyZ2luOiA1cHggMDt9LmNsaWNrYWJsZSwuZXhwYW5k
YWJsZT50ZCB7Y3Vyc29yOiBwb2ludGVyO30uc3Bpbm5lciB7Y29sb3I6ICM5OTk7bGVmdDogNTAl
O3Bvc2l0aW9uOiBhYnNvbHV0ZTt0b3A6IDUwJTt9LnBvd2VyZWQge2JvdHRvbTogMTkwcHg7Y29s
b3I6ICM5RTlFOUU7Zm9udC1zaXplOiBzbWFsbGVyO3Bvc2l0aW9uOiBhYnNvbHV0ZTtyaWdodDog
MjBweDt0cmFuc2Zvcm0tb3JpZ2luOiAxMDAlIDA7dHJhbnNmb3JtOiByb3RhdGUoLTkwZGVnKTt9
LnBvd2VyZWQgYSB7Y29sb3I6ICM2MzYzNjM7fS5wb3dlcmVkIHNwYW4ge2NvbG9yOiAjMDA3YmMz
O30uZHJvcGRvd24taGVhZGVyIHtjb2xvcjogIzAwN2JjMztwYWRkaW5nOiAzcHggMjVweDt0ZXh0
LXRyYW5zZm9ybTogdXBwZXJjYXNlO30uZ2hlYWRlciB7bGV0dGVyLXNwYWNpbmc6IC0xcHg7dGV4
dC10cmFuc2Zvcm06IHVwcGVyY2FzZTt9aDUuZ2hlYWRlciB7bGV0dGVyLXNwYWNpbmc6IDA7fS5w
YW5lbC1oZWFkZXIgaDQuZ2hlYWRlciB7bWFyZ2luLXRvcDogMjBweDt9LnBhbmVsLWhlYWRlciAu
Z2hlYWRlciBzbWFsbCB7Zm9udC1zaXplOiA2OSU7fS8qIE5BVklHQVRJT04gKi9uYXYgey13ZWJr
aXQtdHJhbnNpdGlvbjogbGVmdCAuMnM7YmFja2dyb3VuZDogIzFDMUMxQztib3JkZXItcmlnaHQ6
IDNweCBzb2xpZCAjNWJjMGRlO2hlaWdodDogMTAwJTtsZWZ0OiAtMjM2cHg7b3ZlcmZsb3c6IGhp
ZGRlbjtwb3NpdGlvbjogZml4ZWQ7dG9wOiAwO3RyYW5zaXRpb246IGxlZnQgLjJzO3dpZHRoOiAz
MDBweDt6LWluZGV4OiAyO31uYXYgLm5hdi1saXN0IHtib3R0b206IDA7bGVmdDogMDtvdmVyZmxv
dy15OiBzY3JvbGw7cG9zaXRpb246IGFic29sdXRlO3JpZ2h0OiAtMTdweDt0b3A6IDA7fW5hdiBo
ZWFkZXIge21hcmdpbjogNDBweCAyMHB4IDMwcHg7fW5hdiBoZWFkZXIgYSB7Y29sb3I6IHJnYmEo
MjQwLDI0MCwyNDAsLjcpO2ZvbnQtc2l6ZTogMi43ZW07Zm9udC13ZWlnaHQ6IDMwMDt0ZXh0LXRy
YW5zZm9ybTogdXBwZXJjYXNlO31uYXYgaGVhZGVyIGE6aG92ZXIge2NvbG9yOiAjZWVlO31uYXYu
YWN0aXZlIHtkaXNwbGF5OiBibG9jayAhaW1wb3J0YW50O2xlZnQ6IDA7b3BhY2l0eTogLjk3O31u
YXY6aG92ZXIgfiAjY29udGVudCB7b3BhY2l0eTogLjM7fW5hdi5hY3RpdmUgLm5hdi1iYXJzLG5h
di5hY3RpdmUgLm5hdi1nZWFycyxuYXYuYWN0aXZlIC5uYXYtd3Mtc3RhdHVzIHtvcGFjaXR5OiAw
O31uYXYgLm5hdi1iYXJzLG5hdiAubmF2LWdlYXJzLG5hdiAubmF2LXdzLXN0YXR1cyB7LXdlYmtp
dC10cmFuc2l0aW9uOiBvcGFjaXR5IC4ycztjb2xvcjogIzlFOUU5RTtjdXJzb3I6IHBvaW50ZXI7
ZmxvYXQ6IHJpZ2h0O2ZvbnQtc2l6ZTogMzZweDtoZWlnaHQ6IDMycHg7bGVmdDogMTNweDtsaW5l
LWhlaWdodDogMzJweDtwb3NpdGlvbjogZml4ZWQ7dGV4dC1hbGlnbjogY2VudGVyO3RvcDogMzBw
eDt0cmFuc2l0aW9uOiBvcGFjaXR5IC4yczt3aWR0aDogMzJweDt9bmF2IC5uYXYtZ2VhcnMge3Rv
cDogMTAwcHg7b3BhY2l0eTogMC42O31uYXYgLm5hdi13cy1zdGF0dXMsLm5hdi13cy1zdGF0dXMu
bWluaSB7Y29sb3I6ICM2QTZBNkE7Y3Vyc29yOiBoZWxwO2Rpc3BsYXk6IG5vbmU7Zm9udC1zaXpl
OiAxMnB4O31uYXYgLm5hdi13cy1zdGF0dXMge2xlZnQ6IDI1cHg7dG9wOiAxMjVweDt9Lm5hdi13
cy1zdGF0dXMubWluaSB7dG9wOiAxNHB4O2xlZnQ6IDUwcHg7cG9zaXRpb246IGFic29sdXRlO30u
bmF2LXdzLXN0YXR1cy5jb25uZWN0ZWQge2NvbG9yOiAjNURCNTZBO31uYXYgbGkge3Bvc2l0aW9u
OiByZWxhdGl2ZTt9bmF2IGxpIC50b2dnbGUtcGFuZWwge2N1cnNvcjogcG9pbnRlcjtvcGFjaXR5
OiAwO3BhZGRpbmc6IDlweCAyMHB4O3Bvc2l0aW9uOiBhYnNvbHV0ZTtyaWdodDogMDt0b3A6IDA7
dHJhbnNpdGlvbjogYWxsIC4yczt2aXNpYmlsaXR5OiBoaWRkZW47fW5hdiBsaSAudG9nZ2xlLXBh
bmVsIGkge2NvbG9yOiByZ2JhKDIwMCwyMDAsMjAwLC41KTtvcGFjaXR5OiAwO31uYXYgbGkgLnRv
Z2dsZS1wYW5lbC5hY3RpdmUgaSB7Y29sb3I6ICNlZWU7b3BhY2l0eTogMTt9bmF2LmFjdGl2ZSBs
aSAudG9nZ2xlLXBhbmVsIHt2aXNpYmlsaXR5OiB2aXNpYmxlO29wYWNpdHk6IDE7fW5hdi5hY3Rp
dmUgbGk6aG92ZXIgLnRvZ2dsZS1wYW5lbCBpIHtvcGFjaXR5OiAxO31uYXYgbGkgYSB7Ym9yZGVy
LWxlZnQ6IDNweCBzb2xpZCB0cmFuc3BhcmVudDtjb2xvcjogcmdiYSgyMDAsMjAwLDIwMCwuNSk7
ZGlzcGxheTogYmxvY2s7Zm9udC1zaXplOiBzbWFsbGVyO21heC13aWR0aDogMjM1cHg7b3BhY2l0
eTogMDtvdmVyZmxvdzogaGlkZGVuO3BhZGRpbmc6IDlweCAyMHB4O3RleHQtb3ZlcmZsb3c6IGVs
bGlwc2lzO3RleHQtdHJhbnNmb3JtOiB1cHBlcmNhc2U7dHJhbnNpdGlvbjogb3BhY2l0eSAuMnM7
d2hpdGUtc3BhY2U6IG5vd3JhcDt9bmF2LmFjdGl2ZSBsaSBhIHttYXgtd2lkdGg6IDkwJTtvcGFj
aXR5OiAxO31uYXYgbGk6aG92ZXIgYSxuYXYgbGkuYWN0aXZlIGEge2JhY2tncm91bmQ6IHJnYmEo
MCwwLDAsLjEpO2JvcmRlci1jb2xvcjogIzVCQzBERTtjb2xvcjogI2VlZTt9bmF2IHVsIHtwYWRk
aW5nLWxlZnQ6IDA7bGlzdC1zdHlsZTogbm9uZTt9LyogTmF2aWdhdGlvbiAgLS0gSWNvbiAqL25h
diBhLG5hdiBhOmhvdmVyIHt0ZXh0LWRlY29yYXRpb246IG5vbmU7fW5hdiBoMyB7Y29sb3I6ICNG
RkYgIWltcG9ydGFudDtmb250LXNpemU6IG1lZGl1bTtmb250LXdlaWdodDogYm9sZDttYXJnaW46
IDIwcHggMjVweCAxMHB4O3RleHQtdHJhbnNmb3JtOiB1cHBlcmNhc2U7fS8qIENPTlRBSU5FUiAq
L0BtZWRpYSBzY3JlZW4gYW5kIChtYXgtd2lkdGg6IDc2N3B4KSB7LnJvdy1vZmZjYW52YXMgey13
ZWJraXQtdHJhbnNpdGlvbjogYWxsIC4yNXMgZWFzZS1vdXQ7LW8tdHJhbnNpdGlvbjogYWxsIC4y
NXMgZWFzZS1vdXQ7cG9zaXRpb246IHJlbGF0aXZlO3RyYW5zaXRpb246IGFsbCAuMjVzIGVhc2Ut
b3V0O30ucm93LW9mZmNhbnZhcy1yaWdodCB7cmlnaHQ6IDA7fS5yb3ctb2ZmY2FudmFzLWxlZnQg
e2xlZnQ6IDA7fS5yb3ctb2ZmY2FudmFzLXJpZ2h0LnNpZGViYXItb2ZmY2FudmFzIHtyaWdodDog
LTUwJTt9LnJvdy1vZmZjYW52YXMtbGVmdC5zaWRlYmFyLW9mZmNhbnZhcyB7bGVmdDogLTUwJTt9
LnJvdy1vZmZjYW52YXMtcmlnaHQuYWN0aXZlIHtyaWdodDogNTAlO30ucm93LW9mZmNhbnZhcy1s
ZWZ0LmFjdGl2ZSB7bGVmdDogNTAlO30uc2lkZWJhci1vZmZjYW52YXMge3Bvc2l0aW9uOiBhYnNv
bHV0ZTt0b3A6IDA7d2lkdGg6IDUwJTt9O31AbWVkaWEgKG1pbi13aWR0aDogNzY4cHgpIHsuY29u
dGFpbmVyIHt3aWR0aDogNzUwcHg7fTt9QG1lZGlhIChtYXgtd2lkdGg6IDQ4MHB4KSB7LndyYXAt
Z2VuZXJhbCBoNSwud3JhcC1wYW5lbCBoNSB7d2hpdGUtc3BhY2U6IG5vd3JhcDtvdmVyZmxvdzog
aGlkZGVuO3RleHQtb3ZlcmZsb3c6IGVsbGlwc2lzO30ud3JhcC1nZW5lcmFsIGg1IHt3aWR0aDog
MTAwJX0ud3JhcC1wYW5lbCBoNSB7d2lkdGg6IDcwJX19LmNvbnRhaW5lci1mbHVpZCB7bWFyZ2lu
LWxlZnQ6IDc1cHg7fUBtZWRpYSAobWluLXdpZHRoOiAxMTIwcHgpIHsuY29udGFpbmVyIHt3aWR0
aDogOTcwcHg7fTt9QG1lZGlhIChtaW4td2lkdGg6IDEzMjBweCkgey5jb250YWluZXIge3dpZHRo
OiAxMTcwcHg7fTt9QG1lZGlhIChtYXgtd2lkdGg6IDk5MnB4KSB7LmNvbnRhaW5lci1mbHVpZCB7
bWFyZ2luLWxlZnQ6IGF1dG87fTt9QG1lZGlhIChtYXgtd2lkdGg6IDc2OHB4KSB7LmNvbnRhaW5l
ci1mbHVpZCB7cGFkZGluZy1sZWZ0OiA1cHg7cGFkZGluZy1yaWdodDogNXB4O30ucGFnZS1oZWFk
ZXIge3BhZGRpbmc6IDAgMTBweDt9fS8qIFBBTkVMIFNUWUxFUyAqLy53cmFwLXBhbmVsIC5wYW5l
bC1oZWFkZXIge3Bvc2l0aW9uOiByZWxhdGl2ZTt9ZGl2LndyYXAtcGFuZWwgPiBkaXYge2JhY2tn
cm91bmQ6ICNGRkY7bWFyZ2luLXRvcDogMTBweDtwYWRkaW5nOiAwIDEwcHg7Ym9yZGVyLXRvcDog
MXB4IHNvbGlkIHJnYmEoMCwgMCwgMCwgMC4xNSk7fS8qIFBBTkVMIFRBQkxFUyAqLy53cmFwLXBh
bmVsIHRhYmxlLnRhYmxlLWJvcmRlcmxlc3MgdGJvZHkgdHIgdGQsLndyYXAtcGFuZWwgdGFibGUu
dGFibGUtYm9yZGVybGVzcyB0Ym9keSB0ciB0aCwud3JhcC1wYW5lbCB0YWJsZS50YWJsZS1ib3Jk
ZXJsZXNzIHRoZWFkIHRyIHRoIHtib3JkZXI6IG5vbmU7fS53cmFwLXBhbmVsIHRhYmxlIHRoZWFk
IHRyIHRoIHt0ZXh0LWFsaWduOiByaWdodDtib3JkZXItYm90dG9tLXdpZHRoOiAxcHg7fS53cmFw
LXBhbmVsIHRhYmxlIC5zdHJpbmcsLndyYXAtcGFuZWwgdGFibGUgLmRhdGUge3RleHQtYWxpZ246
IGxlZnQ7fS53cmFwLXBhbmVsIHRhYmxlIC5wZXJjZW50IHtjb2xvcjogIzg5ODk4OTt9LndyYXAt
cGFuZWwgdGFibGUgdGQsLndyYXAtcGFuZWwgdGFibGUgdGgge3doaXRlLXNwYWNlOiBub3dyYXA7
b3ZlcmZsb3c6IGhpZGRlbjt9LndyYXAtcGFuZWwgdGFibGUgdGguc29ydGFibGUge2N1cnNvcjog
cG9pbnRlcjt9LndyYXAtcGFuZWwgdGFibGUudGFibGUtYm9yZGVybGVzcyB0aGVhZD50ci50aGVh
ZC1jb2xzIHRoIHtmb250LXNpemU6IDc4JTt0ZXh0LXRyYW5zZm9ybTogdXBwZXJjYXNlO30ud3Jh
cC1wYW5lbCB0YWJsZSAuY2VsbC1obCB7cGFkZGluZzogMnB4IDNweDtjb2xvcjogI0ZGRjtib3Jk
ZXItcmFkaXVzOiA1cHg7ZGlzcGxheTogYmxvY2s7dGV4dC1hbGlnbjogY2VudGVyO30ud3JhcC1w
YW5lbCB0YWJsZSAuc3Bhbi1obCB7cGFkZGluZzogMnB4IDNweDtib3JkZXItcmFkaXVzOiAzcHg7
Y29sb3I6ICMwMDA7fS53cmFwLXBhbmVsIHRhYmxlIC5zcGFuLWhsLmc1IHtiYWNrZ3JvdW5kOiAj
ZTllY2VmO30ud3JhcC1wYW5lbCB0YWJsZSAuY2VsbC1obC5iMSB7YmFja2dyb3VuZDogIzdGNjY5
RDt9LndyYXAtcGFuZWwgdGFibGUgLmNlbGwtaGwuYjIge2JhY2tncm91bmQ6ICNCQTk0RDE7fS53
cmFwLXBhbmVsIHRhYmxlIC5jZWxsLWhsLmIzIHtiYWNrZ3JvdW5kOiAjREVCQUNFO30ud3JhcC1w
YW5lbCB0YWJsZSAuY2VsbC1obC5kMSB7YmFja2dyb3VuZDogIzlkOWQ5ZDM4O30ud3JhcC1wYW5l
bCB0YWJsZSAuY2VsbC1obC5kMiB7YmFja2dyb3VuZDogIzlkOWQ5ZDYxO30ud3JhcC1wYW5lbCB0
YWJsZSAuY2VsbC1obC5kMyB7YmFja2dyb3VuZDogIzlkOWQ5ZDljO30ud3JhcC1wYW5lbCB0YWJs
ZSAuY2VsbC1obC5kNCB7YmFja2dyb3VuZDogIzlkOWQ5ZDt9LndyYXAtcGFuZWwgdGFibGUgLnNw
YW4taGwubGdybiB7YmFja2dyb3VuZDogI2U2ZjRlYTtjb2xvcjogIzEzNzMzMzt9LndyYXAtcGFu
ZWwgdGFibGUgLnNwYW4taGwubHllbCB7YmFja2dyb3VuZDogI2ZmZjNjZDtjb2xvcjogI2QzOGEx
MDt9LndyYXAtcGFuZWwgdGFibGUgLnNwYW4taGwubHJlZCB7YmFja2dyb3VuZDogI2ZjZThlNjtj
b2xvcjogI2M1MjIxZjt9LndyYXAtcGFuZWwgdGFibGUgLnNwYW4taGwubGJsdSB7YmFja2dyb3Vu
ZDogI2NmZTJmZjtjb2xvcjogIzA1MmM2NTt9LndyYXAtcGFuZWwgdGFibGUgLnNwYW4taGwubHBy
cCB7YmFja2dyb3VuZDogI2NkYzdmZjtjb2xvcjogIzM0MzE1MDt9LyogdGhlYWQgbWV0YSAqLy53
cmFwLXBhbmVsIHRhYmxlIHRoZWFkPnRyLnRoZWFkLW1pbiB0aC5tZXRhLWxhYmVsLC53cmFwLXBh
bmVsIHRhYmxlIHRoZWFkPnRyLnRoZWFkLWF2ZyB0aC5tZXRhLWxhYmVsLC53cmFwLXBhbmVsIHRh
YmxlIHRmb290PnRyLnRmb290LXRvdGFscyB0aC5tZXRhLWxhYmVsLC53cmFwLXBhbmVsIHRhYmxl
IHRoZWFkPnRyLnRoZWFkLW1heCB0aC5tZXRhLWxhYmVsIHtmb250LXdlaWdodDogYm9sZDt0ZXh0
LXRyYW5zZm9ybTogdXBwZXJjYXNlO30ud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtbWluLC53cmFw
LXBhbmVsIHRhYmxlIC50aGVhZC1hdmcsLndyYXAtcGFuZWwgdGFibGUgLnRoZWFkLW1heCB7YmFj
a2dyb3VuZDogI0Y4RjhGODt9LndyYXAtcGFuZWwgdGFibGUgLnRoZWFkLWF2ZyB7Ym9yZGVyLWJv
dHRvbTogMnB4IHNvbGlkICMwMDA7fS53cmFwLXBhbmVsIHRhYmxlIHRoZWFkPnRyLnRoZWFkLW1p
biB0aCwud3JhcC1wYW5lbCB0YWJsZSB0aGVhZD50ci50aGVhZC1hdmcgdGgsLndyYXAtcGFuZWwg
dGFibGUgdGhlYWQ+dHIudGhlYWQtbWF4IHRoIHtmb250LXNpemU6IHNtYWxsZXI7Zm9udC13ZWln
aHQ6IG5vcm1hbDtwYWRkaW5nOiAzcHggOHB4IDNweCA4cHg7dGV4dC10cmFuc2Zvcm06IGluaGVy
aXQ7fS53cmFwLXBhbmVsIHRhYmxlIC50aGVhZC1taW4gdGgsLndyYXAtcGFuZWwgdGFibGUgLnRo
ZWFkLWF2ZyB0aCwud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtbWF4IHRoIHtwYWRkaW5nOiAzcHg7
fS53cmFwLXBhbmVsIHRhYmxlIHRmb290PnRyPnRoIHtib3JkZXItdG9wOiAxcHggZG90dGVkICMw
MDA7cGFkZGluZzogOHB4O30vKiB0aGVhZCBkYXRhICovLndyYXAtcGFuZWwgdGFibGUgdGJvZHku
dGJvZHktZGF0YSB0ciB0ZCB7Ym9yZGVyLXJpZ2h0OiAxcHggc29saWQgI0YxRjFGMTtmb250LXNp
emU6IHNtYWxsZXI7fS53cmFwLXBhbmVsIHRhYmxlIHRib2R5LnRib2R5LWRhdGEgdGQ6bGFzdC1j
aGlsZCB7Ym9yZGVyLXJpZ2h0OiBub25lO30ud3JhcC1wYW5lbCB0YWJsZSB0Ym9keS50Ym9keS1k
YXRhIHRkLnJvdy1pZHgge2ZvbnQtd2VpZ2h0OiA3MDA7fS53cmFwLXBhbmVsIHRhYmxlPnRoZWFk
PnRyLnRoZWFkLWNvbHMge2JvcmRlci1ib3R0b206IDJweCBzb2xpZCAjMjIyO30ud3JhcC1wYW5l
bCB0YWJsZSB0Ym9keS50Ym9keS1kYXRhIHRyLnNoYWRlZCB7YmFja2dyb3VuZC1jb2xvcjogI0Y3
RjdGNzt9LndyYXAtcGFuZWwgdGFibGUgdGJvZHkudGJvZHktZGF0YSB0ci4ge2JhY2tncm91bmQt
Y29sb3I6ICNGN0Y3Rjc7fS53cmFwLXBhbmVsIHRhYmxlIHRib2R5LnRib2R5LWRhdGEgdHIuY2hp
bGQgdGQ6bnRoLWNoaWxkKDEpLC53cmFwLXBhbmVsIHRhYmxlIHRib2R5LnRib2R5LWRhdGEgdHIu
Y2hpbGQgdGQ6bnRoLWNoaWxkKDIpIHtib3JkZXItcmlnaHQ6IG5vbmU7fS53cmFwLXBhbmVsIHRh
YmxlLnRhYmxlLWhvdmVyPnRib2R5PnRyOmhvdmVyIHtiYWNrZ3JvdW5kLWNvbG9yOiAjRUVFO30u
d3JhcC1wYW5lbCAucm93Om50aC1jaGlsZCgyKTpub3QoLnRhYmxlLXdyYXBwZXIpPi5jb2wtbWQt
MTIge3BhZGRpbmc6IDA7fS8qIEdFTkVSQUwgKi8ud3JhcC1nZW5lcmFsIHtwb3NpdGlvbjogcmVs
YXRpdmU7fS5yZXBvcnQtdGl0bGUge2JhY2tncm91bmQ6ICNGRkY7Ym9yZGVyLXJhZGl1czogNHB4
O2JvdHRvbTogLTEwcHg7Y29sb3I6ICM5RTlFOUU7Zm9udC1zaXplOiBzbWFsbDtwYWRkaW5nOiAw
IDEwcHg7cG9zaXRpb246IGFic29sdXRlO3JpZ2h0OiAwO3otaW5kZXg6IDE7fS5wYW5lbC1wbG90
LXdyYXAge3Bvc2l0aW9uOiBhYnNvbHV0ZTtyaWdodDogMDt0b3A6IDE4cHg7fS5jb2wtdGl0bGUg
e2ZvbnQtc2l6ZTogODUlO292ZXJmbG93OiBoaWRkZW47dGV4dC1vdmVyZmxvdzogZWxsaXBzaXM7
dGV4dC1zaGFkb3c6IDFweCAxcHggMCAjRkZGO3doaXRlLXNwYWNlOiBub3dyYXA7d2lkdGg6IDEw
MCU7fS5ncmlkLW1vZHVsZSB7YmFja2dyb3VuZDogI0ZGRjtjb2xvcjogcmdiKDM2LCAzNiwgMzYp
O2ZvbnQtd2VpZ2h0OiBub3JtYWw7bWFyZ2luLXRvcDogNXB4O3BhZGRpbmc6IDdweDt9LmdyaWQt
bW9kdWxlIGgzIHtmb250LXNpemU6IDI1cHg7bWFyZ2luOiAwO292ZXJmbG93OiBoaWRkZW47dGV4
dC1vdmVyZmxvdzogZWxsaXBzaXM7d2hpdGUtc3BhY2U6IG5vd3JhcDt3aWR0aDogMTAwJTt9Lmdy
aWQtbW9kdWxlLmJsYWNrIHtib3JkZXItdG9wOiA0cHggc29saWQgIzBGMTIxNDt9LmdyaWQtbW9k
dWxlLmdyYXkge2JvcmRlci10b3A6IDRweCBzb2xpZCAjOUU5RTlFO30uZ3JpZC1tb2R1bGUucmVk
IHtib3JkZXItdG9wOiA0cHggc29saWQgI0ZGMzAzRTt9LmdyaWQtbW9kdWxlLmJsdWV7Ym9yZGVy
LXRvcDogNHB4IHNvbGlkICMwMEQ0RTE7fS5ncmlkLW1vZHVsZS5ncmVlbiB7Ym9yZGVyLXRvcDog
NHB4IHNvbGlkICMyMjlmNzU7fUBtZWRpYSAobWF4LXdpZHRoOiA3NjdweCkgey5wYW5lbC1wbG90
LXdyYXAge3RvcDogMTBweDt9LnBvd2VyZWQge2JvdHRvbTogMTBweDtsZWZ0OiAyNXB4O3RyYW5z
Zm9ybTogaW5pdGlhbDt9fS8qIENIQVJUUyAqLy5jaGFydC13cmFwIHttYXJnaW4tYm90dG9tOiAx
NXB4O3Bvc2l0aW9uOiByZWxhdGl2ZTt9c3ZnIHtiYWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3BhcmVu
dDtkaXNwbGF5OiBibG9jazt9LmF4aXMgcGF0aCB7ZmlsbDogdHJhbnNwYXJlbnQ7c3Ryb2tlOiBi
bGFjaztzaGFwZS1yZW5kZXJpbmc6IGNyaXNwRWRnZXM7c3Ryb2tlLXdpZHRoOiAxO30uZ3JpZC55
IC50aWNrIGxpbmUsLmdyaWQueCAudGljayBsaW5lIHtzaGFwZS1yZW5kZXJpbmc6IGNyaXNwRWRn
ZXM7c3Ryb2tlOiAjOTk5O3N0cm9rZS1kYXNoYXJyYXk6IDMgMztzdHJva2Utd2lkdGg6IDE7fS5h
eGlzLnggLnRpY2sgbGluZSwuYXhpcy55MCAudGljayBsaW5lLC5heGlzLnkxIC50aWNrIGxpbmUs
LmdyaWQueSAudGljazpmaXJzdC1jaGlsZCBsaW5lIHtzdHJva2U6IGJsYWNrO3N0cm9rZS13aWR0
aDogMTtzaGFwZS1yZW5kZXJpbmc6IGNyaXNwRWRnZXM7fS5iYXJzIHJlY3QuYmFyIHtzaGFwZS1y
ZW5kZXJpbmc6IGNyaXNwRWRnZXM7fS5yZWN0cyByZWN0IHtmaWxsOiB0cmFuc3BhcmVudDt9LmFy
ZWEge29wYWNpdHk6IDAuMjt9LnBvaW50cyB7c3Ryb2tlOiB0cmFuc3BhcmVudDt9bGluZS5pbmRp
Y2F0b3Ige2ZpbGw6IHRyYW5zcGFyZW50O3BvaW50ZXItZXZlbnRzOiBub25lO3NoYXBlLXJlbmRl
cmluZzogY3Jpc3BFZGdlcztzdHJva2U6ICM5OTk7c3Ryb2tlLXdpZHRoOiAxO2Rpc3BsYXk6IG5v
bmU7fS5hcmVhMCwuYmFycy55MCAuYmFyLC5wb2ludHMueTAscmVjdC5sZWdlbmQueTAge2ZpbGw6
ICM0NDdGQjM7fS5hcmVhMSwuYmFycy55MSAuYmFyLC5wb2ludHMueTEscmVjdC5sZWdlbmQueTEg
e2ZpbGw6ICNGRjY4NTQ7fS5saW5lMCwubGluZTEge2ZpbGw6IHRyYW5zcGFyZW50O3N0cm9rZS13
aWR0aDogMTt9LmxpbmUwIHtzdHJva2U6ICMwMDdCQzM7fS5saW5lMSB7c3Ryb2tlOiAjRkYzMDNF
O30uYXhpcyB0ZXh0LC5heGlzLWxhYmVsLHRleHQubGVnZW5kIHtmb250OiAxMHB4IHNhbnMtc2Vy
aWY7fS5heGlzLWxhYmVsLnkwLC5heGlzLWxhYmVsLnkxIHt0ZXh0LWFuY2hvcjogZW5kO31yZWN0
LmxlZ2VuZCB7aGVpZ2h0OiAxMHB4O3dpZHRoOiAxMHB4O30ubGVnZW5kIHtjdXJzb3I6IHBvaW50
ZXI7fS53cmFwLXRleHQgdGV4dCB7dGV4dC1hbmNob3I6IHN0YXJ0IWltcG9ydGFudDt9LyogQ0hB
UlQgVE9PTFRJUCAqLy5jaGFydC10b29sdGlwLXdyYXAge2xlZnQ6IDA7cG9pbnRlci1ldmVudHM6
IG5vbmU7cG9zaXRpb246IGFic29sdXRlO3RvcDogMTBweDt6LWluZGV4OiAxMDt9LmNoYXJ0LXRv
b2x0aXAgey1tb3otYm94LXNoYWRvdzogN3B4IDdweCAxMnB4IC05cHggIzc3Nzc3Nzstd2Via2l0
LWJveC1zaGFkb3c6IDdweCA3cHggMTJweCAtOXB4ICM3Nzc3Nzc7YmFja2dyb3VuZC1jb2xvcjog
I2ZmZjtib3JkZXItY29sbGFwc2U6IGNvbGxhcHNlO2JvcmRlci1zcGFjaW5nOiAwO2JveC1zaGFk
b3c6IDdweCA3cHggMTJweCAtOXB4ICM3Nzc3Nzc7ZW1wdHktY2VsbHM6IHNob3c7b3BhY2l0eTog
MC45O30uY2hhcnQtdG9vbHRpcCB0ciB7Ym9yZGVyOiAxcHggc29saWQgI0NDQzt9LmNoYXJ0LXRv
b2x0aXAgdGgge2JhY2tncm91bmQtY29sb3I6ICNhYWE7Y29sb3I6ICNGRkY7Zm9udC1zaXplOiAx
NHB4O21heC13aWR0aDogMzgwcHg7b3ZlcmZsb3c6IGhpZGRlbjtwYWRkaW5nOiAycHggNXB4O3Rl
eHQtYWxpZ246IGxlZnQ7dGV4dC1vdmVyZmxvdzogZWxsaXBzaXM7d2hpdGUtc3BhY2U6IG5vd3Jh
cDt9LmNoYXJ0LXRvb2x0aXAgdGQge2JvcmRlci1sZWZ0OiAxcHggZG90dGVkICM5OTk7Zm9udC1z
aXplOiAxM3B4O3BhZGRpbmc6IDNweCA2cHg7fS5jaGFydC10b29sdGlwIHRkID4gc3BhbiB7ZGlz
cGxheTogaW5saW5lLWJsb2NrO2hlaWdodDogMTBweDttYXJnaW4tcmlnaHQ6IDZweDt3aWR0aDog
MTBweDt9LmNoYXJ0LXRvb2x0aXAgdGQudmFsdWUge3RleHQtYWxpZ246IHJpZ2h0O30uY2hhcnQt
dG9vbHRpcCAuYmx1ZSB7YmFja2dyb3VuZC1jb2xvcjogIzAwN0JDMzt9LmNoYXJ0LXRvb2x0aXAg
LnJlZCB7YmFja2dyb3VuZC1jb2xvcjogI0ZGMzAzRTt9LyogREFSSyBUSEVNRSAqLy5kYXJrIGgx
IHtjb2xvcjogcmdiYSgyNTUsIDI1NSwgMjU1LCAwLjYpO30uZGFyayBoMywuZGFyayBoNCwuZGFy
ayBoNSB7Y29sb3I6IHJnYmEoMjU1LDI1NSwyNTUsMC40KTt9LmRhcmsgLnRhYmxlLXJlc3BvbnNp
dmUge2JvcmRlcjogbm9uZTt9LmRhcmsgLndyYXAtcGFuZWwgPiBkaXYgPiB0YWJsZSB7Y29sb3I6
ICNEMkQyRDI7fS5kYXJrIC53cmFwLXBhbmVsIHRhYmxlIHRib2R5LnRib2R5LWRhdGEgdHIgdGQg
e2JvcmRlci1yaWdodDogbm9uZTt9LmRhcmsgLndyYXAtcGFuZWwgdGFibGUudGFibGUtaG92ZXI+
dGJvZHkudGJvZHktZGF0YT50cjpob3ZlciB7YmFja2dyb3VuZC1jb2xvcjogcmdiYSgyNTUsIDI1
NSwgMjU1LCAwLjA4KSAhaW1wb3J0YW50O30uZGFyayAuY29sLXRpdGxlIHtjb2xvcjogIzllOWU5
ZTt0ZXh0LXNoYWRvdzpub25lO30uZGFyayAuZ3JpZC1tb2R1bGUgaDMge2NvbG9yOiAjRkZGO30u
ZGFyayAuZHJvcGRvd24tbWVudT5saT5hIHtjb2xvcjogI0ZGRjt9LmRhcmsgZGl2LndyYXAtcGFu
ZWwgPiBkaXYge2NvbG9yOiAjRUVFO21hcmdpbi10b3A6IDEwcHg7cGFkZGluZzogMCAxMHB4O2Jv
cmRlci10b3A6IDFweCBzb2xpZCByZ2JhKDI1NSwgMjU1LCAyNTUsIDAuMTUpO30uZGFyayAud3Jh
cC1wYW5lbCB0YWJsZSAuY2VsbC1obC5kMSB7YmFja2dyb3VuZDogIzE2MTYxNjt9LmRhcmsgLndy
YXAtcGFuZWwgdGFibGUgLmNlbGwtaGwuZDIge2JhY2tncm91bmQ6ICMzYzNjM2M7fS5kYXJrIC53
cmFwLXBhbmVsIHRhYmxlIC5jZWxsLWhsLmQzIHtiYWNrZ3JvdW5kOiAjNWE1YTVhO30uZGFyayAu
d3JhcC1wYW5lbCB0YWJsZSAuY2VsbC1obC5kNCB7YmFja2dyb3VuZDogIzdlN2U3ZTt9LyogREFS
SyBCTFVFIFRIRU1FICovaHRtbC5kYXJrLmJsdWUsLmRhcmsuYmx1ZSBib2R5IHtiYWNrZ3JvdW5k
OiAjMjUyQjMwO30uZGFyay5ibHVlIC5jb250YWluZXIge2JhY2tncm91bmQ6ICMyNTJCMzA7fS5k
YXJrLmJsdWUgLnBhZ2UtaGVhZGVyIHtib3JkZXItYm90dG9tOiAxcHggc29saWQgIzNCNDQ0Qzt9
LmRhcmsuYmx1ZSAubGFiZWwtaW5mbyB7YmFja2dyb3VuZC1jb2xvcjogIzI1MkIzMDt9LmRhcmsu
Ymx1ZSBuYXYge2JvcmRlci1yaWdodDogMXB4IHNvbGlkICMxODFCMUY7YmFja2dyb3VuZDogIzFG
MjMyODt9LmRhcmsuYmx1ZSBkaXYud3JhcC1wYW5lbCA+IGRpdiB7YmFja2dyb3VuZDogIzFGMjMy
ODt9LmRhcmsuYmx1ZSAud3JhcC1wYW5lbCB0YWJsZSB0Zm9vdD50cj50aCB7Ym9yZGVyLXRvcDog
MXB4IGRvdHRlZCAjOTk5O30uZGFyay5ibHVlIC53cmFwLXBhbmVsIHRhYmxlIC50aGVhZC1taW4s
LmRhcmsuYmx1ZSAud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtYXZnLC5kYXJrLmJsdWUgLndyYXAt
cGFuZWwgdGFibGUgLnRoZWFkLW1heCB7YmFja2dyb3VuZDogIzFmMjMyODt9LmRhcmsuYmx1ZSAu
d3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtYXZnIHtib3JkZXItYm90dG9tOiAycHggc29saWQgIzk5
OTt9LmRhcmsuYmx1ZSAud3JhcC1wYW5lbCB0YWJsZT50aGVhZD50ci50aGVhZC1jb2xzIHtib3Jk
ZXItYm90dG9tOiAycHggc29saWQgIzk5OTt9LmRhcmsuYmx1ZSAud3JhcC1wYW5lbCB0YWJsZSB0
Ym9keS50Ym9keS1kYXRhIHRyLnNoYWRlZCB7YmFja2dyb3VuZC1jb2xvcjogIzE4MUIxRjt9LmRh
cmsuYmx1ZSAuZ3JheSB7Ym9yZGVyLXRvcDogNHB4IHNvbGlkICMzQjQ0NEM7fS5kYXJrLmJsdWUg
LmdyaWQtbW9kdWxlIHtiYWNrZ3JvdW5kOiAjMUYyMzI4O30uZGFyay5ibHVlIC5idG4tZGVmYXVs
dCB7Y29sb3I6ICM5RTlFOUU7YmFja2dyb3VuZC1jb2xvcjogIzFGMjMyODtib3JkZXItY29sb3I6
ICMzQjQ0NEM7fS5kYXJrLmJsdWUgLmJ0bi1kZWZhdWx0OmFjdGl2ZSwuZGFyay5ibHVlIC5idG4t
ZGVmYXVsdDpob3ZlciwuZGFyay5ibHVlIC5idG4tZGVmYXVsdC5hY3RpdmUsLmRhcmsuYmx1ZSAu
b3Blbj4uZHJvcGRvd24tdG9nZ2xlLmJ0bi1kZWZhdWx0IHtjb2xvcjogIzNCNDQ0QztiYWNrZ3Jv
dW5kLWNvbG9yOiAjMUYyMzI4O2JvcmRlci1jb2xvcjogIzBGMTIxNDt9LmRhcmsuYmx1ZSAucGFn
aW5hdGlvbj4uZGlzYWJsZWQ+YSwuZGFyay5ibHVlIC5wYWdpbmF0aW9uPi5kaXNhYmxlZD5hOmhv
dmVyLC5kYXJrLmJsdWUgLnBhZ2luYXRpb24+LmRpc2FibGVkPmE6Zm9jdXMge2NvbG9yOiAjNzc3
O30uZGFyay5ibHVlIC5wYWdpbmF0aW9uPmxpPmEge2JhY2tncm91bmQtY29sb3I6ICMxRjIzMjg7
Ym9yZGVyOiAxcHggc29saWQgIzNCNDQ0Qzt9LmRhcmsuYmx1ZSAucGFnaW5hdGlvbj5saT5hOmhv
dmVyLC5kYXJrLmJsdWUgLnBhZ2luYXRpb24+bGk+YTphY3RpdmUsLmRhcmsuYmx1ZSAucGFnaW5h
dGlvbj5saT5hOmZvY3VzIHtjb2xvcjogIzAzNzBCMDtiYWNrZ3JvdW5kLWNvbG9yOiAjMUYyMzI4
O2JvcmRlci1jb2xvcjogIzNCNDQ0Qzt9LmRhcmsuYmx1ZSAuZHJvcGRvd24tbWVudT5saT5hOmhv
dmVyLC5kYXJrLmJsdWUgLmRyb3Bkb3duLW1lbnU+bGk+YTpmb2N1cyB7Y29sb3I6ICNGRkY7YmFj
a2dyb3VuZC1jb2xvcjogIzNCNDQ0Qzt9LmRhcmsuYmx1ZSAuZHJvcGRvd24tbWVudSB7YmFja2dy
b3VuZC1jb2xvcjogIzI1MkIzMDt9LmRhcmsuYmx1ZTo6LXdlYmtpdC1zY3JvbGxiYXItdHJhY2ss
LmRhcmsuYmx1ZSAudGFibGUtcmVzcG9uc2l2ZTo6LXdlYmtpdC1zY3JvbGxiYXItdHJhY2sgey13
ZWJraXQtYm94LXNoYWRvdzogaW5zZXQgMCAwIDZweCByZ2JhKDAsMCwwLDAuMyk7YmFja2dyb3Vu
ZC1jb2xvcjogIzlFOUU5RTt9LmRhcmsuYmx1ZTo6LXdlYmtpdC1zY3JvbGxiYXIsLmRhcmsuYmx1
ZSAudGFibGUtcmVzcG9uc2l2ZTo6LXdlYmtpdC1zY3JvbGxiYXIge3dpZHRoOiAxMHB4O2hlaWdo
dDogMTBweDtiYWNrZ3JvdW5kLWNvbG9yOiAjOUU5RTlFO30uZGFyay5ibHVlOjotd2Via2l0LXNj
cm9sbGJhci10aHVtYiwuZGFyay5ibHVlIC50YWJsZS1yZXNwb25zaXZlOjotd2Via2l0LXNjcm9s
bGJhci10aHVtYiB7YmFja2dyb3VuZC1jb2xvcjogIzNCNDQ0Qzt9LmRhcmsuYmx1ZSAuY2hhcnQt
dG9vbHRpcCB7YmFja2dyb3VuZC1jb2xvcjogIzI1MkIzMDt9LmRhcmsuYmx1ZSAucmVwb3J0LXRp
dGxlIHtiYWNrZ3JvdW5kOiAjMUYyMzI4O30vKiBEQVJLIEdSRVkgVEhFTUUgKi9odG1sLmRhcmsu
Z3JheSwuZGFyay5ncmF5IGJvZHkge2JhY2tncm91bmQ6ICMyMTIxMjE7fS5kYXJrLmdyYXkgLmNv
bnRhaW5lciB7YmFja2dyb3VuZDogIzIxMjEyMTt9LmRhcmsuZ3JheSAucGFnZS1oZWFkZXIge2Jv
cmRlci1ib3R0b206IDFweCBzb2xpZCAjMzAzMDMwO30uZGFyay5ncmF5IC5sYWJlbC1pbmZvIHti
YWNrZ3JvdW5kLWNvbG9yOiAjMzAzMDMwO30uZGFyay5ncmF5IG5hdiB7Ym9yZGVyLXJpZ2h0OiAx
cHggc29saWQgIzM2MzczNztiYWNrZ3JvdW5kOiAjMUMxQzFDO30uZGFyay5ncmF5IGRpdi53cmFw
LXBhbmVsID4gZGl2IHtiYWNrZ3JvdW5kOiAjMUMxQzFDO30uZGFyay5ncmF5IC53cmFwLXBhbmVs
IHRhYmxlIHRmb290PnRyPnRoIHtib3JkZXItdG9wOiAxcHggZG90dGVkICM5OTk7fS5kYXJrLmdy
YXkgLndyYXAtcGFuZWwgdGFibGUgLnRoZWFkLW1pbiwuZGFyay5ncmF5IC53cmFwLXBhbmVsIHRh
YmxlIC50aGVhZC1hdmcsLmRhcmsuZ3JheSAud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtbWF4IHti
YWNrZ3JvdW5kOiAjMWMxYzFjO30uZGFyay5ncmF5IC53cmFwLXBhbmVsIHRhYmxlIC50aGVhZC1h
dmcge2JvcmRlci1ib3R0b206IDJweCBzb2xpZCAjOTk5O30uZGFyay5ncmF5IC53cmFwLXBhbmVs
IHRhYmxlPnRoZWFkPnRyLnRoZWFkLWNvbHMge2JvcmRlci1ib3R0b206IDJweCBzb2xpZCAjOTk5
O30uZGFyay5ncmF5IC53cmFwLXBhbmVsIHRhYmxlIHRib2R5LnRib2R5LWRhdGEgdHIuc2hhZGVk
IHtiYWNrZ3JvdW5kLWNvbG9yOiByZ2JhKDQ4LCA0OCwgNDgsIDAuNDgpO30uZGFyay5ncmF5IC5n
cmF5IHtib3JkZXItdG9wOiA0cHggc29saWQgIzMwMzAzMDt9LmRhcmsuZ3JheSAuZ3JpZC1tb2R1
bGUge2JhY2tncm91bmQ6ICMxQzFDMUM7fS5kYXJrLmdyYXkgLmJ0bi1kZWZhdWx0IHtjb2xvcjog
IzlFOUU5RTtiYWNrZ3JvdW5kLWNvbG9yOiAjMjEyMTIxO2JvcmRlci1jb2xvcjogIzMwMzAzMDt9
LmRhcmsuZ3JheSAuYnRuLWRlZmF1bHQ6YWN0aXZlLC5kYXJrLmdyYXkgLmJ0bi1kZWZhdWx0Omhv
dmVyLC5kYXJrLmdyYXkgLmJ0bi1kZWZhdWx0LmFjdGl2ZSwuZGFyay5ncmF5IC5vcGVuPi5kcm9w
ZG93bi10b2dnbGUuYnRuLWRlZmF1bHQge2NvbG9yOiAjMzYzNzM3O2JhY2tncm91bmQtY29sb3I6
ICMxQzFDMUM7Ym9yZGVyLWNvbG9yOiAjMEYxMjE0O30uZGFyay5ncmF5IC5wYWdpbmF0aW9uPi5k
aXNhYmxlZD5hLC5kYXJrLmdyYXkgLnBhZ2luYXRpb24+LmRpc2FibGVkPmE6aG92ZXIsLmRhcmsu
Z3JheSAucGFnaW5hdGlvbj4uZGlzYWJsZWQ+YTpmb2N1cyB7Y29sb3I6ICM3Nzc7fS5kYXJrLmdy
YXkgLnBhZ2luYXRpb24+bGk+YSB7YmFja2dyb3VuZC1jb2xvcjogIzIxMjEyMTtib3JkZXI6IDFw
eCBzb2xpZCAjMzAzMDMwO30uZGFyay5ncmF5IC5wYWdpbmF0aW9uPmxpPmE6aG92ZXIsLmRhcmsu
Z3JheSAucGFnaW5hdGlvbj5saT5hOmFjdGl2ZSwuZGFyay5ncmF5IC5wYWdpbmF0aW9uPmxpPmE6
Zm9jdXMge2NvbG9yOiAjMDM3MEIwO2JhY2tncm91bmQtY29sb3I6ICMyMTIxMjE7Ym9yZGVyLWNv
bG9yOiAjMzAzMDMwO30uZGFyay5ncmF5IC5kcm9wZG93bi1tZW51PmxpPmEge2NvbG9yOiAjRkZG
O30uZGFyay5ncmF5IC5kcm9wZG93bi1tZW51PmxpPmE6aG92ZXIsLmRhcmsuZ3JheSAuZHJvcGRv
d24tbWVudT5saT5hOmZvY3VzIHtjb2xvcjogI0ZGRjtiYWNrZ3JvdW5kLWNvbG9yOiAjMzAzMDMw
O30uZGFyay5ncmF5IC5kcm9wZG93bi1tZW51IHtiYWNrZ3JvdW5kLWNvbG9yOiAjMjEyMTIxO30u
ZGFyay5ncmF5Ojotd2Via2l0LXNjcm9sbGJhci10cmFjaywuZGFyay5ncmF5IC50YWJsZS1yZXNw
b25zaXZlOjotd2Via2l0LXNjcm9sbGJhci10cmFjayB7LXdlYmtpdC1ib3gtc2hhZG93OiBpbnNl
dCAwIDAgNnB4IHJnYmEoMCwwLDAsMC4zKTtiYWNrZ3JvdW5kLWNvbG9yOiAjOUU5RTlFO30uZGFy
ay5ncmF5Ojotd2Via2l0LXNjcm9sbGJhciwuZGFyay5ncmF5IC50YWJsZS1yZXNwb25zaXZlOjot
d2Via2l0LXNjcm9sbGJhciB7d2lkdGg6IDEwcHg7aGVpZ2h0OiAxMHB4O2JhY2tncm91bmQtY29s
b3I6ICM5RTlFOUU7fS5kYXJrLmdyYXk6Oi13ZWJraXQtc2Nyb2xsYmFyLXRodW1iLC5kYXJrLmdy
YXkgLnRhYmxlLXJlc3BvbnNpdmU6Oi13ZWJraXQtc2Nyb2xsYmFyLXRodW1iIHtiYWNrZ3JvdW5k
LWNvbG9yOiAjMzAzMDMwO30uZGFyay5ncmF5IC5jaGFydC10b29sdGlwIHtiYWNrZ3JvdW5kLWNv
bG9yOiAjMzAzMDMwO30uZGFyay5ncmF5IC5yZXBvcnQtdGl0bGUge2JhY2tncm91bmQ6ICMzMDMw
MzA7fS8qIERBUksgQ0hBUlRTICovLmRhcmsgLmFyZWEge29wYWNpdHk6IDAuMTt9LmRhcmsgLmxp
bmUwLC5kYXJrIC5saW5lMSB7c3Ryb2tlLXdpZHRoOiAyO30uZGFyayAuYXJlYTAsLmRhcmsgLmJh
cnMueTAgLmJhciwuZGFyayByZWN0LmxlZ2VuZC55MCB7ZmlsbDogIzAwN0JDMzt9LmRhcmsgLmFy
ZWExLC5kYXJrIC5iYXJzLnkxIC5iYXIsLmRhcmsgLnBvaW50cy55MSwuZGFyayByZWN0LmxlZ2Vu
ZC55MSB7ZmlsbDogI0ZGMzAzRTt9LmRhcmsgLnBvaW50cy55MCB7ZmlsbDogIzAwRDRFMTt9LmRh
cmsgLmxpbmUwIHtzdHJva2U6ICMwMDdCQzM7fS5kYXJrIC5saW5lMSB7c3Ryb2tlOiAjRkYzMDNF
O30uZGFyayAuZ3JpZC55IC50aWNrIGxpbmUsLmRhcmsgLmdyaWQueCAudGljayBsaW5lIHtzdHJv
a2U6ICM0NDQ3NEI7c3Ryb2tlLWRhc2hhcnJheTogMSAxO30uZGFyayAuYXhpcyB0ZXh0LC5kYXJr
IC5heGlzLWxhYmVsLC5kYXJrIHRleHQubGVnZW5kIHtmaWxsOiAjOUU5RTlFO30uZGFyayAuYXhp
cyBwYXRoIHtzdHJva2U6ICM5OTk5OTk7fS5kYXJrIC5heGlzLnggLnRpY2sgbGluZSwuZGFyayAu
YXhpcy55MCAudGljayBsaW5lLC5kYXJrIC5heGlzLnkxIC50aWNrIGxpbmUsLmRhcmsgLmdyaWQu
eSAudGljazpmaXJzdC1jaGlsZCBsaW5lIHtzdHJva2U6ICMzQjQ0NEM7fS5kYXJrIC5jaGFydC10
b29sdGlwIHRoIHtiYWNrZ3JvdW5kLWNvbG9yOiAjMWMxYzFjO30uZGFyayAuY2hhcnQtdG9vbHRp
cCB0ciB7Ym9yZGVyOiAxcHggc29saWQgIzM2MzczNzt9LyogREFSSyBQVVJQTEUgVEhFTUUgKi9o
dG1sLmRhcmsucHVycGxlLC5kYXJrLnB1cnBsZSBib2R5IHtiYWNrZ3JvdW5kOiAjMWUxZTJmO30u
ZGFyay5wdXJwbGUgLmNvbnRhaW5lciB7YmFja2dyb3VuZDogIzFlMWUyZjt9LmRhcmsucHVycGxl
IC5wYWdlLWhlYWRlciB7Ym9yZGVyLWJvdHRvbTogMXB4IHNvbGlkICMyYjM1NTM7fS5kYXJrLnB1
cnBsZSAubGFiZWwtaW5mbyB7YmFja2dyb3VuZC1jb2xvcjogIzE4MTgyMzt9LmRhcmsucHVycGxl
IG5hdiB7Ym9yZGVyLXJpZ2h0OiAxcHggc29saWQgI2UxNGVjYTtiYWNrZ3JvdW5kOiAjMTgxODIz
O30uZGFyay5wdXJwbGUgZGl2LndyYXAtcGFuZWwgPiBkaXYge2JhY2tncm91bmQ6ICMyNzI5M2Q7
Ym9yZGVyLXRvcDogMXB4IHNvbGlkICMyYjM1NTM7fS5kYXJrLnB1cnBsZSAud3JhcC1wYW5lbCB0
YWJsZSB0Ym9keS50Ym9keS1kYXRhIHRyLnNoYWRlZCB7YmFja2dyb3VuZC1jb2xvcjogIzFlMWUy
Zjt9LmRhcmsucHVycGxlIC53cmFwLXBhbmVsIHRhYmxlIHRmb290PnRyPnRoIHtib3JkZXItdG9w
OiAxcHggZG90dGVkICM5OTk7fS5kYXJrLnB1cnBsZSAud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQt
bWluLC5kYXJrLnB1cnBsZSAud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtYXZnLC5kYXJrLnB1cnBs
ZSAud3JhcC1wYW5lbCB0YWJsZSAudGhlYWQtbWF4IHtiYWNrZ3JvdW5kOiAjMjcyOTNkO30uZGFy
ay5wdXJwbGUgLndyYXAtcGFuZWwgdGFibGUgLnRoZWFkLWF2ZyB7Ym9yZGVyLWJvdHRvbTogMnB4
IHNvbGlkICM5OTk7fS5kYXJrLnB1cnBsZSAud3JhcC1wYW5lbCB0YWJsZT50aGVhZD50ci50aGVh
ZC1jb2xzIHtib3JkZXItYm90dG9tOiAycHggc29saWQgIzk5OTt9LmRhcmsucHVycGxlIC5ncmF5
IHtib3JkZXItdG9wOiA0cHggc29saWQgIzJiMzU1Mzt9LmRhcmsucHVycGxlIC5yZWQge2JvcmRl
ci10b3A6IDRweCBzb2xpZCAjZmQ1ZDkzO30uZGFyay5wdXJwbGUgLmdyZWVuIHtib3JkZXItdG9w
OiA0cHggc29saWQgIzAwZjJjMzt9LmRhcmsucHVycGxlIC5ibHVlIHtib3JkZXItdG9wOiA0cHgg
c29saWQgIzFmOGVmMTt9LmRhcmsucHVycGxlIGgzLCAuZGFyay5wdXJwbGUgaDQsIC5kYXJrLnB1
cnBsZSBoNSB7Y29sb3I6ICM5YTlhOWE7fS5kYXJrLnB1cnBsZSAuZ3JpZC1tb2R1bGUge2JhY2tn
cm91bmQ6ICMyNzI5M2Q7fS5kYXJrLnB1cnBsZSAuZ3JpZC1tb2R1bGUgaDMge2NvbG9yOiAjRkZG
O30uZGFyay5wdXJwbGUgLmJ0bi1kZWZhdWx0IHtjb2xvcjogIzlFOUU5RTtiYWNrZ3JvdW5kLWNv
bG9yOiAjMWUxZTJmO2JvcmRlci1jb2xvcjogIzJiMzU1Mzt9LmRhcmsucHVycGxlIC5idG4tZGVm
YXVsdDphY3RpdmUsLmRhcmsucHVycGxlIC5idG4tZGVmYXVsdDpob3ZlciwuZGFyay5wdXJwbGUg
LmJ0bi1kZWZhdWx0LmFjdGl2ZSwuZGFyay5wdXJwbGUgLm9wZW4+LmRyb3Bkb3duLXRvZ2dsZS5i
dG4tZGVmYXVsdCB7Y29sb3I6ICM1OTU5NWY7YmFja2dyb3VuZC1jb2xvcjogIzFlMWUyZjtib3Jk
ZXItY29sb3I6ICMyYjM1NTM7fS5kYXJrLnB1cnBsZSAucGFnaW5hdGlvbj4uZGlzYWJsZWQ+YSwu
ZGFyay5wdXJwbGUgLnBhZ2luYXRpb24+LmRpc2FibGVkPmE6aG92ZXIsLmRhcmsucHVycGxlIC5w
YWdpbmF0aW9uPi5kaXNhYmxlZD5hOmZvY3VzIHtjb2xvcjogIzc3Nzt9LmRhcmsucHVycGxlIC5w
YWdpbmF0aW9uPmxpPmEge2JhY2tncm91bmQtY29sb3I6ICMxZTFlMmY7Ym9yZGVyOiAxcHggc29s
aWQgIzNCNDQ0Qzt9LmRhcmsucHVycGxlIC5wYWdpbmF0aW9uPmxpPmE6aG92ZXIsLmRhcmsucHVy
cGxlIC5wYWdpbmF0aW9uPmxpPmE6YWN0aXZlLC5kYXJrLnB1cnBsZSAucGFnaW5hdGlvbj5saT5h
OmZvY3VzIHtjb2xvcjogIzAzNzBCMDtiYWNrZ3JvdW5kLWNvbG9yOiAjMTgxODIzO30uZGFyay5w
dXJwbGUgLmRyb3Bkb3duLW1lbnU+bGk+YTpob3ZlciwuZGFyay5wdXJwbGUgLmRyb3Bkb3duLW1l
bnU+bGk+YTpmb2N1cyB7Y29sb3I6ICNGRkY7YmFja2dyb3VuZC1jb2xvcjogIzE4MTgyMzt9LmRh
cmsucHVycGxlIC5kcm9wZG93bi1tZW51IHtiYWNrZ3JvdW5kLWNvbG9yOiAjMWUxZTJmO30uZGFy
ay5wdXJwbGU6Oi13ZWJraXQtc2Nyb2xsYmFyLXRyYWNrLC5kYXJrLnB1cnBsZSAudGFibGUtcmVz
cG9uc2l2ZTo6LXdlYmtpdC1zY3JvbGxiYXItdHJhY2sgey13ZWJraXQtYm94LXNoYWRvdzogaW5z
ZXQgMCAwIDZweCByZ2JhKDAsMCwwLDAuMyk7YmFja2dyb3VuZC1jb2xvcjogIzlFOUU5RTt9LmRh
cmsucHVycGxlOjotd2Via2l0LXNjcm9sbGJhciwuZGFyay5wdXJwbGUgLnRhYmxlLXJlc3BvbnNp
dmU6Oi13ZWJraXQtc2Nyb2xsYmFyIHt3aWR0aDogMTBweDtoZWlnaHQ6IDEwcHg7YmFja2dyb3Vu
ZC1jb2xvcjogIzlFOUU5RTt9LmRhcmsucHVycGxlOjotd2Via2l0LXNjcm9sbGJhci10aHVtYiwu
ZGFyay5wdXJwbGUgLnRhYmxlLXJlc3BvbnNpdmU6Oi13ZWJraXQtc2Nyb2xsYmFyLXRodW1iIHti
YWNrZ3JvdW5kLWNvbG9yOiAjMWUxZTJmO30uZGFyay5wdXJwbGUgLmNoYXJ0LXRvb2x0aXAge2Jh
Y2tncm91bmQtY29sb3I6ICMxODE4MjM7fS5kYXJrLnB1cnBsZSAucmVwb3J0LXRpdGxlIHtiYWNr
Z3JvdW5kOiAjMTgxODIzO30uZGFyay5wdXJwbGUgLmFyZWEwLC5kYXJrLnB1cnBsZSAuYmFycy55
MCAuYmFyLC5kYXJrLnB1cnBsZSByZWN0LmxlZ2VuZC55MCB7ZmlsbDogIzAwN0JDMzt9LmRhcmsu
cHVycGxlIC5hcmVhMSwuZGFyay5wdXJwbGUgLmJhcnMueTEgLmJhciwuZGFyay5wdXJwbGUgLnBv
aW50cy55MSwuZGFyay5wdXJwbGUgcmVjdC5sZWdlbmQueTEge2ZpbGw6ICNkMDQ4YjY7fS5kYXJr
LnB1cnBsZSAucG9pbnRzLnkwIHtmaWxsOiAjMDBENEUxO30uZGFyay5wdXJwbGUgLmxpbmUwIHtz
dHJva2U6ICMwMDdCQzM7fS5kYXJrLnB1cnBsZSAubGluZTEge3N0cm9rZTogI2QwNDhiNjt9PC9z
dHlsZT48L2hlYWQ+PGJvZHk+PG5hdiBjbGFzcz0naGlkZGVuLXhzIGhpZGRlbi1zbSBoaWRlJz48
L25hdj48aSBjbGFzcz0nc3Bpbm5lciBmYSBmYS1jaXJjbGUtby1ub3RjaCBmYS1zcGluIGZhLTN4
IGZhLWZ3Jz48L2k+PGRpdiBjbGFzcz0nY29udGFpbmVyIGhpZGUnPjxkaXYgY2xhc3M9J3dyYXAt
aGVhZGVyJz48ZGl2IGNsYXNzPSdyb3cgcm93LW9mZmNhbnZhcyByb3ctb2ZmY2FudmFzLXJpZ2h0
Jz48ZGl2IGNsYXNzPSdjb2wtbWQtMTInPjxkaXYgY2xhc3M9J3BhZ2UtaGVhZGVyIGNsZWFyZml4
Jz48ZGl2IGNsYXNzPSdwdWxsLXJpZ2h0Jz48aDQ+PHNwYW4gY2xhc3M9J2xhYmVsIGxhYmVsLWlu
Zm8nIHN0eWxlPSdkaXNwbGF5OmJsb2NrJz48c3BhbiBjbGFzcz0naGlkZGVuLXhzJz5MYXN0IFVw
ZGF0ZWQ6IDwvc3Bhbj48c3BhbiBjbGFzcz0nbGFzdC11cGRhdGVkJz4yMDI0LTAyLTE5IDE1OjUy
OjQzICswODAwPC9zcGFuPjwvc3Bhbj48L2g0PjwvZGl2PjxoMSBjbGFzcz0naC1kYXNoYm9hcmQn
PjxzcGFuIGNsYXNzPSdoaWRkZW4teHMgaGlkZGVuLXNtJz48aSBjbGFzcz0nZmEgZmEtdGFjaG9t
ZXRlcic+PC9pPiBEYXNoYm9hcmQ8L3NwYW4+PHNwYW4gY2xhc3M9J3Zpc2libGUteHMgdmlzaWJs
ZS1zbSc+PGkgY2xhc3M9J2ZhIGZhLWJhcnMgbmF2LW1pbmliYXJzJz48L2k+PGkgY2xhc3M9J2Zh
IGZhLWNpcmNsZSBuYXYtd3Mtc3RhdHVzIG1pbmknPjwvaT48L3NwYW4+PC9oMT48ZGl2IGNsYXNz
PSdyZXBvcnQtdGl0bGUnPjwvZGl2PjwvZGl2PjxkaXYgY2xhc3M9J3dyYXAtZ2VuZXJhbCc+PC9k
aXY+PC9kaXY+PC9kaXY+PC9kaXY+PGRpdiBjbGFzcz0nd3JhcC1wYW5lbHMnPjwvZGl2PjwvZGl2
PjwhLS0gVFBMIEdlbmVyYWwgIC0tPjxzY3JpcHQgaWQ9InRwbC1nZW5lcmFsIiB0eXBlPSJ0ZXh0
L3RlbXBsYXRlIj48aDQgY2xhc3M9ImhpZGRlbi14cyBnaGVhZGVyIj57e2hlYWR9fTxzcGFuIGNs
YXNzPSJwdWxsLXJpZ2h0Ij57eyNmcm9tfX08c3BhbiBjbGFzcz0iZnJvbSI+e3tmcm9tfX08L3Nw
YW4+e3svZnJvbX19e3sjdG99fSAmIzgyMTI7IDxzcGFuIGNsYXNzPSJ0byI+e3t0b319PC9zcGFu
Pnt7L3RvfX08L3NwYW4+PC9oND48aDUgY2xhc3M9InZpc2libGUteHMgaGlkZGVuLXNtIGhpZGRl
bi1tZCBoaWRkZW4tbGcgZ2hlYWRlciI+e3toZWFkfX0mbmJzcDsmbmJzcDt7eyNmcm9tfX08c3Bh
biBjbGFzcz0iZnJvbSI+e3tmcm9tfX08L3NwYW4+e3svZnJvbX19e3sjdG99fSAmIzgyMTI7IDxz
cGFuIGNsYXNzPSJ0byI+e3t0b319PC9zcGFuPnt7L3RvfX08L2g1PjxkaXYgY2xhc3M9IndyYXAt
Z2VuZXJhbC1pdGVtcyI+PC9kaXY+PC9zY3JpcHQ+PCEtLSBUUEwgR2VuZXJhbCBJdGVtcyAtLT48
c2NyaXB0IGlkPSJ0cGwtZ2VuZXJhbC1pdGVtcyIgdHlwZT0idGV4dC90ZW1wbGF0ZSI+PGRpdiBj
bGFzcz0iY29sLW1kLTIiPjxkaXYgY2xhc3M9ImdyaWQtbW9kdWxlIHt7I2NsYXNzTmFtZX19e3tj
bGFzc05hbWV9fXt7L2NsYXNzTmFtZX19e3teY2xhc3NOYW1lfX1ncmF5e3svY2xhc3NOYW1lfX0i
PjxkaXYgY2xhc3M9ImNvbC10aXRsZSI+PGkgY2xhc3M9ImZhIGZhLWJhci1jaGFydCI+PC9pPiB7
eyNsYWJlbH19e3tsYWJlbH19e3svbGFiZWx9fTwvZGl2PjxoMyBpZD0ie3tpZH19IiBzdHlsZT0i
cGFkZGluZy10b3A6IDA7Ij57e3ZhbHVlfX08L2gzPjwvZGl2PjwvZGl2Pjwvc2NyaXB0PjwhLS0g
VFBMIFBhbmVsIFRhYmxlIC0tPjxzY3JpcHQgaWQ9InRwbC10YWJsZS1yb3ciIHR5cGU9InRleHQv
dGVtcGxhdGUiPnt7I3Jvd3N9fTx0ciBjbGFzcz0ie3sjY2xhc3NOYW1lfX17e2NsYXNzTmFtZX19
e3svY2xhc3NOYW1lfX0ge3sjaGFzU3ViSXRlbXN9fXt7I2l0ZW1zfX1leHBhbmRhYmxle3svaXRl
bXN9fXt7L2hhc1N1Ykl0ZW1zfX0iIHt7I2lkeH19ZGF0YS1waWQ9Int7aWR4fX0ie3svaWR4fX0g
ZGF0YS1wYW5lbD0ie3twYW5lbH19IiB7eyNrZXl9fWRhdGEta2V5PSJ7e2tleX19Int7L2tleX19
Pnt7I2hhc1N1Ykl0ZW1zfX08dGQgY2xhc3M9InJvdy1leHBhbmRhYmxlIHRleHQtY2VudGVyIHt7
I2l0ZW1zfX1jbGlja2FibGV7ey9pdGVtc319Ij57eyNpdGVtc319PGkgY2xhc3M9ImZhIHt7I2V4
cGFuZGVkfX1mYS1jYXJldC1kb3due3svZXhwYW5kZWR9fXt7XmV4cGFuZGVkfX1mYSBmYS1jYXJl
dC1yaWdodHt7L2V4cGFuZGVkfX0iPjwvaT57ey9pdGVtc319e3teaXRlbXN9fTxpPjwvaT57ey9p
dGVtc319PC90ZD57ey9oYXNTdWJJdGVtc319PHRkIGNsYXNzPSJyb3ctaWR4IHRleHQtcmlnaHQi
Pnt7I2lkeH19e3tpZHh9fXt7L2lkeH19PC90ZD57eyNjZWxsc319PHRkIGNsYXNzPSJ7e2NsYXNz
TmFtZX19IiB7eyNjb2xzcGFufX1jb2xzcGFuPSJ7e2NvbHNwYW59fSJ7ey9jb2xzcGFufX0+PHNw
YW4gY2xhc3M9InZhbHVlIj57e3t2YWx1ZX19fTwvc3Bhbj57eyNwZXJjZW50fX08c3BhbiBjbGFz
cz0icGVyY2VudCI+ICh7e3BlcmNlbnR9fSk8L3NwYW4+e3svcGVyY2VudH19PC90ZD57ey9jZWxs
c319PC90cj57ey9yb3dzfX08L3NjcmlwdD48IS0tIFRQTCBQYW5lbCBUYWJsZSBNZXRhIC0tPjxz
Y3JpcHQgaWQ9InRwbC10YWJsZS1yb3ctbWV0YSIgdHlwZT0idGV4dC90ZW1wbGF0ZSI+e3sjcm93
fX17eyNoYXNTdWJJdGVtc319PHRoIGNsYXNzPSIiPjwvdGg+e3svaGFzU3ViSXRlbXN9fTx0aCBj
bGFzcz0ibWV0YS1sYWJlbCI+e3trZXl9fS48L3RoPnt7I2NlbGxzfX08dGggY2xhc3M9Int7Y2xh
c3NOYW1lfX0iIHt7I2NvbHNwYW59fWNvbHNwYW49Int7Y29sc3Bhbn19Int7L2NvbHNwYW59fT57
eyN2YWx1ZX19PHNwYW4+e3t2YWx1ZX19IHt7I3BlcmNlbnR9fTxzcGFuIGNsYXNzPSJ0ZXh0LW11
dGVkIj4oe3twZXJjZW50fX0lKTwvc3Bhbj57ey9wZXJjZW50fX08L3NwYW4+e3svdmFsdWV9fXt7
XnZhbHVlfX08c3BhbiBjbGFzcz0idGV4dC1tdXRlZCI+Jm1kYXNoOzwvc3Bhbj57ey92YWx1ZX19
PC90aD57ey9jZWxsc319e3svcm93fX08L3NjcmlwdD48IS0tIFRQTCBQYW5lbCBUYWJsZSBUb3Rh
bHMgLS0+PHNjcmlwdCBpZD0idHBsLXRhYmxlLXJvdy10b3RhbHMiIHR5cGU9InRleHQvdGVtcGxh
dGUiPnt7I3Jvd319e3sjaGFzU3ViSXRlbXN9fTx0ZCBjbGFzcz0iIj48L3RkPnt7L2hhc1N1Ykl0
ZW1zfX08dGQgY2xhc3M9Im1ldGEtbGFiZWwiPlRvdC48L3RkPnt7I2NlbGxzfX08dGQgY2xhc3M9
Int7Y2xhc3NOYW1lfX0iIHt7I2NvbHNwYW59fWNvbHNwYW49Int7Y29sc3Bhbn19Int7L2NvbHNw
YW59fT57eyNsYWJlbH19PHNwYW4+e3tsYWJlbH19PC9zcGFuPiZuYnNwO3t7L2xhYmVsfX08c3Bh
bj57e3ZhbHVlfX08L3NwYW4+PC90ZD57ey9jZWxsc319e3svcm93fX08L3NjcmlwdD48IS0tIFRQ
TCBUYWJsZSB0aGVhZCAtLT48c2NyaXB0IGlkPSJ0cGwtdGFibGUtdGhlYWQiIHR5cGU9InRleHQv
dGVtcGxhdGUiPnt7I2hhc1N1Ykl0ZW1zfX08dGg+PC90aD57ey9oYXNTdWJJdGVtc319PHRoPiM8
L3RoPnt7I2l0ZW1zfX08dGggY2xhc3M9Int7ZGF0YVR5cGV9fSB7eyNrZXl9fXNvcnRhYmxle3sv
a2V5fX0iIGRhdGEta2V5PSJ7e2tleX19IiB7eyNzb3J0fX1kYXRhLW9yZGVyPSJ7eyNhc2N9fWFz
Y3t7L2FzY319e3teYXNjfX1kZXNje3svYXNjfX0ie3svc29ydH19Pnt7bGFiZWx9fSA8aSBjbGFz
cz0iZmEgZmEte3tec29ydH19c29ydHt7L3NvcnR9fXt7I3NvcnR9fXt7I2FzY319Y2FyZXQtdXB7
ey9hc2N9fXt7XmFzY319Y2FyZXQtZG93bnt7L2FzY319e3svc29ydH19Ij48L2k+PC90aD57ey9p
dGVtc319PC9zY3JpcHQ+PCEtLSBUUEwgUGFuZWwgT3B0aW9ucyBEcm9wRG93biAtLT48c2NyaXB0
IGlkPSJ0cGwtcGFuZWwtb3B0cyIgdHlwZT0idGV4dC90ZW1wbGF0ZSI+PGxpIGNsYXNzPSJkcm9w
ZG93bi1oZWFkZXIiPnt7IGxhYmVscy5jaGFydF9vcHRzIH19PC9saT48bGk+PGEgaHJlZj0iamF2
YXNjcmlwdDp2b2lkKDApOyIgZGF0YS1wYW5lbD0ie3tpZH19IiBkYXRhLWNoYXJ0PSJ7e3Nob3dD
aGFydH19Ij48aSBjbGFzcz0iZmEgZmEte3sjc2hvd0NoYXJ0fX1jaGVjay17ey9zaG93Q2hhcnR9
fXNxdWFyZS1vIj48L2k+IHt7bGFiZWxzLmNoYXJ0fX08L2E+PC9saT57eyNwbG90Lmxlbmd0aH19
PGxpIGNsYXNzPSJkcm9wZG93bi1oZWFkZXIiPnt7IGxhYmVscy50eXBlIH19PC9saT48bGk+PGEg
aHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgZGF0YS1wYW5lbD0ie3tpZH19IiBkYXRhLWNoYXJ0
LXR5cGU9ImFyZWEtc3BsaW5lIj48aSBjbGFzcz0iZmEgZmEtY2lyY2xle3teYXJlYS1zcGxpbmV9
fS1ve3svYXJlYS1zcGxpbmV9fSI+PC9pPiB7e2xhYmVscy5hcmVhX3NwbGluZX19PC9hPjwvbGk+
PGxpPjxhIGhyZWY9ImphdmFzY3JpcHQ6dm9pZCgwKTsiIGRhdGEtcGFuZWw9Int7aWR9fSIgZGF0
YS1jaGFydC10eXBlPSJiYXIiPjxpIGNsYXNzPSJmYSBmYS1jaXJjbGV7e15iYXJ9fS1ve3svYmFy
fX0iPjwvaT4ge3tsYWJlbHMuYmFyfX08L2E+PC9saT48bGkgY2xhc3M9ImRyb3Bkb3duLWhlYWRl
ciI+e3tsYWJlbHMucGxvdF9tZXRyaWN9fTwvbGk+e3sjcGxvdH19PGxpPjxhIGhyZWY9ImphdmFz
Y3JpcHQ6dm9pZCgwKTsiIGRhdGEtcGFuZWw9Int7aWR9fSIgZGF0YS1wbG90PSJ7e2NsYXNzTmFt
ZX19IiBjbGFzcz0icGFuZWwtcGxvdC17e2NsYXNzTmFtZX19Ij48aSBjbGFzcz0iZmEgZmEtY2ly
Y2xle3tec2VsZWN0ZWR9fS1ve3svc2VsZWN0ZWR9fSI+PC9pPiB7e2xhYmVsfX08L2E+PC9saT57
ey9wbG90fX17ey9wbG90Lmxlbmd0aH19PGxpIGNsYXNzPSJkcm9wZG93bi1oZWFkZXIiPnt7bGFi
ZWxzLnRhYmxlX2NvbHVtbnN9fTwvbGk+e3sjaXRlbXN9fTxsaT48YSBocmVmPSJqYXZhc2NyaXB0
OnZvaWQoMCk7IiBkYXRhLXBhbmVsPSJ7e2lkfX0iIGRhdGEtbWV0cmljPSJ7e2tleX19Ij48aSBj
bGFzcz0iZmEgZmEte3teaGlkZX19Y2hlY2ste3svaGlkZX19c3F1YXJlLW8iPjwvaT4ge3tsYWJl
bH19PC9hPjwvbGk+e3svaXRlbXN9fTwvc2NyaXB0PjwhLS0gVFBMIFRhYmxlIGNvbGdyb3VwIC0t
PjxzY3JpcHQgaWQ9InRwbC10YWJsZS1jb2xncm91cCIgdHlwZT0idGV4dC90ZW1wbGF0ZSI+e3sj
aGFzU3ViSXRlbXN9fTxjb2wgc3R5bGU9IndpZHRoOiAyJTsiPiA8IS0tIHJpZ2h0LWNhcmV0IC0t
Pnt7L2hhc1N1Ykl0ZW1zfX08Y29sIHN0eWxlPSJ3aWR0aDogMyU7Ij4gPCEtLSByb3cgIyAtLT57
eyNpdGVtc319PGNvbCBzdHlsZT0id2lkdGg6e3tjb2xXaWR0aH19Ij57ey9pdGVtc319PC9zY3Jp
cHQ+PCEtLSBUUEwgUGFuZWwgLS0+PHNjcmlwdCBpZD0idHBsLXBhbmVsIiB0eXBlPSJ0ZXh0L3Rl
bXBsYXRlIj48ZGl2IGNsYXNzPSJyb3ciPjxkaXYgY2xhc3M9ImNvbC1tZC0xMiI+PGRpdiBjbGFz
cz0iZm9ybS1ncm91cCBjbGVhcmZpeCBwYW5lbC1oZWFkZXIiPjxoNCBjbGFzcz0icHVsbC1sZWZ0
IGhpZGRlbi14cyBnaGVhZGVyIiBpZD0ie3tpZH19Ij57e2hlYWR9fTxicj48c21hbGw+e3tkZXNj
fX08L3NtYWxsPjwvaDQ+PGg1IGNsYXNzPSJwdWxsLWxlZnQgdmlzaWJsZS14cyBoaWRkZW4tc20g
aGlkZGVuLW1kIGhpZGRlbi1sZyBnaGVhZGVyIiBpZD0ie3tpZH19Ij57e2hlYWR9fTxicj48c21h
bGw+e3tkZXNjfX08L3NtYWxsPjwvaDU+PGRpdiBjbGFzcz0icGFuZWwtcGxvdC13cmFwIj48ZGl2
IGNsYXNzPSJkcm9wZG93biI+PGJ1dHRvbiBjbGFzcz0iYnRuIGJ0bi1kZWZhdWx0IGJ0bi1zbSBk
cm9wZG93bi10b2dnbGUiIHR5cGU9ImJ1dHRvbiIgZGF0YS10b2dnbGU9ImRyb3Bkb3duIiBhcmlh
LWhhc3BvcHVwPSJ0cnVlIiBhcmlhLWV4cGFuZGVkPSJ0cnVlIiBkYXRhLXBhbmVsPSJ7e2lkfX0i
PjxpIGNsYXNzPSJmYSBmYS1nZWFyIj48L2k+IHt7bGFiZWxzLnBhbmVsX29wdHN9fSA8c3BhbiBj
bGFzcz0iZmEgZmEtY2FyZXQtZG93biI+PC9zcGFuPjwvYnV0dG9uPjx1bCBjbGFzcz0iZHJvcGRv
d24tbWVudSBkcm9wZG93bi1tZW51LXJpZ2h0IHBhbmVsLW9wdHMte3tpZH19Ij48L3VsPjwvZGl2
PjwvZGl2PjwvZGl2PjwvZGl2PjwvZGl2Pnt7I2NoYXJ0fX08ZGl2IGNsYXNzPSJyb3ciPjxkaXYg
Y2xhc3M9ImNvbC1tZC0xMiI+PGRpdiBpZD0iY2hhcnQte3tpZH19IiBjbGFzcz0iY2hhcnQtd3Jh
cCI+PC9kaXY+PC9kaXY+PC9kaXY+e3svY2hhcnR9fXt7I3RhYmxlfX08ZGl2IGNsYXNzPSJyb3cg
Y2xlYXJmaXggdGFibGUtd3JhcHBlciB7eyNhdXRvSGlkZVRhYmxlc319aGlkZGVuLXhze3svYXV0
b0hpZGVUYWJsZXN9fSI+PGRpdiBjbGFzcz0iY29sLW1kLTEyIj48ZGl2IGNsYXNzPSJ0YWJsZS1y
ZXNwb25zaXZlIj48dGFibGUgZGF0YS1wYW5lbD0ie3tpZH19IiBjbGFzcz0idGFibGUgdGFibGUt
Ym9yZGVybGVzcyB0YWJsZS1ob3ZlciB0YWJsZS17e2lkfX0iPjxjb2xncm91cD48L2NvbGdyb3Vw
Pjx0aGVhZCA+PHRyIGNsYXNzPSJ0aGVhZC1jb2xzIj48L3RyPjx0ciBjbGFzcz0idGhlYWQtbWlu
Ij48L3RyPjx0ciBjbGFzcz0idGhlYWQtbWF4Ij48L3RyPjx0ciBjbGFzcz0idGhlYWQtYXZnIj48
L3RyPjwvdGhlYWQ+PHRib2R5IGNsYXNzPSJ0Ym9keS1kYXRhIj48L3Rib2R5Pjx0Zm9vdCBjbGFz
cz0idGZvb3QtdG90YWxzIj48dHIgY2xhc3M9InRmb290LXRvdGFscyI+PC90cj48L3Rmb290Pjwv
dGFibGU+PC9kaXY+PHVsIGNsYXNzPSJwYWdpbmF0aW9uIHBhZ2luYXRpb24tc20gcHVsbC1sZWZ0
Ij48bGkgY2xhc3M9ImRpc2FibGVkIj48YSBjbGFzcz0icGFuZWwtZmlyc3QiIGhyZWY9ImphdmFz
Y3JpcHQ6dm9pZCgwKTsiIGFyaWEtbGFiZWw9Int7bGFiZWxzLmZpcnN0fX0iIGRhdGEtcGFuZWw9
Int7aWR9fSIgdGl0bGU9Int7bGFiZWxzLmZpcnN0fX0iPjxpIGNsYXNzPSJmYSBmYS1hbmdsZS1k
b3VibGUtbGVmdCI+PC9pPjwvYT48L2xpPjxsaSBjbGFzcz0iZGlzYWJsZWQiPjxhIGNsYXNzPSJw
YW5lbC1wcmV2IiBocmVmPSJqYXZhc2NyaXB0OnZvaWQoMCk7IiBhcmlhLWxhYmVsPSJ7e2xhYmVs
cy5wcmV2aW91c319IiBkYXRhLXBhbmVsPSJ7e2lkfX0iIHRpdGxlPSJ7e2xhYmVscy5wcmV2aW91
c319Ij48aSBjbGFzcz0iZmEgZmEtYW5nbGUtbGVmdCI+PC9pPjwvYT48L2xpPjxsaT48YSBjbGFz
cz0icGFuZWwtbmV4dCIgaHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgYXJpYS1sYWJlbD0ie3ts
YWJlbHMubmV4dH19IiBkYXRhLXBhbmVsPSJ7e2lkfX0iIHRpdGxlPSJ7e2xhYmVscy5uZXh0fX0i
PjxpIGNsYXNzPSJmYSBmYS1hbmdsZS1yaWdodCI+PC9pPjwvYT48L2xpPjxsaT48YSBjbGFzcz0i
cGFuZWwtbGFzdCIgaHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgYXJpYS1sYWJlbD0ie3tsYWJl
bHMubGFzdH19IiBkYXRhLXBhbmVsPSJ7e2lkfX0iIHRpdGxlPSJ7e2xhYmVscy5sYXN0fX0iPjxp
IGNsYXNzPSJmYSBmYS1hbmdsZS1kb3VibGUtcmlnaHQiPjwvaT48L2E+PC9saT48L3VsPjwvZGl2
PjwvZGl2Pnt7L3RhYmxlfX08L3NjcmlwdD48IS0tIFRQTCBOYXYgQmFyIHdyYXBwZXIgLS0+PHNj
cmlwdCBpZD0idHBsLW5hdi13cmFwIiB0eXBlPSJ0ZXh0L3RlbXBsYXRlIj48ZGl2IGNsYXNzPSJu
YXYtbGlzdCI+PC9kaXY+PGRpdiBjbGFzcz0ibmF2LWJhcnMgZmEgZmEtYmFycyI+PC9kaXY+PGRp
diBjbGFzcz0ibmF2LWdlYXJzIGZhIGZhLWNvZyI+PC9kaXY+PGRpdiBjbGFzcz0ibmF2LXdzLXN0
YXR1cyBmYSBmYS1jaXJjbGUiPjwvZGl2PjxkaXYgY2xhc3M9InBvd2VyZWQgaGlkZGVuLXhzIGhp
ZGRlbi1zbSI+YnkgPGEgaHJlZj0iaHR0cHM6Ly9nb2FjY2Vzcy5pby8iPkdvQWNjZXNzPC9hPiA8
c3Bhbj52e3t2ZXJzaW9ufX08L3NwYW4+IGFuZCA8YSBocmVmPSJodHRwczovL2d3c29ja2V0Lmlv
LyI+R1dTb2NrZXQ8L2E+PC9kaXY+PC9zY3JpcHQ+PCEtLSBUUEwgTmF2IEJhciBpdGVtcyAtLT48
c2NyaXB0IGlkPSJ0cGwtbmF2LW1lbnUiIHR5cGU9InRleHQvdGVtcGxhdGUiPjxoMz57e2xhYmVs
cy5wYW5lbHN9fTwvaDM+PHVsPjxsaSB7eyNvdmVyYWxsX2N1cnJlbnR9fWNsYXNzPSJhY3RpdmUi
e3svb3ZlcmFsbF9jdXJyZW50fX0+PGEgaHJlZj0iIyI+PGkgY2xhc3M9ImZhIGZhLWJhci1jaGFy
dCI+PC9pPiB7e2xhYmVscy50aGVhZH19PC9hPjxzcGFuIHRpdGxlPSJ7e2xhYmVscy50b2dnbGVf
cGFuZWx9fSIgY2xhc3M9InRvZ2dsZS1wYW5lbCB7eyNvdmVyYWxsX2hpZGRlbn19YWN0aXZle3sv
b3ZlcmFsbF9oaWRkZW59fSIgZGF0YS1wYW5lbD0iZ2VuZXJhbCI+PGkgY2xhc3M9ImZhIGZhLWNp
cmNsZS1vIj48L2k+PC9zcGFuPjwvbGk+e3sjbmF2fX08bGkge3sjY3VycmVudH19Y2xhc3M9ImFj
dGl2ZSJ7ey9jdXJyZW50fX0+PGEgaHJlZj0iI3t7a2V5fX0iPjxpIGNsYXNzPSJmYSBmYS17e2lj
b259fSI+PC9pPiB7e2hlYWR9fTwvYT48c3BhbiB0aXRsZT0ie3tsYWJlbHMudG9nZ2xlX3BhbmVs
fX0iIGNsYXNzPSJ0b2dnbGUtcGFuZWwge3sjaGlkZGVufX1hY3RpdmV7ey9oaWRkZW59fSIgZGF0
YS1wYW5lbD0ie3trZXl9fSI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZS1vIj48L2k+PC9zcGFuPjwv
bGk+e3svbmF2fX08L3VsPjwvc2NyaXB0PjwhLS0gVFBMIE5hdiBCYXIgb3B0aW9ucyAtLT48c2Ny
aXB0IGlkPSJ0cGwtbmF2LW9wdHMiIHR5cGU9InRleHQvdGVtcGxhdGUiPjxoMz48aSBjbGFzcz0i
ZmEgZmEtaGFzaHRhZyI+PC9pPiB7e2xhYmVscy50aGVtZX19PC9oMz48dWw+PGxpIHt7I2RhcmtH
cmF5fX1jbGFzcz0iYWN0aXZlInt7L2RhcmtHcmF5fX0+PGEgaHJlZj0iamF2YXNjcmlwdDp2b2lk
KDApOyIgY2xhc3M9InRoZW1lLWRhcmstZ3JheSI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7XmRh
cmtHcmF5fX0tb3t7L2RhcmtHcmF5fX0iPjwvaT4ge3tsYWJlbHMuZGFya19ncmF5fX08L2E+PC9s
aT48bGkge3sjYnJpZ2h0fX1jbGFzcz0iYWN0aXZlInt7L2JyaWdodH19PjxhIGhyZWY9ImphdmFz
Y3JpcHQ6dm9pZCgwKTsiIGNsYXNzPSJ0aGVtZS1icmlnaHQiPjxpIGNsYXNzPSJmYSBmYS1jaXJj
bGV7e15icmlnaHR9fS1ve3svYnJpZ2h0fX0iPjwvaT4ge3tsYWJlbHMuYnJpZ2h0fX08L2E+PC9s
aT48bGkge3sjZGFya0JsdWV9fWNsYXNzPSJhY3RpdmUie3svZGFya0JsdWV9fT48YSBocmVmPSJq
YXZhc2NyaXB0OnZvaWQoMCk7IiBjbGFzcz0idGhlbWUtZGFyay1ibHVlIj48aSBjbGFzcz0iZmEg
ZmEtY2lyY2xle3teZGFya0JsdWV9fS1ve3svZGFya0JsdWV9fSI+PC9pPiB7e2xhYmVscy5kYXJr
X2JsdWV9fTwvYT48L2xpPjxsaSB7eyNkYXJrUHVycGxlfX1jbGFzcz0iYWN0aXZlInt7L2RhcmtQ
dXJwbGV9fT48YSBocmVmPSJqYXZhc2NyaXB0OnZvaWQoMCk7IiBjbGFzcz0idGhlbWUtZGFyay1w
dXJwbGUiPjxpIGNsYXNzPSJmYSBmYS1jaXJjbGV7e15kYXJrUHVycGxlfX0tb3t7L2RhcmtQdXJw
bGV9fSI+PC9pPiB7e2xhYmVscy5kYXJrX3B1cnBsZX19PC9hPjwvbGk+PC91bD48aDM+PGkgY2xh
c3M9ImZhIGZhLWxpc3QtYWx0Ij48L2k+IHt7bGFiZWxzLnBhbmVsc319PC9oMz48dWwgY2xhc3M9
InBlcnBhZ2Utd3JhcCI+PGxpIGNsYXNzPSJkcm9wZG93bi1oZWFkZXIiPjxpIGNsYXNzPSJmYSBm
YS1saXN0Ij48L2k+IHt7bGFiZWxzLml0ZW1zX3Blcl9wYWdlfX08L2xpPjxsaSB7eyNwZXJQYWdl
M319Y2xhc3M9ImFjdGl2ZSJ7ey9wZXJQYWdlM319PjxhIGhyZWY9ImphdmFzY3JpcHQ6dm9pZCgw
KTsiIGRhdGEtcGVycGFnZT0iMyI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7XnBlclBhZ2UzfX0t
b3t7L3BlclBhZ2UzfX0iPjwvaT4gMzwvYT48L2xpPjxsaSB7eyNwZXJQYWdlNX19Y2xhc3M9ImFj
dGl2ZSJ7ey9wZXJQYWdlNX19PjxhIGhyZWY9ImphdmFzY3JpcHQ6dm9pZCgwKTsiIGRhdGEtcGVy
cGFnZT0iNSI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7XnBlclBhZ2U1fX0tb3t7L3BlclBhZ2U1
fX0iPjwvaT4gNTwvYT48L2xpPjxsaSB7eyNwZXJQYWdlN319Y2xhc3M9ImFjdGl2ZSJ7ey9wZXJQ
YWdlN319PjxhIGhyZWY9ImphdmFzY3JpcHQ6dm9pZCgwKTsiIGRhdGEtcGVycGFnZT0iNyI+PGkg
Y2xhc3M9ImZhIGZhLWNpcmNsZXt7XnBlclBhZ2U3fX0tb3t7L3BlclBhZ2U3fX0iPjwvaT4gNzwv
YT48L2xpPjxsaSB7eyNwZXJQYWdlMTB9fWNsYXNzPSJhY3RpdmUie3svcGVyUGFnZTEwfX0+PGEg
aHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgZGF0YS1wZXJwYWdlPSIxMCI+PGkgY2xhc3M9ImZh
IGZhLWNpcmNsZXt7XnBlclBhZ2UxMH19LW97ey9wZXJQYWdlMTB9fSI+PC9pPiAxMDwvYT48L2xp
PjxsaSB7eyNwZXJQYWdlMTV9fWNsYXNzPSJhY3RpdmUie3svcGVyUGFnZTE1fX0+PGEgaHJlZj0i
amF2YXNjcmlwdDp2b2lkKDApOyIgZGF0YS1wZXJwYWdlPSIxNSI+PGkgY2xhc3M9ImZhIGZhLWNp
cmNsZXt7XnBlclBhZ2UxNX19LW97ey9wZXJQYWdlMTV9fSI+PC9pPiAxNTwvYT48L2xpPjxsaSB7
eyNwZXJQYWdlMjB9fWNsYXNzPSJhY3RpdmUie3svcGVyUGFnZTIwfX0+PGEgaHJlZj0iamF2YXNj
cmlwdDp2b2lkKDApOyIgZGF0YS1wZXJwYWdlPSIyMCI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7
XnBlclBhZ2UyMH19LW97ey9wZXJQYWdlMjB9fSI+PC9pPiAyMDwvYT48L2xpPjxsaSB7eyNwZXJQ
YWdlMjR9fWNsYXNzPSJhY3RpdmUie3svcGVyUGFnZTI0fX0+PGEgaHJlZj0iamF2YXNjcmlwdDp2
b2lkKDApOyIgZGF0YS1wZXJwYWdlPSIyNCI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7XnBlclBh
Z2UyNH19LW97ey9wZXJQYWdlMjR9fSI+PC9pPiAyNDwvYT48L2xpPjxsaSBjbGFzcz0iZHJvcGRv
d24taGVhZGVyIj48aSBjbGFzcz0iZmEgZmEtdGFibGUiPjwvaT4ge3tsYWJlbHMudGFibGVzfX08
L2xpPjxsaSB7eyNzaG93VGFibGVzfX1jbGFzcz0iYWN0aXZlInt7L3Nob3dUYWJsZXN9fT48YSBo
cmVmPSJqYXZhc2NyaXB0OnZvaWQoMCk7IiBkYXRhLXNob3ctdGFibGVzPSIxIj48aSBjbGFzcz0i
ZmEgZmEte3sjc2hvd1RhYmxlc319Y2hlY2ste3svc2hvd1RhYmxlc319c3F1YXJlLW8iPjwvaT4g
e3tsYWJlbHMuZGlzcGxheV90YWJsZXN9fTwvYT48L2xpPjxsaSB7eyNhdXRvSGlkZVRhYmxlc319
Y2xhc3M9ImFjdGl2ZSJ7ey9hdXRvSGlkZVRhYmxlc319PjxhIGhyZWY9ImphdmFzY3JpcHQ6dm9p
ZCgwKTsiIGRhdGEtYXV0b2hpZGUtdGFibGVzPSIxIiB0aXRsZT0ie3tsYWJlbHMuYWhfc21hbGxf
dGl0bGV9fSI+PGkgY2xhc3M9ImZhIGZhLXt7I2F1dG9IaWRlVGFibGVzfX1jaGVjay17ey9hdXRv
SGlkZVRhYmxlc319c3F1YXJlLW8iPjwvaT4ge3tsYWJlbHMuYWhfc21hbGx9fTwvYT48L2xpPjwv
dWw+PGgzPjxpIGNsYXNzPSJmYSBmYS10aC1sYXJnZSI+PC9pPiB7e2xhYmVscy5sYXlvdXR9fTwv
aDM+PHVsPjxsaSB7eyNob3Jpem9udGFsfX1jbGFzcz0iYWN0aXZlInt7L2hvcml6b250YWx9fT48
YSBocmVmPSJqYXZhc2NyaXB0OnZvaWQoMCk7IiBjbGFzcz0ibGF5b3V0LWhvcml6b250YWwiPjxp
IGNsYXNzPSJmYSBmYS1jaXJjbGV7e15ob3Jpem9udGFsfX0tb3t7L2hvcml6b250YWx9fSI+PC9p
PiB7e2xhYmVscy5ob3Jpem9udGFsfX08L2E+PC9saT48bGkge3sjd2lkZX19Y2xhc3M9ImFjdGl2
ZSJ7ey93aWRlfX0+PGEgaHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgY2xhc3M9ImxheW91dC13
aWRlIj48aSBjbGFzcz0iZmEgZmEtY2lyY2xle3ted2lkZX19LW97ey93aWRlfX0iPjwvaT4ge3ts
YWJlbHMud2lkZX19PC9hPjwvbGk+PGxpIHt7I3ZlcnRpY2FsfX1jbGFzcz0iYWN0aXZlInt7L3Zl
cnRpY2FsfX0+PGEgaHJlZj0iamF2YXNjcmlwdDp2b2lkKDApOyIgY2xhc3M9ImxheW91dC12ZXJ0
aWNhbCI+PGkgY2xhc3M9ImZhIGZhLWNpcmNsZXt7XnZlcnRpY2FsfX0tb3t7L3ZlcnRpY2FsfX0i
PjwvaT4ge3tsYWJlbHMudmVydGljYWx9fTwvYT48L2xpPjwvdWw+PGgzPjxpIGNsYXNzPSJmYSBm
YS1jb2ciPjwvaT4ge3tsYWJlbHMuZmlsZV9vcHRzfX08L2gzPjx1bD48bGk+PGEgaHJlZj0iamF2
YXNjcmlwdDp2b2lkKDApOyIgY2xhc3M9ImV4cG9ydC1qc29uIj48aSBjbGFzcz0iZmEgZmEtY29k
ZSI+PC9pPiB7e2xhYmVscy5leHBvcnRfanNvbn19PC9hPjwvbGk+PC91bD48L3NjcmlwdD48IS0t
IFRQTCBDaGFydCB0b29sdGlwIC0tPjxzY3JpcHQgaWQ9InRwbC1jaGFydC10b29sdGlwIiB0eXBl
PSJ0ZXh0L3RlbXBsYXRlIj48dGFibGUgY2xhc3M9ImNoYXJ0LXRvb2x0aXAiPjx0Ym9keT48dHI+
PHRoIGNvbHNwYW49IjIiPnt7ZGF0YS4wfX08L3RoPjwvdHI+PHRyPjx0ZCBjbGFzcz0ibmFtZSI+
PHNwYW4gY2xhc3M9ImJsdWUiPjwvc3Bhbj5oaXRzPC90ZD48dGQgY2xhc3M9InZhbHVlIj57e2Rh
dGEuMX19PC90ZD48L3RyPnt7I2RhdGEuMn19PHRyPjx0ZCBjbGFzcz0ibmFtZSI+PHNwYW4gY2xh
c3M9InJlZCI+PC9zcGFuPnZpc2l0b3JzPC90ZD48dGQgY2xhc3M9InZhbHVlIj57e2RhdGEuMn19
PC90ZD48L3RyPnt7L2RhdGEuMn19PC90Ym9keT48L3RhYmxlPjwvc2NyaXB0PjxzY3JpcHQgdHlw
ZT0ndGV4dC9qYXZhc2NyaXB0Jz52YXIganNvbl9pMThuPXsidGhlbWUiOiAiVGhlbWUiLCJkYXJr
X2dyYXkiOiAiRGFyayBHcmF5IiwiYnJpZ2h0IjogIkJyaWdodCIsImRhcmtfYmx1ZSI6ICJEYXJr
IEJsdWUiLCJkYXJrX3B1cnBsZSI6ICJEYXJrIFB1cnBsZSIsInBhbmVscyI6ICJQYW5lbHMiLCJp
dGVtc19wZXJfcGFnZSI6ICJJdGVtcyBwZXIgUGFnZSIsInRhYmxlcyI6ICJUYWJsZXMiLCJkaXNw
bGF5X3RhYmxlcyI6ICJEaXNwbGF5IFRhYmxlcyIsImFoX3NtYWxsIjogIkF1dG8tSGlkZSBvbiBT
bWFsbCBEZXZpY2VzIiwiYWhfc21hbGxfdGl0bGUiOiAiQXV0b21hdGljYWxseSBoaWRlIHRhYmxl
cyBvbiBzbWFsbCBzY3JlZW4gZGV2aWNlcyIsInRvZ2dsZV9wYW5lbCI6ICJUb2dnbGUgUGFuZWwi
LCJsYXlvdXQiOiAiTGF5b3V0IiwiaG9yaXpvbnRhbCI6ICJIb3Jpem9udGFsIiwidmVydGljYWwi
OiAiVmVydGljYWwiLCJ3aWRlIjogIldpZGVTY3JlZW4iLCJmaWxlX29wdHMiOiAiRmlsZSBPcHRp
b25zIiwiZXhwb3J0X2pzb24iOiAiRXhwb3J0IGFzIEpTT04iLCJwYW5lbF9vcHRzIjogIlBhbmVs
IE9wdGlvbnMiLCJwcmV2aW91cyI6ICJQcmV2aW91cyIsIm5leHQiOiAiTmV4dCIsImZpcnN0Ijog
IkZpcnN0IiwibGFzdCI6ICJMYXN0IiwiY2hhcnRfb3B0cyI6ICJDaGFydCBPcHRpb25zIiwiY2hh
cnQiOiAiQ2hhcnQiLCJ0eXBlIjogIlR5cGUiLCJhcmVhX3NwbGluZSI6ICJBcmVhIFNwbGluZSIs
ImJhciI6ICJCYXIiLCJwbG90X21ldHJpYyI6ICJQbG90IE1ldHJpYyIsInRhYmxlX2NvbHVtbnMi
OiAiVGFibGUgQ29sdW1ucyIsInRoZWFkIjogIk92ZXJhbGwgQW5hbHl6ZWQgUmVxdWVzdHMiLCJ2
ZXJzaW9uIjogIjEuOS4xIix9O3ZhciBodG1sX3ByZWZzPXt9O3ZhciB1c2VyX2ludGVyZmFjZT17
ImdlbmVyYWwiOiB7ImhlYWQiOiAiT3ZlcmFsbCBBbmFseXplZCBSZXF1ZXN0cyIsImRlc2MiOiAi
IiwiaXRlbXMiOiB7InRvdGFsX3JlcXVlc3RzIjogeyJjbGFzc05hbWUiOiAiYmxhY2siLCJkYXRh
VHlwZSI6ICJudW1lcmljIiwibGFiZWwiOiAiVG90YWwgUmVxdWVzdHMifSwidmFsaWRfcmVxdWVz
dHMiOiB7ImNsYXNzTmFtZSI6ICJncmVlbiIsImRhdGFUeXBlIjogIm51bWVyaWMiLCJsYWJlbCI6
ICJWYWxpZCBSZXF1ZXN0cyJ9LCJmYWlsZWRfcmVxdWVzdHMiOiB7ImNsYXNzTmFtZSI6ICJyZWQi
LCJkYXRhVHlwZSI6ICJudW1lcmljIiwibGFiZWwiOiAiRmFpbGVkIFJlcXVlc3RzIn0sImdlbmVy
YXRpb25fdGltZSI6IHsiY2xhc3NOYW1lIjogImdyYXkiLCJkYXRhVHlwZSI6ICJzZWNzIiwibGFi
ZWwiOiAiTG9nIFBhcnNpbmcgVGltZSJ9LCJ1bmlxdWVfdmlzaXRvcnMiOiB7ImNsYXNzTmFtZSI6
ICJibHVlIiwiZGF0YVR5cGUiOiAibnVtZXJpYyIsImxhYmVsIjogIlVuaXF1ZSBWaXNpdG9ycyJ9
LCJ1bmlxdWVfZmlsZXMiOiB7ImRhdGFUeXBlIjogIm51bWVyaWMiLCJsYWJlbCI6ICJSZXF1ZXN0
ZWQgRmlsZXMifSwiZXhjbHVkZWRfaGl0cyI6IHsiZGF0YVR5cGUiOiAibnVtZXJpYyIsImxhYmVs
IjogIkV4Y2wuIElQIEhpdHMifSwidW5pcXVlX3JlZmVycmVycyI6IHsiZGF0YVR5cGUiOiAibnVt
ZXJpYyIsImxhYmVsIjogIlJlZmVycmVycyJ9LCJ1bmlxdWVfbm90X2ZvdW5kIjogeyJkYXRhVHlw
ZSI6ICJudW1lcmljIiwibGFiZWwiOiAiTm90IEZvdW5kIn0sInVuaXF1ZV9zdGF0aWNfZmlsZXMi
OiB7ImRhdGFUeXBlIjogIm51bWVyaWMiLCJsYWJlbCI6ICJTdGF0aWMgRmlsZXMifSwibG9nX3Np
emUiOiB7ImRhdGFUeXBlIjogImJ5dGVzIiwibGFiZWwiOiAiTG9nIFNpemUifSwiYmFuZHdpZHRo
IjogeyJkYXRhVHlwZSI6ICJieXRlcyIsImxhYmVsIjogIlR4LiBBbW91bnQifX19LCJ2aXNpdG9y
cyI6IHsiaGVhZCI6ICJVbmlxdWUgdmlzaXRvcnMgcGVyIGRheSAtIEluY2x1ZGluZyBzcGlkZXJz
IiwiZGVzYyI6ICJIaXRzIGhhdmluZyB0aGUgc2FtZSBJUCwgZGF0ZSBhbmQgYWdlbnQgYXJlIGEg
dW5pcXVlIHZpc2l0LiIsImlkIjogInZpc2l0b3JzIiwidGFibGUiOiAxLCJzb3J0IjogeyJmaWVs
ZCI6ICJkYXRhIiwib3JkZXIiOiAiREVTQyJ9LCJwbG90IjogW3siY2xhc3NOYW1lIjogImhpdHMt
dmlzaXRvcnMiLCJsYWJlbCI6ICJIaXRzL1Zpc2l0b3JzIiwiY2hhcnRUeXBlIjogImFyZWEtc3Bs
aW5lIiwiY2hhcnRSZXZlcnNlIjogMSwicmVkcmF3T25FeHBhbmQiOiAxLCJkMyI6IHsieTAiOiB7
ImtleSI6ICJoaXRzIiwibGFiZWwiOiAiSGl0cyJ9LCJ5MSI6IHsia2V5IjogInZpc2l0b3JzIiwi
bGFiZWwiOiAiVmlzaXRvcnMifX19LHsiY2xhc3NOYW1lIjogImJhbmR3aWR0aCIsImxhYmVsIjog
IlR4LiBBbW91bnQiLCJjaGFydFR5cGUiOiAiYXJlYS1zcGxpbmUiLCJjaGFydFJldmVyc2UiOiAx
LCJyZWRyYXdPbkV4cGFuZCI6IDEsImQzIjogeyJ5MCI6IHsia2V5IjogImJ5dGVzIiwibGFiZWwi
OiAiVHguIEFtb3VudCIsImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVtcyI6IFt7ImNvbFdpZHRo
IjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmljIiwia2V5IjogImhp
dHMiLCJsYWJlbCI6ICJIaXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50Iiwi
ZGF0YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxhYmVsIjogIlZpc2l0b3Jz
In0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAiYnl0ZXMi
LCJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJjbGFzc05hbWUiOiAidHJ1
bmMiLCJjb2xXaWR0aCI6ICIxMDAlIiwibWV0YSI6ICJ1bmlxdWUiLCJtZXRhVHlwZSI6ICJudW1l
cmljIiwibWV0YUxhYmVsIjogIlRvdGFsIiwiZGF0YVR5cGUiOiAiZGF0ZSIsImhscmVnZXgiOiAi
e1wiXigxXFxcXGR7Mn18MXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBsYmx1
Jz4kMTwvYj4kMlwiLFwiXigyXFxcXGR7Mn18Mnh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0n
c3Bhbi1obCBsZ3JuJz4kMTwvYj4kMlwiLFwiXigzXFxcXGR7Mn18M3h4KShcXFxccy4qKSRcIjog
XCI8YiBjbGFzcz0nc3Bhbi1obCBscHJwJz4kMTwvYj4kMlwiLFwiXig0XFxcXGR7Mn18NHh4KShc
XFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBseWVsJz4kMTwvYj4kMlwiLFwiXig1XFxc
XGR7Mn18NXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBscmVkJz4kMTwvYj4k
MlwiLFwiXihBU1xcXFxkKylcIjogXCI8Yj4kMTwvYj5cIixcIl4oXFxcXGQrOilcIjogXCI8Yj4k
MTwvYj5cIixcIihcXFxcZCspfCg6XFxcXGQrKXwoOlxcXFxkKzpcXFxcZCspXCI6IFwiJDE8Yj4k
MjwvYj5cIixcIl4oW0EtWl17Mn0pKFxcXFxzLiokKVwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGc1
Jz4kMTwvYj4kMlwifSIsImtleSI6ICJkYXRhIiwibGFiZWwiOiAiRGF0YSJ9XX0sInJlcXVlc3Rz
IjogeyJoZWFkIjogIlJlcXVlc3RlZCBGaWxlcyAoVVJMcykiLCJkZXNjIjogIlRvcCByZXF1ZXN0
cyBzb3J0ZWQgYnkgaGl0cyBbLCBhdmd0cywgY3VtdHMsIG1heHRzLCBtdGhkLCBwcm90b10iLCJp
ZCI6ICJyZXF1ZXN0cyIsInRhYmxlIjogMSwic29ydCI6IHsiZmllbGQiOiAiaGl0cyIsIm9yZGVy
IjogIkRFU0MifSwicGxvdCI6IFt7ImNsYXNzTmFtZSI6ICJoaXRzLXZpc2l0b3JzIiwibGFiZWwi
OiAiSGl0cy9WaXNpdG9ycyIsImNoYXJ0VHlwZSI6ICJiYXIiLCJjaGFydFJldmVyc2UiOiAwLCJy
ZWRyYXdPbkV4cGFuZCI6IDAsImQzIjogeyJ4IjogeyJrZXkiOiBbIm1ldGhvZCIsICJkYXRhIiwg
InByb3RvY29sIl19LCJ5MCI6IHsia2V5IjogImhpdHMiLCJsYWJlbCI6ICJIaXRzIn0sInkxIjog
eyJrZXkiOiAidmlzaXRvcnMiLCJsYWJlbCI6ICJWaXNpdG9ycyJ9fX0seyJjbGFzc05hbWUiOiAi
YmFuZHdpZHRoIiwibGFiZWwiOiAiVHguIEFtb3VudCIsImNoYXJ0VHlwZSI6ICJiYXIiLCJjaGFy
dFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDAsImQzIjogeyJ4IjogeyJrZXkiOiBbIm1l
dGhvZCIsICJwcm90b2NvbCIsICJkYXRhIl19LCJ5MCI6IHsia2V5IjogImJ5dGVzIiwibGFiZWwi
OiAiVHguIEFtb3VudCIsImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVtcyI6IFt7ImNvbFdpZHRo
IjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmljIiwia2V5IjogImhp
dHMiLCJsYWJlbCI6ICJIaXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50Iiwi
ZGF0YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxhYmVsIjogIlZpc2l0b3Jz
In0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAiYnl0ZXMi
LCJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJjb2xXaWR0aCI6ICI2JSIs
ImRhdGFUeXBlIjogInN0cmluZyIsImhscmVnZXgiOiAie1wiKFxcXFxiW0EtWl17M31cXFxcYilc
IjogXCI8YiBjbGFzcz0nY2VsbC1obCBiMSc+JDE8L2I+XCIsXCIoXFxcXGJbQS1aXXs0fVxcXFxi
KVwiOiBcIjxiIGNsYXNzPSdjZWxsLWhsIGIyJz4kMTwvYj5cIixcIihcXFxcYltBLVpdezUsfVxc
XFxiKVwiOiBcIjxiIGNsYXNzPSdjZWxsLWhsIGIzJz4kMTwvYj5cIn0iLCJrZXkiOiAibWV0aG9k
IiwibGFiZWwiOiAiTWV0aG9kIn0seyJjb2xXaWR0aCI6ICI3JSIsImRhdGFUeXBlIjogInN0cmlu
ZyIsImhscmVnZXgiOiAie1wiKFxcXFxiSFRUUC8xLjBcXFxcYilcIjogXCI8YiBjbGFzcz0nY2Vs
bC1obCBkMSc+JDE8L2I+XCIsXCIoXFxcXGJIVFRQLzEuMVxcXFxiKVwiOiBcIjxiIGNsYXNzPSdj
ZWxsLWhsIGQyJz4kMTwvYj5cIixcIihcXFxcYkhUVFAvMlxcXFxiKVwiOiBcIjxiIGNsYXNzPSdj
ZWxsLWhsIGQzJz4kMTwvYj5cIixcIihcXFxcYkhUVFAvM1xcXFxiKVwiOiBcIjxiIGNsYXNzPSdj
ZWxsLWhsIGQ0Jz4kMTwvYj5cIn0iLCJrZXkiOiAicHJvdG9jb2wiLCJsYWJlbCI6ICJQcm90b2Nv
bCJ9LHsiY2xhc3NOYW1lIjogInRydW5jIiwiY29sV2lkdGgiOiAiMTAwJSIsIm1ldGEiOiAidW5p
cXVlIiwibWV0YVR5cGUiOiAibnVtZXJpYyIsIm1ldGFMYWJlbCI6ICJUb3RhbCIsImRhdGFUeXBl
IjogInN0cmluZyIsImhscmVnZXgiOiAie1wiXigxXFxcXGR7Mn18MXh4KShcXFxccy4qKSRcIjog
XCI8YiBjbGFzcz0nc3Bhbi1obCBsYmx1Jz4kMTwvYj4kMlwiLFwiXigyXFxcXGR7Mn18Mnh4KShc
XFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBsZ3JuJz4kMTwvYj4kMlwiLFwiXigzXFxc
XGR7Mn18M3h4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBscHJwJz4kMTwvYj4k
MlwiLFwiXig0XFxcXGR7Mn18NHh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBs
eWVsJz4kMTwvYj4kMlwiLFwiXig1XFxcXGR7Mn18NXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFz
cz0nc3Bhbi1obCBscmVkJz4kMTwvYj4kMlwiLFwiXihBU1xcXFxkKylcIjogXCI8Yj4kMTwvYj5c
IixcIl4oXFxcXGQrOilcIjogXCI8Yj4kMTwvYj5cIixcIihcXFxcZCspfCg6XFxcXGQrKXwoOlxc
XFxkKzpcXFxcZCspXCI6IFwiJDE8Yj4kMjwvYj5cIixcIl4oW0EtWl17Mn0pKFxcXFxzLiokKVwi
OiBcIjxiIGNsYXNzPSdzcGFuLWhsIGc1Jz4kMTwvYj4kMlwifSIsImtleSI6ICJkYXRhIiwibGFi
ZWwiOiAiRGF0YSJ9XX0sInN0YXRpY19yZXF1ZXN0cyI6IHsiaGVhZCI6ICJTdGF0aWMgUmVxdWVz
dHMiLCJkZXNjIjogIlRvcCBzdGF0aWMgcmVxdWVzdHMgc29ydGVkIGJ5IGhpdHMgWywgYXZndHMs
IGN1bXRzLCBtYXh0cywgbXRoZCwgcHJvdG9dIiwiaWQiOiAic3RhdGljX3JlcXVlc3RzIiwidGFi
bGUiOiAxLCJzb3J0IjogeyJmaWVsZCI6ICJoaXRzIiwib3JkZXIiOiAiREVTQyJ9LCJwbG90Ijog
W3siY2xhc3NOYW1lIjogImhpdHMtdmlzaXRvcnMiLCJsYWJlbCI6ICJIaXRzL1Zpc2l0b3JzIiwi
Y2hhcnRUeXBlIjogImJhciIsImNoYXJ0UmV2ZXJzZSI6IDAsInJlZHJhd09uRXhwYW5kIjogMCwi
ZDMiOiB7IngiOiB7ImtleSI6IFsibWV0aG9kIiwgImRhdGEiLCAicHJvdG9jb2wiXX0sInkwIjog
eyJrZXkiOiAiaGl0cyIsImxhYmVsIjogIkhpdHMifSwieTEiOiB7ImtleSI6ICJ2aXNpdG9ycyIs
ImxhYmVsIjogIlZpc2l0b3JzIn19fSx7ImNsYXNzTmFtZSI6ICJiYW5kd2lkdGgiLCJsYWJlbCI6
ICJUeC4gQW1vdW50IiwiY2hhcnRUeXBlIjogImJhciIsImNoYXJ0UmV2ZXJzZSI6IDAsInJlZHJh
d09uRXhwYW5kIjogMCwiZDMiOiB7IngiOiB7ImtleSI6IFsibWV0aG9kIiwgInByb3RvY29sIiwg
ImRhdGEiXX0sInkwIjogeyJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50IiwiZm9y
bWF0IjogImJ5dGVzIn19fV0sIml0ZW1zIjogW3siY29sV2lkdGgiOiAiMTIlIiwibWV0YSI6ICJj
b3VudCIsImRhdGFUeXBlIjogIm51bWVyaWMiLCJrZXkiOiAiaGl0cyIsImxhYmVsIjogIkhpdHMi
fSx7ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmlj
Iiwia2V5IjogInZpc2l0b3JzIiwibGFiZWwiOiAiVmlzaXRvcnMifSx7ImNvbFdpZHRoIjogIjEy
JSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJieXRlcyIsImtleSI6ICJieXRlcyIsImxh
YmVsIjogIlR4LiBBbW91bnQifSx7ImNvbFdpZHRoIjogIjYlIiwiZGF0YVR5cGUiOiAic3RyaW5n
IiwiaGxyZWdleCI6ICJ7XCIoXFxcXGJbQS1aXXszfVxcXFxiKVwiOiBcIjxiIGNsYXNzPSdjZWxs
LWhsIGIxJz4kMTwvYj5cIixcIihcXFxcYltBLVpdezR9XFxcXGIpXCI6IFwiPGIgY2xhc3M9J2Nl
bGwtaGwgYjInPiQxPC9iPlwiLFwiKFxcXFxiW0EtWl17NSx9XFxcXGIpXCI6IFwiPGIgY2xhc3M9
J2NlbGwtaGwgYjMnPiQxPC9iPlwifSIsImtleSI6ICJtZXRob2QiLCJsYWJlbCI6ICJNZXRob2Qi
fSx7ImNvbFdpZHRoIjogIjclIiwiZGF0YVR5cGUiOiAic3RyaW5nIiwiaGxyZWdleCI6ICJ7XCIo
XFxcXGJIVFRQLzEuMFxcXFxiKVwiOiBcIjxiIGNsYXNzPSdjZWxsLWhsIGQxJz4kMTwvYj5cIixc
IihcXFxcYkhUVFAvMS4xXFxcXGIpXCI6IFwiPGIgY2xhc3M9J2NlbGwtaGwgZDInPiQxPC9iPlwi
LFwiKFxcXFxiSFRUUC8yXFxcXGIpXCI6IFwiPGIgY2xhc3M9J2NlbGwtaGwgZDMnPiQxPC9iPlwi
LFwiKFxcXFxiSFRUUC8zXFxcXGIpXCI6IFwiPGIgY2xhc3M9J2NlbGwtaGwgZDQnPiQxPC9iPlwi
fSIsImtleSI6ICJwcm90b2NvbCIsImxhYmVsIjogIlByb3RvY29sIn0seyJjbGFzc05hbWUiOiAi
dHJ1bmMiLCJjb2xXaWR0aCI6ICIxMDAlIiwibWV0YSI6ICJ1bmlxdWUiLCJtZXRhVHlwZSI6ICJu
dW1lcmljIiwibWV0YUxhYmVsIjogIlRvdGFsIiwiZGF0YVR5cGUiOiAic3RyaW5nIiwiaGxyZWdl
eCI6ICJ7XCJeKDFcXFxcZHsyfXwxeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhs
IGxibHUnPiQxPC9iPiQyXCIsXCJeKDJcXFxcZHsyfXwyeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNs
YXNzPSdzcGFuLWhsIGxncm4nPiQxPC9iPiQyXCIsXCJeKDNcXFxcZHsyfXwzeHgpKFxcXFxzLiop
JFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxwcnAnPiQxPC9iPiQyXCIsXCJeKDRcXFxcZHsyfXw0
eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGx5ZWwnPiQxPC9iPiQyXCIsXCJe
KDVcXFxcZHsyfXw1eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxyZWQnPiQx
PC9iPiQyXCIsXCJeKEFTXFxcXGQrKVwiOiBcIjxiPiQxPC9iPlwiLFwiXihcXFxcZCs6KVwiOiBc
IjxiPiQxPC9iPlwiLFwiKFxcXFxkKyl8KDpcXFxcZCspfCg6XFxcXGQrOlxcXFxkKylcIjogXCIk
MTxiPiQyPC9iPlwiLFwiXihbQS1aXXsyfSkoXFxcXHMuKiQpXCI6IFwiPGIgY2xhc3M9J3NwYW4t
aGwgZzUnPiQxPC9iPiQyXCJ9Iiwia2V5IjogImRhdGEiLCJsYWJlbCI6ICJEYXRhIn1dfSwibm90
X2ZvdW5kIjogeyJoZWFkIjogIk5vdCBGb3VuZCBVUkxzICg0MDRzKSIsImRlc2MiOiAiVG9wIG5v
dCBmb3VuZCBVUkxzIHNvcnRlZCBieSBoaXRzIFssIGF2Z3RzLCBjdW10cywgbWF4dHMsIG10aGQs
IHByb3RvXSIsImlkIjogIm5vdF9mb3VuZCIsInRhYmxlIjogMSwic29ydCI6IHsiZmllbGQiOiAi
aGl0cyIsIm9yZGVyIjogIkRFU0MifSwicGxvdCI6IFt7ImNsYXNzTmFtZSI6ICJoaXRzLXZpc2l0
b3JzIiwibGFiZWwiOiAiSGl0cy9WaXNpdG9ycyIsImNoYXJ0VHlwZSI6ICJiYXIiLCJjaGFydFJl
dmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDAsImQzIjogeyJ4IjogeyJrZXkiOiBbIm1ldGhv
ZCIsICJkYXRhIiwgInByb3RvY29sIl19LCJ5MCI6IHsia2V5IjogImhpdHMiLCJsYWJlbCI6ICJI
aXRzIn0sInkxIjogeyJrZXkiOiAidmlzaXRvcnMiLCJsYWJlbCI6ICJWaXNpdG9ycyJ9fX0seyJj
bGFzc05hbWUiOiAiYmFuZHdpZHRoIiwibGFiZWwiOiAiVHguIEFtb3VudCIsImNoYXJ0VHlwZSI6
ICJiYXIiLCJjaGFydFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDAsImQzIjogeyJ4Ijog
eyJrZXkiOiBbIm1ldGhvZCIsICJwcm90b2NvbCIsICJkYXRhIl19LCJ5MCI6IHsia2V5IjogImJ5
dGVzIiwibGFiZWwiOiAiVHguIEFtb3VudCIsImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVtcyI6
IFt7ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmlj
Iiwia2V5IjogImhpdHMiLCJsYWJlbCI6ICJIaXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRh
IjogImNvdW50IiwiZGF0YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxhYmVs
IjogIlZpc2l0b3JzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5
cGUiOiAiYnl0ZXMiLCJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJjb2xX
aWR0aCI6ICI2JSIsImRhdGFUeXBlIjogInN0cmluZyIsImhscmVnZXgiOiAie1wiKFxcXFxiW0Et
Wl17M31cXFxcYilcIjogXCI8YiBjbGFzcz0nY2VsbC1obCBiMSc+JDE8L2I+XCIsXCIoXFxcXGJb
QS1aXXs0fVxcXFxiKVwiOiBcIjxiIGNsYXNzPSdjZWxsLWhsIGIyJz4kMTwvYj5cIixcIihcXFxc
YltBLVpdezUsfVxcXFxiKVwiOiBcIjxiIGNsYXNzPSdjZWxsLWhsIGIzJz4kMTwvYj5cIn0iLCJr
ZXkiOiAibWV0aG9kIiwibGFiZWwiOiAiTWV0aG9kIn0seyJjb2xXaWR0aCI6ICI3JSIsImRhdGFU
eXBlIjogInN0cmluZyIsImhscmVnZXgiOiAie1wiKFxcXFxiSFRUUC8xLjBcXFxcYilcIjogXCI8
YiBjbGFzcz0nY2VsbC1obCBkMSc+JDE8L2I+XCIsXCIoXFxcXGJIVFRQLzEuMVxcXFxiKVwiOiBc
IjxiIGNsYXNzPSdjZWxsLWhsIGQyJz4kMTwvYj5cIixcIihcXFxcYkhUVFAvMlxcXFxiKVwiOiBc
IjxiIGNsYXNzPSdjZWxsLWhsIGQzJz4kMTwvYj5cIixcIihcXFxcYkhUVFAvM1xcXFxiKVwiOiBc
IjxiIGNsYXNzPSdjZWxsLWhsIGQ0Jz4kMTwvYj5cIn0iLCJrZXkiOiAicHJvdG9jb2wiLCJsYWJl
bCI6ICJQcm90b2NvbCJ9LHsiY2xhc3NOYW1lIjogInRydW5jIiwiY29sV2lkdGgiOiAiMTAwJSIs
Im1ldGEiOiAidW5pcXVlIiwibWV0YVR5cGUiOiAibnVtZXJpYyIsIm1ldGFMYWJlbCI6ICJUb3Rh
bCIsImRhdGFUeXBlIjogInN0cmluZyIsImhscmVnZXgiOiAie1wiXigxXFxcXGR7Mn18MXh4KShc
XFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBsYmx1Jz4kMTwvYj4kMlwiLFwiXigyXFxc
XGR7Mn18Mnh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBsZ3JuJz4kMTwvYj4k
MlwiLFwiXigzXFxcXGR7Mn18M3h4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBs
cHJwJz4kMTwvYj4kMlwiLFwiXig0XFxcXGR7Mn18NHh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFz
cz0nc3Bhbi1obCBseWVsJz4kMTwvYj4kMlwiLFwiXig1XFxcXGR7Mn18NXh4KShcXFxccy4qKSRc
IjogXCI8YiBjbGFzcz0nc3Bhbi1obCBscmVkJz4kMTwvYj4kMlwiLFwiXihBU1xcXFxkKylcIjog
XCI8Yj4kMTwvYj5cIixcIl4oXFxcXGQrOilcIjogXCI8Yj4kMTwvYj5cIixcIihcXFxcZCspfCg6
XFxcXGQrKXwoOlxcXFxkKzpcXFxcZCspXCI6IFwiJDE8Yj4kMjwvYj5cIixcIl4oW0EtWl17Mn0p
KFxcXFxzLiokKVwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGc1Jz4kMTwvYj4kMlwifSIsImtleSI6
ICJkYXRhIiwibGFiZWwiOiAiRGF0YSJ9XX0sImhvc3RzIjogeyJoZWFkIjogIlZpc2l0b3IgSG9z
dG5hbWVzIGFuZCBJUHMiLCJkZXNjIjogIlRvcCB2aXNpdG9yIGhvc3RzIHNvcnRlZCBieSBoaXRz
IFssIGF2Z3RzLCBjdW10cywgbWF4dHNdIiwiaWQiOiAiaG9zdHMiLCJ0YWJsZSI6IDEsInNvcnQi
OiB7ImZpZWxkIjogImhpdHMiLCJvcmRlciI6ICJERVNDIn0sInBsb3QiOiBbeyJjbGFzc05hbWUi
OiAiaGl0cy12aXNpdG9ycyIsImxhYmVsIjogIkhpdHMvVmlzaXRvcnMiLCJjaGFydFR5cGUiOiAi
YmFyIiwiY2hhcnRSZXZlcnNlIjogMCwicmVkcmF3T25FeHBhbmQiOiAwLCJkMyI6IHsieTAiOiB7
ImtleSI6ICJoaXRzIiwibGFiZWwiOiAiSGl0cyJ9LCJ5MSI6IHsia2V5IjogInZpc2l0b3JzIiwi
bGFiZWwiOiAiVmlzaXRvcnMifX19LHsiY2xhc3NOYW1lIjogImJhbmR3aWR0aCIsImxhYmVsIjog
IlR4LiBBbW91bnQiLCJjaGFydFR5cGUiOiAiYmFyIiwiY2hhcnRSZXZlcnNlIjogMCwicmVkcmF3
T25FeHBhbmQiOiAwLCJkMyI6IHsieTAiOiB7ImtleSI6ICJieXRlcyIsImxhYmVsIjogIlR4LiBB
bW91bnQiLCJmb3JtYXQiOiAiYnl0ZXMifX19XSwiaXRlbXMiOiBbeyJjb2xXaWR0aCI6ICIxMiUi
LCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJoaXRzIiwibGFi
ZWwiOiAiSGl0cyJ9LHsiY29sV2lkdGgiOiAiMTIlIiwibWV0YSI6ICJjb3VudCIsImRhdGFUeXBl
IjogIm51bWVyaWMiLCJrZXkiOiAidmlzaXRvcnMiLCJsYWJlbCI6ICJWaXNpdG9ycyJ9LHsiY29s
V2lkdGgiOiAiMTIlIiwibWV0YSI6ICJjb3VudCIsImRhdGFUeXBlIjogImJ5dGVzIiwia2V5Ijog
ImJ5dGVzIiwibGFiZWwiOiAiVHguIEFtb3VudCJ9LHsiY2xhc3NOYW1lIjogInRydW5jIiwiY29s
V2lkdGgiOiAiMTAwJSIsIm1ldGEiOiAidW5pcXVlIiwibWV0YVR5cGUiOiAibnVtZXJpYyIsIm1l
dGFMYWJlbCI6ICJUb3RhbCIsImRhdGFUeXBlIjogInN0cmluZyIsImhscmVnZXgiOiAie1wiXigx
XFxcXGR7Mn18MXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBsYmx1Jz4kMTwv
Yj4kMlwiLFwiXigyXFxcXGR7Mn18Mnh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1o
bCBsZ3JuJz4kMTwvYj4kMlwiLFwiXigzXFxcXGR7Mn18M3h4KShcXFxccy4qKSRcIjogXCI8YiBj
bGFzcz0nc3Bhbi1obCBscHJwJz4kMTwvYj4kMlwiLFwiXig0XFxcXGR7Mn18NHh4KShcXFxccy4q
KSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBseWVsJz4kMTwvYj4kMlwiLFwiXig1XFxcXGR7Mn18
NXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBscmVkJz4kMTwvYj4kMlwiLFwi
XihBU1xcXFxkKylcIjogXCI8Yj4kMTwvYj5cIixcIl4oXFxcXGQrOilcIjogXCI8Yj4kMTwvYj5c
IixcIihcXFxcZCspfCg6XFxcXGQrKXwoOlxcXFxkKzpcXFxcZCspXCI6IFwiJDE8Yj4kMjwvYj5c
IixcIl4oW0EtWl17Mn0pKFxcXFxzLiokKVwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGc1Jz4kMTwv
Yj4kMlwifSIsImtleSI6ICJkYXRhIiwibGFiZWwiOiAiRGF0YSJ9XX0sIm9zIjogeyJoZWFkIjog
Ik9wZXJhdGluZyBTeXN0ZW1zIiwiZGVzYyI6ICJUb3AgT3BlcmF0aW5nIFN5c3RlbXMgc29ydGVk
IGJ5IGhpdHMgWywgYXZndHMsIGN1bXRzLCBtYXh0c10iLCJpZCI6ICJvcyIsInRhYmxlIjogMSwi
c29ydCI6IHsiZmllbGQiOiAiaGl0cyIsIm9yZGVyIjogIkRFU0MifSwicGxvdCI6IFt7ImNsYXNz
TmFtZSI6ICJoaXRzLXZpc2l0b3JzIiwibGFiZWwiOiAiSGl0cy9WaXNpdG9ycyIsImNoYXJ0VHlw
ZSI6ICJiYXIiLCJjaGFydFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDEsImQzIjogeyJ5
MCI6IHsia2V5IjogImhpdHMiLCJsYWJlbCI6ICJIaXRzIn0sInkxIjogeyJrZXkiOiAidmlzaXRv
cnMiLCJsYWJlbCI6ICJWaXNpdG9ycyJ9fX0seyJjbGFzc05hbWUiOiAiYmFuZHdpZHRoIiwibGFi
ZWwiOiAiVHguIEFtb3VudCIsImNoYXJ0VHlwZSI6ICJiYXIiLCJjaGFydFJldmVyc2UiOiAwLCJy
ZWRyYXdPbkV4cGFuZCI6IDEsImQzIjogeyJ5MCI6IHsia2V5IjogImJ5dGVzIiwibGFiZWwiOiAi
VHguIEFtb3VudCIsImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVtcyI6IFt7ImNvbFdpZHRoIjog
IjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmljIiwia2V5IjogImhpdHMi
LCJsYWJlbCI6ICJIaXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0
YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxhYmVsIjogIlZpc2l0b3JzIn0s
eyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAiYnl0ZXMiLCJr
ZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJjbGFzc05hbWUiOiAidHJ1bmMi
LCJjb2xXaWR0aCI6ICIxMDAlIiwibWV0YSI6ICJ1bmlxdWUiLCJtZXRhVHlwZSI6ICJudW1lcmlj
IiwibWV0YUxhYmVsIjogIlRvdGFsIiwiZGF0YVR5cGUiOiAic3RyaW5nIiwiaGxyZWdleCI6ICJ7
XCJeKDFcXFxcZHsyfXwxeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxibHUn
PiQxPC9iPiQyXCIsXCJeKDJcXFxcZHsyfXwyeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdz
cGFuLWhsIGxncm4nPiQxPC9iPiQyXCIsXCJeKDNcXFxcZHsyfXwzeHgpKFxcXFxzLiopJFwiOiBc
IjxiIGNsYXNzPSdzcGFuLWhsIGxwcnAnPiQxPC9iPiQyXCIsXCJeKDRcXFxcZHsyfXw0eHgpKFxc
XFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGx5ZWwnPiQxPC9iPiQyXCIsXCJeKDVcXFxc
ZHsyfXw1eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxyZWQnPiQxPC9iPiQy
XCIsXCJeKEFTXFxcXGQrKVwiOiBcIjxiPiQxPC9iPlwiLFwiXihcXFxcZCs6KVwiOiBcIjxiPiQx
PC9iPlwiLFwiKFxcXFxkKyl8KDpcXFxcZCspfCg6XFxcXGQrOlxcXFxkKylcIjogXCIkMTxiPiQy
PC9iPlwiLFwiXihbQS1aXXsyfSkoXFxcXHMuKiQpXCI6IFwiPGIgY2xhc3M9J3NwYW4taGwgZzUn
PiQxPC9iPiQyXCJ9Iiwia2V5IjogImRhdGEiLCJsYWJlbCI6ICJEYXRhIn1dfSwiYnJvd3NlcnMi
OiB7ImhlYWQiOiAiQnJvd3NlcnMiLCJkZXNjIjogIlRvcCBCcm93c2VycyBzb3J0ZWQgYnkgaGl0
cyBbLCBhdmd0cywgY3VtdHMsIG1heHRzXSIsImlkIjogImJyb3dzZXJzIiwidGFibGUiOiAxLCJz
b3J0IjogeyJmaWVsZCI6ICJoaXRzIiwib3JkZXIiOiAiREVTQyJ9LCJwbG90IjogW3siY2xhc3NO
YW1lIjogImhpdHMtdmlzaXRvcnMiLCJsYWJlbCI6ICJIaXRzL1Zpc2l0b3JzIiwiY2hhcnRUeXBl
IjogImJhciIsImNoYXJ0UmV2ZXJzZSI6IDAsInJlZHJhd09uRXhwYW5kIjogMSwiZDMiOiB7Inkw
IjogeyJrZXkiOiAiaGl0cyIsImxhYmVsIjogIkhpdHMifSwieTEiOiB7ImtleSI6ICJ2aXNpdG9y
cyIsImxhYmVsIjogIlZpc2l0b3JzIn19fSx7ImNsYXNzTmFtZSI6ICJiYW5kd2lkdGgiLCJsYWJl
bCI6ICJUeC4gQW1vdW50IiwiY2hhcnRUeXBlIjogImJhciIsImNoYXJ0UmV2ZXJzZSI6IDAsInJl
ZHJhd09uRXhwYW5kIjogMSwiZDMiOiB7InkwIjogeyJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJU
eC4gQW1vdW50IiwiZm9ybWF0IjogImJ5dGVzIn19fV0sIml0ZW1zIjogW3siY29sV2lkdGgiOiAi
MTIlIiwibWV0YSI6ICJjb3VudCIsImRhdGFUeXBlIjogIm51bWVyaWMiLCJrZXkiOiAiaGl0cyIs
ImxhYmVsIjogIkhpdHMifSx7ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRh
VHlwZSI6ICJudW1lcmljIiwia2V5IjogInZpc2l0b3JzIiwibGFiZWwiOiAiVmlzaXRvcnMifSx7
ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJieXRlcyIsImtl
eSI6ICJieXRlcyIsImxhYmVsIjogIlR4LiBBbW91bnQifSx7ImNsYXNzTmFtZSI6ICJ0cnVuYyIs
ImNvbFdpZHRoIjogIjEwMCUiLCJtZXRhIjogInVuaXF1ZSIsIm1ldGFUeXBlIjogIm51bWVyaWMi
LCJtZXRhTGFiZWwiOiAiVG90YWwiLCJkYXRhVHlwZSI6ICJzdHJpbmciLCJobHJlZ2V4IjogIntc
Il4oMVxcXFxkezJ9fDF4eCkoXFxcXHMuKikkXCI6IFwiPGIgY2xhc3M9J3NwYW4taGwgbGJsdSc+
JDE8L2I+JDJcIixcIl4oMlxcXFxkezJ9fDJ4eCkoXFxcXHMuKikkXCI6IFwiPGIgY2xhc3M9J3Nw
YW4taGwgbGdybic+JDE8L2I+JDJcIixcIl4oM1xcXFxkezJ9fDN4eCkoXFxcXHMuKikkXCI6IFwi
PGIgY2xhc3M9J3NwYW4taGwgbHBycCc+JDE8L2I+JDJcIixcIl4oNFxcXFxkezJ9fDR4eCkoXFxc
XHMuKikkXCI6IFwiPGIgY2xhc3M9J3NwYW4taGwgbHllbCc+JDE8L2I+JDJcIixcIl4oNVxcXFxk
ezJ9fDV4eCkoXFxcXHMuKikkXCI6IFwiPGIgY2xhc3M9J3NwYW4taGwgbHJlZCc+JDE8L2I+JDJc
IixcIl4oQVNcXFxcZCspXCI6IFwiPGI+JDE8L2I+XCIsXCJeKFxcXFxkKzopXCI6IFwiPGI+JDE8
L2I+XCIsXCIoXFxcXGQrKXwoOlxcXFxkKyl8KDpcXFxcZCs6XFxcXGQrKVwiOiBcIiQxPGI+JDI8
L2I+XCIsXCJeKFtBLVpdezJ9KShcXFxccy4qJClcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBnNSc+
JDE8L2I+JDJcIn0iLCJrZXkiOiAiZGF0YSIsImxhYmVsIjogIkRhdGEifV19LCJ2aXNpdF90aW1l
IjogeyJoZWFkIjogIlRpbWUgRGlzdHJpYnV0aW9uIiwiZGVzYyI6ICJEYXRhIHNvcnRlZCBieSBo
b3VyIFssIGF2Z3RzLCBjdW10cywgbWF4dHNdIiwiaWQiOiAidmlzaXRfdGltZSIsInRhYmxlIjog
MSwic29ydCI6IHsiZmllbGQiOiAiZGF0YSIsIm9yZGVyIjogIkFTQyJ9LCJwbG90IjogW3siY2xh
c3NOYW1lIjogImhpdHMtdmlzaXRvcnMiLCJsYWJlbCI6ICJIaXRzL1Zpc2l0b3JzIiwiY2hhcnRU
eXBlIjogImFyZWEtc3BsaW5lIiwiY2hhcnRSZXZlcnNlIjogMCwicmVkcmF3T25FeHBhbmQiOiAx
LCJkMyI6IHsieTAiOiB7ImtleSI6ICJoaXRzIiwibGFiZWwiOiAiSGl0cyJ9LCJ5MSI6IHsia2V5
IjogInZpc2l0b3JzIiwibGFiZWwiOiAiVmlzaXRvcnMifX19LHsiY2xhc3NOYW1lIjogImJhbmR3
aWR0aCIsImxhYmVsIjogIlR4LiBBbW91bnQiLCJjaGFydFR5cGUiOiAiYXJlYS1zcGxpbmUiLCJj
aGFydFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDEsImQzIjogeyJ5MCI6IHsia2V5Ijog
ImJ5dGVzIiwibGFiZWwiOiAiVHguIEFtb3VudCIsImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVt
cyI6IFt7ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEiOiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1l
cmljIiwia2V5IjogImhpdHMiLCJsYWJlbCI6ICJIaXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJt
ZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAibnVtZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxh
YmVsIjogIlZpc2l0b3JzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0
YVR5cGUiOiAiYnl0ZXMiLCJrZXkiOiAiYnl0ZXMiLCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJj
bGFzc05hbWUiOiAidHJ1bmMiLCJjb2xXaWR0aCI6ICIxMDAlIiwibWV0YSI6ICJ1bmlxdWUiLCJt
ZXRhVHlwZSI6ICJudW1lcmljIiwibWV0YUxhYmVsIjogIlRvdGFsIiwiZGF0YVR5cGUiOiAic3Ry
aW5nIiwiaGxyZWdleCI6ICJ7XCJeKDFcXFxcZHsyfXwxeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNs
YXNzPSdzcGFuLWhsIGxibHUnPiQxPC9iPiQyXCIsXCJeKDJcXFxcZHsyfXwyeHgpKFxcXFxzLiop
JFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxncm4nPiQxPC9iPiQyXCIsXCJeKDNcXFxcZHsyfXwz
eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxwcnAnPiQxPC9iPiQyXCIsXCJe
KDRcXFxcZHsyfXw0eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGx5ZWwnPiQx
PC9iPiQyXCIsXCJeKDVcXFxcZHsyfXw1eHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFu
LWhsIGxyZWQnPiQxPC9iPiQyXCIsXCJeKEFTXFxcXGQrKVwiOiBcIjxiPiQxPC9iPlwiLFwiXihc
XFxcZCs6KVwiOiBcIjxiPiQxPC9iPlwiLFwiKFxcXFxkKyl8KDpcXFxcZCspfCg6XFxcXGQrOlxc
XFxkKylcIjogXCIkMTxiPiQyPC9iPlwiLFwiXihbQS1aXXsyfSkoXFxcXHMuKiQpXCI6IFwiPGIg
Y2xhc3M9J3NwYW4taGwgZzUnPiQxPC9iPiQyXCJ9Iiwia2V5IjogImRhdGEiLCJsYWJlbCI6ICJE
YXRhIn1dfSwicmVmZXJyaW5nX3NpdGVzIjogeyJoZWFkIjogIlJlZmVycmluZyBTaXRlcyIsImRl
c2MiOiAiVG9wIFJlZmVycmluZyBTaXRlcyBzb3J0ZWQgYnkgaGl0cyBbLCBhdmd0cywgY3VtdHMs
IG1heHRzXSIsImlkIjogInJlZmVycmluZ19zaXRlcyIsInRhYmxlIjogMSwic29ydCI6IHsiZmll
bGQiOiAiaGl0cyIsIm9yZGVyIjogIkRFU0MifSwicGxvdCI6IFt7ImNsYXNzTmFtZSI6ICJoaXRz
LXZpc2l0b3JzIiwibGFiZWwiOiAiSGl0cy9WaXNpdG9ycyIsImNoYXJ0VHlwZSI6ICJiYXIiLCJj
aGFydFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFuZCI6IDAsImQzIjogeyJ5MCI6IHsia2V5Ijog
ImhpdHMiLCJsYWJlbCI6ICJIaXRzIn0sInkxIjogeyJrZXkiOiAidmlzaXRvcnMiLCJsYWJlbCI6
ICJWaXNpdG9ycyJ9fX0seyJjbGFzc05hbWUiOiAiYmFuZHdpZHRoIiwibGFiZWwiOiAiVHguIEFt
b3VudCIsImNoYXJ0VHlwZSI6ICJiYXIiLCJjaGFydFJldmVyc2UiOiAwLCJyZWRyYXdPbkV4cGFu
ZCI6IDAsImQzIjogeyJ5MCI6IHsia2V5IjogImJ5dGVzIiwibGFiZWwiOiAiVHguIEFtb3VudCIs
ImZvcm1hdCI6ICJieXRlcyJ9fX1dLCJpdGVtcyI6IFt7ImNvbFdpZHRoIjogIjEyJSIsIm1ldGEi
OiAiY291bnQiLCJkYXRhVHlwZSI6ICJudW1lcmljIiwia2V5IjogImhpdHMiLCJsYWJlbCI6ICJI
aXRzIn0seyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAibnVt
ZXJpYyIsImtleSI6ICJ2aXNpdG9ycyIsImxhYmVsIjogIlZpc2l0b3JzIn0seyJjb2xXaWR0aCI6
ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAiYnl0ZXMiLCJrZXkiOiAiYnl0ZXMi
LCJsYWJlbCI6ICJUeC4gQW1vdW50In0seyJjbGFzc05hbWUiOiAidHJ1bmMiLCJjb2xXaWR0aCI6
ICIxMDAlIiwibWV0YSI6ICJ1bmlxdWUiLCJtZXRhVHlwZSI6ICJudW1lcmljIiwibWV0YUxhYmVs
IjogIlRvdGFsIiwiZGF0YVR5cGUiOiAic3RyaW5nIiwiaGxyZWdleCI6ICJ7XCJeKDFcXFxcZHsy
fXwxeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxibHUnPiQxPC9iPiQyXCIs
XCJeKDJcXFxcZHsyfXwyeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxncm4n
PiQxPC9iPiQyXCIsXCJeKDNcXFxcZHsyfXwzeHgpKFxcXFxzLiopJFwiOiBcIjxiIGNsYXNzPSdz
cGFuLWhsIGxwcnAnPiQxPC9iPiQyXCIsXCJeKDRcXFxcZHsyfXw0eHgpKFxcXFxzLiopJFwiOiBc
IjxiIGNsYXNzPSdzcGFuLWhsIGx5ZWwnPiQxPC9iPiQyXCIsXCJeKDVcXFxcZHsyfXw1eHgpKFxc
XFxzLiopJFwiOiBcIjxiIGNsYXNzPSdzcGFuLWhsIGxyZWQnPiQxPC9iPiQyXCIsXCJeKEFTXFxc
XGQrKVwiOiBcIjxiPiQxPC9iPlwiLFwiXihcXFxcZCs6KVwiOiBcIjxiPiQxPC9iPlwiLFwiKFxc
XFxkKyl8KDpcXFxcZCspfCg6XFxcXGQrOlxcXFxkKylcIjogXCIkMTxiPiQyPC9iPlwiLFwiXihb
QS1aXXsyfSkoXFxcXHMuKiQpXCI6IFwiPGIgY2xhc3M9J3NwYW4taGwgZzUnPiQxPC9iPiQyXCJ9
Iiwia2V5IjogImRhdGEiLCJsYWJlbCI6ICJEYXRhIn1dfSwic3RhdHVzX2NvZGVzIjogeyJoZWFk
IjogIkhUVFAgU3RhdHVzIENvZGVzIiwiZGVzYyI6ICJUb3AgSFRUUCBTdGF0dXMgQ29kZXMgc29y
dGVkIGJ5IGhpdHMgWywgYXZndHMsIGN1bXRzLCBtYXh0c10iLCJpZCI6ICJzdGF0dXNfY29kZXMi
LCJ0YWJsZSI6IDEsInNvcnQiOiB7ImZpZWxkIjogImhpdHMiLCJvcmRlciI6ICJERVNDIn0sInBs
b3QiOiBbeyJjbGFzc05hbWUiOiAiaGl0cy12aXNpdG9ycyIsImxhYmVsIjogIkhpdHMvVmlzaXRv
cnMiLCJjaGFydFR5cGUiOiAiYmFyIiwiY2hhcnRSZXZlcnNlIjogMCwicmVkcmF3T25FeHBhbmQi
OiAxLCJkMyI6IHsieTAiOiB7ImtleSI6ICJoaXRzIiwibGFiZWwiOiAiSGl0cyJ9LCJ5MSI6IHsi
a2V5IjogInZpc2l0b3JzIiwibGFiZWwiOiAiVmlzaXRvcnMifX19LHsiY2xhc3NOYW1lIjogImJh
bmR3aWR0aCIsImxhYmVsIjogIlR4LiBBbW91bnQiLCJjaGFydFR5cGUiOiAiYmFyIiwiY2hhcnRS
ZXZlcnNlIjogMCwicmVkcmF3T25FeHBhbmQiOiAxLCJkMyI6IHsieTAiOiB7ImtleSI6ICJieXRl
cyIsImxhYmVsIjogIlR4LiBBbW91bnQiLCJmb3JtYXQiOiAiYnl0ZXMifX19XSwiaXRlbXMiOiBb
eyJjb2xXaWR0aCI6ICIxMiUiLCJtZXRhIjogImNvdW50IiwiZGF0YVR5cGUiOiAibnVtZXJpYyIs
ImtleSI6ICJoaXRzIiwibGFiZWwiOiAiSGl0cyJ9LHsiY29sV2lkdGgiOiAiMTIlIiwibWV0YSI6
ICJjb3VudCIsImRhdGFUeXBlIjogIm51bWVyaWMiLCJrZXkiOiAidmlzaXRvcnMiLCJsYWJlbCI6
ICJWaXNpdG9ycyJ9LHsiY29sV2lkdGgiOiAiMTIlIiwibWV0YSI6ICJjb3VudCIsImRhdGFUeXBl
IjogImJ5dGVzIiwia2V5IjogImJ5dGVzIiwibGFiZWwiOiAiVHguIEFtb3VudCJ9LHsiY2xhc3NO
YW1lIjogInRydW5jIiwiY29sV2lkdGgiOiAiMTAwJSIsIm1ldGEiOiAidW5pcXVlIiwibWV0YVR5
cGUiOiAibnVtZXJpYyIsIm1ldGFMYWJlbCI6ICJUb3RhbCIsImRhdGFUeXBlIjogInN0cmluZyIs
ImhscmVnZXgiOiAie1wiXigxXFxcXGR7Mn18MXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0n
c3Bhbi1obCBsYmx1Jz4kMTwvYj4kMlwiLFwiXigyXFxcXGR7Mn18Mnh4KShcXFxccy4qKSRcIjog
XCI8YiBjbGFzcz0nc3Bhbi1obCBsZ3JuJz4kMTwvYj4kMlwiLFwiXigzXFxcXGR7Mn18M3h4KShc
XFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBscHJwJz4kMTwvYj4kMlwiLFwiXig0XFxc
XGR7Mn18NHh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBseWVsJz4kMTwvYj4k
MlwiLFwiXig1XFxcXGR7Mn18NXh4KShcXFxccy4qKSRcIjogXCI8YiBjbGFzcz0nc3Bhbi1obCBs
cmVkJz4kMTwvYj4kMlwiLFwiXihBU1xcXFxkKylcIjogXCI8Yj4kMTwvYj5cIixcIl4oXFxcXGQr
OilcIjogXCI8Yj4kMTwvYj5cIixcIihcXFxcZCspfCg6XFxcXGQrKXwoOlxcXFxkKzpcXFxcZCsp
XCI6IFwiJDE8Yj4kMjwvYj5cIixcIl4oW0EtWl17Mn0pKFxcXFxzLiokKVwiOiBcIjxiIGNsYXNz
PSdzcGFuLWhsIGc1Jz4kMTwvYj4kMlwifSIsImtleSI6ICJkYXRhIiwibGFiZWwiOiAiRGF0YSJ9
XX0sfTwvc2NyaXB0PjxzY3JpcHQgdHlwZT0ndGV4dC9qYXZhc2NyaXB0Jz52YXIganNvbl9kYXRh
PXsiZ2VuZXJhbCI6IHsic3RhcnRfZGF0ZSI6ICIwNi9GZWIvMjAyNCIsImVuZF9kYXRlIjogIjE5
L0ZlYi8yMDI0IiwiZGF0ZV90aW1lIjogIjIwMjQtMDItMTkgMTU6NTI6NDMgKzA4MDAiLCJ0b3Rh
bF9yZXF1ZXN0cyI6IDI1NTMsInZhbGlkX3JlcXVlc3RzIjogMjU1MywiZmFpbGVkX3JlcXVlc3Rz
IjogMCwiZ2VuZXJhdGlvbl90aW1lIjogMSwidW5pcXVlX3Zpc2l0b3JzIjogMTM0NywidW5pcXVl
X2ZpbGVzIjogMCwiZXhjbHVkZWRfaGl0cyI6IDAsInVuaXF1ZV9yZWZlcnJlcnMiOiAwLCJ1bmlx
dWVfbm90X2ZvdW5kIjogMiwidW5pcXVlX3N0YXRpY19maWxlcyI6IDE3MiwibG9nX3NpemUiOiA1
OTgwNjEsImJhbmR3aWR0aCI6IDQ2NDIxODM1MTQxLCJsb2dfcGF0aCI6IFsiZ2V0cmVsZWFzZXhl
bnRnei5sb2ciXX0sInZpc2l0b3JzIjogeyJtZXRhZGF0YSI6IHsiYnl0ZXMiOiB7InRvdGFsIjog
eyJ2YWx1ZSI6IDQ2NDIxODM1MTQxfSwiYXZnIjogeyJ2YWx1ZSI6IDMzMTU4NDUzNzYsInBlcmNl
bnQiOiAiMDcuMTQifSwibWF4IjogeyJ2YWx1ZSI6IDQ5ODE4NjQwNTMsInBlcmNlbnQiOiAiMTAu
NzMifSwibWluIjogeyJ2YWx1ZSI6IDUzMDMwMDI4NCwicGVyY2VudCI6ICIwMS4xNCJ9fSwidmlz
aXRvcnMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDEzNDd9LCJhdmciOiB7InZhbHVlIjogOTYsInBl
cmNlbnQiOiAiMDcuMTMifSwibWF4IjogeyJ2YWx1ZSI6IDEzNCwicGVyY2VudCI6ICIwOS45NSJ9
LCJtaW4iOiB7InZhbHVlIjogMTUsInBlcmNlbnQiOiAiMDEuMTEifX0sImhpdHMiOiB7InRvdGFs
IjogeyJ2YWx1ZSI6IDI1NTN9LCJhdmciOiB7InZhbHVlIjogMTgyLCJwZXJjZW50IjogIjA3LjEz
In0sIm1heCI6IHsidmFsdWUiOiAzMjEsInBlcmNlbnQiOiAiMTIuNTcifSwibWluIjogeyJ2YWx1
ZSI6IDE4LCJwZXJjZW50IjogIjAwLjcxIn19LCJkYXRhIjogeyJ0b3RhbCI6IHsJInZhbHVlIjog
MTR9fX0sImRhdGEiOiBbeyJoaXRzIjogeyJjb3VudCI6IDE4LCJwZXJjZW50IjogIjAwLjcxIn0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDE1LCJwZXJjZW50IjogIjAxLjExIn0sImJ5dGVzIjogeyJj
b3VudCI6IDUzMDMwMDI4NCwicGVyY2VudCI6ICIwMS4xNCJ9LCJkYXRhIjogIjIwMjQwMjE5In0s
eyJoaXRzIjogeyJjb3VudCI6IDEzNiwicGVyY2VudCI6ICIwNS4zMyJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiA4OCwicGVyY2VudCI6ICIwNi41MyJ9LCJieXRlcyI6IHsiY291bnQiOiAzMTM2Njgy
NjM5LCJwZXJjZW50IjogIjA2Ljc2In0sImRhdGEiOiAiMjAyNDAyMTgifSx7ImhpdHMiOiB7ImNv
dW50IjogMTczLCJwZXJjZW50IjogIjA2Ljc4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEwNywi
cGVyY2VudCI6ICIwNy45NCJ9LCJieXRlcyI6IHsiY291bnQiOiAzMzU4OTczODQyLCJwZXJjZW50
IjogIjA3LjI0In0sImRhdGEiOiAiMjAyNDAyMTcifSx7ImhpdHMiOiB7ImNvdW50IjogMjg5LCJw
ZXJjZW50IjogIjExLjMyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEzNCwicGVyY2VudCI6ICIw
OS45NSJ9LCJieXRlcyI6IHsiY291bnQiOiA0NDM5NjU5NzQ2LCJwZXJjZW50IjogIjA5LjU2In0s
ImRhdGEiOiAiMjAyNDAyMTYifSx7ImhpdHMiOiB7ImNvdW50IjogMTk0LCJwZXJjZW50IjogIjA3
LjYwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEyNiwicGVyY2VudCI6ICIwOS4zNSJ9LCJieXRl
cyI6IHsiY291bnQiOiAzODY0NzU1MjExLCJwZXJjZW50IjogIjA4LjMzIn0sImRhdGEiOiAiMjAy
NDAyMTUifSx7ImhpdHMiOiB7ImNvdW50IjogMjE4LCJwZXJjZW50IjogIjA4LjU0In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDExOCwicGVyY2VudCI6ICIwOC43NiJ9LCJieXRlcyI6IHsiY291bnQi
OiA0NjU0Njg2NzAzLCJwZXJjZW50IjogIjEwLjAzIn0sImRhdGEiOiAiMjAyNDAyMTQifSx7Imhp
dHMiOiB7ImNvdW50IjogMTk0LCJwZXJjZW50IjogIjA3LjYwIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEyNywicGVyY2VudCI6ICIwOS40MyJ9LCJieXRlcyI6IHsiY291bnQiOiA0OTgxODY0MDUz
LCJwZXJjZW50IjogIjEwLjczIn0sImRhdGEiOiAiMjAyNDAyMTMifSx7ImhpdHMiOiB7ImNvdW50
IjogMTcxLCJwZXJjZW50IjogIjA2LjcwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDg2LCJwZXJj
ZW50IjogIjA2LjM4In0sImJ5dGVzIjogeyJjb3VudCI6IDMyNDQxNDgyNTMsInBlcmNlbnQiOiAi
MDYuOTkifSwiZGF0YSI6ICIyMDI0MDIxMiJ9LHsiaGl0cyI6IHsiY291bnQiOiAxOTUsInBlcmNl
bnQiOiAiMDcuNjQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNzIsInBlcmNlbnQiOiAiMDUuMzUi
fSwiYnl0ZXMiOiB7ImNvdW50IjogMzIxMDQ3MDEzMSwicGVyY2VudCI6ICIwNi45MiJ9LCJkYXRh
IjogIjIwMjQwMjExIn0seyJoaXRzIjogeyJjb3VudCI6IDEyMywicGVyY2VudCI6ICIwNC44MiJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA3OSwicGVyY2VudCI6ICIwNS44NiJ9LCJieXRlcyI6IHsi
Y291bnQiOiAyODcwNDQ4ODI4LCJwZXJjZW50IjogIjA2LjE4In0sImRhdGEiOiAiMjAyNDAyMTAi
fSx7ImhpdHMiOiB7ImNvdW50IjogMTM0LCJwZXJjZW50IjogIjA1LjI1In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDk0LCJwZXJjZW50IjogIjA2Ljk4In0sImJ5dGVzIjogeyJjb3VudCI6IDMyNDEz
NjQ3ODUsInBlcmNlbnQiOiAiMDYuOTgifSwiZGF0YSI6ICIyMDI0MDIwOSJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAxNDcsInBlcmNlbnQiOiAiMDUuNzYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogODAs
InBlcmNlbnQiOiAiMDUuOTQifSwiYnl0ZXMiOiB7ImNvdW50IjogMjgxMjA2NjE4MywicGVyY2Vu
dCI6ICIwNi4wNiJ9LCJkYXRhIjogIjIwMjQwMjA4In0seyJoaXRzIjogeyJjb3VudCI6IDI0MCwi
cGVyY2VudCI6ICIwOS40MCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxMDksInBlcmNlbnQiOiAi
MDguMDkifSwiYnl0ZXMiOiB7ImNvdW50IjogMzEzMTM4OTM1NywicGVyY2VudCI6ICIwNi43NSJ9
LCJkYXRhIjogIjIwMjQwMjA3In0seyJoaXRzIjogeyJjb3VudCI6IDMyMSwicGVyY2VudCI6ICIx
Mi41NyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxMTIsInBlcmNlbnQiOiAiMDguMzEifSwiYnl0
ZXMiOiB7ImNvdW50IjogMjk0NTAyNTEyNiwicGVyY2VudCI6ICIwNi4zNCJ9LCJkYXRhIjogIjIw
MjQwMjA2In1dfSwicmVxdWVzdHMiOiB7Im1ldGFkYXRhIjogeyJieXRlcyI6IHsidG90YWwiOiB7
InZhbHVlIjogMH0sImF2ZyI6IHsidmFsdWUiOiAwLCJwZXJjZW50IjogIjAwLjAwIn0sIm1heCI6
IHsidmFsdWUiOiAwLCJwZXJjZW50IjogIjAwLjAwIn0sIm1pbiI6IHsidmFsdWUiOiAwLCJwZXJj
ZW50IjogIjAwLjAwIn19LCJ2aXNpdG9ycyI6IHsidG90YWwiOiB7InZhbHVlIjogMH0sImF2ZyI6
IHsidmFsdWUiOiAwLCJwZXJjZW50IjogIjAwLjAwIn0sIm1heCI6IHsidmFsdWUiOiAwLCJwZXJj
ZW50IjogIjAwLjAwIn0sIm1pbiI6IHsidmFsdWUiOiAwLCJwZXJjZW50IjogIjAwLjAwIn19LCJo
aXRzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiAwfSwiYXZnIjogeyJ2YWx1ZSI6IDAsInBlcmNlbnQi
OiAiMDAuMDAifSwibWF4IjogeyJ2YWx1ZSI6IDAsInBlcmNlbnQiOiAiMDAuMDAifSwibWluIjog
eyJ2YWx1ZSI6IDAsInBlcmNlbnQiOiAiMDAuMDAifX0sImRhdGEiOiB7InRvdGFsIjogewkidmFs
dWUiOiAwfX19LCJkYXRhIjogW119LCJzdGF0aWNfcmVxdWVzdHMiOiB7Im1ldGFkYXRhIjogeyJi
eXRlcyI6IHsidG90YWwiOiB7InZhbHVlIjogNDY0MjE4MTc5MjZ9LCJhdmciOiB7InZhbHVlIjog
MjY5ODk0MzA0LCJwZXJjZW50IjogIjAwLjU4In0sIm1heCI6IHsidmFsdWUiOiAzMjA3NDE2NDg3
NiwicGVyY2VudCI6ICI2OS4wOSJ9LCJtaW4iOiB7InZhbHVlIjogNTgyNywicGVyY2VudCI6ICIw
MC4wMCJ9fSwidmlzaXRvcnMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDE4NzR9LCJhdmciOiB7InZh
bHVlIjogMTAsInBlcmNlbnQiOiAiMDAuNTMifSwibWF4IjogeyJ2YWx1ZSI6IDcwNCwicGVyY2Vu
dCI6ICIzNy41NyJ9LCJtaW4iOiB7InZhbHVlIjogMSwicGVyY2VudCI6ICIwMC4wNSJ9fSwiaGl0
cyI6IHsidG90YWwiOiB7InZhbHVlIjogMjU1MH0sImF2ZyI6IHsidmFsdWUiOiAxNCwicGVyY2Vu
dCI6ICIwMC41NSJ9LCJtYXgiOiB7InZhbHVlIjogMTE3MiwicGVyY2VudCI6ICI0NS45NiJ9LCJt
aW4iOiB7InZhbHVlIjogMSwicGVyY2VudCI6ICIwMC4wNCJ9fSwiZGF0YSI6IHsidG90YWwiOiB7
CSJ2YWx1ZSI6IDE3Mn19fSwiZGF0YSI6IFt7ImhpdHMiOiB7ImNvdW50IjogMTE3MiwicGVyY2Vu
dCI6ICI0NS45MSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA3MDQsInBlcmNlbnQiOiAiNTIuMjYi
fSwiYnl0ZXMiOiB7ImNvdW50IjogMzIwNzQxNjQ4NzYsInBlcmNlbnQiOiAiNjkuMDkifSwibWV0
aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hl
blwvNC4xOC4wXC94ZW4tNC4xOC4wLnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAzNTksInBl
cmNlbnQiOiAiMTQuMDYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMzQ3LCJwZXJjZW50IjogIjI1
Ljc2In0sImJ5dGVzIjogeyJjb3VudCI6IDE5MTcxNzQsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0
aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hl
blwvNC4xOC4wXC94ZW4tNC4xOC4wLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogMjEw
LCJwZXJjZW50IjogIjA4LjIzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEzNSwicGVyY2VudCI6
ICIxMC4wMiJ9LCJieXRlcyI6IHsiY291bnQiOiA0NjgzMzk3NzA2LCJwZXJjZW50IjogIjEwLjA5
In0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxl
YXNlXC94ZW5cLzQuOC41XC94ZW4tNC44LjUudGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6IDQx
LCJwZXJjZW50IjogIjAxLjYxIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDI2LCJwZXJjZW50Ijog
IjAxLjkzIn0sImJ5dGVzIjogeyJjb3VudCI6IDk2MDg2NzQwMSwicGVyY2VudCI6ICIwMi4wNyJ9
LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFz
ZVwveGVuXC80LjkuMlwveGVuLTQuOS4yLnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAzNywi
cGVyY2VudCI6ICIwMS40NSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyOSwicGVyY2VudCI6ICIw
Mi4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxNjMwODMwMTQ2LCJwZXJjZW50IjogIjAzLjUxIn0s
Im1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNl
XC94ZW5cLzQuMTcuM1wveGVuLTQuMTcuMy50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMzQs
InBlcmNlbnQiOiAiMDEuMzMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMjYsInBlcmNlbnQiOiAi
MDEuOTMifSwiYnl0ZXMiOiB7ImNvdW50IjogMTk5Njg0LCJwZXJjZW50IjogIjAwLjAwIn0sIm1l
dGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94
ZW5cLzQuMTcuM1wveGVuLTQuMTcuMy50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDMy
LCJwZXJjZW50IjogIjAxLjI1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDE2LCJwZXJjZW50Ijog
IjAxLjE5In0sImJ5dGVzIjogeyJjb3VudCI6IDU1ODQ3NTUwNiwicGVyY2VudCI6ICIwMS4yMCJ9
LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFz
ZVwveGVuXC80LjE3LjJcL3hlbi00LjE3LjIudGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6IDMx
LCJwZXJjZW50IjogIjAxLjIxIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDI4LCJwZXJjZW50Ijog
IjAyLjA4In0sImJ5dGVzIjogeyJjb3VudCI6IDExOTA2NjcxMjYsInBlcmNlbnQiOiAiMDIuNTYi
fSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVh
c2VcL3hlblwvNC4xNS4wXC94ZW4tNC4xNS4wLnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
OSwicGVyY2VudCI6ICIwMS4xNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyNywicGVyY2VudCI6
ICIwMi4wMCJ9LCJieXRlcyI6IHsiY291bnQiOiAxMjYxNDE5ODkwLCJwZXJjZW50IjogIjAyLjcy
In0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxl
YXNlXC94ZW5cLzQuMTYuMVwveGVuLTQuMTYuMS50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50Ijog
MjcsInBlcmNlbnQiOiAiMDEuMDYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMTQsInBlcmNlbnQi
OiAiMDEuMDQifSwiYnl0ZXMiOiB7ImNvdW50IjogNjQ0MjU1NzQ4LCJwZXJjZW50IjogIjAxLjM5
In0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxl
YXNlXC94ZW5cLzQuMTAuNFwveGVuLTQuMTAuNC50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50Ijog
MTcsInBlcmNlbnQiOiAiMDAuNjcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogOCwicGVyY2VudCI6
ICIwMC41OSJ9LCJieXRlcyI6IHsiY291bnQiOiA1Mjg2NDgwNzYsInBlcmNlbnQiOiAiMDEuMTQi
fSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVh
c2VcL3hlblwvNC4xNy4wXC94ZW4tNC4xNy4wLnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAx
MywicGVyY2VudCI6ICIwMC41MSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA4LCJwZXJjZW50Ijog
IjAwLjU5In0sImJ5dGVzIjogeyJjb3VudCI6IDUzMTk4NzM4LCJwZXJjZW50IjogIjAwLjExIn0s
Im1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNl
XC94ZW5cLzQuMTEuMFwveGVuLTQuMTEuMC50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMTEs
InBlcmNlbnQiOiAiMDAuNDMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMTEsInBlcmNlbnQiOiAi
MDAuODIifSwiYnl0ZXMiOiB7ImNvdW50IjogMjE1ODY5NzkyLCJwZXJjZW50IjogIjAwLjQ3In0s
Im1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNl
XC94ZW5cLzQuMTUuNVwveGVuLTQuMTUuNS50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogOSwi
cGVyY2VudCI6ICIwMC4zNSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA5LCJwZXJjZW50IjogIjAw
LjY3In0sImJ5dGVzIjogeyJjb3VudCI6IDIxNjE3ODA5NSwicGVyY2VudCI6ICIwMC40NyJ9LCJt
ZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwv
eGVuXC80LjEyLjBcL3hlbi00LjEyLjAudGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6IDksInBl
cmNlbnQiOiAiMDAuMzUifSwidmlzaXRvcnMiOiB7ImNvdW50IjogOSwicGVyY2VudCI6ICIwMC42
NyJ9LCJieXRlcyI6IHsiY291bnQiOiAyNDA0NDY2MTEsInBlcmNlbnQiOiAiMDAuNTIifSwibWV0
aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hl
blwvNC4xNi41XC94ZW4tNC4xNi41LnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiA4LCJwZXJj
ZW50IjogIjAwLjMxIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDYsInBlcmNlbnQiOiAiMDAuNDUi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNDY4MTUsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjog
IkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4x
NC42XC94ZW4tNC4xNC42LnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogOCwicGVyY2Vu
dCI6ICIwMC4zMSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA4LCJwZXJjZW50IjogIjAwLjU5In0s
ImJ5dGVzIjogeyJjb3VudCI6IDQwNzYzLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJH
RVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMS42
LjFcL3hlbi00LjEuNi4xLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogNywicGVyY2Vu
dCI6ICIwMC4yNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA2LCJwZXJjZW50IjogIjAwLjQ1In0s
ImJ5dGVzIjogeyJjb3VudCI6IDQwODMzLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJH
RVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuNy4w
XC94ZW4tNC43LjAudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA3LCJwZXJjZW50Ijog
IjAwLjI3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDcsInBlcmNlbnQiOiAiMDAuNTIifSwiYnl0
ZXMiOiB7ImNvdW50IjogMjEwMTc1NDgyLCJwZXJjZW50IjogIjAwLjQ1In0sIm1ldGhvZCI6ICJH
RVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTQu
NlwveGVuLTQuMTQuNi50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogNywicGVyY2VudCI6ICIw
MC4yNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA2LCJwZXJjZW50IjogIjAwLjQ1In0sImJ5dGVz
IjogeyJjb3VudCI6IDMwNzMxLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJw
cm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTMuMFwveGVu
LTQuMTMuMC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDcsInBlcmNlbnQiOiAiMDAu
MjcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNywicGVyY2VudCI6ICIwMC41MiJ9LCJieXRlcyI6
IHsiY291bnQiOiAzNDY1OSwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJv
dG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE3LjBcL3hlbi00
LjE3LjAudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA3LCJwZXJjZW50IjogIjAwLjI3
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDcsInBlcmNlbnQiOiAiMDAuNTIifSwiYnl0ZXMiOiB7
ImNvdW50IjogNDA5NzksInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3Rv
Y29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xNi41XC94ZW4tNC4x
Ni41LnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogNywicGVyY2VudCI6ICIwMC4yNyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjM3In0sImJ5dGVzIjogeyJj
b3VudCI6IDI0MjcyOTA4MCwicGVyY2VudCI6ICIwMC41MiJ9LCJtZXRob2QiOiAiR0VUIiwicHJv
dG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE3LjFcL3hlbi00
LjE3LjEudGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6IDYsInBlcmNlbnQiOiAiMDAuMjQifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291
bnQiOiAzNTExNCwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wi
OiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE3LjFcL3hlbi00LjE3LjEu
dGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA2LCJwZXJjZW50IjogIjAwLjI0In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDYsInBlcmNlbnQiOiAiMDAuNDUifSwiYnl0ZXMiOiB7ImNvdW50
IjogMzUwODksInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjog
IkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC43LjZcL3hlbi00LjcuNi50YXIu
Z3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDYsInBlcmNlbnQiOiAiMDAuMjQifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogNiwicGVyY2VudCI6ICIwMC40NSJ9LCJieXRlcyI6IHsiY291bnQiOiAz
NTIxNCwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRU
UC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE3LjJcL3hlbi00LjE3LjIudGFyLmd6
LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA2LCJwZXJjZW50IjogIjAwLjI0In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDYsInBlcmNlbnQiOiAiMDAuNDUifSwiYnl0ZXMiOiB7ImNvdW50IjogMzUx
OTUsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAv
MS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xNi4yXC94ZW4tNC4xNi4yLnRhci5nei5z
aWcifSx7ImhpdHMiOiB7ImNvdW50IjogNiwicGVyY2VudCI6ICIwMC4yNCJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjM3In0sImJ5dGVzIjogeyJjb3VudCI6IDI0MDEz
NTIzMywicGVyY2VudCI6ICIwMC41MiJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRU
UC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE0LjVcL3hlbi00LjE0LjUudGFyLmd6
In0seyJoaXRzIjogeyJjb3VudCI6IDYsInBlcmNlbnQiOiAiMDAuMjQifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogNiwicGVyY2VudCI6ICIwMC40NSJ9LCJieXRlcyI6IHsiY291bnQiOiAzNTE4NSwi
cGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEi
LCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE1LjVcL3hlbi00LjE1LjUudGFyLmd6LnNpZyJ9
LHsiaGl0cyI6IHsiY291bnQiOiA2LCJwZXJjZW50IjogIjAwLjI0In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDYsInBlcmNlbnQiOiAiMDAuNDUifSwiYnl0ZXMiOiB7ImNvdW50IjogMzUwMDUsInBl
cmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwi
ZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4yLjFcL3hlbi00LjIuMS50YXIuZ3ouc2lnIn0seyJo
aXRzIjogeyJjb3VudCI6IDYsInBlcmNlbnQiOiAiMDAuMjQifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogNSwicGVyY2VudCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291bnQiOiAzNTIxMiwicGVyY2Vu
dCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRh
IjogIlwvcmVsZWFzZVwveGVuXC80LjE0LjVcL3hlbi00LjE0LjUudGFyLmd6LnNpZyJ9LHsiaGl0
cyI6IHsiY291bnQiOiA2LCJwZXJjZW50IjogIjAwLjI0In0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDUsInBlcmNlbnQiOiAiMDAuMzcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjg0NjMsInBlcmNlbnQi
OiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6
ICJcL3JlbGVhc2VcL3hlblwvNC4xMC4zXC94ZW4tNC4xMC4zLnRhci5nei5zaWcifSx7ImhpdHMi
OiB7ImNvdW50IjogNiwicGVyY2VudCI6ICIwMC4yNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA2
LCJwZXJjZW50IjogIjAwLjQ1In0sImJ5dGVzIjogeyJjb3VudCI6IDI5OTU4LCJwZXJjZW50Ijog
IjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAi
XC9yZWxlYXNlXC94ZW5cLzQuMTYuMFwveGVuLTQuMTYuMC50YXIuZ3ouc2lnIn0seyJoaXRzIjog
eyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNSwi
cGVyY2VudCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyODYxMSwicGVyY2VudCI6ICIw
MC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwv
cmVsZWFzZVwveGVuXC80LjE1LjNcL3hlbi00LjE1LjMudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsi
Y291bnQiOiA1LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDUsInBl
cmNlbnQiOiAiMDAuMzcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjM5MjAsInBlcmNlbnQiOiAiMDAu
MDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3Jl
bGVhc2VcL3hlblwvNC42LjBcL3hlbi00LjYuMC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3Vu
dCI6IDUsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNSwicGVyY2Vu
dCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyOTI0MiwicGVyY2VudCI6ICIwMC4wMCJ9
LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFz
ZVwveGVuXC80LjkuMFwveGVuLTQuOS4wLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50Ijog
NSwicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0LCJwZXJjZW50Ijog
IjAwLjMwIn0sImJ5dGVzIjogeyJjb3VudCI6IDI4NTk3LCJwZXJjZW50IjogIjAwLjAwIn0sIm1l
dGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94
ZW5cLzQuMTAuMFwveGVuLTQuMTAuMC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDUs
InBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIw
MC4zMCJ9LCJieXRlcyI6IHsiY291bnQiOiAyOTI4MiwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRo
b2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVu
XC80LjE0LjNcL3hlbi00LjE0LjMudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA1LCJw
ZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAu
MzcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjQxMTIsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9k
IjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwv
NC4xMy4xXC94ZW4tNC4xMy4xLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogNSwicGVy
Y2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjM3
In0sImJ5dGVzIjogeyJjb3VudCI6IDIzMTYyLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6
ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQu
My4xXC94ZW4tNC4zLjEudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA1LCJwZXJjZW50
IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMzcifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMjQ3MTcsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdF
VCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xNS4w
XC94ZW4tNC4xNS4wLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6
ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjM3In0sImJ5
dGVzIjogeyJjb3VudCI6IDIzMDg1LCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQi
LCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuNy40XC94
ZW4tNC43LjQudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAw
LjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMzcifSwiYnl0ZXMi
OiB7ImNvdW50IjogMjQ2ODAsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInBy
b3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC44LjVcL3hlbi00
LjguNS50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMjAi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsi
Y291bnQiOiAzMzA3ODEwOCwicGVyY2VudCI6ICIwMC4wNyJ9LCJtZXRob2QiOiAiR0VUIiwicHJv
dG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC8zLjQuNFwveGVuLTMu
NC40LnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjIwIn0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMiOiB7ImNvdW50
IjogMjgzMjIsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjog
IkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xLjBcL3hlbi00LjEuMC50YXIu
Z3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRlcyI6IHsiY291bnQiOiAy
MzkyMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRU
UC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjAuMFwveGVuLTQuMC4wLnRhci5nei5z
aWcifSx7ImhpdHMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjM3In0sImJ5dGVzIjogeyJjb3VudCI6IDE2MTE5
NzExMiwicGVyY2VudCI6ICIwMC4zNSJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRU
UC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE0LjNcL3hlbi00LjE0LjMudGFyLmd6
In0seyJoaXRzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyOTI0MCwi
cGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEi
LCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE2LjRcL3hlbi00LjE2LjQudGFyLmd6LnNpZyJ9
LHsiaGl0cyI6IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMiOiB7ImNvdW50IjogMjUyMjksInBl
cmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwi
ZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvMy40LjRcL3hlbi0zLjQuNC50YXIuZ3ouc2lnIn0seyJo
aXRzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogNSwicGVyY2VudCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyOTEzOSwicGVyY2Vu
dCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRh
IjogIlwvcmVsZWFzZVwveGVuXC80LjEuMlwveGVuLTQuMS4yLnRhci5nei5zaWcifSx7ImhpdHMi
OiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA1
LCJwZXJjZW50IjogIjAwLjM3In0sImJ5dGVzIjogeyJjb3VudCI6IDI4NDAxLCJwZXJjZW50Ijog
IjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAi
XC9yZWxlYXNlXC94ZW5cLzQuOC40XC94ZW4tNC44LjQudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsi
Y291bnQiOiA1LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBl
cmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMiOiB7ImNvdW50IjogMjM3OTMsInBlcmNlbnQiOiAiMDAu
MDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3Jl
bGVhc2VcL3hlblwvNC40LjJcL3hlbi00LjQuMi50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3Vu
dCI6IDUsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNSwicGVyY2Vu
dCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyODQ1MCwicGVyY2VudCI6ICIwMC4wMCJ9
LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFz
ZVwveGVuXC80LjUuMlwveGVuLTQuNS4yLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50Ijog
NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50Ijog
IjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDIzMzQ4LCJwZXJjZW50IjogIjAwLjAwIn0sIm1l
dGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94
ZW5cLzQuMC4zXC94ZW4tNC4wLjMudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJw
ZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAu
MzAifSwiYnl0ZXMiOiB7ImNvdW50IjogMjMzMTEsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9k
IjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwv
NC4wLjJcL3hlbi00LjAuMi50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNl
bnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9
LCJieXRlcyI6IHsiY291bnQiOiAyMjU2OCwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAi
R0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjQu
NFwveGVuLTQuNC40LnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6
ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDIyNTMxLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQi
LCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuNi4xXC94
ZW4tNC42LjEudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAw
LjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMi
OiB7ImNvdW50IjogMTc0MTksInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInBy
b3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC45LjRcL3hlbi00
LjkuNC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRlcyI6IHsi
Y291bnQiOiAxMDA4NzYzNjYsInBlcmNlbnQiOiAiMDAuMjIifSwibWV0aG9kIjogIkdFVCIsInBy
b3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xMS4zXC94ZW4t
NC4xMS4zLnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMiOiB7ImNv
dW50IjogMjI1MzEsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29s
IjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC42LjVcL3hlbi00LjYuNS50
YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRlcyI6IHsiY291bnQi
OiAyMzMxMSwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAi
SFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjcuNVwveGVuLTQuNy41LnRhci5n
ei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjMwIn0sImJ5dGVzIjogeyJjb3VudCI6IDIy
NzMxLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQ
LzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTMuMlwveGVuLTQuMTMuMi50YXIuZ3ou
c2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRlcyI6IHsiY291bnQiOiAyMzMx
MSwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8x
LjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjUuMFwveGVuLTQuNS4wLnRhci5nei5zaWci
fSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiA0LCJwZXJjZW50IjogIjAwLjMwIn0sImJ5dGVzIjogeyJjb3VudCI6IDE3OTgzLCJw
ZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIs
ImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMy4wXC94ZW4tNC4zLjAudGFyLmd6LnNpZyJ9LHsi
aGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMiOiB7ImNvdW50IjogMjI1MzEsInBlcmNl
bnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0
YSI6ICJcL3JlbGVhc2VcL3hlblwvNC40LjNcL3hlbi00LjQuMy50YXIuZ3ouc2lnIn0seyJoaXRz
IjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
NCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzk4MywicGVyY2VudCI6
ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjog
IlwvcmVsZWFzZVwveGVuXC80LjMuM1wveGVuLTQuMy4zLnRhci5nei5zaWcifSx7ImhpdHMiOiB7
ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0LCJw
ZXJjZW50IjogIjAwLjMwIn0sImJ5dGVzIjogeyJjb3VudCI6IDIzMzExLCJwZXJjZW50IjogIjAw
LjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9y
ZWxlYXNlXC94ZW5cLzQuMy4yXC94ZW4tNC4zLjIudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291
bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNl
bnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50IjogMjI1MTIsInBlcmNlbnQiOiAiMDAuMDAi
fSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVh
c2VcL3hlblwvNC4yLjVcL3hlbi00LjIuNS50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6
IDQsInBlcmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6
ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzQxOSwicGVyY2VudCI6ICIwMC4wMCJ9LCJt
ZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwv
eGVuXC80LjExLjFcL3hlbi00LjExLjEudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA0
LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAi
MDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50IjogMjM0NTUsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0
aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hl
blwvNC4xNC4xXC94ZW4tNC4xNC4xLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogNCwi
cGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAw
LjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDIzNDEzLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhv
ZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5c
LzQuMTUuNFwveGVuLTQuMTUuNC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBl
cmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4z
MCJ9LCJieXRlcyI6IHsiY291bnQiOiAyMzQxMywicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2Qi
OiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80
LjE2LjNcL3hlbi00LjE2LjMudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJj
ZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAi
fSwiYnl0ZXMiOiB7ImNvdW50IjogMjM3OTM4NTYsInBlcmNlbnQiOiAiMDAuMDUifSwibWV0aG9k
IjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwv
NC45LjBcL3hlbi00LjkuMC50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6
ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDIzNDMyLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQi
LCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTMuM1wv
eGVuLTQuMTMuMy50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAi
MDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRl
cyI6IHsiY291bnQiOiAyMzIyMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwi
cHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjAuNFwveGVu
LTQuMC40LnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4x
NiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVzIjog
eyJjb3VudCI6IDIyNTMxLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90
b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuNy4zXC94ZW4tNC43
LjMudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNv
dW50IjogMjI1MTIsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29s
IjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC43LjFcL3hlbi00LjcuMS50
YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQi
OiAyMjgwNywicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAi
SFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjEwLjRcL3hlbi00LjEwLjQudGFy
Lmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMiOiB7ImNvdW50Ijog
MjM0NTUsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhU
VFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xMi4yXC94ZW4tNC4xMi4yLnRhci5n
ei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDE3
NTAxLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQ
LzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMS4zXC94ZW4tNC4xLjMudGFyLmd6LnNp
ZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTY3MjEs
InBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4x
IiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC45LjJcL3hlbi00LjkuMi50YXIuZ3ouc2lnIn0s
eyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzUwMCwicGVy
Y2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJk
YXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjExLjJcL3hlbi00LjExLjIudGFyLmd6LnNpZyJ9LHsi
aGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTczOTIsInBlcmNl
bnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0
YSI6ICJcL3JlbGVhc2VcL3hlblwvNC42LjRcL3hlbi00LjYuNC50YXIuZ3ouc2lnIn0seyJoaXRz
IjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNjcyMSwicGVyY2VudCI6
ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjog
IlwvcmVsZWFzZVwveGVuXC80LjYuM1wveGVuLTQuNi4zLnRhci5nei5zaWcifSx7ImhpdHMiOiB7
ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJw
ZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDE3NTAwLCJwZXJjZW50IjogIjAw
LjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9y
ZWxlYXNlXC94ZW5cLzQuMTQuMlwveGVuLTQuMTQuMi50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJj
b3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVy
Y2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzUzNywicGVyY2VudCI6ICIwMC4w
MCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVs
ZWFzZVwveGVuXC80LjE1LjJcL3hlbi00LjE1LjIudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291
bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNl
bnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTI4NzksInBlcmNlbnQiOiAiMDAuMDAi
fSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVh
c2VcL3hlblwvNC4xMi40XC94ZW4tNC4xMi40LnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50
IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50
IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDE3NjA5LCJwZXJjZW50IjogIjAwLjAwIn0s
Im1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNl
XC94ZW5cLzQuMTQuNFwveGVuLTQuMTQuNC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6
IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzM3MywicGVyY2VudCI6ICIwMC4wMCJ9LCJt
ZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwv
eGVuXC80LjYuNlwveGVuLTQuNi42LnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogMywi
cGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAw
LjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDEyMjQ0LCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhv
ZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5c
LzQuOS4zXC94ZW4tNC45LjMudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJj
ZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogMTIyNDQsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjog
IkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4x
NS4xXC94ZW4tNC4xNS4xLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2Vu
dCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0s
ImJ5dGVzIjogeyJjb3VudCI6IDE2NzIxLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJH
RVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuNS4x
XC94ZW4tNC41LjEudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50Ijog
IjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0
ZXMiOiB7ImNvdW50IjogMTc1MDEsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIs
InByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC41LjNcL3hl
bi00LjUuMy50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAu
MTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6
IHsiY291bnQiOiAxNjgyOSwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJv
dG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjEyLjFcL3hlbi00
LjEyLjEudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEy
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTY4MjksInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3Rv
Y29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xMi4zXC94ZW4tNC4x
Mi4zLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJj
b3VudCI6IDE3NjA5LCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2Nv
bCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTQuMFwveGVuLTQuMTQu
MC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291
bnQiOiAxNzQ4MSwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wi
OiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjExLjRcL3hlbi00LjExLjQu
dGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50
IjogMTc1MDEsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjog
IkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4wLjFcL3hlbi00LjAuMS50YXIu
Z3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAx
NzUwMSwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRU
UC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjMuNFwveGVuLTQuMy40LnRhci5nei5z
aWcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDE3NDIw
LCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEu
MSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuOC4wXC94ZW4tNC44LjAudGFyLmd6LnNpZyJ9
LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTIxMzYsInBl
cmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwi
ZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC44LjJcL3hlbi00LjguMi50YXIuZ3ouc2lnIn0seyJo
aXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNjcyMSwicGVyY2Vu
dCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRh
IjogIlwvcmVsZWFzZVwveGVuXC80LjIuMlwveGVuLTQuMi4yLnRhci5nei5zaWcifSx7ImhpdHMi
OiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAz
LCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDE3NTAxLCJwZXJjZW50Ijog
IjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAi
XC9yZWxlYXNlXC94ZW5cLzQuOS4xXC94ZW4tNC45LjEudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBl
cmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTc1MDEsInBlcmNlbnQiOiAiMDAu
MDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3Jl
bGVhc2VcL3hlblwvNC4yLjBcL3hlbi00LjIuMC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3Vu
dCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2Vu
dCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzUwMSwicGVyY2VudCI6ICIwMC4wMCJ9
LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFz
ZVwveGVuXC80LjIuM1wveGVuLTQuMi4zLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50Ijog
MywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50Ijog
IjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDE3NTAxLCJwZXJjZW50IjogIjAwLjAwIn0sIm1l
dGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94
ZW5cLzQuMTEuMFwveGVuLTQuMTEuMC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDMs
InBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIw
MC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzUwMSwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRo
b2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVu
XC80LjEuMVwveGVuLTQuMS4xLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVy
Y2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIy
In0sImJ5dGVzIjogeyJjb3VudCI6IDE3MzkyLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6
ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQu
MS40XC94ZW4tNC4xLjQudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50
IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMTY3MjEsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdF
VCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC40LjFc
L3hlbi00LjQuMS50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAi
MDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRl
cyI6IHsiY291bnQiOiA1NTI0MzU2OCwicGVyY2VudCI6ICIwMC4xMiJ9LCJtZXRob2QiOiAiR0VU
IiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE2LjRc
L3hlbi00LjE2LjQudGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAu
MTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6
IHsiY291bnQiOiAxNzUwMSwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJv
dG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC8zLjQuM1wveGVuLTMu
NC4zLnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJj
b3VudCI6IDE2ODI5LCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2Nv
bCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTMuNFwveGVuLTQuMTMu
NC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291
bnQiOiAxNzYwOSwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wi
OiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjExLjNcL3hlbi00LjExLjMu
dGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50
IjogMjI0NDcyMDQsInBlcmNlbnQiOiAiMDAuMDUifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29s
IjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xLjVcL3hlbi00LjEuNS50
YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDEx
NzM1NzQ2MSwicGVyY2VudCI6ICIwMC4yNSJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAi
SFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjEzLjRcL3hlbi00LjEzLjQudGFy
Lmd6In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAzOTY4
NDgyNiwicGVyY2VudCI6ICIwMC4wOSJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRU
UC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjYuNlwveGVuLTQuNi42LnRhci5neiJ9
LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50IjogNjIyMDc4NTQs
InBlcmNlbnQiOiAiMDAuMTMifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4x
IiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC43LjBcL3hlbi00LjcuMC50YXIuZ3oifSx7Imhp
dHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDIyNjk0Njc0LCJwZXJj
ZW50IjogIjAwLjA1In0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRh
dGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuOC4xXC94ZW4tNC44LjEudGFyLmd6In0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAyMzkwODgyLCJwZXJjZW50Ijog
IjAwLjAxIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAi
XC9yZWxlYXNlXC94ZW5cLzQuMTAuMFwveGVuLTQuMTAuMC50YXIuZ3oifSx7ImhpdHMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDQxNjAxMjcyLCJwZXJjZW50IjogIjAw
LjA5In0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9y
ZWxlYXNlXC94ZW5cLzQuNy42XC94ZW4tNC43LjYudGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAzMTUwMzIsInBlcmNlbnQiOiAiMDAuMDAifSwi
bWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2Vc
L3hlblwvNC4xNS40XC94ZW4tNC4xNS40LnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJw
ZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMTAzOTU3NDksInBlcmNlbnQiOiAiMDAuMDIifSwibWV0
aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hl
blwvNC4xNi4zXC94ZW4tNC4xNi4zLnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUi
fSwiYnl0ZXMiOiB7ImNvdW50IjogMTE1ODIsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjog
IkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC44
LjNcL3hlbi00LjguMy50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQi
OiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJi
eXRlcyI6IHsiY291bnQiOiAyNTY4MDY4NSwicGVyY2VudCI6ICIwMC4wNiJ9LCJtZXRob2QiOiAi
R0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80Ljku
NFwveGVuLTQuOS40LnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMi
OiB7ImNvdW50IjogMjAwMjYyNzgsInBlcmNlbnQiOiAiMDAuMDQifSwibWV0aG9kIjogIkdFVCIs
InByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC42LjFcL3hl
bi00LjYuMS50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJj
b3VudCI6IDExNTgyLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2Nv
bCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuOC4xXC94ZW4tNC44LjEu
dGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50
IjogMTE2NTQsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjog
IkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xMC4yXC94ZW4tNC4xMC4yLnRh
ci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6
IDExNTgyLCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJI
VFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuNy4yXC94ZW4tNC43LjIudGFyLmd6
LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMjUz
NjE5MjcsInBlcmNlbnQiOiAiMDAuMDUifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhU
VFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xMS4xXC94ZW4tNC4xMS4xLnRhci5n
eiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogNzg3NDQy
LCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEu
MSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMi41XC94ZW4tNC4yLjUudGFyLmd6In0seyJo
aXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAyNDEzNTQyLCJwZXJj
ZW50IjogIjAwLjAxIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRh
dGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTAuMVwveGVuLTQuMTAuMS50YXIuZ3oifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDU0ODI5Mjk0LCJwZXJjZW50
IjogIjAwLjEyIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEi
OiAiXC9yZWxlYXNlXC94ZW5cLzQuMTYuMlwveGVuLTQuMTYuMi50YXIuZ3oifSx7ImhpdHMiOiB7
ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJw
ZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDExNTgyLCJwZXJjZW50IjogIjAw
LjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9y
ZWxlYXNlXC94ZW5cLzQuMi40XC94ZW4tNC4yLjQudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMTE1ODIsInBlcmNlbnQiOiAiMDAuMDAi
fSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVh
c2VcL3hlblwvNC41LjVcL3hlbi00LjUuNS50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMTU4MiwicGVyY2VudCI6ICIwMC4wMCJ9LCJt
ZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwv
eGVuXC80LjEwLjFcL3hlbi00LjEwLjEudGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMTE1ODIsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0
aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hl
blwvNC4xLjVcL3hlbi00LjEuNS50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBl
cmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4x
NSJ9LCJieXRlcyI6IHsiY291bnQiOiAyMzYxNDE0LCJwZXJjZW50IjogIjAwLjAxIn0sIm1ldGhv
ZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5c
LzQuOC4wXC94ZW4tNC44LjAudGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQi
OiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJi
eXRlcyI6IHsiY291bnQiOiAyMjI0MjExNiwicGVyY2VudCI6ICIwMC4wNSJ9LCJtZXRob2QiOiAi
R0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80Ljcu
MVwveGVuLTQuNy4xLnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMi
OiB7ImNvdW50IjogMTgwNzU5NzgsInBlcmNlbnQiOiAiMDAuMDQifSwibWV0aG9kIjogIkdFVCIs
InByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xMy4wXC94
ZW4tNC4xMy4wLnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTE1ODIsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3Rv
Y29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC40LjBcL3hlbi00LjQu
MC50YXIuZ3ouc2lnIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291
bnQiOiAxMTY1NCwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wi
OiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjE2LjFcL3hlbi00LjE2LjEu
dGFyLmd6LnNpZyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50
IjogODAwMTk2MDIsInBlcmNlbnQiOiAiMDAuMTcifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29s
IjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xNC4wXC94ZW4tNC4xNC4w
LnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50Ijog
NDU0MTUyNjYsInBlcmNlbnQiOiAiMDAuMTAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjog
IkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xNi4wXC94ZW4tNC4xNi4wLnRh
ci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogODAw
MzA1OTYsInBlcmNlbnQiOiAiMDAuMTcifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhU
VFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xNC40XC94ZW4tNC4xNC40LnRhci5n
eiJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTYzNTEw
LCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEu
MSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTUuMlwveGVuLTQuMTUuMi50YXIuZ3oifSx7
ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDQ3MjI4NiwicGVy
Y2VudCI6ICIwMC4wMCJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJk
YXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjguNFwveGVuLTQuOC40LnRhci5neiJ9LHsiaGl0cyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTMwMDkzLCJwZXJjZW50Ijog
IjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAi
XC9yZWxlYXNlXC94ZW5cLzQuMTAuM1wveGVuLTQuMTAuMy50YXIuZ3oifSx7ImhpdHMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDE4MTY3MjA1LCJwZXJjZW50IjogIjAw
LjA0In0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9y
ZWxlYXNlXC94ZW5cLzQuNC4yXC94ZW4tNC40LjIudGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0MDg2MTI2MywicGVyY2VudCI6ICIwMC4wOSJ9
LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFz
ZVwveGVuXC80LjE1LjFcL3hlbi00LjE1LjEudGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyMzQ1OTk0LCJwZXJjZW50IjogIjAwLjAxIn0sIm1l
dGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94
ZW5cLzQuMy4zXC94ZW4tNC4zLjMudGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9
LCJieXRlcyI6IHsiY291bnQiOiAxNjc0NjAsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjog
IkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC45
LjFcL3hlbi00LjkuMS50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVz
IjogeyJjb3VudCI6IDQwMDI1MDYsInBlcmNlbnQiOiAiMDAuMDEifSwibWV0aG9kIjogIkdFVCIs
InByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4zLjRcL3hl
bi00LjMuNC50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJj
b3VudCI6IDU4ODMsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29s
IjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xMy41XC94ZW4tNC4xMy41
LnRhci5nei5zaWcifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDExMjA4MzYwLCJwZXJjZW50IjogIjAwLjAyIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2Nv
bCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzMuNC4yXC94ZW4tMy40LjIu
dGFyLmd6In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAy
MjQzMTg2LCJwZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJI
VFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuNS4xXC94ZW4tNC41LjEudGFyLmd6
In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyNTIxOTMw
NCwicGVyY2VudCI6ICIwMC4wNSJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8x
LjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVuXC80LjExLjRcL3hlbi00LjExLjQudGFyLmd6In0s
eyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxNTkzMDM0LCJw
ZXJjZW50IjogIjAwLjAwIn0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIs
ImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuOC4zXC94ZW4tNC44LjMudGFyLmd6In0seyJoaXRz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0MDg5NjE2NCwicGVyY2Vu
dCI6ICIwMC4wOSJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjAiLCJkYXRh
IjogIlwvcmVsZWFzZVwveGVuXC80LjE1LjVcL3hlbi00LjE1LjUudGFyLmd6In0seyJoaXRzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxNTYyOTMyMCwicGVyY2VudCI6
ICIwMC4wMyJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjog
IlwvcmVsZWFzZVwveGVuXC80LjIuMlwveGVuLTQuMi4yLnRhci5neiJ9LHsiaGl0cyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjcwNDE5NzksInBlcmNlbnQiOiAiMDAu
MDYifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3Jl
bGVhc2VcL3hlblwvNC4xMi40XC94ZW4tNC4xMi40LnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjIyMjkxNCwicGVyY2VudCI6ICIwMC4wMCJ9
LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFz
ZVwveGVuXC80LjIuNFwveGVuLTQuMi40LnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjIyODcwNiwicGVyY2VudCI6ICIwMC4wMCJ9LCJtZXRo
b2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjogIlwvcmVsZWFzZVwveGVu
XC80LjIuMVwveGVuLTQuMi4xLnRhci5neiJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMTEyMDE0MTAsInBlcmNlbnQiOiAiMDAuMDIifSwibWV0aG9kIjog
IkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvMy40
LjFcL3hlbi0zLjQuMS50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVz
IjogeyJjb3VudCI6IDQ2NTUyNTE0LCJwZXJjZW50IjogIjAwLjEwIn0sIm1ldGhvZCI6ICJHRVQi
LCJwcm90b2NvbCI6ICJIVFRQLzEuMCIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTcuMFwv
eGVuLTQuMTcuMC50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4w
NCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjog
eyJjb3VudCI6IDM5MTMwNTkwLCJwZXJjZW50IjogIjAwLjA4In0sIm1ldGhvZCI6ICJHRVQiLCJw
cm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTMuNVwveGVu
LTQuMTMuNS50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJj
b3VudCI6IDM5MDgyNTc4LCJwZXJjZW50IjogIjAwLjA4In0sIm1ldGhvZCI6ICJHRVQiLCJwcm90
b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuMTMuMVwveGVuLTQu
MTMuMS50YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDIxNzk0NzQsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29s
IjogIkhUVFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4zLjJcL3hlbi00LjMuMi50
YXIuZ3oifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDE2
ODI4MTAsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhU
VFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC45LjNcL3hlbi00LjkuMy50YXIuZ3oi
fSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDIwNTA2MDIs
InBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4x
IiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC43LjJcL3hlbi00LjcuMi50YXIuZ3oifSx7Imhp
dHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDE4MTgyODY5LCJwZXJj
ZW50IjogIjAwLjA0In0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRh
dGEiOiAiXC9yZWxlYXNlXC94ZW5cLzQuNC40XC94ZW4tNC40LjQudGFyLmd6In0seyJoaXRzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0MDAzMjQ3MSwicGVyY2VudCI6
ICIwMC4wOSJ9LCJtZXRob2QiOiAiR0VUIiwicHJvdG9jb2wiOiAiSFRUUC8xLjEiLCJkYXRhIjog
IlwvcmVsZWFzZVwveGVuXC80LjE0LjJcL3hlbi00LjE0LjIudGFyLmd6In0seyJoaXRzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1ODI3LCJwZXJjZW50IjogIjAwLjAw
In0sIm1ldGhvZCI6ICJHRVQiLCJwcm90b2NvbCI6ICJIVFRQLzEuMSIsImRhdGEiOiAiXC9yZWxl
YXNlXC94ZW5cLzQuMTIuMFwveGVuLTQuMTIuMC50YXIuZ3ouc2lnIn1dfSwibm90X2ZvdW5kIjog
eyJtZXRhZGF0YSI6IHsiYnl0ZXMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDE3MjE1fSwiYXZnIjog
eyJ2YWx1ZSI6IDg2MDcsInBlcmNlbnQiOiAiNTAuMDAifSwibWF4IjogeyJ2YWx1ZSI6IDExNDUy
LCJwZXJjZW50IjogIjY2LjUyIn0sIm1pbiI6IHsidmFsdWUiOiA1NzYzLCJwZXJjZW50IjogIjMz
LjQ4In19LCJ2aXNpdG9ycyI6IHsidG90YWwiOiB7InZhbHVlIjogMH0sImF2ZyI6IHsidmFsdWUi
OiAwLCJwZXJjZW50IjogIjAwLjAwIn0sIm1heCI6IHsidmFsdWUiOiAwLCJwZXJjZW50IjogIjAw
LjAwIn0sIm1pbiI6IHsidmFsdWUiOiAwLCJwZXJjZW50IjogIjAwLjAwIn19LCJoaXRzIjogeyJ0
b3RhbCI6IHsidmFsdWUiOiAzfSwiYXZnIjogeyJ2YWx1ZSI6IDEsInBlcmNlbnQiOiAiMzMuMzMi
fSwibWF4IjogeyJ2YWx1ZSI6IDIsInBlcmNlbnQiOiAiNjYuNjcifSwibWluIjogeyJ2YWx1ZSI6
IDEsInBlcmNlbnQiOiAiMzMuMzMifX0sImRhdGEiOiB7InRvdGFsIjogewkidmFsdWUiOiAyfX19
LCJkYXRhIjogW3siaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDAsInBlcmNlbnQiOiAiMDAuMDAifSwiYnl0ZXMiOiB7ImNvdW50Ijog
MTE0NTIsInBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhU
VFAvMS4xIiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xOC4wXC94ZW4tNC4xOC4wLnRhci5n
eiwifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAwLCJwZXJjZW50IjogIjAwLjAwIn0sImJ5dGVzIjogeyJjb3VudCI6IDU3NjMs
InBlcmNlbnQiOiAiMDAuMDAifSwibWV0aG9kIjogIkdFVCIsInByb3RvY29sIjogIkhUVFAvMS4x
IiwiZGF0YSI6ICJcL3JlbGVhc2VcL3hlblwvNC4xOC4xXC94ZW4tNC4xOC4xLnRhci5neiJ9XX0s
Imhvc3RzIjogeyJtZXRhZGF0YSI6IHsiYnl0ZXMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDQ2NDIx
ODM1MTQxfSwiYXZnIjogeyJ2YWx1ZSI6IDQyMjAxNjY4LCJwZXJjZW50IjogIjAwLjA5In0sIm1h
eCI6IHsidmFsdWUiOiA5NjA4Njc0MDEsInBlcmNlbnQiOiAiMDIuMDcifSwibWluIjogeyJ2YWx1
ZSI6IDU5MCwicGVyY2VudCI6ICIwMC4wMCJ9fSwidmlzaXRvcnMiOiB7InRvdGFsIjogeyJ2YWx1
ZSI6IDEzNDd9LCJhdmciOiB7InZhbHVlIjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJtYXgiOiB7
InZhbHVlIjogMjYsInBlcmNlbnQiOiAiMDEuOTMifSwibWluIjogeyJ2YWx1ZSI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifX0sImhpdHMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDI1NTN9LCJhdmciOiB7
InZhbHVlIjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJtYXgiOiB7InZhbHVlIjogMjMxLCJwZXJj
ZW50IjogIjA5LjA1In0sIm1pbiI6IHsidmFsdWUiOiAxLCJwZXJjZW50IjogIjAwLjA0In19LCJk
YXRhIjogeyJ0b3RhbCI6IHsJInZhbHVlIjogMTEwMH19fSwiZGF0YSI6IFt7ImhpdHMiOiB7ImNv
dW50IjogMjMxLCJwZXJjZW50IjogIjA5LjA1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDYsInBl
cmNlbnQiOiAiMDAuNDUifSwiYnl0ZXMiOiB7ImNvdW50IjogMTM0MjA0NiwicGVyY2VudCI6ICIw
MC4wMCJ9LCJkYXRhIjogIjM1LjE2MC4yNy4yMjEifSx7ImhpdHMiOiB7ImNvdW50IjogNTAsInBl
cmNlbnQiOiAiMDEuOTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4w
NyJ9LCJieXRlcyI6IHsiY291bnQiOiA1NzI0MTcyMSwicGVyY2VudCI6ICIwMC4xMiJ9LCJkYXRh
IjogIjEwMi4zOC4zMC4yNDkifSx7ImhpdHMiOiB7ImNvdW50IjogNDEsInBlcmNlbnQiOiAiMDEu
NjEifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMjYsInBlcmNlbnQiOiAiMDEuOTMifSwiYnl0ZXMi
OiB7ImNvdW50IjogOTYwODY3NDAxLCJwZXJjZW50IjogIjAyLjA3In0sImRhdGEiOiAiNjQuMzIu
NDYuMjUxIn0seyJoaXRzIjogeyJjb3VudCI6IDQxLCJwZXJjZW50IjogIjAxLjYxIn0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEwLCJwZXJjZW50IjogIjAwLjc0In0sImJ5dGVzIjogeyJjb3VudCI6
IDM4Nzc3NzIxLCJwZXJjZW50IjogIjAwLjA4In0sImRhdGEiOiAiMjE2LjI0NC42Ni4xOTcifSx7
ImhpdHMiOiB7ImNvdW50IjogMjksInBlcmNlbnQiOiAiMDEuMTQifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1NDQyMjMxNiwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJkYXRhIjogIjExNy4xNTQuODguMTI3In0seyJoaXRzIjogeyJj
b3VudCI6IDI2LCJwZXJjZW50IjogIjAxLjAyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEzLCJw
ZXJjZW50IjogIjAwLjk3In0sImJ5dGVzIjogeyJjb3VudCI6IDY0MzQ3OTM4MiwicGVyY2VudCI6
ICIwMS4zOSJ9LCJkYXRhIjogIjEyOC45My4xNjYuMiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyNiwi
cGVyY2VudCI6ICIwMS4wMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDY5MDU1NzQ3LCJwZXJjZW50IjogIjAwLjE1In0sImRh
dGEiOiAiMTI1LjE2OC4yNTMuNDEifSx7ImhpdHMiOiB7ImNvdW50IjogMjQsInBlcmNlbnQiOiAi
MDAuOTQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRl
cyI6IHsiY291bnQiOiAxMDQ2OTA4MDcsInBlcmNlbnQiOiAiMDAuMjMifSwiZGF0YSI6ICIxNzku
NjYuNjIuMzcifSx7ImhpdHMiOiB7ImNvdW50IjogMjAsInBlcmNlbnQiOiAiMDAuNzgifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQi
OiAxNDY0NzYyMjksInBlcmNlbnQiOiAiMDAuMzIifSwiZGF0YSI6ICIxNDkuMTAyLjI1Mi4xMSJ9
LHsiaGl0cyI6IHsiY291bnQiOiAyMCwicGVyY2VudCI6ICIwMC43OCJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDYyOTkwOTI5
LCJwZXJjZW50IjogIjAwLjE0In0sImRhdGEiOiAiNDEuMjMwLjIxNi4yNDYifSx7ImhpdHMiOiB7
ImNvdW50IjogMTYsInBlcmNlbnQiOiAiMDAuNjMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA2NDYyNzk0OCwicGVyY2VudCI6
ICIwMC4xNCJ9LCJkYXRhIjogIjM3LjE3NC4yNDAuMjQ2In0seyJoaXRzIjogeyJjb3VudCI6IDE2
LCJwZXJjZW50IjogIjAwLjYzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTY1ODc3MjMsInBlcmNlbnQiOiAiMDAuMTIifSwi
ZGF0YSI6ICIxNTQuNzIuMTYyLjEzMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxNiwicGVyY2VudCI6
ICIwMC42MyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5
dGVzIjogeyJjb3VudCI6IDg3MjQyOTQ0LCJwZXJjZW50IjogIjAwLjE5In0sImRhdGEiOiAiMjE3
LjE3OC4xOTQuMTkifSx7ImhpdHMiOiB7ImNvdW50IjogMTUsInBlcmNlbnQiOiAiMDAuNTkifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291
bnQiOiAyNTYwOTY5MzAsInBlcmNlbnQiOiAiMDAuNTUifSwiZGF0YSI6ICIxMDQuMjguMTkyLjk5
In0seyJoaXRzIjogeyJjb3VudCI6IDE1LCJwZXJjZW50IjogIjAwLjU5In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogODM1NDQx
MTUsInBlcmNlbnQiOiAiMDAuMTgifSwiZGF0YSI6ICI0OS40My4yNi44NiJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAxNSwicGVyY2VudCI6ICIwMC41OSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDY2MjYyMTYsInBlcmNlbnQiOiAi
MDAuMDEifSwiZGF0YSI6ICIxMDUuMTAyLjU3LjE3OCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxMywi
cGVyY2VudCI6ICIwMC41MSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDY5ODQ5NDYxLCJwZXJjZW50IjogIjAwLjE1In0sImRh
dGEiOiAiMTA3LjUuMjE4LjExNiJ9LHsiaGl0cyI6IHsiY291bnQiOiAxMywicGVyY2VudCI6ICIw
MC41MSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVz
IjogeyJjb3VudCI6IDM3NTA0OTE1LCJwZXJjZW50IjogIjAwLjA4In0sImRhdGEiOiAiNDYuMTgy
LjUyLjgzIn0seyJoaXRzIjogeyJjb3VudCI6IDEzLCJwZXJjZW50IjogIjAwLjUxIn0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50Ijog
NzQ3MzA4NDQsInBlcmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICI4OS4yMTYuOTUuMTIwIn0seyJo
aXRzIjogeyJjb3VudCI6IDEyLCJwZXJjZW50IjogIjAwLjQ3In0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNzQ5NzMyNjAsInBl
cmNlbnQiOiAiMDAuMTYifSwiZGF0YSI6ICIxMDcuMjExLjIxNi4xNzQifSx7ImhpdHMiOiB7ImNv
dW50IjogMTIsInBlcmNlbnQiOiAiMDAuNDcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVy
Y2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAyNzk0Nzc3MjAsInBlcmNlbnQiOiAi
MDAuNjAifSwiZGF0YSI6ICI5My4xMTIuMjMwLjM5In0seyJoaXRzIjogeyJjb3VudCI6IDExLCJw
ZXJjZW50IjogIjAwLjQzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MDcifSwiYnl0ZXMiOiB7ImNvdW50IjogODc0MjE4MTIsInBlcmNlbnQiOiAiMDAuMTkifSwiZGF0
YSI6ICIxNzEuMTA1LjE3Ny4xMzYifSx7ImhpdHMiOiB7ImNvdW50IjogMTEsInBlcmNlbnQiOiAi
MDAuNDMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRl
cyI6IHsiY291bnQiOiA2MzEwODkyMSwicGVyY2VudCI6ICIwMC4xNCJ9LCJkYXRhIjogIjEwNS4x
NTUuMjQ0LjQ2In0seyJoaXRzIjogeyJjb3VudCI6IDExLCJwZXJjZW50IjogIjAwLjQzIn0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50
IjogMzgzMTE1NjksInBlcmNlbnQiOiAiMDAuMDgifSwiZGF0YSI6ICIzNy4yNi44MS41NCJ9LHsi
aGl0cyI6IHsiY291bnQiOiAxMSwicGVyY2VudCI6ICIwMC40MyJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiA1LCJwZXJjZW50IjogIjAwLjM3In0sImJ5dGVzIjogeyJjb3VudCI6IDI5NTYzMjQ0OCwi
cGVyY2VudCI6ICIwMC42NCJ9LCJkYXRhIjogIjYyLjE2My4xODEuNDMifSx7ImhpdHMiOiB7ImNv
dW50IjogMTAsInBlcmNlbnQiOiAiMDAuMzkifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNSwicGVy
Y2VudCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyMjA5NTE0MDYsInBlcmNlbnQiOiAi
MDAuNDgifSwiZGF0YSI6ICIyMDcuNDYuMTMuMTY4In0seyJoaXRzIjogeyJjb3VudCI6IDEwLCJw
ZXJjZW50IjogIjAwLjM5In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMzk4MTA1NCwicGVyY2VudCI6ICIwMC4wMSJ9LCJkYXRh
IjogIjE5Ni42NC41OC45OSJ9LHsiaGl0cyI6IHsiY291bnQiOiA5LCJwZXJjZW50IjogIjAwLjM1
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7
ImNvdW50IjogMjY3NjgzMCwicGVyY2VudCI6ICIwMC4wMSJ9LCJkYXRhIjogIjE0Ni43MC4xNDUu
OTQifSx7ImhpdHMiOiB7ImNvdW50IjogOSwicGVyY2VudCI6ICIwMC4zNSJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDIwMzUy
NzE0MywicGVyY2VudCI6ICIwMC40NCJ9LCJkYXRhIjogIjE1Ny41NS4zOS40OCJ9LHsiaGl0cyI6
IHsiY291bnQiOiA5LCJwZXJjZW50IjogIjAwLjM1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIs
InBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogNDE4OTcyNjI2LCJwZXJjZW50
IjogIjAwLjkwIn0sImRhdGEiOiAiMTkzLjIyMS4xMzMuMyJ9LHsiaGl0cyI6IHsiY291bnQiOiA4
LCJwZXJjZW50IjogIjAwLjMxIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAi
MDAuMzAifSwiYnl0ZXMiOiB7ImNvdW50IjogMTU4MzA0ODQ0LCJwZXJjZW50IjogIjAwLjM0In0s
ImRhdGEiOiAiMjA3LjQ2LjEzLjcifSx7ImhpdHMiOiB7ImNvdW50IjogOCwicGVyY2VudCI6ICIw
MC4zMSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVz
IjogeyJjb3VudCI6IDE4NjMxODQ4MCwicGVyY2VudCI6ICIwMC40MCJ9LCJkYXRhIjogIjI0LjE4
LjI0NC4yNTAifSx7ImhpdHMiOiB7ImNvdW50IjogOCwicGVyY2VudCI6ICIwMC4zMSJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6
IDE0OTE3NjgyNiwicGVyY2VudCI6ICIwMC4zMiJ9LCJkYXRhIjogIjUyLjE2Ny4xNDQuMTQ1In0s
eyJoaXRzIjogeyJjb3VudCI6IDgsInBlcmNlbnQiOiAiMDAuMzEifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogNiwicGVyY2VudCI6ICIwMC40NSJ9LCJieXRlcyI6IHsiY291bnQiOiAyMjE0MjMxMzcs
InBlcmNlbnQiOiAiMDAuNDgifSwiZGF0YSI6ICIxOTQuMTUzLjg4LjEzNSJ9LHsiaGl0cyI6IHsi
Y291bnQiOiA4LCJwZXJjZW50IjogIjAwLjMxIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTg2MzE4NDgwLCJwZXJjZW50Ijog
IjAwLjQwIn0sImRhdGEiOiAiMzEuMzQuMjMuMTI4In0seyJoaXRzIjogeyJjb3VudCI6IDcsInBl
cmNlbnQiOiAiMDAuMjcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4z
MCJ9LCJieXRlcyI6IHsiY291bnQiOiAxMjY0MjkwNzUsInBlcmNlbnQiOiAiMDAuMjcifSwiZGF0
YSI6ICIxODkuMjcuMjMwLjExNCJ9LHsiaGl0cyI6IHsiY291bnQiOiA3LCJwZXJjZW50IjogIjAw
LjI3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMi
OiB7ImNvdW50IjogMjUzMDc4NDIwLCJwZXJjZW50IjogIjAwLjU1In0sImRhdGEiOiAiNTIuMTY3
LjE0NC4xODEifSx7ImhpdHMiOiB7ImNvdW50IjogNywicGVyY2VudCI6ICIwMC4yNyJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjMwIn0sImJ5dGVzIjogeyJjb3VudCI6
IDIwMTEyMjE2NSwicGVyY2VudCI6ICIwMC40MyJ9LCJkYXRhIjogIjQwLjc3LjE2Ny4yNSJ9LHsi
aGl0cyI6IHsiY291bnQiOiA3LCJwZXJjZW50IjogIjAwLjI3In0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNzMxODY5MCwicGVy
Y2VudCI6ICIwMC4wMiJ9LCJkYXRhIjogIjE4OS41MS45Ni4yMjMifSx7ImhpdHMiOiB7ImNvdW50
IjogNywicGVyY2VudCI6ICIwMC4yNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDI1NjcxNDU4OSwicGVyY2VudCI6ICIwMC41
NSJ9LCJkYXRhIjogIjEwNy4xNDAuNTUuMjEzIn0seyJoaXRzIjogeyJjb3VudCI6IDYsInBlcmNl
bnQiOiAiMDAuMjQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4zMCJ9
LCJieXRlcyI6IHsiY291bnQiOiAxMzY3MDU2NzIsInBlcmNlbnQiOiAiMDAuMjkifSwiZGF0YSI6
ICI1Mi4xNjcuMTQ0LjIyIn0seyJoaXRzIjogeyJjb3VudCI6IDYsInBlcmNlbnQiOiAiMDAuMjQi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsi
Y291bnQiOiAyNjM2OTIwNiwicGVyY2VudCI6ICIwMC4wNiJ9LCJkYXRhIjogIjIwMy45MS44NS41
MSJ9LHsiaGl0cyI6IHsiY291bnQiOiA2LCJwZXJjZW50IjogIjAwLjI0In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogNjg5NTg0
NjYsInBlcmNlbnQiOiAiMDAuMTUifSwiZGF0YSI6ICIxMzEuMTUwLjIxMi4xMjMifSx7ImhpdHMi
OiB7ImNvdW50IjogNiwicGVyY2VudCI6ICIwMC4yNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDEzNTY4NDc2MiwicGVyY2Vu
dCI6ICIwMC4yOSJ9LCJkYXRhIjogIjIwNy40Ni4xMy4xNTMifSx7ImhpdHMiOiB7ImNvdW50Ijog
NiwicGVyY2VudCI6ICIwMC4yNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0LCJwZXJjZW50Ijog
IjAwLjMwIn0sImJ5dGVzIjogeyJjb3VudCI6IDI0NzcyNTYsInBlcmNlbnQiOiAiMDAuMDEifSwi
ZGF0YSI6ICIyMy4yMi4zNS4xNjIifSx7ImhpdHMiOiB7ImNvdW50IjogNiwicGVyY2VudCI6ICIw
MC4yNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVz
IjogeyJjb3VudCI6IDE1MzA4NTk2OSwicGVyY2VudCI6ICIwMC4zMyJ9LCJkYXRhIjogIjE1Ny41
NS4zOS41OCJ9LHsiaGl0cyI6IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50Ijog
NjQ1OTU2OTUsInBlcmNlbnQiOiAiMDAuMTQifSwiZGF0YSI6ICI3OS4xMTcuMjcuMTMyIn0seyJo
aXRzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogNCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDY5NjIzODIsInBl
cmNlbnQiOiAiMDAuMjMifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjIxMiJ9LHsiaGl0cyI6IHsiY291
bnQiOiA1LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogOTgyNzk2MjMsInBlcmNlbnQiOiAiMDAu
MjEifSwiZGF0YSI6ICIxMzYuMjI2LjUxLjExNyJ9LHsiaGl0cyI6IHsiY291bnQiOiA1LCJwZXJj
ZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogOTA0NjI0NjMsInBlcmNlbnQiOiAiMDAuMTkifSwiZGF0YSI6
ICI1Mi4xNjcuMTQ0LjE5In0seyJoaXRzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMjAi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRlcyI6IHsi
Y291bnQiOiA5MDQ1NzA5OCwicGVyY2VudCI6ICIwMC4xOSJ9LCJkYXRhIjogIjE1Ny41NS4zOS41
OSJ9LHsiaGl0cyI6IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMiOiB7ImNvdW50IjogNjc4NTQy
OTEsInBlcmNlbnQiOiAiMDAuMTUifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjE4OSJ9LHsiaGl0cyI6
IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTkzMDM2NDQsInBlcmNlbnQi
OiAiMDAuMTMifSwiZGF0YSI6ICI0Mi4xMDQuMTQxLjE0MCJ9LHsiaGl0cyI6IHsiY291bnQiOiA1
LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAi
MDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTEzMDcwNjM1LCJwZXJjZW50IjogIjAwLjI0In0s
ImRhdGEiOiAiMTU3LjU1LjM5LjE0In0seyJoaXRzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAi
MDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRl
cyI6IHsiY291bnQiOiAxODg2NywicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjY2LjI0OS43
MC4xMDAifSx7ImhpdHMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4yMCJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDY3
ODU0MjU1LCJwZXJjZW50IjogIjAwLjE1In0sImRhdGEiOiAiNDAuNzcuMTY3LjMyIn0seyJoaXRz
IjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
NCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRlcyI6IHsiY291bnQiOiA5MDQ2MjQ2MywicGVyY2Vu
dCI6ICIwMC4xOSJ9LCJkYXRhIjogIjIwNy40Ni4xMy4xMDcifSx7ImhpdHMiOiB7ImNvdW50Ijog
NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50Ijog
IjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDExMDgxMTU2NywicGVyY2VudCI6ICIwMC4yNCJ9
LCJkYXRhIjogIjExOC4yMTAuMjEwLjUifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6
ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5
dGVzIjogeyJjb3VudCI6IDU1MjM4MjI5LCJwZXJjZW50IjogIjAwLjEyIn0sImRhdGEiOiAiMzcu
NDcuMTcxLjE0MSJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50
IjogNDUyNDAxMjgsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjE4MiJ9
LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTQyOTYwMzEs
InBlcmNlbnQiOiAiMDAuMTIifSwiZGF0YSI6ICIyLjUyLjMxLjIwNCJ9LHsiaGl0cyI6IHsiY291
bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTU3MDc4NDksInBlcmNlbnQiOiAiMDAu
MTIifSwiZGF0YSI6ICI5NC4zNC4xOTkuNzUifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2Vu
dCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0s
ImJ5dGVzIjogeyJjb3VudCI6IDU5OTI2ODU4LCJwZXJjZW50IjogIjAwLjEzIn0sImRhdGEiOiAi
NzAuNTEuMjM4LjY0In0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291
bnQiOiA5MDQ1NjUwOCwicGVyY2VudCI6ICIwMC4xOSJ9LCJkYXRhIjogIjE1Ny41NS4zOS40OSJ9
LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogOTMxNTkyNDAs
InBlcmNlbnQiOiAiMDAuMjAifSwiZGF0YSI6ICI2OC45LjI0NC4yMDQifSx7ImhpdHMiOiB7ImNv
dW50IjogNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDU4NDk3ODYzLCJwZXJjZW50IjogIjAw
LjEzIn0sImRhdGEiOiAiMTk0LjIzMC4xNDguMTMxIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBl
cmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4y
MiJ9LCJieXRlcyI6IHsiY291bnQiOiA2Nzg0ODMzNiwicGVyY2VudCI6ICIwMC4xNSJ9LCJkYXRh
IjogIjQwLjc3LjE2Ny41OSJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7
ImNvdW50IjogNTcxNDYxNzAsInBlcmNlbnQiOiAiMDAuMTIifSwiZGF0YSI6ICIxNC4xMzkuMTIy
LjEzOSJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogNTgy
NDA2NjUsInBlcmNlbnQiOiAiMDAuMTMifSwiZGF0YSI6ICI0Ny40Mi4xNy4xMTAifSx7ImhpdHMi
OiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDEwMzkzMTMzMywicGVyY2Vu
dCI6ICIwMC4yMiJ9LCJkYXRhIjogIjUyLjE2Ny4xNDQuMjM2In0seyJoaXRzIjogeyJjb3VudCI6
IDQsInBlcmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1NTQ0Mzg1MCwicGVyY2VudCI6ICIwMC4xMiJ9
LCJkYXRhIjogIjc4LjEzOS45MC4xMTAifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6
ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5
dGVzIjogeyJjb3VudCI6IDU4NTgxODU1LCJwZXJjZW50IjogIjAwLjEzIn0sImRhdGEiOiAiMTk2
Ljc1LjE4LjExOCJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMiOiB7ImNvdW50
IjogMTIzNDc0OCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjIxNy4yMjQuMTg2LjEyMSJ9
LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogOTQ2NzE5NSwi
cGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjogIjE4Ny4xOS4yMjIuMjEwIn0seyJoaXRzIjogeyJj
b3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MzY3MjQzNCwicGVyY2VudCI6ICIw
MC4xMiJ9LCJkYXRhIjogIjIwOS4xODguMzUuMjQyIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBl
cmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4x
NSJ9LCJieXRlcyI6IHsiY291bnQiOiA2Nzg0ODMzNiwicGVyY2VudCI6ICIwMC4xNSJ9LCJkYXRh
IjogIjIwNy40Ni4xMy4xNyJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7
ImNvdW50IjogNjM0MDEzNzgsInBlcmNlbnQiOiAiMDAuMTQifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0
LjE5MSJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTUx
OTk1MjAsInBlcmNlbnQiOiAiMDAuMTIifSwiZGF0YSI6ICI4Ni4xMjcuMjU0LjYyIn0seyJoaXRz
IjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA2MzcwNjEwMywicGVyY2Vu
dCI6ICIwMC4xNCJ9LCJkYXRhIjogIjk0LjQzLjg5Ljc0In0seyJoaXRzIjogeyJjb3VudCI6IDQs
InBlcmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIw
MC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMjY1Mzk1MDUsInBlcmNlbnQiOiAiMDAuMjcifSwi
ZGF0YSI6ICI1Mi4xNjcuMTQ0LjIzOCJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50Ijog
IjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0
ZXMiOiB7ImNvdW50IjogMjg0OTE3LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiMTE3LjIz
My4xNjMuMiJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50Ijog
Njc4NDgzMDAsInBlcmNlbnQiOiAiMDAuMTUifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjIzMCJ9LHsi
aGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjM5ODQ0NDEsInBl
cmNlbnQiOiAiMDAuMDUifSwiZGF0YSI6ICI1OS41Ny44LjE5OCJ9LHsiaGl0cyI6IHsiY291bnQi
OiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQi
OiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogOTA0NTY1MDgsInBlcmNlbnQiOiAiMDAuMTki
fSwiZGF0YSI6ICIxNTcuNTUuMzkuNTUifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6
ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5
dGVzIjogeyJjb3VudCI6IDkwNDU2NTA4LCJwZXJjZW50IjogIjAwLjE5In0sImRhdGEiOiAiNDAu
NzcuMTY3LjQ4In0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQi
OiAxNjExNDAxOSwicGVyY2VudCI6ICIwMC4wMyJ9LCJkYXRhIjogIjE0OS4xNDAuMTU2LjExNCJ9
LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMTU1ODg3Nzcy
LCJwZXJjZW50IjogIjAwLjM0In0sImRhdGEiOiAiNTIuMTY3LjE0NC4yMjYifSx7ImhpdHMiOiB7
ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDc4MDY0NTIsInBlcmNlbnQiOiAi
MDAuMDIifSwiZGF0YSI6ICIyMjAuMTYyLjMuNjEifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVy
Y2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3
In0sImJ5dGVzIjogeyJjb3VudCI6IDU1NTcxNTIzLCJwZXJjZW50IjogIjAwLjEyIn0sImRhdGEi
OiAiMTA5LjE1Ny45MC42In0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsi
Y291bnQiOiA3NDU3MzYzNiwicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjE0Ni43MC4xNzku
MjgifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDc0NTgy
OTk0LCJwZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiNTIuMTY3LjE0NC4xNyJ9LHsiaGl0cyI6
IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMs
InBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50IjogMTUxNzU5MTg5LCJwZXJjZW50
IjogIjAwLjMzIn0sImRhdGEiOiAiNDAuNzcuMTY3LjI4In0seyJoaXRzIjogeyJjb3VudCI6IDQs
InBlcmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIw
MC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA5MDQ1NjUwOCwicGVyY2VudCI6ICIwMC4xOSJ9LCJk
YXRhIjogIjIwNy40Ni4xMy42In0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAu
MTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6
IHsiY291bnQiOiA5NzE5NjUzNSwicGVyY2VudCI6ICIwMC4yMSJ9LCJkYXRhIjogIjUyLjE2Ny4x
NDQuMTcwIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA4
MzU3ODA3LCJwZXJjZW50IjogIjAwLjAyIn0sImRhdGEiOiAiMTAzLjIzMS4yMzYuMiJ9LHsiaGl0
cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMzk1MDYxLCJwZXJjZW50
IjogIjAwLjAwIn0sImRhdGEiOiAiNTEuMjIyLjI1My40In0seyJoaXRzIjogeyJjb3VudCI6IDMs
InBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA4MzQxODE4OSwicGVyY2VudCI6ICIwMC4xOCJ9LCJk
YXRhIjogIjkyLjE4NC4xMDIuOTcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIw
MC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVz
IjogeyJjb3VudCI6IDEwMzkyMTI1OCwicGVyY2VudCI6ICIwMC4yMiJ9LCJkYXRhIjogIjk4LjE4
OC4xMDIuODkifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6
IDE2MDYzOTAsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICIxMzYuMjMuMjEuODYifSx7Imhp
dHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDE1NTg4MTgxNywicGVy
Y2VudCI6ICIwMC4zNCJ9LCJkYXRhIjogIjUyLjE2Ny4xNDQuMjMzIn0seyJoaXRzIjogeyJjb3Vu
dCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2Vu
dCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMjY1Mzg5MTUsInBlcmNlbnQiOiAiMDAu
MjcifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjIzNSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJj
ZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNjc4NDIzODEsInBlcmNlbnQiOiAiMDAuMTUifSwiZGF0YSI6
ICIxNTcuNTUuMzkuNTQifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJj
b3VudCI6IDQ1MjM0MjA5LCJwZXJjZW50IjogIjAwLjEwIn0sImRhdGEiOiAiNTIuMTY3LjE0NC4y
MjQifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDEwMzky
NTM3OCwicGVyY2VudCI6ICIwMC4yMiJ9LCJkYXRhIjogIjQwLjc3LjE2Ny4zIn0seyJoaXRzIjog
eyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxMjQ1OCwicGVyY2VudCI6ICIw
MC4wMCJ9LCJkYXRhIjogIjE4MC4yNDQuMTY0LjE5MCJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJw
ZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTY4NDA2NDksInBlcmNlbnQiOiAiMDAuMTIifSwiZGF0
YSI6ICIxNTcuMjMxLjcuMTEzIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAu
MTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6
IHsiY291bnQiOiAxMTA2ODQ1LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiNTEuMjIyLjI1
My4xMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjAz
NzE1NDIsInBlcmNlbnQiOiAiMDAuMDQifSwiZGF0YSI6ICIxODQuMTYwLjc0LjE1MSJ9LHsiaGl0
cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogOTAzMDI4NTksInBlcmNl
bnQiOiAiMDAuMTkifSwiZGF0YSI6ICIyMDcuNDYuMTMuMTU0In0seyJoaXRzIjogeyJjb3VudCI6
IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyNTYxMDQ0LCJwZXJjZW50IjogIjAwLjAxIn0s
ImRhdGEiOiAiMTAzLjExMi4xMi4xNjIifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6
ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5
dGVzIjogeyJjb3VudCI6IDgwOTc0NDQ1LCJwZXJjZW50IjogIjAwLjE3In0sImRhdGEiOiAiMTAz
LjE1MS4xNzIuMjYifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDEzMzIxNDA1NiwicGVyY2VudCI6ICIwMC4yOSJ9LCJkYXRhIjogIjY4Ljk5LjE0Mi4xMTki
fSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDU4MzU1Nzcz
LCJwZXJjZW50IjogIjAwLjEzIn0sImRhdGEiOiAiMTA4LjM1LjI0LjQ4In0seyJoaXRzIjogeyJj
b3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MjUzODMsInBlcmNlbnQiOiAi
MDAuMjIifSwiZGF0YSI6ICI1LjM5LjIzMy42MSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJj
ZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNjc4MzcwMTYsInBlcmNlbnQiOiAiMDAuMTUifSwiZGF0YSI6
ICIyMDcuNDYuMTMuMTI1In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsi
Y291bnQiOiA0MDc0MjgyMSwicGVyY2VudCI6ICIwMC4wOSJ9LCJkYXRhIjogIjM2LjExMi4xNjUu
NDgifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDgwODI0
MzE4LCJwZXJjZW50IjogIjAwLjE3In0sImRhdGEiOiAiNDkuMzYuMTguMjMzIn0seyJoaXRzIjog
eyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA2Nzg0MjM4MSwicGVyY2VudCI6
ICIwMC4xNSJ9LCJkYXRhIjogIjE1Ny41NS4zOS4yMDAifSx7ImhpdHMiOiB7ImNvdW50IjogMywi
cGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDE3NDY2LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEi
OiAiMTg1LjE5MS4xNzEuNiJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEy
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTA0MzIyNDM3LCJwZXJjZW50IjogIjAwLjIyIn0sImRhdGEiOiAiMTczLjE1LjE2
NC44MSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNDI0
MTExMjIsInBlcmNlbnQiOiAiMDAuMDkifSwiZGF0YSI6ICIxNjYuMTk0LjE0Ny4zNiJ9LHsiaGl0
cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5Njg5NDQsInBlcmNl
bnQiOiAiMDAuMTEifSwiZGF0YSI6ICIyMTYuMjUyLjY2Ljk1In0seyJoaXRzIjogeyJjb3VudCI6
IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTIzMjMzNCwicGVyY2VudCI6ICIwMC4xMSJ9
LCJkYXRhIjogIjQ5LjM3LjMyLjIwMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50Ijog
IjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0
ZXMiOiB7ImNvdW50IjogMTMwMDcwOTIzLCJwZXJjZW50IjogIjAwLjI4In0sImRhdGEiOiAiMTIy
LjE2MC41OS4xNDIifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDU0NDg3Nzg3LCJwZXJjZW50IjogIjAwLjEyIn0sImRhdGEiOiAiMjE3LjIzNS4yMjcuMjEw
In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA2MTY4MDIx
OSwicGVyY2VudCI6ICIwMC4xMyJ9LCJkYXRhIjogIjg3LjEzMC4xMTMuMTA2In0seyJoaXRzIjog
eyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAzODYxMTQ3LCJwZXJjZW50Ijog
IjAwLjAxIn0sImRhdGEiOiAiNjcuNDQuMjA4LjE2NiJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJw
ZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTc0MzAsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6
ICIxODUuMTkxLjE3MS45In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsi
Y291bnQiOiAxMDM5MjUzODMsInBlcmNlbnQiOiAiMDAuMjIifSwiZGF0YSI6ICI3OC4xMjYuMjE1
Ljk3In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMzg3
MjI2MTcsInBlcmNlbnQiOiAiMDAuMzAifSwiZGF0YSI6ICI2OS4xMjcuMjI3LjE1MyJ9LHsiaGl0
cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNjAxMDA2NzQsInBlcmNl
bnQiOiAiMDAuMTMifSwiZGF0YSI6ICI2Ni4xNzcuOTcuMTg4In0seyJoaXRzIjogeyJjb3VudCI6
IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MjU5NjMsInBlcmNlbnQiOiAiMDAuMjIi
fSwiZGF0YSI6ICI3NS43MC4yMTkuOTcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6
ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5
dGVzIjogeyJjb3VudCI6IDY3ODQyMzgxLCJwZXJjZW50IjogIjAwLjE1In0sImRhdGEiOiAiMjA3
LjQ2LjEzLjE1NSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50
IjogNDUyMjg4MDgsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjE5OSJ9
LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTAzOTI1OTYz
LCJwZXJjZW50IjogIjAwLjIyIn0sImRhdGEiOiAiODMuMjA0LjQzLjU3In0seyJoaXRzIjogeyJj
b3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA4MjgxNTE3OCwicGVyY2VudCI6ICIw
MC4xOCJ9LCJkYXRhIjogIjE4NS4xNzkuMTQyLjIyOSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJw
ZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MTUifSwiYnl0ZXMiOiB7ImNvdW50IjogNDUyMzQyMDksInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0
YSI6ICIyMDcuNDYuMTMuMTE2In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAu
MTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6
IHsiY291bnQiOiA3Mjg0MzMwNywicGVyY2VudCI6ICIwMC4xNiJ9LCJkYXRhIjogIjQ1LjEwLjE1
Mi4yMjkifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDEy
MDA2NzU5MywicGVyY2VudCI6ICIwMC4yNiJ9LCJkYXRhIjogIjI3LjMzLjExNi4xNjYifSx7Imhp
dHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDk4NTQxOTg5LCJwZXJj
ZW50IjogIjAwLjIxIn0sImRhdGEiOiAiMTA5LjcwLjEwMC42In0seyJoaXRzIjogeyJjb3VudCI6
IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MzA3NDIsInBlcmNlbnQiOiAiMDAuMjIi
fSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjE3NiJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50
IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMTc2MTMsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICI2Ni4y
NDkuNjQuNjUifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6
IDIyNjI1OTY1LCJwZXJjZW50IjogIjAwLjA1In0sImRhdGEiOiAiNDAuNzcuMTY3LjIzNSJ9LHsi
aGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMzY3MjcwNzUsInBl
cmNlbnQiOiAiMDAuMDgifSwiZGF0YSI6ICI0OS4yMjguMTAxLjExMyJ9LHsiaGl0cyI6IHsiY291
bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogNzA2MywicGVyY2VudCI6ICIwMC4wMCJ9
LCJkYXRhIjogIjQwLjc3LjE2Ny4yNDEifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6
ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5
dGVzIjogeyJjb3VudCI6IDU4Mzc5MTY1LCJwZXJjZW50IjogIjAwLjEzIn0sImRhdGEiOiAiMzcu
MTY5LjE3NC4yNDcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDMwNjEwNDIzLCJwZXJjZW50IjogIjAwLjA3In0sImRhdGEiOiAiMTg1LjY1LjEzNS4xODgi
fSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDkyMDEyMjcs
InBlcmNlbnQiOiAiMDAuMDIifSwiZGF0YSI6ICIxOTYuMTE5Ljg2Ljk0In0seyJoaXRzIjogeyJj
b3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxMzU4NDY5OSwicGVyY2VudCI6ICIw
MC4wMyJ9LCJkYXRhIjogIjU4LjE2MS42NS4xOSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJj
ZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNDUyMjg4MDgsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0YSI6
ICI1Mi4xNjcuMTQ0LjE2NiJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEy
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7
ImNvdW50IjogOTkwMDc0OSwicGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjogIjg3LjE4Mi4xMTQu
MTgyIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzYx
MywicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjY2LjI0OS43NS43MiJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTM1MDkxNzQ2LCJwZXJjZW50Ijog
IjAwLjI5In0sImRhdGEiOiAiNDkuNTAuNjUuMTAzIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBl
cmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4y
MiJ9LCJieXRlcyI6IHsiY291bnQiOiAxNTk5MzA2LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEi
OiAiNTIuNzAuMjQwLjE3MSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEy
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTIzMzksInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICI2Ni4yNDkuNzUuMTAw
In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA4MDA5NzQ3
MCwicGVyY2VudCI6ICIwMC4xNyJ9LCJkYXRhIjogIjc5LjExMy4xODMuMjAzIn0seyJoaXRzIjog
eyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA3NjczMjk0MSwicGVyY2VudCI6
ICIwMC4xNyJ9LCJkYXRhIjogIjE2NS4yMjUuMTA0LjEzMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAz
LCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMTA3NzQyNDQwLCJwZXJjZW50IjogIjAwLjIzIn0s
ImRhdGEiOiAiMTA5LjI0Mi4yMjQuMTAxIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQi
OiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1NzU4MTQwLCJwZXJjZW50IjogIjAwLjAxIn0sImRhdGEiOiAiMTAz
LjIxMS40MC4xNzcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDY3ODQyMzgxLCJwZXJjZW50IjogIjAwLjE1In0sImRhdGEiOiAiMjA3LjQ2LjEzLjEzMCJ9
LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTIwNzIwNDcs
InBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICIxMDcuNzcuMjI3Ljk3In0seyJoaXRzIjogeyJj
b3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0Mzg3MTg3NSwicGVyY2VudCI6ICIw
MC4wOSJ9LCJkYXRhIjogIjYzLjI1MS4xMjIuMTEyIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBl
cmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4x
NSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MzA3NDMsInBlcmNlbnQiOiAiMDAuMjIifSwiZGF0
YSI6ICI0MC43Ny4xNjcuNzcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4x
MiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjog
eyJjb3VudCI6IDE3NDY2LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiMTg1LjE5MS4xNzEu
MTEifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDE1OTA4
MzYyLCJwZXJjZW50IjogIjAwLjAzIn0sImRhdGEiOiAiMTc4LjIwNy45LjIxIn0seyJoaXRzIjog
eyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MjU5NjMsInBlcmNlbnQi
OiAiMDAuMjIifSwiZGF0YSI6ICI3Ni44LjE0Ny43NSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJw
ZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MTUifSwiYnl0ZXMiOiB7ImNvdW50IjogNjg2ODQ1NzcsInBlcmNlbnQiOiAiMDAuMTUifSwiZGF0
YSI6ICIxNTcuNTUuMzkuNjAifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4x
MiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjog
eyJjb3VudCI6IDUxOTY5NTI0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiMTkzLjQ4LjQ1
LjIxMiJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTAz
OTIxMjU4LCJwZXJjZW50IjogIjAwLjIyIn0sImRhdGEiOiAiNzQuODEuMjA2Ljk3In0seyJoaXRz
IjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiA0NTIzNDIwOSwicGVyY2Vu
dCI6ICIwMC4xMCJ9LCJkYXRhIjogIjIwNy40Ni4xMy4xNCJ9LHsiaGl0cyI6IHsiY291bnQiOiAz
LCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTcwMjQ0MDQsInBlcmNlbnQiOiAiMDAuMDQifSwi
ZGF0YSI6ICI1LjEyMi43MC4yNiJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAw
LjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMi
OiB7ImNvdW50IjogMTAzOTIxMjU4LCJwZXJjZW50IjogIjAwLjIyIn0sImRhdGEiOiAiMTAzLjg4
LjEwMS4yMDcifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6
IDUxMTk5NzY2LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiMjIzLjEwNC4xMjIuMjA0In0s
eyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA4MDM3OTc5Niwi
cGVyY2VudCI6ICIwMC4xNyJ9LCJkYXRhIjogIjQ2LjYxLjE2Mi44NiJ9LHsiaGl0cyI6IHsiY291
bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNDM5NDU4MjcsInBlcmNlbnQiOiAiMDAu
MDkifSwiZGF0YSI6ICI5NS45MS4yNDguMTkwIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNl
bnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9
LCJieXRlcyI6IHsiY291bnQiOiA3MDQ0ODUyOCwicGVyY2VudCI6ICIwMC4xNSJ9LCJkYXRhIjog
IjQwLjc3LjE2Ny4xNDMifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJj
b3VudCI6IDEyNjUzODkxNSwicGVyY2VudCI6ICIwMC4yNyJ9LCJkYXRhIjogIjUyLjE2Ny4xNDQu
MjE2In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA2Nzg0
MjM4MSwicGVyY2VudCI6ICIwMC4xNSJ9LCJkYXRhIjogIjUyLjE2Ny4xNDQuMTM4In0seyJoaXRz
IjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1OTE4MTcyMiwicGVyY2Vu
dCI6ICIwMC4xMyJ9LCJkYXRhIjogIjUwLjQ2LjM4LjQwIn0seyJoaXRzIjogeyJjb3VudCI6IDMs
InBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxMzk2NjY3OTQsInBlcmNlbnQiOiAiMDAuMzAifSwi
ZGF0YSI6ICI1OC4yMDkuMTExLjU5In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAi
MDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRl
cyI6IHsiY291bnQiOiA1NzA2NTg1OCwicGVyY2VudCI6ICIwMC4xMiJ9LCJkYXRhIjogIjQ1LjEz
MC4yMDAuMTE3In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQi
OiAxMDgwMzk0NywicGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjogIjg4Ljk3LjQ2LjIyMyJ9LHsi
aGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogOTMxNDU1NzYsInBl
cmNlbnQiOiAiMDAuMjAifSwiZGF0YSI6ICI4Ny45LjE3MC42MCJ9LHsiaGl0cyI6IHsiY291bnQi
OiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTcwNjcxNTAsInBlcmNlbnQiOiAiMDAuMTIi
fSwiZGF0YSI6ICIxNTYuMTQ2LjYwLjEzMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50
IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNDg1NzQxNzEsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0YSI6ICI5
MC4yMTcuMjcuNTQifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4xMiJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDEwMzkzMDY2OCwicGVyY2VudCI6ICIwMC4yMiJ9LCJkYXRhIjogIjExMy4xNTguMTk3Ljg2
In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA2Nzg0MjM4
MSwicGVyY2VudCI6ICIwMC4xNSJ9LCJkYXRhIjogIjUyLjE2Ny4xNDQuMTgwIn0seyJoaXRzIjog
eyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1Nzk4MDc0NSwicGVyY2VudCI6
ICIwMC4xMiJ9LCJkYXRhIjogIjc3LjIwNS4xMTcuMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJw
ZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMTIyNTM3MTk2LCJwZXJjZW50IjogIjAwLjI2In0sImRh
dGEiOiAiMTIuMjAzLjYwLjE2MyJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAw
LjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMi
OiB7ImNvdW50IjogNTE5Njg5NDQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICIzNS4xMzEu
MTgwLjM0In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1
MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjEuMjM5LjE4NS4xMzQifSx7Imhp
dHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJj
ZW50IjogIjAwLjExIn0sImRhdGEiOiAiNzMuMTU1LjIxMy4xMzgifSx7ImhpdHMiOiB7ImNvdW50
IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50
IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDIyNjIwMDQ2LCJwZXJjZW50IjogIjAwLjA1
In0sImRhdGEiOiAiNDAuNzcuMTY3LjUyIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQi
OiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjc1
LjEzNS4xNTAuODcifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDc0MTU1NjgsInBlcmNlbnQiOiAiMDAuMDIifSwiZGF0YSI6ICIxMzYuMjMyLjEuMTY2In0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwi
cGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjIuMjEwLjUyLjIxNSJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjkzMzQzODEsInBlcmNlbnQiOiAiMDAu
MDYifSwiZGF0YSI6ICIxNzguMTQzLjQyLjE3NiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDci
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6
ICIxOTAuMTUzLjg2LjIxNyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7
ImNvdW50IjogNjczODQwNDMsInBlcmNlbnQiOiAiMDAuMTUifSwiZGF0YSI6ICIxNzIuODMuNDcu
MTU2In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyMzQw
NDYxNiwicGVyY2VudCI6ICIwMC4wNSJ9LCJkYXRhIjogIjkzLjg2LjIzNy4yMjUifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDIxNTkzNzM4LCJwZXJjZW50
IjogIjAwLjA1In0sImRhdGEiOiAiMjEzLjEzNC4xODIuMjUzIn0seyJoaXRzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xMSJ9
LCJkYXRhIjogIjg3Ljg4LjE3My4xNTIifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5
dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiMTU2
LjU3LjE2My40MSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50
IjogMTI0MjAxNCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjE0Ni43MC42OC4yNDUifSx7
ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDEwMzkxOTQ1Mywi
cGVyY2VudCI6ICIwMC4yMiJ9LCJkYXRhIjogIjQxLjIxNi4yMDIuMTgwIn0seyJoaXRzIjogeyJj
b3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0Njc4OTYxNCwicGVyY2VudCI6ICIw
MC4xMCJ9LCJkYXRhIjogIjQ1LjE3Ny40My4yNCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDci
fSwiYnl0ZXMiOiB7ImNvdW50IjogOTI0MTY4MiwicGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjog
IjM3Ljc2LjE2LjE4OSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNv
dW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICI4Mi4xNTQuMTYxLjI0
NSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMw
MTQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICI4MC4yOS4xMjYuMjM5In0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6
ICIwMC4xMSJ9LCJkYXRhIjogIjIxMi4yNi43OC42In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBl
cmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4w
NyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRh
IjogIjEwOS4yNDcuMTAwLjUyIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6
IHsiY291bnQiOiAzMTMyODQ3NiwicGVyY2VudCI6ICIwMC4wNyJ9LCJkYXRhIjogIjkwLjg0LjIz
Ny4xMjMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUx
OTYzNTk0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiNDUuMTEuMjAyLjgifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUyNDI2NDg1LCJwZXJjZW50
IjogIjAwLjExIn0sImRhdGEiOiAiODQuMTkuODYuOTEifSx7ImhpdHMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDExNjgxMTU0LCJwZXJjZW50IjogIjAwLjAzIn0sImRh
dGEiOiAiOTEuMTY5LjE3Ni4xMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMi
OiB7ImNvdW50IjogNDUyMjgyNTQsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0YSI6ICI1Mi4xNjcu
MTQ0LjEzNyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50Ijog
NTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICIxMzguNDguMzMuMTA4In0seyJo
aXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVy
Y2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjg2LjQxLjU2LjE4MSJ9LHsiaGl0cyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogOTE5ODQ4NTMsInBlcmNlbnQiOiAiMDAuMjAi
fSwiZGF0YSI6ICIxNjIuMjIxLjE1My41In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQi
OiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjE5
NC42MC4yNDIuNTgifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDk4NTIzNzY5LCJwZXJjZW50IjogIjAwLjIxIn0sImRhdGEiOiAiMjE3LjE0MC4xMDYuMTMi
fSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDE4NzM2NDIs
InBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICIyMTIuNDcuMTM1LjI1MCJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAi
MDAuMTEifSwiZGF0YSI6ICI5My4yMDUuMTM3LjgzIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBl
cmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4x
NSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMzUzODQyMywicGVyY2VudCI6ICIwMC4wMyJ9LCJkYXRh
IjogIjIyMS4xOTQuMTcxLjIyOCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMi
OiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICI4OS45NS4y
MTUuMTE5In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1
MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjc5LjE1NS41Mi43OCJ9LHsiaGl0
cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMjI2MjAwODIsInBlcmNl
bnQiOiAiMDAuMDUifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjE3MSJ9LHsiaGl0cyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjkzNzEyMTQsInBlcmNlbnQiOiAiMDAuMDYi
fSwiZGF0YSI6ICI5MC4xMzkuMTAxLjMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5
dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiNzMu
MjA3LjgwLjExMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50
IjogNzQyMDI3NywicGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjogIjE3Ni4xOTkuMjExLjk1In0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwi
cGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjc5LjExNi41OC45OSJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNDUyMjgyNTQsInBlcmNlbnQiOiAiMDAu
MTAifSwiZGF0YSI6ICIyMDcuNDYuMTMuMTQxIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9
LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjog
IjkxLjQ4LjEyMC4xMzEifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJj
b3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiMTQ3LjEyNC45NS4x
NzYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4
Mjk5LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiMTkxLjYuMTIuMzYifSx7ImhpdHMiOiB7
ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJw
ZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50Ijog
IjAwLjExIn0sImRhdGEiOiAiMTE0LjIwMy4xMTAuNjcifSx7ImhpdHMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDQ2NTc5NjIwLCJwZXJjZW50IjogIjAwLjEwIn0sImRh
dGEiOiAiNzEuMjU1LjE1MC4yNDEifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIw
MC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVz
IjogeyJjb3VudCI6IDEyMzUxNjkwLCJwZXJjZW50IjogIjAwLjAzIn0sImRhdGEiOiAiMTY1Ljcz
LjEyMS4yMTgifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6
IDQ1MjI4MjU0LCJwZXJjZW50IjogIjAwLjEwIn0sImRhdGEiOiAiNTIuMTY3LjE0NC4yMDMifSx7
ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJw
ZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiNDUuODAuMTM4LjIyMCJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAu
MTEifSwiZGF0YSI6ICI5My4zNC45Mi4xNTUifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2Vu
dCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0s
ImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAi
MTk5LjI3LjI1My4yMjMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJj
b3VudCI6IDExNjk0LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiNjYuMjQ5LjcwLjk4In0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA2NTc1LCJwZXJj
ZW50IjogIjAwLjAwIn0sImRhdGEiOiAiMTgxLjIyNC4xMTEuNTAifSx7ImhpdHMiOiB7ImNvdW50
IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDU3MDUzMjA4LCJwZXJjZW50IjogIjAwLjEy
In0sImRhdGEiOiAiNTguODQuNjAuMjIyIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQi
OiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjYw
LjI0My40Ni4zMiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50
IjogNDY1Nzk2MjAsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0YSI6ICIyMTAuODkuMzQuNjkifSx7
ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDc0NTc2NTIxLCJw
ZXJjZW50IjogIjAwLjE2In0sImRhdGEiOiAiNDAuNzcuMTY3LjIyIn0seyJoaXRzIjogeyJjb3Vu
dCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0MzUzOTgyNywicGVyY2VudCI6ICIwMC4w
OSJ9LCJkYXRhIjogIjMxLjUzLjEyOS4xMTYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2Vu
dCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0s
ImJ5dGVzIjogeyJjb3VudCI6IDMwNjczNSwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjUx
LjIyMi4yNTMuMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50
IjogMjcwNDMyMDQsInBlcmNlbnQiOiAiMDAuMDYifSwiZGF0YSI6ICIyMTMuNDcuMTAwLjI2In0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0ODEzMDIxMCwi
cGVyY2VudCI6ICIwMC4xMCJ9LCJkYXRhIjogIjY2LjIyNi4xMDIuMjUxIn0seyJoaXRzIjogeyJj
b3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIw
MC4xMSJ9LCJkYXRhIjogIjEzNC45Ni4xOTEuMjU0In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBl
cmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4x
NSJ9LCJieXRlcyI6IHsiY291bnQiOiAzOTEzOTIsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6
ICI1MS4yMjIuMjUzLjYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJj
b3VudCI6IDIyNjE1MzI3LCJwZXJjZW50IjogIjAwLjA1In0sImRhdGEiOiAiMTk0LjIzNi4zMC45
In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5
NCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjcyLjY1LjU5LjIwMiJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjE2NTU5MTgsInBlcmNlbnQiOiAi
MDAuMDUifSwiZGF0YSI6ICI2Mi4xODEuNTEuNDQifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3
In0sImJ5dGVzIjogeyJjb3VudCI6IDExNTcyLCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAi
ODUuMjA4Ljk2LjIwOCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNv
dW50IjogNDEwMjMyNzQsInBlcmNlbnQiOiAiMDAuMDkifSwiZGF0YSI6ICI5NC4zMi4yNDguMjAz
In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5
NCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjk0LjIzNy40My42NyJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBl
cmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMjI2MjAwODIsInBlcmNlbnQiOiAi
MDAuMDUifSwiZGF0YSI6ICI1Mi4xNjcuMTQ0LjE2OCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJw
ZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNDUyMjgyNTQsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0
YSI6ICI0MC43Ny4xNjcuNDEifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4w
OCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjog
eyJjb3VudCI6IDIwODA0MSwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjE5My43MC44MS4x
MDYifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYz
NTk0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiMTg1LjIwMy4yMTkuMjU0In0seyJoaXRz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAzNzIyMDksInBlcmNlbnQi
OiAiMDAuMDAifSwiZGF0YSI6ICI1MS4yMjIuMjUzLjE1In0seyJoaXRzIjogeyJjb3VudCI6IDIs
InBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xMSJ9LCJk
YXRhIjogIjEwMi4xNjQuOTcuMjUifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIw
MC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVz
IjogeyJjb3VudCI6IDQ1MjI4MjU0LCJwZXJjZW50IjogIjAwLjEwIn0sImRhdGEiOiAiMjA3LjQ2
LjEzLjEyNCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50Ijog
OTAxMjQyOSwicGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjogIjIxMi4yNC4xMzIuNjYifSx7Imhp
dHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJj
ZW50IjogIjAwLjExIn0sImRhdGEiOiAiNzMuMTMuMjI0LjEwNyJ9LHsiaGl0cyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjg3NjkzOTAsInBlcmNlbnQiOiAiMDAuMDYi
fSwiZGF0YSI6ICI5Mi4xODcuMTk0LjcxIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQi
OiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJi
eXRlcyI6IHsiY291bnQiOiA0MDA4MDI3NiwicGVyY2VudCI6ICIwMC4wOSJ9LCJkYXRhIjogIjE5
NS42My4xNTguMTk2In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291
bnQiOiAxNjgyMDA5MCwicGVyY2VudCI6ICIwMC4wNCJ9LCJkYXRhIjogIjc4LjE4Ni4xODQuMTE3
In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5
NCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjIxNi4xOTcuMTU3LjU1In0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyNzAzMDQ2LCJwZXJjZW50Ijog
IjAwLjAxIn0sImRhdGEiOiAiMTc3LjIyOC45Ny4yMDAifSx7ImhpdHMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjExIn0sImRh
dGEiOiAiMjAyLjE3OC4xMjQuMTI2In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRl
cyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjk4LjEx
My4yMDAuMjQzIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQi
OiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjg4LjE2MC44MC40NyJ9LHsi
aGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTc2MjMwLCJwZXJj
ZW50IjogIjAwLjAwIn0sImRhdGEiOiAiNTEuMjIyLjI1My43In0seyJoaXRzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xMSJ9
LCJkYXRhIjogIjkxLjE3My4xMDMuMTk5In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQi
OiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJi
eXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjE3
Ni4xMTQuMTI4LjEwNiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNv
dW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICI2MC4yMjYuMTY1LjU2
In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMTY5NCwi
cGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjY2LjI0OS43NS45OCJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNDY1Nzk2MjAsInBlcmNlbnQiOiAiMDAu
MTAifSwiZGF0YSI6ICI4NC4xNi4yMjQuMjI3In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9
LCJieXRlcyI6IHsiY291bnQiOiA4MTY5MTQ2NCwicGVyY2VudCI6ICIwMC4xOCJ9LCJkYXRhIjog
IjY1LjEwOS4xOC40MiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNv
dW50IjogMTE4NjAsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICIxODkuMjM4Ljg5LjI1MSJ9
LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTks
InBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICIxNzkuMjE3LjI0Ni42MSJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAi
MDAuMTEifSwiZGF0YSI6ICIxODUuMTkwLjI0MS4yMTAifSx7ImhpdHMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDQ2NTc5NjIwLCJwZXJjZW50IjogIjAwLjEwIn0sImRh
dGEiOiAiNzcuMTgyLjEyNC4zNSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMi
OiB7ImNvdW50IjogMTE2NTYsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICIxODUuMTkxLjE3
MS41In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0NTIy
ODI1NCwicGVyY2VudCI6ICIwMC4xMCJ9LCJkYXRhIjogIjE1Ny41NS4zOS4yMDUifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50
IjogIjAwLjExIn0sImRhdGEiOiAiMTAzLjEyMS42OC4xNjEifSx7ImhpdHMiOiB7ImNvdW50Ijog
MiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDExNzEyMjEwLCJwZXJjZW50IjogIjAwLjAzIn0s
ImRhdGEiOiAiNjcuMjIzLjMuMjMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIw
MC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVz
IjogeyJjb3VudCI6IDE2OTA5MjU0LCJwZXJjZW50IjogIjAwLjA0In0sImRhdGEiOiAiMTY1Ljk4
LjEzNy42NiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50Ijog
NDkyMzQ2MywicGVyY2VudCI6ICIwMC4wMSJ9LCJkYXRhIjogIjg3LjE2MS4zMC4yMjMifSx7Imhp
dHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDQ5OTA1MjUsInBlcmNl
bnQiOiAiMDAuMDEifSwiZGF0YSI6ICIxMDkuMjMzLjk1Ljg4In0seyJoaXRzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xMSJ9
LCJkYXRhIjogIjYyLjEyLjEyOS4xMzkifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5
dGVzIjogeyJjb3VudCI6IDUxOTg0ODYxLCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiOTUu
MTYuNjIuMTQifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6
IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiODUuMjE2LjE5LjU3In0seyJo
aXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVy
Y2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjg5LjI0My42MC43MyJ9LHsiaGl0cyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQi
OiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTEi
fSwiZGF0YSI6ICIxNDkuMTIwLjcwLjE4MiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50
IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICIx
NDkuMTk5LjgwLjEyOCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNv
dW50IjogMzE1MjM4LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiNTEuMjIyLjI1My44In0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDM5MjAwMzMs
InBlcmNlbnQiOiAiMDAuMjIifSwiZGF0YSI6ICI2Ny4xNzYuMjAzLjIwMyJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNDUyMjgyNTQsInBlcmNlbnQiOiAi
MDAuMTAifSwiZGF0YSI6ICI0MC43Ny4xNjcuMTkifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3
In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEi
OiAiNzQuMTMxLjE3NC4xMjIifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4w
OCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjog
eyJjb3VudCI6IDM0OTg0MCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjgyLjE0Mi4xNTIu
MTk0In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA3MDMy
ODkwLCJwZXJjZW50IjogIjAwLjAyIn0sImRhdGEiOiAiMTI4LjkwLjQzLjI3In0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6
ICIwMC4xMSJ9LCJkYXRhIjogIjc4LjkyLjExNC4xODQifSx7ImhpdHMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDg1ODkyODgsInBlcmNlbnQiOiAiMDAuMDIifSwiZGF0
YSI6ICIyMTMuMTkyLjIxNC44MCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMi
OiB7ImNvdW50IjogMTAzOTIwMDMzLCJwZXJjZW50IjogIjAwLjIyIn0sImRhdGEiOiAiNzguMzIu
MTUwLjEyMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50Ijog
OTM1NDU3NjksInBlcmNlbnQiOiAiMDAuMjAifSwiZGF0YSI6ICIxODcuMTkwLjE5OS4yMzEifSx7
ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDQ2NTc5NjIwLCJw
ZXJjZW50IjogIjAwLjEwIn0sImRhdGEiOiAiNjYuMjIzLjE3NC4xMDgifSx7ImhpdHMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAw
LjExIn0sImRhdGEiOiAiNDYuMTEyLjAuODQifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2Vu
dCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0s
ImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAi
MTA4LjE2LjUzLjE4MiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNv
dW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICIxODcuMTMuMTczLjE3
NyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjMw
MTQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICIxNTYuMzQuMTg5LjQxIn0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6
ICIwMC4xMSJ9LCJkYXRhIjogIjEzNy42My4yMTguMjA5In0seyJoaXRzIjogeyJjb3VudCI6IDIs
InBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xMSJ9LCJk
YXRhIjogIjIyMC44Ny45Ny41MyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMi
OiB7ImNvdW50IjogNDUyMjgyNTQsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0YSI6ICIxNTcuNTUu
MzkuNjEifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDQ5
ODYxMjIzLCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiODkuNjQuODUuODMifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTY4Mjk5LCJwZXJjZW50
IjogIjAwLjExIn0sImRhdGEiOiAiMjQuOC4xMDguMzAifSx7ImhpdHMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjExIn0sImRh
dGEiOiAiOTEuMTAyLjE4MC4xOTEifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIw
MC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVz
IjogeyJjb3VudCI6IDMwNjM3MDYwLCJwZXJjZW50IjogIjAwLjA3In0sImRhdGEiOiAiMTk0LjM5
LjQ0LjIyOCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50Ijog
NDUyMjgyNTQsInBlcmNlbnQiOiAiMDAuMTAifSwiZGF0YSI6ICI0MC43Ny4xNjcuNjUifSx7Imhp
dHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDExNjk0LCJwZXJjZW50
IjogIjAwLjAwIn0sImRhdGEiOiAiNjYuMjQ5Ljc1LjczIn0seyJoaXRzIjogeyJjb3VudCI6IDIs
InBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxMTY1NiwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRh
IjogIjE4NS4xOTEuMTcxLjE0In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6
IHsiY291bnQiOiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjg2LjkwLjM5
LjIyNiJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5
NjgyOTksInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICIxMDkuMjQ4LjM3LjEyNSJ9LHsiaGl0
cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNl
bnQiOiAiMDAuMTEifSwiZGF0YSI6ICIxODUuMTczLjIwMS4xODMifSx7ImhpdHMiOiB7ImNvdW50
IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50
IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDMxNTIzOCwicGVyY2VudCI6ICIwMC4wMCJ9
LCJkYXRhIjogIjUxLjIyMi4yNTMuMjAifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5
dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiMTA5
LjE1OS4xMi4xNjAifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiOTkuMTA2LjU0LjU4In0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAzODA5MTgwNSwi
cGVyY2VudCI6ICIwMC4wOCJ9LCJkYXRhIjogIjc0LjIxOS4yMzAuMjI2In0seyJoaXRzIjogeyJj
b3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0NjU3OTYyMCwicGVyY2VudCI6ICIw
MC4xMCJ9LCJkYXRhIjogIjUuMjI4LjgyLjcifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2Vu
dCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0s
ImJ5dGVzIjogeyJjb3VudCI6IDQ3MzkyOCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIjUx
LjIyMi4yNTMuOSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50
IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICIxMDguNjcuMjEuMTY2In0s
eyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxODI5MDA5Mywi
cGVyY2VudCI6ICIwMC4wNCJ9LCJkYXRhIjogIjkyLjczLjI2LjEyOSJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogODE2OTE0NjQsInBlcmNlbnQiOiAiMDAu
MTgifSwiZGF0YSI6ICI5NC4xNDAuMTkuMTE4In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9
LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjog
IjE3OC4yNC4yMzMuMTU0In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsi
Y291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjg3LjIwLjEyMy40
MyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1
OTQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICI0NS4xNzcuNzguMjA4In0seyJoaXRzIjog
eyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVyY2VudCI6
ICIwMC4xMSJ9LCJkYXRhIjogIjE4NS4xODcuOTkuNTAifSx7ImhpdHMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzMDE0LCJwZXJjZW50IjogIjAwLjExIn0sImRh
dGEiOiAiMTg1LjE5Mi4xNi41OCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMi
OiB7ImNvdW50IjogNTE5NjMwMTQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICIxMzcuMTAx
LjIzNS45NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50Ijog
MTE2NTYsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICIxODUuMTkxLjE3MS4zIn0seyJoaXRz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAzMzc4MTYsInBlcmNlbnQi
OiAiMDAuMDAifSwiZGF0YSI6ICI1MS4yMjIuMjUzLjMifSx7ImhpdHMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUzNTI5MSwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRh
IjogIjMuMjI0LjIyMC4xMDEifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4w
OCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjog
eyJjb3VudCI6IDEwMzkyNDczOCwicGVyY2VudCI6ICIwMC4yMiJ9LCJkYXRhIjogIjg2LjEyMC4y
MDguMTQ3In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAz
NDA3MTIsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICI1MS4yMjIuMjUzLjEzIn0seyJoaXRz
IjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzQxMDcxLCJwZXJjZW50
IjogIjAwLjAwIn0sImRhdGEiOiAiMTE0Ljc5LjQ2LjIzMyJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTEifSwi
ZGF0YSI6ICI3My4xNjUuMTkyLjI4In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRl
cyI6IHsiY291bnQiOiAxMzk5Mzk1LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiMTE1LjE3
OC4yMzguMTM2In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQi
OiA1MTk2MzU5NCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjg2LjMuMTYwLjQ0In0seyJo
aXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MzU5NCwicGVy
Y2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjIuMTE3LjMzLjQyIn0seyJoaXRzIjogeyJjb3VudCI6
IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0NjU3OTYyMCwicGVyY2VudCI6ICIwMC4xMCJ9
LCJkYXRhIjogIjE3OC44Ni44OC4zNSJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50Ijog
IjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0
ZXMiOiB7ImNvdW50IjogMTQ2NDI2NTUsInBlcmNlbnQiOiAiMDAuMDMifSwiZGF0YSI6ICIxMTUu
MTQ3LjM5LjI3In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQi
OiAxMTA4NjQyNCwicGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjogIjQ2LjExNC4xMzguMzEifSx7
ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDg2NDIwOTYsInBl
cmNlbnQiOiAiMDAuMDIifSwiZGF0YSI6ICIxMDYuMTk4Ljk0LjI0NSJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5ODkwNzAsInBlcmNlbnQiOiAiMDAu
MTEifSwiZGF0YSI6ICIxMzYuMjI2LjE5LjE5MyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDci
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjM1OTQsInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6
ICI3Ni4xNDkuMTMuMjUzIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsi
Y291bnQiOiA1MTk2MzAxNCwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIjY2LjE5OC4yMjIu
MjAzIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA5NDg3
MDkwLCJwZXJjZW50IjogIjAwLjAyIn0sImRhdGEiOiAiNzUuMjEyLjEzMi4xMDAifSx7ImhpdHMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDI4NDg2MjAyLCJwZXJjZW50
IjogIjAwLjA2In0sImRhdGEiOiAiMTk0Ljg3LjM5LjE1NiJ9LHsiaGl0cyI6IHsiY291bnQiOiAy
LCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNjEzMTU0OSwicGVyY2VudCI6ICIwMC4wMSJ9LCJk
YXRhIjogIjE4MS45LjE1NS42In0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6
IHsiY291bnQiOiA3NTE2NTcxLCJwZXJjZW50IjogIjAwLjAyIn0sImRhdGEiOiAiMTE5LjguMTc3
LjEwOCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTAz
NjQxNDk4LCJwZXJjZW50IjogIjAwLjIyIn0sImRhdGEiOiAiNzguODAuODAuMTk3In1dfSwib3Mi
OiB7Im1ldGFkYXRhIjogeyJieXRlcyI6IHsidG90YWwiOiB7InZhbHVlIjogNDY0MjE4MzUxNDF9
LCJhdmciOiB7InZhbHVlIjogMTEwNTI4MTc5MiwicGVyY2VudCI6ICIwMi4zOCJ9LCJtYXgiOiB7
InZhbHVlIjogMjAzODMyMzMxMDcsInBlcmNlbnQiOiAiNDMuOTEifSwibWluIjogeyJ2YWx1ZSI6
IDUxOTQsInBlcmNlbnQiOiAiMDAuMDAifX0sInZpc2l0b3JzIjogeyJ0b3RhbCI6IHsidmFsdWUi
OiAxMzQ3fSwiYXZnIjogeyJ2YWx1ZSI6IDMyLCJwZXJjZW50IjogIjAyLjM4In0sIm1heCI6IHsi
dmFsdWUiOiA1MjMsInBlcmNlbnQiOiAiMzguODMifSwibWluIjogeyJ2YWx1ZSI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifX0sImhpdHMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDI1NTN9LCJhdmciOiB7
InZhbHVlIjogNjAsInBlcmNlbnQiOiAiMDIuMzUifSwibWF4IjogeyJ2YWx1ZSI6IDEwNTgsInBl
cmNlbnQiOiAiNDEuNDQifSwibWluIjogeyJ2YWx1ZSI6IDEsInBlcmNlbnQiOiAiMDAuMDQifX0s
ImRhdGEiOiB7InRvdGFsIjogewkidmFsdWUiOiA0Mn19fSwiZGF0YSI6IFt7ImhpdHMiOiB7ImNv
dW50IjogMTA2MywicGVyY2VudCI6ICI0MS42NCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA1Mjgs
InBlcmNlbnQiOiAiMzkuMjAifSwiYnl0ZXMiOiB7ImNvdW50IjogMjA0OTY3MjQxNzAsInBlcmNl
bnQiOiAiNDQuMTUifSwiZGF0YSI6ICJXaW5kb3dzIiwiaXRlbXMiOiBbeyJoaXRzIjogeyJjb3Vu
dCI6IDEwNTgsInBlcmNlbnQiOiAiNDEuNDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNTIzLCJw
ZXJjZW50IjogIjM4LjgzIn0sImJ5dGVzIjogeyJjb3VudCI6IDIwMzgzMjMzMTA3LCJwZXJjZW50
IjogIjQzLjkxIn0sImRhdGEiOiAiV2luZG93cyAxMCJ9LHsiaGl0cyI6IHsiY291bnQiOiA1LCJw
ZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAu
MzcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTEzNDkxMDYzLCJwZXJjZW50IjogIjAwLjI0In0sImRh
dGEiOiAiV2luZG93cyA3In1dfSx7ImhpdHMiOiB7ImNvdW50IjogNzM4LCJwZXJjZW50IjogIjI4
LjkxIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDM1NCwicGVyY2VudCI6ICIyNi4yOCJ9LCJieXRl
cyI6IHsiY291bnQiOiA3NTM3NjIyODQzLCJwZXJjZW50IjogIjE2LjI0In0sImRhdGEiOiAiQ3Jh
d2xlcnMiLCJpdGVtcyI6IFt7ImhpdHMiOiB7ImNvdW50IjogMzMyLCJwZXJjZW50IjogIjEzLjAw
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIyNiwicGVyY2VudCI6ICIxNi43OCJ9LCJieXRlcyI6
IHsiY291bnQiOiA3Mzg2Njk0MDYzLCJwZXJjZW50IjogIjE1LjkxIn0sImRhdGEiOiAiYmluZ2Jv
dFwvMi4wIn0seyJoaXRzIjogeyJjb3VudCI6IDIzMSwicGVyY2VudCI6ICIwOS4wNSJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiA2LCJwZXJjZW50IjogIjAwLjQ1In0sImJ5dGVzIjogeyJjb3VudCI6
IDEzNDIwNDYsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICJTZW1hbnRpY1NjaG9sYXJCb3Qi
fSx7ImhpdHMiOiB7ImNvdW50IjogNDEsInBlcmNlbnQiOiAiMDEuNjEifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMTAsInBlcmNlbnQiOiAiMDAuNzQifSwiYnl0ZXMiOiB7ImNvdW50IjogMzg3Nzc3
MjEsInBlcmNlbnQiOiAiMDAuMDgifSwiZGF0YSI6ICJEb3RCb3RcLzEuMiJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAzOSwicGVyY2VudCI6ICIwMS41MyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzMCwi
cGVyY2VudCI6ICIwMi4yMyJ9LCJieXRlcyI6IHsiY291bnQiOiAzMTk0NjMyMCwicGVyY2VudCI6
ICIwMC4wNyJ9LCJkYXRhIjogIkdvb2dsZWJvdFwvMi4xIn0seyJoaXRzIjogeyJjb3VudCI6IDMy
LCJwZXJjZW50IjogIjAxLjI1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDI1LCJwZXJjZW50Ijog
IjAxLjg2In0sImJ5dGVzIjogeyJjb3VudCI6IDE4NjIzMiwicGVyY2VudCI6ICIwMC4wMCJ9LCJk
YXRhIjogIlNlbXJ1c2hCb3RcLzd+YmwifSx7ImhpdHMiOiB7ImNvdW50IjogMzIsInBlcmNlbnQi
OiAiMDEuMjUifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMzAsInBlcmNlbnQiOiAiMDIuMjMifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNTY1Nzc5MSwicGVyY2VudCI6ICIwMC4wMSJ9LCJkYXRhIjogIkFo
cmVmc0JvdFwvNy4wIn0seyJoaXRzIjogeyJjb3VudCI6IDE2LCJwZXJjZW50IjogIjAwLjYzIn0s
InZpc2l0b3JzIjogeyJjb3VudCI6IDE2LCJwZXJjZW50IjogIjAxLjE5In0sImJ5dGVzIjogeyJj
b3VudCI6IDY4MDQ4OTA2LCJwZXJjZW50IjogIjAwLjE1In0sImRhdGEiOiAiWWV0aVwvMS4xIn0s
eyJoaXRzIjogeyJjb3VudCI6IDExLCJwZXJjZW50IjogIjAwLjQzIn0sInZpc2l0b3JzIjogeyJj
b3VudCI6IDgsInBlcmNlbnQiOiAiMDAuNTkifSwiYnl0ZXMiOiB7ImNvdW50IjogNDYxMTg1Mywi
cGVyY2VudCI6ICIwMC4wMSJ9LCJkYXRhIjogIihBbWF6b25ib3RcLzAuMSJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAyLCJwZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjA4MDQxLCJwZXJjZW50IjogIjAw
LjAwIn0sImRhdGEiOiAiTUoxMmJvdFwvdjEuNC44In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4w
NyJ9LCJieXRlcyI6IHsiY291bnQiOiAxNDQ2NzYsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6
ICJQSFAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUx
OTQsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICJCYWlkdXNwaWRlclwvMi4wIn1dfSx7Imhp
dHMiOiB7ImNvdW50IjogMjU4LCJwZXJjZW50IjogIjEwLjExIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDE4NSwicGVyY2VudCI6ICIxMy43MyJ9LCJieXRlcyI6IHsiY291bnQiOiA3NDA3MDI4NzI2
LCJwZXJjZW50IjogIjE1Ljk2In0sImRhdGEiOiAiTGludXgiLCJpdGVtcyI6IFt7ImhpdHMiOiB7
ImNvdW50IjogMTU5LCJwZXJjZW50IjogIjA2LjIzIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEx
NSwicGVyY2VudCI6ICIwOC41NCJ9LCJieXRlcyI6IHsiY291bnQiOiA0NDQ4NzE2ODQ0LCJwZXJj
ZW50IjogIjA5LjU4In0sImRhdGEiOiAiTGludXgifSx7ImhpdHMiOiB7ImNvdW50IjogNjQsInBl
cmNlbnQiOiAiMDIuNTEifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNDQsInBlcmNlbnQiOiAiMDMu
MjcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTkxMjM5OTc1NSwicGVyY2VudCI6ICIwNC4xMiJ9LCJk
YXRhIjogImxpbnV4LWdudSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzNSwicGVyY2VudCI6ICIwMS4z
NyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyNiwicGVyY2VudCI6ICIwMS45MyJ9LCJieXRlcyI6
IHsiY291bnQiOiAxMDQ1OTEyMTI3LCJwZXJjZW50IjogIjAyLjI1In0sImRhdGEiOiAiVWJ1bnR1
In1dfSx7ImhpdHMiOiB7ImNvdW50IjogMTk4LCJwZXJjZW50IjogIjA3Ljc2In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEyNCwicGVyY2VudCI6ICIwOS4yMSJ9LCJieXRlcyI6IHsiY291bnQiOiA2
MzYwMDk0MTUxLCJwZXJjZW50IjogIjEzLjcwIn0sImRhdGEiOiAiVW5rbm93biIsIml0ZW1zIjog
W3siaGl0cyI6IHsiY291bnQiOiAxOTgsInBlcmNlbnQiOiAiMDcuNzYifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMTI0LCJwZXJjZW50IjogIjA5LjIxIn0sImJ5dGVzIjogeyJjb3VudCI6IDYzNjAw
OTQxNTEsInBlcmNlbnQiOiAiMTMuNzAifSwiZGF0YSI6ICJVbmtub3duIn1dfSx7ImhpdHMiOiB7
ImNvdW50IjogMTc1LCJwZXJjZW50IjogIjA2Ljg1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDY2
LCJwZXJjZW50IjogIjA0LjkwIn0sImJ5dGVzIjogeyJjb3VudCI6IDE0NzU2MjcyMjYsInBlcmNl
bnQiOiAiMDMuMTgifSwiZGF0YSI6ICJBbmRyb2lkIiwiaXRlbXMiOiBbeyJoaXRzIjogeyJjb3Vu
dCI6IDkxLCJwZXJjZW50IjogIjAzLjU2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDM0LCJwZXJj
ZW50IjogIjAyLjUyIn0sImJ5dGVzIjogeyJjb3VudCI6IDEwNjY5NTI4MDQsInBlcmNlbnQiOiAi
MDIuMzAifSwiZGF0YSI6ICJBbmRyb2lkIDEwIn0seyJoaXRzIjogeyJjb3VudCI6IDI5LCJwZXJj
ZW50IjogIjAxLjE0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDExLCJwZXJjZW50IjogIjAwLjgy
In0sImJ5dGVzIjogeyJjb3VudCI6IDE5Mjc1NzQyMywicGVyY2VudCI6ICIwMC40MiJ9LCJkYXRh
IjogIkFuZHJvaWQgMTMifSx7ImhpdHMiOiB7ImNvdW50IjogMTQsInBlcmNlbnQiOiAiMDAuNTUi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsi
Y291bnQiOiAxMjE2MjEzNzUsInBlcmNlbnQiOiAiMDAuMjYifSwiZGF0YSI6ICJBbmRyb2lkIDEx
In0seyJoaXRzIjogeyJjb3VudCI6IDEzLCJwZXJjZW50IjogIjAwLjUxIn0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMzc1MDQ5
MTUsInBlcmNlbnQiOiAiMDAuMDgifSwiZGF0YSI6ICJMb2xsaXBvcCA1LjEifSx7ImhpdHMiOiB7
ImNvdW50IjogMTEsInBlcmNlbnQiOiAiMDAuNDMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywi
cGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAzMDgxNDA1LCJwZXJjZW50Ijog
IjAwLjAxIn0sImRhdGEiOiAiTWFyc2htYWxsb3cgNi4wIn0seyJoaXRzIjogeyJjb3VudCI6IDQs
InBlcmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIw
MC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAzMTQwNDY2LCJwZXJjZW50IjogIjAwLjAxIn0sImRh
dGEiOiAiTm91Z2F0IDcuMSJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjE2
In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMiOiB7
ImNvdW50IjogNzY0ODg0LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiT3JlbyA4LjAifSx7
ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDIzMjY0LCJwZXJj
ZW50IjogIjAwLjAwIn0sImRhdGEiOiAiTm91Z2F0IDcuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAz
LCJwZXJjZW50IjogIjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTcwMjQ0MDQsInBlcmNlbnQiOiAiMDAuMDQifSwi
ZGF0YSI6ICJNYXJzaG1hbGxvdyA2LjAuMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwi
Ynl0ZXMiOiB7ImNvdW50IjogNTkzMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIkFuZHJv
aWQgQVBJIDM0In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQi
OiAzMjc1MDM1NiwicGVyY2VudCI6ICIwMC4wNyJ9LCJkYXRhIjogIkFuZHJvaWQgMTQifV19LHsi
aGl0cyI6IHsiY291bnQiOiA5OCwicGVyY2VudCI6ICIwMy44NCJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiA3MSwicGVyY2VudCI6ICIwNS4yNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyNzEzOTMwODA1
LCJwZXJjZW50IjogIjA1Ljg1In0sImRhdGEiOiAibWFjT1MiLCJpdGVtcyI6IFt7ImhpdHMiOiB7
ImNvdW50IjogOTIsInBlcmNlbnQiOiAiMDMuNjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNjgs
InBlcmNlbnQiOiAiMDUuMDUifSwiYnl0ZXMiOiB7ImNvdW50IjogMjU1ODAzMTM0NSwicGVyY2Vu
dCI6ICIwNS41MSJ9LCJkYXRhIjogIm1hY09TIDEwLjE1IENhdGFsaW5hIn0seyJoaXRzIjogeyJj
b3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxNTU4OTQyNjIsInBlcmNlbnQiOiAi
MDAuMzQifSwiZGF0YSI6ICJtYWNPUyAxMC4xMyBIaWdoIFNpZXJyYSJ9LHsiaGl0cyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNl
bnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5OCwicGVyY2VudCI6ICIwMC4wMCJ9
LCJkYXRhIjogIk9TIFggMTAuMTAgWW9zZW1pdGUifV19LHsiaGl0cyI6IHsiY291bnQiOiAxMSwi
cGVyY2VudCI6ICIwMC40MyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxMCwicGVyY2VudCI6ICIw
MC43NCJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDA2MzAxMzYsInBlcmNlbnQiOiAiMDAuMjIifSwi
ZGF0YSI6ICJpT1MiLCJpdGVtcyI6IFt7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIw
MC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjIyIn0sImJ5dGVz
IjogeyJjb3VudCI6IDYzOTkxOTgyLCJwZXJjZW50IjogIjAwLjE0In0sImRhdGEiOiAiaVBob25l
IE9TIDE3LjMuMSJ9LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAwLjEyIn0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIifSwiYnl0ZXMiOiB7ImNvdW50
IjogMTc2NTQ4MTIsInBlcmNlbnQiOiAiMDAuMDQifSwiZGF0YSI6ICJpUGhvbmUgT1MgMTcuMi4x
In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxMTU0OTk2
LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiaVBob25lIE9TIDE3LjAuMSJ9LHsiaGl0cyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMTQ2MDQwMjAsInBlcmNlbnQi
OiAiMDAuMDMifSwiZGF0YSI6ICJpUGhvbmUgT1MgMTcuNCJ9LHsiaGl0cyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAi
MDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMzIxODM5NiwicGVyY2VudCI6ICIwMC4wMSJ9LCJk
YXRhIjogImlQYWQgT1MgMTcuMy4xNy4zLjEifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0s
ImJ5dGVzIjogeyJjb3VudCI6IDU5MzAsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICJpUGhv
bmUgT1MgMTcuMS4xIn1dfSx7ImhpdHMiOiB7ImNvdW50IjogNywicGVyY2VudCI6ICIwMC4yNyJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAwLjMwIn0sImJ5dGVzIjogeyJj
b3VudCI6IDEyNTY5NjI2NCwicGVyY2VudCI6ICIwMC4yNyJ9LCJkYXRhIjogIkNocm9tZSBPUyIs
Iml0ZW1zIjogW3siaGl0cyI6IHsiY291bnQiOiA3LCJwZXJjZW50IjogIjAwLjI3In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMiOiB7ImNvdW50Ijog
MTI1Njk2MjY0LCJwZXJjZW50IjogIjAwLjI3In0sImRhdGEiOiAiQ3JPUyJ9XX0seyJoaXRzIjog
eyJjb3VudCI6IDUsInBlcmNlbnQiOiAiMDAuMjAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNSwi
cGVyY2VudCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyMDQ0ODA4MjAsInBlcmNlbnQi
OiAiMDAuNDQifSwiZGF0YSI6ICJCU0QiLCJpdGVtcyI6IFt7ImhpdHMiOiB7ImNvdW50IjogMiwi
cGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDgxNzkyMzI4LCJwZXJjZW50IjogIjAwLjE4In0sImRh
dGEiOiAiTmV0QlNELWZ0cFwvMjAyMzA1MTYifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0s
ImJ5dGVzIjogeyJjb3VudCI6IDQwODk2MTY0LCJwZXJjZW50IjogIjAwLjA5In0sImRhdGEiOiAi
TmV0QlNELWZ0cFwvMjAyMTAxMDYifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVz
IjogeyJjb3VudCI6IDQwODk2MTY0LCJwZXJjZW50IjogIjAwLjA5In0sImRhdGEiOiAiTmV0QlNE
LWZ0cFwvMjAyMTA2MDMifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJj
b3VudCI6IDQwODk2MTY0LCJwZXJjZW50IjogIjAwLjA5In0sImRhdGEiOiAiTmV0QlNELWZ0cFwv
MjAyMzA1MDUifV19XX0sImJyb3dzZXJzIjogeyJtZXRhZGF0YSI6IHsiYnl0ZXMiOiB7InRvdGFs
IjogeyJ2YWx1ZSI6IDQ2NDIxODM1MTQxfSwiYXZnIjogeyJ2YWx1ZSI6IDQ2NDIxODM2OCwicGVy
Y2VudCI6ICIwMS4wMCJ9LCJtYXgiOiB7InZhbHVlIjogMTE3Njg3NzA1MjcsInBlcmNlbnQiOiAi
MjUuMzUifSwibWluIjogeyJ2YWx1ZSI6IDUxOTQsInBlcmNlbnQiOiAiMDAuMDAifX0sInZpc2l0
b3JzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiAxMzQ3fSwiYXZnIjogeyJ2YWx1ZSI6IDEzLCJwZXJj
ZW50IjogIjAwLjk3In0sIm1heCI6IHsidmFsdWUiOiAyOTMsInBlcmNlbnQiOiAiMjEuNzUifSwi
bWluIjogeyJ2YWx1ZSI6IDEsInBlcmNlbnQiOiAiMDAuMDcifX0sImhpdHMiOiB7InRvdGFsIjog
eyJ2YWx1ZSI6IDI1NTN9LCJhdmciOiB7InZhbHVlIjogMjUsInBlcmNlbnQiOiAiMDAuOTgifSwi
bWF4IjogeyJ2YWx1ZSI6IDYzOCwicGVyY2VudCI6ICIyNC45OSJ9LCJtaW4iOiB7InZhbHVlIjog
MSwicGVyY2VudCI6ICIwMC4wNCJ9fSwiZGF0YSI6IHsidG90YWwiOiB7CSJ2YWx1ZSI6IDEwMH19
fSwiZGF0YSI6IFt7ImhpdHMiOiB7ImNvdW50IjogNzczLCJwZXJjZW50IjogIjMwLjI4In0sInZp
c2l0b3JzIjogeyJjb3VudCI6IDM1OCwicGVyY2VudCI6ICIyNi41OCJ9LCJieXRlcyI6IHsiY291
bnQiOiAxMzg5NTgzNTY4NSwicGVyY2VudCI6ICIyOS45MyJ9LCJkYXRhIjogIkNocm9tZSIsIml0
ZW1zIjogW3siaGl0cyI6IHsiY291bnQiOiA2MzgsInBlcmNlbnQiOiAiMjQuOTkifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMjkzLCJwZXJjZW50IjogIjIxLjc1In0sImJ5dGVzIjogeyJjb3VudCI6
IDExNzY4NzcwNTI3LCJwZXJjZW50IjogIjI1LjM1In0sImRhdGEiOiAiQ2hyb21lXC8xMjEuMC4w
LjAifSx7ImhpdHMiOiB7ImNvdW50IjogNDEsInBlcmNlbnQiOiAiMDEuNjEifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMjQsInBlcmNlbnQiOiAiMDEuNzgifSwiYnl0ZXMiOiB7ImNvdW50IjogODk2
MTgwMDQzLCJwZXJjZW50IjogIjAxLjkzIn0sImRhdGEiOiAiQ2hyb21lXC8xMjAuMC4wLjAifSx7
ImhpdHMiOiB7ImNvdW50IjogMjksInBlcmNlbnQiOiAiMDEuMTQifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1NDQyMjMxNiwi
cGVyY2VudCI6ICIwMC4xMiJ9LCJkYXRhIjogIkNocm9tZVwvNjcuMC4zMzk2Ljk5In0seyJoaXRz
IjogeyJjb3VudCI6IDgsInBlcmNlbnQiOiAiMDAuMzEifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
NCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzIwMDE3NzEsInBlcmNl
bnQiOiAiMDAuMzcifSwiZGF0YSI6ICJDaHJvbWVcLzExOS4wLjAuMCJ9LHsiaGl0cyI6IHsiY291
bnQiOiA4LCJwZXJjZW50IjogIjAwLjMxIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMjY1NDE1MzQsInBlcmNlbnQiOiAiMDAu
MDYifSwiZGF0YSI6ICJDaHJvbWVcLzg3LjAuNDI4MC4xNDEifSx7ImhpdHMiOiB7ImNvdW50Ijog
NywicGVyY2VudCI6ICIwMC4yNyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzLCJwZXJjZW50Ijog
IjAwLjIyIn0sImJ5dGVzIjogeyJjb3VudCI6IDEyNTY5NjI2NCwicGVyY2VudCI6ICIwMC4yNyJ9
LCJkYXRhIjogIkNocm9tZVwvMTIzLjAuMC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDYsInBlcmNl
bnQiOiAiMDAuMjQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4zMCJ9
LCJieXRlcyI6IHsiY291bnQiOiAxNTc3NjE5NzQsInBlcmNlbnQiOiAiMDAuMzQifSwiZGF0YSI6
ICJDaHJvbWVcLzEwOS4wLjAuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJjZW50IjogIjAw
LjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMzAifSwiYnl0ZXMi
OiB7ImNvdW50IjogNzY0ODg0LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiQ2hyb21lXC84
Ny4wLjQyODAuODgifSx7ImhpdHMiOiB7ImNvdW50IjogNCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2
aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3Vu
dCI6IDEwMzkyNjYwOCwicGVyY2VudCI6ICIwMC4yMiJ9LCJkYXRhIjogIkNocm9tZVwvMTIyLjAu
MC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxNzAy
NDQwNCwicGVyY2VudCI6ICIwMC4wNCJ9LCJkYXRhIjogIkNocm9tZVwvMTA2LjAuMC4wIn0seyJo
aXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1NzA2NTg1OCwicGVy
Y2VudCI6ICIwMC4xMiJ9LCJkYXRhIjogIkNocm9tZVwvMTE1LjAuMC4wIn0seyJoaXRzIjogeyJj
b3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0Mzg3MTg3NSwicGVyY2VudCI6ICIw
MC4wOSJ9LCJkYXRhIjogIkNocm9tZVwvMTEwLjAuMC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDMs
InBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIw
MC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxOTYxMDEzLCJwZXJjZW50IjogIjAwLjAwIn0sImRh
dGEiOiAiQ2hyb21lXC8xMjEuMC42MTY3LjE2NCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUi
fSwiYnl0ZXMiOiB7ImNvdW50IjogNzg5NDg1ODAsInBlcmNlbnQiOiAiMDAuMTcifSwiZGF0YSI6
ICJDaHJvbWVcLzkwLjAuNDQzMC4yMTIifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6
ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5
dGVzIjogeyJjb3VudCI6IDU3MDQ2MDE5LCJwZXJjZW50IjogIjAwLjEyIn0sImRhdGEiOiAiQ2hy
b21lXC8xMTcuMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJj
b3VudCI6IDExMjM4OSwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIkNocm9tZVwvMTA1LjAu
MC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2
MjM2OSwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIkNocm9tZVwvMTEzLjAuMC4wIn0seyJo
aXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwicGVy
Y2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIkNocm9tZVwvOTEuMC40NDcyLjEwNiJ9LHsiaGl0cyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogMjc3OTY1NjQsInBlcmNlbnQi
OiAiMDAuMDYifSwiZGF0YSI6ICJDaHJvbWVcLzExNi4wLjU4NDUuMTQwIn0seyJoaXRzIjogeyJj
b3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVy
Y2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2MjM2OSwicGVyY2VudCI6ICIw
MC4xMSJ9LCJkYXRhIjogIkhlYWRsZXNzQ2hyb21lXC8xMjAuMC4wLjAifSx7ImhpdHMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYyMzY5LCJwZXJjZW50IjogIjAw
LjExIn0sImRhdGEiOiAiQ2hyb21lXC8xMTIuMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDQ1MDg5ODU3LCJwZXJjZW50IjogIjAwLjEwIn0sImRh
dGEiOiAiQ2hyb21lXC8xMTYuMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6
ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5
dGVzIjogeyJjb3VudCI6IDUxOTYyMzY5LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEiOiAiQ2hy
b21lXC8xMDguMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJj
b3VudCI6IDE5OTMxNiwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIkNocm9tZVwvOTAuMC40
NDMwLjIxMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50Ijog
ODMwMTg0LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiQ2hyb21lXC8xMTguMC4wLjAifSx7
ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDU5MzAsInBlcmNl
bnQiOiAiMDAuMDAifSwiZGF0YSI6ICJDaHJvbWVcLzEyMS4wLjYxNjcuMTQ0In0seyJoaXRzIjog
eyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwi
cGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1OTMwLCJwZXJjZW50IjogIjAw
LjAwIn0sImRhdGEiOiAiQ2hyb21lXC8xMTQuMC4wLjAifV19LHsiaGl0cyI6IHsiY291bnQiOiA3
MzgsInBlcmNlbnQiOiAiMjguOTEifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMzU0LCJwZXJjZW50
IjogIjI2LjI4In0sImJ5dGVzIjogeyJjb3VudCI6IDc1Mzc2MjI4NDMsInBlcmNlbnQiOiAiMTYu
MjQifSwiZGF0YSI6ICJDcmF3bGVycyIsIml0ZW1zIjogW3siaGl0cyI6IHsiY291bnQiOiAzMzIs
InBlcmNlbnQiOiAiMTMuMDAifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMjI2LCJwZXJjZW50Ijog
IjE2Ljc4In0sImJ5dGVzIjogeyJjb3VudCI6IDczODY2OTQwNjMsInBlcmNlbnQiOiAiMTUuOTEi
fSwiZGF0YSI6ICJiaW5nYm90XC8yLjAifSx7ImhpdHMiOiB7ImNvdW50IjogMjMxLCJwZXJjZW50
IjogIjA5LjA1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDYsInBlcmNlbnQiOiAiMDAuNDUifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMTM0MjA0NiwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRhIjogIlNl
bWFudGljU2Nob2xhckJvdCJ9LHsiaGl0cyI6IHsiY291bnQiOiA0MSwicGVyY2VudCI6ICIwMS42
MSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxMCwicGVyY2VudCI6ICIwMC43NCJ9LCJieXRlcyI6
IHsiY291bnQiOiAzODc3NzcyMSwicGVyY2VudCI6ICIwMC4wOCJ9LCJkYXRhIjogIkRvdEJvdFwv
MS4yIn0seyJoaXRzIjogeyJjb3VudCI6IDM5LCJwZXJjZW50IjogIjAxLjUzIn0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDMwLCJwZXJjZW50IjogIjAyLjIzIn0sImJ5dGVzIjogeyJjb3VudCI6IDMx
OTQ2MzIwLCJwZXJjZW50IjogIjAwLjA3In0sImRhdGEiOiAiR29vZ2xlYm90XC8yLjEifSx7Imhp
dHMiOiB7ImNvdW50IjogMzIsInBlcmNlbnQiOiAiMDEuMjUifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMzAsInBlcmNlbnQiOiAiMDIuMjMifSwiYnl0ZXMiOiB7ImNvdW50IjogNTY1Nzc5MSwicGVy
Y2VudCI6ICIwMC4wMSJ9LCJkYXRhIjogIkFocmVmc0JvdFwvNy4wIn0seyJoaXRzIjogeyJjb3Vu
dCI6IDMyLCJwZXJjZW50IjogIjAxLjI1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDI1LCJwZXJj
ZW50IjogIjAxLjg2In0sImJ5dGVzIjogeyJjb3VudCI6IDE4NjIzMiwicGVyY2VudCI6ICIwMC4w
MCJ9LCJkYXRhIjogIlNlbXJ1c2hCb3RcLzd+YmwifSx7ImhpdHMiOiB7ImNvdW50IjogMTYsInBl
cmNlbnQiOiAiMDAuNjMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMTYsInBlcmNlbnQiOiAiMDEu
MTkifSwiYnl0ZXMiOiB7ImNvdW50IjogNjgwNDg5MDYsInBlcmNlbnQiOiAiMDAuMTUifSwiZGF0
YSI6ICJZZXRpXC8xLjEifSx7ImhpdHMiOiB7ImNvdW50IjogMTEsInBlcmNlbnQiOiAiMDAuNDMi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogOCwicGVyY2VudCI6ICIwMC41OSJ9LCJieXRlcyI6IHsi
Y291bnQiOiA0NjExODUzLCJwZXJjZW50IjogIjAwLjAxIn0sImRhdGEiOiAiKEFtYXpvbmJvdFwv
MC4xIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyMDgw
NDEsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICJNSjEyYm90XC92MS40LjgifSx7ImhpdHMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTQsInBlcmNlbnQiOiAi
MDAuMDAifSwiZGF0YSI6ICJCYWlkdXNwaWRlclwvMi4wIn0seyJoaXRzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxNDQ2NzYsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0
YSI6ICJQSFAifV19LHsiaGl0cyI6IHsiY291bnQiOiAzNzksInBlcmNlbnQiOiAiMTQuODUifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogMjMyLCJwZXJjZW50IjogIjE3LjIyIn0sImJ5dGVzIjogeyJj
b3VudCI6IDg0MDE5ODQ3MjgsInBlcmNlbnQiOiAiMTguMTAifSwiZGF0YSI6ICJGaXJlZm94Iiwi
aXRlbXMiOiBbeyJoaXRzIjogeyJjb3VudCI6IDI5NCwicGVyY2VudCI6ICIxMS41MiJ9LCJ2aXNp
dG9ycyI6IHsiY291bnQiOiAxNjcsInBlcmNlbnQiOiAiMTIuNDAifSwiYnl0ZXMiOiB7ImNvdW50
IjogNjM0MjUwNDE4MiwicGVyY2VudCI6ICIxMy42NiJ9LCJkYXRhIjogIkZpcmVmb3hcLzEyMi4w
In0seyJoaXRzIjogeyJjb3VudCI6IDQ3LCJwZXJjZW50IjogIjAxLjg0In0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDM2LCJwZXJjZW50IjogIjAyLjY3In0sImJ5dGVzIjogeyJjb3VudCI6IDEyNjA4
NDgwNTcsInBlcmNlbnQiOiAiMDIuNzIifSwiZGF0YSI6ICJGaXJlZm94XC8xMTUuMCJ9LHsiaGl0
cyI6IHsiY291bnQiOiAxMCwicGVyY2VudCI6ICIwMC4zOSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQi
OiA4LCJwZXJjZW50IjogIjAwLjU5In0sImJ5dGVzIjogeyJjb3VudCI6IDE5NjAyOTA0NywicGVy
Y2VudCI6ICIwMC40MiJ9LCJkYXRhIjogIkZpcmVmb3hcLzEyMS4wIn0seyJoaXRzIjogeyJjb3Vu
dCI6IDgsInBlcmNlbnQiOiAiMDAuMzEifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNSwicGVyY2Vu
dCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291bnQiOiAxMzE4MTY0MTgsInBlcmNlbnQiOiAiMDAu
MjgifSwiZGF0YSI6ICJGaXJlZm94XC8xMjMuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiA0LCJwZXJj
ZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMjIi
fSwiYnl0ZXMiOiB7ImNvdW50IjogMTAzOTMxODkzLCJwZXJjZW50IjogIjAwLjIyIn0sImRhdGEi
OiAiRmlyZWZveFwvMTE5LjAifSx7ImhpdHMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4x
MiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjog
eyJjb3VudCI6IDEwMzkyNTM4MywicGVyY2VudCI6ICIwMC4yMiJ9LCJkYXRhIjogIkZpcmVmb3hc
LzExOC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA4
MDM3OTc5NiwicGVyY2VudCI6ICIwMC4xNyJ9LCJkYXRhIjogIkZpcmVmb3hcLzExNy4wIn0seyJo
aXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA1MTk2ODI5OSwicGVy
Y2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIkZpcmVmb3hcLzEwMi4wIn0seyJoaXRzIjogeyJjb3Vu
dCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAzMTMyODQ3NiwicGVyY2VudCI6ICIwMC4w
NyJ9LCJkYXRhIjogIkZpcmVmb3hcLzEyNC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNl
bnQiOiAiMDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9
LCJieXRlcyI6IHsiY291bnQiOiA0MDAyODUwOCwicGVyY2VudCI6ICIwMC4wOSJ9LCJkYXRhIjog
IkZpcmVmb3hcLzExMi4wIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsi
Y291bnQiOiAyNTE3MDg0MywicGVyY2VudCI6ICIwMC4wNSJ9LCJkYXRhIjogIkZpcmVmb3hcLzg0
LjAifSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDMzOTM1
NTA3LCJwZXJjZW50IjogIjAwLjA3In0sImRhdGEiOiAiRmlyZWZveFwvMTA3LjAifSx7ImhpdHMi
OiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAx
LCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDExMjM4OSwicGVyY2VudCI6
ICIwMC4wMCJ9LCJkYXRhIjogIkZpcmVmb3hcLzEwNC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIw
MC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1OTMwLCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEi
OiAiRmlyZWZveFwvMTE2LjAifV19LHsiaGl0cyI6IHsiY291bnQiOiAyNTQsInBlcmNlbnQiOiAi
MDkuOTUifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMTQ2LCJwZXJjZW50IjogIjEwLjg0In0sImJ5
dGVzIjogeyJjb3VudCI6IDU2NjE0ODYyMDgsInBlcmNlbnQiOiAiMTIuMjAifSwiZGF0YSI6ICJF
ZGdlIiwiaXRlbXMiOiBbeyJoaXRzIjogeyJjb3VudCI6IDIzMSwicGVyY2VudCI6ICIwOS4wNSJ9
LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxMzAsInBlcmNlbnQiOiAiMDkuNjUifSwiYnl0ZXMiOiB7
ImNvdW50IjogNTI1MTE4NzE3OSwicGVyY2VudCI6ICIxMS4zMSJ9LCJkYXRhIjogIkVkZ1wvMTIx
LjAuMC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDcsInBlcmNlbnQiOiAiMDAuMjcifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogNSwicGVyY2VudCI6ICIwMC4zNyJ9LCJieXRlcyI6IHsiY291bnQiOiAy
MjAyMDE4MTEsInBlcmNlbnQiOiAiMDAuNDcifSwiZGF0YSI6ICJFZGdcLzEyMi4wLjAuMCJ9LHsi
aGl0cyI6IHsiY291bnQiOiA1LCJwZXJjZW50IjogIjAwLjIwIn0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0ZXMiOiB7ImNvdW50IjogMjU2MDYwNDUsInBl
cmNlbnQiOiAiMDAuMDYifSwiZGF0YSI6ICJFZGdBXC8xMjEuMC4wLjAifSx7ImhpdHMiOiB7ImNv
dW50IjogMiwicGVyY2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJj
ZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDI4NDg2MjAyLCJwZXJjZW50IjogIjAw
LjA2In0sImRhdGEiOiAiRWRnXC85Mi4wLjkwMi42NyJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJw
ZXJjZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAu
MTUifSwiYnl0ZXMiOiB7ImNvdW50IjogNjI3NzY5NzMsInBlcmNlbnQiOiAiMDAuMTQifSwiZGF0
YSI6ICJFZGdcLzEyMC4wLjAuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAw
LjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMi
OiB7ImNvdW50IjogMjQ5ODc3OTIsInBlcmNlbnQiOiAiMDAuMDUifSwiZGF0YSI6ICJFZGdcLzEy
My4wLjAuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50Ijog
MzE5Mzk0NDQsInBlcmNlbnQiOiAiMDAuMDcifSwiZGF0YSI6ICJFZGdcLzExMy4wLjE3NzQuNTAi
fSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDEwNDQ0OTg0
LCJwZXJjZW50IjogIjAwLjAyIn0sImRhdGEiOiAiRWRnXC8xMTcuMC4yMDQ1LjM2In0seyJoaXRz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50Ijog
MSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiAyODA0MDQsInBlcmNlbnQi
OiAiMDAuMDAifSwiZGF0YSI6ICJFZGdcLzExMi4wLjE3MjIuNTgifSx7ImhpdHMiOiB7ImNvdW50
IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50
IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDU1Njk0NDQsInBlcmNlbnQiOiAiMDAuMDEi
fSwiZGF0YSI6ICJFZGdcLzEyMS4wLjIyNzcuMTA1In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBl
cmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4w
NyJ9LCJieXRlcyI6IHsiY291bnQiOiA1OTMwLCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAi
RWRnXC85Ny4wLjEwNzIuNjkifV19LHsiaGl0cyI6IHsiY291bnQiOiAyMzQsInBlcmNlbnQiOiAi
MDkuMTcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMTQ5LCJwZXJjZW50IjogIjExLjA2In0sImJ5
dGVzIjogeyJjb3VudCI6IDcxMTg1MDkwNTIsInBlcmNlbnQiOiAiMTUuMzMifSwiZGF0YSI6ICJP
dGhlcnMiLCJpdGVtcyI6IFt7ImhpdHMiOiB7ImNvdW50IjogNTQsInBlcmNlbnQiOiAiMDIuMTIi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMjAsInBlcmNlbnQiOiAiMDEuNDgifSwiYnl0ZXMiOiB7
ImNvdW50IjogMTI1MTE1MDM5NiwicGVyY2VudCI6ICIwMi43MCJ9LCJkYXRhIjogImN1cmxcLzgu
MC4xIn0seyJoaXRzIjogeyJjb3VudCI6IDMyLCJwZXJjZW50IjogIjAxLjI1In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDI2LCJwZXJjZW50IjogIjAxLjkzIn0sImJ5dGVzIjogeyJjb3VudCI6IDEw
Nzc1Njc4NDgsInBlcmNlbnQiOiAiMDIuMzIifSwiZGF0YSI6ICJXZ2V0XC8xLjIwLjMifSx7Imhp
dHMiOiB7ImNvdW50IjogMjgsInBlcmNlbnQiOiAiMDEuMTAifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMjYsInBlcmNlbnQiOiAiMDEuOTMifSwiYnl0ZXMiOiB7ImNvdW50IjogMTI1NTgzNTQ3Mywi
cGVyY2VudCI6ICIwMi43MSJ9LCJkYXRhIjogImxpYmZldGNoXC8yLjAifSx7ImhpdHMiOiB7ImNv
dW50IjogMjcsInBlcmNlbnQiOiAiMDEuMDYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMjQsInBl
cmNlbnQiOiAiMDEuNzgifSwiYnl0ZXMiOiB7ImNvdW50IjogMTEwMjgzNDc2NCwicGVyY2VudCI6
ICIwMi4zOCJ9LCJkYXRhIjogIkdvLWh0dHAtY2xpZW50XC8xLjEifSx7ImhpdHMiOiB7ImNvdW50
IjogMjYsInBlcmNlbnQiOiAiMDEuMDIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMTMsInBlcmNl
bnQiOiAiMDAuOTcifSwiYnl0ZXMiOiB7ImNvdW50IjogNjA5NDIxNDM2LCJwZXJjZW50IjogIjAx
LjMxIn0sImRhdGEiOiAiV2dldFwvMS4xNy4xIn0seyJoaXRzIjogeyJjb3VudCI6IDEzLCJwZXJj
ZW50IjogIjAwLjUxIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDci
fSwiYnl0ZXMiOiB7ImNvdW50IjogMzc1MDQ5MTUsInBlcmNlbnQiOiAiMDAuMDgifSwiZGF0YSI6
ICJIVUFXRUkifSx7ImhpdHMiOiB7ImNvdW50IjogMTAsInBlcmNlbnQiOiAiMDAuMzkifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogOCwicGVyY2VudCI6ICIwMC41OSJ9LCJieXRlcyI6IHsiY291bnQi
OiAzNjAzNzUxMzQsInBlcmNlbnQiOiAiMDAuNzgifSwiZGF0YSI6ICJXZ2V0XC8xLjIxLjMifSx7
ImhpdHMiOiB7ImNvdW50IjogMTAsInBlcmNlbnQiOiAiMDAuMzkifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogNCwicGVyY2VudCI6ICIwMC4zMCJ9LCJieXRlcyI6IHsiY291bnQiOiAyMzI4OTgxMDAs
InBlcmNlbnQiOiAiMDAuNTAifSwiZGF0YSI6ICJjdXJsXC84LjIuMSJ9LHsiaGl0cyI6IHsiY291
bnQiOiAxMCwicGVyY2VudCI6ICIwMC4zOSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA3LCJwZXJj
ZW50IjogIjAwLjUyIn0sImJ5dGVzIjogeyJjb3VudCI6IDM1OTgxMzA2MCwicGVyY2VudCI6ICIw
MC43OCJ9LCJkYXRhIjogIldnZXRcLzEuMjEuNCJ9LHsiaGl0cyI6IHsiY291bnQiOiA5LCJwZXJj
ZW50IjogIjAwLjM1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDgsInBlcmNlbnQiOiAiMDAuNTki
fSwiYnl0ZXMiOiB7ImNvdW50IjogMjk4NjA2MDUyLCJwZXJjZW50IjogIjAwLjY0In0sImRhdGEi
OiAiV2dldFwvMS4yMS4yIn0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIwMC4yMiJ9LCJieXRlcyI6IHsi
Y291bnQiOiAxMjAxNzM4MzgsInBlcmNlbnQiOiAiMDAuMjYifSwiZGF0YSI6ICJXZ2V0XC8xLjE1
In0seyJoaXRzIjogeyJjb3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA5ODU0MTkz
MywicGVyY2VudCI6ICIwMC4yMSJ9LCJkYXRhIjogImN1cmxcLzguNi4wIn0seyJoaXRzIjogeyJj
b3VudCI6IDMsInBlcmNlbnQiOiAiMDAuMTIifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDUyMzY2MzMsInBlcmNlbnQiOiAi
MDAuMjMifSwiZGF0YSI6ICJXZ2V0XC8xLjE5LjQifSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVy
Y2VudCI6ICIwMC4wOCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3
In0sImJ5dGVzIjogeyJjb3VudCI6IDUxOTYzNTk0LCJwZXJjZW50IjogIjAwLjExIn0sImRhdGEi
OiAiU2VhTW9ua2V5XC8yLjUzLjExIn0seyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAi
MDAuMDgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRl
cyI6IHsiY291bnQiOiA2NDk3MjExNywicGVyY2VudCI6ICIwMC4xNCJ9LCJkYXRhIjogImN1cmxc
LzguNC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0
NjU1MjUxNCwicGVyY2VudCI6ICIwMC4xMCJ9LCJkYXRhIjogIkx5bnhcLzIuOS4wZGV2LjEwIn0s
eyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA0NTA2MTI0NSwi
cGVyY2VudCI6ICIwMC4xMCJ9LCJkYXRhIjogIldnZXRcLzEuMjEuMSJ9XX0seyJoaXRzIjogeyJj
b3VudCI6IDU2LCJwZXJjZW50IjogIjAyLjE5In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDMwLCJw
ZXJjZW50IjogIjAyLjIzIn0sImJ5dGVzIjogeyJjb3VudCI6IDk1ODI2Nzc2MCwicGVyY2VudCI6
ICIwMi4wNiJ9LCJkYXRhIjogIk9wZXJhIiwiaXRlbXMiOiBbeyJoaXRzIjogeyJjb3VudCI6IDIy
LCJwZXJjZW50IjogIjAwLjg2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDE0LCJwZXJjZW50Ijog
IjAxLjA0In0sImJ5dGVzIjogeyJjb3VudCI6IDU4Njc2NzIzOCwicGVyY2VudCI6ICIwMS4yNiJ9
LCJkYXRhIjogIk9wZXJhXC8xMDYuMC4wLjAifSx7ImhpdHMiOiB7ImNvdW50IjogMTAsInBlcmNl
bnQiOiAiMDAuMzkifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9
LCJieXRlcyI6IHsiY291bnQiOiAzMDc1NDc1LCJwZXJjZW50IjogIjAwLjAxIn0sImRhdGEiOiAi
T3BlcmFcLzcyLjYuMzc2Ny42OTkzNiJ9LHsiaGl0cyI6IHsiY291bnQiOiA5LCJwZXJjZW50Ijog
IjAwLjM1In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMTUifSwiYnl0
ZXMiOiB7ImNvdW50IjogNzE5MzY5NzAsInBlcmNlbnQiOiAiMDAuMTUifSwiZGF0YSI6ICJPcGVy
YVwvNzkuMy40MTk1Ljc2Njc0In0seyJoaXRzIjogeyJjb3VudCI6IDgsInBlcmNlbnQiOiAiMDAu
MzEifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNywicGVyY2VudCI6ICIwMC41MiJ9LCJieXRlcyI6
IHsiY291bnQiOiAyNDEzNzMzODIsInBlcmNlbnQiOiAiMDAuNTIifSwiZGF0YSI6ICJPcGVyYVwv
MTA3LjAuMC4wIn0seyJoaXRzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAiMDAuMTYifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQi
OiAzMTQwNDY2LCJwZXJjZW50IjogIjAwLjAxIn0sImRhdGEiOiAiT3BlcmFcLzc5LjYuNDE5NS43
NzE2OCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5
NjIzNjksInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6ICJPcGVyYVwvOTEuMC40NTE2LjIwIn0s
eyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNv
dW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1OTMwLCJwZXJj
ZW50IjogIjAwLjAwIn0sImRhdGEiOiAiT3BlcmFcLzEwNi4wLjAuMCAoRWRpdGlvbiBZeCkifSx7
ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsiY291
bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDU5MzAsInBlcmNl
bnQiOiAiMDAuMDAifSwiZGF0YSI6ICJPcGVyYVwvMTA2LjAuMC4wIChFZGl0aW9uIHN0ZC0xKSJ9
XX0seyJoaXRzIjogeyJjb3VudCI6IDUxLCJwZXJjZW50IjogIjAyLjAwIn0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDQwLCJwZXJjZW50IjogIjAyLjk3In0sImJ5dGVzIjogeyJjb3VudCI6IDEwMTM5
NzYxMTIsInBlcmNlbnQiOiAiMDIuMTgifSwiZGF0YSI6ICJTYWZhcmkiLCJpdGVtcyI6IFt7Imhp
dHMiOiB7ImNvdW50IjogMzUsInBlcmNlbnQiOiAiMDEuMzcifSwidmlzaXRvcnMiOiB7ImNvdW50
IjogMjYsInBlcmNlbnQiOiAiMDEuOTMifSwiYnl0ZXMiOiB7ImNvdW50IjogOTEzMzE3NTE0LCJw
ZXJjZW50IjogIjAxLjk3In0sImRhdGEiOiAiU2FmYXJpXC82MDUuMS4xNSJ9LHsiaGl0cyI6IHsi
Y291bnQiOiAxMSwicGVyY2VudCI6ICIwMC40MyJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxMCwi
cGVyY2VudCI6ICIwMC43NCJ9LCJieXRlcyI6IHsiY291bnQiOiAxMDA2MzAxMzYsInBlcmNlbnQi
OiAiMDAuMjIifSwiZGF0YSI6ICJTYWZhcmlcLzYwNC4xIn0seyJoaXRzIjogeyJjb3VudCI6IDQs
InBlcmNlbnQiOiAiMDAuMTYifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMywicGVyY2VudCI6ICIw
MC4yMiJ9LCJieXRlcyI6IHsiY291bnQiOiAyMzI2NCwicGVyY2VudCI6ICIwMC4wMCJ9LCJkYXRh
IjogIlNhZmFyaVwvNTM3LjM2In0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6
IHsiY291bnQiOiA1MTk4LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiU2FmYXJpXC82MDAu
MS4yNSJ9XX0seyJoaXRzIjogeyJjb3VudCI6IDQ5LCJwZXJjZW50IjogIjAxLjkyIn0sInZpc2l0
b3JzIjogeyJjb3VudCI6IDI3LCJwZXJjZW50IjogIjAyLjAwIn0sImJ5dGVzIjogeyJjb3VudCI6
IDE0NDc5NDAxMTMsInBlcmNlbnQiOiAiMDMuMTIifSwiZGF0YSI6ICJVbmtub3duIiwiaXRlbXMi
OiBbeyJoaXRzIjogeyJjb3VudCI6IDQ5LCJwZXJjZW50IjogIjAxLjkyIn0sInZpc2l0b3JzIjog
eyJjb3VudCI6IDI3LCJwZXJjZW50IjogIjAyLjAwIn0sImJ5dGVzIjogeyJjb3VudCI6IDE0NDc5
NDAxMTMsInBlcmNlbnQiOiAiMDMuMTIifSwiZGF0YSI6ICJVbmtub3duIn1dfSx7ImhpdHMiOiB7
ImNvdW50IjogMTcsInBlcmNlbnQiOiAiMDAuNjcifSwidmlzaXRvcnMiOiB7ImNvdW50IjogOSwi
cGVyY2VudCI6ICIwMC42NyJ9LCJieXRlcyI6IHsiY291bnQiOiAzNzY2NTIyNDUsInBlcmNlbnQi
OiAiMDAuODEifSwiZGF0YSI6ICJZYW5kZXguQnJvd3MiLCJpdGVtcyI6IFt7ImhpdHMiOiB7ImNv
dW50IjogNiwicGVyY2VudCI6ICIwMC4yNCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0LCJwZXJj
ZW50IjogIjAwLjMwIn0sImJ5dGVzIjogeyJjb3VudCI6IDE1MDkzNjIyOSwicGVyY2VudCI6ICIw
MC4zMyJ9LCJkYXRhIjogIllhQnJvd3NlclwvMjQuMS4wLjAifSx7ImhpdHMiOiB7ImNvdW50Ijog
NCwicGVyY2VudCI6ICIwMC4xNiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDU1NDQzODUwLCJwZXJjZW50IjogIjAwLjEyIn0s
ImRhdGEiOiAiWWFCcm93c2VyXC8yNC4xLjIuOTIuMDEifSx7ImhpdHMiOiB7ImNvdW50IjogMywi
cGVyY2VudCI6ICIwMC4xMiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAw
LjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDU3MDY3MTUwLCJwZXJjZW50IjogIjAwLjEyIn0sImRh
dGEiOiAiWWFCcm93c2VyXC8yMy4xMS4xLjcxNCJ9LHsiaGl0cyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjA4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDci
fSwiYnl0ZXMiOiB7ImNvdW50IjogNTE5NjgyOTksInBlcmNlbnQiOiAiMDAuMTEifSwiZGF0YSI6
ICJZYUJyb3dzZXJcLzIzLjExLjAuMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxLCJwZXJjZW50Ijog
IjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDcifSwiYnl0
ZXMiOiB7ImNvdW50IjogOTI3NDM0OCwicGVyY2VudCI6ICIwMC4wMiJ9LCJkYXRhIjogIllhQnJv
d3NlclwvMjQuMS4yLjkyLjAwIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAu
MDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNyJ9LCJieXRlcyI6
IHsiY291bnQiOiA1MTk2MjM2OSwicGVyY2VudCI6ICIwMC4xMSJ9LCJkYXRhIjogIllhQnJvd3Nl
clwvMjMuMTEuMC4yNDg5In1dfSx7ImhpdHMiOiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4w
OCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJjZW50IjogIjAwLjE1In0sImJ5dGVzIjog
eyJjb3VudCI6IDk1NjAzOTUsInBlcmNlbnQiOiAiMDAuMDIifSwiZGF0YSI6ICJNU0lFIiwiaXRl
bXMiOiBbeyJoaXRzIjogeyJjb3VudCI6IDIsInBlcmNlbnQiOiAiMDAuMDgifSwidmlzaXRvcnMi
OiB7ImNvdW50IjogMiwicGVyY2VudCI6ICIwMC4xNSJ9LCJieXRlcyI6IHsiY291bnQiOiA5NTYw
Mzk1LCJwZXJjZW50IjogIjAwLjAyIn0sImRhdGEiOiAiTVNJRVwvMTEuMCJ9XX1dfSwidmlzaXRf
dGltZSI6IHsibWV0YWRhdGEiOiB7ImJ5dGVzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiA0NjQyMTgz
NTE0MX0sImF2ZyI6IHsidmFsdWUiOiAxOTM0MjQzMjAwLCJwZXJjZW50IjogIjA0LjE3In0sIm1h
eCI6IHsidmFsdWUiOiAzMjc3NzMwODUzLCJwZXJjZW50IjogIjA3LjA2In0sIm1pbiI6IHsidmFs
dWUiOiA5NDA4Njk2MzIsInBlcmNlbnQiOiAiMDIuMDMifX0sInZpc2l0b3JzIjogeyJ0b3RhbCI6
IHsidmFsdWUiOiAxNDgyfSwiYXZnIjogeyJ2YWx1ZSI6IDYxLCJwZXJjZW50IjogIjA0LjEyIn0s
Im1heCI6IHsidmFsdWUiOiA5NiwicGVyY2VudCI6ICIwNi40OCJ9LCJtaW4iOiB7InZhbHVlIjog
MzEsInBlcmNlbnQiOiAiMDIuMDkifX0sImhpdHMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDI1NTN9
LCJhdmciOiB7InZhbHVlIjogMTA2LCJwZXJjZW50IjogIjA0LjE1In0sIm1heCI6IHsidmFsdWUi
OiAyMTIsInBlcmNlbnQiOiAiMDguMzAifSwibWluIjogeyJ2YWx1ZSI6IDQ0LCJwZXJjZW50Ijog
IjAxLjcyIn19LCJkYXRhIjogeyJ0b3RhbCI6IHsJInZhbHVlIjogMjR9fX0sImRhdGEiOiBbeyJo
aXRzIjogeyJjb3VudCI6IDgzLCJwZXJjZW50IjogIjAzLjI1In0sInZpc2l0b3JzIjogeyJjb3Vu
dCI6IDYzLCJwZXJjZW50IjogIjA0LjY4In0sImJ5dGVzIjogeyJjb3VudCI6IDE3OTQ1MjQyOTAs
InBlcmNlbnQiOiAiMDMuODcifSwiZGF0YSI6ICIwMCJ9LHsiaGl0cyI6IHsiY291bnQiOiA1OSwi
cGVyY2VudCI6ICIwMi4zMSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0NiwicGVyY2VudCI6ICIw
My40MSJ9LCJieXRlcyI6IHsiY291bnQiOiAxMjIyOTUzMjI4LCJwZXJjZW50IjogIjAyLjYzIn0s
ImRhdGEiOiAiMDEifSx7ImhpdHMiOiB7ImNvdW50IjogNTcsInBlcmNlbnQiOiAiMDIuMjMifSwi
dmlzaXRvcnMiOiB7ImNvdW50IjogNDMsInBlcmNlbnQiOiAiMDMuMTkifSwiYnl0ZXMiOiB7ImNv
dW50IjogMTAzOTc1Mjg5NCwicGVyY2VudCI6ICIwMi4yNCJ9LCJkYXRhIjogIjAyIn0seyJoaXRz
IjogeyJjb3VudCI6IDkxLCJwZXJjZW50IjogIjAzLjU2In0sInZpc2l0b3JzIjogeyJjb3VudCI6
IDUzLCJwZXJjZW50IjogIjAzLjkzIn0sImJ5dGVzIjogeyJjb3VudCI6IDE3OTY0MzA0NjIsInBl
cmNlbnQiOiAiMDMuODcifSwiZGF0YSI6ICIwMyJ9LHsiaGl0cyI6IHsiY291bnQiOiA3NiwicGVy
Y2VudCI6ICIwMi45OCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA0OCwicGVyY2VudCI6ICIwMy41
NiJ9LCJieXRlcyI6IHsiY291bnQiOiAxMjA2MDkxNzUzLCJwZXJjZW50IjogIjAyLjYwIn0sImRh
dGEiOiAiMDQifSx7ImhpdHMiOiB7ImNvdW50IjogODcsInBlcmNlbnQiOiAiMDMuNDEifSwidmlz
aXRvcnMiOiB7ImNvdW50IjogNTMsInBlcmNlbnQiOiAiMDMuOTMifSwiYnl0ZXMiOiB7ImNvdW50
IjogMTU1NjQ5NzkzMCwicGVyY2VudCI6ICIwMy4zNSJ9LCJkYXRhIjogIjA1In0seyJoaXRzIjog
eyJjb3VudCI6IDc1LCJwZXJjZW50IjogIjAyLjk0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDU1
LCJwZXJjZW50IjogIjA0LjA4In0sImJ5dGVzIjogeyJjb3VudCI6IDE1NDI1MTA2ODAsInBlcmNl
bnQiOiAiMDMuMzIifSwiZGF0YSI6ICIwNiJ9LHsiaGl0cyI6IHsiY291bnQiOiA0NCwicGVyY2Vu
dCI6ICIwMS43MiJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAzMSwicGVyY2VudCI6ICIwMi4zMCJ9
LCJieXRlcyI6IHsiY291bnQiOiA5NDA4Njk2MzIsInBlcmNlbnQiOiAiMDIuMDMifSwiZGF0YSI6
ICIwNyJ9LHsiaGl0cyI6IHsiY291bnQiOiAxMzIsInBlcmNlbnQiOiAiMDUuMTcifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogODMsInBlcmNlbnQiOiAiMDYuMTYifSwiYnl0ZXMiOiB7ImNvdW50Ijog
MjI1NTY2OTk2MiwicGVyY2VudCI6ICIwNC44NiJ9LCJkYXRhIjogIjA4In0seyJoaXRzIjogeyJj
b3VudCI6IDIxMiwicGVyY2VudCI6ICIwOC4zMCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA5Niwi
cGVyY2VudCI6ICIwNy4xMyJ9LCJieXRlcyI6IHsiY291bnQiOiAzMjc3NzMwODUzLCJwZXJjZW50
IjogIjA3LjA2In0sImRhdGEiOiAiMDkifSx7ImhpdHMiOiB7ImNvdW50IjogMTA0LCJwZXJjZW50
IjogIjA0LjA3In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDU4LCJwZXJjZW50IjogIjA0LjMxIn0s
ImJ5dGVzIjogeyJjb3VudCI6IDE5MTAyMTcwNTksInBlcmNlbnQiOiAiMDQuMTEifSwiZGF0YSI6
ICIxMCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxNzcsInBlcmNlbnQiOiAiMDYuOTMifSwidmlzaXRv
cnMiOiB7ImNvdW50IjogNzAsInBlcmNlbnQiOiAiMDUuMjAifSwiYnl0ZXMiOiB7ImNvdW50Ijog
Mjc1MDMzMTYyMywicGVyY2VudCI6ICIwNS45MiJ9LCJkYXRhIjogIjExIn0seyJoaXRzIjogeyJj
b3VudCI6IDExMiwicGVyY2VudCI6ICIwNC4zOSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA3Nywi
cGVyY2VudCI6ICIwNS43MiJ9LCJieXRlcyI6IHsiY291bnQiOiAyNjYwOTQxOTk2LCJwZXJjZW50
IjogIjA1LjczIn0sImRhdGEiOiAiMTIifSx7ImhpdHMiOiB7ImNvdW50IjogOTksInBlcmNlbnQi
OiAiMDMuODgifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNjIsInBlcmNlbnQiOiAiMDQuNjAifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMTkyNjg2ODkwNiwicGVyY2VudCI6ICIwNC4xNSJ9LCJkYXRhIjog
IjEzIn0seyJoaXRzIjogeyJjb3VudCI6IDEzNSwicGVyY2VudCI6ICIwNS4yOSJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiA3NCwicGVyY2VudCI6ICIwNS40OSJ9LCJieXRlcyI6IHsiY291bnQiOiAy
MDY4NjI3ODI0LCJwZXJjZW50IjogIjA0LjQ2In0sImRhdGEiOiAiMTQifSx7ImhpdHMiOiB7ImNv
dW50IjogMTQ0LCJwZXJjZW50IjogIjA1LjY0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDg4LCJw
ZXJjZW50IjogIjA2LjUzIn0sImJ5dGVzIjogeyJjb3VudCI6IDMxMDUzMzA2NDQsInBlcmNlbnQi
OiAiMDYuNjkifSwiZGF0YSI6ICIxNSJ9LHsiaGl0cyI6IHsiY291bnQiOiAxMzEsInBlcmNlbnQi
OiAiMDUuMTMifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNzgsInBlcmNlbnQiOiAiMDUuNzkifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMjUwNjk5NTYwOSwicGVyY2VudCI6ICIwNS40MCJ9LCJkYXRhIjog
IjE2In0seyJoaXRzIjogeyJjb3VudCI6IDEwOCwicGVyY2VudCI6ICIwNC4yMyJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiA3MiwicGVyY2VudCI6ICIwNS4zNSJ9LCJieXRlcyI6IHsiY291bnQiOiAy
NDY0NTA5NTMzLCJwZXJjZW50IjogIjA1LjMxIn0sImRhdGEiOiAiMTcifSx7ImhpdHMiOiB7ImNv
dW50IjogMTI4LCJwZXJjZW50IjogIjA1LjAxIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDYzLCJw
ZXJjZW50IjogIjA0LjY4In0sImJ5dGVzIjogeyJjb3VudCI6IDIxMjQzMzUxNTgsInBlcmNlbnQi
OiAiMDQuNTgifSwiZGF0YSI6ICIxOCJ9LHsiaGl0cyI6IHsiY291bnQiOiAxNTcsInBlcmNlbnQi
OiAiMDYuMTUifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNjQsInBlcmNlbnQiOiAiMDQuNzUifSwi
Ynl0ZXMiOiB7ImNvdW50IjogMTk0MTgxNTcxOCwicGVyY2VudCI6ICIwNC4xOCJ9LCJkYXRhIjog
IjE5In0seyJoaXRzIjogeyJjb3VudCI6IDc4LCJwZXJjZW50IjogIjAzLjA2In0sInZpc2l0b3Jz
IjogeyJjb3VudCI6IDQ4LCJwZXJjZW50IjogIjAzLjU2In0sImJ5dGVzIjogeyJjb3VudCI6IDE2
NjQ1Nzc4MzksInBlcmNlbnQiOiAiMDMuNTkifSwiZGF0YSI6ICIyMCJ9LHsiaGl0cyI6IHsiY291
bnQiOiAxMDcsInBlcmNlbnQiOiAiMDQuMTkifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNjQsInBl
cmNlbnQiOiAiMDQuNzUifSwiYnl0ZXMiOiB7ImNvdW50IjogMTg1MDU4OTc4OCwicGVyY2VudCI6
ICIwMy45OSJ9LCJkYXRhIjogIjIxIn0seyJoaXRzIjogeyJjb3VudCI6IDk5LCJwZXJjZW50Ijog
IjAzLjg4In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDUzLCJwZXJjZW50IjogIjAzLjkzIn0sImJ5
dGVzIjogeyJjb3VudCI6IDE2OTE4NDg4NjcsInBlcmNlbnQiOiAiMDMuNjQifSwiZGF0YSI6ICIy
MiJ9LHsiaGl0cyI6IHsiY291bnQiOiA1OCwicGVyY2VudCI6ICIwMi4yNyJ9LCJ2aXNpdG9ycyI6
IHsiY291bnQiOiA0MCwicGVyY2VudCI6ICIwMi45NyJ9LCJieXRlcyI6IHsiY291bnQiOiAxMTIx
ODEyODkzLCJwZXJjZW50IjogIjAyLjQyIn0sImRhdGEiOiAiMjMifV19LCJyZWZlcnJpbmdfc2l0
ZXMiOiB7Im1ldGFkYXRhIjogeyJieXRlcyI6IHsidG90YWwiOiB7InZhbHVlIjogMzAxMzA4Mzgx
MTl9LCJhdmciOiB7InZhbHVlIjogNTAyMTgwNjA4MCwicGVyY2VudCI6ICIxNi42NyJ9LCJtYXgi
OiB7InZhbHVlIjogMjk3NzM5NTI2MjIsInBlcmNlbnQiOiAiOTguODIifSwibWluIjogeyJ2YWx1
ZSI6IDU3MjYsInBlcmNlbnQiOiAiMDAuMDAifX0sInZpc2l0b3JzIjogeyJ0b3RhbCI6IHsidmFs
dWUiOiA4MTJ9LCJhdmciOiB7InZhbHVlIjogMTM1LCJwZXJjZW50IjogIjE2LjYzIn0sIm1heCI6
IHsidmFsdWUiOiA3OTEsInBlcmNlbnQiOiAiOTcuNDEifSwibWluIjogeyJ2YWx1ZSI6IDEsInBl
cmNlbnQiOiAiMDAuMTIifX0sImhpdHMiOiB7InRvdGFsIjogeyJ2YWx1ZSI6IDE1MDV9LCJhdmci
OiB7InZhbHVlIjogMjUwLCJwZXJjZW50IjogIjE2LjYxIn0sIm1heCI6IHsidmFsdWUiOiAxNDY4
LCJwZXJjZW50IjogIjk3LjU0In0sIm1pbiI6IHsidmFsdWUiOiAxLCJwZXJjZW50IjogIjAwLjA3
In19LCJkYXRhIjogeyJ0b3RhbCI6IHsJInZhbHVlIjogNn19fSwiZGF0YSI6IFt7ImhpdHMiOiB7
ImNvdW50IjogMTQ2OCwicGVyY2VudCI6ICI1Ny41MCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiA3
OTEsInBlcmNlbnQiOiAiNTguNzIifSwiYnl0ZXMiOiB7ImNvdW50IjogMjk3NzM5NTI2MjIsInBl
cmNlbnQiOiAiNjQuMTQifSwiZGF0YSI6ICJ4ZW5wcm9qZWN0Lm9yZyJ9LHsiaGl0cyI6IHsiY291
bnQiOiAyMywicGVyY2VudCI6ICIwMC45MCJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAxNCwicGVy
Y2VudCI6ICIwMS4wNCJ9LCJieXRlcyI6IHsiY291bnQiOiAyNzc3NzE0ODAsInBlcmNlbnQiOiAi
MDAuNjAifSwiZGF0YSI6ICJkb3dubG9hZHMueGVucHJvamVjdC5vcmcifSx7ImhpdHMiOiB7ImNv
dW50IjogOCwicGVyY2VudCI6ICIwMC4zMSJ9LCJ2aXNpdG9ycyI6IHsiY291bnQiOiAyLCJwZXJj
ZW50IjogIjAwLjE1In0sImJ5dGVzIjogeyJjb3VudCI6IDMxNzkwODkzLCJwZXJjZW50IjogIjAw
LjA3In0sImRhdGEiOiAid2lraS54ZW5wcm9qZWN0Lm9yZyJ9LHsiaGl0cyI6IHsiY291bnQiOiA0
LCJwZXJjZW50IjogIjAwLjE2In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDQsInBlcmNlbnQiOiAi
MDAuMzAifSwiYnl0ZXMiOiB7ImNvdW50IjogNzY0ODg0LCJwZXJjZW50IjogIjAwLjAwIn0sImRh
dGEiOiAiYmFpZHUuY29tIn0seyJoaXRzIjogeyJjb3VudCI6IDEsInBlcmNlbnQiOiAiMDAuMDQi
fSwidmlzaXRvcnMiOiB7ImNvdW50IjogMCwicGVyY2VudCI6ICIwMC4wMCJ9LCJieXRlcyI6IHsi
Y291bnQiOiA1NzI2LCJwZXJjZW50IjogIjAwLjAwIn0sImRhdGEiOiAiMTcyLjI1NS41MS4xMTki
fSx7ImhpdHMiOiB7ImNvdW50IjogMSwicGVyY2VudCI6ICIwMC4wNCJ9LCJ2aXNpdG9ycyI6IHsi
Y291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA3In0sImJ5dGVzIjogeyJjb3VudCI6IDQ2NTUyNTE0
LCJwZXJjZW50IjogIjAwLjEwIn0sImRhdGEiOiAid3d3LnNsYWNrYnVpbGRzLm9yZyJ9XX0sInN0
YXR1c19jb2RlcyI6IHsibWV0YWRhdGEiOiB7ImJ5dGVzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiA0
NjQyMTgzNTE0MX0sImF2ZyI6IHsidmFsdWUiOiAxMTYwNTQ1ODk0NCwicGVyY2VudCI6ICIyNS4w
MCJ9LCJtYXgiOiB7InZhbHVlIjogNDM5NjMwNTk3MzgsInBlcmNlbnQiOiAiOTQuNzAifSwibWlu
IjogeyJ2YWx1ZSI6IDU4MSwicGVyY2VudCI6ICIwMC4wMCJ9fSwidmlzaXRvcnMiOiB7InRvdGFs
IjogeyJ2YWx1ZSI6IDE0MTV9LCJhdmciOiB7InZhbHVlIjogMzUzLCJwZXJjZW50IjogIjI0Ljk1
In0sIm1heCI6IHsidmFsdWUiOiAxMzQyLCJwZXJjZW50IjogIjk0Ljg0In0sIm1pbiI6IHsidmFs
dWUiOiAxLCJwZXJjZW50IjogIjAwLjA3In19LCJoaXRzIjogeyJ0b3RhbCI6IHsidmFsdWUiOiAy
NTUzfSwiYXZnIjogeyJ2YWx1ZSI6IDYzOCwicGVyY2VudCI6ICIyNC45OSJ9LCJtYXgiOiB7InZh
bHVlIjogMjEzMCwicGVyY2VudCI6ICI4My40MyJ9LCJtaW4iOiB7InZhbHVlIjogMSwicGVyY2Vu
dCI6ICIwMC4wNCJ9fSwiZGF0YSI6IHsidG90YWwiOiB7CSJ2YWx1ZSI6IDR9fX0sImRhdGEiOiBb
eyJoaXRzIjogeyJjb3VudCI6IDI1NDksInBlcmNlbnQiOiAiOTkuODQifSwidmlzaXRvcnMiOiB7
ImNvdW50IjogMTQxNCwicGVyY2VudCI6ICIxMDQuOTcifSwiYnl0ZXMiOiB7ImNvdW50IjogNDY0
MjE4MTczNDUsInBlcmNlbnQiOiAiMTAwLjAwIn0sImRhdGEiOiAiMnh4IFN1Y2Nlc3MiLCJpdGVt
cyI6IFt7ImhpdHMiOiB7ImNvdW50IjogMjEzMCwicGVyY2VudCI6ICI4My40MyJ9LCJ2aXNpdG9y
cyI6IHsiY291bnQiOiAxMzQyLCJwZXJjZW50IjogIjk5LjYzIn0sImJ5dGVzIjogeyJjb3VudCI6
IDQzOTYzMDU5NzM4LCJwZXJjZW50IjogIjk0LjcwIn0sImRhdGEiOiAiMjAwIC0gT0s6IFRoZSBy
ZXF1ZXN0IHNlbnQgYnkgdGhlIGNsaWVudCB3YXMgc3VjY2Vzc2Z1bCJ9LHsiaGl0cyI6IHsiY291
bnQiOiA0MTksInBlcmNlbnQiOiAiMTYuNDEifSwidmlzaXRvcnMiOiB7ImNvdW50IjogNzIsInBl
cmNlbnQiOiAiMDUuMzUifSwiYnl0ZXMiOiB7ImNvdW50IjogMjQ1ODc1NzYwNywicGVyY2VudCI6
ICIwNS4zMCJ9LCJkYXRhIjogIjIwNiAtIFBhcnRpYWwgQ29udGVudDogVGhlIHBhcnRpYWwgR0VU
IGhhcyBiZWVuIHN1Y2Nlc3NmdWwifV19LHsiaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50Ijog
IjAwLjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDAsInBlcmNlbnQiOiAiMDAuMDAifSwiYnl0
ZXMiOiB7ImNvdW50IjogMTcyMTUsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICI0eHggQ2xp
ZW50IEVycm9ycyIsIml0ZW1zIjogW3siaGl0cyI6IHsiY291bnQiOiAzLCJwZXJjZW50IjogIjAw
LjEyIn0sInZpc2l0b3JzIjogeyJjb3VudCI6IDAsInBlcmNlbnQiOiAiMDAuMDAifSwiYnl0ZXMi
OiB7ImNvdW50IjogMTcyMTUsInBlcmNlbnQiOiAiMDAuMDAifSwiZGF0YSI6ICI0MDQgLSBOb3Qg
Rm91bmQ6IFJlcXVlc3RlZCByZXNvdXJjZSBjb3VsZCBub3QgYmUgZm91bmQifV19LHsiaGl0cyI6
IHsiY291bnQiOiAxLCJwZXJjZW50IjogIjAwLjA0In0sInZpc2l0b3JzIjogeyJjb3VudCI6IDEs
InBlcmNlbnQiOiAiMDAuMDcifSwiYnl0ZXMiOiB7ImNvdW50IjogNTgxLCJwZXJjZW50IjogIjAw
LjAwIn0sImRhdGEiOiAiM3h4IFJlZGlyZWN0aW9uIiwiaXRlbXMiOiBbeyJoaXRzIjogeyJjb3Vu
dCI6IDEsInBlcmNlbnQiOiAiMDAuMDQifSwidmlzaXRvcnMiOiB7ImNvdW50IjogMSwicGVyY2Vu
dCI6ICIwMC4wNyJ9LCJieXRlcyI6IHsiY291bnQiOiA1ODEsInBlcmNlbnQiOiAiMDAuMDAifSwi
ZGF0YSI6ICIzMDIgLSBNb3ZlZCBUZW1wb3JhcmlseSAocmVkaXJlY3QpIn1dfV19fTwvc2NyaXB0
PjxzY3JpcHQ+LyogaHR0cHM6Ly9kM2pzLm9yZyB2Ny44LjQgQ29weXJpZ2h0IDIwMTAtMjAyMyBN
aWtlIEJvc3RvY2sgKi8hZnVuY3Rpb24odCxuKXsib2JqZWN0Ij09dHlwZW9mIGV4cG9ydHMmJiJ1
bmRlZmluZWQiIT10eXBlb2YgbW9kdWxlP24oZXhwb3J0cyk6ImZ1bmN0aW9uIj09dHlwZW9mIGRl
ZmluZSYmZGVmaW5lLmFtZD9kZWZpbmUoWyJleHBvcnRzIl0sbik6bigodD0idW5kZWZpbmVkIiE9
dHlwZW9mIGdsb2JhbFRoaXM/Z2xvYmFsVGhpczp0fHxzZWxmKS5kMz10LmQzfHx7fSl9KHRoaXMs
KGZ1bmN0aW9uKHQpeyJ1c2Ugc3RyaWN0IjtmdW5jdGlvbiBuKHQsbil7cmV0dXJuIG51bGw9PXR8
fG51bGw9PW4/TmFOOnQ8bj8tMTp0Pm4/MTp0Pj1uPzA6TmFOfWZ1bmN0aW9uIGUodCxuKXtyZXR1
cm4gbnVsbD09dHx8bnVsbD09bj9OYU46bjx0Py0xOm4+dD8xOm4+PXQ/MDpOYU59ZnVuY3Rpb24g
cih0KXtsZXQgcixvLGE7ZnVuY3Rpb24gdSh0LG4sZT0wLGk9dC5sZW5ndGgpe2lmKGU8aSl7aWYo
MCE9PXIobixuKSlyZXR1cm4gaTtkb3tjb25zdCByPWUraT4+PjE7byh0W3JdLG4pPDA/ZT1yKzE6
aT1yfXdoaWxlKGU8aSl9cmV0dXJuIGV9cmV0dXJuIDIhPT10Lmxlbmd0aD8ocj1uLG89KGUscik9
Pm4odChlKSxyKSxhPShuLGUpPT50KG4pLWUpOihyPXQ9PT1ufHx0PT09ZT90Omksbz10LGE9dCks
e2xlZnQ6dSxjZW50ZXI6ZnVuY3Rpb24odCxuLGU9MCxyPXQubGVuZ3RoKXtjb25zdCBpPXUodCxu
LGUsci0xKTtyZXR1cm4gaT5lJiZhKHRbaS0xXSxuKT4tYSh0W2ldLG4pP2ktMTppfSxyaWdodDpm
dW5jdGlvbih0LG4sZT0wLGk9dC5sZW5ndGgpe2lmKGU8aSl7aWYoMCE9PXIobixuKSlyZXR1cm4g
aTtkb3tjb25zdCByPWUraT4+PjE7byh0W3JdLG4pPD0wP2U9cisxOmk9cn13aGlsZShlPGkpfXJl
dHVybiBlfX19ZnVuY3Rpb24gaSgpe3JldHVybiAwfWZ1bmN0aW9uIG8odCl7cmV0dXJuIG51bGw9
PT10P05hTjordH1mdW5jdGlvbiphKHQsbil7aWYodm9pZCAwPT09bilmb3IobGV0IG4gb2YgdClu
dWxsIT1uJiYobj0rbik+PW4mJih5aWVsZCBuKTtlbHNle2xldCBlPS0xO2ZvcihsZXQgciBvZiB0
KW51bGwhPShyPW4ociwrK2UsdCkpJiYocj0rcik+PXImJih5aWVsZCByKX19Y29uc3QgdT1yKG4p
LGM9dS5yaWdodCxmPXUubGVmdCxzPXIobykuY2VudGVyO3ZhciBsPWM7Y29uc3QgaD1wKHYpLGQ9
cCgoZnVuY3Rpb24odCl7Y29uc3Qgbj12KHQpO3JldHVybih0LGUscixpLG8pPT57bih0LGUsKHI8
PD0yKSswLChpPDw9MikrMCxvPDw9Miksbih0LGUscisxLGkrMSxvKSxuKHQsZSxyKzIsaSsyLG8p
LG4odCxlLHIrMyxpKzMsbyl9fSkpO2Z1bmN0aW9uIHAodCl7cmV0dXJuIGZ1bmN0aW9uKG4sZSxy
PWUpe2lmKCEoKGU9K2UpPj0wKSl0aHJvdyBuZXcgUmFuZ2VFcnJvcigiaW52YWxpZCByeCIpO2lm
KCEoKHI9K3IpPj0wKSl0aHJvdyBuZXcgUmFuZ2VFcnJvcigiaW52YWxpZCByeSIpO2xldHtkYXRh
Omksd2lkdGg6byxoZWlnaHQ6YX09bjtpZighKChvPU1hdGguZmxvb3IobykpPj0wKSl0aHJvdyBu
ZXcgUmFuZ2VFcnJvcigiaW52YWxpZCB3aWR0aCIpO2lmKCEoKGE9TWF0aC5mbG9vcih2b2lkIDAh
PT1hP2E6aS5sZW5ndGgvbykpPj0wKSl0aHJvdyBuZXcgUmFuZ2VFcnJvcigiaW52YWxpZCBoZWln
aHQiKTtpZighb3x8IWF8fCFlJiYhcilyZXR1cm4gbjtjb25zdCB1PWUmJnQoZSksYz1yJiZ0KHIp
LGY9aS5zbGljZSgpO3JldHVybiB1JiZjPyhnKHUsZixpLG8sYSksZyh1LGksZixvLGEpLGcodSxm
LGksbyxhKSx5KGMsaSxmLG8sYSkseShjLGYsaSxvLGEpLHkoYyxpLGYsbyxhKSk6dT8oZyh1LGks
ZixvLGEpLGcodSxmLGksbyxhKSxnKHUsaSxmLG8sYSkpOmMmJih5KGMsaSxmLG8sYSkseShjLGYs
aSxvLGEpLHkoYyxpLGYsbyxhKSksbn19ZnVuY3Rpb24gZyh0LG4sZSxyLGkpe2ZvcihsZXQgbz0w
LGE9cippO288YTspdChuLGUsbyxvKz1yLDEpfWZ1bmN0aW9uIHkodCxuLGUscixpKXtmb3IobGV0
IG89MCxhPXIqaTtvPHI7KytvKXQobixlLG8sbythLHIpfWZ1bmN0aW9uIHYodCl7Y29uc3Qgbj1N
YXRoLmZsb29yKHQpO2lmKG49PT10KXJldHVybiBmdW5jdGlvbih0KXtjb25zdCBuPTIqdCsxO3Jl
dHVybihlLHIsaSxvLGEpPT57aWYoISgoby09YSk+PWkpKXJldHVybjtsZXQgdT10KnJbaV07Y29u
c3QgYz1hKnQ7Zm9yKGxldCB0PWksbj1pK2M7dDxuO3QrPWEpdSs9cltNYXRoLm1pbihvLHQpXTtm
b3IobGV0IHQ9aSxmPW87dDw9Zjt0Kz1hKXUrPXJbTWF0aC5taW4obyx0K2MpXSxlW3RdPXUvbix1
LT1yW01hdGgubWF4KGksdC1jKV19fSh0KTtjb25zdCBlPXQtbixyPTIqdCsxO3JldHVybih0LGks
byxhLHUpPT57aWYoISgoYS09dSk+PW8pKXJldHVybjtsZXQgYz1uKmlbb107Y29uc3QgZj11Km4s
cz1mK3U7Zm9yKGxldCB0PW8sbj1vK2Y7dDxuO3QrPXUpYys9aVtNYXRoLm1pbihhLHQpXTtmb3Io
bGV0IG49byxsPWE7bjw9bDtuKz11KWMrPWlbTWF0aC5taW4oYSxuK2YpXSx0W25dPShjK2UqKGlb
TWF0aC5tYXgobyxuLXMpXStpW01hdGgubWluKGEsbitzKV0pKS9yLGMtPWlbTWF0aC5tYXgobyxu
LWYpXX19ZnVuY3Rpb24gXyh0LG4pe2xldCBlPTA7aWYodm9pZCAwPT09bilmb3IobGV0IG4gb2Yg
dCludWxsIT1uJiYobj0rbik+PW4mJisrZTtlbHNle2xldCByPS0xO2ZvcihsZXQgaSBvZiB0KW51
bGwhPShpPW4oaSwrK3IsdCkpJiYoaT0raSk+PWkmJisrZX1yZXR1cm4gZX1mdW5jdGlvbiBiKHQp
e3JldHVybiAwfHQubGVuZ3RofWZ1bmN0aW9uIG0odCl7cmV0dXJuISh0PjApfWZ1bmN0aW9uIHgo
dCl7cmV0dXJuIm9iamVjdCIhPXR5cGVvZiB0fHwibGVuZ3RoImluIHQ/dDpBcnJheS5mcm9tKHQp
fWZ1bmN0aW9uIHcodCxuKXtsZXQgZSxyPTAsaT0wLG89MDtpZih2b2lkIDA9PT1uKWZvcihsZXQg
biBvZiB0KW51bGwhPW4mJihuPStuKT49biYmKGU9bi1pLGkrPWUvKytyLG8rPWUqKG4taSkpO2Vs
c2V7bGV0IGE9LTE7Zm9yKGxldCB1IG9mIHQpbnVsbCE9KHU9bih1LCsrYSx0KSkmJih1PSt1KT49
dSYmKGU9dS1pLGkrPWUvKytyLG8rPWUqKHUtaSkpfWlmKHI+MSlyZXR1cm4gby8oci0xKX1mdW5j
dGlvbiBNKHQsbil7Y29uc3QgZT13KHQsbik7cmV0dXJuIGU/TWF0aC5zcXJ0KGUpOmV9ZnVuY3Rp
b24gVCh0LG4pe2xldCBlLHI7aWYodm9pZCAwPT09bilmb3IoY29uc3QgbiBvZiB0KW51bGwhPW4m
Jih2b2lkIDA9PT1lP24+PW4mJihlPXI9bik6KGU+biYmKGU9bikscjxuJiYocj1uKSkpO2Vsc2V7
bGV0IGk9LTE7Zm9yKGxldCBvIG9mIHQpbnVsbCE9KG89bihvLCsraSx0KSkmJih2b2lkIDA9PT1l
P28+PW8mJihlPXI9byk6KGU+byYmKGU9bykscjxvJiYocj1vKSkpfXJldHVybltlLHJdfWNsYXNz
IEF7Y29uc3RydWN0b3IoKXt0aGlzLl9wYXJ0aWFscz1uZXcgRmxvYXQ2NEFycmF5KDMyKSx0aGlz
Ll9uPTB9YWRkKHQpe2NvbnN0IG49dGhpcy5fcGFydGlhbHM7bGV0IGU9MDtmb3IobGV0IHI9MDty
PHRoaXMuX24mJnI8MzI7cisrKXtjb25zdCBpPW5bcl0sbz10K2ksYT1NYXRoLmFicyh0KTxNYXRo
LmFicyhpKT90LShvLWkpOmktKG8tdCk7YSYmKG5bZSsrXT1hKSx0PW99cmV0dXJuIG5bZV09dCx0
aGlzLl9uPWUrMSx0aGlzfXZhbHVlT2YoKXtjb25zdCB0PXRoaXMuX3BhcnRpYWxzO2xldCBuLGUs
cixpPXRoaXMuX24sbz0wO2lmKGk+MCl7Zm9yKG89dFstLWldO2k+MCYmKG49byxlPXRbLS1pXSxv
PW4rZSxyPWUtKG8tbiksIXIpOyk7aT4wJiYocjwwJiZ0W2ktMV08MHx8cj4wJiZ0W2ktMV0+MCkm
JihlPTIqcixuPW8rZSxlPT1uLW8mJihvPW4pKX1yZXR1cm4gb319Y2xhc3MgSW50ZXJuTWFwIGV4
dGVuZHMgTWFwe2NvbnN0cnVjdG9yKHQsbj1rKXtpZihzdXBlcigpLE9iamVjdC5kZWZpbmVQcm9w
ZXJ0aWVzKHRoaXMse19pbnRlcm46e3ZhbHVlOm5ldyBNYXB9LF9rZXk6e3ZhbHVlOm59fSksbnVs
bCE9dClmb3IoY29uc3RbbixlXW9mIHQpdGhpcy5zZXQobixlKX1nZXQodCl7cmV0dXJuIHN1cGVy
LmdldChTKHRoaXMsdCkpfWhhcyh0KXtyZXR1cm4gc3VwZXIuaGFzKFModGhpcyx0KSl9c2V0KHQs
bil7cmV0dXJuIHN1cGVyLnNldChFKHRoaXMsdCksbil9ZGVsZXRlKHQpe3JldHVybiBzdXBlci5k
ZWxldGUoTih0aGlzLHQpKX19Y2xhc3MgSW50ZXJuU2V0IGV4dGVuZHMgU2V0e2NvbnN0cnVjdG9y
KHQsbj1rKXtpZihzdXBlcigpLE9iamVjdC5kZWZpbmVQcm9wZXJ0aWVzKHRoaXMse19pbnRlcm46
e3ZhbHVlOm5ldyBNYXB9LF9rZXk6e3ZhbHVlOm59fSksbnVsbCE9dClmb3IoY29uc3QgbiBvZiB0
KXRoaXMuYWRkKG4pfWhhcyh0KXtyZXR1cm4gc3VwZXIuaGFzKFModGhpcyx0KSl9YWRkKHQpe3Jl
dHVybiBzdXBlci5hZGQoRSh0aGlzLHQpKX1kZWxldGUodCl7cmV0dXJuIHN1cGVyLmRlbGV0ZShO
KHRoaXMsdCkpfX1mdW5jdGlvbiBTKHtfaW50ZXJuOnQsX2tleTpufSxlKXtjb25zdCByPW4oZSk7
cmV0dXJuIHQuaGFzKHIpP3QuZ2V0KHIpOmV9ZnVuY3Rpb24gRSh7X2ludGVybjp0LF9rZXk6bn0s
ZSl7Y29uc3Qgcj1uKGUpO3JldHVybiB0LmhhcyhyKT90LmdldChyKToodC5zZXQocixlKSxlKX1m
dW5jdGlvbiBOKHtfaW50ZXJuOnQsX2tleTpufSxlKXtjb25zdCByPW4oZSk7cmV0dXJuIHQuaGFz
KHIpJiYoZT10LmdldChyKSx0LmRlbGV0ZShyKSksZX1mdW5jdGlvbiBrKHQpe3JldHVybiBudWxs
IT09dCYmIm9iamVjdCI9PXR5cGVvZiB0P3QudmFsdWVPZigpOnR9ZnVuY3Rpb24gQyh0KXtyZXR1
cm4gdH1mdW5jdGlvbiBQKHQsLi4ubil7cmV0dXJuIHEodCxDLEMsbil9ZnVuY3Rpb24geih0LC4u
Lm4pe3JldHVybiBxKHQsQXJyYXkuZnJvbSxDLG4pfWZ1bmN0aW9uICQodCxuKXtmb3IobGV0IGU9
MSxyPW4ubGVuZ3RoO2U8cjsrK2UpdD10LmZsYXRNYXAoKHQ9PnQucG9wKCkubWFwKCgoW24sZV0p
PT5bLi4udCxuLGVdKSkpKTtyZXR1cm4gdH1mdW5jdGlvbiBEKHQsbiwuLi5lKXtyZXR1cm4gcSh0
LEMsbixlKX1mdW5jdGlvbiBSKHQsbiwuLi5lKXtyZXR1cm4gcSh0LEFycmF5LmZyb20sbixlKX1m
dW5jdGlvbiBGKHQpe2lmKDEhPT10Lmxlbmd0aCl0aHJvdyBuZXcgRXJyb3IoImR1cGxpY2F0ZSBr
ZXkiKTtyZXR1cm4gdFswXX1mdW5jdGlvbiBxKHQsbixlLHIpe3JldHVybiBmdW5jdGlvbiB0KGks
byl7aWYobz49ci5sZW5ndGgpcmV0dXJuIGUoaSk7Y29uc3QgYT1uZXcgSW50ZXJuTWFwLHU9cltv
KytdO2xldCBjPS0xO2Zvcihjb25zdCB0IG9mIGkpe2NvbnN0IG49dSh0LCsrYyxpKSxlPWEuZ2V0
KG4pO2U/ZS5wdXNoKHQpOmEuc2V0KG4sW3RdKX1mb3IoY29uc3RbbixlXW9mIGEpYS5zZXQobix0
KGUsbykpO3JldHVybiBuKGEpfSh0LDApfWZ1bmN0aW9uIFUodCxuKXtyZXR1cm4gQXJyYXkuZnJv
bShuLChuPT50W25dKSl9ZnVuY3Rpb24gSSh0LC4uLm4pe2lmKCJmdW5jdGlvbiIhPXR5cGVvZiB0
W1N5bWJvbC5pdGVyYXRvcl0pdGhyb3cgbmV3IFR5cGVFcnJvcigidmFsdWVzIGlzIG5vdCBpdGVy
YWJsZSIpO3Q9QXJyYXkuZnJvbSh0KTtsZXRbZV09bjtpZihlJiYyIT09ZS5sZW5ndGh8fG4ubGVu
Z3RoPjEpe2NvbnN0IHI9VWludDMyQXJyYXkuZnJvbSh0LCgodCxuKT0+bikpO3JldHVybiBuLmxl
bmd0aD4xPyhuPW4ubWFwKChuPT50Lm1hcChuKSkpLHIuc29ydCgoKHQsZSk9Pntmb3IoY29uc3Qg
ciBvZiBuKXtjb25zdCBuPUIoclt0XSxyW2VdKTtpZihuKXJldHVybiBufX0pKSk6KGU9dC5tYXAo
ZSksci5zb3J0KCgodCxuKT0+QihlW3RdLGVbbl0pKSkpLFUodCxyKX1yZXR1cm4gdC5zb3J0KE8o
ZSkpfWZ1bmN0aW9uIE8odD1uKXtpZih0PT09bilyZXR1cm4gQjtpZigiZnVuY3Rpb24iIT10eXBl
b2YgdCl0aHJvdyBuZXcgVHlwZUVycm9yKCJjb21wYXJlIGlzIG5vdCBhIGZ1bmN0aW9uIik7cmV0
dXJuKG4sZSk9Pntjb25zdCByPXQobixlKTtyZXR1cm4gcnx8MD09PXI/cjooMD09PXQoZSxlKSkt
KDA9PT10KG4sbikpfX1mdW5jdGlvbiBCKHQsbil7cmV0dXJuKG51bGw9PXR8fCEodD49dCkpLShu
dWxsPT1ufHwhKG4+PW4pKXx8KHQ8bj8tMTp0Pm4/MTowKX12YXIgWT1BcnJheS5wcm90b3R5cGUu
c2xpY2U7ZnVuY3Rpb24gTCh0KXtyZXR1cm4oKT0+dH1jb25zdCBqPU1hdGguc3FydCg1MCksSD1N
YXRoLnNxcnQoMTApLFg9TWF0aC5zcXJ0KDIpO2Z1bmN0aW9uIEcodCxuLGUpe2NvbnN0IHI9KG4t
dCkvTWF0aC5tYXgoMCxlKSxpPU1hdGguZmxvb3IoTWF0aC5sb2cxMChyKSksbz1yL01hdGgucG93
KDEwLGkpLGE9bz49aj8xMDpvPj1IPzU6bz49WD8yOjE7bGV0IHUsYyxmO3JldHVybiBpPDA/KGY9
TWF0aC5wb3coMTAsLWkpL2EsdT1NYXRoLnJvdW5kKHQqZiksYz1NYXRoLnJvdW5kKG4qZiksdS9m
PHQmJisrdSxjL2Y+biYmLS1jLGY9LWYpOihmPU1hdGgucG93KDEwLGkpKmEsdT1NYXRoLnJvdW5k
KHQvZiksYz1NYXRoLnJvdW5kKG4vZiksdSpmPHQmJisrdSxjKmY+biYmLS1jKSxjPHUmJi41PD1l
JiZlPDI/Ryh0LG4sMiplKTpbdSxjLGZdfWZ1bmN0aW9uIFYodCxuLGUpe2lmKCEoKGU9K2UpPjAp
KXJldHVybltdO2lmKCh0PSt0KT09PShuPStuKSlyZXR1cm5bdF07Y29uc3Qgcj1uPHQsW2ksbyxh
XT1yP0cobix0LGUpOkcodCxuLGUpO2lmKCEobz49aSkpcmV0dXJuW107Y29uc3QgdT1vLWkrMSxj
PW5ldyBBcnJheSh1KTtpZihyKWlmKGE8MClmb3IobGV0IHQ9MDt0PHU7Kyt0KWNbdF09KG8tdCkv
LWE7ZWxzZSBmb3IobGV0IHQ9MDt0PHU7Kyt0KWNbdF09KG8tdCkqYTtlbHNlIGlmKGE8MClmb3Io
bGV0IHQ9MDt0PHU7Kyt0KWNbdF09KGkrdCkvLWE7ZWxzZSBmb3IobGV0IHQ9MDt0PHU7Kyt0KWNb
dF09KGkrdCkqYTtyZXR1cm4gY31mdW5jdGlvbiBXKHQsbixlKXtyZXR1cm4gRyh0PSt0LG49K24s
ZT0rZSlbMl19ZnVuY3Rpb24gWih0LG4sZSl7ZT0rZTtjb25zdCByPShuPStuKTwodD0rdCksaT1y
P1cobix0LGUpOlcodCxuLGUpO3JldHVybihyPy0xOjEpKihpPDA/MS8taTppKX1mdW5jdGlvbiBL
KHQsbixlKXtsZXQgcjtmb3IoOzspe2NvbnN0IGk9Vyh0LG4sZSk7aWYoaT09PXJ8fDA9PT1pfHwh
aXNGaW5pdGUoaSkpcmV0dXJuW3Qsbl07aT4wPyh0PU1hdGguZmxvb3IodC9pKSppLG49TWF0aC5j
ZWlsKG4vaSkqaSk6aTwwJiYodD1NYXRoLmNlaWwodCppKS9pLG49TWF0aC5mbG9vcihuKmkpL2kp
LHI9aX19ZnVuY3Rpb24gUSh0KXtyZXR1cm4gTWF0aC5tYXgoMSxNYXRoLmNlaWwoTWF0aC5sb2co
Xyh0KSkvTWF0aC5MTjIpKzEpfWZ1bmN0aW9uIEooKXt2YXIgdD1DLG49VCxlPVE7ZnVuY3Rpb24g
cihyKXtBcnJheS5pc0FycmF5KHIpfHwocj1BcnJheS5mcm9tKHIpKTt2YXIgaSxvLGEsdT1yLmxl
bmd0aCxjPW5ldyBBcnJheSh1KTtmb3IoaT0wO2k8dTsrK2kpY1tpXT10KHJbaV0saSxyKTt2YXIg
Zj1uKGMpLHM9ZlswXSxoPWZbMV0sZD1lKGMscyxoKTtpZighQXJyYXkuaXNBcnJheShkKSl7Y29u
c3QgdD1oLGU9K2Q7aWYobj09PVQmJihbcyxoXT1LKHMsaCxlKSksKGQ9VihzLGgsZSkpWzBdPD1z
JiYoYT1XKHMsaCxlKSksZFtkLmxlbmd0aC0xXT49aClpZih0Pj1oJiZuPT09VCl7Y29uc3QgdD1X
KHMsaCxlKTtpc0Zpbml0ZSh0KSYmKHQ+MD9oPShNYXRoLmZsb29yKGgvdCkrMSkqdDp0PDAmJiho
PShNYXRoLmNlaWwoaCotdCkrMSkvLXQpKX1lbHNlIGQucG9wKCl9Zm9yKHZhciBwPWQubGVuZ3Ro
LGc9MCx5PXA7ZFtnXTw9czspKytnO2Zvcig7ZFt5LTFdPmg7KS0teTsoZ3x8eTxwKSYmKGQ9ZC5z
bGljZShnLHkpLHA9eS1nKTt2YXIgdixfPW5ldyBBcnJheShwKzEpO2ZvcihpPTA7aTw9cDsrK2kp
KHY9X1tpXT1bXSkueDA9aT4wP2RbaS0xXTpzLHYueDE9aTxwP2RbaV06aDtpZihpc0Zpbml0ZShh
KSl7aWYoYT4wKWZvcihpPTA7aTx1OysraSludWxsIT0obz1jW2ldKSYmczw9byYmbzw9aCYmX1tN
YXRoLm1pbihwLE1hdGguZmxvb3IoKG8tcykvYSkpXS5wdXNoKHJbaV0pO2Vsc2UgaWYoYTwwKWZv
cihpPTA7aTx1OysraSlpZihudWxsIT0obz1jW2ldKSYmczw9byYmbzw9aCl7Y29uc3QgdD1NYXRo
LmZsb29yKChzLW8pKmEpO19bTWF0aC5taW4ocCx0KyhkW3RdPD1vKSldLnB1c2gocltpXSl9fWVs
c2UgZm9yKGk9MDtpPHU7KytpKW51bGwhPShvPWNbaV0pJiZzPD1vJiZvPD1oJiZfW2woZCxvLDAs
cCldLnB1c2gocltpXSk7cmV0dXJuIF99cmV0dXJuIHIudmFsdWU9ZnVuY3Rpb24obil7cmV0dXJu
IGFyZ3VtZW50cy5sZW5ndGg/KHQ9ImZ1bmN0aW9uIj09dHlwZW9mIG4/bjpMKG4pLHIpOnR9LHIu
ZG9tYWluPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhuPSJmdW5jdGlvbiI9
PXR5cGVvZiB0P3Q6TChbdFswXSx0WzFdXSkscik6bn0sci50aHJlc2hvbGRzPWZ1bmN0aW9uKHQp
e3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6TChBcnJh
eS5pc0FycmF5KHQpP1kuY2FsbCh0KTp0KSxyKTplfSxyfWZ1bmN0aW9uIHR0KHQsbil7bGV0IGU7
aWYodm9pZCAwPT09bilmb3IoY29uc3QgbiBvZiB0KW51bGwhPW4mJihlPG58fHZvaWQgMD09PWUm
Jm4+PW4pJiYoZT1uKTtlbHNle2xldCByPS0xO2ZvcihsZXQgaSBvZiB0KW51bGwhPShpPW4oaSwr
K3IsdCkpJiYoZTxpfHx2b2lkIDA9PT1lJiZpPj1pKSYmKGU9aSl9cmV0dXJuIGV9ZnVuY3Rpb24g
bnQodCxuKXtsZXQgZSxyPS0xLGk9LTE7aWYodm9pZCAwPT09bilmb3IoY29uc3QgbiBvZiB0KSsr
aSxudWxsIT1uJiYoZTxufHx2b2lkIDA9PT1lJiZuPj1uKSYmKGU9bixyPWkpO2Vsc2UgZm9yKGxl
dCBvIG9mIHQpbnVsbCE9KG89bihvLCsraSx0KSkmJihlPG98fHZvaWQgMD09PWUmJm8+PW8pJiYo
ZT1vLHI9aSk7cmV0dXJuIHJ9ZnVuY3Rpb24gZXQodCxuKXtsZXQgZTtpZih2b2lkIDA9PT1uKWZv
cihjb25zdCBuIG9mIHQpbnVsbCE9biYmKGU+bnx8dm9pZCAwPT09ZSYmbj49bikmJihlPW4pO2Vs
c2V7bGV0IHI9LTE7Zm9yKGxldCBpIG9mIHQpbnVsbCE9KGk9bihpLCsrcix0KSkmJihlPml8fHZv
aWQgMD09PWUmJmk+PWkpJiYoZT1pKX1yZXR1cm4gZX1mdW5jdGlvbiBydCh0LG4pe2xldCBlLHI9
LTEsaT0tMTtpZih2b2lkIDA9PT1uKWZvcihjb25zdCBuIG9mIHQpKytpLG51bGwhPW4mJihlPm58
fHZvaWQgMD09PWUmJm4+PW4pJiYoZT1uLHI9aSk7ZWxzZSBmb3IobGV0IG8gb2YgdCludWxsIT0o
bz1uKG8sKytpLHQpKSYmKGU+b3x8dm9pZCAwPT09ZSYmbz49bykmJihlPW8scj1pKTtyZXR1cm4g
cn1mdW5jdGlvbiBpdCh0LG4sZT0wLHI9MS8wLGkpe2lmKG49TWF0aC5mbG9vcihuKSxlPU1hdGgu
Zmxvb3IoTWF0aC5tYXgoMCxlKSkscj1NYXRoLmZsb29yKE1hdGgubWluKHQubGVuZ3RoLTEscikp
LCEoZTw9biYmbjw9cikpcmV0dXJuIHQ7Zm9yKGk9dm9pZCAwPT09aT9COk8oaSk7cj5lOyl7aWYo
ci1lPjYwMCl7Y29uc3Qgbz1yLWUrMSxhPW4tZSsxLHU9TWF0aC5sb2cobyksYz0uNSpNYXRoLmV4
cCgyKnUvMyksZj0uNSpNYXRoLnNxcnQodSpjKihvLWMpL28pKihhLW8vMjwwPy0xOjEpO2l0KHQs
bixNYXRoLm1heChlLE1hdGguZmxvb3Iobi1hKmMvbytmKSksTWF0aC5taW4ocixNYXRoLmZsb29y
KG4rKG8tYSkqYy9vK2YpKSxpKX1jb25zdCBvPXRbbl07bGV0IGE9ZSx1PXI7Zm9yKG90KHQsZSxu
KSxpKHRbcl0sbyk+MCYmb3QodCxlLHIpO2E8dTspe2ZvcihvdCh0LGEsdSksKythLC0tdTtpKHRb
YV0sbyk8MDspKythO2Zvcig7aSh0W3VdLG8pPjA7KS0tdX0wPT09aSh0W2VdLG8pP290KHQsZSx1
KTooKyt1LG90KHQsdSxyKSksdTw9biYmKGU9dSsxKSxuPD11JiYocj11LTEpfXJldHVybiB0fWZ1
bmN0aW9uIG90KHQsbixlKXtjb25zdCByPXRbbl07dFtuXT10W2VdLHRbZV09cn1mdW5jdGlvbiBh
dCh0LGU9bil7bGV0IHIsaT0hMTtpZigxPT09ZS5sZW5ndGgpe2xldCBvO2Zvcihjb25zdCBhIG9m
IHQpe2NvbnN0IHQ9ZShhKTsoaT9uKHQsbyk+MDowPT09bih0LHQpKSYmKHI9YSxvPXQsaT0hMCl9
fWVsc2UgZm9yKGNvbnN0IG4gb2YgdCkoaT9lKG4scik+MDowPT09ZShuLG4pKSYmKHI9bixpPSEw
KTtyZXR1cm4gcn1mdW5jdGlvbiB1dCh0LG4sZSl7aWYoKHI9KHQ9RmxvYXQ2NEFycmF5LmZyb20o
YSh0LGUpKSkubGVuZ3RoKSYmIWlzTmFOKG49K24pKXtpZihuPD0wfHxyPDIpcmV0dXJuIGV0KHQp
O2lmKG4+PTEpcmV0dXJuIHR0KHQpO3ZhciByLGk9KHItMSkqbixvPU1hdGguZmxvb3IoaSksdT10
dChpdCh0LG8pLnN1YmFycmF5KDAsbysxKSk7cmV0dXJuIHUrKGV0KHQuc3ViYXJyYXkobysxKSkt
dSkqKGktbyl9fWZ1bmN0aW9uIGN0KHQsbixlPW8pe2lmKChyPXQubGVuZ3RoKSYmIWlzTmFOKG49
K24pKXtpZihuPD0wfHxyPDIpcmV0dXJuK2UodFswXSwwLHQpO2lmKG4+PTEpcmV0dXJuK2UodFty
LTFdLHItMSx0KTt2YXIgcixpPShyLTEpKm4sYT1NYXRoLmZsb29yKGkpLHU9K2UodFthXSxhLHQp
O3JldHVybiB1KygrZSh0W2ErMV0sYSsxLHQpLXUpKihpLWEpfX1mdW5jdGlvbiBmdCh0LG4sZSl7
aWYoKHI9KHQ9RmxvYXQ2NEFycmF5LmZyb20oYSh0LGUpKSkubGVuZ3RoKSYmIWlzTmFOKG49K24p
KXtpZihuPD0wfHxyPDIpcmV0dXJuIHJ0KHQpO2lmKG4+PTEpcmV0dXJuIG50KHQpO3ZhciByLGk9
TWF0aC5mbG9vcigoci0xKSpuKSxvPWl0KFVpbnQzMkFycmF5LmZyb20odCwoKHQsbik9Pm4pKSxp
LDAsci0xLCgobixlKT0+Qih0W25dLHRbZV0pKSk7cmV0dXJuIGF0KG8uc3ViYXJyYXkoMCxpKzEp
LChuPT50W25dKSl9fWZ1bmN0aW9uIHN0KHQpe3JldHVybiBBcnJheS5mcm9tKGZ1bmN0aW9uKih0
KXtmb3IoY29uc3QgbiBvZiB0KXlpZWxkKm59KHQpKX1mdW5jdGlvbiBsdCh0LG4pe3JldHVyblt0
LG5dfWZ1bmN0aW9uIGh0KHQsbixlKXt0PSt0LG49K24sZT0oaT1hcmd1bWVudHMubGVuZ3RoKTwy
PyhuPXQsdD0wLDEpOmk8Mz8xOitlO2Zvcih2YXIgcj0tMSxpPTB8TWF0aC5tYXgoMCxNYXRoLmNl
aWwoKG4tdCkvZSkpLG89bmV3IEFycmF5KGkpOysrcjxpOylvW3JdPXQrciplO3JldHVybiBvfWZ1
bmN0aW9uIGR0KHQsZT1uKXtpZigxPT09ZS5sZW5ndGgpcmV0dXJuIHJ0KHQsZSk7bGV0IHIsaT0t
MSxvPS0xO2Zvcihjb25zdCBuIG9mIHQpKytvLChpPDA/MD09PWUobixuKTplKG4scik8MCkmJihy
PW4saT1vKTtyZXR1cm4gaX12YXIgcHQ9Z3QoTWF0aC5yYW5kb20pO2Z1bmN0aW9uIGd0KHQpe3Jl
dHVybiBmdW5jdGlvbihuLGU9MCxyPW4ubGVuZ3RoKXtsZXQgaT1yLShlPStlKTtmb3IoO2k7KXtj
b25zdCByPXQoKSppLS18MCxvPW5baStlXTtuW2krZV09bltyK2VdLG5bcitlXT1vfXJldHVybiBu
fX1mdW5jdGlvbiB5dCh0KXtpZighKGk9dC5sZW5ndGgpKXJldHVybltdO2Zvcih2YXIgbj0tMSxl
PWV0KHQsdnQpLHI9bmV3IEFycmF5KGUpOysrbjxlOylmb3IodmFyIGksbz0tMSxhPXJbbl09bmV3
IEFycmF5KGkpOysrbzxpOylhW29dPXRbb11bbl07cmV0dXJuIHJ9ZnVuY3Rpb24gdnQodCl7cmV0
dXJuIHQubGVuZ3RofWZ1bmN0aW9uIF90KHQpe3JldHVybiB0IGluc3RhbmNlb2YgSW50ZXJuU2V0
P3Q6bmV3IEludGVyblNldCh0KX1mdW5jdGlvbiBidCh0LG4pe2NvbnN0IGU9dFtTeW1ib2wuaXRl
cmF0b3JdKCkscj1uZXcgU2V0O2Zvcihjb25zdCB0IG9mIG4pe2NvbnN0IG49bXQodCk7aWYoci5o
YXMobikpY29udGludWU7bGV0IGksbztmb3IoOyh7dmFsdWU6aSxkb25lOm99PWUubmV4dCgpKTsp
e2lmKG8pcmV0dXJuITE7Y29uc3QgdD1tdChpKTtpZihyLmFkZCh0KSxPYmplY3QuaXMobix0KSli
cmVha319cmV0dXJuITB9ZnVuY3Rpb24gbXQodCl7cmV0dXJuIG51bGwhPT10JiYib2JqZWN0Ij09
dHlwZW9mIHQ/dC52YWx1ZU9mKCk6dH1mdW5jdGlvbiB4dCh0KXtyZXR1cm4gdH12YXIgd3Q9MSxN
dD0yLFR0PTMsQXQ9NCxTdD0xZS02O2Z1bmN0aW9uIEV0KHQpe3JldHVybiJ0cmFuc2xhdGUoIit0
KyIsMCkifWZ1bmN0aW9uIE50KHQpe3JldHVybiJ0cmFuc2xhdGUoMCwiK3QrIikifWZ1bmN0aW9u
IGt0KHQpe3JldHVybiBuPT4rdChuKX1mdW5jdGlvbiBDdCh0LG4pe3JldHVybiBuPU1hdGgubWF4
KDAsdC5iYW5kd2lkdGgoKS0yKm4pLzIsdC5yb3VuZCgpJiYobj1NYXRoLnJvdW5kKG4pKSxlPT4r
dChlKStufWZ1bmN0aW9uIFB0KCl7cmV0dXJuIXRoaXMuX19heGlzfWZ1bmN0aW9uIHp0KHQsbil7
dmFyIGU9W10scj1udWxsLGk9bnVsbCxvPTYsYT02LHU9MyxjPSJ1bmRlZmluZWQiIT10eXBlb2Yg
d2luZG93JiZ3aW5kb3cuZGV2aWNlUGl4ZWxSYXRpbz4xPzA6LjUsZj10PT09d3R8fHQ9PT1BdD8t
MToxLHM9dD09PUF0fHx0PT09TXQ/IngiOiJ5IixsPXQ9PT13dHx8dD09PVR0P0V0Ok50O2Z1bmN0
aW9uIGgoaCl7dmFyIGQ9bnVsbD09cj9uLnRpY2tzP24udGlja3MuYXBwbHkobixlKTpuLmRvbWFp
bigpOnIscD1udWxsPT1pP24udGlja0Zvcm1hdD9uLnRpY2tGb3JtYXQuYXBwbHkobixlKTp4dDpp
LGc9TWF0aC5tYXgobywwKSt1LHk9bi5yYW5nZSgpLHY9K3lbMF0rYyxfPSt5W3kubGVuZ3RoLTFd
K2MsYj0obi5iYW5kd2lkdGg/Q3Q6a3QpKG4uY29weSgpLGMpLG09aC5zZWxlY3Rpb24/aC5zZWxl
Y3Rpb24oKTpoLHg9bS5zZWxlY3RBbGwoIi5kb21haW4iKS5kYXRhKFtudWxsXSksdz1tLnNlbGVj
dEFsbCgiLnRpY2siKS5kYXRhKGQsbikub3JkZXIoKSxNPXcuZXhpdCgpLFQ9dy5lbnRlcigpLmFw
cGVuZCgiZyIpLmF0dHIoImNsYXNzIiwidGljayIpLEE9dy5zZWxlY3QoImxpbmUiKSxTPXcuc2Vs
ZWN0KCJ0ZXh0Iik7eD14Lm1lcmdlKHguZW50ZXIoKS5pbnNlcnQoInBhdGgiLCIudGljayIpLmF0
dHIoImNsYXNzIiwiZG9tYWluIikuYXR0cigic3Ryb2tlIiwiY3VycmVudENvbG9yIikpLHc9dy5t
ZXJnZShUKSxBPUEubWVyZ2UoVC5hcHBlbmQoImxpbmUiKS5hdHRyKCJzdHJva2UiLCJjdXJyZW50
Q29sb3IiKS5hdHRyKHMrIjIiLGYqbykpLFM9Uy5tZXJnZShULmFwcGVuZCgidGV4dCIpLmF0dHIo
ImZpbGwiLCJjdXJyZW50Q29sb3IiKS5hdHRyKHMsZipnKS5hdHRyKCJkeSIsdD09PXd0PyIwZW0i
OnQ9PT1UdD8iMC43MWVtIjoiMC4zMmVtIikpLGghPT1tJiYoeD14LnRyYW5zaXRpb24oaCksdz13
LnRyYW5zaXRpb24oaCksQT1BLnRyYW5zaXRpb24oaCksUz1TLnRyYW5zaXRpb24oaCksTT1NLnRy
YW5zaXRpb24oaCkuYXR0cigib3BhY2l0eSIsU3QpLmF0dHIoInRyYW5zZm9ybSIsKGZ1bmN0aW9u
KHQpe3JldHVybiBpc0Zpbml0ZSh0PWIodCkpP2wodCtjKTp0aGlzLmdldEF0dHJpYnV0ZSgidHJh
bnNmb3JtIil9KSksVC5hdHRyKCJvcGFjaXR5IixTdCkuYXR0cigidHJhbnNmb3JtIiwoZnVuY3Rp
b24odCl7dmFyIG49dGhpcy5wYXJlbnROb2RlLl9fYXhpcztyZXR1cm4gbCgobiYmaXNGaW5pdGUo
bj1uKHQpKT9uOmIodCkpK2MpfSkpKSxNLnJlbW92ZSgpLHguYXR0cigiZCIsdD09PUF0fHx0PT09
TXQ/YT8iTSIrZiphKyIsIit2KyJIIitjKyJWIitfKyJIIitmKmE6Ik0iK2MrIiwiK3YrIlYiK186
YT8iTSIrdisiLCIrZiphKyJWIitjKyJIIitfKyJWIitmKmE6Ik0iK3YrIiwiK2MrIkgiK18pLHcu
YXR0cigib3BhY2l0eSIsMSkuYXR0cigidHJhbnNmb3JtIiwoZnVuY3Rpb24odCl7cmV0dXJuIGwo
Yih0KStjKX0pKSxBLmF0dHIocysiMiIsZipvKSxTLmF0dHIocyxmKmcpLnRleHQocCksbS5maWx0
ZXIoUHQpLmF0dHIoImZpbGwiLCJub25lIikuYXR0cigiZm9udC1zaXplIiwxMCkuYXR0cigiZm9u
dC1mYW1pbHkiLCJzYW5zLXNlcmlmIikuYXR0cigidGV4dC1hbmNob3IiLHQ9PT1NdD8ic3RhcnQi
OnQ9PT1BdD8iZW5kIjoibWlkZGxlIiksbS5lYWNoKChmdW5jdGlvbigpe3RoaXMuX19heGlzPWJ9
KSl9cmV0dXJuIGguc2NhbGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49
dCxoKTpufSxoLnRpY2tzPWZ1bmN0aW9uKCl7cmV0dXJuIGU9QXJyYXkuZnJvbShhcmd1bWVudHMp
LGh9LGgudGlja0FyZ3VtZW50cz1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8o
ZT1udWxsPT10P1tdOkFycmF5LmZyb20odCksaCk6ZS5zbGljZSgpfSxoLnRpY2tWYWx1ZXM9ZnVu
Y3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9bnVsbD09dD9udWxsOkFycmF5LmZy
b20odCksaCk6ciYmci5zbGljZSgpfSxoLnRpY2tGb3JtYXQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFy
Z3VtZW50cy5sZW5ndGg/KGk9dCxoKTppfSxoLnRpY2tTaXplPWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyhvPWE9K3QsaCk6b30saC50aWNrU2l6ZUlubmVyPWZ1bmN0aW9uKHQp
e3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhvPSt0LGgpOm99LGgudGlja1NpemVPdXRlcj1mdW5j
dGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYT0rdCxoKTphfSxoLnRpY2tQYWRkaW5n
PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh1PSt0LGgpOnV9LGgub2Zmc2V0
PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhjPSt0LGgpOmN9LGh9dmFyICR0
PXt2YWx1ZTooKT0+e319O2Z1bmN0aW9uIER0KCl7Zm9yKHZhciB0LG49MCxlPWFyZ3VtZW50cy5s
ZW5ndGgscj17fTtuPGU7KytuKXtpZighKHQ9YXJndW1lbnRzW25dKyIiKXx8dCBpbiByfHwvW1xz
Ll0vLnRlc3QodCkpdGhyb3cgbmV3IEVycm9yKCJpbGxlZ2FsIHR5cGU6ICIrdCk7clt0XT1bXX1y
ZXR1cm4gbmV3IFJ0KHIpfWZ1bmN0aW9uIFJ0KHQpe3RoaXMuXz10fWZ1bmN0aW9uIEZ0KHQsbil7
Zm9yKHZhciBlLHI9MCxpPXQubGVuZ3RoO3I8aTsrK3IpaWYoKGU9dFtyXSkubmFtZT09PW4pcmV0
dXJuIGUudmFsdWV9ZnVuY3Rpb24gcXQodCxuLGUpe2Zvcih2YXIgcj0wLGk9dC5sZW5ndGg7cjxp
OysrcilpZih0W3JdLm5hbWU9PT1uKXt0W3JdPSR0LHQ9dC5zbGljZSgwLHIpLmNvbmNhdCh0LnNs
aWNlKHIrMSkpO2JyZWFrfXJldHVybiBudWxsIT1lJiZ0LnB1c2goe25hbWU6bix2YWx1ZTplfSks
dH1SdC5wcm90b3R5cGU9RHQucHJvdG90eXBlPXtjb25zdHJ1Y3RvcjpSdCxvbjpmdW5jdGlvbih0
LG4pe3ZhciBlLHIsaT10aGlzLl8sbz0ocj1pLCh0KyIiKS50cmltKCkuc3BsaXQoL158XHMrLyku
bWFwKChmdW5jdGlvbih0KXt2YXIgbj0iIixlPXQuaW5kZXhPZigiLiIpO2lmKGU+PTAmJihuPXQu
c2xpY2UoZSsxKSx0PXQuc2xpY2UoMCxlKSksdCYmIXIuaGFzT3duUHJvcGVydHkodCkpdGhyb3cg
bmV3IEVycm9yKCJ1bmtub3duIHR5cGU6ICIrdCk7cmV0dXJue3R5cGU6dCxuYW1lOm59fSkpKSxh
PS0xLHU9by5sZW5ndGg7aWYoIShhcmd1bWVudHMubGVuZ3RoPDIpKXtpZihudWxsIT1uJiYiZnVu
Y3Rpb24iIT10eXBlb2Ygbil0aHJvdyBuZXcgRXJyb3IoImludmFsaWQgY2FsbGJhY2s6ICIrbik7
Zm9yKDsrK2E8dTspaWYoZT0odD1vW2FdKS50eXBlKWlbZV09cXQoaVtlXSx0Lm5hbWUsbik7ZWxz
ZSBpZihudWxsPT1uKWZvcihlIGluIGkpaVtlXT1xdChpW2VdLHQubmFtZSxudWxsKTtyZXR1cm4g
dGhpc31mb3IoOysrYTx1OylpZigoZT0odD1vW2FdKS50eXBlKSYmKGU9RnQoaVtlXSx0Lm5hbWUp
KSlyZXR1cm4gZX0sY29weTpmdW5jdGlvbigpe3ZhciB0PXt9LG49dGhpcy5fO2Zvcih2YXIgZSBp
biBuKXRbZV09bltlXS5zbGljZSgpO3JldHVybiBuZXcgUnQodCl9LGNhbGw6ZnVuY3Rpb24odCxu
KXtpZigoZT1hcmd1bWVudHMubGVuZ3RoLTIpPjApZm9yKHZhciBlLHIsaT1uZXcgQXJyYXkoZSks
bz0wO288ZTsrK28paVtvXT1hcmd1bWVudHNbbysyXTtpZighdGhpcy5fLmhhc093blByb3BlcnR5
KHQpKXRocm93IG5ldyBFcnJvcigidW5rbm93biB0eXBlOiAiK3QpO2ZvcihvPTAsZT0ocj10aGlz
Ll9bdF0pLmxlbmd0aDtvPGU7KytvKXJbb10udmFsdWUuYXBwbHkobixpKX0sYXBwbHk6ZnVuY3Rp
b24odCxuLGUpe2lmKCF0aGlzLl8uaGFzT3duUHJvcGVydHkodCkpdGhyb3cgbmV3IEVycm9yKCJ1
bmtub3duIHR5cGU6ICIrdCk7Zm9yKHZhciByPXRoaXMuX1t0XSxpPTAsbz1yLmxlbmd0aDtpPG87
KytpKXJbaV0udmFsdWUuYXBwbHkobixlKX19O3ZhciBVdD0iaHR0cDovL3d3dy53My5vcmcvMTk5
OS94aHRtbCIsSXQ9e3N2ZzoiaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciLHhodG1sOlV0LHhs
aW5rOiJodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hsaW5rIix4bWw6Imh0dHA6Ly93d3cudzMub3Jn
L1hNTC8xOTk4L25hbWVzcGFjZSIseG1sbnM6Imh0dHA6Ly93d3cudzMub3JnLzIwMDAveG1sbnMv
In07ZnVuY3Rpb24gT3QodCl7dmFyIG49dCs9IiIsZT1uLmluZGV4T2YoIjoiKTtyZXR1cm4gZT49
MCYmInhtbG5zIiE9PShuPXQuc2xpY2UoMCxlKSkmJih0PXQuc2xpY2UoZSsxKSksSXQuaGFzT3du
UHJvcGVydHkobik/e3NwYWNlOkl0W25dLGxvY2FsOnR9OnR9ZnVuY3Rpb24gQnQodCl7cmV0dXJu
IGZ1bmN0aW9uKCl7dmFyIG49dGhpcy5vd25lckRvY3VtZW50LGU9dGhpcy5uYW1lc3BhY2VVUkk7
cmV0dXJuIGU9PT1VdCYmbi5kb2N1bWVudEVsZW1lbnQubmFtZXNwYWNlVVJJPT09VXQ/bi5jcmVh
dGVFbGVtZW50KHQpOm4uY3JlYXRlRWxlbWVudE5TKGUsdCl9fWZ1bmN0aW9uIFl0KHQpe3JldHVy
biBmdW5jdGlvbigpe3JldHVybiB0aGlzLm93bmVyRG9jdW1lbnQuY3JlYXRlRWxlbWVudE5TKHQu
c3BhY2UsdC5sb2NhbCl9fWZ1bmN0aW9uIEx0KHQpe3ZhciBuPU90KHQpO3JldHVybihuLmxvY2Fs
P1l0OkJ0KShuKX1mdW5jdGlvbiBqdCgpe31mdW5jdGlvbiBIdCh0KXtyZXR1cm4gbnVsbD09dD9q
dDpmdW5jdGlvbigpe3JldHVybiB0aGlzLnF1ZXJ5U2VsZWN0b3IodCl9fWZ1bmN0aW9uIFh0KHQp
e3JldHVybiBudWxsPT10P1tdOkFycmF5LmlzQXJyYXkodCk/dDpBcnJheS5mcm9tKHQpfWZ1bmN0
aW9uIEd0KCl7cmV0dXJuW119ZnVuY3Rpb24gVnQodCl7cmV0dXJuIG51bGw9PXQ/R3Q6ZnVuY3Rp
b24oKXtyZXR1cm4gdGhpcy5xdWVyeVNlbGVjdG9yQWxsKHQpfX1mdW5jdGlvbiBXdCh0KXtyZXR1
cm4gZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5tYXRjaGVzKHQpfX1mdW5jdGlvbiBadCh0KXtyZXR1
cm4gZnVuY3Rpb24obil7cmV0dXJuIG4ubWF0Y2hlcyh0KX19dmFyIEt0PUFycmF5LnByb3RvdHlw
ZS5maW5kO2Z1bmN0aW9uIFF0KCl7cmV0dXJuIHRoaXMuZmlyc3RFbGVtZW50Q2hpbGR9dmFyIEp0
PUFycmF5LnByb3RvdHlwZS5maWx0ZXI7ZnVuY3Rpb24gdG4oKXtyZXR1cm4gQXJyYXkuZnJvbSh0
aGlzLmNoaWxkcmVuKX1mdW5jdGlvbiBubih0KXtyZXR1cm4gbmV3IEFycmF5KHQubGVuZ3RoKX1m
dW5jdGlvbiBlbih0LG4pe3RoaXMub3duZXJEb2N1bWVudD10Lm93bmVyRG9jdW1lbnQsdGhpcy5u
YW1lc3BhY2VVUkk9dC5uYW1lc3BhY2VVUkksdGhpcy5fbmV4dD1udWxsLHRoaXMuX3BhcmVudD10
LHRoaXMuX19kYXRhX189bn1mdW5jdGlvbiBybih0LG4sZSxyLGksbyl7Zm9yKHZhciBhLHU9MCxj
PW4ubGVuZ3RoLGY9by5sZW5ndGg7dTxmOysrdSkoYT1uW3VdKT8oYS5fX2RhdGFfXz1vW3VdLHJb
dV09YSk6ZVt1XT1uZXcgZW4odCxvW3VdKTtmb3IoO3U8YzsrK3UpKGE9blt1XSkmJihpW3VdPWEp
fWZ1bmN0aW9uIG9uKHQsbixlLHIsaSxvLGEpe3ZhciB1LGMsZixzPW5ldyBNYXAsbD1uLmxlbmd0
aCxoPW8ubGVuZ3RoLGQ9bmV3IEFycmF5KGwpO2Zvcih1PTA7dTxsOysrdSkoYz1uW3VdKSYmKGRb
dV09Zj1hLmNhbGwoYyxjLl9fZGF0YV9fLHUsbikrIiIscy5oYXMoZik/aVt1XT1jOnMuc2V0KGYs
YykpO2Zvcih1PTA7dTxoOysrdSlmPWEuY2FsbCh0LG9bdV0sdSxvKSsiIiwoYz1zLmdldChmKSk/
KHJbdV09YyxjLl9fZGF0YV9fPW9bdV0scy5kZWxldGUoZikpOmVbdV09bmV3IGVuKHQsb1t1XSk7
Zm9yKHU9MDt1PGw7Kyt1KShjPW5bdV0pJiZzLmdldChkW3VdKT09PWMmJihpW3VdPWMpfWZ1bmN0
aW9uIGFuKHQpe3JldHVybiB0Ll9fZGF0YV9ffWZ1bmN0aW9uIHVuKHQpe3JldHVybiJvYmplY3Qi
PT10eXBlb2YgdCYmImxlbmd0aCJpbiB0P3Q6QXJyYXkuZnJvbSh0KX1mdW5jdGlvbiBjbih0LG4p
e3JldHVybiB0PG4/LTE6dD5uPzE6dD49bj8wOk5hTn1mdW5jdGlvbiBmbih0KXtyZXR1cm4gZnVu
Y3Rpb24oKXt0aGlzLnJlbW92ZUF0dHJpYnV0ZSh0KX19ZnVuY3Rpb24gc24odCl7cmV0dXJuIGZ1
bmN0aW9uKCl7dGhpcy5yZW1vdmVBdHRyaWJ1dGVOUyh0LnNwYWNlLHQubG9jYWwpfX1mdW5jdGlv
biBsbih0LG4pe3JldHVybiBmdW5jdGlvbigpe3RoaXMuc2V0QXR0cmlidXRlKHQsbil9fWZ1bmN0
aW9uIGhuKHQsbil7cmV0dXJuIGZ1bmN0aW9uKCl7dGhpcy5zZXRBdHRyaWJ1dGVOUyh0LnNwYWNl
LHQubG9jYWwsbil9fWZ1bmN0aW9uIGRuKHQsbil7cmV0dXJuIGZ1bmN0aW9uKCl7dmFyIGU9bi5h
cHBseSh0aGlzLGFyZ3VtZW50cyk7bnVsbD09ZT90aGlzLnJlbW92ZUF0dHJpYnV0ZSh0KTp0aGlz
LnNldEF0dHJpYnV0ZSh0LGUpfX1mdW5jdGlvbiBwbih0LG4pe3JldHVybiBmdW5jdGlvbigpe3Zh
ciBlPW4uYXBwbHkodGhpcyxhcmd1bWVudHMpO251bGw9PWU/dGhpcy5yZW1vdmVBdHRyaWJ1dGVO
Uyh0LnNwYWNlLHQubG9jYWwpOnRoaXMuc2V0QXR0cmlidXRlTlModC5zcGFjZSx0LmxvY2FsLGUp
fX1mdW5jdGlvbiBnbih0KXtyZXR1cm4gdC5vd25lckRvY3VtZW50JiZ0Lm93bmVyRG9jdW1lbnQu
ZGVmYXVsdFZpZXd8fHQuZG9jdW1lbnQmJnR8fHQuZGVmYXVsdFZpZXd9ZnVuY3Rpb24geW4odCl7
cmV0dXJuIGZ1bmN0aW9uKCl7dGhpcy5zdHlsZS5yZW1vdmVQcm9wZXJ0eSh0KX19ZnVuY3Rpb24g
dm4odCxuLGUpe3JldHVybiBmdW5jdGlvbigpe3RoaXMuc3R5bGUuc2V0UHJvcGVydHkodCxuLGUp
fX1mdW5jdGlvbiBfbih0LG4sZSl7cmV0dXJuIGZ1bmN0aW9uKCl7dmFyIHI9bi5hcHBseSh0aGlz
LGFyZ3VtZW50cyk7bnVsbD09cj90aGlzLnN0eWxlLnJlbW92ZVByb3BlcnR5KHQpOnRoaXMuc3R5
bGUuc2V0UHJvcGVydHkodCxyLGUpfX1mdW5jdGlvbiBibih0LG4pe3JldHVybiB0LnN0eWxlLmdl
dFByb3BlcnR5VmFsdWUobil8fGduKHQpLmdldENvbXB1dGVkU3R5bGUodCxudWxsKS5nZXRQcm9w
ZXJ0eVZhbHVlKG4pfWZ1bmN0aW9uIG1uKHQpe3JldHVybiBmdW5jdGlvbigpe2RlbGV0ZSB0aGlz
W3RdfX1mdW5jdGlvbiB4bih0LG4pe3JldHVybiBmdW5jdGlvbigpe3RoaXNbdF09bn19ZnVuY3Rp
b24gd24odCxuKXtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgZT1uLmFwcGx5KHRoaXMsYXJndW1lbnRz
KTtudWxsPT1lP2RlbGV0ZSB0aGlzW3RdOnRoaXNbdF09ZX19ZnVuY3Rpb24gTW4odCl7cmV0dXJu
IHQudHJpbSgpLnNwbGl0KC9efFxzKy8pfWZ1bmN0aW9uIFRuKHQpe3JldHVybiB0LmNsYXNzTGlz
dHx8bmV3IEFuKHQpfWZ1bmN0aW9uIEFuKHQpe3RoaXMuX25vZGU9dCx0aGlzLl9uYW1lcz1Nbih0
LmdldEF0dHJpYnV0ZSgiY2xhc3MiKXx8IiIpfWZ1bmN0aW9uIFNuKHQsbil7Zm9yKHZhciBlPVRu
KHQpLHI9LTEsaT1uLmxlbmd0aDsrK3I8aTspZS5hZGQobltyXSl9ZnVuY3Rpb24gRW4odCxuKXtm
b3IodmFyIGU9VG4odCkscj0tMSxpPW4ubGVuZ3RoOysrcjxpOyllLnJlbW92ZShuW3JdKX1mdW5j
dGlvbiBObih0KXtyZXR1cm4gZnVuY3Rpb24oKXtTbih0aGlzLHQpfX1mdW5jdGlvbiBrbih0KXty
ZXR1cm4gZnVuY3Rpb24oKXtFbih0aGlzLHQpfX1mdW5jdGlvbiBDbih0LG4pe3JldHVybiBmdW5j
dGlvbigpeyhuLmFwcGx5KHRoaXMsYXJndW1lbnRzKT9TbjpFbikodGhpcyx0KX19ZnVuY3Rpb24g
UG4oKXt0aGlzLnRleHRDb250ZW50PSIifWZ1bmN0aW9uIHpuKHQpe3JldHVybiBmdW5jdGlvbigp
e3RoaXMudGV4dENvbnRlbnQ9dH19ZnVuY3Rpb24gJG4odCl7cmV0dXJuIGZ1bmN0aW9uKCl7dmFy
IG49dC5hcHBseSh0aGlzLGFyZ3VtZW50cyk7dGhpcy50ZXh0Q29udGVudD1udWxsPT1uPyIiOm59
fWZ1bmN0aW9uIERuKCl7dGhpcy5pbm5lckhUTUw9IiJ9ZnVuY3Rpb24gUm4odCl7cmV0dXJuIGZ1
bmN0aW9uKCl7dGhpcy5pbm5lckhUTUw9dH19ZnVuY3Rpb24gRm4odCl7cmV0dXJuIGZ1bmN0aW9u
KCl7dmFyIG49dC5hcHBseSh0aGlzLGFyZ3VtZW50cyk7dGhpcy5pbm5lckhUTUw9bnVsbD09bj8i
IjpufX1mdW5jdGlvbiBxbigpe3RoaXMubmV4dFNpYmxpbmcmJnRoaXMucGFyZW50Tm9kZS5hcHBl
bmRDaGlsZCh0aGlzKX1mdW5jdGlvbiBVbigpe3RoaXMucHJldmlvdXNTaWJsaW5nJiZ0aGlzLnBh
cmVudE5vZGUuaW5zZXJ0QmVmb3JlKHRoaXMsdGhpcy5wYXJlbnROb2RlLmZpcnN0Q2hpbGQpfWZ1
bmN0aW9uIEluKCl7cmV0dXJuIG51bGx9ZnVuY3Rpb24gT24oKXt2YXIgdD10aGlzLnBhcmVudE5v
ZGU7dCYmdC5yZW1vdmVDaGlsZCh0aGlzKX1mdW5jdGlvbiBCbigpe3ZhciB0PXRoaXMuY2xvbmVO
b2RlKCExKSxuPXRoaXMucGFyZW50Tm9kZTtyZXR1cm4gbj9uLmluc2VydEJlZm9yZSh0LHRoaXMu
bmV4dFNpYmxpbmcpOnR9ZnVuY3Rpb24gWW4oKXt2YXIgdD10aGlzLmNsb25lTm9kZSghMCksbj10
aGlzLnBhcmVudE5vZGU7cmV0dXJuIG4/bi5pbnNlcnRCZWZvcmUodCx0aGlzLm5leHRTaWJsaW5n
KTp0fWZ1bmN0aW9uIExuKHQpe3JldHVybiBmdW5jdGlvbigpe3ZhciBuPXRoaXMuX19vbjtpZihu
KXtmb3IodmFyIGUscj0wLGk9LTEsbz1uLmxlbmd0aDtyPG87KytyKWU9bltyXSx0LnR5cGUmJmUu
dHlwZSE9PXQudHlwZXx8ZS5uYW1lIT09dC5uYW1lP25bKytpXT1lOnRoaXMucmVtb3ZlRXZlbnRM
aXN0ZW5lcihlLnR5cGUsZS5saXN0ZW5lcixlLm9wdGlvbnMpOysraT9uLmxlbmd0aD1pOmRlbGV0
ZSB0aGlzLl9fb259fX1mdW5jdGlvbiBqbih0LG4sZSl7cmV0dXJuIGZ1bmN0aW9uKCl7dmFyIHIs
aT10aGlzLl9fb24sbz1mdW5jdGlvbih0KXtyZXR1cm4gZnVuY3Rpb24obil7dC5jYWxsKHRoaXMs
bix0aGlzLl9fZGF0YV9fKX19KG4pO2lmKGkpZm9yKHZhciBhPTAsdT1pLmxlbmd0aDthPHU7Kyth
KWlmKChyPWlbYV0pLnR5cGU9PT10LnR5cGUmJnIubmFtZT09PXQubmFtZSlyZXR1cm4gdGhpcy5y
ZW1vdmVFdmVudExpc3RlbmVyKHIudHlwZSxyLmxpc3RlbmVyLHIub3B0aW9ucyksdGhpcy5hZGRF
dmVudExpc3RlbmVyKHIudHlwZSxyLmxpc3RlbmVyPW8sci5vcHRpb25zPWUpLHZvaWQoci52YWx1
ZT1uKTt0aGlzLmFkZEV2ZW50TGlzdGVuZXIodC50eXBlLG8sZSkscj17dHlwZTp0LnR5cGUsbmFt
ZTp0Lm5hbWUsdmFsdWU6bixsaXN0ZW5lcjpvLG9wdGlvbnM6ZX0saT9pLnB1c2gocik6dGhpcy5f
X29uPVtyXX19ZnVuY3Rpb24gSG4odCxuLGUpe3ZhciByPWduKHQpLGk9ci5DdXN0b21FdmVudDsi
ZnVuY3Rpb24iPT10eXBlb2YgaT9pPW5ldyBpKG4sZSk6KGk9ci5kb2N1bWVudC5jcmVhdGVFdmVu
dCgiRXZlbnQiKSxlPyhpLmluaXRFdmVudChuLGUuYnViYmxlcyxlLmNhbmNlbGFibGUpLGkuZGV0
YWlsPWUuZGV0YWlsKTppLmluaXRFdmVudChuLCExLCExKSksdC5kaXNwYXRjaEV2ZW50KGkpfWZ1
bmN0aW9uIFhuKHQsbil7cmV0dXJuIGZ1bmN0aW9uKCl7cmV0dXJuIEhuKHRoaXMsdCxuKX19ZnVu
Y3Rpb24gR24odCxuKXtyZXR1cm4gZnVuY3Rpb24oKXtyZXR1cm4gSG4odGhpcyx0LG4uYXBwbHko
dGhpcyxhcmd1bWVudHMpKX19ZW4ucHJvdG90eXBlPXtjb25zdHJ1Y3RvcjplbixhcHBlbmRDaGls
ZDpmdW5jdGlvbih0KXtyZXR1cm4gdGhpcy5fcGFyZW50Lmluc2VydEJlZm9yZSh0LHRoaXMuX25l
eHQpfSxpbnNlcnRCZWZvcmU6ZnVuY3Rpb24odCxuKXtyZXR1cm4gdGhpcy5fcGFyZW50Lmluc2Vy
dEJlZm9yZSh0LG4pfSxxdWVyeVNlbGVjdG9yOmZ1bmN0aW9uKHQpe3JldHVybiB0aGlzLl9wYXJl
bnQucXVlcnlTZWxlY3Rvcih0KX0scXVlcnlTZWxlY3RvckFsbDpmdW5jdGlvbih0KXtyZXR1cm4g
dGhpcy5fcGFyZW50LnF1ZXJ5U2VsZWN0b3JBbGwodCl9fSxBbi5wcm90b3R5cGU9e2FkZDpmdW5j
dGlvbih0KXt0aGlzLl9uYW1lcy5pbmRleE9mKHQpPDAmJih0aGlzLl9uYW1lcy5wdXNoKHQpLHRo
aXMuX25vZGUuc2V0QXR0cmlidXRlKCJjbGFzcyIsdGhpcy5fbmFtZXMuam9pbigiICIpKSl9LHJl
bW92ZTpmdW5jdGlvbih0KXt2YXIgbj10aGlzLl9uYW1lcy5pbmRleE9mKHQpO24+PTAmJih0aGlz
Ll9uYW1lcy5zcGxpY2UobiwxKSx0aGlzLl9ub2RlLnNldEF0dHJpYnV0ZSgiY2xhc3MiLHRoaXMu
X25hbWVzLmpvaW4oIiAiKSkpfSxjb250YWluczpmdW5jdGlvbih0KXtyZXR1cm4gdGhpcy5fbmFt
ZXMuaW5kZXhPZih0KT49MH19O3ZhciBWbj1bbnVsbF07ZnVuY3Rpb24gV24odCxuKXt0aGlzLl9n
cm91cHM9dCx0aGlzLl9wYXJlbnRzPW59ZnVuY3Rpb24gWm4oKXtyZXR1cm4gbmV3IFduKFtbZG9j
dW1lbnQuZG9jdW1lbnRFbGVtZW50XV0sVm4pfWZ1bmN0aW9uIEtuKHQpe3JldHVybiJzdHJpbmci
PT10eXBlb2YgdD9uZXcgV24oW1tkb2N1bWVudC5xdWVyeVNlbGVjdG9yKHQpXV0sW2RvY3VtZW50
LmRvY3VtZW50RWxlbWVudF0pOm5ldyBXbihbW3RdXSxWbil9V24ucHJvdG90eXBlPVpuLnByb3Rv
dHlwZT17Y29uc3RydWN0b3I6V24sc2VsZWN0OmZ1bmN0aW9uKHQpeyJmdW5jdGlvbiIhPXR5cGVv
ZiB0JiYodD1IdCh0KSk7Zm9yKHZhciBuPXRoaXMuX2dyb3VwcyxlPW4ubGVuZ3RoLHI9bmV3IEFy
cmF5KGUpLGk9MDtpPGU7KytpKWZvcih2YXIgbyxhLHU9bltpXSxjPXUubGVuZ3RoLGY9cltpXT1u
ZXcgQXJyYXkoYykscz0wO3M8YzsrK3MpKG89dVtzXSkmJihhPXQuY2FsbChvLG8uX19kYXRhX18s
cyx1KSkmJigiX19kYXRhX18iaW4gbyYmKGEuX19kYXRhX189by5fX2RhdGFfXyksZltzXT1hKTty
ZXR1cm4gbmV3IFduKHIsdGhpcy5fcGFyZW50cyl9LHNlbGVjdEFsbDpmdW5jdGlvbih0KXt0PSJm
dW5jdGlvbiI9PXR5cGVvZiB0P2Z1bmN0aW9uKHQpe3JldHVybiBmdW5jdGlvbigpe3JldHVybiBY
dCh0LmFwcGx5KHRoaXMsYXJndW1lbnRzKSl9fSh0KTpWdCh0KTtmb3IodmFyIG49dGhpcy5fZ3Jv
dXBzLGU9bi5sZW5ndGgscj1bXSxpPVtdLG89MDtvPGU7KytvKWZvcih2YXIgYSx1PW5bb10sYz11
Lmxlbmd0aCxmPTA7ZjxjOysrZikoYT11W2ZdKSYmKHIucHVzaCh0LmNhbGwoYSxhLl9fZGF0YV9f
LGYsdSkpLGkucHVzaChhKSk7cmV0dXJuIG5ldyBXbihyLGkpfSxzZWxlY3RDaGlsZDpmdW5jdGlv
bih0KXtyZXR1cm4gdGhpcy5zZWxlY3QobnVsbD09dD9RdDpmdW5jdGlvbih0KXtyZXR1cm4gZnVu
Y3Rpb24oKXtyZXR1cm4gS3QuY2FsbCh0aGlzLmNoaWxkcmVuLHQpfX0oImZ1bmN0aW9uIj09dHlw
ZW9mIHQ/dDpadCh0KSkpfSxzZWxlY3RDaGlsZHJlbjpmdW5jdGlvbih0KXtyZXR1cm4gdGhpcy5z
ZWxlY3RBbGwobnVsbD09dD90bjpmdW5jdGlvbih0KXtyZXR1cm4gZnVuY3Rpb24oKXtyZXR1cm4g
SnQuY2FsbCh0aGlzLmNoaWxkcmVuLHQpfX0oImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpadCh0KSkp
fSxmaWx0ZXI6ZnVuY3Rpb24odCl7ImZ1bmN0aW9uIiE9dHlwZW9mIHQmJih0PVd0KHQpKTtmb3Io
dmFyIG49dGhpcy5fZ3JvdXBzLGU9bi5sZW5ndGgscj1uZXcgQXJyYXkoZSksaT0wO2k8ZTsrK2kp
Zm9yKHZhciBvLGE9bltpXSx1PWEubGVuZ3RoLGM9cltpXT1bXSxmPTA7Zjx1OysrZikobz1hW2Zd
KSYmdC5jYWxsKG8sby5fX2RhdGFfXyxmLGEpJiZjLnB1c2gobyk7cmV0dXJuIG5ldyBXbihyLHRo
aXMuX3BhcmVudHMpfSxkYXRhOmZ1bmN0aW9uKHQsbil7aWYoIWFyZ3VtZW50cy5sZW5ndGgpcmV0
dXJuIEFycmF5LmZyb20odGhpcyxhbik7dmFyIGU9bj9vbjpybixyPXRoaXMuX3BhcmVudHMsaT10
aGlzLl9ncm91cHM7ImZ1bmN0aW9uIiE9dHlwZW9mIHQmJih0PWZ1bmN0aW9uKHQpe3JldHVybiBm
dW5jdGlvbigpe3JldHVybiB0fX0odCkpO2Zvcih2YXIgbz1pLmxlbmd0aCxhPW5ldyBBcnJheShv
KSx1PW5ldyBBcnJheShvKSxjPW5ldyBBcnJheShvKSxmPTA7ZjxvOysrZil7dmFyIHM9cltmXSxs
PWlbZl0saD1sLmxlbmd0aCxkPXVuKHQuY2FsbChzLHMmJnMuX19kYXRhX18sZixyKSkscD1kLmxl
bmd0aCxnPXVbZl09bmV3IEFycmF5KHApLHk9YVtmXT1uZXcgQXJyYXkocCk7ZShzLGwsZyx5LGNb
Zl09bmV3IEFycmF5KGgpLGQsbik7Zm9yKHZhciB2LF8sYj0wLG09MDtiPHA7KytiKWlmKHY9Z1ti
XSl7Zm9yKGI+PW0mJihtPWIrMSk7IShfPXlbbV0pJiYrK208cDspO3YuX25leHQ9X3x8bnVsbH19
cmV0dXJuKGE9bmV3IFduKGEscikpLl9lbnRlcj11LGEuX2V4aXQ9YyxhfSxlbnRlcjpmdW5jdGlv
bigpe3JldHVybiBuZXcgV24odGhpcy5fZW50ZXJ8fHRoaXMuX2dyb3Vwcy5tYXAobm4pLHRoaXMu
X3BhcmVudHMpfSxleGl0OmZ1bmN0aW9uKCl7cmV0dXJuIG5ldyBXbih0aGlzLl9leGl0fHx0aGlz
Ll9ncm91cHMubWFwKG5uKSx0aGlzLl9wYXJlbnRzKX0sam9pbjpmdW5jdGlvbih0LG4sZSl7dmFy
IHI9dGhpcy5lbnRlcigpLGk9dGhpcyxvPXRoaXMuZXhpdCgpO3JldHVybiJmdW5jdGlvbiI9PXR5
cGVvZiB0PyhyPXQocikpJiYocj1yLnNlbGVjdGlvbigpKTpyPXIuYXBwZW5kKHQrIiIpLG51bGwh
PW4mJihpPW4oaSkpJiYoaT1pLnNlbGVjdGlvbigpKSxudWxsPT1lP28ucmVtb3ZlKCk6ZShvKSxy
JiZpP3IubWVyZ2UoaSkub3JkZXIoKTppfSxtZXJnZTpmdW5jdGlvbih0KXtmb3IodmFyIG49dC5z
ZWxlY3Rpb24/dC5zZWxlY3Rpb24oKTp0LGU9dGhpcy5fZ3JvdXBzLHI9bi5fZ3JvdXBzLGk9ZS5s
ZW5ndGgsbz1yLmxlbmd0aCxhPU1hdGgubWluKGksbyksdT1uZXcgQXJyYXkoaSksYz0wO2M8YTsr
K2MpZm9yKHZhciBmLHM9ZVtjXSxsPXJbY10saD1zLmxlbmd0aCxkPXVbY109bmV3IEFycmF5KGgp
LHA9MDtwPGg7KytwKShmPXNbcF18fGxbcF0pJiYoZFtwXT1mKTtmb3IoO2M8aTsrK2MpdVtjXT1l
W2NdO3JldHVybiBuZXcgV24odSx0aGlzLl9wYXJlbnRzKX0sc2VsZWN0aW9uOmZ1bmN0aW9uKCl7
cmV0dXJuIHRoaXN9LG9yZGVyOmZ1bmN0aW9uKCl7Zm9yKHZhciB0PXRoaXMuX2dyb3VwcyxuPS0x
LGU9dC5sZW5ndGg7KytuPGU7KWZvcih2YXIgcixpPXRbbl0sbz1pLmxlbmd0aC0xLGE9aVtvXTst
LW8+PTA7KShyPWlbb10pJiYoYSYmNF5yLmNvbXBhcmVEb2N1bWVudFBvc2l0aW9uKGEpJiZhLnBh
cmVudE5vZGUuaW5zZXJ0QmVmb3JlKHIsYSksYT1yKTtyZXR1cm4gdGhpc30sc29ydDpmdW5jdGlv
bih0KXtmdW5jdGlvbiBuKG4sZSl7cmV0dXJuIG4mJmU/dChuLl9fZGF0YV9fLGUuX19kYXRhX18p
OiFuLSFlfXR8fCh0PWNuKTtmb3IodmFyIGU9dGhpcy5fZ3JvdXBzLHI9ZS5sZW5ndGgsaT1uZXcg
QXJyYXkociksbz0wO288cjsrK28pe2Zvcih2YXIgYSx1PWVbb10sYz11Lmxlbmd0aCxmPWlbb109
bmV3IEFycmF5KGMpLHM9MDtzPGM7KytzKShhPXVbc10pJiYoZltzXT1hKTtmLnNvcnQobil9cmV0
dXJuIG5ldyBXbihpLHRoaXMuX3BhcmVudHMpLm9yZGVyKCl9LGNhbGw6ZnVuY3Rpb24oKXt2YXIg
dD1hcmd1bWVudHNbMF07cmV0dXJuIGFyZ3VtZW50c1swXT10aGlzLHQuYXBwbHkobnVsbCxhcmd1
bWVudHMpLHRoaXN9LG5vZGVzOmZ1bmN0aW9uKCl7cmV0dXJuIEFycmF5LmZyb20odGhpcyl9LG5v
ZGU6ZnVuY3Rpb24oKXtmb3IodmFyIHQ9dGhpcy5fZ3JvdXBzLG49MCxlPXQubGVuZ3RoO248ZTsr
K24pZm9yKHZhciByPXRbbl0saT0wLG89ci5sZW5ndGg7aTxvOysraSl7dmFyIGE9cltpXTtpZihh
KXJldHVybiBhfXJldHVybiBudWxsfSxzaXplOmZ1bmN0aW9uKCl7bGV0IHQ9MDtmb3IoY29uc3Qg
biBvZiB0aGlzKSsrdDtyZXR1cm4gdH0sZW1wdHk6ZnVuY3Rpb24oKXtyZXR1cm4hdGhpcy5ub2Rl
KCl9LGVhY2g6ZnVuY3Rpb24odCl7Zm9yKHZhciBuPXRoaXMuX2dyb3VwcyxlPTAscj1uLmxlbmd0
aDtlPHI7KytlKWZvcih2YXIgaSxvPW5bZV0sYT0wLHU9by5sZW5ndGg7YTx1OysrYSkoaT1vW2Fd
KSYmdC5jYWxsKGksaS5fX2RhdGFfXyxhLG8pO3JldHVybiB0aGlzfSxhdHRyOmZ1bmN0aW9uKHQs
bil7dmFyIGU9T3QodCk7aWYoYXJndW1lbnRzLmxlbmd0aDwyKXt2YXIgcj10aGlzLm5vZGUoKTty
ZXR1cm4gZS5sb2NhbD9yLmdldEF0dHJpYnV0ZU5TKGUuc3BhY2UsZS5sb2NhbCk6ci5nZXRBdHRy
aWJ1dGUoZSl9cmV0dXJuIHRoaXMuZWFjaCgobnVsbD09bj9lLmxvY2FsP3NuOmZuOiJmdW5jdGlv
biI9PXR5cGVvZiBuP2UubG9jYWw/cG46ZG46ZS5sb2NhbD9objpsbikoZSxuKSl9LHN0eWxlOmZ1
bmN0aW9uKHQsbixlKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD4xP3RoaXMuZWFjaCgobnVsbD09
bj95bjoiZnVuY3Rpb24iPT10eXBlb2Ygbj9fbjp2bikodCxuLG51bGw9PWU/IiI6ZSkpOmJuKHRo
aXMubm9kZSgpLHQpfSxwcm9wZXJ0eTpmdW5jdGlvbih0LG4pe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPjE/dGhpcy5lYWNoKChudWxsPT1uP21uOiJmdW5jdGlvbiI9PXR5cGVvZiBuP3duOnhuKSh0
LG4pKTp0aGlzLm5vZGUoKVt0XX0sY2xhc3NlZDpmdW5jdGlvbih0LG4pe3ZhciBlPU1uKHQrIiIp
O2lmKGFyZ3VtZW50cy5sZW5ndGg8Mil7Zm9yKHZhciByPVRuKHRoaXMubm9kZSgpKSxpPS0xLG89
ZS5sZW5ndGg7KytpPG87KWlmKCFyLmNvbnRhaW5zKGVbaV0pKXJldHVybiExO3JldHVybiEwfXJl
dHVybiB0aGlzLmVhY2goKCJmdW5jdGlvbiI9PXR5cGVvZiBuP0NuOm4/Tm46a24pKGUsbikpfSx0
ZXh0OmZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoP3RoaXMuZWFjaChudWxsPT10
P1BuOigiZnVuY3Rpb24iPT10eXBlb2YgdD8kbjp6bikodCkpOnRoaXMubm9kZSgpLnRleHRDb250
ZW50fSxodG1sOmZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoP3RoaXMuZWFjaChu
dWxsPT10P0RuOigiZnVuY3Rpb24iPT10eXBlb2YgdD9GbjpSbikodCkpOnRoaXMubm9kZSgpLmlu
bmVySFRNTH0scmFpc2U6ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5lYWNoKHFuKX0sbG93ZXI6ZnVu
Y3Rpb24oKXtyZXR1cm4gdGhpcy5lYWNoKFVuKX0sYXBwZW5kOmZ1bmN0aW9uKHQpe3ZhciBuPSJm
dW5jdGlvbiI9PXR5cGVvZiB0P3Q6THQodCk7cmV0dXJuIHRoaXMuc2VsZWN0KChmdW5jdGlvbigp
e3JldHVybiB0aGlzLmFwcGVuZENoaWxkKG4uYXBwbHkodGhpcyxhcmd1bWVudHMpKX0pKX0saW5z
ZXJ0OmZ1bmN0aW9uKHQsbil7dmFyIGU9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpMdCh0KSxyPW51
bGw9PW4/SW46ImZ1bmN0aW9uIj09dHlwZW9mIG4/bjpIdChuKTtyZXR1cm4gdGhpcy5zZWxlY3Qo
KGZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuaW5zZXJ0QmVmb3JlKGUuYXBwbHkodGhpcyxhcmd1bWVu
dHMpLHIuYXBwbHkodGhpcyxhcmd1bWVudHMpfHxudWxsKX0pKX0scmVtb3ZlOmZ1bmN0aW9uKCl7
cmV0dXJuIHRoaXMuZWFjaChPbil9LGNsb25lOmZ1bmN0aW9uKHQpe3JldHVybiB0aGlzLnNlbGVj
dCh0P1luOkJuKX0sZGF0dW06ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/dGhp
cy5wcm9wZXJ0eSgiX19kYXRhX18iLHQpOnRoaXMubm9kZSgpLl9fZGF0YV9ffSxvbjpmdW5jdGlv
bih0LG4sZSl7dmFyIHIsaSxvPWZ1bmN0aW9uKHQpe3JldHVybiB0LnRyaW0oKS5zcGxpdCgvXnxc
cysvKS5tYXAoKGZ1bmN0aW9uKHQpe3ZhciBuPSIiLGU9dC5pbmRleE9mKCIuIik7cmV0dXJuIGU+
PTAmJihuPXQuc2xpY2UoZSsxKSx0PXQuc2xpY2UoMCxlKSkse3R5cGU6dCxuYW1lOm59fSkpfSh0
KyIiKSxhPW8ubGVuZ3RoO2lmKCEoYXJndW1lbnRzLmxlbmd0aDwyKSl7Zm9yKHU9bj9qbjpMbixy
PTA7cjxhOysrcil0aGlzLmVhY2godShvW3JdLG4sZSkpO3JldHVybiB0aGlzfXZhciB1PXRoaXMu
bm9kZSgpLl9fb247aWYodSlmb3IodmFyIGMsZj0wLHM9dS5sZW5ndGg7ZjxzOysrZilmb3Iocj0w
LGM9dVtmXTtyPGE7KytyKWlmKChpPW9bcl0pLnR5cGU9PT1jLnR5cGUmJmkubmFtZT09PWMubmFt
ZSlyZXR1cm4gYy52YWx1ZX0sZGlzcGF0Y2g6ZnVuY3Rpb24odCxuKXtyZXR1cm4gdGhpcy5lYWNo
KCgiZnVuY3Rpb24iPT10eXBlb2Ygbj9HbjpYbikodCxuKSl9LFtTeW1ib2wuaXRlcmF0b3JdOmZ1
bmN0aW9uKigpe2Zvcih2YXIgdD10aGlzLl9ncm91cHMsbj0wLGU9dC5sZW5ndGg7bjxlOysrbilm
b3IodmFyIHIsaT10W25dLG89MCxhPWkubGVuZ3RoO288YTsrK28pKHI9aVtvXSkmJih5aWVsZCBy
KX19O3ZhciBRbj0wO2Z1bmN0aW9uIEpuKCl7cmV0dXJuIG5ldyB0ZX1mdW5jdGlvbiB0ZSgpe3Ro
aXMuXz0iQCIrKCsrUW4pLnRvU3RyaW5nKDM2KX1mdW5jdGlvbiBuZSh0KXtsZXQgbjtmb3IoO249
dC5zb3VyY2VFdmVudDspdD1uO3JldHVybiB0fWZ1bmN0aW9uIGVlKHQsbil7aWYodD1uZSh0KSx2
b2lkIDA9PT1uJiYobj10LmN1cnJlbnRUYXJnZXQpLG4pe3ZhciBlPW4ub3duZXJTVkdFbGVtZW50
fHxuO2lmKGUuY3JlYXRlU1ZHUG9pbnQpe3ZhciByPWUuY3JlYXRlU1ZHUG9pbnQoKTtyZXR1cm4g
ci54PXQuY2xpZW50WCxyLnk9dC5jbGllbnRZLFsocj1yLm1hdHJpeFRyYW5zZm9ybShuLmdldFNj
cmVlbkNUTSgpLmludmVyc2UoKSkpLngsci55XX1pZihuLmdldEJvdW5kaW5nQ2xpZW50UmVjdCl7
dmFyIGk9bi5nZXRCb3VuZGluZ0NsaWVudFJlY3QoKTtyZXR1cm5bdC5jbGllbnRYLWkubGVmdC1u
LmNsaWVudExlZnQsdC5jbGllbnRZLWkudG9wLW4uY2xpZW50VG9wXX19cmV0dXJuW3QucGFnZVgs
dC5wYWdlWV19dGUucHJvdG90eXBlPUpuLnByb3RvdHlwZT17Y29uc3RydWN0b3I6dGUsZ2V0OmZ1
bmN0aW9uKHQpe2Zvcih2YXIgbj10aGlzLl87IShuIGluIHQpOylpZighKHQ9dC5wYXJlbnROb2Rl
KSlyZXR1cm47cmV0dXJuIHRbbl19LHNldDpmdW5jdGlvbih0LG4pe3JldHVybiB0W3RoaXMuX109
bn0scmVtb3ZlOmZ1bmN0aW9uKHQpe3JldHVybiB0aGlzLl8gaW4gdCYmZGVsZXRlIHRbdGhpcy5f
XX0sdG9TdHJpbmc6ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5ffX07Y29uc3QgcmU9e3Bhc3NpdmU6
ITF9LGllPXtjYXB0dXJlOiEwLHBhc3NpdmU6ITF9O2Z1bmN0aW9uIG9lKHQpe3Quc3RvcEltbWVk
aWF0ZVByb3BhZ2F0aW9uKCl9ZnVuY3Rpb24gYWUodCl7dC5wcmV2ZW50RGVmYXVsdCgpLHQuc3Rv
cEltbWVkaWF0ZVByb3BhZ2F0aW9uKCl9ZnVuY3Rpb24gdWUodCl7dmFyIG49dC5kb2N1bWVudC5k
b2N1bWVudEVsZW1lbnQsZT1Lbih0KS5vbigiZHJhZ3N0YXJ0LmRyYWciLGFlLGllKTsib25zZWxl
Y3RzdGFydCJpbiBuP2Uub24oInNlbGVjdHN0YXJ0LmRyYWciLGFlLGllKToobi5fX25vc2VsZWN0
PW4uc3R5bGUuTW96VXNlclNlbGVjdCxuLnN0eWxlLk1velVzZXJTZWxlY3Q9Im5vbmUiKX1mdW5j
dGlvbiBjZSh0LG4pe3ZhciBlPXQuZG9jdW1lbnQuZG9jdW1lbnRFbGVtZW50LHI9S24odCkub24o
ImRyYWdzdGFydC5kcmFnIixudWxsKTtuJiYoci5vbigiY2xpY2suZHJhZyIsYWUsaWUpLHNldFRp
bWVvdXQoKGZ1bmN0aW9uKCl7ci5vbigiY2xpY2suZHJhZyIsbnVsbCl9KSwwKSksIm9uc2VsZWN0
c3RhcnQiaW4gZT9yLm9uKCJzZWxlY3RzdGFydC5kcmFnIixudWxsKTooZS5zdHlsZS5Nb3pVc2Vy
U2VsZWN0PWUuX19ub3NlbGVjdCxkZWxldGUgZS5fX25vc2VsZWN0KX12YXIgZmU9dD0+KCk9PnQ7
ZnVuY3Rpb24gc2UodCx7c291cmNlRXZlbnQ6bixzdWJqZWN0OmUsdGFyZ2V0OnIsaWRlbnRpZmll
cjppLGFjdGl2ZTpvLHg6YSx5OnUsZHg6YyxkeTpmLGRpc3BhdGNoOnN9KXtPYmplY3QuZGVmaW5l
UHJvcGVydGllcyh0aGlzLHt0eXBlOnt2YWx1ZTp0LGVudW1lcmFibGU6ITAsY29uZmlndXJhYmxl
OiEwfSxzb3VyY2VFdmVudDp7dmFsdWU6bixlbnVtZXJhYmxlOiEwLGNvbmZpZ3VyYWJsZTohMH0s
c3ViamVjdDp7dmFsdWU6ZSxlbnVtZXJhYmxlOiEwLGNvbmZpZ3VyYWJsZTohMH0sdGFyZ2V0Ont2
YWx1ZTpyLGVudW1lcmFibGU6ITAsY29uZmlndXJhYmxlOiEwfSxpZGVudGlmaWVyOnt2YWx1ZTpp
LGVudW1lcmFibGU6ITAsY29uZmlndXJhYmxlOiEwfSxhY3RpdmU6e3ZhbHVlOm8sZW51bWVyYWJs
ZTohMCxjb25maWd1cmFibGU6ITB9LHg6e3ZhbHVlOmEsZW51bWVyYWJsZTohMCxjb25maWd1cmFi
bGU6ITB9LHk6e3ZhbHVlOnUsZW51bWVyYWJsZTohMCxjb25maWd1cmFibGU6ITB9LGR4Ont2YWx1
ZTpjLGVudW1lcmFibGU6ITAsY29uZmlndXJhYmxlOiEwfSxkeTp7dmFsdWU6ZixlbnVtZXJhYmxl
OiEwLGNvbmZpZ3VyYWJsZTohMH0sXzp7dmFsdWU6c319KX1mdW5jdGlvbiBsZSh0KXtyZXR1cm4h
dC5jdHJsS2V5JiYhdC5idXR0b259ZnVuY3Rpb24gaGUoKXtyZXR1cm4gdGhpcy5wYXJlbnROb2Rl
fWZ1bmN0aW9uIGRlKHQsbil7cmV0dXJuIG51bGw9PW4/e3g6dC54LHk6dC55fTpufWZ1bmN0aW9u
IHBlKCl7cmV0dXJuIG5hdmlnYXRvci5tYXhUb3VjaFBvaW50c3x8Im9udG91Y2hzdGFydCJpbiB0
aGlzfWZ1bmN0aW9uIGdlKHQsbixlKXt0LnByb3RvdHlwZT1uLnByb3RvdHlwZT1lLGUuY29uc3Ry
dWN0b3I9dH1mdW5jdGlvbiB5ZSh0LG4pe3ZhciBlPU9iamVjdC5jcmVhdGUodC5wcm90b3R5cGUp
O2Zvcih2YXIgciBpbiBuKWVbcl09bltyXTtyZXR1cm4gZX1mdW5jdGlvbiB2ZSgpe31zZS5wcm90
b3R5cGUub249ZnVuY3Rpb24oKXt2YXIgdD10aGlzLl8ub24uYXBwbHkodGhpcy5fLGFyZ3VtZW50
cyk7cmV0dXJuIHQ9PT10aGlzLl8/dGhpczp0fTt2YXIgX2U9LjcsYmU9MS9fZSxtZT0iXFxzKihb
Ky1dP1xcZCspXFxzKiIseGU9IlxccyooWystXT8oPzpcXGQqXFwuKT9cXGQrKD86W2VFXVsrLV0/
XFxkKyk/KVxccyoiLHdlPSJcXHMqKFsrLV0/KD86XFxkKlxcLik/XFxkKyg/OltlRV1bKy1dP1xc
ZCspPyklXFxzKiIsTWU9L14jKFswLTlhLWZdezMsOH0pJC8sVGU9bmV3IFJlZ0V4cChgXnJnYlxc
KCR7bWV9LCR7bWV9LCR7bWV9XFwpJGApLEFlPW5ldyBSZWdFeHAoYF5yZ2JcXCgke3dlfSwke3dl
fSwke3dlfVxcKSRgKSxTZT1uZXcgUmVnRXhwKGBecmdiYVxcKCR7bWV9LCR7bWV9LCR7bWV9LCR7
eGV9XFwpJGApLEVlPW5ldyBSZWdFeHAoYF5yZ2JhXFwoJHt3ZX0sJHt3ZX0sJHt3ZX0sJHt4ZX1c
XCkkYCksTmU9bmV3IFJlZ0V4cChgXmhzbFxcKCR7eGV9LCR7d2V9LCR7d2V9XFwpJGApLGtlPW5l
dyBSZWdFeHAoYF5oc2xhXFwoJHt4ZX0sJHt3ZX0sJHt3ZX0sJHt4ZX1cXCkkYCksQ2U9e2FsaWNl
Ymx1ZToxNTc5MjM4MyxhbnRpcXVld2hpdGU6MTY0NDQzNzUsYXF1YTo2NTUzNSxhcXVhbWFyaW5l
OjgzODg1NjQsYXp1cmU6MTU3OTQxNzUsYmVpZ2U6MTYxMTkyNjAsYmlzcXVlOjE2NzcwMjQ0LGJs
YWNrOjAsYmxhbmNoZWRhbG1vbmQ6MTY3NzIwNDUsYmx1ZToyNTUsYmx1ZXZpb2xldDo5MDU1MjAy
LGJyb3duOjEwODI0MjM0LGJ1cmx5d29vZDoxNDU5NjIzMSxjYWRldGJsdWU6NjI2NjUyOCxjaGFy
dHJldXNlOjgzODgzNTIsY2hvY29sYXRlOjEzNzg5NDcwLGNvcmFsOjE2NzQ0MjcyLGNvcm5mbG93
ZXJibHVlOjY1OTE5ODEsY29ybnNpbGs6MTY3NzUzODgsY3JpbXNvbjoxNDQyMzEwMCxjeWFuOjY1
NTM1LGRhcmtibHVlOjEzOSxkYXJrY3lhbjozNTcyMyxkYXJrZ29sZGVucm9kOjEyMDkyOTM5LGRh
cmtncmF5OjExMTE5MDE3LGRhcmtncmVlbjoyNTYwMCxkYXJrZ3JleToxMTExOTAxNyxkYXJra2hh
a2k6MTI0MzMyNTksZGFya21hZ2VudGE6OTEwOTY0MyxkYXJrb2xpdmVncmVlbjo1NTk3OTk5LGRh
cmtvcmFuZ2U6MTY3NDc1MjAsZGFya29yY2hpZDoxMDA0MDAxMixkYXJrcmVkOjkxMDk1MDQsZGFy
a3NhbG1vbjoxNTMwODQxMCxkYXJrc2VhZ3JlZW46OTQxOTkxOSxkYXJrc2xhdGVibHVlOjQ3MzQz
NDcsZGFya3NsYXRlZ3JheTozMTAwNDk1LGRhcmtzbGF0ZWdyZXk6MzEwMDQ5NSxkYXJrdHVycXVv
aXNlOjUyOTQ1LGRhcmt2aW9sZXQ6OTY5OTUzOSxkZWVwcGluazoxNjcxNjk0NyxkZWVwc2t5Ymx1
ZTo0OTE1MSxkaW1ncmF5OjY5MDgyNjUsZGltZ3JleTo2OTA4MjY1LGRvZGdlcmJsdWU6MjAwMzE5
OSxmaXJlYnJpY2s6MTE2NzQxNDYsZmxvcmFsd2hpdGU6MTY3NzU5MjAsZm9yZXN0Z3JlZW46MjI2
Mzg0MixmdWNoc2lhOjE2NzExOTM1LGdhaW5zYm9ybzoxNDQ3NDQ2MCxnaG9zdHdoaXRlOjE2MzE2
NjcxLGdvbGQ6MTY3NjY3MjAsZ29sZGVucm9kOjE0MzI5MTIwLGdyYXk6ODQyMTUwNCxncmVlbjoz
Mjc2OCxncmVlbnllbGxvdzoxMTQwMzA1NSxncmV5Ojg0MjE1MDQsaG9uZXlkZXc6MTU3OTQxNjAs
aG90cGluazoxNjczODc0MCxpbmRpYW5yZWQ6MTM0NTg1MjQsaW5kaWdvOjQ5MTUzMzAsaXZvcnk6
MTY3NzcyMDAsa2hha2k6MTU3ODc2NjAsbGF2ZW5kZXI6MTUxMzI0MTAsbGF2ZW5kZXJibHVzaDox
Njc3MzM2NSxsYXduZ3JlZW46ODE5MDk3NixsZW1vbmNoaWZmb246MTY3NzU4ODUsbGlnaHRibHVl
OjExMzkzMjU0LGxpZ2h0Y29yYWw6MTU3NjE1MzYsbGlnaHRjeWFuOjE0NzQ1NTk5LGxpZ2h0Z29s
ZGVucm9keWVsbG93OjE2NDQ4MjEwLGxpZ2h0Z3JheToxMzg4MjMyMyxsaWdodGdyZWVuOjk0OTgy
NTYsbGlnaHRncmV5OjEzODgyMzIzLGxpZ2h0cGluazoxNjc1ODQ2NSxsaWdodHNhbG1vbjoxNjc1
Mjc2MixsaWdodHNlYWdyZWVuOjIxNDI4OTAsbGlnaHRza3libHVlOjg5MDAzNDYsbGlnaHRzbGF0
ZWdyYXk6NzgzMzc1MyxsaWdodHNsYXRlZ3JleTo3ODMzNzUzLGxpZ2h0c3RlZWxibHVlOjExNTg0
NzM0LGxpZ2h0eWVsbG93OjE2Nzc3MTg0LGxpbWU6NjUyODAsbGltZWdyZWVuOjMzMjkzMzAsbGlu
ZW46MTY0NDU2NzAsbWFnZW50YToxNjcxMTkzNSxtYXJvb246ODM4ODYwOCxtZWRpdW1hcXVhbWFy
aW5lOjY3MzczMjIsbWVkaXVtYmx1ZToyMDUsbWVkaXVtb3JjaGlkOjEyMjExNjY3LG1lZGl1bXB1
cnBsZTo5NjYyNjgzLG1lZGl1bXNlYWdyZWVuOjM5NzgwOTcsbWVkaXVtc2xhdGVibHVlOjgwODc3
OTAsbWVkaXVtc3ByaW5nZ3JlZW46NjQxNTQsbWVkaXVtdHVycXVvaXNlOjQ3NzIzMDAsbWVkaXVt
dmlvbGV0cmVkOjEzMDQ3MTczLG1pZG5pZ2h0Ymx1ZToxNjQ0OTEyLG1pbnRjcmVhbToxNjEyMTg1
MCxtaXN0eXJvc2U6MTY3NzAyNzMsbW9jY2FzaW46MTY3NzAyMjksbmF2YWpvd2hpdGU6MTY3Njg2
ODUsbmF2eToxMjgsb2xkbGFjZToxNjY0MzU1OCxvbGl2ZTo4NDIxMzc2LG9saXZlZHJhYjo3MDQ4
NzM5LG9yYW5nZToxNjc1MzkyMCxvcmFuZ2VyZWQ6MTY3MjkzNDQsb3JjaGlkOjE0MzE1NzM0LHBh
bGVnb2xkZW5yb2Q6MTU2NTcxMzAscGFsZWdyZWVuOjEwMDI1ODgwLHBhbGV0dXJxdW9pc2U6MTE1
Mjk5NjYscGFsZXZpb2xldHJlZDoxNDM4MTIwMyxwYXBheWF3aGlwOjE2NzczMDc3LHBlYWNocHVm
ZjoxNjc2NzY3MyxwZXJ1OjEzNDY4OTkxLHBpbms6MTY3NjEwMzUscGx1bToxNDUyNDYzNyxwb3dk
ZXJibHVlOjExNTkxOTEwLHB1cnBsZTo4Mzg4NzM2LHJlYmVjY2FwdXJwbGU6NjY5Nzg4MSxyZWQ6
MTY3MTE2ODAscm9zeWJyb3duOjEyMzU3NTE5LHJveWFsYmx1ZTo0Mjg2OTQ1LHNhZGRsZWJyb3du
OjkxMjcxODcsc2FsbW9uOjE2NDE2ODgyLHNhbmR5YnJvd246MTYwMzI4NjQsc2VhZ3JlZW46MzA1
MDMyNyxzZWFzaGVsbDoxNjc3NDYzOCxzaWVubmE6MTA1MDY3OTcsc2lsdmVyOjEyNjMyMjU2LHNr
eWJsdWU6ODkwMDMzMSxzbGF0ZWJsdWU6Njk3MDA2MSxzbGF0ZWdyYXk6NzM3Mjk0NCxzbGF0ZWdy
ZXk6NzM3Mjk0NCxzbm93OjE2Nzc1OTMwLHNwcmluZ2dyZWVuOjY1NDA3LHN0ZWVsYmx1ZTo0NjIw
OTgwLHRhbjoxMzgwODc4MCx0ZWFsOjMyODk2LHRoaXN0bGU6MTQyMDQ4ODgsdG9tYXRvOjE2NzM3
MDk1LHR1cnF1b2lzZTo0MjUxODU2LHZpb2xldDoxNTYzMTA4Nix3aGVhdDoxNjExMzMzMSx3aGl0
ZToxNjc3NzIxNSx3aGl0ZXNtb2tlOjE2MTE5Mjg1LHllbGxvdzoxNjc3Njk2MCx5ZWxsb3dncmVl
bjoxMDE0NTA3NH07ZnVuY3Rpb24gUGUoKXtyZXR1cm4gdGhpcy5yZ2IoKS5mb3JtYXRIZXgoKX1m
dW5jdGlvbiB6ZSgpe3JldHVybiB0aGlzLnJnYigpLmZvcm1hdFJnYigpfWZ1bmN0aW9uICRlKHQp
e3ZhciBuLGU7cmV0dXJuIHQ9KHQrIiIpLnRyaW0oKS50b0xvd2VyQ2FzZSgpLChuPU1lLmV4ZWMo
dCkpPyhlPW5bMV0ubGVuZ3RoLG49cGFyc2VJbnQoblsxXSwxNiksNj09PWU/RGUobik6Mz09PWU/
bmV3IFVlKG4+PjgmMTV8bj4+NCYyNDAsbj4+NCYxNXwyNDAmbiwoMTUmbik8PDR8MTUmbiwxKTo4
PT09ZT9SZShuPj4yNCYyNTUsbj4+MTYmMjU1LG4+PjgmMjU1LCgyNTUmbikvMjU1KTo0PT09ZT9S
ZShuPj4xMiYxNXxuPj44JjI0MCxuPj44JjE1fG4+PjQmMjQwLG4+PjQmMTV8MjQwJm4sKCgxNSZu
KTw8NHwxNSZuKS8yNTUpOm51bGwpOihuPVRlLmV4ZWModCkpP25ldyBVZShuWzFdLG5bMl0sblsz
XSwxKToobj1BZS5leGVjKHQpKT9uZXcgVWUoMjU1Km5bMV0vMTAwLDI1NSpuWzJdLzEwMCwyNTUq
blszXS8xMDAsMSk6KG49U2UuZXhlYyh0KSk/UmUoblsxXSxuWzJdLG5bM10sbls0XSk6KG49RWUu
ZXhlYyh0KSk/UmUoMjU1Km5bMV0vMTAwLDI1NSpuWzJdLzEwMCwyNTUqblszXS8xMDAsbls0XSk6
KG49TmUuZXhlYyh0KSk/amUoblsxXSxuWzJdLzEwMCxuWzNdLzEwMCwxKToobj1rZS5leGVjKHQp
KT9qZShuWzFdLG5bMl0vMTAwLG5bM10vMTAwLG5bNF0pOkNlLmhhc093blByb3BlcnR5KHQpP0Rl
KENlW3RdKToidHJhbnNwYXJlbnQiPT09dD9uZXcgVWUoTmFOLE5hTixOYU4sMCk6bnVsbH1mdW5j
dGlvbiBEZSh0KXtyZXR1cm4gbmV3IFVlKHQ+PjE2JjI1NSx0Pj44JjI1NSwyNTUmdCwxKX1mdW5j
dGlvbiBSZSh0LG4sZSxyKXtyZXR1cm4gcjw9MCYmKHQ9bj1lPU5hTiksbmV3IFVlKHQsbixlLHIp
fWZ1bmN0aW9uIEZlKHQpe3JldHVybiB0IGluc3RhbmNlb2YgdmV8fCh0PSRlKHQpKSx0P25ldyBV
ZSgodD10LnJnYigpKS5yLHQuZyx0LmIsdC5vcGFjaXR5KTpuZXcgVWV9ZnVuY3Rpb24gcWUodCxu
LGUscil7cmV0dXJuIDE9PT1hcmd1bWVudHMubGVuZ3RoP0ZlKHQpOm5ldyBVZSh0LG4sZSxudWxs
PT1yPzE6cil9ZnVuY3Rpb24gVWUodCxuLGUscil7dGhpcy5yPSt0LHRoaXMuZz0rbix0aGlzLmI9
K2UsdGhpcy5vcGFjaXR5PStyfWZ1bmN0aW9uIEllKCl7cmV0dXJuYCMke0xlKHRoaXMucil9JHtM
ZSh0aGlzLmcpfSR7TGUodGhpcy5iKX1gfWZ1bmN0aW9uIE9lKCl7Y29uc3QgdD1CZSh0aGlzLm9w
YWNpdHkpO3JldHVybmAkezE9PT10PyJyZ2IoIjoicmdiYSgifSR7WWUodGhpcy5yKX0sICR7WWUo
dGhpcy5nKX0sICR7WWUodGhpcy5iKX0kezE9PT10PyIpIjpgLCAke3R9KWB9YH1mdW5jdGlvbiBC
ZSh0KXtyZXR1cm4gaXNOYU4odCk/MTpNYXRoLm1heCgwLE1hdGgubWluKDEsdCkpfWZ1bmN0aW9u
IFllKHQpe3JldHVybiBNYXRoLm1heCgwLE1hdGgubWluKDI1NSxNYXRoLnJvdW5kKHQpfHwwKSl9
ZnVuY3Rpb24gTGUodCl7cmV0dXJuKCh0PVllKHQpKTwxNj8iMCI6IiIpK3QudG9TdHJpbmcoMTYp
fWZ1bmN0aW9uIGplKHQsbixlLHIpe3JldHVybiByPD0wP3Q9bj1lPU5hTjplPD0wfHxlPj0xP3Q9
bj1OYU46bjw9MCYmKHQ9TmFOKSxuZXcgR2UodCxuLGUscil9ZnVuY3Rpb24gSGUodCl7aWYodCBp
bnN0YW5jZW9mIEdlKXJldHVybiBuZXcgR2UodC5oLHQucyx0LmwsdC5vcGFjaXR5KTtpZih0IGlu
c3RhbmNlb2YgdmV8fCh0PSRlKHQpKSwhdClyZXR1cm4gbmV3IEdlO2lmKHQgaW5zdGFuY2VvZiBH
ZSlyZXR1cm4gdDt2YXIgbj0odD10LnJnYigpKS5yLzI1NSxlPXQuZy8yNTUscj10LmIvMjU1LGk9
TWF0aC5taW4obixlLHIpLG89TWF0aC5tYXgobixlLHIpLGE9TmFOLHU9by1pLGM9KG8raSkvMjty
ZXR1cm4gdT8oYT1uPT09bz8oZS1yKS91KzYqKGU8cik6ZT09PW8/KHItbikvdSsyOihuLWUpL3Ur
NCx1Lz1jPC41P28raToyLW8taSxhKj02MCk6dT1jPjAmJmM8MT8wOmEsbmV3IEdlKGEsdSxjLHQu
b3BhY2l0eSl9ZnVuY3Rpb24gWGUodCxuLGUscil7cmV0dXJuIDE9PT1hcmd1bWVudHMubGVuZ3Ro
P0hlKHQpOm5ldyBHZSh0LG4sZSxudWxsPT1yPzE6cil9ZnVuY3Rpb24gR2UodCxuLGUscil7dGhp
cy5oPSt0LHRoaXMucz0rbix0aGlzLmw9K2UsdGhpcy5vcGFjaXR5PStyfWZ1bmN0aW9uIFZlKHQp
e3JldHVybih0PSh0fHwwKSUzNjApPDA/dCszNjA6dH1mdW5jdGlvbiBXZSh0KXtyZXR1cm4gTWF0
aC5tYXgoMCxNYXRoLm1pbigxLHR8fDApKX1mdW5jdGlvbiBaZSh0LG4sZSl7cmV0dXJuIDI1NSoo
dDw2MD9uKyhlLW4pKnQvNjA6dDwxODA/ZTp0PDI0MD9uKyhlLW4pKigyNDAtdCkvNjA6bil9Z2Uo
dmUsJGUse2NvcHkodCl7cmV0dXJuIE9iamVjdC5hc3NpZ24obmV3IHRoaXMuY29uc3RydWN0b3Is
dGhpcyx0KX0sZGlzcGxheWFibGUoKXtyZXR1cm4gdGhpcy5yZ2IoKS5kaXNwbGF5YWJsZSgpfSxo
ZXg6UGUsZm9ybWF0SGV4OlBlLGZvcm1hdEhleDg6ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5yZ2Io
KS5mb3JtYXRIZXg4KCl9LGZvcm1hdEhzbDpmdW5jdGlvbigpe3JldHVybiBIZSh0aGlzKS5mb3Jt
YXRIc2woKX0sZm9ybWF0UmdiOnplLHRvU3RyaW5nOnplfSksZ2UoVWUscWUseWUodmUse2JyaWdo
dGVyKHQpe3JldHVybiB0PW51bGw9PXQ/YmU6TWF0aC5wb3coYmUsdCksbmV3IFVlKHRoaXMucip0
LHRoaXMuZyp0LHRoaXMuYip0LHRoaXMub3BhY2l0eSl9LGRhcmtlcih0KXtyZXR1cm4gdD1udWxs
PT10P19lOk1hdGgucG93KF9lLHQpLG5ldyBVZSh0aGlzLnIqdCx0aGlzLmcqdCx0aGlzLmIqdCx0
aGlzLm9wYWNpdHkpfSxyZ2IoKXtyZXR1cm4gdGhpc30sY2xhbXAoKXtyZXR1cm4gbmV3IFVlKFll
KHRoaXMuciksWWUodGhpcy5nKSxZZSh0aGlzLmIpLEJlKHRoaXMub3BhY2l0eSkpfSxkaXNwbGF5
YWJsZSgpe3JldHVybi0uNTw9dGhpcy5yJiZ0aGlzLnI8MjU1LjUmJi0uNTw9dGhpcy5nJiZ0aGlz
Lmc8MjU1LjUmJi0uNTw9dGhpcy5iJiZ0aGlzLmI8MjU1LjUmJjA8PXRoaXMub3BhY2l0eSYmdGhp
cy5vcGFjaXR5PD0xfSxoZXg6SWUsZm9ybWF0SGV4OkllLGZvcm1hdEhleDg6ZnVuY3Rpb24oKXty
ZXR1cm5gIyR7TGUodGhpcy5yKX0ke0xlKHRoaXMuZyl9JHtMZSh0aGlzLmIpfSR7TGUoMjU1Kihp
c05hTih0aGlzLm9wYWNpdHkpPzE6dGhpcy5vcGFjaXR5KSl9YH0sZm9ybWF0UmdiOk9lLHRvU3Ry
aW5nOk9lfSkpLGdlKEdlLFhlLHllKHZlLHticmlnaHRlcih0KXtyZXR1cm4gdD1udWxsPT10P2Jl
Ok1hdGgucG93KGJlLHQpLG5ldyBHZSh0aGlzLmgsdGhpcy5zLHRoaXMubCp0LHRoaXMub3BhY2l0
eSl9LGRhcmtlcih0KXtyZXR1cm4gdD1udWxsPT10P19lOk1hdGgucG93KF9lLHQpLG5ldyBHZSh0
aGlzLmgsdGhpcy5zLHRoaXMubCp0LHRoaXMub3BhY2l0eSl9LHJnYigpe3ZhciB0PXRoaXMuaCUz
NjArMzYwKih0aGlzLmg8MCksbj1pc05hTih0KXx8aXNOYU4odGhpcy5zKT8wOnRoaXMucyxlPXRo
aXMubCxyPWUrKGU8LjU/ZToxLWUpKm4saT0yKmUtcjtyZXR1cm4gbmV3IFVlKFplKHQ+PTI0MD90
LTI0MDp0KzEyMCxpLHIpLFplKHQsaSxyKSxaZSh0PDEyMD90KzI0MDp0LTEyMCxpLHIpLHRoaXMu
b3BhY2l0eSl9LGNsYW1wKCl7cmV0dXJuIG5ldyBHZShWZSh0aGlzLmgpLFdlKHRoaXMucyksV2Uo
dGhpcy5sKSxCZSh0aGlzLm9wYWNpdHkpKX0sZGlzcGxheWFibGUoKXtyZXR1cm4oMDw9dGhpcy5z
JiZ0aGlzLnM8PTF8fGlzTmFOKHRoaXMucykpJiYwPD10aGlzLmwmJnRoaXMubDw9MSYmMDw9dGhp
cy5vcGFjaXR5JiZ0aGlzLm9wYWNpdHk8PTF9LGZvcm1hdEhzbCgpe2NvbnN0IHQ9QmUodGhpcy5v
cGFjaXR5KTtyZXR1cm5gJHsxPT09dD8iaHNsKCI6ImhzbGEoIn0ke1ZlKHRoaXMuaCl9LCAkezEw
MCpXZSh0aGlzLnMpfSUsICR7MTAwKldlKHRoaXMubCl9JSR7MT09PXQ/IikiOmAsICR7dH0pYH1g
fX0pKTtjb25zdCBLZT1NYXRoLlBJLzE4MCxRZT0xODAvTWF0aC5QSSxKZT0uOTY0MjIsdHI9MSxu
cj0uODI1MjEsZXI9NC8yOSxycj02LzI5LGlyPTMqcnIqcnIsb3I9cnIqcnIqcnI7ZnVuY3Rpb24g
YXIodCl7aWYodCBpbnN0YW5jZW9mIGNyKXJldHVybiBuZXcgY3IodC5sLHQuYSx0LmIsdC5vcGFj
aXR5KTtpZih0IGluc3RhbmNlb2YgZ3IpcmV0dXJuIHlyKHQpO3QgaW5zdGFuY2VvZiBVZXx8KHQ9
RmUodCkpO3ZhciBuLGUscj1ocih0LnIpLGk9aHIodC5nKSxvPWhyKHQuYiksYT1mcigoLjIyMjUw
NDUqcisuNzE2ODc4NippKy4wNjA2MTY5Km8pL3RyKTtyZXR1cm4gcj09PWkmJmk9PT1vP249ZT1h
OihuPWZyKCguNDM2MDc0NypyKy4zODUwNjQ5KmkrLjE0MzA4MDQqbykvSmUpLGU9ZnIoKC4wMTM5
MzIyKnIrLjA5NzEwNDUqaSsuNzE0MTczMypvKS9ucikpLG5ldyBjcigxMTYqYS0xNiw1MDAqKG4t
YSksMjAwKihhLWUpLHQub3BhY2l0eSl9ZnVuY3Rpb24gdXIodCxuLGUscil7cmV0dXJuIDE9PT1h
cmd1bWVudHMubGVuZ3RoP2FyKHQpOm5ldyBjcih0LG4sZSxudWxsPT1yPzE6cil9ZnVuY3Rpb24g
Y3IodCxuLGUscil7dGhpcy5sPSt0LHRoaXMuYT0rbix0aGlzLmI9K2UsdGhpcy5vcGFjaXR5PSty
fWZ1bmN0aW9uIGZyKHQpe3JldHVybiB0Pm9yP01hdGgucG93KHQsMS8zKTp0L2lyK2VyfWZ1bmN0
aW9uIHNyKHQpe3JldHVybiB0PnJyP3QqdCp0OmlyKih0LWVyKX1mdW5jdGlvbiBscih0KXtyZXR1
cm4gMjU1Kih0PD0uMDAzMTMwOD8xMi45Mip0OjEuMDU1Kk1hdGgucG93KHQsMS8yLjQpLS4wNTUp
fWZ1bmN0aW9uIGhyKHQpe3JldHVybih0Lz0yNTUpPD0uMDQwNDU/dC8xMi45MjpNYXRoLnBvdygo
dCsuMDU1KS8xLjA1NSwyLjQpfWZ1bmN0aW9uIGRyKHQpe2lmKHQgaW5zdGFuY2VvZiBncilyZXR1
cm4gbmV3IGdyKHQuaCx0LmMsdC5sLHQub3BhY2l0eSk7aWYodCBpbnN0YW5jZW9mIGNyfHwodD1h
cih0KSksMD09PXQuYSYmMD09PXQuYilyZXR1cm4gbmV3IGdyKE5hTiwwPHQubCYmdC5sPDEwMD8w
Ok5hTix0LmwsdC5vcGFjaXR5KTt2YXIgbj1NYXRoLmF0YW4yKHQuYix0LmEpKlFlO3JldHVybiBu
ZXcgZ3IobjwwP24rMzYwOm4sTWF0aC5zcXJ0KHQuYSp0LmErdC5iKnQuYiksdC5sLHQub3BhY2l0
eSl9ZnVuY3Rpb24gcHIodCxuLGUscil7cmV0dXJuIDE9PT1hcmd1bWVudHMubGVuZ3RoP2RyKHQp
Om5ldyBncih0LG4sZSxudWxsPT1yPzE6cil9ZnVuY3Rpb24gZ3IodCxuLGUscil7dGhpcy5oPSt0
LHRoaXMuYz0rbix0aGlzLmw9K2UsdGhpcy5vcGFjaXR5PStyfWZ1bmN0aW9uIHlyKHQpe2lmKGlz
TmFOKHQuaCkpcmV0dXJuIG5ldyBjcih0LmwsMCwwLHQub3BhY2l0eSk7dmFyIG49dC5oKktlO3Jl
dHVybiBuZXcgY3IodC5sLE1hdGguY29zKG4pKnQuYyxNYXRoLnNpbihuKSp0LmMsdC5vcGFjaXR5
KX1nZShjcix1cix5ZSh2ZSx7YnJpZ2h0ZXIodCl7cmV0dXJuIG5ldyBjcih0aGlzLmwrMTgqKG51
bGw9PXQ/MTp0KSx0aGlzLmEsdGhpcy5iLHRoaXMub3BhY2l0eSl9LGRhcmtlcih0KXtyZXR1cm4g
bmV3IGNyKHRoaXMubC0xOCoobnVsbD09dD8xOnQpLHRoaXMuYSx0aGlzLmIsdGhpcy5vcGFjaXR5
KX0scmdiKCl7dmFyIHQ9KHRoaXMubCsxNikvMTE2LG49aXNOYU4odGhpcy5hKT90OnQrdGhpcy5h
LzUwMCxlPWlzTmFOKHRoaXMuYik/dDp0LXRoaXMuYi8yMDA7cmV0dXJuIG5ldyBVZShscigzLjEz
Mzg1NjEqKG49SmUqc3IobikpLTEuNjE2ODY2NyoodD10cipzcih0KSktLjQ5MDYxNDYqKGU9bnIq
c3IoZSkpKSxscigtLjk3ODc2ODQqbisxLjkxNjE0MTUqdCsuMDMzNDU0KmUpLGxyKC4wNzE5NDUz
Km4tLjIyODk5MTQqdCsxLjQwNTI0MjcqZSksdGhpcy5vcGFjaXR5KX19KSksZ2UoZ3IscHIseWUo
dmUse2JyaWdodGVyKHQpe3JldHVybiBuZXcgZ3IodGhpcy5oLHRoaXMuYyx0aGlzLmwrMTgqKG51
bGw9PXQ/MTp0KSx0aGlzLm9wYWNpdHkpfSxkYXJrZXIodCl7cmV0dXJuIG5ldyBncih0aGlzLmgs
dGhpcy5jLHRoaXMubC0xOCoobnVsbD09dD8xOnQpLHRoaXMub3BhY2l0eSl9LHJnYigpe3JldHVy
biB5cih0aGlzKS5yZ2IoKX19KSk7dmFyIHZyPS0uMTQ4NjEsX3I9MS43ODI3Nyxicj0tLjI5MjI3
LG1yPS0uOTA2NDkseHI9MS45NzI5NCx3cj14ciptcixNcj14cipfcixUcj1fcipici1tcip2cjtm
dW5jdGlvbiBBcih0LG4sZSxyKXtyZXR1cm4gMT09PWFyZ3VtZW50cy5sZW5ndGg/ZnVuY3Rpb24o
dCl7aWYodCBpbnN0YW5jZW9mIFNyKXJldHVybiBuZXcgU3IodC5oLHQucyx0LmwsdC5vcGFjaXR5
KTt0IGluc3RhbmNlb2YgVWV8fCh0PUZlKHQpKTt2YXIgbj10LnIvMjU1LGU9dC5nLzI1NSxyPXQu
Yi8yNTUsaT0oVHIqcit3cipuLU1yKmUpLyhUcit3ci1Nciksbz1yLWksYT0oeHIqKGUtaSktYnIq
bykvbXIsdT1NYXRoLnNxcnQoYSphK28qbykvKHhyKmkqKDEtaSkpLGM9dT9NYXRoLmF0YW4yKGEs
bykqUWUtMTIwOk5hTjtyZXR1cm4gbmV3IFNyKGM8MD9jKzM2MDpjLHUsaSx0Lm9wYWNpdHkpfSh0
KTpuZXcgU3IodCxuLGUsbnVsbD09cj8xOnIpfWZ1bmN0aW9uIFNyKHQsbixlLHIpe3RoaXMuaD0r
dCx0aGlzLnM9K24sdGhpcy5sPStlLHRoaXMub3BhY2l0eT0rcn1mdW5jdGlvbiBFcih0LG4sZSxy
LGkpe3ZhciBvPXQqdCxhPW8qdDtyZXR1cm4oKDEtMyp0KzMqby1hKSpuKyg0LTYqbyszKmEpKmUr
KDErMyp0KzMqby0zKmEpKnIrYSppKS82fWZ1bmN0aW9uIE5yKHQpe3ZhciBuPXQubGVuZ3RoLTE7
cmV0dXJuIGZ1bmN0aW9uKGUpe3ZhciByPWU8PTA/ZT0wOmU+PTE/KGU9MSxuLTEpOk1hdGguZmxv
b3IoZSpuKSxpPXRbcl0sbz10W3IrMV0sYT1yPjA/dFtyLTFdOjIqaS1vLHU9cjxuLTE/dFtyKzJd
OjIqby1pO3JldHVybiBFcigoZS1yL24pKm4sYSxpLG8sdSl9fWZ1bmN0aW9uIGtyKHQpe3ZhciBu
PXQubGVuZ3RoO3JldHVybiBmdW5jdGlvbihlKXt2YXIgcj1NYXRoLmZsb29yKCgoZSU9MSk8MD8r
K2U6ZSkqbiksaT10WyhyK24tMSklbl0sbz10W3Ilbl0sYT10WyhyKzEpJW5dLHU9dFsocisyKSVu
XTtyZXR1cm4gRXIoKGUtci9uKSpuLGksbyxhLHUpfX1nZShTcixBcix5ZSh2ZSx7YnJpZ2h0ZXIo
dCl7cmV0dXJuIHQ9bnVsbD09dD9iZTpNYXRoLnBvdyhiZSx0KSxuZXcgU3IodGhpcy5oLHRoaXMu
cyx0aGlzLmwqdCx0aGlzLm9wYWNpdHkpfSxkYXJrZXIodCl7cmV0dXJuIHQ9bnVsbD09dD9fZTpN
YXRoLnBvdyhfZSx0KSxuZXcgU3IodGhpcy5oLHRoaXMucyx0aGlzLmwqdCx0aGlzLm9wYWNpdHkp
fSxyZ2IoKXt2YXIgdD1pc05hTih0aGlzLmgpPzA6KHRoaXMuaCsxMjApKktlLG49K3RoaXMubCxl
PWlzTmFOKHRoaXMucyk/MDp0aGlzLnMqbiooMS1uKSxyPU1hdGguY29zKHQpLGk9TWF0aC5zaW4o
dCk7cmV0dXJuIG5ldyBVZSgyNTUqKG4rZSoodnIqcitfcippKSksMjU1KihuK2UqKGJyKnIrbXIq
aSkpLDI1NSoobitlKih4cipyKSksdGhpcy5vcGFjaXR5KX19KSk7dmFyIENyPXQ9PigpPT50O2Z1
bmN0aW9uIFByKHQsbil7cmV0dXJuIGZ1bmN0aW9uKGUpe3JldHVybiB0K2Uqbn19ZnVuY3Rpb24g
enIodCxuKXt2YXIgZT1uLXQ7cmV0dXJuIGU/UHIodCxlPjE4MHx8ZTwtMTgwP2UtMzYwKk1hdGgu
cm91bmQoZS8zNjApOmUpOkNyKGlzTmFOKHQpP246dCl9ZnVuY3Rpb24gJHIodCl7cmV0dXJuIDE9
PSh0PSt0KT9EcjpmdW5jdGlvbihuLGUpe3JldHVybiBlLW4/ZnVuY3Rpb24odCxuLGUpe3JldHVy
biB0PU1hdGgucG93KHQsZSksbj1NYXRoLnBvdyhuLGUpLXQsZT0xL2UsZnVuY3Rpb24ocil7cmV0
dXJuIE1hdGgucG93KHQrcipuLGUpfX0obixlLHQpOkNyKGlzTmFOKG4pP2U6bil9fWZ1bmN0aW9u
IERyKHQsbil7dmFyIGU9bi10O3JldHVybiBlP1ByKHQsZSk6Q3IoaXNOYU4odCk/bjp0KX12YXIg
UnI9ZnVuY3Rpb24gdChuKXt2YXIgZT0kcihuKTtmdW5jdGlvbiByKHQsbil7dmFyIHI9ZSgodD1x
ZSh0KSkuciwobj1xZShuKSkuciksaT1lKHQuZyxuLmcpLG89ZSh0LmIsbi5iKSxhPURyKHQub3Bh
Y2l0eSxuLm9wYWNpdHkpO3JldHVybiBmdW5jdGlvbihuKXtyZXR1cm4gdC5yPXIobiksdC5nPWko
biksdC5iPW8obiksdC5vcGFjaXR5PWEobiksdCsiIn19cmV0dXJuIHIuZ2FtbWE9dCxyfSgxKTtm
dW5jdGlvbiBGcih0KXtyZXR1cm4gZnVuY3Rpb24obil7dmFyIGUscixpPW4ubGVuZ3RoLG89bmV3
IEFycmF5KGkpLGE9bmV3IEFycmF5KGkpLHU9bmV3IEFycmF5KGkpO2ZvcihlPTA7ZTxpOysrZSly
PXFlKG5bZV0pLG9bZV09ci5yfHwwLGFbZV09ci5nfHwwLHVbZV09ci5ifHwwO3JldHVybiBvPXQo
byksYT10KGEpLHU9dCh1KSxyLm9wYWNpdHk9MSxmdW5jdGlvbih0KXtyZXR1cm4gci5yPW8odCks
ci5nPWEodCksci5iPXUodCkscisiIn19fXZhciBxcj1GcihOciksVXI9RnIoa3IpO2Z1bmN0aW9u
IElyKHQsbil7bnx8KG49W10pO3ZhciBlLHI9dD9NYXRoLm1pbihuLmxlbmd0aCx0Lmxlbmd0aCk6
MCxpPW4uc2xpY2UoKTtyZXR1cm4gZnVuY3Rpb24obyl7Zm9yKGU9MDtlPHI7KytlKWlbZV09dFtl
XSooMS1vKStuW2VdKm87cmV0dXJuIGl9fWZ1bmN0aW9uIE9yKHQpe3JldHVybiBBcnJheUJ1ZmZl
ci5pc1ZpZXcodCkmJiEodCBpbnN0YW5jZW9mIERhdGFWaWV3KX1mdW5jdGlvbiBCcih0LG4pe3Zh
ciBlLHI9bj9uLmxlbmd0aDowLGk9dD9NYXRoLm1pbihyLHQubGVuZ3RoKTowLG89bmV3IEFycmF5
KGkpLGE9bmV3IEFycmF5KHIpO2ZvcihlPTA7ZTxpOysrZSlvW2VdPVZyKHRbZV0sbltlXSk7Zm9y
KDtlPHI7KytlKWFbZV09bltlXTtyZXR1cm4gZnVuY3Rpb24odCl7Zm9yKGU9MDtlPGk7KytlKWFb
ZV09b1tlXSh0KTtyZXR1cm4gYX19ZnVuY3Rpb24gWXIodCxuKXt2YXIgZT1uZXcgRGF0ZTtyZXR1
cm4gdD0rdCxuPStuLGZ1bmN0aW9uKHIpe3JldHVybiBlLnNldFRpbWUodCooMS1yKStuKnIpLGV9
fWZ1bmN0aW9uIExyKHQsbil7cmV0dXJuIHQ9K3Qsbj0rbixmdW5jdGlvbihlKXtyZXR1cm4gdCoo
MS1lKStuKmV9fWZ1bmN0aW9uIGpyKHQsbil7dmFyIGUscj17fSxpPXt9O2ZvcihlIGluIG51bGwh
PT10JiYib2JqZWN0Ij09dHlwZW9mIHR8fCh0PXt9KSxudWxsIT09biYmIm9iamVjdCI9PXR5cGVv
ZiBufHwobj17fSksbillIGluIHQ/cltlXT1Wcih0W2VdLG5bZV0pOmlbZV09bltlXTtyZXR1cm4g
ZnVuY3Rpb24odCl7Zm9yKGUgaW4gcilpW2VdPXJbZV0odCk7cmV0dXJuIGl9fXZhciBIcj0vWy0r
XT8oPzpcZCtcLj9cZCp8XC4/XGQrKSg/OltlRV1bLStdP1xkKyk/L2csWHI9bmV3IFJlZ0V4cChI
ci5zb3VyY2UsImciKTtmdW5jdGlvbiBHcih0LG4pe3ZhciBlLHIsaSxvPUhyLmxhc3RJbmRleD1Y
ci5sYXN0SW5kZXg9MCxhPS0xLHU9W10sYz1bXTtmb3IodCs9IiIsbis9IiI7KGU9SHIuZXhlYyh0
KSkmJihyPVhyLmV4ZWMobikpOykoaT1yLmluZGV4KT5vJiYoaT1uLnNsaWNlKG8saSksdVthXT91
W2FdKz1pOnVbKythXT1pKSwoZT1lWzBdKT09PShyPXJbMF0pP3VbYV0/dVthXSs9cjp1WysrYV09
cjoodVsrK2FdPW51bGwsYy5wdXNoKHtpOmEseDpMcihlLHIpfSkpLG89WHIubGFzdEluZGV4O3Jl
dHVybiBvPG4ubGVuZ3RoJiYoaT1uLnNsaWNlKG8pLHVbYV0/dVthXSs9aTp1WysrYV09aSksdS5s
ZW5ndGg8Mj9jWzBdP2Z1bmN0aW9uKHQpe3JldHVybiBmdW5jdGlvbihuKXtyZXR1cm4gdChuKSsi
In19KGNbMF0ueCk6ZnVuY3Rpb24odCl7cmV0dXJuIGZ1bmN0aW9uKCl7cmV0dXJuIHR9fShuKToo
bj1jLmxlbmd0aCxmdW5jdGlvbih0KXtmb3IodmFyIGUscj0wO3I8bjsrK3IpdVsoZT1jW3JdKS5p
XT1lLngodCk7cmV0dXJuIHUuam9pbigiIil9KX1mdW5jdGlvbiBWcih0LG4pe3ZhciBlLHI9dHlw
ZW9mIG47cmV0dXJuIG51bGw9PW58fCJib29sZWFuIj09PXI/Q3Iobik6KCJudW1iZXIiPT09cj9M
cjoic3RyaW5nIj09PXI/KGU9JGUobikpPyhuPWUsUnIpOkdyOm4gaW5zdGFuY2VvZiAkZT9Scjpu
IGluc3RhbmNlb2YgRGF0ZT9ZcjpPcihuKT9JcjpBcnJheS5pc0FycmF5KG4pP0JyOiJmdW5jdGlv
biIhPXR5cGVvZiBuLnZhbHVlT2YmJiJmdW5jdGlvbiIhPXR5cGVvZiBuLnRvU3RyaW5nfHxpc05h
TihuKT9qcjpMcikodCxuKX1mdW5jdGlvbiBXcih0LG4pe3JldHVybiB0PSt0LG49K24sZnVuY3Rp
b24oZSl7cmV0dXJuIE1hdGgucm91bmQodCooMS1lKStuKmUpfX12YXIgWnIsS3I9MTgwL01hdGgu
UEksUXI9e3RyYW5zbGF0ZVg6MCx0cmFuc2xhdGVZOjAscm90YXRlOjAsc2tld1g6MCxzY2FsZVg6
MSxzY2FsZVk6MX07ZnVuY3Rpb24gSnIodCxuLGUscixpLG8pe3ZhciBhLHUsYztyZXR1cm4oYT1N
YXRoLnNxcnQodCp0K24qbikpJiYodC89YSxuLz1hKSwoYz10KmUrbipyKSYmKGUtPXQqYyxyLT1u
KmMpLCh1PU1hdGguc3FydChlKmUrcipyKSkmJihlLz11LHIvPXUsYy89dSksdCpyPG4qZSYmKHQ9
LXQsbj0tbixjPS1jLGE9LWEpLHt0cmFuc2xhdGVYOmksdHJhbnNsYXRlWTpvLHJvdGF0ZTpNYXRo
LmF0YW4yKG4sdCkqS3Isc2tld1g6TWF0aC5hdGFuKGMpKktyLHNjYWxlWDphLHNjYWxlWTp1fX1m
dW5jdGlvbiB0aSh0LG4sZSxyKXtmdW5jdGlvbiBpKHQpe3JldHVybiB0Lmxlbmd0aD90LnBvcCgp
KyIgIjoiIn1yZXR1cm4gZnVuY3Rpb24obyxhKXt2YXIgdT1bXSxjPVtdO3JldHVybiBvPXQobyks
YT10KGEpLGZ1bmN0aW9uKHQscixpLG8sYSx1KXtpZih0IT09aXx8ciE9PW8pe3ZhciBjPWEucHVz
aCgidHJhbnNsYXRlKCIsbnVsbCxuLG51bGwsZSk7dS5wdXNoKHtpOmMtNCx4OkxyKHQsaSl9LHtp
OmMtMix4OkxyKHIsbyl9KX1lbHNlKGl8fG8pJiZhLnB1c2goInRyYW5zbGF0ZSgiK2krbitvK2Up
fShvLnRyYW5zbGF0ZVgsby50cmFuc2xhdGVZLGEudHJhbnNsYXRlWCxhLnRyYW5zbGF0ZVksdSxj
KSxmdW5jdGlvbih0LG4sZSxvKXt0IT09bj8odC1uPjE4MD9uKz0zNjA6bi10PjE4MCYmKHQrPTM2
MCksby5wdXNoKHtpOmUucHVzaChpKGUpKyJyb3RhdGUoIixudWxsLHIpLTIseDpMcih0LG4pfSkp
Om4mJmUucHVzaChpKGUpKyJyb3RhdGUoIituK3IpfShvLnJvdGF0ZSxhLnJvdGF0ZSx1LGMpLGZ1
bmN0aW9uKHQsbixlLG8pe3QhPT1uP28ucHVzaCh7aTplLnB1c2goaShlKSsic2tld1goIixudWxs
LHIpLTIseDpMcih0LG4pfSk6biYmZS5wdXNoKGkoZSkrInNrZXdYKCIrbityKX0oby5za2V3WCxh
LnNrZXdYLHUsYyksZnVuY3Rpb24odCxuLGUscixvLGEpe2lmKHQhPT1lfHxuIT09cil7dmFyIHU9
by5wdXNoKGkobykrInNjYWxlKCIsbnVsbCwiLCIsbnVsbCwiKSIpO2EucHVzaCh7aTp1LTQseDpM
cih0LGUpfSx7aTp1LTIseDpMcihuLHIpfSl9ZWxzZSAxPT09ZSYmMT09PXJ8fG8ucHVzaChpKG8p
KyJzY2FsZSgiK2UrIiwiK3IrIikiKX0oby5zY2FsZVgsby5zY2FsZVksYS5zY2FsZVgsYS5zY2Fs
ZVksdSxjKSxvPWE9bnVsbCxmdW5jdGlvbih0KXtmb3IodmFyIG4sZT0tMSxyPWMubGVuZ3RoOysr
ZTxyOyl1WyhuPWNbZV0pLmldPW4ueCh0KTtyZXR1cm4gdS5qb2luKCIiKX19fXZhciBuaT10aSgo
ZnVuY3Rpb24odCl7Y29uc3Qgbj1uZXcoImZ1bmN0aW9uIj09dHlwZW9mIERPTU1hdHJpeD9ET01N
YXRyaXg6V2ViS2l0Q1NTTWF0cml4KSh0KyIiKTtyZXR1cm4gbi5pc0lkZW50aXR5P1FyOkpyKG4u
YSxuLmIsbi5jLG4uZCxuLmUsbi5mKX0pLCJweCwgIiwicHgpIiwiZGVnKSIpLGVpPXRpKChmdW5j
dGlvbih0KXtyZXR1cm4gbnVsbD09dD9RcjooWnJ8fChacj1kb2N1bWVudC5jcmVhdGVFbGVtZW50
TlMoImh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiwiZyIpKSxaci5zZXRBdHRyaWJ1dGUoInRy
YW5zZm9ybSIsdCksKHQ9WnIudHJhbnNmb3JtLmJhc2VWYWwuY29uc29saWRhdGUoKSk/SnIoKHQ9
dC5tYXRyaXgpLmEsdC5iLHQuYyx0LmQsdC5lLHQuZik6UXIpfSksIiwgIiwiKSIsIikiKTtmdW5j
dGlvbiByaSh0KXtyZXR1cm4oKHQ9TWF0aC5leHAodCkpKzEvdCkvMn12YXIgaWk9ZnVuY3Rpb24g
dChuLGUscil7ZnVuY3Rpb24gaSh0LGkpe3ZhciBvLGEsdT10WzBdLGM9dFsxXSxmPXRbMl0scz1p
WzBdLGw9aVsxXSxoPWlbMl0sZD1zLXUscD1sLWMsZz1kKmQrcCpwO2lmKGc8MWUtMTIpYT1NYXRo
LmxvZyhoL2YpL24sbz1mdW5jdGlvbih0KXtyZXR1cm5bdSt0KmQsYyt0KnAsZipNYXRoLmV4cChu
KnQqYSldfTtlbHNle3ZhciB5PU1hdGguc3FydChnKSx2PShoKmgtZipmK3IqZykvKDIqZiplKnkp
LF89KGgqaC1mKmYtcipnKS8oMipoKmUqeSksYj1NYXRoLmxvZyhNYXRoLnNxcnQodip2KzEpLXYp
LG09TWF0aC5sb2coTWF0aC5zcXJ0KF8qXysxKS1fKTthPShtLWIpL24sbz1mdW5jdGlvbih0KXt2
YXIgcj10KmEsaT1yaShiKSxvPWYvKGUqeSkqKGkqZnVuY3Rpb24odCl7cmV0dXJuKCh0PU1hdGgu
ZXhwKDIqdCkpLTEpLyh0KzEpfShuKnIrYiktZnVuY3Rpb24odCl7cmV0dXJuKCh0PU1hdGguZXhw
KHQpKS0xL3QpLzJ9KGIpKTtyZXR1cm5bdStvKmQsYytvKnAsZippL3JpKG4qcitiKV19fXJldHVy
biBvLmR1cmF0aW9uPTFlMyphKm4vTWF0aC5TUVJUMixvfXJldHVybiBpLnJobz1mdW5jdGlvbihu
KXt2YXIgZT1NYXRoLm1heCguMDAxLCtuKSxyPWUqZTtyZXR1cm4gdChlLHIscipyKX0saX0oTWF0
aC5TUVJUMiwyLDQpO2Z1bmN0aW9uIG9pKHQpe3JldHVybiBmdW5jdGlvbihuLGUpe3ZhciByPXQo
KG49WGUobikpLmgsKGU9WGUoZSkpLmgpLGk9RHIobi5zLGUucyksbz1EcihuLmwsZS5sKSxhPURy
KG4ub3BhY2l0eSxlLm9wYWNpdHkpO3JldHVybiBmdW5jdGlvbih0KXtyZXR1cm4gbi5oPXIodCks
bi5zPWkodCksbi5sPW8odCksbi5vcGFjaXR5PWEodCksbisiIn19fXZhciBhaT1vaSh6ciksdWk9
b2koRHIpO2Z1bmN0aW9uIGNpKHQpe3JldHVybiBmdW5jdGlvbihuLGUpe3ZhciByPXQoKG49cHIo
bikpLmgsKGU9cHIoZSkpLmgpLGk9RHIobi5jLGUuYyksbz1EcihuLmwsZS5sKSxhPURyKG4ub3Bh
Y2l0eSxlLm9wYWNpdHkpO3JldHVybiBmdW5jdGlvbih0KXtyZXR1cm4gbi5oPXIodCksbi5jPWko
dCksbi5sPW8odCksbi5vcGFjaXR5PWEodCksbisiIn19fXZhciBmaT1jaSh6ciksc2k9Y2koRHIp
O2Z1bmN0aW9uIGxpKHQpe3JldHVybiBmdW5jdGlvbiBuKGUpe2Z1bmN0aW9uIHIobixyKXt2YXIg
aT10KChuPUFyKG4pKS5oLChyPUFyKHIpKS5oKSxvPURyKG4ucyxyLnMpLGE9RHIobi5sLHIubCks
dT1EcihuLm9wYWNpdHksci5vcGFjaXR5KTtyZXR1cm4gZnVuY3Rpb24odCl7cmV0dXJuIG4uaD1p
KHQpLG4ucz1vKHQpLG4ubD1hKE1hdGgucG93KHQsZSkpLG4ub3BhY2l0eT11KHQpLG4rIiJ9fXJl
dHVybiBlPStlLHIuZ2FtbWE9bixyfSgxKX12YXIgaGk9bGkoenIpLGRpPWxpKERyKTtmdW5jdGlv
biBwaSh0LG4pe3ZvaWQgMD09PW4mJihuPXQsdD1Wcik7Zm9yKHZhciBlPTAscj1uLmxlbmd0aC0x
LGk9blswXSxvPW5ldyBBcnJheShyPDA/MDpyKTtlPHI7KW9bZV09dChpLGk9blsrK2VdKTtyZXR1
cm4gZnVuY3Rpb24odCl7dmFyIG49TWF0aC5tYXgoMCxNYXRoLm1pbihyLTEsTWF0aC5mbG9vcih0
Kj1yKSkpO3JldHVybiBvW25dKHQtbil9fXZhciBnaSx5aSx2aT0wLF9pPTAsYmk9MCxtaT0xZTMs
eGk9MCx3aT0wLE1pPTAsVGk9Im9iamVjdCI9PXR5cGVvZiBwZXJmb3JtYW5jZSYmcGVyZm9ybWFu
Y2Uubm93P3BlcmZvcm1hbmNlOkRhdGUsQWk9Im9iamVjdCI9PXR5cGVvZiB3aW5kb3cmJndpbmRv
dy5yZXF1ZXN0QW5pbWF0aW9uRnJhbWU/d2luZG93LnJlcXVlc3RBbmltYXRpb25GcmFtZS5iaW5k
KHdpbmRvdyk6ZnVuY3Rpb24odCl7c2V0VGltZW91dCh0LDE3KX07ZnVuY3Rpb24gU2koKXtyZXR1
cm4gd2l8fChBaShFaSksd2k9VGkubm93KCkrTWkpfWZ1bmN0aW9uIEVpKCl7d2k9MH1mdW5jdGlv
biBOaSgpe3RoaXMuX2NhbGw9dGhpcy5fdGltZT10aGlzLl9uZXh0PW51bGx9ZnVuY3Rpb24ga2ko
dCxuLGUpe3ZhciByPW5ldyBOaTtyZXR1cm4gci5yZXN0YXJ0KHQsbixlKSxyfWZ1bmN0aW9uIENp
KCl7U2koKSwrK3ZpO2Zvcih2YXIgdCxuPWdpO247KSh0PXdpLW4uX3RpbWUpPj0wJiZuLl9jYWxs
LmNhbGwodm9pZCAwLHQpLG49bi5fbmV4dDstLXZpfWZ1bmN0aW9uIFBpKCl7d2k9KHhpPVRpLm5v
dygpKStNaSx2aT1faT0wO3RyeXtDaSgpfWZpbmFsbHl7dmk9MCxmdW5jdGlvbigpe3ZhciB0LG4s
ZT1naSxyPTEvMDtmb3IoO2U7KWUuX2NhbGw/KHI+ZS5fdGltZSYmKHI9ZS5fdGltZSksdD1lLGU9
ZS5fbmV4dCk6KG49ZS5fbmV4dCxlLl9uZXh0PW51bGwsZT10P3QuX25leHQ9bjpnaT1uKTt5aT10
LCRpKHIpfSgpLHdpPTB9fWZ1bmN0aW9uIHppKCl7dmFyIHQ9VGkubm93KCksbj10LXhpO24+bWkm
JihNaS09bix4aT10KX1mdW5jdGlvbiAkaSh0KXt2aXx8KF9pJiYoX2k9Y2xlYXJUaW1lb3V0KF9p
KSksdC13aT4yND8odDwxLzAmJihfaT1zZXRUaW1lb3V0KFBpLHQtVGkubm93KCktTWkpKSxiaSYm
KGJpPWNsZWFySW50ZXJ2YWwoYmkpKSk6KGJpfHwoeGk9VGkubm93KCksYmk9c2V0SW50ZXJ2YWwo
emksbWkpKSx2aT0xLEFpKFBpKSkpfWZ1bmN0aW9uIERpKHQsbixlKXt2YXIgcj1uZXcgTmk7cmV0
dXJuIG49bnVsbD09bj8wOituLHIucmVzdGFydCgoZT0+e3Iuc3RvcCgpLHQoZStuKX0pLG4sZSks
cn1OaS5wcm90b3R5cGU9a2kucHJvdG90eXBlPXtjb25zdHJ1Y3RvcjpOaSxyZXN0YXJ0OmZ1bmN0
aW9uKHQsbixlKXtpZigiZnVuY3Rpb24iIT10eXBlb2YgdCl0aHJvdyBuZXcgVHlwZUVycm9yKCJj
YWxsYmFjayBpcyBub3QgYSBmdW5jdGlvbiIpO2U9KG51bGw9PWU/U2koKTorZSkrKG51bGw9PW4/
MDorbiksdGhpcy5fbmV4dHx8eWk9PT10aGlzfHwoeWk/eWkuX25leHQ9dGhpczpnaT10aGlzLHlp
PXRoaXMpLHRoaXMuX2NhbGw9dCx0aGlzLl90aW1lPWUsJGkoKX0sc3RvcDpmdW5jdGlvbigpe3Ro
aXMuX2NhbGwmJih0aGlzLl9jYWxsPW51bGwsdGhpcy5fdGltZT0xLzAsJGkoKSl9fTt2YXIgUmk9
RHQoInN0YXJ0IiwiZW5kIiwiY2FuY2VsIiwiaW50ZXJydXB0IiksRmk9W10scWk9MCxVaT0xLElp
PTIsT2k9MyxCaT00LFlpPTUsTGk9NjtmdW5jdGlvbiBqaSh0LG4sZSxyLGksbyl7dmFyIGE9dC5f
X3RyYW5zaXRpb247aWYoYSl7aWYoZSBpbiBhKXJldHVybn1lbHNlIHQuX190cmFuc2l0aW9uPXt9
OyFmdW5jdGlvbih0LG4sZSl7dmFyIHIsaT10Ll9fdHJhbnNpdGlvbjtmdW5jdGlvbiBvKHQpe2Uu
c3RhdGU9VWksZS50aW1lci5yZXN0YXJ0KGEsZS5kZWxheSxlLnRpbWUpLGUuZGVsYXk8PXQmJmEo
dC1lLmRlbGF5KX1mdW5jdGlvbiBhKG8pe3ZhciBmLHMsbCxoO2lmKGUuc3RhdGUhPT1VaSlyZXR1
cm4gYygpO2ZvcihmIGluIGkpaWYoKGg9aVtmXSkubmFtZT09PWUubmFtZSl7aWYoaC5zdGF0ZT09
PU9pKXJldHVybiBEaShhKTtoLnN0YXRlPT09Qmk/KGguc3RhdGU9TGksaC50aW1lci5zdG9wKCks
aC5vbi5jYWxsKCJpbnRlcnJ1cHQiLHQsdC5fX2RhdGFfXyxoLmluZGV4LGguZ3JvdXApLGRlbGV0
ZSBpW2ZdKTorZjxuJiYoaC5zdGF0ZT1MaSxoLnRpbWVyLnN0b3AoKSxoLm9uLmNhbGwoImNhbmNl
bCIsdCx0Ll9fZGF0YV9fLGguaW5kZXgsaC5ncm91cCksZGVsZXRlIGlbZl0pfWlmKERpKChmdW5j
dGlvbigpe2Uuc3RhdGU9PT1PaSYmKGUuc3RhdGU9QmksZS50aW1lci5yZXN0YXJ0KHUsZS5kZWxh
eSxlLnRpbWUpLHUobykpfSkpLGUuc3RhdGU9SWksZS5vbi5jYWxsKCJzdGFydCIsdCx0Ll9fZGF0
YV9fLGUuaW5kZXgsZS5ncm91cCksZS5zdGF0ZT09PUlpKXtmb3IoZS5zdGF0ZT1PaSxyPW5ldyBB
cnJheShsPWUudHdlZW4ubGVuZ3RoKSxmPTAscz0tMTtmPGw7KytmKShoPWUudHdlZW5bZl0udmFs
dWUuY2FsbCh0LHQuX19kYXRhX18sZS5pbmRleCxlLmdyb3VwKSkmJihyWysrc109aCk7ci5sZW5n
dGg9cysxfX1mdW5jdGlvbiB1KG4pe2Zvcih2YXIgaT1uPGUuZHVyYXRpb24/ZS5lYXNlLmNhbGwo
bnVsbCxuL2UuZHVyYXRpb24pOihlLnRpbWVyLnJlc3RhcnQoYyksZS5zdGF0ZT1ZaSwxKSxvPS0x
LGE9ci5sZW5ndGg7KytvPGE7KXJbb10uY2FsbCh0LGkpO2Uuc3RhdGU9PT1ZaSYmKGUub24uY2Fs
bCgiZW5kIix0LHQuX19kYXRhX18sZS5pbmRleCxlLmdyb3VwKSxjKCkpfWZ1bmN0aW9uIGMoKXtm
b3IodmFyIHIgaW4gZS5zdGF0ZT1MaSxlLnRpbWVyLnN0b3AoKSxkZWxldGUgaVtuXSxpKXJldHVy
bjtkZWxldGUgdC5fX3RyYW5zaXRpb259aVtuXT1lLGUudGltZXI9a2kobywwLGUudGltZSl9KHQs
ZSx7bmFtZTpuLGluZGV4OnIsZ3JvdXA6aSxvbjpSaSx0d2VlbjpGaSx0aW1lOm8udGltZSxkZWxh
eTpvLmRlbGF5LGR1cmF0aW9uOm8uZHVyYXRpb24sZWFzZTpvLmVhc2UsdGltZXI6bnVsbCxzdGF0
ZTpxaX0pfWZ1bmN0aW9uIEhpKHQsbil7dmFyIGU9R2kodCxuKTtpZihlLnN0YXRlPnFpKXRocm93
IG5ldyBFcnJvcigidG9vIGxhdGU7IGFscmVhZHkgc2NoZWR1bGVkIik7cmV0dXJuIGV9ZnVuY3Rp
b24gWGkodCxuKXt2YXIgZT1HaSh0LG4pO2lmKGUuc3RhdGU+T2kpdGhyb3cgbmV3IEVycm9yKCJ0
b28gbGF0ZTsgYWxyZWFkeSBydW5uaW5nIik7cmV0dXJuIGV9ZnVuY3Rpb24gR2kodCxuKXt2YXIg
ZT10Ll9fdHJhbnNpdGlvbjtpZighZXx8IShlPWVbbl0pKXRocm93IG5ldyBFcnJvcigidHJhbnNp
dGlvbiBub3QgZm91bmQiKTtyZXR1cm4gZX1mdW5jdGlvbiBWaSh0LG4pe3ZhciBlLHIsaSxvPXQu
X190cmFuc2l0aW9uLGE9ITA7aWYobyl7Zm9yKGkgaW4gbj1udWxsPT1uP251bGw6bisiIixvKShl
PW9baV0pLm5hbWU9PT1uPyhyPWUuc3RhdGU+SWkmJmUuc3RhdGU8WWksZS5zdGF0ZT1MaSxlLnRp
bWVyLnN0b3AoKSxlLm9uLmNhbGwocj8iaW50ZXJydXB0IjoiY2FuY2VsIix0LHQuX19kYXRhX18s
ZS5pbmRleCxlLmdyb3VwKSxkZWxldGUgb1tpXSk6YT0hMTthJiZkZWxldGUgdC5fX3RyYW5zaXRp
b259fWZ1bmN0aW9uIFdpKHQsbil7dmFyIGUscjtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgaT1YaSh0
aGlzLHQpLG89aS50d2VlbjtpZihvIT09ZSlmb3IodmFyIGE9MCx1PShyPWU9bykubGVuZ3RoO2E8
dTsrK2EpaWYoclthXS5uYW1lPT09bil7KHI9ci5zbGljZSgpKS5zcGxpY2UoYSwxKTticmVha31p
LnR3ZWVuPXJ9fWZ1bmN0aW9uIFppKHQsbixlKXt2YXIgcixpO2lmKCJmdW5jdGlvbiIhPXR5cGVv
ZiBlKXRocm93IG5ldyBFcnJvcjtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgbz1YaSh0aGlzLHQpLGE9
by50d2VlbjtpZihhIT09cil7aT0ocj1hKS5zbGljZSgpO2Zvcih2YXIgdT17bmFtZTpuLHZhbHVl
OmV9LGM9MCxmPWkubGVuZ3RoO2M8ZjsrK2MpaWYoaVtjXS5uYW1lPT09bil7aVtjXT11O2JyZWFr
fWM9PT1mJiZpLnB1c2godSl9by50d2Vlbj1pfX1mdW5jdGlvbiBLaSh0LG4sZSl7dmFyIHI9dC5f
aWQ7cmV0dXJuIHQuZWFjaCgoZnVuY3Rpb24oKXt2YXIgdD1YaSh0aGlzLHIpOyh0LnZhbHVlfHwo
dC52YWx1ZT17fSkpW25dPWUuYXBwbHkodGhpcyxhcmd1bWVudHMpfSkpLGZ1bmN0aW9uKHQpe3Jl
dHVybiBHaSh0LHIpLnZhbHVlW25dfX1mdW5jdGlvbiBRaSh0LG4pe3ZhciBlO3JldHVybigibnVt
YmVyIj09dHlwZW9mIG4/THI6biBpbnN0YW5jZW9mICRlP1JyOihlPSRlKG4pKT8obj1lLFJyKTpH
cikodCxuKX1mdW5jdGlvbiBKaSh0KXtyZXR1cm4gZnVuY3Rpb24oKXt0aGlzLnJlbW92ZUF0dHJp
YnV0ZSh0KX19ZnVuY3Rpb24gdG8odCl7cmV0dXJuIGZ1bmN0aW9uKCl7dGhpcy5yZW1vdmVBdHRy
aWJ1dGVOUyh0LnNwYWNlLHQubG9jYWwpfX1mdW5jdGlvbiBubyh0LG4sZSl7dmFyIHIsaSxvPWUr
IiI7cmV0dXJuIGZ1bmN0aW9uKCl7dmFyIGE9dGhpcy5nZXRBdHRyaWJ1dGUodCk7cmV0dXJuIGE9
PT1vP251bGw6YT09PXI/aTppPW4ocj1hLGUpfX1mdW5jdGlvbiBlbyh0LG4sZSl7dmFyIHIsaSxv
PWUrIiI7cmV0dXJuIGZ1bmN0aW9uKCl7dmFyIGE9dGhpcy5nZXRBdHRyaWJ1dGVOUyh0LnNwYWNl
LHQubG9jYWwpO3JldHVybiBhPT09bz9udWxsOmE9PT1yP2k6aT1uKHI9YSxlKX19ZnVuY3Rpb24g
cm8odCxuLGUpe3ZhciByLGksbztyZXR1cm4gZnVuY3Rpb24oKXt2YXIgYSx1LGM9ZSh0aGlzKTtp
ZihudWxsIT1jKXJldHVybihhPXRoaXMuZ2V0QXR0cmlidXRlKHQpKT09PSh1PWMrIiIpP251bGw6
YT09PXImJnU9PT1pP286KGk9dSxvPW4ocj1hLGMpKTt0aGlzLnJlbW92ZUF0dHJpYnV0ZSh0KX19
ZnVuY3Rpb24gaW8odCxuLGUpe3ZhciByLGksbztyZXR1cm4gZnVuY3Rpb24oKXt2YXIgYSx1LGM9
ZSh0aGlzKTtpZihudWxsIT1jKXJldHVybihhPXRoaXMuZ2V0QXR0cmlidXRlTlModC5zcGFjZSx0
LmxvY2FsKSk9PT0odT1jKyIiKT9udWxsOmE9PT1yJiZ1PT09aT9vOihpPXUsbz1uKHI9YSxjKSk7
dGhpcy5yZW1vdmVBdHRyaWJ1dGVOUyh0LnNwYWNlLHQubG9jYWwpfX1mdW5jdGlvbiBvbyh0LG4p
e3ZhciBlLHI7ZnVuY3Rpb24gaSgpe3ZhciBpPW4uYXBwbHkodGhpcyxhcmd1bWVudHMpO3JldHVy
biBpIT09ciYmKGU9KHI9aSkmJmZ1bmN0aW9uKHQsbil7cmV0dXJuIGZ1bmN0aW9uKGUpe3RoaXMu
c2V0QXR0cmlidXRlTlModC5zcGFjZSx0LmxvY2FsLG4uY2FsbCh0aGlzLGUpKX19KHQsaSkpLGV9
cmV0dXJuIGkuX3ZhbHVlPW4saX1mdW5jdGlvbiBhbyh0LG4pe3ZhciBlLHI7ZnVuY3Rpb24gaSgp
e3ZhciBpPW4uYXBwbHkodGhpcyxhcmd1bWVudHMpO3JldHVybiBpIT09ciYmKGU9KHI9aSkmJmZ1
bmN0aW9uKHQsbil7cmV0dXJuIGZ1bmN0aW9uKGUpe3RoaXMuc2V0QXR0cmlidXRlKHQsbi5jYWxs
KHRoaXMsZSkpfX0odCxpKSksZX1yZXR1cm4gaS5fdmFsdWU9bixpfWZ1bmN0aW9uIHVvKHQsbil7
cmV0dXJuIGZ1bmN0aW9uKCl7SGkodGhpcyx0KS5kZWxheT0rbi5hcHBseSh0aGlzLGFyZ3VtZW50
cyl9fWZ1bmN0aW9uIGNvKHQsbil7cmV0dXJuIG49K24sZnVuY3Rpb24oKXtIaSh0aGlzLHQpLmRl
bGF5PW59fWZ1bmN0aW9uIGZvKHQsbil7cmV0dXJuIGZ1bmN0aW9uKCl7WGkodGhpcyx0KS5kdXJh
dGlvbj0rbi5hcHBseSh0aGlzLGFyZ3VtZW50cyl9fWZ1bmN0aW9uIHNvKHQsbil7cmV0dXJuIG49
K24sZnVuY3Rpb24oKXtYaSh0aGlzLHQpLmR1cmF0aW9uPW59fXZhciBsbz1abi5wcm90b3R5cGUu
Y29uc3RydWN0b3I7ZnVuY3Rpb24gaG8odCl7cmV0dXJuIGZ1bmN0aW9uKCl7dGhpcy5zdHlsZS5y
ZW1vdmVQcm9wZXJ0eSh0KX19dmFyIHBvPTA7ZnVuY3Rpb24gZ28odCxuLGUscil7dGhpcy5fZ3Jv
dXBzPXQsdGhpcy5fcGFyZW50cz1uLHRoaXMuX25hbWU9ZSx0aGlzLl9pZD1yfWZ1bmN0aW9uIHlv
KHQpe3JldHVybiBabigpLnRyYW5zaXRpb24odCl9ZnVuY3Rpb24gdm8oKXtyZXR1cm4rK3BvfXZh
ciBfbz1abi5wcm90b3R5cGU7Z28ucHJvdG90eXBlPXlvLnByb3RvdHlwZT17Y29uc3RydWN0b3I6
Z28sc2VsZWN0OmZ1bmN0aW9uKHQpe3ZhciBuPXRoaXMuX25hbWUsZT10aGlzLl9pZDsiZnVuY3Rp
b24iIT10eXBlb2YgdCYmKHQ9SHQodCkpO2Zvcih2YXIgcj10aGlzLl9ncm91cHMsaT1yLmxlbmd0
aCxvPW5ldyBBcnJheShpKSxhPTA7YTxpOysrYSlmb3IodmFyIHUsYyxmPXJbYV0scz1mLmxlbmd0
aCxsPW9bYV09bmV3IEFycmF5KHMpLGg9MDtoPHM7KytoKSh1PWZbaF0pJiYoYz10LmNhbGwodSx1
Ll9fZGF0YV9fLGgsZikpJiYoIl9fZGF0YV9fImluIHUmJihjLl9fZGF0YV9fPXUuX19kYXRhX18p
LGxbaF09YyxqaShsW2hdLG4sZSxoLGwsR2kodSxlKSkpO3JldHVybiBuZXcgZ28obyx0aGlzLl9w
YXJlbnRzLG4sZSl9LHNlbGVjdEFsbDpmdW5jdGlvbih0KXt2YXIgbj10aGlzLl9uYW1lLGU9dGhp
cy5faWQ7ImZ1bmN0aW9uIiE9dHlwZW9mIHQmJih0PVZ0KHQpKTtmb3IodmFyIHI9dGhpcy5fZ3Jv
dXBzLGk9ci5sZW5ndGgsbz1bXSxhPVtdLHU9MDt1PGk7Kyt1KWZvcih2YXIgYyxmPXJbdV0scz1m
Lmxlbmd0aCxsPTA7bDxzOysrbClpZihjPWZbbF0pe2Zvcih2YXIgaCxkPXQuY2FsbChjLGMuX19k
YXRhX18sbCxmKSxwPUdpKGMsZSksZz0wLHk9ZC5sZW5ndGg7Zzx5OysrZykoaD1kW2ddKSYmamko
aCxuLGUsZyxkLHApO28ucHVzaChkKSxhLnB1c2goYyl9cmV0dXJuIG5ldyBnbyhvLGEsbixlKX0s
c2VsZWN0Q2hpbGQ6X28uc2VsZWN0Q2hpbGQsc2VsZWN0Q2hpbGRyZW46X28uc2VsZWN0Q2hpbGRy
ZW4sZmlsdGVyOmZ1bmN0aW9uKHQpeyJmdW5jdGlvbiIhPXR5cGVvZiB0JiYodD1XdCh0KSk7Zm9y
KHZhciBuPXRoaXMuX2dyb3VwcyxlPW4ubGVuZ3RoLHI9bmV3IEFycmF5KGUpLGk9MDtpPGU7Kytp
KWZvcih2YXIgbyxhPW5baV0sdT1hLmxlbmd0aCxjPXJbaV09W10sZj0wO2Y8dTsrK2YpKG89YVtm
XSkmJnQuY2FsbChvLG8uX19kYXRhX18sZixhKSYmYy5wdXNoKG8pO3JldHVybiBuZXcgZ28ocix0
aGlzLl9wYXJlbnRzLHRoaXMuX25hbWUsdGhpcy5faWQpfSxtZXJnZTpmdW5jdGlvbih0KXtpZih0
Ll9pZCE9PXRoaXMuX2lkKXRocm93IG5ldyBFcnJvcjtmb3IodmFyIG49dGhpcy5fZ3JvdXBzLGU9
dC5fZ3JvdXBzLHI9bi5sZW5ndGgsaT1lLmxlbmd0aCxvPU1hdGgubWluKHIsaSksYT1uZXcgQXJy
YXkociksdT0wO3U8bzsrK3UpZm9yKHZhciBjLGY9blt1XSxzPWVbdV0sbD1mLmxlbmd0aCxoPWFb
dV09bmV3IEFycmF5KGwpLGQ9MDtkPGw7KytkKShjPWZbZF18fHNbZF0pJiYoaFtkXT1jKTtmb3Io
O3U8cjsrK3UpYVt1XT1uW3VdO3JldHVybiBuZXcgZ28oYSx0aGlzLl9wYXJlbnRzLHRoaXMuX25h
bWUsdGhpcy5faWQpfSxzZWxlY3Rpb246ZnVuY3Rpb24oKXtyZXR1cm4gbmV3IGxvKHRoaXMuX2dy
b3Vwcyx0aGlzLl9wYXJlbnRzKX0sdHJhbnNpdGlvbjpmdW5jdGlvbigpe2Zvcih2YXIgdD10aGlz
Ll9uYW1lLG49dGhpcy5faWQsZT12bygpLHI9dGhpcy5fZ3JvdXBzLGk9ci5sZW5ndGgsbz0wO288
aTsrK28pZm9yKHZhciBhLHU9cltvXSxjPXUubGVuZ3RoLGY9MDtmPGM7KytmKWlmKGE9dVtmXSl7
dmFyIHM9R2koYSxuKTtqaShhLHQsZSxmLHUse3RpbWU6cy50aW1lK3MuZGVsYXkrcy5kdXJhdGlv
bixkZWxheTowLGR1cmF0aW9uOnMuZHVyYXRpb24sZWFzZTpzLmVhc2V9KX1yZXR1cm4gbmV3IGdv
KHIsdGhpcy5fcGFyZW50cyx0LGUpfSxjYWxsOl9vLmNhbGwsbm9kZXM6X28ubm9kZXMsbm9kZTpf
by5ub2RlLHNpemU6X28uc2l6ZSxlbXB0eTpfby5lbXB0eSxlYWNoOl9vLmVhY2gsb246ZnVuY3Rp
b24odCxuKXt2YXIgZT10aGlzLl9pZDtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aDwyP0dpKHRoaXMu
bm9kZSgpLGUpLm9uLm9uKHQpOnRoaXMuZWFjaChmdW5jdGlvbih0LG4sZSl7dmFyIHIsaSxvPWZ1
bmN0aW9uKHQpe3JldHVybih0KyIiKS50cmltKCkuc3BsaXQoL158XHMrLykuZXZlcnkoKGZ1bmN0
aW9uKHQpe3ZhciBuPXQuaW5kZXhPZigiLiIpO3JldHVybiBuPj0wJiYodD10LnNsaWNlKDAsbikp
LCF0fHwic3RhcnQiPT09dH0pKX0obik/SGk6WGk7cmV0dXJuIGZ1bmN0aW9uKCl7dmFyIGE9byh0
aGlzLHQpLHU9YS5vbjt1IT09ciYmKGk9KHI9dSkuY29weSgpKS5vbihuLGUpLGEub249aX19KGUs
dCxuKSl9LGF0dHI6ZnVuY3Rpb24odCxuKXt2YXIgZT1PdCh0KSxyPSJ0cmFuc2Zvcm0iPT09ZT9l
aTpRaTtyZXR1cm4gdGhpcy5hdHRyVHdlZW4odCwiZnVuY3Rpb24iPT10eXBlb2Ygbj8oZS5sb2Nh
bD9pbzpybykoZSxyLEtpKHRoaXMsImF0dHIuIit0LG4pKTpudWxsPT1uPyhlLmxvY2FsP3RvOkpp
KShlKTooZS5sb2NhbD9lbzpubykoZSxyLG4pKX0sYXR0clR3ZWVuOmZ1bmN0aW9uKHQsbil7dmFy
IGU9ImF0dHIuIit0O2lmKGFyZ3VtZW50cy5sZW5ndGg8MilyZXR1cm4oZT10aGlzLnR3ZWVuKGUp
KSYmZS5fdmFsdWU7aWYobnVsbD09bilyZXR1cm4gdGhpcy50d2VlbihlLG51bGwpO2lmKCJmdW5j
dGlvbiIhPXR5cGVvZiBuKXRocm93IG5ldyBFcnJvcjt2YXIgcj1PdCh0KTtyZXR1cm4gdGhpcy50
d2VlbihlLChyLmxvY2FsP29vOmFvKShyLG4pKX0sc3R5bGU6ZnVuY3Rpb24odCxuLGUpe3ZhciBy
PSJ0cmFuc2Zvcm0iPT0odCs9IiIpP25pOlFpO3JldHVybiBudWxsPT1uP3RoaXMuc3R5bGVUd2Vl
bih0LGZ1bmN0aW9uKHQsbil7dmFyIGUscixpO3JldHVybiBmdW5jdGlvbigpe3ZhciBvPWJuKHRo
aXMsdCksYT0odGhpcy5zdHlsZS5yZW1vdmVQcm9wZXJ0eSh0KSxibih0aGlzLHQpKTtyZXR1cm4g
bz09PWE/bnVsbDpvPT09ZSYmYT09PXI/aTppPW4oZT1vLHI9YSl9fSh0LHIpKS5vbigiZW5kLnN0
eWxlLiIrdCxobyh0KSk6ImZ1bmN0aW9uIj09dHlwZW9mIG4/dGhpcy5zdHlsZVR3ZWVuKHQsZnVu
Y3Rpb24odCxuLGUpe3ZhciByLGksbztyZXR1cm4gZnVuY3Rpb24oKXt2YXIgYT1ibih0aGlzLHQp
LHU9ZSh0aGlzKSxjPXUrIiI7cmV0dXJuIG51bGw9PXUmJih0aGlzLnN0eWxlLnJlbW92ZVByb3Bl
cnR5KHQpLGM9dT1ibih0aGlzLHQpKSxhPT09Yz9udWxsOmE9PT1yJiZjPT09aT9vOihpPWMsbz1u
KHI9YSx1KSl9fSh0LHIsS2kodGhpcywic3R5bGUuIit0LG4pKSkuZWFjaChmdW5jdGlvbih0LG4p
e3ZhciBlLHIsaSxvLGE9InN0eWxlLiIrbix1PSJlbmQuIithO3JldHVybiBmdW5jdGlvbigpe3Zh
ciBjPVhpKHRoaXMsdCksZj1jLm9uLHM9bnVsbD09Yy52YWx1ZVthXT9vfHwobz1obyhuKSk6dm9p
ZCAwO2Y9PT1lJiZpPT09c3x8KHI9KGU9ZikuY29weSgpKS5vbih1LGk9cyksYy5vbj1yfX0odGhp
cy5faWQsdCkpOnRoaXMuc3R5bGVUd2Vlbih0LGZ1bmN0aW9uKHQsbixlKXt2YXIgcixpLG89ZSsi
IjtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgYT1ibih0aGlzLHQpO3JldHVybiBhPT09bz9udWxsOmE9
PT1yP2k6aT1uKHI9YSxlKX19KHQscixuKSxlKS5vbigiZW5kLnN0eWxlLiIrdCxudWxsKX0sc3R5
bGVUd2VlbjpmdW5jdGlvbih0LG4sZSl7dmFyIHI9InN0eWxlLiIrKHQrPSIiKTtpZihhcmd1bWVu
dHMubGVuZ3RoPDIpcmV0dXJuKHI9dGhpcy50d2VlbihyKSkmJnIuX3ZhbHVlO2lmKG51bGw9PW4p
cmV0dXJuIHRoaXMudHdlZW4ocixudWxsKTtpZigiZnVuY3Rpb24iIT10eXBlb2Ygbil0aHJvdyBu
ZXcgRXJyb3I7cmV0dXJuIHRoaXMudHdlZW4ocixmdW5jdGlvbih0LG4sZSl7dmFyIHIsaTtmdW5j
dGlvbiBvKCl7dmFyIG89bi5hcHBseSh0aGlzLGFyZ3VtZW50cyk7cmV0dXJuIG8hPT1pJiYocj0o
aT1vKSYmZnVuY3Rpb24odCxuLGUpe3JldHVybiBmdW5jdGlvbihyKXt0aGlzLnN0eWxlLnNldFBy
b3BlcnR5KHQsbi5jYWxsKHRoaXMsciksZSl9fSh0LG8sZSkpLHJ9cmV0dXJuIG8uX3ZhbHVlPW4s
b30odCxuLG51bGw9PWU/IiI6ZSkpfSx0ZXh0OmZ1bmN0aW9uKHQpe3JldHVybiB0aGlzLnR3ZWVu
KCJ0ZXh0IiwiZnVuY3Rpb24iPT10eXBlb2YgdD9mdW5jdGlvbih0KXtyZXR1cm4gZnVuY3Rpb24o
KXt2YXIgbj10KHRoaXMpO3RoaXMudGV4dENvbnRlbnQ9bnVsbD09bj8iIjpufX0oS2kodGhpcywi
dGV4dCIsdCkpOmZ1bmN0aW9uKHQpe3JldHVybiBmdW5jdGlvbigpe3RoaXMudGV4dENvbnRlbnQ9
dH19KG51bGw9PXQ/IiI6dCsiIikpfSx0ZXh0VHdlZW46ZnVuY3Rpb24odCl7dmFyIG49InRleHQi
O2lmKGFyZ3VtZW50cy5sZW5ndGg8MSlyZXR1cm4obj10aGlzLnR3ZWVuKG4pKSYmbi5fdmFsdWU7
aWYobnVsbD09dClyZXR1cm4gdGhpcy50d2VlbihuLG51bGwpO2lmKCJmdW5jdGlvbiIhPXR5cGVv
ZiB0KXRocm93IG5ldyBFcnJvcjtyZXR1cm4gdGhpcy50d2VlbihuLGZ1bmN0aW9uKHQpe3ZhciBu
LGU7ZnVuY3Rpb24gcigpe3ZhciByPXQuYXBwbHkodGhpcyxhcmd1bWVudHMpO3JldHVybiByIT09
ZSYmKG49KGU9cikmJmZ1bmN0aW9uKHQpe3JldHVybiBmdW5jdGlvbihuKXt0aGlzLnRleHRDb250
ZW50PXQuY2FsbCh0aGlzLG4pfX0ocikpLG59cmV0dXJuIHIuX3ZhbHVlPXQscn0odCkpfSxyZW1v
dmU6ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5vbigiZW5kLnJlbW92ZSIsZnVuY3Rpb24odCl7cmV0
dXJuIGZ1bmN0aW9uKCl7dmFyIG49dGhpcy5wYXJlbnROb2RlO2Zvcih2YXIgZSBpbiB0aGlzLl9f
dHJhbnNpdGlvbilpZigrZSE9PXQpcmV0dXJuO24mJm4ucmVtb3ZlQ2hpbGQodGhpcyl9fSh0aGlz
Ll9pZCkpfSx0d2VlbjpmdW5jdGlvbih0LG4pe3ZhciBlPXRoaXMuX2lkO2lmKHQrPSIiLGFyZ3Vt
ZW50cy5sZW5ndGg8Mil7Zm9yKHZhciByLGk9R2kodGhpcy5ub2RlKCksZSkudHdlZW4sbz0wLGE9
aS5sZW5ndGg7bzxhOysrbylpZigocj1pW29dKS5uYW1lPT09dClyZXR1cm4gci52YWx1ZTtyZXR1
cm4gbnVsbH1yZXR1cm4gdGhpcy5lYWNoKChudWxsPT1uP1dpOlppKShlLHQsbikpfSxkZWxheTpm
dW5jdGlvbih0KXt2YXIgbj10aGlzLl9pZDtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD90aGlzLmVh
Y2goKCJmdW5jdGlvbiI9PXR5cGVvZiB0P3VvOmNvKShuLHQpKTpHaSh0aGlzLm5vZGUoKSxuKS5k
ZWxheX0sZHVyYXRpb246ZnVuY3Rpb24odCl7dmFyIG49dGhpcy5faWQ7cmV0dXJuIGFyZ3VtZW50
cy5sZW5ndGg/dGhpcy5lYWNoKCgiZnVuY3Rpb24iPT10eXBlb2YgdD9mbzpzbykobix0KSk6R2ko
dGhpcy5ub2RlKCksbikuZHVyYXRpb259LGVhc2U6ZnVuY3Rpb24odCl7dmFyIG49dGhpcy5faWQ7
cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/dGhpcy5lYWNoKGZ1bmN0aW9uKHQsbil7aWYoImZ1bmN0
aW9uIiE9dHlwZW9mIG4pdGhyb3cgbmV3IEVycm9yO3JldHVybiBmdW5jdGlvbigpe1hpKHRoaXMs
dCkuZWFzZT1ufX0obix0KSk6R2kodGhpcy5ub2RlKCksbikuZWFzZX0sZWFzZVZhcnlpbmc6ZnVu
Y3Rpb24odCl7aWYoImZ1bmN0aW9uIiE9dHlwZW9mIHQpdGhyb3cgbmV3IEVycm9yO3JldHVybiB0
aGlzLmVhY2goZnVuY3Rpb24odCxuKXtyZXR1cm4gZnVuY3Rpb24oKXt2YXIgZT1uLmFwcGx5KHRo
aXMsYXJndW1lbnRzKTtpZigiZnVuY3Rpb24iIT10eXBlb2YgZSl0aHJvdyBuZXcgRXJyb3I7WGko
dGhpcyx0KS5lYXNlPWV9fSh0aGlzLl9pZCx0KSl9LGVuZDpmdW5jdGlvbigpe3ZhciB0LG4sZT10
aGlzLHI9ZS5faWQsaT1lLnNpemUoKTtyZXR1cm4gbmV3IFByb21pc2UoKGZ1bmN0aW9uKG8sYSl7
dmFyIHU9e3ZhbHVlOmF9LGM9e3ZhbHVlOmZ1bmN0aW9uKCl7MD09LS1pJiZvKCl9fTtlLmVhY2go
KGZ1bmN0aW9uKCl7dmFyIGU9WGkodGhpcyxyKSxpPWUub247aSE9PXQmJigobj0odD1pKS5jb3B5
KCkpLl8uY2FuY2VsLnB1c2godSksbi5fLmludGVycnVwdC5wdXNoKHUpLG4uXy5lbmQucHVzaChj
KSksZS5vbj1ufSkpLDA9PT1pJiZvKCl9KSl9LFtTeW1ib2wuaXRlcmF0b3JdOl9vW1N5bWJvbC5p
dGVyYXRvcl19O2Z1bmN0aW9uIGJvKHQpe3JldHVybigodCo9Mik8PTE/dCp0Oi0tdCooMi10KSsx
KS8yfWZ1bmN0aW9uIG1vKHQpe3JldHVybigodCo9Mik8PTE/dCp0KnQ6KHQtPTIpKnQqdCsyKS8y
fXZhciB4bz1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCl7cmV0dXJuIE1hdGgucG93KHQsbil9
cmV0dXJuIG49K24sZS5leHBvbmVudD10LGV9KDMpLHdvPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24g
ZSh0KXtyZXR1cm4gMS1NYXRoLnBvdygxLXQsbil9cmV0dXJuIG49K24sZS5leHBvbmVudD10LGV9
KDMpLE1vPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0KXtyZXR1cm4oKHQqPTIpPD0xP01hdGgu
cG93KHQsbik6Mi1NYXRoLnBvdygyLXQsbikpLzJ9cmV0dXJuIG49K24sZS5leHBvbmVudD10LGV9
KDMpLFRvPU1hdGguUEksQW89VG8vMjtmdW5jdGlvbiBTbyh0KXtyZXR1cm4oMS1NYXRoLmNvcyhU
byp0KSkvMn1mdW5jdGlvbiBFbyh0KXtyZXR1cm4gMS4wMDA5Nzc1MTcxMDY1NDk0KihNYXRoLnBv
dygyLC0xMCp0KS0uMDAwOTc2NTYyNSl9ZnVuY3Rpb24gTm8odCl7cmV0dXJuKCh0Kj0yKTw9MT9F
bygxLXQpOjItRW8odC0xKSkvMn1mdW5jdGlvbiBrbyh0KXtyZXR1cm4oKHQqPTIpPD0xPzEtTWF0
aC5zcXJ0KDEtdCp0KTpNYXRoLnNxcnQoMS0odC09MikqdCkrMSkvMn12YXIgQ289NC8xMSxQbz02
LzExLHpvPTgvMTEsJG89My80LERvPTkvMTEsUm89MTAvMTEsRm89MTUvMTYscW89MjEvMjIsVW89
NjMvNjQsSW89MS9Dby9DbztmdW5jdGlvbiBPbyh0KXtyZXR1cm4odD0rdCk8Q28/SW8qdCp0OnQ8
em8/SW8qKHQtPVBvKSp0KyRvOnQ8Um8/SW8qKHQtPURvKSp0K0ZvOklvKih0LT1xbykqdCtVb312
YXIgQm89MS43MDE1OCxZbz1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCl7cmV0dXJuKHQ9K3Qp
KnQqKG4qKHQtMSkrdCl9cmV0dXJuIG49K24sZS5vdmVyc2hvb3Q9dCxlfShCbyksTG89ZnVuY3Rp
b24gdChuKXtmdW5jdGlvbiBlKHQpe3JldHVybi0tdCp0KigodCsxKSpuK3QpKzF9cmV0dXJuIG49
K24sZS5vdmVyc2hvb3Q9dCxlfShCbyksam89ZnVuY3Rpb24gdChuKXtmdW5jdGlvbiBlKHQpe3Jl
dHVybigodCo9Mik8MT90KnQqKChuKzEpKnQtbik6KHQtPTIpKnQqKChuKzEpKnQrbikrMikvMn1y
ZXR1cm4gbj0rbixlLm92ZXJzaG9vdD10LGV9KEJvKSxIbz0yKk1hdGguUEksWG89ZnVuY3Rpb24g
dChuLGUpe3ZhciByPU1hdGguYXNpbigxLyhuPU1hdGgubWF4KDEsbikpKSooZS89SG8pO2Z1bmN0
aW9uIGkodCl7cmV0dXJuIG4qRW8oLSAtLXQpKk1hdGguc2luKChyLXQpL2UpfXJldHVybiBpLmFt
cGxpdHVkZT1mdW5jdGlvbihuKXtyZXR1cm4gdChuLGUqSG8pfSxpLnBlcmlvZD1mdW5jdGlvbihl
KXtyZXR1cm4gdChuLGUpfSxpfSgxLC4zKSxHbz1mdW5jdGlvbiB0KG4sZSl7dmFyIHI9TWF0aC5h
c2luKDEvKG49TWF0aC5tYXgoMSxuKSkpKihlLz1Ibyk7ZnVuY3Rpb24gaSh0KXtyZXR1cm4gMS1u
KkVvKHQ9K3QpKk1hdGguc2luKCh0K3IpL2UpfXJldHVybiBpLmFtcGxpdHVkZT1mdW5jdGlvbihu
KXtyZXR1cm4gdChuLGUqSG8pfSxpLnBlcmlvZD1mdW5jdGlvbihlKXtyZXR1cm4gdChuLGUpfSxp
fSgxLC4zKSxWbz1mdW5jdGlvbiB0KG4sZSl7dmFyIHI9TWF0aC5hc2luKDEvKG49TWF0aC5tYXgo
MSxuKSkpKihlLz1Ibyk7ZnVuY3Rpb24gaSh0KXtyZXR1cm4oKHQ9Mip0LTEpPDA/bipFbygtdCkq
TWF0aC5zaW4oKHItdCkvZSk6Mi1uKkVvKHQpKk1hdGguc2luKChyK3QpL2UpKS8yfXJldHVybiBp
LmFtcGxpdHVkZT1mdW5jdGlvbihuKXtyZXR1cm4gdChuLGUqSG8pfSxpLnBlcmlvZD1mdW5jdGlv
bihlKXtyZXR1cm4gdChuLGUpfSxpfSgxLC4zKSxXbz17dGltZTpudWxsLGRlbGF5OjAsZHVyYXRp
b246MjUwLGVhc2U6bW99O2Z1bmN0aW9uIFpvKHQsbil7Zm9yKHZhciBlOyEoZT10Ll9fdHJhbnNp
dGlvbil8fCEoZT1lW25dKTspaWYoISh0PXQucGFyZW50Tm9kZSkpdGhyb3cgbmV3IEVycm9yKGB0
cmFuc2l0aW9uICR7bn0gbm90IGZvdW5kYCk7cmV0dXJuIGV9Wm4ucHJvdG90eXBlLmludGVycnVw
dD1mdW5jdGlvbih0KXtyZXR1cm4gdGhpcy5lYWNoKChmdW5jdGlvbigpe1ZpKHRoaXMsdCl9KSl9
LFpuLnByb3RvdHlwZS50cmFuc2l0aW9uPWZ1bmN0aW9uKHQpe3ZhciBuLGU7dCBpbnN0YW5jZW9m
IGdvPyhuPXQuX2lkLHQ9dC5fbmFtZSk6KG49dm8oKSwoZT1XbykudGltZT1TaSgpLHQ9bnVsbD09
dD9udWxsOnQrIiIpO2Zvcih2YXIgcj10aGlzLl9ncm91cHMsaT1yLmxlbmd0aCxvPTA7bzxpOysr
bylmb3IodmFyIGEsdT1yW29dLGM9dS5sZW5ndGgsZj0wO2Y8YzsrK2YpKGE9dVtmXSkmJmppKGEs
dCxuLGYsdSxlfHxabyhhLG4pKTtyZXR1cm4gbmV3IGdvKHIsdGhpcy5fcGFyZW50cyx0LG4pfTt2
YXIgS289W251bGxdO3ZhciBRbz10PT4oKT0+dDtmdW5jdGlvbiBKbyh0LHtzb3VyY2VFdmVudDpu
LHRhcmdldDplLHNlbGVjdGlvbjpyLG1vZGU6aSxkaXNwYXRjaDpvfSl7T2JqZWN0LmRlZmluZVBy
b3BlcnRpZXModGhpcyx7dHlwZTp7dmFsdWU6dCxlbnVtZXJhYmxlOiEwLGNvbmZpZ3VyYWJsZToh
MH0sc291cmNlRXZlbnQ6e3ZhbHVlOm4sZW51bWVyYWJsZTohMCxjb25maWd1cmFibGU6ITB9LHRh
cmdldDp7dmFsdWU6ZSxlbnVtZXJhYmxlOiEwLGNvbmZpZ3VyYWJsZTohMH0sc2VsZWN0aW9uOnt2
YWx1ZTpyLGVudW1lcmFibGU6ITAsY29uZmlndXJhYmxlOiEwfSxtb2RlOnt2YWx1ZTppLGVudW1l
cmFibGU6ITAsY29uZmlndXJhYmxlOiEwfSxfOnt2YWx1ZTpvfX0pfWZ1bmN0aW9uIHRhKHQpe3Qu
cHJldmVudERlZmF1bHQoKSx0LnN0b3BJbW1lZGlhdGVQcm9wYWdhdGlvbigpfXZhciBuYT17bmFt
ZToiZHJhZyJ9LGVhPXtuYW1lOiJzcGFjZSJ9LHJhPXtuYW1lOiJoYW5kbGUifSxpYT17bmFtZToi
Y2VudGVyIn07Y29uc3R7YWJzOm9hLG1heDphYSxtaW46dWF9PU1hdGg7ZnVuY3Rpb24gY2EodCl7
cmV0dXJuWyt0WzBdLCt0WzFdXX1mdW5jdGlvbiBmYSh0KXtyZXR1cm5bY2EodFswXSksY2EodFsx
XSldfXZhciBzYT17bmFtZToieCIsaGFuZGxlczpbInciLCJlIl0ubWFwKF9hKSxpbnB1dDpmdW5j
dGlvbih0LG4pe3JldHVybiBudWxsPT10P251bGw6W1srdFswXSxuWzBdWzFdXSxbK3RbMV0sblsx
XVsxXV1dfSxvdXRwdXQ6ZnVuY3Rpb24odCl7cmV0dXJuIHQmJlt0WzBdWzBdLHRbMV1bMF1dfX0s
bGE9e25hbWU6InkiLGhhbmRsZXM6WyJuIiwicyJdLm1hcChfYSksaW5wdXQ6ZnVuY3Rpb24odCxu
KXtyZXR1cm4gbnVsbD09dD9udWxsOltbblswXVswXSwrdFswXV0sW25bMV1bMF0sK3RbMV1dXX0s
b3V0cHV0OmZ1bmN0aW9uKHQpe3JldHVybiB0JiZbdFswXVsxXSx0WzFdWzFdXX19LGhhPXtuYW1l
OiJ4eSIsaGFuZGxlczpbIm4iLCJ3IiwiZSIsInMiLCJudyIsIm5lIiwic3ciLCJzZSJdLm1hcChf
YSksaW5wdXQ6ZnVuY3Rpb24odCl7cmV0dXJuIG51bGw9PXQ/bnVsbDpmYSh0KX0sb3V0cHV0OmZ1
bmN0aW9uKHQpe3JldHVybiB0fX0sZGE9e292ZXJsYXk6ImNyb3NzaGFpciIsc2VsZWN0aW9uOiJt
b3ZlIixuOiJucy1yZXNpemUiLGU6ImV3LXJlc2l6ZSIsczoibnMtcmVzaXplIix3OiJldy1yZXNp
emUiLG53OiJud3NlLXJlc2l6ZSIsbmU6Im5lc3ctcmVzaXplIixzZToibndzZS1yZXNpemUiLHN3
OiJuZXN3LXJlc2l6ZSJ9LHBhPXtlOiJ3Iix3OiJlIixudzoibmUiLG5lOiJudyIsc2U6InN3Iixz
dzoic2UifSxnYT17bjoicyIsczoibiIsbnc6InN3IixuZToic2UiLHNlOiJuZSIsc3c6Im53In0s
eWE9e292ZXJsYXk6MSxzZWxlY3Rpb246MSxuOm51bGwsZToxLHM6bnVsbCx3Oi0xLG53Oi0xLG5l
OjEsc2U6MSxzdzotMX0sdmE9e292ZXJsYXk6MSxzZWxlY3Rpb246MSxuOi0xLGU6bnVsbCxzOjEs
dzpudWxsLG53Oi0xLG5lOi0xLHNlOjEsc3c6MX07ZnVuY3Rpb24gX2EodCl7cmV0dXJue3R5cGU6
dH19ZnVuY3Rpb24gYmEodCl7cmV0dXJuIXQuY3RybEtleSYmIXQuYnV0dG9ufWZ1bmN0aW9uIG1h
KCl7dmFyIHQ9dGhpcy5vd25lclNWR0VsZW1lbnR8fHRoaXM7cmV0dXJuIHQuaGFzQXR0cmlidXRl
KCJ2aWV3Qm94Iik/W1sodD10LnZpZXdCb3guYmFzZVZhbCkueCx0LnldLFt0LngrdC53aWR0aCx0
LnkrdC5oZWlnaHRdXTpbWzAsMF0sW3Qud2lkdGguYmFzZVZhbC52YWx1ZSx0LmhlaWdodC5iYXNl
VmFsLnZhbHVlXV19ZnVuY3Rpb24geGEoKXtyZXR1cm4gbmF2aWdhdG9yLm1heFRvdWNoUG9pbnRz
fHwib250b3VjaHN0YXJ0ImluIHRoaXN9ZnVuY3Rpb24gd2EodCl7Zm9yKDshdC5fX2JydXNoOylp
ZighKHQ9dC5wYXJlbnROb2RlKSlyZXR1cm47cmV0dXJuIHQuX19icnVzaH1mdW5jdGlvbiBNYSh0
KXt2YXIgbixlPW1hLHI9YmEsaT14YSxvPSEwLGE9RHQoInN0YXJ0IiwiYnJ1c2giLCJlbmQiKSx1
PTY7ZnVuY3Rpb24gYyhuKXt2YXIgZT1uLnByb3BlcnR5KCJfX2JydXNoIixnKS5zZWxlY3RBbGwo
Ii5vdmVybGF5IikuZGF0YShbX2EoIm92ZXJsYXkiKV0pO2UuZW50ZXIoKS5hcHBlbmQoInJlY3Qi
KS5hdHRyKCJjbGFzcyIsIm92ZXJsYXkiKS5hdHRyKCJwb2ludGVyLWV2ZW50cyIsImFsbCIpLmF0
dHIoImN1cnNvciIsZGEub3ZlcmxheSkubWVyZ2UoZSkuZWFjaCgoZnVuY3Rpb24oKXt2YXIgdD13
YSh0aGlzKS5leHRlbnQ7S24odGhpcykuYXR0cigieCIsdFswXVswXSkuYXR0cigieSIsdFswXVsx
XSkuYXR0cigid2lkdGgiLHRbMV1bMF0tdFswXVswXSkuYXR0cigiaGVpZ2h0Iix0WzFdWzFdLXRb
MF1bMV0pfSkpLG4uc2VsZWN0QWxsKCIuc2VsZWN0aW9uIikuZGF0YShbX2EoInNlbGVjdGlvbiIp
XSkuZW50ZXIoKS5hcHBlbmQoInJlY3QiKS5hdHRyKCJjbGFzcyIsInNlbGVjdGlvbiIpLmF0dHIo
ImN1cnNvciIsZGEuc2VsZWN0aW9uKS5hdHRyKCJmaWxsIiwiIzc3NyIpLmF0dHIoImZpbGwtb3Bh
Y2l0eSIsLjMpLmF0dHIoInN0cm9rZSIsIiNmZmYiKS5hdHRyKCJzaGFwZS1yZW5kZXJpbmciLCJj
cmlzcEVkZ2VzIik7dmFyIHI9bi5zZWxlY3RBbGwoIi5oYW5kbGUiKS5kYXRhKHQuaGFuZGxlcywo
ZnVuY3Rpb24odCl7cmV0dXJuIHQudHlwZX0pKTtyLmV4aXQoKS5yZW1vdmUoKSxyLmVudGVyKCku
YXBwZW5kKCJyZWN0IikuYXR0cigiY2xhc3MiLChmdW5jdGlvbih0KXtyZXR1cm4iaGFuZGxlIGhh
bmRsZS0tIit0LnR5cGV9KSkuYXR0cigiY3Vyc29yIiwoZnVuY3Rpb24odCl7cmV0dXJuIGRhW3Qu
dHlwZV19KSksbi5lYWNoKGYpLmF0dHIoImZpbGwiLCJub25lIikuYXR0cigicG9pbnRlci1ldmVu
dHMiLCJhbGwiKS5vbigibW91c2Vkb3duLmJydXNoIixoKS5maWx0ZXIoaSkub24oInRvdWNoc3Rh
cnQuYnJ1c2giLGgpLm9uKCJ0b3VjaG1vdmUuYnJ1c2giLGQpLm9uKCJ0b3VjaGVuZC5icnVzaCB0
b3VjaGNhbmNlbC5icnVzaCIscCkuc3R5bGUoInRvdWNoLWFjdGlvbiIsIm5vbmUiKS5zdHlsZSgi
LXdlYmtpdC10YXAtaGlnaGxpZ2h0LWNvbG9yIiwicmdiYSgwLDAsMCwwKSIpfWZ1bmN0aW9uIGYo
KXt2YXIgdD1Lbih0aGlzKSxuPXdhKHRoaXMpLnNlbGVjdGlvbjtuPyh0LnNlbGVjdEFsbCgiLnNl
bGVjdGlvbiIpLnN0eWxlKCJkaXNwbGF5IixudWxsKS5hdHRyKCJ4IixuWzBdWzBdKS5hdHRyKCJ5
IixuWzBdWzFdKS5hdHRyKCJ3aWR0aCIsblsxXVswXS1uWzBdWzBdKS5hdHRyKCJoZWlnaHQiLG5b
MV1bMV0tblswXVsxXSksdC5zZWxlY3RBbGwoIi5oYW5kbGUiKS5zdHlsZSgiZGlzcGxheSIsbnVs
bCkuYXR0cigieCIsKGZ1bmN0aW9uKHQpe3JldHVybiJlIj09PXQudHlwZVt0LnR5cGUubGVuZ3Ro
LTFdP25bMV1bMF0tdS8yOm5bMF1bMF0tdS8yfSkpLmF0dHIoInkiLChmdW5jdGlvbih0KXtyZXR1
cm4icyI9PT10LnR5cGVbMF0/blsxXVsxXS11LzI6blswXVsxXS11LzJ9KSkuYXR0cigid2lkdGgi
LChmdW5jdGlvbih0KXtyZXR1cm4ibiI9PT10LnR5cGV8fCJzIj09PXQudHlwZT9uWzFdWzBdLW5b
MF1bMF0rdTp1fSkpLmF0dHIoImhlaWdodCIsKGZ1bmN0aW9uKHQpe3JldHVybiJlIj09PXQudHlw
ZXx8InciPT09dC50eXBlP25bMV1bMV0tblswXVsxXSt1OnV9KSkpOnQuc2VsZWN0QWxsKCIuc2Vs
ZWN0aW9uLC5oYW5kbGUiKS5zdHlsZSgiZGlzcGxheSIsIm5vbmUiKS5hdHRyKCJ4IixudWxsKS5h
dHRyKCJ5IixudWxsKS5hdHRyKCJ3aWR0aCIsbnVsbCkuYXR0cigiaGVpZ2h0IixudWxsKX1mdW5j
dGlvbiBzKHQsbixlKXt2YXIgcj10Ll9fYnJ1c2guZW1pdHRlcjtyZXR1cm4hcnx8ZSYmci5jbGVh
bj9uZXcgbCh0LG4sZSk6cn1mdW5jdGlvbiBsKHQsbixlKXt0aGlzLnRoYXQ9dCx0aGlzLmFyZ3M9
bix0aGlzLnN0YXRlPXQuX19icnVzaCx0aGlzLmFjdGl2ZT0wLHRoaXMuY2xlYW49ZX1mdW5jdGlv
biBoKGUpe2lmKCghbnx8ZS50b3VjaGVzKSYmci5hcHBseSh0aGlzLGFyZ3VtZW50cykpe3ZhciBp
LGEsdSxjLGwsaCxkLHAsZyx5LHYsXz10aGlzLGI9ZS50YXJnZXQuX19kYXRhX18udHlwZSxtPSJz
ZWxlY3Rpb24iPT09KG8mJmUubWV0YUtleT9iPSJvdmVybGF5IjpiKT9uYTpvJiZlLmFsdEtleT9p
YTpyYSx4PXQ9PT1sYT9udWxsOnlhW2JdLHc9dD09PXNhP251bGw6dmFbYl0sTT13YShfKSxUPU0u
ZXh0ZW50LEE9TS5zZWxlY3Rpb24sUz1UWzBdWzBdLEU9VFswXVsxXSxOPVRbMV1bMF0saz1UWzFd
WzFdLEM9MCxQPTAsej14JiZ3JiZvJiZlLnNoaWZ0S2V5LCQ9QXJyYXkuZnJvbShlLnRvdWNoZXN8
fFtlXSwodD0+e2NvbnN0IG49dC5pZGVudGlmaWVyO3JldHVybih0PWVlKHQsXykpLnBvaW50MD10
LnNsaWNlKCksdC5pZGVudGlmaWVyPW4sdH0pKTtWaShfKTt2YXIgRD1zKF8sYXJndW1lbnRzLCEw
KS5iZWZvcmVzdGFydCgpO2lmKCJvdmVybGF5Ij09PWIpe0EmJihnPSEwKTtjb25zdCBuPVskWzBd
LCRbMV18fCRbMF1dO00uc2VsZWN0aW9uPUE9W1tpPXQ9PT1sYT9TOnVhKG5bMF1bMF0sblsxXVsw
XSksdT10PT09c2E/RTp1YShuWzBdWzFdLG5bMV1bMV0pXSxbbD10PT09bGE/TjphYShuWzBdWzBd
LG5bMV1bMF0pLGQ9dD09PXNhP2s6YWEoblswXVsxXSxuWzFdWzFdKV1dLCQubGVuZ3RoPjEmJkko
ZSl9ZWxzZSBpPUFbMF1bMF0sdT1BWzBdWzFdLGw9QVsxXVswXSxkPUFbMV1bMV07YT1pLGM9dSxo
PWwscD1kO3ZhciBSPUtuKF8pLmF0dHIoInBvaW50ZXItZXZlbnRzIiwibm9uZSIpLEY9Ui5zZWxl
Y3RBbGwoIi5vdmVybGF5IikuYXR0cigiY3Vyc29yIixkYVtiXSk7aWYoZS50b3VjaGVzKUQubW92
ZWQ9VSxELmVuZGVkPU87ZWxzZXt2YXIgcT1LbihlLnZpZXcpLm9uKCJtb3VzZW1vdmUuYnJ1c2gi
LFUsITApLm9uKCJtb3VzZXVwLmJydXNoIixPLCEwKTtvJiZxLm9uKCJrZXlkb3duLmJydXNoIiwo
ZnVuY3Rpb24odCl7c3dpdGNoKHQua2V5Q29kZSl7Y2FzZSAxNjp6PXgmJnc7YnJlYWs7Y2FzZSAx
ODptPT09cmEmJih4JiYobD1oLUMqeCxpPWErQyp4KSx3JiYoZD1wLVAqdyx1PWMrUCp3KSxtPWlh
LEkodCkpO2JyZWFrO2Nhc2UgMzI6bSE9PXJhJiZtIT09aWF8fCh4PDA/bD1oLUM6eD4wJiYoaT1h
LUMpLHc8MD9kPXAtUDp3PjAmJih1PWMtUCksbT1lYSxGLmF0dHIoImN1cnNvciIsZGEuc2VsZWN0
aW9uKSxJKHQpKTticmVhaztkZWZhdWx0OnJldHVybn10YSh0KX0pLCEwKS5vbigia2V5dXAuYnJ1
c2giLChmdW5jdGlvbih0KXtzd2l0Y2godC5rZXlDb2RlKXtjYXNlIDE2OnomJih5PXY9ej0hMSxJ
KHQpKTticmVhaztjYXNlIDE4Om09PT1pYSYmKHg8MD9sPWg6eD4wJiYoaT1hKSx3PDA/ZD1wOnc+
MCYmKHU9YyksbT1yYSxJKHQpKTticmVhaztjYXNlIDMyOm09PT1lYSYmKHQuYWx0S2V5Pyh4JiYo
bD1oLUMqeCxpPWErQyp4KSx3JiYoZD1wLVAqdyx1PWMrUCp3KSxtPWlhKTooeDwwP2w9aDp4PjAm
JihpPWEpLHc8MD9kPXA6dz4wJiYodT1jKSxtPXJhKSxGLmF0dHIoImN1cnNvciIsZGFbYl0pLEko
dCkpO2JyZWFrO2RlZmF1bHQ6cmV0dXJufXRhKHQpfSksITApLHVlKGUudmlldyl9Zi5jYWxsKF8p
LEQuc3RhcnQoZSxtLm5hbWUpfWZ1bmN0aW9uIFUodCl7Zm9yKGNvbnN0IG4gb2YgdC5jaGFuZ2Vk
VG91Y2hlc3x8W3RdKWZvcihjb25zdCB0IG9mICQpdC5pZGVudGlmaWVyPT09bi5pZGVudGlmaWVy
JiYodC5jdXI9ZWUobixfKSk7aWYoeiYmIXkmJiF2JiYxPT09JC5sZW5ndGgpe2NvbnN0IHQ9JFsw
XTtvYSh0LmN1clswXS10WzBdKT5vYSh0LmN1clsxXS10WzFdKT92PSEwOnk9ITB9Zm9yKGNvbnN0
IHQgb2YgJCl0LmN1ciYmKHRbMF09dC5jdXJbMF0sdFsxXT10LmN1clsxXSk7Zz0hMCx0YSh0KSxJ
KHQpfWZ1bmN0aW9uIEkodCl7Y29uc3Qgbj0kWzBdLGU9bi5wb2ludDA7dmFyIHI7c3dpdGNoKEM9
blswXS1lWzBdLFA9blsxXS1lWzFdLG0pe2Nhc2UgZWE6Y2FzZSBuYTp4JiYoQz1hYShTLWksdWEo
Ti1sLEMpKSxhPWkrQyxoPWwrQyksdyYmKFA9YWEoRS11LHVhKGstZCxQKSksYz11K1AscD1kK1Ap
O2JyZWFrO2Nhc2UgcmE6JFsxXT8oeCYmKGE9YWEoUyx1YShOLCRbMF1bMF0pKSxoPWFhKFMsdWEo
TiwkWzFdWzBdKSkseD0xKSx3JiYoYz1hYShFLHVhKGssJFswXVsxXSkpLHA9YWEoRSx1YShrLCRb
MV1bMV0pKSx3PTEpKTooeDwwPyhDPWFhKFMtaSx1YShOLWksQykpLGE9aStDLGg9bCk6eD4wJiYo
Qz1hYShTLWwsdWEoTi1sLEMpKSxhPWksaD1sK0MpLHc8MD8oUD1hYShFLXUsdWEoay11LFApKSxj
PXUrUCxwPWQpOnc+MCYmKFA9YWEoRS1kLHVhKGstZCxQKSksYz11LHA9ZCtQKSk7YnJlYWs7Y2Fz
ZSBpYTp4JiYoYT1hYShTLHVhKE4saS1DKngpKSxoPWFhKFMsdWEoTixsK0MqeCkpKSx3JiYoYz1h
YShFLHVhKGssdS1QKncpKSxwPWFhKEUsdWEoayxkK1AqdykpKX1oPGEmJih4Kj0tMSxyPWksaT1s
LGw9cixyPWEsYT1oLGg9cixiIGluIHBhJiZGLmF0dHIoImN1cnNvciIsZGFbYj1wYVtiXV0pKSxw
PGMmJih3Kj0tMSxyPXUsdT1kLGQ9cixyPWMsYz1wLHA9cixiIGluIGdhJiZGLmF0dHIoImN1cnNv
ciIsZGFbYj1nYVtiXV0pKSxNLnNlbGVjdGlvbiYmKEE9TS5zZWxlY3Rpb24pLHkmJihhPUFbMF1b
MF0saD1BWzFdWzBdKSx2JiYoYz1BWzBdWzFdLHA9QVsxXVsxXSksQVswXVswXT09PWEmJkFbMF1b
MV09PT1jJiZBWzFdWzBdPT09aCYmQVsxXVsxXT09PXB8fChNLnNlbGVjdGlvbj1bW2EsY10sW2gs
cF1dLGYuY2FsbChfKSxELmJydXNoKHQsbS5uYW1lKSl9ZnVuY3Rpb24gTyh0KXtpZihmdW5jdGlv
bih0KXt0LnN0b3BJbW1lZGlhdGVQcm9wYWdhdGlvbigpfSh0KSx0LnRvdWNoZXMpe2lmKHQudG91
Y2hlcy5sZW5ndGgpcmV0dXJuO24mJmNsZWFyVGltZW91dChuKSxuPXNldFRpbWVvdXQoKGZ1bmN0
aW9uKCl7bj1udWxsfSksNTAwKX1lbHNlIGNlKHQudmlldyxnKSxxLm9uKCJrZXlkb3duLmJydXNo
IGtleXVwLmJydXNoIG1vdXNlbW92ZS5icnVzaCBtb3VzZXVwLmJydXNoIixudWxsKTtSLmF0dHIo
InBvaW50ZXItZXZlbnRzIiwiYWxsIiksRi5hdHRyKCJjdXJzb3IiLGRhLm92ZXJsYXkpLE0uc2Vs
ZWN0aW9uJiYoQT1NLnNlbGVjdGlvbiksZnVuY3Rpb24odCl7cmV0dXJuIHRbMF1bMF09PT10WzFd
WzBdfHx0WzBdWzFdPT09dFsxXVsxXX0oQSkmJihNLnNlbGVjdGlvbj1udWxsLGYuY2FsbChfKSks
RC5lbmQodCxtLm5hbWUpfX1mdW5jdGlvbiBkKHQpe3ModGhpcyxhcmd1bWVudHMpLm1vdmVkKHQp
fWZ1bmN0aW9uIHAodCl7cyh0aGlzLGFyZ3VtZW50cykuZW5kZWQodCl9ZnVuY3Rpb24gZygpe3Zh
ciBuPXRoaXMuX19icnVzaHx8e3NlbGVjdGlvbjpudWxsfTtyZXR1cm4gbi5leHRlbnQ9ZmEoZS5h
cHBseSh0aGlzLGFyZ3VtZW50cykpLG4uZGltPXQsbn1yZXR1cm4gYy5tb3ZlPWZ1bmN0aW9uKG4s
ZSxyKXtuLnR3ZWVuP24ub24oInN0YXJ0LmJydXNoIiwoZnVuY3Rpb24odCl7cyh0aGlzLGFyZ3Vt
ZW50cykuYmVmb3Jlc3RhcnQoKS5zdGFydCh0KX0pKS5vbigiaW50ZXJydXB0LmJydXNoIGVuZC5i
cnVzaCIsKGZ1bmN0aW9uKHQpe3ModGhpcyxhcmd1bWVudHMpLmVuZCh0KX0pKS50d2VlbigiYnJ1
c2giLChmdW5jdGlvbigpe3ZhciBuPXRoaXMscj1uLl9fYnJ1c2gsaT1zKG4sYXJndW1lbnRzKSxv
PXIuc2VsZWN0aW9uLGE9dC5pbnB1dCgiZnVuY3Rpb24iPT10eXBlb2YgZT9lLmFwcGx5KHRoaXMs
YXJndW1lbnRzKTplLHIuZXh0ZW50KSx1PVZyKG8sYSk7ZnVuY3Rpb24gYyh0KXtyLnNlbGVjdGlv
bj0xPT09dCYmbnVsbD09PWE/bnVsbDp1KHQpLGYuY2FsbChuKSxpLmJydXNoKCl9cmV0dXJuIG51
bGwhPT1vJiZudWxsIT09YT9jOmMoMSl9KSk6bi5lYWNoKChmdW5jdGlvbigpe3ZhciBuPXRoaXMs
aT1hcmd1bWVudHMsbz1uLl9fYnJ1c2gsYT10LmlucHV0KCJmdW5jdGlvbiI9PXR5cGVvZiBlP2Uu
YXBwbHkobixpKTplLG8uZXh0ZW50KSx1PXMobixpKS5iZWZvcmVzdGFydCgpO1ZpKG4pLG8uc2Vs
ZWN0aW9uPW51bGw9PT1hP251bGw6YSxmLmNhbGwobiksdS5zdGFydChyKS5icnVzaChyKS5lbmQo
cil9KSl9LGMuY2xlYXI9ZnVuY3Rpb24odCxuKXtjLm1vdmUodCxudWxsLG4pfSxsLnByb3RvdHlw
ZT17YmVmb3Jlc3RhcnQ6ZnVuY3Rpb24oKXtyZXR1cm4gMT09Kyt0aGlzLmFjdGl2ZSYmKHRoaXMu
c3RhdGUuZW1pdHRlcj10aGlzLHRoaXMuc3RhcnRpbmc9ITApLHRoaXN9LHN0YXJ0OmZ1bmN0aW9u
KHQsbil7cmV0dXJuIHRoaXMuc3RhcnRpbmc/KHRoaXMuc3RhcnRpbmc9ITEsdGhpcy5lbWl0KCJz
dGFydCIsdCxuKSk6dGhpcy5lbWl0KCJicnVzaCIsdCksdGhpc30sYnJ1c2g6ZnVuY3Rpb24odCxu
KXtyZXR1cm4gdGhpcy5lbWl0KCJicnVzaCIsdCxuKSx0aGlzfSxlbmQ6ZnVuY3Rpb24odCxuKXty
ZXR1cm4gMD09LS10aGlzLmFjdGl2ZSYmKGRlbGV0ZSB0aGlzLnN0YXRlLmVtaXR0ZXIsdGhpcy5l
bWl0KCJlbmQiLHQsbikpLHRoaXN9LGVtaXQ6ZnVuY3Rpb24obixlLHIpe3ZhciBpPUtuKHRoaXMu
dGhhdCkuZGF0dW0oKTthLmNhbGwobix0aGlzLnRoYXQsbmV3IEpvKG4se3NvdXJjZUV2ZW50OmUs
dGFyZ2V0OmMsc2VsZWN0aW9uOnQub3V0cHV0KHRoaXMuc3RhdGUuc2VsZWN0aW9uKSxtb2RlOnIs
ZGlzcGF0Y2g6YX0pLGkpfX0sYy5leHRlbnQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/KGU9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpRbyhmYSh0KSksYyk6ZX0sYy5maWx0ZXI9
ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9ImZ1bmN0aW9uIj09dHlwZW9m
IHQ/dDpRbyghIXQpLGMpOnJ9LGMudG91Y2hhYmxlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVu
dHMubGVuZ3RoPyhpPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6UW8oISF0KSxjKTppfSxjLmhhbmRs
ZVNpemU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHU9K3QsYyk6dX0sYy5r
ZXlNb2RpZmllcnM9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG89ISF0LGMp
Om99LGMub249ZnVuY3Rpb24oKXt2YXIgdD1hLm9uLmFwcGx5KGEsYXJndW1lbnRzKTtyZXR1cm4g
dD09PWE/Yzp0fSxjfXZhciBUYT1NYXRoLmFicyxBYT1NYXRoLmNvcyxTYT1NYXRoLnNpbixFYT1N
YXRoLlBJLE5hPUVhLzIsa2E9MipFYSxDYT1NYXRoLm1heCxQYT0xZS0xMjtmdW5jdGlvbiB6YSh0
LG4pe3JldHVybiBBcnJheS5mcm9tKHtsZW5ndGg6bi10fSwoKG4sZSk9PnQrZSkpfWZ1bmN0aW9u
ICRhKHQsbil7dmFyIGU9MCxyPW51bGwsaT1udWxsLG89bnVsbDtmdW5jdGlvbiBhKGEpe3ZhciB1
LGM9YS5sZW5ndGgsZj1uZXcgQXJyYXkoYykscz16YSgwLGMpLGw9bmV3IEFycmF5KGMqYyksaD1u
ZXcgQXJyYXkoYyksZD0wO2E9RmxvYXQ2NEFycmF5LmZyb20oe2xlbmd0aDpjKmN9LG4/KHQsbik9
PmFbbiVjXVtuL2N8MF06KHQsbik9PmFbbi9jfDBdW24lY10pO2ZvcihsZXQgbj0wO248YzsrK24p
e2xldCBlPTA7Zm9yKGxldCByPTA7cjxjOysrcillKz1hW24qYytyXSt0KmFbcipjK25dO2QrPWZb
bl09ZX11PShkPUNhKDAsa2EtZSpjKS9kKT9lOmthL2M7e2xldCBuPTA7ciYmcy5zb3J0KCgodCxu
KT0+cihmW3RdLGZbbl0pKSk7Zm9yKGNvbnN0IGUgb2Ygcyl7Y29uc3Qgcj1uO2lmKHQpe2NvbnN0
IHQ9emEoMSt+YyxjKS5maWx0ZXIoKHQ9PnQ8MD9hW350KmMrZV06YVtlKmMrdF0pKTtpJiZ0LnNv
cnQoKCh0LG4pPT5pKHQ8MD8tYVt+dCpjK2VdOmFbZSpjK3RdLG48MD8tYVt+bipjK2VdOmFbZSpj
K25dKSkpO2Zvcihjb25zdCByIG9mIHQpaWYocjwwKXsobFt+cipjK2VdfHwobFt+cipjK2VdPXtz
b3VyY2U6bnVsbCx0YXJnZXQ6bnVsbH0pKS50YXJnZXQ9e2luZGV4OmUsc3RhcnRBbmdsZTpuLGVu
ZEFuZ2xlOm4rPWFbfnIqYytlXSpkLHZhbHVlOmFbfnIqYytlXX19ZWxzZXsobFtlKmMrcl18fChs
W2UqYytyXT17c291cmNlOm51bGwsdGFyZ2V0Om51bGx9KSkuc291cmNlPXtpbmRleDplLHN0YXJ0
QW5nbGU6bixlbmRBbmdsZTpuKz1hW2UqYytyXSpkLHZhbHVlOmFbZSpjK3JdfX1oW2VdPXtpbmRl
eDplLHN0YXJ0QW5nbGU6cixlbmRBbmdsZTpuLHZhbHVlOmZbZV19fWVsc2V7Y29uc3QgdD16YSgw
LGMpLmZpbHRlcigodD0+YVtlKmMrdF18fGFbdCpjK2VdKSk7aSYmdC5zb3J0KCgodCxuKT0+aShh
W2UqYyt0XSxhW2UqYytuXSkpKTtmb3IoY29uc3QgciBvZiB0KXtsZXQgdDtpZihlPHI/KHQ9bFtl
KmMrcl18fChsW2UqYytyXT17c291cmNlOm51bGwsdGFyZ2V0Om51bGx9KSx0LnNvdXJjZT17aW5k
ZXg6ZSxzdGFydEFuZ2xlOm4sZW5kQW5nbGU6bis9YVtlKmMrcl0qZCx2YWx1ZTphW2UqYytyXX0p
Oih0PWxbcipjK2VdfHwobFtyKmMrZV09e3NvdXJjZTpudWxsLHRhcmdldDpudWxsfSksdC50YXJn
ZXQ9e2luZGV4OmUsc3RhcnRBbmdsZTpuLGVuZEFuZ2xlOm4rPWFbZSpjK3JdKmQsdmFsdWU6YVtl
KmMrcl19LGU9PT1yJiYodC5zb3VyY2U9dC50YXJnZXQpKSx0LnNvdXJjZSYmdC50YXJnZXQmJnQu
c291cmNlLnZhbHVlPHQudGFyZ2V0LnZhbHVlKXtjb25zdCBuPXQuc291cmNlO3Quc291cmNlPXQu
dGFyZ2V0LHQudGFyZ2V0PW59fWhbZV09e2luZGV4OmUsc3RhcnRBbmdsZTpyLGVuZEFuZ2xlOm4s
dmFsdWU6ZltlXX19bis9dX19cmV0dXJuKGw9T2JqZWN0LnZhbHVlcyhsKSkuZ3JvdXBzPWgsbz9s
LnNvcnQobyk6bH1yZXR1cm4gYS5wYWRBbmdsZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD8oZT1DYSgwLHQpLGEpOmV9LGEuc29ydEdyb3Vwcz1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8ocj10LGEpOnJ9LGEuc29ydFN1Ymdyb3Vwcz1mdW5jdGlvbih0KXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oaT10LGEpOml9LGEuc29ydENob3Jkcz1mdW5jdGlvbih0
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obnVsbD09dD9vPW51bGw6KG49dCxvPWZ1bmN0aW9u
KHQsZSl7cmV0dXJuIG4odC5zb3VyY2UudmFsdWUrdC50YXJnZXQudmFsdWUsZS5zb3VyY2UudmFs
dWUrZS50YXJnZXQudmFsdWUpfSkuXz10LGEpOm8mJm8uXzt2YXIgbn0sYX1jb25zdCBEYT1NYXRo
LlBJLFJhPTIqRGEsRmE9MWUtNixxYT1SYS1GYTtmdW5jdGlvbiBVYSh0KXt0aGlzLl8rPXRbMF07
Zm9yKGxldCBuPTEsZT10Lmxlbmd0aDtuPGU7KytuKXRoaXMuXys9YXJndW1lbnRzW25dK3Rbbl19
bGV0IElhPWNsYXNze2NvbnN0cnVjdG9yKHQpe3RoaXMuX3gwPXRoaXMuX3kwPXRoaXMuX3gxPXRo
aXMuX3kxPW51bGwsdGhpcy5fPSIiLHRoaXMuX2FwcGVuZD1udWxsPT10P1VhOmZ1bmN0aW9uKHQp
e2xldCBuPU1hdGguZmxvb3IodCk7aWYoIShuPj0wKSl0aHJvdyBuZXcgRXJyb3IoYGludmFsaWQg
ZGlnaXRzOiAke3R9YCk7aWYobj4xNSlyZXR1cm4gVWE7Y29uc3QgZT0xMCoqbjtyZXR1cm4gZnVu
Y3Rpb24odCl7dGhpcy5fKz10WzBdO2ZvcihsZXQgbj0xLHI9dC5sZW5ndGg7bjxyOysrbil0aGlz
Ll8rPU1hdGgucm91bmQoYXJndW1lbnRzW25dKmUpL2UrdFtuXX19KHQpfW1vdmVUbyh0LG4pe3Ro
aXMuX2FwcGVuZGBNJHt0aGlzLl94MD10aGlzLl94MT0rdH0sJHt0aGlzLl95MD10aGlzLl95MT0r
bn1gfWNsb3NlUGF0aCgpe251bGwhPT10aGlzLl94MSYmKHRoaXMuX3gxPXRoaXMuX3gwLHRoaXMu
X3kxPXRoaXMuX3kwLHRoaXMuX2FwcGVuZGBaYCl9bGluZVRvKHQsbil7dGhpcy5fYXBwZW5kYEwk
e3RoaXMuX3gxPSt0fSwke3RoaXMuX3kxPStufWB9cXVhZHJhdGljQ3VydmVUbyh0LG4sZSxyKXt0
aGlzLl9hcHBlbmRgUSR7K3R9LCR7K259LCR7dGhpcy5feDE9K2V9LCR7dGhpcy5feTE9K3J9YH1i
ZXppZXJDdXJ2ZVRvKHQsbixlLHIsaSxvKXt0aGlzLl9hcHBlbmRgQyR7K3R9LCR7K259LCR7K2V9
LCR7K3J9LCR7dGhpcy5feDE9K2l9LCR7dGhpcy5feTE9K299YH1hcmNUbyh0LG4sZSxyLGkpe2lm
KHQ9K3Qsbj0rbixlPStlLHI9K3IsKGk9K2kpPDApdGhyb3cgbmV3IEVycm9yKGBuZWdhdGl2ZSBy
YWRpdXM6ICR7aX1gKTtsZXQgbz10aGlzLl94MSxhPXRoaXMuX3kxLHU9ZS10LGM9ci1uLGY9by10
LHM9YS1uLGw9ZipmK3MqcztpZihudWxsPT09dGhpcy5feDEpdGhpcy5fYXBwZW5kYE0ke3RoaXMu
X3gxPXR9LCR7dGhpcy5feTE9bn1gO2Vsc2UgaWYobD5GYSlpZihNYXRoLmFicyhzKnUtYypmKT5G
YSYmaSl7bGV0IGg9ZS1vLGQ9ci1hLHA9dSp1K2MqYyxnPWgqaCtkKmQseT1NYXRoLnNxcnQocCks
dj1NYXRoLnNxcnQobCksXz1pKk1hdGgudGFuKChEYS1NYXRoLmFjb3MoKHArbC1nKS8oMip5KnYp
KSkvMiksYj1fL3YsbT1fL3k7TWF0aC5hYnMoYi0xKT5GYSYmdGhpcy5fYXBwZW5kYEwke3QrYipm
fSwke24rYipzfWAsdGhpcy5fYXBwZW5kYEEke2l9LCR7aX0sMCwwLCR7KyhzKmg+ZipkKX0sJHt0
aGlzLl94MT10K20qdX0sJHt0aGlzLl95MT1uK20qY31gfWVsc2UgdGhpcy5fYXBwZW5kYEwke3Ro
aXMuX3gxPXR9LCR7dGhpcy5feTE9bn1gO2Vsc2U7fWFyYyh0LG4sZSxyLGksbyl7aWYodD0rdCxu
PStuLG89ISFvLChlPStlKTwwKXRocm93IG5ldyBFcnJvcihgbmVnYXRpdmUgcmFkaXVzOiAke2V9
YCk7bGV0IGE9ZSpNYXRoLmNvcyhyKSx1PWUqTWF0aC5zaW4ociksYz10K2EsZj1uK3Uscz0xXm8s
bD1vP3ItaTppLXI7bnVsbD09PXRoaXMuX3gxP3RoaXMuX2FwcGVuZGBNJHtjfSwke2Z9YDooTWF0
aC5hYnModGhpcy5feDEtYyk+RmF8fE1hdGguYWJzKHRoaXMuX3kxLWYpPkZhKSYmdGhpcy5fYXBw
ZW5kYEwke2N9LCR7Zn1gLGUmJihsPDAmJihsPWwlUmErUmEpLGw+cWE/dGhpcy5fYXBwZW5kYEEk
e2V9LCR7ZX0sMCwxLCR7c30sJHt0LWF9LCR7bi11fUEke2V9LCR7ZX0sMCwxLCR7c30sJHt0aGlz
Ll94MT1jfSwke3RoaXMuX3kxPWZ9YDpsPkZhJiZ0aGlzLl9hcHBlbmRgQSR7ZX0sJHtlfSwwLCR7
KyhsPj1EYSl9LCR7c30sJHt0aGlzLl94MT10K2UqTWF0aC5jb3MoaSl9LCR7dGhpcy5feTE9bitl
Kk1hdGguc2luKGkpfWApfXJlY3QodCxuLGUscil7dGhpcy5fYXBwZW5kYE0ke3RoaXMuX3gwPXRo
aXMuX3gxPSt0fSwke3RoaXMuX3kwPXRoaXMuX3kxPStufWgke2U9K2V9diR7K3J9aCR7LWV9WmB9
dG9TdHJpbmcoKXtyZXR1cm4gdGhpcy5ffX07ZnVuY3Rpb24gT2EoKXtyZXR1cm4gbmV3IElhfU9h
LnByb3RvdHlwZT1JYS5wcm90b3R5cGU7dmFyIEJhPUFycmF5LnByb3RvdHlwZS5zbGljZTtmdW5j
dGlvbiBZYSh0KXtyZXR1cm4gZnVuY3Rpb24oKXtyZXR1cm4gdH19ZnVuY3Rpb24gTGEodCl7cmV0
dXJuIHQuc291cmNlfWZ1bmN0aW9uIGphKHQpe3JldHVybiB0LnRhcmdldH1mdW5jdGlvbiBIYSh0
KXtyZXR1cm4gdC5yYWRpdXN9ZnVuY3Rpb24gWGEodCl7cmV0dXJuIHQuc3RhcnRBbmdsZX1mdW5j
dGlvbiBHYSh0KXtyZXR1cm4gdC5lbmRBbmdsZX1mdW5jdGlvbiBWYSgpe3JldHVybiAwfWZ1bmN0
aW9uIFdhKCl7cmV0dXJuIDEwfWZ1bmN0aW9uIFphKHQpe3ZhciBuPUxhLGU9amEscj1IYSxpPUhh
LG89WGEsYT1HYSx1PVZhLGM9bnVsbDtmdW5jdGlvbiBmKCl7dmFyIGYscz1uLmFwcGx5KHRoaXMs
YXJndW1lbnRzKSxsPWUuYXBwbHkodGhpcyxhcmd1bWVudHMpLGg9dS5hcHBseSh0aGlzLGFyZ3Vt
ZW50cykvMixkPUJhLmNhbGwoYXJndW1lbnRzKSxwPStyLmFwcGx5KHRoaXMsKGRbMF09cyxkKSks
Zz1vLmFwcGx5KHRoaXMsZCktTmEseT1hLmFwcGx5KHRoaXMsZCktTmEsdj0raS5hcHBseSh0aGlz
LChkWzBdPWwsZCkpLF89by5hcHBseSh0aGlzLGQpLU5hLGI9YS5hcHBseSh0aGlzLGQpLU5hO2lm
KGN8fChjPWY9T2EoKSksaD5QYSYmKFRhKHktZyk+MipoK1BhP3k+Zz8oZys9aCx5LT1oKTooZy09
aCx5Kz1oKTpnPXk9KGcreSkvMixUYShiLV8pPjIqaCtQYT9iPl8/KF8rPWgsYi09aCk6KF8tPWgs
Yis9aCk6Xz1iPShfK2IpLzIpLGMubW92ZVRvKHAqQWEoZykscCpTYShnKSksYy5hcmMoMCwwLHAs
Zyx5KSxnIT09X3x8eSE9PWIpaWYodCl7dmFyIG09di0rdC5hcHBseSh0aGlzLGFyZ3VtZW50cyks
eD0oXytiKS8yO2MucXVhZHJhdGljQ3VydmVUbygwLDAsbSpBYShfKSxtKlNhKF8pKSxjLmxpbmVU
byh2KkFhKHgpLHYqU2EoeCkpLGMubGluZVRvKG0qQWEoYiksbSpTYShiKSl9ZWxzZSBjLnF1YWRy
YXRpY0N1cnZlVG8oMCwwLHYqQWEoXyksdipTYShfKSksYy5hcmMoMCwwLHYsXyxiKTtpZihjLnF1
YWRyYXRpY0N1cnZlVG8oMCwwLHAqQWEoZykscCpTYShnKSksYy5jbG9zZVBhdGgoKSxmKXJldHVy
biBjPW51bGwsZisiInx8bnVsbH1yZXR1cm4gdCYmKGYuaGVhZFJhZGl1cz1mdW5jdGlvbihuKXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOllhKCtuKSxm
KTp0fSksZi5yYWRpdXM9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9aT0i
ZnVuY3Rpb24iPT10eXBlb2YgdD90OllhKCt0KSxmKTpyfSxmLnNvdXJjZVJhZGl1cz1mdW5jdGlv
bih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj0iZnVuY3Rpb24iPT10eXBlb2YgdD90Ollh
KCt0KSxmKTpyfSxmLnRhcmdldFJhZGl1cz1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxl
bmd0aD8oaT0iZnVuY3Rpb24iPT10eXBlb2YgdD90OllhKCt0KSxmKTppfSxmLnN0YXJ0QW5nbGU9
ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG89ImZ1bmN0aW9uIj09dHlwZW9m
IHQ/dDpZYSgrdCksZik6b30sZi5lbmRBbmdsZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD8oYT0iZnVuY3Rpb24iPT10eXBlb2YgdD90OllhKCt0KSxmKTphfSxmLnBhZEFuZ2xl
PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh1PSJmdW5jdGlvbiI9PXR5cGVv
ZiB0P3Q6WWEoK3QpLGYpOnV9LGYuc291cmNlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPyhuPXQsZik6bn0sZi50YXJnZXQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/KGU9dCxmKTplfSxmLmNvbnRleHQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/KGM9bnVsbD09dD9udWxsOnQsZik6Y30sZn12YXIgS2E9QXJyYXkucHJvdG90eXBlLnNs
aWNlO2Z1bmN0aW9uIFFhKHQsbil7cmV0dXJuIHQtbn12YXIgSmE9dD0+KCk9PnQ7ZnVuY3Rpb24g
dHUodCxuKXtmb3IodmFyIGUscj0tMSxpPW4ubGVuZ3RoOysrcjxpOylpZihlPW51KHQsbltyXSkp
cmV0dXJuIGU7cmV0dXJuIDB9ZnVuY3Rpb24gbnUodCxuKXtmb3IodmFyIGU9blswXSxyPW5bMV0s
aT0tMSxvPTAsYT10Lmxlbmd0aCx1PWEtMTtvPGE7dT1vKyspe3ZhciBjPXRbb10sZj1jWzBdLHM9
Y1sxXSxsPXRbdV0saD1sWzBdLGQ9bFsxXTtpZihldShjLGwsbikpcmV0dXJuIDA7cz5yIT1kPnIm
JmU8KGgtZikqKHItcykvKGQtcykrZiYmKGk9LWkpfXJldHVybiBpfWZ1bmN0aW9uIGV1KHQsbixl
KXt2YXIgcixpLG8sYTtyZXR1cm4gZnVuY3Rpb24odCxuLGUpe3JldHVybihuWzBdLXRbMF0pKihl
WzFdLXRbMV0pPT0oZVswXS10WzBdKSooblsxXS10WzFdKX0odCxuLGUpJiYoaT10W3I9Kyh0WzBd
PT09blswXSldLG89ZVtyXSxhPW5bcl0saTw9byYmbzw9YXx8YTw9byYmbzw9aSl9ZnVuY3Rpb24g
cnUoKXt9dmFyIGl1PVtbXSxbW1sxLDEuNV0sWy41LDFdXV0sW1tbMS41LDFdLFsxLDEuNV1dXSxb
W1sxLjUsMV0sWy41LDFdXV0sW1tbMSwuNV0sWzEuNSwxXV1dLFtbWzEsMS41XSxbLjUsMV1dLFtb
MSwuNV0sWzEuNSwxXV1dLFtbWzEsLjVdLFsxLDEuNV1dXSxbW1sxLC41XSxbLjUsMV1dXSxbW1su
NSwxXSxbMSwuNV1dXSxbW1sxLDEuNV0sWzEsLjVdXV0sW1tbLjUsMV0sWzEsLjVdXSxbWzEuNSwx
XSxbMSwxLjVdXV0sW1tbMS41LDFdLFsxLC41XV1dLFtbWy41LDFdLFsxLjUsMV1dXSxbW1sxLDEu
NV0sWzEuNSwxXV1dLFtbWy41LDFdLFsxLDEuNV1dXSxbXV07ZnVuY3Rpb24gb3UoKXt2YXIgdD0x
LG49MSxlPVEscj11O2Z1bmN0aW9uIGkodCl7dmFyIG49ZSh0KTtpZihBcnJheS5pc0FycmF5KG4p
KW49bi5zbGljZSgpLnNvcnQoUWEpO2Vsc2V7Y29uc3QgZT1UKHQsYXUpO2ZvcihuPVYoLi4uSyhl
WzBdLGVbMV0sbiksbik7bltuLmxlbmd0aC0xXT49ZVsxXTspbi5wb3AoKTtmb3IoO25bMV08ZVsw
XTspbi5zaGlmdCgpfXJldHVybiBuLm1hcCgobj0+byh0LG4pKSl9ZnVuY3Rpb24gbyhlLGkpe2Nv
bnN0IG89bnVsbD09aT9OYU46K2k7aWYoaXNOYU4obykpdGhyb3cgbmV3IEVycm9yKGBpbnZhbGlk
IHZhbHVlOiAke2l9YCk7dmFyIHU9W10sYz1bXTtyZXR1cm4gZnVuY3Rpb24oZSxyLGkpe3ZhciBv
LHUsYyxmLHMsbCxoPW5ldyBBcnJheSxkPW5ldyBBcnJheTtvPXU9LTEsZj11dShlWzBdLHIpLGl1
W2Y8PDFdLmZvckVhY2gocCk7Zm9yKDsrK288dC0xOyljPWYsZj11dShlW28rMV0sciksaXVbY3xm
PDwxXS5mb3JFYWNoKHApO2l1W2Y8PDBdLmZvckVhY2gocCk7Zm9yKDsrK3U8bi0xOyl7Zm9yKG89
LTEsZj11dShlW3UqdCt0XSxyKSxzPXV1KGVbdSp0XSxyKSxpdVtmPDwxfHM8PDJdLmZvckVhY2go
cCk7KytvPHQtMTspYz1mLGY9dXUoZVt1KnQrdCtvKzFdLHIpLGw9cyxzPXV1KGVbdSp0K28rMV0s
ciksaXVbY3xmPDwxfHM8PDJ8bDw8M10uZm9yRWFjaChwKTtpdVtmfHM8PDNdLmZvckVhY2gocCl9
bz0tMSxzPWVbdSp0XT49cixpdVtzPDwyXS5mb3JFYWNoKHApO2Zvcig7KytvPHQtMTspbD1zLHM9
dXUoZVt1KnQrbysxXSxyKSxpdVtzPDwyfGw8PDNdLmZvckVhY2gocCk7ZnVuY3Rpb24gcCh0KXt2
YXIgbixlLHI9W3RbMF1bMF0rbyx0WzBdWzFdK3VdLGM9W3RbMV1bMF0rbyx0WzFdWzFdK3VdLGY9
YShyKSxzPWEoYyk7KG49ZFtmXSk/KGU9aFtzXSk/KGRlbGV0ZSBkW24uZW5kXSxkZWxldGUgaFtl
LnN0YXJ0XSxuPT09ZT8obi5yaW5nLnB1c2goYyksaShuLnJpbmcpKTpoW24uc3RhcnRdPWRbZS5l
bmRdPXtzdGFydDpuLnN0YXJ0LGVuZDplLmVuZCxyaW5nOm4ucmluZy5jb25jYXQoZS5yaW5nKX0p
OihkZWxldGUgZFtuLmVuZF0sbi5yaW5nLnB1c2goYyksZFtuLmVuZD1zXT1uKToobj1oW3NdKT8o
ZT1kW2ZdKT8oZGVsZXRlIGhbbi5zdGFydF0sZGVsZXRlIGRbZS5lbmRdLG49PT1lPyhuLnJpbmcu
cHVzaChjKSxpKG4ucmluZykpOmhbZS5zdGFydF09ZFtuLmVuZF09e3N0YXJ0OmUuc3RhcnQsZW5k
Om4uZW5kLHJpbmc6ZS5yaW5nLmNvbmNhdChuLnJpbmcpfSk6KGRlbGV0ZSBoW24uc3RhcnRdLG4u
cmluZy51bnNoaWZ0KHIpLGhbbi5zdGFydD1mXT1uKTpoW2ZdPWRbc109e3N0YXJ0OmYsZW5kOnMs
cmluZzpbcixjXX19aXVbczw8M10uZm9yRWFjaChwKX0oZSxvLChmdW5jdGlvbih0KXtyKHQsZSxv
KSxmdW5jdGlvbih0KXtmb3IodmFyIG49MCxlPXQubGVuZ3RoLHI9dFtlLTFdWzFdKnRbMF1bMF0t
dFtlLTFdWzBdKnRbMF1bMV07KytuPGU7KXIrPXRbbi0xXVsxXSp0W25dWzBdLXRbbi0xXVswXSp0
W25dWzFdO3JldHVybiByfSh0KT4wP3UucHVzaChbdF0pOmMucHVzaCh0KX0pKSxjLmZvckVhY2go
KGZ1bmN0aW9uKHQpe2Zvcih2YXIgbixlPTAscj11Lmxlbmd0aDtlPHI7KytlKWlmKC0xIT09dHUo
KG49dVtlXSlbMF0sdCkpcmV0dXJuIHZvaWQgbi5wdXNoKHQpfSkpLHt0eXBlOiJNdWx0aVBvbHln
b24iLHZhbHVlOmksY29vcmRpbmF0ZXM6dX19ZnVuY3Rpb24gYShuKXtyZXR1cm4gMipuWzBdK25b
MV0qKHQrMSkqNH1mdW5jdGlvbiB1KGUscixpKXtlLmZvckVhY2goKGZ1bmN0aW9uKGUpe3ZhciBv
PWVbMF0sYT1lWzFdLHU9MHxvLGM9MHxhLGY9Y3UocltjKnQrdV0pO28+MCYmbzx0JiZ1PT09byYm
KGVbMF09ZnUobyxjdShyW2MqdCt1LTFdKSxmLGkpKSxhPjAmJmE8biYmYz09PWEmJihlWzFdPWZ1
KGEsY3UoclsoYy0xKSp0K3VdKSxmLGkpKX0pKX1yZXR1cm4gaS5jb250b3VyPW8saS5zaXplPWZ1
bmN0aW9uKGUpe2lmKCFhcmd1bWVudHMubGVuZ3RoKXJldHVyblt0LG5dO3ZhciByPU1hdGguZmxv
b3IoZVswXSksbz1NYXRoLmZsb29yKGVbMV0pO2lmKCEocj49MCYmbz49MCkpdGhyb3cgbmV3IEVy
cm9yKCJpbnZhbGlkIHNpemUiKTtyZXR1cm4gdD1yLG49byxpfSxpLnRocmVzaG9sZHM9ZnVuY3Rp
b24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGU9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpB
cnJheS5pc0FycmF5KHQpP0phKEthLmNhbGwodCkpOkphKHQpLGkpOmV9LGkuc21vb3RoPWZ1bmN0
aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhyPXQ/dTpydSxpKTpyPT09dX0saX1mdW5j
dGlvbiBhdSh0KXtyZXR1cm4gaXNGaW5pdGUodCk/dDpOYU59ZnVuY3Rpb24gdXUodCxuKXtyZXR1
cm4gbnVsbCE9dCYmK3Q+PW59ZnVuY3Rpb24gY3UodCl7cmV0dXJuIG51bGw9PXR8fGlzTmFOKHQ9
K3QpPy0xLzA6dH1mdW5jdGlvbiBmdSh0LG4sZSxyKXtjb25zdCBpPXItbixvPWUtbixhPWlzRmlu
aXRlKGkpfHxpc0Zpbml0ZShvKT9pL286TWF0aC5zaWduKGkpL01hdGguc2lnbihvKTtyZXR1cm4g
aXNOYU4oYSk/dDp0K2EtLjV9ZnVuY3Rpb24gc3UodCl7cmV0dXJuIHRbMF19ZnVuY3Rpb24gbHUo
dCl7cmV0dXJuIHRbMV19ZnVuY3Rpb24gaHUoKXtyZXR1cm4gMX1jb25zdCBkdT0xMzQyMTc3Mjks
cHU9MzMzMDY2OTA3Mzg3NTQ3MDZlLTMyO2Z1bmN0aW9uIGd1KHQsbixlLHIsaSl7bGV0IG8sYSx1
LGMsZj1uWzBdLHM9clswXSxsPTAsaD0wO3M+Zj09cz4tZj8obz1mLGY9blsrK2xdKToobz1zLHM9
clsrK2hdKTtsZXQgZD0wO2lmKGw8dCYmaDxlKWZvcihzPmY9PXM+LWY/KGE9ZitvLHU9by0oYS1m
KSxmPW5bKytsXSk6KGE9cytvLHU9by0oYS1zKSxzPXJbKytoXSksbz1hLDAhPT11JiYoaVtkKytd
PXUpO2w8dCYmaDxlOylzPmY9PXM+LWY/KGE9bytmLGM9YS1vLHU9by0oYS1jKSsoZi1jKSxmPW5b
KytsXSk6KGE9bytzLGM9YS1vLHU9by0oYS1jKSsocy1jKSxzPXJbKytoXSksbz1hLDAhPT11JiYo
aVtkKytdPXUpO2Zvcig7bDx0OylhPW8rZixjPWEtbyx1PW8tKGEtYykrKGYtYyksZj1uWysrbF0s
bz1hLDAhPT11JiYoaVtkKytdPXUpO2Zvcig7aDxlOylhPW8rcyxjPWEtbyx1PW8tKGEtYykrKHMt
Yykscz1yWysraF0sbz1hLDAhPT11JiYoaVtkKytdPXUpO3JldHVybiAwPT09byYmMCE9PWR8fChp
W2QrK109byksZH1mdW5jdGlvbiB5dSh0KXtyZXR1cm4gbmV3IEZsb2F0NjRBcnJheSh0KX1jb25z
dCB2dT0yMjIwNDQ2MDQ5MjUwMzE0NmUtMzIsX3U9MTEwOTMzNTY0Nzk2NzA0ODdlLTQ3LGJ1PXl1
KDQpLG11PXl1KDgpLHh1PXl1KDEyKSx3dT15dSgxNiksTXU9eXUoNCk7ZnVuY3Rpb24gVHUodCxu
LGUscixpLG8pe2NvbnN0IGE9KG4tbykqKGUtaSksdT0odC1pKSooci1vKSxjPWEtdTtpZigwPT09
YXx8MD09PXV8fGE+MCE9dT4wKXJldHVybiBjO2NvbnN0IGY9TWF0aC5hYnMoYSt1KTtyZXR1cm4g
TWF0aC5hYnMoYyk+PTMzMzA2NjkwNzM4NzU0NzE2ZS0zMipmP2M6LWZ1bmN0aW9uKHQsbixlLHIs
aSxvLGEpe2xldCB1LGMsZixzLGwsaCxkLHAsZyx5LHYsXyxiLG0seCx3LE0sVDtjb25zdCBBPXQt
aSxTPWUtaSxFPW4tbyxOPXItbzttPUEqTixoPWR1KkEsZD1oLShoLUEpLHA9QS1kLGg9ZHUqTixn
PWgtKGgtTikseT1OLWcseD1wKnktKG0tZCpnLXAqZy1kKnkpLHc9RSpTLGg9ZHUqRSxkPWgtKGgt
RSkscD1FLWQsaD1kdSpTLGc9aC0oaC1TKSx5PVMtZyxNPXAqeS0ody1kKmctcCpnLWQqeSksdj14
LU0sbD14LXYsYnVbMF09eC0oditsKSsobC1NKSxfPW0rdixsPV8tbSxiPW0tKF8tbCkrKHYtbCks
dj1iLXcsbD1iLXYsYnVbMV09Yi0oditsKSsobC13KSxUPV8rdixsPVQtXyxidVsyXT1fLShULWwp
Kyh2LWwpLGJ1WzNdPVQ7bGV0IGs9ZnVuY3Rpb24odCxuKXtsZXQgZT1uWzBdO2ZvcihsZXQgcj0x
O3I8dDtyKyspZSs9bltyXTtyZXR1cm4gZX0oNCxidSksQz12dSphO2lmKGs+PUN8fC1rPj1DKXJl
dHVybiBrO2lmKGw9dC1BLHU9dC0oQStsKSsobC1pKSxsPWUtUyxmPWUtKFMrbCkrKGwtaSksbD1u
LUUsYz1uLShFK2wpKyhsLW8pLGw9ci1OLHM9ci0oTitsKSsobC1vKSwwPT09dSYmMD09PWMmJjA9
PT1mJiYwPT09cylyZXR1cm4gaztpZihDPV91KmErcHUqTWF0aC5hYnMoayksays9QSpzK04qdS0o
RSpmK1MqYyksaz49Q3x8LWs+PUMpcmV0dXJuIGs7bT11Kk4saD1kdSp1LGQ9aC0oaC11KSxwPXUt
ZCxoPWR1Kk4sZz1oLShoLU4pLHk9Ti1nLHg9cCp5LShtLWQqZy1wKmctZCp5KSx3PWMqUyxoPWR1
KmMsZD1oLShoLWMpLHA9Yy1kLGg9ZHUqUyxnPWgtKGgtUykseT1TLWcsTT1wKnktKHctZCpnLXAq
Zy1kKnkpLHY9eC1NLGw9eC12LE11WzBdPXgtKHYrbCkrKGwtTSksXz1tK3YsbD1fLW0sYj1tLShf
LWwpKyh2LWwpLHY9Yi13LGw9Yi12LE11WzFdPWItKHYrbCkrKGwtdyksVD1fK3YsbD1ULV8sTXVb
Ml09Xy0oVC1sKSsodi1sKSxNdVszXT1UO2NvbnN0IFA9Z3UoNCxidSw0LE11LG11KTttPUEqcyxo
PWR1KkEsZD1oLShoLUEpLHA9QS1kLGg9ZHUqcyxnPWgtKGgtcykseT1zLWcseD1wKnktKG0tZCpn
LXAqZy1kKnkpLHc9RSpmLGg9ZHUqRSxkPWgtKGgtRSkscD1FLWQsaD1kdSpmLGc9aC0oaC1mKSx5
PWYtZyxNPXAqeS0ody1kKmctcCpnLWQqeSksdj14LU0sbD14LXYsTXVbMF09eC0oditsKSsobC1N
KSxfPW0rdixsPV8tbSxiPW0tKF8tbCkrKHYtbCksdj1iLXcsbD1iLXYsTXVbMV09Yi0oditsKSso
bC13KSxUPV8rdixsPVQtXyxNdVsyXT1fLShULWwpKyh2LWwpLE11WzNdPVQ7Y29uc3Qgej1ndShQ
LG11LDQsTXUseHUpO209dSpzLGg9ZHUqdSxkPWgtKGgtdSkscD11LWQsaD1kdSpzLGc9aC0oaC1z
KSx5PXMtZyx4PXAqeS0obS1kKmctcCpnLWQqeSksdz1jKmYsaD1kdSpjLGQ9aC0oaC1jKSxwPWMt
ZCxoPWR1KmYsZz1oLShoLWYpLHk9Zi1nLE09cCp5LSh3LWQqZy1wKmctZCp5KSx2PXgtTSxsPXgt
dixNdVswXT14LSh2K2wpKyhsLU0pLF89bSt2LGw9Xy1tLGI9bS0oXy1sKSsodi1sKSx2PWItdyxs
PWItdixNdVsxXT1iLSh2K2wpKyhsLXcpLFQ9Xyt2LGw9VC1fLE11WzJdPV8tKFQtbCkrKHYtbCks
TXVbM109VDtjb25zdCAkPWd1KHoseHUsNCxNdSx3dSk7cmV0dXJuIHd1WyQtMV19KHQsbixlLHIs
aSxvLGYpfWNvbnN0IEF1PU1hdGgucG93KDIsLTUyKSxTdT1uZXcgVWludDMyQXJyYXkoNTEyKTtj
bGFzcyBFdXtzdGF0aWMgZnJvbSh0LG49JHUsZT1EdSl7Y29uc3Qgcj10Lmxlbmd0aCxpPW5ldyBG
bG9hdDY0QXJyYXkoMipyKTtmb3IobGV0IG89MDtvPHI7bysrKXtjb25zdCByPXRbb107aVsyKm9d
PW4ociksaVsyKm8rMV09ZShyKX1yZXR1cm4gbmV3IEV1KGkpfWNvbnN0cnVjdG9yKHQpe2NvbnN0
IG49dC5sZW5ndGg+PjE7aWYobj4wJiYibnVtYmVyIiE9dHlwZW9mIHRbMF0pdGhyb3cgbmV3IEVy
cm9yKCJFeHBlY3RlZCBjb29yZHMgdG8gY29udGFpbiBudW1iZXJzLiIpO3RoaXMuY29vcmRzPXQ7
Y29uc3QgZT1NYXRoLm1heCgyKm4tNSwwKTt0aGlzLl90cmlhbmdsZXM9bmV3IFVpbnQzMkFycmF5
KDMqZSksdGhpcy5faGFsZmVkZ2VzPW5ldyBJbnQzMkFycmF5KDMqZSksdGhpcy5faGFzaFNpemU9
TWF0aC5jZWlsKE1hdGguc3FydChuKSksdGhpcy5faHVsbFByZXY9bmV3IFVpbnQzMkFycmF5KG4p
LHRoaXMuX2h1bGxOZXh0PW5ldyBVaW50MzJBcnJheShuKSx0aGlzLl9odWxsVHJpPW5ldyBVaW50
MzJBcnJheShuKSx0aGlzLl9odWxsSGFzaD1uZXcgSW50MzJBcnJheSh0aGlzLl9oYXNoU2l6ZSku
ZmlsbCgtMSksdGhpcy5faWRzPW5ldyBVaW50MzJBcnJheShuKSx0aGlzLl9kaXN0cz1uZXcgRmxv
YXQ2NEFycmF5KG4pLHRoaXMudXBkYXRlKCl9dXBkYXRlKCl7Y29uc3R7Y29vcmRzOnQsX2h1bGxQ
cmV2Om4sX2h1bGxOZXh0OmUsX2h1bGxUcmk6cixfaHVsbEhhc2g6aX09dGhpcyxvPXQubGVuZ3Ro
Pj4xO2xldCBhPTEvMCx1PTEvMCxjPS0xLzAsZj0tMS8wO2ZvcihsZXQgbj0wO248bztuKyspe2Nv
bnN0IGU9dFsyKm5dLHI9dFsyKm4rMV07ZTxhJiYoYT1lKSxyPHUmJih1PXIpLGU+YyYmKGM9ZSks
cj5mJiYoZj1yKSx0aGlzLl9pZHNbbl09bn1jb25zdCBzPShhK2MpLzIsbD0odStmKS8yO2xldCBo
LGQscCxnPTEvMDtmb3IobGV0IG49MDtuPG87bisrKXtjb25zdCBlPU51KHMsbCx0WzIqbl0sdFsy
Km4rMV0pO2U8ZyYmKGg9bixnPWUpfWNvbnN0IHk9dFsyKmhdLHY9dFsyKmgrMV07Zz0xLzA7Zm9y
KGxldCBuPTA7bjxvO24rKyl7aWYobj09PWgpY29udGludWU7Y29uc3QgZT1OdSh5LHYsdFsyKm5d
LHRbMipuKzFdKTtlPGcmJmU+MCYmKGQ9bixnPWUpfWxldCBfPXRbMipkXSxiPXRbMipkKzFdLG09
MS8wO2ZvcihsZXQgbj0wO248bztuKyspe2lmKG49PT1ofHxuPT09ZCljb250aW51ZTtjb25zdCBl
PUN1KHksdixfLGIsdFsyKm5dLHRbMipuKzFdKTtlPG0mJihwPW4sbT1lKX1sZXQgeD10WzIqcF0s
dz10WzIqcCsxXTtpZihtPT09MS8wKXtmb3IobGV0IG49MDtuPG87bisrKXRoaXMuX2Rpc3RzW25d
PXRbMipuXS10WzBdfHx0WzIqbisxXS10WzFdO1B1KHRoaXMuX2lkcyx0aGlzLl9kaXN0cywwLG8t
MSk7Y29uc3Qgbj1uZXcgVWludDMyQXJyYXkobyk7bGV0IGU9MDtmb3IobGV0IHQ9MCxyPS0xLzA7
dDxvO3QrKyl7Y29uc3QgaT10aGlzLl9pZHNbdF07dGhpcy5fZGlzdHNbaV0+ciYmKG5bZSsrXT1p
LHI9dGhpcy5fZGlzdHNbaV0pfXJldHVybiB0aGlzLmh1bGw9bi5zdWJhcnJheSgwLGUpLHRoaXMu
dHJpYW5nbGVzPW5ldyBVaW50MzJBcnJheSgwKSx2b2lkKHRoaXMuaGFsZmVkZ2VzPW5ldyBVaW50
MzJBcnJheSgwKSl9aWYoVHUoeSx2LF8sYix4LHcpPDApe2NvbnN0IHQ9ZCxuPV8sZT1iO2Q9cCxf
PXgsYj13LHA9dCx4PW4sdz1lfWNvbnN0IE09ZnVuY3Rpb24odCxuLGUscixpLG8pe2NvbnN0IGE9
ZS10LHU9ci1uLGM9aS10LGY9by1uLHM9YSphK3UqdSxsPWMqYytmKmYsaD0uNS8oYSpmLXUqYyks
ZD10KyhmKnMtdSpsKSpoLHA9bisoYSpsLWMqcykqaDtyZXR1cm57eDpkLHk6cH19KHksdixfLGIs
eCx3KTt0aGlzLl9jeD1NLngsdGhpcy5fY3k9TS55O2ZvcihsZXQgbj0wO248bztuKyspdGhpcy5f
ZGlzdHNbbl09TnUodFsyKm5dLHRbMipuKzFdLE0ueCxNLnkpO1B1KHRoaXMuX2lkcyx0aGlzLl9k
aXN0cywwLG8tMSksdGhpcy5faHVsbFN0YXJ0PWg7bGV0IFQ9MztlW2hdPW5bcF09ZCxlW2RdPW5b
aF09cCxlW3BdPW5bZF09aCxyW2hdPTAscltkXT0xLHJbcF09MixpLmZpbGwoLTEpLGlbdGhpcy5f
aGFzaEtleSh5LHYpXT1oLGlbdGhpcy5faGFzaEtleShfLGIpXT1kLGlbdGhpcy5faGFzaEtleSh4
LHcpXT1wLHRoaXMudHJpYW5nbGVzTGVuPTAsdGhpcy5fYWRkVHJpYW5nbGUoaCxkLHAsLTEsLTEs
LTEpO2ZvcihsZXQgbyxhLHU9MDt1PHRoaXMuX2lkcy5sZW5ndGg7dSsrKXtjb25zdCBjPXRoaXMu
X2lkc1t1XSxmPXRbMipjXSxzPXRbMipjKzFdO2lmKHU+MCYmTWF0aC5hYnMoZi1vKTw9QXUmJk1h
dGguYWJzKHMtYSk8PUF1KWNvbnRpbnVlO2lmKG89ZixhPXMsYz09PWh8fGM9PT1kfHxjPT09cClj
b250aW51ZTtsZXQgbD0wO2ZvcihsZXQgdD0wLG49dGhpcy5faGFzaEtleShmLHMpO3Q8dGhpcy5f
aGFzaFNpemUmJihsPWlbKG4rdCkldGhpcy5faGFzaFNpemVdLC0xPT09bHx8bD09PWVbbF0pO3Qr
Kyk7bD1uW2xdO2xldCBnLHk9bDtmb3IoO2c9ZVt5XSxUdShmLHMsdFsyKnldLHRbMip5KzFdLHRb
MipnXSx0WzIqZysxXSk+PTA7KWlmKHk9Zyx5PT09bCl7eT0tMTticmVha31pZigtMT09PXkpY29u
dGludWU7bGV0IHY9dGhpcy5fYWRkVHJpYW5nbGUoeSxjLGVbeV0sLTEsLTEsclt5XSk7cltjXT10
aGlzLl9sZWdhbGl6ZSh2KzIpLHJbeV09dixUKys7bGV0IF89ZVt5XTtmb3IoO2c9ZVtfXSxUdShm
LHMsdFsyKl9dLHRbMipfKzFdLHRbMipnXSx0WzIqZysxXSk8MDspdj10aGlzLl9hZGRUcmlhbmds
ZShfLGMsZyxyW2NdLC0xLHJbX10pLHJbY109dGhpcy5fbGVnYWxpemUodisyKSxlW19dPV8sVC0t
LF89ZztpZih5PT09bClmb3IoO2c9blt5XSxUdShmLHMsdFsyKmddLHRbMipnKzFdLHRbMip5XSx0
WzIqeSsxXSk8MDspdj10aGlzLl9hZGRUcmlhbmdsZShnLGMseSwtMSxyW3ldLHJbZ10pLHRoaXMu
X2xlZ2FsaXplKHYrMikscltnXT12LGVbeV09eSxULS0seT1nO3RoaXMuX2h1bGxTdGFydD1uW2Nd
PXksZVt5XT1uW19dPWMsZVtjXT1fLGlbdGhpcy5faGFzaEtleShmLHMpXT1jLGlbdGhpcy5faGFz
aEtleSh0WzIqeV0sdFsyKnkrMV0pXT15fXRoaXMuaHVsbD1uZXcgVWludDMyQXJyYXkoVCk7Zm9y
KGxldCB0PTAsbj10aGlzLl9odWxsU3RhcnQ7dDxUO3QrKyl0aGlzLmh1bGxbdF09bixuPWVbbl07
dGhpcy50cmlhbmdsZXM9dGhpcy5fdHJpYW5nbGVzLnN1YmFycmF5KDAsdGhpcy50cmlhbmdsZXNM
ZW4pLHRoaXMuaGFsZmVkZ2VzPXRoaXMuX2hhbGZlZGdlcy5zdWJhcnJheSgwLHRoaXMudHJpYW5n
bGVzTGVuKX1faGFzaEtleSh0LG4pe3JldHVybiBNYXRoLmZsb29yKGZ1bmN0aW9uKHQsbil7Y29u
c3QgZT10LyhNYXRoLmFicyh0KStNYXRoLmFicyhuKSk7cmV0dXJuKG4+MD8zLWU6MStlKS80fSh0
LXRoaXMuX2N4LG4tdGhpcy5fY3kpKnRoaXMuX2hhc2hTaXplKSV0aGlzLl9oYXNoU2l6ZX1fbGVn
YWxpemUodCl7Y29uc3R7X3RyaWFuZ2xlczpuLF9oYWxmZWRnZXM6ZSxjb29yZHM6cn09dGhpczts
ZXQgaT0wLG89MDtmb3IoOzspe2NvbnN0IGE9ZVt0XSx1PXQtdCUzO2lmKG89dSsodCsyKSUzLC0x
PT09YSl7aWYoMD09PWkpYnJlYWs7dD1TdVstLWldO2NvbnRpbnVlfWNvbnN0IGM9YS1hJTMsZj11
Kyh0KzEpJTMscz1jKyhhKzIpJTMsbD1uW29dLGg9blt0XSxkPW5bZl0scD1uW3NdO2lmKGt1KHJb
MipsXSxyWzIqbCsxXSxyWzIqaF0sclsyKmgrMV0sclsyKmRdLHJbMipkKzFdLHJbMipwXSxyWzIq
cCsxXSkpe25bdF09cCxuW2FdPWw7Y29uc3Qgcj1lW3NdO2lmKC0xPT09cil7bGV0IG49dGhpcy5f
aHVsbFN0YXJ0O2Rve2lmKHRoaXMuX2h1bGxUcmlbbl09PT1zKXt0aGlzLl9odWxsVHJpW25dPXQ7
YnJlYWt9bj10aGlzLl9odWxsUHJldltuXX13aGlsZShuIT09dGhpcy5faHVsbFN0YXJ0KX10aGlz
Ll9saW5rKHQsciksdGhpcy5fbGluayhhLGVbb10pLHRoaXMuX2xpbmsobyxzKTtjb25zdCB1PWMr
KGErMSklMztpPFN1Lmxlbmd0aCYmKFN1W2krK109dSl9ZWxzZXtpZigwPT09aSlicmVhazt0PVN1
Wy0taV19fXJldHVybiBvfV9saW5rKHQsbil7dGhpcy5faGFsZmVkZ2VzW3RdPW4sLTEhPT1uJiYo
dGhpcy5faGFsZmVkZ2VzW25dPXQpfV9hZGRUcmlhbmdsZSh0LG4sZSxyLGksbyl7Y29uc3QgYT10
aGlzLnRyaWFuZ2xlc0xlbjtyZXR1cm4gdGhpcy5fdHJpYW5nbGVzW2FdPXQsdGhpcy5fdHJpYW5n
bGVzW2ErMV09bix0aGlzLl90cmlhbmdsZXNbYSsyXT1lLHRoaXMuX2xpbmsoYSxyKSx0aGlzLl9s
aW5rKGErMSxpKSx0aGlzLl9saW5rKGErMixvKSx0aGlzLnRyaWFuZ2xlc0xlbis9MyxhfX1mdW5j
dGlvbiBOdSh0LG4sZSxyKXtjb25zdCBpPXQtZSxvPW4tcjtyZXR1cm4gaSppK28qb31mdW5jdGlv
biBrdSh0LG4sZSxyLGksbyxhLHUpe2NvbnN0IGM9dC1hLGY9bi11LHM9ZS1hLGw9ci11LGg9aS1h
LGQ9by11LHA9cypzK2wqbCxnPWgqaCtkKmQ7cmV0dXJuIGMqKGwqZy1wKmQpLWYqKHMqZy1wKmgp
KyhjKmMrZipmKSoocypkLWwqaCk8MH1mdW5jdGlvbiBDdSh0LG4sZSxyLGksbyl7Y29uc3QgYT1l
LXQsdT1yLW4sYz1pLXQsZj1vLW4scz1hKmErdSp1LGw9YypjK2YqZixoPS41LyhhKmYtdSpjKSxk
PShmKnMtdSpsKSpoLHA9KGEqbC1jKnMpKmg7cmV0dXJuIGQqZCtwKnB9ZnVuY3Rpb24gUHUodCxu
LGUscil7aWYoci1lPD0yMClmb3IobGV0IGk9ZSsxO2k8PXI7aSsrKXtjb25zdCByPXRbaV0sbz1u
W3JdO2xldCBhPWktMTtmb3IoO2E+PWUmJm5bdFthXV0+bzspdFthKzFdPXRbYS0tXTt0W2ErMV09
cn1lbHNle2xldCBpPWUrMSxvPXI7enUodCxlK3I+PjEsaSksblt0W2VdXT5uW3Rbcl1dJiZ6dSh0
LGUsciksblt0W2ldXT5uW3Rbcl1dJiZ6dSh0LGksciksblt0W2VdXT5uW3RbaV1dJiZ6dSh0LGUs
aSk7Y29uc3QgYT10W2ldLHU9blthXTtmb3IoOzspe2Rve2krK313aGlsZShuW3RbaV1dPHUpO2Rv
e28tLX13aGlsZShuW3Rbb11dPnUpO2lmKG88aSlicmVhazt6dSh0LGksbyl9dFtlKzFdPXRbb10s
dFtvXT1hLHItaSsxPj1vLWU/KFB1KHQsbixpLHIpLFB1KHQsbixlLG8tMSkpOihQdSh0LG4sZSxv
LTEpLFB1KHQsbixpLHIpKX19ZnVuY3Rpb24genUodCxuLGUpe2NvbnN0IHI9dFtuXTt0W25dPXRb
ZV0sdFtlXT1yfWZ1bmN0aW9uICR1KHQpe3JldHVybiB0WzBdfWZ1bmN0aW9uIER1KHQpe3JldHVy
biB0WzFdfWNvbnN0IFJ1PTFlLTY7Y2xhc3MgRnV7Y29uc3RydWN0b3IoKXt0aGlzLl94MD10aGlz
Ll95MD10aGlzLl94MT10aGlzLl95MT1udWxsLHRoaXMuXz0iIn1tb3ZlVG8odCxuKXt0aGlzLl8r
PWBNJHt0aGlzLl94MD10aGlzLl94MT0rdH0sJHt0aGlzLl95MD10aGlzLl95MT0rbn1gfWNsb3Nl
UGF0aCgpe251bGwhPT10aGlzLl94MSYmKHRoaXMuX3gxPXRoaXMuX3gwLHRoaXMuX3kxPXRoaXMu
X3kwLHRoaXMuXys9IloiKX1saW5lVG8odCxuKXt0aGlzLl8rPWBMJHt0aGlzLl94MT0rdH0sJHt0
aGlzLl95MT0rbn1gfWFyYyh0LG4sZSl7Y29uc3Qgcj0odD0rdCkrKGU9K2UpLGk9bj0rbjtpZihl
PDApdGhyb3cgbmV3IEVycm9yKCJuZWdhdGl2ZSByYWRpdXMiKTtudWxsPT09dGhpcy5feDE/dGhp
cy5fKz1gTSR7cn0sJHtpfWA6KE1hdGguYWJzKHRoaXMuX3gxLXIpPlJ1fHxNYXRoLmFicyh0aGlz
Ll95MS1pKT5SdSkmJih0aGlzLl8rPSJMIityKyIsIitpKSxlJiYodGhpcy5fKz1gQSR7ZX0sJHtl
fSwwLDEsMSwke3QtZX0sJHtufUEke2V9LCR7ZX0sMCwxLDEsJHt0aGlzLl94MT1yfSwke3RoaXMu
X3kxPWl9YCl9cmVjdCh0LG4sZSxyKXt0aGlzLl8rPWBNJHt0aGlzLl94MD10aGlzLl94MT0rdH0s
JHt0aGlzLl95MD10aGlzLl95MT0rbn1oJHsrZX12JHsrcn1oJHstZX1aYH12YWx1ZSgpe3JldHVy
biB0aGlzLl98fG51bGx9fWNsYXNzIHF1e2NvbnN0cnVjdG9yKCl7dGhpcy5fPVtdfW1vdmVUbyh0
LG4pe3RoaXMuXy5wdXNoKFt0LG5dKX1jbG9zZVBhdGgoKXt0aGlzLl8ucHVzaCh0aGlzLl9bMF0u
c2xpY2UoKSl9bGluZVRvKHQsbil7dGhpcy5fLnB1c2goW3Qsbl0pfXZhbHVlKCl7cmV0dXJuIHRo
aXMuXy5sZW5ndGg/dGhpcy5fOm51bGx9fWNsYXNzIFV1e2NvbnN0cnVjdG9yKHQsW24sZSxyLGld
PVswLDAsOTYwLDUwMF0pe2lmKCEoKHI9K3IpPj0obj0rbikmJihpPStpKT49KGU9K2UpKSl0aHJv
dyBuZXcgRXJyb3IoImludmFsaWQgYm91bmRzIik7dGhpcy5kZWxhdW5heT10LHRoaXMuX2NpcmN1
bWNlbnRlcnM9bmV3IEZsb2F0NjRBcnJheSgyKnQucG9pbnRzLmxlbmd0aCksdGhpcy52ZWN0b3Jz
PW5ldyBGbG9hdDY0QXJyYXkoMip0LnBvaW50cy5sZW5ndGgpLHRoaXMueG1heD1yLHRoaXMueG1p
bj1uLHRoaXMueW1heD1pLHRoaXMueW1pbj1lLHRoaXMuX2luaXQoKX11cGRhdGUoKXtyZXR1cm4g
dGhpcy5kZWxhdW5heS51cGRhdGUoKSx0aGlzLl9pbml0KCksdGhpc31faW5pdCgpe2NvbnN0e2Rl
bGF1bmF5Ontwb2ludHM6dCxodWxsOm4sdHJpYW5nbGVzOmV9LHZlY3RvcnM6cn09dGhpcztsZXQg
aSxvO2NvbnN0IGE9dGhpcy5jaXJjdW1jZW50ZXJzPXRoaXMuX2NpcmN1bWNlbnRlcnMuc3ViYXJy
YXkoMCxlLmxlbmd0aC8zKjIpO2ZvcihsZXQgcix1LGM9MCxmPTAscz1lLmxlbmd0aDtjPHM7Yys9
MyxmKz0yKXtjb25zdCBzPTIqZVtjXSxsPTIqZVtjKzFdLGg9MiplW2MrMl0sZD10W3NdLHA9dFtz
KzFdLGc9dFtsXSx5PXRbbCsxXSx2PXRbaF0sXz10W2grMV0sYj1nLWQsbT15LXAseD12LWQsdz1f
LXAsTT0yKihiKnctbSp4KTtpZihNYXRoLmFicyhNKTwxZS05KXtpZih2b2lkIDA9PT1pKXtpPW89
MDtmb3IoY29uc3QgZSBvZiBuKWkrPXRbMiplXSxvKz10WzIqZSsxXTtpLz1uLmxlbmd0aCxvLz1u
Lmxlbmd0aH1jb25zdCBlPTFlOSpNYXRoLnNpZ24oKGktZCkqdy0oby1wKSp4KTtyPShkK3YpLzIt
ZSp3LHU9KHArXykvMitlKnh9ZWxzZXtjb25zdCB0PTEvTSxuPWIqYittKm0sZT14Kngrdyp3O3I9
ZCsodypuLW0qZSkqdCx1PXArKGIqZS14Km4pKnR9YVtmXT1yLGFbZisxXT11fWxldCB1LGMsZixz
PW5bbi5sZW5ndGgtMV0sbD00KnMsaD10WzIqc10sZD10WzIqcysxXTtyLmZpbGwoMCk7Zm9yKGxl
dCBlPTA7ZTxuLmxlbmd0aDsrK2Upcz1uW2VdLHU9bCxjPWgsZj1kLGw9NCpzLGg9dFsyKnNdLGQ9
dFsyKnMrMV0sclt1KzJdPXJbbF09Zi1kLHJbdSszXT1yW2wrMV09aC1jfXJlbmRlcih0KXtjb25z
dCBuPW51bGw9PXQ/dD1uZXcgRnU6dm9pZCAwLHtkZWxhdW5heTp7aGFsZmVkZ2VzOmUsaW5lZGdl
czpyLGh1bGw6aX0sY2lyY3VtY2VudGVyczpvLHZlY3RvcnM6YX09dGhpcztpZihpLmxlbmd0aDw9
MSlyZXR1cm4gbnVsbDtmb3IobGV0IG49MCxyPWUubGVuZ3RoO248cjsrK24pe2NvbnN0IHI9ZVtu
XTtpZihyPG4pY29udGludWU7Y29uc3QgaT0yKk1hdGguZmxvb3Iobi8zKSxhPTIqTWF0aC5mbG9v
cihyLzMpLHU9b1tpXSxjPW9baSsxXSxmPW9bYV0scz1vW2ErMV07dGhpcy5fcmVuZGVyU2VnbWVu
dCh1LGMsZixzLHQpfWxldCB1LGM9aVtpLmxlbmd0aC0xXTtmb3IobGV0IG49MDtuPGkubGVuZ3Ro
Oysrbil7dT1jLGM9aVtuXTtjb25zdCBlPTIqTWF0aC5mbG9vcihyW2NdLzMpLGY9b1tlXSxzPW9b
ZSsxXSxsPTQqdSxoPXRoaXMuX3Byb2plY3QoZixzLGFbbCsyXSxhW2wrM10pO2gmJnRoaXMuX3Jl
bmRlclNlZ21lbnQoZixzLGhbMF0saFsxXSx0KX1yZXR1cm4gbiYmbi52YWx1ZSgpfXJlbmRlckJv
dW5kcyh0KXtjb25zdCBuPW51bGw9PXQ/dD1uZXcgRnU6dm9pZCAwO3JldHVybiB0LnJlY3QodGhp
cy54bWluLHRoaXMueW1pbix0aGlzLnhtYXgtdGhpcy54bWluLHRoaXMueW1heC10aGlzLnltaW4p
LG4mJm4udmFsdWUoKX1yZW5kZXJDZWxsKHQsbil7Y29uc3QgZT1udWxsPT1uP249bmV3IEZ1OnZv
aWQgMCxyPXRoaXMuX2NsaXAodCk7aWYobnVsbD09PXJ8fCFyLmxlbmd0aClyZXR1cm47bi5tb3Zl
VG8oclswXSxyWzFdKTtsZXQgaT1yLmxlbmd0aDtmb3IoO3JbMF09PT1yW2ktMl0mJnJbMV09PT1y
W2ktMV0mJmk+MTspaS09Mjtmb3IobGV0IHQ9Mjt0PGk7dCs9MilyW3RdPT09clt0LTJdJiZyW3Qr
MV09PT1yW3QtMV18fG4ubGluZVRvKHJbdF0sclt0KzFdKTtyZXR1cm4gbi5jbG9zZVBhdGgoKSxl
JiZlLnZhbHVlKCl9KmNlbGxQb2x5Z29ucygpe2NvbnN0e2RlbGF1bmF5Ontwb2ludHM6dH19PXRo
aXM7Zm9yKGxldCBuPTAsZT10Lmxlbmd0aC8yO248ZTsrK24pe2NvbnN0IHQ9dGhpcy5jZWxsUG9s
eWdvbihuKTt0JiYodC5pbmRleD1uLHlpZWxkIHQpfX1jZWxsUG9seWdvbih0KXtjb25zdCBuPW5l
dyBxdTtyZXR1cm4gdGhpcy5yZW5kZXJDZWxsKHQsbiksbi52YWx1ZSgpfV9yZW5kZXJTZWdtZW50
KHQsbixlLHIsaSl7bGV0IG87Y29uc3QgYT10aGlzLl9yZWdpb25jb2RlKHQsbiksdT10aGlzLl9y
ZWdpb25jb2RlKGUscik7MD09PWEmJjA9PT11PyhpLm1vdmVUbyh0LG4pLGkubGluZVRvKGUscikp
OihvPXRoaXMuX2NsaXBTZWdtZW50KHQsbixlLHIsYSx1KSkmJihpLm1vdmVUbyhvWzBdLG9bMV0p
LGkubGluZVRvKG9bMl0sb1szXSkpfWNvbnRhaW5zKHQsbixlKXtyZXR1cm4obj0rbik9PW4mJihl
PStlKT09ZSYmdGhpcy5kZWxhdW5heS5fc3RlcCh0LG4sZSk9PT10fSpuZWlnaGJvcnModCl7Y29u
c3Qgbj10aGlzLl9jbGlwKHQpO2lmKG4pZm9yKGNvbnN0IGUgb2YgdGhpcy5kZWxhdW5heS5uZWln
aGJvcnModCkpe2NvbnN0IHQ9dGhpcy5fY2xpcChlKTtpZih0KXQ6Zm9yKGxldCByPTAsaT1uLmxl
bmd0aDtyPGk7cis9Milmb3IobGV0IG89MCxhPXQubGVuZ3RoO288YTtvKz0yKWlmKG5bcl09PT10
W29dJiZuW3IrMV09PT10W28rMV0mJm5bKHIrMiklaV09PT10WyhvK2EtMiklYV0mJm5bKHIrMykl
aV09PT10WyhvK2EtMSklYV0pe3lpZWxkIGU7YnJlYWsgdH19fV9jZWxsKHQpe2NvbnN0e2NpcmN1
bWNlbnRlcnM6bixkZWxhdW5heTp7aW5lZGdlczplLGhhbGZlZGdlczpyLHRyaWFuZ2xlczppfX09
dGhpcyxvPWVbdF07aWYoLTE9PT1vKXJldHVybiBudWxsO2NvbnN0IGE9W107bGV0IHU9bztkb3tj
b25zdCBlPU1hdGguZmxvb3IodS8zKTtpZihhLnB1c2goblsyKmVdLG5bMiplKzFdKSx1PXUlMz09
Mj91LTI6dSsxLGlbdV0hPT10KWJyZWFrO3U9clt1XX13aGlsZSh1IT09byYmLTEhPT11KTtyZXR1
cm4gYX1fY2xpcCh0KXtpZigwPT09dCYmMT09PXRoaXMuZGVsYXVuYXkuaHVsbC5sZW5ndGgpcmV0
dXJuW3RoaXMueG1heCx0aGlzLnltaW4sdGhpcy54bWF4LHRoaXMueW1heCx0aGlzLnhtaW4sdGhp
cy55bWF4LHRoaXMueG1pbix0aGlzLnltaW5dO2NvbnN0IG49dGhpcy5fY2VsbCh0KTtpZihudWxs
PT09bilyZXR1cm4gbnVsbDtjb25zdHt2ZWN0b3JzOmV9PXRoaXMscj00KnQ7cmV0dXJuIHRoaXMu
X3NpbXBsaWZ5KGVbcl18fGVbcisxXT90aGlzLl9jbGlwSW5maW5pdGUodCxuLGVbcl0sZVtyKzFd
LGVbcisyXSxlW3IrM10pOnRoaXMuX2NsaXBGaW5pdGUodCxuKSl9X2NsaXBGaW5pdGUodCxuKXtj
b25zdCBlPW4ubGVuZ3RoO2xldCByLGksbyxhLHU9bnVsbCxjPW5bZS0yXSxmPW5bZS0xXSxzPXRo
aXMuX3JlZ2lvbmNvZGUoYyxmKSxsPTA7Zm9yKGxldCBoPTA7aDxlO2grPTIpaWYocj1jLGk9Zixj
PW5baF0sZj1uW2grMV0sbz1zLHM9dGhpcy5fcmVnaW9uY29kZShjLGYpLDA9PT1vJiYwPT09cylh
PWwsbD0wLHU/dS5wdXNoKGMsZik6dT1bYyxmXTtlbHNle2xldCBuLGUsaCxkLHA7aWYoMD09PW8p
e2lmKG51bGw9PT0obj10aGlzLl9jbGlwU2VnbWVudChyLGksYyxmLG8scykpKWNvbnRpbnVlO1tl
LGgsZCxwXT1ufWVsc2V7aWYobnVsbD09PShuPXRoaXMuX2NsaXBTZWdtZW50KGMsZixyLGkscyxv
KSkpY29udGludWU7W2QscCxlLGhdPW4sYT1sLGw9dGhpcy5fZWRnZWNvZGUoZSxoKSxhJiZsJiZ0
aGlzLl9lZGdlKHQsYSxsLHUsdS5sZW5ndGgpLHU/dS5wdXNoKGUsaCk6dT1bZSxoXX1hPWwsbD10
aGlzLl9lZGdlY29kZShkLHApLGEmJmwmJnRoaXMuX2VkZ2UodCxhLGwsdSx1Lmxlbmd0aCksdT91
LnB1c2goZCxwKTp1PVtkLHBdfWlmKHUpYT1sLGw9dGhpcy5fZWRnZWNvZGUodVswXSx1WzFdKSxh
JiZsJiZ0aGlzLl9lZGdlKHQsYSxsLHUsdS5sZW5ndGgpO2Vsc2UgaWYodGhpcy5jb250YWlucyh0
LCh0aGlzLnhtaW4rdGhpcy54bWF4KS8yLCh0aGlzLnltaW4rdGhpcy55bWF4KS8yKSlyZXR1cm5b
dGhpcy54bWF4LHRoaXMueW1pbix0aGlzLnhtYXgsdGhpcy55bWF4LHRoaXMueG1pbix0aGlzLnlt
YXgsdGhpcy54bWluLHRoaXMueW1pbl07cmV0dXJuIHV9X2NsaXBTZWdtZW50KHQsbixlLHIsaSxv
KXtjb25zdCBhPWk8bztmb3IoYSYmKFt0LG4sZSxyLGksb109W2Uscix0LG4sbyxpXSk7Oyl7aWYo
MD09PWkmJjA9PT1vKXJldHVybiBhP1tlLHIsdCxuXTpbdCxuLGUscl07aWYoaSZvKXJldHVybiBu
dWxsO2xldCB1LGMsZj1pfHxvOzgmZj8odT10KyhlLXQpKih0aGlzLnltYXgtbikvKHItbiksYz10
aGlzLnltYXgpOjQmZj8odT10KyhlLXQpKih0aGlzLnltaW4tbikvKHItbiksYz10aGlzLnltaW4p
OjImZj8oYz1uKyhyLW4pKih0aGlzLnhtYXgtdCkvKGUtdCksdT10aGlzLnhtYXgpOihjPW4rKHIt
bikqKHRoaXMueG1pbi10KS8oZS10KSx1PXRoaXMueG1pbiksaT8odD11LG49YyxpPXRoaXMuX3Jl
Z2lvbmNvZGUodCxuKSk6KGU9dSxyPWMsbz10aGlzLl9yZWdpb25jb2RlKGUscikpfX1fY2xpcElu
ZmluaXRlKHQsbixlLHIsaSxvKXtsZXQgYSx1PUFycmF5LmZyb20obik7aWYoKGE9dGhpcy5fcHJv
amVjdCh1WzBdLHVbMV0sZSxyKSkmJnUudW5zaGlmdChhWzBdLGFbMV0pLChhPXRoaXMuX3Byb2pl
Y3QodVt1Lmxlbmd0aC0yXSx1W3UubGVuZ3RoLTFdLGksbykpJiZ1LnB1c2goYVswXSxhWzFdKSx1
PXRoaXMuX2NsaXBGaW5pdGUodCx1KSlmb3IobGV0IG4sZT0wLHI9dS5sZW5ndGgsaT10aGlzLl9l
ZGdlY29kZSh1W3ItMl0sdVtyLTFdKTtlPHI7ZSs9MiluPWksaT10aGlzLl9lZGdlY29kZSh1W2Vd
LHVbZSsxXSksbiYmaSYmKGU9dGhpcy5fZWRnZSh0LG4saSx1LGUpLHI9dS5sZW5ndGgpO2Vsc2Ug
dGhpcy5jb250YWlucyh0LCh0aGlzLnhtaW4rdGhpcy54bWF4KS8yLCh0aGlzLnltaW4rdGhpcy55
bWF4KS8yKSYmKHU9W3RoaXMueG1pbix0aGlzLnltaW4sdGhpcy54bWF4LHRoaXMueW1pbix0aGlz
LnhtYXgsdGhpcy55bWF4LHRoaXMueG1pbix0aGlzLnltYXhdKTtyZXR1cm4gdX1fZWRnZSh0LG4s
ZSxyLGkpe2Zvcig7biE9PWU7KXtsZXQgZSxvO3N3aXRjaChuKXtjYXNlIDU6bj00O2NvbnRpbnVl
O2Nhc2UgNDpuPTYsZT10aGlzLnhtYXgsbz10aGlzLnltaW47YnJlYWs7Y2FzZSA2Om49Mjtjb250
aW51ZTtjYXNlIDI6bj0xMCxlPXRoaXMueG1heCxvPXRoaXMueW1heDticmVhaztjYXNlIDEwOm49
ODtjb250aW51ZTtjYXNlIDg6bj05LGU9dGhpcy54bWluLG89dGhpcy55bWF4O2JyZWFrO2Nhc2Ug
OTpuPTE7Y29udGludWU7Y2FzZSAxOm49NSxlPXRoaXMueG1pbixvPXRoaXMueW1pbn1yW2ldPT09
ZSYmcltpKzFdPT09b3x8IXRoaXMuY29udGFpbnModCxlLG8pfHwoci5zcGxpY2UoaSwwLGUsbyks
aSs9Mil9cmV0dXJuIGl9X3Byb2plY3QodCxuLGUscil7bGV0IGksbyxhLHU9MS8wO2lmKHI8MCl7
aWYobjw9dGhpcy55bWluKXJldHVybiBudWxsOyhpPSh0aGlzLnltaW4tbikvcik8dSYmKGE9dGhp
cy55bWluLG89dCsodT1pKSplKX1lbHNlIGlmKHI+MCl7aWYobj49dGhpcy55bWF4KXJldHVybiBu
dWxsOyhpPSh0aGlzLnltYXgtbikvcik8dSYmKGE9dGhpcy55bWF4LG89dCsodT1pKSplKX1pZihl
PjApe2lmKHQ+PXRoaXMueG1heClyZXR1cm4gbnVsbDsoaT0odGhpcy54bWF4LXQpL2UpPHUmJihv
PXRoaXMueG1heCxhPW4rKHU9aSkqcil9ZWxzZSBpZihlPDApe2lmKHQ8PXRoaXMueG1pbilyZXR1
cm4gbnVsbDsoaT0odGhpcy54bWluLXQpL2UpPHUmJihvPXRoaXMueG1pbixhPW4rKHU9aSkqcil9
cmV0dXJuW28sYV19X2VkZ2Vjb2RlKHQsbil7cmV0dXJuKHQ9PT10aGlzLnhtaW4/MTp0PT09dGhp
cy54bWF4PzI6MCl8KG49PT10aGlzLnltaW4/NDpuPT09dGhpcy55bWF4Pzg6MCl9X3JlZ2lvbmNv
ZGUodCxuKXtyZXR1cm4odDx0aGlzLnhtaW4/MTp0PnRoaXMueG1heD8yOjApfChuPHRoaXMueW1p
bj80Om4+dGhpcy55bWF4Pzg6MCl9X3NpbXBsaWZ5KHQpe2lmKHQmJnQubGVuZ3RoPjQpe2Zvcihs
ZXQgbj0wO248dC5sZW5ndGg7bis9Mil7Y29uc3QgZT0obisyKSV0Lmxlbmd0aCxyPShuKzQpJXQu
bGVuZ3RoOyh0W25dPT09dFtlXSYmdFtlXT09PXRbcl18fHRbbisxXT09PXRbZSsxXSYmdFtlKzFd
PT09dFtyKzFdKSYmKHQuc3BsaWNlKGUsMiksbi09Mil9dC5sZW5ndGh8fCh0PW51bGwpfXJldHVy
biB0fX1jb25zdCBJdT0yKk1hdGguUEksT3U9TWF0aC5wb3c7ZnVuY3Rpb24gQnUodCl7cmV0dXJu
IHRbMF19ZnVuY3Rpb24gWXUodCl7cmV0dXJuIHRbMV19ZnVuY3Rpb24gTHUodCxuLGUpe3JldHVy
blt0K01hdGguc2luKHQrbikqZSxuK01hdGguY29zKHQtbikqZV19Y2xhc3MganV7c3RhdGljIGZy
b20odCxuPUJ1LGU9WXUscil7cmV0dXJuIG5ldyBqdSgibGVuZ3RoImluIHQ/ZnVuY3Rpb24odCxu
LGUscil7Y29uc3QgaT10Lmxlbmd0aCxvPW5ldyBGbG9hdDY0QXJyYXkoMippKTtmb3IobGV0IGE9
MDthPGk7KythKXtjb25zdCBpPXRbYV07b1syKmFdPW4uY2FsbChyLGksYSx0KSxvWzIqYSsxXT1l
LmNhbGwocixpLGEsdCl9cmV0dXJuIG99KHQsbixlLHIpOkZsb2F0NjRBcnJheS5mcm9tKGZ1bmN0
aW9uKih0LG4sZSxyKXtsZXQgaT0wO2Zvcihjb25zdCBvIG9mIHQpeWllbGQgbi5jYWxsKHIsbyxp
LHQpLHlpZWxkIGUuY2FsbChyLG8saSx0KSwrK2l9KHQsbixlLHIpKSl9Y29uc3RydWN0b3IodCl7
dGhpcy5fZGVsYXVuYXRvcj1uZXcgRXUodCksdGhpcy5pbmVkZ2VzPW5ldyBJbnQzMkFycmF5KHQu
bGVuZ3RoLzIpLHRoaXMuX2h1bGxJbmRleD1uZXcgSW50MzJBcnJheSh0Lmxlbmd0aC8yKSx0aGlz
LnBvaW50cz10aGlzLl9kZWxhdW5hdG9yLmNvb3Jkcyx0aGlzLl9pbml0KCl9dXBkYXRlKCl7cmV0
dXJuIHRoaXMuX2RlbGF1bmF0b3IudXBkYXRlKCksdGhpcy5faW5pdCgpLHRoaXN9X2luaXQoKXtj
b25zdCB0PXRoaXMuX2RlbGF1bmF0b3Isbj10aGlzLnBvaW50cztpZih0Lmh1bGwmJnQuaHVsbC5s
ZW5ndGg+MiYmZnVuY3Rpb24odCl7Y29uc3R7dHJpYW5nbGVzOm4sY29vcmRzOmV9PXQ7Zm9yKGxl
dCB0PTA7dDxuLmxlbmd0aDt0Kz0zKXtjb25zdCByPTIqblt0XSxpPTIqblt0KzFdLG89MipuW3Qr
Ml07aWYoKGVbb10tZVtyXSkqKGVbaSsxXS1lW3IrMV0pLShlW2ldLWVbcl0pKihlW28rMV0tZVty
KzFdKT4xZS0xMClyZXR1cm4hMX1yZXR1cm4hMH0odCkpe3RoaXMuY29sbGluZWFyPUludDMyQXJy
YXkuZnJvbSh7bGVuZ3RoOm4ubGVuZ3RoLzJ9LCgodCxuKT0+bikpLnNvcnQoKCh0LGUpPT5uWzIq
dF0tblsyKmVdfHxuWzIqdCsxXS1uWzIqZSsxXSkpO2NvbnN0IHQ9dGhpcy5jb2xsaW5lYXJbMF0s
ZT10aGlzLmNvbGxpbmVhclt0aGlzLmNvbGxpbmVhci5sZW5ndGgtMV0scj1bblsyKnRdLG5bMip0
KzFdLG5bMiplXSxuWzIqZSsxXV0saT0xZS04Kk1hdGguaHlwb3QoclszXS1yWzFdLHJbMl0tclsw
XSk7Zm9yKGxldCB0PTAsZT1uLmxlbmd0aC8yO3Q8ZTsrK3Qpe2NvbnN0IGU9THUoblsyKnRdLG5b
Mip0KzFdLGkpO25bMip0XT1lWzBdLG5bMip0KzFdPWVbMV19dGhpcy5fZGVsYXVuYXRvcj1uZXcg
RXUobil9ZWxzZSBkZWxldGUgdGhpcy5jb2xsaW5lYXI7Y29uc3QgZT10aGlzLmhhbGZlZGdlcz10
aGlzLl9kZWxhdW5hdG9yLmhhbGZlZGdlcyxyPXRoaXMuaHVsbD10aGlzLl9kZWxhdW5hdG9yLmh1
bGwsaT10aGlzLnRyaWFuZ2xlcz10aGlzLl9kZWxhdW5hdG9yLnRyaWFuZ2xlcyxvPXRoaXMuaW5l
ZGdlcy5maWxsKC0xKSxhPXRoaXMuX2h1bGxJbmRleC5maWxsKC0xKTtmb3IobGV0IHQ9MCxuPWUu
bGVuZ3RoO3Q8bjsrK3Qpe2NvbnN0IG49aVt0JTM9PTI/dC0yOnQrMV07LTEhPT1lW3RdJiYtMSE9
PW9bbl18fChvW25dPXQpfWZvcihsZXQgdD0wLG49ci5sZW5ndGg7dDxuOysrdClhW3JbdF1dPXQ7
ci5sZW5ndGg8PTImJnIubGVuZ3RoPjAmJih0aGlzLnRyaWFuZ2xlcz1uZXcgSW50MzJBcnJheSgz
KS5maWxsKC0xKSx0aGlzLmhhbGZlZGdlcz1uZXcgSW50MzJBcnJheSgzKS5maWxsKC0xKSx0aGlz
LnRyaWFuZ2xlc1swXT1yWzBdLG9bclswXV09MSwyPT09ci5sZW5ndGgmJihvW3JbMV1dPTAsdGhp
cy50cmlhbmdsZXNbMV09clsxXSx0aGlzLnRyaWFuZ2xlc1syXT1yWzFdKSl9dm9yb25vaSh0KXty
ZXR1cm4gbmV3IFV1KHRoaXMsdCl9Km5laWdoYm9ycyh0KXtjb25zdHtpbmVkZ2VzOm4saHVsbDpl
LF9odWxsSW5kZXg6cixoYWxmZWRnZXM6aSx0cmlhbmdsZXM6byxjb2xsaW5lYXI6YX09dGhpcztp
ZihhKXtjb25zdCBuPWEuaW5kZXhPZih0KTtyZXR1cm4gbj4wJiYoeWllbGQgYVtuLTFdKSx2b2lk
KG48YS5sZW5ndGgtMSYmKHlpZWxkIGFbbisxXSkpfWNvbnN0IHU9blt0XTtpZigtMT09PXUpcmV0
dXJuO2xldCBjPXUsZj0tMTtkb3tpZih5aWVsZCBmPW9bY10sYz1jJTM9PTI/Yy0yOmMrMSxvW2Nd
IT09dClyZXR1cm47aWYoYz1pW2NdLC0xPT09Yyl7Y29uc3Qgbj1lWyhyW3RdKzEpJWUubGVuZ3Ro
XTtyZXR1cm4gdm9pZChuIT09ZiYmKHlpZWxkIG4pKX19d2hpbGUoYyE9PXUpfWZpbmQodCxuLGU9
MCl7aWYoKHQ9K3QpIT10fHwobj0rbikhPW4pcmV0dXJuLTE7Y29uc3Qgcj1lO2xldCBpO2Zvcig7
KGk9dGhpcy5fc3RlcChlLHQsbikpPj0wJiZpIT09ZSYmaSE9PXI7KWU9aTtyZXR1cm4gaX1fc3Rl
cCh0LG4sZSl7Y29uc3R7aW5lZGdlczpyLGh1bGw6aSxfaHVsbEluZGV4Om8saGFsZmVkZ2VzOmEs
dHJpYW5nbGVzOnUscG9pbnRzOmN9PXRoaXM7aWYoLTE9PT1yW3RdfHwhYy5sZW5ndGgpcmV0dXJu
KHQrMSklKGMubGVuZ3RoPj4xKTtsZXQgZj10LHM9T3Uobi1jWzIqdF0sMikrT3UoZS1jWzIqdCsx
XSwyKTtjb25zdCBsPXJbdF07bGV0IGg9bDtkb3tsZXQgcj11W2hdO2NvbnN0IGw9T3Uobi1jWzIq
cl0sMikrT3UoZS1jWzIqcisxXSwyKTtpZihsPHMmJihzPWwsZj1yKSxoPWglMz09Mj9oLTI6aCsx
LHVbaF0hPT10KWJyZWFrO2lmKGg9YVtoXSwtMT09PWgpe2lmKGg9aVsob1t0XSsxKSVpLmxlbmd0
aF0saCE9PXImJk91KG4tY1syKmhdLDIpK091KGUtY1syKmgrMV0sMik8cylyZXR1cm4gaDticmVh
a319d2hpbGUoaCE9PWwpO3JldHVybiBmfXJlbmRlcih0KXtjb25zdCBuPW51bGw9PXQ/dD1uZXcg
RnU6dm9pZCAwLHtwb2ludHM6ZSxoYWxmZWRnZXM6cix0cmlhbmdsZXM6aX09dGhpcztmb3IobGV0
IG49MCxvPXIubGVuZ3RoO248bzsrK24pe2NvbnN0IG89cltuXTtpZihvPG4pY29udGludWU7Y29u
c3QgYT0yKmlbbl0sdT0yKmlbb107dC5tb3ZlVG8oZVthXSxlW2ErMV0pLHQubGluZVRvKGVbdV0s
ZVt1KzFdKX1yZXR1cm4gdGhpcy5yZW5kZXJIdWxsKHQpLG4mJm4udmFsdWUoKX1yZW5kZXJQb2lu
dHModCxuKXt2b2lkIDAhPT1ufHx0JiYiZnVuY3Rpb24iPT10eXBlb2YgdC5tb3ZlVG98fChuPXQs
dD1udWxsKSxuPW51bGw9PW4/Mjorbjtjb25zdCBlPW51bGw9PXQ/dD1uZXcgRnU6dm9pZCAwLHtw
b2ludHM6cn09dGhpcztmb3IobGV0IGU9MCxpPXIubGVuZ3RoO2U8aTtlKz0yKXtjb25zdCBpPXJb
ZV0sbz1yW2UrMV07dC5tb3ZlVG8oaStuLG8pLHQuYXJjKGksbyxuLDAsSXUpfXJldHVybiBlJiZl
LnZhbHVlKCl9cmVuZGVySHVsbCh0KXtjb25zdCBuPW51bGw9PXQ/dD1uZXcgRnU6dm9pZCAwLHto
dWxsOmUscG9pbnRzOnJ9PXRoaXMsaT0yKmVbMF0sbz1lLmxlbmd0aDt0Lm1vdmVUbyhyW2ldLHJb
aSsxXSk7Zm9yKGxldCBuPTE7bjxvOysrbil7Y29uc3QgaT0yKmVbbl07dC5saW5lVG8ocltpXSxy
W2krMV0pfXJldHVybiB0LmNsb3NlUGF0aCgpLG4mJm4udmFsdWUoKX1odWxsUG9seWdvbigpe2Nv
bnN0IHQ9bmV3IHF1O3JldHVybiB0aGlzLnJlbmRlckh1bGwodCksdC52YWx1ZSgpfXJlbmRlclRy
aWFuZ2xlKHQsbil7Y29uc3QgZT1udWxsPT1uP249bmV3IEZ1OnZvaWQgMCx7cG9pbnRzOnIsdHJp
YW5nbGVzOml9PXRoaXMsbz0yKmlbdCo9M10sYT0yKmlbdCsxXSx1PTIqaVt0KzJdO3JldHVybiBu
Lm1vdmVUbyhyW29dLHJbbysxXSksbi5saW5lVG8oclthXSxyW2ErMV0pLG4ubGluZVRvKHJbdV0s
clt1KzFdKSxuLmNsb3NlUGF0aCgpLGUmJmUudmFsdWUoKX0qdHJpYW5nbGVQb2x5Z29ucygpe2Nv
bnN0e3RyaWFuZ2xlczp0fT10aGlzO2ZvcihsZXQgbj0wLGU9dC5sZW5ndGgvMztuPGU7KytuKXlp
ZWxkIHRoaXMudHJpYW5nbGVQb2x5Z29uKG4pfXRyaWFuZ2xlUG9seWdvbih0KXtjb25zdCBuPW5l
dyBxdTtyZXR1cm4gdGhpcy5yZW5kZXJUcmlhbmdsZSh0LG4pLG4udmFsdWUoKX19dmFyIEh1PXt9
LFh1PXt9LEd1PTM0LFZ1PTEwLFd1PTEzO2Z1bmN0aW9uIFp1KHQpe3JldHVybiBuZXcgRnVuY3Rp
b24oImQiLCJyZXR1cm4geyIrdC5tYXAoKGZ1bmN0aW9uKHQsbil7cmV0dXJuIEpTT04uc3RyaW5n
aWZ5KHQpKyI6IGRbIituKyddIHx8ICIiJ30pKS5qb2luKCIsIikrIn0iKX1mdW5jdGlvbiBLdSh0
KXt2YXIgbj1PYmplY3QuY3JlYXRlKG51bGwpLGU9W107cmV0dXJuIHQuZm9yRWFjaCgoZnVuY3Rp
b24odCl7Zm9yKHZhciByIGluIHQpciBpbiBufHxlLnB1c2gobltyXT1yKX0pKSxlfWZ1bmN0aW9u
IFF1KHQsbil7dmFyIGU9dCsiIixyPWUubGVuZ3RoO3JldHVybiByPG4/bmV3IEFycmF5KG4tcisx
KS5qb2luKDApK2U6ZX1mdW5jdGlvbiBKdSh0KXt2YXIgbixlPXQuZ2V0VVRDSG91cnMoKSxyPXQu
Z2V0VVRDTWludXRlcygpLGk9dC5nZXRVVENTZWNvbmRzKCksbz10LmdldFVUQ01pbGxpc2Vjb25k
cygpO3JldHVybiBpc05hTih0KT8iSW52YWxpZCBEYXRlIjooKG49dC5nZXRVVENGdWxsWWVhcigp
KTwwPyItIitRdSgtbiw2KTpuPjk5OTk/IisiK1F1KG4sNik6UXUobiw0KSkrIi0iK1F1KHQuZ2V0
VVRDTW9udGgoKSsxLDIpKyItIitRdSh0LmdldFVUQ0RhdGUoKSwyKSsobz8iVCIrUXUoZSwyKSsi
OiIrUXUociwyKSsiOiIrUXUoaSwyKSsiLiIrUXUobywzKSsiWiI6aT8iVCIrUXUoZSwyKSsiOiIr
UXUociwyKSsiOiIrUXUoaSwyKSsiWiI6cnx8ZT8iVCIrUXUoZSwyKSsiOiIrUXUociwyKSsiWiI6
IiIpfWZ1bmN0aW9uIHRjKHQpe3ZhciBuPW5ldyBSZWdFeHAoJ1siJyt0KyJcblxyXSIpLGU9dC5j
aGFyQ29kZUF0KDApO2Z1bmN0aW9uIHIodCxuKXt2YXIgcixpPVtdLG89dC5sZW5ndGgsYT0wLHU9
MCxjPW88PTAsZj0hMTtmdW5jdGlvbiBzKCl7aWYoYylyZXR1cm4gWHU7aWYoZilyZXR1cm4gZj0h
MSxIdTt2YXIgbixyLGk9YTtpZih0LmNoYXJDb2RlQXQoaSk9PT1HdSl7Zm9yKDthKys8byYmdC5j
aGFyQ29kZUF0KGEpIT09R3V8fHQuY2hhckNvZGVBdCgrK2EpPT09R3U7KTtyZXR1cm4obj1hKT49
bz9jPSEwOihyPXQuY2hhckNvZGVBdChhKyspKT09PVZ1P2Y9ITA6cj09PVd1JiYoZj0hMCx0LmNo
YXJDb2RlQXQoYSk9PT1WdSYmKythKSx0LnNsaWNlKGkrMSxuLTEpLnJlcGxhY2UoLyIiL2csJyIn
KX1mb3IoO2E8bzspe2lmKChyPXQuY2hhckNvZGVBdChuPWErKykpPT09VnUpZj0hMDtlbHNlIGlm
KHI9PT1XdSlmPSEwLHQuY2hhckNvZGVBdChhKT09PVZ1JiYrK2E7ZWxzZSBpZihyIT09ZSljb250
aW51ZTtyZXR1cm4gdC5zbGljZShpLG4pfXJldHVybiBjPSEwLHQuc2xpY2UoaSxvKX1mb3IodC5j
aGFyQ29kZUF0KG8tMSk9PT1WdSYmLS1vLHQuY2hhckNvZGVBdChvLTEpPT09V3UmJi0tbzsocj1z
KCkpIT09WHU7KXtmb3IodmFyIGw9W107ciE9PUh1JiZyIT09WHU7KWwucHVzaChyKSxyPXMoKTtu
JiZudWxsPT0obD1uKGwsdSsrKSl8fGkucHVzaChsKX1yZXR1cm4gaX1mdW5jdGlvbiBpKG4sZSl7
cmV0dXJuIG4ubWFwKChmdW5jdGlvbihuKXtyZXR1cm4gZS5tYXAoKGZ1bmN0aW9uKHQpe3JldHVy
biBhKG5bdF0pfSkpLmpvaW4odCl9KSl9ZnVuY3Rpb24gbyhuKXtyZXR1cm4gbi5tYXAoYSkuam9p
bih0KX1mdW5jdGlvbiBhKHQpe3JldHVybiBudWxsPT10PyIiOnQgaW5zdGFuY2VvZiBEYXRlP0p1
KHQpOm4udGVzdCh0Kz0iIik/JyInK3QucmVwbGFjZSgvIi9nLCciIicpKyciJzp0fXJldHVybntw
YXJzZTpmdW5jdGlvbih0LG4pe3ZhciBlLGksbz1yKHQsKGZ1bmN0aW9uKHQscil7aWYoZSlyZXR1
cm4gZSh0LHItMSk7aT10LGU9bj9mdW5jdGlvbih0LG4pe3ZhciBlPVp1KHQpO3JldHVybiBmdW5j
dGlvbihyLGkpe3JldHVybiBuKGUociksaSx0KX19KHQsbik6WnUodCl9KSk7cmV0dXJuIG8uY29s
dW1ucz1pfHxbXSxvfSxwYXJzZVJvd3M6cixmb3JtYXQ6ZnVuY3Rpb24obixlKXtyZXR1cm4gbnVs
bD09ZSYmKGU9S3UobikpLFtlLm1hcChhKS5qb2luKHQpXS5jb25jYXQoaShuLGUpKS5qb2luKCJc
biIpfSxmb3JtYXRCb2R5OmZ1bmN0aW9uKHQsbil7cmV0dXJuIG51bGw9PW4mJihuPUt1KHQpKSxp
KHQsbikuam9pbigiXG4iKX0sZm9ybWF0Um93czpmdW5jdGlvbih0KXtyZXR1cm4gdC5tYXAobyku
am9pbigiXG4iKX0sZm9ybWF0Um93Om8sZm9ybWF0VmFsdWU6YX19dmFyIG5jPXRjKCIsIiksZWM9
bmMucGFyc2UscmM9bmMucGFyc2VSb3dzLGljPW5jLmZvcm1hdCxvYz1uYy5mb3JtYXRCb2R5LGFj
PW5jLmZvcm1hdFJvd3MsdWM9bmMuZm9ybWF0Um93LGNjPW5jLmZvcm1hdFZhbHVlLGZjPXRjKCJc
dCIpLHNjPWZjLnBhcnNlLGxjPWZjLnBhcnNlUm93cyxoYz1mYy5mb3JtYXQsZGM9ZmMuZm9ybWF0
Qm9keSxwYz1mYy5mb3JtYXRSb3dzLGdjPWZjLmZvcm1hdFJvdyx5Yz1mYy5mb3JtYXRWYWx1ZTtj
b25zdCB2Yz1uZXcgRGF0ZSgiMjAxOS0wMS0wMVQwMDowMCIpLmdldEhvdXJzKCl8fG5ldyBEYXRl
KCIyMDE5LTA3LTAxVDAwOjAwIikuZ2V0SG91cnMoKTtmdW5jdGlvbiBfYyh0KXtpZighdC5vayl0
aHJvdyBuZXcgRXJyb3IodC5zdGF0dXMrIiAiK3Quc3RhdHVzVGV4dCk7cmV0dXJuIHQuYmxvYigp
fWZ1bmN0aW9uIGJjKHQpe2lmKCF0Lm9rKXRocm93IG5ldyBFcnJvcih0LnN0YXR1cysiICIrdC5z
dGF0dXNUZXh0KTtyZXR1cm4gdC5hcnJheUJ1ZmZlcigpfWZ1bmN0aW9uIG1jKHQpe2lmKCF0Lm9r
KXRocm93IG5ldyBFcnJvcih0LnN0YXR1cysiICIrdC5zdGF0dXNUZXh0KTtyZXR1cm4gdC50ZXh0
KCl9ZnVuY3Rpb24geGModCxuKXtyZXR1cm4gZmV0Y2godCxuKS50aGVuKG1jKX1mdW5jdGlvbiB3
Yyh0KXtyZXR1cm4gZnVuY3Rpb24obixlLHIpe3JldHVybiAyPT09YXJndW1lbnRzLmxlbmd0aCYm
ImZ1bmN0aW9uIj09dHlwZW9mIGUmJihyPWUsZT12b2lkIDApLHhjKG4sZSkudGhlbigoZnVuY3Rp
b24obil7cmV0dXJuIHQobixyKX0pKX19dmFyIE1jPXdjKGVjKSxUYz13YyhzYyk7ZnVuY3Rpb24g
QWModCl7aWYoIXQub2spdGhyb3cgbmV3IEVycm9yKHQuc3RhdHVzKyIgIit0LnN0YXR1c1RleHQp
O2lmKDIwNCE9PXQuc3RhdHVzJiYyMDUhPT10LnN0YXR1cylyZXR1cm4gdC5qc29uKCl9ZnVuY3Rp
b24gU2ModCl7cmV0dXJuKG4sZSk9PnhjKG4sZSkudGhlbigobj0+KG5ldyBET01QYXJzZXIpLnBh
cnNlRnJvbVN0cmluZyhuLHQpKSl9dmFyIEVjPVNjKCJhcHBsaWNhdGlvbi94bWwiKSxOYz1TYygi
dGV4dC9odG1sIiksa2M9U2MoImltYWdlL3N2Zyt4bWwiKTtmdW5jdGlvbiBDYyh0LG4sZSxyKXtp
Zihpc05hTihuKXx8aXNOYU4oZSkpcmV0dXJuIHQ7dmFyIGksbyxhLHUsYyxmLHMsbCxoLGQ9dC5f
cm9vdCxwPXtkYXRhOnJ9LGc9dC5feDAseT10Ll95MCx2PXQuX3gxLF89dC5feTE7aWYoIWQpcmV0
dXJuIHQuX3Jvb3Q9cCx0O2Zvcig7ZC5sZW5ndGg7KWlmKChmPW4+PShvPShnK3YpLzIpKT9nPW86
dj1vLChzPWU+PShhPSh5K18pLzIpKT95PWE6Xz1hLGk9ZCwhKGQ9ZFtsPXM8PDF8Zl0pKXJldHVy
biBpW2xdPXAsdDtpZih1PSt0Ll94LmNhbGwobnVsbCxkLmRhdGEpLGM9K3QuX3kuY2FsbChudWxs
LGQuZGF0YSksbj09PXUmJmU9PT1jKXJldHVybiBwLm5leHQ9ZCxpP2lbbF09cDp0Ll9yb290PXAs
dDtkb3tpPWk/aVtsXT1uZXcgQXJyYXkoNCk6dC5fcm9vdD1uZXcgQXJyYXkoNCksKGY9bj49KG89
KGcrdikvMikpP2c9bzp2PW8sKHM9ZT49KGE9KHkrXykvMikpP3k9YTpfPWF9d2hpbGUoKGw9czw8
MXxmKT09KGg9KGM+PWEpPDwxfHU+PW8pKTtyZXR1cm4gaVtoXT1kLGlbbF09cCx0fWZ1bmN0aW9u
IFBjKHQsbixlLHIsaSl7dGhpcy5ub2RlPXQsdGhpcy54MD1uLHRoaXMueTA9ZSx0aGlzLngxPXIs
dGhpcy55MT1pfWZ1bmN0aW9uIHpjKHQpe3JldHVybiB0WzBdfWZ1bmN0aW9uICRjKHQpe3JldHVy
biB0WzFdfWZ1bmN0aW9uIERjKHQsbixlKXt2YXIgcj1uZXcgUmMobnVsbD09bj96YzpuLG51bGw9
PWU/JGM6ZSxOYU4sTmFOLE5hTixOYU4pO3JldHVybiBudWxsPT10P3I6ci5hZGRBbGwodCl9ZnVu
Y3Rpb24gUmModCxuLGUscixpLG8pe3RoaXMuX3g9dCx0aGlzLl95PW4sdGhpcy5feDA9ZSx0aGlz
Ll95MD1yLHRoaXMuX3gxPWksdGhpcy5feTE9byx0aGlzLl9yb290PXZvaWQgMH1mdW5jdGlvbiBG
Yyh0KXtmb3IodmFyIG49e2RhdGE6dC5kYXRhfSxlPW47dD10Lm5leHQ7KWU9ZS5uZXh0PXtkYXRh
OnQuZGF0YX07cmV0dXJuIG59dmFyIHFjPURjLnByb3RvdHlwZT1SYy5wcm90b3R5cGU7ZnVuY3Rp
b24gVWModCl7cmV0dXJuIGZ1bmN0aW9uKCl7cmV0dXJuIHR9fWZ1bmN0aW9uIEljKHQpe3JldHVy
biAxZS02Kih0KCktLjUpfWZ1bmN0aW9uIE9jKHQpe3JldHVybiB0LngrdC52eH1mdW5jdGlvbiBC
Yyh0KXtyZXR1cm4gdC55K3Qudnl9ZnVuY3Rpb24gWWModCl7cmV0dXJuIHQuaW5kZXh9ZnVuY3Rp
b24gTGModCxuKXt2YXIgZT10LmdldChuKTtpZighZSl0aHJvdyBuZXcgRXJyb3IoIm5vZGUgbm90
IGZvdW5kOiAiK24pO3JldHVybiBlfXFjLmNvcHk9ZnVuY3Rpb24oKXt2YXIgdCxuLGU9bmV3IFJj
KHRoaXMuX3gsdGhpcy5feSx0aGlzLl94MCx0aGlzLl95MCx0aGlzLl94MSx0aGlzLl95MSkscj10
aGlzLl9yb290O2lmKCFyKXJldHVybiBlO2lmKCFyLmxlbmd0aClyZXR1cm4gZS5fcm9vdD1GYyhy
KSxlO2Zvcih0PVt7c291cmNlOnIsdGFyZ2V0OmUuX3Jvb3Q9bmV3IEFycmF5KDQpfV07cj10LnBv
cCgpOylmb3IodmFyIGk9MDtpPDQ7KytpKShuPXIuc291cmNlW2ldKSYmKG4ubGVuZ3RoP3QucHVz
aCh7c291cmNlOm4sdGFyZ2V0OnIudGFyZ2V0W2ldPW5ldyBBcnJheSg0KX0pOnIudGFyZ2V0W2ld
PUZjKG4pKTtyZXR1cm4gZX0scWMuYWRkPWZ1bmN0aW9uKHQpe2NvbnN0IG49K3RoaXMuX3guY2Fs
bChudWxsLHQpLGU9K3RoaXMuX3kuY2FsbChudWxsLHQpO3JldHVybiBDYyh0aGlzLmNvdmVyKG4s
ZSksbixlLHQpfSxxYy5hZGRBbGw9ZnVuY3Rpb24odCl7dmFyIG4sZSxyLGksbz10Lmxlbmd0aCxh
PW5ldyBBcnJheShvKSx1PW5ldyBBcnJheShvKSxjPTEvMCxmPTEvMCxzPS0xLzAsbD0tMS8wO2Zv
cihlPTA7ZTxvOysrZSlpc05hTihyPSt0aGlzLl94LmNhbGwobnVsbCxuPXRbZV0pKXx8aXNOYU4o
aT0rdGhpcy5feS5jYWxsKG51bGwsbikpfHwoYVtlXT1yLHVbZV09aSxyPGMmJihjPXIpLHI+cyYm
KHM9ciksaTxmJiYoZj1pKSxpPmwmJihsPWkpKTtpZihjPnN8fGY+bClyZXR1cm4gdGhpcztmb3Io
dGhpcy5jb3ZlcihjLGYpLmNvdmVyKHMsbCksZT0wO2U8bzsrK2UpQ2ModGhpcyxhW2VdLHVbZV0s
dFtlXSk7cmV0dXJuIHRoaXN9LHFjLmNvdmVyPWZ1bmN0aW9uKHQsbil7aWYoaXNOYU4odD0rdCl8
fGlzTmFOKG49K24pKXJldHVybiB0aGlzO3ZhciBlPXRoaXMuX3gwLHI9dGhpcy5feTAsaT10aGlz
Ll94MSxvPXRoaXMuX3kxO2lmKGlzTmFOKGUpKWk9KGU9TWF0aC5mbG9vcih0KSkrMSxvPShyPU1h
dGguZmxvb3IobikpKzE7ZWxzZXtmb3IodmFyIGEsdSxjPWktZXx8MSxmPXRoaXMuX3Jvb3Q7ZT50
fHx0Pj1pfHxyPm58fG4+PW87KXN3aXRjaCh1PShuPHIpPDwxfHQ8ZSwoYT1uZXcgQXJyYXkoNCkp
W3VdPWYsZj1hLGMqPTIsdSl7Y2FzZSAwOmk9ZStjLG89citjO2JyZWFrO2Nhc2UgMTplPWktYyxv
PXIrYzticmVhaztjYXNlIDI6aT1lK2Mscj1vLWM7YnJlYWs7Y2FzZSAzOmU9aS1jLHI9by1jfXRo
aXMuX3Jvb3QmJnRoaXMuX3Jvb3QubGVuZ3RoJiYodGhpcy5fcm9vdD1mKX1yZXR1cm4gdGhpcy5f
eDA9ZSx0aGlzLl95MD1yLHRoaXMuX3gxPWksdGhpcy5feTE9byx0aGlzfSxxYy5kYXRhPWZ1bmN0
aW9uKCl7dmFyIHQ9W107cmV0dXJuIHRoaXMudmlzaXQoKGZ1bmN0aW9uKG4pe2lmKCFuLmxlbmd0
aClkb3t0LnB1c2gobi5kYXRhKX13aGlsZShuPW4ubmV4dCl9KSksdH0scWMuZXh0ZW50PWZ1bmN0
aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoP3RoaXMuY292ZXIoK3RbMF1bMF0sK3RbMF1b
MV0pLmNvdmVyKCt0WzFdWzBdLCt0WzFdWzFdKTppc05hTih0aGlzLl94MCk/dm9pZCAwOltbdGhp
cy5feDAsdGhpcy5feTBdLFt0aGlzLl94MSx0aGlzLl95MV1dfSxxYy5maW5kPWZ1bmN0aW9uKHQs
bixlKXt2YXIgcixpLG8sYSx1LGMsZixzPXRoaXMuX3gwLGw9dGhpcy5feTAsaD10aGlzLl94MSxk
PXRoaXMuX3kxLHA9W10sZz10aGlzLl9yb290O2ZvcihnJiZwLnB1c2gobmV3IFBjKGcscyxsLGgs
ZCkpLG51bGw9PWU/ZT0xLzA6KHM9dC1lLGw9bi1lLGg9dCtlLGQ9bitlLGUqPWUpO2M9cC5wb3Ao
KTspaWYoISghKGc9Yy5ub2RlKXx8KGk9Yy54MCk+aHx8KG89Yy55MCk+ZHx8KGE9Yy54MSk8c3x8
KHU9Yy55MSk8bCkpaWYoZy5sZW5ndGgpe3ZhciB5PShpK2EpLzIsdj0obyt1KS8yO3AucHVzaChu
ZXcgUGMoZ1szXSx5LHYsYSx1KSxuZXcgUGMoZ1syXSxpLHYseSx1KSxuZXcgUGMoZ1sxXSx5LG8s
YSx2KSxuZXcgUGMoZ1swXSxpLG8seSx2KSksKGY9KG4+PXYpPDwxfHQ+PXkpJiYoYz1wW3AubGVu
Z3RoLTFdLHBbcC5sZW5ndGgtMV09cFtwLmxlbmd0aC0xLWZdLHBbcC5sZW5ndGgtMS1mXT1jKX1l
bHNle3ZhciBfPXQtK3RoaXMuX3guY2FsbChudWxsLGcuZGF0YSksYj1uLSt0aGlzLl95LmNhbGwo
bnVsbCxnLmRhdGEpLG09XypfK2IqYjtpZihtPGUpe3ZhciB4PU1hdGguc3FydChlPW0pO3M9dC14
LGw9bi14LGg9dCt4LGQ9bit4LHI9Zy5kYXRhfX1yZXR1cm4gcn0scWMucmVtb3ZlPWZ1bmN0aW9u
KHQpe2lmKGlzTmFOKG89K3RoaXMuX3guY2FsbChudWxsLHQpKXx8aXNOYU4oYT0rdGhpcy5feS5j
YWxsKG51bGwsdCkpKXJldHVybiB0aGlzO3ZhciBuLGUscixpLG8sYSx1LGMsZixzLGwsaCxkPXRo
aXMuX3Jvb3QscD10aGlzLl94MCxnPXRoaXMuX3kwLHk9dGhpcy5feDEsdj10aGlzLl95MTtpZigh
ZClyZXR1cm4gdGhpcztpZihkLmxlbmd0aClmb3IoOzspe2lmKChmPW8+PSh1PShwK3kpLzIpKT9w
PXU6eT11LChzPWE+PShjPShnK3YpLzIpKT9nPWM6dj1jLG49ZCwhKGQ9ZFtsPXM8PDF8Zl0pKXJl
dHVybiB0aGlzO2lmKCFkLmxlbmd0aClicmVhazsobltsKzEmM118fG5bbCsyJjNdfHxuW2wrMyYz
XSkmJihlPW4saD1sKX1mb3IoO2QuZGF0YSE9PXQ7KWlmKHI9ZCwhKGQ9ZC5uZXh0KSlyZXR1cm4g
dGhpcztyZXR1cm4oaT1kLm5leHQpJiZkZWxldGUgZC5uZXh0LHI/KGk/ci5uZXh0PWk6ZGVsZXRl
IHIubmV4dCx0aGlzKTpuPyhpP25bbF09aTpkZWxldGUgbltsXSwoZD1uWzBdfHxuWzFdfHxuWzJd
fHxuWzNdKSYmZD09PShuWzNdfHxuWzJdfHxuWzFdfHxuWzBdKSYmIWQubGVuZ3RoJiYoZT9lW2hd
PWQ6dGhpcy5fcm9vdD1kKSx0aGlzKToodGhpcy5fcm9vdD1pLHRoaXMpfSxxYy5yZW1vdmVBbGw9
ZnVuY3Rpb24odCl7Zm9yKHZhciBuPTAsZT10Lmxlbmd0aDtuPGU7KytuKXRoaXMucmVtb3ZlKHRb
bl0pO3JldHVybiB0aGlzfSxxYy5yb290PWZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuX3Jvb3R9LHFj
LnNpemU9ZnVuY3Rpb24oKXt2YXIgdD0wO3JldHVybiB0aGlzLnZpc2l0KChmdW5jdGlvbihuKXtp
Zighbi5sZW5ndGgpZG97Kyt0fXdoaWxlKG49bi5uZXh0KX0pKSx0fSxxYy52aXNpdD1mdW5jdGlv
bih0KXt2YXIgbixlLHIsaSxvLGEsdT1bXSxjPXRoaXMuX3Jvb3Q7Zm9yKGMmJnUucHVzaChuZXcg
UGMoYyx0aGlzLl94MCx0aGlzLl95MCx0aGlzLl94MSx0aGlzLl95MSkpO249dS5wb3AoKTspaWYo
IXQoYz1uLm5vZGUscj1uLngwLGk9bi55MCxvPW4ueDEsYT1uLnkxKSYmYy5sZW5ndGgpe3ZhciBm
PShyK28pLzIscz0oaSthKS8yOyhlPWNbM10pJiZ1LnB1c2gobmV3IFBjKGUsZixzLG8sYSkpLChl
PWNbMl0pJiZ1LnB1c2gobmV3IFBjKGUscixzLGYsYSkpLChlPWNbMV0pJiZ1LnB1c2gobmV3IFBj
KGUsZixpLG8scykpLChlPWNbMF0pJiZ1LnB1c2gobmV3IFBjKGUscixpLGYscykpfXJldHVybiB0
aGlzfSxxYy52aXNpdEFmdGVyPWZ1bmN0aW9uKHQpe3ZhciBuLGU9W10scj1bXTtmb3IodGhpcy5f
cm9vdCYmZS5wdXNoKG5ldyBQYyh0aGlzLl9yb290LHRoaXMuX3gwLHRoaXMuX3kwLHRoaXMuX3gx
LHRoaXMuX3kxKSk7bj1lLnBvcCgpOyl7dmFyIGk9bi5ub2RlO2lmKGkubGVuZ3RoKXt2YXIgbyxh
PW4ueDAsdT1uLnkwLGM9bi54MSxmPW4ueTEscz0oYStjKS8yLGw9KHUrZikvMjsobz1pWzBdKSYm
ZS5wdXNoKG5ldyBQYyhvLGEsdSxzLGwpKSwobz1pWzFdKSYmZS5wdXNoKG5ldyBQYyhvLHMsdSxj
LGwpKSwobz1pWzJdKSYmZS5wdXNoKG5ldyBQYyhvLGEsbCxzLGYpKSwobz1pWzNdKSYmZS5wdXNo
KG5ldyBQYyhvLHMsbCxjLGYpKX1yLnB1c2gobil9Zm9yKDtuPXIucG9wKCk7KXQobi5ub2RlLG4u
eDAsbi55MCxuLngxLG4ueTEpO3JldHVybiB0aGlzfSxxYy54PWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyh0aGlzLl94PXQsdGhpcyk6dGhpcy5feH0scWMueT1mdW5jdGlvbih0
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odGhpcy5feT10LHRoaXMpOnRoaXMuX3l9O2NvbnN0
IGpjPTE2NjQ1MjUsSGM9MTAxMzkwNDIyMyxYYz00Mjk0OTY3Mjk2O2Z1bmN0aW9uIEdjKHQpe3Jl
dHVybiB0Lnh9ZnVuY3Rpb24gVmModCl7cmV0dXJuIHQueX12YXIgV2M9TWF0aC5QSSooMy1NYXRo
LnNxcnQoNSkpO2Z1bmN0aW9uIFpjKHQsbil7aWYoKGU9KHQ9bj90LnRvRXhwb25lbnRpYWwobi0x
KTp0LnRvRXhwb25lbnRpYWwoKSkuaW5kZXhPZigiZSIpKTwwKXJldHVybiBudWxsO3ZhciBlLHI9
dC5zbGljZSgwLGUpO3JldHVybltyLmxlbmd0aD4xP3JbMF0rci5zbGljZSgyKTpyLCt0LnNsaWNl
KGUrMSldfWZ1bmN0aW9uIEtjKHQpe3JldHVybih0PVpjKE1hdGguYWJzKHQpKSk/dFsxXTpOYU59
dmFyIFFjLEpjPS9eKD86KC4pPyhbPD49Xl0pKT8oWytcLSggXSk/KFskI10pPygwKT8oXGQrKT8o
LCk/KFwuXGQrKT8ofik/KFthLXolXSk/JC9pO2Z1bmN0aW9uIHRmKHQpe2lmKCEobj1KYy5leGVj
KHQpKSl0aHJvdyBuZXcgRXJyb3IoImludmFsaWQgZm9ybWF0OiAiK3QpO3ZhciBuO3JldHVybiBu
ZXcgbmYoe2ZpbGw6blsxXSxhbGlnbjpuWzJdLHNpZ246blszXSxzeW1ib2w6bls0XSx6ZXJvOm5b
NV0sd2lkdGg6bls2XSxjb21tYTpuWzddLHByZWNpc2lvbjpuWzhdJiZuWzhdLnNsaWNlKDEpLHRy
aW06bls5XSx0eXBlOm5bMTBdfSl9ZnVuY3Rpb24gbmYodCl7dGhpcy5maWxsPXZvaWQgMD09PXQu
ZmlsbD8iICI6dC5maWxsKyIiLHRoaXMuYWxpZ249dm9pZCAwPT09dC5hbGlnbj8iPiI6dC5hbGln
bisiIix0aGlzLnNpZ249dm9pZCAwPT09dC5zaWduPyItIjp0LnNpZ24rIiIsdGhpcy5zeW1ib2w9
dm9pZCAwPT09dC5zeW1ib2w/IiI6dC5zeW1ib2wrIiIsdGhpcy56ZXJvPSEhdC56ZXJvLHRoaXMu
d2lkdGg9dm9pZCAwPT09dC53aWR0aD92b2lkIDA6K3Qud2lkdGgsdGhpcy5jb21tYT0hIXQuY29t
bWEsdGhpcy5wcmVjaXNpb249dm9pZCAwPT09dC5wcmVjaXNpb24/dm9pZCAwOit0LnByZWNpc2lv
bix0aGlzLnRyaW09ISF0LnRyaW0sdGhpcy50eXBlPXZvaWQgMD09PXQudHlwZT8iIjp0LnR5cGUr
IiJ9ZnVuY3Rpb24gZWYodCxuKXt2YXIgZT1aYyh0LG4pO2lmKCFlKXJldHVybiB0KyIiO3ZhciBy
PWVbMF0saT1lWzFdO3JldHVybiBpPDA/IjAuIituZXcgQXJyYXkoLWkpLmpvaW4oIjAiKStyOnIu
bGVuZ3RoPmkrMT9yLnNsaWNlKDAsaSsxKSsiLiIrci5zbGljZShpKzEpOnIrbmV3IEFycmF5KGkt
ci5sZW5ndGgrMikuam9pbigiMCIpfXRmLnByb3RvdHlwZT1uZi5wcm90b3R5cGUsbmYucHJvdG90
eXBlLnRvU3RyaW5nPWZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuZmlsbCt0aGlzLmFsaWduK3RoaXMu
c2lnbit0aGlzLnN5bWJvbCsodGhpcy56ZXJvPyIwIjoiIikrKHZvaWQgMD09PXRoaXMud2lkdGg/
IiI6TWF0aC5tYXgoMSwwfHRoaXMud2lkdGgpKSsodGhpcy5jb21tYT8iLCI6IiIpKyh2b2lkIDA9
PT10aGlzLnByZWNpc2lvbj8iIjoiLiIrTWF0aC5tYXgoMCwwfHRoaXMucHJlY2lzaW9uKSkrKHRo
aXMudHJpbT8ifiI6IiIpK3RoaXMudHlwZX07dmFyIHJmPXsiJSI6KHQsbik9PigxMDAqdCkudG9G
aXhlZChuKSxiOnQ9Pk1hdGgucm91bmQodCkudG9TdHJpbmcoMiksYzp0PT50KyIiLGQ6ZnVuY3Rp
b24odCl7cmV0dXJuIE1hdGguYWJzKHQ9TWF0aC5yb3VuZCh0KSk+PTFlMjE/dC50b0xvY2FsZVN0
cmluZygiZW4iKS5yZXBsYWNlKC8sL2csIiIpOnQudG9TdHJpbmcoMTApfSxlOih0LG4pPT50LnRv
RXhwb25lbnRpYWwobiksZjoodCxuKT0+dC50b0ZpeGVkKG4pLGc6KHQsbik9PnQudG9QcmVjaXNp
b24obiksbzp0PT5NYXRoLnJvdW5kKHQpLnRvU3RyaW5nKDgpLHA6KHQsbik9PmVmKDEwMCp0LG4p
LHI6ZWYsczpmdW5jdGlvbih0LG4pe3ZhciBlPVpjKHQsbik7aWYoIWUpcmV0dXJuIHQrIiI7dmFy
IHI9ZVswXSxpPWVbMV0sbz1pLShRYz0zKk1hdGgubWF4KC04LE1hdGgubWluKDgsTWF0aC5mbG9v
cihpLzMpKSkpKzEsYT1yLmxlbmd0aDtyZXR1cm4gbz09PWE/cjpvPmE/cituZXcgQXJyYXkoby1h
KzEpLmpvaW4oIjAiKTpvPjA/ci5zbGljZSgwLG8pKyIuIityLnNsaWNlKG8pOiIwLiIrbmV3IEFy
cmF5KDEtbykuam9pbigiMCIpK1pjKHQsTWF0aC5tYXgoMCxuK28tMSkpWzBdfSxYOnQ9Pk1hdGgu
cm91bmQodCkudG9TdHJpbmcoMTYpLnRvVXBwZXJDYXNlKCkseDp0PT5NYXRoLnJvdW5kKHQpLnRv
U3RyaW5nKDE2KX07ZnVuY3Rpb24gb2YodCl7cmV0dXJuIHR9dmFyIGFmLHVmPUFycmF5LnByb3Rv
dHlwZS5tYXAsY2Y9WyJ5IiwieiIsImEiLCJmIiwicCIsIm4iLCLCtSIsIm0iLCIiLCJrIiwiTSIs
IkciLCJUIiwiUCIsIkUiLCJaIiwiWSJdO2Z1bmN0aW9uIGZmKHQpe3ZhciBuLGUscj12b2lkIDA9
PT10Lmdyb3VwaW5nfHx2b2lkIDA9PT10LnRob3VzYW5kcz9vZjoobj11Zi5jYWxsKHQuZ3JvdXBp
bmcsTnVtYmVyKSxlPXQudGhvdXNhbmRzKyIiLGZ1bmN0aW9uKHQscil7Zm9yKHZhciBpPXQubGVu
Z3RoLG89W10sYT0wLHU9blswXSxjPTA7aT4wJiZ1PjAmJihjK3UrMT5yJiYodT1NYXRoLm1heCgx
LHItYykpLG8ucHVzaCh0LnN1YnN0cmluZyhpLT11LGkrdSkpLCEoKGMrPXUrMSk+cikpOyl1PW5b
YT0oYSsxKSVuLmxlbmd0aF07cmV0dXJuIG8ucmV2ZXJzZSgpLmpvaW4oZSl9KSxpPXZvaWQgMD09
PXQuY3VycmVuY3k/IiI6dC5jdXJyZW5jeVswXSsiIixvPXZvaWQgMD09PXQuY3VycmVuY3k/IiI6
dC5jdXJyZW5jeVsxXSsiIixhPXZvaWQgMD09PXQuZGVjaW1hbD8iLiI6dC5kZWNpbWFsKyIiLHU9
dm9pZCAwPT09dC5udW1lcmFscz9vZjpmdW5jdGlvbih0KXtyZXR1cm4gZnVuY3Rpb24obil7cmV0
dXJuIG4ucmVwbGFjZSgvWzAtOV0vZywoZnVuY3Rpb24obil7cmV0dXJuIHRbK25dfSkpfX0odWYu
Y2FsbCh0Lm51bWVyYWxzLFN0cmluZykpLGM9dm9pZCAwPT09dC5wZXJjZW50PyIlIjp0LnBlcmNl
bnQrIiIsZj12b2lkIDA9PT10Lm1pbnVzPyLiiJIiOnQubWludXMrIiIscz12b2lkIDA9PT10Lm5h
bj8iTmFOIjp0Lm5hbisiIjtmdW5jdGlvbiBsKHQpe3ZhciBuPSh0PXRmKHQpKS5maWxsLGU9dC5h
bGlnbixsPXQuc2lnbixoPXQuc3ltYm9sLGQ9dC56ZXJvLHA9dC53aWR0aCxnPXQuY29tbWEseT10
LnByZWNpc2lvbix2PXQudHJpbSxfPXQudHlwZTsibiI9PT1fPyhnPSEwLF89ImciKTpyZltfXXx8
KHZvaWQgMD09PXkmJih5PTEyKSx2PSEwLF89ImciKSwoZHx8IjAiPT09biYmIj0iPT09ZSkmJihk
PSEwLG49IjAiLGU9Ij0iKTt2YXIgYj0iJCI9PT1oP2k6IiMiPT09aCYmL1tib3hYXS8udGVzdChf
KT8iMCIrXy50b0xvd2VyQ2FzZSgpOiIiLG09IiQiPT09aD9vOi9bJXBdLy50ZXN0KF8pP2M6IiIs
eD1yZltfXSx3PS9bZGVmZ3BycyVdLy50ZXN0KF8pO2Z1bmN0aW9uIE0odCl7dmFyIGksbyxjLGg9
YixNPW07aWYoImMiPT09XylNPXgodCkrTSx0PSIiO2Vsc2V7dmFyIFQ9KHQ9K3QpPDB8fDEvdDww
O2lmKHQ9aXNOYU4odCk/czp4KE1hdGguYWJzKHQpLHkpLHYmJih0PWZ1bmN0aW9uKHQpe3Q6Zm9y
KHZhciBuLGU9dC5sZW5ndGgscj0xLGk9LTE7cjxlOysrcilzd2l0Y2godFtyXSl7Y2FzZSIuIjpp
PW49cjticmVhaztjYXNlIjAiOjA9PT1pJiYoaT1yKSxuPXI7YnJlYWs7ZGVmYXVsdDppZighK3Rb
cl0pYnJlYWsgdDtpPjAmJihpPTApfXJldHVybiBpPjA/dC5zbGljZSgwLGkpK3Quc2xpY2Uobisx
KTp0fSh0KSksVCYmMD09K3QmJiIrIiE9PWwmJihUPSExKSxoPShUPyIoIj09PWw/bDpmOiItIj09
PWx8fCIoIj09PWw/IiI6bCkraCxNPSgicyI9PT1fP2NmWzgrUWMvM106IiIpK00rKFQmJiIoIj09
PWw/IikiOiIiKSx3KWZvcihpPS0xLG89dC5sZW5ndGg7KytpPG87KWlmKDQ4PihjPXQuY2hhckNv
ZGVBdChpKSl8fGM+NTcpe009KDQ2PT09Yz9hK3Quc2xpY2UoaSsxKTp0LnNsaWNlKGkpKStNLHQ9
dC5zbGljZSgwLGkpO2JyZWFrfX1nJiYhZCYmKHQ9cih0LDEvMCkpO3ZhciBBPWgubGVuZ3RoK3Qu
bGVuZ3RoK00ubGVuZ3RoLFM9QTxwP25ldyBBcnJheShwLUErMSkuam9pbihuKToiIjtzd2l0Y2go
ZyYmZCYmKHQ9cihTK3QsUy5sZW5ndGg/cC1NLmxlbmd0aDoxLzApLFM9IiIpLGUpe2Nhc2UiPCI6
dD1oK3QrTStTO2JyZWFrO2Nhc2UiPSI6dD1oK1MrdCtNO2JyZWFrO2Nhc2UiXiI6dD1TLnNsaWNl
KDAsQT1TLmxlbmd0aD4+MSkraCt0K00rUy5zbGljZShBKTticmVhaztkZWZhdWx0OnQ9UytoK3Qr
TX1yZXR1cm4gdSh0KX1yZXR1cm4geT12b2lkIDA9PT15PzY6L1tncHJzXS8udGVzdChfKT9NYXRo
Lm1heCgxLE1hdGgubWluKDIxLHkpKTpNYXRoLm1heCgwLE1hdGgubWluKDIwLHkpKSxNLnRvU3Ry
aW5nPWZ1bmN0aW9uKCl7cmV0dXJuIHQrIiJ9LE19cmV0dXJue2Zvcm1hdDpsLGZvcm1hdFByZWZp
eDpmdW5jdGlvbih0LG4pe3ZhciBlPWwoKCh0PXRmKHQpKS50eXBlPSJmIix0KSkscj0zKk1hdGgu
bWF4KC04LE1hdGgubWluKDgsTWF0aC5mbG9vcihLYyhuKS8zKSkpLGk9TWF0aC5wb3coMTAsLXIp
LG89Y2ZbOCtyLzNdO3JldHVybiBmdW5jdGlvbih0KXtyZXR1cm4gZShpKnQpK299fX19ZnVuY3Rp
b24gc2Yobil7cmV0dXJuIGFmPWZmKG4pLHQuZm9ybWF0PWFmLmZvcm1hdCx0LmZvcm1hdFByZWZp
eD1hZi5mb3JtYXRQcmVmaXgsYWZ9ZnVuY3Rpb24gbGYodCl7cmV0dXJuIE1hdGgubWF4KDAsLUtj
KE1hdGguYWJzKHQpKSl9ZnVuY3Rpb24gaGYodCxuKXtyZXR1cm4gTWF0aC5tYXgoMCwzKk1hdGgu
bWF4KC04LE1hdGgubWluKDgsTWF0aC5mbG9vcihLYyhuKS8zKSkpLUtjKE1hdGguYWJzKHQpKSl9
ZnVuY3Rpb24gZGYodCxuKXtyZXR1cm4gdD1NYXRoLmFicyh0KSxuPU1hdGguYWJzKG4pLXQsTWF0
aC5tYXgoMCxLYyhuKS1LYyh0KSkrMX10LmZvcm1hdD12b2lkIDAsdC5mb3JtYXRQcmVmaXg9dm9p
ZCAwLHNmKHt0aG91c2FuZHM6IiwiLGdyb3VwaW5nOlszXSxjdXJyZW5jeTpbIiQiLCIiXX0pO3Zh
ciBwZj0xZS02LGdmPTFlLTEyLHlmPU1hdGguUEksdmY9eWYvMixfZj15Zi80LGJmPTIqeWYsbWY9
MTgwL3lmLHhmPXlmLzE4MCx3Zj1NYXRoLmFicyxNZj1NYXRoLmF0YW4sVGY9TWF0aC5hdGFuMixB
Zj1NYXRoLmNvcyxTZj1NYXRoLmNlaWwsRWY9TWF0aC5leHAsTmY9TWF0aC5oeXBvdCxrZj1NYXRo
LmxvZyxDZj1NYXRoLnBvdyxQZj1NYXRoLnNpbix6Zj1NYXRoLnNpZ258fGZ1bmN0aW9uKHQpe3Jl
dHVybiB0PjA/MTp0PDA/LTE6MH0sJGY9TWF0aC5zcXJ0LERmPU1hdGgudGFuO2Z1bmN0aW9uIFJm
KHQpe3JldHVybiB0PjE/MDp0PC0xP3lmOk1hdGguYWNvcyh0KX1mdW5jdGlvbiBGZih0KXtyZXR1
cm4gdD4xP3ZmOnQ8LTE/LXZmOk1hdGguYXNpbih0KX1mdW5jdGlvbiBxZih0KXtyZXR1cm4odD1Q
Zih0LzIpKSp0fWZ1bmN0aW9uIFVmKCl7fWZ1bmN0aW9uIElmKHQsbil7dCYmQmYuaGFzT3duUHJv
cGVydHkodC50eXBlKSYmQmZbdC50eXBlXSh0LG4pfXZhciBPZj17RmVhdHVyZTpmdW5jdGlvbih0
LG4pe0lmKHQuZ2VvbWV0cnksbil9LEZlYXR1cmVDb2xsZWN0aW9uOmZ1bmN0aW9uKHQsbil7Zm9y
KHZhciBlPXQuZmVhdHVyZXMscj0tMSxpPWUubGVuZ3RoOysrcjxpOylJZihlW3JdLmdlb21ldHJ5
LG4pfX0sQmY9e1NwaGVyZTpmdW5jdGlvbih0LG4pe24uc3BoZXJlKCl9LFBvaW50OmZ1bmN0aW9u
KHQsbil7dD10LmNvb3JkaW5hdGVzLG4ucG9pbnQodFswXSx0WzFdLHRbMl0pfSxNdWx0aVBvaW50
OmZ1bmN0aW9uKHQsbil7Zm9yKHZhciBlPXQuY29vcmRpbmF0ZXMscj0tMSxpPWUubGVuZ3RoOysr
cjxpOyl0PWVbcl0sbi5wb2ludCh0WzBdLHRbMV0sdFsyXSl9LExpbmVTdHJpbmc6ZnVuY3Rpb24o
dCxuKXtZZih0LmNvb3JkaW5hdGVzLG4sMCl9LE11bHRpTGluZVN0cmluZzpmdW5jdGlvbih0LG4p
e2Zvcih2YXIgZT10LmNvb3JkaW5hdGVzLHI9LTEsaT1lLmxlbmd0aDsrK3I8aTspWWYoZVtyXSxu
LDApfSxQb2x5Z29uOmZ1bmN0aW9uKHQsbil7TGYodC5jb29yZGluYXRlcyxuKX0sTXVsdGlQb2x5
Z29uOmZ1bmN0aW9uKHQsbil7Zm9yKHZhciBlPXQuY29vcmRpbmF0ZXMscj0tMSxpPWUubGVuZ3Ro
OysrcjxpOylMZihlW3JdLG4pfSxHZW9tZXRyeUNvbGxlY3Rpb246ZnVuY3Rpb24odCxuKXtmb3Io
dmFyIGU9dC5nZW9tZXRyaWVzLHI9LTEsaT1lLmxlbmd0aDsrK3I8aTspSWYoZVtyXSxuKX19O2Z1
bmN0aW9uIFlmKHQsbixlKXt2YXIgcixpPS0xLG89dC5sZW5ndGgtZTtmb3Iobi5saW5lU3RhcnQo
KTsrK2k8bzspcj10W2ldLG4ucG9pbnQoclswXSxyWzFdLHJbMl0pO24ubGluZUVuZCgpfWZ1bmN0
aW9uIExmKHQsbil7dmFyIGU9LTEscj10Lmxlbmd0aDtmb3Iobi5wb2x5Z29uU3RhcnQoKTsrK2U8
cjspWWYodFtlXSxuLDEpO24ucG9seWdvbkVuZCgpfWZ1bmN0aW9uIGpmKHQsbil7dCYmT2YuaGFz
T3duUHJvcGVydHkodC50eXBlKT9PZlt0LnR5cGVdKHQsbik6SWYodCxuKX12YXIgSGYsWGYsR2Ys
VmYsV2YsWmYsS2YsUWYsSmYsdHMsbnMsZXMscnMsaXMsb3MsYXMsdXM9bmV3IEEsY3M9bmV3IEEs
ZnM9e3BvaW50OlVmLGxpbmVTdGFydDpVZixsaW5lRW5kOlVmLHBvbHlnb25TdGFydDpmdW5jdGlv
bigpe3VzPW5ldyBBLGZzLmxpbmVTdGFydD1zcyxmcy5saW5lRW5kPWxzfSxwb2x5Z29uRW5kOmZ1
bmN0aW9uKCl7dmFyIHQ9K3VzO2NzLmFkZCh0PDA/YmYrdDp0KSx0aGlzLmxpbmVTdGFydD10aGlz
LmxpbmVFbmQ9dGhpcy5wb2ludD1VZn0sc3BoZXJlOmZ1bmN0aW9uKCl7Y3MuYWRkKGJmKX19O2Z1
bmN0aW9uIHNzKCl7ZnMucG9pbnQ9aHN9ZnVuY3Rpb24gbHMoKXtkcyhIZixYZil9ZnVuY3Rpb24g
aHModCxuKXtmcy5wb2ludD1kcyxIZj10LFhmPW4sR2Y9dCo9eGYsVmY9QWYobj0obio9eGYpLzIr
X2YpLFdmPVBmKG4pfWZ1bmN0aW9uIGRzKHQsbil7dmFyIGU9KHQqPXhmKS1HZixyPWU+PTA/MTot
MSxpPXIqZSxvPUFmKG49KG4qPXhmKS8yK19mKSxhPVBmKG4pLHU9V2YqYSxjPVZmKm8rdSpBZihp
KSxmPXUqcipQZihpKTt1cy5hZGQoVGYoZixjKSksR2Y9dCxWZj1vLFdmPWF9ZnVuY3Rpb24gcHMo
dCl7cmV0dXJuW1RmKHRbMV0sdFswXSksRmYodFsyXSldfWZ1bmN0aW9uIGdzKHQpe3ZhciBuPXRb
MF0sZT10WzFdLHI9QWYoZSk7cmV0dXJuW3IqQWYobikscipQZihuKSxQZihlKV19ZnVuY3Rpb24g
eXModCxuKXtyZXR1cm4gdFswXSpuWzBdK3RbMV0qblsxXSt0WzJdKm5bMl19ZnVuY3Rpb24gdnMo
dCxuKXtyZXR1cm5bdFsxXSpuWzJdLXRbMl0qblsxXSx0WzJdKm5bMF0tdFswXSpuWzJdLHRbMF0q
blsxXS10WzFdKm5bMF1dfWZ1bmN0aW9uIF9zKHQsbil7dFswXSs9blswXSx0WzFdKz1uWzFdLHRb
Ml0rPW5bMl19ZnVuY3Rpb24gYnModCxuKXtyZXR1cm5bdFswXSpuLHRbMV0qbix0WzJdKm5dfWZ1
bmN0aW9uIG1zKHQpe3ZhciBuPSRmKHRbMF0qdFswXSt0WzFdKnRbMV0rdFsyXSp0WzJdKTt0WzBd
Lz1uLHRbMV0vPW4sdFsyXS89bn12YXIgeHMsd3MsTXMsVHMsQXMsU3MsRXMsTnMsa3MsQ3MsUHMs
enMsJHMsRHMsUnMsRnMscXM9e3BvaW50OlVzLGxpbmVTdGFydDpPcyxsaW5lRW5kOkJzLHBvbHln
b25TdGFydDpmdW5jdGlvbigpe3FzLnBvaW50PVlzLHFzLmxpbmVTdGFydD1Mcyxxcy5saW5lRW5k
PWpzLGlzPW5ldyBBLGZzLnBvbHlnb25TdGFydCgpfSxwb2x5Z29uRW5kOmZ1bmN0aW9uKCl7ZnMu
cG9seWdvbkVuZCgpLHFzLnBvaW50PVVzLHFzLmxpbmVTdGFydD1Pcyxxcy5saW5lRW5kPUJzLHVz
PDA/KFpmPS0oUWY9MTgwKSxLZj0tKEpmPTkwKSk6aXM+cGY/SmY9OTA6aXM8LXBmJiYoS2Y9LTkw
KSxhc1swXT1aZixhc1sxXT1RZn0sc3BoZXJlOmZ1bmN0aW9uKCl7WmY9LShRZj0xODApLEtmPS0o
SmY9OTApfX07ZnVuY3Rpb24gVXModCxuKXtvcy5wdXNoKGFzPVtaZj10LFFmPXRdKSxuPEtmJiYo
S2Y9biksbj5KZiYmKEpmPW4pfWZ1bmN0aW9uIElzKHQsbil7dmFyIGU9Z3MoW3QqeGYsbip4Zl0p
O2lmKHJzKXt2YXIgcj12cyhycyxlKSxpPXZzKFtyWzFdLC1yWzBdLDBdLHIpO21zKGkpLGk9cHMo
aSk7dmFyIG8sYT10LXRzLHU9YT4wPzE6LTEsYz1pWzBdKm1mKnUsZj13ZihhKT4xODA7Zl4odSp0
czxjJiZjPHUqdCk/KG89aVsxXSptZik+SmYmJihKZj1vKTpmXih1KnRzPChjPShjKzM2MCklMzYw
LTE4MCkmJmM8dSp0KT8obz0taVsxXSptZik8S2YmJihLZj1vKToobjxLZiYmKEtmPW4pLG4+SmYm
JihKZj1uKSksZj90PHRzP0hzKFpmLHQpPkhzKFpmLFFmKSYmKFFmPXQpOkhzKHQsUWYpPkhzKFpm
LFFmKSYmKFpmPXQpOlFmPj1aZj8odDxaZiYmKFpmPXQpLHQ+UWYmJihRZj10KSk6dD50cz9Icyha
Zix0KT5IcyhaZixRZikmJihRZj10KTpIcyh0LFFmKT5IcyhaZixRZikmJihaZj10KX1lbHNlIG9z
LnB1c2goYXM9W1pmPXQsUWY9dF0pO248S2YmJihLZj1uKSxuPkpmJiYoSmY9bikscnM9ZSx0cz10
fWZ1bmN0aW9uIE9zKCl7cXMucG9pbnQ9SXN9ZnVuY3Rpb24gQnMoKXthc1swXT1aZixhc1sxXT1R
Zixxcy5wb2ludD1Vcyxycz1udWxsfWZ1bmN0aW9uIFlzKHQsbil7aWYocnMpe3ZhciBlPXQtdHM7
aXMuYWRkKHdmKGUpPjE4MD9lKyhlPjA/MzYwOi0zNjApOmUpfWVsc2UgbnM9dCxlcz1uO2ZzLnBv
aW50KHQsbiksSXModCxuKX1mdW5jdGlvbiBMcygpe2ZzLmxpbmVTdGFydCgpfWZ1bmN0aW9uIGpz
KCl7WXMobnMsZXMpLGZzLmxpbmVFbmQoKSx3Zihpcyk+cGYmJihaZj0tKFFmPTE4MCkpLGFzWzBd
PVpmLGFzWzFdPVFmLHJzPW51bGx9ZnVuY3Rpb24gSHModCxuKXtyZXR1cm4obi09dCk8MD9uKzM2
MDpufWZ1bmN0aW9uIFhzKHQsbil7cmV0dXJuIHRbMF0tblswXX1mdW5jdGlvbiBHcyh0LG4pe3Jl
dHVybiB0WzBdPD10WzFdP3RbMF08PW4mJm48PXRbMV06bjx0WzBdfHx0WzFdPG59dmFyIFZzPXtz
cGhlcmU6VWYscG9pbnQ6V3MsbGluZVN0YXJ0OktzLGxpbmVFbmQ6dGwscG9seWdvblN0YXJ0OmZ1
bmN0aW9uKCl7VnMubGluZVN0YXJ0PW5sLFZzLmxpbmVFbmQ9ZWx9LHBvbHlnb25FbmQ6ZnVuY3Rp
b24oKXtWcy5saW5lU3RhcnQ9S3MsVnMubGluZUVuZD10bH19O2Z1bmN0aW9uIFdzKHQsbil7dCo9
eGY7dmFyIGU9QWYobio9eGYpO1pzKGUqQWYodCksZSpQZih0KSxQZihuKSl9ZnVuY3Rpb24gWnMo
dCxuLGUpeysreHMsTXMrPSh0LU1zKS94cyxUcys9KG4tVHMpL3hzLEFzKz0oZS1BcykveHN9ZnVu
Y3Rpb24gS3MoKXtWcy5wb2ludD1Rc31mdW5jdGlvbiBRcyh0LG4pe3QqPXhmO3ZhciBlPUFmKG4q
PXhmKTtEcz1lKkFmKHQpLFJzPWUqUGYodCksRnM9UGYobiksVnMucG9pbnQ9SnMsWnMoRHMsUnMs
RnMpfWZ1bmN0aW9uIEpzKHQsbil7dCo9eGY7dmFyIGU9QWYobio9eGYpLHI9ZSpBZih0KSxpPWUq
UGYodCksbz1QZihuKSxhPVRmKCRmKChhPVJzKm8tRnMqaSkqYSsoYT1GcypyLURzKm8pKmErKGE9
RHMqaS1ScypyKSphKSxEcypyK1JzKmkrRnMqbyk7d3MrPWEsU3MrPWEqKERzKyhEcz1yKSksRXMr
PWEqKFJzKyhScz1pKSksTnMrPWEqKEZzKyhGcz1vKSksWnMoRHMsUnMsRnMpfWZ1bmN0aW9uIHRs
KCl7VnMucG9pbnQ9V3N9ZnVuY3Rpb24gbmwoKXtWcy5wb2ludD1ybH1mdW5jdGlvbiBlbCgpe2ls
KHpzLCRzKSxWcy5wb2ludD1Xc31mdW5jdGlvbiBybCh0LG4pe3pzPXQsJHM9bix0Kj14ZixuKj14
ZixWcy5wb2ludD1pbDt2YXIgZT1BZihuKTtEcz1lKkFmKHQpLFJzPWUqUGYodCksRnM9UGYobiks
WnMoRHMsUnMsRnMpfWZ1bmN0aW9uIGlsKHQsbil7dCo9eGY7dmFyIGU9QWYobio9eGYpLHI9ZSpB
Zih0KSxpPWUqUGYodCksbz1QZihuKSxhPVJzKm8tRnMqaSx1PUZzKnItRHMqbyxjPURzKmktUnMq
cixmPU5mKGEsdSxjKSxzPUZmKGYpLGw9ZiYmLXMvZjtrcy5hZGQobCphKSxDcy5hZGQobCp1KSxQ
cy5hZGQobCpjKSx3cys9cyxTcys9cyooRHMrKERzPXIpKSxFcys9cyooUnMrKFJzPWkpKSxOcys9
cyooRnMrKEZzPW8pKSxacyhEcyxScyxGcyl9ZnVuY3Rpb24gb2wodCl7cmV0dXJuIGZ1bmN0aW9u
KCl7cmV0dXJuIHR9fWZ1bmN0aW9uIGFsKHQsbil7ZnVuY3Rpb24gZShlLHIpe3JldHVybiBlPXQo
ZSxyKSxuKGVbMF0sZVsxXSl9cmV0dXJuIHQuaW52ZXJ0JiZuLmludmVydCYmKGUuaW52ZXJ0PWZ1
bmN0aW9uKGUscil7cmV0dXJuKGU9bi5pbnZlcnQoZSxyKSkmJnQuaW52ZXJ0KGVbMF0sZVsxXSl9
KSxlfWZ1bmN0aW9uIHVsKHQsbil7cmV0dXJuIHdmKHQpPnlmJiYodC09TWF0aC5yb3VuZCh0L2Jm
KSpiZiksW3Qsbl19ZnVuY3Rpb24gY2wodCxuLGUpe3JldHVybih0JT1iZik/bnx8ZT9hbChzbCh0
KSxsbChuLGUpKTpzbCh0KTpufHxlP2xsKG4sZSk6dWx9ZnVuY3Rpb24gZmwodCl7cmV0dXJuIGZ1
bmN0aW9uKG4sZSl7cmV0dXJuIHdmKG4rPXQpPnlmJiYobi09TWF0aC5yb3VuZChuL2JmKSpiZiks
W24sZV19fWZ1bmN0aW9uIHNsKHQpe3ZhciBuPWZsKHQpO3JldHVybiBuLmludmVydD1mbCgtdCks
bn1mdW5jdGlvbiBsbCh0LG4pe3ZhciBlPUFmKHQpLHI9UGYodCksaT1BZihuKSxvPVBmKG4pO2Z1
bmN0aW9uIGEodCxuKXt2YXIgYT1BZihuKSx1PUFmKHQpKmEsYz1QZih0KSphLGY9UGYobikscz1m
KmUrdSpyO3JldHVybltUZihjKmktcypvLHUqZS1mKnIpLEZmKHMqaStjKm8pXX1yZXR1cm4gYS5p
bnZlcnQ9ZnVuY3Rpb24odCxuKXt2YXIgYT1BZihuKSx1PUFmKHQpKmEsYz1QZih0KSphLGY9UGYo
bikscz1mKmktYypvO3JldHVybltUZihjKmkrZipvLHUqZStzKnIpLEZmKHMqZS11KnIpXX0sYX1m
dW5jdGlvbiBobCh0KXtmdW5jdGlvbiBuKG4pe3JldHVybihuPXQoblswXSp4ZixuWzFdKnhmKSlb
MF0qPW1mLG5bMV0qPW1mLG59cmV0dXJuIHQ9Y2wodFswXSp4Zix0WzFdKnhmLHQubGVuZ3RoPjI/
dFsyXSp4ZjowKSxuLmludmVydD1mdW5jdGlvbihuKXtyZXR1cm4obj10LmludmVydChuWzBdKnhm
LG5bMV0qeGYpKVswXSo9bWYsblsxXSo9bWYsbn0sbn1mdW5jdGlvbiBkbCh0LG4sZSxyLGksbyl7
aWYoZSl7dmFyIGE9QWYobiksdT1QZihuKSxjPXIqZTtudWxsPT1pPyhpPW4rcipiZixvPW4tYy8y
KTooaT1wbChhLGkpLG89cGwoYSxvKSwocj4wP2k8bzppPm8pJiYoaSs9cipiZikpO2Zvcih2YXIg
ZixzPWk7cj4wP3M+bzpzPG87cy09YylmPXBzKFthLC11KkFmKHMpLC11KlBmKHMpXSksdC5wb2lu
dChmWzBdLGZbMV0pfX1mdW5jdGlvbiBwbCh0LG4peyhuPWdzKG4pKVswXS09dCxtcyhuKTt2YXIg
ZT1SZigtblsxXSk7cmV0dXJuKCgtblsyXTwwPy1lOmUpK2JmLXBmKSViZn1mdW5jdGlvbiBnbCgp
e3ZhciB0LG49W107cmV0dXJue3BvaW50OmZ1bmN0aW9uKG4sZSxyKXt0LnB1c2goW24sZSxyXSl9
LGxpbmVTdGFydDpmdW5jdGlvbigpe24ucHVzaCh0PVtdKX0sbGluZUVuZDpVZixyZWpvaW46ZnVu
Y3Rpb24oKXtuLmxlbmd0aD4xJiZuLnB1c2gobi5wb3AoKS5jb25jYXQobi5zaGlmdCgpKSl9LHJl
c3VsdDpmdW5jdGlvbigpe3ZhciBlPW47cmV0dXJuIG49W10sdD1udWxsLGV9fX1mdW5jdGlvbiB5
bCh0LG4pe3JldHVybiB3Zih0WzBdLW5bMF0pPHBmJiZ3Zih0WzFdLW5bMV0pPHBmfWZ1bmN0aW9u
IHZsKHQsbixlLHIpe3RoaXMueD10LHRoaXMuej1uLHRoaXMubz1lLHRoaXMuZT1yLHRoaXMudj0h
MSx0aGlzLm49dGhpcy5wPW51bGx9ZnVuY3Rpb24gX2wodCxuLGUscixpKXt2YXIgbyxhLHU9W10s
Yz1bXTtpZih0LmZvckVhY2goKGZ1bmN0aW9uKHQpe2lmKCEoKG49dC5sZW5ndGgtMSk8PTApKXt2
YXIgbixlLHI9dFswXSxhPXRbbl07aWYoeWwocixhKSl7aWYoIXJbMl0mJiFhWzJdKXtmb3IoaS5s
aW5lU3RhcnQoKSxvPTA7bzxuOysrbylpLnBvaW50KChyPXRbb10pWzBdLHJbMV0pO3JldHVybiB2
b2lkIGkubGluZUVuZCgpfWFbMF0rPTIqcGZ9dS5wdXNoKGU9bmV3IHZsKHIsdCxudWxsLCEwKSks
Yy5wdXNoKGUubz1uZXcgdmwocixudWxsLGUsITEpKSx1LnB1c2goZT1uZXcgdmwoYSx0LG51bGws
ITEpKSxjLnB1c2goZS5vPW5ldyB2bChhLG51bGwsZSwhMCkpfX0pKSx1Lmxlbmd0aCl7Zm9yKGMu
c29ydChuKSxibCh1KSxibChjKSxvPTAsYT1jLmxlbmd0aDtvPGE7KytvKWNbb10uZT1lPSFlO2Zv
cih2YXIgZixzLGw9dVswXTs7KXtmb3IodmFyIGg9bCxkPSEwO2gudjspaWYoKGg9aC5uKT09PWwp
cmV0dXJuO2Y9aC56LGkubGluZVN0YXJ0KCk7ZG97aWYoaC52PWguby52PSEwLGguZSl7aWYoZClm
b3Iobz0wLGE9Zi5sZW5ndGg7bzxhOysrbylpLnBvaW50KChzPWZbb10pWzBdLHNbMV0pO2Vsc2Ug
cihoLngsaC5uLngsMSxpKTtoPWgubn1lbHNle2lmKGQpZm9yKGY9aC5wLnosbz1mLmxlbmd0aC0x
O28+PTA7LS1vKWkucG9pbnQoKHM9ZltvXSlbMF0sc1sxXSk7ZWxzZSByKGgueCxoLnAueCwtMSxp
KTtoPWgucH1mPShoPWgubykueixkPSFkfXdoaWxlKCFoLnYpO2kubGluZUVuZCgpfX19ZnVuY3Rp
b24gYmwodCl7aWYobj10Lmxlbmd0aCl7Zm9yKHZhciBuLGUscj0wLGk9dFswXTsrK3I8bjspaS5u
PWU9dFtyXSxlLnA9aSxpPWU7aS5uPWU9dFswXSxlLnA9aX19ZnVuY3Rpb24gbWwodCl7cmV0dXJu
IHdmKHRbMF0pPD15Zj90WzBdOnpmKHRbMF0pKigod2YodFswXSkreWYpJWJmLXlmKX1mdW5jdGlv
biB4bCh0LG4pe3ZhciBlPW1sKG4pLHI9blsxXSxpPVBmKHIpLG89W1BmKGUpLC1BZihlKSwwXSxh
PTAsdT0wLGM9bmV3IEE7MT09PWk/cj12ZitwZjotMT09PWkmJihyPS12Zi1wZik7Zm9yKHZhciBm
PTAscz10Lmxlbmd0aDtmPHM7KytmKWlmKGg9KGw9dFtmXSkubGVuZ3RoKWZvcih2YXIgbCxoLGQ9
bFtoLTFdLHA9bWwoZCksZz1kWzFdLzIrX2YseT1QZihnKSx2PUFmKGcpLF89MDtfPGg7KytfLHA9
bSx5PXcsdj1NLGQ9Yil7dmFyIGI9bFtfXSxtPW1sKGIpLHg9YlsxXS8yK19mLHc9UGYoeCksTT1B
Zih4KSxUPW0tcCxTPVQ+PTA/MTotMSxFPVMqVCxOPUU+eWYsaz15Knc7aWYoYy5hZGQoVGYoaypT
KlBmKEUpLHYqTStrKkFmKEUpKSksYSs9Tj9UK1MqYmY6VCxOXnA+PWVebT49ZSl7dmFyIEM9dnMo
Z3MoZCksZ3MoYikpO21zKEMpO3ZhciBQPXZzKG8sQyk7bXMoUCk7dmFyIHo9KE5eVD49MD8tMTox
KSpGZihQWzJdKTsocj56fHxyPT09eiYmKENbMF18fENbMV0pKSYmKHUrPU5eVD49MD8xOi0xKX19
cmV0dXJuKGE8LXBmfHxhPHBmJiZjPC1nZileMSZ1fWZ1bmN0aW9uIHdsKHQsbixlLHIpe3JldHVy
biBmdW5jdGlvbihpKXt2YXIgbyxhLHUsYz1uKGkpLGY9Z2woKSxzPW4oZiksbD0hMSxoPXtwb2lu
dDpkLGxpbmVTdGFydDpnLGxpbmVFbmQ6eSxwb2x5Z29uU3RhcnQ6ZnVuY3Rpb24oKXtoLnBvaW50
PXYsaC5saW5lU3RhcnQ9XyxoLmxpbmVFbmQ9YixhPVtdLG89W119LHBvbHlnb25FbmQ6ZnVuY3Rp
b24oKXtoLnBvaW50PWQsaC5saW5lU3RhcnQ9ZyxoLmxpbmVFbmQ9eSxhPXN0KGEpO3ZhciB0PXhs
KG8scik7YS5sZW5ndGg/KGx8fChpLnBvbHlnb25TdGFydCgpLGw9ITApLF9sKGEsVGwsdCxlLGkp
KTp0JiYobHx8KGkucG9seWdvblN0YXJ0KCksbD0hMCksaS5saW5lU3RhcnQoKSxlKG51bGwsbnVs
bCwxLGkpLGkubGluZUVuZCgpKSxsJiYoaS5wb2x5Z29uRW5kKCksbD0hMSksYT1vPW51bGx9LHNw
aGVyZTpmdW5jdGlvbigpe2kucG9seWdvblN0YXJ0KCksaS5saW5lU3RhcnQoKSxlKG51bGwsbnVs
bCwxLGkpLGkubGluZUVuZCgpLGkucG9seWdvbkVuZCgpfX07ZnVuY3Rpb24gZChuLGUpe3Qobixl
KSYmaS5wb2ludChuLGUpfWZ1bmN0aW9uIHAodCxuKXtjLnBvaW50KHQsbil9ZnVuY3Rpb24gZygp
e2gucG9pbnQ9cCxjLmxpbmVTdGFydCgpfWZ1bmN0aW9uIHkoKXtoLnBvaW50PWQsYy5saW5lRW5k
KCl9ZnVuY3Rpb24gdih0LG4pe3UucHVzaChbdCxuXSkscy5wb2ludCh0LG4pfWZ1bmN0aW9uIF8o
KXtzLmxpbmVTdGFydCgpLHU9W119ZnVuY3Rpb24gYigpe3YodVswXVswXSx1WzBdWzFdKSxzLmxp
bmVFbmQoKTt2YXIgdCxuLGUscixjPXMuY2xlYW4oKSxoPWYucmVzdWx0KCksZD1oLmxlbmd0aDtp
Zih1LnBvcCgpLG8ucHVzaCh1KSx1PW51bGwsZClpZigxJmMpe2lmKChuPShlPWhbMF0pLmxlbmd0
aC0xKT4wKXtmb3IobHx8KGkucG9seWdvblN0YXJ0KCksbD0hMCksaS5saW5lU3RhcnQoKSx0PTA7
dDxuOysrdClpLnBvaW50KChyPWVbdF0pWzBdLHJbMV0pO2kubGluZUVuZCgpfX1lbHNlIGQ+MSYm
MiZjJiZoLnB1c2goaC5wb3AoKS5jb25jYXQoaC5zaGlmdCgpKSksYS5wdXNoKGguZmlsdGVyKE1s
KSl9cmV0dXJuIGh9fWZ1bmN0aW9uIE1sKHQpe3JldHVybiB0Lmxlbmd0aD4xfWZ1bmN0aW9uIFRs
KHQsbil7cmV0dXJuKCh0PXQueClbMF08MD90WzFdLXZmLXBmOnZmLXRbMV0pLSgobj1uLngpWzBd
PDA/blsxXS12Zi1wZjp2Zi1uWzFdKX11bC5pbnZlcnQ9dWw7dmFyIEFsPXdsKChmdW5jdGlvbigp
e3JldHVybiEwfSksKGZ1bmN0aW9uKHQpe3ZhciBuLGU9TmFOLHI9TmFOLGk9TmFOO3JldHVybnts
aW5lU3RhcnQ6ZnVuY3Rpb24oKXt0LmxpbmVTdGFydCgpLG49MX0scG9pbnQ6ZnVuY3Rpb24obyxh
KXt2YXIgdT1vPjA/eWY6LXlmLGM9d2Yoby1lKTt3ZihjLXlmKTxwZj8odC5wb2ludChlLHI9KHIr
YSkvMj4wP3ZmOi12ZiksdC5wb2ludChpLHIpLHQubGluZUVuZCgpLHQubGluZVN0YXJ0KCksdC5w
b2ludCh1LHIpLHQucG9pbnQobyxyKSxuPTApOmkhPT11JiZjPj15ZiYmKHdmKGUtaSk8cGYmJihl
LT1pKnBmKSx3ZihvLXUpPHBmJiYoby09dSpwZikscj1mdW5jdGlvbih0LG4sZSxyKXt2YXIgaSxv
LGE9UGYodC1lKTtyZXR1cm4gd2YoYSk+cGY/TWYoKFBmKG4pKihvPUFmKHIpKSpQZihlKS1QZihy
KSooaT1BZihuKSkqUGYodCkpLyhpKm8qYSkpOihuK3IpLzJ9KGUscixvLGEpLHQucG9pbnQoaSxy
KSx0LmxpbmVFbmQoKSx0LmxpbmVTdGFydCgpLHQucG9pbnQodSxyKSxuPTApLHQucG9pbnQoZT1v
LHI9YSksaT11fSxsaW5lRW5kOmZ1bmN0aW9uKCl7dC5saW5lRW5kKCksZT1yPU5hTn0sY2xlYW46
ZnVuY3Rpb24oKXtyZXR1cm4gMi1ufX19KSwoZnVuY3Rpb24odCxuLGUscil7dmFyIGk7aWYobnVs
bD09dClpPWUqdmYsci5wb2ludCgteWYsaSksci5wb2ludCgwLGkpLHIucG9pbnQoeWYsaSksci5w
b2ludCh5ZiwwKSxyLnBvaW50KHlmLC1pKSxyLnBvaW50KDAsLWkpLHIucG9pbnQoLXlmLC1pKSxy
LnBvaW50KC15ZiwwKSxyLnBvaW50KC15ZixpKTtlbHNlIGlmKHdmKHRbMF0tblswXSk+cGYpe3Zh
ciBvPXRbMF08blswXT95ZjoteWY7aT1lKm8vMixyLnBvaW50KC1vLGkpLHIucG9pbnQoMCxpKSxy
LnBvaW50KG8saSl9ZWxzZSByLnBvaW50KG5bMF0sblsxXSl9KSxbLXlmLC12Zl0pO2Z1bmN0aW9u
IFNsKHQpe3ZhciBuPUFmKHQpLGU9Nip4ZixyPW4+MCxpPXdmKG4pPnBmO2Z1bmN0aW9uIG8odCxl
KXtyZXR1cm4gQWYodCkqQWYoZSk+bn1mdW5jdGlvbiBhKHQsZSxyKXt2YXIgaT1bMSwwLDBdLG89
dnMoZ3ModCksZ3MoZSkpLGE9eXMobyxvKSx1PW9bMF0sYz1hLXUqdTtpZighYylyZXR1cm4hciYm
dDt2YXIgZj1uKmEvYyxzPS1uKnUvYyxsPXZzKGksbyksaD1icyhpLGYpO19zKGgsYnMobyxzKSk7
dmFyIGQ9bCxwPXlzKGgsZCksZz15cyhkLGQpLHk9cCpwLWcqKHlzKGgsaCktMSk7aWYoISh5PDAp
KXt2YXIgdj0kZih5KSxfPWJzKGQsKC1wLXYpL2cpO2lmKF9zKF8saCksXz1wcyhfKSwhcilyZXR1
cm4gXzt2YXIgYixtPXRbMF0seD1lWzBdLHc9dFsxXSxNPWVbMV07eDxtJiYoYj1tLG09eCx4PWIp
O3ZhciBUPXgtbSxBPXdmKFQteWYpPHBmO2lmKCFBJiZNPHcmJihiPXcsdz1NLE09YiksQXx8VDxw
Zj9BP3crTT4wXl9bMV08KHdmKF9bMF0tbSk8cGY/dzpNKTp3PD1fWzFdJiZfWzFdPD1NOlQ+eWZe
KG08PV9bMF0mJl9bMF08PXgpKXt2YXIgUz1icyhkLCgtcCt2KS9nKTtyZXR1cm4gX3MoUyxoKSxb
XyxwcyhTKV19fX1mdW5jdGlvbiB1KG4sZSl7dmFyIGk9cj90OnlmLXQsbz0wO3JldHVybiBuPC1p
P298PTE6bj5pJiYob3w9MiksZTwtaT9vfD00OmU+aSYmKG98PTgpLG99cmV0dXJuIHdsKG8sKGZ1
bmN0aW9uKHQpe3ZhciBuLGUsYyxmLHM7cmV0dXJue2xpbmVTdGFydDpmdW5jdGlvbigpe2Y9Yz0h
MSxzPTF9LHBvaW50OmZ1bmN0aW9uKGwsaCl7dmFyIGQscD1bbCxoXSxnPW8obCxoKSx5PXI/Zz8w
OnUobCxoKTpnP3UobCsobDwwP3lmOi15ZiksaCk6MDtpZighbiYmKGY9Yz1nKSYmdC5saW5lU3Rh
cnQoKSxnIT09YyYmKCEoZD1hKG4scCkpfHx5bChuLGQpfHx5bChwLGQpKSYmKHBbMl09MSksZyE9
PWMpcz0wLGc/KHQubGluZVN0YXJ0KCksZD1hKHAsbiksdC5wb2ludChkWzBdLGRbMV0pKTooZD1h
KG4scCksdC5wb2ludChkWzBdLGRbMV0sMiksdC5saW5lRW5kKCkpLG49ZDtlbHNlIGlmKGkmJm4m
JnJeZyl7dmFyIHY7eSZlfHwhKHY9YShwLG4sITApKXx8KHM9MCxyPyh0LmxpbmVTdGFydCgpLHQu
cG9pbnQodlswXVswXSx2WzBdWzFdKSx0LnBvaW50KHZbMV1bMF0sdlsxXVsxXSksdC5saW5lRW5k
KCkpOih0LnBvaW50KHZbMV1bMF0sdlsxXVsxXSksdC5saW5lRW5kKCksdC5saW5lU3RhcnQoKSx0
LnBvaW50KHZbMF1bMF0sdlswXVsxXSwzKSkpfSFnfHxuJiZ5bChuLHApfHx0LnBvaW50KHBbMF0s
cFsxXSksbj1wLGM9ZyxlPXl9LGxpbmVFbmQ6ZnVuY3Rpb24oKXtjJiZ0LmxpbmVFbmQoKSxuPW51
bGx9LGNsZWFuOmZ1bmN0aW9uKCl7cmV0dXJuIHN8KGYmJmMpPDwxfX19KSwoZnVuY3Rpb24obixy
LGksbyl7ZGwobyx0LGUsaSxuLHIpfSkscj9bMCwtdF06Wy15Zix0LXlmXSl9dmFyIEVsLE5sLGts
LENsLFBsPTFlOSx6bD0tUGw7ZnVuY3Rpb24gJGwodCxuLGUscil7ZnVuY3Rpb24gaShpLG8pe3Jl
dHVybiB0PD1pJiZpPD1lJiZuPD1vJiZvPD1yfWZ1bmN0aW9uIG8oaSxvLHUsZil7dmFyIHM9MCxs
PTA7aWYobnVsbD09aXx8KHM9YShpLHUpKSE9PShsPWEobyx1KSl8fGMoaSxvKTwwXnU+MClkb3tm
LnBvaW50KDA9PT1zfHwzPT09cz90OmUscz4xP3I6bil9d2hpbGUoKHM9KHMrdSs0KSU0KSE9PWwp
O2Vsc2UgZi5wb2ludChvWzBdLG9bMV0pfWZ1bmN0aW9uIGEocixpKXtyZXR1cm4gd2YoclswXS10
KTxwZj9pPjA/MDozOndmKHJbMF0tZSk8cGY/aT4wPzI6MTp3ZihyWzFdLW4pPHBmP2k+MD8xOjA6
aT4wPzM6Mn1mdW5jdGlvbiB1KHQsbil7cmV0dXJuIGModC54LG4ueCl9ZnVuY3Rpb24gYyh0LG4p
e3ZhciBlPWEodCwxKSxyPWEobiwxKTtyZXR1cm4gZSE9PXI/ZS1yOjA9PT1lP25bMV0tdFsxXTox
PT09ZT90WzBdLW5bMF06Mj09PWU/dFsxXS1uWzFdOm5bMF0tdFswXX1yZXR1cm4gZnVuY3Rpb24o
YSl7dmFyIGMsZixzLGwsaCxkLHAsZyx5LHYsXyxiPWEsbT1nbCgpLHg9e3BvaW50OncsbGluZVN0
YXJ0OmZ1bmN0aW9uKCl7eC5wb2ludD1NLGYmJmYucHVzaChzPVtdKTt2PSEwLHk9ITEscD1nPU5h
Tn0sbGluZUVuZDpmdW5jdGlvbigpe2MmJihNKGwsaCksZCYmeSYmbS5yZWpvaW4oKSxjLnB1c2go
bS5yZXN1bHQoKSkpO3gucG9pbnQ9dyx5JiZiLmxpbmVFbmQoKX0scG9seWdvblN0YXJ0OmZ1bmN0
aW9uKCl7Yj1tLGM9W10sZj1bXSxfPSEwfSxwb2x5Z29uRW5kOmZ1bmN0aW9uKCl7dmFyIG49ZnVu
Y3Rpb24oKXtmb3IodmFyIG49MCxlPTAsaT1mLmxlbmd0aDtlPGk7KytlKWZvcih2YXIgbyxhLHU9
ZltlXSxjPTEscz11Lmxlbmd0aCxsPXVbMF0saD1sWzBdLGQ9bFsxXTtjPHM7KytjKW89aCxhPWQs
aD0obD11W2NdKVswXSxkPWxbMV0sYTw9cj9kPnImJihoLW8pKihyLWEpPihkLWEpKih0LW8pJiYr
K246ZDw9ciYmKGgtbykqKHItYSk8KGQtYSkqKHQtbykmJi0tbjtyZXR1cm4gbn0oKSxlPV8mJm4s
aT0oYz1zdChjKSkubGVuZ3RoOyhlfHxpKSYmKGEucG9seWdvblN0YXJ0KCksZSYmKGEubGluZVN0
YXJ0KCksbyhudWxsLG51bGwsMSxhKSxhLmxpbmVFbmQoKSksaSYmX2woYyx1LG4sbyxhKSxhLnBv
bHlnb25FbmQoKSk7Yj1hLGM9Zj1zPW51bGx9fTtmdW5jdGlvbiB3KHQsbil7aSh0LG4pJiZiLnBv
aW50KHQsbil9ZnVuY3Rpb24gTShvLGEpe3ZhciB1PWkobyxhKTtpZihmJiZzLnB1c2goW28sYV0p
LHYpbD1vLGg9YSxkPXUsdj0hMSx1JiYoYi5saW5lU3RhcnQoKSxiLnBvaW50KG8sYSkpO2Vsc2Ug
aWYodSYmeSliLnBvaW50KG8sYSk7ZWxzZXt2YXIgYz1bcD1NYXRoLm1heCh6bCxNYXRoLm1pbihQ
bCxwKSksZz1NYXRoLm1heCh6bCxNYXRoLm1pbihQbCxnKSldLG09W289TWF0aC5tYXgoemwsTWF0
aC5taW4oUGwsbykpLGE9TWF0aC5tYXgoemwsTWF0aC5taW4oUGwsYSkpXTshZnVuY3Rpb24odCxu
LGUscixpLG8pe3ZhciBhLHU9dFswXSxjPXRbMV0sZj0wLHM9MSxsPW5bMF0tdSxoPW5bMV0tYztp
ZihhPWUtdSxsfHwhKGE+MCkpe2lmKGEvPWwsbDwwKXtpZihhPGYpcmV0dXJuO2E8cyYmKHM9YSl9
ZWxzZSBpZihsPjApe2lmKGE+cylyZXR1cm47YT5mJiYoZj1hKX1pZihhPWktdSxsfHwhKGE8MCkp
e2lmKGEvPWwsbDwwKXtpZihhPnMpcmV0dXJuO2E+ZiYmKGY9YSl9ZWxzZSBpZihsPjApe2lmKGE8
ZilyZXR1cm47YTxzJiYocz1hKX1pZihhPXItYyxofHwhKGE+MCkpe2lmKGEvPWgsaDwwKXtpZihh
PGYpcmV0dXJuO2E8cyYmKHM9YSl9ZWxzZSBpZihoPjApe2lmKGE+cylyZXR1cm47YT5mJiYoZj1h
KX1pZihhPW8tYyxofHwhKGE8MCkpe2lmKGEvPWgsaDwwKXtpZihhPnMpcmV0dXJuO2E+ZiYmKGY9
YSl9ZWxzZSBpZihoPjApe2lmKGE8ZilyZXR1cm47YTxzJiYocz1hKX1yZXR1cm4gZj4wJiYodFsw
XT11K2YqbCx0WzFdPWMrZipoKSxzPDEmJihuWzBdPXUrcypsLG5bMV09YytzKmgpLCEwfX19fX0o
YyxtLHQsbixlLHIpP3UmJihiLmxpbmVTdGFydCgpLGIucG9pbnQobyxhKSxfPSExKTooeXx8KGIu
bGluZVN0YXJ0KCksYi5wb2ludChjWzBdLGNbMV0pKSxiLnBvaW50KG1bMF0sbVsxXSksdXx8Yi5s
aW5lRW5kKCksXz0hMSl9cD1vLGc9YSx5PXV9cmV0dXJuIHh9fXZhciBEbD17c3BoZXJlOlVmLHBv
aW50OlVmLGxpbmVTdGFydDpmdW5jdGlvbigpe0RsLnBvaW50PUZsLERsLmxpbmVFbmQ9Umx9LGxp
bmVFbmQ6VWYscG9seWdvblN0YXJ0OlVmLHBvbHlnb25FbmQ6VWZ9O2Z1bmN0aW9uIFJsKCl7RGwu
cG9pbnQ9RGwubGluZUVuZD1VZn1mdW5jdGlvbiBGbCh0LG4pe05sPXQqPXhmLGtsPVBmKG4qPXhm
KSxDbD1BZihuKSxEbC5wb2ludD1xbH1mdW5jdGlvbiBxbCh0LG4pe3QqPXhmO3ZhciBlPVBmKG4q
PXhmKSxyPUFmKG4pLGk9d2YodC1ObCksbz1BZihpKSxhPXIqUGYoaSksdT1DbCplLWtsKnIqbyxj
PWtsKmUrQ2wqcipvO0VsLmFkZChUZigkZihhKmErdSp1KSxjKSksTmw9dCxrbD1lLENsPXJ9ZnVu
Y3Rpb24gVWwodCl7cmV0dXJuIEVsPW5ldyBBLGpmKHQsRGwpLCtFbH12YXIgSWw9W251bGwsbnVs
bF0sT2w9e3R5cGU6IkxpbmVTdHJpbmciLGNvb3JkaW5hdGVzOklsfTtmdW5jdGlvbiBCbCh0LG4p
e3JldHVybiBJbFswXT10LElsWzFdPW4sVWwoT2wpfXZhciBZbD17RmVhdHVyZTpmdW5jdGlvbih0
LG4pe3JldHVybiBqbCh0Lmdlb21ldHJ5LG4pfSxGZWF0dXJlQ29sbGVjdGlvbjpmdW5jdGlvbih0
LG4pe2Zvcih2YXIgZT10LmZlYXR1cmVzLHI9LTEsaT1lLmxlbmd0aDsrK3I8aTspaWYoamwoZVty
XS5nZW9tZXRyeSxuKSlyZXR1cm4hMDtyZXR1cm4hMX19LExsPXtTcGhlcmU6ZnVuY3Rpb24oKXty
ZXR1cm4hMH0sUG9pbnQ6ZnVuY3Rpb24odCxuKXtyZXR1cm4gSGwodC5jb29yZGluYXRlcyxuKX0s
TXVsdGlQb2ludDpmdW5jdGlvbih0LG4pe2Zvcih2YXIgZT10LmNvb3JkaW5hdGVzLHI9LTEsaT1l
Lmxlbmd0aDsrK3I8aTspaWYoSGwoZVtyXSxuKSlyZXR1cm4hMDtyZXR1cm4hMX0sTGluZVN0cmlu
ZzpmdW5jdGlvbih0LG4pe3JldHVybiBYbCh0LmNvb3JkaW5hdGVzLG4pfSxNdWx0aUxpbmVTdHJp
bmc6ZnVuY3Rpb24odCxuKXtmb3IodmFyIGU9dC5jb29yZGluYXRlcyxyPS0xLGk9ZS5sZW5ndGg7
KytyPGk7KWlmKFhsKGVbcl0sbikpcmV0dXJuITA7cmV0dXJuITF9LFBvbHlnb246ZnVuY3Rpb24o
dCxuKXtyZXR1cm4gR2wodC5jb29yZGluYXRlcyxuKX0sTXVsdGlQb2x5Z29uOmZ1bmN0aW9uKHQs
bil7Zm9yKHZhciBlPXQuY29vcmRpbmF0ZXMscj0tMSxpPWUubGVuZ3RoOysrcjxpOylpZihHbChl
W3JdLG4pKXJldHVybiEwO3JldHVybiExfSxHZW9tZXRyeUNvbGxlY3Rpb246ZnVuY3Rpb24odCxu
KXtmb3IodmFyIGU9dC5nZW9tZXRyaWVzLHI9LTEsaT1lLmxlbmd0aDsrK3I8aTspaWYoamwoZVty
XSxuKSlyZXR1cm4hMDtyZXR1cm4hMX19O2Z1bmN0aW9uIGpsKHQsbil7cmV0dXJuISghdHx8IUxs
Lmhhc093blByb3BlcnR5KHQudHlwZSkpJiZMbFt0LnR5cGVdKHQsbil9ZnVuY3Rpb24gSGwodCxu
KXtyZXR1cm4gMD09PUJsKHQsbil9ZnVuY3Rpb24gWGwodCxuKXtmb3IodmFyIGUscixpLG89MCxh
PXQubGVuZ3RoO288YTtvKyspe2lmKDA9PT0ocj1CbCh0W29dLG4pKSlyZXR1cm4hMDtpZihvPjAm
JihpPUJsKHRbb10sdFtvLTFdKSk+MCYmZTw9aSYmcjw9aSYmKGUrci1pKSooMS1NYXRoLnBvdygo
ZS1yKS9pLDIpKTxnZippKXJldHVybiEwO2U9cn1yZXR1cm4hMX1mdW5jdGlvbiBHbCh0LG4pe3Jl
dHVybiEheGwodC5tYXAoVmwpLFdsKG4pKX1mdW5jdGlvbiBWbCh0KXtyZXR1cm4odD10Lm1hcChX
bCkpLnBvcCgpLHR9ZnVuY3Rpb24gV2wodCl7cmV0dXJuW3RbMF0qeGYsdFsxXSp4Zl19ZnVuY3Rp
b24gWmwodCxuLGUpe3ZhciByPWh0KHQsbi1wZixlKS5jb25jYXQobik7cmV0dXJuIGZ1bmN0aW9u
KHQpe3JldHVybiByLm1hcCgoZnVuY3Rpb24obil7cmV0dXJuW3Qsbl19KSl9fWZ1bmN0aW9uIEts
KHQsbixlKXt2YXIgcj1odCh0LG4tcGYsZSkuY29uY2F0KG4pO3JldHVybiBmdW5jdGlvbih0KXty
ZXR1cm4gci5tYXAoKGZ1bmN0aW9uKG4pe3JldHVybltuLHRdfSkpfX1mdW5jdGlvbiBRbCgpe3Zh
ciB0LG4sZSxyLGksbyxhLHUsYyxmLHMsbCxoPTEwLGQ9aCxwPTkwLGc9MzYwLHk9Mi41O2Z1bmN0
aW9uIHYoKXtyZXR1cm57dHlwZToiTXVsdGlMaW5lU3RyaW5nIixjb29yZGluYXRlczpfKCl9fWZ1
bmN0aW9uIF8oKXtyZXR1cm4gaHQoU2Yoci9wKSpwLGUscCkubWFwKHMpLmNvbmNhdChodChTZih1
L2cpKmcsYSxnKS5tYXAobCkpLmNvbmNhdChodChTZihuL2gpKmgsdCxoKS5maWx0ZXIoKGZ1bmN0
aW9uKHQpe3JldHVybiB3Zih0JXApPnBmfSkpLm1hcChjKSkuY29uY2F0KGh0KFNmKG8vZCkqZCxp
LGQpLmZpbHRlcigoZnVuY3Rpb24odCl7cmV0dXJuIHdmKHQlZyk+cGZ9KSkubWFwKGYpKX1yZXR1
cm4gdi5saW5lcz1mdW5jdGlvbigpe3JldHVybiBfKCkubWFwKChmdW5jdGlvbih0KXtyZXR1cm57
dHlwZToiTGluZVN0cmluZyIsY29vcmRpbmF0ZXM6dH19KSl9LHYub3V0bGluZT1mdW5jdGlvbigp
e3JldHVybnt0eXBlOiJQb2x5Z29uIixjb29yZGluYXRlczpbcyhyKS5jb25jYXQobChhKS5zbGlj
ZSgxKSxzKGUpLnJldmVyc2UoKS5zbGljZSgxKSxsKHUpLnJldmVyc2UoKS5zbGljZSgxKSldfX0s
di5leHRlbnQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/di5leHRlbnRNYWpv
cih0KS5leHRlbnRNaW5vcih0KTp2LmV4dGVudE1pbm9yKCl9LHYuZXh0ZW50TWFqb3I9ZnVuY3Rp
b24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9K3RbMF1bMF0sZT0rdFsxXVswXSx1PSt0
WzBdWzFdLGE9K3RbMV1bMV0scj5lJiYodD1yLHI9ZSxlPXQpLHU+YSYmKHQ9dSx1PWEsYT10KSx2
LnByZWNpc2lvbih5KSk6W1tyLHVdLFtlLGFdXX0sdi5leHRlbnRNaW5vcj1mdW5jdGlvbihlKXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obj0rZVswXVswXSx0PStlWzFdWzBdLG89K2VbMF1bMV0s
aT0rZVsxXVsxXSxuPnQmJihlPW4sbj10LHQ9ZSksbz5pJiYoZT1vLG89aSxpPWUpLHYucHJlY2lz
aW9uKHkpKTpbW24sb10sW3QsaV1dfSx2LnN0ZXA9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50
cy5sZW5ndGg/di5zdGVwTWFqb3IodCkuc3RlcE1pbm9yKHQpOnYuc3RlcE1pbm9yKCl9LHYuc3Rl
cE1ham9yPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhwPSt0WzBdLGc9K3Rb
MV0sdik6W3AsZ119LHYuc3RlcE1pbm9yPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPyhoPSt0WzBdLGQ9K3RbMV0sdik6W2gsZF19LHYucHJlY2lzaW9uPWZ1bmN0aW9uKGgpe3Jl
dHVybiBhcmd1bWVudHMubGVuZ3RoPyh5PStoLGM9WmwobyxpLDkwKSxmPUtsKG4sdCx5KSxzPVps
KHUsYSw5MCksbD1LbChyLGUseSksdik6eX0sdi5leHRlbnRNYWpvcihbWy0xODAsLTkwK3BmXSxb
MTgwLDkwLXBmXV0pLmV4dGVudE1pbm9yKFtbLTE4MCwtODAtcGZdLFsxODAsODArcGZdXSl9dmFy
IEpsLHRoLG5oLGVoLHJoPXQ9PnQsaWg9bmV3IEEsb2g9bmV3IEEsYWg9e3BvaW50OlVmLGxpbmVT
dGFydDpVZixsaW5lRW5kOlVmLHBvbHlnb25TdGFydDpmdW5jdGlvbigpe2FoLmxpbmVTdGFydD11
aCxhaC5saW5lRW5kPXNofSxwb2x5Z29uRW5kOmZ1bmN0aW9uKCl7YWgubGluZVN0YXJ0PWFoLmxp
bmVFbmQ9YWgucG9pbnQ9VWYsaWguYWRkKHdmKG9oKSksb2g9bmV3IEF9LHJlc3VsdDpmdW5jdGlv
bigpe3ZhciB0PWloLzI7cmV0dXJuIGloPW5ldyBBLHR9fTtmdW5jdGlvbiB1aCgpe2FoLnBvaW50
PWNofWZ1bmN0aW9uIGNoKHQsbil7YWgucG9pbnQ9ZmgsSmw9bmg9dCx0aD1laD1ufWZ1bmN0aW9u
IGZoKHQsbil7b2guYWRkKGVoKnQtbmgqbiksbmg9dCxlaD1ufWZ1bmN0aW9uIHNoKCl7ZmgoSmws
dGgpfXZhciBsaD1haCxoaD0xLzAsZGg9aGgscGg9LWhoLGdoPXBoLHloPXtwb2ludDpmdW5jdGlv
bih0LG4pe3Q8aGgmJihoaD10KTt0PnBoJiYocGg9dCk7bjxkaCYmKGRoPW4pO24+Z2gmJihnaD1u
KX0sbGluZVN0YXJ0OlVmLGxpbmVFbmQ6VWYscG9seWdvblN0YXJ0OlVmLHBvbHlnb25FbmQ6VWYs
cmVzdWx0OmZ1bmN0aW9uKCl7dmFyIHQ9W1toaCxkaF0sW3BoLGdoXV07cmV0dXJuIHBoPWdoPS0o
ZGg9aGg9MS8wKSx0fX07dmFyIHZoLF9oLGJoLG1oLHhoPXloLHdoPTAsTWg9MCxUaD0wLEFoPTAs
U2g9MCxFaD0wLE5oPTAsa2g9MCxDaD0wLFBoPXtwb2ludDp6aCxsaW5lU3RhcnQ6JGgsbGluZUVu
ZDpGaCxwb2x5Z29uU3RhcnQ6ZnVuY3Rpb24oKXtQaC5saW5lU3RhcnQ9cWgsUGgubGluZUVuZD1V
aH0scG9seWdvbkVuZDpmdW5jdGlvbigpe1BoLnBvaW50PXpoLFBoLmxpbmVTdGFydD0kaCxQaC5s
aW5lRW5kPUZofSxyZXN1bHQ6ZnVuY3Rpb24oKXt2YXIgdD1DaD9bTmgvQ2gsa2gvQ2hdOkVoP1tB
aC9FaCxTaC9FaF06VGg/W3doL1RoLE1oL1RoXTpbTmFOLE5hTl07cmV0dXJuIHdoPU1oPVRoPUFo
PVNoPUVoPU5oPWtoPUNoPTAsdH19O2Z1bmN0aW9uIHpoKHQsbil7d2grPXQsTWgrPW4sKytUaH1m
dW5jdGlvbiAkaCgpe1BoLnBvaW50PURofWZ1bmN0aW9uIERoKHQsbil7UGgucG9pbnQ9Umgsemgo
Ymg9dCxtaD1uKX1mdW5jdGlvbiBSaCh0LG4pe3ZhciBlPXQtYmgscj1uLW1oLGk9JGYoZSplK3Iq
cik7QWgrPWkqKGJoK3QpLzIsU2grPWkqKG1oK24pLzIsRWgrPWksemgoYmg9dCxtaD1uKX1mdW5j
dGlvbiBGaCgpe1BoLnBvaW50PXpofWZ1bmN0aW9uIHFoKCl7UGgucG9pbnQ9SWh9ZnVuY3Rpb24g
VWgoKXtPaCh2aCxfaCl9ZnVuY3Rpb24gSWgodCxuKXtQaC5wb2ludD1PaCx6aCh2aD1iaD10LF9o
PW1oPW4pfWZ1bmN0aW9uIE9oKHQsbil7dmFyIGU9dC1iaCxyPW4tbWgsaT0kZihlKmUrcipyKTtB
aCs9aSooYmgrdCkvMixTaCs9aSoobWgrbikvMixFaCs9aSxOaCs9KGk9bWgqdC1iaCpuKSooYmgr
dCksa2grPWkqKG1oK24pLENoKz0zKmksemgoYmg9dCxtaD1uKX12YXIgQmg9UGg7ZnVuY3Rpb24g
WWgodCl7dGhpcy5fY29udGV4dD10fVloLnByb3RvdHlwZT17X3JhZGl1czo0LjUscG9pbnRSYWRp
dXM6ZnVuY3Rpb24odCl7cmV0dXJuIHRoaXMuX3JhZGl1cz10LHRoaXN9LHBvbHlnb25TdGFydDpm
dW5jdGlvbigpe3RoaXMuX2xpbmU9MH0scG9seWdvbkVuZDpmdW5jdGlvbigpe3RoaXMuX2xpbmU9
TmFOfSxsaW5lU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLl9wb2ludD0wfSxsaW5lRW5kOmZ1bmN0aW9u
KCl7MD09PXRoaXMuX2xpbmUmJnRoaXMuX2NvbnRleHQuY2xvc2VQYXRoKCksdGhpcy5fcG9pbnQ9
TmFOfSxwb2ludDpmdW5jdGlvbih0LG4pe3N3aXRjaCh0aGlzLl9wb2ludCl7Y2FzZSAwOnRoaXMu
X2NvbnRleHQubW92ZVRvKHQsbiksdGhpcy5fcG9pbnQ9MTticmVhaztjYXNlIDE6dGhpcy5fY29u
dGV4dC5saW5lVG8odCxuKTticmVhaztkZWZhdWx0OnRoaXMuX2NvbnRleHQubW92ZVRvKHQrdGhp
cy5fcmFkaXVzLG4pLHRoaXMuX2NvbnRleHQuYXJjKHQsbix0aGlzLl9yYWRpdXMsMCxiZil9fSxy
ZXN1bHQ6VWZ9O3ZhciBMaCxqaCxIaCxYaCxHaCxWaD1uZXcgQSxXaD17cG9pbnQ6VWYsbGluZVN0
YXJ0OmZ1bmN0aW9uKCl7V2gucG9pbnQ9Wmh9LGxpbmVFbmQ6ZnVuY3Rpb24oKXtMaCYmS2goamgs
SGgpLFdoLnBvaW50PVVmfSxwb2x5Z29uU3RhcnQ6ZnVuY3Rpb24oKXtMaD0hMH0scG9seWdvbkVu
ZDpmdW5jdGlvbigpe0xoPW51bGx9LHJlc3VsdDpmdW5jdGlvbigpe3ZhciB0PStWaDtyZXR1cm4g
Vmg9bmV3IEEsdH19O2Z1bmN0aW9uIFpoKHQsbil7V2gucG9pbnQ9S2gsamg9WGg9dCxIaD1HaD1u
fWZ1bmN0aW9uIEtoKHQsbil7WGgtPXQsR2gtPW4sVmguYWRkKCRmKFhoKlhoK0doKkdoKSksWGg9
dCxHaD1ufXZhciBRaD1XaDtsZXQgSmgsdGQsbmQsZWQ7Y2xhc3MgcmR7Y29uc3RydWN0b3IodCl7
dGhpcy5fYXBwZW5kPW51bGw9PXQ/aWQ6ZnVuY3Rpb24odCl7Y29uc3Qgbj1NYXRoLmZsb29yKHQp
O2lmKCEobj49MCkpdGhyb3cgbmV3IFJhbmdlRXJyb3IoYGludmFsaWQgZGlnaXRzOiAke3R9YCk7
aWYobj4xNSlyZXR1cm4gaWQ7aWYobiE9PUpoKXtjb25zdCB0PTEwKipuO0poPW4sdGQ9ZnVuY3Rp
b24obil7bGV0IGU9MTt0aGlzLl8rPW5bMF07Zm9yKGNvbnN0IHI9bi5sZW5ndGg7ZTxyOysrZSl0
aGlzLl8rPU1hdGgucm91bmQoYXJndW1lbnRzW2VdKnQpL3QrbltlXX19cmV0dXJuIHRkfSh0KSx0
aGlzLl9yYWRpdXM9NC41LHRoaXMuXz0iIn1wb2ludFJhZGl1cyh0KXtyZXR1cm4gdGhpcy5fcmFk
aXVzPSt0LHRoaXN9cG9seWdvblN0YXJ0KCl7dGhpcy5fbGluZT0wfXBvbHlnb25FbmQoKXt0aGlz
Ll9saW5lPU5hTn1saW5lU3RhcnQoKXt0aGlzLl9wb2ludD0wfWxpbmVFbmQoKXswPT09dGhpcy5f
bGluZSYmKHRoaXMuXys9IloiKSx0aGlzLl9wb2ludD1OYU59cG9pbnQodCxuKXtzd2l0Y2godGhp
cy5fcG9pbnQpe2Nhc2UgMDp0aGlzLl9hcHBlbmRgTSR7dH0sJHtufWAsdGhpcy5fcG9pbnQ9MTti
cmVhaztjYXNlIDE6dGhpcy5fYXBwZW5kYEwke3R9LCR7bn1gO2JyZWFrO2RlZmF1bHQ6aWYodGhp
cy5fYXBwZW5kYE0ke3R9LCR7bn1gLHRoaXMuX3JhZGl1cyE9PW5kfHx0aGlzLl9hcHBlbmQhPT10
ZCl7Y29uc3QgdD10aGlzLl9yYWRpdXMsbj10aGlzLl87dGhpcy5fPSIiLHRoaXMuX2FwcGVuZGBt
MCwke3R9YSR7dH0sJHt0fSAwIDEsMSAwLCR7LTIqdH1hJHt0fSwke3R9IDAgMSwxIDAsJHsyKnR9
emAsbmQ9dCx0ZD10aGlzLl9hcHBlbmQsZWQ9dGhpcy5fLHRoaXMuXz1ufXRoaXMuXys9ZWR9fXJl
c3VsdCgpe2NvbnN0IHQ9dGhpcy5fO3JldHVybiB0aGlzLl89IiIsdC5sZW5ndGg/dDpudWxsfX1m
dW5jdGlvbiBpZCh0KXtsZXQgbj0xO3RoaXMuXys9dFswXTtmb3IoY29uc3QgZT10Lmxlbmd0aDtu
PGU7KytuKXRoaXMuXys9YXJndW1lbnRzW25dK3Rbbl19ZnVuY3Rpb24gb2QodCl7cmV0dXJuIGZ1
bmN0aW9uKG4pe3ZhciBlPW5ldyBhZDtmb3IodmFyIHIgaW4gdCllW3JdPXRbcl07cmV0dXJuIGUu
c3RyZWFtPW4sZX19ZnVuY3Rpb24gYWQoKXt9ZnVuY3Rpb24gdWQodCxuLGUpe3ZhciByPXQuY2xp
cEV4dGVudCYmdC5jbGlwRXh0ZW50KCk7cmV0dXJuIHQuc2NhbGUoMTUwKS50cmFuc2xhdGUoWzAs
MF0pLG51bGwhPXImJnQuY2xpcEV4dGVudChudWxsKSxqZihlLHQuc3RyZWFtKHhoKSksbih4aC5y
ZXN1bHQoKSksbnVsbCE9ciYmdC5jbGlwRXh0ZW50KHIpLHR9ZnVuY3Rpb24gY2QodCxuLGUpe3Jl
dHVybiB1ZCh0LChmdW5jdGlvbihlKXt2YXIgcj1uWzFdWzBdLW5bMF1bMF0saT1uWzFdWzFdLW5b
MF1bMV0sbz1NYXRoLm1pbihyLyhlWzFdWzBdLWVbMF1bMF0pLGkvKGVbMV1bMV0tZVswXVsxXSkp
LGE9K25bMF1bMF0rKHItbyooZVsxXVswXStlWzBdWzBdKSkvMix1PStuWzBdWzFdKyhpLW8qKGVb
MV1bMV0rZVswXVsxXSkpLzI7dC5zY2FsZSgxNTAqbykudHJhbnNsYXRlKFthLHVdKX0pLGUpfWZ1
bmN0aW9uIGZkKHQsbixlKXtyZXR1cm4gY2QodCxbWzAsMF0sbl0sZSl9ZnVuY3Rpb24gc2QodCxu
LGUpe3JldHVybiB1ZCh0LChmdW5jdGlvbihlKXt2YXIgcj0rbixpPXIvKGVbMV1bMF0tZVswXVsw
XSksbz0oci1pKihlWzFdWzBdK2VbMF1bMF0pKS8yLGE9LWkqZVswXVsxXTt0LnNjYWxlKDE1MCpp
KS50cmFuc2xhdGUoW28sYV0pfSksZSl9ZnVuY3Rpb24gbGQodCxuLGUpe3JldHVybiB1ZCh0LChm
dW5jdGlvbihlKXt2YXIgcj0rbixpPXIvKGVbMV1bMV0tZVswXVsxXSksbz0taSplWzBdWzBdLGE9
KHItaSooZVsxXVsxXStlWzBdWzFdKSkvMjt0LnNjYWxlKDE1MCppKS50cmFuc2xhdGUoW28sYV0p
fSksZSl9YWQucHJvdG90eXBlPXtjb25zdHJ1Y3RvcjphZCxwb2ludDpmdW5jdGlvbih0LG4pe3Ro
aXMuc3RyZWFtLnBvaW50KHQsbil9LHNwaGVyZTpmdW5jdGlvbigpe3RoaXMuc3RyZWFtLnNwaGVy
ZSgpfSxsaW5lU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLnN0cmVhbS5saW5lU3RhcnQoKX0sbGluZUVu
ZDpmdW5jdGlvbigpe3RoaXMuc3RyZWFtLmxpbmVFbmQoKX0scG9seWdvblN0YXJ0OmZ1bmN0aW9u
KCl7dGhpcy5zdHJlYW0ucG9seWdvblN0YXJ0KCl9LHBvbHlnb25FbmQ6ZnVuY3Rpb24oKXt0aGlz
LnN0cmVhbS5wb2x5Z29uRW5kKCl9fTt2YXIgaGQ9MTYsZGQ9QWYoMzAqeGYpO2Z1bmN0aW9uIHBk
KHQsbil7cmV0dXJuK24/ZnVuY3Rpb24odCxuKXtmdW5jdGlvbiBlKHIsaSxvLGEsdSxjLGYscyxs
LGgsZCxwLGcseSl7dmFyIHY9Zi1yLF89cy1pLGI9dip2K18qXztpZihiPjQqbiYmZy0tKXt2YXIg
bT1hK2gseD11K2Qsdz1jK3AsTT0kZihtKm0reCp4K3cqdyksVD1GZih3Lz1NKSxBPXdmKHdmKHcp
LTEpPHBmfHx3ZihvLWwpPHBmPyhvK2wpLzI6VGYoeCxtKSxTPXQoQSxUKSxFPVNbMF0sTj1TWzFd
LGs9RS1yLEM9Ti1pLFA9XyprLXYqQzsoUCpQL2I+bnx8d2YoKHYqaytfKkMpL2ItLjUpPi4zfHxh
KmgrdSpkK2MqcDxkZCkmJihlKHIsaSxvLGEsdSxjLEUsTixBLG0vPU0seC89TSx3LGcseSkseS5w
b2ludChFLE4pLGUoRSxOLEEsbSx4LHcsZixzLGwsaCxkLHAsZyx5KSl9fXJldHVybiBmdW5jdGlv
bihuKXt2YXIgcixpLG8sYSx1LGMsZixzLGwsaCxkLHAsZz17cG9pbnQ6eSxsaW5lU3RhcnQ6dixs
aW5lRW5kOmIscG9seWdvblN0YXJ0OmZ1bmN0aW9uKCl7bi5wb2x5Z29uU3RhcnQoKSxnLmxpbmVT
dGFydD1tfSxwb2x5Z29uRW5kOmZ1bmN0aW9uKCl7bi5wb2x5Z29uRW5kKCksZy5saW5lU3RhcnQ9
dn19O2Z1bmN0aW9uIHkoZSxyKXtlPXQoZSxyKSxuLnBvaW50KGVbMF0sZVsxXSl9ZnVuY3Rpb24g
digpe3M9TmFOLGcucG9pbnQ9XyxuLmxpbmVTdGFydCgpfWZ1bmN0aW9uIF8ocixpKXt2YXIgbz1n
cyhbcixpXSksYT10KHIsaSk7ZShzLGwsZixoLGQscCxzPWFbMF0sbD1hWzFdLGY9cixoPW9bMF0s
ZD1vWzFdLHA9b1syXSxoZCxuKSxuLnBvaW50KHMsbCl9ZnVuY3Rpb24gYigpe2cucG9pbnQ9eSxu
LmxpbmVFbmQoKX1mdW5jdGlvbiBtKCl7digpLGcucG9pbnQ9eCxnLmxpbmVFbmQ9d31mdW5jdGlv
biB4KHQsbil7XyhyPXQsbiksaT1zLG89bCxhPWgsdT1kLGM9cCxnLnBvaW50PV99ZnVuY3Rpb24g
dygpe2UocyxsLGYsaCxkLHAsaSxvLHIsYSx1LGMsaGQsbiksZy5saW5lRW5kPWIsYigpfXJldHVy
biBnfX0odCxuKTpmdW5jdGlvbih0KXtyZXR1cm4gb2Qoe3BvaW50OmZ1bmN0aW9uKG4sZSl7bj10
KG4sZSksdGhpcy5zdHJlYW0ucG9pbnQoblswXSxuWzFdKX19KX0odCl9dmFyIGdkPW9kKHtwb2lu
dDpmdW5jdGlvbih0LG4pe3RoaXMuc3RyZWFtLnBvaW50KHQqeGYsbip4Zil9fSk7ZnVuY3Rpb24g
eWQodCxuLGUscixpLG8pe2lmKCFvKXJldHVybiBmdW5jdGlvbih0LG4sZSxyLGkpe2Z1bmN0aW9u
IG8obyxhKXtyZXR1cm5bbit0KihvKj1yKSxlLXQqKGEqPWkpXX1yZXR1cm4gby5pbnZlcnQ9ZnVu
Y3Rpb24obyxhKXtyZXR1cm5bKG8tbikvdCpyLChlLWEpL3QqaV19LG99KHQsbixlLHIsaSk7dmFy
IGE9QWYobyksdT1QZihvKSxjPWEqdCxmPXUqdCxzPWEvdCxsPXUvdCxoPSh1KmUtYSpuKS90LGQ9
KHUqbithKmUpL3Q7ZnVuY3Rpb24gcCh0LG8pe3JldHVybltjKih0Kj1yKS1mKihvKj1pKStuLGUt
Zip0LWMqb119cmV0dXJuIHAuaW52ZXJ0PWZ1bmN0aW9uKHQsbil7cmV0dXJuW3IqKHMqdC1sKm4r
aCksaSooZC1sKnQtcypuKV19LHB9ZnVuY3Rpb24gdmQodCl7cmV0dXJuIF9kKChmdW5jdGlvbigp
e3JldHVybiB0fSkpKCl9ZnVuY3Rpb24gX2QodCl7dmFyIG4sZSxyLGksbyxhLHUsYyxmLHMsbD0x
NTAsaD00ODAsZD0yNTAscD0wLGc9MCx5PTAsdj0wLF89MCxiPTAsbT0xLHg9MSx3PW51bGwsTT1B
bCxUPW51bGwsQT1yaCxTPS41O2Z1bmN0aW9uIEUodCl7cmV0dXJuIGModFswXSp4Zix0WzFdKnhm
KX1mdW5jdGlvbiBOKHQpe3JldHVybih0PWMuaW52ZXJ0KHRbMF0sdFsxXSkpJiZbdFswXSptZix0
WzFdKm1mXX1mdW5jdGlvbiBrKCl7dmFyIHQ9eWQobCwwLDAsbSx4LGIpLmFwcGx5KG51bGwsbihw
LGcpKSxyPXlkKGwsaC10WzBdLGQtdFsxXSxtLHgsYik7cmV0dXJuIGU9Y2woeSx2LF8pLHU9YWwo
bixyKSxjPWFsKGUsdSksYT1wZCh1LFMpLEMoKX1mdW5jdGlvbiBDKCl7cmV0dXJuIGY9cz1udWxs
LEV9cmV0dXJuIEUuc3RyZWFtPWZ1bmN0aW9uKHQpe3JldHVybiBmJiZzPT09dD9mOmY9Z2QoZnVu
Y3Rpb24odCl7cmV0dXJuIG9kKHtwb2ludDpmdW5jdGlvbihuLGUpe3ZhciByPXQobixlKTtyZXR1
cm4gdGhpcy5zdHJlYW0ucG9pbnQoclswXSxyWzFdKX19KX0oZSkoTShhKEEocz10KSkpKSl9LEUu
cHJlY2xpcD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oTT10LHc9dm9pZCAw
LEMoKSk6TX0sRS5wb3N0Y2xpcD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8o
QT10LFQ9cj1pPW89bnVsbCxDKCkpOkF9LEUuY2xpcEFuZ2xlPWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyhNPSt0P1NsKHc9dCp4Zik6KHc9bnVsbCxBbCksQygpKTp3Km1mfSxF
LmNsaXBFeHRlbnQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KEE9bnVsbD09
dD8oVD1yPWk9bz1udWxsLHJoKTokbChUPSt0WzBdWzBdLHI9K3RbMF1bMV0saT0rdFsxXVswXSxv
PSt0WzFdWzFdKSxDKCkpOm51bGw9PVQ/bnVsbDpbW1Qscl0sW2ksb11dfSxFLnNjYWxlPWZ1bmN0
aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhsPSt0LGsoKSk6bH0sRS50cmFuc2xhdGU9
ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGg9K3RbMF0sZD0rdFsxXSxrKCkp
OltoLGRdfSxFLmNlbnRlcj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocD10
WzBdJTM2MCp4ZixnPXRbMV0lMzYwKnhmLGsoKSk6W3AqbWYsZyptZl19LEUucm90YXRlPWZ1bmN0
aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh5PXRbMF0lMzYwKnhmLHY9dFsxXSUzNjAq
eGYsXz10Lmxlbmd0aD4yP3RbMl0lMzYwKnhmOjAsaygpKTpbeSptZix2Km1mLF8qbWZdfSxFLmFu
Z2xlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhiPXQlMzYwKnhmLGsoKSk6
YiptZn0sRS5yZWZsZWN0WD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obT10
Py0xOjEsaygpKTptPDB9LEUucmVmbGVjdFk9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/KHg9dD8tMToxLGsoKSk6eDwwfSxFLnByZWNpc2lvbj1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8oYT1wZCh1LFM9dCp0KSxDKCkpOiRmKFMpfSxFLmZpdEV4dGVudD1m
dW5jdGlvbih0LG4pe3JldHVybiBjZChFLHQsbil9LEUuZml0U2l6ZT1mdW5jdGlvbih0LG4pe3Jl
dHVybiBmZChFLHQsbil9LEUuZml0V2lkdGg9ZnVuY3Rpb24odCxuKXtyZXR1cm4gc2QoRSx0LG4p
fSxFLmZpdEhlaWdodD1mdW5jdGlvbih0LG4pe3JldHVybiBsZChFLHQsbil9LGZ1bmN0aW9uKCl7
cmV0dXJuIG49dC5hcHBseSh0aGlzLGFyZ3VtZW50cyksRS5pbnZlcnQ9bi5pbnZlcnQmJk4saygp
fX1mdW5jdGlvbiBiZCh0KXt2YXIgbj0wLGU9eWYvMyxyPV9kKHQpLGk9cihuLGUpO3JldHVybiBp
LnBhcmFsbGVscz1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD9yKG49dFswXSp4
ZixlPXRbMV0qeGYpOltuKm1mLGUqbWZdfSxpfWZ1bmN0aW9uIG1kKHQsbil7dmFyIGU9UGYodCks
cj0oZStQZihuKSkvMjtpZih3ZihyKTxwZilyZXR1cm4gZnVuY3Rpb24odCl7dmFyIG49QWYodCk7
ZnVuY3Rpb24gZSh0LGUpe3JldHVyblt0Km4sUGYoZSkvbl19cmV0dXJuIGUuaW52ZXJ0PWZ1bmN0
aW9uKHQsZSl7cmV0dXJuW3QvbixGZihlKm4pXX0sZX0odCk7dmFyIGk9MStlKigyKnItZSksbz0k
ZihpKS9yO2Z1bmN0aW9uIGEodCxuKXt2YXIgZT0kZihpLTIqcipQZihuKSkvcjtyZXR1cm5bZSpQ
Zih0Kj1yKSxvLWUqQWYodCldfXJldHVybiBhLmludmVydD1mdW5jdGlvbih0LG4pe3ZhciBlPW8t
bixhPVRmKHQsd2YoZSkpKnpmKGUpO3JldHVybiBlKnI8MCYmKGEtPXlmKnpmKHQpKnpmKGUpKSxb
YS9yLEZmKChpLSh0KnQrZSplKSpyKnIpLygyKnIpKV19LGF9ZnVuY3Rpb24geGQoKXtyZXR1cm4g
YmQobWQpLnNjYWxlKDE1NS40MjQpLmNlbnRlcihbMCwzMy42NDQyXSl9ZnVuY3Rpb24gd2QoKXty
ZXR1cm4geGQoKS5wYXJhbGxlbHMoWzI5LjUsNDUuNV0pLnNjYWxlKDEwNzApLnRyYW5zbGF0ZShb
NDgwLDI1MF0pLnJvdGF0ZShbOTYsMF0pLmNlbnRlcihbLS42LDM4LjddKX1mdW5jdGlvbiBNZCh0
KXtyZXR1cm4gZnVuY3Rpb24obixlKXt2YXIgcj1BZihuKSxpPUFmKGUpLG89dChyKmkpO3JldHVy
biBvPT09MS8wP1syLDBdOltvKmkqUGYobiksbypQZihlKV19fWZ1bmN0aW9uIFRkKHQpe3JldHVy
biBmdW5jdGlvbihuLGUpe3ZhciByPSRmKG4qbitlKmUpLGk9dChyKSxvPVBmKGkpLGE9QWYoaSk7
cmV0dXJuW1RmKG4qbyxyKmEpLEZmKHImJmUqby9yKV19fXZhciBBZD1NZCgoZnVuY3Rpb24odCl7
cmV0dXJuICRmKDIvKDErdCkpfSkpO0FkLmludmVydD1UZCgoZnVuY3Rpb24odCl7cmV0dXJuIDIq
RmYodC8yKX0pKTt2YXIgU2Q9TWQoKGZ1bmN0aW9uKHQpe3JldHVybih0PVJmKHQpKSYmdC9QZih0
KX0pKTtmdW5jdGlvbiBFZCh0LG4pe3JldHVyblt0LGtmKERmKCh2ZituKS8yKSldfWZ1bmN0aW9u
IE5kKHQpe3ZhciBuLGUscixpPXZkKHQpLG89aS5jZW50ZXIsYT1pLnNjYWxlLHU9aS50cmFuc2xh
dGUsYz1pLmNsaXBFeHRlbnQsZj1udWxsO2Z1bmN0aW9uIHMoKXt2YXIgbz15ZiphKCksdT1pKGhs
KGkucm90YXRlKCkpLmludmVydChbMCwwXSkpO3JldHVybiBjKG51bGw9PWY/W1t1WzBdLW8sdVsx
XS1vXSxbdVswXStvLHVbMV0rb11dOnQ9PT1FZD9bW01hdGgubWF4KHVbMF0tbyxmKSxuXSxbTWF0
aC5taW4odVswXStvLGUpLHJdXTpbW2YsTWF0aC5tYXgodVsxXS1vLG4pXSxbZSxNYXRoLm1pbih1
WzFdK28scildXSl9cmV0dXJuIGkuc2NhbGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/KGEodCkscygpKTphKCl9LGkudHJhbnNsYXRlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1
bWVudHMubGVuZ3RoPyh1KHQpLHMoKSk6dSgpfSxpLmNlbnRlcj1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8obyh0KSxzKCkpOm8oKX0saS5jbGlwRXh0ZW50PWZ1bmN0aW9uKHQp
e3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhudWxsPT10P2Y9bj1lPXI9bnVsbDooZj0rdFswXVsw
XSxuPSt0WzBdWzFdLGU9K3RbMV1bMF0scj0rdFsxXVsxXSkscygpKTpudWxsPT1mP251bGw6W1tm
LG5dLFtlLHJdXX0scygpfWZ1bmN0aW9uIGtkKHQpe3JldHVybiBEZigodmYrdCkvMil9ZnVuY3Rp
b24gQ2QodCxuKXt2YXIgZT1BZih0KSxyPXQ9PT1uP1BmKHQpOmtmKGUvQWYobikpL2tmKGtkKG4p
L2tkKHQpKSxpPWUqQ2Yoa2QodCkscikvcjtpZighcilyZXR1cm4gRWQ7ZnVuY3Rpb24gbyh0LG4p
e2k+MD9uPC12ZitwZiYmKG49LXZmK3BmKTpuPnZmLXBmJiYobj12Zi1wZik7dmFyIGU9aS9DZihr
ZChuKSxyKTtyZXR1cm5bZSpQZihyKnQpLGktZSpBZihyKnQpXX1yZXR1cm4gby5pbnZlcnQ9ZnVu
Y3Rpb24odCxuKXt2YXIgZT1pLW4sbz16ZihyKSokZih0KnQrZSplKSxhPVRmKHQsd2YoZSkpKnpm
KGUpO3JldHVybiBlKnI8MCYmKGEtPXlmKnpmKHQpKnpmKGUpKSxbYS9yLDIqTWYoQ2YoaS9vLDEv
cikpLXZmXX0sb31mdW5jdGlvbiBQZCh0LG4pe3JldHVyblt0LG5dfWZ1bmN0aW9uIHpkKHQsbil7
dmFyIGU9QWYodCkscj10PT09bj9QZih0KTooZS1BZihuKSkvKG4tdCksaT1lL3IrdDtpZih3Zihy
KTxwZilyZXR1cm4gUGQ7ZnVuY3Rpb24gbyh0LG4pe3ZhciBlPWktbixvPXIqdDtyZXR1cm5bZSpQ
ZihvKSxpLWUqQWYobyldfXJldHVybiBvLmludmVydD1mdW5jdGlvbih0LG4pe3ZhciBlPWktbixv
PVRmKHQsd2YoZSkpKnpmKGUpO3JldHVybiBlKnI8MCYmKG8tPXlmKnpmKHQpKnpmKGUpKSxbby9y
LGktemYocikqJGYodCp0K2UqZSldfSxvfVNkLmludmVydD1UZCgoZnVuY3Rpb24odCl7cmV0dXJu
IHR9KSksRWQuaW52ZXJ0PWZ1bmN0aW9uKHQsbil7cmV0dXJuW3QsMipNZihFZihuKSktdmZdfSxQ
ZC5pbnZlcnQ9UGQ7dmFyICRkPTEuMzQwMjY0LERkPS0uMDgxMTA2LFJkPTg5M2UtNixGZD0uMDAz
Nzk2LHFkPSRmKDMpLzI7ZnVuY3Rpb24gVWQodCxuKXt2YXIgZT1GZihxZCpQZihuKSkscj1lKmUs
aT1yKnIqcjtyZXR1cm5bdCpBZihlKS8ocWQqKCRkKzMqRGQqcitpKig3KlJkKzkqRmQqcikpKSxl
KigkZCtEZCpyK2kqKFJkK0ZkKnIpKV19ZnVuY3Rpb24gSWQodCxuKXt2YXIgZT1BZihuKSxyPUFm
KHQpKmU7cmV0dXJuW2UqUGYodCkvcixQZihuKS9yXX1mdW5jdGlvbiBPZCh0LG4pe3ZhciBlPW4q
bixyPWUqZTtyZXR1cm5bdCooLjg3MDctLjEzMTk3OSplK3IqKHIqKC4wMDM5NzEqZS0uMDAxNTI5
KnIpLS4wMTM3OTEpKSxuKigxLjAwNzIyNitlKiguMDE1MDg1K3IqKC4wMjg4NzQqZS0uMDQ0NDc1
LS4wMDU5MTYqcikpKV19ZnVuY3Rpb24gQmQodCxuKXtyZXR1cm5bQWYobikqUGYodCksUGYobild
fWZ1bmN0aW9uIFlkKHQsbil7dmFyIGU9QWYobikscj0xK0FmKHQpKmU7cmV0dXJuW2UqUGYodCkv
cixQZihuKS9yXX1mdW5jdGlvbiBMZCh0LG4pe3JldHVybltrZihEZigodmYrbikvMikpLC10XX1m
dW5jdGlvbiBqZCh0LG4pe3JldHVybiB0LnBhcmVudD09PW4ucGFyZW50PzE6Mn1mdW5jdGlvbiBI
ZCh0LG4pe3JldHVybiB0K24ueH1mdW5jdGlvbiBYZCh0LG4pe3JldHVybiBNYXRoLm1heCh0LG4u
eSl9ZnVuY3Rpb24gR2QodCl7dmFyIG49MCxlPXQuY2hpbGRyZW4scj1lJiZlLmxlbmd0aDtpZihy
KWZvcig7LS1yPj0wOyluKz1lW3JdLnZhbHVlO2Vsc2Ugbj0xO3QudmFsdWU9bn1mdW5jdGlvbiBW
ZCh0LG4pe3QgaW5zdGFuY2VvZiBNYXA/KHQ9W3ZvaWQgMCx0XSx2b2lkIDA9PT1uJiYobj1aZCkp
OnZvaWQgMD09PW4mJihuPVdkKTtmb3IodmFyIGUscixpLG8sYSx1PW5ldyBKZCh0KSxjPVt1XTtl
PWMucG9wKCk7KWlmKChpPW4oZS5kYXRhKSkmJihhPShpPUFycmF5LmZyb20oaSkpLmxlbmd0aCkp
Zm9yKGUuY2hpbGRyZW49aSxvPWEtMTtvPj0wOy0tbyljLnB1c2gocj1pW29dPW5ldyBKZChpW29d
KSksci5wYXJlbnQ9ZSxyLmRlcHRoPWUuZGVwdGgrMTtyZXR1cm4gdS5lYWNoQmVmb3JlKFFkKX1m
dW5jdGlvbiBXZCh0KXtyZXR1cm4gdC5jaGlsZHJlbn1mdW5jdGlvbiBaZCh0KXtyZXR1cm4gQXJy
YXkuaXNBcnJheSh0KT90WzFdOm51bGx9ZnVuY3Rpb24gS2QodCl7dm9pZCAwIT09dC5kYXRhLnZh
bHVlJiYodC52YWx1ZT10LmRhdGEudmFsdWUpLHQuZGF0YT10LmRhdGEuZGF0YX1mdW5jdGlvbiBR
ZCh0KXt2YXIgbj0wO2Rve3QuaGVpZ2h0PW59d2hpbGUoKHQ9dC5wYXJlbnQpJiZ0LmhlaWdodDwr
K24pfWZ1bmN0aW9uIEpkKHQpe3RoaXMuZGF0YT10LHRoaXMuZGVwdGg9dGhpcy5oZWlnaHQ9MCx0
aGlzLnBhcmVudD1udWxsfWZ1bmN0aW9uIHRwKHQpe3JldHVybiBudWxsPT10P251bGw6bnAodCl9
ZnVuY3Rpb24gbnAodCl7aWYoImZ1bmN0aW9uIiE9dHlwZW9mIHQpdGhyb3cgbmV3IEVycm9yO3Jl
dHVybiB0fWZ1bmN0aW9uIGVwKCl7cmV0dXJuIDB9ZnVuY3Rpb24gcnAodCl7cmV0dXJuIGZ1bmN0
aW9uKCl7cmV0dXJuIHR9fVVkLmludmVydD1mdW5jdGlvbih0LG4pe2Zvcih2YXIgZSxyPW4saT1y
KnIsbz1pKmkqaSxhPTA7YTwxMiYmKG89KGk9KHItPWU9KHIqKCRkK0RkKmkrbyooUmQrRmQqaSkp
LW4pLygkZCszKkRkKmkrbyooNypSZCs5KkZkKmkpKSkqcikqaSppLCEod2YoZSk8Z2YpKTsrK2Ep
O3JldHVybltxZCp0KigkZCszKkRkKmkrbyooNypSZCs5KkZkKmkpKS9BZihyKSxGZihQZihyKS9x
ZCldfSxJZC5pbnZlcnQ9VGQoTWYpLE9kLmludmVydD1mdW5jdGlvbih0LG4pe3ZhciBlLHI9bixp
PTI1O2Rve3ZhciBvPXIqcixhPW8qbztyLT1lPShyKigxLjAwNzIyNitvKiguMDE1MDg1K2EqKC4w
Mjg4NzQqby0uMDQ0NDc1LS4wMDU5MTYqYSkpKS1uKS8oMS4wMDcyMjYrbyooLjA0NTI1NSthKigu
MjU5ODY2Km8tLjMxMTMyNS0uMDA1OTE2KjExKmEpKSl9d2hpbGUod2YoZSk+cGYmJi0taT4wKTty
ZXR1cm5bdC8oLjg3MDcrKG89cipyKSoobyoobypvKm8qKC4wMDM5NzEtLjAwMTUyOSpvKS0uMDEz
NzkxKS0uMTMxOTc5KSkscl19LEJkLmludmVydD1UZChGZiksWWQuaW52ZXJ0PVRkKChmdW5jdGlv
bih0KXtyZXR1cm4gMipNZih0KX0pKSxMZC5pbnZlcnQ9ZnVuY3Rpb24odCxuKXtyZXR1cm5bLW4s
MipNZihFZih0KSktdmZdfSxKZC5wcm90b3R5cGU9VmQucHJvdG90eXBlPXtjb25zdHJ1Y3RvcjpK
ZCxjb3VudDpmdW5jdGlvbigpe3JldHVybiB0aGlzLmVhY2hBZnRlcihHZCl9LGVhY2g6ZnVuY3Rp
b24odCxuKXtsZXQgZT0tMTtmb3IoY29uc3QgciBvZiB0aGlzKXQuY2FsbChuLHIsKytlLHRoaXMp
O3JldHVybiB0aGlzfSxlYWNoQWZ0ZXI6ZnVuY3Rpb24odCxuKXtmb3IodmFyIGUscixpLG89dGhp
cyxhPVtvXSx1PVtdLGM9LTE7bz1hLnBvcCgpOylpZih1LnB1c2gobyksZT1vLmNoaWxkcmVuKWZv
cihyPTAsaT1lLmxlbmd0aDtyPGk7KytyKWEucHVzaChlW3JdKTtmb3IoO289dS5wb3AoKTspdC5j
YWxsKG4sbywrK2MsdGhpcyk7cmV0dXJuIHRoaXN9LGVhY2hCZWZvcmU6ZnVuY3Rpb24odCxuKXtm
b3IodmFyIGUscixpPXRoaXMsbz1baV0sYT0tMTtpPW8ucG9wKCk7KWlmKHQuY2FsbChuLGksKyth
LHRoaXMpLGU9aS5jaGlsZHJlbilmb3Iocj1lLmxlbmd0aC0xO3I+PTA7LS1yKW8ucHVzaChlW3Jd
KTtyZXR1cm4gdGhpc30sZmluZDpmdW5jdGlvbih0LG4pe2xldCBlPS0xO2Zvcihjb25zdCByIG9m
IHRoaXMpaWYodC5jYWxsKG4sciwrK2UsdGhpcykpcmV0dXJuIHJ9LHN1bTpmdW5jdGlvbih0KXty
ZXR1cm4gdGhpcy5lYWNoQWZ0ZXIoKGZ1bmN0aW9uKG4pe2Zvcih2YXIgZT0rdChuLmRhdGEpfHww
LHI9bi5jaGlsZHJlbixpPXImJnIubGVuZ3RoOy0taT49MDspZSs9cltpXS52YWx1ZTtuLnZhbHVl
PWV9KSl9LHNvcnQ6ZnVuY3Rpb24odCl7cmV0dXJuIHRoaXMuZWFjaEJlZm9yZSgoZnVuY3Rpb24o
bil7bi5jaGlsZHJlbiYmbi5jaGlsZHJlbi5zb3J0KHQpfSkpfSxwYXRoOmZ1bmN0aW9uKHQpe2Zv
cih2YXIgbj10aGlzLGU9ZnVuY3Rpb24odCxuKXtpZih0PT09bilyZXR1cm4gdDt2YXIgZT10LmFu
Y2VzdG9ycygpLHI9bi5hbmNlc3RvcnMoKSxpPW51bGw7dD1lLnBvcCgpLG49ci5wb3AoKTtmb3Io
O3Q9PT1uOylpPXQsdD1lLnBvcCgpLG49ci5wb3AoKTtyZXR1cm4gaX0obix0KSxyPVtuXTtuIT09
ZTspbj1uLnBhcmVudCxyLnB1c2gobik7Zm9yKHZhciBpPXIubGVuZ3RoO3QhPT1lOylyLnNwbGlj
ZShpLDAsdCksdD10LnBhcmVudDtyZXR1cm4gcn0sYW5jZXN0b3JzOmZ1bmN0aW9uKCl7Zm9yKHZh
ciB0PXRoaXMsbj1bdF07dD10LnBhcmVudDspbi5wdXNoKHQpO3JldHVybiBufSxkZXNjZW5kYW50
czpmdW5jdGlvbigpe3JldHVybiBBcnJheS5mcm9tKHRoaXMpfSxsZWF2ZXM6ZnVuY3Rpb24oKXt2
YXIgdD1bXTtyZXR1cm4gdGhpcy5lYWNoQmVmb3JlKChmdW5jdGlvbihuKXtuLmNoaWxkcmVufHx0
LnB1c2gobil9KSksdH0sbGlua3M6ZnVuY3Rpb24oKXt2YXIgdD10aGlzLG49W107cmV0dXJuIHQu
ZWFjaCgoZnVuY3Rpb24oZSl7ZSE9PXQmJm4ucHVzaCh7c291cmNlOmUucGFyZW50LHRhcmdldDpl
fSl9KSksbn0sY29weTpmdW5jdGlvbigpe3JldHVybiBWZCh0aGlzKS5lYWNoQmVmb3JlKEtkKX0s
W1N5bWJvbC5pdGVyYXRvcl06ZnVuY3Rpb24qKCl7dmFyIHQsbixlLHIsaT10aGlzLG89W2ldO2Rv
e2Zvcih0PW8ucmV2ZXJzZSgpLG89W107aT10LnBvcCgpOylpZih5aWVsZCBpLG49aS5jaGlsZHJl
bilmb3IoZT0wLHI9bi5sZW5ndGg7ZTxyOysrZSlvLnB1c2gobltlXSl9d2hpbGUoby5sZW5ndGgp
fX07Y29uc3QgaXA9MTY2NDUyNSxvcD0xMDEzOTA0MjIzLGFwPTQyOTQ5NjcyOTY7ZnVuY3Rpb24g
dXAoKXtsZXQgdD0xO3JldHVybigpPT4odD0oaXAqdCtvcCklYXApL2FwfWZ1bmN0aW9uIGNwKHQs
bil7Zm9yKHZhciBlLHIsaT0wLG89KHQ9ZnVuY3Rpb24odCxuKXtsZXQgZSxyLGk9dC5sZW5ndGg7
Zm9yKDtpOylyPW4oKSppLS18MCxlPXRbaV0sdFtpXT10W3JdLHRbcl09ZTtyZXR1cm4gdH0oQXJy
YXkuZnJvbSh0KSxuKSkubGVuZ3RoLGE9W107aTxvOyllPXRbaV0sciYmbHAocixlKT8rK2k6KHI9
ZHAoYT1mcChhLGUpKSxpPTApO3JldHVybiByfWZ1bmN0aW9uIGZwKHQsbil7dmFyIGUscjtpZiho
cChuLHQpKXJldHVybltuXTtmb3IoZT0wO2U8dC5sZW5ndGg7KytlKWlmKHNwKG4sdFtlXSkmJmhw
KHBwKHRbZV0sbiksdCkpcmV0dXJuW3RbZV0sbl07Zm9yKGU9MDtlPHQubGVuZ3RoLTE7KytlKWZv
cihyPWUrMTtyPHQubGVuZ3RoOysrcilpZihzcChwcCh0W2VdLHRbcl0pLG4pJiZzcChwcCh0W2Vd
LG4pLHRbcl0pJiZzcChwcCh0W3JdLG4pLHRbZV0pJiZocChncCh0W2VdLHRbcl0sbiksdCkpcmV0
dXJuW3RbZV0sdFtyXSxuXTt0aHJvdyBuZXcgRXJyb3J9ZnVuY3Rpb24gc3AodCxuKXt2YXIgZT10
LnItbi5yLHI9bi54LXQueCxpPW4ueS10Lnk7cmV0dXJuIGU8MHx8ZSplPHIqcitpKml9ZnVuY3Rp
b24gbHAodCxuKXt2YXIgZT10LnItbi5yKzFlLTkqTWF0aC5tYXgodC5yLG4uciwxKSxyPW4ueC10
LngsaT1uLnktdC55O3JldHVybiBlPjAmJmUqZT5yKnIraSppfWZ1bmN0aW9uIGhwKHQsbil7Zm9y
KHZhciBlPTA7ZTxuLmxlbmd0aDsrK2UpaWYoIWxwKHQsbltlXSkpcmV0dXJuITE7cmV0dXJuITB9
ZnVuY3Rpb24gZHAodCl7c3dpdGNoKHQubGVuZ3RoKXtjYXNlIDE6cmV0dXJuIGZ1bmN0aW9uKHQp
e3JldHVybnt4OnQueCx5OnQueSxyOnQucn19KHRbMF0pO2Nhc2UgMjpyZXR1cm4gcHAodFswXSx0
WzFdKTtjYXNlIDM6cmV0dXJuIGdwKHRbMF0sdFsxXSx0WzJdKX19ZnVuY3Rpb24gcHAodCxuKXt2
YXIgZT10Lngscj10LnksaT10LnIsbz1uLngsYT1uLnksdT1uLnIsYz1vLWUsZj1hLXIscz11LWks
bD1NYXRoLnNxcnQoYypjK2YqZik7cmV0dXJue3g6KGUrbytjL2wqcykvMix5OihyK2ErZi9sKnMp
LzIscjoobCtpK3UpLzJ9fWZ1bmN0aW9uIGdwKHQsbixlKXt2YXIgcj10LngsaT10Lnksbz10LnIs
YT1uLngsdT1uLnksYz1uLnIsZj1lLngscz1lLnksbD1lLnIsaD1yLWEsZD1yLWYscD1pLXUsZz1p
LXMseT1jLW8sdj1sLW8sXz1yKnIraSppLW8qbyxiPV8tYSphLXUqdStjKmMsbT1fLWYqZi1zKnMr
bCpsLHg9ZCpwLWgqZyx3PShwKm0tZypiKS8oMip4KS1yLE09KGcqeS1wKnYpL3gsVD0oZCpiLWgq
bSkvKDIqeCktaSxBPShoKnYtZCp5KS94LFM9TSpNK0EqQS0xLEU9Mioobyt3Kk0rVCpBKSxOPXcq
dytUKlQtbypvLGs9LShNYXRoLmFicyhTKT4xZS02PyhFK01hdGguc3FydChFKkUtNCpTKk4pKS8o
MipTKTpOL0UpO3JldHVybnt4OnIrdytNKmsseTppK1QrQSprLHI6a319ZnVuY3Rpb24geXAodCxu
LGUpe3ZhciByLGksbyxhLHU9dC54LW4ueCxjPXQueS1uLnksZj11KnUrYypjO2Y/KGk9bi5yK2Uu
cixpKj1pLGE9dC5yK2UucixpPihhKj1hKT8ocj0oZithLWkpLygyKmYpLG89TWF0aC5zcXJ0KE1h
dGgubWF4KDAsYS9mLXIqcikpLGUueD10Lngtcip1LW8qYyxlLnk9dC55LXIqYytvKnUpOihyPShm
K2ktYSkvKDIqZiksbz1NYXRoLnNxcnQoTWF0aC5tYXgoMCxpL2YtcipyKSksZS54PW4ueCtyKnUt
bypjLGUueT1uLnkrcipjK28qdSkpOihlLng9bi54K2UucixlLnk9bi55KX1mdW5jdGlvbiB2cCh0
LG4pe3ZhciBlPXQucituLnItMWUtNixyPW4ueC10LngsaT1uLnktdC55O3JldHVybiBlPjAmJmUq
ZT5yKnIraSppfWZ1bmN0aW9uIF9wKHQpe3ZhciBuPXQuXyxlPXQubmV4dC5fLHI9bi5yK2Uucixp
PShuLngqZS5yK2UueCpuLnIpL3Isbz0obi55KmUucitlLnkqbi5yKS9yO3JldHVybiBpKmkrbypv
fWZ1bmN0aW9uIGJwKHQpe3RoaXMuXz10LHRoaXMubmV4dD1udWxsLHRoaXMucHJldmlvdXM9bnVs
bH1mdW5jdGlvbiBtcCh0LG4pe2lmKCEobz0odD1mdW5jdGlvbih0KXtyZXR1cm4ib2JqZWN0Ij09
dHlwZW9mIHQmJiJsZW5ndGgiaW4gdD90OkFycmF5LmZyb20odCl9KHQpKS5sZW5ndGgpKXJldHVy
biAwO3ZhciBlLHIsaSxvLGEsdSxjLGYscyxsLGg7aWYoKGU9dFswXSkueD0wLGUueT0wLCEobz4x
KSlyZXR1cm4gZS5yO2lmKHI9dFsxXSxlLng9LXIucixyLng9ZS5yLHIueT0wLCEobz4yKSlyZXR1
cm4gZS5yK3Iucjt5cChyLGUsaT10WzJdKSxlPW5ldyBicChlKSxyPW5ldyBicChyKSxpPW5ldyBi
cChpKSxlLm5leHQ9aS5wcmV2aW91cz1yLHIubmV4dD1lLnByZXZpb3VzPWksaS5uZXh0PXIucHJl
dmlvdXM9ZTt0OmZvcihjPTM7YzxvOysrYyl7eXAoZS5fLHIuXyxpPXRbY10pLGk9bmV3IGJwKGkp
LGY9ci5uZXh0LHM9ZS5wcmV2aW91cyxsPXIuXy5yLGg9ZS5fLnI7ZG97aWYobDw9aCl7aWYodnAo
Zi5fLGkuXykpe3I9ZixlLm5leHQ9cixyLnByZXZpb3VzPWUsLS1jO2NvbnRpbnVlIHR9bCs9Zi5f
LnIsZj1mLm5leHR9ZWxzZXtpZih2cChzLl8saS5fKSl7KGU9cykubmV4dD1yLHIucHJldmlvdXM9
ZSwtLWM7Y29udGludWUgdH1oKz1zLl8ucixzPXMucHJldmlvdXN9fXdoaWxlKGYhPT1zLm5leHQp
O2ZvcihpLnByZXZpb3VzPWUsaS5uZXh0PXIsZS5uZXh0PXIucHJldmlvdXM9cj1pLGE9X3AoZSk7
KGk9aS5uZXh0KSE9PXI7KSh1PV9wKGkpKTxhJiYoZT1pLGE9dSk7cj1lLm5leHR9Zm9yKGU9W3Iu
X10saT1yOyhpPWkubmV4dCkhPT1yOyllLnB1c2goaS5fKTtmb3IoaT1jcChlLG4pLGM9MDtjPG87
KytjKShlPXRbY10pLngtPWkueCxlLnktPWkueTtyZXR1cm4gaS5yfWZ1bmN0aW9uIHhwKHQpe3Jl
dHVybiBNYXRoLnNxcnQodC52YWx1ZSl9ZnVuY3Rpb24gd3AodCl7cmV0dXJuIGZ1bmN0aW9uKG4p
e24uY2hpbGRyZW58fChuLnI9TWF0aC5tYXgoMCwrdChuKXx8MCkpfX1mdW5jdGlvbiBNcCh0LG4s
ZSl7cmV0dXJuIGZ1bmN0aW9uKHIpe2lmKGk9ci5jaGlsZHJlbil7dmFyIGksbyxhLHU9aS5sZW5n
dGgsYz10KHIpKm58fDA7aWYoYylmb3Iobz0wO288dTsrK28paVtvXS5yKz1jO2lmKGE9bXAoaSxl
KSxjKWZvcihvPTA7bzx1OysrbylpW29dLnItPWM7ci5yPWErY319fWZ1bmN0aW9uIFRwKHQpe3Jl
dHVybiBmdW5jdGlvbihuKXt2YXIgZT1uLnBhcmVudDtuLnIqPXQsZSYmKG4ueD1lLngrdCpuLngs
bi55PWUueSt0Km4ueSl9fWZ1bmN0aW9uIEFwKHQpe3QueDA9TWF0aC5yb3VuZCh0LngwKSx0Lnkw
PU1hdGgucm91bmQodC55MCksdC54MT1NYXRoLnJvdW5kKHQueDEpLHQueTE9TWF0aC5yb3VuZCh0
LnkxKX1mdW5jdGlvbiBTcCh0LG4sZSxyLGkpe2Zvcih2YXIgbyxhPXQuY2hpbGRyZW4sdT0tMSxj
PWEubGVuZ3RoLGY9dC52YWx1ZSYmKHItbikvdC52YWx1ZTsrK3U8YzspKG89YVt1XSkueTA9ZSxv
LnkxPWksby54MD1uLG8ueDE9bis9by52YWx1ZSpmfXZhciBFcD17ZGVwdGg6LTF9LE5wPXt9LGtw
PXt9O2Z1bmN0aW9uIENwKHQpe3JldHVybiB0LmlkfWZ1bmN0aW9uIFBwKHQpe3JldHVybiB0LnBh
cmVudElkfWZ1bmN0aW9uIHpwKHQpe2xldCBuPXQubGVuZ3RoO2lmKG48MilyZXR1cm4iIjtmb3Io
Oy0tbj4xJiYhJHAodCxuKTspO3JldHVybiB0LnNsaWNlKDAsbil9ZnVuY3Rpb24gJHAodCxuKXtp
ZigiLyI9PT10W25dKXtsZXQgZT0wO2Zvcig7bj4wJiYiXFwiPT09dFstLW5dOykrK2U7aWYoMD09
KDEmZSkpcmV0dXJuITB9cmV0dXJuITF9ZnVuY3Rpb24gRHAodCxuKXtyZXR1cm4gdC5wYXJlbnQ9
PT1uLnBhcmVudD8xOjJ9ZnVuY3Rpb24gUnAodCl7dmFyIG49dC5jaGlsZHJlbjtyZXR1cm4gbj9u
WzBdOnQudH1mdW5jdGlvbiBGcCh0KXt2YXIgbj10LmNoaWxkcmVuO3JldHVybiBuP25bbi5sZW5n
dGgtMV06dC50fWZ1bmN0aW9uIHFwKHQsbixlKXt2YXIgcj1lLyhuLmktdC5pKTtuLmMtPXIsbi5z
Kz1lLHQuYys9cixuLnorPWUsbi5tKz1lfWZ1bmN0aW9uIFVwKHQsbixlKXtyZXR1cm4gdC5hLnBh
cmVudD09PW4ucGFyZW50P3QuYTplfWZ1bmN0aW9uIElwKHQsbil7dGhpcy5fPXQsdGhpcy5wYXJl
bnQ9bnVsbCx0aGlzLmNoaWxkcmVuPW51bGwsdGhpcy5BPW51bGwsdGhpcy5hPXRoaXMsdGhpcy56
PTAsdGhpcy5tPTAsdGhpcy5jPTAsdGhpcy5zPTAsdGhpcy50PW51bGwsdGhpcy5pPW59ZnVuY3Rp
b24gT3AodCxuLGUscixpKXtmb3IodmFyIG8sYT10LmNoaWxkcmVuLHU9LTEsYz1hLmxlbmd0aCxm
PXQudmFsdWUmJihpLWUpL3QudmFsdWU7Kyt1PGM7KShvPWFbdV0pLngwPW4sby54MT1yLG8ueTA9
ZSxvLnkxPWUrPW8udmFsdWUqZn1JcC5wcm90b3R5cGU9T2JqZWN0LmNyZWF0ZShKZC5wcm90b3R5
cGUpO3ZhciBCcD0oMStNYXRoLnNxcnQoNSkpLzI7ZnVuY3Rpb24gWXAodCxuLGUscixpLG8pe2Zv
cih2YXIgYSx1LGMsZixzLGwsaCxkLHAsZyx5LHY9W10sXz1uLmNoaWxkcmVuLGI9MCxtPTAseD1f
Lmxlbmd0aCx3PW4udmFsdWU7Yjx4Oyl7Yz1pLWUsZj1vLXI7ZG97cz1fW20rK10udmFsdWV9d2hp
bGUoIXMmJm08eCk7Zm9yKGw9aD1zLHk9cypzKihnPU1hdGgubWF4KGYvYyxjL2YpLyh3KnQpKSxw
PU1hdGgubWF4KGgveSx5L2wpO208eDsrK20pe2lmKHMrPXU9X1ttXS52YWx1ZSx1PGwmJihsPXUp
LHU+aCYmKGg9dSkseT1zKnMqZywoZD1NYXRoLm1heChoL3kseS9sKSk+cCl7cy09dTticmVha31w
PWR9di5wdXNoKGE9e3ZhbHVlOnMsZGljZTpjPGYsY2hpbGRyZW46Xy5zbGljZShiLG0pfSksYS5k
aWNlP1NwKGEsZSxyLGksdz9yKz1mKnMvdzpvKTpPcChhLGUscix3P2UrPWMqcy93Omksbyksdy09
cyxiPW19cmV0dXJuIHZ9dmFyIExwPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0LGUscixpLG8p
e1lwKG4sdCxlLHIsaSxvKX1yZXR1cm4gZS5yYXRpbz1mdW5jdGlvbihuKXtyZXR1cm4gdCgobj0r
bik+MT9uOjEpfSxlfShCcCk7dmFyIGpwPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0LGUscixp
LG8pe2lmKChhPXQuX3NxdWFyaWZ5KSYmYS5yYXRpbz09PW4pZm9yKHZhciBhLHUsYyxmLHMsbD0t
MSxoPWEubGVuZ3RoLGQ9dC52YWx1ZTsrK2w8aDspe2ZvcihjPSh1PWFbbF0pLmNoaWxkcmVuLGY9
dS52YWx1ZT0wLHM9Yy5sZW5ndGg7ZjxzOysrZil1LnZhbHVlKz1jW2ZdLnZhbHVlO3UuZGljZT9T
cCh1LGUscixpLGQ/cis9KG8tcikqdS52YWx1ZS9kOm8pOk9wKHUsZSxyLGQ/ZSs9KGktZSkqdS52
YWx1ZS9kOmksbyksZC09dS52YWx1ZX1lbHNlIHQuX3NxdWFyaWZ5PWE9WXAobix0LGUscixpLG8p
LGEucmF0aW89bn1yZXR1cm4gZS5yYXRpbz1mdW5jdGlvbihuKXtyZXR1cm4gdCgobj0rbik+MT9u
OjEpfSxlfShCcCk7ZnVuY3Rpb24gSHAodCxuLGUpe3JldHVybihuWzBdLXRbMF0pKihlWzFdLXRb
MV0pLShuWzFdLXRbMV0pKihlWzBdLXRbMF0pfWZ1bmN0aW9uIFhwKHQsbil7cmV0dXJuIHRbMF0t
blswXXx8dFsxXS1uWzFdfWZ1bmN0aW9uIEdwKHQpe2NvbnN0IG49dC5sZW5ndGgsZT1bMCwxXTts
ZXQgcixpPTI7Zm9yKHI9MjtyPG47KytyKXtmb3IoO2k+MSYmSHAodFtlW2ktMl1dLHRbZVtpLTFd
XSx0W3JdKTw9MDspLS1pO2VbaSsrXT1yfXJldHVybiBlLnNsaWNlKDAsaSl9dmFyIFZwPU1hdGgu
cmFuZG9tLFdwPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0LGUpe3JldHVybiB0PW51bGw9PXQ/
MDordCxlPW51bGw9PWU/MTorZSwxPT09YXJndW1lbnRzLmxlbmd0aD8oZT10LHQ9MCk6ZS09dCxm
dW5jdGlvbigpe3JldHVybiBuKCkqZSt0fX1yZXR1cm4gZS5zb3VyY2U9dCxlfShWcCksWnA9ZnVu
Y3Rpb24gdChuKXtmdW5jdGlvbiBlKHQsZSl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg8MiYmKGU9
dCx0PTApLHQ9TWF0aC5mbG9vcih0KSxlPU1hdGguZmxvb3IoZSktdCxmdW5jdGlvbigpe3JldHVy
biBNYXRoLmZsb29yKG4oKSplK3QpfX1yZXR1cm4gZS5zb3VyY2U9dCxlfShWcCksS3A9ZnVuY3Rp
b24gdChuKXtmdW5jdGlvbiBlKHQsZSl7dmFyIHIsaTtyZXR1cm4gdD1udWxsPT10PzA6K3QsZT1u
dWxsPT1lPzE6K2UsZnVuY3Rpb24oKXt2YXIgbztpZihudWxsIT1yKW89cixyPW51bGw7ZWxzZSBk
b3tyPTIqbigpLTEsbz0yKm4oKS0xLGk9cipyK28qb313aGlsZSghaXx8aT4xKTtyZXR1cm4gdCtl
Km8qTWF0aC5zcXJ0KC0yKk1hdGgubG9nKGkpL2kpfX1yZXR1cm4gZS5zb3VyY2U9dCxlfShWcCks
UXA9ZnVuY3Rpb24gdChuKXt2YXIgZT1LcC5zb3VyY2Uobik7ZnVuY3Rpb24gcigpe3ZhciB0PWUu
YXBwbHkodGhpcyxhcmd1bWVudHMpO3JldHVybiBmdW5jdGlvbigpe3JldHVybiBNYXRoLmV4cCh0
KCkpfX1yZXR1cm4gci5zb3VyY2U9dCxyfShWcCksSnA9ZnVuY3Rpb24gdChuKXtmdW5jdGlvbiBl
KHQpe3JldHVybih0PSt0KTw9MD8oKT0+MDpmdW5jdGlvbigpe2Zvcih2YXIgZT0wLHI9dDtyPjE7
LS1yKWUrPW4oKTtyZXR1cm4gZStyKm4oKX19cmV0dXJuIGUuc291cmNlPXQsZX0oVnApLHRnPWZ1
bmN0aW9uIHQobil7dmFyIGU9SnAuc291cmNlKG4pO2Z1bmN0aW9uIHIodCl7aWYoMD09KHQ9K3Qp
KXJldHVybiBuO3ZhciByPWUodCk7cmV0dXJuIGZ1bmN0aW9uKCl7cmV0dXJuIHIoKS90fX1yZXR1
cm4gci5zb3VyY2U9dCxyfShWcCksbmc9ZnVuY3Rpb24gdChuKXtmdW5jdGlvbiBlKHQpe3JldHVy
biBmdW5jdGlvbigpe3JldHVybi1NYXRoLmxvZzFwKC1uKCkpL3R9fXJldHVybiBlLnNvdXJjZT10
LGV9KFZwKSxlZz1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCl7aWYoKHQ9K3QpPDApdGhyb3cg
bmV3IFJhbmdlRXJyb3IoImludmFsaWQgYWxwaGEiKTtyZXR1cm4gdD0xLy10LGZ1bmN0aW9uKCl7
cmV0dXJuIE1hdGgucG93KDEtbigpLHQpfX1yZXR1cm4gZS5zb3VyY2U9dCxlfShWcCkscmc9ZnVu
Y3Rpb24gdChuKXtmdW5jdGlvbiBlKHQpe2lmKCh0PSt0KTwwfHx0PjEpdGhyb3cgbmV3IFJhbmdl
RXJyb3IoImludmFsaWQgcCIpO3JldHVybiBmdW5jdGlvbigpe3JldHVybiBNYXRoLmZsb29yKG4o
KSt0KX19cmV0dXJuIGUuc291cmNlPXQsZX0oVnApLGlnPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24g
ZSh0KXtpZigodD0rdCk8MHx8dD4xKXRocm93IG5ldyBSYW5nZUVycm9yKCJpbnZhbGlkIHAiKTty
ZXR1cm4gMD09PXQ/KCk9PjEvMDoxPT09dD8oKT0+MToodD1NYXRoLmxvZzFwKC10KSxmdW5jdGlv
bigpe3JldHVybiAxK01hdGguZmxvb3IoTWF0aC5sb2cxcCgtbigpKS90KX0pfXJldHVybiBlLnNv
dXJjZT10LGV9KFZwKSxvZz1mdW5jdGlvbiB0KG4pe3ZhciBlPUtwLnNvdXJjZShuKSgpO2Z1bmN0
aW9uIHIodCxyKXtpZigodD0rdCk8MCl0aHJvdyBuZXcgUmFuZ2VFcnJvcigiaW52YWxpZCBrIik7
aWYoMD09PXQpcmV0dXJuKCk9PjA7aWYocj1udWxsPT1yPzE6K3IsMT09PXQpcmV0dXJuKCk9Pi1N
YXRoLmxvZzFwKC1uKCkpKnI7dmFyIGk9KHQ8MT90KzE6dCktMS8zLG89MS8oMypNYXRoLnNxcnQo
aSkpLGE9dDwxPygpPT5NYXRoLnBvdyhuKCksMS90KTooKT0+MTtyZXR1cm4gZnVuY3Rpb24oKXtk
b3tkb3t2YXIgdD1lKCksdT0xK28qdH13aGlsZSh1PD0wKTt1Kj11KnU7dmFyIGM9MS1uKCl9d2hp
bGUoYz49MS0uMDMzMSp0KnQqdCp0JiZNYXRoLmxvZyhjKT49LjUqdCp0K2kqKDEtdStNYXRoLmxv
Zyh1KSkpO3JldHVybiBpKnUqYSgpKnJ9fXJldHVybiByLnNvdXJjZT10LHJ9KFZwKSxhZz1mdW5j
dGlvbiB0KG4pe3ZhciBlPW9nLnNvdXJjZShuKTtmdW5jdGlvbiByKHQsbil7dmFyIHI9ZSh0KSxp
PWUobik7cmV0dXJuIGZ1bmN0aW9uKCl7dmFyIHQ9cigpO3JldHVybiAwPT09dD8wOnQvKHQraSgp
KX19cmV0dXJuIHIuc291cmNlPXQscn0oVnApLHVnPWZ1bmN0aW9uIHQobil7dmFyIGU9aWcuc291
cmNlKG4pLHI9YWcuc291cmNlKG4pO2Z1bmN0aW9uIGkodCxuKXtyZXR1cm4gdD0rdCwobj0rbik+
PTE/KCk9PnQ6bjw9MD8oKT0+MDpmdW5jdGlvbigpe2Zvcih2YXIgaT0wLG89dCxhPW47byphPjE2
JiZvKigxLWEpPjE2Oyl7dmFyIHU9TWF0aC5mbG9vcigobysxKSphKSxjPXIodSxvLXUrMSkoKTtj
PD1hPyhpKz11LG8tPXUsYT0oYS1jKS8oMS1jKSk6KG89dS0xLGEvPWMpfWZvcih2YXIgZj1hPC41
LHM9ZShmP2E6MS1hKSxsPXMoKSxoPTA7bDw9bzsrK2gpbCs9cygpO3JldHVybiBpKyhmP2g6by1o
KX19cmV0dXJuIGkuc291cmNlPXQsaX0oVnApLGNnPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0
LGUscil7dmFyIGk7cmV0dXJuIDA9PSh0PSt0KT9pPXQ9Pi1NYXRoLmxvZyh0KToodD0xL3QsaT1u
PT5NYXRoLnBvdyhuLHQpKSxlPW51bGw9PWU/MDorZSxyPW51bGw9PXI/MTorcixmdW5jdGlvbigp
e3JldHVybiBlK3IqaSgtTWF0aC5sb2cxcCgtbigpKSl9fXJldHVybiBlLnNvdXJjZT10LGV9KFZw
KSxmZz1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCxlKXtyZXR1cm4gdD1udWxsPT10PzA6K3Qs
ZT1udWxsPT1lPzE6K2UsZnVuY3Rpb24oKXtyZXR1cm4gdCtlKk1hdGgudGFuKE1hdGguUEkqbigp
KX19cmV0dXJuIGUuc291cmNlPXQsZX0oVnApLHNnPWZ1bmN0aW9uIHQobil7ZnVuY3Rpb24gZSh0
LGUpe3JldHVybiB0PW51bGw9PXQ/MDordCxlPW51bGw9PWU/MTorZSxmdW5jdGlvbigpe3ZhciBy
PW4oKTtyZXR1cm4gdCtlKk1hdGgubG9nKHIvKDEtcikpfX1yZXR1cm4gZS5zb3VyY2U9dCxlfShW
cCksbGc9ZnVuY3Rpb24gdChuKXt2YXIgZT1vZy5zb3VyY2Uobikscj11Zy5zb3VyY2Uobik7ZnVu
Y3Rpb24gaSh0KXtyZXR1cm4gZnVuY3Rpb24oKXtmb3IodmFyIGk9MCxvPXQ7bz4xNjspe3ZhciBh
PU1hdGguZmxvb3IoLjg3NSpvKSx1PWUoYSkoKTtpZih1Pm8pcmV0dXJuIGkrcihhLTEsby91KSgp
O2krPWEsby09dX1mb3IodmFyIGM9LU1hdGgubG9nMXAoLW4oKSksZj0wO2M8PW87KytmKWMtPU1h
dGgubG9nMXAoLW4oKSk7cmV0dXJuIGkrZn19cmV0dXJuIGkuc291cmNlPXQsaX0oVnApO2NvbnN0
IGhnPTEvNDI5NDk2NzI5NjtmdW5jdGlvbiBkZyh0LG4pe3N3aXRjaChhcmd1bWVudHMubGVuZ3Ro
KXtjYXNlIDA6YnJlYWs7Y2FzZSAxOnRoaXMucmFuZ2UodCk7YnJlYWs7ZGVmYXVsdDp0aGlzLnJh
bmdlKG4pLmRvbWFpbih0KX1yZXR1cm4gdGhpc31mdW5jdGlvbiBwZyh0LG4pe3N3aXRjaChhcmd1
bWVudHMubGVuZ3RoKXtjYXNlIDA6YnJlYWs7Y2FzZSAxOiJmdW5jdGlvbiI9PXR5cGVvZiB0P3Ro
aXMuaW50ZXJwb2xhdG9yKHQpOnRoaXMucmFuZ2UodCk7YnJlYWs7ZGVmYXVsdDp0aGlzLmRvbWFp
bih0KSwiZnVuY3Rpb24iPT10eXBlb2Ygbj90aGlzLmludGVycG9sYXRvcihuKTp0aGlzLnJhbmdl
KG4pfXJldHVybiB0aGlzfWNvbnN0IGdnPVN5bWJvbCgiaW1wbGljaXQiKTtmdW5jdGlvbiB5Zygp
e3ZhciB0PW5ldyBJbnRlcm5NYXAsbj1bXSxlPVtdLHI9Z2c7ZnVuY3Rpb24gaShpKXtsZXQgbz10
LmdldChpKTtpZih2b2lkIDA9PT1vKXtpZihyIT09Z2cpcmV0dXJuIHI7dC5zZXQoaSxvPW4ucHVz
aChpKS0xKX1yZXR1cm4gZVtvJWUubGVuZ3RoXX1yZXR1cm4gaS5kb21haW49ZnVuY3Rpb24oZSl7
aWYoIWFyZ3VtZW50cy5sZW5ndGgpcmV0dXJuIG4uc2xpY2UoKTtuPVtdLHQ9bmV3IEludGVybk1h
cDtmb3IoY29uc3QgciBvZiBlKXQuaGFzKHIpfHx0LnNldChyLG4ucHVzaChyKS0xKTtyZXR1cm4g
aX0saS5yYW5nZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oZT1BcnJheS5m
cm9tKHQpLGkpOmUuc2xpY2UoKX0saS51bmtub3duPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVu
dHMubGVuZ3RoPyhyPXQsaSk6cn0saS5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIHlnKG4sZSkudW5r
bm93bihyKX0sZGcuYXBwbHkoaSxhcmd1bWVudHMpLGl9ZnVuY3Rpb24gdmcoKXt2YXIgdCxuLGU9
eWcoKS51bmtub3duKHZvaWQgMCkscj1lLmRvbWFpbixpPWUucmFuZ2Usbz0wLGE9MSx1PSExLGM9
MCxmPTAscz0uNTtmdW5jdGlvbiBsKCl7dmFyIGU9cigpLmxlbmd0aCxsPWE8byxoPWw/YTpvLGQ9
bD9vOmE7dD0oZC1oKS9NYXRoLm1heCgxLGUtYysyKmYpLHUmJih0PU1hdGguZmxvb3IodCkpLGgr
PShkLWgtdCooZS1jKSkqcyxuPXQqKDEtYyksdSYmKGg9TWF0aC5yb3VuZChoKSxuPU1hdGgucm91
bmQobikpO3ZhciBwPWh0KGUpLm1hcCgoZnVuY3Rpb24obil7cmV0dXJuIGgrdCpufSkpO3JldHVy
biBpKGw/cC5yZXZlcnNlKCk6cCl9cmV0dXJuIGRlbGV0ZSBlLnVua25vd24sZS5kb21haW49ZnVu
Y3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHIodCksbCgpKTpyKCl9LGUucmFuZ2U9
ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KFtvLGFdPXQsbz0rbyxhPSthLGwo
KSk6W28sYV19LGUucmFuZ2VSb3VuZD1mdW5jdGlvbih0KXtyZXR1cm5bbyxhXT10LG89K28sYT0r
YSx1PSEwLGwoKX0sZS5iYW5kd2lkdGg9ZnVuY3Rpb24oKXtyZXR1cm4gbn0sZS5zdGVwPWZ1bmN0
aW9uKCl7cmV0dXJuIHR9LGUucm91bmQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5n
dGg/KHU9ISF0LGwoKSk6dX0sZS5wYWRkaW5nPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPyhjPU1hdGgubWluKDEsZj0rdCksbCgpKTpjfSxlLnBhZGRpbmdJbm5lcj1mdW5jdGlv
bih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYz1NYXRoLm1pbigxLHQpLGwoKSk6Y30sZS5w
YWRkaW5nT3V0ZXI9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGY9K3QsbCgp
KTpmfSxlLmFsaWduPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhzPU1hdGgu
bWF4KDAsTWF0aC5taW4oMSx0KSksbCgpKTpzfSxlLmNvcHk9ZnVuY3Rpb24oKXtyZXR1cm4gdmco
cigpLFtvLGFdKS5yb3VuZCh1KS5wYWRkaW5nSW5uZXIoYykucGFkZGluZ091dGVyKGYpLmFsaWdu
KHMpfSxkZy5hcHBseShsKCksYXJndW1lbnRzKX1mdW5jdGlvbiBfZyh0KXt2YXIgbj10LmNvcHk7
cmV0dXJuIHQucGFkZGluZz10LnBhZGRpbmdPdXRlcixkZWxldGUgdC5wYWRkaW5nSW5uZXIsZGVs
ZXRlIHQucGFkZGluZ091dGVyLHQuY29weT1mdW5jdGlvbigpe3JldHVybiBfZyhuKCkpfSx0fWZ1
bmN0aW9uIGJnKHQpe3JldHVybit0fXZhciBtZz1bMCwxXTtmdW5jdGlvbiB4Zyh0KXtyZXR1cm4g
dH1mdW5jdGlvbiB3Zyh0LG4pe3JldHVybihuLT10PSt0KT9mdW5jdGlvbihlKXtyZXR1cm4oZS10
KS9ufTpmdW5jdGlvbih0KXtyZXR1cm4gZnVuY3Rpb24oKXtyZXR1cm4gdH19KGlzTmFOKG4pP05h
TjouNSl9ZnVuY3Rpb24gTWcodCxuLGUpe3ZhciByPXRbMF0saT10WzFdLG89blswXSxhPW5bMV07
cmV0dXJuIGk8cj8ocj13ZyhpLHIpLG89ZShhLG8pKToocj13ZyhyLGkpLG89ZShvLGEpKSxmdW5j
dGlvbih0KXtyZXR1cm4gbyhyKHQpKX19ZnVuY3Rpb24gVGcodCxuLGUpe3ZhciByPU1hdGgubWlu
KHQubGVuZ3RoLG4ubGVuZ3RoKS0xLGk9bmV3IEFycmF5KHIpLG89bmV3IEFycmF5KHIpLGE9LTE7
Zm9yKHRbcl08dFswXSYmKHQ9dC5zbGljZSgpLnJldmVyc2UoKSxuPW4uc2xpY2UoKS5yZXZlcnNl
KCkpOysrYTxyOylpW2FdPXdnKHRbYV0sdFthKzFdKSxvW2FdPWUoblthXSxuW2ErMV0pO3JldHVy
biBmdW5jdGlvbihuKXt2YXIgZT1sKHQsbiwxLHIpLTE7cmV0dXJuIG9bZV0oaVtlXShuKSl9fWZ1
bmN0aW9uIEFnKHQsbil7cmV0dXJuIG4uZG9tYWluKHQuZG9tYWluKCkpLnJhbmdlKHQucmFuZ2Uo
KSkuaW50ZXJwb2xhdGUodC5pbnRlcnBvbGF0ZSgpKS5jbGFtcCh0LmNsYW1wKCkpLnVua25vd24o
dC51bmtub3duKCkpfWZ1bmN0aW9uIFNnKCl7dmFyIHQsbixlLHIsaSxvLGE9bWcsdT1tZyxjPVZy
LGY9eGc7ZnVuY3Rpb24gcygpe3ZhciB0PU1hdGgubWluKGEubGVuZ3RoLHUubGVuZ3RoKTtyZXR1
cm4gZiE9PXhnJiYoZj1mdW5jdGlvbih0LG4pe3ZhciBlO3JldHVybiB0Pm4mJihlPXQsdD1uLG49
ZSksZnVuY3Rpb24oZSl7cmV0dXJuIE1hdGgubWF4KHQsTWF0aC5taW4obixlKSl9fShhWzBdLGFb
dC0xXSkpLHI9dD4yP1RnOk1nLGk9bz1udWxsLGx9ZnVuY3Rpb24gbChuKXtyZXR1cm4gbnVsbD09
bnx8aXNOYU4obj0rbik/ZTooaXx8KGk9cihhLm1hcCh0KSx1LGMpKSkodChmKG4pKSl9cmV0dXJu
IGwuaW52ZXJ0PWZ1bmN0aW9uKGUpe3JldHVybiBmKG4oKG98fChvPXIodSxhLm1hcCh0KSxMcikp
KShlKSkpfSxsLmRvbWFpbj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYT1B
cnJheS5mcm9tKHQsYmcpLHMoKSk6YS5zbGljZSgpfSxsLnJhbmdlPWZ1bmN0aW9uKHQpe3JldHVy
biBhcmd1bWVudHMubGVuZ3RoPyh1PUFycmF5LmZyb20odCkscygpKTp1LnNsaWNlKCl9LGwucmFu
Z2VSb3VuZD1mdW5jdGlvbih0KXtyZXR1cm4gdT1BcnJheS5mcm9tKHQpLGM9V3IscygpfSxsLmNs
YW1wPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhmPSEhdHx8eGcscygpKTpm
IT09eGd9LGwuaW50ZXJwb2xhdGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KGM9dCxzKCkpOmN9LGwudW5rbm93bj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0
aD8oZT10LGwpOmV9LGZ1bmN0aW9uKGUscil7cmV0dXJuIHQ9ZSxuPXIscygpfX1mdW5jdGlvbiBF
Zygpe3JldHVybiBTZygpKHhnLHhnKX1mdW5jdGlvbiBOZyhuLGUscixpKXt2YXIgbyxhPVoobixl
LHIpO3N3aXRjaCgoaT10ZihudWxsPT1pPyIsZiI6aSkpLnR5cGUpe2Nhc2UicyI6dmFyIHU9TWF0
aC5tYXgoTWF0aC5hYnMobiksTWF0aC5hYnMoZSkpO3JldHVybiBudWxsIT1pLnByZWNpc2lvbnx8
aXNOYU4obz1oZihhLHUpKXx8KGkucHJlY2lzaW9uPW8pLHQuZm9ybWF0UHJlZml4KGksdSk7Y2Fz
ZSIiOmNhc2UiZSI6Y2FzZSJnIjpjYXNlInAiOmNhc2UiciI6bnVsbCE9aS5wcmVjaXNpb258fGlz
TmFOKG89ZGYoYSxNYXRoLm1heChNYXRoLmFicyhuKSxNYXRoLmFicyhlKSkpKXx8KGkucHJlY2lz
aW9uPW8tKCJlIj09PWkudHlwZSkpO2JyZWFrO2Nhc2UiZiI6Y2FzZSIlIjpudWxsIT1pLnByZWNp
c2lvbnx8aXNOYU4obz1sZihhKSl8fChpLnByZWNpc2lvbj1vLTIqKCIlIj09PWkudHlwZSkpfXJl
dHVybiB0LmZvcm1hdChpKX1mdW5jdGlvbiBrZyh0KXt2YXIgbj10LmRvbWFpbjtyZXR1cm4gdC50
aWNrcz1mdW5jdGlvbih0KXt2YXIgZT1uKCk7cmV0dXJuIFYoZVswXSxlW2UubGVuZ3RoLTFdLG51
bGw9PXQ/MTA6dCl9LHQudGlja0Zvcm1hdD1mdW5jdGlvbih0LGUpe3ZhciByPW4oKTtyZXR1cm4g
TmcoclswXSxyW3IubGVuZ3RoLTFdLG51bGw9PXQ/MTA6dCxlKX0sdC5uaWNlPWZ1bmN0aW9uKGUp
e251bGw9PWUmJihlPTEwKTt2YXIgcixpLG89bigpLGE9MCx1PW8ubGVuZ3RoLTEsYz1vW2FdLGY9
b1t1XSxzPTEwO2ZvcihmPGMmJihpPWMsYz1mLGY9aSxpPWEsYT11LHU9aSk7cy0tID4wOyl7aWYo
KGk9VyhjLGYsZSkpPT09cilyZXR1cm4gb1thXT1jLG9bdV09ZixuKG8pO2lmKGk+MCljPU1hdGgu
Zmxvb3IoYy9pKSppLGY9TWF0aC5jZWlsKGYvaSkqaTtlbHNle2lmKCEoaTwwKSlicmVhaztjPU1h
dGguY2VpbChjKmkpL2ksZj1NYXRoLmZsb29yKGYqaSkvaX1yPWl9cmV0dXJuIHR9LHR9ZnVuY3Rp
b24gQ2codCxuKXt2YXIgZSxyPTAsaT0odD10LnNsaWNlKCkpLmxlbmd0aC0xLG89dFtyXSxhPXRb
aV07cmV0dXJuIGE8byYmKGU9cixyPWksaT1lLGU9byxvPWEsYT1lKSx0W3JdPW4uZmxvb3Iobyks
dFtpXT1uLmNlaWwoYSksdH1mdW5jdGlvbiBQZyh0KXtyZXR1cm4gTWF0aC5sb2codCl9ZnVuY3Rp
b24gemcodCl7cmV0dXJuIE1hdGguZXhwKHQpfWZ1bmN0aW9uICRnKHQpe3JldHVybi1NYXRoLmxv
ZygtdCl9ZnVuY3Rpb24gRGcodCl7cmV0dXJuLU1hdGguZXhwKC10KX1mdW5jdGlvbiBSZyh0KXty
ZXR1cm4gaXNGaW5pdGUodCk/KygiMWUiK3QpOnQ8MD8wOnR9ZnVuY3Rpb24gRmcodCl7cmV0dXJu
KG4sZSk9Pi10KC1uLGUpfWZ1bmN0aW9uIHFnKG4pe2NvbnN0IGU9bihQZyx6Zykscj1lLmRvbWFp
bjtsZXQgaSxvLGE9MTA7ZnVuY3Rpb24gdSgpe3JldHVybiBpPWZ1bmN0aW9uKHQpe3JldHVybiB0
PT09TWF0aC5FP01hdGgubG9nOjEwPT09dCYmTWF0aC5sb2cxMHx8Mj09PXQmJk1hdGgubG9nMnx8
KHQ9TWF0aC5sb2codCksbj0+TWF0aC5sb2cobikvdCl9KGEpLG89ZnVuY3Rpb24odCl7cmV0dXJu
IDEwPT09dD9SZzp0PT09TWF0aC5FP01hdGguZXhwOm49Pk1hdGgucG93KHQsbil9KGEpLHIoKVsw
XTwwPyhpPUZnKGkpLG89RmcobyksbigkZyxEZykpOm4oUGcsemcpLGV9cmV0dXJuIGUuYmFzZT1m
dW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYT0rdCx1KCkpOmF9LGUuZG9tYWlu
PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhyKHQpLHUoKSk6cigpfSxlLnRp
Y2tzPXQ9Pntjb25zdCBuPXIoKTtsZXQgZT1uWzBdLHU9bltuLmxlbmd0aC0xXTtjb25zdCBjPXU8
ZTtjJiYoW2UsdV09W3UsZV0pO2xldCBmLHMsbD1pKGUpLGg9aSh1KTtjb25zdCBkPW51bGw9PXQ/
MTA6K3Q7bGV0IHA9W107aWYoIShhJTEpJiZoLWw8ZCl7aWYobD1NYXRoLmZsb29yKGwpLGg9TWF0
aC5jZWlsKGgpLGU+MCl7Zm9yKDtsPD1oOysrbClmb3IoZj0xO2Y8YTsrK2YpaWYocz1sPDA/Zi9v
KC1sKTpmKm8obCksIShzPGUpKXtpZihzPnUpYnJlYWs7cC5wdXNoKHMpfX1lbHNlIGZvcig7bDw9
aDsrK2wpZm9yKGY9YS0xO2Y+PTE7LS1mKWlmKHM9bD4wP2YvbygtbCk6ZipvKGwpLCEoczxlKSl7
aWYocz51KWJyZWFrO3AucHVzaChzKX0yKnAubGVuZ3RoPGQmJihwPVYoZSx1LGQpKX1lbHNlIHA9
VihsLGgsTWF0aC5taW4oaC1sLGQpKS5tYXAobyk7cmV0dXJuIGM/cC5yZXZlcnNlKCk6cH0sZS50
aWNrRm9ybWF0PShuLHIpPT57aWYobnVsbD09biYmKG49MTApLG51bGw9PXImJihyPTEwPT09YT8i
cyI6IiwiKSwiZnVuY3Rpb24iIT10eXBlb2YgciYmKGElMXx8bnVsbCE9KHI9dGYocikpLnByZWNp
c2lvbnx8KHIudHJpbT0hMCkscj10LmZvcm1hdChyKSksbj09PTEvMClyZXR1cm4gcjtjb25zdCB1
PU1hdGgubWF4KDEsYSpuL2UudGlja3MoKS5sZW5ndGgpO3JldHVybiB0PT57bGV0IG49dC9vKE1h
dGgucm91bmQoaSh0KSkpO3JldHVybiBuKmE8YS0uNSYmKG4qPWEpLG48PXU/cih0KToiIn19LGUu
bmljZT0oKT0+cihDZyhyKCkse2Zsb29yOnQ9Pm8oTWF0aC5mbG9vcihpKHQpKSksY2VpbDp0PT5v
KE1hdGguY2VpbChpKHQpKSl9KSksZX1mdW5jdGlvbiBVZyh0KXtyZXR1cm4gZnVuY3Rpb24obil7
cmV0dXJuIE1hdGguc2lnbihuKSpNYXRoLmxvZzFwKE1hdGguYWJzKG4vdCkpfX1mdW5jdGlvbiBJ
Zyh0KXtyZXR1cm4gZnVuY3Rpb24obil7cmV0dXJuIE1hdGguc2lnbihuKSpNYXRoLmV4cG0xKE1h
dGguYWJzKG4pKSp0fX1mdW5jdGlvbiBPZyh0KXt2YXIgbj0xLGU9dChVZyhuKSxJZyhuKSk7cmV0
dXJuIGUuY29uc3RhbnQ9ZnVuY3Rpb24oZSl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/dChVZyhu
PStlKSxJZyhuKSk6bn0sa2coZSl9ZnVuY3Rpb24gQmcodCl7cmV0dXJuIGZ1bmN0aW9uKG4pe3Jl
dHVybiBuPDA/LU1hdGgucG93KC1uLHQpOk1hdGgucG93KG4sdCl9fWZ1bmN0aW9uIFlnKHQpe3Jl
dHVybiB0PDA/LU1hdGguc3FydCgtdCk6TWF0aC5zcXJ0KHQpfWZ1bmN0aW9uIExnKHQpe3JldHVy
biB0PDA/LXQqdDp0KnR9ZnVuY3Rpb24gamcodCl7dmFyIG49dCh4Zyx4ZyksZT0xO3JldHVybiBu
LmV4cG9uZW50PWZ1bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPzE9PT0oZT0rbik/
dCh4Zyx4Zyk6LjU9PT1lP3QoWWcsTGcpOnQoQmcoZSksQmcoMS9lKSk6ZX0sa2cobil9ZnVuY3Rp
b24gSGcoKXt2YXIgdD1qZyhTZygpKTtyZXR1cm4gdC5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIEFn
KHQsSGcoKSkuZXhwb25lbnQodC5leHBvbmVudCgpKX0sZGcuYXBwbHkodCxhcmd1bWVudHMpLHR9
ZnVuY3Rpb24gWGcodCl7cmV0dXJuIE1hdGguc2lnbih0KSp0KnR9Y29uc3QgR2c9bmV3IERhdGUs
Vmc9bmV3IERhdGU7ZnVuY3Rpb24gV2codCxuLGUscil7ZnVuY3Rpb24gaShuKXtyZXR1cm4gdChu
PTA9PT1hcmd1bWVudHMubGVuZ3RoP25ldyBEYXRlOm5ldyBEYXRlKCtuKSksbn1yZXR1cm4gaS5m
bG9vcj1uPT4odChuPW5ldyBEYXRlKCtuKSksbiksaS5jZWlsPWU9Pih0KGU9bmV3IERhdGUoZS0x
KSksbihlLDEpLHQoZSksZSksaS5yb3VuZD10PT57Y29uc3Qgbj1pKHQpLGU9aS5jZWlsKHQpO3Jl
dHVybiB0LW48ZS10P246ZX0saS5vZmZzZXQ9KHQsZSk9PihuKHQ9bmV3IERhdGUoK3QpLG51bGw9
PWU/MTpNYXRoLmZsb29yKGUpKSx0KSxpLnJhbmdlPShlLHIsbyk9Pntjb25zdCBhPVtdO2lmKGU9
aS5jZWlsKGUpLG89bnVsbD09bz8xOk1hdGguZmxvb3IobyksIShlPHImJm8+MCkpcmV0dXJuIGE7
bGV0IHU7ZG97YS5wdXNoKHU9bmV3IERhdGUoK2UpKSxuKGUsbyksdChlKX13aGlsZSh1PGUmJmU8
cik7cmV0dXJuIGF9LGkuZmlsdGVyPWU9PldnKChuPT57aWYobj49bilmb3IoO3QobiksIWUobik7
KW4uc2V0VGltZShuLTEpfSksKCh0LHIpPT57aWYodD49dClpZihyPDApZm9yKDsrK3I8PTA7KWZv
cig7bih0LC0xKSwhZSh0KTspO2Vsc2UgZm9yKDstLXI+PTA7KWZvcig7bih0LDEpLCFlKHQpOyk7
fSkpLGUmJihpLmNvdW50PShuLHIpPT4oR2cuc2V0VGltZSgrbiksVmcuc2V0VGltZSgrciksdChH
ZyksdChWZyksTWF0aC5mbG9vcihlKEdnLFZnKSkpLGkuZXZlcnk9dD0+KHQ9TWF0aC5mbG9vcih0
KSxpc0Zpbml0ZSh0KSYmdD4wP3Q+MT9pLmZpbHRlcihyP249PnIobikldD09MDpuPT5pLmNvdW50
KDAsbikldD09MCk6aTpudWxsKSksaX1jb25zdCBaZz1XZygoKCk9Pnt9KSwoKHQsbik9Pnt0LnNl
dFRpbWUoK3Qrbil9KSwoKHQsbik9Pm4tdCkpO1pnLmV2ZXJ5PXQ9Pih0PU1hdGguZmxvb3IodCks
aXNGaW5pdGUodCkmJnQ+MD90PjE/V2coKG49PntuLnNldFRpbWUoTWF0aC5mbG9vcihuL3QpKnQp
fSksKChuLGUpPT57bi5zZXRUaW1lKCtuK2UqdCl9KSwoKG4sZSk9PihlLW4pL3QpKTpaZzpudWxs
KTtjb25zdCBLZz1aZy5yYW5nZSxRZz0xZTMsSmc9NmU0LHR5PTYwKkpnLG55PTI0KnR5LGV5PTcq
bnkscnk9MzAqbnksaXk9MzY1Km55LG95PVdnKCh0PT57dC5zZXRUaW1lKHQtdC5nZXRNaWxsaXNl
Y29uZHMoKSl9KSwoKHQsbik9Pnt0LnNldFRpbWUoK3QrbipRZyl9KSwoKHQsbik9PihuLXQpL1Fn
KSwodD0+dC5nZXRVVENTZWNvbmRzKCkpKSxheT1veS5yYW5nZSx1eT1XZygodD0+e3Quc2V0VGlt
ZSh0LXQuZ2V0TWlsbGlzZWNvbmRzKCktdC5nZXRTZWNvbmRzKCkqUWcpfSksKCh0LG4pPT57dC5z
ZXRUaW1lKCt0K24qSmcpfSksKCh0LG4pPT4obi10KS9KZyksKHQ9PnQuZ2V0TWludXRlcygpKSks
Y3k9dXkucmFuZ2UsZnk9V2coKHQ9Pnt0LnNldFVUQ1NlY29uZHMoMCwwKX0pLCgodCxuKT0+e3Qu
c2V0VGltZSgrdCtuKkpnKX0pLCgodCxuKT0+KG4tdCkvSmcpLCh0PT50LmdldFVUQ01pbnV0ZXMo
KSkpLHN5PWZ5LnJhbmdlLGx5PVdnKCh0PT57dC5zZXRUaW1lKHQtdC5nZXRNaWxsaXNlY29uZHMo
KS10LmdldFNlY29uZHMoKSpRZy10LmdldE1pbnV0ZXMoKSpKZyl9KSwoKHQsbik9Pnt0LnNldFRp
bWUoK3Qrbip0eSl9KSwoKHQsbik9PihuLXQpL3R5KSwodD0+dC5nZXRIb3VycygpKSksaHk9bHku
cmFuZ2UsZHk9V2coKHQ9Pnt0LnNldFVUQ01pbnV0ZXMoMCwwLDApfSksKCh0LG4pPT57dC5zZXRU
aW1lKCt0K24qdHkpfSksKCh0LG4pPT4obi10KS90eSksKHQ9PnQuZ2V0VVRDSG91cnMoKSkpLHB5
PWR5LnJhbmdlLGd5PVdnKCh0PT50LnNldEhvdXJzKDAsMCwwLDApKSwoKHQsbik9PnQuc2V0RGF0
ZSh0LmdldERhdGUoKStuKSksKCh0LG4pPT4obi10LShuLmdldFRpbWV6b25lT2Zmc2V0KCktdC5n
ZXRUaW1lem9uZU9mZnNldCgpKSpKZykvbnkpLCh0PT50LmdldERhdGUoKS0xKSkseXk9Z3kucmFu
Z2Usdnk9V2coKHQ9Pnt0LnNldFVUQ0hvdXJzKDAsMCwwLDApfSksKCh0LG4pPT57dC5zZXRVVENE
YXRlKHQuZ2V0VVRDRGF0ZSgpK24pfSksKCh0LG4pPT4obi10KS9ueSksKHQ9PnQuZ2V0VVRDRGF0
ZSgpLTEpKSxfeT12eS5yYW5nZSxieT1XZygodD0+e3Quc2V0VVRDSG91cnMoMCwwLDAsMCl9KSwo
KHQsbik9Pnt0LnNldFVUQ0RhdGUodC5nZXRVVENEYXRlKCkrbil9KSwoKHQsbik9PihuLXQpL255
KSwodD0+TWF0aC5mbG9vcih0L255KSkpLG15PWJ5LnJhbmdlO2Z1bmN0aW9uIHh5KHQpe3JldHVy
biBXZygobj0+e24uc2V0RGF0ZShuLmdldERhdGUoKS0obi5nZXREYXkoKSs3LXQpJTcpLG4uc2V0
SG91cnMoMCwwLDAsMCl9KSwoKHQsbik9Pnt0LnNldERhdGUodC5nZXREYXRlKCkrNypuKX0pLCgo
dCxuKT0+KG4tdC0obi5nZXRUaW1lem9uZU9mZnNldCgpLXQuZ2V0VGltZXpvbmVPZmZzZXQoKSkq
SmcpL2V5KSl9Y29uc3Qgd3k9eHkoMCksTXk9eHkoMSksVHk9eHkoMiksQXk9eHkoMyksU3k9eHko
NCksRXk9eHkoNSksTnk9eHkoNiksa3k9d3kucmFuZ2UsQ3k9TXkucmFuZ2UsUHk9VHkucmFuZ2Us
enk9QXkucmFuZ2UsJHk9U3kucmFuZ2UsRHk9RXkucmFuZ2UsUnk9TnkucmFuZ2U7ZnVuY3Rpb24g
RnkodCl7cmV0dXJuIFdnKChuPT57bi5zZXRVVENEYXRlKG4uZ2V0VVRDRGF0ZSgpLShuLmdldFVU
Q0RheSgpKzctdCklNyksbi5zZXRVVENIb3VycygwLDAsMCwwKX0pLCgodCxuKT0+e3Quc2V0VVRD
RGF0ZSh0LmdldFVUQ0RhdGUoKSs3Km4pfSksKCh0LG4pPT4obi10KS9leSkpfWNvbnN0IHF5PUZ5
KDApLFV5PUZ5KDEpLEl5PUZ5KDIpLE95PUZ5KDMpLEJ5PUZ5KDQpLFl5PUZ5KDUpLEx5PUZ5KDYp
LGp5PXF5LnJhbmdlLEh5PVV5LnJhbmdlLFh5PUl5LnJhbmdlLEd5PU95LnJhbmdlLFZ5PUJ5LnJh
bmdlLFd5PVl5LnJhbmdlLFp5PUx5LnJhbmdlLEt5PVdnKCh0PT57dC5zZXREYXRlKDEpLHQuc2V0
SG91cnMoMCwwLDAsMCl9KSwoKHQsbik9Pnt0LnNldE1vbnRoKHQuZ2V0TW9udGgoKStuKX0pLCgo
dCxuKT0+bi5nZXRNb250aCgpLXQuZ2V0TW9udGgoKSsxMioobi5nZXRGdWxsWWVhcigpLXQuZ2V0
RnVsbFllYXIoKSkpLCh0PT50LmdldE1vbnRoKCkpKSxReT1LeS5yYW5nZSxKeT1XZygodD0+e3Qu
c2V0VVRDRGF0ZSgxKSx0LnNldFVUQ0hvdXJzKDAsMCwwLDApfSksKCh0LG4pPT57dC5zZXRVVENN
b250aCh0LmdldFVUQ01vbnRoKCkrbil9KSwoKHQsbik9Pm4uZ2V0VVRDTW9udGgoKS10LmdldFVU
Q01vbnRoKCkrMTIqKG4uZ2V0VVRDRnVsbFllYXIoKS10LmdldFVUQ0Z1bGxZZWFyKCkpKSwodD0+
dC5nZXRVVENNb250aCgpKSksdHY9SnkucmFuZ2UsbnY9V2coKHQ9Pnt0LnNldE1vbnRoKDAsMSks
dC5zZXRIb3VycygwLDAsMCwwKX0pLCgodCxuKT0+e3Quc2V0RnVsbFllYXIodC5nZXRGdWxsWWVh
cigpK24pfSksKCh0LG4pPT5uLmdldEZ1bGxZZWFyKCktdC5nZXRGdWxsWWVhcigpKSwodD0+dC5n
ZXRGdWxsWWVhcigpKSk7bnYuZXZlcnk9dD0+aXNGaW5pdGUodD1NYXRoLmZsb29yKHQpKSYmdD4w
P1dnKChuPT57bi5zZXRGdWxsWWVhcihNYXRoLmZsb29yKG4uZ2V0RnVsbFllYXIoKS90KSp0KSxu
LnNldE1vbnRoKDAsMSksbi5zZXRIb3VycygwLDAsMCwwKX0pLCgobixlKT0+e24uc2V0RnVsbFll
YXIobi5nZXRGdWxsWWVhcigpK2UqdCl9KSk6bnVsbDtjb25zdCBldj1udi5yYW5nZSxydj1XZygo
dD0+e3Quc2V0VVRDTW9udGgoMCwxKSx0LnNldFVUQ0hvdXJzKDAsMCwwLDApfSksKCh0LG4pPT57
dC5zZXRVVENGdWxsWWVhcih0LmdldFVUQ0Z1bGxZZWFyKCkrbil9KSwoKHQsbik9Pm4uZ2V0VVRD
RnVsbFllYXIoKS10LmdldFVUQ0Z1bGxZZWFyKCkpLCh0PT50LmdldFVUQ0Z1bGxZZWFyKCkpKTty
di5ldmVyeT10PT5pc0Zpbml0ZSh0PU1hdGguZmxvb3IodCkpJiZ0PjA/V2coKG49PntuLnNldFVU
Q0Z1bGxZZWFyKE1hdGguZmxvb3Iobi5nZXRVVENGdWxsWWVhcigpL3QpKnQpLG4uc2V0VVRDTW9u
dGgoMCwxKSxuLnNldFVUQ0hvdXJzKDAsMCwwLDApfSksKChuLGUpPT57bi5zZXRVVENGdWxsWWVh
cihuLmdldFVUQ0Z1bGxZZWFyKCkrZSp0KX0pKTpudWxsO2NvbnN0IGl2PXJ2LnJhbmdlO2Z1bmN0
aW9uIG92KHQsbixlLGksbyxhKXtjb25zdCB1PVtbb3ksMSxRZ10sW295LDUsNWUzXSxbb3ksMTUs
MTVlM10sW295LDMwLDNlNF0sW2EsMSxKZ10sW2EsNSw1KkpnXSxbYSwxNSwxNSpKZ10sW2EsMzAs
MzAqSmddLFtvLDEsdHldLFtvLDMsMyp0eV0sW28sNiw2KnR5XSxbbywxMiwxMip0eV0sW2ksMSxu
eV0sW2ksMiwyKm55XSxbZSwxLGV5XSxbbiwxLHJ5XSxbbiwzLDMqcnldLFt0LDEsaXldXTtmdW5j
dGlvbiBjKG4sZSxpKXtjb25zdCBvPU1hdGguYWJzKGUtbikvaSxhPXIoKChbLCx0XSk9PnQpKS5y
aWdodCh1LG8pO2lmKGE9PT11Lmxlbmd0aClyZXR1cm4gdC5ldmVyeShaKG4vaXksZS9peSxpKSk7
aWYoMD09PWEpcmV0dXJuIFpnLmV2ZXJ5KE1hdGgubWF4KFoobixlLGkpLDEpKTtjb25zdFtjLGZd
PXVbby91W2EtMV1bMl08dVthXVsyXS9vP2EtMTphXTtyZXR1cm4gYy5ldmVyeShmKX1yZXR1cm5b
ZnVuY3Rpb24odCxuLGUpe2NvbnN0IHI9bjx0O3ImJihbdCxuXT1bbix0XSk7Y29uc3QgaT1lJiYi
ZnVuY3Rpb24iPT10eXBlb2YgZS5yYW5nZT9lOmModCxuLGUpLG89aT9pLnJhbmdlKHQsK24rMSk6
W107cmV0dXJuIHI/by5yZXZlcnNlKCk6b30sY119Y29uc3RbYXYsdXZdPW92KHJ2LEp5LHF5LGJ5
LGR5LGZ5KSxbY3YsZnZdPW92KG52LEt5LHd5LGd5LGx5LHV5KTtmdW5jdGlvbiBzdih0KXtpZigw
PD10LnkmJnQueTwxMDApe3ZhciBuPW5ldyBEYXRlKC0xLHQubSx0LmQsdC5ILHQuTSx0LlMsdC5M
KTtyZXR1cm4gbi5zZXRGdWxsWWVhcih0LnkpLG59cmV0dXJuIG5ldyBEYXRlKHQueSx0Lm0sdC5k
LHQuSCx0Lk0sdC5TLHQuTCl9ZnVuY3Rpb24gbHYodCl7aWYoMDw9dC55JiZ0Lnk8MTAwKXt2YXIg
bj1uZXcgRGF0ZShEYXRlLlVUQygtMSx0Lm0sdC5kLHQuSCx0Lk0sdC5TLHQuTCkpO3JldHVybiBu
LnNldFVUQ0Z1bGxZZWFyKHQueSksbn1yZXR1cm4gbmV3IERhdGUoRGF0ZS5VVEModC55LHQubSx0
LmQsdC5ILHQuTSx0LlMsdC5MKSl9ZnVuY3Rpb24gaHYodCxuLGUpe3JldHVybnt5OnQsbTpuLGQ6
ZSxIOjAsTTowLFM6MCxMOjB9fWZ1bmN0aW9uIGR2KHQpe3ZhciBuPXQuZGF0ZVRpbWUsZT10LmRh
dGUscj10LnRpbWUsaT10LnBlcmlvZHMsbz10LmRheXMsYT10LnNob3J0RGF5cyx1PXQubW9udGhz
LGM9dC5zaG9ydE1vbnRocyxmPXh2KGkpLHM9d3YoaSksbD14dihvKSxoPXd2KG8pLGQ9eHYoYSks
cD13dihhKSxnPXh2KHUpLHk9d3YodSksdj14dihjKSxfPXd2KGMpLGI9e2E6ZnVuY3Rpb24odCl7
cmV0dXJuIGFbdC5nZXREYXkoKV19LEE6ZnVuY3Rpb24odCl7cmV0dXJuIG9bdC5nZXREYXkoKV19
LGI6ZnVuY3Rpb24odCl7cmV0dXJuIGNbdC5nZXRNb250aCgpXX0sQjpmdW5jdGlvbih0KXtyZXR1
cm4gdVt0LmdldE1vbnRoKCldfSxjOm51bGwsZDpMdixlOkx2LGY6VnYsZzpvXyxHOnVfLEg6anYs
STpIdixqOlh2LEw6R3YsbTpXdixNOlp2LHA6ZnVuY3Rpb24odCl7cmV0dXJuIGlbKyh0LmdldEhv
dXJzKCk+PTEyKV19LHE6ZnVuY3Rpb24odCl7cmV0dXJuIDErfn4odC5nZXRNb250aCgpLzMpfSxR
OkNfLHM6UF8sUzpLdix1OlF2LFU6SnYsVjpuXyx3OmVfLFc6cl8seDpudWxsLFg6bnVsbCx5Omlf
LFk6YV8sWjpjXywiJSI6a199LG09e2E6ZnVuY3Rpb24odCl7cmV0dXJuIGFbdC5nZXRVVENEYXko
KV19LEE6ZnVuY3Rpb24odCl7cmV0dXJuIG9bdC5nZXRVVENEYXkoKV19LGI6ZnVuY3Rpb24odCl7
cmV0dXJuIGNbdC5nZXRVVENNb250aCgpXX0sQjpmdW5jdGlvbih0KXtyZXR1cm4gdVt0LmdldFVU
Q01vbnRoKCldfSxjOm51bGwsZDpmXyxlOmZfLGY6cF8sZzpBXyxHOkVfLEg6c18sSTpsXyxqOmhf
LEw6ZF8sbTpnXyxNOnlfLHA6ZnVuY3Rpb24odCl7cmV0dXJuIGlbKyh0LmdldFVUQ0hvdXJzKCk+
PTEyKV19LHE6ZnVuY3Rpb24odCl7cmV0dXJuIDErfn4odC5nZXRVVENNb250aCgpLzMpfSxROkNf
LHM6UF8sUzp2Xyx1Ol9fLFU6Yl8sVjp4Xyx3OndfLFc6TV8seDpudWxsLFg6bnVsbCx5OlRfLFk6
U18sWjpOXywiJSI6a199LHg9e2E6ZnVuY3Rpb24odCxuLGUpe3ZhciByPWQuZXhlYyhuLnNsaWNl
KGUpKTtyZXR1cm4gcj8odC53PXAuZ2V0KHJbMF0udG9Mb3dlckNhc2UoKSksZStyWzBdLmxlbmd0
aCk6LTF9LEE6ZnVuY3Rpb24odCxuLGUpe3ZhciByPWwuZXhlYyhuLnNsaWNlKGUpKTtyZXR1cm4g
cj8odC53PWguZ2V0KHJbMF0udG9Mb3dlckNhc2UoKSksZStyWzBdLmxlbmd0aCk6LTF9LGI6ZnVu
Y3Rpb24odCxuLGUpe3ZhciByPXYuZXhlYyhuLnNsaWNlKGUpKTtyZXR1cm4gcj8odC5tPV8uZ2V0
KHJbMF0udG9Mb3dlckNhc2UoKSksZStyWzBdLmxlbmd0aCk6LTF9LEI6ZnVuY3Rpb24odCxuLGUp
e3ZhciByPWcuZXhlYyhuLnNsaWNlKGUpKTtyZXR1cm4gcj8odC5tPXkuZ2V0KHJbMF0udG9Mb3dl
ckNhc2UoKSksZStyWzBdLmxlbmd0aCk6LTF9LGM6ZnVuY3Rpb24odCxlLHIpe3JldHVybiBUKHQs
bixlLHIpfSxkOiR2LGU6JHYsZjpJdixnOmt2LEc6TnYsSDpSdixJOlJ2LGo6RHYsTDpVdixtOnp2
LE06RnYscDpmdW5jdGlvbih0LG4sZSl7dmFyIHI9Zi5leGVjKG4uc2xpY2UoZSkpO3JldHVybiBy
Pyh0LnA9cy5nZXQoclswXS50b0xvd2VyQ2FzZSgpKSxlK3JbMF0ubGVuZ3RoKTotMX0scTpQdixR
OkJ2LHM6WXYsUzpxdix1OlR2LFU6QXYsVjpTdix3Ok12LFc6RXYseDpmdW5jdGlvbih0LG4scil7
cmV0dXJuIFQodCxlLG4scil9LFg6ZnVuY3Rpb24odCxuLGUpe3JldHVybiBUKHQscixuLGUpfSx5
Omt2LFk6TnYsWjpDdiwiJSI6T3Z9O2Z1bmN0aW9uIHcodCxuKXtyZXR1cm4gZnVuY3Rpb24oZSl7
dmFyIHIsaSxvLGE9W10sdT0tMSxjPTAsZj10Lmxlbmd0aDtmb3IoZSBpbnN0YW5jZW9mIERhdGV8
fChlPW5ldyBEYXRlKCtlKSk7Kyt1PGY7KTM3PT09dC5jaGFyQ29kZUF0KHUpJiYoYS5wdXNoKHQu
c2xpY2UoYyx1KSksbnVsbCE9KGk9Z3Zbcj10LmNoYXJBdCgrK3UpXSk/cj10LmNoYXJBdCgrK3Up
Omk9ImUiPT09cj8iICI6IjAiLChvPW5bcl0pJiYocj1vKGUsaSkpLGEucHVzaChyKSxjPXUrMSk7
cmV0dXJuIGEucHVzaCh0LnNsaWNlKGMsdSkpLGEuam9pbigiIil9fWZ1bmN0aW9uIE0odCxuKXty
ZXR1cm4gZnVuY3Rpb24oZSl7dmFyIHIsaSxvPWh2KDE5MDAsdm9pZCAwLDEpO2lmKFQobyx0LGUr
PSIiLDApIT1lLmxlbmd0aClyZXR1cm4gbnVsbDtpZigiUSJpbiBvKXJldHVybiBuZXcgRGF0ZShv
LlEpO2lmKCJzImluIG8pcmV0dXJuIG5ldyBEYXRlKDFlMypvLnMrKCJMImluIG8/by5MOjApKTtp
ZihuJiYhKCJaImluIG8pJiYoby5aPTApLCJwImluIG8mJihvLkg9by5IJTEyKzEyKm8ucCksdm9p
ZCAwPT09by5tJiYoby5tPSJxImluIG8/by5xOjApLCJWImluIG8pe2lmKG8uVjwxfHxvLlY+NTMp
cmV0dXJuIG51bGw7InciaW4gb3x8KG8udz0xKSwiWiJpbiBvPyhpPShyPWx2KGh2KG8ueSwwLDEp
KSkuZ2V0VVRDRGF5KCkscj1pPjR8fDA9PT1pP1V5LmNlaWwocik6VXkocikscj12eS5vZmZzZXQo
ciw3KihvLlYtMSkpLG8ueT1yLmdldFVUQ0Z1bGxZZWFyKCksby5tPXIuZ2V0VVRDTW9udGgoKSxv
LmQ9ci5nZXRVVENEYXRlKCkrKG8udys2KSU3KTooaT0ocj1zdihodihvLnksMCwxKSkpLmdldERh
eSgpLHI9aT40fHwwPT09aT9NeS5jZWlsKHIpOk15KHIpLHI9Z3kub2Zmc2V0KHIsNyooby5WLTEp
KSxvLnk9ci5nZXRGdWxsWWVhcigpLG8ubT1yLmdldE1vbnRoKCksby5kPXIuZ2V0RGF0ZSgpKyhv
LncrNiklNyl9ZWxzZSgiVyJpbiBvfHwiVSJpbiBvKSYmKCJ3ImluIG98fChvLnc9InUiaW4gbz9v
LnUlNzoiVyJpbiBvPzE6MCksaT0iWiJpbiBvP2x2KGh2KG8ueSwwLDEpKS5nZXRVVENEYXkoKTpz
dihodihvLnksMCwxKSkuZ2V0RGF5KCksby5tPTAsby5kPSJXImluIG8/KG8udys2KSU3Kzcqby5X
LShpKzUpJTc6by53Kzcqby5VLShpKzYpJTcpO3JldHVybiJaImluIG8/KG8uSCs9by5aLzEwMHww
LG8uTSs9by5aJTEwMCxsdihvKSk6c3Yobyl9fWZ1bmN0aW9uIFQodCxuLGUscil7Zm9yKHZhciBp
LG8sYT0wLHU9bi5sZW5ndGgsYz1lLmxlbmd0aDthPHU7KXtpZihyPj1jKXJldHVybi0xO2lmKDM3
PT09KGk9bi5jaGFyQ29kZUF0KGErKykpKXtpZihpPW4uY2hhckF0KGErKyksIShvPXhbaSBpbiBn
dj9uLmNoYXJBdChhKyspOmldKXx8KHI9byh0LGUscikpPDApcmV0dXJuLTF9ZWxzZSBpZihpIT1l
LmNoYXJDb2RlQXQocisrKSlyZXR1cm4tMX1yZXR1cm4gcn1yZXR1cm4gYi54PXcoZSxiKSxiLlg9
dyhyLGIpLGIuYz13KG4sYiksbS54PXcoZSxtKSxtLlg9dyhyLG0pLG0uYz13KG4sbSkse2Zvcm1h
dDpmdW5jdGlvbih0KXt2YXIgbj13KHQrPSIiLGIpO3JldHVybiBuLnRvU3RyaW5nPWZ1bmN0aW9u
KCl7cmV0dXJuIHR9LG59LHBhcnNlOmZ1bmN0aW9uKHQpe3ZhciBuPU0odCs9IiIsITEpO3JldHVy
biBuLnRvU3RyaW5nPWZ1bmN0aW9uKCl7cmV0dXJuIHR9LG59LHV0Y0Zvcm1hdDpmdW5jdGlvbih0
KXt2YXIgbj13KHQrPSIiLG0pO3JldHVybiBuLnRvU3RyaW5nPWZ1bmN0aW9uKCl7cmV0dXJuIHR9
LG59LHV0Y1BhcnNlOmZ1bmN0aW9uKHQpe3ZhciBuPU0odCs9IiIsITApO3JldHVybiBuLnRvU3Ry
aW5nPWZ1bmN0aW9uKCl7cmV0dXJuIHR9LG59fX12YXIgcHYsZ3Y9eyItIjoiIixfOiIgIiwwOiIw
In0seXY9L15ccypcZCsvLHZ2PS9eJS8sX3Y9L1tcXF4kKis/fFtcXSgpLnt9XS9nO2Z1bmN0aW9u
IGJ2KHQsbixlKXt2YXIgcj10PDA/Ii0iOiIiLGk9KHI/LXQ6dCkrIiIsbz1pLmxlbmd0aDtyZXR1
cm4gcisobzxlP25ldyBBcnJheShlLW8rMSkuam9pbihuKStpOmkpfWZ1bmN0aW9uIG12KHQpe3Jl
dHVybiB0LnJlcGxhY2UoX3YsIlxcJCYiKX1mdW5jdGlvbiB4dih0KXtyZXR1cm4gbmV3IFJlZ0V4
cCgiXig/OiIrdC5tYXAobXYpLmpvaW4oInwiKSsiKSIsImkiKX1mdW5jdGlvbiB3dih0KXtyZXR1
cm4gbmV3IE1hcCh0Lm1hcCgoKHQsbik9Plt0LnRvTG93ZXJDYXNlKCksbl0pKSl9ZnVuY3Rpb24g
TXYodCxuLGUpe3ZhciByPXl2LmV4ZWMobi5zbGljZShlLGUrMSkpO3JldHVybiByPyh0Lnc9K3Jb
MF0sZStyWzBdLmxlbmd0aCk6LTF9ZnVuY3Rpb24gVHYodCxuLGUpe3ZhciByPXl2LmV4ZWMobi5z
bGljZShlLGUrMSkpO3JldHVybiByPyh0LnU9K3JbMF0sZStyWzBdLmxlbmd0aCk6LTF9ZnVuY3Rp
b24gQXYodCxuLGUpe3ZhciByPXl2LmV4ZWMobi5zbGljZShlLGUrMikpO3JldHVybiByPyh0LlU9
K3JbMF0sZStyWzBdLmxlbmd0aCk6LTF9ZnVuY3Rpb24gU3YodCxuLGUpe3ZhciByPXl2LmV4ZWMo
bi5zbGljZShlLGUrMikpO3JldHVybiByPyh0LlY9K3JbMF0sZStyWzBdLmxlbmd0aCk6LTF9ZnVu
Y3Rpb24gRXYodCxuLGUpe3ZhciByPXl2LmV4ZWMobi5zbGljZShlLGUrMikpO3JldHVybiByPyh0
Llc9K3JbMF0sZStyWzBdLmxlbmd0aCk6LTF9ZnVuY3Rpb24gTnYodCxuLGUpe3ZhciByPXl2LmV4
ZWMobi5zbGljZShlLGUrNCkpO3JldHVybiByPyh0Lnk9K3JbMF0sZStyWzBdLmxlbmd0aCk6LTF9
ZnVuY3Rpb24ga3YodCxuLGUpe3ZhciByPXl2LmV4ZWMobi5zbGljZShlLGUrMikpO3JldHVybiBy
Pyh0Lnk9K3JbMF0rKCtyWzBdPjY4PzE5MDA6MmUzKSxlK3JbMF0ubGVuZ3RoKTotMX1mdW5jdGlv
biBDdih0LG4sZSl7dmFyIHI9L14oWil8KFsrLV1cZFxkKSg/Ojo/KFxkXGQpKT8vLmV4ZWMobi5z
bGljZShlLGUrNikpO3JldHVybiByPyh0Llo9clsxXT8wOi0oclsyXSsoclszXXx8IjAwIikpLGUr
clswXS5sZW5ndGgpOi0xfWZ1bmN0aW9uIFB2KHQsbixlKXt2YXIgcj15di5leGVjKG4uc2xpY2Uo
ZSxlKzEpKTtyZXR1cm4gcj8odC5xPTMqclswXS0zLGUrclswXS5sZW5ndGgpOi0xfWZ1bmN0aW9u
IHp2KHQsbixlKXt2YXIgcj15di5leGVjKG4uc2xpY2UoZSxlKzIpKTtyZXR1cm4gcj8odC5tPXJb
MF0tMSxlK3JbMF0ubGVuZ3RoKTotMX1mdW5jdGlvbiAkdih0LG4sZSl7dmFyIHI9eXYuZXhlYyhu
LnNsaWNlKGUsZSsyKSk7cmV0dXJuIHI/KHQuZD0rclswXSxlK3JbMF0ubGVuZ3RoKTotMX1mdW5j
dGlvbiBEdih0LG4sZSl7dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUsZSszKSk7cmV0dXJuIHI/KHQu
bT0wLHQuZD0rclswXSxlK3JbMF0ubGVuZ3RoKTotMX1mdW5jdGlvbiBSdih0LG4sZSl7dmFyIHI9
eXYuZXhlYyhuLnNsaWNlKGUsZSsyKSk7cmV0dXJuIHI/KHQuSD0rclswXSxlK3JbMF0ubGVuZ3Ro
KTotMX1mdW5jdGlvbiBGdih0LG4sZSl7dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUsZSsyKSk7cmV0
dXJuIHI/KHQuTT0rclswXSxlK3JbMF0ubGVuZ3RoKTotMX1mdW5jdGlvbiBxdih0LG4sZSl7dmFy
IHI9eXYuZXhlYyhuLnNsaWNlKGUsZSsyKSk7cmV0dXJuIHI/KHQuUz0rclswXSxlK3JbMF0ubGVu
Z3RoKTotMX1mdW5jdGlvbiBVdih0LG4sZSl7dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUsZSszKSk7
cmV0dXJuIHI/KHQuTD0rclswXSxlK3JbMF0ubGVuZ3RoKTotMX1mdW5jdGlvbiBJdih0LG4sZSl7
dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUsZSs2KSk7cmV0dXJuIHI/KHQuTD1NYXRoLmZsb29yKHJb
MF0vMWUzKSxlK3JbMF0ubGVuZ3RoKTotMX1mdW5jdGlvbiBPdih0LG4sZSl7dmFyIHI9dnYuZXhl
YyhuLnNsaWNlKGUsZSsxKSk7cmV0dXJuIHI/ZStyWzBdLmxlbmd0aDotMX1mdW5jdGlvbiBCdih0
LG4sZSl7dmFyIHI9eXYuZXhlYyhuLnNsaWNlKGUpKTtyZXR1cm4gcj8odC5RPStyWzBdLGUrclsw
XS5sZW5ndGgpOi0xfWZ1bmN0aW9uIFl2KHQsbixlKXt2YXIgcj15di5leGVjKG4uc2xpY2UoZSkp
O3JldHVybiByPyh0LnM9K3JbMF0sZStyWzBdLmxlbmd0aCk6LTF9ZnVuY3Rpb24gTHYodCxuKXty
ZXR1cm4gYnYodC5nZXREYXRlKCksbiwyKX1mdW5jdGlvbiBqdih0LG4pe3JldHVybiBidih0Lmdl
dEhvdXJzKCksbiwyKX1mdW5jdGlvbiBIdih0LG4pe3JldHVybiBidih0LmdldEhvdXJzKCklMTJ8
fDEyLG4sMil9ZnVuY3Rpb24gWHYodCxuKXtyZXR1cm4gYnYoMStneS5jb3VudChudih0KSx0KSxu
LDMpfWZ1bmN0aW9uIEd2KHQsbil7cmV0dXJuIGJ2KHQuZ2V0TWlsbGlzZWNvbmRzKCksbiwzKX1m
dW5jdGlvbiBWdih0LG4pe3JldHVybiBHdih0LG4pKyIwMDAifWZ1bmN0aW9uIFd2KHQsbil7cmV0
dXJuIGJ2KHQuZ2V0TW9udGgoKSsxLG4sMil9ZnVuY3Rpb24gWnYodCxuKXtyZXR1cm4gYnYodC5n
ZXRNaW51dGVzKCksbiwyKX1mdW5jdGlvbiBLdih0LG4pe3JldHVybiBidih0LmdldFNlY29uZHMo
KSxuLDIpfWZ1bmN0aW9uIFF2KHQpe3ZhciBuPXQuZ2V0RGF5KCk7cmV0dXJuIDA9PT1uPzc6bn1m
dW5jdGlvbiBKdih0LG4pe3JldHVybiBidih3eS5jb3VudChudih0KS0xLHQpLG4sMil9ZnVuY3Rp
b24gdF8odCl7dmFyIG49dC5nZXREYXkoKTtyZXR1cm4gbj49NHx8MD09PW4/U3kodCk6U3kuY2Vp
bCh0KX1mdW5jdGlvbiBuXyh0LG4pe3JldHVybiB0PXRfKHQpLGJ2KFN5LmNvdW50KG52KHQpLHQp
Kyg0PT09bnYodCkuZ2V0RGF5KCkpLG4sMil9ZnVuY3Rpb24gZV8odCl7cmV0dXJuIHQuZ2V0RGF5
KCl9ZnVuY3Rpb24gcl8odCxuKXtyZXR1cm4gYnYoTXkuY291bnQobnYodCktMSx0KSxuLDIpfWZ1
bmN0aW9uIGlfKHQsbil7cmV0dXJuIGJ2KHQuZ2V0RnVsbFllYXIoKSUxMDAsbiwyKX1mdW5jdGlv
biBvXyh0LG4pe3JldHVybiBidigodD10Xyh0KSkuZ2V0RnVsbFllYXIoKSUxMDAsbiwyKX1mdW5j
dGlvbiBhXyh0LG4pe3JldHVybiBidih0LmdldEZ1bGxZZWFyKCklMWU0LG4sNCl9ZnVuY3Rpb24g
dV8odCxuKXt2YXIgZT10LmdldERheSgpO3JldHVybiBidigodD1lPj00fHwwPT09ZT9TeSh0KTpT
eS5jZWlsKHQpKS5nZXRGdWxsWWVhcigpJTFlNCxuLDQpfWZ1bmN0aW9uIGNfKHQpe3ZhciBuPXQu
Z2V0VGltZXpvbmVPZmZzZXQoKTtyZXR1cm4obj4wPyItIjoobio9LTEsIisiKSkrYnYobi82MHww
LCIwIiwyKStidihuJTYwLCIwIiwyKX1mdW5jdGlvbiBmXyh0LG4pe3JldHVybiBidih0LmdldFVU
Q0RhdGUoKSxuLDIpfWZ1bmN0aW9uIHNfKHQsbil7cmV0dXJuIGJ2KHQuZ2V0VVRDSG91cnMoKSxu
LDIpfWZ1bmN0aW9uIGxfKHQsbil7cmV0dXJuIGJ2KHQuZ2V0VVRDSG91cnMoKSUxMnx8MTIsbiwy
KX1mdW5jdGlvbiBoXyh0LG4pe3JldHVybiBidigxK3Z5LmNvdW50KHJ2KHQpLHQpLG4sMyl9ZnVu
Y3Rpb24gZF8odCxuKXtyZXR1cm4gYnYodC5nZXRVVENNaWxsaXNlY29uZHMoKSxuLDMpfWZ1bmN0
aW9uIHBfKHQsbil7cmV0dXJuIGRfKHQsbikrIjAwMCJ9ZnVuY3Rpb24gZ18odCxuKXtyZXR1cm4g
YnYodC5nZXRVVENNb250aCgpKzEsbiwyKX1mdW5jdGlvbiB5Xyh0LG4pe3JldHVybiBidih0Lmdl
dFVUQ01pbnV0ZXMoKSxuLDIpfWZ1bmN0aW9uIHZfKHQsbil7cmV0dXJuIGJ2KHQuZ2V0VVRDU2Vj
b25kcygpLG4sMil9ZnVuY3Rpb24gX18odCl7dmFyIG49dC5nZXRVVENEYXkoKTtyZXR1cm4gMD09
PW4/NzpufWZ1bmN0aW9uIGJfKHQsbil7cmV0dXJuIGJ2KHF5LmNvdW50KHJ2KHQpLTEsdCksbiwy
KX1mdW5jdGlvbiBtXyh0KXt2YXIgbj10LmdldFVUQ0RheSgpO3JldHVybiBuPj00fHwwPT09bj9C
eSh0KTpCeS5jZWlsKHQpfWZ1bmN0aW9uIHhfKHQsbil7cmV0dXJuIHQ9bV8odCksYnYoQnkuY291
bnQocnYodCksdCkrKDQ9PT1ydih0KS5nZXRVVENEYXkoKSksbiwyKX1mdW5jdGlvbiB3Xyh0KXty
ZXR1cm4gdC5nZXRVVENEYXkoKX1mdW5jdGlvbiBNXyh0LG4pe3JldHVybiBidihVeS5jb3VudChy
dih0KS0xLHQpLG4sMil9ZnVuY3Rpb24gVF8odCxuKXtyZXR1cm4gYnYodC5nZXRVVENGdWxsWWVh
cigpJTEwMCxuLDIpfWZ1bmN0aW9uIEFfKHQsbil7cmV0dXJuIGJ2KCh0PW1fKHQpKS5nZXRVVENG
dWxsWWVhcigpJTEwMCxuLDIpfWZ1bmN0aW9uIFNfKHQsbil7cmV0dXJuIGJ2KHQuZ2V0VVRDRnVs
bFllYXIoKSUxZTQsbiw0KX1mdW5jdGlvbiBFXyh0LG4pe3ZhciBlPXQuZ2V0VVRDRGF5KCk7cmV0
dXJuIGJ2KCh0PWU+PTR8fDA9PT1lP0J5KHQpOkJ5LmNlaWwodCkpLmdldFVUQ0Z1bGxZZWFyKCkl
MWU0LG4sNCl9ZnVuY3Rpb24gTl8oKXtyZXR1cm4iKzAwMDAifWZ1bmN0aW9uIGtfKCl7cmV0dXJu
IiUifWZ1bmN0aW9uIENfKHQpe3JldHVybit0fWZ1bmN0aW9uIFBfKHQpe3JldHVybiBNYXRoLmZs
b29yKCt0LzFlMyl9ZnVuY3Rpb24gel8obil7cmV0dXJuIHB2PWR2KG4pLHQudGltZUZvcm1hdD1w
di5mb3JtYXQsdC50aW1lUGFyc2U9cHYucGFyc2UsdC51dGNGb3JtYXQ9cHYudXRjRm9ybWF0LHQu
dXRjUGFyc2U9cHYudXRjUGFyc2UscHZ9dC50aW1lRm9ybWF0PXZvaWQgMCx0LnRpbWVQYXJzZT12
b2lkIDAsdC51dGNGb3JtYXQ9dm9pZCAwLHQudXRjUGFyc2U9dm9pZCAwLHpfKHtkYXRlVGltZToi
JXgsICVYIixkYXRlOiIlLW0vJS1kLyVZIix0aW1lOiIlLUk6JU06JVMgJXAiLHBlcmlvZHM6WyJB
TSIsIlBNIl0sZGF5czpbIlN1bmRheSIsIk1vbmRheSIsIlR1ZXNkYXkiLCJXZWRuZXNkYXkiLCJU
aHVyc2RheSIsIkZyaWRheSIsIlNhdHVyZGF5Il0sc2hvcnREYXlzOlsiU3VuIiwiTW9uIiwiVHVl
IiwiV2VkIiwiVGh1IiwiRnJpIiwiU2F0Il0sbW9udGhzOlsiSmFudWFyeSIsIkZlYnJ1YXJ5Iiwi
TWFyY2giLCJBcHJpbCIsIk1heSIsIkp1bmUiLCJKdWx5IiwiQXVndXN0IiwiU2VwdGVtYmVyIiwi
T2N0b2JlciIsIk5vdmVtYmVyIiwiRGVjZW1iZXIiXSxzaG9ydE1vbnRoczpbIkphbiIsIkZlYiIs
Ik1hciIsIkFwciIsIk1heSIsIkp1biIsIkp1bCIsIkF1ZyIsIlNlcCIsIk9jdCIsIk5vdiIsIkRl
YyJdfSk7dmFyICRfPSIlWS0lbS0lZFQlSDolTTolUy4lTFoiO3ZhciBEXz1EYXRlLnByb3RvdHlw
ZS50b0lTT1N0cmluZz9mdW5jdGlvbih0KXtyZXR1cm4gdC50b0lTT1N0cmluZygpfTp0LnV0Y0Zv
cm1hdCgkXyksUl89RF87dmFyIEZfPStuZXcgRGF0ZSgiMjAwMC0wMS0wMVQwMDowMDowMC4wMDBa
Iik/ZnVuY3Rpb24odCl7dmFyIG49bmV3IERhdGUodCk7cmV0dXJuIGlzTmFOKG4pP251bGw6bn06
dC51dGNQYXJzZSgkXykscV89Rl87ZnVuY3Rpb24gVV8odCl7cmV0dXJuIG5ldyBEYXRlKHQpfWZ1
bmN0aW9uIElfKHQpe3JldHVybiB0IGluc3RhbmNlb2YgRGF0ZT8rdDorbmV3IERhdGUoK3QpfWZ1
bmN0aW9uIE9fKHQsbixlLHIsaSxvLGEsdSxjLGYpe3ZhciBzPUVnKCksbD1zLmludmVydCxoPXMu
ZG9tYWluLGQ9ZigiLiVMIikscD1mKCI6JVMiKSxnPWYoIiVJOiVNIikseT1mKCIlSSAlcCIpLHY9
ZigiJWEgJWQiKSxfPWYoIiViICVkIiksYj1mKCIlQiIpLG09ZigiJVkiKTtmdW5jdGlvbiB4KHQp
e3JldHVybihjKHQpPHQ/ZDp1KHQpPHQ/cDphKHQpPHQ/ZzpvKHQpPHQ/eTpyKHQpPHQ/aSh0KTx0
P3Y6XzplKHQpPHQ/YjptKSh0KX1yZXR1cm4gcy5pbnZlcnQ9ZnVuY3Rpb24odCl7cmV0dXJuIG5l
dyBEYXRlKGwodCkpfSxzLmRvbWFpbj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0
aD9oKEFycmF5LmZyb20odCxJXykpOmgoKS5tYXAoVV8pfSxzLnRpY2tzPWZ1bmN0aW9uKG4pe3Zh
ciBlPWgoKTtyZXR1cm4gdChlWzBdLGVbZS5sZW5ndGgtMV0sbnVsbD09bj8xMDpuKX0scy50aWNr
Rm9ybWF0PWZ1bmN0aW9uKHQsbil7cmV0dXJuIG51bGw9PW4/eDpmKG4pfSxzLm5pY2U9ZnVuY3Rp
b24odCl7dmFyIGU9aCgpO3JldHVybiB0JiYiZnVuY3Rpb24iPT10eXBlb2YgdC5yYW5nZXx8KHQ9
bihlWzBdLGVbZS5sZW5ndGgtMV0sbnVsbD09dD8xMDp0KSksdD9oKENnKGUsdCkpOnN9LHMuY29w
eT1mdW5jdGlvbigpe3JldHVybiBBZyhzLE9fKHQsbixlLHIsaSxvLGEsdSxjLGYpKX0sc31mdW5j
dGlvbiBCXygpe3ZhciB0LG4sZSxyLGksbz0wLGE9MSx1PXhnLGM9ITE7ZnVuY3Rpb24gZihuKXty
ZXR1cm4gbnVsbD09bnx8aXNOYU4obj0rbik/aTp1KDA9PT1lPy41OihuPShyKG4pLXQpKmUsYz9N
YXRoLm1heCgwLE1hdGgubWluKDEsbikpOm4pKX1mdW5jdGlvbiBzKHQpe3JldHVybiBmdW5jdGlv
bihuKXt2YXIgZSxyO3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhbZSxyXT1uLHU9dChlLHIpLGYp
Olt1KDApLHUoMSldfX1yZXR1cm4gZi5kb21haW49ZnVuY3Rpb24oaSl7cmV0dXJuIGFyZ3VtZW50
cy5sZW5ndGg/KFtvLGFdPWksdD1yKG89K28pLG49cihhPSthKSxlPXQ9PT1uPzA6MS8obi10KSxm
KTpbbyxhXX0sZi5jbGFtcD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYz0h
IXQsZik6Y30sZi5pbnRlcnBvbGF0b3I9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5n
dGg/KHU9dCxmKTp1fSxmLnJhbmdlPXMoVnIpLGYucmFuZ2VSb3VuZD1zKFdyKSxmLnVua25vd249
ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGk9dCxmKTppfSxmdW5jdGlvbihp
KXtyZXR1cm4gcj1pLHQ9aShvKSxuPWkoYSksZT10PT09bj8wOjEvKG4tdCksZn19ZnVuY3Rpb24g
WV8odCxuKXtyZXR1cm4gbi5kb21haW4odC5kb21haW4oKSkuaW50ZXJwb2xhdG9yKHQuaW50ZXJw
b2xhdG9yKCkpLmNsYW1wKHQuY2xhbXAoKSkudW5rbm93bih0LnVua25vd24oKSl9ZnVuY3Rpb24g
TF8oKXt2YXIgdD1qZyhCXygpKTtyZXR1cm4gdC5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIFlfKHQs
TF8oKSkuZXhwb25lbnQodC5leHBvbmVudCgpKX0scGcuYXBwbHkodCxhcmd1bWVudHMpfWZ1bmN0
aW9uIGpfKCl7dmFyIHQsbixlLHIsaSxvLGEsdT0wLGM9LjUsZj0xLHM9MSxsPXhnLGg9ITE7ZnVu
Y3Rpb24gZCh0KXtyZXR1cm4gaXNOYU4odD0rdCk/YToodD0uNSsoKHQ9K28odCkpLW4pKihzKnQ8
cypuP3I6aSksbChoP01hdGgubWF4KDAsTWF0aC5taW4oMSx0KSk6dCkpfWZ1bmN0aW9uIHAodCl7
cmV0dXJuIGZ1bmN0aW9uKG4pe3ZhciBlLHIsaTtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oW2Us
cixpXT1uLGw9cGkodCxbZSxyLGldKSxkKTpbbCgwKSxsKC41KSxsKDEpXX19cmV0dXJuIGQuZG9t
YWluPWZ1bmN0aW9uKGEpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhbdSxjLGZdPWEsdD1vKHU9
K3UpLG49byhjPStjKSxlPW8oZj0rZikscj10PT09bj8wOi41LyhuLXQpLGk9bj09PWU/MDouNS8o
ZS1uKSxzPW48dD8tMToxLGQpOlt1LGMsZl19LGQuY2xhbXA9ZnVuY3Rpb24odCl7cmV0dXJuIGFy
Z3VtZW50cy5sZW5ndGg/KGg9ISF0LGQpOmh9LGQuaW50ZXJwb2xhdG9yPWZ1bmN0aW9uKHQpe3Jl
dHVybiBhcmd1bWVudHMubGVuZ3RoPyhsPXQsZCk6bH0sZC5yYW5nZT1wKFZyKSxkLnJhbmdlUm91
bmQ9cChXciksZC51bmtub3duPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhh
PXQsZCk6YX0sZnVuY3Rpb24oYSl7cmV0dXJuIG89YSx0PWEodSksbj1hKGMpLGU9YShmKSxyPXQ9
PT1uPzA6LjUvKG4tdCksaT1uPT09ZT8wOi41LyhlLW4pLHM9bjx0Py0xOjEsZH19ZnVuY3Rpb24g
SF8oKXt2YXIgdD1qZyhqXygpKTtyZXR1cm4gdC5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIFlfKHQs
SF8oKSkuZXhwb25lbnQodC5leHBvbmVudCgpKX0scGcuYXBwbHkodCxhcmd1bWVudHMpfWZ1bmN0
aW9uIFhfKHQpe2Zvcih2YXIgbj10Lmxlbmd0aC82fDAsZT1uZXcgQXJyYXkobikscj0wO3I8bjsp
ZVtyXT0iIyIrdC5zbGljZSg2KnIsNiorK3IpO3JldHVybiBlfXZhciBHXz1YXygiMWY3N2I0ZmY3
ZjBlMmNhMDJjZDYyNzI4OTQ2N2JkOGM1NjRiZTM3N2MyN2Y3ZjdmYmNiZDIyMTdiZWNmIiksVl89
WF8oIjdmYzk3ZmJlYWVkNGZkYzA4NmZmZmY5OTM4NmNiMGYwMDI3ZmJmNWIxNzY2NjY2NiIpLFdf
PVhfKCIxYjllNzdkOTVmMDI3NTcwYjNlNzI5OGE2NmE2MWVlNmFiMDJhNjc2MWQ2NjY2NjYiKSxa
Xz1YXygiYTZjZWUzMWY3OGI0YjJkZjhhMzNhMDJjZmI5YTk5ZTMxYTFjZmRiZjZmZmY3ZjAwY2Fi
MmQ2NmEzZDlhZmZmZjk5YjE1OTI4IiksS189WF8oImZiYjRhZWIzY2RlM2NjZWJjNWRlY2JlNGZl
ZDlhNmZmZmZjY2U1ZDhiZGZkZGFlY2YyZjJmMiIpLFFfPVhfKCJiM2UyY2RmZGNkYWNjYmQ1ZThm
NGNhZTRlNmY1YzlmZmYyYWVmMWUyY2NjY2NjY2MiKSxKXz1YXygiZTQxYTFjMzc3ZWI4NGRhZjRh
OTg0ZWEzZmY3ZjAwZmZmZjMzYTY1NjI4Zjc4MWJmOTk5OTk5IiksdGI9WF8oIjY2YzJhNWZjOGQ2
MjhkYTBjYmU3OGFjM2E2ZDg1NGZmZDkyZmU1YzQ5NGIzYjNiMyIpLG5iPVhfKCI4ZGQzYzdmZmZm
YjNiZWJhZGFmYjgwNzI4MGIxZDNmZGI0NjJiM2RlNjlmY2NkZTVkOWQ5ZDliYzgwYmRjY2ViYzVm
ZmVkNmYiKSxlYj1YXygiNGU3OWE3ZjI4ZTJjZTE1NzU5NzZiN2IyNTlhMTRmZWRjOTQ5YWY3YWEx
ZmY5ZGE3OWM3NTVmYmFiMGFiIikscmI9dD0+cXIodFt0Lmxlbmd0aC0xXSksaWI9bmV3IEFycmF5
KDMpLmNvbmNhdCgiZDhiMzY1ZjVmNWY1NWFiNGFjIiwiYTY2MTFhZGZjMjdkODBjZGMxMDE4NTcx
IiwiYTY2MTFhZGZjMjdkZjVmNWY1ODBjZGMxMDE4NTcxIiwiOGM1MTBhZDhiMzY1ZjZlOGMzYzdl
YWU1NWFiNGFjMDE2NjVlIiwiOGM1MTBhZDhiMzY1ZjZlOGMzZjVmNWY1YzdlYWU1NWFiNGFjMDE2
NjVlIiwiOGM1MTBhYmY4MTJkZGZjMjdkZjZlOGMzYzdlYWU1ODBjZGMxMzU5NzhmMDE2NjVlIiwi
OGM1MTBhYmY4MTJkZGZjMjdkZjZlOGMzZjVmNWY1YzdlYWU1ODBjZGMxMzU5NzhmMDE2NjVlIiwi
NTQzMDA1OGM1MTBhYmY4MTJkZGZjMjdkZjZlOGMzYzdlYWU1ODBjZGMxMzU5NzhmMDE2NjVlMDAz
YzMwIiwiNTQzMDA1OGM1MTBhYmY4MTJkZGZjMjdkZjZlOGMzZjVmNWY1YzdlYWU1ODBjZGMxMzU5
NzhmMDE2NjVlMDAzYzMwIikubWFwKFhfKSxvYj1yYihpYiksYWI9bmV3IEFycmF5KDMpLmNvbmNh
dCgiYWY4ZGMzZjdmN2Y3N2ZiZjdiIiwiN2IzMjk0YzJhNWNmYTZkYmEwMDA4ODM3IiwiN2IzMjk0
YzJhNWNmZjdmN2Y3YTZkYmEwMDA4ODM3IiwiNzYyYTgzYWY4ZGMzZTdkNGU4ZDlmMGQzN2ZiZjdi
MWI3ODM3IiwiNzYyYTgzYWY4ZGMzZTdkNGU4ZjdmN2Y3ZDlmMGQzN2ZiZjdiMWI3ODM3IiwiNzYy
YTgzOTk3MGFiYzJhNWNmZTdkNGU4ZDlmMGQzYTZkYmEwNWFhZTYxMWI3ODM3IiwiNzYyYTgzOTk3
MGFiYzJhNWNmZTdkNGU4ZjdmN2Y3ZDlmMGQzYTZkYmEwNWFhZTYxMWI3ODM3IiwiNDAwMDRiNzYy
YTgzOTk3MGFiYzJhNWNmZTdkNGU4ZDlmMGQzYTZkYmEwNWFhZTYxMWI3ODM3MDA0NDFiIiwiNDAw
MDRiNzYyYTgzOTk3MGFiYzJhNWNmZTdkNGU4ZjdmN2Y3ZDlmMGQzYTZkYmEwNWFhZTYxMWI3ODM3
MDA0NDFiIikubWFwKFhfKSx1Yj1yYihhYiksY2I9bmV3IEFycmF5KDMpLmNvbmNhdCgiZTlhM2M5
ZjdmN2Y3YTFkNzZhIiwiZDAxYzhiZjFiNmRhYjhlMTg2NGRhYzI2IiwiZDAxYzhiZjFiNmRhZjdm
N2Y3YjhlMTg2NGRhYzI2IiwiYzUxYjdkZTlhM2M5ZmRlMGVmZTZmNWQwYTFkNzZhNGQ5MjIxIiwi
YzUxYjdkZTlhM2M5ZmRlMGVmZjdmN2Y3ZTZmNWQwYTFkNzZhNGQ5MjIxIiwiYzUxYjdkZGU3N2Fl
ZjFiNmRhZmRlMGVmZTZmNWQwYjhlMTg2N2ZiYzQxNGQ5MjIxIiwiYzUxYjdkZGU3N2FlZjFiNmRh
ZmRlMGVmZjdmN2Y3ZTZmNWQwYjhlMTg2N2ZiYzQxNGQ5MjIxIiwiOGUwMTUyYzUxYjdkZGU3N2Fl
ZjFiNmRhZmRlMGVmZTZmNWQwYjhlMTg2N2ZiYzQxNGQ5MjIxMjc2NDE5IiwiOGUwMTUyYzUxYjdk
ZGU3N2FlZjFiNmRhZmRlMGVmZjdmN2Y3ZTZmNWQwYjhlMTg2N2ZiYzQxNGQ5MjIxMjc2NDE5Iiku
bWFwKFhfKSxmYj1yYihjYiksc2I9bmV3IEFycmF5KDMpLmNvbmNhdCgiOTk4ZWMzZjdmN2Y3ZjFh
MzQwIiwiNWUzYzk5YjJhYmQyZmRiODYzZTY2MTAxIiwiNWUzYzk5YjJhYmQyZjdmN2Y3ZmRiODYz
ZTY2MTAxIiwiNTQyNzg4OTk4ZWMzZDhkYWViZmVlMGI2ZjFhMzQwYjM1ODA2IiwiNTQyNzg4OTk4
ZWMzZDhkYWViZjdmN2Y3ZmVlMGI2ZjFhMzQwYjM1ODA2IiwiNTQyNzg4ODA3M2FjYjJhYmQyZDhk
YWViZmVlMGI2ZmRiODYzZTA4MjE0YjM1ODA2IiwiNTQyNzg4ODA3M2FjYjJhYmQyZDhkYWViZjdm
N2Y3ZmVlMGI2ZmRiODYzZTA4MjE0YjM1ODA2IiwiMmQwMDRiNTQyNzg4ODA3M2FjYjJhYmQyZDhk
YWViZmVlMGI2ZmRiODYzZTA4MjE0YjM1ODA2N2YzYjA4IiwiMmQwMDRiNTQyNzg4ODA3M2FjYjJh
YmQyZDhkYWViZjdmN2Y3ZmVlMGI2ZmRiODYzZTA4MjE0YjM1ODA2N2YzYjA4IikubWFwKFhfKSxs
Yj1yYihzYiksaGI9bmV3IEFycmF5KDMpLmNvbmNhdCgiZWY4YTYyZjdmN2Y3NjdhOWNmIiwiY2Ew
MDIwZjRhNTgyOTJjNWRlMDU3MWIwIiwiY2EwMDIwZjRhNTgyZjdmN2Y3OTJjNWRlMDU3MWIwIiwi
YjIxODJiZWY4YTYyZmRkYmM3ZDFlNWYwNjdhOWNmMjE2NmFjIiwiYjIxODJiZWY4YTYyZmRkYmM3
ZjdmN2Y3ZDFlNWYwNjdhOWNmMjE2NmFjIiwiYjIxODJiZDY2MDRkZjRhNTgyZmRkYmM3ZDFlNWYw
OTJjNWRlNDM5M2MzMjE2NmFjIiwiYjIxODJiZDY2MDRkZjRhNTgyZmRkYmM3ZjdmN2Y3ZDFlNWYw
OTJjNWRlNDM5M2MzMjE2NmFjIiwiNjcwMDFmYjIxODJiZDY2MDRkZjRhNTgyZmRkYmM3ZDFlNWYw
OTJjNWRlNDM5M2MzMjE2NmFjMDUzMDYxIiwiNjcwMDFmYjIxODJiZDY2MDRkZjRhNTgyZmRkYmM3
ZjdmN2Y3ZDFlNWYwOTJjNWRlNDM5M2MzMjE2NmFjMDUzMDYxIikubWFwKFhfKSxkYj1yYihoYiks
cGI9bmV3IEFycmF5KDMpLmNvbmNhdCgiZWY4YTYyZmZmZmZmOTk5OTk5IiwiY2EwMDIwZjRhNTgy
YmFiYWJhNDA0MDQwIiwiY2EwMDIwZjRhNTgyZmZmZmZmYmFiYWJhNDA0MDQwIiwiYjIxODJiZWY4
YTYyZmRkYmM3ZTBlMGUwOTk5OTk5NGQ0ZDRkIiwiYjIxODJiZWY4YTYyZmRkYmM3ZmZmZmZmZTBl
MGUwOTk5OTk5NGQ0ZDRkIiwiYjIxODJiZDY2MDRkZjRhNTgyZmRkYmM3ZTBlMGUwYmFiYWJhODc4
Nzg3NGQ0ZDRkIiwiYjIxODJiZDY2MDRkZjRhNTgyZmRkYmM3ZmZmZmZmZTBlMGUwYmFiYWJhODc4
Nzg3NGQ0ZDRkIiwiNjcwMDFmYjIxODJiZDY2MDRkZjRhNTgyZmRkYmM3ZTBlMGUwYmFiYWJhODc4
Nzg3NGQ0ZDRkMWExYTFhIiwiNjcwMDFmYjIxODJiZDY2MDRkZjRhNTgyZmRkYmM3ZmZmZmZmZTBl
MGUwYmFiYWJhODc4Nzg3NGQ0ZDRkMWExYTFhIikubWFwKFhfKSxnYj1yYihwYikseWI9bmV3IEFy
cmF5KDMpLmNvbmNhdCgiZmM4ZDU5ZmZmZmJmOTFiZmRiIiwiZDcxOTFjZmRhZTYxYWJkOWU5MmM3
YmI2IiwiZDcxOTFjZmRhZTYxZmZmZmJmYWJkOWU5MmM3YmI2IiwiZDczMDI3ZmM4ZDU5ZmVlMDkw
ZTBmM2Y4OTFiZmRiNDU3NWI0IiwiZDczMDI3ZmM4ZDU5ZmVlMDkwZmZmZmJmZTBmM2Y4OTFiZmRi
NDU3NWI0IiwiZDczMDI3ZjQ2ZDQzZmRhZTYxZmVlMDkwZTBmM2Y4YWJkOWU5NzRhZGQxNDU3NWI0
IiwiZDczMDI3ZjQ2ZDQzZmRhZTYxZmVlMDkwZmZmZmJmZTBmM2Y4YWJkOWU5NzRhZGQxNDU3NWI0
IiwiYTUwMDI2ZDczMDI3ZjQ2ZDQzZmRhZTYxZmVlMDkwZTBmM2Y4YWJkOWU5NzRhZGQxNDU3NWI0
MzEzNjk1IiwiYTUwMDI2ZDczMDI3ZjQ2ZDQzZmRhZTYxZmVlMDkwZmZmZmJmZTBmM2Y4YWJkOWU5
NzRhZGQxNDU3NWI0MzEzNjk1IikubWFwKFhfKSx2Yj1yYih5YiksX2I9bmV3IEFycmF5KDMpLmNv
bmNhdCgiZmM4ZDU5ZmZmZmJmOTFjZjYwIiwiZDcxOTFjZmRhZTYxYTZkOTZhMWE5NjQxIiwiZDcx
OTFjZmRhZTYxZmZmZmJmYTZkOTZhMWE5NjQxIiwiZDczMDI3ZmM4ZDU5ZmVlMDhiZDllZjhiOTFj
ZjYwMWE5ODUwIiwiZDczMDI3ZmM4ZDU5ZmVlMDhiZmZmZmJmZDllZjhiOTFjZjYwMWE5ODUwIiwi
ZDczMDI3ZjQ2ZDQzZmRhZTYxZmVlMDhiZDllZjhiYTZkOTZhNjZiZDYzMWE5ODUwIiwiZDczMDI3
ZjQ2ZDQzZmRhZTYxZmVlMDhiZmZmZmJmZDllZjhiYTZkOTZhNjZiZDYzMWE5ODUwIiwiYTUwMDI2
ZDczMDI3ZjQ2ZDQzZmRhZTYxZmVlMDhiZDllZjhiYTZkOTZhNjZiZDYzMWE5ODUwMDA2ODM3Iiwi
YTUwMDI2ZDczMDI3ZjQ2ZDQzZmRhZTYxZmVlMDhiZmZmZmJmZDllZjhiYTZkOTZhNjZiZDYzMWE5
ODUwMDA2ODM3IikubWFwKFhfKSxiYj1yYihfYiksbWI9bmV3IEFycmF5KDMpLmNvbmNhdCgiZmM4
ZDU5ZmZmZmJmOTlkNTk0IiwiZDcxOTFjZmRhZTYxYWJkZGE0MmI4M2JhIiwiZDcxOTFjZmRhZTYx
ZmZmZmJmYWJkZGE0MmI4M2JhIiwiZDUzZTRmZmM4ZDU5ZmVlMDhiZTZmNTk4OTlkNTk0MzI4OGJk
IiwiZDUzZTRmZmM4ZDU5ZmVlMDhiZmZmZmJmZTZmNTk4OTlkNTk0MzI4OGJkIiwiZDUzZTRmZjQ2
ZDQzZmRhZTYxZmVlMDhiZTZmNTk4YWJkZGE0NjZjMmE1MzI4OGJkIiwiZDUzZTRmZjQ2ZDQzZmRh
ZTYxZmVlMDhiZmZmZmJmZTZmNTk4YWJkZGE0NjZjMmE1MzI4OGJkIiwiOWUwMTQyZDUzZTRmZjQ2
ZDQzZmRhZTYxZmVlMDhiZTZmNTk4YWJkZGE0NjZjMmE1MzI4OGJkNWU0ZmEyIiwiOWUwMTQyZDUz
ZTRmZjQ2ZDQzZmRhZTYxZmVlMDhiZmZmZmJmZTZmNTk4YWJkZGE0NjZjMmE1MzI4OGJkNWU0ZmEy
IikubWFwKFhfKSx4Yj1yYihtYiksd2I9bmV3IEFycmF5KDMpLmNvbmNhdCgiZTVmNWY5OTlkOGM5
MmNhMjVmIiwiZWRmOGZiYjJlMmUyNjZjMmE0MjM4YjQ1IiwiZWRmOGZiYjJlMmUyNjZjMmE0MmNh
MjVmMDA2ZDJjIiwiZWRmOGZiY2NlY2U2OTlkOGM5NjZjMmE0MmNhMjVmMDA2ZDJjIiwiZWRmOGZi
Y2NlY2U2OTlkOGM5NjZjMmE0NDFhZTc2MjM4YjQ1MDA1ODI0IiwiZjdmY2ZkZTVmNWY5Y2NlY2U2
OTlkOGM5NjZjMmE0NDFhZTc2MjM4YjQ1MDA1ODI0IiwiZjdmY2ZkZTVmNWY5Y2NlY2U2OTlkOGM5
NjZjMmE0NDFhZTc2MjM4YjQ1MDA2ZDJjMDA0NDFiIikubWFwKFhfKSxNYj1yYih3YiksVGI9bmV3
IEFycmF5KDMpLmNvbmNhdCgiZTBlY2Y0OWViY2RhODg1NmE3IiwiZWRmOGZiYjNjZGUzOGM5NmM2
ODg0MTlkIiwiZWRmOGZiYjNjZGUzOGM5NmM2ODg1NmE3ODEwZjdjIiwiZWRmOGZiYmZkM2U2OWVi
Y2RhOGM5NmM2ODg1NmE3ODEwZjdjIiwiZWRmOGZiYmZkM2U2OWViY2RhOGM5NmM2OGM2YmIxODg0
MTlkNmUwMTZiIiwiZjdmY2ZkZTBlY2Y0YmZkM2U2OWViY2RhOGM5NmM2OGM2YmIxODg0MTlkNmUw
MTZiIiwiZjdmY2ZkZTBlY2Y0YmZkM2U2OWViY2RhOGM5NmM2OGM2YmIxODg0MTlkODEwZjdjNGQw
MDRiIikubWFwKFhfKSxBYj1yYihUYiksU2I9bmV3IEFycmF5KDMpLmNvbmNhdCgiZTBmM2RiYThk
ZGI1NDNhMmNhIiwiZjBmOWU4YmFlNGJjN2JjY2M0MmI4Y2JlIiwiZjBmOWU4YmFlNGJjN2JjY2M0
NDNhMmNhMDg2OGFjIiwiZjBmOWU4Y2NlYmM1YThkZGI1N2JjY2M0NDNhMmNhMDg2OGFjIiwiZjBm
OWU4Y2NlYmM1YThkZGI1N2JjY2M0NGViM2QzMmI4Y2JlMDg1ODllIiwiZjdmY2YwZTBmM2RiY2Nl
YmM1YThkZGI1N2JjY2M0NGViM2QzMmI4Y2JlMDg1ODllIiwiZjdmY2YwZTBmM2RiY2NlYmM1YThk
ZGI1N2JjY2M0NGViM2QzMmI4Y2JlMDg2OGFjMDg0MDgxIikubWFwKFhfKSxFYj1yYihTYiksTmI9
bmV3IEFycmF5KDMpLmNvbmNhdCgiZmVlOGM4ZmRiYjg0ZTM0YTMzIiwiZmVmMGQ5ZmRjYzhhZmM4
ZDU5ZDczMDFmIiwiZmVmMGQ5ZmRjYzhhZmM4ZDU5ZTM0YTMzYjMwMDAwIiwiZmVmMGQ5ZmRkNDll
ZmRiYjg0ZmM4ZDU5ZTM0YTMzYjMwMDAwIiwiZmVmMGQ5ZmRkNDllZmRiYjg0ZmM4ZDU5ZWY2NTQ4
ZDczMDFmOTkwMDAwIiwiZmZmN2VjZmVlOGM4ZmRkNDllZmRiYjg0ZmM4ZDU5ZWY2NTQ4ZDczMDFm
OTkwMDAwIiwiZmZmN2VjZmVlOGM4ZmRkNDllZmRiYjg0ZmM4ZDU5ZWY2NTQ4ZDczMDFmYjMwMDAw
N2YwMDAwIikubWFwKFhfKSxrYj1yYihOYiksQ2I9bmV3IEFycmF5KDMpLmNvbmNhdCgiZWNlMmYw
YTZiZGRiMWM5MDk5IiwiZjZlZmY3YmRjOWUxNjdhOWNmMDI4MThhIiwiZjZlZmY3YmRjOWUxNjdh
OWNmMWM5MDk5MDE2YzU5IiwiZjZlZmY3ZDBkMWU2YTZiZGRiNjdhOWNmMWM5MDk5MDE2YzU5Iiwi
ZjZlZmY3ZDBkMWU2YTZiZGRiNjdhOWNmMzY5MGMwMDI4MThhMDE2NDUwIiwiZmZmN2ZiZWNlMmYw
ZDBkMWU2YTZiZGRiNjdhOWNmMzY5MGMwMDI4MThhMDE2NDUwIiwiZmZmN2ZiZWNlMmYwZDBkMWU2
YTZiZGRiNjdhOWNmMzY5MGMwMDI4MThhMDE2YzU5MDE0NjM2IikubWFwKFhfKSxQYj1yYihDYiks
emI9bmV3IEFycmF5KDMpLmNvbmNhdCgiZWNlN2YyYTZiZGRiMmI4Y2JlIiwiZjFlZWY2YmRjOWUx
NzRhOWNmMDU3MGIwIiwiZjFlZWY2YmRjOWUxNzRhOWNmMmI4Y2JlMDQ1YThkIiwiZjFlZWY2ZDBk
MWU2YTZiZGRiNzRhOWNmMmI4Y2JlMDQ1YThkIiwiZjFlZWY2ZDBkMWU2YTZiZGRiNzRhOWNmMzY5
MGMwMDU3MGIwMDM0ZTdiIiwiZmZmN2ZiZWNlN2YyZDBkMWU2YTZiZGRiNzRhOWNmMzY5MGMwMDU3
MGIwMDM0ZTdiIiwiZmZmN2ZiZWNlN2YyZDBkMWU2YTZiZGRiNzRhOWNmMzY5MGMwMDU3MGIwMDQ1
YThkMDIzODU4IikubWFwKFhfKSwkYj1yYih6YiksRGI9bmV3IEFycmF5KDMpLmNvbmNhdCgiZTdl
MWVmYzk5NGM3ZGQxYzc3IiwiZjFlZWY2ZDdiNWQ4ZGY2NWIwY2UxMjU2IiwiZjFlZWY2ZDdiNWQ4
ZGY2NWIwZGQxYzc3OTgwMDQzIiwiZjFlZWY2ZDRiOWRhYzk5NGM3ZGY2NWIwZGQxYzc3OTgwMDQz
IiwiZjFlZWY2ZDRiOWRhYzk5NGM3ZGY2NWIwZTcyOThhY2UxMjU2OTEwMDNmIiwiZjdmNGY5ZTdl
MWVmZDRiOWRhYzk5NGM3ZGY2NWIwZTcyOThhY2UxMjU2OTEwMDNmIiwiZjdmNGY5ZTdlMWVmZDRi
OWRhYzk5NGM3ZGY2NWIwZTcyOThhY2UxMjU2OTgwMDQzNjcwMDFmIikubWFwKFhfKSxSYj1yYihE
YiksRmI9bmV3IEFycmF5KDMpLmNvbmNhdCgiZmRlMGRkZmE5ZmI1YzUxYjhhIiwiZmVlYmUyZmJi
NGI5Zjc2OGExYWUwMTdlIiwiZmVlYmUyZmJiNGI5Zjc2OGExYzUxYjhhN2EwMTc3IiwiZmVlYmUy
ZmNjNWMwZmE5ZmI1Zjc2OGExYzUxYjhhN2EwMTc3IiwiZmVlYmUyZmNjNWMwZmE5ZmI1Zjc2OGEx
ZGQzNDk3YWUwMTdlN2EwMTc3IiwiZmZmN2YzZmRlMGRkZmNjNWMwZmE5ZmI1Zjc2OGExZGQzNDk3
YWUwMTdlN2EwMTc3IiwiZmZmN2YzZmRlMGRkZmNjNWMwZmE5ZmI1Zjc2OGExZGQzNDk3YWUwMTdl
N2EwMTc3NDkwMDZhIikubWFwKFhfKSxxYj1yYihGYiksVWI9bmV3IEFycmF5KDMpLmNvbmNhdCgi
ZWRmOGIxN2ZjZGJiMmM3ZmI4IiwiZmZmZmNjYTFkYWI0NDFiNmM0MjI1ZWE4IiwiZmZmZmNjYTFk
YWI0NDFiNmM0MmM3ZmI4MjUzNDk0IiwiZmZmZmNjYzdlOWI0N2ZjZGJiNDFiNmM0MmM3ZmI4MjUz
NDk0IiwiZmZmZmNjYzdlOWI0N2ZjZGJiNDFiNmM0MWQ5MWMwMjI1ZWE4MGMyYzg0IiwiZmZmZmQ5
ZWRmOGIxYzdlOWI0N2ZjZGJiNDFiNmM0MWQ5MWMwMjI1ZWE4MGMyYzg0IiwiZmZmZmQ5ZWRmOGIx
YzdlOWI0N2ZjZGJiNDFiNmM0MWQ5MWMwMjI1ZWE4MjUzNDk0MDgxZDU4IikubWFwKFhfKSxJYj1y
YihVYiksT2I9bmV3IEFycmF5KDMpLmNvbmNhdCgiZjdmY2I5YWRkZDhlMzFhMzU0IiwiZmZmZmNj
YzJlNjk5NzhjNjc5MjM4NDQzIiwiZmZmZmNjYzJlNjk5NzhjNjc5MzFhMzU0MDA2ODM3IiwiZmZm
ZmNjZDlmMGEzYWRkZDhlNzhjNjc5MzFhMzU0MDA2ODM3IiwiZmZmZmNjZDlmMGEzYWRkZDhlNzhj
Njc5NDFhYjVkMjM4NDQzMDA1YTMyIiwiZmZmZmU1ZjdmY2I5ZDlmMGEzYWRkZDhlNzhjNjc5NDFh
YjVkMjM4NDQzMDA1YTMyIiwiZmZmZmU1ZjdmY2I5ZDlmMGEzYWRkZDhlNzhjNjc5NDFhYjVkMjM4
NDQzMDA2ODM3MDA0NTI5IikubWFwKFhfKSxCYj1yYihPYiksWWI9bmV3IEFycmF5KDMpLmNvbmNh
dCgiZmZmN2JjZmVjNDRmZDk1ZjBlIiwiZmZmZmQ0ZmVkOThlZmU5OTI5Y2M0YzAyIiwiZmZmZmQ0
ZmVkOThlZmU5OTI5ZDk1ZjBlOTkzNDA0IiwiZmZmZmQ0ZmVlMzkxZmVjNDRmZmU5OTI5ZDk1ZjBl
OTkzNDA0IiwiZmZmZmQ0ZmVlMzkxZmVjNDRmZmU5OTI5ZWM3MDE0Y2M0YzAyOGMyZDA0IiwiZmZm
ZmU1ZmZmN2JjZmVlMzkxZmVjNDRmZmU5OTI5ZWM3MDE0Y2M0YzAyOGMyZDA0IiwiZmZmZmU1ZmZm
N2JjZmVlMzkxZmVjNDRmZmU5OTI5ZWM3MDE0Y2M0YzAyOTkzNDA0NjYyNTA2IikubWFwKFhfKSxM
Yj1yYihZYiksamI9bmV3IEFycmF5KDMpLmNvbmNhdCgiZmZlZGEwZmViMjRjZjAzYjIwIiwiZmZm
ZmIyZmVjYzVjZmQ4ZDNjZTMxYTFjIiwiZmZmZmIyZmVjYzVjZmQ4ZDNjZjAzYjIwYmQwMDI2Iiwi
ZmZmZmIyZmVkOTc2ZmViMjRjZmQ4ZDNjZjAzYjIwYmQwMDI2IiwiZmZmZmIyZmVkOTc2ZmViMjRj
ZmQ4ZDNjZmM0ZTJhZTMxYTFjYjEwMDI2IiwiZmZmZmNjZmZlZGEwZmVkOTc2ZmViMjRjZmQ4ZDNj
ZmM0ZTJhZTMxYTFjYjEwMDI2IiwiZmZmZmNjZmZlZGEwZmVkOTc2ZmViMjRjZmQ4ZDNjZmM0ZTJh
ZTMxYTFjYmQwMDI2ODAwMDI2IikubWFwKFhfKSxIYj1yYihqYiksWGI9bmV3IEFycmF5KDMpLmNv
bmNhdCgiZGVlYmY3OWVjYWUxMzE4MmJkIiwiZWZmM2ZmYmRkN2U3NmJhZWQ2MjE3MWI1IiwiZWZm
M2ZmYmRkN2U3NmJhZWQ2MzE4MmJkMDg1MTljIiwiZWZmM2ZmYzZkYmVmOWVjYWUxNmJhZWQ2MzE4
MmJkMDg1MTljIiwiZWZmM2ZmYzZkYmVmOWVjYWUxNmJhZWQ2NDI5MmM2MjE3MWI1MDg0NTk0Iiwi
ZjdmYmZmZGVlYmY3YzZkYmVmOWVjYWUxNmJhZWQ2NDI5MmM2MjE3MWI1MDg0NTk0IiwiZjdmYmZm
ZGVlYmY3YzZkYmVmOWVjYWUxNmJhZWQ2NDI5MmM2MjE3MWI1MDg1MTljMDgzMDZiIikubWFwKFhf
KSxHYj1yYihYYiksVmI9bmV3IEFycmF5KDMpLmNvbmNhdCgiZTVmNWUwYTFkOTliMzFhMzU0Iiwi
ZWRmOGU5YmFlNGIzNzRjNDc2MjM4YjQ1IiwiZWRmOGU5YmFlNGIzNzRjNDc2MzFhMzU0MDA2ZDJj
IiwiZWRmOGU5YzdlOWMwYTFkOTliNzRjNDc2MzFhMzU0MDA2ZDJjIiwiZWRmOGU5YzdlOWMwYTFk
OTliNzRjNDc2NDFhYjVkMjM4YjQ1MDA1YTMyIiwiZjdmY2Y1ZTVmNWUwYzdlOWMwYTFkOTliNzRj
NDc2NDFhYjVkMjM4YjQ1MDA1YTMyIiwiZjdmY2Y1ZTVmNWUwYzdlOWMwYTFkOTliNzRjNDc2NDFh
YjVkMjM4YjQ1MDA2ZDJjMDA0NDFiIikubWFwKFhfKSxXYj1yYihWYiksWmI9bmV3IEFycmF5KDMp
LmNvbmNhdCgiZjBmMGYwYmRiZGJkNjM2MzYzIiwiZjdmN2Y3Y2NjY2NjOTY5Njk2NTI1MjUyIiwi
ZjdmN2Y3Y2NjY2NjOTY5Njk2NjM2MzYzMjUyNTI1IiwiZjdmN2Y3ZDlkOWQ5YmRiZGJkOTY5Njk2
NjM2MzYzMjUyNTI1IiwiZjdmN2Y3ZDlkOWQ5YmRiZGJkOTY5Njk2NzM3MzczNTI1MjUyMjUyNTI1
IiwiZmZmZmZmZjBmMGYwZDlkOWQ5YmRiZGJkOTY5Njk2NzM3MzczNTI1MjUyMjUyNTI1IiwiZmZm
ZmZmZjBmMGYwZDlkOWQ5YmRiZGJkOTY5Njk2NzM3MzczNTI1MjUyMjUyNTI1MDAwMDAwIikubWFw
KFhfKSxLYj1yYihaYiksUWI9bmV3IEFycmF5KDMpLmNvbmNhdCgiZWZlZGY1YmNiZGRjNzU2YmIx
IiwiZjJmMGY3Y2JjOWUyOWU5YWM4NmE1MWEzIiwiZjJmMGY3Y2JjOWUyOWU5YWM4NzU2YmIxNTQy
NzhmIiwiZjJmMGY3ZGFkYWViYmNiZGRjOWU5YWM4NzU2YmIxNTQyNzhmIiwiZjJmMGY3ZGFkYWVi
YmNiZGRjOWU5YWM4ODA3ZGJhNmE1MWEzNGExNDg2IiwiZmNmYmZkZWZlZGY1ZGFkYWViYmNiZGRj
OWU5YWM4ODA3ZGJhNmE1MWEzNGExNDg2IiwiZmNmYmZkZWZlZGY1ZGFkYWViYmNiZGRjOWU5YWM4
ODA3ZGJhNmE1MWEzNTQyNzhmM2YwMDdkIikubWFwKFhfKSxKYj1yYihRYiksdG09bmV3IEFycmF5
KDMpLmNvbmNhdCgiZmVlMGQyZmM5MjcyZGUyZDI2IiwiZmVlNWQ5ZmNhZTkxZmI2YTRhY2IxODFk
IiwiZmVlNWQ5ZmNhZTkxZmI2YTRhZGUyZDI2YTUwZjE1IiwiZmVlNWQ5ZmNiYmExZmM5MjcyZmI2
YTRhZGUyZDI2YTUwZjE1IiwiZmVlNWQ5ZmNiYmExZmM5MjcyZmI2YTRhZWYzYjJjY2IxODFkOTkw
MDBkIiwiZmZmNWYwZmVlMGQyZmNiYmExZmM5MjcyZmI2YTRhZWYzYjJjY2IxODFkOTkwMDBkIiwi
ZmZmNWYwZmVlMGQyZmNiYmExZmM5MjcyZmI2YTRhZWYzYjJjY2IxODFkYTUwZjE1NjcwMDBkIiku
bWFwKFhfKSxubT1yYih0bSksZW09bmV3IEFycmF5KDMpLmNvbmNhdCgiZmVlNmNlZmRhZTZiZTY1
NTBkIiwiZmVlZGRlZmRiZTg1ZmQ4ZDNjZDk0NzAxIiwiZmVlZGRlZmRiZTg1ZmQ4ZDNjZTY1NTBk
YTYzNjAzIiwiZmVlZGRlZmRkMGEyZmRhZTZiZmQ4ZDNjZTY1NTBkYTYzNjAzIiwiZmVlZGRlZmRk
MGEyZmRhZTZiZmQ4ZDNjZjE2OTEzZDk0ODAxOGMyZDA0IiwiZmZmNWViZmVlNmNlZmRkMGEyZmRh
ZTZiZmQ4ZDNjZjE2OTEzZDk0ODAxOGMyZDA0IiwiZmZmNWViZmVlNmNlZmRkMGEyZmRhZTZiZmQ4
ZDNjZjE2OTEzZDk0ODAxYTYzNjAzN2YyNzA0IikubWFwKFhfKSxybT1yYihlbSk7dmFyIGltPWRp
KEFyKDMwMCwuNSwwKSxBcigtMjQwLC41LDEpKSxvbT1kaShBcigtMTAwLC43NSwuMzUpLEFyKDgw
LDEuNSwuOCkpLGFtPWRpKEFyKDI2MCwuNzUsLjM1KSxBcig4MCwxLjUsLjgpKSx1bT1BcigpO3Zh
ciBjbT1xZSgpLGZtPU1hdGguUEkvMyxzbT0yKk1hdGguUEkvMztmdW5jdGlvbiBsbSh0KXt2YXIg
bj10Lmxlbmd0aDtyZXR1cm4gZnVuY3Rpb24oZSl7cmV0dXJuIHRbTWF0aC5tYXgoMCxNYXRoLm1p
bihuLTEsTWF0aC5mbG9vcihlKm4pKSldfX12YXIgaG09bG0oWF8oIjQ0MDE1NDQ0MDI1NjQ1MDQ1
NzQ1MDU1OTQ2MDc1YTQ2MDg1YzQ2MGE1ZDQ2MGI1ZTQ3MGQ2MDQ3MGU2MTQ3MTA2MzQ3MTE2NDQ3
MTM2NTQ4MTQ2NzQ4MTY2ODQ4MTc2OTQ4MTg2YTQ4MWE2YzQ4MWI2ZDQ4MWM2ZTQ4MWQ2ZjQ4MWY3
MDQ4MjA3MTQ4MjE3MzQ4MjM3NDQ4MjQ3NTQ4MjU3NjQ4MjY3NzQ4Mjg3ODQ4Mjk3OTQ3MmE3YTQ3
MmM3YTQ3MmQ3YjQ3MmU3YzQ3MmY3ZDQ2MzA3ZTQ2MzI3ZTQ2MzM3ZjQ2MzQ4MDQ1MzU4MTQ1Mzc4
MTQ1Mzg4MjQ0Mzk4MzQ0M2E4MzQ0M2I4NDQzM2Q4NDQzM2U4NTQyM2Y4NTQyNDA4NjQyNDE4NjQx
NDI4NzQxNDQ4NzQwNDU4ODQwNDY4ODNmNDc4ODNmNDg4OTNlNDk4OTNlNGE4OTNlNGM4YTNkNGQ4
YTNkNGU4YTNjNGY4YTNjNTA4YjNiNTE4YjNiNTI4YjNhNTM4YjNhNTQ4YzM5NTU4YzM5NTY4YzM4
NTg4YzM4NTk4YzM3NWE4YzM3NWI4ZDM2NWM4ZDM2NWQ4ZDM1NWU4ZDM1NWY4ZDM0NjA4ZDM0NjE4
ZDMzNjI4ZDMzNjM4ZDMyNjQ4ZTMyNjU4ZTMxNjY4ZTMxNjc4ZTMxNjg4ZTMwNjk4ZTMwNmE4ZTJm
NmI4ZTJmNmM4ZTJlNmQ4ZTJlNmU4ZTJlNmY4ZTJkNzA4ZTJkNzE4ZTJjNzE4ZTJjNzI4ZTJjNzM4
ZTJiNzQ4ZTJiNzU4ZTJhNzY4ZTJhNzc4ZTJhNzg4ZTI5Nzk4ZTI5N2E4ZTI5N2I4ZTI4N2M4ZTI4
N2Q4ZTI3N2U4ZTI3N2Y4ZTI3ODA4ZTI2ODE4ZTI2ODI4ZTI2ODI4ZTI1ODM4ZTI1ODQ4ZTI1ODU4
ZTI0ODY4ZTI0ODc4ZTIzODg4ZTIzODk4ZTIzOGE4ZDIyOGI4ZDIyOGM4ZDIyOGQ4ZDIxOGU4ZDIx
OGY4ZDIxOTA4ZDIxOTE4YzIwOTI4YzIwOTI4YzIwOTM4YzFmOTQ4YzFmOTU4YjFmOTY4YjFmOTc4
YjFmOTg4YjFmOTk4YTFmOWE4YTFlOWI4YTFlOWM4OTFlOWQ4OTFmOWU4OTFmOWY4ODFmYTA4ODFm
YTE4ODFmYTE4NzFmYTI4NzIwYTM4NjIwYTQ4NjIxYTU4NTIxYTY4NTIyYTc4NTIyYTg4NDIzYTk4
MzI0YWE4MzI1YWI4MjI1YWM4MjI2YWQ4MTI3YWQ4MTI4YWU4MDI5YWY3ZjJhYjA3ZjJjYjE3ZTJk
YjI3ZDJlYjM3YzJmYjQ3YzMxYjU3YjMyYjY3YTM0YjY3OTM1Yjc3OTM3Yjg3ODM4Yjk3NzNhYmE3
NjNiYmI3NTNkYmM3NDNmYmM3MzQwYmQ3MjQyYmU3MTQ0YmY3MDQ2YzA2ZjQ4YzE2ZTRhYzE2ZDRj
YzI2YzRlYzM2YjUwYzQ2YTUyYzU2OTU0YzU2ODU2YzY2NzU4Yzc2NTVhYzg2NDVjYzg2MzVlYzk2
MjYwY2E2MDYzY2I1ZjY1Y2I1ZTY3Y2M1YzY5Y2Q1YjZjY2Q1YTZlY2U1ODcwY2Y1NzczZDA1Njc1
ZDA1NDc3ZDE1MzdhZDE1MTdjZDI1MDdmZDM0ZTgxZDM0ZDg0ZDQ0Yjg2ZDU0OTg5ZDU0ODhiZDY0
NjhlZDY0NTkwZDc0MzkzZDc0MTk1ZDg0MDk4ZDgzZTliZDkzYzlkZDkzYmEwZGEzOWEyZGEzN2E1
ZGIzNmE4ZGIzNGFhZGMzMmFkZGMzMGIwZGQyZmIyZGQyZGI1ZGUyYmI4ZGUyOWJhZGUyOGJkZGYy
NmMwZGYyNWMyZGYyM2M1ZTAyMWM4ZTAyMGNhZTExZmNkZTExZGQwZTExY2QyZTIxYmQ1ZTIxYWQ4
ZTIxOWRhZTMxOWRkZTMxOGRmZTMxOGUyZTQxOGU1ZTQxOWU3ZTQxOWVhZTUxYWVjZTUxYmVmZTUx
Y2YxZTUxZGY0ZTYxZWY2ZTYyMGY4ZTYyMWZiZTcyM2ZkZTcyNSIpKSxkbT1sbShYXygiMDAwMDA0
MDEwMDA1MDEwMTA2MDEwMTA4MDIwMTA5MDIwMjBiMDIwMjBkMDMwMzBmMDMwMzEyMDQwNDE0MDUw
NDE2MDYwNTE4MDYwNTFhMDcwNjFjMDgwNzFlMDkwNzIwMGEwODIyMGIwOTI0MGMwOTI2MGQwYTI5
MGUwYjJiMTAwYjJkMTEwYzJmMTIwZDMxMTMwZDM0MTQwZTM2MTUwZTM4MTYwZjNiMTgwZjNkMTkx
MDNmMWExMDQyMWMxMDQ0MWQxMTQ3MWUxMTQ5MjAxMTRiMjExMTRlMjIxMTUwMjQxMjUzMjUxMjU1
MjcxMjU4MjkxMTVhMmExMTVjMmMxMTVmMmQxMTYxMmYxMTYzMzExMTY1MzMxMDY3MzQxMDY5MzYx
MDZiMzgxMDZjMzkwZjZlM2IwZjcwM2QwZjcxM2YwZjcyNDAwZjc0NDIwZjc1NDQwZjc2NDUxMDc3
NDcxMDc4NDkxMDc4NGExMDc5NGMxMTdhNGUxMTdiNGYxMjdiNTExMjdjNTIxMzdjNTQxMzdkNTYx
NDdkNTcxNTdlNTkxNTdlNWExNjdlNWMxNjdmNWQxNzdmNWYxODdmNjAxODgwNjIxOTgwNjQxYTgw
NjUxYTgwNjcxYjgwNjgxYzgxNmExYzgxNmIxZDgxNmQxZDgxNmUxZTgxNzAxZjgxNzIxZjgxNzMy
MDgxNzUyMTgxNzYyMTgxNzgyMjgxNzkyMjgyN2IyMzgyN2MyMzgyN2UyNDgyODAyNTgyODEyNTgx
ODMyNjgxODQyNjgxODYyNzgxODgyNzgxODkyODgxOGIyOTgxOGMyOTgxOGUyYTgxOTAyYTgxOTEy
YjgxOTMyYjgwOTQyYzgwOTYyYzgwOTgyZDgwOTkyZDgwOWIyZTdmOWMyZTdmOWUyZjdmYTAyZjdm
YTEzMDdlYTMzMDdlYTUzMTdlYTYzMTdkYTgzMjdkYWEzMzdkYWIzMzdjYWQzNDdjYWUzNDdiYjAz
NTdiYjIzNTdiYjMzNjdhYjUzNjdhYjczNzc5YjgzNzc5YmEzODc4YmMzOTc4YmQzOTc3YmYzYTc3
YzAzYTc2YzIzYjc1YzQzYzc1YzUzYzc0YzczZDczYzgzZTczY2EzZTcyY2MzZjcxY2Q0MDcxY2Y0
MDcwZDA0MTZmZDI0MjZmZDM0MzZlZDU0NDZkZDY0NTZjZDg0NTZjZDk0NjZiZGI0NzZhZGM0ODY5
ZGU0OTY4ZGY0YTY4ZTA0YzY3ZTI0ZDY2ZTM0ZTY1ZTQ0ZjY0ZTU1MDY0ZTc1MjYzZTg1MzYyZTk1
NDYyZWE1NjYxZWI1NzYwZWM1ODYwZWQ1YTVmZWU1YjVlZWY1ZDVlZjA1ZjVlZjE2MDVkZjI2MjVk
ZjI2NDVjZjM2NTVjZjQ2NzVjZjQ2OTVjZjU2YjVjZjY2YzVjZjY2ZTVjZjc3MDVjZjc3MjVjZjg3
NDVjZjg3NjVjZjk3ODVkZjk3OTVkZjk3YjVkZmE3ZDVlZmE3ZjVlZmE4MTVmZmI4MzVmZmI4NTYw
ZmI4NzYxZmM4OTYxZmM4YTYyZmM4YzYzZmM4ZTY0ZmM5MDY1ZmQ5MjY2ZmQ5NDY3ZmQ5NjY4ZmQ5
ODY5ZmQ5YTZhZmQ5YjZiZmU5ZDZjZmU5ZjZkZmVhMTZlZmVhMzZmZmVhNTcxZmVhNzcyZmVhOTcz
ZmVhYTc0ZmVhYzc2ZmVhZTc3ZmViMDc4ZmViMjdhZmViNDdiZmViNjdjZmViNzdlZmViOTdmZmVi
YjgxZmViZDgyZmViZjg0ZmVjMTg1ZmVjMjg3ZmVjNDg4ZmVjNjhhZmVjODhjZmVjYThkZmVjYzhm
ZmVjZDkwZmVjZjkyZmVkMTk0ZmVkMzk1ZmVkNTk3ZmVkNzk5ZmVkODlhZmRkYTljZmRkYzllZmRk
ZWEwZmRlMGExZmRlMmEzZmRlM2E1ZmRlNWE3ZmRlN2E5ZmRlOWFhZmRlYmFjZmNlY2FlZmNlZWIw
ZmNmMGIyZmNmMmI0ZmNmNGI2ZmNmNmI4ZmNmN2I5ZmNmOWJiZmNmYmJkZmNmZGJmIikpLHBtPWxt
KFhfKCIwMDAwMDQwMTAwMDUwMTAxMDYwMTAxMDgwMjAxMGEwMjAyMGMwMjAyMGUwMzAyMTAwNDAz
MTIwNDAzMTQwNTA0MTcwNjA0MTkwNzA1MWIwODA1MWQwOTA2MWYwYTA3MjIwYjA3MjQwYzA4MjYw
ZDA4MjkwZTA5MmIxMDA5MmQxMTBhMzAxMjBhMzIxNDBiMzQxNTBiMzcxNjBiMzkxODBjM2MxOTBj
M2UxYjBjNDExYzBjNDMxZTBjNDUxZjBjNDgyMTBjNGEyMzBjNGMyNDBjNGYyNjBjNTEyODBiNTMy
OTBiNTUyYjBiNTcyZDBiNTkyZjBhNWIzMTBhNWMzMjBhNWUzNDBhNWYzNjA5NjEzODA5NjIzOTA5
NjMzYjA5NjQzZDA5NjUzZTA5NjY0MDBhNjc0MjBhNjg0NDBhNjg0NTBhNjk0NzBiNmE0OTBiNmE0
YTBjNmI0YzBjNmI0ZDBkNmM0ZjBkNmM1MTBlNmM1MjBlNmQ1NDBmNmQ1NTBmNmQ1NzEwNmU1OTEw
NmU1YTExNmU1YzEyNmU1ZDEyNmU1ZjEzNmU2MTEzNmU2MjE0NmU2NDE1NmU2NTE1NmU2NzE2NmU2
OTE2NmU2YTE3NmU2YzE4NmU2ZDE4NmU2ZjE5NmU3MTE5NmU3MjFhNmU3NDFhNmU3NTFiNmU3NzFj
NmQ3ODFjNmQ3YTFkNmQ3YzFkNmQ3ZDFlNmQ3ZjFlNmM4MDFmNmM4MjIwNmM4NDIwNmI4NTIxNmI4
NzIxNmI4ODIyNmE4YTIyNmE4YzIzNjk4ZDIzNjk4ZjI0Njk5MDI1Njg5MjI1Njg5MzI2Njc5NTI2
Njc5NzI3NjY5ODI3NjY5YTI4NjU5YjI5NjQ5ZDI5NjQ5ZjJhNjNhMDJhNjNhMjJiNjJhMzJjNjFh
NTJjNjBhNjJkNjBhODJlNWZhOTJlNWVhYjJmNWVhZDMwNWRhZTMwNWNiMDMxNWJiMTMyNWFiMzMy
NWFiNDMzNTliNjM0NThiNzM1NTdiOTM1NTZiYTM2NTViYzM3NTRiZDM4NTNiZjM5NTJjMDNhNTFj
MTNhNTBjMzNiNGZjNDNjNGVjNjNkNGRjNzNlNGNjODNmNGJjYTQwNGFjYjQxNDljYzQyNDhjZTQz
NDdjZjQ0NDZkMDQ1NDVkMjQ2NDRkMzQ3NDNkNDQ4NDJkNTRhNDFkNzRiM2ZkODRjM2VkOTRkM2Rk
YTRlM2NkYjUwM2JkZDUxM2FkZTUyMzhkZjUzMzdlMDU1MzZlMTU2MzVlMjU3MzRlMzU5MzNlNDVh
MzFlNTVjMzBlNjVkMmZlNzVlMmVlODYwMmRlOTYxMmJlYTYzMmFlYjY0MjllYjY2MjhlYzY3MjZl
ZDY5MjVlZTZhMjRlZjZjMjNlZjZlMjFmMDZmMjBmMTcxMWZmMTczMWRmMjc0MWNmMzc2MWJmMzc4
MTlmNDc5MThmNTdiMTdmNTdkMTVmNjdlMTRmNjgwMTNmNzgyMTJmNzg0MTBmODg1MGZmODg3MGVm
ODg5MGNmOThiMGJmOThjMGFmOThlMDlmYTkwMDhmYTkyMDdmYTk0MDdmYjk2MDZmYjk3MDZmYjk5
MDZmYjliMDZmYjlkMDdmYzlmMDdmY2ExMDhmY2EzMDlmY2E1MGFmY2E2MGNmY2E4MGRmY2FhMGZm
Y2FjMTFmY2FlMTJmY2IwMTRmY2IyMTZmY2I0MThmYmI2MWFmYmI4MWRmYmJhMWZmYmJjMjFmYmJl
MjNmYWMwMjZmYWMyMjhmYWM0MmFmYWM2MmRmOWM3MmZmOWM5MzJmOWNiMzVmOGNkMzdmOGNmM2Fm
N2QxM2RmN2QzNDBmNmQ1NDNmNmQ3NDZmNWQ5NDlmNWRiNGNmNGRkNGZmNGRmNTNmNGUxNTZmM2Uz
NWFmM2U1NWRmMmU2NjFmMmU4NjVmMmVhNjlmMWVjNmRmMWVkNzFmMWVmNzVmMWYxNzlmMmYyN2Rm
MmY0ODJmM2Y1ODZmM2Y2OGFmNGY4OGVmNWY5OTJmNmZhOTZmOGZiOWFmOWZjOWRmYWZkYTFmY2Zm
YTQiKSksZ209bG0oWF8oIjBkMDg4NzEwMDc4ODEzMDc4OTE2MDc4YTE5MDY4YzFiMDY4ZDFkMDY4
ZTIwMDY4ZjIyMDY5MDI0MDY5MTI2MDU5MTI4MDU5MjJhMDU5MzJjMDU5NDJlMDU5NTJmMDU5NjMx
MDU5NzMzMDU5NzM1MDQ5ODM3MDQ5OTM4MDQ5YTNhMDQ5YTNjMDQ5YjNlMDQ5YzNmMDQ5YzQxMDQ5
ZDQzMDM5ZTQ0MDM5ZTQ2MDM5ZjQ4MDM5ZjQ5MDNhMDRiMDNhMTRjMDJhMTRlMDJhMjUwMDJhMjUx
MDJhMzUzMDJhMzU1MDJhNDU2MDFhNDU4MDFhNDU5MDFhNTViMDFhNTVjMDFhNjVlMDFhNjYwMDFh
NjYxMDBhNzYzMDBhNzY0MDBhNzY2MDBhNzY3MDBhODY5MDBhODZhMDBhODZjMDBhODZlMDBhODZm
MDBhODcxMDBhODcyMDFhODc0MDFhODc1MDFhODc3MDFhODc4MDFhODdhMDJhODdiMDJhODdkMDNh
ODdlMDNhODgwMDRhODgxMDRhNzgzMDVhNzg0MDVhNzg2MDZhNjg3MDdhNjg4MDhhNjhhMDlhNThi
MGFhNThkMGJhNThlMGNhNDhmMGRhNDkxMGVhMzkyMGZhMzk0MTBhMjk1MTFhMTk2MTNhMTk4MTRh
MDk5MTU5ZjlhMTY5ZjljMTc5ZTlkMTg5ZDllMTk5ZGEwMWE5Y2ExMWI5YmEyMWQ5YWEzMWU5YWE1
MWY5OWE2MjA5OGE3MjE5N2E4MjI5NmFhMjM5NWFiMjQ5NGFjMjY5NGFkMjc5M2FlMjg5MmIwMjk5
MWIxMmE5MGIyMmI4ZmIzMmM4ZWI0MmU4ZGI1MmY4Y2I2MzA4YmI3MzE4YWI4MzI4OWJhMzM4OGJi
MzQ4OGJjMzU4N2JkMzc4NmJlMzg4NWJmMzk4NGMwM2E4M2MxM2I4MmMyM2M4MWMzM2Q4MGM0M2U3
ZmM1NDA3ZWM2NDE3ZGM3NDI3Y2M4NDM3YmM5NDQ3YWNhNDU3YWNiNDY3OWNjNDc3OGNjNDk3N2Nk
NGE3NmNlNGI3NWNmNGM3NGQwNGQ3M2QxNGU3MmQyNGY3MWQzNTE3MWQ0NTI3MGQ1NTM2ZmQ1NTQ2
ZWQ2NTU2ZGQ3NTY2Y2Q4NTc2YmQ5NTg2YWRhNWE2YWRhNWI2OWRiNWM2OGRjNWQ2N2RkNWU2NmRl
NWY2NWRlNjE2NGRmNjI2M2UwNjM2M2UxNjQ2MmUyNjU2MWUyNjY2MGUzNjg1ZmU0Njk1ZWU1NmE1
ZGU1NmI1ZGU2NmM1Y2U3NmU1YmU3NmY1YWU4NzA1OWU5NzE1OGU5NzI1N2VhNzQ1N2ViNzU1NmVi
NzY1NWVjNzc1NGVkNzk1M2VkN2E1MmVlN2I1MWVmN2M1MWVmN2U1MGYwN2Y0ZmYwODA0ZWYxODE0
ZGYxODM0Y2YyODQ0YmYzODU0YmYzODc0YWY0ODg0OWY0ODk0OGY1OGI0N2Y1OGM0NmY2OGQ0NWY2
OGY0NGY3OTA0NGY3OTE0M2Y3OTM0MmY4OTQ0MWY4OTU0MGY5OTczZmY5OTgzZWY5OWEzZWZhOWIz
ZGZhOWMzY2ZhOWUzYmZiOWYzYWZiYTEzOWZiYTIzOGZjYTMzOGZjYTUzN2ZjYTYzNmZjYTgzNWZj
YTkzNGZkYWIzM2ZkYWMzM2ZkYWUzMmZkYWYzMWZkYjEzMGZkYjIyZmZkYjQyZmZkYjUyZWZlYjcy
ZGZlYjgyY2ZlYmEyY2ZlYmIyYmZlYmQyYWZlYmUyYWZlYzAyOWZkYzIyOWZkYzMyOGZkYzUyN2Zk
YzYyN2ZkYzgyN2ZkY2EyNmZkY2IyNmZjY2QyNWZjY2UyNWZjZDAyNWZjZDIyNWZiZDMyNGZiZDUy
NGZiZDcyNGZhZDgyNGZhZGEyNGY5ZGMyNGY5ZGQyNWY4ZGYyNWY4ZTEyNWY3ZTIyNWY3ZTQyNWY2
ZTYyNmY2ZTgyNmY1ZTkyNmY1ZWIyN2Y0ZWQyN2YzZWUyN2YzZjAyN2YyZjIyN2YxZjQyNmYxZjUy
NWYwZjcyNGYwZjkyMSIpKTtmdW5jdGlvbiB5bSh0KXtyZXR1cm4gZnVuY3Rpb24oKXtyZXR1cm4g
dH19Y29uc3Qgdm09TWF0aC5hYnMsX209TWF0aC5hdGFuMixibT1NYXRoLmNvcyxtbT1NYXRoLm1h
eCx4bT1NYXRoLm1pbix3bT1NYXRoLnNpbixNbT1NYXRoLnNxcnQsVG09MWUtMTIsQW09TWF0aC5Q
SSxTbT1BbS8yLEVtPTIqQW07ZnVuY3Rpb24gTm0odCl7cmV0dXJuIHQ+PTE/U206dDw9LTE/LVNt
Ok1hdGguYXNpbih0KX1mdW5jdGlvbiBrbSh0KXtsZXQgbj0zO3JldHVybiB0LmRpZ2l0cz1mdW5j
dGlvbihlKXtpZighYXJndW1lbnRzLmxlbmd0aClyZXR1cm4gbjtpZihudWxsPT1lKW49bnVsbDtl
bHNle2NvbnN0IHQ9TWF0aC5mbG9vcihlKTtpZighKHQ+PTApKXRocm93IG5ldyBSYW5nZUVycm9y
KGBpbnZhbGlkIGRpZ2l0czogJHtlfWApO249dH1yZXR1cm4gdH0sKCk9Pm5ldyBJYShuKX1mdW5j
dGlvbiBDbSh0KXtyZXR1cm4gdC5pbm5lclJhZGl1c31mdW5jdGlvbiBQbSh0KXtyZXR1cm4gdC5v
dXRlclJhZGl1c31mdW5jdGlvbiB6bSh0KXtyZXR1cm4gdC5zdGFydEFuZ2xlfWZ1bmN0aW9uICRt
KHQpe3JldHVybiB0LmVuZEFuZ2xlfWZ1bmN0aW9uIERtKHQpe3JldHVybiB0JiZ0LnBhZEFuZ2xl
fWZ1bmN0aW9uIFJtKHQsbixlLHIsaSxvLGEpe3ZhciB1PXQtZSxjPW4tcixmPShhP286LW8pL01t
KHUqdStjKmMpLHM9ZipjLGw9LWYqdSxoPXQrcyxkPW4rbCxwPWUrcyxnPXIrbCx5PShoK3ApLzIs
dj0oZCtnKS8yLF89cC1oLGI9Zy1kLG09XypfK2IqYix4PWktbyx3PWgqZy1wKmQsTT0oYjwwPy0x
OjEpKk1tKG1tKDAseCp4Km0tdyp3KSksVD0odypiLV8qTSkvbSxBPSgtdypfLWIqTSkvbSxTPSh3
KmIrXypNKS9tLEU9KC13Kl8rYipNKS9tLE49VC15LGs9QS12LEM9Uy15LFA9RS12O3JldHVybiBO
Kk4rayprPkMqQytQKlAmJihUPVMsQT1FKSx7Y3g6VCxjeTpBLHgwMTotcyx5MDE6LWwseDExOlQq
KGkveC0xKSx5MTE6QSooaS94LTEpfX12YXIgRm09QXJyYXkucHJvdG90eXBlLnNsaWNlO2Z1bmN0
aW9uIHFtKHQpe3JldHVybiJvYmplY3QiPT10eXBlb2YgdCYmImxlbmd0aCJpbiB0P3Q6QXJyYXku
ZnJvbSh0KX1mdW5jdGlvbiBVbSh0KXt0aGlzLl9jb250ZXh0PXR9ZnVuY3Rpb24gSW0odCl7cmV0
dXJuIG5ldyBVbSh0KX1mdW5jdGlvbiBPbSh0KXtyZXR1cm4gdFswXX1mdW5jdGlvbiBCbSh0KXty
ZXR1cm4gdFsxXX1mdW5jdGlvbiBZbSh0LG4pe3ZhciBlPXltKCEwKSxyPW51bGwsaT1JbSxvPW51
bGwsYT1rbSh1KTtmdW5jdGlvbiB1KHUpe3ZhciBjLGYscyxsPSh1PXFtKHUpKS5sZW5ndGgsaD0h
MTtmb3IobnVsbD09ciYmKG89aShzPWEoKSkpLGM9MDtjPD1sOysrYykhKGM8bCYmZShmPXVbY10s
Yyx1KSk9PT1oJiYoKGg9IWgpP28ubGluZVN0YXJ0KCk6by5saW5lRW5kKCkpLGgmJm8ucG9pbnQo
K3QoZixjLHUpLCtuKGYsYyx1KSk7aWYocylyZXR1cm4gbz1udWxsLHMrIiJ8fG51bGx9cmV0dXJu
IHQ9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp2b2lkIDA9PT10P09tOnltKHQpLG49ImZ1bmN0aW9u
Ij09dHlwZW9mIG4/bjp2b2lkIDA9PT1uP0JtOnltKG4pLHUueD1mdW5jdGlvbihuKXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8odD0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOnltKCtuKSx1KTp0fSx1
Lnk9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49ImZ1bmN0aW9uIj09dHlw
ZW9mIHQ/dDp5bSgrdCksdSk6bn0sdS5kZWZpbmVkPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVu
dHMubGVuZ3RoPyhlPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6eW0oISF0KSx1KTplfSx1LmN1cnZl
PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhpPXQsbnVsbCE9ciYmKG89aShy
KSksdSk6aX0sdS5jb250ZXh0PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhu
dWxsPT10P3I9bz1udWxsOm89aShyPXQpLHUpOnJ9LHV9ZnVuY3Rpb24gTG0odCxuLGUpe3ZhciBy
PW51bGwsaT15bSghMCksbz1udWxsLGE9SW0sdT1udWxsLGM9a20oZik7ZnVuY3Rpb24gZihmKXt2
YXIgcyxsLGgsZCxwLGc9KGY9cW0oZikpLmxlbmd0aCx5PSExLHY9bmV3IEFycmF5KGcpLF89bmV3
IEFycmF5KGcpO2ZvcihudWxsPT1vJiYodT1hKHA9YygpKSkscz0wO3M8PWc7KytzKXtpZighKHM8
ZyYmaShkPWZbc10scyxmKSk9PT15KWlmKHk9IXkpbD1zLHUuYXJlYVN0YXJ0KCksdS5saW5lU3Rh
cnQoKTtlbHNle2Zvcih1LmxpbmVFbmQoKSx1LmxpbmVTdGFydCgpLGg9cy0xO2g+PWw7LS1oKXUu
cG9pbnQodltoXSxfW2hdKTt1LmxpbmVFbmQoKSx1LmFyZWFFbmQoKX15JiYodltzXT0rdChkLHMs
ZiksX1tzXT0rbihkLHMsZiksdS5wb2ludChyPytyKGQscyxmKTp2W3NdLGU/K2UoZCxzLGYpOl9b
c10pKX1pZihwKXJldHVybiB1PW51bGwscCsiInx8bnVsbH1mdW5jdGlvbiBzKCl7cmV0dXJuIFlt
KCkuZGVmaW5lZChpKS5jdXJ2ZShhKS5jb250ZXh0KG8pfXJldHVybiB0PSJmdW5jdGlvbiI9PXR5
cGVvZiB0P3Q6dm9pZCAwPT09dD9PbTp5bSgrdCksbj0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOnlt
KHZvaWQgMD09PW4/MDorbiksZT0iZnVuY3Rpb24iPT10eXBlb2YgZT9lOnZvaWQgMD09PWU/Qm06
eW0oK2UpLGYueD1mdW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD0iZnVuY3Rp
b24iPT10eXBlb2Ygbj9uOnltKCtuKSxyPW51bGwsZik6dH0sZi54MD1mdW5jdGlvbihuKXtyZXR1
cm4gYXJndW1lbnRzLmxlbmd0aD8odD0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOnltKCtuKSxmKTp0
fSxmLngxPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhyPW51bGw9PXQ/bnVs
bDoiZnVuY3Rpb24iPT10eXBlb2YgdD90OnltKCt0KSxmKTpyfSxmLnk9ZnVuY3Rpb24odCl7cmV0
dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp5bSgrdCksZT1u
dWxsLGYpOm59LGYueTA9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49ImZ1
bmN0aW9uIj09dHlwZW9mIHQ/dDp5bSgrdCksZik6bn0sZi55MT1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8oZT1udWxsPT10P251bGw6ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp5
bSgrdCksZik6ZX0sZi5saW5lWDA9Zi5saW5lWTA9ZnVuY3Rpb24oKXtyZXR1cm4gcygpLngodCku
eShuKX0sZi5saW5lWTE9ZnVuY3Rpb24oKXtyZXR1cm4gcygpLngodCkueShlKX0sZi5saW5lWDE9
ZnVuY3Rpb24oKXtyZXR1cm4gcygpLngocikueShuKX0sZi5kZWZpbmVkPWZ1bmN0aW9uKHQpe3Jl
dHVybiBhcmd1bWVudHMubGVuZ3RoPyhpPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6eW0oISF0KSxm
KTppfSxmLmN1cnZlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhhPXQsbnVs
bCE9byYmKHU9YShvKSksZik6YX0sZi5jb250ZXh0PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVu
dHMubGVuZ3RoPyhudWxsPT10P289dT1udWxsOnU9YShvPXQpLGYpOm99LGZ9ZnVuY3Rpb24gam0o
dCxuKXtyZXR1cm4gbjx0Py0xOm4+dD8xOm4+PXQ/MDpOYU59ZnVuY3Rpb24gSG0odCl7cmV0dXJu
IHR9VW0ucHJvdG90eXBlPXthcmVhU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLl9saW5lPTB9LGFyZWFF
bmQ6ZnVuY3Rpb24oKXt0aGlzLl9saW5lPU5hTn0sbGluZVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5f
cG9pbnQ9MH0sbGluZUVuZDpmdW5jdGlvbigpeyh0aGlzLl9saW5lfHwwIT09dGhpcy5fbGluZSYm
MT09PXRoaXMuX3BvaW50KSYmdGhpcy5fY29udGV4dC5jbG9zZVBhdGgoKSx0aGlzLl9saW5lPTEt
dGhpcy5fbGluZX0scG9pbnQ6ZnVuY3Rpb24odCxuKXtzd2l0Y2godD0rdCxuPStuLHRoaXMuX3Bv
aW50KXtjYXNlIDA6dGhpcy5fcG9pbnQ9MSx0aGlzLl9saW5lP3RoaXMuX2NvbnRleHQubGluZVRv
KHQsbik6dGhpcy5fY29udGV4dC5tb3ZlVG8odCxuKTticmVhaztjYXNlIDE6dGhpcy5fcG9pbnQ9
MjtkZWZhdWx0OnRoaXMuX2NvbnRleHQubGluZVRvKHQsbil9fX07dmFyIFhtPVZtKEltKTtmdW5j
dGlvbiBHbSh0KXt0aGlzLl9jdXJ2ZT10fWZ1bmN0aW9uIFZtKHQpe2Z1bmN0aW9uIG4obil7cmV0
dXJuIG5ldyBHbSh0KG4pKX1yZXR1cm4gbi5fY3VydmU9dCxufWZ1bmN0aW9uIFdtKHQpe3ZhciBu
PXQuY3VydmU7cmV0dXJuIHQuYW5nbGU9dC54LGRlbGV0ZSB0LngsdC5yYWRpdXM9dC55LGRlbGV0
ZSB0LnksdC5jdXJ2ZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD9uKFZtKHQp
KTpuKCkuX2N1cnZlfSx0fWZ1bmN0aW9uIFptKCl7cmV0dXJuIFdtKFltKCkuY3VydmUoWG0pKX1m
dW5jdGlvbiBLbSgpe3ZhciB0PUxtKCkuY3VydmUoWG0pLG49dC5jdXJ2ZSxlPXQubGluZVgwLHI9
dC5saW5lWDEsaT10LmxpbmVZMCxvPXQubGluZVkxO3JldHVybiB0LmFuZ2xlPXQueCxkZWxldGUg
dC54LHQuc3RhcnRBbmdsZT10LngwLGRlbGV0ZSB0LngwLHQuZW5kQW5nbGU9dC54MSxkZWxldGUg
dC54MSx0LnJhZGl1cz10LnksZGVsZXRlIHQueSx0LmlubmVyUmFkaXVzPXQueTAsZGVsZXRlIHQu
eTAsdC5vdXRlclJhZGl1cz10LnkxLGRlbGV0ZSB0LnkxLHQubGluZVN0YXJ0QW5nbGU9ZnVuY3Rp
b24oKXtyZXR1cm4gV20oZSgpKX0sZGVsZXRlIHQubGluZVgwLHQubGluZUVuZEFuZ2xlPWZ1bmN0
aW9uKCl7cmV0dXJuIFdtKHIoKSl9LGRlbGV0ZSB0LmxpbmVYMSx0LmxpbmVJbm5lclJhZGl1cz1m
dW5jdGlvbigpe3JldHVybiBXbShpKCkpfSxkZWxldGUgdC5saW5lWTAsdC5saW5lT3V0ZXJSYWRp
dXM9ZnVuY3Rpb24oKXtyZXR1cm4gV20obygpKX0sZGVsZXRlIHQubGluZVkxLHQuY3VydmU9ZnVu
Y3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/bihWbSh0KSk6bigpLl9jdXJ2ZX0sdH1m
dW5jdGlvbiBRbSh0LG4pe3JldHVyblsobj0rbikqTWF0aC5jb3ModC09TWF0aC5QSS8yKSxuKk1h
dGguc2luKHQpXX1HbS5wcm90b3R5cGU9e2FyZWFTdGFydDpmdW5jdGlvbigpe3RoaXMuX2N1cnZl
LmFyZWFTdGFydCgpfSxhcmVhRW5kOmZ1bmN0aW9uKCl7dGhpcy5fY3VydmUuYXJlYUVuZCgpfSxs
aW5lU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLl9jdXJ2ZS5saW5lU3RhcnQoKX0sbGluZUVuZDpmdW5j
dGlvbigpe3RoaXMuX2N1cnZlLmxpbmVFbmQoKX0scG9pbnQ6ZnVuY3Rpb24odCxuKXt0aGlzLl9j
dXJ2ZS5wb2ludChuKk1hdGguc2luKHQpLG4qLU1hdGguY29zKHQpKX19O2NsYXNzIEpte2NvbnN0
cnVjdG9yKHQsbil7dGhpcy5fY29udGV4dD10LHRoaXMuX3g9bn1hcmVhU3RhcnQoKXt0aGlzLl9s
aW5lPTB9YXJlYUVuZCgpe3RoaXMuX2xpbmU9TmFOfWxpbmVTdGFydCgpe3RoaXMuX3BvaW50PTB9
bGluZUVuZCgpeyh0aGlzLl9saW5lfHwwIT09dGhpcy5fbGluZSYmMT09PXRoaXMuX3BvaW50KSYm
dGhpcy5fY29udGV4dC5jbG9zZVBhdGgoKSx0aGlzLl9saW5lPTEtdGhpcy5fbGluZX1wb2ludCh0
LG4pe3N3aXRjaCh0PSt0LG49K24sdGhpcy5fcG9pbnQpe2Nhc2UgMDp0aGlzLl9wb2ludD0xLHRo
aXMuX2xpbmU/dGhpcy5fY29udGV4dC5saW5lVG8odCxuKTp0aGlzLl9jb250ZXh0Lm1vdmVUbyh0
LG4pO2JyZWFrO2Nhc2UgMTp0aGlzLl9wb2ludD0yO2RlZmF1bHQ6dGhpcy5feD90aGlzLl9jb250
ZXh0LmJlemllckN1cnZlVG8odGhpcy5feDA9KHRoaXMuX3gwK3QpLzIsdGhpcy5feTAsdGhpcy5f
eDAsbix0LG4pOnRoaXMuX2NvbnRleHQuYmV6aWVyQ3VydmVUbyh0aGlzLl94MCx0aGlzLl95MD0o
dGhpcy5feTArbikvMix0LHRoaXMuX3kwLHQsbil9dGhpcy5feDA9dCx0aGlzLl95MD1ufX1jbGFz
cyB0eHtjb25zdHJ1Y3Rvcih0KXt0aGlzLl9jb250ZXh0PXR9bGluZVN0YXJ0KCl7dGhpcy5fcG9p
bnQ9MH1saW5lRW5kKCl7fXBvaW50KHQsbil7aWYodD0rdCxuPStuLDA9PT10aGlzLl9wb2ludCl0
aGlzLl9wb2ludD0xO2Vsc2V7Y29uc3QgZT1RbSh0aGlzLl94MCx0aGlzLl95MCkscj1RbSh0aGlz
Ll94MCx0aGlzLl95MD0odGhpcy5feTArbikvMiksaT1RbSh0LHRoaXMuX3kwKSxvPVFtKHQsbik7
dGhpcy5fY29udGV4dC5tb3ZlVG8oLi4uZSksdGhpcy5fY29udGV4dC5iZXppZXJDdXJ2ZVRvKC4u
LnIsLi4uaSwuLi5vKX10aGlzLl94MD10LHRoaXMuX3kwPW59fWZ1bmN0aW9uIG54KHQpe3JldHVy
biBuZXcgSm0odCwhMCl9ZnVuY3Rpb24gZXgodCl7cmV0dXJuIG5ldyBKbSh0LCExKX1mdW5jdGlv
biByeCh0KXtyZXR1cm4gbmV3IHR4KHQpfWZ1bmN0aW9uIGl4KHQpe3JldHVybiB0LnNvdXJjZX1m
dW5jdGlvbiBveCh0KXtyZXR1cm4gdC50YXJnZXR9ZnVuY3Rpb24gYXgodCl7bGV0IG49aXgsZT1v
eCxyPU9tLGk9Qm0sbz1udWxsLGE9bnVsbCx1PWttKGMpO2Z1bmN0aW9uIGMoKXtsZXQgYztjb25z
dCBmPUZtLmNhbGwoYXJndW1lbnRzKSxzPW4uYXBwbHkodGhpcyxmKSxsPWUuYXBwbHkodGhpcyxm
KTtpZihudWxsPT1vJiYoYT10KGM9dSgpKSksYS5saW5lU3RhcnQoKSxmWzBdPXMsYS5wb2ludCgr
ci5hcHBseSh0aGlzLGYpLCtpLmFwcGx5KHRoaXMsZikpLGZbMF09bCxhLnBvaW50KCtyLmFwcGx5
KHRoaXMsZiksK2kuYXBwbHkodGhpcyxmKSksYS5saW5lRW5kKCksYylyZXR1cm4gYT1udWxsLGMr
IiJ8fG51bGx9cmV0dXJuIGMuc291cmNlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPyhuPXQsYyk6bn0sYy50YXJnZXQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5n
dGg/KGU9dCxjKTplfSxjLng9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9
ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp5bSgrdCksYyk6cn0sYy55PWZ1bmN0aW9uKHQpe3JldHVy
biBhcmd1bWVudHMubGVuZ3RoPyhpPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6eW0oK3QpLGMpOml9
LGMuY29udGV4dD1mdW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obnVsbD09bj9v
PWE9bnVsbDphPXQobz1uKSxjKTpvfSxjfWNvbnN0IHV4PU1tKDMpO3ZhciBjeD17ZHJhdyh0LG4p
e2NvbnN0IGU9LjU5NDM2Kk1tKG4reG0obi8yOCwuNzUpKSxyPWUvMixpPXIqdXg7dC5tb3ZlVG8o
MCxlKSx0LmxpbmVUbygwLC1lKSx0Lm1vdmVUbygtaSwtciksdC5saW5lVG8oaSxyKSx0Lm1vdmVU
bygtaSxyKSx0LmxpbmVUbyhpLC1yKX19LGZ4PXtkcmF3KHQsbil7Y29uc3QgZT1NbShuL0FtKTt0
Lm1vdmVUbyhlLDApLHQuYXJjKDAsMCxlLDAsRW0pfX0sc3g9e2RyYXcodCxuKXtjb25zdCBlPU1t
KG4vNSkvMjt0Lm1vdmVUbygtMyplLC1lKSx0LmxpbmVUbygtZSwtZSksdC5saW5lVG8oLWUsLTMq
ZSksdC5saW5lVG8oZSwtMyplKSx0LmxpbmVUbyhlLC1lKSx0LmxpbmVUbygzKmUsLWUpLHQubGlu
ZVRvKDMqZSxlKSx0LmxpbmVUbyhlLGUpLHQubGluZVRvKGUsMyplKSx0LmxpbmVUbygtZSwzKmUp
LHQubGluZVRvKC1lLGUpLHQubGluZVRvKC0zKmUsZSksdC5jbG9zZVBhdGgoKX19O2NvbnN0IGx4
PU1tKDEvMyksaHg9MipseDt2YXIgZHg9e2RyYXcodCxuKXtjb25zdCBlPU1tKG4vaHgpLHI9ZSps
eDt0Lm1vdmVUbygwLC1lKSx0LmxpbmVUbyhyLDApLHQubGluZVRvKDAsZSksdC5saW5lVG8oLXIs
MCksdC5jbG9zZVBhdGgoKX19LHB4PXtkcmF3KHQsbil7Y29uc3QgZT0uNjI2MjUqTW0obik7dC5t
b3ZlVG8oMCwtZSksdC5saW5lVG8oZSwwKSx0LmxpbmVUbygwLGUpLHQubGluZVRvKC1lLDApLHQu
Y2xvc2VQYXRoKCl9fSxneD17ZHJhdyh0LG4pe2NvbnN0IGU9Ljg3NTU5Kk1tKG4teG0obi83LDIp
KTt0Lm1vdmVUbygtZSwwKSx0LmxpbmVUbyhlLDApLHQubW92ZVRvKDAsZSksdC5saW5lVG8oMCwt
ZSl9fSx5eD17ZHJhdyh0LG4pe2NvbnN0IGU9TW0obikscj0tZS8yO3QucmVjdChyLHIsZSxlKX19
LHZ4PXtkcmF3KHQsbil7Y29uc3QgZT0uNDQzMSpNbShuKTt0Lm1vdmVUbyhlLGUpLHQubGluZVRv
KGUsLWUpLHQubGluZVRvKC1lLC1lKSx0LmxpbmVUbygtZSxlKSx0LmNsb3NlUGF0aCgpfX07Y29u
c3QgX3g9d20oQW0vMTApL3dtKDcqQW0vMTApLGJ4PXdtKEVtLzEwKSpfeCxteD0tYm0oRW0vMTAp
Kl94O3ZhciB4eD17ZHJhdyh0LG4pe2NvbnN0IGU9TW0oLjg5MDgxMzA5MTUyOTI4NTIqbikscj1i
eCplLGk9bXgqZTt0Lm1vdmVUbygwLC1lKSx0LmxpbmVUbyhyLGkpO2ZvcihsZXQgbj0xO248NTsr
K24pe2NvbnN0IG89RW0qbi81LGE9Ym0obyksdT13bShvKTt0LmxpbmVUbyh1KmUsLWEqZSksdC5s
aW5lVG8oYSpyLXUqaSx1KnIrYSppKX10LmNsb3NlUGF0aCgpfX07Y29uc3Qgd3g9TW0oMyk7dmFy
IE14PXtkcmF3KHQsbil7Y29uc3QgZT0tTW0obi8oMyp3eCkpO3QubW92ZVRvKDAsMiplKSx0Lmxp
bmVUbygtd3gqZSwtZSksdC5saW5lVG8od3gqZSwtZSksdC5jbG9zZVBhdGgoKX19O2NvbnN0IFR4
PU1tKDMpO3ZhciBBeD17ZHJhdyh0LG4pe2NvbnN0IGU9LjY4MjQqTW0obikscj1lLzIsaT1lKlR4
LzI7dC5tb3ZlVG8oMCwtZSksdC5saW5lVG8oaSxyKSx0LmxpbmVUbygtaSxyKSx0LmNsb3NlUGF0
aCgpfX07Y29uc3QgU3g9LS41LEV4PU1tKDMpLzIsTng9MS9NbSgxMiksa3g9MyooTngvMisxKTt2
YXIgQ3g9e2RyYXcodCxuKXtjb25zdCBlPU1tKG4va3gpLHI9ZS8yLGk9ZSpOeCxvPXIsYT1lKk54
K2UsdT0tbyxjPWE7dC5tb3ZlVG8ocixpKSx0LmxpbmVUbyhvLGEpLHQubGluZVRvKHUsYyksdC5s
aW5lVG8oU3gqci1FeCppLEV4KnIrU3gqaSksdC5saW5lVG8oU3gqby1FeCphLEV4Km8rU3gqYSks
dC5saW5lVG8oU3gqdS1FeCpjLEV4KnUrU3gqYyksdC5saW5lVG8oU3gqcitFeCppLFN4KmktRXgq
ciksdC5saW5lVG8oU3gqbytFeCphLFN4KmEtRXgqbyksdC5saW5lVG8oU3gqdStFeCpjLFN4KmMt
RXgqdSksdC5jbG9zZVBhdGgoKX19LFB4PXtkcmF3KHQsbil7Y29uc3QgZT0uNjE4OSpNbShuLXht
KG4vNiwxLjcpKTt0Lm1vdmVUbygtZSwtZSksdC5saW5lVG8oZSxlKSx0Lm1vdmVUbygtZSxlKSx0
LmxpbmVUbyhlLC1lKX19O2NvbnN0IHp4PVtmeCxzeCxkeCx5eCx4eCxNeCxDeF0sJHg9W2Z4LGd4
LFB4LEF4LGN4LHZ4LHB4XTtmdW5jdGlvbiBEeCgpe31mdW5jdGlvbiBSeCh0LG4sZSl7dC5fY29u
dGV4dC5iZXppZXJDdXJ2ZVRvKCgyKnQuX3gwK3QuX3gxKS8zLCgyKnQuX3kwK3QuX3kxKS8zLCh0
Ll94MCsyKnQuX3gxKS8zLCh0Ll95MCsyKnQuX3kxKS8zLCh0Ll94MCs0KnQuX3gxK24pLzYsKHQu
X3kwKzQqdC5feTErZSkvNil9ZnVuY3Rpb24gRngodCl7dGhpcy5fY29udGV4dD10fWZ1bmN0aW9u
IHF4KHQpe3RoaXMuX2NvbnRleHQ9dH1mdW5jdGlvbiBVeCh0KXt0aGlzLl9jb250ZXh0PXR9ZnVu
Y3Rpb24gSXgodCxuKXt0aGlzLl9iYXNpcz1uZXcgRngodCksdGhpcy5fYmV0YT1ufUZ4LnByb3Rv
dHlwZT17YXJlYVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5fbGluZT0wfSxhcmVhRW5kOmZ1bmN0aW9u
KCl7dGhpcy5fbGluZT1OYU59LGxpbmVTdGFydDpmdW5jdGlvbigpe3RoaXMuX3gwPXRoaXMuX3gx
PXRoaXMuX3kwPXRoaXMuX3kxPU5hTix0aGlzLl9wb2ludD0wfSxsaW5lRW5kOmZ1bmN0aW9uKCl7
c3dpdGNoKHRoaXMuX3BvaW50KXtjYXNlIDM6UngodGhpcyx0aGlzLl94MSx0aGlzLl95MSk7Y2Fz
ZSAyOnRoaXMuX2NvbnRleHQubGluZVRvKHRoaXMuX3gxLHRoaXMuX3kxKX0odGhpcy5fbGluZXx8
MCE9PXRoaXMuX2xpbmUmJjE9PT10aGlzLl9wb2ludCkmJnRoaXMuX2NvbnRleHQuY2xvc2VQYXRo
KCksdGhpcy5fbGluZT0xLXRoaXMuX2xpbmV9LHBvaW50OmZ1bmN0aW9uKHQsbil7c3dpdGNoKHQ9
K3Qsbj0rbix0aGlzLl9wb2ludCl7Y2FzZSAwOnRoaXMuX3BvaW50PTEsdGhpcy5fbGluZT90aGlz
Ll9jb250ZXh0LmxpbmVUbyh0LG4pOnRoaXMuX2NvbnRleHQubW92ZVRvKHQsbik7YnJlYWs7Y2Fz
ZSAxOnRoaXMuX3BvaW50PTI7YnJlYWs7Y2FzZSAyOnRoaXMuX3BvaW50PTMsdGhpcy5fY29udGV4
dC5saW5lVG8oKDUqdGhpcy5feDArdGhpcy5feDEpLzYsKDUqdGhpcy5feTArdGhpcy5feTEpLzYp
O2RlZmF1bHQ6UngodGhpcyx0LG4pfXRoaXMuX3gwPXRoaXMuX3gxLHRoaXMuX3gxPXQsdGhpcy5f
eTA9dGhpcy5feTEsdGhpcy5feTE9bn19LHF4LnByb3RvdHlwZT17YXJlYVN0YXJ0OkR4LGFyZWFF
bmQ6RHgsbGluZVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5feDA9dGhpcy5feDE9dGhpcy5feDI9dGhp
cy5feDM9dGhpcy5feDQ9dGhpcy5feTA9dGhpcy5feTE9dGhpcy5feTI9dGhpcy5feTM9dGhpcy5f
eTQ9TmFOLHRoaXMuX3BvaW50PTB9LGxpbmVFbmQ6ZnVuY3Rpb24oKXtzd2l0Y2godGhpcy5fcG9p
bnQpe2Nhc2UgMTp0aGlzLl9jb250ZXh0Lm1vdmVUbyh0aGlzLl94Mix0aGlzLl95MiksdGhpcy5f
Y29udGV4dC5jbG9zZVBhdGgoKTticmVhaztjYXNlIDI6dGhpcy5fY29udGV4dC5tb3ZlVG8oKHRo
aXMuX3gyKzIqdGhpcy5feDMpLzMsKHRoaXMuX3kyKzIqdGhpcy5feTMpLzMpLHRoaXMuX2NvbnRl
eHQubGluZVRvKCh0aGlzLl94MysyKnRoaXMuX3gyKS8zLCh0aGlzLl95MysyKnRoaXMuX3kyKS8z
KSx0aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgpO2JyZWFrO2Nhc2UgMzp0aGlzLnBvaW50KHRoaXMu
X3gyLHRoaXMuX3kyKSx0aGlzLnBvaW50KHRoaXMuX3gzLHRoaXMuX3kzKSx0aGlzLnBvaW50KHRo
aXMuX3g0LHRoaXMuX3k0KX19LHBvaW50OmZ1bmN0aW9uKHQsbil7c3dpdGNoKHQ9K3Qsbj0rbix0
aGlzLl9wb2ludCl7Y2FzZSAwOnRoaXMuX3BvaW50PTEsdGhpcy5feDI9dCx0aGlzLl95Mj1uO2Jy
ZWFrO2Nhc2UgMTp0aGlzLl9wb2ludD0yLHRoaXMuX3gzPXQsdGhpcy5feTM9bjticmVhaztjYXNl
IDI6dGhpcy5fcG9pbnQ9Myx0aGlzLl94ND10LHRoaXMuX3k0PW4sdGhpcy5fY29udGV4dC5tb3Zl
VG8oKHRoaXMuX3gwKzQqdGhpcy5feDErdCkvNiwodGhpcy5feTArNCp0aGlzLl95MStuKS82KTti
cmVhaztkZWZhdWx0OlJ4KHRoaXMsdCxuKX10aGlzLl94MD10aGlzLl94MSx0aGlzLl94MT10LHRo
aXMuX3kwPXRoaXMuX3kxLHRoaXMuX3kxPW59fSxVeC5wcm90b3R5cGU9e2FyZWFTdGFydDpmdW5j
dGlvbigpe3RoaXMuX2xpbmU9MH0sYXJlYUVuZDpmdW5jdGlvbigpe3RoaXMuX2xpbmU9TmFOfSxs
aW5lU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLl94MD10aGlzLl94MT10aGlzLl95MD10aGlzLl95MT1O
YU4sdGhpcy5fcG9pbnQ9MH0sbGluZUVuZDpmdW5jdGlvbigpeyh0aGlzLl9saW5lfHwwIT09dGhp
cy5fbGluZSYmMz09PXRoaXMuX3BvaW50KSYmdGhpcy5fY29udGV4dC5jbG9zZVBhdGgoKSx0aGlz
Ll9saW5lPTEtdGhpcy5fbGluZX0scG9pbnQ6ZnVuY3Rpb24odCxuKXtzd2l0Y2godD0rdCxuPStu
LHRoaXMuX3BvaW50KXtjYXNlIDA6dGhpcy5fcG9pbnQ9MTticmVhaztjYXNlIDE6dGhpcy5fcG9p
bnQ9MjticmVhaztjYXNlIDI6dGhpcy5fcG9pbnQ9Mzt2YXIgZT0odGhpcy5feDArNCp0aGlzLl94
MSt0KS82LHI9KHRoaXMuX3kwKzQqdGhpcy5feTErbikvNjt0aGlzLl9saW5lP3RoaXMuX2NvbnRl
eHQubGluZVRvKGUscik6dGhpcy5fY29udGV4dC5tb3ZlVG8oZSxyKTticmVhaztjYXNlIDM6dGhp
cy5fcG9pbnQ9NDtkZWZhdWx0OlJ4KHRoaXMsdCxuKX10aGlzLl94MD10aGlzLl94MSx0aGlzLl94
MT10LHRoaXMuX3kwPXRoaXMuX3kxLHRoaXMuX3kxPW59fSxJeC5wcm90b3R5cGU9e2xpbmVTdGFy
dDpmdW5jdGlvbigpe3RoaXMuX3g9W10sdGhpcy5feT1bXSx0aGlzLl9iYXNpcy5saW5lU3RhcnQo
KX0sbGluZUVuZDpmdW5jdGlvbigpe3ZhciB0PXRoaXMuX3gsbj10aGlzLl95LGU9dC5sZW5ndGgt
MTtpZihlPjApZm9yKHZhciByLGk9dFswXSxvPW5bMF0sYT10W2VdLWksdT1uW2VdLW8sYz0tMTsr
K2M8PWU7KXI9Yy9lLHRoaXMuX2Jhc2lzLnBvaW50KHRoaXMuX2JldGEqdFtjXSsoMS10aGlzLl9i
ZXRhKSooaStyKmEpLHRoaXMuX2JldGEqbltjXSsoMS10aGlzLl9iZXRhKSoobytyKnUpKTt0aGlz
Ll94PXRoaXMuX3k9bnVsbCx0aGlzLl9iYXNpcy5saW5lRW5kKCl9LHBvaW50OmZ1bmN0aW9uKHQs
bil7dGhpcy5feC5wdXNoKCt0KSx0aGlzLl95LnB1c2goK24pfX07dmFyIE94PWZ1bmN0aW9uIHQo
bil7ZnVuY3Rpb24gZSh0KXtyZXR1cm4gMT09PW4/bmV3IEZ4KHQpOm5ldyBJeCh0LG4pfXJldHVy
biBlLmJldGE9ZnVuY3Rpb24obil7cmV0dXJuIHQoK24pfSxlfSguODUpO2Z1bmN0aW9uIEJ4KHQs
bixlKXt0Ll9jb250ZXh0LmJlemllckN1cnZlVG8odC5feDErdC5fayoodC5feDItdC5feDApLHQu
X3kxK3QuX2sqKHQuX3kyLXQuX3kwKSx0Ll94Mit0Ll9rKih0Ll94MS1uKSx0Ll95Mit0Ll9rKih0
Ll95MS1lKSx0Ll94Mix0Ll95Mil9ZnVuY3Rpb24gWXgodCxuKXt0aGlzLl9jb250ZXh0PXQsdGhp
cy5faz0oMS1uKS82fVl4LnByb3RvdHlwZT17YXJlYVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5fbGlu
ZT0wfSxhcmVhRW5kOmZ1bmN0aW9uKCl7dGhpcy5fbGluZT1OYU59LGxpbmVTdGFydDpmdW5jdGlv
bigpe3RoaXMuX3gwPXRoaXMuX3gxPXRoaXMuX3gyPXRoaXMuX3kwPXRoaXMuX3kxPXRoaXMuX3ky
PU5hTix0aGlzLl9wb2ludD0wfSxsaW5lRW5kOmZ1bmN0aW9uKCl7c3dpdGNoKHRoaXMuX3BvaW50
KXtjYXNlIDI6dGhpcy5fY29udGV4dC5saW5lVG8odGhpcy5feDIsdGhpcy5feTIpO2JyZWFrO2Nh
c2UgMzpCeCh0aGlzLHRoaXMuX3gxLHRoaXMuX3kxKX0odGhpcy5fbGluZXx8MCE9PXRoaXMuX2xp
bmUmJjE9PT10aGlzLl9wb2ludCkmJnRoaXMuX2NvbnRleHQuY2xvc2VQYXRoKCksdGhpcy5fbGlu
ZT0xLXRoaXMuX2xpbmV9LHBvaW50OmZ1bmN0aW9uKHQsbil7c3dpdGNoKHQ9K3Qsbj0rbix0aGlz
Ll9wb2ludCl7Y2FzZSAwOnRoaXMuX3BvaW50PTEsdGhpcy5fbGluZT90aGlzLl9jb250ZXh0Lmxp
bmVUbyh0LG4pOnRoaXMuX2NvbnRleHQubW92ZVRvKHQsbik7YnJlYWs7Y2FzZSAxOnRoaXMuX3Bv
aW50PTIsdGhpcy5feDE9dCx0aGlzLl95MT1uO2JyZWFrO2Nhc2UgMjp0aGlzLl9wb2ludD0zO2Rl
ZmF1bHQ6QngodGhpcyx0LG4pfXRoaXMuX3gwPXRoaXMuX3gxLHRoaXMuX3gxPXRoaXMuX3gyLHRo
aXMuX3gyPXQsdGhpcy5feTA9dGhpcy5feTEsdGhpcy5feTE9dGhpcy5feTIsdGhpcy5feTI9bn19
O3ZhciBMeD1mdW5jdGlvbiB0KG4pe2Z1bmN0aW9uIGUodCl7cmV0dXJuIG5ldyBZeCh0LG4pfXJl
dHVybiBlLnRlbnNpb249ZnVuY3Rpb24obil7cmV0dXJuIHQoK24pfSxlfSgwKTtmdW5jdGlvbiBq
eCh0LG4pe3RoaXMuX2NvbnRleHQ9dCx0aGlzLl9rPSgxLW4pLzZ9angucHJvdG90eXBlPXthcmVh
U3RhcnQ6RHgsYXJlYUVuZDpEeCxsaW5lU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLl94MD10aGlzLl94
MT10aGlzLl94Mj10aGlzLl94Mz10aGlzLl94ND10aGlzLl94NT10aGlzLl95MD10aGlzLl95MT10
aGlzLl95Mj10aGlzLl95Mz10aGlzLl95ND10aGlzLl95NT1OYU4sdGhpcy5fcG9pbnQ9MH0sbGlu
ZUVuZDpmdW5jdGlvbigpe3N3aXRjaCh0aGlzLl9wb2ludCl7Y2FzZSAxOnRoaXMuX2NvbnRleHQu
bW92ZVRvKHRoaXMuX3gzLHRoaXMuX3kzKSx0aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgpO2JyZWFr
O2Nhc2UgMjp0aGlzLl9jb250ZXh0LmxpbmVUbyh0aGlzLl94Myx0aGlzLl95MyksdGhpcy5fY29u
dGV4dC5jbG9zZVBhdGgoKTticmVhaztjYXNlIDM6dGhpcy5wb2ludCh0aGlzLl94Myx0aGlzLl95
MyksdGhpcy5wb2ludCh0aGlzLl94NCx0aGlzLl95NCksdGhpcy5wb2ludCh0aGlzLl94NSx0aGlz
Ll95NSl9fSxwb2ludDpmdW5jdGlvbih0LG4pe3N3aXRjaCh0PSt0LG49K24sdGhpcy5fcG9pbnQp
e2Nhc2UgMDp0aGlzLl9wb2ludD0xLHRoaXMuX3gzPXQsdGhpcy5feTM9bjticmVhaztjYXNlIDE6
dGhpcy5fcG9pbnQ9Mix0aGlzLl9jb250ZXh0Lm1vdmVUbyh0aGlzLl94ND10LHRoaXMuX3k0PW4p
O2JyZWFrO2Nhc2UgMjp0aGlzLl9wb2ludD0zLHRoaXMuX3g1PXQsdGhpcy5feTU9bjticmVhaztk
ZWZhdWx0OkJ4KHRoaXMsdCxuKX10aGlzLl94MD10aGlzLl94MSx0aGlzLl94MT10aGlzLl94Mix0
aGlzLl94Mj10LHRoaXMuX3kwPXRoaXMuX3kxLHRoaXMuX3kxPXRoaXMuX3kyLHRoaXMuX3kyPW59
fTt2YXIgSHg9ZnVuY3Rpb24gdChuKXtmdW5jdGlvbiBlKHQpe3JldHVybiBuZXcgangodCxuKX1y
ZXR1cm4gZS50ZW5zaW9uPWZ1bmN0aW9uKG4pe3JldHVybiB0KCtuKX0sZX0oMCk7ZnVuY3Rpb24g
WHgodCxuKXt0aGlzLl9jb250ZXh0PXQsdGhpcy5faz0oMS1uKS82fVh4LnByb3RvdHlwZT17YXJl
YVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5fbGluZT0wfSxhcmVhRW5kOmZ1bmN0aW9uKCl7dGhpcy5f
bGluZT1OYU59LGxpbmVTdGFydDpmdW5jdGlvbigpe3RoaXMuX3gwPXRoaXMuX3gxPXRoaXMuX3gy
PXRoaXMuX3kwPXRoaXMuX3kxPXRoaXMuX3kyPU5hTix0aGlzLl9wb2ludD0wfSxsaW5lRW5kOmZ1
bmN0aW9uKCl7KHRoaXMuX2xpbmV8fDAhPT10aGlzLl9saW5lJiYzPT09dGhpcy5fcG9pbnQpJiZ0
aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgpLHRoaXMuX2xpbmU9MS10aGlzLl9saW5lfSxwb2ludDpm
dW5jdGlvbih0LG4pe3N3aXRjaCh0PSt0LG49K24sdGhpcy5fcG9pbnQpe2Nhc2UgMDp0aGlzLl9w
b2ludD0xO2JyZWFrO2Nhc2UgMTp0aGlzLl9wb2ludD0yO2JyZWFrO2Nhc2UgMjp0aGlzLl9wb2lu
dD0zLHRoaXMuX2xpbmU/dGhpcy5fY29udGV4dC5saW5lVG8odGhpcy5feDIsdGhpcy5feTIpOnRo
aXMuX2NvbnRleHQubW92ZVRvKHRoaXMuX3gyLHRoaXMuX3kyKTticmVhaztjYXNlIDM6dGhpcy5f
cG9pbnQ9NDtkZWZhdWx0OkJ4KHRoaXMsdCxuKX10aGlzLl94MD10aGlzLl94MSx0aGlzLl94MT10
aGlzLl94Mix0aGlzLl94Mj10LHRoaXMuX3kwPXRoaXMuX3kxLHRoaXMuX3kxPXRoaXMuX3kyLHRo
aXMuX3kyPW59fTt2YXIgR3g9ZnVuY3Rpb24gdChuKXtmdW5jdGlvbiBlKHQpe3JldHVybiBuZXcg
WHgodCxuKX1yZXR1cm4gZS50ZW5zaW9uPWZ1bmN0aW9uKG4pe3JldHVybiB0KCtuKX0sZX0oMCk7
ZnVuY3Rpb24gVngodCxuLGUpe3ZhciByPXQuX3gxLGk9dC5feTEsbz10Ll94MixhPXQuX3kyO2lm
KHQuX2wwMV9hPlRtKXt2YXIgdT0yKnQuX2wwMV8yYSszKnQuX2wwMV9hKnQuX2wxMl9hK3QuX2wx
Ml8yYSxjPTMqdC5fbDAxX2EqKHQuX2wwMV9hK3QuX2wxMl9hKTtyPShyKnUtdC5feDAqdC5fbDEy
XzJhK3QuX3gyKnQuX2wwMV8yYSkvYyxpPShpKnUtdC5feTAqdC5fbDEyXzJhK3QuX3kyKnQuX2ww
MV8yYSkvY31pZih0Ll9sMjNfYT5UbSl7dmFyIGY9Mip0Ll9sMjNfMmErMyp0Ll9sMjNfYSp0Ll9s
MTJfYSt0Ll9sMTJfMmEscz0zKnQuX2wyM19hKih0Ll9sMjNfYSt0Ll9sMTJfYSk7bz0obypmK3Qu
X3gxKnQuX2wyM18yYS1uKnQuX2wxMl8yYSkvcyxhPShhKmYrdC5feTEqdC5fbDIzXzJhLWUqdC5f
bDEyXzJhKS9zfXQuX2NvbnRleHQuYmV6aWVyQ3VydmVUbyhyLGksbyxhLHQuX3gyLHQuX3kyKX1m
dW5jdGlvbiBXeCh0LG4pe3RoaXMuX2NvbnRleHQ9dCx0aGlzLl9hbHBoYT1ufVd4LnByb3RvdHlw
ZT17YXJlYVN0YXJ0OmZ1bmN0aW9uKCl7dGhpcy5fbGluZT0wfSxhcmVhRW5kOmZ1bmN0aW9uKCl7
dGhpcy5fbGluZT1OYU59LGxpbmVTdGFydDpmdW5jdGlvbigpe3RoaXMuX3gwPXRoaXMuX3gxPXRo
aXMuX3gyPXRoaXMuX3kwPXRoaXMuX3kxPXRoaXMuX3kyPU5hTix0aGlzLl9sMDFfYT10aGlzLl9s
MTJfYT10aGlzLl9sMjNfYT10aGlzLl9sMDFfMmE9dGhpcy5fbDEyXzJhPXRoaXMuX2wyM18yYT10
aGlzLl9wb2ludD0wfSxsaW5lRW5kOmZ1bmN0aW9uKCl7c3dpdGNoKHRoaXMuX3BvaW50KXtjYXNl
IDI6dGhpcy5fY29udGV4dC5saW5lVG8odGhpcy5feDIsdGhpcy5feTIpO2JyZWFrO2Nhc2UgMzp0
aGlzLnBvaW50KHRoaXMuX3gyLHRoaXMuX3kyKX0odGhpcy5fbGluZXx8MCE9PXRoaXMuX2xpbmUm
JjE9PT10aGlzLl9wb2ludCkmJnRoaXMuX2NvbnRleHQuY2xvc2VQYXRoKCksdGhpcy5fbGluZT0x
LXRoaXMuX2xpbmV9LHBvaW50OmZ1bmN0aW9uKHQsbil7aWYodD0rdCxuPStuLHRoaXMuX3BvaW50
KXt2YXIgZT10aGlzLl94Mi10LHI9dGhpcy5feTItbjt0aGlzLl9sMjNfYT1NYXRoLnNxcnQodGhp
cy5fbDIzXzJhPU1hdGgucG93KGUqZStyKnIsdGhpcy5fYWxwaGEpKX1zd2l0Y2godGhpcy5fcG9p
bnQpe2Nhc2UgMDp0aGlzLl9wb2ludD0xLHRoaXMuX2xpbmU/dGhpcy5fY29udGV4dC5saW5lVG8o
dCxuKTp0aGlzLl9jb250ZXh0Lm1vdmVUbyh0LG4pO2JyZWFrO2Nhc2UgMTp0aGlzLl9wb2ludD0y
O2JyZWFrO2Nhc2UgMjp0aGlzLl9wb2ludD0zO2RlZmF1bHQ6VngodGhpcyx0LG4pfXRoaXMuX2ww
MV9hPXRoaXMuX2wxMl9hLHRoaXMuX2wxMl9hPXRoaXMuX2wyM19hLHRoaXMuX2wwMV8yYT10aGlz
Ll9sMTJfMmEsdGhpcy5fbDEyXzJhPXRoaXMuX2wyM18yYSx0aGlzLl94MD10aGlzLl94MSx0aGlz
Ll94MT10aGlzLl94Mix0aGlzLl94Mj10LHRoaXMuX3kwPXRoaXMuX3kxLHRoaXMuX3kxPXRoaXMu
X3kyLHRoaXMuX3kyPW59fTt2YXIgWng9ZnVuY3Rpb24gdChuKXtmdW5jdGlvbiBlKHQpe3JldHVy
biBuP25ldyBXeCh0LG4pOm5ldyBZeCh0LDApfXJldHVybiBlLmFscGhhPWZ1bmN0aW9uKG4pe3Jl
dHVybiB0KCtuKX0sZX0oLjUpO2Z1bmN0aW9uIEt4KHQsbil7dGhpcy5fY29udGV4dD10LHRoaXMu
X2FscGhhPW59S3gucHJvdG90eXBlPXthcmVhU3RhcnQ6RHgsYXJlYUVuZDpEeCxsaW5lU3RhcnQ6
ZnVuY3Rpb24oKXt0aGlzLl94MD10aGlzLl94MT10aGlzLl94Mj10aGlzLl94Mz10aGlzLl94ND10
aGlzLl94NT10aGlzLl95MD10aGlzLl95MT10aGlzLl95Mj10aGlzLl95Mz10aGlzLl95ND10aGlz
Ll95NT1OYU4sdGhpcy5fbDAxX2E9dGhpcy5fbDEyX2E9dGhpcy5fbDIzX2E9dGhpcy5fbDAxXzJh
PXRoaXMuX2wxMl8yYT10aGlzLl9sMjNfMmE9dGhpcy5fcG9pbnQ9MH0sbGluZUVuZDpmdW5jdGlv
bigpe3N3aXRjaCh0aGlzLl9wb2ludCl7Y2FzZSAxOnRoaXMuX2NvbnRleHQubW92ZVRvKHRoaXMu
X3gzLHRoaXMuX3kzKSx0aGlzLl9jb250ZXh0LmNsb3NlUGF0aCgpO2JyZWFrO2Nhc2UgMjp0aGlz
Ll9jb250ZXh0LmxpbmVUbyh0aGlzLl94Myx0aGlzLl95MyksdGhpcy5fY29udGV4dC5jbG9zZVBh
dGgoKTticmVhaztjYXNlIDM6dGhpcy5wb2ludCh0aGlzLl94Myx0aGlzLl95MyksdGhpcy5wb2lu
dCh0aGlzLl94NCx0aGlzLl95NCksdGhpcy5wb2ludCh0aGlzLl94NSx0aGlzLl95NSl9fSxwb2lu
dDpmdW5jdGlvbih0LG4pe2lmKHQ9K3Qsbj0rbix0aGlzLl9wb2ludCl7dmFyIGU9dGhpcy5feDIt
dCxyPXRoaXMuX3kyLW47dGhpcy5fbDIzX2E9TWF0aC5zcXJ0KHRoaXMuX2wyM18yYT1NYXRoLnBv
dyhlKmUrcipyLHRoaXMuX2FscGhhKSl9c3dpdGNoKHRoaXMuX3BvaW50KXtjYXNlIDA6dGhpcy5f
cG9pbnQ9MSx0aGlzLl94Mz10LHRoaXMuX3kzPW47YnJlYWs7Y2FzZSAxOnRoaXMuX3BvaW50PTIs
dGhpcy5fY29udGV4dC5tb3ZlVG8odGhpcy5feDQ9dCx0aGlzLl95ND1uKTticmVhaztjYXNlIDI6
dGhpcy5fcG9pbnQ9Myx0aGlzLl94NT10LHRoaXMuX3k1PW47YnJlYWs7ZGVmYXVsdDpWeCh0aGlz
LHQsbil9dGhpcy5fbDAxX2E9dGhpcy5fbDEyX2EsdGhpcy5fbDEyX2E9dGhpcy5fbDIzX2EsdGhp
cy5fbDAxXzJhPXRoaXMuX2wxMl8yYSx0aGlzLl9sMTJfMmE9dGhpcy5fbDIzXzJhLHRoaXMuX3gw
PXRoaXMuX3gxLHRoaXMuX3gxPXRoaXMuX3gyLHRoaXMuX3gyPXQsdGhpcy5feTA9dGhpcy5feTEs
dGhpcy5feTE9dGhpcy5feTIsdGhpcy5feTI9bn19O3ZhciBReD1mdW5jdGlvbiB0KG4pe2Z1bmN0
aW9uIGUodCl7cmV0dXJuIG4/bmV3IEt4KHQsbik6bmV3IGp4KHQsMCl9cmV0dXJuIGUuYWxwaGE9
ZnVuY3Rpb24obil7cmV0dXJuIHQoK24pfSxlfSguNSk7ZnVuY3Rpb24gSngodCxuKXt0aGlzLl9j
b250ZXh0PXQsdGhpcy5fYWxwaGE9bn1KeC5wcm90b3R5cGU9e2FyZWFTdGFydDpmdW5jdGlvbigp
e3RoaXMuX2xpbmU9MH0sYXJlYUVuZDpmdW5jdGlvbigpe3RoaXMuX2xpbmU9TmFOfSxsaW5lU3Rh
cnQ6ZnVuY3Rpb24oKXt0aGlzLl94MD10aGlzLl94MT10aGlzLl94Mj10aGlzLl95MD10aGlzLl95
MT10aGlzLl95Mj1OYU4sdGhpcy5fbDAxX2E9dGhpcy5fbDEyX2E9dGhpcy5fbDIzX2E9dGhpcy5f
bDAxXzJhPXRoaXMuX2wxMl8yYT10aGlzLl9sMjNfMmE9dGhpcy5fcG9pbnQ9MH0sbGluZUVuZDpm
dW5jdGlvbigpeyh0aGlzLl9saW5lfHwwIT09dGhpcy5fbGluZSYmMz09PXRoaXMuX3BvaW50KSYm
dGhpcy5fY29udGV4dC5jbG9zZVBhdGgoKSx0aGlzLl9saW5lPTEtdGhpcy5fbGluZX0scG9pbnQ6
ZnVuY3Rpb24odCxuKXtpZih0PSt0LG49K24sdGhpcy5fcG9pbnQpe3ZhciBlPXRoaXMuX3gyLXQs
cj10aGlzLl95Mi1uO3RoaXMuX2wyM19hPU1hdGguc3FydCh0aGlzLl9sMjNfMmE9TWF0aC5wb3co
ZSplK3Iqcix0aGlzLl9hbHBoYSkpfXN3aXRjaCh0aGlzLl9wb2ludCl7Y2FzZSAwOnRoaXMuX3Bv
aW50PTE7YnJlYWs7Y2FzZSAxOnRoaXMuX3BvaW50PTI7YnJlYWs7Y2FzZSAyOnRoaXMuX3BvaW50
PTMsdGhpcy5fbGluZT90aGlzLl9jb250ZXh0LmxpbmVUbyh0aGlzLl94Mix0aGlzLl95Mik6dGhp
cy5fY29udGV4dC5tb3ZlVG8odGhpcy5feDIsdGhpcy5feTIpO2JyZWFrO2Nhc2UgMzp0aGlzLl9w
b2ludD00O2RlZmF1bHQ6VngodGhpcyx0LG4pfXRoaXMuX2wwMV9hPXRoaXMuX2wxMl9hLHRoaXMu
X2wxMl9hPXRoaXMuX2wyM19hLHRoaXMuX2wwMV8yYT10aGlzLl9sMTJfMmEsdGhpcy5fbDEyXzJh
PXRoaXMuX2wyM18yYSx0aGlzLl94MD10aGlzLl94MSx0aGlzLl94MT10aGlzLl94Mix0aGlzLl94
Mj10LHRoaXMuX3kwPXRoaXMuX3kxLHRoaXMuX3kxPXRoaXMuX3kyLHRoaXMuX3kyPW59fTt2YXIg
dHc9ZnVuY3Rpb24gdChuKXtmdW5jdGlvbiBlKHQpe3JldHVybiBuP25ldyBKeCh0LG4pOm5ldyBY
eCh0LDApfXJldHVybiBlLmFscGhhPWZ1bmN0aW9uKG4pe3JldHVybiB0KCtuKX0sZX0oLjUpO2Z1
bmN0aW9uIG53KHQpe3RoaXMuX2NvbnRleHQ9dH1mdW5jdGlvbiBldyh0KXtyZXR1cm4gdDwwPy0x
OjF9ZnVuY3Rpb24gcncodCxuLGUpe3ZhciByPXQuX3gxLXQuX3gwLGk9bi10Ll94MSxvPSh0Ll95
MS10Ll95MCkvKHJ8fGk8MCYmLTApLGE9KGUtdC5feTEpLyhpfHxyPDAmJi0wKSx1PShvKmkrYSpy
KS8ocitpKTtyZXR1cm4oZXcobykrZXcoYSkpKk1hdGgubWluKE1hdGguYWJzKG8pLE1hdGguYWJz
KGEpLC41Kk1hdGguYWJzKHUpKXx8MH1mdW5jdGlvbiBpdyh0LG4pe3ZhciBlPXQuX3gxLXQuX3gw
O3JldHVybiBlPygzKih0Ll95MS10Ll95MCkvZS1uKS8yOm59ZnVuY3Rpb24gb3codCxuLGUpe3Zh
ciByPXQuX3gwLGk9dC5feTAsbz10Ll94MSxhPXQuX3kxLHU9KG8tcikvMzt0Ll9jb250ZXh0LmJl
emllckN1cnZlVG8ocit1LGkrdSpuLG8tdSxhLXUqZSxvLGEpfWZ1bmN0aW9uIGF3KHQpe3RoaXMu
X2NvbnRleHQ9dH1mdW5jdGlvbiB1dyh0KXt0aGlzLl9jb250ZXh0PW5ldyBjdyh0KX1mdW5jdGlv
biBjdyh0KXt0aGlzLl9jb250ZXh0PXR9ZnVuY3Rpb24gZncodCl7dGhpcy5fY29udGV4dD10fWZ1
bmN0aW9uIHN3KHQpe3ZhciBuLGUscj10Lmxlbmd0aC0xLGk9bmV3IEFycmF5KHIpLG89bmV3IEFy
cmF5KHIpLGE9bmV3IEFycmF5KHIpO2ZvcihpWzBdPTAsb1swXT0yLGFbMF09dFswXSsyKnRbMV0s
bj0xO248ci0xOysrbilpW25dPTEsb1tuXT00LGFbbl09NCp0W25dKzIqdFtuKzFdO2ZvcihpW3It
MV09MixvW3ItMV09NyxhW3ItMV09OCp0W3ItMV0rdFtyXSxuPTE7bjxyOysrbillPWlbbl0vb1tu
LTFdLG9bbl0tPWUsYVtuXS09ZSphW24tMV07Zm9yKGlbci0xXT1hW3ItMV0vb1tyLTFdLG49ci0y
O24+PTA7LS1uKWlbbl09KGFbbl0taVtuKzFdKS9vW25dO2ZvcihvW3ItMV09KHRbcl0raVtyLTFd
KS8yLG49MDtuPHItMTsrK24pb1tuXT0yKnRbbisxXS1pW24rMV07cmV0dXJuW2ksb119ZnVuY3Rp
b24gbHcodCxuKXt0aGlzLl9jb250ZXh0PXQsdGhpcy5fdD1ufWZ1bmN0aW9uIGh3KHQsbil7aWYo
KGk9dC5sZW5ndGgpPjEpZm9yKHZhciBlLHIsaSxvPTEsYT10W25bMF1dLHU9YS5sZW5ndGg7bzxp
Oysrbylmb3Iocj1hLGE9dFtuW29dXSxlPTA7ZTx1OysrZSlhW2VdWzFdKz1hW2VdWzBdPWlzTmFO
KHJbZV1bMV0pP3JbZV1bMF06cltlXVsxXX1mdW5jdGlvbiBkdyh0KXtmb3IodmFyIG49dC5sZW5n
dGgsZT1uZXcgQXJyYXkobik7LS1uPj0wOyllW25dPW47cmV0dXJuIGV9ZnVuY3Rpb24gcHcodCxu
KXtyZXR1cm4gdFtuXX1mdW5jdGlvbiBndyh0KXtjb25zdCBuPVtdO3JldHVybiBuLmtleT10LG59
ZnVuY3Rpb24geXcodCl7dmFyIG49dC5tYXAodncpO3JldHVybiBkdyh0KS5zb3J0KChmdW5jdGlv
bih0LGUpe3JldHVybiBuW3RdLW5bZV19KSl9ZnVuY3Rpb24gdncodCl7Zm9yKHZhciBuLGU9LTEs
cj0wLGk9dC5sZW5ndGgsbz0tMS8wOysrZTxpOykobj0rdFtlXVsxXSk+byYmKG89bixyPWUpO3Jl
dHVybiByfWZ1bmN0aW9uIF93KHQpe3ZhciBuPXQubWFwKGJ3KTtyZXR1cm4gZHcodCkuc29ydCgo
ZnVuY3Rpb24odCxlKXtyZXR1cm4gblt0XS1uW2VdfSkpfWZ1bmN0aW9uIGJ3KHQpe2Zvcih2YXIg
bixlPTAscj0tMSxpPXQubGVuZ3RoOysrcjxpOykobj0rdFtyXVsxXSkmJihlKz1uKTtyZXR1cm4g
ZX1udy5wcm90b3R5cGU9e2FyZWFTdGFydDpEeCxhcmVhRW5kOkR4LGxpbmVTdGFydDpmdW5jdGlv
bigpe3RoaXMuX3BvaW50PTB9LGxpbmVFbmQ6ZnVuY3Rpb24oKXt0aGlzLl9wb2ludCYmdGhpcy5f
Y29udGV4dC5jbG9zZVBhdGgoKX0scG9pbnQ6ZnVuY3Rpb24odCxuKXt0PSt0LG49K24sdGhpcy5f
cG9pbnQ/dGhpcy5fY29udGV4dC5saW5lVG8odCxuKToodGhpcy5fcG9pbnQ9MSx0aGlzLl9jb250
ZXh0Lm1vdmVUbyh0LG4pKX19LGF3LnByb3RvdHlwZT17YXJlYVN0YXJ0OmZ1bmN0aW9uKCl7dGhp
cy5fbGluZT0wfSxhcmVhRW5kOmZ1bmN0aW9uKCl7dGhpcy5fbGluZT1OYU59LGxpbmVTdGFydDpm
dW5jdGlvbigpe3RoaXMuX3gwPXRoaXMuX3gxPXRoaXMuX3kwPXRoaXMuX3kxPXRoaXMuX3QwPU5h
Tix0aGlzLl9wb2ludD0wfSxsaW5lRW5kOmZ1bmN0aW9uKCl7c3dpdGNoKHRoaXMuX3BvaW50KXtj
YXNlIDI6dGhpcy5fY29udGV4dC5saW5lVG8odGhpcy5feDEsdGhpcy5feTEpO2JyZWFrO2Nhc2Ug
Mzpvdyh0aGlzLHRoaXMuX3QwLGl3KHRoaXMsdGhpcy5fdDApKX0odGhpcy5fbGluZXx8MCE9PXRo
aXMuX2xpbmUmJjE9PT10aGlzLl9wb2ludCkmJnRoaXMuX2NvbnRleHQuY2xvc2VQYXRoKCksdGhp
cy5fbGluZT0xLXRoaXMuX2xpbmV9LHBvaW50OmZ1bmN0aW9uKHQsbil7dmFyIGU9TmFOO2lmKG49
K24sKHQ9K3QpIT09dGhpcy5feDF8fG4hPT10aGlzLl95MSl7c3dpdGNoKHRoaXMuX3BvaW50KXtj
YXNlIDA6dGhpcy5fcG9pbnQ9MSx0aGlzLl9saW5lP3RoaXMuX2NvbnRleHQubGluZVRvKHQsbik6
dGhpcy5fY29udGV4dC5tb3ZlVG8odCxuKTticmVhaztjYXNlIDE6dGhpcy5fcG9pbnQ9MjticmVh
aztjYXNlIDI6dGhpcy5fcG9pbnQ9Myxvdyh0aGlzLGl3KHRoaXMsZT1ydyh0aGlzLHQsbikpLGUp
O2JyZWFrO2RlZmF1bHQ6b3codGhpcyx0aGlzLl90MCxlPXJ3KHRoaXMsdCxuKSl9dGhpcy5feDA9
dGhpcy5feDEsdGhpcy5feDE9dCx0aGlzLl95MD10aGlzLl95MSx0aGlzLl95MT1uLHRoaXMuX3Qw
PWV9fX0sKHV3LnByb3RvdHlwZT1PYmplY3QuY3JlYXRlKGF3LnByb3RvdHlwZSkpLnBvaW50PWZ1
bmN0aW9uKHQsbil7YXcucHJvdG90eXBlLnBvaW50LmNhbGwodGhpcyxuLHQpfSxjdy5wcm90b3R5
cGU9e21vdmVUbzpmdW5jdGlvbih0LG4pe3RoaXMuX2NvbnRleHQubW92ZVRvKG4sdCl9LGNsb3Nl
UGF0aDpmdW5jdGlvbigpe3RoaXMuX2NvbnRleHQuY2xvc2VQYXRoKCl9LGxpbmVUbzpmdW5jdGlv
bih0LG4pe3RoaXMuX2NvbnRleHQubGluZVRvKG4sdCl9LGJlemllckN1cnZlVG86ZnVuY3Rpb24o
dCxuLGUscixpLG8pe3RoaXMuX2NvbnRleHQuYmV6aWVyQ3VydmVUbyhuLHQscixlLG8saSl9fSxm
dy5wcm90b3R5cGU9e2FyZWFTdGFydDpmdW5jdGlvbigpe3RoaXMuX2xpbmU9MH0sYXJlYUVuZDpm
dW5jdGlvbigpe3RoaXMuX2xpbmU9TmFOfSxsaW5lU3RhcnQ6ZnVuY3Rpb24oKXt0aGlzLl94PVtd
LHRoaXMuX3k9W119LGxpbmVFbmQ6ZnVuY3Rpb24oKXt2YXIgdD10aGlzLl94LG49dGhpcy5feSxl
PXQubGVuZ3RoO2lmKGUpaWYodGhpcy5fbGluZT90aGlzLl9jb250ZXh0LmxpbmVUbyh0WzBdLG5b
MF0pOnRoaXMuX2NvbnRleHQubW92ZVRvKHRbMF0sblswXSksMj09PWUpdGhpcy5fY29udGV4dC5s
aW5lVG8odFsxXSxuWzFdKTtlbHNlIGZvcih2YXIgcj1zdyh0KSxpPXN3KG4pLG89MCxhPTE7YTxl
OysrbywrK2EpdGhpcy5fY29udGV4dC5iZXppZXJDdXJ2ZVRvKHJbMF1bb10saVswXVtvXSxyWzFd
W29dLGlbMV1bb10sdFthXSxuW2FdKTsodGhpcy5fbGluZXx8MCE9PXRoaXMuX2xpbmUmJjE9PT1l
KSYmdGhpcy5fY29udGV4dC5jbG9zZVBhdGgoKSx0aGlzLl9saW5lPTEtdGhpcy5fbGluZSx0aGlz
Ll94PXRoaXMuX3k9bnVsbH0scG9pbnQ6ZnVuY3Rpb24odCxuKXt0aGlzLl94LnB1c2goK3QpLHRo
aXMuX3kucHVzaCgrbil9fSxsdy5wcm90b3R5cGU9e2FyZWFTdGFydDpmdW5jdGlvbigpe3RoaXMu
X2xpbmU9MH0sYXJlYUVuZDpmdW5jdGlvbigpe3RoaXMuX2xpbmU9TmFOfSxsaW5lU3RhcnQ6ZnVu
Y3Rpb24oKXt0aGlzLl94PXRoaXMuX3k9TmFOLHRoaXMuX3BvaW50PTB9LGxpbmVFbmQ6ZnVuY3Rp
b24oKXswPHRoaXMuX3QmJnRoaXMuX3Q8MSYmMj09PXRoaXMuX3BvaW50JiZ0aGlzLl9jb250ZXh0
LmxpbmVUbyh0aGlzLl94LHRoaXMuX3kpLCh0aGlzLl9saW5lfHwwIT09dGhpcy5fbGluZSYmMT09
PXRoaXMuX3BvaW50KSYmdGhpcy5fY29udGV4dC5jbG9zZVBhdGgoKSx0aGlzLl9saW5lPj0wJiYo
dGhpcy5fdD0xLXRoaXMuX3QsdGhpcy5fbGluZT0xLXRoaXMuX2xpbmUpfSxwb2ludDpmdW5jdGlv
bih0LG4pe3N3aXRjaCh0PSt0LG49K24sdGhpcy5fcG9pbnQpe2Nhc2UgMDp0aGlzLl9wb2ludD0x
LHRoaXMuX2xpbmU/dGhpcy5fY29udGV4dC5saW5lVG8odCxuKTp0aGlzLl9jb250ZXh0Lm1vdmVU
byh0LG4pO2JyZWFrO2Nhc2UgMTp0aGlzLl9wb2ludD0yO2RlZmF1bHQ6aWYodGhpcy5fdDw9MCl0
aGlzLl9jb250ZXh0LmxpbmVUbyh0aGlzLl94LG4pLHRoaXMuX2NvbnRleHQubGluZVRvKHQsbik7
ZWxzZXt2YXIgZT10aGlzLl94KigxLXRoaXMuX3QpK3QqdGhpcy5fdDt0aGlzLl9jb250ZXh0Lmxp
bmVUbyhlLHRoaXMuX3kpLHRoaXMuX2NvbnRleHQubGluZVRvKGUsbil9fXRoaXMuX3g9dCx0aGlz
Ll95PW59fTt2YXIgbXc9dD0+KCk9PnQ7ZnVuY3Rpb24geHcodCx7c291cmNlRXZlbnQ6bix0YXJn
ZXQ6ZSx0cmFuc2Zvcm06cixkaXNwYXRjaDppfSl7T2JqZWN0LmRlZmluZVByb3BlcnRpZXModGhp
cyx7dHlwZTp7dmFsdWU6dCxlbnVtZXJhYmxlOiEwLGNvbmZpZ3VyYWJsZTohMH0sc291cmNlRXZl
bnQ6e3ZhbHVlOm4sZW51bWVyYWJsZTohMCxjb25maWd1cmFibGU6ITB9LHRhcmdldDp7dmFsdWU6
ZSxlbnVtZXJhYmxlOiEwLGNvbmZpZ3VyYWJsZTohMH0sdHJhbnNmb3JtOnt2YWx1ZTpyLGVudW1l
cmFibGU6ITAsY29uZmlndXJhYmxlOiEwfSxfOnt2YWx1ZTppfX0pfWZ1bmN0aW9uIHd3KHQsbixl
KXt0aGlzLms9dCx0aGlzLng9bix0aGlzLnk9ZX13dy5wcm90b3R5cGU9e2NvbnN0cnVjdG9yOnd3
LHNjYWxlOmZ1bmN0aW9uKHQpe3JldHVybiAxPT09dD90aGlzOm5ldyB3dyh0aGlzLmsqdCx0aGlz
LngsdGhpcy55KX0sdHJhbnNsYXRlOmZ1bmN0aW9uKHQsbil7cmV0dXJuIDA9PT10JjA9PT1uP3Ro
aXM6bmV3IHd3KHRoaXMuayx0aGlzLngrdGhpcy5rKnQsdGhpcy55K3RoaXMuaypuKX0sYXBwbHk6
ZnVuY3Rpb24odCl7cmV0dXJuW3RbMF0qdGhpcy5rK3RoaXMueCx0WzFdKnRoaXMuayt0aGlzLnld
fSxhcHBseVg6ZnVuY3Rpb24odCl7cmV0dXJuIHQqdGhpcy5rK3RoaXMueH0sYXBwbHlZOmZ1bmN0
aW9uKHQpe3JldHVybiB0KnRoaXMuayt0aGlzLnl9LGludmVydDpmdW5jdGlvbih0KXtyZXR1cm5b
KHRbMF0tdGhpcy54KS90aGlzLmssKHRbMV0tdGhpcy55KS90aGlzLmtdfSxpbnZlcnRYOmZ1bmN0
aW9uKHQpe3JldHVybih0LXRoaXMueCkvdGhpcy5rfSxpbnZlcnRZOmZ1bmN0aW9uKHQpe3JldHVy
bih0LXRoaXMueSkvdGhpcy5rfSxyZXNjYWxlWDpmdW5jdGlvbih0KXtyZXR1cm4gdC5jb3B5KCku
ZG9tYWluKHQucmFuZ2UoKS5tYXAodGhpcy5pbnZlcnRYLHRoaXMpLm1hcCh0LmludmVydCx0KSl9
LHJlc2NhbGVZOmZ1bmN0aW9uKHQpe3JldHVybiB0LmNvcHkoKS5kb21haW4odC5yYW5nZSgpLm1h
cCh0aGlzLmludmVydFksdGhpcykubWFwKHQuaW52ZXJ0LHQpKX0sdG9TdHJpbmc6ZnVuY3Rpb24o
KXtyZXR1cm4idHJhbnNsYXRlKCIrdGhpcy54KyIsIit0aGlzLnkrIikgc2NhbGUoIit0aGlzLmsr
IikifX07dmFyIE13PW5ldyB3dygxLDAsMCk7ZnVuY3Rpb24gVHcodCl7Zm9yKDshdC5fX3pvb207
KWlmKCEodD10LnBhcmVudE5vZGUpKXJldHVybiBNdztyZXR1cm4gdC5fX3pvb219ZnVuY3Rpb24g
QXcodCl7dC5zdG9wSW1tZWRpYXRlUHJvcGFnYXRpb24oKX1mdW5jdGlvbiBTdyh0KXt0LnByZXZl
bnREZWZhdWx0KCksdC5zdG9wSW1tZWRpYXRlUHJvcGFnYXRpb24oKX1mdW5jdGlvbiBFdyh0KXty
ZXR1cm4hKHQuY3RybEtleSYmIndoZWVsIiE9PXQudHlwZXx8dC5idXR0b24pfWZ1bmN0aW9uIE53
KCl7dmFyIHQ9dGhpcztyZXR1cm4gdCBpbnN0YW5jZW9mIFNWR0VsZW1lbnQ/KHQ9dC5vd25lclNW
R0VsZW1lbnR8fHQpLmhhc0F0dHJpYnV0ZSgidmlld0JveCIpP1tbKHQ9dC52aWV3Qm94LmJhc2VW
YWwpLngsdC55XSxbdC54K3Qud2lkdGgsdC55K3QuaGVpZ2h0XV06W1swLDBdLFt0LndpZHRoLmJh
c2VWYWwudmFsdWUsdC5oZWlnaHQuYmFzZVZhbC52YWx1ZV1dOltbMCwwXSxbdC5jbGllbnRXaWR0
aCx0LmNsaWVudEhlaWdodF1dfWZ1bmN0aW9uIGt3KCl7cmV0dXJuIHRoaXMuX196b29tfHxNd31m
dW5jdGlvbiBDdyh0KXtyZXR1cm4tdC5kZWx0YVkqKDE9PT10LmRlbHRhTW9kZT8uMDU6dC5kZWx0
YU1vZGU/MTouMDAyKSoodC5jdHJsS2V5PzEwOjEpfWZ1bmN0aW9uIFB3KCl7cmV0dXJuIG5hdmln
YXRvci5tYXhUb3VjaFBvaW50c3x8Im9udG91Y2hzdGFydCJpbiB0aGlzfWZ1bmN0aW9uIHp3KHQs
bixlKXt2YXIgcj10LmludmVydFgoblswXVswXSktZVswXVswXSxpPXQuaW52ZXJ0WChuWzFdWzBd
KS1lWzFdWzBdLG89dC5pbnZlcnRZKG5bMF1bMV0pLWVbMF1bMV0sYT10LmludmVydFkoblsxXVsx
XSktZVsxXVsxXTtyZXR1cm4gdC50cmFuc2xhdGUoaT5yPyhyK2kpLzI6TWF0aC5taW4oMCxyKXx8
TWF0aC5tYXgoMCxpKSxhPm8/KG8rYSkvMjpNYXRoLm1pbigwLG8pfHxNYXRoLm1heCgwLGEpKX1U
dy5wcm90b3R5cGU9d3cucHJvdG90eXBlLHQuQWRkZXI9QSx0LkRlbGF1bmF5PWp1LHQuRm9ybWF0
U3BlY2lmaWVyPW5mLHQuSW50ZXJuTWFwPUludGVybk1hcCx0LkludGVyblNldD1JbnRlcm5TZXQs
dC5Ob2RlPUpkLHQuUGF0aD1JYSx0LlZvcm9ub2k9VXUsdC5ab29tVHJhbnNmb3JtPXd3LHQuYWN0
aXZlPWZ1bmN0aW9uKHQsbil7dmFyIGUscixpPXQuX190cmFuc2l0aW9uO2lmKGkpZm9yKHIgaW4g
bj1udWxsPT1uP251bGw6bisiIixpKWlmKChlPWlbcl0pLnN0YXRlPlVpJiZlLm5hbWU9PT1uKXJl
dHVybiBuZXcgZ28oW1t0XV0sS28sbiwrcik7cmV0dXJuIG51bGx9LHQuYXJjPWZ1bmN0aW9uKCl7
dmFyIHQ9Q20sbj1QbSxlPXltKDApLHI9bnVsbCxpPXptLG89JG0sYT1EbSx1PW51bGwsYz1rbShm
KTtmdW5jdGlvbiBmKCl7dmFyIGYscyxsPSt0LmFwcGx5KHRoaXMsYXJndW1lbnRzKSxoPStuLmFw
cGx5KHRoaXMsYXJndW1lbnRzKSxkPWkuYXBwbHkodGhpcyxhcmd1bWVudHMpLVNtLHA9by5hcHBs
eSh0aGlzLGFyZ3VtZW50cyktU20sZz12bShwLWQpLHk9cD5kO2lmKHV8fCh1PWY9YygpKSxoPGwm
JihzPWgsaD1sLGw9cyksaD5UbSlpZihnPkVtLVRtKXUubW92ZVRvKGgqYm0oZCksaCp3bShkKSks
dS5hcmMoMCwwLGgsZCxwLCF5KSxsPlRtJiYodS5tb3ZlVG8obCpibShwKSxsKndtKHApKSx1LmFy
YygwLDAsbCxwLGQseSkpO2Vsc2V7dmFyIHYsXyxiPWQsbT1wLHg9ZCx3PXAsTT1nLFQ9ZyxBPWEu
YXBwbHkodGhpcyxhcmd1bWVudHMpLzIsUz1BPlRtJiYocj8rci5hcHBseSh0aGlzLGFyZ3VtZW50
cyk6TW0obCpsK2gqaCkpLEU9eG0odm0oaC1sKS8yLCtlLmFwcGx5KHRoaXMsYXJndW1lbnRzKSks
Tj1FLGs9RTtpZihTPlRtKXt2YXIgQz1ObShTL2wqd20oQSkpLFA9Tm0oUy9oKndtKEEpKTsoTS09
MipDKT5UbT8oeCs9Qyo9eT8xOi0xLHctPUMpOihNPTAseD13PShkK3ApLzIpLChULT0yKlApPlRt
PyhiKz1QKj15PzE6LTEsbS09UCk6KFQ9MCxiPW09KGQrcCkvMil9dmFyIHo9aCpibShiKSwkPWgq
d20oYiksRD1sKmJtKHcpLFI9bCp3bSh3KTtpZihFPlRtKXt2YXIgRixxPWgqYm0obSksVT1oKndt
KG0pLEk9bCpibSh4KSxPPWwqd20oeCk7aWYoZzxBbSlpZihGPWZ1bmN0aW9uKHQsbixlLHIsaSxv
LGEsdSl7dmFyIGM9ZS10LGY9ci1uLHM9YS1pLGw9dS1vLGg9bCpjLXMqZjtpZighKGgqaDxUbSkp
cmV0dXJuW3QrKGg9KHMqKG4tbyktbCoodC1pKSkvaCkqYyxuK2gqZl19KHosJCxJLE8scSxVLEQs
Uikpe3ZhciBCPXotRlswXSxZPSQtRlsxXSxMPXEtRlswXSxqPVUtRlsxXSxIPTEvd20oZnVuY3Rp
b24odCl7cmV0dXJuIHQ+MT8wOnQ8LTE/QW06TWF0aC5hY29zKHQpfSgoQipMK1kqaikvKE1tKEIq
QitZKlkpKk1tKEwqTCtqKmopKSkvMiksWD1NbShGWzBdKkZbMF0rRlsxXSpGWzFdKTtOPXhtKEUs
KGwtWCkvKEgtMSkpLGs9eG0oRSwoaC1YKS8oSCsxKSl9ZWxzZSBOPWs9MH1UPlRtP2s+VG0/KHY9
Um0oSSxPLHosJCxoLGsseSksXz1SbShxLFUsRCxSLGgsayx5KSx1Lm1vdmVUbyh2LmN4K3YueDAx
LHYuY3krdi55MDEpLGs8RT91LmFyYyh2LmN4LHYuY3ksayxfbSh2LnkwMSx2LngwMSksX20oXy55
MDEsXy54MDEpLCF5KToodS5hcmModi5jeCx2LmN5LGssX20odi55MDEsdi54MDEpLF9tKHYueTEx
LHYueDExKSwheSksdS5hcmMoMCwwLGgsX20odi5jeSt2LnkxMSx2LmN4K3YueDExKSxfbShfLmN5
K18ueTExLF8uY3grXy54MTEpLCF5KSx1LmFyYyhfLmN4LF8uY3ksayxfbShfLnkxMSxfLngxMSks
X20oXy55MDEsXy54MDEpLCF5KSkpOih1Lm1vdmVUbyh6LCQpLHUuYXJjKDAsMCxoLGIsbSwheSkp
OnUubW92ZVRvKHosJCksbD5UbSYmTT5UbT9OPlRtPyh2PVJtKEQsUixxLFUsbCwtTix5KSxfPVJt
KHosJCxJLE8sbCwtTix5KSx1LmxpbmVUbyh2LmN4K3YueDAxLHYuY3krdi55MDEpLE48RT91LmFy
Yyh2LmN4LHYuY3ksTixfbSh2LnkwMSx2LngwMSksX20oXy55MDEsXy54MDEpLCF5KToodS5hcmMo
di5jeCx2LmN5LE4sX20odi55MDEsdi54MDEpLF9tKHYueTExLHYueDExKSwheSksdS5hcmMoMCww
LGwsX20odi5jeSt2LnkxMSx2LmN4K3YueDExKSxfbShfLmN5K18ueTExLF8uY3grXy54MTEpLHkp
LHUuYXJjKF8uY3gsXy5jeSxOLF9tKF8ueTExLF8ueDExKSxfbShfLnkwMSxfLngwMSksIXkpKSk6
dS5hcmMoMCwwLGwsdyx4LHkpOnUubGluZVRvKEQsUil9ZWxzZSB1Lm1vdmVUbygwLDApO2lmKHUu
Y2xvc2VQYXRoKCksZilyZXR1cm4gdT1udWxsLGYrIiJ8fG51bGx9cmV0dXJuIGYuY2VudHJvaWQ9
ZnVuY3Rpb24oKXt2YXIgZT0oK3QuYXBwbHkodGhpcyxhcmd1bWVudHMpKyArbi5hcHBseSh0aGlz
LGFyZ3VtZW50cykpLzIscj0oK2kuYXBwbHkodGhpcyxhcmd1bWVudHMpKyArby5hcHBseSh0aGlz
LGFyZ3VtZW50cykpLzItQW0vMjtyZXR1cm5bYm0ocikqZSx3bShyKSplXX0sZi5pbm5lclJhZGl1
cz1mdW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD0iZnVuY3Rpb24iPT10eXBl
b2Ygbj9uOnltKCtuKSxmKTp0fSxmLm91dGVyUmFkaXVzPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1
bWVudHMubGVuZ3RoPyhuPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6eW0oK3QpLGYpOm59LGYuY29y
bmVyUmFkaXVzPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlPSJmdW5jdGlv
biI9PXR5cGVvZiB0P3Q6eW0oK3QpLGYpOmV9LGYucGFkUmFkaXVzPWZ1bmN0aW9uKHQpe3JldHVy
biBhcmd1bWVudHMubGVuZ3RoPyhyPW51bGw9PXQ/bnVsbDoiZnVuY3Rpb24iPT10eXBlb2YgdD90
OnltKCt0KSxmKTpyfSxmLnN0YXJ0QW5nbGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/KGk9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp5bSgrdCksZik6aX0sZi5lbmRBbmdsZT1m
dW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obz0iZnVuY3Rpb24iPT10eXBlb2Yg
dD90OnltKCt0KSxmKTpvfSxmLnBhZEFuZ2xlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPyhhPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6eW0oK3QpLGYpOmF9LGYuY29udGV4dD1m
dW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odT1udWxsPT10P251bGw6dCxmKTp1
fSxmfSx0LmFyZWE9TG0sdC5hcmVhUmFkaWFsPUttLHQuYXNjZW5kaW5nPW4sdC5hdXRvVHlwZT1m
dW5jdGlvbih0KXtmb3IodmFyIG4gaW4gdCl7dmFyIGUscixpPXRbbl0udHJpbSgpO2lmKGkpaWYo
InRydWUiPT09aSlpPSEwO2Vsc2UgaWYoImZhbHNlIj09PWkpaT0hMTtlbHNlIGlmKCJOYU4iPT09
aSlpPU5hTjtlbHNlIGlmKGlzTmFOKGU9K2kpKXtpZighKHI9aS5tYXRjaCgvXihbLStdXGR7Mn0p
P1xkezR9KC1cZHsyfSgtXGR7Mn0pPyk/KFRcZHsyfTpcZHsyfSg6XGR7Mn0oXC5cZHszfSk/KT8o
WnxbLStdXGR7Mn06XGR7Mn0pPyk/JC8pKSljb250aW51ZTt2YyYmcls0XSYmIXJbN10mJihpPWku
cmVwbGFjZSgvLS9nLCIvIikucmVwbGFjZSgvVC8sIiAiKSksaT1uZXcgRGF0ZShpKX1lbHNlIGk9
ZTtlbHNlIGk9bnVsbDt0W25dPWl9cmV0dXJuIHR9LHQuYXhpc0JvdHRvbT1mdW5jdGlvbih0KXty
ZXR1cm4genQoVHQsdCl9LHQuYXhpc0xlZnQ9ZnVuY3Rpb24odCl7cmV0dXJuIHp0KEF0LHQpfSx0
LmF4aXNSaWdodD1mdW5jdGlvbih0KXtyZXR1cm4genQoTXQsdCl9LHQuYXhpc1RvcD1mdW5jdGlv
bih0KXtyZXR1cm4genQod3QsdCl9LHQuYmluPUosdC5iaXNlY3Q9bCx0LmJpc2VjdENlbnRlcj1z
LHQuYmlzZWN0TGVmdD1mLHQuYmlzZWN0UmlnaHQ9Yyx0LmJpc2VjdG9yPXIsdC5ibG9iPWZ1bmN0
aW9uKHQsbil7cmV0dXJuIGZldGNoKHQsbikudGhlbihfYyl9LHQuYmx1cj1mdW5jdGlvbih0LG4p
e2lmKCEoKG49K24pPj0wKSl0aHJvdyBuZXcgUmFuZ2VFcnJvcigiaW52YWxpZCByIik7bGV0IGU9
dC5sZW5ndGg7aWYoISgoZT1NYXRoLmZsb29yKGUpKT49MCkpdGhyb3cgbmV3IFJhbmdlRXJyb3Io
ImludmFsaWQgbGVuZ3RoIik7aWYoIWV8fCFuKXJldHVybiB0O2NvbnN0IHI9dihuKSxpPXQuc2xp
Y2UoKTtyZXR1cm4gcih0LGksMCxlLDEpLHIoaSx0LDAsZSwxKSxyKHQsaSwwLGUsMSksdH0sdC5i
bHVyMj1oLHQuYmx1ckltYWdlPWQsdC5icnVzaD1mdW5jdGlvbigpe3JldHVybiBNYShoYSl9LHQu
YnJ1c2hTZWxlY3Rpb249ZnVuY3Rpb24odCl7dmFyIG49dC5fX2JydXNoO3JldHVybiBuP24uZGlt
Lm91dHB1dChuLnNlbGVjdGlvbik6bnVsbH0sdC5icnVzaFg9ZnVuY3Rpb24oKXtyZXR1cm4gTWEo
c2EpfSx0LmJydXNoWT1mdW5jdGlvbigpe3JldHVybiBNYShsYSl9LHQuYnVmZmVyPWZ1bmN0aW9u
KHQsbil7cmV0dXJuIGZldGNoKHQsbikudGhlbihiYyl9LHQuY2hvcmQ9ZnVuY3Rpb24oKXtyZXR1
cm4gJGEoITEsITEpfSx0LmNob3JkRGlyZWN0ZWQ9ZnVuY3Rpb24oKXtyZXR1cm4gJGEoITAsITEp
fSx0LmNob3JkVHJhbnNwb3NlPWZ1bmN0aW9uKCl7cmV0dXJuICRhKCExLCEwKX0sdC5jbHVzdGVy
PWZ1bmN0aW9uKCl7dmFyIHQ9amQsbj0xLGU9MSxyPSExO2Z1bmN0aW9uIGkoaSl7dmFyIG8sYT0w
O2kuZWFjaEFmdGVyKChmdW5jdGlvbihuKXt2YXIgZT1uLmNoaWxkcmVuO2U/KG4ueD1mdW5jdGlv
bih0KXtyZXR1cm4gdC5yZWR1Y2UoSGQsMCkvdC5sZW5ndGh9KGUpLG4ueT1mdW5jdGlvbih0KXty
ZXR1cm4gMSt0LnJlZHVjZShYZCwwKX0oZSkpOihuLng9bz9hKz10KG4sbyk6MCxuLnk9MCxvPW4p
fSkpO3ZhciB1PWZ1bmN0aW9uKHQpe2Zvcih2YXIgbjtuPXQuY2hpbGRyZW47KXQ9blswXTtyZXR1
cm4gdH0oaSksYz1mdW5jdGlvbih0KXtmb3IodmFyIG47bj10LmNoaWxkcmVuOyl0PW5bbi5sZW5n
dGgtMV07cmV0dXJuIHR9KGkpLGY9dS54LXQodSxjKS8yLHM9Yy54K3QoYyx1KS8yO3JldHVybiBp
LmVhY2hBZnRlcihyP2Z1bmN0aW9uKHQpe3QueD0odC54LWkueCkqbix0Lnk9KGkueS10LnkpKmV9
OmZ1bmN0aW9uKHQpe3QueD0odC54LWYpLyhzLWYpKm4sdC55PSgxLShpLnk/dC55L2kueToxKSkq
ZX0pfXJldHVybiBpLnNlcGFyYXRpb249ZnVuY3Rpb24obil7cmV0dXJuIGFyZ3VtZW50cy5sZW5n
dGg/KHQ9bixpKTp0fSxpLnNpemU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KHI9ITEsbj0rdFswXSxlPSt0WzFdLGkpOnI/bnVsbDpbbixlXX0saS5ub2RlU2l6ZT1mdW5jdGlv
bih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj0hMCxuPSt0WzBdLGU9K3RbMV0saSk6cj9b
bixlXTpudWxsfSxpfSx0LmNvbG9yPSRlLHQuY29udG91ckRlbnNpdHk9ZnVuY3Rpb24oKXt2YXIg
dD1zdSxuPWx1LGU9aHUscj05NjAsaT01MDAsbz0yMCxhPTIsdT0zKm8sYz1yKzIqdT4+YSxmPWkr
Mip1Pj5hLHM9SmEoMjApO2Z1bmN0aW9uIGwocil7dmFyIGk9bmV3IEZsb2F0MzJBcnJheShjKmYp
LHM9TWF0aC5wb3coMiwtYSksbD0tMTtmb3IoY29uc3QgbyBvZiByKXt2YXIgZD0odChvLCsrbCxy
KSt1KSpzLHA9KG4obyxsLHIpK3UpKnMsZz0rZShvLGwscik7aWYoZyYmZD49MCYmZDxjJiZwPj0w
JiZwPGYpe3ZhciB5PU1hdGguZmxvb3IoZCksdj1NYXRoLmZsb29yKHApLF89ZC15LS41LGI9cC12
LS41O2lbeSt2KmNdKz0oMS1fKSooMS1iKSpnLGlbeSsxK3YqY10rPV8qKDEtYikqZyxpW3krMSso
disxKSpjXSs9XypiKmcsaVt5Kyh2KzEpKmNdKz0oMS1fKSpiKmd9fXJldHVybiBoKHtkYXRhOmks
d2lkdGg6YyxoZWlnaHQ6Zn0sbypzKSxpfWZ1bmN0aW9uIGQodCl7dmFyIG49bCh0KSxlPXMobiks
cj1NYXRoLnBvdygyLDIqYSk7cmV0dXJuIEFycmF5LmlzQXJyYXkoZSl8fChlPVYoTnVtYmVyLk1J
Tl9WQUxVRSx0dChuKS9yLGUpKSxvdSgpLnNpemUoW2MsZl0pLnRocmVzaG9sZHMoZS5tYXAoKHQ9
PnQqcikpKShuKS5tYXAoKCh0LG4pPT4odC52YWx1ZT0rZVtuXSxwKHQpKSkpfWZ1bmN0aW9uIHAo
dCl7cmV0dXJuIHQuY29vcmRpbmF0ZXMuZm9yRWFjaChnKSx0fWZ1bmN0aW9uIGcodCl7dC5mb3JF
YWNoKHkpfWZ1bmN0aW9uIHkodCl7dC5mb3JFYWNoKHYpfWZ1bmN0aW9uIHYodCl7dFswXT10WzBd
Kk1hdGgucG93KDIsYSktdSx0WzFdPXRbMV0qTWF0aC5wb3coMixhKS11fWZ1bmN0aW9uIF8oKXty
ZXR1cm4gYz1yKzIqKHU9MypvKT4+YSxmPWkrMip1Pj5hLGR9cmV0dXJuIGQuY29udG91cnM9ZnVu
Y3Rpb24odCl7dmFyIG49bCh0KSxlPW91KCkuc2l6ZShbYyxmXSkscj1NYXRoLnBvdygyLDIqYSks
aT10PT57dD0rdDt2YXIgaT1wKGUuY29udG91cihuLHQqcikpO3JldHVybiBpLnZhbHVlPXQsaX07
cmV0dXJuIE9iamVjdC5kZWZpbmVQcm9wZXJ0eShpLCJtYXgiLHtnZXQ6KCk9PnR0KG4pL3J9KSxp
fSxkLng9ZnVuY3Rpb24obil7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHQ9ImZ1bmN0aW9uIj09
dHlwZW9mIG4/bjpKYSgrbiksZCk6dH0sZC55PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPyhuPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6SmEoK3QpLGQpOm59LGQud2VpZ2h0PWZ1
bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlPSJmdW5jdGlvbiI9PXR5cGVvZiB0
P3Q6SmEoK3QpLGQpOmV9LGQuc2l6ZT1mdW5jdGlvbih0KXtpZighYXJndW1lbnRzLmxlbmd0aCly
ZXR1cm5bcixpXTt2YXIgbj0rdFswXSxlPSt0WzFdO2lmKCEobj49MCYmZT49MCkpdGhyb3cgbmV3
IEVycm9yKCJpbnZhbGlkIHNpemUiKTtyZXR1cm4gcj1uLGk9ZSxfKCl9LGQuY2VsbFNpemU9ZnVu
Y3Rpb24odCl7aWYoIWFyZ3VtZW50cy5sZW5ndGgpcmV0dXJuIDE8PGE7aWYoISgodD0rdCk+PTEp
KXRocm93IG5ldyBFcnJvcigiaW52YWxpZCBjZWxsIHNpemUiKTtyZXR1cm4gYT1NYXRoLmZsb29y
KE1hdGgubG9nKHQpL01hdGguTE4yKSxfKCl9LGQudGhyZXNob2xkcz1mdW5jdGlvbih0KXtyZXR1
cm4gYXJndW1lbnRzLmxlbmd0aD8ocz0iZnVuY3Rpb24iPT10eXBlb2YgdD90OkFycmF5LmlzQXJy
YXkodCk/SmEoS2EuY2FsbCh0KSk6SmEodCksZCk6c30sZC5iYW5kd2lkdGg9ZnVuY3Rpb24odCl7
aWYoIWFyZ3VtZW50cy5sZW5ndGgpcmV0dXJuIE1hdGguc3FydChvKihvKzEpKTtpZighKCh0PSt0
KT49MCkpdGhyb3cgbmV3IEVycm9yKCJpbnZhbGlkIGJhbmR3aWR0aCIpO3JldHVybiBvPShNYXRo
LnNxcnQoNCp0KnQrMSktMSkvMixfKCl9LGR9LHQuY29udG91cnM9b3UsdC5jb3VudD1fLHQuY3Jl
YXRlPWZ1bmN0aW9uKHQpe3JldHVybiBLbihMdCh0KS5jYWxsKGRvY3VtZW50LmRvY3VtZW50RWxl
bWVudCkpfSx0LmNyZWF0b3I9THQsdC5jcm9zcz1mdW5jdGlvbiguLi50KXtjb25zdCBuPSJmdW5j
dGlvbiI9PXR5cGVvZiB0W3QubGVuZ3RoLTFdJiZmdW5jdGlvbih0KXtyZXR1cm4gbj0+dCguLi5u
KX0odC5wb3AoKSksZT0odD10Lm1hcCh4KSkubWFwKGIpLHI9dC5sZW5ndGgtMSxpPW5ldyBBcnJh
eShyKzEpLmZpbGwoMCksbz1bXTtpZihyPDB8fGUuc29tZShtKSlyZXR1cm4gbztmb3IoOzspe28u
cHVzaChpLm1hcCgoKG4sZSk9PnRbZV1bbl0pKSk7bGV0IGE9cjtmb3IoOysraVthXT09PWVbYV07
KXtpZigwPT09YSlyZXR1cm4gbj9vLm1hcChuKTpvO2lbYS0tXT0wfX19LHQuY3N2PU1jLHQuY3N2
Rm9ybWF0PWljLHQuY3N2Rm9ybWF0Qm9keT1vYyx0LmNzdkZvcm1hdFJvdz11Yyx0LmNzdkZvcm1h
dFJvd3M9YWMsdC5jc3ZGb3JtYXRWYWx1ZT1jYyx0LmNzdlBhcnNlPWVjLHQuY3N2UGFyc2VSb3dz
PXJjLHQuY3ViZWhlbGl4PUFyLHQuY3Vtc3VtPWZ1bmN0aW9uKHQsbil7dmFyIGU9MCxyPTA7cmV0
dXJuIEZsb2F0NjRBcnJheS5mcm9tKHQsdm9pZCAwPT09bj90PT5lKz0rdHx8MDppPT5lKz0rbihp
LHIrKyx0KXx8MCl9LHQuY3VydmVCYXNpcz1mdW5jdGlvbih0KXtyZXR1cm4gbmV3IEZ4KHQpfSx0
LmN1cnZlQmFzaXNDbG9zZWQ9ZnVuY3Rpb24odCl7cmV0dXJuIG5ldyBxeCh0KX0sdC5jdXJ2ZUJh
c2lzT3Blbj1mdW5jdGlvbih0KXtyZXR1cm4gbmV3IFV4KHQpfSx0LmN1cnZlQnVtcFg9bngsdC5j
dXJ2ZUJ1bXBZPWV4LHQuY3VydmVCdW5kbGU9T3gsdC5jdXJ2ZUNhcmRpbmFsPUx4LHQuY3VydmVD
YXJkaW5hbENsb3NlZD1IeCx0LmN1cnZlQ2FyZGluYWxPcGVuPUd4LHQuY3VydmVDYXRtdWxsUm9t
PVp4LHQuY3VydmVDYXRtdWxsUm9tQ2xvc2VkPVF4LHQuY3VydmVDYXRtdWxsUm9tT3Blbj10dyx0
LmN1cnZlTGluZWFyPUltLHQuY3VydmVMaW5lYXJDbG9zZWQ9ZnVuY3Rpb24odCl7cmV0dXJuIG5l
dyBudyh0KX0sdC5jdXJ2ZU1vbm90b25lWD1mdW5jdGlvbih0KXtyZXR1cm4gbmV3IGF3KHQpfSx0
LmN1cnZlTW9ub3RvbmVZPWZ1bmN0aW9uKHQpe3JldHVybiBuZXcgdXcodCl9LHQuY3VydmVOYXR1
cmFsPWZ1bmN0aW9uKHQpe3JldHVybiBuZXcgZncodCl9LHQuY3VydmVTdGVwPWZ1bmN0aW9uKHQp
e3JldHVybiBuZXcgbHcodCwuNSl9LHQuY3VydmVTdGVwQWZ0ZXI9ZnVuY3Rpb24odCl7cmV0dXJu
IG5ldyBsdyh0LDEpfSx0LmN1cnZlU3RlcEJlZm9yZT1mdW5jdGlvbih0KXtyZXR1cm4gbmV3IGx3
KHQsMCl9LHQuZGVzY2VuZGluZz1lLHQuZGV2aWF0aW9uPU0sdC5kaWZmZXJlbmNlPWZ1bmN0aW9u
KHQsLi4ubil7dD1uZXcgSW50ZXJuU2V0KHQpO2Zvcihjb25zdCBlIG9mIG4pZm9yKGNvbnN0IG4g
b2YgZSl0LmRlbGV0ZShuKTtyZXR1cm4gdH0sdC5kaXNqb2ludD1mdW5jdGlvbih0LG4pe2NvbnN0
IGU9bltTeW1ib2wuaXRlcmF0b3JdKCkscj1uZXcgSW50ZXJuU2V0O2Zvcihjb25zdCBuIG9mIHQp
e2lmKHIuaGFzKG4pKXJldHVybiExO2xldCB0LGk7Zm9yKDsoe3ZhbHVlOnQsZG9uZTppfT1lLm5l
eHQoKSkmJiFpOyl7aWYoT2JqZWN0LmlzKG4sdCkpcmV0dXJuITE7ci5hZGQodCl9fXJldHVybiEw
fSx0LmRpc3BhdGNoPUR0LHQuZHJhZz1mdW5jdGlvbigpe3ZhciB0LG4sZSxyLGk9bGUsbz1oZSxh
PWRlLHU9cGUsYz17fSxmPUR0KCJzdGFydCIsImRyYWciLCJlbmQiKSxzPTAsbD0wO2Z1bmN0aW9u
IGgodCl7dC5vbigibW91c2Vkb3duLmRyYWciLGQpLmZpbHRlcih1KS5vbigidG91Y2hzdGFydC5k
cmFnIix5KS5vbigidG91Y2htb3ZlLmRyYWciLHYscmUpLm9uKCJ0b3VjaGVuZC5kcmFnIHRvdWNo
Y2FuY2VsLmRyYWciLF8pLnN0eWxlKCJ0b3VjaC1hY3Rpb24iLCJub25lIikuc3R5bGUoIi13ZWJr
aXQtdGFwLWhpZ2hsaWdodC1jb2xvciIsInJnYmEoMCwwLDAsMCkiKX1mdW5jdGlvbiBkKGEsdSl7
aWYoIXImJmkuY2FsbCh0aGlzLGEsdSkpe3ZhciBjPWIodGhpcyxvLmNhbGwodGhpcyxhLHUpLGEs
dSwibW91c2UiKTtjJiYoS24oYS52aWV3KS5vbigibW91c2Vtb3ZlLmRyYWciLHAsaWUpLm9uKCJt
b3VzZXVwLmRyYWciLGcsaWUpLHVlKGEudmlldyksb2UoYSksZT0hMSx0PWEuY2xpZW50WCxuPWEu
Y2xpZW50WSxjKCJzdGFydCIsYSkpfX1mdW5jdGlvbiBwKHIpe2lmKGFlKHIpLCFlKXt2YXIgaT1y
LmNsaWVudFgtdCxvPXIuY2xpZW50WS1uO2U9aSppK28qbz5sfWMubW91c2UoImRyYWciLHIpfWZ1
bmN0aW9uIGcodCl7S24odC52aWV3KS5vbigibW91c2Vtb3ZlLmRyYWcgbW91c2V1cC5kcmFnIixu
dWxsKSxjZSh0LnZpZXcsZSksYWUodCksYy5tb3VzZSgiZW5kIix0KX1mdW5jdGlvbiB5KHQsbil7
aWYoaS5jYWxsKHRoaXMsdCxuKSl7dmFyIGUscixhPXQuY2hhbmdlZFRvdWNoZXMsdT1vLmNhbGwo
dGhpcyx0LG4pLGM9YS5sZW5ndGg7Zm9yKGU9MDtlPGM7KytlKShyPWIodGhpcyx1LHQsbixhW2Vd
LmlkZW50aWZpZXIsYVtlXSkpJiYob2UodCkscigic3RhcnQiLHQsYVtlXSkpfX1mdW5jdGlvbiB2
KHQpe3ZhciBuLGUscj10LmNoYW5nZWRUb3VjaGVzLGk9ci5sZW5ndGg7Zm9yKG49MDtuPGk7Kytu
KShlPWNbcltuXS5pZGVudGlmaWVyXSkmJihhZSh0KSxlKCJkcmFnIix0LHJbbl0pKX1mdW5jdGlv
biBfKHQpe3ZhciBuLGUsaT10LmNoYW5nZWRUb3VjaGVzLG89aS5sZW5ndGg7Zm9yKHImJmNsZWFy
VGltZW91dChyKSxyPXNldFRpbWVvdXQoKGZ1bmN0aW9uKCl7cj1udWxsfSksNTAwKSxuPTA7bjxv
OysrbikoZT1jW2lbbl0uaWRlbnRpZmllcl0pJiYob2UodCksZSgiZW5kIix0LGlbbl0pKX1mdW5j
dGlvbiBiKHQsbixlLHIsaSxvKXt2YXIgdSxsLGQscD1mLmNvcHkoKSxnPWVlKG98fGUsbik7aWYo
bnVsbCE9KGQ9YS5jYWxsKHQsbmV3IHNlKCJiZWZvcmVzdGFydCIse3NvdXJjZUV2ZW50OmUsdGFy
Z2V0OmgsaWRlbnRpZmllcjppLGFjdGl2ZTpzLHg6Z1swXSx5OmdbMV0sZHg6MCxkeTowLGRpc3Bh
dGNoOnB9KSxyKSkpcmV0dXJuIHU9ZC54LWdbMF18fDAsbD1kLnktZ1sxXXx8MCxmdW5jdGlvbiBl
KG8sYSxmKXt2YXIgeSx2PWc7c3dpdGNoKG8pe2Nhc2Uic3RhcnQiOmNbaV09ZSx5PXMrKzticmVh
aztjYXNlImVuZCI6ZGVsZXRlIGNbaV0sLS1zO2Nhc2UiZHJhZyI6Zz1lZShmfHxhLG4pLHk9c31w
LmNhbGwobyx0LG5ldyBzZShvLHtzb3VyY2VFdmVudDphLHN1YmplY3Q6ZCx0YXJnZXQ6aCxpZGVu
dGlmaWVyOmksYWN0aXZlOnkseDpnWzBdK3UseTpnWzFdK2wsZHg6Z1swXS12WzBdLGR5OmdbMV0t
dlsxXSxkaXNwYXRjaDpwfSkscil9fXJldHVybiBoLmZpbHRlcj1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8oaT0iZnVuY3Rpb24iPT10eXBlb2YgdD90OmZlKCEhdCksaCk6aX0s
aC5jb250YWluZXI9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG89ImZ1bmN0
aW9uIj09dHlwZW9mIHQ/dDpmZSh0KSxoKTpvfSxoLnN1YmplY3Q9ZnVuY3Rpb24odCl7cmV0dXJu
IGFyZ3VtZW50cy5sZW5ndGg/KGE9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpmZSh0KSxoKTphfSxo
LnRvdWNoYWJsZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odT0iZnVuY3Rp
b24iPT10eXBlb2YgdD90OmZlKCEhdCksaCk6dX0saC5vbj1mdW5jdGlvbigpe3ZhciB0PWYub24u
YXBwbHkoZixhcmd1bWVudHMpO3JldHVybiB0PT09Zj9oOnR9LGguY2xpY2tEaXN0YW5jZT1mdW5j
dGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obD0odD0rdCkqdCxoKTpNYXRoLnNxcnQo
bCl9LGh9LHQuZHJhZ0Rpc2FibGU9dWUsdC5kcmFnRW5hYmxlPWNlLHQuZHN2PWZ1bmN0aW9uKHQs
bixlLHIpezM9PT1hcmd1bWVudHMubGVuZ3RoJiYiZnVuY3Rpb24iPT10eXBlb2YgZSYmKHI9ZSxl
PXZvaWQgMCk7dmFyIGk9dGModCk7cmV0dXJuIHhjKG4sZSkudGhlbigoZnVuY3Rpb24odCl7cmV0
dXJuIGkucGFyc2UodCxyKX0pKX0sdC5kc3ZGb3JtYXQ9dGMsdC5lYXNlQmFjaz1qbyx0LmVhc2VC
YWNrSW49WW8sdC5lYXNlQmFja0luT3V0PWpvLHQuZWFzZUJhY2tPdXQ9TG8sdC5lYXNlQm91bmNl
PU9vLHQuZWFzZUJvdW5jZUluPWZ1bmN0aW9uKHQpe3JldHVybiAxLU9vKDEtdCl9LHQuZWFzZUJv
dW5jZUluT3V0PWZ1bmN0aW9uKHQpe3JldHVybigodCo9Mik8PTE/MS1PbygxLXQpOk9vKHQtMSkr
MSkvMn0sdC5lYXNlQm91bmNlT3V0PU9vLHQuZWFzZUNpcmNsZT1rbyx0LmVhc2VDaXJjbGVJbj1m
dW5jdGlvbih0KXtyZXR1cm4gMS1NYXRoLnNxcnQoMS10KnQpfSx0LmVhc2VDaXJjbGVJbk91dD1r
byx0LmVhc2VDaXJjbGVPdXQ9ZnVuY3Rpb24odCl7cmV0dXJuIE1hdGguc3FydCgxLSAtLXQqdCl9
LHQuZWFzZUN1YmljPW1vLHQuZWFzZUN1YmljSW49ZnVuY3Rpb24odCl7cmV0dXJuIHQqdCp0fSx0
LmVhc2VDdWJpY0luT3V0PW1vLHQuZWFzZUN1YmljT3V0PWZ1bmN0aW9uKHQpe3JldHVybi0tdCp0
KnQrMX0sdC5lYXNlRWxhc3RpYz1Hbyx0LmVhc2VFbGFzdGljSW49WG8sdC5lYXNlRWxhc3RpY0lu
T3V0PVZvLHQuZWFzZUVsYXN0aWNPdXQ9R28sdC5lYXNlRXhwPU5vLHQuZWFzZUV4cEluPWZ1bmN0
aW9uKHQpe3JldHVybiBFbygxLSt0KX0sdC5lYXNlRXhwSW5PdXQ9Tm8sdC5lYXNlRXhwT3V0PWZ1
bmN0aW9uKHQpe3JldHVybiAxLUVvKHQpfSx0LmVhc2VMaW5lYXI9dD0+K3QsdC5lYXNlUG9seT1N
byx0LmVhc2VQb2x5SW49eG8sdC5lYXNlUG9seUluT3V0PU1vLHQuZWFzZVBvbHlPdXQ9d28sdC5l
YXNlUXVhZD1ibyx0LmVhc2VRdWFkSW49ZnVuY3Rpb24odCl7cmV0dXJuIHQqdH0sdC5lYXNlUXVh
ZEluT3V0PWJvLHQuZWFzZVF1YWRPdXQ9ZnVuY3Rpb24odCl7cmV0dXJuIHQqKDItdCl9LHQuZWFz
ZVNpbj1Tbyx0LmVhc2VTaW5Jbj1mdW5jdGlvbih0KXtyZXR1cm4gMT09K3Q/MToxLU1hdGguY29z
KHQqQW8pfSx0LmVhc2VTaW5Jbk91dD1Tbyx0LmVhc2VTaW5PdXQ9ZnVuY3Rpb24odCl7cmV0dXJu
IE1hdGguc2luKHQqQW8pfSx0LmV2ZXJ5PWZ1bmN0aW9uKHQsbil7aWYoImZ1bmN0aW9uIiE9dHlw
ZW9mIG4pdGhyb3cgbmV3IFR5cGVFcnJvcigidGVzdCBpcyBub3QgYSBmdW5jdGlvbiIpO2xldCBl
PS0xO2Zvcihjb25zdCByIG9mIHQpaWYoIW4ociwrK2UsdCkpcmV0dXJuITE7cmV0dXJuITB9LHQu
ZXh0ZW50PVQsdC5mY3Vtc3VtPWZ1bmN0aW9uKHQsbil7Y29uc3QgZT1uZXcgQTtsZXQgcj0tMTty
ZXR1cm4gRmxvYXQ2NEFycmF5LmZyb20odCx2b2lkIDA9PT1uP3Q9PmUuYWRkKCt0fHwwKTppPT5l
LmFkZCgrbihpLCsrcix0KXx8MCkpfSx0LmZpbHRlcj1mdW5jdGlvbih0LG4pe2lmKCJmdW5jdGlv
biIhPXR5cGVvZiBuKXRocm93IG5ldyBUeXBlRXJyb3IoInRlc3QgaXMgbm90IGEgZnVuY3Rpb24i
KTtjb25zdCBlPVtdO2xldCByPS0xO2Zvcihjb25zdCBpIG9mIHQpbihpLCsrcix0KSYmZS5wdXNo
KGkpO3JldHVybiBlfSx0LmZsYXRHcm91cD1mdW5jdGlvbih0LC4uLm4pe3JldHVybiAkKHoodCwu
Li5uKSxuKX0sdC5mbGF0Um9sbHVwPWZ1bmN0aW9uKHQsbiwuLi5lKXtyZXR1cm4gJChSKHQsbiwu
Li5lKSxlKX0sdC5mb3JjZUNlbnRlcj1mdW5jdGlvbih0LG4pe3ZhciBlLHI9MTtmdW5jdGlvbiBp
KCl7dmFyIGksbyxhPWUubGVuZ3RoLHU9MCxjPTA7Zm9yKGk9MDtpPGE7KytpKXUrPShvPWVbaV0p
LngsYys9by55O2Zvcih1PSh1L2EtdCkqcixjPShjL2EtbikqcixpPTA7aTxhOysraSkobz1lW2ld
KS54LT11LG8ueS09Y31yZXR1cm4gbnVsbD09dCYmKHQ9MCksbnVsbD09biYmKG49MCksaS5pbml0
aWFsaXplPWZ1bmN0aW9uKHQpe2U9dH0saS54PWZ1bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPyh0PStuLGkpOnR9LGkueT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0
aD8obj0rdCxpKTpufSxpLnN0cmVuZ3RoPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPyhyPSt0LGkpOnJ9LGl9LHQuZm9yY2VDb2xsaWRlPWZ1bmN0aW9uKHQpe3ZhciBuLGUscixp
PTEsbz0xO2Z1bmN0aW9uIGEoKXtmb3IodmFyIHQsYSxjLGYscyxsLGgsZD1uLmxlbmd0aCxwPTA7
cDxvOysrcClmb3IoYT1EYyhuLE9jLEJjKS52aXNpdEFmdGVyKHUpLHQ9MDt0PGQ7Kyt0KWM9blt0
XSxsPWVbYy5pbmRleF0saD1sKmwsZj1jLngrYy52eCxzPWMueStjLnZ5LGEudmlzaXQoZyk7ZnVu
Y3Rpb24gZyh0LG4sZSxvLGEpe3ZhciB1PXQuZGF0YSxkPXQucixwPWwrZDtpZighdSlyZXR1cm4g
bj5mK3B8fG88Zi1wfHxlPnMrcHx8YTxzLXA7aWYodS5pbmRleD5jLmluZGV4KXt2YXIgZz1mLXUu
eC11LnZ4LHk9cy11LnktdS52eSx2PWcqZyt5Knk7djxwKnAmJigwPT09ZyYmKHYrPShnPUljKHIp
KSpnKSwwPT09eSYmKHYrPSh5PUljKHIpKSp5KSx2PShwLSh2PU1hdGguc3FydCh2KSkpL3YqaSxj
LnZ4Kz0oZyo9dikqKHA9KGQqPWQpLyhoK2QpKSxjLnZ5Kz0oeSo9dikqcCx1LnZ4LT1nKihwPTEt
cCksdS52eS09eSpwKX19fWZ1bmN0aW9uIHUodCl7aWYodC5kYXRhKXJldHVybiB0LnI9ZVt0LmRh
dGEuaW5kZXhdO2Zvcih2YXIgbj10LnI9MDtuPDQ7KytuKXRbbl0mJnRbbl0ucj50LnImJih0LnI9
dFtuXS5yKX1mdW5jdGlvbiBjKCl7aWYobil7dmFyIHIsaSxvPW4ubGVuZ3RoO2ZvcihlPW5ldyBB
cnJheShvKSxyPTA7cjxvOysrcilpPW5bcl0sZVtpLmluZGV4XT0rdChpLHIsbil9fXJldHVybiJm
dW5jdGlvbiIhPXR5cGVvZiB0JiYodD1VYyhudWxsPT10PzE6K3QpKSxhLmluaXRpYWxpemU9ZnVu
Y3Rpb24odCxlKXtuPXQscj1lLGMoKX0sYS5pdGVyYXRpb25zPWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyhvPSt0LGEpOm99LGEuc3RyZW5ndGg9ZnVuY3Rpb24odCl7cmV0dXJu
IGFyZ3VtZW50cy5sZW5ndGg/KGk9K3QsYSk6aX0sYS5yYWRpdXM9ZnVuY3Rpb24obil7cmV0dXJu
IGFyZ3VtZW50cy5sZW5ndGg/KHQ9ImZ1bmN0aW9uIj09dHlwZW9mIG4/bjpVYygrbiksYygpLGEp
OnR9LGF9LHQuZm9yY2VMaW5rPWZ1bmN0aW9uKHQpe3ZhciBuLGUscixpLG8sYSx1PVljLGM9ZnVu
Y3Rpb24odCl7cmV0dXJuIDEvTWF0aC5taW4oaVt0LnNvdXJjZS5pbmRleF0saVt0LnRhcmdldC5p
bmRleF0pfSxmPVVjKDMwKSxzPTE7ZnVuY3Rpb24gbChyKXtmb3IodmFyIGk9MCx1PXQubGVuZ3Ro
O2k8czsrK2kpZm9yKHZhciBjLGYsbCxoLGQscCxnLHk9MDt5PHU7Kyt5KWY9KGM9dFt5XSkuc291
cmNlLGg9KGw9Yy50YXJnZXQpLngrbC52eC1mLngtZi52eHx8SWMoYSksZD1sLnkrbC52eS1mLnkt
Zi52eXx8SWMoYSksaCo9cD0oKHA9TWF0aC5zcXJ0KGgqaCtkKmQpKS1lW3ldKS9wKnIqblt5XSxk
Kj1wLGwudngtPWgqKGc9b1t5XSksbC52eS09ZCpnLGYudngrPWgqKGc9MS1nKSxmLnZ5Kz1kKmd9
ZnVuY3Rpb24gaCgpe2lmKHIpe3ZhciBhLGMsZj1yLmxlbmd0aCxzPXQubGVuZ3RoLGw9bmV3IE1h
cChyLm1hcCgoKHQsbik9Plt1KHQsbixyKSx0XSkpKTtmb3IoYT0wLGk9bmV3IEFycmF5KGYpO2E8
czsrK2EpKGM9dFthXSkuaW5kZXg9YSwib2JqZWN0IiE9dHlwZW9mIGMuc291cmNlJiYoYy5zb3Vy
Y2U9TGMobCxjLnNvdXJjZSkpLCJvYmplY3QiIT10eXBlb2YgYy50YXJnZXQmJihjLnRhcmdldD1M
YyhsLGMudGFyZ2V0KSksaVtjLnNvdXJjZS5pbmRleF09KGlbYy5zb3VyY2UuaW5kZXhdfHwwKSsx
LGlbYy50YXJnZXQuaW5kZXhdPShpW2MudGFyZ2V0LmluZGV4XXx8MCkrMTtmb3IoYT0wLG89bmV3
IEFycmF5KHMpO2E8czsrK2EpYz10W2FdLG9bYV09aVtjLnNvdXJjZS5pbmRleF0vKGlbYy5zb3Vy
Y2UuaW5kZXhdK2lbYy50YXJnZXQuaW5kZXhdKTtuPW5ldyBBcnJheShzKSxkKCksZT1uZXcgQXJy
YXkocykscCgpfX1mdW5jdGlvbiBkKCl7aWYocilmb3IodmFyIGU9MCxpPXQubGVuZ3RoO2U8aTsr
K2UpbltlXT0rYyh0W2VdLGUsdCl9ZnVuY3Rpb24gcCgpe2lmKHIpZm9yKHZhciBuPTAsaT10Lmxl
bmd0aDtuPGk7KytuKWVbbl09K2YodFtuXSxuLHQpfXJldHVybiBudWxsPT10JiYodD1bXSksbC5p
bml0aWFsaXplPWZ1bmN0aW9uKHQsbil7cj10LGE9bixoKCl9LGwubGlua3M9ZnVuY3Rpb24obil7
cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHQ9bixoKCksbCk6dH0sbC5pZD1mdW5jdGlvbih0KXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odT10LGwpOnV9LGwuaXRlcmF0aW9ucz1mdW5jdGlvbih0
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocz0rdCxsKTpzfSxsLnN0cmVuZ3RoPWZ1bmN0aW9u
KHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhjPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6VWMo
K3QpLGQoKSxsKTpjfSxsLmRpc3RhbmNlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPyhmPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6VWMoK3QpLHAoKSxsKTpmfSxsfSx0LmZvcmNl
TWFueUJvZHk9ZnVuY3Rpb24oKXt2YXIgdCxuLGUscixpLG89VWMoLTMwKSxhPTEsdT0xLzAsYz0u
ODE7ZnVuY3Rpb24gZihlKXt2YXIgaSxvPXQubGVuZ3RoLGE9RGModCxHYyxWYykudmlzaXRBZnRl
cihsKTtmb3Iocj1lLGk9MDtpPG87KytpKW49dFtpXSxhLnZpc2l0KGgpfWZ1bmN0aW9uIHMoKXtp
Zih0KXt2YXIgbixlLHI9dC5sZW5ndGg7Zm9yKGk9bmV3IEFycmF5KHIpLG49MDtuPHI7KytuKWU9
dFtuXSxpW2UuaW5kZXhdPStvKGUsbix0KX19ZnVuY3Rpb24gbCh0KXt2YXIgbixlLHIsbyxhLHU9
MCxjPTA7aWYodC5sZW5ndGgpe2ZvcihyPW89YT0wO2E8NDsrK2EpKG49dFthXSkmJihlPU1hdGgu
YWJzKG4udmFsdWUpKSYmKHUrPW4udmFsdWUsYys9ZSxyKz1lKm4ueCxvKz1lKm4ueSk7dC54PXIv
Yyx0Lnk9by9jfWVsc2V7KG49dCkueD1uLmRhdGEueCxuLnk9bi5kYXRhLnk7ZG97dSs9aVtuLmRh
dGEuaW5kZXhdfXdoaWxlKG49bi5uZXh0KX10LnZhbHVlPXV9ZnVuY3Rpb24gaCh0LG8sZixzKXtp
ZighdC52YWx1ZSlyZXR1cm4hMDt2YXIgbD10Lngtbi54LGg9dC55LW4ueSxkPXMtbyxwPWwqbCto
Kmg7aWYoZCpkL2M8cClyZXR1cm4gcDx1JiYoMD09PWwmJihwKz0obD1JYyhlKSkqbCksMD09PWgm
JihwKz0oaD1JYyhlKSkqaCkscDxhJiYocD1NYXRoLnNxcnQoYSpwKSksbi52eCs9bCp0LnZhbHVl
KnIvcCxuLnZ5Kz1oKnQudmFsdWUqci9wKSwhMDtpZighKHQubGVuZ3RofHxwPj11KSl7KHQuZGF0
YSE9PW58fHQubmV4dCkmJigwPT09bCYmKHArPShsPUljKGUpKSpsKSwwPT09aCYmKHArPShoPUlj
KGUpKSpoKSxwPGEmJihwPU1hdGguc3FydChhKnApKSk7ZG97dC5kYXRhIT09biYmKGQ9aVt0LmRh
dGEuaW5kZXhdKnIvcCxuLnZ4Kz1sKmQsbi52eSs9aCpkKX13aGlsZSh0PXQubmV4dCl9fXJldHVy
biBmLmluaXRpYWxpemU9ZnVuY3Rpb24obixyKXt0PW4sZT1yLHMoKX0sZi5zdHJlbmd0aD1mdW5j
dGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obz0iZnVuY3Rpb24iPT10eXBlb2YgdD90
OlVjKCt0KSxzKCksZik6b30sZi5kaXN0YW5jZU1pbj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1l
bnRzLmxlbmd0aD8oYT10KnQsZik6TWF0aC5zcXJ0KGEpfSxmLmRpc3RhbmNlTWF4PWZ1bmN0aW9u
KHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh1PXQqdCxmKTpNYXRoLnNxcnQodSl9LGYudGhl
dGE9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGM9dCp0LGYpOk1hdGguc3Fy
dChjKX0sZn0sdC5mb3JjZVJhZGlhbD1mdW5jdGlvbih0LG4sZSl7dmFyIHIsaSxvLGE9VWMoLjEp
O2Z1bmN0aW9uIHUodCl7Zm9yKHZhciBhPTAsdT1yLmxlbmd0aDthPHU7KythKXt2YXIgYz1yW2Fd
LGY9Yy54LW58fDFlLTYscz1jLnktZXx8MWUtNixsPU1hdGguc3FydChmKmYrcypzKSxoPShvW2Fd
LWwpKmlbYV0qdC9sO2MudngrPWYqaCxjLnZ5Kz1zKmh9fWZ1bmN0aW9uIGMoKXtpZihyKXt2YXIg
bixlPXIubGVuZ3RoO2ZvcihpPW5ldyBBcnJheShlKSxvPW5ldyBBcnJheShlKSxuPTA7bjxlOysr
bilvW25dPSt0KHJbbl0sbixyKSxpW25dPWlzTmFOKG9bbl0pPzA6K2EocltuXSxuLHIpfX1yZXR1
cm4iZnVuY3Rpb24iIT10eXBlb2YgdCYmKHQ9VWMoK3QpKSxudWxsPT1uJiYobj0wKSxudWxsPT1l
JiYoZT0wKSx1LmluaXRpYWxpemU9ZnVuY3Rpb24odCl7cj10LGMoKX0sdS5zdHJlbmd0aD1mdW5j
dGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYT0iZnVuY3Rpb24iPT10eXBlb2YgdD90
OlVjKCt0KSxjKCksdSk6YX0sdS5yYWRpdXM9ZnVuY3Rpb24obil7cmV0dXJuIGFyZ3VtZW50cy5s
ZW5ndGg/KHQ9ImZ1bmN0aW9uIj09dHlwZW9mIG4/bjpVYygrbiksYygpLHUpOnR9LHUueD1mdW5j
dGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obj0rdCx1KTpufSx1Lnk9ZnVuY3Rpb24o
dCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGU9K3QsdSk6ZX0sdX0sdC5mb3JjZVNpbXVsYXRp
b249ZnVuY3Rpb24odCl7dmFyIG4sZT0xLHI9LjAwMSxpPTEtTWF0aC5wb3cociwxLzMwMCksbz0w
LGE9LjYsdT1uZXcgTWFwLGM9a2kobCksZj1EdCgidGljayIsImVuZCIpLHM9ZnVuY3Rpb24oKXts
ZXQgdD0xO3JldHVybigpPT4odD0oamMqdCtIYyklWGMpL1hjfSgpO2Z1bmN0aW9uIGwoKXtoKCks
Zi5jYWxsKCJ0aWNrIixuKSxlPHImJihjLnN0b3AoKSxmLmNhbGwoImVuZCIsbikpfWZ1bmN0aW9u
IGgocil7dmFyIGMsZixzPXQubGVuZ3RoO3ZvaWQgMD09PXImJihyPTEpO2Zvcih2YXIgbD0wO2w8
cjsrK2wpZm9yKGUrPShvLWUpKmksdS5mb3JFYWNoKChmdW5jdGlvbih0KXt0KGUpfSkpLGM9MDtj
PHM7KytjKW51bGw9PShmPXRbY10pLmZ4P2YueCs9Zi52eCo9YTooZi54PWYuZngsZi52eD0wKSxu
dWxsPT1mLmZ5P2YueSs9Zi52eSo9YTooZi55PWYuZnksZi52eT0wKTtyZXR1cm4gbn1mdW5jdGlv
biBkKCl7Zm9yKHZhciBuLGU9MCxyPXQubGVuZ3RoO2U8cjsrK2Upe2lmKChuPXRbZV0pLmluZGV4
PWUsbnVsbCE9bi5meCYmKG4ueD1uLmZ4KSxudWxsIT1uLmZ5JiYobi55PW4uZnkpLGlzTmFOKG4u
eCl8fGlzTmFOKG4ueSkpe3ZhciBpPTEwKk1hdGguc3FydCguNStlKSxvPWUqV2M7bi54PWkqTWF0
aC5jb3Mobyksbi55PWkqTWF0aC5zaW4obyl9KGlzTmFOKG4udngpfHxpc05hTihuLnZ5KSkmJihu
LnZ4PW4udnk9MCl9fWZ1bmN0aW9uIHAobil7cmV0dXJuIG4uaW5pdGlhbGl6ZSYmbi5pbml0aWFs
aXplKHQscyksbn1yZXR1cm4gbnVsbD09dCYmKHQ9W10pLGQoKSxuPXt0aWNrOmgscmVzdGFydDpm
dW5jdGlvbigpe3JldHVybiBjLnJlc3RhcnQobCksbn0sc3RvcDpmdW5jdGlvbigpe3JldHVybiBj
LnN0b3AoKSxufSxub2RlczpmdW5jdGlvbihlKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD1l
LGQoKSx1LmZvckVhY2gocCksbik6dH0sYWxwaGE6ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50
cy5sZW5ndGg/KGU9K3Qsbik6ZX0sYWxwaGFNaW46ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50
cy5sZW5ndGg/KHI9K3Qsbik6cn0sYWxwaGFEZWNheTpmdW5jdGlvbih0KXtyZXR1cm4gYXJndW1l
bnRzLmxlbmd0aD8oaT0rdCxuKToraX0sYWxwaGFUYXJnZXQ6ZnVuY3Rpb24odCl7cmV0dXJuIGFy
Z3VtZW50cy5sZW5ndGg/KG89K3Qsbik6b30sdmVsb2NpdHlEZWNheTpmdW5jdGlvbih0KXtyZXR1
cm4gYXJndW1lbnRzLmxlbmd0aD8oYT0xLXQsbik6MS1hfSxyYW5kb21Tb3VyY2U6ZnVuY3Rpb24o
dCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHM9dCx1LmZvckVhY2gocCksbik6c30sZm9yY2U6
ZnVuY3Rpb24odCxlKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD4xPyhudWxsPT1lP3UuZGVsZXRl
KHQpOnUuc2V0KHQscChlKSksbik6dS5nZXQodCl9LGZpbmQ6ZnVuY3Rpb24obixlLHIpe3ZhciBp
LG8sYSx1LGMsZj0wLHM9dC5sZW5ndGg7Zm9yKG51bGw9PXI/cj0xLzA6cio9cixmPTA7ZjxzOysr
ZikoYT0oaT1uLSh1PXRbZl0pLngpKmkrKG89ZS11LnkpKm8pPHImJihjPXUscj1hKTtyZXR1cm4g
Y30sb246ZnVuY3Rpb24odCxlKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD4xPyhmLm9uKHQsZSks
bik6Zi5vbih0KX19fSx0LmZvcmNlWD1mdW5jdGlvbih0KXt2YXIgbixlLHIsaT1VYyguMSk7ZnVu
Y3Rpb24gbyh0KXtmb3IodmFyIGksbz0wLGE9bi5sZW5ndGg7bzxhOysrbykoaT1uW29dKS52eCs9
KHJbb10taS54KSplW29dKnR9ZnVuY3Rpb24gYSgpe2lmKG4pe3ZhciBvLGE9bi5sZW5ndGg7Zm9y
KGU9bmV3IEFycmF5KGEpLHI9bmV3IEFycmF5KGEpLG89MDtvPGE7KytvKWVbb109aXNOYU4ocltv
XT0rdChuW29dLG8sbikpPzA6K2kobltvXSxvLG4pfX1yZXR1cm4iZnVuY3Rpb24iIT10eXBlb2Yg
dCYmKHQ9VWMobnVsbD09dD8wOit0KSksby5pbml0aWFsaXplPWZ1bmN0aW9uKHQpe249dCxhKCl9
LG8uc3RyZW5ndGg9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGk9ImZ1bmN0
aW9uIj09dHlwZW9mIHQ/dDpVYygrdCksYSgpLG8pOml9LG8ueD1mdW5jdGlvbihuKXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8odD0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOlVjKCtuKSxhKCksbyk6
dH0sb30sdC5mb3JjZVk9ZnVuY3Rpb24odCl7dmFyIG4sZSxyLGk9VWMoLjEpO2Z1bmN0aW9uIG8o
dCl7Zm9yKHZhciBpLG89MCxhPW4ubGVuZ3RoO288YTsrK28pKGk9bltvXSkudnkrPShyW29dLWku
eSkqZVtvXSp0fWZ1bmN0aW9uIGEoKXtpZihuKXt2YXIgbyxhPW4ubGVuZ3RoO2ZvcihlPW5ldyBB
cnJheShhKSxyPW5ldyBBcnJheShhKSxvPTA7bzxhOysrbyllW29dPWlzTmFOKHJbb109K3Qobltv
XSxvLG4pKT8wOitpKG5bb10sbyxuKX19cmV0dXJuImZ1bmN0aW9uIiE9dHlwZW9mIHQmJih0PVVj
KG51bGw9PXQ/MDordCkpLG8uaW5pdGlhbGl6ZT1mdW5jdGlvbih0KXtuPXQsYSgpfSxvLnN0cmVu
Z3RoPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhpPSJmdW5jdGlvbiI9PXR5
cGVvZiB0P3Q6VWMoK3QpLGEoKSxvKTppfSxvLnk9ZnVuY3Rpb24obil7cmV0dXJuIGFyZ3VtZW50
cy5sZW5ndGg/KHQ9ImZ1bmN0aW9uIj09dHlwZW9mIG4/bjpVYygrbiksYSgpLG8pOnR9LG99LHQu
Zm9ybWF0RGVmYXVsdExvY2FsZT1zZix0LmZvcm1hdExvY2FsZT1mZix0LmZvcm1hdFNwZWNpZmll
cj10Zix0LmZzdW09ZnVuY3Rpb24odCxuKXtjb25zdCBlPW5ldyBBO2lmKHZvaWQgMD09PW4pZm9y
KGxldCBuIG9mIHQpKG49K24pJiZlLmFkZChuKTtlbHNle2xldCByPS0xO2ZvcihsZXQgaSBvZiB0
KShpPStuKGksKytyLHQpKSYmZS5hZGQoaSl9cmV0dXJuK2V9LHQuZ2VvQWxiZXJzPXdkLHQuZ2Vv
QWxiZXJzVXNhPWZ1bmN0aW9uKCl7dmFyIHQsbixlLHIsaSxvLGE9d2QoKSx1PXhkKCkucm90YXRl
KFsxNTQsMF0pLmNlbnRlcihbLTIsNTguNV0pLnBhcmFsbGVscyhbNTUsNjVdKSxjPXhkKCkucm90
YXRlKFsxNTcsMF0pLmNlbnRlcihbLTMsMTkuOV0pLnBhcmFsbGVscyhbOCwxOF0pLGY9e3BvaW50
OmZ1bmN0aW9uKHQsbil7bz1bdCxuXX19O2Z1bmN0aW9uIHModCl7dmFyIG49dFswXSxhPXRbMV07
cmV0dXJuIG89bnVsbCxlLnBvaW50KG4sYSksb3x8KHIucG9pbnQobixhKSxvKXx8KGkucG9pbnQo
bixhKSxvKX1mdW5jdGlvbiBsKCl7cmV0dXJuIHQ9bj1udWxsLHN9cmV0dXJuIHMuaW52ZXJ0PWZ1
bmN0aW9uKHQpe3ZhciBuPWEuc2NhbGUoKSxlPWEudHJhbnNsYXRlKCkscj0odFswXS1lWzBdKS9u
LGk9KHRbMV0tZVsxXSkvbjtyZXR1cm4oaT49LjEyJiZpPC4yMzQmJnI+PS0uNDI1JiZyPC0uMjE0
P3U6aT49LjE2NiYmaTwuMjM0JiZyPj0tLjIxNCYmcjwtLjExNT9jOmEpLmludmVydCh0KX0scy5z
dHJlYW09ZnVuY3Rpb24oZSl7cmV0dXJuIHQmJm49PT1lP3Q6KHI9W2Euc3RyZWFtKG49ZSksdS5z
dHJlYW0oZSksYy5zdHJlYW0oZSldLGk9ci5sZW5ndGgsdD17cG9pbnQ6ZnVuY3Rpb24odCxuKXtm
b3IodmFyIGU9LTE7KytlPGk7KXJbZV0ucG9pbnQodCxuKX0sc3BoZXJlOmZ1bmN0aW9uKCl7Zm9y
KHZhciB0PS0xOysrdDxpOylyW3RdLnNwaGVyZSgpfSxsaW5lU3RhcnQ6ZnVuY3Rpb24oKXtmb3Io
dmFyIHQ9LTE7Kyt0PGk7KXJbdF0ubGluZVN0YXJ0KCl9LGxpbmVFbmQ6ZnVuY3Rpb24oKXtmb3Io
dmFyIHQ9LTE7Kyt0PGk7KXJbdF0ubGluZUVuZCgpfSxwb2x5Z29uU3RhcnQ6ZnVuY3Rpb24oKXtm
b3IodmFyIHQ9LTE7Kyt0PGk7KXJbdF0ucG9seWdvblN0YXJ0KCl9LHBvbHlnb25FbmQ6ZnVuY3Rp
b24oKXtmb3IodmFyIHQ9LTE7Kyt0PGk7KXJbdF0ucG9seWdvbkVuZCgpfX0pO3ZhciByLGl9LHMu
cHJlY2lzaW9uPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhhLnByZWNpc2lv
bih0KSx1LnByZWNpc2lvbih0KSxjLnByZWNpc2lvbih0KSxsKCkpOmEucHJlY2lzaW9uKCl9LHMu
c2NhbGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGEuc2NhbGUodCksdS5z
Y2FsZSguMzUqdCksYy5zY2FsZSh0KSxzLnRyYW5zbGF0ZShhLnRyYW5zbGF0ZSgpKSk6YS5zY2Fs
ZSgpfSxzLnRyYW5zbGF0ZT1mdW5jdGlvbih0KXtpZighYXJndW1lbnRzLmxlbmd0aClyZXR1cm4g
YS50cmFuc2xhdGUoKTt2YXIgbj1hLnNjYWxlKCksbz0rdFswXSxzPSt0WzFdO3JldHVybiBlPWEu
dHJhbnNsYXRlKHQpLmNsaXBFeHRlbnQoW1tvLS40NTUqbixzLS4yMzgqbl0sW28rLjQ1NSpuLHMr
LjIzOCpuXV0pLnN0cmVhbShmKSxyPXUudHJhbnNsYXRlKFtvLS4zMDcqbixzKy4yMDEqbl0pLmNs
aXBFeHRlbnQoW1tvLS40MjUqbitwZixzKy4xMipuK3BmXSxbby0uMjE0Km4tcGYscysuMjM0Km4t
cGZdXSkuc3RyZWFtKGYpLGk9Yy50cmFuc2xhdGUoW28tLjIwNSpuLHMrLjIxMipuXSkuY2xpcEV4
dGVudChbW28tLjIxNCpuK3BmLHMrLjE2NipuK3BmXSxbby0uMTE1Km4tcGYscysuMjM0Km4tcGZd
XSkuc3RyZWFtKGYpLGwoKX0scy5maXRFeHRlbnQ9ZnVuY3Rpb24odCxuKXtyZXR1cm4gY2Qocyx0
LG4pfSxzLmZpdFNpemU9ZnVuY3Rpb24odCxuKXtyZXR1cm4gZmQocyx0LG4pfSxzLmZpdFdpZHRo
PWZ1bmN0aW9uKHQsbil7cmV0dXJuIHNkKHMsdCxuKX0scy5maXRIZWlnaHQ9ZnVuY3Rpb24odCxu
KXtyZXR1cm4gbGQocyx0LG4pfSxzLnNjYWxlKDEwNzApfSx0Lmdlb0FyZWE9ZnVuY3Rpb24odCl7
cmV0dXJuIGNzPW5ldyBBLGpmKHQsZnMpLDIqY3N9LHQuZ2VvQXppbXV0aGFsRXF1YWxBcmVhPWZ1
bmN0aW9uKCl7cmV0dXJuIHZkKEFkKS5zY2FsZSgxMjQuNzUpLmNsaXBBbmdsZSgxNzkuOTk5KX0s
dC5nZW9BemltdXRoYWxFcXVhbEFyZWFSYXc9QWQsdC5nZW9BemltdXRoYWxFcXVpZGlzdGFudD1m
dW5jdGlvbigpe3JldHVybiB2ZChTZCkuc2NhbGUoNzkuNDE4OCkuY2xpcEFuZ2xlKDE3OS45OTkp
fSx0Lmdlb0F6aW11dGhhbEVxdWlkaXN0YW50UmF3PVNkLHQuZ2VvQm91bmRzPWZ1bmN0aW9uKHQp
e3ZhciBuLGUscixpLG8sYSx1O2lmKEpmPVFmPS0oWmY9S2Y9MS8wKSxvcz1bXSxqZih0LHFzKSxl
PW9zLmxlbmd0aCl7Zm9yKG9zLnNvcnQoWHMpLG49MSxvPVtyPW9zWzBdXTtuPGU7KytuKUdzKHIs
KGk9b3Nbbl0pWzBdKXx8R3MocixpWzFdKT8oSHMoclswXSxpWzFdKT5IcyhyWzBdLHJbMV0pJiYo
clsxXT1pWzFdKSxIcyhpWzBdLHJbMV0pPkhzKHJbMF0sclsxXSkmJihyWzBdPWlbMF0pKTpvLnB1
c2gocj1pKTtmb3IoYT0tMS8wLG49MCxyPW9bZT1vLmxlbmd0aC0xXTtuPD1lO3I9aSwrK24paT1v
W25dLCh1PUhzKHJbMV0saVswXSkpPmEmJihhPXUsWmY9aVswXSxRZj1yWzFdKX1yZXR1cm4gb3M9
YXM9bnVsbCxaZj09PTEvMHx8S2Y9PT0xLzA/W1tOYU4sTmFOXSxbTmFOLE5hTl1dOltbWmYsS2Zd
LFtRZixKZl1dfSx0Lmdlb0NlbnRyb2lkPWZ1bmN0aW9uKHQpe3hzPXdzPU1zPVRzPUFzPVNzPUVz
PU5zPTAsa3M9bmV3IEEsQ3M9bmV3IEEsUHM9bmV3IEEsamYodCxWcyk7dmFyIG49K2tzLGU9K0Nz
LHI9K1BzLGk9TmYobixlLHIpO3JldHVybiBpPGdmJiYobj1TcyxlPUVzLHI9TnMsd3M8cGYmJihu
PU1zLGU9VHMscj1BcyksKGk9TmYobixlLHIpKTxnZik/W05hTixOYU5dOltUZihlLG4pKm1mLEZm
KHIvaSkqbWZdfSx0Lmdlb0NpcmNsZT1mdW5jdGlvbigpe3ZhciB0LG4sZT1vbChbMCwwXSkscj1v
bCg5MCksaT1vbCg2KSxvPXtwb2ludDpmdW5jdGlvbihlLHIpe3QucHVzaChlPW4oZSxyKSksZVsw
XSo9bWYsZVsxXSo9bWZ9fTtmdW5jdGlvbiBhKCl7dmFyIGE9ZS5hcHBseSh0aGlzLGFyZ3VtZW50
cyksdT1yLmFwcGx5KHRoaXMsYXJndW1lbnRzKSp4ZixjPWkuYXBwbHkodGhpcyxhcmd1bWVudHMp
KnhmO3JldHVybiB0PVtdLG49Y2woLWFbMF0qeGYsLWFbMV0qeGYsMCkuaW52ZXJ0LGRsKG8sdSxj
LDEpLGE9e3R5cGU6IlBvbHlnb24iLGNvb3JkaW5hdGVzOlt0XX0sdD1uPW51bGwsYX1yZXR1cm4g
YS5jZW50ZXI9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGU9ImZ1bmN0aW9u
Ij09dHlwZW9mIHQ/dDpvbChbK3RbMF0sK3RbMV1dKSxhKTplfSxhLnJhZGl1cz1mdW5jdGlvbih0
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj0iZnVuY3Rpb24iPT10eXBlb2YgdD90Om9sKCt0
KSxhKTpyfSxhLnByZWNpc2lvbj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8o
aT0iZnVuY3Rpb24iPT10eXBlb2YgdD90Om9sKCt0KSxhKTppfSxhfSx0Lmdlb0NsaXBBbnRpbWVy
aWRpYW49QWwsdC5nZW9DbGlwQ2lyY2xlPVNsLHQuZ2VvQ2xpcEV4dGVudD1mdW5jdGlvbigpe3Zh
ciB0LG4sZSxyPTAsaT0wLG89OTYwLGE9NTAwO3JldHVybiBlPXtzdHJlYW06ZnVuY3Rpb24oZSl7
cmV0dXJuIHQmJm49PT1lP3Q6dD0kbChyLGksbyxhKShuPWUpfSxleHRlbnQ6ZnVuY3Rpb24odSl7
cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9K3VbMF1bMF0saT0rdVswXVsxXSxvPSt1WzFdWzBd
LGE9K3VbMV1bMV0sdD1uPW51bGwsZSk6W1tyLGldLFtvLGFdXX19fSx0Lmdlb0NsaXBSZWN0YW5n
bGU9JGwsdC5nZW9Db25pY0NvbmZvcm1hbD1mdW5jdGlvbigpe3JldHVybiBiZChDZCkuc2NhbGUo
MTA5LjUpLnBhcmFsbGVscyhbMzAsMzBdKX0sdC5nZW9Db25pY0NvbmZvcm1hbFJhdz1DZCx0Lmdl
b0NvbmljRXF1YWxBcmVhPXhkLHQuZ2VvQ29uaWNFcXVhbEFyZWFSYXc9bWQsdC5nZW9Db25pY0Vx
dWlkaXN0YW50PWZ1bmN0aW9uKCl7cmV0dXJuIGJkKHpkKS5zY2FsZSgxMzEuMTU0KS5jZW50ZXIo
WzAsMTMuOTM4OV0pfSx0Lmdlb0NvbmljRXF1aWRpc3RhbnRSYXc9emQsdC5nZW9Db250YWlucz1m
dW5jdGlvbih0LG4pe3JldHVybih0JiZZbC5oYXNPd25Qcm9wZXJ0eSh0LnR5cGUpP1lsW3QudHlw
ZV06amwpKHQsbil9LHQuZ2VvRGlzdGFuY2U9QmwsdC5nZW9FcXVhbEVhcnRoPWZ1bmN0aW9uKCl7
cmV0dXJuIHZkKFVkKS5zY2FsZSgxNzcuMTU4KX0sdC5nZW9FcXVhbEVhcnRoUmF3PVVkLHQuZ2Vv
RXF1aXJlY3Rhbmd1bGFyPWZ1bmN0aW9uKCl7cmV0dXJuIHZkKFBkKS5zY2FsZSgxNTIuNjMpfSx0
Lmdlb0VxdWlyZWN0YW5ndWxhclJhdz1QZCx0Lmdlb0dub21vbmljPWZ1bmN0aW9uKCl7cmV0dXJu
IHZkKElkKS5zY2FsZSgxNDQuMDQ5KS5jbGlwQW5nbGUoNjApfSx0Lmdlb0dub21vbmljUmF3PUlk
LHQuZ2VvR3JhdGljdWxlPVFsLHQuZ2VvR3JhdGljdWxlMTA9ZnVuY3Rpb24oKXtyZXR1cm4gUWwo
KSgpfSx0Lmdlb0lkZW50aXR5PWZ1bmN0aW9uKCl7dmFyIHQsbixlLHIsaSxvLGEsdT0xLGM9MCxm
PTAscz0xLGw9MSxoPTAsZD1udWxsLHA9MSxnPTEseT1vZCh7cG9pbnQ6ZnVuY3Rpb24odCxuKXt2
YXIgZT1iKFt0LG5dKTt0aGlzLnN0cmVhbS5wb2ludChlWzBdLGVbMV0pfX0pLHY9cmg7ZnVuY3Rp
b24gXygpe3JldHVybiBwPXUqcyxnPXUqbCxvPWE9bnVsbCxifWZ1bmN0aW9uIGIoZSl7dmFyIHI9
ZVswXSpwLGk9ZVsxXSpnO2lmKGgpe3ZhciBvPWkqdC1yKm47cj1yKnQraSpuLGk9b31yZXR1cm5b
citjLGkrZl19cmV0dXJuIGIuaW52ZXJ0PWZ1bmN0aW9uKGUpe3ZhciByPWVbMF0tYyxpPWVbMV0t
ZjtpZihoKXt2YXIgbz1pKnQrcipuO3I9cip0LWkqbixpPW99cmV0dXJuW3IvcCxpL2ddfSxiLnN0
cmVhbT1mdW5jdGlvbih0KXtyZXR1cm4gbyYmYT09PXQ/bzpvPXkodihhPXQpKX0sYi5wb3N0Y2xp
cD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odj10LGQ9ZT1yPWk9bnVsbCxf
KCkpOnZ9LGIuY2xpcEV4dGVudD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8o
dj1udWxsPT10PyhkPWU9cj1pPW51bGwscmgpOiRsKGQ9K3RbMF1bMF0sZT0rdFswXVsxXSxyPSt0
WzFdWzBdLGk9K3RbMV1bMV0pLF8oKSk6bnVsbD09ZD9udWxsOltbZCxlXSxbcixpXV19LGIuc2Nh
bGU9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHU9K3QsXygpKTp1fSxiLnRy
YW5zbGF0ZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYz0rdFswXSxmPSt0
WzFdLF8oKSk6W2MsZl19LGIuYW5nbGU9ZnVuY3Rpb24oZSl7cmV0dXJuIGFyZ3VtZW50cy5sZW5n
dGg/KG49UGYoaD1lJTM2MCp4ZiksdD1BZihoKSxfKCkpOmgqbWZ9LGIucmVmbGVjdFg9ZnVuY3Rp
b24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHM9dD8tMToxLF8oKSk6czwwfSxiLnJlZmxl
Y3RZPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhsPXQ/LTE6MSxfKCkpOmw8
MH0sYi5maXRFeHRlbnQ9ZnVuY3Rpb24odCxuKXtyZXR1cm4gY2QoYix0LG4pfSxiLmZpdFNpemU9
ZnVuY3Rpb24odCxuKXtyZXR1cm4gZmQoYix0LG4pfSxiLmZpdFdpZHRoPWZ1bmN0aW9uKHQsbil7
cmV0dXJuIHNkKGIsdCxuKX0sYi5maXRIZWlnaHQ9ZnVuY3Rpb24odCxuKXtyZXR1cm4gbGQoYix0
LG4pfSxifSx0Lmdlb0ludGVycG9sYXRlPWZ1bmN0aW9uKHQsbil7dmFyIGU9dFswXSp4ZixyPXRb
MV0qeGYsaT1uWzBdKnhmLG89blsxXSp4ZixhPUFmKHIpLHU9UGYociksYz1BZihvKSxmPVBmKG8p
LHM9YSpBZihlKSxsPWEqUGYoZSksaD1jKkFmKGkpLGQ9YypQZihpKSxwPTIqRmYoJGYocWYoby1y
KSthKmMqcWYoaS1lKSkpLGc9UGYocCkseT1wP2Z1bmN0aW9uKHQpe3ZhciBuPVBmKHQqPXApL2cs
ZT1QZihwLXQpL2cscj1lKnMrbipoLGk9ZSpsK24qZCxvPWUqdStuKmY7cmV0dXJuW1RmKGkscikq
bWYsVGYobywkZihyKnIraSppKSkqbWZdfTpmdW5jdGlvbigpe3JldHVybltlKm1mLHIqbWZdfTty
ZXR1cm4geS5kaXN0YW5jZT1wLHl9LHQuZ2VvTGVuZ3RoPVVsLHQuZ2VvTWVyY2F0b3I9ZnVuY3Rp
b24oKXtyZXR1cm4gTmQoRWQpLnNjYWxlKDk2MS9iZil9LHQuZ2VvTWVyY2F0b3JSYXc9RWQsdC5n
ZW9OYXR1cmFsRWFydGgxPWZ1bmN0aW9uKCl7cmV0dXJuIHZkKE9kKS5zY2FsZSgxNzUuMjk1KX0s
dC5nZW9OYXR1cmFsRWFydGgxUmF3PU9kLHQuZ2VvT3J0aG9ncmFwaGljPWZ1bmN0aW9uKCl7cmV0
dXJuIHZkKEJkKS5zY2FsZSgyNDkuNSkuY2xpcEFuZ2xlKDkwK3BmKX0sdC5nZW9PcnRob2dyYXBo
aWNSYXc9QmQsdC5nZW9QYXRoPWZ1bmN0aW9uKHQsbil7bGV0IGUscixpPTMsbz00LjU7ZnVuY3Rp
b24gYSh0KXtyZXR1cm4gdCYmKCJmdW5jdGlvbiI9PXR5cGVvZiBvJiZyLnBvaW50UmFkaXVzKCtv
LmFwcGx5KHRoaXMsYXJndW1lbnRzKSksamYodCxlKHIpKSksci5yZXN1bHQoKX1yZXR1cm4gYS5h
cmVhPWZ1bmN0aW9uKHQpe3JldHVybiBqZih0LGUobGgpKSxsaC5yZXN1bHQoKX0sYS5tZWFzdXJl
PWZ1bmN0aW9uKHQpe3JldHVybiBqZih0LGUoUWgpKSxRaC5yZXN1bHQoKX0sYS5ib3VuZHM9ZnVu
Y3Rpb24odCl7cmV0dXJuIGpmKHQsZSh4aCkpLHhoLnJlc3VsdCgpfSxhLmNlbnRyb2lkPWZ1bmN0
aW9uKHQpe3JldHVybiBqZih0LGUoQmgpKSxCaC5yZXN1bHQoKX0sYS5wcm9qZWN0aW9uPWZ1bmN0
aW9uKG4pe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlPW51bGw9PW4/KHQ9bnVsbCxyaCk6KHQ9
bikuc3RyZWFtLGEpOnR9LGEuY29udGV4dD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxl
bmd0aD8ocj1udWxsPT10PyhuPW51bGwsbmV3IHJkKGkpKTpuZXcgWWgobj10KSwiZnVuY3Rpb24i
IT10eXBlb2YgbyYmci5wb2ludFJhZGl1cyhvKSxhKTpufSxhLnBvaW50UmFkaXVzPWZ1bmN0aW9u
KHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhvPSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6KHIu
cG9pbnRSYWRpdXMoK3QpLCt0KSxhKTpvfSxhLmRpZ2l0cz1mdW5jdGlvbih0KXtpZighYXJndW1l
bnRzLmxlbmd0aClyZXR1cm4gaTtpZihudWxsPT10KWk9bnVsbDtlbHNle2NvbnN0IG49TWF0aC5m
bG9vcih0KTtpZighKG4+PTApKXRocm93IG5ldyBSYW5nZUVycm9yKGBpbnZhbGlkIGRpZ2l0czog
JHt0fWApO2k9bn1yZXR1cm4gbnVsbD09PW4mJihyPW5ldyByZChpKSksYX0sYS5wcm9qZWN0aW9u
KHQpLmRpZ2l0cyhpKS5jb250ZXh0KG4pfSx0Lmdlb1Byb2plY3Rpb249dmQsdC5nZW9Qcm9qZWN0
aW9uTXV0YXRvcj1fZCx0Lmdlb1JvdGF0aW9uPWhsLHQuZ2VvU3RlcmVvZ3JhcGhpYz1mdW5jdGlv
bigpe3JldHVybiB2ZChZZCkuc2NhbGUoMjUwKS5jbGlwQW5nbGUoMTQyKX0sdC5nZW9TdGVyZW9n
cmFwaGljUmF3PVlkLHQuZ2VvU3RyZWFtPWpmLHQuZ2VvVHJhbnNmb3JtPWZ1bmN0aW9uKHQpe3Jl
dHVybntzdHJlYW06b2QodCl9fSx0Lmdlb1RyYW5zdmVyc2VNZXJjYXRvcj1mdW5jdGlvbigpe3Zh
ciB0PU5kKExkKSxuPXQuY2VudGVyLGU9dC5yb3RhdGU7cmV0dXJuIHQuY2VudGVyPWZ1bmN0aW9u
KHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoP24oWy10WzFdLHRbMF1dKTpbKHQ9bigpKVsxXSwt
dFswXV19LHQucm90YXRlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoP2UoW3Rb
MF0sdFsxXSx0Lmxlbmd0aD4yP3RbMl0rOTA6OTBdKTpbKHQ9ZSgpKVswXSx0WzFdLHRbMl0tOTBd
fSxlKFswLDAsOTBdKS5zY2FsZSgxNTkuMTU1KX0sdC5nZW9UcmFuc3ZlcnNlTWVyY2F0b3JSYXc9
TGQsdC5ncmF5PWZ1bmN0aW9uKHQsbil7cmV0dXJuIG5ldyBjcih0LDAsMCxudWxsPT1uPzE6bil9
LHQuZ3JlYXRlc3Q9YXQsdC5ncmVhdGVzdEluZGV4PWZ1bmN0aW9uKHQsZT1uKXtpZigxPT09ZS5s
ZW5ndGgpcmV0dXJuIG50KHQsZSk7bGV0IHIsaT0tMSxvPS0xO2Zvcihjb25zdCBuIG9mIHQpKytv
LChpPDA/MD09PWUobixuKTplKG4scik+MCkmJihyPW4saT1vKTtyZXR1cm4gaX0sdC5ncm91cD1Q
LHQuZ3JvdXBTb3J0PWZ1bmN0aW9uKHQsZSxyKXtyZXR1cm4oMiE9PWUubGVuZ3RoP0koRCh0LGUs
ciksKChbdCxlXSxbcixpXSk9Pm4oZSxpKXx8bih0LHIpKSk6SShQKHQsciksKChbdCxyXSxbaSxv
XSk9PmUocixvKXx8bih0LGkpKSkpLm1hcCgoKFt0XSk9PnQpKX0sdC5ncm91cHM9eix0LmhjbD1w
cix0LmhpZXJhcmNoeT1WZCx0Lmhpc3RvZ3JhbT1KLHQuaHNsPVhlLHQuaHRtbD1OYyx0LmltYWdl
PWZ1bmN0aW9uKHQsbil7cmV0dXJuIG5ldyBQcm9taXNlKChmdW5jdGlvbihlLHIpe3ZhciBpPW5l
dyBJbWFnZTtmb3IodmFyIG8gaW4gbilpW29dPW5bb107aS5vbmVycm9yPXIsaS5vbmxvYWQ9ZnVu
Y3Rpb24oKXtlKGkpfSxpLnNyYz10fSkpfSx0LmluZGV4PWZ1bmN0aW9uKHQsLi4ubil7cmV0dXJu
IHEodCxDLEYsbil9LHQuaW5kZXhlcz1mdW5jdGlvbih0LC4uLm4pe3JldHVybiBxKHQsQXJyYXku
ZnJvbSxGLG4pfSx0LmludGVycG9sYXRlPVZyLHQuaW50ZXJwb2xhdGVBcnJheT1mdW5jdGlvbih0
LG4pe3JldHVybihPcihuKT9JcjpCcikodCxuKX0sdC5pbnRlcnBvbGF0ZUJhc2lzPU5yLHQuaW50
ZXJwb2xhdGVCYXNpc0Nsb3NlZD1rcix0LmludGVycG9sYXRlQmx1ZXM9R2IsdC5pbnRlcnBvbGF0
ZUJyQkc9b2IsdC5pbnRlcnBvbGF0ZUJ1R249TWIsdC5pbnRlcnBvbGF0ZUJ1UHU9QWIsdC5pbnRl
cnBvbGF0ZUNpdmlkaXM9ZnVuY3Rpb24odCl7cmV0dXJuIHQ9TWF0aC5tYXgoMCxNYXRoLm1pbigx
LHQpKSwicmdiKCIrTWF0aC5tYXgoMCxNYXRoLm1pbigyNTUsTWF0aC5yb3VuZCgtNC41NC10Kigz
NS4zNC10KigyMzgxLjczLXQqKDY0MDIuNy10Kig3MDI0LjcyLTI3MTAuNTcqdCkpKSkpKSkrIiwg
IitNYXRoLm1heCgwLE1hdGgubWluKDI1NSxNYXRoLnJvdW5kKDMyLjQ5K3QqKDE3MC43Myt0Kig1
Mi44Mi10KigxMzEuNDYtdCooMTc2LjU4LTY3LjM3KnQpKSkpKSkpKyIsICIrTWF0aC5tYXgoMCxN
YXRoLm1pbigyNTUsTWF0aC5yb3VuZCg4MS4yNCt0Kig0NDIuMzYtdCooMjQ4Mi40My10Kig2MTY3
LjI0LXQqKDY2MTQuOTQtMjQ3NS42Nyp0KSkpKSkpKSsiKSJ9LHQuaW50ZXJwb2xhdGVDb29sPWFt
LHQuaW50ZXJwb2xhdGVDdWJlaGVsaXg9aGksdC5pbnRlcnBvbGF0ZUN1YmVoZWxpeERlZmF1bHQ9
aW0sdC5pbnRlcnBvbGF0ZUN1YmVoZWxpeExvbmc9ZGksdC5pbnRlcnBvbGF0ZURhdGU9WXIsdC5p
bnRlcnBvbGF0ZURpc2NyZXRlPWZ1bmN0aW9uKHQpe3ZhciBuPXQubGVuZ3RoO3JldHVybiBmdW5j
dGlvbihlKXtyZXR1cm4gdFtNYXRoLm1heCgwLE1hdGgubWluKG4tMSxNYXRoLmZsb29yKGUqbikp
KV19fSx0LmludGVycG9sYXRlR25CdT1FYix0LmludGVycG9sYXRlR3JlZW5zPVdiLHQuaW50ZXJw
b2xhdGVHcmV5cz1LYix0LmludGVycG9sYXRlSGNsPWZpLHQuaW50ZXJwb2xhdGVIY2xMb25nPXNp
LHQuaW50ZXJwb2xhdGVIc2w9YWksdC5pbnRlcnBvbGF0ZUhzbExvbmc9dWksdC5pbnRlcnBvbGF0
ZUh1ZT1mdW5jdGlvbih0LG4pe3ZhciBlPXpyKCt0LCtuKTtyZXR1cm4gZnVuY3Rpb24odCl7dmFy
IG49ZSh0KTtyZXR1cm4gbi0zNjAqTWF0aC5mbG9vcihuLzM2MCl9fSx0LmludGVycG9sYXRlSW5m
ZXJubz1wbSx0LmludGVycG9sYXRlTGFiPWZ1bmN0aW9uKHQsbil7dmFyIGU9RHIoKHQ9dXIodCkp
LmwsKG49dXIobikpLmwpLHI9RHIodC5hLG4uYSksaT1Ecih0LmIsbi5iKSxvPURyKHQub3BhY2l0
eSxuLm9wYWNpdHkpO3JldHVybiBmdW5jdGlvbihuKXtyZXR1cm4gdC5sPWUobiksdC5hPXIobiks
dC5iPWkobiksdC5vcGFjaXR5PW8obiksdCsiIn19LHQuaW50ZXJwb2xhdGVNYWdtYT1kbSx0Lmlu
dGVycG9sYXRlTnVtYmVyPUxyLHQuaW50ZXJwb2xhdGVOdW1iZXJBcnJheT1Jcix0LmludGVycG9s
YXRlT2JqZWN0PWpyLHQuaW50ZXJwb2xhdGVPclJkPWtiLHQuaW50ZXJwb2xhdGVPcmFuZ2VzPXJt
LHQuaW50ZXJwb2xhdGVQUkduPXViLHQuaW50ZXJwb2xhdGVQaVlHPWZiLHQuaW50ZXJwb2xhdGVQ
bGFzbWE9Z20sdC5pbnRlcnBvbGF0ZVB1QnU9JGIsdC5pbnRlcnBvbGF0ZVB1QnVHbj1QYix0Lmlu
dGVycG9sYXRlUHVPcj1sYix0LmludGVycG9sYXRlUHVSZD1SYix0LmludGVycG9sYXRlUHVycGxl
cz1KYix0LmludGVycG9sYXRlUmFpbmJvdz1mdW5jdGlvbih0KXsodDwwfHx0PjEpJiYodC09TWF0
aC5mbG9vcih0KSk7dmFyIG49TWF0aC5hYnModC0uNSk7cmV0dXJuIHVtLmg9MzYwKnQtMTAwLHVt
LnM9MS41LTEuNSpuLHVtLmw9LjgtLjkqbix1bSsiIn0sdC5pbnRlcnBvbGF0ZVJkQnU9ZGIsdC5p
bnRlcnBvbGF0ZVJkR3k9Z2IsdC5pbnRlcnBvbGF0ZVJkUHU9cWIsdC5pbnRlcnBvbGF0ZVJkWWxC
dT12Yix0LmludGVycG9sYXRlUmRZbEduPWJiLHQuaW50ZXJwb2xhdGVSZWRzPW5tLHQuaW50ZXJw
b2xhdGVSZ2I9UnIsdC5pbnRlcnBvbGF0ZVJnYkJhc2lzPXFyLHQuaW50ZXJwb2xhdGVSZ2JCYXNp
c0Nsb3NlZD1Vcix0LmludGVycG9sYXRlUm91bmQ9V3IsdC5pbnRlcnBvbGF0ZVNpbmVib3c9ZnVu
Y3Rpb24odCl7dmFyIG47cmV0dXJuIHQ9KC41LXQpKk1hdGguUEksY20ucj0yNTUqKG49TWF0aC5z
aW4odCkpKm4sY20uZz0yNTUqKG49TWF0aC5zaW4odCtmbSkpKm4sY20uYj0yNTUqKG49TWF0aC5z
aW4odCtzbSkpKm4sY20rIiJ9LHQuaW50ZXJwb2xhdGVTcGVjdHJhbD14Yix0LmludGVycG9sYXRl
U3RyaW5nPUdyLHQuaW50ZXJwb2xhdGVUcmFuc2Zvcm1Dc3M9bmksdC5pbnRlcnBvbGF0ZVRyYW5z
Zm9ybVN2Zz1laSx0LmludGVycG9sYXRlVHVyYm89ZnVuY3Rpb24odCl7cmV0dXJuIHQ9TWF0aC5t
YXgoMCxNYXRoLm1pbigxLHQpKSwicmdiKCIrTWF0aC5tYXgoMCxNYXRoLm1pbigyNTUsTWF0aC5y
b3VuZCgzNC42MSt0KigxMTcyLjMzLXQqKDEwNzkzLjU2LXQqKDMzMzAwLjEyLXQqKDM4Mzk0LjQ5
LTE0ODI1LjA1KnQpKSkpKSkpKyIsICIrTWF0aC5tYXgoMCxNYXRoLm1pbigyNTUsTWF0aC5yb3Vu
ZCgyMy4zMSt0Kig1NTcuMzMrdCooMTIyNS4zMy10KigzNTc0Ljk2LXQqKDEwNzMuNzcrNzA3LjU2
KnQpKSkpKSkpKyIsICIrTWF0aC5tYXgoMCxNYXRoLm1pbigyNTUsTWF0aC5yb3VuZCgyNy4yK3Qq
KDMyMTEuMS10KigxNTMyNy45Ny10KigyNzgxNC10KigyMjU2OS4xOC02ODM4LjY2KnQpKSkpKSkp
KyIpIn0sdC5pbnRlcnBvbGF0ZVZpcmlkaXM9aG0sdC5pbnRlcnBvbGF0ZVdhcm09b20sdC5pbnRl
cnBvbGF0ZVlsR249QmIsdC5pbnRlcnBvbGF0ZVlsR25CdT1JYix0LmludGVycG9sYXRlWWxPckJy
PUxiLHQuaW50ZXJwb2xhdGVZbE9yUmQ9SGIsdC5pbnRlcnBvbGF0ZVpvb209aWksdC5pbnRlcnJ1
cHQ9VmksdC5pbnRlcnNlY3Rpb249ZnVuY3Rpb24odCwuLi5uKXt0PW5ldyBJbnRlcm5TZXQodCks
bj1uLm1hcChfdCk7dDpmb3IoY29uc3QgZSBvZiB0KWZvcihjb25zdCByIG9mIG4paWYoIXIuaGFz
KGUpKXt0LmRlbGV0ZShlKTtjb250aW51ZSB0fXJldHVybiB0fSx0LmludGVydmFsPWZ1bmN0aW9u
KHQsbixlKXt2YXIgcj1uZXcgTmksaT1uO3JldHVybiBudWxsPT1uPyhyLnJlc3RhcnQodCxuLGUp
LHIpOihyLl9yZXN0YXJ0PXIucmVzdGFydCxyLnJlc3RhcnQ9ZnVuY3Rpb24odCxuLGUpe249K24s
ZT1udWxsPT1lP1NpKCk6K2Usci5fcmVzdGFydCgoZnVuY3Rpb24gbyhhKXthKz1pLHIuX3Jlc3Rh
cnQobyxpKz1uLGUpLHQoYSl9KSxuLGUpfSxyLnJlc3RhcnQodCxuLGUpLHIpfSx0Lmlzb0Zvcm1h
dD1SXyx0Lmlzb1BhcnNlPXFfLHQuanNvbj1mdW5jdGlvbih0LG4pe3JldHVybiBmZXRjaCh0LG4p
LnRoZW4oQWMpfSx0LmxhYj11cix0LmxjaD1mdW5jdGlvbih0LG4sZSxyKXtyZXR1cm4gMT09PWFy
Z3VtZW50cy5sZW5ndGg/ZHIodCk6bmV3IGdyKGUsbix0LG51bGw9PXI/MTpyKX0sdC5sZWFzdD1m
dW5jdGlvbih0LGU9bil7bGV0IHIsaT0hMTtpZigxPT09ZS5sZW5ndGgpe2xldCBvO2Zvcihjb25z
dCBhIG9mIHQpe2NvbnN0IHQ9ZShhKTsoaT9uKHQsbyk8MDowPT09bih0LHQpKSYmKHI9YSxvPXQs
aT0hMCl9fWVsc2UgZm9yKGNvbnN0IG4gb2YgdCkoaT9lKG4scik8MDowPT09ZShuLG4pKSYmKHI9
bixpPSEwKTtyZXR1cm4gcn0sdC5sZWFzdEluZGV4PWR0LHQubGluZT1ZbSx0LmxpbmVSYWRpYWw9
Wm0sdC5saW5rPWF4LHQubGlua0hvcml6b250YWw9ZnVuY3Rpb24oKXtyZXR1cm4gYXgobngpfSx0
LmxpbmtSYWRpYWw9ZnVuY3Rpb24oKXtjb25zdCB0PWF4KHJ4KTtyZXR1cm4gdC5hbmdsZT10Lngs
ZGVsZXRlIHQueCx0LnJhZGl1cz10LnksZGVsZXRlIHQueSx0fSx0LmxpbmtWZXJ0aWNhbD1mdW5j
dGlvbigpe3JldHVybiBheChleCl9LHQubG9jYWw9Sm4sdC5tYXA9ZnVuY3Rpb24odCxuKXtpZigi
ZnVuY3Rpb24iIT10eXBlb2YgdFtTeW1ib2wuaXRlcmF0b3JdKXRocm93IG5ldyBUeXBlRXJyb3Io
InZhbHVlcyBpcyBub3QgaXRlcmFibGUiKTtpZigiZnVuY3Rpb24iIT10eXBlb2Ygbil0aHJvdyBu
ZXcgVHlwZUVycm9yKCJtYXBwZXIgaXMgbm90IGEgZnVuY3Rpb24iKTtyZXR1cm4gQXJyYXkuZnJv
bSh0LCgoZSxyKT0+bihlLHIsdCkpKX0sdC5tYXRjaGVyPVd0LHQubWF4PXR0LHQubWF4SW5kZXg9
bnQsdC5tZWFuPWZ1bmN0aW9uKHQsbil7bGV0IGU9MCxyPTA7aWYodm9pZCAwPT09bilmb3IobGV0
IG4gb2YgdCludWxsIT1uJiYobj0rbik+PW4mJigrK2Uscis9bik7ZWxzZXtsZXQgaT0tMTtmb3Io
bGV0IG8gb2YgdCludWxsIT0obz1uKG8sKytpLHQpKSYmKG89K28pPj1vJiYoKytlLHIrPW8pfWlm
KGUpcmV0dXJuIHIvZX0sdC5tZWRpYW49ZnVuY3Rpb24odCxuKXtyZXR1cm4gdXQodCwuNSxuKX0s
dC5tZWRpYW5JbmRleD1mdW5jdGlvbih0LG4pe3JldHVybiBmdCh0LC41LG4pfSx0Lm1lcmdlPXN0
LHQubWluPWV0LHQubWluSW5kZXg9cnQsdC5tb2RlPWZ1bmN0aW9uKHQsbil7Y29uc3QgZT1uZXcg
SW50ZXJuTWFwO2lmKHZvaWQgMD09PW4pZm9yKGxldCBuIG9mIHQpbnVsbCE9biYmbj49biYmZS5z
ZXQobiwoZS5nZXQobil8fDApKzEpO2Vsc2V7bGV0IHI9LTE7Zm9yKGxldCBpIG9mIHQpbnVsbCE9
KGk9bihpLCsrcix0KSkmJmk+PWkmJmUuc2V0KGksKGUuZ2V0KGkpfHwwKSsxKX1sZXQgcixpPTA7
Zm9yKGNvbnN0W3Qsbl1vZiBlKW4+aSYmKGk9bixyPXQpO3JldHVybiByfSx0Lm5hbWVzcGFjZT1P
dCx0Lm5hbWVzcGFjZXM9SXQsdC5uaWNlPUssdC5ub3c9U2ksdC5wYWNrPWZ1bmN0aW9uKCl7dmFy
IHQ9bnVsbCxuPTEsZT0xLHI9ZXA7ZnVuY3Rpb24gaShpKXtjb25zdCBvPXVwKCk7cmV0dXJuIGku
eD1uLzIsaS55PWUvMix0P2kuZWFjaEJlZm9yZSh3cCh0KSkuZWFjaEFmdGVyKE1wKHIsLjUsbykp
LmVhY2hCZWZvcmUoVHAoMSkpOmkuZWFjaEJlZm9yZSh3cCh4cCkpLmVhY2hBZnRlcihNcChlcCwx
LG8pKS5lYWNoQWZ0ZXIoTXAocixpLnIvTWF0aC5taW4obixlKSxvKSkuZWFjaEJlZm9yZShUcChN
YXRoLm1pbihuLGUpLygyKmkucikpKSxpfXJldHVybiBpLnJhZGl1cz1mdW5jdGlvbihuKXtyZXR1
cm4gYXJndW1lbnRzLmxlbmd0aD8odD10cChuKSxpKTp0fSxpLnNpemU9ZnVuY3Rpb24odCl7cmV0
dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49K3RbMF0sZT0rdFsxXSxpKTpbbixlXX0saS5wYWRkaW5n
PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhyPSJmdW5jdGlvbiI9PXR5cGVv
ZiB0P3Q6cnAoK3QpLGkpOnJ9LGl9LHQucGFja0VuY2xvc2U9ZnVuY3Rpb24odCl7cmV0dXJuIGNw
KHQsdXAoKSl9LHQucGFja1NpYmxpbmdzPWZ1bmN0aW9uKHQpe3JldHVybiBtcCh0LHVwKCkpLHR9
LHQucGFpcnM9ZnVuY3Rpb24odCxuPWx0KXtjb25zdCBlPVtdO2xldCByLGk9ITE7Zm9yKGNvbnN0
IG8gb2YgdClpJiZlLnB1c2gobihyLG8pKSxyPW8saT0hMDtyZXR1cm4gZX0sdC5wYXJ0aXRpb249
ZnVuY3Rpb24oKXt2YXIgdD0xLG49MSxlPTAscj0hMTtmdW5jdGlvbiBpKGkpe3ZhciBvPWkuaGVp
Z2h0KzE7cmV0dXJuIGkueDA9aS55MD1lLGkueDE9dCxpLnkxPW4vbyxpLmVhY2hCZWZvcmUoZnVu
Y3Rpb24odCxuKXtyZXR1cm4gZnVuY3Rpb24ocil7ci5jaGlsZHJlbiYmU3AocixyLngwLHQqKHIu
ZGVwdGgrMSkvbixyLngxLHQqKHIuZGVwdGgrMikvbik7dmFyIGk9ci54MCxvPXIueTAsYT1yLngx
LWUsdT1yLnkxLWU7YTxpJiYoaT1hPShpK2EpLzIpLHU8byYmKG89dT0obyt1KS8yKSxyLngwPWks
ci55MD1vLHIueDE9YSxyLnkxPXV9fShuLG8pKSxyJiZpLmVhY2hCZWZvcmUoQXApLGl9cmV0dXJu
IGkucm91bmQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9ISF0LGkpOnJ9
LGkuc2l6ZT1mdW5jdGlvbihlKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD0rZVswXSxuPStl
WzFdLGkpOlt0LG5dfSxpLnBhZGRpbmc9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5n
dGg/KGU9K3QsaSk6ZX0saX0sdC5wYXRoPU9hLHQucGF0aFJvdW5kPWZ1bmN0aW9uKHQ9Myl7cmV0
dXJuIG5ldyBJYSgrdCl9LHQucGVybXV0ZT1VLHQucGllPWZ1bmN0aW9uKCl7dmFyIHQ9SG0sbj1q
bSxlPW51bGwscj15bSgwKSxpPXltKEVtKSxvPXltKDApO2Z1bmN0aW9uIGEoYSl7dmFyIHUsYyxm
LHMsbCxoPShhPXFtKGEpKS5sZW5ndGgsZD0wLHA9bmV3IEFycmF5KGgpLGc9bmV3IEFycmF5KGgp
LHk9K3IuYXBwbHkodGhpcyxhcmd1bWVudHMpLHY9TWF0aC5taW4oRW0sTWF0aC5tYXgoLUVtLGku
YXBwbHkodGhpcyxhcmd1bWVudHMpLXkpKSxfPU1hdGgubWluKE1hdGguYWJzKHYpL2gsby5hcHBs
eSh0aGlzLGFyZ3VtZW50cykpLGI9XyoodjwwPy0xOjEpO2Zvcih1PTA7dTxoOysrdSkobD1nW3Bb
dV09dV09K3QoYVt1XSx1LGEpKT4wJiYoZCs9bCk7Zm9yKG51bGwhPW4/cC5zb3J0KChmdW5jdGlv
bih0LGUpe3JldHVybiBuKGdbdF0sZ1tlXSl9KSk6bnVsbCE9ZSYmcC5zb3J0KChmdW5jdGlvbih0
LG4pe3JldHVybiBlKGFbdF0sYVtuXSl9KSksdT0wLGY9ZD8odi1oKmIpL2Q6MDt1PGg7Kyt1LHk9
cyljPXBbdV0scz15KygobD1nW2NdKT4wP2wqZjowKStiLGdbY109e2RhdGE6YVtjXSxpbmRleDp1
LHZhbHVlOmwsc3RhcnRBbmdsZTp5LGVuZEFuZ2xlOnMscGFkQW5nbGU6X307cmV0dXJuIGd9cmV0
dXJuIGEudmFsdWU9ZnVuY3Rpb24obil7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHQ9ImZ1bmN0
aW9uIj09dHlwZW9mIG4/bjp5bSgrbiksYSk6dH0sYS5zb3J0VmFsdWVzPWZ1bmN0aW9uKHQpe3Jl
dHVybiBhcmd1bWVudHMubGVuZ3RoPyhuPXQsZT1udWxsLGEpOm59LGEuc29ydD1mdW5jdGlvbih0
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oZT10LG49bnVsbCxhKTplfSxhLnN0YXJ0QW5nbGU9
ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9ImZ1bmN0aW9uIj09dHlwZW9m
IHQ/dDp5bSgrdCksYSk6cn0sYS5lbmRBbmdsZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD8oaT0iZnVuY3Rpb24iPT10eXBlb2YgdD90OnltKCt0KSxhKTppfSxhLnBhZEFuZ2xl
PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhvPSJmdW5jdGlvbiI9PXR5cGVv
ZiB0P3Q6eW0oK3QpLGEpOm99LGF9LHQucGllY2V3aXNlPXBpLHQucG9pbnRSYWRpYWw9UW0sdC5w
b2ludGVyPWVlLHQucG9pbnRlcnM9ZnVuY3Rpb24odCxuKXtyZXR1cm4gdC50YXJnZXQmJih0PW5l
KHQpLHZvaWQgMD09PW4mJihuPXQuY3VycmVudFRhcmdldCksdD10LnRvdWNoZXN8fFt0XSksQXJy
YXkuZnJvbSh0LCh0PT5lZSh0LG4pKSl9LHQucG9seWdvbkFyZWE9ZnVuY3Rpb24odCl7Zm9yKHZh
ciBuLGU9LTEscj10Lmxlbmd0aCxpPXRbci0xXSxvPTA7KytlPHI7KW49aSxpPXRbZV0sbys9blsx
XSppWzBdLW5bMF0qaVsxXTtyZXR1cm4gby8yfSx0LnBvbHlnb25DZW50cm9pZD1mdW5jdGlvbih0
KXtmb3IodmFyIG4sZSxyPS0xLGk9dC5sZW5ndGgsbz0wLGE9MCx1PXRbaS0xXSxjPTA7KytyPGk7
KW49dSx1PXRbcl0sYys9ZT1uWzBdKnVbMV0tdVswXSpuWzFdLG8rPShuWzBdK3VbMF0pKmUsYSs9
KG5bMV0rdVsxXSkqZTtyZXR1cm5bby8oYyo9MyksYS9jXX0sdC5wb2x5Z29uQ29udGFpbnM9ZnVu
Y3Rpb24odCxuKXtmb3IodmFyIGUscixpPXQubGVuZ3RoLG89dFtpLTFdLGE9blswXSx1PW5bMV0s
Yz1vWzBdLGY9b1sxXSxzPSExLGw9MDtsPGk7KytsKWU9KG89dFtsXSlbMF0sKHI9b1sxXSk+dSE9
Zj51JiZhPChjLWUpKih1LXIpLyhmLXIpK2UmJihzPSFzKSxjPWUsZj1yO3JldHVybiBzfSx0LnBv
bHlnb25IdWxsPWZ1bmN0aW9uKHQpe2lmKChlPXQubGVuZ3RoKTwzKXJldHVybiBudWxsO3ZhciBu
LGUscj1uZXcgQXJyYXkoZSksaT1uZXcgQXJyYXkoZSk7Zm9yKG49MDtuPGU7KytuKXJbbl09Wyt0
W25dWzBdLCt0W25dWzFdLG5dO2ZvcihyLnNvcnQoWHApLG49MDtuPGU7KytuKWlbbl09W3Jbbl1b
MF0sLXJbbl1bMV1dO3ZhciBvPUdwKHIpLGE9R3AoaSksdT1hWzBdPT09b1swXSxjPWFbYS5sZW5n
dGgtMV09PT1vW28ubGVuZ3RoLTFdLGY9W107Zm9yKG49by5sZW5ndGgtMTtuPj0wOy0tbilmLnB1
c2godFtyW29bbl1dWzJdXSk7Zm9yKG49K3U7bjxhLmxlbmd0aC1jOysrbilmLnB1c2godFtyW2Fb
bl1dWzJdXSk7cmV0dXJuIGZ9LHQucG9seWdvbkxlbmd0aD1mdW5jdGlvbih0KXtmb3IodmFyIG4s
ZSxyPS0xLGk9dC5sZW5ndGgsbz10W2ktMV0sYT1vWzBdLHU9b1sxXSxjPTA7KytyPGk7KW49YSxl
PXUsbi09YT0obz10W3JdKVswXSxlLT11PW9bMV0sYys9TWF0aC5oeXBvdChuLGUpO3JldHVybiBj
fSx0LnByZWNpc2lvbkZpeGVkPWxmLHQucHJlY2lzaW9uUHJlZml4PWhmLHQucHJlY2lzaW9uUm91
bmQ9ZGYsdC5xdWFkdHJlZT1EYyx0LnF1YW50aWxlPXV0LHQucXVhbnRpbGVJbmRleD1mdCx0LnF1
YW50aWxlU29ydGVkPWN0LHQucXVhbnRpemU9ZnVuY3Rpb24odCxuKXtmb3IodmFyIGU9bmV3IEFy
cmF5KG4pLHI9MDtyPG47KytyKWVbcl09dChyLyhuLTEpKTtyZXR1cm4gZX0sdC5xdWlja3NlbGVj
dD1pdCx0LnJhZGlhbEFyZWE9S20sdC5yYWRpYWxMaW5lPVptLHQucmFuZG9tQmF0ZXM9dGcsdC5y
YW5kb21CZXJub3VsbGk9cmcsdC5yYW5kb21CZXRhPWFnLHQucmFuZG9tQmlub21pYWw9dWcsdC5y
YW5kb21DYXVjaHk9ZmcsdC5yYW5kb21FeHBvbmVudGlhbD1uZyx0LnJhbmRvbUdhbW1hPW9nLHQu
cmFuZG9tR2VvbWV0cmljPWlnLHQucmFuZG9tSW50PVpwLHQucmFuZG9tSXJ3aW5IYWxsPUpwLHQu
cmFuZG9tTGNnPWZ1bmN0aW9uKHQ9TWF0aC5yYW5kb20oKSl7bGV0IG49MHwoMDw9dCYmdDwxP3Qv
aGc6TWF0aC5hYnModCkpO3JldHVybigpPT4obj0xNjY0NTI1Km4rMTAxMzkwNDIyM3wwLGhnKihu
Pj4+MCkpfSx0LnJhbmRvbUxvZ05vcm1hbD1RcCx0LnJhbmRvbUxvZ2lzdGljPXNnLHQucmFuZG9t
Tm9ybWFsPUtwLHQucmFuZG9tUGFyZXRvPWVnLHQucmFuZG9tUG9pc3Nvbj1sZyx0LnJhbmRvbVVu
aWZvcm09V3AsdC5yYW5kb21XZWlidWxsPWNnLHQucmFuZ2U9aHQsdC5yYW5rPWZ1bmN0aW9uKHQs
ZT1uKXtpZigiZnVuY3Rpb24iIT10eXBlb2YgdFtTeW1ib2wuaXRlcmF0b3JdKXRocm93IG5ldyBU
eXBlRXJyb3IoInZhbHVlcyBpcyBub3QgaXRlcmFibGUiKTtsZXQgcj1BcnJheS5mcm9tKHQpO2Nv
bnN0IGk9bmV3IEZsb2F0NjRBcnJheShyLmxlbmd0aCk7MiE9PWUubGVuZ3RoJiYocj1yLm1hcChl
KSxlPW4pO2NvbnN0IG89KHQsbik9PmUoclt0XSxyW25dKTtsZXQgYSx1O3JldHVybih0PVVpbnQz
MkFycmF5LmZyb20ociwoKHQsbik9Pm4pKSkuc29ydChlPT09bj8odCxuKT0+QihyW3RdLHJbbl0p
Ok8obykpLHQuZm9yRWFjaCgoKHQsbik9Pntjb25zdCBlPW8odCx2b2lkIDA9PT1hP3Q6YSk7ZT49
MD8oKHZvaWQgMD09PWF8fGU+MCkmJihhPXQsdT1uKSxpW3RdPXUpOmlbdF09TmFOfSkpLGl9LHQu
cmVkdWNlPWZ1bmN0aW9uKHQsbixlKXtpZigiZnVuY3Rpb24iIT10eXBlb2Ygbil0aHJvdyBuZXcg
VHlwZUVycm9yKCJyZWR1Y2VyIGlzIG5vdCBhIGZ1bmN0aW9uIik7Y29uc3Qgcj10W1N5bWJvbC5p
dGVyYXRvcl0oKTtsZXQgaSxvLGE9LTE7aWYoYXJndW1lbnRzLmxlbmd0aDwzKXtpZigoe2RvbmU6
aSx2YWx1ZTplfT1yLm5leHQoKSksaSlyZXR1cm47KythfWZvcig7KHtkb25lOmksdmFsdWU6b309
ci5uZXh0KCkpLCFpOyllPW4oZSxvLCsrYSx0KTtyZXR1cm4gZX0sdC5yZXZlcnNlPWZ1bmN0aW9u
KHQpe2lmKCJmdW5jdGlvbiIhPXR5cGVvZiB0W1N5bWJvbC5pdGVyYXRvcl0pdGhyb3cgbmV3IFR5
cGVFcnJvcigidmFsdWVzIGlzIG5vdCBpdGVyYWJsZSIpO3JldHVybiBBcnJheS5mcm9tKHQpLnJl
dmVyc2UoKX0sdC5yZ2I9cWUsdC5yaWJib249ZnVuY3Rpb24oKXtyZXR1cm4gWmEoKX0sdC5yaWJi
b25BcnJvdz1mdW5jdGlvbigpe3JldHVybiBaYShXYSl9LHQucm9sbHVwPUQsdC5yb2xsdXBzPVIs
dC5zY2FsZUJhbmQ9dmcsdC5zY2FsZURpdmVyZ2luZz1mdW5jdGlvbiB0KCl7dmFyIG49a2coal8o
KSh4ZykpO3JldHVybiBuLmNvcHk9ZnVuY3Rpb24oKXtyZXR1cm4gWV8obix0KCkpfSxwZy5hcHBs
eShuLGFyZ3VtZW50cyl9LHQuc2NhbGVEaXZlcmdpbmdMb2c9ZnVuY3Rpb24gdCgpe3ZhciBuPXFn
KGpfKCkpLmRvbWFpbihbLjEsMSwxMF0pO3JldHVybiBuLmNvcHk9ZnVuY3Rpb24oKXtyZXR1cm4g
WV8obix0KCkpLmJhc2Uobi5iYXNlKCkpfSxwZy5hcHBseShuLGFyZ3VtZW50cyl9LHQuc2NhbGVE
aXZlcmdpbmdQb3c9SF8sdC5zY2FsZURpdmVyZ2luZ1NxcnQ9ZnVuY3Rpb24oKXtyZXR1cm4gSF8u
YXBwbHkobnVsbCxhcmd1bWVudHMpLmV4cG9uZW50KC41KX0sdC5zY2FsZURpdmVyZ2luZ1N5bWxv
Zz1mdW5jdGlvbiB0KCl7dmFyIG49T2coal8oKSk7cmV0dXJuIG4uY29weT1mdW5jdGlvbigpe3Jl
dHVybiBZXyhuLHQoKSkuY29uc3RhbnQobi5jb25zdGFudCgpKX0scGcuYXBwbHkobixhcmd1bWVu
dHMpfSx0LnNjYWxlSWRlbnRpdHk9ZnVuY3Rpb24gdChuKXt2YXIgZTtmdW5jdGlvbiByKHQpe3Jl
dHVybiBudWxsPT10fHxpc05hTih0PSt0KT9lOnR9cmV0dXJuIHIuaW52ZXJ0PXIsci5kb21haW49
ci5yYW5nZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obj1BcnJheS5mcm9t
KHQsYmcpLHIpOm4uc2xpY2UoKX0sci51bmtub3duPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVu
dHMubGVuZ3RoPyhlPXQscik6ZX0sci5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIHQobikudW5rbm93
bihlKX0sbj1hcmd1bWVudHMubGVuZ3RoP0FycmF5LmZyb20obixiZyk6WzAsMV0sa2cocil9LHQu
c2NhbGVJbXBsaWNpdD1nZyx0LnNjYWxlTGluZWFyPWZ1bmN0aW9uIHQoKXt2YXIgbj1FZygpO3Jl
dHVybiBuLmNvcHk9ZnVuY3Rpb24oKXtyZXR1cm4gQWcobix0KCkpfSxkZy5hcHBseShuLGFyZ3Vt
ZW50cyksa2cobil9LHQuc2NhbGVMb2c9ZnVuY3Rpb24gdCgpe2NvbnN0IG49cWcoU2coKSkuZG9t
YWluKFsxLDEwXSk7cmV0dXJuIG4uY29weT0oKT0+QWcobix0KCkpLmJhc2Uobi5iYXNlKCkpLGRn
LmFwcGx5KG4sYXJndW1lbnRzKSxufSx0LnNjYWxlT3JkaW5hbD15Zyx0LnNjYWxlUG9pbnQ9ZnVu
Y3Rpb24oKXtyZXR1cm4gX2codmcuYXBwbHkobnVsbCxhcmd1bWVudHMpLnBhZGRpbmdJbm5lcigx
KSl9LHQuc2NhbGVQb3c9SGcsdC5zY2FsZVF1YW50aWxlPWZ1bmN0aW9uIHQoKXt2YXIgZSxyPVtd
LGk9W10sbz1bXTtmdW5jdGlvbiBhKCl7dmFyIHQ9MCxuPU1hdGgubWF4KDEsaS5sZW5ndGgpO2Zv
cihvPW5ldyBBcnJheShuLTEpOysrdDxuOylvW3QtMV09Y3Qocix0L24pO3JldHVybiB1fWZ1bmN0
aW9uIHUodCl7cmV0dXJuIG51bGw9PXR8fGlzTmFOKHQ9K3QpP2U6aVtsKG8sdCldfXJldHVybiB1
LmludmVydEV4dGVudD1mdW5jdGlvbih0KXt2YXIgbj1pLmluZGV4T2YodCk7cmV0dXJuIG48MD9b
TmFOLE5hTl06W24+MD9vW24tMV06clswXSxuPG8ubGVuZ3RoP29bbl06cltyLmxlbmd0aC0xXV19
LHUuZG9tYWluPWZ1bmN0aW9uKHQpe2lmKCFhcmd1bWVudHMubGVuZ3RoKXJldHVybiByLnNsaWNl
KCk7cj1bXTtmb3IobGV0IG4gb2YgdCludWxsPT1ufHxpc05hTihuPStuKXx8ci5wdXNoKG4pO3Jl
dHVybiByLnNvcnQobiksYSgpfSx1LnJhbmdlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMu
bGVuZ3RoPyhpPUFycmF5LmZyb20odCksYSgpKTppLnNsaWNlKCl9LHUudW5rbm93bj1mdW5jdGlv
bih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oZT10LHUpOmV9LHUucXVhbnRpbGVzPWZ1bmN0
aW9uKCl7cmV0dXJuIG8uc2xpY2UoKX0sdS5jb3B5PWZ1bmN0aW9uKCl7cmV0dXJuIHQoKS5kb21h
aW4ocikucmFuZ2UoaSkudW5rbm93bihlKX0sZGcuYXBwbHkodSxhcmd1bWVudHMpfSx0LnNjYWxl
UXVhbnRpemU9ZnVuY3Rpb24gdCgpe3ZhciBuLGU9MCxyPTEsaT0xLG89Wy41XSxhPVswLDFdO2Z1
bmN0aW9uIHUodCl7cmV0dXJuIG51bGwhPXQmJnQ8PXQ/YVtsKG8sdCwwLGkpXTpufWZ1bmN0aW9u
IGMoKXt2YXIgdD0tMTtmb3Iobz1uZXcgQXJyYXkoaSk7Kyt0PGk7KW9bdF09KCh0KzEpKnItKHQt
aSkqZSkvKGkrMSk7cmV0dXJuIHV9cmV0dXJuIHUuZG9tYWluPWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyhbZSxyXT10LGU9K2Uscj0rcixjKCkpOltlLHJdfSx1LnJhbmdlPWZ1
bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhpPShhPUFycmF5LmZyb20odCkpLmxl
bmd0aC0xLGMoKSk6YS5zbGljZSgpfSx1LmludmVydEV4dGVudD1mdW5jdGlvbih0KXt2YXIgbj1h
LmluZGV4T2YodCk7cmV0dXJuIG48MD9bTmFOLE5hTl06bjwxP1tlLG9bMF1dOm4+PWk/W29baS0x
XSxyXTpbb1tuLTFdLG9bbl1dfSx1LnVua25vd249ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50
cy5sZW5ndGg/KG49dCx1KTp1fSx1LnRocmVzaG9sZHM9ZnVuY3Rpb24oKXtyZXR1cm4gby5zbGlj
ZSgpfSx1LmNvcHk9ZnVuY3Rpb24oKXtyZXR1cm4gdCgpLmRvbWFpbihbZSxyXSkucmFuZ2UoYSku
dW5rbm93bihuKX0sZGcuYXBwbHkoa2codSksYXJndW1lbnRzKX0sdC5zY2FsZVJhZGlhbD1mdW5j
dGlvbiB0KCl7dmFyIG4sZT1FZygpLHI9WzAsMV0saT0hMTtmdW5jdGlvbiBvKHQpe3ZhciByPWZ1
bmN0aW9uKHQpe3JldHVybiBNYXRoLnNpZ24odCkqTWF0aC5zcXJ0KE1hdGguYWJzKHQpKX0oZSh0
KSk7cmV0dXJuIGlzTmFOKHIpP246aT9NYXRoLnJvdW5kKHIpOnJ9cmV0dXJuIG8uaW52ZXJ0PWZ1
bmN0aW9uKHQpe3JldHVybiBlLmludmVydChYZyh0KSl9LG8uZG9tYWluPWZ1bmN0aW9uKHQpe3Jl
dHVybiBhcmd1bWVudHMubGVuZ3RoPyhlLmRvbWFpbih0KSxvKTplLmRvbWFpbigpfSxvLnJhbmdl
PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhlLnJhbmdlKChyPUFycmF5LmZy
b20odCxiZykpLm1hcChYZykpLG8pOnIuc2xpY2UoKX0sby5yYW5nZVJvdW5kPWZ1bmN0aW9uKHQp
e3JldHVybiBvLnJhbmdlKHQpLnJvdW5kKCEwKX0sby5yb3VuZD1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8oaT0hIXQsbyk6aX0sby5jbGFtcD1mdW5jdGlvbih0KXtyZXR1cm4g
YXJndW1lbnRzLmxlbmd0aD8oZS5jbGFtcCh0KSxvKTplLmNsYW1wKCl9LG8udW5rbm93bj1mdW5j
dGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obj10LG8pOm59LG8uY29weT1mdW5jdGlv
bigpe3JldHVybiB0KGUuZG9tYWluKCkscikucm91bmQoaSkuY2xhbXAoZS5jbGFtcCgpKS51bmtu
b3duKG4pfSxkZy5hcHBseShvLGFyZ3VtZW50cyksa2cobyl9LHQuc2NhbGVTZXF1ZW50aWFsPWZ1
bmN0aW9uIHQoKXt2YXIgbj1rZyhCXygpKHhnKSk7cmV0dXJuIG4uY29weT1mdW5jdGlvbigpe3Jl
dHVybiBZXyhuLHQoKSl9LHBnLmFwcGx5KG4sYXJndW1lbnRzKX0sdC5zY2FsZVNlcXVlbnRpYWxM
b2c9ZnVuY3Rpb24gdCgpe3ZhciBuPXFnKEJfKCkpLmRvbWFpbihbMSwxMF0pO3JldHVybiBuLmNv
cHk9ZnVuY3Rpb24oKXtyZXR1cm4gWV8obix0KCkpLmJhc2Uobi5iYXNlKCkpfSxwZy5hcHBseShu
LGFyZ3VtZW50cyl9LHQuc2NhbGVTZXF1ZW50aWFsUG93PUxfLHQuc2NhbGVTZXF1ZW50aWFsUXVh
bnRpbGU9ZnVuY3Rpb24gdCgpe3ZhciBlPVtdLHI9eGc7ZnVuY3Rpb24gaSh0KXtpZihudWxsIT10
JiYhaXNOYU4odD0rdCkpcmV0dXJuIHIoKGwoZSx0LDEpLTEpLyhlLmxlbmd0aC0xKSl9cmV0dXJu
IGkuZG9tYWluPWZ1bmN0aW9uKHQpe2lmKCFhcmd1bWVudHMubGVuZ3RoKXJldHVybiBlLnNsaWNl
KCk7ZT1bXTtmb3IobGV0IG4gb2YgdCludWxsPT1ufHxpc05hTihuPStuKXx8ZS5wdXNoKG4pO3Jl
dHVybiBlLnNvcnQobiksaX0saS5pbnRlcnBvbGF0b3I9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3Vt
ZW50cy5sZW5ndGg/KHI9dCxpKTpyfSxpLnJhbmdlPWZ1bmN0aW9uKCl7cmV0dXJuIGUubWFwKCgo
dCxuKT0+cihuLyhlLmxlbmd0aC0xKSkpKX0saS5xdWFudGlsZXM9ZnVuY3Rpb24odCl7cmV0dXJu
IEFycmF5LmZyb20oe2xlbmd0aDp0KzF9LCgobixyKT0+dXQoZSxyL3QpKSl9LGkuY29weT1mdW5j
dGlvbigpe3JldHVybiB0KHIpLmRvbWFpbihlKX0scGcuYXBwbHkoaSxhcmd1bWVudHMpfSx0LnNj
YWxlU2VxdWVudGlhbFNxcnQ9ZnVuY3Rpb24oKXtyZXR1cm4gTF8uYXBwbHkobnVsbCxhcmd1bWVu
dHMpLmV4cG9uZW50KC41KX0sdC5zY2FsZVNlcXVlbnRpYWxTeW1sb2c9ZnVuY3Rpb24gdCgpe3Zh
ciBuPU9nKEJfKCkpO3JldHVybiBuLmNvcHk9ZnVuY3Rpb24oKXtyZXR1cm4gWV8obix0KCkpLmNv
bnN0YW50KG4uY29uc3RhbnQoKSl9LHBnLmFwcGx5KG4sYXJndW1lbnRzKX0sdC5zY2FsZVNxcnQ9
ZnVuY3Rpb24oKXtyZXR1cm4gSGcuYXBwbHkobnVsbCxhcmd1bWVudHMpLmV4cG9uZW50KC41KX0s
dC5zY2FsZVN5bWxvZz1mdW5jdGlvbiB0KCl7dmFyIG49T2coU2coKSk7cmV0dXJuIG4uY29weT1m
dW5jdGlvbigpe3JldHVybiBBZyhuLHQoKSkuY29uc3RhbnQobi5jb25zdGFudCgpKX0sZGcuYXBw
bHkobixhcmd1bWVudHMpfSx0LnNjYWxlVGhyZXNob2xkPWZ1bmN0aW9uIHQoKXt2YXIgbixlPVsu
NV0scj1bMCwxXSxpPTE7ZnVuY3Rpb24gbyh0KXtyZXR1cm4gbnVsbCE9dCYmdDw9dD9yW2woZSx0
LDAsaSldOm59cmV0dXJuIG8uZG9tYWluPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPyhlPUFycmF5LmZyb20odCksaT1NYXRoLm1pbihlLmxlbmd0aCxyLmxlbmd0aC0xKSxvKTpl
LnNsaWNlKCl9LG8ucmFuZ2U9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHI9
QXJyYXkuZnJvbSh0KSxpPU1hdGgubWluKGUubGVuZ3RoLHIubGVuZ3RoLTEpLG8pOnIuc2xpY2Uo
KX0sby5pbnZlcnRFeHRlbnQ9ZnVuY3Rpb24odCl7dmFyIG49ci5pbmRleE9mKHQpO3JldHVybltl
W24tMV0sZVtuXV19LG8udW5rbm93bj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0
aD8obj10LG8pOm59LG8uY29weT1mdW5jdGlvbigpe3JldHVybiB0KCkuZG9tYWluKGUpLnJhbmdl
KHIpLnVua25vd24obil9LGRnLmFwcGx5KG8sYXJndW1lbnRzKX0sdC5zY2FsZVRpbWU9ZnVuY3Rp
b24oKXtyZXR1cm4gZGcuYXBwbHkoT18oY3YsZnYsbnYsS3ksd3ksZ3ksbHksdXksb3ksdC50aW1l
Rm9ybWF0KS5kb21haW4oW25ldyBEYXRlKDJlMywwLDEpLG5ldyBEYXRlKDJlMywwLDIpXSksYXJn
dW1lbnRzKX0sdC5zY2FsZVV0Yz1mdW5jdGlvbigpe3JldHVybiBkZy5hcHBseShPXyhhdix1dixy
dixKeSxxeSx2eSxkeSxmeSxveSx0LnV0Y0Zvcm1hdCkuZG9tYWluKFtEYXRlLlVUQygyZTMsMCwx
KSxEYXRlLlVUQygyZTMsMCwyKV0pLGFyZ3VtZW50cyl9LHQuc2Nhbj1mdW5jdGlvbih0LG4pe2Nv
bnN0IGU9ZHQodCxuKTtyZXR1cm4gZTwwP3ZvaWQgMDplfSx0LnNjaGVtZUFjY2VudD1WXyx0LnNj
aGVtZUJsdWVzPVhiLHQuc2NoZW1lQnJCRz1pYix0LnNjaGVtZUJ1R249d2IsdC5zY2hlbWVCdVB1
PVRiLHQuc2NoZW1lQ2F0ZWdvcnkxMD1HXyx0LnNjaGVtZURhcmsyPVdfLHQuc2NoZW1lR25CdT1T
Yix0LnNjaGVtZUdyZWVucz1WYix0LnNjaGVtZUdyZXlzPVpiLHQuc2NoZW1lT3JSZD1OYix0LnNj
aGVtZU9yYW5nZXM9ZW0sdC5zY2hlbWVQUkduPWFiLHQuc2NoZW1lUGFpcmVkPVpfLHQuc2NoZW1l
UGFzdGVsMT1LXyx0LnNjaGVtZVBhc3RlbDI9UV8sdC5zY2hlbWVQaVlHPWNiLHQuc2NoZW1lUHVC
dT16Yix0LnNjaGVtZVB1QnVHbj1DYix0LnNjaGVtZVB1T3I9c2IsdC5zY2hlbWVQdVJkPURiLHQu
c2NoZW1lUHVycGxlcz1RYix0LnNjaGVtZVJkQnU9aGIsdC5zY2hlbWVSZEd5PXBiLHQuc2NoZW1l
UmRQdT1GYix0LnNjaGVtZVJkWWxCdT15Yix0LnNjaGVtZVJkWWxHbj1fYix0LnNjaGVtZVJlZHM9
dG0sdC5zY2hlbWVTZXQxPUpfLHQuc2NoZW1lU2V0Mj10Yix0LnNjaGVtZVNldDM9bmIsdC5zY2hl
bWVTcGVjdHJhbD1tYix0LnNjaGVtZVRhYmxlYXUxMD1lYix0LnNjaGVtZVlsR249T2IsdC5zY2hl
bWVZbEduQnU9VWIsdC5zY2hlbWVZbE9yQnI9WWIsdC5zY2hlbWVZbE9yUmQ9amIsdC5zZWxlY3Q9
S24sdC5zZWxlY3RBbGw9ZnVuY3Rpb24odCl7cmV0dXJuInN0cmluZyI9PXR5cGVvZiB0P25ldyBX
bihbZG9jdW1lbnQucXVlcnlTZWxlY3RvckFsbCh0KV0sW2RvY3VtZW50LmRvY3VtZW50RWxlbWVu
dF0pOm5ldyBXbihbWHQodCldLFZuKX0sdC5zZWxlY3Rpb249Wm4sdC5zZWxlY3Rvcj1IdCx0LnNl
bGVjdG9yQWxsPVZ0LHQuc2h1ZmZsZT1wdCx0LnNodWZmbGVyPWd0LHQuc29tZT1mdW5jdGlvbih0
LG4pe2lmKCJmdW5jdGlvbiIhPXR5cGVvZiBuKXRocm93IG5ldyBUeXBlRXJyb3IoInRlc3QgaXMg
bm90IGEgZnVuY3Rpb24iKTtsZXQgZT0tMTtmb3IoY29uc3QgciBvZiB0KWlmKG4ociwrK2UsdCkp
cmV0dXJuITA7cmV0dXJuITF9LHQuc29ydD1JLHQuc3RhY2s9ZnVuY3Rpb24oKXt2YXIgdD15bShb
XSksbj1kdyxlPWh3LHI9cHc7ZnVuY3Rpb24gaShpKXt2YXIgbyxhLHU9QXJyYXkuZnJvbSh0LmFw
cGx5KHRoaXMsYXJndW1lbnRzKSxndyksYz11Lmxlbmd0aCxmPS0xO2Zvcihjb25zdCB0IG9mIGkp
Zm9yKG89MCwrK2Y7bzxjOysrbykodVtvXVtmXT1bMCwrcih0LHVbb10ua2V5LGYsaSldKS5kYXRh
PXQ7Zm9yKG89MCxhPXFtKG4odSkpO288YzsrK28pdVthW29dXS5pbmRleD1vO3JldHVybiBlKHUs
YSksdX1yZXR1cm4gaS5rZXlzPWZ1bmN0aW9uKG4pe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh0
PSJmdW5jdGlvbiI9PXR5cGVvZiBuP246eW0oQXJyYXkuZnJvbShuKSksaSk6dH0saS52YWx1ZT1m
dW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj0iZnVuY3Rpb24iPT10eXBlb2Yg
dD90OnltKCt0KSxpKTpyfSxpLm9yZGVyPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVu
Z3RoPyhuPW51bGw9PXQ/ZHc6ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDp5bShBcnJheS5mcm9tKHQp
KSxpKTpufSxpLm9mZnNldD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oZT1u
dWxsPT10P2h3OnQsaSk6ZX0saX0sdC5zdGFja09mZnNldERpdmVyZ2luZz1mdW5jdGlvbih0LG4p
e2lmKCh1PXQubGVuZ3RoKT4wKWZvcih2YXIgZSxyLGksbyxhLHUsYz0wLGY9dFtuWzBdXS5sZW5n
dGg7YzxmOysrYylmb3Iobz1hPTAsZT0wO2U8dTsrK2UpKGk9KHI9dFtuW2VdXVtjXSlbMV0tclsw
XSk+MD8oclswXT1vLHJbMV09bys9aSk6aTwwPyhyWzFdPWEsclswXT1hKz1pKTooclswXT0wLHJb
MV09aSl9LHQuc3RhY2tPZmZzZXRFeHBhbmQ9ZnVuY3Rpb24odCxuKXtpZigocj10Lmxlbmd0aCk+
MCl7Zm9yKHZhciBlLHIsaSxvPTAsYT10WzBdLmxlbmd0aDtvPGE7KytvKXtmb3IoaT1lPTA7ZTxy
OysrZSlpKz10W2VdW29dWzFdfHwwO2lmKGkpZm9yKGU9MDtlPHI7KytlKXRbZV1bb11bMV0vPWl9
aHcodCxuKX19LHQuc3RhY2tPZmZzZXROb25lPWh3LHQuc3RhY2tPZmZzZXRTaWxob3VldHRlPWZ1
bmN0aW9uKHQsbil7aWYoKGU9dC5sZW5ndGgpPjApe2Zvcih2YXIgZSxyPTAsaT10W25bMF1dLG89
aS5sZW5ndGg7cjxvOysrcil7Zm9yKHZhciBhPTAsdT0wO2E8ZTsrK2EpdSs9dFthXVtyXVsxXXx8
MDtpW3JdWzFdKz1pW3JdWzBdPS11LzJ9aHcodCxuKX19LHQuc3RhY2tPZmZzZXRXaWdnbGU9ZnVu
Y3Rpb24odCxuKXtpZigoaT10Lmxlbmd0aCk+MCYmKHI9KGU9dFtuWzBdXSkubGVuZ3RoKT4wKXtm
b3IodmFyIGUscixpLG89MCxhPTE7YTxyOysrYSl7Zm9yKHZhciB1PTAsYz0wLGY9MDt1PGk7Kyt1
KXtmb3IodmFyIHM9dFtuW3VdXSxsPXNbYV1bMV18fDAsaD0obC0oc1thLTFdWzFdfHwwKSkvMixk
PTA7ZDx1OysrZCl7dmFyIHA9dFtuW2RdXTtoKz0ocFthXVsxXXx8MCktKHBbYS0xXVsxXXx8MCl9
Yys9bCxmKz1oKmx9ZVthLTFdWzFdKz1lW2EtMV1bMF09byxjJiYoby09Zi9jKX1lW2EtMV1bMV0r
PWVbYS0xXVswXT1vLGh3KHQsbil9fSx0LnN0YWNrT3JkZXJBcHBlYXJhbmNlPXl3LHQuc3RhY2tP
cmRlckFzY2VuZGluZz1fdyx0LnN0YWNrT3JkZXJEZXNjZW5kaW5nPWZ1bmN0aW9uKHQpe3JldHVy
biBfdyh0KS5yZXZlcnNlKCl9LHQuc3RhY2tPcmRlckluc2lkZU91dD1mdW5jdGlvbih0KXt2YXIg
bixlLHI9dC5sZW5ndGgsaT10Lm1hcChidyksbz15dyh0KSxhPTAsdT0wLGM9W10sZj1bXTtmb3Io
bj0wO248cjsrK24pZT1vW25dLGE8dT8oYSs9aVtlXSxjLnB1c2goZSkpOih1Kz1pW2VdLGYucHVz
aChlKSk7cmV0dXJuIGYucmV2ZXJzZSgpLmNvbmNhdChjKX0sdC5zdGFja09yZGVyTm9uZT1kdyx0
LnN0YWNrT3JkZXJSZXZlcnNlPWZ1bmN0aW9uKHQpe3JldHVybiBkdyh0KS5yZXZlcnNlKCl9LHQu
c3RyYXRpZnk9ZnVuY3Rpb24oKXt2YXIgdCxuPUNwLGU9UHA7ZnVuY3Rpb24gcihyKXt2YXIgaSxv
LGEsdSxjLGYscyxsLGg9QXJyYXkuZnJvbShyKSxkPW4scD1lLGc9bmV3IE1hcDtpZihudWxsIT10
KXtjb25zdCBuPWgubWFwKCgobixlKT0+ZnVuY3Rpb24odCl7dD1gJHt0fWA7bGV0IG49dC5sZW5n
dGg7JHAodCxuLTEpJiYhJHAodCxuLTIpJiYodD10LnNsaWNlKDAsLTEpKTtyZXR1cm4iLyI9PT10
WzBdP3Q6YC8ke3R9YH0odChuLGUscikpKSksZT1uLm1hcCh6cCksaT1uZXcgU2V0KG4pLmFkZCgi
Iik7Zm9yKGNvbnN0IHQgb2YgZSlpLmhhcyh0KXx8KGkuYWRkKHQpLG4ucHVzaCh0KSxlLnB1c2go
enAodCkpLGgucHVzaChrcCkpO2Q9KHQsZSk9Pm5bZV0scD0odCxuKT0+ZVtuXX1mb3IoYT0wLGk9
aC5sZW5ndGg7YTxpOysrYSlvPWhbYV0sZj1oW2FdPW5ldyBKZChvKSxudWxsIT0ocz1kKG8sYSxy
KSkmJihzKz0iIikmJihsPWYuaWQ9cyxnLnNldChsLGcuaGFzKGwpP05wOmYpKSxudWxsIT0ocz1w
KG8sYSxyKSkmJihzKz0iIikmJihmLnBhcmVudD1zKTtmb3IoYT0wO2E8aTsrK2EpaWYocz0oZj1o
W2FdKS5wYXJlbnQpe2lmKCEoYz1nLmdldChzKSkpdGhyb3cgbmV3IEVycm9yKCJtaXNzaW5nOiAi
K3MpO2lmKGM9PT1OcCl0aHJvdyBuZXcgRXJyb3IoImFtYmlndW91czogIitzKTtjLmNoaWxkcmVu
P2MuY2hpbGRyZW4ucHVzaChmKTpjLmNoaWxkcmVuPVtmXSxmLnBhcmVudD1jfWVsc2V7aWYodSl0
aHJvdyBuZXcgRXJyb3IoIm11bHRpcGxlIHJvb3RzIik7dT1mfWlmKCF1KXRocm93IG5ldyBFcnJv
cigibm8gcm9vdCIpO2lmKG51bGwhPXQpe2Zvcig7dS5kYXRhPT09a3AmJjE9PT11LmNoaWxkcmVu
Lmxlbmd0aDspdT11LmNoaWxkcmVuWzBdLC0taTtmb3IobGV0IHQ9aC5sZW5ndGgtMTt0Pj0wJiYo
Zj1oW3RdKS5kYXRhPT09a3A7LS10KWYuZGF0YT1udWxsfWlmKHUucGFyZW50PUVwLHUuZWFjaEJl
Zm9yZSgoZnVuY3Rpb24odCl7dC5kZXB0aD10LnBhcmVudC5kZXB0aCsxLC0taX0pKS5lYWNoQmVm
b3JlKFFkKSx1LnBhcmVudD1udWxsLGk+MCl0aHJvdyBuZXcgRXJyb3IoImN5Y2xlIik7cmV0dXJu
IHV9cmV0dXJuIHIuaWQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49dHAo
dCkscik6bn0sci5wYXJlbnRJZD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8o
ZT10cCh0KSxyKTplfSxyLnBhdGg9ZnVuY3Rpb24obil7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KHQ9dHAobikscik6dH0scn0sdC5zdHlsZT1ibix0LnN1YnNldD1mdW5jdGlvbih0LG4pe3JldHVy
biBidChuLHQpfSx0LnN1bT1mdW5jdGlvbih0LG4pe2xldCBlPTA7aWYodm9pZCAwPT09bilmb3Io
bGV0IG4gb2YgdCkobj0rbikmJihlKz1uKTtlbHNle2xldCByPS0xO2ZvcihsZXQgaSBvZiB0KShp
PStuKGksKytyLHQpKSYmKGUrPWkpfXJldHVybiBlfSx0LnN1cGVyc2V0PWJ0LHQuc3ZnPWtjLHQu
c3ltYm9sPWZ1bmN0aW9uKHQsbil7bGV0IGU9bnVsbCxyPWttKGkpO2Z1bmN0aW9uIGkoKXtsZXQg
aTtpZihlfHwoZT1pPXIoKSksdC5hcHBseSh0aGlzLGFyZ3VtZW50cykuZHJhdyhlLCtuLmFwcGx5
KHRoaXMsYXJndW1lbnRzKSksaSlyZXR1cm4gZT1udWxsLGkrIiJ8fG51bGx9cmV0dXJuIHQ9ImZ1
bmN0aW9uIj09dHlwZW9mIHQ/dDp5bSh0fHxmeCksbj0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOnlt
KHZvaWQgMD09PW4/NjQ6K24pLGkudHlwZT1mdW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRzLmxl
bmd0aD8odD0iZnVuY3Rpb24iPT10eXBlb2Ygbj9uOnltKG4pLGkpOnR9LGkuc2l6ZT1mdW5jdGlv
bih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obj0iZnVuY3Rpb24iPT10eXBlb2YgdD90Onlt
KCt0KSxpKTpufSxpLmNvbnRleHQ9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KGU9bnVsbD09dD9udWxsOnQsaSk6ZX0saX0sdC5zeW1ib2xBc3Rlcmlzaz1jeCx0LnN5bWJvbENp
cmNsZT1meCx0LnN5bWJvbENyb3NzPXN4LHQuc3ltYm9sRGlhbW9uZD1keCx0LnN5bWJvbERpYW1v
bmQyPXB4LHQuc3ltYm9sUGx1cz1neCx0LnN5bWJvbFNxdWFyZT15eCx0LnN5bWJvbFNxdWFyZTI9
dngsdC5zeW1ib2xTdGFyPXh4LHQuc3ltYm9sVGltZXM9UHgsdC5zeW1ib2xUcmlhbmdsZT1NeCx0
LnN5bWJvbFRyaWFuZ2xlMj1BeCx0LnN5bWJvbFd5ZT1DeCx0LnN5bWJvbFg9UHgsdC5zeW1ib2xz
PXp4LHQuc3ltYm9sc0ZpbGw9engsdC5zeW1ib2xzU3Ryb2tlPSR4LHQudGV4dD14Yyx0LnRocmVz
aG9sZEZyZWVkbWFuRGlhY29uaXM9ZnVuY3Rpb24odCxuLGUpe2NvbnN0IHI9Xyh0KSxpPXV0KHQs
Ljc1KS11dCh0LC4yNSk7cmV0dXJuIHImJmk/TWF0aC5jZWlsKChlLW4pLygyKmkqTWF0aC5wb3co
ciwtMS8zKSkpOjF9LHQudGhyZXNob2xkU2NvdHQ9ZnVuY3Rpb24odCxuLGUpe2NvbnN0IHI9Xyh0
KSxpPU0odCk7cmV0dXJuIHImJmk/TWF0aC5jZWlsKChlLW4pKk1hdGguY2JydChyKS8oMy40OSpp
KSk6MX0sdC50aHJlc2hvbGRTdHVyZ2VzPVEsdC50aWNrRm9ybWF0PU5nLHQudGlja0luY3JlbWVu
dD1XLHQudGlja1N0ZXA9Wix0LnRpY2tzPVYsdC50aW1lRGF5PWd5LHQudGltZURheXM9eXksdC50
aW1lRm9ybWF0RGVmYXVsdExvY2FsZT16Xyx0LnRpbWVGb3JtYXRMb2NhbGU9ZHYsdC50aW1lRnJp
ZGF5PUV5LHQudGltZUZyaWRheXM9RHksdC50aW1lSG91cj1seSx0LnRpbWVIb3Vycz1oeSx0LnRp
bWVJbnRlcnZhbD1XZyx0LnRpbWVNaWxsaXNlY29uZD1aZyx0LnRpbWVNaWxsaXNlY29uZHM9S2cs
dC50aW1lTWludXRlPXV5LHQudGltZU1pbnV0ZXM9Y3ksdC50aW1lTW9uZGF5PU15LHQudGltZU1v
bmRheXM9Q3ksdC50aW1lTW9udGg9S3ksdC50aW1lTW9udGhzPVF5LHQudGltZVNhdHVyZGF5PU55
LHQudGltZVNhdHVyZGF5cz1SeSx0LnRpbWVTZWNvbmQ9b3ksdC50aW1lU2Vjb25kcz1heSx0LnRp
bWVTdW5kYXk9d3ksdC50aW1lU3VuZGF5cz1reSx0LnRpbWVUaHVyc2RheT1TeSx0LnRpbWVUaHVy
c2RheXM9JHksdC50aW1lVGlja0ludGVydmFsPWZ2LHQudGltZVRpY2tzPWN2LHQudGltZVR1ZXNk
YXk9VHksdC50aW1lVHVlc2RheXM9UHksdC50aW1lV2VkbmVzZGF5PUF5LHQudGltZVdlZG5lc2Rh
eXM9enksdC50aW1lV2Vlaz13eSx0LnRpbWVXZWVrcz1reSx0LnRpbWVZZWFyPW52LHQudGltZVll
YXJzPWV2LHQudGltZW91dD1EaSx0LnRpbWVyPWtpLHQudGltZXJGbHVzaD1DaSx0LnRyYW5zaXRp
b249eW8sdC50cmFuc3Bvc2U9eXQsdC50cmVlPWZ1bmN0aW9uKCl7dmFyIHQ9RHAsbj0xLGU9MSxy
PW51bGw7ZnVuY3Rpb24gaShpKXt2YXIgYz1mdW5jdGlvbih0KXtmb3IodmFyIG4sZSxyLGksbyxh
PW5ldyBJcCh0LDApLHU9W2FdO249dS5wb3AoKTspaWYocj1uLl8uY2hpbGRyZW4pZm9yKG4uY2hp
bGRyZW49bmV3IEFycmF5KG89ci5sZW5ndGgpLGk9by0xO2k+PTA7LS1pKXUucHVzaChlPW4uY2hp
bGRyZW5baV09bmV3IElwKHJbaV0saSkpLGUucGFyZW50PW47cmV0dXJuKGEucGFyZW50PW5ldyBJ
cChudWxsLDApKS5jaGlsZHJlbj1bYV0sYX0oaSk7aWYoYy5lYWNoQWZ0ZXIobyksYy5wYXJlbnQu
bT0tYy56LGMuZWFjaEJlZm9yZShhKSxyKWkuZWFjaEJlZm9yZSh1KTtlbHNle3ZhciBmPWkscz1p
LGw9aTtpLmVhY2hCZWZvcmUoKGZ1bmN0aW9uKHQpe3QueDxmLngmJihmPXQpLHQueD5zLngmJihz
PXQpLHQuZGVwdGg+bC5kZXB0aCYmKGw9dCl9KSk7dmFyIGg9Zj09PXM/MTp0KGYscykvMixkPWgt
Zi54LHA9bi8ocy54K2grZCksZz1lLyhsLmRlcHRofHwxKTtpLmVhY2hCZWZvcmUoKGZ1bmN0aW9u
KHQpe3QueD0odC54K2QpKnAsdC55PXQuZGVwdGgqZ30pKX1yZXR1cm4gaX1mdW5jdGlvbiBvKG4p
e3ZhciBlPW4uY2hpbGRyZW4scj1uLnBhcmVudC5jaGlsZHJlbixpPW4uaT9yW24uaS0xXTpudWxs
O2lmKGUpeyFmdW5jdGlvbih0KXtmb3IodmFyIG4sZT0wLHI9MCxpPXQuY2hpbGRyZW4sbz1pLmxl
bmd0aDstLW8+PTA7KShuPWlbb10pLnorPWUsbi5tKz1lLGUrPW4ucysocis9bi5jKX0obik7dmFy
IG89KGVbMF0ueitlW2UubGVuZ3RoLTFdLnopLzI7aT8obi56PWkueit0KG4uXyxpLl8pLG4ubT1u
Lnotbyk6bi56PW99ZWxzZSBpJiYobi56PWkueit0KG4uXyxpLl8pKTtuLnBhcmVudC5BPWZ1bmN0
aW9uKG4sZSxyKXtpZihlKXtmb3IodmFyIGksbz1uLGE9bix1PWUsYz1vLnBhcmVudC5jaGlsZHJl
blswXSxmPW8ubSxzPWEubSxsPXUubSxoPWMubTt1PUZwKHUpLG89UnAobyksdSYmbzspYz1ScChj
KSwoYT1GcChhKSkuYT1uLChpPXUueitsLW8uei1mK3QodS5fLG8uXykpPjAmJihxcChVcCh1LG4s
ciksbixpKSxmKz1pLHMrPWkpLGwrPXUubSxmKz1vLm0saCs9Yy5tLHMrPWEubTt1JiYhRnAoYSkm
JihhLnQ9dSxhLm0rPWwtcyksbyYmIVJwKGMpJiYoYy50PW8sYy5tKz1mLWgscj1uKX1yZXR1cm4g
cn0obixpLG4ucGFyZW50LkF8fHJbMF0pfWZ1bmN0aW9uIGEodCl7dC5fLng9dC56K3QucGFyZW50
Lm0sdC5tKz10LnBhcmVudC5tfWZ1bmN0aW9uIHUodCl7dC54Kj1uLHQueT10LmRlcHRoKmV9cmV0
dXJuIGkuc2VwYXJhdGlvbj1mdW5jdGlvbihuKXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8odD1u
LGkpOnR9LGkuc2l6ZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocj0hMSxu
PSt0WzBdLGU9K3RbMV0saSk6cj9udWxsOltuLGVdfSxpLm5vZGVTaXplPWZ1bmN0aW9uKHQpe3Jl
dHVybiBhcmd1bWVudHMubGVuZ3RoPyhyPSEwLG49K3RbMF0sZT0rdFsxXSxpKTpyP1tuLGVdOm51
bGx9LGl9LHQudHJlZW1hcD1mdW5jdGlvbigpe3ZhciB0PUxwLG49ITEsZT0xLHI9MSxpPVswXSxv
PWVwLGE9ZXAsdT1lcCxjPWVwLGY9ZXA7ZnVuY3Rpb24gcyh0KXtyZXR1cm4gdC54MD10LnkwPTAs
dC54MT1lLHQueTE9cix0LmVhY2hCZWZvcmUobCksaT1bMF0sbiYmdC5lYWNoQmVmb3JlKEFwKSx0
fWZ1bmN0aW9uIGwobil7dmFyIGU9aVtuLmRlcHRoXSxyPW4ueDArZSxzPW4ueTArZSxsPW4ueDEt
ZSxoPW4ueTEtZTtsPHImJihyPWw9KHIrbCkvMiksaDxzJiYocz1oPShzK2gpLzIpLG4ueDA9cixu
LnkwPXMsbi54MT1sLG4ueTE9aCxuLmNoaWxkcmVuJiYoZT1pW24uZGVwdGgrMV09byhuKS8yLHIr
PWYobiktZSxzKz1hKG4pLWUsKGwtPXUobiktZSk8ciYmKHI9bD0ocitsKS8yKSwoaC09YyhuKS1l
KTxzJiYocz1oPShzK2gpLzIpLHQobixyLHMsbCxoKSl9cmV0dXJuIHMucm91bmQ9ZnVuY3Rpb24o
dCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KG49ISF0LHMpOm59LHMuc2l6ZT1mdW5jdGlvbih0
KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oZT0rdFswXSxyPSt0WzFdLHMpOltlLHJdfSxzLnRp
bGU9ZnVuY3Rpb24obil7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHQ9bnAobikscyk6dH0scy5w
YWRkaW5nPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoP3MucGFkZGluZ0lubmVy
KHQpLnBhZGRpbmdPdXRlcih0KTpzLnBhZGRpbmdJbm5lcigpfSxzLnBhZGRpbmdJbm5lcj1mdW5j
dGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8obz0iZnVuY3Rpb24iPT10eXBlb2YgdD90
OnJwKCt0KSxzKTpvfSxzLnBhZGRpbmdPdXRlcj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD9zLnBhZGRpbmdUb3AodCkucGFkZGluZ1JpZ2h0KHQpLnBhZGRpbmdCb3R0b20odCku
cGFkZGluZ0xlZnQodCk6cy5wYWRkaW5nVG9wKCl9LHMucGFkZGluZ1RvcD1mdW5jdGlvbih0KXty
ZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8oYT0iZnVuY3Rpb24iPT10eXBlb2YgdD90OnJwKCt0KSxz
KTphfSxzLnBhZGRpbmdSaWdodD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8o
dT0iZnVuY3Rpb24iPT10eXBlb2YgdD90OnJwKCt0KSxzKTp1fSxzLnBhZGRpbmdCb3R0b209ZnVu
Y3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGM9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/
dDpycCgrdCkscyk6Y30scy5wYWRkaW5nTGVmdD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRz
Lmxlbmd0aD8oZj0iZnVuY3Rpb24iPT10eXBlb2YgdD90OnJwKCt0KSxzKTpmfSxzfSx0LnRyZWVt
YXBCaW5hcnk9ZnVuY3Rpb24odCxuLGUscixpKXt2YXIgbyxhLHU9dC5jaGlsZHJlbixjPXUubGVu
Z3RoLGY9bmV3IEFycmF5KGMrMSk7Zm9yKGZbMF09YT1vPTA7bzxjOysrbylmW28rMV09YSs9dVtv
XS52YWx1ZTshZnVuY3Rpb24gdChuLGUscixpLG8sYSxjKXtpZihuPj1lLTEpe3ZhciBzPXVbbl07
cmV0dXJuIHMueDA9aSxzLnkwPW8scy54MT1hLHZvaWQocy55MT1jKX12YXIgbD1mW25dLGg9ci8y
K2wsZD1uKzEscD1lLTE7Zm9yKDtkPHA7KXt2YXIgZz1kK3A+Pj4xO2ZbZ108aD9kPWcrMTpwPWd9
aC1mW2QtMV08ZltkXS1oJiZuKzE8ZCYmLS1kO3ZhciB5PWZbZF0tbCx2PXIteTtpZihhLWk+Yy1v
KXt2YXIgXz1yPyhpKnYrYSp5KS9yOmE7dChuLGQseSxpLG8sXyxjKSx0KGQsZSx2LF8sbyxhLGMp
fWVsc2V7dmFyIGI9cj8obyp2K2MqeSkvcjpjO3QobixkLHksaSxvLGEsYiksdChkLGUsdixpLGIs
YSxjKX19KDAsYyx0LnZhbHVlLG4sZSxyLGkpfSx0LnRyZWVtYXBEaWNlPVNwLHQudHJlZW1hcFJl
c3F1YXJpZnk9anAsdC50cmVlbWFwU2xpY2U9T3AsdC50cmVlbWFwU2xpY2VEaWNlPWZ1bmN0aW9u
KHQsbixlLHIsaSl7KDEmdC5kZXB0aD9PcDpTcCkodCxuLGUscixpKX0sdC50cmVlbWFwU3F1YXJp
Znk9THAsdC50c3Y9VGMsdC50c3ZGb3JtYXQ9aGMsdC50c3ZGb3JtYXRCb2R5PWRjLHQudHN2Rm9y
bWF0Um93PWdjLHQudHN2Rm9ybWF0Um93cz1wYyx0LnRzdkZvcm1hdFZhbHVlPXljLHQudHN2UGFy
c2U9c2MsdC50c3ZQYXJzZVJvd3M9bGMsdC51bmlvbj1mdW5jdGlvbiguLi50KXtjb25zdCBuPW5l
dyBJbnRlcm5TZXQ7Zm9yKGNvbnN0IGUgb2YgdClmb3IoY29uc3QgdCBvZiBlKW4uYWRkKHQpO3Jl
dHVybiBufSx0LnVuaXhEYXk9YnksdC51bml4RGF5cz1teSx0LnV0Y0RheT12eSx0LnV0Y0RheXM9
X3ksdC51dGNGcmlkYXk9WXksdC51dGNGcmlkYXlzPVd5LHQudXRjSG91cj1keSx0LnV0Y0hvdXJz
PXB5LHQudXRjTWlsbGlzZWNvbmQ9WmcsdC51dGNNaWxsaXNlY29uZHM9S2csdC51dGNNaW51dGU9
ZnksdC51dGNNaW51dGVzPXN5LHQudXRjTW9uZGF5PVV5LHQudXRjTW9uZGF5cz1IeSx0LnV0Y01v
bnRoPUp5LHQudXRjTW9udGhzPXR2LHQudXRjU2F0dXJkYXk9THksdC51dGNTYXR1cmRheXM9Wnks
dC51dGNTZWNvbmQ9b3ksdC51dGNTZWNvbmRzPWF5LHQudXRjU3VuZGF5PXF5LHQudXRjU3VuZGF5
cz1qeSx0LnV0Y1RodXJzZGF5PUJ5LHQudXRjVGh1cnNkYXlzPVZ5LHQudXRjVGlja0ludGVydmFs
PXV2LHQudXRjVGlja3M9YXYsdC51dGNUdWVzZGF5PUl5LHQudXRjVHVlc2RheXM9WHksdC51dGNX
ZWRuZXNkYXk9T3ksdC51dGNXZWRuZXNkYXlzPUd5LHQudXRjV2Vlaz1xeSx0LnV0Y1dlZWtzPWp5
LHQudXRjWWVhcj1ydix0LnV0Y1llYXJzPWl2LHQudmFyaWFuY2U9dyx0LnZlcnNpb249IjcuOC40
Iix0LndpbmRvdz1nbix0LnhtbD1FYyx0LnppcD1mdW5jdGlvbigpe3JldHVybiB5dChhcmd1bWVu
dHMpfSx0Lnpvb209ZnVuY3Rpb24oKXt2YXIgdCxuLGUscj1FdyxpPU53LG89encsYT1Ddyx1PVB3
LGM9WzAsMS8wXSxmPVtbLTEvMCwtMS8wXSxbMS8wLDEvMF1dLHM9MjUwLGw9aWksaD1EdCgic3Rh
cnQiLCJ6b29tIiwiZW5kIiksZD01MDAscD0xNTAsZz0wLHk9MTA7ZnVuY3Rpb24gdih0KXt0LnBy
b3BlcnR5KCJfX3pvb20iLGt3KS5vbigid2hlZWwuem9vbSIsVCx7cGFzc2l2ZTohMX0pLm9uKCJt
b3VzZWRvd24uem9vbSIsQSkub24oImRibGNsaWNrLnpvb20iLFMpLmZpbHRlcih1KS5vbigidG91
Y2hzdGFydC56b29tIixFKS5vbigidG91Y2htb3ZlLnpvb20iLE4pLm9uKCJ0b3VjaGVuZC56b29t
IHRvdWNoY2FuY2VsLnpvb20iLGspLnN0eWxlKCItd2Via2l0LXRhcC1oaWdobGlnaHQtY29sb3Ii
LCJyZ2JhKDAsMCwwLDApIil9ZnVuY3Rpb24gXyh0LG4pe3JldHVybihuPU1hdGgubWF4KGNbMF0s
TWF0aC5taW4oY1sxXSxuKSkpPT09dC5rP3Q6bmV3IHd3KG4sdC54LHQueSl9ZnVuY3Rpb24gYih0
LG4sZSl7dmFyIHI9blswXS1lWzBdKnQuayxpPW5bMV0tZVsxXSp0Lms7cmV0dXJuIHI9PT10Lngm
Jmk9PT10Lnk/dDpuZXcgd3codC5rLHIsaSl9ZnVuY3Rpb24gbSh0KXtyZXR1cm5bKCt0WzBdWzBd
KyArdFsxXVswXSkvMiwoK3RbMF1bMV0rICt0WzFdWzFdKS8yXX1mdW5jdGlvbiB4KHQsbixlLHIp
e3Qub24oInN0YXJ0Lnpvb20iLChmdW5jdGlvbigpe3codGhpcyxhcmd1bWVudHMpLmV2ZW50KHIp
LnN0YXJ0KCl9KSkub24oImludGVycnVwdC56b29tIGVuZC56b29tIiwoZnVuY3Rpb24oKXt3KHRo
aXMsYXJndW1lbnRzKS5ldmVudChyKS5lbmQoKX0pKS50d2Vlbigiem9vbSIsKGZ1bmN0aW9uKCl7
dmFyIHQ9dGhpcyxvPWFyZ3VtZW50cyxhPXcodCxvKS5ldmVudChyKSx1PWkuYXBwbHkodCxvKSxj
PW51bGw9PWU/bSh1KToiZnVuY3Rpb24iPT10eXBlb2YgZT9lLmFwcGx5KHQsbyk6ZSxmPU1hdGgu
bWF4KHVbMV1bMF0tdVswXVswXSx1WzFdWzFdLXVbMF1bMV0pLHM9dC5fX3pvb20saD0iZnVuY3Rp
b24iPT10eXBlb2Ygbj9uLmFwcGx5KHQsbyk6bixkPWwocy5pbnZlcnQoYykuY29uY2F0KGYvcy5r
KSxoLmludmVydChjKS5jb25jYXQoZi9oLmspKTtyZXR1cm4gZnVuY3Rpb24odCl7aWYoMT09PXQp
dD1oO2Vsc2V7dmFyIG49ZCh0KSxlPWYvblsyXTt0PW5ldyB3dyhlLGNbMF0tblswXSplLGNbMV0t
blsxXSplKX1hLnpvb20obnVsbCx0KX19KSl9ZnVuY3Rpb24gdyh0LG4sZSl7cmV0dXJuIWUmJnQu
X196b29taW5nfHxuZXcgTSh0LG4pfWZ1bmN0aW9uIE0odCxuKXt0aGlzLnRoYXQ9dCx0aGlzLmFy
Z3M9bix0aGlzLmFjdGl2ZT0wLHRoaXMuc291cmNlRXZlbnQ9bnVsbCx0aGlzLmV4dGVudD1pLmFw
cGx5KHQsbiksdGhpcy50YXBzPTB9ZnVuY3Rpb24gVCh0LC4uLm4pe2lmKHIuYXBwbHkodGhpcyxh
cmd1bWVudHMpKXt2YXIgZT13KHRoaXMsbikuZXZlbnQodCksaT10aGlzLl9fem9vbSx1PU1hdGgu
bWF4KGNbMF0sTWF0aC5taW4oY1sxXSxpLmsqTWF0aC5wb3coMixhLmFwcGx5KHRoaXMsYXJndW1l
bnRzKSkpKSxzPWVlKHQpO2lmKGUud2hlZWwpZS5tb3VzZVswXVswXT09PXNbMF0mJmUubW91c2Vb
MF1bMV09PT1zWzFdfHwoZS5tb3VzZVsxXT1pLmludmVydChlLm1vdXNlWzBdPXMpKSxjbGVhclRp
bWVvdXQoZS53aGVlbCk7ZWxzZXtpZihpLms9PT11KXJldHVybjtlLm1vdXNlPVtzLGkuaW52ZXJ0
KHMpXSxWaSh0aGlzKSxlLnN0YXJ0KCl9U3codCksZS53aGVlbD1zZXRUaW1lb3V0KChmdW5jdGlv
bigpe2Uud2hlZWw9bnVsbCxlLmVuZCgpfSkscCksZS56b29tKCJtb3VzZSIsbyhiKF8oaSx1KSxl
Lm1vdXNlWzBdLGUubW91c2VbMV0pLGUuZXh0ZW50LGYpKX19ZnVuY3Rpb24gQSh0LC4uLm4pe2lm
KCFlJiZyLmFwcGx5KHRoaXMsYXJndW1lbnRzKSl7dmFyIGk9dC5jdXJyZW50VGFyZ2V0LGE9dyh0
aGlzLG4sITApLmV2ZW50KHQpLHU9S24odC52aWV3KS5vbigibW91c2Vtb3ZlLnpvb20iLChmdW5j
dGlvbih0KXtpZihTdyh0KSwhYS5tb3ZlZCl7dmFyIG49dC5jbGllbnRYLXMsZT10LmNsaWVudFkt
bDthLm1vdmVkPW4qbitlKmU+Z31hLmV2ZW50KHQpLnpvb20oIm1vdXNlIixvKGIoYS50aGF0Ll9f
em9vbSxhLm1vdXNlWzBdPWVlKHQsaSksYS5tb3VzZVsxXSksYS5leHRlbnQsZikpfSksITApLm9u
KCJtb3VzZXVwLnpvb20iLChmdW5jdGlvbih0KXt1Lm9uKCJtb3VzZW1vdmUuem9vbSBtb3VzZXVw
Lnpvb20iLG51bGwpLGNlKHQudmlldyxhLm1vdmVkKSxTdyh0KSxhLmV2ZW50KHQpLmVuZCgpfSks
ITApLGM9ZWUodCxpKSxzPXQuY2xpZW50WCxsPXQuY2xpZW50WTt1ZSh0LnZpZXcpLEF3KHQpLGEu
bW91c2U9W2MsdGhpcy5fX3pvb20uaW52ZXJ0KGMpXSxWaSh0aGlzKSxhLnN0YXJ0KCl9fWZ1bmN0
aW9uIFModCwuLi5uKXtpZihyLmFwcGx5KHRoaXMsYXJndW1lbnRzKSl7dmFyIGU9dGhpcy5fX3pv
b20sYT1lZSh0LmNoYW5nZWRUb3VjaGVzP3QuY2hhbmdlZFRvdWNoZXNbMF06dCx0aGlzKSx1PWUu
aW52ZXJ0KGEpLGM9ZS5rKih0LnNoaWZ0S2V5Py41OjIpLGw9byhiKF8oZSxjKSxhLHUpLGkuYXBw
bHkodGhpcyxuKSxmKTtTdyh0KSxzPjA/S24odGhpcykudHJhbnNpdGlvbigpLmR1cmF0aW9uKHMp
LmNhbGwoeCxsLGEsdCk6S24odGhpcykuY2FsbCh2LnRyYW5zZm9ybSxsLGEsdCl9fWZ1bmN0aW9u
IEUoZSwuLi5pKXtpZihyLmFwcGx5KHRoaXMsYXJndW1lbnRzKSl7dmFyIG8sYSx1LGMsZj1lLnRv
dWNoZXMscz1mLmxlbmd0aCxsPXcodGhpcyxpLGUuY2hhbmdlZFRvdWNoZXMubGVuZ3RoPT09cyku
ZXZlbnQoZSk7Zm9yKEF3KGUpLGE9MDthPHM7KythKWM9W2M9ZWUodT1mW2FdLHRoaXMpLHRoaXMu
X196b29tLmludmVydChjKSx1LmlkZW50aWZpZXJdLGwudG91Y2gwP2wudG91Y2gxfHxsLnRvdWNo
MFsyXT09PWNbMl18fChsLnRvdWNoMT1jLGwudGFwcz0wKToobC50b3VjaDA9YyxvPSEwLGwudGFw
cz0xKyEhdCk7dCYmKHQ9Y2xlYXJUaW1lb3V0KHQpKSxvJiYobC50YXBzPDImJihuPWNbMF0sdD1z
ZXRUaW1lb3V0KChmdW5jdGlvbigpe3Q9bnVsbH0pLGQpKSxWaSh0aGlzKSxsLnN0YXJ0KCkpfX1m
dW5jdGlvbiBOKHQsLi4ubil7aWYodGhpcy5fX3pvb21pbmcpe3ZhciBlLHIsaSxhLHU9dyh0aGlz
LG4pLmV2ZW50KHQpLGM9dC5jaGFuZ2VkVG91Y2hlcyxzPWMubGVuZ3RoO2ZvcihTdyh0KSxlPTA7
ZTxzOysrZSlpPWVlKHI9Y1tlXSx0aGlzKSx1LnRvdWNoMCYmdS50b3VjaDBbMl09PT1yLmlkZW50
aWZpZXI/dS50b3VjaDBbMF09aTp1LnRvdWNoMSYmdS50b3VjaDFbMl09PT1yLmlkZW50aWZpZXIm
Jih1LnRvdWNoMVswXT1pKTtpZihyPXUudGhhdC5fX3pvb20sdS50b3VjaDEpe3ZhciBsPXUudG91
Y2gwWzBdLGg9dS50b3VjaDBbMV0sZD11LnRvdWNoMVswXSxwPXUudG91Y2gxWzFdLGc9KGc9ZFsw
XS1sWzBdKSpnKyhnPWRbMV0tbFsxXSkqZyx5PSh5PXBbMF0taFswXSkqeSsoeT1wWzFdLWhbMV0p
Knk7cj1fKHIsTWF0aC5zcXJ0KGcveSkpLGk9WyhsWzBdK2RbMF0pLzIsKGxbMV0rZFsxXSkvMl0s
YT1bKGhbMF0rcFswXSkvMiwoaFsxXStwWzFdKS8yXX1lbHNle2lmKCF1LnRvdWNoMClyZXR1cm47
aT11LnRvdWNoMFswXSxhPXUudG91Y2gwWzFdfXUuem9vbSgidG91Y2giLG8oYihyLGksYSksdS5l
eHRlbnQsZikpfX1mdW5jdGlvbiBrKHQsLi4ucil7aWYodGhpcy5fX3pvb21pbmcpe3ZhciBpLG8s
YT13KHRoaXMscikuZXZlbnQodCksdT10LmNoYW5nZWRUb3VjaGVzLGM9dS5sZW5ndGg7Zm9yKEF3
KHQpLGUmJmNsZWFyVGltZW91dChlKSxlPXNldFRpbWVvdXQoKGZ1bmN0aW9uKCl7ZT1udWxsfSks
ZCksaT0wO2k8YzsrK2kpbz11W2ldLGEudG91Y2gwJiZhLnRvdWNoMFsyXT09PW8uaWRlbnRpZmll
cj9kZWxldGUgYS50b3VjaDA6YS50b3VjaDEmJmEudG91Y2gxWzJdPT09by5pZGVudGlmaWVyJiZk
ZWxldGUgYS50b3VjaDE7aWYoYS50b3VjaDEmJiFhLnRvdWNoMCYmKGEudG91Y2gwPWEudG91Y2gx
LGRlbGV0ZSBhLnRvdWNoMSksYS50b3VjaDApYS50b3VjaDBbMV09dGhpcy5fX3pvb20uaW52ZXJ0
KGEudG91Y2gwWzBdKTtlbHNlIGlmKGEuZW5kKCksMj09PWEudGFwcyYmKG89ZWUobyx0aGlzKSxN
YXRoLmh5cG90KG5bMF0tb1swXSxuWzFdLW9bMV0pPHkpKXt2YXIgZj1Lbih0aGlzKS5vbigiZGJs
Y2xpY2suem9vbSIpO2YmJmYuYXBwbHkodGhpcyxhcmd1bWVudHMpfX19cmV0dXJuIHYudHJhbnNm
b3JtPWZ1bmN0aW9uKHQsbixlLHIpe3ZhciBpPXQuc2VsZWN0aW9uP3Quc2VsZWN0aW9uKCk6dDtp
LnByb3BlcnR5KCJfX3pvb20iLGt3KSx0IT09aT94KHQsbixlLHIpOmkuaW50ZXJydXB0KCkuZWFj
aCgoZnVuY3Rpb24oKXt3KHRoaXMsYXJndW1lbnRzKS5ldmVudChyKS5zdGFydCgpLnpvb20obnVs
bCwiZnVuY3Rpb24iPT10eXBlb2Ygbj9uLmFwcGx5KHRoaXMsYXJndW1lbnRzKTpuKS5lbmQoKX0p
KX0sdi5zY2FsZUJ5PWZ1bmN0aW9uKHQsbixlLHIpe3Yuc2NhbGVUbyh0LChmdW5jdGlvbigpe3Jl
dHVybiB0aGlzLl9fem9vbS5rKigiZnVuY3Rpb24iPT10eXBlb2Ygbj9uLmFwcGx5KHRoaXMsYXJn
dW1lbnRzKTpuKX0pLGUscil9LHYuc2NhbGVUbz1mdW5jdGlvbih0LG4sZSxyKXt2LnRyYW5zZm9y
bSh0LChmdW5jdGlvbigpe3ZhciB0PWkuYXBwbHkodGhpcyxhcmd1bWVudHMpLHI9dGhpcy5fX3pv
b20sYT1udWxsPT1lP20odCk6ImZ1bmN0aW9uIj09dHlwZW9mIGU/ZS5hcHBseSh0aGlzLGFyZ3Vt
ZW50cyk6ZSx1PXIuaW52ZXJ0KGEpLGM9ImZ1bmN0aW9uIj09dHlwZW9mIG4/bi5hcHBseSh0aGlz
LGFyZ3VtZW50cyk6bjtyZXR1cm4gbyhiKF8ocixjKSxhLHUpLHQsZil9KSxlLHIpfSx2LnRyYW5z
bGF0ZUJ5PWZ1bmN0aW9uKHQsbixlLHIpe3YudHJhbnNmb3JtKHQsKGZ1bmN0aW9uKCl7cmV0dXJu
IG8odGhpcy5fX3pvb20udHJhbnNsYXRlKCJmdW5jdGlvbiI9PXR5cGVvZiBuP24uYXBwbHkodGhp
cyxhcmd1bWVudHMpOm4sImZ1bmN0aW9uIj09dHlwZW9mIGU/ZS5hcHBseSh0aGlzLGFyZ3VtZW50
cyk6ZSksaS5hcHBseSh0aGlzLGFyZ3VtZW50cyksZil9KSxudWxsLHIpfSx2LnRyYW5zbGF0ZVRv
PWZ1bmN0aW9uKHQsbixlLHIsYSl7di50cmFuc2Zvcm0odCwoZnVuY3Rpb24oKXt2YXIgdD1pLmFw
cGx5KHRoaXMsYXJndW1lbnRzKSxhPXRoaXMuX196b29tLHU9bnVsbD09cj9tKHQpOiJmdW5jdGlv
biI9PXR5cGVvZiByP3IuYXBwbHkodGhpcyxhcmd1bWVudHMpOnI7cmV0dXJuIG8oTXcudHJhbnNs
YXRlKHVbMF0sdVsxXSkuc2NhbGUoYS5rKS50cmFuc2xhdGUoImZ1bmN0aW9uIj09dHlwZW9mIG4/
LW4uYXBwbHkodGhpcyxhcmd1bWVudHMpOi1uLCJmdW5jdGlvbiI9PXR5cGVvZiBlPy1lLmFwcGx5
KHRoaXMsYXJndW1lbnRzKTotZSksdCxmKX0pLHIsYSl9LE0ucHJvdG90eXBlPXtldmVudDpmdW5j
dGlvbih0KXtyZXR1cm4gdCYmKHRoaXMuc291cmNlRXZlbnQ9dCksdGhpc30sc3RhcnQ6ZnVuY3Rp
b24oKXtyZXR1cm4gMT09Kyt0aGlzLmFjdGl2ZSYmKHRoaXMudGhhdC5fX3pvb21pbmc9dGhpcyx0
aGlzLmVtaXQoInN0YXJ0IikpLHRoaXN9LHpvb206ZnVuY3Rpb24odCxuKXtyZXR1cm4gdGhpcy5t
b3VzZSYmIm1vdXNlIiE9PXQmJih0aGlzLm1vdXNlWzFdPW4uaW52ZXJ0KHRoaXMubW91c2VbMF0p
KSx0aGlzLnRvdWNoMCYmInRvdWNoIiE9PXQmJih0aGlzLnRvdWNoMFsxXT1uLmludmVydCh0aGlz
LnRvdWNoMFswXSkpLHRoaXMudG91Y2gxJiYidG91Y2giIT09dCYmKHRoaXMudG91Y2gxWzFdPW4u
aW52ZXJ0KHRoaXMudG91Y2gxWzBdKSksdGhpcy50aGF0Ll9fem9vbT1uLHRoaXMuZW1pdCgiem9v
bSIpLHRoaXN9LGVuZDpmdW5jdGlvbigpe3JldHVybiAwPT0tLXRoaXMuYWN0aXZlJiYoZGVsZXRl
IHRoaXMudGhhdC5fX3pvb21pbmcsdGhpcy5lbWl0KCJlbmQiKSksdGhpc30sZW1pdDpmdW5jdGlv
bih0KXt2YXIgbj1Lbih0aGlzLnRoYXQpLmRhdHVtKCk7aC5jYWxsKHQsdGhpcy50aGF0LG5ldyB4
dyh0LHtzb3VyY2VFdmVudDp0aGlzLnNvdXJjZUV2ZW50LHRhcmdldDp2LHR5cGU6dCx0cmFuc2Zv
cm06dGhpcy50aGF0Ll9fem9vbSxkaXNwYXRjaDpofSksbil9fSx2LndoZWVsRGVsdGE9ZnVuY3Rp
b24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KGE9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDpt
dygrdCksdik6YX0sdi5maWx0ZXI9ZnVuY3Rpb24odCl7cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/
KHI9ImZ1bmN0aW9uIj09dHlwZW9mIHQ/dDptdyghIXQpLHYpOnJ9LHYudG91Y2hhYmxlPWZ1bmN0
aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyh1PSJmdW5jdGlvbiI9PXR5cGVvZiB0P3Q6
bXcoISF0KSx2KTp1fSx2LmV4dGVudD1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0
aD8oaT0iZnVuY3Rpb24iPT10eXBlb2YgdD90Om13KFtbK3RbMF1bMF0sK3RbMF1bMV1dLFsrdFsx
XVswXSwrdFsxXVsxXV1dKSx2KTppfSx2LnNjYWxlRXh0ZW50PWZ1bmN0aW9uKHQpe3JldHVybiBh
cmd1bWVudHMubGVuZ3RoPyhjWzBdPSt0WzBdLGNbMV09K3RbMV0sdik6W2NbMF0sY1sxXV19LHYu
dHJhbnNsYXRlRXh0ZW50PWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhmWzBd
WzBdPSt0WzBdWzBdLGZbMV1bMF09K3RbMV1bMF0sZlswXVsxXT0rdFswXVsxXSxmWzFdWzFdPSt0
WzFdWzFdLHYpOltbZlswXVswXSxmWzBdWzFdXSxbZlsxXVswXSxmWzFdWzFdXV19LHYuY29uc3Ry
YWluPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhvPXQsdik6b30sdi5kdXJh
dGlvbj1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0aD8ocz0rdCx2KTpzfSx2Lmlu
dGVycG9sYXRlPWZ1bmN0aW9uKHQpe3JldHVybiBhcmd1bWVudHMubGVuZ3RoPyhsPXQsdik6bH0s
di5vbj1mdW5jdGlvbigpe3ZhciB0PWgub24uYXBwbHkoaCxhcmd1bWVudHMpO3JldHVybiB0PT09
aD92OnR9LHYuY2xpY2tEaXN0YW5jZT1mdW5jdGlvbih0KXtyZXR1cm4gYXJndW1lbnRzLmxlbmd0
aD8oZz0odD0rdCkqdCx2KTpNYXRoLnNxcnQoZyl9LHYudGFwRGlzdGFuY2U9ZnVuY3Rpb24odCl7
cmV0dXJuIGFyZ3VtZW50cy5sZW5ndGg/KHk9K3Qsdik6eX0sdn0sdC56b29tSWRlbnRpdHk9TXcs
dC56b29tVHJhbnNmb3JtPVR3fSkpOzwvc2NyaXB0PjxzY3JpcHQ+LyoqKiBAcHJlc2VydmUgQ29w
eXJpZ2h0IDIwMTIgVHdpdHRlciwgSW5jLiogQGxpY2Vuc2UgaHR0cDovL3d3dy5hcGFjaGUub3Jn
L2xpY2Vuc2VzL0xJQ0VOU0UtMi4wLnR4dCovdmFyIEhvZ2FuPXt9OyFmdW5jdGlvbih0KXtmdW5j
dGlvbiBuKHQsbixlKXt2YXIgaTtyZXR1cm4gbiYmIm9iamVjdCI9PXR5cGVvZiBuJiYodm9pZCAw
IT09blt0XT9pPW5bdF06ZSYmbi5nZXQmJiJmdW5jdGlvbiI9PXR5cGVvZiBuLmdldCYmKGk9bi5n
ZXQodCkpKSxpfWZ1bmN0aW9uIGUodCxuLGUsaSxyLHMpe2Z1bmN0aW9uIGEoKXt9ZnVuY3Rpb24g
bygpe31hLnByb3RvdHlwZT10LG8ucHJvdG90eXBlPXQuc3Viczt2YXIgdSxjPW5ldyBhO2Muc3Vi
cz1uZXcgbyxjLnN1YnNUZXh0PXt9LGMuYnVmPSIiLGk9aXx8e30sYy5zdGFja1N1YnM9aSxjLnN1
YnNUZXh0PXM7Zm9yKHUgaW4gbilpW3VdfHwoaVt1XT1uW3VdKTtmb3IodSBpbiBpKWMuc3Vic1t1
XT1pW3VdO3I9cnx8e30sYy5zdGFja1BhcnRpYWxzPXI7Zm9yKHUgaW4gZSlyW3VdfHwoclt1XT1l
W3VdKTtmb3IodSBpbiByKWMucGFydGlhbHNbdV09clt1XTtyZXR1cm4gY31mdW5jdGlvbiBpKHQp
e3JldHVybiBTdHJpbmcobnVsbD09PXR8fHZvaWQgMD09PXQ/IiI6dCl9ZnVuY3Rpb24gcih0KXty
ZXR1cm4gdD1pKHQpLGwudGVzdCh0KT90LnJlcGxhY2UocywiJmFtcDsiKS5yZXBsYWNlKGEsIiZs
dDsiKS5yZXBsYWNlKG8sIiZndDsiKS5yZXBsYWNlKHUsIiYjMzk7IikucmVwbGFjZShjLCImcXVv
dDsiKTp0fXQuVGVtcGxhdGU9ZnVuY3Rpb24odCxuLGUsaSl7dD10fHx7fSx0aGlzLnI9dC5jb2Rl
fHx0aGlzLnIsdGhpcy5jPWUsdGhpcy5vcHRpb25zPWl8fHt9LHRoaXMudGV4dD1ufHwiIix0aGlz
LnBhcnRpYWxzPXQucGFydGlhbHN8fHt9LHRoaXMuc3Vicz10LnN1YnN8fHt9LHRoaXMuYnVmPSIi
fSx0LlRlbXBsYXRlLnByb3RvdHlwZT17cjpmdW5jdGlvbigpe3JldHVybiIifSx2OnIsdDppLHJl
bmRlcjpmdW5jdGlvbih0LG4sZSl7cmV0dXJuIHRoaXMucmkoW3RdLG58fHt9LGUpfSxyaTpmdW5j
dGlvbih0LG4sZSl7cmV0dXJuIHRoaXMucih0LG4sZSl9LGVwOmZ1bmN0aW9uKHQsbil7dmFyIGk9
dGhpcy5wYXJ0aWFsc1t0XSxyPW5baS5uYW1lXTtpZihpLmluc3RhbmNlJiZpLmJhc2U9PXIpcmV0
dXJuIGkuaW5zdGFuY2U7aWYoInN0cmluZyI9PXR5cGVvZiByKXtpZighdGhpcy5jKXRocm93IG5l
dyBFcnJvcigiTm8gY29tcGlsZXIgYXZhaWxhYmxlLiIpO3I9dGhpcy5jLmNvbXBpbGUocix0aGlz
Lm9wdGlvbnMpfWlmKCFyKXJldHVybiBudWxsO2lmKHRoaXMucGFydGlhbHNbdF0uYmFzZT1yLGku
c3Vicyl7bi5zdGFja1RleHR8fChuLnN0YWNrVGV4dD17fSk7Zm9yKGtleSBpbiBpLnN1YnMpbi5z
dGFja1RleHRba2V5XXx8KG4uc3RhY2tUZXh0W2tleV09dm9pZCAwIT09dGhpcy5hY3RpdmVTdWIm
Jm4uc3RhY2tUZXh0W3RoaXMuYWN0aXZlU3ViXT9uLnN0YWNrVGV4dFt0aGlzLmFjdGl2ZVN1Yl06
dGhpcy50ZXh0KTtyPWUocixpLnN1YnMsaS5wYXJ0aWFscyx0aGlzLnN0YWNrU3Vicyx0aGlzLnN0
YWNrUGFydGlhbHMsbi5zdGFja1RleHQpfXJldHVybiB0aGlzLnBhcnRpYWxzW3RdLmluc3RhbmNl
PXIscn0scnA6ZnVuY3Rpb24odCxuLGUsaSl7dmFyIHI9dGhpcy5lcCh0LGUpO3JldHVybiByP3Iu
cmkobixlLGkpOiIifSxyczpmdW5jdGlvbih0LG4sZSl7dmFyIGk9dFt0Lmxlbmd0aC0xXTtpZigh
ZihpKSlyZXR1cm4gdm9pZCBlKHQsbix0aGlzKTtmb3IodmFyIHI9MDtyPGkubGVuZ3RoO3IrKyl0
LnB1c2goaVtyXSksZSh0LG4sdGhpcyksdC5wb3AoKX0sczpmdW5jdGlvbih0LG4sZSxpLHIscyxh
KXt2YXIgbztyZXR1cm4gZih0KSYmMD09PXQubGVuZ3RoPyExOigiZnVuY3Rpb24iPT10eXBlb2Yg
dCYmKHQ9dGhpcy5tcyh0LG4sZSxpLHIscyxhKSksbz0hIXQsIWkmJm8mJm4mJm4ucHVzaCgib2Jq
ZWN0Ij09dHlwZW9mIHQ/dDpuW24ubGVuZ3RoLTFdKSxvKX0sZDpmdW5jdGlvbih0LGUsaSxyKXt2
YXIgcyxhPXQuc3BsaXQoIi4iKSxvPXRoaXMuZihhWzBdLGUsaSxyKSx1PXRoaXMub3B0aW9ucy5t
b2RlbEdldCxjPW51bGw7aWYoIi4iPT09dCYmZihlW2UubGVuZ3RoLTJdKSlvPWVbZS5sZW5ndGgt
MV07ZWxzZSBmb3IodmFyIGw9MTtsPGEubGVuZ3RoO2wrKylzPW4oYVtsXSxvLHUpLHZvaWQgMCE9
PXM/KGM9byxvPXMpOm89IiI7cmV0dXJuIHImJiFvPyExOihyfHwiZnVuY3Rpb24iIT10eXBlb2Yg
b3x8KGUucHVzaChjKSxvPXRoaXMubXYobyxlLGkpLGUucG9wKCkpLG8pfSxmOmZ1bmN0aW9uKHQs
ZSxpLHIpe2Zvcih2YXIgcz0hMSxhPW51bGwsbz0hMSx1PXRoaXMub3B0aW9ucy5tb2RlbEdldCxj
PWUubGVuZ3RoLTE7Yz49MDtjLS0paWYoYT1lW2NdLHM9bih0LGEsdSksdm9pZCAwIT09cyl7bz0h
MDticmVha31yZXR1cm4gbz8ocnx8ImZ1bmN0aW9uIiE9dHlwZW9mIHN8fChzPXRoaXMubXYocyxl
LGkpKSxzKTpyPyExOiIifSxsczpmdW5jdGlvbih0LG4sZSxyLHMpe3ZhciBhPXRoaXMub3B0aW9u
cy5kZWxpbWl0ZXJzO3JldHVybiB0aGlzLm9wdGlvbnMuZGVsaW1pdGVycz1zLHRoaXMuYih0aGlz
LmN0KGkodC5jYWxsKG4scikpLG4sZSkpLHRoaXMub3B0aW9ucy5kZWxpbWl0ZXJzPWEsITF9LGN0
OmZ1bmN0aW9uKHQsbixlKXtpZih0aGlzLm9wdGlvbnMuZGlzYWJsZUxhbWJkYSl0aHJvdyBuZXcg
RXJyb3IoIkxhbWJkYSBmZWF0dXJlcyBkaXNhYmxlZC4iKTtyZXR1cm4gdGhpcy5jLmNvbXBpbGUo
dCx0aGlzLm9wdGlvbnMpLnJlbmRlcihuLGUpfSxiOmZ1bmN0aW9uKHQpe3RoaXMuYnVmKz10fSxm
bDpmdW5jdGlvbigpe3ZhciB0PXRoaXMuYnVmO3JldHVybiB0aGlzLmJ1Zj0iIix0fSxtczpmdW5j
dGlvbih0LG4sZSxpLHIscyxhKXt2YXIgbyx1PW5bbi5sZW5ndGgtMV0sYz10LmNhbGwodSk7cmV0
dXJuImZ1bmN0aW9uIj09dHlwZW9mIGM/aT8hMDoobz10aGlzLmFjdGl2ZVN1YiYmdGhpcy5zdWJz
VGV4dCYmdGhpcy5zdWJzVGV4dFt0aGlzLmFjdGl2ZVN1Yl0/dGhpcy5zdWJzVGV4dFt0aGlzLmFj
dGl2ZVN1Yl06dGhpcy50ZXh0LHRoaXMubHMoYyx1LGUsby5zdWJzdHJpbmcocixzKSxhKSk6Y30s
bXY6ZnVuY3Rpb24odCxuLGUpe3ZhciByPW5bbi5sZW5ndGgtMV0scz10LmNhbGwocik7cmV0dXJu
ImZ1bmN0aW9uIj09dHlwZW9mIHM/dGhpcy5jdChpKHMuY2FsbChyKSkscixlKTpzfSxzdWI6ZnVu
Y3Rpb24odCxuLGUsaSl7dmFyIHI9dGhpcy5zdWJzW3RdO3ImJih0aGlzLmFjdGl2ZVN1Yj10LHIo
bixlLHRoaXMsaSksdGhpcy5hY3RpdmVTdWI9ITEpfX07dmFyIHM9LyYvZyxhPS88L2csbz0vPi9n
LHU9L1wnL2csYz0vXCIvZyxsPS9bJjw+XCJcJ10vLGY9QXJyYXkuaXNBcnJheXx8ZnVuY3Rpb24o
dCl7cmV0dXJuIltvYmplY3QgQXJyYXldIj09PU9iamVjdC5wcm90b3R5cGUudG9TdHJpbmcuY2Fs
bCh0KX19KCJ1bmRlZmluZWQiIT10eXBlb2YgZXhwb3J0cz9leHBvcnRzOkhvZ2FuKSxmdW5jdGlv
bih0KXtmdW5jdGlvbiBuKHQpeyJ9Ij09PXQubi5zdWJzdHIodC5uLmxlbmd0aC0xKSYmKHQubj10
Lm4uc3Vic3RyaW5nKDAsdC5uLmxlbmd0aC0xKSl9ZnVuY3Rpb24gZSh0KXtyZXR1cm4gdC50cmlt
P3QudHJpbSgpOnQucmVwbGFjZSgvXlxzKnxccyokL2csIiIpfWZ1bmN0aW9uIGkodCxuLGUpe2lm
KG4uY2hhckF0KGUpIT10LmNoYXJBdCgwKSlyZXR1cm4hMTtmb3IodmFyIGk9MSxyPXQubGVuZ3Ro
O3I+aTtpKyspaWYobi5jaGFyQXQoZStpKSE9dC5jaGFyQXQoaSkpcmV0dXJuITE7cmV0dXJuITB9
ZnVuY3Rpb24gcihuLGUsaSxvKXt2YXIgdT1bXSxjPW51bGwsbD1udWxsLGY9bnVsbDtmb3IobD1p
W2kubGVuZ3RoLTFdO24ubGVuZ3RoPjA7KXtpZihmPW4uc2hpZnQoKSxsJiYiPCI9PWwudGFnJiYh
KGYudGFnIGluIGspKXRocm93IG5ldyBFcnJvcigiSWxsZWdhbCBjb250ZW50IGluIDwgc3VwZXIg
dGFnLiIpO2lmKHQudGFnc1tmLnRhZ108PXQudGFncy4kfHxzKGYsbykpaS5wdXNoKGYpLGYubm9k
ZXM9cihuLGYudGFnLGksbyk7ZWxzZXtpZigiLyI9PWYudGFnKXtpZigwPT09aS5sZW5ndGgpdGhy
b3cgbmV3IEVycm9yKCJDbG9zaW5nIHRhZyB3aXRob3V0IG9wZW5lcjogLyIrZi5uKTtpZihjPWku
cG9wKCksZi5uIT1jLm4mJiFhKGYubixjLm4sbykpdGhyb3cgbmV3IEVycm9yKCJOZXN0aW5nIGVy
cm9yOiAiK2MubisiIHZzLiAiK2Yubik7cmV0dXJuIGMuZW5kPWYuaSx1fSJcbiI9PWYudGFnJiYo
Zi5sYXN0PTA9PW4ubGVuZ3RofHwiXG4iPT1uWzBdLnRhZyl9dS5wdXNoKGYpfWlmKGkubGVuZ3Ro
PjApdGhyb3cgbmV3IEVycm9yKCJtaXNzaW5nIGNsb3NpbmcgdGFnOiAiK2kucG9wKCkubik7cmV0
dXJuIHV9ZnVuY3Rpb24gcyh0LG4pe2Zvcih2YXIgZT0wLGk9bi5sZW5ndGg7aT5lO2UrKylpZihu
W2VdLm89PXQubilyZXR1cm4gdC50YWc9IiMiLCEwfWZ1bmN0aW9uIGEodCxuLGUpe2Zvcih2YXIg
aT0wLHI9ZS5sZW5ndGg7cj5pO2krKylpZihlW2ldLmM9PXQmJmVbaV0ubz09bilyZXR1cm4hMH1m
dW5jdGlvbiBvKHQpe3ZhciBuPVtdO2Zvcih2YXIgZSBpbiB0KW4ucHVzaCgnIicrYyhlKSsnIjog
ZnVuY3Rpb24oYyxwLHQsaSkgeycrdFtlXSsifSIpO3JldHVybiJ7ICIrbi5qb2luKCIsIikrIiB9
In1mdW5jdGlvbiB1KHQpe3ZhciBuPVtdO2Zvcih2YXIgZSBpbiB0LnBhcnRpYWxzKW4ucHVzaCgn
IicrYyhlKSsnIjp7bmFtZToiJytjKHQucGFydGlhbHNbZV0ubmFtZSkrJyIsICcrdSh0LnBhcnRp
YWxzW2VdKSsifSIpO3JldHVybiJwYXJ0aWFsczogeyIrbi5qb2luKCIsIikrIn0sIHN1YnM6ICIr
byh0LnN1YnMpfWZ1bmN0aW9uIGModCl7cmV0dXJuIHQucmVwbGFjZShtLCJcXFxcIikucmVwbGFj
ZSh2LCdcXCInKS5yZXBsYWNlKGIsIlxcbiIpLnJlcGxhY2UoZCwiXFxyIikucmVwbGFjZSh4LCJc
XHUyMDI4IikucmVwbGFjZSh3LCJcXHUyMDI5Iil9ZnVuY3Rpb24gbCh0KXtyZXR1cm5+dC5pbmRl
eE9mKCIuIik/ImQiOiJmIn1mdW5jdGlvbiBmKHQsbil7dmFyIGU9IjwiKyhuLnByZWZpeHx8IiIp
LGk9ZSt0Lm4reSsrO3JldHVybiBuLnBhcnRpYWxzW2ldPXtuYW1lOnQubixwYXJ0aWFsczp7fX0s
bi5jb2RlKz0ndC5iKHQucnAoIicrYyhpKSsnIixjLHAsIicrKHQuaW5kZW50fHwiIikrJyIpKTsn
LGl9ZnVuY3Rpb24gaCh0LG4pe24uY29kZSs9InQuYih0LnQodC4iK2wodC5uKSsnKCInK2ModC5u
KSsnIixjLHAsMCkpKTsnfWZ1bmN0aW9uIHAodCl7cmV0dXJuInQuYigiK3QrIik7In12YXIgZz0v
XFMvLHY9L1wiL2csYj0vXG4vZyxkPS9cci9nLG09L1xcL2cseD0vXHUyMDI4Lyx3PS9cdTIwMjkv
O3QudGFncz17IiMiOjEsIl4iOjIsIjwiOjMsJDo0LCIvIjo1LCIhIjo2LCI+Ijo3LCI9Ijo4LF92
OjksInsiOjEwLCImIjoxMSxfdDoxMn0sdC5zY2FuPWZ1bmN0aW9uKHIscyl7ZnVuY3Rpb24gYSgp
e20ubGVuZ3RoPjAmJih4LnB1c2goe3RhZzoiX3QiLHRleHQ6bmV3IFN0cmluZyhtKX0pLG09IiIp
fWZ1bmN0aW9uIG8oKXtmb3IodmFyIG49ITAsZT15O2U8eC5sZW5ndGg7ZSsrKWlmKG49dC50YWdz
W3hbZV0udGFnXTx0LnRhZ3MuX3Z8fCJfdCI9PXhbZV0udGFnJiZudWxsPT09eFtlXS50ZXh0Lm1h
dGNoKGcpLCFuKXJldHVybiExO3JldHVybiBufWZ1bmN0aW9uIHUodCxuKXtpZihhKCksdCYmbygp
KWZvcih2YXIgZSxpPXk7aTx4Lmxlbmd0aDtpKyspeFtpXS50ZXh0JiYoKGU9eFtpKzFdKSYmIj4i
PT1lLnRhZyYmKGUuaW5kZW50PXhbaV0udGV4dC50b1N0cmluZygpKSx4LnNwbGljZShpLDEpKTtl
bHNlIG58fHgucHVzaCh7dGFnOiJcbiJ9KTt3PSExLHk9eC5sZW5ndGh9ZnVuY3Rpb24gYyh0LG4p
e3ZhciBpPSI9IitTLHI9dC5pbmRleE9mKGksbikscz1lKHQuc3Vic3RyaW5nKHQuaW5kZXhPZigi
PSIsbikrMSxyKSkuc3BsaXQoIiAiKTtyZXR1cm4gVD1zWzBdLFM9c1tzLmxlbmd0aC0xXSxyK2ku
bGVuZ3RoLTF9dmFyIGw9ci5sZW5ndGgsZj0wLGg9MSxwPTIsdj1mLGI9bnVsbCxkPW51bGwsbT0i
Iix4PVtdLHc9ITEsaz0wLHk9MCxUPSJ7eyIsUz0ifX0iO2ZvcihzJiYocz1zLnNwbGl0KCIgIiks
VD1zWzBdLFM9c1sxXSksaz0wO2w+aztrKyspdj09Zj9pKFQscixrKT8oLS1rLGEoKSx2PWgpOiJc
biI9PXIuY2hhckF0KGspP3Uodyk6bSs9ci5jaGFyQXQoayk6dj09aD8oays9VC5sZW5ndGgtMSxk
PXQudGFnc1tyLmNoYXJBdChrKzEpXSxiPWQ/ci5jaGFyQXQoaysxKToiX3YiLCI9Ij09Yj8oaz1j
KHIsayksdj1mKTooZCYmaysrLHY9cCksdz1rKTppKFMscixrKT8oeC5wdXNoKHt0YWc6YixuOmUo
bSksb3RhZzpULGN0YWc6UyxpOiIvIj09Yj93LVQubGVuZ3RoOmsrUy5sZW5ndGh9KSxtPSIiLGsr
PVMubGVuZ3RoLTEsdj1mLCJ7Ij09YiYmKCJ9fSI9PVM/aysrOm4oeFt4Lmxlbmd0aC0xXSkpKTpt
Kz1yLmNoYXJBdChrKTtyZXR1cm4gdSh3LCEwKSx4fTt2YXIgaz17X3Q6ITAsIlxuIjohMCwkOiEw
LCIvIjohMH07dC5zdHJpbmdpZnk9ZnVuY3Rpb24obil7cmV0dXJuIntjb2RlOiBmdW5jdGlvbiAo
YyxwLGkpIHsgIit0LndyYXBNYWluKG4uY29kZSkrIiB9LCIrdShuKSsifSJ9O3ZhciB5PTA7dC5n
ZW5lcmF0ZT1mdW5jdGlvbihuLGUsaSl7eT0wO3ZhciByPXtjb2RlOiIiLHN1YnM6e30scGFydGlh
bHM6e319O3JldHVybiB0LndhbGsobixyKSxpLmFzU3RyaW5nP3RoaXMuc3RyaW5naWZ5KHIsZSxp
KTp0aGlzLm1ha2VUZW1wbGF0ZShyLGUsaSl9LHQud3JhcE1haW49ZnVuY3Rpb24odCl7cmV0dXJu
J3ZhciB0PXRoaXM7dC5iKGk9aXx8IiIpOycrdCsicmV0dXJuIHQuZmwoKTsifSx0LnRlbXBsYXRl
PXQuVGVtcGxhdGUsdC5tYWtlVGVtcGxhdGU9ZnVuY3Rpb24odCxuLGUpe3ZhciBpPXRoaXMubWFr
ZVBhcnRpYWxzKHQpO3JldHVybiBpLmNvZGU9bmV3IEZ1bmN0aW9uKCJjIiwicCIsImkiLHRoaXMu
d3JhcE1haW4odC5jb2RlKSksbmV3IHRoaXMudGVtcGxhdGUoaSxuLHRoaXMsZSl9LHQubWFrZVBh
cnRpYWxzPWZ1bmN0aW9uKHQpe3ZhciBuLGU9e3N1YnM6e30scGFydGlhbHM6dC5wYXJ0aWFscyxu
YW1lOnQubmFtZX07Zm9yKG4gaW4gZS5wYXJ0aWFscyllLnBhcnRpYWxzW25dPXRoaXMubWFrZVBh
cnRpYWxzKGUucGFydGlhbHNbbl0pO2ZvcihuIGluIHQuc3VicyllLnN1YnNbbl09bmV3IEZ1bmN0
aW9uKCJjIiwicCIsInQiLCJpIix0LnN1YnNbbl0pO3JldHVybiBlfSx0LmNvZGVnZW49eyIjIjpm
dW5jdGlvbihuLGUpe2UuY29kZSs9ImlmKHQucyh0LiIrbChuLm4pKycoIicrYyhuLm4pKyciLGMs
cCwxKSxjLHAsMCwnK24uaSsiLCIrbi5lbmQrJywiJytuLm90YWcrIiAiK24uY3RhZysnIikpe3Qu
cnMoYyxwLGZ1bmN0aW9uKGMscCx0KXsnLHQud2FsayhuLm5vZGVzLGUpLGUuY29kZSs9In0pO2Mu
cG9wKCk7fSJ9LCJeIjpmdW5jdGlvbihuLGUpe2UuY29kZSs9ImlmKCF0LnModC4iK2wobi5uKSsn
KCInK2Mobi5uKSsnIixjLHAsMSksYyxwLDEsMCwwLCIiKSl7Jyx0LndhbGsobi5ub2RlcyxlKSxl
LmNvZGUrPSJ9OyJ9LCI+IjpmLCI8IjpmdW5jdGlvbihuLGUpe3ZhciBpPXtwYXJ0aWFsczp7fSxj
b2RlOiIiLHN1YnM6e30saW5QYXJ0aWFsOiEwfTt0LndhbGsobi5ub2RlcyxpKTt2YXIgcj1lLnBh
cnRpYWxzW2YobixlKV07ci5zdWJzPWkuc3VicyxyLnBhcnRpYWxzPWkucGFydGlhbHN9LCQ6ZnVu
Y3Rpb24obixlKXt2YXIgaT17c3Viczp7fSxjb2RlOiIiLHBhcnRpYWxzOmUucGFydGlhbHMscHJl
Zml4Om4ubn07dC53YWxrKG4ubm9kZXMsaSksZS5zdWJzW24ubl09aS5jb2RlLGUuaW5QYXJ0aWFs
fHwoZS5jb2RlKz0ndC5zdWIoIicrYyhuLm4pKyciLGMscCxpKTsnKX0sIlxuIjpmdW5jdGlvbih0
LG4pe24uY29kZSs9cCgnIlxcbiInKyh0Lmxhc3Q/IiI6IiArIGkiKSl9LF92OmZ1bmN0aW9uKHQs
bil7bi5jb2RlKz0idC5iKHQudih0LiIrbCh0Lm4pKycoIicrYyh0Lm4pKyciLGMscCwwKSkpOyd9
LF90OmZ1bmN0aW9uKHQsbil7bi5jb2RlKz1wKCciJytjKHQudGV4dCkrJyInKX0sInsiOmgsIiYi
Omh9LHQud2Fsaz1mdW5jdGlvbihuLGUpe2Zvcih2YXIgaSxyPTAscz1uLmxlbmd0aDtzPnI7cisr
KWk9dC5jb2RlZ2VuW25bcl0udGFnXSxpJiZpKG5bcl0sZSk7cmV0dXJuIGV9LHQucGFyc2U9ZnVu
Y3Rpb24odCxuLGUpe3JldHVybiBlPWV8fHt9LHIodCwiIixbXSxlLnNlY3Rpb25UYWdzfHxbXSl9
LHQuY2FjaGU9e30sdC5jYWNoZUtleT1mdW5jdGlvbih0LG4pe3JldHVyblt0LCEhbi5hc1N0cmlu
ZywhIW4uZGlzYWJsZUxhbWJkYSxuLmRlbGltaXRlcnMsISFuLm1vZGVsR2V0XS5qb2luKCJ8fCIp
fSx0LmNvbXBpbGU9ZnVuY3Rpb24obixlKXtlPWV8fHt9O3ZhciBpPXQuY2FjaGVLZXkobixlKSxy
PXRoaXMuY2FjaGVbaV07aWYocil7dmFyIHM9ci5wYXJ0aWFscztmb3IodmFyIGEgaW4gcylkZWxl
dGUgc1thXS5pbnN0YW5jZTtyZXR1cm4gcn1yZXR1cm4gcj10aGlzLmdlbmVyYXRlKHRoaXMucGFy
c2UodGhpcy5zY2FuKG4sZS5kZWxpbWl0ZXJzKSxuLGUpLG4sZSksdGhpcy5jYWNoZVtpXT1yfX0o
InVuZGVmaW5lZCIhPXR5cGVvZiBleHBvcnRzP2V4cG9ydHM6SG9nYW4pOzwvc2NyaXB0PjxzY3Jp
cHQ+Lypqc2hpbnQgc3ViOnRydWUqLyhmdW5jdGlvbiAoKSB7J3VzZSBzdHJpY3QnO2Z1bmN0aW9u
ICQoc2VsZWN0b3IpIHtyZXR1cm4gZG9jdW1lbnQucXVlcnlTZWxlY3RvcihzZWxlY3Rvcik7fWZ1
bmN0aW9uICQkKHNlbGVjdG9yLCBjYWxsYmFjaykge3ZhciBlbGVtcyA9IGRvY3VtZW50LnF1ZXJ5
U2VsZWN0b3JBbGwoc2VsZWN0b3IpO2ZvciAodmFyIGkgPSAwOyBpIDwgZWxlbXMubGVuZ3RoOyAr
K2kpIHtpZiAoY2FsbGJhY2sgJiYgdHlwZW9mIGNhbGxiYWNrID09ICdmdW5jdGlvbicpY2FsbGJh
Y2suY2FsbCh0aGlzLCBlbGVtc1tpXSk7fX12YXIgZGVib3VuY2UgPSBmdW5jdGlvbiAoZnVuYywg
d2FpdCwgbm93KSB7dmFyIHRpbWVvdXQ7cmV0dXJuIGZ1bmN0aW9uIGRlYm91bmNlZCAoKSB7dmFy
IHRoYXQgPSB0aGlzLCBhcmdzID0gYXJndW1lbnRzO2Z1bmN0aW9uIGRlbGF5ZWQoKSB7aWYgKCFu
b3cpZnVuYy5hcHBseSh0aGF0LCBhcmdzKTt0aW1lb3V0ID0gbnVsbDt9aWYgKHRpbWVvdXQpIHtj
bGVhclRpbWVvdXQodGltZW91dCk7fSBlbHNlIGlmIChub3cpIHtmdW5jLmFwcGx5KG9iaiwgYXJn
cyk7fXRpbWVvdXQgPSBzZXRUaW1lb3V0KGRlbGF5ZWQsIHdhaXQgfHwgMjUwKTt9O307d2luZG93
LkdvQWNjZXNzID0gd2luZG93LkdvQWNjZXNzIHx8IHtpbml0aWFsaXplOiBmdW5jdGlvbiAob3B0
aW9ucykge3RoaXMub3B0cyA9IG9wdGlvbnM7dmFyIGN3ID0gTWF0aC5tYXgoZG9jdW1lbnQuZG9j
dW1lbnRFbGVtZW50LmNsaWVudFdpZHRoIHx8IDAsIHdpbmRvdy5pbm5lcldpZHRoIHx8IDApO3Ro
aXMuQXBwU3RhdGUgID0ge307dGhpcy5BcHBUcGxzICAgPSB7fTt0aGlzLkFwcENoYXJ0cyA9IHt9
O3RoaXMuQXBwVUlEYXRhID0gKHRoaXMub3B0cyB8fCB7fSkudWlEYXRhIHx8IHt9O3RoaXMuQXBw
RGF0YSAgID0gKHRoaXMub3B0cyB8fCB7fSkucGFuZWxEYXRhIHx8IHt9O3RoaXMuQXBwV1NDb25u
ID0gKHRoaXMub3B0cyB8fCB7fSkud3NDb25uZWN0aW9uIHx8IHt9O3RoaXMuaTE4biA9ICh0aGlz
Lm9wdHMgfHwge30pLmkxOG4gfHwge307dGhpcy5BcHBQcmVmcyAgPSB7J2F1dG9IaWRlVGFibGVz
JzogdHJ1ZSwnbGF5b3V0JzogY3cgPiAyNTYwID8gJ3dpZGUnIDogJ2hvcml6b250YWwnLCdwZXJQ
YWdlJzogNywndGhlbWUnOiAod2luZG93Lm1hdGNoTWVkaWEgJiYgd2luZG93Lm1hdGNoTWVkaWEo
JyhwcmVmZXJzLWNvbG9yLXNjaGVtZTogZGFyayknKS5tYXRjaGVzKSA/ICdkYXJrUHVycGxlJyA6
ICdicmlnaHQnLCdoaWRkZW5QYW5lbHMnOiBbXSx9O3RoaXMuQXBwUHJlZnMgPSBHb0FjY2Vzcy5V
dGlsLm1lcmdlKHRoaXMuQXBwUHJlZnMsIHRoaXMub3B0cy5wcmVmcyk7dGhpcy53c0RlbGF5ICAg
ID0gdGhpcy5jdXJyRGVsYXkgPSAxRTM7dGhpcy5tYXhEZWxheSAgID0gMjBFMzt0aGlzLnJldHJp
ZXMgICAgPSAwO3RoaXMubWF4UmV0cmllcyA9IDIwO2lmIChHb0FjY2Vzcy5VdGlsLmhhc0xvY2Fs
U3RvcmFnZSgpKSB7dmFyIGxzID0gSlNPTi5wYXJzZShsb2NhbFN0b3JhZ2UuZ2V0SXRlbSgnQXBw
UHJlZnMnKSk7dGhpcy5BcHBQcmVmcyA9IEdvQWNjZXNzLlV0aWwubWVyZ2UodGhpcy5BcHBQcmVm
cywgbHMpO31pZiAoT2JqZWN0LmtleXModGhpcy5BcHBXU0Nvbm4pLmxlbmd0aCl0aGlzLnNldFdl
YlNvY2tldCh0aGlzLkFwcFdTQ29ubik7fSxnZXRQYW5lbFVJOiBmdW5jdGlvbiAocGFuZWwpIHty
ZXR1cm4gcGFuZWwgPyB0aGlzLkFwcFVJRGF0YVtwYW5lbF0gOiB0aGlzLkFwcFVJRGF0YTt9LGdl
dFByZWZzOiBmdW5jdGlvbiAocGFuZWwpIHtyZXR1cm4gcGFuZWwgPyB0aGlzLkFwcFByZWZzW3Bh
bmVsXSA6IHRoaXMuQXBwUHJlZnM7fSxzZXRQcmVmczogZnVuY3Rpb24gKCkge2lmIChHb0FjY2Vz
cy5VdGlsLmhhc0xvY2FsU3RvcmFnZSgpKSB7bG9jYWxTdG9yYWdlLnNldEl0ZW0oJ0FwcFByZWZz
JywgSlNPTi5zdHJpbmdpZnkoR29BY2Nlc3MuZ2V0UHJlZnMoKSkpO319LGdldFBhbmVsRGF0YTog
ZnVuY3Rpb24gKHBhbmVsKSB7cmV0dXJuIHBhbmVsID8gdGhpcy5BcHBEYXRhW3BhbmVsXSA6IHRo
aXMuQXBwRGF0YTt9LHJlY29ubmVjdDogZnVuY3Rpb24gKHdzQ29ubikge2lmICh0aGlzLnJldHJp
ZXMgPj0gdGhpcy5tYXhSZXRyaWVzKXJldHVybiB3aW5kb3cuY2xlYXJUaW1lb3V0KHRoaXMud3NU
aW1lcik7dGhpcy5yZXRyaWVzKys7aWYgKHRoaXMuY3VyckRlbGF5IDwgdGhpcy5tYXhEZWxheSl0
aGlzLmN1cnJEZWxheSAqPSAyO3RoaXMuc2V0V2ViU29ja2V0KHdzQ29ubik7fSxidWlsZFdTVVJJ
OiBmdW5jdGlvbiAod3NDb25uKSB7dmFyIHVybCA9IG51bGw7aWYgKCF3c0Nvbm4udXJsIHx8ICF3
c0Nvbm4ucG9ydClyZXR1cm4gbnVsbDt1cmwgPSAvXndzcz86XC9cLy9pLnRlc3Qod3NDb25uLnVy
bCkgPyB3c0Nvbm4udXJsIDogd2luZG93LmxvY2F0aW9uLnByb3RvY29sID09PSAiaHR0cHM6IiA/
ICd3c3M6Ly8nICsgd3NDb25uLnVybCA6ICd3czovLycgKyB3c0Nvbm4udXJsO3JldHVybiBuZXcg
VVJMKHVybCkucHJvdG9jb2wgKyAnLy8nICsgbmV3IFVSTCh1cmwpLmhvc3RuYW1lICsgJzonICsg
d3NDb25uLnBvcnQgKyBuZXcgVVJMKHVybCkucGF0aG5hbWU7fSxzZXRXZWJTb2NrZXQ6IGZ1bmN0
aW9uICh3c0Nvbm4pIHt2YXIgaG9zdCA9IG51bGwsIHBpbmdJZCA9IG51bGwsIHVyaSA9IG51bGws
IGRlZlVSSSA9IG51bGwsIHN0ciA9IG51bGw7ZGVmVVJJID0gd2luZG93LmxvY2F0aW9uLmhvc3Ru
YW1lID8gd2luZG93LmxvY2F0aW9uLmhvc3RuYW1lICsgJzonICsgd3NDb25uLnBvcnQgOiAibG9j
YWxob3N0IiArICc6JyArIHdzQ29ubi5wb3J0O3VyaSA9IHdzQ29ubi51cmwgJiYgL14od3NzPzpc
L1wvKT9bXlwvXSs6WzAtOV17MSw1fS8udGVzdCh3c0Nvbm4udXJsKSA/IHdzQ29ubi51cmwgOiB0
aGlzLmJ1aWxkV1NVUkkod3NDb25uKTtzdHIgPSB1cmkgfHwgZGVmVVJJO3N0ciA9ICEvXndzcz86
XC9cLy9pLnRlc3Qoc3RyKSA/ICh3aW5kb3cubG9jYXRpb24ucHJvdG9jb2wgPT09ICJodHRwczoi
ID8gJ3dzczovLycgOiAnd3M6Ly8nKSArIHN0ciA6IHN0cjt2YXIgc29ja2V0ID0gbmV3IFdlYlNv
Y2tldChzdHIpO3NvY2tldC5vbm9wZW4gPSBmdW5jdGlvbiAoZXZlbnQpIHt0aGlzLmN1cnJEZWxh
eSA9IHRoaXMud3NEZWxheTt0aGlzLnJldHJpZXMgPSAwO2lmICh3c0Nvbm4ucGluZ19pbnRlcnZh
bClwaW5nSWQgPSBzZXRJbnRlcnZhbCgoKSA9PiB7IHNvY2tldC5zZW5kKCdwaW5nJyk7IH0sIHdz
Q29ubi5waW5nX2ludGVydmFsICogMUUzKTtHb0FjY2Vzcy5OYXYuV1NPcGVuKHN0cik7fS5iaW5k
KHRoaXMpO3NvY2tldC5vbm1lc3NhZ2UgPSBmdW5jdGlvbiAoZXZlbnQpIHt0aGlzLkFwcFN0YXRl
Wyd1cGRhdGVkJ10gPSB0cnVlO3RoaXMuQXBwRGF0YSA9IEpTT04ucGFyc2UoZXZlbnQuZGF0YSk7
dGhpcy5BcHAucmVuZGVyRGF0YSgpO30uYmluZCh0aGlzKTtzb2NrZXQub25jbG9zZSA9IGZ1bmN0
aW9uIChldmVudCkge0dvQWNjZXNzLk5hdi5XU0Nsb3NlKCk7d2luZG93LmNsZWFySW50ZXJ2YWwo
cGluZ0lkKTtzb2NrZXQgPSBudWxsO3RoaXMud3NUaW1lciA9IHNldFRpbWVvdXQoKCkgPT4geyB0
aGlzLnJlY29ubmVjdCh3c0Nvbm4pOyB9LCB0aGlzLmN1cnJEZWxheSk7fS5iaW5kKHRoaXMpO30s
fTtHb0FjY2Vzcy5VdGlsID0ge21vbnRoczogWyJKYW4iLCAiRmViIiwgIk1hciIsICJBcHIiLCAi
TWF5IiwgIkp1biIsICJKdWwiLCJBdWciLCAiU2VwIiwgIk9jdCIsICJOb3YiLCAiRGVjIl0sbWVy
Z2U6IGZ1bmN0aW9uIChvLCBuKSB7dmFyIG9iaiA9IHt9LCBpID0gMCwgaWwgPSBhcmd1bWVudHMu
bGVuZ3RoLCBrZXk7Zm9yICg7IGkgPCBpbDsgaSsrKSB7Zm9yIChrZXkgaW4gYXJndW1lbnRzW2ld
KSB7aWYgKGFyZ3VtZW50c1tpXS5oYXNPd25Qcm9wZXJ0eShrZXkpKSB7b2JqW2tleV0gPSBhcmd1
bWVudHNbaV1ba2V5XTt9fX1yZXR1cm4gb2JqO30saGFzaENvZGU6IGZ1bmN0aW9uIChzKSB7cmV0
dXJuIChzLnNwbGl0KCcnKS5yZWR1Y2UoZnVuY3Rpb24gKGEsIGIpIHthID0gKChhIDw8IDUpIC0g
YSkgKyBiLmNoYXJDb2RlQXQoMCk7cmV0dXJuIGEmYTt9LCAwKSA+Pj4gMCkudG9TdHJpbmcoMTYp
O30sZm9ybWF0Qnl0ZXM6IGZ1bmN0aW9uIChieXRlcywgZGVjaW1hbHMsIG51bU9ubHkpIHtpZiAo
Ynl0ZXMgPT0gMClyZXR1cm4gbnVtT25seSA/IDAgOiAnMCBCeXRlJzt2YXIgayA9IDEwMjQ7dmFy
IGRtID0gZGVjaW1hbHMgKyAxIHx8IDI7dmFyIHNpemVzID0gWydCJywgJ0tpQicsICdNaUInLCAn
R2lCJywgJ1RpQicsICdQaUInXTt2YXIgaSA9IE1hdGguZmxvb3IoTWF0aC5sb2coYnl0ZXMpIC8g
TWF0aC5sb2coaykpO3JldHVybiBwYXJzZUZsb2F0KChieXRlcyAvIE1hdGgucG93KGssIGkpKS50
b0ZpeGVkKGRtKSkgKyAobnVtT25seSA/ICcnIDogKCcgJyArIHNpemVzW2ldKSk7fSxpc051bWVy
aWM6IGZ1bmN0aW9uIChuKSB7cmV0dXJuICFpc05hTihwYXJzZUZsb2F0KG4pKSAmJiBpc0Zpbml0
ZShuKTt9LHV0aW1lMnN0cjogZnVuY3Rpb24gKHVzZWMpIHtpZiAodXNlYyA+PSA4NjRFOClyZXR1
cm4gKCh1c2VjKSAvIDg2NEU4KS50b0ZpeGVkKDIpICsgJyBkJztlbHNlIGlmICh1c2VjID49IDM2
RTgpcmV0dXJuICgodXNlYykgLyAzNkU4KS50b0ZpeGVkKDIpICsgJyBoJztlbHNlIGlmICh1c2Vj
ID49IDZFNylyZXR1cm4gKCh1c2VjKSAvIDZFNykudG9GaXhlZCgyKSArICcgbSc7ZWxzZSBpZiAo
dXNlYyA+PSAxRTYpcmV0dXJuICgodXNlYykgLyAxRTYpLnRvRml4ZWQoMikgKyAnIHMnO2Vsc2Ug
aWYgKHVzZWMgPj0gMUUzKXJldHVybiAoKHVzZWMpIC8gMUUzKS50b0ZpeGVkKDIpICsgJyBtcyc7
cmV0dXJuICh1c2VjKS50b0ZpeGVkKDIpICsgJyB1cyc7fSxmb3JtYXREYXRlOiBmdW5jdGlvbiAo
c3RyKSB7dmFyIHkgPSBzdHIuc3Vic3RyKDAsNCksIG0gPSBzdHIuc3Vic3RyKDQsMikgLSAxLCBk
ID0gc3RyLnN1YnN0cig2LDIpLGggPSBzdHIuc3Vic3RyKDgsMikgfHwgMCwgaSA9IHN0ci5zdWJz
dHIoMTAsIDIpICB8fCAwLCBzID0gc3RyLnN1YnN0cigxMiwgMikgfHwgMDt2YXIgZGF0ZSA9IG5l
dyBEYXRlKHksbSxkLGgsaSxzKTt2YXIgb3V0ID0gKCcwJyArIGRhdGUuZ2V0RGF0ZSgpKS5zbGlj
ZSgtMikgKyAnLycgKyB0aGlzLm1vbnRoc1tkYXRlLmdldE1vbnRoKCldICsgJy8nICsgZGF0ZS5n
ZXRGdWxsWWVhcigpOzEwIDw9IHN0ci5sZW5ndGggJiYgKG91dCArPSAiOiIgKyBoKTsxMiA8PSBz
dHIubGVuZ3RoICYmIChvdXQgKz0gIjoiICsgaSk7MTQgPD0gc3RyLmxlbmd0aCAmJiAob3V0ICs9
ICI6IiArIHMpO3JldHVybiBvdXQ7fSxzaG9ydE51bTogIGZ1bmN0aW9uIChuKSB7aWYgKG4gPCAx
ZTMpIHJldHVybiBuO2lmIChuID49IDFlMyAmJiBuIDwgMWU2KSByZXR1cm4gKyhuIC8gMWUzKS50
b0ZpeGVkKDEpICsgIksiO2lmIChuID49IDFlNiAmJiBuIDwgMWU5KSByZXR1cm4gKyhuIC8gMWU2
KS50b0ZpeGVkKDEpICsgIk0iO2lmIChuID49IDFlOSAmJiBuIDwgMWUxMikgcmV0dXJuICsobiAv
IDFlOSkudG9GaXhlZCgxKSArICJCIjtpZiAobiA+PSAxZTEyKSByZXR1cm4gKyhuIC8gMWUxMiku
dG9GaXhlZCgxKSArICJUIjt9LGZtdFZhbHVlOiBmdW5jdGlvbiAodmFsdWUsIGRhdGFUeXBlLCBk
ZWNpbWFscywgc2hvcnRlbiwgaGxyZWdleCwgaGx2YWx1ZSkge3ZhciB2YWwgPSAwO2lmICghZGF0
YVR5cGUpdmFsID0gdmFsdWU7c3dpdGNoIChkYXRhVHlwZSkge2Nhc2UgJ3V0aW1lJzp2YWwgPSB0
aGlzLnV0aW1lMnN0cigrdmFsdWUpO2JyZWFrO2Nhc2UgJ2RhdGUnOnZhbCA9IHRoaXMuZm9ybWF0
RGF0ZSh2YWx1ZSk7YnJlYWs7Y2FzZSAnbnVtZXJpYyc6aWYgKHRoaXMuaXNOdW1lcmljKHZhbHVl
KSl2YWwgPSBzaG9ydGVuID8gdGhpcy5zaG9ydE51bSh2YWx1ZSkgOiAoK3ZhbHVlKS50b0xvY2Fs
ZVN0cmluZygpO2JyZWFrO2Nhc2UgJ2J5dGVzJzp2YWwgPSB0aGlzLmZvcm1hdEJ5dGVzKHZhbHVl
LCBkZWNpbWFscyk7YnJlYWs7Y2FzZSAncGVyY2VudCc6dmFsID0gdmFsdWUucmVwbGFjZSgnLCcs
ICcuJykgKyAnJSc7YnJlYWs7Y2FzZSAndGltZSc6aWYgKHRoaXMuaXNOdW1lcmljKHZhbHVlKSl2
YWwgPSB2YWx1ZS50b0xvY2FsZVN0cmluZygpO2JyZWFrO2Nhc2UgJ3NlY3MnOnZhciB0ID0gbmV3
IERhdGUobnVsbCk7dC5zZXRTZWNvbmRzKHZhbHVlKTt2YWwgPSB0LnRvSVNPU3RyaW5nKCkuc3Vi
c3RyKDExLCA4KTticmVhaztkZWZhdWx0OnZhbCA9IHZhbHVlO31pZiAoaGxyZWdleCkge2xldCBv
ID0gSlNPTi5wYXJzZShobHJlZ2V4KSwgdG1wID0gJyc7Zm9yICh2YXIgeCBpbiBvKSB7aWYgKCF2
YWwpIGNvbnRpbnVlO3RtcCA9IHZhbC5yZXBsYWNlKG5ldyBSZWdFeHAoeCwgJ2dpJyksIG9beF0p
O2lmICh0bXAgIT0gdmFsKSB7dmFsID0gdG1wO2JyZWFrO312YWwgPSB0bXA7fX1yZXR1cm4gdmFs
dWUgPT0gMCA/IFN0cmluZyh2YWwpIDogKHZhbCA9PT0gdW5kZWZpbmVkID8gJ+KAlCcgOiB2YWwp
O30saXNQYW5lbEhpZGRlbjogZnVuY3Rpb24gKHBhbmVsKSB7cmV0dXJuIEdvQWNjZXNzLkFwcFBy
ZWZzLmhpZGRlblBhbmVscy5pbmNsdWRlcyhwYW5lbCk7fSxpc1BhbmVsVmFsaWQ6IGZ1bmN0aW9u
IChwYW5lbCkge3ZhciBkYXRhID0gR29BY2Nlc3MuZ2V0UGFuZWxEYXRhKCksIHVpID0gR29BY2Nl
c3MuZ2V0UGFuZWxVSSgpO3JldHVybiAoIXVpLmhhc093blByb3BlcnR5KHBhbmVsKSB8fCAhZGF0
YS5oYXNPd25Qcm9wZXJ0eShwYW5lbCkgfHwgIXVpW3BhbmVsXS5pZCk7fSxnZXRDb3VudDogZnVu
Y3Rpb24gKGl0ZW0pIHtpZiAodGhpcy5pc09iamVjdChpdGVtKSAmJiAnY291bnQnIGluIGl0ZW0p
cmV0dXJuIGl0ZW0uY291bnQ7cmV0dXJuIGl0ZW07fSxnZXRQZXJjZW50OiBmdW5jdGlvbiAoaXRl
bSkge2lmICh0aGlzLmlzT2JqZWN0KGl0ZW0pICYmICdwZXJjZW50JyBpbiBpdGVtKXJldHVybiB0
aGlzLmZtdFZhbHVlKGl0ZW0ucGVyY2VudCwgJ3BlcmNlbnQnKTtyZXR1cm4gbnVsbDt9LGlzT2Jq
ZWN0OiBmdW5jdGlvbiAobykge3JldHVybiBvID09PSBPYmplY3Qobyk7fSxzZXRQcm9wOiBmdW5j
dGlvbiAobywgcywgdikge3ZhciBzY2hlbWEgPSBvO3ZhciBhID0gcy5zcGxpdCgnLicpO2ZvciAo
dmFyIGkgPSAwLCBuID0gYS5sZW5ndGg7IGkgPCBuLTE7ICsraSkge3ZhciBrID0gYVtpXTtpZiAo
IXNjaGVtYVtrXSlzY2hlbWFba10gPSB7fTtzY2hlbWEgPSBzY2hlbWFba107fXNjaGVtYVthW24t
MV1dID0gdjt9LGdldFByb3A6IGZ1bmN0aW9uIChvLCBzKSB7cyA9IHMucmVwbGFjZSgvXFsoXHcr
KVxdL2csICcuJDEnKTtzID0gcy5yZXBsYWNlKC9eXC4vLCAnJyk7dmFyIGEgPSBzLnNwbGl0KCcu
Jyk7Zm9yICh2YXIgaSA9IDAsIG4gPSBhLmxlbmd0aDsgaSA8IG47ICsraSkge3ZhciBrID0gYVtp
XTtpZiAodGhpcy5pc09iamVjdChvKSAmJiBrIGluIG8pIHtvID0gb1trXTt9IGVsc2Uge3JldHVy
bjt9fXJldHVybiBvO30saGFzTG9jYWxTdG9yYWdlOiBmdW5jdGlvbiAoKSB7dHJ5IHtsb2NhbFN0
b3JhZ2Uuc2V0SXRlbSgndGVzdCcsICd0ZXN0Jyk7bG9jYWxTdG9yYWdlLnJlbW92ZUl0ZW0oJ3Rl
c3QnKTtyZXR1cm4gdHJ1ZTt9IGNhdGNoKGUpIHtyZXR1cm4gZmFsc2U7fX0saXNXaXRoaW5WaWV3
UG9ydDogZnVuY3Rpb24gKGVsKSB7dmFyIGVsZW1Ub3AgPSBlbC5nZXRCb3VuZGluZ0NsaWVudFJl
Y3QoKS50b3A7dmFyIGVsZW1Cb3R0b20gPSBlbC5nZXRCb3VuZGluZ0NsaWVudFJlY3QoKS5ib3R0
b207cmV0dXJuIGVsZW1Ub3AgPCB3aW5kb3cuaW5uZXJIZWlnaHQgJiYgZWxlbUJvdHRvbSA+PSAw
O30sdG9nZ2xlUGFuZWw6IGZ1bmN0aW9uKHBhbmVsKSB7dmFyIGluZGV4ID0gR29BY2Nlc3MuQXBw
UHJlZnMuaGlkZGVuUGFuZWxzLmluZGV4T2YocGFuZWwpO2lmIChpbmRleCA9PSAtMSkge0dvQWNj
ZXNzLkFwcFByZWZzLmhpZGRlblBhbmVscy5wdXNoKHBhbmVsKTt9IGVsc2Uge0dvQWNjZXNzLkFw
cFByZWZzLmhpZGRlblBhbmVscy5zcGxpY2UoaW5kZXgsIDEpO31Hb0FjY2Vzcy5zZXRQcmVmcygp
O2RlbGV0ZSBHb0FjY2Vzcy5BcHBDaGFydHNbcGFuZWxdO0dvQWNjZXNzLk92ZXJhbGxTdGF0cy5p
bml0aWFsaXplKCk7R29BY2Nlc3MuUGFuZWxzLmluaXRpYWxpemUoKTtHb0FjY2Vzcy5DaGFydHMu
aW5pdGlhbGl6ZSgpO0dvQWNjZXNzLlRhYmxlcy5pbml0aWFsaXplKCk7fSx9O0dvQWNjZXNzLk92
ZXJhbGxTdGF0cyA9IHt0b3RhbF9yZXF1ZXN0czogMCxyZW5kZXJCb3g6IGZ1bmN0aW9uIChkYXRh
LCB1aSwgcm93LCB4LCBpZHgpIHt2YXIgd3JhcCA9ICQoJy53cmFwLWdlbmVyYWwtaXRlbXMnKTtp
ZiAoaWR4ICUgNiA9PSAwKSB7cm93ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnZGl2Jyk7cm93
LnNldEF0dHJpYnV0ZSgnY2xhc3MnLCAncm93Jyk7d3JhcC5hcHBlbmRDaGlsZChyb3cpO312YXIg
Ym94ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnZGl2Jyk7Ym94LmlubmVySFRNTCA9IEdvQWNj
ZXNzLkFwcFRwbHMuR2VuZXJhbC5pdGVtcy5yZW5kZXIoeydpZCc6IHgsJ2NsYXNzTmFtZSc6IHVp
Lml0ZW1zW3hdLmNsYXNzTmFtZSwnbGFiZWwnOiB1aS5pdGVtc1t4XS5sYWJlbCwndmFsdWUnOiBH
b0FjY2Vzcy5VdGlsLmZtdFZhbHVlKGRhdGFbeF0sIHVpLml0ZW1zW3hdLmRhdGFUeXBlKSx9KTty
b3cuYXBwZW5kQ2hpbGQoYm94KTtyZXR1cm4gcm93O30scmVuZGVyRGF0YTogZnVuY3Rpb24gKGRh
dGEsIHVpKSB7dmFyIGlkeCA9IDAsIHJvdyA9IG51bGw7JCgnLmxhc3QtdXBkYXRlZCcpLmlubmVy
SFRNTCA9IGRhdGEuZGF0ZV90aW1lOyQoJy53cmFwLWdlbmVyYWwnKS5pbm5lckhUTUwgPSAnJztp
ZiAoR29BY2Nlc3MuVXRpbC5pc1BhbmVsSGlkZGVuKCdnZW5lcmFsJykpcmV0dXJuIGZhbHNlOyQo
Jy53cmFwLWdlbmVyYWwnKS5pbm5lckhUTUwgPSBHb0FjY2Vzcy5BcHBUcGxzLkdlbmVyYWwud3Jh
cC5yZW5kZXIoR29BY2Nlc3MuVXRpbC5tZXJnZSh1aSwgeydmcm9tJzogZGF0YS5zdGFydF9kYXRl
LCd0byc6IGRhdGEuZW5kX2RhdGUsfSkpO2ZvciAodmFyIHggaW4gZGF0YSkge2lmICghZGF0YS5o
YXNPd25Qcm9wZXJ0eSh4KSB8fCAhdWkuaXRlbXMuaGFzT3duUHJvcGVydHkoeCkpY29udGludWU7
cm93ID0gdGhpcy5yZW5kZXJCb3goZGF0YSwgdWksIHJvdywgeCwgaWR4KTtpZHgrKzt9fSxpbml0
aWFsaXplOiBmdW5jdGlvbiAoKSB7dmFyIHVpID0gR29BY2Nlc3MuZ2V0UGFuZWxVSSgnZ2VuZXJh
bCcpO3ZhciBkYXRhID0gR29BY2Nlc3MuZ2V0UGFuZWxEYXRhKCdnZW5lcmFsJyk7dGhpcy50b3Rh
bF9yZXF1ZXN0cyA9IGRhdGEudG90YWxfcmVxdWVzdHM7dGhpcy5yZW5kZXJEYXRhKGRhdGEsIHVp
KTt9fTtHb0FjY2Vzcy5OYXYgPSB7ZXZlbnRzOiBmdW5jdGlvbiAoKSB7JCgnLm5hdi1iYXJzJyku
b25jbGljayA9IGZ1bmN0aW9uIChlKSB7ZS5zdG9wUHJvcGFnYXRpb24oKTt0aGlzLnJlbmRlck1l
bnUoZSk7fS5iaW5kKHRoaXMpOyQoJy5uYXYtZ2VhcnMnKS5vbmNsaWNrID0gZnVuY3Rpb24gKGUp
IHtlLnN0b3BQcm9wYWdhdGlvbigpO3RoaXMucmVuZGVyT3B0cyhlKTt9LmJpbmQodGhpcyk7JCgn
Lm5hdi1taW5pYmFycycpLm9uY2xpY2sgPSBmdW5jdGlvbiAoZSkge2Uuc3RvcFByb3BhZ2F0aW9u
KCk7dGhpcy5yZW5kZXJPcHRzKGUpO30uYmluZCh0aGlzKTskKCdib2R5Jykub25jbGljayA9IGZ1
bmN0aW9uIChlKSB7JCgnbmF2JykuY2xhc3NMaXN0LnJlbW92ZSgnYWN0aXZlJyk7fS5iaW5kKHRo
aXMpOyQkKCcuZXhwb3J0LWpzb24nLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9IGZ1
bmN0aW9uIChlKSB7dGhpcy5kb3dubG9hZEpTT04oZSk7fS5iaW5kKHRoaXMpO30uYmluZCh0aGlz
KSk7JCQoJy50aGVtZS1icmlnaHQnLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9IGZ1
bmN0aW9uIChlKSB7dGhpcy5zZXRUaGVtZSgnYnJpZ2h0Jyk7fS5iaW5kKHRoaXMpO30uYmluZCh0
aGlzKSk7JCQoJy50aGVtZS1kYXJrLWJsdWUnLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGlj
ayA9IGZ1bmN0aW9uIChlKSB7dGhpcy5zZXRUaGVtZSgnZGFya0JsdWUnKTt9LmJpbmQodGhpcyk7
fS5iaW5kKHRoaXMpKTskJCgnLnRoZW1lLWRhcmstZ3JheScsIGZ1bmN0aW9uIChpdGVtKSB7aXRl
bS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt0aGlzLnNldFRoZW1lKCdkYXJrR3JheScpO30uYmlu
ZCh0aGlzKTt9LmJpbmQodGhpcykpOyQkKCcudGhlbWUtZGFyay1wdXJwbGUnLCBmdW5jdGlvbiAo
aXRlbSkge2l0ZW0ub25jbGljayA9IGZ1bmN0aW9uIChlKSB7dGhpcy5zZXRUaGVtZSgnZGFya1B1
cnBsZScpO30uYmluZCh0aGlzKTt9LmJpbmQodGhpcykpOyQkKCcubGF5b3V0LWhvcml6b250YWwn
LCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9IGZ1bmN0aW9uIChlKSB7dGhpcy5zZXRM
YXlvdXQoJ2hvcml6b250YWwnKTt9LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTskJCgnLmxheW91
dC12ZXJ0aWNhbCcsIGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUp
IHt0aGlzLnNldExheW91dCgndmVydGljYWwnKTt9LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTsk
JCgnLmxheW91dC13aWRlJywgZnVuY3Rpb24gKGl0ZW0pIHtpdGVtLm9uY2xpY2sgPSBmdW5jdGlv
biAoZSkge3RoaXMuc2V0TGF5b3V0KCd3aWRlJyk7fS5iaW5kKHRoaXMpO30uYmluZCh0aGlzKSk7
JCQoJ1tkYXRhLXBlcnBhZ2VdJywgZnVuY3Rpb24gKGl0ZW0pIHtpdGVtLm9uY2xpY2sgPSBmdW5j
dGlvbiAoZSkge3RoaXMuc2V0UGVyUGFnZShlKTt9LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTsk
JCgnW2RhdGEtc2hvdy10YWJsZXNdJywgZnVuY3Rpb24gKGl0ZW0pIHtpdGVtLm9uY2xpY2sgPSBm
dW5jdGlvbiAoZSkge3RoaXMudG9nZ2xlVGFibGVzKCk7fS5iaW5kKHRoaXMpO30uYmluZCh0aGlz
KSk7JCQoJ1tkYXRhLWF1dG9oaWRlLXRhYmxlc10nLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25j
bGljayA9IGZ1bmN0aW9uIChlKSB7dGhpcy50b2dnbGVBdXRvSGlkZVRhYmxlcygpO30uYmluZCh0
aGlzKTt9LmJpbmQodGhpcykpOyQkKCcudG9nZ2xlLXBhbmVsJywgZnVuY3Rpb24gKGl0ZW0pIHtp
dGVtLm9uY2xpY2sgPSBmdW5jdGlvbiAoZSkge2Uuc3RvcFByb3BhZ2F0aW9uKCk7dmFyIHBhbmVs
ID0gZS5jdXJyZW50VGFyZ2V0LmdldEF0dHJpYnV0ZSgnZGF0YS1wYW5lbCcpO0dvQWNjZXNzLlV0
aWwudG9nZ2xlUGFuZWwocGFuZWwpO2l0ZW0uY2xhc3NMaXN0LnRvZ2dsZSgnYWN0aXZlJyk7fS5i
aW5kKHRoaXMpO30uYmluZCh0aGlzKSk7fSxkb3dubG9hZEpTT046IGZ1bmN0aW9uIChlKSB7dmFy
IHRhcmcgPSBlLmN1cnJlbnRUYXJnZXQ7dmFyIGRhdGEgPSAidGV4dC9qc29uO2NoYXJzZXQ9dXRm
LTgsIiArIGVuY29kZVVSSUNvbXBvbmVudChKU09OLnN0cmluZ2lmeShHb0FjY2Vzcy5nZXRQYW5l
bERhdGEoKSkpO3RhcmcuaHJlZiA9ICdkYXRhOicgKyBkYXRhO3RhcmcuZG93bmxvYWQgPSAnZ29h
Y2Nlc3MtJyArICgrbmV3IERhdGUoKSkgKyAnLmpzb24nO30sc2V0TGF5b3V0OiBmdW5jdGlvbiAo
bGF5b3V0KSB7aWYgKCgnaG9yaXpvbnRhbCcgPT0gbGF5b3V0IHx8ICd3aWRlJyA9PSBsYXlvdXQp
ICYmICQoJy5jb250YWluZXInKSkgeyQoJy5jb250YWluZXInKS5jbGFzc0xpc3QuYWRkKCdjb250
YWluZXItZmx1aWQnKTskKCcuY29udGFpbmVyJykuY2xhc3NMaXN0LnJlbW92ZSgnY29udGFpbmVy
Jyk7fSBlbHNlIGlmICgndmVydGljYWwnID09IGxheW91dCAmJiAkKCcuY29udGFpbmVyLWZsdWlk
JykpIHskKCcuY29udGFpbmVyLWZsdWlkJykuY2xhc3NMaXN0LmFkZCgnY29udGFpbmVyJyk7JCgn
LmNvbnRhaW5lcicpLmNsYXNzTGlzdC5yZW1vdmUoJ2NvbnRhaW5lci1mbHVpZCcpO31Hb0FjY2Vz
cy5BcHBQcmVmc1snbGF5b3V0J10gPSBsYXlvdXQ7R29BY2Nlc3Muc2V0UHJlZnMoKTtHb0FjY2Vz
cy5QYW5lbHMuaW5pdGlhbGl6ZSgpO0dvQWNjZXNzLkNoYXJ0cy5pbml0aWFsaXplKCk7R29BY2Nl
c3MuVGFibGVzLmluaXRpYWxpemUoKTt9LHRvZ2dsZUF1dG9IaWRlVGFibGVzOiBmdW5jdGlvbiAo
ZSkge3ZhciBhdXRvSGlkZVRhYmxlcyA9IEdvQWNjZXNzLlRhYmxlcy5hdXRvSGlkZVRhYmxlcygp
OyQkKCcudGFibGUtd3JhcHBlcicsIGZ1bmN0aW9uIChpdGVtKSB7aWYgKGF1dG9IaWRlVGFibGVz
KSB7aXRlbS5jbGFzc0xpc3QucmVtb3ZlKCdoaWRkZW4teHMnKTt9IGVsc2Uge2l0ZW0uY2xhc3NM
aXN0LmFkZCgnaGlkZGVuLXhzJyk7fX0uYmluZCh0aGlzKSk7R29BY2Nlc3MuQXBwUHJlZnNbJ2F1
dG9IaWRlVGFibGVzJ10gPSAhYXV0b0hpZGVUYWJsZXM7R29BY2Nlc3Muc2V0UHJlZnMoKTt9LHRv
Z2dsZVRhYmxlczogZnVuY3Rpb24gKCkge3ZhciB1aSA9IEdvQWNjZXNzLmdldFBhbmVsVUkoKTt2
YXIgc2hvd1RhYmxlcyA9IEdvQWNjZXNzLlRhYmxlcy5zaG93VGFibGVzKCk7T2JqZWN0LmtleXMo
dWkpLmZvckVhY2goZnVuY3Rpb24gKHBhbmVsLCBpZHgpIHtpZiAoIUdvQWNjZXNzLlV0aWwuaXNQ
YW5lbFZhbGlkKHBhbmVsKSB8fCBHb0FjY2Vzcy5VdGlsLmlzUGFuZWxIaWRkZW4ocGFuZWwpKXVp
W3BhbmVsXVsndGFibGUnXSA9ICFzaG93VGFibGVzO30uYmluZCh0aGlzKSk7R29BY2Nlc3MuQXBw
UHJlZnNbJ3Nob3dUYWJsZXMnXSA9ICFzaG93VGFibGVzO0dvQWNjZXNzLnNldFByZWZzKCk7R29B
Y2Nlc3MuUGFuZWxzLmluaXRpYWxpemUoKTtHb0FjY2Vzcy5DaGFydHMuaW5pdGlhbGl6ZSgpO0dv
QWNjZXNzLlRhYmxlcy5pbml0aWFsaXplKCk7fSxzZXRUaGVtZTogZnVuY3Rpb24gKHRoZW1lKSB7
aWYgKCF0aGVtZSlyZXR1cm47JCgnaHRtbCcpLmNsYXNzTmFtZSA9ICcnO3N3aXRjaCh0aGVtZSkg
e2Nhc2UgJ2RhcmtHcmF5JzokKCdodG1sJykuY2xhc3NMaXN0LmFkZCgnZGFyaycpOyQoJ2h0bWwn
KS5jbGFzc0xpc3QuYWRkKCdncmF5Jyk7YnJlYWs7Y2FzZSAnZGFya0JsdWUnOiQoJ2h0bWwnKS5j
bGFzc0xpc3QuYWRkKCdkYXJrJyk7JCgnaHRtbCcpLmNsYXNzTGlzdC5hZGQoJ2JsdWUnKTticmVh
aztjYXNlICdkYXJrUHVycGxlJzokKCdodG1sJykuY2xhc3NMaXN0LmFkZCgnZGFyaycpOyQoJ2h0
bWwnKS5jbGFzc0xpc3QuYWRkKCdwdXJwbGUnKTticmVhazt9R29BY2Nlc3MuQXBwUHJlZnNbJ3Ro
ZW1lJ10gPSB0aGVtZTtHb0FjY2Vzcy5zZXRQcmVmcygpO30sZ2V0SWNvbjogZnVuY3Rpb24gKGtl
eSkge3N3aXRjaChrZXkpIHtjYXNlICd2aXNpdG9ycycgICAgICAgIDogcmV0dXJuICd1c2Vycyc7
Y2FzZSAncmVxdWVzdHMnICAgICAgICA6IHJldHVybiAnZmlsZSc7Y2FzZSAnc3RhdGljX3JlcXVl
c3RzJyA6IHJldHVybiAnZmlsZS10ZXh0JztjYXNlICdub3RfZm91bmQnICAgICAgIDogcmV0dXJu
ICdmaWxlLW8nO2Nhc2UgJ2hvc3RzJyAgICAgICAgICAgOiByZXR1cm4gJ3VzZXInO2Nhc2UgJ29z
JyAgICAgICAgICAgICAgOiByZXR1cm4gJ2Rlc2t0b3AnO2Nhc2UgJ2Jyb3dzZXJzJyAgICAgICAg
OiByZXR1cm4gJ2Nocm9tZSc7Y2FzZSAndmlzaXRfdGltZScgICAgICA6IHJldHVybiAnY2xvY2st
byc7Y2FzZSAndmhvc3RzJyAgICAgICAgICA6IHJldHVybiAndGgtbGlzdCc7Y2FzZSAncmVmZXJy
ZXJzJyAgICAgICA6IHJldHVybiAnZXh0ZXJuYWwtbGluayc7Y2FzZSAncmVmZXJyaW5nX3NpdGVz
JyA6IHJldHVybiAnZXh0ZXJuYWwtbGluayc7Y2FzZSAna2V5cGhyYXNlcycgICAgICA6IHJldHVy
biAnZ29vZ2xlJztjYXNlICdzdGF0dXNfY29kZXMnICAgIDogcmV0dXJuICd3YXJuaW5nJztjYXNl
ICdyZW1vdGVfdXNlcicgICAgIDogcmV0dXJuICd1c2Vycyc7Y2FzZSAnZ2VvbG9jYXRpb24nICAg
ICA6IHJldHVybiAnbWFwLW1hcmtlcic7Y2FzZSAnYXNuJyAgICAgICAgICAgICA6IHJldHVybiAn
bWFwLW1hcmtlcic7Y2FzZSAnbWltZV90eXBlJyAgICAgICA6IHJldHVybiAnZmlsZS1vJztjYXNl
ICd0bHNfdHlwZScgICAgICAgIDogcmV0dXJuICd3YXJuaW5nJztkZWZhdWx0ICAgICAgICAgICAg
ICAgIDogcmV0dXJuICdwaWUtY2hhcnQnO319LGdldEl0ZW1zOiBmdW5jdGlvbiAoKSB7dmFyIHVp
ID0gR29BY2Nlc3MuZ2V0UGFuZWxVSSgpLCBtZW51ID0gW107Zm9yICh2YXIgcGFuZWwgaW4gdWkp
IHtpZiAoR29BY2Nlc3MuVXRpbC5pc1BhbmVsVmFsaWQocGFuZWwpKWNvbnRpbnVlO21lbnUucHVz
aCh7J2N1cnJlbnQnOiB3aW5kb3cubG9jYXRpb24uaGFzaC5zdWJzdHIoMSkgPT0gcGFuZWwsJ2hl
YWQnOiB1aVtwYW5lbF0uaGVhZCwna2V5JzogcGFuZWwsJ2ljb24nOiB0aGlzLmdldEljb24ocGFu
ZWwpLCdoaWRkZW4nOiBHb0FjY2Vzcy5VdGlsLmlzUGFuZWxIaWRkZW4ocGFuZWwpfSk7fXJldHVy
biBtZW51O30sc2V0UGVyUGFnZTogZnVuY3Rpb24gKGUpIHtHb0FjY2Vzcy5BcHBQcmVmc1sncGVy
UGFnZSddID0gK2UuY3VycmVudFRhcmdldC5nZXRBdHRyaWJ1dGUoJ2RhdGEtcGVycGFnZScpO0dv
QWNjZXNzLkFwcC5yZW5kZXJEYXRhKCk7R29BY2Nlc3Muc2V0UHJlZnMoKTtHb0FjY2Vzcy5UYWJs
ZXMuaW5pdGlhbGl6ZSgpO30sZ2V0VGhlbWU6IGZ1bmN0aW9uICgpIHtyZXR1cm4gR29BY2Nlc3Mu
QXBwUHJlZnMudGhlbWUgfHwgJ2RhcmtHcmF5Jzt9LGdldExheW91dDogZnVuY3Rpb24gKCkge3Jl
dHVybiBHb0FjY2Vzcy5BcHBQcmVmcy5sYXlvdXQgfHwgJ2hvcml6b250YWwnO30sZ2V0UGVyUGFn
ZTogZnVuY3Rpb24gKCkge3JldHVybiBHb0FjY2Vzcy5BcHBQcmVmcy5wZXJQYWdlIHx8IDc7fSxy
ZW5kZXJPcHRzOiBmdW5jdGlvbiAoKSB7dmFyIG8gPSB7fTtvW3RoaXMuZ2V0TGF5b3V0KCldID0g
dHJ1ZTtvW3RoaXMuZ2V0VGhlbWUoKV0gPSB0cnVlO29bJ3BlclBhZ2UnICsgdGhpcy5nZXRQZXJQ
YWdlKCldID0gdHJ1ZTtvWydhdXRvSGlkZVRhYmxlcyddID0gR29BY2Nlc3MuVGFibGVzLmF1dG9I
aWRlVGFibGVzKCk7b1snc2hvd1RhYmxlcyddID0gR29BY2Nlc3MuVGFibGVzLnNob3dUYWJsZXMo
KTtvWydsYWJlbHMnXSA9IEdvQWNjZXNzLmkxOG47JCgnLm5hdi1saXN0JykuaW5uZXJIVE1MID0g
R29BY2Nlc3MuQXBwVHBscy5OYXYub3B0cy5yZW5kZXIobyk7cmVxdWVzdEFuaW1hdGlvbkZyYW1l
KGZ1bmN0aW9uICgpIHskKCduYXYnKS5jbGFzc0xpc3QudG9nZ2xlKCdhY3RpdmUnKTt9KTt0aGlz
LmV2ZW50cygpO30scmVuZGVyTWVudTogZnVuY3Rpb24gKGUpIHskKCcubmF2LWxpc3QnKS5pbm5l
ckhUTUwgPSBHb0FjY2Vzcy5BcHBUcGxzLk5hdi5tZW51LnJlbmRlcih7J25hdic6IHRoaXMuZ2V0
SXRlbXMoKSwnb3ZlcmFsbF9jdXJyZW50Jzogd2luZG93LmxvY2F0aW9uLmhhc2guc3Vic3RyKDEp
ID09ICcnLCdvdmVyYWxsX2hpZGRlbic6IEdvQWNjZXNzLlV0aWwuaXNQYW5lbEhpZGRlbignZ2Vu
ZXJhbCcpLCdsYWJlbHMnOiBHb0FjY2Vzcy5pMThuLH0pO3JlcXVlc3RBbmltYXRpb25GcmFtZShm
dW5jdGlvbiAoKSB7JCgnbmF2JykuY2xhc3NMaXN0LnRvZ2dsZSgnYWN0aXZlJyk7fSk7dGhpcy5l
dmVudHMoKTt9LFdTU3RhdHVzOiBmdW5jdGlvbiAoKSB7aWYgKE9iamVjdC5rZXlzKEdvQWNjZXNz
LkFwcFdTQ29ubikubGVuZ3RoKSQkKCcubmF2LXdzLXN0YXR1cycsIGZ1bmN0aW9uIChpdGVtKSB7
IGl0ZW0uc3R5bGUuZGlzcGxheSA9ICdibG9jayc7IH0pO30sV1NDbG9zZTogZnVuY3Rpb24gKCkg
eyQkKCcubmF2LXdzLXN0YXR1cycsIGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5jbGFzc0xpc3QucmVt
b3ZlKCdjb25uZWN0ZWQnKTtpdGVtLnNldEF0dHJpYnV0ZSgndGl0bGUnLCAnRGlzY29ubmVjdGVk
Jyk7fSk7fSxXU09wZW46IGZ1bmN0aW9uIChzdHIpIHskJCgnLm5hdi13cy1zdGF0dXMnLCBmdW5j
dGlvbiAoaXRlbSkge2l0ZW0uY2xhc3NMaXN0LmFkZCgnY29ubmVjdGVkJyk7aXRlbS5zZXRBdHRy
aWJ1dGUoJ3RpdGxlJywgJ0Nvbm5lY3RlZCB0byAnICsgc3RyKTt9KTt9LHJlbmRlcldyYXA6IGZ1
bmN0aW9uIChuYXYpIHskKCduYXYnKS5pbm5lckhUTUwgPSBHb0FjY2Vzcy5BcHBUcGxzLk5hdi53
cmFwLnJlbmRlcihHb0FjY2Vzcy5pMThuKTt9LGluaXRpYWxpemU6IGZ1bmN0aW9uICgpIHt0aGlz
LnNldFRoZW1lKEdvQWNjZXNzLkFwcFByZWZzLnRoZW1lKTt0aGlzLnJlbmRlcldyYXAoKTt0aGlz
LldTU3RhdHVzKCk7dGhpcy5ldmVudHMoKTt9fTtHb0FjY2Vzcy5QYW5lbHMgPSB7ZXZlbnRzOiBm
dW5jdGlvbiAoKSB7JCQoJ1tkYXRhLXRvZ2dsZT1kcm9wZG93bl0nLCBmdW5jdGlvbiAoaXRlbSkg
e2l0ZW0ub25jbGljayA9IGZ1bmN0aW9uIChlKSB7dGhpcy5vcGVuT3B0cyhlLmN1cnJlbnRUYXJn
ZXQpO30uYmluZCh0aGlzKTtpdGVtLm9uYmx1ciA9IGZ1bmN0aW9uIChlKSB7dGhpcy5jbG9zZU9w
dHMoZSk7fS5iaW5kKHRoaXMpO30uYmluZCh0aGlzKSk7JCQoJ1tkYXRhLXBsb3RdJywgZnVuY3Rp
b24gKGl0ZW0pIHtpdGVtLm9uY2xpY2sgPSBmdW5jdGlvbiAoZSkge0dvQWNjZXNzLkNoYXJ0cy5y
ZWRyYXdDaGFydChlLmN1cnJlbnRUYXJnZXQpO30uYmluZCh0aGlzKTt9LmJpbmQodGhpcykpOyQk
KCdbZGF0YS1jaGFydF0nLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9IGZ1bmN0aW9u
IChlKSB7R29BY2Nlc3MuQ2hhcnRzLnRvZ2dsZUNoYXJ0KGUuY3VycmVudFRhcmdldCk7fS5iaW5k
KHRoaXMpO30uYmluZCh0aGlzKSk7JCQoJ1tkYXRhLWNoYXJ0LXR5cGVdJywgZnVuY3Rpb24gKGl0
ZW0pIHtpdGVtLm9uY2xpY2sgPSBmdW5jdGlvbiAoZSkge0dvQWNjZXNzLkNoYXJ0cy5zZXRDaGFy
dFR5cGUoZS5jdXJyZW50VGFyZ2V0KTt9LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTskJCgnW2Rh
dGEtbWV0cmljXScsIGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUp
IHtHb0FjY2Vzcy5UYWJsZXMudG9nZ2xlQ29sdW1uKGUuY3VycmVudFRhcmdldCk7fS5iaW5kKHRo
aXMpO30uYmluZCh0aGlzKSk7fSxvcGVuT3B0czogZnVuY3Rpb24gKHRhcmcpIHt2YXIgcGFuZWwg
PSB0YXJnLmdldEF0dHJpYnV0ZSgnZGF0YS1wYW5lbCcpO3RhcmcucGFyZW50RWxlbWVudC5jbGFz
c0xpc3QudG9nZ2xlKCdvcGVuJyk7dGhpcy5yZW5kZXJPcHRzKHBhbmVsKTt9LGNsb3NlT3B0czog
ZnVuY3Rpb24gKGUpIHtlLmN1cnJlbnRUYXJnZXQucGFyZW50RWxlbWVudC5jbGFzc0xpc3QucmVt
b3ZlKCdvcGVuJyk7aWYgKGUucmVsYXRlZFRhcmdldCAmJiBlLnJlbGF0ZWRUYXJnZXQuZ2V0QXR0
cmlidXRlKCdkYXRhLXRvZ2dsZScpICE9PSAnZHJvcGRvd24nKWUucmVsYXRlZFRhcmdldC5jbGlj
aygpO30sc2V0UGxvdFNlbGVjdGlvbjogZnVuY3Rpb24gKHVpLCBwcmVmcykge3ZhciBjaGFydFR5
cGUgPSAoKHByZWZzIHx8IHt9KS5wbG90IHx8IHt9KS5jaGFydFR5cGUgfHwgdWkucGxvdFswXS5j
aGFydFR5cGU7dmFyIG1ldHJpYyA9ICgocHJlZnMgfHwge30pLnBsb3QgfHwge30pLm1ldHJpYyB8
fCB1aS5wbG90WzBdLmNsYXNzTmFtZTt1aVtjaGFydFR5cGVdID0gdHJ1ZTtmb3IgKHZhciBpID0g
MCwgbGVuID0gdWkucGxvdC5sZW5ndGg7IGkgPCBsZW47ICsraSlpZiAodWkucGxvdFtpXS5jbGFz
c05hbWUgPT0gbWV0cmljKXVpLnBsb3RbaV1bJ3NlbGVjdGVkJ10gPSB0cnVlO30sc2V0Q29sU2Vs
ZWN0aW9uOiBmdW5jdGlvbiAoaXRlbXMsIHByZWZzKSB7dmFyIGNvbHVtbnMgPSAocHJlZnMgfHwg
e30pLmNvbHVtbnMgfHwge307Zm9yICh2YXIgaSA9IDAsIGxlbiA9IGl0ZW1zLmxlbmd0aDsgaSA8
IGxlbjsgKytpKWlmICgoaXRlbXNbaV0ua2V5IGluIGNvbHVtbnMpICYmIGNvbHVtbnNbaXRlbXNb
aV0ua2V5XVsnaGlkZSddKWl0ZW1zW2ldWydoaWRlJ10gPSB0cnVlO30sc2V0Q2hhcnRTZWxlY3Rp
b246IGZ1bmN0aW9uICh1aSwgcHJlZnMpIHt1aVsnc2hvd0NoYXJ0J10gPSBwcmVmcyAmJiAoJ2No
YXJ0JyBpbiBwcmVmcykgPyBwcmVmcy5jaGFydCA6IHRydWU7fSxzZXRPcHRzOiBmdW5jdGlvbiAo
cGFuZWwpIHt2YXIgdWkgPSBKU09OLnBhcnNlKEpTT04uc3RyaW5naWZ5KEdvQWNjZXNzLmdldFBh
bmVsVUkocGFuZWwpKSksIHByZWZzID0gR29BY2Nlc3MuZ2V0UHJlZnMocGFuZWwpO3RoaXMuc2V0
Q2hhcnRTZWxlY3Rpb24odWksIHByZWZzKTt0aGlzLnNldFBsb3RTZWxlY3Rpb24odWksIHByZWZz
KTt0aGlzLnNldENvbFNlbGVjdGlvbih1aS5pdGVtcywgcHJlZnMpO3JldHVybiBHb0FjY2Vzcy5V
dGlsLm1lcmdlKHVpLCB7J2xhYmVscyc6IEdvQWNjZXNzLmkxOG59KTt9LHJlbmRlck9wdHM6IGZ1
bmN0aW9uIChwYW5lbCkgeyQoJy5wYW5lbC1vcHRzLScgKyBwYW5lbCkuaW5uZXJIVE1MID0gR29B
Y2Nlc3MuQXBwVHBscy5QYW5lbHMub3B0cy5yZW5kZXIodGhpcy5zZXRPcHRzKHBhbmVsKSk7dGhp
cy5ldmVudHMoKTt9LGVuYWJsZVByZXY6IGZ1bmN0aW9uIChwYW5lbCkge3ZhciAkcGFnaW5hdGlv
biA9ICQoJyNwYW5lbC0nICsgcGFuZWwgKyAnIC5wYWdpbmF0aW9uIGEucGFuZWwtcHJldicpO2lm
ICgkcGFnaW5hdGlvbikkcGFnaW5hdGlvbi5wYXJlbnROb2RlLmNsYXNzTGlzdC5yZW1vdmUoJ2Rp
c2FibGVkJyk7fSxkaXNhYmxlUHJldjogZnVuY3Rpb24gKHBhbmVsKSB7dmFyICRwYWdpbmF0aW9u
ID0gJCgnI3BhbmVsLScgKyBwYW5lbCArICcgLnBhZ2luYXRpb24gYS5wYW5lbC1wcmV2Jyk7aWYg
KCRwYWdpbmF0aW9uKSRwYWdpbmF0aW9uLnBhcmVudE5vZGUuY2xhc3NMaXN0LmFkZCgnZGlzYWJs
ZWQnKTt9LGVuYWJsZU5leHQ6IGZ1bmN0aW9uIChwYW5lbCkge3ZhciAkcGFnaW5hdGlvbiA9ICQo
JyNwYW5lbC0nICsgcGFuZWwgKyAnIC5wYWdpbmF0aW9uIGEucGFuZWwtbmV4dCcpO2lmICgkcGFn
aW5hdGlvbikkcGFnaW5hdGlvbi5wYXJlbnROb2RlLmNsYXNzTGlzdC5yZW1vdmUoJ2Rpc2FibGVk
Jyk7fSxkaXNhYmxlTmV4dDogZnVuY3Rpb24gKHBhbmVsKSB7dmFyICRwYWdpbmF0aW9uID0gJCgn
I3BhbmVsLScgKyBwYW5lbCArICcgLnBhZ2luYXRpb24gYS5wYW5lbC1uZXh0Jyk7aWYgKCRwYWdp
bmF0aW9uKSRwYWdpbmF0aW9uLnBhcmVudE5vZGUuY2xhc3NMaXN0LmFkZCgnZGlzYWJsZWQnKTt9
LGVuYWJsZUZpcnN0OiBmdW5jdGlvbiAocGFuZWwpIHt2YXIgJHBhZ2luYXRpb24gPSAkKCcjcGFu
ZWwtJyArIHBhbmVsICsgJyAucGFnaW5hdGlvbiBhLnBhbmVsLWZpcnN0Jyk7aWYgKCRwYWdpbmF0
aW9uKSRwYWdpbmF0aW9uLnBhcmVudE5vZGUuY2xhc3NMaXN0LnJlbW92ZSgnZGlzYWJsZWQnKTt9
LGRpc2FibGVGaXJzdDogZnVuY3Rpb24gKHBhbmVsKSB7dmFyICRwYWdpbmF0aW9uID0gJCgnI3Bh
bmVsLScgKyBwYW5lbCArICcgLnBhZ2luYXRpb24gYS5wYW5lbC1maXJzdCcpO2lmICgkcGFnaW5h
dGlvbikkcGFnaW5hdGlvbi5wYXJlbnROb2RlLmNsYXNzTGlzdC5hZGQoJ2Rpc2FibGVkJyk7fSxl
bmFibGVMYXN0OiBmdW5jdGlvbiAocGFuZWwpIHt2YXIgJHBhZ2luYXRpb24gPSAkKCcjcGFuZWwt
JyArIHBhbmVsICsgJyAucGFnaW5hdGlvbiBhLnBhbmVsLWxhc3QnKTtpZiAoJHBhZ2luYXRpb24p
JHBhZ2luYXRpb24ucGFyZW50Tm9kZS5jbGFzc0xpc3QucmVtb3ZlKCdkaXNhYmxlZCcpO30sZGlz
YWJsZUxhc3Q6IGZ1bmN0aW9uIChwYW5lbCkge3ZhciAkcGFnaW5hdGlvbiA9ICQoJyNwYW5lbC0n
ICsgcGFuZWwgKyAnIC5wYWdpbmF0aW9uIGEucGFuZWwtbGFzdCcpO2lmICgkcGFnaW5hdGlvbikk
cGFnaW5hdGlvbi5wYXJlbnROb2RlLmNsYXNzTGlzdC5hZGQoJ2Rpc2FibGVkJyk7fSxlbmFibGVQ
YWdpbmF0aW9uOiBmdW5jdGlvbiAocGFuZWwpIHt0aGlzLmVuYWJsZVByZXYocGFuZWwpO3RoaXMu
ZW5hYmxlTmV4dChwYW5lbCk7dGhpcy5lbmFibGVGaXJzdChwYW5lbCk7dGhpcy5lbmFibGVMYXN0
KHBhbmVsKTt9LGRpc2FibGVQYWdpbmF0aW9uOiBmdW5jdGlvbiAocGFuZWwpIHt0aGlzLmRpc2Fi
bGVQcmV2KHBhbmVsKTt0aGlzLmRpc2FibGVOZXh0KHBhbmVsKTt0aGlzLmRpc2FibGVGaXJzdChw
YW5lbCk7dGhpcy5kaXNhYmxlTGFzdChwYW5lbCk7fSxoYXNTdWJJdGVtczogZnVuY3Rpb24gKHVp
LCBkYXRhKSB7Zm9yICh2YXIgaSA9IDAsIGxlbiA9IGRhdGEubGVuZ3RoOyBpIDwgbGVuOyArK2kp
IHtpZiAoIWRhdGFbaV0uaXRlbXMpcmV0dXJuICh1aVsnaGFzU3ViSXRlbXMnXSA9IGZhbHNlKTtp
ZiAoZGF0YVtpXS5pdGVtcy5sZW5ndGgpIHtyZXR1cm4gKHVpWydoYXNTdWJJdGVtcyddID0gdHJ1
ZSk7fX1yZXR1cm4gZmFsc2U7fSxzZXRDb21wdXRlZERhdGE6IGZ1bmN0aW9uIChwYW5lbCwgdWks
IGRhdGEpIHt0aGlzLmhhc1N1Ykl0ZW1zKHVpLCBkYXRhLmRhdGEpO0dvQWNjZXNzLkNoYXJ0cy5o
YXNDaGFydChwYW5lbCwgdWkpO0dvQWNjZXNzLlRhYmxlcy5oYXNUYWJsZSh1aSk7fSxyZW5kZXJQ
YW5lbDogZnVuY3Rpb24gKHBhbmVsLCB1aSwgY29sKSB7dmFyIGRhdGEgPSBHb0FjY2Vzcy5nZXRQ
YW5lbERhdGEocGFuZWwpO3RoaXMuc2V0Q29tcHV0ZWREYXRhKHBhbmVsLCB1aSwgZGF0YSk7dmFy
IGJveCA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2RpdicpO2JveC5pZCA9ICdwYW5lbC0nICsg
cGFuZWw7Ym94LmlubmVySFRNTCA9IEdvQWNjZXNzLkFwcFRwbHMuUGFuZWxzLndyYXAucmVuZGVy
KEdvQWNjZXNzLlV0aWwubWVyZ2UodWksIHsnbGFiZWxzJzogR29BY2Nlc3MuaTE4bn0pKTtjb2wu
YXBwZW5kQ2hpbGQoYm94KTtpZiAoZGF0YS5kYXRhLmxlbmd0aCA8PSBHb0FjY2Vzcy5nZXRQcmVm
cygpLnBlclBhZ2UpdGhpcy5kaXNhYmxlUGFnaW5hdGlvbihwYW5lbCk7R29BY2Nlc3MuVGFibGVz
LnJlbmRlclRoZWFkKHBhbmVsLCB1aSk7cmV0dXJuIGNvbDt9LGNyZWF0ZUNvbDogZnVuY3Rpb24g
KHJvdykge3ZhciBsYXlvdXQgPSBHb0FjY2Vzcy5BcHBQcmVmc1snbGF5b3V0J107dmFyIHBlclJv
dyA9ICdob3Jpem9udGFsJyA9PSBsYXlvdXQgPyA2IDogJ3dpZGUnID09IGxheW91dCA/IDMgOiAx
Mjt2YXIgY29sID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnZGl2Jyk7Y29sLnNldEF0dHJpYnV0
ZSgnY2xhc3MnLCAnY29sLW1kLScgKyBwZXJSb3cgKyAnIHdyYXAtcGFuZWwnKTtyb3cuYXBwZW5k
Q2hpbGQoY29sKTtyZXR1cm4gY29sO30sY3JlYXRlUm93OiBmdW5jdGlvbiAocm93LCBpZHgpIHt2
YXIgd3JhcCA9ICQoJy53cmFwLXBhbmVscycpO3ZhciBsYXlvdXQgPSBHb0FjY2Vzcy5BcHBQcmVm
c1snbGF5b3V0J107dmFyIGV2ZXJ5ID0gJ2hvcml6b250YWwnID09IGxheW91dCA/IDIgOiAnd2lk
ZScgPT0gbGF5b3V0ID8gNCA6IDE7aWYgKGlkeCAlIGV2ZXJ5ID09IDApIHtyb3cgPSBkb2N1bWVu
dC5jcmVhdGVFbGVtZW50KCdkaXYnKTtyb3cuc2V0QXR0cmlidXRlKCdjbGFzcycsICdyb3cnICsg
KGV2ZXJ5ID09IDIgfHwgZXZlcnkgPT0gNCA/ICcgZXF1YWwnIDogJycpKTt3cmFwLmFwcGVuZENo
aWxkKHJvdyk7fXJldHVybiByb3c7fSxyZXNldFBhbmVsOiBmdW5jdGlvbiAocGFuZWwpIHt2YXIg
dWkgPSBHb0FjY2Vzcy5nZXRQYW5lbFVJKCk7dmFyIGVsZSA9ICQoJyNwYW5lbC0nICsgcGFuZWwp
O2lmIChHb0FjY2Vzcy5VdGlsLmlzUGFuZWxWYWxpZChwYW5lbCkgfHwgR29BY2Nlc3MuVXRpbC5p
c1BhbmVsSGlkZGVuKHBhbmVsKSlyZXR1cm4gZmFsc2U7dmFyIGNvbCA9IGVsZS5wYXJlbnROb2Rl
O2NvbC5yZW1vdmVDaGlsZChlbGUpO3RoaXMucmVuZGVyUGFuZWwocGFuZWwsIHVpW3BhbmVsXSwg
Y29sKTt0aGlzLmV2ZW50cygpO30scmVuZGVyUGFuZWxzOiBmdW5jdGlvbiAoKSB7dmFyIHVpID0g
R29BY2Nlc3MuZ2V0UGFuZWxVSSgpLCBpZHggPSAwLCByb3cgPSBudWxsLCBjb2wgPSBudWxsOyQo
Jy53cmFwLXBhbmVscycpLmlubmVySFRNTCA9ICcnO2ZvciAodmFyIHBhbmVsIGluIHVpKSB7aWYg
KEdvQWNjZXNzLlV0aWwuaXNQYW5lbFZhbGlkKHBhbmVsKSB8fCBHb0FjY2Vzcy5VdGlsLmlzUGFu
ZWxIaWRkZW4ocGFuZWwpKWNvbnRpbnVlO3JvdyA9IHRoaXMuY3JlYXRlUm93KHJvdywgaWR4Kysp
O2NvbCA9IHRoaXMuY3JlYXRlQ29sKHJvdyk7dGhpcy5yZW5kZXJQYW5lbChwYW5lbCwgdWlbcGFu
ZWxdLCBjb2wpO319LGluaXRpYWxpemU6IGZ1bmN0aW9uICgpIHt0aGlzLnJlbmRlclBhbmVscygp
O3RoaXMuZXZlbnRzKCk7fX07R29BY2Nlc3MuQ2hhcnRzID0ge2l0ZXI6IGZ1bmN0aW9uIChjYWxs
YmFjaykge09iamVjdC5rZXlzKEdvQWNjZXNzLkFwcENoYXJ0cykuZm9yRWFjaChmdW5jdGlvbiAo
cGFuZWwpIHtpZiAoIUdvQWNjZXNzLlV0aWwuaXNXaXRoaW5WaWV3UG9ydCgkKCcjcGFuZWwtJyAr
IHBhbmVsKSkpcmV0dXJuO2lmIChjYWxsYmFjayAmJiB0eXBlb2YgY2FsbGJhY2sgPT09ICdmdW5j
dGlvbicpY2FsbGJhY2suY2FsbCh0aGlzLCBHb0FjY2Vzcy5BcHBDaGFydHNbcGFuZWxdLCBwYW5l
bCk7fSk7fSxnZXRNZXRyaWNLZXlzOiBmdW5jdGlvbiAocGFuZWwsIGtleSkge3JldHVybiBHb0Fj
Y2Vzcy5nZXRQYW5lbFVJKHBhbmVsKVsnaXRlbXMnXS5tYXAoZnVuY3Rpb24gKGEpIHsgcmV0dXJu
IGFba2V5XTsgfSk7fSxnZXRQYW5lbERhdGE6IGZ1bmN0aW9uIChwYW5lbCwgZGF0YSkge3ZhciBw
bG90ID0gR29BY2Nlc3MuVXRpbC5nZXRQcm9wKEdvQWNjZXNzLkFwcFN0YXRlLCBwYW5lbCArICcu
cGxvdCcpO2RhdGEgPSBkYXRhIHx8IHRoaXMucHJvY2Vzc0NoYXJ0RGF0YShHb0FjY2Vzcy5nZXRQ
YW5lbERhdGEocGFuZWwpLmRhdGEpO3JldHVybiBwbG90LmNoYXJ0UmV2ZXJzZSA/IGRhdGEucmV2
ZXJzZSgpIDogZGF0YTt9LGRyYXdQbG90OiBmdW5jdGlvbiAocGFuZWwsIHBsb3RVSSwgZGF0YSkg
e3ZhciBjaGFydCA9IHRoaXMuZ2V0Q2hhcnQocGFuZWwsIHBsb3RVSSwgZGF0YSk7aWYgKCFjaGFy
dClyZXR1cm47dGhpcy5yZW5kZXJDaGFydChwYW5lbCwgY2hhcnQsIGRhdGEpO0dvQWNjZXNzLkFw
cENoYXJ0c1twYW5lbF0gPSBudWxsO0dvQWNjZXNzLkFwcENoYXJ0c1twYW5lbF0gPSBjaGFydDt9
LHNldENoYXJ0VHlwZTogZnVuY3Rpb24gKHRhcmcpIHt2YXIgcGFuZWwgPSB0YXJnLmdldEF0dHJp
YnV0ZSgnZGF0YS1wYW5lbCcpO3ZhciB0eXBlID0gdGFyZy5nZXRBdHRyaWJ1dGUoJ2RhdGEtY2hh
cnQtdHlwZScpO0dvQWNjZXNzLlV0aWwuc2V0UHJvcChHb0FjY2Vzcy5BcHBQcmVmcywgcGFuZWwg
KyAnLnBsb3QuY2hhcnRUeXBlJywgdHlwZSk7R29BY2Nlc3Muc2V0UHJlZnMoKTt2YXIgcGxvdFVJ
ID0gR29BY2Nlc3MuVXRpbC5nZXRQcm9wKEdvQWNjZXNzLkFwcFN0YXRlLCBwYW5lbCArICcucGxv
dCcpO3RoaXMuZHJhd1Bsb3QocGFuZWwsIHBsb3RVSSwgdGhpcy5nZXRQYW5lbERhdGEocGFuZWwp
KTt9LHRvZ2dsZUNoYXJ0OiBmdW5jdGlvbiAodGFyZykge3ZhciBwYW5lbCA9IHRhcmcuZ2V0QXR0
cmlidXRlKCdkYXRhLXBhbmVsJyk7dmFyIHByZWZzID0gR29BY2Nlc3MuZ2V0UHJlZnMocGFuZWwp
LGNoYXJ0ID0gcHJlZnMgJiYgKCdjaGFydCcgaW4gcHJlZnMpID8gcHJlZnMuY2hhcnQgOiB0cnVl
O0dvQWNjZXNzLlV0aWwuc2V0UHJvcChHb0FjY2Vzcy5BcHBQcmVmcywgcGFuZWwgKyAnLmNoYXJ0
JywgIWNoYXJ0KTtHb0FjY2Vzcy5zZXRQcmVmcygpO0dvQWNjZXNzLlBhbmVscy5yZXNldFBhbmVs
KHBhbmVsKTtHb0FjY2Vzcy5DaGFydHMucmVzZXRDaGFydChwYW5lbCk7R29BY2Nlc3MuVGFibGVz
LnJlbmRlckZ1bGxUYWJsZShwYW5lbCk7fSxoYXNDaGFydDogZnVuY3Rpb24gKHBhbmVsLCB1aSkg
e3ZhciBwcmVmcyA9IEdvQWNjZXNzLmdldFByZWZzKHBhbmVsKSxjaGFydCA9IHByZWZzICYmICgn
Y2hhcnQnIGluIHByZWZzKSA/IHByZWZzLmNoYXJ0IDogdHJ1ZTt1aVsnY2hhcnQnXSA9IHVpLnBs
b3QubGVuZ3RoICYmIGNoYXJ0ICYmIGNoYXJ0O30scmVkcmF3Q2hhcnQ6IGZ1bmN0aW9uICh0YXJn
KSB7dmFyIHBsb3QgPSB0YXJnLmdldEF0dHJpYnV0ZSgnZGF0YS1wbG90Jyk7dmFyIHBhbmVsID0g
dGFyZy5nZXRBdHRyaWJ1dGUoJ2RhdGEtcGFuZWwnKTt2YXIgdWkgPSBHb0FjY2Vzcy5nZXRQYW5l
bFVJKHBhbmVsKTt2YXIgcGxvdFVJID0gdWkucGxvdDtHb0FjY2Vzcy5VdGlsLnNldFByb3AoR29B
Y2Nlc3MuQXBwUHJlZnMsIHBhbmVsICsgJy5wbG90Lm1ldHJpYycsIHBsb3QpO0dvQWNjZXNzLnNl
dFByZWZzKCk7Zm9yICh2YXIgeCBpbiBwbG90VUkpIHtpZiAoIXBsb3RVSS5oYXNPd25Qcm9wZXJ0
eSh4KSB8fCBwbG90VUlbeF0uY2xhc3NOYW1lICE9IHBsb3QpY29udGludWU7R29BY2Nlc3MuVXRp
bC5zZXRQcm9wKEdvQWNjZXNzLkFwcFN0YXRlLCBwYW5lbCArICcucGxvdCcsIHBsb3RVSVt4XSk7
dGhpcy5kcmF3UGxvdChwYW5lbCwgcGxvdFVJW3hdLCB0aGlzLmdldFBhbmVsRGF0YShwYW5lbCkp
O2JyZWFrO319LGV4dHJhY3RDb3VudDogZnVuY3Rpb24gKGl0ZW0pIHt2YXIgbyA9IHt9O2ZvciAo
dmFyIHByb3AgaW4gaXRlbSlvW3Byb3BdID0gR29BY2Nlc3MuVXRpbC5nZXRDb3VudChpdGVtW3By
b3BdKTtyZXR1cm4gbzt9LHByb2Nlc3NDaGFydERhdGE6IGZ1bmN0aW9uIChkYXRhKSB7dmFyIG91
dCA9IFtdO2ZvciAodmFyIGkgPSAwOyBpIDwgZGF0YS5sZW5ndGg7ICsraSlvdXQucHVzaCh0aGlz
LmV4dHJhY3RDb3VudChkYXRhW2ldKSk7cmV0dXJuIG91dDt9LGZpbmRVSUl0ZW06IGZ1bmN0aW9u
IChwYW5lbCwga2V5KSB7dmFyIGl0ZW1zID0gR29BY2Nlc3MuZ2V0UGFuZWxVSShwYW5lbCkuaXRl
bXM7Zm9yICh2YXIgaSA9IDA7IGkgPCBpdGVtcy5sZW5ndGg7ICsraSkge2lmIChpdGVtc1tpXS5r
ZXkgPT0ga2V5KXJldHVybiBpdGVtc1tpXTt9cmV0dXJuIG51bGw7fSxnZXRYS2V5OiBmdW5jdGlv
biAoZGF0dW0sIGtleSkge3ZhciBhcnIgPSBbXTtpZiAodHlwZW9mIGtleSA9PT0gJ3N0cmluZycp
cmV0dXJuIGRhdHVtW2tleV07Zm9yICh2YXIgcHJvcCBpbiBrZXkpYXJyLnB1c2goZGF0dW1ba2V5
W3Byb3BdXSk7cmV0dXJuIGFyci5qb2luKCcgJyk7fSxnZXRBcmVhU3BsaW5lOiBmdW5jdGlvbiAo
cGFuZWwsIHBsb3RVSSwgZGF0YSkge3ZhciBkdWFsWWF4aXMgPSBwbG90VUlbJ2QzJ11bJ3kxJ107
dmFyIGNoYXJ0ID0gQXJlYUNoYXJ0KGR1YWxZYXhpcykubGFiZWxzKHt5MDogcGxvdFVJWydkMydd
Wyd5MCddLmxhYmVsLHkxOiBkdWFsWWF4aXMgPyBwbG90VUlbJ2QzJ11bJ3kxJ10ubGFiZWwgOiAn
J30pLngoZnVuY3Rpb24gKGQpIHtpZiAoKCgocGxvdFVJIHx8IHt9KS5kMyB8fCB7fSkueCB8fCB7
fSkua2V5KXJldHVybiB0aGlzLmdldFhLZXkoZCwgcGxvdFVJWydkMyddWyd4J11bJ2tleSddKTty
ZXR1cm4gZC5kYXRhO30uYmluZCh0aGlzKSkueTAoZnVuY3Rpb24gKGQpIHtyZXR1cm4gK2RbcGxv
dFVJWydkMyddWyd5MCddWydrZXknXV07fSkud2lkdGgoJCgiI2NoYXJ0LSIgKyBwYW5lbCkuZ2V0
Qm91bmRpbmdDbGllbnRSZWN0KCkud2lkdGgpLmhlaWdodCgxNzUpLmZvcm1hdCh7eDogKHRoaXMu
ZmluZFVJSXRlbShwYW5lbCwgJ2RhdGEnKSB8fCB7fSkuZGF0YVR5cGUgfHwgbnVsbCx5MDogKChw
bG90VUkuZDMgfHwge30pLnkwIHx8IHt9KS5mb3JtYXQseTE6ICgocGxvdFVJLmQzIHx8IHt9KS55
MSB8fCB7fSkuZm9ybWF0LH0pLm9wdHMocGxvdFVJKTtkdWFsWWF4aXMgJiYgY2hhcnQueTEoZnVu
Y3Rpb24gKGQpIHtyZXR1cm4gK2RbcGxvdFVJWydkMyddWyd5MSddWydrZXknXV07fSk7cmV0dXJu
IGNoYXJ0O30sZ2V0VkJhcjogZnVuY3Rpb24gKHBhbmVsLCBwbG90VUksIGRhdGEpIHt2YXIgZHVh
bFlheGlzID0gcGxvdFVJWydkMyddWyd5MSddO3ZhciBjaGFydCA9IEJhckNoYXJ0KGR1YWxZYXhp
cykubGFiZWxzKHt5MDogcGxvdFVJWydkMyddWyd5MCddLmxhYmVsLHkxOiBkdWFsWWF4aXMgPyBw
bG90VUlbJ2QzJ11bJ3kxJ10ubGFiZWwgOiAnJ30pLngoZnVuY3Rpb24gKGQpIHtpZiAoKCgocGxv
dFVJIHx8IHt9KS5kMyB8fCB7fSkueCB8fCB7fSkua2V5KXJldHVybiB0aGlzLmdldFhLZXkoZCwg
cGxvdFVJWydkMyddWyd4J11bJ2tleSddKTtyZXR1cm4gZC5kYXRhO30uYmluZCh0aGlzKSkueTAo
ZnVuY3Rpb24gKGQpIHtyZXR1cm4gK2RbcGxvdFVJWydkMyddWyd5MCddWydrZXknXV07fSkud2lk
dGgoJCgiI2NoYXJ0LSIgKyBwYW5lbCkuZ2V0Qm91bmRpbmdDbGllbnRSZWN0KCkud2lkdGgpLmhl
aWdodCgxNzUpLmZvcm1hdCh7eDogKHRoaXMuZmluZFVJSXRlbShwYW5lbCwgJ2RhdGEnKSB8fCB7
fSkuZGF0YVR5cGUgfHwgbnVsbCx5MDogKChwbG90VUkuZDMgfHwge30pLnkwIHx8IHt9KS5mb3Jt
YXQseTE6ICgocGxvdFVJLmQzIHx8IHt9KS55MSB8fCB7fSkuZm9ybWF0LH0pLm9wdHMocGxvdFVJ
KTtkdWFsWWF4aXMgJiYgY2hhcnQueTEoZnVuY3Rpb24gKGQpIHtyZXR1cm4gK2RbcGxvdFVJWydk
MyddWyd5MSddWydrZXknXV07fSk7cmV0dXJuIGNoYXJ0O30sZ2V0Q2hhcnRUeXBlOiBmdW5jdGlv
biAocGFuZWwpIHt2YXIgdWkgPSBHb0FjY2Vzcy5nZXRQYW5lbFVJKHBhbmVsKTtpZiAoIXVpLmNo
YXJ0KXJldHVybiAnJztyZXR1cm4gR29BY2Nlc3MuVXRpbC5nZXRQcm9wKEdvQWNjZXNzLmdldFBy
ZWZzKCksIHBhbmVsICsgJy5wbG90LmNoYXJ0VHlwZScpIHx8IHVpLnBsb3RbMF0uY2hhcnRUeXBl
O30sZ2V0UGxvdFVJOiBmdW5jdGlvbiAocGFuZWwsIHVpKSB7dmFyIG1ldHJpYyA9IEdvQWNjZXNz
LlV0aWwuZ2V0UHJvcChHb0FjY2Vzcy5nZXRQcmVmcygpLCBwYW5lbCArICcucGxvdC5tZXRyaWMn
KTtpZiAoIW1ldHJpYylyZXR1cm4gdWkucGxvdFswXTtyZXR1cm4gdWkucGxvdC5maWx0ZXIoZnVu
Y3Rpb24gKHYpIHtyZXR1cm4gdi5jbGFzc05hbWUgPT0gbWV0cmljO30pWzBdO30sZ2V0Q2hhcnQ6
IGZ1bmN0aW9uIChwYW5lbCwgcGxvdFVJLCBkYXRhKSB7dmFyIGNoYXJ0ID0gbnVsbDtzd2l0Y2gg
KHRoaXMuZ2V0Q2hhcnRUeXBlKHBhbmVsKSkge2Nhc2UgJ2FyZWEtc3BsaW5lJzpjaGFydCA9IHRo
aXMuZ2V0QXJlYVNwbGluZShwYW5lbCwgcGxvdFVJLCBkYXRhKTticmVhaztjYXNlICdiYXInOmNo
YXJ0ID0gdGhpcy5nZXRWQmFyKHBhbmVsLCBwbG90VUksIGRhdGEpO2JyZWFrO31yZXR1cm4gY2hh
cnQ7fSxyZW5kZXJDaGFydDogZnVuY3Rpb24gKHBhbmVsLCBjaGFydCwgZGF0YSkge2QzLnNlbGVj
dCgnI2NoYXJ0LScgKyBwYW5lbCArICc+LmNoYXJ0LXRvb2x0aXAtd3JhcCcpLnJlbW92ZSgpO2Qz
LnNlbGVjdCgnI2NoYXJ0LScgKyBwYW5lbCkuc2VsZWN0KCdzdmcnKS5yZW1vdmUoKTtkMy5zZWxl
Y3QoIiNjaGFydC0iICsgcGFuZWwpLmRhdHVtKGRhdGEpLmNhbGwoY2hhcnQpLmFwcGVuZCgiZGl2
IikuYXR0cigiY2xhc3MiLCAiY2hhcnQtdG9vbHRpcC13cmFwIik7fSxhZGRDaGFydDogZnVuY3Rp
b24gKHBhbmVsLCB1aSkge3ZhciBwbG90VUkgPSBudWxsLCBjaGFydCA9IG51bGw7aWYgKCF1aS5w
bG90IHx8ICF1aS5wbG90Lmxlbmd0aClyZXR1cm47cGxvdFVJID0gdGhpcy5nZXRQbG90VUkocGFu
ZWwsIHVpKTtHb0FjY2Vzcy5VdGlsLnNldFByb3AoR29BY2Nlc3MuQXBwU3RhdGUsIHBhbmVsICsg
Jy5wbG90JywgcGxvdFVJKTt2YXIgZGF0YSA9IHRoaXMuZ2V0UGFuZWxEYXRhKHBhbmVsKTtpZiAo
IShjaGFydCA9IHRoaXMuZ2V0Q2hhcnQocGFuZWwsIHBsb3RVSSwgZGF0YSkpKXJldHVybjt0aGlz
LnJlbmRlckNoYXJ0KHBhbmVsLCBjaGFydCwgZGF0YSk7R29BY2Nlc3MuQXBwQ2hhcnRzW3BhbmVs
XSA9IGNoYXJ0O30scmVuZGVyQ2hhcnRzOiBmdW5jdGlvbiAodWkpIHtmb3IgKHZhciBwYW5lbCBp
biB1aSkge2lmIChHb0FjY2Vzcy5VdGlsLmlzUGFuZWxWYWxpZChwYW5lbCkgfHwgR29BY2Nlc3Mu
VXRpbC5pc1BhbmVsSGlkZGVuKHBhbmVsKSljb250aW51ZTt0aGlzLmFkZENoYXJ0KHBhbmVsLCB1
aVtwYW5lbF0pO319LHJlc2V0Q2hhcnQ6IGZ1bmN0aW9uIChwYW5lbCkge3ZhciB1aSA9IHt9O2lm
IChHb0FjY2Vzcy5VdGlsLmlzUGFuZWxWYWxpZChwYW5lbCkgfHwgR29BY2Nlc3MuVXRpbC5pc1Bh
bmVsSGlkZGVuKHBhbmVsKSlyZXR1cm4gZmFsc2U7dWkgPSBHb0FjY2Vzcy5nZXRQYW5lbFVJKHBh
bmVsKTt0aGlzLmFkZENoYXJ0KHBhbmVsLCB1aSk7fSxyZWxvYWRDaGFydDogZnVuY3Rpb24gKGNo
YXJ0LCBwYW5lbCkge3ZhciBzdWJJdGVtcyA9IEdvQWNjZXNzLlRhYmxlcy5nZXRTdWJJdGVtc0Rh
dGEocGFuZWwpO3ZhciBkYXRhID0gKHN1Ykl0ZW1zLmxlbmd0aCA/IHN1Ykl0ZW1zIDogR29BY2Nl
c3MuZ2V0UGFuZWxEYXRhKHBhbmVsKS5kYXRhKS5zbGljZSgwKTtkMy5zZWxlY3QoIiNjaGFydC0i
ICsgcGFuZWwpLmRhdHVtKHRoaXMucHJvY2Vzc0NoYXJ0RGF0YSh0aGlzLmdldFBhbmVsRGF0YShw
YW5lbCwgZGF0YSkpKS5jYWxsKGNoYXJ0LndpZHRoKCQoIiNjaGFydC0iICsgcGFuZWwpLm9mZnNl
dFdpZHRoKSkuYXBwZW5kKCJkaXYiKS5hdHRyKCJjbGFzcyIsICJjaGFydC10b29sdGlwLXdyYXAi
KTt9LHJlbG9hZENoYXJ0czogZnVuY3Rpb24gKCkge3RoaXMuaXRlcihmdW5jdGlvbiAoY2hhcnQs
IHBhbmVsKSB7dGhpcy5yZWxvYWRDaGFydChjaGFydCwgcGFuZWwpO30uYmluZCh0aGlzKSk7R29B
Y2Nlc3MuQXBwU3RhdGUudXBkYXRlZCA9IGZhbHNlO30scmVkcmF3Q2hhcnRzOiBmdW5jdGlvbiAo
KSB7dGhpcy5pdGVyKGZ1bmN0aW9uIChjaGFydCwgcGFuZWwpIHtkMy5zZWxlY3QoIiNjaGFydC0i
ICsgcGFuZWwpLmNhbGwoY2hhcnQud2lkdGgoJCgiI2NoYXJ0LSIgKyBwYW5lbCkub2Zmc2V0V2lk
dGgpKTt9KTt9LGluaXRpYWxpemU6IGZ1bmN0aW9uICgpIHt0aGlzLnJlbmRlckNoYXJ0cyhHb0Fj
Y2Vzcy5nZXRQYW5lbFVJKCkpO2QzLnNlbGVjdCh3aW5kb3cpLm9uKCdzY3JvbGwuY2hhcnRzJywg
ZGVib3VuY2UoZnVuY3Rpb24gKCkge3RoaXMucmVsb2FkQ2hhcnRzKCk7fSwgMjUwLCBmYWxzZSku
YmluZCh0aGlzKSkub24oJ3Jlc2l6ZS5jaGFydHMnLCBmdW5jdGlvbiAoKSB7dGhpcy5yZWRyYXdD
aGFydHMoKTt9LmJpbmQodGhpcykpO319O0dvQWNjZXNzLlRhYmxlcyA9IHtjaGFydERhdGE6IHt9
LGV2ZW50czogZnVuY3Rpb24gKCkgeyQkKCcucGFuZWwtbmV4dCcsIGZ1bmN0aW9uIChpdGVtKSB7
aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt2YXIgcGFuZWwgPSBlLmN1cnJlbnRUYXJnZXQu
Z2V0QXR0cmlidXRlKCdkYXRhLXBhbmVsJyk7dGhpcy5yZW5kZXJUYWJsZShwYW5lbCwgdGhpcy5u
ZXh0UGFnZShwYW5lbCkpO30uYmluZCh0aGlzKTt9LmJpbmQodGhpcykpOyQkKCcucGFuZWwtcHJl
dicsIGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt2YXIgcGFu
ZWwgPSBlLmN1cnJlbnRUYXJnZXQuZ2V0QXR0cmlidXRlKCdkYXRhLXBhbmVsJyk7dGhpcy5yZW5k
ZXJUYWJsZShwYW5lbCwgdGhpcy5wcmV2UGFnZShwYW5lbCkpO30uYmluZCh0aGlzKTt9LmJpbmQo
dGhpcykpOyQkKCcucGFuZWwtZmlyc3QnLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9
IGZ1bmN0aW9uIChlKSB7dmFyIHBhbmVsID0gZS5jdXJyZW50VGFyZ2V0LmdldEF0dHJpYnV0ZSgn
ZGF0YS1wYW5lbCcpO3RoaXMucmVuZGVyVGFibGUocGFuZWwsICJGSVJTVF9QQUdFIik7fS5iaW5k
KHRoaXMpO30uYmluZCh0aGlzKSk7JCQoJy5wYW5lbC1sYXN0JywgZnVuY3Rpb24gKGl0ZW0pIHtp
dGVtLm9uY2xpY2sgPSBmdW5jdGlvbiAoZSkge3ZhciBwYW5lbCA9IGUuY3VycmVudFRhcmdldC5n
ZXRBdHRyaWJ1dGUoJ2RhdGEtcGFuZWwnKTt0aGlzLnJlbmRlclRhYmxlKHBhbmVsLCAiTEFTVF9Q
QUdFIik7fS5iaW5kKHRoaXMpO30uYmluZCh0aGlzKSk7JCQoJy5leHBhbmRhYmxlPnRkJywgZnVu
Y3Rpb24gKGl0ZW0pIHtpdGVtLm9uY2xpY2sgPSBmdW5jdGlvbiAoZSkge2lmICghd2luZG93Lmdl
dFNlbGVjdGlvbigpLnRvU3RyaW5nKCkpdGhpcy50b2dnbGVSb3coZS5jdXJyZW50VGFyZ2V0KTt9
LmJpbmQodGhpcyk7fS5iaW5kKHRoaXMpKTskJCgnLnJvdy1leHBhbmRhYmxlLmNsaWNrYWJsZScs
IGZ1bmN0aW9uIChpdGVtKSB7aXRlbS5vbmNsaWNrID0gZnVuY3Rpb24gKGUpIHt0aGlzLnRvZ2ds
ZVJvdyhlLmN1cnJlbnRUYXJnZXQpO30uYmluZCh0aGlzKTt9LmJpbmQodGhpcykpOyQkKCcuc29y
dGFibGUnLCBmdW5jdGlvbiAoaXRlbSkge2l0ZW0ub25jbGljayA9IGZ1bmN0aW9uIChlKSB7dGhp
cy5zb3J0Q29sdW1uKGUuY3VycmVudFRhcmdldCk7fS5iaW5kKHRoaXMpO30uYmluZCh0aGlzKSk7
fSx0b2dnbGVDb2x1bW46IGZ1bmN0aW9uICh0YXJnKSB7dmFyIHBhbmVsID0gdGFyZy5nZXRBdHRy
aWJ1dGUoJ2RhdGEtcGFuZWwnKTt2YXIgbWV0cmljID0gdGFyZy5nZXRBdHRyaWJ1dGUoJ2RhdGEt
bWV0cmljJyk7dmFyIGNvbHVtbnMgPSAoR29BY2Nlc3MuZ2V0UHJlZnMocGFuZWwpIHx8IHt9KS5j
b2x1bW5zIHx8IHt9O2lmIChtZXRyaWMgaW4gY29sdW1ucykge2RlbGV0ZSBjb2x1bW5zW21ldHJp
Y107fSBlbHNlIHtHb0FjY2Vzcy5VdGlsLnNldFByb3AoY29sdW1ucywgbWV0cmljICsgJy5oaWRl
JywgdHJ1ZSk7fUdvQWNjZXNzLlV0aWwuc2V0UHJvcChHb0FjY2Vzcy5BcHBQcmVmcywgcGFuZWwg
KyAnLmNvbHVtbnMnLCBjb2x1bW5zKTtHb0FjY2Vzcy5zZXRQcmVmcygpO0dvQWNjZXNzLlRhYmxl
cy5yZW5kZXJUaGVhZChwYW5lbCwgR29BY2Nlc3MuZ2V0UGFuZWxVSShwYW5lbCkpO0dvQWNjZXNz
LlRhYmxlcy5yZW5kZXJGdWxsVGFibGUocGFuZWwpO30sc29ydENvbHVtbjogZnVuY3Rpb24gKGVs
ZSkge3ZhciBmaWVsZCA9IGVsZS5nZXRBdHRyaWJ1dGUoJ2RhdGEta2V5Jyk7dmFyIG9yZGVyID0g
ZWxlLmdldEF0dHJpYnV0ZSgnZGF0YS1vcmRlcicpO3ZhciBwYW5lbCA9IGVsZS5wYXJlbnRFbGVt
ZW50LnBhcmVudEVsZW1lbnQucGFyZW50RWxlbWVudC5nZXRBdHRyaWJ1dGUoJ2RhdGEtcGFuZWwn
KTtvcmRlciA9IG9yZGVyID8gJ2FzYycgPT0gb3JkZXIgPyAnZGVzYycgOiAnYXNjJyA6ICdhc2Mn
O0dvQWNjZXNzLkFwcC5zb3J0RGF0YShwYW5lbCwgZmllbGQsIG9yZGVyKTtHb0FjY2Vzcy5VdGls
LnNldFByb3AoR29BY2Nlc3MuQXBwU3RhdGUsIHBhbmVsICsgJy5zb3J0JywgeydmaWVsZCc6IGZp
ZWxkLCdvcmRlcic6IG9yZGVyLH0pO3RoaXMucmVuZGVyVGhlYWQocGFuZWwsIEdvQWNjZXNzLmdl
dFBhbmVsVUkocGFuZWwpKTt0aGlzLnJlbmRlclRhYmxlKHBhbmVsLCB0aGlzLmdldEN1clBhZ2Uo
cGFuZWwpKTtHb0FjY2Vzcy5DaGFydHMucmVsb2FkQ2hhcnQoR29BY2Nlc3MuQXBwQ2hhcnRzW3Bh
bmVsXSwgcGFuZWwpO30sZ2V0RGF0YUJ5S2V5OiBmdW5jdGlvbiAocGFuZWwsIGtleSkge3ZhciBk
YXRhID0gR29BY2Nlc3MuZ2V0UGFuZWxEYXRhKHBhbmVsKS5kYXRhO2ZvciAodmFyIGkgPSAwLCBu
ID0gZGF0YS5sZW5ndGg7IGkgPCBuOyArK2kpIHtpZiAoR29BY2Nlc3MuVXRpbC5oYXNoQ29kZShk
YXRhW2ldLmRhdGEpID09IGtleSlyZXR1cm4gZGF0YVtpXTt9cmV0dXJuIG51bGw7fSxnZXRTdWJJ
dGVtc0RhdGE6IGZ1bmN0aW9uIChwYW5lbCkge3ZhciBvdXQgPSBbXSwgaXRlbXMgPSB0aGlzLmNo
YXJ0RGF0YVtwYW5lbF07Zm9yICh2YXIgeCBpbiBpdGVtcykge2lmICghaXRlbXMuaGFzT3duUHJv
cGVydHkoeCkpY29udGludWU7b3V0ID0gb3V0LmNvbmNhdChpdGVtc1t4XSk7fXJldHVybiBvdXQ7
fSxhZGRDaGFydERhdGE6IGZ1bmN0aW9uIChwYW5lbCwga2V5KSB7dmFyIGRhdGEgPSB0aGlzLmdl
dERhdGFCeUtleShwYW5lbCwga2V5KTt2YXIgcGF0aCA9IHBhbmVsICsgJy4nICsga2V5O2lmICgh
ZGF0YSB8fCAhZGF0YS5pdGVtcylyZXR1cm4gW107R29BY2Nlc3MuVXRpbC5zZXRQcm9wKHRoaXMu
Y2hhcnREYXRhLCBwYXRoLCBkYXRhLml0ZW1zKTtyZXR1cm4gdGhpcy5nZXRTdWJJdGVtc0RhdGEo
cGFuZWwpO30scmVtb3ZlQ2hhcnREYXRhOiBmdW5jdGlvbiAocGFuZWwsIGtleSkge2lmIChHb0Fj
Y2Vzcy5VdGlsLmdldFByb3AodGhpcy5jaGFydERhdGEsIHBhbmVsICsgJy4nICsga2V5KSlkZWxl
dGUgdGhpcy5jaGFydERhdGFbcGFuZWxdW2tleV07aWYgKCF0aGlzLmNoYXJ0RGF0YVtwYW5lbF0g
fHwgT2JqZWN0LmtleXModGhpcy5jaGFydERhdGFbcGFuZWxdKS5sZW5ndGggPT0gMClyZXR1cm4g
R29BY2Nlc3MuZ2V0UGFuZWxEYXRhKHBhbmVsKS5kYXRhO3JldHVybiB0aGlzLmdldFN1Ykl0ZW1z
RGF0YShwYW5lbCk7fSxpc0V4cGFuZGVkOiBmdW5jdGlvbiAocGFuZWwsIGtleSkge3ZhciBwYXRo
ID0gcGFuZWwgKyAnLmV4cGFuZGVkLicgKyBrZXk7cmV0dXJuIEdvQWNjZXNzLlV0aWwuZ2V0UHJv
cChHb0FjY2Vzcy5BcHBTdGF0ZSwgcGF0aCk7fSx0b2dnbGVFeHBhbmRlZDogZnVuY3Rpb24gKHBh
bmVsLCBrZXkpIHt2YXIgcGF0aCA9IHBhbmVsICsgJy5leHBhbmRlZC4nICsga2V5LCByZXQgPSB0
cnVlO2lmICh0aGlzLmlzRXhwYW5kZWQocGFuZWwsIGtleSkpIHtkZWxldGUgR29BY2Nlc3MuQXBw
U3RhdGVbcGFuZWxdWydleHBhbmRlZCddW2tleV07fSBlbHNlIHtHb0FjY2Vzcy5VdGlsLnNldFBy
b3AoR29BY2Nlc3MuQXBwU3RhdGUsIHBhdGgsIHRydWUpLCByZXQgPSBmYWxzZTt9cmV0dXJuIHJl
dDt9LHRvZ2dsZVJvdzogZnVuY3Rpb24gKGVsZSkge3ZhciBoaWRlID0gZmFsc2UsIGRhdGEgPSBb
XTt2YXIgcm93ID0gZWxlLnBhcmVudE5vZGU7dmFyIHBhbmVsID0gcm93LmdldEF0dHJpYnV0ZSgn
ZGF0YS1wYW5lbCcpLCBrZXkgPSByb3cuZ2V0QXR0cmlidXRlKCdkYXRhLWtleScpO3ZhciBwbG90
VUkgPSBHb0FjY2Vzcy5BcHBDaGFydHNbcGFuZWxdLm9wdHMoKTtoaWRlID0gdGhpcy50b2dnbGVF
eHBhbmRlZChwYW5lbCwga2V5KTt0aGlzLnJlbmRlclRhYmxlKHBhbmVsLCB0aGlzLmdldEN1clBh
Z2UocGFuZWwpKTtpZiAoIXBsb3RVSS5yZWRyYXdPbkV4cGFuZClyZXR1cm47aWYgKCFoaWRlKSB7
ZGF0YSA9IEdvQWNjZXNzLkNoYXJ0cy5wcm9jZXNzQ2hhcnREYXRhKHRoaXMuYWRkQ2hhcnREYXRh
KHBhbmVsLCBrZXkpKTt9IGVsc2Uge2RhdGEgPSBHb0FjY2Vzcy5DaGFydHMucHJvY2Vzc0NoYXJ0
RGF0YSh0aGlzLnJlbW92ZUNoYXJ0RGF0YShwYW5lbCwga2V5KSk7fUdvQWNjZXNzLkNoYXJ0cy5k
cmF3UGxvdChwYW5lbCwgcGxvdFVJLCBkYXRhKTt9LGdldEN1clBhZ2U6IGZ1bmN0aW9uIChwYW5l
bCkge3JldHVybiBHb0FjY2Vzcy5VdGlsLmdldFByb3AoR29BY2Nlc3MuQXBwU3RhdGUsIHBhbmVs
ICsgJy5jdXJQYWdlJykgfHwgMDt9LHBhZ2VPZmZTZXQ6IGZ1bmN0aW9uIChwYW5lbCkge3JldHVy
biAoKHRoaXMuZ2V0Q3VyUGFnZShwYW5lbCkgLSAxKSAqIEdvQWNjZXNzLmdldFByZWZzKCkucGVy
UGFnZSk7fSxnZXRUb3RhbFBhZ2VzOiBmdW5jdGlvbiAoZGF0YUl0ZW1zKSB7cmV0dXJuIE1hdGgu
Y2VpbChkYXRhSXRlbXMubGVuZ3RoIC8gR29BY2Nlc3MuZ2V0UHJlZnMoKS5wZXJQYWdlKTt9LGdl
dFBhZ2U6IGZ1bmN0aW9uIChwYW5lbCwgZGF0YUl0ZW1zLCBwYWdlKSB7dmFyIHRvdGFsUGFnZXMg
PSB0aGlzLmdldFRvdGFsUGFnZXMoZGF0YUl0ZW1zKTtpZiAocGFnZSA8IDEpcGFnZSA9IDE7aWYg
KHBhZ2UgPiB0b3RhbFBhZ2VzKXBhZ2UgPSB0b3RhbFBhZ2VzO0dvQWNjZXNzLlV0aWwuc2V0UHJv
cChHb0FjY2Vzcy5BcHBTdGF0ZSwgcGFuZWwgKyAnLmN1clBhZ2UnLCBwYWdlKTt2YXIgc3RhcnQg
PSB0aGlzLnBhZ2VPZmZTZXQocGFuZWwpO3ZhciBlbmQgPSBzdGFydCArIEdvQWNjZXNzLmdldFBy
ZWZzKCkucGVyUGFnZTtyZXR1cm4gZGF0YUl0ZW1zLnNsaWNlKHN0YXJ0LCBlbmQpO30scHJldlBh
Z2U6IGZ1bmN0aW9uIChwYW5lbCkge3JldHVybiB0aGlzLmdldEN1clBhZ2UocGFuZWwpIC0gMTt9
LG5leHRQYWdlOiBmdW5jdGlvbiAocGFuZWwpIHtyZXR1cm4gdGhpcy5nZXRDdXJQYWdlKHBhbmVs
KSArIDE7fSxnZXRNZXRhQ2VsbDogZnVuY3Rpb24gKHVpLCBvLCBrZXkpIHt2YXIgdmFsID0gIG8g
JiYgKGtleSBpbiBvKSAmJiBvW2tleV0udmFsdWUgPyBvW2tleV0udmFsdWUgOiBudWxsO3ZhciBw
ZXJjID0gbyAmJiAgKGtleSBpbiBvKSAmJiBvW2tleV0ucGVyY2VudCA/IG9ba2V5XS5wZXJjZW50
IDogbnVsbDt2YXIgdnR5cGUgPSB1aS5tZXRhVHlwZSB8fCB1aS5kYXRhVHlwZTt2YXIgY2xhc3NO
YW1lID0gdWkuY2xhc3NOYW1lIHx8ICcnO2NsYXNzTmFtZSArPSAhWydzdHJpbmcnXS5pbmNsdWRl
cyh1aS5kYXRhVHlwZSkgPyAndGV4dC1yaWdodCcgOiAnJztyZXR1cm4geydjbGFzc05hbWUnOiBj
bGFzc05hbWUsJ3ZhbHVlJyAgICA6IHZhbCA/IEdvQWNjZXNzLlV0aWwuZm10VmFsdWUodmFsLCB2
dHlwZSkgOiBudWxsLCdwZXJjZW50JyAgOiBwZXJjLCd0aXRsZScgICAgOiB1aS5tZXRhLCdsYWJl
bCcgICAgOiB1aS5tZXRhTGFiZWwgfHwgbnVsbCx9O30saGlkZUNvbHVtbjogZnVuY3Rpb24gKHBh
bmVsLCBjb2wpIHt2YXIgY29sdW1ucyA9IChHb0FjY2Vzcy5nZXRQcmVmcyhwYW5lbCkgfHwge30p
LmNvbHVtbnMgfHwge307cmV0dXJuICgoY29sIGluIGNvbHVtbnMpICYmIGNvbHVtbnNbY29sXVsn
aGlkZSddKTt9LHNob3dUYWJsZXM6IGZ1bmN0aW9uICgpIHtyZXR1cm4gKCdzaG93VGFibGVzJyBp
biBHb0FjY2Vzcy5nZXRQcmVmcygpKSA/IEdvQWNjZXNzLmdldFByZWZzKCkuc2hvd1RhYmxlcyA6
IHRydWU7fSxhdXRvSGlkZVRhYmxlczogZnVuY3Rpb24gKCkge3JldHVybiAoJ2F1dG9IaWRlVGFi
bGVzJyBpbiBHb0FjY2Vzcy5nZXRQcmVmcygpKSA/IEdvQWNjZXNzLmdldFByZWZzKCkuYXV0b0hp
ZGVUYWJsZXMgOiB0cnVlO30saGFzVGFibGU6IGZ1bmN0aW9uICh1aSkge3VpWyd0YWJsZSddID0g
dGhpcy5zaG93VGFibGVzKCk7dWlbJ2F1dG9IaWRlVGFibGVzJ10gPSB0aGlzLmF1dG9IaWRlVGFi
bGVzKCk7fSxnZXRNZXRhUm93czogZnVuY3Rpb24gKHBhbmVsLCB1aSwga2V5KSB7dmFyIGNlbGxz
ID0gW10sIHVpSXRlbXMgPSB1aS5pdGVtczt2YXIgZGF0YSA9IEdvQWNjZXNzLmdldFBhbmVsRGF0
YShwYW5lbCkubWV0YWRhdGE7Zm9yICh2YXIgaSA9IDA7IGkgPCB1aUl0ZW1zLmxlbmd0aDsgKytp
KSB7dmFyIGl0ZW0gPSB1aUl0ZW1zW2ldO2lmICh0aGlzLmhpZGVDb2x1bW4ocGFuZWwsIGl0ZW0u
a2V5KSljb250aW51ZTtjZWxscy5wdXNoKHRoaXMuZ2V0TWV0YUNlbGwoaXRlbSwgZGF0YVtpdGVt
LmtleV0sIGtleSkpO31yZXR1cm4gW3snaGFzU3ViSXRlbXMnOiB1aS5oYXNTdWJJdGVtcywnY2Vs
bHMnOiBjZWxscywna2V5JyA6IGtleS5zdWJzdHJpbmcoMCwgMyksfV07fSxyZW5kZXJNZXRhUm93
OiBmdW5jdGlvbiAocGFuZWwsIG1ldGFyb3dzLCBjbGFzc05hbWUpIHt2YXIgdGFibGUgPSAkKCcu
dGFibGUtJyArIHBhbmVsICsgJyB0ci4nICsgY2xhc3NOYW1lKTtpZiAoIXRhYmxlKXJldHVybjt0
YWJsZS5pbm5lckhUTUwgPSBHb0FjY2Vzcy5BcHBUcGxzLlRhYmxlcy5tZXRhLnJlbmRlcih7cm93
OiBtZXRhcm93c30pO30saXRlclVJSXRlbXM6IGZ1bmN0aW9uIChwYW5lbCwgdWlJdGVtcywgZGF0
YUl0ZW1zLCBjYWxsYmFjaykge3ZhciBvdXQgPSBbXTtmb3IgKHZhciBpID0gMDsgaSA8IHVpSXRl
bXMubGVuZ3RoOyArK2kpIHt2YXIgdWlJdGVtID0gdWlJdGVtc1tpXTtpZiAodGhpcy5oaWRlQ29s
dW1uKHBhbmVsLCB1aUl0ZW0ua2V5KSljb250aW51ZTt2YXIgZGF0YUl0ZW0gPSBkYXRhSXRlbXNb
dWlJdGVtLmtleV07aWYgKGNhbGxiYWNrICYmIHR5cGVvZiBjYWxsYmFjayA9PSAnZnVuY3Rpb24n
KSB7dmFyIHJldCA9IGNhbGxiYWNrLmNhbGwodGhpcywgcGFuZWwsIHVpSXRlbSwgZGF0YUl0ZW0p
O2lmIChyZXQpIG91dC5wdXNoKHJldCk7fX1yZXR1cm4gb3V0O30sZ2V0T2JqZWN0Q2VsbDogZnVu
Y3Rpb24gKHBhbmVsLCB1aSwgdmFsdWUpIHt2YXIgY2xhc3NOYW1lID0gdWkuY2xhc3NOYW1lIHx8
ICcnO2NsYXNzTmFtZSArPSAhWydzdHJpbmcnXS5pbmNsdWRlcyh1aS5kYXRhVHlwZSkgPyAndGV4
dC1yaWdodCcgOiAnJztyZXR1cm4geydjbGFzc05hbWUnOiBjbGFzc05hbWUsJ3BlcmNlbnQnOiBH
b0FjY2Vzcy5VdGlsLmdldFBlcmNlbnQodmFsdWUpLCd2YWx1ZSc6IEdvQWNjZXNzLlV0aWwuZm10
VmFsdWUoR29BY2Nlc3MuVXRpbC5nZXRDb3VudCh2YWx1ZSksIHVpLmRhdGFUeXBlLCBudWxsLCBu
dWxsLCB1aS5obHJlZ2V4LCB1aS5obHZhbHVlLCB1aS5obGlkeCl9O30scmVuZGVyUm93OiBmdW5j
dGlvbiAocGFuZWwsIGNhbGxiYWNrLCB1aSwgZGF0YUl0ZW0sIGlkeCwgc3ViSXRlbSwgcGFyZW50
SWQsIGV4cGFuZGVkKSB7dmFyIHNoYWRlUGFyZW50ID0gKCghc3ViSXRlbSAmJiBpZHggJSAyICE9
IDApID8gJ3NoYWRlZCcgOiAnJyk7dmFyIHNoYWRlQ2hpbGQgPSAoKHBhcmVudElkICUgMiAhPSAw
KSA/ICdzaGFkZWQnIDogJycpO3JldHVybiB7J3BhbmVsJyAgICAgICA6IHBhbmVsLCdpZHgnICAg
ICAgICAgOiAhc3ViSXRlbSAmJiAoU3RyaW5nKChpZHggKyAxKSArIHRoaXMucGFnZU9mZlNldChw
YW5lbCkpKSwna2V5JyAgICAgICAgIDogIXN1Ykl0ZW0gPyBHb0FjY2Vzcy5VdGlsLmhhc2hDb2Rl
KGRhdGFJdGVtLmRhdGEpIDogJycsJ2V4cGFuZGVkJyAgICA6ICFzdWJJdGVtICYmIGV4cGFuZGVk
LCdwYXJlbnRJZCcgICAgOiBzdWJJdGVtID8gU3RyaW5nKHBhcmVudElkKSA6ICcnLCdjbGFzc05h
bWUnICAgOiBzdWJJdGVtID8gJ2NoaWxkICcgKyBzaGFkZUNoaWxkIDogJ3BhcmVudCAnICsgc2hh
ZGVQYXJlbnQsJ2hhc1N1Ykl0ZW1zJyA6IHVpLmhhc1N1Ykl0ZW1zLCdpdGVtcycgICAgICAgOiBk
YXRhSXRlbS5pdGVtcyA/IGRhdGFJdGVtLml0ZW1zLmxlbmd0aCA6IDAsJ2NlbGxzJyAgICAgICA6
IGNhbGxiYWNrLmNhbGwodGhpcyksfTt9LHJlbmRlclJvd3M6IGZ1bmN0aW9uIChyb3dzLCBwYW5l
bCwgdWksIGRhdGFJdGVtcywgc3ViSXRlbSwgcGFyZW50SWQpIHtzdWJJdGVtID0gc3ViSXRlbSB8
fCBmYWxzZTtpZiAoZGF0YUl0ZW1zLmxlbmd0aCA9PSAwICYmIHVpLml0ZW1zLmxlbmd0aCkge3Jv
d3MucHVzaCh7Y2VsbHM6IFt7Y2xhc3NOYW1lOiAndGV4dC1jZW50ZXInLGNvbHNwYW46IHVpLml0
ZW1zLmxlbmd0aCArIDEsdmFsdWU6ICdObyBkYXRhIG9uIHRoaXMgcGFuZWwuJ31dfSk7fXZhciBj
ZWxsY2IgPSBudWxsO2ZvciAodmFyIGkgPSAwOyBpIDwgZGF0YUl0ZW1zLmxlbmd0aDsgKytpKSB7
dmFyIGRhdGFJdGVtID0gZGF0YUl0ZW1zW2ldLCBkYXRhID0gbnVsbCwgZXhwYW5kZWQgPSBmYWxz
ZTtzd2l0Y2godHlwZW9mIGRhdGFJdGVtKSB7Y2FzZSAnc3RyaW5nJzpkYXRhID0gZGF0YUl0ZW07
Y2VsbGNiID0gZnVuY3Rpb24gKCkge3JldHVybiB7J2NvbHNwYW4nOiB1aS5pdGVtcy5sZW5ndGgs
J3ZhbHVlJzogZGF0YX07fTticmVhaztkZWZhdWx0OmRhdGEgPSBkYXRhSXRlbS5kYXRhO2NlbGxj
YiA9IHRoaXMuaXRlclVJSXRlbXMuYmluZCh0aGlzLCBwYW5lbCwgdWkuaXRlbXMsIGRhdGFJdGVt
LCB0aGlzLmdldE9iamVjdENlbGwuYmluZCh0aGlzKSk7fWV4cGFuZGVkID0gdGhpcy5pc0V4cGFu
ZGVkKHBhbmVsLCBHb0FjY2Vzcy5VdGlsLmhhc2hDb2RlKGRhdGEpKTtyb3dzLnB1c2godGhpcy5y
ZW5kZXJSb3cocGFuZWwsIGNlbGxjYiwgdWksIGRhdGFJdGVtLCBpLCBzdWJJdGVtLCBwYXJlbnRJ
ZCwgZXhwYW5kZWQpKTtpZiAoZGF0YUl0ZW0uaXRlbXMgJiYgZGF0YUl0ZW0uaXRlbXMubGVuZ3Ro
ICYmIGV4cGFuZGVkKSB7dGhpcy5yZW5kZXJSb3dzKHJvd3MsIHBhbmVsLCB1aSwgZGF0YUl0ZW0u
aXRlbXMsIHRydWUsIGksIGV4cGFuZGVkKTt9fX0scmVuZGVyRGF0YVJvd3M6IGZ1bmN0aW9uIChw
YW5lbCwgdWksIGRhdGFJdGVtcywgcGFnZSkge3ZhciB0YWJsZSA9ICQoJy50YWJsZS0nICsgcGFu
ZWwgKyAnIHRib2R5LnRib2R5LWRhdGEnKTtpZiAoIXRhYmxlKXJldHVybjtkYXRhSXRlbXMgPSB0
aGlzLmdldFBhZ2UocGFuZWwsIGRhdGFJdGVtcywgcGFnZSk7dmFyIHJvd3MgPSBbXTt0aGlzLnJl
bmRlclJvd3Mocm93cywgcGFuZWwsIHVpLCBkYXRhSXRlbXMpO2lmIChyb3dzLmxlbmd0aCA9PSAw
KXJldHVybjt0YWJsZS5pbm5lckhUTUwgPSBHb0FjY2Vzcy5BcHBUcGxzLlRhYmxlcy5kYXRhLnJl
bmRlcih7cm93czogcm93c30pO30sdG9nZ2xlUGFnaW5hdGlvbjogZnVuY3Rpb24gKHBhbmVsLCBw
YWdlLCBkYXRhSXRlbXMpIHtHb0FjY2Vzcy5QYW5lbHMuZW5hYmxlUGFnaW5hdGlvbihwYW5lbCk7
aWYgKHBhZ2UgPj0gdGhpcy5nZXRUb3RhbFBhZ2VzKGRhdGFJdGVtcykpIHtHb0FjY2Vzcy5QYW5l
bHMuZGlzYWJsZU5leHQocGFuZWwpO0dvQWNjZXNzLlBhbmVscy5kaXNhYmxlTGFzdChwYW5lbCk7
fWlmIChwYWdlIDw9IDEpIHtHb0FjY2Vzcy5QYW5lbHMuZGlzYWJsZVByZXYocGFuZWwpO0dvQWNj
ZXNzLlBhbmVscy5kaXNhYmxlRmlyc3QocGFuZWwpO319LHJlbmRlclRhYmxlOiBmdW5jdGlvbiAo
cGFuZWwsIHBhZ2UpIHt2YXIgZGF0YUl0ZW1zID0gR29BY2Nlc3MuZ2V0UGFuZWxEYXRhKHBhbmVs
KS5kYXRhO3ZhciB1aSA9IEdvQWNjZXNzLmdldFBhbmVsVUkocGFuZWwpO2lmIChwYWdlID09PSAi
TEFTVF9QQUdFIikge3BhZ2UgPSB0aGlzLmdldFRvdGFsUGFnZXMoZGF0YUl0ZW1zKTt9IGVsc2Ug
aWYgKHBhZ2UgPT09ICJGSVJTVF9QQUdFIikge3BhZ2UgPSAxO310aGlzLnRvZ2dsZVBhZ2luYXRp
b24ocGFuZWwsIHBhZ2UsIGRhdGFJdGVtcyk7dGhpcy5yZW5kZXJEYXRhUm93cyhwYW5lbCwgdWks
IGRhdGFJdGVtcywgcGFnZSk7dGhpcy5ldmVudHMoKTt9LHJlbmRlckZ1bGxUYWJsZTogZnVuY3Rp
b24gKHBhbmVsKSB7dmFyIHVpID0gR29BY2Nlc3MuZ2V0UGFuZWxVSShwYW5lbCksIHBhZ2UgPSAw
O3ZhciBkYXRhID0gR29BY2Nlc3MuZ2V0UGFuZWxEYXRhKHBhbmVsKTtpZiAoZGF0YS5oYXNPd25Q
cm9wZXJ0eSgnbWV0YWRhdGEnKSkge3RoaXMucmVuZGVyTWV0YVJvdyhwYW5lbCwgdGhpcy5nZXRN
ZXRhUm93cyhwYW5lbCwgdWksICdtaW4nKSwgJ3RoZWFkLW1pbicpO3RoaXMucmVuZGVyTWV0YVJv
dyhwYW5lbCwgdGhpcy5nZXRNZXRhUm93cyhwYW5lbCwgdWksICdtYXgnKSwgJ3RoZWFkLW1heCcp
O3RoaXMucmVuZGVyTWV0YVJvdyhwYW5lbCwgdGhpcy5nZXRNZXRhUm93cyhwYW5lbCwgdWksICdh
dmcnKSwgJ3RoZWFkLWF2ZycpO31pZiAoZGF0YS5oYXNPd25Qcm9wZXJ0eSgnZGF0YScpKSB7cGFn
ZSA9IHRoaXMuZ2V0Q3VyUGFnZShwYW5lbCk7dGhpcy50b2dnbGVQYWdpbmF0aW9uKHBhbmVsLCBw
YWdlLCBkYXRhLmRhdGEpO3RoaXMucmVuZGVyRGF0YVJvd3MocGFuZWwsIHVpLCBkYXRhLmRhdGEs
IHBhZ2UpO31pZiAoZGF0YS5oYXNPd25Qcm9wZXJ0eSgnbWV0YWRhdGEnKSkge3RoaXMucmVuZGVy
TWV0YVJvdyhwYW5lbCwgdGhpcy5nZXRNZXRhUm93cyhwYW5lbCwgdWksICd0b3RhbCcpLCAndGZv
b3QtdG90YWxzJyk7fX0scmVuZGVyVGFibGVzOiBmdW5jdGlvbiAoZm9yY2UpIHt2YXIgdWkgPSBH
b0FjY2Vzcy5nZXRQYW5lbFVJKCk7Zm9yICh2YXIgcGFuZWwgaW4gdWkpIHtpZiAoR29BY2Nlc3Mu
VXRpbC5pc1BhbmVsVmFsaWQocGFuZWwpIHx8IEdvQWNjZXNzLlV0aWwuaXNQYW5lbEhpZGRlbihw
YW5lbCkgfHwgIXRoaXMuc2hvd1RhYmxlcygpKWNvbnRpbnVlO2lmIChmb3JjZSB8fCBHb0FjY2Vz
cy5VdGlsLmlzV2l0aGluVmlld1BvcnQoJCgnI3BhbmVsLScgKyBwYW5lbCkpKXRoaXMucmVuZGVy
RnVsbFRhYmxlKHBhbmVsKTt9fSxzb3J0MlRwbDogZnVuY3Rpb24gKHBhbmVsLCB1aSkge3ZhciB1
aUNsb25lID0gSlNPTi5wYXJzZShKU09OLnN0cmluZ2lmeSh1aSkpLCBvdXQgPSBbXTt2YXIgc29y
dCA9IEdvQWNjZXNzLlV0aWwuZ2V0UHJvcChHb0FjY2Vzcy5BcHBTdGF0ZSwgcGFuZWwgKyAnLnNv
cnQnKTtmb3IgKHZhciBpID0gMCwgbGVuID0gdWlDbG9uZS5pdGVtcy5sZW5ndGg7IGkgPCBsZW47
ICsraSkge3ZhciBpdGVtID0gdWlDbG9uZS5pdGVtc1tpXTtpZiAodGhpcy5oaWRlQ29sdW1uKHBh
bmVsLCBpdGVtLmtleSkpY29udGludWU7aXRlbVsnc29ydCddID0gZmFsc2U7aWYgKGl0ZW0ua2V5
ID09IHNvcnQuZmllbGQgJiYgc29ydC5vcmRlcikge2l0ZW1bJ3NvcnQnXSA9IHRydWU7aXRlbVtz
b3J0Lm9yZGVyLnRvTG93ZXJDYXNlKCldID0gdHJ1ZTt9b3V0LnB1c2goaXRlbSk7fXVpQ2xvbmUu
aXRlbXMgPSBvdXQ7cmV0dXJuIHVpQ2xvbmU7fSxyZW5kZXJUaGVhZDogZnVuY3Rpb24gKHBhbmVs
LCB1aSkge3ZhciAkdGhlYWQgPSAkKCcudGFibGUtJyArIHBhbmVsICsgJz50aGVhZD50ci50aGVh
ZC1jb2xzJyksJGNvbGdyb3VwID0gJCgnLnRhYmxlLScgKyBwYW5lbCArICc+Y29sZ3JvdXAnKTtp
ZiAoJHRoZWFkICYmICRjb2xncm91cCAmJiB0aGlzLnNob3dUYWJsZXMoKSkge3VpID0gdGhpcy5z
b3J0MlRwbChwYW5lbCwgdWkpOyR0aGVhZC5pbm5lckhUTUwgPSBHb0FjY2Vzcy5BcHBUcGxzLlRh
Ymxlcy5oZWFkLnJlbmRlcih1aSk7JGNvbGdyb3VwLmlubmVySFRNTCA9IEdvQWNjZXNzLkFwcFRw
bHMuVGFibGVzLmNvbGdyb3VwLnJlbmRlcih1aSk7fX0scmVsb2FkVGFibGVzOiBmdW5jdGlvbiAo
KSB7dGhpcy5yZW5kZXJUYWJsZXMoZmFsc2UpO3RoaXMuZXZlbnRzKCk7fSxpbml0aWFsaXplOiBm
dW5jdGlvbiAoKSB7dGhpcy5yZW5kZXJUYWJsZXModHJ1ZSk7dGhpcy5ldmVudHMoKTtkMy5zZWxl
Y3Qod2luZG93KS5vbignc2Nyb2xsLnRhYmxlcycsIGRlYm91bmNlKGZ1bmN0aW9uICgpIHt0aGlz
LnJlbG9hZFRhYmxlcygpO30sIDI1MCwgZmFsc2UpLmJpbmQodGhpcykpO30sfTtHb0FjY2Vzcy5B
cHAgPSB7aGFzRm9jdXM6IHRydWUsdHBsOiBmdW5jdGlvbiAodHBsKSB7cmV0dXJuIEhvZ2FuLmNv
bXBpbGUodHBsKTt9LHNldFRwbHM6IGZ1bmN0aW9uICgpIHtHb0FjY2Vzcy5BcHBUcGxzID0geydO
YXYnOiB7J3dyYXAnOiB0aGlzLnRwbCgkKCcjdHBsLW5hdi13cmFwJykuaW5uZXJIVE1MKSwnbWVu
dSc6IHRoaXMudHBsKCQoJyN0cGwtbmF2LW1lbnUnKS5pbm5lckhUTUwpLCdvcHRzJzogdGhpcy50
cGwoJCgnI3RwbC1uYXYtb3B0cycpLmlubmVySFRNTCksfSwnUGFuZWxzJzogeyd3cmFwJzogdGhp
cy50cGwoJCgnI3RwbC1wYW5lbCcpLmlubmVySFRNTCksJ29wdHMnOiB0aGlzLnRwbCgkKCcjdHBs
LXBhbmVsLW9wdHMnKS5pbm5lckhUTUwpLH0sJ0dlbmVyYWwnOiB7J3dyYXAnOiB0aGlzLnRwbCgk
KCcjdHBsLWdlbmVyYWwnKS5pbm5lckhUTUwpLCdpdGVtcyc6IHRoaXMudHBsKCQoJyN0cGwtZ2Vu
ZXJhbC1pdGVtcycpLmlubmVySFRNTCksfSwnVGFibGVzJzogeydjb2xncm91cCc6IHRoaXMudHBs
KCQoJyN0cGwtdGFibGUtY29sZ3JvdXAnKS5pbm5lckhUTUwpLCdoZWFkJzogdGhpcy50cGwoJCgn
I3RwbC10YWJsZS10aGVhZCcpLmlubmVySFRNTCksJ21ldGEnOiB0aGlzLnRwbCgkKCcjdHBsLXRh
YmxlLXJvdy1tZXRhJykuaW5uZXJIVE1MKSwndG90YWxzJzogdGhpcy50cGwoJCgnI3RwbC10YWJs
ZS1yb3ctdG90YWxzJykuaW5uZXJIVE1MKSwnZGF0YSc6IHRoaXMudHBsKCQoJyN0cGwtdGFibGUt
cm93JykuaW5uZXJIVE1MKSx9LH07fSxzb3J0RmllbGQ6IGZ1bmN0aW9uIChvLCBmaWVsZCkge3Zh
ciBmID0gb1tmaWVsZF07aWYgKEdvQWNjZXNzLlV0aWwuaXNPYmplY3QoZikgJiYgKGYgIT09IG51
bGwpKWYgPSBvW2ZpZWxkXS5jb3VudDtyZXR1cm4gZjt9LHNvcnREYXRhOiBmdW5jdGlvbiAocGFu
ZWwsIGZpZWxkLCBvcmRlcikge3ZhciBwYW5lbERhdGEgPSBHb0FjY2Vzcy5nZXRQYW5lbERhdGEo
cGFuZWwpLmRhdGE7dmFyIHNvcnRBcnJheSA9IGZ1bmN0aW9uKGFycikge2Fyci5zb3J0KGZ1bmN0
aW9uIChhLCBiKSB7YSA9IHRoaXMuc29ydEZpZWxkKGEsIGZpZWxkKTtiID0gdGhpcy5zb3J0Rmll
bGQoYiwgZmllbGQpO2lmICh0eXBlb2YgYSA9PT0gJ3N0cmluZycgJiYgdHlwZW9mIGIgPT09ICdz
dHJpbmcnKXJldHVybiAnYXNjJyA9PSBvcmRlciA/IGEubG9jYWxlQ29tcGFyZShiKSA6IGIubG9j
YWxlQ29tcGFyZShhKTtyZXR1cm4gICdhc2MnID09IG9yZGVyID8gYSAtIGIgOiBiIC0gYTt9LmJp
bmQodGhpcykpO30uYmluZCh0aGlzKTtzb3J0QXJyYXkocGFuZWxEYXRhKTtwYW5lbERhdGEuZm9y
RWFjaChmdW5jdGlvbihpdGVtKSB7aWYgKGl0ZW0uaXRlbXMpIHtzb3J0QXJyYXkoaXRlbS5pdGVt
cyk7fX0pO30sc2V0SW5pdFNvcnQ6IGZ1bmN0aW9uICgpIHt2YXIgdWkgPSBHb0FjY2Vzcy5nZXRQ
YW5lbFVJKCk7Zm9yICh2YXIgcGFuZWwgaW4gdWkpIHtpZiAoR29BY2Nlc3MuVXRpbC5pc1BhbmVs
VmFsaWQocGFuZWwpKWNvbnRpbnVlO0dvQWNjZXNzLlV0aWwuc2V0UHJvcChHb0FjY2Vzcy5BcHBT
dGF0ZSwgcGFuZWwgKyAnLnNvcnQnLCB1aVtwYW5lbF0uc29ydCk7fX0sdmVyaWZ5U29ydDogZnVu
Y3Rpb24gKCkge3ZhciB1aSA9IEdvQWNjZXNzLmdldFBhbmVsVUkoKTtmb3IgKHZhciBwYW5lbCBp
biB1aSkge2lmIChHb0FjY2Vzcy5VdGlsLmlzUGFuZWxWYWxpZChwYW5lbCkgfHwgR29BY2Nlc3Mu
VXRpbC5pc1BhbmVsSGlkZGVuKHBhbmVsKSljb250aW51ZTt2YXIgc29ydCA9IEdvQWNjZXNzLlV0
aWwuZ2V0UHJvcChHb0FjY2Vzcy5BcHBTdGF0ZSwgcGFuZWwgKyAnLnNvcnQnKTtpZiAoSlNPTi5z
dHJpbmdpZnkoc29ydCkgPT09IEpTT04uc3RyaW5naWZ5KHVpW3BhbmVsXS5zb3J0KSljb250aW51
ZTt0aGlzLnNvcnREYXRhKHBhbmVsLCBzb3J0LmZpZWxkLCBzb3J0Lm9yZGVyKTt9fSxpbml0RG9t
OiBmdW5jdGlvbiAoKSB7JCgnbmF2JykuY2xhc3NMaXN0LnJlbW92ZSgnaGlkZScpOyQoJy5jb250
YWluZXInKS5jbGFzc0xpc3QucmVtb3ZlKCdoaWRlJyk7JCgnLnNwaW5uZXInKS5jbGFzc0xpc3Qu
YWRkKCdoaWRlJyk7aWYgKEdvQWNjZXNzLkFwcFByZWZzWydsYXlvdXQnXSA9PSAnaG9yaXpvbnRh
bCcgfHwgR29BY2Nlc3MuQXBwUHJlZnNbJ2xheW91dCddID09ICd3aWRlJykgeyQoJy5jb250YWlu
ZXInKS5jbGFzc0xpc3QuYWRkKCdjb250YWluZXItZmx1aWQnKTskKCcuY29udGFpbmVyLWZsdWlk
JykuY2xhc3NMaXN0LnJlbW92ZSgnY29udGFpbmVyJyk7fX0scmVuZGVyRGF0YTogZnVuY3Rpb24g
KCkge2lmICghdGhpcy5oYXNGb2N1cylyZXR1cm47aWYgKEdvQWNjZXNzLk92ZXJhbGxTdGF0cy50
b3RhbF9yZXF1ZXN0cyA9PSAwICYmIEdvQWNjZXNzLk92ZXJhbGxTdGF0cy50b3RhbF9yZXF1ZXN0
cyAhPSBHb0FjY2Vzcy5BcHBEYXRhLmdlbmVyYWwudG90YWxfcmVxdWVzdHMpR29BY2Nlc3MuUGFu
ZWxzLmluaXRpYWxpemUoKTtHb0FjY2Vzcy5PdmVyYWxsU3RhdHMudG90YWxfcmVxdWVzdHMgPSBH
b0FjY2Vzcy5BcHBEYXRhLmdlbmVyYWwudG90YWxfcmVxdWVzdHM7dGhpcy52ZXJpZnlTb3J0KCk7
R29BY2Nlc3MuT3ZlcmFsbFN0YXRzLmluaXRpYWxpemUoKTtpZiAoIUdvQWNjZXNzLkFwcFN0YXRl
LnVwZGF0ZWQpcmV0dXJuO0dvQWNjZXNzLkNoYXJ0cy5yZWxvYWRDaGFydHMoKTtHb0FjY2Vzcy5U
YWJsZXMucmVsb2FkVGFibGVzKCk7fSxpbml0aWFsaXplOiBmdW5jdGlvbiAoKSB7dGhpcy5zZXRJ
bml0U29ydCgpO3RoaXMuc2V0VHBscygpO0dvQWNjZXNzLk5hdi5pbml0aWFsaXplKCk7dGhpcy5p
bml0RG9tKCk7R29BY2Nlc3MuT3ZlcmFsbFN0YXRzLmluaXRpYWxpemUoKTtHb0FjY2Vzcy5QYW5l
bHMuaW5pdGlhbGl6ZSgpO0dvQWNjZXNzLkNoYXJ0cy5pbml0aWFsaXplKCk7R29BY2Nlc3MuVGFi
bGVzLmluaXRpYWxpemUoKTt9LH07ZG9jdW1lbnQuYWRkRXZlbnRMaXN0ZW5lcigndmlzaWJpbGl0
eWNoYW5nZScsIGZ1bmN0aW9uICgpIHtpZiAoZG9jdW1lbnQudmlzaWJpbGl0eVN0YXRlID09PSAn
aGlkZGVuJylHb0FjY2Vzcy5BcHAuaGFzRm9jdXMgPSBmYWxzZTtpZiAoZG9jdW1lbnQudmlzaWJp
bGl0eVN0YXRlID09PSAndmlzaWJsZScpIHt2YXIgaGFzRm9jdXMgPSBHb0FjY2Vzcy5BcHAuaGFz
Rm9jdXM7R29BY2Nlc3MuQXBwLmhhc0ZvY3VzID0gdHJ1ZTtoYXNGb2N1cyB8fCBHb0FjY2Vzcy5B
cHAucmVuZGVyRGF0YSgpO319KTt3aW5kb3cub25sb2FkID0gZnVuY3Rpb24gKCkge0dvQWNjZXNz
LmluaXRpYWxpemUoeydpMThuJzogd2luZG93Lmpzb25faTE4biwndWlEYXRhJzogd2luZG93LnVz
ZXJfaW50ZXJmYWNlLCdwYW5lbERhdGEnOiB3aW5kb3cuanNvbl9kYXRhLCd3c0Nvbm5lY3Rpb24n
OiB3aW5kb3cuY29ubmVjdGlvbiB8fCBudWxsLCdwcmVmcyc6IHdpbmRvdy5odG1sX3ByZWZzIHx8
IHt9LH0pO0dvQWNjZXNzLkFwcC5pbml0aWFsaXplKCk7fTt9KCkpOzwvc2NyaXB0PjxzY3JpcHQ+
LyoqKiAgICBfX19fX18gICAgICBfX18qICAgLyBfX19fL19fXyAgLyAgIHwgX19fX19fX19fX19f
XyAgX19fX19fX19fXyogIC8gLyBfXy8gX18gXC8gL3wgfC8gX19fLyBfX18vIF8gXC8gX19fLyBf
X18vKiAvIC9fLyAvIC9fLyAvIF9fXyAvIC9fXy8gL19fLyAgX18oX18gIHxfXyAgKSogXF9fX18v
XF9fX18vXy8gIHxfXF9fXy9cX19fL1xfX18vX19fXy9fX19fLyoqIFRoZSBNSVQgTGljZW5zZSAo
TUlUKSogQ29weXJpZ2h0IChjKSAyMDA5LTIwMTggR2VyYXJkbyBPcmVsbGFuYSA8aGVsbG8gQCBn
b2FjY2Vzcy5pbz4qLyd1c2Ugc3RyaWN0JztmdW5jdGlvbiB0cnVuY2F0ZSh0ZXh0LCB3aWR0aCkg
e3RleHQuZWFjaChmdW5jdGlvbiAoKSB7dmFyIHBhcmVudCA9IHRoaXMucGFyZW50Tm9kZSwgJGQz
cGFyZW50ID0gZDMuc2VsZWN0KHBhcmVudCk7dmFyIGd3ID0gJGQzcGFyZW50Lm5vZGUoKS5nZXRC
Qm94KCk7dmFyIHggPSAoTWF0aC5taW4oZ3cud2lkdGgsIHdpZHRoKSAvIDIpICogLTE7aWYgKCdz
dmcnID09IHBhcmVudC5ub2RlTmFtZSkgeyRkM3BhcmVudC5hdHRyKCd3aWR0aCcsIHdpZHRoKS5h
dHRyKCd4JywgeCk7fWVsc2UgeyRkM3BhcmVudC5pbnNlcnQoJ3N2ZycsIGZ1bmN0aW9uICgpIHty
ZXR1cm4gdGhpczt9LmJpbmQodGhpcykpLmF0dHIoJ2NsYXNzJywgJ3dyYXAtdGV4dCcpLmF0dHIo
J3dpZHRoJywgd2lkdGgpLmF0dHIoJ3gnLCB4KS5hcHBlbmQoZnVuY3Rpb24gKCkge3JldHVybiB0
aGlzO30uYmluZCh0aGlzKSk7fX0pO31mdW5jdGlvbiBBcmVhQ2hhcnQoZHVhbFlheGlzKSB7dmFy
IG9wdHMgPSB7fTt2YXIgbWFyZ2luID0ge3RvcCAgICA6IDIwLHJpZ2h0ICA6IDUwLGJvdHRvbSA6
IDQwLGxlZnQgICA6IDUwLH0saGVpZ2h0ID0gMTcwLG5UaWNrcyA9IDEwLHBhZGRpbmcgPSAxMCx3
aWR0aCA9IDc2MDt2YXIgbGFiZWxzID0geyB4OiAnVW5uYW1lZCcsIHkwOiAnVW5uYW1lZCcsIHkx
OiAnVW5uYW1lZCcgfTt2YXIgZm9ybWF0ID0geyB4OiBudWxsLCB5MDogbnVsbCwgeTE6IG51bGx9
O3ZhciB4VmFsdWUgPSBmdW5jdGlvbiAoZCkge3JldHVybiBkWzBdO30seVZhbHVlMCA9IGZ1bmN0
aW9uIChkKSB7cmV0dXJuIGRbMV07fSx5VmFsdWUxID0gZnVuY3Rpb24gKGQpIHtyZXR1cm4gZFsy
XTt9O3ZhciB4U2NhbGUgPSBkMy5zY2FsZUJhbmQoKTt2YXIgeVNjYWxlMCA9IGQzLnNjYWxlTGlu
ZWFyKCkubmljZSgpO3ZhciB5U2NhbGUxID0gZDMuc2NhbGVMaW5lYXIoKS5uaWNlKCk7dmFyIHhB
eGlzID0gZDMuYXhpc0JvdHRvbSh4U2NhbGUpLnRpY2tGb3JtYXQoZnVuY3Rpb24oZCkge2lmIChm
b3JtYXQueClyZXR1cm4gR29BY2Nlc3MuVXRpbC5mbXRWYWx1ZShkLCBmb3JtYXQueCk7cmV0dXJu
IGQ7fSk7dmFyIHlBeGlzMCA9IGQzLmF4aXNMZWZ0KHlTY2FsZTApLnRpY2tGb3JtYXQoZnVuY3Rp
b24oZCkge3JldHVybiBkMy5mb3JtYXQoJy4ycycpKGQpO30pO3ZhciB5QXhpczEgPSBkMy5heGlz
UmlnaHQoeVNjYWxlMSkudGlja0Zvcm1hdChmdW5jdGlvbihkKSB7aWYgKGZvcm1hdC55MSlyZXR1
cm4gR29BY2Nlc3MuVXRpbC5mbXRWYWx1ZShkLCBmb3JtYXQueTEpO3JldHVybiBkMy5mb3JtYXQo
Jy4ycycpKGQpO30pO3ZhciB4R3JpZCA9IGQzLmF4aXNCb3R0b20oeFNjYWxlKTt2YXIgeUdyaWQg
PSBkMy5heGlzTGVmdCh5U2NhbGUwKTt2YXIgYXJlYTAgPSBkMy5hcmVhKCkuY3VydmUoZDMuY3Vy
dmVNb25vdG9uZVgpLngoWCkueTAoaGVpZ2h0KS55MShZMCk7dmFyIGFyZWExID0gZDMuYXJlYSgp
LmN1cnZlKGQzLmN1cnZlTW9ub3RvbmVYKS54KFgpLnkwKFkxKS55MShoZWlnaHQpO3ZhciBsaW5l
MCA9IGQzLmxpbmUoKS5jdXJ2ZShkMy5jdXJ2ZU1vbm90b25lWCkueChYKS55KFkwKTt2YXIgbGlu
ZTEgPSBkMy5saW5lKCkuY3VydmUoZDMuY3VydmVNb25vdG9uZVgpLngoWCkueShZMSk7ZnVuY3Rp
b24gWChkKSB7cmV0dXJuICh4U2NhbGUoZFswXSkgKyB4U2NhbGUuYmFuZHdpZHRoKCkgLyAyKTt9
ZnVuY3Rpb24gWTAoZCkge3JldHVybiB5U2NhbGUwKGRbMV0pO31mdW5jdGlvbiBZMShkKSB7cmV0
dXJuIHlTY2FsZTEoZFsyXSk7fWZ1bmN0aW9uIGlubmVyVygpIHtyZXR1cm4gd2lkdGggLSBtYXJn
aW4ubGVmdCAtIG1hcmdpbi5yaWdodDt9ZnVuY3Rpb24gaW5uZXJIKCkge3JldHVybiBoZWlnaHQg
LSBtYXJnaW4udG9wIC0gbWFyZ2luLmJvdHRvbTt9ZnVuY3Rpb24gZ2V0WFRpY2tzKGRhdGEpIHtj
b25zdCBkb21haW4gPSB4U2NhbGUuZG9tYWluKCk7aWYgKGRhdGEubGVuZ3RoIDwgblRpY2tzKXJl
dHVybiBkb21haW47cmV0dXJuIGQzLnJhbmdlKDAsIG5UaWNrcykubWFwKGZ1bmN0aW9uKGkpIHtj
b25zdCBpbmRleCA9IE1hdGguZmxvb3IoaSAqIChkb21haW4ubGVuZ3RoIC0gMSkgLyAoblRpY2tz
IC0gMSkpO2lmIChpbmRleCA+PSAwICYmIGluZGV4IDwgZG9tYWluLmxlbmd0aClyZXR1cm4gZG9t
YWluW2luZGV4XTtyZXR1cm4gbnVsbDt9KTt9ZnVuY3Rpb24gZ2V0WVRpY2tzKHNjYWxlKSB7dmFy
IGRvbWFpbiA9IHNjYWxlLmRvbWFpbigpO3JldHVybiBkMy5yYW5nZShkb21haW5bMF0sIGRvbWFp
blsxXSwgTWF0aC5jZWlsKGRvbWFpblsxXSAvIG5UaWNrcykpO31mdW5jdGlvbiBtYXBEYXRhKGRh
dGEpIHt2YXIgX2RhdHVtID0gZnVuY3Rpb24gKGQsIGkpIHt2YXIgZGF0dW0gPSBbeFZhbHVlLmNh
bGwoZGF0YSwgZCwgaSksIHlWYWx1ZTAuY2FsbChkYXRhLCBkLCBpKV07ZHVhbFlheGlzICYmIGRh
dHVtLnB1c2goeVZhbHVlMS5jYWxsKGRhdGEsIGQsIGkpKTtyZXR1cm4gZGF0dW07fTtyZXR1cm4g
ZGF0YS5tYXAoZnVuY3Rpb24gKGQsIGkpIHtyZXR1cm4gX2RhdHVtKGQsIGkpO30pO31mdW5jdGlv
biB1cGRhdGVTY2FsZXMoZGF0YSkge3hTY2FsZS5kb21haW4oZGF0YS5tYXAoZnVuY3Rpb24gKGQp
IHtyZXR1cm4gZFswXTt9KSkucmFuZ2UoWzAsIGlubmVyVygpXSk7eVNjYWxlMC5kb21haW4oWzAs
IGQzLm1heChkYXRhLCBmdW5jdGlvbiAoZCkge3JldHVybiBkWzFdO30pXSkucmFuZ2UoW2lubmVy
SCgpLCAwXSk7ZHVhbFlheGlzICYmIHlTY2FsZTEuZG9tYWluKFswLCBkMy5tYXgoZGF0YSwgZnVu
Y3Rpb24gKGQpIHtyZXR1cm4gZFsyXTt9KV0pLnJhbmdlKFtpbm5lckgoKSwgMF0pO31mdW5jdGlv
biB0b2dnbGVPcGFjaXR5KGVsZSwgb3ApIHtkMy5zZWxlY3QoZWxlLnBhcmVudE5vZGUpLnNlbGVj
dEFsbCgnLicgKyAoZWxlLmdldEF0dHJpYnV0ZSgnZGF0YS15YXhpcycpID09ICd5MCcgPyAneTEn
IDogJ3kwJykpLmF0dHIoJ3N0eWxlJywgb3ApO31mdW5jdGlvbiBzZXRMZWdlbmRMYWJlbHMoc3Zn
KSB7dmFyIHJlY3QgPSBzdmcuc2VsZWN0QWxsKCdyZWN0LmxlZ2VuZC55MCcpLmRhdGEoW251bGxd
KTt2YXIgcmVjdEVudGVyID0gcmVjdC5lbnRlcigpLmFwcGVuZCgncmVjdCcpLmF0dHIoJ2NsYXNz
JywgJ2xlZ2VuZCB5MCcpLmF0dHIoJ2RhdGEteWF4aXMnLCAneTAnKS5vbignbW91c2Vtb3ZlJywg
ZnVuY3Rpb24oZCwgaSkge3RvZ2dsZU9wYWNpdHkodGhpcywgJ29wYWNpdHk6MC4xJyk7fSkub24o
J21vdXNlbGVhdmUnLCBmdW5jdGlvbihkLCBpKSB7dG9nZ2xlT3BhY2l0eSh0aGlzLCBudWxsKTt9
KS5hdHRyKCd5JywgKGhlaWdodCAtIDE1KSk7cmVjdEVudGVyLm1lcmdlKHJlY3QpLmF0dHIoJ3gn
LCAod2lkdGggLyAyKSAtIDEwMCk7dmFyIHRleHQgPSBzdmcuc2VsZWN0QWxsKCd0ZXh0LmxlZ2Vu
ZC55MCcpLmRhdGEoW251bGxdKTt2YXIgdGV4dEVudGVyID0gdGV4dC5lbnRlcigpLmFwcGVuZCgn
dGV4dCcpLmF0dHIoJ2NsYXNzJywgJ2xlZ2VuZCB5MCcpLmF0dHIoJ2RhdGEteWF4aXMnLCAneTAn
KS5vbignbW91c2Vtb3ZlJywgZnVuY3Rpb24oZCwgaSkge3RvZ2dsZU9wYWNpdHkodGhpcywgJ29w
YWNpdHk6MC4xJyk7fSkub24oJ21vdXNlbGVhdmUnLCBmdW5jdGlvbihkLCBpKSB7dG9nZ2xlT3Bh
Y2l0eSh0aGlzLCBudWxsKTt9KS5hdHRyKCd5JywgKGhlaWdodCAtIDYpKTt0ZXh0RW50ZXIubWVy
Z2UodGV4dCkuYXR0cigneCcsICh3aWR0aCAvIDIpIC0gODUpLnRleHQobGFiZWxzLnkwKTtpZiAo
IWR1YWxZYXhpcylyZXR1cm47cmVjdCA9IHN2Zy5zZWxlY3RBbGwoJ3JlY3QubGVnZW5kLnkxJyku
ZGF0YShbbnVsbF0pO3ZhciByZWN0RW50ZXIgPSByZWN0LmVudGVyKCkuYXBwZW5kKCdyZWN0Jyku
YXR0cignY2xhc3MnLCAnbGVnZW5kIHkxJykuYXR0cignZGF0YS15YXhpcycsICd5MScpLm9uKCdt
b3VzZW1vdmUnLCBmdW5jdGlvbihkLCBpKSB7dG9nZ2xlT3BhY2l0eSh0aGlzLCAnb3BhY2l0eTow
LjEnKTt9KS5vbignbW91c2VsZWF2ZScsIGZ1bmN0aW9uKGQsIGkpIHt0b2dnbGVPcGFjaXR5KHRo
aXMsIG51bGwpO30pLmF0dHIoJ3knLCAoaGVpZ2h0IC0gMTUpKTtyZWN0RW50ZXIubWVyZ2UocmVj
dCkuYXR0cigneCcsICh3aWR0aCAvIDIpKTt0ZXh0ID0gc3ZnLnNlbGVjdEFsbCgndGV4dC5sZWdl
bmQueTEnKS5kYXRhKFtudWxsXSk7dmFyIHRleHRFbnRlciA9IHRleHQuZW50ZXIoKS5hcHBlbmQo
J3RleHQnKS5hdHRyKCdjbGFzcycsICdsZWdlbmQgeTEnKS5hdHRyKCdkYXRhLXlheGlzJywgJ3kx
Jykub24oJ21vdXNlbW92ZScsIGZ1bmN0aW9uKGQsIGkpIHt0b2dnbGVPcGFjaXR5KHRoaXMsICdv
cGFjaXR5OjAuMScpO30pLm9uKCdtb3VzZWxlYXZlJywgZnVuY3Rpb24oZCwgaSkge3RvZ2dsZU9w
YWNpdHkodGhpcywgbnVsbCk7fSkuYXR0cigneScsIChoZWlnaHQgLSA2KSk7dGV4dEVudGVyLm1l
cmdlKHRleHQpLmF0dHIoJ3gnLCAod2lkdGggLyAyKSArIDE1KS50ZXh0KGxhYmVscy55MSk7fWZ1
bmN0aW9uIHNldEF4aXNMYWJlbHMoc3ZnKSB7c3ZnLnNlbGVjdEFsbCgndGV4dC5heGlzLWxhYmVs
LnkwJykuZGF0YShbbnVsbF0pLmVudGVyKCkuYXBwZW5kKCd0ZXh0JykuYXR0cignY2xhc3MnLCAn
YXhpcy1sYWJlbCB5MCcpLmF0dHIoJ3knLCAxMCkuYXR0cigneCcsIDUzKS50ZXh0KGxhYmVscy55
MCk7aWYgKCFkdWFsWWF4aXMpIHJldHVybjt2YXIgdEVudGVyID0gc3ZnLnNlbGVjdEFsbCgndGV4
dC5heGlzLWxhYmVsLnkxJykuZGF0YShbbnVsbF0pLmVudGVyKCkuYXBwZW5kKCd0ZXh0JykuYXR0
cignY2xhc3MnLCAnYXhpcy1sYWJlbCB5MScpLmF0dHIoJ3knLCAxMCkudGV4dChsYWJlbHMueTEp
O2R1YWxZYXhpcyAmJiB0RW50ZXIuYXR0cigneCcsIHdpZHRoIC0gMjUpO31mdW5jdGlvbiBjcmVh
dGVTa2VsZXRvbihzdmcpIHtjb25zdCBnID0gc3ZnLmFwcGVuZCgnZycpO2cuYXBwZW5kKCdnJyku
YXR0cignY2xhc3MnLCAnbGluZSBsaW5lMCB5MCcpO2R1YWxZYXhpcyAmJiBnLmFwcGVuZCgnZycp
LmF0dHIoJ2NsYXNzJywgJ2xpbmUgbGluZTEgeTEnKTtnLmFwcGVuZCgnZycpLmF0dHIoJ2NsYXNz
JywgJ2FyZWEgYXJlYTAgeTAnKTtkdWFsWWF4aXMgJiYgZy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFz
cycsICdhcmVhIGFyZWExIHkxJyk7Zy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFzcycsICdwb2ludHMg
eTAnKTtkdWFsWWF4aXMgJiYgZy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFzcycsICdwb2ludHMgeTEn
KTtnLmFwcGVuZCgnZycpLmF0dHIoJ2NsYXNzJywgJ3ggZ3JpZCcpO2cuYXBwZW5kKCdnJykuYXR0
cignY2xhc3MnLCAneSBncmlkJyk7Zy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFzcycsICd4IGF4aXMn
KTtnLmFwcGVuZCgnZycpLmF0dHIoJ2NsYXNzJywgJ3kwIGF4aXMnKTtkdWFsWWF4aXMgJiYgZy5h
cHBlbmQoJ2cnKS5hdHRyKCdjbGFzcycsICd5MSBheGlzJyk7Zy5hcHBlbmQoJ2cnKS5hdHRyKCdj
bGFzcycsICdyZWN0cycpO3NldEF4aXNMYWJlbHMoc3ZnKTtzZXRMZWdlbmRMYWJlbHMoc3ZnKTtn
LmFwcGVuZCgnbGluZScpLmF0dHIoJ3kyJywgaW5uZXJIKCkpLmF0dHIoJ3kxJywgMCkuYXR0cign
Y2xhc3MnLCAnaW5kaWNhdG9yJyk7fWZ1bmN0aW9uIHBhdGhMZW4oZCkge3JldHVybiBkLm5vZGUo
KS5nZXRUb3RhbExlbmd0aCgpO31mdW5jdGlvbiBhZGRMaW5lKGcsIGRhdGEsIGxpbmUsIGNOYW1l
KSB7dmFyIHBhdGggPSBnLnNlbGVjdCgnZy4nICsgY05hbWUpLnNlbGVjdEFsbCgncGF0aC4nICsg
Y05hbWUpLmRhdGEoW2RhdGFdKTt2YXIgcGF0aEVudGVyID0gcGF0aC5lbnRlcigpLmFwcGVuZCgn
c3ZnOnBhdGgnKS5hdHRyKCdkJywgbGluZSkuYXR0cignY2xhc3MnLCBjTmFtZSkuYXR0cignc3Ry
b2tlLWRhc2hhcnJheScsIGZ1bmN0aW9uKGQpIHt2YXIgcGwgPSBwYXRoTGVuKGQzLnNlbGVjdCh0
aGlzKSk7cmV0dXJuIHBsICsgJyAnICsgcGw7fSkuYXR0cignc3Ryb2tlLWRhc2hvZmZzZXQnLCBm
dW5jdGlvbihkKSB7cmV0dXJuIHBhdGhMZW4oZDMuc2VsZWN0KHRoaXMpKTt9KTtwYXRoRW50ZXIu
bWVyZ2UocGF0aCkuYXR0cignZCcsIGxpbmUpLnRyYW5zaXRpb24oKS5hdHRyKCdzdHJva2UtZGFz
aGFycmF5JywgZnVuY3Rpb24oZCkge3ZhciBwbCA9IHBhdGhMZW4oZDMuc2VsZWN0KHRoaXMpKTty
ZXR1cm4gcGwgKyAnICcgKyBwbDt9KS5kdXJhdGlvbigyMDAwKS5hdHRyKCdzdHJva2UtZGFzaG9m
ZnNldCcsIDApO3BhdGguZXhpdCgpLnJlbW92ZSgpO31mdW5jdGlvbiBhZGRBcmVhKGcsIGRhdGEs
IGNiLCBjTmFtZSkge3ZhciBhcmVhID0gZy5zZWxlY3QoJ2cuJyArIGNOYW1lKS5zZWxlY3RBbGwo
J3BhdGguJyArIGNOYW1lKS5kYXRhKFtkYXRhXSk7dmFyIGFyZWFFbnRlciA9IGFyZWEuZW50ZXIo
KS5hcHBlbmQoJ3N2ZzpwYXRoJykuYXR0cignY2xhc3MnLCBjTmFtZSk7YXJlYUVudGVyLm1lcmdl
KGFyZWEpLmF0dHIoJ2QnLCBjYik7YXJlYS5leGl0KCkucmVtb3ZlKCk7fWZ1bmN0aW9uIGFkZEFy
ZWFMaW5lcyhnLCBkYXRhKSB7YWRkQXJlYShnLCBkYXRhLCBhcmVhMC55MCh5U2NhbGUwLnJhbmdl
KClbMF0pLCAnYXJlYTAnKTthZGRMaW5lKGcsIGRhdGEsIGxpbmUwLCAnbGluZTAnKTthZGRBcmVh
KGcsIGRhdGEsIGFyZWExLnkxKHlTY2FsZTEucmFuZ2UoKVswXSksICdhcmVhMScpO2FkZExpbmUo
ZywgZGF0YSwgbGluZTEsICdsaW5lMScpO31mdW5jdGlvbiBhZGRQb2ludHMoZywgZGF0YSkge3Zh
ciByYWRpdXMgPSBkYXRhLmxlbmd0aCA+IDEwMCA/IDEgOiAyLjU7dmFyIHBvaW50cyA9IGcuc2Vs
ZWN0KCdnLnBvaW50cy55MCcpLnNlbGVjdEFsbCgnY2lyY2xlLnBvaW50JykuZGF0YShkYXRhKTt2
YXIgcG9pbnRzRW50ZXIgPSBwb2ludHMuZW50ZXIoKS5hcHBlbmQoJ3N2ZzpjaXJjbGUnKS5hdHRy
KCdyJywgcmFkaXVzKS5hdHRyKCdjbGFzcycsICdwb2ludCcpO3BvaW50c0VudGVyLm1lcmdlKHBv
aW50cykuYXR0cignY3gnLCBmdW5jdGlvbihkKSB7cmV0dXJuICh4U2NhbGUoZFswXSkgKyB4U2Nh
bGUuYmFuZHdpZHRoKCkgLyAyKTt9KS5hdHRyKCdjeScsIGZ1bmN0aW9uKGQpIHtyZXR1cm4geVNj
YWxlMChkWzFdKTt9KTtwb2ludHMuZXhpdCgpLnJlbW92ZSgpO2lmICghZHVhbFlheGlzKXJldHVy
bjtwb2ludHMgPSBnLnNlbGVjdCgnZy5wb2ludHMueTEnKS5zZWxlY3RBbGwoJ2NpcmNsZS5wb2lu
dCcpLmRhdGEoZGF0YSk7cG9pbnRzRW50ZXIgPSBwb2ludHMuZW50ZXIoKS5hcHBlbmQoJ3N2Zzpj
aXJjbGUnKS5hdHRyKCdyJywgcmFkaXVzKS5hdHRyKCdjbGFzcycsICdwb2ludCcpO3BvaW50c0Vu
dGVyLm1lcmdlKHBvaW50cykuYXR0cignY3gnLCBmdW5jdGlvbihkKSB7cmV0dXJuICh4U2NhbGUo
ZFswXSkgKyB4U2NhbGUuYmFuZHdpZHRoKCkgLyAyKTt9KS5hdHRyKCdjeScsIGZ1bmN0aW9uKGQp
IHtyZXR1cm4geVNjYWxlMShkWzJdKTt9KTtwb2ludHMuZXhpdCgpLnJlbW92ZSgpO31mdW5jdGlv
biBhZGRBeGlzKGcsIGRhdGEpIHt2YXIgeFRpY2tzID0gZ2V0WFRpY2tzKGRhdGEpO3ZhciB0aWNr
RGlzdGFuY2UgPSB4VGlja3MubGVuZ3RoID4gMSA/ICh4U2NhbGUoeFRpY2tzWzFdKSAtIHhTY2Fs
ZSh4VGlja3NbMF0pKSA6IGlubmVyVygpO3ZhciBsYWJlbFcgPSB0aWNrRGlzdGFuY2UgLSBwYWRk
aW5nO2cuc2VsZWN0KCcueC5heGlzJykuYXR0cigndHJhbnNmb3JtJywgJ3RyYW5zbGF0ZSgwLCcg
KyB5U2NhbGUwLnJhbmdlKClbMF0gKyAnKScpLmNhbGwoeEF4aXMudGlja1ZhbHVlcyh4VGlja3Mp
KS5zZWxlY3RBbGwoIi50aWNrIHRleHQiKS5jYWxsKHRydW5jYXRlLCBsYWJlbFcgPiAwID8gbGFi
ZWxXIDogaW5uZXJXKCkpO2cuc2VsZWN0KCcueTAuYXhpcycpLmNhbGwoeUF4aXMwLnRpY2tWYWx1
ZXMoZ2V0WVRpY2tzKHlTY2FsZTApKSk7aWYgKCFkdWFsWWF4aXMpcmV0dXJuO2cuc2VsZWN0KCcu
eTEuYXhpcycpLmF0dHIoJ3RyYW5zZm9ybScsICd0cmFuc2xhdGUoJyArIGlubmVyVygpICsgJywg
MCknKS5jYWxsKHlBeGlzMS50aWNrVmFsdWVzKGdldFlUaWNrcyh5U2NhbGUxKSkpO31mdW5jdGlv
biBhZGRHcmlkKGcsIGRhdGEpIHtnLnNlbGVjdCgnLnguZ3JpZCcpLmF0dHIoJ3RyYW5zZm9ybScs
ICd0cmFuc2xhdGUoMCwnICsgeVNjYWxlMC5yYW5nZSgpWzBdICsgJyknKS5jYWxsKHhHcmlkLnRp
Y2tWYWx1ZXMoZ2V0WFRpY2tzKGRhdGEpKS50aWNrU2l6ZSgtaW5uZXJIKCksIDAsIDApLnRpY2tT
aXplT3V0ZXIoMCkudGlja0Zvcm1hdCgnJykpO2cuc2VsZWN0KCcueS5ncmlkJykuY2FsbCh5R3Jp
ZC50aWNrVmFsdWVzKGdldFlUaWNrcyh5U2NhbGUwKSkudGlja1NpemUoLWlubmVyVygpLCAwKS50
aWNrU2l6ZU91dGVyKDApLnRpY2tGb3JtYXQoJycpKTt9ZnVuY3Rpb24gZm9ybWF0VG9vbHRpcChk
YXRhKSB7dmFyIGQgPSBkYXRhLnNsaWNlKDApO2RbMF0gPSAoZm9ybWF0LngpID8gR29BY2Nlc3Mu
VXRpbC5mbXRWYWx1ZShkWzBdLCBmb3JtYXQueCkgOiBkWzBdO2RbMV0gPSAoZm9ybWF0LnkwKSA/
IEdvQWNjZXNzLlV0aWwuZm10VmFsdWUoZFsxXSwgZm9ybWF0LnkwKSA6IGQzLmZvcm1hdCgnLCcp
KGRbMV0pO2R1YWxZYXhpcyAmJiAoZFsyXSA9IChmb3JtYXQueTEpID8gR29BY2Nlc3MuVXRpbC5m
bXRWYWx1ZShkWzJdLCBmb3JtYXQueTEpIDogZDMuZm9ybWF0KCcsJykoZFsyXSkpO3ZhciB0ZW1w
bGF0ZSA9IGQzLnNlbGVjdCgnI3RwbC1jaGFydC10b29sdGlwJykuaHRtbCgpO3JldHVybiBIb2dh
bi5jb21waWxlKHRlbXBsYXRlKS5yZW5kZXIoeydkYXRhJzogZH0pO31mdW5jdGlvbiBtb3VzZW92
ZXIoZXZlbnQsIHNlbGVjdGlvbiwgZGF0YSkge3ZhciB0b29sdGlwID0gc2VsZWN0aW9uLnNlbGVj
dCgnLmNoYXJ0LXRvb2x0aXAtd3JhcCcpO3Rvb2x0aXAuaHRtbChmb3JtYXRUb29sdGlwKGRhdGEp
KS5zdHlsZSgnbGVmdCcsIFgoZGF0YSkgKyAncHgnKS5zdHlsZSgndG9wJywgIChkMy5wb2ludGVy
KGV2ZW50KVsxXSArIDEwKSArICdweCcpLnN0eWxlKCdkaXNwbGF5JywgJ2Jsb2NrJyk7c2VsZWN0
aW9uLnNlbGVjdCgnbGluZS5pbmRpY2F0b3InKS5zdHlsZSgnZGlzcGxheScsICdibG9jaycpLmF0
dHIoJ3RyYW5zZm9ybScsICd0cmFuc2xhdGUoJyArIFgoZGF0YSkgKyAnLCcgKyAwICsgJyknKTt9
ZnVuY3Rpb24gbW91c2VvdXQoc2VsZWN0aW9uLCBnKSB7dmFyIHRvb2x0aXAgPSBzZWxlY3Rpb24u
c2VsZWN0KCcuY2hhcnQtdG9vbHRpcC13cmFwJyk7dG9vbHRpcC5zdHlsZSgnZGlzcGxheScsICdu
b25lJyk7Zy5zZWxlY3QoJ2xpbmUuaW5kaWNhdG9yJykuc3R5bGUoJ2Rpc3BsYXknLCAnbm9uZScp
O31mdW5jdGlvbiBhZGRSZWN0cyhzZWxlY3Rpb24sIGcsIGRhdGEpIHt2YXIgdyA9IChpbm5lclco
KSAvIGRhdGEubGVuZ3RoKTt2YXIgcmVjdHMgPSBnLnNlbGVjdCgnZy5yZWN0cycpLnNlbGVjdEFs
bCgncmVjdCcpLmRhdGEoZGF0YSk7dmFyIHJlY3RzRW50ZXIgPSByZWN0cy5lbnRlcigpLmFwcGVu
ZCgnc3ZnOnJlY3QnKS5hdHRyKCdoZWlnaHQnLCBpbm5lckgoKSkuYXR0cignY2xhc3MnLCAncG9p
bnQnKTtyZWN0c0VudGVyLm1lcmdlKHJlY3RzKS5hdHRyKCd3aWR0aCcsIHcpLmF0dHIoJ3gnLCBm
dW5jdGlvbihkLCBpKSB7cmV0dXJuICh3ICogaSk7fSkuYXR0cigneScsIDApLm9uKCdtb3VzZW1v
dmUnLCBmdW5jdGlvbihldmVudCkge21vdXNlb3ZlcihldmVudCwgc2VsZWN0aW9uLCBkMy5zZWxl
Y3QodGhpcykuZGF0dW0oKSk7fSkub24oJ21vdXNlbGVhdmUnLCBmdW5jdGlvbihldmVudCkge21v
dXNlb3V0KHNlbGVjdGlvbiwgZyk7fSk7cmVjdHMuZXhpdCgpLnJlbW92ZSgpO31mdW5jdGlvbiBj
aGFydChzZWxlY3Rpb24pIHtzZWxlY3Rpb24uZWFjaChmdW5jdGlvbiAoZGF0YSkge2RhdGEgPSBt
YXBEYXRhKGRhdGEpO3VwZGF0ZVNjYWxlcyhkYXRhKTtsZXQgc3ZnID0gZDMuc2VsZWN0KHRoaXMp
LnNlbGVjdCgnc3ZnJyk7aWYgKHN2Zy5lbXB0eSgpKSB7c3ZnID0gZDMuc2VsZWN0KHRoaXMpLmFw
cGVuZCgnc3ZnJykuYXR0cignd2lkdGgnLCB3aWR0aCkuYXR0cignaGVpZ2h0JywgaGVpZ2h0KTtj
cmVhdGVTa2VsZXRvbihzdmcpO312YXIgZyA9IHN2Zy5zZWxlY3QoJ2cnKS5hdHRyKCd0cmFuc2Zv
cm0nLCAndHJhbnNsYXRlKCcgKyBtYXJnaW4ubGVmdCArICcsJyArIG1hcmdpbi50b3AgKyAnKScp
O2FkZEdyaWQoZywgZGF0YSk7YWRkQXJlYUxpbmVzKGcsIGRhdGEpO2FkZFBvaW50cyhnLCBkYXRh
KTthZGRBeGlzKGcsIGRhdGEpO2FkZFJlY3RzKHNlbGVjdGlvbiwgZywgZGF0YSk7fSk7fWNoYXJ0
Lm9wdHMgPSBmdW5jdGlvbiAoXykge2lmICghYXJndW1lbnRzLmxlbmd0aCkgcmV0dXJuIG9wdHM7
b3B0cyA9IF87cmV0dXJuIGNoYXJ0O307Y2hhcnQuZm9ybWF0ID0gZnVuY3Rpb24gKF8pIHtpZiAo
IWFyZ3VtZW50cy5sZW5ndGgpIHJldHVybiBmb3JtYXQ7Zm9ybWF0ID0gXztyZXR1cm4gY2hhcnQ7
fTtjaGFydC5sYWJlbHMgPSBmdW5jdGlvbiAoXykge2lmICghYXJndW1lbnRzLmxlbmd0aCkgcmV0
dXJuIGxhYmVscztsYWJlbHMgPSBfO3JldHVybiBjaGFydDt9O2NoYXJ0Lm1hcmdpbiA9IGZ1bmN0
aW9uIChfKSB7aWYgKCFhcmd1bWVudHMubGVuZ3RoKSByZXR1cm4gbWFyZ2luO21hcmdpbiA9IF87
cmV0dXJuIGNoYXJ0O307Y2hhcnQud2lkdGggPSBmdW5jdGlvbiAoXykge2lmICghYXJndW1lbnRz
Lmxlbmd0aCkgcmV0dXJuIHdpZHRoO3dpZHRoID0gXztyZXR1cm4gY2hhcnQ7fTtjaGFydC5oZWln
aHQgPSBmdW5jdGlvbiAoXykge2lmICghYXJndW1lbnRzLmxlbmd0aCkgcmV0dXJuIGhlaWdodDto
ZWlnaHQgPSBfO3JldHVybiBjaGFydDt9O2NoYXJ0LnggPSBmdW5jdGlvbiAoXykge2lmICghYXJn
dW1lbnRzLmxlbmd0aCkgcmV0dXJuIHhWYWx1ZTt4VmFsdWUgPSBfO3JldHVybiBjaGFydDt9O2No
YXJ0LnkwID0gZnVuY3Rpb24gKF8pIHtpZiAoIWFyZ3VtZW50cy5sZW5ndGgpIHJldHVybiB5VmFs
dWUwO3lWYWx1ZTAgPSBfO3JldHVybiBjaGFydDt9O2NoYXJ0LnkxID0gZnVuY3Rpb24gKF8pIHtp
ZiAoIWFyZ3VtZW50cy5sZW5ndGgpIHJldHVybiB5VmFsdWUxO3lWYWx1ZTEgPSBfO3JldHVybiBj
aGFydDt9O3JldHVybiBjaGFydDt9ZnVuY3Rpb24gQmFyQ2hhcnQoZHVhbFlheGlzKSB7dmFyIG9w
dHMgPSB7fTt2YXIgbWFyZ2luID0ge3RvcCAgICA6IDIwLHJpZ2h0ICA6IDUwLGJvdHRvbSA6IDQw
LGxlZnQgICA6IDUwLH0saGVpZ2h0ID0gMTcwLG5UaWNrcyA9IDEwLHBhZGRpbmcgPSAxMCx3aWR0
aCA9IDc2MDt2YXIgbGFiZWxzID0geyB4OiAnVW5uYW1lZCcsIHkwOiAnVW5uYW1lZCcsIHkxOiAn
VW5uYW1lZCcgfTt2YXIgZm9ybWF0ID0geyB4OiBudWxsLCB5MDogbnVsbCwgeTE6IG51bGx9O3Zh
ciB4VmFsdWUgPSBmdW5jdGlvbiAoZCkge3JldHVybiBkWzBdO30seVZhbHVlMCA9IGZ1bmN0aW9u
IChkKSB7cmV0dXJuIGRbMV07fSx5VmFsdWUxID0gZnVuY3Rpb24gKGQpIHtyZXR1cm4gZFsyXTt9
O3ZhciB4U2NhbGUgPSBkMy5zY2FsZUJhbmQoKS5wYWRkaW5nSW5uZXIoMC4xKS5wYWRkaW5nT3V0
ZXIoMC4xKTt2YXIgeVNjYWxlMCA9IGQzLnNjYWxlTGluZWFyKCkubmljZSgpO3ZhciB5U2NhbGUx
ID0gZDMuc2NhbGVMaW5lYXIoKS5uaWNlKCk7dmFyIHhBeGlzID0gZDMuYXhpc0JvdHRvbSh4U2Nh
bGUpLnRpY2tGb3JtYXQoZnVuY3Rpb24gKGQpIHtpZiAoZm9ybWF0LngpcmV0dXJuIEdvQWNjZXNz
LlV0aWwuZm10VmFsdWUoZCwgZm9ybWF0LngpO3JldHVybiBkO30pO3ZhciB5QXhpczAgPSBkMy5h
eGlzTGVmdCh5U2NhbGUwKS50aWNrRm9ybWF0KGZ1bmN0aW9uIChkKSB7cmV0dXJuIGQzLmZvcm1h
dCgnLjJzJykoZCk7fSk7dmFyIHlBeGlzMSA9IGQzLmF4aXNSaWdodCh5U2NhbGUxKS50aWNrRm9y
bWF0KGZ1bmN0aW9uIChkKSB7aWYgKGZvcm1hdC55MSlyZXR1cm4gR29BY2Nlc3MuVXRpbC5mbXRW
YWx1ZShkLCBmb3JtYXQueTEpO3JldHVybiBkMy5mb3JtYXQoJy4ycycpKGQpO30pO3ZhciB4R3Jp
ZCA9IGQzLmF4aXNCb3R0b20oeFNjYWxlKTt2YXIgeUdyaWQgPSBkMy5heGlzTGVmdCh5U2NhbGUw
KTtmdW5jdGlvbiBpbm5lclcoKSB7cmV0dXJuIHdpZHRoIC0gbWFyZ2luLmxlZnQgLSBtYXJnaW4u
cmlnaHQ7fWZ1bmN0aW9uIGlubmVySCgpIHtyZXR1cm4gaGVpZ2h0IC0gbWFyZ2luLnRvcCAtIG1h
cmdpbi5ib3R0b207fWZ1bmN0aW9uIGdldFhUaWNrcyhkYXRhKSB7Y29uc3QgZG9tYWluID0geFNj
YWxlLmRvbWFpbigpO2lmIChkYXRhLmxlbmd0aCA8IG5UaWNrcylyZXR1cm4gZG9tYWluO3JldHVy
biBkMy5yYW5nZSgwLCBuVGlja3MpLm1hcChmdW5jdGlvbihpKSB7Y29uc3QgaW5kZXggPSBNYXRo
LmZsb29yKGkgKiAoZG9tYWluLmxlbmd0aCAtIDEpIC8gKG5UaWNrcyAtIDEpKTtpZiAoaW5kZXgg
Pj0gMCAmJiBpbmRleCA8IGRvbWFpbi5sZW5ndGgpcmV0dXJuIGRvbWFpbltpbmRleF07cmV0dXJu
IG51bGw7fSk7fWZ1bmN0aW9uIGdldFlUaWNrcyhzY2FsZSkge3ZhciBkb21haW4gPSBzY2FsZS5k
b21haW4oKTtyZXR1cm4gZDMucmFuZ2UoZG9tYWluWzBdLCBkb21haW5bMV0sIE1hdGguY2VpbChk
b21haW5bMV0gLyBuVGlja3MpKTt9ZnVuY3Rpb24gWChkKSB7cmV0dXJuICh4U2NhbGUoZFswXSkg
KyB4U2NhbGUuYmFuZHdpZHRoKCkgLyAyKTt9ZnVuY3Rpb24gbWFwRGF0YShkYXRhKSB7dmFyIF9k
YXR1bSA9IGZ1bmN0aW9uIChkLCBpKSB7dmFyIGRhdHVtID0gW3hWYWx1ZS5jYWxsKGRhdGEsIGQs
IGkpLCB5VmFsdWUwLmNhbGwoZGF0YSwgZCwgaSldO2R1YWxZYXhpcyAmJiBkYXR1bS5wdXNoKHlW
YWx1ZTEuY2FsbChkYXRhLCBkLCBpKSk7cmV0dXJuIGRhdHVtO307cmV0dXJuIGRhdGEubWFwKGZ1
bmN0aW9uIChkLCBpKSB7cmV0dXJuIF9kYXR1bShkLCBpKTt9KTt9ZnVuY3Rpb24gdXBkYXRlU2Nh
bGVzKGRhdGEpIHt4U2NhbGUuZG9tYWluKGRhdGEubWFwKGZ1bmN0aW9uIChkKSB7cmV0dXJuIGRb
MF07fSkpLnJhbmdlKFswLCBpbm5lclcoKV0pO3lTY2FsZTAuZG9tYWluKFswLCBkMy5tYXgoZGF0
YSwgZnVuY3Rpb24gKGQpIHtyZXR1cm4gZFsxXTt9KV0pLnJhbmdlKFtpbm5lckgoKSwgMF0pO2R1
YWxZYXhpcyAmJiB5U2NhbGUxLmRvbWFpbihbMCwgZDMubWF4KGRhdGEsIGZ1bmN0aW9uIChkKSB7
cmV0dXJuIGRbMl07fSkgKyAwLjFdKS5yYW5nZShbaW5uZXJIKCksIDBdKTt9ZnVuY3Rpb24gdG9n
Z2xlT3BhY2l0eShlbGUsIG9wKSB7ZDMuc2VsZWN0KGVsZS5wYXJlbnROb2RlKS5zZWxlY3RBbGwo
Jy4nICsgKGVsZS5nZXRBdHRyaWJ1dGUoJ2RhdGEteWF4aXMnKSA9PSAneTAnID8gJ3kxJyA6ICd5
MCcpKS5hdHRyKCdzdHlsZScsIG9wKTt9ZnVuY3Rpb24gc2V0TGVnZW5kTGFiZWxzKHN2Zykge3Zh
ciByZWN0ID0gc3ZnLnNlbGVjdEFsbCgncmVjdC5sZWdlbmQueTAnKS5kYXRhKFtudWxsXSk7dmFy
IHJlY3RFbnRlciA9IHJlY3QuZW50ZXIoKS5hcHBlbmQoJ3JlY3QnKS5hdHRyKCdjbGFzcycsICds
ZWdlbmQgeTAnKS5hdHRyKCdkYXRhLXlheGlzJywgJ3kwJykub24oJ21vdXNlbW92ZScsIGZ1bmN0
aW9uKGQsIGkpIHt0b2dnbGVPcGFjaXR5KHRoaXMsICdvcGFjaXR5OjAuMScpO30pLm9uKCdtb3Vz
ZWxlYXZlJywgZnVuY3Rpb24oZCwgaSkge3RvZ2dsZU9wYWNpdHkodGhpcywgbnVsbCk7fSkuYXR0
cigneScsIChoZWlnaHQgLSAxNSkpO3JlY3RFbnRlci5tZXJnZShyZWN0KS5hdHRyKCd4JywgKHdp
ZHRoIC8gMikgLSAxMDApO3ZhciB0ZXh0ID0gc3ZnLnNlbGVjdEFsbCgndGV4dC5sZWdlbmQueTAn
KS5kYXRhKFtudWxsXSk7dmFyIHRleHRFbnRlciA9IHRleHQuZW50ZXIoKS5hcHBlbmQoJ3RleHQn
KS5hdHRyKCdjbGFzcycsICdsZWdlbmQgeTAnKS5hdHRyKCdkYXRhLXlheGlzJywgJ3kwJykub24o
J21vdXNlbW92ZScsIGZ1bmN0aW9uKGQsIGkpIHt0b2dnbGVPcGFjaXR5KHRoaXMsICdvcGFjaXR5
OjAuMScpO30pLm9uKCdtb3VzZWxlYXZlJywgZnVuY3Rpb24oZCwgaSkge3RvZ2dsZU9wYWNpdHko
dGhpcywgbnVsbCk7fSkuYXR0cigneScsIChoZWlnaHQgLSA2KSk7dGV4dEVudGVyLm1lcmdlKHRl
eHQpLmF0dHIoJ3gnLCAod2lkdGggLyAyKSAtIDg1KS50ZXh0KGxhYmVscy55MCk7aWYgKCFkdWFs
WWF4aXMpcmV0dXJuO3JlY3QgPSBzdmcuc2VsZWN0QWxsKCdyZWN0LmxlZ2VuZC55MScpLmRhdGEo
W251bGxdKTt2YXIgcmVjdEVudGVyID0gcmVjdC5lbnRlcigpLmFwcGVuZCgncmVjdCcpLmF0dHIo
J2NsYXNzJywgJ2xlZ2VuZCB5MScpLmF0dHIoJ2RhdGEteWF4aXMnLCAneTEnKS5vbignbW91c2Vt
b3ZlJywgZnVuY3Rpb24oZCwgaSkge3RvZ2dsZU9wYWNpdHkodGhpcywgJ29wYWNpdHk6MC4xJyk7
fSkub24oJ21vdXNlbGVhdmUnLCBmdW5jdGlvbihkLCBpKSB7dG9nZ2xlT3BhY2l0eSh0aGlzLCBu
dWxsKTt9KS5hdHRyKCd5JywgKGhlaWdodCAtIDE1KSk7cmVjdEVudGVyLm1lcmdlKHJlY3QpLmF0
dHIoJ3gnLCAod2lkdGggLyAyKSk7dGV4dCA9IHN2Zy5zZWxlY3RBbGwoJ3RleHQubGVnZW5kLnkx
JykuZGF0YShbbnVsbF0pO3ZhciB0ZXh0RW50ZXIgPSB0ZXh0LmVudGVyKCkuYXBwZW5kKCd0ZXh0
JykuYXR0cignY2xhc3MnLCAnbGVnZW5kIHkxJykuYXR0cignZGF0YS15YXhpcycsICd5MScpLm9u
KCdtb3VzZW1vdmUnLCBmdW5jdGlvbihkLCBpKSB7dG9nZ2xlT3BhY2l0eSh0aGlzLCAnb3BhY2l0
eTowLjEnKTt9KS5vbignbW91c2VsZWF2ZScsIGZ1bmN0aW9uKGQsIGkpIHt0b2dnbGVPcGFjaXR5
KHRoaXMsIG51bGwpO30pLmF0dHIoJ3knLCAoaGVpZ2h0IC0gNikpO3RleHRFbnRlci5tZXJnZSh0
ZXh0KS5hdHRyKCd4JywgKHdpZHRoIC8gMikgKyAxNSkudGV4dChsYWJlbHMueTEpO31mdW5jdGlv
biBzZXRBeGlzTGFiZWxzKHN2Zykge3N2Zy5zZWxlY3RBbGwoJ3RleHQuYXhpcy1sYWJlbC55MCcp
LmRhdGEoW251bGxdKS5lbnRlcigpLmFwcGVuZCgndGV4dCcpLmF0dHIoJ2NsYXNzJywgJ2F4aXMt
bGFiZWwgeTAnKS5hdHRyKCd5JywgMTApLmF0dHIoJ3gnLCA1MykudGV4dChsYWJlbHMueTApO2lm
ICghZHVhbFlheGlzKSByZXR1cm47dmFyIHRFbnRlciA9IHN2Zy5zZWxlY3RBbGwoJ3RleHQuYXhp
cy1sYWJlbC55MScpLmRhdGEoW251bGxdKS5lbnRlcigpLmFwcGVuZCgndGV4dCcpLmF0dHIoJ2Ns
YXNzJywgJ2F4aXMtbGFiZWwgeTEnKS5hdHRyKCd5JywgMTApLnRleHQobGFiZWxzLnkxKTtkdWFs
WWF4aXMgJiYgdEVudGVyLmF0dHIoJ3gnLCB3aWR0aCAtIDI1KTt9ZnVuY3Rpb24gY3JlYXRlU2tl
bGV0b24oc3ZnKSB7Y29uc3QgZyA9IHN2Zy5hcHBlbmQoJ2cnKTtnLmFwcGVuZCgnZycpLmF0dHIo
J2NsYXNzJywgJ3ggZ3JpZCcpO2cuYXBwZW5kKCdnJykuYXR0cignY2xhc3MnLCAneSBncmlkJyk7
Zy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFzcycsICd4IGF4aXMnKTtnLmFwcGVuZCgnZycpLmF0dHIo
J2NsYXNzJywgJ3kwIGF4aXMnKTtkdWFsWWF4aXMgJiYgZy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFz
cycsICd5MSBheGlzJyk7Zy5hcHBlbmQoJ2cnKS5hdHRyKCdjbGFzcycsICdiYXJzIHkwJyk7ZHVh
bFlheGlzICYmIGcuYXBwZW5kKCdnJykuYXR0cignY2xhc3MnLCAnYmFycyB5MScpO2cuYXBwZW5k
KCdnJykuYXR0cignY2xhc3MnLCAncmVjdHMnKTtzZXRBeGlzTGFiZWxzKHN2Zyk7c2V0TGVnZW5k
TGFiZWxzKHN2Zyk7Zy5hcHBlbmQoJ2xpbmUnKS5hdHRyKCd5MicsIGlubmVySCgpKS5hdHRyKCd5
MScsIDApLmF0dHIoJ2NsYXNzJywgJ2luZGljYXRvcicpO31mdW5jdGlvbiBhZGRCYXJzKGcsIGRh
dGEpIHt2YXIgYmFycyA9IGcuc2VsZWN0KCdnLmJhcnMueTAnKS5zZWxlY3RBbGwoJ3JlY3QuYmFy
JykuZGF0YShkYXRhKTt2YXIgZW50ZXIgPSBiYXJzLmVudGVyKCkuYXBwZW5kKCdzdmc6cmVjdCcp
LmF0dHIoJ2NsYXNzJywgJ2JhcicpLmF0dHIoJ2hlaWdodCcsIDApLmF0dHIoJ3dpZHRoJywgZnVu
Y3Rpb24gKGQsIGkpIHsgcmV0dXJuIHhTY2FsZS5iYW5kd2lkdGgoKSAvIDI7IH0pLmF0dHIoJ3gn
LCBmdW5jdGlvbiAoZCwgaSkgeyByZXR1cm4geFNjYWxlKGRbMF0pOyB9KS5hdHRyKCd5JywgZnVu
Y3Rpb24gKGQsIGkpIHsgcmV0dXJuIGlubmVySCgpOyB9KTtiYXJzLm1lcmdlKGVudGVyKS5hdHRy
KCd3aWR0aCcsIHhTY2FsZS5iYW5kd2lkdGgoKSAvIDIpLmF0dHIoJ3gnLCBmdW5jdGlvbiAoZCkg
eyByZXR1cm4geFNjYWxlKGRbMF0pOyB9KS50cmFuc2l0aW9uKCkuZGVsYXkoZnVuY3Rpb24gKGQs
IGkpIHsgcmV0dXJuIGkgLyBkYXRhLmxlbmd0aCAqIDEwMDA7IH0pLmR1cmF0aW9uKDUwMCkuYXR0
cignaGVpZ2h0JywgZnVuY3Rpb24gKGQsIGkpIHsgcmV0dXJuIGlubmVySCgpIC0geVNjYWxlMChk
WzFdKTsgfSkuYXR0cigneScsIGZ1bmN0aW9uIChkLCBpKSB7IHJldHVybiB5U2NhbGUwKGRbMV0p
OyB9KTtiYXJzLmV4aXQoKS5yZW1vdmUoKTtpZiAoIWR1YWxZYXhpcylyZXR1cm47YmFycyA9IGcu
c2VsZWN0KCdnLmJhcnMueTEnKS5zZWxlY3RBbGwoJ3JlY3QuYmFyJykuZGF0YShkYXRhKTtlbnRl
ciA9IGJhcnMuZW50ZXIoKS5hcHBlbmQoJ3N2ZzpyZWN0JykuYXR0cignY2xhc3MnLCAnYmFyJyku
YXR0cignaGVpZ2h0JywgMCkuYXR0cignd2lkdGgnLCBmdW5jdGlvbiAoZCwgaSkgeyByZXR1cm4g
eFNjYWxlLmJhbmR3aWR0aCgpIC8gMjsgfSkuYXR0cigneCcsIGZ1bmN0aW9uIChkKSB7IHJldHVy
biAoeFNjYWxlKGRbMF0pICsgeFNjYWxlLmJhbmR3aWR0aCgpIC8gMik7IH0pLmF0dHIoJ3knLCBm
dW5jdGlvbiAoZCwgaSkgeyByZXR1cm4gaW5uZXJIKCk7IH0pO2JhcnMubWVyZ2UoZW50ZXIpLmF0
dHIoJ3dpZHRoJywgeFNjYWxlLmJhbmR3aWR0aCgpIC8gMikuYXR0cigneCcsIGZ1bmN0aW9uIChk
KSB7IHJldHVybiAoeFNjYWxlKGRbMF0pICsgeFNjYWxlLmJhbmR3aWR0aCgpIC8gMik7IH0pLnRy
YW5zaXRpb24oKS5kZWxheShmdW5jdGlvbiAoZCwgaSkgeyByZXR1cm4gaSAvIGRhdGEubGVuZ3Ro
ICogMTAwMDsgfSkuZHVyYXRpb24oNTAwKS5hdHRyKCdoZWlnaHQnLCBmdW5jdGlvbiAoZCwgaSkg
eyByZXR1cm4gaW5uZXJIKCkgLSB5U2NhbGUxKGRbMl0pOyB9KS5hdHRyKCd5JywgZnVuY3Rpb24g
KGQsIGkpIHsgcmV0dXJuIHlTY2FsZTEoZFsyXSk7IH0pO2JhcnMuZXhpdCgpLnJlbW92ZSgpO31m
dW5jdGlvbiBhZGRBeGlzKGcsIGRhdGEpIHt2YXIgeFRpY2tzID0gZ2V0WFRpY2tzKGRhdGEpO3Zh
ciB0aWNrRGlzdGFuY2UgPSB4VGlja3MubGVuZ3RoID4gMSA/ICh4U2NhbGUoeFRpY2tzWzFdKSAt
IHhTY2FsZSh4VGlja3NbMF0pKSA6IGlubmVyVygpO3ZhciBsYWJlbFcgPSB0aWNrRGlzdGFuY2Ug
LSBwYWRkaW5nO2cuc2VsZWN0KCcueC5heGlzJykuYXR0cigndHJhbnNmb3JtJywgJ3RyYW5zbGF0
ZSgwLCcgKyB5U2NhbGUwLnJhbmdlKClbMF0gKyAnKScpLmNhbGwoeEF4aXMudGlja1ZhbHVlcyh4
VGlja3MpKS5zZWxlY3RBbGwoIi50aWNrIHRleHQiKS5jYWxsKHRydW5jYXRlLCBsYWJlbFcgPiAw
ID8gbGFiZWxXIDogaW5uZXJXKCkpO2cuc2VsZWN0KCcueTAuYXhpcycpLmNhbGwoeUF4aXMwLnRp
Y2tWYWx1ZXMoZ2V0WVRpY2tzKHlTY2FsZTApKSk7aWYgKCFkdWFsWWF4aXMpcmV0dXJuO2cuc2Vs
ZWN0KCcueTEuYXhpcycpLmF0dHIoJ3RyYW5zZm9ybScsICd0cmFuc2xhdGUoJyArIGlubmVyVygp
ICsgJywgMCknKS5jYWxsKHlBeGlzMS50aWNrVmFsdWVzKGdldFlUaWNrcyh5U2NhbGUxKSkpO31m
dW5jdGlvbiBhZGRHcmlkKGcsIGRhdGEpIHtnLnNlbGVjdCgnLnguZ3JpZCcpLmF0dHIoJ3RyYW5z
Zm9ybScsICd0cmFuc2xhdGUoMCwnICsgeVNjYWxlMC5yYW5nZSgpWzBdICsgJyknKS5jYWxsKHhH
cmlkLnRpY2tWYWx1ZXMoZ2V0WFRpY2tzKGRhdGEpKS50aWNrU2l6ZSgtaW5uZXJIKCksIDAsIDAp
LnRpY2tTaXplT3V0ZXIoMCkudGlja0Zvcm1hdCgnJykpO2cuc2VsZWN0KCcueS5ncmlkJykuY2Fs
bCh5R3JpZC50aWNrVmFsdWVzKGdldFlUaWNrcyh5U2NhbGUwKSkudGlja1NpemUoLWlubmVyVygp
LCAwKS50aWNrU2l6ZU91dGVyKDApLnRpY2tGb3JtYXQoJycpKTt9ZnVuY3Rpb24gZm9ybWF0VG9v
bHRpcChkYXRhKSB7dmFyIGQgPSBkYXRhLnNsaWNlKDApO2RbMF0gPSAoZm9ybWF0LngpID8gR29B
Y2Nlc3MuVXRpbC5mbXRWYWx1ZShkWzBdLCBmb3JtYXQueCkgOiBkWzBdO2RbMV0gPSAoZm9ybWF0
LnkwKSA/IEdvQWNjZXNzLlV0aWwuZm10VmFsdWUoZFsxXSwgZm9ybWF0LnkwKSA6IGQzLmZvcm1h
dCgnLCcpKGRbMV0pO2R1YWxZYXhpcyAmJiAoZFsyXSA9IChmb3JtYXQueTEpID8gR29BY2Nlc3Mu
VXRpbC5mbXRWYWx1ZShkWzJdLCBmb3JtYXQueTEpIDogZDMuZm9ybWF0KCcsJykoZFsyXSkpO3Zh
ciB0ZW1wbGF0ZSA9IGQzLnNlbGVjdCgnI3RwbC1jaGFydC10b29sdGlwJykuaHRtbCgpO3JldHVy
biBIb2dhbi5jb21waWxlKHRlbXBsYXRlKS5yZW5kZXIoeydkYXRhJzogZH0pO31mdW5jdGlvbiBt
b3VzZW92ZXIoZXZlbnQsIHNlbGVjdGlvbiwgZGF0YSkge3ZhciB0b29sdGlwID0gc2VsZWN0aW9u
LnNlbGVjdCgnLmNoYXJ0LXRvb2x0aXAtd3JhcCcpO3Rvb2x0aXAuaHRtbChmb3JtYXRUb29sdGlw
KGRhdGEpKS5zdHlsZSgnbGVmdCcsIFgoZGF0YSkgKyAncHgnKS5zdHlsZSgndG9wJywgIChkMy5w
b2ludGVyKGV2ZW50KVsxXSArIDEwKSArICdweCcpLnN0eWxlKCdkaXNwbGF5JywgJ2Jsb2NrJyk7
c2VsZWN0aW9uLnNlbGVjdCgnbGluZS5pbmRpY2F0b3InKS5zdHlsZSgnZGlzcGxheScsICdibG9j
aycpLmF0dHIoJ3RyYW5zZm9ybScsICd0cmFuc2xhdGUoJyArIFgoZGF0YSkgKyAnLCcgKyAwICsg
JyknKTt9ZnVuY3Rpb24gbW91c2VvdXQoc2VsZWN0aW9uLCBnKSB7dmFyIHRvb2x0aXAgPSBzZWxl
Y3Rpb24uc2VsZWN0KCcuY2hhcnQtdG9vbHRpcC13cmFwJyk7dG9vbHRpcC5zdHlsZSgnZGlzcGxh
eScsICdub25lJyk7Zy5zZWxlY3QoJ2xpbmUuaW5kaWNhdG9yJykuc3R5bGUoJ2Rpc3BsYXknLCAn
bm9uZScpO31mdW5jdGlvbiBhZGRSZWN0cyhzZWxlY3Rpb24sIGcsIGRhdGEpIHt2YXIgdyA9IChp
bm5lclcoKSAvIGRhdGEubGVuZ3RoKTt2YXIgcmVjdHMgPSBnLnNlbGVjdCgnZy5yZWN0cycpLnNl
bGVjdEFsbCgncmVjdCcpLmRhdGEoZGF0YSk7dmFyIHJlY3RzRW50ZXIgPSByZWN0cy5lbnRlcigp
LmFwcGVuZCgnc3ZnOnJlY3QnKS5hdHRyKCdoZWlnaHQnLCBpbm5lckgoKSkuYXR0cignY2xhc3Mn
LCAncG9pbnQnKTtyZWN0c0VudGVyLm1lcmdlKHJlY3RzKS5hdHRyKCd3aWR0aCcsIHcpLmF0dHIo
J3gnLCBmdW5jdGlvbihkLCBpKSB7cmV0dXJuICh3ICogaSk7fSkuYXR0cigneScsIDApLm9uKCdt
b3VzZW1vdmUnLCBmdW5jdGlvbihldmVudCkge21vdXNlb3ZlcihldmVudCwgc2VsZWN0aW9uLCBk
My5zZWxlY3QodGhpcykuZGF0dW0oKSk7fSkub24oJ21vdXNlbGVhdmUnLCBmdW5jdGlvbihldmVu
dCkge21vdXNlb3V0KHNlbGVjdGlvbiwgZyk7fSk7cmVjdHMuZXhpdCgpLnJlbW92ZSgpO31mdW5j
dGlvbiBjaGFydChzZWxlY3Rpb24pIHtzZWxlY3Rpb24uZWFjaChmdW5jdGlvbiAoZGF0YSkge2Rh
dGEgPSBtYXBEYXRhKGRhdGEpO3VwZGF0ZVNjYWxlcyhkYXRhKTtsZXQgc3ZnID0gZDMuc2VsZWN0
KHRoaXMpLnNlbGVjdCgnc3ZnJyk7aWYgKHN2Zy5lbXB0eSgpKSB7c3ZnID0gZDMuc2VsZWN0KHRo
aXMpLmFwcGVuZCgnc3ZnJykuYXR0cignd2lkdGgnLCB3aWR0aCkuYXR0cignaGVpZ2h0JywgaGVp
Z2h0KTtjcmVhdGVTa2VsZXRvbihzdmcpO312YXIgZyA9IHN2Zy5zZWxlY3QoJ2cnKS5hdHRyKCd0
cmFuc2Zvcm0nLCAndHJhbnNsYXRlKCcgKyBtYXJnaW4ubGVmdCArICcsJyArIG1hcmdpbi50b3Ag
KyAnKScpO2FkZEdyaWQoZywgZGF0YSk7YWRkQXhpcyhnLCBkYXRhKTthZGRCYXJzKGcsIGRhdGEp
O2FkZFJlY3RzKHNlbGVjdGlvbiwgZywgZGF0YSk7fSk7fWNoYXJ0Lm9wdHMgPSBmdW5jdGlvbiAo
Xykge2lmICghYXJndW1lbnRzLmxlbmd0aCkgcmV0dXJuIG9wdHM7b3B0cyA9IF87cmV0dXJuIGNo
YXJ0O307Y2hhcnQuZm9ybWF0ID0gZnVuY3Rpb24gKF8pIHtpZiAoIWFyZ3VtZW50cy5sZW5ndGgp
IHJldHVybiBmb3JtYXQ7Zm9ybWF0ID0gXztyZXR1cm4gY2hhcnQ7fTtjaGFydC5sYWJlbHMgPSBm
dW5jdGlvbiAoXykge2lmICghYXJndW1lbnRzLmxlbmd0aCkgcmV0dXJuIGxhYmVscztsYWJlbHMg
PSBfO3JldHVybiBjaGFydDt9O2NoYXJ0LndpZHRoID0gZnVuY3Rpb24gKF8pIHtpZiAoIWFyZ3Vt
ZW50cy5sZW5ndGgpIHJldHVybiB3aWR0aDt3aWR0aCA9IF87cmV0dXJuIGNoYXJ0O307Y2hhcnQu
aGVpZ2h0ID0gZnVuY3Rpb24gKF8pIHtpZiAoIWFyZ3VtZW50cy5sZW5ndGgpIHJldHVybiBoZWln
aHQ7aGVpZ2h0ID0gXztyZXR1cm4gY2hhcnQ7fTtjaGFydC54ID0gZnVuY3Rpb24gKF8pIHtpZiAo
IWFyZ3VtZW50cy5sZW5ndGgpIHJldHVybiB4VmFsdWU7eFZhbHVlID0gXztyZXR1cm4gY2hhcnQ7
fTtjaGFydC55MCA9IGZ1bmN0aW9uIChfKSB7aWYgKCFhcmd1bWVudHMubGVuZ3RoKSByZXR1cm4g
eVZhbHVlMDt5VmFsdWUwID0gXztyZXR1cm4gY2hhcnQ7fTtjaGFydC55MSA9IGZ1bmN0aW9uIChf
KSB7aWYgKCFhcmd1bWVudHMubGVuZ3RoKSByZXR1cm4geVZhbHVlMTt5VmFsdWUxID0gXztyZXR1
cm4gY2hhcnQ7fTtyZXR1cm4gY2hhcnQ7fTwvc2NyaXB0PjwvYm9keT48L2h0bWw+
--000000000000c7064e0611b92fa6--


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 10:31:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 10:31:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682754.1061891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc0vg-00029Z-H3; Mon, 19 Feb 2024 10:31:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682754.1061891; Mon, 19 Feb 2024 10:31:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc0vg-00029S-ER; Mon, 19 Feb 2024 10:31:20 +0000
Received: by outflank-mailman (input) for mailman id 682754;
 Mon, 19 Feb 2024 10:31:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+22N=J4=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rc0vf-00029M-B1
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 10:31:19 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 03cdf62b-cf12-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 11:31:17 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-41262eaa95aso7646515e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 02:31:17 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 21-20020a05600c229500b0041228b2e179sm10826452wmf.39.2024.02.19.02.31.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 19 Feb 2024 02:31:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03cdf62b-cf12-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708338677; x=1708943477; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=Sp973qMCq2uetmFzE4PjtzN4lSg6l5YHUW3n6wOIGwE=;
        b=mnxj4OXbkxmgyGb0s6v++spEGZPu4lPICOHSd2G0uySqRVUzTmI2050WWV3xSmcNuH
         acweViPcFl4VTEb11Qn6s61oQVbcGCk2M418S6YKzC0UfPeW0sWmLnkSqro7V7oBZB5y
         U23XpBGNHPVeQ5H/sr5CkNPerHDlVQWwEgJiU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708338677; x=1708943477;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Sp973qMCq2uetmFzE4PjtzN4lSg6l5YHUW3n6wOIGwE=;
        b=sbL3Pta3DnNO3O/mLGSrhJr7jdhpg++Kt3YkiWk7CvewqF4jVrEJw6ndwRvUSaVAt3
         rXz1zBBoLe6Mxp228WulKmC4VDkh3XbdiuHltRUZw/TpeIkAyX/uatWJrgJktq5M9jZS
         TwxmlBlMfhK/OuNkOcPvGii1k3e6u5dHaoRWaW7/iOOOv9q+hNZrrW2JFTgtHabNkd2U
         L6qXMn8DzcGZSFcig6Qpqb9wB3u1jOofwyZXtgBbkK19RLr4xfd29WdPwpKSQR7dwiKP
         Md7kNMTezPa0eX4fArotmresDTkEtVd0+CRerrdp/KXJK3YWDt5NEf2Eos9Dczl1bNvl
         10AQ==
X-Gm-Message-State: AOJu0Yy76ds1nMfG6PPDSG+uXP5AgI6WIzT/eIk6GIEqzQIkCnDTDQP3
	kk2gidkDmKkwaEAOCnp2NqhWy7S+w15XZGDVbSXhFRJ8NoUvdo8tyL4ekQrGP9w=
X-Google-Smtp-Source: AGHT+IHWXp6GXytviRSlUZcOaYRs8YmDzGesYjIR+AvEKqGEeDzJzYGcJdtWwsKO+pnSnJySynZHtg==
X-Received: by 2002:a05:600c:1f8b:b0:412:4a57:388f with SMTP id je11-20020a05600c1f8b00b004124a57388fmr5755383wmb.15.1708338677068;
        Mon, 19 Feb 2024 02:31:17 -0800 (PST)
Date: Mon, 19 Feb 2024 11:31:15 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: George Dunlap <george.dunlap@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, committers@xenproject.org,
	Kelly Choi <kelly.choi@cloud.com>
Subject: Re: Stats on Xen tarball downloads
Message-ID: <ZdMt88BSNMYtq39n@macbook>
References: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>

On Mon, Feb 19, 2024 at 06:01:54PM +0800, George Dunlap wrote:
> Hey all,
> 
> One of the questions we had with respect to changing our release
> practice (for instance, making the process more light-weight so that
> we could do a point release after every XSA) was, "How many people are
> actually using the tarballs?"

What would this more lightweight process involve from a downstream
PoV?  IOW: in what would the contents of the tarball change compared
to the current releases?

> I finally got access (again) to
> downloads.xenproject.org, and took a look at the logs.  It appears
> that we only keep about 2 weeks of logs there.
> 
> Short answer: It's pretty clear from looking at the logs that there
> are large numbers of automated build systems building various versions
> of Xen from tarballs.  It *looks* like there are over 300 people a
> week downloading 4.18.0 specifically from various web browsers.

As someone who packages Xen for FreeBSD, I've recently switched the
build to use the git sources directly, as otherwise keeping up with
XSA tends to be a pain, specially when XSAs happen to depend on the
context of some of the backports that happened between the point
release and the XSA disclosure.

Overall as a consumer of Xen it would be helpful if we could make a
release for each (batch) or XSAs, as that would possibly make me
switch to build from the release tarballs instead of git.

I don't think it would be much of a disruption if such change to
generate more lightweight tarball is done starting from a major
release (ie: 4.19) and minor releases of previous versions (4.18.x)
are kept using the non-lightweight process.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 10:39:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 10:39:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682760.1061901 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc136-0002od-9D; Mon, 19 Feb 2024 10:39:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682760.1061901; Mon, 19 Feb 2024 10:39:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc136-0002oW-5Z; Mon, 19 Feb 2024 10:39:00 +0000
Received: by outflank-mailman (input) for mailman id 682760;
 Mon, 19 Feb 2024 10:38:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc134-0002oO-U5
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 10:38:58 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 14be030e-cf13-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 11:38:55 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-4125df8f347so11433875e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 02:38:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f9-20020a05600c44c900b00411e3cc0e0asm10708999wmo.44.2024.02.19.02.38.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 02:38:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 14be030e-cf13-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708339135; x=1708943935; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dJ+9saTvcizSh/kaxEm9YiT1T8GFT4Ezz+YzRbfjCkE=;
        b=f1gau9XTEvd71GfCYoSuGz8kM4HqTrprdoCh6WuGgTNAhfT5frUyGfYVHuZsYi3Go4
         pgQCbU1WUtrctYl5Npuw+MbCpwZH8bjW4+NnM1gYr/4jRxOvmys5HdveIOZPWZlHJTyr
         p+MWNgmXvxqavJXYcB/57tHwCW5TXr5tYd9+1aaq4clqiSHCqTko0bn15wjPtqqfEEBm
         8OAtQAayTV2o9f831DJTx/0nmuQ0QywuV69htkcKXcFiaHm7jn2yl5j9H72oXP2fEL10
         7ePMlE4sQf8QrFTK4QTjKrWQFyVEx5HhL4xE9EkEbM3ZSnQSdZIU6AoCF5gnnjylpJvB
         kLuA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708339135; x=1708943935;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dJ+9saTvcizSh/kaxEm9YiT1T8GFT4Ezz+YzRbfjCkE=;
        b=LX80c48WR1ROrlrJCgHou+CaposoR01SyAUIsX3/GjVU1GCfZydcu+Ux7UJaxMZnfQ
         9xLuzLocMvtdf5p6zomteq3wMdsAPTPSU+gSjTz91TEAKsdss7kIedGL2vbfG1hnkHuQ
         W8ObQNg7F8B8PuADcu5r8sgsU37nxsQtx9/qKwaoSkw7hXZ2QOAbulKA+n/MedU8fxWD
         IH4k658y4LJ6DYiGiFfzoPDsmtNHHkrjR+F/XznE8eEL+hBdLpGBUt++D/fEGCl68HNI
         9wHf5CKT8+mqTs50qDKsmTaQciggTVkkwdgquucqs+q7ZSyeleV3AGmWklQS969cJOYg
         0jPw==
X-Gm-Message-State: AOJu0Yy/etomE6Vxq+DXEWV9LYv+cuTmqTjbCQGDDNhkn9Z4cD+9GEwI
	kjEdi8uzH/eHDLxysFmrvCgCrgA+SsKJKOg+ITYC3kBl3DqYay/0/2B1uDt7GA==
X-Google-Smtp-Source: AGHT+IH/7fw7xnboIqUa2Iw8Ff9KwcgTv0f2jT3SH94o252EFByXjlyUUxNJPpganmGldbORipTg4Q==
X-Received: by 2002:a05:600c:4ece:b0:412:6101:915c with SMTP id g14-20020a05600c4ece00b004126101915cmr2398725wmq.19.1708339135126;
        Mon, 19 Feb 2024 02:38:55 -0800 (PST)
Message-ID: <36229344-e2e2-46e1-839b-b7b3b0843f23@suse.com>
Date: Mon, 19 Feb 2024 11:38:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Stats on Xen tarball downloads
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 George Dunlap <george.dunlap@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, committers@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>
References: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
 <ZdMt88BSNMYtq39n@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZdMt88BSNMYtq39n@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 19.02.2024 11:31, Roger Pau Monné wrote:
> On Mon, Feb 19, 2024 at 06:01:54PM +0800, George Dunlap wrote:
>> One of the questions we had with respect to changing our release
>> practice (for instance, making the process more light-weight so that
>> we could do a point release after every XSA) was, "How many people are
>> actually using the tarballs?"
> 
> What would this more lightweight process involve from a downstream
> PoV?  IOW: in what would the contents of the tarball change compared
> to the current releases?

>From all prior discussion my conclusion was "no tarball at all".

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 11:13:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 11:13:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682767.1061911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc1aX-00082I-Ry; Mon, 19 Feb 2024 11:13:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682767.1061911; Mon, 19 Feb 2024 11:13:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc1aX-00082B-Oq; Mon, 19 Feb 2024 11:13:33 +0000
Received: by outflank-mailman (input) for mailman id 682767;
 Mon, 19 Feb 2024 11:13:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DP9f=J4=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1rc1aW-000825-Bn
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 11:13:32 +0000
Received: from wfhigh1-smtp.messagingengine.com
 (wfhigh1-smtp.messagingengine.com [64.147.123.152])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e7e1e7ee-cf17-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 12:13:29 +0100 (CET)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailfhigh.west.internal (Postfix) with ESMTP id DD1B0180006E;
 Mon, 19 Feb 2024 06:13:25 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute1.internal (MEProxy); Mon, 19 Feb 2024 06:13:26 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 19 Feb 2024 06:13:23 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e7e1e7ee-cf17-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1708341205;
	 x=1708427605; bh=YV1OZ/13ji+yvYE/2Z7Ylqtj9sAs1WitQLDbBMED/jc=; b=
	T76nZuRRrrLBk5GwrS9nHrpk3Z+GX11Rj5ip+cqxbCz65TTMguJ+XZFTz2Hg2t1f
	xlRZjjCPfbd+IUb5fbodnUOMy13Nby+hhsu5YlndTl2ljs99JgD9YN+KXaUvfi6C
	kftCUxxkBxdZV6oC8w65Vj4V9VLzG6TI/Zie8+xKsdKzgIVZ/crEU68Jfsxg7p8Q
	bQkAt8u+dsKKumjDYhWXOJikjmd27xFBELgzqE/B+ULuHIS7z94Woi51pkWpQ7v8
	sYRz3Cr8st5xEOCJz/RA1pravLtOVP+CG/xsFu+jD8U76u+PQwg+SQTgyMNeIGlk
	jokBmHwJbjlPqCHNEG+vag==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1708341205; x=1708427605; bh=YV1OZ/13ji+yvYE/2Z7Ylqtj9sAs
	1WitQLDbBMED/jc=; b=e3QvTleVRNgoEpioasp20JUmRu7xZ5baMwCYgER9ulvU
	2dOvNlo/A9n2Jwr7DZBA69D7LFqxuKA66ZsfyrjvIjWR8uUomGlnI2lZFZDCR43c
	Yd8l2LVNrVws4vNNsJZu6CPc4wCd6hgT5EhWowbKEdzpb/0IV/g7BmXZqfHTzS8j
	lpeazkxCHu08jgsiX6OM/QcxNjTxl3r0lfN3GIUwQ1s4WFQQM8lGw4WkJ2zZ0lCt
	01fG2eRk6bLCo6YnqVCmCilF7yFTzPYR86k0YRWQnehHuA/MkfGVZoAW+oOUDybP
	oVXrPFEka7NELZmnmDuoZfccOFIlzVOxIaqzy5XpxA==
X-ME-Sender: <xms:1DfTZVsV5_eQKjlK17znjzB4f8iFrf7gyG7do3O-YZ8qRZk2Y-SddA>
    <xme:1DfTZef1bQwIoH4pMj4IGofSneXZdomHs463ugyUfFcgSF6P1aBW62WdGnDtUopIv
    PiRvOHM24DU7g>
X-ME-Received: <xmr:1DfTZYyVExCKXNJcioXWBpfGHeg-_zlILLwWWpva1TzizmSioyy06r88FBE6CHTCrLtBDnS2m1nq59LMKk0tFHM36GzwRyYxdg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdekgddvgecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpeffhffvvefukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefgudel
    teefvefhfeehieetleeihfejhfeludevteetkeevtedtvdegueetfeejudenucevlhhush
    htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhes
    ihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:1TfTZcPgpfqVkxlW2XqcD7dEdS1vXsy0Nt8lMuifrwDz8Jlh-gkyUw>
    <xmx:1TfTZV-pl0pWvQOnYY6lxehr70qCG5XSrLDZdWyoyEFnncMM83xBDA>
    <xmx:1TfTZcUTOQu4bg-duyJmhnishJvNRZT4M50FYaIQoltzB1nczHw7-g>
    <xmx:1TfTZVyHgDh0L9Uefr5YtdbONhNb0PKUrYFNt_6rbQSzmR6NLWdbycCa5iE>
Feedback-ID: i1568416f:Fastmail
Date: Mon, 19 Feb 2024 12:13:18 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] ns16550: add Asix AX99100 serial card
Message-ID: <ZdM30PNnmlkwW2ZG@mail-itl>
References: <20240218013431.274451-1-marmarek@invisiblethingslab.com>
 <7b60cf83-4d08-46ef-a937-911e42fd9e5d@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="Q6sRfocV+a/6R/xc"
Content-Disposition: inline
In-Reply-To: <7b60cf83-4d08-46ef-a937-911e42fd9e5d@suse.com>


--Q6sRfocV+a/6R/xc
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Mon, 19 Feb 2024 12:13:18 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] ns16550: add Asix AX99100 serial card

On Mon, Feb 19, 2024 at 09:57:49AM +0100, Jan Beulich wrote:
> On 18.02.2024 02:34, Marek Marczykowski-G=C3=B3recki wrote:
> > @@ -1170,6 +1177,11 @@ static const struct ns16550_config __initconst u=
art_config[] =3D
> >          .dev_id =3D 0x7adc,
> >          .param =3D param_intel_lpss
> >      },
> > +    {
> > +        .vendor_id =3D PCI_VENDOR_ID_ASIX,
> > +        .dev_id =3D 9100,
>=20
> As per Linux this is 0x9100.

Right... but then, maybe the patch isn't needed at all, as it does work
for me. Maybe what's needed instead is some other patch already in
staging. Initial attempt that did not work was with 4.17.something.
I guess setting the fifo size isn't that important.

> > +        .param =3D param_asix_ax99100
> > +    },
> >  };
> > =20
> >  static int __init
> > diff --git a/xen/include/xen/pci_ids.h b/xen/include/xen/pci_ids.h
> > index e798477a7e23..2a19f4ab7872 100644
> > --- a/xen/include/xen/pci_ids.h
> > +++ b/xen/include/xen/pci_ids.h
> > @@ -11,3 +11,5 @@
> >  #define PCI_VENDOR_ID_BROADCOM           0x14e4
> > =20
> >  #define PCI_VENDOR_ID_INTEL              0x8086
> > +
> > +#define PCI_VENDOR_ID_ASIX               0x125b
>=20
> Please insert such that numeric sorting is retained.
>=20
> Jan

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--Q6sRfocV+a/6R/xc
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXTN9AACgkQ24/THMrX
1yx19Qf+Pm0vIbG+tD/SXB+dhZBoJ3xa4DDVrUtP/qy7gztFY6G3Kw+ugkVQvON2
5qdQf14HzPiaLJyTTGTnw9NYx4KoxT3VIGNhdx2qCMCgz25jGoNVXSTjsVibweJM
vQPT1iTdJbnrUEUAFBTtAhpmUvSF1XZmKI3q6wPg7uvQbNWOSGE1HY9PGUlPrExP
a1Al2UsmWxNyeXypgcLZkEZh/8bJqOdBtkRrpLatLTN8/STy4+pSzYdlP6hAf8GT
/rrYt/6mZruhXW+pIIftN+qO6LOGekMdqZ1EOFXZrWSZWEGBtsjSZuoZXeKbtz8P
vVHwCUyPKM+YIrbtbgPPQG/kQ6x1WQ==
=9bWI
-----END PGP SIGNATURE-----

--Q6sRfocV+a/6R/xc--


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 11:22:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 11:22:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682773.1061921 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc1jI-0001M9-NH; Mon, 19 Feb 2024 11:22:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682773.1061921; Mon, 19 Feb 2024 11:22:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc1jI-0001M2-Jr; Mon, 19 Feb 2024 11:22:36 +0000
Received: by outflank-mailman (input) for mailman id 682773;
 Mon, 19 Feb 2024 11:22:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc1jH-0001Lv-AD
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 11:22:35 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2ccd4605-cf19-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 12:22:32 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d23114b19dso15200591fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 03:22:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 t18-20020a05600c451200b0040fd1629443sm11197016wmo.18.2024.02.19.03.22.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 03:22:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ccd4605-cf19-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708341752; x=1708946552; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pfPxolcSGvkU83gfDkTy6gFgXMHnooKhqzLG+gYAG5I=;
        b=HNV8kr/6LondVT2fR0zqHIMC8kPjj4pJDWUQh9F3UL4cof3rMtbseC0lEkylnR0P7o
         vDxLHtEgh9GXTt2tYQLBt0LpcpNNL64wmBrK+lfjmby9HM+aYedxTQioKfWPB1v0tMl1
         0C1IMCOoj5wEAbe/HxrVVB8xqu7BvPU/CQycwt/8FA1RgFgJJNUfny+SretsI/OHzMSk
         jLoSnRBt4hv9/3iwMyQ+IpxsQ+bJlOR9z2sUoXM7t6b5rbHiMYS7JRdS5mxZg9qRlKIf
         gLa27dFkHxv1uvhoxjxONFaATFqXrJDEOeHcBKCH9p9G4R9YAva6WwET7e+IrPjwHrJ8
         4OYQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708341752; x=1708946552;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pfPxolcSGvkU83gfDkTy6gFgXMHnooKhqzLG+gYAG5I=;
        b=IoEsJOngJ2pjs0QUNObfNH6jRaohGR+V7OzDFai6k7NlRkiabF275+MMBm3eQ7d04X
         Ro5Puwl79M8gG1s/fLa/VMEaF3jAQOvG4TH2F8XvNeHP6PlF0fj6tfsjsS3ckjh8wUsK
         1zlkkGLwhRPXCc+YJi1GrcgHFywM77dJ4T0MLA26eSuNNVlOcWbeQezVK+K3kUPxUmOS
         bOISo5xzozbmqQvaCaq032j/eNhHFJWRgL37XnpBuwFlg2PSQgYFHeOg1VoxKf59C397
         NNg59XPg74LqygadtuQNp6CVHQ4vvDo5X8k0S+DlAZ1fXN4BGVkvYZNKieeEWjbgPmvv
         e/qA==
X-Forwarded-Encrypted: i=1; AJvYcCVN4EM6uvFD663F08wc7AlYfmldiOgTkHePiWTQlRotC3/rigs4fafA9qU2cljtxck/i74/uyMs70qZCKU4w0HhTx2x+Kh6kLCTydXrvlQ=
X-Gm-Message-State: AOJu0YwL9sSngoK4Sr6UP4ML7ummIwTXmJvQSmjDNRx9xutvzIzScyS+
	SdIJ/qyk7vAOWjbZSrchhMvn+uGh586F4BGidBhXtwCoqFPL0VQt4+t8PMTIvRtgUto+3Pl+hLQ
	=
X-Google-Smtp-Source: AGHT+IHIOEQx3O40dIDP9N8/4SnuFyNe/i6UTB+Wa0fe974lV2YMo3ukMB7PFAmDfR+Ih4HNMCu5Zg==
X-Received: by 2002:a05:651c:626:b0:2d2:21d8:bd6a with SMTP id k38-20020a05651c062600b002d221d8bd6amr4710890lje.44.1708341752297;
        Mon, 19 Feb 2024 03:22:32 -0800 (PST)
Message-ID: <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
Date: Mon, 19 Feb 2024 12:22:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
 <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 15.02.2024 14:41, Oleksii wrote:
>>> +        : "=r" (ret), "+A" (*ptr) \
>>> +        : "r" (new) \
>>> +        : "memory" ); \
>>> +})
>>> +
>>> +#define emulate_xchg_1_2(ptr, new, ret, release_barrier,
>>> acquire_barrier) \
>>> +({ \
>>> +    uint32_t *ptr_32b_aligned = (uint32_t *)ALIGN_DOWN((unsigned
>>> long)ptr, 4); \
>>
>> You now appear to assume that this macro is only used with inputs not
>> crossing word boundaries. That's okay as long as suitably guaranteed
>> at the use sites, but imo wants saying in a comment.
>>
>>> +    uint8_t mask_l = ((unsigned long)(ptr) & (0x8 - sizeof(*ptr)))
>>> * BITS_PER_BYTE; \
>>
>> Why 0x8 (i.e. spanning 64 bits), not 4 (matching the uint32_t use
>> above)?
> The idea to read 8 bytes was to deal with crossing word boundary. So if
> our address is 0x3 and we have to xchg() 2 bytes, what will cross 4
> byte boundary. Instead we align add 0x3, so it will become 0x0 and then
> just always work with 8 bytes.

Then what if my 2-byte access crosses a dword boundary? A cache line
one? A page one?

>>> +    unsigned long new_ = (unsigned long)(new) << mask_l; \
>>> +    unsigned long ret_; \
>>> +    unsigned long rc; \
>>
>> Similarly, why unsigned long here?
> sizeof(unsigned long) is 8 bytes and it was chosen as we are working
> with lc/sc.d which are working with 8 bytes.
> 
>>
>> I also wonder about the mix of underscore suffixed (or not) variable
>> names here.
> If the question about ret_, then the same as before size of ret
> argument of the macros will be 1 or 2, but {lc/sc}.d expected to work
> with 8 bytes.

Then what's the uint32_t * about?

>>> +        release_barrier \
>>> +        "0: lr.d %0, %2\n" \
>>
>> Even here it's an 8-byte access. Even if - didn't check - the insn
>> was
>> okay to use with just a 4-byte aligned pointer, wouldn't it make
>> sense
>> then to 8-byte align it, and be consistent throughout this macro wrt
>> the base unit acted upon? Alternatively, why not use lr.w here, thus
>> reducing possible collisions between multiple CPUs accessing the same
>> cache line?
> According to the docs:
> LR and SC operate on naturally-aligned 64-bit (RV64 only) or 32-bit
> words in memory. Misaligned
> addresses will generate misaligned address exceptions.
> 
> My intention was to deal with 4-byte crossing boundary. so if ptr is 4-
> byte aligned then by reading 8-bytes we shouldn't care about boundary
> crossing, if I am not missing something.

If a ptr is 4-byte aligned, there's no point reading more than 4 bytes.

>>> +        "   and  %1, %0, %z4\n" \
>>> +        "   or   %1, %1, %z3\n" \
>>> +        "   sc.d %1, %1, %2\n" \
>>> +        "   bnez %1, 0b\n" \
>>> +        acquire_barrier \
>>> +        : "=&r" (ret_), "=&r" (rc), "+A" (*ptr_32b_aligned) \
>>> +        : "rJ" (new_), "rJ" (~mask) \
>>
>> I think that as soon as there are more than 2 or maybe 3 operands,
>> legibility is vastly improved by using named asm() operands.
> Just to clarify you mean that it would be better to use instead of %0
> use names?

Yes. Just like you have it in one of the other patches that I looked at
later.

>>> +        : "memory"); \
>>
>> Nit: Missing blank before closing parenthesis.
>>
>>> +    \
>>> +    ret = (__typeof__(*(ptr)))((ret_ & mask) >> mask_l); \
>>> +})
>>
>> Why does "ret" need to be a macro argument? If you had only the
>> expression here, not the the assigment, ...
>>
>>> +#define __xchg_generic(ptr, new, size, sfx, release_barrier,
>>> acquire_barrier) \
>>> +({ \
>>> +    __typeof__(ptr) ptr__ = (ptr); \
>>
>> Is this local variable really needed? Can't you use "ptr" directly
>> in the three macro invocations?
>>
>>> +    __typeof__(*(ptr)) new__ = (new); \
>>> +    __typeof__(*(ptr)) ret__; \
>>> +    switch (size) \
>>> +    { \
>>> +    case 1: \
>>> +    case 2: \
>>> +        emulate_xchg_1_2(ptr__, new__, ret__, release_barrier,
>>> acquire_barrier); \
>>
>> ... this would become
>>
>>         ret__ = emulate_xchg_1_2(ptr__, new__, release_barrier,
>> acquire_barrier); \
>>
>> But, unlike assumed above, there's no enforcement here that a 2-byte
>> quantity won't cross a word, double-word, cache line, or even page
>> boundary. That might be okay if then the code would simply crash
>> (like
>> the AMO insns emitted further down would), but aiui silent
>> misbehavior
>> would result.
> As I mentioned above with 4-byte alignment and then reading and working
> with 8-byte then crossing a word or double-word boundary shouldn't be
> an issue.
> 
> I am not sure that I know how to check that we are crossing cache line
> boundary.
> 
> Regarding page boundary, if the next page is mapped then all should
> work fine, otherwise it will be an exception.

Are you sure lr.d / sc.d are happy to access across such a boundary,
when both pages are mapped?

To me it seems pretty clear that for atomic accesses you want to
demand natural alignment, i.e. 2-byte alignment for 2-byte accesses.
This way you can be sure no potentially problematic boundaries will
be crossed.

>>> +        break; \
>>> +    case 4: \
>>> +        __amoswap_generic(ptr__, new__, ret__,\
>>> +                          ".w" sfx,  release_barrier,
>>> acquire_barrier); \
>>> +        break; \
>>> +    case 8: \
>>> +        __amoswap_generic(ptr__, new__, ret__,\
>>> +                          ".d" sfx,  release_barrier,
>>> acquire_barrier); \
>>> +        break; \
>>> +    default: \
>>> +        STATIC_ASSERT_UNREACHABLE(); \
>>> +    } \
>>> +    ret__; \
>>> +})
>>> +
>>> +#define xchg_relaxed(ptr, x) \
>>> +({ \
>>> +    __typeof__(*(ptr)) x_ = (x); \
>>> +    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)),
>>> "", "", ""); \
>>> +})
>>> +
>>> +#define xchg_acquire(ptr, x) \
>>> +({ \
>>> +    __typeof__(*(ptr)) x_ = (x); \
>>> +    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)), \
>>> +                                       "", "",
>>> RISCV_ACQUIRE_BARRIER); \
>>> +})
>>> +
>>> +#define xchg_release(ptr, x) \
>>> +({ \
>>> +    __typeof__(*(ptr)) x_ = (x); \
>>> +    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)),\
>>> +                                       "", RISCV_RELEASE_BARRIER,
>>> ""); \
>>> +})
>>> +
>>> +#define xchg(ptr,x) \
>>> +({ \
>>> +    __typeof__(*(ptr)) ret__; \
>>> +    ret__ = (__typeof__(*(ptr))) \
>>> +            __xchg_generic(ptr, (unsigned long)(x),
>>> sizeof(*(ptr)), \
>>> +                           ".aqrl", "", ""); \
>>
>> The .aqrl doesn't look to affect the (emulated) 1- and 2-byte cases.
>>
>> Further, amoswap also exists in release-only and acquire-only forms.
>> Why do you prefer explicit barrier insns over those? (Looks to
>> similarly apply to the emulation path as well as to the cmpxchg
>> machinery then, as both lr and sc also come in all four possible
>> acquire/release forms. Perhaps for the emulation path using
>> explicit barriers is better, in case the acquire/release forms of
>> lr/sc - being used inside the loop - might perform worse.)
> As 1- and 2-byte cases are emulated I decided that is not to provide
> sfx argument for emulation macros as it will not have to much affect on
> emulated types and just consume more performance on acquire and release
> version of sc/ld instructions.

Question is whether the common case (4- and 8-byte accesses) shouldn't
be valued higher, with 1- and 2-byte emulation being there just to
allow things to not break altogether.

>> No RISCV_..._BARRIER for use here and ...
>>
>>> +    ret__; \
>>> +})
>>> +
>>> +#define __cmpxchg(ptr, o, n, s) \
>>> +({ \
>>> +    __typeof__(*(ptr)) ret__; \
>>> +    ret__ = (__typeof__(*(ptr))) \
>>> +            __cmpxchg_generic(ptr, (unsigned long)(o), (unsigned
>>> long)(n), \
>>> +                              s, ".rl", "", " fence rw, rw\n"); \
>>
>> ... here? And anyway, wouldn't it make sense to have
>>
>> #define cmpxchg(ptr, o, n) __cmpxchg(ptr, o, n, sizeof(*(ptr))
>>
>> to limit redundancy?
>>
>> Plus wouldn't
>>
>> #define __cmpxchg(ptr, o, n, s) \
>>     ((__typeof__(*(ptr))) \
>>      __cmpxchg_generic(ptr, (unsigned long)(o), (unsigned long)(n), \
>>                        s, ".rl", "", " fence rw, rw\n"))
>>
>> be shorter and thus easier to follow as well? As I notice only now,
>> this would apparently apply further up as well.
> I understand your point about "#define cmpxchg(ptr, o, n) __cmpxchg(",
> but I can't undestand how the definition of __cmxchng should be done
> shorter. Could you please clarify that?

You did notice that in my form there's no local variable, and hence
also no macro-wide scope ( ({ ... }) )?

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 11:26:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 11:26:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682778.1061930 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc1mx-0001za-9W; Mon, 19 Feb 2024 11:26:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682778.1061930; Mon, 19 Feb 2024 11:26:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc1mx-0001zT-6o; Mon, 19 Feb 2024 11:26:23 +0000
Received: by outflank-mailman (input) for mailman id 682778;
 Mon, 19 Feb 2024 11:26:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc1mv-0001zN-Av
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 11:26:21 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b3b26fc7-cf19-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 12:26:19 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-41264195d5cso8134765e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 03:26:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i13-20020a5d55cd000000b0033b198efbedsm10022628wrw.15.2024.02.19.03.26.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 03:26:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b3b26fc7-cf19-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708341979; x=1708946779; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=afWW+G268VY4M/2GXCAHh58cIFj4BYx0rMtuKRAIJTE=;
        b=EBOdzrTjKf7HrFVNnzNp0GeYmM8R9ZrY6ycBUxKaWl1gG4f2eU8AfI4AHKsVMF5sSJ
         +QREBCT7NTzYUxR3Lb0EyeGpyiyxWhmBuPk7ZaOIC9KB/ij/b/sJdm/8aV55FORxmMEa
         Lc5L22C20wT+rXExLrQo7CrajqegZW+aFY/RxuqnG924aKwAOdtqECK8b4LXoRneOjqv
         MtEAVw2QP22aDD31YppZcyNNeqgBuVHRpRiyygruWzg2DnpECodCC+J0cXiOd2ncYDH3
         0yjrD/culpMr1U8G7YDMtNmRNR/qqPuMcsomDFqKa29+SN95hUbYza38eE4bShjCmVUN
         wgWQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708341979; x=1708946779;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=afWW+G268VY4M/2GXCAHh58cIFj4BYx0rMtuKRAIJTE=;
        b=qmSQBAd1wBPly7/ZSGnmTYYCGRn5aNOAFBEIzbOLGL9Q4qhZVUc/fi9Ip2onfgDTFK
         KLe4FnaepFSZEeHb5udfY0wMaeKZ0/na83Indxy6UWt+lL9uGsgVIXlY3PirW+MoXhV0
         c/OSwD6MoVm8k+OBhdOJeyQCfU5OqwO38zeVAqz0n+cE9DF2AtfD2r7Kxg0v39/h1+4C
         1yfpdEKUevaMqJnESmLumcHsY3RS6cdRG2bfjKOqXZbq+YsZXgsThEWxbOA6EuPCLoCI
         fOAVESsWC1hrJG755tZFpk744CZLY4kQMGLKrXfRfhARRihaQ0pEsDgAtorRYcnGpt0M
         ZNfA==
X-Forwarded-Encrypted: i=1; AJvYcCX0ypyYhdauMfAOSUlLzV/YkBCgbYn1tD4KpEBOT5vsFKRMhMKTDbnV+n4TpvfOoepXCZYJDZlcOix9HZHNRS4SS7XWXByGPNk+BnePlmA=
X-Gm-Message-State: AOJu0YyTogZS4rhFNC1U56axajOQJpTTqTfye12iX+vvHNFwmEmBHj3m
	gUAF4F93ecyBowo9nn+H325b6L7Kf7TJJcJYAlEfATkSO+/OHAZ80RXPmXxztQ==
X-Google-Smtp-Source: AGHT+IGmbzwBQSm2AobfSUWydgb5CevYYiTMc1MHKJwTsR4fTmgldtEitVyXlXKc65FuR05e+Bv48A==
X-Received: by 2002:a5d:6e65:0:b0:33d:30af:4623 with SMTP id j37-20020a5d6e65000000b0033d30af4623mr3475850wrz.51.1708341978713;
        Mon, 19 Feb 2024 03:26:18 -0800 (PST)
Message-ID: <92e7c679-fbf5-4f2e-9b0d-ba61a198832c@suse.com>
Date: Mon, 19 Feb 2024 12:26:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 7/7] xen/asm-generic: fold struct devarch into struct
 dev
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Julien Grall <julien@xen.org>, Rahul Singh <rahul.singh@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
 <3a5bf394a9d95a28cecac996f6e0decb788c19fd.1708086092.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3a5bf394a9d95a28cecac996f6e0decb788c19fd.1708086092.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.02.2024 13:39, Oleksii Kurochko wrote:
> The current patch is a follow-up to the patch titled:
>     xen/asm-generic: introduce generic device.h
> Also, a prerequisite for this patch is, without which a compilation
> error will occur:
>     xen/arm: switch Arm to use asm-generic/device.h
> 
> The 'struct dev_archdata' is exclusively used within 'struct device',
> so it could be merged into 'struct device.'
> 
> After the merger, it is necessary to update the 'dev_archdata()'
> macros and the comments above 'struct arm_smmu_xen_device' in
> drivers/passthrough/arm/smmu.c.
> Additionally, it is required to update instances of
> "dev->archdata->iommu" to "dev->iommu".
> 
> Suggested-by: Julien Grall <julien@xen.org>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>   This patch can be merged with patches 4 and 5 of this patch series.
> ---
> Changes in V9:
>  - newly introduced patch.
> ---
>  xen/drivers/passthrough/arm/smmu.c | 12 ++++++------
>  xen/include/asm-generic/device.h   |  8 +-------
>  2 files changed, 7 insertions(+), 13 deletions(-)
> 
> diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
> index 32e2ff279b..4a272c8779 100644
> --- a/xen/drivers/passthrough/arm/smmu.c
> +++ b/xen/drivers/passthrough/arm/smmu.c
> @@ -227,9 +227,9 @@ struct arm_smmu_xen_domain {
>  };
>  
>  /*
> - * Xen: Information about each device stored in dev->archdata.iommu
> + * Xen: Information about each device stored in dev->iommu
>   *
> - * Initially dev->archdata.iommu only stores the iommu_domain (runtime
> + * Initially dev->iommu only stores the iommu_domain (runtime
>   * configuration of the SMMU) but, on Xen, we also have to store the
>   * iommu_group (list of streamIDs associated to the device).
>   *
> @@ -242,7 +242,7 @@ struct arm_smmu_xen_device {
>  	struct iommu_group *group;
>  };
>  
> -#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->archdata.iommu)
> +#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->iommu)

I find in particular the naming here odd. But I'll let Julien judge whether
this really is along the lines of what he suggested.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 11:34:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 11:34:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682784.1061941 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc1ur-0003tM-49; Mon, 19 Feb 2024 11:34:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682784.1061941; Mon, 19 Feb 2024 11:34:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc1ur-0003tF-15; Mon, 19 Feb 2024 11:34:33 +0000
Received: by outflank-mailman (input) for mailman id 682784;
 Mon, 19 Feb 2024 11:34:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=EdXM=J4=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1rc1uq-0003t9-F1
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 11:34:32 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d91a7079-cf1a-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 12:34:31 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-5129cdae3c6so2515426e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 03:34:31 -0800 (PST)
Received: from localhost.localdomain (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 i23-20020a05640200d700b005648745b23bsm567412edu.90.2024.02.19.03.34.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 19 Feb 2024 03:34:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d91a7079-cf1a-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708342470; x=1708947270; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=zuFdWH4GCSkQ1e+xmjUr9ZfgaI6pfJr2DHBQlULfx2w=;
        b=V0uAc++GCSwHPKGF5ef/vBDKcl4eZo4E6JXnPx6nJSnMES8sCMU+z3Dgvh6VfFrAQ1
         YR54+K3AdrZ0cViSzo9x8RMz9q9UWamsSeDnOgOw3CxFXBj16wL78KgrLNtYVRQMg9zi
         wdPiKNSzHZTjVGmi/V6/qHHU2ZSW/9IOGiyBY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708342470; x=1708947270;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=zuFdWH4GCSkQ1e+xmjUr9ZfgaI6pfJr2DHBQlULfx2w=;
        b=P2mmpAIj12YoTcuPyLhdefykALvAkZCuz66mngQ287nZ52KzTsXOgvLaVLc5IizPhy
         CFNkw6E6dlaZYYRjfu9DRvXPc1FUNraWRQqva8Ijl3PCAl4f5B4U4PCQoExvwpREaXhG
         dqqepzfzL7OoZZPJEss88AQqJbMKn2dbnOb1H0jxkigp6ZY7tecSMT0v5NNdbbNBQpur
         NwRpsGxeDdc6xnsgoXH0mgBOhpqQjWp9zJyAI1Ripnpp7M1LIid9/N+6hr8dppXRu2Xe
         I+ISP7ZHGyYpW3nLTTsL/plvXLTvMiBjA64g2om69wnkoAwqlE8ttjfxBlZflNHFsGuM
         BRhQ==
X-Gm-Message-State: AOJu0Yw7cvWYMaZeANVBfLDS7nLdHdMWFnmb/zptclnjBb+tdV2cCEdq
	71cdB5hxxCQbeBq3OtuQRM3rjdqglvwfqzoSb7TzOrs65DgnSbYu74prQChJq4n6KAgFnGqOPIt
	v9Ug=
X-Google-Smtp-Source: AGHT+IHuumVYhHP5Ydeb8lviwRaB+4jWX8EqYpu+eD4O2eYJUrhz2LHSDjOONQlrHzrcY63Og9Clqg==
X-Received: by 2002:ac2:55a7:0:b0:512:ac0c:2f81 with SMTP id y7-20020ac255a7000000b00512ac0c2f81mr2194870lfg.40.1708342470555;
        Mon, 19 Feb 2024 03:34:30 -0800 (PST)
From: Frediano Ziglio <frediano.ziglio@cloud.com>
To: xen-devel@lists.xenproject.org
Cc: Frediano Ziglio <frediano.ziglio@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v4] Reduce assembly code size of entry points
Date: Mon, 19 Feb 2024 11:34:12 +0000
Message-Id: <20240219113412.18445-1-frediano.ziglio@cloud.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

On many entries we push 8-bytes zero and exception constants are
small so we can just write a single byte saving 3 bytes for
instruction.
With ENDBR64 this reduces the size of many entry points from 32 to
16 bytes (due to alignment).
The push and the mov are overlapping stores either way.  Swapping
between movl and movb will make no difference at all on performance.
Similar code is already used in autogen_stubs.

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
--
v4:
- minor space adjustments;
- quoted possible error message;
- update commit subject removing "exception";
- added "Reviewed-by", I hope to got everything asked for.

v3:
- add other missing entries;
- reduce code when TRAP_syscall is used;
- improved commit message.

v2:
- added missing entry points;
- add mention to autogen_stubs code, as suggested.
---
 xen/arch/x86/x86_64/entry.S | 66 ++++++++++++++++++++++---------------
 1 file changed, 40 insertions(+), 26 deletions(-)

diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index ecdd6e5b47..a7bd8f0ca5 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -22,6 +22,14 @@
 #endif
 .endm
 
+.macro BUILD_BUG_ON condstr cond:vararg
+        .if \cond
+        .error "Condition \"\condstr\" not satisfied"
+        .endif
+.endm
+/* preprocessor macro to make error message more user friendly */
+#define BUILD_BUG_ON(cond) BUILD_BUG_ON #cond cond
+
 #ifdef CONFIG_PV
 /* %rbx: struct vcpu */
 FUNC_LOCAL(switch_to_kernel)
@@ -187,7 +195,8 @@ FUNC_LOCAL(restore_all_guest)
         SPEC_CTRL_EXIT_TO_PV    /* Req: a=spec_ctrl %rsp=regs/cpuinfo, Clob: cd */
 
         RESTORE_ALL
-        testw $TRAP_syscall,4(%rsp)
+        BUILD_BUG_ON(TRAP_syscall & 0xff)
+        testb $TRAP_syscall >> 8, 4+1(%rsp)
         jz    iret_exit_to_guest
 
         movq  24(%rsp),%r11           # RFLAGS
@@ -254,7 +263,8 @@ FUNC(lstar_enter)
         pushq $FLAT_KERNEL_CS64
         pushq %rcx
         pushq $0
-        movl  $TRAP_syscall, 4(%rsp)
+        BUILD_BUG_ON(TRAP_syscall & 0xff)
+        movb  $TRAP_syscall >> 8, 4+1(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -292,7 +302,8 @@ FUNC(cstar_enter)
         pushq $FLAT_USER_CS32
         pushq %rcx
         pushq $0
-        movl  $TRAP_syscall, 4(%rsp)
+        BUILD_BUG_ON(TRAP_syscall & 0xff)
+        movb  $TRAP_syscall >> 8, 4+1(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -334,7 +345,8 @@ LABEL(sysenter_eflags_saved, 0)
         pushq $3 /* ring 3 null cs */
         pushq $0 /* null rip */
         pushq $0
-        movl  $TRAP_syscall, 4(%rsp)
+        BUILD_BUG_ON(TRAP_syscall & 0xff)
+        movb  $TRAP_syscall >> 8, 4+1(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -389,7 +401,7 @@ FUNC(entry_int80)
         ENDBR64
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         pushq $0
-        movl  $0x80, 4(%rsp)
+        movb  $0x80, 4(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -561,7 +573,8 @@ __UNLIKELY_END(create_bounce_frame_bad_sp)
         /* Rewrite our stack frame and return to guest-OS mode. */
         /* IA32 Ref. Vol. 3: TF, VM, RF and NT flags are cleared on trap. */
         /* Also clear AC: alignment checks shouldn't trigger in kernel mode. */
-        orl   $TRAP_syscall,UREGS_entry_vector+8(%rsp)
+        BUILD_BUG_ON(TRAP_syscall & 0xff)
+        orb   $TRAP_syscall >> 8, UREGS_entry_vector+8+1(%rsp)
         andl  $~(X86_EFLAGS_AC|X86_EFLAGS_VM|X86_EFLAGS_RF|\
                  X86_EFLAGS_NT|X86_EFLAGS_TF),UREGS_eflags+8(%rsp)
         movq  $FLAT_KERNEL_SS,UREGS_ss+8(%rsp)
@@ -653,7 +666,7 @@ END(ret_from_intr)
         .section .init.text, "ax", @progbits
 FUNC(early_page_fault)
         ENDBR64
-        movl  $X86_EXC_PF, 4(%rsp)
+        movb  $X86_EXC_PF, 4(%rsp)
         SAVE_ALL
         movq  %rsp, %rdi
         call  do_early_page_fault
@@ -722,7 +735,7 @@ END(common_interrupt)
 
 FUNC(entry_PF)
         ENDBR64
-        movl  $X86_EXC_PF, 4(%rsp)
+        movb  $X86_EXC_PF, 4(%rsp)
 END(entry_PF)
 /* No special register assumptions. */
 FUNC(handle_exception, 0)
@@ -898,105 +911,106 @@ END(handle_exception)
 FUNC(entry_DE)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_DE, 4(%rsp)
+        /* no need to update exception type, already 0 */
+        BUILD_BUG_ON(X86_EXC_DE)
         jmp   handle_exception
 END(entry_DE)
 
 FUNC(entry_MF)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_MF, 4(%rsp)
+        movb  $X86_EXC_MF, 4(%rsp)
         jmp   handle_exception
 END(entry_MF)
 
 FUNC(entry_XM)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_XM, 4(%rsp)
+        movb  $X86_EXC_XM, 4(%rsp)
         jmp   handle_exception
 END(entry_XM)
 
 FUNC(entry_NM)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_NM, 4(%rsp)
+        movb  $X86_EXC_NM, 4(%rsp)
         jmp   handle_exception
 END(entry_NM)
 
 FUNC(entry_DB)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_DB, 4(%rsp)
+        movb  $X86_EXC_DB, 4(%rsp)
         jmp   handle_ist_exception
 END(entry_DB)
 
 FUNC(entry_BP)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_BP, 4(%rsp)
+        movb  $X86_EXC_BP, 4(%rsp)
         jmp   handle_exception
 END(entry_BP)
 
 FUNC(entry_OF)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_OF, 4(%rsp)
+        movb  $X86_EXC_OF, 4(%rsp)
         jmp   handle_exception
 END(entry_OF)
 
 FUNC(entry_BR)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_BR, 4(%rsp)
+        movb  $X86_EXC_BR, 4(%rsp)
         jmp   handle_exception
 END(entry_BR)
 
 FUNC(entry_UD)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_UD, 4(%rsp)
+        movb  $X86_EXC_UD, 4(%rsp)
         jmp   handle_exception
 END(entry_UD)
 
 FUNC(entry_TS)
         ENDBR64
-        movl  $X86_EXC_TS, 4(%rsp)
+        movb  $X86_EXC_TS, 4(%rsp)
         jmp   handle_exception
 END(entry_TS)
 
 FUNC(entry_NP)
         ENDBR64
-        movl  $X86_EXC_NP, 4(%rsp)
+        movb  $X86_EXC_NP, 4(%rsp)
         jmp   handle_exception
 END(entry_NP)
 
 FUNC(entry_SS)
         ENDBR64
-        movl  $X86_EXC_SS, 4(%rsp)
+        movb  $X86_EXC_SS, 4(%rsp)
         jmp   handle_exception
 END(entry_SS)
 
 FUNC(entry_GP)
         ENDBR64
-        movl  $X86_EXC_GP, 4(%rsp)
+        movb  $X86_EXC_GP, 4(%rsp)
         jmp   handle_exception
 END(entry_GP)
 
 FUNC(entry_AC)
         ENDBR64
-        movl  $X86_EXC_AC, 4(%rsp)
+        movb  $X86_EXC_AC, 4(%rsp)
         jmp   handle_exception
 END(entry_AC)
 
 FUNC(entry_CP)
         ENDBR64
-        movl  $X86_EXC_CP, 4(%rsp)
+        movb  $X86_EXC_CP, 4(%rsp)
         jmp   handle_exception
 END(entry_CP)
 
 FUNC(entry_DF)
         ENDBR64
-        movl  $X86_EXC_DF, 4(%rsp)
+        movb  $X86_EXC_DF, 4(%rsp)
         /* Set AC to reduce chance of further SMAP faults */
         ALTERNATIVE "", stac, X86_FEATURE_XEN_SMAP
         SAVE_ALL
@@ -1022,7 +1036,7 @@ END(entry_DF)
 FUNC(entry_NMI)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_NMI, 4(%rsp)
+        movb  $X86_EXC_NMI, 4(%rsp)
 END(entry_NMI)
 
 FUNC(handle_ist_exception)
@@ -1158,7 +1172,7 @@ END(handle_ist_exception)
 FUNC(entry_MC)
         ENDBR64
         pushq $0
-        movl  $X86_EXC_MC, 4(%rsp)
+        movb  $X86_EXC_MC, 4(%rsp)
         jmp   handle_ist_exception
 END(entry_MC)
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 19 11:48:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 11:48:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682792.1061951 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc285-0005sb-AL; Mon, 19 Feb 2024 11:48:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682792.1061951; Mon, 19 Feb 2024 11:48:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc285-0005sU-7L; Mon, 19 Feb 2024 11:48:13 +0000
Received: by outflank-mailman (input) for mailman id 682792;
 Mon, 19 Feb 2024 11:48:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GzZz=J4=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rc283-0005sO-Jt
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 11:48:11 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20615.outbound.protection.outlook.com
 [2a01:111:f400:7e88::615])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bfeb6422-cf1c-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 12:48:09 +0100 (CET)
Received: from DM6PR11CA0015.namprd11.prod.outlook.com (2603:10b6:5:190::28)
 by SA0PR12MB4541.namprd12.prod.outlook.com (2603:10b6:806:9e::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.12; Mon, 19 Feb
 2024 11:48:05 +0000
Received: from DS2PEPF0000343E.namprd02.prod.outlook.com
 (2603:10b6:5:190:cafe::70) by DM6PR11CA0015.outlook.office365.com
 (2603:10b6:5:190::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.38 via Frontend
 Transport; Mon, 19 Feb 2024 11:48:05 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS2PEPF0000343E.mail.protection.outlook.com (10.167.18.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 19 Feb 2024 11:48:05 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 19 Feb
 2024 05:48:04 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 19 Feb 2024 05:48:02 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfeb6422-cf1c-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Gus274+yRrro4nVuP60DKP7PvvXdFfhw4eZiDrEt9VDgP5wYZiYMYVAUjuFsZW43VzdnM4qWPpelcyO7Je33z9q9q/hnBeVekA7781M85WUZ0pTFrSrcmAE/J6htHkVM7Qg4Wbn+rOHIBbu25AKSxgx8axr81j22caNK19+Z2WnLVoIpThKxscHm+reaSa48O7Un9619IU16B6XbzRDfHPaD8BXmbsGPej9PJlbV4ifWgXxTBvJYP7wOZGV/dQJT26ErZAFObGope9z+F7+2sMd3oW1AnUjVncdnIV2aWx3RWSPzF8+vAVZwLu/s6R1fS5CiNXkkOHoWcIkpC3sgQA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RncrB+LZkM2n1kaYhG7aciVHltGE6ewfgMj7WhF63eU=;
 b=fw9PtS8VNNV4lUYGX+uiAtaMh7V+OROUgdE2inGb7M9+vv0PPS4RO/s/rhMERRFYuxooxSzaZS2pc/vHJKFomF4rLDpEIccXy09aPcuAx8EXOhp+SIQXp/F4eF/I4g1Ne+7k1CvThjimjcPEPeoc3w3iUQhXiifeex2OsSPmzJy0vXTovssUE8iSNw4OdgJWcH3qIwmWJ3w3PoYBrjDnmxES5hJT9f6CqYwlsW7hZ9wtkTlXhFyiynSAUKZ4CtTmpKkqSO/v8/nKk8Jbw9OmIhFu1Qjb44tuaKBgEsQlguJo/oiIrIBFFBK+Hci5qogsnS3jca2CKbRu8B+2yeVdyQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RncrB+LZkM2n1kaYhG7aciVHltGE6ewfgMj7WhF63eU=;
 b=D7EkUU8fo003+zdGYwlKUZtTJZpWhgOwL0Hgmq5C7/wgaGkA14pelXU3FHEHh5g9Cab0gCMgyCNzVDlK7Vq7XgqwT0qVYI59wrfcL17vmETeTfupA5C2zcfZtNv6EIlq+Kbo3L6ANazBWMRhzshJ7mWn9nfFFW0/JVVgHsaQnjA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Jan Beulich
	<jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Jun Nakajima
	<jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>, Paul Durrant
	<paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>
Subject: [PATCH v13.2 01/14] vpci: use per-domain PCI lock to protect vpci structure
Date: Mon, 19 Feb 2024 06:47:48 -0500
Message-ID: <20240219114750.151730-1-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.2
In-Reply-To: <20240202213321.1920347-2-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com> <20240202213321.1920347-2-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS2PEPF0000343E:EE_|SA0PR12MB4541:EE_
X-MS-Office365-Filtering-Correlation-Id: 050cd7b6-724f-43e1-fe71-08dc3140a273
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ZHsp6aHffNB/wXbBy8mYS8M/C8JOv5PSgSI/WtgMlJ2C/E40AdHwMOsGinhZpnENvWIqBrRlRUn1bRwgtjlxo3UnPi7JGq9n80Lim9F4T3aSVNF735bLUbHJhJtUZAnzD6nSYUzgTUmJW+K+xa9biiwjT+8y19csuvu1ExgZ/2TTvP9Gz4MB8Do61jPJz+d5NvBue/QmS2OxZox5phDcgryFKrULkBDS//7nMgHbwI5k1f0kZDHb1x5R3EpMktYXt8EjvFZVhhe9NLwGMC+HEEBUrXpqnFIN1YD6Sbn690WsvuBOULZVJ2YVg235zIX79qorviZhCI/LTXsJHioVyLWnRzbwuiB99Yq8GGwCBfVgBnPpLWSQbw1FGq8Trm/oIInUa+fSIuqRQ7K6btiqSMy7tz7EhXkMlsavmeDVBXi+Z/JSHKzb5hrxM0fueXkHI9LEJPNScZ+q0TacawoWmKXLqFmydQQ5DYU0wZOrDmSNVjtWiacLaKAxarhCazchvwinQVL2HgyEQnAgWTl8gjaDFd33ljntavbANjxIAxuFWk5HLIV+hQf+Vqo9Sw73L9WNY2zsa4/b9qI3iwMK3Wxvsw0FvHUwp2Hm+SdRCL/uBSChboByAASAllkd1J2XrMuqxNWYrrKxBX15ohXgOWc0tLJuGuQd6t3zwfQesQI=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(46966006)(40470700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Feb 2024 11:48:05.1697
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 050cd7b6-724f-43e1-fe71-08dc3140a273
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS2PEPF0000343E.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4541

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Use the per-domain PCI read/write lock to protect the presence of the
pci device vpci field. This lock can be used (and in a few cases is used
right away) so that vpci removal can be performed while holding the lock
in write mode. Previously such removal could race with vpci_read for
example.

When taking both d->pci_lock and pdev->vpci->lock, they should be
taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
possible deadlock situations.

1. Per-domain's pci_lock is used to protect pdev->vpci structure
from being removed.

2. Writing the command register and ROM BAR register may trigger
modify_bars to run, which in turn may access multiple pdevs while
checking for the existing BAR's overlap. The overlapping check, if
done under the read lock, requires vpci->lock to be acquired on both
devices being compared, which may produce a deadlock. It is not
possible to upgrade read lock to write lock in such a case. So, in
order to prevent the deadlock, use d->pci_lock in write mode instead.

All other code, which doesn't lead to pdev->vpci destruction and does
not access multiple pdevs at the same time, can still use a
combination of the read lock and pdev->vpci->lock.

3. Drop const qualifier where the new rwlock is used and this is
appropriate.

4. Do not call process_pending_softirqs with any locks held. For that
unlock prior the call and re-acquire the locks after. After
re-acquiring the lock there is no need to check if pdev->vpci exists:
 - in apply_map because of the context it is called (no race condition
   possible)
 - for MSI/MSI-X debug code because it is called at the end of
   pdev->vpci access and no further access to pdev->vpci is made

5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
while accessing pdevs in vpci code.

6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
do not go away. The vPCI functions call several MSI-related functions
which already have existing non-vPCI callers. Change those MSI-related
functions to allow using either pcidevs_lock() or d->pci_lock for
ensuring pdevs do not go away. Holding d->pci_lock in read mode is
sufficient. Note that this pdev protection mechanism does not protect
other state or critical sections. These MSI-related functions already
have other race condition and state protection mechanims (e.g.
d->event_lock and msixtbl RCU), so we deduce that the use of the global
pcidevs_lock() is to ensure that pdevs do not go away.

7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
that pdevs do not go away. The purpose of this wrapper is to aid
readability and document the intent of the pdev protection mechanism.

8. When possible, the existing non-vPCI callers of these MSI-related
functions haven't been switched to use the newly introduced per-domain
pci_lock, and will continue to use the global pcidevs_lock(). This is
done to reduce the risk of the new locking scheme introducing
regressions. Those users will be adjusted in due time. One exception
is where the pcidevs_lock() in allocate_and_map_msi_pirq() is moved to
the caller, physdev_map_pirq(): this instance is switched to
read_lock(&d->pci_lock) right away.

Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
Changes in v13.2:
 - clarify comment in vpci_msix_arch_print()
 - add parenthesis around arg in macro definition
 - make the helper macro an ASSERT itself

Changes in v13.1:
 - move pdev_list_is_read_locked() to pci.h
 - use pdev_list_is_read_locked() in more places
 - use d directly in pdev_list_is_read_locked()
 - wrap pdev_list_is_read_locked() helper in #ifndef NDEBUG, and add
   declaration in the #else case with no implementation
 - replace pcidevs_lock() with read_lock(&d->pci_lock) in physdev.c

Changes in v13:
 - hold off adding Roger's R-b tag even though it was provided on v12.2
 - use a wrapper construct to ease readability of odd-looking ASSERTs
 - new placement of ASSERT in __pci_enable_msix(), __pci_enable_msi(),
   and pci_enable_msi(). Rearrange/add pdev NULL check.
 - expand commit description with details about using either
   pcidevs_lock() or d->pci_lock

Changes in v12.2:
 - drop Roger's R-b
 - drop both locks on error paths in vpci_msix_arch_print()
 - add another ASSERT in vpci_msix_arch_print(), to enforce the
   expectation both locks are held before calling vpci_msix_arch_print()
 - move pdev_done label in vpci_dump_msi()
 - update comments in vpci_dump_msi() to say locks (plural)

Changes in v12.1:
 - use read_trylock() in vpci_msix_arch_print()
 - fixup in-code comments (revert double space, use DomXEN) in
   vpci_{read,write}()
 - minor updates in commit message
 - add Roger's R-b

Changes in v12:
 - s/pci_rwlock/pci_lock/ in commit message
 - expand comment about scope of pci_lock in sched.h
 - in vpci_{read,write}, if hwdom is trying to access a device assigned
   to dom_xen, holding hwdom->pci_lock is sufficient (no need to hold
   dom_xen->pci_lock)
 - reintroduce ASSERT in vmx_pi_update_irte()
 - reintroduce ASSERT in __pci_enable_msi{x}()
 - delete note 6. in commit message about removing ASSERTs since we have
   reintroduced them

Changes in v11:
 - Fixed commit message regarding possible spinlocks
 - Removed parameter from allocate_and_map_msi_pirq(), which was added
 in the prev version. Now we are taking pcidevs_lock in
 physdev_map_pirq()
 - Returned ASSERT to pci_enable_msi
 - Fixed case when we took read lock instead of write one
 - Fixed label indentation

Changes in v10:
 - Moved printk pas locked area
 - Returned back ASSERTs
 - Added new parameter to allocate_and_map_msi_pirq() so it knows if
 it should take the global pci lock
 - Added comment about possible improvement in vpci_write
 - Changed ASSERT(rw_is_locked()) to rw_is_write_locked() in
   appropriate places
 - Renamed release_domain_locks() to release_domain_write_locks()
 - moved domain_done label in vpci_dump_msi() to correct place
Changes in v9:
 - extended locked region to protect vpci_remove_device and
   vpci_add_handlers() calls
 - vpci_write() takes lock in the write mode to protect
   potential call to modify_bars()
 - renamed lock releasing function
 - removed ASSERT()s from msi code
 - added trylock in vpci_dump_msi

Changes in v8:
 - changed d->vpci_lock to d->pci_lock
 - introducing d->pci_lock in a separate patch
 - extended locked region in vpci_process_pending
 - removed pcidevs_lockis vpci_dump_msi()
 - removed some changes as they are not needed with
   the new locking scheme
 - added handling for hwdom && dom_xen case
---
 xen/arch/x86/hvm/vmsi.c       | 37 +++++++++++++++++--------
 xen/arch/x86/hvm/vmx/vmx.c    |  2 +-
 xen/arch/x86/irq.c            |  8 +++---
 xen/arch/x86/msi.c            | 20 +++++++++-----
 xen/arch/x86/physdev.c        |  2 ++
 xen/drivers/passthrough/pci.c |  9 +++---
 xen/drivers/vpci/header.c     | 18 ++++++++++++
 xen/drivers/vpci/msi.c        | 30 +++++++++++++++++---
 xen/drivers/vpci/msix.c       | 52 ++++++++++++++++++++++++++++++-----
 xen/drivers/vpci/vpci.c       | 24 ++++++++++++++--
 xen/include/xen/pci.h         | 13 +++++++++
 xen/include/xen/sched.h       |  3 +-
 12 files changed, 177 insertions(+), 41 deletions(-)

diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index 128f23636279..bb1c60782419 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -468,7 +468,7 @@ int msixtbl_pt_register(struct domain *d, struct pirq *pirq, uint64_t gtable)
     struct msixtbl_entry *entry, *new_entry;
     int r = -EINVAL;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     if ( !msixtbl_initialised(d) )
@@ -538,7 +538,7 @@ void msixtbl_pt_unregister(struct domain *d, struct pirq *pirq)
     struct pci_dev *pdev;
     struct msixtbl_entry *entry;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     if ( !msixtbl_initialised(d) )
@@ -684,7 +684,7 @@ static int vpci_msi_update(const struct pci_dev *pdev, uint32_t data,
 {
     unsigned int i;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
 
     if ( (address & MSI_ADDR_BASE_MASK) != MSI_ADDR_HEADER )
     {
@@ -725,8 +725,8 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev)
     int rc;
 
     ASSERT(msi->arch.pirq != INVALID_PIRQ);
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
 
-    pcidevs_lock();
     for ( i = 0; i < msi->vectors && msi->arch.bound; i++ )
     {
         struct xen_domctl_bind_pt_irq unbind = {
@@ -745,7 +745,6 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev)
 
     msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address,
                                        msi->vectors, msi->arch.pirq, msi->mask);
-    pcidevs_unlock();
 }
 
 static int vpci_msi_enable(const struct pci_dev *pdev, unsigned int nr,
@@ -778,15 +777,14 @@ int vpci_msi_arch_enable(struct vpci_msi *msi, const struct pci_dev *pdev,
     int rc;
 
     ASSERT(msi->arch.pirq == INVALID_PIRQ);
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
     rc = vpci_msi_enable(pdev, vectors, 0);
     if ( rc < 0 )
         return rc;
     msi->arch.pirq = rc;
 
-    pcidevs_lock();
     msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address, vectors,
                                        msi->arch.pirq, msi->mask);
-    pcidevs_unlock();
 
     return 0;
 }
@@ -797,8 +795,8 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq,
     unsigned int i;
 
     ASSERT(pirq != INVALID_PIRQ);
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
 
-    pcidevs_lock();
     for ( i = 0; i < nr && bound; i++ )
     {
         struct xen_domctl_bind_pt_irq bind = {
@@ -814,7 +812,6 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq,
     write_lock(&pdev->domain->event_lock);
     unmap_domain_pirq(pdev->domain, pirq);
     write_unlock(&pdev->domain->event_lock);
-    pcidevs_unlock();
 }
 
 void vpci_msi_arch_disable(struct vpci_msi *msi, const struct pci_dev *pdev)
@@ -854,6 +851,7 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
     int rc;
 
     ASSERT(entry->arch.pirq == INVALID_PIRQ);
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
     rc = vpci_msi_enable(pdev, vmsix_entry_nr(pdev->vpci->msix, entry),
                          table_base);
     if ( rc < 0 )
@@ -861,7 +859,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
 
     entry->arch.pirq = rc;
 
-    pcidevs_lock();
     rc = vpci_msi_update(pdev, entry->data, entry->addr, 1, entry->arch.pirq,
                          entry->masked);
     if ( rc )
@@ -869,7 +866,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
         vpci_msi_disable(pdev, entry->arch.pirq, 1, false);
         entry->arch.pirq = INVALID_PIRQ;
     }
-    pcidevs_unlock();
 
     return rc;
 }
@@ -895,6 +891,15 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
 {
     unsigned int i;
 
+    /*
+     * Assert that d->pdev_list doesn't change. ASSERT_PDEV_LIST_IS_READ_LOCKED
+     * is not suitable here because it may allow either pcidevs_lock() or
+     * d->pci_lock to be held, but here we rely on d->pci_lock being held, not
+     * pcidevs_lock().
+     */
+    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
+    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));
+
     for ( i = 0; i < msix->max_entries; i++ )
     {
         const struct vpci_msix_entry *entry = &msix->entries[i];
@@ -913,13 +918,23 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
             struct pci_dev *pdev = msix->pdev;
 
             spin_unlock(&msix->pdev->vpci->lock);
+            read_unlock(&pdev->domain->pci_lock);
             process_pending_softirqs();
+
+            if ( !read_trylock(&pdev->domain->pci_lock) )
+                return -EBUSY;
+
             /* NB: we assume that pdev cannot go away for an alive domain. */
             if ( !pdev->vpci || !spin_trylock(&pdev->vpci->lock) )
+            {
+                read_unlock(&pdev->domain->pci_lock);
                 return -EBUSY;
+            }
+
             if ( pdev->vpci->msix != msix )
             {
                 spin_unlock(&pdev->vpci->lock);
+                read_unlock(&pdev->domain->pci_lock);
                 return -EAGAIN;
             }
         }
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 48376cc32751..80a20b17f8a9 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -413,7 +413,7 @@ static int cf_check vmx_pi_update_irte(const struct vcpu *v,
 
     spin_unlock_irq(&desc->lock);
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(msi_desc->dev->domain);
 
     return iommu_update_ire_from_msi(msi_desc, &msi_desc->msg);
 
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index bbae7751e494..4beafa83f14b 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2164,7 +2164,7 @@ int map_domain_pirq(
         struct pci_dev *pdev;
         unsigned int nr = 0;
 
-        ASSERT(pcidevs_locked());
+        ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
 
         ret = -ENODEV;
         if ( !cpu_has_apic )
@@ -2321,7 +2321,7 @@ int unmap_domain_pirq(struct domain *d, int pirq)
     if ( (pirq < 0) || (pirq >= d->nr_pirqs) )
         return -EINVAL;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     info = pirq_info(d, pirq);
@@ -2886,6 +2886,8 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 {
     int irq, pirq, ret;
 
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
+
     switch ( type )
     {
     case MAP_PIRQ_TYPE_MSI:
@@ -2915,7 +2917,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 
     msi->irq = irq;
 
-    pcidevs_lock();
     /* Verify or get pirq. */
     write_lock(&d->event_lock);
     pirq = allocate_pirq(d, index, *pirq_p, irq, type, &msi->entry_nr);
@@ -2931,7 +2932,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 
  done:
     write_unlock(&d->event_lock);
-    pcidevs_unlock();
     if ( ret )
     {
         switch ( type )
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index 335c0868a225..e721aaf5c001 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -602,7 +602,7 @@ static int msi_capability_init(struct pci_dev *dev,
     unsigned int i, mpos;
     uint16_t control;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(dev->domain);
     pos = pci_find_cap_offset(dev->sbdf, PCI_CAP_ID_MSI);
     if ( !pos )
         return -ENODEV;
@@ -771,7 +771,7 @@ static int msix_capability_init(struct pci_dev *dev,
     if ( !pos )
         return -ENODEV;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(dev->domain);
 
     control = pci_conf_read16(dev->sbdf, msix_control_reg(pos));
     /*
@@ -988,11 +988,11 @@ static int __pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi,
 {
     struct msi_desc *old_desc;
 
-    ASSERT(pcidevs_locked());
-
     if ( !pdev )
         return -ENODEV;
 
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
+
     old_desc = find_msi_entry(pdev, msi->irq, PCI_CAP_ID_MSI);
     if ( old_desc )
     {
@@ -1043,9 +1043,12 @@ static int __pci_enable_msix(struct pci_dev *pdev, struct msi_info *msi,
 {
     struct msi_desc *old_desc;
 
-    ASSERT(pcidevs_locked());
+    if ( !pdev )
+        return -ENODEV;
+
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
 
-    if ( !pdev || !pdev->msix )
+    if ( !pdev->msix )
         return -ENODEV;
 
     if ( msi->entry_nr >= pdev->msix->nr_entries )
@@ -1154,7 +1157,10 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off)
 int pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi,
                    struct msi_desc **desc)
 {
-    ASSERT(pcidevs_locked());
+    if ( !pdev )
+        return -ENODEV;
+
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
 
     if ( !use_msi )
         return -EPERM;
diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
index 47c4da0af7e1..7efa17cf4c1e 100644
--- a/xen/arch/x86/physdev.c
+++ b/xen/arch/x86/physdev.c
@@ -123,7 +123,9 @@ int physdev_map_pirq(domid_t domid, int type, int *index, int *pirq_p,
 
     case MAP_PIRQ_TYPE_MSI:
     case MAP_PIRQ_TYPE_MULTI_MSI:
+        read_lock(&d->pci_lock);
         ret = allocate_and_map_msi_pirq(d, *index, pirq_p, type, msi);
+        read_unlock(&d->pci_lock);
         break;
 
     default:
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 47c0eee7bdcc..c97dd4504a7a 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -750,7 +750,6 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
         pdev->domain = hardware_domain;
         write_lock(&hardware_domain->pci_lock);
         list_add(&pdev->domain_list, &hardware_domain->pdev_list);
-        write_unlock(&hardware_domain->pci_lock);
 
         /*
          * For devices not discovered by Xen during boot, add vPCI handlers
@@ -759,18 +758,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
         ret = vpci_add_handlers(pdev);
         if ( ret )
         {
-            printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret);
-            write_lock(&hardware_domain->pci_lock);
             list_del(&pdev->domain_list);
             write_unlock(&hardware_domain->pci_lock);
             pdev->domain = NULL;
+            printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret);
             goto out;
         }
+        write_unlock(&hardware_domain->pci_lock);
         ret = iommu_add_device(pdev);
         if ( ret )
         {
-            vpci_remove_device(pdev);
             write_lock(&hardware_domain->pci_lock);
+            vpci_remove_device(pdev);
             list_del(&pdev->domain_list);
             write_unlock(&hardware_domain->pci_lock);
             pdev->domain = NULL;
@@ -1146,7 +1145,9 @@ static void __hwdom_init setup_one_hwdom_device(const struct setup_hwdom *ctxt,
     } while ( devfn != pdev->devfn &&
               PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn) );
 
+    write_lock(&ctxt->d->pci_lock);
     err = vpci_add_handlers(pdev);
+    write_unlock(&ctxt->d->pci_lock);
     if ( err )
         printk(XENLOG_ERR "setup of vPCI for d%d failed: %d\n",
                ctxt->d->domain_id, err);
diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 58195549d50a..8f5850b8cf6d 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -173,6 +173,7 @@ bool vpci_process_pending(struct vcpu *v)
         if ( rc == -ERESTART )
             return true;
 
+        write_lock(&v->domain->pci_lock);
         spin_lock(&v->vpci.pdev->vpci->lock);
         /* Disable memory decoding unconditionally on failure. */
         modify_decoding(v->vpci.pdev,
@@ -191,6 +192,7 @@ bool vpci_process_pending(struct vcpu *v)
              * failure.
              */
             vpci_remove_device(v->vpci.pdev);
+        write_unlock(&v->domain->pci_lock);
     }
 
     return false;
@@ -202,8 +204,20 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
     struct map_data data = { .d = d, .map = true };
     int rc;
 
+    ASSERT(rw_is_write_locked(&d->pci_lock));
+
     while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART )
+    {
+        /*
+         * It's safe to drop and reacquire the lock in this context
+         * without risking pdev disappearing because devices cannot be
+         * removed until the initial domain has been started.
+         */
+        write_unlock(&d->pci_lock);
         process_pending_softirqs();
+        write_lock(&d->pci_lock);
+    }
+
     rangeset_destroy(mem);
     if ( !rc )
         modify_decoding(pdev, cmd, false);
@@ -244,6 +258,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
     unsigned int i;
     int rc;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !mem )
         return -ENOMEM;
 
@@ -524,6 +540,8 @@ static int cf_check init_header(struct pci_dev *pdev)
     int rc;
     bool mask_cap_list = false;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     switch ( pci_conf_read8(pdev->sbdf, PCI_HEADER_TYPE) & 0x7f )
     {
     case PCI_HEADER_TYPE_NORMAL:
diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c
index a253ccbd7db7..dc71938e23f5 100644
--- a/xen/drivers/vpci/msi.c
+++ b/xen/drivers/vpci/msi.c
@@ -263,7 +263,7 @@ REGISTER_VPCI_INIT(init_msi, VPCI_PRIORITY_LOW);
 
 void vpci_dump_msi(void)
 {
-    const struct domain *d;
+    struct domain *d;
 
     rcu_read_lock(&domlist_read_lock);
     for_each_domain ( d )
@@ -275,6 +275,9 @@ void vpci_dump_msi(void)
 
         printk("vPCI MSI/MSI-X d%d\n", d->domain_id);
 
+        if ( !read_trylock(&d->pci_lock) )
+            continue;
+
         for_each_pdev ( d, pdev )
         {
             const struct vpci_msi *msi;
@@ -313,17 +316,36 @@ void vpci_dump_msi(void)
                 {
                     /*
                      * On error vpci_msix_arch_print will always return without
-                     * holding the lock.
+                     * holding the locks.
                      */
                     printk("unable to print all MSI-X entries: %d\n", rc);
-                    process_pending_softirqs();
-                    continue;
+                    goto pdev_done;
                 }
             }
 
+            /*
+             * Unlock locks to process pending softirqs. This is
+             * potentially unsafe, as d->pdev_list can be changed in
+             * meantime.
+             */
             spin_unlock(&pdev->vpci->lock);
+            read_unlock(&d->pci_lock);
+        pdev_done:
             process_pending_softirqs();
+            if ( !read_trylock(&d->pci_lock) )
+            {
+                printk("unable to access other devices for the domain\n");
+                goto domain_done;
+            }
         }
+        read_unlock(&d->pci_lock);
+    domain_done:
+        /*
+         * We need this label at the end of the loop, but some
+         * compilers might not be happy about label at the end of the
+         * compound statement so we adding an empty statement here.
+         */
+        ;
     }
     rcu_read_unlock(&domlist_read_lock);
 }
diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
index d1126a417da9..58c16ebdf283 100644
--- a/xen/drivers/vpci/msix.c
+++ b/xen/drivers/vpci/msix.c
@@ -147,6 +147,8 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr)
 {
     struct vpci_msix *msix;
 
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
+
     list_for_each_entry ( msix, &d->arch.hvm.msix_tables, next )
     {
         const struct vpci_bar *bars = msix->pdev->vpci->header.bars;
@@ -163,7 +165,13 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr)
 
 static int cf_check msix_accept(struct vcpu *v, unsigned long addr)
 {
-    return !!msix_find(v->domain, addr);
+    int rc;
+
+    read_lock(&v->domain->pci_lock);
+    rc = !!msix_find(v->domain, addr);
+    read_unlock(&v->domain->pci_lock);
+
+    return rc;
 }
 
 static bool access_allowed(const struct pci_dev *pdev, unsigned long addr,
@@ -358,21 +366,35 @@ static int adjacent_read(const struct domain *d, const struct vpci_msix *msix,
 static int cf_check msix_read(
     struct vcpu *v, unsigned long addr, unsigned int len, unsigned long *data)
 {
-    const struct domain *d = v->domain;
-    struct vpci_msix *msix = msix_find(d, addr);
+    struct domain *d = v->domain;
+    struct vpci_msix *msix;
     const struct vpci_msix_entry *entry;
     unsigned int offset;
 
     *data = ~0UL;
 
+    read_lock(&d->pci_lock);
+
+    msix = msix_find(d, addr);
     if ( !msix )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_RETRY;
+    }
 
     if ( adjacent_handle(msix, addr) )
-        return adjacent_read(d, msix, addr, len, data);
+    {
+        int rc = adjacent_read(d, msix, addr, len, data);
+
+        read_unlock(&d->pci_lock);
+        return rc;
+    }
 
     if ( !access_allowed(msix->pdev, addr, len) )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_OKAY;
+    }
 
     spin_lock(&msix->pdev->vpci->lock);
     entry = get_entry(msix, addr);
@@ -404,6 +426,7 @@ static int cf_check msix_read(
         break;
     }
     spin_unlock(&msix->pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     return X86EMUL_OKAY;
 }
@@ -491,19 +514,33 @@ static int adjacent_write(const struct domain *d, const struct vpci_msix *msix,
 static int cf_check msix_write(
     struct vcpu *v, unsigned long addr, unsigned int len, unsigned long data)
 {
-    const struct domain *d = v->domain;
-    struct vpci_msix *msix = msix_find(d, addr);
+    struct domain *d = v->domain;
+    struct vpci_msix *msix;
     struct vpci_msix_entry *entry;
     unsigned int offset;
 
+    read_lock(&d->pci_lock);
+
+    msix = msix_find(d, addr);
     if ( !msix )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_RETRY;
+    }
 
     if ( adjacent_handle(msix, addr) )
-        return adjacent_write(d, msix, addr, len, data);
+    {
+        int rc = adjacent_write(d, msix, addr, len, data);
+
+        read_unlock(&d->pci_lock);
+        return rc;
+    }
 
     if ( !access_allowed(msix->pdev, addr, len) )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_OKAY;
+    }
 
     spin_lock(&msix->pdev->vpci->lock);
     entry = get_entry(msix, addr);
@@ -579,6 +616,7 @@ static int cf_check msix_write(
         break;
     }
     spin_unlock(&msix->pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     return X86EMUL_OKAY;
 }
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 72ef277c4f8e..475272b173f3 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -42,6 +42,8 @@ extern vpci_register_init_t *const __end_vpci_array[];
 
 void vpci_remove_device(struct pci_dev *pdev)
 {
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !has_vpci(pdev->domain) || !pdev->vpci )
         return;
 
@@ -77,6 +79,8 @@ int vpci_add_handlers(struct pci_dev *pdev)
     const unsigned long *ro_map;
     int rc = 0;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !has_vpci(pdev->domain) )
         return 0;
 
@@ -361,7 +365,7 @@ static uint32_t merge_result(uint32_t data, uint32_t new, unsigned int size,
 
 uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
 {
-    const struct domain *d = current->domain;
+    struct domain *d = current->domain;
     const struct pci_dev *pdev;
     const struct vpci_register *r;
     unsigned int data_offset = 0;
@@ -376,12 +380,18 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
     /*
      * Find the PCI dev matching the address, which for hwdom also requires
      * consulting DomXEN.  Passthrough everything that's not trapped.
+     * If this is hwdom and the device is assigned to DomXEN, acquiring hwdom's
+     * pci_lock is sufficient.
      */
+    read_lock(&d->pci_lock);
     pdev = pci_get_pdev(d, sbdf);
     if ( !pdev && is_hardware_domain(d) )
         pdev = pci_get_pdev(dom_xen, sbdf);
     if ( !pdev || !pdev->vpci )
+    {
+        read_unlock(&d->pci_lock);
         return vpci_read_hw(sbdf, reg, size);
+    }
 
     spin_lock(&pdev->vpci->lock);
 
@@ -428,6 +438,7 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
         ASSERT(data_offset < size);
     }
     spin_unlock(&pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     if ( data_offset < size )
     {
@@ -470,7 +481,7 @@ static void vpci_write_helper(const struct pci_dev *pdev,
 void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
                 uint32_t data)
 {
-    const struct domain *d = current->domain;
+    struct domain *d = current->domain;
     const struct pci_dev *pdev;
     const struct vpci_register *r;
     unsigned int data_offset = 0;
@@ -484,7 +495,13 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
     /*
      * Find the PCI dev matching the address, which for hwdom also requires
      * consulting DomXEN.  Passthrough everything that's not trapped.
+     * If this is hwdom and the device is assigned to DomXEN, acquiring hwdom's
+     * pci_lock is sufficient.
+     *
+     * TODO: We need to take pci_locks in exclusive mode only if we
+     * are modifying BARs, so there is a room for improvement.
      */
+    write_lock(&d->pci_lock);
     pdev = pci_get_pdev(d, sbdf);
     if ( !pdev && is_hardware_domain(d) )
         pdev = pci_get_pdev(dom_xen, sbdf);
@@ -493,6 +510,8 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
         /* Ignore writes to read-only devices, which have no ->vpci. */
         const unsigned long *ro_map = pci_get_ro_map(sbdf.seg);
 
+        write_unlock(&d->pci_lock);
+
         if ( !ro_map || !test_bit(sbdf.bdf, ro_map) )
             vpci_write_hw(sbdf, reg, size, data);
         return;
@@ -534,6 +553,7 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
         ASSERT(data_offset < size);
     }
     spin_unlock(&pdev->vpci->lock);
+    write_unlock(&d->pci_lock);
 
     if ( data_offset < size )
         /* Tailing gap, write the remaining. */
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index aabc5465a7d3..f4577faf75fd 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -171,6 +171,19 @@ void pcidevs_lock(void);
 void pcidevs_unlock(void);
 bool __must_check pcidevs_locked(void);
 
+#ifndef NDEBUG
+/*
+ * Check to ensure there will be no changes to the entries in d->pdev_list (but
+ * not the contents of each entry).
+ * This check is not suitable for protecting other state or critical regions.
+ */
+#define ASSERT_PDEV_LIST_IS_READ_LOCKED(d)                               \
+        /* NB: d may be evaluated multiple times, or not at all */       \
+        ASSERT(pcidevs_locked() || ((d) && rw_is_locked(&(d)->pci_lock)))
+#else
+#define ASSERT_PDEV_LIST_IS_READ_LOCKED(d) ({ (void)(d); })
+#endif
+
 bool pci_known_segment(u16 seg);
 bool pci_device_detect(u16 seg, u8 bus, u8 dev, u8 func);
 int scan_pci_devices(void);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 9da91e0e6244..37f5922f3206 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -462,7 +462,8 @@ struct domain
 #ifdef CONFIG_HAS_PCI
     struct list_head pdev_list;
     /*
-     * pci_lock protects access to pdev_list.
+     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
+     * structure from being removed.
      *
      * Any user *reading* from pdev_list, or from devices stored in pdev_list,
      * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,

base-commit: 78398afae10bfb4ab94e8af17b7ed58510a57d96
-- 
2.43.2



From xen-devel-bounces@lists.xenproject.org Mon Feb 19 11:50:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 11:50:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682795.1061960 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc2Aj-0007l6-SL; Mon, 19 Feb 2024 11:50:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682795.1061960; Mon, 19 Feb 2024 11:50:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc2Aj-0007kz-Pd; Mon, 19 Feb 2024 11:50:57 +0000
Received: by outflank-mailman (input) for mailman id 682795;
 Mon, 19 Feb 2024 11:50:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Ybst=J4=linaro.org=peter.maydell@srs-se1.protection.inumbo.net>)
 id 1rc2Ai-0007kt-3l
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 11:50:56 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 22b90609-cf1d-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 12:50:54 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-563b7b3e3ecso4947796a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 03:50:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22b90609-cf1d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1708343453; x=1708948253; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wRqMCBzf+R3oPVWN5HPILjOg+0s4Zpcn1+zkzkrhkXQ=;
        b=W04H7eKC2iT/QM3ikNKxu4xNpY1bHl26cQbIIws8x8UKOpYvYXL4XnZ0shCHC5taZA
         DY5UCTLae1rDAk2b9Ju26yxfKU+vJryVyzADfJlAsno0HH6sk+7LMY5D2m84HQ74VBOw
         HBIwba5Blxc0d1PAfFo1wbziNQlk9JeM9cE1x3xoas6KVIJmqBpnYDHy0y/eRqNUO9OE
         xvRNmIMlr3MpWLoxcWq97j/yQoQ84GVGHN9cxCCNSG3sMesovmiGquGWJ7w5XM4RA/TL
         Nj5pAPZFEsmR1AV51M3TDNzdVkycxVU8o+Fht47t7CieXNRuCaT0JWEp/RYZu+I55xPm
         X14Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708343453; x=1708948253;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=wRqMCBzf+R3oPVWN5HPILjOg+0s4Zpcn1+zkzkrhkXQ=;
        b=MHdnt669juidcjPQ6ja2wzUo9Be7onu/o6TgRCF8vgpg4lBXro8gpz/c/76x12WsE1
         3e3gnLUYyIzlsnw5pkRtqBC3H73Rz5bysIUX1KSzLzoL9raVeIFM80YUOGBCCAyYroLX
         OBq3Di1QupBQ3M3XZuA99wc3++cfXKCwInUGqUyF61iQvP8GhVqCiDaFQ9Os32V2JcdE
         K/LswyrNBp+cNsbBl8YplpxNxTk7MVAHKwIYQlQzuZHGDlOJaICFpKQCgqf3faNXUFwA
         9b1pSWM4Bn1BoEyfizKIrMWuNKmcQ+5woe3H9sLJtttsgSlY8hGpxhm4aBCJMuTmLR6p
         JhhA==
X-Forwarded-Encrypted: i=1; AJvYcCW4QDre6bbven5bQnT8lIqZZUJBCcrwsRsmS+vioLsnba87t/mpUOLBu1H2WfQLy2O6ivod75l/k+XbBT9VtrIVfAuHpW1JCUekr95wAmc=
X-Gm-Message-State: AOJu0YzrvkLN1ru4nHiKs2FVm7rZu1fgvsPphb8D4iGQGseXa+vW6LDL
	AWRfEgXIRdL5nc0GMxH62d/GBm0kc9j8Pm8Am9VjByr9BnhpXEMcwebl55tFzflvXN6lXelTZw8
	CBbsc1KcNNWDQR7ofpiPPp1ykbYbEH9z8Z6293A==
X-Google-Smtp-Source: AGHT+IEhN/N8srsegjmd62Nb46r+QdlqWsvsBnavJVQa6/fsW9epm6qk4NmB8JLwpKb908Muh40YJqXPthpU+2PGZLI=
X-Received: by 2002:aa7:d64d:0:b0:564:42e1:fa93 with SMTP id
 v13-20020aa7d64d000000b0056442e1fa93mr2860424edr.25.1708343453519; Mon, 19
 Feb 2024 03:50:53 -0800 (PST)
MIME-Version: 1.0
References: <87y1hspiyh.fsf@linaro.org> <alpine.DEB.2.22.394.2308301745530.6458@ubuntu-linux-20-04-desktop>
 <CAFEAcA8Ziov9vA9dW+4vzFE=KkSUqfMNNMZOtvQhqCXyjRytzQ@mail.gmail.com>
 <87cyz3pmgz.fsf@linaro.org> <CAFEAcA-m8G1kfTw52kOGPEvQwWPph0NWc0URVY1aQ2WwVeB_OQ@mail.gmail.com>
 <799b1754-7529-0538-1b5a-d94a362eb74d@amd.com>
In-Reply-To: <799b1754-7529-0538-1b5a-d94a362eb74d@amd.com>
From: Peter Maydell <peter.maydell@linaro.org>
Date: Mon, 19 Feb 2024 11:50:41 +0000
Message-ID: <CAFEAcA8UwpWzD=Z1syjWPcpRev0OLSMnfyqqYg=9UEQk05s4yQ@mail.gmail.com>
Subject: Re: QEMU features useful for Xen development?
To: Ayan Kumar Halder <ayankuma@amd.com>
Cc: =?UTF-8?B?QWxleCBCZW5uw6ll?= <alex.bennee@linaro.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Xen-devel <xen-devel@lists.xenproject.org>, 
	Stewart Hildebrand <stewart.hildebrand@amd.com>, Viresh Kumar <viresh.kumar@linaro.org>, 
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Sergiy Kibrik <Sergiy_Kibrik@epam.com>, 
	QEMU Developers <qemu-devel@nongnu.org>, Vikram Garhwal <vikram.garhwal@amd.com>, 
	Stefano Stabellini <stefano.stabellini@amd.com>, 
	Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>, 
	Jonathan Cameron <jonathan.cameron@huawei.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, 31 Aug 2023 at 11:32, Ayan Kumar Halder <ayankuma@amd.com> wrote:
> On 31/08/2023 11:03, Peter Maydell wrote:
> > On Thu, 31 Aug 2023 at 10:53, Alex Benn=C3=A9e <alex.bennee@linaro.org>=
 wrote:
> >> Peter Maydell <peter.maydell@linaro.org> writes:
> >>> On Thu, 31 Aug 2023 at 01:57, Stefano Stabellini <sstabellini@kernel.=
org> wrote:
> >>>> As Xen is gaining R52 and R82 support, it would be great to be able =
to
> >>>> use QEMU for development and testing there as well, but I don't thin=
k
> >>>> QEMU can emulate EL2 properly for the Cortex-R architecture. We woul=
d
> >>>> need EL2 support in the GIC/timer for R52/R82 as well.

> >>> (What sort of board model would Xen want to use it with?)

> >> We already model a bunch of the mps2/mps3 images so I'm assuming addin=
g
> >> the mps3-an536 would be a fairly simple step to do (mps2tz.c is mostly
> >> tweaking config values). The question is would it be a useful target f=
or
> >> Xen?

> Yes, it will be helpful if Qemu can model this board. We have a
> downstream port of Xen on R52 (upstreaming is in progress).
>
> So, we can test the Qemu model with Xen.

Hi, all. I just wanted to provide an update on this. We've now
completed the mps3-an536 board model, and you can find it if
you check out the head-of-git QEMU. The new board will be in the
9.0 QEMU release, but if you have a chance to give it a spin now
we'll be able to fix any bugs or problems with it before the release.

The documentation for the board is here:
https://www.qemu.org/docs/master/system/arm/mps2.html
and it lists the limitations/missing features. (If any of those
are important let me know and we can look at scheduling the work
to fill them in.)

I'd also like to draw your attention to the note about the UART
ordering on the AN536; unfortunately the hardware setup is a bit
awkward here, so if you have an "I don't see any output" problem
make sure your guest is sending to the same UART you're asking
QEMU to show you the output from :-)

thanks
-- PMM


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 12:00:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 12:00:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682806.1061987 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc2JZ-0001Se-3k; Mon, 19 Feb 2024 12:00:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682806.1061987; Mon, 19 Feb 2024 12:00:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc2JZ-0001S5-08; Mon, 19 Feb 2024 12:00:05 +0000
Received: by outflank-mailman (input) for mailman id 682806;
 Mon, 19 Feb 2024 12:00:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pKiv=J4=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rc2JX-000181-Jr
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 12:00:03 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 68e94f93-cf1e-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 13:00:01 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a3e72ec566aso124209266b.2
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 04:00:01 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 vb7-20020a170907d04700b00a3d23a4e8fdsm2886689ejc.90.2024.02.19.03.59.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 19 Feb 2024 04:00:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68e94f93-cf1e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708344001; x=1708948801; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=xhtHNJqfXbioKX2uexEcvVG0q+OpT+KI4UH7iQl665g=;
        b=KQ7ypgWAUi6/HUvsB+c81GNGxf5klwyECOd44X8sDQSJagWfTZns8I5zPOn4g0FUu0
         vKw6fqkC03hSOZY/UokImuYxlSLzLr/J7t9c2IJHRi8rxkn6tH1WWdO7OZ+PYfllt25M
         qsVnh7feca4dXR5RLGdLqpeYmEUHqojQY/UV5+0qcXdQ+gfFBd4XrTgjpTS2lp4hdDlJ
         Rw05ZUttB6M8BVZaB5rnKiNZCG8Si+3ha2VzXxUv4r9q10S1vhYrQtN6Tx7xpBxhmVJD
         ZH6D92tyBs2K1w5gMt+KimuimRGiW1p/7+lotCxOtd0j4F/ltuaMr9tBD6bNZUrgrjyB
         GdAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708344001; x=1708948801;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xhtHNJqfXbioKX2uexEcvVG0q+OpT+KI4UH7iQl665g=;
        b=SzwZ3+FY2QZFuCpMiZQC+Fx/24BUNSPEVfBLiXr0Iooc/i1OSGytc2YH1dAflnSv0E
         S1omuYYsG4wqxLEso3aP7Z5cCW0f97NLHmXhIxAFFa4AL7zOjwWhL8mQqdRhIPI9XKpd
         qZHceRpeQKtH9t7shxVeNlQbqnDvKCY4JqtrpcVnWrR1cOnPAUYg4nFbdyEl6gNK3kdf
         RLKA8ZgMnFGyIW6L2qJke4qjYXRauyRI9tsTld15A4jhWb3lsxJNPsPsmBYlh5lj9oeO
         zG1wq+TiXIrrRbXNoR+KrofUknf8WI5WoE2nm+e1LmM/kSWSS0x/ioTB8uLeAHysLUey
         4GJg==
X-Forwarded-Encrypted: i=1; AJvYcCX63LmMATWg3xWKNXJeDcBaQts7t+b3YPerok1gh79bFc8f5xJ84gOM/M1rYgK9inVvS9IjEj+BBfaCKslJCR5e6Xy+ugVjW/rHxeMSLvM=
X-Gm-Message-State: AOJu0YxdfcK5eSuFmytPMotb5qqJCnQ1tbeJyiySQMLiDvTUlklltrKA
	qdY/UVMfp7nKB9aba+L3v6ZPYiIweidWq290msCBxLb30CR1c4QR
X-Google-Smtp-Source: AGHT+IFhc5LR8DjAIVxhqrDsqPpRxH6ROeOoJSG0U4sb3I8/SzaRm/1Lg2BO8ck5b4bwV7D2H4WSXw==
X-Received: by 2002:a17:906:3b10:b0:a3d:7d05:a1b8 with SMTP id g16-20020a1709063b1000b00a3d7d05a1b8mr6506996ejf.77.1708344000611;
        Mon, 19 Feb 2024 04:00:00 -0800 (PST)
Message-ID: <8a09e386b709f780f193af39af63b6aeb75c868e.camel@gmail.com>
Subject: Re: [PATCH v4 07/30] xen/asm-generic: introdure nospec.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Jan
 Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>,  Shawn Anastasio
 <sanastasio@raptorengineering.com>, Alistair Francis
 <alistair.francis@wdc.com>, Bob Eshleman <bobbyeshleman@gmail.com>, Connor
 Davis <connojdavis@gmail.com>
Date: Mon, 19 Feb 2024 12:59:59 +0100
In-Reply-To: <510d74ca-5cce-48f8-93f6-407cd727cf7b@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <5889d7a5fa81722472f95cc1448af0be8f359a7d.1707146506.git.oleksii.kurochko@gmail.com>
	 <510d74ca-5cce-48f8-93f6-407cd727cf7b@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

Hi Julien,

On Sun, 2024-02-18 at 18:30 +0000, Julien Grall wrote:
> Hi Oleksii,
>=20
> Title: Typo s/introdure/introduce/
>=20
> On 05/02/2024 15:32, Oleksii Kurochko wrote:
> > The <asm/nospec.h> header is similar between Arm, PPC, and RISC-V,
> > so it has been moved to asm-generic.
>=20
> I am not 100% convinced that moving this header to asm-generic is a
> good=20
> idea. At least for Arm, those helpers ought to be non-empty, what
> about=20
> RISC-V?
For Arm, they are not taking any action, are they? There are no
specific fences or other mechanisms inside
evaluate_nospec()/block_speculation() to address speculation.

For RISC-V, it can be implemented in a similar manner, at least for
now. Since these functions are only used in the grant tables code ( for
Arm and so for RISC-V ), which is not supported by RISC-V.

>=20
> If the answer is they should be non-empty. Then I would consider to
> keep=20
> the duplication to make clear that each architecture should take
> their=20
> own decision in term of security.
>=20
> The alternative, is to have a generic implementation that is safe by=20
> default (if that's even possible).
I am not certain that we can have a generic implementation, as each
architecture may have specific speculation issues.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 12:10:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 12:10:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682814.1061997 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc2Tu-0003XX-3P; Mon, 19 Feb 2024 12:10:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682814.1061997; Mon, 19 Feb 2024 12:10:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc2Tu-0003XQ-09; Mon, 19 Feb 2024 12:10:46 +0000
Received: by outflank-mailman (input) for mailman id 682814;
 Mon, 19 Feb 2024 12:10:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc2Ts-0003XK-Jg
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 12:10:44 +0000
Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com
 [2a00:1450:4864:20::432])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e6fb4ae0-cf1f-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 13:10:42 +0100 (CET)
Received: by mail-wr1-x432.google.com with SMTP id
 ffacd0b85a97d-33d26da3e15so1299256f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 04:10:42 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v15-20020a5d610f000000b0033d4cf751b2sm3533324wrt.33.2024.02.19.04.10.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 04:10:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e6fb4ae0-cf1f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708344642; x=1708949442; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Heu9x30HCReYgHfO2KlQtAZBaU5xCTZkIFS0R3dxQcQ=;
        b=YOZgxcmr2/5HTvKWX7T+L+zN92rRnSDxZkWpEt168iTzgVoywPj7b/31VjVKNJnyux
         iD29Jowa1/m2F1ELVf/2ET7+ydI1ES0qyhg4jUZKef6CodMHRKGPRuRLFmjF8QeTJC5s
         YK5SeXBQz0Al6X4cuPyF6EaXgLnKb/7/vgB8BLqrGj6T71zwfVA9v99mEM0ibGDYHqDe
         sSmHte+mLZtkZuaxnabS5dR/4WmDI/XAtdpqLyzLA1NvTaBgqog4z7GswOEgShNUhNEF
         w5yOTWFrVClntgAXA5eo+UboQx0liVG3yqDOO4RDywrm+czQZjge7gZtkOlX0pPHdLu0
         Pvhg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708344642; x=1708949442;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Heu9x30HCReYgHfO2KlQtAZBaU5xCTZkIFS0R3dxQcQ=;
        b=LbXpyl8mJOzlo4bDAkuxphnuOdpHYTaPrY0ZD+tig88nolI5nwiqF2k2vPYqCuLRMA
         00QtFoPs2kzYJeXP2TxQv0Qeohr1pMXcHN+62KW1+sL4InDYdYeTFN+o8t00vbofcE3X
         lqxYb+Zmg49IF2jcJfvmMo5HTmpgO69zDpf7n8axVqYW0KmcxiJKUh5PkxGN4Ss7CAA8
         5uKIrI+KNDE+9CAy9yPd57/Sptl6nZsV0Zzdte1QIZYUfqSKMpH35+hQubTdxpxhwJ6f
         4pcErMWNMPtVmjr2k8N8eqWEnJw0lpwzdc9dUzKqhBMvhADCqW4/NNozgvFxasyKabrb
         dLsw==
X-Forwarded-Encrypted: i=1; AJvYcCUNoPIDBVbpRiEfYOTEsl9CI1YW8hG3hWjgC2iZrW1/G0pmayixMqRb+c2oHJM2bIdz1uIropDEeZkhqLfjOzeoNh+5lV8Y0zCODAQBDoU=
X-Gm-Message-State: AOJu0YzngP409jgnxAgVOFo7SPnR526/Dkznm1xUEujRWO0yiWKBHIHJ
	nSBxySU/jvCFkJhBY79n0GNOGuJXD6jbZz/vjoKgjLVFd2kQTWwcgg/zr5OonA==
X-Google-Smtp-Source: AGHT+IGRhPtngdLBWD6Dgm0XilfW4EEsMCOyVNL8DfxqJtdu/g2E2uiwhO6CncTHMDqqKkoPJ4GHZA==
X-Received: by 2002:adf:efd2:0:b0:33d:201f:ea9d with SMTP id i18-20020adfefd2000000b0033d201fea9dmr7276183wrp.2.1708344641568;
        Mon, 19 Feb 2024 04:10:41 -0800 (PST)
Message-ID: <8e807612-fd14-4d01-b200-1deb583afa22@suse.com>
Date: Mon, 19 Feb 2024 13:10:40 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13.2 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Paul Durrant <paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <20240219114750.151730-1-stewart.hildebrand@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240219114750.151730-1-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 19.02.2024 12:47, Stewart Hildebrand wrote:
> @@ -895,6 +891,15 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
>  {
>      unsigned int i;
>  
> +    /*
> +     * Assert that d->pdev_list doesn't change. ASSERT_PDEV_LIST_IS_READ_LOCKED
> +     * is not suitable here because it may allow either pcidevs_lock() or
> +     * d->pci_lock to be held, but here we rely on d->pci_lock being held, not
> +     * pcidevs_lock().
> +     */
> +    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
> +    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));

There's no "d" in sight here, so it's a little odd that "d" is being talked
about. But I guess people can infer what's meant without too much trouble.

> @@ -313,17 +316,36 @@ void vpci_dump_msi(void)
>                  {
>                      /*
>                       * On error vpci_msix_arch_print will always return without
> -                     * holding the lock.
> +                     * holding the locks.
>                       */
>                      printk("unable to print all MSI-X entries: %d\n", rc);
> -                    process_pending_softirqs();
> -                    continue;
> +                    goto pdev_done;
>                  }
>              }
>  
> +            /*
> +             * Unlock locks to process pending softirqs. This is
> +             * potentially unsafe, as d->pdev_list can be changed in
> +             * meantime.
> +             */
>              spin_unlock(&pdev->vpci->lock);
> +            read_unlock(&d->pci_lock);
> +        pdev_done:
>              process_pending_softirqs();
> +            if ( !read_trylock(&d->pci_lock) )
> +            {
> +                printk("unable to access other devices for the domain\n");
> +                goto domain_done;
> +            }
>          }
> +        read_unlock(&d->pci_lock);
> +    domain_done:
> +        /*
> +         * We need this label at the end of the loop, but some
> +         * compilers might not be happy about label at the end of the
> +         * compound statement so we adding an empty statement here.
> +         */
> +        ;

As to "some compilers": Are there any which accept a label not followed
by a statement? Depending on the answer, this comment may be viewed as
superfluous. Or else I'd ask about wording: Besides a grammar issue I
also don't view it as appropriate that a comment talks about "adding"
something when its adjacent code that is meant. That something is there
when the comment is there, hence respective wording should imo be used.

> --- a/xen/include/xen/pci.h
> +++ b/xen/include/xen/pci.h
> @@ -171,6 +171,19 @@ void pcidevs_lock(void);
>  void pcidevs_unlock(void);
>  bool __must_check pcidevs_locked(void);
>  
> +#ifndef NDEBUG
> +/*
> + * Check to ensure there will be no changes to the entries in d->pdev_list (but
> + * not the contents of each entry).
> + * This check is not suitable for protecting other state or critical regions.
> + */
> +#define ASSERT_PDEV_LIST_IS_READ_LOCKED(d)                               \
> +        /* NB: d may be evaluated multiple times, or not at all */       \
> +        ASSERT(pcidevs_locked() || ((d) && rw_is_locked(&(d)->pci_lock)))

Is there actually any case where d can be NULL here?

> +#else
> +#define ASSERT_PDEV_LIST_IS_READ_LOCKED(d) ({ (void)(d); })

Evaluating d here isn't very useful when the assertion expression doesn't
guarantee single evaluation. Plus even if it needed evaluating, there would
be no need to use a compiler extension here:

#define ASSERT_PDEV_LIST_IS_READ_LOCKED(d) ((void)(d))

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 12:18:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 12:18:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682819.1062006 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc2be-0004HR-RE; Mon, 19 Feb 2024 12:18:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682819.1062006; Mon, 19 Feb 2024 12:18:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc2be-0004HK-Oi; Mon, 19 Feb 2024 12:18:46 +0000
Received: by outflank-mailman (input) for mailman id 682819;
 Mon, 19 Feb 2024 12:18:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc2bd-0004HE-Rq
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 12:18:45 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 069badd0-cf21-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 13:18:44 +0100 (CET)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2d23114b19dso15934841fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 04:18:44 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 js21-20020a05600c565500b0040fdb244485sm11066418wmb.40.2024.02.19.04.18.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 04:18:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 069badd0-cf21-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708345124; x=1708949924; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nFksg9v+48EyT1bwVhVYyRpB/kruATCnUqr7x+UCQOI=;
        b=Zs912XdQta0rjpY2j4hXzFmOU2bozqnc8Bl/su7l6JD5Abu0Sj5EUJaC/4bIq3sRx1
         HV4EjN9mZN6QmpKWwcQ6/VT1vwBK1/gszjzYABABBT0VH2AW/IS4UoJc7cRRc/MBzeyM
         OunmGhoLLzXIDeD97q/k/+gjRmTu6h2573QatsVxob2HLqKaYYfUD25pJzvLAqMot+w5
         QVh6qOiYzoYO2Bf9BqPXwx89LCvza/HdQjBfgN5T+yWqcRMMtlq7/q2fDNedhlE29aqO
         IetlLCpSNUyS1MQ5G0h+R1+E0E73dQk5XCdcjijOAsYhkcIMDux1Ogm5561UY/0j/Frd
         Wkyg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708345124; x=1708949924;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nFksg9v+48EyT1bwVhVYyRpB/kruATCnUqr7x+UCQOI=;
        b=IbvNhhiaQRCbmJAloTED9GLtRwXEEmolOfrAtyH7mO8+eSX8RyuEH4JmEXJbC3194n
         Mm6/Zwy5VXJbhUSuHzTTm9XBLDSBQjqeeZLTib1S916+ogqrbRz0z+Q6qXHY/4LF5GuQ
         HaA27R+6IFus17pyb/+Dsoyf0Jasdze1y4bNBU+GbyUMcqzOMkvdTuZarbkFPVbTOz31
         OsNzwkcM4cIkgRq+pZM6GlFNtReZMzAViDpCoMm0J2XXX5mTLs9Xpf8sh2tm/dkkcDTZ
         ruuP7Xat2hA11ERAodH+zOppx1Hq+NZL9R0IYYRN4JzfT8RgKjQQrSahUj60NWBlN/4D
         VBdw==
X-Forwarded-Encrypted: i=1; AJvYcCUfgKLar4r3iQ+W/c8rGwNN/gCbtH1kBL5qCVyQaV/WmPY7BOFSuov21jd2uf2vDK9ATZDjmxMrwISVUEO1Kwrxu0zQ/s4mIYUb21WvWaw=
X-Gm-Message-State: AOJu0YzlsfDfUs442KISV49dEGjfCXhkiW0LQSN4SP3GKTVc4to//htw
	4v1mPdYnILZrfqK1dDD01Es1phz2H6HZ4okykoaK0e/M2WCcdGmVDQ8K6amg6Q==
X-Google-Smtp-Source: AGHT+IENjwWLcmiGaPMEf3C+Bwow3hWX+bUqZ4/B94VXSCNCx+1LazEjpTnDBv84OrO0j6SiTTrF3Q==
X-Received: by 2002:a05:6512:234a:b0:512:b1b7:a285 with SMTP id p10-20020a056512234a00b00512b1b7a285mr2991195lfu.51.1708345124250;
        Mon, 19 Feb 2024 04:18:44 -0800 (PST)
Message-ID: <aaf86d31-7ae0-4e33-8386-dda4bd21496a@suse.com>
Date: Mon, 19 Feb 2024 13:18:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 07/30] xen/asm-generic: introdure nospec.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Julien Grall <julien@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <5889d7a5fa81722472f95cc1448af0be8f359a7d.1707146506.git.oleksii.kurochko@gmail.com>
 <510d74ca-5cce-48f8-93f6-407cd727cf7b@xen.org>
 <8a09e386b709f780f193af39af63b6aeb75c868e.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <8a09e386b709f780f193af39af63b6aeb75c868e.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 19.02.2024 12:59, Oleksii wrote:
> Hi Julien,
> 
> On Sun, 2024-02-18 at 18:30 +0000, Julien Grall wrote:
>> Hi Oleksii,
>>
>> Title: Typo s/introdure/introduce/
>>
>> On 05/02/2024 15:32, Oleksii Kurochko wrote:
>>> The <asm/nospec.h> header is similar between Arm, PPC, and RISC-V,
>>> so it has been moved to asm-generic.
>>
>> I am not 100% convinced that moving this header to asm-generic is a
>> good 
>> idea. At least for Arm, those helpers ought to be non-empty, what
>> about 
>> RISC-V?
> For Arm, they are not taking any action, are they? There are no
> specific fences or other mechanisms inside
> evaluate_nospec()/block_speculation() to address speculation.

The question isn't the status quo, but how things should be looking like
if everything was in place that's (in principle) needed.

> For RISC-V, it can be implemented in a similar manner, at least for
> now. Since these functions are only used in the grant tables code ( for
> Arm and so for RISC-V ), which is not supported by RISC-V.

Same here - the question is whether long term, when gnttab is also
supported, RISC-V would get away without doing anything. Still ...

>> If the answer is they should be non-empty. Then I would consider to
>> keep 
>> the duplication to make clear that each architecture should take
>> their 
>> own decision in term of security.
>>
>> The alternative, is to have a generic implementation that is safe by 
>> default (if that's even possible).
> I am not certain that we can have a generic implementation, as each
> architecture may have specific speculation issues.

... it's theoretically possible that there'd be an arch with no
speculation issues, maybe simply because of not speculating.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 12:48:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 12:48:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682834.1062016 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc33s-0000pw-4c; Mon, 19 Feb 2024 12:47:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682834.1062016; Mon, 19 Feb 2024 12:47:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc33s-0000pp-1h; Mon, 19 Feb 2024 12:47:56 +0000
Received: by outflank-mailman (input) for mailman id 682834;
 Mon, 19 Feb 2024 12:47:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GzZz=J4=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rc33q-0000pR-5u
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 12:47:54 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 17cec957-cf25-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 13:47:52 +0100 (CET)
Received: from MN2PR01CA0040.prod.exchangelabs.com (2603:10b6:208:23f::9) by
 DS7PR12MB8370.namprd12.prod.outlook.com (2603:10b6:8:eb::22) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7316.15; Mon, 19 Feb 2024 12:47:47 +0000
Received: from BL6PEPF0001AB71.namprd02.prod.outlook.com
 (2603:10b6:208:23f:cafe::58) by MN2PR01CA0040.outlook.office365.com
 (2603:10b6:208:23f::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.39 via Frontend
 Transport; Mon, 19 Feb 2024 12:47:47 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL6PEPF0001AB71.mail.protection.outlook.com (10.167.242.164) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 19 Feb 2024 12:47:47 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 19 Feb
 2024 06:47:47 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 19 Feb
 2024 04:47:46 -0800
Received: from [172.30.63.250] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 19 Feb 2024 06:47:45 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 17cec957-cf25-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fMuR+2AZnfmgzS3BtBU3TOimf4Jxfb/Tpa9gJPXqjoizct+h0r0neAgMdYjCFYIApsbWEiNUnwjxAoRFLYhkhHz91cayQa9gswln0JtTsz9UP2p9sOLJU7NlRB2Us7Pw2Grwq6iHgwb5N6YAAxWK3w0oOlFMHnwoCjiT136jGqfPIQjdcz90voy5Yrgb0mWoWyFn0WZDCMpV4r58rp0PWBDu2OWRt48AxiuZdwJ/+1r+Cj6oft9uSgqqoqoKQqrJjunILur+RlUnrI6UkFHP7/t+3wCQj8uW2KX8Zo0qA24sDz6FX7nRr8ODzY5yv6HPwapjAH/LMwPTUv1v4hAfcA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=ACzMdtvp0ucmPu0SShkobBEMb/WPAruvaj5zOLYY+5c=;
 b=PqNmYwIy54Me/6RLBbBPZM9GBr8cuRHDYILJ7uIug50878y1NF/Ktmc8Tv+gXI2uyd5ht5r+cx30EsvBOoUgb6KT8LU5fXQxdDO4p/lti46h/YKJOJffnsMpdT1t9NaXkwj50UFnlyIZfEEiD6EYze7WrOSrnS8C8tqkyWde2UCrNZl4wQU7qNoxLlRJ95U0n8Q2gkbXnuSHNLnc/o4Z+GCya2/zmKkxUHys6G53sq469SXVkySKWSJV3hUb9jlQcnR2c6lJX0HHtopdEnGUllF69/FTzeLEXU4rxVD8TY6RF/G8OkCH60q+cOOhmNNVvVb41f8zWWghcJYsdSz/8Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=ACzMdtvp0ucmPu0SShkobBEMb/WPAruvaj5zOLYY+5c=;
 b=LwuIgmp4isZANVPeF371lHI7aba+gInHxYWOrO5x7tAN+xiMFtBtf4mdSLeLVQ9y2jhu466iZahZ/HSLO4DrKSKu8tHbJQgu8iZTwDnjBx/iYmpsxrEBMspK/JttufgH8czOSW/eKk3Us9MRVM2nMQXh3DKascvivpBLcO3MGgA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <68bb972c-5e45-4706-a06f-0dcd65c40581@amd.com>
Date: Mon, 19 Feb 2024 07:47:45 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13.2 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian
	<kevin.tian@intel.com>, Paul Durrant <paul@xen.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>, <xen-devel@lists.xenproject.org>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <20240219114750.151730-1-stewart.hildebrand@amd.com>
 <8e807612-fd14-4d01-b200-1deb583afa22@suse.com>
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <8e807612-fd14-4d01-b200-1deb583afa22@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB71:EE_|DS7PR12MB8370:EE_
X-MS-Office365-Filtering-Correlation-Id: b1b53d80-d5af-46ab-c067-08dc3148f9af
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	M7/nejeTeKolgFRBZOOYZ1HkVG36fEvQ74JyaW33GYCS/s8v4kH265hbHLsUk847EMwVyX9+A1ZS432qwfNsqZPswKnY9h6X2pvUvzop4Ug3qNLf8/+j03hDYPKg9qZ+g5x90iseXDYLLNb4hdbTN7nb5wvvNOtR3qzZ+sdlRDEr4afbXguvydIt4o1/aBejaSkCMN4iH8pZnwKkwbXA81TPJV7xSMQ141zPTaGYjyMVGcDW2AbcyrsSTmwbNiB1/7Bq9i4OkxRnYF1dhJFjtlphQr0CKtb3w117kKAt3y73m6zm8xfnlbJCL88G3wQuCSMgcW3ux8xwBwYnHANRasXuWiT2CQQoVmJLKPfXsOW+tzJ0J/vqBcwwmPVW046xRqgXn0wBDlzrJ4eH7uR8CCc1fmljmjTySoPi19iK2g1Dgd8Zr/Azb+2T+Xr0LsHbBvbzG/+IL5YJkEpzV0Ot+3ejNitCHqriLIbpspI2nDdsKGKydw8JHcz4yL5Ft0O8Zy7IPpT9NWS0V+Qte6hCLWx3Cx+91M6aso//+Xrz9vooGNSuMM7DJTCIXfezEy/kl2uwSoU3Np+LXxBCbqmuxsn1G4hZ8AXwYelHbz1+X8hphy12j/rHF9wU2iPNZfiIxEXSG5dWm6bsL5uowUfkJg==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(46966006)(40470700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Feb 2024 12:47:47.5281
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b1b53d80-d5af-46ab-c067-08dc3148f9af
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB71.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8370

On 2/19/24 07:10, Jan Beulich wrote:
> On 19.02.2024 12:47, Stewart Hildebrand wrote:
>> @@ -895,6 +891,15 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
>>  {
>>      unsigned int i;
>>  
>> +    /*
>> +     * Assert that d->pdev_list doesn't change. ASSERT_PDEV_LIST_IS_READ_LOCKED
>> +     * is not suitable here because it may allow either pcidevs_lock() or
>> +     * d->pci_lock to be held, but here we rely on d->pci_lock being held, not
>> +     * pcidevs_lock().
>> +     */
>> +    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
>> +    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));
> 
> There's no "d" in sight here, so it's a little odd that "d" is being talked
> about. But I guess people can infer what's meant without too much trouble.

I can s/d->pci_lock/msix->pdev->domain->pci_lock/ for the next rev.

> 
>> @@ -313,17 +316,36 @@ void vpci_dump_msi(void)
>>                  {
>>                      /*
>>                       * On error vpci_msix_arch_print will always return without
>> -                     * holding the lock.
>> +                     * holding the locks.
>>                       */
>>                      printk("unable to print all MSI-X entries: %d\n", rc);
>> -                    process_pending_softirqs();
>> -                    continue;
>> +                    goto pdev_done;
>>                  }
>>              }
>>  
>> +            /*
>> +             * Unlock locks to process pending softirqs. This is
>> +             * potentially unsafe, as d->pdev_list can be changed in
>> +             * meantime.
>> +             */
>>              spin_unlock(&pdev->vpci->lock);
>> +            read_unlock(&d->pci_lock);
>> +        pdev_done:
>>              process_pending_softirqs();
>> +            if ( !read_trylock(&d->pci_lock) )
>> +            {
>> +                printk("unable to access other devices for the domain\n");
>> +                goto domain_done;
>> +            }
>>          }
>> +        read_unlock(&d->pci_lock);
>> +    domain_done:
>> +        /*
>> +         * We need this label at the end of the loop, but some
>> +         * compilers might not be happy about label at the end of the
>> +         * compound statement so we adding an empty statement here.
>> +         */
>> +        ;
> 
> As to "some compilers": Are there any which accept a label not followed
> by a statement? Depending on the answer, this comment may be viewed as
> superfluous. Or else I'd ask about wording: Besides a grammar issue I
> also don't view it as appropriate that a comment talks about "adding"
> something when its adjacent code that is meant. That something is there
> when the comment is there, hence respective wording should imo be used.

It seems like hit or miss whether gcc would accept it or not (prior
discussion at [1]). I agree the comment is rather lengthy for what it's
trying to convey. I'd be happy to either remove the comment or reduce
it to:

    domain_done:
        ; /* Empty statement to make some compilers happy */

[1] https://lore.kernel.org/xen-devel/98b8c131-b0b9-f46c-5f46-c2136f2e3b4e@amd.com/

> 
>> --- a/xen/include/xen/pci.h
>> +++ b/xen/include/xen/pci.h
>> @@ -171,6 +171,19 @@ void pcidevs_lock(void);
>>  void pcidevs_unlock(void);
>>  bool __must_check pcidevs_locked(void);
>>  
>> +#ifndef NDEBUG
>> +/*
>> + * Check to ensure there will be no changes to the entries in d->pdev_list (but
>> + * not the contents of each entry).
>> + * This check is not suitable for protecting other state or critical regions.
>> + */
>> +#define ASSERT_PDEV_LIST_IS_READ_LOCKED(d)                               \
>> +        /* NB: d may be evaluated multiple times, or not at all */       \
>> +        ASSERT(pcidevs_locked() || ((d) && rw_is_locked(&(d)->pci_lock)))
> 
> Is there actually any case where d can be NULL here?

Yes, when called from ns16550 driver, if the driver failed to make the
device RO.

> 
>> +#else
>> +#define ASSERT_PDEV_LIST_IS_READ_LOCKED(d) ({ (void)(d); })
> 
> Evaluating d here isn't very useful when the assertion expression doesn't
> guarantee single evaluation. Plus even if it needed evaluating, there would
> be no need to use a compiler extension here:
> 
> #define ASSERT_PDEV_LIST_IS_READ_LOCKED(d) ((void)(d))

OK, I can make this change.


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 12:50:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 12:50:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682789.1062027 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc35x-0002Wd-Le; Mon, 19 Feb 2024 12:50:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682789.1062027; Mon, 19 Feb 2024 12:50:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc35x-0002WA-H8; Mon, 19 Feb 2024 12:50:05 +0000
Received: by outflank-mailman (input) for mailman id 682789;
 Mon, 19 Feb 2024 11:41:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=TU0m=J4=gmail.com=guixiongwei@srs-se1.protection.inumbo.net>)
 id 1rc21X-0005nR-17
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 11:41:27 +0000
Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com
 [2607:f8b0:4864:20::544])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cdf719d4-cf1b-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 12:41:23 +0100 (CET)
Received: by mail-pg1-x544.google.com with SMTP id
 41be03b00d2f7-5cedfc32250so3451510a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 03:41:23 -0800 (PST)
Received: from [10.4.72.30] ([139.177.225.242])
 by smtp.gmail.com with ESMTPSA id
 w3-20020a17090a528300b002990d91d31dsm4956437pjh.15.2024.02.19.03.41.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 03:41:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cdf719d4-cf1b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708342881; x=1708947681; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject
         :user-agent:mime-version:date:message-id:from:to:cc:subject:date
         :message-id:reply-to;
        bh=DseBf/8i17+ETW7/Dma7xksfJzr0mR0ZxNxdl75ecBo=;
        b=N6lEOHHdxvDxzBeKYmSbMgK2LT7CcyqkuRoxLtyIEJNFgmvsePUAwP3VqL6QsVBWpL
         8+yY4N5i16Hx+5uAuoiPtb9xvHakEWxT893pY4Rq3bZ7OFWuSeEktS5ofiZCk89mc7ER
         cd4FHphVZJCvjdxQLcEvoih309u+wRvUu6Tt2SMjgkhDjng4d1CzrIsqZ09i6usEINWb
         khP8fnZJLgO8yUzTzIEZSowu/9BMLJ1iM+Ep+KFkb/ynVoEXvb8CDEuVzSP+tSvrMDKB
         mrkqOMXuiXFnAbkpkZCM/es2huylxiMhuv6IMyScwcTLYpF53N5yaFS5RZ9PbxCkkrkr
         3VPw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708342881; x=1708947681;
        h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject
         :user-agent:mime-version:date:message-id:x-gm-message-state:from:to
         :cc:subject:date:message-id:reply-to;
        bh=DseBf/8i17+ETW7/Dma7xksfJzr0mR0ZxNxdl75ecBo=;
        b=MPNM00Rjg+Q8SJdL3pzeOYQeZ+og8QwC3uPW+nrdQZ8gWxd42SX9GdOgKVvwI03ytI
         wLL7Uu+ClIqjaraaHQt84pKE3jLdIkoGxIpdZqzNAy0eOfEk1+Rnof0ZPchNfKimPEgZ
         9dGfKsWwWv2WlcpdaY0xwi54JRKj9ne1bxRymnlFxUwWUO6pzymxlr8uVecnAEmRl5p7
         rQ/x/DjCQZjmOlr901gr1Bq5WW5tj4LiB6OMp8NzvYcg9KalMXUeDcOsHLCYSJcFB0Wt
         V+a+NAw7h8nN8ppkCYZVs5hLg7jFYzBDRYpsB68L2D1oLK2wazfOn7Bz2XteOHJk80bT
         VjWQ==
X-Forwarded-Encrypted: i=1; AJvYcCWNRI0aE+eYM+u3J1UH9dL213VwOGzjETuJJWL4YNPjiHo7bgpYUf830JwVV2aZZWUZQBjPyT5Iwq+FUnuPRuz2oCW0OuX1OZ0g3vvur6U=
X-Gm-Message-State: AOJu0YwCeIiVlau8Ae7BWko9EikfQyFxUYKj9ctXTZl08IPYCPM+Aiqx
	WDEnUbSkrz/WhyCYFbamkavzGNvEfjvjnO0rKQin38l+FaXSvg49
X-Google-Smtp-Source: AGHT+IHt9rUtlOsN4joDJDaLQ46HZsx+ssQX5mNojSpy9mOPjyyNMVV/S4A0DycY+IBGc04fyk0DsA==
X-Received: by 2002:a17:90a:134f:b0:299:11c0:13c4 with SMTP id y15-20020a17090a134f00b0029911c013c4mr7631636pjf.24.1708342881478;
        Mon, 19 Feb 2024 03:41:21 -0800 (PST)
Message-ID: <7a1c2bf7-df85-431a-ae21-82eeb171f1c4@gmail.com>
Date: Mon, 19 Feb 2024 19:41:13 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RESEND RFC] kernel/ksysfs.c: restrict /sys/kernel/notes to root
 access
To: Kees Cook <keescook@chromium.org>, Greg KH <gregkh@linuxfoundation.org>
Cc: linux-hardening@vger.kernel.org, Juergen Gross <jgross@suse.com>,
 Boris Ostrovsky <boris.ostrovsky@oracle.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
 Guixiong Wei <weiguixiong@bytedance.com>, linux-kernel@vger.kernel.org,
 xen-devel@lists.xenproject.org
References: <20240218073501.54555-1-guixiongwei@gmail.com>
 <2024021825-skiing-trustee-a56a@gregkh> <202402180028.6DB512C50@keescook>
From: Guixiong Wei <guixiongwei@gmail.com>
In-Reply-To: <202402180028.6DB512C50@keescook>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit


On 2024/2/18 17:04, Kees Cook wrote:
> On Sun, Feb 18, 2024 at 08:47:03AM +0100, Greg KH wrote:
>> On Sun, Feb 18, 2024 at 03:35:01PM +0800, Guixiong Wei wrote:
>>> From: Guixiong Wei <weiguixiong@bytedance.com>
>>>
>>> Restrict non-privileged user access to /sys/kernel/notes to
>>> avoid security attack.
>>>
>>> The non-privileged users have read access to notes. The notes
>>> expose the load address of startup_xen. This address could be
>>> used to bypass KASLR.
>> How can it be used to bypass it?
>>
>> KASLR is, for local users, pretty much not an issue, as that's not what
>> it protects from, only remote ones.
>>
>>> For example, the startup_xen is built at 0xffffffff82465180 and
>>> commit_creds is built at 0xffffffff810ad570 which could read from
>>> the /boot/System.map. And the loaded address of startup_xen is
>>> 0xffffffffbc265180 which read from /sys/kernel/notes. So the loaded
>>> address of commit_creds is 0xffffffffbc265180 - (0xffffffff82465180
>>>   - 0xffffffff810ad570) = 0xffffffffbaead570.
>> I've cc: the hardening list on this, I'm sure the developers there have
>> opinions about this.
> Oh eww, why is Xen spewing addresses into the notes section? (This must
> be how it finds its entry point? But that would be before relocations
> happen...)
>
> But yes, I can confirm that relocations are done against the .notes
> section at boot, so the addresses exposed in .notes is an immediate
> KASLR offset exposure.
>
> In /sys/kernel/notes (are there any tools to read this? I wrote my own...)
>
> 	type: 1
> 	name: Xen
> 	desc: 0xb4a711c0 0xffffffff
>
> which matches a privileged read of /proc/kallsysms:
>
> 	ffffffffb4a711c0 T startup_xen
>
> (and the hypercall_page too)
>
> There are all coming from arch/x86/xen/xen-head.S:
>
>          ELFNOTE(Xen, XEN_ELFNOTE_GUEST_OS,       .asciz "linux")
>          ELFNOTE(Xen, XEN_ELFNOTE_GUEST_VERSION,  .asciz "2.6")
>          ELFNOTE(Xen, XEN_ELFNOTE_XEN_VERSION,    .asciz "xen-3.0")
> #ifdef CONFIG_XEN_PV
>          ELFNOTE(Xen, XEN_ELFNOTE_VIRT_BASE,      _ASM_PTR __START_KERNEL_map)
>          /* Map the p2m table to a 512GB-aligned user address. */
>          ELFNOTE(Xen, XEN_ELFNOTE_INIT_P2M,       .quad (PUD_SIZE * PTRS_PER_PUD))
>          ELFNOTE(Xen, XEN_ELFNOTE_ENTRY,          _ASM_PTR startup_xen)
> ...
>
> Introduced in commit 5ead97c84fa7 ("xen: Core Xen implementation")
>
> Exposed in commit da1a679cde9b ("Add /sys/kernel/notes")
>
> Amazingly these both went in on the same release (v2.6.23, 2007). This
> has been exposed for longer than KASLR has been upstream. :P
>
>>> Signed-off-by: Guixiong Wei <weiguixiong@bytedance.com>
>>> ---
>>>   kernel/ksysfs.c | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/kernel/ksysfs.c b/kernel/ksysfs.c
>>> index b1292a57c2a5..09bc0730239b 100644
>>> --- a/kernel/ksysfs.c
>>> +++ b/kernel/ksysfs.c
>>> @@ -199,7 +199,7 @@ static ssize_t notes_read(struct file *filp, struct kobject *kobj,
>>>   static struct bin_attribute notes_attr __ro_after_init  = {
>>>   	.attr = {
>>>   		.name = "notes",
>>> -		.mode = S_IRUGO,
>>> +		.mode = S_IRUSR,
>>>   	},
>>>   	.read = &notes_read,
>>>   };
> Yes please.
>
> Reviewed-by: Kees Cook <keescook@chromium.org>
>
> I wonder if we should also remove relocations that are aimed at the
> .notes section for good measure? If that had already been true, this
> would have just given the same info as System.map.
That's a good idea, but it depends on whether the user space tool can 
accept the remove relocation address.
>> No objection from me, but what userspace tool requires access to this
>> file today?  Will it break if permissions are changed on it?
 From the exposed content, it seems that the main users are Xen-related 
tools. I add Xen list, developers should be able to provide some 
information.
>> And what about the module notes files?  If you change one, shouldn't you
>> change all?
 From what I currently know, the module note files do not expose any 
kernel symbol address, so there is no need for modification.
> Luckily all of _those_ contain what I'd expect: the Linux and
> GNU.build-id notes, which are harmless. But if we're going to suddenly
> have things appearing in here, let's make those root-only too.
Yes, but I also not sure whether the user space tools using this file 
can accept this permission modification.


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 13:12:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 13:12:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682846.1062037 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc3S0-0006Eg-Cv; Mon, 19 Feb 2024 13:12:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682846.1062037; Mon, 19 Feb 2024 13:12:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc3S0-0006EZ-9n; Mon, 19 Feb 2024 13:12:52 +0000
Received: by outflank-mailman (input) for mailman id 682846;
 Mon, 19 Feb 2024 13:12:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc3Rz-0006ET-2D
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 13:12:51 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 939dfef0-cf28-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 14:12:49 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-410e820a4feso29787235e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 05:12:47 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 t12-20020a5d460c000000b0033b48190e5esm10330769wrq.67.2024.02.19.05.12.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 05:12:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 939dfef0-cf28-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708348367; x=1708953167; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+SQhz9bz5DkV8QKGE2lk//F5x/WEOKaWqPgS8b52BRQ=;
        b=ZtNkAtrRlCFtHBoQ16uZdDmrfaC1ikpOMc68/5revpwZ98uNb1Dh4ye6Tvvt/uK+ic
         U20YH/FvwxyHwKbxWCNfwSqnmmPdCPci3r2zJaGBNUXE2q8HX51nuOdcOTjsGYi5Vcxs
         eSY/6i8Onjngf+63ewbIbfAZOnvFSLV741LKl01wC0ZASfmwqSCDuPUPywYNetx/ia9e
         fOPYLbOROk6hajSfVKS5PQUAAeEfckH3vfeaV6WpQ6LCAXm9UQ4yaC9ZF8hkocc/BqTj
         e8rwiqEItUKXz07nK+7tutE9jYEGDKPYB63z5ututNyaOTa77uf5BaSkt7YWkZxg4oSF
         PTiA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708348367; x=1708953167;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+SQhz9bz5DkV8QKGE2lk//F5x/WEOKaWqPgS8b52BRQ=;
        b=SNkyBIIYvnnO3MtIj7kdVnOcvsYA4As4ZsYg2iqXTG1dKVOoFztITrCnrFb+3uS/Ld
         iazBbmoSUROcfPkDEV0pXusFhf8aFdob5qCwRrGib6EK+l51FKvYArBnKksF+JWL2Hn6
         IC3gsfka3K1xvBIlEyXIwXYSbmbLVj83C/St00YyaxFwYky+XmbYqmN7CFGIvXDwATaC
         dkC/7ZrK4vIIpZCCNBNVA3VhNUHJ46/Odt2bnQqxqvi3lAeTKuG4IJjYwmsKtGIO8HAF
         kMRUAA65uLveRoJ8yYuG32lK41u2VWfK5VI1hwz/zmfIO7RQKEDJtalAJyV0VVKdhH1i
         Mpdg==
X-Forwarded-Encrypted: i=1; AJvYcCVyC+EM6oe37wKzt+RXTndjBZ1wNsV9aDxVkE9Bv7x+lCrt2E2X2BI0mUxeF3YQw6NwE7q4MYiUkvbllHnj3zCG0B7Auy5qlt21oxDloHM=
X-Gm-Message-State: AOJu0YxP+ps0lYKcF7t8vQ/Oysl4nhXT5x0CGDbTclCDG9l0rLryZ2x2
	VYT1P3oNPZ01ivgg8rIHAaBCjiH0bsjZuV/7pDon10n7r+0bJamhmuIqkPo6yA==
X-Google-Smtp-Source: AGHT+IG2UxnGZZICregw2HZKS8TCYBUvJRgU+tkohDP1e3pbPoACQ+yLbuyFaV51CaVq+TABtsVu4g==
X-Received: by 2002:adf:e947:0:b0:33d:2567:995c with SMTP id m7-20020adfe947000000b0033d2567995cmr6009730wrn.1.1708348367244;
        Mon, 19 Feb 2024 05:12:47 -0800 (PST)
Message-ID: <39e8845b-8d41-40e5-8e7a-49e734012d78@suse.com>
Date: Mon, 19 Feb 2024 14:12:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13.2 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Paul Durrant <paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <20240219114750.151730-1-stewart.hildebrand@amd.com>
 <8e807612-fd14-4d01-b200-1deb583afa22@suse.com>
 <68bb972c-5e45-4706-a06f-0dcd65c40581@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <68bb972c-5e45-4706-a06f-0dcd65c40581@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 19.02.2024 13:47, Stewart Hildebrand wrote:
> On 2/19/24 07:10, Jan Beulich wrote:
>> On 19.02.2024 12:47, Stewart Hildebrand wrote:
>>> @@ -895,6 +891,15 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
>>>  {
>>>      unsigned int i;
>>>  
>>> +    /*
>>> +     * Assert that d->pdev_list doesn't change. ASSERT_PDEV_LIST_IS_READ_LOCKED
>>> +     * is not suitable here because it may allow either pcidevs_lock() or
>>> +     * d->pci_lock to be held, but here we rely on d->pci_lock being held, not
>>> +     * pcidevs_lock().
>>> +     */
>>> +    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
>>> +    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));
>>
>> There's no "d" in sight here, so it's a little odd that "d" is being talked
>> about. But I guess people can infer what's meant without too much trouble.
> 
> I can s/d->pci_lock/msix->pdev->domain->pci_lock/ for the next rev.

Or simply drop the d-s? That would be better for readability's sake,
I think.

>>> @@ -313,17 +316,36 @@ void vpci_dump_msi(void)
>>>                  {
>>>                      /*
>>>                       * On error vpci_msix_arch_print will always return without
>>> -                     * holding the lock.
>>> +                     * holding the locks.
>>>                       */
>>>                      printk("unable to print all MSI-X entries: %d\n", rc);
>>> -                    process_pending_softirqs();
>>> -                    continue;
>>> +                    goto pdev_done;
>>>                  }
>>>              }
>>>  
>>> +            /*
>>> +             * Unlock locks to process pending softirqs. This is
>>> +             * potentially unsafe, as d->pdev_list can be changed in
>>> +             * meantime.
>>> +             */
>>>              spin_unlock(&pdev->vpci->lock);
>>> +            read_unlock(&d->pci_lock);
>>> +        pdev_done:
>>>              process_pending_softirqs();
>>> +            if ( !read_trylock(&d->pci_lock) )
>>> +            {
>>> +                printk("unable to access other devices for the domain\n");
>>> +                goto domain_done;
>>> +            }
>>>          }
>>> +        read_unlock(&d->pci_lock);
>>> +    domain_done:
>>> +        /*
>>> +         * We need this label at the end of the loop, but some
>>> +         * compilers might not be happy about label at the end of the
>>> +         * compound statement so we adding an empty statement here.
>>> +         */
>>> +        ;
>>
>> As to "some compilers": Are there any which accept a label not followed
>> by a statement? Depending on the answer, this comment may be viewed as
>> superfluous. Or else I'd ask about wording: Besides a grammar issue I
>> also don't view it as appropriate that a comment talks about "adding"
>> something when its adjacent code that is meant. That something is there
>> when the comment is there, hence respective wording should imo be used.
> 
> It seems like hit or miss whether gcc would accept it or not (prior
> discussion at [1]). I agree the comment is rather lengthy for what it's
> trying to convey. I'd be happy to either remove the comment or reduce
> it to:
> 
>     domain_done:
>         ; /* Empty statement to make some compilers happy */
> 
> [1] https://lore.kernel.org/xen-devel/98b8c131-b0b9-f46c-5f46-c2136f2e3b4e@amd.com/

This earlier discussion only proves that there is at least one compiler
objecting. There's no proof there that any compiler exists which, as a
language extension, actually permits such syntax. Yet if the comment
was purely about normal language syntax, then imo it should be zapped
altogether, not just be shrunk.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 13:25:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 13:25:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682852.1062047 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc3e7-00089n-Gl; Mon, 19 Feb 2024 13:25:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682852.1062047; Mon, 19 Feb 2024 13:25:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc3e7-00089g-DG; Mon, 19 Feb 2024 13:25:23 +0000
Received: by outflank-mailman (input) for mailman id 682852;
 Mon, 19 Feb 2024 13:25:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NiMn=J4=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rc3e6-00089a-Mg
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 13:25:22 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 54b036c9-cf2a-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 14:25:21 +0100 (CET)
Received: from Dell.bugseng.com (unknown [37.161.73.215])
 by support.bugseng.com (Postfix) with ESMTPSA id E01804EE073A;
 Mon, 19 Feb 2024 14:25:19 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54b036c9-cf2a-11ee-8a52-1f161083a0e0
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH] automation/eclair: add deviation for MISRA C:2012 Rule 16.3
Date: Mon, 19 Feb 2024 14:24:41 +0100
Message-Id: <b1d2b64c8117d61ea42cf4e9feae128541eb0b61.1708348799.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update ECLAIR configuration to consider safe switch clauses ending
with __{get,put}_user_bad().

Update docs/misra/deviations.rst accordingly.

Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
---
 automation/eclair_analysis/ECLAIR/deviations.ecl | 4 ++++
 docs/misra/deviations.rst                        | 6 ++++++
 2 files changed, 10 insertions(+)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
index fd32ff8a9c..831b5d4c67 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -368,6 +368,10 @@ safe."
 -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/BUG\\(\\);/))))"}
 -doc_end
 
+-doc_begin="Switch clauses ending with constructs \"__get_user_bad()\" and \"__put_user_bad()\" are safe: they denote an unreachable program point."
+-config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/__(put|get)_user_bad\\(\\);/))))"}
+-doc_end
+
 -doc_begin="Switch clauses not ending with the break statement are safe if an
 explicit comment indicating the fallthrough intention is present."
 -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(^(?s).*/\\* [fF]all ?through.? \\*/.*$,0..1))))"}
diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
index 123c78e20a..58f4fac18e 100644
--- a/docs/misra/deviations.rst
+++ b/docs/misra/deviations.rst
@@ -307,6 +307,12 @@ Deviations related to MISRA C:2012 Rules:
      - Switch clauses ending with failure method \"BUG()\" are safe.
      - Tagged as `safe` for ECLAIR.
 
+   * - R16.3
+     - Switch clauses ending with constructs
+       \"__get_user_bad()\" and \"__put_user_bad()\" are safe:
+       they denote an unreachable program point.
+     - Tagged as `safe` for ECLAIR.
+
    * - R16.3
      - Existing switch clauses not ending with the break statement are safe if
        an explicit comment indicating the fallthrough intention is present.
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 19 13:36:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 13:36:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682858.1062060 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc3p7-0001bK-HX; Mon, 19 Feb 2024 13:36:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682858.1062060; Mon, 19 Feb 2024 13:36:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc3p7-0001bD-DW; Mon, 19 Feb 2024 13:36:45 +0000
Received: by outflank-mailman (input) for mailman id 682858;
 Mon, 19 Feb 2024 13:36:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc3p6-0001b7-0V
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 13:36:44 +0000
Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com
 [2a00:1450:4864:20::332])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ea4cc532-cf2b-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 14:36:41 +0100 (CET)
Received: by mail-wm1-x332.google.com with SMTP id
 5b1f17b1804b1-4126aee00b2so1786715e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 05:36:41 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l20-20020a05600c1d1400b0041256ab5becsm8564093wms.26.2024.02.19.05.36.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 05:36:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea4cc532-cf2b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708349801; x=1708954601; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jrKlw3n1B33BnCvjbfAiqSOmbtplbXt1ex1CFFaeOvM=;
        b=FLlhg+IVoeYNQt7WpJq8X0I7e5Bgz7ZupohKAofsRKHX1QCnw1RCMs4xP1Fg/5nvBf
         AakCw/Pf/DT5/wqsRtnH/1J73tVcs708CtLnXNZnWTq4CqjR+4egqG11bX7MzKTO1jqm
         EISACTLo5m6U+xfpdpkf45I+Vd5pvC5ACw5ugW9vzUTsKxdOktYznYtjCZEc2SC7+S9R
         65rtTZUBCORIIxxKn7stxGMqqsCs4OfmoLJv006boSNUUUm6z6RbOcK86rqTOVBi1Vof
         VpkmEUeU90fXBDKcKk9UFCtS9aX2pp15dFB1PfaRJULvBeRoCK7cEkTYNzDvI2Y5IRx+
         zTKA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708349801; x=1708954601;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jrKlw3n1B33BnCvjbfAiqSOmbtplbXt1ex1CFFaeOvM=;
        b=F7hFHv1vXpqknI5iiYS8cKllpd1grtudKrher/VMvjhTlB2+Ix9m2Z/Nh490zyp5H0
         beo8MWJ8CBXv1iMzyAsEgKJiTpKyYBoyViqpWfCRyehp/N/OIZ0EZJlpNWZadEHUCa+G
         GH4Atv3c2PHCEiH2wYmZTzDXGtZV02KdhyLo6GdxbBiXdxglLzH5vjcrFYKZ1bmRCDyp
         pkF4mJPqL0ZHDl/G1+8F6M6DpXhhjQ7idK7/ocu55h1PvADTeJ+Ap30r4ZhlH5sj6sxO
         KpwoFwcoG6DE88jFynhRUQ/zaNKRWw+YipWA8euw3eaML8yf8knwk/DOJckIeFrDi4Fg
         Ltkg==
X-Forwarded-Encrypted: i=1; AJvYcCU6QzX3TkjeqAVyuqZtqPwBHGonbnmYkf0HMFcXjEpDJN7ePj3uCCvzqHlVnGj0wyB8jn5oGKyZvkK+qDY9Oq6APeXjqnHfLwuliE3mzIY=
X-Gm-Message-State: AOJu0YxnI1v4l6YYuhoXeUdiOZuJEDXUjHE14y5RIeVBL4+UpYj2T+XQ
	Fv+1OlHKncq9XBP32PS5Wc8itA0K+rg86vz/8S/oMpiapfO6US9BgJ3RqoSiKQ==
X-Google-Smtp-Source: AGHT+IEz8jMnj3jbA0NmGdJf8EHben7mw302yJEQWn+Atx9WKClUZhrR/jOPkF4EjahffmrmQuIUZw==
X-Received: by 2002:a05:600c:4713:b0:412:6101:9158 with SMTP id v19-20020a05600c471300b0041261019158mr3224184wmo.29.1708349801285;
        Mon, 19 Feb 2024 05:36:41 -0800 (PST)
Message-ID: <66962dfc-db89-40bc-8411-9b92f9458b16@suse.com>
Date: Mon, 19 Feb 2024 14:36:40 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/6] xen/hvm: Convert hap_capabilities into a bitfield
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Jun Nakajima <jun.nakajima@intel.com>,
 Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-2-george.dunlap@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240206012051.3564035-2-george.dunlap@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 02:20, George Dunlap wrote:
> hvm_function_table is an internal structure; rather than manually
> |-ing and &-ing bits, just make it a boolean bitfield and let the
> compiler do all the work.  This makes everything easier to read, and
> presumably allows the compiler more flexibility in producing efficient
> code.
> 
> No functional change intended.
> 
> Signed-off-by: George Dunlap <george.dunlap@cloud.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

> ---
> Questions:
> 
> * Should hap_superpage_2m really be set unconditionally, or should we
>   condition it on cpu_has_svm_npt?

That's HAP capabilities; there's not going to be any use of HAP when
there's no NPT (on an AMD system). IOW - all is fine as is, imo.

> * Do we really need to "!!cpu_has_svm_npt"?  If so, wouldn't it be
>   better to put the "!!"  in the #define, rather than requiring the
>   user to know that it's needed?

Considering that hap_supported is bool now, the !! can simply be
dropped. We've been doing so as code was touched anyway, not in a
concerted effort.

> --- a/xen/arch/x86/hvm/vmx/vmcs.c
> +++ b/xen/arch/x86/hvm/vmx/vmcs.c
> @@ -113,8 +113,8 @@ static int cf_check parse_ept_param_runtime(const char *s)
>      int val;
>  
>      if ( !cpu_has_vmx_ept || !hvm_funcs.hap_supported ||
> -         !(hvm_funcs.hap_capabilities &
> -           (HVM_HAP_SUPERPAGE_2MB | HVM_HAP_SUPERPAGE_1GB)) )
> +         !(hvm_funcs.caps.hap_superpage_2mb ||
> +           hvm_funcs.caps.hap_superpage_1gb) )
>      {
>          printk("VMX: EPT not available, or not in use - ignoring\n");

Just to mention it: The conditional and the log message don't really
fit together. (I was first wondering what the 2mb/1gb checks had to
do here at all, but that's immediately clear when seeing that the
only sub-option here is "exec-sp".)

> @@ -104,8 +96,11 @@ struct hvm_function_table {
>      /* Hardware virtual interrupt delivery enable? */
>      bool virtual_intr_delivery_enabled;
>  
> -    /* Indicate HAP capabilities. */
> -    unsigned int hap_capabilities;
> +    struct {
> +        /* Indicate HAP capabilities. */
> +        bool hap_superpage_1gb:1,
> +            hap_superpage_2mb:1;

Nit: Would be nice imo if the two identifiers aligned vertically with
one another.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 13:43:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 13:43:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682865.1062070 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc3vb-0003WH-Ao; Mon, 19 Feb 2024 13:43:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682865.1062070; Mon, 19 Feb 2024 13:43:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc3vb-0003WA-7j; Mon, 19 Feb 2024 13:43:27 +0000
Received: by outflank-mailman (input) for mailman id 682865;
 Mon, 19 Feb 2024 13:43:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc3vZ-0003W1-2E
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 13:43:25 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id da0af486-cf2c-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 14:43:24 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-41264195d5cso9253545e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 05:43:24 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p8-20020a5d68c8000000b0033b2799815csm10408696wrw.86.2024.02.19.05.43.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 05:43:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da0af486-cf2c-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708350203; x=1708955003; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vrPLVACfRMC6xDUdFgYvzOFYlNcALVZD/fl5SsiDjhM=;
        b=WY6vP9c/b72HNFaLlKZLB4I9jG93NQo2JTEBeAbu7ylUUTyyMvteyayLXFU6pM1fE0
         Z3aj7o1IcfhWVSE72f/4W4oVUbm7PkZjC4CGgQbxaoSDk3hvgG4uEYfXAe4oAqLToyvz
         +o4IL0wRwPjRg76qCucZOydgTUKYcYpLoR30V779sTi8XByW91ko44MHSnGPUiefAJJe
         JyWWwxl68q/aBIaMOuuxLeqmZyg7bpiVnU9ozsQtPAagHoXRwyBBFLQTo2whJJ3505UU
         gjkoUxK1ip7udOLt0l9xtm8eDU45+h2dCyh1jERPvLrHgJOFU+GWvZhYesFX49WQq2ET
         MM3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708350203; x=1708955003;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vrPLVACfRMC6xDUdFgYvzOFYlNcALVZD/fl5SsiDjhM=;
        b=Q77hl55G/SRpSKYK4sB0Zi1NaVmwXsaip6eKy4ikesw7HLZbh36jSrJLxc8wpo5z15
         oST2S8B4FMi2sd+ZLcv51UV9QwDqLO8CfvmRrMaUtfh3ZWZubXhe5Ah73o4UsTvGI3Mm
         pkpvFYXc+HZnSe9iD/Iw0Z4vv52vk/ZiOyt/B4oE+79PcWN/J4SwmW4e/i1cGUQqeNlE
         IBD81ToOSWxQOk2HZ8TG6Xwm7OJpkH+gDmOIYw20gRPgdzgxQie8XWqEczZ0srMHZvhi
         55D3qtLq4JnmQBBIN1EgRpUGnnctJH+04BwAKrl6WD85kOIxAxObbMWJFbo0nad9bEoM
         5qiw==
X-Forwarded-Encrypted: i=1; AJvYcCVC8jNtwTrOQUt8pzF0wx+/2FKXE037gZtkq2U+VyMZc5TuC5N6zSPGAXNwaLA5339kfJbCOhuS1LEpYaiiwat7EOaa/puFgXVw6BHBDL8=
X-Gm-Message-State: AOJu0YznM0hUx3vWXmz5uYvYpmQbPHDABFz3aE8h4ai+ImB97dzbwHXz
	X4foTChwhgqf4n1NZkvpbZcIhKbchOox922py79bcLcR9ww27dDiW2dk+ltLXg==
X-Google-Smtp-Source: AGHT+IF2lK8c7rkP6Aiqra5R6gB8ALiWIYO28aXg+KW4O17lox/081/7gX8uH3BrFn9LjncW69ZmOw==
X-Received: by 2002:adf:e504:0:b0:33c:f35c:a395 with SMTP id j4-20020adfe504000000b0033cf35ca395mr8147693wrm.52.1708350203500;
        Mon, 19 Feb 2024 05:43:23 -0800 (PST)
Message-ID: <a6e8bd32-cc32-4084-907d-e2cd6d46e3e6@suse.com>
Date: Mon, 19 Feb 2024 14:43:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair: add deviation for MISRA C:2012
 Rule 16.3
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <b1d2b64c8117d61ea42cf4e9feae128541eb0b61.1708348799.git.federico.serafini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b1d2b64c8117d61ea42cf4e9feae128541eb0b61.1708348799.git.federico.serafini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 19.02.2024 14:24, Federico Serafini wrote:
> Update ECLAIR configuration to consider safe switch clauses ending
> with __{get,put}_user_bad().
> 
> Update docs/misra/deviations.rst accordingly.
> 
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

As mentioned I'm not happy with this, not the least because of it being
unclear why these two would be deviated, when there's no sign of a
similar deviation for, say, __bad_atomic_size(). Imo this approach
doesn't scale, and that's already leaving aside that the purpose of
identically named (pseudo-)helpers could differ between architectures,
thus putting under question ...

> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> @@ -368,6 +368,10 @@ safe."
>  -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/BUG\\(\\);/))))"}
>  -doc_end
>  
> +-doc_begin="Switch clauses ending with constructs \"__get_user_bad()\" and \"__put_user_bad()\" are safe: they denote an unreachable program point."
> +-config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/__(put|get)_user_bad\\(\\);/))))"}
> +-doc_end

... the global scope of such a deviation. While it may not be a good idea,
even within an arch such (pseudo-)helpers could be used for multiple
distinct purposes.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 13:57:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 13:57:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682871.1062079 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc48s-0005Xf-FS; Mon, 19 Feb 2024 13:57:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682871.1062079; Mon, 19 Feb 2024 13:57:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc48s-0005XY-Cg; Mon, 19 Feb 2024 13:57:10 +0000
Received: by outflank-mailman (input) for mailman id 682871;
 Mon, 19 Feb 2024 13:57:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OQos=J4=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rc48r-0005XO-LU
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 13:57:09 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c406a5e5-cf2e-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 14:57:06 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 2F4431F800;
 Mon, 19 Feb 2024 13:57:05 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 04BFC13585;
 Mon, 19 Feb 2024 13:57:04 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id Y8ktOzBe02W9DAAAn2gu4w
 (envelope-from <jgross@suse.com>); Mon, 19 Feb 2024 13:57:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c406a5e5-cf2e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708351025; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=SwodJ1pTCGhLAwKhDNMO6/ovhBbx7H7Fl1qXIfAUnBE=;
	b=X/lfICm01rh/mCjemMUvY0penYfitc1mzMqdXdVnl0vTZs/N4XpLcT7OaYpWgCpwZh/dHr
	QVoxvVU2BZVJUoyyrcHujB19tD5TbpSyN0LOHSweFu/Z16LyuyFqvgMutdR4HpwCigAv6D
	Q4xVD4XrqLl/G+4BsobRUT7PNm4K210=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708351025; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=SwodJ1pTCGhLAwKhDNMO6/ovhBbx7H7Fl1qXIfAUnBE=;
	b=X/lfICm01rh/mCjemMUvY0penYfitc1mzMqdXdVnl0vTZs/N4XpLcT7OaYpWgCpwZh/dHr
	QVoxvVU2BZVJUoyyrcHujB19tD5TbpSyN0LOHSweFu/Z16LyuyFqvgMutdR4HpwCigAv6D
	Q4xVD4XrqLl/G+4BsobRUT7PNm4K210=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH] tools/9pfsd: add missing va_end() in fill_data()
Date: Mon, 19 Feb 2024 14:56:58 +0100
Message-Id: <20240219135658.3662-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: ***
X-Spam-Score: 3.70
X-Spamd-Result: default: False [3.70 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 TO_DN_SOME(0.00)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[31.25%]
X-Spam-Flag: NO

In xen-9pfsd fill_data() va_end() needs to be called before returning.

Coverity Id CID 1592145

Fixes: bcec59cf7ff4 ("tools/xen-9pfsd: add 9pfs version request support")
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 tools/9pfsd/io.c | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/tools/9pfsd/io.c b/tools/9pfsd/io.c
index ebc4102713..adb887c7d9 100644
--- a/tools/9pfsd/io.c
+++ b/tools/9pfsd/io.c
@@ -445,7 +445,7 @@ static int fill_data(struct ring *ring, const char *fmt, ...)
             if ( !*f || array_sz )
                 fmt_err(fmt);
             if ( !chk_data(ring, data, sizeof(uint16_t)) )
-                return pars;
+                goto out;
             array_sz = get_unaligned((uint16_t *)data);
             data += sizeof(uint16_t);
             *(unsigned int *)par = array_sz;
@@ -455,10 +455,10 @@ static int fill_data(struct ring *ring, const char *fmt, ...)
 
         case 'b':
             if ( !chk_data(ring, data, sizeof(uint8_t)) )
-                return pars;
+                goto out;
             if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint8_t),
                                  data) )
-                return pars;
+                goto out;
             data += sizeof(uint8_t);
             break;
 
@@ -466,48 +466,48 @@ static int fill_data(struct ring *ring, const char *fmt, ...)
             if ( array_sz )
                 fmt_err(fmt);
             if ( !chk_data(ring, data, sizeof(uint32_t)) )
-                return pars;
+                goto out;
             len = get_unaligned((uint32_t *)data);
             data += sizeof(uint32_t);
             *(unsigned int *)par = len;
             par = va_arg(ap, void *);
             if ( !chk_data(ring, data, len) )
-                return pars;
+                goto out;
             memcpy(par, data, len);
             data += len;
             break;
 
         case 'L':
             if ( !chk_data(ring, data, sizeof(uint64_t)) )
-                return pars;
+                goto out;
             if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint64_t),
                                  data) )
-                return pars;
+                goto out;
             data += sizeof(uint64_t);
             break;
 
         case 'S':
             if ( !chk_data(ring, data, sizeof(uint16_t)) )
-                return pars;
+                goto out;
             len = get_unaligned((uint16_t *)data);
             data += sizeof(uint16_t);
             if ( !chk_data(ring, data, len) )
-                return pars;
+                goto out;
             str_off = add_string(ring, data, len);
             if ( str_off == ~0 )
-                return pars;
+                goto out;
             if ( !fill_data_elem(&par, array, &array_sz, sizeof(unsigned int),
                                  &str_off) )
-                return pars;
+                goto out;
             data += len;
             break;
 
         case 'U':
             if ( !chk_data(ring, data, sizeof(uint32_t)) )
-                return pars;
+                goto out;
             if ( !fill_data_elem(&par, array, &array_sz, sizeof(uint32_t),
                                  data) )
-                return pars;
+                goto out;
             data += sizeof(uint32_t);
             break;
 
@@ -520,6 +520,9 @@ static int fill_data(struct ring *ring, const char *fmt, ...)
         pars++;
     }
 
+ out:
+    va_end(ap);
+
     return pars;
 }
 
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:00:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:00:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682874.1062090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4CT-0007MQ-05; Mon, 19 Feb 2024 14:00:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682874.1062090; Mon, 19 Feb 2024 14:00:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4CS-0007MJ-Rs; Mon, 19 Feb 2024 14:00:52 +0000
Received: by outflank-mailman (input) for mailman id 682874;
 Mon, 19 Feb 2024 14:00:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pKiv=J4=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rc4CR-0007MB-A4
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:00:51 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 49bf3146-cf2f-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 15:00:50 +0100 (CET)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2d241ff062cso7077171fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 06:00:50 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 d23-20020a2eb057000000b002d11e348f47sm1124113ljl.87.2024.02.19.06.00.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 19 Feb 2024 06:00:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 49bf3146-cf2f-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708351250; x=1708956050; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=+HwvhZTDNvFwpG7SdNBRsmvr9FEYGM1sjbzBZRGaZQU=;
        b=J1fjqjCunzACF1t4kZd0v/elvo6wNWBMvF5+82/oQX4StrD2pbzlLiZ63AlwH/CDFn
         2dSDKl5/5GcmzMW7wzqQGE3Hk6rw0hhjn4WjY8qz/iHL1YIVZ37ILIDFHdq82fFXBjdt
         FYoQ/w++5DoMwShrVRqG9Xw3qi2IqMF6CoSEez3wOXTIL/twOPjB/zUANYK0+98Xihx/
         SNeyreOFFbnamTJ429BsjNIm6Yr00OfKnJyCkoUAMIbEtqE3R+nAUeYF0ZAxZ06FGAdy
         nFQNNMgm3BQVAdy5Y85pBQFd68LfR5ue8zVp+tjvqh5tjMuqdED0DY0vDfZjQ7owkwoi
         tZPA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708351250; x=1708956050;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+HwvhZTDNvFwpG7SdNBRsmvr9FEYGM1sjbzBZRGaZQU=;
        b=pOFjNxkB6kVHxmgMYUnpWo+tRFp7eQquAfrQT+RwdfpadZb3bzCF6ohHINph3G9vAy
         qkQmamweTFc0faSNp+ScBuyenqlhXb8lXRqdvAfMgEyS6nNHHdrXW33KjolBpw8AkIbr
         QB5xh3zPCdsldwFXzx2KhWK5DS/5oHvyq+WRq/SajnVVclhahjuwTYvP56Qk27S0EmcZ
         Zs//tb1gmPTcy1oz0k6u4OqlGp3I81483WdtgPSwYjkHixaAAAdC9nyPumtex10jeTml
         O2bGThgCim9vKJ9rmY/NfrIY2HCpxUGQGuUKtu6oAYj5ER0AUBnsDXwQIFV2zB3VB5CG
         dn9Q==
X-Forwarded-Encrypted: i=1; AJvYcCVx5E/lsXKS9VQ4c1igb4PY8UMVP0oXsZLOKC+fQdol+PLMkKRC7USOzZwyy9ZW0CgvMMTqupZx7/96upv79/fZuRRTuQs1KBMTX66Y7XA=
X-Gm-Message-State: AOJu0YzBdxpNdtZCaPP4PMCNHQmkjrzM4Th8o0OANoTjCVkutVky+Y33
	PqlW+IEHOpo7NuZqqBWKn2cANLfjvc4spu8NracE6UWxjuJxfjRU
X-Google-Smtp-Source: AGHT+IE3i0oJEA3C53RGkjiIqx3+FL1UAnstwariwJFi9eZrE7fUy2cSVxzwkTHR4nDkTE0LO+BqRA==
X-Received: by 2002:a2e:a454:0:b0:2d2:3c9c:f798 with SMTP id v20-20020a2ea454000000b002d23c9cf798mr1166765ljn.37.1708351249521;
        Mon, 19 Feb 2024 06:00:49 -0800 (PST)
Message-ID: <1ebdc064665b396d1978ac188594682f08601dbc.camel@gmail.com>
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org, Jan
 Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
Date: Mon, 19 Feb 2024 15:00:48 +0100
In-Reply-To: <491937fd-2459-416f-944b-0c63f7c12473@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
	 <491937fd-2459-416f-944b-0c63f7c12473@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Sun, 2024-02-18 at 19:00 +0000, Julien Grall wrote:
>=20
>=20
> On 05/02/2024 15:32, Oleksii Kurochko wrote:
> > The header was taken from Linux kernl 6.4.0-rc1.
> >=20
> > Addionally, were updated:
> > * add emulation of {cmp}xchg for 1/2 byte types
>=20
> This explaination is a little bit light. IIUC, you are implementing
> them=20
> using 32-bit atomic access. Is that correct? If so, please spell it
> out.
Sure, I'll update commit message.

>=20
> Also, I wonder whether it would be better to try to get rid of the
> 1/2=20
> bytes access. Do you know where they are used?
Right now, the issue is with test_and_clear_bool() which is used in
common/sched/core.c:840
[https://gitlab.com/xen-project/xen/-/blob/staging/xen/common/sched/core.c?=
ref_type=3Dheads#L840
]

I don't remember details, but in xen-devel chat someone told me that
grant table requires 1/2 bytes access.

>=20
> > * replace tabs with spaces
> Does this mean you are not planning to backport any Linux fixes?
If it will be any fixes for sure I'll back port them, but it looks like
this code is stable enough and not to many fixes will be done there, so
it shouldn't be hard to backport them and switch to spaces.

>=20
> > * replace __* varialbed with *__
>=20
> s/varialbed/variable/
>=20
> > * introduce generic version of xchg_* and cmpxchg_*.
> >=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > Changes in V4:
> > =C2=A0 - Code style fixes.
> > =C2=A0 - enforce in __xchg_*() has the same type for new and *ptr, also
> > "\n"
> > =C2=A0=C2=A0=C2=A0 was removed at the end of asm instruction.
> > =C2=A0 - dependency from
> > https://lore.kernel.org/xen-devel/cover.1706259490.git.federico.serafin=
i@bugseng.com/
> > =C2=A0 - switch from ASSERT_UNREACHABLE to STATIC_ASSERT_UNREACHABLE().
> > =C2=A0 - drop xchg32(ptr, x) and xchg64(ptr, x) as they aren't used.
> > =C2=A0 - drop cmpxcg{32,64}_{local} as they aren't used.
> > =C2=A0 - introduce generic version of xchg_* and cmpxchg_*.
> > =C2=A0 - update the commit message.
> > ---
> > Changes in V3:
> > =C2=A0 - update the commit message
> > =C2=A0 - add emulation of {cmp}xchg_... for 1 and 2 bytes types
> > ---
> > Changes in V2:
> > =C2=A0 - update the comment at the top of the header.
> > =C2=A0 - change xen/lib.h to xen/bug.h.
> > =C2=A0 - sort inclusion of headers properly.
> > ---
> > =C2=A0 xen/arch/riscv/include/asm/cmpxchg.h | 237
> > +++++++++++++++++++++++++++
> > =C2=A0 1 file changed, 237 insertions(+)
> > =C2=A0 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h
> >=20
> > diff --git a/xen/arch/riscv/include/asm/cmpxchg.h
> > b/xen/arch/riscv/include/asm/cmpxchg.h
> > new file mode 100644
> > index 0000000000..b751a50cbf
> > --- /dev/null
> > +++ b/xen/arch/riscv/include/asm/cmpxchg.h
> > @@ -0,0 +1,237 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/* Copyright (C) 2014 Regents of the University of California */
> > +
> > +#ifndef _ASM_RISCV_CMPXCHG_H
> > +#define _ASM_RISCV_CMPXCHG_H
> > +
> > +#include <xen/compiler.h>
> > +#include <xen/lib.h>
> > +
> > +#include <asm/fence.h>
> > +#include <asm/io.h>
> > +#include <asm/system.h>
> > +
> > +#define ALIGN_DOWN(addr, size)=C2=A0 ((addr) & (~((size) - 1)))
> > +
> > +#define __amoswap_generic(ptr, new, ret, sfx, release_barrier,
> > acquire_barrier) \
> > +({ \
> > +=C2=A0=C2=A0=C2=A0 asm volatile( \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 release_barrier \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 " amoswap" sfx " %0, %2, %1=
\n" \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 acquire_barrier \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "=3Dr" (ret), "+A" (*ptr)=
 \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "r" (new) \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "memory" ); \
> > +})
> > +
> > +#define emulate_xchg_1_2(ptr, new, ret, release_barrier,
> > acquire_barrier) \
> > +({ \
> > +=C2=A0=C2=A0=C2=A0 uint32_t *ptr_32b_aligned =3D (uint32_t *)ALIGN_DOW=
N((unsigned
> > long)ptr, 4); \
> > +=C2=A0=C2=A0=C2=A0 uint8_t mask_l =3D ((unsigned long)(ptr) & (0x8 - s=
izeof(*ptr)))
> > * BITS_PER_BYTE; \
> > +=C2=A0=C2=A0=C2=A0 uint8_t mask_size =3D sizeof(*ptr) * BITS_PER_BYTE;=
 \
> > +=C2=A0=C2=A0=C2=A0 uint8_t mask_h =3D mask_l + mask_size - 1; \
> > +=C2=A0=C2=A0=C2=A0 unsigned long mask =3D GENMASK(mask_h, mask_l); \
> > +=C2=A0=C2=A0=C2=A0 unsigned long new_ =3D (unsigned long)(new) << mask=
_l; \
> > +=C2=A0=C2=A0=C2=A0 unsigned long ret_; \
> > +=C2=A0=C2=A0=C2=A0 unsigned long rc; \
> > +=C2=A0=C2=A0=C2=A0 \
> > +=C2=A0=C2=A0=C2=A0 asm volatile( \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 release_barrier \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "0: lr.d %0, %2\n" \
>=20
> I was going to ask why this is lr.d rather than lr.w. But I see Jan=20
> already asked. I agree with him that it should probably be a lr.w and
> ...
>=20
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "=C2=A0=C2=A0 and=C2=A0 %1,=
 %0, %z4\n" \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "=C2=A0=C2=A0 or=C2=A0=C2=
=A0 %1, %1, %z3\n" \
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "=C2=A0=C2=A0 sc.d %1, %1, =
%2\n" \
>=20
> ... respectively sc.w. The same applies for cmpxchg.

I agree that it would be better, and my initial attempt was to handle
4-byte or 8-byte boundary crossing during 2-byte access:

0 1 2 3 4 5 6 7 8
X X X 1 1 X X X X

In this case, if I align address 3 to address 0 and then read 4 bytes
instead of 8 bytes, I will not process the byte at address 4. This was
the reason why I started to read 8 bytes.

I also acknowledge that there could be an issue in the following case:

X  4094 4095 4096
X    1   1    X
In this situation, when I read 8 bytes, there could be an issue where
the new page (which starts at 4096) will not be mapped. It seems
correct in this case to check that variable is within one page and read
4 bytes instead of 8.

One more thing I am uncertain about is if we change everything to read
4 bytes with 4-byte alignment, what should be done with the first case?
Should we panic? (I am not sure if this is an option.) Should we
perform the operation twice for addresses 0x0 and 0x4?

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:03:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:03:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682880.1062100 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4F1-0007zU-C1; Mon, 19 Feb 2024 14:03:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682880.1062100; Mon, 19 Feb 2024 14:03:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4F1-0007zN-9C; Mon, 19 Feb 2024 14:03:31 +0000
Received: by outflank-mailman (input) for mailman id 682880;
 Mon, 19 Feb 2024 14:03:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc4Ez-0007zH-NS
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:03:29 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a8197e83-cf2f-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 15:03:28 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-51294b40a52so2526409e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 06:03:28 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 cf27-20020a056512281b00b00511627e354asm918916lfb.247.2024.02.19.06.03.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 06:03:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8197e83-cf2f-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708351408; x=1708956208; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2YQFX/wensL8mgkZH6aTdQKBik1QHwjxBa0z6+1wegc=;
        b=DJB6AyAck7Eagqes2P0Bs5MYSBsTk+3KTkJ37sOK4pIqR4K71gqgFKxtcECPNbtk9f
         FqdxhHINbjDWMCjh286Nhku6eXMWwqqwbpJ1EraroKejD76JSBzerEj3jw3BShNJ8szu
         AqbY7MTrZGEZ2VDRpW3ZkoNKZjw9w8e6z6+n1822KAN7YKOdqQh3/0wFpezGY94nOoPW
         zp7/npauBIs9PiWQCKeYZqqarM1ZJ3sUVGfst8fjJKEVKxjrpl0DhIXMkfqeCXzwVy3C
         gpSW9At++nUjwAlrywWG2MiV8s8clMs+DocW41qlKTu1yIEaqcSZdLyyFm7h/rJWGwGN
         4HSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708351408; x=1708956208;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2YQFX/wensL8mgkZH6aTdQKBik1QHwjxBa0z6+1wegc=;
        b=AhrgqrRxdjRHfgDZP/dOT1bZ1XneP21g5tA63WNw4zyrAtBSPaG5BTE2jYZrwELsXi
         WoOOoGwMW/nQ/Qh8ccTX918SoPWU0kqvAZrJRl6iUplupNd/u1Xzw3BMgeFSARaUh4WY
         7cJyBxDeIJ4ceiooJy9+KL/qhVeCuxPD2n8NLrRAkiz9X0U1evjXxEvCjD5ffKddvGQe
         LCDBCt6ptKDRtbNH9/YFbYE9CVpa9IwWaD5HH+IODdWKIk1VRMbrScW62vtCbG+JHYwv
         WIFH0nCmrYRYBEBa1bgNu/14NEc5Uqu3xRTIL3IKL/Ecqzk9LKRnxAmVtilB/rvQaSR9
         mq1g==
X-Forwarded-Encrypted: i=1; AJvYcCWMR7VolQw8O9GTB2TxoL4H6P8k/vj5t0qSHslf6flYy3SlWXG9MPUHJshYiRvLY91cC2tQ2i9++kmqhCvwJX7IvIJCtsPSZX3KAdQEG4g=
X-Gm-Message-State: AOJu0YyFQ5pSK/yO5j+HRaQw+t1A7JGEmM1QcJPqVQOc5JiWQEKHhdPm
	MOuVJl9Du8LLcaP7v4rghlYmQEEcQ0/eTf9KcIrxDOUIWQ7JzPpJu9CLHxHFZw==
X-Google-Smtp-Source: AGHT+IGeXJZeH4j1GUgV4gIvY37NH4BKLPw4dZ8g7aGoZsm6zp18qnqol7abtPSdudsNDkB8p1Q9yQ==
X-Received: by 2002:a05:6512:68c:b0:512:bf10:832d with SMTP id t12-20020a056512068c00b00512bf10832dmr51138lfe.17.1708351408209;
        Mon, 19 Feb 2024 06:03:28 -0800 (PST)
Message-ID: <ebc81051-765a-424a-9b30-97d6755e84c3@suse.com>
Date: Mon, 19 Feb 2024 15:03:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/6] svm: Improve type of cpu_has_svm_feature
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-3-george.dunlap@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240206012051.3564035-3-george.dunlap@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 02:20, George Dunlap wrote:
> The "effective type" of the cpu_has_svm_feature macro is effectively
> an unsigned log with one bit set (or not); at least one place someone
> felt compelled to do a !! to make sure that they got a boolean out of
> it.
> 
> Ideally the whole of this would be folded into the cpufeature.h
> infrastructure.  But for now, duplicate the more type-safe static
> inlines in that file, and remove the !!.
> 
> Signed-off-by: George Dunlap <george.dunlap@cloud.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
albeit preferably with ...

> --- a/xen/arch/x86/include/asm/hvm/svm/svm.h
> +++ b/xen/arch/x86/include/asm/hvm/svm/svm.h
> @@ -38,7 +38,10 @@ extern u32 svm_feature_flags;
>  #define SVM_FEATURE_SSS           19 /* NPT Supervisor Shadow Stacks */
>  #define SVM_FEATURE_SPEC_CTRL     20 /* MSR_SPEC_CTRL virtualisation */
>  
> -#define cpu_has_svm_feature(f) (svm_feature_flags & (1u << (f)))
> +static inline bool cpu_has_svm_feature(unsigned int feat)
> +{
> +    return svm_feature_flags & (1u << (feat));

... the inner pair of parentheses dropped.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:12:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:12:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682889.1062109 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4O4-0001Rj-8I; Mon, 19 Feb 2024 14:12:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682889.1062109; Mon, 19 Feb 2024 14:12:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4O4-0001Rc-5n; Mon, 19 Feb 2024 14:12:52 +0000
Received: by outflank-mailman (input) for mailman id 682889;
 Mon, 19 Feb 2024 14:12:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc4O2-0001RW-CI
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:12:50 +0000
Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com
 [2a00:1450:4864:20::32e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f61ddcb1-cf30-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 15:12:49 +0100 (CET)
Received: by mail-wm1-x32e.google.com with SMTP id
 5b1f17b1804b1-412698ac6f9so3355235e9.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 06:12:49 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 jv14-20020a05600c570e00b0040fe3147babsm11607691wmb.0.2024.02.19.06.12.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 06:12:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f61ddcb1-cf30-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708351968; x=1708956768; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lpM1FgOFVuQs6HeBgyssEQKkmM0PMINwI3V3SPSgV+Y=;
        b=WLt+zxivZcqfrTFsVRnnqJGzrzXu1Ew+mTCFvIDOXOXSjw0ZcTatjPKLUPEElQCW7S
         mnNCv/jNNgVuS+Wy0Knuq5w0UAlEr38G4nDlrrCKW86Pr9KPSpWwjH/W6BaifCDrzhpj
         ps0C7WpLZFRk3U8b9flMexY5BXBFUkBNw5j4hk9TrfWPtpAeBQcv7KBy1Q7aHYXZXt0q
         qLHbjGx9OIE8PSagxihbMY66m0zo5F3QEybpso5H7QhSB+tpyssEoVN3Vs2Mdjd5jUWN
         rrw0E9y4yCfyYo9xci8LFfzdpsmJT+z9nLAgoXPG4ReZxe4OEyLN8a8/bTrgDRPC6DJZ
         PieQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708351968; x=1708956768;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=lpM1FgOFVuQs6HeBgyssEQKkmM0PMINwI3V3SPSgV+Y=;
        b=ISog+riLGcnh43DPTctRkvJPtyhGiY5COrn0uZ4WkkJ6U4cG+eVR1ukjE8ogyk119w
         GEjVIfHXWHjJRrRI2p9xDmY5Kynt1ihE+BquVa0gphOQVJ+Qd72tE5EyiKBHYg39OyiW
         /J5iEGDhOoofbUF+wsY69LL8/C7GohguB/zfndPEtB+Q2DrwyhD7pAoNWHUUSiRzrtBI
         g+0BI6Fq4mT3jZ9nv1mheQen7Zy8ILWe5eUprQHXOFVKH4k+yaxw24N5mzwH6D1hHw66
         uAkIP9hgnvMgFAO7uEKZVpxae7+6TgetC2S2+BblEuOCnf1IWrqai07My8YnCWgLRXHi
         +CWQ==
X-Forwarded-Encrypted: i=1; AJvYcCXoHCD9afIzybPqyzQakLht2gAZMK5PDxjr4Kyhw0oG0Gh0v9zzVF2m0sE2BwH+ef5utNadrW9IuRVEyp7pO+zyCx8kTF08W9h0xpkqQAM=
X-Gm-Message-State: AOJu0YzjxPnxGN3h1qtsr7o4WspbMMJ98Q/3LEUrNBXcrv/dObVOdI8/
	e5Jp1EGuKfdyV+5YJnkWcN0fvEPUIgB93zVPP8BUj51qTrHv39+GhNFIE1onjg==
X-Google-Smtp-Source: AGHT+IFbNQTyR9YmTnaUvioudSePL0NwHbITvwkvmUTjQBjN9Im1dBq7+82ysWZNhtOuHrAia5pQgA==
X-Received: by 2002:a05:600c:1c0a:b0:411:dd82:a23c with SMTP id j10-20020a05600c1c0a00b00411dd82a23cmr10577067wms.0.1708351968559;
        Mon, 19 Feb 2024 06:12:48 -0800 (PST)
Message-ID: <73b4654a-cdf2-48dc-b802-a7890c3fe426@suse.com>
Date: Mon, 19 Feb 2024 15:12:47 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
 <491937fd-2459-416f-944b-0c63f7c12473@xen.org>
 <1ebdc064665b396d1978ac188594682f08601dbc.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <1ebdc064665b396d1978ac188594682f08601dbc.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 19.02.2024 15:00, Oleksii wrote:
> On Sun, 2024-02-18 at 19:00 +0000, Julien Grall wrote:
>> On 05/02/2024 15:32, Oleksii Kurochko wrote:
>>> --- /dev/null
>>> +++ b/xen/arch/riscv/include/asm/cmpxchg.h
>>> @@ -0,0 +1,237 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +/* Copyright (C) 2014 Regents of the University of California */
>>> +
>>> +#ifndef _ASM_RISCV_CMPXCHG_H
>>> +#define _ASM_RISCV_CMPXCHG_H
>>> +
>>> +#include <xen/compiler.h>
>>> +#include <xen/lib.h>
>>> +
>>> +#include <asm/fence.h>
>>> +#include <asm/io.h>
>>> +#include <asm/system.h>
>>> +
>>> +#define ALIGN_DOWN(addr, size)  ((addr) & (~((size) - 1)))
>>> +
>>> +#define __amoswap_generic(ptr, new, ret, sfx, release_barrier,
>>> acquire_barrier) \
>>> +({ \
>>> +    asm volatile( \
>>> +        release_barrier \
>>> +        " amoswap" sfx " %0, %2, %1\n" \
>>> +        acquire_barrier \
>>> +        : "=r" (ret), "+A" (*ptr) \
>>> +        : "r" (new) \
>>> +        : "memory" ); \
>>> +})
>>> +
>>> +#define emulate_xchg_1_2(ptr, new, ret, release_barrier,
>>> acquire_barrier) \
>>> +({ \
>>> +    uint32_t *ptr_32b_aligned = (uint32_t *)ALIGN_DOWN((unsigned
>>> long)ptr, 4); \
>>> +    uint8_t mask_l = ((unsigned long)(ptr) & (0x8 - sizeof(*ptr)))
>>> * BITS_PER_BYTE; \
>>> +    uint8_t mask_size = sizeof(*ptr) * BITS_PER_BYTE; \
>>> +    uint8_t mask_h = mask_l + mask_size - 1; \
>>> +    unsigned long mask = GENMASK(mask_h, mask_l); \
>>> +    unsigned long new_ = (unsigned long)(new) << mask_l; \
>>> +    unsigned long ret_; \
>>> +    unsigned long rc; \
>>> +    \
>>> +    asm volatile( \
>>> +        release_barrier \
>>> +        "0: lr.d %0, %2\n" \
>>
>> I was going to ask why this is lr.d rather than lr.w. But I see Jan 
>> already asked. I agree with him that it should probably be a lr.w and
>> ...
>>
>>> +        "   and  %1, %0, %z4\n" \
>>> +        "   or   %1, %1, %z3\n" \
>>> +        "   sc.d %1, %1, %2\n" \
>>
>> ... respectively sc.w. The same applies for cmpxchg.
> 
> I agree that it would be better, and my initial attempt was to handle
> 4-byte or 8-byte boundary crossing during 2-byte access:
> 
> 0 1 2 3 4 5 6 7 8
> X X X 1 1 X X X X
> 
> In this case, if I align address 3 to address 0 and then read 4 bytes
> instead of 8 bytes, I will not process the byte at address 4. This was
> the reason why I started to read 8 bytes.
> 
> I also acknowledge that there could be an issue in the following case:
> 
> X  4094 4095 4096
> X    1   1    X
> In this situation, when I read 8 bytes, there could be an issue where
> the new page (which starts at 4096) will not be mapped. It seems
> correct in this case to check that variable is within one page and read
> 4 bytes instead of 8.
> 
> One more thing I am uncertain about is if we change everything to read
> 4 bytes with 4-byte alignment, what should be done with the first case?
> Should we panic? (I am not sure if this is an option.)

Counter question (raised elsewhere already): What if a 4-byte access
crosses a word / cache line / page boundary? Ideally exactly the
same would happen for a 2-byte access crossing a respective boundary.
(Which you can achieve relatively easily by masking off only address
bit 1, keeping address bit 0 unaltered.)

> Should we
> perform the operation twice for addresses 0x0 and 0x4?

That wouldn't be atomic then.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:14:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:14:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682893.1062120 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4PZ-0001zL-Ik; Mon, 19 Feb 2024 14:14:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682893.1062120; Mon, 19 Feb 2024 14:14:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4PZ-0001zE-G9; Mon, 19 Feb 2024 14:14:25 +0000
Received: by outflank-mailman (input) for mailman id 682893;
 Mon, 19 Feb 2024 14:14:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=GzZz=J4=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rc4PY-0001z6-5y
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:14:24 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:240a::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2caf5a4a-cf31-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 15:14:21 +0100 (CET)
Received: from SA0PR11CA0170.namprd11.prod.outlook.com (2603:10b6:806:1bb::25)
 by SJ2PR12MB9005.namprd12.prod.outlook.com (2603:10b6:a03:53d::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.20; Mon, 19 Feb
 2024 14:14:15 +0000
Received: from SA2PEPF000015C9.namprd03.prod.outlook.com
 (2603:10b6:806:1bb:cafe::69) by SA0PR11CA0170.outlook.office365.com
 (2603:10b6:806:1bb::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.39 via Frontend
 Transport; Mon, 19 Feb 2024 14:14:15 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF000015C9.mail.protection.outlook.com (10.167.241.199) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 19 Feb 2024 14:14:15 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 19 Feb
 2024 08:14:14 -0600
Received: from [172.30.63.250] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 19 Feb 2024 08:14:13 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2caf5a4a-cf31-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gSzhBZFx/pPqKpy+H7xCJyDmzFgpnoBBJI4rs8qPmnZY16KzwvIadoZVyyhiljxowgdNERJzFnmpdkLw3LV8tqgoSPD2Seu73cFxTZ0AS7e4ZWqH6WnV6VWgP+8ukXLmaynIKQBUDQgs6WBbowVdzfVmZcxcJr5yBZiiUGY7YFxU3HgmPJ/6wo7/8xn2iGgJvXjKznU+i9XGCrPNbm9C8XsTV7E8wenKnlacX7ypbPfvJrA+VINrw7yJA8MakHpJ1Zuy/rmGbexIqPKdNHAj+yzKdGeUQBoqte/0sSTgcKR3k3azAOtyS7oJZMysMyNAs0wUq++RHcnPXhqzqeM6pw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Cj7gA1sm3uWjyRxLyFk5BtQpLh30GWilhCVGfPP9bQk=;
 b=Cg6MjuAs1143ED73DNQ6axZUDf9uvS9R8/533Cq8bjF2Q8tsoo/WImgQiU7hKcn5VwagDk3YrbZ8TjJJPZ1XHbffaT5vDyX3ZzeRpAQfWEJPFqoe8ElEBWarEkjyXfKFiDBiMrVDguHkNlvSzfkaYJFZm0MasseQXKm7AimrsT4hB9UBYWulja4bVfWS+4AxT9k98C5v7PJWtOzqMutn27M1GzllEvMgtSEyR9fpL2vQXHMGu/L5KqF0MK77LPv3XVqlI49gqgXVYhgm1CodXJqjC5SbwtPi8ALspmM1cN+wS8hQwjQadc9O+n3oA2QMhRDz29NE8J/sH56CwAYitA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Cj7gA1sm3uWjyRxLyFk5BtQpLh30GWilhCVGfPP9bQk=;
 b=3opHjx3BdvOd0UNZTsW77b4SHu5tCWWamzt0gGYJNOrksdkokhXZOwe4RqS96mXaPR8GcixB8ZY952VUzsZRWTVZQPP94Lj+rfVM4ygIFGMgIB8s+WPJHHPj4EngHPqg9IjE09osLIWajZYxeCX+KocEy4mjNDN2M4q/30CCNZc=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <6587ee51-5943-4a81-814b-917f7ec71c01@amd.com>
Date: Mon, 19 Feb 2024 09:14:12 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13.2 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
To: Jan Beulich <jbeulich@suse.com>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian
	<kevin.tian@intel.com>, Paul Durrant <paul@xen.org>, Volodymyr Babchuk
	<volodymyr_babchuk@epam.com>, <xen-devel@lists.xenproject.org>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <20240219114750.151730-1-stewart.hildebrand@amd.com>
 <8e807612-fd14-4d01-b200-1deb583afa22@suse.com>
 <68bb972c-5e45-4706-a06f-0dcd65c40581@amd.com>
 <39e8845b-8d41-40e5-8e7a-49e734012d78@suse.com>
Content-Language: en-US
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <39e8845b-8d41-40e5-8e7a-49e734012d78@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF000015C9:EE_|SJ2PR12MB9005:EE_
X-MS-Office365-Filtering-Correlation-Id: 6e43acaa-612b-414a-6092-08dc31550de6
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	D5KV2ol3TSTRn65If5U6yTJbnLls9usiQHlC9/+xj4eq4sPxE3Wi4o1RtW89lgRl8POB8Bof3Cod3iHn8c2ks36VB95gc3Ye031SdmIdn3rqKBTp4AxdnNZib+ZY8UFLVzpqXrSxJE9XzK8ERMc8FcWZKZa8v0qmLVMgkfvozwzkrammzVQ4JzxOUhY3ecCglYCBInYbppb55orm/goMQtPH/SGKRoh8Aqx0jFj0e1WoHaIFlDjwhZWNn2Q3y1jJ+hbuhfNWW95Eq6cosB9BB7cIYBpG8VFRLlzzhJGagzI52oX3Weod+h2T7WW/0SaQpYV0B1bceGiIn3GpAhOt6b10gYDS5xUUi2h6KOeIPe5y8IWURIOfCOArUB7rEPueQzzDyUSCXDuot6XTJFTsAFVo4vB5N2EZPN06IuOrlo72N0ZgyyVfY2SIE30qpwsZihCe/W03zBPSbG4NWp2BlW7lNVsEZyWLelw4NWej4KApJTwSWdztjW9wwveYpgcGg3BBvqBAtOQVBaezAiR8PqGooAwEboshrJm0TAg/oBKeL5g9UgIa39Mlv7KYJGIHc6UUbxXi3REk62HKdlzg+lF2eZ6T+A1jIlXv2ZXQBUnrNGQEQUQgBTGIW4qFLe3urr5kUtCtTZVnivo18p2fcA==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(40470700004)(46966006);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Feb 2024 14:14:15.4325
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6e43acaa-612b-414a-6092-08dc31550de6
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF000015C9.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB9005

On 2/19/24 08:12, Jan Beulich wrote:
> On 19.02.2024 13:47, Stewart Hildebrand wrote:
>> On 2/19/24 07:10, Jan Beulich wrote:
>>> On 19.02.2024 12:47, Stewart Hildebrand wrote:
>>>> @@ -895,6 +891,15 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
>>>>  {
>>>>      unsigned int i;
>>>>  
>>>> +    /*
>>>> +     * Assert that d->pdev_list doesn't change. ASSERT_PDEV_LIST_IS_READ_LOCKED
>>>> +     * is not suitable here because it may allow either pcidevs_lock() or
>>>> +     * d->pci_lock to be held, but here we rely on d->pci_lock being held, not
>>>> +     * pcidevs_lock().
>>>> +     */
>>>> +    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
>>>> +    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));
>>>
>>> There's no "d" in sight here, so it's a little odd that "d" is being talked
>>> about. But I guess people can infer what's meant without too much trouble.
>>
>> I can s/d->pci_lock/msix->pdev->domain->pci_lock/ for the next rev.
> 
> Or simply drop the d-s? That would be better for readability's sake,
> I think.

OK

>>>> @@ -313,17 +316,36 @@ void vpci_dump_msi(void)
>>>>                  {
>>>>                      /*
>>>>                       * On error vpci_msix_arch_print will always return without
>>>> -                     * holding the lock.
>>>> +                     * holding the locks.
>>>>                       */
>>>>                      printk("unable to print all MSI-X entries: %d\n", rc);
>>>> -                    process_pending_softirqs();
>>>> -                    continue;
>>>> +                    goto pdev_done;
>>>>                  }
>>>>              }
>>>>  
>>>> +            /*
>>>> +             * Unlock locks to process pending softirqs. This is
>>>> +             * potentially unsafe, as d->pdev_list can be changed in
>>>> +             * meantime.
>>>> +             */
>>>>              spin_unlock(&pdev->vpci->lock);
>>>> +            read_unlock(&d->pci_lock);
>>>> +        pdev_done:
>>>>              process_pending_softirqs();
>>>> +            if ( !read_trylock(&d->pci_lock) )
>>>> +            {
>>>> +                printk("unable to access other devices for the domain\n");
>>>> +                goto domain_done;
>>>> +            }
>>>>          }
>>>> +        read_unlock(&d->pci_lock);
>>>> +    domain_done:
>>>> +        /*
>>>> +         * We need this label at the end of the loop, but some
>>>> +         * compilers might not be happy about label at the end of the
>>>> +         * compound statement so we adding an empty statement here.
>>>> +         */
>>>> +        ;
>>>
>>> As to "some compilers": Are there any which accept a label not followed
>>> by a statement? Depending on the answer, this comment may be viewed as
>>> superfluous. Or else I'd ask about wording: Besides a grammar issue I
>>> also don't view it as appropriate that a comment talks about "adding"
>>> something when its adjacent code that is meant. That something is there
>>> when the comment is there, hence respective wording should imo be used.
>>
>> It seems like hit or miss whether gcc would accept it or not (prior
>> discussion at [1]). I agree the comment is rather lengthy for what it's
>> trying to convey. I'd be happy to either remove the comment or reduce
>> it to:
>>
>>     domain_done:
>>         ; /* Empty statement to make some compilers happy */
>>
>> [1] https://lore.kernel.org/xen-devel/98b8c131-b0b9-f46c-5f46-c2136f2e3b4e@amd.com/
> 
> This earlier discussion only proves that there is at least one compiler
> objecting. There's no proof there that any compiler exists which, as a
> language extension, actually permits such syntax. Yet if the comment
> was purely about normal language syntax, then imo it should be zapped
> altogether, not just be shrunk.

I'll zap it


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:20:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:20:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682899.1062129 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4VN-0003o3-7V; Mon, 19 Feb 2024 14:20:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682899.1062129; Mon, 19 Feb 2024 14:20:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4VN-0003nw-4h; Mon, 19 Feb 2024 14:20:25 +0000
Received: by outflank-mailman (input) for mailman id 682899;
 Mon, 19 Feb 2024 14:20:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rc4VM-0003nm-5C; Mon, 19 Feb 2024 14:20:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rc4VL-00086s-Sm; Mon, 19 Feb 2024 14:20:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rc4VL-0008Nf-DK; Mon, 19 Feb 2024 14:20:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rc4VL-0004R2-Cq; Mon, 19 Feb 2024 14:20:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dJNEOC2oRlPglRcsWmo9cm7JNAbWJiW1kH9eVs3PPzs=; b=Cj5RZBYOoQAuy7zDcqGqc3jgwg
	k/OjfdtmyzVDGi4Hhp4HeCnVHA6qDogwIbHp2Vp/lNzvsJIcS/v1zn+KpGAdWdHUB/hR0MuwmG6Oq
	akBwbEjqRxVP5jP9eROs8foqkzM9yX4i0rhqRZ8gFZwywEeOMZUAiLPYRzBwaIkDTF2E=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184700-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184700: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-freebsd11-amd64:guest-localmigrate/x10:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=b401b621758e46812da61fa58a67c3fd8d91de0d
X-Osstest-Versions-That:
    linux=6c160f16be5df1f66f6afe186c961ad446d7f94b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 19 Feb 2024 14:20:23 +0000

flight 184700 linux-linus real [real]
flight 184702 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184700/
http://logs.test-lab.xenproject.org/osstest/logs/184702/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-freebsd11-amd64 19 guest-localmigrate/x10 fail pass in 184702-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184697
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184697
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184697
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184697
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184697
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184697
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184697
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184697
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                b401b621758e46812da61fa58a67c3fd8d91de0d
baseline version:
 linux                6c160f16be5df1f66f6afe186c961ad446d7f94b

Last test of basis   184697  2024-02-18 18:43:56 Z    0 days
Testing same since   184700  2024-02-19 03:44:55 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Linus Torvalds <torvalds@linux-foundation.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             fail    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   6c160f16be5d..b401b621758e  b401b621758e46812da61fa58a67c3fd8d91de0d -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:25:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:25:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682907.1062140 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4aR-0004YR-0D; Mon, 19 Feb 2024 14:25:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682907.1062140; Mon, 19 Feb 2024 14:25:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4aQ-0004YK-So; Mon, 19 Feb 2024 14:25:38 +0000
Received: by outflank-mailman (input) for mailman id 682907;
 Mon, 19 Feb 2024 14:25:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rc4aQ-0004YE-5t
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:25:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc4aP-0008C3-Gc; Mon, 19 Feb 2024 14:25:37 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc4aP-0004FN-4J; Mon, 19 Feb 2024 14:25:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=iLCEvlZKhvO/Kr/t8YTvRFm5jGo4UX7cjKzXwoRCQhQ=; b=NxCkBiCAkZ7xyE1kbHKxLCGMYP
	urNrorGBAs98W2ErhuJ3/Mm0AVKBl+WrRVRfWrQ9vBPIIngMqoGpz80BF00v9CHzb8dcGPrpOgpAr
	19yNrWunF3XArb4+ElGRHM+f6z4W4u8UpCfZ4WTK2psqHiLr1Cv6zFCbyxbuJjti8R/g=;
Message-ID: <91e90054-3e07-4aa5-a3bc-a95d7c4800cc@xen.org>
Date: Mon, 19 Feb 2024 14:25:34 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Content-Language: en-GB
To: Oleksii <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org,
 Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
 <491937fd-2459-416f-944b-0c63f7c12473@xen.org>
 <1ebdc064665b396d1978ac188594682f08601dbc.camel@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <1ebdc064665b396d1978ac188594682f08601dbc.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Oleksii,

On 19/02/2024 14:00, Oleksii wrote:
> On Sun, 2024-02-18 at 19:00 +0000, Julien Grall wrote:
>>
>>
>> On 05/02/2024 15:32, Oleksii Kurochko wrote:
>>> The header was taken from Linux kernl 6.4.0-rc1.
>>>
>>> Addionally, were updated:
>>> * add emulation of {cmp}xchg for 1/2 byte types
>>
>> This explaination is a little bit light. IIUC, you are implementing
>> them
>> using 32-bit atomic access. Is that correct? If so, please spell it
>> out.
> Sure, I'll update commit message.
> 
>>
>> Also, I wonder whether it would be better to try to get rid of the
>> 1/2
>> bytes access. Do you know where they are used?
> Right now, the issue is with test_and_clear_bool() which is used in
> common/sched/core.c:840
> [https://gitlab.com/xen-project/xen/-/blob/staging/xen/common/sched/core.c?ref_type=heads#L840
> ]
> 
> I don't remember details, but in xen-devel chat someone told me that
> grant table requires 1/2 bytes access.

Ok :/. This would be part of the ABI then and therefore can't be easily 
changed.

> 
>>
>>> * replace tabs with spaces
>> Does this mean you are not planning to backport any Linux fixes?
> If it will be any fixes for sure I'll back port them, but it looks like
> this code is stable enough and not to many fixes will be done there, so
> it shouldn't be hard to backport them and switch to spaces.

Fair enough.

>>> +        "   and  %1, %0, %z4\n" \
>>> +        "   or   %1, %1, %z3\n" \
>>> +        "   sc.d %1, %1, %2\n" \
>>
>> ... respectively sc.w. The same applies for cmpxchg.
> 
> I agree that it would be better, and my initial attempt was to handle
> 4-byte or 8-byte boundary crossing during 2-byte access:
> 
> 0 1 2 3 4 5 6 7 8
> X X X 1 1 X X X X
> 
> In this case, if I align address 3 to address 0 and then read 4 bytes
> instead of 8 bytes, I will not process the byte at address 4. This was
> the reason why I started to read 8 bytes.

At least on Arm, the architecture doesn't support atomic operations if 
the access is not aligned to its size (this will send a data abort). On 
some architecture, this is supported but potentially very slow.

So all the common code should already use properly aligned address. 
Therefore, I don't really see the reason to add support for unaligned 
access.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:29:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:29:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682912.1062150 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4eG-0005DD-HA; Mon, 19 Feb 2024 14:29:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682912.1062150; Mon, 19 Feb 2024 14:29:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4eG-0005D6-Cl; Mon, 19 Feb 2024 14:29:36 +0000
Received: by outflank-mailman (input) for mailman id 682912;
 Mon, 19 Feb 2024 14:29:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pKiv=J4=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rc4eE-0005Cu-U6
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:29:34 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4c56b851-cf33-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 15:29:32 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a3e8be8a7ddso140204466b.2
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 06:29:32 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 r12-20020a170906704c00b00a3d932d8fa7sm3015143ejj.183.2024.02.19.06.29.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 19 Feb 2024 06:29:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4c56b851-cf33-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708352972; x=1708957772; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=eQCxeGeYb7l53I7iiUj0zXl0o166/IhDB/hbS2xv7h0=;
        b=QZNN6Pd8zR0NgtsNHeCuSnWCKrLvmAQdnngA/Wxi3KX+K1nAXYbC8cVdU1al1PLtO5
         OJe9QTSAwjNbd0NuKJ2IUiWodrPqH65TQzeAZvjWsaxBLkynf3srTw4/m4hCAJkPT8Wn
         wHeRfTD8wV5Le4J19X+e4HpX1jjJA+kBZiARXoV9TJNashp3OyTqbbjHRJxxYwwpJaRF
         CEKKKOmXKiza7zwxlRqcImTH1Hjzl9cI6AXYi+pqSKbODWuYXuhEA0XBWP3Q6XtfuHFa
         35zxbd0R7w3389bzTXsA3JOn2oJPBb9J4xTCxZNjFRm24rfeb8j74vsj1IvIy4siqxoK
         zw5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708352972; x=1708957772;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=eQCxeGeYb7l53I7iiUj0zXl0o166/IhDB/hbS2xv7h0=;
        b=F2tHQo15dEBPrrmHf8Njqa+h+9V98MxnxlprmPpj2qdTlFbjzNoBk63IHKppJLJl/V
         3WnoBckku2TAAui3LOUnR/EFhGdnFUzEX4ZyUoqcjGH4EJFFzlDKRB8fyA6oh+RpMXoB
         apK5m2QodDLSMwBpXykDSAJtKBd9eELWWsEdmQa6CKjOOUzd++EJnQtfSbt2mtG8ISps
         NhmHxfBuJyNPB51zPipP/RDf2gi0UE+SanRLRIOTAROnybMFtmAF7xzITtlyUJssMrdq
         JL7I/LnDetUv6jmXhM7FT1oLDq37s38O5IhUZhsXKX45qPKWQcp+cfyUQgpgeRwNWsRq
         RFTw==
X-Forwarded-Encrypted: i=1; AJvYcCUNbrqX6x0jvY1TeFZjCzECidZREer+Jfuw+dxeqU9o9K1Z0NpXcAkhZxBX3xlUf8zm5ZMoVCyBb99tfZtOjj6oxLu7pbWZL9o0esvLt2E=
X-Gm-Message-State: AOJu0YxwfgzgYGwjMpNTpHih9FXsDGLCADxiXbEed0hljLlHWkOC/WsZ
	+qcmh9qjp2kMc0n+QaR/cLl/e118X1WKdpMIC+JxjB/JMFpR1c/1
X-Google-Smtp-Source: AGHT+IHnd8rqsx06qnbxRJlnN+meMq7aQYrrVwuwNJ3hjnkelnM7bX+eMsvawRbW5odL03sJHTUrmw==
X-Received: by 2002:a17:906:e246:b0:a3e:69eb:45dc with SMTP id gq6-20020a170906e24600b00a3e69eb45dcmr2685436ejb.21.1708352971797;
        Mon, 19 Feb 2024 06:29:31 -0800 (PST)
Message-ID: <1c53b52ee39161a8f59209d28af69fe997479dbc.camel@gmail.com>
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Mon, 19 Feb 2024 15:29:30 +0100
In-Reply-To: <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
	 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
	 <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
	 <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-19 at 12:22 +0100, Jan Beulich wrote:
> On 15.02.2024 14:41, Oleksii wrote:
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "=3Dr" (ret), "+A" (*=
ptr) \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "r" (new) \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "memory" ); \
> > > > +})
> > > > +
> > > > +#define emulate_xchg_1_2(ptr, new, ret, release_barrier,
> > > > acquire_barrier) \
> > > > +({ \
> > > > +=C2=A0=C2=A0=C2=A0 uint32_t *ptr_32b_aligned =3D (uint32_t
> > > > *)ALIGN_DOWN((unsigned
> > > > long)ptr, 4); \
> > >=20
> > > You now appear to assume that this macro is only used with inputs
> > > not
> > > crossing word boundaries. That's okay as long as suitably
> > > guaranteed
> > > at the use sites, but imo wants saying in a comment.
> > >=20
> > > > +=C2=A0=C2=A0=C2=A0 uint8_t mask_l =3D ((unsigned long)(ptr) & (0x8=
 -
> > > > sizeof(*ptr)))
> > > > * BITS_PER_BYTE; \
> > >=20
> > > Why 0x8 (i.e. spanning 64 bits), not 4 (matching the uint32_t use
> > > above)?
> > The idea to read 8 bytes was to deal with crossing word boundary.
> > So if
> > our address is 0x3 and we have to xchg() 2 bytes, what will cross 4
> > byte boundary. Instead we align add 0x3, so it will become 0x0 and
> > then
> > just always work with 8 bytes.
>=20
> Then what if my 2-byte access crosses a dword boundary? A cache line
> one? A page one?
Everything looks okay to me, except in the case of a page boundary.

In the scenario of a dword boundary:

0 1 2 3 4 5 6 7 8 9 ...
X X X X X X X 1 1 X

Assuming a variable starts at address 7, 4-byte alignment will be
enforced, and 8 bytes will be processed starting from address 4.

Concerning a cache line, it should still work, with potential
performance issues arising only if a part of the variable is cached
while another part is not.

Regarding page crossing, I acknowledge that it could be problematic if
the variable is entirely located at the end of a page, as there is no
guarantee that the next page exists. In this case, it would be
preferable to consistently read 4 bytes with 4-byte alignment:

X 4094 4095 4096?
X  1    1    ?

However, if the variable spans two pages, proper page mapping should be
ensured.

It appears sensible to reconsider the macros and implement 4-byte
alignment and 4-byte access, but then this is not clear how better to
deal with first case ( dword boundary ). Panic ? or use the macros
twice for address 4, and address 8?

>=20
> > > > +=C2=A0=C2=A0=C2=A0 unsigned long new_ =3D (unsigned long)(new) << =
mask_l; \
> > > > +=C2=A0=C2=A0=C2=A0 unsigned long ret_; \
> > > > +=C2=A0=C2=A0=C2=A0 unsigned long rc; \
> > >=20
> > > Similarly, why unsigned long here?
> > sizeof(unsigned long) is 8 bytes and it was chosen as we are
> > working
> > with lc/sc.d which are working with 8 bytes.
> >=20
> > >=20
> > > I also wonder about the mix of underscore suffixed (or not)
> > > variable
> > > names here.
> > If the question about ret_, then the same as before size of ret
> > argument of the macros will be 1 or 2, but {lc/sc}.d expected to
> > work
> > with 8 bytes.
>=20
> Then what's the uint32_t * about?
Agree, then it should be also unsigned long.
> >=20

> > > > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) new__ =3D (new); \
> > > > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) ret__; \
> > > > +=C2=A0=C2=A0=C2=A0 switch (size) \
> > > > +=C2=A0=C2=A0=C2=A0 { \
> > > > +=C2=A0=C2=A0=C2=A0 case 1: \
> > > > +=C2=A0=C2=A0=C2=A0 case 2: \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 emulate_xchg_1_2(ptr__,=
 new__, ret__, release_barrier,
> > > > acquire_barrier); \
> > >=20
> > > ... this would become
> > >=20
> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ret__ =3D emulate_xchg_1_2=
(ptr__, new__, release_barrier,
> > > acquire_barrier); \
> > >=20
> > > But, unlike assumed above, there's no enforcement here that a 2-
> > > byte
> > > quantity won't cross a word, double-word, cache line, or even
> > > page
> > > boundary. That might be okay if then the code would simply crash
> > > (like
> > > the AMO insns emitted further down would), but aiui silent
> > > misbehavior
> > > would result.
> > As I mentioned above with 4-byte alignment and then reading and
> > working
> > with 8-byte then crossing a word or double-word boundary shouldn't
> > be
> > an issue.
> >=20
> > I am not sure that I know how to check that we are crossing cache
> > line
> > boundary.
> >=20
> > Regarding page boundary, if the next page is mapped then all should
> > work fine, otherwise it will be an exception.
>=20
> Are you sure lr.d / sc.d are happy to access across such a boundary,
> when both pages are mapped?
If they are mapped, my expectation that lr.d and sc.d should be happy.

>=20
> To me it seems pretty clear that for atomic accesses you want to
> demand natural alignment, i.e. 2-byte alignment for 2-byte accesses.
> This way you can be sure no potentially problematic boundaries will
> be crossed.
It makes sense, but I am not sure that I can guarantee that a user of
macros will always have 2-byte alignment (except during a panic) in the
future.

Even now, I am uncertain that everyone will be willing to add
__alignment(...) to struct vcpu->is_urgent
(xen/include/xen/sched.h:218) and other possible cases to accommodate
RISC-V requirements.

>=20
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 break; \
> > > > +=C2=A0=C2=A0=C2=A0 case 4: \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __amoswap_generic(ptr__=
, new__, ret__,\
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 ".w" sfx,=C2=A0 release_barrier,
> > > > acquire_barrier); \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 break; \
> > > > +=C2=A0=C2=A0=C2=A0 case 8: \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __amoswap_generic(ptr__=
, new__, ret__,\
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 ".d" sfx,=C2=A0 release_barrier,
> > > > acquire_barrier); \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 break; \
> > > > +=C2=A0=C2=A0=C2=A0 default: \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 STATIC_ASSERT_UNREACHAB=
LE(); \
> > > > +=C2=A0=C2=A0=C2=A0 } \
> > > > +=C2=A0=C2=A0=C2=A0 ret__; \
> > > > +})
> > > > +
> > > > +#define xchg_relaxed(ptr, x) \
> > > > +({ \
> > > > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) x_ =3D (x); \
> > > > +=C2=A0=C2=A0=C2=A0 (__typeof__(*(ptr)))__xchg_generic(ptr, x_,
> > > > sizeof(*(ptr)),
> > > > "", "", ""); \
> > > > +})
> > > > +
> > > > +#define xchg_acquire(ptr, x) \
> > > > +({ \
> > > > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) x_ =3D (x); \
> > > > +=C2=A0=C2=A0=C2=A0 (__typeof__(*(ptr)))__xchg_generic(ptr, x_,
> > > > sizeof(*(ptr)), \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 "", "",
> > > > RISCV_ACQUIRE_BARRIER); \
> > > > +})
> > > > +
> > > > +#define xchg_release(ptr, x) \
> > > > +({ \
> > > > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) x_ =3D (x); \
> > > > +=C2=A0=C2=A0=C2=A0 (__typeof__(*(ptr)))__xchg_generic(ptr, x_,
> > > > sizeof(*(ptr)),\
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 "",
> > > > RISCV_RELEASE_BARRIER,
> > > > ""); \
> > > > +})
> > > > +
> > > > +#define xchg(ptr,x) \
> > > > +({ \
> > > > +=C2=A0=C2=A0=C2=A0 __typeof__(*(ptr)) ret__; \
> > > > +=C2=A0=C2=A0=C2=A0 ret__ =3D (__typeof__(*(ptr))) \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 __xchg_generic(ptr, (unsigned long)(x),
> > > > sizeof(*(ptr)), \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 ".aqrl", "", ""); \
> > >=20
> > > The .aqrl doesn't look to affect the (emulated) 1- and 2-byte
> > > cases.
> > >=20
> > > Further, amoswap also exists in release-only and acquire-only
> > > forms.
> > > Why do you prefer explicit barrier insns over those? (Looks to
> > > similarly apply to the emulation path as well as to the cmpxchg
> > > machinery then, as both lr and sc also come in all four possible
> > > acquire/release forms. Perhaps for the emulation path using
> > > explicit barriers is better, in case the acquire/release forms of
> > > lr/sc - being used inside the loop - might perform worse.)
> > As 1- and 2-byte cases are emulated I decided that is not to
> > provide
> > sfx argument for emulation macros as it will not have to much
> > affect on
> > emulated types and just consume more performance on acquire and
> > release
> > version of sc/ld instructions.
>=20
> Question is whether the common case (4- and 8-byte accesses)
> shouldn't
> be valued higher, with 1- and 2-byte emulation being there just to
> allow things to not break altogether.
If I understand you correctly, it would make sense to add the 'sfx'
argument for the 1/2-byte access case, ensuring that all options are
available for 1/2-byte access case as well.


~ Oleksii



From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:30:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:30:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682916.1062159 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4f1-0006n6-Ns; Mon, 19 Feb 2024 14:30:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682916.1062159; Mon, 19 Feb 2024 14:30:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4f1-0006mz-LJ; Mon, 19 Feb 2024 14:30:23 +0000
Received: by outflank-mailman (input) for mailman id 682916;
 Mon, 19 Feb 2024 14:30:22 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rc4f0-0006mh-1T; Mon, 19 Feb 2024 14:30:22 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rc4f0-0008HU-0X; Mon, 19 Feb 2024 14:30:22 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rc4ez-0000AU-JO; Mon, 19 Feb 2024 14:30:21 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rc4ez-0000WU-It; Mon, 19 Feb 2024 14:30:21 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jmeL4WrZ4afM79FqyPp7eWCEu+N1W1Fx+MmjXyFaprM=; b=SUSoMZ4zPg1f/3my6JfwEjeCFb
	LoBTVg5oKywmg7OdegnvIB7b/mmKCf5WLQP42rP27V9xLea4lKZZ/r4vFwGWCHv0M0b/lMaZfGnvK
	Jl8yv7F5M45k5KHL14++UfmznTr4bIxqUyrtd1safRciBKfrQOg6QzjL4zL87t8jNNGg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184701-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184701: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
X-Osstest-Versions-That:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 19 Feb 2024 14:30:21 +0000

flight 184701 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184701/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
baseline version:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3

Last test of basis   184685  2024-02-16 18:03:46 Z    2 days
Testing same since   184701  2024-02-19 12:03:49 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Frediano Ziglio <frediano.ziglio@cloud.com>
  Julien Grall <jgrall@amazon.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   0441c3acc7..c144b9e324  c144b9e32427ba37e0e0443a0d3fa53e9fb878b6 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:32:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:32:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682922.1062169 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4hN-0007WS-5N; Mon, 19 Feb 2024 14:32:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682922.1062169; Mon, 19 Feb 2024 14:32:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4hN-0007WL-2h; Mon, 19 Feb 2024 14:32:49 +0000
Received: by outflank-mailman (input) for mailman id 682922;
 Mon, 19 Feb 2024 14:32:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pKiv=J4=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rc4hM-0007WE-4k
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:32:48 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c04d873a-cf33-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 15:32:47 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-511ac32fe38so6606220e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 06:32:47 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 tb5-20020a1709078b8500b00a3d599f47c2sm3030460ejc.18.2024.02.19.06.32.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 19 Feb 2024 06:32:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c04d873a-cf33-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708353167; x=1708957967; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=IywGlil8rOYVi4LTzlz6gKPp2TTUAfS+ZJIbM80gP9k=;
        b=FV/6svAC1ACqH+g2woiDKE/ilpxzOpab45/tZhV9biwUfvCSt8kbH0iJa30vEvSZUz
         D7g6lrt1svPhVUyUShOPqpTBbU4FJsPIcVxxq7Nd3RCCodt4IA9Cs0Sg/BAZacQSOH/k
         aG+rIzs0g1sTqXp2IvVSEvZRHn8yrE/FFhkIsCYJ38TrI3ypMEg009Fx/p5LBbWkoHpp
         UpW2PlitLstxU0voaF0SuP7w1fJwsdXBln37X03zFHhuKd9VcTWky9ZnpVPGSTgzi5Da
         nAP56EYKK92YsHWXJdPoFs864PCoqPcd0DRuXua88ag9gJ1WFTilFEqUyIQwqEZWxCJB
         SJwQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708353167; x=1708957967;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IywGlil8rOYVi4LTzlz6gKPp2TTUAfS+ZJIbM80gP9k=;
        b=K48OJ5CrQyBUeiYQ+hX8PMW8i9Qnp2LCQhIkAw9YPkZ6AaScMJxxi9v8ChSTWcwX10
         x2pGFGqiVZBCBCJGiHnrIsVFi/BNVOaNajVhBg8GflXWK8uftIOZHoLukcaIbQY2pCDz
         iQqHQwMRm1P1OeddvRhypSILLHhHDzI8vCVJ7gSyfDJiBkDbcT+X9A+AnDxS0Qdxwp2s
         Z2PQLLs6OMC8Vt2HVnjCdafDkOVhKPLUr7riPJ5hyROfL1fg/QxoryvHdHN/JJsRIIfd
         GvwLqC89q+xgT0HBd3rswsFIAqklmaG31J+iqC8TvuatM78zNFk68t6LmypI+bx/a7HX
         PawA==
X-Forwarded-Encrypted: i=1; AJvYcCUbVBPvgR2N+tES7Z3fwkG930A4g7G59MWAGwBJnR3vJyjonUr6nuPIENX4Elt4lNl7LlQNh803lq6XdzStIToHDiSkmYDfXVRdN0O/vx8=
X-Gm-Message-State: AOJu0YxMP4eehwb/5r8QuKjB7/g24tMZuYQaSgNUmtf4FeXj07GM7zm8
	xwuT7fl5GQxnnFyB5yqnIcKEy7cPhu3gIq86/mruhsRUMYUrs4lW
X-Google-Smtp-Source: AGHT+IGa/GO+HHyk1hac0FuhM+9zOCL41IdFHdJnmtKiOdQ27398WRkH3I4d6A1nrplJGt+l/t/eqg==
X-Received: by 2002:ac2:5a5c:0:b0:512:a584:e999 with SMTP id r28-20020ac25a5c000000b00512a584e999mr4777400lfn.22.1708353166621;
        Mon, 19 Feb 2024 06:32:46 -0800 (PST)
Message-ID: <876ba6feab2c22011b027af6211f1f2019228bd9.camel@gmail.com>
Subject: Re: [PATCH v4 13/30] xen/riscv: introduce io.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Date: Mon, 19 Feb 2024 15:32:45 +0100
In-Reply-To: <afa59d63-2310-4081-9fc8-1d8a66ae24fc@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <64afc021f680e55d486f8429c02513bc9bf4b9ea.1707146506.git.oleksii.kurochko@gmail.com>
	 <afa59d63-2310-4081-9fc8-1d8a66ae24fc@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Sun, 2024-02-18 at 19:07 +0000, Julien Grall wrote:
>=20
> > +/*
> > + * Unordered I/O memory access primitives.=C2=A0 These are even more
> > relaxed than
> > + * the relaxed versions, as they don't even order accesses between
> > successive
> > + * operations to the I/O regions.
> > + */
> > +#define readb_cpu(c)		({ uint8_t=C2=A0 __r =3D __raw_readb(c);
> > __r; })
> > +#define readw_cpu(c)		({ uint16_t __r =3D
> > le16_to_cpu((__force __le16)__raw_readw(c)); __r; })
> > +#define readl_cpu(c)		({ uint32_t __r =3D
> > le32_to_cpu((__force __le32)__raw_readl(c)); __r; })
> > +
> > +#define writeb_cpu(v,c)		((void)__raw_writeb(v,c))
> > +#define
> > writew_cpu(v,c)		((void)__raw_writew((__force uint16_t)cpu_to_le16(v),c=
))
> > +#define
> > writel_cpu(v,c)		((void)__raw_writel((__force uint32_t)cpu_to_le32(v),c=
))
>=20
> NIT: __raw_write*() are already returning void. So I am not sure to=20
> understand the pointer of the cast. IIUC, this is coming from Linux,
> are=20
> you intend to keep the code as-is (including style)? If not, then I=20
> woudl consider to drop the cast on the three lines above and ...
Changes have already been made in this header, so it makes sense to
remove these casts. Thanks.


~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:36:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:36:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682929.1062179 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4kX-0008Au-MI; Mon, 19 Feb 2024 14:36:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682929.1062179; Mon, 19 Feb 2024 14:36:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4kX-0008An-Jl; Mon, 19 Feb 2024 14:36:05 +0000
Received: by outflank-mailman (input) for mailman id 682929;
 Mon, 19 Feb 2024 14:36:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pKiv=J4=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rc4kW-0008AD-7t
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:36:04 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 354dde8a-cf34-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 15:36:03 +0100 (CET)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-5649f396269so930535a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 06:36:03 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 pk27-20020a170906d7bb00b00a3ee20b00d0sm13620ejb.4.2024.02.19.06.35.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 19 Feb 2024 06:35:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 354dde8a-cf34-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708353363; x=1708958163; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=1AMKLvPr8rT/i5jcMKX6ZoRGjd3JpDPwqUSMqNzSME4=;
        b=SM97mE/Ebxz65ZhqipFHpOLG3/2ULVNw+EQbDxgn4Zxx7AIpbmveZiv6fytMZaLUqF
         dmHCxC/Tv1aRky/RKOPG01fqMcQGOkt5B1bOn2vu4bdzOgnotjLCdTXmrU3sM5B+HCst
         eUMJl2x6JQvHYxFMWMQUD8z7+MKRJpp0YImNjvabNYZelMZkJtyHVgxvPP/clGMZw7Uk
         EvH03y+OiC4e+rY9nCeTAkWiA2sCAwQItDg1WDZBDGZmOo+FwwxsmBqRwIIgKgB/murM
         wI1cCWNsiso6YX9k1OIUXORYWcRT/NMlYpn92v4OU00b9SxgDU8JTqjqXOUGA+47+s7A
         B8WQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708353363; x=1708958163;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1AMKLvPr8rT/i5jcMKX6ZoRGjd3JpDPwqUSMqNzSME4=;
        b=Ne57P32dKWhIYqO5+ATCvOYU+SQBgDxcL/9OvLeIKmVXGEfwwdImwBGpnM4MziH5l/
         cCZgvLlLeUNNRmHkFAqKuHM+jts70Tos02jw3PySUMUM8LnvIeDvR3wzArViZC07NeGb
         Z9D9XlOPEhOr1q27mW9Uwg6ravwAcqVf75T6rUv16jT1cll9njvb6rQnuFMNd/a+NS+U
         AefuTHodN2R75NLuRQphm6B+geIeEkvZ/t2sjxwIDXVySYLTfmE+mOqb544Iqql0uStR
         fGytvUHTkcoTBFMa98qHSSV1rYGLMr7LvBW1hqdAclTrpuEEV8uVuxdqvneAc+rfdCtb
         eO4Q==
X-Forwarded-Encrypted: i=1; AJvYcCUNtDNcPbFGc2KdutP1CWDy6D70eJNSVQLetcXHQco9BYzcop02laRsZuV1W0o2NudJdV6wZMsGGIInAhMjqlA0TFKkUrgzDVXUb5tBYwc=
X-Gm-Message-State: AOJu0Yw7sVS4z6yuOlkE8MW07dMkNR0AQV5H7rcEEFWDk5VubgmBqPwf
	Gns+OyHnZzBH95kw7BoMFDvaRWo6aVRzd1deAJjDxsLWaJAFu1kW
X-Google-Smtp-Source: AGHT+IH3sFgn1cjYl6SdjJhOCyWM1G0BJo1f54dldDs7aDwGV0a+y+eDnr/OmtgH+DomeNi7EzwiGQ==
X-Received: by 2002:a17:906:6d04:b0:a3e:920d:4f8a with SMTP id m4-20020a1709066d0400b00a3e920d4f8amr2788117ejr.37.1708353358323;
        Mon, 19 Feb 2024 06:35:58 -0800 (PST)
Message-ID: <5617af1e8d94b4f1946a26e18eb10d62fd6c0fbe.camel@gmail.com>
Subject: Re: [PATCH v4 14/30] xen/riscv: introduce atomic.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Bobby Eshleman <bobbyeshleman@gmail.com>, Alistair Francis
 <alistair.francis@wdc.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Date: Mon, 19 Feb 2024 15:35:56 +0100
In-Reply-To: <26e01c13-364d-474b-8b8e-d1721ff0daf0@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <6554f2479e19ed3eae6de842ac1568c31d236461.1707146506.git.oleksii.kurochko@gmail.com>
	 <26e01c13-364d-474b-8b8e-d1721ff0daf0@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

SGkgSnVsaWVuLAoKT24gU3VuLCAyMDI0LTAyLTE4IGF0IDE5OjIyICswMDAwLCBKdWxpZW4gR3Jh
bGwgd3JvdGU6Cj4gSGksCj4gCj4gT24gMDUvMDIvMjAyNCAxNTozMiwgT2xla3NpaSBLdXJvY2hr
byB3cm90ZToKPiA+IEZyb206IEJvYmJ5IEVzaGxlbWFuIDxib2JieWVzaGxlbWFuQGdtYWlsLmNv
bT4KPiA+IAo+ID4gQWRkaXRpb25hbGx5LCB0aGlzIHBhdGNoIGludHJvZHVjZXMgbWFjcm9zIGlu
IGZlbmNlLmgsCj4gPiB3aGljaCBhcmUgdXRpbGl6ZWQgaW4gYXRvbWljLmguCj4gPiAKPiA+IGF0
b21pYyMjcHJlZml4IyNfKnhjaGdfKihhdG9taWMjI3ByZWZpeCMjX3QgKnYsIGNfdCBuKQo+ID4g
d2VyZSB1cGRhdGVkIHRvIHVzZSBfXyp4Y2hnX2dlbmVyaWMoKS4KPiA+IAo+ID4gU2lnbmVkLW9m
Zi1ieTogT2xla3NpaSBLdXJvY2hrbyA8b2xla3NpaS5rdXJvY2hrb0BnbWFpbC5jb20+Cj4gCj4g
VGhlIGF1dGhvciBpcyBCb2JieSwgYnV0IEkgZG9uJ3Qgc2VlIGEgU2lnbmVkLW9mZi1ieS4gRGlk
IHlvdSBmb3Jnb3QKPiBpdD8KSSBtaXNzZWQgdG8gYWRkIHRoYXQgYXMgSSB0aG91Z2h0IHRoYXQg
aXQgd291bGQgYmUgZW5vdWdoIHRvIGNoYW5nZSBhCmNvbW1pdCBhdXRob3IuCgo+IAo+ID4gLS0t
Cj4gPiBDaGFuZ2VzIGluIFY0Ogo+ID4gwqAgLSBkbyBjaGFuZ2VzIHJlbGF0ZWQgdG8gdGhlIHVw
ZGF0ZXMgb2YgW1BBVENIIHYzIDEzLzM0XQo+ID4geGVuL3Jpc2N2OiBpbnRyb2R1Y2UgY21weGNo
Zy5oCj4gPiDCoCAtIGRyb3AgY2FzdHMgaW4gcmVhZF9hdG9taWNfc2l6ZSgpLCB3cml0ZV9hdG9t
aWMoKSwgYWRkX3NpemVkKCkKPiA+IMKgIC0gdGFicyAtPiBzcGFjZXMKPiA+IMKgIC0gZHJvcCAj
aWZkZWYgQ09ORklHX1NNUCAuLi4gI2VuZGlmIGluIGZlbmNlLmhhIGFzIGl0IGlzIHNpbXBsZXIK
PiA+IHRvIGhhbmRsZSBOUl9DUFVTPTEKPiA+IMKgwqDCoCB0aGUgc2FtZSBhcyBOUl9DUFVTPjEg
d2l0aCBhY2NlcHRpbmcgbGVzcyB0aGFuIGlkZWFsCj4gPiBwZXJmb3JtYW5jZS4KPiA+IC0tLQo+
ID4gQ2hhbmdlcyBpbiBWMzoKPiA+IMKgwqAgLSB1cGRhdGUgdGhlIGNvbW1pdCBtZXNzYWdlCj4g
PiDCoMKgIC0gYWRkIFNQRFggZm9yIGZlbmNlLmgKPiA+IMKgwqAgLSBjb2RlIHN0eWxlIGZpeGVz
Cj4gPiDCoMKgIC0gUmVtb3ZlIC8qIFRPRE86IC4uLiAqLyBmb3IgYWRkX3NpemVkIG1hY3Jvcy4g
SXQgbG9va3MgY29ycmVjdAo+ID4gdG8gbWUuCj4gPiDCoMKgIC0gcmUtb3JkZXIgdGhlIHBhdGNo
Cj4gPiDCoMKgIC0gbWVyZ2UgdG8gdGhpcyBwYXRjaCBmZW5jZS5oCj4gPiAtLS0KPiA+IENoYW5n
ZXMgaW4gVjI6Cj4gPiDCoCAtIENoYW5nZSBhbiBhdXRob3Igb2YgY29tbWl0LiBJIGdvdCB0aGlz
IGhlYWRlciBmcm9tIEJvYmJ5J3Mgb2xkCj4gPiByZXBvLgo+ID4gLS0tCj4gPiDCoCB4ZW4vYXJj
aC9yaXNjdi9pbmNsdWRlL2FzbS9hdG9taWMuaCB8IDM5NQo+ID4gKysrKysrKysrKysrKysrKysr
KysrKysrKysrKwo+ID4gwqAgeGVuL2FyY2gvcmlzY3YvaW5jbHVkZS9hc20vZmVuY2UuaMKgIHzC
oMKgIDggKwo+ID4gwqAgMiBmaWxlcyBjaGFuZ2VkLCA0MDMgaW5zZXJ0aW9ucygrKQo+ID4gwqAg
Y3JlYXRlIG1vZGUgMTAwNjQ0IHhlbi9hcmNoL3Jpc2N2L2luY2x1ZGUvYXNtL2F0b21pYy5oCj4g
PiDCoCBjcmVhdGUgbW9kZSAxMDA2NDQgeGVuL2FyY2gvcmlzY3YvaW5jbHVkZS9hc20vZmVuY2Uu
aAo+ID4gCj4gPiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvcmlzY3YvaW5jbHVkZS9hc20vYXRvbWlj
LmgKPiA+IGIveGVuL2FyY2gvcmlzY3YvaW5jbHVkZS9hc20vYXRvbWljLmgKPiA+IG5ldyBmaWxl
IG1vZGUgMTAwNjQ0Cj4gPiBpbmRleCAwMDAwMDAwMDAwLi4yNjdkM2MwODAzCj4gPiAtLS0gL2Rl
di9udWxsCj4gPiArKysgYi94ZW4vYXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9hdG9taWMuaAo+ID4g
QEAgLTAsMCArMSwzOTUgQEAKPiA+ICsvKiBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIu
MC1vbmx5ICovCj4gPiArLyoKPiA+ICsgKiBUYWtlbiBhbmQgbW9kaWZpZWQgZnJvbSBMaW51eC4K
PiAKPiBXaGljaCB2ZXJzaW9uIG9mIExpbnV4PyBDYW4geW91IGFsc28gc3BlbGwgb3V0IHdoYXQg
YXJlIHRoZSBiaWcKPiBjaGFuZ2VzPyAKPiBUaGlzIHdvdWxkIGJlIGhlbHBmdWwgaWYgd2UgbmVl
ZCB0byByZS1zeW5jLgpTdXJlLCBJJ2xsIGFkZCB0aGUgY2hhbmdlcyBoZXJlLgoKPiAKPiA+ICsg
Kgo+ID4gKyAqIGF0b21pYyMjcHJlZml4IyNfKnhjaGdfKihhdG9taWMjI3ByZWZpeCMjX3QgKnYs
IGNfdCBuKSB3ZXJlCj4gPiB1cGRhdGVkIHRvIHVzZQo+ID4gKyAqIF9fKnhjaGdfZ2VuZXJpYygp
Cj4gPiArICoKPiA+ICsgKiBDb3B5cmlnaHQgKEMpIDIwMDcgUmVkIEhhdCwgSW5jLiBBbGwgUmln
aHRzIFJlc2VydmVkLgo+ID4gKyAqIENvcHlyaWdodCAoQykgMjAxMiBSZWdlbnRzIG9mIHRoZSBV
bml2ZXJzaXR5IG9mIENhbGlmb3JuaWEKPiA+ICsgKiBDb3B5cmlnaHQgKEMpIDIwMTcgU2lGaXZl
Cj4gPiArICogQ29weXJpZ2h0IChDKSAyMDIxIFZhdGVzIFNBUwo+ID4gKyAqLwo+ID4gKwo+ID4g
KyNpZm5kZWYgX0FTTV9SSVNDVl9BVE9NSUNfSAo+ID4gKyNkZWZpbmUgX0FTTV9SSVNDVl9BVE9N
SUNfSAo+ID4gKwo+ID4gKyNpbmNsdWRlIDx4ZW4vYXRvbWljLmg+Cj4gPiArI2luY2x1ZGUgPGFz
bS9jbXB4Y2hnLmg+Cj4gPiArI2luY2x1ZGUgPGFzbS9mZW5jZS5oPgo+ID4gKyNpbmNsdWRlIDxh
c20vaW8uaD4KPiA+ICsjaW5jbHVkZSA8YXNtL3N5c3RlbS5oPgo+ID4gKwo+ID4gK3ZvaWQgX19i
YWRfYXRvbWljX3NpemUodm9pZCk7Cj4gPiArCj4gPiArc3RhdGljIGFsd2F5c19pbmxpbmUgdm9p
ZCByZWFkX2F0b21pY19zaXplKGNvbnN0IHZvbGF0aWxlIHZvaWQgKnAsCj4gPiArwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgIHZvaWQgKnJlcywKPiA+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgdW5zaWduZWQgaW50IHNpemUpCj4gPiArewo+ID4gK8KgwqDCoCBzd2l0Y2ggKCBzaXpl
ICkKPiA+ICvCoMKgwqAgewo+ID4gK8KgwqDCoCBjYXNlIDE6ICoodWludDhfdCAqKXJlcyA9IHJl
YWRiKHApOyBicmVhazsKPiA+ICvCoMKgwqAgY2FzZSAyOiAqKHVpbnQxNl90ICopcmVzID0gcmVh
ZHcocCk7IGJyZWFrOwo+ID4gK8KgwqDCoCBjYXNlIDQ6ICoodWludDMyX3QgKilyZXMgPSByZWFk
bChwKTsgYnJlYWs7Cj4gPiArwqDCoMKgIGNhc2UgODogKih1aW50MzJfdCAqKXJlc8KgID0gcmVh
ZHEocCk7IGJyZWFrOwo+ID4gK8KgwqDCoCBkZWZhdWx0OiBfX2JhZF9hdG9taWNfc2l6ZSgpOyBi
cmVhazsKPiA+ICvCoMKgwqAgfQo+ID4gK30KPiA+ICsKPiA+ICsjZGVmaW5lIHJlYWRfYXRvbWlj
KHApICh7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgIFwKPiA+ICvCoMKgwqAgdW5pb24geyB0eXBlb2YoKnApIHZhbDsgY2hhciBjWzBd
OyB9IHhfO8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCByZWFkX2F0b21pY19z
aXplKHAsIHhfLmMsIHNpemVvZigqcCkpO8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+
ICvCoMKgwqAgeF8udmFsO8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK30p
Cj4gPiArCj4gPiArI2RlZmluZSB3cml0ZV9hdG9taWMocCwgeCnCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICsoe8KgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCB0
eXBlb2YoKnApIHhfXyA9ICh4KTvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCBzd2l0Y2ggKCBzaXplb2YoKnAp
ICnCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgXAo+ID4gK8KgwqDCoCB7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoCBcCj4gPiArwqDCoMKgIGNhc2UgMTogd3JpdGViKCh1aW50OF90KXhfXyzCoCBwKTsg
YnJlYWs7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIGNhc2UgMjogd3JpdGV3
KCh1aW50MTZfdCl4X18sIHApOyBicmVhazvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvC
oMKgwqAgY2FzZSA0OiB3cml0ZWwoKHVpbnQzMl90KXhfXywgcCk7IGJyZWFrO8KgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgXAo+ID4gK8KgwqDCoCBjYXNlIDg6IHdyaXRlcSgodWludDY0X3QpeF9fLCBw
KTsgYnJlYWs7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIGRlZmF1bHQ6IF9f
YmFkX2F0b21pY19zaXplKCk7IGJyZWFrO8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBc
Cj4gPiArwqDCoMKgIH3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
IFwKPiA+ICvCoMKgwqAgeF9fO8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg
XAo+ID4gK30pCj4gPiArCj4gPiArI2RlZmluZSBhZGRfc2l6ZWQocCwgeCnCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+
ICsoe8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+
ID4gK8KgwqDCoCB0eXBlb2YoKihwKSkgeF9fID0gKHgpO8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqAgc3dpdGNoICgg
c2l6ZW9mKCoocCkpICnCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoCBcCj4gPiArwqDCoMKgIHvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqAgY2FzZSAxOiB3cml0ZWIocmVhZF9hdG9taWMo
cCkgKyB4X18sIHApOyBicmVhazvCoMKgwqDCoCBcCj4gPiArwqDCoMKgIGNhc2UgMjogd3JpdGV3
KHJlYWRfYXRvbWljKHApICsgeF9fLCBwKTsgYnJlYWs7wqDCoMKgwqAgXAo+ID4gK8KgwqDCoCBj
YXNlIDQ6IHdyaXRlbChyZWFkX2F0b21pYyhwKSArIHhfXywgcCk7IGJyZWFrO8KgwqDCoMKgIFwK
PiA+ICvCoMKgwqAgZGVmYXVsdDogX19iYWRfYXRvbWljX3NpemUoKTsgYnJlYWs7wqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgIFwKPiA+ICvCoMKgwqAgfcKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgXAo+ID4gK30pCj4gPiArCj4gPiArLyoKPiA+ICsgKsKg
IF9fdW5xdWFsX3NjYWxhcl90eXBlb2YoeCkgLSBEZWNsYXJlIGFuIHVucXVhbGlmaWVkIHNjYWxh
cgo+ID4gdHlwZSwgbGVhdmluZwo+ID4gKyAqwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBu
b24tc2NhbGFyIHR5cGVzIHVuY2hhbmdlZC4KPiA+ICsgKgo+ID4gKyAqIFByZWZlciBDMTEgX0dl
bmVyaWMgZm9yIGJldHRlciBjb21waWxlLXRpbWVzIGFuZCBzaW1wbGVyIGNvZGUuCj4gPiBOb3Rl
OiAnY2hhcicKPiAKPiBYZW4gaXMgdGVjaG5pY2FsbHkgYnVpbHQgdXNpbmcgYzk5L2dudTk5LiBT
byBpdCBpcyBmZWVscyBhIGJpdCBvZGQgdG8KPiBpbnRyb2R1Y2UgYSBDMTEgZmVhdHVyZS4gSSBz
ZWUgdGhhdCBfR2VuZXJpYyBpcyBhbHJlYWR5IHVzZWQgaW4KPiBQUEMuLi4gCj4gSG93ZXZlciwg
aWYgd2UgZGVjaWRlIHRvIGFkZCBtb3JlIHVzZSBvZiBpdCwgdGhlbiBJIHRoaW5rIHRoaXMgc2hv
dWxkCj4gYXQgCj4gbWluaW11bSBiZSBkb2N1bWVudGVkIGluIGRvY3MvbWlzcmEvQy1sYW5ndWFn
ZS10b29sY2hhaW4ucnN0ICh0aGUKPiBtb3JlIAo+IGlmIHdlIHBsYW4gdGhlIG1hY3JvIGlzIG1v
dmVkIHRvIGNvbW1vbiBhcyBKYW4gc3VnZ2VzdGVkKS4KPiAKPiBDaGVlcnMsCj4gCgo=



From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:39:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:39:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682933.1062190 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4o4-0000TD-5h; Mon, 19 Feb 2024 14:39:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682933.1062190; Mon, 19 Feb 2024 14:39:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4o4-0000T6-2U; Mon, 19 Feb 2024 14:39:44 +0000
Received: by outflank-mailman (input) for mailman id 682933;
 Mon, 19 Feb 2024 14:39:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc4o2-0000T0-M7
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:39:42 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b72dcec7-cf34-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 15:39:41 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-412698cdd77so3674615e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 06:39:41 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p18-20020a05600c359200b0041256523bf8sm8848960wmq.31.2024.02.19.06.39.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 06:39:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b72dcec7-cf34-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708353581; x=1708958381; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Eg46NAymx8dIIJL3V+k2/dbUTTVI5beeX6j3e+DHUJg=;
        b=PgbKG+UxiqhBfjbDCFKVx5+AQnQBvWsMV0fVapUEVOc9T4fb1ILic8JzG+BX05YeHQ
         TWIuk/anHiOOSK69+8oCEv6le34WYIfcdbQOHHnSjdR+FOIljqQkTwBoqzjkMZjIdrff
         AzvVMZ0nZKSVnQucILkyd0Ju4/H07pxHyXBSs78/cc1Rie9vYz6D026sCc6pYBvCsMa8
         tHGbCx64lgXY3m9AXKmu6RzUKbT5g8u1S2xXXCGdNFRoCrDRGqjUovChbvkcYr8Q3vDi
         Rtl/jseUgfph55cE9Io/QBmgakIXdM9JhUFigv9wy0ppzp71uO1a0PpHzlYjfAYALJ9N
         vwWg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708353581; x=1708958381;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Eg46NAymx8dIIJL3V+k2/dbUTTVI5beeX6j3e+DHUJg=;
        b=uOpJKy0fje6aAYtRUMFCSmXvw5rx7W8y1HgWE+XMEE/bkWyOtQLOB/+FWFiz4DWJ0r
         guLa2RIl92CRGoupJV8Qk439vWhjVL4pMCkL38M/zO0BKagbrw88nnvkCAG5DAvor3rJ
         t8Ms25YMKiud2oF7zyIBIUgG5vKOBUvlsXJ80Gs7NV4rcczaXv0JQJwZptTMJ+PF9jAP
         kVlGUV4YJlSwc3xZpZhp8Oi25yjceo/WpiDtA4XyUmJ13Qv4qeYKNm39qkZHXJMtTJ6L
         ionFnQHsjAV7TyRK/oRS0bd6cy/8WMK2VmUxz8VpdRNPeTd6EhFa3scx3jpSZ6nIfBo4
         52Ug==
X-Forwarded-Encrypted: i=1; AJvYcCUT/gki17Ak1OUvkQ9umyJCqKEOGbzgeWwajcdPqfOYR5dda45MRJaGhQUxAPKmJbfDFEvOgpTl32aRII6TmYAmaKNxh4yTdUVA01RsQqU=
X-Gm-Message-State: AOJu0Yy2e/JNIzbZ/6zSS6gRnqyoOvWdzVSz0DT04zHSrxBAKGUanXX8
	3qYpT3vf3t6AhyOeaKzSHchRiSju2RmIt7+kPs6XiAGZMVq03EufBnb6IlS3Ag==
X-Google-Smtp-Source: AGHT+IHO76xhLBvmz43qHToAPze/ftWK/H0MI8HvorXPID5KWU/GT7d4+o/luhmJHQT7UhtnA7aO4g==
X-Received: by 2002:a05:600c:1c1d:b0:412:529f:934a with SMTP id j29-20020a05600c1c1d00b00412529f934amr7463079wms.19.1708353581025;
        Mon, 19 Feb 2024 06:39:41 -0800 (PST)
Message-ID: <9af4dd5f-771e-4e43-9e04-a393a2cdb746@suse.com>
Date: Mon, 19 Feb 2024 15:39:39 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/6] xen/hvm: Move other hvm_function_table booleans into
 the caps bitfield
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Jun Nakajima <jun.nakajima@intel.com>,
 Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-4-george.dunlap@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240206012051.3564035-4-george.dunlap@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 02:20, George Dunlap wrote:
> @@ -144,7 +144,7 @@ static bool __init hap_supported(struct hvm_function_table *fns)
>  
>      if ( !opt_hap_enabled )
>      {
> -        fns->hap_supported = 0;
> +        fns->caps.hap = 0;

As you touch such, would you mind switching to true/false instead of
1/0 at this occasion?

> @@ -3000,7 +3000,7 @@ const struct hvm_function_table * __init start_vmx(void)
>          vmx_function_table.update_eoi_exit_bitmap = vmx_update_eoi_exit_bitmap;
>          vmx_function_table.process_isr = vmx_process_isr;
>          vmx_function_table.handle_eoi = vmx_handle_eoi;
> -        vmx_function_table.virtual_intr_delivery_enabled = true;
> +        vmx_function_table.caps.virtual_intr_delivery = true;

I'm unsure about this one - it had "enabled" in its name for a good
reason. Then again its (somewhat more involved) derivation from
other capability bits (and a command line option) isn't fundamentally
different from that of, say, hap_supported. Hence I guess with the
other item taken care of
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:40:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:40:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682936.1062200 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4oU-0001v2-EZ; Mon, 19 Feb 2024 14:40:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682936.1062200; Mon, 19 Feb 2024 14:40:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4oU-0001uv-Bc; Mon, 19 Feb 2024 14:40:10 +0000
Received: by outflank-mailman (input) for mailman id 682936;
 Mon, 19 Feb 2024 14:40:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pKiv=J4=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rc4oT-0000og-Ro
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:40:09 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c72613e8-cf34-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 15:40:08 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a3cc2f9621aso456498466b.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 06:40:08 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 r12-20020a170906704c00b00a3d932d8fa7sm3024217ejj.183.2024.02.19.06.40.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 19 Feb 2024 06:40:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c72613e8-cf34-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708353608; x=1708958408; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=f9zwNMGKSNSAwnTlgSx/X1Qxc/SU9hgGoqtSMEumcRw=;
        b=SctYV4akprnNyIEjktktNOsogkjtGUm6mfYucMqSkarxOFAtyPZ23hEqqboQdtn0Bp
         uwyTjhFouoNHzW1hlDSqEZxsalf0IE3MenBve2gnxXWwXrTapa2SCJV7HjgfmSpZIE9h
         LfiqxQ0NvMmI/JwWQajT5dwcTL/Yr2H2B9YdyEDeShg7DvwC4ADUzbjRnTNRWemeJnzC
         ODI5+DyLZy/mkedErmE3eCrYv3fdcIWUNX6SvCTfnEejnmTBcwtAKCC8Pl45Faw082tv
         nD3E//sBncJ31YTGVnQDqtjo9eB7RGFtQlXANVppAzY7iu2L75Gs4lSrWu810dp5lwi/
         kCoA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708353608; x=1708958408;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=f9zwNMGKSNSAwnTlgSx/X1Qxc/SU9hgGoqtSMEumcRw=;
        b=gN3xl2J6VNM5tGCYbELI8ErWLgiDNPzBdZkUHHfU6dWEgNJt29i8LQ97uVTkdKbsYs
         r/IVqplX0IxC/b9pJk4XnrmciA66XNP6OFatzFgdbERsvyErsOlvsiEac6rcqytUs1Yv
         /B+2ztQTpZDc4fNjV1yIrbrtfHiTOZJ/diZ0ksduWmmiJS4xBbowBp/0l1Wqgy5K8pKR
         fjOChfWTKOD7YlEX5lIvlmFyztZHf5q65UG+MWgOClZ00He6Zoh4NNxE0trhF21vD4dx
         PatOXt6zYgB1R1BLAAxKKsKW2VG49TYwAP4Rkwspdk84CNn1qeBVIbspwrNcWUhimFTV
         XzNg==
X-Forwarded-Encrypted: i=1; AJvYcCW4pibPvzvL9xHm43Y/Z3iS+odlQkmHpBzPxOA9C8/6Eh+w9SG+AoJw0Tz5GLOUoCShyuB90fsN/L9Ni7Koti87Uy3BSNGtNWpPnewQfLI=
X-Gm-Message-State: AOJu0YwCgIBWW5GmwmZGa3WamqeCxBCE0ZGbl+/Eg47RURD6FdskVWYE
	cjv2atoYFdQvhZkEDV5gfTYIwqajobgbJ9aQM4iReJQnw59KnfwE
X-Google-Smtp-Source: AGHT+IFHKWkyJZvt8hl8NblYrUGBOj9S7t78T+2+N32yeVHov0fCHGf3N3T291+ocGQ5SJomwFzwSA==
X-Received: by 2002:a17:906:b753:b0:a3e:c2aa:ceb6 with SMTP id fx19-20020a170906b75300b00a3ec2aaceb6mr1224212ejb.32.1708353607575;
        Mon, 19 Feb 2024 06:40:07 -0800 (PST)
Message-ID: <4a8f923188279747387baf1002be5ba9726969d1.camel@gmail.com>
Subject: Re: [PATCH v4 17/30] xen/riscv: introduce regs.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Date: Mon, 19 Feb 2024 15:40:06 +0100
In-Reply-To: <ea8d10ef-39cd-49a2-a445-6705b87d50b3@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <13cfdfa01b303d2597124f09c0ec0734859b697d.1707146506.git.oleksii.kurochko@gmail.com>
	 <ea8d10ef-39cd-49a2-a445-6705b87d50b3@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

Hi Julien,

On Sun, 2024-02-18 at 18:22 +0000, Julien Grall wrote:
> Hi,
>=20
> On 05/02/2024 15:32, Oleksii Kurochko wrote:
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > Acked-by: Jan Beulich <jbeulich@suse.com>
> > ------
> > Changes in V4:
> > =C2=A0 - add Acked-by: Jan Beulich <jbeulich@suse.com>
> > =C2=A0 - s/BUG()/BUG_ON("unimplemented")
> > ---
> > Changes in V3:
> > =C2=A0 - update the commit message
> > =C2=A0 - add Acked-by: Jan Beulich <jbeulich@suse.com>
> > =C2=A0 - remove "include <asm/current.h>" and use a forward declaration
> > instead.
> > ---
> > Changes in V2:
> > =C2=A0 - change xen/lib.h to xen/bug.h
> > =C2=A0 - remove unnecessary empty line
> > ---
> > xen/arch/riscv/include/asm/regs.h | 29
> > +++++++++++++++++++++++++++++
> > =C2=A0 1 file changed, 29 insertions(+)
> > =C2=A0 create mode 100644 xen/arch/riscv/include/asm/regs.h
> >=20
> > diff --git a/xen/arch/riscv/include/asm/regs.h
> > b/xen/arch/riscv/include/asm/regs.h
> > new file mode 100644
> > index 0000000000..c70ea2aa0c
> > --- /dev/null
> > +++ b/xen/arch/riscv/include/asm/regs.h
> > @@ -0,0 +1,29 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +#ifndef __ARM_RISCV_REGS_H__
> > +#define __ARM_RISCV_REGS_H__
> > +
> > +#ifndef __ASSEMBLY__
> > +
> > +#include <xen/bug.h>
> > +
> > +#define hyp_mode(r)=C2=A0=C2=A0=C2=A0=C2=A0 (0)
>=20
> I don't understand where here you return 0 (which should really be=20
> false) but ...
>=20
> > +
> > +struct cpu_user_regs;
> > +
> > +static inline bool guest_mode(const struct cpu_user_regs *r)
> > +{
> > +=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> > +}
>=20
> ... here you return BUG_ON(). But I couldn't find any user of both=20
> guest_mode() and hyp_mode(). So isn't it a bit prematurate to
> introduce=20
> the helpers?

I agree regarding hyp_mode() it can be dropped , but gest_mode() is
used by common/keyhandler.c:142.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:45:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:45:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682948.1062209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4tn-0002sg-Vl; Mon, 19 Feb 2024 14:45:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682948.1062209; Mon, 19 Feb 2024 14:45:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc4tn-0002sZ-T2; Mon, 19 Feb 2024 14:45:39 +0000
Received: by outflank-mailman (input) for mailman id 682948;
 Mon, 19 Feb 2024 14:45:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0k1M=J4=amd.com=ayan.kumar.halder@srs-se1.protection.inumbo.net>)
 id 1rc4tm-0002sT-Jc
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:45:38 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 899df9e4-cf35-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 15:45:35 +0100 (CET)
Received: from PH8PR12MB7326.namprd12.prod.outlook.com (2603:10b6:510:216::7)
 by CH2PR12MB4246.namprd12.prod.outlook.com (2603:10b6:610:a9::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.19; Mon, 19 Feb
 2024 14:45:31 +0000
Received: from PH8PR12MB7326.namprd12.prod.outlook.com
 ([fe80::22eb:b0b:62c0:f3e5]) by PH8PR12MB7326.namprd12.prod.outlook.com
 ([fe80::22eb:b0b:62c0:f3e5%6]) with mapi id 15.20.7316.018; Mon, 19 Feb 2024
 14:45:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 899df9e4-cf35-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=np9xg1etAXtQUmgx6l6Rooy1KvXqp5XqIog7dxjNuMbqtWrUBqkFGHuf+AGP1h50X1Z9Rmd76lC0pjhALImJXohGKw2FKwFZpeCQedwsbqWV/n6oq993OmJ6anRsywWl6AtdrFfDN1/3iK6qBbIushxRIVlllgpkWMfd8cAzdcgf6uJsGTU6C+WSD9ONlIhXklAbcCs5UcxbdUmDbbC1pkyTkq3K3N6FcK60fg8fMdvbPzWWmizLJJt8uwdDsWhjXlytWKGsdUIJyvbWUgmqp29IZye4SksrfwARnGzHLcA98EAJGhQEJl8o87OlgGLMwOs1tcZ+3wKQ+2S2/aNPzQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=X26PrObdoCuCtb1ntRW8CvZKzJ0J6ddqTeobQR3MGlQ=;
 b=lfE8aMtTyiq2wUb39Sit5NEqJPfvQe0edWHKdFyLo5XZBZagrts5xalkzlYEsyGoixUmpGB4SfKY5L0aIrFsP5zYEU9I2YO2a2FjjAPPStGP86pN/Jm/uRozbbPMVd8nbwO++Vnv3oNOBsXbTzkcTY8taN1ILFN6jcZneqvrb5gKRDV60vcAM2KPIejSg+9WpmxnZHjQIYci5u3EOWIWI+ywXl/pm+ao+YjXEbg5uGnJbcFLPYCeSP/qqxaeSnV0NaYogu8sbvEOJL6ESYrRZrw79ZWtTU+yqzqyh++wUWo45Quk0isGmIqficmcKMiwggMMIdSHjDVZ/lg/2lw7mQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=X26PrObdoCuCtb1ntRW8CvZKzJ0J6ddqTeobQR3MGlQ=;
 b=R1RyEOehy0rZfqwacKvQcuf8H0gJdSfR72VB49mkI5ml5gryw3LmIOUsZPu48nfxZGuPNldAlKheT4QfNCZo3RBWndlS6uHwLpPdav/mWg1TC/0M1s//Qk4AZ0ZB+LDNx/wmil9f3jThswM80ioBC5XGh5+y6lX6k0hV4jwS0Do=
Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
Content-Type: multipart/mixed; boundary="------------j04MiwJlrRLEF3xvh8APOAUO"
Message-ID: <7098c9ab-7008-4a49-92d4-6cd201b1490e@amd.com>
Date: Mon, 19 Feb 2024 14:45:25 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v4 2/3] xen/arm: arm64: Add emulation of Debug Data Transfer
 Registers
To: Julien Grall <julien@xen.org>,
 Ayan Kumar Halder <ayan.kumar.halder@amd.com>, xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, michal.orzel@amd.com,
 luca.fancellu@arm.com
References: <20240131121049.225044-1-ayan.kumar.halder@amd.com>
 <20240131121049.225044-3-ayan.kumar.halder@amd.com>
 <0c4e4a40-7327-4377-bfdd-82e69bc838ae@xen.org>
Content-Language: en-GB
From: Ayan Kumar Halder <ayankuma@amd.com>
In-Reply-To: <0c4e4a40-7327-4377-bfdd-82e69bc838ae@xen.org>
X-ClientProxiedBy: LO4P123CA0570.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:276::14) To PH8PR12MB7326.namprd12.prod.outlook.com
 (2603:10b6:510:216::7)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: PH8PR12MB7326:EE_|CH2PR12MB4246:EE_
X-MS-Office365-Filtering-Correlation-Id: 62f5b541-495b-4c48-1729-08dc31596bde
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	uqb+IXQzfwJLa1I9iXVBaP7BvfxnQ0CU8OYKFVuFfeAF3Yx8QGshKM9qQKmezMzUobMXXhTT29PJmaYVG+8zzNNYprj5g1nrSbF5jPcb6L5clNXQzgrJ4jSRcyS8ki/pcOe0USK8Rj39j0AHY4CQzvWpU/uPdRPP0mERzlX9JSju57zwK91FH49QY/8ouJMOGWHdB0B13oJudKkeTWMY300TnsX0dZnofeLhAlN3Ue2sRnes1TeCaCS0iw/S0en9N0/8nGBpTG1dwnNkYOQqSQxLawSqnsII1cwzw+Tm3jgQNrGu7ZZ5vrglrWwnD47el55WWg/F85xOOmLQQiTYoph3V7/HKD0KK4ge2mdEs/CCM+zxdXVtwO4sw7mf9pKi3+g/11PlliJ6d6Zl0miOUky7BdPE19S8frZ19pofiIKLrtZmnI6PX0IkSEyQvbph6DUci5OIM7AzBe/hdhhs2YLcV4NkT5llSJf9Fbd5bFOUmtf9+sxNXZRErqf0WJb5j32ceJlDDeETYyIf7uuAaQ==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH8PR12MB7326.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Ym4vblpDdEwxdktmcUQ4aDVrcUZyY2wxSEpDSis0R1dkTEhOUms3bmxya3p1?=
 =?utf-8?B?QXJ3cjJsWGowMzF1TmpVS0pvZWNBRCtTUEdFWFBQK2tsUllQektJRUQ0cmpz?=
 =?utf-8?B?ZVhMYklBRmtOem5TVUpScjFmVDZmeWdJNXV2NytacWhlWndNaGNxdkJna0pQ?=
 =?utf-8?B?TFpia0hIS2s3Nlp1NWxzZFluL056NXlzVUQxOGtHbTdXZThFanEwcXZ6TE9h?=
 =?utf-8?B?RjRtN0lhbUJkTjQ4azVIZ1JpcjJHMDRyKzNaRXhpL3Q3N3FCLzRHMEJlVm82?=
 =?utf-8?B?Q3MvSXYxaTRMRkt3eHNFSGkyNk1GUWpENHplbHZyRHp1aGVGK1dFaC9MY3ZZ?=
 =?utf-8?B?WWk0enIwNGRzT0FTL3BkaVZrdmRtd2l6SHVaU2plRmtkV01zMExZMmI5emlt?=
 =?utf-8?B?YUxKRWwvOExZbXZVQ0lrOEMydWVJdFV4czRabElBaXJxbnA1R3c1dHlXMUFZ?=
 =?utf-8?B?cHVLS1FTNmkvWkMyM1dOVHpMUWN5YWtCdHZ2KzJtS1E0V2dteEoxTm5xUk5I?=
 =?utf-8?B?QWdUam02TnhDcDBlQkhVWmFXcEpEVHg2SFN4cUhIMHdwWks3bkZ0S093SE5x?=
 =?utf-8?B?Ni9BTUpkTjdncURJUGFNcE9BekVpVHJpcVBqVDJjQkF5VmFiY3RZcTNJSm5t?=
 =?utf-8?B?aEVyMVVUZUpzcDRmZ214UWRGU0MvUUl3VTVzL0JmN2Jqa3QzUW5JOW16NW5v?=
 =?utf-8?B?bUNqNGltNHRaQUdHQ2Fsbzd2YmpYNGthMWIzcWNZV3kvWldqeGpnejBvWGJH?=
 =?utf-8?B?b09XV3c5MC9Hekt0cmFmaUVObEVPMks3cUNickkvSzlwTW5GZDltWVZjcG9i?=
 =?utf-8?B?aC8wbnNycmw5K3k3Tk9OM3Z5MDVSZ2psV0pTVEo0VlNzZkdBRE9ORC9ZdVNr?=
 =?utf-8?B?SU84TnVVNjZPT2NBRS9PUUh4L3NZSXhwajRTQnFmbHN3NC9BeWNCMkJzbnlC?=
 =?utf-8?B?VklhMkhuVnc0eHZGMFRkUWc2OWpXWUV0WDVodk85NGZIZXEzVWVLSGJuTkxk?=
 =?utf-8?B?NUxYVldYQXFvcjBCNXh2ckI4QXVSWEF0dE0xUFhEanZubVNHTFoxSENoSURq?=
 =?utf-8?B?TkFTL2N2VEV3U1RBQzJQQ28xTzhDY0NyTStiancycm5tQUNiQ1VLd2djRnlr?=
 =?utf-8?B?U3I2OEs0NU5EZktlditFajZTUHdOZEhFUlAzOXNBbzJyYkNXSSs2bll6THQw?=
 =?utf-8?B?dzBSekg3QmVuOU42Nll4K1FtdTF1L00rZGR5WGJvSGh0UXpNOGpSVVVjOTR2?=
 =?utf-8?B?bEFRSWZSaUNBYjNyYThNRkRiVUptMHZsZUFHcWtubzExWlVDNHR0T2EvNnBk?=
 =?utf-8?B?cXJCTEl2WUQ0Z2pMZ1J0NUJJUU1ZUFVwSk9nUkFYVGcvUGg1NHUwN041TEgy?=
 =?utf-8?B?RnlKbWRiczdmclU5U3NVV1NqVWxEUzlIQi9jSW9rNDRwMjcwTURwaUV2Yk1m?=
 =?utf-8?B?NEJtakhDbnV3Z0VUWWE1WVg3djZ6b3lvdFppbzJxRjIvQ0dmNlVtLzlDYnF0?=
 =?utf-8?B?UmlWSU82R1NkL25hdzJZR1VLWElTVHdmWkQyV3ozTGh2Y0cwUmY1S08xSE5u?=
 =?utf-8?B?NDl4bEh3Y0c1Sk9WQTU3T1dmL0E3ZFFFVU93WXpxV0Z0c0NmdlpzS1FGRnRD?=
 =?utf-8?B?cC9GamRVRnZpbm1jM2xPUnhwZmNLRVNnSHlTUE9OTVpLNmdkbEpaUjlTdWZt?=
 =?utf-8?B?TEFWSUc2MVc0TGVPK0NCZE5iN0ZNcEhLZ1pTMmZWdXFpRGxzUEF6UGJXdzNo?=
 =?utf-8?B?azNQT05jK1JON0h6VGp4cndrcXVPVG04eHFCSjFzQjZSdWZkdU9UYitqSjEr?=
 =?utf-8?B?MVhodU5NeVp3b1dxYXE3cDhSZnRGek1udGUzSURib1QrWHdIMzUxZ3gzVWgw?=
 =?utf-8?B?dERNbFFBQmZXVWsrQm5WVDBDOEJsclJnZW5ZWWNYOWZWZVVyZGRLT0J1ZkRP?=
 =?utf-8?B?ejVQc29GMGs0b1ZqSUFQZVgrQStNQ0FNMkpzLzdXem4zMkx3WTFhOWNTdkFL?=
 =?utf-8?B?T2ZoNVd0S0tEZGhiejlkRWFiTFZZaTRhM2FtU0hueC9ndFowODNHMHBSTEpP?=
 =?utf-8?B?WVlDUDMrZnlXbWJJYmZPb3pjSmI3cUkxWHZHQVplb2FVQkhwdlhJOUVyWTQ0?=
 =?utf-8?Q?t7SZpChwNo4EYZjv8P7H4+6jo?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 62f5b541-495b-4c48-1729-08dc31596bde
X-MS-Exchange-CrossTenant-AuthSource: PH8PR12MB7326.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Feb 2024 14:45:31.4163
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oAJiksa18m0mmhbXFGVFJfBLKbWZkvCLTC+UCKtdAgg3mkiB6AKyzNhGth6RDwo8+Ar/bnSQNt+t85jTMRDVAA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4246

--------------j04MiwJlrRLEF3xvh8APOAUO
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit


On 06/02/2024 19:05, Julien Grall wrote:
> Hi Ayan,
Hi Julien/Michal,
>
> On 31/01/2024 12:10, Ayan Kumar Halder wrote:
>> From: Michal Orzel <michal.orzel@amd.com>
>>
>> Currently, if user enables HVC_DCC config option in Linux, it invokes 
>> access
>> to debug data transfer registers (i.e. DBGDTRTX_EL0 on arm64, 
>> DBGDTRTXINT on
>> arm32). As these registers are not emulated, Xen injects an undefined
>> exception to the guest and Linux crashes.
>>
>> To prevent this crash, introduce a partial emulation of DBGDTR[TR]X_EL0
>> (these registers share the same encoding) as RAZ/WI and MDCCSR_EL0 as 
>> TXfull.
>>
>> Refer ARM DDI 0487J.a ID042523, D19.3.8, DBGDTRTX_EL0
>> "If TXfull is set to 1, set DTRRX and DTRTX to UNKNOWN".
>>
>> Thus, any OS is expected to read MDCCSR_EL0 and check for TXfull before
>> using DBGDTRTX_EL0. Linux does it via hvc_dcc_init() ---> 
>> hvc_dcc_check(),
>> and returns -ENODEV in case TXfull bit is still set after writing a test
>> character. This way we prevent the guest from making use of HVC DCC as a
>> console.
>>
>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
>> ---
>> Changes from
>>
>> v1 :- 1. DBGDTR_EL0 does not emulate RXfull. This is to avoid giving 
>> the OS any
>> indication that the RX buffer is full and is waiting to be read.
>>
>> 2. In Arm32, DBGOSLSR is emulated. Also DBGDTRTXINT is emulated at 
>> EL0 only.
>>
>> 3. Fixed the commit message and inline code comments.
>>
>> v2 :- 1. Split the patch into two (separate patches for arm64 and 
>> arm32).
>> 2. Removed the "fail" label.
>> 3. Fixed the commit message.
>>
>> v3 :- 1. "HSR_SYSREG_MDCCSR_EL0" emulation differs based on whether
>> partial_emulation_enabled is true or not.
>>
>> 2. If partial_emulation_enabled is false, then access to 
>> HSR_SYSREG_DBGDTR_EL0,
>> HSR_SYSREG_DBGDTRTX_EL0 would lead to undefined exception.
>>
>>   xen/arch/arm/arm64/vsysreg.c         | 28 ++++++++++++++++++++++++----
>>   xen/arch/arm/include/asm/arm64/hsr.h |  3 +++
>>   2 files changed, 27 insertions(+), 4 deletions(-)
>>
>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>> index b5d54c569b..94f0a6c384 100644
>> --- a/xen/arch/arm/arm64/vsysreg.c
>> +++ b/xen/arch/arm/arm64/vsysreg.c
>> @@ -159,9 +159,6 @@ void do_sysreg(struct cpu_user_regs *regs,
>>        *
>>        * Unhandled:
>>        *    MDCCINT_EL1
>> -     *    DBGDTR_EL0
>> -     *    DBGDTRRX_EL0
>> -     *    DBGDTRTX_EL0
>>        *    OSDTRRX_EL1
>>        *    OSDTRTX_EL1
>>        *    OSECCR_EL1
>> @@ -173,10 +170,32 @@ void do_sysreg(struct cpu_user_regs *regs,
>>           return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
>>       case HSR_SYSREG_MDCCSR_EL0:
>>           /*
>> +         * Xen doesn't expose a real (or emulated) Debug 
>> Communications Channel
>> +         * (DCC) to a domain. Yet the Arm ARM implies this is not an 
>> optional
>> +         * feature. So some domains may start to probe it. For 
>> instance, the
>> +         * HVC_DCC driver in Linux (since f377775dc083 and at least 
>> up to v6.7),
>> +         * will try to write some characters and check if the 
>> transmit buffer
>> +         * has emptied.
>> +         *
>> +         * By setting TX status bit (only if partial emulation is 
>> enabled) to
>> +         * indicate the transmit buffer is full, we would hint the 
>> OS that the
>> +         * DCC is probably not working.
>> +         *
>> +         * Bit 29: TX full
>> +         *
>>            * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We 
>> emulate that
>>            * register as RAZ/WI above. So RO at both EL0 and EL1.
>
> The sentence "we emulate that register as ..." seems to be stale?
>
>>            */
>> -        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 0);
>> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, 
>> hsr, 0,
>> +                                  partial_emulation ? (1U << 29) : 0);
>> +
>> +    case HSR_SYSREG_DBGDTR_EL0:
>> +    /* DBGDTR[TR]X_EL0 share the same encoding */
>> +    case HSR_SYSREG_DBGDTRTX_EL0:
>> +        if ( !partial_emulation )
>> +            goto fail;
>> +        return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 0);
>
> AFAICT, all the emulation helpers have an explanation why we are using 
> them. But here this is not the case. Can you add one?
This and..
>
>> +
>>       HSR_SYSREG_DBG_CASES(DBGBVR):
>>       HSR_SYSREG_DBG_CASES(DBGBCR):
>>       HSR_SYSREG_DBG_CASES(DBGWVR):
>> @@ -394,6 +413,7 @@ void do_sysreg(struct cpu_user_regs *regs,
>>        * And all other unknown registers.
>>        */
>>       default:
>> + fail:
>
> AFAICT, this would violate MISRA 15.3 [1]. We didn't seem to have yet 
> (?) accepted the rule, but I don't see we would not given I feel this 
> is similar to what Rule 16.2 is trying to prevent and we accepted it.
>
> I think case, I move all the code within default outside. And then 
> call "goto fail" from the default label.

I am not sure if I have interpreted this correctly.

Is it ok if you can take a look at the attached patch and let me know if 
the explaination and the code change looks sane ?

- Ayan

>
>>           {
>>               const struct hsr_sysreg sysreg = hsr.sysreg;
>>   diff --git a/xen/arch/arm/include/asm/arm64/hsr.h 
>> b/xen/arch/arm/include/asm/arm64/hsr.h
>> index e691d41c17..1495ccddea 100644
>> --- a/xen/arch/arm/include/asm/arm64/hsr.h
>> +++ b/xen/arch/arm/include/asm/arm64/hsr.h
>> @@ -47,6 +47,9 @@
>>   #define HSR_SYSREG_OSDLR_EL1      HSR_SYSREG(2,0,c1,c3,4)
>>   #define HSR_SYSREG_DBGPRCR_EL1    HSR_SYSREG(2,0,c1,c4,4)
>>   #define HSR_SYSREG_MDCCSR_EL0     HSR_SYSREG(2,3,c0,c1,0)
>> +#define HSR_SYSREG_DBGDTR_EL0     HSR_SYSREG(2,3,c0,c4,0)
>> +#define HSR_SYSREG_DBGDTRTX_EL0   HSR_SYSREG(2,3,c0,c5,0)
>> +#define HSR_SYSREG_DBGDTRRX_EL0   HSR_SYSREG(2,3,c0,c5,0)
>>     #define HSR_SYSREG_DBGBVRn_EL1(n) HSR_SYSREG(2,0,c0,c##n,4)
>>   #define HSR_SYSREG_DBGBCRn_EL1(n) HSR_SYSREG(2,0,c0,c##n,5)
>
> Cheers,
>
> [1] 
> https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_15_03.c
> [2] 
> https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_16_02.c
>
--------------j04MiwJlrRLEF3xvh8APOAUO
Content-Type: text/plain; charset=UTF-8;
 name="v5-0002-xen-arm-arm64-Add-emulation-of-Debug-Data-Transfe.patch"
Content-Disposition: attachment;
 filename*0="v5-0002-xen-arm-arm64-Add-emulation-of-Debug-Data-Transfe.pa";
 filename*1="tch"
Content-Transfer-Encoding: base64

RnJvbSA3NjllZmM5MDNlZWQxODgzOTY0MWEwMDNhYTYzY2U2NGRlMjdiYjVmIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBNaWNoYWwgT3J6ZWwgPG1pY2hhbC5vcnplbEBhbWQuY29tPgpE
YXRlOiBXZWQsIDMgSmFuIDIwMjQgMTQ6NDQ6MjIgKzAwMDAKU3ViamVjdDogW1hFTiB2NSAyLzNd
IHhlbi9hcm06IGFybTY0OiBBZGQgZW11bGF0aW9uIG9mIERlYnVnIERhdGEgVHJhbnNmZXIKIFJl
Z2lzdGVycwpUbzogeGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnCkNjOiBzc3RhYmVsbGlu
aUBrZXJuZWwub3JnLAogICAgc3RlZmFuby5zdGFiZWxsaW5pQGFtZC5jb20sCiAgICBqdWxpZW5A
eGVuLm9yZywKICAgIFZvbG9keW15cl9CYWJjaHVrQGVwYW0uY29tLAogICAgYmVydHJhbmQubWFy
cXVpc0Bhcm0uY29tLAogICAgbWljaGFsLm9yemVsQGFtZC5jb20KCkN1cnJlbnRseSwgaWYgdXNl
ciBlbmFibGVzIEhWQ19EQ0MgY29uZmlnIG9wdGlvbiBpbiBMaW51eCwgaXQgaW52b2tlcyBhY2Nl
c3MKdG8gZGVidWcgZGF0YSB0cmFuc2ZlciByZWdpc3RlcnMgKGkuZS4gREJHRFRSVFhfRUwwIG9u
IGFybTY0LCBEQkdEVFJUWElOVCBvbgphcm0zMikuIEFzIHRoZXNlIHJlZ2lzdGVycyBhcmUgbm90
IGVtdWxhdGVkLCBYZW4gaW5qZWN0cyBhbiB1bmRlZmluZWQKZXhjZXB0aW9uIHRvIHRoZSBndWVz
dCBhbmQgTGludXggY3Jhc2hlcy4KClRvIHByZXZlbnQgdGhpcyBjcmFzaCwgaW50cm9kdWNlIGEg
cGFydGlhbCBlbXVsYXRpb24gb2YgREJHRFRSW1RSXVhfRUwwCih0aGVzZSByZWdpc3RlcnMgc2hh
cmUgdGhlIHNhbWUgZW5jb2RpbmcpIGFzIFJBWi9XSSBhbmQgTURDQ1NSX0VMMCBhcyBUWGZ1bGwu
CgpSZWZlciBBUk0gRERJIDA0ODdKLmEgSUQwNDI1MjMsIEQxOS4zLjgsIERCR0RUUlRYX0VMMAoi
SWYgVFhmdWxsIGlzIHNldCB0byAxLCBzZXQgRFRSUlggYW5kIERUUlRYIHRvIFVOS05PV04iLgoK
VGh1cywgYW55IE9TIGlzIGV4cGVjdGVkIHRvIHJlYWQgTURDQ1NSX0VMMCBhbmQgY2hlY2sgZm9y
IFRYZnVsbCBiZWZvcmUKdXNpbmcgREJHRFRSVFhfRUwwLiBMaW51eCBkb2VzIGl0IHZpYSBodmNf
ZGNjX2luaXQoKSAtLS0+IGh2Y19kY2NfY2hlY2soKSwKYW5kIHJldHVybnMgLUVOT0RFViBpbiBj
YXNlIFRYZnVsbCBiaXQgaXMgc3RpbGwgc2V0IGFmdGVyIHdyaXRpbmcgYSB0ZXN0CmNoYXJhY3Rl
ci4gVGhpcyB3YXkgd2UgcHJldmVudCB0aGUgZ3Vlc3QgZnJvbSBtYWtpbmcgdXNlIG9mIEhWQyBE
Q0MgYXMgYQpjb25zb2xlLgoKU2lnbmVkLW9mZi1ieTogTWljaGFsIE9yemVsIDxtaWNoYWwub3J6
ZWxAYW1kLmNvbT4KU2lnbmVkLW9mZi1ieTogQXlhbiBLdW1hciBIYWxkZXIgPGF5YW4ua3VtYXIu
aGFsZGVyQGFtZC5jb20+ClJldmlld2VkLWJ5OiBNaWNoYWwgT3J6ZWwgPG1pY2hhbC5vcnplbEBh
bWQuY29tPgotLS0KIHhlbi9hcmNoL2FybS9hcm02NC92c3lzcmVnLmMgICAgICAgICB8IDY5ICsr
KysrKysrKysrKysrKysrKystLS0tLS0tLS0KIHhlbi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02
NC9oc3IuaCB8ICAzICsrCiAyIGZpbGVzIGNoYW5nZWQsIDUwIGluc2VydGlvbnMoKyksIDIyIGRl
bGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9hcm02NC92c3lzcmVnLmMgYi94
ZW4vYXJjaC9hcm0vYXJtNjQvdnN5c3JlZy5jCmluZGV4IGI1ZDU0YzU2OWIuLjgxM2I0MDQyNWQg
MTAwNjQ0Ci0tLSBhL3hlbi9hcmNoL2FybS9hcm02NC92c3lzcmVnLmMKKysrIGIveGVuL2FyY2gv
YXJtL2FybTY0L3ZzeXNyZWcuYwpAQCAtMTU5LDkgKzE1OSw2IEBAIHZvaWQgZG9fc3lzcmVnKHN0
cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzLAogICAgICAqCiAgICAgICogVW5oYW5kbGVkOgogICAg
ICAqICAgIE1EQ0NJTlRfRUwxCi0gICAgICogICAgREJHRFRSX0VMMAotICAgICAqICAgIERCR0RU
UlJYX0VMMAotICAgICAqICAgIERCR0RUUlRYX0VMMAogICAgICAqICAgIE9TRFRSUlhfRUwxCiAg
ICAgICogICAgT1NEVFJUWF9FTDEKICAgICAgKiAgICBPU0VDQ1JfRUwxCkBAIC0xNzMsMTAgKzE3
MCw0MCBAQCB2b2lkIGRvX3N5c3JlZyhzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywKICAgICAg
ICAgcmV0dXJuIGhhbmRsZV9yYXpfd2kocmVncywgcmVnaWR4LCBoc3Iuc3lzcmVnLnJlYWQsIGhz
ciwgMSk7CiAgICAgY2FzZSBIU1JfU1lTUkVHX01EQ0NTUl9FTDA6CiAgICAgICAgIC8qCisgICAg
ICAgICAqIFhlbiBkb2Vzbid0IGV4cG9zZSBhIHJlYWwgKG9yIGVtdWxhdGVkKSBEZWJ1ZyBDb21t
dW5pY2F0aW9ucyBDaGFubmVsCisgICAgICAgICAqIChEQ0MpIHRvIGEgZG9tYWluLiBZZXQgdGhl
IEFybSBBUk0gaW1wbGllcyB0aGlzIGlzIG5vdCBhbiBvcHRpb25hbAorICAgICAgICAgKiBmZWF0
dXJlLiBTbyBzb21lIGRvbWFpbnMgbWF5IHN0YXJ0IHRvIHByb2JlIGl0LiBGb3IgaW5zdGFuY2Us
IHRoZQorICAgICAgICAgKiBIVkNfRENDIGRyaXZlciBpbiBMaW51eCAoc2luY2UgZjM3Nzc3NWRj
MDgzIGFuZCBhdCBsZWFzdCB1cCB0byB2Ni43KSwKKyAgICAgICAgICogd2lsbCB0cnkgdG8gd3Jp
dGUgc29tZSBjaGFyYWN0ZXJzIGFuZCBjaGVjayBpZiB0aGUgdHJhbnNtaXQgYnVmZmVyCisgICAg
ICAgICAqIGhhcyBlbXB0aWVkLgorICAgICAgICAgKgorICAgICAgICAgKiBCeSBzZXR0aW5nIFRY
IHN0YXR1cyBiaXQgKG9ubHkgaWYgcGFydGlhbCBlbXVsYXRpb24gaXMgZW5hYmxlZCkgdG8KKyAg
ICAgICAgICogaW5kaWNhdGUgdGhlIHRyYW5zbWl0IGJ1ZmZlciBpcyBmdWxsLCB3ZSB3b3VsZCBo
aW50IHRoZSBPUyB0aGF0IHRoZQorICAgICAgICAgKiBEQ0MgaXMgcHJvYmFibHkgbm90IHdvcmtp
bmcuCisgICAgICAgICAqCisgICAgICAgICAqIEJpdCAyOTogVFggZnVsbAorICAgICAgICAgKgog
ICAgICAgICAgKiBBY2Nlc3NpYmxlIGF0IEVMMCBvbmx5IGlmIE1EU0NSX0VMMS5URENDIGlzIHNl
dCB0byAwLiBXZSBlbXVsYXRlIHRoYXQKLSAgICAgICAgICogcmVnaXN0ZXIgYXMgUkFaL1dJIGFi
b3ZlLiBTbyBSTyBhdCBib3RoIEVMMCBhbmQgRUwxLgorICAgICAgICAgKiByZWdpc3RlciBhcyBS
TyBhdCBFTDAgYW5kIGFib3ZlLgogICAgICAgICAgKi8KLSAgICAgICAgcmV0dXJuIGhhbmRsZV9y
b19yYXoocmVncywgcmVnaWR4LCBoc3Iuc3lzcmVnLnJlYWQsIGhzciwgMCk7CisgICAgICAgIHJl
dHVybiBoYW5kbGVfcm9fcmVhZF92YWwocmVncywgcmVnaWR4LCBoc3Iuc3lzcmVnLnJlYWQsIGhz
ciwgMCwKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYXJ0aWFsX2VtdWxhdGlv
biA/ICgxVSA8PCAyOSkgOiAwKTsKKworICAgIGNhc2UgSFNSX1NZU1JFR19EQkdEVFJfRUwwOgor
ICAgIC8qIERCR0RUUltUUl1YX0VMMCBzaGFyZSB0aGUgc2FtZSBlbmNvZGluZyAqLworICAgIGNh
c2UgSFNSX1NZU1JFR19EQkdEVFJUWF9FTDA6CisgICAgICAgIC8qCisgICAgICAgICAqIEFzIHN0
YXRlZCBiZWZvcmUsIFhlbiBkb2VzIG5vdCBzdXBwb3J0IGZ1bGwgZW11bGF0aW9uIG9mIERlYnVn
CisgICAgICAgICAqIENvbW11bmljYXRpb25zIENoYW5uZWwgKERDQykuIFRodXMsIGlmIFhlbiBk
b2VzIG5vdCBzdXBwb3J0IHBhcnRpYWwKKyAgICAgICAgICogZW11bGF0aW9uIGFuZCBhY2Nlc3Mg
dG8gREJHRFRSVFggaXMgdHJhcHBlZCwgaXQgd2lsbCBpbmplY3QgYW4gdW5kZWYKKyAgICAgICAg
ICogZXhjZXB0aW9uLiBPdGhlcndpc2UsIFhlbiBlbXVsYXRlcyB0aGlzIGFzIFJBWi9XSSBhcyBp
dCBlbXVsYXRlcyB0aGUKKyAgICAgICAgICogVFggc3RhdHVzIGFzIGZ1bGwuIFRodXMsIGd1ZXN0
cyBhcmUgbm90IGV4cGVjdGVkIHRvIHVzZSBEQkdEVFJUWCBmb3IKKyAgICAgICAgICogcmVhZGlu
Zy93cml0aW5nLgorICAgICAgICAgKi8KKyAgICAgICAgaWYgKCAhcGFydGlhbF9lbXVsYXRpb24g
KQorICAgICAgICAgICAgZ290byBmYWlsOworICAgICAgICByZXR1cm4gaGFuZGxlX3Jhel93aShy
ZWdzLCByZWdpZHgsIGhzci5zeXNyZWcucmVhZCwgaHNyLCAwKTsKKwogICAgIEhTUl9TWVNSRUdf
REJHX0NBU0VTKERCR0JWUik6CiAgICAgSFNSX1NZU1JFR19EQkdfQ0FTRVMoREJHQkNSKToKICAg
ICBIU1JfU1lTUkVHX0RCR19DQVNFUyhEQkdXVlIpOgpAQCAtMzk0LDIzICs0MjEsMjEgQEAgdm9p
ZCBkb19zeXNyZWcoc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsCiAgICAgICogQW5kIGFsbCBv
dGhlciB1bmtub3duIHJlZ2lzdGVycy4KICAgICAgKi8KICAgICBkZWZhdWx0OgotICAgICAgICB7
Ci0gICAgICAgICAgICBjb25zdCBzdHJ1Y3QgaHNyX3N5c3JlZyBzeXNyZWcgPSBoc3Iuc3lzcmVn
OwotCi0gICAgICAgICAgICBnZHByaW50ayhYRU5MT0dfRVJSLAotICAgICAgICAgICAgICAgICAg
ICAgIiVzICVkLCAlZCwgYyVkLCBjJWQsICVkICVzIHglZCBAIDB4JSJQUklyZWdpc3RlciJcbiIs
Ci0gICAgICAgICAgICAgICAgICAgICBzeXNyZWcucmVhZCA/ICJtcnMiIDogIm1zciIsCi0gICAg
ICAgICAgICAgICAgICAgICBzeXNyZWcub3AwLCBzeXNyZWcub3AxLAotICAgICAgICAgICAgICAg
ICAgICAgc3lzcmVnLmNybiwgc3lzcmVnLmNybSwKLSAgICAgICAgICAgICAgICAgICAgIHN5c3Jl
Zy5vcDIsCi0gICAgICAgICAgICAgICAgICAgICBzeXNyZWcucmVhZCA/ICI9PiIgOiAiPD0iLAot
ICAgICAgICAgICAgICAgICAgICAgc3lzcmVnLnJlZywgcmVncy0+cGMpOwotICAgICAgICAgICAg
Z2RwcmludGsoWEVOTE9HX0VSUiwKLSAgICAgICAgICAgICAgICAgICAgICJ1bmhhbmRsZWQgNjQt
Yml0IHN5c3JlZyBhY2Nlc3MgJSMiUFJJcmVnaXN0ZXIiXG4iLAotICAgICAgICAgICAgICAgICAg
ICAgaHNyLmJpdHMgJiBIU1JfU1lTUkVHX1JFR1NfTUFTSyk7Ci0gICAgICAgICAgICBpbmplY3Rf
dW5kZWZfZXhjZXB0aW9uKHJlZ3MsIGhzcik7Ci0gICAgICAgICAgICByZXR1cm47Ci0gICAgICAg
IH0KKyAgICAgICAgZ290byBmYWlsOworIGZhaWw6CisKKyAgICBjb25zdCBzdHJ1Y3QgaHNyX3N5
c3JlZyBzeXNyZWcgPSBoc3Iuc3lzcmVnOworCisgICAgZ2RwcmludGsoWEVOTE9HX0VSUiwKKyAg
ICAgICAgICAgICAiJXMgJWQsICVkLCBjJWQsIGMlZCwgJWQgJXMgeCVkIEAgMHglIlBSSXJlZ2lz
dGVyIlxuIiwKKyAgICAgICAgICAgICBzeXNyZWcucmVhZCA/ICJtcnMiIDogIm1zciIsIHN5c3Jl
Zy5vcDAsIHN5c3JlZy5vcDEsIHN5c3JlZy5jcm4sCisgICAgICAgICAgICAgc3lzcmVnLmNybSwg
c3lzcmVnLm9wMiwgc3lzcmVnLnJlYWQgPyAiPT4iIDogIjw9Iiwgc3lzcmVnLnJlZywKKyAgICAg
ICAgICAgICByZWdzLT5wYyk7CisgICAgZ2RwcmludGsoWEVOTE9HX0VSUiwKKyAgICAgICAgICAg
ICAidW5oYW5kbGVkIDY0LWJpdCBzeXNyZWcgYWNjZXNzICUjIlBSSXJlZ2lzdGVyIlxuIiwKKyAg
ICAgICAgICAgICBoc3IuYml0cyAmIEhTUl9TWVNSRUdfUkVHU19NQVNLKTsKKyAgICBpbmplY3Rf
dW5kZWZfZXhjZXB0aW9uKHJlZ3MsIGhzcik7CisgICAgcmV0dXJuOwogICAgIH0KIAogICAgIHJl
Z3MtPnBjICs9IDQ7CmRpZmYgLS1naXQgYS94ZW4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQv
aHNyLmggYi94ZW4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvaHNyLmgKaW5kZXggZTY5MWQ0
MWMxNy4uMTQ5NWNjZGRlYSAxMDA2NDQKLS0tIGEveGVuL2FyY2gvYXJtL2luY2x1ZGUvYXNtL2Fy
bTY0L2hzci5oCisrKyBiL3hlbi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9oc3IuaApAQCAt
NDcsNiArNDcsOSBAQAogI2RlZmluZSBIU1JfU1lTUkVHX09TRExSX0VMMSAgICAgIEhTUl9TWVNS
RUcoMiwwLGMxLGMzLDQpCiAjZGVmaW5lIEhTUl9TWVNSRUdfREJHUFJDUl9FTDEgICAgSFNSX1NZ
U1JFRygyLDAsYzEsYzQsNCkKICNkZWZpbmUgSFNSX1NZU1JFR19NRENDU1JfRUwwICAgICBIU1Jf
U1lTUkVHKDIsMyxjMCxjMSwwKQorI2RlZmluZSBIU1JfU1lTUkVHX0RCR0RUUl9FTDAgICAgIEhT
Ul9TWVNSRUcoMiwzLGMwLGM0LDApCisjZGVmaW5lIEhTUl9TWVNSRUdfREJHRFRSVFhfRUwwICAg
SFNSX1NZU1JFRygyLDMsYzAsYzUsMCkKKyNkZWZpbmUgSFNSX1NZU1JFR19EQkdEVFJSWF9FTDAg
ICBIU1JfU1lTUkVHKDIsMyxjMCxjNSwwKQogCiAjZGVmaW5lIEhTUl9TWVNSRUdfREJHQlZSbl9F
TDEobikgSFNSX1NZU1JFRygyLDAsYzAsYyMjbiw0KQogI2RlZmluZSBIU1JfU1lTUkVHX0RCR0JD
Um5fRUwxKG4pIEhTUl9TWVNSRUcoMiwwLGMwLGMjI24sNSkKLS0gCjIuMjUuMQoK

--------------j04MiwJlrRLEF3xvh8APOAUO--


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 14:59:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 14:59:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682955.1062220 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc56y-0005Mm-7T; Mon, 19 Feb 2024 14:59:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682955.1062220; Mon, 19 Feb 2024 14:59:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc56y-0005Mf-4Z; Mon, 19 Feb 2024 14:59:16 +0000
Received: by outflank-mailman (input) for mailman id 682955;
 Mon, 19 Feb 2024 14:59:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NiMn=J4=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rc56w-0005MN-Iz
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 14:59:14 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7078a391-cf37-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 15:59:11 +0100 (CET)
Received: from [192.168.1.229] (93-36-216-194.ip62.fastwebnet.it
 [93.36.216.194])
 by support.bugseng.com (Postfix) with ESMTPSA id 916FA4EE073A;
 Mon, 19 Feb 2024 15:59:10 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7078a391-cf37-11ee-98f5-efadbce2ee36
Message-ID: <f28aed5e-0983-48fa-828f-b2133ca35086@bugseng.com>
Date: Mon, 19 Feb 2024 15:59:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair: add deviation for MISRA C:2012
 Rule 16.3
Content-Language: en-US, it
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <b1d2b64c8117d61ea42cf4e9feae128541eb0b61.1708348799.git.federico.serafini@bugseng.com>
 <a6e8bd32-cc32-4084-907d-e2cd6d46e3e6@suse.com>
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <a6e8bd32-cc32-4084-907d-e2cd6d46e3e6@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 19/02/24 14:43, Jan Beulich wrote:
> On 19.02.2024 14:24, Federico Serafini wrote:
>> Update ECLAIR configuration to consider safe switch clauses ending
>> with __{get,put}_user_bad().
>>
>> Update docs/misra/deviations.rst accordingly.
>>
>> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
> 
> As mentioned I'm not happy with this, not the least because of it being
> unclear why these two would be deviated, when there's no sign of a
> similar deviation for, say, __bad_atomic_size(). Imo this approach
> doesn't scale, and that's already leaving aside that the purpose of
> identically named (pseudo-)helpers could differ between architectures,
> thus putting under question ...
> 
>> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
>> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
>> @@ -368,6 +368,10 @@ safe."
>>   -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/BUG\\(\\);/))))"}
>>   -doc_end
>>   
>> +-doc_begin="Switch clauses ending with constructs \"__get_user_bad()\" and \"__put_user_bad()\" are safe: they denote an unreachable program point."
>> +-config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/__(put|get)_user_bad\\(\\);/))))"}
>> +-doc_end
> 
> ... the global scope of such a deviation. While it may not be a good idea,
> even within an arch such (pseudo-)helpers could be used for multiple
> distinct purposes.

Would you agree with adding the missing break statement after
the uses of __{put,get}_user_bad() (as it is already happening for
uses of __bad_atomic_size())?

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 15:01:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 15:01:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682958.1062230 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc58p-0007EJ-Jm; Mon, 19 Feb 2024 15:01:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682958.1062230; Mon, 19 Feb 2024 15:01:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc58p-0007EB-Ft; Mon, 19 Feb 2024 15:01:11 +0000
Received: by outflank-mailman (input) for mailman id 682958;
 Mon, 19 Feb 2024 15:01:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc58o-0007Dr-9u
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 15:01:10 +0000
Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com
 [2a00:1450:4864:20::42d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b40db1df-cf37-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 16:01:04 +0100 (CET)
Received: by mail-wr1-x42d.google.com with SMTP id
 ffacd0b85a97d-33d509ab80eso431193f8f.3
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 07:01:04 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bo29-20020a056000069d00b0033d5f5571b4sm1544190wrb.44.2024.02.19.07.01.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 07:01:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b40db1df-cf37-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708354864; x=1708959664; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=yMmVpS8COeMnbZvfmiiZ98a0W983xBw30lGDGRuDq3s=;
        b=Svhm895ZWqhEQjzqtwp1+s2VXz1E5AOcggKXsbfap4k+NVQm1Dy58bPmE5Cru4dOsW
         RDAgx7AOGjgposujkNkGX7CdZ4Zh8ocu3zXhgR7mLD60LoR/ozhnHFrMv5W8j5BhFxhy
         6TLN43CGDtxw1QCBWu8cFi2dAMyD2we1Upxa3eAkTjOdXA/gxh3pr6SbrUWkFFPeE89L
         EuGYwnXnyOgiILWEfBOSZhCXa2UUQLiRCuK+tsoeiccXCalSTpI3YiQ5pXN905ylus7S
         DU9xHQL1irUGq2ModI3AWOSnYomMRqyLRSTbkv3h3BfQOsHceUmSs1pI8kEZyBhV4Rtw
         HUww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708354864; x=1708959664;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=yMmVpS8COeMnbZvfmiiZ98a0W983xBw30lGDGRuDq3s=;
        b=Qh9HEQQaMaayyfoJhFozsaf6sA6+NNUEH3XqwIliOH3PqVkuPOPjQlLZznxb6We9Nw
         aV540ARJQ26YRMMM1ztUhNocquobcjZ7CtDnaIn1zrA7+4VWhEZfxgkC+a1MfY8yo5zN
         DdB4pb48g2f18I8M6xaS06Bap4qJzkJe2kD8QOsO4mM8PC2rQgKgcK6bW82CBdlFdQdq
         wcVejCcb7N5JKZ6SuZAD3dJZHypjwqnmdEFX9qynJ2pP3Uc4jZJ4iMlTKz5zEUzg5ZFO
         x1aKB4mOEvlhGzTpAXyR2yg8KtZcJvDo0gQSlBCa1gXMlOYTd1NlHT8uRM2cPdJtLMfT
         525A==
X-Forwarded-Encrypted: i=1; AJvYcCWpwkw1aG2QsyEAfyfAvHIUi7F4uAD1OLbRC04KEJ/tFyafRAj+xwHZeUsMFUC7ChJgPgUiHguTPHHuK9UxWWxKRXm1vEYoEnvOY7APAtY=
X-Gm-Message-State: AOJu0YxCDg47US+aKUyNGARMjWS7vzF3t1qf+7YK9myFBTpAX5MCO2XC
	MAYZh1HaMgTkT4GeJ6f2WvAiPw050l2myj3e7MWE/WMR1RcMJGXLHM5SeW3vEA==
X-Google-Smtp-Source: AGHT+IGhzsbfLNh7ZPzx4xeY6rwsIXkjr9lBN5dAGvnAaKjzy7gQEWbWbUMLzyJgpR+VssziScpyog==
X-Received: by 2002:adf:e751:0:b0:33d:19fc:c4e9 with SMTP id c17-20020adfe751000000b0033d19fcc4e9mr5701381wrn.50.1708354864147;
        Mon, 19 Feb 2024 07:01:04 -0800 (PST)
Message-ID: <3b38fe82-ee0f-4666-93e1-bd78fe69c534@suse.com>
Date: Mon, 19 Feb 2024 16:01:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
 <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
 <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
 <1c53b52ee39161a8f59209d28af69fe997479dbc.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <1c53b52ee39161a8f59209d28af69fe997479dbc.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 19.02.2024 15:29, Oleksii wrote:
> On Mon, 2024-02-19 at 12:22 +0100, Jan Beulich wrote:
>> On 15.02.2024 14:41, Oleksii wrote:
>>> As I mentioned above with 4-byte alignment and then reading and
>>> working
>>> with 8-byte then crossing a word or double-word boundary shouldn't
>>> be
>>> an issue.
>>>
>>> I am not sure that I know how to check that we are crossing cache
>>> line
>>> boundary.
>>>
>>> Regarding page boundary, if the next page is mapped then all should
>>> work fine, otherwise it will be an exception.
>>
>> Are you sure lr.d / sc.d are happy to access across such a boundary,
>> when both pages are mapped?
> If they are mapped, my expectation that lr.d and sc.d should be happy.

How does this expectation of yours fit with the A extension doc having
this:

"For LR and SC, the A extension requires that the address held in rs1 be
 naturally aligned to the size of the operand (i.e., eight-byte aligned
 for 64-bit words and four-byte aligned for 32-bit words). If the
 address is not naturally aligned, an address-misaligned exception or an
 access-fault exception will be generated."

It doesn't even say "may"; it says "will".

>> To me it seems pretty clear that for atomic accesses you want to
>> demand natural alignment, i.e. 2-byte alignment for 2-byte accesses.
>> This way you can be sure no potentially problematic boundaries will
>> be crossed.
> It makes sense, but I am not sure that I can guarantee that a user of
> macros will always have 2-byte alignment (except during a panic) in the
> future.
> 
> Even now, I am uncertain that everyone will be willing to add
> __alignment(...) to struct vcpu->is_urgent
> (xen/include/xen/sched.h:218) and other possible cases to accommodate
> RISC-V requirements.

->is_urgent is bool, i.e. 1 byte and hence okay at any address. For all
normal variables and fields the compiler will guarantee suitable
(natural) alignment. What you prohibit by requiring aligned items is
use of fields of e.g. packed structures.

>>> As 1- and 2-byte cases are emulated I decided that is not to
>>> provide
>>> sfx argument for emulation macros as it will not have to much
>>> affect on
>>> emulated types and just consume more performance on acquire and
>>> release
>>> version of sc/ld instructions.
>>
>> Question is whether the common case (4- and 8-byte accesses)
>> shouldn't
>> be valued higher, with 1- and 2-byte emulation being there just to
>> allow things to not break altogether.
> If I understand you correctly, it would make sense to add the 'sfx'
> argument for the 1/2-byte access case, ensuring that all options are
> available for 1/2-byte access case as well.

That's one of the possibilities. As said, I'm not overly worried about
the emulated cases. For the initial implementation I'd recommend going
with what is easiest there, yielding the best possible result for the
4- and 8-byte cases. If later it turns out repeated acquire/release
accesses are a problem in the emulation loop, things can be changed
to explicit barriers, without touching the 4- and 8-byte cases.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 15:06:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 15:06:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682965.1062240 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5E9-0008GA-4t; Mon, 19 Feb 2024 15:06:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682965.1062240; Mon, 19 Feb 2024 15:06:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5E9-0008G3-1w; Mon, 19 Feb 2024 15:06:41 +0000
Received: by outflank-mailman (input) for mailman id 682965;
 Mon, 19 Feb 2024 15:06:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc5E8-0008Fu-03
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 15:06:40 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7b3d7905-cf38-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 16:06:39 +0100 (CET)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-33d5ec64351so324878f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 07:06:39 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bt21-20020a056000081500b0033d1f25b798sm10322896wrb.82.2024.02.19.07.06.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 07:06:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b3d7905-cf38-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708355198; x=1708959998; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=D36QrkvcP0hG+KI2ygjRzaJwbdaS9gH77TGdUvrLq28=;
        b=cnXX9PQbJr/rC+xFVxWkvHrizb+Veycloet5r2cFvgGy23qSktc8EBTOZ9F9dLP2I0
         lIdPPQ4qYmMOF6v4AyxUYyjIePRefxXxecWsonYiwgoQ45hKBjLGa/cnAGx9jB64J4U7
         l48XZcXvaGxDGy0nAHpbEZ22k6YlqouP9bfp7R/HBzDuUzPgomSiq+2rAb3XxxZDjcAv
         PFlJxXxW8tav8VkgdroG0QprQ+MBp/p6IbuuPsENj2IOX4hL/plXHH6cPUlbDE02IoLO
         EnK5zs4/O2uOKv+/K5Hl1fi97QRDnpY/av78s5iiQaCAsM1tv60rd9dvVvqTrQR7Ut8i
         mXxA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708355198; x=1708959998;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=D36QrkvcP0hG+KI2ygjRzaJwbdaS9gH77TGdUvrLq28=;
        b=LqhYEzP29PxOJyPwzGuepk1E90UzsIpADcpKQpaTg7drIbBJJbljQw1RjC01MaYAoj
         XqgfyNgL7lqPY6C+AATuhy9Sym1mJE1VsHIj3BRnb+0Zc7Lpf4kbgUFlKRC8KN7bhN4k
         NKwmR6mgjZbWfvjEwIbf4oD4tIPHyIFtFqJPOO1NfNnnwulELMGdwAnRzOQhI5VyXiIp
         ktVuNdR9nC7PLROPkeiVW8wKlcAn3kDt0z/BbOTJNI/J6ySStBFUOEsIZE8cC+OmWXlg
         R9F0CnGVFnEl9rZwc9sQhAm9CZ4iGUQOPg9j4P/EO4JyETWLdSazANWU5yzTjT6YEkbk
         9Fmw==
X-Forwarded-Encrypted: i=1; AJvYcCU09G9Atza6+++CQVqJ8/JGe4j0zZgYc3HfhZ8ChBRB1eF+Wjx+nkKjGjynx8Tij7NftOV2jeIzKKsGXvRl7m5bjJLr+WLRwQkawvJGJVw=
X-Gm-Message-State: AOJu0YzP/0/zuwplxjbM4Bw5jY9ws+HGYeTmrbiM3LUTQyZhrFnB+0NS
	KemgWafI5j/QULU/YTWkzz6Q7xkNLhLvBSZuVrTfIAxxoBdStg0EUQ7HmdTcOA==
X-Google-Smtp-Source: AGHT+IHHWw9BowhEJnek4kopE4MGu37ZQlbkOR8Ls49s+4kE4iJMw6fYUxjtKd6/DivPuRHKNdYmGA==
X-Received: by 2002:adf:e80a:0:b0:33d:20ea:7a3a with SMTP id o10-20020adfe80a000000b0033d20ea7a3amr5199552wrm.27.1708355198263;
        Mon, 19 Feb 2024 07:06:38 -0800 (PST)
Message-ID: <4d2c5a69-4807-4af3-84c2-128d16ee2b43@suse.com>
Date: Mon, 19 Feb 2024 16:06:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair: add deviation for MISRA C:2012
 Rule 16.3
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <b1d2b64c8117d61ea42cf4e9feae128541eb0b61.1708348799.git.federico.serafini@bugseng.com>
 <a6e8bd32-cc32-4084-907d-e2cd6d46e3e6@suse.com>
 <f28aed5e-0983-48fa-828f-b2133ca35086@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f28aed5e-0983-48fa-828f-b2133ca35086@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 19.02.2024 15:59, Federico Serafini wrote:
> On 19/02/24 14:43, Jan Beulich wrote:
>> On 19.02.2024 14:24, Federico Serafini wrote:
>>> Update ECLAIR configuration to consider safe switch clauses ending
>>> with __{get,put}_user_bad().
>>>
>>> Update docs/misra/deviations.rst accordingly.
>>>
>>> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
>>
>> As mentioned I'm not happy with this, not the least because of it being
>> unclear why these two would be deviated, when there's no sign of a
>> similar deviation for, say, __bad_atomic_size(). Imo this approach
>> doesn't scale, and that's already leaving aside that the purpose of
>> identically named (pseudo-)helpers could differ between architectures,
>> thus putting under question ...
>>
>>> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
>>> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
>>> @@ -368,6 +368,10 @@ safe."
>>>   -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/BUG\\(\\);/))))"}
>>>   -doc_end
>>>   
>>> +-doc_begin="Switch clauses ending with constructs \"__get_user_bad()\" and \"__put_user_bad()\" are safe: they denote an unreachable program point."
>>> +-config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/__(put|get)_user_bad\\(\\);/))))"}
>>> +-doc_end
>>
>> ... the global scope of such a deviation. While it may not be a good idea,
>> even within an arch such (pseudo-)helpers could be used for multiple
>> distinct purposes.
> 
> Would you agree with adding the missing break statement after
> the uses of __{put,get}_user_bad() (as it is already happening for
> uses of __bad_atomic_size())?

I probably wouldn't mind that (despite being a little pointless).
Perhaps declaring them as noreturn would also help?

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 15:15:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 15:15:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682972.1062250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5MV-0002JF-UW; Mon, 19 Feb 2024 15:15:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682972.1062250; Mon, 19 Feb 2024 15:15:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5MV-0002J8-Rj; Mon, 19 Feb 2024 15:15:19 +0000
Received: by outflank-mailman (input) for mailman id 682972;
 Mon, 19 Feb 2024 15:15:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=3Bhz=J4=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rc5MV-0002Iz-3M
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 15:15:19 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id afc02101-cf39-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 16:15:18 +0100 (CET)
Received: from nico.bugseng.com (unknown [147.123.100.131])
 by support.bugseng.com (Postfix) with ESMTPSA id 7A6884EE073A;
 Mon, 19 Feb 2024 16:15:14 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: afc02101-cf39-11ee-8a52-1f161083a0e0
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	Nicola Vetrini <nicola.vetrini@bugseng.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH] xen: cache clearing and invalidation helpers refactoring
Date: Mon, 19 Feb 2024 16:14:52 +0100
Message-Id: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The cache clearing and invalidation helpers in x86 and Arm didn't
comply with MISRA C Rule 17.7: "The value returned by a function
having non-void return type shall be used". On Arm they
were always returning 0, while some in x86 returned -EOPNOTSUPP
and in common/grant_table the return value is saved.

As a consequence, a common helper arch_grant_cache_flush that returns
an integer is introduced, so that each architecture can choose whether to
return an error value on certain conditions, and the helpers have either
been changed to return void (on Arm) or deleted entirely (on x86).

Signed-off-by: Julien Grall <julien@xen.org>
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
The original refactor idea came from Julien Grall in [1]; I edited that proposal
to fix build errors.

I did introduce a cast to void for the call to flush_area_local on x86, because
even before this patch the return value of that function wasn't checked in all
but one use in x86/smp.c, and in this context the helper (perhaps incidentally)
ignored the return value of flush_area_local.

[1] https://lore.kernel.org/xen-devel/09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org/
---
 xen/arch/arm/include/asm/page.h     | 33 ++++++++++++++++++-----------
 xen/arch/x86/include/asm/flushtlb.h | 23 ++++++++++----------
 xen/common/grant_table.c            |  9 +-------
 3 files changed, 34 insertions(+), 31 deletions(-)

diff --git a/xen/arch/arm/include/asm/page.h b/xen/arch/arm/include/asm/page.h
index 69f817d1e68a..e90c9de3616e 100644
--- a/xen/arch/arm/include/asm/page.h
+++ b/xen/arch/arm/include/asm/page.h
@@ -123,6 +123,7 @@
 
 #ifndef __ASSEMBLY__
 
+#include <public/grant_table.h>
 #include <xen/errno.h>
 #include <xen/types.h>
 #include <xen/lib.h>
@@ -159,13 +160,13 @@ static inline size_t read_dcache_line_bytes(void)
  * if 'range' is large enough we might want to use model-specific
  * full-cache flushes. */
 
-static inline int invalidate_dcache_va_range(const void *p, unsigned long size)
+static inline void invalidate_dcache_va_range(const void *p, unsigned long size)
 {
     size_t cacheline_mask = dcache_line_bytes - 1;
     unsigned long idx = 0;
 
     if ( !size )
-        return 0;
+        return;
 
     /* Passing a region that wraps around is illegal */
     ASSERT(((uintptr_t)p + size - 1) >= (uintptr_t)p);
@@ -188,17 +189,15 @@ static inline int invalidate_dcache_va_range(const void *p, unsigned long size)
         asm volatile (__clean_and_invalidate_dcache_one(0) : : "r" (p + idx));
 
     dsb(sy);           /* So we know the flushes happen before continuing */
-
-    return 0;
 }
 
-static inline int clean_dcache_va_range(const void *p, unsigned long size)
+static inline void clean_dcache_va_range(const void *p, unsigned long size)
 {
     size_t cacheline_mask = dcache_line_bytes - 1;
     unsigned long idx = 0;
 
     if ( !size )
-        return 0;
+        return;
 
     /* Passing a region that wraps around is illegal */
     ASSERT(((uintptr_t)p + size - 1) >= (uintptr_t)p);
@@ -211,18 +210,16 @@ static inline int clean_dcache_va_range(const void *p, unsigned long size)
             idx += dcache_line_bytes, size -= dcache_line_bytes )
         asm volatile (__clean_dcache_one(0) : : "r" (p + idx));
     dsb(sy);           /* So we know the flushes happen before continuing */
-    /* ARM callers assume that dcache_* functions cannot fail. */
-    return 0;
 }
 
-static inline int clean_and_invalidate_dcache_va_range
+static inline void clean_and_invalidate_dcache_va_range
     (const void *p, unsigned long size)
 {
     size_t cacheline_mask = dcache_line_bytes - 1;
     unsigned long idx = 0;
 
     if ( !size )
-        return 0;
+        return;
 
     /* Passing a region that wraps around is illegal */
     ASSERT(((uintptr_t)p + size - 1) >= (uintptr_t)p);
@@ -235,8 +232,6 @@ static inline int clean_and_invalidate_dcache_va_range
             idx += dcache_line_bytes, size -= dcache_line_bytes )
         asm volatile (__clean_and_invalidate_dcache_one(0) : : "r" (p + idx));
     dsb(sy);         /* So we know the flushes happen before continuing */
-    /* ARM callers assume that dcache_* functions cannot fail. */
-    return 0;
 }
 
 /* Macros for flushing a single small item.  The predicate is always
@@ -266,6 +261,20 @@ static inline int clean_and_invalidate_dcache_va_range
             : : "r" (_p), "m" (*_p));                                   \
 } while (0)
 
+static inline int arch_grant_cache_flush(unsigned int op, const void *p,
+                                         unsigned long size)
+{
+    if ( (op & GNTTAB_CACHE_INVAL) && (op & GNTTAB_CACHE_CLEAN) )
+        clean_and_invalidate_dcache_va_range(p, size);
+    else if ( op & GNTTAB_CACHE_INVAL )
+        invalidate_dcache_va_range(p, size);
+    else if ( op & GNTTAB_CACHE_CLEAN )
+        clean_dcache_va_range(p, size);
+
+    /* ARM callers assume that dcache_* functions cannot fail. */
+    return 0;
+}
+
 /*
  * Write a pagetable entry.
  *
diff --git a/xen/arch/x86/include/asm/flushtlb.h b/xen/arch/x86/include/asm/flushtlb.h
index bb0ad58db49b..c37bf4455714 100644
--- a/xen/arch/x86/include/asm/flushtlb.h
+++ b/xen/arch/x86/include/asm/flushtlb.h
@@ -10,6 +10,7 @@
 #ifndef __FLUSHTLB_H__
 #define __FLUSHTLB_H__
 
+#include <public/grant_table.h>
 #include <xen/mm.h>
 #include <xen/percpu.h>
 #include <xen/smp.h>
@@ -182,21 +183,21 @@ void flush_area_mask(const cpumask_t *mask, const void *va,
 }
 
 static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache) {}
-static inline int invalidate_dcache_va_range(const void *p,
-                                             unsigned long size)
-{ return -EOPNOTSUPP; }
-static inline int clean_and_invalidate_dcache_va_range(const void *p,
-                                                       unsigned long size)
+
+static inline int arch_grant_cache_flush(unsigned int op, const void *p,
+                                     unsigned long size)
 {
-    unsigned int order = get_order_from_bytes(size);
+    unsigned int order;
+
+    if ( !(op & GNTTAB_CACHE_CLEAN) )
+        return -EOPNOTSUPP;
+
+    order = get_order_from_bytes(size);
     /* sub-page granularity support needs to be added if necessary */
-    flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
+    (void) flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
+
     return 0;
 }
-static inline int clean_dcache_va_range(const void *p, unsigned long size)
-{
-    return clean_and_invalidate_dcache_va_range(p, size);
-}
 
 unsigned int guest_flush_tlb_flags(const struct domain *d);
 void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask);
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 5721eab22561..8615ea144bb3 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3572,14 +3572,7 @@ static int _cache_flush(const gnttab_cache_flush_t *cflush, grant_ref_t *cur_ref
     v = map_domain_page(mfn);
     v += cflush->offset;
 
-    if ( (cflush->op & GNTTAB_CACHE_INVAL) && (cflush->op & GNTTAB_CACHE_CLEAN) )
-        ret = clean_and_invalidate_dcache_va_range(v, cflush->length);
-    else if ( cflush->op & GNTTAB_CACHE_INVAL )
-        ret = invalidate_dcache_va_range(v, cflush->length);
-    else if ( cflush->op & GNTTAB_CACHE_CLEAN )
-        ret = clean_dcache_va_range(v, cflush->length);
-    else
-        ret = 0;
+    ret = arch_grant_cache_flush(cflush->op, v, cflush->length);
 
     if ( d != owner )
     {
-- 
2.34.1


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 15:21:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 15:21:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682978.1062260 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5Rz-0004Vj-L2; Mon, 19 Feb 2024 15:20:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682978.1062260; Mon, 19 Feb 2024 15:20:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5Rz-0004Vc-HS; Mon, 19 Feb 2024 15:20:59 +0000
Received: by outflank-mailman (input) for mailman id 682978;
 Mon, 19 Feb 2024 15:20:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=pKiv=J4=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rc5Ry-0004VQ-Tv
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 15:20:58 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7b3c43cf-cf3a-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 16:20:58 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-5648d92919dso1117338a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 07:20:57 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 f21-20020a056402195500b0056413012fa8sm2745641edz.62.2024.02.19.07.20.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 19 Feb 2024 07:20:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7b3c43cf-cf3a-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708356057; x=1708960857; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=CCRznYB5/GGkzm3SxrgrZSve2RalzmlGK5ybpDuAZCE=;
        b=JTUfAiipxskLu87nxar9AAvRaLsOOPYsJ/5ryeZMZ6yALD0qOdwwWc4yD6KwRG3JK/
         CgMj0WaMjEfDNsgT5J47TGDksmTh2vbRAOKIqqhKmhGTbdvfb5YHK6b0P/Pw5zSubT5f
         TrqSONjl8oJ0l4SE6cbP+/lXznmKZA0V1whb19MXDEW9+xfjF72o2pdvW0QbL6vfdakV
         +WdUcOusqP5DWK0vxTQcnoP/kvpGHXxzIMnBn7OXuebGrlN6BzF76L4nK2giKmI22wS3
         5R4nww7KZ1ZSkCSsu8iR79L17wUcvj9L/UvzybZQCboe60+HXiK5onvE0AFKiGTHmPYx
         j0QQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708356057; x=1708960857;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=CCRznYB5/GGkzm3SxrgrZSve2RalzmlGK5ybpDuAZCE=;
        b=rPGfYusO2CB1wusJVom4N4M047RiKc1WmIWUgG0eif7EgBnnXPo2EbP9Yfu9SH6ah3
         Q0mVe3C5jHhe46c6AC7cAGn3Jfrl2cLA+L2Dd1/3UBEGYDypAVi//7ATlcdRDqssRdVk
         A84TASpc5SHtrsGUotkH9RHVd5O76XOLjDMNTqgR/d20ugJaCfJ+3cP8ikA+kFrggdac
         P7UJFF7OHQZlkTWMoXc12mlrcBTlH4pKH/YUeEYWaeH3MXdfqW6Wq2z7lex9td8OxReZ
         CLnIZBR8lFZ2UfeejTNHBdfxMjWxCM/tOQD3KtE+PTOXjsU+nNkjAbMWPdMuODFp0L6i
         wweA==
X-Forwarded-Encrypted: i=1; AJvYcCXVvHhYM2AjU5UIW0oV+8ykYB0+YVIXaKty1tf/lZ8Ql2aGXJAjI1dt51bpxk2urzzN1Wi22m7zjHh813pMZBlf3j7vtfmhSB1ac6IbBng=
X-Gm-Message-State: AOJu0Yzczfd2+Of7gY4I6CtYEIgg0/YoWWCJloWjAcvc1RDYmC+82MkW
	7zmXKOt9jyY149xe8M+2oS1ZINJPly4b/k8rPXDMWVc3p2iLaYKe
X-Google-Smtp-Source: AGHT+IEnBxSV1nwoR4bEVd00+EsZoRD3Ifu3dDI4Jx42o3y+bCtbWr7mc8wAut4clxXg17V7cOOWUQ==
X-Received: by 2002:a05:6402:31fb:b0:564:acde:d135 with SMTP id dy27-20020a05640231fb00b00564acded135mr347480edb.29.1708356057249;
        Mon, 19 Feb 2024 07:20:57 -0800 (PST)
Message-ID: <f057507271070bc17e85a7637a9edd4c16b171c5.camel@gmail.com>
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Julien Grall <julien@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Mon, 19 Feb 2024 16:20:55 +0100
In-Reply-To: <73b4654a-cdf2-48dc-b802-a7890c3fe426@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
	 <491937fd-2459-416f-944b-0c63f7c12473@xen.org>
	 <1ebdc064665b396d1978ac188594682f08601dbc.camel@gmail.com>
	 <73b4654a-cdf2-48dc-b802-a7890c3fe426@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-19 at 15:12 +0100, Jan Beulich wrote:
> On 19.02.2024 15:00, Oleksii wrote:
> > On Sun, 2024-02-18 at 19:00 +0000, Julien Grall wrote:
> > > On 05/02/2024 15:32, Oleksii Kurochko wrote:
> > > > --- /dev/null
> > > > +++ b/xen/arch/riscv/include/asm/cmpxchg.h
> > > > @@ -0,0 +1,237 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > +/* Copyright (C) 2014 Regents of the University of California
> > > > */
> > > > +
> > > > +#ifndef _ASM_RISCV_CMPXCHG_H
> > > > +#define _ASM_RISCV_CMPXCHG_H
> > > > +
> > > > +#include <xen/compiler.h>
> > > > +#include <xen/lib.h>
> > > > +
> > > > +#include <asm/fence.h>
> > > > +#include <asm/io.h>
> > > > +#include <asm/system.h>
> > > > +
> > > > +#define ALIGN_DOWN(addr, size)=C2=A0 ((addr) & (~((size) - 1)))
> > > > +
> > > > +#define __amoswap_generic(ptr, new, ret, sfx, release_barrier,
> > > > acquire_barrier) \
> > > > +({ \
> > > > +=C2=A0=C2=A0=C2=A0 asm volatile( \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 release_barrier \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 " amoswap" sfx " %0, %2=
, %1\n" \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 acquire_barrier \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "=3Dr" (ret), "+A" (*=
ptr) \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "r" (new) \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 : "memory" ); \
> > > > +})
> > > > +
> > > > +#define emulate_xchg_1_2(ptr, new, ret, release_barrier,
> > > > acquire_barrier) \
> > > > +({ \
> > > > +=C2=A0=C2=A0=C2=A0 uint32_t *ptr_32b_aligned =3D (uint32_t
> > > > *)ALIGN_DOWN((unsigned
> > > > long)ptr, 4); \
> > > > +=C2=A0=C2=A0=C2=A0 uint8_t mask_l =3D ((unsigned long)(ptr) & (0x8=
 -
> > > > sizeof(*ptr)))
> > > > * BITS_PER_BYTE; \
> > > > +=C2=A0=C2=A0=C2=A0 uint8_t mask_size =3D sizeof(*ptr) * BITS_PER_B=
YTE; \
> > > > +=C2=A0=C2=A0=C2=A0 uint8_t mask_h =3D mask_l + mask_size - 1; \
> > > > +=C2=A0=C2=A0=C2=A0 unsigned long mask =3D GENMASK(mask_h, mask_l);=
 \
> > > > +=C2=A0=C2=A0=C2=A0 unsigned long new_ =3D (unsigned long)(new) << =
mask_l; \
> > > > +=C2=A0=C2=A0=C2=A0 unsigned long ret_; \
> > > > +=C2=A0=C2=A0=C2=A0 unsigned long rc; \
> > > > +=C2=A0=C2=A0=C2=A0 \
> > > > +=C2=A0=C2=A0=C2=A0 asm volatile( \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 release_barrier \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "0: lr.d %0, %2\n" \
> > >=20
> > > I was going to ask why this is lr.d rather than lr.w. But I see
> > > Jan=20
> > > already asked. I agree with him that it should probably be a lr.w
> > > and
> > > ...
> > >=20
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "=C2=A0=C2=A0 and=C2=A0=
 %1, %0, %z4\n" \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "=C2=A0=C2=A0 or=C2=A0=
=C2=A0 %1, %1, %z3\n" \
> > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "=C2=A0=C2=A0 sc.d %1, =
%1, %2\n" \
> > >=20
> > > ... respectively sc.w. The same applies for cmpxchg.
> >=20
> > I agree that it would be better, and my initial attempt was to
> > handle
> > 4-byte or 8-byte boundary crossing during 2-byte access:
> >=20
> > 0 1 2 3 4 5 6 7 8
> > X X X 1 1 X X X X
> >=20
> > In this case, if I align address 3 to address 0 and then read 4
> > bytes
> > instead of 8 bytes, I will not process the byte at address 4. This
> > was
> > the reason why I started to read 8 bytes.
> >=20
> > I also acknowledge that there could be an issue in the following
> > case:
> >=20
> > X=C2=A0 4094 4095 4096
> > X=C2=A0=C2=A0=C2=A0 1=C2=A0=C2=A0 1=C2=A0=C2=A0=C2=A0 X
> > In this situation, when I read 8 bytes, there could be an issue
> > where
> > the new page (which starts at 4096) will not be mapped. It seems
> > correct in this case to check that variable is within one page and
> > read
> > 4 bytes instead of 8.
> >=20
> > One more thing I am uncertain about is if we change everything to
> > read
> > 4 bytes with 4-byte alignment, what should be done with the first
> > case?
> > Should we panic? (I am not sure if this is an option.)
>=20
> Counter question (raised elsewhere already): What if a 4-byte access
> crosses a word / cache line / page boundary? Ideally exactly the
> same would happen for a 2-byte access crossing a respective boundary.
> (Which you can achieve relatively easily by masking off only address
> bit 1, keeping address bit 0 unaltered.)
But if we align down on a 4-byte boundary and then access 4 bytes, we
can't cross a boundary. I agree that the algorithm is not correct, as
it can ignore some values in certain situations. For example:
0 1 2 3 4 5 6 7 8
X X X 1 1 X X X X
In this case, the value at address 4 won't be updated.

I agree that introducing a new macro to check if a variable crosses a
boundary is necessary or as an option we can check that addr is 2-byte
aligned:

#define CHECK_BOUNDARY_CROSSING(start, end, boundary_size)
ASSERT((start / boundary_size) !=3D (end / boundary_size))

Then, it is necessary to check:

CHECK_BOUNDARY_CROSSING(start, end, 4)
CHECK_BOUNDARY_CROSSING(start, end, PAGE_SIZE)

But why do we need to check the cache line boundary? In the case of the
cache, the question will only be about performance, but it should still
work, shouldn't it?

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Mon Feb 19 15:22:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 15:22:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682982.1062269 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5TZ-0005JT-UZ; Mon, 19 Feb 2024 15:22:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682982.1062269; Mon, 19 Feb 2024 15:22:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5TZ-0005JM-Rx; Mon, 19 Feb 2024 15:22:37 +0000
Received: by outflank-mailman (input) for mailman id 682982;
 Mon, 19 Feb 2024 15:22:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc5TZ-0005J8-64
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 15:22:37 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b5b04338-cf3a-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 16:22:35 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d22b8801b9so30802591fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 07:22:35 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p11-20020a05600c1d8b00b004126a0dfd11sm1403795wms.29.2024.02.19.07.22.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 07:22:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b5b04338-cf3a-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708356155; x=1708960955; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/7B2JK78uOtI+4WAcrTSjeehT4XmVYOqJPVysWYKQlo=;
        b=RH4zOJsMczDOG4p8wo3Cud0OKaIXHrst9r90CnLriyKKsyvjjUDMsfs344tgdSez+u
         hifEzUuBZh5qTP0dzD/fjceBrg8AG4b7C2L2Syx4wjUBxEA8FOhJLqTbSdOq1domM9E9
         1Rn+T0XOrppfWKJ+9DGyuoF5syLeu35jJ65pFKmdXMUqamWExMw0WKdGsgjD7/MyWkB2
         FE/hDSC7E3903K2kNrARe3s42awf314Z7ZSipJmXZ7gk+koXJft9peUQYyTzmKN6pDdb
         m3nM/LLIZ2pOSCdofs6W44PhqJ/2y5LjFNtAN7+TQ35KmiobOSMOb3ARK5muZzUT7Yro
         K3ew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708356155; x=1708960955;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/7B2JK78uOtI+4WAcrTSjeehT4XmVYOqJPVysWYKQlo=;
        b=PI+Pn7ZzVj/B2nupG1jYgswj/pLUn8Wpa87Bs3CCz1HkuthzME3KA+S2ZK9NJUb5TQ
         eQmz1N1DbtHPVocDJwd4ZfPBPzk4wgsp9Qt7qDRz5vX4QPuQntrJhMY5vrh6tHUh8M0y
         Lpqj0/ZCjaHZnCIe+FSLGhY9G+oZu3URB4D6IuOklZbDYuC4koGHQL0xMD0F2eJqTd6x
         +Zy3Q+yqALycWVwpajFN7vd07rBO2VdKHu/Drc1yHVUrF3VPc4FfmEeE3Vbel1XhAZHS
         HV2nyYvLVXhMb9lnLnUlNd/222+T+qbKopRAhsR+MBZgzrWNbiYWpt43rHisUECd8ZyJ
         q9Tw==
X-Forwarded-Encrypted: i=1; AJvYcCXrLBhbUZcSLddj/rcfs01sGovrMFlnFQxPa5FM+mNMPvTe8Lm70J8jW9WGuWwhZzUD9Jzv9WKxxobp/OycEVEakCuLeZPb02+wBcxxVJQ=
X-Gm-Message-State: AOJu0YySyNvFOZn6tvj1qJF7rReBmUyZRzj2+Mfw0BBMny9dRuJXvUiG
	lRLq96CCo4lDupFas2puNVfvXd353Jfu45cjACeOGX987iwghytxT4Zk/F0L5g==
X-Google-Smtp-Source: AGHT+IESt0eAT5Jk8d7blPsJb0QGmq/ImUQ6hs8DzyaCF7bmiJNy50g5ZIVjh/XRX7ATqiB+pyaO9g==
X-Received: by 2002:a2e:b8c8:0:b0:2d2:bd9:f100 with SMTP id s8-20020a2eb8c8000000b002d20bd9f100mr9444976ljp.20.1708356155452;
        Mon, 19 Feb 2024 07:22:35 -0800 (PST)
Message-ID: <c5f3a78c-4a2b-499b-93bb-11f60b0afcc0@suse.com>
Date: Mon, 19 Feb 2024 16:22:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/6] nestedsvm: Disable TscRateMSR
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-5-george.dunlap@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240206012051.3564035-5-george.dunlap@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 02:20, George Dunlap wrote:
> For now, just disable the functionality entirely until we can
> implement it properly:
> 
> - Don't set TSCRATEMSR in the host CPUID policy

This goes too far: This way you would (in principle) also affect guests
with nesting disabled. According to the earlier parts of the description
there's also no issue with it in that case. What you want to make sure
it that in the HVM policy the bit isn't set.

While presently resolving to cpu_has_svm_feature(), I think
cpu_has_tsc_ratio really ought to resolve to the host policy field.
Of course then requiring the host policy to reflect reality rather than
having what is "always emulated". IOW ...

> --- a/xen/arch/x86/cpu-policy.c
> +++ b/xen/arch/x86/cpu-policy.c
> @@ -407,8 +407,7 @@ static void __init calculate_host_policy(void)
>                                 (1u << SVM_FEATURE_PAUSEFILTER) |
>                                 (1u << SVM_FEATURE_DECODEASSISTS));
>          /* Enable features which are always emulated. */
> -        p->extd.raw[0xa].d |= ((1u << SVM_FEATURE_VMCBCLEAN) |
> -                               (1u << SVM_FEATURE_TSCRATEMSR));
> +        p->extd.raw[0xa].d |= (1u << SVM_FEATURE_VMCBCLEAN);

... this likely wants replacing altogether by not overriding what we
found in hardware, which would apparently mean moving the two bit
masks to the earlier "clamping" expression.

But then of course Andrew may know of reasons why all of this is done
in calculate_host_policy() in the first place, rather than in HVM
policy calculation.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 15:24:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 15:24:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682987.1062279 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5VA-0005wc-8a; Mon, 19 Feb 2024 15:24:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682987.1062279; Mon, 19 Feb 2024 15:24:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5VA-0005wV-5s; Mon, 19 Feb 2024 15:24:16 +0000
Received: by outflank-mailman (input) for mailman id 682987;
 Mon, 19 Feb 2024 15:24:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc5V8-0005wJ-HK
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 15:24:14 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id efe5cdd9-cf3a-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 16:24:13 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-41269c801eeso4195135e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 07:24:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p11-20020a05600c1d8b00b004126a0dfd11sm1403795wms.29.2024.02.19.07.24.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 07:24:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: efe5cdd9-cf3a-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708356253; x=1708961053; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Z3EQ0FuF/DlxEb+dNpQMGzIUCEU4BkzY8Az1b9ruWr8=;
        b=Mqnneyr/HcFxsSF6CkBxZeqtzKlp8xp/yt+Xf5QpgwC8x7p+t+qo5HzBZy0Gfu050U
         9oqOiSNXLghJ4zUtdoT8EsYsOsAfeWNm1d7pF+QmsmfQjVmrc5SizjWv0ttlZwxktPty
         /au08GDVqduBFniILx8VONTYJDBNcRIc8FXHv9h8ACFCAsue7Mapb6oNd2XdxX4yt/zV
         TAJhDkQ7XSIbZf+9MWnQ0CDZkkaHaT6lER+A9Ra4cYVuvLWinYIc/t/w4a8xeOOgWc/a
         enE3yD2tLRki+AGwOHa9LsMUZMI0nQTEPhjM+6RnNPvJLx/fAcDcv2rPghIlrWpWLnOO
         f9Fw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708356253; x=1708961053;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Z3EQ0FuF/DlxEb+dNpQMGzIUCEU4BkzY8Az1b9ruWr8=;
        b=SGzFw/E8ySdtgzDuH8wbZEShq9FpB6vbGhr4gKMSDsh0i8x+x9yj++zVqDs5jsEn1/
         ivk2jMZeoImCJbAL+44+VDC4DHL9eYtTU6D+H6ELzCrO+oe9WzvG9Bj4ZEz8XQHS9D58
         KHnBZ/D8X1OsqHBkGNvbRzR+C6Bq67Iqc83oKFq7b4HproRMKUrUf9uEbsnpnUpP1oJw
         Jar4ZHFPT24Ylyf9mvCg5Lf3xmbMvu7dHI53b2fg/BwGITBKrnWsqfFSLC1fNesHTB3z
         bKTz3Q46bPCzU9BORZPD97DsKFCsdqUzmK1ORpByCeY9yRbzen5CzYo2wNLhxyRDw/1n
         4ZEQ==
X-Forwarded-Encrypted: i=1; AJvYcCVfVN/mH8tsG4dwpHvKwawITjbQDf72DuTP6RSlEcV7VaMvmnZYZnOqyy58Q7I7SDiruZ+sB7iW3chUOhGo3zDFIx4n1tTBepGauA5y41M=
X-Gm-Message-State: AOJu0YyiI3XS+UXg5QVrtduLrZhNFBdz3ovMbxLiYAsnv0FAeMwzgejl
	6in0zHpAgXvLZg/M2QD+Ngu1hv34zVKCpd/JKbZe8Pl4/khFLSx8vLLRXBikDQ==
X-Google-Smtp-Source: AGHT+IHHPAV6+D/tiaYABLDwjlx8DDvh9bym+Mb5bPehcoqcHQ8Qt/V12R1CD5n/lAGY932pGHKwdg==
X-Received: by 2002:a05:600c:68d7:b0:411:defc:c7e4 with SMTP id jd23-20020a05600c68d700b00411defcc7e4mr8736083wmb.38.1708356253093;
        Mon, 19 Feb 2024 07:24:13 -0800 (PST)
Message-ID: <14f09e7a-76b4-4650-9f90-90e7709379be@suse.com>
Date: Mon, 19 Feb 2024 16:24:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/6] svm: Improve type of cpu_has_svm_feature
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-3-george.dunlap@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240206012051.3564035-3-george.dunlap@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 02:20, George Dunlap wrote:
> --- a/xen/arch/x86/include/asm/hvm/svm/svm.h
> +++ b/xen/arch/x86/include/asm/hvm/svm/svm.h
> @@ -38,7 +38,10 @@ extern u32 svm_feature_flags;
>  #define SVM_FEATURE_SSS           19 /* NPT Supervisor Shadow Stacks */
>  #define SVM_FEATURE_SPEC_CTRL     20 /* MSR_SPEC_CTRL virtualisation */
>  
> -#define cpu_has_svm_feature(f) (svm_feature_flags & (1u << (f)))
> +static inline bool cpu_has_svm_feature(unsigned int feat)
> +{
> +    return svm_feature_flags & (1u << (feat));
> +}
>  #define cpu_has_svm_npt       cpu_has_svm_feature(SVM_FEATURE_NPT)
>  #define cpu_has_svm_lbrv      cpu_has_svm_feature(SVM_FEATURE_LBRV)
>  #define cpu_has_svm_svml      cpu_has_svm_feature(SVM_FEATURE_SVML)

Having seen patch 4 now, I have to raise the question here as well: Why
do we need a separate variable (svm_feature_flags) when we could use
the host policy (provided it isn't abused; see comments on patch 4)?

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 15:29:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 15:29:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.682993.1062291 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5aJ-0006yE-SH; Mon, 19 Feb 2024 15:29:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 682993.1062291; Mon, 19 Feb 2024 15:29:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5aJ-0006y7-No; Mon, 19 Feb 2024 15:29:35 +0000
Received: by outflank-mailman (input) for mailman id 682993;
 Mon, 19 Feb 2024 15:29:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc5aI-0006wl-KM
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 15:29:34 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id adfe6a7b-cf3b-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 16:29:32 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-41263dbdfeaso10498775e9.2
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 07:29:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bo29-20020a056000069d00b0033d5f5571b4sm1611178wrb.44.2024.02.19.07.29.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 07:29:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: adfe6a7b-cf3b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708356572; x=1708961372; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Kbc2VJ6Qu946PsbcJ9T7ZgnhxaE1lcBajh58CAVIoqw=;
        b=eqjfSl0gp86mm9VyHH9TPNPsmSdHwrGamOwm3IBs4KFVU+D7wAs0z3UNrui6Rho28s
         zmCRVW6ajT7hkDqSyAYnqHVYzBcmKP3BZBe6toxmPkOHOcZxeiI9GdsW9WJtytCIP3HM
         PYnV9vMcZ6iMc8XYZxFLadfH8PfxLoGk4HsK492Nc4unKCAroSDEg3b4wuRxhTP5VdIn
         d+4fXVw8pu6PPeOAFJ67UAvCUo2/Qnxp8rEGu1lAJzyb/URQbWYeZr0FkNuomg0lMCwZ
         BhEKVfClsv1Ti/9C58NsoXGJvzCW1IP1DMhb5IGgskD8V0VV6zjWk1PHFKI9Js8UTpkx
         BEuw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708356572; x=1708961372;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Kbc2VJ6Qu946PsbcJ9T7ZgnhxaE1lcBajh58CAVIoqw=;
        b=PLDHtv9fdIT4upX9k0lf6ao9KKNEYx8Vos5h0l9/0SihI/UOoF2+E4DFlPgil1mQAM
         MAbz8ycqzGZg3Qxk71s31fnFEyU7BLvnuwLTGOUtBAwhBqhwozMiyi+1DtoBUJKLSbmZ
         PO0ecigrbt2MIGtxyK813r+ov8/TWQxOb9dAKUFqO0DFK/xBVjbmb/V47Mu7KQfAip2H
         kpps7Ro5mo00gjl+nBOlmo7lLqazXvS0FNgEbxByqGpaQ0LLCxkZoU2mVX9AO1sVpS1V
         u3MEAtffqZLYFAlMH4JRDdpC26nGTtpI6GXL8cUNx1cI6N5fJpNiBC6rIGrnTuN0Dtqw
         bR8w==
X-Forwarded-Encrypted: i=1; AJvYcCV9m8L6bBl4m5pEGIE8bnDNVC6XAInjKHmkjia6u4jck+5YWazBIdtm8kb4JS6TagrwO0Rcq6I/McRvWEM5WGETjQWaJSTg3jxMU4SPSLE=
X-Gm-Message-State: AOJu0Yx+fN9RayFuWxdtFVyyJv0VIIHVIJzgwWEza3SiIBs+XDizGyAZ
	e9ZgLoD0XbDJ+0uoQJO9Uw0HHvhzjr6UNgX3mOBfP9+w6HSQaXFpnmGSOn9Iug==
X-Google-Smtp-Source: AGHT+IFpYlDtW5TvCxvV7x2wa4iusWdKI89Oua0iJ+xBxohaWlB2YoDtvsLfjlBYQOB4Clj69r0ipQ==
X-Received: by 2002:a5d:660c:0:b0:33d:38ed:dc9b with SMTP id n12-20020a5d660c000000b0033d38eddc9bmr3916088wru.56.1708356571912;
        Mon, 19 Feb 2024 07:29:31 -0800 (PST)
Message-ID: <78eb62fd-de6e-4616-9471-ef08d862bd73@suse.com>
Date: Mon, 19 Feb 2024 16:29:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
 <491937fd-2459-416f-944b-0c63f7c12473@xen.org>
 <1ebdc064665b396d1978ac188594682f08601dbc.camel@gmail.com>
 <73b4654a-cdf2-48dc-b802-a7890c3fe426@suse.com>
 <f057507271070bc17e85a7637a9edd4c16b171c5.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f057507271070bc17e85a7637a9edd4c16b171c5.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 19.02.2024 16:20, Oleksii wrote:
> On Mon, 2024-02-19 at 15:12 +0100, Jan Beulich wrote:
>> On 19.02.2024 15:00, Oleksii wrote:
>>> On Sun, 2024-02-18 at 19:00 +0000, Julien Grall wrote:
>>>> On 05/02/2024 15:32, Oleksii Kurochko wrote:
>>>>> --- /dev/null
>>>>> +++ b/xen/arch/riscv/include/asm/cmpxchg.h
>>>>> @@ -0,0 +1,237 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>>> +/* Copyright (C) 2014 Regents of the University of California
>>>>> */
>>>>> +
>>>>> +#ifndef _ASM_RISCV_CMPXCHG_H
>>>>> +#define _ASM_RISCV_CMPXCHG_H
>>>>> +
>>>>> +#include <xen/compiler.h>
>>>>> +#include <xen/lib.h>
>>>>> +
>>>>> +#include <asm/fence.h>
>>>>> +#include <asm/io.h>
>>>>> +#include <asm/system.h>
>>>>> +
>>>>> +#define ALIGN_DOWN(addr, size)  ((addr) & (~((size) - 1)))
>>>>> +
>>>>> +#define __amoswap_generic(ptr, new, ret, sfx, release_barrier,
>>>>> acquire_barrier) \
>>>>> +({ \
>>>>> +    asm volatile( \
>>>>> +        release_barrier \
>>>>> +        " amoswap" sfx " %0, %2, %1\n" \
>>>>> +        acquire_barrier \
>>>>> +        : "=r" (ret), "+A" (*ptr) \
>>>>> +        : "r" (new) \
>>>>> +        : "memory" ); \
>>>>> +})
>>>>> +
>>>>> +#define emulate_xchg_1_2(ptr, new, ret, release_barrier,
>>>>> acquire_barrier) \
>>>>> +({ \
>>>>> +    uint32_t *ptr_32b_aligned = (uint32_t
>>>>> *)ALIGN_DOWN((unsigned
>>>>> long)ptr, 4); \
>>>>> +    uint8_t mask_l = ((unsigned long)(ptr) & (0x8 -
>>>>> sizeof(*ptr)))
>>>>> * BITS_PER_BYTE; \
>>>>> +    uint8_t mask_size = sizeof(*ptr) * BITS_PER_BYTE; \
>>>>> +    uint8_t mask_h = mask_l + mask_size - 1; \
>>>>> +    unsigned long mask = GENMASK(mask_h, mask_l); \
>>>>> +    unsigned long new_ = (unsigned long)(new) << mask_l; \
>>>>> +    unsigned long ret_; \
>>>>> +    unsigned long rc; \
>>>>> +    \
>>>>> +    asm volatile( \
>>>>> +        release_barrier \
>>>>> +        "0: lr.d %0, %2\n" \
>>>>
>>>> I was going to ask why this is lr.d rather than lr.w. But I see
>>>> Jan 
>>>> already asked. I agree with him that it should probably be a lr.w
>>>> and
>>>> ...
>>>>
>>>>> +        "   and  %1, %0, %z4\n" \
>>>>> +        "   or   %1, %1, %z3\n" \
>>>>> +        "   sc.d %1, %1, %2\n" \
>>>>
>>>> ... respectively sc.w. The same applies for cmpxchg.
>>>
>>> I agree that it would be better, and my initial attempt was to
>>> handle
>>> 4-byte or 8-byte boundary crossing during 2-byte access:
>>>
>>> 0 1 2 3 4 5 6 7 8
>>> X X X 1 1 X X X X
>>>
>>> In this case, if I align address 3 to address 0 and then read 4
>>> bytes
>>> instead of 8 bytes, I will not process the byte at address 4. This
>>> was
>>> the reason why I started to read 8 bytes.
>>>
>>> I also acknowledge that there could be an issue in the following
>>> case:
>>>
>>> X  4094 4095 4096
>>> X    1   1    X
>>> In this situation, when I read 8 bytes, there could be an issue
>>> where
>>> the new page (which starts at 4096) will not be mapped. It seems
>>> correct in this case to check that variable is within one page and
>>> read
>>> 4 bytes instead of 8.
>>>
>>> One more thing I am uncertain about is if we change everything to
>>> read
>>> 4 bytes with 4-byte alignment, what should be done with the first
>>> case?
>>> Should we panic? (I am not sure if this is an option.)
>>
>> Counter question (raised elsewhere already): What if a 4-byte access
>> crosses a word / cache line / page boundary? Ideally exactly the
>> same would happen for a 2-byte access crossing a respective boundary.
>> (Which you can achieve relatively easily by masking off only address
>> bit 1, keeping address bit 0 unaltered.)
> But if we align down on a 4-byte boundary and then access 4 bytes, we
> can't cross a boundary. I agree that the algorithm is not correct, as
> it can ignore some values in certain situations. For example:
> 0 1 2 3 4 5 6 7 8
> X X X 1 1 X X X X
> In this case, the value at address 4 won't be updated.
> 
> I agree that introducing a new macro to check if a variable crosses a
> boundary is necessary or as an option we can check that addr is 2-byte
> aligned:
> 
> #define CHECK_BOUNDARY_CROSSING(start, end, boundary_size)
> ASSERT((start / boundary_size) != (end / boundary_size))
> 
> Then, it is necessary to check:
> 
> CHECK_BOUNDARY_CROSSING(start, end, 4)
> CHECK_BOUNDARY_CROSSING(start, end, PAGE_SIZE)
> 
> But why do we need to check the cache line boundary? In the case of the
> cache, the question will only be about performance, but it should still
> work, shouldn't it?

You don't need to check for any of these boundaries. You can simply
leverage what the hardware does for misaligned accesses. See the
various other replies I've sent - I thought things should have become
pretty much crystal clear by now: For 1-byte accesses you access the
containing word, by clearing the low two bits. For 2-byte accesses
you also access the containing word, by clearing only bit 1 (which
the naturally leaves no bit that needs clearing for the projected
[but not necessary] case of handling a 4-byte access). If the resulting
4-byte access then is still misaligned, it'll fault just as a non-
emulated 4-byte access would. And you don't need to care about any of
the boundaries, not at words, not at cache lines, and not at pages.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 15:34:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 15:34:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683003.1062299 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5ew-0000ok-Ex; Mon, 19 Feb 2024 15:34:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683003.1062299; Mon, 19 Feb 2024 15:34:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5ew-0000od-C0; Mon, 19 Feb 2024 15:34:22 +0000
Received: by outflank-mailman (input) for mailman id 683003;
 Mon, 19 Feb 2024 15:34:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=wBJ7=J4=m5p.com=ehem@srs-se1.protection.inumbo.net>)
 id 1rc5ev-0000oV-J1
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 15:34:21 +0000
Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 57e6896f-cf3c-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 16:34:19 +0100 (CET)
Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7])
 by mailhost.m5p.com (8.17.1/8.15.2) with ESMTPS id 41JFY2sT024254
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Mon, 19 Feb 2024 10:34:08 -0500 (EST) (envelope-from ehem@m5p.com)
Received: (from ehem@localhost)
 by m5p.com (8.17.1/8.15.2/Submit) id 41JFY2r5024253;
 Mon, 19 Feb 2024 07:34:02 -0800 (PST) (envelope-from ehem)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57e6896f-cf3c-11ee-8a52-1f161083a0e0
Date: Mon, 19 Feb 2024 07:34:02 -0800
From: Elliott Mitchell <ehem+xen@m5p.com>
To: George Dunlap <george.dunlap@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, committers@xenproject.org,
        Kelly Choi <kelly.choi@cloud.com>
Subject: Re: Stats on Xen tarball downloads
Message-ID: <ZdN06v1k3E8xarLY@mattapan.m5p.com>
References: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
X-Spam-Status: No, score=0.0 required=10.0 tests=KHOP_HELO_FCRDNS
	autolearn=unavailable autolearn_force=no version=4.0.0
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on mattapan.m5p.com

On Mon, Feb 19, 2024 at 06:01:54PM +0800, George Dunlap wrote:
> 
> Looking at the *non*-4.18 downloads, nearly all of them have user
> agents that make it clear they're part of automated build systems:
> user agents like curl and wget, but also "Go-http-client", "libfetch",
                   ^^^^     ^^^^

I reject this claim.  `curl` or `wget` could be part of an interactive
operation.  Telling a browser to copy a URL into the paste buffer, then
using `wget`/`curl` is entirely possible.  I may be the outlier, but I
routinely do this.

I don't know whether Gentoo's `emerge` uses `wget`/`curl`, but that could
be semi-interactive.


> It's not really clear to me why we'd be getting 300-ish people
> downloading the Xen 4.18.0 tarball, 2/3 of which are on Windows.  But
> then I'm also not sure why someone would *fake* hundreds of downloads
> a week from unique IP addresses; and in particular, if you were going
> to fake hundreds of downloads a week, I'm not sure why you'd only fake
> the most recent release.

Remember the browser wars?  At one point many sites were looking for
IE/Windows and sending back error messages without those.  Getting the
tarball on Windows doesn't seem too likely, faking the browser was
pretty common for a while.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@m5p.com  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445




From xen-devel-bounces@lists.xenproject.org Mon Feb 19 15:43:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 15:43:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683008.1062310 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5nc-0003LJ-AL; Mon, 19 Feb 2024 15:43:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683008.1062310; Mon, 19 Feb 2024 15:43:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc5nc-0003LC-6U; Mon, 19 Feb 2024 15:43:20 +0000
Received: by outflank-mailman (input) for mailman id 683008;
 Mon, 19 Feb 2024 15:43:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=k0rs=J4=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rc5na-0003L1-UB
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 15:43:18 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20604.outbound.protection.outlook.com
 [2a01:111:f400:7e88::604])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 992d3d62-cf3d-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 16:43:17 +0100 (CET)
Received: from DS7PR05CA0034.namprd05.prod.outlook.com (2603:10b6:8:2f::35) by
 LV2PR12MB5728.namprd12.prod.outlook.com (2603:10b6:408:17c::7) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7316.19; Mon, 19 Feb 2024 15:43:13 +0000
Received: from CY4PEPF0000EDD3.namprd03.prod.outlook.com
 (2603:10b6:8:2f:cafe::24) by DS7PR05CA0034.outlook.office365.com
 (2603:10b6:8:2f::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.14 via Frontend
 Transport; Mon, 19 Feb 2024 15:43:13 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EDD3.mail.protection.outlook.com (10.167.241.207) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 19 Feb 2024 15:43:12 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 19 Feb
 2024 09:43:12 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 19 Feb
 2024 07:43:12 -0800
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 19 Feb 2024 09:43:10 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 992d3d62-cf3d-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=TTtoowCLLtAMqBhclrd5zWj3ZP+iqIWR6062yh7OCcS7WjCX2/ZucjFqxyiXXkjJx2WhMuTdLzKkfTcdqFwUrxkWV8YYWtW293L8DcSZRXpyJNKAYjzfj5UuiZS5gx41vgYuYsY4ALU37b79feUN6mu+bxXeqEwQpc2txSOuYOpv6qJzqiwAHv3gnzMqWBRU9VH5UoaUuoViDAjXqLnOwYzP09kavKpCsQKqIxCvN3Y+fEJUUOYeMufTyTnfHYHvkaBnwhPvmp3mIZAlDTpHhBRz6uRaBi06tlJ1Ux6SnksS4IZ2BESqQfLEzDbzEUHg83rG4Ou9dIRmXC9bk/E1+w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=AR3nOagl3W0nxyDpSoQSjtwekC9dWEe27HlkS/0Fmz4=;
 b=e5zguS/EJw3DwOPRXZvPRuD0gnpmTn6uG/MTtfjnqkdMDpxMocxKvWZwly3S5OetxeCQY6aBUGnV8cegeDWh1kjDiwtKnsAj7CtWSTuVyCo66P5uZa9TgfmHxDALvYFC+ym/7JOHi+1Y/P9Hf28m/xC1jxGZB2acJOf4Zq2853g2HYuPuSTL2FsAJBSA2dUfUSk/+OMWSR1Vs26891kgeIf9wNwozytbbrGIXT5THi7W/y8gTB3euajFLe8S/BeO6tML7Ks+bqKfKWNydpiwimNVTzrcOl9iKoDm1Pd7Ul+T5z6+rPpw3DveuWgVR34JRRPuHEOOjrDabjsyXzfeUg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=AR3nOagl3W0nxyDpSoQSjtwekC9dWEe27HlkS/0Fmz4=;
 b=mJnHlbKmAQoSl+0URSbS+bcFTxUvYAipmNT7kArOsEtL3j3Kj2HdPepOmroPd3FaaMXFeDqOpTYNNbyb/hVDf1QR+kYwY5HHb7jQq+bogcf+9g2JsFHRrfVp3B2WDGyVNGcMx+bwTgHjJNcTwQcpy1C/zmHhPkmLMKv1bluC+8M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <9d1a545c-a93e-4f8e-bc76-0b5676fa01ea@amd.com>
Date: Mon, 19 Feb 2024 16:43:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v4 2/3] xen/arm: arm64: Add emulation of Debug Data Transfer
 Registers
Content-Language: en-US
To: Ayan Kumar Halder <ayankuma@amd.com>, Julien Grall <julien@xen.org>, "Ayan
 Kumar Halder" <ayan.kumar.halder@amd.com>, <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <stefano.stabellini@amd.com>,
	<Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>,
	<luca.fancellu@arm.com>
References: <20240131121049.225044-1-ayan.kumar.halder@amd.com>
 <20240131121049.225044-3-ayan.kumar.halder@amd.com>
 <0c4e4a40-7327-4377-bfdd-82e69bc838ae@xen.org>
 <7098c9ab-7008-4a49-92d4-6cd201b1490e@amd.com>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <7098c9ab-7008-4a49-92d4-6cd201b1490e@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD3:EE_|LV2PR12MB5728:EE_
X-MS-Office365-Filtering-Correlation-Id: 532aada3-af9e-4db7-ca87-08dc31617b58
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	+V0e6vuXII1+xijbeu/6riYkEoWvuJ0QPJCUw3AuyySnSdFbkXlmqm6J5RnWQOx3cmYmDLqM7wjzD0iClG+3NiscijYgR0ab/ClRw4G67uo/aItMOrVVs3KUYhgOQ9zarQo+/ITcgVaya81cdmuMoop2J/pUrl1SdAOwLrQvRoIRVEtRglIkg06bMAeIhgkBTdkPsOIJUCzu2jgioPDMDk9tmel++3I/23VvqjwsKQaCLN80WPC9vi8TlrGv1khQ9sejZ4mkCSTyvkHwGdNnAcrj/VE+EA8XqWon1JLtuJXlFX+AF1b2y877qmjMj5lzhLNJfnCkuiPZmDMK84IwVhjtaJLxbQG8HjQlEwvIlI3e0lCCme63ieDkmLeg1sa3fIs5o2lX5/LpB8EkRJgwWjQT8KgcqS4yNir+hJnroqUdtor+ilYRIl9nBuQPE8TKpmNNpPK0rU4J3bSRrx0F6HHR2O9QurSDR/+eGz6eTMIwvpfdsOSEw4W5twRzniwcdmMkcLnmTuo4av//i4YvJMwkxLV787UXrhtzj6vL1qhVJ2sFJ6GEBcDl2sOvEUZAf5f1PnEkv3aH+sEh6PMc5EAtWHuCe7yqW/jPvlRcb+71lLojy22JQXZlGegt/mmGIIzhJbJWnmLMDxgMK30/Pb9SX5Vjs+y/nYeolHaAhEg=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(46966006)(40470700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Feb 2024 15:43:12.9679
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 532aada3-af9e-4db7-ca87-08dc31617b58
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EDD3.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5728

Hi Ayan,

On 19/02/2024 15:45, Ayan Kumar Halder wrote:
> 
> On 06/02/2024 19:05, Julien Grall wrote:
>> Hi Ayan,
> Hi Julien/Michal,
>>
>> On 31/01/2024 12:10, Ayan Kumar Halder wrote:
>>> From: Michal Orzel <michal.orzel@amd.com>
>>>
>>> Currently, if user enables HVC_DCC config option in Linux, it invokes 
>>> access
>>> to debug data transfer registers (i.e. DBGDTRTX_EL0 on arm64, 
>>> DBGDTRTXINT on
>>> arm32). As these registers are not emulated, Xen injects an undefined
>>> exception to the guest and Linux crashes.
>>>
>>> To prevent this crash, introduce a partial emulation of DBGDTR[TR]X_EL0
>>> (these registers share the same encoding) as RAZ/WI and MDCCSR_EL0 as 
>>> TXfull.
>>>
>>> Refer ARM DDI 0487J.a ID042523, D19.3.8, DBGDTRTX_EL0
>>> "If TXfull is set to 1, set DTRRX and DTRTX to UNKNOWN".
>>>
>>> Thus, any OS is expected to read MDCCSR_EL0 and check for TXfull before
>>> using DBGDTRTX_EL0. Linux does it via hvc_dcc_init() ---> 
>>> hvc_dcc_check(),
>>> and returns -ENODEV in case TXfull bit is still set after writing a test
>>> character. This way we prevent the guest from making use of HVC DCC as a
>>> console.
>>>
>>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
>>> ---
>>> Changes from
>>>
>>> v1 :- 1. DBGDTR_EL0 does not emulate RXfull. This is to avoid giving 
>>> the OS any
>>> indication that the RX buffer is full and is waiting to be read.
>>>
>>> 2. In Arm32, DBGOSLSR is emulated. Also DBGDTRTXINT is emulated at 
>>> EL0 only.
>>>
>>> 3. Fixed the commit message and inline code comments.
>>>
>>> v2 :- 1. Split the patch into two (separate patches for arm64 and 
>>> arm32).
>>> 2. Removed the "fail" label.
>>> 3. Fixed the commit message.
>>>
>>> v3 :- 1. "HSR_SYSREG_MDCCSR_EL0" emulation differs based on whether
>>> partial_emulation_enabled is true or not.
>>>
>>> 2. If partial_emulation_enabled is false, then access to 
>>> HSR_SYSREG_DBGDTR_EL0,
>>> HSR_SYSREG_DBGDTRTX_EL0 would lead to undefined exception.
>>>
>>>   xen/arch/arm/arm64/vsysreg.c         | 28 ++++++++++++++++++++++++----
>>>   xen/arch/arm/include/asm/arm64/hsr.h |  3 +++
>>>   2 files changed, 27 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>>> index b5d54c569b..94f0a6c384 100644
>>> --- a/xen/arch/arm/arm64/vsysreg.c
>>> +++ b/xen/arch/arm/arm64/vsysreg.c
>>> @@ -159,9 +159,6 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>        *
>>>        * Unhandled:
>>>        *    MDCCINT_EL1
>>> -     *    DBGDTR_EL0
>>> -     *    DBGDTRRX_EL0
>>> -     *    DBGDTRTX_EL0
>>>        *    OSDTRRX_EL1
>>>        *    OSDTRTX_EL1
>>>        *    OSECCR_EL1
>>> @@ -173,10 +170,32 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>           return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
>>>       case HSR_SYSREG_MDCCSR_EL0:
>>>           /*
>>> +         * Xen doesn't expose a real (or emulated) Debug 
>>> Communications Channel
>>> +         * (DCC) to a domain. Yet the Arm ARM implies this is not an 
>>> optional
>>> +         * feature. So some domains may start to probe it. For 
>>> instance, the
>>> +         * HVC_DCC driver in Linux (since f377775dc083 and at least 
>>> up to v6.7),
>>> +         * will try to write some characters and check if the 
>>> transmit buffer
>>> +         * has emptied.
>>> +         *
>>> +         * By setting TX status bit (only if partial emulation is 
>>> enabled) to
>>> +         * indicate the transmit buffer is full, we would hint the 
>>> OS that the
>>> +         * DCC is probably not working.
>>> +         *
>>> +         * Bit 29: TX full
>>> +         *
>>>            * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We 
>>> emulate that
>>>            * register as RAZ/WI above. So RO at both EL0 and EL1.
>>
>> The sentence "we emulate that register as ..." seems to be stale?
I can see that you tried to handle Julien remark here. But I disagree. This statement
is not stale. It explains that because MDSCR_EL1 is RAZ/WI, MDCCSR_EL0 is RO at both
EL0 and EL1. This patch does not change this behavior.

>>
>>>            */
>>> -        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 0);
>>> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, 
>>> hsr, 0,
>>> +                                  partial_emulation ? (1U << 29) : 0);
>>> +
>>> +    case HSR_SYSREG_DBGDTR_EL0:
>>> +    /* DBGDTR[TR]X_EL0 share the same encoding */
>>> +    case HSR_SYSREG_DBGDTRTX_EL0:
>>> +        if ( !partial_emulation )
>>> +            goto fail;
>>> +        return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 0);
>>
>> AFAICT, all the emulation helpers have an explanation why we are using 
>> them. But here this is not the case. Can you add one?
> This and..
>>
>>> +
>>>       HSR_SYSREG_DBG_CASES(DBGBVR):
>>>       HSR_SYSREG_DBG_CASES(DBGBCR):
>>>       HSR_SYSREG_DBG_CASES(DBGWVR):
>>> @@ -394,6 +413,7 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>        * And all other unknown registers.
>>>        */
>>>       default:
>>> + fail:
>>
>> AFAICT, this would violate MISRA 15.3 [1]. We didn't seem to have yet 
>> (?) accepted the rule, but I don't see we would not given I feel this 
>> is similar to what Rule 16.2 is trying to prevent and we accepted it.
>>
>> I think case, I move all the code within default outside. And then 
>> call "goto fail" from the default label.
> 
> I am not sure if I have interpreted this correctly.
> 
> Is it ok if you can take a look at the attached patch and let me know if 
> the explaination and the code change looks sane ?
Looking at the attached patch and fail handling, I don't think it is what Julien meant.
In the default case you should jump to fail that would be defined outside of switch clause.

Something like:
    default:
        goto fail;
    }

    regs->pc += 4;
    return;

fail:
    gdprintk...

When it comes to explanation for HSR_SYSREG_DBGDTRTX_EL0, I will let Julien to provide a comment he believes is right.
To me, it feels strange to repeat almost the same information as for MDCCSR_EL0.

~Michal


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 15:56:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 15:56:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683014.1062320 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc60T-0005cr-Cf; Mon, 19 Feb 2024 15:56:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683014.1062320; Mon, 19 Feb 2024 15:56:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc60T-0005ck-A4; Mon, 19 Feb 2024 15:56:37 +0000
Received: by outflank-mailman (input) for mailman id 683014;
 Mon, 19 Feb 2024 15:56:36 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc60S-0005ce-9x
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 15:56:36 +0000
Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com
 [2a00:1450:4864:20::436])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 74915824-cf3f-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 16:56:34 +0100 (CET)
Received: by mail-wr1-x436.google.com with SMTP id
 ffacd0b85a97d-33d146737e6so2690509f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 07:56:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n6-20020a05600c3b8600b004126a6ee498sm1200844wms.12.2024.02.19.07.56.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 07:56:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 74915824-cf3f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708358193; x=1708962993; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HhzIvAJMiSakWdcNe2qM/9a4+obUP4RnoEcDydP1UWE=;
        b=XlXxTiWHrkftE8QBcTsoqqA4prCH5ivFQFExEkRXkRnhhGFOeFCV4QwlI7pX1AjY1N
         yLr44TC7QowFGwRQouIQAOIjQX8tJR9VQAdB+LnZ1v+vHsgfmoyaJ+KO1WrZ7hJz4THh
         vFW7fXr3qoCZqru9hLngwszQhMLaTkmkBmPtvQUBeOk5AUahcrHSVGMLd7Krqz6MrJPn
         XvRE7YDOngmJN6bZL+rqLG/5TkThi/jA3T2n/wPwPWtGOiPqW2MriKbRKsYBZE2MzZeK
         Sks0zT4oJj0Gc5bji4aYGWoSKN35klQd8xRkAMzTjVOzPpRf11ATyiMLlwG7FYZlWBzd
         qhBw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708358193; x=1708962993;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HhzIvAJMiSakWdcNe2qM/9a4+obUP4RnoEcDydP1UWE=;
        b=DSx1LUTlnVVBk+uZ811g4ehynQ8UsB1sEeTIMf/bBrWo8AKHKnQWdS0SO0AGOmeoqS
         dAVF29ST8BEPOLB18rWt1sBIN92Fio0rNAKw8W2PwwQ0GGl1vDabZB2SwBuMHgt2Oses
         9vZXT00Htfd67MRCYnOebl0jtcdzsxRkQeiNw5ZHOonH+fdsQuW6YupWOYoB6WvWYjiD
         g+Dx3b0yhGQZ92syrLBqXOi4GqNcBEHcJENQJtzk3BRhIsKVBuKyfDt0eu02MxTYtEml
         dgkldlxNRfaCk7KLIiLJY7OCRh7rZhBAQfCNMciGzk4NR3x/DERs1BMZCcR1JleQEEtO
         5rkw==
X-Gm-Message-State: AOJu0Ywfl0WiZ8z1ug85Fdac3btYHYvWyAJ2oE5M5LuSJ1V1h7reQK6q
	7xplTwQH6oCKQ4l6P+rPYyX2d2NSXgrCOp9Wul3XltxFxgY/ePrJj1hwalVHIA==
X-Google-Smtp-Source: AGHT+IEKwQIuUouN3taz5um1itvhh1DN42BegZ52gZdb9Jmilkn2XmJUJtBdcdW21wMtIlw/ESms5Q==
X-Received: by 2002:adf:ec02:0:b0:33d:2b23:5c4f with SMTP id x2-20020adfec02000000b0033d2b235c4fmr5537380wrn.66.1708358193676;
        Mon, 19 Feb 2024 07:56:33 -0800 (PST)
Message-ID: <801618fe-73fe-4d15-8e17-dd02e115cdf8@suse.com>
Date: Mon, 19 Feb 2024 16:56:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/6] nestedsvm: Remove bogus debug message from
 nestedsvm_check_intercepts
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-6-george.dunlap@cloud.com>
Cc: xen-devel@lists.xenproject.org
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240206012051.3564035-6-george.dunlap@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 02:20, George Dunlap wrote:
> Changeset ef3e8db8068 ("x86/hvm: Corrections and improvements to
> unhandled vmexit logging") introduced a printk to the default path of
> the switch statement in nestedsvm_check_intercepts(), complaining of
> an unknown exit reason.
> 
> Unfortunately, the "core" switch statement which is meant to handle
> all vmexit reasons is in nsvm_vmcb_guest_intercepts_exitcode(); the
> switch statement in nestedsvm_check_intercepts() is only meant to
> superimpose on top of that some special-casing for how to interaction
> between L1 and L0 vmexits.
> 
> Remove the printk, and add a comment to prevent future confusion.
> 
> Signed-off-by: George Dunlap <george.dunlap@cloud.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

I wonder if a Fixes: tag is warranted here.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 16:08:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 16:08:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683019.1062329 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc6Bv-0000AO-E5; Mon, 19 Feb 2024 16:08:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683019.1062329; Mon, 19 Feb 2024 16:08:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc6Bv-0000AH-Aa; Mon, 19 Feb 2024 16:08:27 +0000
Received: by outflank-mailman (input) for mailman id 683019;
 Mon, 19 Feb 2024 16:08:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc6Bu-0000AB-8m
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 16:08:26 +0000
Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com
 [2a00:1450:4864:20::430])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1c4f5b7b-cf41-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 17:08:25 +0100 (CET)
Received: by mail-wr1-x430.google.com with SMTP id
 ffacd0b85a97d-33d375993f4so900309f8f.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 08:08:25 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bt21-20020a056000081500b0033d1f25b798sm10491777wrb.82.2024.02.19.08.08.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 08:08:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c4f5b7b-cf41-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708358904; x=1708963704; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=eGyhoNCsN0VjJKCpVFi0p3s4cXPK1xeHiJyUBzBdSZ8=;
        b=eI3rfWnveYKHX51rnQ2aNNvOZC+XIJzW9XZEvVoPomzS5r4I63Nx8OIPXOu2D7gZxf
         sELVsj2Wk+6MM9GH506aaoNplNjKIbR8kKQHSjVyym4/9ls5g6avZlFAkWZGUbwCOTW2
         tPMk4Ia68jUfnNNVZX8aa8xidgN1Wkh8RjfEnnF4yeD+NZxWjQK+cil3zmtovYnnc0Xa
         K/FYNjFdeA5ONbKw2FxM2eWBBPockcjB+PYMkjeSDiu/5YLxPPdr33NjU6jYbxpWJ3tU
         p0yq/gpJ1+snBeat3YzLo/b7KawedmDSfeLAAHIajzfT0sAkqxh2ncPGRbmtgZIQHu0+
         h8mA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708358904; x=1708963704;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=eGyhoNCsN0VjJKCpVFi0p3s4cXPK1xeHiJyUBzBdSZ8=;
        b=v17IvzQj7sPGzkz4Rk+tRwTt4y2muKbqwDo40qvhM5RwSB66Y+RfM7l4MLyBsN3p/0
         1f5HO76Dm+9/MX0y4kEoPK6Z3mO5nwZp0oK7HauAiBlxLuEJ4zlgYjO1CKCheh1O+ZtV
         8DruWJXRMzXSlsaTOyfhuAloI6xMwTpHpn3Unde7dHJxIedx/bIC7TDZncgHFjG2Mg3o
         NE/3hTHM8piTAKNRanQLOZ8l2hsGboh6ufkvSuBUzGUvkCblM9gICnabMdhmScBHl06y
         qrzsJbHOydnLmOkcqenya/gkZvp/nlkGdqx0eCUIROsbQg1CVpjhgoHa3plhJItcptnr
         V1zg==
X-Forwarded-Encrypted: i=1; AJvYcCXVxcK1igEHYh7RtaAhZDg1/KXWN4NjFImSXpACLYHsK7745qfUSUa7oKLZ9xyp9Qyfln4fdnMypM35oYH57bfhvLCCBeZaVsvUULUhnVs=
X-Gm-Message-State: AOJu0YxGZO4vYElowLzpk+obwCLkxM7GKzBUI6qkQc2lwypkpEL77s5a
	i0pqiDqJlYSo9tkHp/1/i04l2M4J6qyloH2Yor4OdQleblyY3+F7K6VCKzCWLw==
X-Google-Smtp-Source: AGHT+IFbKdqg15TUi8M6wCUX9+ImM6WyX0HUV+RhP43wcofJ7nild58zo9bHMVUw0VLN+ZkAfGGcUg==
X-Received: by 2002:a05:6000:1e86:b0:33d:34a9:902f with SMTP id dd6-20020a0560001e8600b0033d34a9902fmr3955786wrb.36.1708358904618;
        Mon, 19 Feb 2024 08:08:24 -0800 (PST)
Message-ID: <046575d2-bc39-4b0e-a656-c501ee1ae4e6@suse.com>
Date: Mon, 19 Feb 2024 17:08:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/6] xen/hvm: Move other hvm_function_table booleans into
 the caps bitfield
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Jun Nakajima <jun.nakajima@intel.com>,
 Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-4-george.dunlap@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240206012051.3564035-4-george.dunlap@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 02:20, George Dunlap wrote:
> --- a/xen/arch/x86/include/asm/hvm/hvm.h
> +++ b/xen/arch/x86/include/asm/hvm/hvm.h
> @@ -86,20 +86,19 @@ struct hvm_vcpu_nonreg_state {
>  struct hvm_function_table {
>      const char *name;
>  
> -    /* Support Hardware-Assisted Paging? */
> -    bool hap_supported;
> -
> -    /* Necessary hardware support for alternate p2m's? */
> -    bool altp2m_supported;
> -    bool singlestep_supported;
> -
> -    /* Hardware virtual interrupt delivery enable? */
> -    bool virtual_intr_delivery_enabled;
> -
>      struct {
>          /* Indicate HAP capabilities. */
> -        bool hap_superpage_1gb:1,
> -            hap_superpage_2mb:1;
> +        bool hap:1,
> +             hap_superpage_1gb:1,
> +             hap_superpage_2mb:1,
> +
> +            /* Altp2m capabilities */
> +            altp2m:1,
> +            singlestep:1,
> +            
> +            /* Hardware virtual interrupt delivery enable? */
> +            virtual_intr_delivery;
> +
>      } caps;

Nit (spotted only while looking at patch 6): You're adding a stray blank
line at the end of the structure. Further I expect virtual_intr_delivery
would also want to be just a single bit?

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 16:25:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 16:25:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683026.1062339 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc6SX-0003rK-Tq; Mon, 19 Feb 2024 16:25:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683026.1062339; Mon, 19 Feb 2024 16:25:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc6SX-0003rD-RG; Mon, 19 Feb 2024 16:25:37 +0000
Received: by outflank-mailman (input) for mailman id 683026;
 Mon, 19 Feb 2024 16:25:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=SL20=J4=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rc6SW-0003r7-K8
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 16:25:36 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 82452733-cf43-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 17:25:35 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-4125cf71eecso14527265e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 08:25:35 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 y21-20020a05600c20d500b0041044e130bfsm11613758wmm.33.2024.02.19.08.25.34
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 08:25:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82452733-cf43-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708359934; x=1708964734; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+KyLwGj6vFDAiGJSuAGHwa4trOZDeOAdto4sZpeY4CE=;
        b=F0Rm3OZJBDned01k9hZNv9nWCV/xot8KZ1VQBAVqRdSeY+YfWnw9TDMX+oKdfr+XFH
         t8H/TcuOs6Ncm8x5+h38QgGGB7v2byzACmkGCnC6L1g/7waWDM1RQ89VRHr/APnWHAcF
         l3DYvlG4gzhFecCIPOua+M+GJP7mnk+gZCp9T8gNApZDR5oPgPOr7Auwm4ooTD0Kpeup
         yDIhRRRjBLRCu+991CfsOak6YKmNkP5HObtL3gfTfMbi03+f0woOLjI8Wqwi4D/NFatQ
         F9xAEWZhMkYLA0pcS/EcXpJqGMXI6B43zi/ZRgZryoju+Z1M50vy9QiwdeFjEh8XOCgw
         YDgg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708359934; x=1708964734;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+KyLwGj6vFDAiGJSuAGHwa4trOZDeOAdto4sZpeY4CE=;
        b=CV2DxcfWHUMPAbPksQYDWBbMLK7sjNxp7xm+je8j8M940MNn9Xn9oStZOL73/rPsdM
         7tGkl7Z0yJsfK0IKnrilKw/HD5MBGGF90M6dILFV5owzpbxW4cf4jaFgwc4vu04qHpQW
         eEhZziod5iZtCTYHFpujde5QdnUfu9R2Z2xax2kPD3VkAOCMSaCh21d5OpvwsYhmKrve
         BGB+2bvh67ZWtD+K5XhcWV1w23zC9+8yi0z7P5SLng89oZGfvc3qnQl2Ld56cDWbATBM
         nMs8Kg9tqUPDSgViAFMqMH8+KilM4pmGRFVCj5BqM3lK7pM/E03AmuBUmZmCbxbornw9
         /Umg==
X-Forwarded-Encrypted: i=1; AJvYcCWuR6qYHbhI6nqI4I790iNlCcX03oNmfXbjCwyWuq8PWLI2cdsYWqB5vnzfiLj0sUOxkJZNKJwbrgmFcXepmxHo4HKJP7B4thcOLlWyHLY=
X-Gm-Message-State: AOJu0YwFDSUKVH6ktQUnzFPR5s+LxkVwq+g0QrbVw053os8Qrxj4Pfxe
	mOhpE7E64kqpi7p4CCx1nPtjrJK4/xMiYZn8I9WZSbrA3kS8FX7oJ9HQ+X4SAQ==
X-Google-Smtp-Source: AGHT+IGzRA0VBYAkCzGHilQZlODScbSJu0Gt9LaysinnC4MnlqMGhkXyrllUcjGk/8usWkIynsJu2g==
X-Received: by 2002:a05:600c:4588:b0:411:e0f0:7a71 with SMTP id r8-20020a05600c458800b00411e0f07a71mr9902390wmo.33.1708359934636;
        Mon, 19 Feb 2024 08:25:34 -0800 (PST)
Message-ID: <8ee35193-9715-4c4b-a1cb-11af04c4dab9@suse.com>
Date: Mon, 19 Feb 2024 17:25:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 6/6] svm/nestedvm: Introduce nested capabilities bit
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-7-george.dunlap@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240206012051.3564035-7-george.dunlap@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 06.02.2024 02:20, George Dunlap wrote:
> --- /dev/null
> +++ b/docs/designs/nested-svm-cpu-features.md
> @@ -0,0 +1,110 @@
> +# Nested SVM (AMD) CPUID requirements
> +
> +The first step in making nested SVM production-ready is to make sure
> +that all features are implemented and well-tested.  To make this
> +tractable, we will initially be limiting the "supported" range of
> +nested virt to a specific subset of host and guest features.  This
> +document describes the criteria for deciding on features, and the
> +rationale behind each feature.
> +
> +For AMD, all virtualization-related features can be found in CPUID
> +leaf 8000000A:edx
> +
> +# Criteria
> +
> +- Processor support: At a minimum we want to support processors from
> +  the last 5 years.  All things being equal, older processors are
> +  better.

Nit: Perhaps missing "covering"? Generally I hope newer processors are
"better".

>  Bits 0:7 were available in the very earliest processors;
> +  and even through bit 15 we should be pretty good support-wise.
> +
> +- Faithfulness to hardware: We need the behavior of the "virtual cpu"
> +  from the L1 hypervisor's perspective to be as close as possible to
> +  the original hardware.  In particular, the behavior of the hardware
> +  on error paths 1) is not easy to understand or test, 2) can be the
> +  source of surprising vulnerabiliies.  (See XSA-7 for an example of a
> +  case where subtle error-handling differences can open up a privilege
> +  escalation.)  We should avoid emulating any bit of the hardware with
> +  complex error paths if we can at all help it.
> +
> +- Cost of implementation: We want to minimize the cost of
> +  implementation (where this includes bringing an existing sub-par
> +  implementation up to speed).  All things being equal, we'll favor a
> +  configuration which does not require any new implementation.
> +
> +- Performance: All things being equal, we'd prefer to choose a set of
> +  L0 / L1 CPUID bits that are faster than slower.
> +
> +
> +# Bits
> +
> +- 0 `NP` *Nested Paging*: Required both for L0 and L1.
> +
> +  Based primarily on faithfulness and performance, as well as
> +  potential cost of implementation.  Available on earliest hardware,
> +  so no compatibility issues.
> +
> +- 1 `LbrVirt` *LBR / debugging virtualization*: Require for L0 and L1.
> +
> +  For L0 this is required for performance: There's no way to tell the
> +  guests not to use the LBR-related registers; and if the guest does,
> +  then you have to save and restore all LBR-related registers on
> +  context switch, which is prohibitive.

"prohibitive" is too strong imo; maybe "undesirable"?

> --- a/xen/arch/x86/hvm/svm/nestedhvm.h
> +++ b/xen/arch/x86/hvm/svm/nestedhvm.h
> @@ -35,6 +35,7 @@ enum nestedhvm_vmexits
>  nestedsvm_check_intercepts(struct vcpu *v, struct cpu_user_regs *regs,
>      uint64_t exitcode);
>  void svm_nested_features_on_efer_update(struct vcpu *v);
> +void __init start_nested_svm(struct hvm_function_table *svm_function_table);

No section placement attributes on declarations, please.

> --- a/xen/arch/x86/hvm/svm/nestedsvm.c
> +++ b/xen/arch/x86/hvm/svm/nestedsvm.c
> @@ -1666,3 +1666,17 @@ void svm_nested_features_on_efer_update(struct vcpu *v)
>          }
>      }
>  }
> +
> +void __init start_nested_svm(struct hvm_function_table *svm_function_table)
> +{
> +    /* 
> +     * Required host functionality to support nested virt.  See
> +     * docs/designs/nested-svm-cpu-features.md for rationale.
> +     */
> +    svm_function_table->caps.nested_virt =
> +        cpu_has_svm_nrips &&
> +        cpu_has_svm_lbrv &&
> +        cpu_has_svm_nrips &&

nrips twice? Was the earlier one meant to be npt?

> --- a/xen/arch/x86/hvm/vmx/vmx.c
> +++ b/xen/arch/x86/hvm/vmx/vmx.c
> @@ -3021,6 +3021,9 @@ const struct hvm_function_table * __init start_vmx(void)
>      if ( cpu_has_vmx_tsc_scaling )
>          vmx_function_table.tsc_scaling.ratio_frac_bits = 48;
>  
> +    /* TODO: Require hardware support before enabling nested virt */
> +    vmx_function_table.caps.nested_virt = vmx_function_table.caps.hap;

This won't have the intended effect if hap_supported() ends up clearing
the bit (used as input here) due to a command line option override. I
wonder if instead this wants doing e.g. in a new hook to be called from
nestedhvm_setup(). On the SVM side the hook function would then be the
start_nested_svm() that you already introduce, with a caps.hap check
added.

Since you leave the other nested-related if() in place in
arch_sanitise_domain_config(), all ought to be well, but I think that
other if() really wants replacing by the one you presently add.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 17:52:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 17:52:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683039.1062355 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc7oN-0007QE-1x; Mon, 19 Feb 2024 17:52:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683039.1062355; Mon, 19 Feb 2024 17:52:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc7oM-0007Q7-VQ; Mon, 19 Feb 2024 17:52:14 +0000
Received: by outflank-mailman (input) for mailman id 683039;
 Mon, 19 Feb 2024 17:52:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NWwU=J4=apertussolutions.com=dpsmith@srs-se1.protection.inumbo.net>)
 id 1rc7oL-0007Q1-Ba
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 17:52:13 +0000
Received: from sender4-of-o50.zoho.com (sender4-of-o50.zoho.com
 [136.143.188.50]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9aa8d64f-cf4f-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 18:52:11 +0100 (CET)
Received: from [10.10.1.138] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1708365125938180.04530272200873;
 Mon, 19 Feb 2024 09:52:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9aa8d64f-cf4f-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; t=1708365127; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=Zt2wcJXe+Uv2oiS/Iifc1MUwJwR1StTVZ+TbSqX+G9xrgqNtWuVr744AIty7y32SvFy/at5Fy9Va9YRYJgSCxq1iHlRb8xGH10RBofDb6NMzpThSbIK7JrteB7b8CHvNgxkoa8EG4vFEkXO42MYiqnwgssxhmRhYAOJ6lnYWCKo=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1708365127; h=Content-Type:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=TNQCLfA9htVpZRkeHZ4Mx7K1oaIJNW/iB3pExkroJRY=; 
	b=HRjVfZApxr8sa+U/svyywHMuwEaQwBaKdFa6risaXSIzTN7bOh0Q5coBjm4lOAjYRtzGMo1eva6bJMhWZ/eKukIzGLuQBYFWXCvhezrQRhk4yBFvYfOkCPoxW1Nzi4AGMqK/4AuScK5k2AAcgCUNlxTGmSpVhx4YKBWFLUsRqho=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1708365127;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Content-Type:Message-ID:Date:Date:MIME-Version:Subject:Subject:To:To:Cc:Cc:References:From:From:In-Reply-To:Message-Id:Reply-To;
	bh=TNQCLfA9htVpZRkeHZ4Mx7K1oaIJNW/iB3pExkroJRY=;
	b=FbSPP8GyIpQO0ZnpcWjSs7Eo2bJwmNMboPij0I+co++OpDNk8Yr33pPRBi08M/Gw
	87dzIhJd9DG8Z8liY9SjAACt2x8g1RnVG9BxRdGfw7Klz5JzqpaXaEEnqu6tpJKqA5K
	k5eqfWvDlG/72Pjj3QIyy5w9ag+44vbmPda1/McM=
Content-Type: multipart/alternative;
 boundary="------------Ho0E0BJPwP6Bi1nFFQEaejQ2"
Message-ID: <b6bf7430-80d6-4d02-a032-ed08a5ee1b10@apertussolutions.com>
Date: Mon, 19 Feb 2024 12:52:03 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] Constify some parameters
Content-Language: en-US
To: Frediano Ziglio <frediano.ziglio@cloud.com>,
 xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 George Dunlap <george.dunlap@citrix.com>, Dario Faggioli <dfaggioli@suse.com>
References: <20240214094734.13533-1-frediano.ziglio@cloud.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Autocrypt: addr=dpsmith@apertussolutions.com; keydata=
 xsJuBFYrueARCACPWL3r2bCSI6TrkIE/aRzj4ksFYPzLkJbWLZGBRlv7HQLvs6i/K4y/b4fs
 JDq5eL4e9BdfdnZm/b+K+Gweyc0Px2poDWwKVTFFRgxKWq9R7McwNnvuZ4nyXJBVn7PTEn/Z
 G7D08iZg94ZsnUdeXfgYdJrqmdiWA6iX9u84ARHUtb0K4r5WpLUMcQ8PVmnv1vVrs/3Wy/Rb
 foxebZNWxgUiSx+d02e3Ad0aEIur1SYXXv71mqKwyi/40CBSHq2jk9eF6zmEhaoFi5+MMMgX
 X0i+fcBkvmT0N88W4yCtHhHQds+RDbTPLGm8NBVJb7R5zbJmuQX7ADBVuNYIU8hx3dF3AQCm
 601w0oZJ0jGOV1vXQgHqZYJGHg5wuImhzhZJCRESIwf+PJxik7TJOgBicko1hUVOxJBZxoe0
 x+/SO6tn+s8wKlR1Yxy8gYN9ZRqV2I83JsWZbBXMG1kLzV0SAfk/wq0PAppA1VzrQ3JqXg7T
 MZ3tFgxvxkYqUP11tO2vrgys+InkZAfjBVMjqXWHokyQPpihUaW0a8mr40w9Qui6DoJj7+Gg
 DtDWDZ7Zcn2hoyrypuht88rUuh1JuGYD434Q6qwQjUDlY+4lgrUxKdMD8R7JJWt38MNlTWvy
 rMVscvZUNc7gxcmnFUn41NPSKqzp4DDRbmf37Iz/fL7i01y7IGFTXaYaF3nEACyIUTr/xxi+
 MD1FVtEtJncZNkRn7WBcVFGKMAf+NEeaeQdGYQ6mGgk++i/vJZxkrC/a9ZXme7BhWRP485U5
 sXpFoGjdpMn4VlC7TFk2qsnJi3yF0pXCKVRy1ukEls8o+4PF2JiKrtkCrWCimB6jxGPIG3lk
 3SuKVS/din3RHz+7Sr1lXWFcGYDENmPd/jTwr1A1FiHrSj+u21hnJEHi8eTa9029F1KRfocp
 ig+k0zUEKmFPDabpanI323O5Tahsy7hwf2WOQwTDLvQ+eqQu40wbb6NocmCNFjtRhNZWGKJS
 b5GrGDGu/No5U6w73adighEuNcCSNBsLyUe48CE0uTO7eAL6Vd+2k28ezi6XY4Y0mgASJslb
 NwW54LzSSM0uRGFuaWVsIFAuIFNtaXRoIDxkcHNtaXRoQGFwZXJ0dXNzb2x1dGlvbnMuY29t
 PsJ6BBMRCAAiBQJWK7ngAhsjBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAAKCRBTc6WbYpR8
 KrQ9AP94+xjtFfJ8gj5c7PVx06Zv9rcmFUqQspZ5wSEkvxOuQQEAg6qEsPYegI7iByLVzNEg
 7B7fUG7pqWIfMqFwFghYhQzOwU0EViu54BAIAL6MXXNlrJ5tRUf+KMBtVz1LJQZRt/uxWrCb
 T06nZjnbp2UcceuYNbISOVHGXTzu38r55YzpkEA8eURQf+5hjtvlrOiHxvpD+Z6WcpV6rrMB
 kcAKWiZTQihW2HoGgVB3gwG9dCh+n0X5OzliAMiGK2a5iqnIZi3o0SeW6aME94bSkTkuj6/7
 OmH9KAzK8UnlhfkoMg3tXW8L6/5CGn2VyrjbB/rcrbIR4mCQ+yCUlocuOjFCJhBd10AG1IcX
 OXUa/ux+/OAV9S5mkr5Fh3kQxYCTcTRt8RY7+of9RGBk10txi94dXiU2SjPbassvagvu/hEi
 twNHms8rpkSJIeeq0/cAAwUH/jV3tXpaYubwcL2tkk5ggL9Do+/Yo2WPzXmbp8vDiJPCvSJW
 rz2NrYkd/RoX+42DGqjfu8Y04F9XehN1zZAFmCDUqBMa4tEJ7kOT1FKJTqzNVcgeKNBGcT7q
 27+wsqbAerM4A0X/F/ctjYcKwNtXck1Bmd/T8kiw2IgyeOC+cjyTOSwKJr2gCwZXGi5g+2V8
 NhJ8n72ISPnOh5KCMoAJXmCF+SYaJ6hIIFARmnuessCIGw4ylCRIU/TiXK94soilx5aCqb1z
 ke943EIUts9CmFAHt8cNPYOPRd20pPu4VFNBuT4fv9Ys0iv0XGCEP+sos7/pgJ3gV3pCOric
 p15jV4PCYQQYEQgACQUCViu54AIbDAAKCRBTc6WbYpR8Khu7AP9NJrBUn94C/3PeNbtQlEGZ
 NV46Mx5HF0P27lH3sFpNrwD/dVdZ5PCnHQYBZ287ZxVfVr4Zuxjo5yJbRjT93Hl0vMY=
In-Reply-To: <20240214094734.13533-1-frediano.ziglio@cloud.com>
X-ZohoMailClient: External

This is a multi-part message in MIME format.
--------------Ho0E0BJPwP6Bi1nFFQEaejQ2
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

For XSM changes,

Ack-by: Daniel P. Smith <dpsmith@apertussolutions.com>

V/r,
Daniel P. Smith
Apertus Solutions, LLC

On 2/14/24 04:47, Frediano Ziglio wrote:
> Make clear they are not changed in the functions.
>
> Signed-off-by: Frediano Ziglio<frediano.ziglio@cloud.com>
> Reviewed-by: Jan Beulich<jbeulich@suse.com>
> --
> v2:
> - fixed typo in commit message.
> ---
>   xen/arch/x86/pv/callback.c | 4 ++--
>   xen/common/sched/compat.c  | 2 +-
>   xen/common/sched/core.c    | 2 +-
>   xen/xsm/flask/flask_op.c   | 8 ++++----
>   4 files changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/xen/arch/x86/pv/callback.c b/xen/arch/x86/pv/callback.c
> index 17829304fe..caec4fb16f 100644
> --- a/xen/arch/x86/pv/callback.c
> +++ b/xen/arch/x86/pv/callback.c
> @@ -48,7 +48,7 @@ static void unregister_guest_nmi_callback(void)
>       memset(t, 0, sizeof(*t));
>   }
>   
> -static long register_guest_callback(struct callback_register *reg)
> +static long register_guest_callback(const struct callback_register *reg)
>   {
>       long ret = 0;
>       struct vcpu *curr = current;
> @@ -102,7 +102,7 @@ static long register_guest_callback(struct callback_register *reg)
>       return ret;
>   }
>   
> -static long unregister_guest_callback(struct callback_unregister *unreg)
> +static long unregister_guest_callback(const struct callback_unregister *unreg)
>   {
>       long ret;
>   
> diff --git a/xen/common/sched/compat.c b/xen/common/sched/compat.c
> index dd97593630..a02204ec9a 100644
> --- a/xen/common/sched/compat.c
> +++ b/xen/common/sched/compat.c
> @@ -26,7 +26,7 @@ CHECK_sched_shutdown;
>   CHECK_sched_remote_shutdown;
>   #undef xen_sched_remote_shutdown
>   
> -static int compat_poll(struct compat_sched_poll *compat)
> +static int compat_poll(const struct compat_sched_poll *compat)
>   {
>       struct sched_poll native;
>   
> diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
> index d177c675c8..c5db373972 100644
> --- a/xen/common/sched/core.c
> +++ b/xen/common/sched/core.c
> @@ -1431,7 +1431,7 @@ static void vcpu_block_enable_events(void)
>       vcpu_block();
>   }
>   
> -static long do_poll(struct sched_poll *sched_poll)
> +static long do_poll(const struct sched_poll *sched_poll)
>   {
>       struct vcpu   *v = current;
>       struct domain *d = v->domain;
> diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
> index b866e8d05f..ea7dd10dc8 100644
> --- a/xen/xsm/flask/flask_op.c
> +++ b/xen/xsm/flask/flask_op.c
> @@ -469,7 +469,7 @@ static int flask_security_load(struct xen_flask_load *load)
>       return ret;
>   }
>   
> -static int flask_devicetree_label(struct xen_flask_devicetree_label *arg)
> +static int flask_devicetree_label(const struct xen_flask_devicetree_label *arg)
>   {
>       int rv;
>       char *buf;
> @@ -492,7 +492,7 @@ static int flask_devicetree_label(struct xen_flask_devicetree_label *arg)
>   
>   #ifndef COMPAT
>   
> -static int flask_ocontext_del(struct xen_flask_ocontext *arg)
> +static int flask_ocontext_del(const struct xen_flask_ocontext *arg)
>   {
>       int rv;
>   
> @@ -506,7 +506,7 @@ static int flask_ocontext_del(struct xen_flask_ocontext *arg)
>       return security_ocontext_del(arg->ocon, arg->low, arg->high);
>   }
>   
> -static int flask_ocontext_add(struct xen_flask_ocontext *arg)
> +static int flask_ocontext_add(const struct xen_flask_ocontext *arg)
>   {
>       int rv;
>   
> @@ -550,7 +550,7 @@ static int flask_get_peer_sid(struct xen_flask_peersid *arg)
>       return rv;
>   }
>   
> -static int flask_relabel_domain(struct xen_flask_relabel *arg)
> +static int flask_relabel_domain(const struct xen_flask_relabel *arg)
>   {
>       int rc;
>       struct domain *d;
--------------Ho0E0BJPwP6Bi1nFFQEaejQ2
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit

<!DOCTYPE html>
<html data-lt-installed="true">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>For XSM changes,<br>
      <br>
      Ack-by: Daniel P. Smith <a class="moz-txt-link-rfc2396E" href="mailto:dpsmith@apertussolutions.com">&lt;dpsmith@apertussolutions.com&gt;</a><br>
    </p>
    <pre class="moz-signature" cols="72">V/r,
Daniel P. Smith
Apertus Solutions, LLC</pre>
    <div class="moz-cite-prefix">On 2/14/24 04:47, Frediano Ziglio
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:20240214094734.13533-1-frediano.ziglio@cloud.com">
      <pre class="moz-quote-pre" wrap="">Make clear they are not changed in the functions.

Signed-off-by: Frediano Ziglio <a class="moz-txt-link-rfc2396E" href="mailto:frediano.ziglio@cloud.com">&lt;frediano.ziglio@cloud.com&gt;</a>
Reviewed-by: Jan Beulich <a class="moz-txt-link-rfc2396E" href="mailto:jbeulich@suse.com">&lt;jbeulich@suse.com&gt;</a>
--
v2:
- fixed typo in commit message.
---
 xen/arch/x86/pv/callback.c | 4 ++--
 xen/common/sched/compat.c  | 2 +-
 xen/common/sched/core.c    | 2 +-
 xen/xsm/flask/flask_op.c   | 8 ++++----
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/pv/callback.c b/xen/arch/x86/pv/callback.c
index 17829304fe..caec4fb16f 100644
--- a/xen/arch/x86/pv/callback.c
+++ b/xen/arch/x86/pv/callback.c
@@ -48,7 +48,7 @@ static void unregister_guest_nmi_callback(void)
     memset(t, 0, sizeof(*t));
 }
 
-static long register_guest_callback(struct callback_register *reg)
+static long register_guest_callback(const struct callback_register *reg)
 {
     long ret = 0;
     struct vcpu *curr = current;
@@ -102,7 +102,7 @@ static long register_guest_callback(struct callback_register *reg)
     return ret;
 }
 
-static long unregister_guest_callback(struct callback_unregister *unreg)
+static long unregister_guest_callback(const struct callback_unregister *unreg)
 {
     long ret;
 
diff --git a/xen/common/sched/compat.c b/xen/common/sched/compat.c
index dd97593630..a02204ec9a 100644
--- a/xen/common/sched/compat.c
+++ b/xen/common/sched/compat.c
@@ -26,7 +26,7 @@ CHECK_sched_shutdown;
 CHECK_sched_remote_shutdown;
 #undef xen_sched_remote_shutdown
 
-static int compat_poll(struct compat_sched_poll *compat)
+static int compat_poll(const struct compat_sched_poll *compat)
 {
     struct sched_poll native;
 
diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index d177c675c8..c5db373972 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -1431,7 +1431,7 @@ static void vcpu_block_enable_events(void)
     vcpu_block();
 }
 
-static long do_poll(struct sched_poll *sched_poll)
+static long do_poll(const struct sched_poll *sched_poll)
 {
     struct vcpu   *v = current;
     struct domain *d = v-&gt;domain;
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index b866e8d05f..ea7dd10dc8 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -469,7 +469,7 @@ static int flask_security_load(struct xen_flask_load *load)
     return ret;
 }
 
-static int flask_devicetree_label(struct xen_flask_devicetree_label *arg)
+static int flask_devicetree_label(const struct xen_flask_devicetree_label *arg)
 {
     int rv;
     char *buf;
@@ -492,7 +492,7 @@ static int flask_devicetree_label(struct xen_flask_devicetree_label *arg)
 
 #ifndef COMPAT
 
-static int flask_ocontext_del(struct xen_flask_ocontext *arg)
+static int flask_ocontext_del(const struct xen_flask_ocontext *arg)
 {
     int rv;
 
@@ -506,7 +506,7 @@ static int flask_ocontext_del(struct xen_flask_ocontext *arg)
     return security_ocontext_del(arg-&gt;ocon, arg-&gt;low, arg-&gt;high);
 }
 
-static int flask_ocontext_add(struct xen_flask_ocontext *arg)
+static int flask_ocontext_add(const struct xen_flask_ocontext *arg)
 {
     int rv;
 
@@ -550,7 +550,7 @@ static int flask_get_peer_sid(struct xen_flask_peersid *arg)
     return rv;
 }
 
-static int flask_relabel_domain(struct xen_flask_relabel *arg)
+static int flask_relabel_domain(const struct xen_flask_relabel *arg)
 {
     int rc;
     struct domain *d;
</pre>
    </blockquote>
  </body>
  <lt-container></lt-container>
</html>

--------------Ho0E0BJPwP6Bi1nFFQEaejQ2--


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 18:17:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 18:17:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683046.1062372 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8CK-0001tt-6a; Mon, 19 Feb 2024 18:17:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683046.1062372; Mon, 19 Feb 2024 18:17:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8CK-0001tR-1H; Mon, 19 Feb 2024 18:17:00 +0000
Received: by outflank-mailman (input) for mailman id 683046;
 Mon, 19 Feb 2024 18:16:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DP9f=J4=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1rc8CJ-0001rg-5p
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 18:16:59 +0000
Received: from wfout5-smtp.messagingengine.com
 (wfout5-smtp.messagingengine.com [64.147.123.148])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 10a6b543-cf53-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 19:16:58 +0100 (CET)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfout.west.internal (Postfix) with ESMTP id 55BAA1C000A1;
 Mon, 19 Feb 2024 13:16:55 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Mon, 19 Feb 2024 13:16:55 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 19 Feb 2024 13:16:53 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 10a6b543-cf53-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:in-reply-to:message-id:mime-version:references:reply-to:subject
	:subject:to:to; s=fm3; t=1708366614; x=1708453014; bh=RDKr5KOEyU
	1P8HpgXnlUzz0XOtyo12r8pUxs5poAuEI=; b=zuCssHKJsXrqqdpwdr9qMqSihV
	HJCmLV+5lv6v6kePzbI2GZYLyzLs/4OT0rct2ONgqs1JGp8XO7QKisx+9aHaYG28
	N33bWum9KxSS6LQmDbDDW/TZSMyO/CT427ViZVzal5xRNnZyP8z4hNXI0g2soJ1w
	RsyR1nSHBetSEXP6Xoas6XWuw+4rsEp4BJm1q9Nv/TaLQ2CbqLI8A5uDAdClTpSZ
	NSlRZH4G6lKdJk9RP2Eloi1rA6K+2aUHaSmidPheFD/mLuvaG5cuSze2tGSRZa2n
	pvDg/Ja9Dr+q62N/D168YGLYAPb4w7uigVx7u9J+jix1AvwElqheC5mhIEIA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
	:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1708366614; x=
	1708453014; bh=RDKr5KOEyU1P8HpgXnlUzz0XOtyo12r8pUxs5poAuEI=; b=h
	gMR+6RhKHwa2e2m2gYM0oq0AWmtRxJLoYYcSrU73SNDH5uzaqT/IVAwFUnt7uuKm
	qAoZhQeg0BLjQJtTn3YLiTdxtrydSwN/nvY8F/wkS2/8YZUlryaam4gGrsLHE0bY
	TxMLTGH4wK8p5sxshGNr/yfB0pgNidxLujfYH1ZmCxBV2IrGe7C49uQQMAP5rC/T
	ujd5r6+avwgnlw677EYDZOJXOH+SWHAeLNic4egCQMdNh+7+nSn59GgKIUB+dzA2
	EUdiD3bT3Al/shDZ2oP93iS9U3JPSHjWvESHwnQ966lEJKH7XKGWzK6cUlEZ5Zv+
	bKC7oeF6YQQWEnmabGsHA==
X-ME-Sender: <xms:FpvTZY7WhVT3Z6agUZFsoA-Qatkr3h36vo3uEWkDYsTY4I96v_86QQ>
    <xme:FpvTZZ4KnnL6m9bioHZYUQ_n4UqqhFInCw-csAdpqAQHeUo4Bf6nhl7Flr952J0xP
    fgyw7X-G2qr0A>
X-ME-Received: <xmr:FpvTZXfR3I9hADHysYXteWjwUeH8CEwb_Od9miHKD6TcKRFfk3SPLDxQHGt2spqErG9b_bFIwibqT0m0UpByYDFyiLrWX_ctAe_gAVRx28TRDTITyEg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdekgddutdelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr
    vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih
    hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg
    ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh
    hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv
    khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:FpvTZdKFzTVfMSVz2plmDB5X91E1l6WFTHsSWp2hcbJcWgZ59ca1kA>
    <xmx:FpvTZcJqmEQUm86f6ArVCAHuqp9ROBFM7Sa_oxjetksiIdFqscQ--g>
    <xmx:FpvTZexcjkx6BLdzEy1WdUeGikx644qGEpNiF6zAeIxs4lbVXzz3vg>
    <xmx:FpvTZU_7baUyXe-ka8eYsk5-kgavH_kt7pef17snKnPX9-JUzSM3OrsBLis>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: qemu-devel@nongnu.org
Cc: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20Pierret=20=28fepitre=29?= <frederic.pierret@qubes-os.org>,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org (open list:X86 Xen CPUs)
Subject: [PATCH 2/2] xen: fix stubdom PCI addr
Date: Mon, 19 Feb 2024 19:16:06 +0100
Message-ID: <20240219181627.282097-2-marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240219181627.282097-1-marmarek@invisiblethingslab.com>
References: <20240219181627.282097-1-marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

From: Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org>

When running in a stubdomain, the config space access via sysfs needs to
use BDF as seen inside stubdomain (connected via xen-pcifront), which is
different from the real BDF. For other purposes (hypercall parameters
etc), the real BDF needs to be used.
Get the in-stubdomain BDF by looking up relevant PV PCI xenstore
entries.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 hw/xen/xen-host-pci-device.c | 77 +++++++++++++++++++++++++++++++++++-
 hw/xen/xen-host-pci-device.h |  6 +++
 2 files changed, 82 insertions(+), 1 deletion(-)

diff --git a/hw/xen/xen-host-pci-device.c b/hw/xen/xen-host-pci-device.c
index 8c6e9a1716..3f8a6f84a8 100644
--- a/hw/xen/xen-host-pci-device.c
+++ b/hw/xen/xen-host-pci-device.c
@@ -9,6 +9,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "qemu/cutils.h"
+#include "hw/xen/xen-legacy-backend.h"
 #include "xen-host-pci-device.h"
 
 #define XEN_HOST_PCI_MAX_EXT_CAP \
@@ -33,13 +34,76 @@
 #define IORESOURCE_PREFETCH     0x00001000      /* No side effects */
 #define IORESOURCE_MEM_64       0x00100000
 
+static void xen_host_pci_fill_local_addr(XenHostPCIDevice *d, Error **errp)
+{
+    unsigned int num_devs, len, i;
+    unsigned int domain, bus, dev, func;
+    char *be_path = NULL;
+    char path[80];
+    char *msg = NULL;
+
+    be_path = qemu_xen_xs_read(xenstore, 0, "device/pci/0/backend", &len);
+    if (!be_path)
+        goto err_out;
+    snprintf(path, sizeof(path), "%s/num_devs", be_path);
+    msg = qemu_xen_xs_read(xenstore, 0, path, &len);
+    if (!msg)
+        goto err_out;
+
+    if (sscanf(msg, "%u", &num_devs) != 1) {
+        error_setg(errp, "Failed to parse %s (%s)\n", msg, path);
+        goto err_out;
+    }
+    free(msg);
+
+    for (i = 0; i < num_devs; i++) {
+        snprintf(path, sizeof(path), "%s/dev-%u", be_path, i);
+        msg = qemu_xen_xs_read(xenstore, 0, path, &len);
+        if (!msg) {
+            error_setg(errp, "Failed to read %s\n", path);
+            goto err_out;
+        }
+        if (sscanf(msg, "%x:%x:%x.%x", &domain, &bus, &dev, &func) != 4) {
+            error_setg(errp, "Failed to parse %s (%s)\n", msg, path);
+            goto err_out;
+        }
+        free(msg);
+        if (domain != d->domain ||
+                bus != d->bus ||
+                dev != d->dev ||
+                func!= d->func)
+            continue;
+        snprintf(path, sizeof(path), "%s/vdev-%u", be_path, i);
+        msg = qemu_xen_xs_read(xenstore, 0, path, &len);
+        if (!msg) {
+            error_setg(errp, "Failed to read %s\n", path);
+            goto out;
+        }
+        if (sscanf(msg, "%x:%x:%x.%x", &domain, &bus, &dev, &func) != 4) {
+            error_setg(errp, "Failed to parse %s (%s)\n", msg, path);
+            goto err_out;
+        }
+        free(msg);
+        d->local_domain = domain;
+        d->local_bus = bus;
+        d->local_dev = dev;
+        d->local_func = func;
+        goto out;
+    }
+
+err_out:
+    free(msg);
+out:
+    free(be_path);
+}
+
 static void xen_host_pci_sysfs_path(const XenHostPCIDevice *d,
                                     const char *name, char *buf, ssize_t size)
 {
     int rc;
 
     rc = snprintf(buf, size, "/sys/bus/pci/devices/%04x:%02x:%02x.%d/%s",
-                  d->domain, d->bus, d->dev, d->func, name);
+                  d->local_domain, d->local_bus, d->local_dev, d->local_func, name);
     assert(rc >= 0 && rc < size);
 }
 
@@ -342,6 +406,17 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
     d->dev = dev;
     d->func = func;
 
+    if (xen_is_stubdomain) {
+        xen_host_pci_fill_local_addr(d, errp);
+        if (*errp)
+            goto error;
+    } else {
+        d->local_domain = d->domain;
+        d->local_bus = d->bus;
+        d->local_dev = d->dev;
+        d->local_func = d->func;
+    }
+
     xen_host_pci_config_open(d, errp);
     if (*errp) {
         goto error;
diff --git a/hw/xen/xen-host-pci-device.h b/hw/xen/xen-host-pci-device.h
index 4d8d34ecb0..270dcb27f7 100644
--- a/hw/xen/xen-host-pci-device.h
+++ b/hw/xen/xen-host-pci-device.h
@@ -23,6 +23,12 @@ typedef struct XenHostPCIDevice {
     uint8_t dev;
     uint8_t func;
 
+    /* different from the above in case of stubdomain */
+    uint16_t local_domain;
+    uint8_t local_bus;
+    uint8_t local_dev;
+    uint8_t local_func;
+
     uint16_t vendor_id;
     uint16_t device_id;
     uint32_t class_code;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 19 18:17:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 18:17:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683045.1062365 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8CJ-0001ry-TP; Mon, 19 Feb 2024 18:16:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683045.1062365; Mon, 19 Feb 2024 18:16:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8CJ-0001rr-Qn; Mon, 19 Feb 2024 18:16:59 +0000
Received: by outflank-mailman (input) for mailman id 683045;
 Mon, 19 Feb 2024 18:16:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DP9f=J4=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1rc8CI-0001rg-Gu
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 18:16:58 +0000
Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com
 [64.147.123.21]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0f53f2dc-cf53-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 19:16:55 +0100 (CET)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
 by mailout.west.internal (Postfix) with ESMTP id 722FE3200922;
 Mon, 19 Feb 2024 13:16:52 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
 by compute6.internal (MEProxy); Mon, 19 Feb 2024 13:16:53 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 19 Feb 2024 13:16:50 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f53f2dc-cf53-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:from:from:in-reply-to
	:message-id:mime-version:reply-to:subject:subject:to:to; s=fm3;
	 t=1708366611; x=1708453011; bh=Uo8NGN15Y+wR+Mg2EFtNRg5h410lOfUh
	X9FiMUkUpNM=; b=ePuwMlpkFCfcozI+MhZqTBm/Dg8ocqkEHLyQsWztZqNGpe9s
	nCqPrTjcFqSoUCgm04bnW1Yymew4DGgov8d55FlgtUzGeq5mFqkbCDDdPb74gbRb
	8181nh6MIynBD3h22iGeiLurm0BK3Xcb/FMpihuMHPK2n52+7z4J0t7c98jcLFch
	wqcQDmy/iUeSLMYlPzTc1EtYqrrfflSGCg+UZgLG5GdUPO9badV6jAKGmfTCa7iX
	KCsn/kiovcGBl2CXn2PHaEDDynOt/ncBEThP+jYpdzujAkLb4AqhG+sVC887CwWl
	o2XERoGrbgTVPWr6Dte/6tYGLYUCGZFw2fM+9w==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-transfer-encoding
	:content-type:content-type:date:date:feedback-id:feedback-id
	:from:from:in-reply-to:message-id:mime-version:reply-to:subject
	:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
	:x-sasl-enc; s=fm1; t=1708366611; x=1708453011; bh=Uo8NGN15Y+wR+
	Mg2EFtNRg5h410lOfUhX9FiMUkUpNM=; b=NwiBHl2JlPG4RfBLrLc3Di0dZEGVk
	1sxrKKmwlUQgPG5xaTShd7X6H56R1Ug/GnCCGIZU1P3qBwgQrb3IP1eEchGYJ5c8
	WqeXs9YER/iDyHtzeySPS/sZsXeEwJl1p0sL5z7Qg6tMk9QoWnO1Rovw6reHREUz
	r5/WwlnAPPVlNWqlVCT05NuUaz1FKPbhaLFaOjhJSOhkqVddHZlrUB1hYRzc/qk2
	Oc2EAiiC46G+AQ1WZRQ+dzeJgfAjr1C7gk+8AdJTjg0rwVNqyBqCDoDm7hoDKrw0
	yo5oRXjabMXLNexRNy8Lh3dkxT9NadIWrz5jse8jAGD+gmTFCOIdvnf0Q==
X-ME-Sender: <xms:E5vTZfqMG8b-uLaFuCcGyr3VfYk09m_3_nyKYJJsNy-dLZZOuQcjcg>
    <xme:E5vTZZq-0ST2NthJrPCm1Yh5tCuFztXwviIqJ9s1Dw-pxYbGfzdDMZjS2SuBLAbdF
    uRJ5_ysYr9etg>
X-ME-Received: <xmr:E5vTZcPb5FccXDpLCin3inCl0UXD9OP6FeWVQUtBIpUY2FBK6V57og2HDpI3e3s4pdipexfHOunEGZnoUlCh6WX74CzlkqbIQOCAWPUrN4yWus9DQbQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdekgddutdelucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhephffvvefufffkofggtgfgsehtkeertdertdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpeelkefh
    udelteelleelteetveeffeetffekteetjeehlefggeekleeghefhtdehvdenucevlhhush
    htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhes
    ihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:E5vTZS4wigD3HxxyaUXVa3yrppnf2lVQzErSE97wvRlJ51bq0Oz4tA>
    <xmx:E5vTZe6by-yFSfjQZqBAdQM-IPOa9iatqsDrOjZRjjr_zCrKbYxjyw>
    <xmx:E5vTZajUF2XL490zVSESdjVosKLRrVwYEAqNkzY8vDbh-qqhjcVG4w>
    <xmx:E5vTZZ3vBlgtfMM6kfe-6PiIL-_US0XMQeXbwFkeQvjM4BfuFO71ww>
Feedback-ID: i1568416f:Fastmail
From: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: qemu-devel@nongnu.org
Cc: =?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org (open list:X86 Xen CPUs)
Subject: [PATCH 1/2] hw/xen: detect when running inside stubdomain
Date: Mon, 19 Feb 2024 19:16:05 +0100
Message-ID: <20240219181627.282097-1-marmarek@invisiblethingslab.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Introduce global xen_is_stubdomain variable when qemu is running inside
a stubdomain instead of dom0. This will be relevant for subsequent
patches, as few things like accessing PCI config space need to be done
differently.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 hw/xen/xen-legacy-backend.c | 15 +++++++++++++++
 include/hw/xen/xen.h        |  1 +
 system/globals.c            |  1 +
 3 files changed, 17 insertions(+)

diff --git a/hw/xen/xen-legacy-backend.c b/hw/xen/xen-legacy-backend.c
index 124dd5f3d6..4a09ea2561 100644
--- a/hw/xen/xen-legacy-backend.c
+++ b/hw/xen/xen-legacy-backend.c
@@ -603,6 +603,19 @@ static void xen_set_dynamic_sysbus(void)
     machine_class_allow_dynamic_sysbus_dev(mc, TYPE_XENSYSDEV);
 }
 
+static bool xen_check_stubdomain(void)
+{
+    char *dm_path = g_strdup_printf("/local/domain/%d/image", xen_domid);
+    uint32_t dm_domid;
+    bool is_stubdom = false;
+
+    if (!xenstore_read_int(dm_path, "device-model-domid", &dm_domid))
+        is_stubdom = dm_domid != 0;
+
+    g_free(dm_path);
+    return is_stubdom;
+}
+
 void xen_be_init(void)
 {
     xenstore = qemu_xen_xs_open();
@@ -616,6 +629,8 @@ void xen_be_init(void)
         exit(1);
     }
 
+    xen_is_stubdomain = xen_check_stubdomain();
+
     xen_sysdev = qdev_new(TYPE_XENSYSDEV);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(xen_sysdev), &error_fatal);
     xen_sysbus = qbus_new(TYPE_XENSYSBUS, xen_sysdev, "xen-sysbus");
diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h
index 37ecc91fc3..ecb89ecfc1 100644
--- a/include/hw/xen/xen.h
+++ b/include/hw/xen/xen.h
@@ -36,6 +36,7 @@ enum xen_mode {
 extern uint32_t xen_domid;
 extern enum xen_mode xen_mode;
 extern bool xen_domid_restrict;
+extern bool xen_is_stubdomain;
 
 int xen_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num);
 int xen_set_pci_link_route(uint8_t link, uint8_t irq);
diff --git a/system/globals.c b/system/globals.c
index b6d4e72530..ac27d88bd4 100644
--- a/system/globals.c
+++ b/system/globals.c
@@ -62,6 +62,7 @@ bool qemu_uuid_set;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_DISABLED;
 bool xen_domid_restrict;
+bool xen_is_stubdomain;
 struct evtchn_backend_ops *xen_evtchn_ops;
 struct gnttab_backend_ops *xen_gnttab_ops;
 struct foreignmem_backend_ops *xen_foreignmem_ops;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 19 18:48:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 18:48:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683060.1062385 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8gy-0006jq-LU; Mon, 19 Feb 2024 18:48:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683060.1062385; Mon, 19 Feb 2024 18:48:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8gy-0006jj-IY; Mon, 19 Feb 2024 18:48:40 +0000
Received: by outflank-mailman (input) for mailman id 683060;
 Mon, 19 Feb 2024 18:48:39 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rc8gx-0006jd-Gf
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 18:48:39 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc8gx-000579-41; Mon, 19 Feb 2024 18:48:39 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc8gw-0007HA-QL; Mon, 19 Feb 2024 18:48:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=LK4bgDKWFBvPwLWO9O8T2vSnAaqiuPT/R8E25MJ30iQ=; b=H+Z+yOhvL6EKHdp6KF8IR/KuoK
	puLhKdygOUfM25drmNBBWoB3sT3gSuF+5LtCymrYxgkLL6baiLlpaKp3+cVUbjYPL/6pwMztDVXtr
	5+ZDf2/TGhencSLFKd1h59Lv3MoWzG7ZUdiK3Zx0thsbZKV5zodplEEn71wkelo94u34=;
Message-ID: <aaeca1c4-8898-400f-a608-6ef758ed9d96@xen.org>
Date: Mon, 19 Feb 2024 18:48:37 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v4 2/3] xen/arm: arm64: Add emulation of Debug Data Transfer
 Registers
Content-Language: en-GB
To: Michal Orzel <michal.orzel@amd.com>, Ayan Kumar Halder
 <ayankuma@amd.com>, Ayan Kumar Halder <ayan.kumar.halder@amd.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, luca.fancellu@arm.com
References: <20240131121049.225044-1-ayan.kumar.halder@amd.com>
 <20240131121049.225044-3-ayan.kumar.halder@amd.com>
 <0c4e4a40-7327-4377-bfdd-82e69bc838ae@xen.org>
 <7098c9ab-7008-4a49-92d4-6cd201b1490e@amd.com>
 <9d1a545c-a93e-4f8e-bc76-0b5676fa01ea@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <9d1a545c-a93e-4f8e-bc76-0b5676fa01ea@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Michal,

On 19/02/2024 15:43, Michal Orzel wrote:

> On 19/02/2024 15:45, Ayan Kumar Halder wrote:
>>
>> On 06/02/2024 19:05, Julien Grall wrote:
>>> Hi Ayan,
>> Hi Julien/Michal,
>>>
>>> On 31/01/2024 12:10, Ayan Kumar Halder wrote:
>>>> From: Michal Orzel <michal.orzel@amd.com>
>>>>
>>>> Currently, if user enables HVC_DCC config option in Linux, it invokes
>>>> access
>>>> to debug data transfer registers (i.e. DBGDTRTX_EL0 on arm64,
>>>> DBGDTRTXINT on
>>>> arm32). As these registers are not emulated, Xen injects an undefined
>>>> exception to the guest and Linux crashes.
>>>>
>>>> To prevent this crash, introduce a partial emulation of DBGDTR[TR]X_EL0
>>>> (these registers share the same encoding) as RAZ/WI and MDCCSR_EL0 as
>>>> TXfull.
>>>>
>>>> Refer ARM DDI 0487J.a ID042523, D19.3.8, DBGDTRTX_EL0
>>>> "If TXfull is set to 1, set DTRRX and DTRTX to UNKNOWN".
>>>>
>>>> Thus, any OS is expected to read MDCCSR_EL0 and check for TXfull before
>>>> using DBGDTRTX_EL0. Linux does it via hvc_dcc_init() --->
>>>> hvc_dcc_check(),
>>>> and returns -ENODEV in case TXfull bit is still set after writing a test
>>>> character. This way we prevent the guest from making use of HVC DCC as a
>>>> console.
>>>>
>>>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>>>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
>>>> ---
>>>> Changes from
>>>>
>>>> v1 :- 1. DBGDTR_EL0 does not emulate RXfull. This is to avoid giving
>>>> the OS any
>>>> indication that the RX buffer is full and is waiting to be read.
>>>>
>>>> 2. In Arm32, DBGOSLSR is emulated. Also DBGDTRTXINT is emulated at
>>>> EL0 only.
>>>>
>>>> 3. Fixed the commit message and inline code comments.
>>>>
>>>> v2 :- 1. Split the patch into two (separate patches for arm64 and
>>>> arm32).
>>>> 2. Removed the "fail" label.
>>>> 3. Fixed the commit message.
>>>>
>>>> v3 :- 1. "HSR_SYSREG_MDCCSR_EL0" emulation differs based on whether
>>>> partial_emulation_enabled is true or not.
>>>>
>>>> 2. If partial_emulation_enabled is false, then access to
>>>> HSR_SYSREG_DBGDTR_EL0,
>>>> HSR_SYSREG_DBGDTRTX_EL0 would lead to undefined exception.
>>>>
>>>>    xen/arch/arm/arm64/vsysreg.c         | 28 ++++++++++++++++++++++++----
>>>>    xen/arch/arm/include/asm/arm64/hsr.h |  3 +++
>>>>    2 files changed, 27 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>>>> index b5d54c569b..94f0a6c384 100644
>>>> --- a/xen/arch/arm/arm64/vsysreg.c
>>>> +++ b/xen/arch/arm/arm64/vsysreg.c
>>>> @@ -159,9 +159,6 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>>         *
>>>>         * Unhandled:
>>>>         *    MDCCINT_EL1
>>>> -     *    DBGDTR_EL0
>>>> -     *    DBGDTRRX_EL0
>>>> -     *    DBGDTRTX_EL0
>>>>         *    OSDTRRX_EL1
>>>>         *    OSDTRTX_EL1
>>>>         *    OSECCR_EL1
>>>> @@ -173,10 +170,32 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>>            return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
>>>>        case HSR_SYSREG_MDCCSR_EL0:
>>>>            /*
>>>> +         * Xen doesn't expose a real (or emulated) Debug
>>>> Communications Channel
>>>> +         * (DCC) to a domain. Yet the Arm ARM implies this is not an
>>>> optional
>>>> +         * feature. So some domains may start to probe it. For
>>>> instance, the
>>>> +         * HVC_DCC driver in Linux (since f377775dc083 and at least
>>>> up to v6.7),
>>>> +         * will try to write some characters and check if the
>>>> transmit buffer
>>>> +         * has emptied.
>>>> +         *
>>>> +         * By setting TX status bit (only if partial emulation is
>>>> enabled) to
>>>> +         * indicate the transmit buffer is full, we would hint the
>>>> OS that the
>>>> +         * DCC is probably not working.
>>>> +         *
>>>> +         * Bit 29: TX full
>>>> +         *
>>>>             * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We
>>>> emulate that
>>>>             * register as RAZ/WI above. So RO at both EL0 and EL1.
>>>
>>> The sentence "we emulate that register as ..." seems to be stale?
> I can see that you tried to handle Julien remark here. But I disagree. This statement
> is not stale. It explains that because MDSCR_EL1 is RAZ/WI, MDCCSR_EL0 is RO at both
> EL0 and EL1. This patch does not change this behavior.

Indeed. I misread the comment. So what I wrote can be ignored here.

> 
>>>
>>>>             */
>>>> -        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 0);
>>>> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read,
>>>> hsr, 0,
>>>> +                                  partial_emulation ? (1U << 29) : 0);
>>>> +
>>>> +    case HSR_SYSREG_DBGDTR_EL0:
>>>> +    /* DBGDTR[TR]X_EL0 share the same encoding */
>>>> +    case HSR_SYSREG_DBGDTRTX_EL0:
>>>> +        if ( !partial_emulation )
>>>> +            goto fail;
>>>> +        return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 0);
>>>
>>> AFAICT, all the emulation helpers have an explanation why we are using
>>> them. But here this is not the case. Can you add one?
>> This and..
>>>
>>>> +
>>>>        HSR_SYSREG_DBG_CASES(DBGBVR):
>>>>        HSR_SYSREG_DBG_CASES(DBGBCR):
>>>>        HSR_SYSREG_DBG_CASES(DBGWVR):
>>>> @@ -394,6 +413,7 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>>         * And all other unknown registers.
>>>>         */
>>>>        default:
>>>> + fail:
>>>
>>> AFAICT, this would violate MISRA 15.3 [1]. We didn't seem to have yet
>>> (?) accepted the rule, but I don't see we would not given I feel this
>>> is similar to what Rule 16.2 is trying to prevent and we accepted it.
>>>
>>> I think case, I move all the code within default outside. And then
>>> call "goto fail" from the default label.
>>
>> I am not sure if I have interpreted this correctly.
>>
>> Is it ok if you can take a look at the attached patch and let me know if
>> the explaination and the code change looks sane ?
> Looking at the attached patch and fail handling, I don't think it is what Julien meant.
> In the default case you should jump to fail that would be defined outside of switch clause.
> 
> Something like:
>      default:
>          goto fail;
>      }
> 
>      regs->pc += 4;
>      return;
> 
> fail:
>      gdprintk...

+1.

> 
> When it comes to explanation for HSR_SYSREG_DBGDTRTX_EL0, I will let Julien to provide a comment he believes is right.
> To me, it feels strange to repeat almost the same information as for MDCCSR_EL0.

It is not clear to me whether you are objecting of adding a comment or 
whether you would be ok with a comment that is not duplicating.

If the latter, you could move the first paragraph outside of the case 
and then have a register specific explanation of the implementation.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 18:54:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 18:54:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683066.1062395 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8mE-0008Tu-7z; Mon, 19 Feb 2024 18:54:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683066.1062395; Mon, 19 Feb 2024 18:54:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8mE-0008Tn-51; Mon, 19 Feb 2024 18:54:06 +0000
Received: by outflank-mailman (input) for mailman id 683066;
 Mon, 19 Feb 2024 18:54:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rc8mC-0008Tf-Lm
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 18:54:04 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc8m7-0005D4-DZ; Mon, 19 Feb 2024 18:53:59 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc8m7-0007N8-3G; Mon, 19 Feb 2024 18:53:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=6SriAx32xfN0JE6QolJyWzGOt83J/1pLDjR8pM5iAzc=; b=uaTK9AqEjUKXyHkcPHcnR9g5eA
	qzm97Ns0NIb7GIuFIBaXKcB0yVLum9Yk0a1tdqcFrmBwJkSK8fnPkT1xRhd9h3ZGcDKsWSB1oYgdH
	rYxlJsvmAThYTBIeXqw3a8OLN7wy0A/wd+JN+MsTH3RWWWS+nTZ0CPyFwgpebpqBKebs=;
Message-ID: <a11912c0-1135-4c0d-a5c1-d4396d5a4579@xen.org>
Date: Mon, 19 Feb 2024 18:53:57 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 7/7] xen/asm-generic: fold struct devarch into struct
 dev
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Rahul Singh <rahul.singh@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
 <3a5bf394a9d95a28cecac996f6e0decb788c19fd.1708086092.git.oleksii.kurochko@gmail.com>
 <92e7c679-fbf5-4f2e-9b0d-ba61a198832c@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <92e7c679-fbf5-4f2e-9b0d-ba61a198832c@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 19/02/2024 11:26, Jan Beulich wrote:
> On 16.02.2024 13:39, Oleksii Kurochko wrote:
>> The current patch is a follow-up to the patch titled:
>>      xen/asm-generic: introduce generic device.h
>> Also, a prerequisite for this patch is, without which a compilation
>> error will occur:
>>      xen/arm: switch Arm to use asm-generic/device.h
>>
>> The 'struct dev_archdata' is exclusively used within 'struct device',
>> so it could be merged into 'struct device.'
>>
>> After the merger, it is necessary to update the 'dev_archdata()'
>> macros and the comments above 'struct arm_smmu_xen_device' in
>> drivers/passthrough/arm/smmu.c.
>> Additionally, it is required to update instances of
>> "dev->archdata->iommu" to "dev->iommu".
>>
>> Suggested-by: Julien Grall <julien@xen.org>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> ---
>>    This patch can be merged with patches 4 and 5 of this patch series.
>> ---
>> Changes in V9:
>>   - newly introduced patch.
>> ---
>>   xen/drivers/passthrough/arm/smmu.c | 12 ++++++------
>>   xen/include/asm-generic/device.h   |  8 +-------
>>   2 files changed, 7 insertions(+), 13 deletions(-)
>>
>> diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
>> index 32e2ff279b..4a272c8779 100644
>> --- a/xen/drivers/passthrough/arm/smmu.c
>> +++ b/xen/drivers/passthrough/arm/smmu.c
>> @@ -227,9 +227,9 @@ struct arm_smmu_xen_domain {
>>   };
>>   
>>   /*
>> - * Xen: Information about each device stored in dev->archdata.iommu
>> + * Xen: Information about each device stored in dev->iommu
>>    *
>> - * Initially dev->archdata.iommu only stores the iommu_domain (runtime
>> + * Initially dev->iommu only stores the iommu_domain (runtime
>>    * configuration of the SMMU) but, on Xen, we also have to store the
>>    * iommu_group (list of streamIDs associated to the device).
>>    *
>> @@ -242,7 +242,7 @@ struct arm_smmu_xen_device {
>>   	struct iommu_group *group;
>>   };
>>   
>> -#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->archdata.iommu)
>> +#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->iommu)
> 
> I find in particular the naming here odd. But I'll let Julien judge whether
> this really is along the lines of what he suggested.

It is. The naming is not great, but the SMMU code is intended to stay 
close to Linux. So we want to do the minimum amount of change (at least 
until we decide we should diverge).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 19:00:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 19:00:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683072.1062406 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8sV-0001wN-UM; Mon, 19 Feb 2024 19:00:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683072.1062406; Mon, 19 Feb 2024 19:00:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8sV-0001wG-RJ; Mon, 19 Feb 2024 19:00:35 +0000
Received: by outflank-mailman (input) for mailman id 683072;
 Mon, 19 Feb 2024 19:00:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rc8sV-0001w7-F0
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 19:00:35 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc8sU-0005Nu-6e; Mon, 19 Feb 2024 19:00:34 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc8sT-0007vg-US; Mon, 19 Feb 2024 19:00:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=zZlXVdLR7KZEi35q3pmO8jPghbVHhV84TtuMxosMGYg=; b=xSuRSWzReqnqr7OjPPJxqsuvha
	pHCH8vdAhyhb705AzqXCTBnRRwLU99erZQXhPkkZwcsaX7NBWQuEjPtkygmWt6+BJ8LLwsJjKocXp
	iIkxumgHx8cutg/vNXg9AQlVg41B9qPYdlQPMDzF2h63V8piwRz6dtAym4M5wGXLbaLw=;
Message-ID: <7a2ed74d-a773-4dc1-8331-a1ee28b17f4a@xen.org>
Date: Mon, 19 Feb 2024 19:00:31 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 7/7] xen/asm-generic: fold struct devarch into struct
 dev
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Rahul Singh <rahul.singh@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
 <3a5bf394a9d95a28cecac996f6e0decb788c19fd.1708086092.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <3a5bf394a9d95a28cecac996f6e0decb788c19fd.1708086092.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 16/02/2024 12:39, Oleksii Kurochko wrote:
> The current patch is a follow-up to the patch titled:
>      xen/asm-generic: introduce generic device.h
> Also, a prerequisite for this patch is, without which a compilation
> error will occur:
>      xen/arm: switch Arm to use asm-generic/device.h
> 
> The 'struct dev_archdata' is exclusively used within 'struct device',
> so it could be merged into 'struct device.'
> 
> After the merger, it is necessary to update the 'dev_archdata()'
> macros and the comments above 'struct arm_smmu_xen_device' in
> drivers/passthrough/arm/smmu.c.
> Additionally, it is required to update instances of
> "dev->archdata->iommu" to "dev->iommu".
> 
> Suggested-by: Julien Grall <julien@xen.org>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>    This patch can be merged with patches 4 and 5 of this patch series.

I am a bit puzzled with this comment. If this is the case, then why was 
it done in a separate patch?

I know I suggested to create the separate patch but this was only in the 
case you decided to handle it after this series is merged. So this 
should have been merged when sending. Maybe I should have been clearer.

Anyway, regardless that:

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 19:04:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 19:04:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683078.1062416 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8w3-0002XK-Dn; Mon, 19 Feb 2024 19:04:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683078.1062416; Mon, 19 Feb 2024 19:04:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8w3-0002XD-AT; Mon, 19 Feb 2024 19:04:15 +0000
Received: by outflank-mailman (input) for mailman id 683078;
 Mon, 19 Feb 2024 19:04:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rc8w2-0002X7-KC
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 19:04:14 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc8w1-0005RH-Qs; Mon, 19 Feb 2024 19:04:13 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc8w1-000857-M2; Mon, 19 Feb 2024 19:04:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=7/1h3QI9c/Ilm0cIOGZSzexY92xMXTkud0URse3Hfv4=; b=fjAnMJx/Wa8cBfwf4CWq0ypWQu
	9fzrgpkkIn/yzh5wfoBNg8ZON9yyEdrJf2D2xLgmpz8F7WqLnQAZhB4ZsHbALqt7vHDrEnQTze4Wu
	TI/9Be9Ff307uLjIDiPsxm2dKSdHULOZuQsKeEJlY8fVMmnDLjNVA1I7AfIZhf16z2AA=;
Message-ID: <d5d5c3a0-94fc-45e8-9b1e-7b32bc648472@xen.org>
Date: Mon, 19 Feb 2024 19:04:11 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 5/7] xen/asm-generic: introduce generic device.h
Content-Language: en-GB
To: Oleksii <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1707499278.git.oleksii.kurochko@gmail.com>
 <4ecb5e5f4a1dbf585ac7a9ebe40b5fd8e4709182.1707499278.git.oleksii.kurochko@gmail.com>
 <319dcf7a-88cd-488e-ae51-6c20c19d8f22@xen.org>
 <9d4f3093de6f95b09e44bc66fe9a9d4b0009dc43.camel@gmail.com>
 <3171ef24-bcb3-4387-ba32-f1c4d6c7220a@xen.org>
 <9fe0221a4c093fb5935ff3a519f2ef7180e569e1.camel@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <9fe0221a4c093fb5935ff3a519f2ef7180e569e1.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Oleksii,

On 15/02/2024 16:54, Oleksii wrote:
>>
>> On 14/02/2024 09:32, Oleksii wrote:
>>> On Tue, 2024-02-13 at 18:09 +0000, Julien Grall wrote:
>>>>> +#ifdef CONFIG_HAS_PASSTHROUGH
>>>>> +    struct iommu_fwspec *iommu_fwspec; /* per-device IOMMU
>>>>> instance data */
>>>>> +#endif
>>>>> +};
>>>>> +
>>>>> +typedef struct device device_t;
>>>>> +
>>>>> +#ifdef CONFIG_HAS_DEVICE_TREE
>>>>> +
>>>>> +#include <xen/device_tree.h>
>>>>> +
>>>>> +#define dev_is_dt(dev)  ((dev)->type == DEV_DT)
>>>>> +
>>>>> +/**
>>>>> + *  device_init - Initialize a device
>>>>> + *  @dev: device to initialize
>>>>> + *  @class: class of the device (serial, network...)
>>>>> + *  @data: specific data for initializing the device
>>>>> + *
>>>>> + *  Return 0 on success.
>>>>> + */
>>>>> +int device_init(struct dt_device_node *dev, enum device_class
>>>>> class,
>>>>> +                const void *data);
>>>>> +
>>>>> +/**
>>>>> + * device_get_type - Get the type of the device
>>>>> + * @dev: device to match
>>>>> + *
>>>>> + * Return the device type on success or DEVICE_ANY on failure
>>>>> + */
>>>>> +enum device_class device_get_class(const struct dt_device_node
>>>>> *dev);
>>>>> +
>>>>> +#define DT_DEVICE_START(name_, namestr_, class_)            \
>>>>> +static const struct device_desc __dev_desc_##name_ __used   \
>>>>> +__section(".dev.info") = {                                  \
>>>>> +    .name = namestr_,                                       \
>>>>> +    .class = class_,
>>>>> +
>>>>> +#define DT_DEVICE_END                                       \
>>>>> +};
>>>>> +
>>>>> +#else /* !CONFIG_HAS_DEVICE_TREE */
>>>>> +#define dev_is_dt(dev) ((void)(dev), false)
>>>>> +#endif /* CONFIG_HAS_DEVICE_TREE */
>>>>> +
>>>>> +#define dev_is_pci(dev) ((dev)->type == DEV_PCI)
>>>>> +
>>>>> +struct device_desc {
>>>>> +    /* Device name */
>>>>> +    const char *name;
>>>>> +    /* Device class */
>>>>> +    enum device_class class;
>>>>> +
>>>>> +#ifdef CONFIG_HAS_DEVICE_TREE
>>>>> +
>>>>> +    /* List of devices supported by this driver */
>>>>> +    const struct dt_device_match *dt_match;
>>>>> +    /*
>>>>> +     * Device initialization.
>>>>> +     *
>>>>> +     * -EAGAIN is used to indicate that device probing is
>>>>> deferred.
>>>>> +     */
>>>>> +    int (*init)(struct dt_device_node *dev, const void *data);
>>>>> +
>>>>> +#endif
>>>>> +};
>>>> I am not sure I fully understand why "device_desc" is not
>>>> protected
>>>> by
>>>> CONFIG_HAS_DEVICE_TREE. The structure doesn't mean much when the
>>>> config
>>>> is disabled. Can you clarify?
>>> I thought that one day struct device_desc and acpi_device_desc will
>>> be
>>> "merged", and so decided just to #ifdef only DEVICE_TREE specific
>>> fields.
>>
>> It might be possible to merge the two if we were using an union for
>> the
>> ACPI/DT specific part. However the majority of the parsing code needs
>> to
>> differ. So I am not convinced there would be any value to merge the
>> two
>> structures.
> In this case, let's have two separate structures.
> 
> This is not the current situation, and I don't have a specific example.
> It appears that all architectures will use Device Tree or ACPI.
> However, does it make sense to keep 'struct device_desc' more generic
> to accommodate non-DT or non-ACPI cases?

I am not entirely sure what else to say. As I wrote before yes it could 
be made generic. But right now I don't see any values.

If you have any idea how to share the structure. Then feel free to make 
a proposal and I will review it.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 19:05:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 19:05:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683084.1062425 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8xT-00037J-Qa; Mon, 19 Feb 2024 19:05:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683084.1062425; Mon, 19 Feb 2024 19:05:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rc8xT-00037C-Nr; Mon, 19 Feb 2024 19:05:43 +0000
Received: by outflank-mailman (input) for mailman id 683084;
 Mon, 19 Feb 2024 19:05:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rc8xS-00036z-18
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 19:05:42 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc8xR-0005UC-5g; Mon, 19 Feb 2024 19:05:41 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rc8xR-000857-0S; Mon, 19 Feb 2024 19:05:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=0KJuC9KyCWK2y5681dj3Jxyb6MB4RCF9ApnXeZ4RnfY=; b=t0WxupyH5gTW8U8Qwdbit8uvtQ
	OPWWm7DCzPZNkNektb/zn7o3KZkW+Ip3kYiLyK0GjHrIQcfprzCqsUs8LphFqDocfuaaeYCOaEeH4
	rHcsqh5ji/1QpSjNihRwXqPbhX8XIeg5kxBRL/+bZLoOqXOCGvSoY4LksDecajOzj2is=;
Message-ID: <39b68eb9-1873-4ecc-8790-25d61ef8c98f@xen.org>
Date: Mon, 19 Feb 2024 19:05:40 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 4/7] xen/asm-generic: introduce generic device.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
 <e6df4fde7de451d089ac680535f44a7206313e15.1708086092.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <e6df4fde7de451d089ac680535f44a7206313e15.1708086092.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 16/02/2024 12:39, Oleksii Kurochko wrote:
> Arm, PPC and RISC-V introduce the same things in asm/device.h, so
> generic device.h was introduced.
> Arm's device.h was taken as a base with the following changes:
>   - #ifdef ACPI related things.
>   - Rename #ifdef guards.
>   - Add SPDX tag.
>   - #ifdef CONFIG_HAS_DEVICE_TREE related things.
>   - #ifdef-ing iommu related things with CONFIG_HAS_PASSTHROUGH.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 20:43:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 20:43:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683103.1062436 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcAUA-0000S8-MI; Mon, 19 Feb 2024 20:43:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683103.1062436; Mon, 19 Feb 2024 20:43:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcAUA-0000S1-IT; Mon, 19 Feb 2024 20:43:34 +0000
Received: by outflank-mailman (input) for mailman id 683103;
 Mon, 19 Feb 2024 20:43:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rUMX=J4=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rcAU9-0000Rv-AV
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 20:43:33 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 898f104b-cf67-11ee-98f5-efadbce2ee36;
 Mon, 19 Feb 2024 21:43:30 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 52C5FCE13D5;
 Mon, 19 Feb 2024 20:43:26 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 761A7C433F1;
 Mon, 19 Feb 2024 20:43:24 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 898f104b-cf67-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708375405;
	bh=etgfJdMjjdz4/PTBfMtqsev8L5RcYHGrkiaTddoYzQg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VH9YRttP2wW8bM7gCwQbVEhQXuBzQmbxIPcM/9OOTNmefTkvrmTw9iE3RklPQkwlM
	 Vjqr1mGOVPwJB6DoUJe119Rqy2Xbkl+uKjagrHbfUauGgu49NUnhn2MYfg+Y9hVnDk
	 iqcoixTSqysrkHL/8RV9Nv8rNCSfdDQ/ECjF+piz45S/njvzLbNZZkS1QvyvDWcNsk
	 cHR/kDa49XIcFM2RIrS/mWGhCCG6KHeWY72C3iYjupH+7lkInEA93NUmGBCpDlGEv+
	 yx9PvrWAm3TZujXXWUbOAZampJCeiG0XWKmhdZK1AbS1b4svxWWAWoDHo94tD09p5g
	 P0IwNhDwAjrmg==
Date: Mon, 19 Feb 2024 12:43:23 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: Jan Beulich <jbeulich@suse.com>, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
In-Reply-To: <56b4a4bb-d3e8-4a21-8c0d-d4ee10c8f9b3@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402191243090.155645@ubuntu-linux-20-04-desktop>
References: <cover.1706259490.git.federico.serafini@bugseng.com> <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop> <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com> <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com> <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com> <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com> <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com> <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com> <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com> <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com> <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com> <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
 <45c19a87-5833-4350-a374-3a70fd3aa9ae@suse.com> <d735e980-c5a5-46b1-a511-cc2d666dac23@bugseng.com> <123c446c-e56f-457e-9cf4-58a43a3b069a@suse.com> <56b4a4bb-d3e8-4a21-8c0d-d4ee10c8f9b3@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 19 Feb 2024, Federico Serafini wrote:
> On 15/02/24 11:32, Jan Beulich wrote:
> > The important difference is: Here we're told that there was a use of
> > __put_user_bad, which is easy to grep for, and thus see how the
> > supplied function / file / line(?) relate to the ultimate problem.
> > 
> > I'm afraid I'm meanwhile confused enough by the various replies
> > containing results of experimentation that I can't really tell
> > anymore what case is best. Hence I can only restate my expectation for
> > an eventual v3: Diagnosing what the issue is, no matter whether the new
> > macro is used in another macro or in an inline function, should not
> > become meaningfully more difficult. In how far this is the case wants
> > clarifying in the description of the change.
> 
> I think the best thing at the moment is to deviate
> __{get,put}_user_bad() for Rule 16.3.
> I'll let maintainers further explore the possibility of having a
> compile-time assertion based on the assembler error.

OK. I hope Jan is OK to deviate by in-code comment.


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 21:50:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 21:50:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683110.1062446 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcBWb-0000gw-Fa; Mon, 19 Feb 2024 21:50:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683110.1062446; Mon, 19 Feb 2024 21:50:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcBWb-0000gp-Cg; Mon, 19 Feb 2024 21:50:09 +0000
Received: by outflank-mailman (input) for mailman id 683110;
 Mon, 19 Feb 2024 21:50:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcBWZ-0000gI-RM
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 21:50:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcBWY-0000Pl-Nh; Mon, 19 Feb 2024 21:50:06 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcBWY-0000ve-Gw; Mon, 19 Feb 2024 21:50:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=T+ua4vRYlqr9kVihPeu+zRr5UflKETSelYwOUFT1PQE=; b=sApXZCNR9vfCG6FWzEuB5SoqN1
	85NIGWIjIblv3m/aVJZxBqNfElg2w7aOsaXnfpWlH1h432qs1oCQqEwSznWiDziZStefkLL3fIa4A
	8Vy2qstyzcE0Gp37s2ply7uKI6ZigMCoM64CF4Qxa1+RLIjENJPd3PdO/OKwtAyFnL1g=;
Message-ID: <102e4fc2-2ced-42d9-9937-a900a79f4993@xen.org>
Date: Mon, 19 Feb 2024 21:50:04 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] gnttab: fully ignore zero-size copy requests
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
 <a73f9ba8-8914-4c41-be55-a57c93c571f0@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <a73f9ba8-8914-4c41-be55-a57c93c571f0@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 05/02/2024 11:03, Jan Beulich wrote:
> Along the line with observations in the context of XSA-448, no field in
> struct gnttab_copy_ptr is relevant when no data is to be copied, much
> like e.g. the pointers passed to memcpy() are irrelevant (and would
> never be "validated") when the passed length is zero.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 22:12:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 22:12:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683116.1062456 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcBrw-0003iY-6d; Mon, 19 Feb 2024 22:12:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683116.1062456; Mon, 19 Feb 2024 22:12:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcBrw-0003iR-3j; Mon, 19 Feb 2024 22:12:12 +0000
Received: by outflank-mailman (input) for mailman id 683116;
 Mon, 19 Feb 2024 22:12:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcBru-0003iF-VQ; Mon, 19 Feb 2024 22:12:10 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcBru-0000sd-7R; Mon, 19 Feb 2024 22:12:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcBrt-0001U2-Ks; Mon, 19 Feb 2024 22:12:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcBrt-00050j-KQ; Mon, 19 Feb 2024 22:12:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=wVQRaH+qPzGJ2WgJPXTOOUwr6vT4s0xQeJm7G0xQd9Q=; b=eLIjwHpZCyeHyCi7uVjfdNVkeW
	Tw4YtUvzeuUx+NwdTnjjIE1lVBcBuPBbfJRS7BdzAf1mYAgDXfxfQA0Px0iBqTxmboFKwp0Mcxw2b
	t9nyPEy1ebBoRfoKg/2YxkfN0baxnLUhKYDgbcutCunPlwzQNDUV7JvQKzuM4+1VL8jE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184703-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184703: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-dom0pvh-xl-amd:guest-start:fail:regression
    xen-unstable:build-arm64-pvops:kernel-build:fail:regression
    xen-unstable:test-arm64-arm64-examine:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
X-Osstest-Versions-That:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 19 Feb 2024 22:12:09 +0000

flight 184703 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184703/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-dom0pvh-xl-amd 14 guest-start           fail REGR. vs. 184699
 build-arm64-pvops             6 kernel-build             fail REGR. vs. 184699

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-examine      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184699
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184699
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184699
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184699
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184699
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184699
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184699
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184699
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184699
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184699
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184699
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184699
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
baseline version:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3

Last test of basis   184699  2024-02-19 01:52:12 Z    0 days
Testing same since   184703  2024-02-19 14:38:55 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Frediano Ziglio <frediano.ziglio@cloud.com>
  Julien Grall <jgrall@amazon.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            fail    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          blocked 
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  blocked 
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  blocked 
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     blocked 
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 blocked 
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      blocked 
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
Author: Frediano Ziglio <frediano.ziglio@cloud.com>
Date:   Mon Feb 19 12:46:21 2024 +0100

    x86: Reduce assembly code size of entry points
    
    On many entries we push 8-bytes zero and exception constants are
    small so we can just write a single byte saving 3 bytes for
    instruction.
    With ENDBR64 this reduces the size of many entry points from 32 to
    16 bytes (due to alignment).
    The push and the mov are overlapping stores either way.  Swapping
    between movl and movb will make no difference at all on performance.
    Similar code is already used in autogen_stubs.
    
    Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
    Reviewed-by: Jan Beulich <jbeulich@suse.com>

commit 1eb013ef972408020013a6fb16a36c5a345c6ff7
Author: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Date:   Mon Feb 19 12:46:07 2024 +0100

    libxl: Add "grant_usage" parameter for virtio disk devices
    
    Allow administrators to control whether Xen grant mappings for
    the virtio disk devices should be used. By default (when new
    parameter is not specified), the existing behavior is retained
    (we enable grants if backend-domid != 0).
    
    Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

commit f93629b18b528a5ab1b1092949c5420069c7226c
Author: Anthony PERARD <anthony.perard@citrix.com>
Date:   Mon Feb 19 12:45:48 2024 +0100

    build: Replace `which` with `command -v`
    
    The `which` command is not standard, may not exist on the build host,
    or may not behave as expected by the build system. It is recommended
    to use `command -v` to find out if a command exist and have its path,
    and it's part of a POSIX shell standard (at least, it seems to be
    mandatory since IEEE Std 1003.1-2008, but was optional before).
    
    Fixes: c8a8645f1efe ("xen/build: Automatically locate a suitable python interpreter")
    Fixes: 3b47bcdb6d38 ("xen/build: Use a distro version of figlet")
    Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
    Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
    Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Reviewed-by: Jan Beulich <jbeulich@suse.com>

commit 4917b9a423cbb09a44403c6a0d292a4baf85b93c
Author: Roger Pau Monné <roger.pau@citrix.com>
Date:   Mon Feb 19 12:44:50 2024 +0100

    mm: add the __must_check attribute to {gfn,mfn,dfn}_add()
    
    It's not obvious from just the function name whether the incremented value will
    be stored in the parameter, or returned to the caller.  That has leads to bugs
    in the past as callers may assume the incremented value is stored in the
    parameter.
    
    Add the __must_check attribute to the function to easily spot callers that
    don't consume the returned value, which signals an error in the caller logic.
    
    No functional change intended.
    
    Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
    Acked-by: Julien Grall <jgrall@amazon.com>
    Reviewed-by: Jan Beulich <jbeulich@suse.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 22:22:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 22:22:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683125.1062468 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcC22-0005WO-A7; Mon, 19 Feb 2024 22:22:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683125.1062468; Mon, 19 Feb 2024 22:22:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcC22-0005WH-7S; Mon, 19 Feb 2024 22:22:38 +0000
Received: by outflank-mailman (input) for mailman id 683125;
 Mon, 19 Feb 2024 22:22:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcC21-0005W9-1x
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 22:22:37 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcC20-00017N-7M; Mon, 19 Feb 2024 22:22:36 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcC20-0004cj-0F; Mon, 19 Feb 2024 22:22:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=JJoNPuNJb1f0baOXVQ3UJzTLMq3ACZ6Sa57X+1/XLwY=; b=s0Kq6qYWYmE/ueZd89Hwrbsaj2
	3rCh0YUAMnWezjwIyLDMWG2MfTdv8q2QDNLBKAmy6BAKYEOPmgCjeywku06+NW/bBUYY1sMk3oM5A
	1ikiMWB58vraLvifUToXDj2NGu4E1IYWE5cjT73KzRm343qmRZdrdOZRWRaZC6gbMEp0=;
Message-ID: <7dd78c97-b82f-4ae1-a257-98f612c6bad2@xen.org>
Date: Mon, 19 Feb 2024 22:22:34 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] almost fully ignore zero-size flush requests
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
 <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

Title: I would add 'gnttab:' to clarify which subsystem you are modifying.

On 05/02/2024 11:03, Jan Beulich wrote:
> Along the line with observations in the context of XSA-448, besides
> "op" no field is relevant when the range to be flushed is empty, much
> like e.g. the pointers passed to memcpy() are irrelevant (and would
> never be "validated") when the passed length is zero. Split the existing
> condition validating "op", "offset", and "length", leaving only the "op"
> part ahead of the check for length being zero (or no flushing to be
> performed).

I am probably missing something here. I understand the theory behind 
reducing the number of checks when len == 0. But an OS cannot rely on it:
   1) older hypervisor would still return an error if the check doesn't 
pass)
   2) it does feel odd to allow "invalid" offset when len == 0 (at least.

So to me, it is better to keep those checks early. That said, I agree 
this is a matter of opinion, so I will not Nack it but also I will not 
Ack it.

> 
> In the course of splitting also simplify the moved part of the condition
> from 3 to 2 conditionals, potentially (depending on the architecture)
> requiring one less (conditional) branch.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -3528,15 +3528,16 @@ static int _cache_flush(const gnttab_cac
>       void *v;
>       int ret;
>   
> -    if ( (cflush->offset >= PAGE_SIZE) ||
> -         (cflush->length > PAGE_SIZE) ||
> -         (cflush->offset + cflush->length > PAGE_SIZE) ||
> -         (cflush->op & ~(GNTTAB_CACHE_INVAL | GNTTAB_CACHE_CLEAN)) )
> +    if ( cflush->op & ~(GNTTAB_CACHE_INVAL | GNTTAB_CACHE_CLEAN) )
>           return -EINVAL;
>   
>       if ( cflush->length == 0 || cflush->op == 0 )
>           return !*cur_ref ? 0 : -EILSEQ;
>   
> +    if ( (cflush->offset | cflush->length) > PAGE_SIZE ||

This is confusing. I understand you are trying to force the compiler to 
optimize. But is it really worth it? After all, the rest of operation 
will outweight this check (cache flush are quite expensive).

We probably should take a more generic decision (and encode in our 
policy) because you seem to like this pattern and I dislike it :). Not 
sure what the others think.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 22:24:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 22:24:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683129.1062479 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcC3Q-00064V-Kv; Mon, 19 Feb 2024 22:24:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683129.1062479; Mon, 19 Feb 2024 22:24:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcC3Q-00064O-Hq; Mon, 19 Feb 2024 22:24:04 +0000
Received: by outflank-mailman (input) for mailman id 683129;
 Mon, 19 Feb 2024 22:24:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DP9f=J4=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1rcC3P-00064E-QX
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 22:24:04 +0000
Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com
 [64.147.123.20]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9425d5b1-cf75-11ee-8a52-1f161083a0e0;
 Mon, 19 Feb 2024 23:24:02 +0100 (CET)
Received: from compute7.internal (compute7.nyi.internal [10.202.2.48])
 by mailout.west.internal (Postfix) with ESMTP id 1A4BE3200A06;
 Mon, 19 Feb 2024 17:23:58 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute7.internal (MEProxy); Mon, 19 Feb 2024 17:23:58 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 19 Feb 2024 17:23:56 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9425d5b1-cf75-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1708381437;
	 x=1708467837; bh=v5KkPOivRpp66kSB6p7Aw+l5wab5MAOcIyIXRWi4RPc=; b=
	SeWkSHMDpbQV5qgpPZfuuPRyOD2/0iHieACLat5EcdACjy0SPmZqqGbBPT5p8e3a
	wlVV6mX14pPSw7n2B4W9PqulLD9HHH825NJFcDgEnJraNpqr8675XZ3aSo8oyXCp
	t4dzEcNXmlU9upYRVXj/PmxDuC/zxKeXPEDwvf2+DvPLBzzLswYtI0RRCcX/sCcv
	TVhrqvp945ltBWdzsu7DWwG9nzVqmWK6HC3HrOTi3TPZYQfnzkdHiGTWc18YBLgJ
	J7HwnEwOjDnFaLmuRB4u+K2/7IcFzsvmDmcCbcjMxs60oBfjmpR2CXSJFPZSQEZm
	lo8uBRrcQnL5J9/KY8GR2g==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1708381437; x=1708467837; bh=v5KkPOivRpp66kSB6p7Aw+l5wab5
	MAOcIyIXRWi4RPc=; b=KUJ7SNc9cJ2RQab3m0ECrAf/Zn3Rcrk2ZUz6QobMEDtu
	YK7oMvLRR7ChaxHzzMue+Bj6K7tXMg2MsfA5qgqSO7KmlFYxpf6VXBEmZcIsR2Dj
	/ixAoXcMVMKrQ7R5lBvVQO+EdpxZU1cbf0Hwu8JGCE0EswsLGzIfHgdbf/VB5kwA
	PFYnsa9ZJqxWv9xTrIj7qjRr+mL5aYxyWx7Bwf4Evg6sxYj+jsMZwZxA+MHDmaWj
	0/XVzU91l/IQOIIv9pOXpoCB79Dk3Z9vrnPl+luVzoZeAGkcPXtZjfUbo7MbsMlk
	rGJnnrN9xOtLj685oO+theGeCG8MuHO/eX4NlEfxZg==
X-ME-Sender: <xms:_dTTZYQyy9fpuLwgHK_1AXv4zzK46ZpJ9_zYm30ky5x9gOvN_-64hA>
    <xme:_dTTZVw2jcIkUbNvGhHl90NvJTAo5VXBiYDsp1o-Y4f_lgglzZVeiXFqEDOfXDP9I
    K-ZcZKPSGwXMw>
X-ME-Received: <xmr:_dTTZV0lJ5E48gMxXKFrcK48tGPIO_I0zR2TlHrRTW4UgyoPoZY1TrKuIRQI26ZmiWhzmMrRGb5Brjjwu4JXMsiu9Gdtuya53Q>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdekgdduheekucetufdoteggodetrfdotf
    fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
    uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
    cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeenucfhrhhomhepofgrrhgv
    khcuofgrrhgtiiihkhhofihskhhiqdfikphrvggtkhhiuceomhgrrhhmrghrvghksehinh
    hvihhsihgslhgvthhhihhnghhslhgrsgdrtghomheqnecuggftrfgrthhtvghrnhepgfdu
    leetfeevhfefheeiteeliefhjefhleduveetteekveettddvgeeuteefjedunecuvehluh
    hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgrrhhmrghrvghk
    sehinhhvihhsihgslhgvthhhihhnghhslhgrsgdrtghomh
X-ME-Proxy: <xmx:_dTTZcDi21UfuTbGTBGD-1K50X-UcChyHOOGpVM6zHLoXO1AKKNQ-g>
    <xmx:_dTTZRjQm-EE3TDOnknUVr2BpAER8SumyyFAJ2UNhi2qxJCVam9Zbw>
    <xmx:_dTTZYrGUzNJm0DGAq5YhjDDiT31BxCGIKTQvtMo6b6mTXhUwjpyTQ>
    <xmx:_dTTZRcmNPMS508i7npDCL1YB8C8lLpJpMoK5ABN3QYUv6kaUG7maQ>
Feedback-ID: i1568416f:Fastmail
Date: Mon, 19 Feb 2024 23:23:53 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: qemu-devel@nongnu.org
Cc: =?utf-8?Q?Fr=C3=A9d=C3=A9ric_Pierret_=28fepitre=29?= <frederic.pierret@qubes-os.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	"open list:X86 Xen CPUs" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 2/2] xen: fix stubdom PCI addr
Message-ID: <ZdPU-Yorm66xAN0Z@mail-itl>
References: <20240219181627.282097-1-marmarek@invisiblethingslab.com>
 <20240219181627.282097-2-marmarek@invisiblethingslab.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="OmIj9+efoChGP1II"
Content-Disposition: inline
In-Reply-To: <20240219181627.282097-2-marmarek@invisiblethingslab.com>


--OmIj9+efoChGP1II
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Mon, 19 Feb 2024 23:23:53 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: qemu-devel@nongnu.org
Cc: =?utf-8?Q?Fr=C3=A9d=C3=A9ric_Pierret_=28fepitre=29?= <frederic.pierret@qubes-os.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	"open list:X86 Xen CPUs" <xen-devel@lists.xenproject.org>
Subject: Re: [PATCH 2/2] xen: fix stubdom PCI addr

On Mon, Feb 19, 2024 at 07:16:06PM +0100, Marek Marczykowski-G=C3=B3recki w=
rote:
> From: Fr=C3=A9d=C3=A9ric Pierret (fepitre) <frederic.pierret@qubes-os.org>

This shouldn't be here, it's my patch.

> When running in a stubdomain, the config space access via sysfs needs to
> use BDF as seen inside stubdomain (connected via xen-pcifront), which is
> different from the real BDF. For other purposes (hypercall parameters
> etc), the real BDF needs to be used.
> Get the in-stubdomain BDF by looking up relevant PV PCI xenstore
> entries.
>=20
> Signed-off-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingsl=
ab.com>
> ---
>  hw/xen/xen-host-pci-device.c | 77 +++++++++++++++++++++++++++++++++++-
>  hw/xen/xen-host-pci-device.h |  6 +++
>  2 files changed, 82 insertions(+), 1 deletion(-)
>=20
> diff --git a/hw/xen/xen-host-pci-device.c b/hw/xen/xen-host-pci-device.c
> index 8c6e9a1716..3f8a6f84a8 100644
> --- a/hw/xen/xen-host-pci-device.c
> +++ b/hw/xen/xen-host-pci-device.c
> @@ -9,6 +9,7 @@
>  #include "qemu/osdep.h"
>  #include "qapi/error.h"
>  #include "qemu/cutils.h"
> +#include "hw/xen/xen-legacy-backend.h"
>  #include "xen-host-pci-device.h"
> =20
>  #define XEN_HOST_PCI_MAX_EXT_CAP \
> @@ -33,13 +34,76 @@
>  #define IORESOURCE_PREFETCH     0x00001000      /* No side effects */
>  #define IORESOURCE_MEM_64       0x00100000
> =20
> +static void xen_host_pci_fill_local_addr(XenHostPCIDevice *d, Error **er=
rp)
> +{
> +    unsigned int num_devs, len, i;
> +    unsigned int domain, bus, dev, func;
> +    char *be_path =3D NULL;
> +    char path[80];
> +    char *msg =3D NULL;
> +
> +    be_path =3D qemu_xen_xs_read(xenstore, 0, "device/pci/0/backend", &l=
en);
> +    if (!be_path)
> +        goto err_out;
> +    snprintf(path, sizeof(path), "%s/num_devs", be_path);
> +    msg =3D qemu_xen_xs_read(xenstore, 0, path, &len);
> +    if (!msg)
> +        goto err_out;
> +
> +    if (sscanf(msg, "%u", &num_devs) !=3D 1) {
> +        error_setg(errp, "Failed to parse %s (%s)\n", msg, path);
> +        goto err_out;
> +    }
> +    free(msg);
> +
> +    for (i =3D 0; i < num_devs; i++) {
> +        snprintf(path, sizeof(path), "%s/dev-%u", be_path, i);
> +        msg =3D qemu_xen_xs_read(xenstore, 0, path, &len);
> +        if (!msg) {
> +            error_setg(errp, "Failed to read %s\n", path);
> +            goto err_out;
> +        }
> +        if (sscanf(msg, "%x:%x:%x.%x", &domain, &bus, &dev, &func) !=3D =
4) {
> +            error_setg(errp, "Failed to parse %s (%s)\n", msg, path);
> +            goto err_out;
> +        }
> +        free(msg);
> +        if (domain !=3D d->domain ||
> +                bus !=3D d->bus ||
> +                dev !=3D d->dev ||
> +                func!=3D d->func)
> +            continue;
> +        snprintf(path, sizeof(path), "%s/vdev-%u", be_path, i);
> +        msg =3D qemu_xen_xs_read(xenstore, 0, path, &len);
> +        if (!msg) {
> +            error_setg(errp, "Failed to read %s\n", path);
> +            goto out;
> +        }
> +        if (sscanf(msg, "%x:%x:%x.%x", &domain, &bus, &dev, &func) !=3D =
4) {
> +            error_setg(errp, "Failed to parse %s (%s)\n", msg, path);
> +            goto err_out;
> +        }
> +        free(msg);
> +        d->local_domain =3D domain;
> +        d->local_bus =3D bus;
> +        d->local_dev =3D dev;
> +        d->local_func =3D func;
> +        goto out;
> +    }
> +
> +err_out:
> +    free(msg);
> +out:
> +    free(be_path);
> +}
> +
>  static void xen_host_pci_sysfs_path(const XenHostPCIDevice *d,
>                                      const char *name, char *buf, ssize_t=
 size)
>  {
>      int rc;
> =20
>      rc =3D snprintf(buf, size, "/sys/bus/pci/devices/%04x:%02x:%02x.%d/%=
s",
> -                  d->domain, d->bus, d->dev, d->func, name);
> +                  d->local_domain, d->local_bus, d->local_dev, d->local_=
func, name);
>      assert(rc >=3D 0 && rc < size);
>  }
> =20
> @@ -342,6 +406,17 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, ui=
nt16_t domain,
>      d->dev =3D dev;
>      d->func =3D func;
> =20
> +    if (xen_is_stubdomain) {
> +        xen_host_pci_fill_local_addr(d, errp);
> +        if (*errp)
> +            goto error;
> +    } else {
> +        d->local_domain =3D d->domain;
> +        d->local_bus =3D d->bus;
> +        d->local_dev =3D d->dev;
> +        d->local_func =3D d->func;
> +    }
> +
>      xen_host_pci_config_open(d, errp);
>      if (*errp) {
>          goto error;
> diff --git a/hw/xen/xen-host-pci-device.h b/hw/xen/xen-host-pci-device.h
> index 4d8d34ecb0..270dcb27f7 100644
> --- a/hw/xen/xen-host-pci-device.h
> +++ b/hw/xen/xen-host-pci-device.h
> @@ -23,6 +23,12 @@ typedef struct XenHostPCIDevice {
>      uint8_t dev;
>      uint8_t func;
> =20
> +    /* different from the above in case of stubdomain */
> +    uint16_t local_domain;
> +    uint8_t local_bus;
> +    uint8_t local_dev;
> +    uint8_t local_func;
> +
>      uint16_t vendor_id;
>      uint16_t device_id;
>      uint32_t class_code;
> --=20
> 2.43.0
>=20

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--OmIj9+efoChGP1II
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXT1PkACgkQ24/THMrX
1yz4cgf/Rpz3Yf45EIjwMbO5QQaP8hMOP4meVWEvJ3dqrZZ52TFG+ZetC3NMLRsJ
ul0qdLLfj0T+Dm0pzkhMs8qhs6Ezr40xCx6nYbB+svICLs4SHYY1wqk14Nwg95EN
2rt8HmqPhJcKg/8mzBq00P/z2q6vmzUe519p62G7rpZIMYQ0baam4RL5YnoMMlZC
SN3h5w8fMRkN23hQ6i8o7Z/airtnUwDWr4pkvSSs7dN/0cUGvCigIQ0hieAXl2Kc
3XgLPDIxsAby3nIaep+VK5QVnH7sV29IC0Yu8pLxqfstqkwAsjEnFD8b2wdzZSZn
1WVrmSzUH2uBWqj/kAjfOYehm5nDuA==
=twIH
-----END PGP SIGNATURE-----

--OmIj9+efoChGP1II--


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 23:06:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 23:06:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683137.1062488 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcCiU-0003YP-PH; Mon, 19 Feb 2024 23:06:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683137.1062488; Mon, 19 Feb 2024 23:06:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcCiU-0003YI-MZ; Mon, 19 Feb 2024 23:06:30 +0000
Received: by outflank-mailman (input) for mailman id 683137;
 Mon, 19 Feb 2024 23:06:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcCiU-0003YC-0q
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 23:06:30 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcCiS-00028h-Cx; Mon, 19 Feb 2024 23:06:28 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcCiS-0006Xh-5W; Mon, 19 Feb 2024 23:06:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=IYKBLoM3Bw7GHAyTLtIc5c6JXHOtHFMytcBDktFyWDE=; b=zNOjPxRg3dCvXeU4/AEX3+RMtx
	4oLx/2BAMpoKvsvqzE0/KoyNp7tkbeK9OA9ScHQjnmQ/oYJJO+uGnqWk/wVQRm+J+BEENPDGR+knC
	xxhJYTeC7PrCXFLNEJ0Iej2ebmgortkVpFOaLW7Ih2tErJqOB7N06cHFGzBLd7380rQw=;
Message-ID: <5d2f6be2-020d-4003-ad47-b20ef07ffefc@xen.org>
Date: Mon, 19 Feb 2024 23:06:26 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 02/15] xen/arm: add initial support for LLC coloring on
 arm64
Content-Language: en-GB
To: Carlo Nonato <carlo.nonato@minervasys.tech>,
 Michal Orzel <michal.orzel@amd.com>
Cc: xen-devel@lists.xenproject.org, andrea.bastoni@minervasys.tech,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-3-carlo.nonato@minervasys.tech>
 <bb28c8a2-c2bd-4e1d-ad28-9bc632e3e474@amd.com>
 <CAG+AhRXsZC7Pjdce42yYTvBy7MyjS1axrDGsY0U=o+APNyxdYA@mail.gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CAG+AhRXsZC7Pjdce42yYTvBy7MyjS1axrDGsY0U=o+APNyxdYA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 14/02/2024 13:52, Carlo Nonato wrote:
> On Wed, Feb 14, 2024 at 11:14 AM Michal Orzel <michal.orzel@amd.com> wrote:
>>> diff --git a/xen/arch/arm/include/asm/processor.h b/xen/arch/arm/include/asm/processor.h
>>> index 8e02410465..336933ee62 100644
>>> --- a/xen/arch/arm/include/asm/processor.h
>>> +++ b/xen/arch/arm/include/asm/processor.h
>>> @@ -18,6 +18,22 @@
>>>   #define CTR_IDC_SHIFT       28
>>>   #define CTR_DIC_SHIFT       29
>>>
>>> +/* CCSIDR Current Cache Size ID Register */
>>> +#define CCSIDR_LINESIZE_MASK            _AC(0x7, ULL)
>> Why ULL and not UL? ccsidr is of register_t type
> 
> Julien, while reviewing an earlier version:
> 
>> Please use ULL here otherwise someone using MASK << SHIFT will have the
>> expected result.
> 
> https://patchew.org/Xen/20220826125111.152261-1-carlo.nonato@minervasys.tech/20220826125111.152261-2-carlo.nonato@minervasys.tech/#08956082-c194-8bae-cb25-44e4e3227689@xen.org

Michal is right. This should be UL. Not sure why I suggested ULL back 
then. Sorry.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 19 23:11:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 19 Feb 2024 23:11:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683142.1062499 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcCnE-0005FC-A9; Mon, 19 Feb 2024 23:11:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683142.1062499; Mon, 19 Feb 2024 23:11:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcCnE-0005F5-7V; Mon, 19 Feb 2024 23:11:24 +0000
Received: by outflank-mailman (input) for mailman id 683142;
 Mon, 19 Feb 2024 23:11:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DP9f=J4=invisiblethingslab.com=marmarek@srs-se1.protection.inumbo.net>)
 id 1rcCnD-0005Ex-6l
 for xen-devel@lists.xenproject.org; Mon, 19 Feb 2024 23:11:23 +0000
Received: from wfout8-smtp.messagingengine.com
 (wfout8-smtp.messagingengine.com [64.147.123.151])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 309e029e-cf7c-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 00:11:20 +0100 (CET)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailfout.west.internal (Postfix) with ESMTP id B166D1C00095;
 Mon, 19 Feb 2024 18:11:17 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute2.internal (MEProxy); Mon, 19 Feb 2024 18:11:18 -0500
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
 19 Feb 2024 18:11:15 -0500 (EST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 309e029e-cf7c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	invisiblethingslab.com; h=cc:cc:content-type:content-type:date
	:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
	:references:reply-to:subject:subject:to:to; s=fm3; t=1708384277;
	 x=1708470677; bh=Thq8OY4uzS1G1pUBHrfS8cOVBxJ5ohRbMVTjwhO7kj4=; b=
	Qlitb0wwu7LoRtajcaY3iLHuZvf6SsJpB8EXsd474NXjIWEDzER8hLyXdg3d/y6r
	tQ51yCgP7kw3CpQ5sYoydmCyN1juG02IZ5cFoBusNk6XTUK+onR2HTikL2H6dTjP
	Y4VnWPINwinpqZ3Y6kQsh85H+i5UBg0WBlpEE0zntIV12oOfDC9Qeg7jdhHO5ttn
	h2nQOu822yfptbbVTZE+j4mXwpLhdNFEU2eGuttA0w5KjYMriKhayQIQivIaUzK3
	/OrymSAp7DENI2HUgu3euDEENo14OAwBr5rR2OkLzKHDpE8Xx4KIg7WHl5O4UZ5k
	IDb1B8WfMJdM7hD5chCQqw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=cc:cc:content-type:content-type:date:date
	:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
	:message-id:mime-version:references:reply-to:subject:subject:to
	:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; t=1708384277; x=1708470677; bh=Thq8OY4uzS1G1pUBHrfS8cOVBxJ5
	ohRbMVTjwhO7kj4=; b=YD9cq3zj8CCUWDO4b49FeYPn9XWumXQlHYiulxnw6CgG
	DjLjvU4XDzyOKO1CKvFVPzB6dxaWVBAeGsmJQjRQlez9VFxb7SX8y7iOb9WraCwy
	rj7V1yWrtItzemdO2aG7TXoDr9NHKxTYFO3Mli/39pwWLjpYb6cOqYkCAJXm36wp
	tmjphinEbH5OKl3Z6HrBlQKeHOkfFbm1BKgJ5XXLZN7/1DmMVnhPZ6x4/F7vZf2B
	IvXS4AAz3Nau8RTGNr5+W+LYNDCqP7blLKTSmkubs1eDzK9NvG3oNAfmnp5sDecr
	IMFuYJWM009jgb4mShURqWjDCn96wriQpXfYQbTbQw==
X-ME-Sender: <xms:FeDTZdqfb3IIWs4Nx2IX0vPVnopERgspMQOqV8rHTcLb-7WYXuVelg>
    <xme:FeDTZfoHE7R5aQ2V0B2aGBa4HQyY9H39xIrh-byLNOsNJnWc7cDum8ATGRkSl03fK
    7vgsXHa8rze_w>
X-ME-Received: <xmr:FeDTZaNZHePu4tjjEYxDGdpCpMf5AWe0EXTtw16fO1ZuflMD7SSYWhyjF9OzYgRPPa_e_rG_dxgZxV6GX57ibq3MkLdQZwth8A>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdelgddtiecutefuodetggdotefrodftvf
    curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
    uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
    fjughrpeffhffvvefukfhfgggtuggjsehgtderredttdejnecuhfhrohhmpeforghrvghk
    ucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesihhnvh
    hishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefgudel
    teefvefhfeehieetleeihfejhfeludevteetkeevtedtvdegueetfeejudenucevlhhush
    htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgvkhes
    ihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm
X-ME-Proxy: <xmx:FeDTZY4IL1MmpX3TibwKOwrv4pO5YvGTfpFf3WQTxfmTx0PJdpXxlA>
    <xmx:FeDTZc784e-wPr2Bio4sO3KtZZcW7rdUUXyq059FC8hpQFvRsiZ2gQ>
    <xmx:FeDTZQhUKF-0G0uCcphsagau6mTa3wzN2gCG-_vd2rRmhupBMErYMw>
    <xmx:FeDTZZuvzMeR9AxW8wIVm6LCsB-SZaOFpPXEgmXu3IRzm568LVj0akPhPgc>
Feedback-ID: i1568416f:Fastmail
Date: Tue, 20 Feb 2024 00:11:13 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] ns16550: add Asix AX99100 serial card
Message-ID: <ZdPgEjt9qg8UMx_Q@mail-itl>
References: <20240218013431.274451-1-marmarek@invisiblethingslab.com>
 <7b60cf83-4d08-46ef-a937-911e42fd9e5d@suse.com>
 <ZdM30PNnmlkwW2ZG@mail-itl>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="muf7oRsE++MJz+xe"
Content-Disposition: inline
In-Reply-To: <ZdM30PNnmlkwW2ZG@mail-itl>


--muf7oRsE++MJz+xe
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 20 Feb 2024 00:11:13 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] ns16550: add Asix AX99100 serial card

On Mon, Feb 19, 2024 at 12:13:18PM +0100, Marek Marczykowski-G=C3=B3recki w=
rote:
> On Mon, Feb 19, 2024 at 09:57:49AM +0100, Jan Beulich wrote:
> > On 18.02.2024 02:34, Marek Marczykowski-G=C3=B3recki wrote:
> > > @@ -1170,6 +1177,11 @@ static const struct ns16550_config __initconst=
 uart_config[] =3D
> > >          .dev_id =3D 0x7adc,
> > >          .param =3D param_intel_lpss
> > >      },
> > > +    {
> > > +        .vendor_id =3D PCI_VENDOR_ID_ASIX,
> > > +        .dev_id =3D 9100,
> >=20
> > As per Linux this is 0x9100.
>=20
> Right... but then, maybe the patch isn't needed at all, as it does work
> for me. Maybe what's needed instead is some other patch already in
> staging. Initial attempt that did not work was with 4.17.something.
> I guess setting the fifo size isn't that important.

Indeed, the patch is not needed after all, plain "master" from today
works.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--muf7oRsE++MJz+xe
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXT4BEACgkQ24/THMrX
1yzDmwf+LOad0J29GJEzHfT6KwDPabxwhfRJKcUREDQ8QJ1F3wJRseDmZj+R58jp
zfQnUC6DtW6d7XWOF1EyHeFN7avYo/xCbxoZBBAJp6MOHZdICTKGnZAgB5dW5K/l
/B6A7whFMYvrGHuc1zy++7kEGkwFBLpmXjk48U9ots3BocF8tDiUXsnwK8Rj19Y9
rjeFEsuL2saNGwgjx0DqpIuVHdpnMtlvTZTxQqHiRmCSyZJ4/OqiXzjL7ryEe8J+
LOrnFDi4MYTV3Qhwt3ibBXEoBC1mu1TjcImkmykPrQ/eYx6cruba0SMpDfQ/PX15
ce4m9BY+xSpQwNkooarbyMoJHNf15A==
=IBoA
-----END PGP SIGNATURE-----

--muf7oRsE++MJz+xe--


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 05:30:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 05:30:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683161.1062520 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcIho-0004Jt-GU; Tue, 20 Feb 2024 05:30:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683161.1062520; Tue, 20 Feb 2024 05:30:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcIho-0004Jm-DV; Tue, 20 Feb 2024 05:30:12 +0000
Received: by outflank-mailman (input) for mailman id 683161;
 Tue, 20 Feb 2024 05:30:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcIhn-0004Ja-6B; Tue, 20 Feb 2024 05:30:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcIhm-0000tJ-Ru; Tue, 20 Feb 2024 05:30:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcIhm-0002XJ-Ee; Tue, 20 Feb 2024 05:30:10 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcIhm-0007O2-EC; Tue, 20 Feb 2024 05:30:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=WwoZ9riVoLm2X4Z6Z1fXtOvN2yQw0+ZwrAcoWkoWH8k=; b=x1wNdWAwhF0kpFrXXVilVcKxJG
	LmossZkAK1n0xFnHd7IKzdQtYkvXLDMmGhp+S4Nbizr6bfCunG8SB0bdInQ6WkhzK9tMlBWz5tkEB
	D9jFnJfxsBjofyUDoxV9DdSvbrSKMtzfgsn01Oxulcxy6D3Q1uH8vR0rX3lzcgjHXngU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184705-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184705: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
X-Osstest-Versions-That:
    xen=0441c3acc7e9e72e984ce49d32e61827894ae4a3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 20 Feb 2024 05:30:10 +0000

flight 184705 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184705/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184699
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184699
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184699
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184699
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184699
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184699
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184699
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184699
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184699
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184699
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184699
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184699
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
baseline version:
 xen                  0441c3acc7e9e72e984ce49d32e61827894ae4a3

Last test of basis   184699  2024-02-19 01:52:12 Z    1 days
Testing same since   184703  2024-02-19 14:38:55 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Frediano Ziglio <frediano.ziglio@cloud.com>
  Julien Grall <jgrall@amazon.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   0441c3acc7..c144b9e324  c144b9e32427ba37e0e0443a0d3fa53e9fb878b6 -> master


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 06:50:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 06:50:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683173.1062547 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcJxX-0004ue-IK; Tue, 20 Feb 2024 06:50:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683173.1062547; Tue, 20 Feb 2024 06:50:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcJxX-0004uX-FN; Tue, 20 Feb 2024 06:50:31 +0000
Received: by outflank-mailman (input) for mailman id 683173;
 Tue, 20 Feb 2024 06:50:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=eXp8=J5=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1rcJxW-0004uR-09
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 06:50:30 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 54eebf70-cfbc-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 07:50:28 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-4126d65ace3so3474355e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 22:50:28 -0800 (PST)
Received: from [192.168.69.100] (mek33-h02-176-184-23-7.dsl.sta.abo.bbox.fr.
 [176.184.23.7]) by smtp.gmail.com with ESMTPSA id
 s6-20020a05600c45c600b00412696bd7d9sm3642279wmo.41.2024.02.19.22.50.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 22:50:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 54eebf70-cfbc-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1708411827; x=1709016627; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:references:cc:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=7IEnePugGgUoICeCL35mG4yEzWVLyAAMfq5V5GWMgX0=;
        b=RoV8qHaPOc03B3S8BN1pgHnk2gmf68fL0wJvEhUsqzgGwqxA/XHhuK7laRLt4EV1R1
         b4XX/VUIT1OgcCiA0DLpErRybRYwVDAffUCYqymFl1J0kruSvxcIeUhWeomrN2zx1Usk
         C7N9Lrn25zXq+byawA9Svr4sgo2erCithRHXI4WvAQW1F0B7Enkjtw0wNug06rSMy2KN
         XYWWDSDmXoEQWuhGbdta0DQP5CIB/8B6j34YvSLO+JH99MxEJdnUeUBTzhXSgSx6qTbb
         l59IiR+QR5cXtSFi3US7uqOPWqeN8ecl03d6NUkYUD4WXzdiOqMVbOh8GPDWgwEg2Omg
         wglA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708411827; x=1709016627;
        h=content-transfer-encoding:in-reply-to:from:references:cc:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=7IEnePugGgUoICeCL35mG4yEzWVLyAAMfq5V5GWMgX0=;
        b=YvYcYYCp9BZksSN/F0aVtwnyRUZQfTpx6zQrbSyJsntQqyOxkuWiw6o22n48oOeFBd
         JVI+A0ChAPeIIV4M74GIu9VwmPYNMBXGhw2xOUlTFPKnU4jUgr7O8CsfNwSh2yVn+IAE
         RuKTn1SQfkv7XwfzAm2e8i/LQAXyxA31fjy2VwFO5trzQd1qb5zbdquJdLUU89xwZ8vH
         q8B9KeTHs9bEOusHu4qsPPUOnkqL/ulKdMYWdOEtecPfHC/Y35oYkQ96kYl8K0ClhjSo
         CTKWY8x0luhNyoyjGYKO5hbQr4yy1Q70EGsL4lc/65tAg6uzuHE4LsHcCdxGY3yNJj79
         g9dQ==
X-Forwarded-Encrypted: i=1; AJvYcCW+5fX8WdK+R4aYL9t1eAP9KS276SbnUvvLSeHZiV0KiqmKZ2+uVB9Rj8TGm5q1HanAL8ZQ+aFLJ29rY0tEQeus3v8un9X9OQGkEhK4Xnc=
X-Gm-Message-State: AOJu0YwE09iZ/728D7tSknK6No2NHVlm5uzocrwp4Y9yuwe5AW20JdfX
	vP4rSavxZitS7GSOO93FPl1t+aF3zvLJLl0UTsleag1quckDKszwRGySZjx/fq8=
X-Google-Smtp-Source: AGHT+IHxO0r30lxscG2YCRQIfJHxQ5ifYp/345djnp2UqyWvY1/X9dmS4mMVsBvkya0ymJckBG5wHw==
X-Received: by 2002:a05:600c:21d8:b0:411:e398:ca7a with SMTP id x24-20020a05600c21d800b00411e398ca7amr11654370wmj.39.1708411827713;
        Mon, 19 Feb 2024 22:50:27 -0800 (PST)
Message-ID: <429a5a27-21b9-45bd-a1a6-a1c2ccc484c9@linaro.org>
Date: Tue, 20 Feb 2024 07:50:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] hw/xen: detect when running inside stubdomain
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>, qemu-devel@nongnu.org,
 Paolo Bonzini <pbonzini@redhat.com>, Claudio Fontana <cfontana@suse.de>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 "open list:X86 Xen CPUs" <xen-devel@lists.xenproject.org>
References: <20240219181627.282097-1-marmarek@invisiblethingslab.com>
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240219181627.282097-1-marmarek@invisiblethingslab.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 19/2/24 19:16, Marek Marczykowski-Górecki wrote:
> Introduce global xen_is_stubdomain variable when qemu is running inside
> a stubdomain instead of dom0. This will be relevant for subsequent
> patches, as few things like accessing PCI config space need to be done
> differently.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
>   hw/xen/xen-legacy-backend.c | 15 +++++++++++++++
>   include/hw/xen/xen.h        |  1 +
>   system/globals.c            |  1 +
>   3 files changed, 17 insertions(+)


> diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h
> index 37ecc91fc3..ecb89ecfc1 100644
> --- a/include/hw/xen/xen.h
> +++ b/include/hw/xen/xen.h
> @@ -36,6 +36,7 @@ enum xen_mode {
>   extern uint32_t xen_domid;
>   extern enum xen_mode xen_mode;
>   extern bool xen_domid_restrict;
> +extern bool xen_is_stubdomain;
>   
>   int xen_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num);
>   int xen_set_pci_link_route(uint8_t link, uint8_t irq);
> diff --git a/system/globals.c b/system/globals.c
> index b6d4e72530..ac27d88bd4 100644
> --- a/system/globals.c
> +++ b/system/globals.c
> @@ -62,6 +62,7 @@ bool qemu_uuid_set;
>   uint32_t xen_domid;
>   enum xen_mode xen_mode = XEN_DISABLED;
>   bool xen_domid_restrict;
> +bool xen_is_stubdomain;

Note for myself, Paolo and Claudio, IIUC these fields belong
to TYPE_XEN_ACCEL in accel/xen/xen-all.c. Maybe resulting in
smth like:

-- >8 --
diff --git a/accel/xen/xen-all.c b/accel/xen/xen-all.c
index 5ff0cb8bd9..fc25d8c912 100644
--- a/accel/xen/xen-all.c
+++ b/accel/xen/xen-all.c
@@ -24,11 +24,31 @@
  #include "migration/global_state.h"
  #include "hw/boards.h"

-bool xen_allowed;
+struct XenAccelState
+{
+    AccelState parent_obj;
+
+    bool xen_allowed;
+
+    enum xen_mode xen_mode;
+
+    uint32_t xen_domid;
+    bool xen_domid_restrict;

  xc_interface *xen_xc;
  xenforeignmemory_handle *xen_fmem;
  xendevicemodel_handle *xen_dmod;
+};
+
+struct XenAccelOpsClass
+{
+    AccelOpsClass parent_class;
+
+    struct evtchn_backend_ops *xen_evtchn_ops;
+    struct gnttab_backend_ops *xen_gnttab_ops;
+    struct foreignmem_backend_ops *xen_foreignmem_ops;
+    struct xenstore_backend_ops *xen_xenstore_ops;
+}

  static void xenstore_record_dm_state(const char *state)
  {
@@ -114,6 +134,13 @@ static int xen_init(MachineState *ms)
      return 0;
  }

+static void xen_accel_init(Object *obj)
+{
+    XenAccelState *s = XEN_ACCEL(obj);
+
+    s->xen_mode = XEN_DISABLED;
+}
+
  static void xen_accel_class_init(ObjectClass *oc, void *data)
  {
      AccelClass *ac = ACCEL_CLASS(oc);
@@ -142,6 +169,8 @@ static void xen_accel_class_init(ObjectClass *oc, 
void *data)
  static const TypeInfo xen_accel_type = {
      .name = TYPE_XEN_ACCEL,
      .parent = TYPE_ACCEL,
+    .instance_size = sizeof(XenAccelState),
+    .instance_init = xen_accel_init,
      .class_init = xen_accel_class_init,
  };

@@ -157,6 +186,7 @@ static const TypeInfo xen_accel_ops_type = {

      .parent = TYPE_ACCEL_OPS,
      .class_init = xen_accel_ops_class_init,
+    .class_size = sizeof(XenAccelOpsClass),
      .abstract = true,
  };

---


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 07:18:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 07:18:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683179.1062557 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcKOu-0007n2-NP; Tue, 20 Feb 2024 07:18:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683179.1062557; Tue, 20 Feb 2024 07:18:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcKOu-0007mv-KO; Tue, 20 Feb 2024 07:18:48 +0000
Received: by outflank-mailman (input) for mailman id 683179;
 Tue, 20 Feb 2024 07:18:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcKOt-0007mm-TS
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 07:18:47 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 47237d12-cfc0-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 08:18:43 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-4127077ee20so382485e9.1
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 23:18:43 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 b1-20020a05600c11c100b00410cc2f5550sm13295465wmi.19.2024.02.19.23.18.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 23:18:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47237d12-cfc0-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708413522; x=1709018322; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=30BNUS+5ObDbWYlvV4vCUl/bgFcqsUNuCVaytBgIjo0=;
        b=Sg8cZyChtDfkOolE7z4bXoyIEIyeWdgK/JOjJD+HWUe4sf9UPXflEKZ5zFGIQbAJnB
         Qo+klvh0SopaUiMjOw8TuUc3RmhHhtDP6hjrMs/41REko4oFORTkzHTLhD0rjMeHFDkf
         eUjA+z6VEJXI/wQ5XJEZLQqZsbEJco046ZujFWrgOqtoOwaw0rdppBMJDP37BI5EL80u
         n9sHhDvkdhRkRQpWIfnrM8BPUeyTu/ynyIclVjQ/4Kgnm2yzFk4JARDYQcL2SYu0d6mQ
         1R2e0VuAENlNiUEQmVbVrevZ8fSQrai8Sqc7oCvRBCqY33ILbL1MmTJM57BXvIX6SlF2
         N6dQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708413522; x=1709018322;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=30BNUS+5ObDbWYlvV4vCUl/bgFcqsUNuCVaytBgIjo0=;
        b=g2iIb6bnGgNviXEIX8Tt0arXyvF8SMCjRp0XLw8xO56z7b6XOw0fpdepEZKmGtBJgw
         +fkVWMfcu97gLLfy3JFaWDlLUUPeVebmdL1Ql9pl79H0H1MLn2I8DvEW1tZ9NhF+MSM5
         adCQXaNtdEO+1LdtTYnOiGKq/gdOxCE739PX+VK0tMOQshmC3hAYil77iKjdH3+hiLID
         V+ZTIFJwniS/ypR3odmHbJL5FcgZ5wzk9knmQse2NSI1anFejdjRWJLVXpZNFJZlLfPS
         NHsXh+izc8dyKc3M2l1hYYD5T8nZzAWYV6IziJYzNcDkUu5/v4KQZ+sL2UXD/tVimojq
         QRAg==
X-Forwarded-Encrypted: i=1; AJvYcCVoDyFP9oSNuGTAPIWjbOdpsgfWeU2Ma69ziUPwmzXvnfkRI2cd7cIO6R+E/61pX+YQnCtASmy/uEHUlkza5Vhn+QQDkxpQqxGlq5aw/Uw=
X-Gm-Message-State: AOJu0YxhieDZHzJm9Xz4ww1wbv7j59lNf/5pIchJ7FrTVKBvhBKDrg4r
	wDdHbYRaceAeKWT3TYhfrqbpvUus8k3hvQ6Ed/DTm+logkLhfNKF8izD9rh+2g==
X-Google-Smtp-Source: AGHT+IFy5rGYcDgjhWYry3W5VheY3OY35thZRmxgZXRXF3bcGp6xjW6Xg11JbBYcnSNh1BaJlM4CuA==
X-Received: by 2002:a05:600c:3545:b0:412:65e7:3639 with SMTP id i5-20020a05600c354500b0041265e73639mr3553337wmq.27.1708413522567;
        Mon, 19 Feb 2024 23:18:42 -0800 (PST)
Message-ID: <dda63b1e-648c-4f11-896c-0359b4099a56@suse.com>
Date: Tue, 20 Feb 2024 08:18:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v2 2/3] x86/uaccess: replace __{get,put}_user_bad()
 with STATIC_ASSERT_UNREACHABLE()
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>,
 Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1706259490.git.federico.serafini@bugseng.com>
 <alpine.DEB.2.22.394.2402061707520.1925432@ubuntu-linux-20-04-desktop>
 <22ee2311-1e6b-4f2e-86e4-12d20b5ba4a2@suse.com>
 <6451696b-0366-4069-b82b-094e63eced8d@bugseng.com>
 <0b0b5a50-7692-4500-baa4-68df8f1c5d7a@suse.com>
 <d7812cd9-7d06-493b-b8a8-d2353f148e8b@bugseng.com>
 <a5d9b91d-4e56-4512-9b15-d6868a383923@suse.com>
 <04a2c7b2-07e5-46fd-835e-42b3c6307547@bugseng.com>
 <a4c2f7c0-66bc-436f-bc99-8f8ba4ec822e@suse.com>
 <fffaa145-16ff-4969-bf4c-cfe0be98b66a@bugseng.com>
 <2117bb4e-94a3-4990-945b-6fb141ae1e63@suse.com>
 <3ed651c3-4ed3-4237-bf57-3a99a790df1a@bugseng.com>
 <3edafece-f15e-4a28-9f17-a4b08d3e0215@suse.com>
 <35249e32-9eb2-4c51-868d-de2cd2d8230b@bugseng.com>
 <45c19a87-5833-4350-a374-3a70fd3aa9ae@suse.com>
 <d735e980-c5a5-46b1-a511-cc2d666dac23@bugseng.com>
 <123c446c-e56f-457e-9cf4-58a43a3b069a@suse.com>
 <56b4a4bb-d3e8-4a21-8c0d-d4ee10c8f9b3@bugseng.com>
 <alpine.DEB.2.22.394.2402191243090.155645@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402191243090.155645@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 19.02.2024 21:43, Stefano Stabellini wrote:
> On Mon, 19 Feb 2024, Federico Serafini wrote:
>> On 15/02/24 11:32, Jan Beulich wrote:
>>> The important difference is: Here we're told that there was a use of
>>> __put_user_bad, which is easy to grep for, and thus see how the
>>> supplied function / file / line(?) relate to the ultimate problem.
>>>
>>> I'm afraid I'm meanwhile confused enough by the various replies
>>> containing results of experimentation that I can't really tell
>>> anymore what case is best. Hence I can only restate my expectation for
>>> an eventual v3: Diagnosing what the issue is, no matter whether the new
>>> macro is used in another macro or in an inline function, should not
>>> become meaningfully more difficult. In how far this is the case wants
>>> clarifying in the description of the change.
>>
>> I think the best thing at the moment is to deviate
>> __{get,put}_user_bad() for Rule 16.3.
>> I'll let maintainers further explore the possibility of having a
>> compile-time assertion based on the assembler error.
> 
> OK. I hope Jan is OK to deviate by in-code comment.

Hmm, the follow-on suggestion was to add break statements? Followed
by me asking whether adding noreturn to the decls wouldn't also help.
(Then again I was under the impression that there was more than just
the "missing" break statements which Misra thought was an issue here.)

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 07:45:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 07:45:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683185.1062567 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcKoP-0003Dq-OO; Tue, 20 Feb 2024 07:45:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683185.1062567; Tue, 20 Feb 2024 07:45:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcKoP-0003Dj-Lf; Tue, 20 Feb 2024 07:45:09 +0000
Received: by outflank-mailman (input) for mailman id 683185;
 Tue, 20 Feb 2024 07:45:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcKoO-0003Db-G0
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 07:45:08 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f75ed2a9-cfc3-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 08:45:07 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-4126dd40a54so3735005e9.3
 for <xen-devel@lists.xenproject.org>; Mon, 19 Feb 2024 23:45:07 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p17-20020a05600c469100b004120b4c57c9sm13842866wmo.4.2024.02.19.23.45.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 19 Feb 2024 23:45:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f75ed2a9-cfc3-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708415107; x=1709019907; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gUHDm++rJxrZmyWvH6jBOaVy8/JQwpnsk5PLwAGeL+c=;
        b=cVFNCzYmmwZXrVr7D7XDbJbdHjTVX3KMbSn1B6IiJomtJqvt9WcOH636taYzeud1sT
         FfoRNKs2muhU/crbsoBUMt68sFsu8wIxdZtNPHXaJcAK6WcYPsi/roD0Fhorr2qCqqlL
         +hQfrJkFLuAs4U/921sEDiuCBWn9PjzSdEKtU1i3YKPFnjay1eHC9bdXC9K15sTJp35L
         5Mh3ZT/nbOi3llPE9bf6hufnZXZ2X5U9SkwRNNpVpufdueMUc70Zm0XYU5k1qNTbF/ah
         hAp966FeGhloRYvwIcytMg0zaBCy+5nP8uHBfzCeidmAKVDKfSoyjzkcK7x+9d519huC
         xArg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708415107; x=1709019907;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gUHDm++rJxrZmyWvH6jBOaVy8/JQwpnsk5PLwAGeL+c=;
        b=w+0LpnrFC2Ohao7GMwNDe7RuOPusFkZreIRdepbYhSbfKQmD1VlS49KqsvIS8w7Lif
         GjwMRCd9BBkJp8IiaTdGS/VXswSHOgMjN+8EkBrzIMKAPZl0SOZiVZHR5E3lJhZE2vCr
         +T7ihrg4dRQUQgbOvHCKM/qw/CcDVQd5yZ6XWQ/jhw5Mn7EuH1TloJw+oiCPkTcuMqSj
         +CJitihj9ipa+zC7fonfv3USCDPuk7OUCJf0PWkilZfP3/cjAOYNxRa6B6bt0ijkuImN
         ZNUAHmpwG4EpWr1vsQQrlc+DRnehK3BsqsOvCfwotK4mK5+kbl8Ew6C/5KsH52+v+zGb
         O8Bg==
X-Forwarded-Encrypted: i=1; AJvYcCXt14t2ccpw3uA/YGjQizLWD6QTydhmnBDypufQc0WoSbs3/WO8ga6HJwZ+tssau+V/BCU+rJxulOnyTF25oPv3Vh0c8nwThkf0beY2LHU=
X-Gm-Message-State: AOJu0YwHOxlXn6/7UTRykzeDGsNvlVgwIG8ga8fyUXqShVUtnm8W3/Cu
	kNOV//ImWbMklQgxtgx9V5YVFCy6E2wXpX/wbRTmeIzGtFwHVNusD8/OxUyb0A==
X-Google-Smtp-Source: AGHT+IFYIAlUadMDqejLTbHkz/pmDNkrmNWdeYu4jJMLp6MzAyvy1taPWooutRgPTOSjBFuyaPhD0Q==
X-Received: by 2002:a05:600c:1f87:b0:411:fc3f:c620 with SMTP id je7-20020a05600c1f8700b00411fc3fc620mr13119388wmb.20.1708415106721;
        Mon, 19 Feb 2024 23:45:06 -0800 (PST)
Message-ID: <d90d98b6-508b-4a2a-ab6a-74a9828a5b94@suse.com>
Date: Tue, 20 Feb 2024 08:45:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 19.02.2024 16:14, Nicola Vetrini wrote:
> The cache clearing and invalidation helpers in x86 and Arm didn't
> comply with MISRA C Rule 17.7: "The value returned by a function
> having non-void return type shall be used". On Arm they
> were always returning 0, while some in x86 returned -EOPNOTSUPP
> and in common/grant_table the return value is saved.
> 
> As a consequence, a common helper arch_grant_cache_flush that returns
> an integer is introduced, so that each architecture can choose whether to
> return an error value on certain conditions, and the helpers have either
> been changed to return void (on Arm) or deleted entirely (on x86).
> 
> Signed-off-by: Julien Grall <julien@xen.org>
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> ---
> The original refactor idea came from Julien Grall in [1]; I edited that proposal
> to fix build errors.
> 
> I did introduce a cast to void for the call to flush_area_local on x86, because
> even before this patch the return value of that function wasn't checked in all
> but one use in x86/smp.c, and in this context the helper (perhaps incidentally)
> ignored the return value of flush_area_local.

I object to such casting to void, at least until there's an overriding
decision that for Misra purposes such casts may be needed.

> --- a/xen/arch/arm/include/asm/page.h
> +++ b/xen/arch/arm/include/asm/page.h
> @@ -123,6 +123,7 @@
>  
>  #ifndef __ASSEMBLY__
>  
> +#include <public/grant_table.h>

This is a no-go, imo (also on x86): Adding this include here effectively
means that nearly every CU will have a dependency on that header, no
matter that most are entirely agnostic of grants. Each arch has a
grant_table.h - is there any reason the new, grant-specific helper can't
be put there?

> @@ -182,21 +183,21 @@ void flush_area_mask(const cpumask_t *mask, const void *va,
>  }
>  
>  static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache) {}
> -static inline int invalidate_dcache_va_range(const void *p,
> -                                             unsigned long size)
> -{ return -EOPNOTSUPP; }
> -static inline int clean_and_invalidate_dcache_va_range(const void *p,
> -                                                       unsigned long size)
> +
> +static inline int arch_grant_cache_flush(unsigned int op, const void *p,
> +                                     unsigned long size)
>  {
> -    unsigned int order = get_order_from_bytes(size);
> +    unsigned int order;
> +
> +    if ( !(op & GNTTAB_CACHE_CLEAN) )
> +        return -EOPNOTSUPP;
> +
> +    order = get_order_from_bytes(size);
>      /* sub-page granularity support needs to be added if necessary */
> -    flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
> +    (void) flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));

As to my objection to the addition of a cast, did you actually check
what this function returns, before saying "incidentally" in the
respective remark? Already the return type being "unsigned int" is
indicative of the return value not being suitable here for handing
on.

In addition there shouldn't be a blank after a cast. Instead, if
already you were to touch this line, it would have been nice if you
took the opportunity and added the missing blanks around the binary
operator involved.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:14:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:14:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683196.1062577 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLGu-0007oG-D9; Tue, 20 Feb 2024 08:14:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683196.1062577; Tue, 20 Feb 2024 08:14:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLGu-0007o9-9w; Tue, 20 Feb 2024 08:14:36 +0000
Received: by outflank-mailman (input) for mailman id 683196;
 Tue, 20 Feb 2024 08:14:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=oQVm=J5=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rcLGs-0007nv-U6
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:14:34 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 13a3a70b-cfc8-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:14:32 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 107B54EE073A;
 Tue, 20 Feb 2024 09:14:32 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13a3a70b-cfc8-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Tue, 20 Feb 2024 09:14:32 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, George
 Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
In-Reply-To: <d90d98b6-508b-4a2a-ab6a-74a9828a5b94@suse.com>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
 <d90d98b6-508b-4a2a-ab6a-74a9828a5b94@suse.com>
Message-ID: <45509cb67ecee3f690b5784489b5ccb4@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-20 08:45, Jan Beulich wrote:
> On 19.02.2024 16:14, Nicola Vetrini wrote:
>> The cache clearing and invalidation helpers in x86 and Arm didn't
>> comply with MISRA C Rule 17.7: "The value returned by a function
>> having non-void return type shall be used". On Arm they
>> were always returning 0, while some in x86 returned -EOPNOTSUPP
>> and in common/grant_table the return value is saved.
>> 
>> As a consequence, a common helper arch_grant_cache_flush that returns
>> an integer is introduced, so that each architecture can choose whether 
>> to
>> return an error value on certain conditions, and the helpers have 
>> either
>> been changed to return void (on Arm) or deleted entirely (on x86).
>> 
>> Signed-off-by: Julien Grall <julien@xen.org>
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>> ---
>> The original refactor idea came from Julien Grall in [1]; I edited 
>> that proposal
>> to fix build errors.
>> 
>> I did introduce a cast to void for the call to flush_area_local on 
>> x86, because
>> even before this patch the return value of that function wasn't 
>> checked in all
>> but one use in x86/smp.c, and in this context the helper (perhaps 
>> incidentally)
>> ignored the return value of flush_area_local.
> 
> I object to such casting to void, at least until there's an overriding
> decision that for Misra purposes such casts may be needed.
> 

There are three choices here:
1. cast to void
2. deviation for flush_area_local, which for the case of the cache 
helpers is what led to this patch; it may still be a viable option, if 
other maintainers agree
3. refactor of flush_area_local; this is not viable here because the 
return value is actually used and useful, as far as I can tell, in smp.c

>> --- a/xen/arch/arm/include/asm/page.h
>> +++ b/xen/arch/arm/include/asm/page.h
>> @@ -123,6 +123,7 @@
>> 
>>  #ifndef __ASSEMBLY__
>> 
>> +#include <public/grant_table.h>
> 
> This is a no-go, imo (also on x86): Adding this include here 
> effectively
> means that nearly every CU will have a dependency on that header, no
> matter that most are entirely agnostic of grants. Each arch has a
> grant_table.h - is there any reason the new, grant-specific helper 
> can't
> be put there?
> 

I would have to test, but I think that can be done

>> @@ -182,21 +183,21 @@ void flush_area_mask(const cpumask_t *mask, 
>> const void *va,
>>  }
>> 
>>  static inline void flush_page_to_ram(unsigned long mfn, bool 
>> sync_icache) {}
>> -static inline int invalidate_dcache_va_range(const void *p,
>> -                                             unsigned long size)
>> -{ return -EOPNOTSUPP; }
>> -static inline int clean_and_invalidate_dcache_va_range(const void *p,
>> -                                                       unsigned long 
>> size)
>> +
>> +static inline int arch_grant_cache_flush(unsigned int op, const void 
>> *p,
>> +                                     unsigned long size)
>>  {
>> -    unsigned int order = get_order_from_bytes(size);
>> +    unsigned int order;
>> +
>> +    if ( !(op & GNTTAB_CACHE_CLEAN) )
>> +        return -EOPNOTSUPP;
>> +
>> +    order = get_order_from_bytes(size);
>>      /* sub-page granularity support needs to be added if necessary */
>> -    flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
>> +    (void) flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
> 
> As to my objection to the addition of a cast, did you actually check
> what this function returns, before saying "incidentally" in the
> respective remark? Already the return type being "unsigned int" is
> indicative of the return value not being suitable here for handing
> on.
> 

My "incidentally" was motivated by the fact that the caller doesn't 
check whether
flags_in != flags_out (effectively tests for the execution of a certain 
code path). It may have been deliberate or not, I don't know. If it was 
accidental, then a check of the return value in arch_grant_cache_flush 
will eliminate the need for a void cast.

> In addition there shouldn't be a blank after a cast. Instead, if
> already you were to touch this line, it would have been nice if you
> took the opportunity and added the missing blanks around the binary
> operator involved.
> 

That's true, thanks.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:16:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:16:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683201.1062587 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLIz-0008Nw-R4; Tue, 20 Feb 2024 08:16:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683201.1062587; Tue, 20 Feb 2024 08:16:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLIz-0008Np-OQ; Tue, 20 Feb 2024 08:16:45 +0000
Received: by outflank-mailman (input) for mailman id 683201;
 Tue, 20 Feb 2024 08:16:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/fGf=J5=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rcLIz-0008Nj-Aq
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:16:45 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 619d90e1-cfc8-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:16:43 +0100 (CET)
Received: from [192.168.1.229] (93-36-216-194.ip62.fastwebnet.it
 [93.36.216.194])
 by support.bugseng.com (Postfix) with ESMTPSA id CF1654EE073A;
 Tue, 20 Feb 2024 09:16:42 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 619d90e1-cfc8-11ee-98f5-efadbce2ee36
Message-ID: <1f1fdfd8-1a34-411e-a0f2-d9bcb5b050fd@bugseng.com>
Date: Tue, 20 Feb 2024 09:16:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair: add deviation for MISRA C:2012
 Rule 16.3
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <b1d2b64c8117d61ea42cf4e9feae128541eb0b61.1708348799.git.federico.serafini@bugseng.com>
 <a6e8bd32-cc32-4084-907d-e2cd6d46e3e6@suse.com>
 <f28aed5e-0983-48fa-828f-b2133ca35086@bugseng.com>
 <4d2c5a69-4807-4af3-84c2-128d16ee2b43@suse.com>
Content-Language: en-US, it
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <4d2c5a69-4807-4af3-84c2-128d16ee2b43@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 19/02/24 16:06, Jan Beulich wrote:
> On 19.02.2024 15:59, Federico Serafini wrote:
>> On 19/02/24 14:43, Jan Beulich wrote:
>>> On 19.02.2024 14:24, Federico Serafini wrote:
>>>> Update ECLAIR configuration to consider safe switch clauses ending
>>>> with __{get,put}_user_bad().
>>>>
>>>> Update docs/misra/deviations.rst accordingly.
>>>>
>>>> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
>>>
>>> As mentioned I'm not happy with this, not the least because of it being
>>> unclear why these two would be deviated, when there's no sign of a
>>> similar deviation for, say, __bad_atomic_size(). Imo this approach
>>> doesn't scale, and that's already leaving aside that the purpose of
>>> identically named (pseudo-)helpers could differ between architectures,
>>> thus putting under question ...
>>>
>>>> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
>>>> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
>>>> @@ -368,6 +368,10 @@ safe."
>>>>    -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/BUG\\(\\);/))))"}
>>>>    -doc_end
>>>>    
>>>> +-doc_begin="Switch clauses ending with constructs \"__get_user_bad()\" and \"__put_user_bad()\" are safe: they denote an unreachable program point."
>>>> +-config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/__(put|get)_user_bad\\(\\);/))))"}
>>>> +-doc_end
>>>
>>> ... the global scope of such a deviation. While it may not be a good idea,
>>> even within an arch such (pseudo-)helpers could be used for multiple
>>> distinct purposes.
>>
>> Would you agree with adding the missing break statement after
>> the uses of __{put,get}_user_bad() (as it is already happening for
>> uses of __bad_atomic_size())?
> 
> I probably wouldn't mind that (despite being a little pointless).
> Perhaps declaring them as noreturn would also help?

Yes, it will help.
Is there any reason to have long as __get_user_bad()'s return value?
It would be nicer to declare it as a void function and then add the
noreturn attribute.

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:26:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:26:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683207.1062597 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLS5-0001eO-NP; Tue, 20 Feb 2024 08:26:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683207.1062597; Tue, 20 Feb 2024 08:26:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLS5-0001eH-Jq; Tue, 20 Feb 2024 08:26:09 +0000
Received: by outflank-mailman (input) for mailman id 683207;
 Tue, 20 Feb 2024 08:26:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLS4-0001eB-MV
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:26:08 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b1bb5665-cfc9-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:26:07 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-4127086bb8aso621215e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:26:07 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 co3-20020a0560000a0300b0033d70dd0e04sm162106wrb.8.2024.02.20.00.26.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:26:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1bb5665-cfc9-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708417567; x=1709022367; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OePRDQOjkk3bAla0m7Qr0etARY5N6IYAZwBZ2aPacag=;
        b=BVsJWvkwdAkiKmItkmzEzgSmRrNOylDjL3dOk1QKquycptia0FFVRs4rHFHFfQnuan
         iimBOG5NyjPF0hz/AzUSLOFTgSpjOnnV59K8T7FMER3XbEd+346kjv3rkMwYJUBN/j9R
         QJIL1WHbqbfiVIxg6ad69gvMPugkGINPgcpcketjMretB9vxAh39qzRjfzloW39P67cl
         mSAj6d+Z2NSWO1YtG3P3037n12SNhOUJ0j3jFVYgBhV4uO8BflDHk/WIxaUzqt4zSaH+
         04Oj5MJNjWkeWdpsM4FNxK7JMJSyMfhnaVt2swLI9YLYqPP9kxCijseXu2ZpxClOAKi7
         NuOg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708417567; x=1709022367;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OePRDQOjkk3bAla0m7Qr0etARY5N6IYAZwBZ2aPacag=;
        b=jOF3ImA/pAV4KZ4U42dgRUkTG3QkYZyLpvggjX1hm1/oj+UC0fPAcvr8EJyGBY8mms
         +0KpPWg1MiX8Jme2u8aq1LAEqBuei/kRF+sGrWntMIvXIWUx+bo26BOJZqzv5veOcS1v
         +GXQLm0NeHmgANipoA1Y+QYWqToc8EDWoqja1Ileoza7vw505bj5S/c1VtWWp/6IR4hC
         5E82+rk3L/rDx7Kej84T5eeTzcdG7E61mLwJZJM/Qcj1u9+PLE07kNd/2tdtmrrgQdNA
         nUdM6S9N26ZlTnx6Mh6InVnTIhyAglNaTn3GKci9HRzujCdjQVQedXcQ+A7GyRU0wnD+
         Cz6w==
X-Forwarded-Encrypted: i=1; AJvYcCW14rjQSxs4SDmWiNTiHqTfSydnSnbyrMbgFncVa2fR0jNaNyscsUJhHWSuvjqIWXjcSxE+0eOjA+jimwCunMfsniCT9QM3wK4dTA4e568=
X-Gm-Message-State: AOJu0YzWqEcGjjy2QqFj0gTS9w57UbLWgWL1CV0OtXwb+O0/xdQd7lA0
	e7shE09i9RMQRmwLlsvuWrv4QLlhG+vebCrP2zgsSlrV1EsDHaEoVjyUNqyiqg==
X-Google-Smtp-Source: AGHT+IFA5neEU9WAepo8LQ4i3BIraJ2gPDzQw/P5tVjxgFdV/ILvvHl3xV1RJqT9S2QvLY2PBjrveg==
X-Received: by 2002:a5d:6652:0:b0:33d:15c1:cfcb with SMTP id f18-20020a5d6652000000b0033d15c1cfcbmr9121464wrw.40.1708417566820;
        Tue, 20 Feb 2024 00:26:06 -0800 (PST)
Message-ID: <6466155e-9883-4743-a897-ef565f14fdf0@suse.com>
Date: Tue, 20 Feb 2024 09:26:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] almost fully ignore zero-size flush requests
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
 <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com>
 <7dd78c97-b82f-4ae1-a257-98f612c6bad2@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <7dd78c97-b82f-4ae1-a257-98f612c6bad2@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 19.02.2024 23:22, Julien Grall wrote:
> Title: I would add 'gnttab:' to clarify which subsystem you are modifying.

That's how I actually have it here; it's not clear to me why I lost the
prefix when sending.

> On 05/02/2024 11:03, Jan Beulich wrote:
>> Along the line with observations in the context of XSA-448, besides
>> "op" no field is relevant when the range to be flushed is empty, much
>> like e.g. the pointers passed to memcpy() are irrelevant (and would
>> never be "validated") when the passed length is zero. Split the existing
>> condition validating "op", "offset", and "length", leaving only the "op"
>> part ahead of the check for length being zero (or no flushing to be
>> performed).
> 
> I am probably missing something here. I understand the theory behind 
> reducing the number of checks when len == 0. But an OS cannot rely on it:
>    1) older hypervisor would still return an error if the check doesn't 
> pass)

Right, but that's no reason to keep the bogus earlier behavior.

>    2) it does feel odd to allow "invalid" offset when len == 0 (at least.

I'm puzzled: You've given R-b for patch 1 (thanks), where exactly the
same reasoning is used, i.e. similarly referring to memcpy() to
justify the (new / supposed) behavior.

>> In the course of splitting also simplify the moved part of the condition
>> from 3 to 2 conditionals, potentially (depending on the architecture)
>> requiring one less (conditional) branch.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>
>> --- a/xen/common/grant_table.c
>> +++ b/xen/common/grant_table.c
>> @@ -3528,15 +3528,16 @@ static int _cache_flush(const gnttab_cac
>>       void *v;
>>       int ret;
>>   
>> -    if ( (cflush->offset >= PAGE_SIZE) ||
>> -         (cflush->length > PAGE_SIZE) ||
>> -         (cflush->offset + cflush->length > PAGE_SIZE) ||
>> -         (cflush->op & ~(GNTTAB_CACHE_INVAL | GNTTAB_CACHE_CLEAN)) )
>> +    if ( cflush->op & ~(GNTTAB_CACHE_INVAL | GNTTAB_CACHE_CLEAN) )
>>           return -EINVAL;
>>   
>>       if ( cflush->length == 0 || cflush->op == 0 )
>>           return !*cur_ref ? 0 : -EILSEQ;
>>   
>> +    if ( (cflush->offset | cflush->length) > PAGE_SIZE ||
> 
> This is confusing. I understand you are trying to force the compiler to 
> optimize. But is it really worth it? After all, the rest of operation 
> will outweight this check (cache flush are quite expensive).

>From purely a performance point of view it may not be worth it. From
code size angle (taken globally) I already view this differently.
Plus I think that we ought to aim at avoiding undesirable patterns,
just because people tend to clone existing code when they can. Thing
is that (as per below) the two of us apparently disagree on what
"undesirable" is in cases like this one.

> We probably should take a more generic decision (and encode in our 
> policy) because you seem to like this pattern and I dislike it :). Not 
> sure what the others think.

Perhaps. If the folding alone was the problem, I'd accept to split (or
even undo) that part. But the earlier aspect you raised also needs
sorting before I can decide whether to adjust or whether to consider
the patch rejected.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:32:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:32:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683212.1062606 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLXk-0003JM-9Q; Tue, 20 Feb 2024 08:32:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683212.1062606; Tue, 20 Feb 2024 08:32:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLXk-0003JF-6p; Tue, 20 Feb 2024 08:32:00 +0000
Received: by outflank-mailman (input) for mailman id 683212;
 Tue, 20 Feb 2024 08:31:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLXi-0003J9-RP
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:31:58 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 809f9139-cfca-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:31:54 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-4126dd40a54so3960985e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:31:54 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l20-20020a1c7914000000b00412706c3ddasm476310wme.18.2024.02.20.00.31.53
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:31:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 809f9139-cfca-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708417914; x=1709022714; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2gsXVGDdG5g7JNoY2F7zo2dgWLlODMgbep354kVe0Xs=;
        b=fk8JKO09S63Q8a8H0MBR5sME5iQQCsWQGRCSmt2c8+kGhhr7XJVtpjU5dfauC6cV7P
         3fHDcrAZZCkhV2wlGWzL56ZkUMKPj3e5NkYk36LwaCDiixkLiYGvc4/4AzsLt3HzL1pA
         wCVa89aUWye4eEmFdx546fiVk7NufA5O+NYt4oRJHX6abV43Hn773FE5QABWYV9gEzGL
         uHQtCrxP/oVtcY+dhMuUyVPIjmLCpG4Ueoxfh4htunntyaJRRRbgqXgskJHcu+jMQG0A
         yHzBIZDbVAT/rq6QYb9qAI0exoNSB/StEg4qce5bWz6SyKgDgUCYZOrkxWSCzHSg7ptl
         /t9g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708417914; x=1709022714;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2gsXVGDdG5g7JNoY2F7zo2dgWLlODMgbep354kVe0Xs=;
        b=rsvAKVyCijHqIQAfNi9T9nrbG54oAwD7K+jomtOsf6GEuduQx0rQGZhH/3xYoTxvi0
         jC7ZHLZ336VxKqVLvTGhq6RKQHiwebARd/ePFcmSBGpJRTNYnQFBD2ujgxs7g5V+aaA1
         99/RVoyRXM2n7t55zfm50t4ewpGlw+mDWnNq+QjcAAiadbnclMvaFouON1bHdpSTXy9v
         0Gugi9SzN4GdCCnRj+y+OsmeIUHb6kRNgv7SAsjHMNECZzYd3CL9lxye4p5eupEuXsI3
         HxIHtjBya7RRC43cSbUK18rWV1XUrz6CB2QRxWkng6YEcT1PBrjASCoQZyzBtQq54pPH
         IWWw==
X-Forwarded-Encrypted: i=1; AJvYcCV8x4Imt5KOhUPSzbexi0GCgtLvchU8EAYyTr6YXQ4k0Boe3zxrWtb8Agc5A1MZsPF04fNtMB7WZ25ueut1R6ZE1Qug57lDzlZuO8aqPak=
X-Gm-Message-State: AOJu0YwHEZBDEKeJe80gPCpTAWlq2T1g2kHSQJVFPYqol00Uc00eXYBl
	GRh7cb26YCG3LHF+Zn0gsAxbQf7ywssLvVnTJQYsp20M3colZE9ghEAlz6IO5g==
X-Google-Smtp-Source: AGHT+IETFDBDCNgCgbw240+OkC+8iXImSaZrM4wEy0CqUYorEKTHJTc1LMybVmXFVydrg01c7o30ug==
X-Received: by 2002:a05:600c:4f45:b0:412:5dfc:238d with SMTP id m5-20020a05600c4f4500b004125dfc238dmr8011150wmq.14.1708417913847;
        Tue, 20 Feb 2024 00:31:53 -0800 (PST)
Message-ID: <4924e95e-0806-4939-90f3-2f4216fada63@suse.com>
Date: Tue, 20 Feb 2024 09:31:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair: add deviation for MISRA C:2012
 Rule 16.3
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <b1d2b64c8117d61ea42cf4e9feae128541eb0b61.1708348799.git.federico.serafini@bugseng.com>
 <a6e8bd32-cc32-4084-907d-e2cd6d46e3e6@suse.com>
 <f28aed5e-0983-48fa-828f-b2133ca35086@bugseng.com>
 <4d2c5a69-4807-4af3-84c2-128d16ee2b43@suse.com>
 <1f1fdfd8-1a34-411e-a0f2-d9bcb5b050fd@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <1f1fdfd8-1a34-411e-a0f2-d9bcb5b050fd@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 20.02.2024 09:16, Federico Serafini wrote:
> On 19/02/24 16:06, Jan Beulich wrote:
>> On 19.02.2024 15:59, Federico Serafini wrote:
>>> On 19/02/24 14:43, Jan Beulich wrote:
>>>> On 19.02.2024 14:24, Federico Serafini wrote:
>>>>> Update ECLAIR configuration to consider safe switch clauses ending
>>>>> with __{get,put}_user_bad().
>>>>>
>>>>> Update docs/misra/deviations.rst accordingly.
>>>>>
>>>>> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
>>>>
>>>> As mentioned I'm not happy with this, not the least because of it being
>>>> unclear why these two would be deviated, when there's no sign of a
>>>> similar deviation for, say, __bad_atomic_size(). Imo this approach
>>>> doesn't scale, and that's already leaving aside that the purpose of
>>>> identically named (pseudo-)helpers could differ between architectures,
>>>> thus putting under question ...
>>>>
>>>>> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
>>>>> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
>>>>> @@ -368,6 +368,10 @@ safe."
>>>>>    -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/BUG\\(\\);/))))"}
>>>>>    -doc_end
>>>>>    
>>>>> +-doc_begin="Switch clauses ending with constructs \"__get_user_bad()\" and \"__put_user_bad()\" are safe: they denote an unreachable program point."
>>>>> +-config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/__(put|get)_user_bad\\(\\);/))))"}
>>>>> +-doc_end
>>>>
>>>> ... the global scope of such a deviation. While it may not be a good idea,
>>>> even within an arch such (pseudo-)helpers could be used for multiple
>>>> distinct purposes.
>>>
>>> Would you agree with adding the missing break statement after
>>> the uses of __{put,get}_user_bad() (as it is already happening for
>>> uses of __bad_atomic_size())?
>>
>> I probably wouldn't mind that (despite being a little pointless).
>> Perhaps declaring them as noreturn would also help?
> 
> Yes, it will help.
> Is there any reason to have long as __get_user_bad()'s return value?
> It would be nicer to declare it as a void function and then add the
> noreturn attribute.

That's a historical leftover, which can be changed. Xen originally
derived quite a bit of code from Linux. If you go look at Linux 2.6.16,
you'll find why it was declared that way.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:44:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:44:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683219.1062617 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLjO-00059I-9Y; Tue, 20 Feb 2024 08:44:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683219.1062617; Tue, 20 Feb 2024 08:44:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLjO-00059B-5Z; Tue, 20 Feb 2024 08:44:02 +0000
Received: by outflank-mailman (input) for mailman id 683219;
 Tue, 20 Feb 2024 08:44:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLjM-000595-QI
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:44:00 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 302d2b1c-cfcc-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:43:58 +0100 (CET)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2d22b8c6e0dso35155281fa.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:43:58 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e4-20020adfe384000000b0033cfa00e497sm12625060wrm.64.2024.02.20.00.43.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:43:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 302d2b1c-cfcc-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708418638; x=1709023438; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=kOgTjEhG2Rt/NwiHYCsYDA5Wdy3noyY+yuNPEFFsPzg=;
        b=O7KtQojJn7TnyfS0OmV0h1QbhMZmJQABGHUu+KSzb6Xzwd17HKUOTFjUlMox1N6hEd
         8GzqIQmu9y8uEI2l1i13zMtfwRSEF/T78v5wkzkXCPBIk7Rq7rKIHRCy2qd1bOEPKv+e
         ITuy9ijBpmG5nE/YO9peOJKhPbUzP+e3m2gi7kbxyFDJFXS83QBzt6c58kXfsjQZs7SQ
         tr00fwdyRLyn5bZh/DvGIccfAf4psFjtX8DO4UzFyBgV1KgRDNlh2QQqXHBYn4Q3hS0g
         3hxO7zf8BXMsBssEEI6uhR8OrtllCBf09ELel1PmlFkUIHbm+K83HrJC0zhfHc4pBusu
         m72A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708418638; x=1709023438;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kOgTjEhG2Rt/NwiHYCsYDA5Wdy3noyY+yuNPEFFsPzg=;
        b=hrsMUif5xoqzPEHZNtMumJqXPl30eH58JjBMusZE/S14NYyga/dnX5OYfIYlPXzzuW
         jTSgmT2B3VdJfb+wAO3FF6709jB8/7PH7RADxdQkQ76eDM3FiDhivf9j2VpvaZJEwd6a
         zamKTForFBku2dDYAFBGF99EdB/uBfgzreXqzMXMxVxsthuO0f0v7xlSz0c3FoP5rjYt
         W5+mWQRw6Hb4wprQI0gPiVX6UFAfhEMVXiFDaI2W8hn93sR/WnzURDlJkQ4McRjuNQjJ
         x+/DHnVQ57G1DhXY/HyIKExgx5TYSlryhYB1W4vteOyMjzK5XRMrl1igOwg+kgZHXb9K
         /OCw==
X-Gm-Message-State: AOJu0YxCFlfg69FCEN7sIWLv4/fPI4WyFMHEGVJUWJcHNV79OXUr7ox+
	IoqaCzTqZLO2P3IcD1W5z20X/T0sZG6zUARBA0jsfZL+XahfLxSHKghFd8ifpn5HXhP6TFrxqPQ
	=
X-Google-Smtp-Source: AGHT+IH+V5abe8WkFg6AOmptl7bUoXVIyt0//P4+uEvxo1d/fUMC9mqKG0K8DRD0ExxEfTDfM6i+aA==
X-Received: by 2002:a2e:6e08:0:b0:2d0:aa06:f496 with SMTP id j8-20020a2e6e08000000b002d0aa06f496mr9653761ljc.40.1708418637886;
        Tue, 20 Feb 2024 00:43:57 -0800 (PST)
Message-ID: <55c3a781-17f8-47f2-9629-515e1aea77aa@suse.com>
Date: Tue, 20 Feb 2024 09:43:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Anthony Perard <anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] build: make sure build fails when running kconfig fails
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Because of using "-include", failure to (re)build auto.conf (with
auto.conf.cmd produced as a secondary target) won't stop make from
continuing the build. Arrange for it being possible to drop the - from
Rules.mk, requiring that the include be skipped for tools-only targets.
Note that relying on the inclusion in those cases wouldn't be correct
anyway, as it might be a stale file (yet to be rebuilt) which would be
included, while during initial build, the file would be absent
altogether.

Fixes: 8d4c17a90b0a ("xen/build: silence make warnings about missing auto.conf*")
Reported-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
An alternative to the %.conf rule adjustment would be to remove the file
only when the sub-make there failed.

--- a/xen/Makefile
+++ b/xen/Makefile
@@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
 # This exploits the 'multi-target pattern rule' trick.
 # The syncconfig should be executed only once to make all the targets.
 include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
+	$(Q)rm -f include/config/$*.conf
 	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
 
 ifeq ($(CONFIG_DEBUG),y)
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -15,7 +15,9 @@ srcdir := $(srctree)/$(src)
 PHONY := __build
 __build:
 
--include $(objtree)/include/config/auto.conf
+ifneq ($(firstword $(subst /, ,$(obj))),tools)
+include $(objtree)/include/config/auto.conf
+endif
 
 include $(XEN_ROOT)/Config.mk
 include $(srctree)/scripts/Kbuild.include


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:45:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:45:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683223.1062627 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLkc-0005eo-IH; Tue, 20 Feb 2024 08:45:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683223.1062627; Tue, 20 Feb 2024 08:45:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLkc-0005eh-Et; Tue, 20 Feb 2024 08:45:18 +0000
Received: by outflank-mailman (input) for mailman id 683223;
 Tue, 20 Feb 2024 08:45:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/fGf=J5=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rcLka-0005eV-LS
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:45:16 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5e10bc10-cfcc-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:45:15 +0100 (CET)
Received: from [192.168.1.229] (93-36-216-194.ip62.fastwebnet.it
 [93.36.216.194])
 by support.bugseng.com (Postfix) with ESMTPSA id B1A4C4EE073A;
 Tue, 20 Feb 2024 09:45:14 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e10bc10-cfcc-11ee-8a52-1f161083a0e0
Message-ID: <beff6113-0260-4dc3-998f-d1ea93088fe7@bugseng.com>
Date: Tue, 20 Feb 2024 09:45:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair: add deviation for MISRA C:2012
 Rule 16.3
Content-Language: en-US, it
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <b1d2b64c8117d61ea42cf4e9feae128541eb0b61.1708348799.git.federico.serafini@bugseng.com>
 <a6e8bd32-cc32-4084-907d-e2cd6d46e3e6@suse.com>
 <f28aed5e-0983-48fa-828f-b2133ca35086@bugseng.com>
 <4d2c5a69-4807-4af3-84c2-128d16ee2b43@suse.com>
 <1f1fdfd8-1a34-411e-a0f2-d9bcb5b050fd@bugseng.com>
 <4924e95e-0806-4939-90f3-2f4216fada63@suse.com>
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <4924e95e-0806-4939-90f3-2f4216fada63@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 20/02/24 09:31, Jan Beulich wrote:
> On 20.02.2024 09:16, Federico Serafini wrote:
>> On 19/02/24 16:06, Jan Beulich wrote:
>>> On 19.02.2024 15:59, Federico Serafini wrote:
>>>> On 19/02/24 14:43, Jan Beulich wrote:
>>>>> On 19.02.2024 14:24, Federico Serafini wrote:
>>>>>> Update ECLAIR configuration to consider safe switch clauses ending
>>>>>> with __{get,put}_user_bad().
>>>>>>
>>>>>> Update docs/misra/deviations.rst accordingly.
>>>>>>
>>>>>> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
>>>>>
>>>>> As mentioned I'm not happy with this, not the least because of it being
>>>>> unclear why these two would be deviated, when there's no sign of a
>>>>> similar deviation for, say, __bad_atomic_size(). Imo this approach
>>>>> doesn't scale, and that's already leaving aside that the purpose of
>>>>> identically named (pseudo-)helpers could differ between architectures,
>>>>> thus putting under question ...
>>>>>
>>>>>> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
>>>>>> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
>>>>>> @@ -368,6 +368,10 @@ safe."
>>>>>>     -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/BUG\\(\\);/))))"}
>>>>>>     -doc_end
>>>>>>     
>>>>>> +-doc_begin="Switch clauses ending with constructs \"__get_user_bad()\" and \"__put_user_bad()\" are safe: they denote an unreachable program point."
>>>>>> +-config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/__(put|get)_user_bad\\(\\);/))))"}
>>>>>> +-doc_end
>>>>>
>>>>> ... the global scope of such a deviation. While it may not be a good idea,
>>>>> even within an arch such (pseudo-)helpers could be used for multiple
>>>>> distinct purposes.
>>>>
>>>> Would you agree with adding the missing break statement after
>>>> the uses of __{put,get}_user_bad() (as it is already happening for
>>>> uses of __bad_atomic_size())?
>>>
>>> I probably wouldn't mind that (despite being a little pointless).
>>> Perhaps declaring them as noreturn would also help?
>>
>> Yes, it will help.
>> Is there any reason to have long as __get_user_bad()'s return value?
>> It would be nicer to declare it as a void function and then add the
>> noreturn attribute.
> 
> That's a historical leftover, which can be changed. Xen originally
> derived quite a bit of code from Linux. If you go look at Linux 2.6.16,
> you'll find why it was declared that way.

Thank you, I'll send a v2.

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:49:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:49:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683232.1062636 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLom-0006U7-4r; Tue, 20 Feb 2024 08:49:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683232.1062636; Tue, 20 Feb 2024 08:49:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLom-0006Ty-2H; Tue, 20 Feb 2024 08:49:36 +0000
Received: by outflank-mailman (input) for mailman id 683232;
 Tue, 20 Feb 2024 08:49:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JDQo=J5=bombadil.srs.infradead.org=BATV+d5dc426667c65ded4f96+7485+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rcLok-0006Sb-Oz
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:49:35 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f6b1af53-cfcc-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:49:32 +0100 (CET)
Received: from
 2a02-8389-2341-5b80-39d3-4735-9a3c-88d8.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:39d3:4735:9a3c:88d8] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rcLof-0000000DnT3-1reG; Tue, 20 Feb 2024 08:49:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6b1af53-cfcc-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:
	Content-ID:Content-Description:In-Reply-To:References;
	bh=6SfuAW972KnH6xK9mYEnu2XAsI1rZD/avgu8EQxehHo=; b=Y5mx2woS3KFrn6CgZJLKOsBj+J
	Xwmwlm0FH7ap7OXCaFYehCJOmmEicnpNRO0Hl3KlNxFgwMkGAcfCcb0mKdslDzLJpaVi4ICjcyc2c
	2+xuChxrG3o/ecPy0dwxKDLLgMNm5N9L8aU5j/RpV+HRRJkL7kKPPU0kE8n7Oce1Twm26S3iaZz7k
	VpaSnYugMXaMxnipTxpMcb/zio2et/Glx3VsFtRm+9HJSkhCDwV064YHupdN+U8f34/63JREO8GmQ
	6Dhn8HBm4NPhr8Bbn6+Sloc6MXlIcauVH5Q+rF4utCQfPY0VwG042LTNq2Yg3ICoUch7V/3Tl27hS
	ivNp54AQ==;
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>,
	xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: convert xen-blkfront to atomic queue limit updates
Date: Tue, 20 Feb 2024 09:49:31 +0100
Message-Id: <20240220084935.3282351-1-hch@lst.de>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Hi all,

this series converts xen-blkfront to the new atomic queue limits update
API in the block tree.  I don't have a Xen setup so this is compile
tested only.

Diffstat:
 xen-blkfront.c |   53 +++++++++++++++++++++++++++--------------------------
 1 file changed, 27 insertions(+), 26 deletions(-)


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:49:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:49:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683233.1062647 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLon-0006jj-DA; Tue, 20 Feb 2024 08:49:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683233.1062647; Tue, 20 Feb 2024 08:49:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLon-0006ja-9K; Tue, 20 Feb 2024 08:49:37 +0000
Received: by outflank-mailman (input) for mailman id 683233;
 Tue, 20 Feb 2024 08:49:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JDQo=J5=bombadil.srs.infradead.org=BATV+d5dc426667c65ded4f96+7485+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rcLom-0006U9-E7
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:49:36 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f83a6192-cfcc-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:49:35 +0100 (CET)
Received: from
 2a02-8389-2341-5b80-39d3-4735-9a3c-88d8.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:39d3:4735:9a3c:88d8] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rcLoi-0000000DnTu-1kAA; Tue, 20 Feb 2024 08:49:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f83a6192-cfcc-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:
	Reply-To:Cc:Content-Type:Content-ID:Content-Description;
	bh=XB8sWvj2sJzz3ZYwbZ8ds+HCirTtw0TSEkRcDSm3wbM=; b=iO3gbqYxXIpxTuUV2zaHjet/E4
	qbRGnvwyzQ2MW6HWb7TOiCVxrGrniN76RG6Cl55RBk38TaWUCAYMDpI/oeiP1WafJ0NibOt9drWiS
	7ZdgZcXtPdA2Nv5UfZZ82AaCGhzZMs5bT+DrnYzh2t8nPzlZx3RNZgEHD4Skvr16yHW60kTGQvrUm
	rdViw5WORq44kHbU2jAlPEjFqyPhJn4KkyylSRCTXiAnGT7FuAYLnmIAiCZ2W/eQs4RDeXNPb1xNh
	WpCctydJ0Fd8j/ehtsJjIwS0eu0OEcf0MxtmI5pe7nswvRbBJzctzU98Tn9aRR3/NZNrYZiVnOFM+
	IJF+6Vqg==;
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>,
	xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: [PATCH 1/4] xen-blkfront: set max_discard/secure erase limits to UINT_MAX
Date: Tue, 20 Feb 2024 09:49:32 +0100
Message-Id: <20240220084935.3282351-2-hch@lst.de>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240220084935.3282351-1-hch@lst.de>
References: <20240220084935.3282351-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Currently xen-blkfront set the max discard limit to the capacity of
the device, which is suboptimal when the capacity changes.  Just set
it to UINT_MAX, which has the same effect except and is simpler.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/block/xen-blkfront.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 4cc2884e748463..f78167cd5a6333 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -944,20 +944,18 @@ static const struct blk_mq_ops blkfront_mq_ops = {
 static void blkif_set_queue_limits(struct blkfront_info *info)
 {
 	struct request_queue *rq = info->rq;
-	struct gendisk *gd = info->gd;
 	unsigned int segments = info->max_indirect_segments ? :
 				BLKIF_MAX_SEGMENTS_PER_REQUEST;
 
 	blk_queue_flag_set(QUEUE_FLAG_VIRT, rq);
 
 	if (info->feature_discard) {
-		blk_queue_max_discard_sectors(rq, get_capacity(gd));
+		blk_queue_max_discard_sectors(rq, UINT_MAX);
 		rq->limits.discard_granularity = info->discard_granularity ?:
 						 info->physical_sector_size;
 		rq->limits.discard_alignment = info->discard_alignment;
 		if (info->feature_secdiscard)
-			blk_queue_max_secure_erase_sectors(rq,
-							   get_capacity(gd));
+			blk_queue_max_secure_erase_sectors(rq, UINT_MAX);
 	}
 
 	/* Hard sector size and max sectors impersonate the equiv. hardware. */
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:49:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:49:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683234.1062657 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLoq-00070C-K2; Tue, 20 Feb 2024 08:49:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683234.1062657; Tue, 20 Feb 2024 08:49:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLoq-000705-Gb; Tue, 20 Feb 2024 08:49:40 +0000
Received: by outflank-mailman (input) for mailman id 683234;
 Tue, 20 Feb 2024 08:49:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JDQo=J5=bombadil.srs.infradead.org=BATV+d5dc426667c65ded4f96+7485+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rcLop-0006Sb-3M
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:49:39 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f9df1a36-cfcc-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:49:37 +0100 (CET)
Received: from
 2a02-8389-2341-5b80-39d3-4735-9a3c-88d8.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:39d3:4735:9a3c:88d8] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rcLol-0000000DnUO-0bhr; Tue, 20 Feb 2024 08:49:35 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9df1a36-cfcc-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:
	Reply-To:Cc:Content-Type:Content-ID:Content-Description;
	bh=qIptjw8o7Oz7t0MBkVD69QjyMkpNVljZIeCgpSOPynU=; b=MiwA9mC1nUXdlp4EQ9i9PgWEcc
	zPQC3qBNpnwdxPm8TZxkbIhUkH9DVy5VbBvDvd5vK9z4cs7wyZpWo7K9ijlkJoBLZ4azIUJsK+AA6
	z+fghEuCvV7VbDZNb6gpNsMwiZBJKstzkf6HRABTqMX0LltW3i1VWQJscMPwb2guGUuN5ZHYm9WoM
	DLYQaU2kaY3KaH9tXa/CiU2lvl4KF5hMAOVh2mGQ++656UQx0Y4BlamhBHh/9nk+sdbD7XvOHvtnS
	SyqZB4+RBR5raaZoQ0BTTcijnSqjTXxxI2DmYhgSweS0VEuuYh8xYSCif+K1Ek+eBXXH6P1jhwGo8
	Oik2Hfdw==;
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>,
	xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: [PATCH 2/4] xen-blkfront: rely on the default discard granularity
Date: Tue, 20 Feb 2024 09:49:33 +0100
Message-Id: <20240220084935.3282351-3-hch@lst.de>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240220084935.3282351-1-hch@lst.de>
References: <20240220084935.3282351-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

The block layer now sets the discard granularity to the physical
block size default.  Take advantage of that in xen-blkfront and only
set the discard granularity if explicitly specified.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/block/xen-blkfront.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index f78167cd5a6333..1258f24b285500 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -951,8 +951,8 @@ static void blkif_set_queue_limits(struct blkfront_info *info)
 
 	if (info->feature_discard) {
 		blk_queue_max_discard_sectors(rq, UINT_MAX);
-		rq->limits.discard_granularity = info->discard_granularity ?:
-						 info->physical_sector_size;
+		if (info->discard_granularity)
+			rq->limits.discard_granularity = info->discard_granularity;
 		rq->limits.discard_alignment = info->discard_alignment;
 		if (info->feature_secdiscard)
 			blk_queue_max_secure_erase_sectors(rq, UINT_MAX);
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:49:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:49:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683235.1062667 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLos-0007Gl-R7; Tue, 20 Feb 2024 08:49:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683235.1062667; Tue, 20 Feb 2024 08:49:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLos-0007Gc-O9; Tue, 20 Feb 2024 08:49:42 +0000
Received: by outflank-mailman (input) for mailman id 683235;
 Tue, 20 Feb 2024 08:49:41 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JDQo=J5=bombadil.srs.infradead.org=BATV+d5dc426667c65ded4f96+7485+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rcLor-0006U9-2J
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:49:41 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fba6f962-cfcc-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:49:40 +0100 (CET)
Received: from
 2a02-8389-2341-5b80-39d3-4735-9a3c-88d8.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:39d3:4735:9a3c:88d8] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rcLoo-0000000DnUj-02wS; Tue, 20 Feb 2024 08:49:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fba6f962-cfcc-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:
	Reply-To:Cc:Content-Type:Content-ID:Content-Description;
	bh=qK76sLiKHDaIugADThNdCeAdCITdHp3R7WCpvZ/50Ys=; b=q+Xuv9UDwXHK+tkLGsYolzNTXG
	R4i+Mt0BZDHHWj0bFjoTaTV4Uf+ZLRjvOqwQnQ1QjdrjASB6LM+HWRzK56SCswy2aYtxd+lwQfOUz
	M1bu2VrcT7cgwkCOsvMNYemtsmevCZLF7gegxUDWG7sE9LYMV7Jb7xt7TyJ/9iUSdR4RG5IIK1ajt
	m5YEP4N9yEmyRAdoE62Vt5brNcei+BJohjnoWUwfYatrm4fKoPJDech8QrTVHR7nWoGlpFKJoBK6e
	/0W1zm/+qrYqHmXJDwLThtVyh3O1D/IfYEdgsCSgxxBHtaCdU8NUXUWYRwFAljvbygirx8S5992Sk
	E75cYLdg==;
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>,
	xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: [PATCH 3/4] xen-blkfront: don't redundantly set max_sements in blkif_recover
Date: Tue, 20 Feb 2024 09:49:34 +0100
Message-Id: <20240220084935.3282351-4-hch@lst.de>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240220084935.3282351-1-hch@lst.de>
References: <20240220084935.3282351-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

blkif_set_queue_limits already sets the max_sements limits, so don't do
it a second time.  Also remove a comment about a long fixe bug in
blk_mq_update_nr_hw_queues.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/block/xen-blkfront.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 1258f24b285500..7664638a0abbfa 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -2008,14 +2008,10 @@ static int blkif_recover(struct blkfront_info *info)
 	struct request *req, *n;
 	int rc;
 	struct bio *bio;
-	unsigned int segs;
 	struct blkfront_ring_info *rinfo;
 
 	blkfront_gather_backend_features(info);
-	/* Reset limits changed by blk_mq_update_nr_hw_queues(). */
 	blkif_set_queue_limits(info);
-	segs = info->max_indirect_segments ? : BLKIF_MAX_SEGMENTS_PER_REQUEST;
-	blk_queue_max_segments(info->rq, segs / GRANTS_PER_PSEG);
 
 	for_each_rinfo(info, rinfo, r_index) {
 		rc = blkfront_setup_indirect(rinfo);
@@ -2035,7 +2031,9 @@ static int blkif_recover(struct blkfront_info *info)
 	list_for_each_entry_safe(req, n, &info->requests, queuelist) {
 		/* Requeue pending requests (flush or discard) */
 		list_del_init(&req->queuelist);
-		BUG_ON(req->nr_phys_segments > segs);
+		BUG_ON(req->nr_phys_segments >
+		       (info->max_indirect_segments ? :
+			BLKIF_MAX_SEGMENTS_PER_REQUEST));
 		blk_mq_requeue_request(req, false);
 	}
 	blk_mq_start_stopped_hw_queues(info->rq, true);
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:49:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:49:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683237.1062677 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLow-0007cU-48; Tue, 20 Feb 2024 08:49:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683237.1062677; Tue, 20 Feb 2024 08:49:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLow-0007c8-0y; Tue, 20 Feb 2024 08:49:46 +0000
Received: by outflank-mailman (input) for mailman id 683237;
 Tue, 20 Feb 2024 08:49:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=JDQo=J5=bombadil.srs.infradead.org=BATV+d5dc426667c65ded4f96+7485+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rcLou-0006U9-7z
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:49:44 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fd812f13-cfcc-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:49:43 +0100 (CET)
Received: from
 2a02-8389-2341-5b80-39d3-4735-9a3c-88d8.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:39d3:4735:9a3c:88d8] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rcLor-0000000DnVL-0v8C; Tue, 20 Feb 2024 08:49:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd812f13-cfcc-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:
	Reply-To:Cc:Content-Type:Content-ID:Content-Description;
	bh=m4XY+o3oHjdzkw4wvaWBW0rkp+IC+rmZPoOQEQCyQz4=; b=efXEsm7JCVBnFfkn7jU1MZ/NFc
	mx9f+yMXUXy0hNPPikyILChtTBHSh0XlQSq8ac8RBIeHbvMaJksVKxk+HmTo2gaPGFMDnl9om2Isl
	w3dXKqPQo1P5Avnwv+qE+OYU6F3kUy2OOT47OtZIjAFeCqBBGE+NGe66Yb/+5+Brfgy/C7d5VmXYW
	QH4P4YTKPMlywHJmCAZpMxe1vgaVZac8kRbVBH6PRTAYo0McQA142uAiWOdB+0pRH6srypy/52W9m
	wvL3iy7q3dZY2Xt4SkPkeeVWwKCM5Umqmcb35w5b1+BSR8L6riFPQerE2vSwenZ38h8ubfhFrCgj5
	3YhiTmqw==;
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>,
	xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: [PATCH 4/4] xen-blkfront: atomically update queue limits
Date: Tue, 20 Feb 2024 09:49:35 +0100
Message-Id: <20240220084935.3282351-5-hch@lst.de>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240220084935.3282351-1-hch@lst.de>
References: <20240220084935.3282351-1-hch@lst.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Pass the initial queue limits to blk_mq_alloc_disk and use the
blkif_set_queue_limits API to update the limits on reconnect.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/block/xen-blkfront.c | 41 ++++++++++++++++++++----------------
 1 file changed, 23 insertions(+), 18 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 7664638a0abbfa..b77707ca2c5aa6 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -941,37 +941,35 @@ static const struct blk_mq_ops blkfront_mq_ops = {
 	.complete = blkif_complete_rq,
 };
 
-static void blkif_set_queue_limits(struct blkfront_info *info)
+static void blkif_set_queue_limits(struct blkfront_info *info,
+		struct queue_limits *lim)
 {
-	struct request_queue *rq = info->rq;
 	unsigned int segments = info->max_indirect_segments ? :
 				BLKIF_MAX_SEGMENTS_PER_REQUEST;
 
-	blk_queue_flag_set(QUEUE_FLAG_VIRT, rq);
-
 	if (info->feature_discard) {
-		blk_queue_max_discard_sectors(rq, UINT_MAX);
+		lim->max_hw_discard_sectors = UINT_MAX;
 		if (info->discard_granularity)
-			rq->limits.discard_granularity = info->discard_granularity;
-		rq->limits.discard_alignment = info->discard_alignment;
+			lim->discard_granularity = info->discard_granularity;
+		lim->discard_alignment = info->discard_alignment;
 		if (info->feature_secdiscard)
-			blk_queue_max_secure_erase_sectors(rq, UINT_MAX);
+			lim->max_secure_erase_sectors = UINT_MAX;
 	}
 
 	/* Hard sector size and max sectors impersonate the equiv. hardware. */
-	blk_queue_logical_block_size(rq, info->sector_size);
-	blk_queue_physical_block_size(rq, info->physical_sector_size);
-	blk_queue_max_hw_sectors(rq, (segments * XEN_PAGE_SIZE) / 512);
+	lim->logical_block_size = info->sector_size;
+	lim->physical_block_size = info->physical_sector_size;
+	lim->max_hw_sectors = (segments * XEN_PAGE_SIZE) / 512;
 
 	/* Each segment in a request is up to an aligned page in size. */
-	blk_queue_segment_boundary(rq, PAGE_SIZE - 1);
-	blk_queue_max_segment_size(rq, PAGE_SIZE);
+	lim->seg_boundary_mask = PAGE_SIZE - 1;
+	lim->max_segment_size = PAGE_SIZE;
 
 	/* Ensure a merged request will fit in a single I/O ring slot. */
-	blk_queue_max_segments(rq, segments / GRANTS_PER_PSEG);
+	lim->max_segments = segments / GRANTS_PER_PSEG;
 
 	/* Make sure buffer addresses are sector-aligned. */
-	blk_queue_dma_alignment(rq, 511);
+	lim->dma_alignment = 511;
 }
 
 static const char *flush_info(struct blkfront_info *info)
@@ -1068,6 +1066,7 @@ static int xlvbd_alloc_gendisk(blkif_sector_t capacity,
 		struct blkfront_info *info, u16 sector_size,
 		unsigned int physical_sector_size)
 {
+	struct queue_limits lim = {};
 	struct gendisk *gd;
 	int nr_minors = 1;
 	int err;
@@ -1134,11 +1133,13 @@ static int xlvbd_alloc_gendisk(blkif_sector_t capacity,
 	if (err)
 		goto out_release_minors;
 
-	gd = blk_mq_alloc_disk(&info->tag_set, NULL, info);
+	blkif_set_queue_limits(info, &lim);
+	gd = blk_mq_alloc_disk(&info->tag_set, &lim, info);
 	if (IS_ERR(gd)) {
 		err = PTR_ERR(gd);
 		goto out_free_tag_set;
 	}
+	blk_queue_flag_set(QUEUE_FLAG_VIRT, gd->queue);
 
 	strcpy(gd->disk_name, DEV_NAME);
 	ptr = encode_disk_name(gd->disk_name + sizeof(DEV_NAME) - 1, offset);
@@ -1160,7 +1161,6 @@ static int xlvbd_alloc_gendisk(blkif_sector_t capacity,
 	info->gd = gd;
 	info->sector_size = sector_size;
 	info->physical_sector_size = physical_sector_size;
-	blkif_set_queue_limits(info);
 
 	xlvbd_flush(info);
 
@@ -2004,14 +2004,19 @@ static int blkfront_probe(struct xenbus_device *dev,
 
 static int blkif_recover(struct blkfront_info *info)
 {
+	struct queue_limits lim;
 	unsigned int r_index;
 	struct request *req, *n;
 	int rc;
 	struct bio *bio;
 	struct blkfront_ring_info *rinfo;
 
+	lim = queue_limits_start_update(info->rq);
 	blkfront_gather_backend_features(info);
-	blkif_set_queue_limits(info);
+	blkif_set_queue_limits(info, &lim);
+	rc = queue_limits_commit_update(info->rq, &lim);
+	if (rc)
+		return rc;
 
 	for_each_rinfo(info, rinfo, r_index) {
 		rc = blkfront_setup_indirect(rinfo);
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:51:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:51:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683252.1062686 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLqS-0001bk-EQ; Tue, 20 Feb 2024 08:51:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683252.1062686; Tue, 20 Feb 2024 08:51:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLqS-0001bd-Bq; Tue, 20 Feb 2024 08:51:20 +0000
Received: by outflank-mailman (input) for mailman id 683252;
 Tue, 20 Feb 2024 08:51:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLqR-0001bV-K2
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:51:19 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 36810bef-cfcd-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:51:18 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-40fb3b5893eso28474595e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:51:18 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020adfcd0e000000b0033d14c96ec1sm12607957wrm.45.2024.02.20.00.51.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:51:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36810bef-cfcd-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708419078; x=1709023878; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=09ANSO5LYiG62Dw0zQaLZ0AeDC8RORPldvSVf/QkRgE=;
        b=FPlNtrgA6DyRRkzeK9Y8ST/ETGUPPkK4DUA5d4nztd01QLt0fPcgoNxvANHq2XfQd/
         Kx5/5wfZKn52ghMCtgUs9gH/rIPdqFcMaNXNgkI6SxWR24Ki6iDURFUufVwhSFawMgR4
         Qfc56KqP1nyd9NtYXDy3jmN7Z1d2nN0GfaDETDRWETLaCjo0Me6jbTuemLCboktvOxLL
         LaBudJx4LqiriidoLH6zEGv667QA9nGkATEMStqepPYImSWKtp6VDHUnndEWWfvMWVmT
         SCcaRrQetJfDntywcdIDgn02p67lGkWP64iXsFzWtiHMNSp8/1qRevWxQkA9nCmqqJTf
         OA7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419078; x=1709023878;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=09ANSO5LYiG62Dw0zQaLZ0AeDC8RORPldvSVf/QkRgE=;
        b=o1/YiBehsH+u2KI3X6luBnf7HTt/A523g3DjI+FPBC8yNFeMxr49tN7d6XgQOVRFWS
         ctZ+gPIKk0ERGPC+xkDz8/TZ95i1Fwz+jrbs0iLV1ihGpVTi6V4uXhMNnie66cIFHWo7
         tkUoFcm5UaLBpgGuU1Mwwko6h0soYSUIGMxOrtZAnZ6PslE7qY9pCTFSypliC98DLlqI
         hagALAECFrcznI6LYnkHs+MNkix0VLSBlnhelxvuxMKgYt51t9HR86xbK0cxanMUTJvA
         rQeFd3GASG0HR1YSdyfkRw45N8M2/aFsapvNngKZ7JrQzYNKYguPiFi7h+7R+wsbg+az
         nGvA==
X-Gm-Message-State: AOJu0YwhcnhNS1b5U/KbzHxbNE6RE7cWy4ULaHyk/eJjAI4sZ21fT6QW
	MpV0x4ywUOV+Xgfj6+NeIF8jcS1MoUOcz91Vdc8DJSj/3wzYZfV+hhP9Fh8uh+qytR2Ug2Hw/ec
	=
X-Google-Smtp-Source: AGHT+IGGnQO3MvuIv3Z0vWdAcnZcAW4DMbgAdvS1W7Z3xJcfLj47AoDY+WEhXjw+0rYVHLwdxYpbiA==
X-Received: by 2002:a5d:698b:0:b0:33d:50cd:b9ba with SMTP id g11-20020a5d698b000000b0033d50cdb9bamr2780820wru.21.1708419078076;
        Tue, 20 Feb 2024 00:51:18 -0800 (PST)
Message-ID: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Date: Tue, 20 Feb 2024 09:51:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v4 0/8] limit passing around of cpu_user_regs
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Unlike (synchronous) exception handlers, interrupt handlers don't normally
have a need to know the outer context's register state. Similarly, the vast
majority of key handlers has no need for such.

1: serial: fake IRQ-regs context in poll handlers
2: keyhandler: drop regs parameter from handle_keyregs()
3: serial: drop serial_rx_fn's regs parameter
4: PV-shim: drop pv_console_rx()'s regs parameter
5: serial: drop serial_[rt]x_interrupt()'s regs parameter
6: IRQ: drop regs parameter from handler functions
7: x86/APIC: drop regs parameter from direct vector handler functions
8: consolidate do_bug_frame() / bug_fn_t

I think it is now only the lack of an ack for the 1st patch which is keeping
this series from going in.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:52:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:52:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683258.1062696 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLrw-0002FI-SD; Tue, 20 Feb 2024 08:52:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683258.1062696; Tue, 20 Feb 2024 08:52:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLrw-0002FB-PZ; Tue, 20 Feb 2024 08:52:52 +0000
Received: by outflank-mailman (input) for mailman id 683258;
 Tue, 20 Feb 2024 08:52:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLrv-0002F1-Gq
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:52:51 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6cb49d55-cfcd-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:52:49 +0100 (CET)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-33d066f8239so2400258f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:52:49 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020adfcd0e000000b0033d14c96ec1sm12607957wrm.45.2024.02.20.00.52.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:52:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6cb49d55-cfcd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708419169; x=1709023969; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=WRod7K1c22IPHMjkp91uZ7nODYMwBBoK3lXCfwDbDuI=;
        b=aip4Y3F4NXVJyTTIb7c3oEiYSHFXiNPPMpPIodVp2NvONU59jEPhAPeSaFufdP4Ycf
         n9CMY4agsSxSchxxvFGKbhpgtBHpAkyJBW2LanWGbxIstkAbNzY7zk8WyLmqHJGS19Yn
         qUmN+mnxkWyDsQhqZ+fLykn/EVKzpxefFIvKbxlGYmLWCYIbWemLa73uQ17PDBHCTu3m
         2DT+lW6CTSAJfa+b0X2FnhLg8zQCn8vOo6R3oytd/eRM4o5TPiSmp8dPD4TimbvuUHJ6
         Rj2fi4ohng09hYDr16M8a204AWYPOcDTbPQ+lJ0XbSRAogYEon0qd/F/YNvLFWKGNxTn
         4sJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419169; x=1709023969;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=WRod7K1c22IPHMjkp91uZ7nODYMwBBoK3lXCfwDbDuI=;
        b=nVOkmC3rAmBAnua82VXcDik0rLfBcOgft4ZVaMT4P9w6vDWioZ0MMgtDVYSnUHloDJ
         g+5PIvP1ekKCyeZP/TIJrBOAFbe8J6OfET91ywpufwX/BWMDtJh5oGPLzTYB27kGn1gW
         3ZOXG67dtbRhwxA02zMSIw8gcc/ZjWNtBW76eA2MavNiVB2SRuTis7lvXPGbjUyAGK0O
         EOy4hbZY1+sHnXnnxenBmzabiQNWjFzbdl3TnIeGGBKcwyhkMadvx5gJFGew2kEg5p2P
         UoA09iYmqmhVmp5TAVc/1Mozi++3hHVZIOnzV42RSA3wI7lVnSAHnvNGN7wBMGFfAqfK
         uo9Q==
X-Gm-Message-State: AOJu0YwDyq5YVUi42HJEV9VhgyCWu7fxa/ddB33WEZlt/nhs1cH5LtuD
	RHHjiRUJjfkRFTfXSZqshWUS1PE76g8rpcGL4yEMENm7L2WJqUpGCu5fsgK7XZ4AyF/NOkF2PoA
	=
X-Google-Smtp-Source: AGHT+IEpHdJFuqVJhMhY5pj5+YOKKSzQlBfKrUOA5rRUTJ7UfOR2TR9gjLx4gSRka+z2g1v92KZ+7w==
X-Received: by 2002:adf:e450:0:b0:33b:60ca:269f with SMTP id t16-20020adfe450000000b0033b60ca269fmr9573558wrm.43.1708419169003;
        Tue, 20 Feb 2024 00:52:49 -0800 (PST)
Message-ID: <a3644326-7514-40f9-939c-2b6294dc5a8b@suse.com>
Date: Tue, 20 Feb 2024 09:52:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v4 1/8] serial: fake IRQ-regs context in poll handlers
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

In preparation of dropping the register parameters from
serial_[rt]x_interrupt() and in turn from IRQ handler functions,
register state needs making available another way for the few key
handlers which need it. Fake IRQ-like state.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
The use of guest_cpu_user_regs() in dbc_uart_poll() is inconsistent with
other console poll functions we have, and it's unclear whether that's
actually generally correct. It is only for this reason that it doesn't
need changing here.

Andrew suggested to move set_irq_regs() to BUGFRAME_run_fn handling;
it's not clear to me whether that would be (a) correct from an abstract
pov (that's exception, not interrupt context after all) and (b) really
beneficial.
---
v4: Drop xhci-dbc.c change as unnecessary with dump_registers() falling
    back to using guest_cpu_user_regs() as of patch 2, i.e. ahead of
    serial_rx_interrupt() having its 2nd parameter dropped.
v2: New.

--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -1253,6 +1253,7 @@ static void cf_check _ehci_dbgp_poll(str
     unsigned long flags;
     unsigned int timeout = MICROSECS(DBGP_CHECK_INTERVAL);
     bool empty = false;
+    struct cpu_user_regs *old_regs;
 
     if ( !dbgp->ehci_debug )
         return;
@@ -1268,12 +1269,17 @@ static void cf_check _ehci_dbgp_poll(str
         spin_unlock_irqrestore(&port->tx_lock, flags);
     }
 
+    /* Mimic interrupt context. */
+    old_regs = set_irq_regs(regs);
+
     if ( dbgp->in.chunk )
         serial_rx_interrupt(port, regs);
 
     if ( empty )
         serial_tx_interrupt(port, regs);
 
+    set_irq_regs(old_regs);
+
     if ( spin_trylock_irqsave(&port->tx_lock, flags) )
     {
         if ( dbgp->state == dbgp_idle && !dbgp->in.chunk &&
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -211,10 +211,14 @@ static void cf_check __ns16550_poll(stru
 {
     struct serial_port *port = this_cpu(poll_port);
     struct ns16550 *uart = port->uart;
+    struct cpu_user_regs *old_regs;
 
     if ( uart->intr_works )
         return; /* Interrupts work - no more polling */
 
+    /* Mimic interrupt context. */
+    old_regs = set_irq_regs(regs);
+
     while ( ns_read_reg(uart, UART_LSR) & UART_LSR_DR )
     {
         if ( ns16550_ioport_invalid(uart) )
@@ -227,6 +231,7 @@ static void cf_check __ns16550_poll(stru
         serial_tx_interrupt(port, regs);
 
 out:
+    set_irq_regs(old_regs);
     set_timer(&uart->timer, NOW() + MILLISECS(uart->timeout_ms));
 }
 



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:53:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:53:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683262.1062707 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLsF-0002g9-4A; Tue, 20 Feb 2024 08:53:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683262.1062707; Tue, 20 Feb 2024 08:53:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLsF-0002g0-18; Tue, 20 Feb 2024 08:53:11 +0000
Received: by outflank-mailman (input) for mailman id 683262;
 Tue, 20 Feb 2024 08:53:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLsD-0002dz-DP
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:53:09 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7785ecbe-cfcd-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:53:07 +0100 (CET)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-33d568fbf62so836196f8f.3
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:53:07 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020adfcd0e000000b0033d14c96ec1sm12607957wrm.45.2024.02.20.00.53.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:53:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7785ecbe-cfcd-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708419187; x=1709023987; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=r6p6O9Is+mZ3Yah69zqykHr7kyUZhreWLXWu9KFJdS8=;
        b=ZwBgJeQGpqX0OtIPWHS+bbvDztCAt56suCEN8s7ALpN6K+25XNRcKgsMsJpe18TxT5
         995Ho4xLTJ9actib/LxOcRHNAXbarlLdfNFgaHLAN+fTB4801hAFO2WHZ6zZoFiWIyeY
         xaTBeOtDLSgH72JrYs1AVwJH41bvXCcj8SrZEvtV5Dr7/zKmgMVdRsWP7SwspYcjMTA9
         8aGsB04AfnJBEbto3twRCRSc/iSDSdjkktcLwgEbUaIByzE1KtRqpBe2uUv+gyHjdnmw
         cjDvljO5nEgiiS//tGewBa4M+F+E5R+MHiRXZJkJ2f1M5hjgzgyCkLxFP3EewK2s6Bs4
         bbdw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419187; x=1709023987;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=r6p6O9Is+mZ3Yah69zqykHr7kyUZhreWLXWu9KFJdS8=;
        b=oUSRWcCl1qPEgw7LANI3N2rdM9bhgjjnykoUqhA4MJYOjTrp/SQZDgOe1PjzR2Q1YC
         tdUIZ3IXkIYtxZ6q6tAv/CqTnXMalYmT3mTx1qRb0gRmfvuBjOWm8CCRDTpA7foOH0rY
         F1HUmGVSpxvUkISUZZKpTIovIivsTfdfhomaAloxf8BEYLr1R+1X2awbHTga/QNYy85n
         gwRtpNy85YgwneAOLg8N/M6HvFIC/uvaDSfZuLkFRfqhwdL2LsbL3vX4o3EjZH6byZBI
         +NQY/9CeV4s3zZ5kH6oSTOlZVBYE82XRW+mvHBixewE/5VmyaCgbBr0h7eentd67Yix/
         yceA==
X-Gm-Message-State: AOJu0YxFTnJJTxXIXXqk8n9mXHVjBMljGIITd70k8IZhE1pHL520HXpy
	leChpNgV4sJbrwEj8OnTAcdIRfHLPX/3A9mHr5vAj7i8Fq8qWDydotOOyWQ3WweGqyXhH15wpC4
	=
X-Google-Smtp-Source: AGHT+IGJcp+HX1PmMQpXJxFiKlDh5AMnw513Hl/8pUFan8/kp8S2U6yj2x7GBWXsGOJUlwYLsqQatQ==
X-Received: by 2002:a5d:6288:0:b0:33d:6bd5:d372 with SMTP id k8-20020a5d6288000000b0033d6bd5d372mr637680wru.15.1708419187200;
        Tue, 20 Feb 2024 00:53:07 -0800 (PST)
Message-ID: <8c76ba07-c850-4e3e-aac8-e5516877d848@suse.com>
Date: Tue, 20 Feb 2024 09:53:06 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v4 2/8] keyhandler: drop regs parameter from handle_keyregs()
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

In preparation for further removal of regs parameters, drop it here. In
the two places where it's actually needed, retrieve IRQ context if
available, or else guest context.

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
As an alternative to the new boolean parameter, I wonder if we couldn't
special-case the idle vCPU case: It's only there where we would not have
proper context retrievable via guest_cpu_user_regs().
---
v3: Re-base.
v2: New.

--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -73,12 +73,12 @@ static struct keyhandler {
 
 static void cf_check keypress_action(void *unused)
 {
-    handle_keypress(keypress_key, NULL);
+    handle_keypress(keypress_key, true);
 }
 
 static DECLARE_TASKLET(keypress_tasklet, keypress_action, NULL);
 
-void handle_keypress(unsigned char key, struct cpu_user_regs *regs)
+void handle_keypress(unsigned char key, bool need_context)
 {
     struct keyhandler *h;
 
@@ -88,7 +88,7 @@ void handle_keypress(unsigned char key,
     if ( !in_irq() || h->irq_callback )
     {
         console_start_log_everything();
-        h->irq_callback ? h->irq_fn(key, regs) : h->fn(key);
+        h->irq_callback ? h->irq_fn(key, need_context) : h->fn(key);
         console_end_log_everything();
     }
     else
@@ -169,7 +169,7 @@ void cf_check dump_execstate(struct cpu_
 }
 
 static void cf_check dump_registers(
-    unsigned char key, struct cpu_user_regs *regs)
+    unsigned char key, bool need_context)
 {
     unsigned int cpu;
 
@@ -182,8 +182,8 @@ static void cf_check dump_registers(
     cpumask_copy(&dump_execstate_mask, &cpu_online_map);
 
     /* Get local execution state out immediately, in case we get stuck. */
-    if ( regs )
-        dump_execstate(regs);
+    if ( !need_context )
+        dump_execstate(get_irq_regs() ?: guest_cpu_user_regs());
     else
         run_in_exception_handler(dump_execstate);
 
@@ -245,8 +245,7 @@ static void cf_check dump_hwdom_register
     }
 }
 
-static void cf_check reboot_machine(
-    unsigned char key, struct cpu_user_regs *regs)
+static void cf_check reboot_machine(unsigned char key, bool unused)
 {
     printk("'%c' pressed -> rebooting machine\n", key);
     machine_restart(0);
@@ -474,8 +473,7 @@ static void cf_check run_all_nonirq_keyh
 static DECLARE_TASKLET(run_all_keyhandlers_tasklet,
                        run_all_nonirq_keyhandlers, NULL);
 
-static void cf_check run_all_keyhandlers(
-    unsigned char key, struct cpu_user_regs *regs)
+static void cf_check run_all_keyhandlers(unsigned char key, bool need_context)
 {
     struct keyhandler *h;
     unsigned int k;
@@ -491,7 +489,7 @@ static void cf_check run_all_keyhandlers
         if ( !h->irq_fn || !h->diagnostic || !h->irq_callback )
             continue;
         printk("[%c: %s]\n", k, h->desc);
-        h->irq_fn(k, regs);
+        h->irq_fn(k, need_context);
     }
 
     watchdog_enable();
@@ -500,8 +498,7 @@ static void cf_check run_all_keyhandlers
     tasklet_schedule(&run_all_keyhandlers_tasklet);
 }
 
-static void cf_check do_toggle_alt_key(
-    unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_toggle_alt_key(unsigned char key, bool unused)
 {
     alt_key_handling = !alt_key_handling;
     printk("'%c' pressed -> using %s key handling\n", key,
@@ -566,7 +563,7 @@ void keyhandler_crash_action(enum crash_
         if ( *action == '+' )
             mdelay(10);
         else
-            handle_keypress(*action, NULL);
+            handle_keypress(*action, true);
         action++;
     }
 }
--- a/xen/common/sysctl.c
+++ b/xen/common/sysctl.c
@@ -134,7 +134,7 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xe
         {
             if ( copy_from_guest_offset(&c, op->u.debug_keys.keys, i, 1) )
                 goto out;
-            handle_keypress(c, guest_cpu_user_regs());
+            handle_keypress(c, false);
         }
         ret = 0;
         copyback = 0;
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -279,7 +279,7 @@ static int *__read_mostly upper_thresh_a
 static int *__read_mostly lower_thresh_adj = &xenlog_lower_thresh;
 static const char *__read_mostly thresh_adj = "standard";
 
-static void cf_check do_toggle_guest(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_toggle_guest(unsigned char key, bool unused)
 {
     if ( upper_thresh_adj == &xenlog_upper_thresh )
     {
@@ -306,13 +306,13 @@ static void do_adj_thresh(unsigned char
            loglvl_str(*upper_thresh_adj));
 }
 
-static void cf_check do_inc_thresh(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_inc_thresh(unsigned char key, bool unused)
 {
     ++*lower_thresh_adj;
     do_adj_thresh(key);
 }
 
-static void cf_check do_dec_thresh(unsigned char key, struct cpu_user_regs *regs)
+static void cf_check do_dec_thresh(unsigned char key, bool unused)
 {
     if ( *lower_thresh_adj )
         --*lower_thresh_adj;
@@ -531,7 +531,7 @@ static void __serial_rx(char c, struct c
     switch ( console_rx )
     {
     case 0:
-        return handle_keypress(c, regs);
+        return handle_keypress(c, false);
 
     case 1:
         /*
--- a/xen/include/xen/keyhandler.h
+++ b/xen/include/xen/keyhandler.h
@@ -24,9 +24,8 @@ typedef void (keyhandler_fn_t)(unsigned
  *
  * Called in hardirq context with interrupts disabled.
  */
-struct cpu_user_regs;
 typedef void (irq_keyhandler_fn_t)(unsigned char key,
-                                   struct cpu_user_regs *regs);
+                                   bool need_context);
 
 /* Initialize keytable with default handlers. */
 void initialize_keytable(void);
@@ -46,7 +45,7 @@ void register_irq_keyhandler(unsigned ch
                              bool diagnostic);
 
 /* Inject a keypress into the key-handling subsystem. */
-extern void handle_keypress(unsigned char key, struct cpu_user_regs *regs);
+extern void handle_keypress(unsigned char key, bool need_context);
 
 enum crash_reason {
     CRASHREASON_PANIC,



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:53:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:53:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683264.1062718 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLsH-0002ve-CH; Tue, 20 Feb 2024 08:53:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683264.1062718; Tue, 20 Feb 2024 08:53:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLsH-0002vV-7g; Tue, 20 Feb 2024 08:53:13 +0000
Received: by outflank-mailman (input) for mailman id 683264;
 Tue, 20 Feb 2024 08:53:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=zzaM=J5=linaro.org=manos.pitsidianakis@srs-se1.protection.inumbo.net>)
 id 1rcLsF-0002dz-GK
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:53:11 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7959e3ec-cfcd-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:53:10 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a3e8c1e4aa7so194147066b.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:53:10 -0800 (PST)
Received: from localhost.localdomain (adsl-245.37.6.163.tellas.gr.
 [37.6.163.245]) by smtp.gmail.com with ESMTPSA id
 cu3-20020a170906ba8300b00a3d014fa12esm3747876ejd.196.2024.02.20.00.53.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 00:53:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7959e3ec-cfcd-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1708419190; x=1709023990; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=74P+PKZ6orTLj/5plkkdrxWuHnGI5wedPF2+fgfkjP0=;
        b=Wy2Zd/LcuH+3l8id/j9RzCXBT3y4P0QzKrhPQ3jsLvYKL5NM7v74NgOJ+tyq2ONb0V
         NTcfTwS5A2Kd1yOxY8L8KDNyUrFrDv+0WzNenc5F6e42yXuA/tH2jZryHP3M+Yebzp8l
         Qd12qScioQ87FOmZtY6fLQi1FM0rwbRoqlj754Y8ESvFKPYlrr9wqhGB79dijbsRmzKk
         +aJ3mawLZskCs+oa5jwUQUH11A5pfrHfHU0Y2e8r6RaY6nB1TWzTdZhkl8Glgj/3Q6EN
         HKtZkZpn1Qp+YsWJYHBGCAiD+sGzoJi7XO9faLfclIqhy0ei0trB9PJ5VNcR98fgBw0H
         frag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419190; x=1709023990;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=74P+PKZ6orTLj/5plkkdrxWuHnGI5wedPF2+fgfkjP0=;
        b=XTIQenuEf85xei1cIa2bz+L2fq9OprFu1JhNOlgGQ9yUqBRgLX42xiKqYbqMrOrc+z
         FjUvFsZexwalEmtBYG6VgIj4AQHRJhfRQtD+EbJsUaOQz+4znaWJgxJ2efRSFWrd6eLF
         UXMNPawYZzB1UoE8WH/YJt4atA8XsWnxNsLhKxc7Qz63qu6OArDOnt98nRHz8OgbSLtg
         EdCStW3yii0dG6kagz8FzvBpl850IMNjDIJN1zsfsxgcB5RYsuNbFCXlxgwWRXaxTS7q
         ArysrVj1Py1py5lRVf6srBjlGkF5h+4EdRNwjjH1sOohIteI+5Tp9bF9gmuWX1Nhh04v
         V4pg==
X-Forwarded-Encrypted: i=1; AJvYcCVXJS8X0Bv4W43tVq2ZY2jdoHnHoQe7zSffnD2G2rIsuA96duDgJ3V/BqV95dA9ESbFc/+Q7xFALKLjjkdL+RK5BFoXYN9fhPJZLmb0yEc=
X-Gm-Message-State: AOJu0YxVl4kQElLKjY5KLEevY5AKxxG2gQhkMdpmnQWzhkOaSxslg2dN
	CNBUfobRyOWAVfj98VWxfozNT5t/bx0Lkl4CHF9l4jIwYN07bksadxb8FZvlfmk=
X-Google-Smtp-Source: AGHT+IH0/5iDtgDsLwpGN/YmUhRGA55XSWPIdNMzHkhCmG1fsWPv29/an+50Zc6oVwnQznVwV1dJEg==
X-Received: by 2002:a17:906:57ca:b0:a3f:ce8:1234 with SMTP id u10-20020a17090657ca00b00a3f0ce81234mr403891ejr.68.1708419190280;
        Tue, 20 Feb 2024 00:53:10 -0800 (PST)
From: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
To: qemu-trivial@nongnu.org
Cc: qemu-devel@nongnu.org,
	Michael Tokarev <mjt@tls.msk.ru>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Paul Durrant <paul@xen.org>,
	xen-devel@lists.xenproject.org
Subject: [PATCH v1 03/21] Xen headers: correct typos
Date: Tue, 20 Feb 2024 10:52:10 +0200
Message-Id: <dca380ada7f7eabfb473828eccafc2902b1985b7.1708419115.git.manos.pitsidianakis@linaro.org>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <cover.1708419115.git.manos.pitsidianakis@linaro.org>
References: <cover.1708419115.git.manos.pitsidianakis@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF8
Content-Transfer-Encoding: 8bit

Correct typos automatically found with the `typos` tool
<https://crates.io/crates/typos>

Signed-off-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
---
 include/hw/xen/interface/arch-x86/xen-x86_64.h | 2 +-
 include/hw/xen/interface/arch-x86/xen.h        | 2 +-
 include/hw/xen/interface/event_channel.h       | 2 +-
 include/hw/xen/interface/grant_table.h         | 2 +-
 include/hw/xen/interface/hvm/hvm_op.h          | 2 +-
 include/hw/xen/interface/io/blkif.h            | 4 ++--
 include/hw/xen/interface/io/fbif.h             | 2 +-
 include/hw/xen/interface/io/kbdif.h            | 2 +-
 include/hw/xen/interface/io/ring.h             | 2 +-
 include/hw/xen/interface/memory.h              | 2 +-
 include/hw/xen/interface/physdev.h             | 4 ++--
 include/hw/xen/interface/xen.h                 | 4 ++--
 12 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/include/hw/xen/interface/arch-x86/xen-x86_64.h b/include/hw/xen/interface/arch-x86/xen-x86_64.h
index 5d9035ed22..3e94eeb0bf 100644
--- a/include/hw/xen/interface/arch-x86/xen-x86_64.h
+++ b/include/hw/xen/interface/arch-x86/xen-x86_64.h
@@ -89,7 +89,7 @@
  *   RING1 -> RING3 kernel mode.
  *   RING2 -> RING3 kernel mode.
  *   RING3 -> RING3 user mode.
- * However RING0 indicates that the guest kernel should return to iteself
+ * However RING0 indicates that the guest kernel should return to itself
  * directly with
  *      orb   $3,1*8(%rsp)
  *      iretq
diff --git a/include/hw/xen/interface/arch-x86/xen.h b/include/hw/xen/interface/arch-x86/xen.h
index c0f4551247..323bd06a63 100644
--- a/include/hw/xen/interface/arch-x86/xen.h
+++ b/include/hw/xen/interface/arch-x86/xen.h
@@ -156,7 +156,7 @@ typedef uint64_t tsc_timestamp_t; /* RDTSC timestamp */
  * information in this structure is updated, the fields read include: fpu_ctxt
  * (if VGCT_I387_VALID is set), flags, user_regs and debugreg[*].
  *
- * Note: VCPUOP_initialise for HVM guests is non-symetric with
+ * Note: VCPUOP_initialise for HVM guests is non-symmetric with
  * DOMCTL_setvcpucontext, and uses struct vcpu_hvm_context from hvm/hvm_vcpu.h
  */
 struct vcpu_guest_context {
diff --git a/include/hw/xen/interface/event_channel.h b/include/hw/xen/interface/event_channel.h
index 0d91a1c4af..d446863230 100644
--- a/include/hw/xen/interface/event_channel.h
+++ b/include/hw/xen/interface/event_channel.h
@@ -302,7 +302,7 @@ typedef struct evtchn_set_priority evtchn_set_priority_t;
  * ` enum neg_errnoval
  * ` HYPERVISOR_event_channel_op_compat(struct evtchn_op *op)
  * `
- * Superceded by new event_channel_op() hypercall since 0x00030202.
+ * Superseded by new event_channel_op() hypercall since 0x00030202.
  */
 struct evtchn_op {
     uint32_t cmd; /* enum event_channel_op */
diff --git a/include/hw/xen/interface/grant_table.h b/include/hw/xen/interface/grant_table.h
index 1dfa17a6d0..7652e8bf81 100644
--- a/include/hw/xen/interface/grant_table.h
+++ b/include/hw/xen/interface/grant_table.h
@@ -607,7 +607,7 @@ DEFINE_XEN_GUEST_HANDLE(gnttab_cache_flush_t);
  /*
   * GNTMAP_contains_pte subflag:
   *  0 => This map request contains a host virtual address.
-  *  1 => This map request contains the machine addess of the PTE to update.
+  *  1 => This map request contains the machine address of the PTE to update.
   */
 #define _GNTMAP_contains_pte    (4)
 #define GNTMAP_contains_pte     (1<<_GNTMAP_contains_pte)
diff --git a/include/hw/xen/interface/hvm/hvm_op.h b/include/hw/xen/interface/hvm/hvm_op.h
index e22adf0319..3defe1c108 100644
--- a/include/hw/xen/interface/hvm/hvm_op.h
+++ b/include/hw/xen/interface/hvm/hvm_op.h
@@ -337,7 +337,7 @@ struct xen_hvm_altp2m_op {
 #define HVMOP_altp2m_vcpu_disable_notify  13
 /* Get the active vcpu p2m index */
 #define HVMOP_altp2m_get_p2m_idx          14
-/* Set the "Supress #VE" bit for a range of pages */
+/* Set the "Suppress #VE" bit for a range of pages */
 #define HVMOP_altp2m_set_suppress_ve_multi 15
 /* Set visibility for a given altp2m view */
 #define HVMOP_altp2m_set_visibility       16
diff --git a/include/hw/xen/interface/io/blkif.h b/include/hw/xen/interface/io/blkif.h
index 22f1eef0c0..4356956975 100644
--- a/include/hw/xen/interface/io/blkif.h
+++ b/include/hw/xen/interface/io/blkif.h
@@ -42,7 +42,7 @@
  * All data in the XenStore is stored as strings.  Nodes specifying numeric
  * values are encoded in decimal.  Integer value ranges listed below are
  * expressed as fixed sized integer types capable of storing the conversion
- * of a properly formated node string, without loss of information.
+ * of a properly formatted node string, without loss of information.
  *
  * Any specified default value is in effect if the corresponding XenBus node
  * is not present in the XenStore.
@@ -406,7 +406,7 @@
  *     further requests may reuse these grants and require write permissions.
  * (9) Linux implementation doesn't have a limit on the maximum number of
  *     grants that can be persistently mapped in the frontend driver, but
- *     due to the frontent driver implementation it should never be bigger
+ *     due to the frontend driver implementation it should never be bigger
  *     than RING_SIZE * BLKIF_MAX_SEGMENTS_PER_REQUEST.
  *(10) The discard-secure property may be present and will be set to 1 if the
  *     backing device supports secure discard.
diff --git a/include/hw/xen/interface/io/fbif.h b/include/hw/xen/interface/io/fbif.h
index 93c73195d8..46046f6775 100644
--- a/include/hw/xen/interface/io/fbif.h
+++ b/include/hw/xen/interface/io/fbif.h
@@ -72,7 +72,7 @@ union xenfb_out_event
  * just ignore it.  Frontends that use the advice should immediately
  * refresh the framebuffer (and send an update notification event if
  * those have been requested), then use the update frequency to guide
- * their periodical refreshs.
+ * their periodical refresh.
  */
 #define XENFB_TYPE_REFRESH_PERIOD 1
 #define XENFB_NO_REFRESH 0
diff --git a/include/hw/xen/interface/io/kbdif.h b/include/hw/xen/interface/io/kbdif.h
index 4bde6b3821..5794cc33f8 100644
--- a/include/hw/xen/interface/io/kbdif.h
+++ b/include/hw/xen/interface/io/kbdif.h
@@ -22,7 +22,7 @@
  * All data in XenStore is stored as strings.  Nodes specifying numeric
  * values are encoded in decimal. Integer value ranges listed below are
  * expressed as fixed sized integer types capable of storing the conversion
- * of a properly formated node string, without loss of information.
+ * of a properly formatted node string, without loss of information.
  *
  *****************************************************************************
  *                            Backend XenBus Nodes
diff --git a/include/hw/xen/interface/io/ring.h b/include/hw/xen/interface/io/ring.h
index 025939278b..6d8f63d7a5 100644
--- a/include/hw/xen/interface/io/ring.h
+++ b/include/hw/xen/interface/io/ring.h
@@ -145,7 +145,7 @@ typedef struct __name##_back_ring __name##_back_ring_t
  *
  * N.B. these macros do NO INTERLOCKS OR FLOW CONTROL.
  * This is OK in 1-for-1 request-response situations where the
- * requestor (front end) never has more than RING_SIZE()-1
+ * requester (front end) never has more than RING_SIZE()-1
  * outstanding requests.
  */
 
diff --git a/include/hw/xen/interface/memory.h b/include/hw/xen/interface/memory.h
index 29cf5c8239..ad11573d90 100644
--- a/include/hw/xen/interface/memory.h
+++ b/include/hw/xen/interface/memory.h
@@ -167,7 +167,7 @@ struct xen_machphys_mfn_list {
 
     /*
      * Pointer to buffer to fill with list of extent starts. If there are
-     * any large discontiguities in the machine address space, 2MB gaps in
+     * any large discontinuities in the machine address space, 2MB gaps in
      * the machphys table will be represented by an MFN base of zero.
      */
     XEN_GUEST_HANDLE(xen_pfn_t) extent_start;
diff --git a/include/hw/xen/interface/physdev.h b/include/hw/xen/interface/physdev.h
index f0c0d4727c..053c045cf5 100644
--- a/include/hw/xen/interface/physdev.h
+++ b/include/hw/xen/interface/physdev.h
@@ -206,7 +206,7 @@ typedef struct physdev_manage_pci_ext physdev_manage_pci_ext_t;
 DEFINE_XEN_GUEST_HANDLE(physdev_manage_pci_ext_t);
 
 /*
- * Argument to physdev_op_compat() hypercall. Superceded by new physdev_op()
+ * Argument to physdev_op_compat() hypercall. Superseded by new physdev_op()
  * hypercall since 0x00030202.
  */
 struct physdev_op {
@@ -332,7 +332,7 @@ DEFINE_XEN_GUEST_HANDLE(physdev_dbgp_op_t);
 
 #if __XEN_INTERFACE_VERSION__ < 0x00040600
 /*
- * These all-capitals physdev operation names are superceded by the new names
+ * These all-capitals physdev operation names are superseded by the new names
  * (defined above) since interface version 0x00030202. The guard above was
  * added post-4.5 only though and hence shouldn't check for 0x00030202.
  */
diff --git a/include/hw/xen/interface/xen.h b/include/hw/xen/interface/xen.h
index 920567e006..b8fe746f78 100644
--- a/include/hw/xen/interface/xen.h
+++ b/include/hw/xen/interface/xen.h
@@ -733,8 +733,8 @@ struct shared_info {
      * are delivered by this mechanism:
      *  1. Bi-directional inter- and intra-domain connections. Domains must
      *     arrange out-of-band to set up a connection (usually by allocating
-     *     an unbound 'listener' port and avertising that via a storage service
-     *     such as xenstore).
+     *     an unbound 'listener' port and advertising that via a storage
+     *     service such as xenstore).
      *  2. Physical interrupts. A domain with suitable hardware-access
      *     privileges can bind an event-channel port to a physical interrupt
      *     source.
-- 
γαῖα πυρί μιχθήτω



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:53:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:53:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683271.1062727 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLsd-0003mk-Oo; Tue, 20 Feb 2024 08:53:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683271.1062727; Tue, 20 Feb 2024 08:53:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLsd-0003mc-LA; Tue, 20 Feb 2024 08:53:35 +0000
Received: by outflank-mailman (input) for mailman id 683271;
 Tue, 20 Feb 2024 08:53:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLsc-0002F1-Hp
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:53:34 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 868ac1f3-cfcd-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:53:32 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-412698cdd77so8099685e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:53:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020adfcd0e000000b0033d14c96ec1sm12607957wrm.45.2024.02.20.00.53.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:53:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 868ac1f3-cfcd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708419212; x=1709024012; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=uPp0elCmf96AfayiOwn8U2QJcfk2O81aOrNPDxDcp70=;
        b=CxZovJAV5o3cIq+PVjJeqWA671HKoDoc5D9KDuXqyRtBYbzZEkgnRb5fFEqdfcYUum
         6JivygeyVIJQvjQWoSJz86C2gi/KP0WSHM+FRddzXMTa+qfFX3YYMBBFVr0atoAgSCeX
         8Y1pawcF4RDU3T2ZGGy7J5BQErRAJp1WJvnoMaRYny+1+idV8zKf3CTzgxl7MEUwTELs
         JhW1MRCho6Ziz5aZmQ4g7AJ27HIYN55oH+Rcqh7zO+XNOPi9AgLwbi2hbteXEA3EOTkC
         NpvcqvsjhAlLNADFTFI/x2tZfq1849QCvLl7Q/c9OGLxH9DMmqYyuAhR8csLPK2GT4lb
         9n1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419212; x=1709024012;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=uPp0elCmf96AfayiOwn8U2QJcfk2O81aOrNPDxDcp70=;
        b=vk2MtnlFsFjvpK4ZMd5CC2CR1XNEbnhIOrBXV+duJIQEqQWgnN4zCg/L4TO4YIOJQ+
         Mc7A8lFEoGMUP4S4x3xpNi7cDoDEviF0ikvtE+qYcq+DQRzilWAdJYZ4q6ppVhoo12LH
         0FiPBF2kKINvbeYPgPsU40Tp/rgfPuD3olkqVFsSayQzxIJPIwvilCVkk/BEd4FkFrob
         O5FWTgFE49DyZsdzCw3NHLP9lepRKpyyZlymw/p1q+GDld4EjCnSgFqhdUze0e1D6e8p
         4C4CsKCLjL8bN+iFPpQPl31Kv/awR5uG+xUH+hNU4dNkIQupH0eXlD5Vu5sp0EeaAq04
         G9/A==
X-Gm-Message-State: AOJu0YzFNyDVEfKFR4sM/b1DkkFvbKjACFZmww0iwOP+sUXDvv3bIf9m
	iTdG/rxGWjOxWTTMTB0szBRT3uRHUiKVHg+NJuVn91gr9Y6rWZnn+sDuLScoQW5RLUBEjbzt64s
	=
X-Google-Smtp-Source: AGHT+IFHtjbHbKo4h5YKKP2rg1k+1j2Twm/B3b+aRITSLkoDCKBr6vLlArxlwCAT4vJ98Z5XQY+t1w==
X-Received: by 2002:a05:600c:45d1:b0:412:6de4:cbe1 with SMTP id s17-20020a05600c45d100b004126de4cbe1mr1344836wmo.15.1708419212431;
        Tue, 20 Feb 2024 00:53:32 -0800 (PST)
Message-ID: <9836c01a-7f9f-4f8e-ba7d-1a7ab7e2e4e1@suse.com>
Date: Tue, 20 Feb 2024 09:53:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v4 3/8] serial: drop serial_rx_fn's regs parameter
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

It's simply not needed anymore.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
v2: Re-base over earlier (new/split) patches.

--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -526,7 +526,7 @@ static void switch_serial_input(void)
     printk("\n");
 }
 
-static void __serial_rx(char c, struct cpu_user_regs *regs)
+static void __serial_rx(char c)
 {
     switch ( console_rx )
     {
@@ -578,7 +578,7 @@ static void __serial_rx(char c, struct c
 #endif
 }
 
-static void cf_check serial_rx(char c, struct cpu_user_regs *regs)
+static void cf_check serial_rx(char c)
 {
     static int switch_code_count = 0;
 
@@ -594,10 +594,10 @@ static void cf_check serial_rx(char c, s
     }
 
     for ( ; switch_code_count != 0; switch_code_count-- )
-        __serial_rx(switch_code, regs);
+        __serial_rx(switch_code);
 
     /* Finally process the just-received character. */
-    __serial_rx(c, regs);
+    __serial_rx(c);
 }
 
 static void cf_check notify_dom0_con_ring(void *unused)
--- a/xen/drivers/char/serial.c
+++ b/xen/drivers/char/serial.c
@@ -68,7 +68,7 @@ void serial_rx_interrupt(struct serial_p
     spin_unlock_irqrestore(&port->rx_lock, flags);
 
     if ( fn != NULL )
-        (*fn)(c & 0x7f, regs);
+        fn(c & 0x7f);
 }
 
 void serial_tx_interrupt(struct serial_port *port, struct cpu_user_regs *regs)
--- a/xen/drivers/char/xen_pv_console.c
+++ b/xen/drivers/char/xen_pv_console.c
@@ -118,7 +118,7 @@ size_t pv_console_rx(struct cpu_user_reg
     {
         c = cons_ring->in[MASK_XENCONS_IDX(cons++, cons_ring->in)];
         if ( cons_rx_handler )
-            cons_rx_handler(c, regs);
+            cons_rx_handler(c);
         recv++;
     }
 
--- a/xen/include/xen/serial.h
+++ b/xen/include/xen/serial.h
@@ -15,7 +15,7 @@
 struct cpu_user_regs;
 
 /* Register a character-receive hook on the specified COM port. */
-typedef void (*serial_rx_fn)(char c, struct cpu_user_regs *regs);
+typedef void (*serial_rx_fn)(char c);
 void serial_set_rx_handler(int handle, serial_rx_fn fn);
 
 /* Number of characters we buffer for a polling receiver. */



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:54:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:54:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683278.1062737 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLt2-0004Li-0E; Tue, 20 Feb 2024 08:54:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683278.1062737; Tue, 20 Feb 2024 08:53:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLt1-0004Lb-TK; Tue, 20 Feb 2024 08:53:59 +0000
Received: by outflank-mailman (input) for mailman id 683278;
 Tue, 20 Feb 2024 08:53:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLt0-0002dz-CL
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:53:58 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 95764024-cfcd-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:53:57 +0100 (CET)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-33d375993f4so1288961f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:53:57 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020adfcd0e000000b0033d14c96ec1sm12607957wrm.45.2024.02.20.00.53.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:53:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 95764024-cfcd-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708419237; x=1709024037; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=YO3g36PvHebDUpK9/W9a6xU16xqzRXgrwigdPTnIuIA=;
        b=MRCDNx/jzxK3ryhZdl7QLnVevdcK5YROT9z2EVNCy9KGgKmXCkICJByY4NpgtiGAog
         bU9XdOdYS7f6r0PHqsI0d4D68FKBjMf0T5u0WmP1FTD5Jj9Uh35QQ15A2H5I+pvyUcex
         0mvXpMunu3a9I7upZcjYS6TaAsLOI21N6Lvl5Vg9oTvqdTaIE2Lzbzb2u2bBEHKd92dy
         nqeRGcJbCN6MbeogPgM5Pftu6ifpx8sgTtNUoFbHy+C42pqpeuna97Hkb3HBqEZAFvPP
         AR1yN+MuPXI7gb9WqjqQ9afSykTHGYfCnxxUayRtgeV2siFadMrsJjyMLy3yRDb6Z8OD
         0YGg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419237; x=1709024037;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=YO3g36PvHebDUpK9/W9a6xU16xqzRXgrwigdPTnIuIA=;
        b=ZfNDLxGGIYfk3LxP8XsVPMsLq87MqT3SND3Cw84B4xNKB26TiL9ZInx6kZ4PJKXYkr
         0jNvnrPHOYkmDYPNSEzsEJeBGQfycpeZ3F+2MgspX86z+dmeTJaZDsCvzcTHP3G722QE
         Iii/Jc3yJIOWEp385xD7mrB8vJE5jJ8UwSR0srfRP8Z5067EWSdUxsljA983G0dFP3Dy
         OokJrovhEEOMqcEzKu7U1u0jDtcA2HUjcnpj4tRE+92SyGknssJftnw3S4CPVCq5EjS0
         21Q4+FcCTK8UZJbr3Nfs6fwvCElWCuUtc/jABZx8rwPKuCZ3BuCyyblDI9SQfne+f0Hz
         fq/A==
X-Gm-Message-State: AOJu0YzslzT7PZmAjOmp/ck7MbihhJaOBtApjnpgNk/ekEfYhO5AEH1I
	xEj0pFl7q67G0hOhjxqixmFdZPV8h+5AxUefdxVaoTesjUneTwN2gTX0Qd0KQD9Kz1ViDX8MNPY
	=
X-Google-Smtp-Source: AGHT+IFFQxNe9Z1KKGyLsocntV93gLhIIqAJrcaLAmuQLTKz+j0cxjaKOyV0sNxP743pK6/Rw7kOIw==
X-Received: by 2002:a5d:5f51:0:b0:33d:6a83:1a61 with SMTP id cm17-20020a5d5f51000000b0033d6a831a61mr1116259wrb.37.1708419237433;
        Tue, 20 Feb 2024 00:53:57 -0800 (PST)
Message-ID: <f0fde2e9-fd19-4489-b795-1115a3a8136e@suse.com>
Date: Tue, 20 Feb 2024 09:53:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v4 4/8] PV-shim: drop pv_console_rx()'s regs parameter
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

It's not needed anymore. This is in preparation of dropping the register
parameters from IRQ handler functions.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

--- a/xen/arch/x86/guest/xen/xen.c
+++ b/xen/arch/x86/guest/xen/xen.c
@@ -181,7 +181,7 @@ static void cf_check xen_evtchn_upcall(s
             port += l1 * BITS_PER_LONG;
 
             if ( pv_console && port == pv_console_evtchn() )
-                pv_console_rx(regs);
+                pv_console_rx();
             else if ( pv_shim )
                 pv_shim_inject_evtchn(port);
         }
--- a/xen/drivers/char/xen_pv_console.c
+++ b/xen/drivers/char/xen_pv_console.c
@@ -94,7 +94,7 @@ evtchn_port_t pv_console_evtchn(void)
     return cons_evtchn;
 }
 
-size_t pv_console_rx(struct cpu_user_regs *regs)
+size_t pv_console_rx(void)
 {
     char c;
     XENCONS_RING_IDX cons, prod;
--- a/xen/include/xen/pv_console.h
+++ b/xen/include/xen/pv_console.h
@@ -9,7 +9,7 @@ void pv_console_init(void);
 void pv_console_set_rx_handler(serial_rx_fn fn);
 void pv_console_init_postirq(void);
 void pv_console_puts(const char *buf, size_t nr);
-size_t pv_console_rx(struct cpu_user_regs *regs);
+size_t pv_console_rx(void);
 evtchn_port_t pv_console_evtchn(void);
 
 #else
@@ -18,7 +18,7 @@ static inline void pv_console_init(void)
 static inline void pv_console_set_rx_handler(serial_rx_fn fn) { }
 static inline void pv_console_init_postirq(void) { }
 static inline void pv_console_puts(const char *buf, size_t nr) { }
-static inline size_t pv_console_rx(struct cpu_user_regs *regs) { return 0; }
+static inline size_t pv_console_rx(void) { return 0; }
 
 #endif /* !CONFIG_XEN_GUEST */
 #endif /* __XEN_PV_CONSOLE_H__ */



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:55:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:55:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683285.1062746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLuI-0004zi-Ao; Tue, 20 Feb 2024 08:55:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683285.1062746; Tue, 20 Feb 2024 08:55:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLuI-0004zZ-7i; Tue, 20 Feb 2024 08:55:18 +0000
Received: by outflank-mailman (input) for mailman id 683285;
 Tue, 20 Feb 2024 08:55:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLuG-0004zL-Mi
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:55:16 +0000
Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com
 [2a00:1450:4864:20::330])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c3ae5476-cfcd-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:55:15 +0100 (CET)
Received: by mail-wm1-x330.google.com with SMTP id
 5b1f17b1804b1-4126f486429so2456985e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:55:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020adfcd0e000000b0033d14c96ec1sm12607957wrm.45.2024.02.20.00.55.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:55:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3ae5476-cfcd-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708419315; x=1709024115; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=rQ9ZnllqLcHW3EN3PU74ZZkd02EgCuqNow+7TRn8kGU=;
        b=SoEk0yBMF1T3cBFn1N+F07hlHweALsNYehpUVvSqOEU97WboHJALxmxT8AP9XPRqDu
         XBTStnhVMlJfw6syceXK8sJJEaYYG/cW5nbWbWZrvM2m+I+ePUMHItWxP3xYuN5MySDI
         TKGnSZPVOSRP6DJek3cByTUDPObibdX32LH2R29z14Z5oU6BVASoyZGQ9ulrWUOg251o
         5HMFEtglXftXRUtyPBRI4tuWPPDDqiTmT8TFL8GXYR/rICd956pKl32Ng7h5Lh2N67fU
         AygiUAlkc9XIYkypGp4i7v7iI7lvbadg9JG5gWS9D0u473SsSl00hDEtEtuAcWsXcr6G
         RIFg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419315; x=1709024115;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=rQ9ZnllqLcHW3EN3PU74ZZkd02EgCuqNow+7TRn8kGU=;
        b=vGbawdjNat03buWI4NIH7qf536vHir2jN3polzTNtHNyW1g9BuiZGqw2ivSglGZh+f
         JppxS4Ulxg9nqtsVdkUzi/+iM+NIvoInFvSfMuasE/GfZO8MUiBbKCLzsJXFlAREWEJJ
         SxaWIjLPmU/s4bb2RCcP+IidqvFpRQNM9Dix7ALTRKtrw5eiULuxzp60DWHyDf6Yjn33
         bOVOKrWX+Pzp6azBP9b9NxbmIZ57VPYEtnevDMpeDFwzlN5CYZKgZTP4TgX/CtBY4kb8
         VaQoDwjER8GdmsgXQePbLRK5ZHdE6I1c4hNb/BwwoIgJfXVF7EObsvXVjcK9uURB04u7
         coQw==
X-Gm-Message-State: AOJu0YyYpSz+YyOo7xoTuToVQ+cYjl7gftkwnAJ5M/gJWNfNl+ksOSdN
	yWGnYwmu27Liqm/uBF6gIN0gyNXcmr0Acmt8bAHWWKfnn5YjesZzBG+w5MrJp7Cp8+8XVW0AxF8
	=
X-Google-Smtp-Source: AGHT+IFDLesNYiLVeXV72g1fEvH0MEGr/FkIU8RrKKBp4bxbJfLh6uIlXLuofaMJTVoKc4FffLuGwg==
X-Received: by 2002:a5d:6288:0:b0:33d:6bd5:d372 with SMTP id k8-20020a5d6288000000b0033d6bd5d372mr641139wru.15.1708419314941;
        Tue, 20 Feb 2024 00:55:14 -0800 (PST)
Message-ID: <6476a662-14f2-4206-83c6-a561c6c4aee5@suse.com>
Date: Tue, 20 Feb 2024 09:55:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v4 5/8] serial: drop serial_[rt]x_interrupt()'s regs parameter
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

They're simply not needed anymore.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
v4: Re-base over changes earlier in the series.
v2: Setting of IRQ regs split off to an earlier patch.

--- a/xen/drivers/char/cadence-uart.c
+++ b/xen/drivers/char/cadence-uart.c
@@ -51,7 +51,7 @@ static void cuart_interrupt(int irq, voi
         /* ACK.  */
         if ( status & UART_SR_INTR_RTRIG )
         {
-            serial_rx_interrupt(port, regs);
+            serial_rx_interrupt(port);
             cuart_write(uart, R_UART_CISR, UART_SR_INTR_RTRIG);
         }
     } while ( status & UART_SR_INTR_RTRIG );
--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -1273,10 +1273,10 @@ static void cf_check _ehci_dbgp_poll(str
     old_regs = set_irq_regs(regs);
 
     if ( dbgp->in.chunk )
-        serial_rx_interrupt(port, regs);
+        serial_rx_interrupt(port);
 
     if ( empty )
-        serial_tx_interrupt(port, regs);
+        serial_tx_interrupt(port);
 
     set_irq_regs(old_regs);
 
--- a/xen/drivers/char/exynos4210-uart.c
+++ b/xen/drivers/char/exynos4210-uart.c
@@ -81,7 +81,7 @@ static void exynos4210_uart_interrupt(in
         if ( status & (UINTM_RXD | UINTM_ERROR) )
         {
             /* uart->regs[UINTM] |= RXD|ERROR; */
-            serial_rx_interrupt(port, regs);
+            serial_rx_interrupt(port);
             /* uart->regs[UINTM] &= ~(RXD|ERROR); */
             exynos4210_write(uart, UINTP, UINTM_RXD | UINTM_ERROR);
         }
@@ -89,7 +89,7 @@ static void exynos4210_uart_interrupt(in
         if ( status & (UINTM_TXD | UINTM_MODEM) )
         {
             /* uart->regs[UINTM] |= TXD|MODEM; */
-            serial_tx_interrupt(port, regs);
+            serial_tx_interrupt(port);
             /* uart->regs[UINTM] &= ~(TXD|MODEM); */
             exynos4210_write(uart, UINTP, UINTM_TXD | UINTM_MODEM);
         }
--- a/xen/drivers/char/imx-lpuart.c
+++ b/xen/drivers/char/imx-lpuart.c
@@ -48,10 +48,10 @@ static void imx_lpuart_interrupt(int irq
     rxcnt = imx_lpuart_read(uart, UARTWATER) >> UARTWATER_RXCNT_OFF;
 
     if ( (sts & UARTSTAT_RDRF) || (rxcnt > 0) )
-	    serial_rx_interrupt(port, regs);
+	    serial_rx_interrupt(port);
 
     if ( sts & UARTSTAT_TDRE )
-	    serial_tx_interrupt(port, regs);
+	    serial_tx_interrupt(port);
 
     imx_lpuart_write(uart, UARTSTAT, sts);
 }
--- a/xen/drivers/char/meson-uart.c
+++ b/xen/drivers/char/meson-uart.c
@@ -69,10 +69,10 @@ static void meson_uart_interrupt(int irq
     uint32_t st = readl(uart->regs + AML_UART_STATUS_REG);
 
     if ( !(st & AML_UART_RX_FIFO_EMPTY) )
-        serial_rx_interrupt(port, regs);
+        serial_rx_interrupt(port);
 
     if ( !(st & AML_UART_TX_FIFO_FULL) )
-        serial_tx_interrupt(port, regs);
+        serial_tx_interrupt(port);
 }
 
 static void __init meson_uart_init_preirq(struct serial_port *port)
--- a/xen/drivers/char/mvebu-uart.c
+++ b/xen/drivers/char/mvebu-uart.c
@@ -76,10 +76,10 @@ static void mvebu3700_uart_interrupt(int
 
     if ( st & (STATUS_RX_RDY | STATUS_OVR_ERR | STATUS_FRM_ERR |
                STATUS_BRK_DET) )
-        serial_rx_interrupt(port, regs);
+        serial_rx_interrupt(port);
 
     if ( st & STATUS_TX_RDY )
-        serial_tx_interrupt(port, regs);
+        serial_tx_interrupt(port);
 }
 
 static void __init mvebu3700_uart_init_preirq(struct serial_port *port)
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -188,9 +188,9 @@ static void cf_check ns16550_interrupt(
         u8 lsr = ns_read_reg(uart, UART_LSR);
 
         if ( (lsr & uart->lsr_mask) == uart->lsr_mask )
-            serial_tx_interrupt(port, regs);
+            serial_tx_interrupt(port);
         if ( lsr & UART_LSR_DR )
-            serial_rx_interrupt(port, regs);
+            serial_rx_interrupt(port);
 
         /* A "busy-detect" condition is observed on Allwinner/sunxi UART
          * after LCR is written during setup. It needs to be cleared at
@@ -224,11 +224,11 @@ static void cf_check __ns16550_poll(stru
         if ( ns16550_ioport_invalid(uart) )
             goto out;
 
-        serial_rx_interrupt(port, regs);
+        serial_rx_interrupt(port);
     }
 
     if ( ( ns_read_reg(uart, UART_LSR) & uart->lsr_mask ) == uart->lsr_mask )
-        serial_tx_interrupt(port, regs);
+        serial_tx_interrupt(port);
 
 out:
     set_irq_regs(old_regs);
--- a/xen/drivers/char/omap-uart.c
+++ b/xen/drivers/char/omap-uart.c
@@ -70,9 +70,9 @@ static void omap_uart_interrupt(int irq,
     {
         lsr = omap_read(uart, UART_LSR) & 0xff;
 	if ( lsr & UART_LSR_THRE )
-            serial_tx_interrupt(port, regs);
+            serial_tx_interrupt(port);
 	if ( lsr & UART_LSR_DR )
-            serial_rx_interrupt(port, regs);
+            serial_rx_interrupt(port);
 
         if ( port->txbufc == port->txbufp ) {
             reg = omap_read(uart, UART_IER);
--- a/xen/drivers/char/pl011.c
+++ b/xen/drivers/char/pl011.c
@@ -95,7 +95,7 @@ static void pl011_interrupt(int irq, voi
             pl011_write(uart, ICR, status & ~(TXI|RTI|RXI));
 
             if ( status & (RTI|RXI) )
-                serial_rx_interrupt(port, regs);
+                serial_rx_interrupt(port);
 
             /* TODO
                 if ( status & (DSRMI|DCDMI|CTSMI|RIMI) )
@@ -103,7 +103,7 @@ static void pl011_interrupt(int irq, voi
             */
 
             if ( status & (TXI) )
-                serial_tx_interrupt(port, regs);
+                serial_tx_interrupt(port);
 
             status = pl011_intr_status(uart);
         } while (status != 0);
--- a/xen/drivers/char/scif-uart.c
+++ b/xen/drivers/char/scif-uart.c
@@ -119,11 +119,11 @@ static void scif_uart_interrupt(int irq,
     {
         /* TX Interrupt */
         if ( status & SCFSR_TDFE )
-            serial_tx_interrupt(port, regs);
+            serial_tx_interrupt(port);
 
         /* RX Interrupt */
         if ( status & (SCFSR_RDF | SCFSR_DR) )
-            serial_rx_interrupt(port, regs);
+            serial_rx_interrupt(port);
 
         /* Error Interrupt */
         if ( status & params->error_mask )
--- a/xen/drivers/char/serial.c
+++ b/xen/drivers/char/serial.c
@@ -45,7 +45,7 @@ static inline void serial_stop_tx(struct
         port->driver->stop_tx(port);
 }
 
-void serial_rx_interrupt(struct serial_port *port, struct cpu_user_regs *regs)
+void serial_rx_interrupt(struct serial_port *port)
 {
     char c;
     serial_rx_fn fn = NULL;
@@ -71,7 +71,7 @@ void serial_rx_interrupt(struct serial_p
         fn(c & 0x7f);
 }
 
-void serial_tx_interrupt(struct serial_port *port, struct cpu_user_regs *regs)
+void serial_tx_interrupt(struct serial_port *port)
 {
     int i, n;
     unsigned long flags;
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -1176,9 +1176,10 @@ static void cf_check dbc_uart_poll(void
     }
 
     while ( dbc_work_ring_size(&dbc->dbc_iwork) )
-        serial_rx_interrupt(port, guest_cpu_user_regs());
+        serial_rx_interrupt(port);
+
+    serial_tx_interrupt(port);
 
-    serial_tx_interrupt(port, guest_cpu_user_regs());
     set_timer(&uart->timer, NOW() + MICROSECS(DBC_POLL_INTERVAL));
 }
 
--- a/xen/include/xen/serial.h
+++ b/xen/include/xen/serial.h
@@ -12,8 +12,6 @@
 #include <xen/init.h>
 #include <xen/spinlock.h>
 
-struct cpu_user_regs;
-
 /* Register a character-receive hook on the specified COM port. */
 typedef void (*serial_rx_fn)(char c);
 void serial_set_rx_handler(int handle, serial_rx_fn fn);
@@ -145,8 +143,8 @@ void serial_register_uart(int idx, struc
 /* Place the serial port into asynchronous transmit mode. */
 void serial_async_transmit(struct serial_port *port);
 /* Process work in interrupt context. */
-void serial_rx_interrupt(struct serial_port *port, struct cpu_user_regs *regs);
-void serial_tx_interrupt(struct serial_port *port, struct cpu_user_regs *regs);
+void serial_rx_interrupt(struct serial_port *port);
+void serial_tx_interrupt(struct serial_port *port);
 
 /*
  * Initialisers for individual uart drivers.



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:56:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:56:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683290.1062757 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLuy-0005V0-IH; Tue, 20 Feb 2024 08:56:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683290.1062757; Tue, 20 Feb 2024 08:56:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLuy-0005Ut-Fe; Tue, 20 Feb 2024 08:56:00 +0000
Received: by outflank-mailman (input) for mailman id 683290;
 Tue, 20 Feb 2024 08:55:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLuw-0005Lw-Fu
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:55:58 +0000
Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com
 [2a00:1450:4864:20::436])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dc286481-cfcd-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:55:56 +0100 (CET)
Received: by mail-wr1-x436.google.com with SMTP id
 ffacd0b85a97d-33d26da3e15so1862341f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:55:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020adfcd0e000000b0033d14c96ec1sm12607957wrm.45.2024.02.20.00.55.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:55:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc286481-cfcd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708419356; x=1709024156; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=vY7h+x/uY/tx4SThVUK3+EyH+28fbreju2/f4B+vk7s=;
        b=TgP4aljbe6DfV4p0ep0ppvmPhxn1dZI8yfesFfFAuzJY7l58BNSXm8YsPp4onKU9xH
         OYGtr05GquwA3P4XMq5a2Rt0VGBroX2/Wttx4lQ/EluiqH4pYRgg8ktuhEWcCT9pgM3c
         6h7ZSAEeaTzRIlxDUZ1KgqHsbSBgACz+s1gKbhXNc+U+6Hb6O+KOyz+z9lEdkDPUg1Z/
         reuDmfYHWJ2Wz9pEB5kXL+vaONiIBxfDMKaNmaLSrpUXBQSZhDtkGqpUf9hRydoagcW5
         CWqhgIjZS0b8sqz25tzUMXZTv3V2/OUPR9MRzmJZhijDw/kdfzW5JgFfagqDX0zDZ09s
         eakw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419356; x=1709024156;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=vY7h+x/uY/tx4SThVUK3+EyH+28fbreju2/f4B+vk7s=;
        b=cbYcWVoCFohYSIEnymwoINrssjSa3ZO9ROWCLdEfo1yEF8TIHzpxU1Rm8uwuU/gxTf
         xUsvX//yumZhSuN3EoQGtno5VSxnMyKsR8hfJAuiKJZ+x9Q7GhgqHifKKDhdrmq/T0Wp
         gwzVoiMpqpdBQGHfIS/ycJbPApOmD+xMUfLqAkg9M1A0bRL8bbozdlqu1YAajGeCinLp
         58cNT1SSmJQPNQoxB8p7jjbZ1Ul0RVUElkg1TZiWNalLzgIGKI1iJhXlMS3e1tEIj9OS
         HvPTKw0B9L8ALMwc/rsGyyoSdHQu+lqX2XNuiEYpx0tKigDpHTR9NzOfAMBqq6OS9hTc
         rgsA==
X-Gm-Message-State: AOJu0YxiH3HccmOk/ZG+Jj4FswptzZESgtjJwngVHGDEsWUd8gyyclCR
	Lcd6atobzxLXIKIW8xCJzeeBBpq1CViZCUmlGI4Q3AlIVuBWqBhsRmTSH7X4bwQH3AFxpchMZEk
	=
X-Google-Smtp-Source: AGHT+IGHn4367SFgbvnMktpZZ1dQ1rWBIxFaDsWpXzI8tbcOjs3bL2NkuZwGgf/P4Aurp9SWp8nsLA==
X-Received: by 2002:a5d:5345:0:b0:33d:1bd1:8ae2 with SMTP id t5-20020a5d5345000000b0033d1bd18ae2mr9444079wrv.19.1708419355920;
        Tue, 20 Feb 2024 00:55:55 -0800 (PST)
Message-ID: <74a35cb6-9211-46ae-85cf-c83ad49d0786@suse.com>
Date: Tue, 20 Feb 2024 09:55:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v4 6/8] IRQ: drop regs parameter from handler functions
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

It's simply not needed anymore. Note how Linux made this change many
years ago already, in 2.6.19 (late 2006, see [1]).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Julien Grall <jgrall@amazon.com>

[1] https://git.kernel.org/torvalds/c/7d12e780e003f93433d49ce78cfedf4b4c52adc5
---
v2: Arm build fixes.

--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -397,7 +397,7 @@ void gic_interrupt(struct cpu_user_regs
     } while (1);
 }
 
-static void maintenance_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
+static void maintenance_interrupt(int irq, void *dev_id)
 {
     /*
      * This is a dummy interrupt handler.
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -182,8 +182,7 @@ void irq_set_affinity(struct irq_desc *d
 }
 
 int request_irq(unsigned int irq, unsigned int irqflags,
-                void (*handler)(int irq, void *dev_id,
-                                struct cpu_user_regs *regs),
+                void (*handler)(int irq, void *dev_id),
                 const char *devname, void *dev_id)
 {
     struct irqaction *action;
@@ -276,7 +275,7 @@ void do_IRQ(struct cpu_user_regs *regs,
 
     do
     {
-        action->handler(irq, action->dev_id, regs);
+        action->handler(irq, action->dev_id);
         action = action->next;
     } while ( action );
 
--- a/xen/arch/arm/time.c
+++ b/xen/arch/arm/time.c
@@ -241,7 +241,7 @@ int reprogram_timer(s_time_t timeout)
 }
 
 /* Handle the firing timer */
-static void htimer_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
+static void htimer_interrupt(int irq, void *dev_id)
 {
     if ( unlikely(!(READ_SYSREG(CNTHP_CTL_EL2) & CNTx_CTL_PENDING)) )
         return;
@@ -255,7 +255,7 @@ static void htimer_interrupt(int irq, vo
     WRITE_SYSREG(0, CNTHP_CTL_EL2);
 }
 
-static void vtimer_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs)
+static void vtimer_interrupt(int irq, void *dev_id)
 {
     /*
      * Edge-triggered interrupts can be used for the virtual timer. Even
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -962,7 +962,7 @@ static int __init cf_check irq_ratelimit
 __initcall(irq_ratelimit_init);
 
 int __init request_irq(unsigned int irq, unsigned int irqflags,
-        void (*handler)(int irq, void *dev_id, struct cpu_user_regs *regs),
+        void (*handler)(int irq, void *dev_id),
         const char * devname, void *dev_id)
 {
     struct irqaction * action;
@@ -2009,7 +2009,7 @@ void do_IRQ(struct cpu_user_regs *regs)
         spin_unlock_irq(&desc->lock);
 
         tsc_in = tb_init_done ? get_cycles() : 0;
-        action->handler(irq, action->dev_id, regs);
+        action->handler(irq, action->dev_id);
         TRACE_3D(TRC_HW_IRQ_HANDLED, irq, tsc_in, get_cycles());
 
         spin_lock_irq(&desc->lock);
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -237,8 +237,7 @@ again:
     }
 }
 
-static void cf_check hpet_interrupt_handler(
-    int irq, void *data, struct cpu_user_regs *regs)
+static void cf_check hpet_interrupt_handler(int irq, void *data)
 {
     struct hpet_event_channel *ch = data;
 
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -198,8 +198,7 @@ static void smp_send_timer_broadcast_ipi
     }
 }
 
-static void cf_check timer_interrupt(
-    int irq, void *dev_id, struct cpu_user_regs *regs)
+static void cf_check timer_interrupt(int irq, void *dev_id)
 {
     ASSERT(local_irq_is_enabled());
 
--- a/xen/common/irq.c
+++ b/xen/common/irq.c
@@ -29,7 +29,7 @@ int init_one_irq_desc(struct irq_desc *d
     return err;
 }
 
-void cf_check no_action(int cpl, void *dev_id, struct cpu_user_regs *regs)
+void cf_check no_action(int cpl, void *dev_id)
 {
 }
 
--- a/xen/drivers/char/cadence-uart.c
+++ b/xen/drivers/char/cadence-uart.c
@@ -40,7 +40,7 @@ static struct cuart {
 #define cuart_read(uart, off)           readl((uart)->regs + (off))
 #define cuart_write(uart, off,val)      writel((val), (uart)->regs + (off))
 
-static void cuart_interrupt(int irq, void *data, struct cpu_user_regs *regs)
+static void cuart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct cuart *uart = port->uart;
--- a/xen/drivers/char/exynos4210-uart.c
+++ b/xen/drivers/char/exynos4210-uart.c
@@ -45,7 +45,7 @@ static struct exynos4210_uart {
 #define exynos4210_read(uart, off)          readl((uart)->regs + off)
 #define exynos4210_write(uart, off, val)    writel(val, (uart->regs) + off)
 
-static void exynos4210_uart_interrupt(int irq, void *data, struct cpu_user_regs *regs)
+static void exynos4210_uart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct exynos4210_uart *uart = port->uart;
--- a/xen/drivers/char/imx-lpuart.c
+++ b/xen/drivers/char/imx-lpuart.c
@@ -37,8 +37,7 @@ static struct imx_lpuart {
     struct vuart_info vuart;
 } imx8_com;
 
-static void imx_lpuart_interrupt(int irq, void *data,
-                                 struct cpu_user_regs *regs)
+static void imx_lpuart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct imx_lpuart *uart = port->uart;
--- a/xen/drivers/char/meson-uart.c
+++ b/xen/drivers/char/meson-uart.c
@@ -61,8 +61,7 @@ static struct meson_uart {
     struct vuart_info vuart;
 } meson_com;
 
-static void meson_uart_interrupt(int irq, void *data,
-                                 struct cpu_user_regs *regs)
+static void meson_uart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct meson_uart *uart = port->uart;
--- a/xen/drivers/char/mvebu-uart.c
+++ b/xen/drivers/char/mvebu-uart.c
@@ -67,8 +67,7 @@ static struct mvebu3700_uart {
 #define mvebu3700_read(uart, off)           readl((uart)->regs + (off))
 #define mvebu3700_write(uart, off, val)     writel(val, (uart)->regs + (off))
 
-static void mvebu3700_uart_interrupt(int irq, void *data,
-                                     struct cpu_user_regs *regs)
+static void mvebu3700_uart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct mvebu3700_uart *uart = port->uart;
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -175,8 +175,7 @@ static void handle_dw_usr_busy_quirk(str
     }
 }
 
-static void cf_check ns16550_interrupt(
-    int irq, void *dev_id, struct cpu_user_regs *regs)
+static void cf_check ns16550_interrupt(int irq, void *dev_id)
 {
     struct serial_port *port = dev_id;
     struct ns16550 *uart = port->uart;
--- a/xen/drivers/char/omap-uart.c
+++ b/xen/drivers/char/omap-uart.c
@@ -59,7 +59,7 @@ static struct omap_uart {
     struct vuart_info vuart;
 } omap_com = {0};
 
-static void omap_uart_interrupt(int irq, void *data, struct cpu_user_regs *regs)
+static void omap_uart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct omap_uart *uart = port->uart;
--- a/xen/drivers/char/pl011.c
+++ b/xen/drivers/char/pl011.c
@@ -82,7 +82,7 @@ static unsigned int pl011_intr_status(st
     return (pl011_read(uart, RIS) & pl011_read(uart, IMSC));
 }
 
-static void pl011_interrupt(int irq, void *data, struct cpu_user_regs *regs)
+static void pl011_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct pl011 *uart = port->uart;
--- a/xen/drivers/char/scif-uart.c
+++ b/xen/drivers/char/scif-uart.c
@@ -102,7 +102,7 @@ static const struct port_params port_par
     },
 };
 
-static void scif_uart_interrupt(int irq, void *data, struct cpu_user_regs *regs)
+static void scif_uart_interrupt(int irq, void *data)
 {
     struct serial_port *port = data;
     struct scif_uart *uart = port->uart;
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -718,8 +718,7 @@ static void cf_check do_amd_iommu_irq(vo
 
 static DECLARE_SOFTIRQ_TASKLET(amd_iommu_irq_tasklet, do_amd_iommu_irq, NULL);
 
-static void cf_check iommu_interrupt_handler(
-    int irq, void *dev_id, struct cpu_user_regs *regs)
+static void cf_check iommu_interrupt_handler(int irq, void *dev_id)
 {
     unsigned long flags;
     struct amd_iommu *iommu = dev_id;
--- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c
+++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
@@ -642,7 +642,7 @@ static void ipmmu_domain_irq(struct ipmm
                         domain->d, status, iova);
 }
 
-static void ipmmu_irq(int irq, void *dev, struct cpu_user_regs *regs)
+static void ipmmu_irq(int irq, void *dev)
 {
     struct ipmmu_vmsa_device *mmu = dev;
     unsigned int i;
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -1097,16 +1097,14 @@ static irqreturn_t arm_smmu_global_fault
 }
 
 /* Xen: Interrupt handlers wrapper */
-static void arm_smmu_context_fault_xen(int irq, void *dev,
-				       struct cpu_user_regs *regs)
+static void arm_smmu_context_fault_xen(int irq, void *dev)
 {
 	arm_smmu_context_fault(irq, dev);
 }
 
 #define arm_smmu_context_fault arm_smmu_context_fault_xen
 
-static void arm_smmu_global_fault_xen(int irq, void *dev,
-				      struct cpu_user_regs *regs)
+static void arm_smmu_global_fault_xen(int irq, void *dev)
 {
 	arm_smmu_global_fault(irq, dev);
 }
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -880,8 +880,7 @@ static void arm_smmu_priq_tasklet(void *
 
 static int arm_smmu_device_disable(struct arm_smmu_device *smmu);
 
-static void arm_smmu_gerror_handler(int irq, void *dev,
-				struct cpu_user_regs *regs)
+static void arm_smmu_gerror_handler(int irq, void *dev)
 {
 	u32 gerror, gerrorn, active;
 	struct arm_smmu_device *smmu = dev;
@@ -926,12 +925,11 @@ static void arm_smmu_gerror_handler(int
 	writel(gerror, smmu->base + ARM_SMMU_GERRORN);
 }
 
-static void arm_smmu_combined_irq_handler(int irq, void *dev,
-				struct cpu_user_regs *regs)
+static void arm_smmu_combined_irq_handler(int irq, void *dev)
 {
 	struct arm_smmu_device *smmu = dev;
 
-	arm_smmu_gerror_handler(irq, dev, regs);
+	arm_smmu_gerror_handler(irq, dev);
 
 	tasklet_schedule(&(smmu->combined_irq_tasklet));
 }
@@ -945,16 +943,14 @@ static void arm_smmu_combined_irq_taskle
 		arm_smmu_priq_tasklet(dev);
 }
 
-static void arm_smmu_evtq_irq_tasklet(int irq, void *dev,
-				struct cpu_user_regs *regs)
+static void arm_smmu_evtq_irq_tasklet(int irq, void *dev)
 {
 	struct arm_smmu_device *smmu = dev;
 
 	tasklet_schedule(&(smmu->evtq_irq_tasklet));
 }
 
-static void arm_smmu_priq_irq_tasklet(int irq, void *dev,
-				struct cpu_user_regs *regs)
+static void arm_smmu_priq_irq_tasklet(int irq, void *dev)
 {
 	struct arm_smmu_device *smmu = dev;
 
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1117,8 +1117,7 @@ static void cf_check do_iommu_page_fault
         __do_iommu_page_fault(drhd->iommu);
 }
 
-static void cf_check iommu_page_fault(
-    int irq, void *dev_id, struct cpu_user_regs *regs)
+static void cf_check iommu_page_fault(int irq, void *dev_id)
 {
     /*
      * Just flag the tasklet as runnable. This is fine, according to VT-d
--- a/xen/include/xen/irq.h
+++ b/xen/include/xen/irq.h
@@ -18,7 +18,7 @@
     ASSERT(!in_irq() && (local_irq_is_enabled() || num_online_cpus() <= 1))
 
 struct irqaction {
-    void (*handler)(int irq, void *dev_id, struct cpu_user_regs *regs);
+    void (*handler)(int irq, void *dev_id);
     const char *name;
     void *dev_id;
     bool free_on_release;
@@ -119,12 +119,11 @@ extern int setup_irq(unsigned int irq, u
                      struct irqaction *new);
 extern void release_irq(unsigned int irq, const void *dev_id);
 extern int request_irq(unsigned int irq, unsigned int irqflags,
-               void (*handler)(int irq, void *dev_id,
-                     struct cpu_user_regs *regs),
+               void (*handler)(int irq, void *dev_id),
                const char *devname, void *dev_id);
 
 extern hw_irq_controller no_irq_type;
-void cf_check no_action(int cpl, void *dev_id, struct cpu_user_regs *regs);
+void cf_check no_action(int cpl, void *dev_id);
 unsigned int cf_check irq_startup_none(struct irq_desc *desc);
 void cf_check irq_actor_none(struct irq_desc *desc);
 #define irq_shutdown_none irq_actor_none



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:56:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:56:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683305.1062766 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLvR-00064Z-Ug; Tue, 20 Feb 2024 08:56:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683305.1062766; Tue, 20 Feb 2024 08:56:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLvR-00064S-Rx; Tue, 20 Feb 2024 08:56:29 +0000
Received: by outflank-mailman (input) for mailman id 683305;
 Tue, 20 Feb 2024 08:56:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLvR-0005pz-97
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:56:29 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ef22c0a5-cfcd-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 09:56:28 +0100 (CET)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2d247c31e1aso5044261fa.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:56:28 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020adfcd0e000000b0033d14c96ec1sm12607957wrm.45.2024.02.20.00.56.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:56:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ef22c0a5-cfcd-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708419388; x=1709024188; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=UDhUrA6tM7xm4y+MMtOUTP3hYExqNaG8GAdW+v0Ujq4=;
        b=bqbyqfPzkmlEAMEMfFq7iuZMpyQVhgIkQz2bT2TbguG9hCjXBC/BIrXFrUCYLiMYjZ
         KQE0MxieGFyVxZVianpSNDqhA7KFK98d7OUcH7t5eEIC+tnxL0MRccthFQcwedDyHNz7
         vW5fG/QTRIFTgnjU4XdrXrdH8TLjNHCODFLrH/ITyg1ECk7fMdE3lbaCsCdg7kFWzMfD
         c+Q+2I3UZ4Z9zgvc9x5SfROb0weIQ/PB63yLziHZ4iD60FJT0hw6dnbDdMt4c4M5ed3m
         eltGbuhVGdBekDipxih3dyWzgb1l2mXJ4nSwY9Y4xr9520jo7x2MYXqIF/Y/r1oRkLpK
         PFeQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419388; x=1709024188;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=UDhUrA6tM7xm4y+MMtOUTP3hYExqNaG8GAdW+v0Ujq4=;
        b=oWAlPuU8ujHHBImbaCzUlTXYJ2vm+GS0uYI4DH9kTab+UZ/eBD430kE6Ibr9WC6FVD
         0fT9ygmvEwy6O7XUKMlmyncFyvolYgJmK7NI2zO4ISxF38BWZovJ5eXZ66X6vOjMbBjH
         65V3kE3VQtX6tQ7xqVjvvmSArnOj6fe4er2oE3J2VRGn95bhDZhSrM9x4nqmyr2t6Eju
         1mCY8u+q7nPAxdROsd+2oZbDjJVZ2MHhNf9eEFhK7kD39jFgDtDIaPw7mGOjYkm4jTNe
         NpQQ9FK5KDbLJSXJxkObGpr9aiTpJYnKvlfHGMfzPgrsf1gGihKKNEUrddnPtRD+F3OJ
         3UfA==
X-Gm-Message-State: AOJu0Yxm/OquBhVDCRLVV8olqcnEAV650vITpQypFJJ1GEJxJq2DD8/D
	+5UNmMpLZVmRHcXBu23s8aM/pmRCxvYb7byt42lDaea9u1YRqSRmtHZwinCUwdG0SAE+xtPrYfE
	=
X-Google-Smtp-Source: AGHT+IHuJs0sbdMFsDfsnlWZQ85DshqCj5umcoFkvdkaMWTKP5kDN7x4qkZSjrcF5Eipjhvdn4z/0w==
X-Received: by 2002:a05:651c:118d:b0:2d2:34d5:63f5 with SMTP id w13-20020a05651c118d00b002d234d563f5mr3365975ljo.52.1708419387794;
        Tue, 20 Feb 2024 00:56:27 -0800 (PST)
Message-ID: <8c9abefd-2871-42e6-b857-6a450f5b833a@suse.com>
Date: Tue, 20 Feb 2024 09:56:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v4 7/8] x86/APIC: drop regs parameter from direct vector
 handler functions
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

The only place it was needed is in the spurious handler, and there we
can use get_irq_regs() instead.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -1322,7 +1322,7 @@ int reprogram_timer(s_time_t timeout)
     return apic_tmict || !timeout;
 }
 
-static void cf_check apic_timer_interrupt(struct cpu_user_regs *regs)
+static void cf_check apic_timer_interrupt(void)
 {
     ack_APIC_irq();
     perfc_incr(apic_timer);
@@ -1341,7 +1341,7 @@ void smp_send_state_dump(unsigned int cp
 /*
  * Spurious interrupts should _never_ happen with our APIC/SMP architecture.
  */
-static void cf_check spurious_interrupt(struct cpu_user_regs *regs)
+static void cf_check spurious_interrupt(void)
 {
     /*
      * Check if this is a vectored interrupt (most likely, as this is probably
@@ -1355,7 +1355,7 @@ static void cf_check spurious_interrupt(
         is_spurious = !nmi_check_continuation();
         if (this_cpu(state_dump_pending)) {
             this_cpu(state_dump_pending) = false;
-            dump_execstate(regs);
+            dump_execstate(get_irq_regs());
             is_spurious = false;
         }
 
@@ -1372,7 +1372,7 @@ static void cf_check spurious_interrupt(
  * This interrupt should never happen with our APIC/SMP architecture
  */
 
-static void cf_check error_interrupt(struct cpu_user_regs *regs)
+static void cf_check error_interrupt(void)
 {
     static const char *const esr_fields[] = {
         ", Send CS error",
@@ -1407,7 +1407,7 @@ static void cf_check error_interrupt(str
  * This interrupt handles performance counters interrupt
  */
 
-static void cf_check pmu_interrupt(struct cpu_user_regs *regs)
+static void cf_check pmu_interrupt(void)
 {
     ack_APIC_irq();
     vpmu_do_interrupt();
--- a/xen/arch/x86/cpu/mcheck/mce_intel.c
+++ b/xen/arch/x86/cpu/mcheck/mce_intel.c
@@ -58,7 +58,7 @@ bool __read_mostly lmce_support;
 #define MCE_RING                0x1
 static DEFINE_PER_CPU(int, last_state);
 
-static void cf_check intel_thermal_interrupt(struct cpu_user_regs *regs)
+static void cf_check intel_thermal_interrupt(void)
 {
     uint64_t msr_content;
     unsigned int cpu = smp_processor_id();
@@ -642,7 +642,7 @@ static void cpu_mcheck_disable(void)
         clear_cmci();
 }
 
-static void cf_check cmci_interrupt(struct cpu_user_regs *regs)
+static void cf_check cmci_interrupt(void)
 {
     mctelem_cookie_t mctc;
     struct mca_summary bs;
--- a/xen/arch/x86/guest/xen/xen.c
+++ b/xen/arch/x86/guest/xen/xen.c
@@ -158,7 +158,7 @@ static void __init init_memmap(void)
     }
 }
 
-static void cf_check xen_evtchn_upcall(struct cpu_user_regs *regs)
+static void cf_check xen_evtchn_upcall(void)
 {
     struct vcpu_info *vcpu_info = this_cpu(vcpu_info);
     unsigned long pending;
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -2782,7 +2782,7 @@ static struct hvm_function_table __initd
 };
 
 /* Handle VT-d posted-interrupt when VCPU is blocked. */
-static void cf_check pi_wakeup_interrupt(struct cpu_user_regs *regs)
+static void cf_check pi_wakeup_interrupt(void)
 {
     struct vmx_vcpu *vmx, *tmp;
     spinlock_t *lock = &per_cpu(vmx_pi_blocking, smp_processor_id()).lock;
@@ -2814,7 +2814,7 @@ static void cf_check pi_wakeup_interrupt
 }
 
 /* Handle VT-d posted-interrupt when VCPU is running. */
-static void cf_check pi_notification_interrupt(struct cpu_user_regs *regs)
+static void cf_check pi_notification_interrupt(void)
 {
     ack_APIC_irq();
     this_cpu(irq_count)++;
--- a/xen/arch/x86/include/asm/irq.h
+++ b/xen/arch/x86/include/asm/irq.h
@@ -72,17 +72,15 @@ extern int opt_irq_vector_map;
 
 #define platform_legacy_irq(irq)	((irq) < 16)
 
-void cf_check event_check_interrupt(struct cpu_user_regs *regs);
-void cf_check invalidate_interrupt(struct cpu_user_regs *regs);
-void cf_check call_function_interrupt(struct cpu_user_regs *regs);
-void cf_check irq_move_cleanup_interrupt(struct cpu_user_regs *regs);
+void cf_check event_check_interrupt(void);
+void cf_check invalidate_interrupt(void);
+void cf_check call_function_interrupt(void);
+void cf_check irq_move_cleanup_interrupt(void);
 
 uint8_t alloc_hipriority_vector(void);
 
-void set_direct_apic_vector(
-    uint8_t vector, void (*handler)(struct cpu_user_regs *regs));
-void alloc_direct_apic_vector(
-    uint8_t *vector, void (*handler)(struct cpu_user_regs *regs));
+void set_direct_apic_vector(uint8_t vector, void (*handler)(void));
+void alloc_direct_apic_vector(uint8_t *vector, void (*handler)(void));
 
 void do_IRQ(struct cpu_user_regs *regs);
 
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -743,7 +743,7 @@ void move_native_irq(struct irq_desc *de
     desc->handler->enable(desc);
 }
 
-void cf_check irq_move_cleanup_interrupt(struct cpu_user_regs *regs)
+void cf_check irq_move_cleanup_interrupt(void)
 {
     unsigned vector, me;
 
@@ -913,16 +913,14 @@ uint8_t alloc_hipriority_vector(void)
     return next++;
 }
 
-static void (*direct_apic_vector[X86_NR_VECTORS])(struct cpu_user_regs *regs);
-void set_direct_apic_vector(
-    uint8_t vector, void (*handler)(struct cpu_user_regs *regs))
+static void (*direct_apic_vector[X86_NR_VECTORS])(void);
+void set_direct_apic_vector(uint8_t vector, void (*handler)(void))
 {
     BUG_ON(direct_apic_vector[vector] != NULL);
     direct_apic_vector[vector] = handler;
 }
 
-void alloc_direct_apic_vector(
-    uint8_t *vector, void (*handler)(struct cpu_user_regs *regs))
+void alloc_direct_apic_vector(uint8_t *vector, void (*handler)(void))
 {
     static DEFINE_SPINLOCK(lock);
 
@@ -1907,7 +1905,7 @@ void do_IRQ(struct cpu_user_regs *regs)
     if ( irq < 0 )
     {
         if ( direct_apic_vector[vector] )
-            direct_apic_vector[vector](regs);
+            direct_apic_vector[vector]();
         else
         {
             const char *kind = ", LAPIC";
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -245,7 +245,7 @@ static cpumask_t flush_cpumask;
 static const void *flush_va;
 static unsigned int flush_flags;
 
-void cf_check invalidate_interrupt(struct cpu_user_regs *regs)
+void cf_check invalidate_interrupt(void)
 {
     unsigned int flags = flush_flags;
     ack_APIC_irq();
@@ -387,14 +387,14 @@ void smp_send_nmi_allbutself(void)
     send_IPI_mask(&cpu_online_map, APIC_DM_NMI);
 }
 
-void cf_check event_check_interrupt(struct cpu_user_regs *regs)
+void cf_check event_check_interrupt(void)
 {
     ack_APIC_irq();
     perfc_incr(ipis);
     this_cpu(irq_count)++;
 }
 
-void cf_check call_function_interrupt(struct cpu_user_regs *regs)
+void cf_check call_function_interrupt(void)
 {
     ack_APIC_irq();
     perfc_incr(ipis);



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:57:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:57:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683310.1062777 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLwJ-0006cL-6j; Tue, 20 Feb 2024 08:57:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683310.1062777; Tue, 20 Feb 2024 08:57:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLwJ-0006cC-3x; Tue, 20 Feb 2024 08:57:23 +0000
Received: by outflank-mailman (input) for mailman id 683310;
 Tue, 20 Feb 2024 08:57:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLwI-0006c0-Hh
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:57:22 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0e1759d4-cfce-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:57:20 +0100 (CET)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-33aeb088324so2737124f8f.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:57:20 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020adfcd0e000000b0033d14c96ec1sm12607957wrm.45.2024.02.20.00.57.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:57:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e1759d4-cfce-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708419440; x=1709024240; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=jWqlogxB928KUsvmRqerAeKEQfMENuclb4BguYyVDTk=;
        b=SF4ohpnW9D7Fm8wjP4cqrFRtqujcxVdbqmbJMquRcWyW5AWDFuV7UQuDdLITV/X3gM
         WaKTpjZqkZTbK0VqmXIniVFYzTk0Gto5X5pPk21WaDEH3fUdlIU2U2J3xqDYwL1WQRLS
         8xts3rTU4P4pszUvofiFJGnwrIArfiDD0SRQ5+Uq/YF42I7d225xM0hya+CyA+c41ciu
         01vLMEvCVIcZFEe0Tp8FH90xeg9JbOaiXx3wdd/ddbV81sen35miUNfhgN4C69ytGN1w
         921LKuQPrPo801kxHxMhETLeZ/pKDQiJ40Q/JAuDE2jOM0oZu/SpyOckVZ/YgEtiA3kd
         0zKw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419440; x=1709024240;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=jWqlogxB928KUsvmRqerAeKEQfMENuclb4BguYyVDTk=;
        b=uBavmlinFU38OUmEH3VxxHOTG9mVMwPWhTRTCh/IKD9zmWuVcIOGXARl2o8evvvMRu
         SSzx7zq14qvhtbiUneIoy1oZX+YFvKLqF06RzVeAe+eyh0HdvXo8hgBLYl+co75xDG1j
         6oT9HJWhWb/REW73fz5gmLTN7hnsLTJGQ0Hpv0w8mUoWSce048gXXOMtPs1koTWMzCNy
         lzHKgJDl3C8O0jCY/pWhFAdQRvqYCmATCOur089p9ejbMywAzHEASgghu8l3bT7EPRpm
         Yr13KnLbwenYk5hN8eBHTnmfAA4aLvI815clDEqlKXQdLSDs5ieP3Jtq6JTWj9C700Hc
         B9Wg==
X-Gm-Message-State: AOJu0Yw07t83kOI6Pl7MNLrb4uDTVgcuNh1cIkkrH4q72GrBmQPGTjpk
	3umwezMEe3wpFlWzGzcewFMoSnLjsL+0zAiRpWmc3Zow6iVRTcTVKytUyL7CHqoGTPygpq/8lgI
	=
X-Google-Smtp-Source: AGHT+IGimlpP3q+zJai3qTgVg7wBX5QVYnZUsKu23U7JIy9kYANwGKOm8T5rfmhwGcF89Jp2y2Ei3A==
X-Received: by 2002:a5d:47ac:0:b0:33d:38cf:aa73 with SMTP id 12-20020a5d47ac000000b0033d38cfaa73mr5898159wrb.47.1708419439684;
        Tue, 20 Feb 2024 00:57:19 -0800 (PST)
Message-ID: <aab24f1f-ed29-4098-b430-152c16965e90@suse.com>
Date: Tue, 20 Feb 2024 09:57:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v4 8/8] consolidate do_bug_frame() / bug_fn_t
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

The type not being used in do_bug_frame() is suspicious. Apparently
that's solely because the type uses a pointer-to-const parameter,
when so far run_in_exception_handler() wanted functions taking pointer-
to-non-const. Expand use of const, in turn requiring common code's
do_bug_frame() as well as [gs]et_irq_regs() to also gain const. This
then brings the former function also closer to the common one, with
Arm's use of vaddr_t remaining as a difference.

While there also replace the bogus use of hard tabs in [gs]et_irq_regs()
(I clearly didn't mean to put it in like this).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Julien Grall <jgrall@amazon.com>
---
This is an alternative proposal to
https://lists.xen.org/archives/html/xen-devel/2023-12/msg01385.html.
---
v4: Re-base over changes earlier in the series (drop xhci-dbc.c change).
v3: Retain / extend use of const. Make part of series.
v2: [skipped]

--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -220,7 +220,7 @@ void do_IRQ(struct cpu_user_regs *regs,
 {
     struct irq_desc *desc = irq_to_desc(irq);
     struct irqaction *action;
-    struct cpu_user_regs *old_regs = set_irq_regs(regs);
+    const struct cpu_user_regs *old_regs = set_irq_regs(regs);
 
     perfc_incr(irqs);
 
--- a/xen/arch/x86/include/asm/processor.h
+++ b/xen/arch/x86/include/asm/processor.h
@@ -409,8 +409,7 @@ static always_inline void rep_nop(void)
 void show_code(const struct cpu_user_regs *regs);
 void show_stack_overflow(unsigned int cpu, const struct cpu_user_regs *regs);
 void show_registers(const struct cpu_user_regs *regs);
-#define dump_execution_state() \
-    run_in_exception_handler(show_execution_state_nonconst)
+#define dump_execution_state() run_in_exception_handler(show_execution_state)
 void show_page_walk(unsigned long addr);
 void noreturn fatal_trap(const struct cpu_user_regs *regs, bool show_remote);
 
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -1896,7 +1896,7 @@ void do_IRQ(struct cpu_user_regs *regs)
     struct irq_desc  *desc;
     unsigned int      vector = (uint8_t)regs->entry_vector;
     int               irq = this_cpu(vector_irq)[vector];
-    struct cpu_user_regs *old_regs = set_irq_regs(regs);
+    const struct cpu_user_regs *old_regs = set_irq_regs(regs);
 
     perfc_incr(irqs);
     this_cpu(irq_count)++;
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -643,7 +643,7 @@ void show_stack_overflow(unsigned int cp
     printk("\n");
 }
 
-void show_execution_state(const struct cpu_user_regs *regs)
+void cf_check show_execution_state(const struct cpu_user_regs *regs)
 {
     /* Prevent interleaving of output. */
     unsigned long flags = console_lock_recursive_irqsave();
@@ -655,11 +655,6 @@ void show_execution_state(const struct c
     console_unlock_recursive_irqrestore(flags);
 }
 
-void cf_check show_execution_state_nonconst(struct cpu_user_regs *regs)
-{
-    show_execution_state(regs);
-}
-
 void vcpu_show_execution_state(struct vcpu *v)
 {
     unsigned long flags = 0;
--- a/xen/common/bug.c
+++ b/xen/common/bug.c
@@ -10,7 +10,7 @@
  * Returns a negative value in case of an error otherwise
  * BUGFRAME_{run_fn, warn, bug, assert}
  */
-int do_bug_frame(struct cpu_user_regs *regs, unsigned long pc)
+int do_bug_frame(const struct cpu_user_regs *regs, unsigned long pc)
 {
     const struct bug_frame *bug = NULL;
     const struct virtual_region *region;
@@ -44,14 +44,10 @@ int do_bug_frame(struct cpu_user_regs *r
 
     if ( id == BUGFRAME_run_fn )
     {
-        void (*fn)(struct cpu_user_regs *) = bug_ptr(bug);
+        bug_fn_t *fn = bug_ptr(bug);
 
         fn(regs);
 
-        /* Re-enforce consistent types, because of the casts involved. */
-        if ( false )
-            run_in_exception_handler(fn);
-
         return id;
     }
 
--- a/xen/common/irq.c
+++ b/xen/common/irq.c
@@ -1,7 +1,7 @@
 #include <xen/irq.h>
 #include <xen/errno.h>
 
-DEFINE_PER_CPU(struct cpu_user_regs *, irq_regs);
+DEFINE_PER_CPU(const struct cpu_user_regs *, irq_regs);
 
 int init_one_irq_desc(struct irq_desc *desc)
 {
--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -135,7 +135,7 @@ static void cf_check show_handlers(unsig
 
 static cpumask_t dump_execstate_mask;
 
-void cf_check dump_execstate(struct cpu_user_regs *regs)
+void cf_check dump_execstate(const struct cpu_user_regs *regs)
 {
     unsigned int cpu = smp_processor_id();
 
--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -1246,14 +1246,14 @@ static int cf_check ehci_dbgp_getc(struc
 /* Safe: ehci_dbgp_poll() runs as timer handler, so not reentrant. */
 static struct serial_port *poll_port;
 
-static void cf_check _ehci_dbgp_poll(struct cpu_user_regs *regs)
+static void cf_check _ehci_dbgp_poll(const struct cpu_user_regs *regs)
 {
     struct serial_port *port = poll_port;
     struct ehci_dbgp *dbgp = port->uart;
     unsigned long flags;
     unsigned int timeout = MICROSECS(DBGP_CHECK_INTERVAL);
     bool empty = false;
-    struct cpu_user_regs *old_regs;
+    const struct cpu_user_regs *old_regs;
 
     if ( !dbgp->ehci_debug )
         return;
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -206,11 +206,11 @@ static void cf_check ns16550_interrupt(i
 /* Safe: ns16550_poll() runs as softirq so not reentrant on a given CPU. */
 static DEFINE_PER_CPU(struct serial_port *, poll_port);
 
-static void cf_check __ns16550_poll(struct cpu_user_regs *regs)
+static void cf_check __ns16550_poll(const struct cpu_user_regs *regs)
 {
     struct serial_port *port = this_cpu(poll_port);
     struct ns16550 *uart = port->uart;
-    struct cpu_user_regs *old_regs;
+    const struct cpu_user_regs *old_regs;
 
     if ( uart->intr_works )
         return; /* Interrupts work - no more polling */
--- a/xen/include/xen/bug.h
+++ b/xen/include/xen/bug.h
@@ -101,8 +101,7 @@ typedef void bug_fn_t(const struct cpu_u
 
 #ifndef run_in_exception_handler
 
-static void always_inline run_in_exception_handler(
-    void (*fn)(struct cpu_user_regs *regs))
+static void always_inline run_in_exception_handler(bug_fn_t *fn)
 {
     BUG_FRAME(BUGFRAME_run_fn, 0, fn, 0, NULL);
 }
@@ -133,7 +132,7 @@ static void always_inline run_in_excepti
  * Returns a negative value in case of an error otherwise
  * BUGFRAME_{run_fn, warn, bug, assert}
  */
-int do_bug_frame(struct cpu_user_regs *regs, unsigned long pc);
+int do_bug_frame(const struct cpu_user_regs *regs, unsigned long pc);
 
 #endif /* CONFIG_GENERIC_BUG_FRAME */
 
--- a/xen/include/xen/irq.h
+++ b/xen/include/xen/irq.h
@@ -134,21 +134,22 @@ void cf_check irq_actor_none(struct irq_
  * Per-cpu interrupted context register state - the inner-most interrupt frame
  * on the stack.
  */
-DECLARE_PER_CPU(struct cpu_user_regs *, irq_regs);
+DECLARE_PER_CPU(const struct cpu_user_regs *, irq_regs);
 
-static inline struct cpu_user_regs *get_irq_regs(void)
+static inline const struct cpu_user_regs *get_irq_regs(void)
 {
-	return this_cpu(irq_regs);
+    return this_cpu(irq_regs);
 }
 
-static inline struct cpu_user_regs *set_irq_regs(struct cpu_user_regs *new_regs)
+static inline const struct cpu_user_regs *set_irq_regs(
+    const struct cpu_user_regs *new_regs)
 {
-	struct cpu_user_regs *old_regs, **pp_regs = &this_cpu(irq_regs);
+    const struct cpu_user_regs *old_regs, **pp_regs = &this_cpu(irq_regs);
 
-	old_regs = *pp_regs;
-	*pp_regs = new_regs;
+    old_regs = *pp_regs;
+    *pp_regs = new_regs;
 
-	return old_regs;
+    return old_regs;
 }
 
 struct domain;
--- a/xen/include/xen/kernel.h
+++ b/xen/include/xen/kernel.h
@@ -110,8 +110,7 @@ extern const unsigned int xen_config_dat
 struct cpu_user_regs;
 struct vcpu;
 
-void show_execution_state(const struct cpu_user_regs *regs);
-void cf_check show_execution_state_nonconst(struct cpu_user_regs *regs);
+void cf_check show_execution_state(const struct cpu_user_regs *regs);
 void vcpu_show_execution_state(struct vcpu *v);
 
 #endif /* _LINUX_KERNEL_H */
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -173,7 +173,7 @@ extern char *print_tainted(char *str);
 extern void add_taint(unsigned int taint);
 
 struct cpu_user_regs;
-void cf_check dump_execstate(struct cpu_user_regs *regs);
+void cf_check dump_execstate(const struct cpu_user_regs *regs);
 
 void init_constructors(void);
 



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 08:58:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 08:58:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683316.1062787 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLxf-0007Cu-Hj; Tue, 20 Feb 2024 08:58:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683316.1062787; Tue, 20 Feb 2024 08:58:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcLxf-0007Cn-EB; Tue, 20 Feb 2024 08:58:47 +0000
Received: by outflank-mailman (input) for mailman id 683316;
 Tue, 20 Feb 2024 08:58:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcLxe-0007Cf-JM
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 08:58:46 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 40508ec3-cfce-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 09:58:44 +0100 (CET)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-3394b892691so2824714f8f.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 00:58:44 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020adfcd0e000000b0033d14c96ec1sm12607957wrm.45.2024.02.20.00.58.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 00:58:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 40508ec3-cfce-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708419524; x=1709024324; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=l7RgsLGIxKZjFjkm6tWwc7IiHgqfN1e9ON21bL9X0ww=;
        b=bJUsJm3TrmLnQ+2PX9b3qObviWTsAZQE1K9RtS2n3l0/HoM1fkO2fsEQFqazUOL5ew
         f7bkFi9tu3mOAsS4weItzVdsYGlsOnPKukEFkAqwMoSF5d+526lgxZOHEWy/U48wVuyx
         EfUBe2aWH2lhAKmz46U2jvAQhSwYrZCsRdBnfuvuie8cQqmzxdKTZW8VzT1lR0vFS4RX
         TxQLLuyvpVwSlop5hDjLl7wy+p8sumvRC9ujwA7tyX5hGeg5mzL97n+P0UGaxUOg5++V
         8M21Sqy165VVqVSNWBREV3FNQxTvLtrCV/6U36oV5rnrfYb2NScviC108n83u9JvJkuX
         X8Gg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708419524; x=1709024324;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=l7RgsLGIxKZjFjkm6tWwc7IiHgqfN1e9ON21bL9X0ww=;
        b=uuqzPN6p3G7PUnAir9wkdla0hdwB3n3CmLNpV6M+GI4C/23ZVy+1xMadXYSCZgCyc3
         E2pDsVz6HktMeHTjz8I9TYzWxfWiXZ6ajn544Ae9iyrlM1k6pfsZFvh6TV97kjQrB6gc
         0KKgknmi4a7LjCBNUHTozRXOZ95Ji8O3rkYHMlyUIRgSbBteRdUag7vEQT0rgz4pcmaq
         PLNFeAxdsoKR20gL/IOIpQ63EF4AdWsgUDHWWCE+IzfzddWHei8Dnz0ZfsydB4wJPI7/
         Jnf2hgAEeO0D9nK8ecZe3WYVqSVOcLlfsnV4Yn5JuV7PQxtC9VUqb/i69Q2f17XoblFy
         7D0w==
X-Gm-Message-State: AOJu0YzB+tfyTARQynJKSkI85Jhrc8RYCxS5d8eKkbw9fU4XBLnB/B+D
	halJ5JMYhJSRKuZgL6lZy/xxT+idqsNUdKeOpll5y9W1Ev5ruL+3TpEJyu17SyfnMaM/y1h8I/4
	=
X-Google-Smtp-Source: AGHT+IFZK/acD1PwUe+3xq0lACWefO9yT1lPal999S1+H4X230qVjE7lWrG2s0Z0WPrTqgjlR727gQ==
X-Received: by 2002:a05:6000:1548:b0:33d:3c0c:ad53 with SMTP id 8-20020a056000154800b0033d3c0cad53mr7639800wry.30.1708419524067;
        Tue, 20 Feb 2024 00:58:44 -0800 (PST)
Message-ID: <80b4ca6f-a9cf-4060-a6ea-03ef61350680@suse.com>
Date: Tue, 20 Feb 2024 09:58:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 0/8] limit passing around of cpu_user_regs
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 20.02.2024 09:51, Jan Beulich wrote:
> Unlike (synchronous) exception handlers, interrupt handlers don't normally
> have a need to know the outer context's register state. Similarly, the vast
> majority of key handlers has no need for such.
> 
> 1: serial: fake IRQ-regs context in poll handlers
> 2: keyhandler: drop regs parameter from handle_keyregs()
> 3: serial: drop serial_rx_fn's regs parameter
> 4: PV-shim: drop pv_console_rx()'s regs parameter
> 5: serial: drop serial_[rt]x_interrupt()'s regs parameter
> 6: IRQ: drop regs parameter from handler functions
> 7: x86/APIC: drop regs parameter from direct vector handler functions
> 8: consolidate do_bug_frame() / bug_fn_t
> 
> I think it is now only the lack of an ack for the 1st patch which is keeping
> this series from going in.

Actually while sending I noticed that further patch 5 still is in need
of an Arm ack.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 09:04:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 09:04:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683337.1062797 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcM35-0001Ka-EY; Tue, 20 Feb 2024 09:04:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683337.1062797; Tue, 20 Feb 2024 09:04:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcM35-0001KT-Bl; Tue, 20 Feb 2024 09:04:23 +0000
Received: by outflank-mailman (input) for mailman id 683337;
 Tue, 20 Feb 2024 09:04:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=00X/=J5=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rcM33-0001KN-U4
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 09:04:21 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 061e304c-cfcf-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 10:04:17 +0100 (CET)
Received: from DS7PR03CA0034.namprd03.prod.outlook.com (2603:10b6:5:3b5::9) by
 CY8PR12MB7339.namprd12.prod.outlook.com (2603:10b6:930:51::22) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7316.20; Tue, 20 Feb 2024 09:04:14 +0000
Received: from DS2PEPF0000343E.namprd02.prod.outlook.com
 (2603:10b6:5:3b5:cafe::f8) by DS7PR03CA0034.outlook.office365.com
 (2603:10b6:5:3b5::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.39 via Frontend
 Transport; Tue, 20 Feb 2024 09:04:14 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS2PEPF0000343E.mail.protection.outlook.com (10.167.18.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 20 Feb 2024 09:04:14 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 20 Feb
 2024 03:04:09 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 20 Feb 2024 03:04:08 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 061e304c-cfcf-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Tz4ugxHzcmNESCrYdRPAzWsLwc8EQzOePtAbn/ZJpJ04NTANNnXjfZJ7mBfmyAUiSqruJuHk8zBR9epyQsmdGOlLT3bXJe6oMfYNHc2fOz9hPnxw9WDqUwLUPS+u+J7WF9bhMeKy4CxgNdI+BZ5/5kX71ZDd1VixRzhIQ6ogqU9YZfWTqkB63+JyfU9scI9r7zKf92aO4MSItO0CNssh+NtqyUmZAa8iO4dJYxm/OZwHnj9pEUkw/de1JwOx/8s6Dv+Hy4kp2G3pYxEjl7UzCaqDd5p1qXUgBO1ACipvb6SXXzZNVzdaybOBSHl7dYZkyTlsdfaGpjRglglDCNNIgw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=GAyCeHGxDJVhPOYZpA+eOHF1amqsieMhyIA/Llp+P6E=;
 b=LwVNYsB7hskSvC7hLjl+xOsj6vXjtc4ql7uVWW4eMe28+5azR4gWkhogNaV1XdcuSartRHC1Y2sis7w46XJBdXE6K4mzih1LfTaYTG2fZSnDPc8W2Q6ZZtRBiV6fV03XUjwiWjQPRffd8HvqlZSPJ0oP4GqIBDB+/0qeKWO2e6vwwrvLAdQiLSnB7N2ZeuMBHmWl25gEPK40HYoCvCRmqz3PnlihoqijxqoSOMhhet52d89RamquI/giuDGbCA2KlpZjsPgA+MFqkBwbWwRFs8NpO4Mgqzbdqn0debVcI0+jB1zTaHDiFZa3GHynr6a3q2QOLT5hg5Pqqxj8rfZHDA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GAyCeHGxDJVhPOYZpA+eOHF1amqsieMhyIA/Llp+P6E=;
 b=nSTbvokvhQQ4Ui+VFNCyPr6Lg9n0DsQ9z++pDr471vBXmiQRb8bHrzANhSw1gOkL7UL4+pVwyTzvZx0epqIXNFeqW/J/1VOg1LpVvYK+exL/igcdSWEu4nRu9YICMw8SuwSu+utvYDeCbt+WRpXb9lowI1taSQwO7xg2wlieGcM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <f075513b-877a-42ea-beb2-0eccd64d3ef1@amd.com>
Date: Tue, 20 Feb 2024 10:04:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v4 2/3] xen/arm: arm64: Add emulation of Debug Data Transfer
 Registers
To: Julien Grall <julien@xen.org>, Ayan Kumar Halder <ayankuma@amd.com>, "Ayan
 Kumar Halder" <ayan.kumar.halder@amd.com>, <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <stefano.stabellini@amd.com>,
	<Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>,
	<luca.fancellu@arm.com>
References: <20240131121049.225044-1-ayan.kumar.halder@amd.com>
 <20240131121049.225044-3-ayan.kumar.halder@amd.com>
 <0c4e4a40-7327-4377-bfdd-82e69bc838ae@xen.org>
 <7098c9ab-7008-4a49-92d4-6cd201b1490e@amd.com>
 <9d1a545c-a93e-4f8e-bc76-0b5676fa01ea@amd.com>
 <aaeca1c4-8898-400f-a608-6ef758ed9d96@xen.org>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <aaeca1c4-8898-400f-a608-6ef758ed9d96@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS2PEPF0000343E:EE_|CY8PR12MB7339:EE_
X-MS-Office365-Filtering-Correlation-Id: 85e6e208-2d20-40f8-6389-08dc31f2e90f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dF+03IJYXmGxz4cm5sEeiJZI+ha36E9NIeirbP9S1zYA1l3QoRiq9QiJLPgounfvsDgWVsvI6gTbzsW9eZ3JfLpLVteTZKCx/PaOoInR0G+Tkp74QpQT5jKtWDHa4admCDwI2Ky9g49uSHk8IVMpEXQP+rZc/p6EQZlF+K28vGBG5j2rIkUeZDUzDkQ08CUwTDNDRvkILnptDR3V15fozwr4iYFce3hmGZW8JJ/MwbFVUth9ikKlmCT96IvjFi5J483NxOfBzmHrLP3RJj5utkyHV7Ik5qR7Vc7hgChXZSCLiFGN42beLiGmIQ+mPWCB6yfS3KW9rip922qggwP6akhn4mwrLFXVLfAe7PHV1LD6kEq5Vtbxey3yL87oOXP4Pcnu+SdEDqrh7az1ZTA646gtJftM0oXqnr20JNYx6wgcka0FvGT1J7w7syZ9y5j1dvqRQoOChqJ+Ci05DkU6Y07pMy/qpDFemRWVl2opdUk4UKx/1Q7XJXOaNeewU7WENxw2hUoCu3h5No2Gb8HjOHQA2odS2NHwn6wSdcPyrY8a2/qARFKad0dC9e1V66yLZB3AObUvXupygjeB9d4YBCcDREKWp32THBRPhpHG5J02bxeFY72W6zaKOXa0y6S4Fi3TXnEinATIbn2qXVr/V/H+UeQBxZ+Qp1vjXArDnrM=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(46966006)(40470700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2024 09:04:14.0933
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 85e6e208-2d20-40f8-6389-08dc31f2e90f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS2PEPF0000343E.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7339

Hi Julien,

On 19/02/2024 19:48, Julien Grall wrote:
> 
> 
> Hi Michal,
> 
> On 19/02/2024 15:43, Michal Orzel wrote:
> 
>> On 19/02/2024 15:45, Ayan Kumar Halder wrote:
>>>
>>> On 06/02/2024 19:05, Julien Grall wrote:
>>>> Hi Ayan,
>>> Hi Julien/Michal,
>>>>
>>>> On 31/01/2024 12:10, Ayan Kumar Halder wrote:
>>>>> From: Michal Orzel <michal.orzel@amd.com>
>>>>>
>>>>> Currently, if user enables HVC_DCC config option in Linux, it invokes
>>>>> access
>>>>> to debug data transfer registers (i.e. DBGDTRTX_EL0 on arm64,
>>>>> DBGDTRTXINT on
>>>>> arm32). As these registers are not emulated, Xen injects an undefined
>>>>> exception to the guest and Linux crashes.
>>>>>
>>>>> To prevent this crash, introduce a partial emulation of DBGDTR[TR]X_EL0
>>>>> (these registers share the same encoding) as RAZ/WI and MDCCSR_EL0 as
>>>>> TXfull.
>>>>>
>>>>> Refer ARM DDI 0487J.a ID042523, D19.3.8, DBGDTRTX_EL0
>>>>> "If TXfull is set to 1, set DTRRX and DTRTX to UNKNOWN".
>>>>>
>>>>> Thus, any OS is expected to read MDCCSR_EL0 and check for TXfull before
>>>>> using DBGDTRTX_EL0. Linux does it via hvc_dcc_init() --->
>>>>> hvc_dcc_check(),
>>>>> and returns -ENODEV in case TXfull bit is still set after writing a test
>>>>> character. This way we prevent the guest from making use of HVC DCC as a
>>>>> console.
>>>>>
>>>>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>>>>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
>>>>> ---
>>>>> Changes from
>>>>>
>>>>> v1 :- 1. DBGDTR_EL0 does not emulate RXfull. This is to avoid giving
>>>>> the OS any
>>>>> indication that the RX buffer is full and is waiting to be read.
>>>>>
>>>>> 2. In Arm32, DBGOSLSR is emulated. Also DBGDTRTXINT is emulated at
>>>>> EL0 only.
>>>>>
>>>>> 3. Fixed the commit message and inline code comments.
>>>>>
>>>>> v2 :- 1. Split the patch into two (separate patches for arm64 and
>>>>> arm32).
>>>>> 2. Removed the "fail" label.
>>>>> 3. Fixed the commit message.
>>>>>
>>>>> v3 :- 1. "HSR_SYSREG_MDCCSR_EL0" emulation differs based on whether
>>>>> partial_emulation_enabled is true or not.
>>>>>
>>>>> 2. If partial_emulation_enabled is false, then access to
>>>>> HSR_SYSREG_DBGDTR_EL0,
>>>>> HSR_SYSREG_DBGDTRTX_EL0 would lead to undefined exception.
>>>>>
>>>>>    xen/arch/arm/arm64/vsysreg.c         | 28 ++++++++++++++++++++++++----
>>>>>    xen/arch/arm/include/asm/arm64/hsr.h |  3 +++
>>>>>    2 files changed, 27 insertions(+), 4 deletions(-)
>>>>>
>>>>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>>>>> index b5d54c569b..94f0a6c384 100644
>>>>> --- a/xen/arch/arm/arm64/vsysreg.c
>>>>> +++ b/xen/arch/arm/arm64/vsysreg.c
>>>>> @@ -159,9 +159,6 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>>>         *
>>>>>         * Unhandled:
>>>>>         *    MDCCINT_EL1
>>>>> -     *    DBGDTR_EL0
>>>>> -     *    DBGDTRRX_EL0
>>>>> -     *    DBGDTRTX_EL0
>>>>>         *    OSDTRRX_EL1
>>>>>         *    OSDTRTX_EL1
>>>>>         *    OSECCR_EL1
>>>>> @@ -173,10 +170,32 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>>>            return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
>>>>>        case HSR_SYSREG_MDCCSR_EL0:
>>>>>            /*
>>>>> +         * Xen doesn't expose a real (or emulated) Debug
>>>>> Communications Channel
>>>>> +         * (DCC) to a domain. Yet the Arm ARM implies this is not an
>>>>> optional
>>>>> +         * feature. So some domains may start to probe it. For
>>>>> instance, the
>>>>> +         * HVC_DCC driver in Linux (since f377775dc083 and at least
>>>>> up to v6.7),
>>>>> +         * will try to write some characters and check if the
>>>>> transmit buffer
>>>>> +         * has emptied.
>>>>> +         *
>>>>> +         * By setting TX status bit (only if partial emulation is
>>>>> enabled) to
>>>>> +         * indicate the transmit buffer is full, we would hint the
>>>>> OS that the
>>>>> +         * DCC is probably not working.
>>>>> +         *
>>>>> +         * Bit 29: TX full
>>>>> +         *
>>>>>             * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We
>>>>> emulate that
>>>>>             * register as RAZ/WI above. So RO at both EL0 and EL1.
>>>>
>>>> The sentence "we emulate that register as ..." seems to be stale?
>> I can see that you tried to handle Julien remark here. But I disagree. This statement
>> is not stale. It explains that because MDSCR_EL1 is RAZ/WI, MDCCSR_EL0 is RO at both
>> EL0 and EL1. This patch does not change this behavior.
> 
> Indeed. I misread the comment. So what I wrote can be ignored here.
> 
>>
>>>>
>>>>>             */
>>>>> -        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 0);
>>>>> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read,
>>>>> hsr, 0,
>>>>> +                                  partial_emulation ? (1U << 29) : 0);
>>>>> +
>>>>> +    case HSR_SYSREG_DBGDTR_EL0:
>>>>> +    /* DBGDTR[TR]X_EL0 share the same encoding */
>>>>> +    case HSR_SYSREG_DBGDTRTX_EL0:
>>>>> +        if ( !partial_emulation )
>>>>> +            goto fail;
>>>>> +        return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 0);
>>>>
>>>> AFAICT, all the emulation helpers have an explanation why we are using
>>>> them. But here this is not the case. Can you add one?
>>> This and..
>>>>
>>>>> +
>>>>>        HSR_SYSREG_DBG_CASES(DBGBVR):
>>>>>        HSR_SYSREG_DBG_CASES(DBGBCR):
>>>>>        HSR_SYSREG_DBG_CASES(DBGWVR):
>>>>> @@ -394,6 +413,7 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>>>         * And all other unknown registers.
>>>>>         */
>>>>>        default:
>>>>> + fail:
>>>>
>>>> AFAICT, this would violate MISRA 15.3 [1]. We didn't seem to have yet
>>>> (?) accepted the rule, but I don't see we would not given I feel this
>>>> is similar to what Rule 16.2 is trying to prevent and we accepted it.
>>>>
>>>> I think case, I move all the code within default outside. And then
>>>> call "goto fail" from the default label.
>>>
>>> I am not sure if I have interpreted this correctly.
>>>
>>> Is it ok if you can take a look at the attached patch and let me know if
>>> the explaination and the code change looks sane ?
>> Looking at the attached patch and fail handling, I don't think it is what Julien meant.
>> In the default case you should jump to fail that would be defined outside of switch clause.
>>
>> Something like:
>>      default:
>>          goto fail;
>>      }
>>
>>      regs->pc += 4;
>>      return;
>>
>> fail:
>>      gdprintk...
> 
> +1.
> 
>>
>> When it comes to explanation for HSR_SYSREG_DBGDTRTX_EL0, I will let Julien to provide a comment he believes is right.
>> To me, it feels strange to repeat almost the same information as for MDCCSR_EL0.
> 
> It is not clear to me whether you are objecting of adding a comment or
> whether you would be ok with a comment that is not duplicating.
Adding a comment is always welcome. I was against duplication.

@Ayan:
Move a paragraph starting with "Xen doesn't expose" above case HSR_SYSREG_MDCCSR_EL0 and leave rest as is.
For HSR_SYSREG_DBGDTRTX_EL0, add sth like:

Emulate as RAZ/WI (only if partial emulation is enabled) to prevent injecting undefined exception.
Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We emulate that register as RAZ/WI.

~Michal


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 09:17:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 09:17:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683347.1062807 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcMFM-0003mB-J6; Tue, 20 Feb 2024 09:17:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683347.1062807; Tue, 20 Feb 2024 09:17:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcMFM-0003m4-FH; Tue, 20 Feb 2024 09:17:04 +0000
Received: by outflank-mailman (input) for mailman id 683347;
 Tue, 20 Feb 2024 09:17:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=g8eU=J5=minervasys.tech=carlo.nonato@srs-se1.protection.inumbo.net>)
 id 1rcMFL-0003ly-1k
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 09:17:03 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cd8a3cfe-cfd0-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 10:17:00 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-563cb3ba9daso5131764a12.3
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 01:17:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd8a3cfe-cfd0-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=minervasys-tech.20230601.gappssmtp.com; s=20230601; t=1708420620; x=1709025420; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=1rXxwlzcWHZiy1kRayV9wJT5zpZnSf9Kb/rmAxXTYWY=;
        b=X6so+6oYcMX9wCrAN4YxZUXiMO2ZuC/O6ktI0e31dRsWwbWNX2mnt/dinUtHeeNnyd
         YYPGKp4kfA6iZBpd3YJN6qtVtHHi8epDZFkq6PyaztmBmIHPBrpE5p0fGf46SQqnYEAz
         n6FdsNlySt0aFPv1IsZk/tlHf8trpd3m4i8czo0frwhoJr1NefKTog+bTm8wamcAETVD
         jxH2FKfvpuIAWHFmz1M+jmCP7BiQvD4lRfallj2UqRj3xdjTCm64B9nKpIsVy6ekXPWp
         av0e2eRmBS8Ju1spfOEDO6nRL9omVuHPyxsLqSPGTDW7TnhgEi3X3OXlNwqXy2y1zQBQ
         gjwg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708420620; x=1709025420;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=1rXxwlzcWHZiy1kRayV9wJT5zpZnSf9Kb/rmAxXTYWY=;
        b=rHl9/GlCXrJtHRqMC6Iya77wpALB8oQU+fEURgPBj+Zi+hRiU4Bal5K0UcLfY2ce2W
         C+xGT2VuRpve38tuIpDHN+o/2DvFHSKxc1rEQCx40HuffR+6frlLWvZTl7KSrvxQDKqo
         J/QPAwOVQIV9dM1QTpFu3n3GiafJGFE4ZfGMz8ccql+VNNSk4SZ6l0Vn0Rongb4I79bs
         RrdYCazArn/dTHIztEXW6sZIWYb/igBakjYAbe0PCe9I/5bM/0kiqE3m0bQ0XCNkEua5
         DQ6r4uAmlvJHfE1m4wn1+uAKIPu8WDAFONyZxDKxaXdW+tphkbyt+nNsWj4xaMP90sdh
         SZ1A==
X-Forwarded-Encrypted: i=1; AJvYcCUndNFKCdBEpPyrGSLSl+w9lWjhDoywxYweOz6sdPjhFFkxSWpa8Sb+hg+BC7VBazyqSY5XbUkvwkkFm5Ezd0llQ5b0VnKMie2ia+gtddQ=
X-Gm-Message-State: AOJu0YyXT8nX38+PhWn/BtLiXesfo4BVvHxgBsSEQMiWKsy3Z0d9ZFez
	2Py7M90Y/MmGdK7PlU4YT41P8Ym92on9EC1FtnfYyyWhBAl8eqzD6qeDnwxVfNmgckbtpQYz8yG
	fIZVdGgGigKujBsxDjcfdG51lgymlcXqm7WsWsw==
X-Google-Smtp-Source: AGHT+IHGqZmj1XoE92SgXdaiZwPbtwJ8uCEiXeV9KzQ1mzqCmHl9rgTyCxLODj5gKNiWMWXNClsXIHVLsfRe6UceBc8=
X-Received: by 2002:aa7:d697:0:b0:564:56ce:5570 with SMTP id
 d23-20020aa7d697000000b0056456ce5570mr3726037edr.33.1708420619889; Tue, 20
 Feb 2024 01:16:59 -0800 (PST)
MIME-Version: 1.0
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-3-carlo.nonato@minervasys.tech> <bb28c8a2-c2bd-4e1d-ad28-9bc632e3e474@amd.com>
 <CAG+AhRXsZC7Pjdce42yYTvBy7MyjS1axrDGsY0U=o+APNyxdYA@mail.gmail.com> <5d2f6be2-020d-4003-ad47-b20ef07ffefc@xen.org>
In-Reply-To: <5d2f6be2-020d-4003-ad47-b20ef07ffefc@xen.org>
From: Carlo Nonato <carlo.nonato@minervasys.tech>
Date: Tue, 20 Feb 2024 10:16:49 +0100
Message-ID: <CAG+AhRX96RBCc0n750K3t+c3j_fKQLrg1K5kLceHig401wS+Ng@mail.gmail.com>
Subject: Re: [PATCH v6 02/15] xen/arm: add initial support for LLC coloring on arm64
To: Julien Grall <julien@xen.org>
Cc: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org, 
	andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
	Marco Solieri <marco.solieri@minervasys.tech>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Julien

On Tue, Feb 20, 2024 at 12:06=E2=80=AFAM Julien Grall <julien@xen.org> wrot=
e:
>
> Hi,
>
> On 14/02/2024 13:52, Carlo Nonato wrote:
> > On Wed, Feb 14, 2024 at 11:14=E2=80=AFAM Michal Orzel <michal.orzel@amd=
.com> wrote:
> >>> diff --git a/xen/arch/arm/include/asm/processor.h b/xen/arch/arm/incl=
ude/asm/processor.h
> >>> index 8e02410465..336933ee62 100644
> >>> --- a/xen/arch/arm/include/asm/processor.h
> >>> +++ b/xen/arch/arm/include/asm/processor.h
> >>> @@ -18,6 +18,22 @@
> >>>   #define CTR_IDC_SHIFT       28
> >>>   #define CTR_DIC_SHIFT       29
> >>>
> >>> +/* CCSIDR Current Cache Size ID Register */
> >>> +#define CCSIDR_LINESIZE_MASK            _AC(0x7, ULL)
> >> Why ULL and not UL? ccsidr is of register_t type
> >
> > Julien, while reviewing an earlier version:
> >
> >> Please use ULL here otherwise someone using MASK << SHIFT will have th=
e
> >> expected result.
> >
> > https://patchew.org/Xen/20220826125111.152261-1-carlo.nonato@minervasys=
.tech/20220826125111.152261-2-carlo.nonato@minervasys.tech/#08956082-c194-8=
bae-cb25-44e4e3227689@xen.org
>
> Michal is right. This should be UL. Not sure why I suggested ULL back
> then. Sorry.

No problem.

If there aren't any other comments I will proceed with sending the v7.
Do you guys want to add something on the arm part?

Thanks to both.

> Cheers,
>
> --
> Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 09:51:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 09:51:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683353.1062817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcMmR-0002Ya-6H; Tue, 20 Feb 2024 09:51:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683353.1062817; Tue, 20 Feb 2024 09:51:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcMmR-0002YT-2o; Tue, 20 Feb 2024 09:51:15 +0000
Received: by outflank-mailman (input) for mailman id 683353;
 Tue, 20 Feb 2024 09:51:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcMmP-0002YN-H8
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 09:51:13 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 94683909-cfd5-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 10:51:12 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-4126dd40a54so4443755e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 01:51:12 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 fs14-20020a05600c3f8e00b00411a595d56bsm13901193wmb.14.2024.02.20.01.51.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 01:51:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 94683909-cfd5-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708422671; x=1709027471; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=P7/Yq2uTii1GCM68bdCxdDu0KhvKbyUGIl+DXx91luE=;
        b=XMkVRcTd7MEakLTK0pOHvTV2UWVwBhlCy0txYu/wXpS4hOy55Dp8zc1B75x1NW0dlq
         un/joKZcjrGuvHCaeDR5Mg+h3V+p7vxK8LzgrlTKhjtmtQm43Ej45eorJhl4PHNy/ccV
         D4tksM+yU8bI+ygV7NIKhgNr7M3ZIC29pIM12TkW4tsUgGm6nkoavXK+EotgMZ6/KxC5
         hIrppxfWQlQTofK9okGDBlIG6KjjjFbUKAMH0q13I+RlB/tRFNv5UyI8+lCqyesY5b/u
         OR6SUc+arD+FuPD37ghkhR8/aVzZzwmkpmKbW+wzvno4gBs2eXOom5k/sZrzIAlevc7K
         r1Pw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708422671; x=1709027471;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=P7/Yq2uTii1GCM68bdCxdDu0KhvKbyUGIl+DXx91luE=;
        b=CAPCNJ/E6JfEphEfIQbw3JGd46OFErwxAWrd1PIwSjJbHCYtDniV9tW7ChEjtjal+5
         9FUYvjVHU93/ILkwwzGNzpo2cWaEP/PFs7YbtIXvxsYK4Ji8+gtJ0i1bDLvuqFMIsimp
         u6YFq7xm5QzHWBBsf0NktQWGQhLjTmVEwigWRXkcfcbSwbKq46IxEKH9V3e/iFxx8nPR
         mb5KvKU0tyueJsrIqY697tNeA3zhpN6K9pVCsYT3JuOPdb4hkmEBWcGBPfycLlArVbDh
         EqykcHcHhRSsMkWjpo+r/XKDzru+zkt7VePd/FGct/xj+Z1J0n5rnF4MdsanWgUjngG6
         eXeA==
X-Forwarded-Encrypted: i=1; AJvYcCUIBprrJPumIhQLNzQPR6sf627d4R+xd/QHH2WlcYFBAMBGZMIC7y4yrkN+fY9rp2hZzk/u3pRxzmKm7OlTBBdL+F9qXmYud2BOak9ZmwY=
X-Gm-Message-State: AOJu0YxSCxNvqTXVSJ9cnyr18hqQ1R3D6Oy3jwr3sjGatqZqFHRIjw/t
	Mf65roKS5bGTfTFQbltbJwLaKgtKs42puwcaYdolQ47NZa1RTIv2fdfObUpOJQ==
X-Google-Smtp-Source: AGHT+IFdDWQvISjNtXBz8Jd95yjWmEtM9cF2XYPeEuuky7lj9OB1Gk53ff3KLjzPXe3qo59LboOO+g==
X-Received: by 2002:a05:600c:1994:b0:410:ac80:bf3e with SMTP id t20-20020a05600c199400b00410ac80bf3emr1960505wmq.37.1708422671532;
        Tue, 20 Feb 2024 01:51:11 -0800 (PST)
Message-ID: <a2ce9820-c57a-4690-9dc6-c15d8a1489f4@suse.com>
Date: Tue, 20 Feb 2024 10:51:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 07/27] x86: Map/unmap pages in
 restore_all_guests
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-8-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240116192611.41112-8-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.01.2024 20:25, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> Before, it assumed the pv cr3 could be accessed via a direct map. This
> is no longer true.

There are a number of terminology issues here, starting with the title:
Unlike (iirc) in an earlier version, no mapping/unmapping occurs in
restore_all_guests itself anymore. When reading just the title I
thought "What? Didn't I say no there?" Then for the sentence above: If
what it stated was true, a bug would have been introduced in one of
the earlier patches. What I think is meant, though, is that this is
going to be no longer true.

Finally the use of "shadow" in identifiers in the code changes
themselves is somewhat problematic imo, seeing the shadow paging mode
we support for both HVM and PV. The nature here is different (it's
merely a secondary mapping aiui), so I think it would be better to
avoid the term "shadow". Maybe ...

> --- a/xen/arch/x86/include/asm/config.h
> +++ b/xen/arch/x86/include/asm/config.h
> @@ -202,7 +202,7 @@ extern unsigned char boot_edid_info[128];
>  /* Slot 260: per-domain mappings (including map cache). */
>  #define PERDOMAIN_VIRT_START    (PML4_ADDR(260))
>  #define PERDOMAIN_SLOT_MBYTES   (PML4_ENTRY_BYTES >> (20 + PAGETABLE_ORDER))
> -#define PERDOMAIN_SLOTS         3
> +#define PERDOMAIN_SLOTS         4
>  #define PERDOMAIN_VIRT_SLOT(s)  (PERDOMAIN_VIRT_START + (s) * \
>                                   (PERDOMAIN_SLOT_MBYTES << 20))
>  /* Slot 4: mirror of per-domain mappings (for compat xlat area accesses). */
> @@ -316,6 +316,16 @@ extern unsigned long xen_phys_start;
>  #define ARG_XLAT_START(v)        \
>      (ARG_XLAT_VIRT_START + ((v)->vcpu_id << ARG_XLAT_VA_SHIFT))
>  
> +/* root_pt shadow mapping area. The fourth per-domain-mapping sub-area */
> +#define SHADOW_ROOT_PT_VIRT_START   PERDOMAIN_VIRT_SLOT(3)
> +#define SHADOW_ROOT_PT_ENTRIES      MAX_VIRT_CPUS
> +#define SHADOW_ROOT_PT_VIRT_END     (SHADOW_ROOT_PT_VIRT_START +    \
> +                                     (SHADOW_ROOT_PT_ENTRIES * PAGE_SIZE))
> +
> +/* The address of a particular VCPU's ROOT_PT */
> +#define SHADOW_ROOT_PT_VCPU_VIRT_START(v) \
> +    (SHADOW_ROOT_PT_VIRT_START + ((v)->vcpu_id * PAGE_SIZE))

... ROOT_PT_MAPPING_* throughout, or PV_ROOT_PT_MAPPING_*?

As to SHADOW_ROOT_PT_VIRT_END - when trying to check where it's used
and hence whether it really needs to use MAX_VIRT_CPUS I couldn't
spot any use. I don't think the constant should be defined when it's
not needed.

> --- a/xen/arch/x86/mm.c
> +++ b/xen/arch/x86/mm.c
> @@ -505,6 +505,13 @@ void share_xen_page_with_guest(struct page_info *page, struct domain *d,
>      spin_unlock(&d->page_alloc_lock);
>  }
>  
> +#define shadow_root_pt_idx(v) \
> +    ((v)->vcpu_id >> PAGETABLE_ORDER)
> +
> +#define pv_shadow_root_pt_pte(v) \
> +    ((v)->domain->arch.pv.shadow_root_pt_l1tab[shadow_root_pt_idx(v)] + \
> +     ((v)->vcpu_id & (L1_PAGETABLE_ENTRIES - 1)))

I think uniformly named constant want using in both macros, i.e. either
some L1_* in the first macro (preferred) or an expression derived from
PAGETABLE_ORDER in the 2nd.

> @@ -524,6 +531,13 @@ void write_ptbase(struct vcpu *v)
>  
>      if ( is_pv_vcpu(v) && v->domain->arch.pv.xpti )
>      {
> +        mfn_t guest_root_pt = _mfn(v->arch.cr3 >> PAGE_SHIFT);

While we do so in several other places, I think we'd be better off not
continuing to assume the top bits to all be zero. IOW MASK_EXTR() may
be better to use here.

> +        l1_pgentry_t *pte = pv_shadow_root_pt_pte(v);
> +
> +        ASSERT(v == current);
> +
> +        l1e_write(pte, l1e_from_mfn(guest_root_pt, __PAGE_HYPERVISOR_RW));

The mapping is the copy source in restore_all_guests, isn't it? In
which case couldn't this be a r/o mapping?

> --- a/xen/arch/x86/pv/domain.c
> +++ b/xen/arch/x86/pv/domain.c
> @@ -288,6 +288,19 @@ static void pv_destroy_gdt_ldt_l1tab(struct vcpu *v)
>                                1U << GDT_LDT_VCPU_SHIFT);
>  }
>  
> +static int pv_create_shadow_root_pt_l1tab(struct vcpu *v)
> +{
> +    return create_perdomain_mapping(v->domain, SHADOW_ROOT_PT_VCPU_VIRT_START(v),

This line looks to be too long. But ...

> +                                    1, v->domain->arch.pv.shadow_root_pt_l1tab,
> +                                    NULL);
> +}
> +
> +static void pv_destroy_shadow_root_pt_l1tab(struct vcpu *v)
> +
> +{
> +    destroy_perdomain_mapping(v->domain, SHADOW_ROOT_PT_VCPU_VIRT_START(v), 1);
> +}

... I'm not convinced of the usefulness of these wrapper functions
anyway, even more so that each is used exactly once.

> @@ -297,6 +310,7 @@ void pv_vcpu_destroy(struct vcpu *v)
>      }
>  
>      pv_destroy_gdt_ldt_l1tab(v);
> +    pv_destroy_shadow_root_pt_l1tab(v);
>      XFREE(v->arch.pv.trap_ctxt);
>  }
>  
> @@ -311,6 +325,13 @@ int pv_vcpu_initialise(struct vcpu *v)
>      if ( rc )
>          return rc;
>  
> +    if ( v->domain->arch.pv.xpti )
> +    {
> +        rc = pv_create_shadow_root_pt_l1tab(v);
> +        if ( rc )
> +            goto done;
> +    }
> +
>      BUILD_BUG_ON(X86_NR_VECTORS * sizeof(*v->arch.pv.trap_ctxt) >
>                   PAGE_SIZE);
>      v->arch.pv.trap_ctxt = xzalloc_array(struct trap_info, X86_NR_VECTORS);
> @@ -346,10 +367,12 @@ void pv_domain_destroy(struct domain *d)
>  
>      destroy_perdomain_mapping(d, GDT_LDT_VIRT_START,
>                                GDT_LDT_MBYTES << (20 - PAGE_SHIFT));
> +    destroy_perdomain_mapping(d, SHADOW_ROOT_PT_VIRT_START, SHADOW_ROOT_PT_ENTRIES);

Too long line again.

>      XFREE(d->arch.pv.cpuidmasks);
>  
>      FREE_XENHEAP_PAGE(d->arch.pv.gdt_ldt_l1tab);
> +    FREE_XENHEAP_PAGE(d->arch.pv.shadow_root_pt_l1tab);
>  }
>  
>  void noreturn cf_check continue_pv_domain(void);
> @@ -371,6 +394,12 @@ int pv_domain_initialise(struct domain *d)
>          goto fail;
>      clear_page(d->arch.pv.gdt_ldt_l1tab);
>  
> +    d->arch.pv.shadow_root_pt_l1tab =
> +        alloc_xenheap_pages(0, MEMF_node(domain_to_node(d)));
> +    if ( !d->arch.pv.shadow_root_pt_l1tab )
> +        goto fail;
> +    clear_page(d->arch.pv.shadow_root_pt_l1tab);

Looks like you simply cloned the GDT/LDT code. That's covering 128k
of VA space per vCPU, though, while here you'd using only 4k. Hence
using a full page looks like a factor 32 over-allocation. And once
using xzalloc() here instead a further question would be whether to
limit to the domain's actual needs - most domains will have far less
than 8k vCPU-s. In the common case (up to 512 vCPU-s) a single slot
will suffice, at which point a yet further question would be whether
to embed the "array" in struct pv_domain instead in that common case
(e.g. by using a union).

> --- a/xen/arch/x86/x86_64/entry.S
> +++ b/xen/arch/x86/x86_64/entry.S
> @@ -165,7 +165,15 @@ restore_all_guest:
>          and   %rsi, %rdi
>          and   %r9, %rsi
>          add   %rcx, %rdi
> +
> +        /*
> +         * The address in the vCPU cr3 is always mapped in the shadow
> +         * root_pt virt area.
> +         */
> +        imul $PAGE_SIZE, VCPU_id(%rbx), %esi

Nit: Another blank please after the insn mnemonic, to match what's
upwards in context as well as ...

> +        movabs $SHADOW_ROOT_PT_VIRT_START, %rcx
>          add   %rcx, %rsi

... this.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 10:07:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 10:07:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683360.1062828 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcN2G-0004xj-KZ; Tue, 20 Feb 2024 10:07:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683360.1062828; Tue, 20 Feb 2024 10:07:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcN2G-0004xc-GQ; Tue, 20 Feb 2024 10:07:36 +0000
Received: by outflank-mailman (input) for mailman id 683360;
 Tue, 20 Feb 2024 10:07:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcN2E-0004xT-Ns
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 10:07:34 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dd102e6f-cfd7-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 11:07:32 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d21e2b2245so52191971fa.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 02:07:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 13-20020a05600c230d00b00411d640085csm13931459wmo.43.2024.02.20.02.07.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 02:07:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dd102e6f-cfd7-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708423652; x=1709028452; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Ti7KMHS/c5+d834QwZjjEwyyFQjvBe63dd9N36vHXJw=;
        b=JQtAWtpShSeqGdG/A0/+0FFnc9w3hzhhWHaivctre4Y2HFSnKS88FVCQ6qsy2Y46I/
         aEng2tffF3Z3sTa+KpD/9c2rXh15KJ2bfMkDdvoSKbv46qrbMBJic+7qrAnfbyGGG8JQ
         jKclsawqZR66Zdsd0o4/qMPsg0SLAUC5rMFpEHnS2BchD4P8myxdt/UCbqvRLiUZpUjF
         D8MboqYV6uMzdcxUiiBUC/x6jde+7IPkqH5+ZlMw+3Y3s9hCmnTf2ej9M+XS/Tyt907+
         4pSMhrr91oweAp5Vwf8EVY18y++FdAY47FX4r3Zj/UB3zmB2dfhbFvExGL2EzglBwAcJ
         rB5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708423652; x=1709028452;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Ti7KMHS/c5+d834QwZjjEwyyFQjvBe63dd9N36vHXJw=;
        b=mBvLgGXK56d8JNU/XKlz4d2UNEx7C7MgMiHGoFICExbcjuntwzHHI6qNESRoFv5KHn
         wJ3F6FMZ3lngrlrQKmOjaSwAUFqm+9k5zhwmyMRkp2F8QaDWe+yuyOaA3fEdMd8R9B60
         CD0mXFvTna8KZ1SpZLUeoSjqL9zOGljdqLelEdfah1Ewg/MYEHfDSzSuY8NWqsf24vff
         DeFHfGXKvF99jUrBiu99k9t41WP60HOIxX/apCgGl8fUlSY9dWG/akankmqN5GaoFpIr
         E+MnvmmFWZdlW4tNMsMbr1z9yiYed9CZc4ZoYNP7KsUbORrwqK1Ugn9sSQLAt0xWrwPn
         5SKA==
X-Forwarded-Encrypted: i=1; AJvYcCVR8lTGHmtxVe07z4z/QWGNp3Xm5Xhc1X4KaPODJVQNU/avCDnjXT7Hs217qL5mB6eMResy2+sAkxSAE5Okdd0XOgaudRyrTmAFRgH3x6M=
X-Gm-Message-State: AOJu0YwNMv5cSJv2Z1StQ4UZcVE1Uk85yfP/btkKy5ZgeFW/BjqeZp11
	K14iCeHAtdsdNRnacfmoRCdcJ/sp2hzDpjPti959NKcddDwYS2FNcjmgIEh0tA==
X-Google-Smtp-Source: AGHT+IGD2VCr6iRahOO1qKyKFmIRSKz6Q5Sa0vXaV0AEAM2cO9cL18DP/9kx3i/v4ol+AymbNEYJBg==
X-Received: by 2002:a2e:92ce:0:b0:2d0:b27e:2437 with SMTP id k14-20020a2e92ce000000b002d0b27e2437mr9437262ljh.39.1708423652428;
        Tue, 20 Feb 2024 02:07:32 -0800 (PST)
Message-ID: <55d42c31-2a10-4edf-8459-66929057c0db@suse.com>
Date: Tue, 20 Feb 2024 11:07:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 08/27] x86/pv: Domheap pages should be mapped
 while relocating initrd
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Wei Liu <wei.liu2@citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Wei Wang <wawei@amazon.de>, Julien Grall <jgrall@amazon.com>,
 xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-9-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240116192611.41112-9-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.01.2024 20:25, Elias El Yandouzi wrote:
> From: Wei Liu <wei.liu2@citrix.com>
> 
> Xen shouldn't use domheap page as if they were xenheap pages. Map and
> unmap pages accordingly.

The title could to with mentioning Dom0. Since it's already a little long,
the mentioning of domheap pages could be left to the description. E.g.
"map pages while relocating Dom0 initrd"?

> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -615,18 +615,25 @@ int __init dom0_construct_pv(struct domain *d,
>          if ( d->arch.physaddr_bitsize &&
>               ((mfn + count - 1) >> (d->arch.physaddr_bitsize - PAGE_SHIFT)) )
>          {
> +            unsigned long nr_pages;
> +
>              order = get_order_from_pages(count);
>              page = alloc_domheap_pages(d, order, MEMF_no_scrub);
>              if ( !page )
>                  panic("Not enough RAM for domain 0 initrd\n");
> +
> +            nr_pages = 1UL << order;

Nit: Is there anything wrong with making this the initializer of the
variable?

>              for ( count = -count; order--; )
>                  if ( count & (1UL << order) )
>                  {
>                      free_domheap_pages(page, order);
>                      page += 1UL << order;
> +                    nr_pages -= 1UL << order;
>                  }
> -            memcpy(page_to_virt(page), mfn_to_virt(initrd->mod_start),
> -                   initrd_len);
> +
> +            for ( i = 0; i < nr_pages; i++ )
> +                copy_domain_page(page_to_mfn(page + i), _mfn(initrd_mfn + i));

There's a type discrepancy between "i" and "nr_pages". If we truly expect
initrd-s of more than 16Tb size, "nr_pages" indeed needs to be unsigned
long, but then "i" can't remain as int. I for one think that having
"nr_pages" be unsigned int is more than enough. It might then still be
good to switch "i" from plain int to unsigned int, albeit maybe in a
(tiny) separate patch.

I further wonder whether it wouldn't be more consistent with the "else"
branch of the containing "if()" if instead of "initrd_mfn" "mfn" would
be used here (and incremented as we go). At which point I think the use
of "i" could be avoided here altogether:

            for ( ; nr_pages--; page++, mfn++ )
                copy_domain_page(page_to_mfn(page), _mfn(mfn));

or something substantially similar (e.g. re-written as while() loop).

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 10:21:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 10:21:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683366.1062837 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNFv-0008UZ-Ps; Tue, 20 Feb 2024 10:21:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683366.1062837; Tue, 20 Feb 2024 10:21:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNFv-0008US-Mt; Tue, 20 Feb 2024 10:21:43 +0000
Received: by outflank-mailman (input) for mailman id 683366;
 Tue, 20 Feb 2024 10:21:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcNFu-0008UM-Fd
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 10:21:42 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcNFs-0007nw-SU; Tue, 20 Feb 2024 10:21:40 +0000
Received: from [15.248.2.238] (helo=[10.24.67.30])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcNFs-0005vo-Hk; Tue, 20 Feb 2024 10:21:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=lPQQ7pGZQgBX+xw8nWCo+MNrxRJU7VJJ+DNGT807UYo=; b=v8aoqx8okU3InnrFE2Me5trZ4X
	n4covUpZywuwf0SIUSd7LQrZKY6Ho0imO4e4VyVZQubaDvkKRawnHb0oJ7kCaHaTD5GDQI7G4o9xS
	KZgnJ21KvBlXOY/QvNhMHoiQTG02ysxYCV0JKE/T2xhImB4AEPlb/eTiNGVvvaek3KSY=;
Message-ID: <814e35c2-ce8b-4e04-b419-a79be9ee445a@xen.org>
Date: Tue, 20 Feb 2024 10:21:37 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 02/15] xen/arm: add initial support for LLC coloring on
 arm64
Content-Language: en-GB
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org,
 andrea.bastoni@minervasys.tech, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Marco Solieri <marco.solieri@minervasys.tech>
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-3-carlo.nonato@minervasys.tech>
 <bb28c8a2-c2bd-4e1d-ad28-9bc632e3e474@amd.com>
 <CAG+AhRXsZC7Pjdce42yYTvBy7MyjS1axrDGsY0U=o+APNyxdYA@mail.gmail.com>
 <5d2f6be2-020d-4003-ad47-b20ef07ffefc@xen.org>
 <CAG+AhRX96RBCc0n750K3t+c3j_fKQLrg1K5kLceHig401wS+Ng@mail.gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CAG+AhRX96RBCc0n750K3t+c3j_fKQLrg1K5kLceHig401wS+Ng@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Carlo,

On 20/02/2024 09:16, Carlo Nonato wrote:
> Hi Julien
> 
> On Tue, Feb 20, 2024 at 12:06 AM Julien Grall <julien@xen.org> wrote:
>>
>> Hi,
>>
>> On 14/02/2024 13:52, Carlo Nonato wrote:
>>> On Wed, Feb 14, 2024 at 11:14 AM Michal Orzel <michal.orzel@amd.com> wrote:
>>>>> diff --git a/xen/arch/arm/include/asm/processor.h b/xen/arch/arm/include/asm/processor.h
>>>>> index 8e02410465..336933ee62 100644
>>>>> --- a/xen/arch/arm/include/asm/processor.h
>>>>> +++ b/xen/arch/arm/include/asm/processor.h
>>>>> @@ -18,6 +18,22 @@
>>>>>    #define CTR_IDC_SHIFT       28
>>>>>    #define CTR_DIC_SHIFT       29
>>>>>
>>>>> +/* CCSIDR Current Cache Size ID Register */
>>>>> +#define CCSIDR_LINESIZE_MASK            _AC(0x7, ULL)
>>>> Why ULL and not UL? ccsidr is of register_t type
>>>
>>> Julien, while reviewing an earlier version:
>>>
>>>> Please use ULL here otherwise someone using MASK << SHIFT will have the
>>>> expected result.
>>>
>>> https://patchew.org/Xen/20220826125111.152261-1-carlo.nonato@minervasys.tech/20220826125111.152261-2-carlo.nonato@minervasys.tech/#08956082-c194-8bae-cb25-44e4e3227689@xen.org
>>
>> Michal is right. This should be UL. Not sure why I suggested ULL back
>> then. Sorry.
> 
> No problem.
> 
> If there aren't any other comments I will proceed with sending the v7.
> Do you guys want to add something on the arm part?

I haven't yet had the chance to fully review the series. It is in my 
TODO list though.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 10:28:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 10:28:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683372.1062847 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNMc-0000h6-Eh; Tue, 20 Feb 2024 10:28:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683372.1062847; Tue, 20 Feb 2024 10:28:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNMc-0000gz-BS; Tue, 20 Feb 2024 10:28:38 +0000
Received: by outflank-mailman (input) for mailman id 683372;
 Tue, 20 Feb 2024 10:28:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcNMa-0000gt-VW
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 10:28:36 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cd153205-cfda-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 11:28:34 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-512b29f82d1so2839925e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 02:28:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w3-20020a05600c014300b0040fccf7e8easm13989864wmm.36.2024.02.20.02.28.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 02:28:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd153205-cfda-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708424914; x=1709029714; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=feBmHTk5QqT0alfBhak8qeu8UsMRRHWdWJtG8l7EVRc=;
        b=AjKtysfHRRoer1u25vOu0lU/US0HHnYVFu3gQuHCz5ImmA7Xyyi597xAt7WYCMpuCj
         B0IYV9x5VtSlPxSD0RJLdI+ilyhz3UbSLgginzmg3kaRFmj96ky4b9bjy5HXRg0eYkSP
         LgZq0mILs1l6vFMmpcbKYCFQGRZjj01QWtZwV5vyfR8OMGZK2wzSd/NVxd811Cfgq3ur
         i4wuUI6IYo3y6oJG93qPOQem2pbUl0kx62MV/v1712f3gOoUIKQV73YyOkmUcjN5Eoy4
         raOSAyHtFeowGWk+bTKTAzi9YmpLrQJdiJxY0ny18/238J0XYpenPuhEsBNbHqF7qXUS
         chuw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708424914; x=1709029714;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=feBmHTk5QqT0alfBhak8qeu8UsMRRHWdWJtG8l7EVRc=;
        b=aEUXEKQI5mo2AykyboPk7geqqR4eI82D4e98AcQgnFGhjOPI+cb1CDsxK2sF+yyB5j
         w5ymXZ3krPI4N7f/36j2Cavsd5BJhy7XShtclxZvnhal6CzhkeW5tJANvX3f1VAve6gS
         jh3tya23cuWPvd1Vam9EXVfanAO3Sp0FZieo3gTZUdqiHB92K3kjIb8MVB4qpFEdVu17
         uRVP9/5CwnJZDdGubhdSKjFgm+0VnmVq15gt0ERofHsHYq0ttAzqqAIC6U97rm9X6ZKR
         8sMdCanKstZ2yUdOyVhMwRLqKK+SF5y3PAUI4J9CaS346e5FNLA2z1YbP53ktFneRj4y
         eiOA==
X-Forwarded-Encrypted: i=1; AJvYcCWwpLfF4sxdImVYdU8uVm/VzS0pWjS65WFOwEiVyiWQPZ+me3TJ4Oi+F9xWaAv2/zTPtDlBhk2aPi0YH0CgJTyKcrycKuqHH0jGG65eP3g=
X-Gm-Message-State: AOJu0YxxYEzuL870dCgrfWXUsise0HEqtPJztMJlGGo3wxvubr8cBIYK
	N9YnPDpGs0zNgtCSYzKK0ilVFGlk7qClv7dK1J4b5z8N6BhEreEsYnqBcsbAZQ==
X-Google-Smtp-Source: AGHT+IEKN6E3c/PKBoceaZsKS7pqEcQjszBO2aJk6zzkeYI9YSsAesCCIlc6pEXEsrrCNKNTcHg+yw==
X-Received: by 2002:a05:6512:3d04:b0:512:bf82:139f with SMTP id d4-20020a0565123d0400b00512bf82139fmr1801336lfv.25.1708424914148;
        Tue, 20 Feb 2024 02:28:34 -0800 (PST)
Message-ID: <7031861b-f925-479c-b236-8b1809996ede@suse.com>
Date: Tue, 20 Feb 2024 11:28:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 09/27] x86/pv: Rewrite how building PV dom0
 handles domheap mappings
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-10-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240116192611.41112-10-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.01.2024 20:25, Elias El Yandouzi wrote:
> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -382,6 +382,10 @@ int __init dom0_construct_pv(struct domain *d,
>      l3_pgentry_t *l3tab = NULL, *l3start = NULL;
>      l2_pgentry_t *l2tab = NULL, *l2start = NULL;
>      l1_pgentry_t *l1tab = NULL, *l1start = NULL;
> +    mfn_t l4start_mfn = INVALID_MFN;
> +    mfn_t l3start_mfn = INVALID_MFN;
> +    mfn_t l2start_mfn = INVALID_MFN;
> +    mfn_t l1start_mfn = INVALID_MFN;

The reason initializers are needed here is, aiui, the overly large scope
of these variables. For example ...

> @@ -708,22 +712,32 @@ int __init dom0_construct_pv(struct domain *d,
>          v->arch.pv.event_callback_cs    = FLAT_COMPAT_KERNEL_CS;
>      }
>  
> +#define UNMAP_MAP_AND_ADVANCE(mfn_var, virt_var, maddr) \
> +do {                                                    \
> +    unmap_domain_page(virt_var);                        \
> +    mfn_var = maddr_to_mfn(maddr);                      \
> +    maddr += PAGE_SIZE;                                 \
> +    virt_var = map_domain_page(mfn_var);                \
> +} while ( false )
> +
>      if ( !compat )
>      {
>          maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l4_page_table;
> -        l4start = l4tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
> +        UNMAP_MAP_AND_ADVANCE(l4start_mfn, l4start, mpt_alloc);
> +        l4tab = l4start;
>          clear_page(l4tab);
> -        init_xen_l4_slots(l4tab, _mfn(virt_to_mfn(l4start)),
> -                          d, INVALID_MFN, true);
> -        v->arch.guest_table = pagetable_from_paddr(__pa(l4start));
> +        init_xen_l4_slots(l4tab, l4start_mfn, d, INVALID_MFN, true);
> +        v->arch.guest_table = pagetable_from_mfn(l4start_mfn);

... looks to be required only here, while ...

>      }
>      else
>      {
>          /* Monitor table already created by switch_compat(). */
> -        l4start = l4tab = __va(pagetable_get_paddr(v->arch.guest_table));
> +        l4start_mfn = pagetable_get_mfn(v->arch.guest_table);
> +        l4start = l4tab = map_domain_page(l4start_mfn);

... in principle the use of the variable could be avoided here. Below
from here there's no further use of it.

> @@ -781,30 +797,34 @@ int __init dom0_construct_pv(struct domain *d,
>  
>      if ( compat )
>      {
> -        l2_pgentry_t *l2t;
> -
>          /* Ensure the first four L3 entries are all populated. */
>          for ( i = 0, l3tab = l3start; i < 4; ++i, ++l3tab )
>          {
>              if ( !l3e_get_intpte(*l3tab) )
>              {
>                  maddr_to_page(mpt_alloc)->u.inuse.type_info = PGT_l2_page_table;
> -                l2tab = __va(mpt_alloc); mpt_alloc += PAGE_SIZE;
> -                clear_page(l2tab);
> -                *l3tab = l3e_from_paddr(__pa(l2tab), L3_PROT);
> +                UNMAP_MAP_AND_ADVANCE(l2start_mfn, l2start, mpt_alloc);
> +                clear_page(l2start);
> +                *l3tab = l3e_from_mfn(l2start_mfn, L3_PROT);
>              }

The updating of l2start is only conditional here, yet ...

>              if ( i == 3 )
>                  l3e_get_page(*l3tab)->u.inuse.type_info |= PGT_pae_xen_l2;
>          }
>  
> -        l2t = map_l2t_from_l3e(l3start[3]);
> -        init_xen_pae_l2_slots(l2t, d);
> -        unmap_domain_page(l2t);
> +        init_xen_pae_l2_slots(l2start, d);

... here you assume it points at the page referenced by the 3rd L3 entry.
Question is why the original code is being replaced here in the first
place: It was already suitably mapping the page in question.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 10:38:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 10:38:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683378.1062857 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNVa-0002i0-9W; Tue, 20 Feb 2024 10:37:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683378.1062857; Tue, 20 Feb 2024 10:37:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNVa-0002ht-6H; Tue, 20 Feb 2024 10:37:54 +0000
Received: by outflank-mailman (input) for mailman id 683378;
 Tue, 20 Feb 2024 10:37:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcNVY-0002hn-Sj
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 10:37:52 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 190fa229-cfdc-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 11:37:51 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-412698ac6f9so8473265e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 02:37:51 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 fs14-20020a05600c3f8e00b00411a595d56bsm14037286wmb.14.2024.02.20.02.37.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 02:37:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 190fa229-cfdc-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708425471; x=1709030271; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6015tbHyN3AB4j82IiQ32hpH7Mru6U9Dq/uVwr4WFAI=;
        b=ClfA8PykBHXP7Z7AqjH0HHDbGLo6Mhl6ffyXIpJYdZWy21rHhB6Xv+eRVTLAEpOIRa
         udbdIvNqK13swXMAmBlc9E6LdyRn+PPrD41tFrcltb0yv6s+poLkhPxh405PzJqVM0gC
         K45r9PaU79gpiPtHyME8+hbVAPTJiY/yGAOfhqaOp/6Hmg41Y5XHqmbPl3txZ1nLkoWL
         zdDN0CW/lCvoR25ZCryj+A58+6lJPxYSg/wxkorZnIsDjXzYavTpvXZUVfMrkjbK44L2
         a/uZQfhwV93FuY2wdVv48fkqT6guWHsOzgin5itqqSoI3Z0HbhHryrXb1evz3V++s7Oe
         MbZg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708425471; x=1709030271;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6015tbHyN3AB4j82IiQ32hpH7Mru6U9Dq/uVwr4WFAI=;
        b=Uib8ih7PViuoNCqNPQLQ9pEbSsVih/ww5O+2kzWFKbYRYo31WlFhp380lVHsXablL2
         wse2++/Bidc6c+lioD7zRINHKMC5MWLQ4jUEU9tJ4hBasHVa2+t12+tF7Z9KrIMWLcWE
         Q73PrsknowpT0ch8R6bRYwCl4s6q7fKzsUUVvh8+bgbdAgVapzIy7kn89oG2LYm9N8Li
         0ItDwXwrw3i9K/aCVFTPU/gOxl8ESDd6aumNZjIXTYOZQwqB85VJvAG7yA4FX0zYBmC9
         2RzbzltyJvPJZfNIfDtDyRyLPSV64Ay4pLZmsU00/WoW9ZH6E8CEti803uq3gM8K410n
         X+7A==
X-Forwarded-Encrypted: i=1; AJvYcCVlRZx6ke5o3Wioe8zq5QSuu7M6Qe9om2GkoxV/MbFvTt+ScnOzfgS1TZfO56bK6ORcXS+dz1Pho4NHkZfFYgI9QLVq2hMuor2bXI+3gjo=
X-Gm-Message-State: AOJu0Ywos8497vCDE5JnJntWLVxjelfUlRsn71KJMe/OEB5jMYWXhcNK
	xjvmEEGQf0Rwpr/4pZ/G5j8YR9mC8zbYlpq5jzckwPz1G6PQEpUBy63da9zlZw==
X-Google-Smtp-Source: AGHT+IG8VCj6Q5ze/WUvaTpnbNapcrqoS+4gjGEYB7cL3AUIDz7jOn4wRaRCZsnmWfgPEAav3Tg2ng==
X-Received: by 2002:a05:600c:4f43:b0:411:e86d:85a3 with SMTP id m3-20020a05600c4f4300b00411e86d85a3mr16253163wmq.16.1708425471215;
        Tue, 20 Feb 2024 02:37:51 -0800 (PST)
Message-ID: <3fdbe1b6-59d0-4a33-8e7e-fb3f31a94fbf@suse.com>
Date: Tue, 20 Feb 2024 11:37:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 10/27] x86/pv: Map L4 page table for shim
 domain
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-11-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240116192611.41112-11-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.01.2024 20:25, Elias El Yandouzi wrote:
> From: Hongyan Xia <hongyxia@amazon.com>
> 
> The root page table is allocated from the domheap and isn't
> mapped by default. Map it on demand to build pv shim domain.
> 
> Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
> Signed-off-by: Elias El Yandouzi <eliasely@amazon.com>

The patch looks correct as is, so
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Still I would have wished that ...

> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -991,8 +991,12 @@ do {                                                    \
>       * !CONFIG_VIDEO case so the logic here can be simplified.
>       */
>      if ( pv_shim )
> +    {
> +        l4start = map_domain_page(l4start_mfn);
>          pv_shim_setup_dom(d, l4start, v_start, vxenstore_start, vconsole_start,
>                            vphysmap_start, si);
> +        UNMAP_DOMAIN_PAGE(l4start);
> +    }

... the function wide "l4start" wasn't clobbered like this.

In fact I think this patch needs either folding into the earlier one,
or moving ahead: The respective UNMAP_DOMAIN_PAGE() added there breaks
the use of l4start here. Yet then why not simply move that
UNMAP_DOMAIN_PAGE() below here, eliminating the need for this patch.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 10:45:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 10:45:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683385.1062867 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNct-0004pH-43; Tue, 20 Feb 2024 10:45:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683385.1062867; Tue, 20 Feb 2024 10:45:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNct-0004pA-1I; Tue, 20 Feb 2024 10:45:27 +0000
Received: by outflank-mailman (input) for mailman id 683385;
 Tue, 20 Feb 2024 10:45:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nvC9=J5=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rcNcr-0004p1-E0
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 10:45:25 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 26d5e8a2-cfdd-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 11:45:24 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-512a96e44e2so3450350e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 02:45:24 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 cs1-20020a0564020c4100b005617cebd051sm3531686edb.78.2024.02.20.02.45.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 02:45:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 26d5e8a2-cfdd-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708425924; x=1709030724; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=XeAtqMosQ2BTmzAb3KiVdCjPDLekNTwtpeI5gDo9Euk=;
        b=jy4a2sViA+ruDGkMl8DnAdhOF82VGTZCiJ34PL4iPMW3mD5Sj4DHHHpKxppt0sAkRj
         FPSxPx+AN+RJ/BG3HIGOGxTNTbxtj89EQOD8kAV3P4L1+W+iE9JVhrfDeGe0UG4F0acY
         zFzu4gi/2ZHUxvnx5vuCjMyl1KmAqRLIaAS71qWSFUc9Eu3Q6AvqjWXankeyP3aifV7j
         WqkCbVhRoe4jhiIzzoSxTAFNpaQ5IMUcLQdJof7YIO3qphnt7hy4ZkuNZ+YNOdO7hSPO
         wZL5cHk4pIRy0ko5NT6506HbkDyv4mej80moIMtGtl+RFAmZM/ilZ8dWLt/GlHBM5Pq3
         8mKQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708425924; x=1709030724;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=XeAtqMosQ2BTmzAb3KiVdCjPDLekNTwtpeI5gDo9Euk=;
        b=kYxmF9EvuEBd6lhc/L/4Ebstzi1DZ7b5RC5SN4ajHOcP9Bz/nqQFvd/f33svhtvWf+
         5lN9CtrDrf4swrCbxzp3ui6b4jOjHZWZHrgibRFm3ML9nXM4j0XQCct9wkteG/UjAmkR
         bvV5JmrtU7GbzWwonoNLkyKxp+7Si7U42yo/XHvh39hBIf900xlgzTqm51M4T+NQyTgD
         huNJVLVUvv+ZvUjkvWqpl8NXR+FAD3hafb5tQorkMx4jWyH7W3JpkK4VCNvW2dClX55E
         g66npyV4dsWxEZ5QitV4mMko8Pqk8NnllV1CsRQ/nyVFn5XzsCMMZqHnfnxmwdAEREYx
         QTzA==
X-Forwarded-Encrypted: i=1; AJvYcCVy3p2L8sjBIwEo6/HIAdt4lQZGjlzTvYP97oW+R9UlG1hYr/ehPx8DpFJvDHzbBZVG+Lf/A0sh/kpQEpu1pEc9iZW0JvYgG4ZjSCOqpwg=
X-Gm-Message-State: AOJu0YwUu0dJV8a3QUUy62SlRV2Zb2ZTypeoFAzu9p0s9EhDj5QIeqik
	FpI3+K2LwtXurmX+vEdcVZRhhuLTxZGDAJ2lim90PiDJH7Ql0D2u
X-Google-Smtp-Source: AGHT+IHc4cvjQ6hx+yVLRYVGLgvmVH/olkOmZzFEsfL6OPWz8qUEiRQAjUOcwpYNVLDFiN1EOxnhtg==
X-Received: by 2002:a05:6512:238c:b0:512:caa2:23da with SMTP id c12-20020a056512238c00b00512caa223damr653221lfv.22.1708425923461;
        Tue, 20 Feb 2024 02:45:23 -0800 (PST)
Message-ID: <806552bc18430cdc9e6dae9fa3321e4f77b22317.camel@gmail.com>
Subject: Re: [PATCH v9 7/7] xen/asm-generic: fold struct devarch into struct
 dev
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Rahul Singh <rahul.singh@arm.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Bertrand Marquis <bertrand.marquis@arm.com>, 
 Michal Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>
Date: Tue, 20 Feb 2024 11:45:22 +0100
In-Reply-To: <7a2ed74d-a773-4dc1-8331-a1ee28b17f4a@xen.org>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
	 <3a5bf394a9d95a28cecac996f6e0decb788c19fd.1708086092.git.oleksii.kurochko@gmail.com>
	 <7a2ed74d-a773-4dc1-8331-a1ee28b17f4a@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

Hi Julien,

On Mon, 2024-02-19 at 19:00 +0000, Julien Grall wrote:
> Hi,
>=20
> On 16/02/2024 12:39, Oleksii Kurochko wrote:
> > The current patch is a follow-up to the patch titled:
> > =C2=A0=C2=A0=C2=A0=C2=A0 xen/asm-generic: introduce generic device.h
> > Also, a prerequisite for this patch is, without which a compilation
> > error will occur:
> > =C2=A0=C2=A0=C2=A0=C2=A0 xen/arm: switch Arm to use asm-generic/device.=
h
> >=20
> > The 'struct dev_archdata' is exclusively used within 'struct
> > device',
> > so it could be merged into 'struct device.'
> >=20
> > After the merger, it is necessary to update the 'dev_archdata()'
> > macros and the comments above 'struct arm_smmu_xen_device' in
> > drivers/passthrough/arm/smmu.c.
> > Additionally, it is required to update instances of
> > "dev->archdata->iommu" to "dev->iommu".
> >=20
> > Suggested-by: Julien Grall <julien@xen.org>
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > =C2=A0=C2=A0 This patch can be merged with patches 4 and 5 of this patc=
h
> > series.
>=20
> I am a bit puzzled with this comment. If this is the case, then why
> was=20
> it done in a separate patch?
>=20
> I know I suggested to create the separate patch but this was only in
> the=20
> case you decided to handle it after this series is merged. So this=20
> should have been merged when sending. Maybe I should have been
> clearer.
I can submit a new version of the patch series in which this patch will
be incorporated into patches 4 and 5, respectively.

In this case, should all the "reviewed-by" and "acked-by" tags for
patch 4 be removed?

>=20
> Anyway, regardless that:
>=20
> Reviewed-by: Julien Grall <jgrall@amazon.com>
Thanks.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 10:46:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 10:46:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683390.1062877 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNeA-0005Kv-ET; Tue, 20 Feb 2024 10:46:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683390.1062877; Tue, 20 Feb 2024 10:46:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNeA-0005Ko-BG; Tue, 20 Feb 2024 10:46:46 +0000
Received: by outflank-mailman (input) for mailman id 683390;
 Tue, 20 Feb 2024 10:46:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcNeA-0005Kg-0o
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 10:46:46 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 56d9420d-cfdd-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 11:46:44 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-512bd533be0so2011006e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 02:46:44 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 t22-20020a7bc3d6000000b0040d5ae2906esm14112971wmj.30.2024.02.20.02.46.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 02:46:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 56d9420d-cfdd-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708426004; x=1709030804; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dQcOrtuylbWlfTWy7gczAjM/7k/LP7uMPpGxr5wNk0Y=;
        b=YtH/cIdtINxIqOUd7Qw3Y0iV4znWNNHRMRiA3xYX20vCksKkwWUXB0vkVKwjJZztz9
         G9EEBgQrvyxLbgXJzbF4NB1tIW6RnOmXQjzUhLQTORZaVayx3nQBcOc68vMLj2OxaDPn
         +YDKrLm3WkX3tfbTuiC8ZrQUQyALZaIW2YGYDqywJZs0YQXxK0j6mFXHwZHMsaT1Q0hz
         A/jfrm+d9WWoSuFNHZIi4Sn7oT0B/Dx+7mn6Q8lk0d5mudIlW9+oofiyZEZEHgsRus5z
         imieaxNtY1XrxRdLM7RMh5Ct3vHlcUjyGHcAcb4WEaOv3mElKAcr5l/wdufBDSohDs23
         gO/A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708426004; x=1709030804;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dQcOrtuylbWlfTWy7gczAjM/7k/LP7uMPpGxr5wNk0Y=;
        b=HJ/fVpBjCrxmV/6rlb4zfwC03Edyii2GBd5XRvtn7nhkcRmf6vPwQ5tWXXl2MX49/Z
         x37uktUZppo7akq2AiJYZhdkAtIvetqdVgTyM1kC7fdXOrc7Jws3FkILx8fPmUrs6Gki
         q3Z4xJAC+fXFfR40aBo/zQvoYvc/J5PIb8W0tQyjhX2MknaRY2NnPxbMKVJDzWkET7Rr
         vkRkc0caTy+099+KII/841IpIpq+tGY8JeS/GOBZ9QEpVbwaEiQYRD7IP3vtEpDBp57v
         sOrVpaagbi/rUWcKcOO4NIE4n8F4sHlYs5j9pdBZ80PeyyXHs3W6cKCq1gIYCM86DINs
         jAsQ==
X-Forwarded-Encrypted: i=1; AJvYcCUrXcqhv2gExU7tCJejdMWCpw3TkCM91nWb9GXPdLO8HG2GVv41K/yJXfah5K5dX0PuKCUWxH7hFu+T5+aL9Ki2Y/Qh1rFT/ZSelc3TYHE=
X-Gm-Message-State: AOJu0Yx1LoDcMVOJbiI5FlfSSig26YoGFgOIJLnuVCYedYeSOGOJUpg7
	3/F8XD5ZyEcETBjX+Xyu0Mui9s7uFKlV6f5c8588WfdYKcRGfEo572POipFDDw==
X-Google-Smtp-Source: AGHT+IEnOfoAYwiGeiHYmc7AhHgRvCnVWzG9BwbqYNLwlhWn8jA0qa/qXNLLcOOqDAkxrhACkecOEw==
X-Received: by 2002:a05:6512:3f10:b0:512:b57d:10da with SMTP id y16-20020a0565123f1000b00512b57d10damr3349218lfa.19.1708426004469;
        Tue, 20 Feb 2024 02:46:44 -0800 (PST)
Message-ID: <1ff19cb2-851d-40cf-bb12-5b7074eee0b8@suse.com>
Date: Tue, 20 Feb 2024 11:46:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 11/27] x86: Lift mapcache variable to the arch
 level
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Wei Liu <wei.liu2@citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Wei Wang <wawei@amazon.de>, Hongyan Xia <hongyxia@amazon.com>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-12-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240116192611.41112-12-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.01.2024 20:25, Elias El Yandouzi wrote:
> From: Wei Liu <wei.liu2@citrix.com>
> 
> It is going to be needed by HVM and idle domain as well, because without
> the direct map, both need a mapcache to map pages.
> 
> This only lifts the mapcache variable up. Whether we populate the
> mapcache for a domain is unchanged in this patch.

Is it? I wonder because of ...

> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -843,6 +843,8 @@ int arch_domain_create(struct domain *d,
>  
>      psr_domain_init(d);
>  
> +    mapcache_domain_init(d);
> +
>      if ( is_hvm_domain(d) )
>      {
>          if ( (rc = hvm_domain_initialise(d, config)) != 0 )
> @@ -850,8 +852,6 @@ int arch_domain_create(struct domain *d,
>      }
>      else if ( is_pv_domain(d) )
>      {
> -        mapcache_domain_init(d);
> -
>          if ( (rc = pv_domain_initialise(d)) != 0 )
>              goto fail;
>      }

... this and ...

> --- a/xen/arch/x86/domain_page.c
> +++ b/xen/arch/x86/domain_page.c
> @@ -82,11 +82,11 @@ void *map_domain_page(mfn_t mfn)
>  #endif
>  
>      v = mapcache_current_vcpu();
> -    if ( !v || !is_pv_vcpu(v) )
> +    if ( !v )
>          return mfn_to_virt(mfn_x(mfn));

... this and yet more changes indicating otherwise.

Yet if which domains have a mapcache set up is already changed here, I
wonder whether the idle domain shouldn't be taken care of here as well.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 10:50:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 10:50:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683396.1062887 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNhT-00077b-R2; Tue, 20 Feb 2024 10:50:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683396.1062887; Tue, 20 Feb 2024 10:50:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNhT-00077U-OB; Tue, 20 Feb 2024 10:50:11 +0000
Received: by outflank-mailman (input) for mailman id 683396;
 Tue, 20 Feb 2024 10:50:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcNhS-00077O-Pi
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 10:50:10 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcNhR-0008Ku-E5; Tue, 20 Feb 2024 10:50:09 +0000
Received: from [15.248.2.238] (helo=[10.24.67.30])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcNhR-0007Wl-53; Tue, 20 Feb 2024 10:50:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=HQ3snlpb44n8vXTacFp+yT86OWJqDU3WzFqVPtSdIvQ=; b=b2e6J3QYRhEVnOcLS9eicOBRyZ
	AAYWWPFxAZzU7KkO9o1cMYqYXWr8V+6etBxquVQYwhktrsfdrtpe51j+CjHQaG1WhzCANaY4tFKIP
	jtU0tRrwWjRI+biAEpgKA/Y41rfM/X+RtnL6BnAcsbOHdUzY8mXGNS4pWe31cjVvhT38=;
Message-ID: <4fa5953d-99b5-40fe-a15b-c58afc4a38cf@xen.org>
Date: Tue, 20 Feb 2024 10:50:06 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 7/7] xen/asm-generic: fold struct devarch into struct
 dev
Content-Language: en-GB
To: Oleksii <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
Cc: Rahul Singh <rahul.singh@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
 <3a5bf394a9d95a28cecac996f6e0decb788c19fd.1708086092.git.oleksii.kurochko@gmail.com>
 <7a2ed74d-a773-4dc1-8331-a1ee28b17f4a@xen.org>
 <806552bc18430cdc9e6dae9fa3321e4f77b22317.camel@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <806552bc18430cdc9e6dae9fa3321e4f77b22317.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Oleksii,

On 20/02/2024 10:45, Oleksii wrote:
> Hi Julien,
> 
> On Mon, 2024-02-19 at 19:00 +0000, Julien Grall wrote:
>> Hi,
>>
>> On 16/02/2024 12:39, Oleksii Kurochko wrote:
>>> The current patch is a follow-up to the patch titled:
>>>       xen/asm-generic: introduce generic device.h
>>> Also, a prerequisite for this patch is, without which a compilation
>>> error will occur:
>>>       xen/arm: switch Arm to use asm-generic/device.h
>>>
>>> The 'struct dev_archdata' is exclusively used within 'struct
>>> device',
>>> so it could be merged into 'struct device.'
>>>
>>> After the merger, it is necessary to update the 'dev_archdata()'
>>> macros and the comments above 'struct arm_smmu_xen_device' in
>>> drivers/passthrough/arm/smmu.c.
>>> Additionally, it is required to update instances of
>>> "dev->archdata->iommu" to "dev->iommu".
>>>
>>> Suggested-by: Julien Grall <julien@xen.org>
>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>> ---
>>>     This patch can be merged with patches 4 and 5 of this patch
>>> series.
>>
>> I am a bit puzzled with this comment. If this is the case, then why
>> was
>> it done in a separate patch?
>>
>> I know I suggested to create the separate patch but this was only in
>> the
>> case you decided to handle it after this series is merged. So this
>> should have been merged when sending. Maybe I should have been
>> clearer.
> I can submit a new version of the patch series in which this patch will
> be incorporated into patches 4 and 5, respectively.

AFAICT the series is fully acked. So no need to send a new version. But 
please keep the request in mind for future series.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 10:51:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 10:51:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683401.1062897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNix-0007zO-4L; Tue, 20 Feb 2024 10:51:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683401.1062897; Tue, 20 Feb 2024 10:51:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNix-0007zH-1R; Tue, 20 Feb 2024 10:51:43 +0000
Received: by outflank-mailman (input) for mailman id 683401;
 Tue, 20 Feb 2024 10:51:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcNiv-0007yc-Ip
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 10:51:41 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 06c0d0c9-cfde-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 11:51:40 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-4125cf71eecso20302215e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 02:51:39 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f2-20020adfe902000000b0033cf24700e5sm12988432wrm.39.2024.02.20.02.51.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 02:51:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 06c0d0c9-cfde-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708426299; x=1709031099; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=7UqccP2mQ1b+ZJFBfNd4qO5pYXo5mGjE9XWtCnkcxxs=;
        b=OxE757AX+xPX6ADrwW9NvIaaUkMeZgh9g6VtQqkXyIJK2r7ENafG59H9v6iG5aBX5X
         dkbuZvICDBHg0WeIOrc8+QRJnvedpnsEbyJucIbyALBGg0cnhyitsQoiw3oNT5LSqvNG
         VuDDUQKsiKID7Ja02cwSN/mptlMXfcGYJNfHAxU/fNO1vLP3/4gqbtmg31IWu8+nDwAI
         dSkpzPNfnJDkoyjZ+5ZXtV3h+gpBr4f8p1pVWL9RdLOt66lnW27UIT0Nfi4pz841rLfc
         PuUWmJ4jDolOjCsfI8PIljcqy7LQmxbbNkgOWKchJf0M31RH2iY29z3I7a9eDg0vgPVH
         jmWQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708426299; x=1709031099;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=7UqccP2mQ1b+ZJFBfNd4qO5pYXo5mGjE9XWtCnkcxxs=;
        b=dg/7Y1jZPvZnDU+8kLJpCW8rLoyY/nWCEoh2bX2rY6ql/7hHN/UCb7/9xBg88HDvzy
         NbG8WYUP9UOkittfzvwR/L1vh86HoAUBUgV8QxYsJgqPM8uUkHXxW1Jwvdrc5rYl7Et3
         TP/4lKpw6XvRry5ycPCdrD4FeIW9MxLlCOpMrL7cCWleeqZDZJE6Ah99NEYDyl0oASH4
         BOSfSTsTXPFlljsqtm4vLHZMNLj3pA/clVbB3315oxuLyEsbg0P1O4LoyPGL/TE7ifHg
         Ua3ABUZ3frwpgskYK+kNfJS854Kw43RWzF3+RRHgxgqR4hjQ1EYY9u04Sk6+jlrlq3pg
         XefQ==
X-Forwarded-Encrypted: i=1; AJvYcCVHstE2hTmb8UJGs1BOllBtFJihCfFZ1u9gUKYkC6ijb/Cdq/QToyO2PkLHHW555TQwK+om7CvLEjvhGiCyYK1mF1dEbzQ46jTHZQHRgdA=
X-Gm-Message-State: AOJu0YxHaoqYQNlFkj1ZlYc+u+hA3Ee43CSmutD+RWR5iZKhyxM6Qiu2
	TRP+FOvwFnqBRcXMxhXmuG8Gy64RoHyEJfq6K2yHgrmrLUakyAHbjMjoENaJyA==
X-Google-Smtp-Source: AGHT+IGJz2eK1iykqgvFOIHk+kqHGI4VZX5H1+joMZevFUUJHDk1Y+RP36Q6qtE2FJFpRxyHWRkGJA==
X-Received: by 2002:adf:f052:0:b0:33d:4cf7:5e65 with SMTP id t18-20020adff052000000b0033d4cf75e65mr3613771wro.16.1708426299441;
        Tue, 20 Feb 2024 02:51:39 -0800 (PST)
Message-ID: <d6f3993e-5e96-4e3d-9334-9b44152f9f81@suse.com>
Date: Tue, 20 Feb 2024 11:51:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 12/27] x86/mapcache: Initialise the mapcache
 for the idle domain
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Wei Liu <wl@xen.org>, Wei Wang <wawei@amazon.de>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-13-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240116192611.41112-13-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.01.2024 20:25, Elias El Yandouzi wrote:
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -750,9 +750,16 @@ int arch_domain_create(struct domain *d,
>  
>      spin_lock_init(&d->arch.e820_lock);
>  
> +    if ( (rc = mapcache_domain_init(d)) != 0)
> +    {
> +        free_perdomain_mappings(d);
> +        return rc;
> +    }
> +
>      /* Minimal initialisation for the idle domain. */
>      if ( unlikely(is_idle_domain(d)) )
>      {
> +        struct page_info *pg = d->arch.perdomain_l3_pg;
>          static const struct arch_csw idle_csw = {
>              .from = paravirt_ctxt_switch_from,
>              .to   = paravirt_ctxt_switch_to,
> @@ -763,6 +770,9 @@ int arch_domain_create(struct domain *d,
>  
>          d->arch.cpu_policy = ZERO_BLOCK_PTR; /* Catch stray misuses. */
>  
> +        idle_pg_table[l4_table_offset(PERDOMAIN_VIRT_START)] =
> +            l4e_from_page(pg, __PAGE_HYPERVISOR_RW);
> +
>          return 0;
>      }

Why not add another call to mapcache_domain_init() right here, allowing
a more specific panic() to be invoked in case of failure (compared to
the BUG_ON() upon failure of creation of the idle domain as a whole)?
Then the other mapcache_domain_init() call doesn't need moving a 2nd
time in close succession.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 10:59:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 10:59:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683410.1062907 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNq7-0000Ch-Qw; Tue, 20 Feb 2024 10:59:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683410.1062907; Tue, 20 Feb 2024 10:59:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcNq7-0000Ca-OC; Tue, 20 Feb 2024 10:59:07 +0000
Received: by outflank-mailman (input) for mailman id 683410;
 Tue, 20 Feb 2024 10:59:06 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcNq6-0000CQ-FW; Tue, 20 Feb 2024 10:59:06 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcNq6-0008Tn-3m; Tue, 20 Feb 2024 10:59:06 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcNq5-0002L2-Pp; Tue, 20 Feb 2024 10:59:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcNq5-0004Wt-PO; Tue, 20 Feb 2024 10:59:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=m5lFFOb0utTMfiTTZOv4lPn4UZOX/j7jhhTI+Nk1mGI=; b=QhZ6l3VZ4Of176wzjP+fzQDXnc
	f2p3ja87eeSPFSniDfPI20Ip9GgC32F05fyJlkq486wZvp4iA3UHtoV1Dr2qVDv/Ui9GTAboGR9V3
	PVjRCUa1PirtIijFiulJml2jkS38m2vPAG8/4Gi8sUi37+6RKQuMDJ2zeDwQwv0l1QsI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184708-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [libvirt test] 184708: tolerable FAIL - PUSHED
X-Osstest-Failures:
    libvirt:test-armhf-armhf-libvirt-qcow2:guest-start/debian.repeat:fail:heisenbug
    libvirt:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    libvirt:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    libvirt:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    libvirt=c8ca9d0118ec21dbff07147e0bbffcd19fb196b7
X-Osstest-Versions-That:
    libvirt=8d9e1f771834f924b41c4dd93f1fde327aef18ff
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 20 Feb 2024 10:59:05 +0000

flight 184708 libvirt real [real]
flight 184710 libvirt real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184708/
http://logs.test-lab.xenproject.org/osstest/logs/184710/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-qcow2 17 guest-start/debian.repeat fail pass in 184710-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184689
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184689
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184689
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-qcow2 15 saverestore-support-check    fail never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass

version targeted for testing:
 libvirt              c8ca9d0118ec21dbff07147e0bbffcd19fb196b7
baseline version:
 libvirt              8d9e1f771834f924b41c4dd93f1fde327aef18ff

Last test of basis   184689  2024-02-17 04:20:47 Z    3 days
Testing same since   184708  2024-02-20 04:18:53 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrea Bolognani <abologna@redhat.com>
  Jiri Denemark <jdenemar@redhat.com>
  Martin Kletzander <mkletzan@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-libvirt                                     pass    
 test-arm64-arm64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-arm64-arm64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               fail    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-libvirt-vhd                                 pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   8d9e1f7718..c8ca9d0118  c8ca9d0118ec21dbff07147e0bbffcd19fb196b7 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 11:14:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 11:14:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683418.1062916 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcO4p-0004Cc-41; Tue, 20 Feb 2024 11:14:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683418.1062916; Tue, 20 Feb 2024 11:14:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcO4p-0004CV-1L; Tue, 20 Feb 2024 11:14:19 +0000
Received: by outflank-mailman (input) for mailman id 683418;
 Tue, 20 Feb 2024 11:14:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcO4o-0004CP-6V
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 11:14:18 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2efa3004-cfe1-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 12:14:15 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-512b700c8ebso2377811e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 03:14:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g8-20020a05600c310800b004126ec2f541sm1659435wmo.0.2024.02.20.03.14.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 03:14:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2efa3004-cfe1-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708427655; x=1709032455; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=0fu04UrW6KIZ54U/Q1q80D52Btmv1ea4nslaOFbkl/g=;
        b=FjG80fYBk7Buz2NLQYdy0skaD+HsPayoB845cnjqSYyI5RgRpVgpLMqqOJK5ZrfGi5
         KisHdm6zcUlv4VKnDuPXlVzk+UE+6WJLNJx9TY8JB5rHeZelcFput9hbwOs8NK6bgy3e
         iS5jAGdJVqx7E2TJ3T9rfC8S+CRT6WIAAO/q7wSSjz5NJIobRy7F6f8pZg/web+BshPK
         05UdQ0dEtvju/p/wk/XNzwd11hNfWMLdKKy1j5qwDMBmeuQVje5X+1tPsezP936DTfH2
         CnC1SoYzzY1p00OebG6OSZRb0LRmJ2KEXCEVhz+M/42dRqRXM7Z4MxzwWvbmu0FELcMe
         O/3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708427655; x=1709032455;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0fu04UrW6KIZ54U/Q1q80D52Btmv1ea4nslaOFbkl/g=;
        b=ZC6i9qOZLyvNdtwxtsVo2y4M75RfmIcg+Wln6aC0tPA4iJCLQ9twECefPZLIuYg/sr
         Mtb4+u8eU31MoroNv84A12v6ZJiq1qixs+iycuT0dV7b39zb5MjqfzbT780cvIxYIymY
         xMg5kg0+bEY3Wqe/lqyW7H7Y+AH110JTySYbW7BP85uztISwHDTHeYCsMSjJeJ9p0M1Q
         ewwvwimuvnNj3BMeZg+X15zKXFXKUbJ2G1SZgH7E3nRxsrg56AoVjMMet1GD4jKEvGOm
         QI0TiWMl1gIBdnjd3d6D97C3MTzEUoKJEABDgpQHKSs5sNyM3r2xwsaReiHdZbgpb8LH
         c2Eg==
X-Forwarded-Encrypted: i=1; AJvYcCX6e6jrmJwl9ckUVqwOirzNmfRedGyMOzsY3mQQr6N5fx0+ZBcEC+AHoecyCtm+hmdIPP4LmnajOJzY9zur8jWInzI6bc5SmWoj38oyyX0=
X-Gm-Message-State: AOJu0YzKLoGpyVsHct0QoPahjPnGYYaKhWgxTm9zp2AXSwiWr1Ugx0zq
	+wvQpRY5kUNiGiyMPHVXU9hnq2Peo75ZZzVdFau+iIS448Z3WoOwCNPa0pq92w==
X-Google-Smtp-Source: AGHT+IHgC3z+xH5eaKqRnifHT+sdO2QvDpPieF5X5R2FmW1NPUuYd/oHQ7DpBTG6NilLo16OpEWxoA==
X-Received: by 2002:a19:f610:0:b0:512:b915:95d4 with SMTP id x16-20020a19f610000000b00512b91595d4mr2977384lfe.61.1708427655294;
        Tue, 20 Feb 2024 03:14:15 -0800 (PST)
Message-ID: <18795bde-bfd5-41ab-bef5-f74819bae956@suse.com>
Date: Tue, 20 Feb 2024 12:14:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 (resend) 13/27] x86: Add a boot option to enable and
 disable the direct map
Content-Language: en-US
To: Elias El Yandouzi <eliasely@amazon.com>
Cc: julien@xen.org, pdurrant@amazon.com, dwmw@amazon.com,
 Hongyan Xia <hongyxia@amazon.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Julien Grall <jgrall@amazon.com>, xen-devel@lists.xenproject.org
References: <20240116192611.41112-1-eliasely@amazon.com>
 <20240116192611.41112-14-eliasely@amazon.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240116192611.41112-14-eliasely@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.01.2024 20:25, Elias El Yandouzi wrote:
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -799,6 +799,18 @@ that enabling this option cannot guarantee anything beyond what underlying
>  hardware guarantees (with, where available and known to Xen, respective
>  tweaks applied).
>  
> +### directmap (x86)
> +> `= <boolean>`
> +
> +> Default: `true`
> +
> +Enable or disable the direct map region in Xen.
> +
> +By default, Xen creates the direct map region which maps physical memory
> +in that region. Setting this to no will remove the direct map, blocking
> +exploits that leak secrets via speculative memory access in the direct
> +map.

I think this wants wording such that the full truth is conveyed: The directmap
doesn't disappear. It's merely only sparsely populated then.

> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -29,6 +29,7 @@ config X86
>  	select HAS_UBSAN
>  	select HAS_VPCI if HVM
>  	select NEEDS_LIBELF
> +	select HAS_SECRET_HIDING

Please respect alphabetic sorting. As to "secret hiding" - personally I
consider this too generic a term. This is about limiting the direct map. Why
not name the option then accordingly?

> --- a/xen/arch/x86/include/asm/mm.h
> +++ b/xen/arch/x86/include/asm/mm.h
> @@ -620,10 +620,18 @@ void write_32bit_pse_identmap(uint32_t *l2);
>  /*
>   * x86 maps part of physical memory via the directmap region.
>   * Return whether the range of MFN falls in the directmap region.
> + *
> + * When boot command line sets directmap=no, we will not have a direct map at
> + * all so this will always return false.
>   */

As with the command line doc, please state the full truth.

>  static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
>  {
> -    unsigned long eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END);
> +    unsigned long eva;
> +
> +    if ( !has_directmap() )
> +        return false;

Hmm. The sole user of this function is init_node_heap(). Would it perhaps make
sense to simply map the indicated number of pages then? init_node_heap() would
fall back to xmalloc(), so the data will be in what's left of the directmap
anyway.

> +    eva = min(DIRECTMAP_VIRT_END, HYPERVISOR_VIRT_END);

Irrespective I don't see a need to replace the initializer by an assignment.

> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -83,6 +83,23 @@ config HAS_UBSAN
>  config MEM_ACCESS_ALWAYS_ON
>  	bool
>  
> +config HAS_SECRET_HIDING
> +	bool

This again wants placing suitably among the other HAS_*.

> +config SECRET_HIDING
> +    bool "Secret hiding"
> +    depends on HAS_SECRET_HIDING
> +    ---help---
> +    The directmap contains mapping for most of the RAM which makes domain
> +    memory easily accessible. While making the performance better, it also makes
> +    the hypervisor more vulnerable to speculation attacks.
> +
> +    Enabling this feature will allow the user to decide whether the memory
> +    is always mapped at boot or mapped only on demand (see the command line
> +    option "directmap").
> +
> +    If unsure, say N.

Nit: Indentation and no ---help--- anymore (just help) please in new Kconfig
entries.

Also as an alternative did you consider making this new setting merely
control the default of opt_directmap? Otherwise the variable shouldn't exist
at all when the Kconfig option is off, but rather be #define-d to "true" in
that case.

> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -165,6 +165,13 @@ extern unsigned long max_page;
>  extern unsigned long total_pages;
>  extern paddr_t mem_hotplug;
>  
> +extern bool opt_directmap;
> +
> +static inline bool has_directmap(void)
> +{
> +    return opt_directmap;
> +}

If opt_directmap isn't static, I see little point in having such a wrapper.
If there are reasons, I think they want stating in the description.

On the whole: Is the placement of this patch in the series an indication
that as of here all directmap uses have gone away? If so, what's the rest of
the series about? Alternatively isn't use of this option still problematic
at this point of the series? Whichever way it is - this wants clarifying in
the description.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 11:18:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 11:18:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683423.1062928 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcO8y-0004mR-Lw; Tue, 20 Feb 2024 11:18:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683423.1062928; Tue, 20 Feb 2024 11:18:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcO8y-0004mK-Hg; Tue, 20 Feb 2024 11:18:36 +0000
Received: by outflank-mailman (input) for mailman id 683423;
 Tue, 20 Feb 2024 11:18:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5vNk=J5=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rcO8x-0004mE-6t
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 11:18:35 +0000
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com
 [2a00:1450:4864:20::329])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c826ed33-cfe1-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 12:18:33 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id
 5b1f17b1804b1-40fb3b5893eso29571785e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 03:18:33 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 c5-20020a5d4f05000000b0033b684d6d5csm13206583wru.20.2024.02.20.03.18.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 03:18:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c826ed33-cfe1-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708427912; x=1709032712; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=XQWief9wkdRpDTRDFY5HHvLHLfu6vYANbFCesfEoTaI=;
        b=eT3FJZyIb7AsbKTkMaVq8PjZ3I25GCTFSGf2ZYmI/RhjNPr+ibzYo2n9S8gufsQGVl
         zoe6V3rj2/TXkBEqJDPV6FKhPC298qKwtbQKp//Oc3aQusOUXJ7xB+GqbWp8U18lPV6D
         nlUN9TJbo1XPZ+LDUKSfsLLJMUlIndPBmzEJU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708427912; x=1709032712;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=XQWief9wkdRpDTRDFY5HHvLHLfu6vYANbFCesfEoTaI=;
        b=RIbneO4X/tXxqYYH8yWDnAiangbNJiO2kc/B1VmaW2OtwQHsCfuj2BKTviPXwTzMMJ
         Yc4HEtoDn08rz7FJq6R5qJu/E1yNRydS7Kj5awEH6RfGf4oppR7/zfNpThXBIH4kixsn
         bHmgYZBpHAeglCvMVLtFD2XXUYjCyVz2TBLy9lpKHjsIOg600Lw4NTG/nQRhI58U84Nj
         GBJHPJtQPfcfTrvI5V0k0lGri2Amfe+oHy02JweEz1WBJcC1pYkHTQVvgIGtcu9nj8ZA
         SOtZvJaf6KtFsfA6NSltQB2xEfJMDMXanAgw35LNVe8C/AqpVXlfFtgyK4J25wg10LUb
         Ry6A==
X-Gm-Message-State: AOJu0Ywfjz9PafUw5dGRJ8Zqtk2L6BHjKl7P0gBQv9TrkzKA3/YEDDu7
	gmOj61smhLloTdcxZK32ss9sLvCNnN82JrUvjaK7SCEU6m1e4yiWu5NWl/PjNpQ=
X-Google-Smtp-Source: AGHT+IFJuoE7Ny2aPZu+noNSiySkulsQ5z5YltJbJG4fRmbptWgxkdCh4rtXGehG1jHiaLMwz35KDw==
X-Received: by 2002:a5d:530f:0:b0:33d:20d3:b876 with SMTP id e15-20020a5d530f000000b0033d20d3b876mr6914057wrv.34.1708427912398;
        Tue, 20 Feb 2024 03:18:32 -0800 (PST)
Date: Tue, 20 Feb 2024 11:18:31 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Subject: Re: [PATCH] build: make sure build fails when running kconfig fails
Message-ID: <b7c22718-ec6c-470e-be72-e613b2af3c54@perard>
References: <55c3a781-17f8-47f2-9629-515e1aea77aa@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <55c3a781-17f8-47f2-9629-515e1aea77aa@suse.com>

On Tue, Feb 20, 2024 at 09:43:56AM +0100, Jan Beulich wrote:
> Because of using "-include", failure to (re)build auto.conf (with
> auto.conf.cmd produced as a secondary target) won't stop make from
> continuing the build. Arrange for it being possible to drop the - from
> Rules.mk, requiring that the include be skipped for tools-only targets.
> Note that relying on the inclusion in those cases wouldn't be correct
> anyway, as it might be a stale file (yet to be rebuilt) which would be
> included, while during initial build, the file would be absent
> altogether.
> 
> Fixes: 8d4c17a90b0a ("xen/build: silence make warnings about missing auto.conf*")
> Reported-by: Roger Pau Monn <roger.pau@citrix.com>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Just to make sure, this patch is a workaround to a harmless bug in older
version of GNU Make which print spurious error messages, and said bug as
been fixed in GNU Make 4.2, right? Bug report:

    bug #102: Make prints an incorrect error for missing includes
    https://savannah.gnu.org/bugs/?func=detailitem&item_id=102

> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
>  # This exploits the 'multi-target pattern rule' trick.
>  # The syncconfig should be executed only once to make all the targets.
>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
> +	$(Q)rm -f include/config/$*.conf

Maybe this should say "include/config/auto.conf" instead of using "$*".
"syncconfig" is going to generate "auto.conf" and not "$*.conf". And it
would make easier to find the "rm" command via "grep".

>  	$(Q)$(MAKE) $(build)=tools/kconfig syncconfig
>  

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 11:26:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 11:26:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683432.1062937 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcOGQ-0006rZ-Ch; Tue, 20 Feb 2024 11:26:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683432.1062937; Tue, 20 Feb 2024 11:26:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcOGQ-0006rS-9f; Tue, 20 Feb 2024 11:26:18 +0000
Received: by outflank-mailman (input) for mailman id 683432;
 Tue, 20 Feb 2024 11:26:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcOGO-0006rM-M4
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 11:26:16 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id db229a04-cfe2-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 12:26:14 +0100 (CET)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a3e5d82ad86so301506366b.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 03:26:14 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ss1-20020a170907c00100b00a3e0c07c1a0sm3744849ejc.23.2024.02.20.03.26.13
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 03:26:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: db229a04-cfe2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708428374; x=1709033174; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=83H65JBn/OBpg4tMqx6OXYsljmCtU8NTWb2zbPb1FwU=;
        b=QuBTKTbWULyjYz75xHO20H3CXa0oS8RC9roQi+WdnXVQxQILGUuTdaI7jBW/PgEYrd
         VW0iJC7mQzboEEtW7g2/KluNiMsGHsRge5BcZFpxnLedwei7A83hY3DxsuwaYYSKgBL3
         SqrZqQXjM54QOcYLpiCOf/hFofLwMD0hOT2opH9F4HPqW3HzIXo7cqni+IF+FsOJh4C8
         XKkH3LHyjTu7qvcbKo1y7EqynhuYpH5iuU72eriezUPkcXZKajg8sknJXN0qxuOMeEYo
         v4y323AQygobCPeFHV3TCAL5Dj8q6uTu9x3qTNX1/amcCE5wKZcJ7DzNB3At4VTjbgv2
         48JQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708428374; x=1709033174;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=83H65JBn/OBpg4tMqx6OXYsljmCtU8NTWb2zbPb1FwU=;
        b=Jdq98MAOC9xxDreoaTBtHk1E0XGZA7JfcH9woL3erETbVt/YDpvyY0JSST98Qu6jIz
         ivWAslgjFb6WTQhOy9GCbfNCH7inQov1amo/WJ1GjpbDPVv3OWM9HtamchHjaf3f9yll
         5c2UbMF9diJmoG1Cq73NLbw6gP9IO5Tk5T86OEkKZTwYcnvjv55E8oENS4ur2YD+dhJ7
         eeRfBRxAkVVd9/cH7ALoqwoSREHzQxECHeJcAtDcQvs7phQziG0PYYmM1288jCfv2a9i
         PapWq0MktuCb9kUFw43UVzrhBMT2HQS2ijtQymqEDRoaRyZqJfKSCGyfL2XyjrB0vH0M
         0WuA==
X-Forwarded-Encrypted: i=1; AJvYcCWSmc4aVNiYNUcZUN/hpX6KLnLmmLdBzaQmgSu0yRC3/+8D5zD2dseJKn92DqngUJGi+KtabUzzAm35bqWgvWPHv5myn3UtOhSxg0AmAJA=
X-Gm-Message-State: AOJu0Yx7EFJAhGTJl0dFwTg3RPq2cFw5xYcgq2HFVjGdqDMNoJjEavbb
	/alQmxFI1GyLiXYWqCJvaWdINwRqp4lsgWMy6PYVz9CofDrBHNysEvkEzfYOEw==
X-Google-Smtp-Source: AGHT+IEJLVOBLUPgvXtZEOn/8NHcVgQez3FHKSRORA/I40EAwtjFl9wMuz0SpFRwfaMyH98/dPh+Dg==
X-Received: by 2002:a17:906:344c:b0:a3f:cfb:9464 with SMTP id d12-20020a170906344c00b00a3f0cfb9464mr787774ejb.21.1708428373740;
        Tue, 20 Feb 2024 03:26:13 -0800 (PST)
Message-ID: <d70ff55d-e3fd-4c76-a053-617f42036295@suse.com>
Date: Tue, 20 Feb 2024 12:26:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 7/7] xen/asm-generic: fold struct devarch into struct
 dev
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Julien Grall <julien@xen.org>, Rahul Singh <rahul.singh@arm.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
 <3a5bf394a9d95a28cecac996f6e0decb788c19fd.1708086092.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3a5bf394a9d95a28cecac996f6e0decb788c19fd.1708086092.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 16.02.2024 13:39, Oleksii Kurochko wrote:
> The current patch is a follow-up to the patch titled:
>     xen/asm-generic: introduce generic device.h
> Also, a prerequisite for this patch is, without which a compilation
> error will occur:
>     xen/arm: switch Arm to use asm-generic/device.h

I've dropped this while committing; it belongs ...

> The 'struct dev_archdata' is exclusively used within 'struct device',
> so it could be merged into 'struct device.'
> 
> After the merger, it is necessary to update the 'dev_archdata()'
> macros and the comments above 'struct arm_smmu_xen_device' in
> drivers/passthrough/arm/smmu.c.
> Additionally, it is required to update instances of
> "dev->archdata->iommu" to "dev->iommu".
> 
> Suggested-by: Julien Grall <julien@xen.org>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>   This patch can be merged with patches 4 and 5 of this patch series.

... somewhere here.

I didn't touch patch 5's subject, but I think the duplicate "Arm" in
there would better have been avoided.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 11:40:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 11:40:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683441.1062958 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcOTk-0001Us-NT; Tue, 20 Feb 2024 11:40:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683441.1062958; Tue, 20 Feb 2024 11:40:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcOTk-0001Ui-KL; Tue, 20 Feb 2024 11:40:04 +0000
Received: by outflank-mailman (input) for mailman id 683441;
 Tue, 20 Feb 2024 11:40:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qAG8=J5=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rcOTj-0001D9-C7
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 11:40:03 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c8c1f4a2-cfe4-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 12:40:02 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-512b29f82d1so2925491e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 03:40:02 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 jv24-20020a05622aa09800b0042dce775a4bsm3386041qtb.3.2024.02.20.03.40.01
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 03:40:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c8c1f4a2-cfe4-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708429202; x=1709034002; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=IGlZzmWp6xz09mlJ1wyOOX/OEeaoUYzGIvASoAaf8N8=;
        b=IQIWO/c0mwTWqMRCz4W8tXmpG0EYhY7eaDC8JgvDrTcIOimuYagBmVAZ3/a/uETKl7
         pwPAZLVJmzP4YP+zILR9eUmEimbadScTAXUcGnCCd9ygjYPWkKbZA2D5Dk8i0Sjww8MK
         /bzQlxhI0D/DzpPJ3E3CHkXaw874qppi994vk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708429202; x=1709034002;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IGlZzmWp6xz09mlJ1wyOOX/OEeaoUYzGIvASoAaf8N8=;
        b=K9OUyNRBBk2fegYBpjqtLFpqRb9CLXDxGIw+GGr7KGLlIHp60feXDArzb49MeuWvUv
         hf+cwormNxLTJ3XgDqb9s7B++Kvt3BnquSEY4kp9TftYdB5UHq2YFr0+2gZaKih+R9zd
         rqEjHMh0tmArTEwVtuUvp769N0KS8i4vlqatzfjNj22U+MuKyzVrgL2eYL3Khz4zEpo5
         yklzhab2oGbd3Sy6lgCcE3KCAhrtA8OAlBQYT59z/1Uh7hMybZa75m+XZUiwkkhcaASi
         Ssc+L9xf1z8bxzhAg4zZCWvujuZXVy6OBL0lVhKyKjKMvwZVOMUZrEaOvFqdFCILICaD
         Owqg==
X-Forwarded-Encrypted: i=1; AJvYcCXpLHeBEdXniBFWmQv4boJRe8Bh6I/I+SIxTOiTw9s9GODsV9TKrjGpNxlp7oVSFbjuhkEqNCz/WoFSOeXA0HK8LkAunwDcI9fRUWh/lJw=
X-Gm-Message-State: AOJu0Ywg0/fdmNW5r+M2AP7hChBvUCz3Lmt5uuP4hCCDVXUtu6pTNlbN
	bq2P9z5WteE1qlS7FLLNQslWiKmAsUxKx98NKtTuJixQF23kU4Ny8DvzFTl6xpQ=
X-Google-Smtp-Source: AGHT+IHbnU1LpWt6trJIlUs3WRM52Tu3ynjqRt6lHOWcYzwBooYpKnWVVL35CFpQMvX5fPKx7bXULg==
X-Received: by 2002:a19:4304:0:b0:512:b3ef:79c6 with SMTP id q4-20020a194304000000b00512b3ef79c6mr3048275lfa.40.1708429201878;
        Tue, 20 Feb 2024 03:40:01 -0800 (PST)
Date: Tue, 20 Feb 2024 12:39:59 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Christoph Hellwig <hch@lst.de>
Cc: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Jens Axboe <axboe@kernel.dk>, xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: Re: [PATCH 1/4] xen-blkfront: set max_discard/secure erase limits to
 UINT_MAX
Message-ID: <ZdSPj32Ww80nKQhM@macbook>
References: <20240220084935.3282351-1-hch@lst.de>
 <20240220084935.3282351-2-hch@lst.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240220084935.3282351-2-hch@lst.de>

On Tue, Feb 20, 2024 at 09:49:32AM +0100, Christoph Hellwig wrote:
> Currently xen-blkfront set the max discard limit to the capacity of
> the device, which is suboptimal when the capacity changes.  Just set
> it to UINT_MAX, which has the same effect except and is simpler.

Extra 'except' in the line above?

> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 11:50:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 11:50:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683447.1062969 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcOdZ-0004GM-Kc; Tue, 20 Feb 2024 11:50:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683447.1062969; Tue, 20 Feb 2024 11:50:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcOdZ-0004GF-Hl; Tue, 20 Feb 2024 11:50:13 +0000
Received: by outflank-mailman (input) for mailman id 683447;
 Tue, 20 Feb 2024 11:50:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qAG8=J5=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rcOdY-0004G9-4J
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 11:50:12 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 32f539c5-cfe6-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 12:50:10 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-512a96e44e2so3529016e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 03:50:10 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 b7-20020a05622a020700b0042e124c7b17sm1679254qtx.60.2024.02.20.03.50.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 03:50:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 32f539c5-cfe6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708429809; x=1709034609; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=L48l2hpsjGRvCKL00qVAvQZynCjle73QSMbq61LuMbw=;
        b=blvXx2ouFz8rNtgKIPFPmuziy5reAgb4hpQ56V+z/LFv9VdDooKHSr6DjtJuQHMWOE
         gUFZy4fO3NYonSEwV4ixcXuwiMll1BDJoJ5JKjIB8Y4nmCu/wYZeu1ea+A3y20xlTFgP
         8YCr9wuitod44Wg54Q+6fUN4NUixoLo/sQyzw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708429809; x=1709034609;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=L48l2hpsjGRvCKL00qVAvQZynCjle73QSMbq61LuMbw=;
        b=PbMbiPFG71eY4LP+7M9eJ40qVmhJo16KxJ4jD3XEhvLhRuJZYbsfoOQWAzKIRj/aHg
         hTW4nLPQqfUj/Ydbtcid69UoRje+IeMweesd3bPMmr+0J+WbfSkOXcMSabLVk2kKLqHd
         C7CNANpFYf3SX24D6y+xGsvrCQ4sJzMz8PN3yAdxRfLLUsvY6Ks6oTNeN396scTKYA3n
         rdw3VROxDivA4yrme5DmuCMTj7l3/ogNyfpwwhpUHjfrTxYySeCEK5AB8Md0PQHSur8C
         gDoOBcQ+AMXZWC2p8Z8IlBL6onOh48tO3GcR+cEMtJ/JXpwhxWiknzLQiAzJ/w1tJoiA
         NVBA==
X-Forwarded-Encrypted: i=1; AJvYcCXTNwAZCfdwOmJcqmRPXZqp2TUub/BEnIWNcS8ZdxJURzdd8htmtc1+umMmDexWO5smuPLtceyfvmN07T2sId0vSj3z5WVRb2MYFfQnByo=
X-Gm-Message-State: AOJu0YxlvgqL8NSQugxMj3bnW8zBYYLru847swqn/zaVhZED2PNvdazV
	0tub3weH3AM3N+yxGU4lNHn5CIy1DQO8OxW6jlocvSR6VraZnMh6s3UlF4kipkw=
X-Google-Smtp-Source: AGHT+IHUZ8iQeWiuL3aFeiidpOcUFxCSR6QyScwRy5qnJiOguqCxJ4LpXQwdFCaOwM+JubG1vE+kxQ==
X-Received: by 2002:a05:6512:1088:b0:512:c9bc:f491 with SMTP id j8-20020a056512108800b00512c9bcf491mr971397lfg.47.1708429809506;
        Tue, 20 Feb 2024 03:50:09 -0800 (PST)
Date: Tue, 20 Feb 2024 12:50:07 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Christoph Hellwig <hch@lst.de>
Cc: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Jens Axboe <axboe@kernel.dk>, xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: Re: [PATCH 2/4] xen-blkfront: rely on the default discard granularity
Message-ID: <ZdSR75QHen-omYCZ@macbook>
References: <20240220084935.3282351-1-hch@lst.de>
 <20240220084935.3282351-3-hch@lst.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240220084935.3282351-3-hch@lst.de>

On Tue, Feb 20, 2024 at 09:49:33AM +0100, Christoph Hellwig wrote:
> The block layer now sets the discard granularity to the physical
> block size default.  Take advantage of that in xen-blkfront and only
> set the discard granularity if explicitly specified.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Acked-by: Roger Pau Monné <roger.apu@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 11:52:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 11:52:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683452.1062979 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcOfi-0004qT-4W; Tue, 20 Feb 2024 11:52:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683452.1062979; Tue, 20 Feb 2024 11:52:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcOfi-0004qM-0e; Tue, 20 Feb 2024 11:52:26 +0000
Received: by outflank-mailman (input) for mailman id 683452;
 Tue, 20 Feb 2024 11:52:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcOfg-0004qE-Bs
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 11:52:24 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcOfe-000106-OE; Tue, 20 Feb 2024 11:52:22 +0000
Received: from [15.248.2.238] (helo=[10.24.67.30])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcOfe-0005IK-I8; Tue, 20 Feb 2024 11:52:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=wbrEDILVp4grnBFXiGbeWqL9LdN5bNqLHXmk79SVRTU=; b=5sulXFrXBVy9DjZEF1NvBCx09n
	fuQXRycHoQyQ4qMNQgY1Ecp0hUeDTNYCRkUmlHiiQrdCNun+NjMO6gdly+ZJeNWmMYyVSn5Ort2Zk
	aZpmW5rKVTeb8dqUht9lPwVW4x7dyWJq8cns5x7h7Uy9nqkdwrQMXznQ2PNkT3MzEgaM=;
Message-ID: <a2348c5f-39ce-42f3-ad2a-a5edf88352e9@xen.org>
Date: Tue, 20 Feb 2024 11:52:20 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] almost fully ignore zero-size flush requests
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
 <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com>
 <7dd78c97-b82f-4ae1-a257-98f612c6bad2@xen.org>
 <6466155e-9883-4743-a897-ef565f14fdf0@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <6466155e-9883-4743-a897-ef565f14fdf0@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 20/02/2024 08:26, Jan Beulich wrote:
> On 19.02.2024 23:22, Julien Grall wrote:
>> Title: I would add 'gnttab:' to clarify which subsystem you are modifying.
> 
> That's how I actually have it here; it's not clear to me why I lost the
> prefix when sending.
> 
>> On 05/02/2024 11:03, Jan Beulich wrote:
>>> Along the line with observations in the context of XSA-448, besides
>>> "op" no field is relevant when the range to be flushed is empty, much
>>> like e.g. the pointers passed to memcpy() are irrelevant (and would
>>> never be "validated") when the passed length is zero. Split the existing
>>> condition validating "op", "offset", and "length", leaving only the "op"
>>> part ahead of the check for length being zero (or no flushing to be
>>> performed).
>>
>> I am probably missing something here. I understand the theory behind
>> reducing the number of checks when len == 0. But an OS cannot rely on it:
>>     1) older hypervisor would still return an error if the check doesn't
>> pass)
> 
> Right, but that's no reason to keep the bogus earlier behavior.

Hmmm... I am not sure why you say the behavior is bogus. From the commit 
message, it seems this is just an optimization that have side effect 
(ignoring the other fields).

> 
>>     2) it does feel odd to allow "invalid" offset when len == 0 (at least.
> 
> I'm puzzled: You've given R-b for patch 1 (thanks), where exactly the
> same reasoning is used, i.e. similarly referring to memcpy() to
> justify the (new / supposed) behavior.

I realize it. But I viewed it slightly different as you are adding the 
check. I think it is a good idea to add the check and ideally it should 
be after.

Here you don't seem to add any check and only re-order it. Hence why I 
treated it differently.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 12:16:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 12:16:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683462.1062989 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP2j-00007z-52; Tue, 20 Feb 2024 12:16:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683462.1062989; Tue, 20 Feb 2024 12:16:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP2j-00007k-0T; Tue, 20 Feb 2024 12:16:13 +0000
Received: by outflank-mailman (input) for mailman id 683462;
 Tue, 20 Feb 2024 12:16:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qAG8=J5=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rcP2h-00007e-Nk
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 12:16:11 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d3d976e2-cfe9-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 13:16:08 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-50eac018059so7937125e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 04:16:08 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 mf18-20020a0562145d9200b0068d05b81fb2sm476529qvb.15.2024.02.20.04.16.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 04:16:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d3d976e2-cfe9-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708431368; x=1709036168; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=knwiP9OAWZECA9KKubxtxSMpv+I26MM185zuFGNZSsA=;
        b=Lab359Edx5L/CRxynNw//tA/0dS5KmbZAYgSsojymFQ1/2GYZXOEjhv3etSMCujyFi
         9ReUYvo5NkGl7sv5gHJ+2T/adKXjtd84zTMTDgsBuMJ7U5p6YOLHeGPLZ20xnReMYkOV
         xAmM1XrVa2yBLlwpLaKHsSl/jHIWWbczIGFV0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708431368; x=1709036168;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=knwiP9OAWZECA9KKubxtxSMpv+I26MM185zuFGNZSsA=;
        b=JaPNZlIrc58luI0UMbNSig/Sd4a36IgJjWF6FBKr82innb5l8mxEJRepxcrxtBGcQf
         UMST1iV/TpyUaMW9/554BzYSLDj2FXnXTk75y+sTdwIkxskXTLhEyv3kRcASdH0KLt0s
         T0TOTFCJ+QR+HvtVKPH9ui/GBQ1qlOSsbKRfyfWmFXk3j31N4suAe4AdtlymGqi504v9
         M2pjB1tr3ibsVnN5OCHsAli/wzJevByTTY2ULKIjyQXUIHAxPw9oqfBOm07Vsg4UrECx
         OnXiWJm2Rb73mwzcJTXhjpnzMFyYkBiWeywB/kx8LEsPifc5jnxZoIQYJgo6l2YgB8J2
         mBaQ==
X-Forwarded-Encrypted: i=1; AJvYcCXe1Q4srKzUq0qlJeDUGJHmdLBdZ1sZgPHoQPzMuxDyqUgTSx5GNybH7RuVPbdX0t+rXhFCzmV14YCtXUB2Vm06M9hwyPVhfbpz4mUxBmI=
X-Gm-Message-State: AOJu0Yyc/w/O4/wMl1XtwHX4lnmSOhpin4R14lH05z7tFJqWNopQ9kp2
	4M/EAIaP+ESdmBv1S5EWN91+Xdb5iGgNJGaBZCd+pdI1Tq79nk+gey7ujVb5Edk=
X-Google-Smtp-Source: AGHT+IFpY0vxmPHLW+oy/ASL3Cytglfrlsy0I7GgUOvlkTMsZmHHJteUe3+LWEIK13qALwuImAYeVA==
X-Received: by 2002:a05:6512:12c7:b0:512:bae7:30f0 with SMTP id p7-20020a05651212c700b00512bae730f0mr3484188lfg.51.1708431367923;
        Tue, 20 Feb 2024 04:16:07 -0800 (PST)
Date: Tue, 20 Feb 2024 13:16:05 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Christoph Hellwig <hch@lst.de>
Cc: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Jens Axboe <axboe@kernel.dk>, xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: Re: [PATCH 3/4] xen-blkfront: don't redundantly set max_sements in
 blkif_recover
Message-ID: <ZdSYBQB-F1CwjoHN@macbook>
References: <20240220084935.3282351-1-hch@lst.de>
 <20240220084935.3282351-4-hch@lst.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240220084935.3282351-4-hch@lst.de>

On Tue, Feb 20, 2024 at 09:49:34AM +0100, Christoph Hellwig wrote:
> blkif_set_queue_limits already sets the max_sements limits, so don't do
> it a second time.  Also remove a comment about a long fixe bug in
> blk_mq_update_nr_hw_queues.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 12:18:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 12:18:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683466.1062999 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP4T-0000ea-FX; Tue, 20 Feb 2024 12:18:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683466.1062999; Tue, 20 Feb 2024 12:18:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP4T-0000eT-C6; Tue, 20 Feb 2024 12:18:01 +0000
Received: by outflank-mailman (input) for mailman id 683466;
 Tue, 20 Feb 2024 12:18:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J7zC=J5=amd.com=ayan.kumar.halder@srs-se1.protection.inumbo.net>)
 id 1rcP4S-0000eL-QP
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 12:18:00 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 157ce610-cfea-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 13:17:59 +0100 (CET)
Received: from DS7PR05CA0029.namprd05.prod.outlook.com (2603:10b6:5:3b9::34)
 by CYXPR12MB9320.namprd12.prod.outlook.com (2603:10b6:930:e6::9) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.19; Tue, 20 Feb
 2024 12:17:55 +0000
Received: from CY4PEPF0000EDD5.namprd03.prod.outlook.com
 (2603:10b6:5:3b9:cafe::1d) by DS7PR05CA0029.outlook.office365.com
 (2603:10b6:5:3b9::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend
 Transport; Tue, 20 Feb 2024 12:17:55 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EDD5.mail.protection.outlook.com (10.167.241.209) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 20 Feb 2024 12:17:54 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 20 Feb
 2024 06:17:53 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 20 Feb
 2024 06:17:53 -0600
Received: from xcbayankuma40.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via
 Frontend Transport; Tue, 20 Feb 2024 06:17:52 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 157ce610-cfea-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WmVkHSQIw+2w3QcGzaThV0wUB+nIV1QkonRVbaqYRIr9JW9XnGI2ovl1ZrGhyJITwEsR8phkXHxqj2rbCOlk9x13QhAeJySmajZJmM3pYlPFrD7N057Ih6u6UhQRPBDuV+8cME9mduh1outpPDrXcaJGvVhdqDgfqPKomxh5KWZbcQvVIiwTTrmKjsuWNhgNdkShhhfmkUKHCXsT1It0U8ILe72Cytk8geBamlVu63cO5x+wLJE7cmcBI/+C1s7fY0M0ZFBwS2rxtq49s3Q7bAWHEBIAJlxTBPkLP1y6r5LkwX5sQLN0IFq5BU62EzvusnIGm0Wf/su2521J7FZ3sQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=2rF9Ud1ZH+JBkCn35IfybOAiJd4oKJew/66Pz/2BAls=;
 b=KiRilViUOcl5Xs/tqsGte64v3R7sjyuHtADIG6ZoNCm4qtXrFMIXonOQNM84ZbXeUFO5cE1o9JLB8s8rP7IeVXmTjo1lAi1hCFvbEATpYpdOmTkkIL0oOpwdoTowDkfB1iCBZsSX3CxhihCvUH9dEHtxUtPNbfZhYe1XIVmOgPLRKHYjZwQ+Qi7DMT8f5KFDOhhgiCMbmdBINHJHVepEolGUCF13LjK4/RGjnRpzBb9ORhxmQy16k0gRSKCE7EYY0m9Dz96BNrZnrbF6sXiOs1POKWSqKRttPjDpB9xEu3/Hgw4fApyfkdnUb1HriWIuMDEqnvS+2f6z0Y8W7pVyPg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=2rF9Ud1ZH+JBkCn35IfybOAiJd4oKJew/66Pz/2BAls=;
 b=g+r0jQfdSGNbzA2cXV4l3TAbVTguSj6ipo8Zvzzm7IGCBn87MO3xP0nbkB5SD4xo7KHCNJRQ9fyLt9FRtDY7fJRUhWkEYIx/QaXR3bglESV5cu7pOV+rBs5XdOaIHpGisG8+LZmsbf8IK0bJdoEjFNH1EtbJ5V6KouapdIhYnAU=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <stefano.stabellini@amd.com>, <julien@xen.org>,
	<Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>,
	<michal.orzel@amd.com>, Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Subject: [XEN v5 0/3] xen/arm: Add emulation of Debug Data Transfer Registers
Date: Tue, 20 Feb 2024 12:17:40 +0000
Message-ID: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD5:EE_|CYXPR12MB9320:EE_
X-MS-Office365-Filtering-Correlation-Id: c78b7dbe-bdc8-4b9d-30e6-08dc320df78c
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	wIdDu81Lxs3qrxDYBIVez2J0YS1nW4IFeJ//yP/ToGUtAY3LM2Bf5GilM11w0uLjazMZyrb11+f0VyTvhM14CKw8uGSJ58s9BVrRw/acm3vmaIPYTJq32KroQB/9XnE3/FskGwXDBe6OvQgvKeWCdUssPvrpHSLU61BCLjNy1gqqAaZCw+Lui0ktnRjG/z4dbhxW9LetnVkrc2DMRSq+fdXJmBhRJmPxDs8jJ62HRXvDZmsIEOElX35y+TGjD96GN4jT8MkPOsiuc1aDhg5QHxOPmch+N9QXqFt5yZ4Ai0QQX50smNnSQ2x9DmrNqD3IDZkDS4pQ/p51BP+phShUF0bWuC5P6cqRAWxaSZhqDMsrdf96cM4z44zaGeb9mSxn4D8TtR9cJn63Tu99BZTwAQiI4MqEU5ZThDFN0dWSyDQplSbcfshUao8DlebVjSyyt3AqzDQsLy8I9cJQbHxqICF6RqnQN5cCctIDp7pQLtbabgnomyWwjFYfpn5OhY9XMCuCZNet2neuzLRX40Ktit2kmBHJfSEx07I4EcbFhvGL27ETknb8wq3Sth5+l7qIsHAcNw6LzaUk1laUd6SQXBKIzGEeimJOn/bEgPKh0gXdmioPm3B+WjVDIsnbLmpbpPvmCe+bzvna3mDRaPUGQQ==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(46966006)(40470700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2024 12:17:54.7797
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c78b7dbe-bdc8-4b9d-30e6-08dc320df78c
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EDD5.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR12MB9320

Hi,

Refer https://lore.kernel.org/all/alpine.DEB.2.22.394.2312071341540.1265976@ubuntu-linux-20-04-desktop/T/
for the previous discussion on this issue.

Also, the linux earlycon hvc driver has been fixed.
See https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty.git/commit/?h=tty-next&id=0ec058ece2f933aed66b76bd5cb9b5e6764853c3

Changes from v1:-
1. Split the change across 3 patches as per the design consensus.

v2 :-
1. Reordered the patches.

v3 :- 
1. Change mentioned in individual patches.

v4 :-
1. Change mentioned in individual patches.

Ayan Kumar Halder (2):
  xen/arm: Introduce CONFIG_PARTIAL_EMULATION and "partial-emulation"
    cmd option
  xen/arm: arm32: Add emulation of Debug Data Transfer Registers

Michal Orzel (1):
  xen/arm: arm64: Add emulation of Debug Data Transfer Registers

 SUPPORT.md                           | 12 +++++
 docs/misc/xen-command-line.pandoc    | 11 +++++
 xen/arch/arm/Kconfig                 |  9 ++++
 xen/arch/arm/arm64/vsysreg.c         | 68 +++++++++++++++++++---------
 xen/arch/arm/include/asm/arm64/hsr.h |  3 ++
 xen/arch/arm/include/asm/cpregs.h    |  2 +
 xen/arch/arm/include/asm/traps.h     |  6 +++
 xen/arch/arm/traps.c                 |  9 ++++
 xen/arch/arm/vcpreg.c                | 63 ++++++++++++++++++--------
 9 files changed, 144 insertions(+), 39 deletions(-)

-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 12:18:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 12:18:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683467.1063009 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP4a-0000vS-NR; Tue, 20 Feb 2024 12:18:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683467.1063009; Tue, 20 Feb 2024 12:18:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP4a-0000vL-Ja; Tue, 20 Feb 2024 12:18:08 +0000
Received: by outflank-mailman (input) for mailman id 683467;
 Tue, 20 Feb 2024 12:18:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J7zC=J5=amd.com=ayan.kumar.halder@srs-se1.protection.inumbo.net>)
 id 1rcP4Z-0000eL-6q
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 12:18:07 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2060d.outbound.protection.outlook.com
 [2a01:111:f400:7e88::60d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 19998b1e-cfea-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 13:18:06 +0100 (CET)
Received: from DS7PR05CA0018.namprd05.prod.outlook.com (2603:10b6:5:3b9::23)
 by DS0PR12MB8815.namprd12.prod.outlook.com (2603:10b6:8:14f::16) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.19; Tue, 20 Feb
 2024 12:18:03 +0000
Received: from CY4PEPF0000EDD5.namprd03.prod.outlook.com
 (2603:10b6:5:3b9:cafe::65) by DS7PR05CA0018.outlook.office365.com
 (2603:10b6:5:3b9::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.32 via Frontend
 Transport; Tue, 20 Feb 2024 12:18:03 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EDD5.mail.protection.outlook.com (10.167.241.209) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 20 Feb 2024 12:18:02 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 20 Feb
 2024 06:18:01 -0600
Received: from xcbayankuma40.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via
 Frontend Transport; Tue, 20 Feb 2024 06:18:00 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19998b1e-cfea-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dT6qtKa1jn2HtlsPJY3QrZixOO6HpPIQOk9L/HFzls2TfJaMb2DZfs161H6Wgy5j3zlGG1SprKm7mdGOknqmEqj06bulb3PrGP88FLLBspvT55UKRZzFz/zyX+IAYJ+nJBsD103qlx9V7thUPl4YwhkVS0QZW/ewXVtjc2BP24OuX6xSi3Mm/aNdqdzhSrbNjcKhEl3OeT4NDunJWSYygh3ZVPjtMCx8XbZ/MDpRJEMfKipLjpOCXM2WK84jAREvJN0AVoEu3WFpIzke2Jaedl4DXphlJT8MGRk/xigDUmQLGKmvVY7nH35q1PKCmi/k9gHVDGTiiWe/Fmjfa9pCbg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=EETal9k4inYY30L6I7esCkTiZAi5uczUE8BepjuzslE=;
 b=HlzR6PCqALmq8oAtDX1J8PnEaKCMTMm5P8nZzioJnTtvzd0N9yqEUVwcWQhINxhaphs9Fv96MCfF3frwCcyQ57mkPJ1FgCJ4nW7kWDxued/e5rOPD8up/NuVk5iSUboCLB8W7Xdea+L+/wiBXX0AKDyHoXGDv1uDJd812/nVehYq4xT/YSu1vVEioVmE/mbZkAANxibpv1X8DbClv3QYerspBjyrRBD4X0EC5d370Tgd5nCLHWVzDgmwuDuzGfb1jmGUNsECaopaMGhUNk04ogUhRxI78R2a/pkeqyKRhjI10zjumohPLOgI2Pqjw/snKRSH4IKsbu0aFq8Q+bn7Vw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=EETal9k4inYY30L6I7esCkTiZAi5uczUE8BepjuzslE=;
 b=HDAL8Gh49RH+c1UjsiJT07FgZYCz32mIHvGEEIn0Z0SSt+bZcLQOTf+b78kG4gS6R3aeoKvEJQUwhW/yqDI1G2DcevHQde+1c+DpoLBUI/hBQDG+CklbKBSNON2WhYUSbvr+FUs9fmN12GpPwfJgG+G16gmJeycUt6AVpkJitH4=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <stefano.stabellini@amd.com>, <julien@xen.org>,
	<Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>,
	<michal.orzel@amd.com>, Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Subject: [XEN v5 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and "partial-emulation" cmd option
Date: Tue, 20 Feb 2024 12:17:41 +0000
Message-ID: <20240220121743.3680715-2-ayan.kumar.halder@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD5:EE_|DS0PR12MB8815:EE_
X-MS-Office365-Filtering-Correlation-Id: c1916796-d8fe-4ce8-20a5-08dc320dfc6b
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	we1UyW+4gi0X5d2+VXFcmabEEwPPHq/c+XQR4xwMzGsQjS4YrTIX7x+AjlHo2KmlwtstXGhgi7hONjEr63kyFoaHiGt2eA9ornWm0dGVCVOP3XdZtpd/IL3Bt9toTx8j9cUaHuXu4f1rtCVnEMJQ9tLoKWfzYq8avZR4uPdVzOZnAIhQgtftrNnRF+euIf3WshI3b9OCKhGAtg10sBZdCGQeh91B0pSw7z9hD2x0fJXEEBmU3qt7ja/4WAqn0LBMSniXXyhmN9AKFuR84dl0iXZy0OYIdu9DVLs2sdbBGZiy1+KOC8MCkwMECAxzvGoHMCzzgYgqfF2kB+xL6tBKqj4iUYVJySa58ncZdkryVO/WLes21BnBn1CipsW9bPMy4O/9purFbzqlo4cCe4owhC9GRPsrEhZQ9Bb8XMYmvIxAwAogbbowFZeguSqtwYk3NUxX/sALvXlyesYrlNULaabaMz1CyFXAfNv4+hkM47c1M+Cjps1F2bRINcJ8dGME78z/Ug6b4EMUhiK019OU1jSLpek5NdXNXHley9nQma5d8C5jc3ciI3i5eHZE6gqwqJVLGn74dL4ha+komapbFIaPLfbP2retrQBwCTnTmSRzRZHj2Ikq+O6Hg2BMSLojKEjOmeas6pY6EJhCaxNn/iCsQXuaYQDoGMFfF+Yxo7k=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(230273577357003)(36860700004)(46966006)(40470700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2024 12:18:02.9673
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c1916796-d8fe-4ce8-20a5-08dc320dfc6b
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EDD5.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8815

There can be situations when the registers cannot be emulated to their full
functionality. This can be due to the complexity involved. In such cases, one
can emulate those registers as RAZ/WI for example. We call them as partial
emulation.

Some registers are non-optional and as such there is nothing preventing an OS
from accessing them.
Instead of injecting undefined exception (thus crashing a guest), one may want
to prefer a partial emulation to let the guest running (in some cases accepting
the fact that it might result in unwanted behavior).

A suitable example of this (as seen in subsequent patches) is emulation of
DBGDTRTX_EL0 (on Arm64) and DBGDTRTXINT(on Arm32). These non-optional
registers can be emulated as RAZ/WI and they can be enclosed within
CONFIG_PARTIAL_EMULATION.

Further, "partial-emulation" command line option allows us to
enable/disable partial emulation at run time. While CONFIG_PARTIAL_EMULATION
enables support for partial emulation at compile time (i.e. adds code for
partial emulation), this option may be enabled or disabled by Yocto or other
build systems. However if the build system turns this option on, users
can use scripts like Imagebuilder to generate uboot-script which will append
"partial-emulation=true" to xen command line to turn on the partial emulation.
Thus, it helps to avoid rebuilding xen.

By default, "CONFIG_PARTIAL_EMULATION=y" and "partial-emulation=false".
This is done so that Xen supports partial emulation. However, customers are
fully aware when they enable partial emulation. It's important to note that
enabling such support might result in unwanted/non-spec compliant behavior.

Added a note in SUPPORT.md to clarify the security support for partial
emulation.

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---

Changes from v1 :-
1. New patch introduced in v2.

v2 :-
1. Reordered the patches so that the config and command line option is
introduced in the first patch.

v3 :-
1. Defined a macro 'partial_emulation' to reduce if-defs.
2. Fixed style issues.

v4 :-
1. Added a note in SUPPORT.md.

 SUPPORT.md                        | 12 ++++++++++++
 docs/misc/xen-command-line.pandoc | 11 +++++++++++
 xen/arch/arm/Kconfig              |  9 +++++++++
 xen/arch/arm/include/asm/traps.h  |  6 ++++++
 xen/arch/arm/traps.c              |  9 +++++++++
 5 files changed, 47 insertions(+)

diff --git a/SUPPORT.md b/SUPPORT.md
index a90d1108c9..037578b10c 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -101,6 +101,18 @@ Extension to the GICv3 interrupt controller to support MSI.
 
     Status: Experimental
 
+### ARM/Partial Emulation
+
+Enable partial emulation of registers, otherwise considered unimplemented,
+that would normally trigger a fault injection.
+
+    Status: Supported, with caveats
+
+Bugs allowing the userspace to attack the guest OS will not be considered
+security vulnerabilities.
+
+Bugs that could compromise Xen will be considered security vulnerabilities.
+
 ### ARM Scalable Vector Extension (SVE/SVE2)
 
 Arm64 domains can use Scalable Vector Extension (SVE/SVE2).
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index be76be8d53..0cac601066 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1942,6 +1942,17 @@ This option is ignored in **pv-shim** mode.
 
 > Default: `on`
 
+### partial-emulation (arm)
+> `= <boolean>`
+
+> Default: `false`
+
+Flag to enable or disable partial emulation of system/coprocessor registers.
+Only effective if CONFIG_PARTIAL_EMULATION is enabled.
+
+**WARNING: Enabling this option might result in unwanted/non-spec compliant
+behavior.**
+
 ### pci
     = List of [ serr=<bool>, perr=<bool> ]
 
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 50e9bfae1a..dc243f4124 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -225,6 +225,15 @@ config STATIC_EVTCHN
 	  This option enables establishing static event channel communication
 	  between domains on a dom0less system (domU-domU as well as domU-dom0).
 
+config PARTIAL_EMULATION
+	bool "Enable partial emulation of system/coprocessor registers"
+	default y
+	help
+	  This option enables partial emulation of registers to prevent guests
+	  crashing when accessing registers which are not optional but have not been
+	  emulated to their complete functionality. Enabling this might result in
+	  unwanted/non-spec compliant behavior.
+
 endmenu
 
 menu "ARM errata workaround via the alternative framework"
diff --git a/xen/arch/arm/include/asm/traps.h b/xen/arch/arm/include/asm/traps.h
index 883dae368e..9a60dbf70e 100644
--- a/xen/arch/arm/include/asm/traps.h
+++ b/xen/arch/arm/include/asm/traps.h
@@ -10,6 +10,12 @@
 # include <asm/arm64/traps.h>
 #endif
 
+#ifdef CONFIG_PARTIAL_EMULATION
+extern bool partial_emulation;
+#else
+#define partial_emulation false
+#endif
+
 /*
  * GUEST_BUG_ON is intended for checking that the guest state has not been
  * corrupted in hardware and/or that the hardware behaves as we
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 9c10e8f78c..d1c7a6c516 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -42,6 +42,15 @@
 #include <asm/vgic.h>
 #include <asm/vtimer.h>
 
+/*
+ * partial_emulation: If true, partial emulation for system/coprocessor
+ * registers will be enabled.
+ */
+#ifdef CONFIG_PARTIAL_EMULATION
+bool __ro_after_init partial_emulation = false;
+boolean_param("partial-emulation", partial_emulation);
+#endif
+
 /* The base of the stack must always be double-word aligned, which means
  * that both the kernel half of struct cpu_user_regs (which is pushed in
  * entry.S) and struct cpu_info (which lives at the bottom of a Xen
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 12:19:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 12:19:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683475.1063019 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP5k-0001n5-0R; Tue, 20 Feb 2024 12:19:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683475.1063019; Tue, 20 Feb 2024 12:19:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP5j-0001my-U5; Tue, 20 Feb 2024 12:19:19 +0000
Received: by outflank-mailman (input) for mailman id 683475;
 Tue, 20 Feb 2024 12:19:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J7zC=J5=amd.com=ayan.kumar.halder@srs-se1.protection.inumbo.net>)
 id 1rcP5i-0001mj-Vm
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 12:19:18 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 43c8ef3f-cfea-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 13:19:17 +0100 (CET)
Received: from DS7PR05CA0022.namprd05.prod.outlook.com (2603:10b6:5:3b9::27)
 by BL0PR12MB4916.namprd12.prod.outlook.com (2603:10b6:208:1ce::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.20; Tue, 20 Feb
 2024 12:19:13 +0000
Received: from CY4PEPF0000EDD5.namprd03.prod.outlook.com
 (2603:10b6:5:3b9:cafe::d1) by DS7PR05CA0022.outlook.office365.com
 (2603:10b6:5:3b9::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.15 via Frontend
 Transport; Tue, 20 Feb 2024 12:19:13 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EDD5.mail.protection.outlook.com (10.167.241.209) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 20 Feb 2024 12:19:13 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 20 Feb
 2024 06:19:12 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 20 Feb
 2024 06:19:12 -0600
Received: from xcbayankuma40.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via
 Frontend Transport; Tue, 20 Feb 2024 06:19:11 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43c8ef3f-cfea-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=f0lV63jXy2bt9vgQdON2yGMcaTkCgTFgDhzL+NRNgozQvyh/vsnhHr57xwc1AdvE09XilBHieknXYSTj+FAZpHfdcWPerWlN3LeQGeK4bNd++GeJwQ4eAm78ZWlCukguqmWY0Y+rNgWJA37iFkdQ8EpWseYm0ilwmnOBZiDU8KJZpPOHZMpH59Qq36B3HRzBo9wnLCijG+UgA/lT7MVgqejK00yHcfANKRDSw0P9343UHWvWq+ZVwMrGo0laJMa9vd5UjlYWyA32QV8+AcTSSYEHTAKQ5ldAaLDC0geA3NPU9KTUxMFPKEAkHg7hOIifJ0ta1LxInR1ElODo00X3wA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=TyheUcKZClDVXN1YlPjAzeLmxnokNyXGDiz7llZyvHQ=;
 b=aUVSLx/hw8NBtHcbgMvjHzeO0ev8HOs9mDverkKEjxz58La/G95x2IbT4v6d30q7tmO/z11WcRxRUPdJtxAjpMxVKCjPlUCpI68pMvNeG0WPrHD6uVmxTcZ3sHgOfQIkwk3H436gasMEalpJvK6JyLgihW+CLm2XXqGyl3mJgTsGouuWJDXdHTW3Y8qWZiuehdZWd9tiYq2/WT+Ike1wKpZtCv2SweAXAHx/6SlyLicZpHwG5jhJYHv/wFywlOR8KH32z1SM8HYV35HWsSGJHClAFpMUbdCYwkmuLKEs1htTDBjEsYTsTEWt0bEdF6ZxbU3RfInB8o/RAK6jR9rcgg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=TyheUcKZClDVXN1YlPjAzeLmxnokNyXGDiz7llZyvHQ=;
 b=ALLVrMhhNMJfoDkrNLePd/ehOGV6H5/hD46zPfrUDkpfAfHpwlUbir2BD63eQNXZmMPo8CzIas9dvK1Y0aeliYIMQpcN/LMo7EN6azns1f5T8JFtVHnlJkV8wS6hPq69uJX064+CMFxPwbqa832lZnzSa1yUVzzQmRDwF8ChRlM=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <stefano.stabellini@amd.com>, <julien@xen.org>,
	<Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>,
	<michal.orzel@amd.com>, Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Subject: [XEN v5 2/3] xen/arm: arm64: Add emulation of Debug Data Transfer Registers
Date: Tue, 20 Feb 2024 12:17:42 +0000
Message-ID: <20240220121743.3680715-3-ayan.kumar.halder@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD5:EE_|BL0PR12MB4916:EE_
X-MS-Office365-Filtering-Correlation-Id: c4deb2e0-a812-42fc-e205-08dc320e266b
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	fgErVotxVtLU/1F9k57JNye2xARh6V9xIOKCdK0h76Uy5Rk5ZlogZ9O46ecKNY+xhC2OmLA6kEavoytjnGonYjnPA4Qh8fxfpEs/g+VGCTKxkppcrkCn57NGECnX9qPVEHL4Q5jeBLYE5y5alCozw7hccMsssk7rrLYqA7br4N0o7dlLwpU4Z7JxMeHOk92RyW0ev/3T9zn1ZRReZV/2uV1/6trR1aWpjNlWewgzI48tmCcmgDPxHpnyrvkv6n2ElrB0Q4yzyecoMtbBZkA7+2wrxj9FPshhuV8gwjDWOcHcQEmvIJ7TauPF0yPCHy0BWS2S7Ax6NdHTMgPI616ZWQrpoLAeEE4bdFUITVncMh+UOM7kR3bUWfVk6+LEbC17G/Up78/duzw1gyv31ksX27Mp6dnwzMjeObQYdiMA2Hg0b6ybHcf4DpVTZOwf/T+cV7CtwEZ1X/AbcpuGOsJyvO5PNNCbFp0uHs3/TYeEfXRYlffByJvIzadE80RDSiuRt5O0XQdYd+LR/xsBal1nVLdYpbtWII9MMbfF5+0keQqQKxbTIn07f+E7uxMv6UDK4lHEcfXSYxBpSz6Fb5gi8Hru93+/c1Bxy7ktA6zCtp/CakzbNi9vXlHoMfO4PXf/Ia/2jfrrA9F6cybK8qDUG4m1RZ/znZZlY/SuqcB9hfM=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(40470700004)(46966006);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2024 12:19:13.4209
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c4deb2e0-a812-42fc-e205-08dc320e266b
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EDD5.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4916

From: Michal Orzel <michal.orzel@amd.com>

Currently, if user enables HVC_DCC config option in Linux, it invokes access
to debug data transfer registers (i.e. DBGDTRTX_EL0 on arm64, DBGDTRTXINT on
arm32). As these registers are not emulated, Xen injects an undefined
exception to the guest and Linux crashes.

To prevent this crash, introduce a partial emulation of DBGDTR[TR]X_EL0
(these registers share the same encoding) as RAZ/WI and MDCCSR_EL0 as TXfull.

Refer ARM DDI 0487J.a ID042523, D19.3.8, DBGDTRTX_EL0
"If TXfull is set to 1, set DTRRX and DTRTX to UNKNOWN".

Thus, any OS is expected to read MDCCSR_EL0 and check for TXfull before
using DBGDTRTX_EL0. Linux does it via hvc_dcc_init() ---> hvc_dcc_check(),
and returns -ENODEV in case TXfull bit is still set after writing a test
character. This way we prevent the guest from making use of HVC DCC as a
console.

Signed-off-by: Michal Orzel <michal.orzel@amd.com>
Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
Changes from

v1 :- 1. DBGDTR_EL0 does not emulate RXfull. This is to avoid giving the OS any
indication that the RX buffer is full and is waiting to be read.

2. In Arm32, DBGOSLSR is emulated. Also DBGDTRTXINT is emulated at EL0 only.

3. Fixed the commit message and inline code comments.

v2 :- 1. Split the patch into two (separate patches for arm64 and arm32).
2. Removed the "fail" label.
3. Fixed the commit message.

v3 :- 1. "HSR_SYSREG_MDCCSR_EL0" emulation differs based on whether 
partial_emulation_enabled is true or not.

2. If partial_emulation_enabled is false, then access to HSR_SYSREG_DBGDTR_EL0,
HSR_SYSREG_DBGDTRTX_EL0 would lead to undefined exception. 

v4 :- 1. Invoked "goto fail" from "default:" to ensure compliance with
MISRA 15.3.

 xen/arch/arm/arm64/vsysreg.c         | 68 +++++++++++++++++++---------
 xen/arch/arm/include/asm/arm64/hsr.h |  3 ++
 2 files changed, 50 insertions(+), 21 deletions(-)

diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
index b5d54c569b..80918bc799 100644
--- a/xen/arch/arm/arm64/vsysreg.c
+++ b/xen/arch/arm/arm64/vsysreg.c
@@ -82,6 +82,7 @@ TVM_REG(CONTEXTIDR_EL1)
 void do_sysreg(struct cpu_user_regs *regs,
                const union hsr hsr)
 {
+    const struct hsr_sysreg sysreg = hsr.sysreg;
     int regidx = hsr.sysreg.reg;
     struct vcpu *v = current;
 
@@ -159,9 +160,6 @@ void do_sysreg(struct cpu_user_regs *regs,
      *
      * Unhandled:
      *    MDCCINT_EL1
-     *    DBGDTR_EL0
-     *    DBGDTRRX_EL0
-     *    DBGDTRTX_EL0
      *    OSDTRRX_EL1
      *    OSDTRTX_EL1
      *    OSECCR_EL1
@@ -171,12 +169,42 @@ void do_sysreg(struct cpu_user_regs *regs,
      */
     case HSR_SYSREG_MDSCR_EL1:
         return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
+
+    /*
+     * Xen doesn't expose a real (or emulated) Debug Communications Channel
+     * (DCC) to a domain. Yet the Arm ARM implies this is not an optional
+     * feature. So some domains may start to probe it. For instance, the
+     * HVC_DCC driver in Linux (since f377775dc083 and at least up to v6.7),
+     * will try to write some characters and check if the transmit buffer
+     * has emptied.
+     */
     case HSR_SYSREG_MDCCSR_EL0:
         /*
+         * By setting TX status bit (only if partial emulation is enabled) to
+         * indicate the transmit buffer is full, we would hint the OS that the
+         * DCC is probably not working.
+         *
+         * Bit 29: TX full
+         *
          * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We emulate that
          * register as RAZ/WI above. So RO at both EL0 and EL1.
          */
-        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 0);
+        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 0,
+                                  partial_emulation ? (1U << 29) : 0);
+
+    case HSR_SYSREG_DBGDTR_EL0:
+    /* DBGDTR[TR]X_EL0 share the same encoding */
+    case HSR_SYSREG_DBGDTRTX_EL0:
+        /*
+         * Emulate as RAZ/WI (only if partial emulation is enabled) to prevent
+         * injecting undefined exception.
+         * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We emulate that
+         * register as RAZ/WI.
+         */
+        if ( !partial_emulation )
+            goto fail;
+        return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 0);
+
     HSR_SYSREG_DBG_CASES(DBGBVR):
     HSR_SYSREG_DBG_CASES(DBGBCR):
     HSR_SYSREG_DBG_CASES(DBGWVR):
@@ -394,26 +422,24 @@ void do_sysreg(struct cpu_user_regs *regs,
      * And all other unknown registers.
      */
     default:
-        {
-            const struct hsr_sysreg sysreg = hsr.sysreg;
-
-            gdprintk(XENLOG_ERR,
-                     "%s %d, %d, c%d, c%d, %d %s x%d @ 0x%"PRIregister"\n",
-                     sysreg.read ? "mrs" : "msr",
-                     sysreg.op0, sysreg.op1,
-                     sysreg.crn, sysreg.crm,
-                     sysreg.op2,
-                     sysreg.read ? "=>" : "<=",
-                     sysreg.reg, regs->pc);
-            gdprintk(XENLOG_ERR,
-                     "unhandled 64-bit sysreg access %#"PRIregister"\n",
-                     hsr.bits & HSR_SYSREG_REGS_MASK);
-            inject_undef_exception(regs, hsr);
-            return;
-        }
+        goto fail;
     }
 
     regs->pc += 4;
+    return;
+
+ fail:
+
+    gdprintk(XENLOG_ERR,
+             "%s %d, %d, c%d, c%d, %d %s x%d @ 0x%"PRIregister"\n",
+             sysreg.read ? "mrs" : "msr", sysreg.op0, sysreg.op1, sysreg.crn,
+             sysreg.crm, sysreg.op2, sysreg.read ? "=>" : "<=", sysreg.reg,
+             regs->pc);
+    gdprintk(XENLOG_ERR,
+             "unhandled 64-bit sysreg access %#"PRIregister"\n",
+             hsr.bits & HSR_SYSREG_REGS_MASK);
+    inject_undef_exception(regs, hsr);
+    return;
 }
 
 /*
diff --git a/xen/arch/arm/include/asm/arm64/hsr.h b/xen/arch/arm/include/asm/arm64/hsr.h
index e691d41c17..1495ccddea 100644
--- a/xen/arch/arm/include/asm/arm64/hsr.h
+++ b/xen/arch/arm/include/asm/arm64/hsr.h
@@ -47,6 +47,9 @@
 #define HSR_SYSREG_OSDLR_EL1      HSR_SYSREG(2,0,c1,c3,4)
 #define HSR_SYSREG_DBGPRCR_EL1    HSR_SYSREG(2,0,c1,c4,4)
 #define HSR_SYSREG_MDCCSR_EL0     HSR_SYSREG(2,3,c0,c1,0)
+#define HSR_SYSREG_DBGDTR_EL0     HSR_SYSREG(2,3,c0,c4,0)
+#define HSR_SYSREG_DBGDTRTX_EL0   HSR_SYSREG(2,3,c0,c5,0)
+#define HSR_SYSREG_DBGDTRRX_EL0   HSR_SYSREG(2,3,c0,c5,0)
 
 #define HSR_SYSREG_DBGBVRn_EL1(n) HSR_SYSREG(2,0,c0,c##n,4)
 #define HSR_SYSREG_DBGBCRn_EL1(n) HSR_SYSREG(2,0,c0,c##n,5)
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 12:19:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 12:19:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683476.1063028 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP5o-00027S-Bz; Tue, 20 Feb 2024 12:19:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683476.1063028; Tue, 20 Feb 2024 12:19:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP5o-00027L-90; Tue, 20 Feb 2024 12:19:24 +0000
Received: by outflank-mailman (input) for mailman id 683476;
 Tue, 20 Feb 2024 12:19:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J7zC=J5=amd.com=ayan.kumar.halder@srs-se1.protection.inumbo.net>)
 id 1rcP5m-0001mj-Nr
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 12:19:22 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 468c6110-cfea-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 13:19:22 +0100 (CET)
Received: from CYZPR11CA0024.namprd11.prod.outlook.com (2603:10b6:930:8d::9)
 by CYYPR12MB8961.namprd12.prod.outlook.com (2603:10b6:930:bf::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.20; Tue, 20 Feb
 2024 12:19:17 +0000
Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com
 (2603:10b6:930:8d:cafe::64) by CYZPR11CA0024.outlook.office365.com
 (2603:10b6:930:8d::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.40 via Frontend
 Transport; Tue, 20 Feb 2024 12:19:17 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.211) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 20 Feb 2024 12:19:17 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 20 Feb
 2024 06:19:15 -0600
Received: from xcbayankuma40.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via
 Frontend Transport; Tue, 20 Feb 2024 06:19:14 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 468c6110-cfea-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=WSsUIBUGmcyd7LuDyoccMlqiRxnGzGDhK2hU3YOdVzJfr7TvsLcFxemCph24hJZaRDEFW638npeauQKY6esEiJ9oSwFQeTIR2g6+8GB1xBygjf0oq9J8EMYEJ3q7RaVXXO1TspQopaJYnInkJGKSa3O+bOa9zXWcqC9pGA241m5PZtEs4ebaD+VgpKGyA7dBUNOMscm3e5vcjTSyNr/LsqnwnUOe1VT5XQyItr9bd40ICufKG3Qs39dtOCTcakOlLr4594MWWNodAcM72DpnYpK+KTzPqdWQihsvLcK/32U4l487lOBde0O6w4JlEGbZVxUeR/4TyC92ZCvjnE3xtg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=PhMwKyVo3kV6ClvTBpqbApB6tIudAzmtxwRqsvXdrFs=;
 b=BaMhWqkfMNVLZVPykLAlaaubHTtmeyarrZyNFWGYqf7qLJra9VMOuZ8cxyP34TIVYJ+5KtHpki7CA9Wd72gYP+DgxrPpSaRIQw/YQmipGVcKXwo5+PM8Ty/dB+OdT/W6VyEC89xhvkyMZp6HSvSOW7vZ/TC31h8SvYOnzIBBAGhAKaB7fLSFqXjozU9d7OMOjFqB9YxGadB8stL2Mon0KnHlRMuV4Dm/MDwGyyiFG1PS5MDMOSBdBjtfks/boxk0qo37ayw35jvxG177QRCOHD1HYYse3BtXgKCmaIyq3v9L0JZdY23ikEefn0ReOW7lrExJancaS0cxeuAl4R/S9Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PhMwKyVo3kV6ClvTBpqbApB6tIudAzmtxwRqsvXdrFs=;
 b=B1NPj/4rKBQE0tFtJjAHtRy01GR+GmMSoHWNV2mRq2tGOh5bvoMt17iKbfOWR4cSufMvasX1SWXxPTTN/aD+OOwZ7sS7/5PIrUgDWG06jGiOPX7+8+f2Len6wtsDl3a6ivn3e+KaPuzTNPq4ErSW0D75Q8/cTxi5MfOApHe0ywg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <stefano.stabellini@amd.com>, <julien@xen.org>,
	<Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>,
	<michal.orzel@amd.com>, Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Subject: [XEN v5 3/3] xen/arm: arm32: Add emulation of Debug Data Transfer Registers
Date: Tue, 20 Feb 2024 12:17:43 +0000
Message-ID: <20240220121743.3680715-4-ayan.kumar.halder@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|CYYPR12MB8961:EE_
X-MS-Office365-Filtering-Correlation-Id: 41435a71-6529-419c-ca3c-08dc320e28b7
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	qJxf9g7TyisYRj5LmoWbUfyKbkbQz/mPtUCK72cUQ7jGlA7lbZ8dynYDp+yIp9oRYPWNLRxsmB3iNHkIGR18X1u45vD+55T3kkc2hJQhbXQzOuwBGLx9oUqKyJRgEg7tcFOTrZd6F4sW+qsiVWwDILMU47M4ANN6key9upJBbtQ24mfY/Q64T19I1nZB5p2qNZ9BN84BYiDEx7QES86hboYPmpeBBOX1/g1Guj7VpcvmxTQOWnVX6+0LbFeO3eSWkhgCIBYEcwFX9TopYc4t/paagxc+ejmsSF1yUiyPbYdbG6sMzKh8GIv7nn41Cxo9p5xYyGRreokcawyEBns/4jqdplr/gyF3pdvtnIqLzxirQg4axkvH8Dfo7pGM51hiNXmD9m1k7TYVvVMk0mUgkjSU4DGHEIg7zQ4TU9vIrtXiC7C2mz2evXa4quHWD9Gc9f/Z0HjDhFsy4kOEF5kUjtYxatHoSbGauMSQkQA3u0nQZA0I011ADtqWdrFooMIk032vYl5JJJwiXxr4Tqfq0GcHMNI3s5nJ+VVd3o2xbzDWyGa32RKYZrGHaeDxPMJaIsD2/O7xyGJfb3SRT2Nv+okf+jhuf6204V44TizGcZVEKgCdsbP3yC9kxF/HHczfQwBqrpu5iYvbNaovOqrGXTMB6/R7HwHeYKNQmAaGxPQ=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(230273577357003)(36860700004)(46966006)(40470700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2024 12:19:17.2738
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 41435a71-6529-419c-ca3c-08dc320e28b7
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EDD7.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8961

When user enables HVC_DCC config option in Linux, it invokes access to debug
transfer register (i.e. DBGDTRTXINT). As this register is not emulated, Xen
injects an undefined exception to the guest and Linux crashes.
To prevent this crash, introduce a partial emulation of DBGDTRTXINT as RAZ/WI
and TXfull should be set to 1. So that Linux will see that TXfull is set, and
it will not thereafter access DBGDTRTXINT (it will access it at least once to
later check if TXfull is set).

As a pre-requisite, DBGOSLSR should be emulated in the same way as its AArch64
variant (ie OSLSR_EL1).
This is to ensure that DBGOSLSR.OSLK is 0, thus DBGDSCRINT.TXfull is treated
as UNK/SBZP. DBGOSLSR.OSLM[1] is set to 1 to mantain consistency with OSLSR_EL1.

Only DBGDSCRINT can be emulated. DBGDSCRINT can be accessed at EL0 as DBGDSCREXT
is emulated as RAZ (as DBGOSLSR.OSLK == 0). So, we took the opportunity to fix
the minimum EL for DBGDSCRINT.DBGDSCRINT.TXfull is set to 1.

Refer ARM DDI 0487J.a ID042523, G8.3.19, DBGDTRTXint
"If TXfull is set to 1, set DTRTX to UNKNOWN".
So, DBGDTR[TR]XINT is emulated as RAZ/WI.

Thus, any OS is expected to read DBGDSCRINT and check for TXfull before using
DBGDTRTXINT.

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
---
Changes from

v1 :- 1. DBGDTR_EL0 does not emulate RXfull. This is to avoid giving the OS any
indication that the RX buffer is full and is waiting to be read.

2. In Arm32, DBGOSLSR is emulated. Also DBGDTRTXINT is emulated at EL0 only.

3. Fixed the commit message and inline code comments.

v2 :- 1. Split the patch into two (separate patches for arm64 and arm32).
2. Fixed in line comments and style related issues.
3. Updated commit message to mention DBGDSCRINT handling.

v3 :- 1. The original emulation of DBGDSCRINT is retained when
'partial_emulation' is false.

2. If 'partial_emulation' is false, then access to DBGDTRTXINT will
lead to undefined exception.

v4 :- 1. Invoked "goto fail" from "default:" to ensure compliance with
MISRA 15.3.

 xen/arch/arm/include/asm/cpregs.h |  2 +
 xen/arch/arm/vcpreg.c             | 63 ++++++++++++++++++++++---------
 2 files changed, 47 insertions(+), 18 deletions(-)

diff --git a/xen/arch/arm/include/asm/cpregs.h b/xen/arch/arm/include/asm/cpregs.h
index 6b083de204..aec9e8f329 100644
--- a/xen/arch/arm/include/asm/cpregs.h
+++ b/xen/arch/arm/include/asm/cpregs.h
@@ -75,6 +75,8 @@
 #define DBGDIDR         p14,0,c0,c0,0   /* Debug ID Register */
 #define DBGDSCRINT      p14,0,c0,c1,0   /* Debug Status and Control Internal */
 #define DBGDSCREXT      p14,0,c0,c2,2   /* Debug Status and Control External */
+#define DBGDTRRXINT     p14,0,c0,c5,0   /* Debug Data Transfer Register, Receive */
+#define DBGDTRTXINT     p14,0,c0,c5,0   /* Debug Data Transfer Register, Transmit */
 #define DBGVCR          p14,0,c0,c7,0   /* Vector Catch */
 #define DBGBVR0         p14,0,c0,c0,4   /* Breakpoint Value 0 */
 #define DBGBCR0         p14,0,c0,c0,5   /* Breakpoint Control 0 */
diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
index a2d0500704..1a2d79b818 100644
--- a/xen/arch/arm/vcpreg.c
+++ b/xen/arch/arm/vcpreg.c
@@ -493,11 +493,12 @@ void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr)
      * ARMv8 (DDI 0487A.d): D1-1509 Table D1-58
      *
      * Unhandled:
-     *    DBGOSLSR
      *    DBGPRCR
      */
     case HSR_CPREG32(DBGOSLAR):
         return handle_wo_wi(regs, regidx, cp32.read, hsr, 1);
+    case HSR_CPREG32(DBGOSLSR):
+        return handle_ro_read_val(regs, regidx, cp32.read, hsr, 1, 1U << 3);
     case HSR_CPREG32(DBGOSDLR):
         return handle_raz_wi(regs, regidx, cp32.read, hsr, 1);
 
@@ -509,8 +510,6 @@ void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr)
      *
      * Unhandled:
      *    DBGDCCINT
-     *    DBGDTRRXint
-     *    DBGDTRTXint
      *    DBGWFAR
      *    DBGDTRTXext
      *    DBGDTRRXext,
@@ -548,12 +547,27 @@ void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr)
         break;
     }
 
+    /*
+     * Xen doesn't expose a real (or emulated) Debug Communications Channel
+     * (DCC) to a domain. Yet the Arm ARM implies this is not an optional
+     * feature. So some domains may start to probe it. For instance, the
+     * HVC_DCC driver in Linux (since f377775dc083 and at least up to v6.7),
+     * will try to write some characters and check if the transmit buffer has
+     * emptied.
+     */
     case HSR_CPREG32(DBGDSCRINT):
         /*
-         * Read-only register. Accessible by EL0 if DBGDSCRext.UDCCdis
-         * is set to 0, which we emulated below.
+         * By setting TX status bit (only if partial emulation is enabled) to
+         * indicate the transmit buffer is full, we would hint the OS that the
+         * DCC is probably not working.
+         *
+         * Bit 29: TX full
+         *
+         * Accessible at EL0 only if DBGDSCRext.UDCCdis is set to 0. We emulate
+         * this as RAZ/WI in the next case. So RO at both EL0 and EL1.
          */
-        return handle_ro_raz(regs, regidx, cp32.read, hsr, 1);
+        return handle_ro_read_val(regs, regidx, cp32.read, hsr, 0,
+                                  partial_emulation ? (1U << 29) : 0);
 
     case HSR_CPREG32(DBGDSCREXT):
         /*
@@ -562,6 +576,18 @@ void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr)
          */
         return handle_raz_wi(regs, regidx, cp32.read, hsr, 1);
 
+    /* DBGDTR[TR]XINT share the same encoding */
+    case HSR_CPREG32(DBGDTRTXINT):
+        /*
+         * Emulate as RAZ/WI (only if partial emulation is enabled) to prevent
+         * injecting undefined exception.
+         * Accessible at EL0 only if DBGDSCREXT is set to 0. We emulate that
+         * register as RAZ/WI.
+         */
+        if ( !partial_emulation )
+            goto fail;
+        return handle_raz_wi(regs, regidx, cp32.read, hsr, 0);
+
     case HSR_CPREG32(DBGVCR):
     case HSR_CPREG32(DBGBVR0):
     case HSR_CPREG32(DBGBCR0):
@@ -591,17 +617,21 @@ void do_cp14_32(struct cpu_user_regs *regs, const union hsr hsr)
      * And all other unknown registers.
      */
     default:
-        gdprintk(XENLOG_ERR,
-                 "%s p14, %d, r%d, cr%d, cr%d, %d @ 0x%"PRIregister"\n",
-                  cp32.read ? "mrc" : "mcr",
-                  cp32.op1, cp32.reg, cp32.crn, cp32.crm, cp32.op2, regs->pc);
-        gdprintk(XENLOG_ERR, "unhandled 32-bit cp14 access %#"PRIregister"\n",
-                 hsr.bits & HSR_CP32_REGS_MASK);
-        inject_undef_exception(regs, hsr);
-        return;
+        goto fail;
     }
 
     advance_pc(regs, hsr);
+    return;
+
+ fail:
+    gdprintk(XENLOG_ERR,
+             "%s p14, %d, r%d, cr%d, cr%d, %d @ 0x%"PRIregister"\n",
+             cp32.read ? "mrc" : "mcr",
+             cp32.op1, cp32.reg, cp32.crn, cp32.crm, cp32.op2, regs->pc);
+    gdprintk(XENLOG_ERR, "unhandled 32-bit cp14 access %#"PRIregister"\n",
+             hsr.bits & HSR_CP32_REGS_MASK);
+    inject_undef_exception(regs, hsr);
+    return;
 }
 
 void do_cp14_64(struct cpu_user_regs *regs, const union hsr hsr)
@@ -659,10 +689,7 @@ void do_cp14_dbg(struct cpu_user_regs *regs, const union hsr hsr)
      * ARMv8 (DDI 0487A.d): D1-1509 Table D1-58
      *
      * Unhandled:
-     *    DBGDTRTXint
-     *    DBGDTRRXint
-     *
-     * And all other unknown registers.
+     * All unknown registers.
      */
     gdprintk(XENLOG_ERR,
              "%s p14, %d, r%d, r%d, cr%d @ 0x%"PRIregister"\n",
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 12:20:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 12:20:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683485.1063038 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP6i-0004Gg-L4; Tue, 20 Feb 2024 12:20:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683485.1063038; Tue, 20 Feb 2024 12:20:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcP6i-0004GZ-IL; Tue, 20 Feb 2024 12:20:20 +0000
Received: by outflank-mailman (input) for mailman id 683485;
 Tue, 20 Feb 2024 12:20:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcP6h-0004GE-SK
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 12:20:19 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 68ef44ce-cfea-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 13:20:18 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-410e820a4feso36042615e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 04:20:18 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h7-20020a056000000700b0033ce214a97csm13294488wrx.17.2024.02.20.04.20.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 04:20:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 68ef44ce-cfea-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708431618; x=1709036418; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gGpH+jB7t66iuiZhxtwTK7HRXVSiLbcivuzs1lr+EUs=;
        b=Z90UEnNIBnt0Iq+C0n+kW+7uFiCaJypTOuIxNanLmkWzET7SIT97rFoe5+3hBzDY14
         AsqdWxztquncWs2uUs8cfk+B/5RNIKqjoVp8YFTyn21i1r7+5cnP0KsZLorCKjOg1CGf
         k2iNlKwbKjXqj4GFQnCaq6OLuYD18yBS2amRUSf2A5IjzvYfaihcINT6E99dkBInkLQd
         0zvUJIwSMrX1rksgc7grcmk5ibTXHyXrdUaE+JhhMFUbcAkaWeELHhIoVdC6EnKi3NPz
         NXtYkFai9e2gYeVoCBN1xYHq7+JeI3ldrIRNYWnEZiUD55m4sKu8i9csWb6b0ShVN/Id
         A/ZQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708431618; x=1709036418;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gGpH+jB7t66iuiZhxtwTK7HRXVSiLbcivuzs1lr+EUs=;
        b=N/92FlGOYvfvRB+01Y2MdDXLGXVlrGDT3em764Mi7l6M2vAmWsKM0JApJOKqzLPvs6
         UBqwK2ImlrgUMECtdlYL+FVh+CFz6Eh8cydRDtgcx4CWESOtUCqtiGveDDCylyZTroZg
         HTgqJSUJrxQ4BTLJp6HD1f7AItLPikTkEBuSXd6edrWrywSBf8BabNeoN8dQ+5uuxJB1
         N0vZtmy8zyNp5RISEJvoBp4pjVyfWyJiTEaZydmrRjMjhjKy7POVzpMwi+dpcQKT9SVn
         VWTFyk8nRxrOI67jF42nTa48trCHzY8P6P3c+pU5hf3QIZSDQNqo8dxKmMB+uea1dOHM
         0doQ==
X-Gm-Message-State: AOJu0Yy+83NYzMQpt3wAUVvmTsV8B0rsw/jzkBjoeTFf2mhP1PCVzC3+
	b6rxK/+6Fyq9O/GOO4/IU0NoGgBcsrtr9rLHHZgqwUOm2QPZ3t+peNDJHvuTrg==
X-Google-Smtp-Source: AGHT+IEp6LZtjnmzI9LcyMMXQ4nn/eBJ+wKlP3BB1yg2+nIueMrhvUp644KGaxtuvfWYkkVUgjcQcQ==
X-Received: by 2002:a5d:42ca:0:b0:33d:4e39:1c45 with SMTP id t10-20020a5d42ca000000b0033d4e391c45mr5576998wrr.6.1708431618024;
        Tue, 20 Feb 2024 04:20:18 -0800 (PST)
Message-ID: <abbae372-5e3e-4013-a2a3-45892ec96c89@suse.com>
Date: Tue, 20 Feb 2024 13:20:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build: make sure build fails when running kconfig fails
Content-Language: en-US
To: Anthony PERARD <anthony.perard@cloud.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <55c3a781-17f8-47f2-9629-515e1aea77aa@suse.com>
 <b7c22718-ec6c-470e-be72-e613b2af3c54@perard>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b7c22718-ec6c-470e-be72-e613b2af3c54@perard>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 20.02.2024 12:18, Anthony PERARD wrote:
> On Tue, Feb 20, 2024 at 09:43:56AM +0100, Jan Beulich wrote:
>> Because of using "-include", failure to (re)build auto.conf (with
>> auto.conf.cmd produced as a secondary target) won't stop make from
>> continuing the build. Arrange for it being possible to drop the - from
>> Rules.mk, requiring that the include be skipped for tools-only targets.
>> Note that relying on the inclusion in those cases wouldn't be correct
>> anyway, as it might be a stale file (yet to be rebuilt) which would be
>> included, while during initial build, the file would be absent
>> altogether.
>>
>> Fixes: 8d4c17a90b0a ("xen/build: silence make warnings about missing auto.conf*")
>> Reported-by: Roger Pau Monné <roger.pau@citrix.com>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Just to make sure, this patch is a workaround to a harmless bug in older
> version of GNU Make which print spurious error messages, and said bug as
> been fixed in GNU Make 4.2, right? Bug report:
> 
>     bug #102: Make prints an incorrect error for missing includes
>     https://savannah.gnu.org/bugs/?func=detailitem&item_id=102

Assuming you mean the change referenced in Fixes: - yes, that's my
understanding. (Whether the referenced make bug is the one I can't tell,
though. But it looks like it is.)

>> --- a/xen/Makefile
>> +++ b/xen/Makefile
>> @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
>>  # This exploits the 'multi-target pattern rule' trick.
>>  # The syncconfig should be executed only once to make all the targets.
>>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
>> +	$(Q)rm -f include/config/$*.conf
> 
> Maybe this should say "include/config/auto.conf" instead of using "$*".
> "syncconfig" is going to generate "auto.conf" and not "$*.conf". And it
> would make easier to find the "rm" command via "grep".

Generally I advocate for using $* wherever possible in pattern rules. If,
however, replacing that is the only way to get an ack for this change, then
I would (hesitantly) do so.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 12:25:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 12:25:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683500.1063049 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcPBL-00050R-7O; Tue, 20 Feb 2024 12:25:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683500.1063049; Tue, 20 Feb 2024 12:25:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcPBL-00050K-4H; Tue, 20 Feb 2024 12:25:07 +0000
Received: by outflank-mailman (input) for mailman id 683500;
 Tue, 20 Feb 2024 12:25:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcPBJ-00050E-Rf
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 12:25:05 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 12d960cb-cfeb-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 13:25:03 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-4126f48411dso3126465e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 04:25:03 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z3-20020a1c4c03000000b004120675e057sm7044964wmf.0.2024.02.20.04.25.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 04:25:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12d960cb-cfeb-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708431903; x=1709036703; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=kE3Vbeq3mNcgQGuLeKzL4FwS7l8YmoQTCMH5OEiHNzI=;
        b=GgHcvDU+MOi9V+YZxSd5eM77B418DNHnxODU5TBOd8pYqyNuBOLcZ9/Yj2YCWwatO+
         1mgsnsXHNzeD2wflrjyxf2NxEQQDKJLiumQpjgKyCAwuF9Siz9sUiUj/ld70ObGMhe1K
         hIdeMewbT1//B8cA+lA/sP9ztEc5N0NWb2Rfa+/Qb9A31syCIUkMVgcKQC+hIhFHh+Tk
         yWohDn1N0cqOhS7HWW5v73Vfjr59tXz0BLL2R/4ewGGi2r2hZIJvTwEn1Tp54KQ/h6o2
         q7WDkv3lJE5qMlMGjNvqh2CDlmbrgLADXlBewg5qHlRhINhf3FOMf6s1296RNEjjggdK
         A+Ag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708431903; x=1709036703;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kE3Vbeq3mNcgQGuLeKzL4FwS7l8YmoQTCMH5OEiHNzI=;
        b=miHNZ7QQqwa+hKvYBB21vPNkmLfxFhUKp3c/B3U7Wc8WWWgEVZ0MW8vi+9/D9vbrNO
         Z84GnNv6afeokriMYZPcmNkr9lfg0TU0QLUFB8sQtv5TEKpcIIfAE0e32HIGsW6NVdCg
         9vmB4x3B/2KanfOdZQH5CykUcWsLGTk09TNuOhzEX6RaWMj2HbCWLmYx9QVC1XDXVQz0
         2E4DuR4hptHCluiGHehPVZg193GFY5wDTdBGqOetUFZ9jhOxFO32TWg552+1LKdwFsst
         DM642kp2VENo/FwDLmA9WwdnT2lrQQspmBQFnEvfe3lwvxjLinK6rcxVR3XqlWX8It8r
         4uwQ==
X-Forwarded-Encrypted: i=1; AJvYcCUiR0LCoqnQ5o8e7aieegCKs6ZaAu/ZvcG1oz9H9FBVI9y/4hADLfr0Ta/k4zu5PATi+pqxHEvZwlxu2eEOubw3PQoGyWHe7rbm4ZLsgFk=
X-Gm-Message-State: AOJu0YyN+3q5hnRmrv/WK1KIQceEK0ppduxppEntwOwGZQx9oam6Xq0S
	vB1IsXedfxpv6F/dmlTdKX7DVQ3+jqonLVq3dd4qdMxrdkVnmWkMosoypu/bJQ==
X-Google-Smtp-Source: AGHT+IF1DKRZCgdB4ZuJfx7XTV6jtnBi3OKyPYlX9Ye7ndXpCrK9PsIiu1cBrmL4iCT5gf9hgIkfAg==
X-Received: by 2002:a05:600c:358b:b0:412:6e83:b89d with SMTP id p11-20020a05600c358b00b004126e83b89dmr1527115wmq.8.1708431903199;
        Tue, 20 Feb 2024 04:25:03 -0800 (PST)
Message-ID: <d0090122-c013-4dbf-97a2-3003352433a1@suse.com>
Date: Tue, 20 Feb 2024 13:25:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] almost fully ignore zero-size flush requests
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
 <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com>
 <7dd78c97-b82f-4ae1-a257-98f612c6bad2@xen.org>
 <6466155e-9883-4743-a897-ef565f14fdf0@suse.com>
 <a2348c5f-39ce-42f3-ad2a-a5edf88352e9@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <a2348c5f-39ce-42f3-ad2a-a5edf88352e9@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 20.02.2024 12:52, Julien Grall wrote:
> Hi Jan,
> 
> On 20/02/2024 08:26, Jan Beulich wrote:
>> On 19.02.2024 23:22, Julien Grall wrote:
>>> Title: I would add 'gnttab:' to clarify which subsystem you are modifying.
>>
>> That's how I actually have it here; it's not clear to me why I lost the
>> prefix when sending.
>>
>>> On 05/02/2024 11:03, Jan Beulich wrote:
>>>> Along the line with observations in the context of XSA-448, besides
>>>> "op" no field is relevant when the range to be flushed is empty, much
>>>> like e.g. the pointers passed to memcpy() are irrelevant (and would
>>>> never be "validated") when the passed length is zero. Split the existing
>>>> condition validating "op", "offset", and "length", leaving only the "op"
>>>> part ahead of the check for length being zero (or no flushing to be
>>>> performed).
>>>
>>> I am probably missing something here. I understand the theory behind
>>> reducing the number of checks when len == 0. But an OS cannot rely on it:
>>>     1) older hypervisor would still return an error if the check doesn't
>>> pass)
>>
>> Right, but that's no reason to keep the bogus earlier behavior.
> 
> Hmmm... I am not sure why you say the behavior is bogus. From the commit 
> message, it seems this is just an optimization that have side effect 
> (ignoring the other fields).

I don't view this as primarily an optimization; I'm in particular after
not raising errors for cases where there is no error to be raised.
Hence the comparison to memcpy(), which you can pass "bogus" pointers
so long as you pass zero size.

>>>     2) it does feel odd to allow "invalid" offset when len == 0 (at least.
>>
>> I'm puzzled: You've given R-b for patch 1 (thanks), where exactly the
>> same reasoning is used, i.e. similarly referring to memcpy() to
>> justify the (new / supposed) behavior.
> 
> I realize it. But I viewed it slightly different as you are adding the 
> check. I think it is a good idea to add the check and ideally it should 
> be after.
> 
> Here you don't seem to add any check and only re-order it. Hence why I 
> treated it differently.

Right, there already was a zero-length check here. Just that zero
length requests still could have an error returned for no reason. So
the "optimization" part that you're talking about above was already
there, but as said, that's secondary to me.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 12:33:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 12:33:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683507.1063058 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcPJ7-0007ST-2Z; Tue, 20 Feb 2024 12:33:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683507.1063058; Tue, 20 Feb 2024 12:33:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcPJ6-0007SM-W9; Tue, 20 Feb 2024 12:33:08 +0000
Received: by outflank-mailman (input) for mailman id 683507;
 Tue, 20 Feb 2024 12:33:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcPJ5-0007SG-Ad
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 12:33:07 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 325c5dc6-cfec-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 13:33:06 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-563e330351dso4552623a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 04:33:06 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i24-20020a170906251800b00a3d9a94b13fsm3919287ejb.136.2024.02.20.04.33.04
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 04:33:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 325c5dc6-cfec-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708432385; x=1709037185; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=VVIlI8kKiUvnaHesQPPaKCxj6fyw45LDej26PUkzPh4=;
        b=CElfueAHMXI+yugycg192MU9I4L3PT6nh9GSSH9+q1ao9WOdiEhMUh3PbI1TAivZ+w
         mfqIDAX1/fz2YlO02sf1eC2SQ2WynWDc3uR6fQ0pgFCyAvHMxMMQbSlqSERVoPIGVblT
         lFiOr53gpQmS/eZqzB8PSbUcHDOT0/2tR/fk9hYhz2nRomuK2Ks59SXUE7ea0M9xBuin
         JiF8P3NzRyrKMEDKu8DmLgCAjrIbudRZTt0QIbXNo2K7dP13yjuFLlKHQNyz/B8XtPgQ
         DF0M4Mu+UmfViY7HE/AFXEWtJGOQ+twntLO3mhMtIMxSWsB6U+eisfwK8TIuF0fTXXET
         3zNQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708432385; x=1709037185;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=VVIlI8kKiUvnaHesQPPaKCxj6fyw45LDej26PUkzPh4=;
        b=OP7o9lobE1Ti3bCZPH9vckIYLpvtm3uQiFtF60zVQhxJHqtoRF0+4Ih9kG44d3MiuN
         fk0xqwdqk42jfVe4sr4uOhq+P0WhCPuEd/yJdNZ7p6A4s1+fa8+BpW6bjR3ROspB4yhD
         bi2UxQl+th/h3TmgcMMlYxRPgRhjcYwhK7N56aN/lrnoy0/sYEPnZzeuDAXwe+qEck5I
         wU4m29jn1UtgoUTP5gdQx3iKwxH18dcnIVcNlMAyMJduJGWuhmeK5NltFNIfC0NPvPmB
         yOvb4fZpoq6STJ3gaKJPvLtb4wftov2dc3zCaqScw4hvzn/WzBC18UsD0xSW96nQ5aeh
         gniQ==
X-Forwarded-Encrypted: i=1; AJvYcCVk3l7ute2+mhRc1cX5fD5zO5J0XAAU/5Hr426KFL3Yz1zvdnqmfdQYUmGWqvduV4vyMD0cJOg9WaQ0tef6WnGiOb6hmRu1t1Qzeas/w8w=
X-Gm-Message-State: AOJu0YxErNPtIEm3F91z0pIcNWqYmKy1GbUII1iD8fUk6dm1vjWsCI38
	I5k2oYNpTDUWq6Xsr9qJmdKRoRaCOcUE+Vfa1Eq+Xlb9IHnzAB358oNNuYCvVg==
X-Google-Smtp-Source: AGHT+IEgDdWlFcexjOL9HwXd1gkWL06cpWeTMDWFJVt2i/DPBUMr07rGktAFx8SGaulsjXsItO+A2A==
X-Received: by 2002:a17:906:cec7:b0:a3e:73c9:7acc with SMTP id si7-20020a170906cec700b00a3e73c97accmr3943998ejb.33.1708432385507;
        Tue, 20 Feb 2024 04:33:05 -0800 (PST)
Message-ID: <e3a8ce5d-841e-4ff1-8d23-48de822972ef@suse.com>
Date: Tue, 20 Feb 2024 13:33:03 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v5 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
Content-Language: en-US
To: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com, julien@xen.org,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, michal.orzel@amd.com,
 xen-devel@lists.xenproject.org
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
 <20240220121743.3680715-2-ayan.kumar.halder@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240220121743.3680715-2-ayan.kumar.halder@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 20.02.2024 13:17, Ayan Kumar Halder wrote:
> There can be situations when the registers cannot be emulated to their full
> functionality. This can be due to the complexity involved. In such cases, one
> can emulate those registers as RAZ/WI for example. We call them as partial
> emulation.
> 
> Some registers are non-optional and as such there is nothing preventing an OS
> from accessing them.
> Instead of injecting undefined exception (thus crashing a guest), one may want
> to prefer a partial emulation to let the guest running (in some cases accepting
> the fact that it might result in unwanted behavior).
> 
> A suitable example of this (as seen in subsequent patches) is emulation of
> DBGDTRTX_EL0 (on Arm64) and DBGDTRTXINT(on Arm32). These non-optional
> registers can be emulated as RAZ/WI and they can be enclosed within
> CONFIG_PARTIAL_EMULATION.
> 
> Further, "partial-emulation" command line option allows us to
> enable/disable partial emulation at run time. While CONFIG_PARTIAL_EMULATION
> enables support for partial emulation at compile time (i.e. adds code for
> partial emulation), this option may be enabled or disabled by Yocto or other
> build systems. However if the build system turns this option on, users
> can use scripts like Imagebuilder to generate uboot-script which will append
> "partial-emulation=true" to xen command line to turn on the partial emulation.
> Thus, it helps to avoid rebuilding xen.
> 
> By default, "CONFIG_PARTIAL_EMULATION=y" and "partial-emulation=false".
> This is done so that Xen supports partial emulation. However, customers are
> fully aware when they enable partial emulation. It's important to note that
> enabling such support might result in unwanted/non-spec compliant behavior.
> 
> Added a note in SUPPORT.md to clarify the security support for partial
> emulation.

There may have been prior discussion of this, but I view this as too little
to justify ...

> --- a/SUPPORT.md
> +++ b/SUPPORT.md
> @@ -101,6 +101,18 @@ Extension to the GICv3 interrupt controller to support MSI.
>  
>      Status: Experimental
>  
> +### ARM/Partial Emulation
> +
> +Enable partial emulation of registers, otherwise considered unimplemented,
> +that would normally trigger a fault injection.
> +
> +    Status: Supported, with caveats
> +
> +Bugs allowing the userspace to attack the guest OS will not be considered
> +security vulnerabilities.
> +
> +Bugs that could compromise Xen will be considered security vulnerabilities.

... the odd statement regarding in-guest vulnerabilities that might be
introduced. I see only two ways of treating this as supported: Either
you simply refuse emulation when the access is from user space, or you
support that mode of emulation as much as that of kernel space accesses.
After all "injecting undefined exception (thus crashing a guest)"
(quoting the description) doesn't seem quite right: If you inject such
an exception for a user space access, all that ought to happen is the
user space process to be killed.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 12:35:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 12:35:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683516.1063070 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcPL7-00081t-HA; Tue, 20 Feb 2024 12:35:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683516.1063070; Tue, 20 Feb 2024 12:35:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcPL7-00081m-DC; Tue, 20 Feb 2024 12:35:13 +0000
Received: by outflank-mailman (input) for mailman id 683516;
 Tue, 20 Feb 2024 12:35:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qAG8=J5=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rcPL5-00080D-I3
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 12:35:11 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7c415af0-cfec-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 13:35:10 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-512bd533be0so2150617e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 04:35:10 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 c14-20020ac8110e000000b0042be1092915sm3362420qtj.10.2024.02.20.04.35.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 04:35:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c415af0-cfec-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708432509; x=1709037309; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=SqpBys1GZlBb1wU68c/Qfm1otX0q/1l9PzfK4bmChbs=;
        b=S7GR/lY162ErsTLEN1yiT1+4ENZ3QpEmsgduzFHdIv1LvjEZMmw61ltOrKsyiYlSnc
         YNtK6r8nzHB/hnNJmOpvGAnGUcQE+lharbN3ExAbezHXt2B6eetjVoOQNVmOQH72NVv0
         MlJ5CAjoAFexjVayGCvhL4rWVnKWfvoWP0w1g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708432509; x=1709037309;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SqpBys1GZlBb1wU68c/Qfm1otX0q/1l9PzfK4bmChbs=;
        b=vxVb3ESsx97JYKQO8tT6aFaNOONDlfm/BwcdoFmPpifkcnt+r/IVy2BbH3xT4z7gJX
         3Q4JHyq2M1C+M/mUcQVKTLb+ebYPtFQAm2CMNm8Bi4AMDLHD/JlxZ8MCylDpoaFDT1/m
         bVGSLHFk2c5o3DU2crHmaZpHnrrp3RZsRFJLbi20CK3RpdwGhcYtO/tpsrvnb3TAYAuk
         gg3PNH73feqq38qn4Zdz8dA0oOiosc+9q/QaWaDdEsg3DgsrDHWktX8U6waGexdmXyJj
         ClRNM1X6NybnEqezs0DWYuiy3ml0Qm06dYWLmJ2aOC37fqV0i3foXjbJLlXXV7lU0OpN
         1Waw==
X-Forwarded-Encrypted: i=1; AJvYcCVLI48BmU9KmwpGs+NCmRs+DRRRICZk98dBcTCSnjqzD/xuaM6VGRBhCxYlGQSEqYOyfswR2lJJk6stl0DCtmex2MTNiOqvqlf9/8cSd+8=
X-Gm-Message-State: AOJu0Yz9o9kSAHcaWI6/4B59P6fRcwbcHZHku6n7lILWf1zFI+ESjnxA
	nt2JUiiCd4lGp1FTTFgL2rA5TB123r0QavD1sJYt2liEofYGIQ0p1HvTBwJRYWo=
X-Google-Smtp-Source: AGHT+IHCfgVrtMYa0fp+z+iiNFStJsXuBsufUna6l982EcnnfOw5X+iFOo070cXM6N0JdfXwfRyDiA==
X-Received: by 2002:a05:6512:3d16:b0:512:c1e7:185 with SMTP id d22-20020a0565123d1600b00512c1e70185mr1640093lfv.61.1708432509503;
        Tue, 20 Feb 2024 04:35:09 -0800 (PST)
Date: Tue, 20 Feb 2024 13:35:07 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Christoph Hellwig <hch@lst.de>
Cc: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Jens Axboe <axboe@kernel.dk>, xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: Re: [PATCH 4/4] xen-blkfront: atomically update queue limits
Message-ID: <ZdScey8AJvBykWa8@macbook>
References: <20240220084935.3282351-1-hch@lst.de>
 <20240220084935.3282351-5-hch@lst.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240220084935.3282351-5-hch@lst.de>

On Tue, Feb 20, 2024 at 09:49:35AM +0100, Christoph Hellwig wrote:
> Pass the initial queue limits to blk_mq_alloc_disk and use the
> blkif_set_queue_limits API to update the limits on reconnect.

Allocating queue_limits on the stack might be a bit risky, as I fear
this struct is likely to grow?

> Signed-off-by: Christoph Hellwig <hch@lst.de>

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Just one addition while you are already modifying a line.

> ---
>  drivers/block/xen-blkfront.c | 41 ++++++++++++++++++++----------------
>  1 file changed, 23 insertions(+), 18 deletions(-)
> 
> diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
> index 7664638a0abbfa..b77707ca2c5aa6 100644
> --- a/drivers/block/xen-blkfront.c
> +++ b/drivers/block/xen-blkfront.c
> @@ -941,37 +941,35 @@ static const struct blk_mq_ops blkfront_mq_ops = {
>  	.complete = blkif_complete_rq,
>  };
>  
> -static void blkif_set_queue_limits(struct blkfront_info *info)
> +static void blkif_set_queue_limits(struct blkfront_info *info,

While there, could you also constify info?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 13:02:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 13:02:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683523.1063078 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcPlr-0005I0-II; Tue, 20 Feb 2024 13:02:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683523.1063078; Tue, 20 Feb 2024 13:02:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcPlr-0005Ht-FE; Tue, 20 Feb 2024 13:02:51 +0000
Received: by outflank-mailman (input) for mailman id 683523;
 Tue, 20 Feb 2024 13:02:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcPlp-0005Hj-Le; Tue, 20 Feb 2024 13:02:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcPlp-0002PD-DM; Tue, 20 Feb 2024 13:02:49 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcPlo-0005P1-Vy; Tue, 20 Feb 2024 13:02:49 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcPlo-0001nG-Vc; Tue, 20 Feb 2024 13:02:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LNPBTbr5QCyWZL0ZddDL7nNS6KS63hEAh+acqfJwUBQ=; b=FCjMqA3WF438zvdnSOs7Vx8qdO
	0qKnl0UOl1Jo03LWHTiFnNuiow26pgWHbGsHRKoLox7vDr9lGew3CYQvGRME/IIEy7i1Zh1YqC/PF
	NuQ3RjnHH8UccQbLsgx+joe1Ee6aw0o6K/s322MDT34yFbG4fmKKE+a4/QwCkxEI95i8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184709-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184709: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
X-Osstest-Versions-That:
    xen=c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 20 Feb 2024 13:02:48 +0000

flight 184709 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184709/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install    fail pass in 184705

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop  fail in 184705 like 184703
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184705
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184705
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184705
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184705
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184705
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184705
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184705
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184705
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184705
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184705
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184705
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
baseline version:
 xen                  c144b9e32427ba37e0e0443a0d3fa53e9fb878b6

Last test of basis   184709  2024-02-20 05:34:24 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 13:35:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 13:35:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683532.1063093 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcQHW-00026s-9K; Tue, 20 Feb 2024 13:35:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683532.1063093; Tue, 20 Feb 2024 13:35:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcQHW-00026l-5m; Tue, 20 Feb 2024 13:35:34 +0000
Received: by outflank-mailman (input) for mailman id 683532;
 Tue, 20 Feb 2024 13:35:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=/fGf=J5=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rcQHU-00026f-I1
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 13:35:32 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ea744172-cff4-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 14:35:31 +0100 (CET)
Received: from Dell.bugseng.com (unknown [37.163.67.214])
 by support.bugseng.com (Postfix) with ESMTPSA id 78BC44EE073A;
 Tue, 20 Feb 2024 14:35:29 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea744172-cff4-11ee-8a52-1f161083a0e0
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH] x86/uaccess: add attribute noreturn to __{get,put}_user_bad()
Date: Tue, 20 Feb 2024 14:35:20 +0100
Message-Id: <1595eac56587d20c7f86128bc5652c31c3a72772.1708436010.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

__get_user_bad() and __put_user_bad() are undefined symbols used
to assert the unreachability of a program point:
a call to one of such functions is optimized away if it is considered
unreachable by the compiler. Otherwise, a linker error is reported.

In accordance with the purpose of such constructs:
1) add the attribute noreturn to __get_user_bad() and __put_user_bad();
2) change return type of __get_user_bad() to void (returning long is a
   leftover from the past).

Point (1) meets the requirements to deviate MISRA C:2012 Rule 16.3
("An unconditional break statement shall terminate every switch
clause") since functions with noreturn attribute are considered
as allowed terminals for switch clauses.

Point (2) addresses several violations of MISRA C:2012 Rule 17.7
("The value returned by a function having non-void return type
shall be used").

No functional change.

Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
---
 xen/arch/x86/include/asm/uaccess.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/include/asm/uaccess.h b/xen/arch/x86/include/asm/uaccess.h
index 7443519d5b..c7bafaf10f 100644
--- a/xen/arch/x86/include/asm/uaccess.h
+++ b/xen/arch/x86/include/asm/uaccess.h
@@ -21,8 +21,8 @@ unsigned int copy_from_guest_ll(void *to, const void __user *from, unsigned int
 unsigned int copy_to_unsafe_ll(void *to, const void *from, unsigned int n);
 unsigned int copy_from_unsafe_ll(void *to, const void *from, unsigned int n);
 
-extern long __get_user_bad(void);
-extern void __put_user_bad(void);
+extern void noreturn __get_user_bad(void);
+extern void noreturn __put_user_bad(void);
 
 #define UA_KEEP(args...) args
 #define UA_DROP(args...)
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 14:08:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 14:08:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683538.1063103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcQmo-0006Wp-L8; Tue, 20 Feb 2024 14:07:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683538.1063103; Tue, 20 Feb 2024 14:07:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcQmo-0006Wi-HN; Tue, 20 Feb 2024 14:07:54 +0000
Received: by outflank-mailman (input) for mailman id 683538;
 Tue, 20 Feb 2024 14:07:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5vNk=J5=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rcQmn-0006Wc-Vu
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 14:07:54 +0000
Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com
 [2a00:1450:4864:20::32d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6febc2bd-cff9-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 15:07:52 +0100 (CET)
Received: by mail-wm1-x32d.google.com with SMTP id
 5b1f17b1804b1-41270d3ce26so2754835e9.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 06:07:52 -0800 (PST)
Received: from localhost.localdomain
 (cpc92320-cmbg19-2-0-cust35.5-4.cable.virginm.net. [82.13.64.36])
 by smtp.gmail.com with ESMTPSA id
 o20-20020a05600c511400b00410395dc7d1sm15205352wms.7.2024.02.20.06.07.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 06:07:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6febc2bd-cff9-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708438071; x=1709042871; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=BIiQPRJmSLHUwUlRUg6vt/WncPrgde1GXyHzWNmW+bk=;
        b=LraDZUIc7owc0KpaXtn4Nahjqb63i71p4Gy2CVwEkw+B34qpaeoArsEt6ny/DhYhAZ
         CI3+gxAxIY5Zp410obK0RgpcIKzMp0bFb7OyiJldROLfiRgIcvlaC6WxE7tuf0tmF7A6
         xFTyj5BXiB3VfeUU3VTtM3MWKm8XPbccv0xOE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708438071; x=1709042871;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=BIiQPRJmSLHUwUlRUg6vt/WncPrgde1GXyHzWNmW+bk=;
        b=gRhUXQsCma0SQwRKR9jynB5JiluFHafPBffMoXyCvjPeY/h8BLvF8D/1bxCsicQDYo
         tYqxkVqgx6qEouWfPa1XN6E2Z+vIoEJvE4fIOJC4ORfxjEY/M2DRLp/a08gyPj0kuutY
         9OLWI83IfVSA6SISqprRi09RLGJwyAkqPHGirvJgq54+E7iM0D7gyy8/9FIsYbgISJ6O
         pEd9qxUTAZ25Ofjyi50B+CYlCHBpRVX4gNl5IR4VaW1FSeJ607zPDja4SYFoXomxcgUj
         yhNRMu5KC4kLRhz858P0sYpqSJYM9uV8Ef4bMqCbXzlsVvL4lacUIlQrYhwSWto9fFh5
         caeA==
X-Gm-Message-State: AOJu0Yyx6s5yRD9pkrQydYxYBFPYiMagZb4Uup4+Gf4JO3DgLydnGfNW
	f++sBKwc0S4SlynnxY+iA3rhwt5ZJUJtoh9yK/cq9/hE7HRremIFhhDqqmeSnPX3rVk+R8RTNx/
	i
X-Google-Smtp-Source: AGHT+IHLEcLmUluugJILFyb4RL78DPOomw6Ma0CjQm6198f9gS5okqvmuHZX/eA/+utfFmN90LRFUw==
X-Received: by 2002:a05:600c:a39d:b0:410:ed31:9e96 with SMTP id hn29-20020a05600ca39d00b00410ed319e96mr11210113wmb.11.1708438071389;
        Tue, 20 Feb 2024 06:07:51 -0800 (PST)
From: Anthony PERARD <anthony.perard@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony.perard@citrix.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH] automation: Rework "build-each-commit-gcc" test
Date: Tue, 20 Feb 2024 14:07:33 +0000
Message-Id: <20240220140733.40980-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable

Current issues with this test are:
- when the job timeout, the log file is lost as there is no chance to
  run the `mv` command.
- GitLab job log is limited in size, so one usually have to download
  the artifacts, which may be missing.

Use $GITLAB_CI to detect when the script is run as part of a GitLab
pipeline. GitLab will add "GITLAB_CI=3Dtrue" in the environment
variables.

When run as part of $GITLAB_CI, ignore "dirty" worktree to allow to
write "build-each-commit-gcc.log", which can then be grabbed as
artifacts, even when the job timeout. The `git clean` command is
changed to keep those build logs.

When run as part of $GITLAB_CI, we will also store the build output in
a log file instead of writing it to stdout, because GitLab's job log
is limited. But we will write the log to stdout in case of error, so
we can find out more quickly why there's been an error.

This patch also make use of a GitLab feature, "log sections", which we
will collapse by default. One section per commit been built.

There's a bit of colour added to the logs.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    Examples:
    - on success:
      https://gitlab.com/xen-project/people/anthonyper/xen/-/jobs/6212972041
    - on failure:
      https://gitlab.com/xen-project/people/anthonyper/xen/-/jobs/6212993231

 automation/gitlab-ci/build-each-commit.sh |  2 +-
 automation/gitlab-ci/test.yaml            |  4 +-
 automation/scripts/build-test.sh          | 55 ++++++++++++++++++++---
 3 files changed, 50 insertions(+), 11 deletions(-)

diff --git a/automation/gitlab-ci/build-each-commit.sh b/automation/gitlab-=
ci/build-each-commit.sh
index 19e337b468..08fddecbbe 100755
--- a/automation/gitlab-ci/build-each-commit.sh
+++ b/automation/gitlab-ci/build-each-commit.sh
@@ -15,4 +15,4 @@ fi
 echo "Building ${BASE}..${TIP}"
=20
 NON_SYMBOLIC_REF=3D1 ./automation/scripts/build-test.sh ${BASE} ${TIP} \
-    bash -c "git clean -ffdx && ./automation/scripts/build"
+    bash -c "git clean -ffdx -e '/build-*.log' && ./automation/scripts/bui=
ld"
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 3b27cc9f41..50056c1372 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -136,9 +136,7 @@ build-each-commit-gcc:
     XEN_TARGET_ARCH: x86_64
     CC: gcc
   script:
-    - BASE=3D${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=3D${TIP_SHA:-${CI_CO=
MMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-=
each-commit-gcc.log
-  after_script:
-    - mv ../build-each-commit-gcc.log .
+    - BASE=3D${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=3D${TIP_SHA:-${CI_CO=
MMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee build-eac=
h-commit-gcc.log
   artifacts:
     paths:
       - '*.log'
diff --git a/automation/scripts/build-test.sh b/automation/scripts/build-te=
st.sh
index da643adc01..355d4bf7f6 100755
--- a/automation/scripts/build-test.sh
+++ b/automation/scripts/build-test.sh
@@ -9,6 +9,37 @@
 # Set NON_SYMBOLIC_REF=3D1 if you want to use this script in detached HEAD=
 state.
 # This is currently used by automated test system.
=20
+# Colors with ANSI escape sequences
+txt_info=3D'=1B[32m'
+txt_err=3D'=1B[31m'
+txt_clr=3D'=1B[0m'
+
+# $GITLAB_CI should be "true" or "false".
+if [ "$GITLAB_CI" !=3D true ]; then
+    GITLAB_CI=3Dfalse
+fi
+
+gitlab_log_section() {
+    if $GITLAB_CI; then
+        echo -n "=1B[0Ksection_$1:$(date +%s):$2=0D=1B[0K"
+    fi
+    if [ $# -ge 3 ]; then
+        echo "$3"
+    fi
+}
+log_section_last=3D
+log_section_start() {
+    log_section_last=3D"${1%\[collapsed=3Dtrue\]}"
+    gitlab_log_section 'start' "$1" "${txt_info}$2${txt_clr}"
+}
+log_section_end() {
+    if [ "$log_section_last" ]; then
+        gitlab_log_section 'end' "$log_section_last"
+        log_section_last=3D
+    fi
+}
+
+
 if test $# -lt 2 ; then
     echo "Usage:"
     echo " $0 <BASE> <TIP> [CMD]"
@@ -19,10 +50,12 @@ fi
=20
 pushd `git rev-parse --show-toplevel`
=20
-status=3D`git status -s`
-if test -n "$status"; then
-    echo "Tree is dirty, aborted"
-    exit 1
+if ! $GITLAB_CI; then
+    status=3D`git status -s`
+    if test -n "$status"; then
+        echo "Tree is dirty, aborted"
+        exit 1
+    fi
 fi
=20
 BASE=3D$1; shift
@@ -40,26 +73,34 @@ fi
=20
 ret=3D1
 while read num rev; do
-    echo "Testing $num $rev"
+    log_section_start "commit_$rev[collapsed=3Dtrue]" "Testing #$num $(git=
 log -1 --abbrev=3D12 --format=3Dtformat:'%h ("%s")' $rev)"
=20
     git checkout $rev
     ret=3D$?
     if test $ret -ne 0; then
-        echo "Failed to checkout $num $rev with $ret"
+        log_section_end
+        echo "${txt_err}Failed to checkout $num $rev with $ret${txt_clr}"
         break
     fi
=20
     if test $# -eq 0 ; then
         git clean -fdx && ./configure && make -j4
+    elif $GITLAB_CI; then
+        "$@" > "build-$num.log" 2>&1
     else
         "$@"
     fi
     ret=3D$?
     if test $ret -ne 0; then
-        echo "Failed at $num $rev with $ret"
+        if $GITLAB_CI; then
+            cat "build-$num.log"
+        fi
+        log_section_end
+        echo "${txt_err}Failed at $num $rev with $ret${txt_clr}"
         break
     fi
     echo
+    log_section_end
 done < <(git rev-list $BASE..$TIP | nl -ba | tac)
=20
 echo "Restoring original HEAD"
--=20
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 14:28:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 14:28:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683546.1063113 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcR6B-0000xQ-7a; Tue, 20 Feb 2024 14:27:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683546.1063113; Tue, 20 Feb 2024 14:27:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcR6B-0000xJ-3e; Tue, 20 Feb 2024 14:27:55 +0000
Received: by outflank-mailman (input) for mailman id 683546;
 Tue, 20 Feb 2024 14:27:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcR6A-0000x9-1N; Tue, 20 Feb 2024 14:27:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcR69-0003tt-S3; Tue, 20 Feb 2024 14:27:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcR68-0007OL-TG; Tue, 20 Feb 2024 14:27:52 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcR68-0002oa-St; Tue, 20 Feb 2024 14:27:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=74J2FmXwh18KMz1vg5t8C7pMhQGw+bU5cEzswbC3L6E=; b=tEU+oC8S9LyCpBO7HYf+g7JnUX
	EDHukXwY/V1VaxvVVXwyqkxlTto/57JI08et0siUMLX+TFcG9R56BhPRsG4GVtsu8BDIrjmExThAl
	p3nVLGIqO1BDidsh+0Q6zWehRcEcDSwZ10guIfWse2saUCjFsaBA3tteKUox9GUoWkt8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184711-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184711: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=981f722efa876e3f1766629f444d10e8806adc93
X-Osstest-Versions-That:
    xen=c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 20 Feb 2024 14:27:52 +0000

flight 184711 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184711/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  981f722efa876e3f1766629f444d10e8806adc93
baseline version:
 xen                  c144b9e32427ba37e0e0443a0d3fa53e9fb878b6

Last test of basis   184701  2024-02-19 12:03:49 Z    1 days
Testing same since   184711  2024-02-20 12:02:07 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   c144b9e324..981f722efa  981f722efa876e3f1766629f444d10e8806adc93 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 14:54:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 14:54:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683558.1063123 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRVW-0004dZ-8V; Tue, 20 Feb 2024 14:54:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683558.1063123; Tue, 20 Feb 2024 14:54:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRVW-0004dS-5Y; Tue, 20 Feb 2024 14:54:06 +0000
Received: by outflank-mailman (input) for mailman id 683558;
 Tue, 20 Feb 2024 14:54:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5vNk=J5=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rcRVV-0004dM-82
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 14:54:05 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e393cb32-cfff-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 15:54:03 +0100 (CET)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-33d6cc6d2fcso374273f8f.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 06:54:04 -0800 (PST)
Received: from localhost.localdomain
 (cpc92320-cmbg19-2-0-cust35.5-4.cable.virginm.net. [82.13.64.36])
 by smtp.gmail.com with ESMTPSA id
 14-20020a056000154e00b0033d10bd6612sm14050711wry.81.2024.02.20.06.54.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 06:54:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e393cb32-cfff-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708440843; x=1709045643; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=poJh6RFQdroxhR+KU74eDDRDn7U0Xmv5gjyyB0XXq3k=;
        b=ka0fcQUjyXFUbpVM0MfxPY2tLT+26osae8jkNJXHLbbicuq4dBSEosrpYKRxvLOAE4
         nVkrD3paJ6Lx6rhHRv9p2ftDt5r1Y0m7Uq5IrLEPXGqJ1mcAUFZZFWSH1jwI1TFY1EYw
         0n3L5z7HcxIjthZ6m665Mfo6m43kwOxTMrDoU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708440843; x=1709045643;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=poJh6RFQdroxhR+KU74eDDRDn7U0Xmv5gjyyB0XXq3k=;
        b=Jm5tHuXAK7+O4mZ84LCByhod1nfuOwhTCiS2HpCTgNzBALwNxUiSAoYu/c3PnFjsmJ
         yI256LKeF7lPaqpUbKdcdn8ZyXXJv+GhmiZDX2fhFaXPc4YZQdoxiVR4zZUQHNMzaCmo
         iSgM7kmS3gjTX9K7f4sR5A1Xzo3N9HVfxF/btlZKrDyMQjwmrWyw+GzGKEM/+YqFtSqo
         9URFke5vfxTzF2UxdKudSV4wZryknhpPAgKUggrTfLucCllX6olHLXTXqr/rkUo/QbFI
         uHMXGotb0dB/9ePOBhWLxGuKkJZ4F4mbAlp5F15OZl4clZTc7jNBCRCgRkqsDri+f6Ha
         36cg==
X-Gm-Message-State: AOJu0Yy1HAfpw+89VBAp5VGJ1YfRuHmSapF2aD2mZfsb/kdJ7fsfgP75
	SKJkUDEfqbN9nTSF9Y2ilgs1H3n+y4UuF6Qv5miYyq++SvwpWfC0R6bv0nexMFP5GBMtV7HWqGT
	/
X-Google-Smtp-Source: AGHT+IGrKLRi6pW1r/jgiBNLV+BFOpYI2xCfLoWEsSsup3/ZaqvK9juW8XlYASDtrVpphu/WR6sUSg==
X-Received: by 2002:a5d:408e:0:b0:33d:1585:b51c with SMTP id o14-20020a5d408e000000b0033d1585b51cmr8930119wrp.46.1708440843055;
        Tue, 20 Feb 2024 06:54:03 -0800 (PST)
From: Anthony PERARD <anthony.perard@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Roger Pau Monne <roger.pau@citrix.com>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [OSSTEST PATCH] ap-common: Switch to Linux 6.1 by default on Arm
Date: Tue, 20 Feb 2024 14:53:52 +0000
Message-Id: <20240220145352.26264-1-anthony.perard@citrix.com>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The "linux-arm-xen" tree currently point to Linux 5.4.17 + 3 commits:
  69117aaf07f7 ("xen/privcmd: replace kcalloc() by kvcalloc() when allocating empty pages")
  bca5bc0229a6 ("xen/privcmd: fix error handling in mmap-resource processing")
  f0f0e602f7c9 ("xen/privcmd: drop "pages" parameter from xen_remap_pfn()")

This stable version is old and doesn't build on Debian Bookworm.

The 3 extra commits are present in at least Linux 5.15, so there's
currently no reason to have a separate tree when we could use a stable
tree that osstest can update regulary.

So switch to 6.1. This will change the default Linux tree for all Arm
jobs that aren't on "linux-*" osstest branches.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    I plan to also change to use Linux 6.1 on X86 a bit later.

 ap-common | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ap-common b/ap-common
index 292553ed..48a95a22 100644
--- a/ap-common
+++ b/ap-common
@@ -60,7 +60,7 @@
 : ${PUSH_TREE_LINUX:=$XENBITS:/home/xen/git/linux-pvops.git}
 : ${BASE_TREE_LINUX:=git://xenbits.xen.org/linux-pvops.git}
 : ${BASE_TAG_LINUX:=tested/linux-4.19}
-: ${BASE_TAG_LINUX_ARM:=tested/linux-arm-xen}
+: ${BASE_TAG_LINUX_ARM:=tested/linux-6.1}
 
 if [ "x${TREE_LINUX}" = x ]; then
 	: ${TREE_LINUX:=${BASE_TREE_LINUX}}
-- 
Anthony PERARD



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 15:14:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 15:14:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683565.1063132 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRov-0007OE-Ry; Tue, 20 Feb 2024 15:14:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683565.1063132; Tue, 20 Feb 2024 15:14:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRov-0007O7-P8; Tue, 20 Feb 2024 15:14:09 +0000
Received: by outflank-mailman (input) for mailman id 683565;
 Tue, 20 Feb 2024 15:14:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nvC9=J5=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rcRou-0007O1-D8
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 15:14:08 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b0ecc3a7-d002-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 16:14:07 +0100 (CET)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a30f7c9574eso708769166b.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 07:14:07 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 jw2-20020a170906e94200b00a3f0d98c125sm474554ejb.155.2024.02.20.07.14.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 07:14:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b0ecc3a7-d002-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708442046; x=1709046846; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=H2TPE3JBhRfKrzD8Uz0RMfbxMcBRoHd2NveFlZmWBXc=;
        b=eJC8vgpj6reCDaRquS3ydXvi9uL2Ftg7AnINGZLTiSRNnc4QhrTxXAF7sOFGolCh8e
         sxyl23y73LkIh/oP93jQBVnHGVZFRyVa/qjKGr9CkQD6P8mqw59bOhSbvwellsJJ4tyz
         TEitb0RZYH3bIHKzgrykR+2SpLVYl7RDxvuDqMsmMI6rpqJPTVxk5AVBL3CUeUuE4jGD
         qXITLq5hwMdSGDOgf1caMTqnooQajDmaS4NATEWPTMVG/scpMZcsgDr0NltPLrydvypN
         XhIWxj6VVqKvWakROmz59oYUBmk6EoZfDyNtnQNr7xrRFPNzYBinQoc9nWJb+9Immbqb
         rpnw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708442046; x=1709046846;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=H2TPE3JBhRfKrzD8Uz0RMfbxMcBRoHd2NveFlZmWBXc=;
        b=K5pBJAbpNk8Sereuzt4pMVfyKo7xG6J1lmsCTRlbai1VxuXyQIBm2OmjSZmVtn3Fu6
         bLZ0TSesL526CtatOQn0CXg9hO+Rr7KY7saxi4ahTHsnpGBl8OgpDkoBqHcLA+wXOdgH
         WnZ0g8wCo4QLWfe+uJCK2CpqFt+Y9BQX579FPrhxPGSQ/tH0Yf7AMImOfZIsVeTUQizh
         /lr0AFPi1YepJ1qq5WyrPy4m3Tsp9IE3Uz0tCssCWZ9JG6K9O+ufKGrSkqCTikQNMEXB
         CD6skeWLhdZyZVJKCan5pwU8N+NiSV+ScV60D2YMGJpELKUMiuW0dSvVadCAyFLxJUO7
         F81g==
X-Gm-Message-State: AOJu0YzU6aMRuLUZYS2llwA76rZYsq4aA4npFiaA0eEyhD1gtWo2RQ2s
	ozvPv25DkZpuO+OxCgIwyx7/X1cRBNFRDSY+GA9rk5gSsa1Lag1XGNTzvLMW
X-Google-Smtp-Source: AGHT+IG4XoHtOWZLR2fMnnmr2h2K6DdbmI51sfL+KqY7WYPdlItpKQoS1t41/0L+5iZGKJ8NrPMVEw==
X-Received: by 2002:a17:906:4544:b0:a3e:71fb:348e with SMTP id s4-20020a170906454400b00a3e71fb348emr4579361ejq.63.1708442045914;
        Tue, 20 Feb 2024 07:14:05 -0800 (PST)
Message-ID: <172fd36027d4b95636d1ede472bec2792298ea97.camel@gmail.com>
Subject: Re: [PATCH v9 2/7] xen/asm-generic: introduce stub header monitor.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org, Shawn Anastasio
	 <sanastasio@raptorengineering.com>
Cc: Tamas K Lengyel <tamas@tklengyel.com>, Alexandru Isaila
 <aisaila@bitdefender.com>, Petre Pircalabu <ppircalabu@bitdefender.com>, 
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,  Michal Orzel
 <michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, Jan
 Beulich <jbeulich@suse.com>, Julien Grall <jgrall@amazon.com>
Date: Tue, 20 Feb 2024 16:14:04 +0100
In-Reply-To: <296530c13b8400f6872d4afc8cc14e2fcfc7a2d5.1708086092.git.oleksii.kurochko@gmail.com>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
	 <296530c13b8400f6872d4afc8cc14e2fcfc7a2d5.1708086092.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

Oleksii

On Fri, 2024-02-16 at 13:39 +0100, Oleksii Kurochko wrote:
> The header is shared between several archs so it is
> moved to asm-generic.
>=20
> Switch partly Arm and PPC to asm-generic/monitor.h and only
> arch_monitor_get_capabilities() left in arch-specific/monitor.h.
>=20
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
> Acked-by: Julien Grall <jgrall@amazon.com>
> ---
> Changes in V9:
> =C2=A0- Add Acked-by: Julien Grall <jgrall@amazon.com> for Arm part
> ---
> Changes in V8:
> =C2=A0- Add Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
> ---
> Changes in V7:
> =C2=A0- Drop definition of arch_monitor_domctl_event for PPC.
> =C2=A0- define arch_monitor_domctl_event in asm-generic/monitor.h.
> =C2=A0- add "define HAS_ARCH_MONITOR_DOMCTL_EVENT" in arm/.../monitor.h a=
s
> it has arch specific implementation.
> ---
> Changes in V6:
> =C2=A0- Rebase only.
> ---
> Changes in V5:
> =C2=A0 - Switched partly Arm and PPC to asm-generic monitor.h only
> =C2=A0=C2=A0=C2=A0 arch_monitor_get_capabilities() left in arch-specific/=
monitor.h.
> =C2=A0 - Updated the commit message.
> ---
> Changes in V4:
> =C2=A0- Removed the double blank line.
> =C2=A0- Added Acked-by: Jan Beulich <jbeulich@suse.com>.
> =C2=A0- Update the commit message
> ---
> Changes in V3:
> =C2=A0- Use forward-declaration of struct domain instead of " #include
> <xen/sched.h> ".
> =C2=A0- Add ' include <xen/errno.h> '
> =C2=A0- Drop PPC's monitor.h.
> ---
> Changes in V2:
> 	- remove inclusion of "+#include <public/domctl.h>"
> 	- add "struct xen_domctl_monitor_op;"
> 	- remove one of SPDX tags.
> ---
> =C2=A0xen/arch/arm/include/asm/monitor.h | 25 +-----------
> =C2=A0xen/arch/ppc/include/asm/monitor.h | 28 +------------
> =C2=A0xen/arch/ppc/stubs.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 8 ----
> =C2=A0xen/include/asm-generic/monitor.h=C2=A0 | 64
> ++++++++++++++++++++++++++++++
> =C2=A04 files changed, 67 insertions(+), 58 deletions(-)
> =C2=A0create mode 100644 xen/include/asm-generic/monitor.h
>=20
> diff --git a/xen/arch/arm/include/asm/monitor.h
> b/xen/arch/arm/include/asm/monitor.h
> index 7567be66bd..77a3c1a36c 100644
> --- a/xen/arch/arm/include/asm/monitor.h
> +++ b/xen/arch/arm/include/asm/monitor.h
> @@ -25,34 +25,13 @@
> =C2=A0#include <xen/sched.h>
> =C2=A0#include <public/domctl.h>
> =C2=A0
> -static inline
> -void arch_monitor_allow_userspace(struct domain *d, bool
> allow_userspace)
> -{
> -}
> +#define HAS_ARCH_MONITOR_DOMCTL_EVENT
> =C2=A0
> -static inline
> -int arch_monitor_domctl_op(struct domain *d, struct
> xen_domctl_monitor_op *mop)
> -{
> -=C2=A0=C2=A0=C2=A0 /* No arch-specific monitor ops on ARM. */
> -=C2=A0=C2=A0=C2=A0 return -EOPNOTSUPP;
> -}
> +#include <asm-generic/monitor.h>
> =C2=A0
> =C2=A0int arch_monitor_domctl_event(struct domain *d,
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct xen_domctl_monitor_op *mop);
> =C2=A0
> -static inline
> -int arch_monitor_init_domain(struct domain *d)
> -{
> -=C2=A0=C2=A0=C2=A0 /* No arch-specific domain initialization on ARM. */
> -=C2=A0=C2=A0=C2=A0 return 0;
> -}
> -
> -static inline
> -void arch_monitor_cleanup_domain(struct domain *d)
> -{
> -=C2=A0=C2=A0=C2=A0 /* No arch-specific domain cleanup on ARM. */
> -}
> -
> =C2=A0static inline uint32_t arch_monitor_get_capabilities(struct domain
> *d)
> =C2=A0{
> =C2=A0=C2=A0=C2=A0=C2=A0 uint32_t capabilities =3D 0;
> diff --git a/xen/arch/ppc/include/asm/monitor.h
> b/xen/arch/ppc/include/asm/monitor.h
> index e5b0282bf1..89000dacc6 100644
> --- a/xen/arch/ppc/include/asm/monitor.h
> +++ b/xen/arch/ppc/include/asm/monitor.h
> @@ -6,33 +6,7 @@
> =C2=A0#include <public/domctl.h>
> =C2=A0#include <xen/errno.h>
> =C2=A0
> -static inline
> -void arch_monitor_allow_userspace(struct domain *d, bool
> allow_userspace)
> -{
> -}
> -
> -static inline
> -int arch_monitor_domctl_op(struct domain *d, struct
> xen_domctl_monitor_op *mop)
> -{
> -=C2=A0=C2=A0=C2=A0 /* No arch-specific monitor ops on PPC. */
> -=C2=A0=C2=A0=C2=A0 return -EOPNOTSUPP;
> -}
> -
> -int arch_monitor_domctl_event(struct domain *d,
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct xen_domctl_monitor_op *mop);
> -
> -static inline
> -int arch_monitor_init_domain(struct domain *d)
> -{
> -=C2=A0=C2=A0=C2=A0 /* No arch-specific domain initialization on PPC. */
> -=C2=A0=C2=A0=C2=A0 return 0;
> -}
> -
> -static inline
> -void arch_monitor_cleanup_domain(struct domain *d)
> -{
> -=C2=A0=C2=A0=C2=A0 /* No arch-specific domain cleanup on PPC. */
> -}
> +#include <asm-generic/monitor.h>
> =C2=A0
> =C2=A0static inline uint32_t arch_monitor_get_capabilities(struct domain
> *d)
> =C2=A0{
> diff --git a/xen/arch/ppc/stubs.c b/xen/arch/ppc/stubs.c
> index a96e45626d..da193839bd 100644
> --- a/xen/arch/ppc/stubs.c
> +++ b/xen/arch/ppc/stubs.c
> @@ -95,14 +95,6 @@ void arch_get_domain_info(const struct domain *d,
> =C2=A0=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> =C2=A0}
> =C2=A0
> -/* monitor.c */
> -
> -int arch_monitor_domctl_event(struct domain *d,
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct xen_domctl_monitor_op *mop)
> -{
> -=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> -}
> -
> =C2=A0/* smp.c */
> =C2=A0
> =C2=A0void arch_flush_tlb_mask(const cpumask_t *mask)
> diff --git a/xen/include/asm-generic/monitor.h b/xen/include/asm-
> generic/monitor.h
> new file mode 100644
> index 0000000000..1ade289099
> --- /dev/null
> +++ b/xen/include/asm-generic/monitor.h
> @@ -0,0 +1,64 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * include/asm-generic/monitor.h
> + *
> + * Arch-specific monitor_op domctl handler.
> + *
> + * Copyright (c) 2015 Tamas K Lengyel (tamas@tklengyel.com)
> + * Copyright (c) 2016, Bitdefender S.R.L.
> + *
> + */
> +
> +#ifndef __ASM_GENERIC_MONITOR_H__
> +#define __ASM_GENERIC_MONITOR_H__
> +
> +#include <xen/errno.h>
> +#include <xen/lib.h>
> +
> +struct domain;
> +struct xen_domctl_monitor_op;
> +
> +static inline
> +void arch_monitor_allow_userspace(struct domain *d, bool
> allow_userspace)
> +{
> +}
> +
> +static inline
> +int arch_monitor_domctl_op(struct domain *d, struct
> xen_domctl_monitor_op *mop)
> +{
> +=C2=A0=C2=A0=C2=A0 /* No arch-specific monitor ops on GENERIC. */
> +=C2=A0=C2=A0=C2=A0 return -EOPNOTSUPP;
> +}
> +
> +#ifndef HAS_ARCH_MONITOR_DOMCTL_EVENT
> +static inline
> +int arch_monitor_domctl_event(struct domain *d,
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct xen_domctl_monitor_op *mop)
> +{
> +=C2=A0=C2=A0=C2=A0 BUG_ON("unimplemented");
> +}
> +#endif
> +
> +static inline
> +int arch_monitor_init_domain(struct domain *d)
> +{
> +=C2=A0=C2=A0=C2=A0 /* No arch-specific domain initialization on GENERIC.=
 */
> +=C2=A0=C2=A0=C2=A0 return 0;
> +}
> +
> +static inline
> +void arch_monitor_cleanup_domain(struct domain *d)
> +{
> +=C2=A0=C2=A0=C2=A0 /* No arch-specific domain cleanup on GENERIC. */
> +}
> +
> +#endif /* __ASM_GENERIC_MONITOR_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: BSD
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 15:14:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 15:14:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683570.1063144 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRpP-0007sv-9V; Tue, 20 Feb 2024 15:14:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683570.1063144; Tue, 20 Feb 2024 15:14:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRpP-0007so-4X; Tue, 20 Feb 2024 15:14:39 +0000
Received: by outflank-mailman (input) for mailman id 683570;
 Tue, 20 Feb 2024 15:14:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nvC9=J5=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rcRpN-0007kg-Tb
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 15:14:37 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c23201e9-d002-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 16:14:36 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a3e7f7b3d95so297051566b.3
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 07:14:36 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 vb7-20020a170907d04700b00a3dabd0d000sm3997443ejc.209.2024.02.20.07.14.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 07:14:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c23201e9-d002-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708442075; x=1709046875; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:to:from:subject:message-id:from:to:cc:subject:date
         :message-id:reply-to;
        bh=eCo217VIQDdyMozUTvAlDqcrb3LIxMx88qr1WrJ9aH4=;
        b=ImCLZZXcCo5NdRouzLfxSrj/6NRcCsbJtYC0GFVHq3MHx3PC/Ds33uh8QDkJkJCqHG
         FDhDmEMDZ5/dszvmcFjWzXQqoDG4umpUzz2amYb8l2M+e0V7Ddde3NnbHpt5l/PLlTAk
         NzcwFEN4HU9WYxCCLFu6szHWJy5DLoCFgfcNTlorvbbHNEFDVPHN9nSXp/AZcxuVmzum
         lC/vOCauMFcuAOGAd/L5hx1no7JtgWV37cClGSnB4wRyr+YOuIir3W53j75sOLuagjvo
         +nlkKc+ajCTSXweHkh11kfwXmtzEERUTN6iAfPsoQHQ9tyQ4SKGywjGROdhTMvlNnWtE
         zvxQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708442075; x=1709046875;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:to:from:subject:message-id:x-gm-message-state:from
         :to:cc:subject:date:message-id:reply-to;
        bh=eCo217VIQDdyMozUTvAlDqcrb3LIxMx88qr1WrJ9aH4=;
        b=NbLNgx9GwCrCqjLScyHrYZd815zOhRxGqvl4YSkVYfNwSJU0fS9BdX7JpzNHOp/Fq8
         WyjkqQshfRp/tj1p8ByOGJ71NgyJ+DDZSKHo8X9LepVIsoUHEMZmhT6di1QgPOdBJXFQ
         eTbzVsqXQePzZ/jMjz6T8VB+DGKqPl97xgEMNIZ4A6zvXm59CAwUqoaYYg4YMTvbwxBZ
         TVvfPSbuMHx8imCwC7bUqTa569YQKS4Z0ijNYhSpY16aGV9JhnvtPf0cFTwGWNRznEsm
         pvFdRV9WryNPXJI+8LwAjmxg+fVq0J/kLvx37FjN35zahTMafE8j2GY+aMfZHmVpmJ+C
         bAOg==
X-Gm-Message-State: AOJu0Ywh7ION1vxjvDm4T2GyEzZ+LfY0cloteKuXV5dibS4hAkhmnzVC
	LSujYhR5DvXwoNHyBAtHSNfkJg6LC+q9O1O6cOc8ZKZnHwcRb6lpZjEgCABA
X-Google-Smtp-Source: AGHT+IHdcPnjyR9dZN/l51OaGsncFKH8H69QwSlDLhsizRrbN3tqUurK4a2/H7nzIbUPZ3nnPF2gyg==
X-Received: by 2002:a17:906:40d8:b0:a3e:e678:556 with SMTP id a24-20020a17090640d800b00a3ee6780556mr2217599ejk.58.1708442075022;
        Tue, 20 Feb 2024 07:14:35 -0800 (PST)
Message-ID: <c3efe22792dd732522b1a9240f8807f53756e6c0.camel@gmail.com>
Subject: Re: [PATCH v9 6/7] xen/ppc: switch PPC to use asm-generic/device.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org, Shawn Anastasio
	 <sanastasio@raptorengineering.com>
Date: Tue, 20 Feb 2024 16:14:34 +0100
In-Reply-To: <222ae66639312712e145bd04d5dd1ebd2c511d43.1708086092.git.oleksii.kurochko@gmail.com>
References: <cover.1708086091.git.oleksii.kurochko@gmail.com>
	 <222ae66639312712e145bd04d5dd1ebd2c511d43.1708086092.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

SGVsbG8gU2hhd24sCgpXb3VsZCB5b3UgbWluZCB0YWtpbmcgYSBsb29rIGF0IHRoaXMgcGF0Y2g/
CgpUaGFua3MgaW4gYWR2YW5jZS4KCkJlc3QgcmVnYXJkcywKIE9sZWtzaWkKCk9uIEZyaSwgMjAy
NC0wMi0xNiBhdCAxMzozOSArMDEwMCwgT2xla3NpaSBLdXJvY2hrbyB3cm90ZToKPiBTaWduZWQt
b2ZmLWJ5OiBPbGVrc2lpIEt1cm9jaGtvIDxvbGVrc2lpLmt1cm9jaGtvQGdtYWlsLmNvbT4KPiAt
LS0KPiBDaGFuZ2VkIGluIFY4Ogo+IMKgLSBub3RoaW5nIGNoYW5nZWQuIG9ubHkgcmViYXNlLgo+
IC0tLQo+IENoYW5nZXMgaW4gVjc6Cj4gwqAtIG5ld2x5IGludHJvZHVjZWQgcGF0Y2ggd2hpY2gg
aXMgYmFzZWQgb24gdGhlIHByZXZpb3VzIHZlcnNpb24gb2YKPiB0aGUgcGF0Y2g6Cj4gwqDCoMKg
wqAgW1BBVENIIHY2IDkvOV0geGVuL2FzbS1nZW5lcmljOiBpbnRyb2R1Y2UgZ2VuZXJpYyBkZXZp
Y2UuaAo+IC0tLQo+IMKgeGVuL2FyY2gvcHBjL2luY2x1ZGUvYXNtL01ha2VmaWxlIHzCoCAxICsK
PiDCoHhlbi9hcmNoL3BwYy9pbmNsdWRlL2FzbS9kZXZpY2UuaCB8IDUzIC0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tCj4gLS0KPiDCoDIgZmlsZXMgY2hhbmdlZCwgMSBpbnNlcnRpb24oKyks
IDUzIGRlbGV0aW9ucygtKQo+IMKgZGVsZXRlIG1vZGUgMTAwNjQ0IHhlbi9hcmNoL3BwYy9pbmNs
dWRlL2FzbS9kZXZpY2UuaAo+IAo+IGRpZmYgLS1naXQgYS94ZW4vYXJjaC9wcGMvaW5jbHVkZS9h
c20vTWFrZWZpbGUKPiBiL3hlbi9hcmNoL3BwYy9pbmNsdWRlL2FzbS9NYWtlZmlsZQo+IGluZGV4
IDNmZDg5M2YzZTAuLmNlZDAyZTI2ZWQgMTAwNjQ0Cj4gLS0tIGEveGVuL2FyY2gvcHBjL2luY2x1
ZGUvYXNtL01ha2VmaWxlCj4gKysrIGIveGVuL2FyY2gvcHBjL2luY2x1ZGUvYXNtL01ha2VmaWxl
Cj4gQEAgLTEsNSArMSw2IEBACj4gwqAjIFNQRFgtTGljZW5zZS1JZGVudGlmaWVyOiBHUEwtMi4w
LW9ubHkKPiDCoGdlbmVyaWMteSArPSBhbHRwMm0uaAo+ICtnZW5lcmljLXkgKz0gZGV2aWNlLmgK
PiDCoGdlbmVyaWMteSArPSBkaXY2NC5oCj4gwqBnZW5lcmljLXkgKz0gaGFyZGlycS5oCj4gwqBn
ZW5lcmljLXkgKz0gaHlwZXJjYWxsLmgKPiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvcHBjL2luY2x1
ZGUvYXNtL2RldmljZS5oCj4gYi94ZW4vYXJjaC9wcGMvaW5jbHVkZS9hc20vZGV2aWNlLmgKPiBk
ZWxldGVkIGZpbGUgbW9kZSAxMDA2NDQKPiBpbmRleCA4MjUzZTYxZDUxLi4wMDAwMDAwMDAwCj4g
LS0tIGEveGVuL2FyY2gvcHBjL2luY2x1ZGUvYXNtL2RldmljZS5oCj4gKysrIC9kZXYvbnVsbAo+
IEBAIC0xLDUzICswLDAgQEAKPiAtLyogU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAt
b25seSAqLwo+IC0jaWZuZGVmIF9fQVNNX1BQQ19ERVZJQ0VfSF9fCj4gLSNkZWZpbmUgX19BU01f
UFBDX0RFVklDRV9IX18KPiAtCj4gLWVudW0gZGV2aWNlX3R5cGUKPiAtewo+IC3CoMKgwqAgREVW
X0RULAo+IC3CoMKgwqAgREVWX1BDSSwKPiAtfTsKPiAtCj4gLXN0cnVjdCBkZXZpY2Ugewo+IC3C
oMKgwqAgZW51bSBkZXZpY2VfdHlwZSB0eXBlOwo+IC0jaWZkZWYgQ09ORklHX0hBU19ERVZJQ0Vf
VFJFRQo+IC3CoMKgwqAgc3RydWN0IGR0X2RldmljZV9ub2RlICpvZl9ub2RlOyAvKiBVc2VkIGJ5
IGRyaXZlcnMgaW1wb3J0ZWQgZnJvbQo+IExpbnV4ICovCj4gLSNlbmRpZgo+IC19Owo+IC0KPiAt
ZW51bSBkZXZpY2VfY2xhc3MKPiAtewo+IC3CoMKgwqAgREVWSUNFX1NFUklBTCwKPiAtwqDCoMKg
IERFVklDRV9JT01NVSwKPiAtwqDCoMKgIERFVklDRV9QQ0lfSE9TVEJSSURHRSwKPiAtwqDCoMKg
IC8qIFVzZSBmb3IgZXJyb3IgKi8KPiAtwqDCoMKgIERFVklDRV9VTktOT1dOLAo+IC19Owo+IC0K
PiAtc3RydWN0IGRldmljZV9kZXNjIHsKPiAtwqDCoMKgIC8qIERldmljZSBuYW1lICovCj4gLcKg
wqDCoCBjb25zdCBjaGFyICpuYW1lOwo+IC3CoMKgwqAgLyogRGV2aWNlIGNsYXNzICovCj4gLcKg
wqDCoCBlbnVtIGRldmljZV9jbGFzcyBjbGFzczsKPiAtwqDCoMKgIC8qIExpc3Qgb2YgZGV2aWNl
cyBzdXBwb3J0ZWQgYnkgdGhpcyBkcml2ZXIgKi8KPiAtwqDCoMKgIGNvbnN0IHN0cnVjdCBkdF9k
ZXZpY2VfbWF0Y2ggKmR0X21hdGNoOwo+IC3CoMKgwqAgLyoKPiAtwqDCoMKgwqAgKiBEZXZpY2Ug
aW5pdGlhbGl6YXRpb24uCj4gLcKgwqDCoMKgICoKPiAtwqDCoMKgwqAgKiAtRUFHQUlOIGlzIHVz
ZWQgdG8gaW5kaWNhdGUgdGhhdCBkZXZpY2UgcHJvYmluZyBpcyBkZWZlcnJlZC4KPiAtwqDCoMKg
wqAgKi8KPiAtwqDCoMKgIGludCAoKmluaXQpKHN0cnVjdCBkdF9kZXZpY2Vfbm9kZSAqZGV2LCBj
b25zdCB2b2lkICpkYXRhKTsKPiAtfTsKPiAtCj4gLXR5cGVkZWYgc3RydWN0IGRldmljZSBkZXZp
Y2VfdDsKPiAtCj4gLSNkZWZpbmUgRFRfREVWSUNFX1NUQVJUKG5hbWVfLCBuYW1lc3RyXywgY2xh
c3NfKcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgCj4gXAo+IC1zdGF0aWMg
Y29uc3Qgc3RydWN0IGRldmljZV9kZXNjIF9fZGV2X2Rlc2NfIyNuYW1lXyBfX3VzZWTCoMKgwqDC
oMKgwqDCoMKgwqDCoAo+IFwKPiAtX19zZWN0aW9uKCIuZGV2LmluZm8iKSA9IHvCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgCj4gXAo+IC3CoMKgwqAgLm5hbWUgPSBuYW1lc3RyXyzCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoAo+IFwKPiAtwqDCoMKgIC5jbGFzcyA9IGNsYXNzXyzC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgCj4gXAo+IC0KPiAtI2RlZmluZSBE
VF9ERVZJQ0VfRU5EwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAKPiBcCj4gLX07
Cj4gLQo+IC0jZW5kaWYgLyogX19BU01fUFBDX0RFVklDRV9IX18gKi8KCg==



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 15:18:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 15:18:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683577.1063152 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRsx-00006m-M7; Tue, 20 Feb 2024 15:18:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683577.1063152; Tue, 20 Feb 2024 15:18:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRsx-00006f-JW; Tue, 20 Feb 2024 15:18:19 +0000
Received: by outflank-mailman (input) for mailman id 683577;
 Tue, 20 Feb 2024 15:18:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YSgb=J5=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcRsw-00006Z-O9
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 15:18:18 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 457c7aba-d003-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 16:18:16 +0100 (CET)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-563e330351dso4787254a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 07:18:16 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i16-20020a0564020f1000b0056411b3fc4bsm3731051eda.30.2024.02.20.07.18.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 07:18:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 457c7aba-d003-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708442296; x=1709047096; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=5hib8pgheuZH5W/3YrL7nsCGE8Fb2vhnUkezVvlGO6E=;
        b=fRT9wwCjoKSAz62MtP6zjuNsNxHCz+iMvcda0qGKsGVbWqVFt0V/DTuazOLH35VQVY
         NGFqS4Knr1klIT5ogMdVsa8zBpiL+D+oE1lIrmYunjIPt72si4umbA8m/IH7uqCkWE6K
         h5eMy/bvKssTBymI1zOATkWxQDCdwQ0fVzkn4zluu0nLPx7c0/VVO5RRjYE1AlB9UqiO
         nKNLHG9YD2Wnw28jizz9++scCFowYUrJNV2H8HZ8WJYCGdQmDdKQ7jqHwxioNwGIo7MS
         Ul7WCKgurWxyv/u7Ompn2aym4NR8sMyvRm4txE61HTyYoalQ6QTtelRfinmLgjqagVNU
         cM+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708442296; x=1709047096;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5hib8pgheuZH5W/3YrL7nsCGE8Fb2vhnUkezVvlGO6E=;
        b=VobUGRu+DhfjWnuYjfUPo9eMVb7CG/1cZVhFR2vJAY95OXiqbADFBLfQRAd0SYsdlw
         CxY6kiesVzN3xL+NxpQQsCUn+dmGD660eUZ/DuqgUG1EGdScBHlLskwENHLXUaHdmTih
         rB7xY2XZ4bl9u8idnWrz/GzXfQ84KDvGG/aTW5qq6xLAp0Iwig5pnTnFoNQ0ijDp7Kqw
         xQY/CXPMhpggS4fE8O9sJ6WwZDkWiRWNFUx6BtzFYcW6vueSMVNpuQBINTGaqkxljFcb
         7QOR6F5zjCnfdr7olh4alXqy6oLUZzLzJrLCeNW9OtSGKDFaTJZFbNe9gOyv8izFCrrf
         ZVIw==
X-Forwarded-Encrypted: i=1; AJvYcCWFyzuXPHEvjutjw92FHoy7soIFkSbC6wQF8L0YzCe+9j7Ss/6y9/S/+7msH97oILdf1ysu7QEPgUL6/QlfbefIkvt4EVmvXteRtCfb7cU=
X-Gm-Message-State: AOJu0YwNUxdefNupCk5RVOQX5aMkGdvBJIwsZ0I67T75FK0XrEdi0RYt
	vDw7/mbNNjZrCPiK09w8cpvozChV/vox0UjuacgMW/mYSYazZWVPXuVr7taTtA==
X-Google-Smtp-Source: AGHT+IF41Sl+oL0ivwp+lQ5LmNlH7AdQjVyUU295wIGDzZgHGrP02focl0uVOEIj1OJTfgmJxjR0vw==
X-Received: by 2002:aa7:c383:0:b0:564:1c4e:eb57 with SMTP id k3-20020aa7c383000000b005641c4eeb57mr6714101edq.17.1708442296083;
        Tue, 20 Feb 2024 07:18:16 -0800 (PST)
Message-ID: <d2c4d6e9-a776-4dd2-aff6-22081da2f9cf@suse.com>
Date: Tue, 20 Feb 2024 16:18:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] x86/uaccess: add attribute noreturn to
 __{get,put}_user_bad()
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <1595eac56587d20c7f86128bc5652c31c3a72772.1708436010.git.federico.serafini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <1595eac56587d20c7f86128bc5652c31c3a72772.1708436010.git.federico.serafini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 20.02.2024 14:35, Federico Serafini wrote:
> __get_user_bad() and __put_user_bad() are undefined symbols used
> to assert the unreachability of a program point:
> a call to one of such functions is optimized away if it is considered
> unreachable by the compiler. Otherwise, a linker error is reported.
> 
> In accordance with the purpose of such constructs:
> 1) add the attribute noreturn to __get_user_bad() and __put_user_bad();
> 2) change return type of __get_user_bad() to void (returning long is a
>    leftover from the past).
> 
> Point (1) meets the requirements to deviate MISRA C:2012 Rule 16.3
> ("An unconditional break statement shall terminate every switch
> clause") since functions with noreturn attribute are considered
> as allowed terminals for switch clauses.
> 
> Point (2) addresses several violations of MISRA C:2012 Rule 17.7
> ("The value returned by a function having non-void return type
> shall be used").
> 
> No functional change.
> 
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
preferably ...

> --- a/xen/arch/x86/include/asm/uaccess.h
> +++ b/xen/arch/x86/include/asm/uaccess.h
> @@ -21,8 +21,8 @@ unsigned int copy_from_guest_ll(void *to, const void __user *from, unsigned int
>  unsigned int copy_to_unsafe_ll(void *to, const void *from, unsigned int n);
>  unsigned int copy_from_unsafe_ll(void *to, const void *from, unsigned int n);
>  
> -extern long __get_user_bad(void);
> -extern void __put_user_bad(void);
> +extern void noreturn __get_user_bad(void);
> +extern void noreturn __put_user_bad(void);

... with the "extern" dropped at the same time (see other function
decls in context). I'll try to remember to drop them while committing.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 15:18:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 15:18:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683578.1063163 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRt6-0000Ps-VE; Tue, 20 Feb 2024 15:18:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683578.1063163; Tue, 20 Feb 2024 15:18:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRt6-0000Pj-Qd; Tue, 20 Feb 2024 15:18:28 +0000
Received: by outflank-mailman (input) for mailman id 683578;
 Tue, 20 Feb 2024 15:18:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nvC9=J5=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rcRt5-0000Oq-EK
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 15:18:27 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4b03358f-d003-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 16:18:26 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d244b28b95so14496431fa.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 07:18:25 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 f34-20020a0565123b2200b00512aa52a37esm1090390lfv.51.2024.02.20.07.18.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 07:18:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4b03358f-d003-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708442304; x=1709047104; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=gkavrmItIQeqPwKMSOFu6cj3eEHdZbxS4SZMMZneueo=;
        b=eJKEKgyAJdXAPuT5dMvjlIrwhNbQJxp0qvT7/r4OUEo67aRsjulmCdS9jdcrEaPh+M
         aP6eQsgtuuj7E4JM8QXx+JA/2nUsdgFI9FhaS58bJpMTWZOh1ENeupk42gkVN9pFaeVN
         8X9KUf0WqitIGcfBcqaavIyQnniuvMS+McNTBQ1cPd8VqF/UBIruAOR31OqcNeumWabh
         XeU/fHVG2F9aW4ALjBggEbWoE0sowOwQ2VD+/u7Fq3spCVaIYJpBPrJrhjLJyVtkJJ5j
         pNZ3x9JdiL2rHY4+058MJYPP8aW9pRuRUUv2UcvBzAtN+724euppSro7kvcuaRgJHewK
         9FWQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708442304; x=1709047104;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gkavrmItIQeqPwKMSOFu6cj3eEHdZbxS4SZMMZneueo=;
        b=ZzHjTDBCqL3pkV8CDt+FFYFahndcdAZU6pa1ei1maywXlIrbodHpxhN3KdNePxJk5g
         fOiOVcrhflnlWtXp0TEZXdSoAyNnSf9jNjtP7OLVG8CaAkkP2aVnnD9kp6UfUQNuhHZS
         KnMbPZbHN0ekGTLX4YZ/W4/fpOA2Rr8TQ5EgyLMJiXvX2Arf/OugaaLbKf83kYQ2Was3
         hJfrHUV6tfoUn3jWos6sGIdKU6MpJgiTxm1MlO+a/ybEEqL1kCG0pJXY1tCWVC+zchZX
         lSbop7EpV0LhQJAM6XTfNzSGfWGlbDSxnVXoJo61fo6DPG1UXvnuL+sLRISDTVxaI0i7
         Hg7w==
X-Gm-Message-State: AOJu0YykySO2hAx/B65hbzHoUEqtJoURvSbjrWiXCUpNiL6bjvCQdpFg
	u/77W+F0QRBJJ9TpJkpbITUFbmil5SQWhERVlBbMHFtYCH75jdefsJr0mPaM
X-Google-Smtp-Source: AGHT+IEOWcKv59Z3JJNpBcpFd4rE0ewvVsYjebTuSFUDcCHmUOhM2dCheuwfR8QKBWZgBA+/aUL3YQ==
X-Received: by 2002:a05:6512:2342:b0:512:c1ba:9115 with SMTP id p2-20020a056512234200b00512c1ba9115mr2058064lfu.57.1708442304311;
        Tue, 20 Feb 2024 07:18:24 -0800 (PST)
Message-ID: <ee34f7eb61a1f09c18e3d5ed915e189087135a9a.camel@gmail.com>
Subject: Re: [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(),
 ASSERT_UNREACHABLE() to xen/bug.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org, Shawn Anastasio
	 <sanastasio@raptorengineering.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>,  Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>, Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>, "Daniel P. Smith"
 <dpsmith@apertussolutions.com>
Date: Tue, 20 Feb 2024 16:18:23 +0100
In-Reply-To: <4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
References: 
	<4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

Hello Shawn,

Could you please take a look at the patch and PPC-related changes in
xen/arch/ppc/include/asm/time.h ?

Thanks in advance.

~ Oleksii

On Fri, 2024-02-02 at 20:03 +0200, Oleksii Kurochko wrote:
> Move the macros mentioned in the commit subject to their appropriate
> locations.
> Additionally, eliminate the dependency of xen/lib.h from xen/bug.h
> and
> include "xen/bug.h" in files where xen/bug.h macros are utilized.
>=20
> Most of the changes were made because a file requires macros from
> xen/bug.h,
> except for some files for Arm which require definitions of
> BUG_OPCODE,
> BUG_INSTR, BUG_FN_REG.
>=20
> xen/lib.h was added to list-sort.c ( otherwise compilation errors
> related
> to {d}printk occur during compilation of list-sort.c. ) as xen/lib.h
> was
> removed from xen/list.h. Since nothing in xen/list.h depends on
> xen/lib.h
> functionality and only xen/bug.h is needed.
>=20
> cpufeature.h requires the inclusion of <xen/cache.h>;
> otherwise, the following error will occur:
> ld:
> common/monitor.o:/build/xen/./arch/x86/include/asm/cpufeature.h:41:
> multiple definitions of `__cacheline_aligned';
>=20
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> =C2=A0xen/arch/arm/arm32/insn.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 3 ++-
> =C2=A0xen/arch/arm/arm64/cpufeature.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 |=C2=A0 1 +
> =C2=A0xen/arch/arm/arm64/insn.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 1 +
> =C2=A0xen/arch/arm/cpufeature.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 1 +
> =C2=A0xen/arch/arm/include/asm/arm32/cmpxchg.h |=C2=A0 1 +
> =C2=A0xen/arch/arm/include/asm/arm64/cmpxchg.h |=C2=A0 2 ++
> =C2=A0xen/arch/arm/include/asm/regs.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 |=C2=A0 2 +-
> =C2=A0xen/arch/arm/include/asm/vgic.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 |=C2=A0 1 +
> =C2=A0xen/arch/ppc/include/asm/time.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 |=C2=A0 2 +-
> =C2=A0xen/arch/x86/bitops.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=
=A0 2 +-
> =C2=A0xen/arch/x86/include/asm/cpufeature.h=C2=A0=C2=A0=C2=A0 |=C2=A0 1 +
> =C2=A0xen/arch/x86/include/asm/system.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 |=C2=A0 2 +-
> =C2=A0xen/arch/x86/include/asm/x86_64/page.h=C2=A0=C2=A0 |=C2=A0 2 ++
> =C2=A0xen/arch/x86/x86_emulate/private.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 |=C2=A0 1 +
> =C2=A0xen/common/efi/common-stub.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 2 +-
> =C2=A0xen/common/version.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 |=C2=A0 1 +
> =C2=A0xen/include/public/hvm/save.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 2 +-
> =C2=A0xen/include/xen/bug.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 19 =
+++++++++++++++++++
> =C2=A0xen/include/xen/cpumask.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 1 +
> =C2=A0xen/include/xen/device_tree.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 1 +
> =C2=A0xen/include/xen/lib.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 19 =
-------------------
> =C2=A0xen/include/xen/list.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 2 +=
-
> =C2=A0xen/include/xen/livepatch.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 2 ++
> =C2=A0xen/include/xen/mm.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
 |=C2=A0 1 +
> =C2=A0xen/include/xen/param.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 1 +
> =C2=A0xen/lib/list-sort.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 |=C2=A0 1 +
> =C2=A0xen/xsm/flask/ss/ebitmap.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 1 +
> =C2=A027 files changed, 48 insertions(+), 27 deletions(-)
>=20
> diff --git a/xen/arch/arm/arm32/insn.c b/xen/arch/arm/arm32/insn.c
> index 49953a042a..2a62bb9cce 100644
> --- a/xen/arch/arm/arm32/insn.c
> +++ b/xen/arch/arm/arm32/insn.c
> @@ -13,8 +13,9 @@
> =C2=A0=C2=A0 * You should have received a copy of the GNU General Public
> License
> =C2=A0=C2=A0 * along with this program.=C2=A0 If not, see
> <http://www.gnu.org/licenses/>.
> =C2=A0=C2=A0 */
> -#include <xen/lib.h>
> =C2=A0#include <xen/bitops.h>
> +#include <xen/bug.h>
> +#include <xen/lib.h>
> =C2=A0#include <xen/sizes.h>
> =C2=A0#include <asm/insn.h>
> =C2=A0
> diff --git a/xen/arch/arm/arm64/cpufeature.c
> b/xen/arch/arm/arm64/cpufeature.c
> index b4656ff4d8..864413d9cc 100644
> --- a/xen/arch/arm/arm64/cpufeature.c
> +++ b/xen/arch/arm/arm64/cpufeature.c
> @@ -69,6 +69,7 @@
> =C2=A0 *=C2=A0=C2=A0 KVM guests.
> =C2=A0 */
> =C2=A0
> +#include <xen/bug.h>
> =C2=A0#include <xen/types.h>
> =C2=A0#include <xen/kernel.h>
> =C2=A0#include <asm/sysregs.h>
> diff --git a/xen/arch/arm/arm64/insn.c b/xen/arch/arm/arm64/insn.c
> index 22f2bdebd5..773c3749d1 100644
> --- a/xen/arch/arm/arm64/insn.c
> +++ b/xen/arch/arm/arm64/insn.c
> @@ -18,6 +18,7 @@
> =C2=A0 * You should have received a copy of the GNU General Public Licens=
e
> =C2=A0 * along with this program.=C2=A0 If not, see
> <http://www.gnu.org/licenses/>.
> =C2=A0 */
> +#include <xen/bug.h>
> =C2=A0#include <xen/types.h>
> =C2=A0#include <xen/lib.h>
> =C2=A0#include <xen/errno.h>
> diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
> index f43d5cb338..ef77473bf8 100644
> --- a/xen/arch/arm/cpufeature.c
> +++ b/xen/arch/arm/cpufeature.c
> @@ -5,6 +5,7 @@
> =C2=A0 * Copyright (C) 2015 ARM Ltd.
> =C2=A0 */
> =C2=A0
> +#include <xen/bug.h>
> =C2=A0#include <xen/types.h>
> =C2=A0#include <xen/init.h>
> =C2=A0#include <xen/smp.h>
> diff --git a/xen/arch/arm/include/asm/arm32/cmpxchg.h
> b/xen/arch/arm/include/asm/arm32/cmpxchg.h
> index 37b2d64eb6..8455eb7cc3 100644
> --- a/xen/arch/arm/include/asm/arm32/cmpxchg.h
> +++ b/xen/arch/arm/include/asm/arm32/cmpxchg.h
> @@ -1,6 +1,7 @@
> =C2=A0#ifndef __ASM_ARM32_CMPXCHG_H
> =C2=A0#define __ASM_ARM32_CMPXCHG_H
> =C2=A0
> +#include <xen/bug.h>
> =C2=A0#include <xen/prefetch.h>
> =C2=A0
> =C2=A0extern void __bad_xchg(volatile void *ptr, int size);
> diff --git a/xen/arch/arm/include/asm/arm64/cmpxchg.h
> b/xen/arch/arm/include/asm/arm64/cmpxchg.h
> index 031fa6d92a..f160e8e7bc 100644
> --- a/xen/arch/arm/include/asm/arm64/cmpxchg.h
> +++ b/xen/arch/arm/include/asm/arm64/cmpxchg.h
> @@ -1,6 +1,8 @@
> =C2=A0#ifndef __ASM_ARM64_CMPXCHG_H
> =C2=A0#define __ASM_ARM64_CMPXCHG_H
> =C2=A0
> +#include <xen/bug.h>
> +
> =C2=A0extern void __bad_xchg(volatile void *ptr, int size);
> =C2=A0
> =C2=A0static inline unsigned long __xchg(unsigned long x, volatile void
> *ptr, int size)
> diff --git a/xen/arch/arm/include/asm/regs.h
> b/xen/arch/arm/include/asm/regs.h
> index f998aedff5..0d9f239a77 100644
> --- a/xen/arch/arm/include/asm/regs.h
> +++ b/xen/arch/arm/include/asm/regs.h
> @@ -5,7 +5,7 @@
> =C2=A0
> =C2=A0#ifndef __ASSEMBLY__
> =C2=A0
> -#include <xen/lib.h>
> +#include <xen/bug.h>
> =C2=A0#include <xen/types.h>
> =C2=A0#include <public/xen.h>
> =C2=A0#include <asm/current.h>
> diff --git a/xen/arch/arm/include/asm/vgic.h
> b/xen/arch/arm/include/asm/vgic.h
> index 922779ce14..79b73a0dbb 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -22,6 +22,7 @@
> =C2=A0#include <asm/new_vgic.h>
> =C2=A0#else
> =C2=A0
> +#include <xen/bug.h>
> =C2=A0#include <xen/radix-tree.h>
> =C2=A0#include <xen/rbtree.h>
> =C2=A0
> diff --git a/xen/arch/ppc/include/asm/time.h
> b/xen/arch/ppc/include/asm/time.h
> index aa9dda82a3..f23eec173e 100644
> --- a/xen/arch/ppc/include/asm/time.h
> +++ b/xen/arch/ppc/include/asm/time.h
> @@ -2,7 +2,7 @@
> =C2=A0#ifndef __ASM_PPC_TIME_H__
> =C2=A0#define __ASM_PPC_TIME_H__
> =C2=A0
> -#include <xen/lib.h>
> +#include <xen/bug.h>
> =C2=A0#include <asm/processor.h>
> =C2=A0#include <asm/regs.h>
> =C2=A0
> diff --git a/xen/arch/x86/bitops.c b/xen/arch/x86/bitops.c
> index f6ee71512c..720ab32362 100644
> --- a/xen/arch/x86/bitops.c
> +++ b/xen/arch/x86/bitops.c
> @@ -1,6 +1,6 @@
> =C2=A0
> =C2=A0#include <xen/bitops.h>
> -#include <xen/lib.h>
> +#include <xen/bug.h>
> =C2=A0
> =C2=A0unsigned int __find_first_bit(
> =C2=A0=C2=A0=C2=A0=C2=A0 const unsigned long *addr, unsigned int size)
> diff --git a/xen/arch/x86/include/asm/cpufeature.h
> b/xen/arch/x86/include/asm/cpufeature.h
> index 896483a900..be16492c68 100644
> --- a/xen/arch/x86/include/asm/cpufeature.h
> +++ b/xen/arch/x86/include/asm/cpufeature.h
> @@ -6,6 +6,7 @@
> =C2=A0#ifndef __ASM_I386_CPUFEATURE_H
> =C2=A0#define __ASM_I386_CPUFEATURE_H
> =C2=A0
> +#include <xen/cache.h>
> =C2=A0#include <xen/const.h>
> =C2=A0#include <asm/cpuid.h>
> =C2=A0
> diff --git a/xen/arch/x86/include/asm/system.h
> b/xen/arch/x86/include/asm/system.h
> index 6e40a03cb3..debf6bfa17 100644
> --- a/xen/arch/x86/include/asm/system.h
> +++ b/xen/arch/x86/include/asm/system.h
> @@ -1,8 +1,8 @@
> =C2=A0#ifndef __ASM_SYSTEM_H
> =C2=A0#define __ASM_SYSTEM_H
> =C2=A0
> -#include <xen/lib.h>
> =C2=A0#include <xen/bitops.h>
> +#include <xen/bug.h>
> =C2=A0#include <asm/cpufeature.h>
> =C2=A0#include <asm/x86-defns.h>
> =C2=A0
> diff --git a/xen/arch/x86/include/asm/x86_64/page.h
> b/xen/arch/x86/include/asm/x86_64/page.h
> index f49e10475f..19ca64d792 100644
> --- a/xen/arch/x86/include/asm/x86_64/page.h
> +++ b/xen/arch/x86/include/asm/x86_64/page.h
> @@ -19,6 +19,8 @@ static inline unsigned long
> canonicalise_addr(unsigned long addr)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return addr & ~CANONICAL=
_MASK;
> =C2=A0}
> =C2=A0
> +#include <xen/bug.h>
> +
> =C2=A0#include <xen/types.h>
> =C2=A0
> =C2=A0#include <xen/pdx.h>
> diff --git a/xen/arch/x86/x86_emulate/private.h
> b/xen/arch/x86/x86_emulate/private.h
> index 33f2870508..0fa26ba00a 100644
> --- a/xen/arch/x86/x86_emulate/private.h
> +++ b/xen/arch/x86/x86_emulate/private.h
> @@ -8,6 +8,7 @@
> =C2=A0
> =C2=A0#ifdef __XEN__
> =C2=A0
> +# include <xen/bug.h>
> =C2=A0# include <xen/kernel.h>
> =C2=A0# include <asm/endbr.h>
> =C2=A0# include <asm/msr-index.h>
> diff --git a/xen/common/efi/common-stub.c b/xen/common/efi/common-
> stub.c
> index 5a91fe28cc..77f138a6c5 100644
> --- a/xen/common/efi/common-stub.c
> +++ b/xen/common/efi/common-stub.c
> @@ -1,6 +1,6 @@
> +#include <xen/bug.h>
> =C2=A0#include <xen/efi.h>
> =C2=A0#include <xen/errno.h>
> -#include <xen/lib.h>
> =C2=A0
> =C2=A0bool efi_enabled(unsigned int feature)
> =C2=A0{
> diff --git a/xen/common/version.c b/xen/common/version.c
> index e807ef4de0..80869430fc 100644
> --- a/xen/common/version.c
> +++ b/xen/common/version.c
> @@ -1,3 +1,4 @@
> +#include <xen/bug.h>
> =C2=A0#include <xen/compile.h>
> =C2=A0#include <xen/init.h>
> =C2=A0#include <xen/errno.h>
> diff --git a/xen/include/public/hvm/save.h
> b/xen/include/public/hvm/save.h
> index ff0048e5f8..5561495b27 100644
> --- a/xen/include/public/hvm/save.h
> +++ b/xen/include/public/hvm/save.h
> @@ -51,7 +51,7 @@ struct hvm_save_descriptor {
> =C2=A0=C2=A0=C2=A0=C2=A0 struct __HVM_SAVE_TYPE_##_x { _type t; char c[_c=
ode]; char
> cpt[2];};=C2=A0 \
> =C2=A0=C2=A0=C2=A0=C2=A0 struct __HVM_SAVE_TYPE_COMPAT_##_x { _ctype t; }
> =C2=A0
> -# include <xen/lib.h> /* BUG() */
> +# include <xen/bug.h> /* BUG() */
> =C2=A0# define DECLARE_HVM_SAVE_TYPE(_x, _code,
> _type)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 \
> =C2=A0=C2=A0=C2=A0=C2=A0 static inline int __HVM_SAVE_FIX_COMPAT_##_x(voi=
d *h, uint32_t
> size) \
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 { BUG(); return -1;
> }=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> diff --git a/xen/include/xen/bug.h b/xen/include/xen/bug.h
> index 4bd0be5088..7c150b7032 100644
> --- a/xen/include/xen/bug.h
> +++ b/xen/include/xen/bug.h
> @@ -16,6 +16,7 @@
> =C2=A0
> =C2=A0#ifndef __ASSEMBLY__
> =C2=A0
> +#include <xen/compiler.h>
> =C2=A0#include <xen/macros.h>
> =C2=A0#include <xen/types.h>
> =C2=A0
> @@ -127,6 +128,24 @@ static void always_inline
> run_in_exception_handler(
> =C2=A0} while ( false )
> =C2=A0#endif
> =C2=A0
> +#define BUG_ON(p)=C2=A0 do { if (unlikely(p)) BUG();=C2=A0 } while (0)
> +#define WARN_ON(p)=C2=A0 ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> +=C2=A0=C2=A0=C2=A0 bool ret_warn_on_ =3D (p);=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 \
> +=C2=A0=C2=A0=C2=A0 if ( unlikely(ret_warn_on_) )=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 \
> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 WARN();=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> +=C2=A0=C2=A0=C2=A0 unlikely(ret_warn_on_);=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> +})
> +
> +#ifndef NDEBUG
> +#define ASSERT(p) \
> +=C2=A0=C2=A0=C2=A0 do { if ( unlikely(!(p)) ) assert_failed(#p); } while=
 (0)
> +#define ASSERT_UNREACHABLE() assert_failed("unreachable")
> +#else
> +#define ASSERT(p) do { if ( 0 && (p) ) {} } while (0)
> +#define ASSERT_UNREACHABLE() do { } while (0)
> +#endif
> +
> =C2=A0#ifdef CONFIG_GENERIC_BUG_FRAME
> =C2=A0
> =C2=A0/*
> diff --git a/xen/include/xen/cpumask.h b/xen/include/xen/cpumask.h
> index 145e140481..46e75d54ba 100644
> --- a/xen/include/xen/cpumask.h
> +++ b/xen/include/xen/cpumask.h
> @@ -55,6 +55,7 @@
> =C2=A0 * for_each_present_cpu(cpu)		for-loop cpu over
> cpu_present_map
> =C2=A0 */
> =C2=A0
> +#include <xen/bug.h>
> =C2=A0#include <xen/bitmap.h>
> =C2=A0#include <xen/kernel.h>
> =C2=A0#include <xen/random.h>
> diff --git a/xen/include/xen/device_tree.h
> b/xen/include/xen/device_tree.h
> index 94a836cb4e..6fe2fa8b21 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -14,6 +14,7 @@
> =C2=A0#include <asm/device.h>
> =C2=A0#include <public/xen.h>
> =C2=A0#include <public/device_tree_defs.h>
> +#include <xen/bug.h>
> =C2=A0#include <xen/kernel.h>
> =C2=A0#include <xen/string.h>
> =C2=A0#include <xen/types.h>
> diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
> index 1793be5b6b..4c755cc275 100644
> --- a/xen/include/xen/lib.h
> +++ b/xen/include/xen/lib.h
> @@ -5,31 +5,12 @@
> =C2=A0
> =C2=A0#ifndef __ASSEMBLY__
> =C2=A0
> -#include <xen/bug.h>
> =C2=A0#include <xen/inttypes.h>
> =C2=A0#include <xen/stdarg.h>
> =C2=A0#include <xen/types.h>
> =C2=A0#include <xen/xmalloc.h>
> =C2=A0#include <xen/string.h>
> =C2=A0
> -#define BUG_ON(p)=C2=A0 do { if (unlikely(p)) BUG();=C2=A0 } while (0)
> -#define WARN_ON(p)=C2=A0 ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> -=C2=A0=C2=A0=C2=A0 bool ret_warn_on_ =3D (p);=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 \
> -=C2=A0=C2=A0=C2=A0 if ( unlikely(ret_warn_on_) )=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 \
> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 WARN();=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> -=C2=A0=C2=A0=C2=A0 unlikely(ret_warn_on_);=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> -})
> -
> -#ifndef NDEBUG
> -#define ASSERT(p) \
> -=C2=A0=C2=A0=C2=A0 do { if ( unlikely(!(p)) ) assert_failed(#p); } while=
 (0)
> -#define ASSERT_UNREACHABLE() assert_failed("unreachable")
> -#else
> -#define ASSERT(p) do { if ( 0 && (p) ) {} } while (0)
> -#define ASSERT_UNREACHABLE() do { } while (0)
> -#endif
> -
> =C2=A0#define __ACCESS_ONCE(x) ({=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 \
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
(void)(typeof(x))0; /* Scalar typecheck. */ \
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
(volatile typeof(x) *)&(x); })
> diff --git a/xen/include/xen/list.h b/xen/include/xen/list.h
> index dc5a8c461b..b5eab3a1eb 100644
> --- a/xen/include/xen/list.h
> +++ b/xen/include/xen/list.h
> @@ -7,7 +7,7 @@
> =C2=A0#ifndef __XEN_LIST_H__
> =C2=A0#define __XEN_LIST_H__
> =C2=A0
> -#include <xen/lib.h>
> +#include <xen/bug.h>
> =C2=A0#include <asm/system.h>
> =C2=A0
> =C2=A0/*
> diff --git a/xen/include/xen/livepatch.h
> b/xen/include/xen/livepatch.h
> index 45df4bba4f..ad0eae28bd 100644
> --- a/xen/include/xen/livepatch.h
> +++ b/xen/include/xen/livepatch.h
> @@ -18,6 +18,8 @@ struct xen_sysctl_livepatch_op;
> =C2=A0
> =C2=A0#ifdef CONFIG_LIVEPATCH
> =C2=A0
> +#include <xen/lib.h>
> +
> =C2=A0/*
> =C2=A0 * We use alternative and exception table code - which by default
> are __init
> =C2=A0 * only, however we need them during runtime. These macros allows u=
s
> to build
> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> index 3d9b2d05a5..bb29b352ec 100644
> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -61,6 +61,7 @@
> =C2=A0#ifndef __XEN_MM_H__
> =C2=A0#define __XEN_MM_H__
> =C2=A0
> +#include <xen/bug.h>
> =C2=A0#include <xen/compiler.h>
> =C2=A0#include <xen/mm-frame.h>
> =C2=A0#include <xen/types.h>
> diff --git a/xen/include/xen/param.h b/xen/include/xen/param.h
> index 9170455cde..d8471ffe18 100644
> --- a/xen/include/xen/param.h
> +++ b/xen/include/xen/param.h
> @@ -1,6 +1,7 @@
> =C2=A0#ifndef _XEN_PARAM_H
> =C2=A0#define _XEN_PARAM_H
> =C2=A0
> +#include <xen/bug.h>
> =C2=A0#include <xen/hypfs.h>
> =C2=A0#include <xen/init.h>
> =C2=A0#include <xen/lib.h>
> diff --git a/xen/lib/list-sort.c b/xen/lib/list-sort.c
> index de1af2ef8b..251764804c 100644
> --- a/xen/lib/list-sort.c
> +++ b/xen/lib/list-sort.c
> @@ -15,6 +15,7 @@
> =C2=A0 * this program; If not, see <http://www.gnu.org/licenses/>.
> =C2=A0 */
> =C2=A0
> +#include <xen/lib.h>
> =C2=A0#include <xen/list.h>
> =C2=A0#include <xen/list_sort.h>
> =C2=A0
> diff --git a/xen/xsm/flask/ss/ebitmap.h b/xen/xsm/flask/ss/ebitmap.h
> index bb43de891d..7b2da8179a 100644
> --- a/xen/xsm/flask/ss/ebitmap.h
> +++ b/xen/xsm/flask/ss/ebitmap.h
> @@ -15,6 +15,7 @@
> =C2=A0#define _SS_EBITMAP_H_
> =C2=A0
> =C2=A0#include <xen/bitmap.h>
> +#include <xen/bug.h>
> =C2=A0
> =C2=A0#define EBITMAP_UNIT_NUMS	((32 - sizeof(void *) -
> sizeof(u32))	\
> =C2=A0					/ sizeof(unsigned long))



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 15:20:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 15:20:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683585.1063173 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRuh-0002Er-D4; Tue, 20 Feb 2024 15:20:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683585.1063173; Tue, 20 Feb 2024 15:20:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRuh-0002Ek-8i; Tue, 20 Feb 2024 15:20:07 +0000
Received: by outflank-mailman (input) for mailman id 683585;
 Tue, 20 Feb 2024 15:20:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nvC9=J5=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rcRuf-0001xY-A1
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 15:20:05 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 85afd66b-d003-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 16:20:04 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-512b4388dafso2734076e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 07:20:04 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 t5-20020ac243a5000000b00512c9bf8f89sm187037lfl.99.2024.02.20.07.20.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 07:20:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85afd66b-d003-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708442403; x=1709047203; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=99gK8LGzCfvguh8HVJdGJvMrvrsSm5+rJoS5zjH7FTA=;
        b=gmtVr3pCSBpDsL0VEVccratj+s2lryPxR+tpwASIF9YILISVJbfzXUQxHRk7/9ytkT
         Tml1W9cKuNp+j/5zdmwSbcNuLddW+zHToaGDvCk3T/uQ/vC+3zF94r7E6V/DgRMPEeKw
         oP93QvPhwoyalWy46VyNLAh8rFciU6fMdFRyI0ddsDBbjOfmycz7D0OqCyTKndBcgFBb
         kL2YrusOaKTn8civeQHt75RbbdINWigIUiKDbjhKwBXumX2R5zJBQqG4LgGAAGvnOUmd
         sKVftjLm0sdFUmltuCRNsTsQDUKoP3hYDCUbuuLzxQBjzJFC4l/ACFch6fQL3caTZiN5
         8IsA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708442403; x=1709047203;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=99gK8LGzCfvguh8HVJdGJvMrvrsSm5+rJoS5zjH7FTA=;
        b=B0L4GQSdWiLtztsrJDhlV1CzVH3TSFKHndosXoy4TkMc//TmBNtKBMe2v/Kt8pmVq5
         RE6rBJ5Mjre2Q33C7OOVEqwrPz5zeiBXB3Sfm0LcJSPuMzSc7mYVqjTgz8U+Zxeqf1J0
         j3iGUfDgVF9PwGa5v11QXGsvLqUV1gYeM/kryztD50FxWRNBM1Cd2iRaaZwuTw22rpD1
         6SRhYqusmD4OxWcM62xZQ9oeG7BaGsEUf9NbXfR1ZntpNbobSmCfxeTBGmyKUPHQu7aZ
         Zggxd8OgU0n0Hu8NIQmRn6vGhoRFGX6LPOOU2Kb1LXWE3kqTOe+j4r8Y8fw3XLUJ/ifY
         yW2Q==
X-Gm-Message-State: AOJu0YzfC85pctMOp1YSAYCiGZtvg0aVxRopxHedAeQ1LhBJRd4PZnqJ
	5n2P6ebSODwsiNgy9C/kHdeKs0fz1xYEFpymE/1OiP8lmo1mWkmsyJfhT/iG
X-Google-Smtp-Source: AGHT+IE+0fQT+kpQQuc+qaH9lNES1SYFsCx0tU2bsSfI2Eh1Qn5Dbx905RUQP0mV+W+ppfvjme8WEg==
X-Received: by 2002:a05:6512:691:b0:512:ccd2:5cb0 with SMTP id t17-20020a056512069100b00512ccd25cb0mr444998lfe.29.1708442403124;
        Tue, 20 Feb 2024 07:20:03 -0800 (PST)
Message-ID: <0d10bc590e3b6574eea35f72f324c509b4824d69.camel@gmail.com>
Subject: Re: [PATCH v2] xen/lib: introduce generic find next bit operations
From: Oleksii <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org, Shawn Anastasio
	 <sanastasio@raptorengineering.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	 <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu
	 <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal Orzel
	 <michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Date: Tue, 20 Feb 2024 16:20:02 +0100
In-Reply-To: <fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
References: 
	<fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

SGkgU2hhd24sCgpDb3VsZCB5b3UgcGxlYXNlIHRha2UgYSBsb29rIGF0IHRoZSBwYXRjaCBhbmQg
UFBDLXJlbGF0ZWQgY2hhbmdlcyAoCnhlbi9hcmNoL3BwYy9pbmNsdWRlL2FzbS9iaXRvcHMuaCAp
PwoKVGhhbmtzIGluIGFkdmFuY2UuCgp+IE9sZWtzaWkKCk9uIEZyaSwgMjAyNC0wMi0wOSBhdCAx
ODo1OCArMDEwMCwgT2xla3NpaSBLdXJvY2hrbyB3cm90ZToKPiBmaW5kLW5leHQtYml0LmMgaXMg
Y29tbW9uIGZvciBBcm02NCwgUFBDIGFuZCBSSVNDVjY0LAo+IHNvIGl0IGlzIG1vdmVkIHRvIHhl
bi9saWIuCj4gCj4gUFBDIGhhcyBiZWVuIHRyYW5zaXRpb25lZCB0byBnZW5lcmljIGZ1bmN0aW9u
cyBmcm9tIGZpbmQtbmV4dC1iaXQuYwo+IHNpbmNlIGl0IG5vdyBzaGFyZXMgdGhlIHNhbWUgaW1w
bGVtZW50YXRpb24gYXMgdGhlIFBQQy1zcGVjaWZpYyBjb2RlLgo+IAo+IFRoZSBNSVNSQSBleGNs
dWRlIGxpc3QgaGFzIGJlZW4gdXBkYXRlZCB0byB2ZXJpZnkKPiBsaWIvZmluZC1uZXh0LWJpdC5j
IGluc3RlYWQgb2YgQXJtJ3MgZmluZF9uZXh0X2JpdC5jLAo+IGFzIEFybSdzIGZpbmRfbmV4dF9i
aXQuYyBoYXMgYmVlbiByZWxvY2F0ZWQgdG8geGVuL2xpYi8uCj4gCj4gRGVzcGl0ZSBDT05GSUdf
R0VORVJJQ19GSU5EX0ZJUlNUX0JJVCBub3QgY3VycmVudGx5IGJlaW5nIHVzZWQgaW4KPiBYZW4g
YW5kIGJlaW5nIHJlbW92ZWQgZnJvbSB0aGUgTGludXgga2VybmVsIFsxXSwgaXQgY291bGQKPiB0
aGVvcmV0aWNhbGx5Cj4gcHJvdmUgdXNlZnVsIGZvciBYZW4uIFRoaXMgaXMgYmVjYXVzZSB0aGUg
TGludXgga2VybmVsIHRyYW5zaXRpb25lZAo+IEFybSB0byB0aGUgZ2VuZXJpYyB2ZXJzaW9uIG9m
IGZpbmRfZmlyc3RfYml0KCkgYW5kCj4gZmluZF9maXJzdF96ZXJvX2JpdCgpIGR1ZSB0byBpbXBy
b3ZlbWVudHMgWzFdIGluIGJvdGggcGVyZm9ybWFuY2UKPiBhbmQgLnRleHQgc2l6ZS4KPiBJdCB3
b3VsZCBiZSBwcnVkZW50IHRvIGludmVzdGlnYXRlIHdoZXRoZXIgWzFdIGlzIGFwcGxpY2FibGUg
dG8gWGVuCj4gYXMgd2VsbCBhbmQsIGlmIHNvLCBpbXBsZW1lbnQgdGhlIG5lY2Vzc2FyeSBjaGFu
Z2VzIGluIGEgc2VwYXJhdGUKPiBwYXRjaC4KPiAKPiBbMV0KPiBodHRwczovL2xvcmUua2VybmVs
Lm9yZy9saW51eC1hcmNoLzIwMjExMDA1MDU0MDU5LjQ3NTYzNC01LXl1cnkubm9yb3ZAZ21haWwu
Y29tLwo+IAo+IFN1Z2dlc3RlZC1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgo+
IFNpZ25lZC1vZmYtYnk6IE9sZWtzaWkgS3Vyb2Noa28gPG9sZWtzaWkua3Vyb2Noa29AZ21haWwu
Y29tPgo+IC0tLQo+IMKgZG9jcy9taXNyYS9leGNsdWRlLWxpc3QuanNvbsKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqAgfMKgwqAgOCArLQo+IMKgeGVuL2FyY2gvYXJtL2FybTY0L2xp
Yi9NYWtlZmlsZcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgfMKgwqAgMiArLQo+IMKgeGVu
L2FyY2gvYXJtL2luY2x1ZGUvYXNtL2FybTY0L2JpdG9wcy5owqDCoMKgwqDCoMKgIHzCoCA0NiAr
LS0tLS0tCj4gwqB4ZW4vYXJjaC9wcGMvaW5jbHVkZS9hc20vYml0b3BzLmjCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgfCAxMTUgLS0tLS0tLS0tLS0tLS0tLQo+IC0tCj4gwqB4ZW4vaW5jbHVkZS94
ZW4vYml0b3BzLmjCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgfMKg
IDQ4ICsrKysrKysrCj4gwqB4ZW4vbGliL01ha2VmaWxlwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB8wqDCoCAxICsKPiDCoC4uLi9maW5k
X25leHRfYml0LmMgPT4gbGliL2ZpbmQtbmV4dC1iaXQuY33CoMKgIHzCoMKgIDAKPiDCoDcgZmls
ZXMgY2hhbmdlZCwgNTUgaW5zZXJ0aW9ucygrKSwgMTY1IGRlbGV0aW9ucygtKQo+IMKgcmVuYW1l
IHhlbi97YXJjaC9hcm0vYXJtNjQvbGliL2ZpbmRfbmV4dF9iaXQuYyA9PiBsaWIvZmluZC1uZXh0
LQo+IGJpdC5jfSAoMTAwJSkKPiAKPiBkaWZmIC0tZ2l0IGEvZG9jcy9taXNyYS9leGNsdWRlLWxp
c3QuanNvbiBiL2RvY3MvbWlzcmEvZXhjbHVkZS0KPiBsaXN0Lmpzb24KPiBpbmRleCA3OTcxZDBl
NzBmLi5iMjgxNzRkMGViIDEwMDY0NAo+IC0tLSBhL2RvY3MvbWlzcmEvZXhjbHVkZS1saXN0Lmpz
b24KPiArKysgYi9kb2NzL21pc3JhL2V4Y2x1ZGUtbGlzdC5qc29uCj4gQEAgLTEzLDEwICsxMyw2
IEBACj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICJyZWxfcGF0aCI6ICJhcmNoL2FybS9hcm02
NC9pbnNuLmMiLAo+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAiY29tbWVudCI6ICJJbXBvcnRl
ZCBvbiBMaW51eCwgaWdub3JlIGZvciBub3ciCj4gwqDCoMKgwqDCoMKgwqDCoCB9LAo+IC3CoMKg
wqDCoMKgwqDCoCB7Cj4gLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgInJlbF9wYXRoIjogImFyY2gv
YXJtL2FybTY0L2xpYi9maW5kX25leHRfYml0LmMiLAo+IC3CoMKgwqDCoMKgwqDCoMKgwqDCoMKg
ICJjb21tZW50IjogIkltcG9ydGVkIGZyb20gTGludXgsIGlnbm9yZSBmb3Igbm93Igo+IC3CoMKg
wqDCoMKgwqDCoCB9LAo+IMKgwqDCoMKgwqDCoMKgwqAgewo+IMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoCAicmVsX3BhdGgiOiAiYXJjaC94ODYvYWNwaS9ib290LmMiLAo+IMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoCAiY29tbWVudCI6ICJJbXBvcnRlZCBmcm9tIExpbnV4LCBpZ25vcmUgZm9yIG5v
dyIKPiBAQCAtMjIxLDYgKzIxNywxMCBAQAo+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAicmVs
X3BhdGgiOiAiaW5jbHVkZS94ZW4vZGVjb21wcmVzcy5oIiwKPiDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgImNvbW1lbnQiOiAiSW1wb3J0ZWQgZnJvbSBMaW51eCwgaWdub3JlIGZvciBub3ciCj4g
wqDCoMKgwqDCoMKgwqDCoCB9LAo+ICvCoMKgwqDCoMKgwqDCoCB7Cj4gK8KgwqDCoMKgwqDCoMKg
wqDCoMKgwqAgInJlbF9wYXRoIjogImxpYi9maW5kLW5leHQtYml0LmMiLAo+ICvCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgICJjb21tZW50IjogIkltcG9ydGVkIGZyb20gTGludXgsIGlnbm9yZSBmb3Ig
bm93Igo+ICvCoMKgwqDCoMKgwqDCoCB9LAo+IMKgwqDCoMKgwqDCoMKgwqAgewo+IMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoCAicmVsX3BhdGgiOiAibGliL2xpc3Qtc29ydC5jIiwKPiDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgImNvbW1lbnQiOiAiSW1wb3J0ZWQgZnJvbSBMaW51eCwgaWdub3Jl
IGZvciBub3ciCj4gZGlmZiAtLWdpdCBhL3hlbi9hcmNoL2FybS9hcm02NC9saWIvTWFrZWZpbGUK
PiBiL3hlbi9hcmNoL2FybS9hcm02NC9saWIvTWFrZWZpbGUKPiBpbmRleCAxYjljN2E5NWU2Li42
NmNmYWM0MzVhIDEwMDY0NAo+IC0tLSBhL3hlbi9hcmNoL2FybS9hcm02NC9saWIvTWFrZWZpbGUK
PiArKysgYi94ZW4vYXJjaC9hcm0vYXJtNjQvbGliL01ha2VmaWxlCj4gQEAgLTEsNCArMSw0IEBA
Cj4gwqBvYmoteSArPSBtZW1jcHkubyBtZW1jbXAubyBtZW1tb3ZlLm8gbWVtc2V0Lm8gbWVtY2hy
Lm8KPiDCoG9iai15ICs9IGNsZWFyX3BhZ2Uubwo+IC1vYmoteSArPSBiaXRvcHMubyBmaW5kX25l
eHRfYml0Lm8KPiArb2JqLXkgKz0gYml0b3BzLm8KPiDCoG9iai15ICs9IHN0cmNoci5vIHN0cmNt
cC5vIHN0cmxlbi5vIHN0cm5jbXAubyBzdHJubGVuLm8gc3RycmNoci5vCj4gZGlmZiAtLWdpdCBh
L3hlbi9hcmNoL2FybS9pbmNsdWRlL2FzbS9hcm02NC9iaXRvcHMuaAo+IGIveGVuL2FyY2gvYXJt
L2luY2x1ZGUvYXNtL2FybTY0L2JpdG9wcy5oCj4gaW5kZXggZDg1YTQ5YmNhNC4uMGVmZGUyOTA2
OCAxMDA2NDQKPiAtLS0gYS94ZW4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvYml0b3BzLmgK
PiArKysgYi94ZW4vYXJjaC9hcm0vaW5jbHVkZS9hc20vYXJtNjQvYml0b3BzLmgKPiBAQCAtMzYs
NTcgKzM2LDEzIEBAIHN0YXRpYyBpbmxpbmUgaW50IGZsc2wodW5zaWduZWQgbG9uZyB4KQo+IMKg
Cj4gwqAvKiBCYXNlZCBvbiBsaW51eC9pbmNsdWRlL2FzbS1nZW5lcmljL2JpdG9wcy9maW5kLmgg
Ki8KPiDCoAo+IC0jaWZuZGVmIGZpbmRfbmV4dF9iaXQKPiAtLyoqCj4gLSAqIGZpbmRfbmV4dF9i
aXQgLSBmaW5kIHRoZSBuZXh0IHNldCBiaXQgaW4gYSBtZW1vcnkgcmVnaW9uCj4gLSAqIEBhZGRy
OiBUaGUgYWRkcmVzcyB0byBiYXNlIHRoZSBzZWFyY2ggb24KPiAtICogQG9mZnNldDogVGhlIGJp
dG51bWJlciB0byBzdGFydCBzZWFyY2hpbmcgYXQKPiAtICogQHNpemU6IFRoZSBiaXRtYXAgc2l6
ZSBpbiBiaXRzCj4gLSAqLwo+IC1leHRlcm4gdW5zaWduZWQgbG9uZyBmaW5kX25leHRfYml0KGNv
bnN0IHVuc2lnbmVkIGxvbmcgKmFkZHIsCj4gdW5zaWduZWQgbG9uZwo+IC0JCXNpemUsIHVuc2ln
bmVkIGxvbmcgb2Zmc2V0KTsKPiAtI2VuZGlmCj4gLQo+IC0jaWZuZGVmIGZpbmRfbmV4dF96ZXJv
X2JpdAo+IC0vKioKPiAtICogZmluZF9uZXh0X3plcm9fYml0IC0gZmluZCB0aGUgbmV4dCBjbGVh
cmVkIGJpdCBpbiBhIG1lbW9yeSByZWdpb24KPiAtICogQGFkZHI6IFRoZSBhZGRyZXNzIHRvIGJh
c2UgdGhlIHNlYXJjaCBvbgo+IC0gKiBAb2Zmc2V0OiBUaGUgYml0bnVtYmVyIHRvIHN0YXJ0IHNl
YXJjaGluZyBhdAo+IC0gKiBAc2l6ZTogVGhlIGJpdG1hcCBzaXplIGluIGJpdHMKPiAtICovCj4g
LWV4dGVybiB1bnNpZ25lZCBsb25nIGZpbmRfbmV4dF96ZXJvX2JpdChjb25zdCB1bnNpZ25lZCBs
b25nICphZGRyLAo+IHVuc2lnbmVkCj4gLQkJbG9uZyBzaXplLCB1bnNpZ25lZCBsb25nIG9mZnNl
dCk7Cj4gLSNlbmRpZgo+IC0KPiAtI2lmZGVmIENPTkZJR19HRU5FUklDX0ZJTkRfRklSU1RfQklU
Cj4gLQo+IC0vKioKPiAtICogZmluZF9maXJzdF9iaXQgLSBmaW5kIHRoZSBmaXJzdCBzZXQgYml0
IGluIGEgbWVtb3J5IHJlZ2lvbgo+IC0gKiBAYWRkcjogVGhlIGFkZHJlc3MgdG8gc3RhcnQgdGhl
IHNlYXJjaCBhdAo+IC0gKiBAc2l6ZTogVGhlIG1heGltdW0gc2l6ZSB0byBzZWFyY2gKPiAtICoK
PiAtICogUmV0dXJucyB0aGUgYml0IG51bWJlciBvZiB0aGUgZmlyc3Qgc2V0IGJpdC4KPiAtICov
Cj4gLWV4dGVybiB1bnNpZ25lZCBsb25nIGZpbmRfZmlyc3RfYml0KGNvbnN0IHVuc2lnbmVkIGxv
bmcgKmFkZHIsCj4gLQkJCQnCoMKgwqAgdW5zaWduZWQgbG9uZyBzaXplKTsKPiAtCj4gLS8qKgo+
IC0gKiBmaW5kX2ZpcnN0X3plcm9fYml0IC0gZmluZCB0aGUgZmlyc3QgY2xlYXJlZCBiaXQgaW4g
YSBtZW1vcnkKPiByZWdpb24KPiAtICogQGFkZHI6IFRoZSBhZGRyZXNzIHRvIHN0YXJ0IHRoZSBz
ZWFyY2ggYXQKPiAtICogQHNpemU6IFRoZSBtYXhpbXVtIHNpemUgdG8gc2VhcmNoCj4gLSAqCj4g
LSAqIFJldHVybnMgdGhlIGJpdCBudW1iZXIgb2YgdGhlIGZpcnN0IGNsZWFyZWQgYml0Lgo+IC0g
Ki8KPiAtZXh0ZXJuIHVuc2lnbmVkIGxvbmcgZmluZF9maXJzdF96ZXJvX2JpdChjb25zdCB1bnNp
Z25lZCBsb25nICphZGRyLAo+IC0JCQkJCSB1bnNpZ25lZCBsb25nIHNpemUpOwo+IC0jZWxzZSAv
KiBDT05GSUdfR0VORVJJQ19GSU5EX0ZJUlNUX0JJVCAqLwo+ICsjaWZuZGVmIENPTkZJR19HRU5F
UklDX0ZJTkRfRklSU1RfQklUCj4gwqAKPiDCoCNkZWZpbmUgZmluZF9maXJzdF9iaXQoYWRkciwg
c2l6ZSkgZmluZF9uZXh0X2JpdCgoYWRkciksIChzaXplKSwgMCkKPiDCoCNkZWZpbmUgZmluZF9m
aXJzdF96ZXJvX2JpdChhZGRyLCBzaXplKSBmaW5kX25leHRfemVyb19iaXQoKGFkZHIpLAo+IChz
aXplKSwgMCkKPiDCoAo+IMKgI2VuZGlmIC8qIENPTkZJR19HRU5FUklDX0ZJTkRfRklSU1RfQklU
ICovCj4gwqAKPiAtCj4gwqAjZW5kaWYgLyogX0FSTV9BUk02NF9CSVRPUFNfSCAqLwo+IMKgLyoK
PiDCoCAqIExvY2FsIHZhcmlhYmxlczoKPiBkaWZmIC0tZ2l0IGEveGVuL2FyY2gvcHBjL2luY2x1
ZGUvYXNtL2JpdG9wcy5oCj4gYi94ZW4vYXJjaC9wcGMvaW5jbHVkZS9hc20vYml0b3BzLmgKPiBp
bmRleCA1ZTdmMzZjMjFkLi41ODIwYjljZTdiIDEwMDY0NAo+IC0tLSBhL3hlbi9hcmNoL3BwYy9p
bmNsdWRlL2FzbS9iaXRvcHMuaAo+ICsrKyBiL3hlbi9hcmNoL3BwYy9pbmNsdWRlL2FzbS9iaXRv
cHMuaAo+IEBAIC0yMTcsMTE5ICsyMTcsNCBAQCBzdGF0aWMgYWx3YXlzX2lubGluZSB1bnNpZ25l
ZCBsb25nCj4gX19mZnModW5zaWduZWQgbG9uZyB3b3JkKQo+IMKgICovCj4gwqAjZGVmaW5lIGZp
bmRfZmlyc3Rfc2V0X2JpdCh4KSAoZmZzbCh4KSAtIDEpCj4gwqAKPiAtLyoKPiAtICogRmluZCB0
aGUgZmlyc3Qgc2V0IGJpdCBpbiBhIG1lbW9yeSByZWdpb24uCj4gLSAqLwo+IC1zdGF0aWMgaW5s
aW5lIHVuc2lnbmVkIGxvbmcgZmluZF9maXJzdF9iaXQoY29uc3QgdW5zaWduZWQgbG9uZwo+ICph
ZGRyLAo+IC3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdW5zaWduZWQgbG9uZyBzaXplKQo+
IC17Cj4gLcKgwqDCoCBjb25zdCB1bnNpZ25lZCBsb25nICpwID0gYWRkcjsKPiAtwqDCoMKgIHVu
c2lnbmVkIGxvbmcgcmVzdWx0ID0gMDsKPiAtwqDCoMKgIHVuc2lnbmVkIGxvbmcgdG1wOwo+IC0K
PiAtwqDCoMKgIHdoaWxlICggc2l6ZSAmIH4oQklUU19QRVJfTE9ORyAtIDEpICkKPiAtwqDCoMKg
IHsKPiAtwqDCoMKgwqDCoMKgwqAgaWYgKCAodG1wID0gKihwKyspKSApCj4gLcKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgZ290byBmb3VuZDsKPiAtwqDCoMKgwqDCoMKgwqAgcmVzdWx0ICs9IEJJVFNf
UEVSX0xPTkc7Cj4gLcKgwqDCoMKgwqDCoMKgIHNpemUgLT0gQklUU19QRVJfTE9ORzsKPiAtwqDC
oMKgIH0KPiAtwqDCoMKgIGlmICggIXNpemUgKQo+IC3CoMKgwqDCoMKgwqDCoCByZXR1cm4gcmVz
dWx0Owo+IC0KPiAtwqDCoMKgIHRtcCA9ICgqcCkgJiAofjBVTCA+PiAoQklUU19QRVJfTE9ORyAt
IHNpemUpKTsKPiAtwqDCoMKgIGlmICggdG1wID09IDBVTCApwqDCoMKgwqDCoMKgwqDCoCAvKiBB
cmUgYW55IGJpdHMgc2V0PyAqLwo+IC3CoMKgwqDCoMKgwqDCoCByZXR1cm4gcmVzdWx0ICsgc2l6
ZTsgLyogTm9wZS4gKi8KPiAtIGZvdW5kOgo+IC3CoMKgwqAgcmV0dXJuIHJlc3VsdCArIF9fZmZz
KHRtcCk7Cj4gLX0KPiAtCj4gLXN0YXRpYyBpbmxpbmUgdW5zaWduZWQgbG9uZyBmaW5kX25leHRf
Yml0KGNvbnN0IHVuc2lnbmVkIGxvbmcgKmFkZHIsCj4gLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqAgdW5zaWduZWQgbG9uZyBzaXplLAo+IC3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVuc2ln
bmVkIGxvbmcgb2Zmc2V0KQo+IC17Cj4gLcKgwqDCoCBjb25zdCB1bnNpZ25lZCBsb25nICpwID0g
YWRkciArIEJJVE9QX1dPUkQob2Zmc2V0KTsKPiAtwqDCoMKgIHVuc2lnbmVkIGxvbmcgcmVzdWx0
ID0gb2Zmc2V0ICYgfihCSVRTX1BFUl9MT05HIC0gMSk7Cj4gLcKgwqDCoCB1bnNpZ25lZCBsb25n
IHRtcDsKPiAtCj4gLcKgwqDCoCBpZiAoIG9mZnNldCA+PSBzaXplICkKPiAtwqDCoMKgwqDCoMKg
wqAgcmV0dXJuIHNpemU7Cj4gLcKgwqDCoCBzaXplIC09IHJlc3VsdDsKPiAtwqDCoMKgIG9mZnNl
dCAlPSBCSVRTX1BFUl9MT05HOwo+IC3CoMKgwqAgaWYgKCBvZmZzZXQgKQo+IC3CoMKgwqAgewo+
IC3CoMKgwqDCoMKgwqDCoCB0bXAgPSAqKHArKyk7Cj4gLcKgwqDCoMKgwqDCoMKgIHRtcCAmPSAo
fjBVTCA8PCBvZmZzZXQpOwo+IC3CoMKgwqDCoMKgwqDCoCBpZiAoIHNpemUgPCBCSVRTX1BFUl9M
T05HICkKPiAtwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBnb3RvIGZvdW5kX2ZpcnN0Owo+IC3CoMKg
wqDCoMKgwqDCoCBpZiAoIHRtcCApCj4gLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgZ290byBmb3Vu
ZF9taWRkbGU7Cj4gLcKgwqDCoMKgwqDCoMKgIHNpemUgLT0gQklUU19QRVJfTE9ORzsKPiAtwqDC
oMKgwqDCoMKgwqAgcmVzdWx0ICs9IEJJVFNfUEVSX0xPTkc7Cj4gLcKgwqDCoCB9Cj4gLcKgwqDC
oCB3aGlsZSAoIHNpemUgJiB+KEJJVFNfUEVSX0xPTkcgLSAxKSApCj4gLcKgwqDCoCB7Cj4gLcKg
wqDCoMKgwqDCoMKgIGlmICggKHRtcCA9ICoocCsrKSkgKQo+IC3CoMKgwqDCoMKgwqDCoMKgwqDC
oMKgIGdvdG8gZm91bmRfbWlkZGxlOwo+IC3CoMKgwqDCoMKgwqDCoCByZXN1bHQgKz0gQklUU19Q
RVJfTE9ORzsKPiAtwqDCoMKgwqDCoMKgwqAgc2l6ZSAtPSBCSVRTX1BFUl9MT05HOwo+IC3CoMKg
wqAgfQo+IC3CoMKgwqAgaWYgKCAhc2l6ZSApCj4gLcKgwqDCoMKgwqDCoMKgIHJldHVybiByZXN1
bHQ7Cj4gLcKgwqDCoCB0bXAgPSAqcDsKPiAtCj4gLSBmb3VuZF9maXJzdDoKPiAtwqDCoMKgIHRt
cCAmPSAofjBVTCA+PiAoQklUU19QRVJfTE9ORyAtIHNpemUpKTsKPiAtwqDCoMKgIGlmICggdG1w
ID09IDBVTCApwqDCoMKgwqDCoMKgwqDCoCAvKiBBcmUgYW55IGJpdHMgc2V0PyAqLwo+IC3CoMKg
wqDCoMKgwqDCoCByZXR1cm4gcmVzdWx0ICsgc2l6ZTsgLyogTm9wZS4gKi8KPiAtIGZvdW5kX21p
ZGRsZToKPiAtwqDCoMKgIHJldHVybiByZXN1bHQgKyBfX2Zmcyh0bXApOwo+IC19Cj4gLQo+IC0v
Kgo+IC0gKiBUaGlzIGltcGxlbWVudGF0aW9uIG9mIGZpbmRfe2ZpcnN0LG5leHR9X3plcm9fYml0
IHdhcyBzdG9sZW4gZnJvbQo+IC0gKiBMaW51cycgYXNtLWFscGhhL2JpdG9wcy5oLgo+IC0gKi8K
PiAtc3RhdGljIGlubGluZSB1bnNpZ25lZCBsb25nIGZpbmRfbmV4dF96ZXJvX2JpdChjb25zdCB1
bnNpZ25lZCBsb25nCj4gKmFkZHIsCj4gLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgIHVuc2lnbmVkIGxvbmcgc2l6ZSwKPiAtwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgdW5zaWduZWQgbG9uZyBvZmZzZXQpCj4gLXsKPiAtwqDCoMKgIGNvbnN0IHVuc2lnbmVk
IGxvbmcgKnAgPSBhZGRyICsgQklUT1BfV09SRChvZmZzZXQpOwo+IC3CoMKgwqAgdW5zaWduZWQg
bG9uZyByZXN1bHQgPSBvZmZzZXQgJiB+KEJJVFNfUEVSX0xPTkcgLSAxKTsKPiAtwqDCoMKgIHVu
c2lnbmVkIGxvbmcgdG1wOwo+IC0KPiAtwqDCoMKgIGlmICggb2Zmc2V0ID49IHNpemUgKQo+IC3C
oMKgwqDCoMKgwqDCoCByZXR1cm4gc2l6ZTsKPiAtwqDCoMKgIHNpemUgLT0gcmVzdWx0Owo+IC3C
oMKgwqAgb2Zmc2V0ICU9IEJJVFNfUEVSX0xPTkc7Cj4gLcKgwqDCoCBpZiAoIG9mZnNldCApCj4g
LcKgwqDCoCB7Cj4gLcKgwqDCoMKgwqDCoMKgIHRtcCA9ICoocCsrKTsKPiAtwqDCoMKgwqDCoMKg
wqAgdG1wIHw9IH4wVUwgPj4gKEJJVFNfUEVSX0xPTkcgLSBvZmZzZXQpOwo+IC3CoMKgwqDCoMKg
wqDCoCBpZiAoIHNpemUgPCBCSVRTX1BFUl9MT05HICkKPiAtwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oCBnb3RvIGZvdW5kX2ZpcnN0Owo+IC3CoMKgwqDCoMKgwqDCoCBpZiAoIH50bXAgKQo+IC3CoMKg
wqDCoMKgwqDCoMKgwqDCoMKgIGdvdG8gZm91bmRfbWlkZGxlOwo+IC3CoMKgwqDCoMKgwqDCoCBz
aXplIC09IEJJVFNfUEVSX0xPTkc7Cj4gLcKgwqDCoMKgwqDCoMKgIHJlc3VsdCArPSBCSVRTX1BF
Ul9MT05HOwo+IC3CoMKgwqAgfQo+IC3CoMKgwqAgd2hpbGUgKCBzaXplICYgfihCSVRTX1BFUl9M
T05HIC0gMSkgKQo+IC3CoMKgwqAgewo+IC3CoMKgwqDCoMKgwqDCoCBpZiAoIH4odG1wID0gKihw
KyspKSApCj4gLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgZ290byBmb3VuZF9taWRkbGU7Cj4gLcKg
wqDCoMKgwqDCoMKgIHJlc3VsdCArPSBCSVRTX1BFUl9MT05HOwo+IC3CoMKgwqDCoMKgwqDCoCBz
aXplIC09IEJJVFNfUEVSX0xPTkc7Cj4gLcKgwqDCoCB9Cj4gLcKgwqDCoCBpZiAoICFzaXplICkK
PiAtwqDCoMKgwqDCoMKgwqAgcmV0dXJuIHJlc3VsdDsKPiAtwqDCoMKgIHRtcCA9ICpwOwo+IC0K
PiAtIGZvdW5kX2ZpcnN0Ogo+IC3CoMKgwqAgdG1wIHw9IH4wVUwgPDwgc2l6ZTsKPiAtwqDCoMKg
IGlmICggdG1wID09IH4wVUwgKcKgwqDCoMKgwqDCoMKgIC8qIEFyZSBhbnkgYml0cyB6ZXJvPyAq
Lwo+IC3CoMKgwqDCoMKgwqDCoCByZXR1cm4gcmVzdWx0ICsgc2l6ZTsgLyogTm9wZS4gKi8KPiAt
IGZvdW5kX21pZGRsZToKPiAtwqDCoMKgIHJldHVybiByZXN1bHQgKyBmZnoodG1wKTsKPiAtfQo+
IC0KPiDCoCNlbmRpZiAvKiBfQVNNX1BQQ19CSVRPUFNfSCAqLwo+IGRpZmYgLS1naXQgYS94ZW4v
aW5jbHVkZS94ZW4vYml0b3BzLmggYi94ZW4vaW5jbHVkZS94ZW4vYml0b3BzLmgKPiBpbmRleCAy
Y2I3ODkyYmNjLi5lM2M1YTRjY2YzIDEwMDY0NAo+IC0tLSBhL3hlbi9pbmNsdWRlL3hlbi9iaXRv
cHMuaAo+ICsrKyBiL3hlbi9pbmNsdWRlL3hlbi9iaXRvcHMuaAo+IEBAIC0xMDUsNiArMTA1LDU0
IEBAIHN0YXRpYyBpbmxpbmUgaW50IGdlbmVyaWNfZmxzbCh1bnNpZ25lZCBsb25nIHgpCj4gwqAg
Ki8KPiDCoCNpbmNsdWRlIDxhc20vYml0b3BzLmg+Cj4gwqAKPiArI2lmbmRlZiBmaW5kX25leHRf
Yml0Cj4gKy8qKgo+ICsgKiBmaW5kX25leHRfYml0IC0gZmluZCB0aGUgbmV4dCBzZXQgYml0IGlu
IGEgbWVtb3J5IHJlZ2lvbgo+ICsgKiBAYWRkcjogVGhlIGFkZHJlc3MgdG8gYmFzZSB0aGUgc2Vh
cmNoIG9uCj4gKyAqIEBvZmZzZXQ6IFRoZSBiaXRudW1iZXIgdG8gc3RhcnQgc2VhcmNoaW5nIGF0
Cj4gKyAqIEBzaXplOiBUaGUgYml0bWFwIHNpemUgaW4gYml0cwo+ICsgKi8KPiArZXh0ZXJuIHVu
c2lnbmVkIGxvbmcgZmluZF9uZXh0X2JpdChjb25zdCB1bnNpZ25lZCBsb25nICphZGRyLAo+ICvC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoCB1bnNpZ25lZCBsb25nIHNpemUsCj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVuc2lnbmVkIGxv
bmcgb2Zmc2V0KTsKPiArI2VuZGlmCj4gKwo+ICsjaWZuZGVmIGZpbmRfbmV4dF96ZXJvX2JpdAo+
ICsvKioKPiArICogZmluZF9uZXh0X3plcm9fYml0IC0gZmluZCB0aGUgbmV4dCBjbGVhcmVkIGJp
dCBpbiBhIG1lbW9yeSByZWdpb24KPiArICogQGFkZHI6IFRoZSBhZGRyZXNzIHRvIGJhc2UgdGhl
IHNlYXJjaCBvbgo+ICsgKiBAb2Zmc2V0OiBUaGUgYml0bnVtYmVyIHRvIHN0YXJ0IHNlYXJjaGlu
ZyBhdAo+ICsgKiBAc2l6ZTogVGhlIGJpdG1hcCBzaXplIGluIGJpdHMKPiArICovCj4gK2V4dGVy
biB1bnNpZ25lZCBsb25nIGZpbmRfbmV4dF96ZXJvX2JpdChjb25zdCB1bnNpZ25lZCBsb25nICph
ZGRyLAo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdW5zaWduZWQgbG9uZyBzaXplLAo+ICvCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgdW5zaWduZWQgbG9uZyBvZmZzZXQpOwo+ICsjZW5kaWYKPiArCj4g
KyNpZm5kZWYgZmluZF9maXJzdF9iaXQKPiArLyoqCj4gKyAqIGZpbmRfZmlyc3RfYml0IC0gZmlu
ZCB0aGUgZmlyc3Qgc2V0IGJpdCBpbiBhIG1lbW9yeSByZWdpb24KPiArICogQGFkZHI6IFRoZSBh
ZGRyZXNzIHRvIHN0YXJ0IHRoZSBzZWFyY2ggYXQKPiArICogQHNpemU6IFRoZSBtYXhpbXVtIHNp
emUgdG8gc2VhcmNoCj4gKyAqCj4gKyAqIFJldHVybnMgdGhlIGJpdCBudW1iZXIgb2YgdGhlIGZp
cnN0IHNldCBiaXQuCj4gKyAqLwo+ICtleHRlcm4gdW5zaWduZWQgbG9uZyBmaW5kX2ZpcnN0X2Jp
dChjb25zdCB1bnNpZ25lZCBsb25nICphZGRyLAo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVuc2lnbmVkIGxv
bmcgc2l6ZSk7Cj4gKyNlbmRpZgo+ICsKPiArI2lmbmRlZiBmaW5kX2ZpcnN0X3plcm9fYml0Cj4g
Ky8qKgo+ICsgKiBmaW5kX2ZpcnN0X3plcm9fYml0IC0gZmluZCB0aGUgZmlyc3QgY2xlYXJlZCBi
aXQgaW4gYSBtZW1vcnkKPiByZWdpb24KPiArICogQGFkZHI6IFRoZSBhZGRyZXNzIHRvIHN0YXJ0
IHRoZSBzZWFyY2ggYXQKPiArICogQHNpemU6IFRoZSBtYXhpbXVtIHNpemUgdG8gc2VhcmNoCj4g
KyAqCj4gKyAqIFJldHVybnMgdGhlIGJpdCBudW1iZXIgb2YgdGhlIGZpcnN0IGNsZWFyZWQgYml0
Lgo+ICsgKi8KPiArZXh0ZXJuIHVuc2lnbmVkIGxvbmcgZmluZF9maXJzdF96ZXJvX2JpdChjb25z
dCB1bnNpZ25lZCBsb25nICphZGRyLAo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB1bnNpZ25l
ZCBsb25nIHNpemUpOwo+ICsjZW5kaWYKPiArCj4gwqAjaWYgQklUU19QRVJfTE9ORyA9PSA2NAo+
IMKgIyBkZWZpbmUgZmxzNjQgZmxzbAo+IMKgIyBkZWZpbmUgZmZzNjQgZmZzbAo+IGRpZmYgLS1n
aXQgYS94ZW4vbGliL01ha2VmaWxlIGIveGVuL2xpYi9NYWtlZmlsZQo+IGluZGV4IDJkOWViYjk0
NWYuLmU2Mzc5OGUxZDQgMTAwNjQ0Cj4gLS0tIGEveGVuL2xpYi9NYWtlZmlsZQo+ICsrKyBiL3hl
bi9saWIvTWFrZWZpbGUKPiBAQCAtMyw2ICszLDcgQEAgb2JqLSQoQ09ORklHX1g4NikgKz0geDg2
Lwo+IMKgbGliLXkgKz0gYnNlYXJjaC5vCj4gwqBsaWIteSArPSBjdG9ycy5vCj4gwqBsaWIteSAr
PSBjdHlwZS5vCj4gK2xpYi15ICs9IGZpbmQtbmV4dC1iaXQubwo+IMKgbGliLXkgKz0gbGlzdC1z
b3J0Lm8KPiDCoGxpYi15ICs9IG1lbWNoci5vCj4gwqBsaWIteSArPSBtZW1jaHJfaW52Lm8KPiBk
aWZmIC0tZ2l0IGEveGVuL2FyY2gvYXJtL2FybTY0L2xpYi9maW5kX25leHRfYml0LmMgYi94ZW4v
bGliL2ZpbmQtCj4gbmV4dC1iaXQuYwo+IHNpbWlsYXJpdHkgaW5kZXggMTAwJQo+IHJlbmFtZSBm
cm9tIHhlbi9hcmNoL2FybS9hcm02NC9saWIvZmluZF9uZXh0X2JpdC5jCj4gcmVuYW1lIHRvIHhl
bi9saWIvZmluZC1uZXh0LWJpdC5jCgo=



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 15:22:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 15:22:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683591.1063182 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRwu-00035H-Np; Tue, 20 Feb 2024 15:22:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683591.1063182; Tue, 20 Feb 2024 15:22:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcRwu-00035A-LJ; Tue, 20 Feb 2024 15:22:24 +0000
Received: by outflank-mailman (input) for mailman id 683591;
 Tue, 20 Feb 2024 15:22:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J7zC=J5=amd.com=ayan.kumar.halder@srs-se1.protection.inumbo.net>)
 id 1rcRws-000352-M8
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 15:22:22 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d69375db-d003-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 16:22:21 +0100 (CET)
Received: from PH8PR12MB7326.namprd12.prod.outlook.com (2603:10b6:510:216::7)
 by BY5PR12MB4324.namprd12.prod.outlook.com (2603:10b6:a03:209::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.20; Tue, 20 Feb
 2024 15:22:14 +0000
Received: from PH8PR12MB7326.namprd12.prod.outlook.com
 ([fe80::22eb:b0b:62c0:f3e5]) by PH8PR12MB7326.namprd12.prod.outlook.com
 ([fe80::22eb:b0b:62c0:f3e5%6]) with mapi id 15.20.7316.018; Tue, 20 Feb 2024
 15:22:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d69375db-d003-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gXfWUbgO0hHS0YdZa1OnXX2Z/3j+kvse4ByVQ+X0yqMKCwQG5b3x0OkTTiOGQmNLI0qwljVOPwK7P0lbtvJNk7cBhSizAjJhk9iulYB2v5YKIDLnyRgYRr78+CSDuW4uzu6VgSjxBv6m65qpvpqPf+POJRroni2DdZ+mmbU+4AcNrBN4aLVAdVTPPah9C4BK6fX9pvFQ+JFr1QrNQJGMGHIKzFzqAA+O27N34F0gPPUrgC16wJQW86/L3DWOZc1sGXl0A1fAIpCioGyvj+M9v9QkNr0PnXdRxwfZNrEO7GTdNw77sHuAVLuTNnCv+k+0BOtuRugP+BSeH2362HtY6A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=cT26UHKv7tMDiRIIHmrwq2RY7bBETYsV5OnSwpm1rxI=;
 b=NcQOuR5rkAmRjB/W8r7umRJFMRkbqxR7W78forBZ5BHg7kZANQsdazkV2zI+5WKypw2Tt+u7s1EN248XHU3JJ6N8Qa+DRGs27ADmmGAt1PCJ64YobEz8/Wfxs4i2XOfc+SScUm1DOsJ7KVkMONtT5PfIiLqZDKrYp0zZLJj6iwJnu4Zu3z5B/7yiuvWmZB4LsHv0DgpaXlomNuezrAMTaclHysjbEahdVS69I1Ag/uPA3EUYaGaJO8LN+q4Viv0jE5j93cxPbjtQLDbKE+eABQ/bek60Q8Zw0j3/HK9Cl1qL0rUJu2sGBNEByNR0u49AQ82NhWrQMHmbWKp1SfEobw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cT26UHKv7tMDiRIIHmrwq2RY7bBETYsV5OnSwpm1rxI=;
 b=jGLjxqGYJHUWMjlrpASYghlkpr/K/Ip6Bcxtl2HAprap0ql9e3iuo1haRmIZqN/SMhQxRfpGBBK6RX+IXb9Too+8Bdi/YPpkkal3SnWhBCpVHqQjH/vlDgJfsbP8dnxJcv35h/pN4dmIhFfF+BQ4wPPGeNC7EtsSCD/UgKmpu9g=
Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
Message-ID: <5f526deb-7376-4c34-85d1-d22b1b14d90e@amd.com>
Date: Tue, 20 Feb 2024 15:22:09 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v5 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
To: Jan Beulich <jbeulich@suse.com>,
 Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com, julien@xen.org,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, michal.orzel@amd.com,
 xen-devel@lists.xenproject.org
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
 <20240220121743.3680715-2-ayan.kumar.halder@amd.com>
 <e3a8ce5d-841e-4ff1-8d23-48de822972ef@suse.com>
Content-Language: en-GB
From: Ayan Kumar Halder <ayankuma@amd.com>
In-Reply-To: <e3a8ce5d-841e-4ff1-8d23-48de822972ef@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: DU2P250CA0009.EURP250.PROD.OUTLOOK.COM
 (2603:10a6:10:231::14) To PH8PR12MB7326.namprd12.prod.outlook.com
 (2603:10b6:510:216::7)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: PH8PR12MB7326:EE_|BY5PR12MB4324:EE_
X-MS-Office365-Filtering-Correlation-Id: 4ce0d645-b553-4183-3100-08dc3227b78a
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	iD2n98dwVvdhdcNyhimFPdGnxRIgZLQ43jut9y1fv6j8ZtQxXnHWBpsDKzzHddiR6rdUQnWfwra+HA3fvqKljJOLfjs4aX64ICN9HGa9FS7IOZc/vpbs7O7IZ62tX8yTIU3gFb0H2Fqxa687z/EAxLg/enoIEw14mNUQAq9S7eHrdtOoHjLwLzg4FcJUm1EYv08ouFG5+RIRB0Y4r0D+wfVoc9BeDMZfkWT5n5GbJdKWNwlAyr/5YdAzHZmYXzIIoEgBSqeXQPne8KjQ9iJUO6WWCJlsJ+cFp6FjNus4DpA+QkCp62AFETfnoMvVLfSB0HbV6yLTL9dlpfgUoDPovahkjmdX/RCbmI4k+bFd3Xv+BS9qUtHDkl1Kpb0ZiP0+E/Ke/NSRTW/06YjqAUoof7SprEReGB6spOqYnFyhsWA2kjEoDnrSUQ8hMLL5CEHM6/eqlKUx5KwTKeKY7wDRhgmbQou19Ey3WreJ7A5TY9vZPddrehvUoXlcy/E6HSpwwiPWCDglJp5DebYsajE1e4Jg8gut1qFBJBVra7edWN4=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH8PR12MB7326.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cDZERnovNTV1UzVrb051TllnTkJpNGJVR2tJUklNQmRuN3FuQWtUaHZ2Qkd2?=
 =?utf-8?B?cGhKNmxSTUlacnFmS2xqczZ5N3VlYzk4N0l6Z2dlZ3Jvd08zT3p2R0VXbDZx?=
 =?utf-8?B?dDQ1RWwxQXdEQ3p3by81Rng3RmxBR2s2MXJ6eHlhb2N2bFFRL2RIZThacW1i?=
 =?utf-8?B?aTVHYXRVMEE2RjVaNFQ0UHgzdzVpS0ExTERvYnBPRTFmVmpOMm83M3NCSE1u?=
 =?utf-8?B?UDdmK2F5K2l4VDRZMjZ3dTdqNVNUUjk0SXB0TzJMSUpzSTJ0MEtGUWZWMnhT?=
 =?utf-8?B?T21YZVZySnp6OEkrUWtXL3dhQU9salZ0QUFZNjB1TFVUSFhUbE50N084WmxQ?=
 =?utf-8?B?dXVybG1NaUVxL3BmZXlRdHpUQjJIamN6ZlJaUURleVBEUjBIVHU0Qk9YWmVO?=
 =?utf-8?B?N3daWjhvelJieWR6dUwxZHZZWmI2YURvNENRUU1aN1ZtWXo0em1KVTBPZlFH?=
 =?utf-8?B?aVhEWXQvUmdKQWdKZDhiR1hQUmdCWEs5Z1RqenBZRjZSakF4bTlubDl6N1Mr?=
 =?utf-8?B?ZFpCVHZuWmRmWkNBWHZqRW5JV1dTMkpvRDljTGEyc1FuaEU1ZHZCN0l2VW13?=
 =?utf-8?B?ZCtVZkZETjM5a1RLWnc2VTlvdUlWRnNVaDFxRkplMzZzR1RvSWpWMk9WRW5j?=
 =?utf-8?B?TStQZHMvRVY3NGlsTURSaENmNFhJcmo4QnFpMTVzaUdDR2lnclNqWVJGc3ln?=
 =?utf-8?B?bzZhSFUxTWxhTDkzcU01RmQvZlVXZ3dNSWdyUG8wMklPMTFSdlZTYkp5dVVC?=
 =?utf-8?B?aHJrL0o5bllqa29NTzk2VE9vT05zalRxY2djenVUdEw4eTM3YUtxT1dyWDNz?=
 =?utf-8?B?UlVubnZuOGc2UzdtY09RdGJJQ2NWampjR3NvTE5uckxCOENHd1hLcVc5VXN5?=
 =?utf-8?B?V3llYlI3WVkyb04xWjk1dExiTWN1S3VUdkZCMGg3a242T2RIamxKcFkvc0dT?=
 =?utf-8?B?T2VLYUI5OExWT3Jmb0lCb25ObmRBVlora0NleGhtMmsxRWVlcGVuNWNKZ0RK?=
 =?utf-8?B?bGZLelhrek5BblFoVUgrOCthS2MxT1dIM2FyNUZxdGRpMTMzZHRZaWNzRnhm?=
 =?utf-8?B?TFJ3ZTkrOHNJRjVEUk9DZlVUOVFPaUNWQzZMTSs1Y0JDWWo2N3MrN1U1bU93?=
 =?utf-8?B?dEI3MUtkaE05VExYTVRxa0Y2ODJMSGF0U3ZKNFNpZEVaR2dxQTFQbEl1SnFP?=
 =?utf-8?B?c1Z1MlBjZDhhRmUzU1VrRlRaQzRXZ0lOU2hnZ3ZxUk1UZTFSeWh4WjRsZTF3?=
 =?utf-8?B?SUlJbUhEY3pBMERHaVQ1bjZ4cE1ZNjdyVkN6MlEraEVENGhFa2djNGdiTFFj?=
 =?utf-8?B?QkN4SXBGTCtBM25DZUpXUG00dDVaQzFLUU5GM202dVFIOThxaERjUVMwQ2lM?=
 =?utf-8?B?dzVPY2w3Z1ZRUG1ScFpjV0dqVGlNazQ3cWpwZEJkeGh1ZlhScGlQUEdWSWlU?=
 =?utf-8?B?bnVjTkFVNldPUzRTa2lOSVMvTjNjQUg1SXR2cDdRSitGcmhkZ0FOOWZSWldt?=
 =?utf-8?B?cUVXMmlFcDd5ZmdLSkNQeVFxb01oaVNLMkxHeC9hZVU5bkhnMzZPY3dJN0xa?=
 =?utf-8?B?OER2Q3ZXbUR1QUdvRDdjei9CMitSRmttcW1pTjJTdFFLTmY2ZmRPeDJ1ZXBu?=
 =?utf-8?B?U0tFNklxRWpEUGhoWklYU0RRVkpNVmh0cGgxNDFvVDRpSms0czcyTnE1MFZQ?=
 =?utf-8?B?NWFrZlZ4cFFpRWVuY2ZpczJRS01DdUpvM09veG9hN3llVkRLdy9DV0tnTG5H?=
 =?utf-8?B?RUJ6ejFUZDBJc3ZtVzhGejhFdnQ2NnBQMzhlWWNOUVhVM01LQ25zTkg1Mysr?=
 =?utf-8?B?blpLaHBKVXM2V1ZiellXV1djOGhwNnZOcnp1SzZ1T01rZkNMSFB5akFZTzV6?=
 =?utf-8?B?a3kvb3VPUVdLVGdPMlo5ckIrOCs5SEsyamVtV3hKQm1rV1AyVU1kb09zdnFm?=
 =?utf-8?B?UWZYYmsvR2xhSGVEblcySjJBeXZYbUJwWStlRUx4RUpLeTFlMkd5MllUenkr?=
 =?utf-8?B?Q2tPTks3Mm5SYVZmYlEzMEpHSUd2aHRVSktWTTAyQytCT2VNQytBTGcxSDNt?=
 =?utf-8?B?WWdwR2l0RmNaMGFFTjEvaG5WUjZaYnV5MDhJTFZiRkt4WXdtUnFCOW1vbGRC?=
 =?utf-8?Q?JJBXykXGn4JsoPjA6+l3wWQBF?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4ce0d645-b553-4183-3100-08dc3227b78a
X-MS-Exchange-CrossTenant-AuthSource: PH8PR12MB7326.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2024 15:22:14.5305
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +mlsgQpsJOPqWBlmbjbT6jHIcSU6GAZDshaISLSwVPuzj3vPtuiHD9lmsdzxUfWbbRpquRBz99borctS/30PeA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4324

Hi Jan,

On 20/02/2024 12:33, Jan Beulich wrote:
> On 20.02.2024 13:17, Ayan Kumar Halder wrote:
>> There can be situations when the registers cannot be emulated to their full
>> functionality. This can be due to the complexity involved. In such cases, one
>> can emulate those registers as RAZ/WI for example. We call them as partial
>> emulation.
>>
>> Some registers are non-optional and as such there is nothing preventing an OS
>> from accessing them.
>> Instead of injecting undefined exception (thus crashing a guest), one may want
>> to prefer a partial emulation to let the guest running (in some cases accepting
>> the fact that it might result in unwanted behavior).
>>
>> A suitable example of this (as seen in subsequent patches) is emulation of
>> DBGDTRTX_EL0 (on Arm64) and DBGDTRTXINT(on Arm32). These non-optional
>> registers can be emulated as RAZ/WI and they can be enclosed within
>> CONFIG_PARTIAL_EMULATION.
>>
>> Further, "partial-emulation" command line option allows us to
>> enable/disable partial emulation at run time. While CONFIG_PARTIAL_EMULATION
>> enables support for partial emulation at compile time (i.e. adds code for
>> partial emulation), this option may be enabled or disabled by Yocto or other
>> build systems. However if the build system turns this option on, users
>> can use scripts like Imagebuilder to generate uboot-script which will append
>> "partial-emulation=true" to xen command line to turn on the partial emulation.
>> Thus, it helps to avoid rebuilding xen.
>>
>> By default, "CONFIG_PARTIAL_EMULATION=y" and "partial-emulation=false".
>> This is done so that Xen supports partial emulation. However, customers are
>> fully aware when they enable partial emulation. It's important to note that
>> enabling such support might result in unwanted/non-spec compliant behavior.
>>
>> Added a note in SUPPORT.md to clarify the security support for partial
>> emulation.
> There may have been prior discussion of this, but I view this as too little
> to justify ...
>
>> --- a/SUPPORT.md
>> +++ b/SUPPORT.md
>> @@ -101,6 +101,18 @@ Extension to the GICv3 interrupt controller to support MSI.
>>   
>>       Status: Experimental
>>   
>> +### ARM/Partial Emulation
>> +
>> +Enable partial emulation of registers, otherwise considered unimplemented,
>> +that would normally trigger a fault injection.
>> +
>> +    Status: Supported, with caveats
>> +
>> +Bugs allowing the userspace to attack the guest OS will not be considered
>> +security vulnerabilities.
>> +
>> +Bugs that could compromise Xen will be considered security vulnerabilities.
> ... the odd statement regarding in-guest vulnerabilities that might be
> introduced. I see only two ways of treating this as supported: Either
> you simply refuse emulation when the access is from user space,

I am wondering how do we enforce that.

Let me try to understand this with the current implementation of partial 
emulation for system registers.

1. DBGDTRTX_EL0 :- I understand that EL0 access to this register will 
cause a trap to EL2. The reason being MDCR_EL2.TDA == 1.

In that case, if we refuse emulation then an undef exception is injected 
to the guest (this is the behavior as of today even without this patch).

So, are you saying that the undef exception is to be injected to the 
user space process. This may be possible for Arm64 guests 
(inject_undef64_exception() needs to be changed).

However for Arm32 guests, this may not be possible as the mode changes 
to PSR_MODE_UND.

Let me know if I understood you correctly.

> or you
> support that mode of emulation as much as that of kernel space accesses.

Do you mean we support partial emulation only for traps from EL1 mode ?

- Ayan

> After all "injecting undefined exception (thus crashing a guest)"
> (quoting the description) doesn't seem quite right: If you inject such
> an exception for a user space access, all that ought to happen is the
> user space process to be killed.
>
> Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 15:42:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 15:42:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683605.1063192 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcSFi-0006o2-Dw; Tue, 20 Feb 2024 15:41:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683605.1063192; Tue, 20 Feb 2024 15:41:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcSFi-0006nv-B5; Tue, 20 Feb 2024 15:41:50 +0000
Received: by outflank-mailman (input) for mailman id 683605;
 Tue, 20 Feb 2024 15:41:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=UtGB=J5=apertussolutions.com=dpsmith@srs-se1.protection.inumbo.net>)
 id 1rcSFg-0006np-TH
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 15:41:48 +0000
Received: from sender4-of-o50.zoho.com (sender4-of-o50.zoho.com
 [136.143.188.50]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8d005dda-d006-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 16:41:46 +0100 (CET)
Received: from [10.10.1.138] (static-72-81-132-2.bltmmd.fios.verizon.net
 [72.81.132.2]) by mx.zohomail.com
 with SMTPS id 1708443698629666.9694419562685;
 Tue, 20 Feb 2024 07:41:38 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d005dda-d006-11ee-8a52-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; t=1708443700; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=bDC+gZTP7tPVhpHKVnJDkPnHDhDfUjSUnMNofHvMR068ffliKbtNmChmbbszVJ+M9ugAsfIblbVg3cNP3FFPmUvKBeuv4RYAAkoXdDWQMk610EZZLe3524envUgikeipqLx4+mnVecapWayQjPzx0SfFle4HyAMobYUj7+w9RPQ=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1708443700; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=irvYXm4uYz0DwWBjM/Ejk/RAcsYLz7Pr2lEZ9AJYBmg=; 
	b=TWXDdJA2ca1GvQsLipz9QPuZCSFdUZICkjstd26wBMjFomPYD8QJqRh/Pt1I9wsKc6okm/BrXqEdz/IfzKilUjtNtk8wE3OhOPX3A+pqHiS422zED6fcZdhrJnNOwMBL0mu1/+TKYFwNBJqM0+Xvm6o1eDThIZf/n6vOZSt2nrI=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=apertussolutions.com;
	spf=pass  smtp.mailfrom=dpsmith@apertussolutions.com;
	dmarc=pass header.from=<dpsmith@apertussolutions.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1708443700;
	s=zoho; d=apertussolutions.com; i=dpsmith@apertussolutions.com;
	h=Message-ID:Date:Date:MIME-Version:Subject:Subject:To:To:Cc:Cc:References:From:From:In-Reply-To:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=irvYXm4uYz0DwWBjM/Ejk/RAcsYLz7Pr2lEZ9AJYBmg=;
	b=C3macS6wFvyJHuAam1EixoDqvZ2I1GBAiSLHfQM3MpKOia42H4Jmms3HaPqIi5kQ
	ZVFKhzKqzBgy5szpQsYn8IWnX+/g3txiuJ+dRFY17q2J4bV6fjg/vBaakquvhUhxCBB
	cuCq1Z+JgFI/vc3+lqIu2DUhiy+kM/ltcpg/xPAw=
Message-ID: <a2d94dc8-aca3-4fa4-a5a4-1de268010747@apertussolutions.com>
Date: Tue, 20 Feb 2024 10:41:35 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen: move BUG_ON(), WARN_ON(), ASSERT(),
 ASSERT_UNREACHABLE() to xen/bug.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>, Shawn Anastasio <sanastasio@raptorengineering.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>
References: <4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
Autocrypt: addr=dpsmith@apertussolutions.com; keydata=
 xsJuBFYrueARCACPWL3r2bCSI6TrkIE/aRzj4ksFYPzLkJbWLZGBRlv7HQLvs6i/K4y/b4fs
 JDq5eL4e9BdfdnZm/b+K+Gweyc0Px2poDWwKVTFFRgxKWq9R7McwNnvuZ4nyXJBVn7PTEn/Z
 G7D08iZg94ZsnUdeXfgYdJrqmdiWA6iX9u84ARHUtb0K4r5WpLUMcQ8PVmnv1vVrs/3Wy/Rb
 foxebZNWxgUiSx+d02e3Ad0aEIur1SYXXv71mqKwyi/40CBSHq2jk9eF6zmEhaoFi5+MMMgX
 X0i+fcBkvmT0N88W4yCtHhHQds+RDbTPLGm8NBVJb7R5zbJmuQX7ADBVuNYIU8hx3dF3AQCm
 601w0oZJ0jGOV1vXQgHqZYJGHg5wuImhzhZJCRESIwf+PJxik7TJOgBicko1hUVOxJBZxoe0
 x+/SO6tn+s8wKlR1Yxy8gYN9ZRqV2I83JsWZbBXMG1kLzV0SAfk/wq0PAppA1VzrQ3JqXg7T
 MZ3tFgxvxkYqUP11tO2vrgys+InkZAfjBVMjqXWHokyQPpihUaW0a8mr40w9Qui6DoJj7+Gg
 DtDWDZ7Zcn2hoyrypuht88rUuh1JuGYD434Q6qwQjUDlY+4lgrUxKdMD8R7JJWt38MNlTWvy
 rMVscvZUNc7gxcmnFUn41NPSKqzp4DDRbmf37Iz/fL7i01y7IGFTXaYaF3nEACyIUTr/xxi+
 MD1FVtEtJncZNkRn7WBcVFGKMAf+NEeaeQdGYQ6mGgk++i/vJZxkrC/a9ZXme7BhWRP485U5
 sXpFoGjdpMn4VlC7TFk2qsnJi3yF0pXCKVRy1ukEls8o+4PF2JiKrtkCrWCimB6jxGPIG3lk
 3SuKVS/din3RHz+7Sr1lXWFcGYDENmPd/jTwr1A1FiHrSj+u21hnJEHi8eTa9029F1KRfocp
 ig+k0zUEKmFPDabpanI323O5Tahsy7hwf2WOQwTDLvQ+eqQu40wbb6NocmCNFjtRhNZWGKJS
 b5GrGDGu/No5U6w73adighEuNcCSNBsLyUe48CE0uTO7eAL6Vd+2k28ezi6XY4Y0mgASJslb
 NwW54LzSSM0uRGFuaWVsIFAuIFNtaXRoIDxkcHNtaXRoQGFwZXJ0dXNzb2x1dGlvbnMuY29t
 PsJ6BBMRCAAiBQJWK7ngAhsjBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAAKCRBTc6WbYpR8
 KrQ9AP94+xjtFfJ8gj5c7PVx06Zv9rcmFUqQspZ5wSEkvxOuQQEAg6qEsPYegI7iByLVzNEg
 7B7fUG7pqWIfMqFwFghYhQzOwU0EViu54BAIAL6MXXNlrJ5tRUf+KMBtVz1LJQZRt/uxWrCb
 T06nZjnbp2UcceuYNbISOVHGXTzu38r55YzpkEA8eURQf+5hjtvlrOiHxvpD+Z6WcpV6rrMB
 kcAKWiZTQihW2HoGgVB3gwG9dCh+n0X5OzliAMiGK2a5iqnIZi3o0SeW6aME94bSkTkuj6/7
 OmH9KAzK8UnlhfkoMg3tXW8L6/5CGn2VyrjbB/rcrbIR4mCQ+yCUlocuOjFCJhBd10AG1IcX
 OXUa/ux+/OAV9S5mkr5Fh3kQxYCTcTRt8RY7+of9RGBk10txi94dXiU2SjPbassvagvu/hEi
 twNHms8rpkSJIeeq0/cAAwUH/jV3tXpaYubwcL2tkk5ggL9Do+/Yo2WPzXmbp8vDiJPCvSJW
 rz2NrYkd/RoX+42DGqjfu8Y04F9XehN1zZAFmCDUqBMa4tEJ7kOT1FKJTqzNVcgeKNBGcT7q
 27+wsqbAerM4A0X/F/ctjYcKwNtXck1Bmd/T8kiw2IgyeOC+cjyTOSwKJr2gCwZXGi5g+2V8
 NhJ8n72ISPnOh5KCMoAJXmCF+SYaJ6hIIFARmnuessCIGw4ylCRIU/TiXK94soilx5aCqb1z
 ke943EIUts9CmFAHt8cNPYOPRd20pPu4VFNBuT4fv9Ys0iv0XGCEP+sos7/pgJ3gV3pCOric
 p15jV4PCYQQYEQgACQUCViu54AIbDAAKCRBTc6WbYpR8Khu7AP9NJrBUn94C/3PeNbtQlEGZ
 NV46Mx5HF0P27lH3sFpNrwD/dVdZ5PCnHQYBZ287ZxVfVr4Zuxjo5yJbRjT93Hl0vMY=
In-Reply-To: <4887b2d91a4bf2e8b4b66f03964259651981403b.1706897023.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ZohoMailClient: External

On 2/2/24 13:03, Oleksii Kurochko wrote:
> Move the macros mentioned in the commit subject to their appropriate
> locations.
> Additionally, eliminate the dependency of xen/lib.h from xen/bug.h and
> include "xen/bug.h" in files where xen/bug.h macros are utilized.
> 
> Most of the changes were made because a file requires macros from xen/bug.h,
> except for some files for Arm which require definitions of BUG_OPCODE,
> BUG_INSTR, BUG_FN_REG.
> 
> xen/lib.h was added to list-sort.c ( otherwise compilation errors related
> to {d}printk occur during compilation of list-sort.c. ) as xen/lib.h was
> removed from xen/list.h. Since nothing in xen/list.h depends on xen/lib.h
> functionality and only xen/bug.h is needed.
> 
> cpufeature.h requires the inclusion of <xen/cache.h>;
> otherwise, the following error will occur:
> ld: common/monitor.o:/build/xen/./arch/x86/include/asm/cpufeature.h:41:
> multiple definitions of `__cacheline_aligned';
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---

<snip/>

> diff --git a/xen/xsm/flask/ss/ebitmap.h b/xen/xsm/flask/ss/ebitmap.h
> index bb43de891d..7b2da8179a 100644
> --- a/xen/xsm/flask/ss/ebitmap.h
> +++ b/xen/xsm/flask/ss/ebitmap.h
> @@ -15,6 +15,7 @@
>   #define _SS_EBITMAP_H_
>   
>   #include <xen/bitmap.h>
> +#include <xen/bug.h>
>   
>   #define EBITMAP_UNIT_NUMS	((32 - sizeof(void *) - sizeof(u32))	\
>   					/ sizeof(unsigned long))

Acked-by: Daniel P. Smith <dpsmith@apertussolutions.com>


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 15:50:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 15:50:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683612.1063203 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcSON-0000HY-7f; Tue, 20 Feb 2024 15:50:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683612.1063203; Tue, 20 Feb 2024 15:50:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcSON-0000HR-4i; Tue, 20 Feb 2024 15:50:47 +0000
Received: by outflank-mailman (input) for mailman id 683612;
 Tue, 20 Feb 2024 15:50:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vPn8=J5=lst.de=hch@srs-se1.protection.inumbo.net>)
 id 1rcSOL-0000Fw-HE
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 15:50:45 +0000
Received: from verein.lst.de (verein.lst.de [213.95.11.211])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ce701a5a-d007-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 16:50:44 +0100 (CET)
Received: by verein.lst.de (Postfix, from userid 2407)
 id C66C968D07; Tue, 20 Feb 2024 16:50:41 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce701a5a-d007-11ee-8a52-1f161083a0e0
Date: Tue, 20 Feb 2024 16:50:41 +0100
From: Christoph Hellwig <hch@lst.de>
To: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Cc: Christoph Hellwig <hch@lst.de>, Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Jens Axboe <axboe@kernel.dk>, xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: Re: [PATCH 1/4] xen-blkfront: set max_discard/secure erase limits
 to UINT_MAX
Message-ID: <20240220155041.GA17393@lst.de>
References: <20240220084935.3282351-1-hch@lst.de> <20240220084935.3282351-2-hch@lst.de> <ZdSPj32Ww80nKQhM@macbook>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZdSPj32Ww80nKQhM@macbook>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Tue, Feb 20, 2024 at 12:39:59PM +0100, Roger Pau Monn wrote:
> On Tue, Feb 20, 2024 at 09:49:32AM +0100, Christoph Hellwig wrote:
> > Currently xen-blkfront set the max discard limit to the capacity of
> > the device, which is suboptimal when the capacity changes.  Just set
> > it to UINT_MAX, which has the same effect except and is simpler.
> 
> Extra 'except' in the line above?

Yes, thanks.


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 15:53:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 15:53:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683615.1063212 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcSRD-0000sY-KM; Tue, 20 Feb 2024 15:53:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683615.1063212; Tue, 20 Feb 2024 15:53:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcSRD-0000sR-Hs; Tue, 20 Feb 2024 15:53:43 +0000
Received: by outflank-mailman (input) for mailman id 683615;
 Tue, 20 Feb 2024 15:53:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=vPn8=J5=lst.de=hch@srs-se1.protection.inumbo.net>)
 id 1rcSRB-0000sE-T8
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 15:53:41 +0000
Received: from verein.lst.de (verein.lst.de [213.95.11.211])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 36dfb430-d008-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 16:53:39 +0100 (CET)
Received: by verein.lst.de (Postfix, from userid 2407)
 id B777D68D07; Tue, 20 Feb 2024 16:53:36 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 36dfb430-d008-11ee-98f5-efadbce2ee36
Date: Tue, 20 Feb 2024 16:53:36 +0100
From: Christoph Hellwig <hch@lst.de>
To: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Cc: Christoph Hellwig <hch@lst.de>, Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Jens Axboe <axboe@kernel.dk>, xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: Re: [PATCH 4/4] xen-blkfront: atomically update queue limits
Message-ID: <20240220155336.GB17393@lst.de>
References: <20240220084935.3282351-1-hch@lst.de> <20240220084935.3282351-5-hch@lst.de> <ZdScey8AJvBykWa8@macbook>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ZdScey8AJvBykWa8@macbook>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Tue, Feb 20, 2024 at 01:35:07PM +0100, Roger Pau Monn wrote:
> On Tue, Feb 20, 2024 at 09:49:35AM +0100, Christoph Hellwig wrote:
> > Pass the initial queue limits to blk_mq_alloc_disk and use the
> > blkif_set_queue_limits API to update the limits on reconnect.
> 
> Allocating queue_limits on the stack might be a bit risky, as I fear
> this struct is likely to grow?

It might grow a little bit, but it's not actually that large, epecially
in a simple probe context that isn't in memory reclaim or similar.

> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> 
> Acked-by: Roger Pau Monn <roger.pau@citrix.com>
> 
> Just one addition while you are already modifying a line.
> 
> > ---
> >  drivers/block/xen-blkfront.c | 41 ++++++++++++++++++++----------------
> >  1 file changed, 23 insertions(+), 18 deletions(-)
> > 
> > diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
> > index 7664638a0abbfa..b77707ca2c5aa6 100644
> > --- a/drivers/block/xen-blkfront.c
> > +++ b/drivers/block/xen-blkfront.c
> > @@ -941,37 +941,35 @@ static const struct blk_mq_ops blkfront_mq_ops = {
> >  	.complete = blkif_complete_rq,
> >  };
> >  
> > -static void blkif_set_queue_limits(struct blkfront_info *info)
> > +static void blkif_set_queue_limits(struct blkfront_info *info,
> 
> While there, could you also constify info?

Sure.


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 17:36:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 17:36:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683633.1063227 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcU23-00058l-S4; Tue, 20 Feb 2024 17:35:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683633.1063227; Tue, 20 Feb 2024 17:35:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcU23-00058e-Pc; Tue, 20 Feb 2024 17:35:51 +0000
Received: by outflank-mailman (input) for mailman id 683633;
 Tue, 20 Feb 2024 17:35:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=YWH4=J5=tls.msk.ru=mjt@srs-se1.protection.inumbo.net>)
 id 1rcU23-00058Y-Ad
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 17:35:51 +0000
Received: from isrv.corpit.ru (isrv.corpit.ru [86.62.121.231])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7cb9c66c-d016-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 18:35:49 +0100 (CET)
Received: from tsrv.corpit.ru (tsrv.tls.msk.ru [192.168.177.2])
 by isrv.corpit.ru (Postfix) with ESMTP id F2C214F090;
 Tue, 20 Feb 2024 20:36:07 +0300 (MSK)
Received: from [192.168.177.130] (mjt.wg.tls.msk.ru [192.168.177.130])
 by tsrv.corpit.ru (Postfix) with ESMTP id C310F85499;
 Tue, 20 Feb 2024 20:35:47 +0300 (MSK)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7cb9c66c-d016-11ee-8a52-1f161083a0e0
Message-ID: <0049ecc3-f3f9-4668-a13f-11120a7e8627@tls.msk.ru>
Date: Tue, 20 Feb 2024 20:35:47 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v1 03/21] Xen headers: correct typos
Content-Language: en-US
To: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>,
 qemu-trivial@nongnu.org
Cc: qemu-devel@nongnu.org, Stefano Stabellini <sstabellini@kernel.org>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708419115.git.manos.pitsidianakis@linaro.org>
 <dca380ada7f7eabfb473828eccafc2902b1985b7.1708419115.git.manos.pitsidianakis@linaro.org>
From: Michael Tokarev <mjt@tls.msk.ru>
Autocrypt: addr=mjt@tls.msk.ru; keydata=
 xsBLBETIiwkBCADh3cFB56BQYPjtMZCfK6PSLR8lw8EB20rsrPeJtd91IoNZlnCjSoxd9Th1
 bLUR8YlpRJ2rjc6O1Bc04VghqUOHgS/tYt8vLjcGWixzdhSLJgPDK3QQZPAvBjMbCt1B6euC
 WuD87Pv5Udlpnzf4aMwxkgfTusx+ynae/o+T5r7tXD+isccbC3SiGhmAPxFyY3zGcFk4+Rxc
 0tP8YY2FWE/baHu+lBDTUN79efWAkHhex1XzVZsV7ZD16rzDbXFK5m6ApvGJWlr5YDEEydTF
 WwmvwBfr4OINVxzEG/ujNiG4fpMf2NsnFGyB9aSbFjXZevB4qWkduYYW+xpK1EryszHtAAYp
 zSBNaWNoYWVsIFRva2FyZXYgPG1qdEB0bHMubXNrLnJ1PsLAlgQTAQoAQAIbAwYLCQgHAwIE
 FQIIAwQWAgMBAh4BAheAAhkBFiEEbuGV0Yhuj/uBDUMkRXzgoIBEZcUFAmBbcjwFCS5e6jMA
 CgkQRXzgoIBEZcUTIQgA1hPsOF82pXxbcJXBMc4zB9OQu4AlnZvERoGyw7I2222QzaN3RFuj
 Fia//mapXzpIQNF08l/AA6cx+CKPeGnXwyZfF9fLa4RfifmdNKME8C00XlqnoJDZBGzq8yMy
 LAKDxl9OQWFcDwDxV+irg5U3fbtNVhvV0kLbS2TyQ0aU5w60ERS2NcyDWplOo7AOzZWChcA4
 UFf78oVdZdCW8YDtU0uQFhA9moNnrePy1HSFqduxnlFHEI+fDj/TiOm2ci48b8SBBJOIJFjl
 SBgH8+SfT9ZqkzhN9vh3YJ49831NwASVm0x1rDHcIwWD32VFZViZ3NjehogRNH9br0PSUYOC
 3s7ATQRX2BjLAQgAnak3m0imYOkv2tO/olULFa686tlwuvl5kL0NWCdGQeXv2uMxy36szcrh
 K1uYhpiQv4r2qNd8BJtYlnYIK16N8GBdkplaDIHcBMbU4t+6bQzEIJIaWoq1hzakmHHngE2a
 pNMnUf/01GFvCRPlv3imkujE/5ILbagjtdyJaHF0wGOSlTnNT4W8j+zPJ/XK0I5EVQwtbmoc
 GY62LKxxz2pID6sPZV4zQVY4JdUQaFvOz1emnBxakkt0cq3Qnnqso1tjiy7vyH9CAwPR/48W
 fpK6dew4Fk+STYtBeixOTfSUS8qRS/wfpUeNa5RnEdTtFQ9IcjpQ/nPrvJJsu9FqwlpjMwAR
 AQABwsBlBBgBCAAPBQJX2BjLAhsMBQkSzAMAAAoJEEV84KCARGXFUKcH/jqKETECkbyPktdP
 cWVqw2ZIsmGxMkIdnZTbPwhORseGXMHadQODayhU9GWfCDdSPkWDWzMamD+qStfl9MhlVT60
 HTbo6wu1W/ogUS70qQPTY9IfsvAj6f8TlSlK0eLMa3s2UxL2oe5FkNs2CnVeRlr4Yqvp/ZQV
 6LXtew4GPRrmplUT/Cre9QIUqR4pxYCQaMoOXQQw3Y0csBwoDYUQujn3slbDJRIweHoppBzT
 rM6ZG5ldWQN3n3d71pVuv80guylX8+TSB8Mvkqwb5I36/NAFKl0CbGbTuQli7SmNiTAKilXc
 Y5Uh9PIrmixt0JrmGVRzke6+11mTjVlio/J5dCM=
In-Reply-To: <dca380ada7f7eabfb473828eccafc2902b1985b7.1708419115.git.manos.pitsidianakis@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

20.02.2024 11:52, Manos Pitsidianakis пишет:
> Correct typos automatically found with the `typos` tool
> <https://crates.io/crates/typos>
> 
> Signed-off-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
> ---
>   include/hw/xen/interface/arch-x86/xen-x86_64.h | 2 +-
>   include/hw/xen/interface/arch-x86/xen.h        | 2 +-
>   include/hw/xen/interface/event_channel.h       | 2 +-
>   include/hw/xen/interface/grant_table.h         | 2 +-
>   include/hw/xen/interface/hvm/hvm_op.h          | 2 +-
>   include/hw/xen/interface/io/blkif.h            | 4 ++--
>   include/hw/xen/interface/io/fbif.h             | 2 +-
>   include/hw/xen/interface/io/kbdif.h            | 2 +-
>   include/hw/xen/interface/io/ring.h             | 2 +-
>   include/hw/xen/interface/memory.h              | 2 +-
>   include/hw/xen/interface/physdev.h             | 4 ++--
>   include/hw/xen/interface/xen.h                 | 4 ++--
>   12 files changed, 15 insertions(+), 15 deletions(-)

include/hw/xen/ is imported from xen, IIRC it should not be edited
directly in qemu.

/mjt


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 17:44:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 17:44:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683638.1063239 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcUAN-0006eE-Oc; Tue, 20 Feb 2024 17:44:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683638.1063239; Tue, 20 Feb 2024 17:44:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcUAN-0006e7-Jy; Tue, 20 Feb 2024 17:44:27 +0000
Received: by outflank-mailman (input) for mailman id 683638;
 Tue, 20 Feb 2024 17:44:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qAG8=J5=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rcUAM-0006e1-Ns
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 17:44:26 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b045ece1-d017-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 18:44:25 +0100 (CET)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2d208be133bso60005891fa.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 09:44:25 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 cx19-20020a05620a51d300b007873df5a625sm3585374qkb.97.2024.02.20.09.44.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 09:44:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b045ece1-d017-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708451064; x=1709055864; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=fN3AioTtNiF2G1C/gJr0Z8XOqGfeC2tvwzpfkBsnwog=;
        b=l6U2u9BnqGtp9DLXSmgyk0OzVjPnjk69P5Nh4tZ9yB/u5bLjmnDT47+78UJfQjEg3V
         numByEzCrGYePO1GlYSQ1IPLeXxSJ56XYAEktnRFDjNGhvB1waINZ9ymHkr9JkbCNZOo
         xPxoLO/4SCQmLgUOfuvcmC2b4imMSe/XeQIVw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708451064; x=1709055864;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=fN3AioTtNiF2G1C/gJr0Z8XOqGfeC2tvwzpfkBsnwog=;
        b=kFwxMQJbsrzDZFkhS8L1gQeIp3vhJEuMK1Of84udYMPybyTe6xUTK8PY/3n2IrQ9Wi
         lGUS4OxCAsnJAkp3e6lZXRrLuqN1J2IGXalSyMADcSIUAysXvJ+Ha+S3sanV8WV6/u9W
         cf/5/+IugVuGfCVeTQVKCVnDSoEeFsLNIXuxDl0R/ZK/EI4jccG9LOvBkKBKYJyd8Ia3
         ThckamYWzFzQhfNoTqF4P+0R8qwX1kUDtg3rTysdaxT/qGYzOmw3WpuIAaF/zzakRhFp
         CKgPYB9aOTcwd+b5AdefCaCZfqYUzjtsTeCzjnS8Y0IzNUVVw7pMgoit09j1qBzGl+Se
         SZ4A==
X-Gm-Message-State: AOJu0YzNJZxE61AZAI0RYXFtwNNG7JvBPkeGFoi6AkxUGE7zrF6n6+nl
	OQFnYoJFsB7eeeYOnOWRBbf8RcsdCank7cyaWHEbxLMEnVaRfCYn5M6reuEcAxy/Z/y3RiOp4i2
	p
X-Google-Smtp-Source: AGHT+IHmGvbkMFw3zkVleaXq/kyUxVdLLslfkum2vmnLrPA+IyzJUIHNEXK0WeJJv+hyOqfnaNUTaA==
X-Received: by 2002:a2e:b005:0:b0:2d2:3757:c6b with SMTP id y5-20020a2eb005000000b002d237570c6bmr4961778ljk.0.1708451064254;
        Tue, 20 Feb 2024 09:44:24 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	x86@kernel.org,
	"H. Peter Anvin" <hpa@zytor.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	linux-kernel@vger.kernel.org,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH RFC] x86/xen: attempt to inflate the memory balloon on PVH
Date: Tue, 20 Feb 2024 18:43:41 +0100
Message-ID: <20240220174341.56131-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When running as PVH or HVM Linux will use holes in the memory map as scratch
space to map grants, foreign domain pages and possibly miscellaneous other
stuff.  However the usage of such memory map holes for Xen purposes can be
problematic.  The request of holesby Xen happen quite early in the kernel boot
process (grant table setup already uses scratch map space), and it's possible
that by then not all devices have reclaimed their MMIO space.  It's not
unlikely for chunks of Xen scratch map space to end up using PCI bridge MMIO
window memory, which (as expected) causes quite a lot of issues in the system.

At least for PVH dom0 we have the possibility of using regions marked as
UNUSABLE in the e820 memory map.  Either if the region is UNUSABLE in the
native memory map, or it has been converted into UNUSABLE in order to hide RAM
regions from dom0, the second stage translation page-tables can populate those
areas without issues.

PV already has this kind of logic, where the balloon driver is inflated at
boot.  Re-use the current logic in order to also inflate it when running as
PVH.  onvert UNUSABLE regions up to the ratio specified in EXTRA_MEM_RATIO to
RAM, while reserving them using xen_add_extra_mem() (which is also moved so
it's no longer tied to CONFIG_PV).

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
RFC reasons:

 * Note that it would be preferred for the hypervisor to provide an explicit
   range to be used as scratch mapping space, but that requires changes to Xen,
   and it's not fully clear whether Xen can figure out the position of all MMIO
   regions at boot in order to suggest a scratch mapping region for dom0.

 * Should the whole set of xen_{add,del,chk,inv}_extra_mem() functions be moved
   to a different file?  For the purposes of PVH only xen_add_extra_mem() is
   moved and the chk and inv ones are PV specific and might not want moving to
   a separate file just to guard them with CONFIG_PV.
---
 arch/x86/include/asm/xen/hypervisor.h |  1 +
 arch/x86/platform/pvh/enlighten.c     |  3 ++
 arch/x86/xen/enlighten.c              | 32 +++++++++++++
 arch/x86/xen/enlighten_pvh.c          | 68 +++++++++++++++++++++++++++
 arch/x86/xen/setup.c                  | 44 -----------------
 arch/x86/xen/xen-ops.h                | 14 ++++++
 drivers/xen/balloon.c                 |  2 -
 7 files changed, 118 insertions(+), 46 deletions(-)

diff --git a/arch/x86/include/asm/xen/hypervisor.h b/arch/x86/include/asm/xen/hypervisor.h
index a9088250770f..31e2bf8d5db7 100644
--- a/arch/x86/include/asm/xen/hypervisor.h
+++ b/arch/x86/include/asm/xen/hypervisor.h
@@ -62,6 +62,7 @@ void xen_arch_unregister_cpu(int num);
 #ifdef CONFIG_PVH
 void __init xen_pvh_init(struct boot_params *boot_params);
 void __init mem_map_via_hcall(struct boot_params *boot_params_p);
+void __init xen_reserve_extra_memory(struct boot_params *bootp);
 #endif
 
 /* Lazy mode for batching updates / context switch */
diff --git a/arch/x86/platform/pvh/enlighten.c b/arch/x86/platform/pvh/enlighten.c
index 00a92cb2c814..a12117f3d4de 100644
--- a/arch/x86/platform/pvh/enlighten.c
+++ b/arch/x86/platform/pvh/enlighten.c
@@ -74,6 +74,9 @@ static void __init init_pvh_bootparams(bool xen_guest)
 	} else
 		xen_raw_printk("Warning: Can fit ISA range into e820\n");
 
+	if (xen_guest)
+		xen_reserve_extra_memory(&pvh_bootparams);
+
 	pvh_bootparams.hdr.cmd_line_ptr =
 		pvh_start_info.cmdline_paddr;
 
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 3c61bb98c10e..a01ca255b0c6 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -6,6 +6,7 @@
 #include <linux/console.h>
 #include <linux/cpu.h>
 #include <linux/kexec.h>
+#include <linux/memblock.h>
 #include <linux/slab.h>
 #include <linux/panic_notifier.h>
 
@@ -350,3 +351,34 @@ void xen_arch_unregister_cpu(int num)
 }
 EXPORT_SYMBOL(xen_arch_unregister_cpu);
 #endif
+
+/* Amount of extra memory space we add to the e820 ranges */
+struct xen_memory_region xen_extra_mem[XEN_EXTRA_MEM_MAX_REGIONS] __initdata;
+
+void __init xen_add_extra_mem(unsigned long start_pfn, unsigned long n_pfns)
+{
+	unsigned int i;
+
+	/*
+	 * No need to check for zero size, should happen rarely and will only
+	 * write a new entry regarded to be unused due to zero size.
+	 */
+	for (i = 0; i < XEN_EXTRA_MEM_MAX_REGIONS; i++) {
+		/* Add new region. */
+		if (xen_extra_mem[i].n_pfns == 0) {
+			xen_extra_mem[i].start_pfn = start_pfn;
+			xen_extra_mem[i].n_pfns = n_pfns;
+			break;
+		}
+		/* Append to existing region. */
+		if (xen_extra_mem[i].start_pfn + xen_extra_mem[i].n_pfns ==
+		    start_pfn) {
+			xen_extra_mem[i].n_pfns += n_pfns;
+			break;
+		}
+	}
+	if (i == XEN_EXTRA_MEM_MAX_REGIONS)
+		printk(KERN_WARNING "Warning: not enough extra memory regions\n");
+
+	memblock_reserve(PFN_PHYS(start_pfn), PFN_PHYS(n_pfns));
+}
diff --git a/arch/x86/xen/enlighten_pvh.c b/arch/x86/xen/enlighten_pvh.c
index ada3868c02c2..c28f073c1df5 100644
--- a/arch/x86/xen/enlighten_pvh.c
+++ b/arch/x86/xen/enlighten_pvh.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 #include <linux/acpi.h>
 #include <linux/export.h>
+#include <linux/mm.h>
 
 #include <xen/hvc-console.h>
 
@@ -72,3 +73,70 @@ void __init mem_map_via_hcall(struct boot_params *boot_params_p)
 	}
 	boot_params_p->e820_entries = memmap.nr_entries;
 }
+
+/*
+ * Reserve e820 UNUSABLE regions to inflate the memory balloon.
+ *
+ * On PVH dom0 the host memory map is used, RAM regions available to dom0 are
+ * located as the same place as in the native memory map, but since dom0 gets
+ * less memory than the total amount of host RAM the ranges that can't be
+ * populated are converted from RAM -> UNUSABLE.  Use such regions (up to the
+ * ratio signaled in EXTRA_MEM_RATIO) in order to inflate the balloon driver at
+ * boot.  Doing so prevents the guest (even if just temporary) from using holes
+ * in the memory map in order to map grants or foreign addresses, and
+ * hopefully limits the risk of a clash with a device MMIO region.  Ideally the
+ * hypervisor should notify us which memory ranges are suitable for creating
+ * foreign mappings, but that's not yet implemented.
+ */
+void __init xen_reserve_extra_memory(struct boot_params *bootp)
+{
+	unsigned int i, ram_pages = 0, extra_pages;
+
+	for (i = 0; i < bootp->e820_entries; i++) {
+		struct boot_e820_entry *e = &bootp->e820_table[i];
+
+		if (e->type != E820_TYPE_RAM)
+			continue;
+		ram_pages += PFN_DOWN(e->addr + e->size) - PFN_UP(e->addr);
+	}
+
+	/* Max amount of extra memory. */
+	extra_pages = EXTRA_MEM_RATIO * ram_pages;
+
+	/*
+	 * Convert UNUSABLE ranges to RAM and reserve them for foreign mapping
+	 * purposes.
+	 */
+	for (i = 0; i < bootp->e820_entries && extra_pages; i++) {
+		struct boot_e820_entry *e = &bootp->e820_table[i];
+		unsigned long pages;
+
+		if (e->type != E820_TYPE_UNUSABLE)
+			continue;
+
+		pages = min(extra_pages,
+			PFN_DOWN(e->addr + e->size) - PFN_UP(e->addr));
+
+		if (pages != (PFN_DOWN(e->addr + e->size) - PFN_UP(e->addr))) {
+			struct boot_e820_entry *next;
+
+			if (bootp->e820_entries ==
+			    ARRAY_SIZE(bootp->e820_table))
+				/* No space left to split - skip region. */
+				continue;
+
+			/* Split entry. */
+			next = e + 1;
+			memmove(next, e,
+				(bootp->e820_entries - i) * sizeof(*e));
+			bootp->e820_entries++;
+			next->addr = PAGE_ALIGN(e->addr) + PFN_PHYS(pages);
+			e->size = next->addr - e->addr;
+			next->size -= e->size;
+		}
+		e->type = E820_TYPE_RAM;
+		extra_pages -= pages;
+
+		xen_add_extra_mem(PFN_UP(e->addr), pages);
+	}
+}
diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index b3e37961065a..380591028cb8 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -38,9 +38,6 @@
 
 #define GB(x) ((uint64_t)(x) * 1024 * 1024 * 1024)
 
-/* Amount of extra memory space we add to the e820 ranges */
-struct xen_memory_region xen_extra_mem[XEN_EXTRA_MEM_MAX_REGIONS] __initdata;
-
 /* Number of pages released from the initial allocation. */
 unsigned long xen_released_pages;
 
@@ -64,18 +61,6 @@ static struct {
 } xen_remap_buf __initdata __aligned(PAGE_SIZE);
 static unsigned long xen_remap_mfn __initdata = INVALID_P2M_ENTRY;
 
-/*
- * The maximum amount of extra memory compared to the base size.  The
- * main scaling factor is the size of struct page.  At extreme ratios
- * of base:extra, all the base memory can be filled with page
- * structures for the extra memory, leaving no space for anything
- * else.
- *
- * 10x seems like a reasonable balance between scaling flexibility and
- * leaving a practically usable system.
- */
-#define EXTRA_MEM_RATIO		(10)
-
 static bool xen_512gb_limit __initdata = IS_ENABLED(CONFIG_XEN_512GB);
 
 static void __init xen_parse_512gb(void)
@@ -96,35 +81,6 @@ static void __init xen_parse_512gb(void)
 	xen_512gb_limit = val;
 }
 
-static void __init xen_add_extra_mem(unsigned long start_pfn,
-				     unsigned long n_pfns)
-{
-	int i;
-
-	/*
-	 * No need to check for zero size, should happen rarely and will only
-	 * write a new entry regarded to be unused due to zero size.
-	 */
-	for (i = 0; i < XEN_EXTRA_MEM_MAX_REGIONS; i++) {
-		/* Add new region. */
-		if (xen_extra_mem[i].n_pfns == 0) {
-			xen_extra_mem[i].start_pfn = start_pfn;
-			xen_extra_mem[i].n_pfns = n_pfns;
-			break;
-		}
-		/* Append to existing region. */
-		if (xen_extra_mem[i].start_pfn + xen_extra_mem[i].n_pfns ==
-		    start_pfn) {
-			xen_extra_mem[i].n_pfns += n_pfns;
-			break;
-		}
-	}
-	if (i == XEN_EXTRA_MEM_MAX_REGIONS)
-		printk(KERN_WARNING "Warning: not enough extra memory regions\n");
-
-	memblock_reserve(PFN_PHYS(start_pfn), PFN_PHYS(n_pfns));
-}
-
 static void __init xen_del_extra_mem(unsigned long start_pfn,
 				     unsigned long n_pfns)
 {
diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
index a87ab36889e7..79cf93f2c92f 100644
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -163,4 +163,18 @@ void xen_hvm_post_suspend(int suspend_cancelled);
 static inline void xen_hvm_post_suspend(int suspend_cancelled) {}
 #endif
 
+/*
+ * The maximum amount of extra memory compared to the base size.  The
+ * main scaling factor is the size of struct page.  At extreme ratios
+ * of base:extra, all the base memory can be filled with page
+ * structures for the extra memory, leaving no space for anything
+ * else.
+ *
+ * 10x seems like a reasonable balance between scaling flexibility and
+ * leaving a practically usable system.
+ */
+#define EXTRA_MEM_RATIO		(10)
+
+void xen_add_extra_mem(unsigned long start_pfn, unsigned long n_pfns);
+
 #endif /* XEN_OPS_H */
diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index 976c6cdf9ee6..aaf2514fcfa4 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -672,7 +672,6 @@ EXPORT_SYMBOL(xen_free_ballooned_pages);
 
 static void __init balloon_add_regions(void)
 {
-#if defined(CONFIG_XEN_PV)
 	unsigned long start_pfn, pages;
 	unsigned long pfn, extra_pfn_end;
 	unsigned int i;
@@ -696,7 +695,6 @@ static void __init balloon_add_regions(void)
 
 		balloon_stats.total_pages += extra_pfn_end - start_pfn;
 	}
-#endif
 }
 
 static int __init balloon_init(void)
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Tue Feb 20 18:00:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 18:00:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683645.1063247 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcUPG-0008PX-1u; Tue, 20 Feb 2024 17:59:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683645.1063247; Tue, 20 Feb 2024 17:59:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcUPF-0008PQ-VZ; Tue, 20 Feb 2024 17:59:49 +0000
Received: by outflank-mailman (input) for mailman id 683645;
 Tue, 20 Feb 2024 17:59:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=bzUj=J5=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1rcUPE-0008PK-WE
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 17:59:49 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d61bbf68-d019-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 18:59:48 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a3e91993d73so365980366b.3
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 09:59:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d61bbf68-d019-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708451987; x=1709056787; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=j5a+NGR/JG/kXWxFoWSe7UwRlrGp+vtqbaGZKpx+uOY=;
        b=HFOGVp75bYG6B6j5hiJZnxrrc1pn8yIoISA9sdpkJYNgJc2FjNHPSkrrVLne2sVPF6
         PVWwueBSMZ5btzZ6GGGMP+LAzF6FU/0LZ0oFM9j53QAInDA5zOpAhk3YlxPC3X6NAah/
         she/Wt94Zavmu0fZu48ehA8D7sZYmID5OQchw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708451987; x=1709056787;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=j5a+NGR/JG/kXWxFoWSe7UwRlrGp+vtqbaGZKpx+uOY=;
        b=NAYApUqU/6lrlcoIcj9/CaC7FeoTTJNwsl9j3tjjnmj1dXk0zYRmLQjIYccLbIlWVf
         mp0ujMpMOFyDztOH9bFcMdUDEhJhuOW56o97SyX3tqNfKNVT7okO1BcWrD/EK/Ghljvq
         /7AzOMBfgepGXonv4fhWBbw/suYUpvvipulUA8p5GtcHY3FPJ6tX5RdWvQogAQEQSdg1
         t0ywqv+3xluAY64MtDQjZWGZwQauHd5G4XTdI+Jmn7waHzxsHquiOudgNHaGFDBgysdQ
         SXjg8QvotdeSY7ru5w4yf0mvLIqFd87Goh+EOxL52kx5ux1NlRDNMSI/VeiKQJK7oVd6
         Fm4A==
X-Gm-Message-State: AOJu0YwZf48JV405MrBY3gp10XrSPXfR+sLIpPKbqF7nhXiyjTIzQI0v
	9Jo5GZ48+EpBwVf5rSRLtcryivaT/Z2lB32bEsL3GFPot60KJTwJqJCuv1ez7snpp9vnOBEenUZ
	OgPt4L4DKp9yk+kyn+cd/4pTxFy1nlJClXTlo+LfWPpwDotqCppZFMg==
X-Google-Smtp-Source: AGHT+IHQSZn5aaQ5eQ/FxjNfqR3eGv+Z5c06xzYKrW0ziAWdcJxsF1IXMqXwFR70eIexjF9BypBbGuU9tOtt07D1pzQ=
X-Received: by 2002:a17:906:c7c4:b0:a3f:1192:723b with SMTP id
 dc4-20020a170906c7c400b00a3f1192723bmr1281268ejb.70.1708451987052; Tue, 20
 Feb 2024 09:59:47 -0800 (PST)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Tue, 20 Feb 2024 17:59:11 +0000
Message-ID: <CAO-mL=wkWuin=yy-TUfs86sD8x55v5wb-ttgB1os_Xo=UYbp=w@mail.gmail.com>
Subject: XEN SOCIAL TOMORROW (21st Feb 2024)!
To: xen-devel@lists.xenproject.org, xen-users@lists.xenproject.org
Content-Type: multipart/alternative; boundary="000000000000cdfb090611d3f942"

--000000000000cdfb090611d3f942
Content-Type: text/plain; charset="UTF-8"

A reminder our Xen Project social is tomorrow!

Connect with the community and your fellow peers - Food and drinks will be
provided.

*If you haven't already, please reply to me directly if you're interested
in attending to anticipate food orders.*

*Location:*
The Portland Arms
129 Chesterton Rd Cambridge
www.theportlandarms.co.uk
<https://www.google.com/url?q=http%3A%2F%2Fwww.theportlandarms.co.uk%2F&sa=D&ust=1708179900000000&usg=AOvVaw39V5Gc5y-9wOaDF__Bwsv5>
T: 01223 357268

*Getting to the venue (more information here
<https://www.google.com/url?q=https%3A%2F%2Ftheportlandarms.co.uk%2Fwp%2Faccess-info%2F&sa=D&ust=1708179900000000&usg=AOvVaw2OZgE_3TLKdcLGYks6qbUW>):*
There are pay and display spaces (free after 5pm) on Milton Road to the
East of us or opposite on Chesterton Road. The nearest bus stops are:
Chesterton Road, opposite the pub outside Hing Hung restaurant, and on
Milton Road, near Westbrook centre, Citi 1, 9, X9 and The Busway. Cambridge
North rail station is 2.3 miles away and Cambridge Central station is 2.4
miles.

Many thanks,
Kelly Choi

Community Manager
Xen Project

--000000000000cdfb090611d3f942
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">A reminder our Xen Project social is tomorrow!=C2=A0<div><=
br><div><div>Connect with the community and your fellow peers - Food and dr=
inks will be provided.<br></div><div><b><font color=3D"#000000"><br></font>=
</b></div><div><b><font color=3D"#000000">If you haven&#39;t already, pleas=
e reply to me directly if you&#39;re interested in attending to=C2=A0antici=
pate=C2=A0food orders.</font></b></div><div><br></div><div><span style=3D"c=
olor:rgb(60,64,67);font-family:Roboto,sans-serif;font-size:14px;letter-spac=
ing:0.2px"><b>Location:</b></span><br style=3D"color:rgb(60,64,67);font-fam=
ily:Roboto,sans-serif;font-size:14px;letter-spacing:0.2px"><span style=3D"c=
olor:rgb(60,64,67);font-family:Roboto,sans-serif;font-size:14px;letter-spac=
ing:0.2px">The Portland Arms</span><br style=3D"color:rgb(60,64,67);font-fa=
mily:Roboto,sans-serif;font-size:14px;letter-spacing:0.2px"><span style=3D"=
color:rgb(60,64,67);font-family:Roboto,sans-serif;font-size:14px;letter-spa=
cing:0.2px">129 Chesterton Rd Cambridge</span><br style=3D"color:rgb(60,64,=
67);font-family:Roboto,sans-serif;font-size:14px;letter-spacing:0.2px"><a h=
ref=3D"https://www.google.com/url?q=3Dhttp%3A%2F%2Fwww.theportlandarms.co.u=
k%2F&amp;sa=3DD&amp;ust=3D1708179900000000&amp;usg=3DAOvVaw39V5Gc5y-9wOaDF_=
_Bwsv5" target=3D"_blank" style=3D"font-family:Roboto,sans-serif;font-size:=
14px;letter-spacing:0.2px">www.theportlandarms.co.uk</a><br style=3D"color:=
rgb(60,64,67);font-family:Roboto,sans-serif;font-size:14px;letter-spacing:0=
.2px"><span style=3D"color:rgb(60,64,67);font-family:Roboto,sans-serif;font=
-size:14px;letter-spacing:0.2px">T: 01223 357268</span><br style=3D"color:r=
gb(60,64,67);font-family:Roboto,sans-serif;font-size:14px;letter-spacing:0.=
2px"><br style=3D"color:rgb(60,64,67);font-family:Roboto,sans-serif;font-si=
ze:14px;letter-spacing:0.2px"><b><span style=3D"color:rgb(60,64,67);font-fa=
mily:Roboto,sans-serif;font-size:14px;letter-spacing:0.2px">Getting to the =
venue (</span><a href=3D"https://www.google.com/url?q=3Dhttps%3A%2F%2Fthepo=
rtlandarms.co.uk%2Fwp%2Faccess-info%2F&amp;sa=3DD&amp;ust=3D170817990000000=
0&amp;usg=3DAOvVaw2OZgE_3TLKdcLGYks6qbUW" target=3D"_blank" style=3D"font-f=
amily:Roboto,sans-serif;font-size:14px;letter-spacing:0.2px">more informati=
on here</a><span style=3D"color:rgb(60,64,67);font-family:Roboto,sans-serif=
;font-size:14px;letter-spacing:0.2px">):</span></b><br style=3D"color:rgb(6=
0,64,67);font-family:Roboto,sans-serif;font-size:14px;letter-spacing:0.2px"=
><span style=3D"color:rgb(60,64,67);font-family:Roboto,sans-serif;font-size=
:14px;letter-spacing:0.2px">There are pay and display spaces (free after 5p=
m) on Milton Road to the East of us or opposite on Chesterton Road. The nea=
rest bus stops are: Chesterton Road, opposite the pub outside Hing Hung res=
taurant, and on Milton Road, near Westbrook centre, Citi 1, 9, X9 and The B=
usway. Cambridge North rail station is 2.3 miles away and Cambridge Central=
 station is 2.4 miles.</span><br></div><div><br></div><div><div dir=3D"ltr"=
 class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=3D"l=
tr"><div>Many thanks,</div><div>Kelly Choi</div><div><br></div><div><div st=
yle=3D"color:rgb(136,136,136)">Community Manager</div><div style=3D"color:r=
gb(136,136,136)">Xen Project=C2=A0<br></div></div></div></div></div></div><=
/div></div>

--000000000000cdfb090611d3f942--


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 18:58:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 18:58:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683711.1063279 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcVJD-0000S9-Hm; Tue, 20 Feb 2024 18:57:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683711.1063279; Tue, 20 Feb 2024 18:57:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcVJD-0000S2-Du; Tue, 20 Feb 2024 18:57:39 +0000
Received: by outflank-mailman (input) for mailman id 683711;
 Tue, 20 Feb 2024 18:57:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nvC9=J5=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rcVJC-0000Rw-Fk
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 18:57:38 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ea16af28-d021-11ee-8a52-1f161083a0e0;
 Tue, 20 Feb 2024 19:57:37 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-5129e5b8cecso4777371e87.3
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 10:57:37 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 ep14-20020a056512484e00b00511740efdd1sm1359588lfb.39.2024.02.20.10.57.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 10:57:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea16af28-d021-11ee-8a52-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708455457; x=1709060257; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=HWDibD0Fx2KzEZtfYZhxa0gkumK9CdCgWsdGtaVvLts=;
        b=ZmN10KgvnMrrbdSurRo/Jx8Ic6VKyJKX7wl14U5SJ0DQYd+m3EST2HvFW9FeVtB9oo
         WBby/Az1kSrH4/ysSYcbRc2WHanUBnHXyIiw+5Q4bTnTs5pAi3MKxlPRmdYejpGKo1vX
         6JT5w30C4UXMx9pQlQXRJ9pLG90OcC1Ux7f0ziPfbviFNGW/74F6ZpmvMXIVRz7I8GGk
         3nr7RhJHGspQJ8LhPEZed2Git/eN12pazMrK+anKPjwKJQ+BnrA8o4BgRMgLyYEkvCzA
         rcILmrPjTNB3C8cagrH9eHCnRGbnRO4l7QImU44aesFeo8BmRD+vD9N1lmf/vYAKBX2O
         CS+A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708455457; x=1709060257;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HWDibD0Fx2KzEZtfYZhxa0gkumK9CdCgWsdGtaVvLts=;
        b=g7TZ9Su1gruhsPPnOSDUDv1YTOZRET9ox1s+l2CjxeMkWpm1ugi6RktE+ZzuCOtlP2
         RQNP7+6QFtoskgofTUsvZ86xWf+Kprx4U77vGUcp7bZ598zl9GVjLFlvhUlecOFrUnsM
         KXwGqus+UNqQGDt9+kc2A+LQ8mBLdl4VmzdYmfes2qPwanDCRevA+F3VN8x4+Ww07R04
         r3t+mVH9lnvKJ+k7F75BwMBpckLx69xkqFubCZhK5ioGzxvCsQ5rpoMPz1pxdnknTygr
         b8yKtD2rje1qa99hKKhBb1udfaxyzEdpMfoVZfJrpm7h8qWyTEUphWnCWybjAlYn1D6j
         2Sbg==
X-Forwarded-Encrypted: i=1; AJvYcCUTxRXido6gpA0cYQAcNqVR7U8zLNJNZOlM8Wl5zkeeBF2uihHsv9qZ4x+4fvF6pbTS4hkSu09nwabSBnQenf2b5r52nfWnfNtIIk9Dv7E=
X-Gm-Message-State: AOJu0YyV37n8FaOJE2raYtLCrzi1TucBE9cb4WpIBCVgRTkMwLgbqwJP
	8y2m8Or7Y+ZIO8y3qNPKk41SU1fnher2+N1zgwILN8beBKdgU0hY
X-Google-Smtp-Source: AGHT+IGwH/Ldxke2P1hxqOEqYmORolFTk35OPOUwEzazEcurI1VcyKvFnRtxyTQHSfdW8VUPKxo+5g==
X-Received: by 2002:a05:6512:1024:b0:511:87b7:6d88 with SMTP id r4-20020a056512102400b0051187b76d88mr10469147lfr.32.1708455456771;
        Tue, 20 Feb 2024 10:57:36 -0800 (PST)
Message-ID: <35641c47eda255e74cf79a66d2e3d04020be261f.camel@gmail.com>
Subject: Re: [PATCH v4 02/30] xen/riscv: use some asm-generic headers
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Tue, 20 Feb 2024 19:57:35 +0100
In-Reply-To: <5b8d6ff3-508c-49aa-8b25-3624d165eae1@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <a721f0c092306b589fae5f44bdaafcd94c60ed14.1707146506.git.oleksii.kurochko@gmail.com>
	 <16baca98-44fe-42ba-b61d-ff1945e0d2b5@suse.com>
	 <d2cc798462de9e0d7a89777ea0743fe6dce4d061.camel@gmail.com>
	 <5b8d6ff3-508c-49aa-8b25-3624d165eae1@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Wed, 2024-02-14 at 11:03 +0100, Jan Beulich wrote:
> On 14.02.2024 10:54, Oleksii wrote:
> > On Mon, 2024-02-12 at 16:03 +0100, Jan Beulich wrote:
> > > On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > > > =C2=A0As [PATCH v6 0/9] Introduce generic headers
> > > > =C2=A0(
> > > > https://lore.kernel.org/xen-devel/cover.1703072575.git.oleksii.kuro=
chko@gmail.com
> > > > /)
> > > > =C2=A0is not stable, the list in asm/Makefile can be changed, but
> > > > the
> > > > changes will
> > > > =C2=A0be easy.
> > >=20
> > > Or wait - doesn't this mean the change here can't be committed
> > > yet? I
> > > know the cover letter specifies dependencies, yet I think we need
> > > to
> > > come
> > > to a point where this large series won't need re-posting again
> > > and
> > > again.
> > We can't committed it now because asm-generic version of device.h,
> > which is not commited yet.
> >=20
> > We can drop the change " generic-y +=3D device.h ", and commit the
> > current one patch, but it sill will require to create a new patch
> > for
> > using of asm-generic/device.h. Or as an option, I can merge
> > "generic-y
> > +=3D device.h" into PATCH 29/30 xen/riscv: enable full Xen build.
> >=20
> > I don't expect that the of asm-generic headers will changed in
> > riscv/include/asm/Makefile, but it looks to me that it is better to
> > wait until asm-generic/device.h will be in staging branch.
> >=20
> >=20
> > If you have better ideas, please share it with me.
>=20
> My main point was that the interdependencies here have grown too far,
> imo. The more that while having dependencies stated in the cover
> letter
> is useful, while committing (and also reviewing) I for one would
> typically only look at the individual patches.
>=20
> For this patch alone, maybe it would be more obvious that said
> dependency exists if it was last on the asm-generic series, rather
> than part of the series here (which depends on that other series
> anyway). That series now looks to be making some progress, and it
> being
> a prereq for here it may be prudent to focus on getting that one in,
> before re-posting here.
I'll be more specific next time regarding dependencies and specify what
a prereq changes are.

Considering that asm-generic/device.h was merged to staging branch. It
seems to me that there are no more additional prereqs for this patch.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 19:28:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 19:28:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683720.1063288 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcVnD-0004gy-NW; Tue, 20 Feb 2024 19:28:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683720.1063288; Tue, 20 Feb 2024 19:28:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcVnD-0004gr-KX; Tue, 20 Feb 2024 19:28:39 +0000
Received: by outflank-mailman (input) for mailman id 683720;
 Tue, 20 Feb 2024 19:28:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcVnC-0004gh-Ep; Tue, 20 Feb 2024 19:28:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcVnC-0000y0-8v; Tue, 20 Feb 2024 19:28:38 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcVnC-0004RV-0s; Tue, 20 Feb 2024 19:28:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcVnB-0001AF-UW; Tue, 20 Feb 2024 19:28:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=wVtYBgi8cYClyI8mGDqFIO1VuyBWZY3g2QhM3mLtgA4=; b=S+nA2PHwBM2mQku+xDsUC+tO8L
	Ot+o86e3gyrAhrETqSX76oPFDVvQeQVjLH/Re0vHGTxFHVvjQma+iPy6hXSB57jlfkEeptcqkNaVW
	ZfW39OM1LSsWUj8FWK3HtCY4aDZYrv9bSSIIHxClnm00QYoS9ZmQ1PcFXz7hdxTVDngg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184713-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184713: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f8791d0fd3adbda3701e7eb9db63a9351b478365
X-Osstest-Versions-That:
    xen=981f722efa876e3f1766629f444d10e8806adc93
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 20 Feb 2024 19:28:37 +0000

flight 184713 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184713/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  f8791d0fd3adbda3701e7eb9db63a9351b478365
baseline version:
 xen                  981f722efa876e3f1766629f444d10e8806adc93

Last test of basis   184711  2024-02-20 12:02:07 Z    0 days
Testing same since   184713  2024-02-20 15:00:29 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Julien Grall <jgrall@amazon.com>
  Luca Fancellu <luca.fancellu@arm.com>
  Michal Orzel <michal.orzel@amd.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   981f722efa..f8791d0fd3  f8791d0fd3adbda3701e7eb9db63a9351b478365 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 20:30:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 20:30:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683728.1063299 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcWl2-0004Kk-7I; Tue, 20 Feb 2024 20:30:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683728.1063299; Tue, 20 Feb 2024 20:30:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcWl2-0004Kd-2H; Tue, 20 Feb 2024 20:30:28 +0000
Received: by outflank-mailman (input) for mailman id 683728;
 Tue, 20 Feb 2024 20:30:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nvC9=J5=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rcWl1-0004KX-08
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 20:30:27 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e0485907-d02e-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 21:30:24 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-5129e5b8cecso4898286e87.3
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 12:30:24 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 y2-20020a199142000000b00511570bf1desm1450031lfj.240.2024.02.20.12.30.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 20 Feb 2024 12:30:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0485907-d02e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708461024; x=1709065824; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Lbwkuduf+qdvroYy9Nltc2Q0VqNnF6JVyR/T8bbpGHo=;
        b=IMl7wcS4dxV2939dNJwJOrmOvoJvcDW8nBpTErKtG6FyR9zKXN2dmenZMOqAvWrbRD
         ao8kTMLymBte7sZV6IJUl45ujrsjrTBzIkVfX7HJorKXYWaohXhT3c0bjsdMRNPRRo04
         UQvm5klz7BAVDj8uqIHDZQJX/7bNldo+poFMicyE16X8mXWBpbvnuKQo/1XTh8h1jIBR
         iboncJX2G4piNAE6K+EVBH0qKRKZGC2iYtrF7zpDYnyaKzc/c/91SHDslK0MJLqc8k2K
         jFYRLEOZ8vpssL06w1M1S6Z48BGP2QQcnQSOkek+raJ1mUwHYjkN8N0QNwl+VxB3ebGr
         Hajg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708461024; x=1709065824;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Lbwkuduf+qdvroYy9Nltc2Q0VqNnF6JVyR/T8bbpGHo=;
        b=CJeohzYOCudWkdPUwiCRpjt3RyisSGl33a2u3C/B5hGsBQx9ZGqwmYG0oRzXx0cYFa
         i0kia1VbbQ4SuzPv6aGEDv2I+jRD/L76W8ro0AViuYCqZXZSryEEo7m8yQ7ZA4h0Thcx
         rNCcYR1YvtifnhHQnRFsStCIfXvRGFV1IijcR8KRX0MxOj8P9yeDpbLByxRTqaFitVb3
         QsHlXluvfyWBNGWM9gZBkVBLa2ndFwLnsrPoW8zJROmijRG22Vz4IagYRB/i6An7ZLvS
         AZI6B7pjQHO5PQph4M3jdkoubbCJxwbFKjWqocidVPINquy8ugi5jTO4JHcaLvAp+a9w
         RBAQ==
X-Forwarded-Encrypted: i=1; AJvYcCVjChxPZ9T4pogzjwNlPKrrjzpaSF4l6qChISqS9Fqjo2Fme2Nv1L5IawadpUlYsZUueZ5nPLB1M5IUfMV3tAjQRJ9rovy8zMS5GaiWsGs=
X-Gm-Message-State: AOJu0Yx2HKGHh3uYdmrtWOOj8T58pZqSNSNtkwTjQJLZC449FXJvVc9k
	e4Zwo8/FdL+hN482gQIpWUPVUirGYLQx9YDploMqTMaSS9cTdgHv
X-Google-Smtp-Source: AGHT+IG42Y7mHPXbC+/kWmdTEpgcWa/OVZJ2SqxHd1CNTvHiBtxAw9LRQu2P+poXm5oKVU6q58F3Jg==
X-Received: by 2002:ac2:4a68:0:b0:512:a37a:bde2 with SMTP id q8-20020ac24a68000000b00512a37abde2mr6506385lfp.67.1708461023724;
        Tue, 20 Feb 2024 12:30:23 -0800 (PST)
Message-ID: <ab040f3c8ee35f49fffac385053b55053c52da8c.camel@gmail.com>
Subject: Re: [PATCH v4 07/30] xen/asm-generic: introdure nospec.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Wei
 Liu <wl@xen.org>, Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,  Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Julien
 Grall <julien@xen.org>,  xen-devel@lists.xenproject.org
Date: Tue, 20 Feb 2024 21:30:22 +0100
In-Reply-To: <aaf86d31-7ae0-4e33-8386-dda4bd21496a@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <5889d7a5fa81722472f95cc1448af0be8f359a7d.1707146506.git.oleksii.kurochko@gmail.com>
	 <510d74ca-5cce-48f8-93f6-407cd727cf7b@xen.org>
	 <8a09e386b709f780f193af39af63b6aeb75c868e.camel@gmail.com>
	 <aaf86d31-7ae0-4e33-8386-dda4bd21496a@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-19 at 13:18 +0100, Jan Beulich wrote:
> On 19.02.2024 12:59, Oleksii wrote:
> > Hi Julien,
> >=20
> > On Sun, 2024-02-18 at 18:30 +0000, Julien Grall wrote:
> > > Hi Oleksii,
> > >=20
> > > Title: Typo s/introdure/introduce/
> > >=20
> > > On 05/02/2024 15:32, Oleksii Kurochko wrote:
> > > > The <asm/nospec.h> header is similar between Arm, PPC, and
> > > > RISC-V,
> > > > so it has been moved to asm-generic.
> > >=20
> > > I am not 100% convinced that moving this header to asm-generic is
> > > a
> > > good=20
> > > idea. At least for Arm, those helpers ought to be non-empty, what
> > > about=20
> > > RISC-V?
> > For Arm, they are not taking any action, are they? There are no
> > specific fences or other mechanisms inside
> > evaluate_nospec()/block_speculation() to address speculation.
>=20
> The question isn't the status quo, but how things should be looking
> like
> if everything was in place that's (in principle) needed.
>=20
> > For RISC-V, it can be implemented in a similar manner, at least for
> > now. Since these functions are only used in the grant tables code (
> > for
> > Arm and so for RISC-V ), which is not supported by RISC-V.
>=20
> Same here - the question is whether long term, when gnttab is also
> supported, RISC-V would get away without doing anything. Still ...
>=20
> > > If the answer is they should be non-empty. Then I would consider
> > > to
> > > keep=20
> > > the duplication to make clear that each architecture should take
> > > their=20
> > > own decision in term of security.
> > >=20
> > > The alternative, is to have a generic implementation that is safe
> > > by=20
> > > default (if that's even possible).
> > I am not certain that we can have a generic implementation, as each
> > architecture may have specific speculation issues.
>=20
> ... it's theoretically possible that there'd be an arch with no
> speculation issues, maybe simply because of not speculating.

I am not sure that understand your and Julien point.

For example, modern CPU uses speculative execution to reduce the cost
of conditional branch instructions using schemes that predict the
execution path of a program based on the history of branch executions.

Arm CPUs are vulnerable for speculative execution, but if to look at
the code of evaluate_nospec()/block_speculation() functions they are
doing nothing for Arm.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 21:56:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 21:56:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683735.1063308 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcY65-0004nA-Nk; Tue, 20 Feb 2024 21:56:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683735.1063308; Tue, 20 Feb 2024 21:56:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcY65-0004n3-L0; Tue, 20 Feb 2024 21:56:17 +0000
Received: by outflank-mailman (input) for mailman id 683735;
 Tue, 20 Feb 2024 21:56:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=C80L=J5=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rcY63-0004mh-Ue
 for xen-devel@lists.xenproject.org; Tue, 20 Feb 2024 21:56:16 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dc58171a-d03a-11ee-98f5-efadbce2ee36;
 Tue, 20 Feb 2024 22:56:12 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 7BCAE61210;
 Tue, 20 Feb 2024 21:56:11 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1F91FC433C7;
 Tue, 20 Feb 2024 21:56:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc58171a-d03a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708466171;
	bh=SxhF36u+/ccAB7bpf8YCfKxFTOxe0Qb+HUoXZTCXu1E=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=ItVhe2bQ3ZOghNzHV/HmNqPSIXfQgbn+70Z+DRnafE5akK94FNh+waKObFT/+WKet
	 f91lTV8XfvZ7bgZIjHhsC3HZh6EixmK4E34ABJs5AV49SHsQ64p+1JyjiRsEE29bog
	 dZW9PDDeAfxUubFTJgv9alF4jhmZO/ic0AqrAvOMllwoqmOvQn+ipzSAaeAjxjBh18
	 jNTl8+WJhDyumZxSejXwqGs4VH3ei+Xub0iHMsNlU4IPWywieclfuhjALuvD7vAzZR
	 VyhFvRIxhPD5Cf+WUlI8+RuyU8RrO+Z47IOAEi2krDF+cSWzvK9OQRjujO9qcfNWIq
	 496Ta0eJQCMWg==
Date: Tue, 20 Feb 2024 13:56:08 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Anthony PERARD <anthony.perard@citrix.com>
cc: xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Michal Orzel <michal.orzel@amd.com>, 
    Roger Pau Monne <roger.pau@citrix.com>
Subject: Re: [OSSTEST PATCH] ap-common: Switch to Linux 6.1 by default on
 Arm
In-Reply-To: <20240220145352.26264-1-anthony.perard@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2402201355410.754277@ubuntu-linux-20-04-desktop>
References: <20240220145352.26264-1-anthony.perard@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 20 Feb 2024, Anthony PERARD wrote:
> The "linux-arm-xen" tree currently point to Linux 5.4.17 + 3 commits:
>   69117aaf07f7 ("xen/privcmd: replace kcalloc() by kvcalloc() when allocating empty pages")
>   bca5bc0229a6 ("xen/privcmd: fix error handling in mmap-resource processing")
>   f0f0e602f7c9 ("xen/privcmd: drop "pages" parameter from xen_remap_pfn()")
> 
> This stable version is old and doesn't build on Debian Bookworm.
> 
> The 3 extra commits are present in at least Linux 5.15, so there's
> currently no reason to have a separate tree when we could use a stable
> tree that osstest can update regulary.
> 
> So switch to 6.1. This will change the default Linux tree for all Arm
> jobs that aren't on "linux-*" osstest branches.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

I think that's fine. Assuming that you tested it and it works with
OSSTest as expected:

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> 
> Notes:
>     I plan to also change to use Linux 6.1 on X86 a bit later.
> 
>  ap-common | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/ap-common b/ap-common
> index 292553ed..48a95a22 100644
> --- a/ap-common
> +++ b/ap-common
> @@ -60,7 +60,7 @@
>  : ${PUSH_TREE_LINUX:=$XENBITS:/home/xen/git/linux-pvops.git}
>  : ${BASE_TREE_LINUX:=git://xenbits.xen.org/linux-pvops.git}
>  : ${BASE_TAG_LINUX:=tested/linux-4.19}
> -: ${BASE_TAG_LINUX_ARM:=tested/linux-arm-xen}
> +: ${BASE_TAG_LINUX_ARM:=tested/linux-6.1}
>  
>  if [ "x${TREE_LINUX}" = x ]; then
>  	: ${TREE_LINUX:=${BASE_TREE_LINUX}}
> -- 
> Anthony PERARD
> 


From xen-devel-bounces@lists.xenproject.org Tue Feb 20 23:19:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 20 Feb 2024 23:19:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683744.1063319 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcZOh-0005T4-O9; Tue, 20 Feb 2024 23:19:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683744.1063319; Tue, 20 Feb 2024 23:19:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcZOh-0005Sx-Kz; Tue, 20 Feb 2024 23:19:35 +0000
Received: by outflank-mailman (input) for mailman id 683744;
 Tue, 20 Feb 2024 23:19:35 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcZOg-0005Sn-VK; Tue, 20 Feb 2024 23:19:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcZOg-0004ol-Oj; Tue, 20 Feb 2024 23:19:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcZOg-00040I-Bw; Tue, 20 Feb 2024 23:19:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcZOg-0002nk-BG; Tue, 20 Feb 2024 23:19:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=eP7+5EmuEVH9/f9PowcTkKkaYmu3Hxu1H3OS8oauYoQ=; b=p8luzTUdpxR3XybUnfjToxmy5q
	5cDOp83da/S+LUjcnJRs7sV20R0yH9XTZKIsI3ftSFebdZTFNvRjFuk8UuNrlC1bOrZriqoF23wiB
	5Jb61x7x/70K1Qvs2KlzylUguiSuk8lEDUyoOP/FwVFCj7t1Hx1YEO2JAVne9myizZ7Y=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184712-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184712: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt-qcow2:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=981f722efa876e3f1766629f444d10e8806adc93
X-Osstest-Versions-That:
    xen=c144b9e32427ba37e0e0443a0d3fa53e9fb878b6
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 20 Feb 2024 23:19:34 +0000

flight 184712 xen-unstable real [real]
flight 184715 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184712/
http://logs.test-lab.xenproject.org/osstest/logs/184715/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-qcow2 17 guest-start/debian.repeat fail pass in 184715-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184705
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184709
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184709
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184709
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184709
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184709
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184709
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184709
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184709
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184709
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184709
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184709
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  981f722efa876e3f1766629f444d10e8806adc93
baseline version:
 xen                  c144b9e32427ba37e0e0443a0d3fa53e9fb878b6

Last test of basis   184709  2024-02-20 05:34:24 Z    0 days
Testing same since   184712  2024-02-20 14:39:53 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               fail    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   c144b9e324..981f722efa  981f722efa876e3f1766629f444d10e8806adc93 -> master


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 02:16:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 02:16:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683755.1063340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcc9H-0006SW-PD; Wed, 21 Feb 2024 02:15:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683755.1063340; Wed, 21 Feb 2024 02:15:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcc9H-0006SP-MW; Wed, 21 Feb 2024 02:15:51 +0000
Received: by outflank-mailman (input) for mailman id 683755;
 Wed, 21 Feb 2024 02:15:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rcc9G-0006SJ-VO
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 02:15:51 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 215c38f0-d05f-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 03:15:49 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d23a5a7c18so31913871fa.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 18:15:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 215c38f0-d05f-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708481749; x=1709086549; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=mQF1qTpurz95qEjIjguEtPOhesntQW0/pAl5GKuAv2k=;
        b=BtTLnp6VsGICeU+0rFbRPV8R+gRhmkgpFtypAMJkAJhwzQG9TtFWWd7LZ+WJcGBD1C
         RCzliYNQnp9OT6/tk2PEeWbT3B5uN7ctrRP/BhpADrPLn2ekgdF4rf7kjJMawozFZn83
         yoECIpsOlqkQfbPSa4LZJuD4mKiAMs5CPT7zc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708481749; x=1709086549;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=mQF1qTpurz95qEjIjguEtPOhesntQW0/pAl5GKuAv2k=;
        b=bvqIvRFKygPad1ooikAhSGtQ2KlFl9KE5IHN/w/PHdK/2cJn/f3lqewvhATx3jBzEI
         D7CUxXNuTF22nWYZS63Gbmmobv/XXHy2dCMs7Vlj5f1IO2eOCasezkaaa9YIreDbJLcg
         FrWtp816jqvvlUoVSeHh83R0PF1x6XfoeVNyRQaAkJVPhyUzHAPg7bHU/2ZcvblgkwD1
         w9Pc/7LDjKZ4A9XOkHwpfrDGhHmSQdzbnwdPLmcU4s7fiZls1qWMVXNFicUAlI7sjONl
         JLyQSJB6egF+6ySvqGMyk20fCAAbpuE2eWRNMRIrGblWvHlNZ6Dy8PpHfAEYgIxuhA9E
         nSkA==
X-Gm-Message-State: AOJu0Yx6TvmDo5wQjXaxqC9kM3xQyilmE1h8q/YPhajef55stCeF8BPo
	7rVIy0LA05wM1h6vcxaLRDt+ynomU2PUYSH6yCnU8M6wcIo/O7Ol9ClBBt5kCwSKIrQHVKt1Nom
	BeqjiguyBdZJREPi0IrGFZOdbYzKsa7b4X3JW2w==
X-Google-Smtp-Source: AGHT+IHEqS+kxul4evQoa7DKb0efNc+gF3WvmJ/sN39vtA4IA2glOXVr3TQ5fT2S3I+0Q1Pp2loQvzXZASk6fYDqWQk=
X-Received: by 2002:a2e:a272:0:b0:2d2:4388:63fc with SMTP id
 k18-20020a2ea272000000b002d2438863fcmr3130404ljm.44.1708481749060; Tue, 20
 Feb 2024 18:15:49 -0800 (PST)
MIME-Version: 1.0
References: <20240214094734.13533-1-frediano.ziglio@cloud.com>
In-Reply-To: <20240214094734.13533-1-frediano.ziglio@cloud.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 10:15:37 +0800
Message-ID: <CA+zSX=Y113S6zJqGExRw+bWSxPZSNmz9zY+yC=-yC0LLxoWXtQ@mail.gmail.com>
Subject: Re: [PATCH v2] Constify some parameters
To: Frediano Ziglio <frediano.ziglio@cloud.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
	Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, Dario Faggioli <dfaggioli@suse.com>, 
	"Daniel P. Smith" <dpsmith@apertussolutions.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 14, 2024 at 5:48=E2=80=AFPM Frediano Ziglio
<frediano.ziglio@cloud.com> wrote:
>
> Make clear they are not changed in the functions.
>
> Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Sched bits:

Acked-by: George Dunlap <george.dunlap@cloud.com>


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 02:32:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 02:32:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683761.1063351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rccPY-0000aU-5k; Wed, 21 Feb 2024 02:32:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683761.1063351; Wed, 21 Feb 2024 02:32:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rccPY-0000aN-1q; Wed, 21 Feb 2024 02:32:40 +0000
Received: by outflank-mailman (input) for mailman id 683761;
 Wed, 21 Feb 2024 02:32:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rccPW-0000aH-Rn
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 02:32:38 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7a5dd93f-d061-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 03:32:37 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d244b28b95so21807491fa.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 18:32:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a5dd93f-d061-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708482757; x=1709087557; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=sNsSMd/avjPHTwuybE9n0Q/zH0doxEeoNutnqoCLcTo=;
        b=aAFFaWgFZBAC5J34u+xIPefc/vMazhLhM9wyDE3rkhjcczuzOfrVt2fQw5iLzfEKYU
         LBMLi2VdxF/8hHOwks5qJIbbnzAp2HpDZGMM8rSKeA168338mHhmr7gSoXrs3grJdGiY
         2iZBRnjudRJZOc9jWPijDxnQ5OImXURFYXy3I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708482757; x=1709087557;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=sNsSMd/avjPHTwuybE9n0Q/zH0doxEeoNutnqoCLcTo=;
        b=eJLTPV3buM3rvrh/7Ia7f1OgQNvDMhO9VmbHc1RaQLZvQYt5Wy/59dGcpmYXtrprmx
         IgmtPzB6nGdPLDqIrF6H8KnXSchM0z5LX9PuXqVJgpFZen7gW3WCnHexR1irB2oqKPeD
         w6ZgP+uOiQUMHR4Jufa3IGVUbAjfw6mAwBh4KQWN9Rg+xY3gieNCM+SzflmdMJAL3mLh
         S00Py6X61L3iM6YUwHhLwub9U72TogBt8YHQDWFD62nYoDlLX3RB8KMKQwa03EQtTI63
         7eSkqljPXlZp3OqzMSFthzsxmAfK2hxyL1FdXeH7ySEHjzhAEE3Pp4FuDqYNe00Yz2yO
         i2fg==
X-Forwarded-Encrypted: i=1; AJvYcCWeGnxhsgnJwZ3Zd0T9FN1tu6jvIwHCO4+H9g8rrMPltzIfWVqaZ7FEAwSy4JM8Q2OeG9a4+VNvITuLgJphmavAkI08zG3DrMh/2FOBBxY=
X-Gm-Message-State: AOJu0YzgCCrI6sSi2ywUisvMzSRZm4bpy7xg8vD6wIUlUU/0MdjUiBwY
	3f0oiDnuDMicekbs3soXRuUhEpzWT82zzjEJ40rJPjetZB1N3GWY3ZOZfzRLUTHyccSGr7rb4P2
	m+dS2BMs4LdfJ/MQlzOI99iw93zZAEdlXCW75Aw==
X-Google-Smtp-Source: AGHT+IHhu572k5XuC3mikpoqZkrtWjcC8JZaV3R3axhCoNIDzQZCITo8FkH4vDabUK/kVW3eV+yy95zbjO5Rm8nVs1A=
X-Received: by 2002:a2e:a585:0:b0:2d2:4ccb:7449 with SMTP id
 m5-20020a2ea585000000b002d24ccb7449mr2488937ljp.44.1708482757302; Tue, 20 Feb
 2024 18:32:37 -0800 (PST)
MIME-Version: 1.0
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
 <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com> <7dd78c97-b82f-4ae1-a257-98f612c6bad2@xen.org>
 <6466155e-9883-4743-a897-ef565f14fdf0@suse.com>
In-Reply-To: <6466155e-9883-4743-a897-ef565f14fdf0@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 10:32:26 +0800
Message-ID: <CA+zSX=aJ2mVbGzFj74+DEGZvw=XWTZKAeWfrwMNHaYhsdaytHA@mail.gmail.com>
Subject: Re: [PATCH 2/2] almost fully ignore zero-size flush requests
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 20, 2024 at 4:26=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
> >> +    if ( (cflush->offset | cflush->length) > PAGE_SIZE ||
> >
> > This is confusing. I understand you are trying to force the compiler to
> > optimize. But is it really worth it? After all, the rest of operation
> > will outweight this check (cache flush are quite expensive).
>
> From purely a performance point of view it may not be worth it. From
> code size angle (taken globally) I already view this differently.
> Plus I think that we ought to aim at avoiding undesirable patterns,
> just because people tend to clone existing code when they can. Thing
> is that (as per below) the two of us apparently disagree on what
> "undesirable" is in cases like this one.
>
> > We probably should take a more generic decision (and encode in our
> > policy) because you seem to like this pattern and I dislike it :). Not
> > sure what the others think.

This is similar to the policy question I raised among the x86
committers a few weeks ago: You're manually specifying a more specific
behavior than is required, rather than specifying what you want and
then letting the compiler optimize things.  The problem with this is
twofold:

1. It's harder for humans to read and understand the intent

2. It ties the compiler's hands.  If you write your intent, then the
compiler is free to apply the optimization or not, or apply a
different optimization.  If you specify this optimization, then the
compiler has fewer ways that it's allowed to compile the code.

#1 by itself is probably enough to counterindicate this kind of
behavior.  Add them together, and I'm inclined to say that we should
write a policy against such optimizations, without specific
justifications.

 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 02:46:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 02:46:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683768.1063360 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcccs-0002LR-DS; Wed, 21 Feb 2024 02:46:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683768.1063360; Wed, 21 Feb 2024 02:46:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcccs-0002LK-Ar; Wed, 21 Feb 2024 02:46:26 +0000
Received: by outflank-mailman (input) for mailman id 683768;
 Wed, 21 Feb 2024 02:46:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HakF=J6=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rcccq-0002LE-DD
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 02:46:24 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 63a0d3ec-d063-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 03:46:19 +0100 (CET)
Received: from CYXPR02CA0075.namprd02.prod.outlook.com (2603:10b6:930:ce::28)
 by BL3PR12MB6618.namprd12.prod.outlook.com (2603:10b6:208:38d::19)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.19; Wed, 21 Feb
 2024 02:46:15 +0000
Received: from CY4PEPF0000EE38.namprd03.prod.outlook.com
 (2603:10b6:930:ce:cafe::77) by CYXPR02CA0075.outlook.office365.com
 (2603:10b6:930:ce::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.40 via Frontend
 Transport; Wed, 21 Feb 2024 02:46:15 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000EE38.mail.protection.outlook.com (10.167.242.12) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7249.19 via Frontend Transport; Wed, 21 Feb 2024 02:46:14 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 20 Feb
 2024 20:46:14 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 20 Feb 2024 20:46:12 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 63a0d3ec-d063-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ybu739SSI5GM8p+sGlwapppA1VKASNAwpRjNk8rhfs4rc8V94rbicqsVymRbi//tjzMjyDhvnTEuzwhWhi+ZUugLQOzb0x1/OwE+m5BLch5QOe6wD8WSxAvc8VNzSuUs6S/T9SaSEkHFOef1T2MxWFcxNUqoOH0YfJxmkjaA9K9xip2pmKn7UoZLdauV5M7L0EatxTEm+1V7hWbITLlIWskDxCya9qhqCNua7uBELJ+W3QnL/KLU6shBNJxAPk/c53g1vGO8dmfkL/8Bb0b3R296rJQOzlrJi5iFnWnwpM4xOgSuBPiNafcnNTFR7ZbBBbJB/Qi75TRzAEALinsBIg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=QFou7oGcuxDYVvS3pJtjBxOeAm+1ypFnd2a4gCrmbbM=;
 b=YOyX5Hj3TxSvNi1d/MZY2wqtq5DzJGLP+Dg39P9duE7VTej5skj5NDrRRCxICfGOPzucCIWVx16CX+IploFh2CwivYDUsxKOAPr/ERo/3u9N8kqPFWQ+pLH7sjlFx89acsY49OAT3d4B34Xcu0bqBB3CNDPEoXLgDHx4EX4MVCEFQZc/mfUDz7yyBZAGTSreOfbFbx7nAq4Dgxfihv9xKcf/v0iR3HyF/DXIHb3QBUg0sBc3OvpUcRqlBZWVECpf62XBExdtFnCecROQ/SflcE8uBJAOBo9rwe8BLNNqDWjTy/uGel6wj/ZBO55dsirp3E295c9j1gCW10Ud2AZnSA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=QFou7oGcuxDYVvS3pJtjBxOeAm+1ypFnd2a4gCrmbbM=;
 b=TdaU2su8UeLqphfmGfb8VwHdX42a5wr6692Uux/b6q3bUlH6Y3lvCbOS/R5h1Reb5la8JYtbeeB0hLO/9BYcDFBchYV6jTlLXH3OkVeaUFjDAE/K+1YcnFNmp859UMgrNXgEI5md1qDYRYIlriGuX0uEO3LDteKeOAtfREzEZ28=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>, Jan Beulich
	<jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Jun Nakajima
	<jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>, Paul Durrant
	<paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>, "Stewart
 Hildebrand" <stewart.hildebrand@amd.com>
Subject: [PATCH v13.3 01/14] vpci: use per-domain PCI lock to protect vpci structure
Date: Tue, 20 Feb 2024 21:45:03 -0500
Message-ID: <20240221024504.357840-1-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.2
In-Reply-To: <20240202213321.1920347-2-stewart.hildebrand@amd.com>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com> <20240202213321.1920347-2-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE38:EE_|BL3PR12MB6618:EE_
X-MS-Office365-Filtering-Correlation-Id: 24930d24-c5e9-4097-aef8-08dc3287458c
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	a+kgcFX6iK9VqqeZioH8J6pNampuX52E8DVpe01iXW3piXPIKXV60zvDyeHqUWGE0RunBzzzfG9j2TF8wQ23dAHJz+9IpWaEgq6UlpeFBMAFivfFSDQ5Yk4hoSVOQALNNZKF91ovQnrWcRJj1Bc5YFFZsUHmBMCCJSDvbWxZyyFbv0WgquuYBJUdnYO2a1OjBG0cS37pdcBpqwA4MMVn098DwrqeCse0DASUkV1RTK63pkTfGgT6yzpNFmhILpUPa+PP7cbEpxP/SieK3d1QkMqH/aXsLklIxoECiVbtZ6Ia/POyyUZ251igHWG2TyGFtVsfqy40OZ7Fu/2L60SN9X3YGR1ZEtLol7w+ZFMYTrJESLghZjcW514NUuvaLaGm/sEcNZ1BvhZy9GLo32dVjBSdfEVTQULg3dQjiEY40zDh7QpwZqDQCBNJHE3XG39ina9g71HSWzngHBTpshg8sI0sN0iD1fvBnpCfaTJpB0kcIQM08k7yTTQELQBSOYrm3FfTKimkxLqTKFHfSdL/1tgQVEveS5kajVfXhrGe06n76eZqBT9NdYpwJAR0gJ1WPqDPOI7QAiuyFZbxy2ULwdeWoJzDaYhruCTMsuTjM2D0Wtxi2epwmW74b+3Ol9BscQ1DUiHEf/fwmGt0kIVtbpp5rUZyRPbYU57BbbGnCyw=
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(40470700004)(46966006);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2024 02:46:14.7495
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 24930d24-c5e9-4097-aef8-08dc3287458c
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000EE38.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6618

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Use the per-domain PCI read/write lock to protect the presence of the
pci device vpci field. This lock can be used (and in a few cases is used
right away) so that vpci removal can be performed while holding the lock
in write mode. Previously such removal could race with vpci_read for
example.

When taking both d->pci_lock and pdev->vpci->lock, they should be
taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
possible deadlock situations.

1. Per-domain's pci_lock is used to protect pdev->vpci structure
from being removed.

2. Writing the command register and ROM BAR register may trigger
modify_bars to run, which in turn may access multiple pdevs while
checking for the existing BAR's overlap. The overlapping check, if
done under the read lock, requires vpci->lock to be acquired on both
devices being compared, which may produce a deadlock. It is not
possible to upgrade read lock to write lock in such a case. So, in
order to prevent the deadlock, use d->pci_lock in write mode instead.

All other code, which doesn't lead to pdev->vpci destruction and does
not access multiple pdevs at the same time, can still use a
combination of the read lock and pdev->vpci->lock.

3. Drop const qualifier where the new rwlock is used and this is
appropriate.

4. Do not call process_pending_softirqs with any locks held. For that
unlock prior the call and re-acquire the locks after. After
re-acquiring the lock there is no need to check if pdev->vpci exists:
 - in apply_map because of the context it is called (no race condition
   possible)
 - for MSI/MSI-X debug code because it is called at the end of
   pdev->vpci access and no further access to pdev->vpci is made

5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
while accessing pdevs in vpci code.

6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
do not go away. The vPCI functions call several MSI-related functions
which already have existing non-vPCI callers. Change those MSI-related
functions to allow using either pcidevs_lock() or d->pci_lock for
ensuring pdevs do not go away. Holding d->pci_lock in read mode is
sufficient. Note that this pdev protection mechanism does not protect
other state or critical sections. These MSI-related functions already
have other race condition and state protection mechanims (e.g.
d->event_lock and msixtbl RCU), so we deduce that the use of the global
pcidevs_lock() is to ensure that pdevs do not go away.

7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
that pdevs do not go away. The purpose of this wrapper is to aid
readability and document the intent of the pdev protection mechanism.

8. When possible, the existing non-vPCI callers of these MSI-related
functions haven't been switched to use the newly introduced per-domain
pci_lock, and will continue to use the global pcidevs_lock(). This is
done to reduce the risk of the new locking scheme introducing
regressions. Those users will be adjusted in due time. One exception
is where the pcidevs_lock() in allocate_and_map_msi_pirq() is moved to
the caller, physdev_map_pirq(): this instance is switched to
read_lock(&d->pci_lock) right away.

Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
Changes in v13.3:
 - remove d's in comment in vpci_msix_arch_print()
 - remove comment after domain_done: label in vpci_dump_msi()
 - slightly tweak ASSERT_PDEV_LIST_IS_READ_LOCKED() definition in the
   #else case

Changes in v13.2:
 - clarify comment in vpci_msix_arch_print()
 - add parenthesis around arg in macro definition
 - make the helper macro an ASSERT itself

Changes in v13.1:
 - move pdev_list_is_read_locked() to pci.h
 - use pdev_list_is_read_locked() in more places
 - use d directly in pdev_list_is_read_locked()
 - wrap pdev_list_is_read_locked() helper in #ifndef NDEBUG, and add
   declaration in the #else case with no implementation
 - replace pcidevs_lock() with read_lock(&d->pci_lock) in physdev.c

Changes in v13:
 - hold off adding Roger's R-b tag even though it was provided on v12.2
 - use a wrapper construct to ease readability of odd-looking ASSERTs
 - new placement of ASSERT in __pci_enable_msix(), __pci_enable_msi(),
   and pci_enable_msi(). Rearrange/add pdev NULL check.
 - expand commit description with details about using either
   pcidevs_lock() or d->pci_lock

Changes in v12.2:
 - drop Roger's R-b
 - drop both locks on error paths in vpci_msix_arch_print()
 - add another ASSERT in vpci_msix_arch_print(), to enforce the
   expectation both locks are held before calling vpci_msix_arch_print()
 - move pdev_done label in vpci_dump_msi()
 - update comments in vpci_dump_msi() to say locks (plural)

Changes in v12.1:
 - use read_trylock() in vpci_msix_arch_print()
 - fixup in-code comments (revert double space, use DomXEN) in
   vpci_{read,write}()
 - minor updates in commit message
 - add Roger's R-b

Changes in v12:
 - s/pci_rwlock/pci_lock/ in commit message
 - expand comment about scope of pci_lock in sched.h
 - in vpci_{read,write}, if hwdom is trying to access a device assigned
   to dom_xen, holding hwdom->pci_lock is sufficient (no need to hold
   dom_xen->pci_lock)
 - reintroduce ASSERT in vmx_pi_update_irte()
 - reintroduce ASSERT in __pci_enable_msi{x}()
 - delete note 6. in commit message about removing ASSERTs since we have
   reintroduced them

Changes in v11:
 - Fixed commit message regarding possible spinlocks
 - Removed parameter from allocate_and_map_msi_pirq(), which was added
 in the prev version. Now we are taking pcidevs_lock in
 physdev_map_pirq()
 - Returned ASSERT to pci_enable_msi
 - Fixed case when we took read lock instead of write one
 - Fixed label indentation

Changes in v10:
 - Moved printk pas locked area
 - Returned back ASSERTs
 - Added new parameter to allocate_and_map_msi_pirq() so it knows if
 it should take the global pci lock
 - Added comment about possible improvement in vpci_write
 - Changed ASSERT(rw_is_locked()) to rw_is_write_locked() in
   appropriate places
 - Renamed release_domain_locks() to release_domain_write_locks()
 - moved domain_done label in vpci_dump_msi() to correct place
Changes in v9:
 - extended locked region to protect vpci_remove_device and
   vpci_add_handlers() calls
 - vpci_write() takes lock in the write mode to protect
   potential call to modify_bars()
 - renamed lock releasing function
 - removed ASSERT()s from msi code
 - added trylock in vpci_dump_msi

Changes in v8:
 - changed d->vpci_lock to d->pci_lock
 - introducing d->pci_lock in a separate patch
 - extended locked region in vpci_process_pending
 - removed pcidevs_lockis vpci_dump_msi()
 - removed some changes as they are not needed with
   the new locking scheme
 - added handling for hwdom && dom_xen case
---
 xen/arch/x86/hvm/vmsi.c       | 37 +++++++++++++++++--------
 xen/arch/x86/hvm/vmx/vmx.c    |  2 +-
 xen/arch/x86/irq.c            |  8 +++---
 xen/arch/x86/msi.c            | 20 +++++++++-----
 xen/arch/x86/physdev.c        |  2 ++
 xen/drivers/passthrough/pci.c |  9 +++---
 xen/drivers/vpci/header.c     | 18 ++++++++++++
 xen/drivers/vpci/msi.c        | 25 ++++++++++++++---
 xen/drivers/vpci/msix.c       | 52 ++++++++++++++++++++++++++++++-----
 xen/drivers/vpci/vpci.c       | 24 ++++++++++++++--
 xen/include/xen/pci.h         | 13 +++++++++
 xen/include/xen/sched.h       |  3 +-
 12 files changed, 172 insertions(+), 41 deletions(-)

diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index 128f23636279..211442eba704 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -468,7 +468,7 @@ int msixtbl_pt_register(struct domain *d, struct pirq *pirq, uint64_t gtable)
     struct msixtbl_entry *entry, *new_entry;
     int r = -EINVAL;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     if ( !msixtbl_initialised(d) )
@@ -538,7 +538,7 @@ void msixtbl_pt_unregister(struct domain *d, struct pirq *pirq)
     struct pci_dev *pdev;
     struct msixtbl_entry *entry;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     if ( !msixtbl_initialised(d) )
@@ -684,7 +684,7 @@ static int vpci_msi_update(const struct pci_dev *pdev, uint32_t data,
 {
     unsigned int i;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
 
     if ( (address & MSI_ADDR_BASE_MASK) != MSI_ADDR_HEADER )
     {
@@ -725,8 +725,8 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev)
     int rc;
 
     ASSERT(msi->arch.pirq != INVALID_PIRQ);
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
 
-    pcidevs_lock();
     for ( i = 0; i < msi->vectors && msi->arch.bound; i++ )
     {
         struct xen_domctl_bind_pt_irq unbind = {
@@ -745,7 +745,6 @@ void vpci_msi_arch_update(struct vpci_msi *msi, const struct pci_dev *pdev)
 
     msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address,
                                        msi->vectors, msi->arch.pirq, msi->mask);
-    pcidevs_unlock();
 }
 
 static int vpci_msi_enable(const struct pci_dev *pdev, unsigned int nr,
@@ -778,15 +777,14 @@ int vpci_msi_arch_enable(struct vpci_msi *msi, const struct pci_dev *pdev,
     int rc;
 
     ASSERT(msi->arch.pirq == INVALID_PIRQ);
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
     rc = vpci_msi_enable(pdev, vectors, 0);
     if ( rc < 0 )
         return rc;
     msi->arch.pirq = rc;
 
-    pcidevs_lock();
     msi->arch.bound = !vpci_msi_update(pdev, msi->data, msi->address, vectors,
                                        msi->arch.pirq, msi->mask);
-    pcidevs_unlock();
 
     return 0;
 }
@@ -797,8 +795,8 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq,
     unsigned int i;
 
     ASSERT(pirq != INVALID_PIRQ);
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
 
-    pcidevs_lock();
     for ( i = 0; i < nr && bound; i++ )
     {
         struct xen_domctl_bind_pt_irq bind = {
@@ -814,7 +812,6 @@ static void vpci_msi_disable(const struct pci_dev *pdev, int pirq,
     write_lock(&pdev->domain->event_lock);
     unmap_domain_pirq(pdev->domain, pirq);
     write_unlock(&pdev->domain->event_lock);
-    pcidevs_unlock();
 }
 
 void vpci_msi_arch_disable(struct vpci_msi *msi, const struct pci_dev *pdev)
@@ -854,6 +851,7 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
     int rc;
 
     ASSERT(entry->arch.pirq == INVALID_PIRQ);
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
     rc = vpci_msi_enable(pdev, vmsix_entry_nr(pdev->vpci->msix, entry),
                          table_base);
     if ( rc < 0 )
@@ -861,7 +859,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
 
     entry->arch.pirq = rc;
 
-    pcidevs_lock();
     rc = vpci_msi_update(pdev, entry->data, entry->addr, 1, entry->arch.pirq,
                          entry->masked);
     if ( rc )
@@ -869,7 +866,6 @@ int vpci_msix_arch_enable_entry(struct vpci_msix_entry *entry,
         vpci_msi_disable(pdev, entry->arch.pirq, 1, false);
         entry->arch.pirq = INVALID_PIRQ;
     }
-    pcidevs_unlock();
 
     return rc;
 }
@@ -895,6 +891,15 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
 {
     unsigned int i;
 
+    /*
+     * Assert that pdev_list doesn't change. ASSERT_PDEV_LIST_IS_READ_LOCKED
+     * is not suitable here because it may allow either pcidevs_lock() or
+     * pci_lock to be held, but here we rely on pci_lock being held, not
+     * pcidevs_lock().
+     */
+    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
+    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));
+
     for ( i = 0; i < msix->max_entries; i++ )
     {
         const struct vpci_msix_entry *entry = &msix->entries[i];
@@ -913,13 +918,23 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
             struct pci_dev *pdev = msix->pdev;
 
             spin_unlock(&msix->pdev->vpci->lock);
+            read_unlock(&pdev->domain->pci_lock);
             process_pending_softirqs();
+
+            if ( !read_trylock(&pdev->domain->pci_lock) )
+                return -EBUSY;
+
             /* NB: we assume that pdev cannot go away for an alive domain. */
             if ( !pdev->vpci || !spin_trylock(&pdev->vpci->lock) )
+            {
+                read_unlock(&pdev->domain->pci_lock);
                 return -EBUSY;
+            }
+
             if ( pdev->vpci->msix != msix )
             {
                 spin_unlock(&pdev->vpci->lock);
+                read_unlock(&pdev->domain->pci_lock);
                 return -EAGAIN;
             }
         }
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 48376cc32751..80a20b17f8a9 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -413,7 +413,7 @@ static int cf_check vmx_pi_update_irte(const struct vcpu *v,
 
     spin_unlock_irq(&desc->lock);
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(msi_desc->dev->domain);
 
     return iommu_update_ire_from_msi(msi_desc, &msi_desc->msg);
 
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index bbae7751e494..4beafa83f14b 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2164,7 +2164,7 @@ int map_domain_pirq(
         struct pci_dev *pdev;
         unsigned int nr = 0;
 
-        ASSERT(pcidevs_locked());
+        ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
 
         ret = -ENODEV;
         if ( !cpu_has_apic )
@@ -2321,7 +2321,7 @@ int unmap_domain_pirq(struct domain *d, int pirq)
     if ( (pirq < 0) || (pirq >= d->nr_pirqs) )
         return -EINVAL;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
     ASSERT(rw_is_write_locked(&d->event_lock));
 
     info = pirq_info(d, pirq);
@@ -2886,6 +2886,8 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 {
     int irq, pirq, ret;
 
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
+
     switch ( type )
     {
     case MAP_PIRQ_TYPE_MSI:
@@ -2915,7 +2917,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 
     msi->irq = irq;
 
-    pcidevs_lock();
     /* Verify or get pirq. */
     write_lock(&d->event_lock);
     pirq = allocate_pirq(d, index, *pirq_p, irq, type, &msi->entry_nr);
@@ -2931,7 +2932,6 @@ int allocate_and_map_msi_pirq(struct domain *d, int index, int *pirq_p,
 
  done:
     write_unlock(&d->event_lock);
-    pcidevs_unlock();
     if ( ret )
     {
         switch ( type )
diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c
index 335c0868a225..e721aaf5c001 100644
--- a/xen/arch/x86/msi.c
+++ b/xen/arch/x86/msi.c
@@ -602,7 +602,7 @@ static int msi_capability_init(struct pci_dev *dev,
     unsigned int i, mpos;
     uint16_t control;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(dev->domain);
     pos = pci_find_cap_offset(dev->sbdf, PCI_CAP_ID_MSI);
     if ( !pos )
         return -ENODEV;
@@ -771,7 +771,7 @@ static int msix_capability_init(struct pci_dev *dev,
     if ( !pos )
         return -ENODEV;
 
-    ASSERT(pcidevs_locked());
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(dev->domain);
 
     control = pci_conf_read16(dev->sbdf, msix_control_reg(pos));
     /*
@@ -988,11 +988,11 @@ static int __pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi,
 {
     struct msi_desc *old_desc;
 
-    ASSERT(pcidevs_locked());
-
     if ( !pdev )
         return -ENODEV;
 
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
+
     old_desc = find_msi_entry(pdev, msi->irq, PCI_CAP_ID_MSI);
     if ( old_desc )
     {
@@ -1043,9 +1043,12 @@ static int __pci_enable_msix(struct pci_dev *pdev, struct msi_info *msi,
 {
     struct msi_desc *old_desc;
 
-    ASSERT(pcidevs_locked());
+    if ( !pdev )
+        return -ENODEV;
+
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
 
-    if ( !pdev || !pdev->msix )
+    if ( !pdev->msix )
         return -ENODEV;
 
     if ( msi->entry_nr >= pdev->msix->nr_entries )
@@ -1154,7 +1157,10 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off)
 int pci_enable_msi(struct pci_dev *pdev, struct msi_info *msi,
                    struct msi_desc **desc)
 {
-    ASSERT(pcidevs_locked());
+    if ( !pdev )
+        return -ENODEV;
+
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(pdev->domain);
 
     if ( !use_msi )
         return -EPERM;
diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
index 47c4da0af7e1..7efa17cf4c1e 100644
--- a/xen/arch/x86/physdev.c
+++ b/xen/arch/x86/physdev.c
@@ -123,7 +123,9 @@ int physdev_map_pirq(domid_t domid, int type, int *index, int *pirq_p,
 
     case MAP_PIRQ_TYPE_MSI:
     case MAP_PIRQ_TYPE_MULTI_MSI:
+        read_lock(&d->pci_lock);
         ret = allocate_and_map_msi_pirq(d, *index, pirq_p, type, msi);
+        read_unlock(&d->pci_lock);
         break;
 
     default:
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 47c0eee7bdcc..c97dd4504a7a 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -750,7 +750,6 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
         pdev->domain = hardware_domain;
         write_lock(&hardware_domain->pci_lock);
         list_add(&pdev->domain_list, &hardware_domain->pdev_list);
-        write_unlock(&hardware_domain->pci_lock);
 
         /*
          * For devices not discovered by Xen during boot, add vPCI handlers
@@ -759,18 +758,18 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
         ret = vpci_add_handlers(pdev);
         if ( ret )
         {
-            printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret);
-            write_lock(&hardware_domain->pci_lock);
             list_del(&pdev->domain_list);
             write_unlock(&hardware_domain->pci_lock);
             pdev->domain = NULL;
+            printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret);
             goto out;
         }
+        write_unlock(&hardware_domain->pci_lock);
         ret = iommu_add_device(pdev);
         if ( ret )
         {
-            vpci_remove_device(pdev);
             write_lock(&hardware_domain->pci_lock);
+            vpci_remove_device(pdev);
             list_del(&pdev->domain_list);
             write_unlock(&hardware_domain->pci_lock);
             pdev->domain = NULL;
@@ -1146,7 +1145,9 @@ static void __hwdom_init setup_one_hwdom_device(const struct setup_hwdom *ctxt,
     } while ( devfn != pdev->devfn &&
               PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn) );
 
+    write_lock(&ctxt->d->pci_lock);
     err = vpci_add_handlers(pdev);
+    write_unlock(&ctxt->d->pci_lock);
     if ( err )
         printk(XENLOG_ERR "setup of vPCI for d%d failed: %d\n",
                ctxt->d->domain_id, err);
diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c
index 58195549d50a..8f5850b8cf6d 100644
--- a/xen/drivers/vpci/header.c
+++ b/xen/drivers/vpci/header.c
@@ -173,6 +173,7 @@ bool vpci_process_pending(struct vcpu *v)
         if ( rc == -ERESTART )
             return true;
 
+        write_lock(&v->domain->pci_lock);
         spin_lock(&v->vpci.pdev->vpci->lock);
         /* Disable memory decoding unconditionally on failure. */
         modify_decoding(v->vpci.pdev,
@@ -191,6 +192,7 @@ bool vpci_process_pending(struct vcpu *v)
              * failure.
              */
             vpci_remove_device(v->vpci.pdev);
+        write_unlock(&v->domain->pci_lock);
     }
 
     return false;
@@ -202,8 +204,20 @@ static int __init apply_map(struct domain *d, const struct pci_dev *pdev,
     struct map_data data = { .d = d, .map = true };
     int rc;
 
+    ASSERT(rw_is_write_locked(&d->pci_lock));
+
     while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == -ERESTART )
+    {
+        /*
+         * It's safe to drop and reacquire the lock in this context
+         * without risking pdev disappearing because devices cannot be
+         * removed until the initial domain has been started.
+         */
+        write_unlock(&d->pci_lock);
         process_pending_softirqs();
+        write_lock(&d->pci_lock);
+    }
+
     rangeset_destroy(mem);
     if ( !rc )
         modify_decoding(pdev, cmd, false);
@@ -244,6 +258,8 @@ static int modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_only)
     unsigned int i;
     int rc;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !mem )
         return -ENOMEM;
 
@@ -524,6 +540,8 @@ static int cf_check init_header(struct pci_dev *pdev)
     int rc;
     bool mask_cap_list = false;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     switch ( pci_conf_read8(pdev->sbdf, PCI_HEADER_TYPE) & 0x7f )
     {
     case PCI_HEADER_TYPE_NORMAL:
diff --git a/xen/drivers/vpci/msi.c b/xen/drivers/vpci/msi.c
index a253ccbd7db7..2913adf4f64a 100644
--- a/xen/drivers/vpci/msi.c
+++ b/xen/drivers/vpci/msi.c
@@ -263,7 +263,7 @@ REGISTER_VPCI_INIT(init_msi, VPCI_PRIORITY_LOW);
 
 void vpci_dump_msi(void)
 {
-    const struct domain *d;
+    struct domain *d;
 
     rcu_read_lock(&domlist_read_lock);
     for_each_domain ( d )
@@ -275,6 +275,9 @@ void vpci_dump_msi(void)
 
         printk("vPCI MSI/MSI-X d%d\n", d->domain_id);
 
+        if ( !read_trylock(&d->pci_lock) )
+            continue;
+
         for_each_pdev ( d, pdev )
         {
             const struct vpci_msi *msi;
@@ -313,17 +316,31 @@ void vpci_dump_msi(void)
                 {
                     /*
                      * On error vpci_msix_arch_print will always return without
-                     * holding the lock.
+                     * holding the locks.
                      */
                     printk("unable to print all MSI-X entries: %d\n", rc);
-                    process_pending_softirqs();
-                    continue;
+                    goto pdev_done;
                 }
             }
 
+            /*
+             * Unlock locks to process pending softirqs. This is
+             * potentially unsafe, as d->pdev_list can be changed in
+             * meantime.
+             */
             spin_unlock(&pdev->vpci->lock);
+            read_unlock(&d->pci_lock);
+        pdev_done:
             process_pending_softirqs();
+            if ( !read_trylock(&d->pci_lock) )
+            {
+                printk("unable to access other devices for the domain\n");
+                goto domain_done;
+            }
         }
+        read_unlock(&d->pci_lock);
+    domain_done:
+        ;
     }
     rcu_read_unlock(&domlist_read_lock);
 }
diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
index d1126a417da9..58c16ebdf283 100644
--- a/xen/drivers/vpci/msix.c
+++ b/xen/drivers/vpci/msix.c
@@ -147,6 +147,8 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr)
 {
     struct vpci_msix *msix;
 
+    ASSERT_PDEV_LIST_IS_READ_LOCKED(d);
+
     list_for_each_entry ( msix, &d->arch.hvm.msix_tables, next )
     {
         const struct vpci_bar *bars = msix->pdev->vpci->header.bars;
@@ -163,7 +165,13 @@ static struct vpci_msix *msix_find(const struct domain *d, unsigned long addr)
 
 static int cf_check msix_accept(struct vcpu *v, unsigned long addr)
 {
-    return !!msix_find(v->domain, addr);
+    int rc;
+
+    read_lock(&v->domain->pci_lock);
+    rc = !!msix_find(v->domain, addr);
+    read_unlock(&v->domain->pci_lock);
+
+    return rc;
 }
 
 static bool access_allowed(const struct pci_dev *pdev, unsigned long addr,
@@ -358,21 +366,35 @@ static int adjacent_read(const struct domain *d, const struct vpci_msix *msix,
 static int cf_check msix_read(
     struct vcpu *v, unsigned long addr, unsigned int len, unsigned long *data)
 {
-    const struct domain *d = v->domain;
-    struct vpci_msix *msix = msix_find(d, addr);
+    struct domain *d = v->domain;
+    struct vpci_msix *msix;
     const struct vpci_msix_entry *entry;
     unsigned int offset;
 
     *data = ~0UL;
 
+    read_lock(&d->pci_lock);
+
+    msix = msix_find(d, addr);
     if ( !msix )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_RETRY;
+    }
 
     if ( adjacent_handle(msix, addr) )
-        return adjacent_read(d, msix, addr, len, data);
+    {
+        int rc = adjacent_read(d, msix, addr, len, data);
+
+        read_unlock(&d->pci_lock);
+        return rc;
+    }
 
     if ( !access_allowed(msix->pdev, addr, len) )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_OKAY;
+    }
 
     spin_lock(&msix->pdev->vpci->lock);
     entry = get_entry(msix, addr);
@@ -404,6 +426,7 @@ static int cf_check msix_read(
         break;
     }
     spin_unlock(&msix->pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     return X86EMUL_OKAY;
 }
@@ -491,19 +514,33 @@ static int adjacent_write(const struct domain *d, const struct vpci_msix *msix,
 static int cf_check msix_write(
     struct vcpu *v, unsigned long addr, unsigned int len, unsigned long data)
 {
-    const struct domain *d = v->domain;
-    struct vpci_msix *msix = msix_find(d, addr);
+    struct domain *d = v->domain;
+    struct vpci_msix *msix;
     struct vpci_msix_entry *entry;
     unsigned int offset;
 
+    read_lock(&d->pci_lock);
+
+    msix = msix_find(d, addr);
     if ( !msix )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_RETRY;
+    }
 
     if ( adjacent_handle(msix, addr) )
-        return adjacent_write(d, msix, addr, len, data);
+    {
+        int rc = adjacent_write(d, msix, addr, len, data);
+
+        read_unlock(&d->pci_lock);
+        return rc;
+    }
 
     if ( !access_allowed(msix->pdev, addr, len) )
+    {
+        read_unlock(&d->pci_lock);
         return X86EMUL_OKAY;
+    }
 
     spin_lock(&msix->pdev->vpci->lock);
     entry = get_entry(msix, addr);
@@ -579,6 +616,7 @@ static int cf_check msix_write(
         break;
     }
     spin_unlock(&msix->pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     return X86EMUL_OKAY;
 }
diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
index 72ef277c4f8e..475272b173f3 100644
--- a/xen/drivers/vpci/vpci.c
+++ b/xen/drivers/vpci/vpci.c
@@ -42,6 +42,8 @@ extern vpci_register_init_t *const __end_vpci_array[];
 
 void vpci_remove_device(struct pci_dev *pdev)
 {
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !has_vpci(pdev->domain) || !pdev->vpci )
         return;
 
@@ -77,6 +79,8 @@ int vpci_add_handlers(struct pci_dev *pdev)
     const unsigned long *ro_map;
     int rc = 0;
 
+    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
+
     if ( !has_vpci(pdev->domain) )
         return 0;
 
@@ -361,7 +365,7 @@ static uint32_t merge_result(uint32_t data, uint32_t new, unsigned int size,
 
 uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
 {
-    const struct domain *d = current->domain;
+    struct domain *d = current->domain;
     const struct pci_dev *pdev;
     const struct vpci_register *r;
     unsigned int data_offset = 0;
@@ -376,12 +380,18 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
     /*
      * Find the PCI dev matching the address, which for hwdom also requires
      * consulting DomXEN.  Passthrough everything that's not trapped.
+     * If this is hwdom and the device is assigned to DomXEN, acquiring hwdom's
+     * pci_lock is sufficient.
      */
+    read_lock(&d->pci_lock);
     pdev = pci_get_pdev(d, sbdf);
     if ( !pdev && is_hardware_domain(d) )
         pdev = pci_get_pdev(dom_xen, sbdf);
     if ( !pdev || !pdev->vpci )
+    {
+        read_unlock(&d->pci_lock);
         return vpci_read_hw(sbdf, reg, size);
+    }
 
     spin_lock(&pdev->vpci->lock);
 
@@ -428,6 +438,7 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size)
         ASSERT(data_offset < size);
     }
     spin_unlock(&pdev->vpci->lock);
+    read_unlock(&d->pci_lock);
 
     if ( data_offset < size )
     {
@@ -470,7 +481,7 @@ static void vpci_write_helper(const struct pci_dev *pdev,
 void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
                 uint32_t data)
 {
-    const struct domain *d = current->domain;
+    struct domain *d = current->domain;
     const struct pci_dev *pdev;
     const struct vpci_register *r;
     unsigned int data_offset = 0;
@@ -484,7 +495,13 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
     /*
      * Find the PCI dev matching the address, which for hwdom also requires
      * consulting DomXEN.  Passthrough everything that's not trapped.
+     * If this is hwdom and the device is assigned to DomXEN, acquiring hwdom's
+     * pci_lock is sufficient.
+     *
+     * TODO: We need to take pci_locks in exclusive mode only if we
+     * are modifying BARs, so there is a room for improvement.
      */
+    write_lock(&d->pci_lock);
     pdev = pci_get_pdev(d, sbdf);
     if ( !pdev && is_hardware_domain(d) )
         pdev = pci_get_pdev(dom_xen, sbdf);
@@ -493,6 +510,8 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
         /* Ignore writes to read-only devices, which have no ->vpci. */
         const unsigned long *ro_map = pci_get_ro_map(sbdf.seg);
 
+        write_unlock(&d->pci_lock);
+
         if ( !ro_map || !test_bit(sbdf.bdf, ro_map) )
             vpci_write_hw(sbdf, reg, size, data);
         return;
@@ -534,6 +553,7 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
         ASSERT(data_offset < size);
     }
     spin_unlock(&pdev->vpci->lock);
+    write_unlock(&d->pci_lock);
 
     if ( data_offset < size )
         /* Tailing gap, write the remaining. */
diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h
index aabc5465a7d3..1df1863b1331 100644
--- a/xen/include/xen/pci.h
+++ b/xen/include/xen/pci.h
@@ -171,6 +171,19 @@ void pcidevs_lock(void);
 void pcidevs_unlock(void);
 bool __must_check pcidevs_locked(void);
 
+#ifndef NDEBUG
+/*
+ * Check to ensure there will be no changes to the entries in d->pdev_list (but
+ * not the contents of each entry).
+ * This check is not suitable for protecting other state or critical regions.
+ */
+#define ASSERT_PDEV_LIST_IS_READ_LOCKED(d)                               \
+        /* NB: d may be evaluated multiple times, or not at all */       \
+        ASSERT(pcidevs_locked() || ((d) && rw_is_locked(&(d)->pci_lock)))
+#else
+#define ASSERT_PDEV_LIST_IS_READ_LOCKED(d) ((void)(d))
+#endif
+
 bool pci_known_segment(u16 seg);
 bool pci_device_detect(u16 seg, u8 bus, u8 dev, u8 func);
 int scan_pci_devices(void);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 9da91e0e6244..37f5922f3206 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -462,7 +462,8 @@ struct domain
 #ifdef CONFIG_HAS_PCI
     struct list_head pdev_list;
     /*
-     * pci_lock protects access to pdev_list.
+     * pci_lock protects access to pdev_list. pci_lock also protects pdev->vpci
+     * structure from being removed.
      *
      * Any user *reading* from pdev_list, or from devices stored in pdev_list,
      * should hold either pcidevs_lock() or pci_lock in read mode. Optionally,

base-commit: f8791d0fd3adbda3701e7eb9db63a9351b478365
-- 
2.43.2



From xen-devel-bounces@lists.xenproject.org Wed Feb 21 02:53:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 02:53:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683775.1063372 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rccj8-00044d-8B; Wed, 21 Feb 2024 02:52:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683775.1063372; Wed, 21 Feb 2024 02:52:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rccj8-00044W-3X; Wed, 21 Feb 2024 02:52:54 +0000
Received: by outflank-mailman (input) for mailman id 683775;
 Wed, 21 Feb 2024 02:52:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rccj7-00044Q-02
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 02:52:53 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4d4f2409-d064-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 03:52:50 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d21a68dd3bso61303381fa.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 18:52:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4d4f2409-d064-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708483970; x=1709088770; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=kzN08dfURlX5uofvO0Xj4FiOZNs4wbcnJfxRfrsSU80=;
        b=lHXV+CJqQvW5VdxpKZS3cvTdYVh8GLXXx1rAtREK5QTv6A4+60orZLlevCF4sRLl94
         4jeew1zmMD1ZUKQeIEKcQuIrJvEszWWgquyrzm73bFGc4Fbd4Tls/b3cM7tAug16YFgE
         CHP5IrsNmVkauSgczAPp7B2tMNPEJK1DSKdRQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708483970; x=1709088770;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=kzN08dfURlX5uofvO0Xj4FiOZNs4wbcnJfxRfrsSU80=;
        b=gYnI0skUXrTmoIUw4r41vxoPv+ShjxTDrjLwB3uOzHPbPL9Z88tt0T8Kww33aqdvJ8
         3r92FcEZADo5AYkVV3g/aExzpi2JNmkmvfdNpXrOHsMaZw3d12XQLqqpVkz+YTh4yNF5
         GtiJ2HHwTauoYKwQM+A+V6l7r0rEO2JBB/16+Uzam112Ol0GWpdjB8++p8Ro07e1QWaH
         YC/XhOZaQHG3VyaIKkiSMjoydkO9498huvIo4n4IS/2SfxzdQww5MJcP1dW3SX64rcQM
         pujYLLpuk8WiB6r3ppxsd2N81y7t70el/djAyAf1HGri7lkEXqB247ndFQtyQAbBPbTZ
         osbQ==
X-Gm-Message-State: AOJu0Yy+U2sJabZ2u65RIKbQRvdlugWtpVyvbwGxEw4ZeDgijojcCPFf
	iNfgAZJcZ4uW3ZuAvO6Wpoi8Fs4bWh9B1Q3bneAtQMDUWFlYx8iV1bYjKFe8O/+8UiFg1GKfDDz
	fMdWgLSdwlBKhxjmselXQl9wmXLBiG/qfxf2pEcOBdR9A8jmfXmQ=
X-Google-Smtp-Source: AGHT+IGpDhcZ8opn9LBoHgxEWsHqspX4nd8HJr69NFgJlL8vMKOv6Ldu704rOF3QE85MYOf2LFxFLK43Hf02Y+I/R0k=
X-Received: by 2002:ac2:538a:0:b0:512:bce5:1e0f with SMTP id
 g10-20020ac2538a000000b00512bce51e0fmr3843407lfh.64.1708483970241; Tue, 20
 Feb 2024 18:52:50 -0800 (PST)
MIME-Version: 1.0
References: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
 <ZdN06v1k3E8xarLY@mattapan.m5p.com>
In-Reply-To: <ZdN06v1k3E8xarLY@mattapan.m5p.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 10:52:39 +0800
Message-ID: <CA+zSX=aFyoJCZVbj4u5m3Rm1dTQDH_fLfuWuV5ms04qhc38Q7w@mail.gmail.com>
Subject: Re: Stats on Xen tarball downloads
To: Elliott Mitchell <ehem+xen@m5p.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, committers@xenproject.org, 
	Kelly Choi <kelly.choi@cloud.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 19, 2024 at 11:34=E2=80=AFPM Elliott Mitchell <ehem+xen@m5p.com=
> wrote:
>
> On Mon, Feb 19, 2024 at 06:01:54PM +0800, George Dunlap wrote:
> >
> > Looking at the *non*-4.18 downloads, nearly all of them have user
> > agents that make it clear they're part of automated build systems:
> > user agents like curl and wget, but also "Go-http-client", "libfetch",
>                    ^^^^     ^^^^
>
> I reject this claim.  `curl` or `wget` could be part of an interactive
> operation.  Telling a browser to copy a URL into the paste buffer, then
> using `wget`/`curl` is entirely possible.  I may be the outlier, but I
> routinely do this.

It's not just the user agent; there are certain statistical
regularities that make me think it's automated.  e.g., a specific
version of curl always downloading a specific version of the tarball,
the tar.gz and the tar.gz.sig being downloaded exactly the same time
distance apart.  There certainly *are* manual wget / curl invocations,
but the majority of them look to me like they're part of automated
systems.

(And the "Go-http-client" instances are kind of fascinating to me --
someone wrote something on golang to download the Xen tarball?  And
always 4.15.1?  And it's being run from both NH, USA and from Finland,
and a handful of other places that seem unrelated?  What project is
this?)

> > It's not really clear to me why we'd be getting 300-ish people
> > downloading the Xen 4.18.0 tarball, 2/3 of which are on Windows.  But
> > then I'm also not sure why someone would *fake* hundreds of downloads
> > a week from unique IP addresses; and in particular, if you were going
> > to fake hundreds of downloads a week, I'm not sure why you'd only fake
> > the most recent release.
>
> Remember the browser wars?  At one point many sites were looking for
> IE/Windows and sending back error messages without those.  Getting the
> tarball on Windows doesn't seem too likely, faking the browser was
> pretty common for a while.

Right, which is why I wanted to look more into the rest of the data to
see if I could get a feel for it.  There are very few Windows user
agents for the other versions; the handful of browser agents for
non-4.18.0 tarballs look very normal and unix-y.  So the question is,
why would you fake loads of downloads for Chrome / Firefox / Edge on
Windows *only* for 4.18.0?

I agree that none of the current explanations make a lot of sense; but
I continue to believe that the "We have loads of actual humans
downloading the 4.18.0 tarball via browsers, even on Windows" is the
least-bad fit.  (Feel free to propose others, though.)

 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 02:55:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 02:55:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683779.1063380 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rccm1-0004dv-JS; Wed, 21 Feb 2024 02:55:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683779.1063380; Wed, 21 Feb 2024 02:55:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rccm1-0004do-Gf; Wed, 21 Feb 2024 02:55:53 +0000
Received: by outflank-mailman (input) for mailman id 683779;
 Wed, 21 Feb 2024 02:55:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rccm0-0004di-LW
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 02:55:52 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b881c6b3-d064-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 03:55:50 +0100 (CET)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2d2505352e6so12046151fa.3
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 18:55:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b881c6b3-d064-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708484150; x=1709088950; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wYzFasVS6k5md+txoN4+vxSF9iSmkvpQ0PJpRJ5AdxA=;
        b=FKUj2xN2Orh6l3+qfWaw0A37tlnrHtI/00fOA/e4XJhmtSYajb1KiA71M2+rbDgmKj
         w7IqdkaSejNz1xNydhOi5jgoTEkNEARvG6KPnBxF+zRJMgcz9jgU/OEpAAG40M2grqka
         F5hXMFXY0EqnWtDIHHHWkuhCQzsD5ZQnfm5Go=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708484150; x=1709088950;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=wYzFasVS6k5md+txoN4+vxSF9iSmkvpQ0PJpRJ5AdxA=;
        b=ajdz8rLlEPWYVD4xljfzUT8bFS1ABddrvsyiO2zxycqEtRWLIVLqajyHXexW9oCBUu
         NcCNH0+/Kkcupjn0791y0zE2zA1ojOgtyS+09t0CoFMcOBMMssPx73zkoLUKvWPL+ukB
         kmbFiXMSWLhd3YyyMiEDLhvqJhCK2s3VZ4KRMgNLlzXn5Um1SNWLF018IoLecjrExMf4
         eGveWvkFmnksSKLk5q+FvaCKAf2QdFrnkoVjBaE40aqXMZ+KeWgrVBQr24ySN9b5Tblt
         UV8UJ1vxCJw2LPa5qFTkZVh3Th0pUzgD4OCU/nv99j+l218N6/2G5gGC24rvSQGWXhmq
         9SdQ==
X-Forwarded-Encrypted: i=1; AJvYcCVnimyPf5CufIdJBNmN/RkP8Vtwb4zlCLUg2pF3c9AmIyhKXRkobmZhDUi/DbFa1k4uXLcFegWfAOFzdCx71SysS3E/679cZ6li7/n9bwE=
X-Gm-Message-State: AOJu0YxIWWFPnwwD/3WLJFXy7kZdiFC+HcyfPvES/usHJIc39719GIpY
	SgeC1zkLMkEhn9p1s+nGiom03e87OyxLq9WuOKPcpewrDrSWzXhtoame0XYZrHGJAkQLSXsxnSk
	igROzz2RA4dkna0ywaauwqWxsVasWUyCF+m3RzQ==
X-Google-Smtp-Source: AGHT+IFeOG94QipeGLYIa1c1ry3/xlDVG7VRvY5SOqbG4yKwY47QErdnvEz7QgW3uPmQIbfrQImhvRkHh06J0IaKBSE=
X-Received: by 2002:a2e:9dda:0:b0:2d2:4fa4:f4b2 with SMTP id
 x26-20020a2e9dda000000b002d24fa4f4b2mr1538664ljj.28.1708484150236; Tue, 20
 Feb 2024 18:55:50 -0800 (PST)
MIME-Version: 1.0
References: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
 <ZdMt88BSNMYtq39n@macbook> <36229344-e2e2-46e1-839b-b7b3b0843f23@suse.com>
In-Reply-To: <36229344-e2e2-46e1-839b-b7b3b0843f23@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 10:55:39 +0800
Message-ID: <CA+zSX=YvTgEMC0S5OqSFvxfVDeruiO=FphyVx5fv75oT-QW01A@mail.gmail.com>
Subject: Re: Stats on Xen tarball downloads
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Xen-devel <xen-devel@lists.xenproject.org>, committers@xenproject.org, 
	Kelly Choi <kelly.choi@cloud.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 19, 2024 at 6:38=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 19.02.2024 11:31, Roger Pau Monn=C3=A9 wrote:
> > On Mon, Feb 19, 2024 at 06:01:54PM +0800, George Dunlap wrote:
> >> One of the questions we had with respect to changing our release
> >> practice (for instance, making the process more light-weight so that
> >> we could do a point release after every XSA) was, "How many people are
> >> actually using the tarballs?"
> >
> > What would this more lightweight process involve from a downstream
> > PoV?  IOW: in what would the contents of the tarball change compared
> > to the current releases?
>
> From all prior discussion my conclusion was "no tarball at all".

Or at very least, the tarball would be a simple `git archive` of a
release tag.   Right now the tarball creation has a number of
annoyingly manual parts about it.


 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 03:06:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 03:06:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683787.1063390 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rccvp-0006U8-GL; Wed, 21 Feb 2024 03:06:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683787.1063390; Wed, 21 Feb 2024 03:06:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rccvp-0006U1-DU; Wed, 21 Feb 2024 03:06:01 +0000
Received: by outflank-mailman (input) for mailman id 683787;
 Wed, 21 Feb 2024 03:06:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rccvo-0006Tv-KA
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 03:06:00 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 23730c57-d066-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 04:05:59 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-512aafb3ca8so3261790e87.3
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 19:05:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 23730c57-d066-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708484759; x=1709089559; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=CDVWoH+nKiVWuUc0iKL9rSHj/ShAiT6i2MROjC3kJK0=;
        b=EWRLju4j2T545fPdl5AMRp1wlbH31YyBhtHz/NzB2P9Tr13PRfMZ5rP6NHnNxTC+6T
         0EyjE8yJqnFJzkxFFV8MaMdls/94bUpom1a4C8OhjzcgZMZ+HLInyNTfwFN92g91KZQH
         KSJWLjeHQtFt8VA6bECv+d0XdxUycIVTteaCg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708484759; x=1709089559;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=CDVWoH+nKiVWuUc0iKL9rSHj/ShAiT6i2MROjC3kJK0=;
        b=d0UfIozng1Ov0unHqgRWauyuEiOlc4cWygv03kHComd293yfvRaU2FGZ0OBoNzwjb2
         sHeYErbMS4tzjsZgkgOwl4ve5FWwxjV4K9g4i5AM8dehxpIMIsPlEwx/tfCeNF6/o4nS
         xfh2sLmU7sDeFYutcWA6nln9oJ7Cn86mKIfv+pcvt/D/5K7/pvsFdav8WJCcZHO1ThZ5
         fd5JuJ1bKwRGm2ukgnEl7U78JgSxtMMf+7Y5UK9lBErnQpqvqUMvflDYb6KA0ikBlIiN
         5S5ulRCo8ulEdmaXpC8loeyOVEmMRWs6WCEdrA490czew6ssl10n1OFPm+43GHOfi70C
         sB1g==
X-Gm-Message-State: AOJu0Yx7zFiO01/aCOSO+OZWuULVVUNrm747664yIQF7EU4xYWrHZ8cp
	C4/SteAJxcTPcuwPsSksISmTn8iAPQEZ42HrbUdOPI0Wu2/mZngjiAg8xVPQ4fljN9sqS/lnwdg
	iEG4Sa/hy9pZ1jEZSQm3Onkr8+JIYPcRFyLnYrQ==
X-Google-Smtp-Source: AGHT+IHPepeK4YFR8HmlROaoNLv3XTMXF4DHgTC+llsoJqDqDIxMyy7bURyiPYhiWIR4aJSpuOad2MVc/ehhL1KXrIQ=
X-Received: by 2002:ac2:4859:0:b0:512:b788:2c0e with SMTP id
 25-20020ac24859000000b00512b7882c0emr4231908lfy.43.1708484759122; Tue, 20 Feb
 2024 19:05:59 -0800 (PST)
MIME-Version: 1.0
References: <20240215150644.2477378-1-olekstysh@gmail.com>
In-Reply-To: <20240215150644.2477378-1-olekstysh@gmail.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 11:05:47 +0800
Message-ID: <CA+zSX=bXcSa6JWCSVkNSmcbtJ10St8PAmSeTrhRMt3XL95M6Ng@mail.gmail.com>
Subject: Re: [PATCH V3] libxl: Add "grant_usage" parameter for virtio disk devices
To: Oleksandr Tyshchenko <olekstysh@gmail.com>
Cc: xen-devel@lists.xenproject.org, 
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>, Nick Rosbrook <rosbrookn@gmail.com>, 
	Juergen Gross <jgross@suse.com>, Viresh Kumar <viresh.kumar@linaro.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 15, 2024 at 11:07=E2=80=AFPM Oleksandr Tyshchenko
<olekstysh@gmail.com> wrote:
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> Allow administrators to control whether Xen grant mappings for
> the virtio disk devices should be used. By default (when new
> parameter is not specified), the existing behavior is retained
> (we enable grants if backend-domid !=3D 0).
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Golang bits:

Acked-by: George Dunlap <george.dunlap@cloud.com>


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 03:56:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 03:56:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683793.1063401 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcdiW-0004bG-8M; Wed, 21 Feb 2024 03:56:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683793.1063401; Wed, 21 Feb 2024 03:56:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcdiW-0004b9-43; Wed, 21 Feb 2024 03:56:20 +0000
Received: by outflank-mailman (input) for mailman id 683793;
 Wed, 21 Feb 2024 03:56:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcdiU-0004aj-Bi; Wed, 21 Feb 2024 03:56:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcdiT-0000Fu-VA; Wed, 21 Feb 2024 03:56:17 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcdiN-0000l0-2u; Wed, 21 Feb 2024 03:56:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcdiN-0006Dt-23; Wed, 21 Feb 2024 03:56:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ne/kl5IvuzQb6rjS2HynZIM7at59ujyCvOVoCZGlhRg=; b=iPoJu/ocvOG5f4Yq9uzsLmk/XV
	KDAD1ex2TzfNREK8FwRz7uSyZE9QFNEpurKbRHpJlsXxHRmYvN8/8ys/HHzLBSlUwesQxkwsQgOVL
	XYCk5M8PWYTW1j0IlNqJLOjBwUWWG2x4oyGQniOX8233tgDh+CusNsR1wp0r6KexEcFA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184717-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184717: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=2ca8d559744319025592df10ada0f714fc3b8e15
X-Osstest-Versions-That:
    ovmf=edc6681206c1a8791981a2f911d2fb8b3d2f5768
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 21 Feb 2024 03:56:11 +0000

flight 184717 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184717/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 2ca8d559744319025592df10ada0f714fc3b8e15
baseline version:
 ovmf                 edc6681206c1a8791981a2f911d2fb8b3d2f5768

Last test of basis   184669  2024-02-14 17:44:06 Z    6 days
Testing same since   184717  2024-02-21 01:43:32 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jiaxin Wu <jiaxin.wu@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   edc6681206..2ca8d55974  2ca8d559744319025592df10ada0f714fc3b8e15 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 06:31:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 06:31:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683802.1063410 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcg8V-00051t-V8; Wed, 21 Feb 2024 06:31:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683802.1063410; Wed, 21 Feb 2024 06:31:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcg8V-00051m-S8; Wed, 21 Feb 2024 06:31:19 +0000
Received: by outflank-mailman (input) for mailman id 683802;
 Wed, 21 Feb 2024 06:31:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcg8U-00051c-Gv; Wed, 21 Feb 2024 06:31:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcg8U-0003Qh-4i; Wed, 21 Feb 2024 06:31:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcg8T-0006Yc-Oc; Wed, 21 Feb 2024 06:31:17 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcg8T-0003UR-O4; Wed, 21 Feb 2024 06:31:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hXsO11xkjYBnSV2iDjljmMuyL/dX5ckZRXpxAjrsFHQ=; b=VMwKkjO+3JX9xSk+AU6lOexzgd
	O/p604jUhvX4svUah8ONjFehRe6zAK64JwgT3kPSs/rAf6twFgoEYjD+NQE85u25tb0t1+9PsWZEu
	7tMhUBnpbv4Ec91SU/RNRD7pI4T4j3fYx1kqzDQXK3XZG3JEb1Ychb1Rc+klyiNXKUH4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184714-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184714: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-start/freebsd.repeat:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=fca7526b7d8910c6125cb1ebc3e78ccd5f50ec52
X-Osstest-Versions-That:
    linux=b401b621758e46812da61fa58a67c3fd8d91de0d
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 21 Feb 2024 06:31:17 +0000

flight 184714 linux-linus real [real]
flight 184718 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184714/
http://logs.test-lab.xenproject.org/osstest/logs/184718/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-freebsd12-amd64 21 guest-start/freebsd.repeat fail pass in 184718-retest
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184718-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184700
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184700
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184700
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184700
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184700
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184700
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184700
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184700
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                fca7526b7d8910c6125cb1ebc3e78ccd5f50ec52
baseline version:
 linux                b401b621758e46812da61fa58a67c3fd8d91de0d

Last test of basis   184700  2024-02-19 03:44:55 Z    2 days
Testing same since   184714  2024-02-20 21:12:20 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Borislav Petkov <bp@alien8.de>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Torvalds <torvalds@linuxfoundation.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   b401b621758e..fca7526b7d89  fca7526b7d8910c6125cb1ebc3e78ccd5f50ec52 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 07:02:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 07:02:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683811.1063420 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgcc-0000Bl-HQ; Wed, 21 Feb 2024 07:02:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683811.1063420; Wed, 21 Feb 2024 07:02:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgcc-0000Bd-Ex; Wed, 21 Feb 2024 07:02:26 +0000
Received: by outflank-mailman (input) for mailman id 683811;
 Wed, 21 Feb 2024 07:02:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rcgcb-0000BX-OV
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 07:02:25 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 29a5dbdf-d087-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 08:02:23 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-512b29f82d1so4404510e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 23:02:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 29a5dbdf-d087-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708498943; x=1709103743; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=0ZDuYmCw68tSZSZV15ysOk1bl/+jDWERuA9QV+0SCDw=;
        b=f7WWbUHlfp/jFxA+/VbPZqVyqhlbDTxJhPa6wtaKtFwwIY2QkQ96Y6KwkOgwFRtO1J
         PFCzccyatNLy9tY5vVoEr7UdZ8VtqGkB9ZmBoHQm1eM8lgO8S9h55pE/TutN5zVLxdgA
         w/V0wPlcnpFQnHZYkijsmr4QfFZw7qUEjOpww=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708498943; x=1709103743;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=0ZDuYmCw68tSZSZV15ysOk1bl/+jDWERuA9QV+0SCDw=;
        b=TV+pF8w2h/WJ1PfiwynZhhaltV7kgnMxpDkCo8x3+IdOqWkiFmr8HWViErdcWbXEyZ
         ULmQjkq57WYsYWszP4pCaC58CXpXj8g9BpwaEY8rHXsZ6MiB4dVE/0hNpvm/VauzSCFB
         ncG6JBOlw1Ak4B5f6oCs0BaXSYz1VMe7wr+uMdfr6QpDS3Sp7uU7pAHMWsCzwD4JdThx
         +2HREd5sDyVe9vsvgseUFlkSmcfhsALhG21PTFohL3K91vO7NTfgS6XAGvYfy2L7i5+i
         X1xHXZkodNmD8jbMyUkm621leK2pQ9Pp0OMcjX6CyW2iklUXAkXHOdMZC8y3CyeB5lRo
         I8Jg==
X-Forwarded-Encrypted: i=1; AJvYcCXFnwHeZj3i2/SM65z8mfpq9dRhYZSR2XuryaH490k7UqST3ayTWCCunuXiT5mN7k8mYhVVc6dFxaE7ikBy8V5FojgMbCHcxTZmM+111Ls=
X-Gm-Message-State: AOJu0YzdqiCKBtpcNEctoKaYx6yiexUMy66ZdN10i7kagVzXqN96q+mk
	XeWSaVDg7RGch5+nbXuHEvm0W1KEDZkM5DKU7qm4Mvgg8bmLYH2WGGLvzX/vR2vYK+VJFtUOXm8
	OG84eWtZfwRDgK1EQFYKT0z2OMlusCtDnXMrT6g==
X-Google-Smtp-Source: AGHT+IEuPx0Dxmb1fEwJS9ag9XvHSGKUCqc9RozVlhurOAtTecM0QJqWBIaAKTY30AbN6O2OBLFLCXL6ddtpkVnzj6Y=
X-Received: by 2002:ac2:4f8a:0:b0:511:7259:370d with SMTP id
 z10-20020ac24f8a000000b005117259370dmr10189457lfs.33.1708498942856; Tue, 20
 Feb 2024 23:02:22 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-2-george.dunlap@cloud.com> <66962dfc-db89-40bc-8411-9b92f9458b16@suse.com>
In-Reply-To: <66962dfc-db89-40bc-8411-9b92f9458b16@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 15:02:11 +0800
Message-ID: <CA+zSX=ZwUT4Wf7eY3DFyp=Bf_MndExoKCZBVGEhF-ByaOEH7tw@mail.gmail.com>
Subject: Re: [PATCH 1/6] xen/hvm: Convert hap_capabilities into a bitfield
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 19, 2024 at 9:36=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 06.02.2024 02:20, George Dunlap wrote:
> > hvm_function_table is an internal structure; rather than manually
> > |-ing and &-ing bits, just make it a boolean bitfield and let the
> > compiler do all the work.  This makes everything easier to read, and
> > presumably allows the compiler more flexibility in producing efficient
> > code.
> >
> > No functional change intended.
> >
> > Signed-off-by: George Dunlap <george.dunlap@cloud.com>
>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
> > ---
> > Questions:
> >
> > * Should hap_superpage_2m really be set unconditionally, or should we
> >   condition it on cpu_has_svm_npt?
>
> That's HAP capabilities; there's not going to be any use of HAP when
> there's no NPT (on an AMD system). IOW - all is fine as is, imo.

Basically there are two stances to take:

1. hap_superpage_2m always has meaning.  If there's no HAP, then there
are no HAP superpages, so we should say that there are no superpages.

2. hap_superpage_2m only has meaning if hap is enabled.  If there's no
HAP, then the question "are there superpages" is moot; nobody should
be paying attention to it.

The second is not without precedent, but is it really the best stance?
 It means that before reading hap_superpage_2m, you have to first
check hap; and it introduces a risk (no matter how small) that someone
will forget to check, and end up doing the wrong thing.

Not a huge deal, but overall my vote would be #1.  I may send a patch
at some point in the future.

> > * Do we really need to "!!cpu_has_svm_npt"?  If so, wouldn't it be
> >   better to put the "!!"  in the #define, rather than requiring the
> >   user to know that it's needed?
>
> Considering that hap_supported is bool now, the !! can simply be
> dropped. We've been doing so as code was touched anyway, not in a
> concerted effort.

Right, forgot to actually delete this para after adding a patch to address =
it.

> > --- a/xen/arch/x86/hvm/vmx/vmcs.c
> > +++ b/xen/arch/x86/hvm/vmx/vmcs.c
> > @@ -113,8 +113,8 @@ static int cf_check parse_ept_param_runtime(const c=
har *s)
> >      int val;
> >
> >      if ( !cpu_has_vmx_ept || !hvm_funcs.hap_supported ||
> > -         !(hvm_funcs.hap_capabilities &
> > -           (HVM_HAP_SUPERPAGE_2MB | HVM_HAP_SUPERPAGE_1GB)) )
> > +         !(hvm_funcs.caps.hap_superpage_2mb ||
> > +           hvm_funcs.caps.hap_superpage_1gb) )
> >      {
> >          printk("VMX: EPT not available, or not in use - ignoring\n");
>
> Just to mention it: The conditional and the log message don't really
> fit together. (I was first wondering what the 2mb/1gb checks had to
> do here at all, but that's immediately clear when seeing that the
> only sub-option here is "exec-sp".)

So you mean basically that the checks & error message are poorly
factored, because there's only a single sub-option?  (i.e., if there
were options which didn't rely on superpages, the check would be
incorrect?)

Let me know if there's something concrete you'd like me to do here.

> > @@ -104,8 +96,11 @@ struct hvm_function_table {
> >      /* Hardware virtual interrupt delivery enable? */
> >      bool virtual_intr_delivery_enabled;
> >
> > -    /* Indicate HAP capabilities. */
> > -    unsigned int hap_capabilities;
> > +    struct {
> > +        /* Indicate HAP capabilities. */
> > +        bool hap_superpage_1gb:1,
> > +            hap_superpage_2mb:1;
>
> Nit: Would be nice imo if the two identifiers aligned vertically with
> one another.

Done.

 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 07:09:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 07:09:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683817.1063431 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgjI-0000uS-7c; Wed, 21 Feb 2024 07:09:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683817.1063431; Wed, 21 Feb 2024 07:09:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgjI-0000uL-4O; Wed, 21 Feb 2024 07:09:20 +0000
Received: by outflank-mailman (input) for mailman id 683817;
 Wed, 21 Feb 2024 07:09:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rcgjH-0000uF-5O
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 07:09:19 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2051665f-d088-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 08:09:17 +0100 (CET)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2d204e102a9so80618281fa.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 23:09:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2051665f-d088-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708499357; x=1709104157; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=6+KSlJ1bHrB5YosM9lWw6VXzlk/IiNkl3Wa0OgtGAhc=;
        b=LDisUKiQXdVzFPnZfmIJ528cncP8VVucSYjFGm5xQXAWKac8Djzg5PxG/6HzJRKbLz
         40N/Cv5MvZ+Xy138W0OrVE8Us8Vxe8liNcdc3V7iZ5I1WsohXd9aFQN8rvap5THIdoFJ
         qMkqCiND2QzvgLyIfcxPM2+PwhQT5Lezj0hJs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708499357; x=1709104157;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=6+KSlJ1bHrB5YosM9lWw6VXzlk/IiNkl3Wa0OgtGAhc=;
        b=vZp/OjaZNdSDNJy6xGZrg6F/TxJ9Ep0JRWRFZ/GP0WZfGp5fFQMvI+XHgegmvCnDU7
         2mNrYvrccOvzC4v6yXNqzVWfMS21B7t1qn0iPQTlfcs9pjEM+jpSz+USseqPXfgA+5A0
         ZLzslAu3k3aPzlfIDP7YyGHphwrV3c0Chc/X2jFJWTn0cLqFhCjjXgqpRQ2D2fOi99q8
         MWTM80XSt7eHLfhNSdkNhMFVnpmRMPupqvCZtle46K4wo33As2lPHCX/5BBkPemb1Cg+
         2gz3r3YcL2LmjwmaoSDD/DmPp8iFyirXThnKAxBKvUo8J7Nfotl5y1BNnTRMyoL4iRfd
         Q19Q==
X-Forwarded-Encrypted: i=1; AJvYcCVzaYFQWx5cwweN5Z+UQjDTMo++kRa+p8YL89BVg/+vTDucyhyhB+VWoXNbPoazdK5W0nz+EsLHkcNApRU3al+ahTSLxhyo1ekKmxqyrWw=
X-Gm-Message-State: AOJu0YzSTyScT9Brb15hdHtFqTTRBQWhMpCezy3ct7pR0ZtbS4/2aNeS
	8QUplwPB+WESAbp4+hN1SwjHMPiyE0VaKqbtjipueAzxmEwBUEkGo3Nb+Xp/UY4Aszl2gH9iZ6t
	CvnFJgcYQTkyRnfaa9cYAMjOMrLGP86i3XQP08Q==
X-Google-Smtp-Source: AGHT+IGK7PqP3f+3KPNnrcLSC8JDdjkkLvnkykJy7LMEEPWttvqUwb+pfyS7jGDNhe9aB8kpp65bV6jBMGlPGcrZvY8=
X-Received: by 2002:a2e:9612:0:b0:2d2:4774:7642 with SMTP id
 v18-20020a2e9612000000b002d247747642mr2516813ljh.44.1708499356743; Tue, 20
 Feb 2024 23:09:16 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-3-george.dunlap@cloud.com> <ebc81051-765a-424a-9b30-97d6755e84c3@suse.com>
In-Reply-To: <ebc81051-765a-424a-9b30-97d6755e84c3@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 15:09:05 +0800
Message-ID: <CA+zSX=YB3TnBF_fLT_htEtvgzV+1uLpNPE5p1kQgbyMKYDBBAQ@mail.gmail.com>
Subject: Re: [PATCH 2/6] svm: Improve type of cpu_has_svm_feature
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 19, 2024 at 10:03=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wr=
ote:
>
> On 06.02.2024 02:20, George Dunlap wrote:
> > The "effective type" of the cpu_has_svm_feature macro is effectively
> > an unsigned log with one bit set (or not); at least one place someone
> > felt compelled to do a !! to make sure that they got a boolean out of
> > it.
> >
> > Ideally the whole of this would be folded into the cpufeature.h
> > infrastructure.  But for now, duplicate the more type-safe static
> > inlines in that file, and remove the !!.
> >
> > Signed-off-by: George Dunlap <george.dunlap@cloud.com>
>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> albeit preferably with ...
>
> > --- a/xen/arch/x86/include/asm/hvm/svm/svm.h
> > +++ b/xen/arch/x86/include/asm/hvm/svm/svm.h
> > @@ -38,7 +38,10 @@ extern u32 svm_feature_flags;
> >  #define SVM_FEATURE_SSS           19 /* NPT Supervisor Shadow Stacks *=
/
> >  #define SVM_FEATURE_SPEC_CTRL     20 /* MSR_SPEC_CTRL virtualisation *=
/
> >
> > -#define cpu_has_svm_feature(f) (svm_feature_flags & (1u << (f)))
> > +static inline bool cpu_has_svm_feature(unsigned int feat)
> > +{
> > +    return svm_feature_flags & (1u << (feat));
>
> ... the inner pair of parentheses dropped.

Done, thanks.

 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 07:09:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 07:09:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683818.1063442 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgjM-0001A5-Ff; Wed, 21 Feb 2024 07:09:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683818.1063442; Wed, 21 Feb 2024 07:09:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgjM-00019y-BL; Wed, 21 Feb 2024 07:09:24 +0000
Received: by outflank-mailman (input) for mailman id 683818;
 Wed, 21 Feb 2024 07:09:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TA2L=J6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcgjL-00019P-I2
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 07:09:23 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 21f17d81-d088-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 08:09:20 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a3e891b5e4eso333104166b.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 23:09:20 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 tz3-20020a170907c78300b00a3e0b7e7217sm4571618ejc.48.2024.02.20.23.09.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 23:09:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21f17d81-d088-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708499359; x=1709104159; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=LJduH8fD1eLpHOA380mFlZlKNFPfToyyPfK1YmR13nE=;
        b=Kf8PIHnBh3IIx3RCfXKvgtUELvsDeyIv0J7jVcvhxvbfjX5AtnGLlBzE0YDyGKLB+f
         ZBa+QVR9E5UyIg+ZYvhkNCHhrOKHpw9q804iFJtG+/1B8ZMkbG6g4OUBX20PnKvRWARg
         2/GI6uGFhwOK3qwWSLp/g+LnV2jpHBsJz93j0SmGlFG3Z5u4O2bKIw8asHDqey69VlfC
         /rmZ89Ixi2B6kTiZyrGDKs/66QiAGIJtRIiU7/XVe+mZS+xMYhwSwTZcViwTZe/sFvk/
         FriFUKGBhAbk/FMMlaRWJsz6suwtMOGP0cEbOatt1f/5jErn7MrTNg9oT2dOAmu8ErHo
         kfpw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708499359; x=1709104159;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=LJduH8fD1eLpHOA380mFlZlKNFPfToyyPfK1YmR13nE=;
        b=hSHy5CFYPvP17rBVybXcR8BVme9H0ivyOAnnF/1FwG6f4nmGhCjimLCClEq5wxfrST
         73yEk0gcVis8Taq84SoZIfCapB1zcYIfRmPnr90OqBNbYITvNBCFDbAAKZBwocht2TFb
         zbTvplEulrraL9ddl6CFnyovNNkcsSaFES75tiiHZIofolPEsBYMPXNbtB7qBhwRXk1M
         hPiKM466wd81I7VEpow7yTMNirG2YPFrtQrVjkybNQjCdM/oCI/KVogNSC2Qrj26PRng
         /fQUb65DDaO1vLu2Q/AON5hmXTIwXSLByxdeH8gxPduoHF9w13vanxy4MJ3YyFS/Pqd2
         t0ng==
X-Forwarded-Encrypted: i=1; AJvYcCWZUjKLhCuQ0+ce6Ku6EFBg7T8FzqTEp1eX9Vp/O/HzolbA2dWyDkL0Xj7LApiz8f+WJoadKVRrhugxQ/v9xxSwctMj4seWqHCOOGSuJgg=
X-Gm-Message-State: AOJu0YxLy/z8E/8tUIKW2cOHJ5c8Q7D41qcodb5vNFKw0xxZGGVhA5T9
	dz4d37fq5xzKX4U7Uu6DBhUCjXbopfqQHiaL2vmiNcnOOkeKCYN1zJVRdppAQQ==
X-Google-Smtp-Source: AGHT+IE8k2jkPTWX+k0kZM2Xv3gZbFg0kL2x6CD392XXuVLw4m3xphkdDt46CMG5FaeYKs7Uzix0MA==
X-Received: by 2002:a17:906:7d4:b0:a3e:cc03:d40e with SMTP id m20-20020a17090607d400b00a3ecc03d40emr4222580ejc.63.1708499359398;
        Tue, 20 Feb 2024 23:09:19 -0800 (PST)
Message-ID: <e7c4b08f-724a-4419-977a-caa1342c47eb@suse.com>
Date: Wed, 21 Feb 2024 08:09:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v5 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
Content-Language: en-US
To: Ayan Kumar Halder <ayankuma@amd.com>,
 Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com, julien@xen.org,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, michal.orzel@amd.com,
 xen-devel@lists.xenproject.org
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
 <20240220121743.3680715-2-ayan.kumar.halder@amd.com>
 <e3a8ce5d-841e-4ff1-8d23-48de822972ef@suse.com>
 <5f526deb-7376-4c34-85d1-d22b1b14d90e@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <5f526deb-7376-4c34-85d1-d22b1b14d90e@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 20.02.2024 16:22, Ayan Kumar Halder wrote:
> On 20/02/2024 12:33, Jan Beulich wrote:
>> On 20.02.2024 13:17, Ayan Kumar Halder wrote:
>>> --- a/SUPPORT.md
>>> +++ b/SUPPORT.md
>>> @@ -101,6 +101,18 @@ Extension to the GICv3 interrupt controller to support MSI.
>>>   
>>>       Status: Experimental
>>>   
>>> +### ARM/Partial Emulation
>>> +
>>> +Enable partial emulation of registers, otherwise considered unimplemented,
>>> +that would normally trigger a fault injection.
>>> +
>>> +    Status: Supported, with caveats
>>> +
>>> +Bugs allowing the userspace to attack the guest OS will not be considered
>>> +security vulnerabilities.
>>> +
>>> +Bugs that could compromise Xen will be considered security vulnerabilities.
>> ... the odd statement regarding in-guest vulnerabilities that might be
>> introduced. I see only two ways of treating this as supported: Either
>> you simply refuse emulation when the access is from user space,
> 
> I am wondering how do we enforce that.
> 
> Let me try to understand this with the current implementation of partial 
> emulation for system registers.
> 
> 1. DBGDTRTX_EL0 :- I understand that EL0 access to this register will 
> cause a trap to EL2. The reason being MDCR_EL2.TDA == 1.
> 
> In that case, if we refuse emulation then an undef exception is injected 
> to the guest (this is the behavior as of today even without this patch).
> 
> So, are you saying that the undef exception is to be injected to the 
> user space process. This may be possible for Arm64 guests 
> (inject_undef64_exception() needs to be changed).

No, injection is always to the guest, not to a specific entity within the
guest. That ought to be the same on bare hardware: An exception when
raised has an architecturally defined entry point for handling. That'll
typically be kernel code. The handler then figures out whether the source
of the exception was in user or kernel mode. For user mode code, the
kernel may or may not try to handle the exception and then continue the
user mode process. If it can't or doesn't want to handle it, it'll raise
(in UNIX terms) a signal to the process. That signal, in turn, may or may
not be fatal to the process. But such an exception from user mode should
never be fatal to the guest as a whole.

> However for Arm32 guests, this may not be possible as the mode changes 
> to PSR_MODE_UND.

I'm afraid my Arm foo isn't good enough to understand this. On the surface
it looks to violate above outlined principle.

> Let me know if I understood you correctly.
> 
>> or you
>> support that mode of emulation as much as that of kernel space accesses.
> 
> Do you mean we support partial emulation only for traps from EL1 mode ?

Possibly.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 07:13:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 07:13:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683827.1063450 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgnA-0002sJ-So; Wed, 21 Feb 2024 07:13:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683827.1063450; Wed, 21 Feb 2024 07:13:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgnA-0002sC-QE; Wed, 21 Feb 2024 07:13:20 +0000
Received: by outflank-mailman (input) for mailman id 683827;
 Wed, 21 Feb 2024 07:13:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rcgn9-0002s6-9k
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 07:13:19 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id afcfad74-d088-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 08:13:17 +0100 (CET)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2d10ad265d5so68889351fa.0
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 23:13:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: afcfad74-d088-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708499597; x=1709104397; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=5nr74LCFagsjWslOgZf57SG/Js4h6QFS74f9G6WYs40=;
        b=KuTZT+o/YqISIpt3aZkXiyDcL1b5zz1pQEZNf0dewnlHg6Y45itj8AnOsLGveNNdxK
         WgkKWpUFJr3FjF5XdaGXluI5Qx9Sa28/wTY+vV30kfP14yJTt2qWzPXgQUJqyYN2nnW5
         BBuTdNcweihXa7auqusfnRe2pBlDepartXJAU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708499597; x=1709104397;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=5nr74LCFagsjWslOgZf57SG/Js4h6QFS74f9G6WYs40=;
        b=tXXV4uxi7DuDqsc19E/AgeHJLU9YFGUYaWek+EbmEO7aYcPzqFuNELIdbpbnTbyc0z
         HdnzT8TtBhbnoZIERnn+YC7WU5j0eYTCPjrd0uoBP7Evk80bMQUvc5JFcSwQ207hYCfy
         3bPqgPkTFYKLn1+PMCsnfjoc+GaWPeHfKAHwKCkwrnf5fV5gtPlHqpjWHZNKmI2/WIc3
         ksN3hrkEWAJ3dsmzqLZWcBT6zNMP6Op7bIDW8SxSi9+y82lKNLoedPrVbc/gD3SL8gmm
         COjQ57hbZdlgX1d2kdOIDeY2W3tVjdJmVEfid9Z5RLjjDj3Q345W/qaBjPeVg+CIz2Tr
         45lg==
X-Forwarded-Encrypted: i=1; AJvYcCUWyR/AaOkKjKrPyG++IQcKEJaCZ3D7CVUvXE6/eh3IlSkKMlTaFy38Ys5dbeL1BcVJeB8DOARyNXQU6mzULy/AD314YGpFb2WN4HdIrUc=
X-Gm-Message-State: AOJu0YxAxFTpeh7kTYzxJoQbgFn/z97POA41qSjICbUvWjLTLFr/pr6p
	j8pwKCHTQ2s3QNCRCFl6sEk7NsXcZJIESw85ppWHMbIJN4kWXrBNK+r3hvOizOjTyBg0dIrFQxX
	08VFMztyiZ1o0pX1nDGQEnypKfznjCgSOmNNFMA==
X-Google-Smtp-Source: AGHT+IH8mrsxu8WW8+MHevP+GQ8WvqFc+p76S/FGwgq4Y9u3yTcAFgsBPbJalxMxlxYPc51l1HxB8VfS7BVWFLmD5GY=
X-Received: by 2002:a2e:3e15:0:b0:2d2:50bc:99d6 with SMTP id
 l21-20020a2e3e15000000b002d250bc99d6mr1678949lja.35.1708499597544; Tue, 20
 Feb 2024 23:13:17 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-3-george.dunlap@cloud.com> <14f09e7a-76b4-4650-9f90-90e7709379be@suse.com>
In-Reply-To: <14f09e7a-76b4-4650-9f90-90e7709379be@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 15:13:06 +0800
Message-ID: <CA+zSX=aARhiXDSVFXUvQOuNf-jNNcq3XDakZUcshj6z5UN9HCQ@mail.gmail.com>
Subject: Re: [PATCH 2/6] svm: Improve type of cpu_has_svm_feature
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 19, 2024 at 11:24=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wr=
ote:
>
> On 06.02.2024 02:20, George Dunlap wrote:
> > --- a/xen/arch/x86/include/asm/hvm/svm/svm.h
> > +++ b/xen/arch/x86/include/asm/hvm/svm/svm.h
> > @@ -38,7 +38,10 @@ extern u32 svm_feature_flags;
> >  #define SVM_FEATURE_SSS           19 /* NPT Supervisor Shadow Stacks *=
/
> >  #define SVM_FEATURE_SPEC_CTRL     20 /* MSR_SPEC_CTRL virtualisation *=
/
> >
> > -#define cpu_has_svm_feature(f) (svm_feature_flags & (1u << (f)))
> > +static inline bool cpu_has_svm_feature(unsigned int feat)
> > +{
> > +    return svm_feature_flags & (1u << (feat));
> > +}
> >  #define cpu_has_svm_npt       cpu_has_svm_feature(SVM_FEATURE_NPT)
> >  #define cpu_has_svm_lbrv      cpu_has_svm_feature(SVM_FEATURE_LBRV)
> >  #define cpu_has_svm_svml      cpu_has_svm_feature(SVM_FEATURE_SVML)
>
> Having seen patch 4 now, I have to raise the question here as well: Why
> do we need a separate variable (svm_feature_flags) when we could use
> the host policy (provided it isn't abused; see comments on patch 4)?

We certainly don't need an extra variable; in fact, moving all of
these into the host cpuid policy thing would make it easier, for
example, to test some of the guest creation restrictions: One could
use the Xen command-line parameters to disable some of the bits, then
try to create a nested SVM guest and verify that it fails as expected.

I would like to do that eventually, but this patch is already done and
improves the code, so I just kept it.

Let me know if you'd like me to simply drop this patch instead.

 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 07:17:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 07:17:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683832.1063461 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgrB-0003Qi-FA; Wed, 21 Feb 2024 07:17:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683832.1063461; Wed, 21 Feb 2024 07:17:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgrB-0003Qb-BF; Wed, 21 Feb 2024 07:17:29 +0000
Received: by outflank-mailman (input) for mailman id 683832;
 Wed, 21 Feb 2024 07:17:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TA2L=J6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcgr9-0003QV-Cq
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 07:17:27 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 431fc0da-d089-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 08:17:25 +0100 (CET)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-563c2b2bddbso8966545a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 23:17:25 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 vx10-20020a170907a78a00b00a3ee2530a11sm1668395ejc.194.2024.02.20.23.17.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 23:17:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 431fc0da-d089-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708499844; x=1709104644; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=VXykqclW9twT/8BX1EFASN1yE6LphN1JKGbGziHqQUM=;
        b=Pw2tPulGzRRzjgq7aZGYNMLNOQ04bGc1BiZaY6DnOMbo43JokYL/IpVvlYic4OEVhm
         eEaoZvEMYF9u++Bjt7eiN9kJ/9jBYjuqpOglgGe9DzIzoV5otNVpNCxxv3JSg8zC3rse
         Eaat8gqPqyVGaB3Z6b94sA9oCu0fwfs1qYT1Y1XD7e2CqrDr6yeOHiakbGUtPPnFRonw
         alOzAtYeE+D5kgMwWtsebXNodFjDAxOKRU4ifZ1V10dFj6OToP8ZmWmfyY9rFJqJrP9M
         FNOT1rGLixxs6ZqvUcWAHlrXBGbwHtlgIRwKYrveAObCvkv6wydIMraY3j5pDYSyxsGR
         //7A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708499844; x=1709104644;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=VXykqclW9twT/8BX1EFASN1yE6LphN1JKGbGziHqQUM=;
        b=XmlsEf+fq0Bh/nnkpsCvY4OFDjGcPdgYNcvAAkGW+g5wjOmc9kCC/rzRHxmrbeoNDW
         G5aEWEv2QrnITYXfX6QnTAW/Rrw5MOuPz/P55iYu8Mk2GbXA946WGbOytBqLwmBM/MQh
         2wgvsGAVRwZ6xfHLEPYUZaokqI4jvfDd1kuucAAZG7/NnFGYhFmS9KeRVmwEt3waAgW9
         5QEm2Nv6wkCz0uxEBJXR57Uf7a95X5LIlyW4Fl/+FnHqE1d/pp9KP+Zs4MFj9nnl7+ty
         IdRgMpKUb+sZ9lMz2PV7VTmJwc37NLA5Smbg9vkJwTKdwJeRSDnC7+oFvA7xLjyxFQdH
         q9SQ==
X-Forwarded-Encrypted: i=1; AJvYcCUS5aMPFtZvKQ51N+QB5OJn2CxlkUF2V0IHmcGKn6pcJuARvidUrkviVCbmNkC2Ozu8vDwyWYKJq91vpOjhShR5P5ZzN6DwrRZkjPMVgbU=
X-Gm-Message-State: AOJu0Yyu0WrqASfTdqzEWRFq4N6rBUAIg44qiwJRxlvRselsKWqRpDJR
	pV86cdV2IAKtPyPMBdWywnRFcyMjkdHsPOUGkMg2MO4mPt9U9VQTuaDLD6Hhow==
X-Google-Smtp-Source: AGHT+IHwtgUzYUW93f0pcwXvZEdUJBAeYMjxu07b6Q62pr+l7b+bajVUxkmnpeDRswWy7om/r/ECqQ==
X-Received: by 2002:a17:906:7d46:b0:a3e:4553:ba31 with SMTP id l6-20020a1709067d4600b00a3e4553ba31mr6374470ejp.28.1708499844685;
        Tue, 20 Feb 2024 23:17:24 -0800 (PST)
Message-ID: <5a28331f-663f-421f-9845-45ff5eb1810c@suse.com>
Date: Wed, 21 Feb 2024 08:17:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] almost fully ignore zero-size flush requests
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Julien Grall <julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
 <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com>
 <7dd78c97-b82f-4ae1-a257-98f612c6bad2@xen.org>
 <6466155e-9883-4743-a897-ef565f14fdf0@suse.com>
 <CA+zSX=aJ2mVbGzFj74+DEGZvw=XWTZKAeWfrwMNHaYhsdaytHA@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CA+zSX=aJ2mVbGzFj74+DEGZvw=XWTZKAeWfrwMNHaYhsdaytHA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.02.2024 03:32, George Dunlap wrote:
> On Tue, Feb 20, 2024 at 4:26 PM Jan Beulich <jbeulich@suse.com> wrote:
>>>> +    if ( (cflush->offset | cflush->length) > PAGE_SIZE ||
>>>
>>> This is confusing. I understand you are trying to force the compiler to
>>> optimize. But is it really worth it? After all, the rest of operation
>>> will outweight this check (cache flush are quite expensive).
>>
>> From purely a performance point of view it may not be worth it. From
>> code size angle (taken globally) I already view this differently.
>> Plus I think that we ought to aim at avoiding undesirable patterns,
>> just because people tend to clone existing code when they can. Thing
>> is that (as per below) the two of us apparently disagree on what
>> "undesirable" is in cases like this one.
>>
>>> We probably should take a more generic decision (and encode in our
>>> policy) because you seem to like this pattern and I dislike it :). Not
>>> sure what the others think.
> 
> This is similar to the policy question I raised among the x86
> committers a few weeks ago: You're manually specifying a more specific
> behavior than is required, rather than specifying what you want and
> then letting the compiler optimize things.  The problem with this is
> twofold:
> 
> 1. It's harder for humans to read and understand the intent

Depends.

> 2. It ties the compiler's hands.  If you write your intent, then the
> compiler is free to apply the optimization or not, or apply a
> different optimization.  If you specify this optimization, then the
> compiler has fewer ways that it's allowed to compile the code.

I'm inclined to believe that no compiler will do this kind of
optimization, unless a specific request was raised against it. The
pattern may not seem overly complex, but to recognize it would
require effort that on the whole may simply not be justified by
the gains (from the compiler's perspective).

> #1 by itself is probably enough to counterindicate this kind of
> behavior.  Add them together, and I'm inclined to say that we should
> write a policy against such optimizations, without specific
> justifications.

It's not like I didn't give any justification. So I guess you mean
without better (whatever that means) justification.

But yes, I'll undo that part of the change then and submit a v2,
albeit with not overly much hope for it to then be accepted.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 07:21:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 07:21:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683845.1063470 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgux-0005L0-1Z; Wed, 21 Feb 2024 07:21:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683845.1063470; Wed, 21 Feb 2024 07:21:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcguw-0005Kt-Ul; Wed, 21 Feb 2024 07:21:22 +0000
Received: by outflank-mailman (input) for mailman id 683845;
 Wed, 21 Feb 2024 07:21:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rcguv-0005KU-Sy
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 07:21:21 +0000
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com
 [2a00:1450:4864:20::22f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cefe04df-d089-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 08:21:19 +0100 (CET)
Received: by mail-lj1-x22f.google.com with SMTP id
 38308e7fff4ca-2d094bc2244so87009281fa.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 23:21:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cefe04df-d089-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708500079; x=1709104879; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ZNI40zGYNZp2oHOueSeNqiaK3+ljK33pV5I9xmzz7XY=;
        b=D9teRyOQJbZPvf70TNTAx2hvrHywfwHPkS6mk1qRJKb4hMys+uzsF7z9E3wV9p6OgK
         1kqYLU/OZrOU61Scplw3yf88a8ZQX5LGleI9V29pQHcltFnkc7KThwWpY0UPAYy+IjCi
         SFYI7peyQ6OSrdbF5+hRLFqo2wlXKSE14+HTs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708500079; x=1709104879;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ZNI40zGYNZp2oHOueSeNqiaK3+ljK33pV5I9xmzz7XY=;
        b=CGcyfbHCnt0HJZgY6LsdbyQtkXszsgS+AEKf5vEfmsh54ERF2NCLLgWhocjmyNytph
         nBet87ZGztbJefYu9usEIDNQgAbW1oeIkUp03iGjJ0mvgbbCm7tEUJRpXAzLXdpDqdkp
         nQXCHwKx099D6k81Ph+SOHUP2CpGiTciCxv/u3opBAC9/Lpsm+39RMZgg8FiOmnGt+XO
         m7JS5I6ESHDt0d6MpDgbTB7R75eGhSOKb5+SkBK34+dKhihaPPJneK0PAr2f1EMmtO7e
         dHOXltgquXQMhrqZ6SiX7lTGzbPd/q/qqnIu3NMmaIHor8jDNS+ofToCYySfa13Kf5U1
         Ltzg==
X-Forwarded-Encrypted: i=1; AJvYcCWZrWMd9QIIBinmLM1WygnzOpMj9DA1VLSDsC5ZAL3+m6hlbxg2Pl+mlYawBb7HPmfaUXBgIbnT1291Cuf1cUb4oiZiD6A6HcEDUf/MiHA=
X-Gm-Message-State: AOJu0YzGfKC06JJTYPYC/h9fR2IbJBA3+ilxqopWFCd8AwarpQw+ZDuc
	KgXw09z2vBhyxhn+8yn3TraenJX0CLEyHPBfchQzbvdXeOSh0GaBjTLPsOy6EklJZ6nT4Ylujte
	i7YQ8j8tEPPHMQRvvB/BjCS/LBTma+Md+wEs0Zw==
X-Google-Smtp-Source: AGHT+IHCNbYIYiR9zMt9JbV6+ZAeTnQxtMRbzmNKD8EWYTSR+RhNij9svb020PfvWJD+Z6gamzvlxnlGL+JYtqvBGhY=
X-Received: by 2002:a2e:880d:0:b0:2d2:4017:d79f with SMTP id
 x13-20020a2e880d000000b002d24017d79fmr4540584ljh.37.1708500079293; Tue, 20
 Feb 2024 23:21:19 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-4-george.dunlap@cloud.com> <046575d2-bc39-4b0e-a656-c501ee1ae4e6@suse.com>
In-Reply-To: <046575d2-bc39-4b0e-a656-c501ee1ae4e6@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 15:21:08 +0800
Message-ID: <CA+zSX=aNL30KP_sCC38DJ0bS=8f2e-i17SuUfkXKzP4b6wx0tA@mail.gmail.com>
Subject: Re: [PATCH 3/6] xen/hvm: Move other hvm_function_table booleans into
 the caps bitfield
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 20, 2024 at 12:08=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wr=
ote:
>
> On 06.02.2024 02:20, George Dunlap wrote:
> > --- a/xen/arch/x86/include/asm/hvm/hvm.h
> > +++ b/xen/arch/x86/include/asm/hvm/hvm.h
> > @@ -86,20 +86,19 @@ struct hvm_vcpu_nonreg_state {
> >  struct hvm_function_table {
> >      const char *name;
> >
> > -    /* Support Hardware-Assisted Paging? */
> > -    bool hap_supported;
> > -
> > -    /* Necessary hardware support for alternate p2m's? */
> > -    bool altp2m_supported;
> > -    bool singlestep_supported;
> > -
> > -    /* Hardware virtual interrupt delivery enable? */
> > -    bool virtual_intr_delivery_enabled;
> > -
> >      struct {
> >          /* Indicate HAP capabilities. */
> > -        bool hap_superpage_1gb:1,
> > -            hap_superpage_2mb:1;
> > +        bool hap:1,
> > +             hap_superpage_1gb:1,
> > +             hap_superpage_2mb:1,
> > +
> > +            /* Altp2m capabilities */
> > +            altp2m:1,
> > +            singlestep:1,
> > +
> > +            /* Hardware virtual interrupt delivery enable? */
> > +            virtual_intr_delivery;
> > +
> >      } caps;
>
> Nit (spotted only while looking at patch 6): You're adding a stray blank
> line at the end of the structure. Further I expect virtual_intr_delivery
> would also want to be just a single bit?

Oh yes, good catch.  (I kind of feel like ":1" should be the default
for bools, but hey...)

I'll fix up this and the 0/1 =3D> false/true thing.

 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 07:23:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 07:23:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683850.1063481 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgww-0005tC-Cs; Wed, 21 Feb 2024 07:23:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683850.1063481; Wed, 21 Feb 2024 07:23:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcgww-0005t5-A8; Wed, 21 Feb 2024 07:23:26 +0000
Received: by outflank-mailman (input) for mailman id 683850;
 Wed, 21 Feb 2024 07:23:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TA2L=J6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcgwv-0005sx-7R
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 07:23:25 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 19142fc8-d08a-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 08:23:24 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a3e8be8a7ddso369668766b.2
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 23:23:24 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h11-20020a17090634cb00b00a3d7bcfb9a1sm4685554ejb.128.2024.02.20.23.23.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 20 Feb 2024 23:23:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 19142fc8-d08a-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708500203; x=1709105003; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xhERbpoE4VXwROCKxrt3FxWiUBFTqvpBx+V/ORx+9Ys=;
        b=Pd82sxdVc6MHqAdk06Nl24YtqbMsx9BNwbca3O+/QcQxV9Dtm5ftjJYTkcja0ZSFYx
         g5Mi4FN4cevHj/MI8FmaO0onhADsD28YLXnWwtyGZpV++Tq5kn6NlPKtI3kpuqPNFXhK
         yk52mWe1NHwF9RQ2xfKCVVVbFseXrhXwY6HtkVTZn9XeGQa7nLY9CpmTC7uCHHG5GeZJ
         rTXkL/rnrzKadt+rPyLrdGsD1IMKFWjAZazm4nrnSHl4n72PREJOsImsUmSmnmq2DW0P
         ADc0ATvg98ISaUfp1aXNXUJ2WXKNbUMKJk0+brChAjrZmMh3Fx+iCowW+Z/XHltPa2Nf
         ghJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708500203; x=1709105003;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xhERbpoE4VXwROCKxrt3FxWiUBFTqvpBx+V/ORx+9Ys=;
        b=E5LCRkrw5r3K/hpm2h+CKchrVdj2L8f5fn7EwzURw4MdJJ1RMn52VLulgfo98IBZm/
         dc4H+8e4ovU+PWEpZLeiJQ12kwHVWem/Io2REBX0/3vUp1wqrlroXu+i2X+K6rNPMsuX
         nXvC+wfwFS2tdNyNsr/1pt+/84F6cliU9Y1wlqiwk/0NGefJqSlmg4g1CzCckX02olKv
         YBVCIm9SpVabpQN7tWDLIQJmIPZ65wLTYr+RixCMlvAsmSNqd+/r9wwQKfJg6zkT07Oc
         tHK9oh/+Ug42N8xnFshAciY6hDSDooe7MWwMuKoYcORdVxaPYmK92qYInjTUh/u+S9yI
         eBdw==
X-Forwarded-Encrypted: i=1; AJvYcCWFdJIWmZ1Hpw5qv20rKWqdKHnq3ijSsDcHdoj2RkaotDJcjzxPDhIZjWA96NgEsTmcZrbW4m9Tl4u55jEf0o6vcA6s0JqZ0kObA1OB3fo=
X-Gm-Message-State: AOJu0YyfpOliJEnvO0deRplnJVvN0iIvrOD6Kf6Y9qWRdINneszkfYhU
	+B49MdosMMYpDaycnWfgsrcs22WsjN6xGQqzrgL65RlSbHsQ8XrLnk+WEN8F1w==
X-Google-Smtp-Source: AGHT+IHa7NVEZplEZ3l18DhGyDV51nw+mgUNhbkMoTobZvdJOC76JwEjs0du3eMCGQuvan/wb5F/ag==
X-Received: by 2002:a17:906:d043:b0:a3d:a63f:2db0 with SMTP id bo3-20020a170906d04300b00a3da63f2db0mr11037432ejb.28.1708500203681;
        Tue, 20 Feb 2024 23:23:23 -0800 (PST)
Message-ID: <088414f7-c5f3-4776-80a2-c9c3da618c22@suse.com>
Date: Wed, 21 Feb 2024 08:23:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/6] xen/hvm: Convert hap_capabilities into a bitfield
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Jun Nakajima <jun.nakajima@intel.com>,
 Kevin Tian <kevin.tian@intel.com>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-2-george.dunlap@cloud.com>
 <66962dfc-db89-40bc-8411-9b92f9458b16@suse.com>
 <CA+zSX=ZwUT4Wf7eY3DFyp=Bf_MndExoKCZBVGEhF-ByaOEH7tw@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CA+zSX=ZwUT4Wf7eY3DFyp=Bf_MndExoKCZBVGEhF-ByaOEH7tw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.02.2024 08:02, George Dunlap wrote:
> On Mon, Feb 19, 2024 at 9:36 PM Jan Beulich <jbeulich@suse.com> wrote:
>> On 06.02.2024 02:20, George Dunlap wrote:
>>> --- a/xen/arch/x86/hvm/vmx/vmcs.c
>>> +++ b/xen/arch/x86/hvm/vmx/vmcs.c
>>> @@ -113,8 +113,8 @@ static int cf_check parse_ept_param_runtime(const char *s)
>>>      int val;
>>>
>>>      if ( !cpu_has_vmx_ept || !hvm_funcs.hap_supported ||
>>> -         !(hvm_funcs.hap_capabilities &
>>> -           (HVM_HAP_SUPERPAGE_2MB | HVM_HAP_SUPERPAGE_1GB)) )
>>> +         !(hvm_funcs.caps.hap_superpage_2mb ||
>>> +           hvm_funcs.caps.hap_superpage_1gb) )
>>>      {
>>>          printk("VMX: EPT not available, or not in use - ignoring\n");
>>
>> Just to mention it: The conditional and the log message don't really
>> fit together. (I was first wondering what the 2mb/1gb checks had to
>> do here at all, but that's immediately clear when seeing that the
>> only sub-option here is "exec-sp".)
> 
> So you mean basically that the checks & error message are poorly
> factored, because there's only a single sub-option?  (i.e., if there
> were options which didn't rely on superpages, the check would be
> incorrect?)

Right.

> Let me know if there's something concrete you'd like me to do here.

Nothing. I meant to express this by starting with "Just to mention it".
If you were eager, you might deal with this right away, but there's no
expectation to this effect. If and when it becomes a problem, it'll
then need sorting. And the message being potentially misleading is,
well, just an annoyance - as with many other things, finding it confusing
will likely lead to looking up where it's issued, then realizing that
what it says isn't what it means.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 07:38:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 07:38:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683863.1063491 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rchBX-0007vb-P2; Wed, 21 Feb 2024 07:38:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683863.1063491; Wed, 21 Feb 2024 07:38:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rchBX-0007vU-M4; Wed, 21 Feb 2024 07:38:31 +0000
Received: by outflank-mailman (input) for mailman id 683863;
 Wed, 21 Feb 2024 07:38:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rchBW-0007vO-0z
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 07:38:30 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 33debf08-d08c-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 08:38:28 +0100 (CET)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2d0cd9871b3so67191271fa.1
 for <xen-devel@lists.xenproject.org>; Tue, 20 Feb 2024 23:38:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 33debf08-d08c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708501107; x=1709105907; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=kM3OowUkSdEyn+kO36RUprrqXgAkORPjN1L25sxdVgQ=;
        b=DtXxjSoNaVzxYlwzpcFwMc+LwO+9uTbzULJcdHa1LwQZsBC14DgznaBoSNsRhxvRup
         p286PDJoUZSI+X6qzFTqkg/A5WheHtAfhcOjzrscItHVl8oDaEDJMRyWjYWvhkyJDWot
         3vMAr0tSIsD7fmr/En+1xqjZ3NYI0l+wn0CUw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708501107; x=1709105907;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=kM3OowUkSdEyn+kO36RUprrqXgAkORPjN1L25sxdVgQ=;
        b=oX5yk1aFqlNwu6ZU0DlomEZydKvlccZIU5p9fcqAnTRsGnygJcYCy1vjJcZATEqWMO
         BsNhKKJgHKj5JrhfuKff9KawOgPtwbbnWMsqBDiw6eZyT0DQBc4APDniv0yKgKMerv57
         4mdug/zH/BE/VdZhNAeqN7omAUgLG3VI5X2rgxG8x0vvbw7yLSeY5WvkH0S1BHWy1Eg+
         1qnovoPgWAwiUlZrMNYhpdGgOLvt/6+chYdw0/fbufUpng5U6bDkApNcvBYiLlN37U/S
         lXVNGdNF68EsFRjevJcR/R3eD/eD9M28tgp3c07WSggIXvAZ/J+eYuYPbAdT2Jy5UmHB
         cd+Q==
X-Forwarded-Encrypted: i=1; AJvYcCUnATFbZEZsyANODgGq2vc/4jcFwZLF704KXIH4lhT5Hjui3YqRKjiAujv82pS4K6fj4zB5pi7ITmhmoIDidwFPEVa4V43IxJ9X3LnA/Mw=
X-Gm-Message-State: AOJu0Yy2V37Nu1N5Rige9kDTWyxsK2LzunECR0HT7jwOydNB1kLsRLES
	d8cQ2poiDknO+LtoJv9FIMYl8Hn4Ge1eAMtyy1Vl99XX7diP2KEuhuEV6DW0YIPXuLjuvGl9qcw
	ovxwFQC4/fK9yJTdwcT/UwgoKqb4tLeGE9sUsuQ==
X-Google-Smtp-Source: AGHT+IEx2n75EVJP3+E0tFds5KhS6fxBn1YLAIf2HQJDYNZr+XZO6vD5YYeLVF3hd+WX4TE4EqmPzAaxXlh6u7uSEMA=
X-Received: by 2002:a2e:9d10:0:b0:2d0:af40:7058 with SMTP id
 t16-20020a2e9d10000000b002d0af407058mr4938455lji.14.1708501107532; Tue, 20
 Feb 2024 23:38:27 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-2-george.dunlap@cloud.com> <66962dfc-db89-40bc-8411-9b92f9458b16@suse.com>
 <CA+zSX=ZwUT4Wf7eY3DFyp=Bf_MndExoKCZBVGEhF-ByaOEH7tw@mail.gmail.com> <088414f7-c5f3-4776-80a2-c9c3da618c22@suse.com>
In-Reply-To: <088414f7-c5f3-4776-80a2-c9c3da618c22@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 15:38:16 +0800
Message-ID: <CA+zSX=a5-Bfp==DESE-2UZdt6yCfSxRxCf65PotsJ2Emrg2GCQ@mail.gmail.com>
Subject: Re: [PATCH 1/6] xen/hvm: Convert hap_capabilities into a bitfield
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 21, 2024 at 3:23=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 21.02.2024 08:02, George Dunlap wrote:
> > On Mon, Feb 19, 2024 at 9:36=E2=80=AFPM Jan Beulich <jbeulich@suse.com>=
 wrote:
> >> On 06.02.2024 02:20, George Dunlap wrote:
> >>> --- a/xen/arch/x86/hvm/vmx/vmcs.c
> >>> +++ b/xen/arch/x86/hvm/vmx/vmcs.c
> >>> @@ -113,8 +113,8 @@ static int cf_check parse_ept_param_runtime(const=
 char *s)
> >>>      int val;
> >>>
> >>>      if ( !cpu_has_vmx_ept || !hvm_funcs.hap_supported ||
> >>> -         !(hvm_funcs.hap_capabilities &
> >>> -           (HVM_HAP_SUPERPAGE_2MB | HVM_HAP_SUPERPAGE_1GB)) )
> >>> +         !(hvm_funcs.caps.hap_superpage_2mb ||
> >>> +           hvm_funcs.caps.hap_superpage_1gb) )
> >>>      {
> >>>          printk("VMX: EPT not available, or not in use - ignoring\n")=
;
> >>
> >> Just to mention it: The conditional and the log message don't really
> >> fit together. (I was first wondering what the 2mb/1gb checks had to
> >> do here at all, but that's immediately clear when seeing that the
> >> only sub-option here is "exec-sp".)
> >
> > So you mean basically that the checks & error message are poorly
> > factored, because there's only a single sub-option?  (i.e., if there
> > were options which didn't rely on superpages, the check would be
> > incorrect?)
>
> Right.
>
> > Let me know if there's something concrete you'd like me to do here.
>
> Nothing. I meant to express this by starting with "Just to mention it".

Right, and when I said "let me know" I meant, "I'm going to ignore
this unless you say something, feel free to say nothing". :-D

I understood that you weren't asking for anything, but maybe coming
back to this after a few days you'd've had a simple fix.  I wouldn't
mind changing the text of the message, but I didn't feel like finding
a better text.  Reorganizing the checks (which seems closer to the
Right Thing) is off-topic for this patch of course.

 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 08:48:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 08:48:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683880.1063500 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rciHQ-0001Zx-5E; Wed, 21 Feb 2024 08:48:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683880.1063500; Wed, 21 Feb 2024 08:48:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rciHQ-0001Zq-2K; Wed, 21 Feb 2024 08:48:40 +0000
Received: by outflank-mailman (input) for mailman id 683880;
 Wed, 21 Feb 2024 08:48:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rciHO-0001Zk-Bl
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 08:48:38 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 00e43bc8-d096-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 09:48:37 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-512b13bf764so3934029e87.0
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 00:48:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00e43bc8-d096-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708505317; x=1709110117; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SqkHSlulIprfbeRURSygRzcZQTs86nWaFlFZb9d2gL4=;
        b=Am9FAffZKV6+ZhA54AkCQ7F58QhiM+L0nQtXVGVcYYDSg6UHKwZFgBT6UIvyGJUWJd
         FxxALCCijgm2oc8x9hPq8CvA7nbH5jLFNNl/uTi/I/yayVWkupaQoAqRKDgDd8vHlvRh
         KQlBDd7fOJ9dP4MyUXooSnef55+8pCSdMumBQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708505317; x=1709110117;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=SqkHSlulIprfbeRURSygRzcZQTs86nWaFlFZb9d2gL4=;
        b=QW6tk8+mY9en4Fh/qDZCQeUUQ3qHmBVsIPSZUEMPNObkhYQIdgeLtOMHvO9WlFaYZL
         FOS/UQU/3AHJ/g8E2XUe9HeQ+EwK9BWit7sxjuAcMCP0CzD3HWUVPw5C8IsuEPoklerQ
         u9s6If0C81piX/MbddsLdU9EX1uq2l5ZL4rsuV/MiOYH20ZnYkV3zdsBmwZOFsQ/4qFb
         jXwRi1ch9F/v6CEkuIe6tTj1YJN+V1DaPn552C7ZjgzTt14iPUo7VOovE7XoVfCvhgE7
         2FUVfSzAqdWkCw20izClOh1irjsB3i0Nm0niwQn37wSXIXSu17ylwwFNHtOsGTEE/nVO
         XGgg==
X-Forwarded-Encrypted: i=1; AJvYcCXoSctGKfYqgsmGRDlD02GtdbBVipMoljFdRuL52zVrFj+IF/LH4EipUBufJ1PWniVjV3wzW5b8bsKFlY5Bbp6Uu3bdC6J45btsUKpv9L4=
X-Gm-Message-State: AOJu0Yw6Rje8A3uo4pL3OrDnX64da9kYBsUYrQFDArFnIvXh4qVmtYPm
	6h+7NeZHijfsmcvP4MrGL9ueqTsr1LfQapHm1PsYZtOr0wvjOxGRx+QE535eC6WAyLWFIPZv761
	qQ+tsuAkwk27Zh6J2X4lInuzwziRNQ0lCiq1khg==
X-Google-Smtp-Source: AGHT+IEZP5lkOhf+MRS12LtEfR6Tc68lk2YllWkU9FD+bDklc3PcPKYDsBSdYTYmS24BkzbKuCTCz0WgJEYQUSBwH2I=
X-Received: by 2002:a05:6512:ba6:b0:512:be4d:36f1 with SMTP id
 b38-20020a0565120ba600b00512be4d36f1mr7090163lfv.21.1708505316863; Wed, 21
 Feb 2024 00:48:36 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-5-george.dunlap@cloud.com> <c5f3a78c-4a2b-499b-93bb-11f60b0afcc0@suse.com>
In-Reply-To: <c5f3a78c-4a2b-499b-93bb-11f60b0afcc0@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 16:48:25 +0800
Message-ID: <CA+zSX=acH-4B01=vUBstAbsNBMVgcRe_A5v0eXn66BdsuPtgiA@mail.gmail.com>
Subject: Re: [PATCH 4/6] nestedsvm: Disable TscRateMSR
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 19, 2024 at 11:22=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wr=
ote:
>
> On 06.02.2024 02:20, George Dunlap wrote:
> > For now, just disable the functionality entirely until we can
> > implement it properly:
> >
> > - Don't set TSCRATEMSR in the host CPUID policy
>
> This goes too far: This way you would (in principle) also affect guests
> with nesting disabled. According to the earlier parts of the description
> there's also no issue with it in that case. What you want to make sure
> it that in the HVM policy the bit isn't set.
>
> While presently resolving to cpu_has_svm_feature(), I think
> cpu_has_tsc_ratio really ought to resolve to the host policy field.
> Of course then requiring the host policy to reflect reality rather than
> having what is "always emulated". IOW ...
>
> > --- a/xen/arch/x86/cpu-policy.c
> > +++ b/xen/arch/x86/cpu-policy.c
> > @@ -407,8 +407,7 @@ static void __init calculate_host_policy(void)
> >                                 (1u << SVM_FEATURE_PAUSEFILTER) |
> >                                 (1u << SVM_FEATURE_DECODEASSISTS));
> >          /* Enable features which are always emulated. */
> > -        p->extd.raw[0xa].d |=3D ((1u << SVM_FEATURE_VMCBCLEAN) |
> > -                               (1u << SVM_FEATURE_TSCRATEMSR));
> > +        p->extd.raw[0xa].d |=3D (1u << SVM_FEATURE_VMCBCLEAN);
>
> ... this likely wants replacing altogether by not overriding what we
> found in hardware, which would apparently mean moving the two bit
> masks to the earlier "clamping" expression.
>
> But then of course Andrew may know of reasons why all of this is done
> in calculate_host_policy() in the first place, rather than in HVM
> policy calculation.

It sounds like maybe you're confusing host_policy with
x86_capabilities?  From what I can tell:

*  the "basic" cpu_has_X macros resolve to boot_cpu_has(), which
resolves to cpu_has(&boot_cpu_data, ...), which is completely
independent of the cpu-policy.c:host_cpu_policy

* cpu-policy.c:host_cpu_policy only affects what is advertised to
guests, via {pv,hvm}_cpu_policy and featureset bits.  Most notably a
quick skim doesn't show any mechanism by which host_cpu_policy could
affect what features Xen itself decides to use.

Not sure exactly why the nested virt stuff is done at the
host_cpu_policy level rather than the hvm_cpu_policy level, but since
that's where it is, that's where we need to change it.

FWIW, as I said in response to your comment on 2/6, it would be nicer
if we moved svm_feature_flags into the "capabilities" section; but
that's a different set of work.

 -George


 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 09:14:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 09:14:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683898.1063516 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcige-0005cM-De; Wed, 21 Feb 2024 09:14:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683898.1063516; Wed, 21 Feb 2024 09:14:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcige-0005cF-7q; Wed, 21 Feb 2024 09:14:44 +0000
Received: by outflank-mailman (input) for mailman id 683898;
 Wed, 21 Feb 2024 09:14:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcigd-0005c3-4g; Wed, 21 Feb 2024 09:14:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcigd-0006xh-1Y; Wed, 21 Feb 2024 09:14:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcigc-0004xx-Hk; Wed, 21 Feb 2024 09:14:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcigc-0002n2-Gq; Wed, 21 Feb 2024 09:14:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=whYFy89Z+6+WC8hfupAljqAD6/QfG3MVyZwTtp+3lLE=; b=A8CYfdIDbI6whXCQmGyl8c7AqV
	fJALhHdORI10OzIqovI0iDsSBEGFl/7kESsiRYlSMXpAb+u6Pc1xPwPaPPPZSid7G8kCQpFaetTaG
	tioFWRo7uzQaF93LYl0mvZpgDoFUQcj4xw34Go+9KZfaiKSgtNWP9sOPzlt0MG6Li97s=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184716-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184716: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f8791d0fd3adbda3701e7eb9db63a9351b478365
X-Osstest-Versions-That:
    xen=981f722efa876e3f1766629f444d10e8806adc93
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 21 Feb 2024 09:14:42 +0000

flight 184716 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184716/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184712
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184712
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184712
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184712
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184712
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184712
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184712
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184712
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184712
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184712
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184712
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184712
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  f8791d0fd3adbda3701e7eb9db63a9351b478365
baseline version:
 xen                  981f722efa876e3f1766629f444d10e8806adc93

Last test of basis   184712  2024-02-20 14:39:53 Z    0 days
Testing same since   184716  2024-02-20 23:39:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Julien Grall <jgrall@amazon.com>
  Luca Fancellu <luca.fancellu@arm.com>
  Michal Orzel <michal.orzel@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   981f722efa..f8791d0fd3  f8791d0fd3adbda3701e7eb9db63a9351b478365 -> master


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 09:17:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 09:17:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683905.1063525 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcij9-0006DN-Ue; Wed, 21 Feb 2024 09:17:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683905.1063525; Wed, 21 Feb 2024 09:17:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcij9-0006DG-Qt; Wed, 21 Feb 2024 09:17:19 +0000
Received: by outflank-mailman (input) for mailman id 683905;
 Wed, 21 Feb 2024 09:17:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CeSM=J6=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rcij9-0006DA-7f
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 09:17:19 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 01fdcb00-d09a-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 10:17:17 +0100 (CET)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2d22b8c6e0dso50571991fa.2
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 01:17:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01fdcb00-d09a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708507037; x=1709111837; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=rsQ0Vfl6ZhEilW3byA6G/qo88FKVThxA350DbsuYXOg=;
        b=WJ5F0CZEmPntgarIVqYqFRC5r+tZ1gHziqco9NQGIO9Cmx6M1mQs1PhOELC1dK64a0
         iaWPtmtl1BKAtQgYE7EOrVEbHD3EWZf4fhSOP8StynFFm7M78yFKFEbtofGGtB8ow4ku
         XZf+bPmi/h/F3DWmWSpSceb0EIaBra0LYzsKE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708507037; x=1709111837;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=rsQ0Vfl6ZhEilW3byA6G/qo88FKVThxA350DbsuYXOg=;
        b=JSPDBaa9r3K0OcpgHsvnp+H33U9AwQZPAneKbqFmHEvg3dLR4rLtB5A0TNjNbY7ij5
         Z1mRhB9vvC/IzNaGDYgdCLbpWMvZpM6WVWrc7OyFS5pNHyyGl+WUdfO0MCKMrg+8cQn8
         aWPM54p3axpdLHtzUTJbjvMcIzJMJLZlA5YgN/YuQbmBJK8KBrfM9cGYDU3tiHELLC2Y
         nMqAE8cRhWUhjmkVIBEzlwZWxoKuIMEfeW12x/1ttXZUaWBgCBJMycVMV9XiHLYQhwvO
         xK/E/W12w+a0OArVmxGE8BWFsPLDSumwLMZwS5AUF8gSuZ8Lb21EMFLz8oCWysdezgRJ
         h9qg==
X-Forwarded-Encrypted: i=1; AJvYcCXI61Zpuydn+5f3u2lCXWZODAOpDV6SjNdtWv/LZHlzH5+CRO6DOT6NFlWoo1kh6JF0K3S9RwvTmAwGgHIynTLK7aDPxMl+DyTmH20JzKU=
X-Gm-Message-State: AOJu0YxxeF7Xd2pRmLOY2Qa+ORAE8IBsgmGhhIdc5nm9jLFEgLBmJITy
	3dhVWaLyJZqg4oJNy0OKA/a/lO579398R/IZ6fWKs7Fp46df0qV+03PQFk6AR9UDDkOZEnlj8lf
	F0FFytsGwIb2crjCl3tvH6Ot8uvZSLukWdYjobg==
X-Google-Smtp-Source: AGHT+IFkFu7ROa2MjXixgEtrEtmThVE0gGHJKc/LxSFS6ej/W6OZB7fzHFhVmSWneDKyWtHT0L1AY3qrGRR8Ht5bfy8=
X-Received: by 2002:a19:654c:0:b0:512:d473:803f with SMTP id
 c12-20020a19654c000000b00512d473803fmr614167lfj.21.1708507036713; Wed, 21 Feb
 2024 01:17:16 -0800 (PST)
MIME-Version: 1.0
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
 <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com> <7dd78c97-b82f-4ae1-a257-98f612c6bad2@xen.org>
 <6466155e-9883-4743-a897-ef565f14fdf0@suse.com> <CA+zSX=aJ2mVbGzFj74+DEGZvw=XWTZKAeWfrwMNHaYhsdaytHA@mail.gmail.com>
 <5a28331f-663f-421f-9845-45ff5eb1810c@suse.com>
In-Reply-To: <5a28331f-663f-421f-9845-45ff5eb1810c@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Wed, 21 Feb 2024 17:17:05 +0800
Message-ID: <CA+zSX=YtWXC1vkE6dhwrNpmft4X3ZHtroHjTaL1j_Gn_VFfr6Q@mail.gmail.com>
Subject: Re: [PATCH 2/2] almost fully ignore zero-size flush requests
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 21, 2024 at 3:17=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
> > #1 by itself is probably enough to counterindicate this kind of
> > behavior.  Add them together, and I'm inclined to say that we should
> > write a policy against such optimizations, without specific
> > justifications.
>
> It's not like I didn't give any justification. So I guess you mean
> without better (whatever that means) justification.

Sorry, what I meant was that the policy would have to include a sketch
for what sorts of justifications would be acceptable.

For instance, here's a justification I would consider for this sort of thin=
g:

A. In use-case X, there is hard limit Y on the binary size.  For X's
configuration, with a reasonably small number of features enabled, we
are already close to 90% of the way there.  If we were to consistently
use this sort of manual code size optimization techniques across the
codebase, we could cut down the total size of the code base by 25%.

Here's a situation I would absolutely not consider worth it:

B. If we consistently use this sort of code size optimization
techniques across the codebase, we could cut down the entire size of
the codebase by 0.1%.  There are no hard limits, we're just trying to
generally keep things smaller.

Filling our codebase with these sorts of logic puzzles ("Why are we
binary or-ing the offset and the length?") makes it more difficult for
people to understand the code base and increases the risk of someone
making a mistake as they try to change it.  For instance, is this
change really equivalent, given that previously one of the comparisons
had >=3D?  It turns out yes, but only because we filter out situations
where the length is 0; what if we were to move things around again,
such that we actually can get here with length 0?

Making the binary 0.1% smaller is absolutely not worth the cost of
that.  I'm not sure even 5% would be worth that cost, given that we
don't really have any hard limits we're in danger of exceeding (at
least that I'm aware of).

But a minimum justification for allowing these sorts of things would
need to include a concrete prediction of the improvement we would get
by applying these sorts of things all over the place; not simply, "in
this instance it goes from three to two branches".

 -George


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 09:34:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 09:34:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683913.1063535 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcizo-0000b3-5s; Wed, 21 Feb 2024 09:34:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683913.1063535; Wed, 21 Feb 2024 09:34:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcizo-0000aw-2A; Wed, 21 Feb 2024 09:34:32 +0000
Received: by outflank-mailman (input) for mailman id 683913;
 Wed, 21 Feb 2024 09:34:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcizm-0000aq-Rd
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 09:34:30 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcizl-0007R0-3l; Wed, 21 Feb 2024 09:34:29 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcizk-0003uX-Uo; Wed, 21 Feb 2024 09:34:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=HpeaWn16Dj4m5UJcWha4wY7PV7aKK81DpJk4qFhDKNg=; b=PsjWQzK0x2EGofoUHScOpvCOPw
	P5gw6RwPf0brJKFwJ1Oq9xOeGDKenTvcG/YpeZoF2IfgenUfyntrKDhP8BoIHFTLl1uvKHMytSgs/
	DjL11mDG8SOopkadzAoTr7xvTYl3DuMFNfcEV4b7lao1U2ZP6EU2b+rB2taw9mLedQW8=;
Message-ID: <3a277c20-462d-44f1-bedb-dc0adba04224@xen.org>
Date: Wed, 21 Feb 2024 09:34:27 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] almost fully ignore zero-size flush requests
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
 <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com>
 <7dd78c97-b82f-4ae1-a257-98f612c6bad2@xen.org>
 <6466155e-9883-4743-a897-ef565f14fdf0@suse.com>
 <a2348c5f-39ce-42f3-ad2a-a5edf88352e9@xen.org>
 <d0090122-c013-4dbf-97a2-3003352433a1@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <d0090122-c013-4dbf-97a2-3003352433a1@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 20/02/2024 12:25, Jan Beulich wrote:
> On 20.02.2024 12:52, Julien Grall wrote:
>> Hi Jan,
>>
>> On 20/02/2024 08:26, Jan Beulich wrote:
>>> On 19.02.2024 23:22, Julien Grall wrote:
>>>> Title: I would add 'gnttab:' to clarify which subsystem you are modifying.
>>>
>>> That's how I actually have it here; it's not clear to me why I lost the
>>> prefix when sending.
>>>
>>>> On 05/02/2024 11:03, Jan Beulich wrote:
>>>>> Along the line with observations in the context of XSA-448, besides
>>>>> "op" no field is relevant when the range to be flushed is empty, much
>>>>> like e.g. the pointers passed to memcpy() are irrelevant (and would
>>>>> never be "validated") when the passed length is zero. Split the existing
>>>>> condition validating "op", "offset", and "length", leaving only the "op"
>>>>> part ahead of the check for length being zero (or no flushing to be
>>>>> performed).
>>>>
>>>> I am probably missing something here. I understand the theory behind
>>>> reducing the number of checks when len == 0. But an OS cannot rely on it:
>>>>      1) older hypervisor would still return an error if the check doesn't
>>>> pass)
>>>
>>> Right, but that's no reason to keep the bogus earlier behavior.
>>
>> Hmmm... I am not sure why you say the behavior is bogus. From the commit
>> message, it seems this is just an optimization that have side effect
>> (ignoring the other fields).
> 
> I don't view this as primarily an optimization; I'm in particular after
> not raising errors for cases where there is no error to be raised.
> Hence the comparison to memcpy(), which you can pass "bogus" pointers
> so long as you pass zero size.

The part I am missing is why this approach is better than what we have. 
So far what you described is just a matter of taste.

To give a concrete example, if tomorrow a contributor decides to send a 
patch undoing what you did (IOW enforcing the check for zero-length or 
replace | with two branches), then on what grounds I will be able to 
refuse their patch?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 10:27:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 10:27:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683926.1063544 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcjov-00073R-1V; Wed, 21 Feb 2024 10:27:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683926.1063544; Wed, 21 Feb 2024 10:27:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcjou-00073K-Uu; Wed, 21 Feb 2024 10:27:20 +0000
Received: by outflank-mailman (input) for mailman id 683926;
 Wed, 21 Feb 2024 10:27:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TA2L=J6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcjot-00073E-IL
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 10:27:19 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c970d564-d0a3-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 11:27:17 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-5650ac9a440so474931a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 02:27:17 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ev24-20020a056402541800b0056163b46393sm4443346edb.64.2024.02.21.02.27.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 21 Feb 2024 02:27:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c970d564-d0a3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708511237; x=1709116037; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=pJTa6YW+NtX8rkgi+1nHxJa7bodneRdIU2pBJkxOBgo=;
        b=IWPyDHSEBXHzmdLR4zjxNnOGQ06uXKBDDzWZxQ5IaJIMjN/8lmZr8ZRMQzoA5T6bR7
         yMcBIFWmZ4mSHKSJ5LwOIaTNymrZT2z5RyJkpBDGKtxdjSX7OPp6dnqzX5EaQAgghmb3
         uNtksK0fGtqpTtirps4HP7058dpFBVMNabBP7ufXQQQq8lIWcz9rTfebCLSOHx4qOX0g
         X7raQqB9zBGHQTbUXYyDpN7VP7+cfYgfFY0KunH/H+FO5bhHeYLoqC5jXKk644iUP48c
         obJIzCzQG9W9n4cm1STvUQ34m+s1fOgRL2Rf4Do8p85G23GILi14Waq1UHhUUoQRHAZS
         HT6g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708511237; x=1709116037;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pJTa6YW+NtX8rkgi+1nHxJa7bodneRdIU2pBJkxOBgo=;
        b=lA7kM6xtuEDulPJxavKwujCWhSrAJ23qwqs1GZEuubGL/5NgqMMLCdFr1kTo0XyLNp
         9Wny+OLwm/c4VbNEoFXDOsyXKyKvPiKf97maiVnBodp6jbXG/7H+xT6PklRWgzl+pjtS
         j8mQZYXqotcaCb6rpHArHF+y6Yg2pbRSv0JDlRu5EVI+ym3ajuNo4CuoGh5YJ5ORd28P
         oncmFr0Td5dpexslPsdTFy8/soOI5XOnU2msnUK4Gub+sI1pGyeImfCBhCBOmrVun6L0
         HddwmECfgSdZByn8szuFSLZM8FYWzxpx4Trm89MItpu/pdbO93x0CJzN6zPxx8SSG6LC
         /BfA==
X-Gm-Message-State: AOJu0YxxoQNPK5gUWIFGiGVh3yzACUFwFM4njwPS8dB8VYe/bETys/W6
	v5R3YicFnxSzi+ArwoAKwN+mIw2F47c/R5Uxvg5uqlVGR5+YRqLe257P3Hn0VxkkLfyh0sVj4Uo
	=
X-Google-Smtp-Source: AGHT+IHfD1H/dIIewbpyDmudEspruYyxzjEhxGnQbtYtGGWudeJdT8Q0aKoBqyRPcvIk9xO3C8gj1w==
X-Received: by 2002:a05:6402:14d0:b0:564:66c1:d527 with SMTP id f16-20020a05640214d000b0056466c1d527mr6144651edx.36.1708511236934;
        Wed, 21 Feb 2024 02:27:16 -0800 (PST)
Message-ID: <52d2b102-1285-41fc-a550-ca9437df612b@suse.com>
Date: Wed, 21 Feb 2024 11:27:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86emul: add missing EVEX.R' checks
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

EVEX.R' is not ignored in 64-bit code when encoding a GPR or mask
register. While for mask registers suitable checks are in place (there
also covering EVEX.R), they were missing for the few cases where in
EVEX-encoded instructions ModR/M.reg encodes a GPR. While for VPEXTRW
the bit is replaced before an emulation stub is invoked, for
VCVT{,T}{S,D,H}2{,U}SI this actually would have led to #UD from inside
an emulation stub, in turn raising #UD to the guest, but accompanied by
log messages indicating something's wrong in Xen nevertheless.

Fixes: 001bd91ad864 ("x86emul: support AVX512{F,BW,DQ} extract insns")
Fixes: baf4a376f550 ("x86emul: support AVX512F legacy-equivalent scalar int/FP conversion insns")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
As to Fixes: tags - there are more affected commits, but they both only
re-use the expression introduced by the 2nd of the ones mentioned.

--- a/xen/arch/x86/x86_emulate/x86_emulate.c
+++ b/xen/arch/x86/x86_emulate/x86_emulate.c
@@ -3686,7 +3686,8 @@ x86_emulate(
     CASE_SIMD_SCALAR_FP(_EVEX, 0x0f, 0x2d): /* vcvts{s,d}2si xmm/mem,reg */
     CASE_SIMD_SCALAR_FP(_EVEX, 0x0f, 0x78): /* vcvtts{s,d}2usi xmm/mem,reg */
     CASE_SIMD_SCALAR_FP(_EVEX, 0x0f, 0x79): /* vcvts{s,d}2usi xmm/mem,reg */
-        generate_exception_if((evex.reg != 0xf || !evex.RX || evex.opmsk ||
+        generate_exception_if((evex.reg != 0xf || !evex.RX || !evex.R ||
+                               evex.opmsk ||
                                (ea.type != OP_REG && evex.brs)),
                               X86_EXC_UD);
         host_and_vcpu_must_have(avx512f);
@@ -7327,7 +7328,7 @@ x86_emulate(
         goto pextr;
 
     case X86EMUL_OPC_EVEX_66(0x0f, 0xc5):   /* vpextrw $imm8,xmm,reg */
-        generate_exception_if(ea.type != OP_REG, X86_EXC_UD);
+        generate_exception_if(ea.type != OP_REG || !evex.R, X86_EXC_UD);
         /* Convert to alternative encoding: We want to use a memory operand. */
         evex.opcx = ext_0f3a;
         b = 0x15;


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 10:27:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 10:27:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683928.1063555 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcjpL-0007S3-8e; Wed, 21 Feb 2024 10:27:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683928.1063555; Wed, 21 Feb 2024 10:27:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcjpL-0007Rw-61; Wed, 21 Feb 2024 10:27:47 +0000
Received: by outflank-mailman (input) for mailman id 683928;
 Wed, 21 Feb 2024 10:27:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TA2L=J6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcjpJ-00073E-Ok
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 10:27:45 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d960d9c4-d0a3-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 11:27:44 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-564a53b8133so3341759a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 02:27:44 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ev24-20020a056402541800b0056163b46393sm4443346edb.64.2024.02.21.02.27.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 21 Feb 2024 02:27:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d960d9c4-d0a3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708511263; x=1709116063; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=46RcPl9//p9Ch8d0wZpyLAwhHfYopm7zJrbgb+zTroo=;
        b=ELe0e1UHaUzVjGKV82+ip7io3/8zt5c9AXV5X/UgS0mIIKfmxury3WTAGk3ypcnTbN
         FTsbeos7kx4284J98NHBK+dSDkfs2YQ1FDcpXdh5nDt2lWN/vvaB79iz2ZNtXQ80wS1e
         QofGr4mszfKe0jkqU+and00Jjr+8sM97qmLk70T/s1iI86gwsXMT8ULBt8NfD9YaMF2Z
         37k3lTKxQllxL2spugqsSgA5jrCYbF9qXn+vvlIpKwt8UnpOsZ4fUywzixJTclPZINmy
         UrXj3qjJpDTXQDf/L902oxWC3zeflV3VV5hF8V87ED+ZNu7K54Xt6dfKLsXRLie8MZwW
         qkZw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708511263; x=1709116063;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=46RcPl9//p9Ch8d0wZpyLAwhHfYopm7zJrbgb+zTroo=;
        b=i7AQD/uIURESpLnZHezs5UztHhtl9uhleBe+a6d0KieQojmPIXXiXcsyT3scAGWob9
         xgxMIEK7+6SY9VfDTqS9cNwLV/JwbBICX5yNkBybykLMpyWsTPXBiI0MZafsobTayGmZ
         3O0E2gnoYqRiPpKKJCkAi0ZWwB67IhnZQmOZdG+WM/KqrkjgRzpCc3x4vmJxV0ZmfmMO
         o2XRLRDQvbsDyf3/2BxJGputomjg6AFHpEVzDRWhFCVUwzWH+yRj3k+IwZRNkLDEMgYV
         R1wVgjDyYTcUk/syXnzMyA5fS0AC4HofAbiBYMKYA7VG8Y5tgI3k0OKDcYe1FSbXr7IN
         ZZrQ==
X-Gm-Message-State: AOJu0Yzk8WTA8qKWzYYQRxTZpCHe0G4GPHQXFnJ62p4h6LWRuiI+IeHM
	Y/bAkCnXAHOl7hxT71wvphQ6gSAlQIqgEbh2N4iVkx08YMhOSFNZ7qWjqyM+KflqYnI2+mDwhw4
	=
X-Google-Smtp-Source: AGHT+IECpUKHCo05KBvASBTqUsc244lRcYUgcSz+BHIrP6lrkksPor9jBjWgzROiQx2pOQJ4WwcPKw==
X-Received: by 2002:a05:6402:502:b0:564:e57a:18b4 with SMTP id m2-20020a056402050200b00564e57a18b4mr1534613edv.20.1708511263700;
        Wed, 21 Feb 2024 02:27:43 -0800 (PST)
Message-ID: <6da6639d-f11b-47ac-af50-071d451fa60a@suse.com>
Date: Wed, 21 Feb 2024 11:27:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86emul: make run32 test harness goal work again
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

When re-working library call wrapping the sed invocation didn't account
for all sources living in the parent directory when building the 32-bit
harness binary.

Fixes: 6fba45ca3be1 ("x86emul: rework wrapping of libc functions in test and fuzzing harnesses")
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/tools/tests/x86_emulator/Makefile
+++ b/tools/tests/x86_emulator/Makefile
@@ -255,7 +255,8 @@ OBJS := x86-emulate.o cpuid.o test_x86_e
 OBJS += x86_emulate/0f01.o x86_emulate/0fae.o x86_emulate/0fc7.o
 OBJS += x86_emulate/blk.o x86_emulate/decode.o x86_emulate/fpu.o x86_emulate/util.o
 
-WRAPPED := $(shell sed -n 's,^ *WRAP(\([[:alnum:]_]*\));,\1,p' x86-emulate.h)
+WRAPPED := $(shell sed -n 's,^ *WRAP(\([[:alnum:]_]*\));,\1,p' \
+                       $(firstword $(wildcard ../x86-emulate.h) x86-emulate.h))
 
 $(TARGET): $(OBJS)
 	$(HOSTCC) $(HOSTCFLAGS) $(addprefix -Wl$(comma)--wrap=,$(WRAPPED)) -o $@ $^


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 10:32:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 10:32:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683938.1063565 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcju2-0000qj-SB; Wed, 21 Feb 2024 10:32:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683938.1063565; Wed, 21 Feb 2024 10:32:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcju2-0000qc-OS; Wed, 21 Feb 2024 10:32:38 +0000
Received: by outflank-mailman (input) for mailman id 683938;
 Wed, 21 Feb 2024 10:32:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TA2L=J6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcju1-0000qT-Mu
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 10:32:37 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 87cb8e8c-d0a4-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 11:32:36 +0100 (CET)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-564d9b0e96dso1909125a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 02:32:36 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 q29-20020a50cc9d000000b00563a3ff30basm4767647edi.59.2024.02.21.02.32.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 21 Feb 2024 02:32:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87cb8e8c-d0a4-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708511556; x=1709116356; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=7h4uJdEKu2CZ2gxhOJx5jx71sYHCisevUny+2+RNbFU=;
        b=PAdRi2/Wzo+azYDJBu1eNkbc626VjVe9SAPrsnCWFFTQuLZpdmVcJbfadZwELEtX9/
         UJONa6uzTtrUiiJOU53tLd9gOkojW/L+DHZ4SvPkjJgwLUbGK5qIkAaZ25VJmgYer+Vo
         AjeeP7wVvB4ZOuIvmDol9BTIOXAj6Tl/4c7qHCec0WhlHD8gq4UuQI/ss/9793RbNzSu
         qtQcieYarcjhuIr+nlQYPoWQxYyU/i+EJnfcEYPdiuXS6kZhHf8T1tivGkf5EXPsdNCd
         fzpxNtI+rf4ln9CAqoc47Dql9jIegcddgB4rJI8+V9sP8DZ6kVOLXdGs66npFpuPjJDx
         VJvw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708511556; x=1709116356;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=7h4uJdEKu2CZ2gxhOJx5jx71sYHCisevUny+2+RNbFU=;
        b=ObCuyoboCf20lyL4zjUPJl4gFo8haMj6tlkZOXVEiz9o/yzC5/DKd6mR53Zy3nBfbz
         Q5+sZQxJD+nV80fJXOAJ2cdvx2evjqWYq4ol53qOJ1JWrP1dz+m1B/XmuQCTKHOFwmJJ
         BQSMeyNKyE0FOgV+BgZ+928y6OU4dGhyzr4OOnIEoiA6In59lQ4c3bKARTj0PdwiMl5S
         GBUPhHDg8zGMcWsQvqfpty8wHeW+hOdg3KxfvJyY9i5aNMfU8veFC/HxFbN+7UIhhbMI
         iPgP/H1+I/Y4vkSna36NuaWKmy+9fkhu7YCBPA6DJjT9tm07zGVIYw+jo1F+HXyBeqGR
         AO6w==
X-Forwarded-Encrypted: i=1; AJvYcCWyvxOEa5tRcBmYBTzudY79wPNb7RbrsXPuy8u4L0W/clHrTPAPhMMvt8oePt2gwIma39+/Thxm1ZMBMgcT6wsO5nfdf4KJL87tjbQR5bU=
X-Gm-Message-State: AOJu0Yw3kjaNVOZXqXh/0lq0KvYou44u7ILhxQMtLrt3z7GxCsNkO8Or
	QSwhJ4TGl+3BIJuhiCbE7J/VxygM0LwIsMGiSxKtXmCIyPOS/CmGt22NyxyeeQ==
X-Google-Smtp-Source: AGHT+IEApuCKf6SmUPmwql2xgmSZSO/BbQ86WTWMc2R6ueDTLctemSCEwDnYSAWJneRQL39l7IgsLQ==
X-Received: by 2002:a05:6402:608:b0:564:3bca:8bb7 with SMTP id n8-20020a056402060800b005643bca8bb7mr7968056edv.12.1708511556307;
        Wed, 21 Feb 2024 02:32:36 -0800 (PST)
Message-ID: <2002c36b-1e48-4778-abb3-c2d12189df50@suse.com>
Date: Wed, 21 Feb 2024 11:32:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] almost fully ignore zero-size flush requests
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <a0a7fa35-4615-4a9b-9887-569353f05e33@suse.com>
 <26ff8f42-2a76-4f8d-9af6-5830b0aae739@suse.com>
 <7dd78c97-b82f-4ae1-a257-98f612c6bad2@xen.org>
 <6466155e-9883-4743-a897-ef565f14fdf0@suse.com>
 <a2348c5f-39ce-42f3-ad2a-a5edf88352e9@xen.org>
 <d0090122-c013-4dbf-97a2-3003352433a1@suse.com>
 <3a277c20-462d-44f1-bedb-dc0adba04224@xen.org>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3a277c20-462d-44f1-bedb-dc0adba04224@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 21.02.2024 10:34, Julien Grall wrote:
> Hi,
> 
> On 20/02/2024 12:25, Jan Beulich wrote:
>> On 20.02.2024 12:52, Julien Grall wrote:
>>> Hi Jan,
>>>
>>> On 20/02/2024 08:26, Jan Beulich wrote:
>>>> On 19.02.2024 23:22, Julien Grall wrote:
>>>>> Title: I would add 'gnttab:' to clarify which subsystem you are modifying.
>>>>
>>>> That's how I actually have it here; it's not clear to me why I lost the
>>>> prefix when sending.
>>>>
>>>>> On 05/02/2024 11:03, Jan Beulich wrote:
>>>>>> Along the line with observations in the context of XSA-448, besides
>>>>>> "op" no field is relevant when the range to be flushed is empty, much
>>>>>> like e.g. the pointers passed to memcpy() are irrelevant (and would
>>>>>> never be "validated") when the passed length is zero. Split the existing
>>>>>> condition validating "op", "offset", and "length", leaving only the "op"
>>>>>> part ahead of the check for length being zero (or no flushing to be
>>>>>> performed).
>>>>>
>>>>> I am probably missing something here. I understand the theory behind
>>>>> reducing the number of checks when len == 0. But an OS cannot rely on it:
>>>>>      1) older hypervisor would still return an error if the check doesn't
>>>>> pass)
>>>>
>>>> Right, but that's no reason to keep the bogus earlier behavior.
>>>
>>> Hmmm... I am not sure why you say the behavior is bogus. From the commit
>>> message, it seems this is just an optimization that have side effect
>>> (ignoring the other fields).
>>
>> I don't view this as primarily an optimization; I'm in particular after
>> not raising errors for cases where there is no error to be raised.
>> Hence the comparison to memcpy(), which you can pass "bogus" pointers
>> so long as you pass zero size.
> 
> The part I am missing is why this approach is better than what we have. 
> So far what you described is just a matter of taste.
> 
> To give a concrete example, if tomorrow a contributor decides to send a 
> patch undoing what you did (IOW enforcing the check for zero-length or 
> replace | with two branches), then on what grounds I will be able to 
> refuse their patch?

On the grounds of the argument I gave before: Consistency with other
more or less similar operations, where length 0 simply means "no-op",
up to and including "no errors from arguments specifying the address(es)
to operate on".

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 10:34:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 10:34:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683943.1063575 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcjvj-0001Sj-Au; Wed, 21 Feb 2024 10:34:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683943.1063575; Wed, 21 Feb 2024 10:34:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcjvj-0001Sc-76; Wed, 21 Feb 2024 10:34:23 +0000
Received: by outflank-mailman (input) for mailman id 683943;
 Wed, 21 Feb 2024 10:34:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=E9i0=J6=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rcjvi-0001SW-JS
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 10:34:22 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c670bad8-d0a4-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 11:34:21 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a3e4765c86eso473688466b.0
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 02:34:21 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 vo5-20020a170907a80500b00a3d559c6113sm4803279ejc.204.2024.02.21.02.34.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 21 Feb 2024 02:34:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c670bad8-d0a4-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708511661; x=1709116461; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=F6BVW3ETWXZLxSHalU1zubHFF93FEpizsrFcZD8Bpsc=;
        b=tycE1m+1JlLgs/nHNFI2lkSB0XL9HwHkwzJlUw1+MLXx5eTiprTGDxokUrWj945j+O
         sn1VmgXvZWX+KPYAg12Ecv8nhiY5404mjCJ60XWtWwE8iG1WQ7F5OvXimhwKHe+PesVe
         ZfBW1lKoTvWIqmylsNzNd50XF/1ktdWOc5R/Y=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708511661; x=1709116461;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=F6BVW3ETWXZLxSHalU1zubHFF93FEpizsrFcZD8Bpsc=;
        b=DvJoxBgIZdUEIX+oYH65PUOBN+PdIYtyn6JfuFczen2Cm8wiQqSqjPq+DIa5aNNebX
         fNufE96Kx+nJEtsDcCvsaeJ3Yih2ZwzPXCicJIlPHtIuFAfvsvlcBkx7yC2c/MScrDAO
         BK155IyfhqTTxXSXZ/e0Bdkvugty+NWXgzMl+g1UNFck6YgZA05wy1f6E17ZFdr/Lnsy
         pNjIiuwvDlZeVrTFDCSG4rYGUJAEN0vxzg2Lhr0xKjKl8uQbP/BvdS28ITQQo6kfje25
         4eCXOvURf03h9GdgeEbbYQnEQiD5ne1QEEmg+iVlf0gmdkUo2yNawN/7N0tCxsl5gRHk
         IK5Q==
X-Forwarded-Encrypted: i=1; AJvYcCUqlBpnqSbqvO9pyh6nNmuUvqJUg4Q/vbfHnpozPZ83BKcccYY+PrVSxxuV7HxR220qFsSwm73N6aBPJeZ7DCcziVtC/6QHD98KxKCQark=
X-Gm-Message-State: AOJu0YwlQ/yMrKXp7UJjshC+a7TxvxbE82e4jAiKtjJV+vOD54F4irIn
	KgEYEXMKggYnkuDzo9MW04FCuehKt5CGcoJP6pTk13L4poLqOeaTyNXA92MXYfA=
X-Google-Smtp-Source: AGHT+IE/z24U31/40OP8e7eimpzKvRB+cl5LnXoUfJiKgSYiUEGjoCZbHT4bFuDFedwHbJ9j0YSToQ==
X-Received: by 2002:a17:906:504d:b0:a3e:1225:2d7d with SMTP id e13-20020a170906504d00b00a3e12252d7dmr7837461ejk.38.1708511661478;
        Wed, 21 Feb 2024 02:34:21 -0800 (PST)
Message-ID: <db5c58e2-09c1-4c99-bffc-df16010403b4@citrix.com>
Date: Wed, 21 Feb 2024 10:34:21 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86emul: make run32 test harness goal work again
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <6da6639d-f11b-47ac-af50-071d451fa60a@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <6da6639d-f11b-47ac-af50-071d451fa60a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 21/02/2024 10:27 am, Jan Beulich wrote:
> When re-working library call wrapping the sed invocation didn't account
> for all sources living in the parent directory when building the 32-bit
> harness binary.
>
> Fixes: 6fba45ca3be1 ("x86emul: rework wrapping of libc functions in test and fuzzing harnesses")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 10:52:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 10:52:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683957.1063585 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rckCx-0004js-Ol; Wed, 21 Feb 2024 10:52:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683957.1063585; Wed, 21 Feb 2024 10:52:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rckCx-0004jl-Ln; Wed, 21 Feb 2024 10:52:11 +0000
Received: by outflank-mailman (input) for mailman id 683957;
 Wed, 21 Feb 2024 10:52:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TA2L=J6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rckCw-0004jd-RI
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 10:52:10 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4248a7d6-d0a7-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 11:52:08 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a3e75e30d36so592009866b.1
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 02:52:08 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s8-20020a170906168800b00a3e7a2d9ac4sm3339978ejd.6.2024.02.21.02.52.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 21 Feb 2024 02:52:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4248a7d6-d0a7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708512728; x=1709117528; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ydcPULZ+yw3cK2VhFEG/Cg908Xb9yky5xkBswdzDVWI=;
        b=TaXzlcYrtbJdT5Yld7h/4voKjE19lvd/L4Jq7wbsICaSJS7dLtmxV3ISdAXkctbdKJ
         qPeJe9rgxBCv1NnBoaIFCSDiErb6yqA6VBbTz1SQRND6MT8nOgHVnQ1+OR5vVJgbFVe3
         aV1giqZbHkHn1EceZupD7hE5rkZ1M/K0hdW5/B8pC16Q05EbhBvap/1LSjbKIrSzpEvk
         mDC5BnERPufPGqCOo8rrc9C7X5OsZc80AvChBiHndFJlvuT0UFyUj9jOHeAIEISDf55I
         DCk26FY0POlOx5UY+cT7jBbVA6a3jCnEhzGGrNWBTAExuzU5qOEjRLhl2XSgqhvUelDX
         7gmQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708512728; x=1709117528;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ydcPULZ+yw3cK2VhFEG/Cg908Xb9yky5xkBswdzDVWI=;
        b=m0EK43qWIW23VoYknyjGEXiwpSM5IVQ+sNfqM6hG0ynKcASTAi3bBKvD8dWmILaT4W
         79gaUaq6yc4b3bGI1GrgQhChla0PFOVf2ySwIMCJN2iqWpE/DnF/jB1WyQMwdfQLKbui
         Sj8lC8oB/b+VN7zDSUfcgtBAA1OOisH3h9OZpmAZci4C87/3ef+0x8IqEj/KnpSL8fLo
         fxUW4Ia2sORxk/3TweW9djr80mxdG/BpMcSXcVGv4FcSTmT8CHlcmbouDYrOuDaDt/j+
         XGy7rOTxpbwbSi35rHBT/vkBXQp0jeqKzZiWAYZKNQE2BmRMzSE7KT+qymMvagqK56Zy
         ZuRA==
X-Forwarded-Encrypted: i=1; AJvYcCUpmzbeOgr+NMEuAS7TTlRBL1zYsjrc1SS6pIOIuNkSNDyea5A3BDU7qL+6aFzHP+rXvB+qbcujf/aBBy+/LS+9BrlMpjWTpDHDZOS91h4=
X-Gm-Message-State: AOJu0Yzi6b52DyMPMpoE7644EFx7I2mxwopUIkBQDKTWEGF2kVbnwqb7
	BTXxbcaPfF3vKpkxYskxHlBQcxY/Bg2WGJzD3WE0PAv+Na0dXW8xJ+L3bO3ihA==
X-Google-Smtp-Source: AGHT+IFgmhIdfe3tQDzSuE6lmF0tET3TzlCpKxgBN+msncAH6UpUi2V6dQGkh4HQ3QguSnaSLie6iw==
X-Received: by 2002:a17:906:7f18:b0:a3e:7d76:6559 with SMTP id d24-20020a1709067f1800b00a3e7d766559mr9532780ejr.33.1708512728119;
        Wed, 21 Feb 2024 02:52:08 -0800 (PST)
Message-ID: <74ef7b2b-b105-4060-afaa-73ec272a7449@suse.com>
Date: Wed, 21 Feb 2024 11:52:06 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/6] nestedsvm: Disable TscRateMSR
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-5-george.dunlap@cloud.com>
 <c5f3a78c-4a2b-499b-93bb-11f60b0afcc0@suse.com>
 <CA+zSX=acH-4B01=vUBstAbsNBMVgcRe_A5v0eXn66BdsuPtgiA@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CA+zSX=acH-4B01=vUBstAbsNBMVgcRe_A5v0eXn66BdsuPtgiA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.02.2024 09:48, George Dunlap wrote:
> On Mon, Feb 19, 2024 at 11:22 PM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 06.02.2024 02:20, George Dunlap wrote:
>>> For now, just disable the functionality entirely until we can
>>> implement it properly:
>>>
>>> - Don't set TSCRATEMSR in the host CPUID policy
>>
>> This goes too far: This way you would (in principle) also affect guests
>> with nesting disabled. According to the earlier parts of the description
>> there's also no issue with it in that case. What you want to make sure
>> it that in the HVM policy the bit isn't set.
>>
>> While presently resolving to cpu_has_svm_feature(), I think
>> cpu_has_tsc_ratio really ought to resolve to the host policy field.
>> Of course then requiring the host policy to reflect reality rather than
>> having what is "always emulated". IOW ...
>>
>>> --- a/xen/arch/x86/cpu-policy.c
>>> +++ b/xen/arch/x86/cpu-policy.c
>>> @@ -407,8 +407,7 @@ static void __init calculate_host_policy(void)
>>>                                 (1u << SVM_FEATURE_PAUSEFILTER) |
>>>                                 (1u << SVM_FEATURE_DECODEASSISTS));
>>>          /* Enable features which are always emulated. */
>>> -        p->extd.raw[0xa].d |= ((1u << SVM_FEATURE_VMCBCLEAN) |
>>> -                               (1u << SVM_FEATURE_TSCRATEMSR));
>>> +        p->extd.raw[0xa].d |= (1u << SVM_FEATURE_VMCBCLEAN);
>>
>> ... this likely wants replacing altogether by not overriding what we
>> found in hardware, which would apparently mean moving the two bit
>> masks to the earlier "clamping" expression.
>>
>> But then of course Andrew may know of reasons why all of this is done
>> in calculate_host_policy() in the first place, rather than in HVM
>> policy calculation.
> 
> It sounds like maybe you're confusing host_policy with
> x86_capabilities?  From what I can tell:
> 
> *  the "basic" cpu_has_X macros resolve to boot_cpu_has(), which
> resolves to cpu_has(&boot_cpu_data, ...), which is completely
> independent of the cpu-policy.c:host_cpu_policy
> 
> * cpu-policy.c:host_cpu_policy only affects what is advertised to
> guests, via {pv,hvm}_cpu_policy and featureset bits.  Most notably a
> quick skim doesn't show any mechanism by which host_cpu_policy could
> affect what features Xen itself decides to use.

I'm not mixing the two, no; the two are still insufficiently disentangled.
There's really no reason (long term) to have both host policy and
x86_capabilities. Therefore I'd prefer if new code (including a basically
fundamental re-write as is going to be needed for nested) to avoid
needlessly further extending x86_capabilities. Unless of course there's
something fundamentally wrong with eliminating the redundancy, which
likely Andrew would be in the best position to point out.

> Not sure exactly why the nested virt stuff is done at the
> host_cpu_policy level rather than the hvm_cpu_policy level, but since
> that's where it is, that's where we need to change it.
> 
> FWIW, as I said in response to your comment on 2/6, it would be nicer
> if we moved svm_feature_flags into the "capabilities" section; but
> that's a different set of work.

Can as well reply here then, rather than there: If the movement from
host to HVM policy was done first, the patch here could more sanely go
on top, and patch 2 could then also go on top, converting the separate
variable to host policy accesses, quite possibly introducing a similar
wrapper as you introduce there right now.

But no, I'm not meaning to make this a requirement; this would merely be
an imo better approach. My ack there stands, in case you want to keep
(and commit) the change as is.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 11:00:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 11:00:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683964.1063595 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rckLE-0006ew-IQ; Wed, 21 Feb 2024 11:00:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683964.1063595; Wed, 21 Feb 2024 11:00:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rckLE-0006ep-Ej; Wed, 21 Feb 2024 11:00:44 +0000
Received: by outflank-mailman (input) for mailman id 683964;
 Wed, 21 Feb 2024 11:00:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TA2L=J6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rckLC-0006ei-HC
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 11:00:42 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 73520f18-d0a8-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 12:00:40 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a3f4464c48dso50047366b.3
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 03:00:40 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w8-20020a1709067c8800b00a3e9ea30a95sm2806625ejo.5.2024.02.21.03.00.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 21 Feb 2024 03:00:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73520f18-d0a8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708513240; x=1709118040; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=k9wBkQspzqmzlIJCcBa4X8dSTnv3wOlrmhWnq465fmQ=;
        b=L3SeMVM/lode/aLwqbLIpuIUBUoDkG4X94dSPoUd+jrw2WrJ3eoI54Stoon5yGa1+T
         EWpyvfz4BlQKHIqZY9BOzFRynLGfoYxAmuzzvA02US8xsRM1OziZaOeG99IUFupUSmHp
         5Ne4QpNNfYdchZqxKCO823y1/F9TrXY5pWIeeyCKNppXmuTjMu8XKq5QreITt/Djrz6l
         Rom+XAnoL4W+Q4IzyU1Mr9jChPPJM0ckknwcqn+u+wiUCheUsfn2OgpjJPgbcLRSLDfI
         /QDfkZAJZswiqySB2FHprGaohP/Y1W1yW/pXiJWmyIVjZs+71Prc3h5Yyrv//iDjzBfa
         KXNw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708513240; x=1709118040;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=k9wBkQspzqmzlIJCcBa4X8dSTnv3wOlrmhWnq465fmQ=;
        b=k6LEeyIOPJppl64kE8Sj3vhlPLaVRiMkIifjn8X0DNOpmMJo8Z6MIEIRzP3F7w0iBn
         sY0KOfkDCVw+IFXhMSjQjNTcou/E3R+ztBxy0hQErBhb+xicIRZSTzJr7zERV/mNHiqV
         0kax4btwRiM7oLrhvPtpkMPfpoCm1rrNm4FdDwbfbZLbmcVCQHB1qPGAMzzhOtaJK35a
         cizTx1BGpb4t1BOipSIJnlgOmdKjioLzNkhzYiCsW594OMByRo8nIWaROhHz9sJf6qTO
         R7NLPbN5kqCkykCL7FUHSO0PCvWXbo+BSKW5dodyhd8Psc1AZRgVn8K28O5VRRivtu4m
         kRBg==
X-Forwarded-Encrypted: i=1; AJvYcCWrp/uHfJKc9cMZLUwasBSWwlJKKO/NUpfm1mK7+uZ9JWNyl8IEGv54ijE2K4eyJOU8pmZTqFAH2ztv9CMPp75PzaDTG60bZj007QHFyi8=
X-Gm-Message-State: AOJu0YyaNYUvBmL6ZYq2If1CszDW3oZkODUTA4xyvZzG7cqhBoEQLnBy
	+4th7g9cMXiAWsHsQ5lIdNmx2ecwlB49ZOQeOJd/h+lgM/D438WXxf+38noHCA==
X-Google-Smtp-Source: AGHT+IFMpZZn29jRiyCr9qtxWDYgM4DPbZMmmtYKB36ZhzPLJcWyP3RyWcveKJemYCcmquBpQksakQ==
X-Received: by 2002:a17:906:488c:b0:a3f:7bc:7278 with SMTP id v12-20020a170906488c00b00a3f07bc7278mr3368845ejq.31.1708513239887;
        Wed, 21 Feb 2024 03:00:39 -0800 (PST)
Message-ID: <a6394d8a-63d0-42bf-8ed7-a7722cb7e71c@suse.com>
Date: Wed, 21 Feb 2024 12:00:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 07/30] xen/asm-generic: introdure nospec.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Julien Grall <julien@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <5889d7a5fa81722472f95cc1448af0be8f359a7d.1707146506.git.oleksii.kurochko@gmail.com>
 <510d74ca-5cce-48f8-93f6-407cd727cf7b@xen.org>
 <8a09e386b709f780f193af39af63b6aeb75c868e.camel@gmail.com>
 <aaf86d31-7ae0-4e33-8386-dda4bd21496a@suse.com>
 <ab040f3c8ee35f49fffac385053b55053c52da8c.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ab040f3c8ee35f49fffac385053b55053c52da8c.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 20.02.2024 21:30, Oleksii wrote:
> On Mon, 2024-02-19 at 13:18 +0100, Jan Beulich wrote:
>> On 19.02.2024 12:59, Oleksii wrote:
>>> Hi Julien,
>>>
>>> On Sun, 2024-02-18 at 18:30 +0000, Julien Grall wrote:
>>>> Hi Oleksii,
>>>>
>>>> Title: Typo s/introdure/introduce/
>>>>
>>>> On 05/02/2024 15:32, Oleksii Kurochko wrote:
>>>>> The <asm/nospec.h> header is similar between Arm, PPC, and
>>>>> RISC-V,
>>>>> so it has been moved to asm-generic.
>>>>
>>>> I am not 100% convinced that moving this header to asm-generic is
>>>> a
>>>> good 
>>>> idea. At least for Arm, those helpers ought to be non-empty, what
>>>> about 
>>>> RISC-V?
>>> For Arm, they are not taking any action, are they? There are no
>>> specific fences or other mechanisms inside
>>> evaluate_nospec()/block_speculation() to address speculation.
>>
>> The question isn't the status quo, but how things should be looking
>> like
>> if everything was in place that's (in principle) needed.
>>
>>> For RISC-V, it can be implemented in a similar manner, at least for
>>> now. Since these functions are only used in the grant tables code (
>>> for
>>> Arm and so for RISC-V ), which is not supported by RISC-V.
>>
>> Same here - the question is whether long term, when gnttab is also
>> supported, RISC-V would get away without doing anything. Still ...
>>
>>>> If the answer is they should be non-empty. Then I would consider
>>>> to
>>>> keep 
>>>> the duplication to make clear that each architecture should take
>>>> their 
>>>> own decision in term of security.
>>>>
>>>> The alternative, is to have a generic implementation that is safe
>>>> by 
>>>> default (if that's even possible).
>>> I am not certain that we can have a generic implementation, as each
>>> architecture may have specific speculation issues.
>>
>> ... it's theoretically possible that there'd be an arch with no
>> speculation issues, maybe simply because of not speculating.
> 
> I am not sure that understand your and Julien point.
> 
> For example, modern CPU uses speculative execution to reduce the cost
> of conditional branch instructions using schemes that predict the
> execution path of a program based on the history of branch executions.
> 
> Arm CPUs are vulnerable for speculative execution, but if to look at
> the code of evaluate_nospec()/block_speculation() functions they are
> doing nothing for Arm.

Which, as I understood Julien say, likely isn't correct. In which case
this header shouldn't be dropped, using the generic one instead. The
generic headers, as pointed out several times before, should imo be used
only if their use results in correct behavior. What is acceptable is if
their use results in sub-optimal behavior (e.g. reduced performance or
lack of a certain feature that another architecture maybe implements).

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 11:33:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 11:33:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683974.1063604 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rckr1-00032A-4G; Wed, 21 Feb 2024 11:33:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683974.1063604; Wed, 21 Feb 2024 11:33:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rckr1-000323-1i; Wed, 21 Feb 2024 11:33:35 +0000
Received: by outflank-mailman (input) for mailman id 683974;
 Wed, 21 Feb 2024 11:33:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=i+Ye=J6=amd.com=ayan.kumar.halder@srs-se1.protection.inumbo.net>)
 id 1rckqz-00031v-RM
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 11:33:33 +0000
Received: from NAM12-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2418::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 09ae7f58-d0ad-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 12:33:31 +0100 (CET)
Received: from PH8PR12MB7326.namprd12.prod.outlook.com (2603:10b6:510:216::7)
 by MN6PR12MB8490.namprd12.prod.outlook.com (2603:10b6:208:470::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.17; Wed, 21 Feb
 2024 11:33:27 +0000
Received: from PH8PR12MB7326.namprd12.prod.outlook.com
 ([fe80::22eb:b0b:62c0:f3e5]) by PH8PR12MB7326.namprd12.prod.outlook.com
 ([fe80::22eb:b0b:62c0:f3e5%6]) with mapi id 15.20.7316.018; Wed, 21 Feb 2024
 11:33:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09ae7f58-d0ad-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZxRhdwy5Xq2euSxVx5Vt4ANXCB9HPfAAJl7ymv9dRR/mkZWuAcPgAKYVdQvw1EiBHMdZ94bwPJZC+fuXI/ynz30TEmmhLtQjLIUWqd4YgdpYB/RazFA9yWQwKTqhEXkboNz4+mSZP1/qQmJJCZE4c1ezKhPkJ7+FKg+fc0RBWQLsg++ZevOiYJCZ1O53KRJCbsS70MPmtuHX7TzsFgAVustaML/atXKQn9W03/5klHC+y3SRR9neDviRlDaIGcUolXnSp59mXhPa49Q4qJIawJRAXn6YqpkrxkYnmgX5osY5GOQqnJntXvLHqa8lhDRlJcjXYkSNUCQTYR4FyyWweA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=yuYUIaR3PwI91XYPj4p6fO0tb69+oRj4kDID5JeBdm8=;
 b=K55rtfeVmWpY6wMuRvXHitKZdoALUG+rBEQpFMAIhmBe+ZIz0ELl3/i0KJiacSHlT97UYfCfTfgMGgPmMu+N96VGTiYoQpLN3AJNYSjgHr2Pl64ja42bLWEvoAL/heeZPBqG6a6aRtgOPTLOpsGzuk6ExMgk1BWPaFTB6Dq4KPrRCJReipWncM7+0lc0DXxGxd16Jq2jHxCdPK49BQ+NU4yIn9+ykcjaUncg9asSzVkKdrUK3Oje9XjXHuow0KoHbTlEp9XUjsdURMdnicQ3pfNLJ2INIClgaO3atCEUnX4e1EE18w5sxpzvy45f9IDSMmvwRE77+qd7BQCttZqLtA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=yuYUIaR3PwI91XYPj4p6fO0tb69+oRj4kDID5JeBdm8=;
 b=11G34yo+/OyVOoZPuh+IG0SVVhgKBsFepByMxNTpr2O3CiujT1me8OMttAnwqqjjXZ07I9coSYpgN62+DhcnZMsZs+ks+Q2fE5icNcY5CmQrCQHzR+R+7lomVvoDW8Ohx7KYTzcwDPShav/ScoAfOsEEpShCahu6k8hHODlg9W4=
Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
Message-ID: <62e047c5-a913-410d-9b24-e9a26d555d78@amd.com>
Date: Wed, 21 Feb 2024 11:33:22 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v5 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com, julien@xen.org,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, michal.orzel@amd.com,
 xen-devel@lists.xenproject.org
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
 <20240220121743.3680715-2-ayan.kumar.halder@amd.com>
 <e3a8ce5d-841e-4ff1-8d23-48de822972ef@suse.com>
 <5f526deb-7376-4c34-85d1-d22b1b14d90e@amd.com>
 <e7c4b08f-724a-4419-977a-caa1342c47eb@suse.com>
From: Ayan Kumar Halder <ayankuma@amd.com>
In-Reply-To: <e7c4b08f-724a-4419-977a-caa1342c47eb@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO4P265CA0118.GBRP265.PROD.OUTLOOK.COM
 (2603:10a6:600:2c6::8) To PH8PR12MB7326.namprd12.prod.outlook.com
 (2603:10b6:510:216::7)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: PH8PR12MB7326:EE_|MN6PR12MB8490:EE_
X-MS-Office365-Filtering-Correlation-Id: ddb76728-7105-42d0-33a5-08dc32d0ebdc
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	oLfY/29pcIMIM+C3ZwbH1FyB55im6uUFB4HLhSO7CUwJ+t2dKLBCJ1IQ3Cpxx7+KeFZFcww4e7sxo4hYi1Nv/FM/45DFzR3Ynu8ucGdLKN/5dckEgFhIoszub00BjT1fD7N1/CAkqnNgaAXmNPfGwxmB3EatGZze20j3qFj8KV35gjw+b+DFcf5dyqlz3v/8UeyV7ZcWccQ60znjDCrIxdyw+aGWLwVS5hpxsI57iNz61nyCV8UVjIcMeV6ZqErzPbSEY+ypEzskKfOD/CNZ1174gu//F7mswoLAvMjotfEAa0MdvYznvmu0txswDFjBOjaw5UjQ4ykMRHZVzgc21Djj1axC++x9Ub6qzu1POnmhfmNmsJMNGfi7kSsY76bctGzv2UUohQsFpMdo1iob9UfXX7GZYQVH6TJfALUAtPZDM7/keSAEwdZiTeEgueH9lnFIPnwzxctkz2A+Aj/wnUgKujIywn7hRIbGsiocAFGsse+4bviI+9gIbL+VgE09JMivBBiycpwFtmTYdVnvblcA9gthlExna/36hZmskyU=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH8PR12MB7326.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OWVNODZSbXZQbU5Qa3Bld3E3SnlGVHhzM2YvRWxvbExaN3JNTW5ncVVNamRk?=
 =?utf-8?B?aTdqWEZHQ3JnYmxkRVVyditmVU1JNlVEQXNPWE1FeVBoMURSRzVKc3VGUFdr?=
 =?utf-8?B?YTFOekRZa3QvbjZ3bU15K29ueUlqUjgwUW9JdlZqTkRLU1NIMDd0QWF2emNh?=
 =?utf-8?B?bTB6WlZwbE1BemVzUDBCVFBXbUlGaWh1NFR2QVE0clh2RTRFZ28xcWFwYmpk?=
 =?utf-8?B?NFpHV3MwRk1HQm1lVHRCVHZuTFdGdTBGWCtpdjNMN09sbzF6Mm1ua01WQ1pr?=
 =?utf-8?B?K2FOMHpaQXIrYS9YTENDWTNrMnRBbzV0VHU1emNzUXV0OExOY3h5S0ZRZGNK?=
 =?utf-8?B?OUFJeXQ0QzFkUVNHQWxXRjVjVWFCSGZCd01DbkNCVGNKRlZCUkRDckROTDN5?=
 =?utf-8?B?WHBUem50akFSN2tpUDI2aldmYzJkbEh3eTYrZjlSa0pjcThEK1piWG1YL0lt?=
 =?utf-8?B?c3FEWUdPZDlvUHBRZUFqM0syRFNSTXZTWHJSL3dLT1BvV0J0VEZCaUk0VW1S?=
 =?utf-8?B?dDJWTE5TWDJoMUpUMlJKQ3lFby91MXpXU2E3SkQ5NUFxMmlPZVlyUHc2bkFy?=
 =?utf-8?B?NEdSS1FiU2pMOE5vbHUvaFZZbUEvbkIxYlNsUWMvQTNaV3E0ekE5NFRUbEpp?=
 =?utf-8?B?cTZFNDkvQ0dKN2hjdnViUGZIODBDNVY5YUExYTErU3JXVWtDbG9USUg4ZS84?=
 =?utf-8?B?Ym1yL2Z3MzFoRUJNVWdsbTFmUitkY2pLaVBVOU85NTkzMlkwZjBLUmZDaGJp?=
 =?utf-8?B?RWtnT0xRQ2hraUpPQlc1ckdJaUppaUEwbnBhL0puTnQ0ajVUZG51UU1RM1No?=
 =?utf-8?B?eFdPSCt2UFY5cEkyTTN6eG9JdmpaamdYaUxSZXg2Q1dOVDg4Z3RTMDM2OXZ2?=
 =?utf-8?B?R1h1alhGNFdDdWVVSjVBWUVPZGM3M3lqaVBaazZCQWFvNDZjWCtVb0RvT2R0?=
 =?utf-8?B?Rmd0dHBDMHB0TVRMR0YxTWhXV0NxNmJhbEdFa0xCcUVqSnhPZmtvcFVjSDdV?=
 =?utf-8?B?Y2E5bm9PejlWMzZVUVhIdi94SE9WVHAxV0J1aXpiTmIwU2k3NitWcVJnamhI?=
 =?utf-8?B?RXhlMW16K1l4dmpRMldNejVRamQ2WGt2YXhyNWo5ZHhNS1BESlNvSmR4Smdt?=
 =?utf-8?B?MWJ6aDFFTkU4dnlkRmRMUmUrU3dsckRpVEpBd0hFdG8rOVVBbXpTelF6Znpr?=
 =?utf-8?B?UThrUGdGaVlEVUJKMUV4K0tIUzhnSXowcXRlWVJxNklzdWhhdGlmZWZyTW5i?=
 =?utf-8?B?OG93UGxxb05pZ1JtaU80RXFoRDhmc2puQU5iNTY1RVRuNElXaHB3ZXEveG5H?=
 =?utf-8?B?UW9Ta2ZGdWVJaVdtd1BVaVBmZzJXOEI4R2xjK05Xbldtdy9ibjZQTzQ3d3pH?=
 =?utf-8?B?T3U2dzRwYzNyUVM1MnBtaHArUUZYdjFCTWJ2NDV0Y1pidm5yemd5cmxtS2lK?=
 =?utf-8?B?QTdOdEx2WFhOaC8vRTJjTTQvekd3a08zWGJxb3JCTFBFWGx1OFNicS9YVnU1?=
 =?utf-8?B?QkpLbFQzczNUSm1JYXU4Q1E1cGpvbS9qUHIzQ1VKbU1VbFRpc3EvdER4M0JX?=
 =?utf-8?B?T0wxVzcrNTlzTlVkNDJnUHFCZ0FDdFJGQUFEdk4wNGlEdnQ3VythSTVlOWFT?=
 =?utf-8?B?d1E3anBhZGlQc1pjM3F2NHY2WWFIZlFOQjJ0V3o0ZSt6U01WSU9SMnRvZFVK?=
 =?utf-8?B?R1F5TDQ3K0w0OEx3ZExUU1gxcUxqV2lycDdRUW9UdXVzRm8wUTFDbWxVeWR3?=
 =?utf-8?B?bEd0VlJGMVpvalAxb3J4RnZ2RUdCd2FLTjNKYkNmaVlwSW9BQTRQaGdjdHJ4?=
 =?utf-8?B?Wi9zYzJ3TmpaMWFWd1ErZ0tha2xZUUpFc0c4OEtmUjBlTnhlNVo0ejBxR0Q4?=
 =?utf-8?B?QWllMGdmYzgrTGE5YmtCT2c0TGwwdmJ6Zk5NaGtzR1c2RDB1RnNOYzVjZVkw?=
 =?utf-8?B?cGFhNjRBdmltaFFpV3ZUUU91bkoraWUrT01RZHJjanA3RnZ0Nm1zbHRsTE9m?=
 =?utf-8?B?NTQ5MEFJQStjOXBYbzNTTFFUNWJjcmhPSWlJd2VGS201bjRNd2xpOE96RmVS?=
 =?utf-8?B?UWFxSWEyOHR3VkRuSzlTTTBMTyt2OE01aDFhcTJJZkVoNUFEdmhXdU55emQ3?=
 =?utf-8?Q?FleBc7yqzNcfAZT5b52tCHCr5?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ddb76728-7105-42d0-33a5-08dc32d0ebdc
X-MS-Exchange-CrossTenant-AuthSource: PH8PR12MB7326.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2024 11:33:27.4003
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: s852zYdR57eO/k1YQhSjxGS8d9fk2Yl8DUhg7jnfpJwY4pKFk5vvBPiI1iYfMMd1AWqcYcKCeRGWJo4HRgDUFg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN6PR12MB8490

Hi Jan,

On 21/02/2024 07:09, Jan Beulich wrote:
> On 20.02.2024 16:22, Ayan Kumar Halder wrote:
>> On 20/02/2024 12:33, Jan Beulich wrote:
>>> On 20.02.2024 13:17, Ayan Kumar Halder wrote:
>>>> --- a/SUPPORT.md
>>>> +++ b/SUPPORT.md
>>>> @@ -101,6 +101,18 @@ Extension to the GICv3 interrupt controller to support MSI.
>>>>    
>>>>        Status: Experimental
>>>>    
>>>> +### ARM/Partial Emulation
>>>> +
>>>> +Enable partial emulation of registers, otherwise considered unimplemented,
>>>> +that would normally trigger a fault injection.
>>>> +
>>>> +    Status: Supported, with caveats
>>>> +
>>>> +Bugs allowing the userspace to attack the guest OS will not be considered
>>>> +security vulnerabilities.
>>>> +
>>>> +Bugs that could compromise Xen will be considered security vulnerabilities.
>>> ... the odd statement regarding in-guest vulnerabilities that might be
>>> introduced. I see only two ways of treating this as supported: Either
>>> you simply refuse emulation when the access is from user space,
>> I am wondering how do we enforce that.
>>
>> Let me try to understand this with the current implementation of partial
>> emulation for system registers.
>>
>> 1. DBGDTRTX_EL0 :- I understand that EL0 access to this register will
>> cause a trap to EL2. The reason being MDCR_EL2.TDA == 1.
>>
>> In that case, if we refuse emulation then an undef exception is injected
>> to the guest (this is the behavior as of today even without this patch).
>>
>> So, are you saying that the undef exception is to be injected to the
>> user space process. This may be possible for Arm64 guests
>> (inject_undef64_exception() needs to be changed).
> No, injection is always to the guest, not to a specific entity within the
> guest. That ought to be the same on bare hardware: An exception when
> raised has an architecturally defined entry point for handling. That'll
> typically be kernel code. The handler then figures out whether the source
> of the exception was in user or kernel mode. For user mode code, the
> kernel may or may not try to handle the exception and then continue the
> user mode process. If it can't or doesn't want to handle it, it'll raise
> (in UNIX terms) a signal to the process. That signal, in turn, may or may
> not be fatal to the process. But such an exception from user mode should
> never be fatal to the guest as a whole.
Thanks for explaining it so well.
>
>> However for Arm32 guests, this may not be possible as the mode changes
>> to PSR_MODE_UND.
> I'm afraid my Arm foo isn't good enough to understand this. On the surface
> it looks to violate above outlined principle.
>
>> Let me know if I understood you correctly.
>>
>>> or you
>>> support that mode of emulation as much as that of kernel space accesses.
>> Do you mean we support partial emulation only for traps from EL1 mode ?
> Possibly.

Now, I understand you. We will allow partial_emulation only from kernel 
mode.

So we need to do sth :-

if ( 'partial_emulation == true' && '!regs_mode_is_user(regs)' )

{

      /* partial_emulation logic */

}

I am ok with this.

And the updates message will be

### ARM/Partial Emulation

Enable partial emulation of registers, otherwise considered unimplemented,
that would normally trigger a fault injection.

     Status: Supported, with caveats

Partial emulation for traps from userspace is not allowed.

Bugs that could compromise Xen will be considered security vulnerabilities.

Also, want @Julien, @Michal to comment on this.

- Ayan

>
> Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 12:09:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 12:09:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683993.1063618 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rclPC-00085y-3y; Wed, 21 Feb 2024 12:08:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683993.1063618; Wed, 21 Feb 2024 12:08:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rclPC-00085r-0k; Wed, 21 Feb 2024 12:08:54 +0000
Received: by outflank-mailman (input) for mailman id 683993;
 Wed, 21 Feb 2024 12:08:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QV5I=J6=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rclPB-00085l-1q
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 12:08:53 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f9f33532-d0b1-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 13:08:51 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 5BBD94EE0737;
 Wed, 21 Feb 2024 13:08:51 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9f33532-d0b1-11ee-8a53-1f161083a0e0
MIME-Version: 1.0
Date: Wed, 21 Feb 2024 13:08:51 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper3
 <andrew.cooper3@citrix.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, George
 Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
In-Reply-To: <45509cb67ecee3f690b5784489b5ccb4@bugseng.com>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
 <d90d98b6-508b-4a2a-ab6a-74a9828a5b94@suse.com>
 <45509cb67ecee3f690b5784489b5ccb4@bugseng.com>
Message-ID: <1743b4248d30a4e8b68a150c25724caa@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-20 09:14, Nicola Vetrini wrote:
> On 2024-02-20 08:45, Jan Beulich wrote:
>> On 19.02.2024 16:14, Nicola Vetrini wrote:
>>> The cache clearing and invalidation helpers in x86 and Arm didn't
>>> comply with MISRA C Rule 17.7: "The value returned by a function
>>> having non-void return type shall be used". On Arm they
>>> were always returning 0, while some in x86 returned -EOPNOTSUPP
>>> and in common/grant_table the return value is saved.
>>> 
>>> As a consequence, a common helper arch_grant_cache_flush that returns
>>> an integer is introduced, so that each architecture can choose 
>>> whether to
>>> return an error value on certain conditions, and the helpers have 
>>> either
>>> been changed to return void (on Arm) or deleted entirely (on x86).
>>> 
>>> Signed-off-by: Julien Grall <julien@xen.org>
>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>> ---
>>> The original refactor idea came from Julien Grall in [1]; I edited 
>>> that proposal
>>> to fix build errors.
>>> 
>>> I did introduce a cast to void for the call to flush_area_local on 
>>> x86, because
>>> even before this patch the return value of that function wasn't 
>>> checked in all
>>> but one use in x86/smp.c, and in this context the helper (perhaps 
>>> incidentally)
>>> ignored the return value of flush_area_local.
>> 
>> I object to such casting to void, at least until there's an overriding
>> decision that for Misra purposes such casts may be needed.
>> 
> 
> There are three choices here:
> 1. cast to void
> 2. deviation for flush_area_local, which for the case of the cache 
> helpers is what led to this patch; it may still be a viable option, if 
> other maintainers agree
> 3. refactor of flush_area_local; this is not viable here because the 
> return value is actually used and useful, as far as I can tell, in 
> smp.c
> 
>>> --- a/xen/arch/arm/include/asm/page.h
>>> +++ b/xen/arch/arm/include/asm/page.h
>>> @@ -123,6 +123,7 @@
>>> 
>>>  #ifndef __ASSEMBLY__
>>> 
>>> +#include <public/grant_table.h>
>> 
>> This is a no-go, imo (also on x86): Adding this include here 
>> effectively
>> means that nearly every CU will have a dependency on that header, no
>> matter that most are entirely agnostic of grants. Each arch has a
>> grant_table.h - is there any reason the new, grant-specific helper 
>> can't
>> be put there?
>> 
> 
> I would have to test, but I think that can be done
> 

The only blocker so far is that this triggers a build error due to a 
circular dependency between xen/mm.h and asm/flushtlb.h on x86. Also 
found some earlier evidence [1] that there are some oddities around 
asm/flushtlb's inclusion.

[1] 
https://lore.kernel.org/xen-devel/20200318210540.5602-1-andrew.cooper3@citrix.com/

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 12:20:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 12:20:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.683999.1063627 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rclac-0002Y3-3s; Wed, 21 Feb 2024 12:20:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 683999.1063627; Wed, 21 Feb 2024 12:20:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rclac-0002Xw-1C; Wed, 21 Feb 2024 12:20:42 +0000
Received: by outflank-mailman (input) for mailman id 683999;
 Wed, 21 Feb 2024 12:20:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rclaa-0002Xq-Im
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 12:20:40 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rclaW-00030k-N0; Wed, 21 Feb 2024 12:20:36 +0000
Received: from 54-240-197-234.amazon.com ([54.240.197.234]
 helo=[192.168.14.101]) by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rclaW-00058t-GT; Wed, 21 Feb 2024 12:20:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=hcj4fMldxUZ/Ct5Kb6EqPXji4TnklCL+6XMbONF2EBk=; b=K+2AmZVbTNzYs+KWiz2TWLD9+5
	gJFtDFqxSkTD0VFox+CmZ62QkjuUfgxzBhJEEobFSzwwvaBAqqkgQ0sBPXvsKBhxKaMLOXeN+UWbJ
	yZvLORplNiyDXBDbvalrB++1PXH3EImaGkm5qdZXPvqOh4AzJkwakFW2Jmi8oian4CqY=;
Message-ID: <10e8a485-61b7-4c9d-930f-975e25e8ec45@xen.org>
Date: Wed, 21 Feb 2024 12:20:34 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v5 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
Content-Language: en-GB
To: Ayan Kumar Halder <ayankuma@amd.com>, Jan Beulich <jbeulich@suse.com>,
 Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, michal.orzel@amd.com,
 xen-devel@lists.xenproject.org
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
 <20240220121743.3680715-2-ayan.kumar.halder@amd.com>
 <e3a8ce5d-841e-4ff1-8d23-48de822972ef@suse.com>
 <5f526deb-7376-4c34-85d1-d22b1b14d90e@amd.com>
 <e7c4b08f-724a-4419-977a-caa1342c47eb@suse.com>
 <62e047c5-a913-410d-9b24-e9a26d555d78@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <62e047c5-a913-410d-9b24-e9a26d555d78@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit



On 21/02/2024 11:33, Ayan Kumar Halder wrote:
> Hi Jan,
> 
> On 21/02/2024 07:09, Jan Beulich wrote:
>> On 20.02.2024 16:22, Ayan Kumar Halder wrote:
>>> On 20/02/2024 12:33, Jan Beulich wrote:
>>>> On 20.02.2024 13:17, Ayan Kumar Halder wrote:
>>>>> --- a/SUPPORT.md
>>>>> +++ b/SUPPORT.md
>>>>> @@ -101,6 +101,18 @@ Extension to the GICv3 interrupt controller to 
>>>>> support MSI.
>>>>>        Status: Experimental
>>>>> +### ARM/Partial Emulation
>>>>> +
>>>>> +Enable partial emulation of registers, otherwise considered 
>>>>> unimplemented,
>>>>> +that would normally trigger a fault injection.
>>>>> +
>>>>> +    Status: Supported, with caveats
>>>>> +
>>>>> +Bugs allowing the userspace to attack the guest OS will not be 
>>>>> considered
>>>>> +security vulnerabilities.
>>>>> +
>>>>> +Bugs that could compromise Xen will be considered security 
>>>>> vulnerabilities.
>>>> ... the odd statement regarding in-guest vulnerabilities that might be
>>>> introduced. I see only two ways of treating this as supported: Either
>>>> you simply refuse emulation when the access is from user space,
>>> I am wondering how do we enforce that.
>>>
>>> Let me try to understand this with the current implementation of partial
>>> emulation for system registers.
>>>
>>> 1. DBGDTRTX_EL0 :- I understand that EL0 access to this register will
>>> cause a trap to EL2. The reason being MDCR_EL2.TDA == 1.
>>>
>>> In that case, if we refuse emulation then an undef exception is injected
>>> to the guest (this is the behavior as of today even without this patch).
>>>
>>> So, are you saying that the undef exception is to be injected to the
>>> user space process. This may be possible for Arm64 guests
>>> (inject_undef64_exception() needs to be changed).
>> No, injection is always to the guest, not to a specific entity within the
>> guest. That ought to be the same on bare hardware: An exception when
>> raised has an architecturally defined entry point for handling. That'll
>> typically be kernel code. The handler then figures out whether the source
>> of the exception was in user or kernel mode. For user mode code, the
>> kernel may or may not try to handle the exception and then continue the
>> user mode process. If it can't or doesn't want to handle it, it'll raise
>> (in UNIX terms) a signal to the process. That signal, in turn, may or may
>> not be fatal to the process. But such an exception from user mode should
>> never be fatal to the guest as a whole.
> Thanks for explaining it so well.
>>
>>> However for Arm32 guests, this may not be possible as the mode changes
>>> to PSR_MODE_UND.
>> I'm afraid my Arm foo isn't good enough to understand this. On the 
>> surface
>> it looks to violate above outlined principle.
>>
>>> Let me know if I understood you correctly.
>>>
>>>> or you
>>>> support that mode of emulation as much as that of kernel space 
>>>> accesses.
>>> Do you mean we support partial emulation only for traps from EL1 mode ?
>> Possibly.
> 
> Now, I understand you. We will allow partial_emulation only from kernel 
> mode.
> 
> So we need to do sth :-
> 
> if ( 'partial_emulation == true' && '!regs_mode_is_user(regs)' )
> 
> {
> 
>       /* partial_emulation logic */
> 
> }
> 
> I am ok with this.

The helpers take a min_el. So you can have simpler code and set the 
value to 1 (rather than 0 today).

> 
> And the updates message will be
> 
> ### ARM/Partial Emulation
> 
> Enable partial emulation of registers, otherwise considered unimplemented,
> that would normally trigger a fault injection.
> 
>      Status: Supported, with caveats
> 
> Partial emulation for traps from userspace is not allowed.

I am not sure how this helps...
> 
> Bugs that could compromise Xen will be considered security vulnerabilities.

... you are still implying that any userland breaching the kernel will 
not be supported because of a bug.

But, I don't see how preventing the userland to access a register will 
help. In theory you could have a register that can only be accessed by 
EL1 but has an impact to EL0.

By definition, it means that we don't faithfully follow the Arm Arm and 
anything can happen. Which is why I wanted to exclude security support 
from userland and kernel.

I think the current registers are low risk. But I am under the 
impression that you may wan to add more partial emulation. So we need to 
make some sort of statement that doesn't put any burden to the security 
team for future registers.

An option would be to explicitly list the registers in SUPPORT.md. As I 
mentioned above, I think the registers you currently emulate are 
low-risk to introduce a security bug. So I would be ok to fully security 
support them. This would need to be re-assessed for new registers.

I am open to other suggestions.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 12:47:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 12:47:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684007.1063638 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcm0U-0005l6-60; Wed, 21 Feb 2024 12:47:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684007.1063638; Wed, 21 Feb 2024 12:47:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcm0U-0005kz-2i; Wed, 21 Feb 2024 12:47:26 +0000
Received: by outflank-mailman (input) for mailman id 684007;
 Wed, 21 Feb 2024 12:47:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tO1j=J6=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rcm0S-0005ka-6c
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 12:47:24 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5afc927b-d0b7-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 13:47:22 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d10ad265d5so72467941fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 04:47:22 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 w18-20020a05651c103200b002d250e323edsm378476ljm.89.2024.02.21.04.47.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 21 Feb 2024 04:47:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5afc927b-d0b7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708519641; x=1709124441; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=bh3Pt3NERQ8sAQrTrTMndX28uy4hyOof3AfoSJBo/MM=;
        b=Qo1KDwsaiGmsLIiWlAVhINFfm0nURxa8kh20o6uzGu355QtHAisLfvuuwxAai+/0ku
         Xuou5oZwUTZUxl5iI8UdLES/6/LuMFlTK/Ii6YUtjbysoPwKNhdfe4AWg8jdylxQp2a/
         shkhi3IqQzho5wNIBTyIiJVfPnRxCICwSxXR5sb6VyWmzIdhzzsC19fxX35vIMpXMz7s
         lfxFmsLwsJxtCeJWSYVDcUGWc/KWur6+SWcSq/tNz7zRzbog8zxNGSwAthX/fyVWd7LW
         EKM1WmKYa2fHDrlXqvcO3e/kPp/xiP5CNon71MhV026F3Js20BD2dybFBnvKVZKkiT+/
         92ZQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708519641; x=1709124441;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bh3Pt3NERQ8sAQrTrTMndX28uy4hyOof3AfoSJBo/MM=;
        b=XGOQTVBRVY8iO7H6gfAbmK3YU2SnepYCNfs1pQjNp16NfCuQPh/bQcVS+3t2gLijLz
         FsCXGyqd8XCYFv4WzAPhbJTi8ZzwJ4U/sDUhaQJ7dkhLjeE89xl6EtpHC2mI4cXSO90P
         +8vOTMlql45EVHSgDvk+sg4JdNIZKenP1MuygNK1rjBEA3FdcVBOc5wRlkuG9CxPSHYO
         1sL1anAs+ESyAEVtjpjU6ZOEhQnkDPsemZX4yL6r5DJ+/tIyhYHU8BQvkHS8xOeUKb3g
         zeOladn9my+ti6S+JJQ6as5fBFIi1ucr4IfwFoW80Y1vNPNySGO+SjUZvQ5uXYicA+Dm
         Eufg==
X-Forwarded-Encrypted: i=1; AJvYcCVBQJXeK9jrEvJsYoMXAyNzpzBH5UrRqZFK9n3adDfOKwGoeWl0o1aRcODndH+ZF2V6FAmrh9yjzzo2OxWPe0a4dtGp/zIiH7vw2OClEYM=
X-Gm-Message-State: AOJu0Yz48gY6Mt2BDZTUaz5auA4SDXZ02oEmDBGrkpDzirFGs0lLW3Zp
	yKNj+SYphw5peMink7oG9hL0hEgsh3uvOxcgmBJEz/KTaA9eHFeV
X-Google-Smtp-Source: AGHT+IHEv2NoRIaXgY/O+6NofEY9+X1hHJnSiU3+cHQnm/kUqzszER7JKjArHoeyorpnk3znuotcYQ==
X-Received: by 2002:a05:651c:1043:b0:2d2:329f:7a7e with SMTP id x3-20020a05651c104300b002d2329f7a7emr7477351ljm.6.1708519641080;
        Wed, 21 Feb 2024 04:47:21 -0800 (PST)
Message-ID: <3e0c83fd0e3dc424059575cf9da9d57a87d90736.camel@gmail.com>
Subject: Re: [PATCH v4 07/30] xen/asm-generic: introdure nospec.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
 <bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Wei
 Liu <wl@xen.org>, Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,  Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, 
 xen-devel@lists.xenproject.org
Date: Wed, 21 Feb 2024 13:47:19 +0100
In-Reply-To: <a6394d8a-63d0-42bf-8ed7-a7722cb7e71c@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <5889d7a5fa81722472f95cc1448af0be8f359a7d.1707146506.git.oleksii.kurochko@gmail.com>
	 <510d74ca-5cce-48f8-93f6-407cd727cf7b@xen.org>
	 <8a09e386b709f780f193af39af63b6aeb75c868e.camel@gmail.com>
	 <aaf86d31-7ae0-4e33-8386-dda4bd21496a@suse.com>
	 <ab040f3c8ee35f49fffac385053b55053c52da8c.camel@gmail.com>
	 <a6394d8a-63d0-42bf-8ed7-a7722cb7e71c@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Wed, 2024-02-21 at 12:00 +0100, Jan Beulich wrote:
> On 20.02.2024 21:30, Oleksii wrote:
> > On Mon, 2024-02-19 at 13:18 +0100, Jan Beulich wrote:
> > > On 19.02.2024 12:59, Oleksii wrote:
> > > > Hi Julien,
> > > >=20
> > > > On Sun, 2024-02-18 at 18:30 +0000, Julien Grall wrote:
> > > > > Hi Oleksii,
> > > > >=20
> > > > > Title: Typo s/introdure/introduce/
> > > > >=20
> > > > > On 05/02/2024 15:32, Oleksii Kurochko wrote:
> > > > > > The <asm/nospec.h> header is similar between Arm, PPC, and
> > > > > > RISC-V,
> > > > > > so it has been moved to asm-generic.
> > > > >=20
> > > > > I am not 100% convinced that moving this header to asm-
> > > > > generic is
> > > > > a
> > > > > good=20
> > > > > idea. At least for Arm, those helpers ought to be non-empty,
> > > > > what
> > > > > about=20
> > > > > RISC-V?
> > > > For Arm, they are not taking any action, are they? There are no
> > > > specific fences or other mechanisms inside
> > > > evaluate_nospec()/block_speculation() to address speculation.
> > >=20
> > > The question isn't the status quo, but how things should be
> > > looking
> > > like
> > > if everything was in place that's (in principle) needed.
> > >=20
> > > > For RISC-V, it can be implemented in a similar manner, at least
> > > > for
> > > > now. Since these functions are only used in the grant tables
> > > > code (
> > > > for
> > > > Arm and so for RISC-V ), which is not supported by RISC-V.
> > >=20
> > > Same here - the question is whether long term, when gnttab is
> > > also
> > > supported, RISC-V would get away without doing anything. Still
> > > ...
> > >=20
> > > > > If the answer is they should be non-empty. Then I would
> > > > > consider
> > > > > to
> > > > > keep=20
> > > > > the duplication to make clear that each architecture should
> > > > > take
> > > > > their=20
> > > > > own decision in term of security.
> > > > >=20
> > > > > The alternative, is to have a generic implementation that is
> > > > > safe
> > > > > by=20
> > > > > default (if that's even possible).
> > > > I am not certain that we can have a generic implementation, as
> > > > each
> > > > architecture may have specific speculation issues.
> > >=20
> > > ... it's theoretically possible that there'd be an arch with no
> > > speculation issues, maybe simply because of not speculating.
> >=20
> > I am not sure that understand your and Julien point.
> >=20
> > For example, modern CPU uses speculative execution to reduce the
> > cost
> > of conditional branch instructions using schemes that predict the
> > execution path of a program based on the history of branch
> > executions.
> >=20
> > Arm CPUs are vulnerable for speculative execution, but if to look
> > at
> > the code of evaluate_nospec()/block_speculation() functions they
> > are
> > doing nothing for Arm.
>=20
> Which, as I understood Julien say, likely isn't correct. In which
> case
> this header shouldn't be dropped, using the generic one instead. The
> generic headers, as pointed out several times before, should imo be
> used
> only if their use results in correct behavior. What is acceptable is
> if
> their use results in sub-optimal behavior (e.g. reduced performance
> or
> lack of a certain feature that another architecture maybe
> implements).

As I understand it, evaluate_nospec()/block_speculation() were
introduced for x86 to address the L1TF vulnerability specific to x86
CPUs. This vulnerability is exclusive to x86 architectures [1], which
explains why evaluate_nospec()/block_speculation() are left empty for
Arm, RISC-V, and PPC.

It is unclear whether these functions should be utilized to mitigate
other speculation vulnerabilities. If they should, then, based on the
current implementation, the Arm platform seems to accept having
speculative vulnerabilities.

The question arises: why can't other architectures make their own
decisions regarding security? By default, if an architecture leaves the
mentioned functions empty, it implies an agreement to potentially have
speculative vulnerabilities. Subsequently, if an architecture needs to
address such vulnerabilities, they can develop arch-specific nospec.h
to implement the required code.

If reaching an agreement to potentially have speculative
vulnerabilities is deemed unfavorable, I am open to reconsidering these
changes and reverting to the use of arch-specific nospec.h.
Your input on this matter is appreciated.

[1] https://docs.kernel.org/admin-guide/hw-vuln/l1tf.html



From xen-devel-bounces@lists.xenproject.org Wed Feb 21 12:59:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 12:59:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684015.1063668 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmBh-00085O-QY; Wed, 21 Feb 2024 12:59:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684015.1063668; Wed, 21 Feb 2024 12:59:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmBh-00085G-Mt; Wed, 21 Feb 2024 12:59:01 +0000
Received: by outflank-mailman (input) for mailman id 684015;
 Wed, 21 Feb 2024 12:59:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RCxi=J6=bombadil.srs.infradead.org=BATV+bafd7931fd2c4139f05c+7486+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rcmBg-00082S-KX
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 12:59:00 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f9a37b58-d0b8-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 13:58:58 +0100 (CET)
Received: from
 2a02-8389-2341-5b80-39d3-4735-9a3c-88d8.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:39d3:4735:9a3c:88d8] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rcmBb-00000000wEZ-31In; Wed, 21 Feb 2024 12:58:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9a37b58-d0b8-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:
	From:Sender:Reply-To:Cc:Content-ID:Content-Description;
	bh=AJ2bO5DndZrmPq3Dd1zge+YVJmxcGzOjpW+1dV9o8hU=; b=0UI5+v3RaoEG0RwA7cWThgCcBh
	esLhJcrhMRUggkzC3n/4WDY0UPLEB3eIPfNGc5SfzVyCRwnouge7req2d+k03fvIdw33zXtgsi5N4
	SZGC0hXnZr/F5F7mVsOi7dpCqpk+Mq9cOndOQxcfPMFQ4ASMLkvPk1XrXCqxdTVSraaJ9eTxWDEWs
	EKoQDODwTqxpDQD+A+1HdUu6hSb8QnA+pWAlYPN2+tlhTtdm2dQrPOCI335+Xq60VKFT0y1pGHBZC
	pJh9NMBWck+arfFAIvRkghaw/C0wKXw5g2H2GBmr5DGLBJexBghbC6ZrNWRZOQYDcbqnNl3e2Qlz2
	OUPaQu3g==;
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>,
	xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: [PATCH 2/4] xen-blkfront: rely on the default discard granularity
Date: Wed, 21 Feb 2024 13:58:43 +0100
Message-Id: <20240221125845.3610668-3-hch@lst.de>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240221125845.3610668-1-hch@lst.de>
References: <20240221125845.3610668-1-hch@lst.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

The block layer now sets the discard granularity to the physical
block size default.  Take advantage of that in xen-blkfront and only
set the discard granularity if explicitly specified.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
---
 drivers/block/xen-blkfront.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index f78167cd5a6333..1258f24b285500 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -951,8 +951,8 @@ static void blkif_set_queue_limits(struct blkfront_info *info)
 
 	if (info->feature_discard) {
 		blk_queue_max_discard_sectors(rq, UINT_MAX);
-		rq->limits.discard_granularity = info->discard_granularity ?:
-						 info->physical_sector_size;
+		if (info->discard_granularity)
+			rq->limits.discard_granularity = info->discard_granularity;
 		rq->limits.discard_alignment = info->discard_alignment;
 		if (info->feature_secdiscard)
 			blk_queue_max_secure_erase_sectors(rq, UINT_MAX);
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed Feb 21 12:59:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 12:59:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684013.1063647 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmBf-0007bp-4q; Wed, 21 Feb 2024 12:58:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684013.1063647; Wed, 21 Feb 2024 12:58:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmBf-0007bi-1y; Wed, 21 Feb 2024 12:58:59 +0000
Received: by outflank-mailman (input) for mailman id 684013;
 Wed, 21 Feb 2024 12:58:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RCxi=J6=bombadil.srs.infradead.org=BATV+bafd7931fd2c4139f05c+7486+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rcmBc-0007bV-4N
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 12:58:57 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f5b8fee7-d0b8-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 13:58:52 +0100 (CET)
Received: from
 2a02-8389-2341-5b80-39d3-4735-9a3c-88d8.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:39d3:4735:9a3c:88d8] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rcmBU-00000000wAK-1ase; Wed, 21 Feb 2024 12:58:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5b8fee7-d0b8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	MIME-Version:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:
	Content-ID:Content-Description:In-Reply-To:References;
	bh=TLqbe5eB45zE6JDeJk2bwZfFOKKXQJ5aMNlrvDgw0Fs=; b=TLLFdF3hIPpuyvuEvHNUa/tPbJ
	D3wPfOyi+liltFjp06FdZVYZl+ndpbqTNHOg5WaQMvN51FsQpycpPo3LWgpFdF5ECcChTDVyXrmEx
	G1cdzOrAuFLGwBEL/r69hIkvu31pidcEkYJMDDUfndvC8oyVBi16xG3UXLKpsBq+dqTmlN7rbEqlC
	ML7OCIydIMJk4sFKe53MLlbOxWekXLOyGWNXZJKO3aU9WvULwUnlAVVHC9/N3saQ4RxUxI2lB3rFv
	/8Z80oV7eGgtrXTGM9ZJsi/1kgv6jg404yDrtwwkh5dQWFxSHtp2sQZXyQkYMeuZZWcn8SS1IXva9
	3TFHREWg==;
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>,
	xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: convert xen-blkfront to atomic queue limit updates v2
Date: Wed, 21 Feb 2024 13:58:41 +0100
Message-Id: <20240221125845.3610668-1-hch@lst.de>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Hi all,

this series converts xen-blkfront to the new atomic queue limits update
API in the block tree.  I don't have a Xen setup so this is compile
tested only.

Changes since v1:
 - constify the info argument to blkif_set_queue_limits
 - remove a spurious word from a commit message

Diffstat:
 xen-blkfront.c |   53 +++++++++++++++++++++++++++--------------------------
 1 file changed, 27 insertions(+), 26 deletions(-)


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 12:59:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 12:59:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684014.1063654 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmBf-0007f3-F5; Wed, 21 Feb 2024 12:58:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684014.1063654; Wed, 21 Feb 2024 12:58:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmBf-0007er-9b; Wed, 21 Feb 2024 12:58:59 +0000
Received: by outflank-mailman (input) for mailman id 684014;
 Wed, 21 Feb 2024 12:58:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RCxi=J6=bombadil.srs.infradead.org=BATV+bafd7931fd2c4139f05c+7486+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rcmBd-0007bV-W7
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 12:58:57 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f7cf6c8e-d0b8-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 13:58:55 +0100 (CET)
Received: from
 2a02-8389-2341-5b80-39d3-4735-9a3c-88d8.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:39d3:4735:9a3c:88d8] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rcmBY-00000000wCC-3HMp; Wed, 21 Feb 2024 12:58:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7cf6c8e-d0b8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:
	From:Sender:Reply-To:Cc:Content-ID:Content-Description;
	bh=YF5q7+3VG37xUS7LPqZfsPI94l1e0iY4W/WI3XMTmts=; b=dkELOht3p/0mBMKM01rj7u26/q
	qcbpOHS8c81tO6wKhp3hfJBcoa9XaxFluK9tPHejblLd+YoNybr/5oUbaCH6Rz6uiXfPWnQRhCaCI
	8JvnTbk3uqKcRVo8gGzY4kSYXPC2O9d1ni5VzzUVcyzyvc8B6Z7xGk5dPY1+W6x7wI+hTqhgT6106
	3fsxLlpNBthXWs28REaKn48G5qJyHe2DJluaZaYrWr1D7R3zkPF91U0ZlZoCG2dBNgE3U6bNPOHfB
	q7LUzpc1YJEJS+PLzs6CgoNCN9j/YbATIzIf7LVo1thF4XI+Uwjv1CP8TJge/Xr1yokbgtT25ftf8
	hqSdMJ2g==;
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>,
	xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: [PATCH 1/4] xen-blkfront: set max_discard/secure erase limits to UINT_MAX
Date: Wed, 21 Feb 2024 13:58:42 +0100
Message-Id: <20240221125845.3610668-2-hch@lst.de>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240221125845.3610668-1-hch@lst.de>
References: <20240221125845.3610668-1-hch@lst.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Currently xen-blkfront set the max discard limit to the capacity of
the device, which is suboptimal when the capacity changes.  Just set
it to UINT_MAX, which has the same effect and is simpler.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
---
 drivers/block/xen-blkfront.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 4cc2884e748463..f78167cd5a6333 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -944,20 +944,18 @@ static const struct blk_mq_ops blkfront_mq_ops = {
 static void blkif_set_queue_limits(struct blkfront_info *info)
 {
 	struct request_queue *rq = info->rq;
-	struct gendisk *gd = info->gd;
 	unsigned int segments = info->max_indirect_segments ? :
 				BLKIF_MAX_SEGMENTS_PER_REQUEST;
 
 	blk_queue_flag_set(QUEUE_FLAG_VIRT, rq);
 
 	if (info->feature_discard) {
-		blk_queue_max_discard_sectors(rq, get_capacity(gd));
+		blk_queue_max_discard_sectors(rq, UINT_MAX);
 		rq->limits.discard_granularity = info->discard_granularity ?:
 						 info->physical_sector_size;
 		rq->limits.discard_alignment = info->discard_alignment;
 		if (info->feature_secdiscard)
-			blk_queue_max_secure_erase_sectors(rq,
-							   get_capacity(gd));
+			blk_queue_max_secure_erase_sectors(rq, UINT_MAX);
 	}
 
 	/* Hard sector size and max sectors impersonate the equiv. hardware. */
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed Feb 21 12:59:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 12:59:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684016.1063678 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmBk-0008M9-19; Wed, 21 Feb 2024 12:59:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684016.1063678; Wed, 21 Feb 2024 12:59:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmBj-0008Lv-U5; Wed, 21 Feb 2024 12:59:03 +0000
Received: by outflank-mailman (input) for mailman id 684016;
 Wed, 21 Feb 2024 12:59:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RCxi=J6=bombadil.srs.infradead.org=BATV+bafd7931fd2c4139f05c+7486+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rcmBi-00082S-Fl
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 12:59:02 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fbb219fa-d0b8-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 13:59:02 +0100 (CET)
Received: from
 2a02-8389-2341-5b80-39d3-4735-9a3c-88d8.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:39d3:4735:9a3c:88d8] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rcmBf-00000000wFJ-0P7d; Wed, 21 Feb 2024 12:58:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fbb219fa-d0b8-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:
	From:Sender:Reply-To:Cc:Content-ID:Content-Description;
	bh=2bzGn47cjUdgN+x/FL0i0dqH9SVUpthf30bI0lT03/8=; b=XeCr1flD8EXP6yvVh2cqn3wx1r
	cpqVTcboC5Q7w6b9DjTaqoQQVy2vjQ3SsPPdLq6P1FpHNVzOqrR4WOj9+RIbWgwbJKzs9DjwHhOog
	dR8jo0Q5bmgAuH/eqJD6sZYXkafMp8LYbF7U8VtO/5NwXtQymU6GqHIouoYxytvNIFpYv7NbtKiH/
	ScPvL4Yn1BCrYfXn4CYGc2DDSZf8IlB4rASU66L3wde0FZtfA7WCGheu9dnmYxoO6q5v+cAFUJPkJ
	o5LWzv45IGz7c9Asx7N53BismN8ObaC/zgUuaCaBFQXvIdGE5FzHNdGJCLSNr3ISXDED+Hi0HZoZL
	jE5LCtYQ==;
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>,
	xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: [PATCH 3/4] xen-blkfront: don't redundantly set max_sements in blkif_recover
Date: Wed, 21 Feb 2024 13:58:44 +0100
Message-Id: <20240221125845.3610668-4-hch@lst.de>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240221125845.3610668-1-hch@lst.de>
References: <20240221125845.3610668-1-hch@lst.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

blkif_set_queue_limits already sets the max_sements limits, so don't do
it a second time.  Also remove a comment about a long fixe bug in
blk_mq_update_nr_hw_queues.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
---
 drivers/block/xen-blkfront.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 1258f24b285500..7664638a0abbfa 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -2008,14 +2008,10 @@ static int blkif_recover(struct blkfront_info *info)
 	struct request *req, *n;
 	int rc;
 	struct bio *bio;
-	unsigned int segs;
 	struct blkfront_ring_info *rinfo;
 
 	blkfront_gather_backend_features(info);
-	/* Reset limits changed by blk_mq_update_nr_hw_queues(). */
 	blkif_set_queue_limits(info);
-	segs = info->max_indirect_segments ? : BLKIF_MAX_SEGMENTS_PER_REQUEST;
-	blk_queue_max_segments(info->rq, segs / GRANTS_PER_PSEG);
 
 	for_each_rinfo(info, rinfo, r_index) {
 		rc = blkfront_setup_indirect(rinfo);
@@ -2035,7 +2031,9 @@ static int blkif_recover(struct blkfront_info *info)
 	list_for_each_entry_safe(req, n, &info->requests, queuelist) {
 		/* Requeue pending requests (flush or discard) */
 		list_del_init(&req->queuelist);
-		BUG_ON(req->nr_phys_segments > segs);
+		BUG_ON(req->nr_phys_segments >
+		       (info->max_indirect_segments ? :
+			BLKIF_MAX_SEGMENTS_PER_REQUEST));
 		blk_mq_requeue_request(req, false);
 	}
 	blk_mq_start_stopped_hw_queues(info->rq, true);
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed Feb 21 12:59:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 12:59:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684017.1063689 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmBo-0000HO-Dn; Wed, 21 Feb 2024 12:59:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684017.1063689; Wed, 21 Feb 2024 12:59:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmBo-0000H9-71; Wed, 21 Feb 2024 12:59:08 +0000
Received: by outflank-mailman (input) for mailman id 684017;
 Wed, 21 Feb 2024 12:59:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=RCxi=J6=bombadil.srs.infradead.org=BATV+bafd7931fd2c4139f05c+7486+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rcmBm-0007bV-D7
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 12:59:06 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fd65de6c-d0b8-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 13:59:04 +0100 (CET)
Received: from
 2a02-8389-2341-5b80-39d3-4735-9a3c-88d8.cable.dynamic.v6.surfer.at
 ([2a02:8389:2341:5b80:39d3:4735:9a3c:88d8] helo=localhost)
 by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux))
 id 1rcmBi-00000000wH4-0Pli; Wed, 21 Feb 2024 12:59:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd65de6c-d0b8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:
	Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:
	From:Sender:Reply-To:Cc:Content-ID:Content-Description;
	bh=ClBF4HdQdZoudt8WElKT+rvsm/9JyMAWyPnUsdGhZ5c=; b=ry/NuscqMOGf4bi5P73rmAi3BH
	HPoTpm+PFte7RXlBzNZupzOIuXurSaaiEbm7qy2KuV5KMgJL3CPZuQMX1HLYQyx/jVc+0q+oXo8fo
	Nh2XDSwkDhKLq7tbFoWLG3kjAQH9wMcD73ZcKyntH/or0abk3OAITizpF0SN9pdXfOeWQ54zRv5CQ
	DPgkDcE6AjceQ9WSGqjn65FdYSPyX9yfkdIUQXV0t3IW9gUy9aNDzxLlYjGDsAADdGNpJScdisyXl
	r/WFMicA2TDsJU1Wmk49a8HpuK1aXbcTK24bvDfDQcI1BmbfauN+fS4N4/BSWPNHMjmO/jDL3yAFv
	eJI5G3hA==;
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>,
	xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: [PATCH 4/4] xen-blkfront: atomically update queue limits
Date: Wed, 21 Feb 2024 13:58:45 +0100
Message-Id: <20240221125845.3610668-5-hch@lst.de>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240221125845.3610668-1-hch@lst.de>
References: <20240221125845.3610668-1-hch@lst.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

Pass the initial queue limits to blk_mq_alloc_disk and use the
blkif_set_queue_limits API to update the limits on reconnect.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
---
 drivers/block/xen-blkfront.c | 41 ++++++++++++++++++++----------------
 1 file changed, 23 insertions(+), 18 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 7664638a0abbfa..fd7c0ff2139cee 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -941,37 +941,35 @@ static const struct blk_mq_ops blkfront_mq_ops = {
 	.complete = blkif_complete_rq,
 };
 
-static void blkif_set_queue_limits(struct blkfront_info *info)
+static void blkif_set_queue_limits(const struct blkfront_info *info,
+		struct queue_limits *lim)
 {
-	struct request_queue *rq = info->rq;
 	unsigned int segments = info->max_indirect_segments ? :
 				BLKIF_MAX_SEGMENTS_PER_REQUEST;
 
-	blk_queue_flag_set(QUEUE_FLAG_VIRT, rq);
-
 	if (info->feature_discard) {
-		blk_queue_max_discard_sectors(rq, UINT_MAX);
+		lim->max_hw_discard_sectors = UINT_MAX;
 		if (info->discard_granularity)
-			rq->limits.discard_granularity = info->discard_granularity;
-		rq->limits.discard_alignment = info->discard_alignment;
+			lim->discard_granularity = info->discard_granularity;
+		lim->discard_alignment = info->discard_alignment;
 		if (info->feature_secdiscard)
-			blk_queue_max_secure_erase_sectors(rq, UINT_MAX);
+			lim->max_secure_erase_sectors = UINT_MAX;
 	}
 
 	/* Hard sector size and max sectors impersonate the equiv. hardware. */
-	blk_queue_logical_block_size(rq, info->sector_size);
-	blk_queue_physical_block_size(rq, info->physical_sector_size);
-	blk_queue_max_hw_sectors(rq, (segments * XEN_PAGE_SIZE) / 512);
+	lim->logical_block_size = info->sector_size;
+	lim->physical_block_size = info->physical_sector_size;
+	lim->max_hw_sectors = (segments * XEN_PAGE_SIZE) / 512;
 
 	/* Each segment in a request is up to an aligned page in size. */
-	blk_queue_segment_boundary(rq, PAGE_SIZE - 1);
-	blk_queue_max_segment_size(rq, PAGE_SIZE);
+	lim->seg_boundary_mask = PAGE_SIZE - 1;
+	lim->max_segment_size = PAGE_SIZE;
 
 	/* Ensure a merged request will fit in a single I/O ring slot. */
-	blk_queue_max_segments(rq, segments / GRANTS_PER_PSEG);
+	lim->max_segments = segments / GRANTS_PER_PSEG;
 
 	/* Make sure buffer addresses are sector-aligned. */
-	blk_queue_dma_alignment(rq, 511);
+	lim->dma_alignment = 511;
 }
 
 static const char *flush_info(struct blkfront_info *info)
@@ -1068,6 +1066,7 @@ static int xlvbd_alloc_gendisk(blkif_sector_t capacity,
 		struct blkfront_info *info, u16 sector_size,
 		unsigned int physical_sector_size)
 {
+	struct queue_limits lim = {};
 	struct gendisk *gd;
 	int nr_minors = 1;
 	int err;
@@ -1134,11 +1133,13 @@ static int xlvbd_alloc_gendisk(blkif_sector_t capacity,
 	if (err)
 		goto out_release_minors;
 
-	gd = blk_mq_alloc_disk(&info->tag_set, NULL, info);
+	blkif_set_queue_limits(info, &lim);
+	gd = blk_mq_alloc_disk(&info->tag_set, &lim, info);
 	if (IS_ERR(gd)) {
 		err = PTR_ERR(gd);
 		goto out_free_tag_set;
 	}
+	blk_queue_flag_set(QUEUE_FLAG_VIRT, gd->queue);
 
 	strcpy(gd->disk_name, DEV_NAME);
 	ptr = encode_disk_name(gd->disk_name + sizeof(DEV_NAME) - 1, offset);
@@ -1160,7 +1161,6 @@ static int xlvbd_alloc_gendisk(blkif_sector_t capacity,
 	info->gd = gd;
 	info->sector_size = sector_size;
 	info->physical_sector_size = physical_sector_size;
-	blkif_set_queue_limits(info);
 
 	xlvbd_flush(info);
 
@@ -2004,14 +2004,19 @@ static int blkfront_probe(struct xenbus_device *dev,
 
 static int blkif_recover(struct blkfront_info *info)
 {
+	struct queue_limits lim;
 	unsigned int r_index;
 	struct request *req, *n;
 	int rc;
 	struct bio *bio;
 	struct blkfront_ring_info *rinfo;
 
+	lim = queue_limits_start_update(info->rq);
 	blkfront_gather_backend_features(info);
-	blkif_set_queue_limits(info);
+	blkif_set_queue_limits(info, &lim);
+	rc = queue_limits_commit_update(info->rq, &lim);
+	if (rc)
+		return rc;
 
 	for_each_rinfo(info, rinfo, r_index) {
 		rc = blkfront_setup_indirect(rinfo);
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Wed Feb 21 13:38:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 13:38:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684044.1063697 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmnu-0006fE-Am; Wed, 21 Feb 2024 13:38:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684044.1063697; Wed, 21 Feb 2024 13:38:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcmnu-0006f7-7m; Wed, 21 Feb 2024 13:38:30 +0000
Received: by outflank-mailman (input) for mailman id 684044;
 Wed, 21 Feb 2024 13:38:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c6My=J6=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rcmnt-0006f1-KR
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 13:38:29 +0000
Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com
 [2a00:1450:4864:20::42b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7ec93c9b-d0be-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 14:38:28 +0100 (CET)
Received: by mail-wr1-x42b.google.com with SMTP id
 ffacd0b85a97d-33d7b8f563eso571746f8f.0
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 05:38:28 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 x4-20020adff644000000b0033b792ed609sm17038166wrp.91.2024.02.21.05.38.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 21 Feb 2024 05:38:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ec93c9b-d0be-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708522708; x=1709127508; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=kdDF83QJKZeIDA4jDwU37uslvlpmH0PGAxQ9Ot6TN1o=;
        b=Q4Cn0lThHG53fWksVNObIcXpdTBIkPiBfhYbmXEC/VDjom6AcT4baSAeWjlIsMLpji
         cbRr0CzCqb4bEH4gNiu8E5KEmvJdFsnRZ4GGPyVE1SQNkp1lUgxBeWp4bAvZWoVw6ntL
         LKu3hqLLTdY6IgRNlfWBA02dH14Zh8RlcmdJQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708522708; x=1709127508;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=kdDF83QJKZeIDA4jDwU37uslvlpmH0PGAxQ9Ot6TN1o=;
        b=JhnsDIrX2QDXsC0/w8anMrImQpqiWbSPdZspsf4ndzNlGhRJci1iCmnB6fS7C0PR+o
         YkwfkyvBr0Em7LTAnF+ojCcjd/TMrVdyI0GxdSu0Bk8qPst+YbQwAQti4y/x9v2lH6n9
         FMBIfZgCcFKBNXCr7utW0Wji2vfF+8ciV/Oylk3Vq5XnsHTK/HF8uDt+DsRa4q9Q7xHo
         bOwAQeoC8bzkn983YOF0JhMhDqPA6K3ntdFDRMmY/j/lzbEd0iBd9HoeonWFUobBBpcJ
         JDmxi1OfwsaCycz8XwtKfEq3AxaeOoVhWjIORtBkYwet1tR0ji1E/GB7Zp75OQZkJwfe
         ooKQ==
X-Gm-Message-State: AOJu0YzINGfHI+4BdcGA4uGF5rxcSo6hKByvDvNSa602fbXJazn0GbJE
	G3DiXRuTHGK3lm2ygGBaUNmcNbnEQTN5yPq4Ycb7m3rtP/iBhByk08xFNpcH4Ng=
X-Google-Smtp-Source: AGHT+IEJyxJyicierypr9dbfpGgT7XCWyOfWZf7N9RonvyfyOqT1Ef5YI7SlhA9vApHVDJVDQQ859A==
X-Received: by 2002:adf:f44f:0:b0:33d:5f6a:60e3 with SMTP id f15-20020adff44f000000b0033d5f6a60e3mr6281581wrp.43.1708522708070;
        Wed, 21 Feb 2024 05:38:28 -0800 (PST)
Date: Wed, 21 Feb 2024 13:38:27 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Jiqian Chen <Jiqian.Chen@amd.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	"Daniel P . Smith" <dpsmith@apertussolutions.com>,
	Stewart Hildebrand <Stewart.Hildebrand@amd.com>,
	Huang Rui <Ray.Huang@amd.com>
Subject: Re: [RFC XEN PATCH v5 4/5] libxl: Use gsi instead of irq for mapping
 pirq
Message-ID: <cf52f61b-1cf2-48e6-a3ba-7cdb12adacfa@perard>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-5-Jiqian.Chen@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240112061317.418658-5-Jiqian.Chen@amd.com>

Hi Jiqian,

On Fri, Jan 12, 2024 at 02:13:16PM +0800, Jiqian Chen wrote:
> In PVH dom0, it uses the linux local interrupt mechanism,
> when it allocs irq for a gsi, it is dynamic, and follow
> the principle of applying first, distributing first. And
> the irq number is alloced from small to large, but the
> applying gsi number is not, may gsi 38 comes before gsi
> 28, that causes the irq number is not equal with the gsi
> number. And when passthrough a device, xl wants to use
> gsi to map pirq, see pci_add_dm_done->xc_physdev_map_pirq,
> but the gsi number is got from file
> /sys/bus/pci/devices/<sbdf>/irq in current code, so it
> will fail when mapping.
> 
> So, use real gsi number read from gsi sysfs.
> 
> Co-developed-by: Huang Rui <ray.huang@amd.com>

The "Co-developed-by" tag isn't really defined in Xen, it's fine to keep
I guess, but it mean that another tag is missing, I'm pretty sure you
need to add "Signed-off-by: Huang Rui <ray.huang@amd.com>"

Beyond that, the patch looks good, I've only coding style issues.

> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
>  tools/libs/light/libxl_pci.c | 25 +++++++++++++++++++++++--
>  1 file changed, 23 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
> index 96cb4da0794e..a1c6e82631e9 100644
> --- a/tools/libs/light/libxl_pci.c
> +++ b/tools/libs/light/libxl_pci.c
> @@ -1478,8 +1478,19 @@ static void pci_add_dm_done(libxl__egc *egc,
>      fclose(f);
>      if (!pci_supp_legacy_irq())
>          goto out_no_irq;
> -    sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
> +    sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/gsi", pci->domain,
>                                  pci->bus, pci->dev, pci->func);
> +
> +    if ( access(sysfs_path, F_OK) != 0 ) {

So, the coding style in libxl is a bit different, first could you store
the return value of access() in 'r', then check that value? Then
"if (r)" will be enough, no need for "!= 0".

Second, there's no around space the condition in the if statement, so
just "if (r)".

> +        if ( errno == ENOENT )
> +            sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
> +                                pci->bus, pci->dev, pci->func);

Has the else part of this if..else is in a {}-block, could you add { }
here, for the if part? To quote the CODING_STYLE libxl document: "To
avoid confusion, either all the blocks in an if...else chain have
braces, or none of them do.

> +        else {
> +            LOGED(ERROR, domainid, "Can't access %s", sysfs_path);

This error message is kind of redundant, we could could simply let the code
try fopen() on this "/gsi" path, even if we know that it's not going to
work, as the error check on fopen() will produce the same error message.
;-)

And without that else part, the code could be simplified to:

    /* If /gsi path doesn't exist, fallback to /irq */
    if (r && errno == ENOENT)
        sysfs_path = "..../irq";



And those comments also apply to the changes in pci_remove_detached().

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 13:58:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 13:58:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684050.1063707 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcn7M-00017r-Uc; Wed, 21 Feb 2024 13:58:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684050.1063707; Wed, 21 Feb 2024 13:58:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcn7M-00017k-RW; Wed, 21 Feb 2024 13:58:36 +0000
Received: by outflank-mailman (input) for mailman id 684050;
 Wed, 21 Feb 2024 13:58:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WOnV=J6=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rcn7L-00017e-8o
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 13:58:35 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4beb73af-d0c1-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 14:58:33 +0100 (CET)
Received: from BN9PR03CA0711.namprd03.prod.outlook.com (2603:10b6:408:ef::26)
 by SA0PR12MB4446.namprd12.prod.outlook.com (2603:10b6:806:71::18)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.21; Wed, 21 Feb
 2024 13:58:29 +0000
Received: from BN3PEPF0000B072.namprd04.prod.outlook.com
 (2603:10b6:408:ef:cafe::41) by BN9PR03CA0711.outlook.office365.com
 (2603:10b6:408:ef::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.41 via Frontend
 Transport; Wed, 21 Feb 2024 13:58:28 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN3PEPF0000B072.mail.protection.outlook.com (10.167.243.117) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Wed, 21 Feb 2024 13:58:28 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 21 Feb
 2024 07:58:27 -0600
Received: from [10.71.193.58] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 21 Feb 2024 07:58:26 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4beb73af-d0c1-11ee-8a53-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bAF9PYEi0uN3wGf2YI5U+isAWoDhpDIm+VM+W6fPQLNMSf53ldXUTUqD93Kfd5XykYjtA0EvlXIIxZzdunrpYqmczJQBUqIQqVld2IwB/ExRI4EoARCTnd+l0qrJUxkPDIgLjbxhoZS0+czavWcABjlJlARGs2HJy4YuIvPWcFRP6sevCZAxvF1dD/t0IJ8choJPQNMCRXu/fTyAZY2YDK6fzozJ5EgTe8R1Hohc0OGpf9bwjFEjWjrIZ6VMVrqsdCRsfMdntc624lxKhicDZtZi5Wavyx9DoksSMBS/qvTj9UYkmGwYFxkWwlcYOT3Vo2ZZ+d4tiDsdY7V2T/KrdQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=RgWEbqC/1v7O6VAUqDSFOH2ByYy2Epr0gQSCvExcFtU=;
 b=MWSXsHtOZmmRRSisUMfV2nAWiHDmIgJUVvvu3+PJ4tz8Jv0x8JZinTibg8sBfElM+5Ij9SFGUvFqQShdqHRluOM7pHFXmYX0TZyEKcUKVFYpEWvDtp/6i8KAWQlC8GJXDauPu2gPcIWGT6JUpeVau8CAkaVxL7GG0f3lwev+ACCMr8xJu9qXupdonM1OOIiiCt2ZMij5cDrnR/hjRvae32AA4S4ySWElKCpd+lor6N9Uaq8P/vvQ9B1efatE3QyQ8uIdDIhVg+ULRnwOZF94FIKQQhK8hHymlFPb+HQh5TYcg5HKS+I8PtNQyWIshlShU6FN9vr9VF8SVvFP8xIg0g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=RgWEbqC/1v7O6VAUqDSFOH2ByYy2Epr0gQSCvExcFtU=;
 b=u1M+N2p1EsHw7ouclZrGwRHEvHu5yOpXs7d+D1wxe3gSv14fPWFwTUdVrVNdZN0s/LimfM8YpM1MBvEaZK2BY0bxPy+njbDYxHQdLexpu6L8JX3ust6vnyKscjIh25V8I8H9YudyEkGhI8WmwfCEYNvo1EPQRf+DQJgAcwsG5+M=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <32d9736e-965a-451f-9227-f774c20df215@amd.com>
Date: Wed, 21 Feb 2024 14:58:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v5 1/3] xen/arm: Introduce CONFIG_PARTIAL_EMULATION and
 "partial-emulation" cmd option
Content-Language: en-US
To: Julien Grall <julien@xen.org>, Ayan Kumar Halder <ayankuma@amd.com>, "Jan
 Beulich" <jbeulich@suse.com>, Ayan Kumar Halder <ayan.kumar.halder@amd.com>
CC: <sstabellini@kernel.org>, <stefano.stabellini@amd.com>,
	<Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>,
	<xen-devel@lists.xenproject.org>
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
 <20240220121743.3680715-2-ayan.kumar.halder@amd.com>
 <e3a8ce5d-841e-4ff1-8d23-48de822972ef@suse.com>
 <5f526deb-7376-4c34-85d1-d22b1b14d90e@amd.com>
 <e7c4b08f-724a-4419-977a-caa1342c47eb@suse.com>
 <62e047c5-a913-410d-9b24-e9a26d555d78@amd.com>
 <10e8a485-61b7-4c9d-930f-975e25e8ec45@xen.org>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <10e8a485-61b7-4c9d-930f-975e25e8ec45@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN3PEPF0000B072:EE_|SA0PR12MB4446:EE_
X-MS-Office365-Filtering-Correlation-Id: 420bcb25-32b7-4c8d-3fa7-08dc32e52e39
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XcRAhjFb1GTzwhVdx4E4uIGXy0EtBzzo0qE+JMcsktBjJBn+6aezMC/Qgam9DTTvnHVnP3YsxC+PyP9ZRaPvD+rW3fG4WJR6mFb2hQHOjKXR+r4F+Xel+PkTwoaYT3q4RH7d/THe6R66PGxIrj0gOxQWB8OQ+i2sR3rcM5mQ6SLbtiZ9bL3mhiuVC9OyDp2Gt8+purUnSjx0eB3T9fLLBWZIG3w8m/PPwdFBa9RQCIuhNJgJBRkkd3okmJQ95qy9zNxDrp+BXEkHHwWFbK5XMcvbR876NaaBBwOachfIythNNhapHtIhsK2B4L9cgocEpIr34ktiec4NMvq0UUb02jIFFzoLlqpwmgrhS+x2BPwON7GOm+5fCP9Eha6Wg72MbJcMgIU9+RJ8z3Uo1+cSoCG6kHzcbQRO0BzWM+0eYrOgqbk69lTyBqCWxVdhtoy+8pB7ID0TRQaII35odPOb33Kp3OFyRGBkGXgum+kN2K4npl5KPG1+kggopV6F9qNR0eGx+ahyuYT+v5xFLkaatZ5OVguLWOLpqPTjyJB7pX+19nsJ2MfUV1i3NCRv/kD/JQOXdo1JJYYBmAtq4ejzrhcGC5XYVWfSvwWY0znjPC4xsgjcf9iMNxS28PGYLIq0ChD45QsSdwVQ5NjNLLpzcQ==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(40470700004)(46966006);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2024 13:58:28.3702
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 420bcb25-32b7-4c8d-3fa7-08dc32e52e39
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN3PEPF0000B072.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4446



On 21/02/2024 13:20, Julien Grall wrote:
> 
> 
> On 21/02/2024 11:33, Ayan Kumar Halder wrote:
>> Hi Jan,
>>
>> On 21/02/2024 07:09, Jan Beulich wrote:
>>> On 20.02.2024 16:22, Ayan Kumar Halder wrote:
>>>> On 20/02/2024 12:33, Jan Beulich wrote:
>>>>> On 20.02.2024 13:17, Ayan Kumar Halder wrote:
>>>>>> --- a/SUPPORT.md
>>>>>> +++ b/SUPPORT.md
>>>>>> @@ -101,6 +101,18 @@ Extension to the GICv3 interrupt controller to
>>>>>> support MSI.
>>>>>>        Status: Experimental
>>>>>> +### ARM/Partial Emulation
>>>>>> +
>>>>>> +Enable partial emulation of registers, otherwise considered
>>>>>> unimplemented,
>>>>>> +that would normally trigger a fault injection.
>>>>>> +
>>>>>> +    Status: Supported, with caveats
>>>>>> +
>>>>>> +Bugs allowing the userspace to attack the guest OS will not be
>>>>>> considered
>>>>>> +security vulnerabilities.
>>>>>> +
>>>>>> +Bugs that could compromise Xen will be considered security
>>>>>> vulnerabilities.
>>>>> ... the odd statement regarding in-guest vulnerabilities that might be
>>>>> introduced. I see only two ways of treating this as supported: Either
>>>>> you simply refuse emulation when the access is from user space,
>>>> I am wondering how do we enforce that.
>>>>
>>>> Let me try to understand this with the current implementation of partial
>>>> emulation for system registers.
>>>>
>>>> 1. DBGDTRTX_EL0 :- I understand that EL0 access to this register will
>>>> cause a trap to EL2. The reason being MDCR_EL2.TDA == 1.
>>>>
>>>> In that case, if we refuse emulation then an undef exception is injected
>>>> to the guest (this is the behavior as of today even without this patch).
>>>>
>>>> So, are you saying that the undef exception is to be injected to the
>>>> user space process. This may be possible for Arm64 guests
>>>> (inject_undef64_exception() needs to be changed).
>>> No, injection is always to the guest, not to a specific entity within the
>>> guest. That ought to be the same on bare hardware: An exception when
>>> raised has an architecturally defined entry point for handling. That'll
>>> typically be kernel code. The handler then figures out whether the source
>>> of the exception was in user or kernel mode. For user mode code, the
>>> kernel may or may not try to handle the exception and then continue the
>>> user mode process. If it can't or doesn't want to handle it, it'll raise
>>> (in UNIX terms) a signal to the process. That signal, in turn, may or may
>>> not be fatal to the process. But such an exception from user mode should
>>> never be fatal to the guest as a whole.
>> Thanks for explaining it so well.
>>>
>>>> However for Arm32 guests, this may not be possible as the mode changes
>>>> to PSR_MODE_UND.
>>> I'm afraid my Arm foo isn't good enough to understand this. On the
>>> surface
>>> it looks to violate above outlined principle.
>>>
>>>> Let me know if I understood you correctly.
>>>>
>>>>> or you
>>>>> support that mode of emulation as much as that of kernel space
>>>>> accesses.
>>>> Do you mean we support partial emulation only for traps from EL1 mode ?
>>> Possibly.
>>
>> Now, I understand you. We will allow partial_emulation only from kernel
>> mode.
>>
>> So we need to do sth :-
>>
>> if ( 'partial_emulation == true' && '!regs_mode_is_user(regs)' )
>>
>> {
>>
>>       /* partial_emulation logic */
>>
>> }
>>
>> I am ok with this.
> 
> The helpers take a min_el. So you can have simpler code and set the
> value to 1 (rather than 0 today).
> 
>>
>> And the updates message will be
>>
>> ### ARM/Partial Emulation
>>
>> Enable partial emulation of registers, otherwise considered unimplemented,
>> that would normally trigger a fault injection.
>>
>>      Status: Supported, with caveats
>>
>> Partial emulation for traps from userspace is not allowed.
> 
> I am not sure how this helps...
>>
>> Bugs that could compromise Xen will be considered security vulnerabilities.
> 
> ... you are still implying that any userland breaching the kernel will
> not be supported because of a bug.
> 
> But, I don't see how preventing the userland to access a register will
> help. In theory you could have a register that can only be accessed by
> EL1 but has an impact to EL0.
> 
> By definition, it means that we don't faithfully follow the Arm Arm and
> anything can happen. Which is why I wanted to exclude security support
> from userland and kernel.
> 
> I think the current registers are low risk. But I am under the
> impression that you may wan to add more partial emulation. So we need to
> make some sort of statement that doesn't put any burden to the security
> team for future registers.
> 
> An option would be to explicitly list the registers in SUPPORT.md. As I
> mentioned above, I think the registers you currently emulate are
> low-risk to introduce a security bug. So I would be ok to fully security
> support them. This would need to be re-assessed for new registers.
> 
> I am open to other suggestions.
+1. Providing partial emulation support only for kernel mode where the Arm ARM allows for user mode
access is confusing and would require excessive number of comments to back up the implementation.
While it might be ok for DCC, imposing such limit on all the future use cases sounds too severe.

I think the following would be ok:

Status: Supported, with caveats

Only the following system registers are security supported:
 - ...

~Michal



From xen-devel-bounces@lists.xenproject.org Wed Feb 21 14:08:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 14:08:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684057.1063717 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcnGd-0002xD-NL; Wed, 21 Feb 2024 14:08:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684057.1063717; Wed, 21 Feb 2024 14:08:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcnGd-0002x6-Kq; Wed, 21 Feb 2024 14:08:11 +0000
Received: by outflank-mailman (input) for mailman id 684057;
 Wed, 21 Feb 2024 14:08:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcnGc-0002x0-2z
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 14:08:10 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcnGT-0005BU-RU; Wed, 21 Feb 2024 14:08:01 +0000
Received: from 54-240-197-234.amazon.com ([54.240.197.234]
 helo=[192.168.14.101]) by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcnGT-0002qM-9O; Wed, 21 Feb 2024 14:08:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=tqftxFxC3TX3Acmbj0snE5eMI1bwehitJaMwlEaMN9o=; b=dG0gBKp9+sl5m48GfzDQOtv6/0
	VvvlKaQCnwRX33e93gWeBsvnXg6O0O0LJsciNcKoMlpxcQ+rCEvH36ibXVyz5OCEN2tQaKqsE/z7D
	HwCh/TyJe63raPhunYZR8gf5Kds9tJchiTbNWFBs80VOjo6oSG7AQKx97JxWmGJJNn+4=;
Message-ID: <eeb48ddf-6925-4394-8d5a-288ba23e55d3@xen.org>
Date: Wed, 21 Feb 2024 14:07:57 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 07/30] xen/asm-generic: introdure nospec.h
Content-Language: en-GB
To: Oleksii <oleksii.kurochko@gmail.com>, Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <5889d7a5fa81722472f95cc1448af0be8f359a7d.1707146506.git.oleksii.kurochko@gmail.com>
 <510d74ca-5cce-48f8-93f6-407cd727cf7b@xen.org>
 <8a09e386b709f780f193af39af63b6aeb75c868e.camel@gmail.com>
 <aaf86d31-7ae0-4e33-8386-dda4bd21496a@suse.com>
 <ab040f3c8ee35f49fffac385053b55053c52da8c.camel@gmail.com>
 <a6394d8a-63d0-42bf-8ed7-a7722cb7e71c@suse.com>
 <3e0c83fd0e3dc424059575cf9da9d57a87d90736.camel@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <3e0c83fd0e3dc424059575cf9da9d57a87d90736.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 21/02/2024 12:47, Oleksii wrote:
> On Wed, 2024-02-21 at 12:00 +0100, Jan Beulich wrote:
>> On 20.02.2024 21:30, Oleksii wrote:
>>> On Mon, 2024-02-19 at 13:18 +0100, Jan Beulich wrote:
>>>> On 19.02.2024 12:59, Oleksii wrote:
>>>>> Hi Julien,
>>>>>
>>>>> On Sun, 2024-02-18 at 18:30 +0000, Julien Grall wrote:
>>>>>> Hi Oleksii,
>>>>>>
>>>>>> Title: Typo s/introdure/introduce/
>>>>>>
>>>>>> On 05/02/2024 15:32, Oleksii Kurochko wrote:
>>>>>>> The <asm/nospec.h> header is similar between Arm, PPC, and
>>>>>>> RISC-V,
>>>>>>> so it has been moved to asm-generic.
>>>>>>
>>>>>> I am not 100% convinced that moving this header to asm-
>>>>>> generic is
>>>>>> a
>>>>>> good
>>>>>> idea. At least for Arm, those helpers ought to be non-empty,
>>>>>> what
>>>>>> about
>>>>>> RISC-V?
>>>>> For Arm, they are not taking any action, are they? There are no
>>>>> specific fences or other mechanisms inside
>>>>> evaluate_nospec()/block_speculation() to address speculation.
>>>>
>>>> The question isn't the status quo, but how things should be
>>>> looking
>>>> like
>>>> if everything was in place that's (in principle) needed.
>>>>
>>>>> For RISC-V, it can be implemented in a similar manner, at least
>>>>> for
>>>>> now. Since these functions are only used in the grant tables
>>>>> code (
>>>>> for
>>>>> Arm and so for RISC-V ), which is not supported by RISC-V.
>>>>
>>>> Same here - the question is whether long term, when gnttab is
>>>> also
>>>> supported, RISC-V would get away without doing anything. Still
>>>> ...
>>>>
>>>>>> If the answer is they should be non-empty. Then I would
>>>>>> consider
>>>>>> to
>>>>>> keep
>>>>>> the duplication to make clear that each architecture should
>>>>>> take
>>>>>> their
>>>>>> own decision in term of security.
>>>>>>
>>>>>> The alternative, is to have a generic implementation that is
>>>>>> safe
>>>>>> by
>>>>>> default (if that's even possible).
>>>>> I am not certain that we can have a generic implementation, as
>>>>> each
>>>>> architecture may have specific speculation issues.
>>>>
>>>> ... it's theoretically possible that there'd be an arch with no
>>>> speculation issues, maybe simply because of not speculating.
>>>
>>> I am not sure that understand your and Julien point.
>>>
>>> For example, modern CPU uses speculative execution to reduce the
>>> cost
>>> of conditional branch instructions using schemes that predict the
>>> execution path of a program based on the history of branch
>>> executions.
>>>
>>> Arm CPUs are vulnerable for speculative execution, but if to look
>>> at
>>> the code of evaluate_nospec()/block_speculation() functions they
>>> are
>>> doing nothing for Arm.
>>
>> Which, as I understood Julien say, likely isn't correct. In which
>> case
>> this header shouldn't be dropped, using the generic one instead. The
>> generic headers, as pointed out several times before, should imo be
>> used
>> only if their use results in correct behavior. What is acceptable is
>> if
>> their use results in sub-optimal behavior (e.g. reduced performance
>> or
>> lack of a certain feature that another architecture maybe
>> implements).
> 
> As I understand it, evaluate_nospec()/block_speculation() were
> introduced for x86 to address the L1TF vulnerability specific to x86
> CPUs. This vulnerability is exclusive to x86 architectures [1], which
> explains why evaluate_nospec()/block_speculation() are left empty for
> Arm, RISC-V, and PPC.
> 
> It is unclear whether these functions should be utilized to mitigate
> other speculation vulnerabilities. 

The name is generic enough that someone may want to use it for other 
speculations. If we think this is only related to L1TF, then the 
functions names should reflect it. But see below.

> If they should, then, based on the
> current implementation, the Arm platform seems to accept having
> speculative vulnerabilities.

Looking at some of the use in common code (such as the grant-table 
code), it is unclear to me why it is empty on Arm. I think we need a 
speculation barrier.

I would raise the same question for RISC-V/PPC.

> 
> The question arises: why can't other architectures make their own
> decisions regarding security? 

Each architecture can make there own decision. I am not trying to 
prevent that. What I am trying to prevent is a developper including the 
asm-generic without realizing that the header doesn't provide a safe 
version.

> By default, if an architecture leaves the
> mentioned functions empty, it implies an agreement to potentially have
> speculative vulnerabilities. 

See above. That agreement is somewhat implicit. It would be better if 
this is explicit.

So overall, I would prefer if that header is not part of asm-generic.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 14:58:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 14:58:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684067.1063736 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rco3M-000187-GW; Wed, 21 Feb 2024 14:58:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684067.1063736; Wed, 21 Feb 2024 14:58:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rco3M-000180-Dw; Wed, 21 Feb 2024 14:58:32 +0000
Received: by outflank-mailman (input) for mailman id 684067;
 Wed, 21 Feb 2024 14:58:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TA2L=J6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rco3K-00017u-ID
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 14:58:30 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aa40ee7d-d0c9-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 15:58:28 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a28a6cef709so820813766b.1
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 06:58:26 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ts7-20020a170907c5c700b00a3f4bb02bc8sm486131ejc.42.2024.02.21.06.58.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 21 Feb 2024 06:58:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aa40ee7d-d0c9-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708527505; x=1709132305; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=G/p1pS4DzE5YyDMUJcVgfeMjgWT7V9bvuL4NGd3j3LQ=;
        b=AfGbJyeHGYksO/D6bb/o0wzlb/AXV56MynckxgxdbMWLKlSLzwDrASBBMGf2lWhltk
         38Tn0Ez7deIrYfxokdZfr79cxARCQipCHo8pzq9zuntDnTDRgXkVt+DhoPjEZsnOA+s2
         jG3xkyd7G8O036vOqBoxQJpLIOTjmG1l6M3/UW0YAQs7UnrKH3Iy6Ts7R6hVUAYcJEq8
         IV2Xusz7EVHpZ1zOBQoNfhCkVCDTU9fEzDAZbX4HMY929dKH4Pe086+e7CZmVkpc3ZTk
         iNsCJwcjqofJC+2mpj2bNSVS68Yiy68uOPjr6hLaOuYI79jWfBNX2wmAQf9omawL3OZd
         oMoQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708527505; x=1709132305;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=G/p1pS4DzE5YyDMUJcVgfeMjgWT7V9bvuL4NGd3j3LQ=;
        b=jNZvtAqViJsl7sv9/Q3T9fz+QeMMMcdBqvfBlJxQMhvRQlyTCgal2RQJA7+F8b8oWW
         o+ovK2YiRHClsgR+uSdPMiRdRaPppuDMuysXs+VAj/qcSfa1y79lQ68kcap3i2OUGJm2
         Kf+srJsf5+loEong/PQ9dyHchPLe3BNjdhR69ppyB780V7y3HC+MyJG0WVQfa0/iERSQ
         5zMStkXNClKnK+H/E0IDtmEYmV2BYflbCg+Nauyg8moGGQf3p355VI3Z6qsooIAqJRas
         zp0X94eJdAAd+wphJMP2U+z9psPWoBUJK53+Iq04lMNK1ZqeqF+iOFnLBFL5mXvCNWHt
         Owjw==
X-Forwarded-Encrypted: i=1; AJvYcCVoWSTN+yNAwKuQYaqop3c7tl3Aq9wayXqohap0A5Pyt9c98X5bW4OAUv0aucbGkwXqSF3AxzFrTP/lVE8zv0MkqPfv3Q8OrFjulX3u/U4=
X-Gm-Message-State: AOJu0YyRlvIJEVh9iDzUJ5uFCYx2hPCKLgz4gMiZpnRjzNSi9omXpOaE
	wGnc2IpCs3mE5g6SxerG2cMTHRo4tTOfvfaF8qoMCQN2AUJIWksBQZGghEP8vg==
X-Google-Smtp-Source: AGHT+IHd3SEnXzBwjccwyim0c94d45naHaXVQjzgG/i4PVGFKqZfqn+sH7HwH+3n5VtdBMnKOFpI0A==
X-Received: by 2002:a17:907:1044:b0:a3e:590f:6348 with SMTP id oy4-20020a170907104400b00a3e590f6348mr7681451ejb.41.1708527505509;
        Wed, 21 Feb 2024 06:58:25 -0800 (PST)
Message-ID: <dfcfcab7-50b4-45b2-a25a-8898b74054a9@suse.com>
Date: Wed, 21 Feb 2024 15:58:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 07/30] xen/asm-generic: introdure nospec.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org,
 Julien Grall <julien@xen.org>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <5889d7a5fa81722472f95cc1448af0be8f359a7d.1707146506.git.oleksii.kurochko@gmail.com>
 <510d74ca-5cce-48f8-93f6-407cd727cf7b@xen.org>
 <8a09e386b709f780f193af39af63b6aeb75c868e.camel@gmail.com>
 <aaf86d31-7ae0-4e33-8386-dda4bd21496a@suse.com>
 <ab040f3c8ee35f49fffac385053b55053c52da8c.camel@gmail.com>
 <a6394d8a-63d0-42bf-8ed7-a7722cb7e71c@suse.com>
 <3e0c83fd0e3dc424059575cf9da9d57a87d90736.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3e0c83fd0e3dc424059575cf9da9d57a87d90736.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 21.02.2024 13:47, Oleksii wrote:
> As I understand it, evaluate_nospec()/block_speculation() were
> introduced for x86 to address the L1TF vulnerability specific to x86
> CPUs.

Well, if you look at one of the later commits altering x86'es variant,
you'll find that this wasn't really correct.

> This vulnerability is exclusive to x86 architectures [1], which
> explains why evaluate_nospec()/block_speculation() are left empty for
> Arm, RISC-V, and PPC.
> 
> It is unclear whether these functions should be utilized to mitigate
> other speculation vulnerabilities. If they should, then, based on the
> current implementation, the Arm platform seems to accept having
> speculative vulnerabilities.
> 
> The question arises: why can't other architectures make their own
> decisions regarding security? By default, if an architecture leaves the
> mentioned functions empty, it implies an agreement to potentially have
> speculative vulnerabilities. Subsequently, if an architecture needs to
> address such vulnerabilities, they can develop arch-specific nospec.h
> to implement the required code.

You can't take different perspectives on security. There is some
hardening which one architecture may go farther with than another,
but e.g. information leaks are information leaks and hence need
addressing. Of course if an arch knew it had no (known) issues, then
using a generic form of this header would be okay (until such time
where an issue would be found).

And btw, looking at how xen/nospec.h came about, it looks pretty clear
to me that array_index_mask_nospec() should move from system.h to
nospec.h. That would make Arm's form immediately different from what
a generic one might have, and quite likely an inline assembly variant
could also do better on RISC-V (unless, as said, RISC-V simply has no
such issues). Then again I notice Arm64 has no override here ...

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 15:03:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 15:03:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684072.1063747 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rco7q-0002mA-1t; Wed, 21 Feb 2024 15:03:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684072.1063747; Wed, 21 Feb 2024 15:03:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rco7p-0002m3-Un; Wed, 21 Feb 2024 15:03:09 +0000
Received: by outflank-mailman (input) for mailman id 684072;
 Wed, 21 Feb 2024 15:03:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c6My=J6=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rco7o-0002lx-Fr
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 15:03:08 +0000
Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com
 [2a00:1450:4864:20::42f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 514c7ce0-d0ca-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 16:03:06 +0100 (CET)
Received: by mail-wr1-x42f.google.com with SMTP id
 ffacd0b85a97d-33d2710f3acso3165235f8f.0
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 07:03:06 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 f2-20020adfe902000000b0033cf24700e5sm17073229wrm.39.2024.02.21.07.03.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 21 Feb 2024 07:03:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 514c7ce0-d0ca-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708527786; x=1709132586; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=t0lmC0kXh+qRlgHgi5grcPHTJtMDtOQOJnj+ljZmAjw=;
        b=L2GYW2dPSGqWheK36CgnVbnod5LX4HRiLeizprMLpdkHJO+E6xFuWUmCyENWXavSw1
         MiIoju7KY9v42DcUD1r7gS30cFPgJq70x76fNfFMq4JpAqFivmz4MaqQQImPOMY6pnP2
         pETnuCs6eTpcN5UYAnrYIdOziWcLKt2JW6fBA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708527786; x=1709132586;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=t0lmC0kXh+qRlgHgi5grcPHTJtMDtOQOJnj+ljZmAjw=;
        b=LSZapRPbPpS1a97FhnTyKa0+AZ+w4xxwZ0SRBGvKW9Cmg8mHGCZYkswPTrORAkbnj/
         SJrv7/lBmGVIWbm898naPVEElgTh+7AoJkC3HK2F+zz+v+jR1K6t+CQWCipC7PLnFquN
         wmuvRy4433sMJ6YZKHzFcFLPNwSoqPbGmtFLnfC5xVSQ53nagdQxuWMvYlRsywTQNEp2
         9rhwgAMufK0N6Q8psnbWTqbLxbetkTPrW0wa9wXbT7MUCBaypvnW2ockfT59/g+IZoVP
         nXFDqp5pQ1yPDt3H3Wwa7g2RIR5Ws6Uy8Q73GweV3D0qucrUWgd6EASPYWqVTwJhdAUF
         dh2Q==
X-Gm-Message-State: AOJu0YyUnME9npBSYCWssGArY7EmXcWRjlpW1atZ5kAv35Ne7tedJgo9
	i++Ua8z9fKGE+jks7IcCzzs85a3likA7k061+pgNuGbv0IbwJs388COAgNF1PjY=
X-Google-Smtp-Source: AGHT+IH+UwhX1wSEGNJtpA8AZncuYndy8Qb1tQLKki7gTlFDMwq/oKN9wZ+VOIdTiJW/bMsAwzv6Mw==
X-Received: by 2002:a5d:4404:0:b0:33d:e27:f053 with SMTP id z4-20020a5d4404000000b0033d0e27f053mr14120345wrq.6.1708527785596;
        Wed, 21 Feb 2024 07:03:05 -0800 (PST)
Date: Wed, 21 Feb 2024 15:03:04 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Jiqian Chen <Jiqian.Chen@amd.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	"Daniel P . Smith" <dpsmith@apertussolutions.com>,
	Stewart Hildebrand <Stewart.Hildebrand@amd.com>,
	Huang Rui <Ray.Huang@amd.com>
Subject: Re: [RFC XEN PATCH v5 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Message-ID: <09290ba4-6915-4f0d-8e16-3e14713d02ba@perard>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-6-Jiqian.Chen@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240112061317.418658-6-Jiqian.Chen@amd.com>

On Fri, Jan 12, 2024 at 02:13:17PM +0800, Jiqian Chen wrote:
> diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
> index f2d9d14b4d9f..448ba2c59ae1 100644
> --- a/tools/libs/ctrl/xc_domain.c
> +++ b/tools/libs/ctrl/xc_domain.c
> @@ -1394,6 +1394,21 @@ int xc_domain_irq_permission(xc_interface *xch,
>      return do_domctl(xch, &domctl);
>  }
>  
> +int xc_domain_gsi_permission(xc_interface *xch,
> +                             uint32_t domid,
> +                             uint32_t gsi,
> +                             bool allow_access)
> +{
> +    struct xen_domctl domctl = {};
> +
> +    domctl.cmd = XEN_DOMCTL_gsi_permission;
> +    domctl.domain = domid;
> +    domctl.u.gsi_permission.gsi = gsi;
> +    domctl.u.gsi_permission.allow_access = allow_access;

This could be written as:
    struct xen_domctl domctl = {
        .cmd = XEN_DOMCTL_gsi_permission,
        .domain = domid,
        .u.gsi_permission.gsi = gsi,
        .u.gsi_permission.allow_access = allow_access,
    };

but your change is fine too.


> +
> +    return do_domctl(xch, &domctl);
> +}
> +
>  int xc_domain_iomem_permission(xc_interface *xch,
>                                 uint32_t domid,
>                                 unsigned long first_mfn,
> diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
> index a1c6e82631e9..4136a860a048 100644
> --- a/tools/libs/light/libxl_pci.c
> +++ b/tools/libs/light/libxl_pci.c
> @@ -1421,6 +1421,8 @@ static void pci_add_dm_done(libxl__egc *egc,
>      uint32_t flag = XEN_DOMCTL_DEV_RDM_RELAXED;
>      uint32_t domainid = domid;
>      bool isstubdom = libxl_is_stubdom(ctx, domid, &domainid);
> +    int gsi;
> +    bool has_gsi = true;

Why is this function has "gsi" variable, and "gsi = irq" but the next
function pci_remove_detached() does only have "irq" variable?

So, gsi can only be positive integer, right? Could we forgo `has_gsi` and
just set "gsi = -1" when there's no gsi?

>      /* Convenience aliases */
>      bool starting = pas->starting;
> @@ -1482,6 +1484,7 @@ static void pci_add_dm_done(libxl__egc *egc,
>                                  pci->bus, pci->dev, pci->func);
>  
>      if ( access(sysfs_path, F_OK) != 0 ) {
> +        has_gsi = false;
>          if ( errno == ENOENT )
>              sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
>                                  pci->bus, pci->dev, pci->func);
> @@ -1497,6 +1500,7 @@ static void pci_add_dm_done(libxl__egc *egc,
>          goto out_no_irq;
>      }
>      if ((fscanf(f, "%u", &irq) == 1) && irq) {
> +        gsi = irq;

Why do you do this, that that mean that `gsi` and `irq` are the same? Or
does `irq` happen to sometime contain a `gsi` value?

>          r = xc_physdev_map_pirq(ctx->xch, domid, irq, &irq);
>          if (r < 0) {
>              LOGED(ERROR, domainid, "xc_physdev_map_pirq irq=%d (error=%d)",
> @@ -1505,7 +1509,10 @@ static void pci_add_dm_done(libxl__egc *egc,
>              rc = ERROR_FAIL;
>              goto out;
>          }
> -        r = xc_domain_irq_permission(ctx->xch, domid, irq, 1);
> +        if ( has_gsi )
> +            r = xc_domain_gsi_permission(ctx->xch, domid, gsi, 1);
> +        if ( !has_gsi || r == -EOPNOTSUPP )
> +            r = xc_domain_irq_permission(ctx->xch, domid, irq, 1);
>          if (r < 0) {
>              LOGED(ERROR, domainid,
>                    "xc_domain_irq_permission irq=%d (error=%d)", irq, r);

Looks like this error message could be wrong, I think we want to know if
which call between xc_domain_irq_permission() and
xc_domain_gsi_permission() has failed.

> @@ -2185,6 +2192,7 @@ static void pci_remove_detached(libxl__egc *egc,
>      FILE *f;
>      uint32_t domainid = prs->domid;
>      bool isstubdom;
> +    bool has_gsi = true;
>  
>      /* Convenience aliases */
>      libxl_device_pci *const pci = &prs->pci;
> @@ -2244,6 +2252,7 @@ skip_bar:
>                             pci->bus, pci->dev, pci->func);
>  
>      if ( access(sysfs_path, F_OK) != 0 ) {
> +        has_gsi = false;
>          if ( errno == ENOENT )
>              sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain,
>                                  pci->bus, pci->dev, pci->func);
> @@ -2270,7 +2279,10 @@ skip_bar:
>               */
>              LOGED(ERROR, domid, "xc_physdev_unmap_pirq irq=%d", irq);
>          }
> -        rc = xc_domain_irq_permission(ctx->xch, domid, irq, 0);
> +        if ( has_gsi )
> +            rc = xc_domain_gsi_permission(ctx->xch, domid, irq, 0);

Why do you use the xc_domain_gsi_permission() with "irq" here, but not
in pci_add_dm_done() ?

> +        if ( !has_gsi || rc == -EOPNOTSUPP )
> +            rc = xc_domain_irq_permission(ctx->xch, domid, irq, 0);
>          if (rc < 0) {
>              LOGED(ERROR, domid, "xc_domain_irq_permission irq=%d", irq);
>          }


These changes to libxl are quite confusing, there's a mixed up between
"gsi" and "irq", it's hard to follow.

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 15:22:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 15:22:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684080.1063756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcoQl-0005mD-GQ; Wed, 21 Feb 2024 15:22:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684080.1063756; Wed, 21 Feb 2024 15:22:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcoQl-0005m6-Dn; Wed, 21 Feb 2024 15:22:43 +0000
Received: by outflank-mailman (input) for mailman id 684080;
 Wed, 21 Feb 2024 15:22:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c6My=J6=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rcoQk-0005lZ-Mo
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 15:22:42 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0cc4df94-d0cd-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 16:22:42 +0100 (CET)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-5650ac9a440so917201a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 07:22:39 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 u16-20020aa7db90000000b005645c4af6c7sm3380979edt.13.2024.02.21.07.22.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 21 Feb 2024 07:22:38 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0cc4df94-d0cd-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708528959; x=1709133759; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=bBlTt9poW/uW95sB+S7+mNOQfYUBO7fagy8zO2v9oDY=;
        b=dgvNiqAgmSaboMlmI6adJbfLFtXvjRZM76q/hKGwJX1yxO0zL/VEv4rpS8nhOEyC0j
         s7Ha04Wfd/cbVRSvS0qZPz70xMo0+qn5G+cPPIdqSjU7B/OuE7u8K3gaE+wvMGphKu0v
         W/NYckseRtu8hYGxGC2pyeeyWS2tTBeiQJTvc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708528959; x=1709133759;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bBlTt9poW/uW95sB+S7+mNOQfYUBO7fagy8zO2v9oDY=;
        b=QpTMaCXGVI/jmkI+K/DrOtUiN35MtM56VziAQwggx1IeqeNQl+WWLAqPx6xMSKsIx1
         WiZNOid9d2vWWXyngxBz+xtCPzpxSF2vqsSrIlN5lFiizrTiop1dju7XO6Cb0yANLYeq
         am2zCAkcz7E1Mk/iO+1Py82W4UMi1/2E/uTejD+MkQayHSVfCfwafVUqa2ywhjmdMQoZ
         3gYs1LmBdzu6t1QEmZMJxfTmQPaFF0rZoOF1naFRXWRmKlGxuTdeusG0rDy7cn64bCN8
         bO9S9ebcReOd99AYX0SNMqZ6SoTw2cSps2Vv8ih0k3cWQSrpRhmF2TnjLax0H/vIKF/k
         lk5A==
X-Gm-Message-State: AOJu0Yy4EId6T1DsbdWCZoHrwDUFF8kOiNR8FA0QnxTy2ow3dYjIW8tr
	mhVmWUvm7FFr/3xA/y9rUdRJWhjBk+CyaPwOHgXz3xxfWnSbinWryGZZt2gMa0w=
X-Google-Smtp-Source: AGHT+IFzIAwS5710ytBbzLuG8+QaJwD8W5MOmhFdD1ZDEyz0wKc+aoR5WlDuxHU2zMP1FHZPf6bghw==
X-Received: by 2002:a05:6402:1656:b0:564:7007:e14a with SMTP id s22-20020a056402165600b005647007e14amr6178279edx.6.1708528959172;
        Wed, 21 Feb 2024 07:22:39 -0800 (PST)
Date: Wed, 21 Feb 2024 15:22:38 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>
Subject: Re: [PATCH] build: make sure build fails when running kconfig fails
Message-ID: <ed53b507-8ce1-4496-b2c2-cb0089f384c0@perard>
References: <55c3a781-17f8-47f2-9629-515e1aea77aa@suse.com>
 <b7c22718-ec6c-470e-be72-e613b2af3c54@perard>
 <abbae372-5e3e-4013-a2a3-45892ec96c89@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <abbae372-5e3e-4013-a2a3-45892ec96c89@suse.com>

On Tue, Feb 20, 2024 at 01:20:16PM +0100, Jan Beulich wrote:
> On 20.02.2024 12:18, Anthony PERARD wrote:
> > On Tue, Feb 20, 2024 at 09:43:56AM +0100, Jan Beulich wrote:
> >> Because of using "-include", failure to (re)build auto.conf (with
> >> auto.conf.cmd produced as a secondary target) won't stop make from
> >> continuing the build. Arrange for it being possible to drop the - from
> >> Rules.mk, requiring that the include be skipped for tools-only targets.
> >> Note that relying on the inclusion in those cases wouldn't be correct
> >> anyway, as it might be a stale file (yet to be rebuilt) which would be
> >> included, while during initial build, the file would be absent
> >> altogether.
> >>
> >> Fixes: 8d4c17a90b0a ("xen/build: silence make warnings about missing auto.conf*")
> >> Reported-by: Roger Pau Monn <roger.pau@citrix.com>
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> > 
> > Just to make sure, this patch is a workaround to a harmless bug in older
> > version of GNU Make which print spurious error messages, and said bug as
> > been fixed in GNU Make 4.2, right? Bug report:
> > 
> >     bug #102: Make prints an incorrect error for missing includes
> >     https://savannah.gnu.org/bugs/?func=detailitem&item_id=102
> 
> Assuming you mean the change referenced in Fixes: - yes, that's my
> understanding. (Whether the referenced make bug is the one I can't tell,
> though. But it looks like it is.)
> 
> >> --- a/xen/Makefile
> >> +++ b/xen/Makefile
> >> @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
> >>  # This exploits the 'multi-target pattern rule' trick.
> >>  # The syncconfig should be executed only once to make all the targets.
> >>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
> >> +	$(Q)rm -f include/config/$*.conf
> > 
> > Maybe this should say "include/config/auto.conf" instead of using "$*".
> > "syncconfig" is going to generate "auto.conf" and not "$*.conf". And it
> > would make easier to find the "rm" command via "grep".
> 
> Generally I advocate for using $* wherever possible in pattern rules. If,

Well, it might look like a pattern rule, but it isn't one. It's a
"groupes targets" rule that works on version of Make that doesn't have
this brand new feature. It would be written as:
    include/config/auto.conf include/config/auto.conf.cmd&: $(KCONFIG_CONFIG)

Second, the other command in the recipe doesn't use "$*", so it's not
possible to use this rule with a value of $* other than "auto".

> however, replacing that is the only way to get an ack for this change, then
> I would (hesitantly) do so.

Yes please, even if I don't like the patch. I hope it isn't going to
byte us later if something in "tools/" happens to start depending on
Kconfig (only fixdep and kconfig need to not depend on kconfig at the
moment). But the patch currently works and looks ok. So, with "$*"
replace by "auto", you can add my "Reviewed-by: Anthony PERARD
<anthony.perard@citrix.com>".

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 15:32:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 15:32:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684086.1063766 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcoZw-0007Wu-Fj; Wed, 21 Feb 2024 15:32:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684086.1063766; Wed, 21 Feb 2024 15:32:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcoZw-0007Wn-Cz; Wed, 21 Feb 2024 15:32:12 +0000
Received: by outflank-mailman (input) for mailman id 684086;
 Wed, 21 Feb 2024 15:32:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcoZv-0007Wd-FY; Wed, 21 Feb 2024 15:32:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcoZu-0006zq-U5; Wed, 21 Feb 2024 15:32:10 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcoZu-0004UE-HB; Wed, 21 Feb 2024 15:32:10 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcoZu-00085y-Go; Wed, 21 Feb 2024 15:32:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=9rwVtyHQsrzkgIuliWsvK1lcPQcBpA5AXtAoU+sP/JA=; b=yz3AX1SMNPgYQTNAVp0qUawJqB
	3XQc/toa9lcs+swtlrQkNvFBvvFD4xisjOIeZwaKSA6j8FrU88aLVi33hn9zbljcJvjmlX1WosD9D
	zZzGdHCbX0DM/wzK+VeXkmfpnzu0tHIgr6rf5Fo8Rb2MI/Cs2V5XjlNI5PsJUChWqjZE=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184719-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184719: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=9fc1ccccfd8d53dc7936fe6d633f2373fc9f62e8
X-Osstest-Versions-That:
    linux=fca7526b7d8910c6125cb1ebc3e78ccd5f50ec52
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 21 Feb 2024 15:32:10 +0000

flight 184719 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184719/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184714
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184714
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184714
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184714
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184714
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184714
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184714
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184714
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                9fc1ccccfd8d53dc7936fe6d633f2373fc9f62e8
baseline version:
 linux                fca7526b7d8910c6125cb1ebc3e78ccd5f50ec52

Last test of basis   184714  2024-02-20 21:12:20 Z    0 days
Testing same since   184719  2024-02-21 06:34:23 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Arnd Bergmann <arnd@arndb.de>
  Bart Van Assche <bvanassche@acm.org>
  Bhargava Chenna Marreddy <bhargava.marreddy@broadcom.com>
  Daniel Vacek <neelx@redhat.com>
  Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
  Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
  Kamal Heib <kheib@redhat.com>
  Kashyap Desai <kashyap.desai@broadcom.com>
  Leon Romanovsky <leon@kernel.org>
  Leon Romanovsky <leonro@nvidia.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Mark Zhang <markzhang@nvidia.com>
  Mats Kronberg <kronberg@nsc.liu.se>
  Mike Marciniszyn <mike.marciniszyn@intel.com>
  Mustafa Ismail <mustafa.ismail@intel.com>
  Selvin Xavier <selvin.xavier@broadcom.com>
  Shiraz Saleem <shiraz.saleem@intel.com>
  Sindhu Devale <sindhu.devale@gmail.com>
  Sindhu Devale <sindhu.devale@intel.com>
  Tamar Mashiah <tmashiah@nvidia.com>
  Yishai Hadas <yishaih@nvidia.com>
  Zhipeng Lu <alexious@zju.edu.cn>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   fca7526b7d89..9fc1ccccfd8d  9fc1ccccfd8d53dc7936fe6d633f2373fc9f62e8 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 15:40:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 15:40:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684093.1063777 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcohn-0000m0-Ao; Wed, 21 Feb 2024 15:40:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684093.1063777; Wed, 21 Feb 2024 15:40:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcohn-0000lt-7X; Wed, 21 Feb 2024 15:40:19 +0000
Received: by outflank-mailman (input) for mailman id 684093;
 Wed, 21 Feb 2024 15:40:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c6My=J6=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rcohm-0000ln-Ip
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 15:40:18 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 82a189e8-d0cf-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 16:40:16 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-41273f53c71so6882045e9.1
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 07:40:16 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 j6-20020a05600c190600b00411d0b58056sm19113894wmq.5.2024.02.21.07.40.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 21 Feb 2024 07:40:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 82a189e8-d0cf-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708530016; x=1709134816; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=zF3CJpOvKAO/nkojOSXwjNL9njN21cd0Ma6oA+EsuYU=;
        b=GfnyVL0aatpZf3OxWF8c4fRED5McVyS9vF8d5D7sb/jbT3vhzzFEzX0DF31bSom8mt
         TOxJ0cfVNW4CFswwNPBSZ99Iu0fsI0FdO6ZM+cJktaCHO9aL0xYkfLHEGfC49nY2X2uW
         wc5/PS+83+nNzuGOgfh2rK4Jy+SrXvmFGQWog=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708530016; x=1709134816;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zF3CJpOvKAO/nkojOSXwjNL9njN21cd0Ma6oA+EsuYU=;
        b=DRndAh6Gb60FWmkkk+Z8oNNcppH0kHHO3raJMy0fqqXut0eVGrOUhazNl1vrhip3DY
         8QjMvsOK1lX7C75uerLh4Zc6HXgvYijyzZTfnKX1a21Chr+On9ArnAHAewhlhatSoIlU
         hqovC8CdrCct1lrJ1pWREPj+Sd3JHDXwYKG/RhOw189obktOVv/MZqdFodXRIdV8mgcf
         zx9xjsag10ttmW+EPl9N/KxGX8m0yeGbjP+f5d8hGxxvLNSAtEl9YNo5e+ShSOuzYEGt
         BI/o+r4cJAhvl3mKnegRl+5xfEoLGBHeolMeCGAt/mWzan9GK9JpbwgiwlV+8Knj0SfK
         9Xrw==
X-Gm-Message-State: AOJu0YzJI7USUgckeAAZzwojJswzkiOaHG3VlhfL/KsmK5AbYOTo2Wf0
	kkmKdm0dK9V3zWpuAabCJAsNEHWiokHuGmyhcG/UqNAyNX5UHMKCqOnDs+AGC1U=
X-Google-Smtp-Source: AGHT+IHL+fYNLryDvEv0dGfWzKQ8XRIDk3SOyHSRnGd3ERou/ZUPXADUX15xB1olbZFkSb36+vEmkA==
X-Received: by 2002:adf:cc88:0:b0:33d:620b:49 with SMTP id p8-20020adfcc88000000b0033d620b0049mr5545821wrj.32.1708530015940;
        Wed, 21 Feb 2024 07:40:15 -0800 (PST)
Date: Wed, 21 Feb 2024 15:40:15 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] tools/9pfsd: add missing va_end() in fill_data()
Message-ID: <a58105f3-1dd6-462c-a454-0cee1f5911df@perard>
References: <20240219135658.3662-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240219135658.3662-1-jgross@suse.com>

On Mon, Feb 19, 2024 at 02:56:58PM +0100, Juergen Gross wrote:
> In xen-9pfsd fill_data() va_end() needs to be called before returning.
> 
> Coverity Id CID 1592145
> 
> Fixes: bcec59cf7ff4 ("tools/xen-9pfsd: add 9pfs version request support")
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 15:46:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 15:46:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684100.1063786 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rconY-0001OX-T6; Wed, 21 Feb 2024 15:46:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684100.1063786; Wed, 21 Feb 2024 15:46:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rconY-0001OQ-QW; Wed, 21 Feb 2024 15:46:16 +0000
Received: by outflank-mailman (input) for mailman id 684100;
 Wed, 21 Feb 2024 15:46:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QV5I=J6=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rconX-0001OK-BR
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 15:46:15 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5738f7e4-d0d0-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 16:46:13 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 8F0CD4EE0737;
 Wed, 21 Feb 2024 16:46:12 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5738f7e4-d0d0-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Wed, 21 Feb 2024 16:46:12 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper3
 <andrew.cooper3@citrix.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, julien@xen.org, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, George Dunlap <george.dunlap@citrix.com>, Wei
 Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
In-Reply-To: <1743b4248d30a4e8b68a150c25724caa@bugseng.com>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
 <d90d98b6-508b-4a2a-ab6a-74a9828a5b94@suse.com>
 <45509cb67ecee3f690b5784489b5ccb4@bugseng.com>
 <1743b4248d30a4e8b68a150c25724caa@bugseng.com>
Message-ID: <2ff52df443fc080875fd05614d89764d@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-21 13:08, Nicola Vetrini wrote:
> On 2024-02-20 09:14, Nicola Vetrini wrote:
>> On 2024-02-20 08:45, Jan Beulich wrote:
>>> On 19.02.2024 16:14, Nicola Vetrini wrote:
>>>> The cache clearing and invalidation helpers in x86 and Arm didn't
>>>> comply with MISRA C Rule 17.7: "The value returned by a function
>>>> having non-void return type shall be used". On Arm they
>>>> were always returning 0, while some in x86 returned -EOPNOTSUPP
>>>> and in common/grant_table the return value is saved.
>>>> 
>>>> As a consequence, a common helper arch_grant_cache_flush that 
>>>> returns
>>>> an integer is introduced, so that each architecture can choose 
>>>> whether to
>>>> return an error value on certain conditions, and the helpers have 
>>>> either
>>>> been changed to return void (on Arm) or deleted entirely (on x86).
>>>> 
>>>> Signed-off-by: Julien Grall <julien@xen.org>
>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>>> ---
>>>> The original refactor idea came from Julien Grall in [1]; I edited 
>>>> that proposal
>>>> to fix build errors.
>>>> 
>>>> I did introduce a cast to void for the call to flush_area_local on 
>>>> x86, because
>>>> even before this patch the return value of that function wasn't 
>>>> checked in all
>>>> but one use in x86/smp.c, and in this context the helper (perhaps 
>>>> incidentally)
>>>> ignored the return value of flush_area_local.
>>> 
>>> I object to such casting to void, at least until there's an 
>>> overriding
>>> decision that for Misra purposes such casts may be needed.
>>> 
>> 
>> There are three choices here:
>> 1. cast to void
>> 2. deviation for flush_area_local, which for the case of the cache 
>> helpers is what led to this patch; it may still be a viable option, if 
>> other maintainers agree
>> 3. refactor of flush_area_local; this is not viable here because the 
>> return value is actually used and useful, as far as I can tell, in 
>> smp.c
>> 
>>>> --- a/xen/arch/arm/include/asm/page.h
>>>> +++ b/xen/arch/arm/include/asm/page.h
>>>> @@ -123,6 +123,7 @@
>>>> 
>>>>  #ifndef __ASSEMBLY__
>>>> 
>>>> +#include <public/grant_table.h>
>>> 
>>> This is a no-go, imo (also on x86): Adding this include here 
>>> effectively
>>> means that nearly every CU will have a dependency on that header, no
>>> matter that most are entirely agnostic of grants. Each arch has a
>>> grant_table.h - is there any reason the new, grant-specific helper 
>>> can't
>>> be put there?
>>> 
>> 
>> I would have to test, but I think that can be done
>> 
> 
> The only blocker so far is that this triggers a build error due to a 
> circular dependency between xen/mm.h and asm/flushtlb.h on x86. Also 
> found some earlier evidence [1] that there are some oddities around 
> asm/flushtlb's inclusion.
> 
> [1] 
> https://lore.kernel.org/xen-devel/20200318210540.5602-1-andrew.cooper3@citrix.com/

There could be a way of untangling asm/flushtlb.h from xen/mm.h, by 
moving "accumulate_tlbflush" and "filtered_flush_tlb_mask" introduced by 
commit 80943aa40e30 ("replace tlbflush check and operation with inline 
functions") [1].
However, these function should then be part of a generic xen/flushtlb.h 
header, since they are used in common code (e.g., common/page_alloc) and 
a bunch of common code source files should move their includes (see [2] 
for a partial non-working patch). Do you feel that this is a feasible 
route?
In passing it should be noted that the header ordering in 
x86/alternative.c is not the one usually prescribed, so that may be 
taken care of as well.

[1] 
https://lore.kernel.org/xen-devel/1474338664-5054-1-git-send-email-dongli.zhang@oracle.com/
[2] 
https://gitlab.com/xen-project/people/bugseng/xen/-/commit/a2be0927f724e7e9f891d1e00831739137c29041

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 15:47:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 15:47:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684103.1063797 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcoob-0001tb-5d; Wed, 21 Feb 2024 15:47:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684103.1063797; Wed, 21 Feb 2024 15:47:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcoob-0001tU-2T; Wed, 21 Feb 2024 15:47:21 +0000
Received: by outflank-mailman (input) for mailman id 684103;
 Wed, 21 Feb 2024 15:47:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=TA2L=J6=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rcooZ-0001tK-La
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 15:47:19 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7e2ad398-d0d0-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 16:47:18 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a293f2280c7so981638366b.1
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 07:47:18 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s8-20020a170906500800b00a3de4c7bf00sm5061791ejj.79.2024.02.21.07.47.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 21 Feb 2024 07:47:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e2ad398-d0d0-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708530438; x=1709135238; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=BaSZPrN+BoA1InZG490Cz4viy5OnTLYfAijdnOVQWCY=;
        b=ATVEwDhKS4/Dzu/Zr72u9G7wGmMT2fAy7cCkJdouLnZhRu9nlVFiLlRctbr/GwBx5I
         ArAWJgHDLfDj2A0lSdo8zb3WIhsdMRQNP8L+MJ3CKZ2MtUKMfaqEu2qfya29J/nD+LLn
         HxcsNyxv57d85hmsvlyronkBfCDtYcCLX8FzWSj9W9gl/eQxqd5pe+zZkTNTDN6MImMT
         kctzcA1yLuQowJ03hSOp3ry6KbLNF0Yn2iPb49eH+a5ZoFBaXOJHR4zHHRVCaf7IXZbj
         XEm6Wki9PVtODgXyGwz4Vwlf0ogaCTM5uGD9kqZ9P7Lq9BF/JqEi4TQpZ/UPaPH8/zfx
         rqIA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708530438; x=1709135238;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=BaSZPrN+BoA1InZG490Cz4viy5OnTLYfAijdnOVQWCY=;
        b=TOxnaE+yxzDkm9X/QcnNj/4tSy7i3kS6ntdkGZl+b4j5HJRpsGKutLOkepxc1ppsxb
         XiWZzjKe+CYYIhlB0aHkxn6bwsMdFxDWckyngDhkJ32SBOajjESAnCU1e094A66zDoPR
         PdWCWngcX9KKappwTq7ADO9l+XYBqlXklc7kn5+orfS17gFXEHkpiKF067d7eFDaRmFt
         DiV6kUV+uA2P5KmWFzSfoGDwsmqC3KNAlyhzU667Vg4nhLCATE0k+UDgY/kx0bVGHkQd
         FBdREp7MHD4iXZLaqIBwkE0ESQ+gz2AugdOuSALkrzUKISklK46sM3Ocx3YAzpFgF4vK
         AFgA==
X-Gm-Message-State: AOJu0Yy2ooDnZ9/PRWvOzgr658xqB4T4vWRc4MlTkINaj5KZuLEy/LXH
	KSePQFtYfWlC8rgPJ7RIPdtpXKCy4JK1H58sNvdjgFrEPNicvvnI4dneAeUTIQ==
X-Google-Smtp-Source: AGHT+IG/L/BmGQ2U9fC5PJH7SYIWbjHtKtF0h1j52+bqP4RxaHBdT3TcvG77WJj2gJmrEU1v05A0+Q==
X-Received: by 2002:a17:906:a0ce:b0:a3d:8f37:c37f with SMTP id bh14-20020a170906a0ce00b00a3d8f37c37fmr14099147ejb.6.1708530437934;
        Wed, 21 Feb 2024 07:47:17 -0800 (PST)
Message-ID: <8dbf3be5-4954-493d-acdc-aed0603943f0@suse.com>
Date: Wed, 21 Feb 2024 16:47:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] build: make sure build fails when running kconfig fails
Content-Language: en-US
To: Anthony PERARD <anthony.perard@cloud.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <55c3a781-17f8-47f2-9629-515e1aea77aa@suse.com>
 <b7c22718-ec6c-470e-be72-e613b2af3c54@perard>
 <abbae372-5e3e-4013-a2a3-45892ec96c89@suse.com>
 <ed53b507-8ce1-4496-b2c2-cb0089f384c0@perard>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ed53b507-8ce1-4496-b2c2-cb0089f384c0@perard>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.02.2024 16:22, Anthony PERARD wrote:
> On Tue, Feb 20, 2024 at 01:20:16PM +0100, Jan Beulich wrote:
>> On 20.02.2024 12:18, Anthony PERARD wrote:
>>> On Tue, Feb 20, 2024 at 09:43:56AM +0100, Jan Beulich wrote:
>>>> Because of using "-include", failure to (re)build auto.conf (with
>>>> auto.conf.cmd produced as a secondary target) won't stop make from
>>>> continuing the build. Arrange for it being possible to drop the - from
>>>> Rules.mk, requiring that the include be skipped for tools-only targets.
>>>> Note that relying on the inclusion in those cases wouldn't be correct
>>>> anyway, as it might be a stale file (yet to be rebuilt) which would be
>>>> included, while during initial build, the file would be absent
>>>> altogether.
>>>>
>>>> Fixes: 8d4c17a90b0a ("xen/build: silence make warnings about missing auto.conf*")
>>>> Reported-by: Roger Pau Monné <roger.pau@citrix.com>
>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>
>>> Just to make sure, this patch is a workaround to a harmless bug in older
>>> version of GNU Make which print spurious error messages, and said bug as
>>> been fixed in GNU Make 4.2, right? Bug report:
>>>
>>>     bug #102: Make prints an incorrect error for missing includes
>>>     https://savannah.gnu.org/bugs/?func=detailitem&item_id=102
>>
>> Assuming you mean the change referenced in Fixes: - yes, that's my
>> understanding. (Whether the referenced make bug is the one I can't tell,
>> though. But it looks like it is.)
>>
>>>> --- a/xen/Makefile
>>>> +++ b/xen/Makefile
>>>> @@ -375,6 +375,7 @@ $(KCONFIG_CONFIG): tools_fixdep
>>>>  # This exploits the 'multi-target pattern rule' trick.
>>>>  # The syncconfig should be executed only once to make all the targets.
>>>>  include/config/%.conf include/config/%.conf.cmd: $(KCONFIG_CONFIG)
>>>> +	$(Q)rm -f include/config/$*.conf
>>>
>>> Maybe this should say "include/config/auto.conf" instead of using "$*".
>>> "syncconfig" is going to generate "auto.conf" and not "$*.conf". And it
>>> would make easier to find the "rm" command via "grep".
>>
>> Generally I advocate for using $* wherever possible in pattern rules. If,
> 
> Well, it might look like a pattern rule, but it isn't one. It's a
> "groupes targets" rule that works on version of Make that doesn't have
> this brand new feature. It would be written as:
>     include/config/auto.conf include/config/auto.conf.cmd&: $(KCONFIG_CONFIG)

Assuming new enough make.

> Second, the other command in the recipe doesn't use "$*", so it's not
> possible to use this rule with a value of $* other than "auto".

True. "auto" isn't used there either, though.

To emphasize this isn't really a pattern rule, how about shrinking the
stem along the lines of what I've done in

https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=79d635fc64ba9ca97c24c0c52c9924dba77e9527

e.g. making it just the (first) dot here?

>> however, replacing that is the only way to get an ack for this change, then
>> I would (hesitantly) do so.
> 
> Yes please, even if I don't like the patch. I hope it isn't going to
> byte us later if something in "tools/" happens to start depending on
> Kconfig (only fixdep and kconfig need to not depend on kconfig at the
> moment). But the patch currently works and looks ok. So, with "$*"
> replace by "auto", you can add my "Reviewed-by: Anthony PERARD
> <anthony.perard@citrix.com>".

Thanks.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 16:49:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 16:49:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684112.1063807 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcpmQ-0001jJ-Q9; Wed, 21 Feb 2024 16:49:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684112.1063807; Wed, 21 Feb 2024 16:49:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcpmQ-0001jC-MM; Wed, 21 Feb 2024 16:49:10 +0000
Received: by outflank-mailman (input) for mailman id 684112;
 Wed, 21 Feb 2024 16:49:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=E9i0=J6=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rcpmO-0001j4-PI
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 16:49:08 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 20ae7dba-d0d9-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 17:49:07 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a3e75e30d36so655808366b.1
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 08:49:07 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 vx11-20020a170907a78b00b00a3ee688f7ddsm2020021ejc.106.2024.02.21.08.49.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 21 Feb 2024 08:49:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 20ae7dba-d0d9-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708534146; x=1709138946; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1FPP5xYfSLLFQ+cTbSnxV4l0XE95TJRefz79jjMu0+4=;
        b=ANFrxg1+s2r7/lMy2tjszgk9hGcV/Yn18hBgFMOWE7YpvCoEXjxF4D/JQlT4/bokKq
         I0KS/BKsYAZIpIq3J2I4Bk9cdcwa5oC25ParDuQqZl2tnT2Ab3FTu34V6EBgVejX7UEc
         J8Sf6UMliID3x59h4DZ0Fqs8E/6zQLx5jKB18=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708534146; x=1709138946;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1FPP5xYfSLLFQ+cTbSnxV4l0XE95TJRefz79jjMu0+4=;
        b=FrbByCtvOW9kJ/jy+mPlRsN9t8k+ti4teKGRcEtzsetXGYLPhlyjxdIE0p9i6YfJhi
         BFxxufQ/C5qy36lvCy9mZtNiLaA5F52Imzqkqw/b731B1cUA9REH9TbgniKQ18sigiQf
         ljgjmOsYMPptWK0PsFJilwegCWRSCChZAOqn63NgPrB+5KYYG6h35U+4hcVV8d7vgjzY
         B7qg+0iG9fFfH+EZHR6zDizI2U3NG7UM6ORB4zuAUxqWQ8/4YzDruaXqz2cMQC7iB7g8
         gEOR7EYbRAEsPBB6L/ObtnBv1z94IC2e3PSuB4A030l8cdlhuYfVpf4rFFngFYBve35d
         Upqg==
X-Forwarded-Encrypted: i=1; AJvYcCWL/iHi4a+RmF8ZE/+NHf/mRlBrXMZ6c5KjzkUtIJ3biaKg+IidPpTmU/0dohJJF69t13SiM1BFaRDQ5HcMupFfJ6bprWisSmYCD8EOWUI=
X-Gm-Message-State: AOJu0YwnSTtEBrlQ4MdoFbB6hVtyTg9y6ur7uIcfCu8cio/AFK9tIIyH
	H53WOEWOc3MLEKFePoBl88VOP9UCLTiThJua1LTfPbVwHiDHRR5EWKrFevwy/+UJ0+Kh11miv9b
	P
X-Google-Smtp-Source: AGHT+IECSfH69AooszsXLhwE04VrNP7ZTOaXeNKO2rRUs9XwZUWR+tHwcRfm1KaYC9Z9FHd/63n1GA==
X-Received: by 2002:a17:906:2845:b0:a3f:50f0:7a0a with SMTP id s5-20020a170906284500b00a3f50f07a0amr37855ejc.20.1708534146560;
        Wed, 21 Feb 2024 08:49:06 -0800 (PST)
Message-ID: <7c816246-3a98-4696-ba54-4db804b57e8b@citrix.com>
Date: Wed, 21 Feb 2024 16:49:05 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86emul: add missing EVEX.R' checks
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <52d2b102-1285-41fc-a550-ca9437df612b@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <52d2b102-1285-41fc-a550-ca9437df612b@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 21/02/2024 10:27 am, Jan Beulich wrote:
> EVEX.R' is not ignored in 64-bit code when encoding a GPR or mask
> register. While for mask registers suitable checks are in place (there
> also covering EVEX.R), they were missing for the few cases where in
> EVEX-encoded instructions ModR/M.reg encodes a GPR. While for VPEXTRW
> the bit is replaced before an emulation stub is invoked, for
> VCVT{,T}{S,D,H}2{,U}SI this actually would have led to #UD from inside
> an emulation stub, in turn raising #UD to the guest, but accompanied by
> log messages indicating something's wrong in Xen nevertheless.
>
> Fixes: 001bd91ad864 ("x86emul: support AVX512{F,BW,DQ} extract insns")
> Fixes: baf4a376f550 ("x86emul: support AVX512F legacy-equivalent scalar int/FP conversion insns")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 16:55:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 16:55:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684117.1063817 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcpsp-0003Hu-ED; Wed, 21 Feb 2024 16:55:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684117.1063817; Wed, 21 Feb 2024 16:55:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcpsp-0003Hn-Ap; Wed, 21 Feb 2024 16:55:47 +0000
Received: by outflank-mailman (input) for mailman id 684117;
 Wed, 21 Feb 2024 16:55:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c6My=J6=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rcpso-0003Hh-Nz
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 16:55:46 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0cfff80a-d0da-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 17:55:43 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-4127190ad83so12119805e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 08:55:43 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 l15-20020a1c790f000000b0040fe308ff25sm3022782wme.24.2024.02.21.08.55.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 21 Feb 2024 08:55:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0cfff80a-d0da-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708534543; x=1709139343; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=KwUFUpoW5hHVQuEktH7p0pFcmfkKAd6VvCgn5VWIBq8=;
        b=ELe4qwHwT5qp0VIQCWnfRQh5WUXM2A2bXX3YQ3MUVMsqdaQkmUX290EXj5EluAoLfI
         taqeXj1duddWdDM9eLrvqboUI7jy4lKl4edAz41+FI++yB6rcrdNAwBmIDTXTUb6+U4K
         Nij98QWLO2DZebtCtTen1Bmjm/IWFVzaHL3L4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708534543; x=1709139343;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=KwUFUpoW5hHVQuEktH7p0pFcmfkKAd6VvCgn5VWIBq8=;
        b=H0oZcYQSuYdl50ZiXM+UzJ8UgEid6djdmUq2MFVe0vXCdks+50F69+8wpzCdA9YOFz
         lFt3/8dc1j0qThCap/US8QJYtZSyfpzI9+8DTpPLpqNZV6evK5MXPn/F268LS2Zm+0bF
         3njiKyn5z3mpm/X9zNcgJw+wWLCwrUccnzw7fqHHzBE3g41IkiqxLjscYqUbmr/GxmT2
         Xwc8yy6MieVEVCmC0r/YNXUWWdHSzjbNaLQHYTaLILE6iUjsYbFo0Lw9yARHb682XddJ
         4PqEZ/QWo4sHbxj3udv8mD82MuRb6T1VGVFqNOcko8Xted8IwCb1KbvR+cqYDioX0YDb
         fxKQ==
X-Gm-Message-State: AOJu0YxZTTHhzwH3BvWuIN29nVS6izDDIdttIl/g++5SBmOdCKdHmT7p
	WbAPAj/7l9BK5k/Lhduc6VInX7QSCaFtga8VW7iC+Kqyy9AEhWDRpCuDYCHOtlI=
X-Google-Smtp-Source: AGHT+IGnkfV7PLqsPGHrgrCeBxbJAklnA9H4oNODwrkUkGyxHGxiLDvgJLOBi56XP4A1vXSm/fEgDw==
X-Received: by 2002:a05:600c:4747:b0:412:71af:8af5 with SMTP id w7-20020a05600c474700b0041271af8af5mr3207487wmo.16.1708534543122;
        Wed, 21 Feb 2024 08:55:43 -0800 (PST)
Date: Wed, 21 Feb 2024 16:55:42 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] libxl: Use vkb=[] for HVMs
Message-ID: <6e22cbfa-84db-4556-b8cf-2563810ab558@perard>
References: <20240109171631.40071-1-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240109171631.40071-1-jandryuk@gmail.com>

On Tue, Jan 09, 2024 at 12:16:31PM -0500, Jason Andryuk wrote:
> xl/libxl only applies vkb=[] to PV & PVH guests.  HVM gets only a single
> vkb by default, but that can be disabled by the vkb_device boolean.
> Notably the HVM vkb cannot be configured, so feature-abs-pointer or the
> backend-type cannot be specified.
> 
> Re-arrange the logic so that vkb=[] is handled regardless of domain
> type.  If vkb is empty or unspecified, follow the vkb_device boolean for
> HVMs.  Nothing changes for PVH & PV.  HVMs can now get a configured vkb
> instead of just the default one.
> 
> The chance for regression is an HVM config with
> vkb=["$something"]
> vkb_device=false
> 
> Which would now get a vkb.
> 
> This is useful for vGlass which provides a VKB to HVMs.  vGlass wants to
> specify feature-abs-pointer, but that is racily written by vGlass
> instead of coming through the xl.cfg.  Unhelpfully, Linux xen-kbdfront
> reads the backend nodes without checking that the backend is in
> InitWait.
> 
> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>

I guess we would want an entry in the CHANGELOG about "vkb=[]" been
available on HVM guests.


> diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
> index ce1d431103..39c50b3711 100644
> --- a/tools/libs/light/libxl_create.c
> +++ b/tools/libs/light/libxl_create.c
> @@ -1771,24 +1771,31 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
>          libxl__device_add(gc, domid, &libxl__virtio_devtype,
>                            &d_config->virtios[i]);
>  
> +    if (d_config->num_vkbs) {
> +        for (i = 0; i < d_config->num_vkbs; i++) {
> +            libxl__device_add(gc, domid, &libxl__vkb_devtype,
> +                              &d_config->vkbs[i]);

While there, could you check the return value of that function? That
would be having:
    ret = libxl__device_add();
    if (ret) goto error_out;

With that: Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 17:03:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 17:03:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684124.1063828 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcq0X-00053i-76; Wed, 21 Feb 2024 17:03:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684124.1063828; Wed, 21 Feb 2024 17:03:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcq0X-00053b-2t; Wed, 21 Feb 2024 17:03:45 +0000
Received: by outflank-mailman (input) for mailman id 684124;
 Wed, 21 Feb 2024 17:03:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=8C+a=J6=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rcq0W-00053V-5G
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 17:03:44 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2a2d4c01-d0db-11ee-98f5-efadbce2ee36;
 Wed, 21 Feb 2024 18:03:42 +0100 (CET)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2d0cdbd67f0so95483661fa.3
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 09:03:42 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 p10-20020a05620a056a00b00785d9e3b744sm4465716qkp.108.2024.02.21.09.03.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 21 Feb 2024 09:03:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2a2d4c01-d0db-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708535021; x=1709139821; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=vP4gMZWRt3kH7TAC3U/8MIPnSOHiEExJGIwVsdIZ4b0=;
        b=dBDPsVWC1bT52aZCo785iFYPTYDgY4qjzeZrxejlRvyH94c/ubU2SS3i021MntkU5v
         CcEy5qOAg8+LNNMalktdVyxISdGdoQLr2ylY3SFgEEuSbq86CrA9e/mILgsRKZD4PeMF
         ziu0c+oxeTCJW2LlE5GinPqt0gLkgjaddn8GU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708535021; x=1709139821;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=vP4gMZWRt3kH7TAC3U/8MIPnSOHiEExJGIwVsdIZ4b0=;
        b=iUmObAP/y70yJxR669iOLitdNRTEjSSFyBSOvU9wiQfDmlfcr5ZGSQQ+GYmamo1tQ7
         ZX6t58icY358n577o3GiMbrXOLvxK3uvv6freCKzEwQwqQojQiBHp1ero3x22VGb0rq1
         rGbZihPaZQOqWcwgeYGyxY38L7T7n1G1U8htQOljJdD7Ig8FruFdfCR3mG2PAApf9cGA
         z3r3cJcvTzdG0RFubXLZQ8LdOLt6qodNGIcsN4d7knCffV9mxZidl79mHyrgSbJ6FhQA
         sh4DX8RkO/MhzIrQ/WhM4VEQuH+FR9gqhmAbsJoRuiSXjQR0BHE326Mm6HMSqMRd0qRa
         sGZQ==
X-Gm-Message-State: AOJu0YyVJLCk/CVW18LiMD9turuhURoOUHIpPFEDf4p2ybRlcQfpUR45
	O+XLPwlR8SbbTC9hWpgFTe44tHHNbdkQ2FnU/tUhkwQ37BkrzHzG4+jv2qMWVodOQkP2zOGDDFU
	9
X-Google-Smtp-Source: AGHT+IH5j3kGz37N+2rV3nlptBNUSFLzJus9O05ZAevSgc1+blIeBsAlCf2EbPFkdjAOP3Cjm6bcGw==
X-Received: by 2002:a05:651c:1988:b0:2d2:2fb7:b30f with SMTP id bx8-20020a05651c198800b002d22fb7b30fmr9457535ljb.7.1708535020648;
        Wed, 21 Feb 2024 09:03:40 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/altcall: use an union as register type for function parameters
Date: Wed, 21 Feb 2024 18:03:31 +0100
Message-ID: <20240221170331.60605-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The current code for alternative calls uses the caller parameter types as the
types for the register variables that serve as function parameters:

uint8_t foo;
[...]
alternative_call(myfunc, foo);

Would expand roughly into:

register unint8_t a1_ asm("rdi") = foo;
register unsigned long a2_ asm("rsi");
[...]
asm volatile ("call *%c[addr](%%rip)"...);

However under certain circumstances clang >= 16.0.0 with -O2 can generate
incorrect code, given the following example:

unsigned int func(uint8_t t)
{
    return t;
}

static void bar(uint8_t b)
{
    int ret_;
    register uint8_t di asm("rdi") = b;
    register unsigned long si asm("rsi");
    register unsigned long dx asm("rdx");
    register unsigned long cx asm("rcx");
    register unsigned long r8 asm("r8");
    register unsigned long r9 asm("r9");
    register unsigned long r10 asm("r10");
    register unsigned long r11 asm("r11");

    asm volatile ( "call %c[addr]"
                   : "+r" (di), "=r" (si), "=r" (dx),
                     "=r" (cx), "=r" (r8), "=r" (r9),
                     "=r" (r10), "=r" (r11), "=a" (ret_)
                   : [addr] "i" (&(func)), "g" (func)
                   : "memory" );
}

void foo(unsigned int a)
{
    bar(a);
}

Clang generates the following code:

func:                                   # @func
        movl    %edi, %eax
        retq
foo:                                    # @foo
        callq   func
        retq

Note the truncation of the unsigned int parameter 'a' of foo() to uint8_t when
passed into bar() is lost.

The above can be worked around by using an union when defining the register
variables, so that `di` becomes:

register union {
    uint8_t e;
    unsigned long r;
} di asm("rdi") = { .e = b };

Which results in following code generated for `foo()`:

foo:                                    # @foo
        movzbl  %dil, %edi
        callq   func
        retq

So the truncation is not longer lost.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Seems like all gitlab build tests are OK with this approach.
---
 xen/arch/x86/include/asm/alternative.h | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/include/asm/alternative.h b/xen/arch/x86/include/asm/alternative.h
index a1cd6a9fe5b8..837dfd953d2f 100644
--- a/xen/arch/x86/include/asm/alternative.h
+++ b/xen/arch/x86/include/asm/alternative.h
@@ -167,9 +167,18 @@ extern void alternative_branches(void);
 #define ALT_CALL_arg5 "r8"
 #define ALT_CALL_arg6 "r9"
 
-#define ALT_CALL_ARG(arg, n) \
-    register typeof(arg) a ## n ## _ asm ( ALT_CALL_arg ## n ) = \
-        ({ BUILD_BUG_ON(sizeof(arg) > sizeof(void *)); (arg); })
+/*
+ * Use an union with an unsigned long in order to prevent clang >= 16.0.0 from
+ * skipping a possible truncation of the value.  By using the union any
+ * truncation is carried before the call instruction.
+ */
+#define ALT_CALL_ARG(arg, n)                                            \
+    register union {                                                    \
+        typeof(arg) e;                                                  \
+        unsigned long r;                                                \
+    } a ## n ## _ asm ( ALT_CALL_arg ## n ) = {                         \
+        .e = ({ BUILD_BUG_ON(sizeof(arg) > sizeof(void *)); (arg); })   \
+    }
 #define ALT_CALL_NO_ARG(n) \
     register unsigned long a ## n ## _ asm ( ALT_CALL_arg ## n )
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 21 17:11:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 17:11:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684129.1063837 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcq7Z-0006ni-Su; Wed, 21 Feb 2024 17:11:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684129.1063837; Wed, 21 Feb 2024 17:11:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcq7Z-0006na-PV; Wed, 21 Feb 2024 17:11:01 +0000
Received: by outflank-mailman (input) for mailman id 684129;
 Wed, 21 Feb 2024 17:11:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=c6My=J6=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rcq7Y-0006nU-BY
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 17:11:00 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2ee6d943-d0dc-11ee-8a53-1f161083a0e0;
 Wed, 21 Feb 2024 18:10:59 +0100 (CET)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a30f7c9574eso875309966b.0
 for <xen-devel@lists.xenproject.org>; Wed, 21 Feb 2024 09:10:59 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 cw4-20020a170907160400b00a3e45a18a5asm4302508ejd.61.2024.02.21.09.10.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 21 Feb 2024 09:10:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2ee6d943-d0dc-11ee-8a53-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708535459; x=1709140259; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=cs7RnsYILH+rArcNup3mExB8svxqC+WstkwV98sQgS4=;
        b=dsQpH/u63ZVjr7CU9RQS+L3m4IceEeLQjcOLiVzkk83id9bf74Vv7f8eIShTfB9XEf
         /V+JvOmhLiurDA9qhyR8RvmDtbwIoGeyhBxZFHtWpGGYu6ESFZO4XftcLQiq2xFYMUjm
         VOpVDCfg5zUSim7abobl5cEljCJIBrVg/P7B0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708535459; x=1709140259;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cs7RnsYILH+rArcNup3mExB8svxqC+WstkwV98sQgS4=;
        b=Q7xcAEIIvOnHGUGy1tDZmzHjObt3UxRUmt+kBP3zK+xoNCr0k2rJdhmLLXp0RXOMcD
         lXj57zZFWuTwNywI3DQjyOjZJC79pTrYe7Z0gJYMZxPbjvMxgrcK13TbZwOvCT471zT2
         hUofP3Xv+LtfcnLqdYu54c4ef2LAAR+1aN+n6nKQGKHIVludd8pG+2kBrZLZz6NtlDSw
         pP9Mlq0aemgM8kHWIyiMEUp/Xg/yNaPwxzPeQNpU9vsSA4Ik2FP4SPU5zj5O1zIlXDDL
         n0h78Bg4SWhPU9qQTFsi/6zyqhh331YtESwhhvssf6hM2rzKECEqJbJtGoTmJavaXswM
         A02w==
X-Forwarded-Encrypted: i=1; AJvYcCXxhtr0KUtWti6THe4AkYtgv4xntNRd9TeaVZUrRg8EmG6b1AImGLcODeM/mwK+QhHB66sF9huS/BB6x/ZqvzrN4WCAT7pqSekEAaI53vQ=
X-Gm-Message-State: AOJu0Yy+XZB8M/vPK881PyxIj8u95j4FKDB1XRW3Cnz05rRTmdw4IkPD
	x6iphxF3jvcBoCQThYMuE0CkdhixU1rLBdjiI0GaUqXpnT6DP4NLjkjBXqmjc0U=
X-Google-Smtp-Source: AGHT+IFxik8yBEUOqNWyWl0K1/hJtildBCKKylIhwwPL53CFiUXCI82BUqp2CnKsfb4IQQoZG09OrQ==
X-Received: by 2002:a17:906:2990:b0:a3f:218b:444d with SMTP id x16-20020a170906299000b00a3f218b444dmr2644387eje.6.1708535459005;
        Wed, 21 Feb 2024 09:10:59 -0800 (PST)
Date: Wed, 21 Feb 2024 17:10:58 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: Marek =?iso-8859-1?Q?Marczykowski-G=F3recki?= <marmarek@invisiblethingslab.com>,
	xen-devel@lists.xenproject.org, Neowutran <xen@neowutran.ovh>,
	Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH] libxl: Disable relocating memory for qemu-xen in
 stubdomain too
Message-ID: <5ce1bb5a-9605-49ff-a783-44e90ef3df06@perard>
References: <20231227023544.1253277-1-marmarek@invisiblethingslab.com>
 <CAKf6xpsa_8nRyR-dY8sTQ6HsywZBmDvu29WwerxNkvJFqHq+fQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAKf6xpsa_8nRyR-dY8sTQ6HsywZBmDvu29WwerxNkvJFqHq+fQ@mail.gmail.com>

On Wed, Dec 27, 2023 at 07:18:12PM -0500, Jason Andryuk wrote:
> On Tue, Dec 26, 2023 at 11:49 PM Marek Marczykowski-Górecki
> <marmarek@invisiblethingslab.com> wrote:
> >
> > According to comments (and experiments) qemu-xen cannot handle memory
> > reolcation done by hvmloader. The code was already disabled when running
> > qemu-xen in dom0 (see libxl__spawn_local_dm()), but it was missed when
> > adding qemu-xen support to stubdomain. Adjust libxl__spawn_stub_dm() to
> > be consistent in this regard.
> >
> > Reported-by: Neowutran <xen@neowutran.ovh>
> > Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> 
> Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 18:56:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 18:56:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684139.1063847 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcrln-0002PD-Ef; Wed, 21 Feb 2024 18:56:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684139.1063847; Wed, 21 Feb 2024 18:56:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcrln-0002P6-Bx; Wed, 21 Feb 2024 18:56:39 +0000
Received: by outflank-mailman (input) for mailman id 684139;
 Wed, 21 Feb 2024 18:56:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcrll-0002Ow-VZ; Wed, 21 Feb 2024 18:56:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcrll-0002tX-PC; Wed, 21 Feb 2024 18:56:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcrll-0003SO-Bx; Wed, 21 Feb 2024 18:56:37 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcrll-0005Yx-BT; Wed, 21 Feb 2024 18:56:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=OZnIQnF8g/e7JWkgQXSz4zS5oR0Bv3tGCGVl9Bi7Xys=; b=04lJ22qoMUfNe9xgCgdmw/Pbaz
	e+QvKuseFKwDEZZzMh/qS0i9MteQpgmOSsi9bSkz4eEjGPaDBwLcsXkMq43BDF+nyHHL7x/cH5Mpf
	OA06eQq68R/QMMY9wESVOY8iBGsQnvMjeMxGRDmTh44k6/MpEm1c9M6EBTH0JZCJG58c=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184720-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184720: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f8791d0fd3adbda3701e7eb9db63a9351b478365
X-Osstest-Versions-That:
    xen=f8791d0fd3adbda3701e7eb9db63a9351b478365
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 21 Feb 2024 18:56:37 +0000

flight 184720 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184720/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184716

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184716
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184716
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184716
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184716
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184716
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184716
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184716
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184716
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184716
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184716
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184716
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184716
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  f8791d0fd3adbda3701e7eb9db63a9351b478365
baseline version:
 xen                  f8791d0fd3adbda3701e7eb9db63a9351b478365

Last test of basis   184720  2024-02-21 09:18:51 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Wed Feb 21 22:54:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 22:54:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684152.1063857 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcvTR-00022d-NR; Wed, 21 Feb 2024 22:53:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684152.1063857; Wed, 21 Feb 2024 22:53:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcvTR-00022W-KW; Wed, 21 Feb 2024 22:53:57 +0000
Received: by outflank-mailman (input) for mailman id 684152;
 Wed, 21 Feb 2024 22:53:56 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcvTQ-00022Q-GZ
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 22:53:56 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcvTL-0007Pw-06; Wed, 21 Feb 2024 22:53:51 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcvTK-0005OV-OY; Wed, 21 Feb 2024 22:53:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=HG2wS8bjsKT7RcLwSoCzx+ndRPVLoiaDbJOeW7HDit4=; b=ZbmHzZL9IsZejGeKRWEysPdlHn
	QJfUSS5jS4/7MCriMWXXEZnptbQZmgtprq73KpoTqALYEqYZLAscKN148R3UVe1WIj6u690TBewfn
	pCrhcK+8Hoi7OpJcULSDoqdYJziy1Dnu85vXUPC5FdFpU0bFGgJZvF3DXmngcYu3TNMM=;
Message-ID: <0936a6c5-bac2-4de2-9155-1722d3bb28ca@xen.org>
Date: Wed, 21 Feb 2024 22:53:49 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Stats on Xen tarball downloads
Content-Language: en-GB
To: George Dunlap <george.dunlap@cloud.com>, Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Xen-devel <xen-devel@lists.xenproject.org>, committers@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>
References: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
 <ZdMt88BSNMYtq39n@macbook> <36229344-e2e2-46e1-839b-b7b3b0843f23@suse.com>
 <CA+zSX=YvTgEMC0S5OqSFvxfVDeruiO=FphyVx5fv75oT-QW01A@mail.gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <CA+zSX=YvTgEMC0S5OqSFvxfVDeruiO=FphyVx5fv75oT-QW01A@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi George,

On 21/02/2024 02:55, George Dunlap wrote:
> On Mon, Feb 19, 2024 at 6:38 PM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 19.02.2024 11:31, Roger Pau Monné wrote:
>>> On Mon, Feb 19, 2024 at 06:01:54PM +0800, George Dunlap wrote:
>>>> One of the questions we had with respect to changing our release
>>>> practice (for instance, making the process more light-weight so that
>>>> we could do a point release after every XSA) was, "How many people are
>>>> actually using the tarballs?"
>>>
>>> What would this more lightweight process involve from a downstream
>>> PoV?  IOW: in what would the contents of the tarball change compared
>>> to the current releases?
>>
>>  From all prior discussion my conclusion was "no tarball at all".
> 
> Or at very least, the tarball would be a simple `git archive` of a
> release tag.   Right now the tarball creation has a number of
> annoyingly manual parts about it.
At the moment we have the following steps:

1) Checkout tag
2) Create the tarball
3) Check the source tarball can build
4) Sign the tarball
5) Upload it

I managed to script it so I have only two commands to execute (mostly 
because I build and sign on a different host).

AFAIU, your command 'git archive' will only replace 2. Am I correct? If 
so, it is not entirely clear how your proposal is going to make it better.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 23:01:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 23:01:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684160.1063867 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcvaM-0003kj-H9; Wed, 21 Feb 2024 23:01:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684160.1063867; Wed, 21 Feb 2024 23:01:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcvaM-0003kc-Do; Wed, 21 Feb 2024 23:01:06 +0000
Received: by outflank-mailman (input) for mailman id 684160;
 Wed, 21 Feb 2024 23:01:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcvaL-0003kW-Ep
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 23:01:05 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcvaK-0007jM-Ho; Wed, 21 Feb 2024 23:01:04 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcvaK-0005ze-Aj; Wed, 21 Feb 2024 23:01:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=/UnwQ4f25YcwzxadFU1xxhOehoEUl+3dH/muFsfVpSo=; b=xC/hi43GphoZA/CjIWLlj7fRF3
	WM2m6MEeP2FSadT8o1MR4BqsD2uDf/QrQjmWhoqS8ZFLsLQH4ENYAeBuUztru6Sz/zJjpZ7ljG7xN
	35rVogJtMXhsVPLyq6DYYWKvPiuN17Drp3qP0JenE3igxCy0qNOc+UJwekNaoW3lMcr8=;
Message-ID: <f7a07d36-10ac-4f89-9ba7-cfa337eee13b@xen.org>
Date: Wed, 21 Feb 2024 23:01:02 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 1/8] serial: fake IRQ-regs context in poll handlers
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
 <a3644326-7514-40f9-939c-2b6294dc5a8b@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <a3644326-7514-40f9-939c-2b6294dc5a8b@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 20/02/2024 08:52, Jan Beulich wrote:
> In preparation of dropping the register parameters from
> serial_[rt]x_interrupt() and in turn from IRQ handler functions,
> register state needs making available another way for the few key
> handlers which need it. Fake IRQ-like state.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 21 23:02:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 21 Feb 2024 23:02:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684165.1063877 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcvbn-0004Ga-Qb; Wed, 21 Feb 2024 23:02:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684165.1063877; Wed, 21 Feb 2024 23:02:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcvbn-0004GR-Mv; Wed, 21 Feb 2024 23:02:35 +0000
Received: by outflank-mailman (input) for mailman id 684165;
 Wed, 21 Feb 2024 23:02:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rcvbl-0004GG-Rr
 for xen-devel@lists.xenproject.org; Wed, 21 Feb 2024 23:02:33 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcvbk-0007kp-OQ; Wed, 21 Feb 2024 23:02:32 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rcvbk-0005ze-IR; Wed, 21 Feb 2024 23:02:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=LyW+Ud3uGdVm+rDUBBEm/zt10hSu5jEXlATe7kRh9RI=; b=fpqYylFUY7en5+ZvQUYBNIOn04
	9nbQD5UC2f9Qit6vwJpHENKPcZBFMgeNhhcGs0gKWm/AFbeXC89VGox42Pdp4QbDValM8Q3j6ueNQ
	5zrak9B2DvzK3HJO2Rvu8BKJ9ro2I2HbJEPhFbgVNmYITa4f7Jy7S3pi+HvHTrFrFnfs=;
Message-ID: <801ed065-cdc1-491b-b34e-8cc4a964257d@xen.org>
Date: Wed, 21 Feb 2024 23:02:30 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 5/8] serial: drop serial_[rt]x_interrupt()'s regs
 parameter
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Michal Orzel <michal.orzel@amd.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <e5f62a45-5b68-404d-908e-909f84f87b44@suse.com>
 <6476a662-14f2-4206-83c6-a561c6c4aee5@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <6476a662-14f2-4206-83c6-a561c6c4aee5@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 20/02/2024 08:55, Jan Beulich wrote:
> They're simply not needed anymore.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 01:04:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 01:04:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684176.1063887 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcxVd-0000ss-El; Thu, 22 Feb 2024 01:04:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684176.1063887; Thu, 22 Feb 2024 01:04:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rcxVd-0000sl-C1; Thu, 22 Feb 2024 01:04:21 +0000
Received: by outflank-mailman (input) for mailman id 684176;
 Thu, 22 Feb 2024 01:04:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcxVc-0000sb-22; Thu, 22 Feb 2024 01:04:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcxVb-0000cu-JD; Thu, 22 Feb 2024 01:04:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rcxVb-0005Ia-3C; Thu, 22 Feb 2024 01:04:19 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rcxVb-0001TW-2p; Thu, 22 Feb 2024 01:04:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NZE+UvjFXY/xsfoK/o7hBI/s3Mq/Wpc7iTP45gFxGb0=; b=1dfpun8ecpEIMYwvuGDka+ZR6R
	ty3RM+mILWKcvXXhKFAzCnbWvvlN72NQlOpJjAVf36UxDndI0FwkfBigi0La1n06G7or7U4A0PAee
	hVvc5tpADncpAk3aKmas7YfqSAyG1y4GQrYfocqNFXCS9ox7PAV8a6n9d5naUqIEzh9A=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184721-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184721: regressions - FAIL
X-Osstest-Failures:
    linux-linus:build-arm64-pvops:kernel-build:fail:regression
    linux-linus:test-arm64-arm64-examine:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=39133352cbed6626956d38ed72012f49b0421e7b
X-Osstest-Versions-That:
    linux=9fc1ccccfd8d53dc7936fe6d633f2373fc9f62e8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 22 Feb 2024 01:04:19 +0000

flight 184721 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184721/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvops             6 kernel-build             fail REGR. vs. 184719

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-examine      1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184719
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184719
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184719
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184719
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184719
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184719
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184719
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184719
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                39133352cbed6626956d38ed72012f49b0421e7b
baseline version:
 linux                9fc1ccccfd8d53dc7936fe6d633f2373fc9f62e8

Last test of basis   184719  2024-02-21 06:34:23 Z    0 days
Testing same since   184721  2024-02-21 17:43:51 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  David Sterba <dsterba@suse.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Jason Wang <jasowang@redhat.com>
  Josef Bacik <josef@toxicpanda.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Marc Zyngier <maz@kernel.org>
  Michael S. Tsirkin <mst@redhat.com>
  Nathan Chancellor <nathan@kernel.org> # build
  Oliver Upton <oliver.upton@linux.dev>
  Paolo Bonzini <pbonzini@redhat.com>
  Qu Wenruo <wqu@suse.com>
  zhenwei pi <pizhenwei@bytedance.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            fail    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          blocked 
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  blocked 
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  blocked 
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     blocked 
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 blocked 
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      blocked 
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 39133352cbed6626956d38ed72012f49b0421e7b
Merge: 8da8d88455eb c48617fbbe83
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed Feb 21 09:13:27 2024 -0800

    Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
    
    Pull kvm fixes from Paolo Bonzini:
     "Two fixes for ARM ITS emulation. Unmapped interrupts were used instead
      of ignored, causing NULL pointer dereferences"
    
    * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm:
      KVM: arm64: vgic-its: Test for valid IRQ in MOVALL handler
      KVM: arm64: vgic-its: Test for valid IRQ in its_sync_lpi_pending_table()

commit 8da8d88455ebbb4e05423cf60cff985e92d43754
Merge: d8be5a55b8e3 b0ad381fa769
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed Feb 21 08:45:07 2024 -0800

    Merge tag 'for-6.8-rc5-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux
    
    Pull btrfs fixes from David Sterba:
    
     - Fix a deadlock in fiemap.
    
       There was a big lock around the whole operation that can interfere
       with a page fault and mkwrite.
    
       Reducing the lock scope can also speed up fiemap
    
     - Fix range condition for extent defragmentation which could lead to
       worse layout in some cases
    
    * tag 'for-6.8-rc5-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux:
      btrfs: fix deadlock with fiemap and extent locking
      btrfs: defrag: avoid unnecessary defrag caused by incorrect extent size

commit d8be5a55b8e3f7eab8f36ceed2512f457f914318
Merge: 9fc1ccccfd8d c0ec2a712daf
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed Feb 21 08:37:49 2024 -0800

    Merge tag 'v6.8-p4' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
    
    Pull crypto fix from Herbert Xu:
     "Fix a stack overflow in virtio"
    
    * tag 'v6.8-p4' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6:
      crypto: virtio/akcipher - Fix stack overflow on memcpy

commit c48617fbbe831d4c80fe84056033f17b70a31136
Merge: 9895ceeb5cd6 85a71ee9a070
Author: Paolo Bonzini <pbonzini@redhat.com>
Date:   Wed Feb 21 05:18:56 2024 -0500

    Merge tag 'kvmarm-fixes-6.8-3' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
    
    KVM/arm64 fixes for 6.8, take #3
    
    - Check for the validity of interrupts handled by a MOVALL
      command
    
    - Check for the validity of interrupts while reading the
      pending state on enabling LPIs.

commit 85a71ee9a0700f6c18862ef3b0011ed9dad99aca
Author: Oliver Upton <oliver.upton@linux.dev>
Date:   Wed Feb 21 09:27:32 2024 +0000

    KVM: arm64: vgic-its: Test for valid IRQ in MOVALL handler
    
    It is possible that an LPI mapped in a different ITS gets unmapped while
    handling the MOVALL command. If that is the case, there is no state that
    can be migrated to the destination. Silently ignore it and continue
    migrating other LPIs.
    
    Cc: stable@vger.kernel.org
    Fixes: ff9c114394aa ("KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE")
    Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
    Link: https://lore.kernel.org/r/20240221092732.4126848-3-oliver.upton@linux.dev
    Signed-off-by: Marc Zyngier <maz@kernel.org>

commit 8d3a7dfb801d157ac423261d7cd62c33e95375f8
Author: Oliver Upton <oliver.upton@linux.dev>
Date:   Wed Feb 21 09:27:31 2024 +0000

    KVM: arm64: vgic-its: Test for valid IRQ in its_sync_lpi_pending_table()
    
    vgic_get_irq() may not return a valid descriptor if there is no ITS that
    holds a valid translation for the specified INTID. If that is the case,
    it is safe to silently ignore it and continue processing the LPI pending
    table.
    
    Cc: stable@vger.kernel.org
    Fixes: 33d3bc9556a7 ("KVM: arm64: vgic-its: Read initial LPI pending table")
    Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
    Link: https://lore.kernel.org/r/20240221092732.4126848-2-oliver.upton@linux.dev
    Signed-off-by: Marc Zyngier <maz@kernel.org>

commit b0ad381fa7690244802aed119b478b4bdafc31dd
Author: Josef Bacik <josef@toxicpanda.com>
Date:   Mon Feb 12 11:56:02 2024 -0500

    btrfs: fix deadlock with fiemap and extent locking
    
    While working on the patchset to remove extent locking I got a lockdep
    splat with fiemap and pagefaulting with my new extent lock replacement
    lock.
    
    This deadlock exists with our normal code, we just don't have lockdep
    annotations with the extent locking so we've never noticed it.
    
    Since we're copying the fiemap extent to user space on every iteration
    we have the chance of pagefaulting.  Because we hold the extent lock for
    the entire range we could mkwrite into a range in the file that we have
    mmap'ed.  This would deadlock with the following stack trace
    
    [<0>] lock_extent+0x28d/0x2f0
    [<0>] btrfs_page_mkwrite+0x273/0x8a0
    [<0>] do_page_mkwrite+0x50/0xb0
    [<0>] do_fault+0xc1/0x7b0
    [<0>] __handle_mm_fault+0x2fa/0x460
    [<0>] handle_mm_fault+0xa4/0x330
    [<0>] do_user_addr_fault+0x1f4/0x800
    [<0>] exc_page_fault+0x7c/0x1e0
    [<0>] asm_exc_page_fault+0x26/0x30
    [<0>] rep_movs_alternative+0x33/0x70
    [<0>] _copy_to_user+0x49/0x70
    [<0>] fiemap_fill_next_extent+0xc8/0x120
    [<0>] emit_fiemap_extent+0x4d/0xa0
    [<0>] extent_fiemap+0x7f8/0xad0
    [<0>] btrfs_fiemap+0x49/0x80
    [<0>] __x64_sys_ioctl+0x3e1/0xb50
    [<0>] do_syscall_64+0x94/0x1a0
    [<0>] entry_SYSCALL_64_after_hwframe+0x6e/0x76
    
    I wrote an fstest to reproduce this deadlock without my replacement lock
    and verified that the deadlock exists with our existing locking.
    
    To fix this simply don't take the extent lock for the entire duration of
    the fiemap.  This is safe in general because we keep track of where we
    are when we're searching the tree, so if an ordered extent updates in
    the middle of our fiemap call we'll still emit the correct extents
    because we know what offset we were on before.
    
    The only place we maintain the lock is searching delalloc.  Since the
    delalloc stuff can change during writeback we want to lock the extent
    range so we have a consistent view of delalloc at the time we're
    checking to see if we need to set the delalloc flag.
    
    With this patch applied we no longer deadlock with my testcase.
    
    CC: stable@vger.kernel.org # 6.1+
    Reviewed-by: Filipe Manana <fdmanana@suse.com>
    Signed-off-by: Josef Bacik <josef@toxicpanda.com>
    Reviewed-by: David Sterba <dsterba@suse.com>
    Signed-off-by: David Sterba <dsterba@suse.com>

commit e42b9d8b9ea2672811285e6a7654887ff64d23f3
Author: Qu Wenruo <wqu@suse.com>
Date:   Wed Feb 7 10:00:42 2024 +1030

    btrfs: defrag: avoid unnecessary defrag caused by incorrect extent size
    
    [BUG]
    With the following file extent layout, defrag would do unnecessary IO
    and result more on-disk space usage.
    
      # mkfs.btrfs -f $dev
      # mount $dev $mnt
      # xfs_io -f -c "pwrite 0 40m" $mnt/foobar
      # sync
      # xfs_io -f -c "pwrite 40m 16k" $mnt/foobar
      # sync
    
    Above command would lead to the following file extent layout:
    
            item 6 key (257 EXTENT_DATA 0) itemoff 15816 itemsize 53
                    generation 7 type 1 (regular)
                    extent data disk byte 298844160 nr 41943040
                    extent data offset 0 nr 41943040 ram 41943040
                    extent compression 0 (none)
            item 7 key (257 EXTENT_DATA 41943040) itemoff 15763 itemsize 53
                    generation 8 type 1 (regular)
                    extent data disk byte 13631488 nr 16384
                    extent data offset 0 nr 16384 ram 16384
                    extent compression 0 (none)
    
    Which is mostly fine. We can allow the final 16K to be merged with the
    previous 40M, but it's upon the end users' preference.
    
    But if we defrag the file using the default parameters, it would result
    worse file layout:
    
     # btrfs filesystem defrag $mnt/foobar
     # sync
    
            item 6 key (257 EXTENT_DATA 0) itemoff 15816 itemsize 53
                    generation 7 type 1 (regular)
                    extent data disk byte 298844160 nr 41943040
                    extent data offset 0 nr 8650752 ram 41943040
                    extent compression 0 (none)
            item 7 key (257 EXTENT_DATA 8650752) itemoff 15763 itemsize 53
                    generation 9 type 1 (regular)
                    extent data disk byte 340787200 nr 33292288
                    extent data offset 0 nr 33292288 ram 33292288
                    extent compression 0 (none)
            item 8 key (257 EXTENT_DATA 41943040) itemoff 15710 itemsize 53
                    generation 8 type 1 (regular)
                    extent data disk byte 13631488 nr 16384
                    extent data offset 0 nr 16384 ram 16384
                    extent compression 0 (none)
    
    Note the original 40M extent is still there, but a new 32M extent is
    created for no benefit at all.
    
    [CAUSE]
    There is an existing check to make sure we won't defrag a large enough
    extent (the threshold is by default 32M).
    
    But the check is using the length to the end of the extent:
    
            range_len = em->len - (cur - em->start);
    
            /* Skip too large extent */
            if (range_len >= extent_thresh)
                    goto next;
    
    This means, for the first 8MiB of the extent, the range_len is always
    smaller than the default threshold, and would not be defragged.
    But after the first 8MiB, the remaining part would fit the requirement,
    and be defragged.
    
    Such different behavior inside the same extent caused the above problem,
    and we should avoid different defrag decision inside the same extent.
    
    [FIX]
    Instead of using @range_len, just use @em->len, so that we have a
    consistent decision among the same file extent.
    
    Now with this fix, we won't touch the extent, thus not making it any
    worse.
    
    Reported-by: Filipe Manana <fdmanana@suse.com>
    Fixes: 0cb5950f3f3b ("btrfs: fix deadlock when reserving space during defrag")
    CC: stable@vger.kernel.org # 6.1+
    Reviewed-by: Boris Burkov <boris@bur.io>
    Reviewed-by: Filipe Manana <fdmanana@suse.com>
    Signed-off-by: Qu Wenruo <wqu@suse.com>
    Signed-off-by: David Sterba <dsterba@suse.com>

commit c0ec2a712daf133d9996a8a1b7ee2d4996080363
Author: zhenwei pi <pizhenwei@bytedance.com>
Date:   Tue Jan 30 19:27:40 2024 +0800

    crypto: virtio/akcipher - Fix stack overflow on memcpy
    
    sizeof(struct virtio_crypto_akcipher_session_para) is less than
    sizeof(struct virtio_crypto_op_ctrl_req::u), copying more bytes from
    stack variable leads stack overflow. Clang reports this issue by
    commands:
    make -j CC=clang-14 mrproper >/dev/null 2>&1
    make -j O=/tmp/crypto-build CC=clang-14 allmodconfig >/dev/null 2>&1
    make -j O=/tmp/crypto-build W=1 CC=clang-14 drivers/crypto/virtio/
      virtio_crypto_akcipher_algs.o
    
    Fixes: 59ca6c93387d ("virtio-crypto: implement RSA algorithm")
    Link: https://lore.kernel.org/all/0a194a79-e3a3-45e7-be98-83abd3e1cb7e@roeck-us.net/
    Cc: <stable@vger.kernel.org>
    Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
    Tested-by: Nathan Chancellor <nathan@kernel.org> # build
    Acked-by: Michael S. Tsirkin <mst@redhat.com>
    Acked-by: Jason Wang <jasowang@redhat.com>
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 06:23:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 06:23:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684189.1063897 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd2Tl-0001Mf-IS; Thu, 22 Feb 2024 06:22:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684189.1063897; Thu, 22 Feb 2024 06:22:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd2Tl-0001MY-ET; Thu, 22 Feb 2024 06:22:45 +0000
Received: by outflank-mailman (input) for mailman id 684189;
 Thu, 22 Feb 2024 06:22:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F9EN=J7=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1rd2Tj-0001MN-Tj
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 06:22:44 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2061c.outbound.protection.outlook.com
 [2a01:111:f400:7e88::61c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c7d3ce6e-d14a-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 07:22:40 +0100 (CET)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by DM6PR12MB4958.namprd12.prod.outlook.com (2603:10b6:5:20a::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.21; Thu, 22 Feb
 2024 06:22:37 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa%5]) with mapi id 15.20.7316.018; Thu, 22 Feb 2024
 06:22:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7d3ce6e-d14a-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Tva159OzZ62B3CGu9Rt6nCKkJ3JyOo0q7UB0gG9Uw8I5duhFj/ketWaTNjAGJT0tf6FLeDd6Jfax7pVa9KutW6Jrp6783ZbErcI3zbbfKy2opF6ZnTfjt1KTnRVQi/RcWvFwsgZ7Laj0GMKAVuJAvcNHMdAPwQhV/jWdByMFxTFMl8aBZjZKZtNlUmGgwOti65jqyXbljminLFheIgTk1Wd5vsD3x6tlTCrgq4KWGf1Qd/7PJ7eP9lD2pu/lbUQlfzLHnMXhq+N0QBc0G0R5FMJKdReKF1cBxvtPOihE7jvZvgt/T5zo7wjGGLkvw2oN9jTDvSsh/4GljaJOSeLZVA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=/8JDNaLthTdtyHVQWbnOQqPmvGWO/rBfwadANohq0Tg=;
 b=ViSCMAa2rx8HrKMLD0KYNlkvsEmHr0YTTi3yK8eKP3Gq0n8Zj9AnZDmBJhFnTYIpn0Brf3++qrkQSbNw5u6PUSKhAZoxSyiia6pWfsXbi7hDnBsUe5jr7s1J6woWXhz6pQVxsfw+xnYX/0ONxCPx4vT96FfDI8YAgDzNsPleybW+jUS16k6gHdn4I9ZgvF77Tcamq7BTbNF8X6qVRod7Tt5WOerBtWp0RhaxdYNB7pX76c3hfDypg9bC3Kwf0u3vJ64XNPyNKVVG8XAhz7GCzS2EUPgfgqD7MWA0jx/a75BmylZz2fe9okgaMhX6ZtIm5q0R8la+Chwh4QlBltVLyQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/8JDNaLthTdtyHVQWbnOQqPmvGWO/rBfwadANohq0Tg=;
 b=sXt9gKDPsYqQhtN4eEQ6ppQvniuImFrMpeass8rCi6+/QdZZ5u3gj2zZsPbq4bj0L2WcMwLIFmbfL36/csEDwJMuBlxewuAD0onKcpdxv2lf2liVkrBNQNXCPzhWH0eODlE3k04ZyufF3kOmk8j7Tr4HYVSrunU9HMJeq4t7+UE=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Anthony PERARD <anthony.perard@citrix.com>, Juergen
 Gross <jgross@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>,
	"Huang, Ray" <Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v5 1/5] xen/vpci: Clear all vpci status of device
Thread-Topic: [RFC XEN PATCH v5 1/5] xen/vpci: Clear all vpci status of device
Thread-Index: AQHaRR6lSBH6S20cFEGVGIBPBL8sdbECeg+AgBQujAA=
Date: Thu, 22 Feb 2024 06:22:37 +0000
Message-ID:
 <BL1PR12MB58493E40588F4A7BF351B596E7562@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-2-Jiqian.Chen@amd.com>
 <9b4ce4c7-6f4e-47ec-b3a4-0b417cd2660c@amd.com>
In-Reply-To: <9b4ce4c7-6f4e-47ec-b3a4-0b417cd2660c@amd.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL0PR12MB5539.namprd12.prod.outlook.com
 (15.20.7292.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|DM6PR12MB4958:EE_
x-ms-office365-filtering-correlation-id: d2d31197-489f-4e9b-275b-08dc336ea9fe
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 fhHbksfbiB05iFKxhPpR/xSd8997RE6Bk+Ki87T0AFhdclkSJIJ/YahKiQaTz7xg2l8KqqYAcYjlNwtQMQzsDzJYHJJRv1yaB8YFdwMG9uyoZxW4p3aoXvcN2PW7uM8AC+xPUaMnSgqvHa/8gtI9sinhUUVY9EAObdeDY+Ib0CRwQx7KgA8ehkijCRJJS51vtu8dimSB9+Vg2jj2oTonq7cVgfKYmFxvAGCw/2vsOvc9gFroUUVqPydpgUNfPLjy+SFwtAklu4wflE12GVpMMTaK1kn7kHJ95ZjX+681DgIz0YLh6FOD7Wq81OK1d/EhPZLyGidVPEL5MfwDXxDSczeg/jYgm/ivJv7g2uZhD+Qqu1ZXqia5EoVmu3/U0Dv5FHRWgqrvSsKpsWJUeIk/q30kMfgTvY2chOgbGZwQUhCWaaqsux+HQunQWLVNjqxcKwZaIjjY2bNvVDMKcYMAZOCDxjlFcbTaZnPN/rWUC95jzYZA+Q0IcKPsAA4Mu07P0w49aAB79XAcsQf7+aFHNO9J/eLCtOyMS/ywgliYZTG14NLag62tYVndp/ieysUIsnJPgM6Ca4oQxwqz0TqOHFfg3epRqaaljUwYLN5oi5M=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?UnlpMXk5QXhnQXdlamJMTGh5ZDN5Z3VBb2tLT1hBL1o5WGJHOWxWRmZsVlBZ?=
 =?utf-8?B?MkJZSngycWd5cEtFcVhEeGpjYTRiVTR5ZExGZHlnNGpQTTl3UkJkY0djakVi?=
 =?utf-8?B?L2Y3QmRoRlRDcW5waVo0NXdxS053bHBEQnlJT1Y5T2xRVGIvdkk0OW1sYk0w?=
 =?utf-8?B?VWFKbTdnSitpNmNtRjM4NktHaElOWVU1WThMZ3g2bnY5OWRjU3g4SnZOdnph?=
 =?utf-8?B?THBQbjM3NzNwS2crU3dzVG9FRGtPWGJQVTlZa1NNM3k5U1JWeHBjbWpLdy85?=
 =?utf-8?B?QUpsUUdlS2ZMUDk0YVQ4VnJBTElWWkNUY2Q0NDZoeit6enFJb29EYmlabm11?=
 =?utf-8?B?dnFvbkZwL2IzSTFUOEpPZ0Naa0d6SnJmeSs1N0ZGb2JqbGRXT0hVbjdiS3Zo?=
 =?utf-8?B?dFF0Z3gwTkp3WlgrcENhSlI5akZ1eStzN2pTY1JrUGJqZXRHSTNJd2NQZDh1?=
 =?utf-8?B?dzN3Y2NwWnorWGNSWHViN094Zmh1TVRHTlZrSGdPOVFzR3VhYmtGdUhHWnBr?=
 =?utf-8?B?cTQ2bTQvdk5Sby9Mb0tkUXphZVdibXZHRVl0dEJQaHB3YitGYm9kNWREa3dM?=
 =?utf-8?B?N3psM05MK0RIV1FJK0hrd25Gb0JpMnI5dFZ6VmhUcmNKM2dyTWh1emNYRmk2?=
 =?utf-8?B?dGsydVcrU1JtQkxFM0FRcHQwb1NwZGhzWGxhd0N2TmgwRXZaWmlGT3pQNG9T?=
 =?utf-8?B?My9OOG9hV2FlTWorMUVTLzVvQ3JjK1lJdXNWSThGR0VQcHQzbnRvMnF3eFoy?=
 =?utf-8?B?a1QrRG9MNmcya1NhMkEzME9vMFA0OEJNQ09LT2FYZDI0eGRlYUtYUVpyVnNx?=
 =?utf-8?B?cTFMUTRLZ2xYcXduZndLNGdVMFEybTZTcmJ0dzRJM2NjaGxkZTZsbjBnMmY1?=
 =?utf-8?B?UlFQZXZURmozT0NrRzNWWTIrV1dIcWRsazhSalJGQ21UenFkS2prU3A4OXE2?=
 =?utf-8?B?R0J1WnJJa3o0cHRjV29PbFc1OE44ajdLbHMwczJtbW5EdWNob1I0YkNpOGc4?=
 =?utf-8?B?QmJRL3FxVy81QzFIcjc4eTRidTk4eEpmYm1KcDNsMkhncE1kWVkza2p1Nzl1?=
 =?utf-8?B?aGpvd0pJczhjQXVyY1VzZ2p0eVlieWFQTnVOUWxkRU1lanFudmdJTzdFa0Z0?=
 =?utf-8?B?c3lpU2tsOGpwWnpkcS9rc2RrSGxLNU42WlB6MHlQdVI4MGJseVFQOEo5YWZU?=
 =?utf-8?B?WnBCTE1WYWxwdDlYMGRhWDY3NDRFNW1NOEl4VnZRNHcvV3FlUGhxK2xaMUtM?=
 =?utf-8?B?WmlKeUFyNGlNUG9rcmd0bUpNcDZ4V0hZSGJQaHZYS0JxMU1SYmtrOEFCR2VO?=
 =?utf-8?B?a1FndWM2clRBUTJ2VnhMSVkzelJqeGVOcjZIM2hXM21CT01PTDNweGFVV1hZ?=
 =?utf-8?B?MVpXNWYrU1V0alVXMkNtVkhRc1dwWCtONlRYcWh1RksvZWEwSWJiLzNaY2NJ?=
 =?utf-8?B?YVhVTGVaQ21qNm45aEZEdEZLV2RRQlJGTnVsZ3dUc0llRkhDd21GdXNyRmhn?=
 =?utf-8?B?U01nVjdXeVQ1TDl4bHJQUVM1S29HZk5yMUtaOFZYVzJGNCt0eDZRZzJSUHhi?=
 =?utf-8?B?VUdBMEtDbHZPWElZSWx6QVo0Q3JWNkxYYktsbWdUeGVzbzhKSmtlb1Q4T2V0?=
 =?utf-8?B?MWtkWEd6L0dBbzdYNHFkUGdHd0w0TUYxT2RLaUlpUXBFbGVoa2ZPNzhQd0E2?=
 =?utf-8?B?YVQrN0ZDSTBybmNISmd0Ym8wS1ZNamRmQSttb0pqbnFLYjRYbXFDTXpwbHBI?=
 =?utf-8?B?enllWS9McXAvTjR5QkdRNk9IdDNkYU9qaXV1WEFkelN5Mmx2NXFFNE9SL29x?=
 =?utf-8?B?ZmxIRFZWbmlRMC9GZFRuazJZcDB6bzBTSjN1RTNBY21PRmh6ZjZwMVpZWmZi?=
 =?utf-8?B?bG9sM1VrU21RcVptQXZTSkdycnpqQTZLOEUrTXUzMHZHY2FDdG5hZCtzTGtz?=
 =?utf-8?B?bEJJTHM3VjBpY0ZBTUJPaExwSFJtdHVVQU5xMFpHa3R5WHJsWEVwSjBaV2pY?=
 =?utf-8?B?YW9rUGRTVnJyUkxCSWc2a2d6Yll1akFmSFRqWlFiUm1KQ2Fydk5DV1RCWFRU?=
 =?utf-8?B?SVhoTkJrYWtuQUtVVlF3MFBEZlNzUW9yYktaNU1ldmo0eGhaQ1FHTGhEWEcw?=
 =?utf-8?Q?qh64=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <C57769D79BA4074ABB5A2552915C6BC8@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d2d31197-489f-4e9b-275b-08dc336ea9fe
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Feb 2024 06:22:37.0982
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: qqb6ee6Dajw6zqXD1t6y0KIhuhIyLk/g6VI0EbY0kX/RKULgWjbmz1Cxr3zsz7bHW4RwFybQ3VpTBaudthRwNg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4958

SGkgU3Rld2FydCwNCg0KT24gMjAyNC8yLzEwIDAyOjAyLCBTdGV3YXJ0IEhpbGRlYnJhbmQgd3Jv
dGU6DQo+IE9uIDEvMTIvMjQgMDE6MTMsIEppcWlhbiBDaGVuIHdyb3RlOg0KPj4gV2hlbiBhIGRl
dmljZSBoYXMgYmVlbiByZXNldCBvbiBkb20wIHNpZGUsIHRoZSB2cGNpIG9uIFhlbg0KPj4gc2lk
ZSB3b24ndCBnZXQgbm90aWZpY2F0aW9uLCBzbyB0aGUgY2FjaGVkIHN0YXRlIGluIHZwY2kgaXMN
Cj4+IGFsbCBvdXQgb2YgZGF0ZSBjb21wYXJlIHdpdGggdGhlIHJlYWwgZGV2aWNlIHN0YXRlLg0K
Pj4gVG8gc29sdmUgdGhhdCBwcm9ibGVtLCBhZGQgYSBuZXcgaHlwZXJjYWxsIHRvIGNsZWFyIGFs
bCB2cGNpDQo+PiBkZXZpY2Ugc3RhdGUuIFdoZW4gdGhlIHN0YXRlIG9mIGRldmljZSBpcyByZXNl
dCBvbiBkb20wIHNpZGUsDQo+PiBkb20wIGNhbiBjYWxsIHRoaXMgaHlwZXJjYWxsIHRvIG5vdGlm
eSB2cGNpLg0KPj4NCj4+IENvLWRldmVsb3BlZC1ieTogSHVhbmcgUnVpIDxyYXkuaHVhbmdAYW1k
LmNvbT4NCj4+IFNpZ25lZC1vZmYtYnk6IEppcWlhbiBDaGVuIDxKaXFpYW4uQ2hlbkBhbWQuY29t
Pg0KPiANCj4gUmV2aWV3ZWQtYnk6IFN0ZXdhcnQgSGlsZGVicmFuZCA8c3Rld2FydC5oaWxkZWJy
YW5kQGFtZC5jb20+DQpUaGFua3MsIEkgd2lsbCBhZGQgaW4gbmV4dCB2ZXJzaW9uLg0KDQo+IA0K
PiBJZiB5b3Ugc2VuZCBhbm90aGVyIHZlcnNpb24sIHRoZSBSRkMgdGFnIG1heSBiZSBkcm9wcGVk
Lg0KRG9lcyBvbmx5IHRoaXMgb25lIHBhdGNoLCBvciBhbGwgcGF0Y2hlcyBvZiB0aGlzIHNlcmll
cywgbmVlZCB0byBkcm9wIFJGQyB0YWc/DQoNCj4gDQo+IE9uZSB0aGluZyB0byBrZWVwIGFuIGV5
ZSBvdXQgZm9yIGJlbG93IChub3QgcmVxdWVzdGluZyBhbnkgY2hhbmdlcykuDQpUaGFua3MgZm9y
IHJlbWluZGluZyBtZSwgSSB3aWxsIGFsd2F5cyBrZWVwIHJlYmFzaW5nIG15IGNvZGUgZnJvbSBs
YXRlc3Qgc3RhZ2luZyBicmFuY2ggYmVmb3JlIHNlbmRpbmcgbmV3IHZlcnNpb24uDQoNCj4gDQo+
PiAtLS0NCj4+IGRpZmYgLS1naXQgYS94ZW4vZHJpdmVycy92cGNpL3ZwY2kuYyBiL3hlbi9kcml2
ZXJzL3ZwY2kvdnBjaS5jDQo+PiBpbmRleCA3MmVmMjc3YzRmOGUuLmM2ZGYyYzZhOTU2MSAxMDA2
NDQNCj4+IC0tLSBhL3hlbi9kcml2ZXJzL3ZwY2kvdnBjaS5jDQo+PiArKysgYi94ZW4vZHJpdmVy
cy92cGNpL3ZwY2kuYw0KPj4gQEAgLTEwNyw2ICsxMDcsMTYgQEAgaW50IHZwY2lfYWRkX2hhbmRs
ZXJzKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQ0KPj4gIA0KPj4gICAgICByZXR1cm4gcmM7DQo+PiAg
fQ0KPj4gKw0KPj4gK2ludCB2cGNpX3Jlc2V0X2RldmljZV9zdGF0ZShzdHJ1Y3QgcGNpX2RldiAq
cGRldikNCj4+ICt7DQo+PiArICAgIEFTU0VSVChwY2lkZXZzX2xvY2tlZCgpKTsNCj4+ICsgICAg
QVNTRVJUKHJ3X2lzX3dyaXRlX2xvY2tlZCgmcGRldi0+ZG9tYWluLT5wY2lfbG9jaykpOw0KPj4g
Kw0KPj4gKyAgICB2cGNpX3JlbW92ZV9kZXZpY2UocGRldik7DQo+PiArICAgIHJldHVybiB2cGNp
X2FkZF9oYW5kbGVycyhwZGV2KTsNCj4gDQo+IE5vdGUgdGhhdCB0aGVzZSB0d28gZnVuY3Rpb25z
IG1heSBiZSByZW5hbWVkIHNvb24gYnkgdGhlIHBhdGNoIGF0IFsxXS4NCj4gV2hpY2hldmVyIHBh
dGNoIGdvZXMgaW4gbGF0ZXIgd2lsbCBuZWVkIHRvIGJlIHJlYmFzZWQgdG8gYWNjb3VudCBmb3Ig
dGhlDQo+IHJlbmFtZS4NCj4gDQo+IFsxXSBodHRwczovL2xpc3RzLnhlbnByb2plY3Qub3JnL2Fy
Y2hpdmVzL2h0bWwveGVuLWRldmVsLzIwMjQtMDIvbXNnMDAxMzQuaHRtbA0KPiANCj4+ICt9DQo+
PiArDQo+PiAgI2VuZGlmIC8qIF9fWEVOX18gKi8NCj4+ICANCj4+ICBzdGF0aWMgaW50IHZwY2lf
cmVnaXN0ZXJfY21wKGNvbnN0IHN0cnVjdCB2cGNpX3JlZ2lzdGVyICpyMSwNCj4gDQoNCi0tIA0K
QmVzdCByZWdhcmRzLA0KSmlxaWFuIENoZW4uDQo=


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 06:49:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 06:49:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684198.1063907 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd2t5-00045f-If; Thu, 22 Feb 2024 06:48:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684198.1063907; Thu, 22 Feb 2024 06:48:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd2t5-00045Y-G0; Thu, 22 Feb 2024 06:48:55 +0000
Received: by outflank-mailman (input) for mailman id 684198;
 Thu, 22 Feb 2024 06:48:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F9EN=J7=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1rd2t4-00045S-B2
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 06:48:54 +0000
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2416::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6fae7a68-d14e-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 07:48:52 +0100 (CET)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by MW6PR12MB8913.namprd12.prod.outlook.com (2603:10b6:303:247::11)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.22; Thu, 22 Feb
 2024 06:48:47 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa%5]) with mapi id 15.20.7316.018; Thu, 22 Feb 2024
 06:48:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6fae7a68-d14e-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iiqJcXYUmILCMol7m5G5CXvKI4Txm5VVPmAailYx2C/QmhehQB2p3i0pmJEWN2MfczEyzzr6Mm2Tcz0uQfaYMsCsb9rfeYnKVz6JX4PSQW1/TxbHqzCrADsAZ04EFEnHmFNcfYKCWU/8zXJYn94ICxNXxa+I+wldn/dGD8dwD56BO7Uzd9/RMRVdgOmHcS4oc6NWUyCongdJaPYAenRepYOmGtkjrPus2eNKTgo7Qg60CLH6eT4Yxsj12fqhcYDDbCH/QxLjXqa1bTyEWkcm6dEj75QYdD+TMhEISLzr0wqOh1TT836OUpZT4EtvliMb82Y9i0UYJ8/7jvrdL6ekPA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=1Nfz7P6UJHvUs6YcEs4ycdOvGyo632KuQH4oNfnI+78=;
 b=Q37wswcbCTUs7+BFvvJR2rPo4MT4e2Ji7b1ROqcnU6aTGPp0S++F0JjZwIi6XwKOYHr4YE6e9qXyN3fkKKFjMPIx4hToRfd3w/PVHcqLg/9EKZgIxrK7dREPuaZFPjANXR+KN/wHZrRPS9QnWvb7YlT8QpNC/6c/6pSRC7Sh5zZWctbTMUq+2dzDdHDE548ezzw87WOMoNHycYzEO6MSLzElxcNePKe6CRDfm4vq37KbPcZ8kLRtsQMhM7CbH5GRbI5IdhnJngiOF/d/UE0Y6q54wCuFyPEy4dHchGLFY2jkq3jG5W+DDE1J/6mIwvalegoRqathq73AUK8sfs2xvA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=1Nfz7P6UJHvUs6YcEs4ycdOvGyo632KuQH4oNfnI+78=;
 b=B6KIOOwmUHByXCbq7fNyCO6z6Wi7IK2BVa/l26eLOzngIPs/0DKssUiWwYkfSinOdoojLu4WibEBYjFzhv4DFvEsSPP9ruf8fy9llWGy+WW3KS4PzeCZTCRAjMm2NXnCd5LN1pf/WWTzCcPC5veOWIft8HCMMEU0eYzr88e360A=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Anthony PERARD <anthony.perard@cloud.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Jan
 Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Juergen Gross
	<jgross@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v5 4/5] libxl: Use gsi instead of irq for mapping
 pirq
Thread-Topic: [RFC XEN PATCH v5 4/5] libxl: Use gsi instead of irq for mapping
 pirq
Thread-Index: AQHaRR630AyI7Bl4yEKb4z8ADq0CA7EVDEGAgAGgyAA=
Date: Thu, 22 Feb 2024 06:48:47 +0000
Message-ID:
 <BL1PR12MB584975750099A63AA77CA013E7562@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-5-Jiqian.Chen@amd.com>
 <cf52f61b-1cf2-48e6-a3ba-7cdb12adacfa@perard>
In-Reply-To: <cf52f61b-1cf2-48e6-a3ba-7cdb12adacfa@perard>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL0PR12MB5539.namprd12.prod.outlook.com
 (15.20.7292.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|MW6PR12MB8913:EE_
x-ms-office365-filtering-correlation-id: 2e2a663a-8840-42ba-aefc-08dc33725221
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 lhXO0ygu3cWRMc87m8iJqskO4/I7/3gtNXr8rnueqy+urjjlw01bw2eesS4aSLhBbWUSCMC/cymH6SDvnG0iGbt2ZQg0KmluZ9yLqyw2QgPWIsWVp30Jd2GDeSPngicLwL/JN/4Hz3XpAYXkgdi9fcnojPDe1em/wGpKndZUsCRuuKqmzsIg5BSjvE+k75Iq8cIfb4WAFI70duzFKjODW0e2BLPfyDLfuf8Q9+JxcM8PpQmBX7lQGHLiEWBEzRqSGl9WY3baThsyimyy6tM1eSr5FEikQBaThHqGDGZeSXihsbc+QCHd9cNTpdjtzgK6uusNeLwiHj9qtYdK6M6vVlNkEBoaJXQoDRtcjguwsjNfz1DdE6U6pCXJaVzpoPi4Q76/vPLqt7kAL7N+CLTEeOGQFnmIEPBOBe6sPkZR+K5SuG3bMtUh3PYAR/XRpyxwhGTC3rkFzyN1pzDCs20vFE/0XOaSjncaY0Ix849AeOSzAi/PIznvf7s3m9B6OttD1OTGjOB0iR647RfR3OHvPXjCnNidqCU4b1jUBdaWj04FkzSFcmvuJ6Njf7tAU1BGtnCznM37HWREAGcd4OUwiu3jVQwpFMcKbA4UrkrYa0xSSTnN+2YalOMtfQVp4G12
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?ZUlOa1ArY0xNbDNFVEUrajFpN0JZZU1RL1NCVVVKRTNjd1ZNTWpkRXRWU0dT?=
 =?utf-8?B?U1dIb3RINzBWK0Q2dVFhK3Q0QVYwNHZ0b1d0NlRKQTBOejZCOU9idTZPZHp4?=
 =?utf-8?B?RkNhclBVRjN4WUZLMmFycFh5STN5V3dDNVpxeVBmUjFVOWtCY0J4cnExSVZM?=
 =?utf-8?B?QVRJM1I5Ty93TGs0ZTFadlhMVmJWbmdBUWxhbmNneG5UbjJaeUZZYUdmQkx3?=
 =?utf-8?B?V2t4YkpYcUtZdUtiT0FsRXpOeWtiQldoTnZNRTVRQ1pDUTVtSG8zM1hPY0ZG?=
 =?utf-8?B?NWJrTXNVZW5hbzFWbDU1TmxoTlJGa2I1ejRZbDZJMW9tdXFvakFvVGhrMHpW?=
 =?utf-8?B?S0lHU3BYN005VjRiUmp2elU0bnVtZWtzMFA1cGJIZHd6aDZabHFSb2JxK2E3?=
 =?utf-8?B?YXlxRWlQRUpicFNTbGRuUjlxbVZMNFhLY1A5SFV2UG5KcTRMVXVlQktZYVZC?=
 =?utf-8?B?OGVyTEdSRXZEL2JlM3FFaXRSOHhBWVFJd3ZFZ3BDZWxlbllCaU1tVzVjR1Bx?=
 =?utf-8?B?YVJ2T1ZxVk10b2d6VndjeTZ4dHByVVc0SUt4THpGOXMzbGM5T3JqckJlLzlS?=
 =?utf-8?B?UXBVT3d1VW91dWhhTGxxUkhGSWN1dEk0U3BrM0VudEtwaENhbWs5TWJ2dDdV?=
 =?utf-8?B?Q1Zoa0JxTjFaYnlwR0ZUOFNkK0VXeCtNa0ZuZHhRMXFkdVM2QncxMDlZem52?=
 =?utf-8?B?VHBYSktDN3dNbmJodkMyWEFIaENZUVFzbC9BeTMxOHI1dnEvVERwQ2ZhYisx?=
 =?utf-8?B?WTMwRWJCbmNvZFZ4RVhzWU8vdEtmWFBvQks3Rm5OUDZCbHBIRXM1QzY1bStm?=
 =?utf-8?B?YXUyS3Fla004RWdCL2lGdzRrVzk1eDBRcHJYODF6ckNxamxPckxtaUlmVlpZ?=
 =?utf-8?B?Y2pncTlpb2k5TWVOTWZyUzI1ZU0zQ0dCeUt1TnJLdmo0SE0xcmp0NlZIOGVw?=
 =?utf-8?B?YmJPay90WXlsWWtCUUNkZlFiVDczbnA1NVR4RnZmWW5oOXMwNk1BRDhOL3VB?=
 =?utf-8?B?QllIay9lL3RhVUF6Q1ltbDdOQUx2cGtzWTNrdVpoWmt2cDUxUXRDcmpxUXpI?=
 =?utf-8?B?SHlmRVZKQ25HL2ZwenhsbDBaZzVkYm5mNm0zSHhBa1VUdFpRVGNUYzFWOWFm?=
 =?utf-8?B?NnFPd1ErUjd3Q2tiZUpOMGJkZ2xDTVZ2WXRVZm1RZWhVZnh0dDYwb1FzdmRv?=
 =?utf-8?B?RngwY2c5YWNuUUwxUEhobW1adWlKelE3RVkwYjBUMDdxMzNVVVhVK2NWa2lk?=
 =?utf-8?B?Q3d0SzFSNWVtOXkrRDhTVGhnQmxnNGFmN05ndE81bVRmcXVVbXk5emF2NlFC?=
 =?utf-8?B?aEwwNFZtRXd4RlMzNU04a1FGTk9SdUp4WW5mSkk1U3ZEd0I4SzROWmxobEhj?=
 =?utf-8?B?UlhXdHI4aEN0ZjBpZnVvTlpadGxjVEg0OVZBYmNpelpEOHhDaDR5c0tUQ2dV?=
 =?utf-8?B?ZWJsT2NjR3lvamxSUmtpSlFwWFZEdUZ0WWhUSEI1YjVobDR3WnNsUUJjTzNz?=
 =?utf-8?B?dnIvUlozR1BGalBwWjZSWGVyeURaR1FsSUNjdTh1THVXQWVzZTRpLzU3bFNP?=
 =?utf-8?B?OTNReWQ2RUpZMzl5WEQydE42dkU0SEtyMCtNM3hlejlCaWRtNG9ZbXBJTE9B?=
 =?utf-8?B?UE9jZlBDVkV5L012cWU2TFlPT3A3SFZTVlk2cWJuam5tVFBUQTBIUkczNDJ2?=
 =?utf-8?B?c0tKaHJYbG9kSTg1NDRYT05rSkhvUm12cDFJS1NVK1RXYU9SOGt3a3cxUUo3?=
 =?utf-8?B?N1FubDNRSGhoeStTVXNpZ0ZaYXFlcXdnSEZiYnhKNjN4R201Y1VuSUUwNTRt?=
 =?utf-8?B?N0F0TWVJUHJ4RU9wYnJpTmN4OU5GVnkvdElMR1RSSEpDNmRnUnNBZ212dXBC?=
 =?utf-8?B?VFJ0UHJteUsvckIyS0NkRnF3OWYzaDN4emdScCtpbWx5c2FpME5GcDZsYjls?=
 =?utf-8?B?Z0JnQm03RlNGMVdMY3ROQUltQ1pVcDVTOHpURFdCeG1mL29sOHNtbklqc29m?=
 =?utf-8?B?SGRScVZ1eWNKV0VZR3lyQU5td0lndHJhYnBFYTlTK1hrK2d6dWM1cEtrTXZV?=
 =?utf-8?B?dTd0UHZ1YUdDeVR2T1VNWWxlanIvTkwzM0QwaXlVM2kvNUIxQXVuWS81Q05q?=
 =?utf-8?Q?MB1M=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <03CC3E8A67D46748B45C761E56D658F4@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2e2a663a-8840-42ba-aefc-08dc33725221
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Feb 2024 06:48:47.6517
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 6uGdlNRJChCO6+7ilYdirkuMt5kMZwi7OKXpA9IkPBQMFrHDHbXCWMQ8snaX6j9b4AshG1/SPnAA2XbbkwDBXg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8913

SGkgQW50aG9ueSwNCg0KT24gMjAyNC8yLzIxIDIxOjM4LCBBbnRob255IFBFUkFSRCB3cm90ZToN
Cj4gSGkgSmlxaWFuLA0KPiANCj4gT24gRnJpLCBKYW4gMTIsIDIwMjQgYXQgMDI6MTM6MTZQTSAr
MDgwMCwgSmlxaWFuIENoZW4gd3JvdGU6DQo+PiBJbiBQVkggZG9tMCwgaXQgdXNlcyB0aGUgbGlu
dXggbG9jYWwgaW50ZXJydXB0IG1lY2hhbmlzbSwNCj4+IHdoZW4gaXQgYWxsb2NzIGlycSBmb3Ig
YSBnc2ksIGl0IGlzIGR5bmFtaWMsIGFuZCBmb2xsb3cNCj4+IHRoZSBwcmluY2lwbGUgb2YgYXBw
bHlpbmcgZmlyc3QsIGRpc3RyaWJ1dGluZyBmaXJzdC4gQW5kDQo+PiB0aGUgaXJxIG51bWJlciBp
cyBhbGxvY2VkIGZyb20gc21hbGwgdG8gbGFyZ2UsIGJ1dCB0aGUNCj4+IGFwcGx5aW5nIGdzaSBu
dW1iZXIgaXMgbm90LCBtYXkgZ3NpIDM4IGNvbWVzIGJlZm9yZSBnc2kNCj4+IDI4LCB0aGF0IGNh
dXNlcyB0aGUgaXJxIG51bWJlciBpcyBub3QgZXF1YWwgd2l0aCB0aGUgZ3NpDQo+PiBudW1iZXIu
IEFuZCB3aGVuIHBhc3N0aHJvdWdoIGEgZGV2aWNlLCB4bCB3YW50cyB0byB1c2UNCj4+IGdzaSB0
byBtYXAgcGlycSwgc2VlIHBjaV9hZGRfZG1fZG9uZS0+eGNfcGh5c2Rldl9tYXBfcGlycSwNCj4+
IGJ1dCB0aGUgZ3NpIG51bWJlciBpcyBnb3QgZnJvbSBmaWxlDQo+PiAvc3lzL2J1cy9wY2kvZGV2
aWNlcy88c2JkZj4vaXJxIGluIGN1cnJlbnQgY29kZSwgc28gaXQNCj4+IHdpbGwgZmFpbCB3aGVu
IG1hcHBpbmcuDQo+Pg0KPj4gU28sIHVzZSByZWFsIGdzaSBudW1iZXIgcmVhZCBmcm9tIGdzaSBz
eXNmcy4NCj4+DQo+PiBDby1kZXZlbG9wZWQtYnk6IEh1YW5nIFJ1aSA8cmF5Lmh1YW5nQGFtZC5j
b20+DQo+IA0KPiBUaGUgIkNvLWRldmVsb3BlZC1ieSIgdGFnIGlzbid0IHJlYWxseSBkZWZpbmVk
IGluIFhlbiwgaXQncyBmaW5lIHRvIGtlZXANCj4gSSBndWVzcywgYnV0IGl0IG1lYW4gdGhhdCBh
bm90aGVyIHRhZyBpcyBtaXNzaW5nLCBJJ20gcHJldHR5IHN1cmUgeW91DQo+IG5lZWQgdG8gYWRk
ICJTaWduZWQtb2ZmLWJ5OiBIdWFuZyBSdWkgPHJheS5odWFuZ0BhbWQuY29tPiINCk9rLCB3aWxs
IGFkZCB0aGlzIGluIG5leHQgdmVyc2lvbi4NCg0KPiANCj4gQmV5b25kIHRoYXQsIHRoZSBwYXRj
aCBsb29rcyBnb29kLCBJJ3ZlIG9ubHkgY29kaW5nIHN0eWxlIGlzc3Vlcy4NClRoZXJlIGFyZSB0
d28gZW5jb2Rpbmcgc3R5bGVzIGluIHRoZSBjdXJyZW50IGZpbGUsIGFuZCBJIHdhcyBhbHNvIHN0
cnVnZ2xpbmcgd2l0aCB3aGljaCBvbmUgdG8gZm9sbG93IHdoZW4gSSB3cml0ZSBteSBjb2RlLCBp
dCBzZWVtcyB0aGF0IEkgbWFkZSB0aGUgd3JvbmcgY2hvaWNlLg0KVGhhbmsgeW91IHZlcnkgbXVj
aCBmb3IgeW91ciBzdWdnZXN0aW9ucy4gSSB3aWxsIGZpeCBhbGwgY29kaW5nIHN0eWxlIGlzc3Vl
cyB0aGF0IHlvdSBwb2ludGVkIG91dCBpbiBuZXh0IHZlcnNpb24uDQoNCj4gDQo+PiBTaWduZWQt
b2ZmLWJ5OiBKaXFpYW4gQ2hlbiA8SmlxaWFuLkNoZW5AYW1kLmNvbT4NCj4+IFJldmlld2VkLWJ5
OiBTdGVmYW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQo+PiAtLS0NCj4+
ICB0b29scy9saWJzL2xpZ2h0L2xpYnhsX3BjaS5jIHwgMjUgKysrKysrKysrKysrKysrKysrKysr
KystLQ0KPj4gIDEgZmlsZSBjaGFuZ2VkLCAyMyBpbnNlcnRpb25zKCspLCAyIGRlbGV0aW9ucygt
KQ0KPj4NCj4+IGRpZmYgLS1naXQgYS90b29scy9saWJzL2xpZ2h0L2xpYnhsX3BjaS5jIGIvdG9v
bHMvbGlicy9saWdodC9saWJ4bF9wY2kuYw0KPj4gaW5kZXggOTZjYjRkYTA3OTRlLi5hMWM2ZTgy
NjMxZTkgMTAwNjQ0DQo+PiAtLS0gYS90b29scy9saWJzL2xpZ2h0L2xpYnhsX3BjaS5jDQo+PiAr
KysgYi90b29scy9saWJzL2xpZ2h0L2xpYnhsX3BjaS5jDQo+PiBAQCAtMTQ3OCw4ICsxNDc4LDE5
IEBAIHN0YXRpYyB2b2lkIHBjaV9hZGRfZG1fZG9uZShsaWJ4bF9fZWdjICplZ2MsDQo+PiAgICAg
IGZjbG9zZShmKTsNCj4+ICAgICAgaWYgKCFwY2lfc3VwcF9sZWdhY3lfaXJxKCkpDQo+PiAgICAg
ICAgICBnb3RvIG91dF9ub19pcnE7DQo+PiAtICAgIHN5c2ZzX3BhdGggPSBHQ1NQUklOVEYoU1lT
RlNfUENJX0RFViIvIlBDSV9CREYiL2lycSIsIHBjaS0+ZG9tYWluLA0KPj4gKyAgICBzeXNmc19w
YXRoID0gR0NTUFJJTlRGKFNZU0ZTX1BDSV9ERVYiLyJQQ0lfQkRGIi9nc2kiLCBwY2ktPmRvbWFp
biwNCj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBjaS0+YnVzLCBwY2ktPmRl
diwgcGNpLT5mdW5jKTsNCj4+ICsNCj4+ICsgICAgaWYgKCBhY2Nlc3Moc3lzZnNfcGF0aCwgRl9P
SykgIT0gMCApIHsNCj4gDQo+IFNvLCB0aGUgY29kaW5nIHN0eWxlIGluIGxpYnhsIGlzIGEgYml0
IGRpZmZlcmVudCwgZmlyc3QgY291bGQgeW91IHN0b3JlDQo+IHRoZSByZXR1cm4gdmFsdWUgb2Yg
YWNjZXNzKCkgaW4gJ3InLCB0aGVuIGNoZWNrIHRoYXQgdmFsdWU/IFRoZW4NCj4gImlmIChyKSIg
d2lsbCBiZSBlbm91Z2gsIG5vIG5lZWQgZm9yICIhPSAwIi4NCj4gDQo+IFNlY29uZCwgdGhlcmUn
cyBubyBhcm91bmQgc3BhY2UgdGhlIGNvbmRpdGlvbiBpbiB0aGUgaWYgc3RhdGVtZW50LCBzbw0K
PiBqdXN0ICJpZiAocikiLg0KPiANCj4+ICsgICAgICAgIGlmICggZXJybm8gPT0gRU5PRU5UICkN
Cj4+ICsgICAgICAgICAgICBzeXNmc19wYXRoID0gR0NTUFJJTlRGKFNZU0ZTX1BDSV9ERVYiLyJQ
Q0lfQkRGIi9pcnEiLCBwY2ktPmRvbWFpbiwNCj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHBjaS0+YnVzLCBwY2ktPmRldiwgcGNpLT5mdW5jKTsNCj4gDQo+IEhhcyB0aGUgZWxz
ZSBwYXJ0IG9mIHRoaXMgaWYuLmVsc2UgaXMgaW4gYSB7fS1ibG9jaywgY291bGQgeW91IGFkZCB7
IH0NCj4gaGVyZSwgZm9yIHRoZSBpZiBwYXJ0PyBUbyBxdW90ZSB0aGUgQ09ESU5HX1NUWUxFIGxp
YnhsIGRvY3VtZW50OiAiVG8NCj4gYXZvaWQgY29uZnVzaW9uLCBlaXRoZXIgYWxsIHRoZSBibG9j
a3MgaW4gYW4gaWYuLi5lbHNlIGNoYWluIGhhdmUNCj4gYnJhY2VzLCBvciBub25lIG9mIHRoZW0g
ZG8uDQo+IA0KPj4gKyAgICAgICAgZWxzZSB7DQo+PiArICAgICAgICAgICAgTE9HRUQoRVJST1Is
IGRvbWFpbmlkLCAiQ2FuJ3QgYWNjZXNzICVzIiwgc3lzZnNfcGF0aCk7DQo+IA0KPiBUaGlzIGVy
cm9yIG1lc3NhZ2UgaXMga2luZCBvZiByZWR1bmRhbnQsIHdlIGNvdWxkIGNvdWxkIHNpbXBseSBs
ZXQgdGhlIGNvZGUNCj4gdHJ5IGZvcGVuKCkgb24gdGhpcyAiL2dzaSIgcGF0aCwgZXZlbiBpZiB3
ZSBrbm93IHRoYXQgaXQncyBub3QgZ29pbmcgdG8NCj4gd29yaywgYXMgdGhlIGVycm9yIGNoZWNr
IG9uIGZvcGVuKCkgd2lsbCBwcm9kdWNlIHRoZSBzYW1lIGVycm9yIG1lc3NhZ2UuDQo+IDstKQ0K
PiANCj4gQW5kIHdpdGhvdXQgdGhhdCBlbHNlIHBhcnQsIHRoZSBjb2RlIGNvdWxkIGJlIHNpbXBs
aWZpZWQgdG86DQo+IA0KPiAgICAgLyogSWYgL2dzaSBwYXRoIGRvZXNuJ3QgZXhpc3QsIGZhbGxi
YWNrIHRvIC9pcnEgKi8NCj4gICAgIGlmIChyICYmIGVycm5vID09IEVOT0VOVCkNCj4gICAgICAg
ICBzeXNmc19wYXRoID0gIi4uLi4vaXJxIjsNCj4gDQo+IA0KPiANCj4gQW5kIHRob3NlIGNvbW1l
bnRzIGFsc28gYXBwbHkgdG8gdGhlIGNoYW5nZXMgaW4gcGNpX3JlbW92ZV9kZXRhY2hlZCgpLg0K
PiANCj4gVGhhbmtzLA0KPiANCg0KLS0gDQpCZXN0IHJlZ2FyZHMsDQpKaXFpYW4gQ2hlbi4NCg==


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 06:49:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 06:49:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684199.1063917 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd2tD-0004Lg-P6; Thu, 22 Feb 2024 06:49:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684199.1063917; Thu, 22 Feb 2024 06:49:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd2tD-0004LZ-MI; Thu, 22 Feb 2024 06:49:03 +0000
Received: by outflank-mailman (input) for mailman id 684199;
 Thu, 22 Feb 2024 06:49:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd2tB-0004Kq-Uj; Thu, 22 Feb 2024 06:49:01 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd2tB-0007xk-RV; Thu, 22 Feb 2024 06:49:01 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd2tB-0004kr-Hz; Thu, 22 Feb 2024 06:49:01 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rd2tB-0004H3-HV; Thu, 22 Feb 2024 06:49:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=I+riVAc5USmfVZ7q50rWBRR2nBYFt194y1/s+duPGCY=; b=gK76TRAswnYH900pkY3uALtw6o
	D05a/9xzlyDBtDLy+jZkggqb+vzlyMXYuqTBRTwtLKQHCbgXc2TJE33L3E9I9EzKqh0iS+BZ1967a
	R0/oxoUwXB9YEVhzef97W5ZiDdMGAJQabQh0HrGeEtaUUftHuRPAo05bBaTdJ0WoW+cA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184724-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184724: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=8ccd63d14da5678a4b95df0aa954a2378355af9b
X-Osstest-Versions-That:
    ovmf=2ca8d559744319025592df10ada0f714fc3b8e15
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 22 Feb 2024 06:49:01 +0000

flight 184724 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184724/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 8ccd63d14da5678a4b95df0aa954a2378355af9b
baseline version:
 ovmf                 2ca8d559744319025592df10ada0f714fc3b8e15

Last test of basis   184717  2024-02-21 01:43:32 Z    1 days
Testing same since   184724  2024-02-22 03:43:20 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Zhiguang Liu <zhiguang.liu@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   2ca8d55974..8ccd63d14d  8ccd63d14da5678a4b95df0aa954a2378355af9b -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 07:23:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 07:23:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684215.1063926 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd3Pw-00016C-Ec; Thu, 22 Feb 2024 07:22:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684215.1063926; Thu, 22 Feb 2024 07:22:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd3Pw-000165-Bv; Thu, 22 Feb 2024 07:22:52 +0000
Received: by outflank-mailman (input) for mailman id 684215;
 Thu, 22 Feb 2024 07:22:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=F9EN=J7=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1rd3Pv-00015z-2w
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 07:22:51 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2415::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2e550cfc-d153-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 08:22:49 +0100 (CET)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by DM4PR12MB6373.namprd12.prod.outlook.com (2603:10b6:8:a4::7) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7316.22; Thu, 22 Feb 2024 07:22:46 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa%5]) with mapi id 15.20.7316.018; Thu, 22 Feb 2024
 07:22:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e550cfc-d153-11ee-8a55-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UoJ5NQLRYoR1WXSWOH/LQoy/hWfuGFYKx1hr8Xo5d0G6s65x/WpG5HSYJWMI81wdn87qFlJbaMjUFRIo/JweO/jvp1v7rYFlWlLEQwBto/DIpjKj5R2gny1zpSOF+PXGV1NJaaF9g9sDtg36gzWbDUks1jds0xTnVHxSYIS7oBd3Nx9AR8gPz8qld15keeZ0SUPnDgsDQXrc4eUKeIrnvLDGbSz3qNpG7yS27HwQ+HF1/cED7Gc1JOo8gmvJ3q5MDOW+2ujPEV1SKh8h1ogga+0mmyzJ8TllVdmc8Yd8IXBqzPon8JnK7i3XAimfa0/CugSjmpN2G19y7Z6gw7qnRQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=7/loTcbC9TVRCF249PAs5O2zhfdT+KpcAMI8mcyq5pU=;
 b=jbnimwtK1SPxcTJizLTOTCZ1QYjnG2L+2AK3CAOrQ4tBCtsWdfOFrXfsGdTPAazmZCa9FoVrWWZAN6ibWAc80Hn687DqI3fUtJvwcbT5PJW//lmej4bf+/bXiWLBmj2klwSmmCxhus/fwtw5UVADAUfQu6JSv90aldEk9GdywC6ilYLhS0RxQ9NXL3DhIqMFuRWipRZdMOY1gZ+iCD73PXOY0cqkOGNOtPIjhKP/5lDl2P/YjstctrzGyZffUKx0c8Trs4uQaD4rk0gXCTR3SZiTxjAEQjuA8kOuCAYgBCHIaEqhtMpFqM7fPdB3VC8/eCLQ+BU/wQ6VTyDS3R4ZvQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7/loTcbC9TVRCF249PAs5O2zhfdT+KpcAMI8mcyq5pU=;
 b=rzniROR4/lGylXkgxBSEqN1aMl9XRf/R3twjL0cIR0qbJZkbNrjBxZS2YP/RiMxWGv8otB69HOHdSbQ8VJdmzIxG9ttsgcMgzl5lfKj9ZoMpVUQ91ZFoBIN6LPXQqbNMBCMwN+D1HJQsGnw2etVMkGcLaDeLRoWQwpnM4r+pmhM=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Anthony PERARD <anthony.perard@cloud.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Jan
 Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Juergen Gross
	<jgross@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v5 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Topic: [RFC XEN PATCH v5 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Index: AQHaRR65Xby346gk7ESzHNZtxrKssbEVI+YAgAGO+4A=
Date: Thu, 22 Feb 2024 07:22:45 +0000
Message-ID:
 <BL1PR12MB5849F3787ED9245E6986872CE7562@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-6-Jiqian.Chen@amd.com>
 <09290ba4-6915-4f0d-8e16-3e14713d02ba@perard>
In-Reply-To: <09290ba4-6915-4f0d-8e16-3e14713d02ba@perard>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: BL0PR12MB5539.namprd12.prod.outlook.com
 (15.20.7292.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|DM4PR12MB6373:EE_
x-ms-office365-filtering-correlation-id: 1e472d82-4344-4afa-b31d-08dc33771104
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 ixJbzSjFVPiz+7iS6eOCoL3ZxqEvHEUUFnAU5xLwxBNxZRF+tzurGH7c8IP+gda05BEtcRFDGD8//wVbntEATbUQ8GHP6mvVeGYobwKIMa3BwYpaXskegsgYq24ifgGyOThqDuyxyRm+nRGrDc2C0l7cdX0Hg9iIULx5f39hWJ22OVhFvCQ5Jxl0M9gQd5zCcAqo4e9cO0XwjAcZvNRHXRwauAGKMqZtUbSd/WaWQJOoh6egl/J3yt9q/ZJOTPlrRyAk1l0jfU26OpyIHwKJu3XoKeDxsKPdDouosck56MQnh7b4p+lOP+iw1vr+F/bA/hPapw7KiC2n59fQRz47A05T2bRpHgwSm30xbTVcuy3g6drMUdT1jmpZQQaCTkoWIZrjPYMT/SKWVgYqvoRR+RHbHPLAySyOddDswDIoV1/M4vKiBoCFPVTUN0pR75A7OLmqbaUCs8s5+hul+z1+fa+JVgTX7d00S4BVqnXMAjaUPj79+CNJ0Vg4LrK4YR6G8QpXPT3iTZRAuxYKoocjo5JTRkBHXm9rr5o8r4TexCMAip439H+25pLf277B95t9/j4lnKmeniVaW1G/SAn07AF90fCwxZG5IojDPF8PjG0=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?QjNldFlmWnl3UUZIQnZVc2M3MUhVckMrbkxXUlUweUVHS04yblJzakxCaC9j?=
 =?utf-8?B?QWFrMDNXc0x6dFhMcEtIeHQ4ZFlhV25haVI3UGV3RVlNZnl1UHJEUkNFb0U2?=
 =?utf-8?B?YUJkbE1PVjYzbDlUYVNkMFplcCtnaWg5R0tOMmxqdXFhbFlCSVQyLytZQTgx?=
 =?utf-8?B?d2hXZE82RDA5RTg4UEVuUHM5ZHVxMmFza1Vsb3g4Q09BZzMya2NhcDV3R2Rs?=
 =?utf-8?B?YWVHV1R1VVBCeXJ4QWUyZHFTcmZzUDBaaVpMVUMrVitHbVRVNlZuV2FYTDFi?=
 =?utf-8?B?SFJJQlNuQ3VFdzgvVWR1ZHRnb3JZU1NrSlM2YmMzTjYvb2tKTHJyWlQxTzZZ?=
 =?utf-8?B?RU1NMitzWlV3VEloWC9KUmQ0Qm5wZUJMbzYyNXdwb0VCUGJuSlFldVRRL1JO?=
 =?utf-8?B?MENMelBUaC9LRU9YWjlXbXN6V1pvMW9GdGNLbk5sSU9FOUdwcTc3ckpHWTBM?=
 =?utf-8?B?SFpPSmFzUlJiaGVoWEZuSVVMTFZ4SUE2dU53ckV3dDBxbGxDRVliTndnUnAx?=
 =?utf-8?B?NDE5MDBLN0VEQXRHck93QUtWeHJGOWNOZXBnU3BqdDhwa0JMbHo1cFptTjZq?=
 =?utf-8?B?azZ3RXlnZzMzUEt6MFNuQjB1U3ZuaTFjUWxzUDRNQlIxSWlZNnNiSWJBb1Jz?=
 =?utf-8?B?aVU3UVF3Y090T3cyb0JuMElzck93cm8zcjRoWnlQeW92c3E2aXBrc0IwTU1r?=
 =?utf-8?B?dTVFTVFKOWF4VTEvQzNPWTZlQXd4aVFDRmxScTFDKzVLKzVwQUJtYmJDd3Av?=
 =?utf-8?B?bEpDUkprZEJ2cjVUbmJ0eGdiNWx3R3RhbXE0Zi8rYzBrUSszc2tGSTlwb2Fv?=
 =?utf-8?B?QVU2ZTI4ZDBlRC9PUXlreXRpejVoV05uKzlkRDk0eDNHcjRVQ2hPV21EdGpm?=
 =?utf-8?B?dnlzek5JWkdJbEhnZ3F3b2VDdENXU2Q2SzVzUHFHM2lRRjR3Q2YvTVVLMndW?=
 =?utf-8?B?Z2NycEM1TFlkb1JvOG1PbGZDeHc0bUIwSTY0dzlmL1dKd1JDT0RlQUZ5U040?=
 =?utf-8?B?eFMzdE5IRWRhTGRyeEVqbmNUS2VkcnFadnd6V3RJVmVodFdHdkp0ZFEvcWg5?=
 =?utf-8?B?dUpXK0pVMk1RVjdwaU5jZW5YTVhHUmU4UG01VmdCWGRJRFEwSm1tWksyU1JM?=
 =?utf-8?B?am9jdU95R3JubmdoZlZrdVVzUG9lQWZWWHpLL1hwUStvM2s0SkpvOVVZMC9Y?=
 =?utf-8?B?dTJWcXhHNjlZUVZBaFRkNnV5Q2V2eVJPQUR0ZFNreDluWnMyMXBpczIxQU54?=
 =?utf-8?B?b2tFWGNCS1psQlZLRmNsSXVVaEpobUt5cFJMdU1ZWjM0QXYwUnk0T0s5Qk13?=
 =?utf-8?B?aVdwMGpPaGlwYm1ScEpxUGg4bysrMzIvTFFrN3pmWDBLREVzMWt1UzlFNy9a?=
 =?utf-8?B?bjBzc0I1L09GWVpDb0JhQ3BvSS9ucjZVY0FxdzZLc09PTnpOSk1DS0FOMW5s?=
 =?utf-8?B?SHJWUTZWakp2OVljNmNSMnQ0UklOY2diUEFsRm5YUm9tSy9qRU9sdHV6TG1y?=
 =?utf-8?B?M2dGaXNMQUpEZWpDeXdZcXB6SUVRVHRrMkdoN0N6WnhMQ0NDTjVCa1JINDNI?=
 =?utf-8?B?bFdUNnh5VmVBNngwcFhCcnAra2EyaklvQzNycVZxb25LRU15L2hZb1NaU2pj?=
 =?utf-8?B?SHN2MmJzVCtLdHUvdkxWL1pELzRwTGZ2L0NCZVJUV2wyZW1aVUpaU09aYW1H?=
 =?utf-8?B?aFF6LzFSWS83UkkwWHdIcFRzcVRBb1BmTDNYZEJxWVZ5dDdGamp1UzBzN2Rx?=
 =?utf-8?B?bjh3ZU01bFhCUFdvSCtTUzk2eldETTJqSmExVWhxdll4RkZiU29kaksxTnFM?=
 =?utf-8?B?OGNkQ3EvMTRGYXZ2MjVoekdTYjBGUDBNTkc3OVhUSFVEdTVWWW9vSHJNeDNs?=
 =?utf-8?B?K2kzNWVaRGlmNWlTQ1NncHdjdEtZc3UvR0UwbmF3SjZFUlVBM0ptN0ltdXA4?=
 =?utf-8?B?MHdrNmpkL3VtT0pMQVlqc29mSUJ4aFU4dFhqTHh3eGZ5UzN5RHhuRnV3RlFR?=
 =?utf-8?B?YVo2bS9uYXBsV0dXaTVsdG5nMW14Ynhnbm9CV0d6cGJhYW90eXIrNEpLc25l?=
 =?utf-8?B?elZ1aFZYMWhia1p0N3FpN21rRTVod1NZRzZoMDZkVGQxMFpaazJpSitSdE4v?=
 =?utf-8?Q?FTNc=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <B35BB279D943BD4AA77ACF9A9DF40B06@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e472d82-4344-4afa-b31d-08dc33771104
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Feb 2024 07:22:45.9168
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 64OAAxnGndNUZnVg/fOX5vRP/GbhN8FuGEXvK0f+Ap4/bzsAmq6LOuZXe7OA6vRPBktZE6TU/woPH1oe0MS5+A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6373

SGkgQW50aG9ueSwNCg0KT24gMjAyNC8yLzIxIDIzOjAzLCBBbnRob255IFBFUkFSRCB3cm90ZToN
Cj4gT24gRnJpLCBKYW4gMTIsIDIwMjQgYXQgMDI6MTM6MTdQTSArMDgwMCwgSmlxaWFuIENoZW4g
d3JvdGU6DQo+PiBkaWZmIC0tZ2l0IGEvdG9vbHMvbGlicy9jdHJsL3hjX2RvbWFpbi5jIGIvdG9v
bHMvbGlicy9jdHJsL3hjX2RvbWFpbi5jDQo+PiBpbmRleCBmMmQ5ZDE0YjRkOWYuLjQ0OGJhMmM1
OWFlMSAxMDA2NDQNCj4+IC0tLSBhL3Rvb2xzL2xpYnMvY3RybC94Y19kb21haW4uYw0KPj4gKysr
IGIvdG9vbHMvbGlicy9jdHJsL3hjX2RvbWFpbi5jDQo+PiBAQCAtMTM5NCw2ICsxMzk0LDIxIEBA
IGludCB4Y19kb21haW5faXJxX3Blcm1pc3Npb24oeGNfaW50ZXJmYWNlICp4Y2gsDQo+PiAgICAg
IHJldHVybiBkb19kb21jdGwoeGNoLCAmZG9tY3RsKTsNCj4+ICB9DQo+PiAgDQo+PiAraW50IHhj
X2RvbWFpbl9nc2lfcGVybWlzc2lvbih4Y19pbnRlcmZhY2UgKnhjaCwNCj4+ICsgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIHVpbnQzMl90IGRvbWlkLA0KPj4gKyAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgdWludDMyX3QgZ3NpLA0KPj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgYm9vbCBhbGxvd19hY2Nlc3MpDQo+PiArew0KPj4gKyAgICBzdHJ1Y3QgeGVuX2RvbWN0bCBk
b21jdGwgPSB7fTsNCj4+ICsNCj4+ICsgICAgZG9tY3RsLmNtZCA9IFhFTl9ET01DVExfZ3NpX3Bl
cm1pc3Npb247DQo+PiArICAgIGRvbWN0bC5kb21haW4gPSBkb21pZDsNCj4+ICsgICAgZG9tY3Rs
LnUuZ3NpX3Blcm1pc3Npb24uZ3NpID0gZ3NpOw0KPj4gKyAgICBkb21jdGwudS5nc2lfcGVybWlz
c2lvbi5hbGxvd19hY2Nlc3MgPSBhbGxvd19hY2Nlc3M7DQo+IA0KPiBUaGlzIGNvdWxkIGJlIHdy
aXR0ZW4gYXM6DQo+ICAgICBzdHJ1Y3QgeGVuX2RvbWN0bCBkb21jdGwgPSB7DQo+ICAgICAgICAg
LmNtZCA9IFhFTl9ET01DVExfZ3NpX3Blcm1pc3Npb24sDQo+ICAgICAgICAgLmRvbWFpbiA9IGRv
bWlkLA0KPiAgICAgICAgIC51LmdzaV9wZXJtaXNzaW9uLmdzaSA9IGdzaSwNCj4gICAgICAgICAu
dS5nc2lfcGVybWlzc2lvbi5hbGxvd19hY2Nlc3MgPSBhbGxvd19hY2Nlc3MsDQo+ICAgICB9Ow0K
PiANClRoYXQncyBmaW5lLCBJIHdpbGwgY2hhbmdlIHRvIHRoaXMgaW4gbmV4dCB2ZXJzaW9uLg0K
DQo+IGJ1dCB5b3VyIGNoYW5nZSBpcyBmaW5lIHRvby4NCj4gDQo+IA0KPj4gKw0KPj4gKyAgICBy
ZXR1cm4gZG9fZG9tY3RsKHhjaCwgJmRvbWN0bCk7DQo+PiArfQ0KPj4gKw0KPj4gIGludCB4Y19k
b21haW5faW9tZW1fcGVybWlzc2lvbih4Y19pbnRlcmZhY2UgKnhjaCwNCj4+ICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgdWludDMyX3QgZG9taWQsDQo+PiAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIHVuc2lnbmVkIGxvbmcgZmlyc3RfbWZuLA0KPj4gZGlmZiAtLWdpdCBh
L3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfcGNpLmMgYi90b29scy9saWJzL2xpZ2h0L2xpYnhsX3Bj
aS5jDQo+PiBpbmRleCBhMWM2ZTgyNjMxZTkuLjQxMzZhODYwYTA0OCAxMDA2NDQNCj4+IC0tLSBh
L3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfcGNpLmMNCj4+ICsrKyBiL3Rvb2xzL2xpYnMvbGlnaHQv
bGlieGxfcGNpLmMNCj4+IEBAIC0xNDIxLDYgKzE0MjEsOCBAQCBzdGF0aWMgdm9pZCBwY2lfYWRk
X2RtX2RvbmUobGlieGxfX2VnYyAqZWdjLA0KPj4gICAgICB1aW50MzJfdCBmbGFnID0gWEVOX0RP
TUNUTF9ERVZfUkRNX1JFTEFYRUQ7DQo+PiAgICAgIHVpbnQzMl90IGRvbWFpbmlkID0gZG9taWQ7
DQo+PiAgICAgIGJvb2wgaXNzdHViZG9tID0gbGlieGxfaXNfc3R1YmRvbShjdHgsIGRvbWlkLCAm
ZG9tYWluaWQpOw0KPj4gKyAgICBpbnQgZ3NpOw0KPj4gKyAgICBib29sIGhhc19nc2kgPSB0cnVl
Ow0KPiANCj4gV2h5IGlzIHRoaXMgZnVuY3Rpb24gaGFzICJnc2kiIHZhcmlhYmxlLCBhbmQgImdz
aSA9IGlycSIgYnV0IHRoZSBuZXh0DQo+IGZ1bmN0aW9uIHBjaV9yZW1vdmVfZGV0YWNoZWQoKSBk
b2VzIG9ubHkgaGF2ZSAiaXJxIiB2YXJpYWJsZT8NCkJlY2F1c2UgaW4gcGNpX2FkZF9kbV9kb25l
LCBpdCBjYWxscyAiIHIgPSB4Y19waHlzZGV2X21hcF9waXJxKGN0eC0+eGNoLCBkb21pZCwgaXJx
LCAmaXJxKTsgIiwgaXQgcGFzc2VzIHRoZSBwb2ludGVyIG9mIGlycSBpbiwgYW5kIHRoZW4gaXJx
IHdpbGwgYmUgY2hhbmdlZCwgc28gSSBuZWVkIHRvIHVzZSBnc2kgdG8gc2F2ZSB0aGUgb3JpZ2lu
IHZhbHVlLg0KDQo+IA0KPiBTbywgZ3NpIGNhbiBvbmx5IGJlIHBvc2l0aXZlIGludGVnZXIsIHJp
Z2h0PyBDb3VsZCB3ZSBmb3JnbyBgaGFzX2dzaWAgYW5kDQo+IGp1c3Qgc2V0ICJnc2kgPSAtMSIg
d2hlbiB0aGVyZSdzIG5vIGdzaT8NCk5vLCB3ZSBjYW4ndCBmb3JnbyAnaGFzX2dzaScsIGJlY2F1
c2Ugd2UgbmVlZCB0byBzZXQgZ3NpID0gaXJxIHRvIHNhdmUgdGhlIG9yaWdpbmFsIHZhbC4NCg0K
PiANCj4+ICAgICAgLyogQ29udmVuaWVuY2UgYWxpYXNlcyAqLw0KPj4gICAgICBib29sIHN0YXJ0
aW5nID0gcGFzLT5zdGFydGluZzsNCj4+IEBAIC0xNDgyLDYgKzE0ODQsNyBAQCBzdGF0aWMgdm9p
ZCBwY2lfYWRkX2RtX2RvbmUobGlieGxfX2VnYyAqZWdjLA0KPj4gICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgcGNpLT5idXMsIHBjaS0+ZGV2LCBwY2ktPmZ1bmMpOw0KPj4gIA0KPj4g
ICAgICBpZiAoIGFjY2VzcyhzeXNmc19wYXRoLCBGX09LKSAhPSAwICkgew0KPj4gKyAgICAgICAg
aGFzX2dzaSA9IGZhbHNlOw0KPj4gICAgICAgICAgaWYgKCBlcnJubyA9PSBFTk9FTlQgKQ0KPj4g
ICAgICAgICAgICAgIHN5c2ZzX3BhdGggPSBHQ1NQUklOVEYoU1lTRlNfUENJX0RFViIvIlBDSV9C
REYiL2lycSIsIHBjaS0+ZG9tYWluLA0KPj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgcGNpLT5idXMsIHBjaS0+ZGV2LCBwY2ktPmZ1bmMpOw0KPj4gQEAgLTE0OTcsNiArMTUwMCw3
IEBAIHN0YXRpYyB2b2lkIHBjaV9hZGRfZG1fZG9uZShsaWJ4bF9fZWdjICplZ2MsDQo+PiAgICAg
ICAgICBnb3RvIG91dF9ub19pcnE7DQo+PiAgICAgIH0NCj4+ICAgICAgaWYgKChmc2NhbmYoZiwg
IiV1IiwgJmlycSkgPT0gMSkgJiYgaXJxKSB7DQo+PiArICAgICAgICBnc2kgPSBpcnE7DQo+IA0K
PiBXaHkgZG8geW91IGRvIHRoaXMsIHRoYXQgdGhhdCBtZWFuIHRoYXQgYGdzaWAgYW5kIGBpcnFg
IGFyZSB0aGUgc2FtZT8gT3INCj4gZG9lcyBgaXJxYCBoYXBwZW4gdG8gc29tZXRpbWUgY29udGFp
biBhIGBnc2lgIHZhbHVlPw0KQWJvdmUgcmVhc29uLg0KDQo+IA0KPj4gICAgICAgICAgciA9IHhj
X3BoeXNkZXZfbWFwX3BpcnEoY3R4LT54Y2gsIGRvbWlkLCBpcnEsICZpcnEpOw0KPj4gICAgICAg
ICAgaWYgKHIgPCAwKSB7DQo+PiAgICAgICAgICAgICAgTE9HRUQoRVJST1IsIGRvbWFpbmlkLCAi
eGNfcGh5c2Rldl9tYXBfcGlycSBpcnE9JWQgKGVycm9yPSVkKSIsDQo+PiBAQCAtMTUwNSw3ICsx
NTA5LDEwIEBAIHN0YXRpYyB2b2lkIHBjaV9hZGRfZG1fZG9uZShsaWJ4bF9fZWdjICplZ2MsDQo+
PiAgICAgICAgICAgICAgcmMgPSBFUlJPUl9GQUlMOw0KPj4gICAgICAgICAgICAgIGdvdG8gb3V0
Ow0KPj4gICAgICAgICAgfQ0KPj4gLSAgICAgICAgciA9IHhjX2RvbWFpbl9pcnFfcGVybWlzc2lv
bihjdHgtPnhjaCwgZG9taWQsIGlycSwgMSk7DQo+PiArICAgICAgICBpZiAoIGhhc19nc2kgKQ0K
Pj4gKyAgICAgICAgICAgIHIgPSB4Y19kb21haW5fZ3NpX3Blcm1pc3Npb24oY3R4LT54Y2gsIGRv
bWlkLCBnc2ksIDEpOw0KPj4gKyAgICAgICAgaWYgKCAhaGFzX2dzaSB8fCByID09IC1FT1BOT1RT
VVBQICkNCj4+ICsgICAgICAgICAgICByID0geGNfZG9tYWluX2lycV9wZXJtaXNzaW9uKGN0eC0+
eGNoLCBkb21pZCwgaXJxLCAxKTsNCj4+ICAgICAgICAgIGlmIChyIDwgMCkgew0KPj4gICAgICAg
ICAgICAgIExPR0VEKEVSUk9SLCBkb21haW5pZCwNCj4+ICAgICAgICAgICAgICAgICAgICAieGNf
ZG9tYWluX2lycV9wZXJtaXNzaW9uIGlycT0lZCAoZXJyb3I9JWQpIiwgaXJxLCByKTsNCj4gDQo+
IExvb2tzIGxpa2UgdGhpcyBlcnJvciBtZXNzYWdlIGNvdWxkIGJlIHdyb25nLCBJIHRoaW5rIHdl
IHdhbnQgdG8ga25vdyBpZg0KPiB3aGljaCBjYWxsIGJldHdlZW4geGNfZG9tYWluX2lycV9wZXJt
aXNzaW9uKCkgYW5kDQo+IHhjX2RvbWFpbl9nc2lfcGVybWlzc2lvbigpIGhhcyBmYWlsZWQuDQpZ
b3UgYXJlIHJpZ2h0LCBJIHdpbGwgc2VwYXJhdGUgdGhlbSBpbiBuZXh0IHZlcnNpb24uDQoNCj4g
DQo+PiBAQCAtMjE4NSw2ICsyMTkyLDcgQEAgc3RhdGljIHZvaWQgcGNpX3JlbW92ZV9kZXRhY2hl
ZChsaWJ4bF9fZWdjICplZ2MsDQo+PiAgICAgIEZJTEUgKmY7DQo+PiAgICAgIHVpbnQzMl90IGRv
bWFpbmlkID0gcHJzLT5kb21pZDsNCj4+ICAgICAgYm9vbCBpc3N0dWJkb207DQo+PiArICAgIGJv
b2wgaGFzX2dzaSA9IHRydWU7DQo+PiAgDQo+PiAgICAgIC8qIENvbnZlbmllbmNlIGFsaWFzZXMg
Ki8NCj4+ICAgICAgbGlieGxfZGV2aWNlX3BjaSAqY29uc3QgcGNpID0gJnBycy0+cGNpOw0KPj4g
QEAgLTIyNDQsNiArMjI1Miw3IEBAIHNraXBfYmFyOg0KPj4gICAgICAgICAgICAgICAgICAgICAg
ICAgICAgIHBjaS0+YnVzLCBwY2ktPmRldiwgcGNpLT5mdW5jKTsNCj4+ICANCj4+ICAgICAgaWYg
KCBhY2Nlc3Moc3lzZnNfcGF0aCwgRl9PSykgIT0gMCApIHsNCj4+ICsgICAgICAgIGhhc19nc2kg
PSBmYWxzZTsNCj4+ICAgICAgICAgIGlmICggZXJybm8gPT0gRU5PRU5UICkNCj4+ICAgICAgICAg
ICAgICBzeXNmc19wYXRoID0gR0NTUFJJTlRGKFNZU0ZTX1BDSV9ERVYiLyJQQ0lfQkRGIi9pcnEi
LCBwY2ktPmRvbWFpbiwNCj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBjaS0+
YnVzLCBwY2ktPmRldiwgcGNpLT5mdW5jKTsNCj4+IEBAIC0yMjcwLDcgKzIyNzksMTAgQEAgc2tp
cF9iYXI6DQo+PiAgICAgICAgICAgICAgICovDQo+PiAgICAgICAgICAgICAgTE9HRUQoRVJST1Is
IGRvbWlkLCAieGNfcGh5c2Rldl91bm1hcF9waXJxIGlycT0lZCIsIGlycSk7DQo+PiAgICAgICAg
ICB9DQo+PiAtICAgICAgICByYyA9IHhjX2RvbWFpbl9pcnFfcGVybWlzc2lvbihjdHgtPnhjaCwg
ZG9taWQsIGlycSwgMCk7DQo+PiArICAgICAgICBpZiAoIGhhc19nc2kgKQ0KPj4gKyAgICAgICAg
ICAgIHJjID0geGNfZG9tYWluX2dzaV9wZXJtaXNzaW9uKGN0eC0+eGNoLCBkb21pZCwgaXJxLCAw
KTsNCj4gDQo+IFdoeSBkbyB5b3UgdXNlIHRoZSB4Y19kb21haW5fZ3NpX3Blcm1pc3Npb24oKSB3
aXRoICJpcnEiIGhlcmUsIGJ1dCBub3QNCj4gaW4gcGNpX2FkZF9kbV9kb25lKCkgPw0KQmVjYXVz
ZSB4Y19waHlzZGV2X3VubWFwX3BpcnEgZG9lc24ndCBjaGFuZ2UgdGhlIHZhbHVlIG9mIGlycSwg
YnV0IGluIHBjaV9hZGRfZG1fZG9uZSwgdGhlIHZhbHVlIG9mIGlycSBpcyBjaGFuZ2VkIGJ5IHhj
X3BoeXNkZXZfbWFwX3BpcnEuDQoNCj4gDQo+PiArICAgICAgICBpZiAoICFoYXNfZ3NpIHx8IHJj
ID09IC1FT1BOT1RTVVBQICkNCj4+ICsgICAgICAgICAgICByYyA9IHhjX2RvbWFpbl9pcnFfcGVy
bWlzc2lvbihjdHgtPnhjaCwgZG9taWQsIGlycSwgMCk7DQo+PiAgICAgICAgICBpZiAocmMgPCAw
KSB7DQo+PiAgICAgICAgICAgICAgTE9HRUQoRVJST1IsIGRvbWlkLCAieGNfZG9tYWluX2lycV9w
ZXJtaXNzaW9uIGlycT0lZCIsIGlycSk7DQo+PiAgICAgICAgICB9DQo+IA0KPiANCj4gVGhlc2Ug
Y2hhbmdlcyB0byBsaWJ4bCBhcmUgcXVpdGUgY29uZnVzaW5nLCB0aGVyZSdzIGEgbWl4ZWQgdXAg
YmV0d2Vlbg0KPiAiZ3NpIiBhbmQgImlycSIsIGl0J3MgaGFyZCB0byBmb2xsb3cuDQo+IA0KPiBU
aGFua3MsDQo+IA0KDQotLSANCkJlc3QgcmVnYXJkcywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 08:21:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 08:21:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684227.1063937 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd4Kk-0000VG-40; Thu, 22 Feb 2024 08:21:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684227.1063937; Thu, 22 Feb 2024 08:21:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd4Kj-0000V9-W6; Thu, 22 Feb 2024 08:21:33 +0000
Received: by outflank-mailman (input) for mailman id 684227;
 Thu, 22 Feb 2024 08:21:33 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd4Kj-0000Uz-2J; Thu, 22 Feb 2024 08:21:33 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd4Ki-0001kB-Ox; Thu, 22 Feb 2024 08:21:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd4Ki-0000Co-BL; Thu, 22 Feb 2024 08:21:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rd4Ki-0007XO-Aw; Thu, 22 Feb 2024 08:21:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=pmhR7Bcpi4UvJeXGZhDz7JEWQ/aBFmJkYdjWyedS8wE=; b=HKCPT9pbn7cUFUp7XOELgDTp2v
	wlPOmNxLyVfSZkiBlezX8VsRrt6zgtofSEvr9H2ED56RBhc7MVphae4U1sT4u2K0dr6YGLafEQYoV
	GPp0Trvmh4chOg4YnP8n2GIvyy2YYtulg6cnVh/ofCcNYcGFAo0fBPVSfoGW7UtIw9So=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184722-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184722: regressions - FAIL
X-Osstest-Failures:
    linux-linus:build-arm64-pvops:kernel-build:fail:regression
    linux-linus:test-amd64-amd64-xl-credit1:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-examine:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=39133352cbed6626956d38ed72012f49b0421e7b
X-Osstest-Versions-That:
    linux=9fc1ccccfd8d53dc7936fe6d633f2373fc9f62e8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 22 Feb 2024 08:21:32 +0000

flight 184722 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184722/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvops             6 kernel-build   fail in 184721 REGR. vs. 184719

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-credit1  22 guest-start/debian.repeat  fail pass in 184721

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit1   1 build-check(1)           blocked in 184721 n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)           blocked in 184721 n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)           blocked in 184721 n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)           blocked in 184721 n/a
 test-arm64-arm64-examine      1 build-check(1)           blocked in 184721 n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)           blocked in 184721 n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)           blocked in 184721 n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)           blocked in 184721 n/a
 test-arm64-arm64-xl           1 build-check(1)           blocked in 184721 n/a
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184719
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184719
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184719
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184719
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184719
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184719
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184719
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184719
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                39133352cbed6626956d38ed72012f49b0421e7b
baseline version:
 linux                9fc1ccccfd8d53dc7936fe6d633f2373fc9f62e8

Last test of basis   184719  2024-02-21 06:34:23 Z    1 days
Testing same since   184721  2024-02-21 17:43:51 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  David Sterba <dsterba@suse.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Jason Wang <jasowang@redhat.com>
  Josef Bacik <josef@toxicpanda.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Marc Zyngier <maz@kernel.org>
  Michael S. Tsirkin <mst@redhat.com>
  Nathan Chancellor <nathan@kernel.org> # build
  Oliver Upton <oliver.upton@linux.dev>
  Paolo Bonzini <pbonzini@redhat.com>
  Qu Wenruo <wqu@suse.com>
  zhenwei pi <pizhenwei@bytedance.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  fail    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 39133352cbed6626956d38ed72012f49b0421e7b
Merge: 8da8d88455eb c48617fbbe83
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed Feb 21 09:13:27 2024 -0800

    Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
    
    Pull kvm fixes from Paolo Bonzini:
     "Two fixes for ARM ITS emulation. Unmapped interrupts were used instead
      of ignored, causing NULL pointer dereferences"
    
    * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm:
      KVM: arm64: vgic-its: Test for valid IRQ in MOVALL handler
      KVM: arm64: vgic-its: Test for valid IRQ in its_sync_lpi_pending_table()

commit 8da8d88455ebbb4e05423cf60cff985e92d43754
Merge: d8be5a55b8e3 b0ad381fa769
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed Feb 21 08:45:07 2024 -0800

    Merge tag 'for-6.8-rc5-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux
    
    Pull btrfs fixes from David Sterba:
    
     - Fix a deadlock in fiemap.
    
       There was a big lock around the whole operation that can interfere
       with a page fault and mkwrite.
    
       Reducing the lock scope can also speed up fiemap
    
     - Fix range condition for extent defragmentation which could lead to
       worse layout in some cases
    
    * tag 'for-6.8-rc5-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux:
      btrfs: fix deadlock with fiemap and extent locking
      btrfs: defrag: avoid unnecessary defrag caused by incorrect extent size

commit d8be5a55b8e3f7eab8f36ceed2512f457f914318
Merge: 9fc1ccccfd8d c0ec2a712daf
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed Feb 21 08:37:49 2024 -0800

    Merge tag 'v6.8-p4' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
    
    Pull crypto fix from Herbert Xu:
     "Fix a stack overflow in virtio"
    
    * tag 'v6.8-p4' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6:
      crypto: virtio/akcipher - Fix stack overflow on memcpy

commit c48617fbbe831d4c80fe84056033f17b70a31136
Merge: 9895ceeb5cd6 85a71ee9a070
Author: Paolo Bonzini <pbonzini@redhat.com>
Date:   Wed Feb 21 05:18:56 2024 -0500

    Merge tag 'kvmarm-fixes-6.8-3' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
    
    KVM/arm64 fixes for 6.8, take #3
    
    - Check for the validity of interrupts handled by a MOVALL
      command
    
    - Check for the validity of interrupts while reading the
      pending state on enabling LPIs.

commit 85a71ee9a0700f6c18862ef3b0011ed9dad99aca
Author: Oliver Upton <oliver.upton@linux.dev>
Date:   Wed Feb 21 09:27:32 2024 +0000

    KVM: arm64: vgic-its: Test for valid IRQ in MOVALL handler
    
    It is possible that an LPI mapped in a different ITS gets unmapped while
    handling the MOVALL command. If that is the case, there is no state that
    can be migrated to the destination. Silently ignore it and continue
    migrating other LPIs.
    
    Cc: stable@vger.kernel.org
    Fixes: ff9c114394aa ("KVM: arm/arm64: GICv4: Handle MOVALL applied to a vPE")
    Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
    Link: https://lore.kernel.org/r/20240221092732.4126848-3-oliver.upton@linux.dev
    Signed-off-by: Marc Zyngier <maz@kernel.org>

commit 8d3a7dfb801d157ac423261d7cd62c33e95375f8
Author: Oliver Upton <oliver.upton@linux.dev>
Date:   Wed Feb 21 09:27:31 2024 +0000

    KVM: arm64: vgic-its: Test for valid IRQ in its_sync_lpi_pending_table()
    
    vgic_get_irq() may not return a valid descriptor if there is no ITS that
    holds a valid translation for the specified INTID. If that is the case,
    it is safe to silently ignore it and continue processing the LPI pending
    table.
    
    Cc: stable@vger.kernel.org
    Fixes: 33d3bc9556a7 ("KVM: arm64: vgic-its: Read initial LPI pending table")
    Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
    Link: https://lore.kernel.org/r/20240221092732.4126848-2-oliver.upton@linux.dev
    Signed-off-by: Marc Zyngier <maz@kernel.org>

commit b0ad381fa7690244802aed119b478b4bdafc31dd
Author: Josef Bacik <josef@toxicpanda.com>
Date:   Mon Feb 12 11:56:02 2024 -0500

    btrfs: fix deadlock with fiemap and extent locking
    
    While working on the patchset to remove extent locking I got a lockdep
    splat with fiemap and pagefaulting with my new extent lock replacement
    lock.
    
    This deadlock exists with our normal code, we just don't have lockdep
    annotations with the extent locking so we've never noticed it.
    
    Since we're copying the fiemap extent to user space on every iteration
    we have the chance of pagefaulting.  Because we hold the extent lock for
    the entire range we could mkwrite into a range in the file that we have
    mmap'ed.  This would deadlock with the following stack trace
    
    [<0>] lock_extent+0x28d/0x2f0
    [<0>] btrfs_page_mkwrite+0x273/0x8a0
    [<0>] do_page_mkwrite+0x50/0xb0
    [<0>] do_fault+0xc1/0x7b0
    [<0>] __handle_mm_fault+0x2fa/0x460
    [<0>] handle_mm_fault+0xa4/0x330
    [<0>] do_user_addr_fault+0x1f4/0x800
    [<0>] exc_page_fault+0x7c/0x1e0
    [<0>] asm_exc_page_fault+0x26/0x30
    [<0>] rep_movs_alternative+0x33/0x70
    [<0>] _copy_to_user+0x49/0x70
    [<0>] fiemap_fill_next_extent+0xc8/0x120
    [<0>] emit_fiemap_extent+0x4d/0xa0
    [<0>] extent_fiemap+0x7f8/0xad0
    [<0>] btrfs_fiemap+0x49/0x80
    [<0>] __x64_sys_ioctl+0x3e1/0xb50
    [<0>] do_syscall_64+0x94/0x1a0
    [<0>] entry_SYSCALL_64_after_hwframe+0x6e/0x76
    
    I wrote an fstest to reproduce this deadlock without my replacement lock
    and verified that the deadlock exists with our existing locking.
    
    To fix this simply don't take the extent lock for the entire duration of
    the fiemap.  This is safe in general because we keep track of where we
    are when we're searching the tree, so if an ordered extent updates in
    the middle of our fiemap call we'll still emit the correct extents
    because we know what offset we were on before.
    
    The only place we maintain the lock is searching delalloc.  Since the
    delalloc stuff can change during writeback we want to lock the extent
    range so we have a consistent view of delalloc at the time we're
    checking to see if we need to set the delalloc flag.
    
    With this patch applied we no longer deadlock with my testcase.
    
    CC: stable@vger.kernel.org # 6.1+
    Reviewed-by: Filipe Manana <fdmanana@suse.com>
    Signed-off-by: Josef Bacik <josef@toxicpanda.com>
    Reviewed-by: David Sterba <dsterba@suse.com>
    Signed-off-by: David Sterba <dsterba@suse.com>

commit e42b9d8b9ea2672811285e6a7654887ff64d23f3
Author: Qu Wenruo <wqu@suse.com>
Date:   Wed Feb 7 10:00:42 2024 +1030

    btrfs: defrag: avoid unnecessary defrag caused by incorrect extent size
    
    [BUG]
    With the following file extent layout, defrag would do unnecessary IO
    and result more on-disk space usage.
    
      # mkfs.btrfs -f $dev
      # mount $dev $mnt
      # xfs_io -f -c "pwrite 0 40m" $mnt/foobar
      # sync
      # xfs_io -f -c "pwrite 40m 16k" $mnt/foobar
      # sync
    
    Above command would lead to the following file extent layout:
    
            item 6 key (257 EXTENT_DATA 0) itemoff 15816 itemsize 53
                    generation 7 type 1 (regular)
                    extent data disk byte 298844160 nr 41943040
                    extent data offset 0 nr 41943040 ram 41943040
                    extent compression 0 (none)
            item 7 key (257 EXTENT_DATA 41943040) itemoff 15763 itemsize 53
                    generation 8 type 1 (regular)
                    extent data disk byte 13631488 nr 16384
                    extent data offset 0 nr 16384 ram 16384
                    extent compression 0 (none)
    
    Which is mostly fine. We can allow the final 16K to be merged with the
    previous 40M, but it's upon the end users' preference.
    
    But if we defrag the file using the default parameters, it would result
    worse file layout:
    
     # btrfs filesystem defrag $mnt/foobar
     # sync
    
            item 6 key (257 EXTENT_DATA 0) itemoff 15816 itemsize 53
                    generation 7 type 1 (regular)
                    extent data disk byte 298844160 nr 41943040
                    extent data offset 0 nr 8650752 ram 41943040
                    extent compression 0 (none)
            item 7 key (257 EXTENT_DATA 8650752) itemoff 15763 itemsize 53
                    generation 9 type 1 (regular)
                    extent data disk byte 340787200 nr 33292288
                    extent data offset 0 nr 33292288 ram 33292288
                    extent compression 0 (none)
            item 8 key (257 EXTENT_DATA 41943040) itemoff 15710 itemsize 53
                    generation 8 type 1 (regular)
                    extent data disk byte 13631488 nr 16384
                    extent data offset 0 nr 16384 ram 16384
                    extent compression 0 (none)
    
    Note the original 40M extent is still there, but a new 32M extent is
    created for no benefit at all.
    
    [CAUSE]
    There is an existing check to make sure we won't defrag a large enough
    extent (the threshold is by default 32M).
    
    But the check is using the length to the end of the extent:
    
            range_len = em->len - (cur - em->start);
    
            /* Skip too large extent */
            if (range_len >= extent_thresh)
                    goto next;
    
    This means, for the first 8MiB of the extent, the range_len is always
    smaller than the default threshold, and would not be defragged.
    But after the first 8MiB, the remaining part would fit the requirement,
    and be defragged.
    
    Such different behavior inside the same extent caused the above problem,
    and we should avoid different defrag decision inside the same extent.
    
    [FIX]
    Instead of using @range_len, just use @em->len, so that we have a
    consistent decision among the same file extent.
    
    Now with this fix, we won't touch the extent, thus not making it any
    worse.
    
    Reported-by: Filipe Manana <fdmanana@suse.com>
    Fixes: 0cb5950f3f3b ("btrfs: fix deadlock when reserving space during defrag")
    CC: stable@vger.kernel.org # 6.1+
    Reviewed-by: Boris Burkov <boris@bur.io>
    Reviewed-by: Filipe Manana <fdmanana@suse.com>
    Signed-off-by: Qu Wenruo <wqu@suse.com>
    Signed-off-by: David Sterba <dsterba@suse.com>

commit c0ec2a712daf133d9996a8a1b7ee2d4996080363
Author: zhenwei pi <pizhenwei@bytedance.com>
Date:   Tue Jan 30 19:27:40 2024 +0800

    crypto: virtio/akcipher - Fix stack overflow on memcpy
    
    sizeof(struct virtio_crypto_akcipher_session_para) is less than
    sizeof(struct virtio_crypto_op_ctrl_req::u), copying more bytes from
    stack variable leads stack overflow. Clang reports this issue by
    commands:
    make -j CC=clang-14 mrproper >/dev/null 2>&1
    make -j O=/tmp/crypto-build CC=clang-14 allmodconfig >/dev/null 2>&1
    make -j O=/tmp/crypto-build W=1 CC=clang-14 drivers/crypto/virtio/
      virtio_crypto_akcipher_algs.o
    
    Fixes: 59ca6c93387d ("virtio-crypto: implement RSA algorithm")
    Link: https://lore.kernel.org/all/0a194a79-e3a3-45e7-be98-83abd3e1cb7e@roeck-us.net/
    Cc: <stable@vger.kernel.org>
    Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
    Tested-by: Nathan Chancellor <nathan@kernel.org> # build
    Acked-by: Michael S. Tsirkin <mst@redhat.com>
    Acked-by: Jason Wang <jasowang@redhat.com>
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 08:28:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 08:28:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684236.1063947 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd4RA-0001Dl-TX; Thu, 22 Feb 2024 08:28:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684236.1063947; Thu, 22 Feb 2024 08:28:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd4RA-0001De-Qi; Thu, 22 Feb 2024 08:28:12 +0000
Received: by outflank-mailman (input) for mailman id 684236;
 Thu, 22 Feb 2024 08:28:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MkW+=J7=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rd4R9-0001DY-Eq
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 08:28:11 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4f1da88b-d15c-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 09:28:09 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id A6C1F1F44E;
 Thu, 22 Feb 2024 08:28:08 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8BD01133DC;
 Thu, 22 Feb 2024 08:28:08 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id /+0RIJgF12XZbwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 22 Feb 2024 08:28:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4f1da88b-d15c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708590488; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=5m00YemeUG0BUhaAa1PF536kpt34ewP3xHc930PyLHk=;
	b=jzbutF+O89hZi+JHWmN5kEZiBQsbPLObixLymDupoUsQ1woNQQtaI9yMHUgBf/LYwmd3sG
	+9a2qp8vUhviFkADtRyYXyp0IFvMw2rXVBFMKiayco/RC6X8llTYtdkhKqffZnieidPFyH
	zPrztDt1aB7wi6M6zaPa4hTM/Fc0KJA=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708590488; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=5m00YemeUG0BUhaAa1PF536kpt34ewP3xHc930PyLHk=;
	b=jzbutF+O89hZi+JHWmN5kEZiBQsbPLObixLymDupoUsQ1woNQQtaI9yMHUgBf/LYwmd3sG
	+9a2qp8vUhviFkADtRyYXyp0IFvMw2rXVBFMKiayco/RC6X8llTYtdkhKqffZnieidPFyH
	zPrztDt1aB7wi6M6zaPa4hTM/Fc0KJA=
Message-ID: <6d6646ce-be79-4d7c-bca6-0d44262d30b5@suse.com>
Date: Thu, 22 Feb 2024 09:28:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [linux-linus test] 184722: regressions - FAIL
To: osstest service owner <osstest-admin@xenproject.org>,
 xen-devel@lists.xenproject.org
References: <osstest-184722-mainreport@xen.org>
Content-Language: en-US
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <osstest-184722-mainreport@xen.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------VQhtSiN7qj71Z0n6VxUBhTrg"
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [-0.99 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 HAS_ATTACHMENT(0.00)[];
	 MIME_BASE64_TEXT_BOGUS(1.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MIME_BASE64_TEXT(0.10)[];
	 RCPT_COUNT_TWO(0.00)[2];
	 SIGNED_PGP(-2.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[];
	 MIME_UNKNOWN(0.10)[application/pgp-keys]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: -0.99

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------VQhtSiN7qj71Z0n6VxUBhTrg
Content-Type: multipart/mixed; boundary="------------frDl0zaZjc9wTRC5UQpqIKin";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: osstest service owner <osstest-admin@xenproject.org>,
 xen-devel@lists.xenproject.org
Message-ID: <6d6646ce-be79-4d7c-bca6-0d44262d30b5@suse.com>
Subject: Re: [linux-linus test] 184722: regressions - FAIL
References: <osstest-184722-mainreport@xen.org>
In-Reply-To: <osstest-184722-mainreport@xen.org>

--------------frDl0zaZjc9wTRC5UQpqIKin
Content-Type: multipart/mixed; boundary="------------GXwV9dSDNUFd10c9i3iNi0es"

--------------GXwV9dSDNUFd10c9i3iNi0es
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMjIuMDIuMjQgMDk6MjEsIG9zc3Rlc3Qgc2VydmljZSBvd25lciB3cm90ZToNCj4gZmxp
Z2h0IDE4NDcyMiBsaW51eC1saW51cyByZWFsIFtyZWFsXQ0KPiBodHRwOi8vbG9ncy50ZXN0
LWxhYi54ZW5wcm9qZWN0Lm9yZy9vc3N0ZXN0L2xvZ3MvMTg0NzIyLw0KPiANCj4gUmVncmVz
c2lvbnMgOi0oDQo+IA0KPiBUZXN0cyB3aGljaCBkaWQgbm90IHN1Y2NlZWQgYW5kIGFyZSBi
bG9ja2luZywNCj4gaW5jbHVkaW5nIHRlc3RzIHdoaWNoIGNvdWxkIG5vdCBiZSBydW46DQo+
ICAgYnVpbGQtYXJtNjQtcHZvcHMgICAgICAgICAgICAgNiBrZXJuZWwtYnVpbGQgICBmYWls
IGluIDE4NDcyMSBSRUdSLiB2cy4gMTg0NzE5DQoNCkxvZyBzYXlzOg0KDQpnY2M6IGludGVy
bmFsIGNvbXBpbGVyIGVycm9yOiBTZWdtZW50YXRpb24gZmF1bHQgc2lnbmFsIHRlcm1pbmF0
ZWQgcHJvZ3JhbSBjYzENClBsZWFzZSBzdWJtaXQgYSBmdWxsIGJ1ZyByZXBvcnQsDQp3aXRo
IHByZXByb2Nlc3NlZCBzb3VyY2UgaWYgYXBwcm9wcmlhdGUuDQpTZWUgPGZpbGU6Ly8vdXNy
L3NoYXJlL2RvYy9nY2MtOC9SRUFETUUuQnVncz4gZm9yIGluc3RydWN0aW9ucy4NCm1ha2Vb
NV06ICoqKiBbc2NyaXB0cy9NYWtlZmlsZS5idWlsZDoyNDM6IGRyaXZlcnMvaWlvL2FkYy9t
YXg5NjExLm9dIEVycm9yIDQNCm1ha2VbNV06ICoqKiBXYWl0aW5nIGZvciB1bmZpbmlzaGVk
IGpvYnMuLi4uDQoNCg0KSnVlcmdlbg0K
--------------GXwV9dSDNUFd10c9i3iNi0es
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------GXwV9dSDNUFd10c9i3iNi0es--

--------------frDl0zaZjc9wTRC5UQpqIKin--

--------------VQhtSiN7qj71Z0n6VxUBhTrg
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXXBZgFAwAAAAAACgkQsN6d1ii/Ey9h
egf/ctsZum7AwVLuB7fvRsQcFyvi9pt5ievxGKNx03krBUbA2yw/BiLnBt+27rP0xugN+grlf0A1
BRKhZxRd4989UMo2jt7lKRaHueceswjYwJdtPDl9yF27zJF+HzKQ8UMkyhn51N5JpxlynHPpr/vO
WJqq4b/g3t/mP2ba0qEp/I6VqnWKm10TvraQ7LFWZY0A/LNWZjDeAL30KKs2OOuf0d5kQ3GKshFE
2+gLcy1wBhu+3gYNrqcHg3msbOy6E2mD1fYskCHeW3AcYjUB4PnPv2eUBhybJLa3ggA0A/PJUWlV
LoGrvTnuWQxLJNLfTlB9Tx3uSBodzXwSGBkQUJBgfA==
=8wyI
-----END PGP SIGNATURE-----

--------------VQhtSiN7qj71Z0n6VxUBhTrg--


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 08:33:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 08:33:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684240.1063957 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd4WV-0002tV-GD; Thu, 22 Feb 2024 08:33:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684240.1063957; Thu, 22 Feb 2024 08:33:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd4WV-0002tO-Cu; Thu, 22 Feb 2024 08:33:43 +0000
Received: by outflank-mailman (input) for mailman id 684240;
 Thu, 22 Feb 2024 08:33:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+gCR=J7=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rd4WT-0002tE-PT
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 08:33:41 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 140fbabf-d15d-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 09:33:39 +0100 (CET)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2d1094b5568so97689831fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 00:33:39 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 ka7-20020a05600c584700b004101543e843sm5273202wmb.10.2024.02.22.00.33.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 22 Feb 2024 00:33:38 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 140fbabf-d15d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708590818; x=1709195618; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=e8K4AzAt7zREk2uoZWOW/KP6G4e1UyXKQkZ20gvIn3I=;
        b=ULp4KoD/Rx+S2u95vxvPopbo87/xihSPPWwMPxOAdqxiRUkj55vuyGIO0VjjZ3KorG
         7cGCeKDuAi8jbQ8HAksd5t5FIJJ+k5lRXRvFbaW8TnZXcwROOegVTrlvgTIwWJCIaoZA
         /B1EJ5WAXXT2sRqLykUurPDayBQ7+Mc+Tunx4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708590818; x=1709195618;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=e8K4AzAt7zREk2uoZWOW/KP6G4e1UyXKQkZ20gvIn3I=;
        b=Ay7VfeHEsNXdpGn9HYsgSc3DJyO0jg0UyiQQZVUcxlP+/tncJvNRZD84wWIeiW6Ddj
         8l9k4Htf1vHA3SXboySMQI5rKd3qfHL5aGNfZsbVOcg44Y6fIrNpL1hNsYEWUgbacw9W
         D54YgVVe7k5Ns+FIV1k+xOPvY5+IuYgINQdKng4+l2Z25sPg1iZn7Hfne58/VhUwqCR5
         yLP7jOTEzLEavBbYOqvA9Sx2djQ0qFdxx4oPI3wKnjh/bkoYa7tVIiqhU+yQGxZf8IRv
         THOgJKBrDBkGwJKzVKdQk8cqZakVrlo/VTQCx0I3fNiuyOdEciopM/Mf5YB37AzfTslk
         /FrA==
X-Gm-Message-State: AOJu0Yx+2cAZ9Od+nsjGgSDbixau5v9+NV1bza3bEINFJwnfUQTG48tT
	KAA+//jvFtYHi3tlprlFP1J5IIpCdScwSklxECBomaqHSRR9C5+CKtRP0L04Bj/o/KH67K4hdbP
	U
X-Google-Smtp-Source: AGHT+IHkHxwPgrYdJRUoW/hybj9DTiquKRcIf5cOXAiHWArerCXkARgUbJ6ngE5V2a+ePQZ8ARjT3w==
X-Received: by 2002:a2e:a49a:0:b0:2d2:4315:4d8b with SMTP id h26-20020a2ea49a000000b002d243154d8bmr5646080lji.2.1708590818324;
        Thu, 22 Feb 2024 00:33:38 -0800 (PST)
Date: Thu, 22 Feb 2024 09:33:37 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86/altcall: use an union as register type for function
 parameters
Message-ID: <ZdcG4bSMael4aunD@macbook>
References: <20240221170331.60605-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20240221170331.60605-1-roger.pau@citrix.com>

On Wed, Feb 21, 2024 at 06:03:31PM +0100, Roger Pau Monne wrote:
> The current code for alternative calls uses the caller parameter types as the
> types for the register variables that serve as function parameters:
> 
> uint8_t foo;
> [...]
> alternative_call(myfunc, foo);
> 
> Would expand roughly into:
> 
> register unint8_t a1_ asm("rdi") = foo;
> register unsigned long a2_ asm("rsi");
> [...]
> asm volatile ("call *%c[addr](%%rip)"...);
> 
> However under certain circumstances clang >= 16.0.0 with -O2 can generate
> incorrect code, given the following example:
> 
> unsigned int func(uint8_t t)
> {
>     return t;
> }
> 
> static void bar(uint8_t b)
> {
>     int ret_;
>     register uint8_t di asm("rdi") = b;
>     register unsigned long si asm("rsi");
>     register unsigned long dx asm("rdx");
>     register unsigned long cx asm("rcx");
>     register unsigned long r8 asm("r8");
>     register unsigned long r9 asm("r9");
>     register unsigned long r10 asm("r10");
>     register unsigned long r11 asm("r11");
> 
>     asm volatile ( "call %c[addr]"
>                    : "+r" (di), "=r" (si), "=r" (dx),
>                      "=r" (cx), "=r" (r8), "=r" (r9),
>                      "=r" (r10), "=r" (r11), "=a" (ret_)
>                    : [addr] "i" (&(func)), "g" (func)
>                    : "memory" );
> }
> 
> void foo(unsigned int a)
> {
>     bar(a);
> }
> 
> Clang generates the following code:
> 
> func:                                   # @func
>         movl    %edi, %eax
>         retq
> foo:                                    # @foo
>         callq   func
>         retq
> 
> Note the truncation of the unsigned int parameter 'a' of foo() to uint8_t when
> passed into bar() is lost.
> 
> The above can be worked around by using an union when defining the register
> variables, so that `di` becomes:
> 
> register union {
>     uint8_t e;
>     unsigned long r;
> } di asm("rdi") = { .e = b };
> 
> Which results in following code generated for `foo()`:
> 
> foo:                                    # @foo
>         movzbl  %dil, %edi
>         callq   func
>         retq
> 
> So the truncation is not longer lost.
> 

This is missing:

Reported-by: Matthew Grooms <mgrooms@shrew.net>
Link: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277200
Link: https://github.com/llvm/llvm-project/issues/82598

Last one is the bug report against llvm.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 08:42:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 08:42:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684246.1063966 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd4eg-0004eu-8U; Thu, 22 Feb 2024 08:42:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684246.1063966; Thu, 22 Feb 2024 08:42:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd4eg-0004en-59; Thu, 22 Feb 2024 08:42:10 +0000
Received: by outflank-mailman (input) for mailman id 684246;
 Thu, 22 Feb 2024 08:42:08 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6628=J7=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rd4ee-0004ef-TM
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 08:42:08 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 42a51df9-d15e-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 09:42:07 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id A0E8D4EE0737;
 Thu, 22 Feb 2024 09:42:06 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42a51df9-d15e-11ee-8a55-1f161083a0e0
MIME-Version: 1.0
Date: Thu, 22 Feb 2024 09:42:06 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com, jbeulich@suse.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Simone Ballarin <simone.ballarin@bugseng.com>, Doug
 Goldstein <cardoe@cardoe.com>, George Dunlap <george.dunlap@citrix.com>, Wei
 Liu <wl@xen.org>
Subject: Re: [XEN PATCH v2] automation/eclair_analysis: deviate certain macros
 for Rule 20.12
In-Reply-To: <7bc72f6ccd858e0405d62d49c32449e3b5abd790.1707996317.git.nicola.vetrini@bugseng.com>
References: <7bc72f6ccd858e0405d62d49c32449e3b5abd790.1707996317.git.nicola.vetrini@bugseng.com>
Message-ID: <d31f6e4cb46de29898f8869991d3cbad@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-15 14:06, Nicola Vetrini wrote:
> Certain macros are allowed to violate the Rule, since their meaning and
> intended use is well-known to all Xen developers.
> 
> Variadic macros that rely on the GCC extension for removing a trailing
> comma when token pasting the variable argument are similarly
> well-understood and therefore allowed.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> ---
> Changes in v2:
> - Restrict deviation for GENERATE_CASE to vcpreg.c.
> - Improve deviation justifications.
> ---
>  .../eclair_analysis/ECLAIR/deviations.ecl     | 20 +++++++++++++++++
>  docs/misra/deviations.rst                     | 22 +++++++++++++++++++
>  2 files changed, 42 insertions(+)
> 

Ping?

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 09:06:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 09:06:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684259.1064000 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd52d-00088a-QH; Thu, 22 Feb 2024 09:06:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684259.1064000; Thu, 22 Feb 2024 09:06:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd52d-00088P-MN; Thu, 22 Feb 2024 09:06:55 +0000
Received: by outflank-mailman (input) for mailman id 684259;
 Thu, 22 Feb 2024 09:06:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+gCR=J7=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rd52c-00087Q-JV
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 09:06:54 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b80a6e5c-d161-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 10:06:52 +0100 (CET)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2d21cdbc85bso21242421fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 01:06:52 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 bj13-20020a0560001e0d00b0033d926bf7b5sm1243304wrb.76.2024.02.22.01.06.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 22 Feb 2024 01:06:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b80a6e5c-d161-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708592811; x=1709197611; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=KyY62PKoEMOxefDcZ5yAyBP77/0u5doyNW7ljPYkb9c=;
        b=FJM57vDYmMbbustH6UW/uv3X/ELhxzTk6neugH5/jwkvsUormaaJ4Y906RA/VW5aUa
         Vt8FzkRtiGp02/1XAMNdJrLyaCRAELkjk6i5N6Aqa45XzD1AIoLJLdYOkQuCT+EBziaY
         eBQrJsV6wY+4eFWF6omBkMa/AZ5PC9t8lifN8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708592811; x=1709197611;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=KyY62PKoEMOxefDcZ5yAyBP77/0u5doyNW7ljPYkb9c=;
        b=YNDZiDjRJuk+j7n/rIEn4j5Ind1HZYlivw3QjOCXisGioZi/C6jzwP2pbYn0xpaCVK
         OGNFlFi4xkJljfkteYlmqlvzRUFEcOMyvSa8P4SmHQKu+WP7fGd2kUlA+8OyZksjj2kV
         x1MstY4pVAdkkf4swN7M6sGHJWrDN5AzR9iBZKv5wUVxhhZoBIJhTVRrydZB2kF29T/+
         5PvlpmdNI0lNZ2NXoYgt0hitiX1JKHoqxxqpuntyLM2J+6uR+gxCrRyLKQJq/Cx5vqI5
         wPJYNVukQwbnG9ZzldJbu919l64h44RzzjtXxws5HAxjsRfN4vJB/cbMqWe61agFvUr1
         sj4w==
X-Gm-Message-State: AOJu0YxB7rRP1Mz/4wk01YMPF8Qsh2oxL1dcRqmkJgEyS/ayBLbJ3lyO
	2UzBeTizOQbLNDFkI8fVskkwbcmUklABNanAfTfT/QX1waMsIUan3bH6q8/MdAgdZv1YoLoSu+H
	o
X-Google-Smtp-Source: AGHT+IH0LcjYx+aEuc0oDfHapt2lkKd4+JT9s/wqOVAcCMosCzuSAIkK0yHnfTo1EzuGmanEP66ulg==
X-Received: by 2002:a2e:300d:0:b0:2d2:39fa:822c with SMTP id w13-20020a2e300d000000b002d239fa822cmr6653432ljw.35.1708592811460;
        Thu, 22 Feb 2024 01:06:51 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 2/2] x86/hpet: use an atomic add instead of a cmpxchg loop
Date: Thu, 22 Feb 2024 10:05:30 +0100
Message-ID: <20240222090530.62530-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240222090530.62530-1-roger.pau@citrix.com>
References: <20240222090530.62530-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The usage of a cmpxchg loop in hpet_get_channel() is unnecessary, as the same
can be achieved with an atomic increment, which is both simpler to read, and
avoid any need for a loop.

Note there can be a small divergence in the channel returned if next_channel
overflows, but returned channel will always be in the [0, num_hpets_used)
range, and that's fine for the purpose of balancing HPET channels across CPUs.
This is also theoretical, as there's no system currently with 2^32 CPUs (as
long as next_channel is 32bit width).

Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/hpet.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
index d982b0f6b2c9..4777dc859d96 100644
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -458,11 +458,7 @@ static struct hpet_event_channel *hpet_get_channel(unsigned int cpu)
     if ( num_hpets_used >= nr_cpu_ids )
         return &hpet_events[cpu];
 
-    do {
-        next = next_channel;
-        if ( (i = next + 1) == num_hpets_used )
-            i = 0;
-    } while ( cmpxchg(&next_channel, next, i) != next );
+    next = arch_fetch_and_add(&next_channel, 1) % num_hpets_used;
 
     /* try unused channel first */
     for ( i = next; i < next + num_hpets_used; i++ )
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 22 09:06:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 09:06:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684258.1063985 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd52a-0007hV-EM; Thu, 22 Feb 2024 09:06:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684258.1063985; Thu, 22 Feb 2024 09:06:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd52a-0007ge-AT; Thu, 22 Feb 2024 09:06:52 +0000
Received: by outflank-mailman (input) for mailman id 684258;
 Thu, 22 Feb 2024 09:06:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+gCR=J7=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rd52Z-0007eV-Li
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 09:06:51 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b73d0d3c-d161-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 10:06:51 +0100 (CET)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-512be6fda52so4687372e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 01:06:51 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 fa27-20020a05600c519b00b004128812dcb6sm1172138wmb.28.2024.02.22.01.06.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 22 Feb 2024 01:06:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b73d0d3c-d161-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708592810; x=1709197610; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=bH2mQPDEXai+re4kmllotrxpSIRK6T6FaHn93utR0MA=;
        b=Jjfj6KJsT+4Bn3V6yBgQeTeQFryg49hEUkmx70T1hp7w0ayV3tS/NWsVnu0dQ8uIfC
         GHJv1GsrrzWPPbD8Hu7H3KSuPQEsOOKiBQKr0eeq4nC2xKAECEr/ChcDzrlZqsBcORXq
         vXh8lpuot5pAz88YVQjpyIg9Qma4xrYsIuEQg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708592810; x=1709197610;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=bH2mQPDEXai+re4kmllotrxpSIRK6T6FaHn93utR0MA=;
        b=XBF7L0wK6y72Bu9Lmm1TmjtxUUMGiarpH5RQxhSH0AAgpiYVuZOnyx1ptg7jFVCXZs
         clLelE3hNMtdGkXQzWvFi/SgwYD8fzMjLk21xFoIo0xbvkFB0WFPM8T4k50c62PUVrXr
         jObQtp+z+RNKAnM3dsCN/mwbP0rp6ytSNZmjRSXAxwJpXxzxoj6Y7/novLTea3foGV6w
         X/bWQao1MiStUW8bRbhNGqMgnV7EHWwK8OVtRVZktxEkLY8GW+zL67d/PDuM4gIa8fzP
         anphTgTn5rqDLNdbRbQVfxWX/PTzcv2DeKIyiXGWiQPFJ0nK+wYDwrXvJAuAJERY7Rtu
         IDig==
X-Gm-Message-State: AOJu0YyEWxCkxy5ZGe3lUnzPP52nhbsj6AmLcJ7rRV4pIKikQoBVI3bn
	v1nanrgcjD+EVzTxheX7Qvfs2uxlG8KRQT0FpajZBRULg2bld8hZimvo2BGCzldLmphnqahHUit
	3
X-Google-Smtp-Source: AGHT+IEVOhWolRmh36S3w5sq/YHv1CBFSFLI/qadi605FiMuwznoTB0h/XtNqavitfnG68w/Z8bmEw==
X-Received: by 2002:a05:6512:12c3:b0:511:8d49:4b4f with SMTP id p3-20020a05651212c300b005118d494b4fmr16125552lfg.6.1708592809958;
        Thu, 22 Feb 2024 01:06:49 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 1/2] x86/memsharing: use an atomic add instead of a cmpxchg loop
Date: Thu, 22 Feb 2024 10:05:29 +0100
Message-ID: <20240222090530.62530-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240222090530.62530-1-roger.pau@citrix.com>
References: <20240222090530.62530-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The usage of a cmpxchg loop in get_next_handle() is unnecessary, as the same
can be achieved with an atomic increment, which is both simpler to read, and
avoid any need for a loop.

The cmpxchg usage is likely a remnant of 32bit support, which didn't have an
instruction to do an atomic 64bit add, and instead a cmpxchg had to be used.

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/mm/mem_sharing.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
index 4f810706a315..fe299a2bf9aa 100644
--- a/xen/arch/x86/mm/mem_sharing.c
+++ b/xen/arch/x86/mm/mem_sharing.c
@@ -179,13 +179,7 @@ static void mem_sharing_page_unlock(struct page_info *pg)
 
 static shr_handle_t get_next_handle(void)
 {
-    /* Get the next handle get_page style */
-    uint64_t x, y = next_handle;
-    do {
-        x = y;
-    }
-    while ( (y = cmpxchg(&next_handle, x, x + 1)) != x );
-    return x + 1;
+    return arch_fetch_and_add(&next_handle, 1) + 1;
 }
 
 static atomic_t nr_saved_mfns   = ATOMIC_INIT(0);
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 22 09:06:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 09:06:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684257.1063981 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd52a-0007en-8W; Thu, 22 Feb 2024 09:06:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684257.1063981; Thu, 22 Feb 2024 09:06:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd52a-0007eg-3L; Thu, 22 Feb 2024 09:06:52 +0000
Received: by outflank-mailman (input) for mailman id 684257;
 Thu, 22 Feb 2024 09:06:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+gCR=J7=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rd52Y-0007eV-KV
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 09:06:50 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b66e2838-d161-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 10:06:49 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d23a22233fso49256691fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 01:06:49 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 ch19-20020a5d5d13000000b0033d817eddd3sm4414473wrb.13.2024.02.22.01.06.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 22 Feb 2024 01:06:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b66e2838-d161-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708592808; x=1709197608; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=fXk63syKH0DiHAkRMGb27p2YfWstaLmmfAzDV6w563M=;
        b=cyY/k7EUt/NzYPmsYzD2wXMMh22YOngpoGuwQ3Nil/hfeyS7IQU/soukRf6n+BP7d2
         o11Q3VFW2becoV8H0NngZ8WHKeoL8+/WT+X9ii/Uv4rqhRvwvvOsX86s2paGH5My1UAd
         7GRk6WqDf2c0qZP9tXTfcb8z7Qy2OUMi5QvcM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708592808; x=1709197608;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=fXk63syKH0DiHAkRMGb27p2YfWstaLmmfAzDV6w563M=;
        b=GDsmp61BihovjdQTUFiX+KsJkHDApsDDstBQG6NxgD0WFbPMaCgLuTR5LA3YiBl5le
         fCKGaYWeQq+mhwiB3GHrzXdWaBLrD3NpF3qyVjH9NcKofdALtdEh9uU5OvtLllDupGdF
         u5fgAta14Rb6iWBOrvRZ96TcFN+TtmElC7IzKIp0TSe1TuGSxaf+uPBqqDJpLAWFTFJz
         D5B7+vXoELNEmA/4SBsfCNX8fa2e7tEl3QoMYrZ719KlYosibd7RloABipzCAYjdLHPp
         qbJn8ycpQowOCIwH1GviPcLuzpayr089dmnzglWJGvAzrKI2QAyAO5U82mQUUtJjOfTW
         r+Ng==
X-Gm-Message-State: AOJu0Yz9dB7o9yRVY1qipXqt1ctJBADh4yAMzvPMljlxL4TTQ/OgJiXd
	Ic+NKRbqaLP1FZ0jpuvb51YprSCHU5mEGnszT8xJypEBc/Z9rlpBHLpM/6L4zLs3Gj4NT4vo5ro
	P
X-Google-Smtp-Source: AGHT+IEdpaCEOWZ2GcD0LzLeBwXq+tsb4ah3z17bTN9fnrvDf2wCOma5A5FTrXnzls7Y9z7/MtSVlw==
X-Received: by 2002:a2e:8ec9:0:b0:2d2:500a:1dd with SMTP id e9-20020a2e8ec9000000b002d2500a01ddmr3639377ljl.23.1708592808612;
        Thu, 22 Feb 2024 01:06:48 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 0/2] xen/x86: cmpxchg cleanup
Date: Thu, 22 Feb 2024 10:05:28 +0100
Message-ID: <20240222090530.62530-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

Following series replace a couple of cmpxchg loops with an atomic inc.
The usage of such loops probably dates back to 32bit support, which
didn't have an instruction to do an atomic 64bit addition.

Thanks, Roger.

Roger Pau Monne (2):
  x86/memsharing: use an atomic add instead of a cmpxchg loop
  x86/hpet: use an atomic add instead of a cmpxchg loop

 xen/arch/x86/hpet.c           | 6 +-----
 xen/arch/x86/mm/mem_sharing.c | 8 +-------
 2 files changed, 2 insertions(+), 12 deletions(-)

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 22 09:31:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 09:31:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684277.1064010 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd5Pf-0004iU-La; Thu, 22 Feb 2024 09:30:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684277.1064010; Thu, 22 Feb 2024 09:30:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd5Pf-0004iN-I4; Thu, 22 Feb 2024 09:30:43 +0000
Received: by outflank-mailman (input) for mailman id 684277;
 Thu, 22 Feb 2024 09:30:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aBgv=J7=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rd5Pd-0004iF-St
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 09:30:41 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0ab2d89d-d165-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 10:30:39 +0100 (CET)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2d2533089f6so21373371fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 01:30:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0ab2d89d-d165-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708594239; x=1709199039; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=0XHkumgOBcmnyxht3ocEmnFXRRBS8/vHQ1R5Cq89CQU=;
        b=PYxHzwiOvQKV/UJRy3p0qa1YOl7ts54LaCXpWuAoRzH0L1+gsZ2PSNqxCjB7QvmxMY
         4Gne2GAlaqYFZLSetXus1l43jS2j3DUQHYabmfZdTh0urAVtAaiYBB606rwKBoT5tmuG
         z69SnCwpt5fbN1QG3pE3VQ0hjt/Gb94g6dlBk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708594239; x=1709199039;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=0XHkumgOBcmnyxht3ocEmnFXRRBS8/vHQ1R5Cq89CQU=;
        b=c5+xnDf4k8os5iJffFGTvbmMKigatuwNNuaBPL8rDRFcEPSvCO48WfGBHLGrOTdpWL
         qJ/TbRAyWPcdw30D8ixyOHRJr6IBhYjOJMnGjjMqoT/OC5Aevh23ni9KZ8/zYfPw9ukF
         cIKuIVTCc5QpNLAqElzzPU13fUsyBqnDH2vlFWWp9quv+TtawwshzZ61Tfxnmtrom8fJ
         v3riwD4m4aniQlhKA4Dv6V+q1ppeD+IbLbDjCavjmtsgaBOuF/tndt/eEI6AzPXNvN5F
         izgBgAk9AQKR1tWuRrwjWd1lZNNdnZ7KatQIy4Hy5GQnVf6FJdQQnJWTNqQDMOan2KHd
         ZdXw==
X-Forwarded-Encrypted: i=1; AJvYcCWD85fPKG20rrYVKKK4hageyQHAXuozjgo/aQq5f6YDGkEftBnLF0/MFiUUynNSlZzelakY2lZSQ9UkBfer3e75UBCiA/hh3cKhrRYba3Y=
X-Gm-Message-State: AOJu0Yw6aCzojdsd3haSGpkyUP/KcIrLaxd3+GreIYyiRaHbvQzMLRj6
	oHTvpFe//IsryNlNDj+U4vNEbjxhMhVIgUQgxlIYDgbpTHAlQuN//aWHA+uTS/Nj/nagOcXqQXt
	IlkIfem4Dw0ts0lbuyB30Nl8mqYxyNQFWOn8wzENBeIaAmXul
X-Google-Smtp-Source: AGHT+IFbbM2DkbxE03rn5dvwqrjCkCSxxcaSjPRknWnrKeg4Un0/apsfbwFV28nZiZJA0pOObIciFiXL3LFqV+BWRYs=
X-Received: by 2002:a05:651c:2227:b0:2d2:20ad:a51 with SMTP id
 y39-20020a05651c222700b002d220ad0a51mr12896111ljq.0.1708594239310; Thu, 22
 Feb 2024 01:30:39 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-5-george.dunlap@cloud.com> <c5f3a78c-4a2b-499b-93bb-11f60b0afcc0@suse.com>
 <CA+zSX=acH-4B01=vUBstAbsNBMVgcRe_A5v0eXn66BdsuPtgiA@mail.gmail.com> <74ef7b2b-b105-4060-afaa-73ec272a7449@suse.com>
In-Reply-To: <74ef7b2b-b105-4060-afaa-73ec272a7449@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Thu, 22 Feb 2024 17:30:27 +0800
Message-ID: <CA+zSX=ZPKScDQO6XXGOp6gv1arEs4-iWTgX8NSezYo4LGqkoLA@mail.gmail.com>
Subject: Re: [PATCH 4/6] nestedsvm: Disable TscRateMSR
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 21, 2024 at 6:52=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
> >> But then of course Andrew may know of reasons why all of this is done
> >> in calculate_host_policy() in the first place, rather than in HVM
> >> policy calculation.
> >
> > It sounds like maybe you're confusing host_policy with
> > x86_capabilities?  From what I can tell:
> >
> > *  the "basic" cpu_has_X macros resolve to boot_cpu_has(), which
> > resolves to cpu_has(&boot_cpu_data, ...), which is completely
> > independent of the cpu-policy.c:host_cpu_policy
> >
> > * cpu-policy.c:host_cpu_policy only affects what is advertised to
> > guests, via {pv,hvm}_cpu_policy and featureset bits.  Most notably a
> > quick skim doesn't show any mechanism by which host_cpu_policy could
> > affect what features Xen itself decides to use.
>
> I'm not mixing the two, no; the two are still insufficiently disentangled=
.
> There's really no reason (long term) to have both host policy and
> x86_capabilities. Therefore I'd prefer if new code (including a basically
> fundamental re-write as is going to be needed for nested) to avoid
> needlessly further extending x86_capabilities. Unless of course there's
> something fundamentally wrong with eliminating the redundancy, which
> likely Andrew would be in the best position to point out.

So I don't know the history of how things got to be the way they are,
nor really much about the code but what I've gathered from skimming
through while creating this patch series.  But from that impression,
the only issue I really see with the current code is the confusing
naming.  The cpufeature.h code has this nice infrastructure to allow
you to, for instance, enable or disable certain bits on the
command-line; and the interface for querying all the different bits of
functionality is all nicely put in one place.  Moving the
svm_feature_flags into x86_capabilities would immediately allow SVM to
take advantage of this infrastructure; it's not clear to me how this
would be "needless".

Furthermore, it looks to me like host_cpu_policy is used as a starting
point for generating pv_cpu_policy and hvm_cpu_policy, both of which
are only used for guest cpuid generation.  Given that the format of
those policies is fixed, and there's a lot of "copy this bit from the
host policy wholesale", it seems like no matter what, you'd want a
host_cpu_policy.

And in any case -- all that is kind of moot.  *Right now*,
host_cpu_policy is only used for guest cpuid policy creation; *right
now*, the nested virt features of AMD are handled in the
host_cpu_policy; *right now*, we're advertising to guests bits which
are not properly virtualized; *right now* these bits are actually set
unconditionally, regardless of whether they're even available on the
hardware; *right now*, Xen uses svm_feature_flags to determine its own
use of TscRateMSR; so *right now*, removing this bit from
host_cpu_policy won't prevent Xen from using TscRateMSR itself.

(Unless my understanding of the code is wrong, in which case I'd
appreciate a correction.)

If at some point in the future x86_capabilities and host_cpu_policy
were merged somehow, whoever did the merging would have to untangle
the twiddling of these bits anyway.  What I'm changing in this patch
wouldn't make that any harder.

> > Not sure exactly why the nested virt stuff is done at the
> > host_cpu_policy level rather than the hvm_cpu_policy level, but since
> > that's where it is, that's where we need to change it.
> >
> > FWIW, as I said in response to your comment on 2/6, it would be nicer
> > if we moved svm_feature_flags into the "capabilities" section; but
> > that's a different set of work.
>
> Can as well reply here then, rather than there: If the movement from
> host to HVM policy was done first, the patch here could more sanely go
> on top, and patch 2 could then also go on top, converting the separate
> variable to host policy accesses, quite possibly introducing a similar
> wrapper as you introduce there right now.
>
> But no, I'm not meaning to make this a requirement; this would merely be
> an imo better approach. My ack there stands, in case you want to keep
> (and commit) the change as is.

I mean, I don't mind doing a bit more prep work, if I know that's the
direction we want to go in.  "Actually, since you're doing a bit of
clean-up anyway -- right now host_cpu_policy is only used to calculate
guest policy, but we'd like to move over to being the Source of Truth
for the host instead of x86_capabilities.  While you're here, would
you mind moving the nested virt policy stuff into hvm_cpu_policy
instead?"

I certainly wouldn't want to move svm_feature_flags into
host_cpu_policy while it's still got random other "guest-only" policy
bits in it; and auditing it for such policy bits is out of the scope
of this work.

 -George


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 09:49:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 09:49:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684283.1064020 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd5hq-0006jK-5h; Thu, 22 Feb 2024 09:49:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684283.1064020; Thu, 22 Feb 2024 09:49:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd5hq-0006jD-1S; Thu, 22 Feb 2024 09:49:30 +0000
Received: by outflank-mailman (input) for mailman id 684283;
 Thu, 22 Feb 2024 09:49:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+gCR=J7=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rd5hn-0006gW-UX
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 09:49:27 +0000
Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com
 [2a00:1450:4864:20::42e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a9b028b0-d167-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 10:49:25 +0100 (CET)
Received: by mail-wr1-x42e.google.com with SMTP id
 ffacd0b85a97d-33aeb088324so4200846f8f.2
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 01:49:25 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 i3-20020a05600011c300b0033cf453f2bbsm19735292wrx.35.2024.02.22.01.49.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 22 Feb 2024 01:49:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a9b028b0-d167-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708595365; x=1709200165; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=eH/louA0em6mnyS2A46pkXSaK/sHzRF1mM8GXmk5/nQ=;
        b=iqe8He3WEZNZ6TaLVUVY1Fp5WGjqXlBeVPpseA6WGY/Sw/E7naRIFK9gOp7o6vFpAf
         1T8F/PWPC4ZeCUKH+TdQRiPKjfXEru5zNd1UQZ9ytqjZnOmt2Q6NZEY/jWNLRmKOO7SG
         ToYhKiZAK2wjdnUAT9iL3DTL033yfyRRu52Ok=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708595365; x=1709200165;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=eH/louA0em6mnyS2A46pkXSaK/sHzRF1mM8GXmk5/nQ=;
        b=ufacysC/cUS50gOcUpN6OduO3MEFmuIob0AIcL4nKfxJjLa0CjgcFPKbtBQGON6aBh
         KEgneNRsoiPZOAsYvFIYYDlw/RSBPh0cnVTuDN3N+ldk6E1rBMGnKN6nEyThwIWeupJN
         Y4Q41DqGDtamo82qbclsPdoEG3HxIeTJQb7znMVune2jNyEN5ue1v+ONJkl3jYJOegyZ
         Bqjt1C8lT+5f/q/qQR3k3HcNveEUX4oARo9QYBE1bvD23Cc4NZvshCnq+azJmFr6OT27
         ssA33nMHU0zjsWIAh5pQ4md2rfR+KG6NhuHOzqRCwhEZWe29L7Y4b86W2Yf+U2U/cfkO
         2+pA==
X-Forwarded-Encrypted: i=1; AJvYcCUeEeh1r8Rk6jeOL9F1Qa/2x6CbtHv/6lU/pEsib9x71TYou7Ug3qxwwoTe/F2TClBia/zMFC1TO/x3rHTn581H/snebzcMA/dM+M4/Zyk=
X-Gm-Message-State: AOJu0YyYIHiaOqHNcInuRpu4LMwMVKzaFpDkRaDy8V52sVy+E11NiRBS
	XJ783WX0YEtNQi4Cw93RpH0HUkVj3r8fRVlcLc3P+uqnRWkFO6qjv4D/71IQzP4=
X-Google-Smtp-Source: AGHT+IH7yDrRgNTIQqNr563gE6DXDJt+4EmmOyyu4t1w6EF4Q9Ah/RDyPD1RVHBC2W1cTeeWbtQK1g==
X-Received: by 2002:a05:6000:1543:b0:33d:6ede:249a with SMTP id 3-20020a056000154300b0033d6ede249amr5816964wry.69.1708595365100;
        Thu, 22 Feb 2024 01:49:25 -0800 (PST)
Date: Thu, 22 Feb 2024 10:49:23 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Julien Grall <julien@xen.org>
Cc: George Dunlap <george.dunlap@cloud.com>,
	Jan Beulich <jbeulich@suse.com>,
	Xen-devel <xen-devel@lists.xenproject.org>,
	committers@xenproject.org, Kelly Choi <kelly.choi@cloud.com>
Subject: Re: Stats on Xen tarball downloads
Message-ID: <ZdcYo3sGHVYaP6kM@macbook>
References: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
 <ZdMt88BSNMYtq39n@macbook>
 <36229344-e2e2-46e1-839b-b7b3b0843f23@suse.com>
 <CA+zSX=YvTgEMC0S5OqSFvxfVDeruiO=FphyVx5fv75oT-QW01A@mail.gmail.com>
 <0936a6c5-bac2-4de2-9155-1722d3bb28ca@xen.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <0936a6c5-bac2-4de2-9155-1722d3bb28ca@xen.org>

On Wed, Feb 21, 2024 at 10:53:49PM +0000, Julien Grall wrote:
> Hi George,
> 
> On 21/02/2024 02:55, George Dunlap wrote:
> > On Mon, Feb 19, 2024 at 6:38 PM Jan Beulich <jbeulich@suse.com> wrote:
> > > 
> > > On 19.02.2024 11:31, Roger Pau Monné wrote:
> > > > On Mon, Feb 19, 2024 at 06:01:54PM +0800, George Dunlap wrote:
> > > > > One of the questions we had with respect to changing our release
> > > > > practice (for instance, making the process more light-weight so that
> > > > > we could do a point release after every XSA) was, "How many people are
> > > > > actually using the tarballs?"
> > > > 
> > > > What would this more lightweight process involve from a downstream
> > > > PoV?  IOW: in what would the contents of the tarball change compared
> > > > to the current releases?
> > > 
> > >  From all prior discussion my conclusion was "no tarball at all".
> > 
> > Or at very least, the tarball would be a simple `git archive` of a
> > release tag.   Right now the tarball creation has a number of
> > annoyingly manual parts about it.
> At the moment we have the following steps:
> 
> 1) Checkout tag
> 2) Create the tarball
> 3) Check the source tarball can build
> 4) Sign the tarball
> 5) Upload it
> 
> I managed to script it so I have only two commands to execute (mostly
> because I build and sign on a different host).
> 
> AFAIU, your command 'git archive' will only replace 2. Am I correct? If so,
> it is not entirely clear how your proposal is going to make it better.

IMO building for release tarballs is easier than from a git checkout
(or archive).  It's a bit annoying to have to pre-download the
external project sources, now even more as QEMU is using git
submodules.

Most distro binary builders have infrastructure to deal with all this,
but requires a bit more logic in the recipe than a plain just fetch a
tarball and build from it.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 09:50:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 09:50:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684287.1064030 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd5jF-0008Ay-EW; Thu, 22 Feb 2024 09:50:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684287.1064030; Thu, 22 Feb 2024 09:50:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd5jF-0008Ar-Bf; Thu, 22 Feb 2024 09:50:57 +0000
Received: by outflank-mailman (input) for mailman id 684287;
 Thu, 22 Feb 2024 09:50:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rd5jE-0008Ah-7M
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 09:50:56 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id de3d57e8-d167-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 10:50:53 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-563c2b2bddbso10915294a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 01:50:53 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i7-20020aa7c9c7000000b005649f17558bsm3062566edt.42.2024.02.22.01.50.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 01:50:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: de3d57e8-d167-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708595453; x=1709200253; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/EP+k7IchCG5AUxEyx665O9ED9ZcjaJbrg7Qwar7VTs=;
        b=ExWXdSD96ZNyupVREZ+C8OSEP/c2NxYy/UoIkIH9lsJ9loKgzfub6qcf1tKUL5A6a6
         y99LPIJQZvs9O70zwT+1wizWXgTpyelFe4ueEkcpKDzLTuQuPs9r38kQooFIMKNDjREh
         yZYiOFtKY1vgAa7cnELt45Vj5Hg5uylt4lcjyKQufK1T+5tddBzfnH5GowX7POg+Tbih
         dFHZHNhkwhFegxVZ5ktBN5fyI5uYU3/IZNFasEVvAJ5ddsDtvmnZ6xzyoaQkGaNMjHiY
         /vqH8ihDiCY5BUPuHJvfJ25M0nF+r1DQxLWhdRgq/VMZ/PR9bIpfqG19mWdfF/sI/rBM
         C6+Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708595453; x=1709200253;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/EP+k7IchCG5AUxEyx665O9ED9ZcjaJbrg7Qwar7VTs=;
        b=IZs4eYzafhkHD/oNruZEEtml49cUDm3UeR9vSzDjnIa7jAwyEklaGBH2f6TzQ/mv7W
         NqjT1vhgNT3sKmHIeQ1c0eYKbRrWE2nZcFOEG9109vX/19/E5XCDRxqGhI5I2Bl+y7gY
         WX1pXHXBq7R61N9xBGLWZ2BDIhm/+y6/Fj7zXhpse8yMWPDt2KLKytiXjYtthkZDVGQv
         P4UsgNn38M6uvDjL05esjyNart2MB46AqKekUL1wQtp+dGv+AG1rXjty2/p2njeih507
         /ja+aU6ZcjEhIiPQmHIjTlNok66Et1ieUge2A95sxzfvdpxfzM4Zx8xHGjrkB3i9yli4
         U96w==
X-Forwarded-Encrypted: i=1; AJvYcCVRuxtiraMylPZQB8JpVxsOCx0wsjjCf5lnvsZ4TJYDBddFfLTyZfzkyYyfVe2Mfv64zvwoZOWM02iD4AMcCrwerRx1JE+Inb5bDbLDJN4=
X-Gm-Message-State: AOJu0Yw5S7meng/72F/3rxbis6Psufu+CMLuZxV4ue5qR7DBkI9Wh3Yf
	r/0cSPi2J6U+evhwHi31flVXIVpf5zEm7jEPyMO/gkrXgPy1+/cIvmITMqcgWw==
X-Google-Smtp-Source: AGHT+IHW5ZOTMp+9xCGM1j/AxW7tmsK/1xvFIR/CmMt32p029fbweCpPbpPQENaKciF4xqvU5NyDNw==
X-Received: by 2002:a05:6402:6d5:b0:564:7bff:846e with SMTP id n21-20020a05640206d500b005647bff846emr6801868edy.3.1708595453131;
        Thu, 22 Feb 2024 01:50:53 -0800 (PST)
Message-ID: <40ff2158-78aa-478b-b591-02bbcb1bf88f@suse.com>
Date: Thu, 22 Feb 2024 10:50:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/6] nestedsvm: Disable TscRateMSR
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-5-george.dunlap@cloud.com>
 <c5f3a78c-4a2b-499b-93bb-11f60b0afcc0@suse.com>
 <CA+zSX=acH-4B01=vUBstAbsNBMVgcRe_A5v0eXn66BdsuPtgiA@mail.gmail.com>
 <74ef7b2b-b105-4060-afaa-73ec272a7449@suse.com>
 <CA+zSX=ZPKScDQO6XXGOp6gv1arEs4-iWTgX8NSezYo4LGqkoLA@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CA+zSX=ZPKScDQO6XXGOp6gv1arEs4-iWTgX8NSezYo4LGqkoLA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.02.2024 10:30, George Dunlap wrote:
> On Wed, Feb 21, 2024 at 6:52 PM Jan Beulich <jbeulich@suse.com> wrote:
>>>> But then of course Andrew may know of reasons why all of this is done
>>>> in calculate_host_policy() in the first place, rather than in HVM
>>>> policy calculation.
>>>
>>> It sounds like maybe you're confusing host_policy with
>>> x86_capabilities?  From what I can tell:
>>>
>>> *  the "basic" cpu_has_X macros resolve to boot_cpu_has(), which
>>> resolves to cpu_has(&boot_cpu_data, ...), which is completely
>>> independent of the cpu-policy.c:host_cpu_policy
>>>
>>> * cpu-policy.c:host_cpu_policy only affects what is advertised to
>>> guests, via {pv,hvm}_cpu_policy and featureset bits.  Most notably a
>>> quick skim doesn't show any mechanism by which host_cpu_policy could
>>> affect what features Xen itself decides to use.
>>
>> I'm not mixing the two, no; the two are still insufficiently disentangled.
>> There's really no reason (long term) to have both host policy and
>> x86_capabilities. Therefore I'd prefer if new code (including a basically
>> fundamental re-write as is going to be needed for nested) to avoid
>> needlessly further extending x86_capabilities. Unless of course there's
>> something fundamentally wrong with eliminating the redundancy, which
>> likely Andrew would be in the best position to point out.
> 
> So I don't know the history of how things got to be the way they are,
> nor really much about the code but what I've gathered from skimming
> through while creating this patch series.  But from that impression,
> the only issue I really see with the current code is the confusing
> naming.  The cpufeature.h code has this nice infrastructure to allow
> you to, for instance, enable or disable certain bits on the
> command-line; and the interface for querying all the different bits of
> functionality is all nicely put in one place.  Moving the
> svm_feature_flags into x86_capabilities would immediately allow SVM to
> take advantage of this infrastructure; it's not clear to me how this
> would be "needless".
> 
> Furthermore, it looks to me like host_cpu_policy is used as a starting
> point for generating pv_cpu_policy and hvm_cpu_policy, both of which
> are only used for guest cpuid generation.  Given that the format of
> those policies is fixed, and there's a lot of "copy this bit from the
> host policy wholesale", it seems like no matter what, you'd want a
> host_cpu_policy.
> 
> And in any case -- all that is kind of moot.  *Right now*,
> host_cpu_policy is only used for guest cpuid policy creation; *right
> now*, the nested virt features of AMD are handled in the
> host_cpu_policy; *right now*, we're advertising to guests bits which
> are not properly virtualized; *right now* these bits are actually set
> unconditionally, regardless of whether they're even available on the
> hardware; *right now*, Xen uses svm_feature_flags to determine its own
> use of TscRateMSR; so *right now*, removing this bit from
> host_cpu_policy won't prevent Xen from using TscRateMSR itself.
> 
> (Unless my understanding of the code is wrong, in which case I'd
> appreciate a correction.)

There's nothing wrong afaics, just missing at least one aspect: Did you
see all the featureset <-> policy conversions in cpu-policy.c? That (to
me at least) clearly is a sign of unnecessary duplication of the same
data. This goes as far as seeding the host policy from the raw one, just
to then immediately run x86_cpu_featureset_to_policy(), thus overwriting
a fair part of what was first taken from the raw policy. That's necessary
right now, because setup_{force,clear}_cpu_cap() act on
boot_cpu_data.x86_capability[], not the host policy.

As to the "needless" further up, it's only as far as moving those bits
into x86_capability[] would further duplicate information, rather than
(for that piece at least) putting them into the policies right away. But
yes, if the goal is to have setup_{force,clear}_cpu_cap() be able to
control those bits as well, then going the intermediate step would be
unavoidable at this point in time.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 09:56:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 09:56:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684294.1064040 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd5oL-0000Ra-5O; Thu, 22 Feb 2024 09:56:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684294.1064040; Thu, 22 Feb 2024 09:56:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd5oL-0000RT-1q; Thu, 22 Feb 2024 09:56:13 +0000
Received: by outflank-mailman (input) for mailman id 684294;
 Thu, 22 Feb 2024 09:56:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MkW+=J7=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rd5oJ-0000RL-MV
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 09:56:11 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9b0c1feb-d168-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 10:56:10 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id E273221FCB;
 Thu, 22 Feb 2024 09:56:09 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8C31E13A8C;
 Thu, 22 Feb 2024 09:56:09 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id eJ94IDka12WGBgAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 22 Feb 2024 09:56:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b0c1feb-d168-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708595770; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=bhBvpF6rC4GRr7UEGIKWfHyqFYFYRBZM7h4JhhQ9DLk=;
	b=U07aJ8QyP+Eju4qSrerUHLKB9WEdVG3Qz1Ndf8KVkEbUkat8VACe07A3kd8alH0fMP9vun
	T+ibQCW3q4RUfY6drtMI4FSVgmBVjBSbW975OlaRruahjgYXVQ7gaInW/t/q7+oxcG5LeO
	NP6GUIRAHWIUrY69s5cxci4+KClV8Lg=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708595769; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=bhBvpF6rC4GRr7UEGIKWfHyqFYFYRBZM7h4JhhQ9DLk=;
	b=Mm9JU0slxnfX90szfxYw1tyUgAObiMHxtCvKakkObic+4HEoA1rhlCYuDsJbGqamvzJYXL
	0fi1m1EtBRV89lSkZFnpy+VmhBg+RdDaK3LNB++L7HJD566URheT2am7eBXHzL2dkqDlCR
	UrzfrHIO3ngGzRL02Pus6RwGHrsDmYw=
Message-ID: <2e02fe28-d964-46fe-b5ba-4b5e0998979e@suse.com>
Date: Thu, 22 Feb 2024 10:56:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Stats on Xen tarball downloads
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Julien Grall <julien@xen.org>
Cc: George Dunlap <george.dunlap@cloud.com>, Jan Beulich <jbeulich@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>, committers@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>
References: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
 <ZdMt88BSNMYtq39n@macbook> <36229344-e2e2-46e1-839b-b7b3b0843f23@suse.com>
 <CA+zSX=YvTgEMC0S5OqSFvxfVDeruiO=FphyVx5fv75oT-QW01A@mail.gmail.com>
 <0936a6c5-bac2-4de2-9155-1722d3bb28ca@xen.org> <ZdcYo3sGHVYaP6kM@macbook>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <ZdcYo3sGHVYaP6kM@macbook>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------6w0HBUR4SwheZVelV9f7w0eb"
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=Mm9JU0sl
X-Spamd-Result: default: False [-6.20 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 HAS_ATTACHMENT(0.00)[];
	 DWL_DNSWL_LOW(-1.00)[suse.com:dkim];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[];
	 MIME_UNKNOWN(0.10)[application/pgp-keys]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: E273221FCB
X-Spam-Level: 
X-Spam-Score: -6.20
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------6w0HBUR4SwheZVelV9f7w0eb
Content-Type: multipart/mixed; boundary="------------0JYDU4o3Bg3NJ1vFNpOXoia0";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Julien Grall <julien@xen.org>
Cc: George Dunlap <george.dunlap@cloud.com>, Jan Beulich <jbeulich@suse.com>,
 Xen-devel <xen-devel@lists.xenproject.org>, committers@xenproject.org,
 Kelly Choi <kelly.choi@cloud.com>
Message-ID: <2e02fe28-d964-46fe-b5ba-4b5e0998979e@suse.com>
Subject: Re: Stats on Xen tarball downloads
References: <CA+zSX=aEdcF-D3PUQN=1V2puz0AEMiLn4-uQFGrLWf7EYSZ7WA@mail.gmail.com>
 <ZdMt88BSNMYtq39n@macbook> <36229344-e2e2-46e1-839b-b7b3b0843f23@suse.com>
 <CA+zSX=YvTgEMC0S5OqSFvxfVDeruiO=FphyVx5fv75oT-QW01A@mail.gmail.com>
 <0936a6c5-bac2-4de2-9155-1722d3bb28ca@xen.org> <ZdcYo3sGHVYaP6kM@macbook>
In-Reply-To: <ZdcYo3sGHVYaP6kM@macbook>
Autocrypt-Gossip: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJ3BBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AAIQkQoDSui/t3IH4WIQQ+pJkfkcoLMCa4X6CgNK6L+3cgfgn7AJ9DmMd0SMJE
 ePbc7/m22D2v04iu7ACffXTdZQhNl557tJuDXZSBxDmW/tLOwU0EWTecRBAIAIK5OMKMU5R2
 Lk2bbjgX7vyQuCFFyKf9rC/4itNwhYWFSlKzVj3WJBDsoi2KvPm7AI+XB6NIkNAkshL5C0kd
 pcNd5Xo0jRR5/WE/bT7LyrJ0OJWS/qUit5eNNvsO+SxGAk28KRa1ieVLeZi9D03NL0+HIAtZ
 tecfqwgl3Y72UpLUyt+r7LQhcI/XR5IUUaD4C/chB4Vq2QkDKO7Q8+2HJOrFIjiVli4lU+Sf
 OBp64m//Y1xys++Z4ODoKh7tkh5DxiO3QBHG7bHK0CSQsJ6XUvPVYubAuy1XfSDzSeSBl//C
 v78Fclb+gi9GWidSTG/4hsEzd1fY5XwCZG/XJJY9M/sAAwUH/09Ar9W2U1Qm+DwZeP2ii3Ou
 14Z9VlVVPhcEmR/AFykL9dw/OV2O/7cdi52+l00reUu6Nd4Dl8s4f5n8b1YFzmkVVIyhwjvU
 jxtPyUgDOt6DRa+RaDlXZZmxQyWcMv2anAgYWGVszeB8Myzsw8y7xhBEVV1S+1KloCzw4V8Z
 DSJrcsZlyMDoiTb7FyqxwQnM0f6qHxWbmOOnbzJmBqpNpFuDcz/4xNsymJylm6oXiucHQBAP
 Xb/cE1YNHpuaH4SRhIxwQilCYEznWowQphNAbJtEKOmcocY7EbSt8VjXTzmYENkIfkrHRyXQ
 dUm5AoL51XZljkCqNwrADGkTvkwsWSvCSQQYEQIACQUCWTecRAIbDAAKCRCgNK6L+3cgfuef
 AJ9wlZQNQUp0KwEf8Tl37RmcxCL4bQCcC5alCSMzUBJ5DBIcR4BY+CyQFAs=

--------------0JYDU4o3Bg3NJ1vFNpOXoia0
Content-Type: multipart/mixed; boundary="------------my0Avvjrsjnt4pg8df8a0JI5"

--------------my0Avvjrsjnt4pg8df8a0JI5
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMjIuMDIuMjQgMTA6NDksIFJvZ2VyIFBhdSBNb25uw6kgd3JvdGU6DQo+IE9uIFdlZCwg
RmViIDIxLCAyMDI0IGF0IDEwOjUzOjQ5UE0gKzAwMDAsIEp1bGllbiBHcmFsbCB3cm90ZToN
Cj4+IEhpIEdlb3JnZSwNCj4+DQo+PiBPbiAyMS8wMi8yMDI0IDAyOjU1LCBHZW9yZ2UgRHVu
bGFwIHdyb3RlOg0KPj4+IE9uIE1vbiwgRmViIDE5LCAyMDI0IGF0IDY6MzjigK9QTSBKYW4g
QmV1bGljaCA8amJldWxpY2hAc3VzZS5jb20+IHdyb3RlOg0KPj4+Pg0KPj4+PiBPbiAxOS4w
Mi4yMDI0IDExOjMxLCBSb2dlciBQYXUgTW9ubsOpIHdyb3RlOg0KPj4+Pj4gT24gTW9uLCBG
ZWIgMTksIDIwMjQgYXQgMDY6MDE6NTRQTSArMDgwMCwgR2VvcmdlIER1bmxhcCB3cm90ZToN
Cj4+Pj4+PiBPbmUgb2YgdGhlIHF1ZXN0aW9ucyB3ZSBoYWQgd2l0aCByZXNwZWN0IHRvIGNo
YW5naW5nIG91ciByZWxlYXNlDQo+Pj4+Pj4gcHJhY3RpY2UgKGZvciBpbnN0YW5jZSwgbWFr
aW5nIHRoZSBwcm9jZXNzIG1vcmUgbGlnaHQtd2VpZ2h0IHNvIHRoYXQNCj4+Pj4+PiB3ZSBj
b3VsZCBkbyBhIHBvaW50IHJlbGVhc2UgYWZ0ZXIgZXZlcnkgWFNBKSB3YXMsICJIb3cgbWFu
eSBwZW9wbGUgYXJlDQo+Pj4+Pj4gYWN0dWFsbHkgdXNpbmcgdGhlIHRhcmJhbGxzPyINCj4+
Pj4+DQo+Pj4+PiBXaGF0IHdvdWxkIHRoaXMgbW9yZSBsaWdodHdlaWdodCBwcm9jZXNzIGlu
dm9sdmUgZnJvbSBhIGRvd25zdHJlYW0NCj4+Pj4+IFBvVj8gIElPVzogaW4gd2hhdCB3b3Vs
ZCB0aGUgY29udGVudHMgb2YgdGhlIHRhcmJhbGwgY2hhbmdlIGNvbXBhcmVkDQo+Pj4+PiB0
byB0aGUgY3VycmVudCByZWxlYXNlcz8NCj4+Pj4NCj4+Pj4gICBGcm9tIGFsbCBwcmlvciBk
aXNjdXNzaW9uIG15IGNvbmNsdXNpb24gd2FzICJubyB0YXJiYWxsIGF0IGFsbCIuDQo+Pj4N
Cj4+PiBPciBhdCB2ZXJ5IGxlYXN0LCB0aGUgdGFyYmFsbCB3b3VsZCBiZSBhIHNpbXBsZSBg
Z2l0IGFyY2hpdmVgIG9mIGENCj4+PiByZWxlYXNlIHRhZy4gICBSaWdodCBub3cgdGhlIHRh
cmJhbGwgY3JlYXRpb24gaGFzIGEgbnVtYmVyIG9mDQo+Pj4gYW5ub3lpbmdseSBtYW51YWwg
cGFydHMgYWJvdXQgaXQuDQo+PiBBdCB0aGUgbW9tZW50IHdlIGhhdmUgdGhlIGZvbGxvd2lu
ZyBzdGVwczoNCj4+DQo+PiAxKSBDaGVja291dCB0YWcNCj4+IDIpIENyZWF0ZSB0aGUgdGFy
YmFsbA0KPj4gMykgQ2hlY2sgdGhlIHNvdXJjZSB0YXJiYWxsIGNhbiBidWlsZA0KPj4gNCkg
U2lnbiB0aGUgdGFyYmFsbA0KPj4gNSkgVXBsb2FkIGl0DQo+Pg0KPj4gSSBtYW5hZ2VkIHRv
IHNjcmlwdCBpdCBzbyBJIGhhdmUgb25seSB0d28gY29tbWFuZHMgdG8gZXhlY3V0ZSAobW9z
dGx5DQo+PiBiZWNhdXNlIEkgYnVpbGQgYW5kIHNpZ24gb24gYSBkaWZmZXJlbnQgaG9zdCku
DQo+Pg0KPj4gQUZBSVUsIHlvdXIgY29tbWFuZCAnZ2l0IGFyY2hpdmUnIHdpbGwgb25seSBy
ZXBsYWNlIDIuIEFtIEkgY29ycmVjdD8gSWYgc28sDQo+PiBpdCBpcyBub3QgZW50aXJlbHkg
Y2xlYXIgaG93IHlvdXIgcHJvcG9zYWwgaXMgZ29pbmcgdG8gbWFrZSBpdCBiZXR0ZXIuDQo+
IA0KPiBJTU8gYnVpbGRpbmcgZm9yIHJlbGVhc2UgdGFyYmFsbHMgaXMgZWFzaWVyIHRoYW4g
ZnJvbSBhIGdpdCBjaGVja291dA0KPiAob3IgYXJjaGl2ZSkuICBJdCdzIGEgYml0IGFubm95
aW5nIHRvIGhhdmUgdG8gcHJlLWRvd25sb2FkIHRoZQ0KPiBleHRlcm5hbCBwcm9qZWN0IHNv
dXJjZXMsIG5vdyBldmVuIG1vcmUgYXMgUUVNVSBpcyB1c2luZyBnaXQNCj4gc3VibW9kdWxl
cy4NCj4gDQo+IE1vc3QgZGlzdHJvIGJpbmFyeSBidWlsZGVycyBoYXZlIGluZnJhc3RydWN0
dXJlIHRvIGRlYWwgd2l0aCBhbGwgdGhpcywNCj4gYnV0IHJlcXVpcmVzIGEgYml0IG1vcmUg
bG9naWMgaW4gdGhlIHJlY2lwZSB0aGFuIGEgcGxhaW4ganVzdCBmZXRjaCBhDQo+IHRhcmJh
bGwgYW5kIGJ1aWxkIGZyb20gaXQuDQoNCkkgaGF2ZSBhbiB1bmZpbmlzaGVkIHBhdGNoIHNl
cmllcyBseWluZyBhcm91bmQgZG9pbmcgdGhlIGRvd25sb2FkIHN0ZXBzDQpfYmVmb3JlXyBz
dGFydGluZyB0aGUgYnVpbGQuIFRoaXMgaW5jbHVkZXMgbWFrZSB0YXJnZXRzIGZvciBkb3du
bG9hZGluZw0KdGhlIHJlcXVpcmVkIGNvbXBvbmVudHMsIG9yIGFsbCBjb21wb25lbnRzIGlm
IGNvbmZpZ3VyZSBzaG91bGQgYmUgY2FsbGVkDQphZnRlcndhcmRzLg0KDQpDcmVhdGluZyB0
aGUgdGFyYmFsbCBhZnRlciBoYXZpbmcgZG93bmxvYWRlZCBhbGwgY29tcG9uZW50cyBpcyB0
cml2aWFsLg0KDQpUaGVyZSBhcmUgYSBmZXcgYnVncyBpbiB0aGUgc2VyaWVzIEkgZGlkbid0
IGhhdmUgdGltZSB5ZXQgdG8gZml4LiBJZiBzb21lb25lDQppcyBpbnRlcmVzdGVkIGluIHdv
cmtpbmcgb24gaXQsIEkgY2FuIHBvc3QgdGhlIHNlcmllcy4NCg0KDQpKdWVyZ2VuDQoNCg==

--------------my0Avvjrsjnt4pg8df8a0JI5
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------my0Avvjrsjnt4pg8df8a0JI5--

--------------0JYDU4o3Bg3NJ1vFNpOXoia0--

--------------6w0HBUR4SwheZVelV9f7w0eb
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXXGjkFAwAAAAAACgkQsN6d1ii/Ey9D
SQf/bBGRDg4IfNts7O/O3/IjATd1gE+u40uohi0T5DJCo0Wkbk1/LDSrV1O7XH8fM5rnBUDp1V49
cawYUwSzQ0pwQ8jhiEpy7R3OenX0/OGE1S+Ew2KsWLiTxnt3VhVkFYmI7cQ08tRzFTsukYPnaTUV
S4iuWS612B4YjxJm/ucDjHsn5o0AhLYBm4x6QjQrjowuRGi7C0fwKsEmmRNojNw7QF0+NEnKceCf
yOECU+JYAuTSo4re8uJhNeWDyI6eCt6Uv0yhv93ZVJgF0UQo6GLp2BUNiRfyK8xyBx8rUtt9UGTP
74+9hzqPIa9wA9C/bb16UDh5hjCm4ZNI4GGtwxRNHQ==
=Pi3o
-----END PGP SIGNATURE-----

--------------6w0HBUR4SwheZVelV9f7w0eb--


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 10:06:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 10:06:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684299.1064051 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd5yd-0002Tk-5i; Thu, 22 Feb 2024 10:06:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684299.1064051; Thu, 22 Feb 2024 10:06:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd5yd-0002Td-0W; Thu, 22 Feb 2024 10:06:51 +0000
Received: by outflank-mailman (input) for mailman id 684299;
 Thu, 22 Feb 2024 10:06:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rd5yb-0002TV-Kq
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 10:06:49 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1747582f-d16a-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 11:06:48 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-55f279dca99so2004147a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 02:06:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 x3-20020aa7d383000000b005646c7b9712sm3887432edq.16.2024.02.22.02.06.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 02:06:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1747582f-d16a-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708596408; x=1709201208; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jr6cF09++WzpyKRiizw7qNFbitKB6CkyWilkhKAB7Lw=;
        b=gxCX6N+k+Ty7qj6lsBrYcEMqLYK73cLMVfJeo5lufpjL6ujGSYHL8INXjYEi0NyU9u
         CVYSv3a43KNvN0jWEj9oNKz2LcYJ0ffnbnVYTlhu5dWzcJA9dchL7QEq1Ofr4tWIFCCH
         Ku/fTuLE2Im7T0N/06ehlsphXzLzPUnjOrER2IQoxCJXIK5c2txNlBfgSGzAFhvVKLRh
         Ae5ht7kW40ZC619kI7+A/senPN3B1i0Jl/xo7yZQezFPcPuK6cf8MbcYKNxWAKgAjrGz
         zUTbQk+jZdU4093Q9ddX2u6YssgpZ+bxXsZG/whpTsg9wss5oO3nd//Ggm4Bzrs9YDQW
         k4/w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708596408; x=1709201208;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jr6cF09++WzpyKRiizw7qNFbitKB6CkyWilkhKAB7Lw=;
        b=xJccG/E9bheQ2YFPd/oLApLUtBFhh7DZ3rQOBAxuulA/lUXyN6y+4tM8k5R10NzS27
         jrwbdILIHK2H5MWiNqbEaq9UgaWJu6cY92VMjtGqu/TC53cBRaXEVTY4qAdbg3Wegydo
         NkH2bOYL7qyvSrW3RcJmH+KUwBB2FdXpxHT6aORT1qpxvxfIjIzODUuxEUv/UjUnyGyX
         vRcPM1hmLuqRZA1TjiJQs9M/EQ5U92XuVaPafKSkIhv35xeMH0HjgvdPithvK2rfR8v0
         9L4j2AHVn71NDHJjUzNUUZdmT7s3lV3TJGjYcmfuxXOQQkLVnjVX4efTlMjs+KEakwiU
         3a8A==
X-Forwarded-Encrypted: i=1; AJvYcCXaLrXNvcUqfWkSMrH7B+QPRfjA3PZijkldi02ytJXaQ6gWJYI/t20srZumhlXFUtPnpH1tUitk+Pdh+7bPcGghBqMIWSBnvcdsVu0WUNY=
X-Gm-Message-State: AOJu0YwuLiej+R+L4pxDenf+H3K08kcOW5vmJG+RgcXnTorG3MGnWocs
	7+BeDOI/nEFJLDZY8Q1UjCzaNXovpl9p7eXfv0S6P5Xdsq0gSW5Ey/k27hgUiw==
X-Google-Smtp-Source: AGHT+IFRc9oXLanoTt8ZFK0yLdYL9uWAYn76sOlvgQ8KvQ5NpLqhlldSAgaxKw3iLtSwObZDPYUD6g==
X-Received: by 2002:a05:6402:351:b0:565:1a91:94ad with SMTP id r17-20020a056402035100b005651a9194admr1928130edw.22.1708596407833;
        Thu, 22 Feb 2024 02:06:47 -0800 (PST)
Message-ID: <d8939069-a1cb-4794-a6aa-75a3e4d97884@suse.com>
Date: Thu, 22 Feb 2024 11:06:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] x86/memsharing: use an atomic add instead of a
 cmpxchg loop
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>,
 Tamas K Lengyel <tamas@tklengyel.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-2-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240222090530.62530-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.02.2024 10:05, Roger Pau Monne wrote:
> The usage of a cmpxchg loop in get_next_handle() is unnecessary, as the same
> can be achieved with an atomic increment, which is both simpler to read, and
> avoid any need for a loop.
> 
> The cmpxchg usage is likely a remnant of 32bit support, which didn't have an
> instruction to do an atomic 64bit add, and instead a cmpxchg had to be used.
> 
> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
albeit ...

> --- a/xen/arch/x86/mm/mem_sharing.c
> +++ b/xen/arch/x86/mm/mem_sharing.c
> @@ -179,13 +179,7 @@ static void mem_sharing_page_unlock(struct page_info *pg)
>  
>  static shr_handle_t get_next_handle(void)
>  {
> -    /* Get the next handle get_page style */
> -    uint64_t x, y = next_handle;
> -    do {
> -        x = y;
> -    }
> -    while ( (y = cmpxchg(&next_handle, x, x + 1)) != x );
> -    return x + 1;
> +    return arch_fetch_and_add(&next_handle, 1) + 1;
>  }

... the adding of 1 here is a little odd when taken together with
next_handle's initializer. Tamas, you've not written that code, but do
you have any thoughts towards the possible removal of either the
initializer or the adding here? Plus that variable of course could
very well do with moving into this function.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 10:10:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 10:10:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684305.1064060 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd62c-0004AF-Jg; Thu, 22 Feb 2024 10:10:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684305.1064060; Thu, 22 Feb 2024 10:10:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd62c-0004A8-GH; Thu, 22 Feb 2024 10:10:58 +0000
Received: by outflank-mailman (input) for mailman id 684305;
 Thu, 22 Feb 2024 10:10:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rd62b-0004A2-Do
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 10:10:57 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ab3218bf-d16a-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 11:10:56 +0100 (CET)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a3e550ef31cso165303366b.3
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 02:10:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m12-20020a170906160c00b00a3d0a094574sm5776908ejd.66.2024.02.22.02.10.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 02:10:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ab3218bf-d16a-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708596656; x=1709201456; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=nqfegFgQFplvZynzmmWeCSQTM2EV90XkHsxyRQ1eDf0=;
        b=TkRGuabBYq0zUVPOziQhkmz6VBpUu849Yh8+cmq999F2DeKxcYGTwB/pflIEmXFH6X
         1LV55TbICWuv8F4hcncTyurRE4+xR2RkPNm/xCp/xa6BYCflDdfIzFG8IzVZ4VgpKmkI
         /HoNGn9z48+7cmUtHxSofR5rQ5po4o+tKeJr0bUhBbGlt3Pz685Y7ZI0nUfLbNtLasMo
         5d0iggW23lVtH8dvC6UYkWzHYB0Rtg/ChzHdyMj+Hhvb9axUlUhS48ZGQvqAhD/IaZwj
         l6LUg2eE4COaxEbQfO6RsOGUkCvQBMPJYhGSfmuXfYN+0nJ1b1z0iM2ZqWvfH6EmKTDO
         Od8g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708596656; x=1709201456;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=nqfegFgQFplvZynzmmWeCSQTM2EV90XkHsxyRQ1eDf0=;
        b=GtR8uV8H64BLjoVFpelvWvE83qGPt7q3oIsa1PjuYsBElfJ6COLMphVFblzza2nBGw
         wBGlx2tE0wKrkWcfjqs1+W7NcrJvzc12UbCT7TmkV5Q5Ml72zJwnIC+C2l++tNqyVUld
         auMU9BqsEkATZBoYNmN4/bmsfcrfuIJ4bWh3CPxgDgE311Q41nG5ATwzjM3jzf7t/Izi
         vcZhEWB8Kw37NE3KHy+I6BCsuydfz3EozFGRNOHJafBN8JoAT4jaB0nb4PyxJ2wzGpyV
         Gf16Rlh7UnmBlQT8fctujJVYyUuX8MUckPzZmFvzhueNuLLX8PZWQ8CaWhevQEfZ9cCB
         zO2A==
X-Forwarded-Encrypted: i=1; AJvYcCV1JMB/PE2v97TxcdiXWKpqZ/j7z2mtOtN/j7AtNqgz5Vvyfosi2tlpiQLsC2XL+xpTs7AnvC+t3L2Hs4z/v9qcXP5evx9WnaNocZg8ekw=
X-Gm-Message-State: AOJu0Yz2R8vFwLVOoUv+kIpiNsRm3JCgyJkXzVb8WRUTTD/gCD96GUKy
	+Y9yF3+B8jrgsk4PrySb0tYtXPv8G2pyjJM9h89YINSemtY5m0VZZcdO0FJWzg==
X-Google-Smtp-Source: AGHT+IFYPi2B9ry2vTe6Wv7cu8R4GmHVtb7g/vMCNahOOCm26CiN6dbX+LS62/w/Z3ZQ+GeixxcI5g==
X-Received: by 2002:a17:906:46ce:b0:a3e:42cf:f6ae with SMTP id k14-20020a17090646ce00b00a3e42cff6aemr7289413ejs.19.1708596656056;
        Thu, 22 Feb 2024 02:10:56 -0800 (PST)
Message-ID: <d37335f7-ae3c-4dc8-854a-625df275f5da@suse.com>
Date: Thu, 22 Feb 2024 11:10:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] x86/hpet: use an atomic add instead of a cmpxchg loop
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-3-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240222090530.62530-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.02.2024 10:05, Roger Pau Monne wrote:
> The usage of a cmpxchg loop in hpet_get_channel() is unnecessary, as the same
> can be achieved with an atomic increment, which is both simpler to read, and
> avoid any need for a loop.
> 
> Note there can be a small divergence in the channel returned if next_channel
> overflows, but returned channel will always be in the [0, num_hpets_used)
> range, and that's fine for the purpose of balancing HPET channels across CPUs.
> This is also theoretical, as there's no system currently with 2^32 CPUs (as
> long as next_channel is 32bit width).

The code change looks good to me, but I fail to see the connection to
2^32 CPUs. So it feels like I'm missing something, in which case I'd
rather not offer any R-b.

Jan

> Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
>  xen/arch/x86/hpet.c | 6 +-----
>  1 file changed, 1 insertion(+), 5 deletions(-)
> 
> diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
> index d982b0f6b2c9..4777dc859d96 100644
> --- a/xen/arch/x86/hpet.c
> +++ b/xen/arch/x86/hpet.c
> @@ -458,11 +458,7 @@ static struct hpet_event_channel *hpet_get_channel(unsigned int cpu)
>      if ( num_hpets_used >= nr_cpu_ids )
>          return &hpet_events[cpu];
>  
> -    do {
> -        next = next_channel;
> -        if ( (i = next + 1) == num_hpets_used )
> -            i = 0;
> -    } while ( cmpxchg(&next_channel, next, i) != next );
> +    next = arch_fetch_and_add(&next_channel, 1) % num_hpets_used;
>  
>      /* try unused channel first */
>      for ( i = next; i < next + num_hpets_used; i++ )



From xen-devel-bounces@lists.xenproject.org Thu Feb 22 10:22:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 10:22:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684311.1064069 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6Dx-0006Rr-MY; Thu, 22 Feb 2024 10:22:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684311.1064069; Thu, 22 Feb 2024 10:22:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6Dx-0006Rk-Jd; Thu, 22 Feb 2024 10:22:41 +0000
Received: by outflank-mailman (input) for mailman id 684311;
 Thu, 22 Feb 2024 10:22:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=E4eK=J7=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rd6Dw-0006Re-TL
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 10:22:40 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4dd335ab-d16c-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 11:22:38 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d180d6bd32so88656451fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 02:22:38 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4dd335ab-d16c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708597358; x=1709202158; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nPki2FRl4cYglDmI6UvFuJHMZb7WqNcD7CgJmC/747Q=;
        b=ajMxWy0/nAdJ5MwEK/lrxmmB04334eDKMtWvuglVogCReqDKAVax5qhCel2HT1Knvx
         TPb8hjFaTXsmKVZ0oa5Jbrwr5irpWR1jlqq0gb+rynPeGkfwZFausUqYTFIqtdAin7CG
         voHP8AMzjsJYjuoFp1Lh3E0sIY//1sJf8XcmQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708597358; x=1709202158;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=nPki2FRl4cYglDmI6UvFuJHMZb7WqNcD7CgJmC/747Q=;
        b=pZjM4PoFMInrhhQvMBPf++CAYlyl6abVGzLA6Ngb+hIOfyakWGgihVxFD2yb+m/AOk
         G9nmV3C0/p6yWBtod9PuD4iGwSLSjTr6j/ge3U2TeG5RrhamCO1PD+aJ6vtTrpct9gN1
         SbbjPAmEPvXGeIygkIA2YkAIKublmpIfR0QhqGkkhgu1y5AKXWwpJClVbjIGDdyAEFrD
         uUPFQAAXkJb22I0j5OZrjdmLARXsX7yYGzd1vifl6YBYVB+juaNXqg0t+bQ3OsZvWosp
         es8ZQFwBOOoCXsKCE9nzfqP4f/McSh+5qz36vwZyvU6oex5jOVRWqkMJhapPJGr3Kujo
         A1MQ==
X-Forwarded-Encrypted: i=1; AJvYcCVbitw97yhuQrVlyiEjlQX9Gzk6bn/WhQea2G1q/bLzKdrHnqfJmXLU5tg2PqV4Uz8eFyyji7xlZFp8v2YJO0awtGdmHk8T5AqlDxAE1sg=
X-Gm-Message-State: AOJu0YxateENTBxNBtUgzMaDpMmaMxhdvliRrHC50/w+8qmoSycwp6gR
	gRLa9yepBYfSNMylT3TRKs5vR+xdl43X0/k1wnh4+SbWlg68qjsR0QGtOiIeIO1cod5dOb6ZouW
	Y/SdOip8R+p1wcZPdeHuLF2E4KLE26uoxMe5r8mU6BzckhxCR
X-Google-Smtp-Source: AGHT+IHu9b79I7PtKnmK849UopJkbaTTyB6PvraeFfpanfMU6TlvwcJR/7+mqaHZmK7LgiEkgV5LmkKlMliFbxgWkC8=
X-Received: by 2002:a2e:b70e:0:b0:2d2:3e9c:18db with SMTP id
 j14-20020a2eb70e000000b002d23e9c18dbmr6999960ljo.14.1708597358331; Thu, 22
 Feb 2024 02:22:38 -0800 (PST)
MIME-Version: 1.0
References: <osstest-184722-mainreport@xen.org> <6d6646ce-be79-4d7c-bca6-0d44262d30b5@suse.com>
In-Reply-To: <6d6646ce-be79-4d7c-bca6-0d44262d30b5@suse.com>
From: Anthony PERARD <anthony.perard@cloud.com>
Date: Thu, 22 Feb 2024 10:22:27 +0000
Message-ID: <CAKoJhMRdZ1mJucuHOdBfM1VvpaQFjcEEtEBZzfjUXaD6Hcv0YQ@mail.gmail.com>
Subject: Re: [linux-linus test] 184722: regressions - FAIL
To: Juergen Gross <jgross@suse.com>
Cc: osstest service owner <osstest-admin@xenproject.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 22, 2024 at 8:28=E2=80=AFAM Juergen Gross <jgross@suse.com> wro=
te:
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> >   build-arm64-pvops             6 kernel-build   fail in 184721 REGR. v=
s. 184719
>
> Log says:
>
> gcc: internal compiler error: Segmentation fault signal terminated progra=
m cc1
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See <file:///usr/share/doc/gcc-8/README.Bugs> for instructions.
> make[5]: *** [scripts/Makefile.build:243: drivers/iio/adc/max9611.o] Erro=
r 4
> make[5]: *** Waiting for unfinished jobs....

Yeah, that happens from time to time. My plan to fix this is to
upgrade to a newer version of Debian, it just takes time.

Cheers,

--=20
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 10:32:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 10:32:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684316.1064080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6NI-0008Qe-KX; Thu, 22 Feb 2024 10:32:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684316.1064080; Thu, 22 Feb 2024 10:32:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6NI-0008QX-G2; Thu, 22 Feb 2024 10:32:20 +0000
Received: by outflank-mailman (input) for mailman id 684316;
 Thu, 22 Feb 2024 10:32:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rd6NH-0008QR-Rj
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 10:32:19 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a647c878-d16d-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 11:32:16 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a3d01a9a9a2so156913366b.1
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 02:32:16 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 jj12-20020a170907984c00b00a3e64bcd2c1sm4610852ejc.142.2024.02.22.02.32.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 02:32:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a647c878-d16d-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708597936; x=1709202736; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lOjwQD/mvtqgt+BJpMBjKQhb7S1DGSamUJBOXm3VbCE=;
        b=QYepXBC2s4lr+f1vLOlBL2uJJ1h5NB+x94dup/Ttj9jSJ8P5kuokgwtkIKiZSxy79Y
         dUa8h95n8LTx/qIws05vXyfRwrbQLX+ElQ6l1/sldtW/95Gf9NcUqj1TUEX3arBheZFH
         x1Exluy++GyPWU62IVUnRLQEaFRyedrCswJk7xE6HjGQSHmVjalwIC4QKKQgWtBf7cm4
         D2Hg+MDzF01QUKxWfY4FsApX7l3jcQvei/qlSWphVwdEGtI1r139fo+nFB8DUK5c9Rd+
         9LXqUvZ9sDjFzbA4NE64T+89gr9OXOjZHsjbaerxD2Txz6l97e68APvXeDHkOFoEkN87
         E8fw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708597936; x=1709202736;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=lOjwQD/mvtqgt+BJpMBjKQhb7S1DGSamUJBOXm3VbCE=;
        b=XSlHQF4jfm14pbqc8/DuZFmEc65BSFPrlGXKeFgVA24cO5INkrWJ8Z7dcSHhG4l/pO
         tDK2QKyp2K93s6uAkgCMZWsWgpc70CZea+5nY1qV+Qc8NM7BFlGYOK4Q78/YpokOgE8p
         HvfQPWcy3pRat9k8EkMJdlG+6DQVIBS81GI1PaLN7sEoo18/3i82RPvjazI40C5vCKve
         zVAk5HVzNR+Jhc3MMaJpmW2DRSOIcgo+eQQ16sRla7w0oEH7gQbY8YaOhXa1LHaJtkNS
         37CsbxLqZDnTp1MBKktJtIO38Ze2YZUc3ImXVyyaWjPu9GtfdXKT/fnzru92N3M+8d7z
         VtCw==
X-Forwarded-Encrypted: i=1; AJvYcCW1MxHGkrmWMPXIRXA3jlOwfHFU2O++X9qMeyql/EBsZeecrtwfcD4y03eST8NlHcRMqD4DIAqD3USERQHoO+SMyJ3fBw6+nnbejsrhtgM=
X-Gm-Message-State: AOJu0Yxfdy7wnbh+PPLXI19KOw2HIOUHM2idlq+ypukOVtu5AsZROP7u
	lGeFyWNPHXu3jfyWA83jLoaFdIuGhciSjwAaSr8RTqSasFOErb+iGSAZa9ISKw==
X-Google-Smtp-Source: AGHT+IFVwAs/tfw9ZhmvnHzPVQUJxPKWOEfffpywd1wLFGZfX9YZD8auVNUDMmwkvGNRFTdCWr29iQ==
X-Received: by 2002:a17:906:4f01:b0:a3f:1fd6:6474 with SMTP id t1-20020a1709064f0100b00a3f1fd66474mr4134155eju.43.1708597936257;
        Thu, 22 Feb 2024 02:32:16 -0800 (PST)
Message-ID: <c236f02c-241b-4f1d-b037-1d66a8f87d5a@suse.com>
Date: Thu, 22 Feb 2024 11:32:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/altcall: use an union as register type for function
 parameters
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240221170331.60605-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240221170331.60605-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 21.02.2024 18:03, Roger Pau Monne wrote:
> The current code for alternative calls uses the caller parameter types as the
> types for the register variables that serve as function parameters:
> 
> uint8_t foo;
> [...]
> alternative_call(myfunc, foo);
> 
> Would expand roughly into:
> 
> register unint8_t a1_ asm("rdi") = foo;
> register unsigned long a2_ asm("rsi");
> [...]
> asm volatile ("call *%c[addr](%%rip)"...);
> 
> However under certain circumstances clang >= 16.0.0 with -O2 can generate
> incorrect code,

Considering that the related (wider) ABI issue looks to also be present on
Clang5, is the more specific issue here really limited to >= 16?

> given the following example:
> 
> unsigned int func(uint8_t t)
> {
>     return t;
> }
> 
> static void bar(uint8_t b)
> {
>     int ret_;
>     register uint8_t di asm("rdi") = b;
>     register unsigned long si asm("rsi");
>     register unsigned long dx asm("rdx");
>     register unsigned long cx asm("rcx");
>     register unsigned long r8 asm("r8");
>     register unsigned long r9 asm("r9");
>     register unsigned long r10 asm("r10");
>     register unsigned long r11 asm("r11");
> 
>     asm volatile ( "call %c[addr]"
>                    : "+r" (di), "=r" (si), "=r" (dx),
>                      "=r" (cx), "=r" (r8), "=r" (r9),
>                      "=r" (r10), "=r" (r11), "=a" (ret_)
>                    : [addr] "i" (&(func)), "g" (func)
>                    : "memory" );
> }
> 
> void foo(unsigned int a)
> {
>     bar(a);
> }
> 
> Clang generates the following code:
> 
> func:                                   # @func
>         movl    %edi, %eax
>         retq
> foo:                                    # @foo
>         callq   func
>         retq
> 
> Note the truncation of the unsigned int parameter 'a' of foo() to uint8_t when
> passed into bar() is lost.
> 
> The above can be worked around by using an union when defining the register
> variables, so that `di` becomes:
> 
> register union {
>     uint8_t e;
>     unsigned long r;
> } di asm("rdi") = { .e = b };
> 
> Which results in following code generated for `foo()`:
> 
> foo:                                    # @foo
>         movzbl  %dil, %edi
>         callq   func
>         retq
> 
> So the truncation is not longer lost.

But how do you explain this behavior? I see absolutely no reason why filling
the one union field should lead to zero-extension. If I'm not mistaken the
language allows the rest of the union to retain undefined contents. So to me
this looks like you're converting something that failed to build due to a
(presumed) bug in Clang to something that any compiler would be permitted to
translate to other than what we want.

In this context note in particular that the spec distinguishes aggregates
from unions, and the clause regarding filling unmentioned fields is limited
to aggregates:

"If there are fewer initializers in a brace-enclosed list than there are
 elements or members of an aggregate, or fewer characters in a string
 literal used to initialize an array of known size than there are elements
 in the array, the remainder of the aggregate shall be initialized
 implicitly the same as objects that have static storage duration."

Which makes sense, as a union initializer shall mention a single of the
members only anyway.

All of this of course doesn't invalidate your approach as a possible
workaround, but it then needs limiting to Clang versions where we are sure
the more strict behavior than demanded by the standard actually applies.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 10:50:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 10:50:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684323.1064090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6el-0003K0-1G; Thu, 22 Feb 2024 10:50:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684323.1064090; Thu, 22 Feb 2024 10:50:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6ek-0003Jt-UD; Thu, 22 Feb 2024 10:50:22 +0000
Received: by outflank-mailman (input) for mailman id 684323;
 Thu, 22 Feb 2024 10:50:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+gCR=J7=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rd6ej-0003Jm-Kq
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 10:50:21 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2b9c4f1b-d170-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 11:50:19 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-512dc7b7fc2so628107e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 02:50:19 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 c11-20020ac86e8b000000b0042e033e5211sm4274506qtv.7.2024.02.22.02.50.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 22 Feb 2024 02:50:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b9c4f1b-d170-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708599019; x=1709203819; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=ZsRaHjr0LZ2aX1MqEL6SY1faoWh6haTUtLtlbCJdG/Q=;
        b=pukRTSeGbwowcJOV5ToMlBBFRtoGw3gl0lQTEJZzf8qbmDMLA7xWmwuPdtagaRvR9y
         S9PKqZsgFL4rp207GHQbgin+oucb55rhHsWCvcSQQTjzDR+b6TOeCyCtBIoS7dgNie0P
         BgsyPJBrjMvBbChz8lE8XyvsuZVMgLbaG/9Gc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708599019; x=1709203819;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ZsRaHjr0LZ2aX1MqEL6SY1faoWh6haTUtLtlbCJdG/Q=;
        b=hvkFhPt8jqf3bgSC8fSUswpsIkxg044YRsrhEhGgh47mRI6ZH6YWlQAL8kJUHPbMzZ
         7YL9beOHXNPuPb3JFOfSU3Rhf4/jIsNAqy2cv4qWmWMEX745TKQ/sutI4TXeP3u/c7ml
         qAA295gKIIuUNvc4gtkLjM6oVm/bFxdlv7ZvSnzv7xauODugqHYautMpU/Vjf1xT4vQj
         1eelvL62L6QW+P2JW9qquoPDscnKsROHQJZIL6SIg7hYfQUrFw63SQq6RDDRfMpPh2mZ
         Nt5xNXSp2GAMHTu56UaQwklA9bOw1NQTTQpHjibMzLYzV8MYzk9iRMvlvgFEalWMtYsB
         0RHg==
X-Forwarded-Encrypted: i=1; AJvYcCXEla2aX/o/AeakLcvTKefJECchANyOFFGZnpK9atNvraaOOzu5L537kZZS4QQIzWA4aA3biSLsrLiMf1x8XdXWHBFJvpowwU00AdtdmBw=
X-Gm-Message-State: AOJu0YxPX4mbPXUzkBZiYubUvnRy3c+LDSkm/uid8lDPl1bZFBwi2b0d
	5D+Hu2w7dTH8d/JtRipZyCu5JvHHZExXO5rTP4Xf6lLaQhE9AmVkZWSxP3CP5Ew=
X-Google-Smtp-Source: AGHT+IGSQeg3RLx6Nmp4v04FR3QUyZ87agz8WzomJsB69kiHK/wFWTB62U2NlsQWxc65PztlsHNO/A==
X-Received: by 2002:ac2:4e08:0:b0:512:b935:c542 with SMTP id e8-20020ac24e08000000b00512b935c542mr8571866lfr.59.1708599018781;
        Thu, 22 Feb 2024 02:50:18 -0800 (PST)
Date: Thu, 22 Feb 2024 11:50:16 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] x86/altcall: use an union as register type for function
 parameters
Message-ID: <Zdcm6CrINxGeKU1M@macbook>
References: <20240221170331.60605-1-roger.pau@citrix.com>
 <c236f02c-241b-4f1d-b037-1d66a8f87d5a@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <c236f02c-241b-4f1d-b037-1d66a8f87d5a@suse.com>

On Thu, Feb 22, 2024 at 11:32:14AM +0100, Jan Beulich wrote:
> On 21.02.2024 18:03, Roger Pau Monne wrote:
> > The current code for alternative calls uses the caller parameter types as the
> > types for the register variables that serve as function parameters:
> > 
> > uint8_t foo;
> > [...]
> > alternative_call(myfunc, foo);
> > 
> > Would expand roughly into:
> > 
> > register unint8_t a1_ asm("rdi") = foo;
> > register unsigned long a2_ asm("rsi");
> > [...]
> > asm volatile ("call *%c[addr](%%rip)"...);
> > 
> > However under certain circumstances clang >= 16.0.0 with -O2 can generate
> > incorrect code,
> 
> Considering that the related (wider) ABI issue looks to also be present on
> Clang5, is the more specific issue here really limited to >= 16?

No, this is wrong.  I did check clang 15.0.0 and I messed up the
output, all clang versions (on godbolt) seem to be affected.

> > given the following example:
> > 
> > unsigned int func(uint8_t t)
> > {
> >     return t;
> > }
> > 
> > static void bar(uint8_t b)
> > {
> >     int ret_;
> >     register uint8_t di asm("rdi") = b;
> >     register unsigned long si asm("rsi");
> >     register unsigned long dx asm("rdx");
> >     register unsigned long cx asm("rcx");
> >     register unsigned long r8 asm("r8");
> >     register unsigned long r9 asm("r9");
> >     register unsigned long r10 asm("r10");
> >     register unsigned long r11 asm("r11");
> > 
> >     asm volatile ( "call %c[addr]"
> >                    : "+r" (di), "=r" (si), "=r" (dx),
> >                      "=r" (cx), "=r" (r8), "=r" (r9),
> >                      "=r" (r10), "=r" (r11), "=a" (ret_)
> >                    : [addr] "i" (&(func)), "g" (func)
> >                    : "memory" );
> > }
> > 
> > void foo(unsigned int a)
> > {
> >     bar(a);
> > }
> > 
> > Clang generates the following code:
> > 
> > func:                                   # @func
> >         movl    %edi, %eax
> >         retq
> > foo:                                    # @foo
> >         callq   func
> >         retq
> > 
> > Note the truncation of the unsigned int parameter 'a' of foo() to uint8_t when
> > passed into bar() is lost.
> > 
> > The above can be worked around by using an union when defining the register
> > variables, so that `di` becomes:
> > 
> > register union {
> >     uint8_t e;
> >     unsigned long r;
> > } di asm("rdi") = { .e = b };
> > 
> > Which results in following code generated for `foo()`:
> > 
> > foo:                                    # @foo
> >         movzbl  %dil, %edi
> >         callq   func
> >         retq
> > 
> > So the truncation is not longer lost.
> 
> But how do you explain this behavior? I see absolutely no reason why filling
> the one union field should lead to zero-extension. If I'm not mistaken the
> language allows the rest of the union to retain undefined contents. So to me
> this looks like you're converting something that failed to build due to a
> (presumed) bug in Clang to something that any compiler would be permitted to
> translate to other than what we want.

Oh, right, I was expecting the compiler to zero extend it, confabulating
how unmentioned fields are initialized in structs.

However, if as mentioned in the psABI thread, the callee is required
to do any zero extension as necessary, using the union shouldn't cause
issues for compilers that implement the ABI properly.

IOW: I don't think the proposed workaround would cause issues for gcc.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 10:59:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 10:59:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684329.1064099 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6n9-00043A-Ro; Thu, 22 Feb 2024 10:59:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684329.1064099; Thu, 22 Feb 2024 10:59:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6n9-000433-P8; Thu, 22 Feb 2024 10:59:03 +0000
Received: by outflank-mailman (input) for mailman id 684329;
 Thu, 22 Feb 2024 10:59:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+gCR=J7=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rd6n8-00042x-Ga
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 10:59:02 +0000
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com
 [2a00:1450:4864:20::229])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 62f10988-d171-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 11:59:01 +0100 (CET)
Received: by mail-lj1-x229.google.com with SMTP id
 38308e7fff4ca-2d23a22233fso50312511fa.2
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 02:59:01 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 lb7-20020a056214318700b0068f11ceb309sm1379613qvb.128.2024.02.22.02.59.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 22 Feb 2024 02:59:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62f10988-d171-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708599541; x=1709204341; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=yrRVt+FqIt1XO7ECPMcpBijREWcbS9Su+cJM3EM6zJ4=;
        b=A2tPpPz2LYhvV/NdFavv9pQlqZWBw/O4G9yj14vSIk0UxHLoLRqMweyYwaX0YHOdya
         KJmH8Xq3fPRKciz3dgLU/Wv2UxmaWFDvNorAn/JrVASTaRI83+a3wCVjar+tEmOI86qa
         yevRVb9b/In9hY3fiX0AHEo0NnZamhhla3+M0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708599541; x=1709204341;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=yrRVt+FqIt1XO7ECPMcpBijREWcbS9Su+cJM3EM6zJ4=;
        b=Ijv4qQiK3oHX+BAer60dRpuGEU4HZFBLNCwyrkXOBcOGmvSIHhYgUUUfjRlsQNb2Nj
         VJ8LasCA7G7h6HallJtw+iV5rCv49ymReEYRt31CIIfiSdCAPJVUFMvLp6u5HfwB+lnB
         fDSKqVoST+D/4ETaEi9cKfqASvzgpy9OBaL5R17+Gp3Qdy/dw+8siuLBLY9pkZ00GycV
         UsygIVTeEIjnU4HD3MKRREDKwLs0/gFHvF5GOiEUH4QJOFUMlgzXiuB47RHFol5nTbnA
         hzuxjHDhZWScQtBTgQfyA9olK620hHL0jL96P7tMb4Jqc6cBdDkEiNxVo4udk26KZBfz
         bWqg==
X-Forwarded-Encrypted: i=1; AJvYcCUIpMo6rkodga3vwMVyMxH/dHiVbmEx7LBo6h0yq8AGf7QEWb4lSpSDmZKyTbQrHoGEuYxvuQYIfvosS4rAsjA68c8HMuA1u4JBMpSt7TE=
X-Gm-Message-State: AOJu0YzmeeKLa0IUtr7YxZDM4l/tgoe0fabl0U/otR3EzrYC5AWo7HWs
	CqaLqI7fFxid7hQmC9OBmfW7zi3Nv+Bv7dJvAZRLKappa0e17GoURkzp7ht3XN8=
X-Google-Smtp-Source: AGHT+IGOGez26HBWMgpC2gmOJYPEEoH2zUil/w2k/Z1ofT8g18Ynzm2Rm56n99MBDV8ZVfhf/2Xkng==
X-Received: by 2002:a05:6512:4cd:b0:512:c802:7a9 with SMTP id w13-20020a05651204cd00b00512c80207a9mr4756736lfq.38.1708599541199;
        Thu, 22 Feb 2024 02:59:01 -0800 (PST)
Date: Thu, 22 Feb 2024 11:58:58 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH 2/2] x86/hpet: use an atomic add instead of a cmpxchg loop
Message-ID: <Zdco8nMu2N4WDWwx@macbook>
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-3-roger.pau@citrix.com>
 <d37335f7-ae3c-4dc8-854a-625df275f5da@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <d37335f7-ae3c-4dc8-854a-625df275f5da@suse.com>

On Thu, Feb 22, 2024 at 11:10:54AM +0100, Jan Beulich wrote:
> On 22.02.2024 10:05, Roger Pau Monne wrote:
> > The usage of a cmpxchg loop in hpet_get_channel() is unnecessary, as the same
> > can be achieved with an atomic increment, which is both simpler to read, and
> > avoid any need for a loop.
> > 
> > Note there can be a small divergence in the channel returned if next_channel
> > overflows, but returned channel will always be in the [0, num_hpets_used)
> > range, and that's fine for the purpose of balancing HPET channels across CPUs.
> > This is also theoretical, as there's no system currently with 2^32 CPUs (as
> > long as next_channel is 32bit width).
> 
> The code change looks good to me, but I fail to see the connection to
> 2^32 CPUs. So it feels like I'm missing something, in which case I'd
> rather not offer any R-b.

The purpose of hpet_get_channel() is to distribute HPET channels
across CPUs, so that each CPU gets assigned an HPET channel, balancing
the number of CPUs that use each channel.

This is done by (next_channel++ % num_hpets_used), so the value of
next_channel after this change can be > num_hpets_used, which
previously wasn't.  This can lead to a different channel returned for
the 2^32 call to the function, as the counter would overflow.  Note
calls to the function are restricted to the number of CPUs in the
host, as per-CPU channel assignment is done only once (and the channel
is then stored in a per-cpu variable).

Feel free to adjust the commit message if you think all this is too
much data, or too confusing.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 10:59:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 10:59:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684333.1064110 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6nl-0004fz-5C; Thu, 22 Feb 2024 10:59:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684333.1064110; Thu, 22 Feb 2024 10:59:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6nl-0004fs-1S; Thu, 22 Feb 2024 10:59:41 +0000
Received: by outflank-mailman (input) for mailman id 684333;
 Thu, 22 Feb 2024 10:59:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rd6nj-0004St-HS
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 10:59:39 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 786b1307-d171-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 11:59:37 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a28a6cef709so962711766b.1
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 02:59:37 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 hu17-20020a170907a09100b00a3e6c6f2140sm4446117ejc.117.2024.02.22.02.59.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 02:59:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 786b1307-d171-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708599577; x=1709204377; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PdpM/T7oirETjJPsD87ga5uMfssPawpKdYm3AcDRhag=;
        b=WgAuoFBpoe17YpWQaXs+gZyxD4xo8uktVXVpGSfNhauiu0nHRUGFaNfzJ5jnSHlReV
         antnshXbTjHC6yaB+EdVIa3KSpIy5yRKR+rMmRh6ptPs/ew6QptZZBns0u5txWZXoydK
         6IVXaeQVNwubKiQkiIKhoGk9vtUiXr1NPAe7FBBGUL/mJS4qnGCI6Ne5pnDMq5xOIM4s
         aix4/pKAxaTWXW0NUHB3H1hNRXFRAgbwsv2wfMXY+48TD02XQEw2LYd7Oevy598BukLF
         vN0x6DnxFk+Nl52WRXjyjv2VdeI8atp9j7jCAa9zT1rPFbVuYA7T7dovNiQjaq6MNbN1
         NTBg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708599577; x=1709204377;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PdpM/T7oirETjJPsD87ga5uMfssPawpKdYm3AcDRhag=;
        b=rVOnLw1Rm4QTqgUeUeppNq7yJsCRLxVUJiuXSyfuV4G7e/r0uC6RPi1R/CwEHn4Y4G
         LqMGjFWWWJe2fPWbAIWIAw3U49ZtbfdcQnt9Bqu3W6GGHeF1jksmMMthuEAl+nhYGHsJ
         5Kjc2nA3crV8FVP9Nirw9Ts56shIb358QzI4Ycp+qp/A2CG8VqGk81O+32onqreX6+PP
         uGFADpkLTV6Q6Wl+c83/H3cum45DKc1a+CZ85+IuA3wA6XP7V3vCC2zsaiy9pIkd2dqc
         vBfa+KLLRRyEWmgDWR+0T/SBZ5GtxZ1NusC6pH6Mu13uIgvhOWGT+yK0eQusICZQEHr6
         /QhQ==
X-Forwarded-Encrypted: i=1; AJvYcCW9qTQ5yf/+R1MkSZbygG5CTOY9+C9lGXzlrINGW17LaKGaEeB8xY/aNw/OGHuNYQA073JgUfwqbyFJiYWeVxJ/sfOHz1oLzQ4jDYvcfQk=
X-Gm-Message-State: AOJu0YwahhyV8oMuv31hOFKX/kUPOpxATK0MgcD+U6ZmZgcrIpVk4Dwo
	3PuYb0EcbbrAKrK1ieEkRwaIS6MU7NuQxyzr/1TYbPkfKJ+JGG4VWz2xnP9X6Q==
X-Google-Smtp-Source: AGHT+IHw5W6RYLXt0djKTEwzsfBZdmZU5AEmZup3/2gfVvFwbekEq8RZOJX2uTjR/NZGIYCvHS/sgQ==
X-Received: by 2002:a17:906:a86:b0:a3f:97cf:c41e with SMTP id y6-20020a1709060a8600b00a3f97cfc41emr560844ejf.20.1708599577281;
        Thu, 22 Feb 2024 02:59:37 -0800 (PST)
Message-ID: <6af07ce2-7987-4ac0-a5d5-14004146aa08@suse.com>
Date: Thu, 22 Feb 2024 11:59:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/altcall: use an union as register type for function
 parameters
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240221170331.60605-1-roger.pau@citrix.com>
 <c236f02c-241b-4f1d-b037-1d66a8f87d5a@suse.com> <Zdcm6CrINxGeKU1M@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zdcm6CrINxGeKU1M@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.02.2024 11:50, Roger Pau Monné wrote:
> On Thu, Feb 22, 2024 at 11:32:14AM +0100, Jan Beulich wrote:
>> On 21.02.2024 18:03, Roger Pau Monne wrote:
>>> The above can be worked around by using an union when defining the register
>>> variables, so that `di` becomes:
>>>
>>> register union {
>>>     uint8_t e;
>>>     unsigned long r;
>>> } di asm("rdi") = { .e = b };
>>>
>>> Which results in following code generated for `foo()`:
>>>
>>> foo:                                    # @foo
>>>         movzbl  %dil, %edi
>>>         callq   func
>>>         retq
>>>
>>> So the truncation is not longer lost.
>>
>> But how do you explain this behavior? I see absolutely no reason why filling
>> the one union field should lead to zero-extension. If I'm not mistaken the
>> language allows the rest of the union to retain undefined contents. So to me
>> this looks like you're converting something that failed to build due to a
>> (presumed) bug in Clang to something that any compiler would be permitted to
>> translate to other than what we want.
> 
> Oh, right, I was expecting the compiler to zero extend it, confabulating
> how unmentioned fields are initialized in structs.
> 
> However, if as mentioned in the psABI thread, the callee is required
> to do any zero extension as necessary, using the union shouldn't cause
> issues for compilers that implement the ABI properly.
> 
> IOW: I don't think the proposed workaround would cause issues for gcc.

Well, that's making a lot of assumptions then. There may indeed be little
reason for the compiler to emit different code, but it is absolutely free
to do so. I continue to think that we want to limit this workaround to as
narrow a set of compilers as possible. Clearly for Clang the code is
better than without the workaround, so there's no reason to take this as
an improvement even if in some obscure case it would still cause an issue.
But for a compiler that produces correct code without this workaround, we
shouldn't chance the workaround breaking some case somewhere.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 11:01:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 11:01:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684345.1064146 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6p2-0006c0-SS; Thu, 22 Feb 2024 11:01:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684345.1064146; Thu, 22 Feb 2024 11:01:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6p2-0006br-Pc; Thu, 22 Feb 2024 11:01:00 +0000
Received: by outflank-mailman (input) for mailman id 684345;
 Thu, 22 Feb 2024 11:00:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aBgv=J7=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rd6p1-0006bc-Il
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 11:00:59 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a7daf1a0-d171-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 12:00:57 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-512b4388dafso5216944e87.2
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 03:00:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a7daf1a0-d171-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708599657; x=1709204457; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=XBhBSYJB3FxNbdMA+DfG7SOVbzaBlWRh1j33ezEklCE=;
        b=UrFxk6U4hVxLpqFgyBSLHpu0Pm+HROI/8dZ3q5Aj08NkAvPXpjrtyQcYO+oC8Q1z06
         EvNpWUfq0oSMFzIaUjd5N48w2VtBM5mXsW6LhE8MF6C99VhEe0o4KzLIN7A1MqP6IlT0
         mYbc/P6khondKuntd0Ny10+G02OcFYXUKYwXk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708599657; x=1709204457;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=XBhBSYJB3FxNbdMA+DfG7SOVbzaBlWRh1j33ezEklCE=;
        b=NSsT6qx6gkg6DKwe/hGsZnu9VgckqOTt89790wOR45gpAtmhC2z4NEKsCCPPos+i2d
         zn3Ce+z/aVI6SG3MObDfl5rCOnye9hHFvGD36VSgXRDuf3m8v7Dsjg2ufvPLRYPFBQqa
         u8K27TZs/qRBpLORjoSPWdxPtkQTIkJkO8IxLsboDE3FaKHs2egV39C2XEmZ+HMhGOzx
         RaX59AZjhlTZVvn8z6ZNS2fDramySYQi/+E942+tO0wfxG0NIQRocPa0YavITL5QaQwv
         roDPJ1Q8mRiSQwtTPMiirhkWQKtEuAAiCq2WioaGlOxRGPLKYNf4Mtpj8dP9+udhsbck
         1M1w==
X-Forwarded-Encrypted: i=1; AJvYcCXu73wLB6c/Q0+plBKw0rGDa12MOcgxcKx+/pgLruu3JS+Zls561PoagEeIUaIJgPjPu/ysFcLu9sz1+3sMxYbKgxZMA8LoxK86VlfIbO8=
X-Gm-Message-State: AOJu0YzjzyrV/gCQd7DN4FOQMdR/xyF9NPQVF1BRXItZnj+idPsRzm8s
	HxVoez222CVAyi100RmIEJgY0c6nydziOQstZdAvGL4TQVafQcxhU6haTspYsfwVeVzlBbIMTgM
	V2etRcUTK8pCFUGXhvPXQXR4JKOhAChJweQsoDw==
X-Google-Smtp-Source: AGHT+IGVsDMheNxhedNH/tUJBtlpHFkfEiCXyTglLSRUGfUkzlYbN3hTQzo5FdCvS/qUZFypSwjWGpKBZcELwshHPSM=
X-Received: by 2002:a05:6512:308d:b0:512:a8bd:4bee with SMTP id
 z13-20020a056512308d00b00512a8bd4beemr9818068lfd.45.1708599656725; Thu, 22
 Feb 2024 03:00:56 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-5-george.dunlap@cloud.com> <c5f3a78c-4a2b-499b-93bb-11f60b0afcc0@suse.com>
 <CA+zSX=acH-4B01=vUBstAbsNBMVgcRe_A5v0eXn66BdsuPtgiA@mail.gmail.com>
 <74ef7b2b-b105-4060-afaa-73ec272a7449@suse.com> <CA+zSX=ZPKScDQO6XXGOp6gv1arEs4-iWTgX8NSezYo4LGqkoLA@mail.gmail.com>
 <40ff2158-78aa-478b-b591-02bbcb1bf88f@suse.com>
In-Reply-To: <40ff2158-78aa-478b-b591-02bbcb1bf88f@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Thu, 22 Feb 2024 19:00:45 +0800
Message-ID: <CA+zSX=b4DKJ1_s9E8N9EGu09=zzQC+NPVMHeF-C1gHxT5RgSog@mail.gmail.com>
Subject: Re: [PATCH 4/6] nestedsvm: Disable TscRateMSR
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 22, 2024 at 5:50=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 22.02.2024 10:30, George Dunlap wrote:
> > On Wed, Feb 21, 2024 at 6:52=E2=80=AFPM Jan Beulich <jbeulich@suse.com>=
 wrote:
> >>>> But then of course Andrew may know of reasons why all of this is don=
e
> >>>> in calculate_host_policy() in the first place, rather than in HVM
> >>>> policy calculation.
> >>>
> >>> It sounds like maybe you're confusing host_policy with
> >>> x86_capabilities?  From what I can tell:
> >>>
> >>> *  the "basic" cpu_has_X macros resolve to boot_cpu_has(), which
> >>> resolves to cpu_has(&boot_cpu_data, ...), which is completely
> >>> independent of the cpu-policy.c:host_cpu_policy
> >>>
> >>> * cpu-policy.c:host_cpu_policy only affects what is advertised to
> >>> guests, via {pv,hvm}_cpu_policy and featureset bits.  Most notably a
> >>> quick skim doesn't show any mechanism by which host_cpu_policy could
> >>> affect what features Xen itself decides to use.
> >>
> >> I'm not mixing the two, no; the two are still insufficiently disentang=
led.
> >> There's really no reason (long term) to have both host policy and
> >> x86_capabilities. Therefore I'd prefer if new code (including a basica=
lly
> >> fundamental re-write as is going to be needed for nested) to avoid
> >> needlessly further extending x86_capabilities. Unless of course there'=
s
> >> something fundamentally wrong with eliminating the redundancy, which
> >> likely Andrew would be in the best position to point out.
> >
> > So I don't know the history of how things got to be the way they are,
> > nor really much about the code but what I've gathered from skimming
> > through while creating this patch series.  But from that impression,
> > the only issue I really see with the current code is the confusing
> > naming.  The cpufeature.h code has this nice infrastructure to allow
> > you to, for instance, enable or disable certain bits on the
> > command-line; and the interface for querying all the different bits of
> > functionality is all nicely put in one place.  Moving the
> > svm_feature_flags into x86_capabilities would immediately allow SVM to
> > take advantage of this infrastructure; it's not clear to me how this
> > would be "needless".
> >
> > Furthermore, it looks to me like host_cpu_policy is used as a starting
> > point for generating pv_cpu_policy and hvm_cpu_policy, both of which
> > are only used for guest cpuid generation.  Given that the format of
> > those policies is fixed, and there's a lot of "copy this bit from the
> > host policy wholesale", it seems like no matter what, you'd want a
> > host_cpu_policy.
> >
> > And in any case -- all that is kind of moot.  *Right now*,
> > host_cpu_policy is only used for guest cpuid policy creation; *right
> > now*, the nested virt features of AMD are handled in the
> > host_cpu_policy; *right now*, we're advertising to guests bits which
> > are not properly virtualized; *right now* these bits are actually set
> > unconditionally, regardless of whether they're even available on the
> > hardware; *right now*, Xen uses svm_feature_flags to determine its own
> > use of TscRateMSR; so *right now*, removing this bit from
> > host_cpu_policy won't prevent Xen from using TscRateMSR itself.
> >
> > (Unless my understanding of the code is wrong, in which case I'd
> > appreciate a correction.)
>
> There's nothing wrong afaics, just missing at least one aspect: Did you
> see all the featureset <-> policy conversions in cpu-policy.c? That (to
> me at least) clearly is a sign of unnecessary duplication of the same
> data. This goes as far as seeding the host policy from the raw one, just
> to then immediately run x86_cpu_featureset_to_policy(), thus overwriting
> a fair part of what was first taken from the raw policy. That's necessary
> right now, because setup_{force,clear}_cpu_cap() act on
> boot_cpu_data.x86_capability[], not the host policy.
>
> As to the "needless" further up, it's only as far as moving those bits
> into x86_capability[] would further duplicate information, rather than
> (for that piece at least) putting them into the policies right away. But
> yes, if the goal is to have setup_{force,clear}_cpu_cap() be able to
> control those bits as well, then going the intermediate step would be
> unavoidable at this point in time.

I'm still not sure of what needs to happen to move this forward.

As I said, I'm not opposed to doing some prep work; but I don't want
to randomly guess as to what kinds of clean-up needs to be done, only
to be told it was wrong (either by you when I post it or by Andy
sometime after it's checked in).

I could certainly move svm_feature_flags into host_cpu_policy, and
have cpu_svm_feature_* reference host_cpu_policy instead (after moving
the nested virt "guest policy" tweaks into hvm_cpu_policy); but as far
as I can tell, that would be the *very first* instance of Xen using
host_cpu_policy in that manner.  I'd like more clarity that this is
the long-term direction that things are going before then.

If you (plural) don't have time now to refresh your memory / make an
informed decision about what you want to happen, then please consider
just taking the patch as it is; it doesn't make future changes any
harder.

 -George


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 11:02:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 11:02:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684353.1064170 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6qO-0007Nd-Cq; Thu, 22 Feb 2024 11:02:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684353.1064170; Thu, 22 Feb 2024 11:02:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd6qO-0007NW-8X; Thu, 22 Feb 2024 11:02:24 +0000
Received: by outflank-mailman (input) for mailman id 684353;
 Thu, 22 Feb 2024 11:02:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rd6qM-0007NO-Q5
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 11:02:22 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id da06ca89-d171-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 12:02:21 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a293f2280c7so1107493566b.1
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 03:02:21 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 qo3-20020a170907874300b00a3e4efbfdacsm4928763ejc.225.2024.02.22.03.02.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 03:02:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da06ca89-d171-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708599741; x=1709204541; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Vng0Xhu5bNeOx6NGjpq55auRrEyg23sjRCgd8WKvvMg=;
        b=D3d947bjLSCDRFJp0PjUe3qBiOUqLVbfq/eRjWfoNFasXD/arHUyxWywqWd1mcJOYC
         8YwOZxMu328pOVu8F+ZzPxv9F8koPgjm1X9EctW4l4F08KtohW/ImZpOPPOdmO5Miwda
         94Gw1jA8vp4WZLiREn7HbzIibyAWqJNkFOhBhRJjzjYvQ63+LlLMmFAHfXxXsk+u6c7t
         qDNqzrMDyllxO3EbuXnLA8Eejy1u9J7e/PVnqHlCtZZS5lxBZxAM0dBe8xb/d5fb1Mnv
         MuS+SOtgHYly4kxkZ87cr/6vLVGF4jitL5tADPds5LNdfGOJXGrX4aHVFAWk3TkEcnHW
         lUSw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708599741; x=1709204541;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Vng0Xhu5bNeOx6NGjpq55auRrEyg23sjRCgd8WKvvMg=;
        b=qdnags2fXsx/OQJAH2UVI8CCcvGh/u4JbXGWd/0EM2L45r/wAm6uYPUf8dDs/MVykr
         +93eCbeI0CUDCs+MuFhNcNu5uOVhtX80rD3p9N3JjlG5nR51bxiNkp6xfpQJ9TJs/eyk
         iRs8ucW6OEM+5G6/N2MkPRMVLbi8x9JIJcYkoEqhQcQfJ1DX5IjLeQMdnAkdNlDnHqQz
         Ok6dPmyOD87CHgpO7gosboymuMsZt+UL74NVwYUQBbXV/NogcKv62U1ISNBhln3zLhtH
         rE/jaUXftX52xuCCpckhLNKl2woESZhoFUXrC54XkqxrnotNaAoE0Za/ouVAM2P2q8OQ
         ltZA==
X-Forwarded-Encrypted: i=1; AJvYcCVkjjFSL2KvrZ2opZehkElQDeVpx4HKg+OOvOi8Yp37Kt7I3LadZ4VCQpSFogqEk8FQcvtez5ULIairRKql0vRXmNTY5pTKlnWfQB1xIzk=
X-Gm-Message-State: AOJu0YzQpDigxl3nFJF4az7Q1UkWBN8UtGZXK3tjT6pYeh2Z79zfxzXc
	5wSjsA1BEu+xBC2aFz1q4EwrjQA8JoHHFeEC0jOHGa0nFIsmrnNsqpe6D4Po/GQcaaxYuv/MzAA
	=
X-Google-Smtp-Source: AGHT+IHCYhqrTEY4kmjioazyb++vPy/MGjCzH1ocMcr+yH6ScBeRCLt617o1eI7vjDoDJNZw74xhyg==
X-Received: by 2002:a17:906:b796:b0:a3f:33c5:ffb5 with SMTP id dt22-20020a170906b79600b00a3f33c5ffb5mr3077706ejb.76.1708599741094;
        Thu, 22 Feb 2024 03:02:21 -0800 (PST)
Message-ID: <ca9b7c65-cd75-4fb0-b6a9-93fe62c03de5@suse.com>
Date: Thu, 22 Feb 2024 12:02:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] x86/hpet: use an atomic add instead of a cmpxchg loop
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-3-roger.pau@citrix.com>
 <d37335f7-ae3c-4dc8-854a-625df275f5da@suse.com> <Zdco8nMu2N4WDWwx@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zdco8nMu2N4WDWwx@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.02.2024 11:58, Roger Pau Monné wrote:
> On Thu, Feb 22, 2024 at 11:10:54AM +0100, Jan Beulich wrote:
>> On 22.02.2024 10:05, Roger Pau Monne wrote:
>>> The usage of a cmpxchg loop in hpet_get_channel() is unnecessary, as the same
>>> can be achieved with an atomic increment, which is both simpler to read, and
>>> avoid any need for a loop.
>>>
>>> Note there can be a small divergence in the channel returned if next_channel
>>> overflows, but returned channel will always be in the [0, num_hpets_used)
>>> range, and that's fine for the purpose of balancing HPET channels across CPUs.
>>> This is also theoretical, as there's no system currently with 2^32 CPUs (as
>>> long as next_channel is 32bit width).
>>
>> The code change looks good to me, but I fail to see the connection to
>> 2^32 CPUs. So it feels like I'm missing something, in which case I'd
>> rather not offer any R-b.
> 
> The purpose of hpet_get_channel() is to distribute HPET channels
> across CPUs, so that each CPU gets assigned an HPET channel, balancing
> the number of CPUs that use each channel.
> 
> This is done by (next_channel++ % num_hpets_used), so the value of
> next_channel after this change can be > num_hpets_used, which
> previously wasn't.  This can lead to a different channel returned for
> the 2^32 call to the function, as the counter would overflow.  Note
> calls to the function are restricted to the number of CPUs in the
> host, as per-CPU channel assignment is done only once (and the channel
> is then stored in a per-cpu variable).

That's once per CPU being brought up, not once per CPU in the system.

> Feel free to adjust the commit message if you think all this is too
> much data, or too confusing.

I'll simply drop that last sentence then, without which
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 11:13:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 11:13:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684361.1064180 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd70Y-000172-B2; Thu, 22 Feb 2024 11:12:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684361.1064180; Thu, 22 Feb 2024 11:12:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd70Y-00016v-6r; Thu, 22 Feb 2024 11:12:54 +0000
Received: by outflank-mailman (input) for mailman id 684361;
 Thu, 22 Feb 2024 11:12:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rd70W-00016p-5D
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 11:12:52 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rd70V-0004yE-PP; Thu, 22 Feb 2024 11:12:51 +0000
Received: from [15.248.2.233] (helo=[10.24.67.33])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rd70V-0003JO-HD; Thu, 22 Feb 2024 11:12:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=kuIMoXlm/WIgCgAjrPoiKKWqVYCqvNK25pngRZV6Ntk=; b=cuuzIGStbGcTCXEtdYqWv6ubCU
	st6reHUg1xr+KT2kcyiZnHHvjtE1OpP6Bx0rhaOIHcvxCoZ8LYVxP8MCVsXCYv7m1ehZHejUs+dKc
	kFQmur6eHIAq2MYhAhvDzeGjraZp5Rm0rRKsRo57FWhnfXvcD23qJgm+Y++I1ZDnRbE0=;
Message-ID: <b8e39e0e-213f-436a-ad8e-65679be2a3b6@xen.org>
Date: Thu, 22 Feb 2024 11:12:49 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] x86/memsharing: use an atomic add instead of a
 cmpxchg loop
Content-Language: en-GB
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Tamas K Lengyel <tamas@tklengyel.com>, Jan Beulich <jbeulich@suse.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-2-roger.pau@citrix.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240222090530.62530-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Roger,

On 22/02/2024 09:05, Roger Pau Monne wrote:
> The usage of a cmpxchg loop in get_next_handle() is unnecessary, as the same
> can be achieved with an atomic increment, which is both simpler to read, and
> avoid any need for a loop.
> 
> The cmpxchg usage is likely a remnant of 32bit support, which didn't have an
> instruction to do an atomic 64bit add, and instead a cmpxchg had to be used.
> 
> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 11:16:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 11:16:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684367.1064188 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd74M-0001iG-Q4; Thu, 22 Feb 2024 11:16:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684367.1064188; Thu, 22 Feb 2024 11:16:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd74M-0001i9-Mp; Thu, 22 Feb 2024 11:16:50 +0000
Received: by outflank-mailman (input) for mailman id 684367;
 Thu, 22 Feb 2024 11:16:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rd74L-0001i3-MD
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 11:16:49 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rd74L-00052w-JH; Thu, 22 Feb 2024 11:16:49 +0000
Received: from [15.248.2.233] (helo=[10.24.67.33])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rd74L-0003Oh-Dy; Thu, 22 Feb 2024 11:16:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=quvwbhyJIdAjkyDw7gtuYWlKsRRKpUE5bMg+BaPidSs=; b=qc9duCbeYWq6sUctDULNTLCqO6
	vqU8MyW12DKDY+fbSCf9cdZemK/8yqH3LhPRcaav4/bASPA7U1lO0gY9+qLMdhQiDFH7x3mQzpFK+
	Gc83z85J5CfWIzjWkgoG2K8bQi03ncepKrCdWEZLgYG7cGnySSp/NXBR7tnHjMnGED6A=;
Message-ID: <67314ab0-1c8f-4617-9b0f-94488e3f6d9c@xen.org>
Date: Thu, 22 Feb 2024 11:16:47 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] x86/hpet: use an atomic add instead of a cmpxchg loop
Content-Language: en-GB
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
 <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-3-roger.pau@citrix.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240222090530.62530-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Roger,

On 22/02/2024 09:05, Roger Pau Monne wrote:
> The usage of a cmpxchg loop in hpet_get_channel() is unnecessary, as the same
> can be achieved with an atomic increment, which is both simpler to read, and
> avoid any need for a loop.
> 
> Note there can be a small divergence in the channel returned if next_channel
> overflows, but returned channel will always be in the [0, num_hpets_used)
> range, and that's fine for the purpose of balancing HPET channels across CPUs.
> This is also theoretical, as there's no system currently with 2^32 CPUs (as
> long as next_channel is 32bit width).

This would also happen if we decide to reduce the size next_channel.

Reviewed-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 11:26:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 11:26:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684384.1064231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd7Db-0004JT-2x; Thu, 22 Feb 2024 11:26:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684384.1064231; Thu, 22 Feb 2024 11:26:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd7Da-0004JM-UX; Thu, 22 Feb 2024 11:26:22 +0000
Received: by outflank-mailman (input) for mailman id 684384;
 Thu, 22 Feb 2024 11:26:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rd7DZ-0004JC-EG
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 11:26:21 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 32f16d73-d175-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 12:26:19 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a3ddc13bbb3so129254666b.0
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 03:26:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 rg8-20020a1709076b8800b00a3e28471fa4sm5339560ejc.59.2024.02.22.03.26.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 03:26:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 32f16d73-d175-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708601179; x=1709205979; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=zgKhvR11LyYdIZ9/4bMkbgjO3M131DXJ/4QsMosSG64=;
        b=DL8mrl+7+S+7Z6mcbtOhoWFYjSx+ru+NmBSvnbmcfi0vNqKZenjE0dIuU8+VTqAPZT
         SQqcZRFIOAvR9DR3PZvjSTKCsZeESprFlSdfWzhzonwoub6/Y+4EMk4aTigGrKsxK765
         cXg4WKuSbAy0gZ2qD1iw1vJYaxvqcMMmYahjdXDHnYjOPE/MM2qlrMeNqB+5QIPW+GS3
         mgLs7mV6XldnjK05NjDQp6HSui7BYZH3hYEFsCH7mLo3kBBB3+2qtvl07qIsYzzXhob6
         HAlIo21xEy3oc0/pv7fR4RaKPLkrPCitEUat2VS4ADGB+HIB356I4LJeGiwJlUZYJdZ5
         cw4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708601179; x=1709205979;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zgKhvR11LyYdIZ9/4bMkbgjO3M131DXJ/4QsMosSG64=;
        b=Tgm/UGG2YtgNgVcXtt8txFHWcBvWu3e6lFxOoB+qPzN8TJuB1cEGVlbJfm9A2jHMjI
         xk7lU9GLCySGZRPQv6QKv16OmIYDSrVG/b6akM4oPyl2yPpqwsfHuJ3B6FBS01uvqKJL
         ZVnSlWP1/yEhRhPIOSVUG/hczFFg0+ARXV8xOXRj9g8taKs1IzTY+uwKQW+SW23q/Jea
         UDLR3lYn49fRH/JbonFAr4Wpb05TKFeQnjRYzlv5Qer1gjJKjhO44H9+h72S6ONgoQkE
         CLiCOQhl7NqsdAJ+8l1D7TTqfeCa35aNosSKXl7lib3QcESbd0yoP9IfWeifdW5wqruF
         y09g==
X-Forwarded-Encrypted: i=1; AJvYcCVNcBQoyRzn6tlM2PjNzTyVX5AEDILFctvSbEC7yb+VMoACxHdrwXVSwd2Ya9wwweVqb1m1khsOPlDdYoLGN2/qLUEPq8tQxknp28tT4OY=
X-Gm-Message-State: AOJu0YxJodOUzBJoxvodRjG+6AL/N1+tZYjk4L8PPMKWwocP4L46FnAK
	bhtJuju6wfd2Zz23Vbo/uHV5eH2kVRgtS5bH81q6uNN6ctX4jG+r9dmB3t18Bg==
X-Google-Smtp-Source: AGHT+IGidJzeOmy8hmXeN2kzUtYGC73k08fYyOFm4rIaQ9wNe2VRQRzgbo+MIv5VZRdzSQ9jsd1pPw==
X-Received: by 2002:a17:907:9712:b0:a3f:2596:790c with SMTP id jg18-20020a170907971200b00a3f2596790cmr2744806ejc.8.1708601178656;
        Thu, 22 Feb 2024 03:26:18 -0800 (PST)
Message-ID: <bace3eda-3505-497d-a028-86bafc03de9b@suse.com>
Date: Thu, 22 Feb 2024 12:26:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/6] nestedsvm: Disable TscRateMSR
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-5-george.dunlap@cloud.com>
 <c5f3a78c-4a2b-499b-93bb-11f60b0afcc0@suse.com>
 <CA+zSX=acH-4B01=vUBstAbsNBMVgcRe_A5v0eXn66BdsuPtgiA@mail.gmail.com>
 <74ef7b2b-b105-4060-afaa-73ec272a7449@suse.com>
 <CA+zSX=ZPKScDQO6XXGOp6gv1arEs4-iWTgX8NSezYo4LGqkoLA@mail.gmail.com>
 <40ff2158-78aa-478b-b591-02bbcb1bf88f@suse.com>
 <CA+zSX=b4DKJ1_s9E8N9EGu09=zzQC+NPVMHeF-C1gHxT5RgSog@mail.gmail.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CA+zSX=b4DKJ1_s9E8N9EGu09=zzQC+NPVMHeF-C1gHxT5RgSog@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.02.2024 12:00, George Dunlap wrote:
> On Thu, Feb 22, 2024 at 5:50 PM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 22.02.2024 10:30, George Dunlap wrote:
>>> On Wed, Feb 21, 2024 at 6:52 PM Jan Beulich <jbeulich@suse.com> wrote:
>>>>>> But then of course Andrew may know of reasons why all of this is done
>>>>>> in calculate_host_policy() in the first place, rather than in HVM
>>>>>> policy calculation.
>>>>>
>>>>> It sounds like maybe you're confusing host_policy with
>>>>> x86_capabilities?  From what I can tell:
>>>>>
>>>>> *  the "basic" cpu_has_X macros resolve to boot_cpu_has(), which
>>>>> resolves to cpu_has(&boot_cpu_data, ...), which is completely
>>>>> independent of the cpu-policy.c:host_cpu_policy
>>>>>
>>>>> * cpu-policy.c:host_cpu_policy only affects what is advertised to
>>>>> guests, via {pv,hvm}_cpu_policy and featureset bits.  Most notably a
>>>>> quick skim doesn't show any mechanism by which host_cpu_policy could
>>>>> affect what features Xen itself decides to use.
>>>>
>>>> I'm not mixing the two, no; the two are still insufficiently disentangled.
>>>> There's really no reason (long term) to have both host policy and
>>>> x86_capabilities. Therefore I'd prefer if new code (including a basically
>>>> fundamental re-write as is going to be needed for nested) to avoid
>>>> needlessly further extending x86_capabilities. Unless of course there's
>>>> something fundamentally wrong with eliminating the redundancy, which
>>>> likely Andrew would be in the best position to point out.
>>>
>>> So I don't know the history of how things got to be the way they are,
>>> nor really much about the code but what I've gathered from skimming
>>> through while creating this patch series.  But from that impression,
>>> the only issue I really see with the current code is the confusing
>>> naming.  The cpufeature.h code has this nice infrastructure to allow
>>> you to, for instance, enable or disable certain bits on the
>>> command-line; and the interface for querying all the different bits of
>>> functionality is all nicely put in one place.  Moving the
>>> svm_feature_flags into x86_capabilities would immediately allow SVM to
>>> take advantage of this infrastructure; it's not clear to me how this
>>> would be "needless".
>>>
>>> Furthermore, it looks to me like host_cpu_policy is used as a starting
>>> point for generating pv_cpu_policy and hvm_cpu_policy, both of which
>>> are only used for guest cpuid generation.  Given that the format of
>>> those policies is fixed, and there's a lot of "copy this bit from the
>>> host policy wholesale", it seems like no matter what, you'd want a
>>> host_cpu_policy.
>>>
>>> And in any case -- all that is kind of moot.  *Right now*,
>>> host_cpu_policy is only used for guest cpuid policy creation; *right
>>> now*, the nested virt features of AMD are handled in the
>>> host_cpu_policy; *right now*, we're advertising to guests bits which
>>> are not properly virtualized; *right now* these bits are actually set
>>> unconditionally, regardless of whether they're even available on the
>>> hardware; *right now*, Xen uses svm_feature_flags to determine its own
>>> use of TscRateMSR; so *right now*, removing this bit from
>>> host_cpu_policy won't prevent Xen from using TscRateMSR itself.
>>>
>>> (Unless my understanding of the code is wrong, in which case I'd
>>> appreciate a correction.)
>>
>> There's nothing wrong afaics, just missing at least one aspect: Did you
>> see all the featureset <-> policy conversions in cpu-policy.c? That (to
>> me at least) clearly is a sign of unnecessary duplication of the same
>> data. This goes as far as seeding the host policy from the raw one, just
>> to then immediately run x86_cpu_featureset_to_policy(), thus overwriting
>> a fair part of what was first taken from the raw policy. That's necessary
>> right now, because setup_{force,clear}_cpu_cap() act on
>> boot_cpu_data.x86_capability[], not the host policy.
>>
>> As to the "needless" further up, it's only as far as moving those bits
>> into x86_capability[] would further duplicate information, rather than
>> (for that piece at least) putting them into the policies right away. But
>> yes, if the goal is to have setup_{force,clear}_cpu_cap() be able to
>> control those bits as well, then going the intermediate step would be
>> unavoidable at this point in time.
> 
> I'm still not sure of what needs to happen to move this forward.
> 
> As I said, I'm not opposed to doing some prep work; but I don't want
> to randomly guess as to what kinds of clean-up needs to be done, only
> to be told it was wrong (either by you when I post it or by Andy
> sometime after it's checked in).
> 
> I could certainly move svm_feature_flags into host_cpu_policy, and
> have cpu_svm_feature_* reference host_cpu_policy instead (after moving
> the nested virt "guest policy" tweaks into hvm_cpu_policy); but as far
> as I can tell, that would be the *very first* instance of Xen using
> host_cpu_policy in that manner.  I'd like more clarity that this is
> the long-term direction that things are going before then.
> 
> If you (plural) don't have time now to refresh your memory / make an
> informed decision about what you want to happen, then please consider
> just taking the patch as it is; it doesn't make future changes any
> harder.

I'd be willing to ack the patch as is if I understood the piece of code
in calculate_host_policy() that is being changed here. My take is that
in a prereq patch that wants adjusting, by moving it wherever applicable.
Without indications to the contrary, it living there is plain wrong, and
the patch here touching that function isn't quite right either, even if
only as a result of the original issue.

If without at least a little bit of input from Andrew we can't move
forward, then that's the way it is - we need to wait. As you likely know
I have dozens of patches in that state ...

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 12:58:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 12:58:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684399.1064240 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd8ec-0000VX-PC; Thu, 22 Feb 2024 12:58:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684399.1064240; Thu, 22 Feb 2024 12:58:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd8ec-0000VQ-MJ; Thu, 22 Feb 2024 12:58:22 +0000
Received: by outflank-mailman (input) for mailman id 684399;
 Thu, 22 Feb 2024 12:58:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aBgv=J7=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rd8ea-0000VK-HA
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 12:58:20 +0000
Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com
 [2a00:1450:4864:20::22b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0c929cc7-d182-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 13:58:18 +0100 (CET)
Received: by mail-lj1-x22b.google.com with SMTP id
 38308e7fff4ca-2d0bc402c8eso8163131fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 04:58:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c929cc7-d182-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708606698; x=1709211498; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=fjre2Q/BSMGJzCXdj7238/HXa8ZceCmEO0mTF1CT35I=;
        b=KAAtgH2Q5wqK1czdl0vHDxxMqsZoua448STTHAbmEVz24kKyQzvPb/ntF37W9EW6u2
         zM7d/ou/Yu8uIGIYpIdd9Ayn6u30kDSmpyrYmka2G2sDcmJmrsJ8XnXfsGdZoH5tSUZl
         2rZqQhlIgGKJMxrTr+DQBSi9qeTQyWhI+p/vk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708606698; x=1709211498;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=fjre2Q/BSMGJzCXdj7238/HXa8ZceCmEO0mTF1CT35I=;
        b=epFlojbhHbmpnsrgz7OFLJ/aDm2NxWek+phK8tgKvm4MW75Ba8fQtMa8wEjWtIlci5
         30lNRfhWlDhHJ+Gp9SS7E0UuOpi0ponWg0bnu+Kah8eR350kW0MCnGh1mEGOobjquGPN
         P70xy6uwVgFAlq23MzSET2pYZgn+KpOTMzS4LNIvIkw78qcbXb/ST6Fdfal8B9l9PVNI
         SoeesNID/hn3NsENlAtZOgkMA7nPCZB5PETvwMC3W8TE0J3D7oVXI2LBPboy24UhiBq/
         4BiYogFOXz7F7dQHtD/fUXLfRB+Q8PrTmgShWGM2BvEsJb2U0aL2TnI+J+bzrLtdUvk9
         EOXA==
X-Gm-Message-State: AOJu0YxTz2umwTJqOaiUWAM+s6HWr6/0xkJa0C1JiYxWu+FS32x3uuXf
	11dEnds4h457MsWh10S6YDvKmbyHU0rYXD1xJ4fQFjEf+9L4AReLLpCFIVnxlZk5LgQygkpAs9t
	wQi/fcWS0je9iAHrwaboPjcTR1Ym9gsfvHSo46Q==
X-Google-Smtp-Source: AGHT+IGrGuMknL/Lt6eaFLg7q4My4LtxTYnRntmZ1GVvJHg4kN41xZtwmsRfczYMkCjeX+I/bv2PkldgxzgK90/wQWw=
X-Received: by 2002:a2e:9241:0:b0:2d0:e4ef:f1ec with SMTP id
 v1-20020a2e9241000000b002d0e4eff1ecmr931412ljg.18.1708606697811; Thu, 22 Feb
 2024 04:58:17 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-6-george.dunlap@cloud.com> <801618fe-73fe-4d15-8e17-dd02e115cdf8@suse.com>
In-Reply-To: <801618fe-73fe-4d15-8e17-dd02e115cdf8@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Thu, 22 Feb 2024 20:58:06 +0800
Message-ID: <CA+zSX=Y3W-CXyXzOZYXNhjSOUMJpS6=mLjjFJiUFxOOpPeB2sg@mail.gmail.com>
Subject: Re: [PATCH 5/6] nestedsvm: Remove bogus debug message from nestedsvm_check_intercepts
To: Jan Beulich <jbeulich@suse.com>
Cc: xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 19, 2024 at 11:56=E2=80=AFPM Jan Beulich <jbeulich@suse.com> wr=
ote:
>
> On 06.02.2024 02:20, George Dunlap wrote:
> > Changeset ef3e8db8068 ("x86/hvm: Corrections and improvements to
> > unhandled vmexit logging") introduced a printk to the default path of
> > the switch statement in nestedsvm_check_intercepts(), complaining of
> > an unknown exit reason.
> >
> > Unfortunately, the "core" switch statement which is meant to handle
> > all vmexit reasons is in nsvm_vmcb_guest_intercepts_exitcode(); the
> > switch statement in nestedsvm_check_intercepts() is only meant to
> > superimpose on top of that some special-casing for how to interaction
> > between L1 and L0 vmexits.
> >
> > Remove the printk, and add a comment to prevent future confusion.
> >
> > Signed-off-by: George Dunlap <george.dunlap@cloud.com>
>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>
> I wonder if a Fixes: tag is warranted here.

Yes, I think probably so.


 -George


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 13:15:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 13:15:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684409.1064250 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd8vQ-0003Nz-8Q; Thu, 22 Feb 2024 13:15:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684409.1064250; Thu, 22 Feb 2024 13:15:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd8vQ-0003Ns-5j; Thu, 22 Feb 2024 13:15:44 +0000
Received: by outflank-mailman (input) for mailman id 684409;
 Thu, 22 Feb 2024 13:15:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd8vO-0003Ni-U1; Thu, 22 Feb 2024 13:15:42 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd8vO-00074v-RS; Thu, 22 Feb 2024 13:15:42 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd8vO-0006kV-I4; Thu, 22 Feb 2024 13:15:42 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rd8vO-0001j9-HQ; Thu, 22 Feb 2024 13:15:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=zRRy/eVjfWMlDcIedCTF49Vb5Yk7XABcnVd0FryFkh4=; b=Q2cnVeJO6Yyu8LG1gl6pHJI7hY
	resR7CaPo/52FBQG6NGJiiqnEO8kxPc1eNK7NlWQ91z2N2hKz259ur/yUke5gkDg82XV6Q9okOLJh
	cOyc/reX9D0Ly3W+UeJqvh7sFtWS+Pc58pv8udrGsb+JClfjp+XmhJn7ebTR/mYMT4Mw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184723-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184723: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-i386-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f8791d0fd3adbda3701e7eb9db63a9351b478365
X-Osstest-Versions-That:
    xen=f8791d0fd3adbda3701e7eb9db63a9351b478365
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 22 Feb 2024 13:15:42 +0000

flight 184723 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184723/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail in 184720 pass in 184723
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184720

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184720
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184720
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184720
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184720
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184720
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184720
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184720
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184720
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184720
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184720
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184720
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184720
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  f8791d0fd3adbda3701e7eb9db63a9351b478365
baseline version:
 xen                  f8791d0fd3adbda3701e7eb9db63a9351b478365

Last test of basis   184723  2024-02-22 01:52:14 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Thu Feb 22 13:40:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 13:40:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684417.1064261 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd9Iz-0006kN-7I; Thu, 22 Feb 2024 13:40:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684417.1064261; Thu, 22 Feb 2024 13:40:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd9Iz-0006jt-3Q; Thu, 22 Feb 2024 13:40:05 +0000
Received: by outflank-mailman (input) for mailman id 684417;
 Thu, 22 Feb 2024 13:40:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd9Iy-0006eW-J6; Thu, 22 Feb 2024 13:40:04 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd9Iy-0007U8-HV; Thu, 22 Feb 2024 13:40:04 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rd9Iy-0008MK-3q; Thu, 22 Feb 2024 13:40:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rd9Iy-0005Qz-3E; Thu, 22 Feb 2024 13:40:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HJPaZ8slHpAs1WGMQUx9HcO6b5I8osRwPzaOVqAoCz0=; b=1v8ilpykI8T1qLjY3bIe7XJHn6
	LAfl2SUXX2xZLGhyJDPpRMcpg11szVycTJNXpqtqmyF7VkRRBy0donGF3gSe1FHAarhn+BjUI+Xe3
	HfqFVes5KyBTIvF+EP7o4mBaK/tTWKUZIfg33NDPtg1+SM3vCZsgf5+ilixKSO0PyuD0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184727-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184727: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9ee7dc877b8754ce2fc82500feea52c04d4e6409
X-Osstest-Versions-That:
    xen=f8791d0fd3adbda3701e7eb9db63a9351b478365
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 22 Feb 2024 13:40:04 +0000

flight 184727 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184727/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  9ee7dc877b8754ce2fc82500feea52c04d4e6409
baseline version:
 xen                  f8791d0fd3adbda3701e7eb9db63a9351b478365

Last test of basis   184713  2024-02-20 15:00:29 Z    1 days
Testing same since   184727  2024-02-22 11:00:37 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com> # XSM
  Federico Serafini <federico.serafini@bugseng.com>
  Frediano Ziglio <frediano.ziglio@cloud.com>
  George Dunlap <george.dunlap@cloud.com> # sched
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f8791d0fd3..9ee7dc877b  9ee7dc877b8754ce2fc82500feea52c04d4e6409 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 13:48:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 13:48:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684423.1064271 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd9Qr-0007eO-2V; Thu, 22 Feb 2024 13:48:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684423.1064271; Thu, 22 Feb 2024 13:48:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd9Qq-0007eH-Vl; Thu, 22 Feb 2024 13:48:12 +0000
Received: by outflank-mailman (input) for mailman id 684423;
 Thu, 22 Feb 2024 13:48:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rd9Qp-0007eB-E8
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 13:48:11 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 03aa47ef-d189-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 14:48:09 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a3e8c1e4aa7so227514666b.2
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 05:48:09 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 q18-20020a170906145200b00a3d2d81daafsm6098028ejc.172.2024.02.22.05.48.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 05:48:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03aa47ef-d189-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708609689; x=1709214489; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=CB32mqdCo2HKG4yTh7aIIABKIUipuqMQWkLbNKvQVmw=;
        b=eXDc3e/7vBFrHSzd9voBLVMcxCMECUvjyilXSI3WNLHdAlUuyllBTxk7iA6c+x7OHV
         mqAT5Wx+ooVvr2KiKV4VpGFkNyddJW7N5qAg25cIApd6gCu2LAwyIo4M7iXwjZTFFtF+
         gNE80ThVCohRv62JZsIUqbEoNFZpixOK9aGO2hjtuGBEG0zot4w/Mj3Rkw7Nl+CJtrOy
         iU4X4FekM5F5VknkJhZIfauctWYX+hIXc5LAbx3GcAyHxzOgja1MI/Ab5d0abv8evgEJ
         RSie3VacdBUXqUmB9yhAvHfHJoEt5LVj6Pf16Wh3NEh0ufHH/fj1WNxl0SLw08wRUJU8
         TX0g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708609689; x=1709214489;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=CB32mqdCo2HKG4yTh7aIIABKIUipuqMQWkLbNKvQVmw=;
        b=bhqDDZyzz51vl4Z0R+1Pcq252YvR3uQvuWyAJArxeFGrKvMzEU1D4fKNt7GXE18NE6
         Y9vp6AsxMABiLW7LItpgmX8LbfFlahMTEC22e86nIRgwdBNyLjuAC/3wJBAgNJEb85q0
         ikP2xwFS8oBfg5lyPgOqShPSmgWi82ntJdyfI2Eb1SKMk/KfKGC3iwVGPKIezqP0GQxA
         TOYTOmj9TeiBFFocq8KMVsOM3DJvWp3oLLt+NZy9b8MSEOgXMbXzCurfq+ayUJs9/b6F
         2Ixl6BwZVyvpjd17KWgI+rBtX/QNHvTvLYVcr+iq2XMEq+eUifQfLkz6o/79BVdsz85Y
         mg7A==
X-Forwarded-Encrypted: i=1; AJvYcCWNFXhqa/03tEZ5leCPFKWJJ9MYP8N39kLerMuwY4fQphyGlCXrzMaNdWs5UlWJ88s2yXogBUFYnMT59AmKOPjnSRLPPPYuZT1vX/BadPg=
X-Gm-Message-State: AOJu0YzauyNTtkk6JG3p0RL5IwISfBN2KywbkqkM1CFLrbIImixjRSX/
	fPyxcoCEWTui1E0xbant5mJaYRUDz0tSS6pHENrNQQZiVfJrbJ9/LM1/rxeVOA==
X-Google-Smtp-Source: AGHT+IHkGa0up2723rnQ5fe3oeMXa6ee1ak3iosmbRyUy02I02toNIv535jC90y4hfqEDdgxgeyXUg==
X-Received: by 2002:a17:906:6b90:b0:a3e:665a:c870 with SMTP id l16-20020a1709066b9000b00a3e665ac870mr8898866ejr.76.1708609689329;
        Thu, 22 Feb 2024 05:48:09 -0800 (PST)
Message-ID: <4b121e48-9541-4b53-8352-939c904f4f1c@suse.com>
Date: Thu, 22 Feb 2024 14:48:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, julien@xen.org,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
 <d90d98b6-508b-4a2a-ab6a-74a9828a5b94@suse.com>
 <45509cb67ecee3f690b5784489b5ccb4@bugseng.com>
 <1743b4248d30a4e8b68a150c25724caa@bugseng.com>
 <2ff52df443fc080875fd05614d89764d@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <2ff52df443fc080875fd05614d89764d@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 21.02.2024 16:46, Nicola Vetrini wrote:
> On 2024-02-21 13:08, Nicola Vetrini wrote:
>> On 2024-02-20 09:14, Nicola Vetrini wrote:
>>> On 2024-02-20 08:45, Jan Beulich wrote:
>>>> On 19.02.2024 16:14, Nicola Vetrini wrote:
>>>>> The cache clearing and invalidation helpers in x86 and Arm didn't
>>>>> comply with MISRA C Rule 17.7: "The value returned by a function
>>>>> having non-void return type shall be used". On Arm they
>>>>> were always returning 0, while some in x86 returned -EOPNOTSUPP
>>>>> and in common/grant_table the return value is saved.
>>>>>
>>>>> As a consequence, a common helper arch_grant_cache_flush that 
>>>>> returns
>>>>> an integer is introduced, so that each architecture can choose 
>>>>> whether to
>>>>> return an error value on certain conditions, and the helpers have 
>>>>> either
>>>>> been changed to return void (on Arm) or deleted entirely (on x86).
>>>>>
>>>>> Signed-off-by: Julien Grall <julien@xen.org>
>>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>>>> ---
>>>>> The original refactor idea came from Julien Grall in [1]; I edited 
>>>>> that proposal
>>>>> to fix build errors.
>>>>>
>>>>> I did introduce a cast to void for the call to flush_area_local on 
>>>>> x86, because
>>>>> even before this patch the return value of that function wasn't 
>>>>> checked in all
>>>>> but one use in x86/smp.c, and in this context the helper (perhaps 
>>>>> incidentally)
>>>>> ignored the return value of flush_area_local.
>>>>
>>>> I object to such casting to void, at least until there's an 
>>>> overriding
>>>> decision that for Misra purposes such casts may be needed.
>>>>
>>>
>>> There are three choices here:
>>> 1. cast to void
>>> 2. deviation for flush_area_local, which for the case of the cache 
>>> helpers is what led to this patch; it may still be a viable option, if 
>>> other maintainers agree
>>> 3. refactor of flush_area_local; this is not viable here because the 
>>> return value is actually used and useful, as far as I can tell, in 
>>> smp.c
>>>
>>>>> --- a/xen/arch/arm/include/asm/page.h
>>>>> +++ b/xen/arch/arm/include/asm/page.h
>>>>> @@ -123,6 +123,7 @@
>>>>>
>>>>>  #ifndef __ASSEMBLY__
>>>>>
>>>>> +#include <public/grant_table.h>
>>>>
>>>> This is a no-go, imo (also on x86): Adding this include here 
>>>> effectively
>>>> means that nearly every CU will have a dependency on that header, no
>>>> matter that most are entirely agnostic of grants. Each arch has a
>>>> grant_table.h - is there any reason the new, grant-specific helper 
>>>> can't
>>>> be put there?
>>>>
>>>
>>> I would have to test, but I think that can be done
>>>
>>
>> The only blocker so far is that this triggers a build error due to a 
>> circular dependency between xen/mm.h and asm/flushtlb.h on x86. Also 
>> found some earlier evidence [1] that there are some oddities around 
>> asm/flushtlb's inclusion.
>>
>> [1] 
>> https://lore.kernel.org/xen-devel/20200318210540.5602-1-andrew.cooper3@citrix.com/
> 
> There could be a way of untangling asm/flushtlb.h from xen/mm.h, by 
> moving "accumulate_tlbflush" and "filtered_flush_tlb_mask" introduced by 
> commit 80943aa40e30 ("replace tlbflush check and operation with inline 
> functions") [1].
> However, these function should then be part of a generic xen/flushtlb.h 
> header, since they are used in common code (e.g., common/page_alloc) and 
> a bunch of common code source files should move their includes (see [2] 
> for a partial non-working patch). Do you feel that this is a feasible 
> route?

Yeah, introducing xen/flushtlb.h to hold these sounds pretty sensible.

> In passing it should be noted that the header ordering in 
> x86/alternative.c is not the one usually prescribed, so that may be 
> taken care of as well.

I'm afraid I don't understand this remark.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 14:18:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 14:18:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684400.1064281 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd9uE-0003Gt-9P; Thu, 22 Feb 2024 14:18:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684400.1064281; Thu, 22 Feb 2024 14:18:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rd9uE-0003Gm-65; Thu, 22 Feb 2024 14:18:34 +0000
Received: by outflank-mailman (input) for mailman id 684400;
 Thu, 22 Feb 2024 12:58:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=uKx3=J7=thalesgroup.com=dominique.larrieu@srs-se1.protection.inumbo.net>)
 id 1rd8ee-0000jb-4w
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 12:58:24 +0000
Received: from esa.hc1631-21.eu.iphmx.com (esa.hc1631-21.eu.iphmx.com
 [23.90.122.185]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0e5cb3ef-d182-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 13:58:22 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e5cb3ef-d182-11ee-8a55-1f161083a0e0
X-CSE-ConnectionGUID: uzEjowYURbKIur4w0SKPjQ==
X-CSE-MsgGUID: c/RyoclOTNSXO94av7gSwg==
Authentication-Results: ob1.hc1631-21.eu.iphmx.com; dkim=pass (signature verified) header.i=@thalesgroup.com
X-IronPort-AV: E=McAfee;i="6600,9927,10991"; a="10015327"
X-IronPort-AV: E=Sophos;i="6.06,177,1705359600"; 
   d="png'150?scan'150,208,217,150";a="10015327"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=thalesgroup.com; i=@thalesgroup.com; s=bbmfo20230504;
  t=1708606701;
  h=from:to:cc:subject:date:message-id:references:
   in-reply-to:mime-version;
  bh=Q6bRqOBoMvb/8u+PUqU4fAEhWGGx25bsEDUQblyrk2c=;
  b=n53oK8lXXubCSAIoEv1iCKf2fwkwD9t9oj9FXCG6yTIAQEKv7HpbX5OT
   bPkGmtfwBp35yOsMnJVT38gfF/PAg62YT8y6wLSbd+aXFWraQxhA+enf4
   DXO1CHgIFS81xggM4N/woQWNNvgT2D4EggMH0Kp1278yCIY4v6M8NpPPd
   dmrxfHkFl2YQf0jt2Fdj59Rycz4U23x9ETQSGNEULSsJF45AgcahZKIDf
   Z1RRjZqoxHmWGvZu0KaIQZnBuF2lfJo10Dq9+tuGRQqw50TTBHwieKZtq
   dHjYCklpRSqNo23G9Q9MHc6YB7Lltb4zl9dsVEquPumafGpiKRSSIrbeW
   w==;
X-CSE-ConnectionGUID: ei7jUqx0TaiXxirwgLEs3Q==
X-CSE-MsgGUID: vFWUi/NkSzWSeBH/wLYEzQ==
X-CSE-ConnectionGUID: is24IsAjRlGdxjCa9CNdAw==
X-CSE-MsgGUID: 9bvOb9nrR9G4KbIuTOTUzA==
X-IronPort-AV: E=McAfee;i="6600,9927,10991"; a="18623120"
X-IronPort-AV: E=Sophos;i="6.06,177,1705359600"; 
   d="png'150?scan'150,208,217,150";a="18623120"
From: LARRIEU Dominique <dominique.larrieu@thalesgroup.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: WILLEMS Louis <louis.willems@thalesgroup.com>, GRUO Nicolas
	<nicolas-n.gruo@thalesgroup.com>, GOURLOT Francois
	<francois.gourlot@thalesgroup.com>, Kelly Choi <kelly.choi@cloud.com>
Subject: Need help on USB port virtualization with Xen hypervisor
Thread-Topic: Need help on USB port virtualization with Xen hypervisor
Thread-Index: AdpZrSx+TQ/bw3NbSziyPQV7i3t4XAADFUoAAvVGyzA=
Sensitivity: company-confidential
Date: Thu, 22 Feb 2024 12:58:19 +0000
Message-ID: <11692a929ee941eb8db9a1682ca42bd3@thalesgroup.com>
References: <32acf7d97f83494aa142fa3b614e5e7d@thalesgroup.com>
 <CAO-mL=xnWR9EC5by8YhX12XcAFrQ35h5=LF1DJQbiCA7N+nAFg@mail.gmail.com>
In-Reply-To: <CAO-mL=xnWR9EC5by8YhX12XcAFrQ35h5=LF1DJQbiCA7N+nAFg@mail.gmail.com>
Accept-Language: fr-FR, en-US
Content-Language: fr-FR
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
x-ms-exchange-nodisclaimer: 0
Content-Type: multipart/related;
	boundary="_004_11692a929ee941eb8db9a1682ca42bd3thalesgroupcom_";
	type="multipart/alternative"
MIME-Version: 1.0

--_004_11692a929ee941eb8db9a1682ca42bd3thalesgroupcom_
Content-Type: multipart/alternative;
	boundary="_000_11692a929ee941eb8db9a1682ca42bd3thalesgroupcom_"

--_000_11692a929ee941eb8db9a1682ca42bd3thalesgroupcom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

RGVhciBhbGwsDQoNCldlIGFyZSBkZXRlY3Rpbmcgc2V2ZXJhbCBpc3N1ZXMgd2l0aCBVU0IgcG9y
dCB2aXJ0dWFsaXphdGlvbiB3aXRoIHRoZSBYZW4gaHlwZXJ2aXNvci4NCi0gV2UgY2Fubm90IGRv
IFBDSSBwYXNzdGhyb3VnaCBvZiB0aGUgUENJIHVzYiBidXMgb24gYSBXaW5kb3dzIDEwIDE2MDcg
NjQtYml0IHZpcnR1YWwgbWFjaGluZS4gVGhlIGJhZCByZXN1bHQgaXMgYSBXaW5kb3dzIGJsdWUg
c2NyZWVuLg0KLSBXaGVuIHdlIHVzZSB0aGUgcGFzc3Rocm91Z2ggZnVuY3Rpb25hbGl0eSBvbiBh
IFdpbmRvd3MgMjFIMiB2aXJ0dWFsIG1hY2hpbmUsIHdlIG5vdGljZSB0aGF0IHRoZSBzcGVlZCBv
ZiB0aGUgVVNCIHBvcnQgaXMgbm90IGhpZ2ggc3BlZWQgYnV0IGZ1bGwgc3BlZWQgb24gYSBVU0Ig
My4wIHBvcnQNCi0gV2Ugbm90aWNlIGluc3RhYmlsaXRpZXMgd2hlbiB1c2luZyB0aGUgbmVjLXVz
Yi14aGNpIGRyaXZlciwgIFVTQiAyLjAga2V5cyBhcmUgbm90IHJlY29nbml6ZWQgYnkgdGhlIFdp
bmRvd3MgdmlydHVhbCBtYWNoaW5lIChpbmNvcnJlY3QgZGVzY3JpcHRvcikNCg0KV2UgbmVlZCB5
b3VyIGhlbHAgdG8gZmluZCBhIHNvbHV0aW9uIGZvciB0aGVzZSBwcm9ibGVtcy4NCg0KVGhlIFNv
ZnR3YXJlIHVzZWQgYXJlIDoNCg0KLSAgICAgICAgRGViaWFuIDExIHZlcnNpb24gNS4xMC4wLTIw
DQoNCi0gICAgICAgIFhlbiB2ZXJzaW9uIDQuMTQNCg0KLSAgICAgICAgV2luZG93cyAxMCAxNjA3
IGFuZCAyMUgyIGZvciB2aXJ0dWFsIG1hY2hpbmVzLiBWaXJ0dWFsIE1hY2hpbmUgSFZNDQoNClRo
YW5rcyBpbiBhZHZhbmNlIGZvciB5b3VyIGhlbHAuDQoNCkJlc3QgcmVnYXJkcywNCg0KDQoNCg0K
RG9taW5pcXVlIExBUlJJRVUNCg0KQ3liZXIgUHJvamVjdCBNYW5hZ2VyDQoNClRoYWxlcw0KDQoN
CjQsIGF2ZW51ZSBkZXMgTG91dnJlc3Nlcw0KOTIyMzAgR2VubmV2aWxsaWVycywgRnJhbmNlDQoN
CltjaWQ6aW1hZ2UwMDEucG5nQDAxRDdFMURFLjc3ODYyQkIwXQ0KDQpSZXRyb3V2ZXogVGhhbGVz
IHN1ciBsZXMgcsOpc2VhdXggc29jaWF1eCBldCBzdXIgd3d3LnRoYWxlc2dyb3VwLmNvbTxodHRw
czovL3d3dy50aGFsZXNncm91cC5jb20vPg0KDQoNCg0KDQoNCg0K

--_000_11692a929ee941eb8db9a1682ca42bd3thalesgroupcom_
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64

PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy
bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt
YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj
cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv
VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg
Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv
ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPCEtLVtp
ZiAhbXNvXT48c3R5bGU+dlw6KiB7YmVoYXZpb3I6dXJsKCNkZWZhdWx0I1ZNTCk7fQ0Kb1w6KiB7
YmVoYXZpb3I6dXJsKCNkZWZhdWx0I1ZNTCk7fQ0Kd1w6KiB7YmVoYXZpb3I6dXJsKCNkZWZhdWx0
I1ZNTCk7fQ0KLnNoYXBlIHtiZWhhdmlvcjp1cmwoI2RlZmF1bHQjVk1MKTt9DQo8L3N0eWxlPjwh
W2VuZGlmXS0tPjxzdHlsZT48IS0tDQovKiBGb250IERlZmluaXRpb25zICovDQpAZm9udC1mYWNl
DQoJe2ZvbnQtZmFtaWx5OiJDYW1icmlhIE1hdGgiOw0KCXBhbm9zZS0xOjIgNCA1IDMgNSA0IDYg
MyAyIDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpDYWxpYnJpOw0KCXBhbm9zZS0xOjIg
MTUgNSAyIDIgMiA0IDMgMiA0O30NCi8qIFN0eWxlIERlZmluaXRpb25zICovDQpwLk1zb05vcm1h
bCwgbGkuTXNvTm9ybWFsLCBkaXYuTXNvTm9ybWFsDQoJe21hcmdpbjowY207DQoJbWFyZ2luLWJv
dHRvbTouMDAwMXB0Ow0KCWZvbnQtc2l6ZToxMi4wcHQ7DQoJZm9udC1mYW1pbHk6IlRpbWVzIE5l
dyBSb21hbiIsc2VyaWY7fQ0KYTpsaW5rLCBzcGFuLk1zb0h5cGVybGluaw0KCXttc28tc3R5bGUt
cHJpb3JpdHk6OTk7DQoJY29sb3I6Ymx1ZTsNCgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lO30N
CmE6dmlzaXRlZCwgc3Bhbi5Nc29IeXBlcmxpbmtGb2xsb3dlZA0KCXttc28tc3R5bGUtcHJpb3Jp
dHk6OTk7DQoJY29sb3I6cHVycGxlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQ0KcC5t
c29ub3JtYWwwLCBsaS5tc29ub3JtYWwwLCBkaXYubXNvbm9ybWFsMA0KCXttc28tc3R5bGUtbmFt
ZTptc29ub3JtYWw7DQoJbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87DQoJbWFyZ2luLXJpZ2h0OjBj
bTsNCgltc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzsNCgltYXJnaW4tbGVmdDowY207DQoJZm9u
dC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZjt9DQpw
LmdtYWlsLXAxLCBsaS5nbWFpbC1wMSwgZGl2LmdtYWlsLXAxDQoJe21zby1zdHlsZS1uYW1lOmdt
YWlsLXAxOw0KCW1zby1tYXJnaW4tdG9wLWFsdDphdXRvOw0KCW1hcmdpbi1yaWdodDowY207DQoJ
bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87DQoJbWFyZ2luLWxlZnQ6MGNtOw0KCWZvbnQtc2l6
ZToxMi4wcHQ7DQoJZm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2VyaWY7fQ0KcC5tMzg2
MzUwMjE2NDQyOTQ5MTA2bXNvbGlzdHBhcmFncmFwaCwgbGkubTM4NjM1MDIxNjQ0Mjk0OTEwNm1z
b2xpc3RwYXJhZ3JhcGgsIGRpdi5tMzg2MzUwMjE2NDQyOTQ5MTA2bXNvbGlzdHBhcmFncmFwaA0K
CXttc28tc3R5bGUtbmFtZTptXzM4NjM1MDIxNjQ0Mjk0OTEwNm1zb2xpc3RwYXJhZ3JhcGg7DQoJ
bXNvLW1hcmdpbi10b3AtYWx0OmF1dG87DQoJbWFyZ2luLXJpZ2h0OjBjbTsNCgltc28tbWFyZ2lu
LWJvdHRvbS1hbHQ6YXV0bzsNCgltYXJnaW4tbGVmdDowY207DQoJZm9udC1zaXplOjEyLjBwdDsN
Cglmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZjt9DQpzcGFuLkVtYWlsU3R5bGUy
MA0KCXttc28tc3R5bGUtdHlwZTpwZXJzb25hbC1yZXBseTsNCglmb250LWZhbWlseToiQ2FsaWJy
aSIsc2Fucy1zZXJpZjsNCgljb2xvcjojMUY0OTdEO30NCi5Nc29DaHBEZWZhdWx0DQoJe21zby1z
dHlsZS10eXBlOmV4cG9ydC1vbmx5Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIixzYW5zLXNlcmlm
Ow0KCW1zby1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTO30NCkBwYWdlIFdvcmRTZWN0aW9uMQ0KCXtz
aXplOjYxMi4wcHQgNzkyLjBwdDsNCgltYXJnaW46NzAuODVwdCA3MC44NXB0IDcwLjg1cHQgNzAu
ODVwdDt9DQpkaXYuV29yZFNlY3Rpb24xDQoJe3BhZ2U6V29yZFNlY3Rpb24xO30NCi0tPjwvc3R5
bGU+PCEtLVtpZiBndGUgbXNvIDldPjx4bWw+DQo8bzpzaGFwZWRlZmF1bHRzIHY6ZXh0PSJlZGl0
IiBzcGlkbWF4PSIxMDI2IiAvPg0KPC94bWw+PCFbZW5kaWZdLS0+PCEtLVtpZiBndGUgbXNvIDld
Pjx4bWw+DQo8bzpzaGFwZWxheW91dCB2OmV4dD0iZWRpdCI+DQo8bzppZG1hcCB2OmV4dD0iZWRp
dCIgZGF0YT0iMSIgLz4NCjwvbzpzaGFwZWxheW91dD48L3htbD48IVtlbmRpZl0tLT4NCjwvaGVh
ZD4NCjxib2R5IGxhbmc9IkZSIiBsaW5rPSJibHVlIiB2bGluaz0icHVycGxlIj4NCjxkaXYgY2xh
c3M9IldvcmRTZWN0aW9uMSI+DQo8ZGl2Pg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25l
O2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2LjBw
dDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tcmlnaHQ6MGNtIj4NCjxkaXY+DQo8ZGl2Pg0KPGRp
dj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt
c28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+PHNwYW4gc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPkRl
YXIgYWxsLDwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxl
PSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+PHNw
YW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48
L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0
OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHls
ZT0iY29sb3I6IzJGNTQ5NiI+V2UgYXJlIGRldGVjdGluZzwvc3Bhbj48c3BhbiBsYW5nPSJFTi1V
UyIgc3R5bGU9ImNvbG9yOiMwMEIwNTAiPg0KPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHls
ZT0iY29sb3I6IzJGNTQ5NiI+c2V2ZXJhbCBpc3N1ZXMgd2l0aCBVU0IgcG9ydCB2aXJ0dWFsaXph
dGlvbiB3aXRoDQo8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdE
Ij50aGUgWGVuIGh5cGVydmlzb3I8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xv
cjojMkY1NDk2Ij4uPC9zcGFuPjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIg
c3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRv
O21hcmdpbi1sZWZ0OjM1LjRwdCI+DQo8c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMy
RjU0OTYiPi0gV2UgY2Fubm90IGRvIFBDSSBwYXNzdGhyb3VnaCBvZiB0aGUgUENJIHVzYiBidXMg
b24gYSBXaW5kb3dzIDEwIDE2MDcgNjQtYml0IHZpcnR1YWwgbWFjaGluZTwvc3Bhbj48c3BhbiBs
YW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPi4NCjwvc3Bhbj48c3BhbiBsYW5nPSJF
Ti1VUyIgc3R5bGU9ImNvbG9yOiMyRjU0OTYiPlRoZSBiYWQgcmVzdWx0IGlzIGEgV2luZG93cyBi
bHVlIHNjcmVlbi48L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBz
dHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87
bWFyZ2luLWxlZnQ6MzUuNHB0Ij4NCjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzJG
NTQ5NiI+LSBXaGVuIHdlIHVzZSB0aGUgcGFzc3Rocm91Z2ggZnVuY3Rpb25hbGl0eSBvbiBhIFdp
bmRvd3MgMjFIMiB2aXJ0dWFsIG1hY2hpbmUsIHdlIG5vdGljZSB0aGF0PC9zcGFuPjxzcGFuIGxh
bmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzAwQjA1MCI+DQo8L3NwYW4+PHNwYW4gbGFuZz0iRU4t
VVMiIHN0eWxlPSJjb2xvcjojMkY1NDk2Ij50aGUgc3BlZWQgb2YgdGhlIFVTQiBwb3J0IGlzIG5v
dCBoaWdoIHNwZWVkIGJ1dCBmdWxsIHNwZWVkIG9uIGEgVVNCIDMuMCBwb3J0PC9zcGFuPjxvOnA+
PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFs
dDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21hcmdpbi1sZWZ0OjM1LjRwdCI+DQo8
c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMyRjU0OTYiPi0gV2Ugbm90aWNlIGluc3Rh
YmlsaXRpZXMgd2hlbiB1c2luZyB0aGUgbmVjLXVzYi14aGNpIGRyaXZlciwgJm5ic3A7VVNCIDIu
MCBrZXlzIGFyZSBub3QgcmVjb2duaXplZCBieSB0aGUgV2luZG93cyB2aXJ0dWFsIG1hY2hpbmUg
KGluY29ycmVjdCBkZXNjcmlwdG9yKTwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv
bS1hbHQ6YXV0byI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4mbmJz
cDs8L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNv
LW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFuIGxh
bmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzJGNTQ5NiI+V2UNCjwvc3Bhbj48c3BhbiBsYW5nPSJF
Ti1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPm5lZWQ8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMi
IHN0eWxlPSJjb2xvcjojMDBCMDUwIj4NCjwvc3Bhbj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9
ImNvbG9yOiMyRjU0OTYiPnlvdXIgPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29s
b3I6IzFGNDk3RCI+aGVscDwvc3Bhbj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMy
RjU0OTYiPiB0byBmaW5kIGEgc29sdXRpb24gZm9yIHRoZXNlIHByb2JsZW1zLjwvc3Bhbj48bzpw
PjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1h
bHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0
eWxlPSJjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i
b3R0b20tYWx0OmF1dG8iPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+
VGhlIFNvZnR3YXJlIHVzZWQgYXJlIDoNCjwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNz
PSJtMzg2MzUwMjE2NDQyOTQ5MTA2bXNvbGlzdHBhcmFncmFwaCI+PHNwYW4gbGFuZz0iRU4tVVMi
IHN0eWxlPSJjb2xvcjojMUY0OTdEIj4tPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0i
Zm9udC1zaXplOjcuMHB0O2NvbG9yOiMxRjQ5N0QiPiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu
YnNwOyZuYnNwOyZuYnNwOw0KPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6
IzJGNTQ5NiI+RGViaTwvc3Bhbj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5
N0QiPmE8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMkY1NDk2Ij5uIDEx
IHZlcnNpb248L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4N
Cjwvc3Bhbj48c3BhbiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+NS4xMC4wLTIwPC9zcGFuPjxvOnA+
PC9vOnA+PC9wPg0KPHAgY2xhc3M9Im0zODYzNTAyMTY0NDI5NDkxMDZtc29saXN0cGFyYWdyYXBo
Ij48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPi08L3NwYW4+PHNwYW4g
bGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6Ny4wcHQ7Y29sb3I6IzFGNDk3RCI+Jm5ic3A7
Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7DQo8L3NwYW4+PHNwYW4gbGFuZz0i
RU4tVVMiIHN0eWxlPSJjb2xvcjojMkY1NDk2Ij5YZW4gdmVyc2lvbjwvc3Bhbj48c3BhbiBsYW5n
PSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPg0KPC9zcGFuPjxzcGFuIHN0eWxlPSJjb2xv
cjojMUY0OTdEIj40LjE0PC9zcGFuPjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Im0zODYzNTAy
MTY0NDI5NDkxMDZtc29saXN0cGFyYWdyYXBoIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNv
bG9yOiMxRjQ5N0QiPi08L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6
Ny4wcHQ7Y29sb3I6IzFGNDk3RCI+Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7
Jm5ic3A7DQo8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMkY1NDk2Ij5X
aW5kb3dzIDEwIDE2MDcgYW5kIDIxSDIgZm9yIHZpcnR1YWwgbWFjaGluZXMuIFZpcnR1YWwgTWFj
aGluZSBIVk08L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHls
ZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxz
cGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+Jm5ic3A7PC9zcGFuPjxvOnA+
PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFs
dDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5
bGU9ImNvbG9yOiMyRjU0OTYiPlRoYW5rcyBpbiBhZHZhbmNlPC9zcGFuPjxzcGFuIGxhbmc9IkVO
LVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+IGZvciB5b3VyIGhlbHAuPC9zcGFuPjxvOnA+PC9v
OnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDph
dXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9
ImNvbG9yOiMxRjQ5N0QiPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv
bS1hbHQ6YXV0byI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdEIj5CZXN0
IHJlZ2FyZHMsPC9zcGFuPjxvOnA+PC9vOnA+PC9wPg0KPHRhYmxlIGNsYXNzPSJNc29Ob3JtYWxU
YWJsZSIgYm9yZGVyPSIwIiBjZWxsc3BhY2luZz0iMCIgY2VsbHBhZGRpbmc9IjAiIHN0eWxlPSJi
b3JkZXItY29sbGFwc2U6Y29sbGFwc2UiPg0KPHRib2R5Pg0KPHRyPg0KPHRkIHdpZHRoPSI0ODAi
IHZhbGlnbj0idG9wIiBzdHlsZT0id2lkdGg6MzYwLjBwdDtib3JkZXI6bm9uZTtib3JkZXItYm90
dG9tOnNvbGlkICNCRkJGQkYgMS4wcHQ7cGFkZGluZzozLjBwdCAwY20gMS41cHQgMGNtIj4NCjxw
IGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFy
Z2luLWJvdHRvbS1hbHQ6YXV0byI+PGI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9u
dC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMjQyQTc1Ij4mbmJz
cDs8L3NwYW4+PC9iPjxvOnA+PC9vOnA+PC9wPg0KPC90ZD4NCjwvdHI+DQo8dHI+DQo8dGQgd2lk
dGg9IjQ4MCIgdmFsaWduPSJ0b3AiIHN0eWxlPSJ3aWR0aDozNjAuMHB0O3BhZGRpbmc6My4wcHQg
MGNtIDEuNXB0IDBjbSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10
b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxiPjxzcGFuIHN0eWxlPSJm
b250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMtc2VyaWY7
Y29sb3I6IzI0MkE3NSI+Jm5ic3A7PC9zcGFuPjwvYj48bzpwPjwvbzpwPjwvcD4NCjwvdGQ+DQo8
L3RyPg0KPHRyPg0KPHRkIHdpZHRoPSI0ODAiIHZhbGlnbj0idG9wIiBzdHlsZT0id2lkdGg6MzYw
LjBwdDtwYWRkaW5nOjMuMHB0IDBjbSAxLjVwdCAwY20iPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIg
c3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRv
Ij48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlh
bCZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMyNDJBNzUiPkRvbWluaXF1ZSBMQVJSSUVVDQo8L3Nw
YW4+PC9iPjxvOnA+PC9vOnA+PC9wPg0KPC90ZD4NCjwvdHI+DQo8dHI+DQo8dGQgd2lkdGg9IjQ4
MCIgdmFsaWduPSJ0b3AiIHN0eWxlPSJ3aWR0aDozNjAuMHB0O3BhZGRpbmc6Ljc1cHQgMGNtIC43
NXB0IDBjbSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0
OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxiPjxzcGFuIHN0eWxlPSJmb250LXNp
emU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6
Izc5Nzk3OSI+Q3liZXIgUHJvamVjdCBNYW5hZ2VyPC9zcGFuPjwvYj48bzpwPjwvbzpwPjwvcD4N
CjwvdGQ+DQo8L3RyPg0KPHRyIHN0eWxlPSJoZWlnaHQ6MTcuNTVwdCI+DQo8dGQgd2lkdGg9IjQ4
MCIgdmFsaWduPSJ0b3AiIHN0eWxlPSJ3aWR0aDozNjAuMHB0O3BhZGRpbmc6Ljc1cHQgMGNtIDcu
NXB0IDBjbTtoZWlnaHQ6MTcuNTVwdCI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNv
LW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxiPjxzcGFu
IHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNh
bnMtc2VyaWY7Y29sb3I6Izc5Nzk3OSI+VGhhbGVzPC9zcGFuPjwvYj48bzpwPjwvbzpwPjwvcD4N
CjwvdGQ+DQo8L3RyPg0KPHRyIHN0eWxlPSJoZWlnaHQ6Ljg1cHQiPg0KPHRkIHdpZHRoPSI0ODAi
IHZhbGlnbj0idG9wIiBzdHlsZT0id2lkdGg6MzYwLjBwdDtib3JkZXI6bm9uZTtib3JkZXItdG9w
OnNvbGlkICMwMEJCREQgMi4yNXB0O3BhZGRpbmc6Ny41cHQgMGNtIDMuNzVwdCAwY207aGVpZ2h0
Oi44NXB0Ij4NCjwvdGQ+DQo8L3RyPg0KPHRyIHN0eWxlPSJoZWlnaHQ6My4wcHQiPg0KPHRkIHdp
ZHRoPSI0ODAiIHZhbGlnbj0idG9wIiBzdHlsZT0id2lkdGg6MzYwLjBwdDtwYWRkaW5nOjMuNzVw
dCAwY20gNy41cHQgMGNtO2hlaWdodDozLjBwdCI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHls
ZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxz
cGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7
LHNhbnMtc2VyaWY7Y29sb3I6Izc5Nzk3OSI+NCwgYXZlbnVlIGRlcyBMb3V2cmVzc2VzDQo8L3Nw
YW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdp
bi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFuIHN0eWxlPSJm
b250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMtc2VyaWY7
Y29sb3I6Izc5Nzk3OSI+OTIyMzAgR2VubmV2aWxsaWVycywgRnJhbmNlPC9zcGFuPjxvOnA+PC9v
OnA+PC9wPg0KPC90ZD4NCjwvdHI+DQo8dHI+DQo8dGQgd2lkdGg9IjQ4MCIgdmFsaWduPSJ0b3Ai
IHN0eWxlPSJ3aWR0aDozNjAuMHB0O2JvcmRlcjpub25lO2JvcmRlci10b3A6c29saWQgIzAwQkJE
RCAyLjI1cHQ7cGFkZGluZzoxMS4yNXB0IDBjbSAzLjBwdCAwY20iPg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFs
dDphdXRvIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtB
cmlhbCZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiM3OTc2NzYiPjxpbWcgd2lkdGg9IjE2MCIgaGVp
Z2h0PSIzNiIgc3R5bGU9IndpZHRoOjEuNjY2NmluO2hlaWdodDouMzc1aW4iIGlkPSJtXzM4NjM1
MDIxNjQ0Mjk0OTEwNkltYWdlX3gwMDIwXzEiIHNyYz0iY2lkOmltYWdlMDAxLnBuZ0AwMURBNjU5
Ny4yRjU1N0UxMCIgYWx0PSJjaWQ6aW1hZ2UwMDEucG5nQDAxRDdFMURFLjc3ODYyQkIwIj48L3Nw
YW4+PG86cD48L286cD48L3A+DQo8L3RkPg0KPC90cj4NCjx0cj4NCjx0ZCB3aWR0aD0iNDgwIiB2
YWxpZ249InRvcCIgc3R5bGU9IndpZHRoOjM2MC4wcHQ7cGFkZGluZzo1LjI1cHQgMGNtIDUuMjVw
dCAwY20iPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDph
dXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXpl
OjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6Izc5
NzY3NiI+UmV0cm91dmV6IFRoYWxlcyBzdXIgbGVzIHLDqXNlYXV4IHNvY2lhdXggZXQgc3VyDQo8
L3NwYW4+PC9iPjxzcGFuIHN0eWxlPSJmb250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7
QXJpYWwmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojNzk3Njc2Ij48YSBocmVmPSJodHRwczovL3d3
dy50aGFsZXNncm91cC5jb20vIiB0YXJnZXQ9Il9ibGFuayI+d3d3LnRoYWxlc2dyb3VwLmNvbTwv
YT48L3NwYW4+PG86cD48L286cD48L3A+DQo8L3RkPg0KPC90cj4NCjx0cj4NCjx0ZCB3aWR0aD0i
NDgwIiB2YWxpZ249InRvcCIgc3R5bGU9IndpZHRoOjM2MC4wcHQ7cGFkZGluZzo1LjI1cHQgMGNt
IDUuMjVwdCAwY20iPg0KPC90ZD4NCjwvdHI+DQo8dHI+DQo8dGQgd2lkdGg9IjQ4MCIgdmFsaWdu
PSJ0b3AiIHN0eWxlPSJ3aWR0aDozNjAuMHB0O3BhZGRpbmc6MTAuNXB0IDBjbSAxMC41cHQgMGNt
Ij4NCjwvdGQ+DQo8L3RyPg0KPC90Ym9keT4NCjwvdGFibGU+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
IiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1
dG8iPjxzcGFuIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48L286cD48
L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87
bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFuIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4m
bmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0i
bXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFu
IHN0eWxlPSJjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+DQo8L2Rp
dj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ib2R5
Pg0KPC9odG1sPg0K

--_000_11692a929ee941eb8db9a1682ca42bd3thalesgroupcom_--

--_004_11692a929ee941eb8db9a1682ca42bd3thalesgroupcom_
Content-Type: image/png; name="image001.png"
Content-Description: image001.png
Content-Disposition: inline; filename="image001.png"; size=3376;
	creation-date="Thu, 22 Feb 2024 12:58:18 GMT";
	modification-date="Thu, 22 Feb 2024 12:58:18 GMT"
Content-ID: <image001.png@01DA6597.2F557E10>
Content-Transfer-Encoding: base64

iVBORw0KGgoAAAANSUhEUgAAAKAAAAAkCAYAAAAO7jHjAAAACXBIWXMAAAsTAAALEwEAmpwYAAAA
AXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAzFSURBVHgB7VzNUhtJEs6sbmHHXtA+AfJtbP6a
094W6QlgrhMzg4jYu2WIiTkCx401RjwBkj2xsTeLJ5DY80bQAmHPzc0TWOxsbNi0unIzq7tF6w8k
BnYYS1+oQ1L9V1dWVlZmVuEz5wfHgsAhDTOEkEHANIB8g3wD8W/oA473ovjomzwk8FBhPQDlvXP/
5vbL5ziFtAZrLSByuC6pJ4MIxRN3dx+GxLyzucoV7kV1NxVCTYF96Lp/9eAWcJwfMz75W30jUdXP
3JdFuCVmFzcOOgIImmcnr17AHSEeP9CwqBHS8TsNqxpq7JpIJOPmaQjqDbdYgyEg4xiAleWyZpJj
yVFpimhnUH1A5AqdWGC7tiJ9TJIk/LRBNzQg0blMWElYAJF8BR4HPemXLwjUKikoYlSb1EOa0jAK
AkqTwkxctyZY1eRLGTtwC7SotcftWe0byR2acwrusAPTDS4331Ecksdfd0aA/cZv1LEj5gBkMlnw
dGkz8/5493xQXocJvkV6r0WQjetKjuUw9QHiqtTXghYzjjHHnLOZ5a/VaxORtQUTmJUiIF3ln1m4
I9gw5mCx4aB75hpxAjCTCMoyF8zelgt+KQjAz1IoosVooiyptwS/96bdqO/idYnmFja2mGVud2Qk
yJ+d7Jbhd455ZyPPfckkw5j4SohY5h/VjsQhF6zBA4dMnrP6qydwD5B9AnRSi3ta383Br8B4L8GE
PUtrgHqn4e7WoJfYDBeECe4UY0uAhvtBL/f72S165g/2bmiQrAOY4E7xIGRAXvIcIYihMxAuw6/F
AO4X/xYuOLe4WYOEwC0EO+tsFM7cV7dWy9w3RI3Wo/rpl47QbZwMr/oaAIffUXWUDLG6zlapsqjN
HgQBkuxCCVfh/wQhon6yX5v7xRAuSJ07Ph64LdaBlVy3eGvh+56R7lb99AUaEWMkAmTCOadOGVA2
JFkYAbG6jlVfBdZh5sZuCRZVgiJ83h2e5H4xBsiC6ZZWz2EMYVm6IhwM7gZp1mHujR0BsvJzoOyX
rx6n89VGPnzeh2n6yIIsMxTEEgBjBuH6PFFl11uDu4HzMJZgoqL2cWi5ykrxks3mOxgRwv2Y9feV
/YTgNIiSlTJRKHxXbbx4k5srdsuCcMUFb2V5uU/cpxpGEIkpRvXy9OnmzCh5bZtybMFKyqfph7EJ
YYXk+/eDzT/dmJ/fuBDz0agI+hBfzP3+hH+pQpdcyDVsMVcs/evFTz2yYMgFfyzf1v78JWCUMROw
1cnrtteNzRIs3I/7nu8Oj2U/or7CdFpbU8uDZMF+BD3BaBgbU9x13C/6K7vadG8aDHe7fXbEQtCs
nC6PaqL7yilkYET07NC/EIwFAUayX747PLnzJQh2EKy9riTumz8/O5If/fSCYcbRTHRiY7bJ+gAj
4iYvlWT584ubI5cfI8Dg63dusa8r3YKz+ZxXigLcFtTpMcO/3bEgwCG4H7zJLRa/q56wcV02F5jW
AKX/wGWHngw1FklRtquoB+eoMMgPcBhYAU4PitPiNoeYgTsCElW+eAIUdytxnugO76f3e5NbKPFX
aVBZp6cvD9nK0O0p87txVHhAqPGYlM5OXpVvJMCAsK6QSh1hWo+0+0nCt3TN1mqbZ5IDRmsPmbac
NTTUR2R1A4Vym8dqHI+UdTgotah58Kq+NAdUfq7fTqYiDTtKgeGo4uEbmpbQHZQetcoSaofTZZJt
GMZbOS4bSXvv63vnQ5SficrPRGXcVEf7/fGk8qSeXyyqwwC0lC6nAnRJ+pEYvwF1GVetq34AP+Ry
+Ufew7UiTTDBBBNMMMEEE0wwwQQTDEDoOXI8tEdIlD4DE0yQgP199bTCG/OVRFgTlSqVl59de3ZV
Q0o8YTP8/PH7akPOpnpvcrNfJ9N8Wz1ZVaDetgCWSMEaqzAK31QbS3/Pzblwjwg9W4JjMOoOrHS3
qzOtTKJUpnTPbXroMFYOHkMLWkctsg4a9d2OdybuZ3fphBuXp1h3JJrvJijcNo/obrQufFs9zV5X
AIHeBxWbZeSoXu/hcrYqmDDLUtO+hrKU/1/wPbhnaKXXQPR9ShXQGqxYFhCm3rLV4y2MOVgx7Ojw
ggCetOR0xzNRVtkcOJL71SCwqfDA15ZherEiuvl6edZYBr6rNs5ZgXigFM4IEVqIa5/J3/lHbsnj
uLxCWC5n59ajqxiEeDuOZ+arDYcQjMewTrg3KdAZJJWRHgpntFBdcV1krqut/VLuqWfyM7dkyuC8
1Azj/J1SbqkZln+cZqJp22xZKeopfbkfx5uJo8m497PiMxvooMxcfhsVTpeX5wxXX6s1Dgitmtb+
BYX1pCUMyX8hZUvc6+Wn5bgvPsH+FE8cDXaBJ+xMWLbPZdkcj2vdfYjbNrew+dxWQVlmujnUDS3n
1N2tmEFwNvIBWK5cYSLcJxp8iM9KJN+pOTyvKToHg005y2EOVckxSfNy9ZGYAqUODa2VuCweQ09B
cJjkXMYurv21KL45zJUmchWKHGGwAypwWw5tCFy5XkXqkSs2IoV27EFu0sfXsyTbGV/bQqQdREhz
uovYHavtCcxEZYjnUvtHQmRixkpZj0wBzC6XY7MWym/de6OABnxL5qqMq/tlws5ai5LXmno0baG1
Ir8ljTyyNBMGW+38WuIoLbNS4gLr0WJHHVE+Q+CatrWaarvIo2VRIh30a6vUjRRko3/pZNruOPlv
K1hsgcVhuGU4hZlYj9PJvpo2G4fWBBBWWfww3CTQvhjyD67agM+toDVtCJHrULz8yTRtUes4WYQQ
THhGWYhFrBWSTgqkj2EetiKR9Vbs0S1oyTvZlnB5+He++/jAJ/gMifgs13fjAaaYwGSJ1qCaLc3E
x/2WMgiCppkc7QliEq6g1otCiJxuK65P0mrwpSy2kFBT/rcJkF/mljzSZ66w9hgew6gI3dgpw5Ra
fJOby9ENh14kjTwQvozMMPmF01ngf83PujwSxpwu0y5TvFcUGi4zNYWFn3ILlUH1SxzPRJH9mlzf
esxF+yEWJ8hShbDNrQyYZR6L0g4m8oq0PbkxE9MTaMtMHjFfiWkqcbbYOT3dO+KIFR6w4snJbrlx
8tKsQsmlTrgmf9WE60kasUdLOOc9DPOYk20uacvkkYGVcHlQUzkymbUhDhhxvI3BC4KrdzcIwtmk
7YEFlfDSKRJzZs3Uf40TBhlzHE8cZR/FaaV+YwokrMn/eAn2XudmjRu3LGFMrVWtaIXZ0QX8Jhh8
8EWWb56F0awVGzLBbwrhwJDajm73AeHwEF9XQVAnRcsicLeEmzN3Ygpe5GVM4mphEkizWLM9t7iZ
l/89V12Yi5hC222MyL2szblCe/pAdMjm0XIqjKbZoqgF9wTjwrawWfLBr3L/PBvt9e7lvscZIfb0
QJbZuGn18Pdot1eNmr4nP2EWumTLGJZSspymX+fmjE8+78CHeIOGUI3a6Doul0g/RPsjBwreWMXy
c09bLV3TpLZ4+TZcjNe+Osjga/OOzK5buIE2niGDrzrBrvZEXBEa0bUYItTDkGDie87LY0G4L0+M
jE+qCveIiEPvzy5sFIQQoevWtJgAM90DyUtGzQI61/InoMpa7Uy8Jq4dGBHAeaNS4xdc4PSrN6Xv
l5/b4XK+PH+v9ktDLMSKpMjxH3nZaxLdTH8ky6zIapD6wO3qyiNyFGU5/IOdwuzlpa7xKK3K/5BB
0IC2zrncBk84IKfNh0VRJd7oCFj493jmpw3REVR4yXN9UOK1wrIRlqLiXeGAvBGR3eeFyFmxMC+Q
o5C8A92TO3pQwbkQfiSTOaHqhPMQhAQ+BIyHCmsHWPZ8wivJjLzLYWGRXnecHyotHXSE8wQ6txRu
cXtEtruINpDmKrdA7i40HcVFneDuLPqs8kpwzhsb2uENQuVKjcIdxCtP4G+qx0u2mloxw4CqGc98
k4/VK2HLoBDfEcdUvp4SgR1ZJonS/xJ8qv8BUhe2srzg8tNFAHaZ5YLaVc+u8l+yfCf5UVkzpEk2
MvlkZ18vP9v//p9nnvQvbJPl+TrocCFqaTqM6wrzzO5wHrdfns90uTOlpjxxCQsuP1/4AOtT6hH3
N3ofnPZSfzLvYko9zvvBp7ZbFLc1d5UWm1rrHlcmftHr3I/FltKHLPM0n81v7HDYdMzxzI52/oWn
Y/cmwo/J/LKD/copLNmkRI86I/GN05c1zpOP87AgX/y3Cg6lwYG2C3FecX1LBXZHeT7LfbY2m8AZ
kRcVT/aQPqAcYOCl5MJPrXq8njlfztZq7VJ2DV2ub+9OX5W4PR+v2kOlQAVHTHzAdU1HtOVy2E5U
1o6UxRuxmdGPlt0zkioa2WHyV5oH+omogWCCLw4P8FQccxIK1SL8qfCuMzshvi8X/wPlgS2kdDEs
VgAAAABJRU5ErkJggg==

--_004_11692a929ee941eb8db9a1682ca42bd3thalesgroupcom_--


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 14:33:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 14:33:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684443.1064290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdA8f-00065l-Kq; Thu, 22 Feb 2024 14:33:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684443.1064290; Thu, 22 Feb 2024 14:33:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdA8f-00065e-IG; Thu, 22 Feb 2024 14:33:29 +0000
Received: by outflank-mailman (input) for mailman id 684443;
 Thu, 22 Feb 2024 14:33:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aBgv=J7=cloud.com=george.dunlap@srs-se1.protection.inumbo.net>)
 id 1rdA8e-00065Y-JU
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 14:33:28 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 577c7514-d18f-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 15:33:27 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-512b13bf764so5827964e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 06:33:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 577c7514-d18f-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708612407; x=1709217207; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iWO4O0D1YfqyoCsIR6LchQGI1Au4b7dSpCaMRlBeBaM=;
        b=KM/FROad2G20+ed4DwEcNFa9wO2vzU6LhZmU2XxoeXf8gNtKjbuN2eMGkf0MeS5XyP
         wP38yywrUwzcdoik9yO9+9rGC3RPAMTHPoZmVZoasAZUW8Bf1u8q4Lmp2a3XlxVGN6ff
         /zrhkTH2a3HjM2DcQMmsvgCXf1LHzMuRIE77A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708612407; x=1709217207;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=iWO4O0D1YfqyoCsIR6LchQGI1Au4b7dSpCaMRlBeBaM=;
        b=bRCTCzQQ80r0e6xDEDgFaVCixMj6TroS8DcH29gZM2Mcbm6lEWJ2vTzx2UfqEclJdH
         g8N1BQ6U1IUsMZqSXaTq+IL20HS3b5mQP3rCzkNjpacd7JyLFAubcW4DWMijJtjM355q
         arIxLaP/OV2rBtHsjtJ+zHQV+8C0J3jGqMV2+z0zrIouMUY/DI+5lgchn29PTxG+K16w
         xWLeuV1nuhYAp8zjZYXGWtuK82shJsgsV2S5ONIpw1zqIU+uTYULE9oQSsH2qPGCeqNt
         TwF+kqvA5VvJ6+z4d9aW2Q1ah4q491LjoW8z1AtQubJVEZEOdJnIbON9Or483rS6Dyfs
         qB/g==
X-Forwarded-Encrypted: i=1; AJvYcCUdEL8M5ElFJ7GArimeDHGdPCDMPAua5pzYAoE+trYi3d7fQkSoyb2sKULnV46Ev3e7lpMjQks1eWBLHk6c5rTDLnrGe5aE/dNMA6khz0g=
X-Gm-Message-State: AOJu0YzaqyuMJt9Lma7wGXI/RksBqysdrrpSTtwMZtUR9Ba/YsZ0OhTb
	lbAXLd2SyKpgBvE8c+0KgdBqco6hIV4bHAQvv8uRNiHCy6TYT0hLiMhLmwZAqi9RbOM+ie2Kw6J
	ICDxAv6HNC55+K2Bv30YrnTRKFzL5epnhPMP8sQ==
X-Google-Smtp-Source: AGHT+IFS17Htyjz+pfob5P7AjVa74GpQ4vlmDCG+qdnfsml0zv30QdGqOt+wc3EaC4FRCBsiXMoQ07G/6TPszxz8WeQ=
X-Received: by 2002:a05:6512:2313:b0:512:cb9f:b279 with SMTP id
 o19-20020a056512231300b00512cb9fb279mr6169771lfu.66.1708612406838; Thu, 22
 Feb 2024 06:33:26 -0800 (PST)
MIME-Version: 1.0
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-7-george.dunlap@cloud.com> <8ee35193-9715-4c4b-a1cb-11af04c4dab9@suse.com>
In-Reply-To: <8ee35193-9715-4c4b-a1cb-11af04c4dab9@suse.com>
From: George Dunlap <george.dunlap@cloud.com>
Date: Thu, 22 Feb 2024 22:33:15 +0800
Message-ID: <CA+zSX=YuLmZRz=1wwV=svOrhfQ66E-CZ9TEtqNCVM++JVQ5CzQ@mail.gmail.com>
Subject: Re: [PATCH 6/6] svm/nestedvm: Introduce nested capabilities bit
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>, 
	xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 20, 2024 at 12:25=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wr=
ote:
>
> On 06.02.2024 02:20, George Dunlap wrote:
> > --- /dev/null
> > +++ b/docs/designs/nested-svm-cpu-features.md
> > @@ -0,0 +1,110 @@
> > +# Nested SVM (AMD) CPUID requirements
> > +
> > +The first step in making nested SVM production-ready is to make sure
> > +that all features are implemented and well-tested.  To make this
> > +tractable, we will initially be limiting the "supported" range of
> > +nested virt to a specific subset of host and guest features.  This
> > +document describes the criteria for deciding on features, and the
> > +rationale behind each feature.
> > +
> > +For AMD, all virtualization-related features can be found in CPUID
> > +leaf 8000000A:edx
> > +
> > +# Criteria
> > +
> > +- Processor support: At a minimum we want to support processors from
> > +  the last 5 years.  All things being equal, older processors are
> > +  better.
>
> Nit: Perhaps missing "covering"? Generally I hope newer processors are
> "better".

I've reworded it.

> > +  For L0 this is required for performance: There's no way to tell the
> > +  guests not to use the LBR-related registers; and if the guest does,
> > +  then you have to save and restore all LBR-related registers on
> > +  context switch, which is prohibitive.
>
> "prohibitive" is too strong imo; maybe "undesirable"?

Prohibitive sounds strong, but I don't think it really is; here's the
dictionary definition:

"(of a price or charge) so high as to prevent something being done or bough=
t."

The cost of saving the registers on every context switch is so high as
to prevent us from wanting to do it.

I could change it to "too expensive".

> > --- a/xen/arch/x86/hvm/svm/nestedhvm.h
> > +++ b/xen/arch/x86/hvm/svm/nestedhvm.h
> > @@ -35,6 +35,7 @@ enum nestedhvm_vmexits
> >  nestedsvm_check_intercepts(struct vcpu *v, struct cpu_user_regs *regs,
> >      uint64_t exitcode);
> >  void svm_nested_features_on_efer_update(struct vcpu *v);
> > +void __init start_nested_svm(struct hvm_function_table *svm_function_t=
able);
>
> No section placement attributes on declarations, please.

Ack.

> > --- a/xen/arch/x86/hvm/svm/nestedsvm.c
> > +++ b/xen/arch/x86/hvm/svm/nestedsvm.c
> > @@ -1666,3 +1666,17 @@ void svm_nested_features_on_efer_update(struct v=
cpu *v)
> >          }
> >      }
> >  }
> > +
> > +void __init start_nested_svm(struct hvm_function_table *svm_function_t=
able)
> > +{
> > +    /*
> > +     * Required host functionality to support nested virt.  See
> > +     * docs/designs/nested-svm-cpu-features.md for rationale.
> > +     */
> > +    svm_function_table->caps.nested_virt =3D
> > +        cpu_has_svm_nrips &&
> > +        cpu_has_svm_lbrv &&
> > +        cpu_has_svm_nrips &&
>
> nrips twice? Was the earlier one meant to be npt?

Er, yes exactly.



> > --- a/xen/arch/x86/hvm/vmx/vmx.c
> > +++ b/xen/arch/x86/hvm/vmx/vmx.c
> > @@ -3021,6 +3021,9 @@ const struct hvm_function_table * __init start_vm=
x(void)
> >      if ( cpu_has_vmx_tsc_scaling )
> >          vmx_function_table.tsc_scaling.ratio_frac_bits =3D 48;
> >
> > +    /* TODO: Require hardware support before enabling nested virt */
> > +    vmx_function_table.caps.nested_virt =3D vmx_function_table.caps.ha=
p;
>
> This won't have the intended effect if hap_supported() ends up clearing
> the bit (used as input here) due to a command line option override. I
> wonder if instead this wants doing e.g. in a new hook to be called from
> nestedhvm_setup(). On the SVM side the hook function would then be the
> start_nested_svm() that you already introduce, with a caps.hap check
> added.

I take it presmp_initcall()'s are guaranteed to run before __initcall()'s?

If so that's probably a good idea.

> Since you leave the other nested-related if() in place in
> arch_sanitise_domain_config(), all ought to be well, but I think that
> other if() really wants replacing by the one you presently add.

Ack.

I'll probably check in patches 1,2,3, and 5, and resend the other two,
unless you'd like to see all the changes again...

 -George




 -George


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 14:37:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 14:37:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684448.1064301 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdACr-0006ea-5Q; Thu, 22 Feb 2024 14:37:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684448.1064301; Thu, 22 Feb 2024 14:37:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdACr-0006eT-1O; Thu, 22 Feb 2024 14:37:49 +0000
Received: by outflank-mailman (input) for mailman id 684448;
 Thu, 22 Feb 2024 14:37:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DiTz=J7=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rdACq-0006eN-1u
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 14:37:48 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f0f3068c-d18f-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 15:37:46 +0100 (CET)
Received: from SJ0PR03CA0079.namprd03.prod.outlook.com (2603:10b6:a03:331::24)
 by MN2PR12MB4502.namprd12.prod.outlook.com (2603:10b6:208:263::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.24; Thu, 22 Feb
 2024 14:37:41 +0000
Received: from SJ5PEPF000001D0.namprd05.prod.outlook.com
 (2603:10b6:a03:331:cafe::82) by SJ0PR03CA0079.outlook.office365.com
 (2603:10b6:a03:331::24) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.43 via Frontend
 Transport; Thu, 22 Feb 2024 14:37:40 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SJ5PEPF000001D0.mail.protection.outlook.com (10.167.242.52) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 22 Feb 2024 14:37:39 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 22 Feb
 2024 08:37:38 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 22 Feb
 2024 08:37:38 -0600
Received: from [172.30.63.250] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 22 Feb 2024 08:37:37 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0f3068c-d18f-11ee-8a55-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Y0PXmEDdcmKPjQ8np0+yfFpqyDkZ0J5garsEBqTk9xXn/g30UfRaE4VWqv98+tjORO35oQ/N8UPpsL1cYbUPN9xFDZBVbuiYdiHDyt0F2IO/7yplOxFi4B2pyEzYp+25jduZ6avVtnpNGDTbAoptD3OPj3rf9n38TGoQYEQgsYv/DxWhozT8GKr8V9K3ywG63Sw8rjCTu6m44w0RStnbhWYHqluen1Wf3SsGDDB/i/iaT18vEkzQeN2OeUTOdUUG6pATZMPiHkhpd7hcPJ3rqMz5KMAeWCnjBUwrxdGt/YY8/eRTWdsbF+5sb8CO9JD8iUZeQ1fklQj1jeErd1C0Tw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=JL/sZFRb6nwZUYNapXnnprE+amw5XNd3uGcjchCdbqo=;
 b=IioYlcoRCLeMni+/InoQdXWfrecvdDCyv8DR22H/eSau+ILHDCHqg89taE42SJYuX9AyEVbL4+4+RyzUwNscPnQOJkMfVshoydQK+KNEHDXz2BuBQwR4x0sAEnQ9RmHcCGnTOnOC5fHBqWsyfaPZX19VlOgUMhNqDAiriD9/PTosWsPwylYyehHUed4c0erGkVYe71KF3AuMErq21YGF+BIKlozXPLYqn4c+e/e4HC4mHvV2v3+qrV96b1xVIeXBgn/Md61IgwDQZNjzYaLRRQRgN4aHZYJkC4TQnYRoBX2OB1glWTNt1BUEHehrwatWSn6h8Wv5QHPYDcFhaem1cA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=JL/sZFRb6nwZUYNapXnnprE+amw5XNd3uGcjchCdbqo=;
 b=AqNgnSCBJ23yWuIwwgraVqW+OkrVeWKQR6qWdPbkCTRJW+91FNmjiPBaMcWohAAeSBu45zS62AB2zVAX4+QEC06D4oMMGaeqiHIU/Yb9J1nnS2APJNed7d4ETrf2JfoVwHokc/eNEVzRxZ+ZhW8ozLUN8xU/F9TlI4qv1sLv3eg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <a159e7ec-18b7-4898-8274-e02dc4810d54@amd.com>
Date: Thu, 22 Feb 2024 09:37:31 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC XEN PATCH v5 1/5] xen/vpci: Clear all vpci status of device
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Jan Beulich <jbeulich@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Wei Liu <wl@xen.org>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Anthony PERARD <anthony.perard@citrix.com>, Juergen
 Gross <jgross@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>,
	"Huang, Ray" <Ray.Huang@amd.com>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-2-Jiqian.Chen@amd.com>
 <9b4ce4c7-6f4e-47ec-b3a4-0b417cd2660c@amd.com>
 <BL1PR12MB58493E40588F4A7BF351B596E7562@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Language: en-US
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <BL1PR12MB58493E40588F4A7BF351B596E7562@BL1PR12MB5849.namprd12.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ5PEPF000001D0:EE_|MN2PR12MB4502:EE_
X-MS-Office365-Filtering-Correlation-Id: 18eca679-62dc-49f8-bbe5-08dc33b3d261
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	CsNu8k/ybg0UvVhj9PEU0rhTKXvOiK/l9MK0LWOxa2NZ0xC6/M72vPXBJy9XBpbOyjsY8xzZpiBKRipoRio+DlGBzTIcNd7dl5btSo2mijZofF810C+hMmsoCMWihWjvqmXLBVMK/5llXa6/k+DF2/+fbiKxXlM62DfBfj6EKUPNLsagidQz1tKD6shUOoyce1oDH+rsfjSmHoglKQvWbKxPL5sZsr2u/IhnGDlz3/mczUXO3Mwhu4dgMTkd83ixuZrl13fTMgJDWxfDX7IDtJlgkSQLrD2N+fZzcaJ0wDSonLDi+FH5ya9nQf5/IP9zL5U/LUBLgfEr6dch9tpg+xyPkbQectjZLh2rTmdruikw5qdLXxuMCCxi2fIB7GtiWrlpTStS95OjwYUgznygyk1OUq9X7bvrbLLtJxrZsUVsp3J/R+hESEGCb8KkD8SKCYM8n62Vo+APkqErfG5Kazk30RZiBVOj9FGLkTnIwwZhUFubUK7cC+bMVqC+P3z9UY6kIMlF40Nsmk6EocKpAz4/MjJyg3nZnVcu1EMHi+cCSu5erXZm1ggdNAnHlQUB3XjfYpT8XTMNHjjpbpkMMJIBfA6szdC3YEWKCxycHD+J8mtGAA9IkvAoCwEC9VL9S5Sn6k9FV7wx9WbmEkjARw==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(46966006)(40470700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Feb 2024 14:37:39.9519
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 18eca679-62dc-49f8-bbe5-08dc33b3d261
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ5PEPF000001D0.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4502

On 2/22/24 01:22, Chen, Jiqian wrote:
> Hi Stewart,
> 
> On 2024/2/10 02:02, Stewart Hildebrand wrote:
>> On 1/12/24 01:13, Jiqian Chen wrote:
>>> When a device has been reset on dom0 side, the vpci on Xen
>>> side won't get notification, so the cached state in vpci is
>>> all out of date compare with the real device state.
>>> To solve that problem, add a new hypercall to clear all vpci
>>> device state. When the state of device is reset on dom0 side,
>>> dom0 can call this hypercall to notify vpci.
>>>
>>> Co-developed-by: Huang Rui <ray.huang@amd.com>
>>> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
>>
>> Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> Thanks, I will add in next version.
> 
>>
>> If you send another version, the RFC tag may be dropped.
> Does only this one patch, or all patches of this series, need to drop RFC tag?

In my opinion at least patches 1, 2, and 3. If you decide to retain the
RFC tag on e.g. patches 4 and/or 5, you may want to include after the
commit description a scissors line --- followed by a brief explanation
for the RFC tag. For example:

---
RFC: discussions ongoing on the Linux side where/how to expose the gsi

>>
>> One thing to keep an eye out for below (not requesting any changes).
> Thanks for reminding me, I will always keep rebasing my code from latest staging branch before sending new version.
> 
>>
>>> ---
>>> diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c
>>> index 72ef277c4f8e..c6df2c6a9561 100644
>>> --- a/xen/drivers/vpci/vpci.c
>>> +++ b/xen/drivers/vpci/vpci.c
>>> @@ -107,6 +107,16 @@ int vpci_add_handlers(struct pci_dev *pdev)
>>>  
>>>      return rc;
>>>  }
>>> +
>>> +int vpci_reset_device_state(struct pci_dev *pdev)
>>> +{
>>> +    ASSERT(pcidevs_locked());
>>> +    ASSERT(rw_is_write_locked(&pdev->domain->pci_lock));
>>> +
>>> +    vpci_remove_device(pdev);
>>> +    return vpci_add_handlers(pdev);
>>
>> Note that these two functions may be renamed soon by the patch at [1].
>> Whichever patch goes in later will need to be rebased to account for the
>> rename.
>>
>> [1] https://lists.xenproject.org/archives/html/xen-devel/2024-02/msg00134.html
>>
>>> +}
>>> +
>>>  #endif /* __XEN__ */
>>>  
>>>  static int vpci_register_cmp(const struct vpci_register *r1,
>>
> 



From xen-devel-bounces@lists.xenproject.org Thu Feb 22 14:43:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 14:43:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684455.1064311 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdAIH-0008Tb-OU; Thu, 22 Feb 2024 14:43:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684455.1064311; Thu, 22 Feb 2024 14:43:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdAIH-0008TU-Lb; Thu, 22 Feb 2024 14:43:25 +0000
Received: by outflank-mailman (input) for mailman id 684455;
 Thu, 22 Feb 2024 14:43:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=6628=J7=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rdAIG-0008TO-Fi
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 14:43:24 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b9f4828c-d190-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 15:43:22 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 849064EE0737;
 Thu, 22 Feb 2024 15:43:21 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b9f4828c-d190-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Thu, 22 Feb 2024 15:43:21 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, julien@xen.org, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, George Dunlap <george.dunlap@citrix.com>, Wei
 Liu <wl@xen.org>, xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
In-Reply-To: <4b121e48-9541-4b53-8352-939c904f4f1c@suse.com>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
 <d90d98b6-508b-4a2a-ab6a-74a9828a5b94@suse.com>
 <45509cb67ecee3f690b5784489b5ccb4@bugseng.com>
 <1743b4248d30a4e8b68a150c25724caa@bugseng.com>
 <2ff52df443fc080875fd05614d89764d@bugseng.com>
 <4b121e48-9541-4b53-8352-939c904f4f1c@suse.com>
Message-ID: <be36b86a08f7573b93edc4c03aff93ef@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit


>>>> 
>>>>>> --- a/xen/arch/arm/include/asm/page.h
>>>>>> +++ b/xen/arch/arm/include/asm/page.h
>>>>>> @@ -123,6 +123,7 @@
>>>>>> 
>>>>>>  #ifndef __ASSEMBLY__
>>>>>> 
>>>>>> +#include <public/grant_table.h>
>>>>> 
>>>>> This is a no-go, imo (also on x86): Adding this include here
>>>>> effectively
>>>>> means that nearly every CU will have a dependency on that header, 
>>>>> no
>>>>> matter that most are entirely agnostic of grants. Each arch has a
>>>>> grant_table.h - is there any reason the new, grant-specific helper
>>>>> can't
>>>>> be put there?
>>>>> 
>>>> 
>>>> I would have to test, but I think that can be done
>>>> 
>>> 
>>> The only blocker so far is that this triggers a build error due to a
>>> circular dependency between xen/mm.h and asm/flushtlb.h on x86. Also
>>> found some earlier evidence [1] that there are some oddities around
>>> asm/flushtlb's inclusion.
>>> 
>>> [1]
>>> https://lore.kernel.org/xen-devel/20200318210540.5602-1-andrew.cooper3@citrix.com/
>> 
>> There could be a way of untangling asm/flushtlb.h from xen/mm.h, by
>> moving "accumulate_tlbflush" and "filtered_flush_tlb_mask" introduced 
>> by
>> commit 80943aa40e30 ("replace tlbflush check and operation with inline
>> functions") [1].
>> However, these function should then be part of a generic 
>> xen/flushtlb.h
>> header, since they are used in common code (e.g., common/page_alloc) 
>> and
>> a bunch of common code source files should move their includes (see 
>> [2]
>> for a partial non-working patch). Do you feel that this is a feasible
>> route?
> 
> Yeah, introducing xen/flushtlb.h to hold these sounds pretty sensible.
> 

There is some shuffling of includes to be done to get it to build, which 
I'm still trying to address. Most problems are down to the use of struct 
page_info in page_set_tlbflush_timestamp in x86/.*/asm/flushtlb.h which 
then prompts the inclusion asm/mm.h probably.

>> In passing it should be noted that the header ordering in
>> x86/alternative.c is not the one usually prescribed, so that may be
>> taken care of as well.
> 
> I'm afraid I don't understand this remark.
> 

I just meant to say that this

#include <xen/delay.h>
#include <xen/types.h>
#include <asm/apic.h>
#include <asm/endbr.h>
#include <asm/processor.h>
#include <asm/alternative.h>
#include <xen/init.h>
#include <asm/setup.h>
#include <asm/system.h>
#include <asm/traps.h>
#include <asm/nmi.h>
#include <asm/nops.h>
#include <xen/livepatch.h>

is not the usual order of xen/*.h then asm/*.h and there is no comment 
justifying that ordering. So in the process of including asm/flushtlb.h 
here the inclusion order can be tidied up (or also indipendently), 
unless there is some reason I'm missing that disallows it.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 15:05:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 15:05:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684461.1064321 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdAdW-0003EA-G7; Thu, 22 Feb 2024 15:05:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684461.1064321; Thu, 22 Feb 2024 15:05:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdAdW-0003E3-C5; Thu, 22 Feb 2024 15:05:22 +0000
Received: by outflank-mailman (input) for mailman id 684461;
 Thu, 22 Feb 2024 15:05:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rdAdU-0003Dx-9U
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 15:05:20 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c9d780f6-d193-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 16:05:18 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-56454c695e6so1806877a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 07:05:17 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 em8-20020a056402364800b005652389a8b9sm922226edb.11.2024.02.22.07.05.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 07:05:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c9d780f6-d193-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708614317; x=1709219117; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=VC908Y92RI65RPaybLWx5sFvrvu8aEfhnIPjrp0p2/k=;
        b=TA1djCiimKt1IMHd+46h1tOSkOvgZeiY18TOh5eOQ9T7SQZ/4Jisgx0cyGeVHqbCYh
         ODEl/FIvB78Ry9oKgnLrFkPcIOLgUBhp9pf+TQmev89kDmUu/gdV4SniAJg+BHdslSuw
         jeqTOMGFdedrhzoUxFSWFwQ9Mo1Z8VAa/g2elg3oI0C/xXKQQUN/nDxDLJChllf9ecSC
         ErNwPw127ThwT59gjeWkVKVwNPwAZKE2AHBLRe0kADCuCKshZUkYWhpxarbnerglvC9t
         /WVmaNITcpQVON4z5qvh+5+2gNTDjY+qOFK1siqKFjTNTtvIg/TnyUtcnEYbUcOd+lBf
         xdyA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708614317; x=1709219117;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=VC908Y92RI65RPaybLWx5sFvrvu8aEfhnIPjrp0p2/k=;
        b=eaduam4+8Qhc/WU0O1+OUdOMTe5YnMUl/Q3hRO36QkdEnlOmeQzc6obAg5wjpMlUgv
         HzhtBJe8MBtQ5HGw1gh0kNS9tz707EBa/rceyIudQgbhCb7kvZfqMuZeothd7f0ctdXB
         dReOVWr2cK3XvuX2a1k/Q0JMRVXLiQD80NuV8UlKWb4ol/BpaqhNt6MorALl+DQHwsxe
         wachJyal7RCjTkbBDBCUHbBidgQ4NavHGmXR5b9Wv2qIZWW91a0ubt3yrdTrMh6Bk8sV
         YWAb70kZ3oIAHnVzCLqryln5exLuGt/WNicijwogJeyH0zmjfGth94qeI/IG7HTpIY4n
         EgKQ==
X-Forwarded-Encrypted: i=1; AJvYcCXTSiUZ6B/6x8tZGp61Tz1GrjGEMeMEsf6fNyOqbGQcrkBeW+TjV6xV/mFlLA6RcfR01ZfV8ObUVPqpwQn3m4Kyi7HSwzkrEkvQUDEFMGc=
X-Gm-Message-State: AOJu0YzBoEoQ3HrCQYZxu6xTv1mbcdZO2rZVuXwdruFmoI7OV/pEp+Ha
	PhKFDByHyeQRPT4m99eFscmPNNAwN61zYgxy8p56qJyT7e0AkSTZW+POIEEzcA==
X-Google-Smtp-Source: AGHT+IGdWqQol7qeXwuwWvkMLmgV48dZ227GYIX1gCB7P2039NhGRafa922tP8YA38wAm1jFQVY6mg==
X-Received: by 2002:a05:6402:d4f:b0:565:32ec:204b with SMTP id ec15-20020a0564020d4f00b0056532ec204bmr1845137edb.2.1708614316755;
        Thu, 22 Feb 2024 07:05:16 -0800 (PST)
Message-ID: <34c2c2a9-85e9-4caf-a936-b574ac710ee5@suse.com>
Date: Thu, 22 Feb 2024 16:05:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 6/6] svm/nestedvm: Introduce nested capabilities bit
Content-Language: en-US
To: George Dunlap <george.dunlap@cloud.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-7-george.dunlap@cloud.com>
 <8ee35193-9715-4c4b-a1cb-11af04c4dab9@suse.com>
 <CA+zSX=YuLmZRz=1wwV=svOrhfQ66E-CZ9TEtqNCVM++JVQ5CzQ@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CA+zSX=YuLmZRz=1wwV=svOrhfQ66E-CZ9TEtqNCVM++JVQ5CzQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.02.2024 15:33, George Dunlap wrote:
> On Tue, Feb 20, 2024 at 12:25 AM Jan Beulich <jbeulich@suse.com> wrote:
>> On 06.02.2024 02:20, George Dunlap wrote:
>>> --- a/xen/arch/x86/hvm/vmx/vmx.c
>>> +++ b/xen/arch/x86/hvm/vmx/vmx.c
>>> @@ -3021,6 +3021,9 @@ const struct hvm_function_table * __init start_vmx(void)
>>>      if ( cpu_has_vmx_tsc_scaling )
>>>          vmx_function_table.tsc_scaling.ratio_frac_bits = 48;
>>>
>>> +    /* TODO: Require hardware support before enabling nested virt */
>>> +    vmx_function_table.caps.nested_virt = vmx_function_table.caps.hap;
>>
>> This won't have the intended effect if hap_supported() ends up clearing
>> the bit (used as input here) due to a command line option override. I
>> wonder if instead this wants doing e.g. in a new hook to be called from
>> nestedhvm_setup(). On the SVM side the hook function would then be the
>> start_nested_svm() that you already introduce, with a caps.hap check
>> added.
> 
> I take it presmp_initcall()'s are guaranteed to run before __initcall()'s?

Yes - the former happen ahead of AP bringup, the latter after.

>> Since you leave the other nested-related if() in place in
>> arch_sanitise_domain_config(), all ought to be well, but I think that
>> other if() really wants replacing by the one you presently add.
> 
> Ack.
> 
> I'll probably check in patches 1,2,3, and 5, and resend the other two,
> unless you'd like to see all the changes again...

No need imo to re-post anything that was agreed upon.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 15:07:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 15:07:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684466.1064331 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdAfq-0003pT-U8; Thu, 22 Feb 2024 15:07:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684466.1064331; Thu, 22 Feb 2024 15:07:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdAfq-0003pM-Qx; Thu, 22 Feb 2024 15:07:46 +0000
Received: by outflank-mailman (input) for mailman id 684466;
 Thu, 22 Feb 2024 15:07:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rdAfq-0003pE-Bc
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 15:07:46 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 217aa545-d194-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 16:07:44 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a3ed9cae56fso194634966b.1
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 07:07:44 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 hu20-20020a170907a09400b00a3fa3411416sm226522ejc.64.2024.02.22.07.07.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 07:07:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 217aa545-d194-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708614464; x=1709219264; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pi53rr+pvqYkR7hrdATKlyfk1Nk3hEqdGpA+XlwjUfU=;
        b=STgB73I/vG5QObMSnnhcsMOQa9ik07PpXtPu6C2wyqi8SHn6dIVbq6UlD/+O2AOQFN
         +Rbk2/7Y7uUHqtpbabZ5QT++b+FcbyEXxXmreseZ/42x3ggFbU6BSml8u6duzh3LBIOe
         wvQafitlL6A3B55RATYjPlgsNvKt4j04EkREqdYLA3vHX8tctBL8KdyinHMKMgghL9bM
         bpq/upqyQD1AtUL44InHYnGYrHfezxUyouES3zsCtkGHXoGX03jIoBQDt2+gWtdiNUdf
         yO18AAKOB/La5CWll+CisobZe0wKhCGEpXev8iJJcZ+repkOmae8ug43nXHCugpNdZGW
         kqWw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708614464; x=1709219264;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pi53rr+pvqYkR7hrdATKlyfk1Nk3hEqdGpA+XlwjUfU=;
        b=fgfE0iTJx7XvxkJ/MPVofGpQvpSQdrJf8NnMKdKRMWN0hQl8Xqmj/sa4FyQWg6btKV
         /WQd7vVDFOXUWvHXjoj2MMVvJNsVaZRzUTKpnM7F5K5NbhEAZoxFhPGbErrNLJABmmsX
         zdaCl0tXKloNrURtppEsM5ys24ajEs0o5K91E2KzgCaair805pQpH76MmjYl4lcmkzzn
         fLjrpzWdOKQV1PgMX5I2oeI+1EIzNk0i/CILJAPOTkxKm8mMYrNMQFxyQEYU2d5SBmVP
         597xx/PG78yJjcxR3yJrnsJ6NHbcBMjR6C1rREdOcXc6N4uaRB1F8UfllziI5wigHMxn
         VwXQ==
X-Forwarded-Encrypted: i=1; AJvYcCWlOuVqAol0BeKqxnjaAcjFg7s4ZCOFr4BWfLQs4Uxb06D/2RqO2sp1L+b1Q3jD1xWmFc2eRBMoGUKi272uIQViHoa/87Wdk7yEfM1cnbQ=
X-Gm-Message-State: AOJu0YzSGCjn1SylRayJ+9gLoLLUdM9ok9retVCgzXE6pl2sEFdsS1kJ
	Qkx8WmA85gpo7Ey/YRSH5EHBTxnnGnSThUKCv4EeKUwCFhMl5ohxtRevKgkmIg==
X-Google-Smtp-Source: AGHT+IHNDodOaxePkkFBzLSd1iIJqD0urWy7U9x0QlyKI6dTx0wCe/KRDswxBXRJUyIi63mA0XL9pQ==
X-Received: by 2002:a17:906:1316:b0:a3f:2ffd:c67e with SMTP id w22-20020a170906131600b00a3f2ffdc67emr2349133ejb.15.1708614463730;
        Thu, 22 Feb 2024 07:07:43 -0800 (PST)
Message-ID: <c376a5e2-c9ae-4df5-ae2c-23b9bf299903@suse.com>
Date: Thu, 22 Feb 2024 16:07:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, julien@xen.org,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
 <d90d98b6-508b-4a2a-ab6a-74a9828a5b94@suse.com>
 <45509cb67ecee3f690b5784489b5ccb4@bugseng.com>
 <1743b4248d30a4e8b68a150c25724caa@bugseng.com>
 <2ff52df443fc080875fd05614d89764d@bugseng.com>
 <4b121e48-9541-4b53-8352-939c904f4f1c@suse.com>
 <be36b86a08f7573b93edc4c03aff93ef@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <be36b86a08f7573b93edc4c03aff93ef@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22.02.2024 15:43, Nicola Vetrini wrote:
>>> In passing it should be noted that the header ordering in
>>> x86/alternative.c is not the one usually prescribed, so that may be
>>> taken care of as well.
>>
>> I'm afraid I don't understand this remark.
>>
> 
> I just meant to say that this
> 
> #include <xen/delay.h>
> #include <xen/types.h>
> #include <asm/apic.h>
> #include <asm/endbr.h>
> #include <asm/processor.h>
> #include <asm/alternative.h>
> #include <xen/init.h>
> #include <asm/setup.h>
> #include <asm/system.h>
> #include <asm/traps.h>
> #include <asm/nmi.h>
> #include <asm/nops.h>
> #include <xen/livepatch.h>
> 
> is not the usual order of xen/*.h then asm/*.h and there is no comment 
> justifying that ordering.

Well, you'll find such in many other places. It hasn't been for that long
that we've been trying to get #include-s into a more predictable shape.

> So in the process of including asm/flushtlb.h 
> here the inclusion order can be tidied up (or also indipendently), 
> unless there is some reason I'm missing that disallows it.

Independently, if at all possible, would be better. Unless of course you
need to touch almost all of that block anyway.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 16:06:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 16:06:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684477.1064341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdBaQ-0004IE-W6; Thu, 22 Feb 2024 16:06:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684477.1064341; Thu, 22 Feb 2024 16:06:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdBaQ-0004I7-Se; Thu, 22 Feb 2024 16:06:14 +0000
Received: by outflank-mailman (input) for mailman id 684477;
 Thu, 22 Feb 2024 16:06:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MkW+=J7=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rdBaP-0004I0-L7
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 16:06:13 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4bcb950b-d19c-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 17:06:11 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 752671F745;
 Thu, 22 Feb 2024 16:06:10 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 494BA133DC;
 Thu, 22 Feb 2024 16:06:10 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id jaY1EPJw12XDYAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 22 Feb 2024 16:06:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4bcb950b-d19c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708617970; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=AxMGWf9jPwozcCAbCztlwBdL0qC8GgPQfaHVNyREvY4=;
	b=DqpU49Uw2CHf7SqD97HVoxaegW0CcWFre+iCBtvGbXf8C+tfjzUfcxWlx3q5oLriEHmTwC
	ol5ePFDSdgFnyxAQNCaOwY+eF4nGE0KfSh+mkp7WyKB/aP+wr1UYK/Mj4r1Z8+/0vzaDZV
	3YC18hEMp+7I7z142/XeMiQztDY1zvw=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708617970; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=AxMGWf9jPwozcCAbCztlwBdL0qC8GgPQfaHVNyREvY4=;
	b=DqpU49Uw2CHf7SqD97HVoxaegW0CcWFre+iCBtvGbXf8C+tfjzUfcxWlx3q5oLriEHmTwC
	ol5ePFDSdgFnyxAQNCaOwY+eF4nGE0KfSh+mkp7WyKB/aP+wr1UYK/Mj4r1Z8+/0vzaDZV
	3YC18hEMp+7I7z142/XeMiQztDY1zvw=
Message-ID: <a12f113b-bd97-4807-97c9-b66b55e247f8@suse.com>
Date: Thu, 22 Feb 2024 17:06:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 3/8] stubdom: extend xenstore stubdom configs
Content-Language: en-US
To: xen-devel@lists.xenproject.org,
 Samuel Thibault <samuel.thibault@ens-lyon.org>
Cc: Jason Andryuk <jandryuk@gmail.com>
References: <20240216163139.9093-1-jgross@suse.com>
 <20240216163139.9093-4-jgross@suse.com>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <20240216163139.9093-4-jgross@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------oCu170TXGfK3lZiRdv1Uop6M"
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=DqpU49Uw
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-4.65 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 TO_DN_SOME(0.00)[];
	 HAS_ATTACHMENT(0.00)[];
	 MIME_BASE64_TEXT_BOGUS(1.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 NEURAL_HAM_SHORT(-0.20)[-0.991];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 MID_RHS_MATCH_FROM(0.00)[];
	 BAYES_HAM(-2.25)[96.45%];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[3];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]
X-Spam-Score: -4.65
X-Rspamd-Queue-Id: 752671F745
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------oCu170TXGfK3lZiRdv1Uop6M
Content-Type: multipart/mixed; boundary="------------bi34RBb4shja2g5GSkfVpFZO";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org,
 Samuel Thibault <samuel.thibault@ens-lyon.org>
Cc: Jason Andryuk <jandryuk@gmail.com>
Message-ID: <a12f113b-bd97-4807-97c9-b66b55e247f8@suse.com>
Subject: Re: [PATCH v8 3/8] stubdom: extend xenstore stubdom configs
References: <20240216163139.9093-1-jgross@suse.com>
 <20240216163139.9093-4-jgross@suse.com>
In-Reply-To: <20240216163139.9093-4-jgross@suse.com>

--------------bi34RBb4shja2g5GSkfVpFZO
Content-Type: multipart/mixed; boundary="------------lT1YyORpBnzWltSDA0ZUAFQ9"

--------------lT1YyORpBnzWltSDA0ZUAFQ9
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMTYuMDIuMjQgMTc6MzEsIEp1ZXJnZW4gR3Jvc3Mgd3JvdGU6DQo+IEV4dGVuZCB0aGUg
Y29uZmlnIGZpbGVzIG9mIHRoZSBYZW5zdG9yZSBzdHViZG9tcyB0byBpbmNsdWRlIFhFTkJV
Uw0KPiBhbmQgOVBGUk9OVCBpdGVtcyBpbiBvcmRlciB0byBzdXBwb3J0IGZpbGUgYmFzZWQg
bG9nZ2luZy4NCj4gDQo+IFNpZ25lZC1vZmYtYnk6IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0Bz
dXNlLmNvbT4NCj4gUmV2aWV3ZWQtYnk6IEphc29uIEFuZHJ5dWsgPGphbmRyeXVrQGdtYWls
LmNvbT4NCg0KU2FtdWVsLCBhcmUgeW91IGZpbmUgd2l0aCB0aGlzIHBhdGNoPw0KDQoNCkp1
ZXJnZW4NCg0KPiAtLS0NCj4gICBzdHViZG9tL3hlbnN0b3JlLW1pbmlvcy5jZmcgICAgfCAy
ICstDQo+ICAgc3R1YmRvbS94ZW5zdG9yZXB2aC1taW5pb3MuY2ZnIHwgMiArLQ0KPiAgIDIg
ZmlsZXMgY2hhbmdlZCwgMiBpbnNlcnRpb25zKCspLCAyIGRlbGV0aW9ucygtKQ0KPiANCj4g
ZGlmZiAtLWdpdCBhL3N0dWJkb20veGVuc3RvcmUtbWluaW9zLmNmZyBiL3N0dWJkb20veGVu
c3RvcmUtbWluaW9zLmNmZw0KPiBpbmRleCBhNDE3MDRiYjZiLi4yMzlkYTUxOWI5IDEwMDY0
NA0KPiAtLS0gYS9zdHViZG9tL3hlbnN0b3JlLW1pbmlvcy5jZmcNCj4gKysrIGIvc3R1YmRv
bS94ZW5zdG9yZS1taW5pb3MuY2ZnDQo+IEBAIC0zLDcgKzMsNyBAQCBDT05GSUdfTkVURlJP
TlQ9bg0KPiAgIENPTkZJR19GQkZST05UPW4NCj4gICBDT05GSUdfS0JERlJPTlQ9bg0KPiAg
IENPTkZJR19DT05TRlJPTlQ9bg0KPiAtQ09ORklHX1hFTkJVUz1uDQo+ICAgQ09ORklHX0xX
SVA9bg0KPiArQ09ORklHXzlQRlJPTlQ9eQ0KPiAgIENPTkZJR19CQUxMT09OPXkNCj4gICBY
RU5fSU5URVJGQUNFX1ZFUlNJT049X19YRU5fTEFURVNUX0lOVEVSRkFDRV9WRVJTSU9OX18N
Cj4gZGlmZiAtLWdpdCBhL3N0dWJkb20veGVuc3RvcmVwdmgtbWluaW9zLmNmZyBiL3N0dWJk
b20veGVuc3RvcmVwdmgtbWluaW9zLmNmZw0KPiBpbmRleCA2YWY1MWY1NzUzLi43NTJiOTBk
N2QzIDEwMDY0NA0KPiAtLS0gYS9zdHViZG9tL3hlbnN0b3JlcHZoLW1pbmlvcy5jZmcNCj4g
KysrIGIvc3R1YmRvbS94ZW5zdG9yZXB2aC1taW5pb3MuY2ZnDQo+IEBAIC00LDcgKzQsNyBA
QCBDT05GSUdfTkVURlJPTlQ9bg0KPiAgIENPTkZJR19GQkZST05UPW4NCj4gICBDT05GSUdf
S0JERlJPTlQ9bg0KPiAgIENPTkZJR19DT05TRlJPTlQ9bg0KPiAtQ09ORklHX1hFTkJVUz1u
DQo+ICAgQ09ORklHX0xXSVA9bg0KPiArQ09ORklHXzlQRlJPTlQ9eQ0KPiAgIENPTkZJR19C
QUxMT09OPXkNCj4gICBYRU5fSU5URVJGQUNFX1ZFUlNJT049X19YRU5fTEFURVNUX0lOVEVS
RkFDRV9WRVJTSU9OX18NCg0K
--------------lT1YyORpBnzWltSDA0ZUAFQ9
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------lT1YyORpBnzWltSDA0ZUAFQ9--

--------------bi34RBb4shja2g5GSkfVpFZO--

--------------oCu170TXGfK3lZiRdv1Uop6M
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXXcPEFAwAAAAAACgkQsN6d1ii/Ey/B
Dwf7BySdIF7C0d1csA8XIdRMbSdcPMi5hEYseAImDBTIHCDVFHmAb1tcINqFAC+sZq00H+C6kYYu
0oumtSS2VEYEnXCd/gwCIwPt7uxpP02u5hx/YzJrJcowmpcqUOlfmQJaxqW6rfxn7WyRusZUC6BF
XIwzFZgpKkQe7m8Wqo6RnSHbPQvVtop739UKuP5a2eaTbMd5qOW6GNy3U/4PCBqngQp83tKCbznc
JS6jlsKkRCXTAyJovrOfwE1DQbIUe2z9ReIT3SqPI/i1lFhZJMY4IAvgFuFm/EXs171AFQwOMvA2
Khf2EwuU0L7ZuFjQaddbYVeVNPg+Ij156+e0MEPqrw==
=FvY+
-----END PGP SIGNATURE-----

--------------oCu170TXGfK3lZiRdv1Uop6M--


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 16:13:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 16:13:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684484.1064363 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdBhr-0006AX-Pb; Thu, 22 Feb 2024 16:13:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684484.1064363; Thu, 22 Feb 2024 16:13:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdBhr-0006AQ-My; Thu, 22 Feb 2024 16:13:55 +0000
Received: by outflank-mailman (input) for mailman id 684484;
 Thu, 22 Feb 2024 16:13:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rdBhq-0006AK-CX
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 16:13:54 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rdBho-0002Bw-9N; Thu, 22 Feb 2024 16:13:52 +0000
Received: from [15.248.2.233] (helo=[10.24.67.33])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rdBho-0006zG-3Z; Thu, 22 Feb 2024 16:13:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=CiMqrbBdPVU+W4HhbnQ/ZpuHekVHjKIcVoIfUbi4hqk=; b=I3CoaIoXH2ufT/QY0ayJc6VckT
	SzKT5sSA/nWHTusP6SwtYjf4s8EuyyV6wtihe0+khnvBX7m2GSXs66tnFtuGp+i27U6EbG+ipf8xr
	lCRg4TxJPJO/eTIcPmlAJp9r966Irrh/w7NyJwZwydI0z9GPBgcc5PRszJn9ppWLX6WI=;
Message-ID: <e75c2613-ce22-4dd5-97b4-a95994d98231@xen.org>
Date: Thu, 22 Feb 2024 16:13:50 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 0/3] xen-analysis.py: Enable Xen deviation tag at the end
 of the line
Content-Language: en-GB
To: Stefano Stabellini <sstabellini@kernel.org>,
 Luca Fancellu <luca.fancellu@arm.com>
Cc: xen-devel@lists.xenproject.org, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
References: <20240131110241.3951995-1-luca.fancellu@arm.com>
 <alpine.DEB.2.22.394.2402141707290.1925432@ubuntu-linux-20-04-desktop>
From: Julien Grall <julien@xen.org>
In-Reply-To: <alpine.DEB.2.22.394.2402141707290.1925432@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 15/02/2024 01:08, Stefano Stabellini wrote:
> On Wed, 31 Jan 2024, Luca Fancellu wrote:
>> This serie is enabling the xen-analysis tool to parse and substitute correctly
>> a deviation tag put at the end of the line containing a deviation to be deviated.
>>
>> Before this serie the only way to deviate a violation was to put the tag in the
>> line above:
>>
>> /* SAF-<id>-safe deviate the bla bla bla */
>> <line containing the violation>
>>
>> But there are places in the code base where using the tag in the line above is
>> not convinient, for example:
>>
>> if ( (expression) &&
>>       ((expression with violation) ||
>>       (expression) )
>> {
>>    [...]
>> }
>>
>> In the above example is better to have the suppression comment at the end of the
>> line:
>>
>> if ( (expression) &&
>>       ((expression with violation) || /* SAF-<id>-safe deviate the bla bla bla */
>>       (expression) )
>> {
>>    [...]
>> }
>>
>> This clearly brings up the question about the code style line length, which in
>> this case needs to be amended for Xen deviation tags that goes above the limit.
> 
> 
> Hi Luca,
> 
> I tested the series in a number of configurations and everything works
> as expected. Great!
> 
> For the whole series:
> 
> Acked-by: Stefano Stabellini <sstabellini@kernel.org>

This is now committed.

Cheers,


-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 16:39:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 16:39:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684490.1064373 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdC6E-0000fU-ML; Thu, 22 Feb 2024 16:39:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684490.1064373; Thu, 22 Feb 2024 16:39:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdC6E-0000fN-JN; Thu, 22 Feb 2024 16:39:06 +0000
Received: by outflank-mailman (input) for mailman id 684490;
 Thu, 22 Feb 2024 16:39:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdC6D-0000f9-8s; Thu, 22 Feb 2024 16:39:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdC6D-0002d9-6p; Thu, 22 Feb 2024 16:39:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdC6C-0007wN-Nu; Thu, 22 Feb 2024 16:39:04 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdC6C-0002zg-NV; Thu, 22 Feb 2024 16:39:04 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=PWgyZ1nFYavzLkkpFrLAuq0OBliGrgzM5Mfl5lrIhdA=; b=StcCo00gRp11/YJnEwcqvIULhg
	zp6K2QGv5Q33FqJV3j2FzMEik+8GBavKRLKS8Lt+FVVFoTbrb8v143PTHzCaObt4JUdcrkNyVHqKm
	0sfp2trUHLdcG4i9wkEsQnAPWImtDXe8qUpM7BkKbgWPLFr1kIv+Piw43CabX1ec3+Ao=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184728-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184728: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=8c5e4ce14509ca85b3f5faa298a766b8ffdd4f85
X-Osstest-Versions-That:
    xen=9ee7dc877b8754ce2fc82500feea52c04d4e6409
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 22 Feb 2024 16:39:04 +0000

flight 184728 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184728/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  8c5e4ce14509ca85b3f5faa298a766b8ffdd4f85
baseline version:
 xen                  9ee7dc877b8754ce2fc82500feea52c04d4e6409

Last test of basis   184727  2024-02-22 11:00:37 Z    0 days
Testing same since   184728  2024-02-22 14:04:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   9ee7dc877b..8c5e4ce145  8c5e4ce14509ca85b3f5faa298a766b8ffdd4f85 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 16:42:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 16:42:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684495.1064383 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdC9R-0002Fm-9K; Thu, 22 Feb 2024 16:42:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684495.1064383; Thu, 22 Feb 2024 16:42:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdC9R-0002Ff-63; Thu, 22 Feb 2024 16:42:25 +0000
Received: by outflank-mailman (input) for mailman id 684495;
 Thu, 22 Feb 2024 16:42:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qXTC=J7=ens-lyon.org=samuel.thibault@bounce.ens-lyon.org>)
 id 1rdC9Q-0002FW-69
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 16:42:24 +0000
Received: from sonata.ens-lyon.org (sonata.ens-lyon.org [140.77.166.138])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 59162a93-d1a1-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 17:42:21 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by sonata.ens-lyon.org (Postfix) with ESMTP id B1336A019C;
 Thu, 22 Feb 2024 17:42:20 +0100 (CET)
Received: from sonata.ens-lyon.org ([127.0.0.1])
 by localhost (sonata.ens-lyon.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id bZ7SmCzCoOcV; Thu, 22 Feb 2024 17:42:20 +0100 (CET)
Received: from begin (nat-inria-interne-52-gw-01-bso.bordeaux.inria.fr
 [194.199.1.52])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by sonata.ens-lyon.org (Postfix) with ESMTPSA id 75767A017F;
 Thu, 22 Feb 2024 17:42:20 +0100 (CET)
Received: from samy by begin with local (Exim 4.97)
 (envelope-from <samuel.thibault@ens-lyon.org>)
 id 1rdC9L-0000000Harf-3wSW; Thu, 22 Feb 2024 17:42:19 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 59162a93-d1a1-11ee-98f5-efadbce2ee36
Date: Thu, 22 Feb 2024 17:42:19 +0100
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: xen-devel@lists.xenproject.org, Jason Andryuk <jandryuk@gmail.com>
Subject: Re: [PATCH v8 3/8] stubdom: extend xenstore stubdom configs
Message-ID: <20240222164219.rmeybzido2rhpjkb@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, xen-devel@lists.xenproject.org,
	Jason Andryuk <jandryuk@gmail.com>
References: <20240216163139.9093-1-jgross@suse.com>
 <20240216163139.9093-4-jgross@suse.com>
 <a12f113b-bd97-4807-97c9-b66b55e247f8@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <a12f113b-bd97-4807-97c9-b66b55e247f8@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Juergen Gross, le jeu. 22 févr. 2024 17:06:09 +0100, a ecrit:
> On 16.02.24 17:31, Juergen Gross wrote:
> > Extend the config files of the Xenstore stubdoms to include XENBUS
> > and 9PFRONT items in order to support file based logging.
> > 
> > Signed-off-by: Juergen Gross <jgross@suse.com>
> > Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
> 
> Samuel, are you fine with this patch?

Yes!

Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Samuel

> 
> 
> Juergen
> 
> > ---
> >   stubdom/xenstore-minios.cfg    | 2 +-
> >   stubdom/xenstorepvh-minios.cfg | 2 +-
> >   2 files changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/stubdom/xenstore-minios.cfg b/stubdom/xenstore-minios.cfg
> > index a41704bb6b..239da519b9 100644
> > --- a/stubdom/xenstore-minios.cfg
> > +++ b/stubdom/xenstore-minios.cfg
> > @@ -3,7 +3,7 @@ CONFIG_NETFRONT=n
> >   CONFIG_FBFRONT=n
> >   CONFIG_KBDFRONT=n
> >   CONFIG_CONSFRONT=n
> > -CONFIG_XENBUS=n
> >   CONFIG_LWIP=n
> > +CONFIG_9PFRONT=y
> >   CONFIG_BALLOON=y
> >   XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
> > diff --git a/stubdom/xenstorepvh-minios.cfg b/stubdom/xenstorepvh-minios.cfg
> > index 6af51f5753..752b90d7d3 100644
> > --- a/stubdom/xenstorepvh-minios.cfg
> > +++ b/stubdom/xenstorepvh-minios.cfg
> > @@ -4,7 +4,7 @@ CONFIG_NETFRONT=n
> >   CONFIG_FBFRONT=n
> >   CONFIG_KBDFRONT=n
> >   CONFIG_CONSFRONT=n
> > -CONFIG_XENBUS=n
> >   CONFIG_LWIP=n
> > +CONFIG_9PFRONT=y
> >   CONFIG_BALLOON=y
> >   XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
> 


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 16:47:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 16:47:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684498.1064393 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdCES-0002qh-RF; Thu, 22 Feb 2024 16:47:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684498.1064393; Thu, 22 Feb 2024 16:47:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdCES-0002qa-Nh; Thu, 22 Feb 2024 16:47:36 +0000
Received: by outflank-mailman (input) for mailman id 684498;
 Thu, 22 Feb 2024 16:47:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+gCR=J7=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdCER-0002qT-5Q
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 16:47:35 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 12ffce5d-d1a2-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 17:47:34 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-512bde3d197so4831511e87.0
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 08:47:32 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 or6-20020a056214468600b0068f7f26d74asm4320658qvb.138.2024.02.22.08.47.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 22 Feb 2024 08:47:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 12ffce5d-d1a2-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708620452; x=1709225252; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=m6OftHvItKM5qkGRVczxkILrn6fjNA4NJKNo1579rFo=;
        b=uh2BdqG+SV2Ep2YkZvY1nOVmJn6x/W5CCFMYvf0w1OPinSToz2pyucbl/3PQ6xsTY4
         G3P/uQEVy5qVEAkQMbzua38mlXRZkv0a3GzM7pk3keaEcr1tlGpZQ/UQELbLx1Rav2VI
         X60aK8HtfqCojrUwpgV89B2llET99r/loS9+A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708620452; x=1709225252;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=m6OftHvItKM5qkGRVczxkILrn6fjNA4NJKNo1579rFo=;
        b=h9Qy2foDzrRsMWG6Xr4eoBLBW+OdVAi5WGLAY0XZcfDhRJTUp3oJvBGaUO0g4CpwSc
         4ZxLTJBM5pFpy4KEQNgvgWOHakKa+MnBQwP5qL3U/tu9sTL9PtkwmncIGrf+7xvl30BV
         O58/B94XQRCTNOMHN+nN3QiTaifh1JZRgU/ZuUEbKL4ze44MbQJ9HHhOWlwtGffuiUjo
         /XNGf2rSPkYg2lCd83swCnzjVsNBpbr+eJhxKsxZ3EkRAqueejeNZYTonRACJQ6voHB8
         zR4eGwb4leqa1Ju/8D5ieG39DU7pxHUmSRtw48L9jywLa6EaCAej93T+sGHi+jVgqodh
         iU/A==
X-Gm-Message-State: AOJu0YzkQPROb94gTOipya3FpXOoW8JDuMR2scE3bKzxf0t+mYUzaNdz
	4xoPKcwC+nYfVsqTNnIBUIHtcMgchqag6Ibr36JcjPlCcgXmEYkqGdcael181GJdz1M2M5hyviI
	p
X-Google-Smtp-Source: AGHT+IGxMja/Obx6Is7A875tQAOe8Ul7LHdFqcKWHRAQ8tm2mIESByI3NPYREhBgits+8sqNbSEpzQ==
X-Received: by 2002:ac2:489a:0:b0:511:5f38:76e with SMTP id x26-20020ac2489a000000b005115f38076emr13237993lfc.1.1708620452135;
        Thu, 22 Feb 2024 08:47:32 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>,
	Matthew Grooms <mgrooms@shrew.net>
Subject: [PATCH v2] x86/altcall: use an union as register type for function parameters
Date: Thu, 22 Feb 2024 17:44:55 +0100
Message-ID: <20240222164455.67248-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The current code for alternative calls uses the caller parameter types as the
types for the register variables that serve as function parameters:

uint8_t foo;
[...]
alternative_call(myfunc, foo);

Would expand roughly into:

register unint8_t a1_ asm("rdi") = foo;
register unsigned long a2_ asm("rsi");
[...]
asm volatile ("call *%c[addr](%%rip)"...);

However under certain circumstances clang >= 16.0.0 with -O2 can generate
incorrect code, given the following example:

unsigned int func(uint8_t t)
{
    return t;
}

static void bar(uint8_t b)
{
    int ret_;
    register uint8_t di asm("rdi") = b;
    register unsigned long si asm("rsi");
    register unsigned long dx asm("rdx");
    register unsigned long cx asm("rcx");
    register unsigned long r8 asm("r8");
    register unsigned long r9 asm("r9");
    register unsigned long r10 asm("r10");
    register unsigned long r11 asm("r11");

    asm volatile ( "call %c[addr]"
                   : "+r" (di), "=r" (si), "=r" (dx),
                     "=r" (cx), "=r" (r8), "=r" (r9),
                     "=r" (r10), "=r" (r11), "=a" (ret_)
                   : [addr] "i" (&(func)), "g" (func)
                   : "memory" );
}

void foo(unsigned int a)
{
    bar(a);
}

Clang generates the following code:

func:                                   # @func
        movl    %edi, %eax
        retq
foo:                                    # @foo
        callq   func
        retq

Note the truncation of the unsigned int parameter 'a' of foo() to uint8_t when
passed into bar() is lost.

The above can be worked around by using an union when defining the register
variables, so that `di` becomes:

register union {
    uint8_t e;
    unsigned long r;
} di asm("rdi") = { .e = b };

Which results in following code generated for `foo()`:

foo:                                    # @foo
        movzbl  %dil, %edi
        callq   func
        retq

So the truncation is not longer lost.  Apply such workaround only when built
with clang.

Reported-by: Matthew Grooms <mgrooms@shrew.net>
Link: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277200
Link: https://github.com/llvm/llvm-project/issues/82598
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Only apply the union workaround with clang.

Seems like all gitlab build tests are OK with this approach.
---
 xen/arch/x86/include/asm/alternative.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/xen/arch/x86/include/asm/alternative.h b/xen/arch/x86/include/asm/alternative.h
index a1cd6a9fe5b8..3fe27ea791bf 100644
--- a/xen/arch/x86/include/asm/alternative.h
+++ b/xen/arch/x86/include/asm/alternative.h
@@ -167,9 +167,25 @@ extern void alternative_branches(void);
 #define ALT_CALL_arg5 "r8"
 #define ALT_CALL_arg6 "r9"
 
+#ifdef CONFIG_CC_IS_CLANG
+/*
+ * Use an union with an unsigned long in order to prevent clang from skipping a
+ * possible truncation of the value.  By using the union any truncation is
+ * carried before the call instruction.
+ * https://github.com/llvm/llvm-project/issues/82598
+ */
+#define ALT_CALL_ARG(arg, n)                                            \
+    register union {                                                    \
+        typeof(arg) e;                                                  \
+        unsigned long r;                                                \
+    } a ## n ## _ asm ( ALT_CALL_arg ## n ) = {                         \
+        .e = ({ BUILD_BUG_ON(sizeof(arg) > sizeof(void *)); (arg); })   \
+    }
+#else
 #define ALT_CALL_ARG(arg, n) \
     register typeof(arg) a ## n ## _ asm ( ALT_CALL_arg ## n ) = \
         ({ BUILD_BUG_ON(sizeof(arg) > sizeof(void *)); (arg); })
+#endif
 #define ALT_CALL_NO_ARG(n) \
     register unsigned long a ## n ## _ asm ( ALT_CALL_arg ## n )
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 22 16:55:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 16:55:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684505.1064403 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdCLf-0004Vx-Jf; Thu, 22 Feb 2024 16:55:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684505.1064403; Thu, 22 Feb 2024 16:55:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdCLf-0004Vq-FV; Thu, 22 Feb 2024 16:55:03 +0000
Received: by outflank-mailman (input) for mailman id 684505;
 Thu, 22 Feb 2024 16:55:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=1gXq=J7=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rdCLe-0004VI-LC
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 16:55:02 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1ea5bdae-d1a3-11ee-8a55-1f161083a0e0;
 Thu, 22 Feb 2024 17:55:02 +0100 (CET)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-563cb3ba9daso8452972a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 08:55:02 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p24-20020aa7c898000000b005617a858f21sm5571507eds.93.2024.02.22.08.55.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 08:55:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1ea5bdae-d1a3-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708620901; x=1709225701; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+wodz11AuxqAk56yN/n24hcumQ6Q7S3p+gEEetmoBQI=;
        b=Ie5a5Euz11NLDRUvqqgduC4n5/A8WGXJQpBTERDTjpB8EpIo4uPZXjzkQpHDea5It4
         VW9RwpUFDhO1eztcTuSMlCgqDIo5pgxUgU50RABpjfjpRrpXHpyQpKv/pzp1ZansPs0Q
         7JgEE01vhhCp0qUNgNB7YXe4wBh05dVaBOupWEE6zKVZmANVcR202sDNFiys8URMjuLu
         faubLsl1K/h9yaLCFiWa3tmNTash6APVMcNmZ4UQbjnQa7raqj0nLDdsA0szZ8uUlv4K
         +DZoHeDQ0hBIBR8KLcYhONdCrZVILIH0qe/Xdao7yvb3/QCaZiN7I4yQWmOnVpz6i2x8
         V02w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708620901; x=1709225701;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+wodz11AuxqAk56yN/n24hcumQ6Q7S3p+gEEetmoBQI=;
        b=wE82tj7FZczoLrggTbVlQrUpNjASFLDYV+PxdXGWm0BH6w6AL1fsnvmmXX7UjSvNjg
         0cg82+2XhPNIPSIGGexQ1mnRlRTaxiRiUt5SlGBEpwC2KROFaJCnICpFdrd9IgYobD+B
         nXnm3NP07DGsnt+MLcV06nkJ6Xh0SbLNsbaEaTQtnJObtVRej0vF0kBF/svg1Hp1XqlY
         iJ6T+DjONmKhF3iPqkZCRf+fIH2uZe2yWALmRU16H+G/b8bD0AaKoSp3QsbdvX82vxWb
         fZAWO1a2UMWKew7lQYAc8kSSILRBvkSa5nUaXfhiCpKPfIvhRTfviEL3zG6z99P4T8S+
         nNxQ==
X-Forwarded-Encrypted: i=1; AJvYcCXbKvcdLytCtDofC9vHCL/5eziWdbl2i+7jBRp/fyX0yICBCvCB8mykb6kt/tKocTa/L7H9BvIMEeVbmXWSPj7VPfpspiDp57cDDiqUy1w=
X-Gm-Message-State: AOJu0YwZeCLXQmp3/1q7BRbROSfdLa6D6ki33Gx9a4bmlZ/UzsKI0VCH
	xZdLdbQF2o+XAcBM4m2j8EPRyzn5DJK36OmeobLEkmhvfYbMGXptGxDciOrjmg==
X-Google-Smtp-Source: AGHT+IG19+3X4IMoUDl/gRiGJ1CIyyb0uylRUMZEsNLc1FpTEbZLzZcwj6iE/g/FSekcBXcVt/f11A==
X-Received: by 2002:aa7:de0a:0:b0:564:bbdc:ac3f with SMTP id h10-20020aa7de0a000000b00564bbdcac3fmr6334225edv.16.1708620901599;
        Thu, 22 Feb 2024 08:55:01 -0800 (PST)
Message-ID: <b1c96b93-fd75-4077-b622-4781b7811ee8@suse.com>
Date: Thu, 22 Feb 2024 17:55:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/altcall: use an union as register type for
 function parameters
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 Matthew Grooms <mgrooms@shrew.net>, xen-devel@lists.xenproject.org
References: <20240222164455.67248-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240222164455.67248-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22.02.2024 17:44, Roger Pau Monne wrote:
> --- a/xen/arch/x86/include/asm/alternative.h
> +++ b/xen/arch/x86/include/asm/alternative.h
> @@ -167,9 +167,25 @@ extern void alternative_branches(void);
>  #define ALT_CALL_arg5 "r8"
>  #define ALT_CALL_arg6 "r9"
>  
> +#ifdef CONFIG_CC_IS_CLANG
> +/*
> + * Use an union with an unsigned long in order to prevent clang from skipping a
> + * possible truncation of the value.  By using the union any truncation is
> + * carried before the call instruction.
> + * https://github.com/llvm/llvm-project/issues/82598
> + */

I think it needs saying that this is relying on compiler behavior not
mandated by the standard, thus explaining why it's restricted to
Clang (down the road we may even want to restrict to old versions,
assuming they fix the issue at some point). Plus also giving future
readers a clear understanding that if something breaks with this, it's
not really a surprise.

Aiui this bug is only a special case of the other, much older one, so
referencing that one here too would seem advisable.

As a nit: I think it is "a union" (spelling according to pronunciation),
and I guess the title could now do with saying "optionally" or
mentioning Clang or some such.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 18:04:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 18:04:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684514.1064413 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdDR3-0004Rv-Gd; Thu, 22 Feb 2024 18:04:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684514.1064413; Thu, 22 Feb 2024 18:04:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdDR3-0004Ro-D7; Thu, 22 Feb 2024 18:04:41 +0000
Received: by outflank-mailman (input) for mailman id 684514;
 Thu, 22 Feb 2024 18:04:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mbB9=J7=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1rdDR2-0004Ri-4y
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 18:04:40 +0000
Received: from sender3-op-o18.zoho.com (sender3-op-o18.zoho.com
 [136.143.184.18]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d6245396-d1ac-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 19:04:36 +0100 (CET)
Received: from mail-yw1-f176.google.com (mail-yw1-f176.google.com
 [209.85.128.176]) by mx.zohomail.com
 with SMTPS id 1708625072049441.3639280308985;
 Thu, 22 Feb 2024 10:04:32 -0800 (PST)
Received: by mail-yw1-f176.google.com with SMTP id
 00721157ae682-60822b444c9so37573257b3.2
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 10:04:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d6245396-d1ac-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; t=1708625074; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=jEA7GQola9rqHS98s0Ex2x5iLJXmXJbp13lAozlLoeEjwM/c1GDtZ4cR+uf5Ou8lh8xtVXYeVHmKG2jCRCHNTpNVa8efHk+yHBl8iaf8k4EZytPCYzG23JxUTcH7TMP4ZvKw0fi45bGbTXZVzQWpdM5Bsmb7GDc6mm9ruOcZxm4=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1708625074; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=gZFGRZm0r/chvCibWsTL41TnqvgsC7Ny3x7SAr9/GyA=; 
	b=En6xbggBkvZs/X+jE4Ar+CCHW9VuJBtHUaCn7zIGljKVvhEsid6ET8Z7NYq4mMId3yf9XWyFc8lBJfR9cB0uKPky+pM6YVKWxCUUGUvPqsdCgVa7fp4SSdeWOByddI8EH5XnLKJjDBnaROn3vyR38zhcdxKAtAWTk5IvYYQFKSA=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1708625074;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=gZFGRZm0r/chvCibWsTL41TnqvgsC7Ny3x7SAr9/GyA=;
	b=gW1LVELxPuDUlXtnpDuxXUAg/zA2/Ap6C+Z9gZ8P0cgzDLHRm6G99EKIbUj4v3fv
	OOVuJREpcxrQs80vPIy9VYVMhPpsTTZIkTujTgPTQNEAVHubscv/OUcqaXN1YU4xHat
	o5867eVstbi7msXQ2r+J9TVHhuN9GDNGplsQov78=
X-Forwarded-Encrypted: i=1; AJvYcCUbh2yvPF6rj1IGTO1lhmc6KBtOoT5BYIF+zFKeW0/SIbqM+2PRilgGkM3OjFYs+uNpxXqDlXP9YwwjHtX/zmVdXAw8CcQcGNE5lC/zO1g=
X-Gm-Message-State: AOJu0Yy9VMppVDAgc6G9lwff3GHaWDWYEu4zErmMUg0M6tCV60KxJXXU
	Q92R0xbIrp+2ueayW/SJK0Xo1Ae/SewZrmlvL6cKwuVR7lckAhRHKSWr4PSg2R/6X3MpNW5j4Ab
	zfTW0vwi37iNeNlVucuMf15TLXgo=
X-Google-Smtp-Source: AGHT+IFSTXMUjgYOgiBiX66D/d8gwDzXRGnSaETmzkXucjYsmcmKUzXbMGet77X7htDMwzJoSccSCb+WfDDkNQt0mVE=
X-Received: by 2002:a81:e706:0:b0:608:7485:9df4 with SMTP id
 x6-20020a81e706000000b0060874859df4mr6872139ywl.14.1708625071104; Thu, 22 Feb
 2024 10:04:31 -0800 (PST)
MIME-Version: 1.0
References: <20240222090530.62530-1-roger.pau@citrix.com> <20240222090530.62530-2-roger.pau@citrix.com>
 <d8939069-a1cb-4794-a6aa-75a3e4d97884@suse.com>
In-Reply-To: <d8939069-a1cb-4794-a6aa-75a3e4d97884@suse.com>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Thu, 22 Feb 2024 13:03:55 -0500
X-Gmail-Original-Message-ID: <CABfawhkhH7Pakkq+RgjfmW-f60CbOKxr6wWxvBSr2AkN+m1OdQ@mail.gmail.com>
Message-ID: <CABfawhkhH7Pakkq+RgjfmW-f60CbOKxr6wWxvBSr2AkN+m1OdQ@mail.gmail.com>
Subject: Re: [PATCH 1/2] x86/memsharing: use an atomic add instead of a
 cmpxchg loop
To: Jan Beulich <jbeulich@suse.com>
Cc: Roger Pau Monne <roger.pau@citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Feb 22, 2024 at 5:06=E2=80=AFAM Jan Beulich <jbeulich@suse.com> wro=
te:
>
> On 22.02.2024 10:05, Roger Pau Monne wrote:
> > The usage of a cmpxchg loop in get_next_handle() is unnecessary, as the=
 same
> > can be achieved with an atomic increment, which is both simpler to read=
, and
> > avoid any need for a loop.
> >
> > The cmpxchg usage is likely a remnant of 32bit support, which didn't ha=
ve an
> > instruction to do an atomic 64bit add, and instead a cmpxchg had to be =
used.
> >
> > Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > Signed-of-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> albeit ...
>
> > --- a/xen/arch/x86/mm/mem_sharing.c
> > +++ b/xen/arch/x86/mm/mem_sharing.c
> > @@ -179,13 +179,7 @@ static void mem_sharing_page_unlock(struct page_in=
fo *pg)
> >
> >  static shr_handle_t get_next_handle(void)
> >  {
> > -    /* Get the next handle get_page style */
> > -    uint64_t x, y =3D next_handle;
> > -    do {
> > -        x =3D y;
> > -    }
> > -    while ( (y =3D cmpxchg(&next_handle, x, x + 1)) !=3D x );
> > -    return x + 1;
> > +    return arch_fetch_and_add(&next_handle, 1) + 1;
> >  }
>
> ... the adding of 1 here is a little odd when taken together with
> next_handle's initializer. Tamas, you've not written that code, but do
> you have any thoughts towards the possible removal of either the
> initializer or the adding here? Plus that variable of course could
> very well do with moving into this function.

I have to say I find the existing logic here hard to parse but by the
looks I don't think we need the + 1 once we switch to
arch_fetch_and_add. Also could go without initializing next_handle to
1. Moving it into the function would not really accomplish anything
other than style AFAICT?

Tamas


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 18:43:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 18:43:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684522.1064423 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdE24-0000ff-5o; Thu, 22 Feb 2024 18:42:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684522.1064423; Thu, 22 Feb 2024 18:42:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdE24-0000fY-2B; Thu, 22 Feb 2024 18:42:56 +0000
Received: by outflank-mailman (input) for mailman id 684522;
 Thu, 22 Feb 2024 18:42:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Gl4T=J7=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rdE22-0000fQ-Rt
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 18:42:54 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2f692a6b-d1b2-11ee-98f5-efadbce2ee36;
 Thu, 22 Feb 2024 19:42:52 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a3e7ce7dac9so6392266b.1
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 10:42:52 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 r16-20020a1709067fd000b00a3e88f99cf1sm4543829ejs.149.2024.02.22.10.42.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 10:42:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f692a6b-d1b2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708627372; x=1709232172; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=S5Cgx/f/+pUbHdT1JWXdvp//wDZx48ETdBnkDTXsdq0=;
        b=JjsaDPkfsHI0+ERgsZa5vNTFluDQAMf4/0x39HTDxCJ+YIrgsViMmnJh8NiHkTqHaU
         xDXXhmtzoC/sMoAnW42+LyY2EwIFBiLwEQUnYjwCR56cHDSM4xg7ue4nI5OiE7DJ4GK5
         3lzyJp/SAHifveqZsQRoCotLChsdJo+EXdFs4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708627372; x=1709232172;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=S5Cgx/f/+pUbHdT1JWXdvp//wDZx48ETdBnkDTXsdq0=;
        b=OLlhG4yST9f664VnR6h+vNZDNkaZ+HqdzIzZ+rRL9LzAYsX/WE3up2VJVIq+z3Z90G
         dH8MK8WdxnKARGCn1KK9raaonVS7tmDJdiRtKeWw1AofalaU5ltI6S93244q9AyAl5CX
         rpuxYM5FYn1hdbB+xIDgo5POgikmskpjA3wdbjt4bw5E2XdTik/E0BR6RcJhjn+0opZb
         8hYs+ObXJQDKRmjUS6/Ib+KubSjY9rWnT8vljaGq3gV1H+sCcWjXSTxTTFsfiUpCdYiP
         +wssbsCvv/aQ2NqnJQaj5Wh8prGMa2Su+JgRJTifDPeKBNtxdqwuMUOzY5sDdVnzSRl/
         pUtA==
X-Forwarded-Encrypted: i=1; AJvYcCVyTxMg6/P7z2F7tq0Fu9NE7nThclb3rPZjbLQXalwdls2meo5v1ltWCJOe5lJhzjQuIDvKXfc6QM0lNDpk5hbILp4aN3teCTKf1LENGyg=
X-Gm-Message-State: AOJu0Yx/kpAnr7tTriHrgm9Lzxkw7W2tbN/Kxlp8glOpEUXCuj5hBLJ2
	JoCnhTzRKYeqrwxCeqGPY/SnikJhYLli8sQKjjNgP5XVX6C2TubIQSq+4PNWDpU=
X-Google-Smtp-Source: AGHT+IHgUigpyW/1rngs2gZGSzdDDPpLnRBIrRtPYyw0t2kYl0KgQiAAQvPSte/H7lECfO7NVz4vZg==
X-Received: by 2002:a17:906:1148:b0:a3f:986c:3c3 with SMTP id i8-20020a170906114800b00a3f986c03c3mr1500669eja.40.1708627372175;
        Thu, 22 Feb 2024 10:42:52 -0800 (PST)
Message-ID: <06fba67f-17d0-41f5-bcf0-d31a42acfc3f@citrix.com>
Date: Thu, 22 Feb 2024 18:42:51 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/altcall: use an union as register type for
 function parameters
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>, Roger Pau Monne <roger.pau@citrix.com>
Cc: Wei Liu <wl@xen.org>, Matthew Grooms <mgrooms@shrew.net>,
 xen-devel@lists.xenproject.org
References: <20240222164455.67248-1-roger.pau@citrix.com>
 <b1c96b93-fd75-4077-b622-4781b7811ee8@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <b1c96b93-fd75-4077-b622-4781b7811ee8@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22/02/2024 4:55 pm, Jan Beulich wrote:
> On 22.02.2024 17:44, Roger Pau Monne wrote:
>> --- a/xen/arch/x86/include/asm/alternative.h
>> +++ b/xen/arch/x86/include/asm/alternative.h
>> @@ -167,9 +167,25 @@ extern void alternative_branches(void);
>>  #define ALT_CALL_arg5 "r8"
>>  #define ALT_CALL_arg6 "r9"
>>  
>> +#ifdef CONFIG_CC_IS_CLANG
>> +/*
>> + * Use an union with an unsigned long in order to prevent clang from skipping a
>> + * possible truncation of the value.  By using the union any truncation is
>> + * carried before the call instruction.
>> + * https://github.com/llvm/llvm-project/issues/82598
>> + */
> I think it needs saying that this is relying on compiler behavior not
> mandated by the standard, thus explaining why it's restricted to
> Clang (down the road we may even want to restrict to old versions,
> assuming they fix the issue at some point). Plus also giving future
> readers a clear understanding that if something breaks with this, it's
> not really a surprise.
>
> Aiui this bug is only a special case of the other, much older one, so
> referencing that one here too would seem advisable.
>
> As a nit: I think it is "a union" (spelling according to pronunciation),
> and I guess the title could now do with saying "optionally" or
> mentioning Clang or some such.

Yes.  "a union" is the right form here.

~Andrew

P.S. Spoken, "an union" would come across as "an onion", which is
something very different.

P.P.S.  As I noted to Matthew concerning
https://mjg59.dreamwidth.org/66907.html

"There's an interesting sidechannel in your blog post.  You seem to call
it an S-O-C and not a soc(k), based on the "an""


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 20:39:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 20:39:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684533.1064433 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdFqu-000530-B4; Thu, 22 Feb 2024 20:39:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684533.1064433; Thu, 22 Feb 2024 20:39:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdFqu-00052t-7u; Thu, 22 Feb 2024 20:39:32 +0000
Received: by outflank-mailman (input) for mailman id 684533;
 Thu, 22 Feb 2024 20:39:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdFqs-00052j-T0; Thu, 22 Feb 2024 20:39:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdFqs-0007Wy-QR; Thu, 22 Feb 2024 20:39:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdFqs-0001wW-Gr; Thu, 22 Feb 2024 20:39:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdFqs-0002LL-GD; Thu, 22 Feb 2024 20:39:30 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=gFmpMjBJNFtYwXvRaoo1e+BLUzlfUc8FdrO/XzA9/sM=; b=l2v7zpmzwKv12DeUAVVo77Ow2+
	RKI65gHH7//tRbTtRTlBoZrqzmA/7fTmDqpA1PGjz0Hhbr98yUFnVGamyynyCy69VMgX8xxuW4WhW
	ayACEfd2yChauOR5JoPimMNfNddGKswznnif1DKjm+nIE8PrMWyc3IKnIUKY/+9QOGPI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184730-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184730: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b
X-Osstest-Versions-That:
    xen=8c5e4ce14509ca85b3f5faa298a766b8ffdd4f85
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 22 Feb 2024 20:39:30 +0000

flight 184730 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184730/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b
baseline version:
 xen                  8c5e4ce14509ca85b3f5faa298a766b8ffdd4f85

Last test of basis   184728  2024-02-22 14:04:05 Z    0 days
Testing same since   184730  2024-02-22 17:02:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Luca Fancellu <luca.fancellu@arm.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   8c5e4ce145..812bdc99f4  812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 21:24:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 21:24:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684542.1064443 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdGXx-00029q-K7; Thu, 22 Feb 2024 21:24:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684542.1064443; Thu, 22 Feb 2024 21:24:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdGXx-00029j-HX; Thu, 22 Feb 2024 21:24:01 +0000
Received: by outflank-mailman (input) for mailman id 684542;
 Thu, 22 Feb 2024 21:24:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdGXw-00029Z-0F; Thu, 22 Feb 2024 21:24:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdGXv-00009U-RV; Thu, 22 Feb 2024 21:23:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdGXv-00038z-BY; Thu, 22 Feb 2024 21:23:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdGXv-0001NH-B7; Thu, 22 Feb 2024 21:23:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=+cf1gBNHeBZIFKklPm8Aw6/anveqda3ZQ2d/d/XcQ2c=; b=k2uQsnlScfFQa8Ndgrb10BqixC
	JqO320IvA0TwxX4UBQlih1j43hUNSu/9BSEypeH6IWrsACA6d1pVdddsw1Pd/KbegrsK2vJpapXfy
	qyiUP3G0Rl8kF9OvwJjp7hHbSgtALAuvKunJuF/e1lNYqWOPg8UQoITpPLZjTelILkrI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184725-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184725: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-credit1:guest-start/debian.repeat:fail:heisenbug
    linux-linus:test-amd64-amd64-freebsd12-amd64:guest-start/freebsd.repeat:fail:heisenbug
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=39133352cbed6626956d38ed72012f49b0421e7b
X-Osstest-Versions-That:
    linux=9fc1ccccfd8d53dc7936fe6d633f2373fc9f62e8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 22 Feb 2024 21:23:59 +0000

flight 184725 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184725/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-credit1 22 guest-start/debian.repeat fail in 184722 pass in 184725
 test-amd64-amd64-freebsd12-amd64 21 guest-start/freebsd.repeat fail pass in 184722

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184719
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184719
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184719
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184719
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184719
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184719
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184719
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184719
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                39133352cbed6626956d38ed72012f49b0421e7b
baseline version:
 linux                9fc1ccccfd8d53dc7936fe6d633f2373fc9f62e8

Last test of basis   184719  2024-02-21 06:34:23 Z    1 days
Testing same since   184721  2024-02-21 17:43:51 Z    1 days    3 attempts

------------------------------------------------------------
People who touched revisions under test:
  David Sterba <dsterba@suse.com>
  Herbert Xu <herbert@gondor.apana.org.au>
  Jason Wang <jasowang@redhat.com>
  Josef Bacik <josef@toxicpanda.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Marc Zyngier <maz@kernel.org>
  Michael S. Tsirkin <mst@redhat.com>
  Nathan Chancellor <nathan@kernel.org> # build
  Oliver Upton <oliver.upton@linux.dev>
  Paolo Bonzini <pbonzini@redhat.com>
  Qu Wenruo <wqu@suse.com>
  zhenwei pi <pizhenwei@bytedance.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             fail    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   9fc1ccccfd8d..39133352cbed  39133352cbed6626956d38ed72012f49b0421e7b -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Thu Feb 22 23:35:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 22 Feb 2024 23:35:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684555.1064452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdIb8-0007Je-MV; Thu, 22 Feb 2024 23:35:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684555.1064452; Thu, 22 Feb 2024 23:35:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdIb8-0007JX-Jw; Thu, 22 Feb 2024 23:35:26 +0000
Received: by outflank-mailman (input) for mailman id 684555;
 Thu, 22 Feb 2024 23:35:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jbig=J7=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdIb8-0007JQ-3Q
 for xen-devel@lists.xenproject.org; Thu, 22 Feb 2024 23:35:26 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0c6f088d-d1db-11ee-8a55-1f161083a0e0;
 Fri, 23 Feb 2024 00:35:24 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 39B9F6335F;
 Thu, 22 Feb 2024 23:35:22 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 66E43C433C7;
 Thu, 22 Feb 2024 23:35:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c6f088d-d1db-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708644921;
	bh=jB0OHEs3+SXJygcUqLzKgnLozk5rRzzVc+fKeeXWZcI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=an7VVQUUcsoPElTUDxhyXz6N4UpjrfEpZZ+ImI/5qT9oqswRpK5Da0L6pFbnSo6CN
	 7aVX5i0++hes/DHqlJRKN5NcDP2lxtwQPRJaIskkQWiaXbhLJn+xsRTWOERCEVgRAp
	 3EcraJun6Z0JuMJx+ELygmgIMghcFt+4z+Xf80f/EOyhd7WnyvAbfOwiLBmWOH2TbM
	 +TCfyZPWQ4ITwvzRc6mrFlr4xU/3azPvD2GW0T+JARfTEhm7k0/Dm8apusJ98sDOsJ
	 R7tTDNxj/JJsGok1SgZF2WTiysiFZdOPf93dVigNe6ht/maJhbGQHvPrsr0QLdKjz0
	 y0lTCL2o8bWQg==
Date: Thu, 22 Feb 2024 15:35:19 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: ZHANG Zewei <zewei.zhang@forvia.com>
cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    "ray.huang@amd.com" <ray.huang@amd.com>
Subject: Re: AMD Working On VirtIO GPU Support For Xen Virtualization
 issues
In-Reply-To: <PR0P264MB2423CA7C396A70D21D08274C80B4A@PR0P264MB2423.FRAP264.PROD.OUTLOOK.COM>
Message-ID: <alpine.DEB.2.22.394.2402221534240.754277@ubuntu-linux-20-04-desktop>
References: <PR0P264MB2423CA7C396A70D21D08274C80B4A@PR0P264MB2423.FRAP264.PROD.OUTLOOK.COM>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-1917931811-1708644921=:754277"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1917931811-1708644921=:754277
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

Hi Zewei,

Did you manage to make progress on this issue?

I apologize for answering so late, but this email fell through the
cracks,

Cheers,

Stefano


On Mon, 20 Nov 2023, ZHANG Zewei wrote:
> INTERNAL & PARTNERS
> 
> 
> Hi，Xen-devel：
> 
>        We are trying to implement the igpu sharing function for domU on the AMD (v2000 series) development board environment, and the
> software architecture is xen + ubuntu 22.04 (dom0) + Ubuntu 22.04 (Dom U). We found that AMD engineers like Mr. Huang @ray.huang@amd.com
> have posted information about virtio-gpu  for Xen virtualization to the open source community and uploaded the relevant code.
> 
>  
> 
>        And We refer to the relevant information linked below to deploy on my AMD hardware platform:
> 
> [RFC PATCH 0/5] Add Xen PVH dom0 support for GPU - Huang Rui (kernel.org)
> 
>  
> 
>        However, when we deploy virtio-GPU related software, we encounter the following issues:
> 
> 1.    My code of xen /qemu /Virglrenderer checkout to the upstream-for-xen or upstream-for-xen-v2 branch, the xl configuration file is set
> as follows, see< ubuntuhvm_virtIO-gpu.cfg>:
> 
>               device_model_args_hvm= ["-display", "sdl,gl=on", "-device", "virtio-vga-gl,context_init=true,blob=true,hostmem=4G"]
> 
>  
> 
> 2.    After the domU is started with the xl command, qemu and xen hypervisor will report the following errors:
> 
>          xen hypervisor report < xl_dmesg_upstream-for-xen-v2.txt>:
> 
>               (XEN) d0v8 Over-allocation for d1: 1048833 > 1048832
> 
> (XEN) common/memory.c:277:d0v8 Could not allocate order=0 extent: id=1 memflags=0xc0 (192 of 2048)
> 
>               qemu report  in log file < qemu-dm-ubuntuU_b.hvm.log>:
> 
>               qemu-system-i386: -device virtio-vga-gl,context_init=true,blob=true,hostmem=4G: xen: failed to populate ram at 110080000
> 
>  
> 
> 3.    I tried to start pvh dom0 but it didn't work, it looks the same regardless of whether I'm using upstream-for-xen-v2 or
> upstream-for-xen branch linux ：
> 
> Logs in attachment:< serial_xen_hypervisor-linux-upstream-for-xen-v2.txt>
> 
>  
> 
>  
> 
>        So I have some questions for you :
> 
> 1.     Does Dom 0 need to be configured with PVH for VirtIO-GPU?
> 
> 2.     What may cause the above issues?
> 
>  
> 
> We look forward to hearing from you!
> 
>  
> 
> Thanks & Best regards!
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
> 
> 5acXjzUk
> 
> 
> 
--8323329-1917931811-1708644921=:754277--


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 00:18:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 00:18:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684559.1064463 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJGc-0004SP-LF; Fri, 23 Feb 2024 00:18:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684559.1064463; Fri, 23 Feb 2024 00:18:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJGc-0004SI-HK; Fri, 23 Feb 2024 00:18:18 +0000
Received: by outflank-mailman (input) for mailman id 684559;
 Fri, 23 Feb 2024 00:18:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdJGb-0004SB-7J
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 00:18:17 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 08ac98a3-d1e1-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 01:18:14 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 07FD861576;
 Fri, 23 Feb 2024 00:18:13 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C75A5C433C7;
 Fri, 23 Feb 2024 00:18:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 08ac98a3-d1e1-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708647492;
	bh=/G3WPmBRvHL0fQTmn66Ut0QyFHARNRNblDjaZFfez/c=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hlxrDVqi28sPJZBicQlLwLyRh9U4XeEujfJ0EFXtO9br3dnF2mFBBMflGjQcl5sf3
	 m64smT5OWhiZNtV7EEuXe8uEPcdgW549x03T0y467QC2NKZx0rNa16MbydqNH6/KuG
	 3NfQs6tQ5dC2D1lyl2xE6qa+u+E8I3ustDK803Ust/MUgChlKwTNa09VpBj31bkpLP
	 8RYwXA98chqcnzdAmsMQ9XRyQHQtyQyN5+bCs0MoO3vGXvYU/AAgBgCTtCgVKJtYkX
	 4TBnN2D1gH0tozMZM1O5lbxj06kgzmetx5E/deNYgJ/NxjfOaEPs6LH9DqX3MzUhif
	 Wqzj8oehIpS9A==
Date: Thu, 22 Feb 2024 16:18:09 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jiqian Chen <Jiqian.Chen@amd.com>
cc: Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Bjorn Helgaas <bhelgaas@google.com>, 
    "Rafael J . Wysocki" <rafael@kernel.org>, Len Brown <lenb@kernel.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org, 
    linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, 
    Stewart Hildebrand <Stewart.Hildebrand@amd.com>, 
    Huang Rui <Ray.Huang@amd.com>, Huang Rui <ray.huang@amd.com>
Subject: Re: [RFC KERNEL PATCH v4 1/3] xen/pci: Add xen_reset_device_state
 function
In-Reply-To: <20240105062217.349645-2-Jiqian.Chen@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402221617590.754277@ubuntu-linux-20-04-desktop>
References: <20240105062217.349645-1-Jiqian.Chen@amd.com> <20240105062217.349645-2-Jiqian.Chen@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 5 Jan 2024, Jiqian Chen wrote:
> When device on dom0 side has been reset, the vpci on Xen side
> won't get notification, so that the cached state in vpci is
> all out of date with the real device state.
> To solve that problem, add a new function to clear all vpci
> device state when device is reset on dom0 side.
> 
> And call that function in pcistub_init_device. Because when
> using "pci-assignable-add" to assign a passthrough device in
> Xen, it will reset passthrough device and the vpci state will
> out of date, and then device will fail to restore bar state.
> 
> Co-developed-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  drivers/xen/pci.c                  | 12 ++++++++++++
>  drivers/xen/xen-pciback/pci_stub.c | 18 +++++++++++++++---
>  include/xen/interface/physdev.h    |  7 +++++++
>  include/xen/pci.h                  |  6 ++++++
>  4 files changed, 40 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/xen/pci.c b/drivers/xen/pci.c
> index 72d4e3f193af..e9b30bc09139 100644
> --- a/drivers/xen/pci.c
> +++ b/drivers/xen/pci.c
> @@ -177,6 +177,18 @@ static int xen_remove_device(struct device *dev)
>  	return r;
>  }
>  
> +int xen_reset_device_state(const struct pci_dev *dev)
> +{
> +	struct physdev_pci_device device = {
> +		.seg = pci_domain_nr(dev->bus),
> +		.bus = dev->bus->number,
> +		.devfn = dev->devfn
> +	};
> +
> +	return HYPERVISOR_physdev_op(PHYSDEVOP_pci_device_state_reset, &device);
> +}
> +EXPORT_SYMBOL_GPL(xen_reset_device_state);
> +
>  static int xen_pci_notifier(struct notifier_block *nb,
>  			    unsigned long action, void *data)
>  {
> diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
> index e34b623e4b41..46c40ec8a18e 100644
> --- a/drivers/xen/xen-pciback/pci_stub.c
> +++ b/drivers/xen/xen-pciback/pci_stub.c
> @@ -89,6 +89,16 @@ static struct pcistub_device *pcistub_device_alloc(struct pci_dev *dev)
>  	return psdev;
>  }
>  
> +static int pcistub_reset_device_state(struct pci_dev *dev)
> +{
> +	__pci_reset_function_locked(dev);
> +
> +	if (!xen_pv_domain())
> +		return xen_reset_device_state(dev);
> +	else
> +		return 0;
> +}
> +
>  /* Don't call this directly as it's called by pcistub_device_put */
>  static void pcistub_device_release(struct kref *kref)
>  {
> @@ -107,7 +117,7 @@ static void pcistub_device_release(struct kref *kref)
>  	/* Call the reset function which does not take lock as this
>  	 * is called from "unbind" which takes a device_lock mutex.
>  	 */
> -	__pci_reset_function_locked(dev);
> +	pcistub_reset_device_state(dev);
>  	if (dev_data &&
>  	    pci_load_and_free_saved_state(dev, &dev_data->pci_saved_state))
>  		dev_info(&dev->dev, "Could not reload PCI state\n");
> @@ -284,7 +294,7 @@ void pcistub_put_pci_dev(struct pci_dev *dev)
>  	 * (so it's ready for the next domain)
>  	 */
>  	device_lock_assert(&dev->dev);
> -	__pci_reset_function_locked(dev);
> +	pcistub_reset_device_state(dev);
>  
>  	dev_data = pci_get_drvdata(dev);
>  	ret = pci_load_saved_state(dev, dev_data->pci_saved_state);
> @@ -420,7 +430,9 @@ static int pcistub_init_device(struct pci_dev *dev)
>  		dev_err(&dev->dev, "Could not store PCI conf saved state!\n");
>  	else {
>  		dev_dbg(&dev->dev, "resetting (FLR, D3, etc) the device\n");
> -		__pci_reset_function_locked(dev);
> +		err = pcistub_reset_device_state(dev);
> +		if (err)
> +			goto config_release;
>  		pci_restore_state(dev);
>  	}
>  	/* Now disable the device (this also ensures some private device
> diff --git a/include/xen/interface/physdev.h b/include/xen/interface/physdev.h
> index a237af867873..8609770e28f5 100644
> --- a/include/xen/interface/physdev.h
> +++ b/include/xen/interface/physdev.h
> @@ -256,6 +256,13 @@ struct physdev_pci_device_add {
>   */
>  #define PHYSDEVOP_prepare_msix          30
>  #define PHYSDEVOP_release_msix          31
> +/*
> + * Notify the hypervisor that a PCI device has been reset, so that any
> + * internally cached state is regenerated.  Should be called after any
> + * device reset performed by the hardware domain.
> + */
> +#define PHYSDEVOP_pci_device_state_reset     32
> +
>  struct physdev_pci_device {
>      /* IN */
>      uint16_t seg;
> diff --git a/include/xen/pci.h b/include/xen/pci.h
> index b8337cf85fd1..b2e2e856efd6 100644
> --- a/include/xen/pci.h
> +++ b/include/xen/pci.h
> @@ -4,10 +4,16 @@
>  #define __XEN_PCI_H__
>  
>  #if defined(CONFIG_XEN_DOM0)
> +int xen_reset_device_state(const struct pci_dev *dev);
>  int xen_find_device_domain_owner(struct pci_dev *dev);
>  int xen_register_device_domain_owner(struct pci_dev *dev, uint16_t domain);
>  int xen_unregister_device_domain_owner(struct pci_dev *dev);
>  #else
> +static inline int xen_reset_device_state(const struct pci_dev *dev)
> +{
> +	return -1;
> +}
> +
>  static inline int xen_find_device_domain_owner(struct pci_dev *dev)
>  {
>  	return -1;
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 00:24:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 00:24:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684563.1064472 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJM9-00066i-Bv; Fri, 23 Feb 2024 00:24:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684563.1064472; Fri, 23 Feb 2024 00:24:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJM9-00066b-9E; Fri, 23 Feb 2024 00:24:01 +0000
Received: by outflank-mailman (input) for mailman id 684563;
 Fri, 23 Feb 2024 00:24:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdJM7-00066V-Uy
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 00:23:59 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d53846fb-d1e1-11ee-8a55-1f161083a0e0;
 Fri, 23 Feb 2024 01:23:58 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 203E8CE2817;
 Fri, 23 Feb 2024 00:23:54 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 73C99C433C7;
 Fri, 23 Feb 2024 00:23:51 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d53846fb-d1e1-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708647833;
	bh=N50MF0FEA09+JdWWeSvMMaGLUQyg9qJDly4KxW/WH/E=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=Jl+0Mx5FR+/lTfq3bBupa6egNmH/ZvbEnlWJLi7B0LT49a8HDDkw9VRnwuer3NtUz
	 G7fdAwswNBE4Mak1W1T/wp1OUGuxg3UMITwset0GoLfaZblcN8lMQKR3eDyqLrT2Ac
	 rhGhgjJsCXOnHrPAOvz75j36TkWXbr70dezzFDhMOvEAxwMAu3iwraUHU6O3Rs9B8X
	 jamAjUL6RXaFCPCnLZPWJIfJMLcDN9qQbXIcW64fpMgF/wIFTlOLZ6fqP9OppflKQO
	 aDQkrepAuXioLa2VBKcgNjeXUNBYnUDLF71P4pQBhaXth2/TsjVNAwv/lmFOd7mzhO
	 rp4ZNAOSWcQrg==
Date: Thu, 22 Feb 2024 16:23:49 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jiqian Chen <Jiqian.Chen@amd.com>
cc: Juergen Gross <jgross@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Bjorn Helgaas <bhelgaas@google.com>, 
    "Rafael J . Wysocki" <rafael@kernel.org>, Len Brown <lenb@kernel.org>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org, 
    linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, 
    Stewart Hildebrand <Stewart.Hildebrand@amd.com>, 
    Huang Rui <Ray.Huang@amd.com>, Huang Rui <ray.huang@amd.com>
Subject: Re: [RFC KERNEL PATCH v4 2/3] xen/pvh: Setup gsi for passthrough
 device
In-Reply-To: <20240105062217.349645-3-Jiqian.Chen@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402221622000.754277@ubuntu-linux-20-04-desktop>
References: <20240105062217.349645-1-Jiqian.Chen@amd.com> <20240105062217.349645-3-Jiqian.Chen@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 5 Jan 2024, Jiqian Chen wrote:
> In PVH dom0, the gsis don't get registered, but the gsi of
> a passthrough device must be configured for it to be able to be
> mapped into a domU.
> 
> When assign a device to passthrough, proactively setup the gsi
> of the device during that process.
> 
> Co-developed-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> ---
>  arch/x86/xen/enlighten_pvh.c       | 90 ++++++++++++++++++++++++++++++
>  drivers/acpi/pci_irq.c             |  2 +-
>  drivers/xen/xen-pciback/pci_stub.c |  8 +++
>  include/linux/acpi.h               |  1 +
>  include/xen/acpi.h                 |  6 ++
>  5 files changed, 106 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/xen/enlighten_pvh.c b/arch/x86/xen/enlighten_pvh.c
> index ada3868c02c2..ecadd966c684 100644
> --- a/arch/x86/xen/enlighten_pvh.c
> +++ b/arch/x86/xen/enlighten_pvh.c
> @@ -1,6 +1,7 @@
>  // SPDX-License-Identifier: GPL-2.0
>  #include <linux/acpi.h>
>  #include <linux/export.h>
> +#include <linux/pci.h>
>  
>  #include <xen/hvc-console.h>
>  
> @@ -25,6 +26,95 @@
>  bool __ro_after_init xen_pvh;
>  EXPORT_SYMBOL_GPL(xen_pvh);
>  
> +typedef struct gsi_info {
> +	int gsi;
> +	int trigger;
> +	int polarity;
> +} gsi_info_t;
> +
> +struct acpi_prt_entry {
> +	struct acpi_pci_id	id;
> +	u8			pin;
> +	acpi_handle		link;
> +	u32			index;		/* GSI, or link _CRS index */
> +};
> +
> +static int xen_pvh_get_gsi_info(struct pci_dev *dev,
> +								gsi_info_t *gsi_info)
> +{
> +	int gsi;
> +	u8 pin = 0;
> +	struct acpi_prt_entry *entry;
> +	int trigger = ACPI_LEVEL_SENSITIVE;
> +	int polarity = acpi_irq_model == ACPI_IRQ_MODEL_GIC ?
> +				      ACPI_ACTIVE_HIGH : ACPI_ACTIVE_LOW;
> +
> +	if (dev)
> +		pin = dev->pin;

This is minor, but you can just move the pin = dev->pin after the !dev
check below.

With that change, and assuming the Xen-side and QEMU-side patches are
accepted:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>




> +	if (!dev || !pin || !gsi_info)
> +		return -EINVAL;
> +
> +	entry = acpi_pci_irq_lookup(dev, pin);
> +	if (entry) {
> +		if (entry->link)
> +			gsi = acpi_pci_link_allocate_irq(entry->link,
> +							 entry->index,
> +							 &trigger, &polarity,
> +							 NULL);
> +		else
> +			gsi = entry->index;
> +	} else
> +		gsi = -1;
> +
> +	if (gsi < 0)
> +		return -EINVAL;
> +
> +	gsi_info->gsi = gsi;
> +	gsi_info->trigger = trigger;
> +	gsi_info->polarity = polarity;
> +
> +	return 0;
> +}
> +
> +static int xen_pvh_setup_gsi(gsi_info_t *gsi_info)
> +{
> +	struct physdev_setup_gsi setup_gsi;
> +
> +	if (!gsi_info)
> +		return -EINVAL;
> +
> +	setup_gsi.gsi = gsi_info->gsi;
> +	setup_gsi.triggering = (gsi_info->trigger == ACPI_EDGE_SENSITIVE ? 0 : 1);
> +	setup_gsi.polarity = (gsi_info->polarity == ACPI_ACTIVE_HIGH ? 0 : 1);
> +
> +	return HYPERVISOR_physdev_op(PHYSDEVOP_setup_gsi, &setup_gsi);
> +}
> +
> +int xen_pvh_passthrough_gsi(struct pci_dev *dev)
> +{
> +	int ret;
> +	gsi_info_t gsi_info;
> +
> +	if (!dev)
> +		return -EINVAL;
> +
> +	ret = xen_pvh_get_gsi_info(dev, &gsi_info);
> +	if (ret) {
> +		xen_raw_printk("Fail to get gsi info!\n");
> +		return ret;
> +	}
> +
> +	ret = xen_pvh_setup_gsi(&gsi_info);
> +	if (ret == -EEXIST) {
> +		xen_raw_printk("Already setup the GSI :%d\n", gsi_info.gsi);
> +		ret = 0;
> +	} else if (ret)
> +		xen_raw_printk("Fail to setup GSI (%d)!\n", gsi_info.gsi);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(xen_pvh_passthrough_gsi);
> +
>  void __init xen_pvh_init(struct boot_params *boot_params)
>  {
>  	u32 msr;
> diff --git a/drivers/acpi/pci_irq.c b/drivers/acpi/pci_irq.c
> index ff30ceca2203..630fe0a34bc6 100644
> --- a/drivers/acpi/pci_irq.c
> +++ b/drivers/acpi/pci_irq.c
> @@ -288,7 +288,7 @@ static int acpi_reroute_boot_interrupt(struct pci_dev *dev,
>  }
>  #endif /* CONFIG_X86_IO_APIC */
>  
> -static struct acpi_prt_entry *acpi_pci_irq_lookup(struct pci_dev *dev, int pin)
> +struct acpi_prt_entry *acpi_pci_irq_lookup(struct pci_dev *dev, int pin)
>  {
>  	struct acpi_prt_entry *entry = NULL;
>  	struct pci_dev *bridge;
> diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
> index 46c40ec8a18e..22d4380d2b04 100644
> --- a/drivers/xen/xen-pciback/pci_stub.c
> +++ b/drivers/xen/xen-pciback/pci_stub.c
> @@ -20,6 +20,7 @@
>  #include <linux/atomic.h>
>  #include <xen/events.h>
>  #include <xen/pci.h>
> +#include <xen/acpi.h>
>  #include <xen/xen.h>
>  #include <asm/xen/hypervisor.h>
>  #include <xen/interface/physdev.h>
> @@ -435,6 +436,13 @@ static int pcistub_init_device(struct pci_dev *dev)
>  			goto config_release;
>  		pci_restore_state(dev);
>  	}
> +
> +	if (xen_initial_domain() && xen_pvh_domain()) {
> +		err = xen_pvh_passthrough_gsi(dev);
> +		if (err)
> +			goto config_release;
> +	}
> +
>  	/* Now disable the device (this also ensures some private device
>  	 * data is setup before we export)
>  	 */
> diff --git a/include/linux/acpi.h b/include/linux/acpi.h
> index 4db54e928b36..7ea3be981cc3 100644
> --- a/include/linux/acpi.h
> +++ b/include/linux/acpi.h
> @@ -360,6 +360,7 @@ void acpi_unregister_gsi (u32 gsi);
>  
>  struct pci_dev;
>  
> +struct acpi_prt_entry *acpi_pci_irq_lookup(struct pci_dev *dev, int pin);
>  int acpi_pci_irq_enable (struct pci_dev *dev);
>  void acpi_penalize_isa_irq(int irq, int active);
>  bool acpi_isa_irq_available(int irq);
> diff --git a/include/xen/acpi.h b/include/xen/acpi.h
> index b1e11863144d..17c4d37f1e60 100644
> --- a/include/xen/acpi.h
> +++ b/include/xen/acpi.h
> @@ -67,10 +67,16 @@ static inline void xen_acpi_sleep_register(void)
>  		acpi_suspend_lowlevel = xen_acpi_suspend_lowlevel;
>  	}
>  }
> +int xen_pvh_passthrough_gsi(struct pci_dev *dev);
>  #else
>  static inline void xen_acpi_sleep_register(void)
>  {
>  }
> +
> +static inline int xen_pvh_passthrough_gsi(struct pci_dev *dev)
> +{
> +	return -1;
> +}
>  #endif
>  
>  #endif	/* _XEN_ACPI_H */
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 00:26:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 00:26:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684567.1064483 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJOD-0006dj-PG; Fri, 23 Feb 2024 00:26:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684567.1064483; Fri, 23 Feb 2024 00:26:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJOD-0006dc-LM; Fri, 23 Feb 2024 00:26:09 +0000
Received: by outflank-mailman (input) for mailman id 684567;
 Fri, 23 Feb 2024 00:26:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdJOC-0006dW-Eu
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 00:26:08 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 21cbbcea-d1e2-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 01:26:06 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id DFF2A6339A;
 Fri, 23 Feb 2024 00:26:04 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8412AC433C7;
 Fri, 23 Feb 2024 00:26:03 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21cbbcea-d1e2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708647964;
	bh=S6XLWwOlyw5kVkrghNQ9x3mICtxXeoGmEqulSxH1LPs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=L4qne9PBS9NG8ikgBrkpnxHesLLuFCKpMU/uxjIadOUGu1nxweNwNTc/UJb45QVoa
	 0BEJoxj4wDUrYkMj+Qx1Q59exNDVRQjRdqz3GRn1TRAoTYoNG+Pfhkrl2e3DXNQiDr
	 vVQQ79mADinRLkS9KcUGwcYC+pGU9pHnVekOAxi0J0AKo48CkovTiRq9wJFD54Ql5j
	 UXkqENTUd5tazjvBpKSJNIWG67Ajs9+YCUl4o5CSNAUE2XQ4AwVO1SOYAZV97VT7zn
	 VHfSefBNL1TzL+j9BXg/uWEYMu9pQ305dJlAmtkOiAwp3QngODZ/Aj76PXWWavhxWe
	 AM8AfJk4jggeA==
Date: Thu, 22 Feb 2024 16:26:01 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jiqian Chen <Jiqian.Chen@amd.com>
cc: qemu-devel@nongnu.org, Stefano Stabellini <sstabellini@kernel.org>, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    xen-devel@lists.xenproject.org, Huang Rui <Ray.Huang@amd.com>, 
    Huang Rui <ray.huang@amd.com>
Subject: Re: [RFC QEMU PATCH v4 1/1] xen: Use gsi instead of irq for mapping
 pirq
In-Reply-To: <20240105071919.350397-2-Jiqian.Chen@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402221625040.754277@ubuntu-linux-20-04-desktop>
References: <20240105071919.350397-1-Jiqian.Chen@amd.com> <20240105071919.350397-2-Jiqian.Chen@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 5 Jan 2024, Jiqian Chen wrote:
> In PVH dom0, it uses the linux local interrupt mechanism,
> when it allocs irq for a gsi, it is dynamic, and follow
> the principle of applying first, distributing first. And
> the irq number is alloced from small to large, but the
> applying gsi number is not, may gsi 38 comes before gsi
> 28, that causes the irq number is not equal with the gsi
> number. And when passthrough a device, qemu wants to use
> gsi to map pirq, xen_pt_realize->xc_physdev_map_pirq, but
> the gsi number is got from file
> /sys/bus/pci/devices/<sbdf>/irq in current code, so it
> will fail when mapping.
> 
> Add gsi into XenHostPCIDevice and use gsi number that
> read from gsi sysfs if it exists.
> 
> Co-developed-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-host-pci-device.c | 7 +++++++
>  hw/xen/xen-host-pci-device.h | 1 +
>  hw/xen/xen_pt.c              | 6 +++++-
>  3 files changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/xen/xen-host-pci-device.c b/hw/xen/xen-host-pci-device.c
> index 8c6e9a1716a2..5be3279aa25b 100644
> --- a/hw/xen/xen-host-pci-device.c
> +++ b/hw/xen/xen-host-pci-device.c
> @@ -370,6 +370,13 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
>      }
>      d->irq = v;
>  
> +    xen_host_pci_get_dec_value(d, "gsi", &v, errp);
> +    if (*errp) {
> +        d->gsi = -1;
> +    } else {
> +        d->gsi = v;
> +    }
> +
>      xen_host_pci_get_hex_value(d, "class", &v, errp);
>      if (*errp) {
>          goto error;
> diff --git a/hw/xen/xen-host-pci-device.h b/hw/xen/xen-host-pci-device.h
> index 4d8d34ecb024..74c552bb5548 100644
> --- a/hw/xen/xen-host-pci-device.h
> +++ b/hw/xen/xen-host-pci-device.h
> @@ -27,6 +27,7 @@ typedef struct XenHostPCIDevice {
>      uint16_t device_id;
>      uint32_t class_code;
>      int irq;
> +    int gsi;
>  
>      XenHostPCIIORegion io_regions[PCI_NUM_REGIONS - 1];
>      XenHostPCIIORegion rom;
> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
> index 36e6f93c372f..d448f3a17306 100644
> --- a/hw/xen/xen_pt.c
> +++ b/hw/xen/xen_pt.c
> @@ -839,7 +839,11 @@ static void xen_pt_realize(PCIDevice *d, Error **errp)
>          goto out;
>      }
>  
> -    machine_irq = s->real_device.irq;
> +    if (s->real_device.gsi < 0) {
> +        machine_irq = s->real_device.irq;
> +    } else {
> +        machine_irq = s->real_device.gsi;
> +    }
>      if (machine_irq == 0) {
>          XEN_PT_LOG(d, "machine irq is 0\n");
>          cmd |= PCI_COMMAND_INTX_DISABLE;
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 00:27:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 00:27:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684573.1064493 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJPj-0007AF-2x; Fri, 23 Feb 2024 00:27:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684573.1064493; Fri, 23 Feb 2024 00:27:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJPj-0007A8-0F; Fri, 23 Feb 2024 00:27:43 +0000
Received: by outflank-mailman (input) for mailman id 684573;
 Fri, 23 Feb 2024 00:27:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdJPh-0007A2-B6
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 00:27:41 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 589bda61-d1e2-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 01:27:39 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id B74BFCE2817;
 Fri, 23 Feb 2024 00:27:36 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 50DEAC433F1;
 Fri, 23 Feb 2024 00:27:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 589bda61-d1e2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708648056;
	bh=o1hUzMwoVVSEjKKW2uxZHQ9eLm2xHzU/158rOozK1oA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=r2DX7ExVRIddodt0pMzkzqXbgJZL6swLh6TLGMev62gEODPBwdotAm+yEQZ46SkRP
	 LILn5ZqCA6sZ5qsRmtP3+FNdDh4Gj57xsnkH7qGarI6gDYTbIEl++5fyO7gGI4+p3V
	 EqMhM6dYQu3OJc7nzCutMYrafBl/C5AvbXJSCECDq3IjPjnPRZVLKb3wyTM+DBLLut
	 sIJYDvvoPa8TdcsbMmUKvzpk9LGAg3I/PwF4MPoXX4bAYsd4P5YS4wQE+C+b316wt/
	 cxXlSVgA0rNzZn9bgOAn2uRFnYCEvGiEApVk66/o/ZM0I6rDaBz1e2eqIL4X8rROUN
	 gMVbriXIkr+Tg==
Date: Thu, 22 Feb 2024 16:27:32 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
cc: Jiqian Chen <Jiqian.Chen@amd.com>, xen-devel@lists.xenproject.org, 
    Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Anthony PERARD <anthony.perard@citrix.com>, 
    Juergen Gross <jgross@suse.com>, 
    "Daniel P . Smith" <dpsmith@apertussolutions.com>, 
    Huang Rui <Ray.Huang@amd.com>
Subject: Re: [RFC XEN PATCH v5 1/5] xen/vpci: Clear all vpci status of
 device
In-Reply-To: <9b4ce4c7-6f4e-47ec-b3a4-0b417cd2660c@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402221627250.754277@ubuntu-linux-20-04-desktop>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com> <20240112061317.418658-2-Jiqian.Chen@amd.com> <9b4ce4c7-6f4e-47ec-b3a4-0b417cd2660c@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 9 Feb 2024, Stewart Hildebrand wrote:
> On 1/12/24 01:13, Jiqian Chen wrote:
> > When a device has been reset on dom0 side, the vpci on Xen
> > side won't get notification, so the cached state in vpci is
> > all out of date compare with the real device state.
> > To solve that problem, add a new hypercall to clear all vpci
> > device state. When the state of device is reset on dom0 side,
> > dom0 can call this hypercall to notify vpci.
> > 
> > Co-developed-by: Huang Rui <ray.huang@amd.com>
> > Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> 
> Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 00:41:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 00:41:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684578.1064502 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJcq-0001wn-7b; Fri, 23 Feb 2024 00:41:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684578.1064502; Fri, 23 Feb 2024 00:41:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJcq-0001wg-4g; Fri, 23 Feb 2024 00:41:16 +0000
Received: by outflank-mailman (input) for mailman id 684578;
 Fri, 23 Feb 2024 00:41:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdJco-0001wa-Eo
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 00:41:14 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3d19c244-d1e4-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 01:41:12 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 389DECE28A9;
 Fri, 23 Feb 2024 00:41:09 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C3119C433F1;
 Fri, 23 Feb 2024 00:40:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d19c244-d1e4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708648867;
	bh=2eAFc96uRl3Ua7F+T8Syhw9JISW3nl4V1PqSlH0ngOY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VN1OQdFD4fM0xXIULwFJs9e8yo/38jscI4CxtSNXZx8nWFYrNNsdXuPgJR8umTVDH
	 mJncfrIIr4+wLYK0I1soVekD0xsHIg+B965iWEXa3U0xfvjhp0GHgzeY8fgLMXFx0C
	 Q1+vVpSX8Dq6WkDLibm7gTW8euUTh37pa/ZeeUusi67h2nZe0Ro3pd8ud35MaCs0yh
	 EdOD4veViQ8tAwC7DXiMTPWXZDLwQIX/QAEzx+D8mIPaxrX8V5hcQdYuYMP71AyRJO
	 Gc8CSYB6eCJXRqf06Dx8NlHHmjMTmxtgHoCvZ63pr88l278ebA8NS5uewSBtIle5XO
	 L9P6D/NomF6ow==
Date: Thu, 22 Feb 2024 16:40:58 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jiqian Chen <Jiqian.Chen@amd.com>
cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Anthony PERARD <anthony.perard@citrix.com>, 
    Juergen Gross <jgross@suse.com>, 
    "Daniel P . Smith" <dpsmith@apertussolutions.com>, 
    Stewart Hildebrand <Stewart.Hildebrand@amd.com>, 
    Huang Rui <Ray.Huang@amd.com>, Huang Rui <ray.huang@amd.com>
Subject: Re: [RFC XEN PATCH v5 2/5] x86/pvh: Allow (un)map_pirq when dom0 is
 PVH
In-Reply-To: <20240112061317.418658-3-Jiqian.Chen@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402221632170.754277@ubuntu-linux-20-04-desktop>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com> <20240112061317.418658-3-Jiqian.Chen@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 12 Jan 2024, Jiqian Chen wrote:
> If run Xen with PVH dom0 and hvm domU, hvm will map a pirq for
> a passthrough device by using gsi, see
> xen_pt_realize->xc_physdev_map_pirq and
> pci_add_dm_done->xc_physdev_map_pirq. Then xc_physdev_map_pirq
> will call into Xen, but in hvm_physdev_op, PHYSDEVOP_map_pirq
> is not allowed because currd is PVH dom0 and PVH has no
> X86_EMU_USE_PIRQ flag, it will fail at has_pirq check.
> 
> So, allow PHYSDEVOP_map_pirq when dom0 is PVH and also allow
> PHYSDEVOP_unmap_pirq for the failed path to unmap pirq. And
> add a new check to prevent self map when caller has no PIRQ
> flag.
> 
> Co-developed-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> ---
>  xen/arch/x86/hvm/hypercall.c |  2 ++
>  xen/arch/x86/physdev.c       | 22 ++++++++++++++++++++++
>  2 files changed, 24 insertions(+)
> 
> diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
> index 6ad5b4d5f11f..493998b42ec5 100644
> --- a/xen/arch/x86/hvm/hypercall.c
> +++ b/xen/arch/x86/hvm/hypercall.c
> @@ -74,6 +74,8 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>      {
>      case PHYSDEVOP_map_pirq:
>      case PHYSDEVOP_unmap_pirq:
> +        break;
> +
>      case PHYSDEVOP_eoi:
>      case PHYSDEVOP_irq_status_query:
>      case PHYSDEVOP_get_free_pirq:
> diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
> index 47c4da0af7e1..7f2422c2a483 100644
> --- a/xen/arch/x86/physdev.c
> +++ b/xen/arch/x86/physdev.c
> @@ -303,11 +303,22 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>      case PHYSDEVOP_map_pirq: {
>          physdev_map_pirq_t map;
>          struct msi_info msi;
> +        struct domain *d;
>  
>          ret = -EFAULT;
>          if ( copy_from_guest(&map, arg, 1) != 0 )
>              break;
>  
> +        d = rcu_lock_domain_by_any_id(map.domid);
> +        if ( d == NULL )
> +            return -ESRCH;
> +        if ( !is_pv_domain(d) && !has_pirq(d) )

I think this could just be:

    if ( !has_pirq(d) )

Right?


> +        {
> +            rcu_unlock_domain(d);
> +            return -EOPNOTSUPP;
> +        }
> +        rcu_unlock_domain(d);
> +
>          switch ( map.type )
>          {
>          case MAP_PIRQ_TYPE_MSI_SEG:
> @@ -341,11 +352,22 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>  
>      case PHYSDEVOP_unmap_pirq: {
>          struct physdev_unmap_pirq unmap;
> +        struct domain *d;
>  
>          ret = -EFAULT;
>          if ( copy_from_guest(&unmap, arg, 1) != 0 )
>              break;
>  
> +        d = rcu_lock_domain_by_any_id(unmap.domid);
> +        if ( d == NULL )
> +            return -ESRCH;
> +        if ( !is_pv_domain(d) && !has_pirq(d) )

same here


Other than that, everything looks fine to me


> +        {
> +            rcu_unlock_domain(d);
> +            return -EOPNOTSUPP;
> +        }
> +        rcu_unlock_domain(d);
> +
>          ret = physdev_unmap_pirq(unmap.domid, unmap.pirq);
>          break;
>      }
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 00:44:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 00:44:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684581.1064512 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJfp-0002Vj-LV; Fri, 23 Feb 2024 00:44:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684581.1064512; Fri, 23 Feb 2024 00:44:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJfp-0002Vc-Im; Fri, 23 Feb 2024 00:44:21 +0000
Received: by outflank-mailman (input) for mailman id 684581;
 Fri, 23 Feb 2024 00:44:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdJfo-0002VW-C6
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 00:44:20 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aca63fbf-d1e4-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 01:44:18 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 10CA663340;
 Fri, 23 Feb 2024 00:44:17 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id ECE2AC433C7;
 Fri, 23 Feb 2024 00:44:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aca63fbf-d1e4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708649056;
	bh=VSsnso58d5B5E6qzjFbJgzNWT1xjYoXRuyFXxmOrLxA=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=J5cgbiTTxLqPDVKVOttl443bP4SazqAHIpIk+iZ+XsTE9bXDnQ7ZcTUuUax9jnoMf
	 aVHuvrUehfROqpCGdNJMCR++JWFmgi8dZlexaQLlktBAII8Pg4faPE4XZq2N3OKrMP
	 JHLpcQSbkb84Jt52MMkgg6mUPlYZ832AhbDPQdU9LHJ4w8ozeauLDuoF2gSpC8kUjq
	 MG5SGcMwRPGVTp1vMgiGAARZIAUIdO4wJugByR0zBWUSqOs5027ciOECBRkCe0mhrd
	 VbgksYhZd9E1yzEL3M35e1F4M4xevgUqCzqGhyVRfQHJZVf3+/Am6JEajm4gHudfI1
	 EtHN0YnbMVeNw==
Date: Thu, 22 Feb 2024 16:44:13 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jiqian Chen <Jiqian.Chen@amd.com>
cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, 
    Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    Anthony PERARD <anthony.perard@citrix.com>, 
    Juergen Gross <jgross@suse.com>, 
    "Daniel P . Smith" <dpsmith@apertussolutions.com>, 
    Stewart Hildebrand <Stewart.Hildebrand@amd.com>, 
    Huang Rui <Ray.Huang@amd.com>, Huang Rui <ray.huang@amd.com>
Subject: Re: [RFC XEN PATCH v5 3/5] x86/pvh: Add PHYSDEVOP_setup_gsi for PVH
 dom0
In-Reply-To: <20240112061317.418658-4-Jiqian.Chen@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402221642530.754277@ubuntu-linux-20-04-desktop>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com> <20240112061317.418658-4-Jiqian.Chen@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 12 Jan 2024, Jiqian Chen wrote:
> On PVH dom0, the gsis don't get registered, but
> the gsi of a passthrough device must be configured for it to
> be able to be mapped into a hvm domU.
> On Linux kernel side, it calles PHYSDEVOP_setup_gsi for
> passthrough devices to register gsi when dom0 is PVH.
> So, add PHYSDEVOP_setup_gsi for above purpose.
> 
> Co-developed-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> ---
>  xen/arch/x86/hvm/hypercall.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
> index 493998b42ec5..46f51ee459f6 100644
> --- a/xen/arch/x86/hvm/hypercall.c
> +++ b/xen/arch/x86/hvm/hypercall.c
> @@ -76,6 +76,12 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>      case PHYSDEVOP_unmap_pirq:
>          break;
>  
> +    case PHYSDEVOP_setup_gsi:
> +        if ( !is_hardware_domain(currd) )
> +            return -EOPNOTSUPP;
> +        ASSERT(!has_pirq(currd));

Do we really need this assert? I understand that the use case right now
is for !has_pirq(currd) but in general it doesn't seem to me that
PHYSDEVOP_setup_gsi and !has_pirq should be tied together.

Aside from that, it looks fine.


> +        break;
> +
>      case PHYSDEVOP_eoi:
>      case PHYSDEVOP_irq_status_query:
>      case PHYSDEVOP_get_free_pirq:
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 00:58:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 00:58:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684587.1064525 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJt6-0004Yj-Oo; Fri, 23 Feb 2024 00:58:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684587.1064525; Fri, 23 Feb 2024 00:58:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdJt6-0004Yc-MJ; Fri, 23 Feb 2024 00:58:04 +0000
Received: by outflank-mailman (input) for mailman id 684587;
 Fri, 23 Feb 2024 00:58:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdJt5-0004YW-6l
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 00:58:03 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9501a8fa-d1e6-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 01:57:58 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id AAD2BCE2999;
 Fri, 23 Feb 2024 00:57:55 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 51BD6C433C7;
 Fri, 23 Feb 2024 00:57:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9501a8fa-d1e6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708649874;
	bh=njJxp1qIWrm69elf9jtOs1FBORbdgE+hleoAV17yg98=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=DTSLI0ZbO6TWYGW72vmkAdITmO8kFz5yGeoDg7N4RgjXJG5T4DA+bf4j/B73SCVbD
	 nz8KTv1c4HOmy7ZVyZszEUI4d1prKmBBpLLhg5Ajsvt+CW2RmsExZ2pV9SSAsKexny
	 XVPalUTb6h+lEJ5SXzh/ICAFhYmY+FGh2Yvw+bgzn+6eWxpHJ0f0VrwLZT14iAaEyw
	 +FFEdftUsQT05T28klerAFwaG1ewLQUnxOPbfXD2kJPPFllRo9SueM/Lx+wYbZCKVE
	 APC5bCxmhrW2CZaQg9bBouujhPNUw8VsIfVJsCBcgkq99xVMFaepgPkW9T6BV+ss9B
	 09sb3E6+CcBqw==
Date: Thu, 22 Feb 2024 16:57:52 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Anthony PERARD <anthony.perard@citrix.com>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH] automation: Rework "build-each-commit-gcc" test
In-Reply-To: <20240220140733.40980-1-anthony.perard@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2402221656150.754277@ubuntu-linux-20-04-desktop>
References: <20240220140733.40980-1-anthony.perard@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 20 Feb 2024, Anthony PERARD wrote:

> Current issues with this test are:
> - when the job timeout, the log file is lost as there is no chance to
>   run the `mv` command.
> - GitLab job log is limited in size, so one usually have to download
>   the artifacts, which may be missing.
> 
> Use $GITLAB_CI to detect when the script is run as part of a GitLab
> pipeline. GitLab will add "GITLAB_CI=true" in the environment
> variables.
> 
> When run as part of $GITLAB_CI, ignore "dirty" worktree to allow to
> write "build-each-commit-gcc.log", which can then be grabbed as
> artifacts, even when the job timeout. The `git clean` command is
> changed to keep those build logs.
> 
> When run as part of $GITLAB_CI, we will also store the build output in
> a log file instead of writing it to stdout, because GitLab's job log
> is limited. But we will write the log to stdout in case of error, so
> we can find out more quickly why there's been an error.
> 
> This patch also make use of a GitLab feature, "log sections", which we
> will collapse by default. One section per commit been built.
> 
> There's a bit of colour added to the logs.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> 
> Notes:
>     Examples:
>     - on success:
>       https://gitlab.com/xen-project/people/anthonyper/xen/-/jobs/6212972041
>     - on failure:
>       https://gitlab.com/xen-project/people/anthonyper/xen/-/jobs/6212993231
> 
>  automation/gitlab-ci/build-each-commit.sh |  2 +-
>  automation/gitlab-ci/test.yaml            |  4 +-
>  automation/scripts/build-test.sh          | 55 ++++++++++++++++++++---
>  3 files changed, 50 insertions(+), 11 deletions(-)
> 
> diff --git a/automation/gitlab-ci/build-each-commit.sh b/automation/gitlab-ci/build-each-commit.sh
> index 19e337b468..08fddecbbe 100755
> --- a/automation/gitlab-ci/build-each-commit.sh
> +++ b/automation/gitlab-ci/build-each-commit.sh
> @@ -15,4 +15,4 @@ fi
>  echo "Building ${BASE}..${TIP}"
>  
>  NON_SYMBOLIC_REF=1 ./automation/scripts/build-test.sh ${BASE} ${TIP} \
> -    bash -c "git clean -ffdx && ./automation/scripts/build"
> +    bash -c "git clean -ffdx -e '/build-*.log' && ./automation/scripts/build"
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 3b27cc9f41..50056c1372 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -136,9 +136,7 @@ build-each-commit-gcc:
>      XEN_TARGET_ARCH: x86_64
>      CC: gcc
>    script:
> -    - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
> -  after_script:
> -    - mv ../build-each-commit-gcc.log .
> +    - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee build-each-commit-gcc.log
>    artifacts:
>      paths:
>        - '*.log'
> diff --git a/automation/scripts/build-test.sh b/automation/scripts/build-test.sh
> index da643adc01..355d4bf7f6 100755
> --- a/automation/scripts/build-test.sh
> +++ b/automation/scripts/build-test.sh
> @@ -9,6 +9,37 @@
>  # Set NON_SYMBOLIC_REF=1 if you want to use this script in detached HEAD state.
>  # This is currently used by automated test system.
>  
> +# Colors with ANSI escape sequences
> +txt_info='[32m'
> +txt_err='[31m'
> +txt_clr='[0m'
> +
> +# $GITLAB_CI should be "true" or "false".
> +if [ "$GITLAB_CI" != true ]; then
> +    GITLAB_CI=false
> +fi
> +
> +gitlab_log_section() {
> +    if $GITLAB_CI; then
> +        echo -n "[0Ksection_$1:$(date +%s):$2[0K"
> +    fi
> +    if [ $# -ge 3 ]; then
> +        echo "$3"
> +    fi
> +}
> +log_section_last=
> +log_section_start() {
> +    log_section_last="${1%\[collapsed=true\]}"
> +    gitlab_log_section 'start' "$1" "${txt_info}$2${txt_clr}"
> +}
> +log_section_end() {
> +    if [ "$log_section_last" ]; then
> +        gitlab_log_section 'end' "$log_section_last"
> +        log_section_last=
> +    fi
> +}
> +
> +
>  if test $# -lt 2 ; then
>      echo "Usage:"
>      echo " $0 <BASE> <TIP> [CMD]"
> @@ -19,10 +50,12 @@ fi
>  
>  pushd `git rev-parse --show-toplevel`
>  
> -status=`git status -s`
> -if test -n "$status"; then
> -    echo "Tree is dirty, aborted"
> -    exit 1
> +if ! $GITLAB_CI; then
> +    status=`git status -s`
> +    if test -n "$status"; then
> +        echo "Tree is dirty, aborted"
> +        exit 1
> +    fi
>  fi
>  
>  BASE=$1; shift
> @@ -40,26 +73,34 @@ fi
>  
>  ret=1
>  while read num rev; do
> -    echo "Testing $num $rev"
> +    log_section_start "commit_$rev[collapsed=true]" "Testing #$num $(git log -1 --abbrev=12 --format=tformat:'%h ("%s")' $rev)"
>  
>      git checkout $rev
>      ret=$?
>      if test $ret -ne 0; then
> -        echo "Failed to checkout $num $rev with $ret"
> +        log_section_end
> +        echo "${txt_err}Failed to checkout $num $rev with $ret${txt_clr}"
>          break
>      fi
>  
>      if test $# -eq 0 ; then
>          git clean -fdx && ./configure && make -j4
> +    elif $GITLAB_CI; then
> +        "$@" > "build-$num.log" 2>&1
>      else
>          "$@"
>      fi
>      ret=$?
>      if test $ret -ne 0; then
> -        echo "Failed at $num $rev with $ret"
> +        if $GITLAB_CI; then
> +            cat "build-$num.log"
> +        fi
> +        log_section_end
> +        echo "${txt_err}Failed at $num $rev with $ret${txt_clr}"
>          break
>      fi
>      echo
> +    log_section_end
>  done < <(git rev-list $BASE..$TIP | nl -ba | tac)
>  
>  echo "Restoring original HEAD"
> -- 
> Anthony PERARD
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 01:16:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 01:16:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684593.1064535 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdKAl-0006S4-A6; Fri, 23 Feb 2024 01:16:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684593.1064535; Fri, 23 Feb 2024 01:16:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdKAl-0006Rx-7Z; Fri, 23 Feb 2024 01:16:19 +0000
Received: by outflank-mailman (input) for mailman id 684593;
 Fri, 23 Feb 2024 01:16:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdKAj-0006Rp-MU
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 01:16:17 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 22e1a803-d1e9-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 02:16:14 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 17DBD6333E;
 Fri, 23 Feb 2024 01:16:13 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id D81F6C433C7;
 Fri, 23 Feb 2024 01:16:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 22e1a803-d1e9-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708650972;
	bh=bt0v5yeGHnD4YYUl0ZoLlEH5P5CVzY1MYew/SO2Ese0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=glBwmgDH1ptQVJ4b0DpxjYfcyEXDrD7YlmTYcmiliqnUHiPOkfEqi/vv/iobUoqcC
	 pBELv3sz4jxMjv0VGzP5fU12dzptwCKlXpWQAbLvrNpzbuKNlVKUzwH4/sIhbtOjIL
	 ZPpnlR9D+Owg82HbufKpIfUuVdoxWH2+uJnuTjQ5vsRXYufwjWluxjq/HVyD3fWt/N
	 DmuHSkE0lFvXyyk3l9qFNBFTs4PA3hPoUR6jJ8Vkq4qoYPogk1r2Lt+MISUjp6m+wD
	 UisZaeGQcVq57Ti3lt9aqu+s5CVQL0QaZxZPDahQToIObQWKSfP+7uzeJmiEGQw7zA
	 agxHTPjf1HzQQ==
Date: Thu, 22 Feb 2024 17:16:09 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Roger Pau Monne <roger.pau@citrix.com>
cc: xen-devel@lists.xenproject.org, Juergen Gross <jgross@suse.com>, 
    Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
    Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, 
    Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>, 
    x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
    linux-kernel@vger.kernel.org, Jan Beulich <jbeulich@suse.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH RFC] x86/xen: attempt to inflate the memory balloon on
 PVH
In-Reply-To: <20240220174341.56131-1-roger.pau@citrix.com>
Message-ID: <alpine.DEB.2.22.394.2402221701190.754277@ubuntu-linux-20-04-desktop>
References: <20240220174341.56131-1-roger.pau@citrix.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-1734207608-1708650207=:754277"
Content-ID: <alpine.DEB.2.22.394.2402221703300.754277@ubuntu-linux-20-04-desktop>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1734207608-1708650207=:754277
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.22.394.2402221703301.754277@ubuntu-linux-20-04-desktop>

On Tue, 20 Feb 2024, Roger Pau Monne wrote:
> When running as PVH or HVM Linux will use holes in the memory map as scratch
> space to map grants, foreign domain pages and possibly miscellaneous other
> stuff.  However the usage of such memory map holes for Xen purposes can be
> problematic.  The request of holesby Xen happen quite early in the kernel boot
> process (grant table setup already uses scratch map space), and it's possible
> that by then not all devices have reclaimed their MMIO space.  It's not
> unlikely for chunks of Xen scratch map space to end up using PCI bridge MMIO
> window memory, which (as expected) causes quite a lot of issues in the system.

Am I understanding correctly that XEN_BALLOON_MEMORY_HOTPLUG doesn't
help because it becomes available too late in the PVH boot sequence? 



> At least for PVH dom0 we have the possibility of using regions marked as
> UNUSABLE in the e820 memory map.  Either if the region is UNUSABLE in the
> native memory map, or it has been converted into UNUSABLE in order to hide RAM
> regions from dom0, the second stage translation page-tables can populate those
> areas without issues.
> 
> PV already has this kind of logic, where the balloon driver is inflated at
> boot.  Re-use the current logic in order to also inflate it when running as
> PVH.  onvert UNUSABLE regions up to the ratio specified in EXTRA_MEM_RATIO to
> RAM, while reserving them using xen_add_extra_mem() (which is also moved so
> it's no longer tied to CONFIG_PV).
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> RFC reasons:
> 
>  * Note that it would be preferred for the hypervisor to provide an explicit
>    range to be used as scratch mapping space, but that requires changes to Xen,
>    and it's not fully clear whether Xen can figure out the position of all MMIO
>    regions at boot in order to suggest a scratch mapping region for dom0.
> 
>  * Should the whole set of xen_{add,del,chk,inv}_extra_mem() functions be moved
>    to a different file?  For the purposes of PVH only xen_add_extra_mem() is
>    moved and the chk and inv ones are PV specific and might not want moving to
>    a separate file just to guard them with CONFIG_PV.
> ---
>  arch/x86/include/asm/xen/hypervisor.h |  1 +
>  arch/x86/platform/pvh/enlighten.c     |  3 ++
>  arch/x86/xen/enlighten.c              | 32 +++++++++++++
>  arch/x86/xen/enlighten_pvh.c          | 68 +++++++++++++++++++++++++++
>  arch/x86/xen/setup.c                  | 44 -----------------
>  arch/x86/xen/xen-ops.h                | 14 ++++++
>  drivers/xen/balloon.c                 |  2 -
>  7 files changed, 118 insertions(+), 46 deletions(-)
> 
> diff --git a/arch/x86/include/asm/xen/hypervisor.h b/arch/x86/include/asm/xen/hypervisor.h
> index a9088250770f..31e2bf8d5db7 100644
> --- a/arch/x86/include/asm/xen/hypervisor.h
> +++ b/arch/x86/include/asm/xen/hypervisor.h
> @@ -62,6 +62,7 @@ void xen_arch_unregister_cpu(int num);
>  #ifdef CONFIG_PVH
>  void __init xen_pvh_init(struct boot_params *boot_params);
>  void __init mem_map_via_hcall(struct boot_params *boot_params_p);
> +void __init xen_reserve_extra_memory(struct boot_params *bootp);
>  #endif
>  
>  /* Lazy mode for batching updates / context switch */
> diff --git a/arch/x86/platform/pvh/enlighten.c b/arch/x86/platform/pvh/enlighten.c
> index 00a92cb2c814..a12117f3d4de 100644
> --- a/arch/x86/platform/pvh/enlighten.c
> +++ b/arch/x86/platform/pvh/enlighten.c
> @@ -74,6 +74,9 @@ static void __init init_pvh_bootparams(bool xen_guest)
>  	} else
>  		xen_raw_printk("Warning: Can fit ISA range into e820\n");
>  
> +	if (xen_guest)
> +		xen_reserve_extra_memory(&pvh_bootparams);
> +
>  	pvh_bootparams.hdr.cmd_line_ptr =
>  		pvh_start_info.cmdline_paddr;
>  
> diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
> index 3c61bb98c10e..a01ca255b0c6 100644
> --- a/arch/x86/xen/enlighten.c
> +++ b/arch/x86/xen/enlighten.c
> @@ -6,6 +6,7 @@
>  #include <linux/console.h>
>  #include <linux/cpu.h>
>  #include <linux/kexec.h>
> +#include <linux/memblock.h>
>  #include <linux/slab.h>
>  #include <linux/panic_notifier.h>
>  
> @@ -350,3 +351,34 @@ void xen_arch_unregister_cpu(int num)
>  }
>  EXPORT_SYMBOL(xen_arch_unregister_cpu);
>  #endif
> +
> +/* Amount of extra memory space we add to the e820 ranges */
> +struct xen_memory_region xen_extra_mem[XEN_EXTRA_MEM_MAX_REGIONS] __initdata;
> +
> +void __init xen_add_extra_mem(unsigned long start_pfn, unsigned long n_pfns)
> +{
> +	unsigned int i;
> +
> +	/*
> +	 * No need to check for zero size, should happen rarely and will only
> +	 * write a new entry regarded to be unused due to zero size.
> +	 */
> +	for (i = 0; i < XEN_EXTRA_MEM_MAX_REGIONS; i++) {
> +		/* Add new region. */
> +		if (xen_extra_mem[i].n_pfns == 0) {
> +			xen_extra_mem[i].start_pfn = start_pfn;
> +			xen_extra_mem[i].n_pfns = n_pfns;
> +			break;
> +		}
> +		/* Append to existing region. */
> +		if (xen_extra_mem[i].start_pfn + xen_extra_mem[i].n_pfns ==
> +		    start_pfn) {
> +			xen_extra_mem[i].n_pfns += n_pfns;
> +			break;
> +		}
> +	}
> +	if (i == XEN_EXTRA_MEM_MAX_REGIONS)
> +		printk(KERN_WARNING "Warning: not enough extra memory regions\n");
> +
> +	memblock_reserve(PFN_PHYS(start_pfn), PFN_PHYS(n_pfns));
> +}
> diff --git a/arch/x86/xen/enlighten_pvh.c b/arch/x86/xen/enlighten_pvh.c
> index ada3868c02c2..c28f073c1df5 100644
> --- a/arch/x86/xen/enlighten_pvh.c
> +++ b/arch/x86/xen/enlighten_pvh.c
> @@ -1,6 +1,7 @@
>  // SPDX-License-Identifier: GPL-2.0
>  #include <linux/acpi.h>
>  #include <linux/export.h>
> +#include <linux/mm.h>
>  
>  #include <xen/hvc-console.h>
>  
> @@ -72,3 +73,70 @@ void __init mem_map_via_hcall(struct boot_params *boot_params_p)
>  	}
>  	boot_params_p->e820_entries = memmap.nr_entries;
>  }
> +
> +/*
> + * Reserve e820 UNUSABLE regions to inflate the memory balloon.
> + *
> + * On PVH dom0 the host memory map is used, RAM regions available to dom0 are
> + * located as the same place as in the native memory map, but since dom0 gets
> + * less memory than the total amount of host RAM the ranges that can't be
> + * populated are converted from RAM -> UNUSABLE.  Use such regions (up to the
> + * ratio signaled in EXTRA_MEM_RATIO) in order to inflate the balloon driver at
> + * boot.  Doing so prevents the guest (even if just temporary) from using holes
> + * in the memory map in order to map grants or foreign addresses, and
> + * hopefully limits the risk of a clash with a device MMIO region.  Ideally the
> + * hypervisor should notify us which memory ranges are suitable for creating
> + * foreign mappings, but that's not yet implemented.
> + */
> +void __init xen_reserve_extra_memory(struct boot_params *bootp)
> +{
> +	unsigned int i, ram_pages = 0, extra_pages;
> +
> +	for (i = 0; i < bootp->e820_entries; i++) {
> +		struct boot_e820_entry *e = &bootp->e820_table[i];
> +
> +		if (e->type != E820_TYPE_RAM)
> +			continue;
> +		ram_pages += PFN_DOWN(e->addr + e->size) - PFN_UP(e->addr);
> +	}
> +
> +	/* Max amount of extra memory. */
> +	extra_pages = EXTRA_MEM_RATIO * ram_pages;
> +
> +	/*
> +	 * Convert UNUSABLE ranges to RAM and reserve them for foreign mapping
> +	 * purposes.
> +	 */
> +	for (i = 0; i < bootp->e820_entries && extra_pages; i++) {
> +		struct boot_e820_entry *e = &bootp->e820_table[i];
> +		unsigned long pages;
> +
> +		if (e->type != E820_TYPE_UNUSABLE)
> +			continue;
> +
> +		pages = min(extra_pages,
> +			PFN_DOWN(e->addr + e->size) - PFN_UP(e->addr));
> +
> +		if (pages != (PFN_DOWN(e->addr + e->size) - PFN_UP(e->addr))) {
> +			struct boot_e820_entry *next;
> +
> +			if (bootp->e820_entries ==
> +			    ARRAY_SIZE(bootp->e820_table))
> +				/* No space left to split - skip region. */
> +				continue;
> +
> +			/* Split entry. */
> +			next = e + 1;
> +			memmove(next, e,
> +				(bootp->e820_entries - i) * sizeof(*e));
> +			bootp->e820_entries++;
> +			next->addr = PAGE_ALIGN(e->addr) + PFN_PHYS(pages);
> +			e->size = next->addr - e->addr;
> +			next->size -= e->size;

Is this really worth doing? Can we just skip this range and continue or
simply break out and call it a day? Or even add the whole range instead?

The reason I am asking is that I am expecting E820_TYPE_UNUSABLE regions
not to be huge. Splitting one just to cover the few remaining pages out
of extra_pages doesn't seem worth it?

Everything else looks OK to me.


> +		}
> +		e->type = E820_TYPE_RAM;
> +		extra_pages -= pages;
> +
> +		xen_add_extra_mem(PFN_UP(e->addr), pages);
> +	}
> +}
> diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
> index b3e37961065a..380591028cb8 100644
> --- a/arch/x86/xen/setup.c
> +++ b/arch/x86/xen/setup.c
> @@ -38,9 +38,6 @@
>  
>  #define GB(x) ((uint64_t)(x) * 1024 * 1024 * 1024)
>  
> -/* Amount of extra memory space we add to the e820 ranges */
> -struct xen_memory_region xen_extra_mem[XEN_EXTRA_MEM_MAX_REGIONS] __initdata;
> -
>  /* Number of pages released from the initial allocation. */
>  unsigned long xen_released_pages;
>  
> @@ -64,18 +61,6 @@ static struct {
>  } xen_remap_buf __initdata __aligned(PAGE_SIZE);
>  static unsigned long xen_remap_mfn __initdata = INVALID_P2M_ENTRY;
>  
> -/*
> - * The maximum amount of extra memory compared to the base size.  The
> - * main scaling factor is the size of struct page.  At extreme ratios
> - * of base:extra, all the base memory can be filled with page
> - * structures for the extra memory, leaving no space for anything
> - * else.
> - *
> - * 10x seems like a reasonable balance between scaling flexibility and
> - * leaving a practically usable system.
> - */
> -#define EXTRA_MEM_RATIO		(10)
> -
>  static bool xen_512gb_limit __initdata = IS_ENABLED(CONFIG_XEN_512GB);
>  
>  static void __init xen_parse_512gb(void)
> @@ -96,35 +81,6 @@ static void __init xen_parse_512gb(void)
>  	xen_512gb_limit = val;
>  }
>  
> -static void __init xen_add_extra_mem(unsigned long start_pfn,
> -				     unsigned long n_pfns)
> -{
> -	int i;
> -
> -	/*
> -	 * No need to check for zero size, should happen rarely and will only
> -	 * write a new entry regarded to be unused due to zero size.
> -	 */
> -	for (i = 0; i < XEN_EXTRA_MEM_MAX_REGIONS; i++) {
> -		/* Add new region. */
> -		if (xen_extra_mem[i].n_pfns == 0) {
> -			xen_extra_mem[i].start_pfn = start_pfn;
> -			xen_extra_mem[i].n_pfns = n_pfns;
> -			break;
> -		}
> -		/* Append to existing region. */
> -		if (xen_extra_mem[i].start_pfn + xen_extra_mem[i].n_pfns ==
> -		    start_pfn) {
> -			xen_extra_mem[i].n_pfns += n_pfns;
> -			break;
> -		}
> -	}
> -	if (i == XEN_EXTRA_MEM_MAX_REGIONS)
> -		printk(KERN_WARNING "Warning: not enough extra memory regions\n");
> -
> -	memblock_reserve(PFN_PHYS(start_pfn), PFN_PHYS(n_pfns));
> -}
> -
>  static void __init xen_del_extra_mem(unsigned long start_pfn,
>  				     unsigned long n_pfns)
>  {
> diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
> index a87ab36889e7..79cf93f2c92f 100644
> --- a/arch/x86/xen/xen-ops.h
> +++ b/arch/x86/xen/xen-ops.h
> @@ -163,4 +163,18 @@ void xen_hvm_post_suspend(int suspend_cancelled);
>  static inline void xen_hvm_post_suspend(int suspend_cancelled) {}
>  #endif
>  
> +/*
> + * The maximum amount of extra memory compared to the base size.  The
> + * main scaling factor is the size of struct page.  At extreme ratios
> + * of base:extra, all the base memory can be filled with page
> + * structures for the extra memory, leaving no space for anything
> + * else.
> + *
> + * 10x seems like a reasonable balance between scaling flexibility and
> + * leaving a practically usable system.
> + */
> +#define EXTRA_MEM_RATIO		(10)
> +
> +void xen_add_extra_mem(unsigned long start_pfn, unsigned long n_pfns);
> +
>  #endif /* XEN_OPS_H */
> diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
> index 976c6cdf9ee6..aaf2514fcfa4 100644
> --- a/drivers/xen/balloon.c
> +++ b/drivers/xen/balloon.c
> @@ -672,7 +672,6 @@ EXPORT_SYMBOL(xen_free_ballooned_pages);
>  
>  static void __init balloon_add_regions(void)
>  {
> -#if defined(CONFIG_XEN_PV)
>  	unsigned long start_pfn, pages;
>  	unsigned long pfn, extra_pfn_end;
>  	unsigned int i;
> @@ -696,7 +695,6 @@ static void __init balloon_add_regions(void)
>  
>  		balloon_stats.total_pages += extra_pfn_end - start_pfn;
>  	}
> -#endif
>  }
>  
>  static int __init balloon_init(void)
> -- 
> 2.43.0
> 
--8323329-1734207608-1708650207=:754277--


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 01:28:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 01:28:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684598.1064546 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdKMI-00005T-Bn; Fri, 23 Feb 2024 01:28:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684598.1064546; Fri, 23 Feb 2024 01:28:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdKMI-00005M-8O; Fri, 23 Feb 2024 01:28:14 +0000
Received: by outflank-mailman (input) for mailman id 684598;
 Fri, 23 Feb 2024 01:28:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdKMH-00005G-1J
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 01:28:13 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ce073b2a-d1ea-11ee-8a55-1f161083a0e0;
 Fri, 23 Feb 2024 02:28:11 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 63BF7CE28EA;
 Fri, 23 Feb 2024 01:28:07 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C22A4C433C7;
 Fri, 23 Feb 2024 01:28:03 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce073b2a-d1ea-11ee-8a55-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708651686;
	bh=46goDBKRAJzr8WceLOh1hdvsIDopHFd3Eu4bfTNcDb0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=QKZRClp7rQQWEYKGiCIk+EupIbY/kt+Ccrp16QgSV3gw7yOr0y7RUTYLr44pbpFrZ
	 BlpyaMg/jPEQIjaliY/+/GbmhYztDMtZRxtnf4KLgi38GcblrV3x8JACVkt/kVnqMk
	 iYc6jqToNm5lNC4KsNn3YM0S/cRynhSpyepZRchafJbhtn1OcDD2+IypfHc25WvTpf
	 JpdfNOMDQG4XHqMuHuRgr9MUBTjKwwzOx7xDPYGa2RgYRDUPveacpsMLqZrltcgsnG
	 V0k18OTFUJDf9K30mG7FqwMD3kz4v7lvk1pw4eIK6KBb+q1PrWbICUgqR20SMhs/uY
	 a42LJLdfHN95A==
Date: Thu, 22 Feb 2024 17:28:02 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, bertrand.marquis@arm.com, julien@xen.org, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH v2] automation/eclair_analysis: deviate certain macros
 for Rule 20.12
In-Reply-To: <7bc72f6ccd858e0405d62d49c32449e3b5abd790.1707996317.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402221727370.754277@ubuntu-linux-20-04-desktop>
References: <7bc72f6ccd858e0405d62d49c32449e3b5abd790.1707996317.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 15 Feb 2024, Nicola Vetrini wrote:
> Certain macros are allowed to violate the Rule, since their meaning and
> intended use is well-known to all Xen developers.
> 
> Variadic macros that rely on the GCC extension for removing a trailing
> comma when token pasting the variable argument are similarly
> well-understood and therefore allowed.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>



> ---
> Changes in v2:
> - Restrict deviation for GENERATE_CASE to vcpreg.c.
> - Improve deviation justifications.
> ---
>  .../eclair_analysis/ECLAIR/deviations.ecl     | 20 +++++++++++++++++
>  docs/misra/deviations.rst                     | 22 +++++++++++++++++++
>  2 files changed, 42 insertions(+)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
> index fd32ff8a9cae..04cb41e16a50 100644
> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> @@ -387,6 +387,26 @@ in assignments."
>  {safe, "left_right(^[(,\\[]$,^[),\\]]$)"}
>  -doc_end
>  
> +-doc_begin="Uses of variadic macros that have one of their arguments defined as
> +a macro and used within the body for both ordinary parameter expansion and as an
> +operand to the # or ## operators have a behavior that is well-understood and
> +deliberate."
> +-config=MC3R1.R20.12,macros+={deliberate, "variadic()"}
> +-doc_end
> +
> +-doc_begin="Uses of a macro parameter for ordinary expansion and as an operand
> +to the # or ## operators within the following macros are deliberate, to provide
> +useful diagnostic messages to the user."
> +-config=MC3R1.R20.12,macros+={deliberate, "name(ASSERT||BUILD_BUG_ON||BUILD_BUG_ON_ZERO)"}
> +-doc_end
> +
> +-doc_begin="The helper macro GENERATE_CASE may use a macro parameter for ordinary
> +expansion and token pasting to improve readability. Only instances where this
> +leads to a violation of the Rule are deviated."
> +-file_tag+={deliberate_generate_case, "^xen/arch/arm/vcpreg\\.c$"}
> +-config=MC3R1.R20.12,macros+={deliberate, "name(GENERATE_CASE)&&loc(file(deliberate_generate_case))"}
> +-doc_end
> +
>  #
>  # General
>  #
> diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
> index 123c78e20a01..84da637ef888 100644
> --- a/docs/misra/deviations.rst
> +++ b/docs/misra/deviations.rst
> @@ -330,6 +330,28 @@ Deviations related to MISRA C:2012 Rules:
>         (4) as lhs in assignments.
>       - Tagged as `safe` for ECLAIR.
>  
> +   * - R20.12
> +     - Variadic macros that use token pasting often employ the gcc extension
> +       `ext_paste_comma`, as detailed in `C-language-toolchain.rst`, which is
> +       not easily replaceable; macros that in addition perform regular argument
> +       expansion on the same argument subject to the # or ## operators violate
> +       the Rule if the argument is a macro. 
> +     - Tagged as `deliberate` for ECLAIR.
> +
> +   * - R20.12
> +     - Macros that are used for runtime or build-time assertions contain
> +       deliberate uses of an argument as both a regular argument and a
> +       stringification token, to provide useful diagnostic messages.
> +     - Tagged as `deliberate` for ECLAIR.
> +
> +   * - R20.12
> +     - GENERATE_CASE is a local helper macro that allows some selected switch
> +       statements to be more compact and readable. As such, the risk of
> +       developer confusion in using such macro is deemed negligible. This
> +       construct is deviated only in Translation Units that present a violation
> +       of the Rule due to uses of this macro.
> +     - Tagged as `deliberate` for ECLAIR.
> +
>  Other deviations:
>  -----------------
>  
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 01:32:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 01:32:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684601.1064556 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdKQd-00026n-VV; Fri, 23 Feb 2024 01:32:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684601.1064556; Fri, 23 Feb 2024 01:32:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdKQd-00026g-Qh; Fri, 23 Feb 2024 01:32:43 +0000
Received: by outflank-mailman (input) for mailman id 684601;
 Fri, 23 Feb 2024 01:32:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdKQd-00026a-Ea
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 01:32:43 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6e7a369a-d1eb-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 02:32:40 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 4B0EA632EE;
 Fri, 23 Feb 2024 01:32:39 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 878DAC433C7;
 Fri, 23 Feb 2024 01:32:35 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6e7a369a-d1eb-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708651959;
	bh=nWVJG5spi+bRiLS0xJbmbmJBV5/WP5FVK+t19DwXej0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=d/rzkEhFKB9rvzUoH1JhGpBvjrfSa1JmDj0RHz/a9wEpeiAiKgcOQkWx/K8/47fdE
	 25L7socT6E25oRPs0u79xSzKbNff/D43h9Irpi4l9ZfwazmCDDQu7/pyhY1rxvNRRI
	 TXYGVoLSWp14KQwBXjiPB78v3Eo3SpxvJ6YKZU860JM5DKDS/X1VzwuEgViaUni5iX
	 zvRpsnwsaJ0cE5zhHJpn/7wvgQrwZgEowDeUH19L2rTL0sdphpR8HALabH8ZJFGIv6
	 7o+FTlOr/4lRpl0F7HwwY4mhFcRUviBRf8VPXeaF/Pj0JeKnksRf5ZkoHnRzFgxCtY
	 pU/wHMHsEEQkw==
Date: Thu, 22 Feb 2024 17:32:33 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Simone Ballarin <simone.ballarin@bugseng.com>, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 3/4] xen/include: add pure and const attributes
In-Reply-To: <8f8141e2-c55c-2e58-a078-d5d8764a02df@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402221730270.754277@ubuntu-linux-20-04-desktop>
References: <cover.1697638210.git.simone.ballarin@bugseng.com> <fd5421162a00aa782e0776324ff6497193c1e3d3.1697638210.git.simone.ballarin@bugseng.com> <89778285-5cba-8fb5-70bc-710b6dd30a10@suse.com> <0032186f-80c7-4dba-b46e-10d4a8e2a8cb@bugseng.com>
 <e8bf9817-fd54-9bf4-4302-dcee682f9172@suse.com> <alpine.DEB.2.22.394.2310231417260.3516@ubuntu-linux-20-04-desktop> <8f8141e2-c55c-2e58-a078-d5d8764a02df@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="8323329-195102866-1708651958=:754277"

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-195102866-1708651958=:754277
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Tue, 24 Oct 2023, Jan Beulich wrote:
> On 24.10.2023 00:05, Stefano Stabellini wrote:
> > On Mon, 23 Oct 2023, Jan Beulich wrote:
> >> On 23.10.2023 17:23, Simone Ballarin wrote:
> >>> On 23/10/23 15:34, Jan Beulich wrote:
> >>>> On 18.10.2023 16:18, Simone Ballarin wrote:
> >>>>> --- a/xen/include/xen/pdx.h
> >>>>> +++ b/xen/include/xen/pdx.h
> >>>>> @@ -164,7 +164,7 @@ static inline unsigned long pfn_to_pdx(unsigned long pfn)
> >>>>>    * @param pdx Page index
> >>>>>    * @return Obtained pfn after decompressing the pdx
> >>>>>    */
> >>>>> -static inline unsigned long pdx_to_pfn(unsigned long pdx)
> >>>>> +static inline __attribute_pure__ unsigned long pdx_to_pfn(unsigned long pdx)
> >>>>>   {
> >>>>>       return (pdx & pfn_pdx_bottom_mask) |
> >>>>>              ((pdx << pfn_pdx_hole_shift) & pfn_top_mask);
> >>>>
> >>>> Taking this as an example for what I've said above: The compiler can't
> >>>> know that the globals used by the functions won't change value. Even
> >>>> within Xen it is only by convention that these variables are assigned
> >>>> their values during boot, and then aren't changed anymore. Which makes
> >>>> me wonder: Did you check carefully that around the time the variables
> >>>> have their values established, no calls to the functions exist (which
> >>>> might then be subject to folding)?
> >>>
> >>> There is no need to check that, the GCC documentation explicitly says:
> >>>
> >>> However, functions declared with the pure attribute *can safely read any 
> >>> non-volatile objects*, and modify the value of objects in a way that 
> >>> does not affect their return value or the observable state of the program.
> >>
> >> I did quote this same text in response to what Andrew has said, but I also
> >> did note there that this needs to be taken with a grain of salt: The
> >> compiler generally assumes a single-threaded environment, i.e. no changes
> >> to globals behind the back of the code it is processing.
> > 
> > Let's start from the beginning. The reason for Simone to add
> > __attribute_pure__ to pdx_to_pfn and other functions is for
> > documentation purposes. It is OK if it doesn't serve any purpose other
> > than documentation.
> > 
> > Andrew, for sure we do not want to lie to the compiler and introduce
> > undefined behavior. If we think there is a risk of it, we should not do
> > it.
> > 
> > So, what do we want to document? We want to document that the function
> > does not have side effects according to MISRA's definition of it, which
> > might subtly differ from GCC's definition.
> > 
> > Looking at GCC's definition of __attribute_pure__, with the
> > clarification statement copy/pasted above by both Simone and Jan, it
> > seems that __attribute_pure__ matches MISRA's definition of a function
> > without side effects. It also seems that pdx_to_pfn abides to that
> > definition.
> > 
> > Jan has a point that GCC might be making other assumptions
> > (single-thread execution) that might not hold true in our case. Given
> > the way the GCC statement is written I think this is low risk. But maybe
> > not all GCC versions we want to support in the project might have the
> > same definition of __attribute_pure__. So we could end up using
> > __attribute_pure__ correctly for the GCC version used for safety (GCC
> > 12.1, see docs/misra/C-language-toolchain.rst) but it might actually
> > break an older GCC version.
> > 
> > 
> > So Option#1 is to use __attribute_pure__ taking the risk that a GCC or
> > Clang version might interpret __attribute_pure__ differently and
> > potentially misbehave.
> > 
> > Option#2 is to avoid this risk, by not using __attribute_pure__.
> > Instead, we can use SAF-xx-safe or deviations.rst to document that
> > pdx_to_pfn and other functions like it are without side effects
> > according to MISRA's definition.
> > 
> > 
> > Both options have pros and cons. To me the most important factor is how
> > many GCC versions come with the statement "pure attribute can safely
> > read any non-volatile objects, and modify the value of objects in a way
> > that does not affect their return value or the observable state of the
> > program".
> > 
> > I checked and these are the results:
> > - gcc 4.0.2: no statement
> > - gcc 5.1.0: no statement
> > - gcc 6.1.0: no statement
> > - gcc 7.1.0: no statement
> > - gcc 8.1.0: alternative statement "The pure attribute imposes similar
> >   but looser restrictions on a function’s definition than the const
> >   attribute: it allows the function to read global variables."
> > - gcc 9.1.0: yes statement
> > 
> > 
> > So based on the above, __attribute_pure__ comes with its current
> > definition only from gcc 9 onward. I don't know if as a Xen community we
> > clearly declare a range of supported compilers, but I would imagine we
> > would still want to support gcc versions older than 9? (Not to mention
> > clang, which I haven't checked.)
> > 
> > It doesn't seem to me that __attribute_pure__ could be correctly used on
> > pdx_to_pfn with GCC 7.1.0 for example.
> 
> The absence of documentation doesn't mean the attribute had different
> (or even undefined) meaning in earlier versions. Instead it means one
> would need to consult other places (source code?) to figure out whether
> there was any behavioral difference (I don't think there was).
> 
> That said, ...
> 
> > So in conclusion, I think it is better to avoid __attribute_pure__ and
> > use SAF-xx-safe or an alternative approach instead.
> 
> ... I agree here. We just don't want to take chances.

Let me resurrect this thread.

Could we use something like "pure" that we #define as we want?

Depending on the compiler version or other options we could #define pure
to __attribute_pure__ or to nothing.

Opinions?
--8323329-195102866-1708651958=:754277--


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 04:53:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 04:53:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684610.1064566 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdNZ5-0000Y6-DD; Fri, 23 Feb 2024 04:53:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684610.1064566; Fri, 23 Feb 2024 04:53:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdNZ5-0000Xz-7y; Fri, 23 Feb 2024 04:53:39 +0000
Received: by outflank-mailman (input) for mailman id 684610;
 Fri, 23 Feb 2024 04:53:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdNZ4-0000Xp-3o; Fri, 23 Feb 2024 04:53:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdNZ3-0008VE-T4; Fri, 23 Feb 2024 04:53:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdNZ3-0001bc-Gf; Fri, 23 Feb 2024 04:53:37 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdNZ3-0000EM-GA; Fri, 23 Feb 2024 04:53:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=3Q3+b8RUMRiUuc5GKERof25jBGyRdm6ukQ2uV/p3u64=; b=CiiCT+nf48V5J4ZqoOZ3lbsXHh
	jyeH/6xNmWqd1OboCkVvIyrYTN0YxB8BjcLn+HK5aAZf/XbhLiqxZlvcfAH6Tdyio9Mp91SwCWbYN
	Dj4SGwChl+JfhV+/ZagYlKi2lWzTP+Bg/bK3yvMKHMqTkNXcAEeTzOgi6vboPJNYkA1U=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184729-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184729: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=9ee7dc877b8754ce2fc82500feea52c04d4e6409
X-Osstest-Versions-That:
    xen=f8791d0fd3adbda3701e7eb9db63a9351b478365
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 23 Feb 2024 04:53:37 +0000

flight 184729 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184729/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184723
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184723
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184723
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184723
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184723
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184723
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail like 184723
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184723
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184723
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184723
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184723
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184723
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184723
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 xen                  9ee7dc877b8754ce2fc82500feea52c04d4e6409
baseline version:
 xen                  f8791d0fd3adbda3701e7eb9db63a9351b478365

Last test of basis   184723  2024-02-22 01:52:14 Z    1 days
Testing same since   184729  2024-02-22 14:07:38 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Daniel P. Smith <dpsmith@apertussolutions.com> # XSM
  Federico Serafini <federico.serafini@bugseng.com>
  Frediano Ziglio <frediano.ziglio@cloud.com>
  George Dunlap <george.dunlap@cloud.com> # sched
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f8791d0fd3..9ee7dc877b  9ee7dc877b8754ce2fc82500feea52c04d4e6409 -> master


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 06:04:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 06:04:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684630.1064611 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdOfX-0000Pv-V4; Fri, 23 Feb 2024 06:04:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684630.1064611; Fri, 23 Feb 2024 06:04:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdOfX-0000Po-SR; Fri, 23 Feb 2024 06:04:23 +0000
Received: by outflank-mailman (input) for mailman id 684630;
 Fri, 23 Feb 2024 06:04:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SdR/=KA=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1rdOfW-0000Pi-Js
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 06:04:22 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2412::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 624a535e-d211-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 07:04:20 +0100 (CET)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by MW4PR12MB7440.namprd12.prod.outlook.com (2603:10b6:303:223::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.24; Fri, 23 Feb
 2024 06:04:16 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa%5]) with mapi id 15.20.7316.018; Fri, 23 Feb 2024
 06:04:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 624a535e-d211-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=SQZoSsZFK7i+Z0OJkr6Pw9M7wqMJxUyt0McpAIjhsxroK2rRd9iQ6PK5tO10A23vRmi2ufVZrrJN3936YQFST2uWrF1yFNs8oruAz7nB5UKuM6JhGWwwlUusYf7gkxHAA7SxyFOLX0fAuHWj2A2Fg5LYiy9tuRJLoawXkIrEZEJL6EtLN3vIfwZ2p1qAg6HOjpYf/GNjcM1OEQhas+q6NnwfUJyeGYaDPeZPAfu7VedCUyJhoP6pbdaDjAtm36qm4VTWSJnfcFTluI6yBpNMsb83QkcJv0vJwWKbcy8DGcd7D0l5DOFE7nACOkfv5JDcrPKWORoDPMDidk2GKHs08A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=6h8pS0n132sUF0hKdhZ2d9lg8e0IXpLC/UbvJyNzD50=;
 b=mRm+wW3v3ZlPrD1Tl41MNQ77bIIK4sHXXqMRrmOMEyFbGTU+fA2hIC8FURmAxA14KKW8DG8VQOY0rUk4Vffa0AHRcU62mGZyQh0jDEVaXYEuylgo59y9FlmZF4wlMiXWBk4ZnY2NaNCFclyTbTtBRcnc+LESZYvetnlyhm2WomeQh0pI78MjrjOo728SwsNs23lTWSKOo9looJ9L+5TSeVEIIfTSyIcsLIMLH/oFaaGMuLS83ZJV9wgbAXXxFVzrZfhi0tz4jep7fo80CpfZIg/enaiqeNQStGuG/o1s+FCzKywQG6w+jLlRZK1i5YVteHOu1KkU2TRssZ2MEmdPjA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=6h8pS0n132sUF0hKdhZ2d9lg8e0IXpLC/UbvJyNzD50=;
 b=4mvV7D2lZNVLykxYg7yhll/kNPuxZ6SPVJIvQgehnw0eAKKNB/VOL9kG7c+yuHlQ3RrenTX2zESQwUo8/BDBcgzGuZWzbKz+w8kH9sHp1JEuKBlGHJ3enCuBFjNzUROucDzYISGV/CY3xnlLcDVBAEb+g8VrOQ/9oHUAfRGBvcs=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Jan
 Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v5 2/5] x86/pvh: Allow (un)map_pirq when dom0 is
 PVH
Thread-Topic: [RFC XEN PATCH v5 2/5] x86/pvh: Allow (un)map_pirq when dom0 is
 PVH
Thread-Index: AQHaRR6zD8fXe9R1qEqrvmeKDYHBArEXV7EAgADevoA=
Date: Fri, 23 Feb 2024 06:04:14 +0000
Message-ID:
 <BL1PR12MB58497693D450A36C84EE8B22E7552@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-3-Jiqian.Chen@amd.com>
 <alpine.DEB.2.22.394.2402221632170.754277@ubuntu-linux-20-04-desktop>
In-Reply-To:
 <alpine.DEB.2.22.394.2402221632170.754277@ubuntu-linux-20-04-desktop>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: MN0PR12MB6150.namprd12.prod.outlook.com
 (15.20.7292.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|MW4PR12MB7440:EE_
x-ms-office365-filtering-correlation-id: c133f575-0025-49a1-afdc-08dc34354348
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 WloyhT6puv2EpfsGWrCyWDJT454ZClmO9yAsRh8WJvFu4xZw6SYrqMvtiNIap7xC4tIafPoHnOGmBcy5c4IZUnnS0HRNZjqUffOiRUVewg/q6xN2auNVuBGaM/C/buASG8yHSLXxEkAHJIwT9mwwxRBLoDezU++Ir8oqEYOgUdVqGMVD9s2gTQhjBqSBZRazDbSKpAoOVk86ajJ5RP3863kFk+qTz3DULTonRfqqhkuLO1ge6qzsQ7296vRv3iiHzSn4im8yQogWUqbg6l9Rg/8ezx7quz4bFiXXSjb1dSZf5r4iOc5hOmWUeiFu+p745ehj/hxYbsIoT4X6POxjTxzPNvKmsDBF18BD4HjCaSPzX6YvNaLvAriedCdbQ4NyArlPQJ/5iH21CBYYVPScRnJzRD5vi2q3+oN1ZGWLAUY+3xg0YuRcVy7/pmlJIAzYxzpX2HJmLix/Yc9YcVX38Ta168d7GdjwRnC0br4UPrECbvQ+yDkYdGWMicafhDRuqSPeWwoQkRPKlErcpw2QYJ7jRQVKKc6FrxxSWY3s9OmIY/3bavk9GxkwaCRIG+a/4M5jFpz3GUzjnwwbdEyebc2nZnrUScGQ+WIMJbSsXPL+prGixd46HKTk6JHFyO82
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?SkxiMVF3WFdlSmZMek9XbzJLd1VQNWpCaE9PNXg2d1NVcnBNWVMwUkZIVnVC?=
 =?utf-8?B?bVloUEtyMmt2RUJJV3h4VzN5eW1WVHgxcW5EZG1MeEJ0Unl0Zmc1alBiMjgz?=
 =?utf-8?B?Sm8xS3ZrNTR1clFEYUFxVXR5VE5LeW04Wk80YndtMVVCUUVPVjFSci9HNEtC?=
 =?utf-8?B?d0VOZ1F6ZzAvaHE5ZlJzNThJMXgrdXJyWGJJNzZLemx4ZWNVS1kvd0djU2NL?=
 =?utf-8?B?UmIyRGYxbmJUMEdXVEV0bDdKSmwzWTFYKzc0d0ludE9KVXZ0VEQwd0RicHFU?=
 =?utf-8?B?eWpXUUpmeDYyVkJDRVVKU01XRjdvWUQrOXZPd3ZCci9YKzlNMEQ3TDhrdExn?=
 =?utf-8?B?QURwRC9WbGlNRkJ0RlJReEdHMHVLd1FoWjhlUkhFL24zZ0hWbVZYaU9aaFNx?=
 =?utf-8?B?STFGODFla0J0RWg1bStnQXpqMnZCMHlVaXA3bXkrWFRBcVNmaWV5NGYyUUtB?=
 =?utf-8?B?SHpISUdSZ2x6eUcyc2theVRnTFFWMzVIVCs2R3lkSXdlS0lOVWROS2czRXVp?=
 =?utf-8?B?VWFCU2hJbjFvbTllcndKSmVPYTVVSDB2cER0ZVhINk5EUC9uMi9hQTE2MFRH?=
 =?utf-8?B?N3dUdXdLZEhUUlBrRUc2S2s3Ky9HMUJxYVBmbHJYSEN4ZWFFN0JuOUNlcjgv?=
 =?utf-8?B?cnlnSkg3ZGpYdHhXMDdGdnltM1JXSWs5ZXVSUW1iem1IVDVnLzV0c0xQWHh5?=
 =?utf-8?B?SG81dVgrdEt2ZEFmbjRMWGZRNndsZGhUdDlpQmx6SFZHNEludjgybVM3UUh5?=
 =?utf-8?B?SytjclNlZDFxd3p4aWFsNkp5Nk16WmN2c3VSdXFSZXdUK3NSYzYwd0Zhamh3?=
 =?utf-8?B?U2hlMUJjQUxWZEJza3NCWnUwSGRicXMvbVh0enZYQ0kzMVRkSXBNNk5UVXRY?=
 =?utf-8?B?WUI4RG9RMG10T2h3REZTVURSOTB0d1ZNSkdRdTFhZ1BQTThhMTVtK3c0Z2w1?=
 =?utf-8?B?ZnFKbUJZNFc5UnIzOFNsT0hVSUpHN0FUaTZVM3F1K29lSWxIczJmTU90Tk4r?=
 =?utf-8?B?d1FLaFhRZXRWUnIwNkp6YW9JWm9NRTdSRElyUkYyeUhmL2Z2eExYVFdiRG5F?=
 =?utf-8?B?TWEwNGJkei90dUVodk1LU0FGYkNaZkc4OGxWSzRXMUtxdnRldURmYi91dmND?=
 =?utf-8?B?UjNGMHQ3dk9ZREYrcnhidGE3Y0NjeUN1YzA0U3l3aG5PUXQ5VFV3aDRoeWR6?=
 =?utf-8?B?VFpvVjFVeGcxLzVKOEZmWmZFVCttajhRSWlwTHFvdnA4eHNiSmVFTVg0VXY2?=
 =?utf-8?B?OWhyQXFXS0dJUXdIa01CUXNiOVRVdUYzYVdCbjZONkZXbzJ5MmNGSFpsNlkv?=
 =?utf-8?B?YTdpSzgzd29qYVJmTjAvTHZJZ0NGeW9MeFF6K0lxcGpndnlFQUtiL3QvUkUw?=
 =?utf-8?B?WE9UWXpld2VaT2p5T2pqTTdJemZYd09PRnViNWtXU0ZtNGU0T0ZQS21VdjVy?=
 =?utf-8?B?NEo3amNVRXZCTWNiUGkvUlQ1UERsT1FPMXVuM2xuOXhldWJEWWV1cTZPU2gz?=
 =?utf-8?B?enNGSnFDdzYzWnc4UGxHakpNZHBLQjJIM0s0U2tyMlcxRTBLUDZGeGxGTFJR?=
 =?utf-8?B?MjVhV1R6OFdEaW5kZmJuMERlVzZ4Kzd0cE1wdGphZzVTam1XakZhWWJPMXRS?=
 =?utf-8?B?RHJIQU55YnN0di9ZR3FJcys1T0kxTFp1UUgraktGWlpBSGdwMFZUOERoeGl3?=
 =?utf-8?B?bUpZODZlZXo5Lzg5WVBqTzZOWGFLMUU4VW5BMnNxdkRISkp1MkVTcE0wT25G?=
 =?utf-8?B?N0EzWUo0SDFUUTFBa2FnQ0doL0lIRU1zWEU0WUpCUUJPM0V2S2xuM01tZVlJ?=
 =?utf-8?B?V0R2aDh4LzdaWVhlWTJvbHAvVGFtSTZNanJIWE1OZDBXMWJlMzh4YkViUDFW?=
 =?utf-8?B?ajM3aGUya1dpZHdLOFBOdVk4b0Q1YlY1TGQzR3ZlWWZhR2lWZDJMQUFqb1VB?=
 =?utf-8?B?QW1GRFAzRXh3NWVCekpkL3VIL0xDcU9UZGFuRHdVaVVEZ0dUYUh1WStMYTNx?=
 =?utf-8?B?cktyZDdvcWs0K3k5dWc5UVB3UDFlRUpXc0NWQjk3T1FVeW5rbU5mSURZc1lq?=
 =?utf-8?B?MWNsYktFUmlZdE9YMTZBMG42aTFLeG5SU0NCMHhzMEVaODc1a2J0bmRFUE9x?=
 =?utf-8?Q?VySM=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <95985E6D6ED7F748BDC67DEEF2164EEB@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c133f575-0025-49a1-afdc-08dc34354348
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Feb 2024 06:04:14.6085
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Tk/NZn0GNcirDsb972nYzWPkKEJzyu5sEHQu1O/7KuUBd1DEVoJhaVBgQfghIJa7iIBwB7/UKVXf/Sevdl2pBw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7440

T24gMjAyNC8yLzIzIDA4OjQwLCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+IE9uIEZyaSwg
MTIgSmFuIDIwMjQsIEppcWlhbiBDaGVuIHdyb3RlOg0KPj4gSWYgcnVuIFhlbiB3aXRoIFBWSCBk
b20wIGFuZCBodm0gZG9tVSwgaHZtIHdpbGwgbWFwIGEgcGlycSBmb3INCj4+IGEgcGFzc3Rocm91
Z2ggZGV2aWNlIGJ5IHVzaW5nIGdzaSwgc2VlDQo+PiB4ZW5fcHRfcmVhbGl6ZS0+eGNfcGh5c2Rl
dl9tYXBfcGlycSBhbmQNCj4+IHBjaV9hZGRfZG1fZG9uZS0+eGNfcGh5c2Rldl9tYXBfcGlycS4g
VGhlbiB4Y19waHlzZGV2X21hcF9waXJxDQo+PiB3aWxsIGNhbGwgaW50byBYZW4sIGJ1dCBpbiBo
dm1fcGh5c2Rldl9vcCwgUEhZU0RFVk9QX21hcF9waXJxDQo+PiBpcyBub3QgYWxsb3dlZCBiZWNh
dXNlIGN1cnJkIGlzIFBWSCBkb20wIGFuZCBQVkggaGFzIG5vDQo+PiBYODZfRU1VX1VTRV9QSVJR
IGZsYWcsIGl0IHdpbGwgZmFpbCBhdCBoYXNfcGlycSBjaGVjay4NCj4+DQo+PiBTbywgYWxsb3cg
UEhZU0RFVk9QX21hcF9waXJxIHdoZW4gZG9tMCBpcyBQVkggYW5kIGFsc28gYWxsb3cNCj4+IFBI
WVNERVZPUF91bm1hcF9waXJxIGZvciB0aGUgZmFpbGVkIHBhdGggdG8gdW5tYXAgcGlycS4gQW5k
DQo+PiBhZGQgYSBuZXcgY2hlY2sgdG8gcHJldmVudCBzZWxmIG1hcCB3aGVuIGNhbGxlciBoYXMg
bm8gUElSUQ0KPj4gZmxhZy4NCj4+DQo+PiBDby1kZXZlbG9wZWQtYnk6IEh1YW5nIFJ1aSA8cmF5
Lmh1YW5nQGFtZC5jb20+DQo+PiBTaWduZWQtb2ZmLWJ5OiBKaXFpYW4gQ2hlbiA8SmlxaWFuLkNo
ZW5AYW1kLmNvbT4NCj4+IC0tLQ0KPj4gIHhlbi9hcmNoL3g4Ni9odm0vaHlwZXJjYWxsLmMgfCAg
MiArKw0KPj4gIHhlbi9hcmNoL3g4Ni9waHlzZGV2LmMgICAgICAgfCAyMiArKysrKysrKysrKysr
KysrKysrKysrDQo+PiAgMiBmaWxlcyBjaGFuZ2VkLCAyNCBpbnNlcnRpb25zKCspDQo+Pg0KPj4g
ZGlmZiAtLWdpdCBhL3hlbi9hcmNoL3g4Ni9odm0vaHlwZXJjYWxsLmMgYi94ZW4vYXJjaC94ODYv
aHZtL2h5cGVyY2FsbC5jDQo+PiBpbmRleCA2YWQ1YjRkNWYxMWYuLjQ5Mzk5OGI0MmVjNSAxMDA2
NDQNCj4+IC0tLSBhL3hlbi9hcmNoL3g4Ni9odm0vaHlwZXJjYWxsLmMNCj4+ICsrKyBiL3hlbi9h
cmNoL3g4Ni9odm0vaHlwZXJjYWxsLmMNCj4+IEBAIC03NCw2ICs3NCw4IEBAIGxvbmcgaHZtX3Bo
eXNkZXZfb3AoaW50IGNtZCwgWEVOX0dVRVNUX0hBTkRMRV9QQVJBTSh2b2lkKSBhcmcpDQo+PiAg
ICAgIHsNCj4+ICAgICAgY2FzZSBQSFlTREVWT1BfbWFwX3BpcnE6DQo+PiAgICAgIGNhc2UgUEhZ
U0RFVk9QX3VubWFwX3BpcnE6DQo+PiArICAgICAgICBicmVhazsNCj4+ICsNCj4+ICAgICAgY2Fz
ZSBQSFlTREVWT1BfZW9pOg0KPj4gICAgICBjYXNlIFBIWVNERVZPUF9pcnFfc3RhdHVzX3F1ZXJ5
Og0KPj4gICAgICBjYXNlIFBIWVNERVZPUF9nZXRfZnJlZV9waXJxOg0KPj4gZGlmZiAtLWdpdCBh
L3hlbi9hcmNoL3g4Ni9waHlzZGV2LmMgYi94ZW4vYXJjaC94ODYvcGh5c2Rldi5jDQo+PiBpbmRl
eCA0N2M0ZGEwYWY3ZTEuLjdmMjQyMmMyYTQ4MyAxMDA2NDQNCj4+IC0tLSBhL3hlbi9hcmNoL3g4
Ni9waHlzZGV2LmMNCj4+ICsrKyBiL3hlbi9hcmNoL3g4Ni9waHlzZGV2LmMNCj4+IEBAIC0zMDMs
MTEgKzMwMywyMiBAQCByZXRfdCBkb19waHlzZGV2X29wKGludCBjbWQsIFhFTl9HVUVTVF9IQU5E
TEVfUEFSQU0odm9pZCkgYXJnKQ0KPj4gICAgICBjYXNlIFBIWVNERVZPUF9tYXBfcGlycTogew0K
Pj4gICAgICAgICAgcGh5c2Rldl9tYXBfcGlycV90IG1hcDsNCj4+ICAgICAgICAgIHN0cnVjdCBt
c2lfaW5mbyBtc2k7DQo+PiArICAgICAgICBzdHJ1Y3QgZG9tYWluICpkOw0KPj4gIA0KPj4gICAg
ICAgICAgcmV0ID0gLUVGQVVMVDsNCj4+ICAgICAgICAgIGlmICggY29weV9mcm9tX2d1ZXN0KCZt
YXAsIGFyZywgMSkgIT0gMCApDQo+PiAgICAgICAgICAgICAgYnJlYWs7DQo+PiAgDQo+PiArICAg
ICAgICBkID0gcmN1X2xvY2tfZG9tYWluX2J5X2FueV9pZChtYXAuZG9taWQpOw0KPj4gKyAgICAg
ICAgaWYgKCBkID09IE5VTEwgKQ0KPj4gKyAgICAgICAgICAgIHJldHVybiAtRVNSQ0g7DQo+PiAr
ICAgICAgICBpZiAoICFpc19wdl9kb21haW4oZCkgJiYgIWhhc19waXJxKGQpICkNCj4gDQo+IEkg
dGhpbmsgdGhpcyBjb3VsZCBqdXN0IGJlOg0KPiANCj4gICAgIGlmICggIWhhc19waXJxKGQpICkN
Cj4gDQo+IFJpZ2h0Pw0KTm8uIEluIHRoZSBiZWdpbm5pbmcsIEkgb25seSBzZXQgdGhpcyBjb25k
aXRpb24gaGVyZSwgYnV0IGl0IGRlc3Ryb3llZCBQViBkb20wLg0KQmVjYXVzZSAgUFYgaGFzIG5v
IHBpcnEgZmxhZyB0b28sIGl0IGNhbiBtYXRjaCB0aGlzIGNvbmRpdGlvbiBhbmQgcmV0dXJuIC1F
T1BOT1RTVVBQLCBQSFlTREVWT1BfbWFwX3BpcnEgd2lsbCBmYWlsLg0KDQo+IA0KPiANCj4+ICsg
ICAgICAgIHsNCj4+ICsgICAgICAgICAgICByY3VfdW5sb2NrX2RvbWFpbihkKTsNCj4+ICsgICAg
ICAgICAgICByZXR1cm4gLUVPUE5PVFNVUFA7DQo+PiArICAgICAgICB9DQo+PiArICAgICAgICBy
Y3VfdW5sb2NrX2RvbWFpbihkKTsNCj4+ICsNCj4+ICAgICAgICAgIHN3aXRjaCAoIG1hcC50eXBl
ICkNCj4+ICAgICAgICAgIHsNCj4+ICAgICAgICAgIGNhc2UgTUFQX1BJUlFfVFlQRV9NU0lfU0VH
Og0KPj4gQEAgLTM0MSwxMSArMzUyLDIyIEBAIHJldF90IGRvX3BoeXNkZXZfb3AoaW50IGNtZCwg
WEVOX0dVRVNUX0hBTkRMRV9QQVJBTSh2b2lkKSBhcmcpDQo+PiAgDQo+PiAgICAgIGNhc2UgUEhZ
U0RFVk9QX3VubWFwX3BpcnE6IHsNCj4+ICAgICAgICAgIHN0cnVjdCBwaHlzZGV2X3VubWFwX3Bp
cnEgdW5tYXA7DQo+PiArICAgICAgICBzdHJ1Y3QgZG9tYWluICpkOw0KPj4gIA0KPj4gICAgICAg
ICAgcmV0ID0gLUVGQVVMVDsNCj4+ICAgICAgICAgIGlmICggY29weV9mcm9tX2d1ZXN0KCZ1bm1h
cCwgYXJnLCAxKSAhPSAwICkNCj4+ICAgICAgICAgICAgICBicmVhazsNCj4+ICANCj4+ICsgICAg
ICAgIGQgPSByY3VfbG9ja19kb21haW5fYnlfYW55X2lkKHVubWFwLmRvbWlkKTsNCj4+ICsgICAg
ICAgIGlmICggZCA9PSBOVUxMICkNCj4+ICsgICAgICAgICAgICByZXR1cm4gLUVTUkNIOw0KPj4g
KyAgICAgICAgaWYgKCAhaXNfcHZfZG9tYWluKGQpICYmICFoYXNfcGlycShkKSApDQo+IA0KPiBz
YW1lIGhlcmUNCj4gDQo+IA0KPiBPdGhlciB0aGFuIHRoYXQsIGV2ZXJ5dGhpbmcgbG9va3MgZmlu
ZSB0byBtZQ0KPiANCj4gDQo+PiArICAgICAgICB7DQo+PiArICAgICAgICAgICAgcmN1X3VubG9j
a19kb21haW4oZCk7DQo+PiArICAgICAgICAgICAgcmV0dXJuIC1FT1BOT1RTVVBQOw0KPj4gKyAg
ICAgICAgfQ0KPj4gKyAgICAgICAgcmN1X3VubG9ja19kb21haW4oZCk7DQo+PiArDQo+PiAgICAg
ICAgICByZXQgPSBwaHlzZGV2X3VubWFwX3BpcnEodW5tYXAuZG9taWQsIHVubWFwLnBpcnEpOw0K
Pj4gICAgICAgICAgYnJlYWs7DQo+PiAgICAgIH0NCj4+IC0tIA0KPj4gMi4zNC4xDQo+Pg0KDQot
LSANCkJlc3QgcmVnYXJkcywNCkppcWlhbiBDaGVuLg0K


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 06:08:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 06:08:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684634.1064621 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdOjL-000136-Gr; Fri, 23 Feb 2024 06:08:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684634.1064621; Fri, 23 Feb 2024 06:08:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdOjL-00012z-EC; Fri, 23 Feb 2024 06:08:19 +0000
Received: by outflank-mailman (input) for mailman id 684634;
 Fri, 23 Feb 2024 06:08:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SdR/=KA=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1rdOjJ-00012t-JH
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 06:08:17 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20600.outbound.protection.outlook.com
 [2a01:111:f403:2009::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id edcb6cb7-d211-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 07:08:15 +0100 (CET)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by SJ2PR12MB9161.namprd12.prod.outlook.com (2603:10b6:a03:566::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.22; Fri, 23 Feb
 2024 06:08:08 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa%5]) with mapi id 15.20.7316.018; Fri, 23 Feb 2024
 06:08:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: edcb6cb7-d211-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=oY59rj+7x5HW36sWPw/J3xYxeYQloT0GzS84344OmSVws9YG8vKeudJGnrc6E6BwB7l/5xGN0hShNqSVRKSV33Q3pIqZ4nSmQf0CProBlgMC2ZyECiTWVfSazWBC/jw9ntgnZbsS6u+sAWo1htpGknFiMAULBnnEWRXayGdk+YBGqV3G6qC+axHNxS8XIATwjUYTVe9gKX5YwIKMho56Ny9oOyQ77lIWZDLc3isL1wb2k70IXQ1WexQ4eJIpcO6tmONBEHn8WG6Pj6wljbZbu3a/sMi+A8/rDx/gWmC6t6D++UnLQ8hM3M5weZumzKgqjXZt1nqoOI/lHckr+HvoMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=cSFXgVI8MFKZIpnZrwxrAEdKHoKZxwuw+/OdAizMT1s=;
 b=aXixU3jO30f2RWfZCIn15FNHmqOia2J0mOtlMRyXIGXPw8ZISnp249Qiy7zcEjwQE0IFHKWLbJWWm8gB9ize5uojNowvBcenLvAyY5zN6qv3q9DsOs5+sTAwfEWW/NRy0x4wuF2oCzcQIVXDeFFcFUOofMU6rsvrUv13D/t/s4e1uV8orqBSFaYA9Urn6KE6riS2BUCKvFvKJ5KeXwzRgmrcunY+ys9aRavmD9tWVXmlD6TzQKH9BeFSKmMsdmnOcl4aZ3YX2/2376g650yXPudc4Djoxn+jB9eXUJafpTFoZ10hfxRkgf8w2I3GasJk01G+BDPo9R3lwxosty2dFw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=cSFXgVI8MFKZIpnZrwxrAEdKHoKZxwuw+/OdAizMT1s=;
 b=wS1vEOBZNMzUX7EHxoJsFmtS0OfzlVAZv7vPj+i10LH08B+4+7kIqrgtdpp7K0NOVI9zcApp2y2+iqD7Ja0QjUUPmz1d7TsDtncx4pI3dP4JvODI8s6y6ODb0JorhzNZkU7a7UOM4JxvHMhSDOWmjXZQBRKz1Ou1NXi+gEyqi4A=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Juergen Gross <jgross@suse.com>, Oleksandr Tyshchenko
	<oleksandr_tyshchenko@epam.com>, Boris Ostrovsky
	<boris.ostrovsky@oracle.com>, Bjorn Helgaas <bhelgaas@google.com>, "Rafael J
 . Wysocki" <rafael@kernel.org>, Len Brown <lenb@kernel.org>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>, "Hildebrand,
 Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray" <Ray.Huang@amd.com>,
	"Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC KERNEL PATCH v4 2/3] xen/pvh: Setup gsi for passthrough
 device
Thread-Topic: [RFC KERNEL PATCH v4 2/3] xen/pvh: Setup gsi for passthrough
 device
Thread-Index: AQHaP5+cDtnYXqI88Ey+AFMsAvIGlrEXXeWAgADmGoA=
Date: Fri, 23 Feb 2024 06:08:08 +0000
Message-ID:
 <BL1PR12MB58492E9C5247A711D3C26AFAE7552@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240105062217.349645-1-Jiqian.Chen@amd.com>
 <20240105062217.349645-3-Jiqian.Chen@amd.com>
 <alpine.DEB.2.22.394.2402221622000.754277@ubuntu-linux-20-04-desktop>
In-Reply-To:
 <alpine.DEB.2.22.394.2402221622000.754277@ubuntu-linux-20-04-desktop>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: MN0PR12MB6150.namprd12.prod.outlook.com
 (15.20.7292.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|SJ2PR12MB9161:EE_
x-ms-office365-filtering-correlation-id: 47e91f7c-7d66-45e2-cb69-08dc3435ceaa
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 ETwzAr0X4aGYjLqDF8XV8DW6480QDADpDxXXGrfwzpKexkDGCsBW5MDMCbTnFFdUYuTKBcJfO1IBtVrum14nAiGscOTNpPz1QRqugL/3967VQbyly2MPJF2+as0LGCu5ljByL648B3CTuDjDi6FEFI4UZYX/avNsxNc1KL8XMqrRW5PfcLSPkO/eGQnjScSHodBGxdq66G0BuDObbYxakbx0o4xGOJTmQ3IUctTdrdzGSO8V32TJRMLMaTJS6cs6Bj6Q9yVHIeypV+RqZlmTUDqWfob/24u6W119+CSXNC5nhflMaENRc6cTdEWycD1QurWSstTFkHzNnldZHigSF1AF9QN+5RhZFay7hPc8NFqiOdqQPJceZDTgZYnGsG8dvpiQGZuuPaiNRN4olNUDs6tXpE+TbpGhDVHzlUsxoyz57TYm7zCQWkdUeCSWcvGjXEhCapY9aG4Sjh5YaclPSIasV4ClImFOG9MXZVbIcPEL22tzvD6sqXGq+TaZR5btJgeTd34T7G7eZo1trvZJTStjLtaN3Rxh+7kPtw4Pd538+ru8od/FXjCUE47J4O0RgAlQKs/XLjjl7odRAG3VeuV4BSERauhxbpM0T20BOM7kTzsS6hns+9jlTziNY4oc
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?NytmeTBhS0FYc3FIQlNseTlUdHVXRGhYbUtWZVplVEk4dy83RlZaZHpRakNT?=
 =?utf-8?B?OC9rRGwyZlVJSkhlL3RPd2JERlRjMGkwVzV0NjVGOXM3dTJYWGFYdGZkV1Jq?=
 =?utf-8?B?enRDKzBRNGFFYVdQUUpaS0JaeEtNNkdJeHpYQ0RwTDZMaWtQNTRtb3BaUURH?=
 =?utf-8?B?ZzZjR1VyNEFwK0xsZlhGUVN5UEpyK295a3o0Z3NpK2oxSlBWd25kMk9KVk5Y?=
 =?utf-8?B?SnhQY1Y4bkp3aXR3dUNxVSsxVGhESHVyZEd1RzBZQjNTcDUxZTQ3a2VNb1JQ?=
 =?utf-8?B?TFRFb3ZQVEd3RGkxZFV0akRBdFVRRXplblZkd09RMzRxS0dSMTAxYjdjTjgz?=
 =?utf-8?B?bjdKazk3VW83Z0hvSEUyTTVSYW45azh2aHRWWHMyeWpEYkZQWmxrZlpDTWxC?=
 =?utf-8?B?Ukdmd3Y1Y2k5N04wNDNuMUxKSlRxZDlGMnpHanArOGszTlNiM1ZBbXQvNm5r?=
 =?utf-8?B?eUVuR1J3ZWVlYjduZVQxalZ0d2ZSM2R5d1NUa1ZaTDdRQWJBcy84K2hndms1?=
 =?utf-8?B?QlVFZG5ib1dibFlHTURmbTBZR3pMbitoVTBSMkFlVzVmR0hHQzhsSTBFWjhF?=
 =?utf-8?B?aEVVaXdaaXRZS0NWZ0VKVEZzdlJFWUhPdllVb1M1elJFK2RWMUtTYXlCWndF?=
 =?utf-8?B?eElHOEFxV0o3ODc4aWtEVVhtNzRLODNPZi9tRHFnUWhrczZsOFdCMHRkaEpT?=
 =?utf-8?B?d3Nhb2xJV2NGMmlZb3JsOWc3M0IxSXZQNS9LbEJ3Z0JzZVAxQS80YWJMSUl4?=
 =?utf-8?B?WkJSUGpMdUJYdGk4TUlrZERoaHZMWXhFNy9CeW1ORTU0VDVQTk5aZDNycjZi?=
 =?utf-8?B?dWtNbDZHbmRDYkNnTWhLNkFBZ25La200Qk5FQkpGbC8zSVJnTGx5Q2lmZmVX?=
 =?utf-8?B?Yk0xM0NISFhpNmM4RnNLWnFFTDRBQUxsNXNBSjBFV3hTSElkQ1hGcE5scGxB?=
 =?utf-8?B?U0xCQWV4c2NrWm9qZXU2bmF6SFQzajFMNUxFVVdBS0RlbHgvbVZnYXhUNENI?=
 =?utf-8?B?K1RacnFHWDRHaVNBd2txeUdFK2JRdUlUWE1jM25yMVhLUVN2cUdoNlJ0REFj?=
 =?utf-8?B?eUYzVXdvOEsxM0d4Y0FnK1B6YnNzL25wcGh0bTJFY04xNThvbVNzT3I4T2Qw?=
 =?utf-8?B?ZG54RllhSjQ2N2JqSk0xOEorcUdoeENySzFzZFhibk5FZDdLT2x0VE9uSjdL?=
 =?utf-8?B?UEFUdFc2OEJnR3V1VEhSS2pFS2lUd0kzdUI1bGJwYTNVd2UrOEhDMllzb0tD?=
 =?utf-8?B?N1V2L05xQ1RJSUg1WEhwd2J1dkwxSW1xdW5hNlZlMUZLM1VrT2lZaWlKVmxp?=
 =?utf-8?B?ZG9HNDRlbkJtM0FXb05WbFRuTXBQbnVub1BtaTRwRHVpdk1iRE0vWFZvK1lz?=
 =?utf-8?B?cG9xMmFEcjUrQmdacWk2Z2czNmJ4MzhlYlNnZm1sZ3FQeE9zcmVoU0FLNWhL?=
 =?utf-8?B?TEpHTWExT0hPeU1Zd2pXUGFjRER4REo1VlZpZDdKaVo3c1NZcTdNbG4wajRN?=
 =?utf-8?B?dlJoY2xhTmlFVERWenQrdTJ5eG1CaGpuM2Z2TTBzdFBqejZZVS9rNi9vSG9j?=
 =?utf-8?B?SG8yK0tubWxFdVNmZnNoRnd5cytHTEtiNXcyTHF0MGhMcEVKWEtVZUlNTXZN?=
 =?utf-8?B?emlUSVpaQ1IxVkhSdUNHcWM3SW9vTmJnTFJRSjlMU29KN0xiSzdZUWVGNHpa?=
 =?utf-8?B?TzIyWGpSNm56ZnJGZnZGbHAyaHdWVk85b2U0VEwwRVl2Qm1hWWZIRWpTQzdz?=
 =?utf-8?B?MVEvUTlISGVvWmVwejZNTnNReHFQaVQ5UXcreW95RzRxcEM1Y2swM0pZYUtn?=
 =?utf-8?B?c01BckdHb28rYnVtdVJxcGRNL2dNajVwZUtrNGVLRzJTRXBHdEZaR1BwNUZy?=
 =?utf-8?B?TUVwWEI3SmRwVXZBVGYxTHJzVjM5b0VBeWg3QjBRcGtSM2ZMWVNxVkxvTUcr?=
 =?utf-8?B?ZTVxUDdQcytieEJacVRGU1pkcWlYSzgxWE8wM1ZEWVEzQkswQkxNSEg5VVR6?=
 =?utf-8?B?SVdadE5PaUNmS3NPY0RUVzJTd0p4QWIvQmNmQXBQWHBtc3ZBbWRZbCt1V01R?=
 =?utf-8?B?MmlRTStLL3lEbFc2cnZEN3BQVTQyR2NKeUlkeW1qMEI5THdpaFQyMG1UZXda?=
 =?utf-8?Q?UjGQ=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <10FED9D7D2B42544B6FEAC7926674405@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 47e91f7c-7d66-45e2-cb69-08dc3435ceaa
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Feb 2024 06:08:08.4794
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: aJ004It+mnhP++1uIsPAcgwIRrsM3NR0yl0px3q0vvSGydPduBWFb1FJJdp8MUkmjtTK+u9foZQ60TNuMpjx+A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB9161

T24gMjAyNC8yLzIzIDA4OjIzLCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+IE9uIEZyaSwg
NSBKYW4gMjAyNCwgSmlxaWFuIENoZW4gd3JvdGU6DQo+PiBJbiBQVkggZG9tMCwgdGhlIGdzaXMg
ZG9uJ3QgZ2V0IHJlZ2lzdGVyZWQsIGJ1dCB0aGUgZ3NpIG9mDQo+PiBhIHBhc3N0aHJvdWdoIGRl
dmljZSBtdXN0IGJlIGNvbmZpZ3VyZWQgZm9yIGl0IHRvIGJlIGFibGUgdG8gYmUNCj4+IG1hcHBl
ZCBpbnRvIGEgZG9tVS4NCj4+DQo+PiBXaGVuIGFzc2lnbiBhIGRldmljZSB0byBwYXNzdGhyb3Vn
aCwgcHJvYWN0aXZlbHkgc2V0dXAgdGhlIGdzaQ0KPj4gb2YgdGhlIGRldmljZSBkdXJpbmcgdGhh
dCBwcm9jZXNzLg0KPj4NCj4+IENvLWRldmVsb3BlZC1ieTogSHVhbmcgUnVpIDxyYXkuaHVhbmdA
YW1kLmNvbT4NCj4+IFNpZ25lZC1vZmYtYnk6IEppcWlhbiBDaGVuIDxKaXFpYW4uQ2hlbkBhbWQu
Y29tPg0KPj4gLS0tDQo+PiAgYXJjaC94ODYveGVuL2VubGlnaHRlbl9wdmguYyAgICAgICB8IDkw
ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKw0KPj4gIGRyaXZlcnMvYWNwaS9wY2lfaXJx
LmMgICAgICAgICAgICAgfCAgMiArLQ0KPj4gIGRyaXZlcnMveGVuL3hlbi1wY2liYWNrL3BjaV9z
dHViLmMgfCAgOCArKysNCj4+ICBpbmNsdWRlL2xpbnV4L2FjcGkuaCAgICAgICAgICAgICAgIHwg
IDEgKw0KPj4gIGluY2x1ZGUveGVuL2FjcGkuaCAgICAgICAgICAgICAgICAgfCAgNiArKw0KPj4g
IDUgZmlsZXMgY2hhbmdlZCwgMTA2IGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkNCj4+DQo+
PiBkaWZmIC0tZ2l0IGEvYXJjaC94ODYveGVuL2VubGlnaHRlbl9wdmguYyBiL2FyY2gveDg2L3hl
bi9lbmxpZ2h0ZW5fcHZoLmMNCj4+IGluZGV4IGFkYTM4NjhjMDJjMi4uZWNhZGQ5NjZjNjg0IDEw
MDY0NA0KPj4gLS0tIGEvYXJjaC94ODYveGVuL2VubGlnaHRlbl9wdmguYw0KPj4gKysrIGIvYXJj
aC94ODYveGVuL2VubGlnaHRlbl9wdmguYw0KPj4gQEAgLTEsNiArMSw3IEBADQo+PiAgLy8gU1BE
WC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjANCj4+ICAjaW5jbHVkZSA8bGludXgvYWNwaS5o
Pg0KPj4gICNpbmNsdWRlIDxsaW51eC9leHBvcnQuaD4NCj4+ICsjaW5jbHVkZSA8bGludXgvcGNp
Lmg+DQo+PiAgDQo+PiAgI2luY2x1ZGUgPHhlbi9odmMtY29uc29sZS5oPg0KPj4gIA0KPj4gQEAg
LTI1LDYgKzI2LDk1IEBADQo+PiAgYm9vbCBfX3JvX2FmdGVyX2luaXQgeGVuX3B2aDsNCj4+ICBF
WFBPUlRfU1lNQk9MX0dQTCh4ZW5fcHZoKTsNCj4+ICANCj4+ICt0eXBlZGVmIHN0cnVjdCBnc2lf
aW5mbyB7DQo+PiArCWludCBnc2k7DQo+PiArCWludCB0cmlnZ2VyOw0KPj4gKwlpbnQgcG9sYXJp
dHk7DQo+PiArfSBnc2lfaW5mb190Ow0KPj4gKw0KPj4gK3N0cnVjdCBhY3BpX3BydF9lbnRyeSB7
DQo+PiArCXN0cnVjdCBhY3BpX3BjaV9pZAlpZDsNCj4+ICsJdTgJCQlwaW47DQo+PiArCWFjcGlf
aGFuZGxlCQlsaW5rOw0KPj4gKwl1MzIJCQlpbmRleDsJCS8qIEdTSSwgb3IgbGluayBfQ1JTIGlu
ZGV4ICovDQo+PiArfTsNCj4+ICsNCj4+ICtzdGF0aWMgaW50IHhlbl9wdmhfZ2V0X2dzaV9pbmZv
KHN0cnVjdCBwY2lfZGV2ICpkZXYsDQo+PiArCQkJCQkJCQlnc2lfaW5mb190ICpnc2lfaW5mbykN
Cj4+ICt7DQo+PiArCWludCBnc2k7DQo+PiArCXU4IHBpbiA9IDA7DQo+PiArCXN0cnVjdCBhY3Bp
X3BydF9lbnRyeSAqZW50cnk7DQo+PiArCWludCB0cmlnZ2VyID0gQUNQSV9MRVZFTF9TRU5TSVRJ
VkU7DQo+PiArCWludCBwb2xhcml0eSA9IGFjcGlfaXJxX21vZGVsID09IEFDUElfSVJRX01PREVM
X0dJQyA/DQo+PiArCQkJCSAgICAgIEFDUElfQUNUSVZFX0hJR0ggOiBBQ1BJX0FDVElWRV9MT1c7
DQo+PiArDQo+PiArCWlmIChkZXYpDQo+PiArCQlwaW4gPSBkZXYtPnBpbjsNCj4gDQo+IFRoaXMg
aXMgbWlub3IsIGJ1dCB5b3UgY2FuIGp1c3QgbW92ZSB0aGUgcGluID0gZGV2LT5waW4gYWZ0ZXIg
dGhlICFkZXYNCj4gY2hlY2sgYmVsb3cuDQpXaWxsIGNoYW5nZSBpbiBuZXh0IHZlcnNpb24uDQoN
Cj4gDQo+IFdpdGggdGhhdCBjaGFuZ2UsIGFuZCBhc3N1bWluZyB0aGUgWGVuLXNpZGUgYW5kIFFF
TVUtc2lkZSBwYXRjaGVzIGFyZQ0KPiBhY2NlcHRlZDoNCj4gDQo+IFJldmlld2VkLWJ5OiBTdGVm
YW5vIFN0YWJlbGxpbmkgPHNzdGFiZWxsaW5pQGtlcm5lbC5vcmc+DQpUaGFuayB5b3UgdmVyeSBt
dWNoIQ0KDQo+IA0KPiANCj4gDQo+IA0KPj4gKwlpZiAoIWRldiB8fCAhcGluIHx8ICFnc2lfaW5m
bykNCj4+ICsJCXJldHVybiAtRUlOVkFMOw0KPj4gKw0KPj4gKwllbnRyeSA9IGFjcGlfcGNpX2ly
cV9sb29rdXAoZGV2LCBwaW4pOw0KPj4gKwlpZiAoZW50cnkpIHsNCj4+ICsJCWlmIChlbnRyeS0+
bGluaykNCj4+ICsJCQlnc2kgPSBhY3BpX3BjaV9saW5rX2FsbG9jYXRlX2lycShlbnRyeS0+bGlu
aywNCj4+ICsJCQkJCQkJIGVudHJ5LT5pbmRleCwNCj4+ICsJCQkJCQkJICZ0cmlnZ2VyLCAmcG9s
YXJpdHksDQo+PiArCQkJCQkJCSBOVUxMKTsNCj4+ICsJCWVsc2UNCj4+ICsJCQlnc2kgPSBlbnRy
eS0+aW5kZXg7DQo+PiArCX0gZWxzZQ0KPj4gKwkJZ3NpID0gLTE7DQo+PiArDQo+PiArCWlmIChn
c2kgPCAwKQ0KPj4gKwkJcmV0dXJuIC1FSU5WQUw7DQo+PiArDQo+PiArCWdzaV9pbmZvLT5nc2kg
PSBnc2k7DQo+PiArCWdzaV9pbmZvLT50cmlnZ2VyID0gdHJpZ2dlcjsNCj4+ICsJZ3NpX2luZm8t
PnBvbGFyaXR5ID0gcG9sYXJpdHk7DQo+PiArDQo+PiArCXJldHVybiAwOw0KPj4gK30NCj4+ICsN
Cj4+ICtzdGF0aWMgaW50IHhlbl9wdmhfc2V0dXBfZ3NpKGdzaV9pbmZvX3QgKmdzaV9pbmZvKQ0K
Pj4gK3sNCj4+ICsJc3RydWN0IHBoeXNkZXZfc2V0dXBfZ3NpIHNldHVwX2dzaTsNCj4+ICsNCj4+
ICsJaWYgKCFnc2lfaW5mbykNCj4+ICsJCXJldHVybiAtRUlOVkFMOw0KPj4gKw0KPj4gKwlzZXR1
cF9nc2kuZ3NpID0gZ3NpX2luZm8tPmdzaTsNCj4+ICsJc2V0dXBfZ3NpLnRyaWdnZXJpbmcgPSAo
Z3NpX2luZm8tPnRyaWdnZXIgPT0gQUNQSV9FREdFX1NFTlNJVElWRSA/IDAgOiAxKTsNCj4+ICsJ
c2V0dXBfZ3NpLnBvbGFyaXR5ID0gKGdzaV9pbmZvLT5wb2xhcml0eSA9PSBBQ1BJX0FDVElWRV9I
SUdIID8gMCA6IDEpOw0KPj4gKw0KPj4gKwlyZXR1cm4gSFlQRVJWSVNPUl9waHlzZGV2X29wKFBI
WVNERVZPUF9zZXR1cF9nc2ksICZzZXR1cF9nc2kpOw0KPj4gK30NCj4+ICsNCj4+ICtpbnQgeGVu
X3B2aF9wYXNzdGhyb3VnaF9nc2koc3RydWN0IHBjaV9kZXYgKmRldikNCj4+ICt7DQo+PiArCWlu
dCByZXQ7DQo+PiArCWdzaV9pbmZvX3QgZ3NpX2luZm87DQo+PiArDQo+PiArCWlmICghZGV2KQ0K
Pj4gKwkJcmV0dXJuIC1FSU5WQUw7DQo+PiArDQo+PiArCXJldCA9IHhlbl9wdmhfZ2V0X2dzaV9p
bmZvKGRldiwgJmdzaV9pbmZvKTsNCj4+ICsJaWYgKHJldCkgew0KPj4gKwkJeGVuX3Jhd19wcmlu
dGsoIkZhaWwgdG8gZ2V0IGdzaSBpbmZvIVxuIik7DQo+PiArCQlyZXR1cm4gcmV0Ow0KPj4gKwl9
DQo+PiArDQo+PiArCXJldCA9IHhlbl9wdmhfc2V0dXBfZ3NpKCZnc2lfaW5mbyk7DQo+PiArCWlm
IChyZXQgPT0gLUVFWElTVCkgew0KPj4gKwkJeGVuX3Jhd19wcmludGsoIkFscmVhZHkgc2V0dXAg
dGhlIEdTSSA6JWRcbiIsIGdzaV9pbmZvLmdzaSk7DQo+PiArCQlyZXQgPSAwOw0KPj4gKwl9IGVs
c2UgaWYgKHJldCkNCj4+ICsJCXhlbl9yYXdfcHJpbnRrKCJGYWlsIHRvIHNldHVwIEdTSSAoJWQp
IVxuIiwgZ3NpX2luZm8uZ3NpKTsNCj4+ICsNCj4+ICsJcmV0dXJuIHJldDsNCj4+ICt9DQo+PiAr
RVhQT1JUX1NZTUJPTF9HUEwoeGVuX3B2aF9wYXNzdGhyb3VnaF9nc2kpOw0KPj4gKw0KPj4gIHZv
aWQgX19pbml0IHhlbl9wdmhfaW5pdChzdHJ1Y3QgYm9vdF9wYXJhbXMgKmJvb3RfcGFyYW1zKQ0K
Pj4gIHsNCj4+ICAJdTMyIG1zcjsNCj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2FjcGkvcGNpX2ly
cS5jIGIvZHJpdmVycy9hY3BpL3BjaV9pcnEuYw0KPj4gaW5kZXggZmYzMGNlY2EyMjAzLi42MzBm
ZTBhMzRiYzYgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL2FjcGkvcGNpX2lycS5jDQo+PiArKysg
Yi9kcml2ZXJzL2FjcGkvcGNpX2lycS5jDQo+PiBAQCAtMjg4LDcgKzI4OCw3IEBAIHN0YXRpYyBp
bnQgYWNwaV9yZXJvdXRlX2Jvb3RfaW50ZXJydXB0KHN0cnVjdCBwY2lfZGV2ICpkZXYsDQo+PiAg
fQ0KPj4gICNlbmRpZiAvKiBDT05GSUdfWDg2X0lPX0FQSUMgKi8NCj4+ICANCj4+IC1zdGF0aWMg
c3RydWN0IGFjcGlfcHJ0X2VudHJ5ICphY3BpX3BjaV9pcnFfbG9va3VwKHN0cnVjdCBwY2lfZGV2
ICpkZXYsIGludCBwaW4pDQo+PiArc3RydWN0IGFjcGlfcHJ0X2VudHJ5ICphY3BpX3BjaV9pcnFf
bG9va3VwKHN0cnVjdCBwY2lfZGV2ICpkZXYsIGludCBwaW4pDQo+PiAgew0KPj4gIAlzdHJ1Y3Qg
YWNwaV9wcnRfZW50cnkgKmVudHJ5ID0gTlVMTDsNCj4+ICAJc3RydWN0IHBjaV9kZXYgKmJyaWRn
ZTsNCj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2lfc3R1Yi5jIGIv
ZHJpdmVycy94ZW4veGVuLXBjaWJhY2svcGNpX3N0dWIuYw0KPj4gaW5kZXggNDZjNDBlYzhhMThl
Li4yMmQ0MzgwZDJiMDQgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9w
Y2lfc3R1Yi5jDQo+PiArKysgYi9kcml2ZXJzL3hlbi94ZW4tcGNpYmFjay9wY2lfc3R1Yi5jDQo+
PiBAQCAtMjAsNiArMjAsNyBAQA0KPj4gICNpbmNsdWRlIDxsaW51eC9hdG9taWMuaD4NCj4+ICAj
aW5jbHVkZSA8eGVuL2V2ZW50cy5oPg0KPj4gICNpbmNsdWRlIDx4ZW4vcGNpLmg+DQo+PiArI2lu
Y2x1ZGUgPHhlbi9hY3BpLmg+DQo+PiAgI2luY2x1ZGUgPHhlbi94ZW4uaD4NCj4+ICAjaW5jbHVk
ZSA8YXNtL3hlbi9oeXBlcnZpc29yLmg+DQo+PiAgI2luY2x1ZGUgPHhlbi9pbnRlcmZhY2UvcGh5
c2Rldi5oPg0KPj4gQEAgLTQzNSw2ICs0MzYsMTMgQEAgc3RhdGljIGludCBwY2lzdHViX2luaXRf
ZGV2aWNlKHN0cnVjdCBwY2lfZGV2ICpkZXYpDQo+PiAgCQkJZ290byBjb25maWdfcmVsZWFzZTsN
Cj4+ICAJCXBjaV9yZXN0b3JlX3N0YXRlKGRldik7DQo+PiAgCX0NCj4+ICsNCj4+ICsJaWYgKHhl
bl9pbml0aWFsX2RvbWFpbigpICYmIHhlbl9wdmhfZG9tYWluKCkpIHsNCj4+ICsJCWVyciA9IHhl
bl9wdmhfcGFzc3Rocm91Z2hfZ3NpKGRldik7DQo+PiArCQlpZiAoZXJyKQ0KPj4gKwkJCWdvdG8g
Y29uZmlnX3JlbGVhc2U7DQo+PiArCX0NCj4+ICsNCj4+ICAJLyogTm93IGRpc2FibGUgdGhlIGRl
dmljZSAodGhpcyBhbHNvIGVuc3VyZXMgc29tZSBwcml2YXRlIGRldmljZQ0KPj4gIAkgKiBkYXRh
IGlzIHNldHVwIGJlZm9yZSB3ZSBleHBvcnQpDQo+PiAgCSAqLw0KPj4gZGlmZiAtLWdpdCBhL2lu
Y2x1ZGUvbGludXgvYWNwaS5oIGIvaW5jbHVkZS9saW51eC9hY3BpLmgNCj4+IGluZGV4IDRkYjU0
ZTkyOGIzNi4uN2VhM2JlOTgxY2MzIDEwMDY0NA0KPj4gLS0tIGEvaW5jbHVkZS9saW51eC9hY3Bp
LmgNCj4+ICsrKyBiL2luY2x1ZGUvbGludXgvYWNwaS5oDQo+PiBAQCAtMzYwLDYgKzM2MCw3IEBA
IHZvaWQgYWNwaV91bnJlZ2lzdGVyX2dzaSAodTMyIGdzaSk7DQo+PiAgDQo+PiAgc3RydWN0IHBj
aV9kZXY7DQo+PiAgDQo+PiArc3RydWN0IGFjcGlfcHJ0X2VudHJ5ICphY3BpX3BjaV9pcnFfbG9v
a3VwKHN0cnVjdCBwY2lfZGV2ICpkZXYsIGludCBwaW4pOw0KPj4gIGludCBhY3BpX3BjaV9pcnFf
ZW5hYmxlIChzdHJ1Y3QgcGNpX2RldiAqZGV2KTsNCj4+ICB2b2lkIGFjcGlfcGVuYWxpemVfaXNh
X2lycShpbnQgaXJxLCBpbnQgYWN0aXZlKTsNCj4+ICBib29sIGFjcGlfaXNhX2lycV9hdmFpbGFi
bGUoaW50IGlycSk7DQo+PiBkaWZmIC0tZ2l0IGEvaW5jbHVkZS94ZW4vYWNwaS5oIGIvaW5jbHVk
ZS94ZW4vYWNwaS5oDQo+PiBpbmRleCBiMWUxMTg2MzE0NGQuLjE3YzRkMzdmMWU2MCAxMDA2NDQN
Cj4+IC0tLSBhL2luY2x1ZGUveGVuL2FjcGkuaA0KPj4gKysrIGIvaW5jbHVkZS94ZW4vYWNwaS5o
DQo+PiBAQCAtNjcsMTAgKzY3LDE2IEBAIHN0YXRpYyBpbmxpbmUgdm9pZCB4ZW5fYWNwaV9zbGVl
cF9yZWdpc3Rlcih2b2lkKQ0KPj4gIAkJYWNwaV9zdXNwZW5kX2xvd2xldmVsID0geGVuX2FjcGlf
c3VzcGVuZF9sb3dsZXZlbDsNCj4+ICAJfQ0KPj4gIH0NCj4+ICtpbnQgeGVuX3B2aF9wYXNzdGhy
b3VnaF9nc2koc3RydWN0IHBjaV9kZXYgKmRldik7DQo+PiAgI2Vsc2UNCj4+ICBzdGF0aWMgaW5s
aW5lIHZvaWQgeGVuX2FjcGlfc2xlZXBfcmVnaXN0ZXIodm9pZCkNCj4+ICB7DQo+PiAgfQ0KPj4g
Kw0KPj4gK3N0YXRpYyBpbmxpbmUgaW50IHhlbl9wdmhfcGFzc3Rocm91Z2hfZ3NpKHN0cnVjdCBw
Y2lfZGV2ICpkZXYpDQo+PiArew0KPj4gKwlyZXR1cm4gLTE7DQo+PiArfQ0KPj4gICNlbmRpZg0K
Pj4gIA0KPj4gICNlbmRpZgkvKiBfWEVOX0FDUElfSCAqLw0KPj4gLS0gDQo+PiAyLjM0LjENCj4+
DQoNCi0tIA0KQmVzdCByZWdhcmRzLA0KSmlxaWFuIENoZW4uDQo=


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 06:36:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 06:36:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684641.1064632 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdPA8-0004xw-OA; Fri, 23 Feb 2024 06:36:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684641.1064632; Fri, 23 Feb 2024 06:36:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdPA8-0004xp-Kv; Fri, 23 Feb 2024 06:36:00 +0000
Received: by outflank-mailman (input) for mailman id 684641;
 Fri, 23 Feb 2024 06:35:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqZf=KA=amd.com=Ray.Huang@srs-se1.protection.inumbo.net>)
 id 1rdPA7-0004xj-3D
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 06:35:59 +0000
Received: from NAM04-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:240a::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cb37161b-d215-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 07:35:55 +0100 (CET)
Received: from SJ2PR12MB8690.namprd12.prod.outlook.com (2603:10b6:a03:540::10)
 by BY5PR12MB4066.namprd12.prod.outlook.com (2603:10b6:a03:207::22)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.26; Fri, 23 Feb
 2024 06:35:50 +0000
Received: from SJ2PR12MB8690.namprd12.prod.outlook.com
 ([fe80::e7a:5022:4b7d:ade1]) by SJ2PR12MB8690.namprd12.prod.outlook.com
 ([fe80::e7a:5022:4b7d:ade1%6]) with mapi id 15.20.7316.023; Fri, 23 Feb 2024
 06:35:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb37161b-d215-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=iCkWhrwx0s8oFhENfvUQwM3f7GfVMPhx5KiwNiRSLHTczlRgzwGE2fRf6gGR8OIn51RfN9iTbg47LBo8cnoftJW5YaQqJXAfavGEWy24M8fUrxnRczdeppIxomN09hXqwzIxr0MhNWXCi1M6M4DneEvyICBdofGrfPq0pqvBNb53L5TRNrl+jriU85FYpz4MSV8zQZOuZOgeqbIFe/zWeXiUnawJ7oHzDX9C06q1V8I3WFIBHPENzaF7BiIF6ose0Uogsxqk8PRnEa3aV4dzA1tdAtLe/7wxyyfzfwh3OZICkujekvauOkgd4LE1wGFxZMur3UJ4uAftwsQ1oX7J6Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=MhACpc4kiVFwEdFtg6pExw1NBrA0ebSBvcu4VULNahc=;
 b=bssT6RMFhd+vVMYOxekW7wsGUdxP7Hqu34yVwm9KShlsT8jI8EltuHIsfbj8faJnOd6ZPvysdloXPqYC4AG96rDz5t5QJnpfQi00UJXpHPYvodNN6KqVvH1qfsk4RR6ByxNsYiNM+zDNReEO2JYejmAN23W5FNDl12TMbs8vBkVE750CZTEuvCUBi44oFLupEdXADZ1tUL6TZvJqrERewI26KAfHRqk/qLGmfBF3f2HPXZaQ4gicCDF64fHyRp334XvGwvfHhRf/joIHEKaScKH4mQpUbJt3VJMfPTC8l5rMGibqxKJbqDjdHQ8CLCE93cxQViYL2KkZxONJAtaHwA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=MhACpc4kiVFwEdFtg6pExw1NBrA0ebSBvcu4VULNahc=;
 b=YJfJ5NmQOGkkHVjw7ubUzGNdyWHxpNdG6uBietYTfMe9/+rQnYJV9WbW6anM2o87TyE2hGmHmTGmbBXQ3hqr+j8WE9I1XgGB41eCSIi3d9Ift1YFvxgzTtIoV/2j19ql1e9oFvgQiepGMlrYSKmN11h9gxgk9ngzhmJhsLxR8BQ=
Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
Date: Fri, 23 Feb 2024 14:34:53 +0800
From: Huang Rui <ray.huang@amd.com>
To: Pierre-Eric Pelloux-Prayer <pierre-eric@damsy.net>
Cc: Akihiko Odaki <akihiko.odaki@daynix.com>,
	=?iso-8859-1?Q?Marc-Andr=E9?= Lureau <marcandre.lureau@gmail.com>,
	Philippe =?iso-8859-1?Q?Mathieu-Daud=E9?= <philmd@linaro.org>,
	Gerd Hoffmann <kraxel@redhat.com>,
	"Michael S . Tsirkin" <mst@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Antonio Caggiano <quic_acaggian@quicinc.com>,
	"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
	Robert Beckett <bob.beckett@collabora.com>,
	Dmitry Osipenko <dmitry.osipenko@collabora.com>,
	Gert Wollny <gert.wollny@collabora.com>,
	Alex =?iso-8859-1?Q?Benn=E9e?= <alex.bennee@linaro.org>,
	"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Gurchetan Singh <gurchetansingh@chromium.org>,
	"ernunes@redhat.com" <ernunes@redhat.com>,
	Alyssa Ross <hi@alyssa.is>,
	Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
	"Deucher, Alexander" <Alexander.Deucher@amd.com>,
	"Stabellini, Stefano" <stefano.stabellini@amd.com>,
	"Koenig, Christian" <Christian.Koenig@amd.com>,
	"Ragiadakou, Xenia" <Xenia.Ragiadakou@amd.com>,
	"Pelloux-Prayer, Pierre-Eric" <Pierre-eric.Pelloux-prayer@amd.com>,
	"Huang, Honglei1" <Honglei1.Huang@amd.com>,
	"Zhang, Julia" <Julia.Zhang@amd.com>,
	"Chen, Jiqian" <Jiqian.Chen@amd.com>,
	Antonio Caggiano <antonio.caggiano@collabora.com>
Subject: Re: [PATCH v6 07/11] virtio-gpu: Handle resource blob commands
Message-ID: <Zdg8jYYI1e3+xyJi@amd.com>
References: <20231219075320.165227-1-ray.huang@amd.com>
 <20231219075320.165227-8-ray.huang@amd.com>
 <ccc34ce0-44af-425e-8634-6f7a0583ee12@damsy.net>
 <c986f552-60b8-471a-a439-a8714936fa47@damsy.net>
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <c986f552-60b8-471a-a439-a8714936fa47@damsy.net>
X-ClientProxiedBy: SI2PR04CA0007.apcprd04.prod.outlook.com
 (2603:1096:4:197::19) To SJ2PR12MB8690.namprd12.prod.outlook.com
 (2603:10b6:a03:540::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ2PR12MB8690:EE_|BY5PR12MB4066:EE_
X-MS-Office365-Filtering-Correlation-Id: ce0c9367-f83f-4423-586b-08dc3439ad03
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kect6Fcjm+yRcHAebXQQQHvGmmeDzRRYh16DT8bDiI7rf+9pBYeEyQnR1FK9zDhA+IxgdcvdOrS4iWcz7O9Bb1fkuQWUS6qORSxAo+GipgNc41jwADFM0S21/k42Z5MUGULnqA+hPEcNNWEVbwo9NWq0hVP7IyWhNXxWxPHL8HNCVDlAKFUff1Q92eTCp+Dj3HAOcJ1aanbamKGmYydn6Ap2Z+EW1vFGXg7iJOGiFovjB9BhsSSLEQegTHYSFw7gFFH2cE0TL86MUVZwMoORAe1RMttWzvSA4D+FCMRpjmQyryhwR5vYuVnMVw+L7h94mNMj+AvF6EbWCpqA9YWck+zzfV4XlWG67mGcPwQucvbRxtA/Lkoo0piajRVea/3f+26yla1qUXiKQd4WjqPWKFlDK6Lq+RH5AvXFbniJrpeHuhkIksqY/Z+nTdln+9C0rHz86ImZq4SnKQIvCGj4CI7qD3w6H5oimQCcxqqax/vh4xvl1VPlZS2MQwSgGgnmEa2TnyFNHVRFYRZoszlTLaA9thXQl5kwucqEjSYLm4k=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ2PR12MB8690.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?iso-8859-1?Q?oYPm5RAVwy/29ce+1TAKFZLEOna8JbXsEfylnSw20zVudHIEo8SebfQX3x?=
 =?iso-8859-1?Q?ZdVQ7Fw9EHiOLJDm7CBLXowJAKIjqVwdO8zUqh5pYT6H8UhVRUdMvL9y7H?=
 =?iso-8859-1?Q?GFAPDlCNf2lEm6eBTb++lIIt/Mc0tJ/Si3KoI0GbIywjZsfKQJGyjqsRem?=
 =?iso-8859-1?Q?yz8KbbPvK0scY/FSkzzOnd5AXarmaK0cVzLsnYjY+i2BQhuKYaCXI1pFkw?=
 =?iso-8859-1?Q?iVREN90xYbQFrds9pmSNwF+wgto6CoiUvr0yk722YjFs4tA1X1kmNfukN2?=
 =?iso-8859-1?Q?eR71IfdiwdRTykhMzko5jhEDvTciU2FkgLv7DMW+RnEAjX6o1txo6BNpVF?=
 =?iso-8859-1?Q?4wOTV8zMamFw4SErhqiiKz5gG6qpGdfSWo9W6f++/KUgcTL5W30WiPFW0T?=
 =?iso-8859-1?Q?E+9MZpzeglUjeAFSrRTeooY/idn8sUchE1LgZ2Ko7LE3kQvGuPATYg2/r8?=
 =?iso-8859-1?Q?KJHuPB1iXHgmfVy9lK7pyrf8LolXYrGbYz19c9BT8wX/eh0RA6ooSTjp67?=
 =?iso-8859-1?Q?Gn7dmxfiJLV01ixcy0s/wsquCyURUbTb48LQ66HixyG5thz7UtdPa7Y216?=
 =?iso-8859-1?Q?mfo0/eAm/oJJe0bXrmShBjkZC361TssFHRs2ptpk400ErQSgyanyTuT8sU?=
 =?iso-8859-1?Q?F7+wSDY5aIsa6qeYQC6L9woPCUwkW1qRJ6hiUG4OmUcHa7x+wb62O+SO6h?=
 =?iso-8859-1?Q?QZKugQM+F5VNydmaqzDNQphGJcsU31yXE3LTwu/FIMyf/9noUAaUBDTauH?=
 =?iso-8859-1?Q?661WUagOANxQdOmh+tTLNyYliz2BlVK6+MmjDjyHfMUuqwtl7gUPOz8L/3?=
 =?iso-8859-1?Q?yU70BLpa2wvF76p6Y05lWUTmnZl9p4b6i260DDs8W/3khZ1yriYZRnt46N?=
 =?iso-8859-1?Q?GJNSUKxYHBBoL8HRJSdiVWXfTLBACWJAgJDb0kaERGRUdrEJFdcOrlJUQE?=
 =?iso-8859-1?Q?C9TtJ/rrVYBVD38CB3/5DfcVJ2StIZIUJBEEbfNSAB3uTHpudNeF/Yed1v?=
 =?iso-8859-1?Q?Tt3dKnQOZNjVlMPD82N5jQKEH4SWzTD7u281Z/bvHtF6+InASiJqKZjbJO?=
 =?iso-8859-1?Q?O7/zUwLefsU/njDvHLr3oAPrudhOPN2VnE/4Iad8hL1wPNH8ryK7YwZk5j?=
 =?iso-8859-1?Q?7WT9mv9IGgQYTTdpq5le0J1pYOj8upcsekQvdWqoIcBUYQFtCamWBpHTEt?=
 =?iso-8859-1?Q?nAv8P7XNoaZBHIYB0REqD281zgs99go3znUT1wHVgR/0gH+MwfWQpvF7dl?=
 =?iso-8859-1?Q?QvM5K3+dDlT1o09qRavL3xIgCY05NtFzesG9+s6BjaMm6pNmzHa/2NQN1b?=
 =?iso-8859-1?Q?mOXppNU1oCNHFeyTLT/9epwiU9M8xBJE/plN63PSlS4CNe+DhAXPHx0pSj?=
 =?iso-8859-1?Q?GtwKwMLwnbdrhW3HvsPWvBGrQOqgyf+YViQQSq4puqEe/GCr5rL33nhDSe?=
 =?iso-8859-1?Q?FlkGZCtynNsZXPGvRnhRLUJp66LTP8OXpmSBsbKRfyZNGMDQyz24ADNJPN?=
 =?iso-8859-1?Q?klEZTfe9WpjK2QxAPgzRqqPUXgRC1izqonrCk91gVvRT23+mBVH9Kvz8kJ?=
 =?iso-8859-1?Q?GnOeNaT9f9O0f93r+N1Xs6NxWKCHwHPWgiWPRgxTLE36AyTiRV2r7nMgXG?=
 =?iso-8859-1?Q?sq/zW84mkUDJg/hfIlM8leENyWGcG0q90w?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ce0c9367-f83f-4423-586b-08dc3439ad03
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR12MB8690.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2024 06:35:50.3883
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uWlO1D35SxwrtwgiIpy08JL/O/34S56jTfjPffcRww0m5di7qnZTfzpIAxDt6F1ELzZG3gocVeJJZ10aoxpm5A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4066

On Wed, Jan 10, 2024 at 04:51:31PM +0800, Pierre-Eric Pelloux-Prayer wrote:
> 
> 
> Le 09/01/2024  17:50, Pierre-Eric Pelloux-Prayer a crit:
> > 
> > 
> > Le 19/12/2023  08:53, Huang Rui a crit:
> >> From: Antonio Caggiano <antonio.caggiano@collabora.com>
> >>
> >> Support BLOB resources creation, mapping and unmapping by calling the
> >> new stable virglrenderer 0.10 interface. Only enabled when available and
> >> via the blob config. E.g. -device virtio-vga-gl,blob=true
> >>
> >> Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
> >> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> >> Signed-off-by: Xenia Ragiadakou <xenia.ragiadakou@amd.com>
> >> Signed-off-by: Huang Rui <ray.huang@amd.com>
> >> ---
> >>
> >> Changes in v6:
> >> - Use new struct virgl_gpu_resource.
> >> - Unmap, unref and destroy the resource only after the memory region
> >>  has been completely removed.
> >> - In unref check whether the resource is still mapped.
> >> - In unmap_blob check whether the resource has been already unmapped.
> >> - Fix coding style
> >>
> >>  hw/display/virtio-gpu-virgl.c | 274 +++++++++++++++++++++++++++++++++-
> >>  hw/display/virtio-gpu.c | 4 +-
> >>  meson.build | 4 +
> >>  3 files changed, 276 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c
> >> index faab374336..5a3a292f79 100644
> >> --- a/hw/display/virtio-gpu-virgl.c
> >> +++ b/hw/display/virtio-gpu-virgl.c
> >> @@ -17,6 +17,7 @@
> >>  #include "trace.h"
> >>  #include "hw/virtio/virtio.h"
> >>  #include "hw/virtio/virtio-gpu.h"
> >> +#include "hw/virtio/virtio-gpu-bswap.h"
> >>  #include "ui/egl-helpers.h"
> >> @@ -24,8 +25,62 @@
> >>  struct virgl_gpu_resource {
> >>  struct virtio_gpu_simple_resource res;
> >> + uint32_t ref;
> >> + VirtIOGPU *g;
> >> +
> >> +#ifdef HAVE_VIRGL_RESOURCE_BLOB
> >> + /* only blob resource needs this region to be mapped as guest mmio */
> >> + MemoryRegion *region;
> >> +#endif
> >>  };
> >> +static void vres_get_ref(struct virgl_gpu_resource *vres)
> >> +{
> >> + uint32_t ref;
> >> +
> >> + ref = qatomic_fetch_inc(&vres->ref);
> >> + g_assert(ref < INT_MAX);
> >> +}
> >> +
> >> +static void virgl_resource_destroy(struct virgl_gpu_resource *vres)
> >> +{
> >> + struct virtio_gpu_simple_resource *res;
> >> + VirtIOGPU *g;
> >> +
> >> + if (!vres) {
> >> + return;
> >> + }
> >> +
> >> + g = vres->g;
> >> + res = &vres->res;
> >> + QTAILQ_REMOVE(&g->reslist, res, next);
> >> + virtio_gpu_cleanup_mapping(g, res);
> >> + g_free(vres);
> >> +}
> >> +
> >> +static void virgl_resource_unref(struct virgl_gpu_resource *vres)
> >> +{
> >> + struct virtio_gpu_simple_resource *res;
> >> +
> >> + if (!vres) {
> >> + return;
> >> + }
> >> +
> >> + res = &vres->res;
> >> + virgl_renderer_resource_detach_iov(res->resource_id, NULL, NULL);
> >> + virgl_renderer_resource_unref(res->resource_id);
> >> +}
> >> +
> >> +static void vres_put_ref(struct virgl_gpu_resource *vres)
> >> +{
> >> + g_assert(vres->ref > 0);
> >> +
> >> + if (qatomic_fetch_dec(&vres->ref) == 1) {
> >> + virgl_resource_unref(vres);
> >> + virgl_resource_destroy(vres);
> >> + }
> >> +}
> >> +
> >>  static struct virgl_gpu_resource *
> >>  virgl_gpu_find_resource(VirtIOGPU *g, uint32_t resource_id)
> >>  {
> >> @@ -59,6 +114,8 @@ static void virgl_cmd_create_resource_2d(VirtIOGPU *g,
> >>  c2d.width, c2d.height);
> >>  vres = g_new0(struct virgl_gpu_resource, 1);
> >> + vres_get_ref(vres);
> >> + vres->g = g;
> >>  vres->res.width = c2d.width;
> >>  vres->res.height = c2d.height;
> >>  vres->res.format = c2d.format;
> >> @@ -91,6 +148,8 @@ static void virgl_cmd_create_resource_3d(VirtIOGPU *g,
> >>  c3d.width, c3d.height, c3d.depth);
> >>  vres = g_new0(struct virgl_gpu_resource, 1);
> >> + vres_get_ref(vres);
> >> + vres->g = g;
> >>  vres->res.width = c3d.width;
> >>  vres->res.height = c3d.height;
> >>  vres->res.format = c3d.format;
> >> @@ -126,12 +185,21 @@ static void virgl_cmd_resource_unref(VirtIOGPU *g,
> >>  return;
> >>  }
> >> - virgl_renderer_resource_detach_iov(unref.resource_id, NULL, NULL);
> >> - virgl_renderer_resource_unref(unref.resource_id);
> >> +#ifdef HAVE_VIRGL_RESOURCE_BLOB
> >> + if (vres->region) {
> >> + VirtIOGPUBase *b = VIRTIO_GPU_BASE(g);
> >> + MemoryRegion *mr = vres->region;
> >> +
> >> + warn_report("%s: blob resource %d not unmapped",
> >> + __func__, unref.resource_id);
> >> + vres->region = NULL;
> > 
> > Shouldn't there be a call to memory_region_unref(mr)?
> > 
> >> + memory_region_set_enabled(mr, false);
> >> + memory_region_del_subregion(&b->hostmem, mr);
> >> + object_unparent(OBJECT(mr));
> >> + }
> >> +#endif /* HAVE_VIRGL_RESOURCE_BLOB */
> >> - QTAILQ_REMOVE(&g->reslist, &vres->res, next);
> >> - virtio_gpu_cleanup_mapping(g, &vres->res);
> >> - g_free(vres);
> >> + vres_put_ref(vres);
> >>  }
> >>  static void virgl_cmd_context_create(VirtIOGPU *g,
> >> @@ -470,6 +538,191 @@ static void virgl_cmd_get_capset(VirtIOGPU *g,
> >>  g_free(resp);
> >>  }
> >> +#ifdef HAVE_VIRGL_RESOURCE_BLOB
> >> +
> >> +static void virgl_resource_unmap(struct virgl_gpu_resource *vres)
> >> +{
> >> + if (!vres) {
> >> + return;
> >> + }
> >> +
> >> + virgl_renderer_resource_unmap(vres->res.resource_id);
> >> +
> >> + vres_put_ref(vres);
> >> +}
> >> +
> >> +static void virgl_resource_blob_async_unmap(void *obj)
> >> +{
> >> + MemoryRegion *mr = MEMORY_REGION(obj);
> >> + struct virgl_gpu_resource *vres = mr->opaque;
> >> +
> >> + virgl_resource_unmap(vres);
> >> +
> >> + g_free(obj);
> >> +}
> >> +
> >> +static void virgl_cmd_resource_create_blob(VirtIOGPU *g,
> >> + struct virtio_gpu_ctrl_command *cmd)
> >> +{
> >> + struct virgl_gpu_resource *vres;
> >> + struct virtio_gpu_resource_create_blob cblob;
> >> + struct virgl_renderer_resource_create_blob_args virgl_args = { 0 };
> >> + int ret;
> >> +
> >> + VIRTIO_GPU_FILL_CMD(cblob);
> >> + virtio_gpu_create_blob_bswap(&cblob);
> >> + trace_virtio_gpu_cmd_res_create_blob(cblob.resource_id, cblob.size);
> >> +
> >> + if (cblob.resource_id == 0) {
> >> + qemu_log_mask(LOG_GUEST_ERROR, "%s: resource id 0 is not allowed\n",
> >> + __func__);
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
> >> + return;
> >> + }
> >> +
> >> + vres = virgl_gpu_find_resource(g, cblob.resource_id);
> >> + if (vres) {
> >> + qemu_log_mask(LOG_GUEST_ERROR, "%s: resource already exists %d\n",
> >> + __func__, cblob.resource_id);
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
> >> + return;
> >> + }
> >> +
> >> + vres = g_new0(struct virgl_gpu_resource, 1);
> >> + vres_get_ref(vres);
> >> + vres->g = g;
> >> + vres->res.resource_id = cblob.resource_id;
> >> + vres->res.blob_size = cblob.size;
> >> +
> >> + if (cblob.blob_mem != VIRTIO_GPU_BLOB_MEM_HOST3D) {
> >> + ret = virtio_gpu_create_mapping_iov(g, cblob.nr_entries, sizeof(cblob),
> >> + cmd, &vres->res.addrs,
> >> + &vres->res.iov, &vres->res.iov_cnt);
> >> + if (!ret) {
> >> + g_free(vres);
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC;
> >> + return;
> >> + }
> >> + }
> >> +
> >> + QTAILQ_INSERT_HEAD(&g->reslist, &vres->res, next);
> >> +
> >> + virgl_args.res_handle = cblob.resource_id;
> >> + virgl_args.ctx_id = cblob.hdr.ctx_id;
> >> + virgl_args.blob_mem = cblob.blob_mem;
> >> + virgl_args.blob_id = cblob.blob_id;
> >> + virgl_args.blob_flags = cblob.blob_flags;
> >> + virgl_args.size = cblob.size;
> >> + virgl_args.iovecs = vres->res.iov;
> >> + virgl_args.num_iovs = vres->res.iov_cnt;
> >> +
> >> + ret = virgl_renderer_resource_create_blob(&virgl_args);
> >> + if (ret) {
> >> + qemu_log_mask(LOG_GUEST_ERROR, "%s: virgl blob create error: %s\n",
> >> + __func__, strerror(-ret));
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC;
> >> + }
> >> +}
> >> +
> >> +static void virgl_cmd_resource_map_blob(VirtIOGPU *g,
> >> + struct virtio_gpu_ctrl_command *cmd)
> >> +{
> >> + struct virgl_gpu_resource *vres;
> >> + struct virtio_gpu_resource_map_blob mblob;
> >> + int ret;
> >> + void *data;
> >> + uint64_t size;
> >> + struct virtio_gpu_resp_map_info resp;
> >> + VirtIOGPUBase *b = VIRTIO_GPU_BASE(g);
> >> +
> >> + VIRTIO_GPU_FILL_CMD(mblob);
> >> + virtio_gpu_map_blob_bswap(&mblob);
> >> +
> >> + if (mblob.resource_id == 0) {
> >> + qemu_log_mask(LOG_GUEST_ERROR, "%s: resource id 0 is not allowed\n",
> >> + __func__);
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
> >> + return;
> >> + }
> >> +
> >> + vres = virgl_gpu_find_resource(g, mblob.resource_id);
> >> + if (!vres) {
> >> + qemu_log_mask(LOG_GUEST_ERROR, "%s: resource does not exist %d\n",
> >> + __func__, mblob.resource_id);
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
> >> + return;
> >> + }
> >> + if (vres->region) {
> >> + qemu_log_mask(LOG_GUEST_ERROR, "%s: resource already mapped %d\n",
> >> + __func__, mblob.resource_id);
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
> >> + return;
> >> + }
> >> +
> >> + ret = virgl_renderer_resource_map(vres->res.resource_id, &data, &size);
> >> + if (ret) {
> >> + qemu_log_mask(LOG_GUEST_ERROR, "%s: resource map error: %s\n",
> >> + __func__, strerror(-ret));
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
> >> + return;
> >> + }
> >> +
> >> + vres_get_ref(vres);
> > 
> > Why is this needed? And if it is, shouldn't virgl_cmd_resource_unmap_blob
> > call "vres_put_ref(vres)" ?
> > 
> >> + vres->region = g_new0(MemoryRegion, 1);
> >> + memory_region_init_ram_ptr(vres->region, OBJECT(g), NULL, size, data);
> >> + vres->region->opaque = vres;
> >> + OBJECT(vres->region)->free = virgl_resource_blob_async_unmap;
> >> + memory_region_add_subregion(&b->hostmem, mblob.offset, vres->region);
> >> + memory_region_set_enabled(vres->region, true);
> >> +
> >> + memset(&resp, 0, sizeof(resp));
> >> + resp.hdr.type = VIRTIO_GPU_RESP_OK_MAP_INFO;
> >> + virgl_renderer_resource_get_map_info(mblob.resource_id, &resp.map_info);
> >> + virtio_gpu_ctrl_response(g, cmd, &resp.hdr, sizeof(resp));
> >> +}
> >> +
> >> +static void virgl_cmd_resource_unmap_blob(VirtIOGPU *g,
> >> + struct virtio_gpu_ctrl_command *cmd)
> >> +{
> >> + struct virgl_gpu_resource *vres;
> >> + struct virtio_gpu_resource_unmap_blob ublob;
> >> + VirtIOGPUBase *b = VIRTIO_GPU_BASE(g);
> >> + MemoryRegion *mr;
> >> +
> >> + VIRTIO_GPU_FILL_CMD(ublob);
> >> + virtio_gpu_unmap_blob_bswap(&ublob);
> >> +
> >> + if (ublob.resource_id == 0) {
> >> + qemu_log_mask(LOG_GUEST_ERROR, "%s: resource id 0 is not allowed\n",
> >> + __func__);
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
> >> + return;
> >> + }
> >> +
> >> + vres = virgl_gpu_find_resource(g, ublob.resource_id);
> >> + if (!vres) {
> >> + qemu_log_mask(LOG_GUEST_ERROR, "%s: resource does not exist %d\n",
> >> + __func__, ublob.resource_id);
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
> >> + return;
> >> + }
> >> +
> >> + if (!vres->region) {
> >> + qemu_log_mask(LOG_GUEST_ERROR, "%s: resource already unmapped %d\n",
> >> + __func__, ublob.resource_id);
> >> + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID;
> >> + return;
> >> + }
> >> +
> >> + mr = vres->region;
> >> + vres->region = NULL;
> > 
> > memory_region_unref(mr)?
> > 
> > Note that AFAICT without the added memory_region_unref() calls virgl_resource_unmap()
> > was never called.
> 
> 
> Xenia and I figured out the refcounting issue: this code is written based on the
> assumption that:
>   
>     object_unparent(OBJECT(mr));
> 
> 
> Will decrement the refcount. But this assumption is only true if mr->parent_obj.parent
> is non-NULL.
> 
> The map_blob function uses the following arguments:
> 
>     memory_region_init_ram_ptr(vres->region, OBJECT(g), NULL, size, data);
> 
> Since name is NULL, mr won't be added as a child of 'g' and thus object_unparent()
> does nothing.
> 
> I'd suggest 2 changes:
>     * use a name ("blob_memory"?) to so mr can be a child of g
>     * increment mr's refcount when setting vres->region and decrement it when clearing it.
>       This change is not needed technically but when a variable is refcounted it seems
>       clearer to increment/decrement the refcount in these situations.
> 

Yes, issue is caused by NULL in name field, I will update according to your
suggestions in V7.

Thanks,
Ray

> 
> Pierre-Eric
> 
> 
> > 
> >> + memory_region_set_enabled(mr, false);
> >> + memory_region_del_subregion(&b->hostmem, mr);
> >> + object_unparent(OBJECT(mr));
> >> +}
> >> +
> >> +#endif /* HAVE_VIRGL_RESOURCE_BLOB */
> >> +
> >>  void virtio_gpu_virgl_process_cmd(VirtIOGPU *g,
> >>  struct virtio_gpu_ctrl_command *cmd)
> >>  {
> >> @@ -536,6 +789,17 @@ void virtio_gpu_virgl_process_cmd(VirtIOGPU *g,
> >>  case VIRTIO_GPU_CMD_GET_EDID:
> >>  virtio_gpu_get_edid(g, cmd);
> >>  break;
> >> +#ifdef HAVE_VIRGL_RESOURCE_BLOB
> >> + case VIRTIO_GPU_CMD_RESOURCE_CREATE_BLOB:
> >> + virgl_cmd_resource_create_blob(g, cmd);
> >> + break;
> >> + case VIRTIO_GPU_CMD_RESOURCE_MAP_BLOB:
> >> + virgl_cmd_resource_map_blob(g, cmd);
> >> + break;
> >> + case VIRTIO_GPU_CMD_RESOURCE_UNMAP_BLOB:
> >> + virgl_cmd_resource_unmap_blob(g, cmd);
> >> + break;
> >> +#endif /* HAVE_VIRGL_RESOURCE_BLOB */
> >>  default:
> >>  cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC;
> >>  break;
> >> diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
> >> index 4c3ec9d0ea..8189c392dc 100644
> >> --- a/hw/display/virtio-gpu.c
> >> +++ b/hw/display/virtio-gpu.c
> >> @@ -1449,10 +1449,12 @@ void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
> >>  return;
> >>  }
> >> +#ifndef HAVE_VIRGL_RESOURCE_BLOB
> >>  if (virtio_gpu_virgl_enabled(g->parent_obj.conf)) {
> >> - error_setg(errp, "blobs and virgl are not compatible (yet)");
> >> + error_setg(errp, "Linked virglrenderer does not support blob resources");
> >>  return;
> >>  }
> >> +#endif
> >>  }
> >>  if (!virtio_gpu_base_device_realize(qdev,
> >> diff --git a/meson.build b/meson.build
> >> index ea52ef1b9c..629407128e 100644
> >> --- a/meson.build
> >> +++ b/meson.build
> >> @@ -1054,6 +1054,10 @@ if not get_option('virglrenderer').auto() or have_system or have_vhost_user_gpu
> >>  cc.has_function('virgl_renderer_context_create_with_flags',
> >>  prefix: '#include <virglrenderer.h>',
> >>  dependencies: virgl))
> >> + config_host_data.set('HAVE_VIRGL_RESOURCE_BLOB',
> >> + cc.has_function('virgl_renderer_resource_create_blob',
> >> + prefix: '#include <virglrenderer.h>',
> >> + dependencies: virgl))
> >>  endif
> >>  endif
> >>  rutabaga = not_found
> > 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 07:36:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 07:36:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684649.1064641 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdQ6l-0003ia-0c; Fri, 23 Feb 2024 07:36:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684649.1064641; Fri, 23 Feb 2024 07:36:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdQ6k-0003iT-Tn; Fri, 23 Feb 2024 07:36:34 +0000
Received: by outflank-mailman (input) for mailman id 684649;
 Fri, 23 Feb 2024 07:36:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6083=KA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rdQ6j-0003iN-O8
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 07:36:33 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 42e9baac-d21e-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 08:36:31 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-564372fb762so585040a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 23:36:31 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n10-20020aa7c78a000000b005643e2ea283sm5396721eds.39.2024.02.22.23.36.29
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 23:36:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 42e9baac-d21e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708673790; x=1709278590; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pmkbwePMKm9EByqKANYIYNp4pxpH8D4joyd4XABIw00=;
        b=KP7ZLwl44ePsLgCnhhgjQM5TZglmweNT5sXG/ZwtwktjK0B1P68sf1SFOd9692AJgu
         bJf0WsctMLoOY6+iBxTceM9KKSfMugYqnErvTNkPb5b3EKATKg/57OqS+PCy9TMt1TAy
         IdCCKYJx8LXFFqn6bNFSPeMFTWYKnOfZLOdcxLiLt75hf2ipfg0Q5bb3CRSXtLpYmRR7
         BuM/1wI1q0j15uuNzZXxUhO+5Uo+kse3OT+jWaVSEZqyytuPSLBHThg8RnCtAYVQTOXS
         7b1kx+Xs8sT6aSl2XTLPaPio0/pIRZvZ7vSE0d/NqV0GjgPMrYgHZt8ChBRh5d/r2VF7
         8umw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708673790; x=1709278590;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pmkbwePMKm9EByqKANYIYNp4pxpH8D4joyd4XABIw00=;
        b=kpQ9aocZMOM9qK1m3lrkvBjovZEXnHsmOAAJkqOyqqqcMVO9cAtMyM0cwQiA/+FTuC
         E3BN/QCHpsmX4Owr9auB/SIFsTd+h0xcEUCfXEgpNtD99Yzm6P/D5jobET7GzmM748Nd
         uspUIZgHB3qTxRWq7qF/FeeAe9+odcC4kZLDpzRqpF5cuN44gOneJxkv8fPztEXmmSOi
         XPPRzPHx85ZJX/K+ospctYmrLub4z1KjJ8G15qjC593aV+ONXMM3M7Wyjgf+m/9GnQ6Q
         aB2V+WZY33hd4CoPLAugKP0wNAtEClYm62yO1RFYdKTyUiK4wmbkmx3XaX0kkluTQiAG
         O/fw==
X-Forwarded-Encrypted: i=1; AJvYcCWTsLCWlXCJ6v9IpVkphIqzwasH6yTKqW2+g+oFo6akhgerjTX7y+KP+bNiRksQYGe7clOxjCdv8fv0k8CB80DWYtLjbY5jI7AoeLQItlA=
X-Gm-Message-State: AOJu0YzgCumCOq0jM+Y/euHkrH+aiQ/QzoINo5D62temWnBsTTr8ejBh
	FXX8Sq8wKTbm7DaNmvkX6eepbL3XwyzoRafnrsvf9SZSKbICKajO0d9I4VVweA==
X-Google-Smtp-Source: AGHT+IHUGsvWsDinspQmhmNmSwelLqeba8knyJ3g1Crk+PnEId2SGU43NU/AwviPHaTD7REiYQLoNQ==
X-Received: by 2002:aa7:c515:0:b0:565:7eff:58d5 with SMTP id o21-20020aa7c515000000b005657eff58d5mr118625edq.41.1708673790442;
        Thu, 22 Feb 2024 23:36:30 -0800 (PST)
Message-ID: <6eb4dc7c-c6a5-496a-a37f-125b00e9aaa3@suse.com>
Date: Fri, 23 Feb 2024 08:36:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 3/4] xen/include: add pure and const attributes
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Simone Ballarin <simone.ballarin@bugseng.com>, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1697638210.git.simone.ballarin@bugseng.com>
 <fd5421162a00aa782e0776324ff6497193c1e3d3.1697638210.git.simone.ballarin@bugseng.com>
 <89778285-5cba-8fb5-70bc-710b6dd30a10@suse.com>
 <0032186f-80c7-4dba-b46e-10d4a8e2a8cb@bugseng.com>
 <e8bf9817-fd54-9bf4-4302-dcee682f9172@suse.com>
 <alpine.DEB.2.22.394.2310231417260.3516@ubuntu-linux-20-04-desktop>
 <8f8141e2-c55c-2e58-a078-d5d8764a02df@suse.com>
 <alpine.DEB.2.22.394.2402221730270.754277@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402221730270.754277@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.02.2024 02:32, Stefano Stabellini wrote:
> On Tue, 24 Oct 2023, Jan Beulich wrote:
>> On 24.10.2023 00:05, Stefano Stabellini wrote:
>>> On Mon, 23 Oct 2023, Jan Beulich wrote:
>>>> On 23.10.2023 17:23, Simone Ballarin wrote:
>>>>> On 23/10/23 15:34, Jan Beulich wrote:
>>>>>> On 18.10.2023 16:18, Simone Ballarin wrote:
>>>>>>> --- a/xen/include/xen/pdx.h
>>>>>>> +++ b/xen/include/xen/pdx.h
>>>>>>> @@ -164,7 +164,7 @@ static inline unsigned long pfn_to_pdx(unsigned long pfn)
>>>>>>>    * @param pdx Page index
>>>>>>>    * @return Obtained pfn after decompressing the pdx
>>>>>>>    */
>>>>>>> -static inline unsigned long pdx_to_pfn(unsigned long pdx)
>>>>>>> +static inline __attribute_pure__ unsigned long pdx_to_pfn(unsigned long pdx)
>>>>>>>   {
>>>>>>>       return (pdx & pfn_pdx_bottom_mask) |
>>>>>>>              ((pdx << pfn_pdx_hole_shift) & pfn_top_mask);
>>>>>>
>>>>>> Taking this as an example for what I've said above: The compiler can't
>>>>>> know that the globals used by the functions won't change value. Even
>>>>>> within Xen it is only by convention that these variables are assigned
>>>>>> their values during boot, and then aren't changed anymore. Which makes
>>>>>> me wonder: Did you check carefully that around the time the variables
>>>>>> have their values established, no calls to the functions exist (which
>>>>>> might then be subject to folding)?
>>>>>
>>>>> There is no need to check that, the GCC documentation explicitly says:
>>>>>
>>>>> However, functions declared with the pure attribute *can safely read any 
>>>>> non-volatile objects*, and modify the value of objects in a way that 
>>>>> does not affect their return value or the observable state of the program.
>>>>
>>>> I did quote this same text in response to what Andrew has said, but I also
>>>> did note there that this needs to be taken with a grain of salt: The
>>>> compiler generally assumes a single-threaded environment, i.e. no changes
>>>> to globals behind the back of the code it is processing.
>>>
>>> Let's start from the beginning. The reason for Simone to add
>>> __attribute_pure__ to pdx_to_pfn and other functions is for
>>> documentation purposes. It is OK if it doesn't serve any purpose other
>>> than documentation.
>>>
>>> Andrew, for sure we do not want to lie to the compiler and introduce
>>> undefined behavior. If we think there is a risk of it, we should not do
>>> it.
>>>
>>> So, what do we want to document? We want to document that the function
>>> does not have side effects according to MISRA's definition of it, which
>>> might subtly differ from GCC's definition.
>>>
>>> Looking at GCC's definition of __attribute_pure__, with the
>>> clarification statement copy/pasted above by both Simone and Jan, it
>>> seems that __attribute_pure__ matches MISRA's definition of a function
>>> without side effects. It also seems that pdx_to_pfn abides to that
>>> definition.
>>>
>>> Jan has a point that GCC might be making other assumptions
>>> (single-thread execution) that might not hold true in our case. Given
>>> the way the GCC statement is written I think this is low risk. But maybe
>>> not all GCC versions we want to support in the project might have the
>>> same definition of __attribute_pure__. So we could end up using
>>> __attribute_pure__ correctly for the GCC version used for safety (GCC
>>> 12.1, see docs/misra/C-language-toolchain.rst) but it might actually
>>> break an older GCC version.
>>>
>>>
>>> So Option#1 is to use __attribute_pure__ taking the risk that a GCC or
>>> Clang version might interpret __attribute_pure__ differently and
>>> potentially misbehave.
>>>
>>> Option#2 is to avoid this risk, by not using __attribute_pure__.
>>> Instead, we can use SAF-xx-safe or deviations.rst to document that
>>> pdx_to_pfn and other functions like it are without side effects
>>> according to MISRA's definition.
>>>
>>>
>>> Both options have pros and cons. To me the most important factor is how
>>> many GCC versions come with the statement "pure attribute can safely
>>> read any non-volatile objects, and modify the value of objects in a way
>>> that does not affect their return value or the observable state of the
>>> program".
>>>
>>> I checked and these are the results:
>>> - gcc 4.0.2: no statement
>>> - gcc 5.1.0: no statement
>>> - gcc 6.1.0: no statement
>>> - gcc 7.1.0: no statement
>>> - gcc 8.1.0: alternative statement "The pure attribute imposes similar
>>>   but looser restrictions on a function’s definition than the const
>>>   attribute: it allows the function to read global variables."
>>> - gcc 9.1.0: yes statement
>>>
>>>
>>> So based on the above, __attribute_pure__ comes with its current
>>> definition only from gcc 9 onward. I don't know if as a Xen community we
>>> clearly declare a range of supported compilers, but I would imagine we
>>> would still want to support gcc versions older than 9? (Not to mention
>>> clang, which I haven't checked.)
>>>
>>> It doesn't seem to me that __attribute_pure__ could be correctly used on
>>> pdx_to_pfn with GCC 7.1.0 for example.
>>
>> The absence of documentation doesn't mean the attribute had different
>> (or even undefined) meaning in earlier versions. Instead it means one
>> would need to consult other places (source code?) to figure out whether
>> there was any behavioral difference (I don't think there was).
>>
>> That said, ...
>>
>>> So in conclusion, I think it is better to avoid __attribute_pure__ and
>>> use SAF-xx-safe or an alternative approach instead.
>>
>> ... I agree here. We just don't want to take chances.
> 
> Let me resurrect this thread.
> 
> Could we use something like "pure" that we #define as we want?
> 
> Depending on the compiler version or other options we could #define pure
> to __attribute_pure__ or to nothing.

While we can do about anything, I don't think it's a good idea to overload
a well known term with something having somewhat different meaning. If a
differently named custom attribute helps, that might be a possible option.

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 07:43:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 07:43:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684653.1064651 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdQDR-0005Lf-MY; Fri, 23 Feb 2024 07:43:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684653.1064651; Fri, 23 Feb 2024 07:43:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdQDR-0005LY-Jt; Fri, 23 Feb 2024 07:43:29 +0000
Received: by outflank-mailman (input) for mailman id 684653;
 Fri, 23 Feb 2024 07:43:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6083=KA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rdQDQ-0005LS-BJ
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 07:43:28 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3a6b21c9-d21f-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 08:43:26 +0100 (CET)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-563b7b3e3ecso572754a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 22 Feb 2024 23:43:26 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 da16-20020a056402177000b00564e4debe29sm2517408edb.78.2024.02.22.23.43.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 22 Feb 2024 23:43:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a6b21c9-d21f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708674205; x=1709279005; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=j7HsQO81s/fhD77QVQwHTCJVNUl5T8w6MyX423vUmDg=;
        b=cuvwuLzABNJmxEyEYrBO395PhshU/qetcYVtdkorRnfQUF+7LZe6m3d2xs+cxv8MMp
         MdWv6r5LwYG+J8XtHfnUGrzFn4zFqBW4lCLMOHFNNVBQiRuMJITW82AEBlUd4JQMFhKS
         gWsZJu8nsg/CWE2Z+j7m9ENPjnHZvJHAa9/hmbkpZlsx23nXvq6UzXif1bspXlFizW1K
         sDgJhlxxJOnMF2LGr9NZMnKm++U82cGTwxK2CooIyjGUvA0bCa3iyHu3YaHjhTCLMj+t
         pvtrvA8XXqnZ6Pbl91rTEhvIY3W+fwBbbJdTcd+lGPySl+BTlGc7Po/L1K1fuInDapeu
         f+5w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708674205; x=1709279005;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=j7HsQO81s/fhD77QVQwHTCJVNUl5T8w6MyX423vUmDg=;
        b=suUDMaAx7WFuFfsEGqT/q7URE1N+IZLixYgHSExj7j4YsNc1E9bNcquj2rgxkoXhKW
         iJ3U2ia+gcW/xqsyQsKN131a6DOY1myCxHVNWkAs8O4LzJ7zde1CyDTRZ3myrVcZ+IaV
         wjW5ViW9ixYSz9xfqxl7+/7P6ICPGYq07HIG1WzGY//NI4/wDMTnNArJJHUoj403hFJZ
         OghvZg9oH04SySe2Eu1kS2rVk1RIC2T11/bm8znpXmnLiTvN6AXvn4FidTx1ExcFTaLm
         aTM58opY2Ci7NNVwn2Ah0zmP8US81c8PqkiQgU077ndEPHpMC0q+NWppXFF1B/DFa6Gd
         ZbpA==
X-Forwarded-Encrypted: i=1; AJvYcCVWCVEURBDmRampV2kSlsa1LR102VYFG6dpAjDrj4j6XYPVcVT371fLG2/UqFigGaOXEATiE/CZTlQ0FM0/xe420qy/CviRptLsoS70P+c=
X-Gm-Message-State: AOJu0Yy63GDDaxdTiRizTtNXJRplBiAYisnJfVNiQtj/Chvf84WH6aah
	upI+HPNqAptaLn5i5X84+zTH6DJW+YjXNfRJM9TRT1Hwtw0erQVmz+DpO4DO2w==
X-Google-Smtp-Source: AGHT+IHfBWxBo4rWvgTMPqfQ6qvGftE8tA46rGuCuKHtT4T9sFBA31ibVrLSST45Q6VBuvvkwEPyxw==
X-Received: by 2002:aa7:d50a:0:b0:564:b822:9204 with SMTP id y10-20020aa7d50a000000b00564b8229204mr613516edq.17.1708674205655;
        Thu, 22 Feb 2024 23:43:25 -0800 (PST)
Message-ID: <2f68276d-0b10-4cec-8636-5456314c194e@suse.com>
Date: Fri, 23 Feb 2024 08:43:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] x86/memsharing: use an atomic add instead of a
 cmpxchg loop
Content-Language: en-US
To: Tamas K Lengyel <tamas@tklengyel.com>
Cc: Roger Pau Monne <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-2-roger.pau@citrix.com>
 <d8939069-a1cb-4794-a6aa-75a3e4d97884@suse.com>
 <CABfawhkhH7Pakkq+RgjfmW-f60CbOKxr6wWxvBSr2AkN+m1OdQ@mail.gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <CABfawhkhH7Pakkq+RgjfmW-f60CbOKxr6wWxvBSr2AkN+m1OdQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.02.2024 19:03, Tamas K Lengyel wrote:
> On Thu, Feb 22, 2024 at 5:06 AM Jan Beulich <jbeulich@suse.com> wrote:
>> On 22.02.2024 10:05, Roger Pau Monne wrote:
>>> The usage of a cmpxchg loop in get_next_handle() is unnecessary, as the same
>>> can be achieved with an atomic increment, which is both simpler to read, and
>>> avoid any need for a loop.
>>>
>>> The cmpxchg usage is likely a remnant of 32bit support, which didn't have an
>>> instruction to do an atomic 64bit add, and instead a cmpxchg had to be used.
>>>
>>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>
>>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>> albeit ...
>>
>>> --- a/xen/arch/x86/mm/mem_sharing.c
>>> +++ b/xen/arch/x86/mm/mem_sharing.c
>>> @@ -179,13 +179,7 @@ static void mem_sharing_page_unlock(struct page_info *pg)
>>>
>>>  static shr_handle_t get_next_handle(void)
>>>  {
>>> -    /* Get the next handle get_page style */
>>> -    uint64_t x, y = next_handle;
>>> -    do {
>>> -        x = y;
>>> -    }
>>> -    while ( (y = cmpxchg(&next_handle, x, x + 1)) != x );
>>> -    return x + 1;
>>> +    return arch_fetch_and_add(&next_handle, 1) + 1;
>>>  }
>>
>> ... the adding of 1 here is a little odd when taken together with
>> next_handle's initializer. Tamas, you've not written that code, but do
>> you have any thoughts towards the possible removal of either the
>> initializer or the adding here? Plus that variable of course could
>> very well do with moving into this function.
> 
> I have to say I find the existing logic here hard to parse but by the
> looks I don't think we need the + 1 once we switch to
> arch_fetch_and_add. Also could go without initializing next_handle to
> 1. Moving it into the function would not really accomplish anything
> other than style AFAICT?

Well, limiting scope of things can be viewed as purely style, but I
think it's more than that: It makes intentions more clear and reduces
the chance of abuse (deliberate or unintentional).

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 07:58:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 07:58:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684657.1064662 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdQS4-00078L-V7; Fri, 23 Feb 2024 07:58:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684657.1064662; Fri, 23 Feb 2024 07:58:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdQS4-00078E-Ry; Fri, 23 Feb 2024 07:58:36 +0000
Received: by outflank-mailman (input) for mailman id 684657;
 Fri, 23 Feb 2024 07:58:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hL0v=KA=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rdQS2-000788-TY
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 07:58:34 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 57272a4e-d221-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 08:58:33 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id CB65A4EE073D;
 Fri, 23 Feb 2024 08:58:32 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57272a4e-d221-11ee-8a57-1f161083a0e0
MIME-Version: 1.0
Date: Fri, 23 Feb 2024 08:58:32 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com, jbeulich@suse.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, George
 Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
In-Reply-To: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
Message-ID: <4f3b3d52a3ba20b05ecfa068b916b804@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-19 16:14, Nicola Vetrini wrote:
> The cache clearing and invalidation helpers in x86 and Arm didn't
> comply with MISRA C Rule 17.7: "The value returned by a function
> having non-void return type shall be used". On Arm they
> were always returning 0, while some in x86 returned -EOPNOTSUPP
> and in common/grant_table the return value is saved.
> 
> As a consequence, a common helper arch_grant_cache_flush that returns
> an integer is introduced, so that each architecture can choose whether 
> to
> return an error value on certain conditions, and the helpers have 
> either
> been changed to return void (on Arm) or deleted entirely (on x86).
> 
> Signed-off-by: Julien Grall <julien@xen.org>
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> ---
> The original refactor idea came from Julien Grall in [1]; I edited that 
> proposal
> to fix build errors.
> 
> I did introduce a cast to void for the call to flush_area_local on x86, 
> because
> even before this patch the return value of that function wasn't checked 
> in all
> but one use in x86/smp.c, and in this context the helper (perhaps 
> incidentally)
> ignored the return value of flush_area_local.
> 
> [1] 
> https://lore.kernel.org/xen-devel/09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org/
> ---
>  xen/arch/arm/include/asm/page.h     | 33 ++++++++++++++++++-----------
>  xen/arch/x86/include/asm/flushtlb.h | 23 ++++++++++----------
>  xen/common/grant_table.c            |  9 +-------
>  3 files changed, 34 insertions(+), 31 deletions(-)
> 

I'll put this patch in the backlog at the moment: too many intricacies 
while trying to untangle xen/flushtlb from xen/mm.h, and there are 
easier cases that can be done faster. If someone is interested I can 
post the partial work I've done so far, even though it doesn't
build on x86.

> diff --git a/xen/arch/arm/include/asm/page.h 
> b/xen/arch/arm/include/asm/page.h
> index 69f817d1e68a..e90c9de3616e 100644
> --- a/xen/arch/arm/include/asm/page.h
> +++ b/xen/arch/arm/include/asm/page.h
> @@ -123,6 +123,7 @@
> 
>  #ifndef __ASSEMBLY__
> 
> +#include <public/grant_table.h>
>  #include <xen/errno.h>
>  #include <xen/types.h>
>  #include <xen/lib.h>
> @@ -159,13 +160,13 @@ static inline size_t read_dcache_line_bytes(void)
>   * if 'range' is large enough we might want to use model-specific
>   * full-cache flushes. */
> 
> -static inline int invalidate_dcache_va_range(const void *p, unsigned 
> long size)
> +static inline void invalidate_dcache_va_range(const void *p, unsigned 
> long size)
>  {
>      size_t cacheline_mask = dcache_line_bytes - 1;
>      unsigned long idx = 0;
> 
>      if ( !size )
> -        return 0;
> +        return;
> 
>      /* Passing a region that wraps around is illegal */
>      ASSERT(((uintptr_t)p + size - 1) >= (uintptr_t)p);
> @@ -188,17 +189,15 @@ static inline int 
> invalidate_dcache_va_range(const void *p, unsigned long size)
>          asm volatile (__clean_and_invalidate_dcache_one(0) : : "r" (p 
> + idx));
> 
>      dsb(sy);           /* So we know the flushes happen before 
> continuing */
> -
> -    return 0;
>  }
> 
> -static inline int clean_dcache_va_range(const void *p, unsigned long 
> size)
> +static inline void clean_dcache_va_range(const void *p, unsigned long 
> size)
>  {
>      size_t cacheline_mask = dcache_line_bytes - 1;
>      unsigned long idx = 0;
> 
>      if ( !size )
> -        return 0;
> +        return;
> 
>      /* Passing a region that wraps around is illegal */
>      ASSERT(((uintptr_t)p + size - 1) >= (uintptr_t)p);
> @@ -211,18 +210,16 @@ static inline int clean_dcache_va_range(const 
> void *p, unsigned long size)
>              idx += dcache_line_bytes, size -= dcache_line_bytes )
>          asm volatile (__clean_dcache_one(0) : : "r" (p + idx));
>      dsb(sy);           /* So we know the flushes happen before 
> continuing */
> -    /* ARM callers assume that dcache_* functions cannot fail. */
> -    return 0;
>  }
> 
> -static inline int clean_and_invalidate_dcache_va_range
> +static inline void clean_and_invalidate_dcache_va_range
>      (const void *p, unsigned long size)
>  {
>      size_t cacheline_mask = dcache_line_bytes - 1;
>      unsigned long idx = 0;
> 
>      if ( !size )
> -        return 0;
> +        return;
> 
>      /* Passing a region that wraps around is illegal */
>      ASSERT(((uintptr_t)p + size - 1) >= (uintptr_t)p);
> @@ -235,8 +232,6 @@ static inline int 
> clean_and_invalidate_dcache_va_range
>              idx += dcache_line_bytes, size -= dcache_line_bytes )
>          asm volatile (__clean_and_invalidate_dcache_one(0) : : "r" (p 
> + idx));
>      dsb(sy);         /* So we know the flushes happen before 
> continuing */
> -    /* ARM callers assume that dcache_* functions cannot fail. */
> -    return 0;
>  }
> 
>  /* Macros for flushing a single small item.  The predicate is always
> @@ -266,6 +261,20 @@ static inline int 
> clean_and_invalidate_dcache_va_range
>              : : "r" (_p), "m" (*_p));                                  
>  \
>  } while (0)
> 
> +static inline int arch_grant_cache_flush(unsigned int op, const void 
> *p,
> +                                         unsigned long size)
> +{
> +    if ( (op & GNTTAB_CACHE_INVAL) && (op & GNTTAB_CACHE_CLEAN) )
> +        clean_and_invalidate_dcache_va_range(p, size);
> +    else if ( op & GNTTAB_CACHE_INVAL )
> +        invalidate_dcache_va_range(p, size);
> +    else if ( op & GNTTAB_CACHE_CLEAN )
> +        clean_dcache_va_range(p, size);
> +
> +    /* ARM callers assume that dcache_* functions cannot fail. */
> +    return 0;
> +}
> +
>  /*
>   * Write a pagetable entry.
>   *
> diff --git a/xen/arch/x86/include/asm/flushtlb.h 
> b/xen/arch/x86/include/asm/flushtlb.h
> index bb0ad58db49b..c37bf4455714 100644
> --- a/xen/arch/x86/include/asm/flushtlb.h
> +++ b/xen/arch/x86/include/asm/flushtlb.h
> @@ -10,6 +10,7 @@
>  #ifndef __FLUSHTLB_H__
>  #define __FLUSHTLB_H__
> 
> +#include <public/grant_table.h>
>  #include <xen/mm.h>
>  #include <xen/percpu.h>
>  #include <xen/smp.h>
> @@ -182,21 +183,21 @@ void flush_area_mask(const cpumask_t *mask, const 
> void *va,
>  }
> 
>  static inline void flush_page_to_ram(unsigned long mfn, bool 
> sync_icache) {}
> -static inline int invalidate_dcache_va_range(const void *p,
> -                                             unsigned long size)
> -{ return -EOPNOTSUPP; }
> -static inline int clean_and_invalidate_dcache_va_range(const void *p,
> -                                                       unsigned long 
> size)
> +
> +static inline int arch_grant_cache_flush(unsigned int op, const void 
> *p,
> +                                     unsigned long size)
>  {
> -    unsigned int order = get_order_from_bytes(size);
> +    unsigned int order;
> +
> +    if ( !(op & GNTTAB_CACHE_CLEAN) )
> +        return -EOPNOTSUPP;
> +
> +    order = get_order_from_bytes(size);
>      /* sub-page granularity support needs to be added if necessary */
> -    flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
> +    (void) flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
> +
>      return 0;
>  }
> -static inline int clean_dcache_va_range(const void *p, unsigned long 
> size)
> -{
> -    return clean_and_invalidate_dcache_va_range(p, size);
> -}
> 
>  unsigned int guest_flush_tlb_flags(const struct domain *d);
>  void guest_flush_tlb_mask(const struct domain *d, const cpumask_t 
> *mask);
> diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
> index 5721eab22561..8615ea144bb3 100644
> --- a/xen/common/grant_table.c
> +++ b/xen/common/grant_table.c
> @@ -3572,14 +3572,7 @@ static int _cache_flush(const 
> gnttab_cache_flush_t *cflush, grant_ref_t *cur_ref
>      v = map_domain_page(mfn);
>      v += cflush->offset;
> 
> -    if ( (cflush->op & GNTTAB_CACHE_INVAL) && (cflush->op & 
> GNTTAB_CACHE_CLEAN) )
> -        ret = clean_and_invalidate_dcache_va_range(v, cflush->length);
> -    else if ( cflush->op & GNTTAB_CACHE_INVAL )
> -        ret = invalidate_dcache_va_range(v, cflush->length);
> -    else if ( cflush->op & GNTTAB_CACHE_CLEAN )
> -        ret = clean_dcache_va_range(v, cflush->length);
> -    else
> -        ret = 0;
> +    ret = arch_grant_cache_flush(cflush->op, v, cflush->length);
> 
>      if ( d != owner )
>      {

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:05:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:05:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684678.1064672 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRUq-0007Yq-9c; Fri, 23 Feb 2024 09:05:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684678.1064672; Fri, 23 Feb 2024 09:05:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRUq-0007Yj-5t; Fri, 23 Feb 2024 09:05:32 +0000
Received: by outflank-mailman (input) for mailman id 684678;
 Fri, 23 Feb 2024 09:05:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqZf=KA=amd.com=Ray.Huang@srs-se1.protection.inumbo.net>)
 id 1rdRUo-0007Yd-KW
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:05:30 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id af30bb49-d22a-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 10:05:27 +0100 (CET)
Received: from SJ2PR12MB8690.namprd12.prod.outlook.com (2603:10b6:a03:540::10)
 by DM3PR12MB9326.namprd12.prod.outlook.com (2603:10b6:0:3d::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.24; Fri, 23 Feb
 2024 09:05:24 +0000
Received: from SJ2PR12MB8690.namprd12.prod.outlook.com
 ([fe80::e7a:5022:4b7d:ade1]) by SJ2PR12MB8690.namprd12.prod.outlook.com
 ([fe80::e7a:5022:4b7d:ade1%6]) with mapi id 15.20.7316.023; Fri, 23 Feb 2024
 09:05:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af30bb49-d22a-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Lx7YqNic0jCnKxQxNUtNJ0D2NvhbsHKXko7iv4CAg4/OiGb78mZLGYpjaddgVCuLRbH3beDjfkPG6aelcMALvZ6s5x/OjXTE7/5eXik8tcbxYw7qQmAJ89AFmkGf3HYQ24RZFuWQGPOekoVP92cQGZ2o8t/kuEYXn2Cz/yoQheHaaPEP/shrmF92XWM8UaD4ga1aPfYa/NOjl/FqDMXdx9pPdNY9NqV0r+MTiHkN799bKpiPZSVavIdJEk+6KpoEqn3yu/cwAM6l5i4FCM/V7JUCbHFPnPBjx3PW/jyRR/Vuz0O4rk/lMBiuKmkLGdWaYKIxGAG8YFpx3pymRdQKMA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Xh62kUWafKq/zvrKyfXL6ltH0QXBb9WHQ12LE//thAc=;
 b=O6mdSkNOFLy6R+WNp/uDJxqPmIgxryGv5gWKiLJl0Qf8c9ykBBrM/iTwflcYDSEiomhx+O7QwtG6RP3+PSDD9q7PXsRlz/S4YgzUs5yTAaF5HbRYD7k1pRGENps03I5+Z/2BR34eX2P6BeJjMLpa5ppslKWc9ImNZql/X/KiBt3GzB1uc5mublhAC0Wffg3q+epFtLgts3OlTY214e05YkpeKzOOrI393sSe8/PeMBiukVvgZ5E8V6+hVUHlweTOpt5AkTP/dK8cTTNbHt4DQu7JFwLQIeGC89fwohSreknbKILrsmrUF76BWUJTCHmBq/Gtw155H+idV8jr2ycltA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Xh62kUWafKq/zvrKyfXL6ltH0QXBb9WHQ12LE//thAc=;
 b=f/nAZJuzCSZB0l4IYvI+wr7a2UVrzhKrzHoK2qAH5PoYkV5zzZlZZYGbaP6++QN/hG9CmsRTrwJVcf719i5i2cDpEViS/qd2Xjwnbsg4HMYucHj+gRL+HfilbkExfDxzTDW+i06NahkIKKJ08pi2wjVkthE/wZlaRH5/8tC58r4=
Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
Date: Fri, 23 Feb 2024 17:04:24 +0800
From: Huang Rui <ray.huang@amd.com>
To: =?iso-8859-1?Q?Marc-Andr=E9?= Lureau <marcandre.lureau@gmail.com>
Cc: Akihiko Odaki <akihiko.odaki@daynix.com>,
	Philippe =?iso-8859-1?Q?Mathieu-Daud=E9?= <philmd@linaro.org>,
	Gerd Hoffmann <kraxel@redhat.com>,
	"Michael S . Tsirkin" <mst@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Antonio Caggiano <quic_acaggian@quicinc.com>,
	"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
	Robert Beckett <bob.beckett@collabora.com>,
	Dmitry Osipenko <dmitry.osipenko@collabora.com>,
	Gert Wollny <gert.wollny@collabora.com>,
	Alex =?iso-8859-1?Q?Benn=E9e?= <alex.bennee@linaro.org>,
	"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Gurchetan Singh <gurchetansingh@chromium.org>,
	"ernunes@redhat.com" <ernunes@redhat.com>,
	Alyssa Ross <hi@alyssa.is>,
	Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
	"Deucher, Alexander" <Alexander.Deucher@amd.com>,
	"Stabellini, Stefano" <stefano.stabellini@amd.com>,
	"Koenig, Christian" <Christian.Koenig@amd.com>,
	"Ragiadakou, Xenia" <Xenia.Ragiadakou@amd.com>,
	"Pelloux-Prayer, Pierre-Eric" <Pierre-eric.Pelloux-prayer@amd.com>,
	"Huang, Honglei1" <Honglei1.Huang@amd.com>,
	"Zhang, Julia" <Julia.Zhang@amd.com>,
	"Chen, Jiqian" <Jiqian.Chen@amd.com>,
	Antonio Caggiano <antonio.caggiano@collabora.com>
Subject: Re: [PATCH v6 08/11] virtio-gpu: Resource UUID
Message-ID: <ZdhfmJZC8ws4KIhi@amd.com>
References: <20231219075320.165227-1-ray.huang@amd.com>
 <20231219075320.165227-9-ray.huang@amd.com>
 <CAJ+F1C+NbeFkiGkN=JRifbs6QU2zyiMKUfQxA9KdonfFrL1CUg@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAJ+F1C+NbeFkiGkN=JRifbs6QU2zyiMKUfQxA9KdonfFrL1CUg@mail.gmail.com>
X-ClientProxiedBy: TYCP286CA0135.JPNP286.PROD.OUTLOOK.COM
 (2603:1096:400:2b6::19) To SJ2PR12MB8690.namprd12.prod.outlook.com
 (2603:10b6:a03:540::10)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ2PR12MB8690:EE_|DM3PR12MB9326:EE_
X-MS-Office365-Filtering-Correlation-Id: 395f87f5-cca7-4c24-a5da-08dc344e91a9
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	/3LotZfBoyVYR1wtDhmZ0W4rlX75e3wOPkYMw2vORoUA87C5FScd8tXcrSKjzu81PdXtpptw0Lyh0O/KbOYBCyFtUhjFG/8XEtgys8c1Ryp09PQI6nfu9QTVDWjeNJt6CVqJSZdaqg9OAgnCmX7gpFEIwJOIh4yardah0GahYzKQgV5hcmzPfLV0mT9mLh9j+NR2jvP2fXaWFvgVIb6mVcIi0o4gJCgbo2pIu3jxvhCqFGOKcBhBYJ8MUMlg29GKJbVaQBeFLNzDHfJdpcQ/wzCgYHz36FPOoqgU5Gwno8MqKRhRkX973/cZkGtqEusXqpc8X/lcALcNbNcmeBfEkzCJB0GfoZbDsa1qnGFgitdKT8kAua9cCx3KGPZ4SEYteFZ6EyJB8zYpwRVx35+AlCSfxGwFFWJ4J8rAf/pB//fT9WgyhyrlNMu+HqWT/5hdVgpTx1arTArD246ZIot+ghp8t1Yso5aKeRQd1pc/K0iKBKKSretQZ28yjbuQGq1O+6MzlJ7oW9MMGSKygu8Xe4b51K3tKpouzhkyKdhbuNA=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ2PR12MB8690.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NWpoYW40WUtNeEwydEZwL1BuSDI3dDNEVnNOS2pacEd0Rmx2SWI0bm1YZndD?=
 =?utf-8?B?ZnZLWWFGMlNPc2xoYVVFY1JjWTdjRUR3UWY2RFdyVlI0RGpIaXd3N3N0dC91?=
 =?utf-8?B?N3h5ajdENVlKNmFoTys3cVdkdUdodFVLYWR0RzJvaTlla0tLMVFyVTMyNThk?=
 =?utf-8?B?OXg4bHZybWUvWFdBbTVFTTJ4NGRrbi9lMk1LZS9UdFZ1RU1YYnAxckIwZGhs?=
 =?utf-8?B?NHRONE9SOWFWVHhjTEoxcEk1bUtWMFpzK05UTTh4SWRMQjl5STRNaURUTjAy?=
 =?utf-8?B?QUtRK0lvVFVSUnZTVk9LcC9Ka24rVENwV0VLVUxreDEyYnlENUROTTg0MDk0?=
 =?utf-8?B?UHNJZXBxY2ZhRncwR01aeVB0RnhDR3B3U0x4eVJOMVVWeGZ6ZVlOQTNtRkU2?=
 =?utf-8?B?UjdLY0R4K0hGME1WcHJJa2pRbWtYczRnbmRocmZSRW51bWJKT1FNYnl6M1d3?=
 =?utf-8?B?M3M1T3lsMTlueWZTa25FSlJTZkJ4Y0puSytTNXpaSnRsZUdXa21keEhOQUR3?=
 =?utf-8?B?VFQzNTltNVlnNmxDVFNYNS90Ym5lQVVoM0RNRTZFVFc0WXpWbjZ4a1VZYlBZ?=
 =?utf-8?B?eXFWamtMQ09HM1FZbldGU0I4V1Z5YUhxUkY1OWdyVVd5a1RFNGxXa21ycTNa?=
 =?utf-8?B?cm4vbXl1aUc3eHRzSVJOblNFNWxUSXBIZlh4TVpTdVRhbUIrRGg0UzkwNHh0?=
 =?utf-8?B?S29lUUMrVTQyRnpLOUk2Z3NURkJUZHJvNVhoQmUwTXRHZVdleTUraFZEenZK?=
 =?utf-8?B?QUd5MFY5dklOWXl0czgwYjRER09MbnNnS0t3ZmIxcmswbEExWVhuSC9ya2JD?=
 =?utf-8?B?WEtxL1hvZ2g3TVVwdnJUNEx1dmRjeWlIeTc3K2ZnQjJ0WmVVQ0c1ZURKSWJ0?=
 =?utf-8?B?K0dOcEQ5R01XREtDcHVyL2swWHZudUR0SXdUNFB6czVLbzg2ZVpNRGhzczhL?=
 =?utf-8?B?Q2lra1ZKVVZEVm5NTWs1WkwrWVJMSXFUSHVvNFAvQlB0MkNSRFFyUC9MOGdD?=
 =?utf-8?B?dFpIMXVoazUxdk5BQzZYaHI1T0ZWNUJHM1YvRUw1WW5xVHJUTGdyR01UZzhN?=
 =?utf-8?B?aWtvNldVWHFpSDB5MVozL1E5Zm82aHpyZjZqdnAwdnBkNEppNHBQZHBxMXpo?=
 =?utf-8?B?WGM0TzIvZXdkcDRsZWVZbjRNUHpmQk9GL0F1ckhKdk1lUzJvOEwyeUtHeUEr?=
 =?utf-8?B?U2JGUzUxS09GKzZ6SFNxdXFhbTZtZitWQ3lzRnMyUXJKYlBVNU9PL3pUTHZI?=
 =?utf-8?B?RGpwSWFiLzNBeHRSVVI5WFF5ZCtIamNQUlVUbVpTaFZNMHFaTGNMR2hBYitN?=
 =?utf-8?B?eHMzWEY3RHpCRW9vRVBKTXJaMHNpT2xqODBMMmVmK1o5OTRXRUEwV2pRZXNT?=
 =?utf-8?B?aTd0bEg1NWpON1BwalR1VHBFRGt5S1FLeGl6MnRrenYwVTRBTEJldVYwUm5T?=
 =?utf-8?B?ci8xd3VOWDVrKytkanU1OWs2b056QUJqcUY5bzVOWHk5d09tOUoreDA4WEF5?=
 =?utf-8?B?cS9WNWYwVWFKZTZWbnJBSDg4Z0lDM2dTYU9BaUFyaFNqZkRmUXdVYTB1QUQ2?=
 =?utf-8?B?NFBIYUsvNlRjQTlpcE81RW5MeldodmVLblJPM29iQzQ4N2dCVTB6QSs1L1hR?=
 =?utf-8?B?TnQzaC9yUDZkMTdNa1NjQXNIU0duaTJ0VG1vQ3FVbmlLM0MrT3RaVm0reVJh?=
 =?utf-8?B?MnRTYVFzWjRQTHQyU3dLT2dVMk9YWDJVbU44NmU5VnorUzNvNGVKKzNBam95?=
 =?utf-8?B?RDhiL2RHSk02SmJyeFJXWUJtaldTdGpIWEkrd2tWdDdhVmxwWHRKRnpxelB5?=
 =?utf-8?B?cVdiVW1hU3JNTHJsTUJsWW44dFJIREV5VVZwUlJsU21pbXRjQzFUcmFST1VH?=
 =?utf-8?B?UHM3U0puVTRnK3pPMzZiN1NLR0lIN3Nobm8zZ2FJSXJISXNVWDNGejZFTldU?=
 =?utf-8?B?K0t2a3c3cDFVaUViTHk4dlZrSUNTM2xrc3Z5U1dqZmpJVDNncXRSand2WFNZ?=
 =?utf-8?B?NVdNQ2ZWcGQxMnBZR3RLUzJRMzVGV2svZkxhMy81bmdKdlhJWHZDak03Skly?=
 =?utf-8?B?aXpQUEhKQktzdHdHaklGMUwzZDBUY1Q0Ly9GTklkVFZmcXZwSGNMbEpramM2?=
 =?utf-8?Q?cbg83vUmCyxtssz9EWgcHAM1/?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 395f87f5-cca7-4c24-a5da-08dc344e91a9
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR12MB8690.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2024 09:05:23.8932
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eSgAaWTo3cx1Rhy5Q0Hjpv46UEc+mtPmmNAKHKE2rRC8CG0c02eZ11U0IrVxApUnC4gOwL/bghX6KXZSJWlpow==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9326

On Tue, Jan 02, 2024 at 08:49:54PM +0800, Marc-André Lureau wrote:
> Hi
> 
> On Tue, Dec 19, 2023 at 11:55 AM Huang Rui <ray.huang@amd.com> wrote:
> >
> > From: Antonio Caggiano <antonio.caggiano@collabora.com>
> >
> > Enable resource UUID feature and implement command resource assign UUID.
> > This is done by introducing a hash table to map resource IDs to their
> > UUIDs.
> 
> I agree with Akihiko, what about putting QemuUUID in struct
> virtio_gpu_simple_resource?

OK, I will add a member of UUID in simple resource structure.

> 
> (I also doubt about the hash table usefulness, but I don't know
> how/why the UUID is used)
> 

The system cannot be without this patch, let me figure it out the reason.

Thanks,
Ray

> >
> > Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
> > Signed-off-by: Huang Rui <ray.huang@amd.com>
> > ---
> >
> > Changes in v6:
> > - Set resource uuid as option.
> > - Implement optional subsection of vmstate_virtio_gpu_resource_uuid_state
> >   or virtio live migration.
> > - Use g_int_hash/g_int_equal instead of the default.
> > - Move virtio_vgpu_simple_resource initialization in the earlier new patch
> >   "virtio-gpu: Introduce virgl_gpu_resource structure"
> >
> >  hw/display/trace-events        |   1 +
> >  hw/display/virtio-gpu-base.c   |   4 ++
> >  hw/display/virtio-gpu-virgl.c  |   3 +
> >  hw/display/virtio-gpu.c        | 119 +++++++++++++++++++++++++++++++++
> >  include/hw/virtio/virtio-gpu.h |   7 ++
> >  5 files changed, 134 insertions(+)
> >
> > diff --git a/hw/display/trace-events b/hw/display/trace-events
> > index 2336a0ca15..54d6894c59 100644
> > --- a/hw/display/trace-events
> > +++ b/hw/display/trace-events
> > @@ -41,6 +41,7 @@ virtio_gpu_cmd_res_create_blob(uint32_t res, uint64_t size) "res 0x%x, size %" P
> >  virtio_gpu_cmd_res_unref(uint32_t res) "res 0x%x"
> >  virtio_gpu_cmd_res_back_attach(uint32_t res) "res 0x%x"
> >  virtio_gpu_cmd_res_back_detach(uint32_t res) "res 0x%x"
> > +virtio_gpu_cmd_res_assign_uuid(uint32_t res) "res 0x%x"
> >  virtio_gpu_cmd_res_xfer_toh_2d(uint32_t res) "res 0x%x"
> >  virtio_gpu_cmd_res_xfer_toh_3d(uint32_t res) "res 0x%x"
> >  virtio_gpu_cmd_res_xfer_fromh_3d(uint32_t res) "res 0x%x"
> > diff --git a/hw/display/virtio-gpu-base.c b/hw/display/virtio-gpu-base.c
> > index 37af256219..6bcee3882f 100644
> > --- a/hw/display/virtio-gpu-base.c
> > +++ b/hw/display/virtio-gpu-base.c
> > @@ -236,6 +236,10 @@ virtio_gpu_base_get_features(VirtIODevice *vdev, uint64_t features,
> >          features |= (1 << VIRTIO_GPU_F_CONTEXT_INIT);
> >      }
> >
> > +    if (virtio_gpu_resource_uuid_enabled(g->conf)) {
> > +        features |= (1 << VIRTIO_GPU_F_RESOURCE_UUID);
> > +    }
> > +
> >      return features;
> >  }
> >
> > diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c
> > index 5a3a292f79..be9da6e780 100644
> > --- a/hw/display/virtio-gpu-virgl.c
> > +++ b/hw/display/virtio-gpu-virgl.c
> > @@ -777,6 +777,9 @@ void virtio_gpu_virgl_process_cmd(VirtIOGPU *g,
> >          /* TODO add security */
> >          virgl_cmd_ctx_detach_resource(g, cmd);
> >          break;
> > +    case VIRTIO_GPU_CMD_RESOURCE_ASSIGN_UUID:
> > +        virtio_gpu_resource_assign_uuid(g, cmd);
> > +        break;
> >      case VIRTIO_GPU_CMD_GET_CAPSET_INFO:
> >          virgl_cmd_get_capset_info(g, cmd);
> >          break;
> > diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
> > index 8189c392dc..466debb256 100644
> > --- a/hw/display/virtio-gpu.c
> > +++ b/hw/display/virtio-gpu.c
> > @@ -958,6 +958,37 @@ virtio_gpu_resource_detach_backing(VirtIOGPU *g,
> >      virtio_gpu_cleanup_mapping(g, res);
> >  }
> >
> > +void virtio_gpu_resource_assign_uuid(VirtIOGPU *g,
> > +                                     struct virtio_gpu_ctrl_command *cmd)
> > +{
> > +    struct virtio_gpu_simple_resource *res;
> > +    struct virtio_gpu_resource_assign_uuid assign;
> > +    struct virtio_gpu_resp_resource_uuid resp;
> > +    QemuUUID *uuid;
> > +
> > +    VIRTIO_GPU_FILL_CMD(assign);
> > +    virtio_gpu_bswap_32(&assign, sizeof(assign));
> > +    trace_virtio_gpu_cmd_res_assign_uuid(assign.resource_id);
> > +
> > +    res = virtio_gpu_find_check_resource(g, assign.resource_id, false, __func__, &cmd->error);
> > +    if (!res) {
> > +        return;
> > +    }
> > +
> > +    memset(&resp, 0, sizeof(resp));
> > +    resp.hdr.type = VIRTIO_GPU_RESP_OK_RESOURCE_UUID;
> > +
> > +    uuid = g_hash_table_lookup(g->resource_uuids, &assign.resource_id);
> > +    if (!uuid) {
> > +        uuid = g_new(QemuUUID, 1);
> > +        qemu_uuid_generate(uuid);
> > +        g_hash_table_insert(g->resource_uuids, &assign.resource_id, uuid);
> > +    }
> > +
> > +    memcpy(resp.uuid, uuid, sizeof(QemuUUID));
> > +    virtio_gpu_ctrl_response(g, cmd, &resp.hdr, sizeof(resp));
> > +}
> > +
> >  void virtio_gpu_simple_process_cmd(VirtIOGPU *g,
> >                                     struct virtio_gpu_ctrl_command *cmd)
> >  {
> > @@ -1006,6 +1037,9 @@ void virtio_gpu_simple_process_cmd(VirtIOGPU *g,
> >      case VIRTIO_GPU_CMD_RESOURCE_DETACH_BACKING:
> >          virtio_gpu_resource_detach_backing(g, cmd);
> >          break;
> > +    case VIRTIO_GPU_CMD_RESOURCE_ASSIGN_UUID:
> > +        virtio_gpu_resource_assign_uuid(g, cmd);
> > +        break;
> >      default:
> >          cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC;
> >          break;
> > @@ -1400,6 +1434,57 @@ static int virtio_gpu_blob_load(QEMUFile *f, void *opaque, size_t size,
> >      return 0;
> >  }
> >
> > +static int virtio_gpu_resource_uuid_save(QEMUFile *f, void *opaque, size_t size,
> > +                                         const VMStateField *field,
> > +                                         JSONWriter *vmdesc)
> > +{
> > +    VirtIOGPU *g = opaque;
> > +    struct virtio_gpu_simple_resource *res;
> > +    QemuUUID *uuid;
> > +
> > +    /* in 2d mode we should never find unprocessed commands here */
> > +    assert(QTAILQ_EMPTY(&g->cmdq));
> > +
> > +    QTAILQ_FOREACH(res, &g->reslist, next) {
> > +        qemu_put_be32(f, res->resource_id);
> > +        uuid = g_hash_table_lookup(g->resource_uuids, &res->resource_id);
> > +        qemu_put_buffer(f, (void *)uuid, sizeof(QemuUUID));
> > +    }
> > +    qemu_put_be32(f, 0); /* end of list */
> > +
> > +    g_hash_table_destroy(g->resource_uuids);
> > +
> > +    return 0;
> > +}
> > +
> > +static int virtio_gpu_resource_uuid_load(QEMUFile *f, void *opaque, size_t size,
> > +                                         const VMStateField *field)
> > +{
> > +    VirtIOGPU *g = opaque;
> > +    struct virtio_gpu_simple_resource *res;
> > +    uint32_t resource_id;
> > +    QemuUUID *uuid = NULL;
> > +
> > +    g->resource_uuids = g_hash_table_new_full(g_int_hash, g_int_equal, NULL, g_free);
> > +    resource_id = qemu_get_be32(f);
> > +    while (resource_id != 0) {
> > +        res = virtio_gpu_find_resource(g, resource_id);
> > +        if (res) {
> > +            return -EINVAL;
> > +        }
> > +
> > +        res = g_new0(struct virtio_gpu_simple_resource, 1);
> > +        res->resource_id = resource_id;
> > +
> > +        qemu_get_buffer(f, (void *)uuid, sizeof(QemuUUID));
> > +        g_hash_table_insert(g->resource_uuids, &res->resource_id, uuid);
> > +
> > +        resource_id = qemu_get_be32(f);
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> >  static int virtio_gpu_post_load(void *opaque, int version_id)
> >  {
> >      VirtIOGPU *g = opaque;
> > @@ -1475,12 +1560,15 @@ void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
> >      QTAILQ_INIT(&g->reslist);
> >      QTAILQ_INIT(&g->cmdq);
> >      QTAILQ_INIT(&g->fenceq);
> > +
> > +    g->resource_uuids = g_hash_table_new_full(g_int_hash, g_int_equal, NULL, g_free);
> >  }
> >
> >  static void virtio_gpu_device_unrealize(DeviceState *qdev)
> >  {
> >      VirtIOGPU *g = VIRTIO_GPU(qdev);
> >
> > +    g_hash_table_destroy(g->resource_uuids);
> 
> better:
> g_clear_pointer(&g->resource_uuids, g_hash_table_unref);
> 
> >      g_clear_pointer(&g->ctrl_bh, qemu_bh_delete);
> >      g_clear_pointer(&g->cursor_bh, qemu_bh_delete);
> >      g_clear_pointer(&g->reset_bh, qemu_bh_delete);
> > @@ -1534,6 +1622,8 @@ void virtio_gpu_reset(VirtIODevice *vdev)
> >          g_free(cmd);
> >      }
> >
> > +    g_hash_table_remove_all(g->resource_uuids);
> > +
> >      virtio_gpu_base_reset(VIRTIO_GPU_BASE(vdev));
> >  }
> >
> > @@ -1583,6 +1673,32 @@ const VMStateDescription vmstate_virtio_gpu_blob_state = {
> >      },
> >  };
> >
> > +static bool virtio_gpu_resource_uuid_state_needed(void *opaque)
> > +{
> > +    VirtIOGPU *g = VIRTIO_GPU(opaque);
> > +
> > +    return virtio_gpu_resource_uuid_enabled(g->parent_obj.conf);
> > +}
> > +
> > +const VMStateDescription vmstate_virtio_gpu_resource_uuid_state = {
> > +    .name = "virtio-gpu/resource_uuid",
> > +    .minimum_version_id = VIRTIO_GPU_VM_VERSION,
> > +    .version_id = VIRTIO_GPU_VM_VERSION,
> > +    .needed = virtio_gpu_resource_uuid_state_needed,
> > +    .fields = (const VMStateField[]){
> > +        {
> > +            .name = "virtio-gpu/resource_uuid",
> > +            .info = &(const VMStateInfo) {
> > +                .name = "resource_uuid",
> > +                .get = virtio_gpu_resource_uuid_load,
> > +                .put = virtio_gpu_resource_uuid_save,
> > +            },
> > +            .flags = VMS_SINGLE,
> > +        } /* device */,
> > +        VMSTATE_END_OF_LIST()
> > +    },
> > +};
> > +
> >  /*
> >   * For historical reasons virtio_gpu does not adhere to virtio migration
> >   * scheme as described in doc/virtio-migration.txt, in a sense that no
> > @@ -1610,6 +1726,7 @@ static const VMStateDescription vmstate_virtio_gpu = {
> >      },
> >      .subsections = (const VMStateDescription * []) {
> >          &vmstate_virtio_gpu_blob_state,
> > +        &vmstate_virtio_gpu_resource_uuid_state,
> >          NULL
> >      },
> >      .post_load = virtio_gpu_post_load,
> > @@ -1622,6 +1739,8 @@ static Property virtio_gpu_properties[] = {
> >      DEFINE_PROP_BIT("blob", VirtIOGPU, parent_obj.conf.flags,
> >                      VIRTIO_GPU_FLAG_BLOB_ENABLED, false),
> >      DEFINE_PROP_SIZE("hostmem", VirtIOGPU, parent_obj.conf.hostmem, 0),
> > +    DEFINE_PROP_BIT("resource_uuid", VirtIOGPU, parent_obj.conf.flags,
> > +                    VIRTIO_GPU_FLAG_RESOURCE_UUID_ENABLED, false),
> 
> why not enable it by default? (and set it to false for machine < 9.0
> 
> >  #ifdef HAVE_VIRGL_CONTEXT_CREATE_WITH_FLAGS
> >      DEFINE_PROP_BIT("context_init", VirtIOGPU, parent_obj.conf.flags,
> >                      VIRTIO_GPU_FLAG_CONTEXT_INIT_ENABLED, true),
> > diff --git a/include/hw/virtio/virtio-gpu.h b/include/hw/virtio/virtio-gpu.h
> > index 584ba2ed73..76b410fe91 100644
> > --- a/include/hw/virtio/virtio-gpu.h
> > +++ b/include/hw/virtio/virtio-gpu.h
> > @@ -98,6 +98,7 @@ enum virtio_gpu_base_conf_flags {
> >      VIRTIO_GPU_FLAG_BLOB_ENABLED,
> >      VIRTIO_GPU_FLAG_CONTEXT_INIT_ENABLED,
> >      VIRTIO_GPU_FLAG_RUTABAGA_ENABLED,
> > +    VIRTIO_GPU_FLAG_RESOURCE_UUID_ENABLED,
> >  };
> >
> >  #define virtio_gpu_virgl_enabled(_cfg) \
> > @@ -114,6 +115,8 @@ enum virtio_gpu_base_conf_flags {
> >      (_cfg.flags & (1 << VIRTIO_GPU_FLAG_CONTEXT_INIT_ENABLED))
> >  #define virtio_gpu_rutabaga_enabled(_cfg) \
> >      (_cfg.flags & (1 << VIRTIO_GPU_FLAG_RUTABAGA_ENABLED))
> > +#define virtio_gpu_resource_uuid_enabled(_cfg) \
> > +    (_cfg.flags & (1 << VIRTIO_GPU_FLAG_RESOURCE_UUID_ENABLED))
> >  #define virtio_gpu_hostmem_enabled(_cfg) \
> >      (_cfg.hostmem > 0)
> >
> > @@ -209,6 +212,8 @@ struct VirtIOGPU {
> >          QTAILQ_HEAD(, VGPUDMABuf) bufs;
> >          VGPUDMABuf *primary[VIRTIO_GPU_MAX_SCANOUTS];
> >      } dmabuf;
> > +
> > +    GHashTable *resource_uuids;
> >  };
> >
> >  struct VirtIOGPUClass {
> > @@ -307,6 +312,8 @@ void virtio_gpu_cleanup_mapping_iov(VirtIOGPU *g,
> >                                      struct iovec *iov, uint32_t count);
> >  void virtio_gpu_cleanup_mapping(VirtIOGPU *g,
> >                                  struct virtio_gpu_simple_resource *res);
> > +void virtio_gpu_resource_assign_uuid(VirtIOGPU *g,
> > +                                     struct virtio_gpu_ctrl_command *cmd);
> >  void virtio_gpu_process_cmdq(VirtIOGPU *g);
> >  void virtio_gpu_device_realize(DeviceState *qdev, Error **errp);
> >  void virtio_gpu_reset(VirtIODevice *vdev);
> > --
> > 2.25.1
> >
> 
> 
> -- 
> Marc-André Lureau


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:07:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:07:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684683.1064681 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRWX-00084C-Jn; Fri, 23 Feb 2024 09:07:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684683.1064681; Fri, 23 Feb 2024 09:07:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRWX-000845-Gb; Fri, 23 Feb 2024 09:07:17 +0000
Received: by outflank-mailman (input) for mailman id 684683;
 Fri, 23 Feb 2024 09:07:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hXte=KA=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rdRWV-00083u-Sw
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:07:15 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ef0b37b7-d22a-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 10:07:13 +0100 (CET)
Received: from Dell.bugsneg.com (unknown [37.161.242.66])
 by support.bugseng.com (Postfix) with ESMTPSA id 5290C4EE073F;
 Fri, 23 Feb 2024 10:07:12 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ef0b37b7-d22a-11ee-98f5-efadbce2ee36
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH 0/2] automation/eclair: update deviations and tags
Date: Fri, 23 Feb 2024 10:06:58 +0100
Message-Id: <cover.1708677243.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Deviate Rules 5.7 and 18.7.

Tag Rule 8.2 ad clean.

Federico Serafini (2):
  automation/eclair: fully deviate MISRA C:2012 Rules 5.7 and 18.7
  automation/eclair: tag MISRA C:2012 Rule 8.2 as clean

 automation/eclair_analysis/ECLAIR/deviations.ecl | 14 ++++++++++++++
 automation/eclair_analysis/ECLAIR/tagging.ecl    |  2 +-
 2 files changed, 15 insertions(+), 1 deletion(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:07:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:07:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684684.1064687 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRWX-00086s-Th; Fri, 23 Feb 2024 09:07:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684684.1064687; Fri, 23 Feb 2024 09:07:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRWX-00086c-OX; Fri, 23 Feb 2024 09:07:17 +0000
Received: by outflank-mailman (input) for mailman id 684684;
 Fri, 23 Feb 2024 09:07:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hXte=KA=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rdRWW-00083u-In
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:07:16 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ef75a4a9-d22a-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 10:07:14 +0100 (CET)
Received: from Dell.bugsneg.com (unknown [37.161.242.66])
 by support.bugseng.com (Postfix) with ESMTPSA id 9416F4EE0741;
 Fri, 23 Feb 2024 10:07:13 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ef75a4a9-d22a-11ee-98f5-efadbce2ee36
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH 1/2] automation/eclair: fully deviate MISRA C:2012 Rules 5.7 and 18.7
Date: Fri, 23 Feb 2024 10:06:59 +0100
Message-Id: <1fee04ad07b03c13c44f66a20c4ee70737069650.1708677243.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1708677243.git.federico.serafini@bugseng.com>
References: <cover.1708677243.git.federico.serafini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update ECLAIR configuration to fully deviate Rules 5.7 and 18.7
as agreed during MISRA meeetings.

Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
---
 automation/eclair_analysis/ECLAIR/deviations.ecl | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
index fd32ff8a9c..02eae39786 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -107,6 +107,11 @@ number of guest paging levels."
 -config=MC3R1.R5.6,reports+={deliberate,"any_area(any_loc(file(adopted_r5_6)))"}
 -doc_end
 
+-doc_begin="The project intentionally reuses tag names in order to have identifiers matching the applicable external specifications as well as established internal conventions.
+As there is little possibility for developer confusion not resulting into compilation errors, the risk of renaming outweighs the potential advantages of compliance."
+-config=MC3R1.R5.7,reports+={deliberate,"any()"}
+-doc_end
+
 #
 # Series 7.
 #
@@ -373,6 +378,15 @@ explicit comment indicating the fallthrough intention is present."
 -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(^(?s).*/\\* [fF]all ?through.? \\*/.*$,0..1))))"}
 -doc_end
 
+#
+# Series 18.
+#
+
+-doc_begin="Flexible array members are deliberately used and XEN developers are aware of the dangers related to them:
+unexpected result when the structure is given as argument to a sizeof() operator and the truncation in assignment between structures."
+-config=MC3R1.R18.7,reports+={deliberate, "any()"}
+-doc_end
+
 #
 # Series 20.
 #
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:07:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:07:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684685.1064702 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRWZ-000061-3X; Fri, 23 Feb 2024 09:07:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684685.1064702; Fri, 23 Feb 2024 09:07:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRWY-00005s-Vg; Fri, 23 Feb 2024 09:07:18 +0000
Received: by outflank-mailman (input) for mailman id 684685;
 Fri, 23 Feb 2024 09:07:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hXte=KA=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rdRWX-00083u-Im
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:07:17 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f03a5837-d22a-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 10:07:15 +0100 (CET)
Received: from Dell.bugsneg.com (unknown [37.161.242.66])
 by support.bugseng.com (Postfix) with ESMTPSA id C26024EE0C8A;
 Fri, 23 Feb 2024 10:07:14 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f03a5837-d22a-11ee-98f5-efadbce2ee36
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [XEN PATCH 2/2] automation/eclair: tag MISRA C:2012 Rule 8.2 as clean
Date: Fri, 23 Feb 2024 10:07:00 +0100
Message-Id: <59db8ce679a2ab46b23b39e8f1591cd9fc1c9218.1708677243.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1708677243.git.federico.serafini@bugseng.com>
References: <cover.1708677243.git.federico.serafini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update ECLAIR configuration to consider Rule 8.2 as clean.

Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
---
 automation/eclair_analysis/ECLAIR/tagging.ecl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
index 900c532196..a1dea32b21 100644
--- a/automation/eclair_analysis/ECLAIR/tagging.ecl
+++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
@@ -30,7 +30,7 @@
 
 -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
 
--service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R2.2||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R8.1||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R9.2||MC3R1.R9.4||MC3R1.R9.5||MC3R1.R12.5||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6"
+-service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R2.2||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R8.1||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R9.2||MC3R1.R9.4||MC3R1.R9.5||MC3R1.R12.5||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6"
 }
 
 -setq=target,getenv("XEN_TARGET_ARCH")
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:16:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:16:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684694.1064712 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRfD-0002Ru-2b; Fri, 23 Feb 2024 09:16:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684694.1064712; Fri, 23 Feb 2024 09:16:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRfC-0002Rn-VX; Fri, 23 Feb 2024 09:16:14 +0000
Received: by outflank-mailman (input) for mailman id 684694;
 Fri, 23 Feb 2024 09:16:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=oqZf=KA=amd.com=Ray.Huang@srs-se1.protection.inumbo.net>)
 id 1rdRfC-0002Rh-Bx
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:16:14 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20609.outbound.protection.outlook.com
 [2a01:111:f400:7e88::609])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2f59b811-d22c-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 10:16:12 +0100 (CET)
Received: from SJ2PR12MB8690.namprd12.prod.outlook.com (2603:10b6:a03:540::10)
 by PH7PR12MB7965.namprd12.prod.outlook.com (2603:10b6:510:270::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.23; Fri, 23 Feb
 2024 09:16:07 +0000
Received: from SJ2PR12MB8690.namprd12.prod.outlook.com
 ([fe80::e7a:5022:4b7d:ade1]) by SJ2PR12MB8690.namprd12.prod.outlook.com
 ([fe80::e7a:5022:4b7d:ade1%6]) with mapi id 15.20.7316.023; Fri, 23 Feb 2024
 09:16:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f59b811-d22c-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=naU1w/LOtv2UFE9wqpEegdkxZCmy4GjGOn+E4gHxTLvgVCWa1vko7u7HJcAnkL7/nnd5766U+0xuOLSPb86I1YSu41+uamk1i6XhO+H15XQy+PSHljZ8YodDt1+NslYVrzwfWtxT5d9tZ0BwNeVBHXz93JkhTBEawiYMwT7ckGMvZ62vefs7+Ix755KB3r25KmEm+LEMavegnD1I0DxfjkfcWS+sF9Z2PnR3JTxH9fO0g3YdUjJ/SBhPh7WOotEnL9SFa6Cy3lRW2ngFzBj0Jjte5Z//canJTylAJV1tSjSxbMs8wP35UxfXkqBd7Go2GGQBsnw5Iu/aBpubdbER2w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=W0+q/t21RIbG/uUyv0SqG41fSql+NgSoPURFOny8XXs=;
 b=dtms5A4mVOhqIbrgc5IVWFRyQIV5k9dAv4n17mY6nZeE9ltpkDSz7MTEYrIxNzi17dI9n8XntdjV2XS3MiQdp0jjeB/KBGnfLE/wI6o3Ym677Pif6P9pO1oVfGdt8bRW+N3cLfzRfTB6YsMSXSluHdQxCg2AZPz93FxQdpFoq58g/6lbSSkwHoVVAzvPNXtwtp9qyepMnK3w0VYLaHJ7ynN70bJW4rD3iIh8rR0WwxA27sZuvjWxBiSbtUPEhZR2Gk68+xFKfJsZ0qwfu5JTGLKLfjRhvQJ5o+zF0Hu7VYdbVX6iYUZwYw6tYLDfgrQPHv9IjrWrUreJewfiJZB8sw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=W0+q/t21RIbG/uUyv0SqG41fSql+NgSoPURFOny8XXs=;
 b=EsHBZc0Rr4QVBUpUHLVF4CRUhH83yls80a4Or8fqhxsLDai2aiTtg6jtWi3yXHshVVmOQzWntMvtFK6EVCf19h4tH2iMh+5yQjWvajVYXa6CnbvtWq0iD8bkiElVOASGhTWegLTDM01jPo9BAx4OHh5xHxrhJH6xRORZl4lq8Q4=
Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
Date: Fri, 23 Feb 2024 17:15:39 +0800
From: Huang Rui <ray.huang@amd.com>
To: =?iso-8859-1?Q?Marc-Andr=E9?= Lureau <marcandre.lureau@gmail.com>
Cc: Akihiko Odaki <akihiko.odaki@daynix.com>,
	Philippe =?iso-8859-1?Q?Mathieu-Daud=E9?= <philmd@linaro.org>,
	Gerd Hoffmann <kraxel@redhat.com>,
	"Michael S . Tsirkin" <mst@redhat.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Antonio Caggiano <quic_acaggian@quicinc.com>,
	"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
	Robert Beckett <bob.beckett@collabora.com>,
	Dmitry Osipenko <dmitry.osipenko@collabora.com>,
	Gert Wollny <gert.wollny@collabora.com>,
	Alex =?iso-8859-1?Q?Benn=E9e?= <alex.bennee@linaro.org>,
	"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Gurchetan Singh <gurchetansingh@chromium.org>,
	"ernunes@redhat.com" <ernunes@redhat.com>,
	Alyssa Ross <hi@alyssa.is>,
	Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
	"Deucher, Alexander" <Alexander.Deucher@amd.com>,
	"Stabellini, Stefano" <stefano.stabellini@amd.com>,
	"Koenig, Christian" <Christian.Koenig@amd.com>,
	"Ragiadakou, Xenia" <Xenia.Ragiadakou@amd.com>,
	"Pelloux-Prayer, Pierre-Eric" <Pierre-eric.Pelloux-prayer@amd.com>,
	"Huang, Honglei1" <Honglei1.Huang@amd.com>,
	"Zhang, Julia" <Julia.Zhang@amd.com>,
	"Chen, Jiqian" <Jiqian.Chen@amd.com>,
	Antonio Caggiano <antonio.caggiano@collabora.com>
Subject: Re: [PATCH v6 10/11] virtio-gpu: Initialize Venus
Message-ID: <ZdhiO0ZEZS9lmAeo@amd.com>
References: <20231219075320.165227-1-ray.huang@amd.com>
 <20231219075320.165227-11-ray.huang@amd.com>
 <CAJ+F1C+gQ7kwAD=Tv7uAgKtgA57wPkZcE_zadAogtMUEFUmFnQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAJ+F1C+gQ7kwAD=Tv7uAgKtgA57wPkZcE_zadAogtMUEFUmFnQ@mail.gmail.com>
X-ClientProxiedBy: TYCPR01CA0008.jpnprd01.prod.outlook.com (2603:1096:405::20)
 To MW6PR12MB8706.namprd12.prod.outlook.com (2603:10b6:303:249::5)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ2PR12MB8690:EE_|PH7PR12MB7965:EE_
X-MS-Office365-Filtering-Correlation-Id: b6889da4-d0ed-4de0-a4c3-08dc345010f7
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	t7mbcuid9WlPnW7brOrlUs3lPfeKGAnixyeM1EHfgESV/c7Yhi8HHgTLuQXKwshb0t6PFdNqqS52DphgBq8hj0gZLbJD/aaXEYzJgzt1sJyDsoCGr946ZmGv7SZneW/zo3p/A7WujYcRz/h/70+r9oy65Gz5ikkXDGY5MgqSRLFW5B/982Y0BGXyAZlOXJRCktv0/9WXyVrIh0Za2qWbtC72I/89+6VnauKzy5QeqII3f+LB/hFmVDTTrPJS2xgXdUlBe5zpHp9yYCZMcdwu11XC4zmZRrUZQG5oVrLXbtO3/yQ35bAkQtuPOaI63gD8/6Aq2HoIeTHfAE4wziQaaROIEJVkPjYlJx7yKTCAzhMuA9Y8sXZMbM+GkAZG5j8DJAKI6osTM8ET7z7kWbzjw8ELFzC/SXa/kBz5B/56bHcj+6V3mbYQZ7ph4de6hsdvErnQsnoCTlURXXvGZ6v6lnhCPeXMrEdyz8Rt6qRYtjuu1q3SQASndwTK0cmc/QvWgwnlUSjah8ftjlqyW3gg1eCxKfuf1o8ntkKJ1yseH9U=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ2PR12MB8690.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?c1pJUExlR1lOMnJhK3Z5YjVMWXdxYnlzTzVQVEdObHdleXRZN0tnZFFQb1Vo?=
 =?utf-8?B?ZHJRSEpENDQ2b2grU0JYQUZ5VEErekpsbDUxbVNTdCtQRGRJb2lCcTZncldW?=
 =?utf-8?B?UDVacnZvYnBtVkRScU8zL0QvZWlPdjZ4ZzlSMVF2MG94NFFudXYwWXdDUGdq?=
 =?utf-8?B?WkVacFhkbU4vdW5FV1Z4MUdSaHVTaHJjSGl2S3cvb3RLK2JJTm5WZXgzV3pY?=
 =?utf-8?B?V1BLeHhQK0M4RXdDL3BzS0dGa2FySU9yVVlBbVhLc0t4cUE2VGRwbGRGZFhP?=
 =?utf-8?B?aUsxSlBYamdUU25XQUFLM2ZSeDVRWklLN3RHU2t5NWp4NjdMNjdjQkFpemVw?=
 =?utf-8?B?YVdJSjhXSE4rQnF4ai8zZ2lvMnJaSFNaK1RYeEs5NUtVU0g1M2JzeUpRVktk?=
 =?utf-8?B?RmUvZ3V0SkJTNmxBU0hYSThKYTIwaGllWWFScEpWUExyWDZhUHJqZTk2U0o3?=
 =?utf-8?B?eTlSQlA5NmlWb20zdlpLV05qUVdEa2FJbUY3dUNzOHZaeVdpTWE2S2Q4NTYy?=
 =?utf-8?B?U0NYa3JHTlM2MFpUQi9WbFcydHRFSlBIaTNlYm5LL3ZNTDVRMDdrTkkrK1Bj?=
 =?utf-8?B?VUFDOGFxanJyT0R5YVFaQjVMSkdnbnprKzd6OGEySWFlaXAyeVI5SDN4UzZs?=
 =?utf-8?B?MzdtRUtJeVlqTGppdGZTUDdSdXZOblgrZnZvamU3STA4czdzc3kwVzVhd1Qv?=
 =?utf-8?B?ZHdmWlYrUTdoQjIwRjgvYWFJK3lZdk52Y2hQUmJBbndQcmlQbDF5MHh2azBk?=
 =?utf-8?B?M0pqL0xuOEYvQ09DVGp1azc4RW5RbjNiYlFnNkFWN3EwM2p6Y1NyaUVMNDlM?=
 =?utf-8?B?bDg0YVEwdDkrWHRsU2NTbE9YNFpjTXBweG1oR0JrdnRpaCtOQm1EbHBMcUFC?=
 =?utf-8?B?Y0NBay9rdG1EN2Vuem93aDRKRm5hcmtBY0t3YTFabk5PTHVzc0ora2pBZFFX?=
 =?utf-8?B?MUUvUWh6VXJJVjhNMWRVZ3c2N3N2cmJiQzBFejJZQzJDRlpVNy9nN0xuaFpV?=
 =?utf-8?B?RVg1ekdYSmRRSjNkNC9SQm1wTnRCZFd0ZHdtc0I0ZSt4cVlkcTVhNnFQMHIr?=
 =?utf-8?B?NEpYR0tFUjZwRmZpb252YTQ2UGVVdWYzQ0FiSER0SEpGY29vc1l2U1JDMUZV?=
 =?utf-8?B?N2lyL3VJSmxYZHIwbUZqalF3NldaazVlajFLRmxlZFF1YmJOK1dpMS9mV0JB?=
 =?utf-8?B?YnIxYXBtd0dYZkJHeTI5ZFN4aDhTTEQ1cWFTWmFUT3hxcHRZZUNyMUpwWHBj?=
 =?utf-8?B?cVhkZHdTUlJlWFMrZGx4aHZVbUMzTHpFRVdGRHhZdEJ6QW1nTWF1djJBWEwx?=
 =?utf-8?B?eDJNblFtSU1KQzNqaDJxZ0pTODV0ZUE2ZWo4N293Z21aZ2IxcS9TaXZ1eVJM?=
 =?utf-8?B?WVFONy9aMFFrQWRBR1diN3hBcjRHclh5RkhjVDJqcG1HOTV5QVlSREZVVW1k?=
 =?utf-8?B?amh6S2pRc2p4cTdEb2RLS3BmdW1ZTE9hQU9RODlRU1VaMFpQRi9OdW1JcGQy?=
 =?utf-8?B?eFdVSVk3Y3REQjNvNXhJd3lUYUJLRldqVTdzVDMzemppS000WWZjTEF2b05C?=
 =?utf-8?B?WU9qZnlQUmhBT0xlU2Q2d2NUZzlJSy95Ylc4NG4yVXdQY3ZNNUZjZEErdGN3?=
 =?utf-8?B?eEF5OWpnWDZ4MUhZS0dnbWl5cHA1UXViUFlwUG9pRTVrNkE1bjBuOG9UbnlL?=
 =?utf-8?B?MjgwVnViU1kvR2NPQmZZb1c4MFVCMCswdnlqVkdkeUZOV1cwSEt1SGF2bExQ?=
 =?utf-8?B?UWJUTGxGNUVTL1gybjRJaVR0bmFsejFDUWM2YTJmWHUzdmxFVXFCN04yZjhq?=
 =?utf-8?B?d1ROOUhJMmNLV2tZWFJJNVBKRUZDRVZZazNuU1ZmMUxzbkFYTm5CN0huZ0cr?=
 =?utf-8?B?bjZjbEhMWC92eGp0dytBT1FLSmp0THRoWm1HczlNZnoyZm50YWl6aFJsand0?=
 =?utf-8?B?TndjNTYyd1d4WS9WRFJ1MFlDOFBOTWRLcHIxSDkza1ArWFNJbXQ5OGYreFI3?=
 =?utf-8?B?Z0tVdE1XUEZwQUNqTTgxUG42QmZWRXdXVUg0SFNGZGt0cUd2K3ovaTJoY3Js?=
 =?utf-8?B?QmNTd2I2R3JTNzNxcTFLVnhicmNtaTRPSWJ3Nzg2Q2RFMUs4Zy9zQUhPS0to?=
 =?utf-8?Q?u8zWHl+vAP6Lz++Fu8h/LIbth?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b6889da4-d0ed-4de0-a4c3-08dc345010f7
X-MS-Exchange-CrossTenant-AuthSource: MW6PR12MB8706.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2024 09:16:07.5265
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kmsU6z2T6VGF017LkW8lo0OOVv+4BUBUoEn6a14yQNL4jbWby9ILjHN/YVfhyDPhzwgLUrVnoVu3M9A7bthctQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7965

On Tue, Jan 02, 2024 at 09:33:11PM +0800, Marc-André Lureau wrote:
> Hi
> 
> On Tue, Dec 19, 2023 at 11:55 AM Huang Rui <ray.huang@amd.com> wrote:
> >
> > From: Antonio Caggiano <antonio.caggiano@collabora.com>
> >
> > Request Venus when initializing VirGL.
> >
> > Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
> > Signed-off-by: Huang Rui <ray.huang@amd.com>
> > ---
> >
> > Changes in v6:
> > - Remove the unstable API flags check because virglrenderer is already 1.0.
> > - Squash the render server flag support into "Initialize Venus".
> >
> >  hw/display/virtio-gpu-virgl.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c
> > index f35a751824..c523a6717a 100644
> > --- a/hw/display/virtio-gpu-virgl.c
> > +++ b/hw/display/virtio-gpu-virgl.c
> > @@ -964,6 +964,10 @@ int virtio_gpu_virgl_init(VirtIOGPU *g)
> >      }
> >  #endif
> >
> > +#ifdef VIRGL_RENDERER_VENUS
> > +    flags |= VIRGL_RENDERER_VENUS | VIRGL_RENDERER_RENDER_SERVER;
> > +#endif
> > +
> 
> I wonder if it's a good idea to initialize venus by default. It
> doesn't seem to require vulkan during initialization, but this may
> evolve. Make it optional?
> 

I am fine. In fact, vulkan is widely used for graphic area such as gaming,
compute, VR/AR, etc.

Thanks,
Ray

> >      ret = virgl_renderer_init(g, flags, &virtio_gpu_3d_cbs);
> >      if (ret != 0) {
> >          error_report("virgl could not be initialized: %d", ret);
> > --
> > 2.25.1
> >
> 
> 
> -- 
> Marc-André Lureau


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:19:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:19:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684700.1064721 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRi7-00038W-GL; Fri, 23 Feb 2024 09:19:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684700.1064721; Fri, 23 Feb 2024 09:19:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRi7-00038P-Cz; Fri, 23 Feb 2024 09:19:15 +0000
Received: by outflank-mailman (input) for mailman id 684700;
 Fri, 23 Feb 2024 09:19:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdRi6-00035f-4P
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:19:14 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9b294b29-d22c-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 10:19:11 +0100 (CET)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-512cba0f953so738066e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 01:19:12 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 mu19-20020a056214329300b0068fbd69defesm1871516qvb.87.2024.02.23.01.19.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 01:19:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b294b29-d22c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708679951; x=1709284751; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=d7hQCuUachth0pW2nuQ1/VeIz0I8udEWdG6ieEd+U/o=;
        b=a66ODQlA3E+q5SddnNSLt4OxaHLcSq8miZgRf38shHh7PJvLxsJgGjH5ww4/Q2HK1i
         cmF+N/tU4vHHmZ1ejks09hsfsWAV35aKedZ9rAKzW3jctykeF2GWkPhXkNqqR7tyDUkk
         y1aQfWss+8GnKe7jJx9HmSL2eKgrwQFjpFfg4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708679951; x=1709284751;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=d7hQCuUachth0pW2nuQ1/VeIz0I8udEWdG6ieEd+U/o=;
        b=OJi0xv6/7Y9h49ga/x2TqlF94o9sqGXnEix/S/g5qVptNjySemT4KzCXEy3W1Lbady
         BtpguMMfkm+uTJpOD3CdDptseUBjia/kVXIWMEA5agekbSkmUpSGgUSuY+q5AK6Lz4bz
         G9HsBt8TP3k+YfF5bpBEGF9eqoWiG46cnj7/CeL7VIF1JoKp4BNv+F2wJSw3YVBE0lIE
         i4laacsY6shcaYri1c9cH5/UgGTwjzy0M4nngnYSnvycV+dbqIvM/JA/EV94Mg81Hlxi
         ckcQQhOs2Pvrk6tqbSoXs/B2mrsGteiPodMBEDuNuRon9QM1Go2AP4wgqYtHSjYitD1d
         SL9Q==
X-Forwarded-Encrypted: i=1; AJvYcCULmo26i5EGxN7vNMuWOnBtEDe70mBRgJu13BCQZzCqqRjz/ot9GEyh1PqkmOctm0rl6JRu2kdrV2rEXxyt/cuDdhRSM16HqZ+xlArlYSQ=
X-Gm-Message-State: AOJu0YxTBWuHgGx7uNUYHsrEKbXKpVfu2rEbz4AGZ5OyWwrZ/iYglPxq
	QSC5Cvweo1w7hq5sXwFg07EeIG0bYQpSNqJ/3+b5WrYLsimNZWlTNG3E5sgJ5PE=
X-Google-Smtp-Source: AGHT+IFh8sCmJJ0Uj3+IH25ZkymPg0MrwVYtvDLQmvWJVlI7H90HIuaZ8c1jMaezxFY6Gp2/yEZ+Hg==
X-Received: by 2002:ac2:48ab:0:b0:512:cfac:3d81 with SMTP id u11-20020ac248ab000000b00512cfac3d81mr813189lfg.68.1708679951392;
        Fri, 23 Feb 2024 01:19:11 -0800 (PST)
Date: Fri, 23 Feb 2024 10:19:08 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	Matthew Grooms <mgrooms@shrew.net>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH v2] x86/altcall: use an union as register type for
 function parameters
Message-ID: <ZdhjDBlHRtO4MIhM@macbook>
References: <20240222164455.67248-1-roger.pau@citrix.com>
 <b1c96b93-fd75-4077-b622-4781b7811ee8@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <b1c96b93-fd75-4077-b622-4781b7811ee8@suse.com>

On Thu, Feb 22, 2024 at 05:55:00PM +0100, Jan Beulich wrote:
> On 22.02.2024 17:44, Roger Pau Monne wrote:
> > --- a/xen/arch/x86/include/asm/alternative.h
> > +++ b/xen/arch/x86/include/asm/alternative.h
> > @@ -167,9 +167,25 @@ extern void alternative_branches(void);
> >  #define ALT_CALL_arg5 "r8"
> >  #define ALT_CALL_arg6 "r9"
> >  
> > +#ifdef CONFIG_CC_IS_CLANG
> > +/*
> > + * Use an union with an unsigned long in order to prevent clang from skipping a
> > + * possible truncation of the value.  By using the union any truncation is
> > + * carried before the call instruction.
> > + * https://github.com/llvm/llvm-project/issues/82598
> > + */
> 
> I think it needs saying that this is relying on compiler behavior not
> mandated by the standard, thus explaining why it's restricted to
> Clang (down the road we may even want to restrict to old versions,
> assuming they fix the issue at some point). Plus also giving future
> readers a clear understanding that if something breaks with this, it's
> not really a surprise.

What about:

Use a union with an unsigned long in order to prevent clang from
skipping a possible truncation of the value.  By using the union any
truncation is carried before the call instruction.  Note this
behavior is not mandated by the standard, and hence could stop being
a viable workaround, or worse, could cause a different set of
code-generation issues in future clang versions.

This has been reported upstream at:
https://github.com/llvm/llvm-project/issues/82598

> Aiui this bug is only a special case of the other, much older one, so
> referencing that one here too would seem advisable.

My report has been resolved as a duplicate of:

https://github.com/llvm/llvm-project/issues/43573

FWIW, I think for the context the link is used in (altcall) my bug
report is more representative, and readers can always follow the trail
into the other inter-related bugs.

> As a nit: I think it is "a union" (spelling according to pronunciation),
> and I guess the title could now do with saying "optionally" or
> mentioning Clang or some such.

OK.

Thanks, Roger


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:35:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:35:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684705.1064732 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRy7-0005xW-RW; Fri, 23 Feb 2024 09:35:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684705.1064732; Fri, 23 Feb 2024 09:35:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRy7-0005xP-Os; Fri, 23 Feb 2024 09:35:47 +0000
Received: by outflank-mailman (input) for mailman id 684705;
 Fri, 23 Feb 2024 09:35:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hL0v=KA=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rdRy6-0005xJ-4F
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:35:46 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ea80ec04-d22e-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 10:35:44 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id B75ED4EE073F;
 Fri, 23 Feb 2024 10:35:42 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea80ec04-d22e-11ee-98f5-efadbce2ee36
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	consulting@bugseng.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH 0/2] address some violations of MISRA C Rule 17.7
Date: Fri, 23 Feb 2024 10:35:35 +0100
Message-Id: <cover.1708680104.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 17.7 states: "The value returned by a function having non-void
return type shall be used". Therefore the functions that are subject to this
series are refactored to address these concerns.

Nicola Vetrini (2):
  xen/console: drop return value from consoled_guest_rx/tx
  xen/cpu: address MISRA C Rule 17.7

 xen/common/cpu.c            | 31 +++++++++++++++++++------------
 xen/drivers/char/consoled.c | 17 +++++------------
 xen/include/xen/consoled.h  |  4 ++--
 3 files changed, 26 insertions(+), 26 deletions(-)

-- 
2.34.1


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:35:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:35:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684706.1064742 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRyB-0006CA-2f; Fri, 23 Feb 2024 09:35:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684706.1064742; Fri, 23 Feb 2024 09:35:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRyA-0006C3-VV; Fri, 23 Feb 2024 09:35:50 +0000
Received: by outflank-mailman (input) for mailman id 684706;
 Fri, 23 Feb 2024 09:35:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hL0v=KA=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rdRy9-0006BX-1V
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:35:49 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ecdb525a-d22e-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 10:35:48 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id B667E4EE0741;
 Fri, 23 Feb 2024 10:35:46 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ecdb525a-d22e-11ee-8a57-1f161083a0e0
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	consulting@bugseng.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH 1/2] xen/console: drop return value from consoled_guest_rx/tx
Date: Fri, 23 Feb 2024 10:35:36 +0100
Message-Id: <4998ec735bd7e5a50a229507e2b92ae56ec1ba4b.1708680104.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1708680104.git.nicola.vetrini@bugseng.com>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

These functions never saw a usage of their return value since
they were introduced, so it can be dropped since their usages
violate MISRA C Rule 17.7:
"The value returned by a function having non-void return type shall be used".

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/drivers/char/consoled.c | 17 +++++------------
 xen/include/xen/consoled.h  |  4 ++--
 2 files changed, 7 insertions(+), 14 deletions(-)

diff --git a/xen/drivers/char/consoled.c b/xen/drivers/char/consoled.c
index 222e01844271..b415b632cecc 100644
--- a/xen/drivers/char/consoled.c
+++ b/xen/drivers/char/consoled.c
@@ -43,13 +43,13 @@ struct xencons_interface *consoled_get_ring_addr(void)
 static char buf[BUF_SZ + 1];
 
 /* Receives characters from a domain's PV console */
-size_t consoled_guest_rx(void)
+void consoled_guest_rx(void)
 {
-    size_t recv = 0, idx = 0;
+    size_t idx = 0;
     XENCONS_RING_IDX cons, prod;
 
     if ( !cons_ring )
-        return 0;
+        return;
 
     spin_lock(&rx_lock);
 
@@ -73,7 +73,6 @@ size_t consoled_guest_rx(void)
         char c = cons_ring->out[MASK_XENCONS_IDX(cons++, cons_ring->out)];
 
         buf[idx++] = c;
-        recv++;
 
         if ( idx >= BUF_SZ )
         {
@@ -92,18 +91,15 @@ size_t consoled_guest_rx(void)
 
  out:
     spin_unlock(&rx_lock);
-
-    return recv;
 }
 
 /* Sends a character into a domain's PV console */
-size_t consoled_guest_tx(char c)
+void consoled_guest_tx(char c)
 {
-    size_t sent = 0;
     XENCONS_RING_IDX cons, prod;
 
     if ( !cons_ring )
-        return 0;
+        return;
 
     cons = ACCESS_ONCE(cons_ring->in_cons);
     prod = cons_ring->in_prod;
@@ -121,7 +117,6 @@ size_t consoled_guest_tx(char c)
         goto notify;
 
     cons_ring->in[MASK_XENCONS_IDX(prod++, cons_ring->in)] = c;
-    sent++;
 
     /* Write to the ring before updating the pointer */
     smp_wmb();
@@ -130,8 +125,6 @@ size_t consoled_guest_tx(char c)
  notify:
     /* Always notify the guest: prevents receive path from getting stuck. */
     pv_shim_inject_evtchn(pv_console_evtchn());
-
-    return sent;
 }
 
 /*
diff --git a/xen/include/xen/consoled.h b/xen/include/xen/consoled.h
index 2b30516b3a0a..bd7ab6329ee8 100644
--- a/xen/include/xen/consoled.h
+++ b/xen/include/xen/consoled.h
@@ -5,8 +5,8 @@
 
 void consoled_set_ring_addr(struct xencons_interface *ring);
 struct xencons_interface *consoled_get_ring_addr(void);
-size_t consoled_guest_rx(void);
-size_t consoled_guest_tx(char c);
+void consoled_guest_rx(void);
+void consoled_guest_tx(char c);
 
 #endif /* __XEN_CONSOLED_H__ */
 /*
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:35:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:35:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684707.1064752 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRyD-0006RU-8h; Fri, 23 Feb 2024 09:35:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684707.1064752; Fri, 23 Feb 2024 09:35:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdRyD-0006RL-5w; Fri, 23 Feb 2024 09:35:53 +0000
Received: by outflank-mailman (input) for mailman id 684707;
 Fri, 23 Feb 2024 09:35:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=hL0v=KA=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rdRyC-0006BX-2n
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:35:52 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id eeee753b-d22e-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 10:35:51 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id 4FDB34EE0C8A;
 Fri, 23 Feb 2024 10:35:50 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eeee753b-d22e-11ee-8a57-1f161083a0e0
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	consulting@bugseng.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
Date: Fri, 23 Feb 2024 10:35:37 +0100
Message-Id: <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1708680104.git.nicola.vetrini@bugseng.com>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Refactor cpu_notifier_call_chain into two functions:
- the variant that is allowed to fail loses the nofail flag
- the variant that shouldn't fail is encapsulated in a call
  to the failing variant, with an additional check.

This prevents uses of the function that are not supposed to
fail from ignoring the return value, thus violating Rule 17.7:
"The value returned by a function having non-void return type shall
be used".

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/common/cpu.c | 31 +++++++++++++++++++------------
 1 file changed, 19 insertions(+), 12 deletions(-)

diff --git a/xen/common/cpu.c b/xen/common/cpu.c
index 8709db4d2957..0b7cf54c4264 100644
--- a/xen/common/cpu.c
+++ b/xen/common/cpu.c
@@ -78,20 +78,27 @@ void __init register_cpu_notifier(struct notifier_block *nb)
 }
 
 static int cpu_notifier_call_chain(unsigned int cpu, unsigned long action,
-                                   struct notifier_block **nb, bool nofail)
+                                   struct notifier_block **nb)
 {
     void *hcpu = (void *)(long)cpu;
     int notifier_rc = notifier_call_chain(&cpu_chain, action, hcpu, nb);
     int ret =  notifier_to_errno(notifier_rc);
 
-    BUG_ON(ret && nofail);
-
     return ret;
 }
 
+static void cpu_notifier_call_chain_nofail(unsigned int cpu,
+                                           unsigned long action,
+                                           struct notifier_block **nb)
+{
+    int ret = cpu_notifier_call_chain(cpu, action, nb);
+
+    BUG_ON(ret);
+}
+
 static void cf_check _take_cpu_down(void *unused)
 {
-    cpu_notifier_call_chain(smp_processor_id(), CPU_DYING, NULL, true);
+    cpu_notifier_call_chain_nofail(smp_processor_id(), CPU_DYING, NULL);
     __cpu_disable();
 }
 
@@ -116,7 +123,7 @@ int cpu_down(unsigned int cpu)
     if ( !cpu_online(cpu) )
         goto out;
 
-    err = cpu_notifier_call_chain(cpu, CPU_DOWN_PREPARE, &nb, false);
+    err = cpu_notifier_call_chain(cpu, CPU_DOWN_PREPARE, &nb);
     if ( err )
         goto fail;
 
@@ -129,14 +136,14 @@ int cpu_down(unsigned int cpu)
     err = cpu_online(cpu);
     BUG_ON(err);
 
-    cpu_notifier_call_chain(cpu, CPU_DEAD, NULL, true);
+    cpu_notifier_call_chain_nofail(cpu, CPU_DEAD, NULL);
 
     send_global_virq(VIRQ_PCPU_STATE);
     cpu_hotplug_done();
     return 0;
 
  fail:
-    cpu_notifier_call_chain(cpu, CPU_DOWN_FAILED, &nb, true);
+    cpu_notifier_call_chain_nofail(cpu, CPU_DOWN_FAILED, &nb);
  out:
     cpu_hotplug_done();
     return err;
@@ -157,7 +164,7 @@ int cpu_up(unsigned int cpu)
     if ( cpu_online(cpu) )
         goto out;
 
-    err = cpu_notifier_call_chain(cpu, CPU_UP_PREPARE, &nb, false);
+    err = cpu_notifier_call_chain(cpu, CPU_UP_PREPARE, &nb);
     if ( err )
         goto fail;
 
@@ -165,7 +172,7 @@ int cpu_up(unsigned int cpu)
     if ( err < 0 )
         goto fail;
 
-    cpu_notifier_call_chain(cpu, CPU_ONLINE, NULL, true);
+    cpu_notifier_call_chain_nofail(cpu, CPU_ONLINE, NULL);
 
     send_global_virq(VIRQ_PCPU_STATE);
 
@@ -173,7 +180,7 @@ int cpu_up(unsigned int cpu)
     return 0;
 
  fail:
-    cpu_notifier_call_chain(cpu, CPU_UP_CANCELED, &nb, true);
+    cpu_notifier_call_chain_nofail(cpu, CPU_UP_CANCELED, &nb);
  out:
     cpu_hotplug_done();
     return err;
@@ -181,7 +188,7 @@ int cpu_up(unsigned int cpu)
 
 void notify_cpu_starting(unsigned int cpu)
 {
-    cpu_notifier_call_chain(cpu, CPU_STARTING, NULL, true);
+    cpu_notifier_call_chain_nofail(cpu, CPU_STARTING, NULL);
 }
 
 static cpumask_t frozen_cpus;
@@ -237,7 +244,7 @@ void enable_nonboot_cpus(void)
     }
 
     for_each_cpu ( cpu, &frozen_cpus )
-        cpu_notifier_call_chain(cpu, CPU_RESUME_FAILED, NULL, true);
+        cpu_notifier_call_chain_nofail(cpu, CPU_RESUME_FAILED, NULL);
 
     cpumask_clear(&frozen_cpus);
 }
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:43:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:43:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684717.1064762 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdS53-0000L4-2y; Fri, 23 Feb 2024 09:42:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684717.1064762; Fri, 23 Feb 2024 09:42:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdS53-0000Kx-02; Fri, 23 Feb 2024 09:42:57 +0000
Received: by outflank-mailman (input) for mailman id 684717;
 Fri, 23 Feb 2024 09:42:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdS51-0000Kr-V2
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:42:55 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id eaba9a08-d22f-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 10:42:54 +0100 (CET)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-3394b892691so364032f8f.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 01:42:54 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 pv18-20020ad45492000000b0068f2ea5c678sm4670047qvb.118.2024.02.23.01.42.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 01:42:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eaba9a08-d22f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708681373; x=1709286173; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=BXPA50Xn8D6xZzjKg4dTG6GJFYvuP5BIsnkJUOBPMQE=;
        b=SVbxGN4+u2D+A3dXaM//KrjrtHc9X6VBypTB5w6w1d2SNvFiz1iOwZnHM6IVUaxkNs
         ihlzONy02Wd1aG0LOGyICowETVxYTB7fVwHWGclOkRkI2rNheZoJSVHSEcCZsz6fkoQ1
         Tks+GjKRFqC36E64NyngkgUFWkUkZt3JrvQWg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708681373; x=1709286173;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=BXPA50Xn8D6xZzjKg4dTG6GJFYvuP5BIsnkJUOBPMQE=;
        b=FSTKVslM3AVzMmKDvKuYbNBQav3XB/20EdNBLlnkeF/5ebE7zzKOCstKky7VHlnIIC
         6RHE4B0YG8esqzOiB7kkbaGOZ+UPtCLzV76xCY+Stn0HTRoYT2ffYaTO80XBYp5rLYQr
         wOrcaueUzwar/hfM14aou2wTeZr9wRDuCB232fPrpnOHnEogvPo153XXUEwCc4Wtms2F
         mJ1mCrB59Stxk/V9C24keybGozYaSkHRJz+lZ+CsX79tOny83TeowB99lxGMADQzNG3/
         RWwNtnQxvLdvf1i1YBL7bw67YgKwcRvHOBsrrC/84KvM6/Clft0j+97p4bUoJM96Euiz
         b6cA==
X-Gm-Message-State: AOJu0Yz3uDtc04XO8czVvqt6XUIeOTO3HXVQA/YcDJ0epZ9Ehhjj/Moz
	pPwyWYH4YPWt/5xBpjwj+BLNcl1l22PBpPD7K29uOA8CDFP6IOdkO1MotPGtZq9lZdKk8kZtwnn
	2
X-Google-Smtp-Source: AGHT+IG4tu8jAfWiGsF9LamOkjO6IxRmmwB883d58ervno1AesmGmmyXLGVU5LTzlzMWDzh+ntWzxw==
X-Received: by 2002:adf:cf04:0:b0:33d:90c2:c7f4 with SMTP id o4-20020adfcf04000000b0033d90c2c7f4mr3584212wrj.14.1708681373223;
        Fri, 23 Feb 2024 01:42:53 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/spec: fix BRANCH_HARDEN option to only be set when build-enabled
Date: Fri, 23 Feb 2024 10:42:15 +0100
Message-ID: <20240223094215.71889-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The current logic to handle the BRANCH_HARDEN option will report it as enabled
even when build-time disabled. Fix this by only allowing the option to be set
when support for it is built into Xen.

Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/spec_ctrl.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 421fe3f640df..e634c6b559b4 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
 int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
 int8_t __read_mostly opt_eager_fpu = -1;
 int8_t __read_mostly opt_l1d_flush = -1;
-static bool __initdata opt_branch_harden = true;
+static bool __initdata opt_branch_harden =
+    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
 
 bool __initdata bsp_delay_spec_ctrl;
 uint8_t __read_mostly default_xen_spec_ctrl;
@@ -267,7 +268,8 @@ static int __init cf_check parse_spec_ctrl(const char *s)
             opt_eager_fpu = val;
         else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
             opt_l1d_flush = val;
-        else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
+        else if ( IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) &&
+                  (val = parse_boolean("branch-harden", s, ss)) >= 0 )
             opt_branch_harden = val;
         else if ( (val = parse_boolean("srb-lock", s, ss)) >= 0 )
             opt_srb_lock = val;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:45:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:45:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684720.1064772 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdS7V-0000tp-EL; Fri, 23 Feb 2024 09:45:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684720.1064772; Fri, 23 Feb 2024 09:45:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdS7V-0000ti-BR; Fri, 23 Feb 2024 09:45:29 +0000
Received: by outflank-mailman (input) for mailman id 684720;
 Fri, 23 Feb 2024 09:45:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jp8H=KA=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1rdS7T-0000tc-U5
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:45:27 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 45de15ad-d230-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 10:45:26 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-41295fd7847so1288635e9.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 01:45:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 45de15ad-d230-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708681526; x=1709286326; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=nOX50ZzAMGvnHyaJn2dczFDwxeACzI/aEsvZXuqY8/I=;
        b=Ro8W+HunoMHBlcJoo+c3eRC0yK3iubCGUMCzl3UYbe8TUqQx75hnc7Tg3hdkV4D9Az
         mOo609Q93Tmcm9sFol+MKUJkXNNFSNbuD3v23XrJ5uSVOQ2WqOHWXZ0KkvSPCQlBUCQt
         TLDgVuwYIHDwvhOGVOh/TE/gElHeEUbssALfE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708681526; x=1709286326;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=nOX50ZzAMGvnHyaJn2dczFDwxeACzI/aEsvZXuqY8/I=;
        b=P2P4tJrCJh8s3KxDnoWypmRUWrIh+LgVh+jtl4QKx19AiqCYG5AydsrX1b1FyCaTYU
         Ef/3TI147NMETWtiiG+NivLRSe1g5OyK0ipgdPUlXANAuNPYRILXbRQrEPmryFKJ8Fh/
         hyKCfXSPKUM6FHjR3QOjq8eu8B5pSB94f0UupCQ4cVe20MXIzn4BB/JhcvLsM7EYnoCr
         y5lBG6faTa4QptzFcfqK+RdaFwu36DBRdvlXu8C+l87GJadi6oiMGDYVDe9dHLpE8K1y
         vWd/BrCLRvkJiMC+NPPaMpIfhPQNocyHOh2v547zg36+itCFQnJPIL9DMXom7G3rdf7i
         AlCA==
X-Gm-Message-State: AOJu0Yw6uQTrMMjU3cZqXHLINHZMh6wPqMbM2PCZU8PJbx65CTdP2JFa
	/U5k0j9wktynqIb/P4SErWL/SOORAkEpSleskw3aPPqWm2KIPyL1xgtx+mpiOq3oAVt2/7tFlSn
	XdzKxcA9/5pCsLQsZtfHj240MEznFp3tZocat
X-Google-Smtp-Source: AGHT+IFT0ErA0nfP2BE1trHVau5z8enNuPNiK+5R85eWp2AG2dwv6ShEQQV/rFtcTGH53RGHuIC9lkfzqGuTSXDhs/4=
X-Received: by 2002:a05:600c:3acd:b0:411:e7d5:cc5 with SMTP id
 d13-20020a05600c3acd00b00411e7d50cc5mr836680wms.4.1708681526400; Fri, 23 Feb
 2024 01:45:26 -0800 (PST)
MIME-Version: 1.0
References: <20231130142944.46322-1-roger.pau@citrix.com> <20231130142944.46322-2-roger.pau@citrix.com>
In-Reply-To: <20231130142944.46322-2-roger.pau@citrix.com>
From: Ross Lagerwall <ross.lagerwall@cloud.com>
Date: Fri, 23 Feb 2024 09:45:15 +0000
Message-ID: <CAG7k0Eq+G7w8M2JFTnikv10SNocfCBGwvbEwz-pbR9v99uStGg@mail.gmail.com>
Subject: Re: [PATCH 1/5] xen/livepatch: register livepatch regions when loaded
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
	Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Nov 30, 2023 at 2:30=E2=80=AFPM Roger Pau Monne <roger.pau@citrix.c=
om> wrote:
>
> Currently livepatch regions are registered as virtual regions only after =
the
> livepatch has been applied.
>
> This can lead to issues when using the pre-apply or post-revert hooks, as=
 at
> the point the livepatch is not in the virtual regions list.  If a livepat=
ch
> pre-apply hook contains a WARN() it would trigger an hypervisor crash, as=
 the
> code to handle the bug frame won't be able to find the instruction pointe=
r that
> triggered the #UD in any of the registered virtual regions, and hence cra=
sh.
>
> Fix this by adding the livepatch payloads as virtual regions as soon as l=
oaded,
> and only remove them once the payload is unloaded.  This requires some ch=
anges
> to the virtual regions code, as the removal of the virtual regions is no =
longer
> done in stop machine context, and hence an RCU barrier is added in order =
to
> make sure there are no users of the virtual region after it's been remove=
d from
> the list.
>
> Fixes: 8313c864fa95 ('livepatch: Implement pre-|post- apply|revert hooks'=
)
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
>  xen/common/livepatch.c      |  5 +++--
>  xen/common/virtual_region.c | 40 +++++++++++--------------------------
>  2 files changed, 15 insertions(+), 30 deletions(-)
>
> diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
> index 1209fea2566c..3199432f11f5 100644
> --- a/xen/common/livepatch.c
> +++ b/xen/common/livepatch.c
> @@ -942,6 +942,8 @@ static int prepare_payload(struct payload *payload,
>          }
>      }
>
> +    register_virtual_region(region);
> +
>      return 0;
>  }
>

The region is registered in prepare_payload() but if e.g. the build id
check in load_payload_data() fails, it won't unregister the region
since the failure path calls free_payload_data(), not free_payload().
Perhaps the call to register_virtual_region() could be done as the last
thing in load_payload_data()?

Ross


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:46:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:46:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684725.1064782 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdS8W-0001Pc-NS; Fri, 23 Feb 2024 09:46:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684725.1064782; Fri, 23 Feb 2024 09:46:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdS8W-0001PV-KY; Fri, 23 Feb 2024 09:46:32 +0000
Received: by outflank-mailman (input) for mailman id 684725;
 Fri, 23 Feb 2024 09:46:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0yaO=KA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rdS8U-0001P9-Tr
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:46:30 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6aebb05e-d230-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 10:46:29 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-412949fd224so928155e9.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 01:46:29 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 m6-20020a7bce06000000b00410bca333b7sm1683825wmc.27.2024.02.23.01.46.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 23 Feb 2024 01:46:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6aebb05e-d230-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708681588; x=1709286388; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=GMLGYht9EikJPYSB8/VdWj+YkiECcKdadrFo+N57dhc=;
        b=Hw5I04Cw+RkbMLYM6em8TbPpEqvwHBn2XRFVSfRtHYoJur+68o5sTHqpd1bQdQksQq
         7hqw5RTqwx+Nrbx89LWyM9sKpOQEEcSTmJqLdxrf9tueM819LqRaZA92z52ALtQ5Mcw6
         1cZ1RDuv5Z87jkLVu6TST9Bs0+LeVrCi7Xu3s=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708681588; x=1709286388;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GMLGYht9EikJPYSB8/VdWj+YkiECcKdadrFo+N57dhc=;
        b=Ce5xTeaPr7rpu9VJAWwGoQnsAW//tOh2UxBx/HuC7P8m6zIfRBBnDfeiX43Y1QAe8M
         0IiHJfCd+irHXOL4S+rqBCX/OG1va4A367jJUKLVi76qNpLX5bIu7gzBEytm0xnz/4CK
         oaVbhhZDYmDbXmWZbx3ovQ/K0mwLtujZN9NRuT2v97K7c2G9x0IrusL5hk5+AEePSvkb
         VhgPSHwnOPbOBpV8ctNT6aCz4yL0g7IZHNWILUdGQrFUFJxsbS30eZTbvz5NPbJajdns
         nVzN33D+UmO1NFSOQ+D6z3zBYLLuJLyxmQmcLfev6LR04y0lqg7RzWi6dn9sbrrBjhFJ
         AhZA==
X-Forwarded-Encrypted: i=1; AJvYcCW2OPgQNLR8MDXRkvINEPCmFRT41zZLA0DJKmp7tbRV2yUCvfFTTG86uvN59tKeeb7rO0yyi0bIxtbSdDPzeI6osr0F9hv+lC6NtMXQvSo=
X-Gm-Message-State: AOJu0Yz8Dkg14f1jtrD2iVU73VzMzcqPTOGiyk65c5nI+El6SOVcvEyt
	Me1hkEb6DBXUlpt9mLeK5HmPk61XCaN9y+7ruEIHYX0+KEcuM9xrBpwaR1GmuSZDrNvFw+BPoQY
	b
X-Google-Smtp-Source: AGHT+IGy2IXZKTAwHpAaT1uAZiPYWeCGA30GvOYCOsSCN3kTAk/uEIgbbKbtG1WLmY5IXObDLHRVog==
X-Received: by 2002:adf:f5c4:0:b0:33d:3631:f78d with SMTP id k4-20020adff5c4000000b0033d3631f78dmr4666718wrp.18.1708681588504;
        Fri, 23 Feb 2024 01:46:28 -0800 (PST)
Message-ID: <b5696664-0e85-414b-8360-242349a87a7e@citrix.com>
Date: Fri, 23 Feb 2024 09:46:27 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/spec: fix BRANCH_HARDEN option to only be set when
 build-enabled
Content-Language: en-GB
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Jan Beulich <jbeulich@suse.com>, Wei Liu <wl@xen.org>
References: <20240223094215.71889-1-roger.pau@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240223094215.71889-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/02/2024 9:42 am, Roger Pau Monne wrote:
> The current logic to handle the BRANCH_HARDEN option will report it as enabled
> even when build-time disabled. Fix this by only allowing the option to be set
> when support for it is built into Xen.
>
> Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
>  xen/arch/x86/spec_ctrl.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
> index 421fe3f640df..e634c6b559b4 100644
> --- a/xen/arch/x86/spec_ctrl.c
> +++ b/xen/arch/x86/spec_ctrl.c
> @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
>  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
>  int8_t __read_mostly opt_eager_fpu = -1;
>  int8_t __read_mostly opt_l1d_flush = -1;
> -static bool __initdata opt_branch_harden = true;
> +static bool __initdata opt_branch_harden =
> +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
>  
>  bool __initdata bsp_delay_spec_ctrl;
>  uint8_t __read_mostly default_xen_spec_ctrl;
> @@ -267,7 +268,8 @@ static int __init cf_check parse_spec_ctrl(const char *s)
>              opt_eager_fpu = val;
>          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
>              opt_l1d_flush = val;
> -        else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> +        else if ( IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) &&
> +                  (val = parse_boolean("branch-harden", s, ss)) >= 0 )
>              opt_branch_harden = val;

Yeah, we should definitely fix this, but could we use no_config_param()
here for the compiled-out case ?

See cet= for an example.  If we're going to ignore what the user asks,
we should tell them why.

And given this as an example, shouldn't we do the same with
CONFIG_INDIRECT_THUNK and bti=thunk= too ?

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 09:58:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 09:58:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684730.1064791 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSJg-0003Q5-Mv; Fri, 23 Feb 2024 09:58:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684730.1064791; Fri, 23 Feb 2024 09:58:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSJg-0003Py-KT; Fri, 23 Feb 2024 09:58:04 +0000
Received: by outflank-mailman (input) for mailman id 684730;
 Fri, 23 Feb 2024 09:58:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdSJe-0003Ps-SC
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 09:58:02 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 078fbf7f-d232-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 10:58:01 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-41241f64c6bso5191355e9.0
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 01:58:01 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 fc19-20020a05600c525300b0041285ffec13sm1743787wmb.41.2024.02.23.01.58.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 01:58:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 078fbf7f-d232-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708682281; x=1709287081; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Yz4otW/pwgozuw7DlysuN7mX+Rcs2WUAXHt11rHZ1hY=;
        b=GtB3a0pGY1XLVSy5sG+n/LHY/Q+iUua7gnsJ7gCtVsKH9//DFKV9+RWEhX/d4CKSDm
         lYYIjQBEhit6YLe8gHhdb5Y0bhZrZrIstKFmNrdBm+ZqMnxEtkpfmdXGVA5Ro/NKrvSx
         Xl5l4kPt0cGWtX9hBmCagqPBiD4uX+QVg5Uxg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708682281; x=1709287081;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Yz4otW/pwgozuw7DlysuN7mX+Rcs2WUAXHt11rHZ1hY=;
        b=Lz2Y4ld3VeeAt440ARR3FPBNqPVCoc0G6lS4fvU7g0epGqSOOrNbJPaInEGWTb69hp
         UYJ8fyRDBJaC3CN3AwJHr4idjM7Z4XZ3lTAxQz/NO6pqIQHwX9aON4uiFZm/ZsiIYosH
         czTsd2/WyXiDCoWi6Ayg3vF5gZ1TB4Zx+iMf0vqeP590SAQUgvPUnW5RCzLsUFjm64UP
         924Pk8W1swxw+obh+yuq0WUtyvFlSLfjOzzw6mqXsiAML4gyPidmaytrykMUEbtkEKWY
         QQhjCyGz+IV27Bixw3+qcWu0WBFQecxH9mHXWS/Sgvu3SFScRCbLldGdTSahZ6KNgi2n
         T6kA==
X-Gm-Message-State: AOJu0Yy+JR+P2m1kTRXVjITe/fDamDRTwzs8hugy6roxa5x10dB7Wkf+
	9Yl4/p945b2xbMNMaqmg4HFX6sEx6Kj+DSRaj0JQksPFxtveonasC4psa/p0OIY=
X-Google-Smtp-Source: AGHT+IGTsa/MHcy6FouYaoChNreOOYgQdrChkSch3kqq2XqKsHrEtd1hEcRbvjatSIMZNkAo4m67Yw==
X-Received: by 2002:a05:600c:154a:b0:412:96f2:2df9 with SMTP id f10-20020a05600c154a00b0041296f22df9mr172170wmg.26.1708682280792;
        Fri, 23 Feb 2024 01:58:00 -0800 (PST)
Date: Fri, 23 Feb 2024 10:57:59 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86/spec: fix BRANCH_HARDEN option to only be set when
 build-enabled
Message-ID: <ZdhsJ3dWbLuSsoCu@macbook>
References: <20240223094215.71889-1-roger.pau@citrix.com>
 <b5696664-0e85-414b-8360-242349a87a7e@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <b5696664-0e85-414b-8360-242349a87a7e@citrix.com>

On Fri, Feb 23, 2024 at 09:46:27AM +0000, Andrew Cooper wrote:
> On 23/02/2024 9:42 am, Roger Pau Monne wrote:
> > The current logic to handle the BRANCH_HARDEN option will report it as enabled
> > even when build-time disabled. Fix this by only allowing the option to be set
> > when support for it is built into Xen.
> >
> > Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> >  xen/arch/x86/spec_ctrl.c | 6 ++++--
> >  1 file changed, 4 insertions(+), 2 deletions(-)
> >
> > diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
> > index 421fe3f640df..e634c6b559b4 100644
> > --- a/xen/arch/x86/spec_ctrl.c
> > +++ b/xen/arch/x86/spec_ctrl.c
> > @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
> >  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
> >  int8_t __read_mostly opt_eager_fpu = -1;
> >  int8_t __read_mostly opt_l1d_flush = -1;
> > -static bool __initdata opt_branch_harden = true;
> > +static bool __initdata opt_branch_harden =
> > +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
> >  
> >  bool __initdata bsp_delay_spec_ctrl;
> >  uint8_t __read_mostly default_xen_spec_ctrl;
> > @@ -267,7 +268,8 @@ static int __init cf_check parse_spec_ctrl(const char *s)
> >              opt_eager_fpu = val;
> >          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
> >              opt_l1d_flush = val;
> > -        else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> > +        else if ( IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) &&
> > +                  (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> >              opt_branch_harden = val;
> 
> Yeah, we should definitely fix this, but could we use no_config_param()
> here for the compiled-out case ?

Oh, didn't know that helper existed.

> See cet= for an example.  If we're going to ignore what the user asks,
> we should tell them why.
> 
> And given this as an example, shouldn't we do the same with
> CONFIG_INDIRECT_THUNK and bti=thunk= too ?

Checked for SPECULATIVE_HARDEN_ARRAY and
SPECULATIVE_HARDEN_GUEST_ACCESS, but not the thunk, will add it as a
followup patch.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:11:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:11:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684736.1064802 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSW4-0006kC-RG; Fri, 23 Feb 2024 10:10:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684736.1064802; Fri, 23 Feb 2024 10:10:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSW4-0006k5-O9; Fri, 23 Feb 2024 10:10:52 +0000
Received: by outflank-mailman (input) for mailman id 684736;
 Fri, 23 Feb 2024 10:10:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jp8H=KA=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1rdSW3-0006jz-Qw
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:10:51 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d15ef228-d233-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 11:10:49 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-4128cfb6c1aso4164095e9.0
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:10:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d15ef228-d233-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708683049; x=1709287849; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=kbvQLKnWegIgTosnvez+76k27dwEsKDLBPZ1TSP0Xkk=;
        b=YGtlVgvME6ogoiH7M8i+O3z2ZotNfzy5sgMEa7kuEq47JFMYDrA52BSBkTw45Lytn1
         WrF5PuMm3TL5bT2duxI5AIkCfc+mbq0EcJopDZxtbjmrCAeZN7+4OEiwQNYxeiSzA2kU
         6lyGOzPTOti97pO8OgXysdCplqsqTQFt+q1U8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708683049; x=1709287849;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=kbvQLKnWegIgTosnvez+76k27dwEsKDLBPZ1TSP0Xkk=;
        b=wfsM5VFTlIqUdIrG3BjQakdUJQnM7NTgu4EHuV6V965nAe1ExQVSJTnY5N46ypErsI
         XITHNkKc6sEsWaYYzvF3GBrDNRN7zYQVBtjnko8bT1Xsew5ppYVHVnbH2Pt7v5p0M7SY
         VloU5QOSyIbaIKbm6PE4nwk8dDA7MreSkF6thkmUJ4TjSCSE4q2uXJE6po1NdslzmIsB
         RKtnfkF8LaaVbUC0Rral4rZk8OQce4xoBjj/nu0X6AWdZalIaqG9e8ykbwueXgVKC7YZ
         jLYRo5dTCL5b8EpS8cWM4yZc0rL2KpZQb+5artVEs+jl0EIrP6zw/92HGFvMVqPW88Ek
         GRCw==
X-Gm-Message-State: AOJu0YwAXTJf/VJFA8sr1KkWllADuGYVCz3EyHvmN+ULEHguG5M0gMQ2
	QSsbndkDtZEvyoGw4ojbf+c43Ok2/R4BLg/B8rmxb8a6kT2LZn2Re9yDnh5nH5txTMBfGOreK3o
	RL+2PP/7I6TpcoSo8L0rx1jnyATY2oeL+rKtK
X-Google-Smtp-Source: AGHT+IELAouiUrZFKYG9zCBlrgQFNjkhEIqm9WixiX6M2uRhrK/FNBRBcXLTalHBN8JSkIVODogNUt8alvm1TI2GpiY=
X-Received: by 2002:a05:600c:1c09:b0:412:9739:717a with SMTP id
 j9-20020a05600c1c0900b004129739717amr218801wms.12.1708683049017; Fri, 23 Feb
 2024 02:10:49 -0800 (PST)
MIME-Version: 1.0
References: <20231130142944.46322-1-roger.pau@citrix.com> <20231130142944.46322-4-roger.pau@citrix.com>
In-Reply-To: <20231130142944.46322-4-roger.pau@citrix.com>
From: Ross Lagerwall <ross.lagerwall@citrix.com>
Date: Fri, 23 Feb 2024 10:10:38 +0000
Message-ID: <CAG7k0ErX0yydgTyZQM=CU+7ZgWnrvqJGXRGmUtM8=O9_UH=12A@mail.gmail.com>
Subject: Re: [PATCH 3/5] xen/livepatch: fix norevert test attempt to open-code revert
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Nov 30, 2023 at 2:30=E2=80=AFPM Roger Pau Monne <roger.pau@citrix.c=
om> wrote:
>
> The purpose of the norevert test is to install a dummy handler that repla=
ces
> the internal Xen revert code, and then perform the revert in the post-rev=
ert
> hook.  For that purpose the usage of the previous common_livepatch_revert=
() is
> not enough, as that just reverts specific functions, but not the whole st=
ate of
> the payload.
>
> Remove both common_livepatch_{apply,revert}() and instead expose
> revert_payload{,_tail}() in order to perform the patch revert from the
> post-revert hook.
>
> Fixes: 6047104c3ccc ('livepatch: Add per-function applied/reverted state =
tracking marker')
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
>  xen/common/livepatch.c                        | 41 +++++++++++++++++--
>  xen/include/xen/livepatch.h                   | 32 ++-------------
>  .../livepatch/xen_action_hooks_norevert.c     | 22 +++-------
>  3 files changed, 46 insertions(+), 49 deletions(-)
>
> diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
> index 4e775be66571..d81f3d11d655 100644
> --- a/xen/common/livepatch.c
> +++ b/xen/common/livepatch.c
> @@ -1367,7 +1367,22 @@ static int apply_payload(struct payload *data)
>      ASSERT(!local_irq_is_enabled());
>
>      for ( i =3D 0; i < data->nfuncs; i++ )
> -        common_livepatch_apply(&data->funcs[i], &data->fstate[i]);
> +    {
> +        const struct livepatch_func *func =3D &data->funcs[i];
> +        struct livepatch_fstate *state =3D &data->fstate[i];
> +
> +        /* If the action has been already executed on this function, do =
nothing. */
> +        if ( state->applied =3D=3D LIVEPATCH_FUNC_APPLIED )
> +        {
> +            printk(XENLOG_WARNING LIVEPATCH
> +                   "%s: %s has been already applied before\n",
> +                   __func__, func->name);
> +            continue;
> +        }
> +
> +        arch_livepatch_apply(func, state);
> +        state->applied =3D LIVEPATCH_FUNC_APPLIED;
> +    }
>
>      arch_livepatch_revive();
>
> @@ -1383,7 +1398,7 @@ static inline void apply_payload_tail(struct payloa=
d *data)
>      data->state =3D LIVEPATCH_STATE_APPLIED;
>  }
>
> -static int revert_payload(struct payload *data)
> +int revert_payload(struct payload *data)
>  {
>      unsigned int i;
>      int rc;
> @@ -1398,7 +1413,25 @@ static int revert_payload(struct payload *data)
>      }
>
>      for ( i =3D 0; i < data->nfuncs; i++ )
> -        common_livepatch_revert(&data->funcs[i], &data->fstate[i]);
> +    {
> +        const struct livepatch_func *func =3D &data->funcs[i];
> +        struct livepatch_fstate *state =3D &data->fstate[i];
> +
> +        /*
> +         * If the apply action hasn't been executed on this function, do
> +         * nothing.
> +         */
> +        if ( !func->old_addr || state->applied =3D=3D LIVEPATCH_FUNC_NOT=
_APPLIED )
> +        {
> +            printk(XENLOG_WARNING LIVEPATCH
> +                   "%s: %s has not been applied before\n",
> +                   __func__, func->name);
> +            continue;
> +        }
> +
> +        arch_livepatch_revert(func, state);
> +        state->applied =3D LIVEPATCH_FUNC_NOT_APPLIED;
> +    }
>
>      /*
>       * Since we are running with IRQs disabled and the hooks may call co=
mmon
> @@ -1416,7 +1449,7 @@ static int revert_payload(struct payload *data)
>      return 0;
>  }
>
> -static inline void revert_payload_tail(struct payload *data)
> +void revert_payload_tail(struct payload *data)
>  {
>      list_del(&data->applied_list);
>
> diff --git a/xen/include/xen/livepatch.h b/xen/include/xen/livepatch.h
> index df339a134e40..9da8b6939878 100644
> --- a/xen/include/xen/livepatch.h
> +++ b/xen/include/xen/livepatch.h
> @@ -136,35 +136,11 @@ void arch_livepatch_post_action(void);
>  void arch_livepatch_mask(void);
>  void arch_livepatch_unmask(void);
>
> -static inline void common_livepatch_apply(const struct livepatch_func *f=
unc,
> -                                          struct livepatch_fstate *state=
)
> -{
> -    /* If the action has been already executed on this function, do noth=
ing. */
> -    if ( state->applied =3D=3D LIVEPATCH_FUNC_APPLIED )
> -    {
> -        printk(XENLOG_WARNING LIVEPATCH "%s: %s has been already applied=
 before\n",
> -                __func__, func->name);
> -        return;
> -    }
> -
> -    arch_livepatch_apply(func, state);
> -    state->applied =3D LIVEPATCH_FUNC_APPLIED;
> -}
> +/* Only for testing purposes. */
> +struct payload;
> +int revert_payload(struct payload *data);
> +void revert_payload_tail(struct payload *data);
>
> -static inline void common_livepatch_revert(const struct livepatch_func *=
func,
> -                                           struct livepatch_fstate *stat=
e)
> -{
> -    /* If the apply action hasn't been executed on this function, do not=
hing. */
> -    if ( !func->old_addr || state->applied =3D=3D LIVEPATCH_FUNC_NOT_APP=
LIED )
> -    {
> -        printk(XENLOG_WARNING LIVEPATCH "%s: %s has not been applied bef=
ore\n",
> -                __func__, func->name);
> -        return;
> -    }
> -
> -    arch_livepatch_revert(func, state);
> -    state->applied =3D LIVEPATCH_FUNC_NOT_APPLIED;
> -}
>  #else
>
>  /*
> diff --git a/xen/test/livepatch/xen_action_hooks_norevert.c b/xen/test/li=
vepatch/xen_action_hooks_norevert.c
> index 3e21ade6abfc..074f8e1d56ce 100644
> --- a/xen/test/livepatch/xen_action_hooks_norevert.c
> +++ b/xen/test/livepatch/xen_action_hooks_norevert.c
> @@ -96,26 +96,14 @@ static int revert_hook(livepatch_payload_t *payload)
>
>  static void post_revert_hook(livepatch_payload_t *payload)
>  {
> -    int i;
> +    unsigned long flags;
>
>      printk(KERN_DEBUG "%s: Hook starting.\n", __func__);
>
> -    for (i =3D 0; i < payload->nfuncs; i++)
> -    {
> -        const struct livepatch_func *func =3D &payload->funcs[i];
> -        struct livepatch_fstate *fstate =3D &payload->fstate[i];
> -
> -        BUG_ON(revert_cnt !=3D 1);
> -        BUG_ON(fstate->applied !=3D LIVEPATCH_FUNC_APPLIED);
> -
> -        /* Outside of quiesce zone: MAY TRIGGER HOST CRASH/UNDEFINED BEH=
AVIOR */
> -        arch_livepatch_quiesce();
> -        common_livepatch_revert(payload);
> -        arch_livepatch_revive();
> -        BUG_ON(fstate->applied =3D=3D LIVEPATCH_FUNC_APPLIED);
> -
> -        printk(KERN_DEBUG "%s: post reverted: %s\n", __func__, func->nam=
e);
> -    }
> +    local_irq_save(flags);
> +    revert_payload(payload);

Should this check or assert the return value of revert_payload()?

Ross

> +    revert_payload_tail(payload);
> +    local_irq_restore(flags);
>
>      printk(KERN_DEBUG "%s: Hook done.\n", __func__);
>  }
> --
> 2.43.0
>


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:12:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:12:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684742.1064812 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSXH-0007L4-97; Fri, 23 Feb 2024 10:12:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684742.1064812; Fri, 23 Feb 2024 10:12:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSXH-0007Kx-63; Fri, 23 Feb 2024 10:12:07 +0000
Received: by outflank-mailman (input) for mailman id 684742;
 Fri, 23 Feb 2024 10:12:06 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jp8H=KA=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1rdSXG-0007Ki-92
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:12:06 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id feeca44a-d233-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 11:12:05 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-4128cfb6c1aso4170955e9.0
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:12:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: feeca44a-d233-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708683125; x=1709287925; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=tud49X6dABoy6o/VSHh9Xymi32eP9RvcgR36vgY0Oyw=;
        b=Z4bbhH4JeJGoyghkedVIHHq+sfrbMWtGxbk5OLh5XEeNzBscD4P8hUA+bQe2Ie1a5C
         X0PCJeC7ld1U3zIb4CA8/vwP4nBfTMjkPmUTY8sPGdit7caIZ5cm+KhVIMPdEQBwl59V
         91vreLWd37oRi5Zz/MiA9MghYvmudi3hNNRNE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708683125; x=1709287925;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=tud49X6dABoy6o/VSHh9Xymi32eP9RvcgR36vgY0Oyw=;
        b=b4/67KfNtzjAXLrUHjcls65+3V22zJlbjse4tARp4X4tsIFFM7+MA53jTBfQVUTvdC
         P+3D2DIk8ni8/UFW8/YSI7ROTcDURZ9lw7UqYHhgdDYwOJsSgIApI1jYE5prOlj4Bcmz
         6Dqukyvl2vAZrUYasVvP9aphaVN5NX1otql2QgKYFU0+BGGJFS0w+s/QLCNwBODzYJCu
         Rccx3au1APb/O0B27DkffDv0sPM7IxuYvbTBXmD7jmPF1vjP1p8XFUL4AHzhgnLG8p+W
         RbKZzPShBfbJgzn+rbY6hzGa9XkJEXJdRbAJY6GLpv+6BsAC1EwN4xYyaeOiUyngUJrL
         rWFA==
X-Gm-Message-State: AOJu0Yx6mdWT6Rt96Casfr4WSgmOODEYCnLeBILsyXlXXabY8H25X4ht
	qKYtNNgl7wXSOUWb8ld6QAqJJ1Rkam9onJ1Pz1LxqZIqViSfgDAKp/UuHwvf1vJ0GnHloQpwAPn
	pG+kg04lAnZ4L25atteHnkdIAPb9lHanUQzUN
X-Google-Smtp-Source: AGHT+IF+KpkcUfbRCrtj0JvxarGhXAeaGdOZ/RSFgUOG5AiGhlzz98xeRMDmmwzG9GntKY3PsIGG1a8KOf074jZ/k4w=
X-Received: by 2002:a05:600c:358a:b0:412:73f5:3aa5 with SMTP id
 p10-20020a05600c358a00b0041273f53aa5mr4675737wmq.16.1708683125315; Fri, 23
 Feb 2024 02:12:05 -0800 (PST)
MIME-Version: 1.0
References: <20231130142944.46322-1-roger.pau@citrix.com> <20231130142944.46322-3-roger.pau@citrix.com>
In-Reply-To: <20231130142944.46322-3-roger.pau@citrix.com>
From: Ross Lagerwall <ross.lagerwall@citrix.com>
Date: Fri, 23 Feb 2024 10:11:54 +0000
Message-ID: <CAG7k0EphLJwEoR5cx7hBTHhB+dRGtBuKFPxf+f4scWopKMrj7g@mail.gmail.com>
Subject: Re: [PATCH 2/5] xen/livepatch: search for symbols in all loaded payloads
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Nov 30, 2023 at 2:30=E2=80=AFPM Roger Pau Monne <roger.pau@citrix.c=
om> wrote:
>
> When checking if an address belongs to a patch, or when resolving a symbo=
l,
> take into account all loaded livepatch payloads, even if not applied.
>
> This is required in order for the pre-apply and post-revert hooks to work
> properly, or else Xen won't detect the intruction pointer belonging to th=
ose
> hooks as being part of the currently active text.
>
> Move the RCU handling to be used for payload_list instead of applied_list=
, as
> now the calls from trap code will iterate over the payload_list.
>
> Fixes: 8313c864fa95 ('livepatch: Implement pre-|post- apply|revert hooks'=
)
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>

Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:14:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:14:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684747.1064822 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSZ4-0007v8-M5; Fri, 23 Feb 2024 10:13:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684747.1064822; Fri, 23 Feb 2024 10:13:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSZ4-0007v1-Hs; Fri, 23 Feb 2024 10:13:58 +0000
Received: by outflank-mailman (input) for mailman id 684747;
 Fri, 23 Feb 2024 10:13:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jp8H=KA=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1rdSZ2-0007ut-Pj
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:13:56 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3fd2ae7a-d234-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 11:13:54 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-41294136c27so2080565e9.3
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:13:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3fd2ae7a-d234-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708683234; x=1709288034; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=sYx/i9GrfI+eJBfQvx0CCnnCR908o5f36pXd3cbpKyA=;
        b=INL/6K6LjkIUR2mlE1/A3n3O0GoNVEWt8cZCu2l2Um9L/EzBgaNHJdoB1E1DCQ9lZG
         aqu0lDblNSI5zoGqZg8tFrfq0InwU7IzgL+iAjR42hmys9gwpZB3eJZSk2H8UVocN2zq
         5E6f11jBppILulbtWZRr/RoboyF0D9aniHCEE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708683234; x=1709288034;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=sYx/i9GrfI+eJBfQvx0CCnnCR908o5f36pXd3cbpKyA=;
        b=r+1boFtagkk6jvCsf26t5u/uEDjJrl1TmNc3oYfJpd0/nxSSvcliQcIpqdDjWQvpso
         JP/mVy9pzkjRdfjvMB3oGtW5L3OShSjOMnXOnZN83QqvgPe1H7z5e6gju4Z47fKuf6hY
         V+1j/l/wzeYt6YvkX5RIFN4wYYnkzHREqQdgtWZXhxA/rRyFZkzY9KRX9wDpO3RnGFEa
         5KWtaGiJQ7iRJPF3kpGvl3bOf07PeJQ21OdKZ4COeUaG8OwelGgFB55ytt/ACyFsN6Ne
         3tX9uWntM7Rnem4HQAGx9OcuPoIHPuBPEbxtSHkGBOACWEuQPWPtEGjCtyING0t9rz4X
         JGVA==
X-Gm-Message-State: AOJu0YyIvRezktUt2y+1oqP0FDqA1v8LEWICMWhvrNy3MQHK1RuklNXL
	VeR/8JkxK4Jtyc7N0OzpbFU6gquQEXl4pm3FcFiI8HPURoCcpgjDbkEF8Vz2owkN/Rh+kQL1DTI
	e5OeZsAin2+QXIrnZFNgs6jji8ZKGAYQ/kOluMlqFpXBnZ0mGA3Tm
X-Google-Smtp-Source: AGHT+IF8ULCnUOBOfC87fhhptkF1WEfmbUh5b+9+pjyiUNGEXy7rRztjkyMx8yd6nnoQ1bzMK61SCnQgfG0jlXufxFA=
X-Received: by 2002:a05:600c:524f:b0:412:8fef:7f with SMTP id
 fc15-20020a05600c524f00b004128fef007fmr938713wmb.1.1708683234388; Fri, 23 Feb
 2024 02:13:54 -0800 (PST)
MIME-Version: 1.0
References: <20231130142944.46322-1-roger.pau@citrix.com> <20231130142944.46322-5-roger.pau@citrix.com>
In-Reply-To: <20231130142944.46322-5-roger.pau@citrix.com>
From: Ross Lagerwall <ross.lagerwall@citrix.com>
Date: Fri, 23 Feb 2024 10:13:43 +0000
Message-ID: <CAG7k0EpTaMiZthxdGGRm6mgxCKFw+g7JaAtCmRM3z+PzAQyKyQ@mail.gmail.com>
Subject: Re: [PATCH 4/5] xen/livepatch: fix norevert test hook setup typo
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Nov 30, 2023 at 2:31=E2=80=AFPM Roger Pau Monne <roger.pau@citrix.c=
om> wrote:
>
> The test code has a typo in using LIVEPATCH_APPLY_HOOK() instead of
> LIVEPATCH_REVERT_HOOK().
>
> Fixes: 6047104c3ccc ('livepatch: Add per-function applied/reverted state =
tracking marker')
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
>  xen/test/livepatch/xen_action_hooks_norevert.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/xen/test/livepatch/xen_action_hooks_norevert.c b/xen/test/li=
vepatch/xen_action_hooks_norevert.c
> index 074f8e1d56ce..cdfff156cede 100644
> --- a/xen/test/livepatch/xen_action_hooks_norevert.c
> +++ b/xen/test/livepatch/xen_action_hooks_norevert.c
> @@ -108,7 +108,7 @@ static void post_revert_hook(livepatch_payload_t *pay=
load)
>      printk(KERN_DEBUG "%s: Hook done.\n", __func__);
>  }
>
> -LIVEPATCH_APPLY_HOOK(revert_hook);
> +LIVEPATCH_REVERT_HOOK(revert_hook);
>
>  LIVEPATCH_PREAPPLY_HOOK(pre_apply_hook);
>  LIVEPATCH_POSTAPPLY_HOOK(post_apply_hook);
> --

Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:17:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:17:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684754.1064832 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdScd-0008Ux-3w; Fri, 23 Feb 2024 10:17:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684754.1064832; Fri, 23 Feb 2024 10:17:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdScd-0008Uq-0u; Fri, 23 Feb 2024 10:17:39 +0000
Received: by outflank-mailman (input) for mailman id 684754;
 Fri, 23 Feb 2024 10:17:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdScb-0008Uk-40
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:17:37 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c3ec05f1-d234-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 11:17:36 +0100 (CET)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2d23d301452so9015681fa.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:17:36 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 z16-20020ac81010000000b0042e44960c15sm1829642qti.52.2024.02.23.02.17.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 02:17:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3ec05f1-d234-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708683456; x=1709288256; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=PJuPG1SwJFZRoP8Zs5JfSqF/t6MzFnevhz34Cumr+LA=;
        b=ZWQ3tJye286lstslwKcqQu42RecLyvkMVe+pWqHBONX3hCawKIe70xbzaPvHjkaFrh
         DwbCJvK/xBDLOMEPjMzki85TWM9qUWiYHY04odE/sNiwdHW1+rxs8GERz+ESUCAYrRLz
         SmGzcIbT+IKBZDXnO5P9EjYuz8X9FyMmf1fGY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708683456; x=1709288256;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PJuPG1SwJFZRoP8Zs5JfSqF/t6MzFnevhz34Cumr+LA=;
        b=UAtQ4/cDfrq5gMcsQ1IBt+V7caHvgkhLMJNRwPuaGdFfhi8+h4g/7JHeRz3oGemt9K
         hEWQPUA0Q25jpdMxjvliK9dZzuP3Le9bgpiC2NNhTGffb4nR87f2nmdp0IYfJPF5/m0Y
         GioxG4vop6RcAOQ7A0lQUIiTg3yi8jNH/w0KIR+KOsZAmAbyxRqfIdoA9r8OmlHDJvJB
         QtnLchiAUOX+X0bMJ3lf4qdPe6QNUIRbpFRji6+ylhUhkJMjUUZjGMrRxMI1/8WO2xx0
         hEcwx19nc7a4YCMXz2lGIruigJ2cbHPFdgujC47vAzHgE8mfMiHowfZKhI9tKFB1tnOR
         y9Ow==
X-Gm-Message-State: AOJu0YzQKov7u6Hry3aBxvaBP5ga4yHjy05F6Bts35mQB1wyVvoKe7vl
	nUAK2CnB96/eatPCFSTGNCkhVLhxQxs/MMg5Gxo5p7ReaWNUKTelnWiP3no5glQ=
X-Google-Smtp-Source: AGHT+IFazMLQE14TL77JxmyGlT5E/Gz9A4HELcRlVGSJhrsPFC/7ZuZviGvUqT9u9QqydjRuORsnFQ==
X-Received: by 2002:ac2:5df2:0:b0:512:9f9b:ec50 with SMTP id z18-20020ac25df2000000b005129f9bec50mr1071039lfq.38.1708683455723;
        Fri, 23 Feb 2024 02:17:35 -0800 (PST)
Date: Fri, 23 Feb 2024 11:17:33 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86/spec: fix BRANCH_HARDEN option to only be set when
 build-enabled
Message-ID: <Zdhwve9-YQekWb0B@macbook>
References: <20240223094215.71889-1-roger.pau@citrix.com>
 <b5696664-0e85-414b-8360-242349a87a7e@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <b5696664-0e85-414b-8360-242349a87a7e@citrix.com>

On Fri, Feb 23, 2024 at 09:46:27AM +0000, Andrew Cooper wrote:
> On 23/02/2024 9:42 am, Roger Pau Monne wrote:
> > The current logic to handle the BRANCH_HARDEN option will report it as enabled
> > even when build-time disabled. Fix this by only allowing the option to be set
> > when support for it is built into Xen.
> >
> > Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> >  xen/arch/x86/spec_ctrl.c | 6 ++++--
> >  1 file changed, 4 insertions(+), 2 deletions(-)
> >
> > diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
> > index 421fe3f640df..e634c6b559b4 100644
> > --- a/xen/arch/x86/spec_ctrl.c
> > +++ b/xen/arch/x86/spec_ctrl.c
> > @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
> >  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
> >  int8_t __read_mostly opt_eager_fpu = -1;
> >  int8_t __read_mostly opt_l1d_flush = -1;
> > -static bool __initdata opt_branch_harden = true;
> > +static bool __initdata opt_branch_harden =
> > +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
> >  
> >  bool __initdata bsp_delay_spec_ctrl;
> >  uint8_t __read_mostly default_xen_spec_ctrl;
> > @@ -267,7 +268,8 @@ static int __init cf_check parse_spec_ctrl(const char *s)
> >              opt_eager_fpu = val;
> >          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
> >              opt_l1d_flush = val;
> > -        else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> > +        else if ( IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) &&
> > +                  (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> >              opt_branch_harden = val;
> 
> Yeah, we should definitely fix this, but could we use no_config_param()
> here for the compiled-out case ?
> 
> See cet= for an example.  If we're going to ignore what the user asks,
> we should tell them why.

Maybe I'm missing something: I've looked into using no_config_param(),
but there's no difference really, because cmdline_parse() is called
before the console is initialized, so those messages seem to be
lost.

Should this go into some kind of buffer which is then printed by
__start_xen() once the console has been initialized? (just after
printing cmdline itself).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:26:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:26:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684759.1064842 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSl0-0002He-Tk; Fri, 23 Feb 2024 10:26:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684759.1064842; Fri, 23 Feb 2024 10:26:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSl0-0002HX-Q5; Fri, 23 Feb 2024 10:26:18 +0000
Received: by outflank-mailman (input) for mailman id 684759;
 Fri, 23 Feb 2024 10:26:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0yaO=KA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rdSl0-0002HI-3P
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:26:18 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fa27b28b-d235-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 11:26:16 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a3f48422fb6so94417666b.3
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:26:16 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 bu1-20020a170907930100b00a3f99497444sm1138454ejc.138.2024.02.23.02.26.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 23 Feb 2024 02:26:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa27b28b-d235-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708683976; x=1709288776; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RwphasSeaK34H33sQimio7VghGNDm4K1S6k5frzZQ8w=;
        b=H92v3/y+yPG6Md8ubHUkJQ9au3ainvMM45Mrhn+sT5JYOosvOlclcqnHXEVC5nmlbC
         8CpZodiYurzrvHFusQB+X0iUBfgyT7BQubKk1iyTmu0GIAdS76IezKU/iKINkXfC7w0J
         VbPDtiN2Y1lwMBXnGRWGikQRiI9UTESFFZK2M=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708683976; x=1709288776;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RwphasSeaK34H33sQimio7VghGNDm4K1S6k5frzZQ8w=;
        b=kQtHh3Jx6o/X1pz9WFNXfLYOUQhv8AYWQQN146NLNpKWlGi1d34NjjMOdjWfkxkB6j
         u9f4i2SlHDYxLAiAn8XWzRbQv9SlvJwZEDxvDbQl68sp1tL9KzP4v7NvJiGvf1LYdhpr
         PK5aXAAe7Z0WnPRzvl1Q4piiTmy/+fWvn6BBoTkVvM9urmxhw79Htm1sVde4xuYnCIqK
         gxyx/3lnvAFhJPFLZa7kNZoGgtY8E2HAE59MRdb0ufLBhc9X24gXXqur0Jdr+H9t8PGV
         Dp6W0G+Bkq1fwS+W8NYSoNZk+5CY17pnNa0rcPDbpqBeOJYyYJkAn2JiqhgPMefI+wm2
         mlHA==
X-Gm-Message-State: AOJu0YwCy2VLpZdeWmGu1bSXH796FgckHtj4QO8jknAagjG+5WNuSmrp
	sQdQegumgF20j+IRys21S5fILPeT1ql7fdxtwAxlPqaHL4Q+vTEB3kQVCUL0aps=
X-Google-Smtp-Source: AGHT+IFaGDfrk7l6LCRfSoDbk0pKBOEMDxyuzBo8DLX39P/bpE48VkXKjMxdP7cawWO7YQVOPGBghw==
X-Received: by 2002:a17:906:aac9:b0:a3e:39f7:52b4 with SMTP id kt9-20020a170906aac900b00a3e39f752b4mr905701ejb.49.1708683976240;
        Fri, 23 Feb 2024 02:26:16 -0800 (PST)
Message-ID: <cf60c604-5a4a-46be-badf-1f55a760e421@citrix.com>
Date: Fri, 23 Feb 2024 10:26:15 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/spec: fix BRANCH_HARDEN option to only be set when
 build-enabled
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>
References: <20240223094215.71889-1-roger.pau@citrix.com>
 <b5696664-0e85-414b-8360-242349a87a7e@citrix.com> <Zdhwve9-YQekWb0B@macbook>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <Zdhwve9-YQekWb0B@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/02/2024 10:17 am, Roger Pau Monné wrote:
> On Fri, Feb 23, 2024 at 09:46:27AM +0000, Andrew Cooper wrote:
>> On 23/02/2024 9:42 am, Roger Pau Monne wrote:
>>> The current logic to handle the BRANCH_HARDEN option will report it as enabled
>>> even when build-time disabled. Fix this by only allowing the option to be set
>>> when support for it is built into Xen.
>>>
>>> Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>> ---
>>>  xen/arch/x86/spec_ctrl.c | 6 ++++--
>>>  1 file changed, 4 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
>>> index 421fe3f640df..e634c6b559b4 100644
>>> --- a/xen/arch/x86/spec_ctrl.c
>>> +++ b/xen/arch/x86/spec_ctrl.c
>>> @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
>>>  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
>>>  int8_t __read_mostly opt_eager_fpu = -1;
>>>  int8_t __read_mostly opt_l1d_flush = -1;
>>> -static bool __initdata opt_branch_harden = true;
>>> +static bool __initdata opt_branch_harden =
>>> +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
>>>  
>>>  bool __initdata bsp_delay_spec_ctrl;
>>>  uint8_t __read_mostly default_xen_spec_ctrl;
>>> @@ -267,7 +268,8 @@ static int __init cf_check parse_spec_ctrl(const char *s)
>>>              opt_eager_fpu = val;
>>>          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
>>>              opt_l1d_flush = val;
>>> -        else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
>>> +        else if ( IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) &&
>>> +                  (val = parse_boolean("branch-harden", s, ss)) >= 0 )
>>>              opt_branch_harden = val;
>> Yeah, we should definitely fix this, but could we use no_config_param()
>> here for the compiled-out case ?
>>
>> See cet= for an example.  If we're going to ignore what the user asks,
>> we should tell them why.
> Maybe I'm missing something: I've looked into using no_config_param(),
> but there's no difference really, because cmdline_parse() is called
> before the console is initialized, so those messages seem to be
> lost.

Look at `xl dmesg` rather than the console.  They also do appear on vga
in some configurations.

There's a separate todo to get these out in a slightly nicer way, but
they at least exist in logs.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:31:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:31:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684762.1064852 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSpV-0004Mu-EG; Fri, 23 Feb 2024 10:30:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684762.1064852; Fri, 23 Feb 2024 10:30:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSpV-0004Mn-Bb; Fri, 23 Feb 2024 10:30:57 +0000
Received: by outflank-mailman (input) for mailman id 684762;
 Fri, 23 Feb 2024 10:30:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jp8H=KA=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1rdSpU-0004Mg-B7
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:30:56 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a010a0bc-d236-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 11:30:55 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-412897a2f9eso4166575e9.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:30:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a010a0bc-d236-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708684255; x=1709289055; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=qCWjiNDCj+h0MsE3wlX/EEpGpaYvh60VyHnAyqQMKss=;
        b=XkjPUMagFFkeNpj5kgPK1kQf4x4DrRSbmf+ry8JqPryxuzpisxd5VSI1l6R+e1Hk+J
         +1dzBgv0ZMq56ifEwzQCuF/ratjsoAgFmX3mjBsDZ0ZcVYY+l/zimmYlvBgYbGCKPT7G
         lEUau1fDrqnXA28AtaJ6BjQNmBk0bKnGlV1Uo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708684255; x=1709289055;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=qCWjiNDCj+h0MsE3wlX/EEpGpaYvh60VyHnAyqQMKss=;
        b=YSPq/gvNJs1aewL8R+OM/Jz3uurz/6g/cClZa89LafXULP7XdwjgOX/O+EZip9i034
         O0hQpTxRoJjX7kzl3rrs8qcfBq4whsEX5lzAEnRIheSW7HHR7X4z6CI+Lik9ji88H+z5
         6Ia0h84Dt7VB9zNAEBnDjV3JfqMaRbimFT253jZZ+idlk2uLHgG3GZZUMT5nx4vwVwTh
         LU/ZBSCwccq6x05e1w/TNAs0O7QWvvGL5EMNQFnZLTo1OgafH6QxiPglIxdf1Q421l6M
         aQM44F0FqAKscy3Mu0VzxFI2zSd6vKgy5IuiFcglskEvZcEe3LYGslxdBeD1M7V2glIo
         KWSA==
X-Gm-Message-State: AOJu0Yz6kP0wQ8R/sob4beqkQj5CN9s7ySSDjpP5X67MbulqLwPc1sVA
	JbwqXdIw9oduNWy3TF157JgKaiI8/LipRCuGwvj9P1Iw0qKUeyeqK65PuzNQ3nOfXRH4AbBSK+v
	z1qUav0GMZaba7/j6FJqKj04rI6PLYwDZf941
X-Google-Smtp-Source: AGHT+IEmeol1KnsaUNNTUhza2l7R+0Xs3TeLmYz3KnK0XbgAo78Pwom1mSXIosnMJPZ59Dsros5Na5Eyw/EYAiIiFSw=
X-Received: by 2002:a05:600c:3ca6:b0:411:c8a7:7b09 with SMTP id
 bg38-20020a05600c3ca600b00411c8a77b09mr1353538wmb.10.1708684254679; Fri, 23
 Feb 2024 02:30:54 -0800 (PST)
MIME-Version: 1.0
References: <20231130142944.46322-1-roger.pau@citrix.com> <20231130142944.46322-6-roger.pau@citrix.com>
In-Reply-To: <20231130142944.46322-6-roger.pau@citrix.com>
From: Ross Lagerwall <ross.lagerwall@citrix.com>
Date: Fri, 23 Feb 2024 10:30:43 +0000
Message-ID: <CAG7k0EpCJ9h=d4VF8GycrmxbVRQwT+=bB0GfPE_R=dcFRh7uBQ@mail.gmail.com>
Subject: Re: [PATCH 5/5] xen/livepatch: properly build the noapply and
 norevert tests
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Nov 30, 2023 at 2:31=E2=80=AFPM Roger Pau Monne <roger.pau@citrix.c=
om> wrote:
>
> It seems the build variables for those tests where copy-pasted from
> xen_action_hooks_marker-objs and not adjusted to use the correct source f=
iles.
>
> Fixes: 6047104c3ccc ('livepatch: Add per-function applied/reverted state =
tracking marker')
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
>  xen/test/livepatch/Makefile | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/xen/test/livepatch/Makefile b/xen/test/livepatch/Makefile
> index c258ab0b5940..d987a8367f15 100644
> --- a/xen/test/livepatch/Makefile
> +++ b/xen/test/livepatch/Makefile
> @@ -118,12 +118,12 @@ xen_action_hooks_marker-objs :=3D xen_action_hooks_=
marker.o xen_hello_world_func.o
>  $(obj)/xen_action_hooks_noapply.o: $(obj)/config.h
>
>  extra-y +=3D xen_action_hooks_noapply.livepatch
> -xen_action_hooks_noapply-objs :=3D xen_action_hooks_marker.o xen_hello_w=
orld_func.o note.o xen_note.o
> +xen_action_hooks_noapply-objs :=3D xen_action_hooks_noapply.o xen_hello_=
world_func.o note.o xen_note.o
>
>  $(obj)/xen_action_hooks_norevert.o: $(obj)/config.h
>
>  extra-y +=3D xen_action_hooks_norevert.livepatch
> -xen_action_hooks_norevert-objs :=3D xen_action_hooks_marker.o xen_hello_=
world_func.o note.o xen_note.o
> +xen_action_hooks_norevert-objs :=3D xen_action_hooks_norevert.o xen_hell=
o_world_func.o note.o xen_note.o
>
>  EXPECT_BYTES_COUNT :=3D 8
>  CODE_GET_EXPECT=3D$(shell $(OBJDUMP) -d --insn-width=3D1 $(1) | sed -n -=
e '/<'$(2)'>:$$/,/^$$/ p' | tail -n +2 | head -n $(EXPECT_BYTES_COUNT) | aw=
k '{$$0=3D$$2; printf "%s", substr($$0,length-1)}' | sed 's/.\{2\}/0x&,/g' =
| sed 's/^/{/;s/,$$/}/g')
> --

Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:35:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:35:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684766.1064862 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSuJ-00051G-31; Fri, 23 Feb 2024 10:35:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684766.1064862; Fri, 23 Feb 2024 10:35:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdSuJ-000519-08; Fri, 23 Feb 2024 10:35:55 +0000
Received: by outflank-mailman (input) for mailman id 684766;
 Fri, 23 Feb 2024 10:35:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0yaO=KA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rdSuH-000513-Ru
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:35:53 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 51a41c3c-d237-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 11:35:53 +0100 (CET)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a3f4464c48dso77615066b.3
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:35:53 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 qo3-20020a170907874300b00a3e4efbfdacsm5848002ejc.225.2024.02.23.02.35.51
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 02:35:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51a41c3c-d237-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708684552; x=1709289352; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=xwyptn1bE6WztZatqA7hkN5iai3sDIG7sJEM1euaKoI=;
        b=tnyV/xRiZWfo45JcqQVzUWiq/Va86KAZ3kb1nv6X+CLOyekNre0oFZcE+pvjXGxHmH
         NaO+Pp4UXKWozBNOmAtlsE4h0OhfgYbGlK6VEt5HcscXximEq7RsvvnQk1V4FDcDgopH
         ItgLWPW7SS049ZMsdFLGxP9oGFHv0Ps2MT4vA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708684552; x=1709289352;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=xwyptn1bE6WztZatqA7hkN5iai3sDIG7sJEM1euaKoI=;
        b=NSaYTcRDVyMllzF5S9ufloBMFLoyER+pQg3lXna1gcUVIjNtEW6Smf6/L8jbzBZXNe
         Ela676/PLAg64T8yfmMHpJCiUIb9fuwu1zRwaa7/sdc4LwGKkGVfPJb8WIPu6SNfYrzI
         8RxycGAa/wZRV333l8yzGxN8mxH+RyIu21I29HJKzDkBwjTIj6bkYKbnt4hoDZUG9l7F
         G5hLPgm/VRZEc9pwcvtl8DCZHPengvi3SacOg/QNdZg7nTOGAIvXCRAxGuPVjAAW9WZ+
         vhr+BiKnp1Y0AryqUY5ZJE8OZpOgYKgMzi99YVaWrawISfQCJV+PI5FO223BYNxU+E7C
         aQ/g==
X-Gm-Message-State: AOJu0YzYDVlQO8W/Rs//bom+BhdakhjrJRmAjYpo3E5gr4oQefnygTRK
	667FPBcvpIMdynE4TLKvM1HB/q8NPSW2I16Sr+ZLPUZdefgRZqWIgnhEQHQbxFL1UhiB4P6JgfQ
	E
X-Google-Smtp-Source: AGHT+IF4cX7OjdPJKpaxLe4BF/4+9ASstV92JjsetBxaxcGs0gSd3EwTstH2z3rTqPJwY4znjcEXUA==
X-Received: by 2002:a17:906:f94b:b0:a3e:980c:699d with SMTP id ld11-20020a170906f94b00b00a3e980c699dmr909759ejb.57.1708684551996;
        Fri, 23 Feb 2024 02:35:51 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>
Subject: [PATCH LIVEPATCH] Fix inclusion of new object files
Date: Fri, 23 Feb 2024 10:35:50 +0000
Message-Id: <20240223103550.299992-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Right now, there's a mixup over the xen/ part of the path for new files.

  + NEW_FILES=./arch/x86/lp-hooks.o
  + for i in '$NEW_FILES'
  ++ dirname ./arch/x86/lp-hooks.o
  + mkdir -p output/./arch/x86
  + cp patched/./arch/x86/lp-hooks.o output/./arch/x86/lp-hooks.o
  cp: cannot stat 'patched/./arch/x86/lp-hooks.o': No such file or directory

Alter the `cd` and `find` runes to use paths relative to the root of the
different source trees.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Ross Lagerwall <ross.lagerwall@citrix.com>

I'm unsure whether there's a useful fixes tag to use.  AFAICT it was broken
even when 0564f0c7a57 claimed to have it working.
---
 livepatch-build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/livepatch-build b/livepatch-build
index cdb852cc7fea..948b2acfc2f6 100755
--- a/livepatch-build
+++ b/livepatch-build
@@ -227,7 +227,7 @@ function create_patch()
         fi
     done
 
-    NEW_FILES=$(comm -23 <(cd patched/xen && find . -type f -name '*.o' | sort) <(cd original/xen && find . -type f -name '*.o' | sort))
+    NEW_FILES=$(comm -23 <(cd patched && find xen/ -type f -name '*.o' | sort) <(cd original && find xen/ -type f -name '*.o' | sort))
     for i in $NEW_FILES; do
         mkdir -p "output/$(dirname "$i")"
         cp "patched/$i" "output/$i"

base-commit: 1b5b03b3ce4187ce99bad580fd0ee36c6337313f
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:42:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:42:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684772.1064873 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdT0R-00079m-PZ; Fri, 23 Feb 2024 10:42:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684772.1064873; Fri, 23 Feb 2024 10:42:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdT0R-00079f-Kb; Fri, 23 Feb 2024 10:42:15 +0000
Received: by outflank-mailman (input) for mailman id 684772;
 Fri, 23 Feb 2024 10:42:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jp8H=KA=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1rdT0Q-00079Z-OU
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:42:14 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 348e84a4-d238-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 11:42:13 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-512cca90f38so796218e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:42:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 348e84a4-d238-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708684933; x=1709289733; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=OfbuVucXTMtEm7kzIgqF5TiJSPGTVWUBfO2y9Xa82TE=;
        b=Z3X55NPPTLVHvZZmbzkm/4eYXgvS8ihDpawaxqk5RVemNIS5JeUWeAyW5vtDArYd2C
         Zj5S9pSrrppMRFu+CfyVO2MbdAV0YuKel6Zj9WtN0HynFlFOQetF9p3HfB1w+rcRlM3h
         6l45RQEQu8bfnRJUWhSOeF1gEJp2WXjFlRjEU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708684933; x=1709289733;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=OfbuVucXTMtEm7kzIgqF5TiJSPGTVWUBfO2y9Xa82TE=;
        b=ZTljkitv562O6LW802kRkJ0pYVkaBv6ageXmgJu5gClEwOgmJo31Uu/RZ5PZyUQ9NA
         k8sAdSN97wqdFnKUcONEmpnbNhs6HinVyJ5pUnH3EU01WjRD8MYfkK8CQZOB038GjKtv
         AD6DUfLNZ0SLG0qdT+fnwO6vIRGAFgBzqxhQZ5xW+777qVaQULfIGzWJEYUidWz27hJr
         YF23aJr5zqXTQPdNCRXH6Bd9eeIPJsPfVtjdZR30hUbLQn/6/Vdt/UdcCcAQC8FKdIre
         8OIlwdpu87Y0OZhReznhYT/exZXrqbok6CXHx1xSSaGSYkH6Rx7/f65w+LKmwZI0HWhF
         rj+Q==
X-Gm-Message-State: AOJu0YxSgssNsgxMhPg11vkJk2hO0lwZAn0kr+0z/+r2w1I2efWaM3dL
	tr4wWQHXSUD3jWWDyEGO6U3g2JuO6uvzqTsucMYfAinLtkpIl6DOPaty/Cy30MvwSkAPr1b0XUr
	vtt3HjrXtXW+oHHyqK6JQHstzEON+fjZKx5Gh
X-Google-Smtp-Source: AGHT+IH86tpGVKxziV/xtEnL1Z62l7b6NDVMFmNFnFBRwIbAiSxsSf6Ht9k4f280umEB5Y5JOi45ov9AfWb0CEglNbg=
X-Received: by 2002:ac2:5fa8:0:b0:512:ea66:9852 with SMTP id
 s8-20020ac25fa8000000b00512ea669852mr576108lfe.52.1708684933207; Fri, 23 Feb
 2024 02:42:13 -0800 (PST)
MIME-Version: 1.0
References: <20240223103550.299992-1-andrew.cooper3@citrix.com>
In-Reply-To: <20240223103550.299992-1-andrew.cooper3@citrix.com>
From: Ross Lagerwall <ross.lagerwall@citrix.com>
Date: Fri, 23 Feb 2024 10:42:01 +0000
Message-ID: <CAG7k0Eryh734q2=ejp2XHLnZVC6ksi0jsV_ftPHh6vQPP5vyiA@mail.gmail.com>
Subject: Re: [PATCH LIVEPATCH] Fix inclusion of new object files
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Feb 23, 2024 at 10:35=E2=80=AFAM Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
>
> Right now, there's a mixup over the xen/ part of the path for new files.
>
>   + NEW_FILES=3D./arch/x86/lp-hooks.o
>   + for i in '$NEW_FILES'
>   ++ dirname ./arch/x86/lp-hooks.o
>   + mkdir -p output/./arch/x86
>   + cp patched/./arch/x86/lp-hooks.o output/./arch/x86/lp-hooks.o
>   cp: cannot stat 'patched/./arch/x86/lp-hooks.o': No such file or direct=
ory
>
> Alter the `cd` and `find` runes to use paths relative to the root of the
> different source trees.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> CC: Ross Lagerwall <ross.lagerwall@citrix.com>
>
> I'm unsure whether there's a useful fixes tag to use.  AFAICT it was brok=
en
> even when 0564f0c7a57 claimed to have it working.
> ---
>  livepatch-build | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/livepatch-build b/livepatch-build
> index cdb852cc7fea..948b2acfc2f6 100755
> --- a/livepatch-build
> +++ b/livepatch-build
> @@ -227,7 +227,7 @@ function create_patch()
>          fi
>      done
>
> -    NEW_FILES=3D$(comm -23 <(cd patched/xen && find . -type f -name '*.o=
' | sort) <(cd original/xen && find . -type f -name '*.o' | sort))
> +    NEW_FILES=3D$(comm -23 <(cd patched && find xen/ -type f -name '*.o'=
 | sort) <(cd original && find xen/ -type f -name '*.o' | sort))
>      for i in $NEW_FILES; do
>          mkdir -p "output/$(dirname "$i")"
>          cp "patched/$i" "output/$i"
>
> base-commit: 1b5b03b3ce4187ce99bad580fd0ee36c6337313f
> --

Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:43:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:43:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684775.1064882 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdT1T-0007fV-1A; Fri, 23 Feb 2024 10:43:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684775.1064882; Fri, 23 Feb 2024 10:43:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdT1S-0007fO-Ti; Fri, 23 Feb 2024 10:43:18 +0000
Received: by outflank-mailman (input) for mailman id 684775;
 Fri, 23 Feb 2024 10:43:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6083=KA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rdT1R-0007fI-IC
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:43:17 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 59db54d8-d238-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 11:43:16 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-55f50cf2021so917123a12.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:43:16 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l15-20020a056402124f00b00564c8800f66sm3201461edw.14.2024.02.23.02.43.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 23 Feb 2024 02:43:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 59db54d8-d238-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708684996; x=1709289796; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KgRvYUCzO9ua3jx3wY953h1RsOgnIZJoPBbhabQAgvA=;
        b=RoP4Bed3Q8iodn22LB0iTIvm91uzqIcK+VbxjOYi459ZXynInYecM9wFigSPcaxspW
         Vn1I7y4z2DF8aHnAfbOOwhF4pZ1aWE65EQtaIHEv4iVcGYBuQyUE7i2GVM6yMfp5p7oZ
         fo4IBUFy6ls1coQyNpNEj5VZwfewgfPVkIiKGh/gPrXoGdco9mWon9Stc2AvMbApu2QE
         waq7XKC0J97iuh/6JS0WCcjIWTVaGbyw2ba5S9RFUaHilmjn7A3mqFJaJI671AKkIHm0
         PEuzdtl81s15PuRLhVWs+KUBoDEX8ZBkxmco1GxYnA7N7RNyTmlogUW2LfY4ZmxS7/f2
         4pfw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708684996; x=1709289796;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KgRvYUCzO9ua3jx3wY953h1RsOgnIZJoPBbhabQAgvA=;
        b=l47Hr8Q5lXdnqCMkZChQoY1RsPLlybGwJGScigv/No/iXrBxZhI/beymnNtBLdbqst
         YcGvk7mpGUdkbeQAJX/e7WknNuaWJycJzmItQ1qic/mT4WW3urwEhGARHxESVlgd75IP
         SW7MK2vyjz/ZCo+iCEnsVq8WjV0xXv0MK7Xi5GnGDqDjRcbOXCS3kHM1XiK74lM1J/x1
         hXHwq5EKoAD4Uj1hi0QhywbRxEsZvuZ7OjVbUQH/vl5DaWPBnC5DNfby1Q71gxR7tfGz
         Jq6mudimmNtJLR5+Yteg9W7fGgRQ8xDKmk7rLgtFy8ODLwvSlXyQB2I3EuDPsAdI+s8A
         QY6A==
X-Forwarded-Encrypted: i=1; AJvYcCUOQK2pPGxJF4dvLiN/Eo+swXfjEG2kInOi3zrhlLSNos9/ShBte+Cv0pN7I1U3TPN2sWugLHLOWdQA08aTz+SYtHuO2OKmrnOeSVNR8n4=
X-Gm-Message-State: AOJu0Yx38biafHqZRqR7gDN/C6wgkqQ2WbSioYjuVvUr1j1oQgI7ma9W
	8m2P0EPgiPUroyqwYBSsl+Oq8qRxXf8WlLe9MLrhb1Mr2wOltH8K6Z9c8lgVLQ==
X-Google-Smtp-Source: AGHT+IE/xsuzIbBqb9tUoFzg4G9E8Ymr0Ym7dT86fz2lSweyCkLfET3uslwp/X4AduYio3bjHqJXyg==
X-Received: by 2002:aa7:d3d4:0:b0:565:5924:9d8b with SMTP id o20-20020aa7d3d4000000b0056559249d8bmr918908edr.35.1708684995909;
        Fri, 23 Feb 2024 02:43:15 -0800 (PST)
Message-ID: <b62f223d-8baa-4d5b-aeba-9890c101b28c@suse.com>
Date: Fri, 23 Feb 2024 11:43:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/altcall: use an union as register type for
 function parameters
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 Matthew Grooms <mgrooms@shrew.net>, xen-devel@lists.xenproject.org
References: <20240222164455.67248-1-roger.pau@citrix.com>
 <b1c96b93-fd75-4077-b622-4781b7811ee8@suse.com> <ZdhjDBlHRtO4MIhM@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZdhjDBlHRtO4MIhM@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.02.2024 10:19, Roger Pau Monné wrote:
> On Thu, Feb 22, 2024 at 05:55:00PM +0100, Jan Beulich wrote:
>> On 22.02.2024 17:44, Roger Pau Monne wrote:
>>> --- a/xen/arch/x86/include/asm/alternative.h
>>> +++ b/xen/arch/x86/include/asm/alternative.h
>>> @@ -167,9 +167,25 @@ extern void alternative_branches(void);
>>>  #define ALT_CALL_arg5 "r8"
>>>  #define ALT_CALL_arg6 "r9"
>>>  
>>> +#ifdef CONFIG_CC_IS_CLANG
>>> +/*
>>> + * Use an union with an unsigned long in order to prevent clang from skipping a
>>> + * possible truncation of the value.  By using the union any truncation is
>>> + * carried before the call instruction.
>>> + * https://github.com/llvm/llvm-project/issues/82598
>>> + */
>>
>> I think it needs saying that this is relying on compiler behavior not
>> mandated by the standard, thus explaining why it's restricted to
>> Clang (down the road we may even want to restrict to old versions,
>> assuming they fix the issue at some point). Plus also giving future
>> readers a clear understanding that if something breaks with this, it's
>> not really a surprise.
> 
> What about:
> 
> Use a union with an unsigned long in order to prevent clang from
> skipping a possible truncation of the value.  By using the union any
> truncation is carried before the call instruction.

..., in turn covering for ABI-non-compliance in that the necessary
clipping / extension of the value is supposed to be carried out in
the callee.

>  Note this
> behavior is not mandated by the standard, and hence could stop being
> a viable workaround, or worse, could cause a different set of
> code-generation issues in future clang versions.
> 
> This has been reported upstream at:
> https://github.com/llvm/llvm-project/issues/82598
> 
>> Aiui this bug is only a special case of the other, much older one, so
>> referencing that one here too would seem advisable.
> 
> My report has been resolved as a duplicate of:
> 
> https://github.com/llvm/llvm-project/issues/43573
> 
> FWIW, I think for the context the link is used in (altcall) my bug
> report is more representative, and readers can always follow the trail
> into the other inter-related bugs.

While true, the comment extension suggested above goes beyond that
territory, and there the other bug is quite relevant directly. After all
what your change does is papering over a knock-on effect of them not
following the ABI. And that simply because it is pretty hard to see how
we could work around the ABI non-conformance itself (which btw could
bite us if we had any affected C function called from assembly).

43537 looks to be a newer instance of 12579; funny they didn't close
that as a duplicate then, too.

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:46:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:46:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684778.1064891 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdT4G-0008G4-DT; Fri, 23 Feb 2024 10:46:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684778.1064891; Fri, 23 Feb 2024 10:46:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdT4G-0008Fx-Ak; Fri, 23 Feb 2024 10:46:12 +0000
Received: by outflank-mailman (input) for mailman id 684778;
 Fri, 23 Feb 2024 10:46:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdT4F-0008Fr-Fj
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:46:11 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c0932f3b-d238-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 11:46:08 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-512a65cd2c7so1049149e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:46:08 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 mw6-20020a05621433c600b0068fda3873b6sm429509qvb.110.2024.02.23.02.46.07
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 02:46:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c0932f3b-d238-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708685168; x=1709289968; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=43f4rOlcrZts1hyjm0iyNDxb3Fodzfg94cEQ/eHhvSc=;
        b=rmXx+f+nRFKsNKSD6JFOPQKx6N0qgxVcjVVIMt0G6GShVG9JLThndECBK4FL/MPFvP
         cjNmBN55hY4yflYaRl2DwGvuSpMlhgKjWCj2mQJR+znyHi0q2rzf4imOTiabqPSoNkBL
         53lDDSuL3LltwBP5Fb2PHrAYsRm2bTI5eUBpI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708685168; x=1709289968;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=43f4rOlcrZts1hyjm0iyNDxb3Fodzfg94cEQ/eHhvSc=;
        b=TKcSo18xGC/DXSf0qHnzvMLtdWgXXUOrzvoHbsvdYg+6Le/MsClr+6LtBmIJScTgNj
         J8m3S9C82E8knuKzrwl2hu050qNQN88du5/EehzEZTDSdMi8ZJBxkmjlEDG7ZDQPDFVx
         SpQkImgHKgy5dz2OHVChKH4rMTkWw+qvZGU0Wi92toVMNm4PNe4KduTKkpj9Rn0ChDeQ
         g1yVNxXUhjxHxjozswLywCmxlzcT9hbylfFzzeGQfPjkXh3xp6CwjF9ImZLBrp7vMqyg
         0CdMWW92HfgjtCJ7eE3IBIqHvz9FjK22S4tge1rLwTHic0toLu/LERaOB90k0/vVN2sJ
         b2KQ==
X-Gm-Message-State: AOJu0YwhnHfR3VRxshuVmE8geqFFV5eem9kcGXV3mdatNpkO+rKqnDwA
	zdf3SHfrrrK13pIs7uU4zvCd+LS0BIPQaje0ZenRCIOm1bODwXuTjlHAk4h9Fbr3kkw4x5gr7A2
	t
X-Google-Smtp-Source: AGHT+IHiWfMgBNi5VxsY+XJjtihpkobcAUtpnUTpml2CY5MU8SmW/hhEPNqh0zJu7KbzhyH3P3GotA==
X-Received: by 2002:a05:6512:505:b0:512:b0c4:71c8 with SMTP id o5-20020a056512050500b00512b0c471c8mr1200597lfb.29.1708685168237;
        Fri, 23 Feb 2024 02:46:08 -0800 (PST)
Date: Fri, 23 Feb 2024 11:46:05 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Ross Lagerwall <ross.lagerwall@citrix.com>
Subject: Re: [PATCH LIVEPATCH] Fix inclusion of new object files
Message-ID: <Zdh3bfjS1vqbnEdC@macbook>
References: <20240223103550.299992-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240223103550.299992-1-andrew.cooper3@citrix.com>

On Fri, Feb 23, 2024 at 10:35:50AM +0000, Andrew Cooper wrote:
> Right now, there's a mixup over the xen/ part of the path for new files.
> 
>   + NEW_FILES=./arch/x86/lp-hooks.o
>   + for i in '$NEW_FILES'
>   ++ dirname ./arch/x86/lp-hooks.o
>   + mkdir -p output/./arch/x86
>   + cp patched/./arch/x86/lp-hooks.o output/./arch/x86/lp-hooks.o
>   cp: cannot stat 'patched/./arch/x86/lp-hooks.o': No such file or directory
> 
> Alter the `cd` and `find` runes to use paths relative to the root of the
> different source trees.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Hm, this never worked AFAICT.

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 10:48:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 10:48:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684784.1064902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdT6E-0000MU-PM; Fri, 23 Feb 2024 10:48:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684784.1064902; Fri, 23 Feb 2024 10:48:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdT6E-0000MN-MU; Fri, 23 Feb 2024 10:48:14 +0000
Received: by outflank-mailman (input) for mailman id 684784;
 Fri, 23 Feb 2024 10:48:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6083=KA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rdT6E-0000MH-4R
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 10:48:14 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0a16dfa9-d239-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 11:48:12 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a3122b70439so35972766b.3
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 02:48:12 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 cu11-20020a170906ba8b00b00a3ee5c19ee5sm3702290ejd.109.2024.02.23.02.48.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 23 Feb 2024 02:48:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a16dfa9-d239-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708685291; x=1709290091; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PTF254Ui2YukHBaZx1UjzeAVwpyYnHmdjc0m2ICWVMY=;
        b=Wc8btW/PktMf1FEzU927BJY668Zdt69qZqpuNVvZZC65FOiQtWdVgywciNktyQHat4
         vcdOJFOL/xZum51GRSioNMwV5eYaEx1n5ESafoNaf5Yw/W71VSPx7UyeNsEtMJkj039e
         s/1nlQSlyyp69gg3J3nFimc8yspuM4NVmVu4NWfNXxK780TAtt6Ke+O91QL9M/tE0ir2
         85sJGfma8mltswIF+o0nUS29m1J2WNlCQmMo26ZNJ/Jcs7jEwARZ+Zqv9tNyUp8XCKAb
         aIBzLnY2Eh+Kow3+0K8KXyKrP3uKOi8MTWWJAKERoHEzEfvUnz/GNBRIj0m5O936Bs6B
         v/3g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708685291; x=1709290091;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PTF254Ui2YukHBaZx1UjzeAVwpyYnHmdjc0m2ICWVMY=;
        b=ROKGYaB80WOOhjCx+rRU6Ei028QJEqwRT44IG1TCcHeu4L4pkY+wR2eoDDIzUCa4ft
         FqmjyqU3zn/ZrM2cgUTBAq41zGU1mbN6nXsONg9/WD75pvdEasJcAMUvSYMqoOr0b/ej
         aD3bmMFgcaJdwP3fzUdKA5WQg+vHqdeYepnD1IY0UR0PBazer7a1Ww6j0/RQD79L1uhG
         2gz9lrB21Z1VLnbxk/cSzj1DPC0wsvV+1MCL0Wv2ZlFMc/nfUU2k7N7f6Ht45ebuYQzX
         u4WXMoz+uURJOwpFqGuXGYir5P74kBxh/19A6L3XfGCoj8zCVh7H7tJfxIBVSoUNEGaq
         9q9w==
X-Forwarded-Encrypted: i=1; AJvYcCUrr4Ux9W9McRY2ZtFzVUc4qwHeOcTc7FP2BWgwlONRmnWin4+Kk8WJzNl3aosEYXhmEYRXDe/mwaEbcJOOCZXmxrt+sWn2AYC0Q0Cc5UM=
X-Gm-Message-State: AOJu0YxzWDlGN4+PF2MzfgRSrgIGt4qTVwNAvI7OFRlyL4qOsm+w0rCe
	Ki+CnpSy6+GLFJGZ3FX0Vx48/pd930cDEF5m8U9ASS0KlHHYVwWlAvRMrhEG0w==
X-Google-Smtp-Source: AGHT+IFekVhUB2r/pkY53WQNPv3Wa1VtthywsZqroK0m/ZbwdV+zXEzaP0H9T8AcQYIzK8rlwD2NQA==
X-Received: by 2002:a17:906:3e01:b0:a3e:7cd8:3db7 with SMTP id k1-20020a1709063e0100b00a3e7cd83db7mr923779eji.68.1708685291561;
        Fri, 23 Feb 2024 02:48:11 -0800 (PST)
Message-ID: <6143d253-1d5c-48c8-81b9-3652b21dd9c0@suse.com>
Date: Fri, 23 Feb 2024 11:48:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 0/5] xen/livepatch: fixes for the pre-apply / post-revert
 hooks
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231130142944.46322-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20231130142944.46322-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 30.11.2023 15:29, Roger Pau Monne wrote:
> Hello,
> 
> The follow series contain a misc of fixes mostly related to the usage of
> the pre-apply / post-revert hooks.  The norevert test is also fixed to
> work as I think was expected.  Finally both the no{apply,revert}
> tests are fixed to build properly, as the files where previously
> unhooked from the build system completely.
> 
> I'm unsure how useful the apply and revert hooks really are, as without
> calling the internal apply/revert functions the state of the payload
> structure is quite likely inconsistent with the code expectations.
> 
> Thanks, Roger.
> 
> Roger Pau Monne (5):
>   xen/livepatch: register livepatch regions when loaded
>   xen/livepatch: search for symbols in all loaded payloads
>   xen/livepatch: fix norevert test attempt to open-code revert
>   xen/livepatch: fix norevert test hook setup typo
>   xen/livepatch: properly build the noapply and norevert tests

With the R-b-s that have arrived, could you clarify in how far these
can be committed out of order?

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 11:12:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 11:12:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684796.1064911 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdTT2-0005yq-QA; Fri, 23 Feb 2024 11:11:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684796.1064911; Fri, 23 Feb 2024 11:11:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdTT2-0005yj-Na; Fri, 23 Feb 2024 11:11:48 +0000
Received: by outflank-mailman (input) for mailman id 684796;
 Fri, 23 Feb 2024 11:11:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdTT1-0005yd-I1
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 11:11:47 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 552bae5c-d23c-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 12:11:46 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-512e4f4e463so1003536e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 03:11:46 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 on12-20020a056214448c00b0068f51bfb6c8sm7346215qvb.132.2024.02.23.03.11.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 03:11:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 552bae5c-d23c-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708686706; x=1709291506; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=F9fH0kJ9IeL3G35Vgyq2Tqf5MGK5iAc0aBgni/nudck=;
        b=RqXXF85VkZ4cR56xXVOJRczRXQk2KZpgQUjwl1crxJ/n5rUWxyij+E90qgRu+V/MS/
         z0uoYJrtLonIfLKR0Fu0hBJTZ/8RnrRfKwRyLnBXUh7ZjOlas/b1eW+yVeGm3QHiruUM
         JxZkf5gUWa6NNWAjglM4gOXYC/COlVchJJD5w=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708686706; x=1709291506;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=F9fH0kJ9IeL3G35Vgyq2Tqf5MGK5iAc0aBgni/nudck=;
        b=SZHYhMxF6cLMHtE/vZWcAa9cUXOUbHpv3wd7lqHefmkFekSx8emwOFK7Bn+ufSc7Hu
         KPk1OWtsqokA2S+C+o0GXMpZrOWS4ZC0uM2tDfxzzzRhe8fr/Krhj5XBmX44+y4DJdm4
         nCvSJWz1JERNV2YbTcljhjHa+3dZs+iMvDI2nVpkpP96DZqUofxO3ZIi47t1iH0WB+nR
         UJa4YdyDt0RyYx6oZR8+W6CDuqYiYvTH5rUW9mSIjadS6FpOmatn6r9c06zPFlyfnpMX
         r5vi7PTcIj2KMTrtyMNisngG5Zw/NnPiw3W7NqEDHcZgDWlLaM/gMXSn3b4sQgowefnZ
         YjMg==
X-Gm-Message-State: AOJu0Yzy6TKfAasMRbUIGS9JQ/uE7kQ3r4F5LE2a9IVq787BupwcHc9X
	U4z9tFjfcoTJgHyyuf1lvO8AcNBYA6MrBM7jGOOMhcLeMjMMStHyXGHUhLFBdx8=
X-Google-Smtp-Source: AGHT+IFic857UkGzEfMM8UHD2ptpj2AXvv92ERn5KaOJ2Bp2F33GxVE48aOzF9Z8Ftpdk++Hic3b/w==
X-Received: by 2002:a05:6512:e9f:b0:512:e5bf:f342 with SMTP id bi31-20020a0565120e9f00b00512e5bff342mr1417708lfb.37.1708686705990;
        Fri, 23 Feb 2024 03:11:45 -0800 (PST)
Date: Fri, 23 Feb 2024 12:11:43 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86/spec: fix BRANCH_HARDEN option to only be set when
 build-enabled
Message-ID: <Zdh9b4vNHfJnFXPR@macbook>
References: <20240223094215.71889-1-roger.pau@citrix.com>
 <b5696664-0e85-414b-8360-242349a87a7e@citrix.com>
 <Zdhwve9-YQekWb0B@macbook>
 <cf60c604-5a4a-46be-badf-1f55a760e421@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <cf60c604-5a4a-46be-badf-1f55a760e421@citrix.com>

On Fri, Feb 23, 2024 at 10:26:15AM +0000, Andrew Cooper wrote:
> On 23/02/2024 10:17 am, Roger Pau Monné wrote:
> > On Fri, Feb 23, 2024 at 09:46:27AM +0000, Andrew Cooper wrote:
> >> On 23/02/2024 9:42 am, Roger Pau Monne wrote:
> >>> The current logic to handle the BRANCH_HARDEN option will report it as enabled
> >>> even when build-time disabled. Fix this by only allowing the option to be set
> >>> when support for it is built into Xen.
> >>>
> >>> Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
> >>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> >>> ---
> >>>  xen/arch/x86/spec_ctrl.c | 6 ++++--
> >>>  1 file changed, 4 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
> >>> index 421fe3f640df..e634c6b559b4 100644
> >>> --- a/xen/arch/x86/spec_ctrl.c
> >>> +++ b/xen/arch/x86/spec_ctrl.c
> >>> @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
> >>>  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
> >>>  int8_t __read_mostly opt_eager_fpu = -1;
> >>>  int8_t __read_mostly opt_l1d_flush = -1;
> >>> -static bool __initdata opt_branch_harden = true;
> >>> +static bool __initdata opt_branch_harden =
> >>> +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
> >>>  
> >>>  bool __initdata bsp_delay_spec_ctrl;
> >>>  uint8_t __read_mostly default_xen_spec_ctrl;
> >>> @@ -267,7 +268,8 @@ static int __init cf_check parse_spec_ctrl(const char *s)
> >>>              opt_eager_fpu = val;
> >>>          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
> >>>              opt_l1d_flush = val;
> >>> -        else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> >>> +        else if ( IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) &&
> >>> +                  (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> >>>              opt_branch_harden = val;
> >> Yeah, we should definitely fix this, but could we use no_config_param()
> >> here for the compiled-out case ?
> >>
> >> See cet= for an example.  If we're going to ignore what the user asks,
> >> we should tell them why.
> > Maybe I'm missing something: I've looked into using no_config_param(),
> > but there's no difference really, because cmdline_parse() is called
> > before the console is initialized, so those messages seem to be
> > lost.
> 
> Look at `xl dmesg` rather than the console.  They also do appear on vga
> in some configurations.

Oh, my internal buffer was too small on those also got truncated, had
to bump it.

> There's a separate todo to get these out in a slightly nicer way, but
> they at least exist in logs.

I've created:

https://gitlab.com/xen-project/xen/-/issues/184

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 11:13:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 11:13:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684798.1064922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdTV6-0006WA-6I; Fri, 23 Feb 2024 11:13:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684798.1064922; Fri, 23 Feb 2024 11:13:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdTV6-0006W3-37; Fri, 23 Feb 2024 11:13:56 +0000
Received: by outflank-mailman (input) for mailman id 684798;
 Fri, 23 Feb 2024 11:13:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0yaO=KA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rdTV4-0006Ve-Ru
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 11:13:54 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a04e007e-d23c-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 12:13:52 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-512bde3d197so170186e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 03:13:52 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 e11-20020a05620a12cb00b00787b6750619sm304075qkl.130.2024.02.23.03.13.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 23 Feb 2024 03:13:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a04e007e-d23c-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708686832; x=1709291632; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KGKWwxph69B4iBv+6rJEvJhMpAG8cYrWDwfcQGnn0OM=;
        b=qCDzR65ldPq41oBbxJS9cEjVqbZM6fRLqjkJI4Z8LwnGbu/2l4FqWFp0wKRyFa8CWh
         xD89cM0mpyQX0Xkq6+k8pZHOXgbI7JgRpLwgIqwHHXxBz6nY1/BoFxKzpPRrJoS/a6RY
         0GKm2jLCf/pREKtK2KkSM+r14y9dP5mZLPvvk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708686832; x=1709291632;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KGKWwxph69B4iBv+6rJEvJhMpAG8cYrWDwfcQGnn0OM=;
        b=phft936HYVk7iUUUsn0BSEC6Tw+6K9eHUKkjxTOG7xlLaNY7BE4E3R6ffEMFhYdTv0
         4c8EYjcVUGHoOSjxZMLezWY6fa7e8a+KjdHHw2qYCfxPI422zPe+Mn+atNdHrHbwMo14
         b6LfNmngMjSjEWZlLJJJ+TIAnjtp0LhsFf8UQ8CYx8ev7gcmdi0AdAECTcx37orioH5g
         kGABJ68IrwaZfDHpCSElpBMvds26lLuv6GsLyXSr+o5lKklkZcOouO187NHNG0jTswQ4
         GPmlD9LpvNbKujKiFZRlmxzGypl3McTvmlza0WzZwFrqznmlwlPfQ1weC8nNF4r2ds6B
         7HPg==
X-Gm-Message-State: AOJu0YzmE2Zkkc13OcTwxyXDgMZaVFPrBKfuQctaH4zr1bIumjeG02JJ
	Iep+XPAUvASBZhtvUbih1PHIpPz16Srh0tni8/xw4qT2FvLqbF9lcrQ4U5bScSk=
X-Google-Smtp-Source: AGHT+IE/5cZk5AWk5a7xzqUN850yzLQTtt1/weVUf1dWbhJkHnGCWF9EoUAzPJ3NJRrF2UVAiN8CUg==
X-Received: by 2002:a05:6512:3b89:b0:512:e506:1c6c with SMTP id g9-20020a0565123b8900b00512e5061c6cmr1459678lfv.34.1708686832039;
        Fri, 23 Feb 2024 03:13:52 -0800 (PST)
Message-ID: <6bd41e23-e7c9-4cfc-8227-b508b0d4ed13@citrix.com>
Date: Fri, 23 Feb 2024 11:13:49 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/spec: fix BRANCH_HARDEN option to only be set when
 build-enabled
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
 Wei Liu <wl@xen.org>
References: <20240223094215.71889-1-roger.pau@citrix.com>
 <b5696664-0e85-414b-8360-242349a87a7e@citrix.com> <Zdhwve9-YQekWb0B@macbook>
 <cf60c604-5a4a-46be-badf-1f55a760e421@citrix.com> <Zdh9b4vNHfJnFXPR@macbook>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <Zdh9b4vNHfJnFXPR@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/02/2024 11:11 am, Roger Pau Monné wrote:
> On Fri, Feb 23, 2024 at 10:26:15AM +0000, Andrew Cooper wrote:
>> On 23/02/2024 10:17 am, Roger Pau Monné wrote:
>>> On Fri, Feb 23, 2024 at 09:46:27AM +0000, Andrew Cooper wrote:
>>>> On 23/02/2024 9:42 am, Roger Pau Monne wrote:
>>>>> The current logic to handle the BRANCH_HARDEN option will report it as enabled
>>>>> even when build-time disabled. Fix this by only allowing the option to be set
>>>>> when support for it is built into Xen.
>>>>>
>>>>> Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
>>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>>>> ---
>>>>>  xen/arch/x86/spec_ctrl.c | 6 ++++--
>>>>>  1 file changed, 4 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
>>>>> index 421fe3f640df..e634c6b559b4 100644
>>>>> --- a/xen/arch/x86/spec_ctrl.c
>>>>> +++ b/xen/arch/x86/spec_ctrl.c
>>>>> @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
>>>>>  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
>>>>>  int8_t __read_mostly opt_eager_fpu = -1;
>>>>>  int8_t __read_mostly opt_l1d_flush = -1;
>>>>> -static bool __initdata opt_branch_harden = true;
>>>>> +static bool __initdata opt_branch_harden =
>>>>> +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
>>>>>  
>>>>>  bool __initdata bsp_delay_spec_ctrl;
>>>>>  uint8_t __read_mostly default_xen_spec_ctrl;
>>>>> @@ -267,7 +268,8 @@ static int __init cf_check parse_spec_ctrl(const char *s)
>>>>>              opt_eager_fpu = val;
>>>>>          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
>>>>>              opt_l1d_flush = val;
>>>>> -        else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
>>>>> +        else if ( IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) &&
>>>>> +                  (val = parse_boolean("branch-harden", s, ss)) >= 0 )
>>>>>              opt_branch_harden = val;
>>>> Yeah, we should definitely fix this, but could we use no_config_param()
>>>> here for the compiled-out case ?
>>>>
>>>> See cet= for an example.  If we're going to ignore what the user asks,
>>>> we should tell them why.
>>> Maybe I'm missing something: I've looked into using no_config_param(),
>>> but there's no difference really, because cmdline_parse() is called
>>> before the console is initialized, so those messages seem to be
>>> lost.
>> Look at `xl dmesg` rather than the console.  They also do appear on vga
>> in some configurations.
> Oh, my internal buffer was too small on those also got truncated, had
> to bump it.

Yeah - the default buffer size in Xen is too small.  It has been for
more than a decade, which is how long the adjustment in XenServer has
been around.

>
>> There's a separate todo to get these out in a slightly nicer way, but
>> they at least exist in logs.
> I've created:
>
> https://gitlab.com/xen-project/xen/-/issues/184

Thanks.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 11:36:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 11:36:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684809.1064933 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdTqV-0001oF-TJ; Fri, 23 Feb 2024 11:36:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684809.1064933; Fri, 23 Feb 2024 11:36:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdTqV-0001o8-Ox; Fri, 23 Feb 2024 11:36:03 +0000
Received: by outflank-mailman (input) for mailman id 684809;
 Fri, 23 Feb 2024 11:36:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=BhOy=KA=cloud.com=edwin.torok@srs-se1.protection.inumbo.net>)
 id 1rdTqT-0001o0-Ml
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 11:36:01 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b7b4c1a3-d23f-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 12:36:00 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a3566c0309fso37876266b.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 03:36:00 -0800 (PST)
Received: from smtpclient.apple ([217.156.233.157])
 by smtp.gmail.com with ESMTPSA id
 vk7-20020a170907cbc700b00a3f29591dcdsm2679381ejc.95.2024.02.23.03.35.59
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 23 Feb 2024 03:35:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7b4c1a3-d23f-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708688160; x=1709292960; darn=lists.xenproject.org;
        h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
         :from:from:to:cc:subject:date:message-id:reply-to;
        bh=JyQ6xbFYJAJFa0prrCGrknfBke4TIhg5l9pGph2tG5s=;
        b=Pnx4UxO2XpTXPwA6i2zNhnyP2fwEvQTe6RwrRQ0HJShPqWL0rFQWabDJIC+78gjgzI
         yLV062Pfl6ovPTT2sDwHRFVlfWocpzPt+decQ91ti8ovEiOi20cfbG+3UXsprpDF8VfP
         RA5zZ3eAX7hzGiE6xS6qwzSJ/Z3xBbRm/HCLg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708688160; x=1709292960;
        h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
         :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=JyQ6xbFYJAJFa0prrCGrknfBke4TIhg5l9pGph2tG5s=;
        b=AiE2puRd4jQA6WttTqfguJkK5r+6hxddTydEtqQCNyOzMkDugz7hKDL8aVwujMZb7P
         ia2atehpexgBpwnrAY40yvH83F3i8qGQh3nmm2IHur2bzGPx+edJuBTMxMxRJnwNyt14
         aj4ufOJ9DVuoveNY0g9qFtcIqj1/ahxIXEDJuHDk+5EzrhmbBwV75knKrzqB8O3wjAz9
         I9XkiMUeUjIg4VXPim2jO9nAS8389/4vUAndcq7GEoNocXSsfU5VlLU46wNxs9YdOf7D
         n/iKKJzgeSGKN597lGRd5qkVTeI1YgUiSF/1DDsyJSDn+ED4By4e51Jf9Uq9EJupzCVG
         V8sA==
X-Gm-Message-State: AOJu0YysdgYbdZRH9qtN2H+R2O1L1aeRlafc+/e9lATErQAwdK69rVQC
	LJ+rlh7HLlG33obboKLiQU2svL7P/n/cVjRtFhvV+4GFERbhr3Gf2TalOe5FaK4=
X-Google-Smtp-Source: AGHT+IESfYKAQD1tPOcq9XL6vnw7Y2EnMg7oFDUe40QIQR5Lr+u5T9Ai//+Uel3YngrKBtLdVvC8Qw==
X-Received: by 2002:a17:906:f24e:b0:a3e:fce7:9393 with SMTP id gy14-20020a170906f24e00b00a3efce79393mr1063956ejb.10.1708688159760;
        Fri, 23 Feb 2024 03:35:59 -0800 (PST)
From: Edwin Torok <edwin.torok@cloud.com>
Message-Id: <9301CE74-1D0D-4264-9987-D52AA21F525B@cloud.com>
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_1105AA12-93D6-4804-BE20-A928B5937E54"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\))
Subject: Re: [PATCH v1 2/2] oxenstored: make Quota.t pure
Date: Fri, 23 Feb 2024 11:35:48 +0000
In-Reply-To: <400C98F1-A3C6-4E14-90DE-91E314C48697@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Christian Lindig <christian.lindig@citrix.com>,
 David Scott <dave@recoil.org>,
 Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>
To: Christian Lindig <christian.lindig@cloud.com>
References: <cover.1706697858.git.edwin.torok@cloud.com>
 <f98edc633527b6d9a6855af0aff4fb77970454cc.1706697858.git.edwin.torok@cloud.com>
 <5879EE8F-6FAB-4026-BEDB-1AEEA7BF23DB@cloud.com>
 <400C98F1-A3C6-4E14-90DE-91E314C48697@cloud.com>
X-Mailer: Apple Mail (2.3774.400.31)


--Apple-Mail=_1105AA12-93D6-4804-BE20-A928B5937E54
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8



> On 31 Jan 2024, at 16:27, Edwin Torok <edwin.torok@cloud.com> wrote:
>=20
>=20
>=20
>> On 31 Jan 2024, at 11:17, Christian Lindig =
<christian.lindig@cloud.com> wrote:
>>=20
>>=20
>>=20
>>> On 31 Jan 2024, at 10:52, Edwin T=C3=B6r=C3=B6k =
<edwin.torok@cloud.com> wrote:
>>>=20
>>> Now that we no longer have a hashtable inside we can make Quota.t =
pure,
>>> and push the mutable update to its callers.
>>> Store.t already had a mutable Quota.t field.
>>>=20
>>> No functional change.
>>=20
>> Acked-by: Christian Lindig <christian.lindig@cloud.com>
>>=20
>> This is shifting copying working to GC work, at least potentially. I =
would agree that this is a good trade-off and the code looks correct to =
me. But I think we should see more testing and benchmarking before =
merging this unless we are fine merging speculative improvements.
>>=20
>> =E2=80=94 C
>>=20
>>=20
>=20
>=20
> I=E2=80=99ve written this [1] microbenchmark which creates ~300_000 =
entries in xenstore, assigns quota to 1000 domains and then measure how =
long it takes to list xenstore
> (It doesn=E2=80=99t matter whether these domains exist or not).
> It shows a measurable improvement with this patch, once I=E2=80=99ve =
run a more realistic test on the original machine with 1000 real VMs =
I=E2=80=99ll post those results too:

The machine that can run this test is offline now due to a lab move, but =
I managed to get this data before it went away, and I think this patch =
series is ready to be committed.

Flamegraph without my changes, where Hashtbl.copy takes up a significant =
amount of oxenstored time: =
https://cdn.jsdelivr.net/gh/edwintorok/xen@oxenstored-coverletter/docs/ori=
ginal.svg?x=3D153.0&y=3D1269
Flamegraph with this patch series, where Hashtbl.copy does not show up =
at all: =
https://cdn.jsdelivr.net/gh/edwintorok/xen@oxenstored-coverletter/docs/oxe=
nstored_no_hashtbl_copy.svg?x=3D294.3&y=3D1301
(There are of course still hashtbl in the flame graph, due to the =
well-known inefficient poll_select implementation, and we see hashtbl =
iteration as a parent caller, which is fine)

IMHO this means the patch series is a worthwhile improvement: it removes =
a codepath that was previously a hotspot completely from oxenstored.

The timings on the test also show improvements with this patch:

Booting 575 VMs:
* without this patch series: 1099s
* with this patch series: 604s

Booting 626 VMs:
* without this patch series: 4027s
* with this patch series: 2115s

Booting 627 VMs:
* without this patch series: 4038s
* with this patch series: 4120s

This shows that *with* the patch series the system scales better until =
it hits a breaking point around 627 VMs where everything is ~equally =
slow

Not everything is ideal, there is also a slowdown around the 789 booted =
VM mark:
* without this patch series: 168s
* with this patch series: 394s
I wouldn=E2=80=99t worry about this result, because by this point some =
VMs have already crashed, and I=E2=80=99d consider the test to have =
failed by this point. What results you get at this point depends on how =
much CPU oxenstored gets compared to the qemus ioreq handler that is =
spinning handling the crash on the serial port.
To actually reach 1000 VMs it will require more fixes outsides of =
oxenstored (e.g. wrt to using correct groups with tapdisk, qemu, etc., =
or making qemu better cope with flooding on serial port from the guest), =
and probably some fixes to the poll_select in oxenstored that I may =
address in a future patch series.



Best regards,
=E2=80=94Edwin

>=20
> On an Intel Xeon Gold 6354 @ 3.0 Ghz:
> * without my patch: 12.756 +- 0.152 seconds time elapsed  ( +-  1.19% =
)
> * with my patch: 5.6051 +- 0.0467 seconds time elapsed  ( +-  0.83% )
>=20
> [1]
> # cat >create.py <<EOF
> #!/usr/bin/env python3
> from xen.lowlevel.xs import xs
>=20
> xenstore =3D xs()
>=20
> for i in range(1,1000):
>   txn =3D xenstore.transaction_start()
>   for j in range(1,10):
>     for k in range(1,30):
>         path=3Df"/quotatest/{i}/{j}/{k}/x"
>         xenstore.write(txn, path, "val")
>         # assign quota to domid i
>         xenstore.set_permissions(txn, path, [{"dom": i}])
>   xenstore.transaction_end(txn)
> EOF
> # python3 create.py
> # perf stat -r 10 sh -c 'xenstore-ls $(for i in $(seq 1 100); do echo =
"/quotatest/$i"; done) >/dev/null=E2=80=99
>=20
> Best regards,
> =E2=80=94Edwin


--Apple-Mail=_1105AA12-93D6-4804-BE20-A928B5937E54
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=utf-8

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; =
charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;"><br =
id=3D"lineBreakAtBeginningOfMessage"><div><br><blockquote =
type=3D"cite"><div>On 31 Jan 2024, at 16:27, Edwin Torok =
&lt;edwin.torok@cloud.com&gt; wrote:</div><br =
class=3D"Apple-interchange-newline"><div><meta http-equiv=3D"content-type"=
 content=3D"text/html; charset=3Dutf-8"><div style=3D"overflow-wrap: =
break-word; -webkit-nbsp-mode: space; line-break: =
after-white-space;"><br =
id=3D"lineBreakAtBeginningOfMessage"><div><br><blockquote =
type=3D"cite"><div>On 31 Jan 2024, at 11:17, Christian Lindig =
&lt;christian.lindig@cloud.com&gt; wrote:</div><br =
class=3D"Apple-interchange-newline"><div><div><br><br><blockquote =
type=3D"cite">On 31 Jan 2024, at 10:52, Edwin T=C3=B6r=C3=B6k =
&lt;edwin.torok@cloud.com&gt; wrote:<br><br>Now that we no longer have a =
hashtable inside we can make Quota.t pure,<br>and push the mutable =
update to its callers.<br>Store.t already had a mutable Quota.t =
field.<br><br>No functional change.<br></blockquote><br>Acked-by: =
Christian Lindig &lt;christian.lindig@cloud.com&gt;<br><br>This is =
shifting copying working to GC work, at least potentially. I would agree =
that this is a good trade-off and the code looks correct to me. But I =
think we should see more testing and benchmarking before merging this =
unless we are fine merging speculative improvements.<br><br>=E2=80=94 =
C<br><br><br></div></div></blockquote></div><br><div><br></div><div>I=E2=80=
=99ve written this [1] microbenchmark which creates ~300_000 entries in =
xenstore, assigns quota to 1000 domains and then measure how long it =
takes to list xenstore</div><div>(It doesn=E2=80=99t matter whether =
these domains exist or not).</div><div>It shows a measurable improvement =
with this patch, once I=E2=80=99ve run a more realistic test on the =
original machine with 1000 real VMs I=E2=80=99ll post those results =
too:</div></div></div></blockquote><div><br></div><div>The machine that =
can run this test is offline now due to a lab move, but I managed to get =
this data before it went away, and I think this patch series is ready to =
be committed.</div><div><br></div><div>Flamegraph without my changes, =
where Hashtbl.copy takes up a significant amount of oxenstored =
time:&nbsp;<a =
href=3D"https://cdn.jsdelivr.net/gh/edwintorok/xen@oxenstored-coverletter/=
docs/original.svg?x=3D153.0&amp;y=3D1269">https://cdn.jsdelivr.net/gh/edwi=
ntorok/xen@oxenstored-coverletter/docs/original.svg?x=3D153.0&amp;y=3D1269=
</a></div><div>Flamegraph with this patch series, where Hashtbl.copy =
does not show up at all:&nbsp;<a =
href=3D"https://cdn.jsdelivr.net/gh/edwintorok/xen@oxenstored-coverletter/=
docs/oxenstored_no_hashtbl_copy.svg?x=3D294.3&amp;y=3D1301">https://cdn.js=
delivr.net/gh/edwintorok/xen@oxenstored-coverletter/docs/oxenstored_no_has=
htbl_copy.svg?x=3D294.3&amp;y=3D1301</a></div><div>(There are of course =
still hashtbl in the flame graph, due to the well-known inefficient =
poll_select implementation, and we see hashtbl iteration as a parent =
caller, which is fine)</div><div><br></div><div>IMHO this means the =
patch series is a worthwhile improvement: it removes a codepath that was =
previously a hotspot completely from =
oxenstored.</div><div><br></div><div>The timings on the test also show =
improvements with this patch:</div><div><br></div><div>Booting 575 =
VMs:</div><div>* without this patch series: 1099s</div><div>* with this =
patch series: 604s</div><div><br></div><div>Booting 626 VMs:</div><div>* =
without this patch series:&nbsp;<span style=3D"caret-color: rgb(0, 0, =
0); color: rgb(0, 0, 0);">4027s</span></div><div>* with this patch =
series: 2115s</div><div><br></div><div>Booting 627 VMs:</div><div>* =
without this patch series: 4038s</div><div>* with this patch series: =
4120s</div><div><br></div><div>This shows that *with* the patch series =
the system scales better until it hits a breaking point around 627 VMs =
where everything is ~equally slow</div><div><br></div><div>Not =
everything is ideal, there is also a slowdown around the 789 booted VM =
mark:</div><div>* without this patch series: 168s</div><div>* with this =
patch series: 394s</div><div>I wouldn=E2=80=99t worry about this result, =
because by this point some VMs have already crashed, and I=E2=80=99d =
consider the test to have failed by this point. What results you get at =
this point depends on how much CPU oxenstored gets compared to the qemus =
ioreq handler that is spinning handling the crash on the serial =
port.</div><div>To actually reach 1000 VMs it will require more fixes =
outsides of oxenstored (e.g. wrt to using correct groups with tapdisk, =
qemu, etc., or making qemu better cope with flooding on serial port from =
the guest), and probably some fixes to the poll_select in oxenstored =
that I may address in a future patch =
series.</div><div><br></div><div><br></div><div><br></div><div>Best =
regards,</div><div>=E2=80=94Edwin</div><br><blockquote =
type=3D"cite"><div><div style=3D"overflow-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: =
after-white-space;"><div><br></div><div>On an Intel Xeon Gold 6354 @ 3.0 =
Ghz:</div><div>* without my patch:&nbsp;12.756 +- 0.152 seconds time =
elapsed &nbsp;( +- &nbsp;1.19% )</div><div>* with my patch:&nbsp;5.6051 =
+- 0.0467 seconds time elapsed &nbsp;( +- &nbsp;0.83% =
)</div><div><br></div><div>[1]</div><div># cat &gt;create.py =
&lt;&lt;EOF</div><div><div>#!/usr/bin/env python3</div><div>from =
xen.lowlevel.xs import xs</div><div><br></div><div>xenstore =3D =
xs()</div><div><br></div><div>for i in range(1,1000):</div><div>&nbsp; =
txn =3D xenstore.transaction_start()</div><div>&nbsp; for j in =
range(1,10):</div><div>&nbsp; &nbsp; for k in =
range(1,30):</div><div>&nbsp; &nbsp; &nbsp; &nbsp; =
path=3Df"/quotatest/{i}/{j}/{k}/x"</div><div>&nbsp; &nbsp; &nbsp; &nbsp; =
xenstore.write(txn, path, "val")</div><div>&nbsp; &nbsp; &nbsp; &nbsp; # =
assign quota to domid i</div><div>&nbsp; &nbsp; &nbsp; &nbsp; =
xenstore.set_permissions(txn, path, [{"dom": i}])</div><div>&nbsp; =
xenstore.transaction_end(txn)</div></div><div>EOF</div><div># python3 =
create.py</div><div>#&nbsp;<span style=3D"caret-color: rgb(0, 0, =
0);">perf stat -r 10 sh -c 'xenstore-ls $(for i in $(seq 1 100); do echo =
"/quotatest/$i"; done) &gt;/dev/null</span><font><span =
style=3D"caret-color: rgb(0, 0, 0);">=E2=80=99</span></font></div><div><sp=
an style=3D"caret-color: rgb(0, 0, 0);"><br></span></div><div><span =
style=3D"caret-color: rgb(0, 0, 0);">Best =
regards,</span></div><div><font><span style=3D"caret-color: rgb(0, 0, =
0);">=E2=80=94Edwin</span></font></div></div></div></blockquote></div><br>=
</body></html>=

--Apple-Mail=_1105AA12-93D6-4804-BE20-A928B5937E54--


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 12:06:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 12:06:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684815.1064942 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUJp-0006dv-9Y; Fri, 23 Feb 2024 12:06:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684815.1064942; Fri, 23 Feb 2024 12:06:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUJp-0006do-5h; Fri, 23 Feb 2024 12:06:21 +0000
Received: by outflank-mailman (input) for mailman id 684815;
 Fri, 23 Feb 2024 12:06:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=0yaO=KA=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rdUJo-0006di-0V
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 12:06:20 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f2f4c005-d243-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 13:06:17 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-512b700c8ebso987189e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 04:06:17 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 od6-20020a0562142f0600b0068fdc6f89cbsm408247qvb.133.2024.02.23.04.06.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 23 Feb 2024 04:06:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f2f4c005-d243-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708689977; x=1709294777; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=yxkkrvkgc+DSjiHCxjUfb9IZNAwxhD7Q4naR3cAtQYg=;
        b=aw+EbZPCuM9wtrCPaDjDfs4H3y9uEticDSI6QWeMwmAcxzO+ASC/w7QYyhK/ZnV4Pu
         U4hpDawLgTKbvzH8VZ6VA6hx1ETdU0At/wiCGu7O0qxp8u+mjXuP5wzuRyCi7+1/T1Vp
         tgNT8IPC0mtNeoJHUrwLISKzHB2VbsDiRo2TI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708689977; x=1709294777;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=yxkkrvkgc+DSjiHCxjUfb9IZNAwxhD7Q4naR3cAtQYg=;
        b=P0UQw0LFhGknuWkLC0VVDW4VXP6H5C1N28f3fhvvyz9HUtmsTYU7XZzBP26RtqzUFp
         y0M77Plq+S2XFbR5q4+cVVzXmhdYCE4fJDAMGas20FOSyJTH2hNw775Nl1dFinP4dAkd
         J07Q9KBLWhi5h553SnERi80BVN+5+glV1LE/E367bduEaYYbKxtinybYiBEJPFs7kt2e
         Fmbppxukr1XLT7ynuYuZQJX2IjugR9olrknx1Nria/o+n78Y0hZN5h/82G1xMTsg7d5b
         5G2a8+tfA9mQJPZVTHuSUMixPg6B9NR864vkia7ElE0Lv5tbfvC9zlJR05uoq03ByV19
         WTkQ==
X-Gm-Message-State: AOJu0YxxO+5CUgqKDXfv5mdi4QGMr0kJJVskWQw/G6DJkt12LO9/SM1t
	FAuWzDR8lINSJNaUvr+h0i9sTjIUJAFZRYcYivZLQojg7CYFoY3vozLKISkr2bQ=
X-Google-Smtp-Source: AGHT+IF0jj9M58YIHOQY0Mk98hfGMBpXeO6qDsBxwGe6PYWRBrWUWjnLvBfJaOd6O7n9PEcmwqj6Vg==
X-Received: by 2002:ac2:5f76:0:b0:512:c2e7:27a3 with SMTP id c22-20020ac25f76000000b00512c2e727a3mr1219685lfc.33.1708689977141;
        Fri, 23 Feb 2024 04:06:17 -0800 (PST)
Message-ID: <1d2ab544-b7c3-4d98-867e-1d76f933b512@citrix.com>
Date: Fri, 23 Feb 2024 12:06:14 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v1 2/2] oxenstored: make Quota.t pure
Content-Language: en-GB
To: Edwin Torok <edwin.torok@cloud.com>,
 Christian Lindig <christian.lindig@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott
 <dave@recoil.org>, Wei Liu <wl@xen.org>,
 Anthony PERARD <anthony.perard@citrix.com>
References: <cover.1706697858.git.edwin.torok@cloud.com>
 <f98edc633527b6d9a6855af0aff4fb77970454cc.1706697858.git.edwin.torok@cloud.com>
 <5879EE8F-6FAB-4026-BEDB-1AEEA7BF23DB@cloud.com>
 <400C98F1-A3C6-4E14-90DE-91E314C48697@cloud.com>
 <9301CE74-1D0D-4264-9987-D52AA21F525B@cloud.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <9301CE74-1D0D-4264-9987-D52AA21F525B@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23/02/2024 11:35 am, Edwin Torok wrote:
>> On 31 Jan 2024, at 16:27, Edwin Torok <edwin.torok@cloud.com> wrote:
>>> On 31 Jan 2024, at 11:17, Christian Lindig
>>> <christian.lindig@cloud.com> wrote:
>>>> On 31 Jan 2024, at 10:52, Edwin Török <edwin.torok@cloud.com> wrote:
>>>>
>>>> Now that we no longer have a hashtable inside we can make Quota.t pure,
>>>> and push the mutable update to its callers.
>>>> Store.t already had a mutable Quota.t field.
>>>>
>>>> No functional change.
>>>
>>> Acked-by: Christian Lindig <christian.lindig@cloud.com>
>>>
>>> This is shifting copying working to GC work, at least potentially. I
>>> would agree that this is a good trade-off and the code looks correct
>>> to me. But I think we should see more testing and benchmarking
>>> before merging this unless we are fine merging speculative improvements.
>>>
>>> — C
>>>
>>>
>>
>>
>> I’ve written this [1] microbenchmark which creates ~300_000 entries
>> in xenstore, assigns quota to 1000 domains and then measure how long
>> it takes to list xenstore
>> (It doesn’t matter whether these domains exist or not).
>> It shows a measurable improvement with this patch, once I’ve run a
>> more realistic test on the original machine with 1000 real VMs I’ll
>> post those results too:
>
> The machine that can run this test is offline now due to a lab move,
> but I managed to get this data before it went away, and I think this
> patch series is ready to be committed.
>
> Flamegraph without my changes, where Hashtbl.copy takes up a
> significant amount of oxenstored
> time: https://cdn.jsdelivr.net/gh/edwintorok/xen@oxenstored-coverletter/docs/original.svg?x=153.0&y=1269
> <https://cdn.jsdelivr.net/gh/edwintorok/xen@oxenstored-coverletter/docs/original.svg?x=153.0&y=1269>
> Flamegraph with this patch series, where Hashtbl.copy does not show up
> at
> all: https://cdn.jsdelivr.net/gh/edwintorok/xen@oxenstored-coverletter/docs/oxenstored_no_hashtbl_copy.svg?x=294.3&y=1301
> <https://cdn.jsdelivr.net/gh/edwintorok/xen@oxenstored-coverletter/docs/oxenstored_no_hashtbl_copy.svg?x=294.3&y=1301>
> (There are of course still hashtbl in the flame graph, due to the
> well-known inefficient poll_select implementation, and we see hashtbl
> iteration as a parent caller, which is fine)
>
> IMHO this means the patch series is a worthwhile improvement: it
> removes a codepath that was previously a hotspot completely from
> oxenstored.

Agreed.  I'll queue this series in due course.

~Andrew


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 12:06:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 12:06:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684816.1064952 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUJz-0006uT-FM; Fri, 23 Feb 2024 12:06:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684816.1064952; Fri, 23 Feb 2024 12:06:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUJz-0006uM-Ca; Fri, 23 Feb 2024 12:06:31 +0000
Received: by outflank-mailman (input) for mailman id 684816;
 Fri, 23 Feb 2024 12:06:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdUJx-0006di-U6
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 12:06:29 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f9657c30-d243-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 13:06:28 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-512dfa097b1so1051745e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 04:06:28 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 oi8-20020a05621443c800b0068fb9bdd7absm2186266qvb.74.2024.02.23.04.06.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 04:06:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f9657c30-d243-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708689988; x=1709294788; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=sGVjziCIVDQTtjsVgjM5FvSGR0Kdkzwj0ZWP3aarH/A=;
        b=JcQoGMz5khAPiHpgWjvP23r8lOelqwUN7i7IluqSDaabbfjkFy8ChbBtP9rYhaBZj6
         B4zj7d85yaj9tDNNPBq9vORBRIGBw0GNzs8tuHekT18a3MiYaw+BrVav93E6+v++yZZ2
         mVl12iIVUEXRgQIIq51g9h67HsHV5IX6572OU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708689988; x=1709294788;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=sGVjziCIVDQTtjsVgjM5FvSGR0Kdkzwj0ZWP3aarH/A=;
        b=hgC5HXnYnmyd5KN0aygyjp9arE9jwzjWrI/8hKBlFBTLJWCKGxLoUPW25nihl7MjZF
         JxliSO8t+yliyL0U553/04fApVtY6ciRP6Hwi0kgLRecyEf9//jFZkoXwZRyFgbfHL7P
         AiOIO8YNbDQqvy0q4aM42EklwlSFcJZtOggxtr7MG6PKTOI9JNNkzcMsou9rflI7fnDT
         6ClRGi/3yM1uvakeDA1gr6IYTA8ZwtMRtgvWTbgkvjrA8mjaRU3Bi/74Edw/uEylOi+s
         48f8nDQSxxsfAusi1r3YwQeK00DOry2YwvQtJNHQQwymfJE8LzRy/NRytk3fhNQkL7Jv
         rNjg==
X-Gm-Message-State: AOJu0YzFt7zcluyt4/mVdmDluiw/kucQdWkbI5k9Dk7ZHzvERhVfaPPA
	hKuCMXJROxC7ywXBMr24UPlRKcnvY1om3ftnoYDopConM8HM3WiY0EGV8mbd3uWIKlff7XiocJS
	g
X-Google-Smtp-Source: AGHT+IFcLLC+oL9Adf9+8TDIGW+8rKQLVInW0WmgPPDKtHQfpV7bsa8+45vroPspfGJS4OhW6k3uWw==
X-Received: by 2002:a05:6512:32bb:b0:512:a7a1:147b with SMTP id q27-20020a05651232bb00b00512a7a1147bmr1213715lfe.63.1708689987842;
        Fri, 23 Feb 2024 04:06:27 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2 0/3] x86/spec: improve command line parsing
Date: Fri, 23 Feb 2024 13:06:13 +0100
Message-ID: <20240223120616.73503-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

A couple of improvements for the spec-ctrl command line parsing, and one
bugfix for no_config_param().

Thanks, Roger.

Roger Pau Monne (3):
  xen/cmdline: fix printf format specifier in no_config_param()
  x86/spec: fix BRANCH_HARDEN option to only be set when build-enabled
  x86/spec: fix INDIRECT_THUNK option to only be set when build-enabled

 xen/arch/x86/spec_ctrl.c | 15 ++++++++++++++-
 xen/include/xen/param.h  |  2 +-
 2 files changed, 15 insertions(+), 2 deletions(-)

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 12:06:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 12:06:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684817.1064962 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUK1-0007A9-MW; Fri, 23 Feb 2024 12:06:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684817.1064962; Fri, 23 Feb 2024 12:06:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUK1-00079y-JZ; Fri, 23 Feb 2024 12:06:33 +0000
Received: by outflank-mailman (input) for mailman id 684817;
 Fri, 23 Feb 2024 12:06:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdUK0-0006xc-B7
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 12:06:32 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fae8aaf4-d243-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 13:06:31 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-51197ca63f5so1247513e87.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 04:06:30 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 kf3-20020a056214524300b0068fe3170b0esm88772qvb.11.2024.02.23.04.06.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 04:06:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fae8aaf4-d243-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708689990; x=1709294790; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=jp86iadmfvDn3kY5s/NcHvROQQfc/eEcQpXnyfCy0t8=;
        b=KJhAtRRhazOY55NyMGHBOdbs6ygrXx1bV746XYR78n0cdBDHgF07du3mwcOZwQUBuq
         9KvFaOEe0Q8UDpIzflWzBScbmdrr0cOd3A7EVsrzIalnfzF5rUy/SaOdISVy+j4I+NVP
         gSaLTTUVF7M8Xj34OAPZMilwzOZIKCLEyoJU0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708689990; x=1709294790;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=jp86iadmfvDn3kY5s/NcHvROQQfc/eEcQpXnyfCy0t8=;
        b=lgTvdhFV0PFsjXyhIXTy2hKZTmtGZ03/VUXVLFa7sHiCuLXWQ5RdDvzxSVcndrWiGF
         FJ5FPcID8sq2pmYXO7eoCjy5p4r83cXdKhqJSyd/wiySGMvgq5/PAfO7GdHPdlgL1BG4
         xA6vb+htrVyJE/t8LmvMqsPFfkasjxhaw0TZzazQyY4wE2Rb5ufqIHU5oarhUgHsELMq
         zyZNaLod5lIy1hSv0xs9NSH18aHpCHivHV+hTnnrpXiYEt4V+JXFoppPSGJ6zrcHmRXW
         PqH63gFbawCg/rYHZq1Q6OJwDR/EKizTqh0HE2vEoAiicjA8galM/w2WB5AYwCeGHLJf
         jNTg==
X-Gm-Message-State: AOJu0YxcuXItC0qoOPvsJMyTbdjVxaqc2xLFvKfF2s6ZObhyqAdxgc97
	iQNaqYw3NDzvZtY/q2eRrSSmX7wDL4jKFoA/6ZrNB4/ZNsob0VC+d4hvCITp4N6nw5KmAWKLs3y
	C
X-Google-Smtp-Source: AGHT+IGsFXYkQjVxuWz/a4tHq/rp5C8ES+MZ65o6SqjJB1Uhw3VOCrsFHqjMlvc4Rg8ZC/gh6YiWPw==
X-Received: by 2002:a05:6512:3194:b0:512:b90f:fc4e with SMTP id i20-20020a056512319400b00512b90ffc4emr1603543lfe.46.1708689990023;
        Fri, 23 Feb 2024 04:06:30 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2 1/3] xen/cmdline: fix printf format specifier in no_config_param()
Date: Fri, 23 Feb 2024 13:06:14 +0100
Message-ID: <20240223120616.73503-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240223120616.73503-1-roger.pau@citrix.com>
References: <20240223120616.73503-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

'*' sets the width field, which is the minimum number of characters to output,
but what we want in no_config_param() is the precision instead, which is '.*'
as it imposes a maximum limit on the output.

Fixes: 68d757df8dd2 ('x86/pv: Options to disable and/or compile out 32bit PV support')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - New in this version.
---
 xen/include/xen/param.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/xen/param.h b/xen/include/xen/param.h
index 9170455cde5c..6442a92aff8e 100644
--- a/xen/include/xen/param.h
+++ b/xen/include/xen/param.h
@@ -191,7 +191,7 @@ static inline void no_config_param(const char *cfg, const char *param,
 {
     int len = e ? ({ ASSERT(e >= s); e - s; }) : strlen(s);
 
-    printk(XENLOG_INFO "CONFIG_%s disabled - ignoring '%s=%*s' setting\n",
+    printk(XENLOG_INFO "CONFIG_%s disabled - ignoring '%s=%.*s' setting\n",
            cfg, param, len, s);
 }
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 12:06:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 12:06:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684818.1064972 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUK4-0007Rz-Vw; Fri, 23 Feb 2024 12:06:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684818.1064972; Fri, 23 Feb 2024 12:06:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUK4-0007Rs-Qt; Fri, 23 Feb 2024 12:06:36 +0000
Received: by outflank-mailman (input) for mailman id 684818;
 Fri, 23 Feb 2024 12:06:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdUK3-0006di-36
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 12:06:35 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fc67deed-d243-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 13:06:33 +0100 (CET)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2d2505352e6so7990361fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 04:06:33 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 d4-20020a056214184400b0068f97928376sm4150972qvy.17.2024.02.23.04.06.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 04:06:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc67deed-d243-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708689992; x=1709294792; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/ja3agDhYaU4Xo2ppsREM/G2NT1u2hAe6akh+/GuxDM=;
        b=Dx95aEgYlAbBARDUcKB6d7uEqiRX2L8I2wOMelk01Ajl/Y/LJKoy6vcPZ6nqI++VDm
         B0YmeGJPTiBfvTa/ig71JzN0vixWxgbt5ylY5bK/veZ85S8h+AfgK8RFuPwj6h2Hars/
         2v684NRfXATlbIsNk36tz0bWFitnKnc11fT1g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708689992; x=1709294792;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=/ja3agDhYaU4Xo2ppsREM/G2NT1u2hAe6akh+/GuxDM=;
        b=tmD8EYunH32SQuQOnr+DYy5Bv9NDB/2VDmmCCR3W6myXCBbrOL42VzV8sF/vBITBNm
         CDPyl8UQq/VwUMrhht7xRDHqWgam5BDpOelYFv5wVUmpCRYQlRWEtY+gf9cVrnJu9lWL
         YkzVZBDKVs9Mn2+/pKiXL9ao9U0IBI41wpV5fTx94UAbcHkXOCRR2ymiu9gWL5CTCpPl
         0JuTuIdWnTcWubDOdJWzB/Bbm6OGrB+mnOfyVC6D/+QJzvK4DsF6YcZS4K+KWUUupG1R
         6SJGUwKTcwtdHgfrdXKKiErz3A68mHFxh35SFZpiwA7XPEQZQPiQ8yz/WxibptTgbADu
         UbdQ==
X-Gm-Message-State: AOJu0YyxM6poQZ0slfM5QpXnZH/nX8S5pKeWyBJA1o97aP504BqMOdTv
	BSHwpcjsW7U4z3LuByAvPoXFcR0BZda064q2y3Q+YHqmd/vOONk+eB8s+MCF+/5Ffp4gGPiFbjX
	Q
X-Google-Smtp-Source: AGHT+IF0k4Kr9lbT9FetBX+abq3S57WnQWAFi/UWuRO0TO+qnTkBGj9QAvYwkgJaW9/nk/eZloIAOQ==
X-Received: by 2002:ac2:4ece:0:b0:512:e051:ac2c with SMTP id p14-20020ac24ece000000b00512e051ac2cmr1167099lfr.3.1708689992373;
        Fri, 23 Feb 2024 04:06:32 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2 2/3] x86/spec: fix BRANCH_HARDEN option to only be set when build-enabled
Date: Fri, 23 Feb 2024 13:06:15 +0100
Message-ID: <20240223120616.73503-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240223120616.73503-1-roger.pau@citrix.com>
References: <20240223120616.73503-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The current logic to handle the BRANCH_HARDEN option will report it as enabled
even when build-time disabled. Fix this by only allowing the option to be set
when support for it is built into Xen.

Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Use no_config_param().
---
 xen/arch/x86/spec_ctrl.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 421fe3f640df..4ce8a7a0b8ef 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
 int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
 int8_t __read_mostly opt_eager_fpu = -1;
 int8_t __read_mostly opt_l1d_flush = -1;
-static bool __initdata opt_branch_harden = true;
+static bool __initdata opt_branch_harden =
+    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
 
 bool __initdata bsp_delay_spec_ctrl;
 uint8_t __read_mostly default_xen_spec_ctrl;
@@ -268,7 +269,14 @@ static int __init cf_check parse_spec_ctrl(const char *s)
         else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
             opt_l1d_flush = val;
         else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
+        {
+#ifdef CONFIG_SPECULATIVE_HARDEN_BRANCH
             opt_branch_harden = val;
+#else
+            no_config_param("SPECULATIVE_HARDEN_BRANCH", "spec-ctrl", s, ss);
+            rc = -EINVAL;
+#endif
+        }
         else if ( (val = parse_boolean("srb-lock", s, ss)) >= 0 )
             opt_srb_lock = val;
         else if ( (val = parse_boolean("unpriv-mmio", s, ss)) >= 0 )
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 12:06:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 12:06:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684819.1064982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUK7-0007jV-64; Fri, 23 Feb 2024 12:06:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684819.1064982; Fri, 23 Feb 2024 12:06:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUK7-0007jJ-3G; Fri, 23 Feb 2024 12:06:39 +0000
Received: by outflank-mailman (input) for mailman id 684819;
 Fri, 23 Feb 2024 12:06:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdUK5-0006di-Pc
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 12:06:37 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fdef2b12-d243-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 13:06:36 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-512bde3d197so210620e87.0
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 04:06:36 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 da35-20020a05620a362300b007854018044bsm6336939qkb.134.2024.02.23.04.06.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 04:06:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fdef2b12-d243-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708689995; x=1709294795; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=5cdfP1drv1LQ9YGslYJKltpSCLlPkSQYqQfve8zYol4=;
        b=WzJmNMYr1BCm1TZv+pfQ6eJYyox7xB7bTNJP7rUZskBurArWF4jllYZoEZWUvCIdpN
         RdhvrTJqNQQQl7wUGdFqbFwI9XgEFLJCqIX7I3B/HgL/2rgv0W32zUwX6kVOkoeOenum
         RL82WESEaMu/47eUo1R5KUOPzINgJ/v/xeLbk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708689995; x=1709294795;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=5cdfP1drv1LQ9YGslYJKltpSCLlPkSQYqQfve8zYol4=;
        b=e8+8SNnST+zyTmWp4D2bttlPCkudrecNMD2zXbzYtJwHgehVvVMPDfsnatkNQEWv/r
         oq+B14tQr9DnZYv6YKWXHo0UdDboVD1Qtj44GwctKe8w/7KbQzlbXaFS+YvIaGjzrnx1
         ByCRwlNvIV6C9cuoKCUi2ETbzg8wx6byuQE96DaUQ36RPg68CifXrhggFLaaCa0KCOv/
         Efo7rb4EqNb5xx2H/dRQYwR6us3TWl9fYL0Uqz3+RiLSnTfpqZS2VexSyeXek7tbx7EP
         wOVfUqu36hGZzsX/WQN1N9rfvnKK9QKsotTQCwslwu03NaaDhYoBQdRpFZlh/DG1cAmd
         BIPw==
X-Gm-Message-State: AOJu0YxJTH1aBuuAeamw18EwEVAic0DkHeT85nPJUpncYP1UhZ/u8Rye
	fCLWPwegedebUlNfquXRGTC+IyFyvZFXPkq3naV4As01yl+fQaTQ9vzOcjktR3Mb/cuCdOjKsFL
	Y
X-Google-Smtp-Source: AGHT+IHpeNl7z21jCtIKBCr1pJW/x8CWDv/TpSnAFd34Glij7WpFsufIBx3xHtkP91uwcHGLYJlfrw==
X-Received: by 2002:a19:5f08:0:b0:512:d689:51cb with SMTP id t8-20020a195f08000000b00512d68951cbmr1104193lfb.26.1708689994947;
        Fri, 23 Feb 2024 04:06:34 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2 3/3] x86/spec: fix INDIRECT_THUNK option to only be set when build-enabled
Date: Fri, 23 Feb 2024 13:06:16 +0100
Message-ID: <20240223120616.73503-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240223120616.73503-1-roger.pau@citrix.com>
References: <20240223120616.73503-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Attempt to provide a more helpful error message when the user attempts to set
spec-ctrl=bti-thunk option but the support is build-time disabled.

Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - New in this version.
---
 xen/arch/x86/spec_ctrl.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 4ce8a7a0b8ef..f3432f1a6c80 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -239,6 +239,7 @@ static int __init cf_check parse_spec_ctrl(const char *s)
         /* Xen's speculative sidechannel mitigation settings. */
         else if ( !strncmp(s, "bti-thunk=", 10) )
         {
+#ifdef CONFIG_INDIRECT_THUNK
             s += 10;
 
             if ( !cmdline_strcmp(s, "retpoline") )
@@ -249,6 +250,10 @@ static int __init cf_check parse_spec_ctrl(const char *s)
                 opt_thunk = THUNK_JMP;
             else
                 rc = -EINVAL;
+#else
+            no_config_param("INDIRECT_THUNK", "spec-ctrl", s, ss);
+            rc = -EINVAL;
+#endif
         }
 
         /* Bits in MSR_SPEC_CTRL. */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 12:10:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 12:10:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684831.1064992 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUNj-0002PT-Q6; Fri, 23 Feb 2024 12:10:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684831.1064992; Fri, 23 Feb 2024 12:10:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUNj-0002PM-NV; Fri, 23 Feb 2024 12:10:23 +0000
Received: by outflank-mailman (input) for mailman id 684831;
 Fri, 23 Feb 2024 12:10:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=mvZq=KA=nxp.com=peng.fan@srs-se1.protection.inumbo.net>)
 id 1rdUNi-0002PG-9h
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 12:10:22 +0000
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
 (mail-db3eur04on0627.outbound.protection.outlook.com
 [2a01:111:f400:fe0c::627])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 836d8d55-d244-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 13:10:20 +0100 (CET)
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com (2603:10a6:10:358::11)
 by AM9PR04MB8844.eurprd04.prod.outlook.com (2603:10a6:20b:40b::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.32; Fri, 23 Feb
 2024 12:10:18 +0000
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::4ac3:3559:4200:38e1]) by DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::4ac3:3559:4200:38e1%4]) with mapi id 15.20.7292.036; Fri, 23 Feb 2024
 12:10:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 836d8d55-d244-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MtavfOpLmmktXl29XoSBANRMm08NpW5JEH4HPixU28wvoX8yCA05MkeBZjAEHM+FhnttaGhcX2MIF8oazi4xmO8ZwpQc2OkmqTiU9LeLXw1JQArzDx8jpsWFNJPa+tYfQi2dyXmYcWLwlmKGJt4Bpgdcw7bc1wj0RZjhmL24gntZB0BohqN9LY91DuuXhMi415gB9twFBWZTRiNM6iQUa8YuWBdHKDuCcE8jliLWSjOWfdrW5DCSCnU6SE9tbpReL/rGwIygA1tHOHnv5qrVlFTNRXifemBq1L2SznvYzUT7MO+vpR9xnsYPUxdgqN5UmvD1rVfGOUth374PWRCLFA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=pfu5C4ujnHdSPWzaSeFwKT7LbJnPZHwHqjo7MkanO/Q=;
 b=NKofgqmYF7BKonp04xMDGPcT1wZ1TVdrm51VZIil2HdUvUjlTzFT72Enw4ru+3oiaXDE9YhDOCa3B8sJWYhrsNhzRTYVt1fOPRfk9pBtIdyIlBoV4u1gaFP0lz8h1BRrlP/QqvyA2Ph/yz7EW56PbiSkz7SgI0zVlOlQ+pGjjtOTC9/swryjusPmMrtcJqG4HWEf3nr5WtcnLmdVBU2RTanOBUllDjji6qzq9toiKgRdSCm3hGWg8mJplhO0tt/QJUOf8MUG8nQN9R6eW6Pbn9TdigltoH+hjHQ0CFVS3dI1hC3ffZ5AQdmluar9RyJz9aAOBTLnAzXOCH6GTsgSTw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass
 header.d=nxp.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pfu5C4ujnHdSPWzaSeFwKT7LbJnPZHwHqjo7MkanO/Q=;
 b=djhmb+SJI13v3vTHeNsNnHNKua1ELEtJiGaM74LLMODifSkn2bgdjXzKi6ffqaOJiiwJG0pfy8AKhpj1bsXqvc4fJYnVSdYBj2QQqoIHyy0DvTQS1FpkPFu5+aiSfWj3oBCecDwtCq1DUpelhgCzfHIftZ087JOa2oIjlSaz/R8=
From: Peng Fan <peng.fan@nxp.com>
To: "stefano.stabellini@amd.com" <stefano.stabellini@amd.com>, Julien Grall
	<julien@xen.org>, "vikram.garhwal@amd.com" <vikram.garhwal@amd.com>,
	"oleksandr_tyshchenko@epam.com" <oleksandr_tyshchenko@epam.com>, Leo Yan
	<leo.yan@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>,
	"stewart.hildebrand@amd.com" <stewart.hildebrand@amd.com>,
	"sgarzare@redhat.com" <sgarzare@redhat.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Peng
 Fan <peng.fan@nxp.com>
Subject: question about virtio-vsock on xen
Thread-Topic: question about virtio-vsock on xen
Thread-Index: AdpmUSXKZvg8Aft6RRu9smUwoggtbw==
Date: Fri, 23 Feb 2024 12:10:18 +0000
Message-ID:
 <DU0PR04MB941734DA793D87B7FF3A491488552@DU0PR04MB9417.eurprd04.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=nxp.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DU0PR04MB9417:EE_|AM9PR04MB8844:EE_
x-ms-office365-filtering-correlation-id: b99384cd-4f98-47bf-934e-08dc34686690
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 IugZv6lDnmZwrQBXIeUY/87odEBpbkbJqFSwARZyKcBHsOiDjoISQA4FkWa8dtStTyh7nQzwiwn+tiFqFEOmvm+m0pn9oeEXR+uiQLLZEsUwwl5Fv54oNIeVD4JPTmxEXgeXs29ezwLbEXdmvi2DNtXSIkJWaB8WfJPDuLMowJRLY/uh1G12/Xev03m2DXnTIArQwsL9cTWvAYNQ84AaAOC4WM0z/VKI2q/FbgbellyGKFMzmeO4DjNhtsNAMdTyoAffWfRFVSbGPQ+bY0gPhKwkSgpzsedfM7aU1OH9QN5X6AHCZcrVcmlbHUHWbWaJOT/VeYoLQiGg765ITfZaRh92tTLPNh7v79+sYHs/d1KiHLsYV897wQrfgQ6V35ywEUO6kbzyRkpWxX+ZWivoFdjbyDaODWYaoayMKoz/zYj47afCgmvyV7ng6Z2BHoBVPGR4H/5t8ps94FYgL1BZfuauJRUAgwJi8rAPpgSN/6fFP8ZqCx/1GQk3yv3tewVOlP37qEEfhIzzkFElh/0pzWGjlNz3V7+PhqyYGT189kM0dA1amnJ3/ExPWrybr2B/6SqaSwiku25DHZ3YL3d7GtCvltU5TkoiGcfLPf4mxQMqPBOiKk1hXO7Llk5HKCCq
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU0PR04MB9417.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?B08cS3UZivHozT3M3F2wChgFB9ansk+hBrM6AfLVlAoBubLZLuxMIcnoTWzd?=
 =?us-ascii?Q?2CoEntLAGRhKGUZKA64TzyOA/0d41YLUZPMmft1sftHkdWlm/44n+VCJEBiB?=
 =?us-ascii?Q?R3xaUe8l2ue7m9EFSWIZrQNRxgeHtU54Uu2fSerPq9IcFxTFJkDHHWd4E69P?=
 =?us-ascii?Q?dB1hewfN7IrTq+u7tDfhu9+cqfz9wZA8IVXlMGJX2eVcRNa3Gd5HRanOkqoe?=
 =?us-ascii?Q?kem4wwmpERA+Quw/pzhn07Zb6r/0vz5+Zj4tjrPqtZRfGAXVRgG8Uwh2B95x?=
 =?us-ascii?Q?QxnsHF67iTIkLnx8ocG0wTyMFj7U7NFm7sb1OxtDdzMewabGqSzzuCJGPGyy?=
 =?us-ascii?Q?wm+Ifr8eM6LkigJMRrdnhb1GEYWOndZuw5OYANs9m8aZmYpSENm5yzM4xijx?=
 =?us-ascii?Q?K89qU9w20nIO8nDRx9aC5g6YIPA8Gkc4Z9u1uNKtR9Xpkt/0PEQoP6kLcl82?=
 =?us-ascii?Q?C+If5KTFtpDNnbzuxMIxGvXwp2PjxuskxAHwcKErvOjsYK/TeZarQ+/Ta29d?=
 =?us-ascii?Q?fFEjPb28n8A6yGWI6xi/iXVw6O/+BTjtlMZC8EzMlAWpqxWXKaEJHr7nAEHL?=
 =?us-ascii?Q?5bg7IFKa92z8/qE5f5CUipnCuoN7yZ3e5KTvC7hP2fxjXLON2Dok11l8XLe9?=
 =?us-ascii?Q?oZqO6HrX0DvlH4WjDhs3mDzUcRov1J4xW2FOPewLF56hZNvkTW1eXWqBc/UY?=
 =?us-ascii?Q?ggtEh5A9xbJb1EVuqVsrGzPQ5C2B0qxC+XFqHvS+dsJk7nkU/tjKEJG2WDgB?=
 =?us-ascii?Q?48rR2iL9nBOi0i0FLfB7UiOdHfh0Ng7bps2jnVFMlOj5xPQFvG1JSDWwKbuo?=
 =?us-ascii?Q?RwypcL/ZanF0AfwYGj3RCFXmfSgoeKWVhWwtnCDgsZ/622e5p3nl9OTcX9p7?=
 =?us-ascii?Q?dhPv4HVLP3TVauJUcTySNkoTO5zPHd15ZmPCFgQEDbDl1mKLuO7y5IvqAAlc?=
 =?us-ascii?Q?oUp0it5Wsm3Yx8THYhTn7wRQZWEN1fGTJ6e5PAAHAdD15044Czno2v+d78MF?=
 =?us-ascii?Q?/+65umEw/5m1k3o4hZraCeVE9iViN/O5oNwtZsWuYlKt+MHoh2y+PrS/6wbO?=
 =?us-ascii?Q?rvGqwgJaqgHRXWRTAuslJWLz0EUn76Ww8rR3MsijoroqB8/7Hn2ajNS3spAk?=
 =?us-ascii?Q?iZYx8lHtPXpZ3G16BZlPZU594B28ZiAGsOybYS3bYd6Qfa8vmGEG5MMHrAD8?=
 =?us-ascii?Q?/OZxXZcCCIqNeZ5KWyrpWIBbA4d2GfwYA/FVHJ7BA7+B7lxu+oO18cwZDgdo?=
 =?us-ascii?Q?fZmXCMDr9k3IvNxoBLSxczav8bGMlKcwIUHYPkimtTbG5Z3mpOBfoY20Zp4M?=
 =?us-ascii?Q?LiVgI4MVdnF6ibb4GdBPhtk6nFctMPtBDLP/h+RarE4J76pOpThrTz7y7u2O?=
 =?us-ascii?Q?kqJuI66ICu0+3oMnLD+MfIVvrBS9uJFMgsDtwFhu/ffyvpd2u+hBdIXUg5x8?=
 =?us-ascii?Q?DtQ509neEiB5Ia8hpyqgCA2dYtyUOHqtKZOLWU6bzCEZwhQYIY6qqIeskhTw?=
 =?us-ascii?Q?qsKrTU9oQ4cMHG7fxpNGliyQVBjQQNHJ4Om6fKD7YxlaHW+focEQ5WbbpLjv?=
 =?us-ascii?Q?e44B3xBET8erFm8f86o=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DU0PR04MB9417.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b99384cd-4f98-47bf-934e-08dc34686690
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Feb 2024 12:10:18.1201
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: pscayf22oatOshqZpFEQq+eZ89Ev9rjMK3ybVHjrMJeaJqIoXsCcpJlU6O8mcX2kOkorwzIo9Xy0y2I9Qd9nFA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR04MB8844

Hi All,

Has anyone make virtio-vsock on xen work? My dm args as below:

virtio =3D [
'backend=3D0,type=3Dvirtio,device,transport=3Dpci,bdf=3D05:00.0,backend_typ=
e=3Dqemu,grant_usage=3Dtrue'
]
device_model_args =3D [
'-D', '/home/root/qemu_log.txt',
'-d', 'trace:*vsock*,trace:*vhost*,trace:*virtio*,trace:*pci_update*,trace:=
*pci_route*,trace:*handle_ioreq*,trace:*xen*',
'-device', 'vhost-vsock-pci,iommu_platform=3Dfalse,id=3Dvhost-vsock-pci0,bu=
s=3Dpcie.0,addr=3D5.0,guest-cid=3D3']

During my test, it always return failure in dom0 kernel in below code:

vhost_transport_do_send_pkt {
...
               nbytes =3D copy_to_iter(hdr, sizeof(*hdr), &iov_iter);      =
                         =20
                if (nbytes !=3D sizeof(*hdr)) {                            =
                          =20
                        vq_err(vq, "Faulted on copying pkt hdr %x %x %x %px=
\n", nbytes, sizeof(*hdr),                       =20
__builtin_object_size(hdr, 0), &iov_iter);
                        kfree_skb(skb);                                    =
                        =20
                        break;                                             =
                        =20
                } =20
}

I checked copy_to_iter, it is copy data to __user addr, but it never pass,=
=20
the copy to __user addr always return 0 bytes copied.

The asm code "sttr x7, [x6]" will trigger data abort, the kernel will run
into do_page_fault, but lock_mm_and_find_vma report it is VM_FAULT_BADMAP,
that means the __user addr is not mapped, no vma has this addr.

I am not sure what may cause this. Appreciate if any comments.

BTW: I tested blk pci, it works, so the virtio pci should work on my setup.

Thanks,
Peng.


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 12:19:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 12:19:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684840.1065001 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUVt-00034a-IR; Fri, 23 Feb 2024 12:18:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684840.1065001; Fri, 23 Feb 2024 12:18:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUVt-00034T-Fu; Fri, 23 Feb 2024 12:18:49 +0000
Received: by outflank-mailman (input) for mailman id 684840;
 Fri, 23 Feb 2024 12:18:48 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdUVs-00034K-2v
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 12:18:48 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b19f03d3-d245-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 13:18:47 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-512db550e3fso1019888e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 04:18:47 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 d11-20020a0ce44b000000b0068efc28e3d3sm8114801qvm.45.2024.02.23.04.18.45
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 04:18:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b19f03d3-d245-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708690726; x=1709295526; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=H3KsMeY6NbpcuY/BnLd20nttKnCgOMyXNwdS9PjQ60A=;
        b=tpaf3pO5O6tsnajOjO6rmQ4myoHwP5eXhNp2x6ki1QsB6RyfQAHGexPT3SU8mEJA/i
         l7xgaLOWtOLAMsVHfXTUFV8W68AJF8re8s6cVBSR0Epw1j25m8xcb2BqCCqjnGg7Ek/C
         7upWOYl22rP1pJlr3Drt8Y+53jf5MFJhppv1g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708690726; x=1709295526;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=H3KsMeY6NbpcuY/BnLd20nttKnCgOMyXNwdS9PjQ60A=;
        b=NMDl2eLTfVUOV3Hwy8kKLcAn/u4o0HrlEe497F67Gdb07zuc81HriKaaBIzCJOkL7X
         gYrYHGJnmZDxnEOFJJu3Mj0ZoM3n/H2F8jbZZsFU0okheVP2KhVch+zQHBe+ffGp4k+H
         y/xTdCaiMAHSewr4ZiiW6umc5VWolo9/3/MQzl1r/dv1FXsVhpmOwZ0vIOVEyY2NHcOu
         zvNO0z/iSQiiryB1IXWkNqyNQvPVaFlepnJvZLGmENOikJiXbAlrtmyQ/v1gqkxH8WWF
         VlClg3PBxPBy98becMczjwrF+YmS7Fti4RLlwDrFk1HG4AFjuc5acqVzbngueTtLdiBV
         8zDw==
X-Forwarded-Encrypted: i=1; AJvYcCXH17oQqXyc2385ArVdkVNSrO/EvhOooT8QM4Y4GxrxIohgozKUQya0jOqMWWti6CgoFUM50edfYHrsQKtT+OqzANgbhUAZZtM3juvLcLU=
X-Gm-Message-State: AOJu0YysAEGtKaoxjDWPhxJljAHHWEoXEik1D2Arv25nvdQGrINnY5hm
	NtRx6a6wmJjV8BjHZk0MPjSgU9IczIlcbDQAkhZYjWTCADgKwAKRtTBlccznB6o=
X-Google-Smtp-Source: AGHT+IF9V3tSRHOCwWW57WGz4nvQvTlPP7uv6Tw47RmHw6bhHcXHhXLaOTPxbtOmc1zlMp5cTaDmJQ==
X-Received: by 2002:a05:6512:1093:b0:512:e704:44bc with SMTP id j19-20020a056512109300b00512e70444bcmr1284861lfg.23.1708690726414;
        Fri, 23 Feb 2024 04:18:46 -0800 (PST)
Date: Fri, 23 Feb 2024 13:18:44 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	Matthew Grooms <mgrooms@shrew.net>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH v2] x86/altcall: use an union as register type for
 function parameters
Message-ID: <ZdiNJHErP1V-3CIi@macbook>
References: <20240222164455.67248-1-roger.pau@citrix.com>
 <b1c96b93-fd75-4077-b622-4781b7811ee8@suse.com>
 <ZdhjDBlHRtO4MIhM@macbook>
 <b62f223d-8baa-4d5b-aeba-9890c101b28c@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <b62f223d-8baa-4d5b-aeba-9890c101b28c@suse.com>

On Fri, Feb 23, 2024 at 11:43:14AM +0100, Jan Beulich wrote:
> On 23.02.2024 10:19, Roger Pau Monné wrote:
> > On Thu, Feb 22, 2024 at 05:55:00PM +0100, Jan Beulich wrote:
> >> On 22.02.2024 17:44, Roger Pau Monne wrote:
> >>> --- a/xen/arch/x86/include/asm/alternative.h
> >>> +++ b/xen/arch/x86/include/asm/alternative.h
> >>> @@ -167,9 +167,25 @@ extern void alternative_branches(void);
> >>>  #define ALT_CALL_arg5 "r8"
> >>>  #define ALT_CALL_arg6 "r9"
> >>>  
> >>> +#ifdef CONFIG_CC_IS_CLANG
> >>> +/*
> >>> + * Use an union with an unsigned long in order to prevent clang from skipping a
> >>> + * possible truncation of the value.  By using the union any truncation is
> >>> + * carried before the call instruction.
> >>> + * https://github.com/llvm/llvm-project/issues/82598
> >>> + */
> >>
> >> I think it needs saying that this is relying on compiler behavior not
> >> mandated by the standard, thus explaining why it's restricted to
> >> Clang (down the road we may even want to restrict to old versions,
> >> assuming they fix the issue at some point). Plus also giving future
> >> readers a clear understanding that if something breaks with this, it's
> >> not really a surprise.
> > 
> > What about:
> > 
> > Use a union with an unsigned long in order to prevent clang from
> > skipping a possible truncation of the value.  By using the union any
> > truncation is carried before the call instruction.
> 
> ..., in turn covering for ABI-non-compliance in that the necessary
> clipping / extension of the value is supposed to be carried out in
> the callee.
> 
> >  Note this
> > behavior is not mandated by the standard, and hence could stop being
> > a viable workaround, or worse, could cause a different set of
> > code-generation issues in future clang versions.
> > 
> > This has been reported upstream at:
> > https://github.com/llvm/llvm-project/issues/82598
> > 
> >> Aiui this bug is only a special case of the other, much older one, so
> >> referencing that one here too would seem advisable.
> > 
> > My report has been resolved as a duplicate of:
> > 
> > https://github.com/llvm/llvm-project/issues/43573
> > 
> > FWIW, I think for the context the link is used in (altcall) my bug
> > report is more representative, and readers can always follow the trail
> > into the other inter-related bugs.
> 
> While true, the comment extension suggested above goes beyond that
> territory, and there the other bug is quite relevant directly. After all
> what your change does is papering over a knock-on effect of them not
> following the ABI. And that simply because it is pretty hard to see how
> we could work around the ABI non-conformance itself (which btw could
> bite us if we had any affected C function called from assembly).
> 
> 43537 looks to be a newer instance of 12579; funny they didn't close
> that as a duplicate then, too.

So would you be OK with the following:

Use a union with an unsigned long in order to prevent clang from
skipping a possible truncation of the value.  By using the union any
truncation is carried before the call instruction, in turn covering
for ABI-non-compliance in that the necessary clipping / extension of
the value is supposed to be carried out in the callee.

Note this behavior is not mandated by the standard, and hence could
stop being a viable workaround, or worse, could cause a different set
of code-generation issues in future clang versions.

This has been reported upstream at:
https://github.com/llvm/llvm-project/issues/12579

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 12:23:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 12:23:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684844.1065012 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUaT-0005DO-30; Fri, 23 Feb 2024 12:23:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684844.1065012; Fri, 23 Feb 2024 12:23:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUaS-0005DH-WC; Fri, 23 Feb 2024 12:23:33 +0000
Received: by outflank-mailman (input) for mailman id 684844;
 Fri, 23 Feb 2024 12:23:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=X/z8=KA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rdUaR-0005DB-I1
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 12:23:31 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5ab767b5-d246-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 13:23:30 +0100 (CET)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-512bb2ed1f7so324011e87.3
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 04:23:30 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 d11-20020ac2544b000000b00511ac5e89f7sm2408289lfn.300.2024.02.23.04.23.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 04:23:29 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5ab767b5-d246-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708691010; x=1709295810; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=y2xfGF03HxvHDka7iopRcCUIz+v7jFXuNdrwq7NKmek=;
        b=VsF+9raG9jb4h2qccDKikOtN2BjHw4xsnL/ZoMR57prQchWhuw1rCjaWtroAFHs2ah
         XU3Q0aDoWZ9N3EOxMLcaO+MCsu5BlA4XpBlmGPjLGB/GpKjConXY0kxy48ycgBihbOqK
         4cMxjr6yL91ugnXz6nVpsoufWdkVgMbBYiJLHynw4Ka4xSFmw33s0el/fx7XJrhbTCIP
         nzp7zbU2hqhJLfPY1hPSAmfzPC8I5Mh9hD/9Taw6wzsvIiFeM0Zk7IK0vGonemTyPjPN
         TeseUD9zobuaJLvBRJcgTJlllotBH5UPVRjKb+nAhUIiThl95oHjzh87JZks+pjSU5i4
         i9JA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708691010; x=1709295810;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=y2xfGF03HxvHDka7iopRcCUIz+v7jFXuNdrwq7NKmek=;
        b=EpZsPzVl0gWKwg2dkiX0tj44G/C0hJ5sYkgpJA6mAtu5iraa6lOI5a/ml7D/TTVGz6
         hEH5Fv+nDdHpsC1UYDxQyw5GD9n0NZ4Y5yhrPQM5+spziz53jRDPlbv8xfAT/mBjYiN7
         fPEsjWQl0J4MefwCTi+mB4nBi5W/BmL6X9BzPhLb0/QvmLlYBjB4klOdTtjUK9hxWu/j
         oWYIVL19jTcI+Ttf6PkQCyXoYsLzd5XUheNm3Md+lsdveRwhDzs2MwvSKzgsBbEMchzU
         qWc9Zjg4ext+0Ma2XTYqvuiOZ9+gLV8Gd1j00gIY/I6F6HzTTe7CKoAnHHVR/LtM6oEU
         Rnsg==
X-Forwarded-Encrypted: i=1; AJvYcCVBC9IJg+JuJm6mYWwIIDTygWVNCoy+npaAPpUfU6u4nQGaw7Roy2fsvUk6M6rMM+Z9gAKz+x0D0kuzjqZuJoaRiPPLZNyaLah7GClCxlk=
X-Gm-Message-State: AOJu0YzBtYwnnSmVbQGG0oDn/pIBED/gNMLbhd+zh6bD2XZ4AZFouK/d
	TUWT5J3y0anXMtOw4iPUsrRRDAkt8YRwIzfx25S3PMPSoGPzbfue
X-Google-Smtp-Source: AGHT+IEJElHU+nDwe3nm6XMstI9N3hjNwEKsDwOKEoMupGiR8lWmTof+odxyFL13CfL8Nyn7KrG+hw==
X-Received: by 2002:ac2:5934:0:b0:512:d81f:c22e with SMTP id v20-20020ac25934000000b00512d81fc22emr1206630lfi.59.1708691009929;
        Fri, 23 Feb 2024 04:23:29 -0800 (PST)
Message-ID: <5d2d35fe014094c991363e42c3e2ad9ca2af3938.camel@gmail.com>
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Fri, 23 Feb 2024 13:23:28 +0100
In-Reply-To: <3b38fe82-ee0f-4666-93e1-bd78fe69c534@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
	 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
	 <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
	 <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
	 <1c53b52ee39161a8f59209d28af69fe997479dbc.camel@gmail.com>
	 <3b38fe82-ee0f-4666-93e1-bd78fe69c534@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

>=20
> > > > As 1- and 2-byte cases are emulated I decided that is not to
> > > > provide
> > > > sfx argument for emulation macros as it will not have to much
> > > > affect on
> > > > emulated types and just consume more performance on acquire and
> > > > release
> > > > version of sc/ld instructions.
> > >=20
> > > Question is whether the common case (4- and 8-byte accesses)
> > > shouldn't
> > > be valued higher, with 1- and 2-byte emulation being there just
> > > to
> > > allow things to not break altogether.
> > If I understand you correctly, it would make sense to add the 'sfx'
> > argument for the 1/2-byte access case, ensuring that all options
> > are
> > available for 1/2-byte access case as well.
>=20
> That's one of the possibilities. As said, I'm not overly worried
> about
> the emulated cases. For the initial implementation I'd recommend
> going
> with what is easiest there, yielding the best possible result for the
> 4- and 8-byte cases. If later it turns out repeated acquire/release
> accesses are a problem in the emulation loop, things can be changed
> to explicit barriers, without touching the 4- and 8-byte cases.
I am confused then a little bit if emulated case is not an issue.

For 4- and 8-byte cases for xchg .aqrl is used, for relaxed and aqcuire
version of xchg barries are used.

The similar is done for cmpxchg.

If something will be needed to change in emulation loop it won't
require to change 4- and 8-byte cases.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 12:24:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 12:24:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684847.1065021 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUba-0005jW-Be; Fri, 23 Feb 2024 12:24:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684847.1065021; Fri, 23 Feb 2024 12:24:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUba-0005jP-8t; Fri, 23 Feb 2024 12:24:42 +0000
Received: by outflank-mailman (input) for mailman id 684847;
 Fri, 23 Feb 2024 12:24:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=6083=KA=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rdUbY-0005jD-OI
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 12:24:40 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 831c6e53-d246-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 13:24:38 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-5643ae47cd3so891108a12.3
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 04:24:38 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 q29-20020a50cc9d000000b00563a3ff30basm6806208edi.59.2024.02.23.04.24.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 23 Feb 2024 04:24:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 831c6e53-d246-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708691078; x=1709295878; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=B4I9sZOXJcqzreICLCN3hytYAXvuabcNleeaVNRUsbI=;
        b=bHDZMfdUkxgkX4TffbTVl573m4sYD7zU2ifIsrqIvziOs6sS7/ZMeuMTZxMrH/VdUr
         63Vq5Qj8boBl5+8ERUTXQ611mbEftLGs4/2SGfpHGhuijFbCQ+EfFiUKPnvGZAhLN1O0
         A7vF76lKnvZMgsztkDMY4MiO5lTdESVLYdrvCrZUC9iXUOCb/Eo5VM/xPrWkS53IMFV9
         nf5G4r5uEfx4JiSQzi6sugJx0F7+z5idz+og6dcZJefc+8/2qyn32J1QilAOpPYwAOQS
         v7QsSC1AvWheouzr4COLq+0J+xvpTN67natc6pV5D3tvEHv/oW//gi20HiGOJRr2Xrve
         p+yQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708691078; x=1709295878;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=B4I9sZOXJcqzreICLCN3hytYAXvuabcNleeaVNRUsbI=;
        b=iD2c64L4wEBMtOcu4Eiygq7YKgsxdzCwZ2iwO2ptgotrQ8tOmbLxcYnd9hTlABEvEK
         9MCvH3c0X52KTvDbJHCIJmyTCnvAa7Yx/w5cjhHTLxOBuRbHkspNE5mvJOW/F7fUqIn7
         sVItN9tjevtbJZdth1K7Vd7isq5IwPWC6mhr1nhEFY3h/2Y1hrb1N2aUylBN1cAFtDY+
         I2hiPW64AmkIc7RusFFkfQOe5ZIQZeHS0y8OCnUq1tzGiHS4L9iF87jgUuOEpxpgLjcJ
         g9dw2xlCzN1538+RnQCnrEdyDAC/MblUlpOfhET/x7Uvs255HQ8W+WKneze9CMnUazYM
         qKuA==
X-Forwarded-Encrypted: i=1; AJvYcCV+fSEizMW8k/HV1c1ATSW5EaaVSnk3S7oPMsyaFka48LPFOwrDEHHqdoIetgT3N9Fof0LrZFQkDlb12956qxPdIBe43m3C2VOraUSjPNU=
X-Gm-Message-State: AOJu0YxCLj8j6Wn7HamnNQ3G/1dnKmCqWQZNo5kXj8gfZzttruOv19Sk
	ivkQF+dHzu9iJuQsPtEmWamgU9g6r+rrp6yEy6VznkbM/iuDLN4UcG4MIuVx0g==
X-Google-Smtp-Source: AGHT+IEqiazWVbPH8vIlUohZwm2+g5dBndk+3IyGp9Ma66OsdTPaK0lu7KlAqN/37lZg0rHIEsxJ2w==
X-Received: by 2002:a05:6402:254c:b0:565:7114:ef37 with SMTP id l12-20020a056402254c00b005657114ef37mr1438498edb.12.1708691078043;
        Fri, 23 Feb 2024 04:24:38 -0800 (PST)
Message-ID: <fd3e6d0f-897b-490f-937f-5127a5c13636@suse.com>
Date: Fri, 23 Feb 2024 13:24:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/altcall: use an union as register type for
 function parameters
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 Matthew Grooms <mgrooms@shrew.net>, xen-devel@lists.xenproject.org
References: <20240222164455.67248-1-roger.pau@citrix.com>
 <b1c96b93-fd75-4077-b622-4781b7811ee8@suse.com> <ZdhjDBlHRtO4MIhM@macbook>
 <b62f223d-8baa-4d5b-aeba-9890c101b28c@suse.com> <ZdiNJHErP1V-3CIi@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZdiNJHErP1V-3CIi@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.02.2024 13:18, Roger Pau Monné wrote:
> On Fri, Feb 23, 2024 at 11:43:14AM +0100, Jan Beulich wrote:
>> On 23.02.2024 10:19, Roger Pau Monné wrote:
>>> On Thu, Feb 22, 2024 at 05:55:00PM +0100, Jan Beulich wrote:
>>>> On 22.02.2024 17:44, Roger Pau Monne wrote:
>>>>> --- a/xen/arch/x86/include/asm/alternative.h
>>>>> +++ b/xen/arch/x86/include/asm/alternative.h
>>>>> @@ -167,9 +167,25 @@ extern void alternative_branches(void);
>>>>>  #define ALT_CALL_arg5 "r8"
>>>>>  #define ALT_CALL_arg6 "r9"
>>>>>  
>>>>> +#ifdef CONFIG_CC_IS_CLANG
>>>>> +/*
>>>>> + * Use an union with an unsigned long in order to prevent clang from skipping a
>>>>> + * possible truncation of the value.  By using the union any truncation is
>>>>> + * carried before the call instruction.
>>>>> + * https://github.com/llvm/llvm-project/issues/82598
>>>>> + */
>>>>
>>>> I think it needs saying that this is relying on compiler behavior not
>>>> mandated by the standard, thus explaining why it's restricted to
>>>> Clang (down the road we may even want to restrict to old versions,
>>>> assuming they fix the issue at some point). Plus also giving future
>>>> readers a clear understanding that if something breaks with this, it's
>>>> not really a surprise.
>>>
>>> What about:
>>>
>>> Use a union with an unsigned long in order to prevent clang from
>>> skipping a possible truncation of the value.  By using the union any
>>> truncation is carried before the call instruction.
>>
>> ..., in turn covering for ABI-non-compliance in that the necessary
>> clipping / extension of the value is supposed to be carried out in
>> the callee.
>>
>>>  Note this
>>> behavior is not mandated by the standard, and hence could stop being
>>> a viable workaround, or worse, could cause a different set of
>>> code-generation issues in future clang versions.
>>>
>>> This has been reported upstream at:
>>> https://github.com/llvm/llvm-project/issues/82598
>>>
>>>> Aiui this bug is only a special case of the other, much older one, so
>>>> referencing that one here too would seem advisable.
>>>
>>> My report has been resolved as a duplicate of:
>>>
>>> https://github.com/llvm/llvm-project/issues/43573
>>>
>>> FWIW, I think for the context the link is used in (altcall) my bug
>>> report is more representative, and readers can always follow the trail
>>> into the other inter-related bugs.
>>
>> While true, the comment extension suggested above goes beyond that
>> territory, and there the other bug is quite relevant directly. After all
>> what your change does is papering over a knock-on effect of them not
>> following the ABI. And that simply because it is pretty hard to see how
>> we could work around the ABI non-conformance itself (which btw could
>> bite us if we had any affected C function called from assembly).
>>
>> 43537 looks to be a newer instance of 12579; funny they didn't close
>> that as a duplicate then, too.
> 
> So would you be OK with the following:

Yes, ...

> Use a union with an unsigned long in order to prevent clang from
> skipping a possible truncation of the value.  By using the union any
> truncation is carried before the call instruction, in turn covering
> for ABI-non-compliance in that the necessary clipping / extension of
> the value is supposed to be carried out in the callee.
> 
> Note this behavior is not mandated by the standard, and hence could
> stop being a viable workaround, or worse, could cause a different set
> of code-generation issues in future clang versions.
> 
> This has been reported upstream at:
> https://github.com/llvm/llvm-project/issues/12579

... yet perhaps still list your new bug report here as well.

Jan


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 12:37:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 12:37:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684855.1065031 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUo3-00086f-IS; Fri, 23 Feb 2024 12:37:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684855.1065031; Fri, 23 Feb 2024 12:37:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdUo3-00086Y-Fj; Fri, 23 Feb 2024 12:37:35 +0000
Received: by outflank-mailman (input) for mailman id 684855;
 Fri, 23 Feb 2024 12:37:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdUo2-00086O-S5; Fri, 23 Feb 2024 12:37:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdUo2-0001Lo-Op; Fri, 23 Feb 2024 12:37:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdUo2-0002M5-9L; Fri, 23 Feb 2024 12:37:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdUo2-0000vK-6N; Fri, 23 Feb 2024 12:37:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=6wqB65vyE4qaB29fhdh/wYqxF+Cb0eu44xNEzqr9jl0=; b=dOw5GIRlrIwGlmS1PCOv2V5baZ
	3bVHUPd9kzlDEtmNTID9R6YBlSB6FACFZGu6SRwy99q4lpes+t3L71lWeYNNHFAMuYj7tzYsNNRec
	I1/yUhUfhlLG8PeVON5VvF0vwILdYYAlVZ7ylaNBi8Z6dfEUznJHP9tjN/5MK5POZE9E=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184731-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184731: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-xl:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=ffd2cb6b718e189e7e2d5d0c19c25611f92e061a
X-Osstest-Versions-That:
    linux=39133352cbed6626956d38ed72012f49b0421e7b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 23 Feb 2024 12:37:34 +0000

flight 184731 linux-linus real [real]
flight 184735 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184731/
http://logs.test-lab.xenproject.org/osstest/logs/184735/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl           8 xen-boot            fail pass in 184735-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl         15 migrate-support-check fail in 184735 never pass
 test-armhf-armhf-xl     16 saverestore-support-check fail in 184735 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184725
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184725
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184725
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184725
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184725
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184725
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184725
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184725
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                ffd2cb6b718e189e7e2d5d0c19c25611f92e061a
baseline version:
 linux                39133352cbed6626956d38ed72012f49b0421e7b

Last test of basis   184725  2024-02-22 08:27:10 Z    1 days
Testing same since   184731  2024-02-22 21:44:16 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Elder <elder@linaro.org>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Kobel <a-kobel@a-kobel.de>
  Alexei Starovoitov <ast@kernel.org>
  Andrii Nakryiko <andrii@kernel.org>
  Avihai Horon <avihaih@nvidia.com>
  Baokun Li <libaokun1@huawei.com>
  Bart Van Assche <bvanassche@acm.org>
  Christian Brauner <brauner@kernel.org>
  Daniel Borkmann <daniel@iogearbox.net>
  Daniil Dulov <d.dulov@aladdin.ru>
  David Howells <dhowells@redhat.com>
  David S. Miller <davem@davemloft.net>
  David Vernet <void@manifault.com>
  Dominique Martinet <asmadeus@codewreck.org>
  Eric Dumazet <edumazet@google.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Florian Westphal <fw@strlen.de>
  Geliang Tang <tanggeliang@kylinos.cn>
  Gianmarco Lusvardi <glusvardi@posteo.net>
  Greg Joyce <gjoyce@linux.ibm.com>
  Hangbin Liu <liuhangbin@gmail.com>
  Hans de Goede <hdegoede@redhat.com>
  Hari Bathini <hbathini@linux.ibm.com>
  Horatiu Vultur <horatiu.vultur@microchip.com>
  Hou Tao <houtao1@huawei.com>
  Jakub Kicinski <kuba@kernel.org>
  Jamal Hadi Salim <jhs@mojatatu.com>
  Jason Gunthorpe <jgg@nvidia.com>
  Jens Axboe <axboe@kernel.dk>
  Jeremy Kerr <jk@codeconstruct.com.au>
  Jianbo Liu <jianbol@nvidia.com>
  Jiri Olsa <jolsa@kernel.org>
  Jiri Pirko <jiri@nvidia.com>
  Joao Martins <joao.m.martins@oracle.com>
  John Fastabend <john.fastabend@gmail.com>
  Justin Chen <justin.chen@broadcom.com>
  Justin Iurman <justin.iurman@uliege.be>
  Kees Cook <keescook@chromium.org>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Kurt Kanzenbach <kurt@linutronix.de>
  Linus Torvalds <torvalds@linux-foundation.org>
  Maciej Fijalkowski <maciej.fijalkowski@intel.com>
  Marc Dionne <marc.dionne@auristor.com>
  Mario Limonciello <mario.limonciello@amd.com>
  Mark Pearson <mpearson-lenovo@squebb.ca>
  Martin KaFai Lau <martin.lau@kernel.org>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Muhammad Usama Anjum <usama.anjum@collabora.com>
  Nicolas Dichtel <nicolas.dichtel@6wind.com>
  Oleg Nesterov <oleg@redhat.com>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Paolo Abeni <pabeni@redhat.com>
  Patil Rajesh Reddy <Patil.Reddy@amd.com>
  Pavel Sakharov <p.sakharov@ispras.ru>
  Randy Dunlap <rdunlap@infradead.org>
  Rob Herring <robh@kernel.org>
  Rémi Denis-Courmont <courmisch@gmail.com>
  Sabrina Dubroca <sd@queasysnail.net>
  Sebastian Andrzej Siewior <bigeasy@linutronix.de>
  Shannon Nelson <shannon.nelson@amd.com>
  Shigeru Yoshida <syoshida@redhat.com>
  Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
  Siddharth Vadapalli <s-vadapalli@ti.com>
  Simon Horman <horms@kernel.org>
  Song Liu <song@kernel.org>
  Stanislav Fomichev <sdf@google.com>
  Stephen Boyd <sboyd@kernel.org>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Subbaraya Sundeep <sbhatta@marvell.com>
  syzbot+fd7b34375c1c8ce29c93@syzkaller.appspotmail.com
  Thomas Gleixner <tglx@linutronix.de>
  Tobias Waldekranz <tobias@waldekranz.com>
  Tom Parkin <tparkin@katalix.com>
  Tudor Ambarus <tudor.ambarus@linaro.org>
  Vasiliy Kovalev <kovalev@altlinux.org>
  Yafang Shao <laoar.shao@gmail.com>
  Yonghong Song <yonghong.song@linux.dev>
  Yu Kuai <yukuai3@huawei.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          fail    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   39133352cbed6..ffd2cb6b718e1  ffd2cb6b718e189e7e2d5d0c19c25611f92e061a -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 13:04:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 13:04:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684862.1065042 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdVE9-0004qg-N9; Fri, 23 Feb 2024 13:04:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684862.1065042; Fri, 23 Feb 2024 13:04:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdVE9-0004qZ-JT; Fri, 23 Feb 2024 13:04:33 +0000
Received: by outflank-mailman (input) for mailman id 684862;
 Fri, 23 Feb 2024 13:04:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdVE9-0004qT-6F
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 13:04:33 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 15eaccdd-d24c-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 14:04:32 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-512cca90f38so921229e87.2
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 05:04:32 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 r19-20020ac84253000000b0042e30153d7csm3115670qtm.2.2024.02.23.05.04.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 05:04:22 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15eaccdd-d24c-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708693471; x=1709298271; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=9i4075X9PLnoKVpgo6Uj90eLDaJ7j9BjLkIR0nRpAH0=;
        b=V8fXwPYTUBOKWFemTyuBiUNbxCddOkBGhNNO6GjtpOskKQFA8bXfOyYDkQ41UyEbj1
         zEKJ+ZrQhxvlv+OHXUYPVN0uZEY/XB7EZbsd5K0YOG9VK9i+KIqRcnW3+KpxQTuzOZ9j
         Bf2Yt+7xQaI+ehI4+OezjKETWZWEhJYdvv554=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708693471; x=1709298271;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=9i4075X9PLnoKVpgo6Uj90eLDaJ7j9BjLkIR0nRpAH0=;
        b=epuHDKyO9tv17N7M1y1+kLZgK+LQuyn+1/zvnvVYF8vuRAWlUwgoOIqTCFq4rqaovd
         scg44dYmwXjMs6VsePV8gogtWHx2tyUM+zI6p4GNHdw2ZR//yynMYW3rQDkpeLPEhLNv
         mnnZwtzm7XZZKJt+34ezLN4DoKA9SQeW5coMe4af+zHTFWlvFCED4d1M9qVb36c4f7Ul
         rzPhk2mAFnXBbnrAf9MTyXXGndnArGHKF46dlPlFuv36o4+A0HYc5whEaH7WXhB2M3++
         tGAK3gh1+0AjgJD76mZ/LlTvlQ9vnmpWGos1+/oGD7cvmB3v8ywgaUqAgtUaiYMKFrYi
         3gZA==
X-Gm-Message-State: AOJu0YwOMMekln+i0xDy9JWe7omumn8cssNJ88N0Gu7DX80RBaxfudRN
	TkqCM3o35DUfJV4KSQE8tqMbQwl/l2SZu2awnBQYw1+NtBZhG5O1VjurmTeqUWvko0NzuN3pPwJ
	2
X-Google-Smtp-Source: AGHT+IECUZ0RXhIwP4VagDHxPgB4V67aKSns1JCI1uE9dRzz8mQ/h0v3rWTvi+g0md61K3QbgD/GXQ==
X-Received: by 2002:ac2:4d03:0:b0:512:b24c:dba with SMTP id r3-20020ac24d03000000b00512b24c0dbamr1184494lfi.60.1708693470944;
        Fri, 23 Feb 2024 05:04:30 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>,
	Matthew Grooms <mgrooms@shrew.net>
Subject: [PATCH v3] x86/altcall: use an union as register type for function parameters on clang
Date: Fri, 23 Feb 2024 14:03:54 +0100
Message-ID: <20240223130354.73936-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The current code for alternative calls uses the caller parameter types as the
types for the register variables that serve as function parameters:

uint8_t foo;
[...]
alternative_call(myfunc, foo);

Would expand roughly into:

register unint8_t a1_ asm("rdi") = foo;
register unsigned long a2_ asm("rsi");
[...]
asm volatile ("call *%c[addr](%%rip)"...);

However with -O2 clang will generate incorrect code, given the following
example:

unsigned int func(uint8_t t)
{
    return t;
}

static void bar(uint8_t b)
{
    int ret_;
    register uint8_t di asm("rdi") = b;
    register unsigned long si asm("rsi");
    register unsigned long dx asm("rdx");
    register unsigned long cx asm("rcx");
    register unsigned long r8 asm("r8");
    register unsigned long r9 asm("r9");
    register unsigned long r10 asm("r10");
    register unsigned long r11 asm("r11");

    asm volatile ( "call %c[addr]"
                   : "+r" (di), "=r" (si), "=r" (dx),
                     "=r" (cx), "=r" (r8), "=r" (r9),
                     "=r" (r10), "=r" (r11), "=a" (ret_)
                   : [addr] "i" (&(func)), "g" (func)
                   : "memory" );
}

void foo(unsigned int a)
{
    bar(a);
}

Clang generates the following assembly code:

func:                                   # @func
        movl    %edi, %eax
        retq
foo:                                    # @foo
        callq   func
        retq

Note the truncation of the unsigned int parameter 'a' of foo() to uint8_t when
passed into bar() is lost.  clang doesn't zero extend the parameters in the
callee when required, as the psABI mandates.

The above can be worked around by using an union when defining the register
variables, so that `di` becomes:

register union {
    uint8_t e;
    unsigned long r;
} di asm("rdi") = { .e = b };

Which results in following code generated for `foo()`:

foo:                                    # @foo
        movzbl  %dil, %edi
        callq   func
        retq

So the truncation is not longer lost.  Apply such workaround only when built
with clang.

Reported-by: Matthew Grooms <mgrooms@shrew.net>
Link: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277200
Link: https://github.com/llvm/llvm-project/issues/12579
Link: https://github.com/llvm/llvm-project/issues/82598
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v2:
 - Expand the code comment.

Changes since v1:
 - Only apply the union workaround with clang.

Seems like all gitlab build tests are OK with this approach.
---
 xen/arch/x86/include/asm/alternative.h | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/xen/arch/x86/include/asm/alternative.h b/xen/arch/x86/include/asm/alternative.h
index a1cd6a9fe5b8..3c14db5078ba 100644
--- a/xen/arch/x86/include/asm/alternative.h
+++ b/xen/arch/x86/include/asm/alternative.h
@@ -167,9 +167,34 @@ extern void alternative_branches(void);
 #define ALT_CALL_arg5 "r8"
 #define ALT_CALL_arg6 "r9"
 
+#ifdef CONFIG_CC_IS_CLANG
+/*
+ * Use a union with an unsigned long in order to prevent clang from
+ * skipping a possible truncation of the value.  By using the union any
+ * truncation is carried before the call instruction, in turn covering
+ * for ABI-non-compliance in that the necessary clipping / extension of
+ * the value is supposed to be carried out in the callee.
+ *
+ * Note this behavior is not mandated by the standard, and hence could
+ * stop being a viable workaround, or worse, could cause a different set
+ * of code-generation issues in future clang versions.
+ *
+ * This has been reported upstream:
+ * https://github.com/llvm/llvm-project/issues/12579
+ * https://github.com/llvm/llvm-project/issues/82598
+ */
+#define ALT_CALL_ARG(arg, n)                                            \
+    register union {                                                    \
+        typeof(arg) e;                                                  \
+        unsigned long r;                                                \
+    } a ## n ## _ asm ( ALT_CALL_arg ## n ) = {                         \
+        .e = ({ BUILD_BUG_ON(sizeof(arg) > sizeof(void *)); (arg); })   \
+    }
+#else
 #define ALT_CALL_ARG(arg, n) \
     register typeof(arg) a ## n ## _ asm ( ALT_CALL_arg ## n ) = \
         ({ BUILD_BUG_ON(sizeof(arg) > sizeof(void *)); (arg); })
+#endif
 #define ALT_CALL_NO_ARG(n) \
     register unsigned long a ## n ## _ asm ( ALT_CALL_arg ## n )
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 14:56:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 14:56:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684870.1065052 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdWy3-0001yr-Hy; Fri, 23 Feb 2024 14:56:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684870.1065052; Fri, 23 Feb 2024 14:56:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdWy3-0001yi-FH; Fri, 23 Feb 2024 14:56:03 +0000
Received: by outflank-mailman (input) for mailman id 684870;
 Fri, 23 Feb 2024 14:56:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=lysb=KA=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rdWy2-0001yc-AH
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 14:56:02 +0000
Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com
 [2a00:1450:4864:20::231])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a8db7696-d25b-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 15:56:01 +0100 (CET)
Received: by mail-lj1-x231.google.com with SMTP id
 38308e7fff4ca-2d0a4e1789cso11754261fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 06:56:01 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 bi9-20020a05620a318900b00787af8b5c02sm881863qkb.39.2024.02.23.06.55.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 06:56:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8db7696-d25b-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708700161; x=1709304961; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=eZS4hr9rfgbLObhuwrrnLB/QJjAbInDizKuJeP1mGJo=;
        b=NEfWTrv7zjugtuoFIBx2penBjVSBGO02uEebsJWqf5hiWTYLTgCdUe3qRCQOGDtzaF
         WR3+lmXLIM6en4rU5MWeeMPHDEEhYjMed5/X3MxmvYnpr5aatxHYUQmXThQwOgGbNzPE
         dQzf/WlrveaVhW53ZOidfxeM0Ts1mEc5zFT7w=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708700161; x=1709304961;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=eZS4hr9rfgbLObhuwrrnLB/QJjAbInDizKuJeP1mGJo=;
        b=XOrivJj1dm1O5p6m5zyl5DKweVFmH3fNE7g6ISRuMbt3XcxJdyzIzmsUgpp4cZvMDG
         KGlgYCPYGJ/mutYD8Kg49qtM4aeVJOhuMb1ZrHX75i+oEtXKpg7k+A/1MXFd4NTq6ZDB
         HaP7WwwhU0pmKK/SENjdQe/8iu5ISrN/egP1cssmpDYyANiyTAIYHStfojufbYMhI/Fq
         32esm7QAQq9jHnM+/pDrc3Rx8iNs9R+Y0eQFt1c/wK5Jg7Yo1eN5LjVx9iVJPlTv/V6l
         SSgj6QLcjD54OyxcplhttKelKdRGWi4TDyjRXmrTfqpHcSOWsuMpmaFquDtYECRqRhRu
         JjOA==
X-Gm-Message-State: AOJu0YwNFO9MabobRwQmG1/lWgnNLyqoFvLsKI/c/sNB3v6Fdm3UmijQ
	gBHN/ErrQu7GL0kEGZzRxJkuGG8QNEAUGw3QwEKl7LxWCmri+U6bFc/AqPML5sQ=
X-Google-Smtp-Source: AGHT+IHEm3daiiwXU0yFlknywEAOBblajlB571JAVqKO5xxWi0iOK2dyC+4t7C+MNa3oDhOeXk++iA==
X-Received: by 2002:a2e:9d0a:0:b0:2d2:4ce4:bf4a with SMTP id t10-20020a2e9d0a000000b002d24ce4bf4amr37564lji.39.1708700160744;
        Fri, 23 Feb 2024 06:56:00 -0800 (PST)
Date: Fri, 23 Feb 2024 15:55:58 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Ross Lagerwall <ross.lagerwall@cloud.com>
Cc: xen-devel@lists.xenproject.org,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH 1/5] xen/livepatch: register livepatch regions when loaded
Message-ID: <Zdix_iZFYnvweQuR@macbook>
References: <20231130142944.46322-1-roger.pau@citrix.com>
 <20231130142944.46322-2-roger.pau@citrix.com>
 <CAG7k0Eq+G7w8M2JFTnikv10SNocfCBGwvbEwz-pbR9v99uStGg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAG7k0Eq+G7w8M2JFTnikv10SNocfCBGwvbEwz-pbR9v99uStGg@mail.gmail.com>

On Fri, Feb 23, 2024 at 09:45:15AM +0000, Ross Lagerwall wrote:
> On Thu, Nov 30, 2023 at 2:30 PM Roger Pau Monne <roger.pau@citrix.com> wrote:
> >
> > Currently livepatch regions are registered as virtual regions only after the
> > livepatch has been applied.
> >
> > This can lead to issues when using the pre-apply or post-revert hooks, as at
> > the point the livepatch is not in the virtual regions list.  If a livepatch
> > pre-apply hook contains a WARN() it would trigger an hypervisor crash, as the
> > code to handle the bug frame won't be able to find the instruction pointer that
> > triggered the #UD in any of the registered virtual regions, and hence crash.
> >
> > Fix this by adding the livepatch payloads as virtual regions as soon as loaded,
> > and only remove them once the payload is unloaded.  This requires some changes
> > to the virtual regions code, as the removal of the virtual regions is no longer
> > done in stop machine context, and hence an RCU barrier is added in order to
> > make sure there are no users of the virtual region after it's been removed from
> > the list.
> >
> > Fixes: 8313c864fa95 ('livepatch: Implement pre-|post- apply|revert hooks')
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> >  xen/common/livepatch.c      |  5 +++--
> >  xen/common/virtual_region.c | 40 +++++++++++--------------------------
> >  2 files changed, 15 insertions(+), 30 deletions(-)
> >
> > diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
> > index 1209fea2566c..3199432f11f5 100644
> > --- a/xen/common/livepatch.c
> > +++ b/xen/common/livepatch.c
> > @@ -942,6 +942,8 @@ static int prepare_payload(struct payload *payload,
> >          }
> >      }
> >
> > +    register_virtual_region(region);
> > +
> >      return 0;
> >  }
> >
> 
> The region is registered in prepare_payload() but if e.g. the build id
> check in load_payload_data() fails, it won't unregister the region
> since the failure path calls free_payload_data(), not free_payload().
> Perhaps the call to register_virtual_region() could be done as the last
> thing in load_payload_data()?

I've moved it to livepatch_upload(), as I think it's more natural to
be done together with the addition to the list of livepatches.  Thanks
for spotting it, I guess I got confused between free_payload_data()
free_payload().

Roger.


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 15:09:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 15:09:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684875.1065062 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdXBL-0003kT-TG; Fri, 23 Feb 2024 15:09:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684875.1065062; Fri, 23 Feb 2024 15:09:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdXBL-0003kM-PZ; Fri, 23 Feb 2024 15:09:47 +0000
Received: by outflank-mailman (input) for mailman id 684875;
 Fri, 23 Feb 2024 15:09:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdXBK-0003kC-IW; Fri, 23 Feb 2024 15:09:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdXBK-0004RK-B8; Fri, 23 Feb 2024 15:09:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdXBK-000247-25; Fri, 23 Feb 2024 15:09:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdXBK-0007qM-1d; Fri, 23 Feb 2024 15:09:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=0Q8LwGneS4af054MEmDQVuymhYmqAg9qorqzY6vVAi0=; b=PhW7+q/dhWHFMHzDCOZIy8W0rZ
	LQ2O2HITibKAc+yKR82ti1qxv/08de8oDQ4CGs98iOibcN+4l1+nChVSBVOh9L4yMvFEnM/NjeSA0
	hzF/HYHkSy4e06KNfACBd9hpbTo2MTDw2dA7gs+6l8bTK1TNFRrJgO+AqiAluSgPMIXc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184736-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184736: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=11ad164bcea6b0ed3628d595090f84892c367086
X-Osstest-Versions-That:
    ovmf=8ccd63d14da5678a4b95df0aa954a2378355af9b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 23 Feb 2024 15:09:46 +0000

flight 184736 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184736/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 11ad164bcea6b0ed3628d595090f84892c367086
baseline version:
 ovmf                 8ccd63d14da5678a4b95df0aa954a2378355af9b

Last test of basis   184724  2024-02-22 03:43:20 Z    1 days
Testing same since   184736  2024-02-23 13:13:11 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Dhaval <dhaval@rivosinc.com>
  Dhaval Sharma <dhaval@rivosinc.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   8ccd63d14d..11ad164bce  11ad164bcea6b0ed3628d595090f84892c367086 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 15:23:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 15:23:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684888.1065100 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdXOH-0006VH-Ar; Fri, 23 Feb 2024 15:23:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684888.1065100; Fri, 23 Feb 2024 15:23:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdXOH-0006VA-7a; Fri, 23 Feb 2024 15:23:09 +0000
Received: by outflank-mailman (input) for mailman id 684888;
 Fri, 23 Feb 2024 15:23:08 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdXOG-0006Us-6X; Fri, 23 Feb 2024 15:23:08 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdXOG-0004gx-4H; Fri, 23 Feb 2024 15:23:08 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdXOF-0002Ma-O4; Fri, 23 Feb 2024 15:23:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdXOF-0004gy-Nf; Fri, 23 Feb 2024 15:23:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=X/iMk6JAu48AbAetDFB7AR6Xae9RYPeaT3hvxx9NStE=; b=w7a6dJpezBgkLLPOq7roCLL5Oy
	sOBReUTIt/gNn+rEa1jWoxYfTxUMC1YkOq9DyIe2PxFTzP2+hB7pD66v3wNaD/MSaYxmf38wcpYrZ
	cMET3VVKMYUbzgkNuqfsJRgm9stLBst8zJ59qYuN1F/OktKsCKV0Fc/1jvNtYNeQ2Sts=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184732-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184732: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b
X-Osstest-Versions-That:
    xen=9ee7dc877b8754ce2fc82500feea52c04d4e6409
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 23 Feb 2024 15:23:07 +0000

flight 184732 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184732/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184729
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184729
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184729
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184729
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184729
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184729
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184729
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184729
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184729
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184729
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184729
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184729
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 xen                  812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b
baseline version:
 xen                  9ee7dc877b8754ce2fc82500feea52c04d4e6409

Last test of basis   184729  2024-02-22 14:07:38 Z    1 days
Testing same since   184732  2024-02-23 04:57:51 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Luca Fancellu <luca.fancellu@arm.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   9ee7dc877b..812bdc99f4  812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b -> master


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 15:41:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 15:41:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684900.1065127 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdXfu-0000s2-VF; Fri, 23 Feb 2024 15:41:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684900.1065127; Fri, 23 Feb 2024 15:41:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdXfu-0000rv-Ql; Fri, 23 Feb 2024 15:41:22 +0000
Received: by outflank-mailman (input) for mailman id 684900;
 Fri, 23 Feb 2024 15:41:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nHXq=KA=amd.com=ayan.kumar.halder@srs-se1.protection.inumbo.net>)
 id 1rdXft-0000rp-Tc
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 15:41:22 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fc3705e8-d261-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 16:41:19 +0100 (CET)
Received: from PH8PR12MB7326.namprd12.prod.outlook.com (2603:10b6:510:216::7)
 by DM4PR12MB5246.namprd12.prod.outlook.com (2603:10b6:5:399::17) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.24; Fri, 23 Feb
 2024 15:41:15 +0000
Received: from PH8PR12MB7326.namprd12.prod.outlook.com
 ([fe80::22eb:b0b:62c0:f3e5]) by PH8PR12MB7326.namprd12.prod.outlook.com
 ([fe80::22eb:b0b:62c0:f3e5%6]) with mapi id 15.20.7316.023; Fri, 23 Feb 2024
 15:41:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc3705e8-d261-11ee-8a57-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gXlTMmglhcrhqV0RbcCZNkOIfAltjwSbkY6Rdzc5o5TKEzUqtnE71Dityq1zpqBEDjLCx6xT0p+tpJVGuB6oFbZbXBIaiuIRI9d9KZvoj4ZOeNmJSrBkEoAmkB04iBagfoDkfhbrB5QsKmCQMarPjfxMC7BbwopZnnb1G5LCwxoHMJ8T3rBE8XiT6oXdwWgEkSQJDJJBa0lteWFBm/kyR5jqitBhisfMjTzj+cjIuuqHK+KwTr2d3nYANCKgtV/ebl9j63yn1LSY0DONKICYSDpeGYNTFFSnIa6zCLmi6fg5Xnb9GqlScwDpmQlfixtsGgxTs4Km9CkuYzUrgcHJig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=5q0K7Dq6T/xYtj3KkhpP9soL7g+L03o69A9EYnaFMMs=;
 b=FrVH4Efec5+Iq+aaK27nFx2917rkHdPCzOD3Wku3M0Ew4+2ayy2kOP0Hkw/ub+wSsc5w6hmA2HDomPqeF7N9mPtER4g5JJwp0Kr4zRzF2t0xGxOiccCFssRvg/07U916uTPdD6YQwrCOLxVUijjaVERvXF0nSfbNjmtv1trSovCRz0kPMaeICtEw4MTEWpQDrpcrExJiSI52fAXQtozk1evSsKYg7u6+gpxTivkIrcMKtPCxcXsIh3sCiifvZvdKes7wHX7TfiEAUpMVEPzXOR3JOJXO53uYCKvNXMNYcP7qd5zv2p1oiCG+HjkQCpz3/DzvosKD3kpVsbPBS2+mNA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5q0K7Dq6T/xYtj3KkhpP9soL7g+L03o69A9EYnaFMMs=;
 b=S1yq+Lo0cMH/OQIiOpRa45mliydj51WbK8ImGzEaz5tBADUKqnV1Yj/Uic9IuxA0SoY3TbeAFeBwZX0J7oL4KEmYiv5LbQO9ax7712gmkFWmoMirfaAeOocNH5XRpDqaxTRQ78KRRIVTV3Ux/M92lmz9lZOFYkPuMVRLQNG8csA=
Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
Message-ID: <c5c0a46b-8793-481d-a678-da0929a0d6e9@amd.com>
Date: Fri, 23 Feb 2024 15:41:10 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v5 2/3] xen/arm: arm64: Add emulation of Debug Data Transfer
 Registers
To: Ayan Kumar Halder <ayan.kumar.halder@amd.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com, julien@xen.org,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com, michal.orzel@amd.com
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
 <20240220121743.3680715-3-ayan.kumar.halder@amd.com>
Content-Language: en-GB
From: Ayan Kumar Halder <ayankuma@amd.com>
In-Reply-To: <20240220121743.3680715-3-ayan.kumar.halder@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: LO4P123CA0526.GBRP123.PROD.OUTLOOK.COM
 (2603:10a6:600:2c5::12) To PH8PR12MB7326.namprd12.prod.outlook.com
 (2603:10b6:510:216::7)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: PH8PR12MB7326:EE_|DM4PR12MB5246:EE_
X-MS-Office365-Filtering-Correlation-Id: 5e3d8eca-cae8-4ef1-fa38-08dc3485dea9
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	IU9z36TpQD5+e6oYsXwyjcGSVj2wghj4DIDZhPPlRoKsFdxt1OWud6ZuljL8vpNk0YOpOJryADJy3rSh5aTtE2MIWR5616fRmkSmELTHR0JNtsZsQQKMeRKBzPIrXGhK1GTkgngUJBxojNwAtckXFWsjI28KbL2Iujl4O6UVIsKqQzc76Ne04kyGBAunPOuptpJLbXHs9nBQDIncWaJWfNaENFiGBr5x2lFyeBU3CmNoY8Av5PqkEF1kNWTJpZ9shk77CingQqEZjjITR1Lh3/XcyCT1bzqWmM+Y4dIxL+wV08rkWkgSYKEENZReilvmIQlRIDoP/E5c6B2dz56jG2TXsp6KPyIGcOwbk38euW/WgJSeKTbphF6SlpPkfNzgHR8pvqXd0ZEnJaH6adj34clKsuzCCgs74IBAYjQDSGlwvtz29ED6AKB6sRacvVBsmFWquyr29WRZNLtbWKxxHKbdv15LScWWYsnksKPoLfl7YVBr+lZU2KZmK1aeB98c9lYM85To9pFRK9lztZJ2S3XkZtklr54b+geyzT3UPCc=
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH8PR12MB7326.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?LzgxT2RHSHdVbVNqZGZhYUxuSEVQZ0hVaENORTQvZG1xRXJhbEkyVk9jcThv?=
 =?utf-8?B?aHlITldwY1dJZkVmYVhhS0cyNzlBeGVyRW5tMVpTMXF3NWxNQ1Npa2JYM2lS?=
 =?utf-8?B?YThPUG41dVlSTXRLb2tTZFZFYXpNQ2RGVkl4WHM2NFRnM0oxMUlSWFdxSGRC?=
 =?utf-8?B?NlVSMldWRWJTUlJIVlBHZFh4V2tWWXJkMUN5ckpLSFJpWjhsNVhWOXJuUURJ?=
 =?utf-8?B?MDV4ZVNnclVrSzllSUxkd3o4V0pKNGh2Qm5FZUs2MHlGYVUxUDVySU94Sitx?=
 =?utf-8?B?M1ZtK1NubW42bzYwQ0F2TTZRbGdrc2tocURnamFqQ2I2UzlJbCtJSTVuQXh5?=
 =?utf-8?B?ZVI2UlJYZzlPN0lOMzJuTDZXTjJudUNRK1ZZYldRaEhtdEVqYzdjOFJIYXYy?=
 =?utf-8?B?VDRmZmowTHJYOXFQemQ4bWloZVhDUDViZHZTMFFaNm92VzYyTUhWZ0NwdmQx?=
 =?utf-8?B?UDUvYjhvN1AyNzYvYVgveUNGUDdBdm5SYnVnN0dRMVQ4MlJHRVdqOE5XMUcv?=
 =?utf-8?B?cU1BOW5aMnJFcnp0TkNQSVRDNEJZZkNGYzVjdXVRL3VoZkdlVi80NDBXTnE0?=
 =?utf-8?B?NEJOQzlROWhWMk5FVHVONnpxU2VvR0dDVUpSRGEyUjJLN3N5MHVPejNzejZo?=
 =?utf-8?B?dGhzdEhQK2JFeUdqdTNNWnQ1eEpMVEZudDJLQS9QZmttZzYzOEt4b056bzZm?=
 =?utf-8?B?amIrU3lUQk1GaHBnbUhGVnZnR3YwRWhDN3YycjkvbGNLZUU3VFczWnhVa29z?=
 =?utf-8?B?WGcveVVGQ05FYVBQL2FUV2gwbzJiK0p1YWFmZnlFbjZsZHFVQ0RLQUJYbS9z?=
 =?utf-8?B?d2RzcVlpVWJEY3NCSDZwOXlJVDlvV011UmdBRTVmV3IvcUZLbS9tV01VVy90?=
 =?utf-8?B?VGJDMHEyRHlIVUZLUkJDSFc5UmtYSFFqODdneFhrNjlDemY4VkxEbTBja2xO?=
 =?utf-8?B?Yi82QXQyQ0FoUU1SNG0rWjNxR2xFUHRUOGhPVVFid1lROVhvR0xWOUxFUk5T?=
 =?utf-8?B?dmp6czZiMXBuYmJheVFrMWs1bGg1czh2Zm5YZTh2aGZGSzZoVmVCM2JrbEdq?=
 =?utf-8?B?RUtiSXV4cGY4T2M1QTFYRXhpdC9pM0dDVjZjTElYcE4vSXFra1RmOWxYZkNy?=
 =?utf-8?B?NW9Da1k1bjFrWkcxckRQWC84WmdPMXdYU1pQb2VQZzlwd2tOMkl6MmphblEw?=
 =?utf-8?B?NllSWHh0WU1OUkNuRW90YWZkYXRldk8yVGdkMXFUZlR1S0VOclRhUUlYUHBo?=
 =?utf-8?B?djVIYlZEcFR3VnJiZ055SzhrQXdicWZpMHkzWUlPQ2VYcUNyUDVSeGJvYkE3?=
 =?utf-8?B?QU1vNU9wWVdsUnE0YTdQMEZXZXJnREZBS2hUTDZXREdBNXM2UE5NTFlvMXgx?=
 =?utf-8?B?UUtodmN0bXFObk5CRW50NDBMN3QybTIyNE5neVRQME00bUIzWldTb2VpMGtT?=
 =?utf-8?B?MVdOZko5MFFzVEh6SW1tKzZuOURtVXZBS2VoOVBUS1h0ZWI2YzJNSHFhd3Vo?=
 =?utf-8?B?ZGxvbEpmZDMxbWZSNEdwc2hJVTk5RWZMclZxbnVSN0dFQVhXbVpvSFdYdmZ1?=
 =?utf-8?B?cjVYbDR6ZG1sZzhUMGhac3hFZHZLZ3VSUFFSUTVLUkdsZ2ZFV0hGQXF5L0hD?=
 =?utf-8?B?bkNBbzgySDRIK3lGRTdnUnFpMW9UTzNQRGpZVWp1UDNMVlA1L1ZjRHYwU0s0?=
 =?utf-8?B?UkdrT3FHZExBTW5jUUxqbEVRejE3clpzWFFlYkVvZlpISGNGd2FFT2hBS2Rm?=
 =?utf-8?B?eE1UL1NFdkRhK09UQm5BazV0QVlyZWpkSjN0Mms0VHRaSVo1bHZkKzFjNXc3?=
 =?utf-8?B?ZytvOG9xYXhhekZ1NjZ4eURHYkNWUkdXUjlWclF1aUo3Q3hsYk14dmxRNXJR?=
 =?utf-8?B?Vjd1TzBadzBURldxckNQOGxXeHJEbU84dTJoTmNCV0p5T0IwUkp0UkdIYU53?=
 =?utf-8?B?MDlKbGMzSHlSem9WU2xIRHBIQ0ROT2VxMjc2aUx0Q05FaDAvZkFoZXk2MXUr?=
 =?utf-8?B?VnEyZUM0UlFNK2h6NFlQeVlsSndJRmF3OE1jYWFYWUN0VlZRWEJTUkFoWnFX?=
 =?utf-8?B?ZHF5M2VhaFE3ckM3a2JxdlM0YVdMZ0N5VEZqekxZUEloVmxWVWt3YVRXQ2tt?=
 =?utf-8?Q?CqVqPBB0hpWQG75ECOv81ErBq?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5e3d8eca-cae8-4ef1-fa38-08dc3485dea9
X-MS-Exchange-CrossTenant-AuthSource: PH8PR12MB7326.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2024 15:41:15.2035
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Cs9wjPnrvhbjO8A9JGcahhesFF4QO57qPQIAHVLrJA97ew+ZPT5ZsgmDfeg8u6tu9ixfooqr9qJ6OYVFfkrJwg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5246

Hi,

On 20/02/2024 12:17, Ayan Kumar Halder wrote:
> From: Michal Orzel <michal.orzel@amd.com>
>
> Currently, if user enables HVC_DCC config option in Linux, it invokes access
> to debug data transfer registers (i.e. DBGDTRTX_EL0 on arm64, DBGDTRTXINT on
> arm32). As these registers are not emulated, Xen injects an undefined
> exception to the guest and Linux crashes.
>
> To prevent this crash, introduce a partial emulation of DBGDTR[TR]X_EL0
> (these registers share the same encoding) as RAZ/WI and MDCCSR_EL0 as TXfull.
>
> Refer ARM DDI 0487J.a ID042523, D19.3.8, DBGDTRTX_EL0
> "If TXfull is set to 1, set DTRRX and DTRTX to UNKNOWN".
>
> Thus, any OS is expected to read MDCCSR_EL0 and check for TXfull before
> using DBGDTRTX_EL0. Linux does it via hvc_dcc_init() ---> hvc_dcc_check(),
> and returns -ENODEV in case TXfull bit is still set after writing a test
> character. This way we prevent the guest from making use of HVC DCC as a
> console.
>
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> Reviewed-by: Michal Orzel <michal.orzel@amd.com>
> ---
> Changes from
>
> v1 :- 1. DBGDTR_EL0 does not emulate RXfull. This is to avoid giving the OS any
> indication that the RX buffer is full and is waiting to be read.
>
> 2. In Arm32, DBGOSLSR is emulated. Also DBGDTRTXINT is emulated at EL0 only.
>
> 3. Fixed the commit message and inline code comments.
>
> v2 :- 1. Split the patch into two (separate patches for arm64 and arm32).
> 2. Removed the "fail" label.
> 3. Fixed the commit message.
>
> v3 :- 1. "HSR_SYSREG_MDCCSR_EL0" emulation differs based on whether
> partial_emulation_enabled is true or not.
>
> 2. If partial_emulation_enabled is false, then access to HSR_SYSREG_DBGDTR_EL0,
> HSR_SYSREG_DBGDTRTX_EL0 would lead to undefined exception.
>
> v4 :- 1. Invoked "goto fail" from "default:" to ensure compliance with
> MISRA 15.3.
>
>   xen/arch/arm/arm64/vsysreg.c         | 68 +++++++++++++++++++---------
>   xen/arch/arm/include/asm/arm64/hsr.h |  3 ++
>   2 files changed, 50 insertions(+), 21 deletions(-)
>
> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
> index b5d54c569b..80918bc799 100644
> --- a/xen/arch/arm/arm64/vsysreg.c
> +++ b/xen/arch/arm/arm64/vsysreg.c
> @@ -82,6 +82,7 @@ TVM_REG(CONTEXTIDR_EL1)
>   void do_sysreg(struct cpu_user_regs *regs,
>                  const union hsr hsr)
>   {
> +    const struct hsr_sysreg sysreg = hsr.sysreg;
>       int regidx = hsr.sysreg.reg;
>       struct vcpu *v = current;
>   
> @@ -159,9 +160,6 @@ void do_sysreg(struct cpu_user_regs *regs,
>        *
>        * Unhandled:
>        *    MDCCINT_EL1
> -     *    DBGDTR_EL0
> -     *    DBGDTRRX_EL0
> -     *    DBGDTRTX_EL0
>        *    OSDTRRX_EL1
>        *    OSDTRTX_EL1
>        *    OSECCR_EL1
> @@ -171,12 +169,42 @@ void do_sysreg(struct cpu_user_regs *regs,
>        */
>       case HSR_SYSREG_MDSCR_EL1:
>           return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
> +
> +    /*
> +     * Xen doesn't expose a real (or emulated) Debug Communications Channel
> +     * (DCC) to a domain. Yet the Arm ARM implies this is not an optional
> +     * feature. So some domains may start to probe it. For instance, the
> +     * HVC_DCC driver in Linux (since f377775dc083 and at least up to v6.7),
> +     * will try to write some characters and check if the transmit buffer
> +     * has emptied.
> +     */
>       case HSR_SYSREG_MDCCSR_EL0:
>           /*
> +         * By setting TX status bit (only if partial emulation is enabled) to
> +         * indicate the transmit buffer is full, we would hint the OS that the
> +         * DCC is probably not working.
> +         *
> +         * Bit 29: TX full
> +         *
>            * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We emulate that
>            * register as RAZ/WI above. So RO at both EL0 and EL1.
>            */
> -        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 0);
> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 0,
> +                                  partial_emulation ? (1U << 29) : 0);
> +
> +    case HSR_SYSREG_DBGDTR_EL0:
> +    /* DBGDTR[TR]X_EL0 share the same encoding */
> +    case HSR_SYSREG_DBGDTRTX_EL0:
> +        /*
> +         * Emulate as RAZ/WI (only if partial emulation is enabled) to prevent
> +         * injecting undefined exception.
> +         * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We emulate that
> +         * register as RAZ/WI.
> +         */
> +        if ( !partial_emulation )
> +            goto fail;
> +        return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 0);
> +
>       HSR_SYSREG_DBG_CASES(DBGBVR):
>       HSR_SYSREG_DBG_CASES(DBGBCR):
>       HSR_SYSREG_DBG_CASES(DBGWVR):
> @@ -394,26 +422,24 @@ void do_sysreg(struct cpu_user_regs *regs,
>        * And all other unknown registers.
>        */
>       default:
> -        {
> -            const struct hsr_sysreg sysreg = hsr.sysreg;
> -
> -            gdprintk(XENLOG_ERR,
> -                     "%s %d, %d, c%d, c%d, %d %s x%d @ 0x%"PRIregister"\n",
> -                     sysreg.read ? "mrs" : "msr",
> -                     sysreg.op0, sysreg.op1,
> -                     sysreg.crn, sysreg.crm,
> -                     sysreg.op2,
> -                     sysreg.read ? "=>" : "<=",
> -                     sysreg.reg, regs->pc);
> -            gdprintk(XENLOG_ERR,
> -                     "unhandled 64-bit sysreg access %#"PRIregister"\n",
> -                     hsr.bits & HSR_SYSREG_REGS_MASK);
> -            inject_undef_exception(regs, hsr);
> -            return;
> -        }
> +        goto fail;
>       }
>   
>       regs->pc += 4;
> +    return;
> +
> + fail:
> +
> +    gdprintk(XENLOG_ERR,
> +             "%s %d, %d, c%d, c%d, %d %s x%d @ 0x%"PRIregister"\n",
> +             sysreg.read ? "mrs" : "msr", sysreg.op0, sysreg.op1, sysreg.crn,
> +             sysreg.crm, sysreg.op2, sysreg.read ? "=>" : "<=", sysreg.reg,
> +             regs->pc);
> +    gdprintk(XENLOG_ERR,
> +             "unhandled 64-bit sysreg access %#"PRIregister"\n",
> +             hsr.bits & HSR_SYSREG_REGS_MASK);
> +    inject_undef_exception(regs, hsr);
> +    return;

The last 'return' needs to be removed (spotted by Michal)

- Ayan



From xen-devel-bounces@lists.xenproject.org Fri Feb 23 16:00:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 16:00:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684907.1065135 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdXy1-0003yh-Ls; Fri, 23 Feb 2024 16:00:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684907.1065135; Fri, 23 Feb 2024 16:00:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdXy1-0003xt-Ii; Fri, 23 Feb 2024 16:00:05 +0000
Received: by outflank-mailman (input) for mailman id 684907;
 Fri, 23 Feb 2024 16:00:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SHka=KA=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rdXxz-0003aU-TD
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 16:00:03 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 99008269-d264-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 17:00:01 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a3566c0309fso75053266b.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 08:00:00 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ot18-20020a170906ccd200b00a3ecdd0ba23sm4413236ejb.52.2024.02.23.07.59.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 07:59:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99008269-d264-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708704000; x=1709308800; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=ssD+CzyXnBVEZMfxZkqRKLy8uXBrLLZsUTuV+Jfdq+Q=;
        b=ZZ0drDS0FcI7FYtXwRm7dPqBLA5vx7xO63L4w1MBf5EECxBQGieXy3D3y9e3GizlfQ
         yADfwzuOS09Da2PnZJNb1PnkIyTXEDM3mRZVnvjF9nIJeiXGXLd6fUF5RfFVokHnkFwA
         KqR73xwmCo8oX0nktdlBdjIbotB1ts/syEPS8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708704000; x=1709308800;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ssD+CzyXnBVEZMfxZkqRKLy8uXBrLLZsUTuV+Jfdq+Q=;
        b=NsgQSK/lIyuej7XZqD7OjEJ843S6+BA+X9LQlrebTFEjUgXAmuk0G4jRDex0AILrW3
         CAjHdVpsRvLxROPfD7vvx+rSw4cQ/ivkesd0ujjfz4Zb+GwzT9SdG1dkw/8L/NTl1pWQ
         GzN/izWC3HOzqR9fFH+Za3+9ARH9QDE6RcfLI7uIX8e/f4nu+BjrekFEDDmLAivii7+J
         lrfD0AeuugUwR9BwzJfs8k3Bt7Fe6/N0Kq1YhZ/9V7zG8b2OlinpRtbPwKz9bEu4a7Lj
         tULl3PZJtmcKYMTURvUG4PxrOJPGvCZKUX8+8GNXCGnS86FhTHkLL7zFHjpAoEu5UKUe
         HUtA==
X-Gm-Message-State: AOJu0YwiXOyDXcs3WVuP5qZ+y79TY6aP12h5UjNaDDWx985DoI6Zm+UJ
	bN25pW8bo0SSxHvnfgZVaAMORZpSjJIlXCWyqiE5x3fEdo75JekEbV8eweLHZsI=
X-Google-Smtp-Source: AGHT+IGDivtW5FFVIARkidjXJCZPcfL03/nDJKsjOPlYFct8cLuj+JWf7/f5xD/A5xhykCulpHYc6g==
X-Received: by 2002:a17:906:4809:b0:a3f:988f:b9c4 with SMTP id w9-20020a170906480900b00a3f988fb9c4mr168559ejq.7.1708703999672;
        Fri, 23 Feb 2024 07:59:59 -0800 (PST)
Date: Fri, 23 Feb 2024 15:59:58 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Juergen Gross <jgross@suse.com>,
	"Daniel P . Smith" <dpsmith@apertussolutions.com>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
	"Huang, Ray" <Ray.Huang@amd.com>
Subject: Re: [RFC XEN PATCH v5 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Message-ID: <d0adfc76-5142-46b1-acaa-3a4f0331cfb0@perard>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-6-Jiqian.Chen@amd.com>
 <09290ba4-6915-4f0d-8e16-3e14713d02ba@perard>
 <BL1PR12MB5849F3787ED9245E6986872CE7562@BL1PR12MB5849.namprd12.prod.outlook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BL1PR12MB5849F3787ED9245E6986872CE7562@BL1PR12MB5849.namprd12.prod.outlook.com>

On Thu, Feb 22, 2024 at 07:22:45AM +0000, Chen, Jiqian wrote:
> On 2024/2/21 23:03, Anthony PERARD wrote:
> > On Fri, Jan 12, 2024 at 02:13:17PM +0800, Jiqian Chen wrote:
> >> diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
> >> index a1c6e82631e9..4136a860a048 100644
> >> --- a/tools/libs/light/libxl_pci.c
> >> +++ b/tools/libs/light/libxl_pci.c
> >> @@ -1421,6 +1421,8 @@ static void pci_add_dm_done(libxl__egc *egc,
> >>      uint32_t flag = XEN_DOMCTL_DEV_RDM_RELAXED;
> >>      uint32_t domainid = domid;
> >>      bool isstubdom = libxl_is_stubdom(ctx, domid, &domainid);
> >> +    int gsi;
> >> +    bool has_gsi = true;
> > 
> > Why is this function has "gsi" variable, and "gsi = irq" but the next
> > function pci_remove_detached() does only have "irq" variable?
> Because in pci_add_dm_done, it calls " r = xc_physdev_map_pirq(ctx->xch, domid, irq, &irq); ", it passes the pointer of irq in, and then irq will be changed, so I need to use gsi to save the origin value.

I'm sorry but unconditional "gsi = irq" looks very wrong, that line
needs to be removed or changed, otherwise we have two functions doing the
same thing just after that (xc_domain_irq_permission and
xc_domain_gsi_permission). Instead, my guess is that the
arguments of xc_physdev_map_pirq() needs to be changes. What does
contain `irq` after the map_pirq() call? A "pirq" of some kind? Maybe
xc_physdev_map_pirq(ctx->xch, domid, irq, &pirq) would make things more
clear about what's going on.


And BTW, maybe renaming the variable "has_gsi" to "is_gsi" might make
things a bit clearer as well, as in: "the variable 'irq' $is_gsi",
instead of a variable that have a meaning of "the system $has_gsi"
without necessarily meaning that the code is using it.

Maybe using (abusing?) the variable name "irq" might be a bit wrong now?
What does "GSI" stand for anyway? And about "PIRQ"? This is just some
question to figure out if there's potentially a better name for the
variables names.

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 17:00:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 17:00:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684916.1065146 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdYub-0003M6-3b; Fri, 23 Feb 2024 17:00:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684916.1065146; Fri, 23 Feb 2024 17:00:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdYua-0003Lz-W5; Fri, 23 Feb 2024 17:00:36 +0000
Received: by outflank-mailman (input) for mailman id 684916;
 Fri, 23 Feb 2024 17:00:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=X/z8=KA=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rdYuY-0003Ls-VW
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 17:00:35 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0e3322de-d26d-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 18:00:34 +0100 (CET)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2d269b2ff48so6745101fa.3
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 09:00:32 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 q14-20020a2e914e000000b002d0a8143f31sm2667879ljg.50.2024.02.23.09.00.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 23 Feb 2024 09:00:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0e3322de-d26d-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708707632; x=1709312432; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=xfcegJCkN+4l/qAbHquimEKw/cizmOVIAMbbX7WIHZw=;
        b=l+UAa1z0yHGaFNvbn7EFTdC+D5U7prJAlfdXRv2GYjRHDWo+PJfW33I/BmkpcWZe0Z
         DuZ0sOk83nXKErSL2IQUCtUQmOa4KJ1CCtcJe1DrpYuUTiHqvOjwOKRksdqnDXIutaLW
         E1u66G6Cc7dfOmG5zaiR8BI7Ao/iR+g7GxFp4CYHC5YfisSaMfIyOaujBoFf/6w43OIc
         h1nmL9pFpl6CgOeY4QvWOkFSwrFq8afa7fPeK+7tCV0ES09WMliJukjHeHIdIF/LWWqU
         v4Kvob2W90+NFl4XZmQZyvpxlWKtR5FTivVMqSK64Qis/GMroJ6WHnowRmkeIo3q1KO+
         5uww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708707632; x=1709312432;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xfcegJCkN+4l/qAbHquimEKw/cizmOVIAMbbX7WIHZw=;
        b=tuYPi4jnfqhdT3IUeNa3FDEZwlnhEJz8/WMusw6qfMxzn1bMdlu+57tbDyMMm+QPfV
         RqdXxLiq1k5E3VzL6bG4S4NPAtG0gkqTKUGMys2dw2HxhAv+7FwFNLACg7zG4bj6650h
         sKef5a7i2srRSGJlm9JiRerkWdMKKYVmcos6EEdKa2CD93QQ/mBg5vZ4zmXmF18/qbW/
         QNxopdafy934uo5VsVp0GBboy1P60OOSpCAm0dvlR72B81jrrpaetlOHRvtfKb/S/zGX
         KwywdUXpunnqC7G0Zl4K5eUalcM+6Lx8uo8QnUFTvo4D0fOvQ04SdxilxE0OBcF+ZFGu
         JxzA==
X-Forwarded-Encrypted: i=1; AJvYcCU8pIvmsOB0pKYekDFreN7VJ0en9Uqm/rE5l3F2DgDfVxfxEjumO5GEGyC/JJfRlegBh/wWlXYTMLhFRdmT4XiSN7GwKzfYLpN7LixMHZU=
X-Gm-Message-State: AOJu0Yzk0S7s830Oqgz09HUbDE3UlJFVL4+/rwPkyESMuOsIUuh+4uXa
	A4ZzFm0FNJvtLbdRSIjr+i4bfBxNaiMMKs/TWnWTphvIWk38boLK
X-Google-Smtp-Source: AGHT+IEgTk+NxAELDXdrspIgiDNNJxy45Rj71Fb2P360WbueqQJxKy8e2bYfXC4ZUHpofIPjCB0crQ==
X-Received: by 2002:a2e:6a0d:0:b0:2d2:7580:e220 with SMTP id f13-20020a2e6a0d000000b002d27580e220mr251065ljc.15.1708707631792;
        Fri, 23 Feb 2024 09:00:31 -0800 (PST)
Message-ID: <6cd4d415ccf2163d9fdf690f29d898d61b077472.camel@gmail.com>
Subject: Re: [PATCH v4 25/30] xen/riscv: add minimal stuff to processor.h to
 build full Xen
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, Alistair Francis
 <alistair.francis@wdc.com>, Bob Eshleman <bobbyeshleman@gmail.com>, Connor
 Davis <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
Date: Fri, 23 Feb 2024 18:00:30 +0100
In-Reply-To: <5de8f721-461c-4a0e-a11d-63aa7c93a742@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <9efbc232f64b6192cf83f865b8987846fe082720.1707146506.git.oleksii.kurochko@gmail.com>
	 <6be5102a-624c-463a-9821-c618d110ce7a@suse.com>
	 <095b8031eaaa5324cdae9fee75f9521a795feb46.camel@gmail.com>
	 <2f1e4d2d-5b33-47ff-912b-c4693744d0e9@suse.com>
	 <0bafef389b30251bc9898bb61604aa3efaabe48c.camel@gmail.com>
	 <5de8f721-461c-4a0e-a11d-63aa7c93a742@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-19 at 09:06 +0100, Jan Beulich wrote:
> On 16.02.2024 12:16, Oleksii wrote:
> > On Thu, 2024-02-15 at 17:43 +0100, Jan Beulich wrote:
> > > On 15.02.2024 17:38, Oleksii wrote:
> > > > On Tue, 2024-02-13 at 14:33 +0100, Jan Beulich wrote:
> > > > > On 05.02.2024 16:32, Oleksii Kurochko wrote:
> > > > > > +	depends on LLD_VERSION >=3D 150000 || LD_VERSION >=3D
> > > > > > 23600
> > > > >=20
> > > > > What's the linker dependency here? Depending on the answer I
> > > > > might
> > > > > further
> > > > > ask why "TOOLCHAIN" when elsewhere we use CC_HAS_ or HAS_CC_
> > > > > or
> > > > > HAS_AS_.
> > > > I missed to introduce {L}LLD_VERSION config. It should output
> > > > from
> > > > the
> > > > command:
> > > > =C2=A0 riscv64-linux-gnu-ld --version
> > >=20
> > > Doesn't answer my question though where the linker version
> > > matters
> > > here.
> > Then I misinterpreted your initial question.
> > Could you please provide further clarification or rephrase it for
> > better understanding?
> >=20
> > Probably, your question was about why linker dependency is needed
> > here,
> > then
> > it is not sufficient to check if a toolchain supports a particular=C2=
=A0
> > extension without checking if the linker supports that extension=C2=A0=
=C2=A0=20
> > too.
> > For example, Clang 15 supports Zihintpause but GNU bintutils
> > 2.35.2 does not, leading build errors like so:
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0 riscv64-linux-gnu-ld: -march=3Drv64i_zihintpause2p0: Inval=
id or
> > =C2=A0=C2=A0 unknown z ISA extension: 'zihintpause'
>=20
> Hmm, that's certainly "interesting" behavior of the RISC-V linker.
> Yet
> isn't the linker capability expected to be tied to that of gas? I
> would
> find it far more natural if a gas dependency existed here. If such a
> connection cannot be taken for granted, I'm pretty sure you'd need to
> probe both then anyway.

Wouldn't it be enough in this case instead of introducing of new
configs and etc, just to do the following:
   +ifeq ($(CONFIG_RISCV_64),y)
   +has_zihintpause =3D $(call as-insn,$(CC) -mabi=3Dlp64 -
   march=3Drv64i_zihintpause, "pause",_zihintpause,)
   +else
   +has_zihintpause =3D $(call as-insn,$(CC) -mabi=3Dilp32 -
   march=3Drv32i_zihintpause, "pause",_zihintpause,)
   +endif
   +
    riscv-march-$(CONFIG_RISCV_ISA_RV64G) :=3D rv64g
    riscv-march-$(CONFIG_RISCV_ISA_C)       :=3D $(riscv-march-y)c
   -riscv-march-$(CONFIG_TOOLCHAIN_HAS_ZIHINTPAUSE) :=3D $(riscv-march-
   y)_zihintpause
   =20
    # Note that -mcmodel=3Dmedany is used so that Xen can be mapped
    # into the upper half _or_ the lower half of the address space.
    # -mcmodel=3Dmedlow would force Xen into the lower half.
   =20
   -CFLAGS +=3D -march=3D$(riscv-march-y) -mstrict-align -mcmodel=3Dmedany
   +CFLAGS +=3D -march=3D$(riscv-march-y)$(has_zihintpause) -mstrict-align
   -
   mcmodel=3Dmedany

Probably, it would be better:
   ...
   +CFLAGS +=3D -march=3D$(riscv-march-y)$(call or,$(has_zihintpause)) -
   mstrict-align -
   mcmodel=3Dmedany


~ Oleksii


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 21:42:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 21:42:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684939.1065208 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rddJH-00084U-1I; Fri, 23 Feb 2024 21:42:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684939.1065208; Fri, 23 Feb 2024 21:42:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rddJG-00084N-Ue; Fri, 23 Feb 2024 21:42:22 +0000
Received: by outflank-mailman (input) for mailman id 684939;
 Fri, 23 Feb 2024 21:42:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nQMd=KA=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1rddJF-00084H-JD
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 21:42:21 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 696cbbfa-d294-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 22:42:17 +0100 (CET)
Received: from BN9PR03CA0278.namprd03.prod.outlook.com (2603:10b6:408:f5::13)
 by BY5PR12MB4276.namprd12.prod.outlook.com (2603:10b6:a03:20f::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.24; Fri, 23 Feb
 2024 21:42:13 +0000
Received: from BN3PEPF0000B071.namprd04.prod.outlook.com
 (2603:10b6:408:f5:cafe::1b) by BN9PR03CA0278.outlook.office365.com
 (2603:10b6:408:f5::13) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.43 via Frontend
 Transport; Fri, 23 Feb 2024 21:42:12 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN3PEPF0000B071.mail.protection.outlook.com (10.167.243.116) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Fri, 23 Feb 2024 21:42:12 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 23 Feb
 2024 15:42:12 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 23 Feb
 2024 15:42:11 -0600
Received: from ubuntu-20.04.2-arm64.shared (10.180.168.240) by
 SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.35 via Frontend Transport; Fri, 23 Feb 2024 15:42:11 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 696cbbfa-d294-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eS3C1nnct3mCy8hPMmX3wP0s1xMLiiSyYqjCBLByd6gqLvQvDg+V0eJfiuCo2SLxBLclhnz611obQduXayW+EmE4yJCmX0YwJN7BsCmGlxyiKseC7WvF2OjKwsC/tW57RPpVsAWV/uu3y+vsiI7XEknjYujLpMUu34KThKmnxaBeak/MpwexXTjlLm3VkfpnPE8ZT+qYhI57i4suHI3uu0tTPK270EUscoNVe9Sh+g18UQbENG/eZQS4KJ+nUcOzbSmoqv+D8f15+lAz+Gas4vnuhJp//J73+mxn3ZlGE00xLphARHdNtVL4FTwd/8brI/WDhNf7qja3Al1OxNGKeg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=brzpYEEybsreMV1sY2Ki8Ly4xbWrIx5anpAufpPOS70=;
 b=hz5hSs4Fc+w4dkQ1bQG58729PRwycpd+d5IgzPSHS2c/rwXYnnwZoMdnDDsf79HiFPhW8E/UDkIQpTPtjHzGSY7dtGWgLFvT0fxLxI88kPrWh5VQaIzkIMtIlp9HBJO0kCtRufrfdzv5250aNCgP86uJ/pzZdSJkwrhlW84o6WoDrMgegomPVHt0CRr+gVSpgOB7hZEyxZJFo+5JKyANNZA1qfwNHt7S0RpiaDjsqr69Ua13LgC0CyH78r8cudMcHIQEL6xvo+JS4eaTmgmh/weZhH/o9XuMPRfwcfLE4rE/HD/BdTnsLcIewxUSJGwNflSD9Linq83STl9DoxLEfA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=nxp.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=brzpYEEybsreMV1sY2Ki8Ly4xbWrIx5anpAufpPOS70=;
 b=G7sYzAF1EgEbPw2om+LQznZPBkgksvx32rhOeLP4efnHKtPTPMGcfxLvvEx23hUADRWbRNJOGr00GX4GNjUrKQFoNTknwmPw3SIu97NBYAgwMKnCvgc83EXm7bggjRFE+pJZrdkY3bt3ql3e+AC3HyFqBuhPEdQgV6iOaQWYHMs=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Date: Fri, 23 Feb 2024 13:42:10 -0800
From: Stefano Stabellini <stefano.stabellini@amd.com>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Peng Fan <peng.fan@nxp.com>
CC: "stefano.stabellini@amd.com" <stefano.stabellini@amd.com>, Julien Grall
	<julien@xen.org>, "vikram.garhwal@amd.com" <vikram.garhwal@amd.com>,
	"oleksandr_tyshchenko@epam.com" <oleksandr_tyshchenko@epam.com>, Leo Yan
	<leo.yan@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>,
	"stewart.hildebrand@amd.com" <stewart.hildebrand@amd.com>,
	"sgarzare@redhat.com" <sgarzare@redhat.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: question about virtio-vsock on xen
In-Reply-To: <DU0PR04MB941734DA793D87B7FF3A491488552@DU0PR04MB9417.eurprd04.prod.outlook.com>
Message-ID: <alpine.DEB.2.22.394.2402231332580.754277@ubuntu-linux-20-04-desktop>
References: <DU0PR04MB941734DA793D87B7FF3A491488552@DU0PR04MB9417.eurprd04.prod.outlook.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Received-SPF: None (SATLEXMB05.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN3PEPF0000B071:EE_|BY5PR12MB4276:EE_
X-MS-Office365-Filtering-Correlation-Id: 157e3131-6f19-4f45-9ace-08dc34b84b7f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kO+V4s9OdN91nn/QK39thiS8VYDf5mwOL4PA6U9vPHwLaYOmY+HNZlD3jSHddezGEyNKtQ+oZAqlOIidwiYr5asAKWG0wgYI6/XzzrNhVVYC3pfPknYMBrsw/Q4LPKnU+gH07wM7g/ozryMn47d9bhgt0V4RgNKkYbjk6eboMza88cvxmv0Y8ngr5eUV8HfZmbu2Pl294O9uuAM4r98aYxDvF/tY9ecezsW4CkVCEZ9aoMWvaoxgTQc8d80qMbWnGakjGF8T10BkPtiGBt7eGcny9ZUNwYu7nLya7RFvWi+Gw8n9wDBgdgO22ZNp7UJY2FFc3XT+dWgiOoujatmovkuAU50e2UGK1sfTkSeAk4w8AazwYqq6uwhPw1UeK6ItfI1qVCgZME74XknXZSodCNnzGfbmkm6IxhBtDyMaYAvE2kG+UnDgL0m1rmviWHIhENm7lBpBP27xd2qwwUonBL34C74f9KQOnzn2J7UoMNv4/aU6pssIphI/sxkc3Zc4b1k6/53SbxyQ8CJkGKIkbpEqDpAuoW9Kc2fDO9LBl5viWMH+LooT6gGw6x/bUZcUQdf+zZnZ5vMZyYMxafomBcFWO2rp/Vr9mgQ9e5SNqGgWgv2Zdo38tpylS5u/8P4AvMlHJLBVHT2qwAqDv8ygKQ==
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(40470700004)(46966006);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2024 21:42:12.4736
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 157e3131-6f19-4f45-9ace-08dc34b84b7f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN3PEPF0000B071.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4276

Hi Peng,

We haven't tried to setup virtio-vsock yet.

In general, I am very supportive of using QEMU for virtio backends. We
use QEMU to provide virtio-net, virtio-block, virtio-console and more.

However, typically virtio-vsock comes into play for VM-to-VM
communication, which is different. Going via QEMU in Dom0 just to have 1
VM communicate with another VM is not an ideal design: it adds latency
and uses resources in Dom0 when actually we could do without it.

A better model for VM-to-VM communication would be to have the VM talk
to each other directly via grant table or pre-shared memory (see the
static shared memory feature) or via Xen hypercalls (see Argo.)

For a good Xen design, I think the virtio-vsock backend would need to be
in Xen itself (the hypervisor).

Of course that is more work and it doesn't help you with the specific
question you had below :-)

For that, I don't have a pointer to help you but maybe others in CC
have.

Cheers,

Stefano


On Fri, 23 Feb 2024, Peng Fan wrote:
> Hi All,
> 
> Has anyone make virtio-vsock on xen work? My dm args as below:
> 
> virtio = [
> 'backend=0,type=virtio,device,transport=pci,bdf=05:00.0,backend_type=qemu,grant_usage=true'
> ]
> device_model_args = [
> '-D', '/home/root/qemu_log.txt',
> '-d', 'trace:*vsock*,trace:*vhost*,trace:*virtio*,trace:*pci_update*,trace:*pci_route*,trace:*handle_ioreq*,trace:*xen*',
> '-device', 'vhost-vsock-pci,iommu_platform=false,id=vhost-vsock-pci0,bus=pcie.0,addr=5.0,guest-cid=3']
> 
> During my test, it always return failure in dom0 kernel in below code:
> 
> vhost_transport_do_send_pkt {
> ...
>                nbytes = copy_to_iter(hdr, sizeof(*hdr), &iov_iter);                                
>                 if (nbytes != sizeof(*hdr)) {                                                       
>                         vq_err(vq, "Faulted on copying pkt hdr %x %x %x %px\n", nbytes, sizeof(*hdr),                        
> __builtin_object_size(hdr, 0), &iov_iter);
>                         kfree_skb(skb);                                                             
>                         break;                                                                      
>                 }  
> }
> 
> I checked copy_to_iter, it is copy data to __user addr, but it never pass, 
> the copy to __user addr always return 0 bytes copied.
> 
> The asm code "sttr x7, [x6]" will trigger data abort, the kernel will run
> into do_page_fault, but lock_mm_and_find_vma report it is VM_FAULT_BADMAP,
> that means the __user addr is not mapped, no vma has this addr.
> 
> I am not sure what may cause this. Appreciate if any comments.
> 
> BTW: I tested blk pci, it works, so the virtio pci should work on my setup.
> 
> Thanks,
> Peng.
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 22:06:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 22:06:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684945.1065217 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rddgL-0002W2-NH; Fri, 23 Feb 2024 22:06:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684945.1065217; Fri, 23 Feb 2024 22:06:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rddgL-0002Vv-KW; Fri, 23 Feb 2024 22:06:13 +0000
Received: by outflank-mailman (input) for mailman id 684945;
 Fri, 23 Feb 2024 22:06:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rddgJ-0002Vl-Kx; Fri, 23 Feb 2024 22:06:11 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rddgJ-0004tP-Gh; Fri, 23 Feb 2024 22:06:11 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rddgJ-00073v-6Y; Fri, 23 Feb 2024 22:06:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rddgJ-0001da-63; Fri, 23 Feb 2024 22:06:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=NvygsFnjEAKi3A7Ke4kNeyLbloeAaUVt9XhSU2Qpa50=; b=lJZIUDYjASTP8SjlAybEG/IXp8
	UKZMGxrcRUMt8wK4wvW0v0WJpUMqqEFQpoXmlT1qm1JjichVywZd6fjU4zlY55aMIRZU1+Y9NwhL8
	S6z25thD9spvCUnZ+MnsFv2RUgdrjJ3Oo+Zg01qkIjQyiWKTJ2M7Cf20MLhysKQEFrKA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184733-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-5.4 test] 184733: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-5.4:test-armhf-armhf-xl-credit1:guest-start/debian.repeat:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit2:guest-start:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-5.4:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-5.4:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-5.4:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-5.4:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=6e1f54a4985b63bc1b55a09e5e75a974c5d6719b
X-Osstest-Versions-That:
    linux=f0602893f43a54097fcf22bd8c2f7b8e75ca643e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 23 Feb 2024 22:06:11 +0000

flight 184733 linux-5.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184733/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit1  18 guest-start/debian.repeat    fail  like 184511
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184523
 test-armhf-armhf-xl-credit2  14 guest-start                  fail  like 184523
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184523
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184523
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184523
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184523
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184523
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184523
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184523
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184523
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184523
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184523
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184523
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 linux                6e1f54a4985b63bc1b55a09e5e75a974c5d6719b
baseline version:
 linux                f0602893f43a54097fcf22bd8c2f7b8e75ca643e

Last test of basis   184523  2024-01-29 23:43:48 Z   24 days
Testing same since   184733  2024-02-23 07:45:41 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Guilherme G. Piccoli" <gpiccoli@igalia.com>
  Adrian Hunter <adrian.hunter@intel.com>
  Ajay Kaher <ajay.kaher@broadcom.com>
  Akinobu Mita <akinobu.mita@gmail.com>
  Al Viro <viro@zeniv.linux.org.uk>
  Alan Maguire <alan.maguire@oracle.com>
  Aleksander Mazur <deweloper@wp.pl>
  Alex Deucher <alexander.deucher@amd.com>
  Alex Henrie <alexhenrie24@gmail.com>
  Alex Lyakas <alex.lyakas@zadara.com>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Hansen <alexander.hansen@9elements.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexandre Belloni <alexandre.belloni@bootlin.com>
  Alexei Starovoitov <ast@kernel.org>
  Alexey Khoroshilov <khoroshilov@ispras.ru>
  Alfred Piccioni <alpic@google.com>
  Amir Goldstein <amir73il@gmail.com>
  Amit Pundir <amit.pundir@linaro.org>
  Andi Shyti <andi.shyti@kernel.org>
  Andrew Morton <akpm@linux-foundation.org>
  Andrii Nakryiko <andrii@kernel.org>
  Anna Schumaker <Anna.Schumaker@Netapp.com>
  Anton Ivanov <anton.ivanov@cambridgegreys.com>
  Arnaldo Carvalho de Melo <acme@redhat.com>
  Arnd Bergmann <arnd@arndb.de>
  Avri Altman <avri.altman@wdc.com>
  Baokun Li <libaokun1@huawei.com>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  Benjamin Berg <bberg@redhat.com>
  Benjamin Berg <benjamin.berg@intel.com>
  Benjamin Berg <benjamin@sipsolutions.net>
  Bjorn Andersson <andersson@kernel.org>
  Bjorn Helgaas <bhelgaas@google.com>
  bo liu <bo.liu@senarytech.com>
  Boris Burkov <boris@bur.io>
  Borislav Petkov (AMD) <bp@alien8.de>
  Borislav Petkov <bp@alien8.de>
  Brad Spengler <spender@grsecurity.net>
  Breno Leitao <leitao@debian.org>
  Carlos Llamas <cmllamas@google.com>
  Celeste Liu <CoelacanthusHex@gmail.com>
  Chao Yu <chao@kernel.org>
  Chris Riches <chris.riches@nutanix.com>
  Christian A. Ehrhardt <lk@c--e.de>
  Christian Brauner (Microsoft) <brauner@kernel.org>
  Christoph Hellwig <hch@lst.de>
  Christophe JAILLET <christophe.jaillet@wanadoo.fr>
  Chung-Chiang Cheng <cccheng@synology.com>
  Chunyan Zhang <zhang.lyra@gmail.com>
  Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Basilio <daniel.basilio@corigine.com>
  Daniel de Villiers <daniel.devilliers@corigine.com>
  Daniel Lezcano <daniel.lezcano@linaro.org>
  Daniel Stodden <dns@arista.com>
  Daniel Vacek <neelx@redhat.com>
  Dave Airlie <airlied@redhat.com>
  Dave Hansen <dave.hansen@linux.intel.com>
  Dave Kleikamp <dave.kleikamp@oracle.com>
  Dave Switzer <david.switzer@intel.com>
  David Howells <dhowells@redhat.com>
  David S. Miller <davem@davemloft.net>
  David Schiller <david.schiller@jku.at>
  David Sterba <dsterba@suse.com>
  David Wang <00107082@163.com>
  Denis Efremov <efremov@linux.com>
  Dmitry Antipov <dmantipov@yandex.ru>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Doug Berger <opendmb@gmail.com>
  Douglas Anderson <dianders@chromium.org>
  Edson Juliano Drosdeck <edson.drosdeck@gmail.com>
  Edward Adam Davis <eadavis@qq.com>
  Ekansh Gupta <quic_ekangupt@quicinc.com>
  Eric Biggers <ebiggers@google.com>
  Eric Dumazet <edumazet@google.com>
  Fabio Estevam <festevam@denx.de>
  Fedor Pchelkin <pchelkin@ispras.ru>
  Felix Kuehling <Felix.Kuehling@amd.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Florian Westphal <fw@strlen.de>
  Frank Li <Frank.Li@nxp.com>
  Frank Rowand <frank.rowand@sony.com>
  Frederic Weisbecker <frederic@kernel.org>
  free5lot <mail@free5lot.com>
  Furong Xu <0x1207@gmail.com>
  Gabriel Krisman Bertazi <krisman@suse.de>
  Ghanshyam Agrawal <ghanshyam1898@gmail.com>
  Greg KH <gregkh@linuxfoundation.org>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Gregory CLEMENT <gregory.clement@bootlin.com>
  Guenter Roeck <linux@roeck-us.net>
  Guilherme G. Piccoli <gpiccoli@igalia.com>
  H. Nikolaus Schaller <hns@goldelico.com>
  H. Peter Anvin <hpa@zytor.com>
  Hannes Reinecke <hare@suse.de>
  Hans de Goede <hdegoede@redhat.com>
  Hans Verkuil <hverkuil-cisco@xs4all.nl>
  Hardik Gajjar <hgajjar@de.adit-jv.com>
  Harshit Mogalapalli <harshit.m.mogalapalli@oracle.com>
  Harshit Shah <harshitshah.opendev@gmail.com>
  Heiko Carstens <hca@linux.ibm.com>
  Heiko Stuebner <heiko@sntech.de>
  Heiner Kallweit <hkallweit1@gmail.com>
  Helge Deller <deller@gmx.de>
  Helge Deller <deller@kernel.org>
  Herbert Xu <herbert@gondor.apana.org.au>
  Hongchen Zhang <zhanghongchen@loongson.cn>
  Hou Tao <houtao1@huawei.com>
  Hugo Villeneuve <hvilleneuve@dimonoff.com>
  Ian Rogers <irogers@google.com>
  Ido Schimmel <idosch@nvidia.com>
  Ilya Dryomov <idryomov@gmail.com>
  Inki Dae <inki.dae@samsung.com>
  Ivan Vecera <ivecera@redhat.com>
  Jack Wang <jinpu.wang@ionos.com>
  JackBB Wu <wojackbb@gmail.com>
  Jaegeuk Kim <jaegeuk@kernel.org>
  JaimeLiao <jaimeliao@mxic.com.tw>
  Jakub Kicinski <kuba@kernel.org>
  Jan Beulich <jbeulich@suse.com>
  Jason Gerecke <jason.gerecke@wacom.com>
  Jason Gerecke <killertofu@gmail.com>
  Javier Martinez Canillas <javierm@redhat.com>
  Jay Vosburgh <jay.vosburgh@canonical.com>
  Jean Delvare <jdelvare@suse.de>
  Jedrzej Jagielski <jedrzej.jagielski@intel.com>
  Jens Axboe <axboe@kernel.dk>
  Jianjun Wang <jianjun.wang@mediatek.com>
  Jiri Kosina <jkosina@suse.com>
  Jiri Kosina <jkosina@suse.cz>
  Johan Hovold <johan+linaro@kernel.org>
  Johan Hovold <johan@kernel.org>
  Johan Jonker <jbx6244@gmail.com>
  Johannes Berg <johannes.berg@intel.com>
  Jon Hunter <jonathanh@nvidia.com>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Jozsef Kadlecsik <kadlec@netfilter.org>
  Julian Wiedmann <jwi@linux.ibm.com>
  Junxiao Bi <junxiao.bi@oracle.com>
  Justin Tee <justin.tee@broadcom.com>
  Kalle Valo <kvalo@kernel.org>
  Kalle Valo <quic_kvalo@quicinc.com>
  Kamal Dasu <kamal.dasu@broadcom.com>
  Karsten Graul <kgraul@linux.ibm.com>
  Kees Cook <keescook@chromium.org>
  kernelci.org bot <bot@kernelci.org>
  Konrad Dybcio <konrad.dybcio@linaro.org>
  Krzysztof Wilczyński <kwilczynski@kernel.org>
  Kuan-Wei Chiu <visitorckw@gmail.com>
  Kuniyuki Iwashima <kuniyu@amazon.com>
  Kunwu Chan <chentao@kylinos.cn>
  Lee Duncan <lduncan@suse.com>
  Lee Jones <lee@kernel.org>
  Leon Romanovsky <leon@kernel.org>
  Leonard Dallmayr <leonard.dallmayr@mailbox.org>
  Lin Ma <linma@zju.edu.cn>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Torvalds <torvalds@linuxfoundation.org>
  Loic Prylli <lprylli@netflix.com>
  Louis Peens <louis.peens@corigine.com>
  Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
  Mahmoud Adam <mngyadam@amazon.com>
  Manas Ghandat <ghandatmanas@gmail.com>
  Mao Jinlong <quic_jinlmao@quicinc.com>
  Marc Kleine-Budde <mkl@pengutronix.de>
  Marc Zyngier <maz@kernel.org>
  Marek Szyprowski <m.szyprowski@samsung.com>
  Mario Limonciello <mario.limonciello@amd.com>
  Mark Brown <broonie@kernel.org>
  Martin K. Petersen <martin.petersen@oracle.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Mathieu Poirier <mathieu.poirier@linaro.org>
  Matthew Wilcox (Oracle) <willy@infradead.org>
  Max Kellermann <max.kellermann@ionos.com>
  Max Krummenacher <max.krummenacher@toradex.com>
  Maxime Ripard <mripard@kernel.org>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael S. Tsirkin <mst@redhat.com>
  Michael Tretter <m.tretter@pengutronix.de>
  Mike Rapoport <rppt@linux.ibm.com>
  Ming Lei <ming.lei@redhat.com>
  Minsuk Kang <linuxlovemin@yonsei.ac.kr>
  Miquel Raynal <miquel.raynal@bootlin.com>
  Miri Korenblit <miriam.rachel.korenblit@intel.com>
  Nathan Chancellor <nathan@kernel.org>
  Naveen N Rao <naveen@kernel.org>
  Nikolay Borisov <nik.borisov@suse.com>
  Oleg Nesterov <oleg@redhat.com>
  Oleksij Rempel <o.rempel@pengutronix.de>
  Oliver Neukum <oneukum@suse.com>
  Oliver Upton <oliver.upton@linux.dev>
  Osama Muhammad <osmtendev@gmail.com>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Paolo Abeni <pabeni@redhat.com>
  Paul E. McKenney <paulmck@kernel.org>
  Paul Menzel <pmenzel@molgen.mpg.de>
  Paul Moore <paul@paul-moore.com>
  Peter Robinson <pbrobinson@gmail.com>
  Peter Zijlstra (Intel) <peterz@infradead.org>
  Peter Zijlstra <peterz@infradead.org>
  Petr Machata <petrm@nvidia.com>
  Petr Pavlu <petr.pavlu@suse.com>
  Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
  Piotr Skajewski <piotrx.skajewski@intel.com>
  Prarit Bhargava <prarit@redhat.com>
  Prathu Baronia <prathubaronia2011@gmail.com>
  Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
  Puliang Lu <puliang.lu@fibocom.com>
  qizhong cheng <qizhong.cheng@mediatek.com>
  Qu Wenruo <wqu@suse.com>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Rafael J. Wysocki <rafael@kernel.org>
  Richard Palethorpe <rpalethorpe@suse.com>
  Richard Weinberger <richard@nod.at>
  Rob Clark <robdclark@chromium.org>
  Rob Herring <robh@kernel.org>
  Robert Foss <rfoss@kernel.org>
  Rui Zhang <zr.zhang@vivo.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Saeed Mahameed <saeedm@mellanox.com>
  Saeed Mahameed <saeedm@nvidia.com>
  Salvatore Dipietro <dipiets@amazon.com>
  Sasha Levin <sashal@kernel.org>
  Serge Semin <fancer.lancer@gmail.com>
  Sharath Srinivasan <sharath.srinivasan@oracle.com>
  Shawn Guo <shawnguo@kernel.org>
  Shenwei Wang <shenwei.wang@nxp.com>
  Shigeru Yoshida <syoshida@redhat.com>
  Shiji Yang <yangshiji66@outlook.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Simon Horman <horms@kernel.org>
  Sjoerd Simons <sjoerd@collabora.com>
  Song Liu <song@kernel.org>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Stanislaw Gruszka <stf_xl@wp.pl>
  Stephen Boyd <sboyd@kernel.org>
  Stephen Rothwell <sfr@canb.auug.org.au>
  Steve Wahl <steve.wahl@hpe.com>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Su Hui <suhui@nfschina.com>
  Sunitha Mekala <sunithax.d.mekala@intel.com> (A Contingent worker at Intel)
  Suraj Jitindar Singh <surajjs@amazon.com>
  Takashi Iwai <tiwai@suse.de>
  Takashi Sakamoto <o-takashi@sakamocchi.jp>
  Tatsunosuke Tobita <tatsunosuke.tobita@wacom.com>
  Tejun Heo <tj@kernel.org>
  Theodore Ts'o <tytso@mit.edu>
  Thomas Bogendoerfer <tsbogend@alpha.franken.de>
  Thomas Bourgoin <thomas.bourgoin@foss.st.com>
  Thomas Gleixner <tglx@linutronix.de>
  Thomas Richter <tmricht@linux.ibm.com>
  Tim Chen <tim.c.chen@linux.intel.com>
  Toke Høiland-Jørgensen <toke@toke.dk>
  Tom Rix <trix@redhat.com>
  Tom Zanussi <tom.zanussi@linux.intel.com>
  Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
  Tony Lindgren <tony@atomide.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Ulf Hansson <ulf.hansson@linaro.org>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Ville Syrjälä <ville.syrjala@linux.intel.com>
  Vincent Donnefort <vdonnefort@google.com>
  Vinod Koul <vkoul@kernel.org>
  Weichen Chen <weichen.chen@mediatek.com>
  Weihao Li <liweihao@loongson.cn>
  Wen Gu <guwen@linux.alibaba.com>
  Wenhua Lin <Wenhua.Lin@unisoc.com>
  Wolfram Sang <wsa@kernel.org>
  Xi Ruoyao <xry111@xry111.site>
  Xiaolei Wang <xiaolei.wang@windriver.com>
  Xiubo Li <xiubli@redhat.com>
  Yang Xu <xuyang2018.jy@fujitsu.com>
  Ye Bin <yebin10@huawei.com>
  Yevgeny Kliteynik <kliteyn@nvidia.com>
  Yonghong Song <yonghong.song@linux.dev>
  Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
  Youquan Song <youquan.song@intel.com>
  yuan linyu <yuanlinyu@hihonor.com>
  Yuluo Qiu <qyl27@outlook.com>
  Zach O'Keefe <zokeefe@google.com>
  Zakhar Semenov <mail@free5lot.com>
  Zenm Chen <zenmchen@gmail.com>
  Zhang Rui <rui.zhang@intel.com>
  Zhengchao Shao <shaozhengchao@huawei.com>
  Zhihao Cheng <chengzhihao1@huawei.com>
  zhili.liu <zhili.liu@ucas.com.cn>
  Zhipeng Lu <alexious@zju.edu.cn>
  Zhiquan Li <zhiquan1.li@intel.com>
  Zhu Yanjun <yanjun.zhu@linux.dev>
  Стас Ничипорович <stasn77@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   f0602893f43a5..6e1f54a4985b6  6e1f54a4985b63bc1b55a09e5e75a974c5d6719b -> tested/linux-5.4


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 22:30:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 22:30:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684952.1065227 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rde3j-0006Hb-Ci; Fri, 23 Feb 2024 22:30:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684952.1065227; Fri, 23 Feb 2024 22:30:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rde3j-0006HU-AA; Fri, 23 Feb 2024 22:30:23 +0000
Received: by outflank-mailman (input) for mailman id 684952;
 Fri, 23 Feb 2024 22:30:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rde3i-0006HO-B6
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 22:30:22 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1effd2a4-d29b-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 23:30:19 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id C5EFECE2E7A;
 Fri, 23 Feb 2024 22:30:14 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 83B17C433F1;
 Fri, 23 Feb 2024 22:30:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1effd2a4-d29b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708727414;
	bh=XDpdY/K24aHYoj+1hE2HmwgSYhlLtwJ0OGq3GfCguro=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=T44fXxbNneK6gM2J33cw7IEC2ywU463ss60xjpuzd2lagrgOsot7GGbPhRl1i3cL7
	 HyOJah0Yne3SJbms/JvzHuEah2CwHwFOu0porp7tAaGH5eJNmIA6o2JmGn5viEsitb
	 d7OTzcH/grO8rXi0guYPzHFsRo1H29YUAbN9GpxBgVvVFmp1/svXBRTAL4w470ixfR
	 cXcrwT1myPsoUxjfuqZKwg3VFNWXAHku9Nw5qbNNuBA12emIYNWzOH1awL/u53BMqL
	 8uWrP9Up1eMmGgR2iuz8GA+KCOQhfGBQdodNzxEnuRrc/3qRhHykekw9ARFsa/WVoM
	 uod3epd2OttzA==
Date: Fri, 23 Feb 2024 14:30:09 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: "Chen, Jiqian" <Jiqian.Chen@amd.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
    =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
    Wei Liu <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, 
    Julien Grall <julien@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, 
    Juergen Gross <jgross@suse.com>, 
    "Daniel P . Smith" <dpsmith@apertussolutions.com>, 
    "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>, 
    "Huang, Ray" <Ray.Huang@amd.com>
Subject: Re: [RFC XEN PATCH v5 2/5] x86/pvh: Allow (un)map_pirq when dom0 is
 PVH
In-Reply-To: <BL1PR12MB58497693D450A36C84EE8B22E7552@BL1PR12MB5849.namprd12.prod.outlook.com>
Message-ID: <alpine.DEB.2.22.394.2402231425300.754277@ubuntu-linux-20-04-desktop>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com> <20240112061317.418658-3-Jiqian.Chen@amd.com> <alpine.DEB.2.22.394.2402221632170.754277@ubuntu-linux-20-04-desktop> <BL1PR12MB58497693D450A36C84EE8B22E7552@BL1PR12MB5849.namprd12.prod.outlook.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 23 Feb 2024, Chen, Jiqian wrote:
> On 2024/2/23 08:40, Stefano Stabellini wrote:
> > On Fri, 12 Jan 2024, Jiqian Chen wrote:
> >> If run Xen with PVH dom0 and hvm domU, hvm will map a pirq for
> >> a passthrough device by using gsi, see
> >> xen_pt_realize->xc_physdev_map_pirq and
> >> pci_add_dm_done->xc_physdev_map_pirq. Then xc_physdev_map_pirq
> >> will call into Xen, but in hvm_physdev_op, PHYSDEVOP_map_pirq
> >> is not allowed because currd is PVH dom0 and PVH has no
> >> X86_EMU_USE_PIRQ flag, it will fail at has_pirq check.
> >>
> >> So, allow PHYSDEVOP_map_pirq when dom0 is PVH and also allow
> >> PHYSDEVOP_unmap_pirq for the failed path to unmap pirq. And
> >> add a new check to prevent self map when caller has no PIRQ
> >> flag.
> >>
> >> Co-developed-by: Huang Rui <ray.huang@amd.com>
> >> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> >> ---
> >>  xen/arch/x86/hvm/hypercall.c |  2 ++
> >>  xen/arch/x86/physdev.c       | 22 ++++++++++++++++++++++
> >>  2 files changed, 24 insertions(+)
> >>
> >> diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
> >> index 6ad5b4d5f11f..493998b42ec5 100644
> >> --- a/xen/arch/x86/hvm/hypercall.c
> >> +++ b/xen/arch/x86/hvm/hypercall.c
> >> @@ -74,6 +74,8 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> >>      {
> >>      case PHYSDEVOP_map_pirq:
> >>      case PHYSDEVOP_unmap_pirq:
> >> +        break;
> >> +
> >>      case PHYSDEVOP_eoi:
> >>      case PHYSDEVOP_irq_status_query:
> >>      case PHYSDEVOP_get_free_pirq:
> >> diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
> >> index 47c4da0af7e1..7f2422c2a483 100644
> >> --- a/xen/arch/x86/physdev.c
> >> +++ b/xen/arch/x86/physdev.c
> >> @@ -303,11 +303,22 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> >>      case PHYSDEVOP_map_pirq: {
> >>          physdev_map_pirq_t map;
> >>          struct msi_info msi;
> >> +        struct domain *d;
> >>  
> >>          ret = -EFAULT;
> >>          if ( copy_from_guest(&map, arg, 1) != 0 )
> >>              break;
> >>  
> >> +        d = rcu_lock_domain_by_any_id(map.domid);
> >> +        if ( d == NULL )
> >> +            return -ESRCH;
> >> +        if ( !is_pv_domain(d) && !has_pirq(d) )
> > 
> > I think this could just be:
> > 
> >     if ( !has_pirq(d) )
> > 
> > Right?
> No. In the beginning, I only set this condition here, but it destroyed PV dom0.
> Because  PV has no pirq flag too, it can match this condition and return -EOPNOTSUPP, PHYSDEVOP_map_pirq will fail.

Yes I get it now. Thanks for the explanation. I think "has_pirq" is
misnamed and should probably be hvm_has_pirq or something like that.

I am not asking to fix it, but maybe an in-code comment here would help,
like:

/* if it is an HVM guest, check if it has PIRQs */

in any case:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 22:36:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 22:36:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684956.1065238 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rde9o-0006u3-W3; Fri, 23 Feb 2024 22:36:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684956.1065238; Fri, 23 Feb 2024 22:36:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rde9o-0006tw-TC; Fri, 23 Feb 2024 22:36:40 +0000
Received: by outflank-mailman (input) for mailman id 684956;
 Fri, 23 Feb 2024 22:36:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rde9n-0006to-6u
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 22:36:39 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 00de4ecc-d29c-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 23:36:37 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 48E67615B9;
 Fri, 23 Feb 2024 22:36:36 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 834DCC433C7;
 Fri, 23 Feb 2024 22:36:34 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00de4ecc-d29c-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708727796;
	bh=lUyCHS8sm4/gGGwJKCtAw1+ZW3FxxNSA1nxAgPdcSR8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=sa0HIh3x/8W5CnK/QQDrRBW6d83bqDf03806hGtnjpbdfH8PsdUXDfL1Ur0jrX/5R
	 DOfX/QMoEqszzsNoeEVEiT7Ig4n0E5YwmHMbAHU0KZ0cUuhrql3q/EKW5L1aEa9xOX
	 4sgAhow/oKFa/p+OBAdIMMIdbTacoXlToAqI6dCVqk+z2nf4SMZJaZnmqoGVK3SA4S
	 zJUu9AxCMYSas+BtouWyCFAdw+x6IYhQRLx8USxf0KKEFhvFGJJhG6yX05RDkM+Rj3
	 L/M/pCClaOtEWP0iB6thkX/j7+sckv8xcCWBmT8hTs66uFCo94v+Xrr7IlMRfu1P2d
	 utLoWAjVYGyAA==
Date: Fri, 23 Feb 2024 14:36:32 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Simone Ballarin <simone.ballarin@bugseng.com>, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 3/4] xen/include: add pure and const attributes
In-Reply-To: <6eb4dc7c-c6a5-496a-a37f-125b00e9aaa3@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402231432090.754277@ubuntu-linux-20-04-desktop>
References: <cover.1697638210.git.simone.ballarin@bugseng.com> <fd5421162a00aa782e0776324ff6497193c1e3d3.1697638210.git.simone.ballarin@bugseng.com> <89778285-5cba-8fb5-70bc-710b6dd30a10@suse.com> <0032186f-80c7-4dba-b46e-10d4a8e2a8cb@bugseng.com>
 <e8bf9817-fd54-9bf4-4302-dcee682f9172@suse.com> <alpine.DEB.2.22.394.2310231417260.3516@ubuntu-linux-20-04-desktop> <8f8141e2-c55c-2e58-a078-d5d8764a02df@suse.com> <alpine.DEB.2.22.394.2402221730270.754277@ubuntu-linux-20-04-desktop>
 <6eb4dc7c-c6a5-496a-a37f-125b00e9aaa3@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-628198823-1708727577=:754277"
Content-ID: <alpine.DEB.2.22.394.2402231433310.754277@ubuntu-linux-20-04-desktop>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-628198823-1708727577=:754277
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.22.394.2402231433311.754277@ubuntu-linux-20-04-desktop>

On Fri, 23 Feb 2024, Jan Beulich wrote:
> On 23.02.2024 02:32, Stefano Stabellini wrote:
> > On Tue, 24 Oct 2023, Jan Beulich wrote:
> >> On 24.10.2023 00:05, Stefano Stabellini wrote:
> >>> On Mon, 23 Oct 2023, Jan Beulich wrote:
> >>>> On 23.10.2023 17:23, Simone Ballarin wrote:
> >>>>> On 23/10/23 15:34, Jan Beulich wrote:
> >>>>>> On 18.10.2023 16:18, Simone Ballarin wrote:
> >>>>>>> --- a/xen/include/xen/pdx.h
> >>>>>>> +++ b/xen/include/xen/pdx.h
> >>>>>>> @@ -164,7 +164,7 @@ static inline unsigned long pfn_to_pdx(unsigned long pfn)
> >>>>>>>    * @param pdx Page index
> >>>>>>>    * @return Obtained pfn after decompressing the pdx
> >>>>>>>    */
> >>>>>>> -static inline unsigned long pdx_to_pfn(unsigned long pdx)
> >>>>>>> +static inline __attribute_pure__ unsigned long pdx_to_pfn(unsigned long pdx)
> >>>>>>>   {
> >>>>>>>       return (pdx & pfn_pdx_bottom_mask) |
> >>>>>>>              ((pdx << pfn_pdx_hole_shift) & pfn_top_mask);
> >>>>>>
> >>>>>> Taking this as an example for what I've said above: The compiler can't
> >>>>>> know that the globals used by the functions won't change value. Even
> >>>>>> within Xen it is only by convention that these variables are assigned
> >>>>>> their values during boot, and then aren't changed anymore. Which makes
> >>>>>> me wonder: Did you check carefully that around the time the variables
> >>>>>> have their values established, no calls to the functions exist (which
> >>>>>> might then be subject to folding)?
> >>>>>
> >>>>> There is no need to check that, the GCC documentation explicitly says:
> >>>>>
> >>>>> However, functions declared with the pure attribute *can safely read any 
> >>>>> non-volatile objects*, and modify the value of objects in a way that 
> >>>>> does not affect their return value or the observable state of the program.
> >>>>
> >>>> I did quote this same text in response to what Andrew has said, but I also
> >>>> did note there that this needs to be taken with a grain of salt: The
> >>>> compiler generally assumes a single-threaded environment, i.e. no changes
> >>>> to globals behind the back of the code it is processing.
> >>>
> >>> Let's start from the beginning. The reason for Simone to add
> >>> __attribute_pure__ to pdx_to_pfn and other functions is for
> >>> documentation purposes. It is OK if it doesn't serve any purpose other
> >>> than documentation.
> >>>
> >>> Andrew, for sure we do not want to lie to the compiler and introduce
> >>> undefined behavior. If we think there is a risk of it, we should not do
> >>> it.
> >>>
> >>> So, what do we want to document? We want to document that the function
> >>> does not have side effects according to MISRA's definition of it, which
> >>> might subtly differ from GCC's definition.
> >>>
> >>> Looking at GCC's definition of __attribute_pure__, with the
> >>> clarification statement copy/pasted above by both Simone and Jan, it
> >>> seems that __attribute_pure__ matches MISRA's definition of a function
> >>> without side effects. It also seems that pdx_to_pfn abides to that
> >>> definition.
> >>>
> >>> Jan has a point that GCC might be making other assumptions
> >>> (single-thread execution) that might not hold true in our case. Given
> >>> the way the GCC statement is written I think this is low risk. But maybe
> >>> not all GCC versions we want to support in the project might have the
> >>> same definition of __attribute_pure__. So we could end up using
> >>> __attribute_pure__ correctly for the GCC version used for safety (GCC
> >>> 12.1, see docs/misra/C-language-toolchain.rst) but it might actually
> >>> break an older GCC version.
> >>>
> >>>
> >>> So Option#1 is to use __attribute_pure__ taking the risk that a GCC or
> >>> Clang version might interpret __attribute_pure__ differently and
> >>> potentially misbehave.
> >>>
> >>> Option#2 is to avoid this risk, by not using __attribute_pure__.
> >>> Instead, we can use SAF-xx-safe or deviations.rst to document that
> >>> pdx_to_pfn and other functions like it are without side effects
> >>> according to MISRA's definition.
> >>>
> >>>
> >>> Both options have pros and cons. To me the most important factor is how
> >>> many GCC versions come with the statement "pure attribute can safely
> >>> read any non-volatile objects, and modify the value of objects in a way
> >>> that does not affect their return value or the observable state of the
> >>> program".
> >>>
> >>> I checked and these are the results:
> >>> - gcc 4.0.2: no statement
> >>> - gcc 5.1.0: no statement
> >>> - gcc 6.1.0: no statement
> >>> - gcc 7.1.0: no statement
> >>> - gcc 8.1.0: alternative statement "The pure attribute imposes similar
> >>>   but looser restrictions on a function’s definition than the const
> >>>   attribute: it allows the function to read global variables."
> >>> - gcc 9.1.0: yes statement
> >>>
> >>>
> >>> So based on the above, __attribute_pure__ comes with its current
> >>> definition only from gcc 9 onward. I don't know if as a Xen community we
> >>> clearly declare a range of supported compilers, but I would imagine we
> >>> would still want to support gcc versions older than 9? (Not to mention
> >>> clang, which I haven't checked.)
> >>>
> >>> It doesn't seem to me that __attribute_pure__ could be correctly used on
> >>> pdx_to_pfn with GCC 7.1.0 for example.
> >>
> >> The absence of documentation doesn't mean the attribute had different
> >> (or even undefined) meaning in earlier versions. Instead it means one
> >> would need to consult other places (source code?) to figure out whether
> >> there was any behavioral difference (I don't think there was).
> >>
> >> That said, ...
> >>
> >>> So in conclusion, I think it is better to avoid __attribute_pure__ and
> >>> use SAF-xx-safe or an alternative approach instead.
> >>
> >> ... I agree here. We just don't want to take chances.
> > 
> > Let me resurrect this thread.
> > 
> > Could we use something like "pure" that we #define as we want?
> > 
> > Depending on the compiler version or other options we could #define pure
> > to __attribute_pure__ or to nothing.
> 
> While we can do about anything, I don't think it's a good idea to overload
> a well known term with something having somewhat different meaning. If a
> differently named custom attribute helps, that might be a possible option.

It doesn't have a different meaning. If it had a different meaning I'd
agree with you.

The goal is for the #define to have exactly the same meaning as the gcc
definition from gcc 9 onward. However, other versions of gcc or other
compilers could have different semantics. Also we might not want to
allow gcc to perform the optimizations that it might want to do if the
attribute is passed.

So the definition would be clear and 100% aligned with the modern gcc
definition. However we would be able to control the behavior better.
--8323329-628198823-1708727577=:754277--


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 22:53:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 22:53:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684960.1065248 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdeQ9-0001QQ-D9; Fri, 23 Feb 2024 22:53:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684960.1065248; Fri, 23 Feb 2024 22:53:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdeQ9-0001QJ-98; Fri, 23 Feb 2024 22:53:33 +0000
Received: by outflank-mailman (input) for mailman id 684960;
 Fri, 23 Feb 2024 22:53:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdeQ8-0001QD-MX
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 22:53:32 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5cd9dddf-d29e-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 23:53:30 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 6D94561A0F;
 Fri, 23 Feb 2024 22:53:29 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5023EC433C7;
 Fri, 23 Feb 2024 22:53:28 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5cd9dddf-d29e-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708728809;
	bh=ber/363D8X2ld/t6L5y424sXzcWWrZxiUdpGvaBWudk=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=EEiDsF+mWilZIskklE1ev588FwyFs0FWHroBJVia2k2GYrvGR1KXyrHdYjil2ng3T
	 fG5IfT7dwOc6qwSujjH5mYGta7GR36EiD5gXRtbbGnOWdca04O9LP0gDY1WDW21xvU
	 z7TQ14o0OLEIJPlNQ1UScwvijaXGhTllo/qH7jP/9cQiyLj6juoKM89TR+gSZNv1ex
	 0v61x0sTwmtkh6CZ5P5A5sbBMko821yq9OwBqbN9Bogfghhf8zZGeX6YajBDpIwzk0
	 3wa0f6/KNpi9PV6RcXv9TaLbJVcId/ba4THzbNyWly6UWQdheWtla9GeaLtvpbUyB8
	 SLj91cgyPcaVw==
Date: Fri, 23 Feb 2024 14:53:26 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH 2/2] automation/eclair: tag MISRA C:2012 Rule 8.2 as
 clean
In-Reply-To: <59db8ce679a2ab46b23b39e8f1591cd9fc1c9218.1708677243.git.federico.serafini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402231453210.754277@ubuntu-linux-20-04-desktop>
References: <cover.1708677243.git.federico.serafini@bugseng.com> <59db8ce679a2ab46b23b39e8f1591cd9fc1c9218.1708677243.git.federico.serafini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 23 Feb 2024, Federico Serafini wrote:
> Update ECLAIR configuration to consider Rule 8.2 as clean.
> 
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/eclair_analysis/ECLAIR/tagging.ecl | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl b/automation/eclair_analysis/ECLAIR/tagging.ecl
> index 900c532196..a1dea32b21 100644
> --- a/automation/eclair_analysis/ECLAIR/tagging.ecl
> +++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
> @@ -30,7 +30,7 @@
>  
>  -doc_begin="Clean guidelines: new violations for these guidelines are not accepted."
>  
> --service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R2.2||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R8.1||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R9.2||MC3R1.R9.4||MC3R1.R9.5||MC3R1.R12.5||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6"
> +-service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R2.2||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R4.2||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R5.6||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R8.1||MC3R1.R8.2||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R9.2||MC3R1.R9.4||MC3R1.R9.5||MC3R1.R12.5||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.6||MC3R1.R20.13||MC3R1.R20.14||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6"
>  }
>  
>  -setq=target,getenv("XEN_TARGET_ARCH")
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 22:54:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 22:54:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684962.1065258 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdeQw-0001uC-MR; Fri, 23 Feb 2024 22:54:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684962.1065258; Fri, 23 Feb 2024 22:54:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdeQw-0001u5-I7; Fri, 23 Feb 2024 22:54:22 +0000
Received: by outflank-mailman (input) for mailman id 684962;
 Fri, 23 Feb 2024 22:54:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdeQv-0001sk-8u
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 22:54:21 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7a22bd2e-d29e-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 23:54:20 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id C684C61408;
 Fri, 23 Feb 2024 22:54:18 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id ADC4BC433F1;
 Fri, 23 Feb 2024 22:54:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a22bd2e-d29e-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708728858;
	bh=lf3iFOIl9mvNIQh9fRC8B7vVoC3x2fksP2YI5SZZc2I=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=nsIrGW5D5sHD91RnjlAIEO1/BwqNTqgbH24dztjHy83lzBw+FhyiLi88ETt/FoKb8
	 iRXotBaLPOV6WGTf3jo6BdPmD/7f8Dv5RDdEzGhQKyqTnrz6flBl7KkIVJu76Q3gwT
	 xlV8os0vdDlj/35ZwSDnaJhJYkNvy7fzPzwb1xvwHbvgDNWCdNcaJ5eCbvU8RC47dp
	 bS8z7nCD1j1M61ZrgpWEmSg8mlPorKZe07i2pdvXlA1qKowJU0tGqwm2MJw/iS+9QH
	 N/EucOUeEO2Fwy2v1HByhEFTOeiIsEBMbSrq6fyJjVjG7pwZims6B7vEaIG0A21sKD
	 iXASdASskLPKQ==
Date: Fri, 23 Feb 2024 14:54:16 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH 1/2] automation/eclair: fully deviate MISRA C:2012
 Rules 5.7 and 18.7
In-Reply-To: <1fee04ad07b03c13c44f66a20c4ee70737069650.1708677243.git.federico.serafini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402231453490.754277@ubuntu-linux-20-04-desktop>
References: <cover.1708677243.git.federico.serafini@bugseng.com> <1fee04ad07b03c13c44f66a20c4ee70737069650.1708677243.git.federico.serafini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 23 Feb 2024, Federico Serafini wrote:
> Update ECLAIR configuration to fully deviate Rules 5.7 and 18.7
> as agreed during MISRA meeetings.
> 
> Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  automation/eclair_analysis/ECLAIR/deviations.ecl | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
> 
> diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
> index fd32ff8a9c..02eae39786 100644
> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> @@ -107,6 +107,11 @@ number of guest paging levels."
>  -config=MC3R1.R5.6,reports+={deliberate,"any_area(any_loc(file(adopted_r5_6)))"}
>  -doc_end
>  
> +-doc_begin="The project intentionally reuses tag names in order to have identifiers matching the applicable external specifications as well as established internal conventions.
> +As there is little possibility for developer confusion not resulting into compilation errors, the risk of renaming outweighs the potential advantages of compliance."
> +-config=MC3R1.R5.7,reports+={deliberate,"any()"}
> +-doc_end
> +
>  #
>  # Series 7.
>  #
> @@ -373,6 +378,15 @@ explicit comment indicating the fallthrough intention is present."
>  -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(^(?s).*/\\* [fF]all ?through.? \\*/.*$,0..1))))"}
>  -doc_end
>  
> +#
> +# Series 18.
> +#
> +
> +-doc_begin="Flexible array members are deliberately used and XEN developers are aware of the dangers related to them:
> +unexpected result when the structure is given as argument to a sizeof() operator and the truncation in assignment between structures."
> +-config=MC3R1.R18.7,reports+={deliberate, "any()"}
> +-doc_end
> +
>  #
>  # Series 20.
>  #
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 22:56:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 22:56:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684966.1065267 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdeTM-0002Uy-1Q; Fri, 23 Feb 2024 22:56:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684966.1065267; Fri, 23 Feb 2024 22:56:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdeTL-0002Ur-V3; Fri, 23 Feb 2024 22:56:51 +0000
Received: by outflank-mailman (input) for mailman id 684966;
 Fri, 23 Feb 2024 22:56:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdeTK-0002Ue-M8
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 22:56:50 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d1e5ca62-d29e-11ee-98f5-efadbce2ee36;
 Fri, 23 Feb 2024 23:56:48 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 5946DCE286D;
 Fri, 23 Feb 2024 22:56:45 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 639C9C433F1;
 Fri, 23 Feb 2024 22:56:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1e5ca62-d29e-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708729004;
	bh=8tFUB0YujzU0UFzQ+rQJCSmQzCxHBhbk3AZvnWwBN9c=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=qK42QPUpJGP9IZWf4210kWO/VboKmV82Esq7856ZDrRoMo3WtWtekUVGVBmvvWwJ9
	 KLB/q7QQZsmKes+nj49fXBbFbNz7TiyVypHI2P9h3/dk6+M48H4h1GMyk58EZg24h7
	 s2U1PxVuTXe3Z0c8FjZr6cnIkEfhloQs2LjvAT6UUpjUvyvYYRGpAbDAfluMEqMn+F
	 RFQNDQkQntSB+fqoZZiejroSYCe3VVz8+ir3uo/0gPQ/vAEWKvgxRnASdMz4RoMwNx
	 7hEAw4wlxOo1/dvFmRCY4am8nrSrJ3mUferJW6XBdFkh3Q/yAa67wGN6DbkZ9+44iT
	 kvKw64zLIvaJQ==
Date: Fri, 23 Feb 2024 14:56:41 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH 1/2] xen/console: drop return value from
 consoled_guest_rx/tx
In-Reply-To: <4998ec735bd7e5a50a229507e2b92ae56ec1ba4b.1708680104.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402231456350.754277@ubuntu-linux-20-04-desktop>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com> <4998ec735bd7e5a50a229507e2b92ae56ec1ba4b.1708680104.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 23 Feb 2024, Nicola Vetrini wrote:
> These functions never saw a usage of their return value since
> they were introduced, so it can be dropped since their usages
> violate MISRA C Rule 17.7:
> "The value returned by a function having non-void return type shall be used".
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/drivers/char/consoled.c | 17 +++++------------
>  xen/include/xen/consoled.h  |  4 ++--
>  2 files changed, 7 insertions(+), 14 deletions(-)
> 
> diff --git a/xen/drivers/char/consoled.c b/xen/drivers/char/consoled.c
> index 222e01844271..b415b632cecc 100644
> --- a/xen/drivers/char/consoled.c
> +++ b/xen/drivers/char/consoled.c
> @@ -43,13 +43,13 @@ struct xencons_interface *consoled_get_ring_addr(void)
>  static char buf[BUF_SZ + 1];
>  
>  /* Receives characters from a domain's PV console */
> -size_t consoled_guest_rx(void)
> +void consoled_guest_rx(void)
>  {
> -    size_t recv = 0, idx = 0;
> +    size_t idx = 0;
>      XENCONS_RING_IDX cons, prod;
>  
>      if ( !cons_ring )
> -        return 0;
> +        return;
>  
>      spin_lock(&rx_lock);
>  
> @@ -73,7 +73,6 @@ size_t consoled_guest_rx(void)
>          char c = cons_ring->out[MASK_XENCONS_IDX(cons++, cons_ring->out)];
>  
>          buf[idx++] = c;
> -        recv++;
>  
>          if ( idx >= BUF_SZ )
>          {
> @@ -92,18 +91,15 @@ size_t consoled_guest_rx(void)
>  
>   out:
>      spin_unlock(&rx_lock);
> -
> -    return recv;
>  }
>  
>  /* Sends a character into a domain's PV console */
> -size_t consoled_guest_tx(char c)
> +void consoled_guest_tx(char c)
>  {
> -    size_t sent = 0;
>      XENCONS_RING_IDX cons, prod;
>  
>      if ( !cons_ring )
> -        return 0;
> +        return;
>  
>      cons = ACCESS_ONCE(cons_ring->in_cons);
>      prod = cons_ring->in_prod;
> @@ -121,7 +117,6 @@ size_t consoled_guest_tx(char c)
>          goto notify;
>  
>      cons_ring->in[MASK_XENCONS_IDX(prod++, cons_ring->in)] = c;
> -    sent++;
>  
>      /* Write to the ring before updating the pointer */
>      smp_wmb();
> @@ -130,8 +125,6 @@ size_t consoled_guest_tx(char c)
>   notify:
>      /* Always notify the guest: prevents receive path from getting stuck. */
>      pv_shim_inject_evtchn(pv_console_evtchn());
> -
> -    return sent;
>  }
>  
>  /*
> diff --git a/xen/include/xen/consoled.h b/xen/include/xen/consoled.h
> index 2b30516b3a0a..bd7ab6329ee8 100644
> --- a/xen/include/xen/consoled.h
> +++ b/xen/include/xen/consoled.h
> @@ -5,8 +5,8 @@
>  
>  void consoled_set_ring_addr(struct xencons_interface *ring);
>  struct xencons_interface *consoled_get_ring_addr(void);
> -size_t consoled_guest_rx(void);
> -size_t consoled_guest_tx(char c);
> +void consoled_guest_rx(void);
> +void consoled_guest_tx(char c);
>  
>  #endif /* __XEN_CONSOLED_H__ */
>  /*
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 22:57:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 22:57:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684967.1065278 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdeTU-0002nD-Cl; Fri, 23 Feb 2024 22:57:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684967.1065278; Fri, 23 Feb 2024 22:57:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdeTU-0002n4-A6; Fri, 23 Feb 2024 22:57:00 +0000
Received: by outflank-mailman (input) for mailman id 684967;
 Fri, 23 Feb 2024 22:56:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdeTS-0002m9-Nz; Fri, 23 Feb 2024 22:56:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdeTS-00068Q-MC; Fri, 23 Feb 2024 22:56:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdeTS-0008Rk-C9; Fri, 23 Feb 2024 22:56:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdeTS-0004YO-Bl; Fri, 23 Feb 2024 22:56:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=IktdaiOBHhc26/mQrXqv1NOE3cu7yoFbVy7R26MpJcM=; b=C30ihgACwD5fGut3duHcI0Lo0L
	DPMr5BzhZyYvxDYhFOXlkUJnHX4ZgiqV+AeQeMrALdsLwCAmNYkkX8ieuldNpxQxvQFfbEAL6l+Km
	pjOg7M7NdAr8762uW9yg/Hm00E8cuzegZfSzbfTCT7tqhpNpDQhAFw81oSo0y65FuPOU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184739-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184739: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f5e1c527d0a0d09ca0cb1dcd8d4ab4a1a5261e91
X-Osstest-Versions-That:
    xen=812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Fri, 23 Feb 2024 22:56:58 +0000

flight 184739 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184739/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  f5e1c527d0a0d09ca0cb1dcd8d4ab4a1a5261e91
baseline version:
 xen                  812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b

Last test of basis   184730  2024-02-22 17:02:09 Z    1 days
Testing same since   184739  2024-02-23 18:00:28 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Christian Lindig <christian.lindig@cloud.com>
  Edwin Török <edwin.torok@cloud.com>
  George Dunlap <george.dunlap@cloud.com> # Golang bits
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Petr Beneš <w1benny@gmail.com>
  Samuel Thibault <samuel.thibault@ens-lyon.org>
  Shawn Anastasio <sanastasio@raptorengineering.com>
  Tamas K Lengyel <tamas@tklengyel.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   812bdc99f4..f5e1c527d0  f5e1c527d0a0d09ca0cb1dcd8d4ab4a1a5261e91 -> smoke


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 22:59:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 22:59:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684977.1065288 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdeVV-0003ds-Q7; Fri, 23 Feb 2024 22:59:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684977.1065288; Fri, 23 Feb 2024 22:59:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdeVV-0003dl-NK; Fri, 23 Feb 2024 22:59:05 +0000
Received: by outflank-mailman (input) for mailman id 684977;
 Fri, 23 Feb 2024 22:59:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdeVU-0003db-7y
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 22:59:04 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 222f9d44-d29f-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 23:59:02 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 208C0CE2F51;
 Fri, 23 Feb 2024 22:59:00 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 345ADC43390;
 Fri, 23 Feb 2024 22:58:57 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 222f9d44-d29f-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708729139;
	bh=8AkDEPaYhqFxix1alBDBCdbW/wA1+l01GSxZ2Uy6Ma0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=IAVyJKYuQhLf7YFRskANhVzJCBF3odrUwSDVwL6gXJhcqJqMRDICH6n+vsJ1QIyd/
	 Bj8vHirz2s+Ec7yZcZxioNRyHlhB4rbubmzq9x2N17kJ4wkCA3LVCPTN1ruWS0huM2
	 gTB2dnVNc13elVyp02pzBCfxdHavJlJsLNfN8ApqxLHssLR5DwR4ut8fSMoOuhSmpd
	 Frldmq3tgbQgtqbesqiPgOKZPo6eW9vml90eJKu484Ywip5f5ATRbNBHZbzacEm7eZ
	 UMFw4oau60EYyMPWQDJmFTPS16dqJo8+4UH3cVkEeO6f7YV2X9nNygtIXZOu1vpk2v
	 LwOqMUQZ/3qCw==
Date: Fri, 23 Feb 2024 14:58:48 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
In-Reply-To: <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402231458400.754277@ubuntu-linux-20-04-desktop>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com> <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 23 Feb 2024, Nicola Vetrini wrote:
> Refactor cpu_notifier_call_chain into two functions:
> - the variant that is allowed to fail loses the nofail flag
> - the variant that shouldn't fail is encapsulated in a call
>   to the failing variant, with an additional check.
> 
> This prevents uses of the function that are not supposed to
> fail from ignoring the return value, thus violating Rule 17.7:
> "The value returned by a function having non-void return type shall
> be used".
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/common/cpu.c | 31 +++++++++++++++++++------------
>  1 file changed, 19 insertions(+), 12 deletions(-)
> 
> diff --git a/xen/common/cpu.c b/xen/common/cpu.c
> index 8709db4d2957..0b7cf54c4264 100644
> --- a/xen/common/cpu.c
> +++ b/xen/common/cpu.c
> @@ -78,20 +78,27 @@ void __init register_cpu_notifier(struct notifier_block *nb)
>  }
>  
>  static int cpu_notifier_call_chain(unsigned int cpu, unsigned long action,
> -                                   struct notifier_block **nb, bool nofail)
> +                                   struct notifier_block **nb)
>  {
>      void *hcpu = (void *)(long)cpu;
>      int notifier_rc = notifier_call_chain(&cpu_chain, action, hcpu, nb);
>      int ret =  notifier_to_errno(notifier_rc);
>  
> -    BUG_ON(ret && nofail);
> -
>      return ret;
>  }
>  
> +static void cpu_notifier_call_chain_nofail(unsigned int cpu,
> +                                           unsigned long action,
> +                                           struct notifier_block **nb)
> +{
> +    int ret = cpu_notifier_call_chain(cpu, action, nb);
> +
> +    BUG_ON(ret);
> +}
> +
>  static void cf_check _take_cpu_down(void *unused)
>  {
> -    cpu_notifier_call_chain(smp_processor_id(), CPU_DYING, NULL, true);
> +    cpu_notifier_call_chain_nofail(smp_processor_id(), CPU_DYING, NULL);
>      __cpu_disable();
>  }
>  
> @@ -116,7 +123,7 @@ int cpu_down(unsigned int cpu)
>      if ( !cpu_online(cpu) )
>          goto out;
>  
> -    err = cpu_notifier_call_chain(cpu, CPU_DOWN_PREPARE, &nb, false);
> +    err = cpu_notifier_call_chain(cpu, CPU_DOWN_PREPARE, &nb);
>      if ( err )
>          goto fail;
>  
> @@ -129,14 +136,14 @@ int cpu_down(unsigned int cpu)
>      err = cpu_online(cpu);
>      BUG_ON(err);
>  
> -    cpu_notifier_call_chain(cpu, CPU_DEAD, NULL, true);
> +    cpu_notifier_call_chain_nofail(cpu, CPU_DEAD, NULL);
>  
>      send_global_virq(VIRQ_PCPU_STATE);
>      cpu_hotplug_done();
>      return 0;
>  
>   fail:
> -    cpu_notifier_call_chain(cpu, CPU_DOWN_FAILED, &nb, true);
> +    cpu_notifier_call_chain_nofail(cpu, CPU_DOWN_FAILED, &nb);
>   out:
>      cpu_hotplug_done();
>      return err;
> @@ -157,7 +164,7 @@ int cpu_up(unsigned int cpu)
>      if ( cpu_online(cpu) )
>          goto out;
>  
> -    err = cpu_notifier_call_chain(cpu, CPU_UP_PREPARE, &nb, false);
> +    err = cpu_notifier_call_chain(cpu, CPU_UP_PREPARE, &nb);
>      if ( err )
>          goto fail;
>  
> @@ -165,7 +172,7 @@ int cpu_up(unsigned int cpu)
>      if ( err < 0 )
>          goto fail;
>  
> -    cpu_notifier_call_chain(cpu, CPU_ONLINE, NULL, true);
> +    cpu_notifier_call_chain_nofail(cpu, CPU_ONLINE, NULL);
>  
>      send_global_virq(VIRQ_PCPU_STATE);
>  
> @@ -173,7 +180,7 @@ int cpu_up(unsigned int cpu)
>      return 0;
>  
>   fail:
> -    cpu_notifier_call_chain(cpu, CPU_UP_CANCELED, &nb, true);
> +    cpu_notifier_call_chain_nofail(cpu, CPU_UP_CANCELED, &nb);
>   out:
>      cpu_hotplug_done();
>      return err;
> @@ -181,7 +188,7 @@ int cpu_up(unsigned int cpu)
>  
>  void notify_cpu_starting(unsigned int cpu)
>  {
> -    cpu_notifier_call_chain(cpu, CPU_STARTING, NULL, true);
> +    cpu_notifier_call_chain_nofail(cpu, CPU_STARTING, NULL);
>  }
>  
>  static cpumask_t frozen_cpus;
> @@ -237,7 +244,7 @@ void enable_nonboot_cpus(void)
>      }
>  
>      for_each_cpu ( cpu, &frozen_cpus )
> -        cpu_notifier_call_chain(cpu, CPU_RESUME_FAILED, NULL, true);
> +        cpu_notifier_call_chain_nofail(cpu, CPU_RESUME_FAILED, NULL);
>  
>      cpumask_clear(&frozen_cpus);
>  }
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 23:05:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 23:05:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684983.1065297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdebP-0005ay-Cs; Fri, 23 Feb 2024 23:05:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684983.1065297; Fri, 23 Feb 2024 23:05:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdebP-0005ar-A3; Fri, 23 Feb 2024 23:05:11 +0000
Received: by outflank-mailman (input) for mailman id 684983;
 Fri, 23 Feb 2024 23:05:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdebN-0005al-Bs
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 23:05:09 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fba4aae8-d29f-11ee-98f5-efadbce2ee36;
 Sat, 24 Feb 2024 00:05:07 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id EA7EC615B9;
 Fri, 23 Feb 2024 23:05:03 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id AFE4DC433C7;
 Fri, 23 Feb 2024 23:05:01 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fba4aae8-d29f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708729503;
	bh=vy5raAayEUzIBDSjL2ecol3dPMK+Bwnm2dkQrQw7L40=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=cJvQL8PqyNSuycwskI44peNrHonBUGe5jd72G5PtEb9WKALYNCRNqGnBYwOrjzlgC
	 PfxyuCHkkjG2059WWAtj83tHB3sA5BQR0XJP+U1XUM+cSdwiiiG3zqCnURymPjtVWE
	 U5KYceWGKSnUn75s2AU7UBwip081yML5Qy7dumu/sYA/wEGAcZCHtFBxEQ4u4PPq6J
	 WRk70s5OQsot4eOumv4vYzm0KYBhAW7/v5x1LjW0hN6mpsfUzFg7U+G5d1EJOfLYyt
	 ywZtMZ5v9l+Qr/GQ8LhFoh4FpSpqaMXepkhSDAinsKt3H3VlYS728wTjprhbrIR9gu
	 QOILmgTAjAzqQ==
Date: Fri, 23 Feb 2024 15:05:00 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, bertrand.marquis@arm.com, julien@xen.org, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
In-Reply-To: <4f3b3d52a3ba20b05ecfa068b916b804@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402231502540.754277@ubuntu-linux-20-04-desktop>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com> <4f3b3d52a3ba20b05ecfa068b916b804@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Fri, 23 Feb 2024, Nicola Vetrini wrote:
> On 2024-02-19 16:14, Nicola Vetrini wrote:
> > The cache clearing and invalidation helpers in x86 and Arm didn't
> > comply with MISRA C Rule 17.7: "The value returned by a function
> > having non-void return type shall be used". On Arm they
> > were always returning 0, while some in x86 returned -EOPNOTSUPP
> > and in common/grant_table the return value is saved.
> > 
> > As a consequence, a common helper arch_grant_cache_flush that returns
> > an integer is introduced, so that each architecture can choose whether to
> > return an error value on certain conditions, and the helpers have either
> > been changed to return void (on Arm) or deleted entirely (on x86).
> > 
> > Signed-off-by: Julien Grall <julien@xen.org>
> > Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> > ---
> > The original refactor idea came from Julien Grall in [1]; I edited that
> > proposal
> > to fix build errors.
> > 
> > I did introduce a cast to void for the call to flush_area_local on x86,
> > because
> > even before this patch the return value of that function wasn't checked in
> > all
> > but one use in x86/smp.c, and in this context the helper (perhaps
> > incidentally)
> > ignored the return value of flush_area_local.
> > 
> > [1]
> > https://lore.kernel.org/xen-devel/09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org/
> > ---
> >  xen/arch/arm/include/asm/page.h     | 33 ++++++++++++++++++-----------
> >  xen/arch/x86/include/asm/flushtlb.h | 23 ++++++++++----------
> >  xen/common/grant_table.c            |  9 +-------
> >  3 files changed, 34 insertions(+), 31 deletions(-)
> > 
> 
> I'll put this patch in the backlog at the moment: too many intricacies while
> trying to untangle xen/flushtlb from xen/mm.h, and there are easier cases that
> can be done faster. If someone is interested I can post the partial work I've
> done so far, even though it doesn't
> build on x86.

I understand that the blocker is:

diff --git a/xen/arch/arm/include/asm/page.h b/xen/arch/arm/include/asm/page.h
index 69f817d1e6..e90c9de361 100644
--- a/xen/arch/arm/include/asm/page.h
+++ b/xen/arch/arm/include/asm/page.h
@@ -123,6 +123,7 @@
 
 #ifndef __ASSEMBLY__
 
+#include <public/grant_table.h>
 #include <xen/errno.h>
 #include <xen/types.h>
 #include <xen/lib.h>


And the headers disentagling required to solve it, right?


Let me ask a silly question. public/grant_table.h seems needed by
arch_grant_cache_flush. Can we move arch_grant_cache_flush somewhere
else? It is not like page.h is a perfect fit for it anyway.

For instance, can we move it to 

xen/arch/arm/include/asm/grant_table.h

?


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 23:19:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 23:19:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684987.1065307 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdepF-0007kJ-HZ; Fri, 23 Feb 2024 23:19:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684987.1065307; Fri, 23 Feb 2024 23:19:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdepF-0007kC-Ex; Fri, 23 Feb 2024 23:19:29 +0000
Received: by outflank-mailman (input) for mailman id 684987;
 Fri, 23 Feb 2024 23:19:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdepE-0007k6-GZ
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 23:19:28 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fb35dad8-d2a1-11ee-98f5-efadbce2ee36;
 Sat, 24 Feb 2024 00:19:25 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id D58BFCE2DA3;
 Fri, 23 Feb 2024 23:19:19 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4AE2DC433C7;
 Fri, 23 Feb 2024 23:19:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb35dad8-d2a1-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708730359;
	bh=31gof/eXW9PGdivtf3skAXmx8VLQQWBRudDGTL0QyGo=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=uXMEnjPeVDU1gOEWBDKg2kotasVjZdHQrx1sJ4L9UaS4VWySq46022t1CJXGkiDNE
	 vRvXSSiOXJRrPuENEibOQAhWWeqWnzMX1BJzBLlgf2oDux2m4L2TbWxzmnH9iB+Ex0
	 tvFkVE+8unyZa21F2Ug1N1e0GhITD5GD+ljsvSWdalqVnYtCoS85i5V5dk0n1E1+h+
	 i7+Kki4d8eTFO6CS00159rar3klcQP/4Zy1cplrwU2ytw7sUIuny6A2/cpd14kkApl
	 fphhmY4hv10Om5QctoWkeeobocbtqQYDmLRtlLFeF9tl4KAEaSSMZfavKDfPXx3opd
	 O+T1KMGlYsvwQ==
Date: Fri, 23 Feb 2024 15:19:16 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Simone Ballarin <simone.ballarin@bugseng.com>
cc: xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>, 
    Shawn Anastasio <sanastasio@raptorengineering.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH v2 2/6] xen/ppc: address violations of MISRA C:2012 Rule
 11.8.
In-Reply-To: <9270a4fe1712cff6a99e60c7862de1c1b2dde3d6.1702982442.git.maria.celeste.cesario@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402231518390.754277@ubuntu-linux-20-04-desktop>
References: <cover.1702982442.git.maria.celeste.cesario@bugseng.com> <9270a4fe1712cff6a99e60c7862de1c1b2dde3d6.1702982442.git.maria.celeste.cesario@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

Shawn,

I am thinking of committing this, if you disagree please say something
in the next couple of days


On Tue, 19 Dec 2023, Simone Ballarin wrote:
> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
> 
> The xen sources contain violations of MISRA C:2012 Rule 11.8 whose
> headline states:
> "A conversion shall not remove any const, volatile or _Atomic qualification
> from the type pointed to by a pointer".
> 
> Fix violation by adding missing const qualifier in cast.
> 
> Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> Adaptation requested by the community to make the code more consistent.
> ---
>  xen/arch/ppc/include/asm/atomic.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/arch/ppc/include/asm/atomic.h b/xen/arch/ppc/include/asm/atomic.h
> index 64168aa3f1..fe778579fb 100644
> --- a/xen/arch/ppc/include/asm/atomic.h
> +++ b/xen/arch/ppc/include/asm/atomic.h
> @@ -16,7 +16,7 @@
>  
>  static inline int atomic_read(const atomic_t *v)
>  {
> -    return *(volatile int *)&v->counter;
> +    return *(const volatile int *)&v->counter;
>  }
>  
>  static inline int _atomic_read(atomic_t v)
> -- 
> 2.40.0
> 


From xen-devel-bounces@lists.xenproject.org Fri Feb 23 23:22:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Fri, 23 Feb 2024 23:22:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684990.1065318 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdesQ-00012T-W4; Fri, 23 Feb 2024 23:22:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684990.1065318; Fri, 23 Feb 2024 23:22:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdesQ-00012M-Sw; Fri, 23 Feb 2024 23:22:46 +0000
Received: by outflank-mailman (input) for mailman id 684990;
 Fri, 23 Feb 2024 23:22:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OEAM=KA=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rdesP-00012G-Lh
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 23:22:45 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 71040861-d2a2-11ee-98f5-efadbce2ee36;
 Sat, 24 Feb 2024 00:22:43 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 34596CE2F27;
 Fri, 23 Feb 2024 23:22:41 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id D8B08C433C7;
 Fri, 23 Feb 2024 23:22:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 71040861-d2a2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708730560;
	bh=YE1QDpgyw5X4rwqnczvC39zy2K4+xIdimH9zq6pXTc8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=VMfQQsqr9XgiRWnIcMdNPvJzmO102BqmCgG+JW7FybR7nzb/Hz55Z8mED5idfNKvJ
	 aXbDVHC+1J1T3iWVgMiVm6Xb8JeM3ckNf3x4KK70sK23JMPjOFEmfj2ivTIEAeX09K
	 mQkUn63wwPx81/92C5Mv9yaR9Hb6uaNJADvGAMT8GvXjakwE+tDpryIpDO3CCZBcxm
	 CYgfwKHn2D1G3NfxTUZec0b/ehRXkEJtl6Aff9rEutsfQPv5tHA9CbEw2IEp0YPl8C
	 QRrmGcp5mpp5KgtqsXEP7PNvQDXPM4GyEIWinhHJ1LarzkhGWnoE+E0oiaU7+DBbvD
	 ho9RtFCoNhbPw==
Date: Fri, 23 Feb 2024 15:22:38 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Olaf Hering <olaf@aepfle.de>
cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [PATCH v1] automation: remove bin86/dev86 from tumbleweed
 image
In-Reply-To: <20231213202515.30587-1-olaf@aepfle.de>
Message-ID: <alpine.DEB.2.22.394.2402231522020.754277@ubuntu-linux-20-04-desktop>
References: <20231213202515.30587-1-olaf@aepfle.de>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 13 Dec 2023, Olaf Hering wrote:
> https://build.opensuse.org/request/show/1126240
> 
> Signed-off-by: Olaf Hering <olaf@aepfle.de>

Hi Olaf,

Do you have a successful gitlab pipeline with this patch applied that
you can give me as proof of testing and success?


> ---
>  automation/build/suse/opensuse-tumbleweed.dockerfile | 2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/automation/build/suse/opensuse-tumbleweed.dockerfile b/automation/build/suse/opensuse-tumbleweed.dockerfile
> index 38f6fda2ff..f00e03eda7 100644
> --- a/automation/build/suse/opensuse-tumbleweed.dockerfile
> +++ b/automation/build/suse/opensuse-tumbleweed.dockerfile
> @@ -11,13 +11,11 @@ RUN zypper ref && zypper dup -y --no-recommends
>  RUN zypper install -y --no-recommends \
>          acpica \
>          bc \
> -        bin86 \
>          bison \
>          bzip2 \
>          checkpolicy \
>          clang \
>          cmake \
> -        dev86 \
>          diffutils \
>          discount \
>          flex \
> 


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 01:10:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 01:10:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684908.1065328 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgYN-0005ef-GH; Sat, 24 Feb 2024 01:10:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684908.1065328; Sat, 24 Feb 2024 01:10:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgYN-0005eY-C9; Sat, 24 Feb 2024 01:10:11 +0000
Received: by outflank-mailman (input) for mailman id 684908;
 Fri, 23 Feb 2024 16:00:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7Fpr=KA=thalesgroup.com=francois.gourlot@srs-se1.protection.inumbo.net>)
 id 1rdXyQ-0004dh-0M
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 16:00:30 +0000
Received: from esa.hc1631-21.eu.iphmx.com (esa.hc1631-21.eu.iphmx.com
 [23.90.123.40]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a8e37864-d264-11ee-8a57-1f161083a0e0;
 Fri, 23 Feb 2024 17:00:28 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8e37864-d264-11ee-8a57-1f161083a0e0
X-CSE-ConnectionGUID: ns9mFFqhT8efsREh8IYF0A==
X-CSE-MsgGUID: akeB61nrR06bBgRBcUwKNA==
Authentication-Results: ob1.hc1631-21.eu.iphmx.com; dkim=pass (signature verified) header.i=@thalesgroup.com
X-IronPort-AV: E=McAfee;i="6600,9927,10993"; a="10596996"
X-IronPort-AV: E=Sophos;i="6.06,180,1705359600"; 
   d="png'150?scan'150,208,217,150";a="10596996"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=thalesgroup.com; i=@thalesgroup.com; s=bbmfo20230504;
  t=1708704026;
  h=from:to:cc:subject:date:message-id:mime-version;
  bh=HIH+zALnBEStBj4jTN0n0a9LiBx35tTeOyR+6u0KA6M=;
  b=uHwVve4K7HesYakkllLUouQj39amCtpjmot6/rh9bnUOFaVW94C5n5Rp
   jQPksz8mvkGPxh+tZNh23LTaq4z1ZYuLBQo1N/3n5qMoDy1IU2mMrMN0W
   nwfe2ENd2E5p7M0r+NzWaNzO116GhWNV6+dt6TlE+fS8t2BWbyu3X9KeY
   QnBfl0Vph+GtKHrzQ4qiCVS2idx4A/LytGoiUr5srLDR3g13OMQZBARPp
   zerEgDmO+MAJVcRb92o3TB1QAuT9jr8PtX+gc7e0jEMvyxELy12dONrP7
   z1jz4diqPvrpE4LcrQ7yy8sGinwImIvoy7A/LduXWTavwYJAd59jwQvUv
   g==;
X-CSE-ConnectionGUID: WduDQ+O/S+atuga8foMxZQ==
X-CSE-MsgGUID: h9z/sXUhQTabfJYcWFx/WQ==
X-CSE-ConnectionGUID: b3aEZtJ9Q1qoDqsXTBngcw==
X-CSE-MsgGUID: ZIthwiM0Qu6IIVWfBkuCfQ==
X-IronPort-AV: E=McAfee;i="6600,9927,10993"; a="18692764"
X-IronPort-AV: E=Sophos;i="6.06,180,1705359600"; 
   d="png'150?scan'150,208,217,150";a="18692764"
From: GOURLOT Francois <francois.gourlot@thalesgroup.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
CC: Kelly Choi <kelly.choi@cloud.com>
Subject: IMPORTANT - : Need help on USB port virtualization with Xen
 hypervisor
Thread-Topic: IMPORTANT - : Need help on USB port virtualization with Xen
 hypervisor
Thread-Index: AdpmcWIlf+ro0u5NSE2ZDRGqfQbNXA==
Sensitivity: company-confidential
Date: Fri, 23 Feb 2024 16:00:24 +0000
Message-ID: <d2de4ae9ecb34efc962dea7f8b4e7cbd@thalesgroup.com>
Accept-Language: fr-FR, en-US
Content-Language: fr-FR
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
x-ms-exchange-nodisclaimer: 0
Content-Type: multipart/related;
	boundary="_004_d2de4ae9ecb34efc962dea7f8b4e7cbdthalesgroupcom_";
	type="multipart/alternative"
MIME-Version: 1.0

--_004_d2de4ae9ecb34efc962dea7f8b4e7cbdthalesgroupcom_
Content-Type: multipart/alternative;
	boundary="_000_d2de4ae9ecb34efc962dea7f8b4e7cbdthalesgroupcom_"

--_000_d2de4ae9ecb34efc962dea7f8b4e7cbdthalesgroupcom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

RGVhciBBbGwsDQoNCldlIHNlbmQgeW91IGEgbWVzc2FnZSBmZXcgZGF5cyBhZ28uDQoNCldlIGhh
dmUgbWFqb3IgcGVyZm9ybWFuY2Ugd2l0aCBYRU4gVVNCIERyaXZlcnMuIFdlIHVzZSB5b3VyIGRy
aXZlciB0byBsb2FkIGRhdGEgaW4gYSBkZXZpY2UgYW5kIHdlIG5lZWQgVVNCMyBIaWdoIFNwZWVk
IFVTQiBwcm90b2NvbGUNCg0KRG8geW91IGhhdmUgYSBuZXcgaXNzdWUgb2YgdGhlIEhDSSBhbmQg
TkVDIFVTQiBYZW4gRHJpdmVyID8NCg0KV2UgaG9wZSB5b3Ugd2lsbCBjb250YWN0IHVzIHF1aWNr
bHkuDQoNCkJlc3QgcmVnYXJkcw0KDQpQcm9ncmFtIE1hbmFnZXINCg0KVGhhbGVzIFNJWCBHVFMg
RnJhbmNlDQoNClQ6ICszMygwKSAxIDQxIDMwIDI2IDY3DQpNOiArMzMgKDApIDYgODUgMjMgMTIg
MzANCg0KNCBBdmVudWUgZGVzIExvdXZyZXNzZXMNCjkyNjIyIEdlbm5ldmlsbGllcnMNCkZyYW5j
ZQ0KDQoNCg0KDQoNCg0KRGUgOiBMQVJSSUVVIERvbWluaXF1ZSA8ZG9taW5pcXVlLmxhcnJpZXVA
dGhhbGVzZ3JvdXAuY29tPg0KRW52b3nDqSA6IGpldWRpIDIyIGbDqXZyaWVyIDIwMjQgMTM6NTgN
CsOAIDogeGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnDQpDYyA6IFdJTExFTVMgTG91aXMg
PGxvdWlzLndpbGxlbXNAdGhhbGVzZ3JvdXAuY29tPjsgR1JVTyBOaWNvbGFzIDxuaWNvbGFzLW4u
Z3J1b0B0aGFsZXNncm91cC5jb20+OyBHT1VSTE9UIEZyYW5jb2lzIDxmcmFuY29pcy5nb3VybG90
QHRoYWxlc2dyb3VwLmNvbT47IEtlbGx5IENob2kgPGtlbGx5LmNob2lAY2xvdWQuY29tPg0KT2Jq
ZXQgOiBOZWVkIGhlbHAgb24gVVNCIHBvcnQgdmlydHVhbGl6YXRpb24gd2l0aCBYZW4gaHlwZXJ2
aXNvcg0KQ3JpdMOocmUgZGUgZGlmZnVzaW9uIDogQ29uZmlkZW50aWVsDQoNCkRlYXIgYWxsLA0K
DQpXZSBhcmUgZGV0ZWN0aW5nIHNldmVyYWwgaXNzdWVzIHdpdGggVVNCIHBvcnQgdmlydHVhbGl6
YXRpb24gd2l0aCB0aGUgWGVuIGh5cGVydmlzb3IuDQotIFdlIGNhbm5vdCBkbyBQQ0kgcGFzc3Ro
cm91Z2ggb2YgdGhlIFBDSSB1c2IgYnVzIG9uIGEgV2luZG93cyAxMCAxNjA3IDY0LWJpdCB2aXJ0
dWFsIG1hY2hpbmUuIFRoZSBiYWQgcmVzdWx0IGlzIGEgV2luZG93cyBibHVlIHNjcmVlbi4NCi0g
V2hlbiB3ZSB1c2UgdGhlIHBhc3N0aHJvdWdoIGZ1bmN0aW9uYWxpdHkgb24gYSBXaW5kb3dzIDIx
SDIgdmlydHVhbCBtYWNoaW5lLCB3ZSBub3RpY2UgdGhhdCB0aGUgc3BlZWQgb2YgdGhlIFVTQiBw
b3J0IGlzIG5vdCBoaWdoIHNwZWVkIGJ1dCBmdWxsIHNwZWVkIG9uIGEgVVNCIDMuMCBwb3J0DQot
IFdlIG5vdGljZSBpbnN0YWJpbGl0aWVzIHdoZW4gdXNpbmcgdGhlIG5lYy11c2IteGhjaSBkcml2
ZXIsICBVU0IgMi4wIGtleXMgYXJlIG5vdCByZWNvZ25pemVkIGJ5IHRoZSBXaW5kb3dzIHZpcnR1
YWwgbWFjaGluZSAoaW5jb3JyZWN0IGRlc2NyaXB0b3IpDQoNCldlIG5lZWQgeW91ciBoZWxwIHRv
IGZpbmQgYSBzb2x1dGlvbiBmb3IgdGhlc2UgcHJvYmxlbXMuDQoNClRoZSBTb2Z0d2FyZSB1c2Vk
IGFyZSA6DQoNCi0gICAgICAgIERlYmlhbiAxMSB2ZXJzaW9uIDUuMTAuMC0yMA0KDQotICAgICAg
ICBYZW4gdmVyc2lvbiA0LjE0DQoNCi0gICAgICAgIFdpbmRvd3MgMTAgMTYwNyBhbmQgMjFIMiBm
b3IgdmlydHVhbCBtYWNoaW5lcy4gVmlydHVhbCBNYWNoaW5lIEhWTQ0KDQpUaGFua3MgaW4gYWR2
YW5jZSBmb3IgeW91ciBoZWxwLg0KDQpCZXN0IHJlZ2FyZHMsDQoNCg0KDQoNCkRvbWluaXF1ZSBM
QVJSSUVVDQoNCkN5YmVyIFByb2plY3QgTWFuYWdlcg0KDQpUaGFsZXMNCg0KDQo0LCBhdmVudWUg
ZGVzIExvdXZyZXNzZXMNCjkyMjMwIEdlbm5ldmlsbGllcnMsIEZyYW5jZQ0KDQpbY2lkOmltYWdl
MDAxLnBuZ0AwMUQ3RTFERS43Nzg2MkJCMF0NCg0KUmV0cm91dmV6IFRoYWxlcyBzdXIgbGVzIHLD
qXNlYXV4IHNvY2lhdXggZXQgc3VyIHd3dy50aGFsZXNncm91cC5jb208aHR0cHM6Ly93d3cudGhh
bGVzZ3JvdXAuY29tLz4NCg0KDQoNCg0KDQoNCg==

--_000_d2de4ae9ecb34efc962dea7f8b4e7cbdthalesgroupcom_
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64

PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy
bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt
YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj
cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv
VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg
Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv
ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPCEtLVtp
ZiAhbXNvXT48c3R5bGU+dlw6KiB7YmVoYXZpb3I6dXJsKCNkZWZhdWx0I1ZNTCk7fQ0Kb1w6KiB7
YmVoYXZpb3I6dXJsKCNkZWZhdWx0I1ZNTCk7fQ0Kd1w6KiB7YmVoYXZpb3I6dXJsKCNkZWZhdWx0
I1ZNTCk7fQ0KLnNoYXBlIHtiZWhhdmlvcjp1cmwoI2RlZmF1bHQjVk1MKTt9DQo8L3N0eWxlPjwh
W2VuZGlmXS0tPjxzdHlsZT48IS0tDQovKiBGb250IERlZmluaXRpb25zICovDQpAZm9udC1mYWNl
DQoJe2ZvbnQtZmFtaWx5OiJDYW1icmlhIE1hdGgiOw0KCXBhbm9zZS0xOjIgNCA1IDMgNSA0IDYg
MyAyIDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpDYWxpYnJpOw0KCXBhbm9zZS0xOjIg
MTUgNSAyIDIgMiA0IDMgMiA0O30NCi8qIFN0eWxlIERlZmluaXRpb25zICovDQpwLk1zb05vcm1h
bCwgbGkuTXNvTm9ybWFsLCBkaXYuTXNvTm9ybWFsDQoJe21hcmdpbjowY207DQoJbWFyZ2luLWJv
dHRvbTouMDAwMXB0Ow0KCWZvbnQtc2l6ZToxMi4wcHQ7DQoJZm9udC1mYW1pbHk6IlRpbWVzIE5l
dyBSb21hbiIsc2VyaWY7fQ0KYTpsaW5rLCBzcGFuLk1zb0h5cGVybGluaw0KCXttc28tc3R5bGUt
cHJpb3JpdHk6OTk7DQoJY29sb3I6Ymx1ZTsNCgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lO30N
CmE6dmlzaXRlZCwgc3Bhbi5Nc29IeXBlcmxpbmtGb2xsb3dlZA0KCXttc28tc3R5bGUtcHJpb3Jp
dHk6OTk7DQoJY29sb3I6cHVycGxlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQ0KcC5t
c29ub3JtYWwwLCBsaS5tc29ub3JtYWwwLCBkaXYubXNvbm9ybWFsMA0KCXttc28tc3R5bGUtbmFt
ZTptc29ub3JtYWw7DQoJbXNvLW1hcmdpbi10b3AtYWx0OmF1dG87DQoJbWFyZ2luLXJpZ2h0OjBj
bTsNCgltc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0bzsNCgltYXJnaW4tbGVmdDowY207DQoJZm9u
dC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZjt9DQpw
LmdtYWlsLXAxLCBsaS5nbWFpbC1wMSwgZGl2LmdtYWlsLXAxDQoJe21zby1zdHlsZS1uYW1lOmdt
YWlsLXAxOw0KCW1zby1tYXJnaW4tdG9wLWFsdDphdXRvOw0KCW1hcmdpbi1yaWdodDowY207DQoJ
bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87DQoJbWFyZ2luLWxlZnQ6MGNtOw0KCWZvbnQtc2l6
ZToxMi4wcHQ7DQoJZm9udC1mYW1pbHk6IlRpbWVzIE5ldyBSb21hbiIsc2VyaWY7fQ0KcC5tMzg2
MzUwMjE2NDQyOTQ5MTA2bXNvbGlzdHBhcmFncmFwaCwgbGkubTM4NjM1MDIxNjQ0Mjk0OTEwNm1z
b2xpc3RwYXJhZ3JhcGgsIGRpdi5tMzg2MzUwMjE2NDQyOTQ5MTA2bXNvbGlzdHBhcmFncmFwaA0K
CXttc28tc3R5bGUtbmFtZTptXzM4NjM1MDIxNjQ0Mjk0OTEwNm1zb2xpc3RwYXJhZ3JhcGg7DQoJ
bXNvLW1hcmdpbi10b3AtYWx0OmF1dG87DQoJbWFyZ2luLXJpZ2h0OjBjbTsNCgltc28tbWFyZ2lu
LWJvdHRvbS1hbHQ6YXV0bzsNCgltYXJnaW4tbGVmdDowY207DQoJZm9udC1zaXplOjEyLjBwdDsN
Cglmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZjt9DQpzcGFuLkVtYWlsU3R5bGUy
MA0KCXttc28tc3R5bGUtdHlwZTpwZXJzb25hbDsNCglmb250LWZhbWlseToiQ2FsaWJyaSIsc2Fu
cy1zZXJpZjsNCgljb2xvcjojMUY0OTdEO30NCnNwYW4uRW1haWxTdHlsZTIxDQoJe21zby1zdHls
ZS10eXBlOnBlcnNvbmFsLXJlcGx5Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIixzYW5zLXNlcmlm
Ow0KCWNvbG9yOiMxRjQ5N0Q7fQ0KLk1zb0NocERlZmF1bHQNCgl7bXNvLXN0eWxlLXR5cGU6ZXhw
b3J0LW9ubHk7DQoJZm9udC1zaXplOjEwLjBwdDt9DQpAcGFnZSBXb3JkU2VjdGlvbjENCgl7c2l6
ZTo2MTIuMHB0IDc5Mi4wcHQ7DQoJbWFyZ2luOjcwLjg1cHQgNzAuODVwdCA3MC44NXB0IDcwLjg1
cHQ7fQ0KZGl2LldvcmRTZWN0aW9uMQ0KCXtwYWdlOldvcmRTZWN0aW9uMTt9DQotLT48L3N0eWxl
PjwhLS1baWYgZ3RlIG1zbyA5XT48eG1sPg0KPG86c2hhcGVkZWZhdWx0cyB2OmV4dD0iZWRpdCIg
c3BpZG1heD0iMTAyNiIgLz4NCjwveG1sPjwhW2VuZGlmXS0tPjwhLS1baWYgZ3RlIG1zbyA5XT48
eG1sPg0KPG86c2hhcGVsYXlvdXQgdjpleHQ9ImVkaXQiPg0KPG86aWRtYXAgdjpleHQ9ImVkaXQi
IGRhdGE9IjEiIC8+DQo8L286c2hhcGVsYXlvdXQ+PC94bWw+PCFbZW5kaWZdLS0+DQo8L2hlYWQ+
DQo8Ym9keSBsYW5nPSJGUiIgbGluaz0iYmx1ZSIgdmxpbms9InB1cnBsZSI+DQo8ZGl2IGNsYXNz
PSJXb3JkU2VjdGlvbjEiPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMi
IHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDss
c2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEO21zby1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTIj5EZWFy
IEFsbCw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBs
YW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2Fs
aWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6
RU4tVVMiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi
PjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTom
cXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1s
YW5ndWFnZTpFTi1VUyI+V2Ugc2VuZCB5b3UgYSBtZXNzYWdlIGZldyBkYXlzIGFnby48bzpwPjwv
bzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIg
c3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90Oyxz
YW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPjxvOnA+
Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9
IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJp
JnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpFTi1V
UyI+V2UgaGF2ZSBtYWpvciBwZXJmb3JtYW5jZSB3aXRoIFhFTiBVU0IgRHJpdmVycy4gV2UgdXNl
IHlvdXIgZHJpdmVyIHRvIGxvYWQgZGF0YSBpbiBhIGRldmljZSBhbmQgd2UgbmVlZCBVU0IzIEhp
Z2ggU3BlZWQNCiBVU0IgcHJvdG9jb2xlPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2Zv
bnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEO21z
by1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8
cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZTox
MS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMx
RjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPkRvIHlvdSBoYXZlIGEgbmV3IGlzc3Vl
IG9mIHRoZSBIQ0kgYW5kIE5FQyBVU0IgWGVuIERyaXZlciA/PG86cD48L286cD48L3NwYW4+PC9w
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNp
emU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xv
cjojMUY0OTdEO21zby1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTIj48bzpwPiZuYnNwOzwvbzpwPjwv
c3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9
ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNl
cmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPldlIGhvcGUgeW91
IHdpbGwgY29udGFjdCB1cyBxdWlja2x5LjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNz
PSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtm
b250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RDtt
c28tZmFyZWFzdC1sYW5ndWFnZTpFTi1VUyI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0K
PHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6
MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjoj
MUY0OTdEO21zby1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTIj5CZXN0IHJlZ2FyZHM8bzpwPjwvbzpw
Pjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5
bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5z
LXNlcmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPjxvOnA+Jm5i
c3A7PC9vOnA+PC9zcGFuPjwvcD4NCjx0YWJsZSBjbGFzcz0iTXNvTm9ybWFsVGFibGUiIGJvcmRl
cj0iMCIgY2VsbHNwYWNpbmc9IjAiIGNlbGxwYWRkaW5nPSIwIiBzdHlsZT0iYm9yZGVyLWNvbGxh
cHNlOmNvbGxhcHNlIj4NCjx0Ym9keT4NCjx0cj4NCjx0ZCB3aWR0aD0iNDcyIiB2YWxpZ249InRv
cCIgc3R5bGU9IndpZHRoOjM2MC4wcHQ7cGFkZGluZzozLjBwdCAwY20gMS41cHQgMGNtIj4NCjxw
IGNsYXNzPSJNc29Ob3JtYWwiPjxiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQt
ZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6Izc5Nzk3OSI+UHJvZ3Jh
bSBNYW5hZ2VyPG86cD48L286cD48L3NwYW4+PC9iPjwvcD4NCjwvdGQ+DQo8L3RyPg0KPHRyPg0K
PHRkIHdpZHRoPSI0NzIiIHZhbGlnbj0idG9wIiBzdHlsZT0id2lkdGg6MzYwLjBwdDtwYWRkaW5n
Oi43NXB0IDBjbSAuNzVwdCAwY20iPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PGI+PHNwYW4gc3R5
bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssc2Fucy1z
ZXJpZjtjb2xvcjojNzk3OTc5Ij5UaGFsZXMgU0lYIEdUUyBGcmFuY2U8bzpwPjwvbzpwPjwvc3Bh
bj48L2I+PC9wPg0KPC90ZD4NCjwvdHI+DQo8dHIgc3R5bGU9ImhlaWdodDoyLjVwdCI+DQo8dGQg
d2lkdGg9IjQ3MiIgdmFsaWduPSJ0b3AiIHN0eWxlPSJ3aWR0aDozNjAuMHB0O3BhZGRpbmc6Ljc1
cHQgMGNtIDcuNXB0IDBjbTtoZWlnaHQ6Mi41cHQiPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNw
YW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDss
c2Fucy1zZXJpZjtjb2xvcjojNzk3OTc5Ij5UOiAmIzQzOzMzKDApIDEgNDEgMzAgMjYgNjc8bzpw
PjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9u
dC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OyxzYW5zLXNlcmlmO2Nv
bG9yOiM3OTc5NzkiPk06ICYjNDM7MzMgKDApIDYgODUgMjMgMTIgMzA8bzpwPjwvbzpwPjwvc3Bh
bj48L3A+DQo8L3RkPg0KPC90cj4NCjx0ciBzdHlsZT0iaGVpZ2h0OjMuMHB0Ij4NCjx0ZCB3aWR0
aD0iNDcyIiB2YWxpZ249InRvcCIgc3R5bGU9IndpZHRoOjM2MC4wcHQ7Ym9yZGVyOm5vbmU7Ym9y
ZGVyLXRvcDpzb2xpZCAjMDBCQkREIDIuMjVwdDtwYWRkaW5nOjcuNXB0IDBjbSAzLjc1cHQgMGNt
O2hlaWdodDozLjBwdCI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1z
aXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OyxzYW5zLXNlcmlmO2NvbG9y
OiM3OTc5NzkiPjQgQXZlbnVlIGRlcyBMb3V2cmVzc2VzPG86cD48L286cD48L3NwYW4+PC9wPg0K
PHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1m
YW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojNzk3OTc5Ij45MjYyMiBH
ZW5uZXZpbGxpZXJzPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+
PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVv
dDssc2Fucy1zZXJpZjtjb2xvcjojNzk3OTc5Ij5GcmFuY2U8bzpwPjwvbzpwPjwvc3Bhbj48L3A+
DQo8L3RkPg0KPC90cj4NCjwvdGJvZHk+DQo8L3RhYmxlPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+
PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZx
dW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMi
PjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFu
IHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDss
c2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEO21zby1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTIj48bzpw
PiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHls
ZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMt
c2VyaWY7Y29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpFTi1VUyI+PG86cD4mbmJz
cDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZv
bnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlm
O2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPjxvOnA+Jm5ic3A7PC9v
OnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNp
emU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xv
cjojMUY0OTdEO21zby1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTIj48bzpwPjwvbzpwPjwvc3Bhbj48
L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtm
b250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RDtt
c28tZmFyZWFzdC1sYW5ndWFnZTpFTi1VUyI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0K
PGRpdj4NCjxkaXYgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci10b3A6c29saWQgI0UxRTFFMSAx
LjBwdDtwYWRkaW5nOjMuMHB0IDBjbSAwY20gMGNtIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxi
PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkm
cXVvdDssc2Fucy1zZXJpZiI+RGUmbmJzcDs6PC9zcGFuPjwvYj48c3BhbiBzdHlsZT0iZm9udC1z
aXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPiBM
QVJSSUVVIERvbWluaXF1ZSAmbHQ7ZG9taW5pcXVlLmxhcnJpZXVAdGhhbGVzZ3JvdXAuY29tJmd0
Ow0KPGJyPg0KPGI+RW52b3nDqSZuYnNwOzo8L2I+IGpldWRpIDIyIGbDqXZyaWVyIDIwMjQgMTM6
NTg8YnI+DQo8Yj7DgCZuYnNwOzo8L2I+IHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZzxi
cj4NCjxiPkNjJm5ic3A7OjwvYj4gV0lMTEVNUyBMb3VpcyAmbHQ7bG91aXMud2lsbGVtc0B0aGFs
ZXNncm91cC5jb20mZ3Q7OyBHUlVPIE5pY29sYXMgJmx0O25pY29sYXMtbi5ncnVvQHRoYWxlc2dy
b3VwLmNvbSZndDs7IEdPVVJMT1QgRnJhbmNvaXMgJmx0O2ZyYW5jb2lzLmdvdXJsb3RAdGhhbGVz
Z3JvdXAuY29tJmd0OzsgS2VsbHkgQ2hvaSAmbHQ7a2VsbHkuY2hvaUBjbG91ZC5jb20mZ3Q7PGJy
Pg0KPGI+T2JqZXQmbmJzcDs6PC9iPiBOZWVkIGhlbHAgb24gVVNCIHBvcnQgdmlydHVhbGl6YXRp
b24gd2l0aCBYZW4gaHlwZXJ2aXNvcjxicj4NCjxiPkNyaXTDqHJlIGRlIGRpZmZ1c2lvbiZuYnNw
Ozo8L2I+IENvbmZpZGVudGllbDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0KPC9kaXY+
DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxkaXY+DQo8Ymxv
Y2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBw
dDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi10b3A6
NS4wcHQ7bWFyZ2luLXJpZ2h0OjBjbTttYXJnaW4tYm90dG9tOjUuMHB0Ij4NCjxkaXY+DQo8ZGl2
Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6
YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+PHNwYW4gc3R5bGU9ImNvbG9yOiMxRjQ5
N0QiPkRlYXIgYWxsLDwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi
IHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0
byI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+
PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10
b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFuIGxhbmc9IkVOLVVT
IiBzdHlsZT0iY29sb3I6IzJGNTQ5NiI+V2UgYXJlIGRldGVjdGluZzwvc3Bhbj48c3BhbiBsYW5n
PSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMwMEIwNTAiPg0KPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVT
IiBzdHlsZT0iY29sb3I6IzJGNTQ5NiI+c2V2ZXJhbCBpc3N1ZXMgd2l0aCBVU0IgcG9ydCB2aXJ0
dWFsaXphdGlvbiB3aXRoDQo8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjoj
MUY0OTdEIj50aGUgWGVuIGh5cGVydmlzb3I8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxl
PSJjb2xvcjojMkY1NDk2Ij4uPC9zcGFuPjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFs
dDphdXRvO21hcmdpbi1sZWZ0OjM1LjRwdCI+DQo8c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNv
bG9yOiMyRjU0OTYiPi0gV2UgY2Fubm90IGRvIFBDSSBwYXNzdGhyb3VnaCBvZiB0aGUgUENJIHVz
YiBidXMgb24gYSBXaW5kb3dzIDEwIDE2MDcgNjQtYml0IHZpcnR1YWwgbWFjaGluZTwvc3Bhbj48
c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPi4NCjwvc3Bhbj48c3BhbiBs
YW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMyRjU0OTYiPlRoZSBiYWQgcmVzdWx0IGlzIGEgV2lu
ZG93cyBibHVlIHNjcmVlbi48L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9y
bWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0
OmF1dG87bWFyZ2luLWxlZnQ6MzUuNHB0Ij4NCjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29s
b3I6IzJGNTQ5NiI+LSBXaGVuIHdlIHVzZSB0aGUgcGFzc3Rocm91Z2ggZnVuY3Rpb25hbGl0eSBv
biBhIFdpbmRvd3MgMjFIMiB2aXJ0dWFsIG1hY2hpbmUsIHdlIG5vdGljZSB0aGF0PC9zcGFuPjxz
cGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzAwQjA1MCI+DQo8L3NwYW4+PHNwYW4gbGFu
Zz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMkY1NDk2Ij50aGUgc3BlZWQgb2YgdGhlIFVTQiBwb3J0
IGlzIG5vdCBoaWdoIHNwZWVkIGJ1dCBmdWxsIHNwZWVkIG9uIGEgVVNCIDMuMCBwb3J0PC9zcGFu
PjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4t
dG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvO21hcmdpbi1sZWZ0OjM1LjRw
dCI+DQo8c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMyRjU0OTYiPi0gV2Ugbm90aWNl
IGluc3RhYmlsaXRpZXMgd2hlbiB1c2luZyB0aGUgbmVjLXVzYi14aGNpIGRyaXZlciwgJm5ic3A7
VVNCIDIuMCBrZXlzIGFyZSBub3QgcmVjb2duaXplZCBieSB0aGUgV2luZG93cyB2aXJ0dWFsIG1h
Y2hpbmUgKGluY29ycmVjdCBkZXNjcmlwdG9yKTwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNs
YXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2lu
LWJvdHRvbS1hbHQ6YXV0byI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdE
Ij4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHls
ZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxz
cGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzJGNTQ5NiI+V2UNCjwvc3Bhbj48c3BhbiBs
YW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPm5lZWQ8L3NwYW4+PHNwYW4gbGFuZz0i
RU4tVVMiIHN0eWxlPSJjb2xvcjojMDBCMDUwIj4NCjwvc3Bhbj48c3BhbiBsYW5nPSJFTi1VUyIg
c3R5bGU9ImNvbG9yOiMyRjU0OTYiPnlvdXIgPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHls
ZT0iY29sb3I6IzFGNDk3RCI+aGVscDwvc3Bhbj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNv
bG9yOiMyRjU0OTYiPiB0byBmaW5kIGEgc29sdXRpb24gZm9yIHRoZXNlIHByb2JsZW1zLjwvc3Bh
bj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2lu
LXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+PHNwYW4gbGFuZz0iRU4t
VVMiIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+DQo8
cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1h
cmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFG
NDk3RCI+VGhlIFNvZnR3YXJlIHVzZWQgYXJlIDoNCjwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxw
IGNsYXNzPSJtMzg2MzUwMjE2NDQyOTQ5MTA2bXNvbGlzdHBhcmFncmFwaCI+PHNwYW4gbGFuZz0i
RU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4tPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVTIiBz
dHlsZT0iZm9udC1zaXplOjcuMHB0O2NvbG9yOiMxRjQ5N0QiPiZuYnNwOyZuYnNwOyZuYnNwOyZu
YnNwOyZuYnNwOyZuYnNwOyZuYnNwOw0KPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0i
Y29sb3I6IzJGNTQ5NiI+RGViaTwvc3Bhbj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9y
OiMxRjQ5N0QiPmE8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMkY1NDk2
Ij5uIDExIHZlcnNpb248L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0
OTdEIj4NCjwvc3Bhbj48c3BhbiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+NS4xMC4wLTIwPC9zcGFu
PjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Im0zODYzNTAyMTY0NDI5NDkxMDZtc29saXN0cGFy
YWdyYXBoIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPi08L3NwYW4+
PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6Ny4wcHQ7Y29sb3I6IzFGNDk3RCI+
Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7DQo8L3NwYW4+PHNwYW4g
bGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMkY1NDk2Ij5YZW4gdmVyc2lvbjwvc3Bhbj48c3Bh
biBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPg0KPC9zcGFuPjxzcGFuIHN0eWxl
PSJjb2xvcjojMUY0OTdEIj40LjE0PC9zcGFuPjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Im0z
ODYzNTAyMTY0NDI5NDkxMDZtc29saXN0cGFyYWdyYXBoIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5
bGU9ImNvbG9yOiMxRjQ5N0QiPi08L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250
LXNpemU6Ny4wcHQ7Y29sb3I6IzFGNDk3RCI+Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7
Jm5ic3A7Jm5ic3A7DQo8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMkY1
NDk2Ij5XaW5kb3dzIDEwIDE2MDcgYW5kIDIxSDIgZm9yIHZpcnR1YWwgbWFjaGluZXMuIFZpcnR1
YWwgTWFjaGluZSBIVk08L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
IiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1
dG8iPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+Jm5ic3A7PC9zcGFu
PjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4t
dG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj48c3BhbiBsYW5nPSJFTi1V
UyIgc3R5bGU9ImNvbG9yOiMyRjU0OTYiPlRoYW5rcyBpbiBhZHZhbmNlPC9zcGFuPjxzcGFuIGxh
bmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+IGZvciB5b3VyIGhlbHAuPC9zcGFuPjxv
OnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9w
LWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj48c3BhbiBsYW5nPSJFTi1VUyIg
c3R5bGU9ImNvbG9yOiMxRjQ5N0QiPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNs
YXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2lu
LWJvdHRvbS1hbHQ6YXV0byI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdE
Ij5CZXN0IHJlZ2FyZHMsPC9zcGFuPjxvOnA+PC9vOnA+PC9wPg0KPHRhYmxlIGNsYXNzPSJNc29O
b3JtYWxUYWJsZSIgYm9yZGVyPSIwIiBjZWxsc3BhY2luZz0iMCIgY2VsbHBhZGRpbmc9IjAiIHN0
eWxlPSJib3JkZXItY29sbGFwc2U6Y29sbGFwc2UiPg0KPHRib2R5Pg0KPHRyPg0KPHRkIHdpZHRo
PSI0NjMiIHZhbGlnbj0idG9wIiBzdHlsZT0id2lkdGg6MzYwLjBwdDtib3JkZXI6bm9uZTtib3Jk
ZXItYm90dG9tOnNvbGlkICNCRkJGQkYgMS4wcHQ7cGFkZGluZzozLjBwdCAwY20gMS41cHQgMGNt
Ij4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt
c28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+PGI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMC4w
cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7QXJpYWwmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMjQyQTc1
Ij4mbmJzcDs8L3NwYW4+PC9iPjxvOnA+PC9vOnA+PC9wPg0KPC90ZD4NCjwvdHI+DQo8dHI+DQo8
dGQgd2lkdGg9IjQ2MyIgdmFsaWduPSJ0b3AiIHN0eWxlPSJ3aWR0aDozNjAuMHB0O3BhZGRpbmc6
My4wcHQgMGNtIDEuNXB0IDBjbSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1h
cmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxiPjxzcGFuIHN0
eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMt
c2VyaWY7Y29sb3I6IzI0MkE3NSI+Jm5ic3A7PC9zcGFuPjwvYj48bzpwPjwvbzpwPjwvcD4NCjwv
dGQ+DQo8L3RyPg0KPHRyPg0KPHRkIHdpZHRoPSI0NjMiIHZhbGlnbj0idG9wIiBzdHlsZT0id2lk
dGg6MzYwLjBwdDtwYWRkaW5nOjMuMHB0IDBjbSAxLjVwdCAwY20iPg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFs
dDphdXRvIj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVv
dDtBcmlhbCZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMyNDJBNzUiPkRvbWluaXF1ZSBMQVJSSUVV
DQo8L3NwYW4+PC9iPjxvOnA+PC9vOnA+PC9wPg0KPC90ZD4NCjwvdHI+DQo8dHI+DQo8dGQgd2lk
dGg9IjQ2MyIgdmFsaWduPSJ0b3AiIHN0eWxlPSJ3aWR0aDozNjAuMHB0O3BhZGRpbmc6Ljc1cHQg
MGNtIC43NXB0IDBjbSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10
b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxiPjxzcGFuIHN0eWxlPSJm
b250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMtc2VyaWY7
Y29sb3I6Izc5Nzk3OSI+Q3liZXIgUHJvamVjdCBNYW5hZ2VyPC9zcGFuPjwvYj48bzpwPjwvbzpw
PjwvcD4NCjwvdGQ+DQo8L3RyPg0KPHRyIHN0eWxlPSJoZWlnaHQ6MTcuNTVwdCI+DQo8dGQgd2lk
dGg9IjQ2MyIgdmFsaWduPSJ0b3AiIHN0eWxlPSJ3aWR0aDozNjAuMHB0O3BhZGRpbmc6Ljc1cHQg
MGNtIDcuNXB0IDBjbTtoZWlnaHQ6MTcuNTVwdCI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHls
ZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxi
PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1
b3Q7LHNhbnMtc2VyaWY7Y29sb3I6Izc5Nzk3OSI+VGhhbGVzPC9zcGFuPjwvYj48bzpwPjwvbzpw
PjwvcD4NCjwvdGQ+DQo8L3RyPg0KPHRyIHN0eWxlPSJoZWlnaHQ6Ljg1cHQiPg0KPHRkIHdpZHRo
PSI0NjMiIHZhbGlnbj0idG9wIiBzdHlsZT0id2lkdGg6MzYwLjBwdDtib3JkZXI6bm9uZTtib3Jk
ZXItdG9wOnNvbGlkICMwMEJCREQgMi4yNXB0O3BhZGRpbmc6Ny41cHQgMGNtIDMuNzVwdCAwY207
aGVpZ2h0Oi44NXB0Ij4NCjwvdGQ+DQo8L3RyPg0KPHRyIHN0eWxlPSJoZWlnaHQ6My4wcHQiPg0K
PHRkIHdpZHRoPSI0NjMiIHZhbGlnbj0idG9wIiBzdHlsZT0id2lkdGg6MzYwLjBwdDtwYWRkaW5n
OjMuNzVwdCAwY20gNy41cHQgMGNtO2hlaWdodDozLjBwdCI+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
IiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1
dG8iPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFs
JnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6Izc5Nzk3OSI+NCwgYXZlbnVlIGRlcyBMb3V2cmVzc2Vz
DQo8L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNv
LW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFuIHN0
eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMt
c2VyaWY7Y29sb3I6Izc5Nzk3OSI+OTIyMzAgR2VubmV2aWxsaWVycywgRnJhbmNlPC9zcGFuPjxv
OnA+PC9vOnA+PC9wPg0KPC90ZD4NCjwvdHI+DQo8dHI+DQo8dGQgd2lkdGg9IjQ2MyIgdmFsaWdu
PSJ0b3AiIHN0eWxlPSJ3aWR0aDozNjAuMHB0O2JvcmRlcjpub25lO2JvcmRlci10b3A6c29saWQg
IzAwQkJERCAyLjI1cHQ7cGFkZGluZzoxMS4yNXB0IDBjbSAzLjBwdCAwY20iPg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90
dG9tLWFsdDphdXRvIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTom
cXVvdDtBcmlhbCZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiM3OTc2NzYiPjxpbWcgd2lkdGg9IjE2
MCIgaGVpZ2h0PSIzNiIgc3R5bGU9IndpZHRoOjEuNjY2NmluO2hlaWdodDouMzc1aW4iIGlkPSJt
XzM4NjM1MDIxNjQ0Mjk0OTEwNkltYWdlX3gwMDIwXzEiIHNyYz0iY2lkOmltYWdlMDAxLnBuZ0Aw
MURBNjY3OS41ODQ0QTJBMCIgYWx0PSJjaWQ6aW1hZ2UwMDEucG5nQDAxRDdFMURFLjc3ODYyQkIw
Ij48L3NwYW4+PG86cD48L286cD48L3A+DQo8L3RkPg0KPC90cj4NCjx0cj4NCjx0ZCB3aWR0aD0i
NDYzIiB2YWxpZ249InRvcCIgc3R5bGU9IndpZHRoOjM2MC4wcHQ7cGFkZGluZzo1LjI1cHQgMGNt
IDUuMjVwdCAwY20iPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9w
LWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj48Yj48c3BhbiBzdHlsZT0iZm9u
dC1zaXplOjkuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMtc2VyaWY7Y29s
b3I6Izc5NzY3NiI+UmV0cm91dmV6IFRoYWxlcyBzdXIgbGVzIHLDqXNlYXV4IHNvY2lhdXggZXQg
c3VyDQo8L3NwYW4+PC9iPjxzcGFuIHN0eWxlPSJmb250LXNpemU6OS4wcHQ7Zm9udC1mYW1pbHk6
JnF1b3Q7QXJpYWwmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojNzk3Njc2Ij48YSBocmVmPSJodHRw
czovL3d3dy50aGFsZXNncm91cC5jb20vIiB0YXJnZXQ9Il9ibGFuayI+d3d3LnRoYWxlc2dyb3Vw
LmNvbTwvYT48L3NwYW4+PG86cD48L286cD48L3A+DQo8L3RkPg0KPC90cj4NCjx0cj4NCjx0ZCB3
aWR0aD0iNDYzIiB2YWxpZ249InRvcCIgc3R5bGU9IndpZHRoOjM2MC4wcHQ7cGFkZGluZzo1LjI1
cHQgMGNtIDUuMjVwdCAwY20iPg0KPC90ZD4NCjwvdHI+DQo8dHI+DQo8dGQgd2lkdGg9IjQ2MyIg
dmFsaWduPSJ0b3AiIHN0eWxlPSJ3aWR0aDozNjAuMHB0O3BhZGRpbmc6MTAuNXB0IDBjbSAxMC41
cHQgMGNtIj4NCjwvdGQ+DQo8L3RyPg0KPC90Ym9keT4NCjwvdGFibGU+DQo8cCBjbGFzcz0iTXNv
Tm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20t
YWx0OmF1dG8iPjxzcGFuIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48
L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0
OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFuIHN0eWxlPSJjb2xvcjojMUY0
OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBz
dHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8i
PjxzcGFuIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+
DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvZGl2Pg0K
PC9ib2R5Pg0KPC9odG1sPg0K

--_000_d2de4ae9ecb34efc962dea7f8b4e7cbdthalesgroupcom_--

--_004_d2de4ae9ecb34efc962dea7f8b4e7cbdthalesgroupcom_
Content-Type: image/png; name="image001.png"
Content-Description: image001.png
Content-Disposition: inline; filename="image001.png"; size=3376;
	creation-date="Fri, 23 Feb 2024 16:00:24 GMT";
	modification-date="Fri, 23 Feb 2024 16:00:24 GMT"
Content-ID: <image001.png@01DA6679.5844A2A0>
Content-Transfer-Encoding: base64

iVBORw0KGgoAAAANSUhEUgAAAKAAAAAkCAYAAAAO7jHjAAAACXBIWXMAAAsTAAALEwEAmpwYAAAA
AXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAzFSURBVHgB7VzNUhtJEs6sbmHHXtA+AfJtbP6a
094W6QlgrhMzg4jYu2WIiTkCx401RjwBkj2xsTeLJ5DY80bQAmHPzc0TWOxsbNi0unIzq7tF6w8k
BnYYS1+oQ1L9V1dWVlZmVuEz5wfHgsAhDTOEkEHANIB8g3wD8W/oA473ovjomzwk8FBhPQDlvXP/
5vbL5ziFtAZrLSByuC6pJ4MIxRN3dx+GxLyzucoV7kV1NxVCTYF96Lp/9eAWcJwfMz75W30jUdXP
3JdFuCVmFzcOOgIImmcnr17AHSEeP9CwqBHS8TsNqxpq7JpIJOPmaQjqDbdYgyEg4xiAleWyZpJj
yVFpimhnUH1A5AqdWGC7tiJ9TJIk/LRBNzQg0blMWElYAJF8BR4HPemXLwjUKikoYlSb1EOa0jAK
AkqTwkxctyZY1eRLGTtwC7SotcftWe0byR2acwrusAPTDS4331Ecksdfd0aA/cZv1LEj5gBkMlnw
dGkz8/5493xQXocJvkV6r0WQjetKjuUw9QHiqtTXghYzjjHHnLOZ5a/VaxORtQUTmJUiIF3ln1m4
I9gw5mCx4aB75hpxAjCTCMoyF8zelgt+KQjAz1IoosVooiyptwS/96bdqO/idYnmFja2mGVud2Qk
yJ+d7Jbhd455ZyPPfckkw5j4SohY5h/VjsQhF6zBA4dMnrP6qydwD5B9AnRSi3ta383Br8B4L8GE
PUtrgHqn4e7WoJfYDBeECe4UY0uAhvtBL/f72S165g/2bmiQrAOY4E7xIGRAXvIcIYihMxAuw6/F
AO4X/xYuOLe4WYOEwC0EO+tsFM7cV7dWy9w3RI3Wo/rpl47QbZwMr/oaAIffUXWUDLG6zlapsqjN
HgQBkuxCCVfh/wQhon6yX5v7xRAuSJ07Ph64LdaBlVy3eGvh+56R7lb99AUaEWMkAmTCOadOGVA2
JFkYAbG6jlVfBdZh5sZuCRZVgiJ83h2e5H4xBsiC6ZZWz2EMYVm6IhwM7gZp1mHujR0BsvJzoOyX
rx6n89VGPnzeh2n6yIIsMxTEEgBjBuH6PFFl11uDu4HzMJZgoqL2cWi5ykrxks3mOxgRwv2Y9feV
/YTgNIiSlTJRKHxXbbx4k5srdsuCcMUFb2V5uU/cpxpGEIkpRvXy9OnmzCh5bZtybMFKyqfph7EJ
YYXk+/eDzT/dmJ/fuBDz0agI+hBfzP3+hH+pQpdcyDVsMVcs/evFTz2yYMgFfyzf1v78JWCUMROw
1cnrtteNzRIs3I/7nu8Oj2U/or7CdFpbU8uDZMF+BD3BaBgbU9x13C/6K7vadG8aDHe7fXbEQtCs
nC6PaqL7yilkYET07NC/EIwFAUayX747PLnzJQh2EKy9riTumz8/O5If/fSCYcbRTHRiY7bJ+gAj
4iYvlWT584ubI5cfI8Dg63dusa8r3YKz+ZxXigLcFtTpMcO/3bEgwCG4H7zJLRa/q56wcV02F5jW
AKX/wGWHngw1FklRtquoB+eoMMgPcBhYAU4PitPiNoeYgTsCElW+eAIUdytxnugO76f3e5NbKPFX
aVBZp6cvD9nK0O0p87txVHhAqPGYlM5OXpVvJMCAsK6QSh1hWo+0+0nCt3TN1mqbZ5IDRmsPmbac
NTTUR2R1A4Vym8dqHI+UdTgotah58Kq+NAdUfq7fTqYiDTtKgeGo4uEbmpbQHZQetcoSaofTZZJt
GMZbOS4bSXvv63vnQ5SficrPRGXcVEf7/fGk8qSeXyyqwwC0lC6nAnRJ+pEYvwF1GVetq34AP+Ry
+Ufew7UiTTDBBBNMMMEEE0wwwQQTDEDoOXI8tEdIlD4DE0yQgP199bTCG/OVRFgTlSqVl59de3ZV
Q0o8YTP8/PH7akPOpnpvcrNfJ9N8Wz1ZVaDetgCWSMEaqzAK31QbS3/Pzblwjwg9W4JjMOoOrHS3
qzOtTKJUpnTPbXroMFYOHkMLWkctsg4a9d2OdybuZ3fphBuXp1h3JJrvJijcNo/obrQufFs9zV5X
AIHeBxWbZeSoXu/hcrYqmDDLUtO+hrKU/1/wPbhnaKXXQPR9ShXQGqxYFhCm3rLV4y2MOVgx7Ojw
ggCetOR0xzNRVtkcOJL71SCwqfDA15ZherEiuvl6edZYBr6rNs5ZgXigFM4IEVqIa5/J3/lHbsnj
uLxCWC5n59ajqxiEeDuOZ+arDYcQjMewTrg3KdAZJJWRHgpntFBdcV1krqut/VLuqWfyM7dkyuC8
1Azj/J1SbqkZln+cZqJp22xZKeopfbkfx5uJo8m497PiMxvooMxcfhsVTpeX5wxXX6s1Dgitmtb+
BYX1pCUMyX8hZUvc6+Wn5bgvPsH+FE8cDXaBJ+xMWLbPZdkcj2vdfYjbNrew+dxWQVlmujnUDS3n
1N2tmEFwNvIBWK5cYSLcJxp8iM9KJN+pOTyvKToHg005y2EOVckxSfNy9ZGYAqUODa2VuCweQ09B
cJjkXMYurv21KL45zJUmchWKHGGwAypwWw5tCFy5XkXqkSs2IoV27EFu0sfXsyTbGV/bQqQdREhz
uovYHavtCcxEZYjnUvtHQmRixkpZj0wBzC6XY7MWym/de6OABnxL5qqMq/tlws5ai5LXmno0baG1
Ir8ljTyyNBMGW+38WuIoLbNS4gLr0WJHHVE+Q+CatrWaarvIo2VRIh30a6vUjRRko3/pZNruOPlv
K1hsgcVhuGU4hZlYj9PJvpo2G4fWBBBWWfww3CTQvhjyD67agM+toDVtCJHrULz8yTRtUes4WYQQ
THhGWYhFrBWSTgqkj2EetiKR9Vbs0S1oyTvZlnB5+He++/jAJ/gMifgs13fjAaaYwGSJ1qCaLc3E
x/2WMgiCppkc7QliEq6g1otCiJxuK65P0mrwpSy2kFBT/rcJkF/mljzSZ66w9hgew6gI3dgpw5Ra
fJOby9ENh14kjTwQvozMMPmF01ngf83PujwSxpwu0y5TvFcUGi4zNYWFn3ILlUH1SxzPRJH9mlzf
esxF+yEWJ8hShbDNrQyYZR6L0g4m8oq0PbkxE9MTaMtMHjFfiWkqcbbYOT3dO+KIFR6w4snJbrlx
8tKsQsmlTrgmf9WE60kasUdLOOc9DPOYk20uacvkkYGVcHlQUzkymbUhDhhxvI3BC4KrdzcIwtmk
7YEFlfDSKRJzZs3Uf40TBhlzHE8cZR/FaaV+YwokrMn/eAn2XudmjRu3LGFMrVWtaIXZ0QX8Jhh8
8EWWb56F0awVGzLBbwrhwJDajm73AeHwEF9XQVAnRcsicLeEmzN3Ygpe5GVM4mphEkizWLM9t7iZ
l/89V12Yi5hC222MyL2szblCe/pAdMjm0XIqjKbZoqgF9wTjwrawWfLBr3L/PBvt9e7lvscZIfb0
QJbZuGn18Pdot1eNmr4nP2EWumTLGJZSspymX+fmjE8+78CHeIOGUI3a6Doul0g/RPsjBwreWMXy
c09bLV3TpLZ4+TZcjNe+Osjga/OOzK5buIE2niGDrzrBrvZEXBEa0bUYItTDkGDie87LY0G4L0+M
jE+qCveIiEPvzy5sFIQQoevWtJgAM90DyUtGzQI61/InoMpa7Uy8Jq4dGBHAeaNS4xdc4PSrN6Xv
l5/b4XK+PH+v9ktDLMSKpMjxH3nZaxLdTH8ky6zIapD6wO3qyiNyFGU5/IOdwuzlpa7xKK3K/5BB
0IC2zrncBk84IKfNh0VRJd7oCFj493jmpw3REVR4yXN9UOK1wrIRlqLiXeGAvBGR3eeFyFmxMC+Q
o5C8A92TO3pQwbkQfiSTOaHqhPMQhAQ+BIyHCmsHWPZ8wivJjLzLYWGRXnecHyotHXSE8wQ6txRu
cXtEtruINpDmKrdA7i40HcVFneDuLPqs8kpwzhsb2uENQuVKjcIdxCtP4G+qx0u2mloxw4CqGc98
k4/VK2HLoBDfEcdUvp4SgR1ZJonS/xJ8qv8BUhe2srzg8tNFAHaZ5YLaVc+u8l+yfCf5UVkzpEk2
MvlkZ18vP9v//p9nnvQvbJPl+TrocCFqaTqM6wrzzO5wHrdfns90uTOlpjxxCQsuP1/4AOtT6hH3
N3ofnPZSfzLvYko9zvvBp7ZbFLc1d5UWm1rrHlcmftHr3I/FltKHLPM0n81v7HDYdMzxzI52/oWn
Y/cmwo/J/LKD/copLNmkRI86I/GN05c1zpOP87AgX/y3Cg6lwYG2C3FecX1LBXZHeT7LfbY2m8AZ
kRcVT/aQPqAcYOCl5MJPrXq8njlfztZq7VJ2DV2ub+9OX5W4PR+v2kOlQAVHTHzAdU1HtOVy2E5U
1o6UxRuxmdGPlt0zkioa2WHyV5oH+omogWCCLw4P8FQccxIK1SL8qfCuMzshvi8X/wPlgS2kdDEs
VgAAAABJRU5ErkJggg==

--_004_d2de4ae9ecb34efc962dea7f8b4e7cbdthalesgroupcom_--


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 01:10:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 01:10:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685002.1065353 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgYO-00061Q-ML; Sat, 24 Feb 2024 01:10:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685002.1065353; Sat, 24 Feb 2024 01:10:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgYO-0005zc-EN; Sat, 24 Feb 2024 01:10:12 +0000
Received: by outflank-mailman (input) for mailman id 685002;
 Fri, 23 Feb 2024 23:57:47 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nFrk=KA=gmail.com=alexei.starovoitov@srs-se1.protection.inumbo.net>)
 id 1rdfQJ-0005zS-Gf
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 23:57:47 +0000
Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com
 [2607:f8b0:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 567fba97-d2a7-11ee-98f5-efadbce2ee36;
 Sat, 24 Feb 2024 00:57:45 +0100 (CET)
Received: by mail-pl1-x636.google.com with SMTP id
 d9443c01a7336-1dc0d11d1b7so8497715ad.2
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 15:57:45 -0800 (PST)
Received: from localhost.localdomain ([2620:10d:c090:400::4:45de])
 by smtp.gmail.com with ESMTPSA id
 d11-20020a170902728b00b001dc23e877c1sm6175189pll.265.2024.02.23.15.57.42
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 23 Feb 2024 15:57:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 567fba97-d2a7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708732664; x=1709337464; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=4T/JxyeGclar7cIIHLQMvHUD6oGOKffCuXl9nke3dJg=;
        b=PVL2fwFX95asjvNqbgQ4wubZN/VAg0zoG27fW6SMbPx1rcE70rz2V+qpivYsSlXEyA
         xY8KU3Q3i1kPlkuFNf+3C4vjS/lm+AErX/bOzj2Fe6eIDYBGJl36vixok6HxomnqQ+r0
         t5ZFdwcgwDVt+82A+HvZJWgRdGxV3ADiQv728xmmNCG9jUb+ADloHUaKpBH6Q/4PDjDh
         RmSrsCW674acpRIjmQljvipaswO2ZUm1f8w7n/3ptc7QKrPdzHZuyvRoT0/XZV9ZD/4t
         mB/nMU01mll89/elTukKrvud1za2bhIhgNyypTKmMFS8CKGVn05Yyb1fhuxCUnHBODhT
         nXPQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708732664; x=1709337464;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=4T/JxyeGclar7cIIHLQMvHUD6oGOKffCuXl9nke3dJg=;
        b=OAWPTBHY4dKwq7YvW76b0RE/ga+bjyHeutKw6IDziRrBL40jmElMaSgQ5OwA0Dex39
         7f/CeR2o5oOyNKy1+6a/ku+e8cUm5oI+V2Vbycc64IVw4zk9uNlmagYXHL2A3ICTSgTr
         5dldtCFpSGhaTnXPwssip8QP1CHXb1lvAVnAK5qAn12oMyXZ+jsZ6yiXZk4tGBCslj8B
         S2TesUcIHKjn7Cu0GS/LnCTgppZwjEHj9eEDEDnIY3hBDvNZ4ZQdFrBOhMX/Zm2nQZOM
         EoIyDdF1+qnpn+p86XsHu5Gqfz7V0n0Ji2W8wzk0iqr7TUIlpoOyEOVKWsnCw04eirqi
         XoWA==
X-Forwarded-Encrypted: i=1; AJvYcCVXjDdd5hQYBGin0C7nUSCWsggiSGKps3jy+PLOM5xVCk0OqIfszD9a7MxxbtZbLVedn2NHD9z5DF5XlNLNIonQcNGRAv//ynPA6KRKcbs=
X-Gm-Message-State: AOJu0YyaSoqO9tCiMwaXg3RDKjZW2pu+lK+a8cjCk9+a6UN+J2VBkHGk
	Zq37xYs0njE1unJweFf/6TyKKosJZj1vprzXD189vTU9BbPkYGQQ8w+sK1wQ
X-Google-Smtp-Source: AGHT+IGrC0E10ADJh+Q28iXnJRIwpr5YwuUUqZaBfkmopbBhR99u1mcgLWatOYemqm7CQuFVwzUw8A==
X-Received: by 2002:a17:902:c40d:b0:1d9:d341:b150 with SMTP id k13-20020a170902c40d00b001d9d341b150mr1582992plk.40.1708732664121;
        Fri, 23 Feb 2024 15:57:44 -0800 (PST)
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: bpf@vger.kernel.org
Cc: daniel@iogearbox.net,
	andrii@kernel.org,
	torvalds@linux-foundation.org,
	brho@google.com,
	hannes@cmpxchg.org,
	lstoakes@gmail.com,
	akpm@linux-foundation.org,
	urezki@gmail.com,
	hch@infradead.org,
	boris.ostrovsky@oracle.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	linux-mm@kvack.org,
	xen-devel@lists.xenproject.org,
	kernel-team@fb.com
Subject: [PATCH v2 bpf-next 3/3] mm: Introduce VM_SPARSE kind and vm_area_[un]map_pages().
Date: Fri, 23 Feb 2024 15:57:28 -0800
Message-Id: <20240223235728.13981-4-alexei.starovoitov@gmail.com>
X-Mailer: git-send-email 2.39.3 (Apple Git-145)
In-Reply-To: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
References: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Alexei Starovoitov <ast@kernel.org>

vmap/vmalloc APIs are used to map a set of pages into contiguous kernel virtual space.

get_vm_area() with appropriate flag is used to request an area of kernel address range.
It'se used for vmalloc, vmap, ioremap, xen use cases.
- vmalloc use case dominates the usage. Such vm areas have VM_ALLOC flag.
- the areas created by vmap() function should be tagged with VM_MAP.
- ioremap areas are tagged with VM_IOREMAP.
- xen use cases are VM_XEN.

BPF would like to extend the vmap API to implement a lazily-populated
sparse, yet contiguous kernel virtual space.
Introduce VM_SPARSE vm_area flag and
vm_area_map_pages(area, start_addr, count, pages) API to map a set
of pages within a given area.
It has the same sanity checks as vmap() does.
It also checks that get_vm_area() was created with VM_SPARSE flag
which identifies such areas in /proc/vmallocinfo
and returns zero pages on read through /proc/kcore.

The next commits will introduce bpf_arena which is a sparsely populated shared
memory region between bpf program and user space process. It will map
privately-managed pages into a sparse vm area with the following steps:

  area = get_vm_area(area_size, VM_SPARSE);  // at bpf prog verification time
  vm_area_map_pages(area, kaddr, 1, page);   // on demand
                    // it will return an error if kaddr is out of range
  vm_area_unmap_pages(area, kaddr, 1);
  free_vm_area(area);                        // after bpf prog is unloaded

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
 include/linux/vmalloc.h |  4 +++
 mm/vmalloc.c            | 55 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 57 insertions(+), 2 deletions(-)

diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index 223e51c243bc..416bc7b0b4db 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -29,6 +29,7 @@ struct iov_iter;		/* in uio.h */
 #define VM_MAP_PUT_PAGES	0x00000200	/* put pages and free array in vfree */
 #define VM_ALLOW_HUGE_VMAP	0x00000400      /* Allow for huge pages on archs with HAVE_ARCH_HUGE_VMALLOC */
 #define VM_XEN			0x00000800	/* xen use cases */
+#define VM_SPARSE		0x00001000	/* sparse vm_area. not all pages are present. */
 
 #if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
 	!defined(CONFIG_KASAN_VMALLOC)
@@ -233,6 +234,9 @@ static inline bool is_vm_area_hugepages(const void *addr)
 }
 
 #ifdef CONFIG_MMU
+int vm_area_map_pages(struct vm_struct *area, unsigned long addr, unsigned int count,
+		      struct page **pages);
+int vm_area_unmap_pages(struct vm_struct *area, unsigned long addr, unsigned int count);
 void vunmap_range(unsigned long addr, unsigned long end);
 static inline void set_vm_flush_reset_perms(void *addr)
 {
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index d769a65bddad..a05dfbbacb78 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -648,6 +648,54 @@ static int vmap_pages_range(unsigned long addr, unsigned long end,
 	return err;
 }
 
+/**
+ * vm_area_map_pages - map pages inside given vm_area
+ * @area: vm_area
+ * @addr: start address inside vm_area
+ * @count: number of pages
+ * @pages: pages to map (always PAGE_SIZE pages)
+ */
+int vm_area_map_pages(struct vm_struct *area, unsigned long addr, unsigned int count,
+		      struct page **pages)
+{
+	unsigned long size = ((unsigned long)count) * PAGE_SIZE;
+	unsigned long end = addr + size;
+
+	might_sleep();
+	if (WARN_ON_ONCE(area->flags & VM_FLUSH_RESET_PERMS))
+		return -EINVAL;
+	if (WARN_ON_ONCE(area->flags & VM_NO_GUARD))
+		return -EINVAL;
+	if (WARN_ON_ONCE(!(area->flags & VM_SPARSE)))
+		return -EINVAL;
+	if (count > totalram_pages())
+		return -E2BIG;
+	if (addr < (unsigned long)area->addr || (void *)end > area->addr + area->size)
+		return -ERANGE;
+
+	return vmap_pages_range(addr, end, PAGE_KERNEL, pages, PAGE_SHIFT);
+}
+
+/**
+ * vm_area_unmap_pages - unmap pages inside given vm_area
+ * @area: vm_area
+ * @addr: start address inside vm_area
+ * @count: number of pages to unmap
+ */
+int vm_area_unmap_pages(struct vm_struct *area, unsigned long addr, unsigned int count)
+{
+	unsigned long size = ((unsigned long)count) * PAGE_SIZE;
+	unsigned long end = addr + size;
+
+	if (WARN_ON_ONCE(!(area->flags & VM_SPARSE)))
+		return -EINVAL;
+	if (addr < (unsigned long)area->addr || (void *)end > area->addr + area->size)
+		return -ERANGE;
+
+	vunmap_range(addr, end);
+	return 0;
+}
+
 int is_vmalloc_or_module_addr(const void *x)
 {
 	/*
@@ -3822,9 +3870,9 @@ long vread_iter(struct iov_iter *iter, const char *addr, size_t count)
 
 		if (flags & VMAP_RAM)
 			copied = vmap_ram_vread_iter(iter, addr, n, flags);
-		else if (!(vm && (vm->flags & (VM_IOREMAP | VM_XEN))))
+		else if (!(vm && (vm->flags & (VM_IOREMAP | VM_XEN | VM_SPARSE))))
 			copied = aligned_vread_iter(iter, addr, n);
-		else /* IOREMAP|XEN area is treated as memory hole */
+		else /* IOREMAP|XEN|SPARSE area is treated as memory hole */
 			copied = zero_iter(iter, n);
 
 		addr += copied;
@@ -4418,6 +4466,9 @@ static int s_show(struct seq_file *m, void *p)
 	if (v->flags & VM_XEN)
 		seq_puts(m, " xen");
 
+	if (v->flags & VM_SPARSE)
+		seq_puts(m, " sparse");
+
 	if (v->flags & VM_ALLOC)
 		seq_puts(m, " vmalloc");
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat Feb 24 01:10:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 01:10:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684998.1065341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgYO-0005mc-3L; Sat, 24 Feb 2024 01:10:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684998.1065341; Sat, 24 Feb 2024 01:10:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgYN-0005lI-Rd; Sat, 24 Feb 2024 01:10:11 +0000
Received: by outflank-mailman (input) for mailman id 684998;
 Fri, 23 Feb 2024 23:57:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nFrk=KA=gmail.com=alexei.starovoitov@srs-se1.protection.inumbo.net>)
 id 1rdfQC-0005zS-Ux
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 23:57:40 +0000
Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com
 [2607:f8b0:4864:20::102e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 51dfb343-d2a7-11ee-98f5-efadbce2ee36;
 Sat, 24 Feb 2024 00:57:38 +0100 (CET)
Received: by mail-pj1-x102e.google.com with SMTP id
 98e67ed59e1d1-29a378040daso1051030a91.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 15:57:38 -0800 (PST)
Received: from localhost.localdomain ([2620:10d:c090:400::4:45de])
 by smtp.gmail.com with ESMTPSA id
 si4-20020a17090b528400b002963cab9e2asm2161658pjb.2.2024.02.23.15.57.34
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 23 Feb 2024 15:57:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51dfb343-d2a7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708732656; x=1709337456; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=A7v8QcQGKEfLwaJltNJW0mYz5w9KdmkVXr/8REE7aG4=;
        b=Ua9UR9Nkf7947Ao70UBTRIPhKLlen+kIXRvptVzPvLoTuHUqQVZLd8bXtCwwMMaVe7
         /sWEgWotLBNsj47oWGbK31kCqIq5JCYIgDkygRNIWCbqQFLd6Xbnc44dgpl3Lz6LYyln
         tcBAFtS97aC5TD+MTFrW9F+rOeZbL2WLU7lecXImmAbln6pvvosAd18yWtOhD6r3kmjC
         fPJiKHO196k6q6cbEMMavAF0o1dMGTZI+Z007iqCvmVvr3or9TRfYVobc+ge/Fui6boG
         fW3TlTZfxEQpY459mWBzm2MmeUcBYtyYFcKQFCfX8pFnEAW1ztcgNdwoxE/VgXkvmT7a
         7z3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708732656; x=1709337456;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=A7v8QcQGKEfLwaJltNJW0mYz5w9KdmkVXr/8REE7aG4=;
        b=LnnTHZEG6MlJ52DWwAu2qqvnj3L2OVqIYHfWqg6JLRDc6UTdSV46t3GLL84F00Jid6
         BbTYCN+SFYKDsPcAYARlHNxzkSFcAEv91ZVu89BHp9IOKlKBk8n8pCCbv8Sccb5QzB3T
         NjBU2w2P3FVhErTQ9ge9OamUmCJS8OMviOszOhYRrxqsR0Tsn3WnqoVwprmY8I4QdcqD
         NFmirRT7nGojPYreO31PisusORg++1sQd2NPnEOY88i4AluW4jCu3H/RhXt7rXZWMh07
         cQ0/MU59lISZpYYY7rBQmBKCiyZurt7V+YP/ZHLH7thgTR9vrjzX173he444HtmlbQnd
         zcFA==
X-Forwarded-Encrypted: i=1; AJvYcCW/n37LmpjHxmzzrTgwGOKIST1rU8GUraVvF7xrVNZTVBEQyNTcX+T5E7BX1I+KTzYXopH4E8ZqndhuoQvsN/WYyiMq9UxBvKBdeGaFutg=
X-Gm-Message-State: AOJu0Yz8BrfmkGVXQepVEPXsB+2aHZ3ZvcLEf4yF8naYDIuOoW3vKiqe
	2SM7A1kHw0qCSjfV6PRvl3FPaivWs5Z8vMSyXqDVgFWxyNM7oJQr
X-Google-Smtp-Source: AGHT+IHG/DO/OVIktjtp11/4CHtLMBPBSMQzb2KGZX31cgPqxxDf+7H9BLoJUXmr0UjElOG7XOtMJQ==
X-Received: by 2002:a17:90b:1050:b0:29a:9ba0:8a5b with SMTP id gq16-20020a17090b105000b0029a9ba08a5bmr411918pjb.5.1708732656476;
        Fri, 23 Feb 2024 15:57:36 -0800 (PST)
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: bpf@vger.kernel.org
Cc: daniel@iogearbox.net,
	andrii@kernel.org,
	torvalds@linux-foundation.org,
	brho@google.com,
	hannes@cmpxchg.org,
	lstoakes@gmail.com,
	akpm@linux-foundation.org,
	urezki@gmail.com,
	hch@infradead.org,
	boris.ostrovsky@oracle.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	linux-mm@kvack.org,
	xen-devel@lists.xenproject.org,
	kernel-team@fb.com
Subject: [PATCH v2 bpf-next 1/3] mm: Enforce VM_IOREMAP flag and range in ioremap_page_range.
Date: Fri, 23 Feb 2024 15:57:26 -0800
Message-Id: <20240223235728.13981-2-alexei.starovoitov@gmail.com>
X-Mailer: git-send-email 2.39.3 (Apple Git-145)
In-Reply-To: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
References: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Alexei Starovoitov <ast@kernel.org>

There are various users of get_vm_area() + ioremap_page_range() APIs.
Enforce that get_vm_area() was requested as VM_IOREMAP type and range passed to
ioremap_page_range() matches created vm_area to avoid accidentally ioremap-ing
into wrong address range.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
 mm/vmalloc.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index d12a17fc0c17..f42f98a127d5 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -307,8 +307,21 @@ static int vmap_range_noflush(unsigned long addr, unsigned long end,
 int ioremap_page_range(unsigned long addr, unsigned long end,
 		phys_addr_t phys_addr, pgprot_t prot)
 {
+	struct vm_struct *area;
 	int err;
 
+	area = find_vm_area((void *)addr);
+	if (!area || !(area->flags & VM_IOREMAP)) {
+		WARN_ONCE(1, "vm_area at addr %lx is not marked as VM_IOREMAP\n", addr);
+		return -EINVAL;
+	}
+	if (addr != (unsigned long)area->addr ||
+	    (void *)end != area->addr + get_vm_area_size(area)) {
+		WARN_ONCE(1, "ioremap request [%lx,%lx) doesn't match vm_area [%lx, %lx)\n",
+			  addr, end, (long)area->addr,
+			  (long)area->addr + get_vm_area_size(area));
+		return -ERANGE;
+	}
 	err = vmap_range_noflush(addr, end, phys_addr, pgprot_nx(prot),
 				 ioremap_max_page_shift);
 	flush_cache_vmap(addr, end);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat Feb 24 01:10:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 01:10:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684999.1065344 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgYO-0005s5-7k; Sat, 24 Feb 2024 01:10:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684999.1065344; Sat, 24 Feb 2024 01:10:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgYO-0005pG-21; Sat, 24 Feb 2024 01:10:12 +0000
Received: by outflank-mailman (input) for mailman id 684999;
 Fri, 23 Feb 2024 23:57:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nFrk=KA=gmail.com=alexei.starovoitov@srs-se1.protection.inumbo.net>)
 id 1rdfQE-0005yy-GZ
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 23:57:42 +0000
Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com
 [2607:f8b0:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5433a7fd-d2a7-11ee-8a57-1f161083a0e0;
 Sat, 24 Feb 2024 00:57:42 +0100 (CET)
Received: by mail-pl1-x635.google.com with SMTP id
 d9443c01a7336-1dc418fa351so5821525ad.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 15:57:41 -0800 (PST)
Received: from localhost.localdomain ([2620:10d:c090:400::4:45de])
 by smtp.gmail.com with ESMTPSA id
 bc7-20020a170902930700b001dc486f0cb2sm3796745plb.208.2024.02.23.15.57.38
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 23 Feb 2024 15:57:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5433a7fd-d2a7-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708732660; x=1709337460; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Wgy7UuZvicma9Y2PX48aaJ5ZDCArBODZAL7mSlN3pk0=;
        b=YELVKYQWjyHrpXdKoo1gSdbC5UAdJBNpVzVnBN2J70KHFCWih4VpBXw7WQEgl3nmT2
         nduA1il6SF2NcgUXhTVhGhn/v81Rkbei1qlP/fG9aMJcOmUiqbYulE5aceVqzvtBGrIj
         4dNeVJ7NQXEzoh/qX1a5uGP+J/pdpXXNtIM4i0bNatk62/E2D+BAWwjC9JS+UNAP+S42
         tTjrfGSVLHluiVNOANYb+wUCaNe/2WDiOKgao2MwEqahk7RsAORibYqRuhkDu8zBX1/+
         NN+CVmW+DJZdy256fPVtU2I8/2mbKGtk8ohPdFd/DQFlc54LLdEiBJsOh+Rcr8MzxXCd
         OM3A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708732660; x=1709337460;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Wgy7UuZvicma9Y2PX48aaJ5ZDCArBODZAL7mSlN3pk0=;
        b=of7Bl1ruXvOlC7p3+naVW5h3FtUgqb49JWrmXbpCvFfG0lW7pPsJigG5/QT1ehdrqc
         k0P5UXqPCOrU7PD+DyET44/WVow8g8BjPE2wqvbfarJvu9xjyrjHIDGFs6AaMADROtLt
         ucwSppxY6TTOyMecVR1TnjyOvjd7tUuztBR8xqHko9sEqD4SGdX9PZgoPG1d0eNvrZW6
         PNgKKfBwFd61fHl1/NrSyDSAJCjpBMYRbaWRng7pgNNWCLISF/EMviAdHnik+g9W5+DD
         HZi2C2wx1fysftAe8N9q/3M0IUi1bU8349XLSiiFmgwxHjfkYGvCCS0vuIDbngSOb3he
         3czQ==
X-Forwarded-Encrypted: i=1; AJvYcCVmGqN0hB8yCtdvlji6npZkn1nPECxHPVo0gHLxMyeRZogWcw6Qws1tWsHyl+ojZVvS/ylr80kZS0k6F5y0L2iQTV9XA6WTm4JZen4OOOk=
X-Gm-Message-State: AOJu0YwMC6sMS/exdGpw5zXGrMAI1oUlytFV3fNP2rHnQC9/JeHswaOx
	xMt176HU8jFkPx+bvDbbRj1peBfzYpAoiGzd+pypoI/aUaGBHKh6
X-Google-Smtp-Source: AGHT+IGELhXnvr3n524zH97gDxBoJwFKXzefQUh+8dOBuQvkiJz95RKjpkB2JqjALLNdOQ3Mm4DyFA==
X-Received: by 2002:a17:902:f645:b0:1dc:b64:13cd with SMTP id m5-20020a170902f64500b001dc0b6413cdmr1551144plg.27.1708732660213;
        Fri, 23 Feb 2024 15:57:40 -0800 (PST)
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: bpf@vger.kernel.org
Cc: daniel@iogearbox.net,
	andrii@kernel.org,
	torvalds@linux-foundation.org,
	brho@google.com,
	hannes@cmpxchg.org,
	lstoakes@gmail.com,
	akpm@linux-foundation.org,
	urezki@gmail.com,
	hch@infradead.org,
	boris.ostrovsky@oracle.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	linux-mm@kvack.org,
	xen-devel@lists.xenproject.org,
	kernel-team@fb.com
Subject: [PATCH v2 bpf-next 2/3] mm, xen: Separate xen use cases from ioremap.
Date: Fri, 23 Feb 2024 15:57:27 -0800
Message-Id: <20240223235728.13981-3-alexei.starovoitov@gmail.com>
X-Mailer: git-send-email 2.39.3 (Apple Git-145)
In-Reply-To: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
References: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Alexei Starovoitov <ast@kernel.org>

xen grant table and xenbus ring are not ioremap the way arch specific code is using it,
so let's add VM_XEN flag to separate them from VM_IOREMAP users.
xen will not and should not be calling ioremap_page_range() on that range.
/proc/vmallocinfo will print such region as "xen" instead of "ioremap" as well.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
 arch/x86/xen/grant-table.c         | 2 +-
 drivers/xen/xenbus/xenbus_client.c | 2 +-
 include/linux/vmalloc.h            | 1 +
 mm/vmalloc.c                       | 7 +++++--
 4 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/arch/x86/xen/grant-table.c b/arch/x86/xen/grant-table.c
index 1e681bf62561..b816db0349c4 100644
--- a/arch/x86/xen/grant-table.c
+++ b/arch/x86/xen/grant-table.c
@@ -104,7 +104,7 @@ static int arch_gnttab_valloc(struct gnttab_vm_area *area, unsigned nr_frames)
 	area->ptes = kmalloc_array(nr_frames, sizeof(*area->ptes), GFP_KERNEL);
 	if (area->ptes == NULL)
 		return -ENOMEM;
-	area->area = get_vm_area(PAGE_SIZE * nr_frames, VM_IOREMAP);
+	area->area = get_vm_area(PAGE_SIZE * nr_frames, VM_XEN);
 	if (!area->area)
 		goto out_free_ptes;
 	if (apply_to_page_range(&init_mm, (unsigned long)area->area->addr,
diff --git a/drivers/xen/xenbus/xenbus_client.c b/drivers/xen/xenbus/xenbus_client.c
index 32835b4b9bc5..b9c81a2d578b 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -758,7 +758,7 @@ static int xenbus_map_ring_pv(struct xenbus_device *dev,
 	bool leaked = false;
 	int err = -ENOMEM;
 
-	area = get_vm_area(XEN_PAGE_SIZE * nr_grefs, VM_IOREMAP);
+	area = get_vm_area(XEN_PAGE_SIZE * nr_grefs, VM_XEN);
 	if (!area)
 		return -ENOMEM;
 	if (apply_to_page_range(&init_mm, (unsigned long)area->addr,
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index c720be70c8dd..223e51c243bc 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -28,6 +28,7 @@ struct iov_iter;		/* in uio.h */
 #define VM_FLUSH_RESET_PERMS	0x00000100	/* reset direct map and flush TLB on unmap, can't be freed in atomic context */
 #define VM_MAP_PUT_PAGES	0x00000200	/* put pages and free array in vfree */
 #define VM_ALLOW_HUGE_VMAP	0x00000400      /* Allow for huge pages on archs with HAVE_ARCH_HUGE_VMALLOC */
+#define VM_XEN			0x00000800	/* xen use cases */
 
 #if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
 	!defined(CONFIG_KASAN_VMALLOC)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index f42f98a127d5..d769a65bddad 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -3822,9 +3822,9 @@ long vread_iter(struct iov_iter *iter, const char *addr, size_t count)
 
 		if (flags & VMAP_RAM)
 			copied = vmap_ram_vread_iter(iter, addr, n, flags);
-		else if (!(vm && (vm->flags & VM_IOREMAP)))
+		else if (!(vm && (vm->flags & (VM_IOREMAP | VM_XEN))))
 			copied = aligned_vread_iter(iter, addr, n);
-		else /* IOREMAP area is treated as memory hole */
+		else /* IOREMAP|XEN area is treated as memory hole */
 			copied = zero_iter(iter, n);
 
 		addr += copied;
@@ -4415,6 +4415,9 @@ static int s_show(struct seq_file *m, void *p)
 	if (v->flags & VM_IOREMAP)
 		seq_puts(m, " ioremap");
 
+	if (v->flags & VM_XEN)
+		seq_puts(m, " xen");
+
 	if (v->flags & VM_ALLOC)
 		seq_puts(m, " vmalloc");
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat Feb 24 01:10:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 01:10:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.684996.1065335 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgYN-0005ht-S3; Sat, 24 Feb 2024 01:10:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 684996.1065335; Sat, 24 Feb 2024 01:10:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgYN-0005hH-K2; Sat, 24 Feb 2024 01:10:11 +0000
Received: by outflank-mailman (input) for mailman id 684996;
 Fri, 23 Feb 2024 23:57:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nFrk=KA=gmail.com=alexei.starovoitov@srs-se1.protection.inumbo.net>)
 id 1rdfQ8-0005yy-AH
 for xen-devel@lists.xenproject.org; Fri, 23 Feb 2024 23:57:36 +0000
Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com
 [2607:f8b0:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4fdbaf6e-d2a7-11ee-8a57-1f161083a0e0;
 Sat, 24 Feb 2024 00:57:34 +0100 (CET)
Received: by mail-pl1-x62d.google.com with SMTP id
 d9443c01a7336-1dc1e7c0e29so5686935ad.1
 for <xen-devel@lists.xenproject.org>; Fri, 23 Feb 2024 15:57:34 -0800 (PST)
Received: from localhost.localdomain ([2620:10d:c090:400::4:45de])
 by smtp.gmail.com with ESMTPSA id
 o16-20020a17090323d000b001d9edac54b1sm12313162plh.171.2024.02.23.15.57.30
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Fri, 23 Feb 2024 15:57:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4fdbaf6e-d2a7-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708732653; x=1709337453; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=DSYW/f6ddyBB9WtPg1xLnsJlpquY1/ikpKTyYScqzh0=;
        b=DHZAR66PGWFk6Ya12h0SLjd5waoX7yF749XqIMC+0K8CcCpH0QvkpHHgKAvp4XHgLU
         HDXd6rDbYyvw6qmmhHtlmLGu5OiuRWR+LySPBfHHcCxL0SrELyswRj9+dXVHdB7kBCvc
         E1LvwrHb7SYMYmkOZHfIrBMLrvWRDiv60x+6gE6IXlLOgGgdt8g9QBmZiwsKrzKrLQ4E
         u8fykTFFMb5UMeNer0Q+szbztPsh+A+aRexxF9KObeRsbgFDB3pmEyQM7AxbGyugFMd1
         /HqPchRKVPeM+1reIpKA5c/SoCuaC9Smj9REGGEbvnDSdziYLz1fIcjcUTRk3zqviROv
         nfgA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708732653; x=1709337453;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=DSYW/f6ddyBB9WtPg1xLnsJlpquY1/ikpKTyYScqzh0=;
        b=OTmHJo3ggs2kUP+36TJvG+9T/Ev5Xsc9KH8VLs83bc2EbXFsrw5OcPc4l7gguu3CHH
         2nFPZfumYNeWQm1tSYYN2VkCvKBZH74Ch1yWcyLwTCPO6ySQnvM5AXuMexNYKpfMas1h
         xBLKWOzzwBu4IBJ+8ikJN2Ks7WeuIjZ22vEcmNeiJ4gEVT832jQrwtf4FLA293GHr2Vj
         GXqhFWMCmY3zm41za+bLTvfnnwPDeeYmrRPMJRWp9LnloACsIEIMxqmKIZpiZQ+QIYLO
         cPcVukvxUml7nIgff/z1DLFBTp8OFdeUi5BT7xYKIjk3QewciDWrptMfQ0sMFlcz+b+A
         +y2A==
X-Forwarded-Encrypted: i=1; AJvYcCVskONynI8UeLrfM3OxesUG9POWCKG4DesZKyyDssMHS+/cyVbyjg0997FMGfibPsX8lk2cIsbc/cywbCS5mxC7Fa25uG2J/4dRMZrp/6A=
X-Gm-Message-State: AOJu0YxY1sIl5WdoGit3xDx1FoFQG9arLEB1dio0clPMSztG6qC3isPJ
	gAIQZeTe5vFREwb3QzQN372cektv2brMndTDc2EWB5eNyTGT8JfU
X-Google-Smtp-Source: AGHT+IF03Ip9HN7hOBgclqSMTFdHG7TNdTZF9AvtgtNN5+5+EnBr4r8zdxXO0SElDm498lBK65E9Ag==
X-Received: by 2002:a17:902:d2c4:b0:1db:5213:222 with SMTP id n4-20020a170902d2c400b001db52130222mr1899819plc.5.1708732652680;
        Fri, 23 Feb 2024 15:57:32 -0800 (PST)
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: bpf@vger.kernel.org
Cc: daniel@iogearbox.net,
	andrii@kernel.org,
	torvalds@linux-foundation.org,
	brho@google.com,
	hannes@cmpxchg.org,
	lstoakes@gmail.com,
	akpm@linux-foundation.org,
	urezki@gmail.com,
	hch@infradead.org,
	boris.ostrovsky@oracle.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	linux-mm@kvack.org,
	xen-devel@lists.xenproject.org,
	kernel-team@fb.com
Subject: [PATCH v2 bpf-next 0/3] mm: Cleanup and identify various users of kernel virtual address space
Date: Fri, 23 Feb 2024 15:57:25 -0800
Message-Id: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
X-Mailer: git-send-email 2.39.3 (Apple Git-145)
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Alexei Starovoitov <ast@kernel.org>

There are various users of kernel virtual address space: vmalloc, vmap, ioremap, xen.

- vmalloc use case dominates the usage. Such vm areas have VM_ALLOC flag
and these areas are treated differently by KASAN.

- the areas created by vmap() function should be tagged with VM_MAP
(as majority of the users do).

- ioremap areas are tagged with VM_IOREMAP and vm area start is aligned to size
of the area unlike vmalloc/vmap.

- there is also xen usage that is marked as VM_IOREMAP, but it doesn't
call ioremap_page_range() unlike all other VM_IOREMAP users.

To clean this up:
1. Enforce that ioremap_page_range() checks the range and VM_IOREMAP flag.
2. Introduce VM_XEN flag to separate xen us cases from ioremap.

In addition BPF would like to reserve regions of kernel virtual address
space and populate it lazily, similar to xen use cases.
For that reason, introduce VM_SPARSE flag and vm_area_[un]map_pages() helpers
to populate this sparse area.

In the end the /proc/vmallocinfo will show
"vmalloc"
"vmap"
"ioremap"
"xen"
"sparse"
categories for different kinds of address regions.

ioremap, xen, sparse will return zero when dumped through /proc/kcore

Alexei Starovoitov (3):
  mm: Enforce VM_IOREMAP flag and range in ioremap_page_range.
  mm, xen: Separate xen use cases from ioremap.
  mm: Introduce VM_SPARSE kind and vm_area_[un]map_pages().

 arch/x86/xen/grant-table.c         |  2 +-
 drivers/xen/xenbus/xenbus_client.c |  2 +-
 include/linux/vmalloc.h            |  5 +++
 mm/vmalloc.c                       | 71 +++++++++++++++++++++++++++++-
 4 files changed, 76 insertions(+), 4 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Sat Feb 24 01:31:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 01:31:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685025.1065378 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgt9-0002fl-JO; Sat, 24 Feb 2024 01:31:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685025.1065378; Sat, 24 Feb 2024 01:31:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdgt9-0002fe-Dx; Sat, 24 Feb 2024 01:31:39 +0000
Received: by outflank-mailman (input) for mailman id 685025;
 Sat, 24 Feb 2024 01:31:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdgt7-0002fU-Ui; Sat, 24 Feb 2024 01:31:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdgt7-00006l-QL; Sat, 24 Feb 2024 01:31:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdgt7-00076M-5N; Sat, 24 Feb 2024 01:31:37 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdgt7-0000iW-0v; Sat, 24 Feb 2024 01:31:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=CYOEDsz8syK/EsrrYX3rXD0HiSckSFEy96phruzCBag=; b=DNtcSK9WTLpxwPCGo1aWah8J6m
	VaJpAKc3KGbCwmjLH7AgzF5swY1qCGBbWe3tFsh6xYI3nAPkMg/PylugWrRUzLF0WuKvwEIj6nR7X
	b8GWSZdppKC2e3o5qS39MyE2QNShIkIjeTHH75wAEuU0aEvuFmVj0mzuh8x1YxrWnfkQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184745-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184745: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=92babc88f67ed0ef3dc575a8b9534040274678ee
X-Osstest-Versions-That:
    xen=f5e1c527d0a0d09ca0cb1dcd8d4ab4a1a5261e91
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 24 Feb 2024 01:31:37 +0000

flight 184745 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184745/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  92babc88f67ed0ef3dc575a8b9534040274678ee
baseline version:
 xen                  f5e1c527d0a0d09ca0cb1dcd8d4ab4a1a5261e91

Last test of basis   184739  2024-02-23 18:00:28 Z    0 days
Testing same since   184745  2024-02-23 23:03:51 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f5e1c527d0..92babc88f6  92babc88f67ed0ef3dc575a8b9534040274678ee -> smoke


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 05:41:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 05:41:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685037.1065387 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdkme-0006oQ-Px; Sat, 24 Feb 2024 05:41:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685037.1065387; Sat, 24 Feb 2024 05:41:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdkme-0006oJ-N8; Sat, 24 Feb 2024 05:41:12 +0000
Received: by outflank-mailman (input) for mailman id 685037;
 Sat, 24 Feb 2024 05:41:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=PObI=KB=aepfle.de=olaf@srs-se1.protection.inumbo.net>)
 id 1rdkmd-0006oD-4Z
 for xen-devel@lists.xenproject.org; Sat, 24 Feb 2024 05:41:11 +0000
Received: from mo4-p00-ob.smtp.rzone.de (mo4-p00-ob.smtp.rzone.de
 [81.169.146.221]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4e1a3b0b-d2d7-11ee-98f5-efadbce2ee36;
 Sat, 24 Feb 2024 06:41:06 +0100 (CET)
Received: from sender by smtp.strato.de (RZmta 50.0.4 AUTH)
 with ESMTPSA id Q60eb001O5f21mk
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits))
 (Client did not present a certificate);
 Sat, 24 Feb 2024 06:41:02 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e1a3b0b-d2d7-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; t=1708753262; cv=none;
    d=strato.com; s=strato-dkim-0002;
    b=fWNRmebLBzL1O6xOB8jGE3f8EPuSROIx2RUB2J9V7zjmP/EUQeKumbBvLQCvogDoJL
    /LROM52IA2ZvFFZk0rTroie2VSGhgFifdABCPpwjpmobFB/0O+yRF491UDHRJcZB+KP4
    IYhwZjjm+NEuf0Cl3HqL9Da4frINas2qiCXe6/eQW3uwXMk2QBocCQBBnx4tFGy/z3xa
    fxbP8L72Avbfk/7bleB9y8HmwpYv9M6eR1Ua9/Y2TEyLGW9kfTSehYPsYi23/s2dvSxI
    7dFi9d+tHPghK+5pmCwwbL4lyfXgImBU2uWAQ0TljQS/KwbACUMQkJ11aqAnM1gd4IDn
    IG1A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1708753262;
    s=strato-dkim-0002; d=strato.com;
    h=References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date:Cc:Date:
    From:Subject:Sender;
    bh=tiVa+cmzDxUAxS2MsGAu2HlnRcl+rBEDZXnON/1r4tc=;
    b=fhR5U/PyT7DAEyG1nd15+S0YvNjnqh12fEqTiRXZC6snhYkhi4KCMy3AKc8ZD7jLP+
    bcGTcprjhB10pfTtnaP8rwWOfSc+fAMgLnzTU3y0iLHO9hRqWqozQfa7sgPu7x2aNE0C
    66nEHfcp4OqX/ChRS1dTlFgVZ6FPlBjJRanAlyhWGlBP2a4xyx7idsOAwbEvoMPoMPI9
    bqX811Az5TiS53ZxV9CWo2ifMicKYcKaXUxuYfvNubUTGal6FSaEL8S2kZRopBKU07Vv
    QsEDucHtUKZ35vnIrzganfXrXrmKxCCGSdA7/oXrxvt8GwhLvhPADd7Lg5q08ODjR11Y
    QXag==
ARC-Authentication-Results: i=1; strato.com;
    arc=none;
    dkim=none
X-RZG-CLASS-ID: mo00
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1708753262;
    s=strato-dkim-0002; d=aepfle.de;
    h=References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date:Cc:Date:
    From:Subject:Sender;
    bh=tiVa+cmzDxUAxS2MsGAu2HlnRcl+rBEDZXnON/1r4tc=;
    b=mbTG5YD81dF9xg9/dtcwBYh0ecK/FCSCh3bUWKGpV3zLAw344odjT3Bko3soNMSI7o
    lQOoAdSb+jj0EDUIbHWwt6z0g9PJYCTwkhs3nfXPPY0TgWZCWlpWhUwQa338POPrg/Rg
    XotgRAeTu01sgcJX1OFN6p1TJgJyxQvGDfTHRTn0HoEQKJRnt3gcGLd0Yr7A+HZH3CNE
    0dasbc5sZspM/QJPpEAo3zLOn7AX0YNeT4889MnAGJkIjmU8KqV7j1xFOn5U2h2M4DA8
    b9vTQUCl4R6JxqyjekCo+V2ANMkYqusi6NBq6AEhdR37EmlVEArDCvLioqaRsJEd+Oui
    l/Bw==
DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; t=1708753262;
    s=strato-dkim-0003; d=aepfle.de;
    h=References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date:Cc:Date:
    From:Subject:Sender;
    bh=tiVa+cmzDxUAxS2MsGAu2HlnRcl+rBEDZXnON/1r4tc=;
    b=v3a+swVB3hNMjInf8dVmbUyQXtb649avWbfRU6hW5V5vVc8yvjQCjAuP4eleCPvLbi
    IPJ/MTPn4ChA+/WPniDg==
X-RZG-AUTH: ":P2EQZWCpfu+qG7CngxMFH1J+3q8wa/QLpd5ylWvMDX3y/OuD5rXVisR5B0GIOR3sVV7O9CNMudHTkv9gywkxX9QhB6i+AQ=="
Date: Sat, 24 Feb 2024 06:40:55 +0100
From: Olaf Hering <olaf@aepfle.de>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>
Subject: Re: [PATCH v1] automation: remove bin86/dev86 from tumbleweed image
Message-ID: <20240224064055.69dc7fb8.olaf@aepfle.de>
In-Reply-To: <alpine.DEB.2.22.394.2402231522020.754277@ubuntu-linux-20-04-desktop>
References: <20231213202515.30587-1-olaf@aepfle.de>
	<alpine.DEB.2.22.394.2402231522020.754277@ubuntu-linux-20-04-desktop>
X-Mailer: Claws Mail (olh) 20240213T072151.4501d00e hat ein Softwareproblem, kann man nichts machen.
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="Sig_/mVkE72fFxpJwgeXMNm7lIXN";
 protocol="application/pgp-signature"; micalg=pgp-sha256
Content-Transfer-Encoding: 7bit

--Sig_/mVkE72fFxpJwgeXMNm7lIXN
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

Fri, 23 Feb 2024 15:22:38 -0800 (PST) Stefano Stabellini <sstabellini@kerne=
l.org>:

> Do you have a successful gitlab pipeline with this patch applied that
> you can give me as proof of testing and success?

Yes, all of them since the patch went out.


Olaf

--Sig_/mVkE72fFxpJwgeXMNm7lIXN
Content-Type: application/pgp-signature
Content-Description: Digitale Signatur von OpenPGP

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE97o7Um30LT3B+5b/86SN7mm1DoAFAmXZgWcACgkQ86SN7mm1
DoAWoA/+PVWPpu93tN9CE2mgjW6ZHVbQFrubpY/WgLzFaXfDwjH9gqYOrjspQn/7
5k/TQo/E33NdJ8GGSb3VcVcUoQHMunsaDGofo/LwuMn0MeOyOW7Vy8ovarxrD8Bv
5P3YoWJbH4+RmhG6uRKTXyOAP8g5vDcl+hfDX3xh/4oN9jYUO0lhLJ03atV4h4XB
bREZMv8MSsZe+uuiWPrv0+zbmhDHvPdmXOxJCDVwiKqPe8kg+8/r7buG022YQMln
HCioxErIy018NyBZFkUpEbVQEGyEOJf4qKbCksKq/IMIfhTmAHIFmWxT4eqEu9RU
iNdnUX1w6OBd5swLt7z5CZJSEaAc3JA767A/rj1u2ac4eZutHAx9542a/e8w6YGv
ivRfw8kUMSjuoGF/p01JNW/1UXLCilt+bnhvXKJ+MA+hiIHXEDzPHjBJxvLJ9TVt
yQTsvLOwG+jAIuUDcFIgsZ9412vIZ3c8rtkeLJL83dtux0UeratsymXeBpBSP/vg
t1kEKDLNFBkBrf3KB6UofFBCwll5NzCOifW90K3lD6FV+Zh5YZ68SNbhb5fQ1+gU
p4dxQT8DeZLSgSwsw8u09j7IqEQrdaMBoSVmJ5+ybQZc1zhWOmg4gCjg2n6MrsVV
J489uePONN5cZTftYDe+Ys4obQS1QhOYag1qMKG0fgtwj13ccY8=
=uPaz
-----END PGP SIGNATURE-----

--Sig_/mVkE72fFxpJwgeXMNm7lIXN--


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 05:55:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 05:55:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685040.1065398 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdl0B-0008Po-0P; Sat, 24 Feb 2024 05:55:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685040.1065398; Sat, 24 Feb 2024 05:55:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdl0A-0008Ph-Ta; Sat, 24 Feb 2024 05:55:10 +0000
Received: by outflank-mailman (input) for mailman id 685040;
 Sat, 24 Feb 2024 05:55:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdl09-0008PX-1N; Sat, 24 Feb 2024 05:55:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdl08-00060c-Rs; Sat, 24 Feb 2024 05:55:08 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdl08-0003vZ-GV; Sat, 24 Feb 2024 05:55:08 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdl08-000423-DF; Sat, 24 Feb 2024 05:55:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=RXLzOD+zj4xkz3ErAblYLMPzzrE0atzAKNHaFp0b7Ko=; b=xbTwNwr6SKjwEIgrF+7v98XajC
	bb8m9yP78nRoll6qr6jeJ76ZqU4UbClIC9d/XPlq62wu/xLBXHhz6mTZkiCkakrr9bHqJdsNHJcCU
	Pe5SduskK9K1FbrJQYgXI30ZHfKgxaoKFnTIjgAWv9OhG6nVfBfnQovA57feJr+m69V0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184734-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-6.1 test] 184734: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:debian-hvm-install:fail:heisenbug
    linux-6.1:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-6.1:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-6.1:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-6.1:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=81e1dc2f70014b9523dd02ca763788e4f81e5bac
X-Osstest-Versions-That:
    linux=8b4118fabd6eb75fed19483b04dab3a036886489
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 24 Feb 2024 05:55:08 +0000

flight 184734 linux-6.1 real [real]
flight 184747 linux-6.1 real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184734/
http://logs.test-lab.xenproject.org/osstest/logs/184747/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 12 debian-hvm-install fail pass in 184747-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail in 184747 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184687
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184687
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184687
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184687
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184687
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184687
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184687
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184687
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 linux                81e1dc2f70014b9523dd02ca763788e4f81e5bac
baseline version:
 linux                8b4118fabd6eb75fed19483b04dab3a036886489

Last test of basis   184687  2024-02-16 18:13:56 Z    7 days
Testing same since   184734  2024-02-23 08:41:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Christian A. Ehrhardt" <lk@c--e.de>
  Aaron Conole <aconole@redhat.com>
  Aleksander Mazur <deweloper@wp.pl>
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexandra Winter <wintera@linux.ibm.com>
  Alexandru Gagniuc <alexandru.gagniuc@hp.com>
  Alexey Khoroshilov <khoroshilov@ispras.ru>
  Allen Pais <apais@linux.microsoft.com>
  Andi Shyti <andi.shyti@kernel.org>
  Andrei Vagin <avagin@google.com>
  Andrejs Cainikovs <andrejs.cainikovs@toradex.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andy Chi <andy.chi@canonical.com>
  Arnd Bergmann <arnd@arndb.de>
  Audra Mitchell <audra@redhat.com>
  Bagas Sanjaya <bagasdotme@gmail.com>
  Baokun Li <libaokun1@huawei.com>
  Bibo Mao <maobibo@loongson.cn>
  Bjorn Andersson <andersson@kernel.org>
  bo liu <bo.liu@senarytech.com>
  Boris Burkov <boris@bur.io>
  Borislav Petkov (AMD) <bp@alien8.de>
  Borislav Petkov <bp@alien8.de>
  Brad Spengler <spender@grsecurity.net>
  Breno Leitao <leitao@debian.org>
  Brian Norris <briannorris@chromium.org>
  Bryan O'Donoghue <bryan.odonoghue@linaro.org> # qrb5165-rb5
  Carlos Llamas <cmllamas@google.com>
  Christian A. Ehrhardt <lk@c--e.de>
  Christian Brauner <brauner@kernel.org>
  Christian König <christian.koenig@amd.com>
  Christoph Paasch <cpaasch@apple.com>
  Chuck Lever <chuck.lever@oracle.com>
  Damien Le Moal <dlemoal@kernel.org>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Basilio <daniel.basilio@corigine.com>
  Daniel Borkmann <daniel@iogearbox.net>
  Daniel Bristot de Oliveira <bristot@kernel.org>
  Daniel de Villiers <daniel.devilliers@corigine.com>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Danilo Krummrich <dakr@redhat.com>
  Dave Hansen <dave.hansen@linux.intel.com>
  Dave Jiang <dave.jiang@intel.com>
  David Engraf <david.engraf@sysgo.com>
  David Howells <dhowells@redhat.com>
  David Lin <yu-hao.lin@nxp.com>
  David S. Miller <davem@davemloft.net>
  David Schiller <david.schiller@jku.at>
  David Senoner <seda18@rolmail.net>
  David Sterba <dsterba@suse.com>
  David Wang <00107082@163.com>
  Davidlohr Bueso <dave@stgolabs.net>
  Dinghao Liu <dinghao.liu@zju.edu.cn>
  Dmitry Osipenko <dmitry.osipenko@collabora.com>
  Dmitry Torokhov <dmitry.torokhov@gmail.com>
  Doug Berger <opendmb@gmail.com>
  Dylan Hatch <dylanbhatch@google.com>
  Easwar Hariharan <eahariha@linux.microsoft.com>
  Edson Juliano Drosdeck <edson.drosdeck@gmail.com>
  Ekansh Gupta <quic_ekangupt@quicinc.com>
  Eniac Zhang <eniac-xw.zhang@hp.com>
  Eric Dumazet <edumazet@google.com>
  Esben Haabendal <esben@geanix.com>
  Fedor Pchelkin <pchelkin@ispras.ru>
  Felix Kuehling <Felix.Kuehling@amd.com>
  Felix Moessbauer <felix.moessbauer@siemens.com>
  Filipe Manana <fdmanana@suse.com>
  Florian Fainelli <florian.fainelli@broadcom.com>
  Fred Ai <fred.ai@bayhubtech.com>
  Gavin Shan <gshan@redhat.com>
  Geliang Tang <geliang.tang@linux.dev>
  Geliang Tang <geliang@kernel.org>
  Georgi Djakov <djakov@kernel.org>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Gregory CLEMENT <gregory.clement@bootlin.com>
  Guenter Roeck <linux@roeck-us.net>
  Guruswamy Basavaiah <guruswamy.basavaiah@broadcom.com>
  H. Peter Anvin <hpa@zytor.com>
  Heikki Krogerus <heikki.krogerus@linux.intel.com>
  Helge Deller <deller@gmx.de>
  Herbert Xu <herbert@gondor.apana.org.au>
  Horatiu Vultur <horatiu.vultur@microchip.com>
  Huacai Chen <chenhuacai@loongson.cn>
  Hugo Villeneuve <hvilleneuve@dimonoff.com>
  Hui Zhou <hui.zhou@corigine.com>
  Ido Schimmel <idosch@nvidia.com>
  Ilya Dryomov <idryomov@gmail.com>
  Ivan Vecera <ivecera@redhat.com>
  Jakub Kicinski <kuba@kernel.org>
  Jan Beulich <jbeulich@suse.com>
  Jann Horn <jannh@google.com>
  Jason Gerecke <jason.gerecke@wacom.com>
  Jason Gerecke <killertofu@gmail.com>
  Jason Gunthorpe <jgg@nvidia.com>
  Jean Delvare <jdelvare@suse.de>
  Jens Axboe <axboe@kernel.dk>
  Jiangfeng Xiao <xiaojiangfeng@huawei.com>
  Jiri Kosina <jkosina@suse.com>
  Jiri Olsa <jolsa@kernel.org>
  Johan Hovold <johan+linaro@kernel.org>
  Johannes Berg <johannes.berg@intel.com>
  John Allen <john.allen@amd.com>
  John Johansen <john.johansen@canonical.com>
  John Kacur <jkacur@redhat.com>
  Jon Hunter <jonathanh@nvidia.com>
  Jonathan Cameron <Jonathan.Cameron@huawei.com>
  Josef Bacik <josef@toxicpanda.com>
  José Relvas <josemonsantorelvas@gmail.com>
  Jozsef Kadlecsik <kadlec@netfilter.org>
  Junxiao Bi <junxiao.bi@oracle.com>
  Kalle Valo <kvalo@kernel.org>
  Kelsey Steele <kelseysteele@linux.microsoft.com>
  kernelci.org bot <bot@kernelci.org>
  Kim Phillips <kim.phillips@amd.com>
  Konrad Dybcio <konrad.dybcio@linaro.org>
  Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
  Lee Duncan <lduncan@suse.com>
  limingming3 <limingming3@lixiang.com>
  limingming3 <limingming890315@gmail.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Linus Torvalds <torvalds@linuxfoundation.org>
  Linux Kernel Functional Testing <lkft@linaro.org>
  Lokesh Gidra <lokeshgidra@google.com>
  Louis Peens <louis.peens@corigine.com>
  Luka Guzenko <l.guzenko@web.de>
  Lukas Wunner <lukas@wunner.de>
  Marc Kleine-Budde <mkl@pengutronix.de>
  Marc Zyngier <maz@kernel.org>
  Marcel Ziswiler <marcel.ziswiler@toradex.com> # Verdin AM62 (IW416)
  Mario Limonciello <mario.limonciello@amd.com>
  Mark Brown <broonie@kernel.org>
  Mark Rutland <mark.rutland@arm.com>
  Martin K. Petersen <martin.petersen@oracle.com>
  Masahiro Yamada <masahiroy@kernel.org>
  Masami Hiramatsu (Google) <mhiramat@kernel.org>
  Mateusz Jończyk <mat.jonczyk@o2.pl>
  Matthieu Baerts (NGI0) <matttbe@kernel.org>
  Mauro Carvalho Chehab <mchehab@kernel.org>
  Maxime Jayat <maxime.jayat@mobile-devices.fr>
  Michael Ellerman <mpe@ellerman.id.au>
  Michael Kelley <mhklinux@outlook.com>
  Michal Hocko <mhocko@suse.com>
  Mike Marciniszyn <mike.marciniszyn@intel.com>
  Mike Rapoport (IBM) <rppt@kernel.org>
  Mike Snitzer <snitzer@kernel.org>
  Mikulas Patocka <mpatocka@redhat.com>
  Mingwei Zhang <mizhang@google.com>
  Muchun Song <muchun.song@linux.dev>
  Mukesh Kumar Savaliya <quic_msavaliy@quicinc.com>
  Namjae Jeon <linkinjeon@kernel.org>
  Nathan Chancellor <nathan@kernel.org>
  Naveen N Rao <naveen@kernel.org>
  NeilBrown <neilb@suse.de>
  Nikita Zhandarovich <n.zhandarovich@fintech.ru>
  Nuno Sa <nuno.sa@analog.com>
  Oleg Nesterov <oleg@redhat.com>
  Oleksij Rempel <o.rempel@pengutronix.de>
  Oliver Neukum <oneukum@suse.com>
  Ondrej Mosnacek <omosnace@redhat.com>
  Oscar Salvador <osalvador@suse.de>
  Pablo Neira Ayuso <pablo@netfilter.org>
  Paolo Abeni <pabeni@redhat.com>
  Paul Moore <paul@paul-moore.com>
  Paulo Alcantara (SUSE) <pc@manguebit.com>
  Paulo Alcantara <pc@manguebit.com>
  Pavel Machek (CIP) <pavel@denx.de>
  Peter Zijlstra (Intel) <peterz@infradead.org>
  Peter Zijlstra <peterz@infradead.org>
  Petr Machata <petrm@nvidia.com>
  Philip Yang <Philip.Yang@amd.com>
  Prakash Sangappa <prakash.sangappa@oracle.com>
  Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
  Qu Wenruo <wqu@suse.com>
  Radek Krejci <radek.krejci@oracle.com>
  Rafal Romanowski <rafal.romanowski@intel.com>
  Randy Dunlap <rdunlap@infradead.org>
  Rishabh Dave <ridave@redhat.com>
  Rob Clark <robdclark@chromium.org>
  Rob Herring <robh@kernel.org>
  Ron Economos <re@w6rz.net>
  Ryan Roberts <ryan.roberts@arm.com>
  Ryusuke Konishi <konishi.ryusuke@gmail.com>
  Sabrina Dubroca <sd@queasysnail.net>
  Salvatore Bonaccorso <carnil@debian.org>
  Samuel Holland <samuel.holland@sifive.com>
  Saravana Kannan <saravanak@google.com>
  Sasha Levin <sashal@kernel.org>
  Sean Anderson <sean.anderson@seco.com>
  Sean Christopherson <seanjc@google.com>
  Sean Young <sean@mess.org>
  Sebastian Ott <sebott@redhat.com>
  SeongJae Park <sj@kernel.org>
  Shawn Guo <shawnguo@kernel.org>
  Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
  Shiraz Saleem <shiraz.saleem@intel.com>
  Shradha Gupta <shradhagupta@linux.microsoft.com>
  Shrikanth Hegde <sshegde@linux.ibm.com>
  Shuah Khan <skhan@linuxfoundation.org>
  Sinthu Raja <sinthu.raja@ti.com>
  Sjoerd Simons <sjoerd@collabora.com>
  Song Liu <song@kernel.org>
  Souradeep Chakrabarti <schakrabarti@linux.microsoft.com>
  Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
  Steffen Klassert <steffen.klassert@secunet.com>
  Stephan Gerhold <stephan@gerhold.net>
  Steve French <stfrench@microsoft.com>
  Steve Wahl <steve.wahl@hpe.com>
  Steven Rostedt (Google) <rostedt@goodmis.org>
  Takashi Iwai <tiwai@suse.de>
  Takashi Sakamoto <o-takashi@sakamocchi.jp>
  Tatsunosuke Tobita <tatsunosuke.tobita@wacom.com>
  Techno Mooney <techno.mooney@gmail.com>
  Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
  Theodore Ts'o <tytso@mit.edu>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Thomas Bogendoerfer <tsbogend@alpha.franken.de>
  Thomas Gleixner <tglx@linutronix.de>
  Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
  Tom Chung <chiahsuan.chung@amd.com>
  Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
  Tony Nguyen <anthony.l.nguyen@intel.com>
  Ulf Hansson <ulf.hansson@linaro.org>
  Uttkarsh Aggarwal <quic_uaggarwa@quicinc.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Viken Dadhaniya <quic_vdadhani@quicinc.com>
  Vincent Donnefort <vdonnefort@google.com>
  Vitaly Rodionov <vitalyr@opensource.cirrus.com>
  Vivek Kasireddy <vivek.kasireddy@intel.com>
  Will Deacon <will@kernel.org>
  Xu Yang <xu.yang_2@nxp.com>
  Yann Sionneau<ysionneau@kalrayinc.com>
  Yonghong Song <yhs@fb.com>
  yuan linyu <yuanlinyu@hihonor.com>
  Zach O'Keefe <zokeefe@google.com>
  Zhenyu Zhang <zhenyzha@redhat.com>
  zhili.liu <zhili.liu@ucas.com.cn>
  Zhipeng Lu <alexious@zju.edu.cn>
  Ziqi Zhao <astrajoan@yahoo.com>
  Стас Ничипорович <stasn77@gmail.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           fail    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   8b4118fabd6eb..81e1dc2f70014  81e1dc2f70014b9523dd02ca763788e4f81e5bac -> tested/linux-6.1


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 10:33:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 10:33:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685067.1065408 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdpLN-0003vW-O2; Sat, 24 Feb 2024 10:33:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685067.1065408; Sat, 24 Feb 2024 10:33:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdpLN-0003vO-H0; Sat, 24 Feb 2024 10:33:21 +0000
Received: by outflank-mailman (input) for mailman id 685067;
 Sat, 24 Feb 2024 10:33:20 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdpLM-0003vE-RD; Sat, 24 Feb 2024 10:33:20 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdpLM-00037A-Nv; Sat, 24 Feb 2024 10:33:20 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdpLM-0002cR-BZ; Sat, 24 Feb 2024 10:33:20 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdpLM-0003Gp-B1; Sat, 24 Feb 2024 10:33:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=2DAkFllEIWcvNRRAl1y+i4jzXXpCPAu4/R8ZGaghPJA=; b=Dm+cZkELFE/+OIf2y2pQ2DRGJs
	4cYdKE4R+jurCAGmrBri9OLX0TMiv5jSwzrjnn3VCJGLEcpRGjyHF+bPbuky5fbw1hQqG2UY1kUHa
	L/YXyaJqglbBJ3lladKNsymmauXjcYxOvN8XhaZOOt6D7Wnf3WM/rBPDdvtGB2K39L3g=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184738-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184738: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt-raw:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=b6d69282db550689ab5980e06eedd23b64584a73
X-Osstest-Versions-That:
    linux=ffd2cb6b718e189e7e2d5d0c19c25611f92e061a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 24 Feb 2024 10:33:20 +0000

flight 184738 linux-linus real [real]
flight 184748 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184738/
http://logs.test-lab.xenproject.org/osstest/logs/184748/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-raw  8 xen-boot            fail pass in 184748-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check fail in 184748 like 184731
 test-armhf-armhf-libvirt-raw 14 migrate-support-check fail in 184748 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184731
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184731
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184731
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184731
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184731
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184731
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184731
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                b6d69282db550689ab5980e06eedd23b64584a73
baseline version:
 linux                ffd2cb6b718e189e7e2d5d0c19c25611f92e061a

Last test of basis   184731  2024-02-22 21:44:16 Z    1 days
Testing same since   184738  2024-02-23 17:42:09 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  "Andrey Jr. Melnikov" <temnota.am@gmail.com>
  Ahmad Khalifa <ahmad@khalifa.ws>
  Andrey Jr. Melnikov <temnota.am@gmail.com>
  Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
  Damien Le Moal <dlemoal@kernel.org>
  Emil Renner Berthing <emil.renner.berthing@canonical.com>
  Guenter Roeck <linux@roeck-us.net>
  Jisheng Zhang <jszhang@kernel.org>
  Linus Torvalds <torvalds@linux-foundation.org>
  Niklas Cassel <cassel@kernel.org>
  Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   ffd2cb6b718e1..b6d69282db550  b6d69282db550689ab5980e06eedd23b64584a73 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 11:40:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 11:40:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685073.1065418 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdqO2-00030A-FD; Sat, 24 Feb 2024 11:40:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685073.1065418; Sat, 24 Feb 2024 11:40:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdqO2-000303-Bx; Sat, 24 Feb 2024 11:40:10 +0000
Received: by outflank-mailman (input) for mailman id 685073;
 Sat, 24 Feb 2024 11:40:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=s0+h=KB=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rdqO0-0002w3-9a
 for xen-devel@lists.xenproject.org; Sat, 24 Feb 2024 11:40:08 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 73a45d8f-d309-11ee-98f5-efadbce2ee36;
 Sat, 24 Feb 2024 12:40:04 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id CF8E24EE0739;
 Sat, 24 Feb 2024 12:40:03 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73a45d8f-d309-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Sat, 24 Feb 2024 12:40:03 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, michal.orzel@amd.com,
 xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, consulting@bugseng.com,
 jbeulich@suse.com, andrew.cooper3@citrix.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, julien@xen.org, Volodymyr Babchuk
 <Volodymyr_Babchuk@epam.com>, George Dunlap <george.dunlap@citrix.com>, Wei
 Liu <wl@xen.org>
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
In-Reply-To: <alpine.DEB.2.22.394.2402231502540.754277@ubuntu-linux-20-04-desktop>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
 <4f3b3d52a3ba20b05ecfa068b916b804@bugseng.com>
 <alpine.DEB.2.22.394.2402231502540.754277@ubuntu-linux-20-04-desktop>
Message-ID: <fb460b92d758891c1de42f7e588d5ffb@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 2024-02-24 00:05, Stefano Stabellini wrote:
> On Fri, 23 Feb 2024, Nicola Vetrini wrote:
>> On 2024-02-19 16:14, Nicola Vetrini wrote:
>> > The cache clearing and invalidation helpers in x86 and Arm didn't
>> > comply with MISRA C Rule 17.7: "The value returned by a function
>> > having non-void return type shall be used". On Arm they
>> > were always returning 0, while some in x86 returned -EOPNOTSUPP
>> > and in common/grant_table the return value is saved.
>> >
>> > As a consequence, a common helper arch_grant_cache_flush that returns
>> > an integer is introduced, so that each architecture can choose whether to
>> > return an error value on certain conditions, and the helpers have either
>> > been changed to return void (on Arm) or deleted entirely (on x86).
>> >
>> > Signed-off-by: Julien Grall <julien@xen.org>
>> > Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>> > ---
>> > The original refactor idea came from Julien Grall in [1]; I edited that
>> > proposal
>> > to fix build errors.
>> >
>> > I did introduce a cast to void for the call to flush_area_local on x86,
>> > because
>> > even before this patch the return value of that function wasn't checked in
>> > all
>> > but one use in x86/smp.c, and in this context the helper (perhaps
>> > incidentally)
>> > ignored the return value of flush_area_local.
>> >
>> > [1]
>> > https://lore.kernel.org/xen-devel/09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org/
>> > ---
>> >  xen/arch/arm/include/asm/page.h     | 33 ++++++++++++++++++-----------
>> >  xen/arch/x86/include/asm/flushtlb.h | 23 ++++++++++----------
>> >  xen/common/grant_table.c            |  9 +-------
>> >  3 files changed, 34 insertions(+), 31 deletions(-)
>> >
>> 
>> I'll put this patch in the backlog at the moment: too many intricacies 
>> while
>> trying to untangle xen/flushtlb from xen/mm.h, and there are easier 
>> cases that
>> can be done faster. If someone is interested I can post the partial 
>> work I've
>> done so far, even though it doesn't
>> build on x86.
> 
> I understand that the blocker is:
> 
> diff --git a/xen/arch/arm/include/asm/page.h 
> b/xen/arch/arm/include/asm/page.h
> index 69f817d1e6..e90c9de361 100644
> --- a/xen/arch/arm/include/asm/page.h
> +++ b/xen/arch/arm/include/asm/page.h
> @@ -123,6 +123,7 @@
> 
>  #ifndef __ASSEMBLY__
> 
> +#include <public/grant_table.h>
>  #include <xen/errno.h>
>  #include <xen/types.h>
>  #include <xen/lib.h>
> 
> 
> And the headers disentagling required to solve it, right?
> 
> 
> Let me ask a silly question. public/grant_table.h seems needed by
> arch_grant_cache_flush. Can we move arch_grant_cache_flush somewhere
> else? It is not like page.h is a perfect fit for it anyway.
> 
> For instance, can we move it to
> 
> xen/arch/arm/include/asm/grant_table.h
> 
> ?

Yes, this is what was suggested and what I was trying to accomplish.
Basically my plan is:

1. move the arch_grant_cache_flush helper to asm/grant_table.h for both 
architectures
2. pull out of xen/mm.h this hunk (note the inclusion of asm/flushtlb in 
the middle of the file) because there is a build error on 
tlbflush_current_time() induced in some .c file (don't remember which) 
by the earlier movement

-#include <asm/flushtlb.h>
-
-static inline void accumulate_tlbflush(bool *need_tlbflush,
-                                       const struct page_info *page,
-                                       uint32_t *tlbflush_timestamp)
-{
-    if ( page->u.free.need_tlbflush &&
-         page->tlbflush_timestamp <= tlbflush_current_time() &&
-         (!*need_tlbflush ||
-          page->tlbflush_timestamp > *tlbflush_timestamp) )
-    {
-        *need_tlbflush = true;
-        *tlbflush_timestamp = page->tlbflush_timestamp;
-    }
-}
-
-static inline void filtered_flush_tlb_mask(uint32_t tlbflush_timestamp)
-{
-    cpumask_t mask;
-
-    cpumask_copy(&mask, &cpu_online_map);
-    tlbflush_filter(&mask, tlbflush_timestamp);
-    if ( !cpumask_empty(&mask) )
-    {
-        perfc_incr(need_flush_tlb_flush);
-        arch_flush_tlb_mask(&mask);
-    }
-}
-

which is going to be in a new header xen/flushtlb.h
3. replace various inclusions the previously relied on the fact that 
xen/mm.h included asm/flushtlb.h (some even stating this as evidenced 
from the hunk below)

diff --git a/xen/arch/x86/cpu/microcode/amd.c 
b/xen/arch/x86/cpu/microcode/amd.c
index 75fc84e445ce..91ee7e6ec39e 100644
--- a/xen/arch/x86/cpu/microcode/amd.c
+++ b/xen/arch/x86/cpu/microcode/amd.c
@@ -15,8 +15,8 @@
   */

  #include <xen/err.h>
+#include <xen/flushtlb.h>
  #include <xen/init.h>
-#include <xen/mm.h> /* TODO: Fix asm/tlbflush.h breakage */

and then make everything build.
However, the dependencies tied to xen/mm.h are quite numerous on x86, 
and I'm not seeing an obvious way to avoid touching xen/mm.h. See this 
tree [1] for the latest state of the patch.

If anyone has an idea how to tackle this in a smarter way, I'm open to 
suggestions.
Specifically in step (2) there might be a way to avoid doing that 
modification, perhaps.

[1] 
https://gitlab.com/xen-project/people/bugseng/xen/-/commits/cache_helpers

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 13:37:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 13:37:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685082.1065427 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdsDI-0006Hm-SJ; Sat, 24 Feb 2024 13:37:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685082.1065427; Sat, 24 Feb 2024 13:37:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdsDI-0006Hf-Pn; Sat, 24 Feb 2024 13:37:12 +0000
Received: by outflank-mailman (input) for mailman id 685082;
 Sat, 24 Feb 2024 13:37:11 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rdsDH-0006HZ-D9
 for xen-devel@lists.xenproject.org; Sat, 24 Feb 2024 13:37:11 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rdsDG-0006BP-OY; Sat, 24 Feb 2024 13:37:10 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rdsDG-0001u1-Df; Sat, 24 Feb 2024 13:37:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=6+3ORigXTMHF7gWR/0z8B7LLduvQ7kjHteB4pKe2qEo=; b=fcImsQKY288DqlUHK0DIeqse99
	1m9laWtroX4Y1v5x2r6ifPsgShemOmOoFcHhlSmqHDZbPzf8Mj+3yKfGIo3yhI6yaHAfSPQjcAnBl
	i5BhL1F4dHjeHy6x64XN72DeamYFb4gPv5Vgx/eW1X73Z2POEvnDCrrh4mspkPwv48is=;
Message-ID: <bb2485e5-7818-435b-8d9c-dda88100979f@xen.org>
Date: Sat, 24 Feb 2024 13:37:08 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: IMPORTANT - : Need help on USB port virtualization with Xen
 hypervisor
Content-Language: en-GB
To: GOURLOT Francois <francois.gourlot@thalesgroup.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Kelly Choi <kelly.choi@cloud.com>, Jan Beulich <jbeulich@suse.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Juergen Gross <jgross@suse.com>
References: <d2de4ae9ecb34efc962dea7f8b4e7cbd@thalesgroup.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <d2de4ae9ecb34efc962dea7f8b4e7cbd@thalesgroup.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Dominique and Francois,

Thank you for the report. I am CCing some person who I think could help.

I also have some questions (see below).

On 23/02/2024 16:00, GOURLOT Francois wrote:
> Dear All,
> 
> We send you a message few days ago.
> 
> We have major performance with XEN USB Drivers. We use your driver to 
> load data in a device and we need USB3 High Speed USB protocole
> 
> Do you have a new issue of the HCI and NEC USB Xen Driver ?

[...]

> *De :*LARRIEU Dominique <dominique.larrieu@thalesgroup.com>
> *Envoyé :* jeudi 22 février 2024 13:58
> *À :* xen-devel@lists.xenproject.org
> *Cc :* WILLEMS Louis <louis.willems@thalesgroup.com>; GRUO Nicolas 
> <nicolas-n.gruo@thalesgroup.com>; GOURLOT Francois 
> <francois.gourlot@thalesgroup.com>; Kelly Choi <kelly.choi@cloud.com>
> *Objet :* Need help on USB port virtualization with Xen hypervisor
> *Critère de diffusion :* Confidentiel
> 
>     Dear all,
> 
>     We are detectingseveral issues with USB port virtualization with the
>     Xen hypervisor.

Just to clarify, you are assigning the PCI USB bus (rather than just the 
USB device) to the guest. Is that correct?

If so...

 >
 > Do you have a new issue of the HCI and NEC USB Xen Driver ?

... I am not sure what Xen driver you are talking about. If the USB bus 
is assigned to the guest, then I would expect no specific Xen driver are 
required. Do you have any pointer to the existing one?

>     - We cannot do PCI passthrough of the PCI usb bus on a Windows 10
>     1607 64-bit virtual machine. The bad result is a Windows blue screen.
> 
>     - When we use the passthrough functionality on a Windows 21H2
>     virtual machine, we notice thatthe speed of the USB port is not high
>     speed but full speed on a USB 3.0 port
> 
>     - We notice instabilities when using the nec-usb-xhci driver,  USB
>     2.0 keys are not recognized by the Windows virtual machine
>     (incorrect descriptor)

Do you have any logs you could share from the guest OS? Also, would you 
be able to share more details on how you configure the guest? (e.g. xl 
configuration, libvirt...)?

> 
>     We needyour helpto find a solution for these problems.
> 
>     The Software used are :

Can you share some details of the HW you are using?

> 
>     -Debian 11 version5.10.0-20
> 
>     -Xen version4.14

Are minor version of Xen 4.14 are you using? Any patches on top?

Xen 4.14 has also been out of support for a few months now. If that's 
possible, I would recommend to try a new Xen version (the last stable is 
4.18) just in case your issue as been fixed in newer release.

> 
>     -Windows 10 1607 and 21H2 for virtual machines. Virtual Machine HVM

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 14:19:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 14:19:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685086.1065437 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdss0-0002kk-WB; Sat, 24 Feb 2024 14:19:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685086.1065437; Sat, 24 Feb 2024 14:19:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdss0-0002kd-TS; Sat, 24 Feb 2024 14:19:16 +0000
Received: by outflank-mailman (input) for mailman id 685086;
 Sat, 24 Feb 2024 14:19:15 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdsrz-0002kT-PZ; Sat, 24 Feb 2024 14:19:15 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdsrz-000700-MH; Sat, 24 Feb 2024 14:19:15 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdsrz-00031i-Au; Sat, 24 Feb 2024 14:19:15 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdsrz-0002xk-AS; Sat, 24 Feb 2024 14:19:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=P8JESzndlLePrIZccpCq43bKPP4VKFHK8Mfj7HOmjH8=; b=Rsih9hqPNFxdRTJWsOy/Ti4KZd
	5PsQLM7EEqpEGZLtebnp5Mw9LfdgbLI1yNXEAWXwWzt6Us9JeEwPdUNoxxYEc0gOErUUI63l7QXDL
	5bhhLebiSlOVruHKhgBPqY2WSXF3TZAZ7bw6lGPzCLEXjx4bKfvNLPVjG68cpZ7E4gzI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184746-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184746: regressions - FAIL
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt-raw:guest-start/debian.repeat:fail:regression
    xen-unstable:test-armhf-armhf-libvirt-raw:guest-start:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=f5e1c527d0a0d09ca0cb1dcd8d4ab4a1a5261e91
X-Osstest-Versions-That:
    xen=812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 24 Feb 2024 14:19:15 +0000

flight 184746 xen-unstable real [real]
flight 184750 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184746/
http://logs.test-lab.xenproject.org/osstest/logs/184750/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt-raw 17 guest-start/debian.repeat fail REGR. vs. 184732

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-raw 13 guest-start      fail in 184750 pass in 184746
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184750-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184732
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184732
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184732
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184732
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184732
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184732
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184732
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184732
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184732
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184732
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184732
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184732
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 xen                  f5e1c527d0a0d09ca0cb1dcd8d4ab4a1a5261e91
baseline version:
 xen                  812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b

Last test of basis   184732  2024-02-23 04:57:51 Z    1 days
Testing same since   184746  2024-02-23 23:07:23 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Christian Lindig <christian.lindig@cloud.com>
  Edwin Török <edwin.torok@cloud.com>
  George Dunlap <george.dunlap@cloud.com> # Golang bits
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Petr Beneš <w1benny@gmail.com>
  Samuel Thibault <samuel.thibault@ens-lyon.org>
  Shawn Anastasio <sanastasio@raptorengineering.com>
  Tamas K Lengyel <tamas@tklengyel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 fail    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit f5e1c527d0a0d09ca0cb1dcd8d4ab4a1a5261e91
Author: Juergen Gross <jgross@suse.com>
Date:   Fri Feb 16 17:31:39 2024 +0100

    tools/xenstored: have a single do_control_memreport()
    
    With 9pfs now available in Xenstore-stubdom, there is no reason to
    have distinct do_control_memreport() variants for the daemon and the
    stubdom implementations.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

commit a76f8809f10f1764d4627844461906dac93e68df
Author: Juergen Gross <jgross@suse.com>
Date:   Fri Feb 16 17:31:38 2024 +0100

    tools/xenstored: support complete log capabilities in stubdom
    
    With 9pfs being fully available in Xenstore-stubdom now, there is no
    reason to not fully support all logging capabilities in stubdom.
    
    Open the logfile on stubdom only after the 9pfs file system has been
    mounted.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>

commit 93892648ffaf78a682edef48ea2d108650b3b547
Author: Juergen Gross <jgross@suse.com>
Date:   Fri Feb 16 17:31:37 2024 +0100

    tools/xenstored: add helpers for filename handling
    
    Add some helpers for handling filenames which might need different
    implementations between stubdom and daemon environments:
    
    - expansion of relative filenames (those are not really defined today,
      just expand them to be relative to /var/lib/xen/xenstore)
    - expansion of xenstore_daemon_rundir() (used e.g. for saving the state
      file in case of live update - needs to be unchanged in the daemon
      case, but should result in /var/lib/xen/xenstore for stubdom)
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit 297a2587e6b5ab2edbadce90cec0d5192625973c
Author: Juergen Gross <jgross@suse.com>
Date:   Fri Feb 16 17:31:36 2024 +0100

    tools/xenstored: mount 9pfs device in stubdom
    
    Mount the 9pfs device in stubdom enabling it to use files.
    
    This has to happen in a worker thread in order to allow the main thread
    handling the required Xenstore accesses in parallel.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Julien Grall <jgrall@amazon.com>

commit e21d9dbf4457a8c1f42d6e4a2ef2d3fe7c62616b
Author: Juergen Gross <jgross@suse.com>
Date:   Fri Feb 16 17:31:35 2024 +0100

    tools: add 9pfs device to xenstore-stubdom
    
    Add a 9pfs device to Xenstore stubdom in order to allow it to do e.g.
    logging into a dom0 file.
    
    Use the following parameters for the new device:
    
    - tag = "Xen"
    - type = "xen_9pfsd"
    - path = "/var/lib/xen/xenstore"
    - security-model = "none"
    
    For now don't limit allowed file space or number of files.
    
    Add a new libxl function for adding it similar to the function for
    adding the console device.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

commit 0b761781c4de2e6637203d0f51273a5a1f8873df
Author: Juergen Gross <jgross@suse.com>
Date:   Fri Feb 16 17:31:34 2024 +0100

    stubdom: extend xenstore stubdom configs
    
    Extend the config files of the Xenstore stubdoms to include XENBUS
    and 9PFRONT items in order to support file based logging.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

commit 4287a71f9b9d6a7dc1e51ae89b5ae04e1ba49d7c
Author: Juergen Gross <jgross@suse.com>
Date:   Fri Feb 16 17:31:33 2024 +0100

    tools/xl: support new 9pfs backend xen_9pfsd
    
    Add support for the new 9pfs backend "xen_9pfsd". For this backend type
    the tag defaults to "Xen" and the host side path to
    "/var/log/xen/guests/<dom-name>".
    
    Do most of the default settings in libxl. Unfortunately the default
    path can't easily be set in libxl, as the domain name isn't available
    in the related 9pfs specific function.
    
    Settings the defaults in libxl requires to move the sanity checking
    of 9pfs parameters from xl to libxl, too.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

commit ab5a6b598651279ae9c47c22bbb0c072ad60d878
Author: Juergen Gross <jgross@suse.com>
Date:   Fri Feb 16 17:31:32 2024 +0100

    tools/libs/light: add backend type for 9pfs PV devices
    
    Make the backend type of 9pfs PV devices configurable. The default is
    "qemu" with the related Xenstore backend-side directory being "9pfs".
    
    Add another type "xen_9pfsd" with the related Xenstore backend-side
    directory "xen_9pfs".
    
    As additional security features it is possible to specify:
    - "max-space" for limiting the maximum space consumed on the filesystem
      in MBs
    - "max-files" for limiting the maximum number of files in the
      filesystem
    - "max-open-files" for limiting the maximum number of concurrent open
      files
    
    For convenience "auto-delete" is available to let the backend delete the
    oldest file of the guest in case otherwise "max-space" or "max-files"
    would be violated.
    
    The xen-9pfsd daemon will be started by libxenlight automatically when
    the first "xen_9pfs" device is being created.
    
    Signed-off-by: Juergen Gross <jgross@suse.com>
    Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
    Acked-by: George Dunlap <george.dunlap@cloud.com> # Golang bits
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

commit 098d868e52ac0165b7f36e22b767ea70cef70054
Author: Edwin Török <edwin.torok@cloud.com>
Date:   Wed Jan 31 10:52:56 2024 +0000

    tools/oxenstored: Make Quota.t pure
    
    Now that we no longer have a hashtable inside we can make Quota.t pure, and
    push the mutable update to its callers.  Store.t already had a mutable Quota.t
    field.
    
    No functional change.
    
    Signed-off-by: Edwin Török <edwin.torok@cloud.com>
    Acked-by: Christian Lindig <christian.lindig@cloud.com>

commit b6cf604207fd0a04451a48f2ce6d05fb66c612ab
Author: Edwin Török <edwin.torok@cloud.com>
Date:   Wed Jan 31 10:52:55 2024 +0000

    tools/oxenstored: Use Map instead of Hashtbl for quotas
    
    On a stress test running 1000 VMs flamegraphs have shown that
    `oxenstored` spends a large amount of time in `Hashtbl.copy` and the GC.
    
    Hashtable complexity:
     * read/write: O(1) average
     * copy: O(domains) -- copying the entire table
    
    Map complexity:
     * read/write: O(log n) worst case
     * copy: O(1) -- a word copy
    
    We always perform at least one 'copy' when processing each xenstore
    packet (regardless whether it is a readonly operation or inside a
    transaction or not), so the actual complexity per packet is:
      * Hashtbl: O(domains)
      * Map: O(log domains)
    
    Maps are the clear winner, and a better fit for the immutable xenstore
    tree.
    
    Signed-off-by: Edwin Török <edwin.torok@cloud.com>
    Acked-by: Christian Lindig <christian.lindig@cloud.com>

commit 7e5b662b289ac66ee347113f94c46c89d127e774
Author: Petr Beneš <w1benny@gmail.com>
Date:   Tue Feb 6 10:08:22 2024 +0000

    x86/altp2m: p2m_altp2m_get_or_propagate() should honor ap2m->default_access
    
    This patch addresses a behavior discrepancy in the handling of altp2m views,
    where upon the creation and subsequent EPT violation, the page access
    permissions were incorrectly inherited from the hostp2m instead of respecting
    the altp2m default_access.
    
    Previously, when a new altp2m view was established with restrictive
    default_access permissions and activated via xc_altp2m_switch_to_view(),
    it failed to trigger an event on the first access violation.  This behavior
    diverged from the intended mechanism, where the altp2m's default_access
    should dictate the initial permissions, ensuring proper event triggering on
    access violations.
    
    The correction involves modifying the handling mechanism to respect the
    altp2m view's default_access upon its activation, eliminating the need for
    setting memory access permissions for the entire altp2m range (e.g. within
    xen-access.c).  This change not only aligns the behavior with the expected
    access control logic but also results in a significant performance improvement
    by reducing the overhead associated with setting memory access permissions
    across the altp2m range.
    
    Signed-off-by: Petr Beneš <w1benny@gmail.com>
    Acked-by: Tamas K Lengyel <tamas@tklengyel.com>

commit 98e79be5b60ac5f4fbcb064a9e11c30024929ff1
Author: Andrew Cooper <andrew.cooper3@citrix.com>
Date:   Mon Jan 29 11:39:32 2024 +0000

    xen/*/asm-offset: Fix bad copy&paste from x86
    
    All architectures have copy&pasted bad logic from x86.
    
    OFFSET() having a trailing semi-colon within the macro expansion can be a
    problematic pattern.  It's benign in this case, but fix it anyway.
    
    Perform style fixes for the other macros, and tame the mess of BLANK()
    position to be consistent (one BLANK() after each block) so the intermediate
    form is legible too.
    
    No functional change.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com>
    Reviewed-by: Jan Beulich <jbeulich@suse.com>
    Acked-by: Julien Grall <jgrall@amazon.com>
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 15:37:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 15:37:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685100.1065451 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdu5D-0002zs-Pu; Sat, 24 Feb 2024 15:36:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685100.1065451; Sat, 24 Feb 2024 15:36:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdu5D-0002zl-NE; Sat, 24 Feb 2024 15:36:59 +0000
Received: by outflank-mailman (input) for mailman id 685100;
 Sat, 24 Feb 2024 15:36:58 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdu5C-0002zb-Fr; Sat, 24 Feb 2024 15:36:58 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdu5C-00005P-Cw; Sat, 24 Feb 2024 15:36:58 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdu5C-0005Oy-0b; Sat, 24 Feb 2024 15:36:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdu5C-0005Ab-06; Sat, 24 Feb 2024 15:36:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=x2o82TiMfOVeqYphLO19sKn8xD5epgGMEKjzAGMEr6s=; b=qF7i0b/XLPiNBukf82GvnncE+M
	ZCigmQ9tcOCWAiPNslAqJfCe1HRJzz9kiy1vnOCo7B5xOrhNlvG9kMtLkUbti7B4ptfIsde3GJbQT
	x4XVtgCGB5j9KeM7RiwCFOSOKcswmF57xANNIuZPzdJwGU3MBSUO8fn89S5esoVUivH8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184751-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184751: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=7fa4a984c438b897086f5e2fea6b9e9ad17867c4
X-Osstest-Versions-That:
    ovmf=11ad164bcea6b0ed3628d595090f84892c367086
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 24 Feb 2024 15:36:58 +0000

flight 184751 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184751/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 7fa4a984c438b897086f5e2fea6b9e9ad17867c4
baseline version:
 ovmf                 11ad164bcea6b0ed3628d595090f84892c367086

Last test of basis   184736  2024-02-23 13:13:11 Z    1 days
Testing same since   184751  2024-02-24 13:41:24 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Gua Guo <gua.guo@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   11ad164bce..7fa4a984c4  7fa4a984c438b897086f5e2fea6b9e9ad17867c4 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 18:30:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 18:30:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685111.1065461 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdwmT-00046I-V7; Sat, 24 Feb 2024 18:29:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685111.1065461; Sat, 24 Feb 2024 18:29:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdwmT-00046B-QT; Sat, 24 Feb 2024 18:29:49 +0000
Received: by outflank-mailman (input) for mailman id 685111;
 Sat, 24 Feb 2024 18:29:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Y+9+=KB=epam.com=prvs=8784076af6=oleksandr_tyshchenko@srs-se1.protection.inumbo.net>)
 id 1rdwmS-000465-AO
 for xen-devel@lists.xenproject.org; Sat, 24 Feb 2024 18:29:48 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ad1f204d-d342-11ee-98f5-efadbce2ee36;
 Sat, 24 Feb 2024 19:29:44 +0100 (CET)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id
 41O4Qn1V010989; Sat, 24 Feb 2024 18:29:38 GMT
Received: from eur05-vi1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2168.outbound.protection.outlook.com [104.47.17.168])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3wf9edh1nx-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Sat, 24 Feb 2024 18:29:38 +0000 (GMT)
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com (2603:10a6:10:ed::15)
 by AM7PR03MB6213.eurprd03.prod.outlook.com (2603:10a6:20b:133::20) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.33; Sat, 24 Feb
 2024 18:29:33 +0000
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167]) by DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167%4]) with mapi id 15.20.7316.031; Sat, 24 Feb 2024
 18:29:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad1f204d-d342-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=E9nkhVCNt/mo3BBKIs2B4gA8+51wEKwyiNfyJGmppjV6+hN4ySYUErzRF0ZVftRIkbygrhIlb4q87BzcHbjr/DbOpueIpAvlbz/W8kB8KpFJ4vxyKBioTMneTcufYsulSNpMQ1Q0MinjwHguBuK81d009kABoANIAOzkjbFbsa6Z14F3Ij77IlMJgTsCOs+HVNHPxTBxu2V9Ay6FfkyO7613HICi0CCOIVcOOz+UzC2KCnEAb20rqG3ttQ1XOrbhOxE0+3VRLTRKzoNQjAAHjbS/QPHsYwZbdYrFQtwMIvYJQljtkXwVK1vX6dUQYBUlyP/Eh3lQ0AV1AYYbbEA/aw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=kRqDv77eX0nJWM6k/FgQqWszLWJKFqMdp3Sby4DuzZI=;
 b=BsfagtrVYNzNmygwbtiqWLDowpP28VG49XMdYmM/zjI8kMWtSj8V4sYc4QCX+gDDBNqYG9nmk10GAco6OLyslFWR6ZyYwefVIJB1XTh3c+2RuWw8q7FhxscnCiclZTTECYU5vDAmNS8oiktTX5m3KEIfWlA80RDwgHTXpDv0kq/G9evJ0ad+Cv9sJn8oQ1fDNj9JfWzrhVV2hR6Jyvu2o0Wrg7JFAUF8qzIrhXlrvFJQl4/ENusld6suSjfKI6ggX0yF4CEKwXvhcVsFfqYCINUIdUcWWWK42Z48boMwdT1VLApL1KTIKWPr+uqCdS21l4kFpctxfUgtE0gv1W+dcw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=kRqDv77eX0nJWM6k/FgQqWszLWJKFqMdp3Sby4DuzZI=;
 b=aoUNlt1UM3thSudUOU6h2gRHabLe5Q661ybfWWBh+uuray/ur3/yRGWFNpCAzYkxX7Z142/ulfoP2zANatOYG+eb+Y+KjO2aWOcNMIE1xhQS36RSei+s6RirjepzrYLB+Td1mVB/zKmle3s+TIm8VSO3P8LNQ7ykJfAmV7aSIIrlYx20hkfXxU5lFDu63YjgkBtvfXDmYD2OOz9zP6rmvjq0Qj3tz8xUQ8R+vVSy1d4agprApHoMGUlx5g2hd6OGazUTSos2H5HMq9tXRq0WjgYbirx0wMMJoftvF8wv7aHvBxegDFeUKHkNxjd6aOuqJOE3eLhNHort1OXhy0PGIA==
From: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
To: Stefano Stabellini <stefano.stabellini@amd.com>,
        Peng Fan
	<peng.fan@nxp.com>
CC: Julien Grall <julien@xen.org>,
        "vikram.garhwal@amd.com"
	<vikram.garhwal@amd.com>,
        Leo Yan <leo.yan@linaro.org>, Viresh Kumar
	<viresh.kumar@linaro.org>,
        Stewart Hildebrand <stewart.hildebrand@amd.com>,
        "sgarzare@redhat.com" <sgarzare@redhat.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: Re: question about virtio-vsock on xen
Thread-Topic: question about virtio-vsock on xen
Thread-Index: AdpmUSXKZvg8Aft6RRu9smUwoggtbwAUAHAAACuPlYA=
Date: Sat, 24 Feb 2024 18:29:32 +0000
Message-ID: <ee69684a-5aad-417a-8522-1cd1b5322bbe@epam.com>
References: 
 <DU0PR04MB941734DA793D87B7FF3A491488552@DU0PR04MB9417.eurprd04.prod.outlook.com>
 <alpine.DEB.2.22.394.2402231332580.754277@ubuntu-linux-20-04-desktop>
In-Reply-To: 
 <alpine.DEB.2.22.394.2402231332580.754277@ubuntu-linux-20-04-desktop>
Accept-Language: en-US, ru-RU
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DB8PR03MB6108:EE_|AM7PR03MB6213:EE_
x-ms-office365-filtering-correlation-id: 437c4e30-1d0e-4cb9-2b22-08dc35668be7
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 qjoZd5/TfrsEGLs4nO/21NIOmZ3lxRMRs7E07pN4zP2591YR6G3HdtQyRQkR86jkzVFE3LDwOvef4uCNjjBnDeKb2Z+GtJV5wrNRYKuQSuTfK14fmMezvOY4sxBBVqsZMWgb5NHHKnm4HF3UnphPqhLpDipYzKhniWKuBPsVAKJ+qTlz7wwUZu0qTjOcFBa4kUCDwNH7TwLwjqs9NkLsig294sHiL/ACeMSedk8lVbBWB1AvpZK4hVAlfX0huou3oV+s15v9jLvSynYp2u2+rEPizim7ggMtRPNtCWemldOsO8qXTMrpVas13Xhh444x81P+CxfFLLLPtzWAgPdC3qiNnUiiuxJuVJBHpnQqrczagJQCp9EnbBzrxmKGCBnAYK8BCsbBSdxcByWOrboqrLYCpWq2OCSZvsOU2qmgR4V92M/80sj4d93J824q2rHINsLnEgyknPyvbvztq1jKDvb1g9T4xw+GUlp/auQp5jjMeCvhkFuT/KtEv55jbmC59fc9Ui1BBHaoHtzHyipkDl5DSETfSHFvLl/vYRJB+mG1XLbsslX4NjUwHNw6DaHqWimc1qK0xMM2QvH1Rqzw+sCjngOqZG54btfz+98bWEg=
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR03MB6108.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?NzAvNXhDVTIzNlRyQk1xZk5va3BrbnlYUXpJbDBYZUNaRktnUVRoVGEzWFp6?=
 =?utf-8?B?Zkw4YllUOFlFMlRYVGJUbkpFOExNOG1OQ0xiUU1tYnJaWlRrU0JOSExSemZt?=
 =?utf-8?B?Q0FnNzhjbHE4OE1HSGxFYUcwbkl3SWRtV0JUZVFRWHBLVTlNbkJYNlZOcDYz?=
 =?utf-8?B?eVJwQWo5YjF3OVZReWgxVUo5dVVtWDNlbmM2UVlBaER3eVkxL2JPM0FOQkpR?=
 =?utf-8?B?eWdPUW13cVZ2RS9oTnIvKzlsVTdFc1dabnpncHNjR3I2WFZVcmhHajI3ZzRU?=
 =?utf-8?B?SzBNeUpIZ0s5REFObHByQ21ZU0pIUUE4clFIL1RqeWhJekJwcVphK1pRTTVa?=
 =?utf-8?B?dEUxc3ppbGloOGxNeERqb2JQV3l3blphcmtJamVzVGdLZ05KK3hqa2dFdTZ1?=
 =?utf-8?B?NEFXWXFCY2F5OXkyaDNJVFBCQ0w0NUdqUWhsQ05XNEdKZ0tETnJDWTRFQUNJ?=
 =?utf-8?B?TjFqUVk1dWkxQXpkaWkvQitLN3ZVQWd4V2VxN3JwTkI0L0VkUis5M1dFTGE2?=
 =?utf-8?B?SzhZTTkwMm54RlNadVNXaHRVWXdlcnRZb3djOG9MUXkwcjBOenNsUG5vVmls?=
 =?utf-8?B?UHlMMi9BUk9sRTR5R0ZKeDZiZVl3RWpLSDRYN1BIamJRU00vN3VKMkc2UU1H?=
 =?utf-8?B?QmhMY2dRN21va1lKQThJT1FubzlTYWNpTFBhcll2bUpEVGNPd2RrbkxQTTN2?=
 =?utf-8?B?YXhaT3FaenZkLzBXNUUrUXBBdjYzM0FvSE1HYTNRVTdxTHRvblNtMzZMSHlX?=
 =?utf-8?B?MFRhQTIvY0Jsc0V4MWJwdGNTRi84MTR0NUwwLzFEQmJRajI5N1hBZnd4VFN2?=
 =?utf-8?B?UHBiMUlPeVpNc29XVXg3a05uSTNYVERmS09lY2t2Ui9ReUVCbXNlUGI2aTJ5?=
 =?utf-8?B?eVFJR3FFVDhzTE1tRytpZm9LLzA0RG5mUGVtM3lzcGRCeDNTZEpaUmIvU0M0?=
 =?utf-8?B?cXgxaWx6VUJvdTVqd2Erc1UyWmNMRjI1WitnUlFZa3ZKUXJuMEV6eWZCWjhk?=
 =?utf-8?B?WFZlc2trOEZFVGU2RjkyNmpXbmI4bWhlRmIrUVpSc2p4dHhTbTNVUkRQQUcr?=
 =?utf-8?B?Q2lNWjR3ZGtpMW12YUYrU2wxWTFEYnlNRWxieEFoS2dNdERMM09sT2JUU0RN?=
 =?utf-8?B?YWRyaDBvejk4cmRhbTFwRVRRYzkrZkw3aVlYZ2pMMDBQVWxzMENFN2VNQlAr?=
 =?utf-8?B?b2xaWDEyNTUvZWYxNnBWVFBzTzVQZ3dpRG1GeEpQUU1JQWhOdkQ2REhXS2hx?=
 =?utf-8?B?TlhnQlB6QiswOTF5QmhQamIvWmRuV3pPSGltaGdmMmZtdXI3RlBiVVlGdUNI?=
 =?utf-8?B?azduSkhhRVhKdW9VNmxmL2hSdlgwZXJQZUZWRUpEcmM5U0JsSythRjNDMEdo?=
 =?utf-8?B?bUVYRkszVm51eXlEYnQwbkpuL0Q0Z21ac3NNdlg3RHpvcm9XR0F4V2l4T3R5?=
 =?utf-8?B?aDVreDMxS1FsS0lFUHkyU3ZXSnlZRGV4a2FTRFZuellhRXBnZ3lzRHk1SXM1?=
 =?utf-8?B?U3hkamJHaWx6V3BkVlFHOHBDOTBCR2k2amp1S05hMnpmdGtvR2hpNWgrVGZ6?=
 =?utf-8?B?eWFJUHdwY3hKV2FHTmx6TllhN25FVFV2V0tnSzlYVUp5TEdZZGgrWEMyYmxa?=
 =?utf-8?B?aENrcy9iWWliVTNxNmpMQ24yYzd6QlFGUDRYNUtVWlMxM29GVXFaa3ZmbzRW?=
 =?utf-8?B?MVdraURBZW5sRkx2NmtnTzF4V0Z4L2JLSktFQ3ZkVDJMT1Zta0Rhc1ZhL21O?=
 =?utf-8?B?dFk2ODYvZGMzTmV4L1FvOW9uWm9QWlFpMFcyZjl0NFlmVWV6SGVJNU5WZ1Nz?=
 =?utf-8?B?V0pwVjE5SkhWbVBTeUNQOWV6dzM3bEwrbE01QTdFamN5dkxIV3d3cVpIU1Yx?=
 =?utf-8?B?TlpicnVDWmY1WXdSVERtU1QvV0xMazMrOUppNkNoREtFM08zSTdQeEtVV2U5?=
 =?utf-8?B?Ukg3cnFUQjhwLzNETDZlTzdxVVdWVXNYRGp2MlhRZkVrZVJJWittTzRka2t6?=
 =?utf-8?B?N2I1NWY4cHFmK3dwK1dJaWU5RVZEV1FjaUJnV0djRDg2THcwSHlZcjZwTkF4?=
 =?utf-8?B?OXJNai9LZUhRYjZCRTd0dmxrNElvSy9qWGUxcG94VjdxMnZPcHdpOXJnYzVu?=
 =?utf-8?B?eC9pUXI5UytnQzRNVkl4TDlNM2ZUVjJ5YkhOZExqU2lQMHB1Yzllc3lUSWJp?=
 =?utf-8?Q?p6jgiS0AZuSD5Ff7/07wyBA=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <5704C126A50E5843859A76BC896C5010@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB8PR03MB6108.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 437c4e30-1d0e-4cb9-2b22-08dc35668be7
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Feb 2024 18:29:32.9821
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: z07OaLiuzJu85eQNbr7+rcRbwgb8V3pf+OPxB3KWnO7l36k+Jn5QnUac74tvvIZg9BLC6/3WpiPq0jN+D8ePvP+EYQSiG1pohEGrD4Xi730=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR03MB6213
X-Proofpoint-GUID: kPyHaw7B8u5j66o8FAhKxxrKOYoKvH2X
X-Proofpoint-ORIG-GUID: kPyHaw7B8u5j66o8FAhKxxrKOYoKvH2X
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-24_13,2024-02-23_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0
 spamscore=0 phishscore=0 adultscore=0 impostorscore=0 bulkscore=0
 suspectscore=0 clxscore=1011 malwarescore=0 mlxlogscore=999
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2402120000 definitions=main-2402240155

DQoNCk9uIDIzLjAyLjI0IDIzOjQyLCBTdGVmYW5vIFN0YWJlbGxpbmkgd3JvdGU6DQo+IEhpIFBl
bmcsDQoNCkhlbGxvIFBlbmcsIFN0ZWZhbm8NCg0KDQo+IA0KPiBXZSBoYXZlbid0IHRyaWVkIHRv
IHNldHVwIHZpcnRpby12c29jayB5ZXQuDQo+IA0KPiBJbiBnZW5lcmFsLCBJIGFtIHZlcnkgc3Vw
cG9ydGl2ZSBvZiB1c2luZyBRRU1VIGZvciB2aXJ0aW8gYmFja2VuZHMuIFdlDQo+IHVzZSBRRU1V
IHRvIHByb3ZpZGUgdmlydGlvLW5ldCwgdmlydGlvLWJsb2NrLCB2aXJ0aW8tY29uc29sZSBhbmQg
bW9yZS4NCj4gDQo+IEhvd2V2ZXIsIHR5cGljYWxseSB2aXJ0aW8tdnNvY2sgY29tZXMgaW50byBw
bGF5IGZvciBWTS10by1WTQ0KPiBjb21tdW5pY2F0aW9uLCB3aGljaCBpcyBkaWZmZXJlbnQuIEdv
aW5nIHZpYSBRRU1VIGluIERvbTAganVzdCB0byBoYXZlIDENCj4gVk0gY29tbXVuaWNhdGUgd2l0
aCBhbm90aGVyIFZNIGlzIG5vdCBhbiBpZGVhbCBkZXNpZ246IGl0IGFkZHMgbGF0ZW5jeQ0KPiBh
bmQgdXNlcyByZXNvdXJjZXMgaW4gRG9tMCB3aGVuIGFjdHVhbGx5IHdlIGNvdWxkIGRvIHdpdGhv
dXQgaXQuDQo+IA0KPiBBIGJldHRlciBtb2RlbCBmb3IgVk0tdG8tVk0gY29tbXVuaWNhdGlvbiB3
b3VsZCBiZSB0byBoYXZlIHRoZSBWTSB0YWxrDQo+IHRvIGVhY2ggb3RoZXIgZGlyZWN0bHkgdmlh
IGdyYW50IHRhYmxlIG9yIHByZS1zaGFyZWQgbWVtb3J5IChzZWUgdGhlDQo+IHN0YXRpYyBzaGFy
ZWQgbWVtb3J5IGZlYXR1cmUpIG9yIHZpYSBYZW4gaHlwZXJjYWxscyAoc2VlIEFyZ28uKQ0KPiAN
Cj4gRm9yIGEgZ29vZCBYZW4gZGVzaWduLCBJIHRoaW5rIHRoZSB2aXJ0aW8tdnNvY2sgYmFja2Vu
ZCB3b3VsZCBuZWVkIHRvIGJlDQo+IGluIFhlbiBpdHNlbGYgKHRoZSBoeXBlcnZpc29yKS4NCj4g
DQo+IE9mIGNvdXJzZSB0aGF0IGlzIG1vcmUgd29yayBhbmQgaXQgZG9lc24ndCBoZWxwIHlvdSB3
aXRoIHRoZSBzcGVjaWZpYw0KPiBxdWVzdGlvbiB5b3UgaGFkIGJlbG93IDotKQ0KPiANCj4gRm9y
IHRoYXQsIEkgZG9uJ3QgaGF2ZSBhIHBvaW50ZXIgdG8gaGVscCB5b3UgYnV0IG1heWJlIG90aGVy
cyBpbiBDQw0KPiBoYXZlLg0KDQoNClllcywgSSB3aWxsIHRyeSB0byBwcm92aWRlIHNvbWUgaW5m
byAuLi4NCg0KDQo+IA0KPiBDaGVlcnMsDQo+IA0KPiBTdGVmYW5vDQo+IA0KPiANCj4gT24gRnJp
LCAyMyBGZWIgMjAyNCwgUGVuZyBGYW4gd3JvdGU6DQo+PiBIaSBBbGwsDQo+Pg0KPj4gSGFzIGFu
eW9uZSBtYWtlIHZpcnRpby12c29jayBvbiB4ZW4gd29yaz8gTXkgZG0gYXJncyBhcyBiZWxvdzoN
Cj4+DQo+PiB2aXJ0aW8gPSBbDQo+PiAnYmFja2VuZD0wLHR5cGU9dmlydGlvLGRldmljZSx0cmFu
c3BvcnQ9cGNpLGJkZj0wNTowMC4wLGJhY2tlbmRfdHlwZT1xZW11LGdyYW50X3VzYWdlPXRydWUn
DQo+PiBdDQo+PiBkZXZpY2VfbW9kZWxfYXJncyA9IFsNCj4+ICctRCcsICcvaG9tZS9yb290L3Fl
bXVfbG9nLnR4dCcsDQo+PiAnLWQnLCAndHJhY2U6KnZzb2NrKix0cmFjZToqdmhvc3QqLHRyYWNl
Oip2aXJ0aW8qLHRyYWNlOipwY2lfdXBkYXRlKix0cmFjZToqcGNpX3JvdXRlKix0cmFjZToqaGFu
ZGxlX2lvcmVxKix0cmFjZToqeGVuKicsDQo+PiAnLWRldmljZScsICd2aG9zdC12c29jay1wY2ks
aW9tbXVfcGxhdGZvcm09ZmFsc2UsaWQ9dmhvc3QtdnNvY2stcGNpMCxidXM9cGNpZS4wLGFkZHI9
NS4wLGd1ZXN0LWNpZD0zJ10NCj4+DQo+PiBEdXJpbmcgbXkgdGVzdCwgaXQgYWx3YXlzIHJldHVy
biBmYWlsdXJlIGluIGRvbTAga2VybmVsIGluIGJlbG93IGNvZGU6DQo+Pg0KPj4gdmhvc3RfdHJh
bnNwb3J0X2RvX3NlbmRfcGt0IHsNCj4+IC4uLg0KPj4gICAgICAgICAgICAgICAgIG5ieXRlcyA9
IGNvcHlfdG9faXRlcihoZHIsIHNpemVvZigqaGRyKSwgJmlvdl9pdGVyKTsNCj4+ICAgICAgICAg
ICAgICAgICAgaWYgKG5ieXRlcyAhPSBzaXplb2YoKmhkcikpIHsNCj4+ICAgICAgICAgICAgICAg
ICAgICAgICAgICB2cV9lcnIodnEsICJGYXVsdGVkIG9uIGNvcHlpbmcgcGt0IGhkciAleCAleCAl
eCAlcHhcbiIsIG5ieXRlcywgc2l6ZW9mKCpoZHIpLA0KPj4gX19idWlsdGluX29iamVjdF9zaXpl
KGhkciwgMCksICZpb3ZfaXRlcik7DQo+PiAgICAgICAgICAgICAgICAgICAgICAgICAga2ZyZWVf
c2tiKHNrYik7DQo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgYnJlYWs7DQo+PiAgICAgICAg
ICAgICAgICAgIH0NCj4+IH0NCj4+DQo+PiBJIGNoZWNrZWQgY29weV90b19pdGVyLCBpdCBpcyBj
b3B5IGRhdGEgdG8gX191c2VyIGFkZHIsIGJ1dCBpdCBuZXZlciBwYXNzLA0KPj4gdGhlIGNvcHkg
dG8gX191c2VyIGFkZHIgYWx3YXlzIHJldHVybiAwIGJ5dGVzIGNvcGllZC4NCj4+DQo+PiBUaGUg
YXNtIGNvZGUgInN0dHIgeDcsIFt4Nl0iIHdpbGwgdHJpZ2dlciBkYXRhIGFib3J0LCB0aGUga2Vy
bmVsIHdpbGwgcnVuDQo+PiBpbnRvIGRvX3BhZ2VfZmF1bHQsIGJ1dCBsb2NrX21tX2FuZF9maW5k
X3ZtYSByZXBvcnQgaXQgaXMgVk1fRkFVTFRfQkFETUFQLA0KPj4gdGhhdCBtZWFucyB0aGUgX191
c2VyIGFkZHIgaXMgbm90IG1hcHBlZCwgbm8gdm1hIGhhcyB0aGlzIGFkZHIuDQo+Pg0KPj4gSSBh
bSBub3Qgc3VyZSB3aGF0IG1heSBjYXVzZSB0aGlzLiBBcHByZWNpYXRlIGlmIGFueSBjb21tZW50
cy4NCg0KDQogICAuLi4gUGVuZywgd2UgaGF2ZSB2aG9zdC12c29jayAoYW5kIHZob3N0LW5ldCkg
WGVuIFBvQy4gQWx0aG91Z2ggaXQgaXMgDQpub24tdXBzdHJlYW1hYmxlIGluIGl0cyBjdXJyZW50
IHNoYXBlIChiYXNlZCBvbiBvbGQgTGludXggdmVyc2lvbiwgDQpyZXF1aXJlcyBzb21lIHJld29y
ayBhbmQgcHJvcGVyIGludGVncmF0aW9uLCBtb3N0IGxpa2VseSByZXF1aXJlcyANCmludm9sdmlu
ZyBRZW11IGFuZCBwcm90b2NvbCBjaGFuZ2VzIHRvIHBhc3MgYW4gYWRkaXRpb25hbCBpbmZvIHRv
IA0Kdmhvc3QpLCBpdCB3b3JrcyB3aXRoIExpbnV4IHY1LjEwICsgcGF0Y2hlZCBRZW11IHY3LjAs
IHNvIHlvdSBjYW4gcmVmZXIgDQp0byB0aGUgWW9jdG8gbWV0YSBsYXllciB3aGljaCBjb250YWlu
cyBrZXJuZWwgcGF0Y2hlcyBmb3IgdGhlIGRldGFpbHMgWzFdLg0KDQpJbiBhIG51dHNoZWxsLCBi
ZWZvcmUgYWNjZXNzaW5nIHRoZSBndWVzdCBkYXRhIHRoZSBob3N0IG1vZHVsZSBuZWVkcyB0byAN
Cm1hcCBkZXNjcmlwdG9ycyBpbiB2aXJ0aW8gcmluZ3Mgd2hpY2ggY29udGFpbiBlaXRoZXIgZ3Vl
c3QgZ3JhbnQgYmFzZWQgDQpETUEgYWRkcmVzc2VzIChieSB1c2luZyBYZW4gZ3JhbnQgbWFwcGlu
Z3MpIG9yIGd1ZXN0IHBzZXVkby1waHlzaWNhbCANCmFkZHJlc3NlcyAoYnkgdXNpbmcgWGVuIGZv
cmVpZ24gbWFwcGluZ3MpLiBBZnRlciBhY2Nlc3NpbmcgdGhlIGd1ZXN0IA0KZGF0YSB0aGUgaG9z
dCBtb2R1bGUgbmVlZHMgdG8gdW5tYXAgdGhlbS4NCg0KQWxzbyBub3RlLCBpbiB0aGF0IFBvQyB0
aGUgdGFyZ2V0IG1hcHBpbmcgc2NoZW1lIGlzIGNvbnRyb2xsZWQgdmlhIA0KbW9kdWxlIHBhcmFt
IGFuZCBndWVzdCBkb21haW4gaWQgaXMgcmV0cmlldmVkIGZyb20gdGhlIGRldmljZS1tb2RlbCAN
CnNwZWNpZmljIHBhcnQgaW4gdGhlIFhlbnN0b3JlIChzbyBRZW11L3Byb3RvY29sIGFyZSB1bm1v
ZGlmaWVkKS4gQnV0IHlvdSANCm1pZ2h0IHdhbnQgdG8gbG9vayBhdCBbMl0gYXMgYW4gZXhhbXBs
ZSBvZiB2aG9zdC11c2VyIHByb3RvY29sIGNoYW5nZXMgDQpob3cgdG8gcGFzcyB0aGF0IGFkZGl0
aW9uYWwgaW5mby4NCg0KSG9wZSB0aGF0IGhlbHBzLg0KDQpbMV0gaHR0cHM6Ly9naXRodWIuY29t
L3hlbi10cm9vcHMvbWV0YS14dC12aG9zdC9jb21taXRzL21haW4vDQpbMl0gaHR0cHM6Ly93d3cu
bWFpbC1hcmNoaXZlLmNvbS9xZW11LWRldmVsQG5vbmdudS5vcmcvbXNnOTQ4MzI3Lmh0bWwNCg0K
UC5TLiBNYXkgYW5zd2VyIHdpdGggYSBkZWxheS4NCg0KDQo+Pg0KPj4gQlRXOiBJIHRlc3RlZCBi
bGsgcGNpLCBpdCB3b3Jrcywgc28gdGhlIHZpcnRpbyBwY2kgc2hvdWxkIHdvcmsgb24gbXkgc2V0
dXAuDQo+Pg0KPj4gVGhhbmtzLA0KPj4gUGVuZy4NCj4+


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 19:40:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 19:40:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685116.1065471 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdxsT-0004P9-QA; Sat, 24 Feb 2024 19:40:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685116.1065471; Sat, 24 Feb 2024 19:40:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rdxsT-0004P2-Ld; Sat, 24 Feb 2024 19:40:05 +0000
Received: by outflank-mailman (input) for mailman id 685116;
 Sat, 24 Feb 2024 19:40:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdxsS-0004As-Bc; Sat, 24 Feb 2024 19:40:04 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdxsS-0004q3-6A; Sat, 24 Feb 2024 19:40:04 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rdxsR-0001bk-Ok; Sat, 24 Feb 2024 19:40:03 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rdxsR-0000Av-OA; Sat, 24 Feb 2024 19:40:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ZImTgzsr56Tx9pmhp7uMTzHnt92QfGpaqUXKN3s4XP4=; b=ugHn4+j7VdNGEvrd5MCvsoPGbR
	lMZkTmFZazBqucJZ2Sn+xbVAKwxL4qKu+b80i5hBfZxZNQquv6OLkiidAy7twwDN1zTtrjMDBnW0K
	2luULG3gEPlv1Fs45pvQj8AToAAgCdazhj/ILrEM0yCttahGZs3cMuj3a/trkkXscXXM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184749-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184749: regressions - trouble: broken/fail/pass
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:<job status>:broken:regression
    linux-linus:test-amd64-amd64-xl-pvhv2-intel:host-install(5):broken:regression
    linux-linus:test-armhf-armhf-xl-multivcpu:xen-boot:fail:regression
    linux-linus:test-amd64-amd64-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:regression
    linux-linus:test-armhf-armhf-xl-rtds:guest-start/debian.repeat:fail:allowable
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=603c04e27c3e9891ce7afa5cd6b496bfacff4206
X-Osstest-Versions-That:
    linux=b6d69282db550689ab5980e06eedd23b64584a73
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 24 Feb 2024 19:40:03 +0000

flight 184749 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184749/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvhv2-intel    <job status>                 broken
 test-amd64-amd64-xl-pvhv2-intel  5 host-install(5)     broken REGR. vs. 184738
 test-armhf-armhf-xl-multivcpu  8 xen-boot                fail REGR. vs. 184738
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. vs. 184738

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds    18 guest-start/debian.repeat fail REGR. vs. 184738

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check fail blocked in 184738
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184738
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184738
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184738
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184738
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184738
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184738
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184738
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                603c04e27c3e9891ce7afa5cd6b496bfacff4206
baseline version:
 linux                b6d69282db550689ab5980e06eedd23b64584a73

Last test of basis   184738  2024-02-23 17:42:09 Z    1 days
Testing same since   184749  2024-02-24 10:36:24 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexander Sverdlin <alexander.sverdlin@gmail.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrzej Kacprowski <Andrzej.Kacprowski@intel.com>
  Andy Yan <andyshrk@163.com>
  Anshuman Khandual <anshuman.khandual@arm.com>
  Armin Wolf <W_Armin@gmx.de>
  Arnd Bergmann <arnd@arndb.de>
  Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
  Ashutosh Dixit <ashutosh.dixit@intel.com>
  Benjamin Gray <bgray@linux.ibm.com>
  Chanho Min <chanho.min@lge.com>
  Chengming Zhou <zhouchengming@bytedance.com>
  Chris Li <chrisl@kernel.org>
  Chris Morgan <macromorgan@hotmail.com>
  Christian König <christian.koenig@amd.com>
  Colin Ian King <colin.i.king@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Danilo Krummrich <dakr@redhat.com>
  Dave Airlie <airlied@redhat.com>
  David Hildenbrand <david@redhat.com>
  Erik Kurzinger <ekurzinger@nvidia.com>
  Fabio Estevam <festevam@denx.de>
  Florian Fainelli <florian.fainelli@broadcom.com> #Broadcom
  Geert Uytterhoeven <geert+renesas@glider.be>
  Guenter Roeck <linux@roeck-us.net>
  Heiko Carstens <hca@linux.ibm.com>
  Heiko Stuebner <heiko.stuebner@cherry.de>
  Heiko Stuebner <heiko@sntech.de>
  Helge Deller <deller@gmx.de>
  Hojin Nam <hj96.nam@samsung.com>
  Jacek Lawrynowicz <jacek.lawrynowicz@linux.intel.com>
  Jason Gunthorpe <jgg@nvidia.com>
  Javier Martinez Canillas <javierm@redhat.com>
  Johan Jonker <jbx6244@gmail.com>
  Johannes Weiner <hannes@cmpxchg.org>
  Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
  José Roberto de Souza <jose.souza@intel.com>
  Kairui Song <kasong@tencent.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lewis Huang <lewis.huang@amd.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lucas Stach <l.stach@pengutronix.de>
  Ma Jun <Jun.Ma2@amd.com>
  Marek Vasut <marex@denx.de>
  Mark Brown <broonie@kernel.org>
  Martin Blumenstingl <martin.blumenstingl@googlemail.com>
  Matthew Auld <matthew.auld@intel.com>
  Matthew Brost <matthew.brost@intel.com>
  Max Kellermann <max.kellermann@ionos.com>
  Maxime Ripard <mripard@kernel.org>
  Maxime Ripard <mripard@redhat.com>
  Melissa Wen <mwen@igalia.com>
  Michal Hocko <mhocko@suse.com>
  Mike Snitzer <snitzer@kernel.org>
  Mikulas Patocka <mpatocka@redhat.com>
  Neil Armstrong <neil.armstrong@linaro.org>
  Nhat Pham <nphamcs@gmail.com>
  Nikita Shubin <nikita.shubin@maquefel.me>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Peter Oberparleiter <oberpar@linux.ibm.com>
  Rob Herring <robh@kernel.org>
  Rodrigo Siqueira <rodrigo.siqueira@amd.com>
  Rodrigo Vivi <rodrigo.vivi@intel.com>
  Samasth Norway Ananda <samasth.norway.ananda@oracle.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  SeongJae Park <sj@kernel.org>
  Shakeel Butt <shakeel.butt@linux.dev>
  Shakeel Butt <shakeelb@google.com>
  Shawn Guo <shawnguo@kernel.org>
  Simon Ser <contact@emersion.fr>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Steve Morvai <stevemorvai@hotmail.com>
  Swapnil Patel <swapnil.patel@amd.com>
  Terry Tritton <terry.tritton@linaro.org>
  Thierry Reding <treding@nvidia.com>
  Thomas Hellstrom <thomas.hellstrom@linux.intel.com>
  Thomas Hellström <thomas.hellstrom@linux.intel.com>
  Thomas Zimmermann <tzimmermann@suse.de>
  Thorsten Winkler <twinkler@linux.ibm.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  Wayne Lin <wayne.lin@amd.com>
  Will Deacon <will@kernel.org>
  Yosry Ahmed <yosryahmed@google.com>
  Yu Zhao <yuzhao@google.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              broken  
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                fail    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-amd64-amd64-xl-pvhv2-intel broken
broken-step test-amd64-amd64-xl-pvhv2-intel host-install(5)

Not pushing.

(No revision log; it would be 2494 lines long.)


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 22:39:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 22:39:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685125.1065487 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1re0fS-00063T-9v; Sat, 24 Feb 2024 22:38:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685125.1065487; Sat, 24 Feb 2024 22:38:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1re0fS-00063M-6M; Sat, 24 Feb 2024 22:38:50 +0000
Received: by outflank-mailman (input) for mailman id 685125;
 Sat, 24 Feb 2024 22:38:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NmKg=KB=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1re0fQ-00063G-Tl
 for xen-devel@lists.xenproject.org; Sat, 24 Feb 2024 22:38:48 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 787a80f6-d365-11ee-8a57-1f161083a0e0;
 Sat, 24 Feb 2024 23:38:47 +0100 (CET)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-5643eccad0bso3013244a12.1
 for <xen-devel@lists.xenproject.org>; Sat, 24 Feb 2024 14:38:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 787a80f6-d365-11ee-8a57-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708814325; x=1709419125; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=tEluI7LvDw7LFX5VQEn063ItBW3HwsptymCLrHlIUGI=;
        b=cYX5yIyDwX6eLdJFt7Mmm8klzXu0DNj4gr7aEgbawu+qmiH+vaJm0LS7wE4WKIY/e0
         yXhn7KtTOQGQW9M5Zjo0iryL0ja1Ayy/CF1IiWZVtecj6egStG8Kta1RgOaLyJsJQtny
         d1JUDNgRgRR/ne4LZR0uGwMyaN0IzXKm4YPOUAXSScthS9eLxkh6SZC8K2N7xSjmrmXb
         YQH40+zdswVbThVe9SNaA30t/nHqpfcxDwwaKHKCJt5GkljI0bBefrf3TqtdTInUNiBY
         JN244UgUK2GKJvz5IMxTnzZp8JqhD9LkO40NuPhUAZolceVryXpBT9MSiz3HqhxrYq0C
         TNgQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708814325; x=1709419125;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=tEluI7LvDw7LFX5VQEn063ItBW3HwsptymCLrHlIUGI=;
        b=YOFMm8ontsmeydgkE0GYaVMkJuJgd2P7p+rwHBewzFDgajeH0eeZ1c0DV6jNtCOVq6
         Q/HrzkVh9Sht78FZGQvVl2pPBWf6h/IYSINl6GuCWLEKYkbQgnyBWPkNs09h5qlpUKPx
         /f7d3eWxmhRDwI8j63RaPdO/n8eiEiw0uDhkXV8Ky/U6J+9Up8w3HXSNyg/NtO3XWlp7
         6aCBCQuuqHmVIEk5MjwKgk2btiR0/YKLMivUDPaA9SJEUVgS7078id/kyysrGX3HfgVX
         6YqeCzhJRRxt3TXlcGo1NElYS2HiD/uWTNOZKbIybY7yy5zHqWjGvt6NfJPW9eSldFoi
         IAFg==
X-Forwarded-Encrypted: i=1; AJvYcCU5Y28vA3MZrxfsjhxn3GewFpR07Kray7rmSvqw4kS7kjLRXzqWfbH0qnbbb3AHC5HiRxLP0oYK/UJcvuLvLWXyZz+eyhWBw4ifjESCiSA=
X-Gm-Message-State: AOJu0YxjWPhp2hjTSmqc8fxFw1vvSM6vXEYHHeU8mINKhARgl4/XCCke
	50oCVhPno/cZujU/sigGQ+K1bSuAH8LsJ5uTcz53tlnjLAY77oj0wkKFn7Agqagt+WSAbR9aHxB
	ZILWQzei9tzKjPyLOJKnKD0hMlYU=
X-Google-Smtp-Source: AGHT+IGwfCWYBQmFql5XzZmlAw8qa0O4J9dpcMXcpQCXBSN7Uguk3jfvHidORqTaQL7tfMsnLqfxDxur1pU+ccJMDvc=
X-Received: by 2002:aa7:d385:0:b0:565:9b29:ad3e with SMTP id
 x5-20020aa7d385000000b005659b29ad3emr1736557edq.6.1708814325480; Sat, 24 Feb
 2024 14:38:45 -0800 (PST)
MIME-Version: 1.0
References: <20240219181627.282097-1-marmarek@invisiblethingslab.com> <20240219181627.282097-2-marmarek@invisiblethingslab.com>
In-Reply-To: <20240219181627.282097-2-marmarek@invisiblethingslab.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Sat, 24 Feb 2024 17:38:33 -0500
Message-ID: <CAKf6xpsA5dDgLavGFbT1Jyd1LQS2ZEwUH9G=GdLYnA+1ntohwg@mail.gmail.com>
Subject: Re: [PATCH 2/2] xen: fix stubdom PCI addr
To: =?UTF-8?Q?Marek_Marczykowski=2DG=C3=B3recki?= <marmarek@invisiblethingslab.com>
Cc: qemu-devel@nongnu.org, 
	=?UTF-8?B?RnLDqWTDqXJpYyBQaWVycmV0IChmZXBpdHJlKQ==?= <frederic.pierret@qubes-os.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Anthony Perard <anthony.perard@citrix.com>, 
	Paul Durrant <paul@xen.org>, "open list:X86 Xen CPUs" <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 19, 2024 at 1:49=E2=80=AFPM Marek Marczykowski-G=C3=B3recki
<marmarek@invisiblethingslab.com> wrote:
>
> From: Fr=C3=A9d=C3=A9ric Pierret (fepitre) <frederic.pierret@qubes-os.org=
>
>
> When running in a stubdomain, the config space access via sysfs needs to
> use BDF as seen inside stubdomain (connected via xen-pcifront), which is
> different from the real BDF. For other purposes (hypercall parameters
> etc), the real BDF needs to be used.
> Get the in-stubdomain BDF by looking up relevant PV PCI xenstore
> entries.
>
> Signed-off-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingsl=
ab.com>

Anthony made these comments on a different version of this patch:
https://lore.kernel.org/xen-devel/48c55d33-aa16-4867-a477-f6df45c7d9d9@pera=
rd/

(Sorry I lost track of addressing them at the time.)

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Sat Feb 24 23:43:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sat, 24 Feb 2024 23:43:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685131.1065497 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1re1fI-0005uW-NK; Sat, 24 Feb 2024 23:42:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685131.1065497; Sat, 24 Feb 2024 23:42:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1re1fI-0005uP-Iu; Sat, 24 Feb 2024 23:42:44 +0000
Received: by outflank-mailman (input) for mailman id 685131;
 Sat, 24 Feb 2024 23:42:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1re1fH-0005uF-SI; Sat, 24 Feb 2024 23:42:43 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1re1fH-0000u4-P7; Sat, 24 Feb 2024 23:42:43 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1re1fH-0002ih-AT; Sat, 24 Feb 2024 23:42:43 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1re1fH-00044p-9z; Sat, 24 Feb 2024 23:42:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=mU+VYNKEFtSJYd6EvTp/R9MUgFsMNyLVLKfifPMAR/M=; b=YjtHenk2aAaujq1UjAbiUlJF/K
	cB1M04jAfvPAuQ505NczV+YiICAD7f1jlUfzkaJOiz9mVBPDR0PATFypqQhn1Gi2rooLc0JLPdW1d
	sFgUSy8B0Tra2DxNKIxae7XW7DPdypimgCAHt1NYeBe+f4XQWQd75Bwco2X9UmbYiUrA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184752-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184752: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=92babc88f67ed0ef3dc575a8b9534040274678ee
X-Osstest-Versions-That:
    xen=812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sat, 24 Feb 2024 23:42:43 +0000

flight 184752 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184752/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184732
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184732
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184732
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184732
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184732
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184732
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184732
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184732
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184732
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184732
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184732
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184732
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 xen                  92babc88f67ed0ef3dc575a8b9534040274678ee
baseline version:
 xen                  812bdc99f4c5d05d20b6fac03b90920c0dbf9a2b

Last test of basis   184732  2024-02-23 04:57:51 Z    1 days
Failing since        184746  2024-02-23 23:07:23 Z    1 days    2 attempts
Testing same since   184752  2024-02-24 14:23:33 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Christian Lindig <christian.lindig@cloud.com>
  Edwin Török <edwin.torok@cloud.com>
  George Dunlap <george.dunlap@cloud.com> # Golang bits
  Juergen Gross <jgross@suse.com>
  Julien Grall <jgrall@amazon.com>
  Petr Beneš <w1benny@gmail.com>
  Samuel Thibault <samuel.thibault@ens-lyon.org>
  Shawn Anastasio <sanastasio@raptorengineering.com>
  Tamas K Lengyel <tamas@tklengyel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   812bdc99f4..92babc88f6  92babc88f67ed0ef3dc575a8b9534040274678ee -> master


From xen-devel-bounces@lists.xenproject.org Sun Feb 25 02:50:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 25 Feb 2024 02:50:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685151.1065563 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1re4aO-0001Ur-73; Sun, 25 Feb 2024 02:49:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685151.1065563; Sun, 25 Feb 2024 02:49:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1re4aO-0001Uk-16; Sun, 25 Feb 2024 02:49:52 +0000
Received: by outflank-mailman (input) for mailman id 685151;
 Sun, 25 Feb 2024 02:49:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1re4aN-0001Ua-7D; Sun, 25 Feb 2024 02:49:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1re4aN-0003XP-0h; Sun, 25 Feb 2024 02:49:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1re4aM-0007B1-EM; Sun, 25 Feb 2024 02:49:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1re4aM-0006E1-Db; Sun, 25 Feb 2024 02:49:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=eB9vAUS+2sQ6ZWjCQlNZ+spChbykPNKjoQlMudl84g8=; b=FmSzdrSeq1jwiLMJDQuGHfCer2
	Dwqb/mQyOFQ/cNSYLD/Qp9jd2grd88o7Ag0BRDMm3O7m2KtEHMAgaVIq4BjcbDdxP7v1MIMBpTOMG
	Sk7eS9BYfze27rFofejvr5IWIxCgUkykox7EXYmeNDd5lO2evmRY4ni6h9EJV7lVFxs0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184754-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184754: regressions - FAIL
X-Osstest-Failures:
    linux-linus:build-arm64:xen-build:fail:regression
    linux-linus:test-arm64-arm64-examine:build-check(1):blocked:nonblocking
    linux-linus:build-arm64-libvirt:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:build-check(1):blocked:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:build-check(1):blocked:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=f2e367d6ad3bdc527c2b14e759c2f010d6b2b7a1
X-Osstest-Versions-That:
    linux=b6d69282db550689ab5980e06eedd23b64584a73
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 25 Feb 2024 02:49:50 +0000

flight 184754 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184754/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64                   6 xen-build                fail REGR. vs. 184738

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-examine      1 build-check(1)               blocked  n/a
 build-arm64-libvirt           1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)               blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl           1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               blocked  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               blocked  n/a
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check fail blocked in 184738
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184738
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184738
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184738
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184738
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184738
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184738
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184738
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                f2e367d6ad3bdc527c2b14e759c2f010d6b2b7a1
baseline version:
 linux                b6d69282db550689ab5980e06eedd23b64584a73

Last test of basis   184738  2024-02-23 17:42:09 Z    1 days
Failing since        184749  2024-02-24 10:36:24 Z    0 days    2 attempts
Testing same since   184754  2024-02-24 20:14:27 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexander Sverdlin <alexander.sverdlin@gmail.com>
  Andi Shyti <andi.shyti@kernel.org>
  Andrew Manley <andrew.manley@sealingtech.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrzej Kacprowski <Andrzej.Kacprowski@intel.com>
  Andy Yan <andyshrk@163.com>
  Anshuman Khandual <anshuman.khandual@arm.com>
  Armin Wolf <W_Armin@gmx.de>
  Arnd Bergmann <arnd@arndb.de>
  Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
  Ashutosh Dixit <ashutosh.dixit@intel.com>
  Benjamin Gray <bgray@linux.ibm.com>
  Chanho Min <chanho.min@lge.com>
  Chengming Zhou <zhouchengming@bytedance.com>
  Chris Li <chrisl@kernel.org>
  Chris Morgan <macromorgan@hotmail.com>
  Christian König <christian.koenig@amd.com>
  Colin Ian King <colin.i.king@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Corey Minyard <minyard@acm.org>
  Dan Carpenter <dan.carpenter@linaro.org>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Danilo Krummrich <dakr@redhat.com>
  Dave Airlie <airlied@redhat.com>
  David Hildenbrand <david@redhat.com>
  Don Brace <don.brace@microchip.com>
  Erik Kurzinger <ekurzinger@nvidia.com>
  Fabio Estevam <festevam@denx.de>
  Florian Fainelli <florian.fainelli@broadcom.com> #Broadcom
  Geert Uytterhoeven <geert+renesas@glider.be>
  Guenter Roeck <linux@roeck-us.net>
  Heiko Carstens <hca@linux.ibm.com>
  Heiko Stuebner <heiko.stuebner@cherry.de>
  Heiko Stuebner <heiko@sntech.de>
  Helge Deller <deller@gmx.de>
  Hojin Nam <hj96.nam@samsung.com>
  Huacai Chen <chenhuacai@loongson.cn>
  Jacek Lawrynowicz <jacek.lawrynowicz@linux.intel.com>
  Jason Gunthorpe <jgg@nvidia.com>
  Javier Martinez Canillas <javierm@redhat.com>
  Johan Jonker <jbx6244@gmail.com>
  Johannes Weiner <hannes@cmpxchg.org>
  Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
  José Roberto de Souza <jose.souza@intel.com>
  Julian Sikorski <belegdol@gmail.com>
  Kairui Song <kasong@tencent.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lewis Huang <lewis.huang@amd.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lucas Stach <l.stach@pengutronix.de>
  Ma Jun <Jun.Ma2@amd.com>
  Marek Vasut <marex@denx.de>
  Mark Brown <broonie@kernel.org>
  Martin Blumenstingl <martin.blumenstingl@googlemail.com>
  Martin K. Petersen <martin.petersen@oracle.com>
  Matthew Auld <matthew.auld@intel.com>
  Matthew Brost <matthew.brost@intel.com>
  Max Kellermann <max.kellermann@ionos.com>
  Maxime Ripard <mripard@kernel.org>
  Maxime Ripard <mripard@redhat.com>
  Melissa Wen <mwen@igalia.com>
  Michal Hocko <mhocko@suse.com>
  Mike Snitzer <snitzer@kernel.org>
  Mikulas Patocka <mpatocka@redhat.com>
  Naohiro Aota <naohiro.aota@wdc.com>
  Neil Armstrong <neil.armstrong@linaro.org>
  Nhat Pham <nphamcs@gmail.com>
  Nikita Shubin <nikita.shubin@maquefel.me>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Oreoluwa Babatunde <quic_obabatun@quicinc.com>
  Peter Oberparleiter <oberpar@linux.ibm.com>
  Randy Dunlap <rdunlap@infradead.org>
  Rob Herring <robh@kernel.org>
  Rodrigo Siqueira <rodrigo.siqueira@amd.com>
  Rodrigo Vivi <rodrigo.vivi@intel.com>
  Samasth Norway Ananda <samasth.norway.ananda@oracle.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  SeongJae Park <sj@kernel.org>
  Shakeel Butt <shakeel.butt@linux.dev>
  Shakeel Butt <shakeelb@google.com>
  Shawn Guo <shawnguo@kernel.org>
  Simon Ser <contact@emersion.fr>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Steve Morvai <stevemorvai@hotmail.com>
  Swapnil Patel <swapnil.patel@amd.com>
  Tasos Sahanidis <tasos@tasossah.com>
  Terry Tritton <terry.tritton@linaro.org>
  Thierry Reding <treding@nvidia.com>
  Thomas Hellstrom <thomas.hellstrom@linux.intel.com>
  Thomas Hellström <thomas.hellstrom@linux.intel.com>
  Thomas Zimmermann <tzimmermann@suse.de>
  Thorsten Winkler <twinkler@linux.ibm.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  WANG Xuerui <git@xen0n.name>
  Wayne Lin <wayne.lin@amd.com>
  Will Deacon <will@kernel.org>
  Wolfram Sang <wsa+renesas@sang-engineering.com>
  Yogesh Chandra Pandey <YogeshChandra.Pandey@microchip.com>
  Yosry Ahmed <yosryahmed@google.com>
  Yu Zhao <yuzhao@google.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  fail    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          blocked 
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          blocked 
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 blocked 
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  blocked 
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  blocked 
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     blocked 
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 blocked 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 blocked 
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      blocked 
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3048 lines long.)


From xen-devel-bounces@lists.xenproject.org Sun Feb 25 08:47:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 25 Feb 2024 08:47:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685167.1065572 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reAAP-0006y1-2I; Sun, 25 Feb 2024 08:47:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685167.1065572; Sun, 25 Feb 2024 08:47:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reAAO-0006xu-Vr; Sun, 25 Feb 2024 08:47:24 +0000
Received: by outflank-mailman (input) for mailman id 685167;
 Sun, 25 Feb 2024 08:47:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reAAN-0006xk-Gu; Sun, 25 Feb 2024 08:47:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reAAN-0003SF-Ca; Sun, 25 Feb 2024 08:47:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reAAN-0001Of-0s; Sun, 25 Feb 2024 08:47:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1reAAN-0002ta-06; Sun, 25 Feb 2024 08:47:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=CN8eJXRbZadyD753eB2DK2IMKc/g6ZBL/dSLPpN2cBo=; b=WNs70xPoPDJNl5QsRcDxp89Wl1
	/dWpU1VFfA78/GB5OnaFEEWzZj3e6fGG0jr87gNT2ZMeMAu9HY872PHNqYmKIVpYilrxLuW3TdEEx
	47Zay2NurvAwC2VhClRlUCWBDkBx1lf7yYVfSOGiUtEUR4vNiNZZnpW7fuGNlVON8PQI=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184756-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184756: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=92babc88f67ed0ef3dc575a8b9534040274678ee
X-Osstest-Versions-That:
    xen=92babc88f67ed0ef3dc575a8b9534040274678ee
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 25 Feb 2024 08:47:23 +0000

flight 184756 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184756/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 12 debian-hvm-install fail pass in 184752

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184752
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184752
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184752
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184752
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184752
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184752
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184752
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184752
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184752
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184752
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184752
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184752
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 xen                  92babc88f67ed0ef3dc575a8b9534040274678ee
baseline version:
 xen                  92babc88f67ed0ef3dc575a8b9534040274678ee

Last test of basis   184756  2024-02-25 01:53:48 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             fail    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Sun Feb 25 12:20:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 25 Feb 2024 12:20:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685194.1065581 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reDU2-0003RR-PY; Sun, 25 Feb 2024 12:19:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685194.1065581; Sun, 25 Feb 2024 12:19:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reDU2-0003RK-Mc; Sun, 25 Feb 2024 12:19:54 +0000
Received: by outflank-mailman (input) for mailman id 685194;
 Sun, 25 Feb 2024 12:19:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=J/ns=KC=gmail.com=rob.townley@srs-se1.protection.inumbo.net>)
 id 1reDU1-0003RE-BF
 for xen-devel@lists.xenproject.org; Sun, 25 Feb 2024 12:19:53 +0000
Received: from mail-yb1-xb2e.google.com (mail-yb1-xb2e.google.com
 [2607:f8b0:4864:20::b2e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2b212630-d3d8-11ee-98f5-efadbce2ee36;
 Sun, 25 Feb 2024 13:19:49 +0100 (CET)
Received: by mail-yb1-xb2e.google.com with SMTP id
 3f1490d57ef6-dcbcea9c261so2294958276.3
 for <xen-devel@lists.xenproject.org>; Sun, 25 Feb 2024 04:19:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2b212630-d3d8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708863588; x=1709468388; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:reply-to:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=W2E0Tttmj/DoPM7LRU51gbJvIjSSBzxFpbiR1OJL1cw=;
        b=aUdruPgnUd1JQuTeD5R3yKKoao6k4dMN/ke8+SvUgsl/cjYjq6gl9jkqBImSd31o4t
         uRE8MHIwpLYbNuwa4dN251BjzkOo6OoxLzDahEugDbGhdSKjGA+u0mzilgqnb0em5mUZ
         axya7Pm2s2Lig0cKnV7XQnklMDQlwLdFHfuN1mY9exGPNvIlCABCesA1nxfaJYlWjbAV
         sMzkTRBOMDybm3VBrhw6HqDY86f0zJ8gJ+tbqChMAGyANwpD9smrE2xCtqmI/kquWJva
         9vNGyq8dp/Z3/wrNIPam9DTsYIYkkuCqmuWVJOW4dSKVT9ZrVPHN5f/PqEZCzfECef4h
         I8SQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708863588; x=1709468388;
        h=cc:to:subject:message-id:date:from:reply-to:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=W2E0Tttmj/DoPM7LRU51gbJvIjSSBzxFpbiR1OJL1cw=;
        b=O1hz3rxbC7IKC25GM63f4+YFfDufoFljcPVlQUy7MpuluN/4vQyL6VkysqkyIqJKTR
         SagJ1XKZZwNpBy3EQZxHcIdbnRRpnEev84gRuxPMvnopwndUvKLSh2EzkV3IUw7RZvFP
         eKp6ir5WLkylA/GYXa9/4D73kT9rYwIGv5N0qB2/Yg7KrUD4TWwNiPZxxAmSapp8l2HR
         dd7GQInIRICp43ztMaljAxEdp/SjAchQIJ9WMi25Tnqvb4uQe1Xatgo6vQPRpQF7dp3X
         umvYJw5KiN7PlHx2mrUdCRNBncG6ppAyk6LndVO1xrSDCYNxm5NK2Jjtjm9qd/vRHOKS
         CV0g==
X-Forwarded-Encrypted: i=1; AJvYcCUHM1vyZtlxohCN/am2GjtPiX2VnkPhhNN9szNXJNQNrfqiQP7K8GF/QMUxUd6anchkiBXIb5RBR0mUF2J2k2VZkVKAOJ6H52nd8dvY1yw=
X-Gm-Message-State: AOJu0YxrWCKpgaUv/ucSsF3akEWrpx1ytQo3TRPoXfmvVCl2F5uUK1A3
	HqW6hyiJErbKDLspwIBSsgSFlQhucDKLRyk+nIGYHILNcX4Nzw3YJDg0QBbRpjuS8hfNO0U8sZd
	Aa3ob9RY8xdPu24RRa5jlKL4Wi+8=
X-Google-Smtp-Source: AGHT+IHBpdMueIx32lg1q5sCYr6wtPcOb4aIxsc4Qzt1tdi1kX6fWxBdf8TuGsBcYwQFcJP8bJtT0Nhj9RZhFAmQoVI=
X-Received: by 2002:a25:26c5:0:b0:dc6:18d0:95b0 with SMTP id
 m188-20020a2526c5000000b00dc618d095b0mr2306857ybm.8.1708863587972; Sun, 25
 Feb 2024 04:19:47 -0800 (PST)
MIME-Version: 1.0
References: <d2de4ae9ecb34efc962dea7f8b4e7cbd@thalesgroup.com> <bb2485e5-7818-435b-8d9c-dda88100979f@xen.org>
In-Reply-To: <bb2485e5-7818-435b-8d9c-dda88100979f@xen.org>
Reply-To: Rob.Townley@gmail.com
From: Rob Townley <rob.townley@gmail.com>
Date: Sun, 25 Feb 2024 06:19:37 -0600
Message-ID: <CA+VdTb8razTDCZdAssAf=HHbDhV+w8+k-XO+n6msv31izUfQEw@mail.gmail.com>
Subject: Re: IMPORTANT - : Need help on USB port virtualization with Xen hypervisor
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, 
	GOURLOT Francois <francois.gourlot@thalesgroup.com>, George Dunlap <george.dunlap@citrix.com>, 
	Jan Beulich <jbeulich@suse.com>, Juergen Gross <jgross@suse.com>, Kelly Choi <kelly.choi@cloud.com>, 
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Content-Type: multipart/alternative; boundary="000000000000217d27061233cfe1"

--000000000000217d27061233cfe1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

In short, purchase an addon PCIe USB card to isolate traffic.  USB is a
shared medium and is rooted in the cheapest electronics.   It was never
considered enterprise grade hardware for most all of history.    The letter
B in  USB  stands for BUS which actually means a shared roadway that other
USB devices can and do impede traffic.   We experienced interfering traffic
on inexpensive laptops running  XEN based Qubes-OS.  Turned out the WiFi
card was actually wired onto the USB bus.   Unplugging all external USB
electronics reduced inexplicable WiFi disconnects and reliably improved
throughput.    However, there was no way to remove the keyboard, touchpad,
and touchscreen which all used the USB roadway.

Those were cheap laptops, but even enterprise class hardware such as HPE
 and Dell Servers running XCP-NG did not have great speeds.   But in this
scenario, we could simplify the problem and  isolate the issue by purchasin=
g
a  new add-on PCIe card with USB ports and then assign the entire PCIe card
to the VM.   The VM would then have entire control of the PCIe card and
isolate traffic to devices plugged into that card.

I did not know until later that Wireshark has plugins for sniffing USB
traffic, but my name is  not Heisenberg and not sure if that traffic
monitoring does not also slow down other devices.

On Sat, Feb 24, 2024 at 8:39 AM Julien Grall <julien@xen.org> wrote:

> Hi Dominique and Francois,
>
> Thank you for the report. I am CCing some person who I think could help.
>
> I also have some questions (see below).
>
> On 23/02/2024 16:00, GOURLOT Francois wrote:
> > Dear All,
> >
> > We send you a message few days ago.
> >
> > We have major performance with XEN USB Drivers. We use your driver to
> > load data in a device and we need USB3 High Speed USB protocole
> >
> > Do you have a new issue of the HCI and NEC USB Xen Driver ?
> p
> [...]
>
> > *De :*LARRIEU Dominique <dominique.larrieu@thalesgroup.com>
> > *Envoy=C3=A9 :* jeudi 22 f=C3=A9vrier 2024 13:58
> > *=C3=80 :* xen-devel@lists.xenproject.org
> > *Cc :* WILLEMS Louis <louis.willems@thalesgroup.com>; GRUO Nicolas
> > <nicolas-n.gruo@thalesgroup.com>; GOURLOT Francois
> > <francois.gourlot@thalesgroup.com>; Kelly Choi <kelly.choi@cloud.com>
> > *Objet :* Need help on USB port virtualization with Xen hypervisor
> > *Crit=C3=A8re de diffusion :* Confidentiel
> >
> >     Dear all,
> >
> >     We are detectingseveral issues with USB port virtualization with th=
e
> >     Xen hypervisor.
>
> Just to clarify, you are assigning the PCI USB bus (rather than just the
> USB device) to the guest. Is that correct?
>
> If so...
>
>  >
>  > Do you have a new issue of the HCI and NEC USB Xen Driver ?
>
> ... I am not sure what Xen driver you are talking about. If the USB bus
> is assigned to the guest, then I would expect no specific Xen driver are
> required. Do you have any pointer to the existing one?
>
> >     - We cannot do PCI passthrough of the PCI usb bus on a Windows 10
> >     1607 64-bit virtual machine. The bad result is a Windows blue scree=
n.
> >
> >     - When we use the passthrough functionality on a Windows 21H2
> >     virtual machine, we notice thatthe speed of the USB port is not hig=
h
> >     speed but full speed on a USB 3.0 port
> >
> >     - We notice instabilities when using the nec-usb-xhci driver,  USB
> >     2.0 keys are not recognized by the Windows virtual machine
> >     (incorrect descriptor)
>
> Do you have any logs you could share from the guest OS? Also, would you
> be able to share more details on how you configure the guest? (e.g. xl
> configuration, libvirt...)?
>
> >
> >     We needyour helpto find a solution for these problems.
> >
> >     The Software used are :
>
> Can you share some details of the HW you are using?
>
> >
> >     -Debian 11 version5.10.0-20
> >
> >     -Xen version4.14
>
> Are minor version of Xen 4.14 are you using? Any patches on top?
>
> Xen 4.14 has also been out of support for a few months now. If that's
> possible, I would recommend to try a new Xen version (the last stable is
> 4.18) just in case your issue as been fixed in newer release.
>
> >
> >     -Windows 10 1607 and 21H2 for virtual machines. Virtual Machine HVM
>
> Cheers,
>
> --
> Julien Grall
>
>

--000000000000217d27061233cfe1
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div><div dir=3D"auto" style=3D"font-family:-apple-system=
,helveticaneue;font-size:18px;font-style:normal;font-weight:400;letter-spac=
ing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spac=
ing:0px;text-decoration:none;background-color:rgba(0,0,0,0);border-color:rg=
b(0,0,0);color:rgb(0,0,0)"><span style=3D"font-family:-apple-system,helveti=
caneue;background-color:rgba(0,0,0,0);border-color:rgb(0,0,0);color:rgb(0,0=
,0)"><div style=3D"font-family:-apple-system,helveticaneue" dir=3D"auto"><s=
pan style=3D"font-family:-apple-system,helveticaneue;font-size:18px;font-st=
yle:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-trans=
form:none;white-space:normal;word-spacing:0px;text-decoration:none;float:no=
ne;display:inline!important;background-color:rgba(0,0,0,0);border-color:rgb=
(0,0,0);color:rgb(0,0,0)">In short, purchase an addon PCIe USB card to isol=
ate traffic.=C2=A0 USB is a shared medium and is rooted in the cheapest ele=
ctronics. =C2=A0 It was never considered enterprise grade hardware for most=
 all of history. =C2=A0 =C2=A0</span><span style=3D"font-family:-apple-syst=
em,helveticaneue;background-color:rgba(0,0,0,0);border-color:rgb(0,0,0);col=
or:rgb(0,0,0)">The letter B in =C2=A0USB =C2=A0stands for BUS which actuall=
y means a shared roadway that other USB devices can and do impede traffic. =
=C2=A0 We experienced interfering traffic on inexpensive laptops running =
=C2=A0XEN based Qubes-OS.=C2=A0 Turned out the WiFi card was actually wired=
 onto the USB bus. =C2=A0 Unplugging all external USB electronics reduced i=
nexplicable WiFi disconnects and reliably improved throughput. =C2=A0 =C2=
=A0However, there was no way to remove the keyboard, touchpad, and touchscr=
een which all used the USB roadway. =C2=A0=C2=A0</span></div><div style=3D"=
font-family:-apple-system,helveticaneue" dir=3D"auto"><span style=3D"font-f=
amily:-apple-system,helveticaneue;background-color:rgba(0,0,0,0);border-col=
or:rgb(0,0,0);color:rgb(0,0,0)"><br></span></div><div style=3D"font-family:=
-apple-system,helveticaneue" dir=3D"auto"><span style=3D"font-family:-apple=
-system,helveticaneue;background-color:rgba(0,0,0,0);border-color:rgb(0,0,0=
);color:rgb(0,0,0)">Those were cheap laptops, but even=C2=A0</span><span st=
yle=3D"font-family:-apple-system,helveticaneue;background-color:rgba(0,0,0,=
0);border-color:rgb(0,0,0);color:rgb(0,0,0)">enterprise class hardware such=
 as HPE =C2=A0and Dell Servers running XCP-NG did not have great speeds. =
=C2=A0 But in this scenario, we could simplify the problem and =C2=A0isolat=
e the issue by=C2=A0</span><span>purchasing a =C2=A0new add-on=C2=A0</span>=
<span style=3D"font-family:-apple-system,helveticaneue;background-color:rgb=
a(0,0,0,0);border-color:rgb(0,0,0);color:rgb(0,0,0)">PCIe card with USB por=
ts and then assign the entire PCIe card to the VM. =C2=A0 The VM would then=
 have entire control of the PCIe card and isolate traffic to devices plugge=
d into that card. =C2=A0</span></div><div style=3D"font-family:-apple-syste=
m,helveticaneue" dir=3D"auto"><span style=3D"font-family:-apple-system,helv=
eticaneue;background-color:rgba(0,0,0,0);border-color:rgb(0,0,0);color:rgb(=
0,0,0)"><br></span></div><div style=3D"font-family:-apple-system,helvetican=
eue" dir=3D"auto"><span style=3D"font-family:-apple-system,helveticaneue;ba=
ckground-color:rgba(0,0,0,0);border-color:rgb(0,0,0);color:rgb(0,0,0)">I di=
d not know until later that Wireshark has plugins for sniffing USB traffic,=
 but my name is =C2=A0not Heisenberg and not sure if that traffic monitorin=
g does not also slow down other devices. =C2=A0</span><br></div></span></di=
v></div></div><div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D=
"gmail_attr">On Sat, Feb 24, 2024 at 8:39 AM Julien Grall &lt;<a href=3D"ma=
ilto:julien@xen.org">julien@xen.org</a>&gt; wrote:<br></div><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;=
border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)=
" dir=3D"auto">Hi Dominique and Francois,<br>
<br>
Thank you for the report. I am CCing some person who I think could help.<br=
>
<br>
I also have some questions (see below).<br>
<br>
On 23/02/2024 16:00, GOURLOT Francois wrote:<br>
&gt; Dear All,<br>
&gt; <br>
&gt; We send you a message few days ago.<br>
&gt; <br>
&gt; We have major performance with XEN USB Drivers. We use your driver to =
<br>
&gt; load data in a device and we need USB3 High Speed USB protocole<br>
&gt; <br>
&gt; Do you have a new issue of the HCI and NEC USB Xen Driver ?<br>
p<br>
[...]<br>
<br>
&gt; *De=C2=A0:*LARRIEU Dominique &lt;<a href=3D"mailto:dominique.larrieu@t=
halesgroup.com" target=3D"_blank">dominique.larrieu@thalesgroup.com</a>&gt;=
<br>
&gt; *Envoy=C3=A9=C2=A0:* jeudi 22 f=C3=A9vrier 2024 13:58<br>
&gt; *=C3=80=C2=A0:* <a href=3D"mailto:xen-devel@lists.xenproject.org" targ=
et=3D"_blank">xen-devel@lists.xenproject.org</a><br>
&gt; *Cc=C2=A0:* WILLEMS Louis &lt;<a href=3D"mailto:louis.willems@thalesgr=
oup.com" target=3D"_blank">louis.willems@thalesgroup.com</a>&gt;; GRUO Nico=
las <br>
&gt; &lt;<a href=3D"mailto:nicolas-n.gruo@thalesgroup.com" target=3D"_blank=
">nicolas-n.gruo@thalesgroup.com</a>&gt;; GOURLOT Francois <br>
&gt; &lt;<a href=3D"mailto:francois.gourlot@thalesgroup.com" target=3D"_bla=
nk">francois.gourlot@thalesgroup.com</a>&gt;; Kelly Choi &lt;<a href=3D"mai=
lto:kelly.choi@cloud.com" target=3D"_blank">kelly.choi@cloud.com</a>&gt;<br=
>
&gt; *Objet=C2=A0:* Need help on USB port virtualization with Xen hyperviso=
r<br>
&gt; *Crit=C3=A8re de diffusion=C2=A0:* Confidentiel<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0Dear all,<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0We are detectingseveral issues with USB port virtua=
lization with the<br>
&gt;=C2=A0 =C2=A0 =C2=A0Xen hypervisor.<br>
<br>
Just to clarify, you are assigning the PCI USB bus (rather than just the <b=
r>
USB device) to the guest. Is that correct?<br>
<br>
If so...<br>
<br>
=C2=A0&gt;<br>
=C2=A0&gt; Do you have a new issue of the HCI and NEC USB Xen Driver ?<br>
<br>
... I am not sure what Xen driver you are talking about. If the USB bus <br=
>
is assigned to the guest, then I would expect no specific Xen driver are <b=
r>
required. Do you have any pointer to the existing one?<br>
<br>
&gt;=C2=A0 =C2=A0 =C2=A0- We cannot do PCI passthrough of the PCI usb bus o=
n a Windows 10<br>
&gt;=C2=A0 =C2=A0 =C2=A01607 64-bit virtual machine. The bad result is a Wi=
ndows blue screen.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0- When we use the passthrough functionality on a Wi=
ndows 21H2<br>
&gt;=C2=A0 =C2=A0 =C2=A0virtual machine, we notice thatthe speed of the USB=
 port is not high<br>
&gt;=C2=A0 =C2=A0 =C2=A0speed but full speed on a USB 3.0 port<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0- We notice instabilities when using the nec-usb-xh=
ci driver, =C2=A0USB<br>
&gt;=C2=A0 =C2=A0 =C2=A02.0 keys are not recognized by the Windows virtual =
machine<br>
&gt;=C2=A0 =C2=A0 =C2=A0(incorrect descriptor)<br>
<br>
Do you have any logs you could share from the guest OS? Also, would you <br=
>
be able to share more details on how you configure the guest? (e.g. xl <br>
configuration, libvirt...)?<br>
<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0We needyour helpto find a solution for these proble=
ms.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0The Software used are :<br>
<br>
Can you share some details of the HW you are using?<br>
<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0-Debian 11 version5.10.0-20<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0-Xen version4.14<br>
<br>
Are minor version of Xen 4.14 are you using? Any patches on top?<br>
<br>
Xen 4.14 has also been out of support for a few months now. If that&#39;s <=
br>
possible, I would recommend to try a new Xen version (the last stable is <b=
r>
4.18) just in case your issue as been fixed in newer release.<br>
<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0-Windows 10 1607 and 21H2 for virtual machines. Vir=
tual Machine HVM<br>
<br>
Cheers,<br>
<br>
-- <br>
Julien Grall<br>
<br>
</blockquote></div></div>

--000000000000217d27061233cfe1--


From xen-devel-bounces@lists.xenproject.org Sun Feb 25 16:00:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 25 Feb 2024 16:00:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685201.1065592 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reGvF-00039Q-6K; Sun, 25 Feb 2024 16:00:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685201.1065592; Sun, 25 Feb 2024 16:00:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reGvF-00039J-1O; Sun, 25 Feb 2024 16:00:13 +0000
Received: by outflank-mailman (input) for mailman id 685201;
 Sun, 25 Feb 2024 16:00:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reGvE-000399-AD; Sun, 25 Feb 2024 16:00:12 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reGvE-0003cH-5D; Sun, 25 Feb 2024 16:00:12 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reGvD-0006zR-O7; Sun, 25 Feb 2024 16:00:11 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1reGvD-0006UK-NT; Sun, 25 Feb 2024 16:00:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=nn0W7/PSYloL3uEPVru4FdVxIXsYUniFCdxalDqR3wk=; b=B2Rk/+qfA4+qv4AK8EZY95axP6
	qedejNB0hB+H9K8XHDesGZMLyFtJ+hbPWcv0QhxgRFNDP5AgaCRagSxFPwQeec03kh3kREsIeXqFp
	LDKeNAINNEDF8qExhL/o50Cd629uY7hN2ZkcT0HMTWfLxQ1uRYN8YxmmGrQD+pfNvMF0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184757-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184757: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt-raw:guest-start:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=ab0a97cffa0bb3b529ca08b0caea772ddb3e0b5c
X-Osstest-Versions-That:
    linux=b6d69282db550689ab5980e06eedd23b64584a73
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 25 Feb 2024 16:00:11 +0000

flight 184757 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184757/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 13 guest-start             fail blocked in 184738
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184738
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184738
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184738
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184738
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184738
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184738
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184738
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                ab0a97cffa0bb3b529ca08b0caea772ddb3e0b5c
baseline version:
 linux                b6d69282db550689ab5980e06eedd23b64584a73

Last test of basis   184738  2024-02-23 17:42:09 Z    1 days
Failing since        184749  2024-02-24 10:36:24 Z    1 days    3 attempts
Testing same since   184757  2024-02-25 02:59:57 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Alex Deucher <alexander.deucher@amd.com>
  Alexander Gordeev <agordeev@linux.ibm.com>
  Alexander Stein <alexander.stein@ew.tq-group.com>
  Alexander Sverdlin <alexander.sverdlin@gmail.com>
  Alison Schofield <alison.schofield@intel.com>
  Amit Machhiwal <amachhiw@linux.ibm.com>
  Andi Shyti <andi.shyti@kernel.org>
  Andrew Manley <andrew.manley@sealingtech.com>
  Andrew Morton <akpm@linux-foundation.org>
  Andrzej Kacprowski <Andrzej.Kacprowski@intel.com>
  Andy Yan <andyshrk@163.com>
  Anshuman Khandual <anshuman.khandual@arm.com>
  Armin Wolf <W_Armin@gmx.de>
  Arnd Bergmann <arnd@arndb.de>
  Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
  Ashutosh Dixit <ashutosh.dixit@intel.com>
  Benjamin Gray <bgray@linux.ibm.com>
  Chanho Min <chanho.min@lge.com>
  Chengming Zhou <zhouchengming@bytedance.com>
  Chris Li <chrisl@kernel.org>
  Chris Morgan <macromorgan@hotmail.com>
  Christian König <christian.koenig@amd.com>
  Colin Ian King <colin.i.king@gmail.com>
  Conor Dooley <conor.dooley@microchip.com>
  Corey Minyard <minyard@acm.org>
  Dan Carpenter <dan.carpenter@linaro.org>
  Dan Williams <dan.j.williams@intel.com>
  Daniel Wheeler <daniel.wheeler@amd.com>
  Danilo Krummrich <dakr@redhat.com>
  Dave Airlie <airlied@redhat.com>
  Dave Hansen <dave.hansen@linux.intel.com>
  Dave Jiang <dave.jiang@intel.com>
  David Hildenbrand <david@redhat.com>
  Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
  Don Brace <don.brace@microchip.com>
  Erik Kurzinger <ekurzinger@nvidia.com>
  Fabio Estevam <festevam@denx.de>
  Florian Fainelli <florian.fainelli@broadcom.com> #Broadcom
  Gaurav Batra <gbatra@linux.ibm.com>
  Geert Uytterhoeven <geert+renesas@glider.be>
  Guenter Roeck <linux@roeck-us.net>
  Heiko Carstens <hca@linux.ibm.com>
  Heiko Stuebner <heiko.stuebner@cherry.de>
  Heiko Stuebner <heiko@sntech.de>
  Helge Deller <deller@gmx.de>
  Hojin Nam <hj96.nam@samsung.com>
  Huacai Chen <chenhuacai@loongson.cn>
  Jacek Lawrynowicz <jacek.lawrynowicz@linux.intel.com>
  Jason Gunthorpe <jgg@nvidia.com>
  Javier Martinez Canillas <javierm@redhat.com>
  Joerg Roedel <jroedel@suse.de>
  Johan Jonker <jbx6244@gmail.com>
  Johannes Weiner <hannes@cmpxchg.org>
  Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
  José Roberto de Souza <jose.souza@intel.com>
  Julian Sikorski <belegdol@gmail.com>
  Kairui Song <kasong@tencent.com>
  Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
  Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
  Lewis Huang <lewis.huang@amd.com>
  Li Ming <ming4.li@intel.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lu Baolu <baolu.lu@linux.intel.com>
  Lucas Stach <l.stach@pengutronix.de>
  Ma Jun <Jun.Ma2@amd.com>
  Marek Vasut <marex@denx.de>
  Mark Brown <broonie@kernel.org>
  Martin Blumenstingl <martin.blumenstingl@googlemail.com>
  Martin K. Petersen <martin.petersen@oracle.com>
  Matthew Auld <matthew.auld@intel.com>
  Matthew Brost <matthew.brost@intel.com>
  Max Kellermann <max.kellermann@ionos.com>
  Maxime Ripard <mripard@kernel.org>
  Maxime Ripard <mripard@redhat.com>
  Melissa Wen <mwen@igalia.com>
  Michael Ellerman <mpe@ellerman.id.au>
  Michal Hocko <mhocko@suse.com>
  Mike Rapoport (IBM) <rppt@kernel.org>
  Mike Snitzer <snitzer@kernel.org>
  Mikulas Patocka <mpatocka@redhat.com>
  Naohiro Aota <naohiro.aota@wdc.com>
  Neil Armstrong <neil.armstrong@linaro.org>
  Nhat Pham <nphamcs@gmail.com>
  Nikita Shubin <nikita.shubin@maquefel.me>
  Niklas Schnelle <schnelle@linux.ibm.com>
  Oreoluwa Babatunde <quic_obabatun@quicinc.com>
  Peter Oberparleiter <oberpar@linux.ibm.com>
  Randy Dunlap <rdunlap@infradead.org>
  Rob Herring <robh@kernel.org>
  Robert Richter <rrichter@amd.com>
  Robin Murphy <robin.murphy@arm.com>
  Rodrigo Siqueira <rodrigo.siqueira@amd.com>
  Rodrigo Vivi <rodrigo.vivi@intel.com>
  Samasth Norway Ananda <samasth.norway.ananda@oracle.com>
  Sebastian Reichel <sebastian.reichel@collabora.com>
  SeongJae Park <sj@kernel.org>
  Shakeel Butt <shakeel.butt@linux.dev>
  Shakeel Butt <shakeelb@google.com>
  Shawn Guo <shawnguo@kernel.org>
  Simon Ser <contact@emersion.fr>
  Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
  Steve Morvai <stevemorvai@hotmail.com>
  Swapnil Patel <swapnil.patel@amd.com>
  Tasos Sahanidis <tasos@tasossah.com>
  Terry Tritton <terry.tritton@linaro.org>
  Thierry Reding <treding@nvidia.com>
  Thomas Hellstrom <thomas.hellstrom@linux.intel.com>
  Thomas Hellström <thomas.hellstrom@linux.intel.com>
  Thomas Zimmermann <tzimmermann@suse.de>
  Thorsten Winkler <twinkler@linux.ibm.com>
  Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
  WANG Xuerui <git@xen0n.name>
  Wayne Lin <wayne.lin@amd.com>
  Will Deacon <will@kernel.org>
  Wolfram Sang <wsa+renesas@sang-engineering.com>
  Wonjae Lee <wj28.lee@samsung.com>
  Yi Liu <yi.l.liu@intel.com>
  Yi Sun <yi.y.sun@linux.intel.com>
  Yogesh Chandra Pandey <YogeshChandra.Pandey@microchip.com>
  Yosry Ahmed <yosryahmed@google.com>
  Yu Zhao <yuzhao@google.com>
  Zhangfei Gao <zhangfei.gao@linaro.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 fail    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   b6d69282db550..ab0a97cffa0bb  ab0a97cffa0bb3b529ca08b0caea772ddb3e0b5c -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Sun Feb 25 19:49:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 25 Feb 2024 19:49:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685210.1065601 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reKVK-0000qR-LP; Sun, 25 Feb 2024 19:49:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685210.1065601; Sun, 25 Feb 2024 19:49:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reKVK-0000qK-If; Sun, 25 Feb 2024 19:49:42 +0000
Received: by outflank-mailman (input) for mailman id 685210;
 Sun, 25 Feb 2024 19:49:40 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reKVI-0000qA-J8; Sun, 25 Feb 2024 19:49:40 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reKVI-0008If-HL; Sun, 25 Feb 2024 19:49:40 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reKVI-0005Wj-9C; Sun, 25 Feb 2024 19:49:40 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1reKVI-0004U2-8h; Sun, 25 Feb 2024 19:49:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=fotXBprdfnK+Td3/V29nr+eRDldi6Ddwn1FxNR6ws0E=; b=AuUAwoqCNLrWfHWkcu9LK92cg1
	ygU6yJ1PbBLvfDysBMvkmeVSJ9tEyVKKBkPVkZycTk2RpHfJRj/WO8BCIfBgeDsJoNtatiaLZei3V
	EE7p2zMHSv4FP3G+ekFQIl6VpLcTIS0wSM/GaO6GiCYD+HWI//RByFr4RQ7QOu3DnA44=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184759-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184759: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=f881b4d129602a49e3403043fc27550a74453234
X-Osstest-Versions-That:
    ovmf=7fa4a984c438b897086f5e2fea6b9e9ad17867c4
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 25 Feb 2024 19:49:40 +0000

flight 184759 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184759/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 f881b4d129602a49e3403043fc27550a74453234
baseline version:
 ovmf                 7fa4a984c438b897086f5e2fea6b9e9ad17867c4

Last test of basis   184751  2024-02-24 13:41:24 Z    1 days
Testing same since   184759  2024-02-25 17:44:36 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Gerd Hoffmann <kraxel@redhat.com>
  Jiewen Yao <Jiewen.yao@intel.com>
  Laszlo Ersek <lersek@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   7fa4a984c4..f881b4d129  f881b4d129602a49e3403043fc27550a74453234 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Sun Feb 25 22:05:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 25 Feb 2024 22:05:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685217.1065612 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reMcm-00077t-8N; Sun, 25 Feb 2024 22:05:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685217.1065612; Sun, 25 Feb 2024 22:05:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reMcm-00077m-5H; Sun, 25 Feb 2024 22:05:32 +0000
Received: by outflank-mailman (input) for mailman id 685217;
 Sun, 25 Feb 2024 22:05:30 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FNuJ=KC=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1reMck-00077g-4o
 for xen-devel@lists.xenproject.org; Sun, 25 Feb 2024 22:05:30 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fb8aa509-d429-11ee-98f5-efadbce2ee36;
 Sun, 25 Feb 2024 23:05:28 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-5640fef9fa6so3124997a12.0
 for <xen-devel@lists.xenproject.org>; Sun, 25 Feb 2024 14:05:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb8aa509-d429-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708898727; x=1709503527; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=DkKvfyrMBWj0+DlETuXifPbtp+G+CbFdvNnXMM5MeUs=;
        b=cWXYEHM4vV0ILOgSo0IjiUGm22SkLcOAiC3cB7sFNCRmMcDv4Fd8OFJgWQyQK6V/hH
         zy+kDx7BpRWBM8nfxrEBAzz40JdCROcq22/h3L6UF3o/Dt+zrXs6X0kHZqL4IrJ8dcYp
         gbh1FHPZYmM1if/1wulAzE8GW8dEf66M2xnsmpE+RIRfK5BWNU+hnh2a7wgZG7gJwBnu
         uFZDs/VWWAygdY2y5bzySsMl275hFsQzkB/qDPlmrlMeqe23fLyMqBQlsDrJKAnE0ro9
         idu+VBVAIEUbmtS0jdis9Gb4ow91TBEF+fUZdty8IkviN0PNCELmhgPzre1+auIx/sFk
         rO+w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708898727; x=1709503527;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=DkKvfyrMBWj0+DlETuXifPbtp+G+CbFdvNnXMM5MeUs=;
        b=tuyfIoDRsCFi3BwC44ftsNYtKOyDaz7OcDr+AfEXiIn/5YW1KJqHxFFibgVmEYcB2K
         Ys7vFOl3ZWmqxmYYj4lMhBpqCjODTn2gK4DFbjCYFNTkiOWkdFIWaEzqxmnJ/sOq+MnK
         HsBUfdnQTrgitUJo65z3Uv+6mXWeuKlBeV/0izPY13ORt8AIFx6Pa+X0QDqSDlCXzdeQ
         woivWAdjSZtcjoo41Rp42tTBU+n3Y2UDsrlAFuKOg5n1x95i3nkK1erBgFxyGTnfXTzH
         vZAlRPhTCjC+br8hSif4U78+D2VOIALuQrgSAayWYnaC4+lDrg0M3hFQtH74TATrjVCw
         inkw==
X-Forwarded-Encrypted: i=1; AJvYcCUmqBwbDWkQJtCrAZVJcGDQutH4UCYmgZsPKHRSiBHuG981BPNHafZaORhbvoFa6zflBRPlK5yai31259KUkQlMRisOYp1g2CA9cr/CiJA=
X-Gm-Message-State: AOJu0Yw9rmPlDNLzootJBGsQC3CxPTXS1cHcdut+I12qiSP/A5qJa0/R
	960zQoUNnfTE6/shfatICJ0AfJqgzlqr92sx1fHfnQei7wjzjuV9YoFb0aabCFLlrQ+W3rD+snj
	lcENuymWtesWb5mk7oltZ7VzjRwg=
X-Google-Smtp-Source: AGHT+IHPh56IyeB2dNiGOUUfRg2lexOS+IfInlHodGvxOzUuY8w69pmiHjOZgl3KhqRNlKV7KBSPMR5uatl6c+8GdWo=
X-Received: by 2002:aa7:df83:0:b0:564:ded0:6072 with SMTP id
 b3-20020aa7df83000000b00564ded06072mr3954699edy.1.1708898726812; Sun, 25 Feb
 2024 14:05:26 -0800 (PST)
MIME-Version: 1.0
References: <20240219181627.282097-1-marmarek@invisiblethingslab.com>
In-Reply-To: <20240219181627.282097-1-marmarek@invisiblethingslab.com>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Sun, 25 Feb 2024 17:05:15 -0500
Message-ID: <CAKf6xps-wWgFiwwdAF26-HxXPdaLism5viD=NY=g5EWJGLTCVg@mail.gmail.com>
Subject: Re: [PATCH 1/2] hw/xen: detect when running inside stubdomain
To: =?UTF-8?Q?Marek_Marczykowski=2DG=C3=B3recki?= <marmarek@invisiblethingslab.com>
Cc: qemu-devel@nongnu.org, Stefano Stabellini <sstabellini@kernel.org>, 
	Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
	"open list:X86 Xen CPUs" <xen-devel@lists.xenproject.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 19, 2024 at 1:17=E2=80=AFPM Marek Marczykowski-G=C3=B3recki
<marmarek@invisiblethingslab.com> wrote:
>
> Introduce global xen_is_stubdomain variable when qemu is running inside
> a stubdomain instead of dom0. This will be relevant for subsequent
> patches, as few things like accessing PCI config space need to be done
> differently.
>
> Signed-off-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblethingsl=
ab.com>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

Thanks,
Jason


From xen-devel-bounces@lists.xenproject.org Sun Feb 25 22:09:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 25 Feb 2024 22:09:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685219.1065623 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reMh4-0007ou-Sm; Sun, 25 Feb 2024 22:09:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685219.1065623; Sun, 25 Feb 2024 22:09:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reMh4-0007on-Nr; Sun, 25 Feb 2024 22:09:58 +0000
Received: by outflank-mailman (input) for mailman id 685219;
 Sun, 25 Feb 2024 22:09:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FNuJ=KC=gmail.com=jandryuk@srs-se1.protection.inumbo.net>)
 id 1reMh3-0007of-Mw
 for xen-devel@lists.xenproject.org; Sun, 25 Feb 2024 22:09:57 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9b83d86c-d42a-11ee-98f5-efadbce2ee36;
 Sun, 25 Feb 2024 23:09:55 +0100 (CET)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-56533e30887so2382574a12.1
 for <xen-devel@lists.xenproject.org>; Sun, 25 Feb 2024 14:09:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9b83d86c-d42a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708898995; x=1709503795; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=rpmwRK0QweucH9b9Rs/GnZrRDx/RfA9K31A46yAtE+M=;
        b=B9qU65vkuf8irOrijXLQWYgWiXYAC67LH71PghTogwaa2cPxbT12zjGZcdbv/vO6CB
         ouvPcAOgJ4LYU5A86XTHrhCIUmnQ/m/n8yzESayHByNyfFIlw5eIb7a5SO+IK6OQnE4h
         UAb5t/FuwzLu4yTmkVMXyJoLAxib5SAyEmcvLO1rpw8ahuFJ1AQ1m8ae7Q60dvMVjoZd
         AKzXecQeMPoWLLukLPudB9N/9ff8U9akIStRqizWTKMZSPbXbI1siY9mW+t404nkh5bS
         qtKWNjndH+bOvusPvOQX0dTUWLDCtv9gysWjgf4r8i2viNf88qF9nz0s0E/ZcAlFDgHH
         HZwg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708898995; x=1709503795;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=rpmwRK0QweucH9b9Rs/GnZrRDx/RfA9K31A46yAtE+M=;
        b=AwZSjtAErW0Z6fSTYgkYJN5aScNRWfFOak65Q0sVYFQ8SNpK8R//mZJOUY6BG5I6E5
         D048WrTPHroAJonFuORT4jpLeUKlZ66jAdv8er+iAnQgn2xh57I+9X9fM5sxi6wqsKuO
         J3U3umR60Tj/KJ6NjiW6cjAX9tCVpjqlj4JmrhkcA1FEIi5HNL4Rgoyqt5ZDT8K6QFAl
         BgaDC5w17ALHHmkIj2qB3byaowe4qLLBzkPqMcQhEAMvehpGd6eEOnQ34/zFpEtDzCGQ
         xS/f5iVqcYySTAG9rc+tBCaFQ4JKY0OEVjrdHM8nD5FuFkByXZGXeNxyV0UpSChePCEG
         hbXw==
X-Forwarded-Encrypted: i=1; AJvYcCUHkXEMxtJYghDBMv0WvMpF+mww0YJBXMu35ScypW1EH3CyTH1Q9FwLmUOhcqUinYMoBWa9M7B5QSvW2ja4rIBe8NTXekbzoiv7D48MCGQ=
X-Gm-Message-State: AOJu0YwybCug1Jd3pDuNasOJskt3GAWVadBmD4Cl8w7wMPsFCOmVVmIL
	2IBjwfiaEKk6ktQMktMe6xUxeLvT8ol/WETSNwHekRGQKN11/Ju731lb8bDeE0JDLgkRNea3kmr
	yWHQhk9/3Y1zpivVSfAGf1x80mOs=
X-Google-Smtp-Source: AGHT+IH5E3VRU8H5r8MxSJ69s61CROiFH/aDaz3qpmcnhpDcgzaqoQggtpdDq4fx2fyMmLEawOzXhXI2jdHbOujNZkE=
X-Received: by 2002:a05:6402:717:b0:565:cc2d:533 with SMTP id
 w23-20020a056402071700b00565cc2d0533mr1513727edx.1.1708898995198; Sun, 25 Feb
 2024 14:09:55 -0800 (PST)
MIME-Version: 1.0
References: <20240219181627.282097-1-marmarek@invisiblethingslab.com> <429a5a27-21b9-45bd-a1a6-a1c2ccc484c9@linaro.org>
In-Reply-To: <429a5a27-21b9-45bd-a1a6-a1c2ccc484c9@linaro.org>
From: Jason Andryuk <jandryuk@gmail.com>
Date: Sun, 25 Feb 2024 17:09:43 -0500
Message-ID: <CAKf6xpsG9bhRv9QPwhXtCoM+Zy+JCy+ceFEP2j-ZVVZC_yOe1w@mail.gmail.com>
Subject: Re: [PATCH 1/2] hw/xen: detect when running inside stubdomain
To: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= <philmd@linaro.org>
Cc: =?UTF-8?Q?Marek_Marczykowski=2DG=C3=B3recki?= <marmarek@invisiblethingslab.com>, 
	qemu-devel@nongnu.org, Paolo Bonzini <pbonzini@redhat.com>, 
	Claudio Fontana <cfontana@suse.de>, Stefano Stabellini <sstabellini@kernel.org>, 
	Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
	"open list:X86 Xen CPUs" <xen-devel@lists.xenproject.org>, David Woodhouse <dwmw2@infradead.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 20, 2024 at 1:50=E2=80=AFAM Philippe Mathieu-Daud=C3=A9
<philmd@linaro.org> wrote:
>
> On 19/2/24 19:16, Marek Marczykowski-G=C3=B3recki wrote:
> > Introduce global xen_is_stubdomain variable when qemu is running inside
> > a stubdomain instead of dom0. This will be relevant for subsequent
> > patches, as few things like accessing PCI config space need to be done
> > differently.
> >
> > Signed-off-by: Marek Marczykowski-G=C3=B3recki <marmarek@invisiblething=
slab.com>
> > ---
> >   hw/xen/xen-legacy-backend.c | 15 +++++++++++++++
> >   include/hw/xen/xen.h        |  1 +
> >   system/globals.c            |  1 +
> >   3 files changed, 17 insertions(+)
>
>
> > diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h
> > index 37ecc91fc3..ecb89ecfc1 100644
> > --- a/include/hw/xen/xen.h
> > +++ b/include/hw/xen/xen.h
> > @@ -36,6 +36,7 @@ enum xen_mode {
> >   extern uint32_t xen_domid;
> >   extern enum xen_mode xen_mode;
> >   extern bool xen_domid_restrict;
> > +extern bool xen_is_stubdomain;
> >
> >   int xen_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num);
> >   int xen_set_pci_link_route(uint8_t link, uint8_t irq);
> > diff --git a/system/globals.c b/system/globals.c
> > index b6d4e72530..ac27d88bd4 100644
> > --- a/system/globals.c
> > +++ b/system/globals.c
> > @@ -62,6 +62,7 @@ bool qemu_uuid_set;
> >   uint32_t xen_domid;
> >   enum xen_mode xen_mode =3D XEN_DISABLED;
> >   bool xen_domid_restrict;
> > +bool xen_is_stubdomain;
>
> Note for myself, Paolo and Claudio, IIUC these fields belong
> to TYPE_XEN_ACCEL in accel/xen/xen-all.c. Maybe resulting in
> smth like:

I think some of these are used by the KVM Xen-emulation, so they can't
just be moved into the Xen accelerator.  David?

Regards,
Jason


From xen-devel-bounces@lists.xenproject.org Sun Feb 25 22:55:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Sun, 25 Feb 2024 22:55:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685223.1065631 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reNOf-0005ZS-1K; Sun, 25 Feb 2024 22:55:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685223.1065631; Sun, 25 Feb 2024 22:55:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reNOe-0005ZL-Un; Sun, 25 Feb 2024 22:55:00 +0000
Received: by outflank-mailman (input) for mailman id 685223;
 Sun, 25 Feb 2024 22:55:00 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reNOe-0005ZB-6w; Sun, 25 Feb 2024 22:55:00 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reNOe-0003Xr-2g; Sun, 25 Feb 2024 22:55:00 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reNOd-0003Tp-K6; Sun, 25 Feb 2024 22:54:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1reNOd-0000C1-Jh; Sun, 25 Feb 2024 22:54:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Eqr8O2rwvQGi28uLtowinEGx7awVmOpsdX4JzPd5CfI=; b=CBh+S7ZXqUqNcMyM04T8yojc1B
	U5IsISM1NTFgny4J36M6pRRz7d7wzuOtgKBvlqwbPIWZazeZvNte1pAAL42fIjx5tqdsWoa/qnIGb
	/YOU6Mby7fmq/FA0ufJNNi8HXH0JjpeOSXI0IEmV64rMOcuoZZgxApXVeAxAFIlfnE6A=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184761-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184761: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=ba96acd963e794e86633ffd5e7f96cbc03db673c
X-Osstest-Versions-That:
    ovmf=f881b4d129602a49e3403043fc27550a74453234
From: osstest service owner <osstest-admin@xenproject.org>
Date: Sun, 25 Feb 2024 22:54:59 +0000

flight 184761 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184761/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 ba96acd963e794e86633ffd5e7f96cbc03db673c
baseline version:
 ovmf                 f881b4d129602a49e3403043fc27550a74453234

Last test of basis   184759  2024-02-25 17:44:36 Z    0 days
Testing same since   184761  2024-02-25 20:12:59 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  David Woodhouse <dwmw@amazon.co.uk>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   f881b4d129..ba96acd963  ba96acd963e794e86633ffd5e7f96cbc03db673c -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 01:03:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 01:03:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685239.1065642 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rePOd-0002as-QV; Mon, 26 Feb 2024 01:03:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685239.1065642; Mon, 26 Feb 2024 01:03:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rePOd-0002al-MV; Mon, 26 Feb 2024 01:03:07 +0000
Received: by outflank-mailman (input) for mailman id 685239;
 Mon, 26 Feb 2024 01:03:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rePOb-0002ab-Jo; Mon, 26 Feb 2024 01:03:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rePOb-00053G-Db; Mon, 26 Feb 2024 01:03:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rePOb-0006dK-3V; Mon, 26 Feb 2024 01:03:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rePOb-0006YM-3B; Mon, 26 Feb 2024 01:03:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Gt2RACoktzyx+qe/klxSEE5FC+KBrglQz9neF8rbLpo=; b=4emAUTQelSRrYyOeJvWk9OS1ec
	3LElUT2G9mWKLrY9+OfKYCN4gyJ/FWQqXV9dhyvX2tIsblz/a0fBYk/He5PQU4aADa/+OkgWuFCq2
	Ek1GIMlp3X2LgZ2TNIxuP7jiJMl1WDLeYotCijtBJimxn8BDzcMi7gVam+asfCLHCP8c=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184760-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184760: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=4ca0d9894fd517a2f2c0c10d26ebe99ab4396fe3
X-Osstest-Versions-That:
    linux=ab0a97cffa0bb3b529ca08b0caea772ddb3e0b5c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 26 Feb 2024 01:03:05 +0000

flight 184760 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184760/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check fail blocked in 184757
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184757
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184757
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184757
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184757
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184757
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184757
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184757
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                4ca0d9894fd517a2f2c0c10d26ebe99ab4396fe3
baseline version:
 linux                ab0a97cffa0bb3b529ca08b0caea772ddb3e0b5c

Last test of basis   184757  2024-02-25 02:59:57 Z    0 days
Testing same since   184760  2024-02-25 18:12:13 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Al Viro <viro@zeniv.linux.org.uk>
  Christian Brauner <brauner@kernel.org>
  Gao Xiang <hsiangkao@linux.alibaba.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Miklos Szeredi <mszeredi@redhat.com>
  Sandeep Dhavale <dhavale@google.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   ab0a97cffa0bb..4ca0d9894fd51  4ca0d9894fd517a2f2c0c10d26ebe99ab4396fe3 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 01:20:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 01:20:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685245.1065652 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rePes-0004J8-CB; Mon, 26 Feb 2024 01:19:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685245.1065652; Mon, 26 Feb 2024 01:19:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rePes-0004J1-9T; Mon, 26 Feb 2024 01:19:54 +0000
Received: by outflank-mailman (input) for mailman id 685245;
 Mon, 26 Feb 2024 01:19:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fECz=KD=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1rePer-0004Iv-S6
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 01:19:53 +0000
Received: from NAM02-SN1-obe.outbound.protection.outlook.com
 (mail-sn1nam02on20618.outbound.protection.outlook.com
 [2a01:111:f400:7ea9::618])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 230c73b8-d445-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 02:19:52 +0100 (CET)
Received: from DM6PR04CA0027.namprd04.prod.outlook.com (2603:10b6:5:334::32)
 by IA1PR12MB6092.namprd12.prod.outlook.com (2603:10b6:208:3ec::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.34; Mon, 26 Feb
 2024 01:19:47 +0000
Received: from CY4PEPF0000FCBF.namprd03.prod.outlook.com
 (2603:10b6:5:334:cafe::4b) by DM6PR04CA0027.outlook.office365.com
 (2603:10b6:5:334::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend
 Transport; Mon, 26 Feb 2024 01:19:47 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000FCBF.mail.protection.outlook.com (10.167.242.101) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 26 Feb 2024 01:19:46 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 25 Feb
 2024 19:19:46 -0600
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 25 Feb 2024 19:19:42 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 230c73b8-d445-11ee-8a58-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=e7ClIod9/T+1uLPj74NWlg5VZUwLxUwK+i2tAteqpYaD+SQe46ClpeKIDuO7oeo25lenf8FfLriAzDywCYYt//icJRwjk17VUgtGcNigITM9VysKM//BoTB+4Z2bM+iTR1FeRhd2kR4YrWI/R/r5COt95j/KpP0EfJAvHM4GZI1obmwr59luL5rapDIw2H8yhtQZUsV/ODnySkClTMExDSQ1GYt8MfFC6riYHtS2rX0gBbGKypfpIWurEj9PnWVMkqTYeeEWtd16e+jUZ2tQFGYC4JXQg0NbEHWyI7Qr9OcRVaCNIqCY+/56u1kGgPt6V/eTO64rN/ltMkfabV6PsA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=SbHROJquyYqurmbhug7zXj703l2PSAlL8189exDZXOA=;
 b=BL24vCh8KL2RX8d0DTfksMMknxn3C+1MG4PyvAkHGhF7yX9F8uf1oDRfMp1gw3ftcx17gb5Qg9WiNQTdyCiGVYXTs7BoeQ+abB3YWFdSXplqKylD6ykUkZxze5GHT7WnF0o9zrOArBEA0MOLQxjPT1ohGUsZz47NItQhneCbmvK1LSRgUKtHAFK4/3g5FpAHdymd8LpvfIIRiOtRvTGNOr/b2gmqfSKbqJXw/nPJ+oa9pzmkAPOUXRlG0/4JRbr8/GWFiKOHM5gAmoa03T0lH2RxBUnllrMSEljR4P2KSNZGVxTF3q4p/ZVfIqCA0DsNcAN+/fr4bfbTusX0NaZwqg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=SbHROJquyYqurmbhug7zXj703l2PSAlL8189exDZXOA=;
 b=KNzUkG1FueEvtx1+jD24lm5xZ3SAUglMBqV8J4kD3GJQQJZbX2+Y8sKo3fL6UxgfuGnmAiOgUJ6aVRsBDdDUZ8qiyAMzzUN1nEdeAfOrzqjwPWW3CrB7T8kxhrI8jw+sc9eRFJtD5eT/Ag9OyAs5uthr4jLO89wUJVQmXNFKiDg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Wei Liu <wl@xen.org>, Anthony PERARD
	<anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Shawn Anastasio
	<sanastasio@raptorengineering.com>, Alistair Francis
	<alistair.francis@wdc.com>, Bob Eshleman <bobbyeshleman@gmail.com>, "Connor
 Davis" <connojdavis@gmail.com>, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?=
	<roger.pau@citrix.com>, Alec Kwapis <alec.kwapis@medtronic.com>
Subject: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct mapped domains
Date: Mon, 26 Feb 2024 09:19:35 +0800
Message-ID: <20240226011935.169462-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCBF:EE_|IA1PR12MB6092:EE_
X-MS-Office365-Filtering-Correlation-Id: 391c704a-55fe-4af5-1076-08dc3669056e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	eyn99XQB/SQCOD1e7A2jwMzKjbEdV1qri+tm5VsMuP/YzgHHM6szpAVr1iaY1g3XcjhGCNUc0sB95ZtO0xVwhqTHhAgo4LhHahAxF8hUtDmVVgnz9gf41+PBoLxq9YFgwvSX+lDCDRMds3X841YIWgt7KivAgFLN253Mq+j4uzh+rbfcwj08hqCEG6Z2TlgXwkjYT433lGWXdevf66Ndn2eOwuQK3w6TJLA0YS6uidzzXKELlquCtNDuhfMDJ/UrDVTRs0R/XVa0Wd+7dSl4S5iC4AzT1cmhquTmwO5Or8iZdwvZPg9So4t1wZksDCydjbTNh4ulCRVnumitmwT3oN8nZWXyFgY+3ePMxg04RV+AKFnFPDnAt9H7MTxYe40Kmq5/o5MquP3zo2X1Bod0SWPPxK6/ih98D7Hm0q846r0rL1eCcudIYugQoa+0+w0wveXSdauxcygb3KTXlgcw9MgmEwtt2pwqcQsfEUr9g1fUnJAbDtK9/DJvcrLEbN9hA4Gf7X6UnG0godr6IrsB5k/mgld+RfPx+ghMjZm7FreHetlwkxoQ1Zfowcf4r159FmkXwx7pavn86lF/ds68G+ZCl8pos1InnQNSOTHC48EKD5MaTzoJqUkvqUNB7dRkuo1hyaWawXA0C7rv83tfj7qaJMN7sz+8yJEbM16sbua+1bygqGc9LLQohSLAYU5wCUtLVsyD4/sE8oOmt2AwhytzndPsGUhp5n0pxd1FvyhoRS8VpDTcJ0OwokgtZo8C
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 01:19:46.9395
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 391c704a-55fe-4af5-1076-08dc3669056e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCBF.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6092

An error message can seen from the init-dom0less application on
direct-mapped 1:1 domains:
```
Allocating magic pages
memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
Error on alloc magic pages
```

This is because populate_physmap() automatically assumes gfn == mfn
for direct mapped domains. This cannot be true for the magic pages
that are allocated later for Dom0less DomUs from the init-dom0less
helper application executed in Dom0.

Force populate_physmap to take the "normal" memory allocation route for
the magic pages even for 1:1 Dom0less DomUs. This should work as long
as the 1:1 Dom0less DomU doesn't have anything else mapped at the same
guest address as the magic pages:
- gfn 0x39000 address 0x39000000
- gfn 0x39001 address 0x39001000
- gfn 0x39002 address 0x39002000
- gfn 0x39003 address 0x39003000
Create helper is_magic_gpfn() for Arm to assist this and stub helpers
for non-Arm architectures to avoid #ifdef. Move the definition of the
magic pages on Arm to a more common place.

Note that the init-dom0less application of the diffenent Xen version
may allocate all or part of four magic pages for each DomU.

Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
 tools/libs/guest/xg_dom_arm.c   |  6 ------
 xen/arch/arm/include/asm/mm.h   | 13 +++++++++++++
 xen/arch/ppc/include/asm/mm.h   |  5 +++++
 xen/arch/riscv/include/asm/mm.h |  6 ++++++
 xen/arch/x86/include/asm/mm.h   |  5 +++++
 xen/common/memory.c             |  2 +-
 xen/include/public/arch-arm.h   |  6 ++++++
 7 files changed, 36 insertions(+), 7 deletions(-)

diff --git a/tools/libs/guest/xg_dom_arm.c b/tools/libs/guest/xg_dom_arm.c
index 2fd8ee7ad4..8c579d7576 100644
--- a/tools/libs/guest/xg_dom_arm.c
+++ b/tools/libs/guest/xg_dom_arm.c
@@ -25,12 +25,6 @@
 
 #include "xg_private.h"
 
-#define NR_MAGIC_PAGES 4
-#define CONSOLE_PFN_OFFSET 0
-#define XENSTORE_PFN_OFFSET 1
-#define MEMACCESS_PFN_OFFSET 2
-#define VUART_PFN_OFFSET 3
-
 #define LPAE_SHIFT 9
 
 #define PFN_4K_SHIFT  (0)
diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h
index cbcf3bf147..17149b4635 100644
--- a/xen/arch/arm/include/asm/mm.h
+++ b/xen/arch/arm/include/asm/mm.h
@@ -428,6 +428,19 @@ static inline void page_set_xenheap_gfn(struct page_info *p, gfn_t gfn)
     } while ( (y = cmpxchg(&p->u.inuse.type_info, x, nx)) != x );
 }
 
+#define MAGIC_PAGE_N_GPFN(n)     ((GUEST_MAGIC_BASE >> PAGE_SHIFT) + n)
+static inline bool is_magic_gpfn(xen_pfn_t gpfn)
+{
+    unsigned int i;
+    for ( i = 0; i < NR_MAGIC_PAGES; i++ )
+    {
+        if ( gpfn == MAGIC_PAGE_N_GPFN(i) )
+            return true;
+    }
+
+    return false;
+}
+
 #endif /*  __ARCH_ARM_MM__ */
 /*
  * Local variables:
diff --git a/xen/arch/ppc/include/asm/mm.h b/xen/arch/ppc/include/asm/mm.h
index a433936076..8ad81d9552 100644
--- a/xen/arch/ppc/include/asm/mm.h
+++ b/xen/arch/ppc/include/asm/mm.h
@@ -256,4 +256,9 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
     return true;
 }
 
+static inline bool is_magic_gpfn(xen_pfn_t gpfn)
+{
+    return false;
+}
+
 #endif /* _ASM_PPC_MM_H */
diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
index 07c7a0abba..a376a77e29 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -3,6 +3,7 @@
 #ifndef _ASM_RISCV_MM_H
 #define _ASM_RISCV_MM_H
 
+#include <public/xen.h>
 #include <asm/page-bits.h>
 
 #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
@@ -20,4 +21,9 @@ unsigned long calc_phys_offset(void);
 
 void turn_on_mmu(unsigned long ra);
 
+static inline bool is_magic_gpfn(xen_pfn_t gpfn)
+{
+    return false;
+}
+
 #endif /* _ASM_RISCV_MM_H */
diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
index 7d26d9cd2f..f385f36d78 100644
--- a/xen/arch/x86/include/asm/mm.h
+++ b/xen/arch/x86/include/asm/mm.h
@@ -628,4 +628,9 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
     return (mfn + nr) <= (virt_to_mfn(eva - 1) + 1);
 }
 
+static inline bool is_magic_gpfn(xen_pfn_t gpfn)
+{
+    return false;
+}
+
 #endif /* __ASM_X86_MM_H__ */
diff --git a/xen/common/memory.c b/xen/common/memory.c
index b3b05c2ec0..ab4bad79e2 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
         }
         else
         {
-            if ( is_domain_direct_mapped(d) )
+            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
             {
                 mfn = _mfn(gpfn);
 
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index a25e87dbda..58aa6ff05b 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -476,6 +476,12 @@ typedef uint64_t xen_callback_t;
 #define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
 #define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
 
+#define NR_MAGIC_PAGES 4
+#define CONSOLE_PFN_OFFSET 0
+#define XENSTORE_PFN_OFFSET 1
+#define MEMACCESS_PFN_OFFSET 2
+#define VUART_PFN_OFFSET 3
+
 #define GUEST_RAM_BANKS   2
 
 /*
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 03:02:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 03:02:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685249.1065662 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reRFh-0008Jq-GP; Mon, 26 Feb 2024 03:02:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685249.1065662; Mon, 26 Feb 2024 03:02:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reRFh-0008Jj-Cz; Mon, 26 Feb 2024 03:02:01 +0000
Received: by outflank-mailman (input) for mailman id 685249;
 Mon, 26 Feb 2024 03:01:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fECz=KD=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1reRFf-0008Jd-I9
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 03:01:59 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6622f974-d453-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 04:01:56 +0100 (CET)
Received: from BYAPR21CA0029.namprd21.prod.outlook.com (2603:10b6:a03:114::39)
 by DS0PR12MB7678.namprd12.prod.outlook.com (2603:10b6:8:135::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.34; Mon, 26 Feb
 2024 03:01:53 +0000
Received: from SJ5PEPF000001CC.namprd05.prod.outlook.com
 (2603:10b6:a03:114:cafe::4b) by BYAPR21CA0029.outlook.office365.com
 (2603:10b6:a03:114::39) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.22 via Frontend
 Transport; Mon, 26 Feb 2024 03:01:53 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ5PEPF000001CC.mail.protection.outlook.com (10.167.242.41) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 26 Feb 2024 03:01:52 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Sun, 25 Feb
 2024 21:01:50 -0600
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Sun, 25 Feb 2024 21:01:48 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6622f974-d453-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fLGKUXYMu+28U4BGUOgJ67q9z2lePw+QZx4RmDiLCps4THboznNFCHg3gkJERS2yRdPL060RIKby4QbqgpNNDdA/wfz5kW2r1k0G9SQuxhIf3BVXeRJx57Bju1KjTwSv3orCLq6GQMm0Zmcv/z5YMreo07Alc45NJDS+XPRFWOvHnWPclr3uUX3nW/BSsI+B5pD6juEdjs+zS1Dncrt9hoeLtorNAyO6G3KJEfT266t7JkeMXevg8MKU4Ierx0ymSX4Tcqkcq8aNFt97AfO1MsEuxVbDnnUu4HSR8pNI0Ay7MulfoIXmkeUFwIxqzVWwxKbf/eD2Bm2v3alyS8s+/Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Sb+v1j9xyb+ru/VnYV1pVUwgPOA04LuMrPTFdg1rbn4=;
 b=Rw0wdxDknUavq/CDSAYI/tXLdpyd4cNJufEaDiaG3VaDtaw81kprJEY0eS0gnrYIxkp/6uEE8QdDruNqwkXBcNci32Uuu8PP6QoRG0NgFow7tbLhx3hCSYjXScw1bwwPeDz6Zb7l0aaw/yLX7IOVOsWea4cO9LzpJewVrH6ZPP1j+IQAkJQ5firhBmGCMUS1R8mVjgLyDTcgnVwlv4ifTYBLhBC4/Wpv5LCy9eq1r12HmGkR+0dOUQ/pXXQ1itfTbDYJiLXcFW2kLDNvhyLvS8x9zxy9SN0DbHDqrsSfiH5dvJcmAhsClvzr9BBmj/ah7mB2H5ltUZ6JAW1/zG7vhg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Sb+v1j9xyb+ru/VnYV1pVUwgPOA04LuMrPTFdg1rbn4=;
 b=EadhCkETQXE065DGOvare7F3O/eoy0I5x82lTMUsNzTovLg97w1viyuqCbVXUy8+2aXoD/VHOa3rFNEJt9raxR8Gy8Lm0uinDBEwpadqsCxPMn1x0cee4SI6YevdKMTSMFQIAgs8OWbS5NvEyalyLBcXLrJRJKHqk3oS0Bm6d9Y=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
Subject: [PATCH] xen/arm: Set correct per-cpu cpu_core_mask
Date: Mon, 26 Feb 2024 11:01:46 +0800
Message-ID: <20240226030146.179465-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ5PEPF000001CC:EE_|DS0PR12MB7678:EE_
X-MS-Office365-Filtering-Correlation-Id: 93a58300-57cd-4f5c-c4b6-08dc367748bd
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	nxRUaZoVmBj4tGx+iH7zBh5CxNrWO2qDiEYg80ZgzTYWR1oAYHZV2VYuG19MlYvvb6jlE8Tm0y/v9rT85IKz//lfY/BTKSL89kK1LB7jkNnTKC6acKUNElztJbML2UtnYPE+V41YKNcwCQaZLcwdehlQJdLhK7G5qFIUJAyLZDnt9ReOMHgnn67oSKPjN30xn3yy8k61IMtzv1RxbPEL9y4X1OC7iBL+aRdQ+KHfTg5v5YW86ttOIi/DlDDgbVJcNt2a9iHT8Qa8Idpuf7e65Wbt+bD4iUCY7r2C2TTH0TiKu3PDN21nGSFSTS/uyPexkNkoGuRybK1PEbHTA1FXWCqFfl25lhyFxcPq+CYXSgTvqNFwXXz1SMuoZtETbr7KJo6nYJWsMTW7Br/bWT4opxKeYEjRCGD3Y9CzgQpMskUvHkxETl2QzFSK0TCS0lzY4sz74IxgdT0FXmOz2IXd+DRo/LkF7Lpnd7+KWSaMSkqrHj1OaglqPuCetCOCdjxjfMT0ibAn+wHJWdkZVNX3YaCyW61nrQB6ZkV8EY1V7yxc6s4uAYB/2SxPFfcyzt+9mRnFcn0y47N/lDHnkxfGLKDuaSAE8ihxz0t1jgJj3s6dVNFKaf/bnLA+pU7nRZHAnYUeLR4p3gsxIPSwgSZlcwEq9DBI2XfOlVu07/mceMQ90rnV+HQwwnCUas1jhiEnsrQrk0kfdgxu6ER+BluT+m0byfys+SC3VuZUj15A7Ls0njHCbzQDY8hoDXZUx3K2
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 03:01:52.8078
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 93a58300-57cd-4f5c-c4b6-08dc367748bd
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ5PEPF000001CC.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7678

In the common sysctl command XEN_SYSCTL_physinfo, the value of
cores_per_socket is calculated based on the cpu_core_mask of CPU0.
Currently on Arm this is a fixed value 1 (can be checked via xl info),
which is not correct. This is because during the Arm CPU online
process at boot time, setup_cpu_sibling_map() only sets the per-cpu
cpu_core_mask for itself.

cores_per_socket refers to the number of cores that belong to the same
socket (NUMA node). Currently Xen on Arm does not support physical
CPU hotplug and NUMA. Therefore if the MT bit (bit 24) in MPIDR_EL1
is 0, cores_per_socket means all possible CPUs detected from the device
tree. Setting the per-cpu cpu_core_mask in setup_cpu_sibling_map()
accordingly. Drop the in-code comment which seems to be outdated.

Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
 xen/arch/arm/smpboot.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index a84e706d77..d616778655 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -66,7 +66,6 @@ static bool cpu_is_dead;
 
 /* ID of the PCPU we're running on */
 DEFINE_PER_CPU(unsigned int, cpu_id);
-/* XXX these seem awfully x86ish... */
 /* representing HT siblings of each logical CPU */
 DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
 /* representing HT and core siblings of each logical CPU */
@@ -89,6 +88,11 @@ static int setup_cpu_sibling_map(int cpu)
     cpumask_set_cpu(cpu, per_cpu(cpu_sibling_mask, cpu));
     cpumask_set_cpu(cpu, per_cpu(cpu_core_mask, cpu));
 
+    /* PE not implemented using a multithreading type approach. */
+    if ( system_cpuinfo.mpidr.mt == 0 )
+        cpumask_or(per_cpu(cpu_core_mask, cpu),
+                   per_cpu(cpu_core_mask, cpu), &cpu_possible_map);
+
     return 0;
 }
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 03:05:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 03:05:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685251.1065673 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reRIt-0000Rq-Va; Mon, 26 Feb 2024 03:05:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685251.1065673; Mon, 26 Feb 2024 03:05:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reRIt-0000Rj-R2; Mon, 26 Feb 2024 03:05:19 +0000
Received: by outflank-mailman (input) for mailman id 685251;
 Mon, 26 Feb 2024 03:05:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ieDH=KD=nxp.com=peng.fan@srs-se1.protection.inumbo.net>)
 id 1reRIs-0000Rb-Ca
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 03:05:18 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05on2060e.outbound.protection.outlook.com
 [2a01:111:f400:7e1a::60e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dc811242-d453-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 04:05:14 +0100 (CET)
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com (2603:10a6:10:358::11)
 by DU2PR04MB8935.eurprd04.prod.outlook.com (2603:10a6:10:2e2::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.31; Mon, 26 Feb
 2024 03:05:12 +0000
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::1232:ed97:118f:72fd]) by DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::1232:ed97:118f:72fd%4]) with mapi id 15.20.7316.034; Mon, 26 Feb 2024
 03:05:12 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dc811242-d453-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Ckp0MSWmZ2eMeJFGoxkGm+HVdnq+69VwSWjUFK7wHX3YEXJ+G+dyHATMQWRmk31i/MbbGNKqjAHyrUhkmkaMcyv3optaBU7hS8BpYXcBZkbIaiVUQ6sdQwAyV79zrtyStb8/wF7ZYnoHjJXBNVE+SypRqKehKL8cx/1MZzvNT/EcODMpAd+7CKqhKkRbiSKFwtwrxurMjqH3fQgQKckwQYBogZJcY3SweXFLlwRIBA4zWO+WRGN+iz4WZ4zn7UEdR6hgYVhSYM3YiuiTRVf++MsP3uoLYc9hjhxZKUQ8EMnByzhEx1+cQ0hfgCxFkE530eirn3luA7bDwHmvSyQLQg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=5z3JfmvSQXMXM0K6dZ2WyrioM8mQUu8oA3IKTQPYO9Q=;
 b=ETNkzpv5o/wNDyUul9D0vaSnnBjGnLSSiKo3RhKCkbwQpxACqenEamnKmB+55iCDuL0S1XMMpEmvBfh63Gbc9lKkCpdDO4IfUF3rYnGPsHCd0na7bcN7rkcNtDQIeAkIZeXa0u/Idfo+kwyWjE7IExZIUr5oQy7PSPan0K06leLWhDNB+St4sbHapC+IY4Ln6YVyDnpLDWbgsv9Dg4UdXJQq7wnOWNa8/k4HgILxzSFtVy8x0NMWAzpe0RKV2zkul+WtJzvb8lBNoaL34bSZKRjswdUTkFamGA3+80tZ46GHcbpu0KuSVsLH/g52tGag1gpUBK4PHnYMhQ/TZucb5A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass
 header.d=nxp.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5z3JfmvSQXMXM0K6dZ2WyrioM8mQUu8oA3IKTQPYO9Q=;
 b=eFNintCmIIYcR2l56PKsUPZOzxm7ZweAFgcdXsvtY9dNnzlUByvdh9T76pRGqAbTjiOMjnEcw+nI22HtHeSKKJ17u99ZYQ6XFcIUi3b5klxXIQXpzZAqxknoSDpuGnX3QASkHejKvyEp+dm+ryFeuenWtnBYH3QeNdz2lmcPVUY=
From: Peng Fan <peng.fan@nxp.com>
To: Stefano Stabellini <stefano.stabellini@amd.com>
CC: Julien Grall <julien@xen.org>, "vikram.garhwal@amd.com"
	<vikram.garhwal@amd.com>, "oleksandr_tyshchenko@epam.com"
	<oleksandr_tyshchenko@epam.com>, Leo Yan <leo.yan@linaro.org>, Viresh Kumar
	<viresh.kumar@linaro.org>, "stewart.hildebrand@amd.com"
	<stewart.hildebrand@amd.com>, "sgarzare@redhat.com" <sgarzare@redhat.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: RE: question about virtio-vsock on xen
Thread-Topic: question about virtio-vsock on xen
Thread-Index: AdpmUSXKZvg8Aft6RRu9smUwoggtbwAUAHAAAG+7FYA=
Date: Mon, 26 Feb 2024 03:05:12 +0000
Message-ID:
 <DU0PR04MB941746B1F385B6A514704F2B885A2@DU0PR04MB9417.eurprd04.prod.outlook.com>
References:
 <DU0PR04MB941734DA793D87B7FF3A491488552@DU0PR04MB9417.eurprd04.prod.outlook.com>
 <alpine.DEB.2.22.394.2402231332580.754277@ubuntu-linux-20-04-desktop>
In-Reply-To:
 <alpine.DEB.2.22.394.2402231332580.754277@ubuntu-linux-20-04-desktop>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=nxp.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DU0PR04MB9417:EE_|DU2PR04MB8935:EE_
x-ms-office365-filtering-correlation-id: bf74059b-9df0-4132-264d-08dc3677bfa4
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 EdvJkoWHZ2/kYvPps8MOvd6Iog8CAkzHzT0gqTquOyKA2p6FJ105vQz4Ik6dnk4uN5eC4K6RCIcT9pqjoUXSBeEJvGQL2CSixtSSCiI4FOduK0Wt5yThCaGz0TR60ilUMwOFOxVTP2pJl6gQhSj1JKUtwNzO/MQBDQHFricZgaKRnfINGzJsYQ1mlal1bltYDtrTq6HD9qgZgdfhHegSZhb+1VFurfZpRjBChfSo1tMa0I/+29Pyxpw2YZskefb2pVSKzX3WWfSMjNzhW/QEQgdSpgMCzY8xIzWvKIx5uw1IUSJ/J/0HbNL/nKLRLiTzYEuZYBwhkkYGFP9O9Fjm815zdBflkGtRAuVAQOZdSXYPgwLGlqJzEhP4htEjUvjZzRwDSUarvnMsx/gcyWWwqmRCoYUcjjJsKiEdOftyAKMLC4+GrgeIWN1LeMpmkpUajKVqRVIEs9ZEdkDMahssErkI/Yac9y60MXms5OhLa4tQ4EM9J128Ftk/WS3R8yIX5RdM0ZBDNm0xJWQ4lMB1rqF7fSaTmrlabYvLOaAwWUE+NjTFhQKjX30JmfK4q/u45UbsGber77EJmjFrN2hY9mzbfDPPqWJ2E8GTXiSf1lpC3CWelZhovykEZVZ3TO5LB0VWmUlIBYmz/1NNMXoCjh6S8UoUIyyvsbFDFUssW/k9ND0vpKBHeyMpCdN8CDE7zJCzZl1ri4rSfYQaxtQ8N11m9he/d3mCjixLv4D0fk4=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU0PR04MB9417.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?h6OoFXsaYBcoaNbJRgOoFlQ7uJedUCoHMLKWDy0mxpI2vxBHt3o5mSopK8g3?=
 =?us-ascii?Q?LaxP06G915411kNL1Z5UYaX0t4ydDYGoa4jO5vyzthhfP6MrVQ3+bTtYpKhg?=
 =?us-ascii?Q?3k5+dtMgtHQUHKM0zDJmqZKeKqsUsVQAOZMZSt51gKSieJOMgnbMV6iDXt56?=
 =?us-ascii?Q?cVci3BFD99m9iHSQFu+fiwaV0BeyZ9UEn2JcSe5TWaMcKVmiiROc9skETF8y?=
 =?us-ascii?Q?kD6WGZzwifPEwekz2mjtVmDtdsyDDswketGl/eNSAph3hteYPr+WRG1gcKu2?=
 =?us-ascii?Q?RuY/7852c9Z8jsDqHsX07pNEckzUWNHVTesC7yvmbIL1Dx5LekynPJiZ+FdU?=
 =?us-ascii?Q?RtuZZHPzfzvgcl0ksXWrAUWM9vvYNYELjFxYMuDjjMnyHugHmnoszVhu6xMQ?=
 =?us-ascii?Q?FhFR6SqcLxYxPBj5HUdYhQPzTDGgnskJrALjHaCsSLh3xv3DIYfxUSWdiXAJ?=
 =?us-ascii?Q?PhhgmQFiS4zjWJM3E8vJc3/oCyLNazmxeyzDLgn2eQSNU2PDNQkNSL/tVV13?=
 =?us-ascii?Q?JvZiKe9IYjEGeHQlT3PFDXlZglmU65H8asSIWs/0+VXySrU2m6EeMRgI4Weg?=
 =?us-ascii?Q?dO1mSGFVpwOiVgL5SQuhj57nbjBnsyvWlMzwOOWtEyhAOPXDA/NfQ6f1NpXX?=
 =?us-ascii?Q?He8ycAv5ORCNZO9gLlSqm+G8jFHgNkfjL7S4X/lpJNXUARv7JLLUD6Kl+qiw?=
 =?us-ascii?Q?0jcZt2AdKSl4e7IvL9G/K7FgdTUBl65SfrWBjSpPAD0HfrYdiPx/z6Pygb7L?=
 =?us-ascii?Q?V91rODOpypO+wrhFNCODJr+lQZNs1bs+3roJcrj3y+84aAL2rjO+qYfO3oRJ?=
 =?us-ascii?Q?yEca6dDmbbQLhZrUZUlH3b+FUdVUFYkj/lqTnhDG6fZAdWHxZK4UyBmZ7k8I?=
 =?us-ascii?Q?0QvDXJWPC7Kj6/XlnsFb19UJNiANvk9zSLCuYj5t7KW5svJ+bkzsB8STk+6o?=
 =?us-ascii?Q?ZKFIEmgEozplsWJ1qZboCkeNN+pKtpVYLBBASanc8umvLLSILRnVM9k88am1?=
 =?us-ascii?Q?eneeydKnQWNeHLK3vdoIpSBfUfH9PtSVDW6XpSocu/GVCYdqAbx2ii1H7C88?=
 =?us-ascii?Q?QrEe/gfm3oanInKjD2tVQRN/IWh8VNPOraDXGrQubGs471HiiCmpajf9nmEL?=
 =?us-ascii?Q?reV389E6+5lP7srj4dXFVdG6rYuWTwDV3lr/iuYng60G718JPtvFmJyfTCbX?=
 =?us-ascii?Q?uENWBv39uJE1BZlRfIZYo2qJ9QV8hgw6pDUCzgKZawbmETmKXyR/T8cIk7gx?=
 =?us-ascii?Q?HW/4LNIoXK5Y0UnIbHoriXN8whKTK6d/KjMZLy4BXN8QzeZFYHLrE6g1N9Aq?=
 =?us-ascii?Q?pAE5cqQ/exoDC/7qDp2A4W/NNxq3hBzIDTkMOC3boPlD2tIldjo8r5Zjf5VG?=
 =?us-ascii?Q?tVTyDVOE6NyF/p3z8Ak43V2Z3nKLP+2xgeqbmluRvsGb1J3407cLNO5eiqoH?=
 =?us-ascii?Q?Vfkc4oxF1tw6x0Uaptj0svwUUvp3f0jn9XFjKhOWubLIxbwuUHtZN/wa+G6l?=
 =?us-ascii?Q?4PBUl3f7NF6HZWK/5ieD/+UcGI/DnaCFRvRWnigHZCR2N/Q7DJ8rRGEn1ofL?=
 =?us-ascii?Q?jiIFnNq2xeoIYrH0fv0=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DU0PR04MB9417.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bf74059b-9df0-4132-264d-08dc3677bfa4
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Feb 2024 03:05:12.3671
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: iXMtzwi4OhWRxsJ35IjcCCE9LhTVY7OAw956D0QLjtoLA2+XBIjzREYM5Wj+VuwmzGYPzrM0Q96zyxN7fm+xOg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB8935

Hi Stefano,

> Subject: Re: question about virtio-vsock on xen
>=20
> Hi Peng,
>=20
> We haven't tried to setup virtio-vsock yet.
>=20
> In general, I am very supportive of using QEMU for virtio backends. We us=
e
> QEMU to provide virtio-net, virtio-block, virtio-console and more.

Would you mind share how to setup virtio-console using qemu + xen?

>=20
> However, typically virtio-vsock comes into play for VM-to-VM
> communication, which is different. Going via QEMU in Dom0 just to have 1
> VM communicate with another VM is not an ideal design: it adds latency an=
d
> uses resources in Dom0 when actually we could do without it.
>=20
> A better model for VM-to-VM communication would be to have the VM talk
> to each other directly via grant table or pre-shared memory (see the stat=
ic
> shared memory feature) or via Xen hypercalls (see Argo.)

The goal is to make android trout VM run with XEN + i.MX95, so need vsock.

Thanks,
Peng.

>=20
> For a good Xen design, I think the virtio-vsock backend would need to be =
in
> Xen itself (the hypervisor).
>=20
> Of course that is more work and it doesn't help you with the specific que=
stion
> you had below :-)
>=20
> For that, I don't have a pointer to help you but maybe others in CC have.
>=20
> Cheers,
>=20
> Stefano
>=20
>=20
> On Fri, 23 Feb 2024, Peng Fan wrote:
> > Hi All,
> >
> > Has anyone make virtio-vsock on xen work? My dm args as below:
> >
> > virtio =3D [
> >
> 'backend=3D0,type=3Dvirtio,device,transport=3Dpci,bdf=3D05:00.0,backend_t=
ype=3Dqem
> u,grant_usage=3Dtrue'
> > ]
> > device_model_args =3D [
> > '-D', '/home/root/qemu_log.txt',
> > '-d',
> > 'trace:*vsock*,trace:*vhost*,trace:*virtio*,trace:*pci_update*,trace:*
> > pci_route*,trace:*handle_ioreq*,trace:*xen*',
> > '-device',
> > 'vhost-vsock-pci,iommu_platform=3Dfalse,id=3Dvhost-vsock-pci0,bus=3Dpci=
e.0,a
> > ddr=3D5.0,guest-cid=3D3']
> >
> > During my test, it always return failure in dom0 kernel in below code:
> >
> > vhost_transport_do_send_pkt {
> > ...
> >                nbytes =3D copy_to_iter(hdr, sizeof(*hdr), &iov_iter);
> >                 if (nbytes !=3D sizeof(*hdr)) {
> >                         vq_err(vq, "Faulted on copying pkt hdr %x %x %x=
 %px\n",
> nbytes, sizeof(*hdr),
> > __builtin_object_size(hdr, 0), &iov_iter);
> >                         kfree_skb(skb);
> >                         break;
> >                 }
> > }
> >
> > I checked copy_to_iter, it is copy data to __user addr, but it never
> > pass, the copy to __user addr always return 0 bytes copied.
> >
> > The asm code "sttr x7, [x6]" will trigger data abort, the kernel will
> > run into do_page_fault, but lock_mm_and_find_vma report it is
> > VM_FAULT_BADMAP, that means the __user addr is not mapped, no vma
> has this addr.
> >
> > I am not sure what may cause this. Appreciate if any comments.
> >
> > BTW: I tested blk pci, it works, so the virtio pci should work on my se=
tup.
> >
> > Thanks,
> > Peng.
> >


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 03:09:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 03:09:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685256.1065682 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reRMe-00018R-GP; Mon, 26 Feb 2024 03:09:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685256.1065682; Mon, 26 Feb 2024 03:09:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reRMe-00018K-DK; Mon, 26 Feb 2024 03:09:12 +0000
Received: by outflank-mailman (input) for mailman id 685256;
 Mon, 26 Feb 2024 03:09:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=ieDH=KD=nxp.com=peng.fan@srs-se1.protection.inumbo.net>)
 id 1reRMd-00018E-EB
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 03:09:11 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
 (mail-db8eur05on2061b.outbound.protection.outlook.com
 [2a01:111:f400:7e1a::61b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 688c2eec-d454-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 04:09:09 +0100 (CET)
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com (2603:10a6:10:358::11)
 by DU2PR04MB8935.eurprd04.prod.outlook.com (2603:10a6:10:2e2::20)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.31; Mon, 26 Feb
 2024 03:09:07 +0000
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::1232:ed97:118f:72fd]) by DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::1232:ed97:118f:72fd%4]) with mapi id 15.20.7316.034; Mon, 26 Feb 2024
 03:09:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 688c2eec-d454-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=GA/lek6+CyvjTmFcO2K6TdKf9lvh5QLoCU5MObkaxZznNLja6teehfmXuLz6BSTdHJCrPQZfq0+9S86aqbIiW6T5SsoAZGJrJ7u3MhSsiO0uFigRqm+u7B3wBfMF8q6F7EHmWpqkPxisZEl3sdqokjQv3AbQcvr1TxNqwUIzw6WiaRWK+Kl5BQRl0hUkRgpPbF+NFPR7McWuFpnozNJEWNUPdabimP3r9xZEIU88IluM3SiTudnrsB6DQUdgMKMrTdqNPDMJ9eQF7ixu3U+gldeyZIzlr1OCgMRMahuAbKEjsGbh09L1G50GVRa3OI+tf2sGR2Q7a4vfBhrCbiYIOQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=j6FBDVc9DuZ4LImNhHXFOD8atrPcOK44Z0QGLn59K/Y=;
 b=TBXa5N1wnE0GahRL9vOqIMSEMHh06+9NY57QXplulRw6t0dZsOTnI103DwMWbNAQT67ssDCuO5y4dP/0Pf/wHU3FRwWpitKTpJLyf/yZOcwe4HG7sCln+egBgOq08/dGrZ+tMEmuRL8DAmaWu/z761pgHoLj7wA3UJrrKLAHSZmnlPavSziEcE0AicCAgv1jsZS8FT6soHUZ+WBPPqEyZyaRrmW3g8o3UUbQAdr0sv39A2LFpkwqQDVaEqYAhxxavNQdufpv7eiaI4xsKCjDj/5IfsYinZyJ88OGIULxNxnVVcrz5q4C29E9tNAQCTytdNpdhe7PZGyWOGXsfLUe6w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass
 header.d=nxp.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=j6FBDVc9DuZ4LImNhHXFOD8atrPcOK44Z0QGLn59K/Y=;
 b=sSiA9JWljTw1rPeC1q9PsF04ee3XHor1Qu1WmvzlH/PL3HbmyLiy/jLFWHENG6uNjD7z59d43K+qqYNJmL5duRpIA038KjxE6FThVOs5Fqu5rcWIpDOdIV9yqroV4suEUEfmFMKOpYyZYJot2yEfKl22/xQzNoBVPIO2YGdy1bA=
From: Peng Fan <peng.fan@nxp.com>
To: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>, Stefano Stabellini
	<stefano.stabellini@amd.com>
CC: Julien Grall <julien@xen.org>, "vikram.garhwal@amd.com"
	<vikram.garhwal@amd.com>, Leo Yan <leo.yan@linaro.org>, Viresh Kumar
	<viresh.kumar@linaro.org>, Stewart Hildebrand <stewart.hildebrand@amd.com>,
	"sgarzare@redhat.com" <sgarzare@redhat.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: RE: question about virtio-vsock on xen
Thread-Topic: question about virtio-vsock on xen
Thread-Index: AdpmUSXKZvg8Aft6RRu9smUwoggtbwAUAHAAACuPlYAARFRfYA==
Date: Mon, 26 Feb 2024 03:09:07 +0000
Message-ID:
 <DU0PR04MB94175684F8B7678A7B885559885A2@DU0PR04MB9417.eurprd04.prod.outlook.com>
References:
 <DU0PR04MB941734DA793D87B7FF3A491488552@DU0PR04MB9417.eurprd04.prod.outlook.com>
 <alpine.DEB.2.22.394.2402231332580.754277@ubuntu-linux-20-04-desktop>
 <ee69684a-5aad-417a-8522-1cd1b5322bbe@epam.com>
In-Reply-To: <ee69684a-5aad-417a-8522-1cd1b5322bbe@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=nxp.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DU0PR04MB9417:EE_|DU2PR04MB8935:EE_
x-ms-office365-filtering-correlation-id: 8d301486-af32-4fb8-c49d-08dc36784bca
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 d80EalE6pLzqjT0JkulFYqdGg6nd6wLFsl1/XcE8BW8FYIQ51fvWCqnIi801rbJItPXD5R/Fnuk9zdg/KUhfy+PCnO05EgburY2RNDRSYJEA7crxr/JDxDHmZ4NzcnmyetQ9lf5MtofrSyqDbYJNghSN5qPOgJLrFaxXrBwO1dMufB1yaW+kR1Y7SDqzRu2dlN8cZSmG+dMW1nuvWE7xlnBjzCNv0LsojkOvorWMH9v88vxdAZSG/1tdXvALs1sLoO+MxtpN5i61c4BoJvvSLC5xEF62057B+XhhczHWMx6/opZXj3wvie8pTOKwxHgjngbSo2E9XauYJdj8SVxVSWl4gr5OI1m1QcCH701XQuaZ01Sc5B+EFhr9uswrjsKNAIMlEX4HZxHhfwDt4u2bd1I0pf1hMKc+smdLd6J28zXGeGE0qj7NAudxo7oidO3TZQaSkrTj7cmxl7zh/581jN0oRMbc+ULrSyNFTqx8fnOI5AxPn8qq7CVMBdN3EsE0dmgR1zcJiFxNYHmDCdnAo7quYPmji+ngKoIjC8qEBM1iN9RSp7wRgcJOqgPB3SoRYLW0O7qtIwxV6hILK1VM4oBHoiupYcGX695bD1zRc4o8fjiBoHXokPSmVnmpyQDEcn5hejlb74MrojQN+VeE6XoUKQ8SioTygC1NomdT+uUqsL4knP7t0uozGbhDglzIDOFyFRZFBmUyqq5r9NzPqg==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU0PR04MB9417.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?2RXG6+BGl+sBkIGsP0nn/+7Vs47vBvyXV5IEdraIWFPtAyGDMp+2jqPorPRq?=
 =?us-ascii?Q?d2WElVUJP+6eyxfCeTLgTOQ2imJehtOFURBJbLynGg1nrKMe4SVtuOdCYrvW?=
 =?us-ascii?Q?jAIUBUQbUcJH8FyI8Pc1tWdsWNE3WUNLG0OHF0kAaiw1MbZ0YSigKG1BSdHo?=
 =?us-ascii?Q?GOylZrciMbl5018Nrek5tBZtxPKz2nfCJr0NMpapBUQVL2bD8m+iGIfEPEIc?=
 =?us-ascii?Q?kvkUoUuZ0UpDl6tDgUI2S9VPzktgXe0Crjn6hAzcYxDd80nT/cJp+RjOFqrO?=
 =?us-ascii?Q?qkcbLhhKsFMFrjDVujGiQgCvFRenSiZDnsjchIjIT9ZGOvrQHwMQf7A3fjc4?=
 =?us-ascii?Q?pQSxdtDaZFZBZP/0PA3KdZXmu4vHvvrgMNWtn9ogabymKBqa2DSmKNwv2EhF?=
 =?us-ascii?Q?NQaxCXlgXQmoFeEZnD27jsJAZTA00LEAVHwFlHZkdA3IvSEqKZGvRhYWh1qa?=
 =?us-ascii?Q?2o/zrUdrhionbOKcqvjJcbw5USlDY4z9grtsPMn96asHiCdcWzTQ8paGrz33?=
 =?us-ascii?Q?KPsQHWXlDZ9lOgTDV8XMKUHECqEVon+PnL8R/G2ehO6yqLRW4xGxah0fbwYY?=
 =?us-ascii?Q?oKO39u8cgYIdxKKLReS/QB0s3a3lykLj+y9JaOlABSGzGRw+AQaHApWNnLoP?=
 =?us-ascii?Q?o6oSXI8DaD5l4n7kWd1qrnMf7h2RotFqARR9gl6demd64AOC8q+m3+VTnY4p?=
 =?us-ascii?Q?HMBs9nyTR6XfwL2+eWILfYj4q4nbof/1lz1kjHN082pg8NgZQyzlHbw2j0uF?=
 =?us-ascii?Q?kvgsbXJ0jTQa9T6HgdT7XLrym7yiC3NoDN0ehTPFYsUAf0uKspIjiL4I3SWg?=
 =?us-ascii?Q?WPFaIao29ADOU0zBgiVx80cHFTCu0vbXLngm5jLrXjk35LzcLuIsjCp2pViN?=
 =?us-ascii?Q?FkyAGRqA3HMEM2MqUoV8Y8SB7DC91k4eCBk9JOFgkoafZ8nDbT0apKjN+Fd3?=
 =?us-ascii?Q?Dlt9P17LfWhnvUoMaGr76BjQiYxZNpAsjkAxN7M7m54P3eJIDj5ejgL4KUxO?=
 =?us-ascii?Q?iS1L6UNgqcEvTq+5M+a/+cAHh4q9xwYFazNICpEevKyZ9ofNpjJZF38uq80x?=
 =?us-ascii?Q?U/uBI6bq4wXNU89ACQk9rEhY7a3+yvwb+6EcOKrgyaGWeqpEUm77oUJXk2yD?=
 =?us-ascii?Q?23HGxEg45uu3EUaLi7AR+eYVFY5MeNBP5Q+5aE9ANrGaCIh8OB28f5ZixbGJ?=
 =?us-ascii?Q?cern11RVGQeAQLQD+H/NRax3t9OuHfmvtSe4xzUtOYHldk9WdjtBSMd5fjZs?=
 =?us-ascii?Q?nmOe/uc+lN6zVt8whXF/F/L4FCY/4h34waCeiEdgpfwq8EXAsGRaEO0cZ9ZL?=
 =?us-ascii?Q?ixBCIauxHaXW/R0IEuVIic1tyE/ubC8d6SxsKZ1rxheAoY3dBplpWUh7L8ez?=
 =?us-ascii?Q?hUhlfd3zmFh6WOnCZjr2/6asXZQXtVDbeNbd07PfTqPH5GwTqTV9m7iGpaLn?=
 =?us-ascii?Q?EV7hwrmZYNPbfFDncVPA9d+55vbvkSSuc1ORj6ej/OJIBle1iGXoOjqpjF/T?=
 =?us-ascii?Q?tQJ7kUdOLYftpJk0PxsztPjUjX49/OxHcsVLglH/6u9TIWVh+GGkIikIIbNh?=
 =?us-ascii?Q?7LwyphH4plRCvBMQiF0=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DU0PR04MB9417.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8d301486-af32-4fb8-c49d-08dc36784bca
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Feb 2024 03:09:07.4753
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: TRTgU0glGpF3rdS3cM1i+BezGoCIjjAYE3r2S1q4csfA3jnwqBXF0kNwswvDJ9x3HacJZ4+cptUc7/EjiudTBw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB8935

Hi Oleksandr,

> Subject: Re: question about virtio-vsock on xen
>=20
>=20
>=20
> On 23.02.24 23:42, Stefano Stabellini wrote:
> > Hi Peng,
>=20
> Hello Peng, Stefano
>=20
>=20
> >
> > We haven't tried to setup virtio-vsock yet.
> >
> > In general, I am very supportive of using QEMU for virtio backends. We
> > use QEMU to provide virtio-net, virtio-block, virtio-console and more.
> >
> > However, typically virtio-vsock comes into play for VM-to-VM
> > communication, which is different. Going via QEMU in Dom0 just to have
> > 1 VM communicate with another VM is not an ideal design: it adds
> > latency and uses resources in Dom0 when actually we could do without it=
.
> >
> > A better model for VM-to-VM communication would be to have the VM talk
> > to each other directly via grant table or pre-shared memory (see the
> > static shared memory feature) or via Xen hypercalls (see Argo.)
> >
> > For a good Xen design, I think the virtio-vsock backend would need to
> > be in Xen itself (the hypervisor).
> >
> > Of course that is more work and it doesn't help you with the specific
> > question you had below :-)
> >
> > For that, I don't have a pointer to help you but maybe others in CC
> > have.
>=20
>=20
> Yes, I will try to provide some info ...
>=20
>=20
> >
> > Cheers,
> >
> > Stefano
> >
> >
> > On Fri, 23 Feb 2024, Peng Fan wrote:
> >> Hi All,
> >>
> >> Has anyone make virtio-vsock on xen work? My dm args as below:
> >>
> >> virtio =3D [
> >>
> 'backend=3D0,type=3Dvirtio,device,transport=3Dpci,bdf=3D05:00.0,backend_t=
ype=3Dqem
> u,grant_usage=3Dtrue'
> >> ]
> >> device_model_args =3D [
> >> '-D', '/home/root/qemu_log.txt',
> >> '-d',
> >> 'trace:*vsock*,trace:*vhost*,trace:*virtio*,trace:*pci_update*,trace:
> >> *pci_route*,trace:*handle_ioreq*,trace:*xen*',
> >> '-device',
> >> 'vhost-vsock-pci,iommu_platform=3Dfalse,id=3Dvhost-vsock-pci0,bus=3Dpc=
ie.0,
> >> addr=3D5.0,guest-cid=3D3']
> >>
> >> During my test, it always return failure in dom0 kernel in below code:
> >>
> >> vhost_transport_do_send_pkt {
> >> ...
> >>                 nbytes =3D copy_to_iter(hdr, sizeof(*hdr), &iov_iter);
> >>                  if (nbytes !=3D sizeof(*hdr)) {
> >>                          vq_err(vq, "Faulted on copying pkt hdr %x %x
> >> %x %px\n", nbytes, sizeof(*hdr), __builtin_object_size(hdr, 0), &iov_i=
ter);
> >>                          kfree_skb(skb);
> >>                          break;
> >>                  }
> >> }
> >>
> >> I checked copy_to_iter, it is copy data to __user addr, but it never
> >> pass, the copy to __user addr always return 0 bytes copied.
> >>
> >> The asm code "sttr x7, [x6]" will trigger data abort, the kernel will
> >> run into do_page_fault, but lock_mm_and_find_vma report it is
> >> VM_FAULT_BADMAP, that means the __user addr is not mapped, no vma
> has this addr.
> >>
> >> I am not sure what may cause this. Appreciate if any comments.
>=20
>=20
>    ... Peng, we have vhost-vsock (and vhost-net) Xen PoC. Although it is =
non-
> upstreamable in its current shape (based on old Linux version, requires s=
ome
> rework and proper integration, most likely requires involving Qemu and
> protocol changes to pass an additional info to vhost), it works with Linu=
x
> v5.10 + patched Qemu v7.0, so you can refer to the Yocto meta layer which
> contains kernel patches for the details [1].

Thanks for the pointer, I am reading the code.

>=20
> In a nutshell, before accessing the guest data the host module needs to m=
ap
> descriptors in virtio rings which contain either guest grant based DMA
> addresses (by using Xen grant mappings) or guest pseudo-physical addresse=
s
> (by using Xen foreign mappings). After accessing the guest data the host
> module needs to unmap them.

Ok, I thought  the current xen virtio code already map every ready.

>=20
> Also note, in that PoC the target mapping scheme is controlled via module
> param and guest domain id is retrieved from the device-model specific par=
t in
> the Xenstore (so Qemu/protocol are unmodified). But you might want to loo=
k
> at [2] as an example of vhost-user protocol changes how to pass that
> additional info.

Sure, thanks very much on the link. Giving a look.
>=20
> Hope that helps.

Definitely.

Thanks,
Peng.

>=20
> [1]
> https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithu=
b
> .com%2Fxen-troops%2Fmeta-xt-
> vhost%2Fcommits%2Fmain%2F&data=3D05%7C02%7Cpeng.fan%40nxp.com%7
> C56a4b63510da43a9ddf808dc356694fa%7C686ea1d3bc2b4c6fa92cd99c5c3
> 01635%7C0%7C0%7C638443961893736252%7CUnknown%7CTWFpbGZsb3
> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0
> %3D%7C0%7C%7C%7C&sdata=3DhDZLtjhzHGHzvuzPmQLwoNI8mwKeZDtVWgn
> VTF%2BX1TQ%3D&reserved=3D0
> [2]
> https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwww.
> mail-archive.com%2Fqemu-
> devel%40nongnu.org%2Fmsg948327.html&data=3D05%7C02%7Cpeng.fan%40
> nxp.com%7C56a4b63510da43a9ddf808dc356694fa%7C686ea1d3bc2b4c6fa
> 92cd99c5c301635%7C0%7C0%7C638443961893746102%7CUnknown%7CT
> WFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLC
> JXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=3DygBpkAS%2F1aCcyB20ymt%2BZ9jt
> 9T1l%2F8hzPwRQrGb35jg%3D&reserved=3D0
>=20
> P.S. May answer with a delay.
>=20
>=20
> >>
> >> BTW: I tested blk pci, it works, so the virtio pci should work on my s=
etup.
> >>
> >> Thanks,
> >> Peng.
> >>


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 06:04:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 06:04:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685259.1065692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reU61-00052d-O4; Mon, 26 Feb 2024 06:04:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685259.1065692; Mon, 26 Feb 2024 06:04:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reU61-00052W-Jb; Mon, 26 Feb 2024 06:04:13 +0000
Received: by outflank-mailman (input) for mailman id 685259;
 Mon, 26 Feb 2024 06:04:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=dckQ=KD=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1reU60-00052Q-0r
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 06:04:12 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2409::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d9cb9d64-d46c-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 07:04:08 +0100 (CET)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by LV2PR12MB5917.namprd12.prod.outlook.com (2603:10b6:408:175::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.36; Mon, 26 Feb
 2024 06:04:01 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa%5]) with mapi id 15.20.7316.032; Mon, 26 Feb 2024
 06:04:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d9cb9d64-d46c-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=bBSrLIHo3R5UUwBkElMiUPVv1elX2nl8ALqF/lt1EZFEIEmAV6qRo/pbkUh/I+y7kiUlk1lzZK++XA/Oa/4QZEw1YihC/I4g7Vb03XhhJx8aveTPp8FI1Mlnq01reasF8TdlbcQ/LxY/8SDD8gn74j72tyS9rQcTSewxlC/RJNwH/Bd1wjsSZ9XJ7/w5La7GPSG2ZM9H9BFwdRpkHPPiPRS5VAaGsBQlQ1Do4NSyLASGr37AOrBop8eLIDiWTkkQbCjxGGL876tpe7xHXOUwUFsNj2aP2n3D7Z9w2TEBbGTlssSkVUe/Frv69bnMYafttnHxRj4gnAiS+AyxxLLpiA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=Q1uDBRPAeRQ7Hl1rBOx0+UzHvS6fO69g6G8GdJbVNWs=;
 b=AKdHyI0S7zVpyYRt6KO4IyKo/MG93CUrarU1IIH0iLOig/gXxEAn/CWtY7oDcqRcCa73XL9E/SwscKhi9cX5eb1zRwIdBDBnhVyi+pyLAkOnrMWYIF1Z/RM+bbubydrChI1yuz4aaBqzsvPm0FGJ4BCMc398g0ekr4Rsd+dutfzEyFUiDxunKHLClxOOvNrSVYqS96DXaru9OUE5wlvNJyt/L5dt089QNoEvIkpNb2a1GLuF1/lpn7zgXS6r1YsWMeE/eN5adGOBQifbyKyZ8Kn33BAcSvG0WUTzXigAyVdFXcnwcrGBSODKIqmrqppjuXywBk53bh+eJR8TZMVYkg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=Q1uDBRPAeRQ7Hl1rBOx0+UzHvS6fO69g6G8GdJbVNWs=;
 b=u6lD0LizyC5oGIYIGn0udNdDezlsIGOkCV+zZCAyVoC+UOBIT/TeIrsAoY7/BPN6N78Ncni+qlfbZyJpoDQ4AtHz7mMaTmhSbx/keG/LvvBiIDHdoaZguelcmw1vYrEoFc+Mwscw6dNKfLaQwl57sEj+i69R0wKwENlaX0XPfDQ=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Stefano Stabellini <sstabellini@kernel.org>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Jan
 Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v5 3/5] x86/pvh: Add PHYSDEVOP_setup_gsi for PVH
 dom0
Thread-Topic: [RFC XEN PATCH v5 3/5] x86/pvh: Add PHYSDEVOP_setup_gsi for PVH
 dom0
Thread-Index: AQHaRR62LJvUheASc0e0D+dOwe/lNLEXWJqAgAWWDwA=
Date: Mon, 26 Feb 2024 06:04:01 +0000
Message-ID:
 <BL1PR12MB58498142AF67EB5CE27F72AEE75A2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-4-Jiqian.Chen@amd.com>
 <alpine.DEB.2.22.394.2402221642530.754277@ubuntu-linux-20-04-desktop>
In-Reply-To:
 <alpine.DEB.2.22.394.2402221642530.754277@ubuntu-linux-20-04-desktop>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: MN0PR12MB6150.namprd12.prod.outlook.com
 (15.20.7292.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|LV2PR12MB5917:EE_
x-ms-office365-filtering-correlation-id: 008e39a6-bbe2-402f-656d-08dc3690baa2
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 nEknTQk3Xxi8YTQh13y6bjzD8CLeHgUvk+NqNifc3hXcsqNfsoTG+5fvMOTE5tiisG20GpITFjW3eIJ9qsc5nqeztY7hvJxXLKfMfvE+SJMpNOafodPoAUeMD2rHib+4cI75Eu/zKW4Hiyq9yRsznsF+V3P3i6QGN0SKFlAWUzmnk77rn0XepbpfrGKNrWk5ApUrffVtFxpbzub+fVaoRv1nlXMoFrUbW/a2VvxqcjOQTpN2+YbI/apOROJ5kEZGZn0H4Z/ENikj9hbVNtxOZdkCPIFPY4CGP4JbXsh5FL7/u2OEb+ckrSJEcQSxr18uIgPL8mrqE6D+7bYjp0gWBlzDBFDLRQ+hlO8chaiXh3McoV3GuHPeblbkk2TBbQTUV93xsZJrEUiV72UynqmA7056SBtpTVowpeScHOAT8aZQbv9cW5nkZVFCANMZgJXPotTkjkiZA1PdY6+R56avQneVOv5XR03CT3dIYiHNX0PGBGfk5Kg5o0UVcud9H9dSX7emAPjocj6JR2S/JYdl5xyhvuJPRLLkZxw93WnPOds/TN9jcK7sq8VWea8TGievipxvzVMHa/xH64VHd/oPo2YZ5jxWsixR5JZhHRjOUMZ4lUpbIl8jLMDa3+EXmL07YPVYCQDy331mCzlJ7/ms75BiA9TPCO0Kq2KsoL+tf3GijaCOC3jhEIZBoL/GW5jMAxoOSHOEUSbL26ORvnCvI/YSuHQwSdGoSA/Ea4Xc1sM=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?S0MyVGNka1Y3dzNJSkpTS05iYVNFanJTVFk0bGl6akRmaU9lNTlZOWVxWXJG?=
 =?utf-8?B?Z2pMQWhSbmU1SWFaSVplZUhRMXRiOUtWMmJOc00raEtNdnprS052bUlnRkFF?=
 =?utf-8?B?Yjc3ak5ISzNiUnd0eEJtK3N3SGkwWWQ5bXdjREc1WldHVGgveVUvM0NQTUlz?=
 =?utf-8?B?aVNTNUJaWncxdUJzQVFjZ3pDTzRlWjBRU0NIRHNXRTR5OWpHSzZva3ZHUFRB?=
 =?utf-8?B?UDdLZUtYZjZGeGNDeHZsVTdhSmJpUUxZSVliaVc0Snlqb2lXUHMrZ2RBWjQ0?=
 =?utf-8?B?Zzg2K1Z1MjdZMnZmdFB4Nk11NHFGcFJLa1k3SkdWU3oxTjZhMVlJUWVFSm80?=
 =?utf-8?B?clFJT0M2dUFhcTJBSDVmazZxb0V5WmtkN2dqQk00UU1JUncvMUZObHNEamhY?=
 =?utf-8?B?bWJIN1YrS3VqWlArbGkxZUR3NFhtNzQvLzZoeCsyMXFHT3dOZ2hTOXRGZHg0?=
 =?utf-8?B?UzN6ZjYwMSsxLzk4cW5JbkZhN1gvVFU3NThHSWI4QTg5VmhBckMxa0RnbVdh?=
 =?utf-8?B?Vy9kZ1RkUEQybDBmMHFFaGprbHBoNUZVd0JTYXJJYVZ2MFVEalhOWCtLcmNm?=
 =?utf-8?B?d1g5clMxQjc4bUliZnlsSlhlS05Gd2FiWmlvUGFKWk9IbHhQdmtrVlhpVEVW?=
 =?utf-8?B?UUhhV3dJMGlMWDhJRTlSYTJXTWZ6bDdPaHlVcWxqcEVJQlNoMDU0OXlBZmN1?=
 =?utf-8?B?OUpIUjNVNml5RWgrTmZ2dStMeitPcHZuMlhHekNYSWpBemxZWGZydjZXV1Zm?=
 =?utf-8?B?QzExU0MrT2F6MGRHOHRBeVpiL2Z6NXBZaGlvN2tQZDFQTnRSUGdTZ3NMUEdm?=
 =?utf-8?B?TWlqVzJKejZUb3JJMURGNUdGbVZZbXFhSGhwN2cvR21RUmdnZG9jNm9PNitn?=
 =?utf-8?B?YWJ1WDQvanUwWEVHaVF4aE9YamNPSXZUc2JLdUlFM21NWS9SbDN3cFgxT055?=
 =?utf-8?B?c2QvTDVXVnZCTXBuR1VlMGJSWUdOUWRGTDN2M2c0cU0vMlk3dTkvd1RkVzI1?=
 =?utf-8?B?SndtNllJRklGTEhqQ3MraTFPVDkvWHBoMlJuVUV1ZmRxTC9QNkJsdmhURWpW?=
 =?utf-8?B?Mm1iVDRleHdOZTJCVENFVUZ5bFdoKzZQV3A5WTh5MTYzbzZ1Z2NTaEJ5VERW?=
 =?utf-8?B?dnZHNGFrUWRWdUNncWlkRy9TNDE4ZHYxOUhHUTl6ZHBWZk9iYWZOODBpUUx1?=
 =?utf-8?B?bDBPU2wyelhhRFUyYitTYm1QNkhkblVHdU1XR29uck9Tb3c2S0JwNXFXKzlq?=
 =?utf-8?B?YVppWk1YQVBTbElKTFhvZzlvNGxKY2Nvck1Kd1llak9keWE1U1Uzc2FIQ3Rz?=
 =?utf-8?B?cTQwSHVRVTBkNnRxVlR5dmJTT095czREb1VBTlpuTGJ1V20zdXRSUFp0ZmhL?=
 =?utf-8?B?cFN4L1RhRXFQaUhXTU5BZmRMZ1U1WUlMYmllU3ZpTXYyd1NiWlZScGFuSld6?=
 =?utf-8?B?VVVHd0pXNFBRSHlYL1dDdm1paGVXcUx1TkNYN3d2L1BudXNsSFNkdkE4UHJE?=
 =?utf-8?B?WnYzdi8xTlpRbDhaSVFzblp2OVJPK3BtRUx6MTY4WEowTTRPSnpra3NKNDJq?=
 =?utf-8?B?TUtHRTZlUnlNQUwrKzFEc2h0dmtNbkxLdDM2eGM0TjRBajVuWTMzUXFPK1B6?=
 =?utf-8?B?anZTMElQeTdWcVZSUVlrTG5ST0NlVkQvWk91KzhXMEZLZ2crczc2TWZpK20w?=
 =?utf-8?B?emFObjZDaDY1Qk85ZlV0a0ZoVUNPOEpWNXFsa2owUXY1UG5LUzJhRVNRWWhp?=
 =?utf-8?B?eEdDNXFaWDV6UTZQQXlqaTkxajJjc1lSMXNkWGxJY1ZtZmY4ZHhrQTFBNnVK?=
 =?utf-8?B?cTBNdWtRanVoNUlOV3JBbmgyeVd2b25MR1FnVFBSb3lmRi9tNXJydFVEcHVH?=
 =?utf-8?B?bFBmOFhmZUo0WW0zbU0yaXI3cGVjK2E1eG9Fa1MvekpVRFJEM2Z1d2Q5SXZU?=
 =?utf-8?B?U2xpSnY2NkExZXRwWkhENm45SitJWXJLdFVmVzdEbHE2VDQyVEpISEJ2MnJL?=
 =?utf-8?B?QWZWRHBVV2pJMzVlSkJrQXd6cEkrY0l1dE8xbmVlaGVRQitSMVF0a2NsUHAz?=
 =?utf-8?B?TG91UUdodGFBdWdxb0s3TlFEWFVGMGtoMTF2Zz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <900B64BC21E6E44693ADC0C7128D7816@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 008e39a6-bbe2-402f-656d-08dc3690baa2
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Feb 2024 06:04:01.3922
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: EV0vYhUcyPT9+ZnMN3lMfzApd1iFesOrmXdoWoh+QxFYTU73vA6QCkWUUGMgi/qiINu13oVuB6volb4foAM/Nw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5917

DQpPbiAyMDI0LzIvMjMgMDg6NDQsIFN0ZWZhbm8gU3RhYmVsbGluaSB3cm90ZToNCj4gT24gRnJp
LCAxMiBKYW4gMjAyNCwgSmlxaWFuIENoZW4gd3JvdGU6DQo+PiBPbiBQVkggZG9tMCwgdGhlIGdz
aXMgZG9uJ3QgZ2V0IHJlZ2lzdGVyZWQsIGJ1dA0KPj4gdGhlIGdzaSBvZiBhIHBhc3N0aHJvdWdo
IGRldmljZSBtdXN0IGJlIGNvbmZpZ3VyZWQgZm9yIGl0IHRvDQo+PiBiZSBhYmxlIHRvIGJlIG1h
cHBlZCBpbnRvIGEgaHZtIGRvbVUuDQo+PiBPbiBMaW51eCBrZXJuZWwgc2lkZSwgaXQgY2FsbGVz
IFBIWVNERVZPUF9zZXR1cF9nc2kgZm9yDQo+PiBwYXNzdGhyb3VnaCBkZXZpY2VzIHRvIHJlZ2lz
dGVyIGdzaSB3aGVuIGRvbTAgaXMgUFZILg0KPj4gU28sIGFkZCBQSFlTREVWT1Bfc2V0dXBfZ3Np
IGZvciBhYm92ZSBwdXJwb3NlLg0KPj4NCj4+IENvLWRldmVsb3BlZC1ieTogSHVhbmcgUnVpIDxy
YXkuaHVhbmdAYW1kLmNvbT4NCj4+IFNpZ25lZC1vZmYtYnk6IEppcWlhbiBDaGVuIDxKaXFpYW4u
Q2hlbkBhbWQuY29tPg0KPj4gLS0tDQo+PiAgeGVuL2FyY2gveDg2L2h2bS9oeXBlcmNhbGwuYyB8
IDYgKysrKysrDQo+PiAgMSBmaWxlIGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygrKQ0KPj4NCj4+IGRp
ZmYgLS1naXQgYS94ZW4vYXJjaC94ODYvaHZtL2h5cGVyY2FsbC5jIGIveGVuL2FyY2gveDg2L2h2
bS9oeXBlcmNhbGwuYw0KPj4gaW5kZXggNDkzOTk4YjQyZWM1Li40NmY1MWVlNDU5ZjYgMTAwNjQ0
DQo+PiAtLS0gYS94ZW4vYXJjaC94ODYvaHZtL2h5cGVyY2FsbC5jDQo+PiArKysgYi94ZW4vYXJj
aC94ODYvaHZtL2h5cGVyY2FsbC5jDQo+PiBAQCAtNzYsNiArNzYsMTIgQEAgbG9uZyBodm1fcGh5
c2Rldl9vcChpbnQgY21kLCBYRU5fR1VFU1RfSEFORExFX1BBUkFNKHZvaWQpIGFyZykNCj4+ICAg
ICAgY2FzZSBQSFlTREVWT1BfdW5tYXBfcGlycToNCj4+ICAgICAgICAgIGJyZWFrOw0KPj4gIA0K
Pj4gKyAgICBjYXNlIFBIWVNERVZPUF9zZXR1cF9nc2k6DQo+PiArICAgICAgICBpZiAoICFpc19o
YXJkd2FyZV9kb21haW4oY3VycmQpICkNCj4+ICsgICAgICAgICAgICByZXR1cm4gLUVPUE5PVFNV
UFA7DQo+PiArICAgICAgICBBU1NFUlQoIWhhc19waXJxKGN1cnJkKSk7DQo+IA0KPiBEbyB3ZSBy
ZWFsbHkgbmVlZCB0aGlzIGFzc2VydD8gSSB1bmRlcnN0YW5kIHRoYXQgdGhlIHVzZSBjYXNlIHJp
Z2h0IG5vdw0KPiBpcyBmb3IgIWhhc19waXJxKGN1cnJkKSBidXQgaW4gZ2VuZXJhbCBpdCBkb2Vz
bid0IHNlZW0gdG8gbWUgdGhhdA0KPiBQSFlTREVWT1Bfc2V0dXBfZ3NpIGFuZCAhaGFzX3BpcnEg
c2hvdWxkIGJlIHRpZWQgdG9nZXRoZXIuDQpNYWtlIHNlbnNlLCB0aGFua3MgZm9yIGV4cGxhbmF0
aW9uLCBJIHdpbGwgZGVsZXRlIHRoaXMgaW4gbmV4dCB2ZXJzaW9uLg0KDQo+IA0KPiBBc2lkZSBm
cm9tIHRoYXQsIGl0IGxvb2tzIGZpbmUuDQo+IA0KPiANCj4+ICsgICAgICAgIGJyZWFrOw0KPj4g
Kw0KPj4gICAgICBjYXNlIFBIWVNERVZPUF9lb2k6DQo+PiAgICAgIGNhc2UgUEhZU0RFVk9QX2ly
cV9zdGF0dXNfcXVlcnk6DQo+PiAgICAgIGNhc2UgUEhZU0RFVk9QX2dldF9mcmVlX3BpcnE6DQo+
PiAtLSANCj4+IDIuMzQuMQ0KPj4NCg0KLS0gDQpCZXN0IHJlZ2FyZHMsDQpKaXFpYW4gQ2hlbi4N
Cg==


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 07:26:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 07:26:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685264.1065703 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reVNp-0005jz-FZ; Mon, 26 Feb 2024 07:26:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685264.1065703; Mon, 26 Feb 2024 07:26:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reVNp-0005js-A0; Mon, 26 Feb 2024 07:26:41 +0000
Received: by outflank-mailman (input) for mailman id 685264;
 Mon, 26 Feb 2024 07:26:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reVNo-0005jR-85
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 07:26:40 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 607b8556-d478-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 08:26:37 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a4348aaa705so71435766b.0
 for <xen-devel@lists.xenproject.org>; Sun, 25 Feb 2024 23:26:37 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 mj9-20020a170906af8900b00a3e94142018sm2130011ejb.132.2024.02.25.23.26.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 25 Feb 2024 23:26:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 607b8556-d478-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708932397; x=1709537197; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cbE0HxKHF5m/mQDiNYWry/QZ2qCtnZ+KpgaeK35W+aE=;
        b=M17zKsfLl6kZiTyvr8yIEA//Q900QLZzGmO/nXzUf6uxA67sFEUghKrPFcWRoKZWOW
         hFheIlpZ0WerOMf52vTgW5QFHEOr1pQxEJlGNY5okvJpH5rWlBHRstG++zZZv0wj2Z5z
         P2WNCsyYfH221Ic4CoD/piDNs+V3OHjMgIAqB3NMIefn0fQwtMxsmgmEjedcPAhELhIA
         YXYWr38BqO6oc8gM+fmOiKNrJmUK6gUtN0/EiWYIDh/WSAXn40VE7K/SFvJ6ZqcyMud2
         vB1LUNorwUh7pjVUhp+rigUU3yAiYyGPNqkGg7FCW7bVVtujV78ZfBrcgxuIvIXGyTZ4
         zR9A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708932397; x=1709537197;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cbE0HxKHF5m/mQDiNYWry/QZ2qCtnZ+KpgaeK35W+aE=;
        b=ixl3a40F+wYFgpqEs47Vvo1W+1KH3GFjbqRmEuLvogNTHu/kiwHepCraY+NkiTiaoI
         XYlL2i0pRjz1iDShiFOPscVwSke8XchI9Y2G+l3KnG/KUz5+P2WMurnHjJhsiq5Ktnfr
         wfrT3EN1A6SXwbxpAizN5dsBdlED0YGdyTz348mLqi2Sqjnd2oPWk56vt1DT1lNW3ZX4
         oxZrnZ5UlFhOSBs1lNB8ItLBus8sSWA+HMKeMbJjOHbB3IATpSyFNrgkydF3xdc4DhuA
         NclI6o7iHObWElByC4oiATiQS8XPzAqDYNsQA8vRyyckYVCTVsnsasy/6u6g3zX/qfBr
         5OQw==
X-Forwarded-Encrypted: i=1; AJvYcCVpo6XaA6iObI4L3dJWeRc0nojOfV5kNp3cl4nLQWK1ZG20LBwg87dKZHMNTfauUTHMzXeJTSJ6BvBOqK4uT8nl7Gh9+wjt5Y+Tr1r0p6E=
X-Gm-Message-State: AOJu0YxcsAaXqu9fGRhi5Rlbq6pX51X3KpYmb+6f0iaRuNR8W+0mqKAM
	QrKlTImk3iXDr2EaKjYSj/bBr9J5UxphsDK68F8IndWI0ZD/T/59QSQAWsrEZA==
X-Google-Smtp-Source: AGHT+IF/pZBU9ErNsuF/xnt1bFkVHK+wYsQ4q1ooM5syOa0t1ozLF+y8SNN8p5X2G83i28e1IoecIA==
X-Received: by 2002:a17:906:3659:b0:a3e:da6:85f8 with SMTP id r25-20020a170906365900b00a3e0da685f8mr3976031ejb.30.1708932397082;
        Sun, 25 Feb 2024 23:26:37 -0800 (PST)
Message-ID: <65c33a27-3c8d-4943-8ce7-4fe50ebb45d0@suse.com>
Date: Mon, 26 Feb 2024 08:26:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 25/30] xen/riscv: add minimal stuff to processor.h to
 build full Xen
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <9efbc232f64b6192cf83f865b8987846fe082720.1707146506.git.oleksii.kurochko@gmail.com>
 <6be5102a-624c-463a-9821-c618d110ce7a@suse.com>
 <095b8031eaaa5324cdae9fee75f9521a795feb46.camel@gmail.com>
 <2f1e4d2d-5b33-47ff-912b-c4693744d0e9@suse.com>
 <0bafef389b30251bc9898bb61604aa3efaabe48c.camel@gmail.com>
 <5de8f721-461c-4a0e-a11d-63aa7c93a742@suse.com>
 <6cd4d415ccf2163d9fdf690f29d898d61b077472.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <6cd4d415ccf2163d9fdf690f29d898d61b077472.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.02.2024 18:00, Oleksii wrote:
> On Mon, 2024-02-19 at 09:06 +0100, Jan Beulich wrote:
>> On 16.02.2024 12:16, Oleksii wrote:
>>> On Thu, 2024-02-15 at 17:43 +0100, Jan Beulich wrote:
>>>> On 15.02.2024 17:38, Oleksii wrote:
>>>>> On Tue, 2024-02-13 at 14:33 +0100, Jan Beulich wrote:
>>>>>> On 05.02.2024 16:32, Oleksii Kurochko wrote:
>>>>>>> +	depends on LLD_VERSION >= 150000 || LD_VERSION >=
>>>>>>> 23600
>>>>>>
>>>>>> What's the linker dependency here? Depending on the answer I
>>>>>> might
>>>>>> further
>>>>>> ask why "TOOLCHAIN" when elsewhere we use CC_HAS_ or HAS_CC_
>>>>>> or
>>>>>> HAS_AS_.
>>>>> I missed to introduce {L}LLD_VERSION config. It should output
>>>>> from
>>>>> the
>>>>> command:
>>>>>   riscv64-linux-gnu-ld --version
>>>>
>>>> Doesn't answer my question though where the linker version
>>>> matters
>>>> here.
>>> Then I misinterpreted your initial question.
>>> Could you please provide further clarification or rephrase it for
>>> better understanding?
>>>
>>> Probably, your question was about why linker dependency is needed
>>> here,
>>> then
>>> it is not sufficient to check if a toolchain supports a particular 
>>> extension without checking if the linker supports that extension   
>>> too.
>>> For example, Clang 15 supports Zihintpause but GNU bintutils
>>> 2.35.2 does not, leading build errors like so:
>>>     
>>>    riscv64-linux-gnu-ld: -march=rv64i_zihintpause2p0: Invalid or
>>>    unknown z ISA extension: 'zihintpause'
>>
>> Hmm, that's certainly "interesting" behavior of the RISC-V linker.
>> Yet
>> isn't the linker capability expected to be tied to that of gas? I
>> would
>> find it far more natural if a gas dependency existed here. If such a
>> connection cannot be taken for granted, I'm pretty sure you'd need to
>> probe both then anyway.
> 
> Wouldn't it be enough in this case instead of introducing of new
> configs and etc, just to do the following:
>    +ifeq ($(CONFIG_RISCV_64),y)
>    +has_zihintpause = $(call as-insn,$(CC) -mabi=lp64 -
>    march=rv64i_zihintpause, "pause",_zihintpause,)
>    +else
>    +has_zihintpause = $(call as-insn,$(CC) -mabi=ilp32 -
>    march=rv32i_zihintpause, "pause",_zihintpause,)
>    +endif
>    +
>     riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
>     riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
>    -riscv-march-$(CONFIG_TOOLCHAIN_HAS_ZIHINTPAUSE) := $(riscv-march-
>    y)_zihintpause
>     
>     # Note that -mcmodel=medany is used so that Xen can be mapped
>     # into the upper half _or_ the lower half of the address space.
>     # -mcmodel=medlow would force Xen into the lower half.
>     
>    -CFLAGS += -march=$(riscv-march-y) -mstrict-align -mcmodel=medany
>    +CFLAGS += -march=$(riscv-march-y)$(has_zihintpause) -mstrict-align
>    -
>    mcmodel=medany

Yes, this is kind of what I'd expect.

> Probably, it would be better:
>    ...
>    +CFLAGS += -march=$(riscv-march-y)$(call or,$(has_zihintpause)) -
>    mstrict-align -
>    mcmodel=medany

Why the use of "or"? IOW right now I don't see what's "better" here.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 07:27:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 07:27:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685267.1065711 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reVOX-0006HN-Nx; Mon, 26 Feb 2024 07:27:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685267.1065711; Mon, 26 Feb 2024 07:27:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reVOX-0006HG-LL; Mon, 26 Feb 2024 07:27:25 +0000
Received: by outflank-mailman (input) for mailman id 685267;
 Mon, 26 Feb 2024 07:27:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=dckQ=KD=amd.com=Jiqian.Chen@srs-se1.protection.inumbo.net>)
 id 1reVOV-0006FH-Po
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 07:27:23 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7ab1d8e2-d478-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 08:27:21 +0100 (CET)
Received: from BL1PR12MB5849.namprd12.prod.outlook.com (2603:10b6:208:384::18)
 by PH7PR12MB7355.namprd12.prod.outlook.com (2603:10b6:510:20e::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.34; Mon, 26 Feb
 2024 07:27:17 +0000
Received: from BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa]) by BL1PR12MB5849.namprd12.prod.outlook.com
 ([fe80::debf:178c:f5df:5efa%5]) with mapi id 15.20.7316.032; Mon, 26 Feb 2024
 07:27:17 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ab1d8e2-d478-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=LRCl4H2JMgM3UwNg41p55Hqnss5614X0HizPQQT8b5bb1eYsOfG2AbeZLpOS9zmF/Q2oYYsX+jY6gmNrNhaKQBNSg1CItLm0swSNGqqyp1X2Ai9KEUHKMX1NmbSQQL0F2ZFjG4EgLbuIsM2DDsl+cEyRRAhlzqSgoQ7MWud47SYZsG/fkc9f8hf47lyQIFFVmi0Llo0bWI1JxyK86QufbqYt28B5ziL+UXw+ZaAlkRBNZHOUrQv1fGNQcnMEx/cb/zTdV82x7GrQ7ofFDyonwRAcohr+C4hIaUJrxFy03Px6B97A34+vBz1Cdp9lhs9Cn88bqM0Z/3e+R7fG9xLbsQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=xu0pfrPxmlGqZrRP6Gxt1NJJQ0n8AY6GEFqapd9lgrI=;
 b=SO0pTEle0s+8H9AbeeM/cvjdT+AC+3bS4iUCfoIZ7CsW4hX3MBFMoselNrvUxJBsblVSvvc7YJGJYJXn0Jx2l6MUKeui2IXg1Tmw+dZ1D064laKoYkG3luuaD90ReGMBKRTSWCJ6tE2izbOf4UuIyTwBRre4Fg5WltElvnz3LK94H3H81wy++EdWa8MSSfinH+RMbUJ2pDmBduMcBdenEfrA3aIUx6hMEvM/Bdywovv/zQm7uOAjTmIw21J56iZOij+8nLpgcr1nkvd/fa80RIUDtovO9EtuaRjSzq/Z0aHQ9wCRQCUr8PYdvZcwikAYnAVfMR91JHgQtLs6qfQJ6g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=xu0pfrPxmlGqZrRP6Gxt1NJJQ0n8AY6GEFqapd9lgrI=;
 b=z17km6Ig3Pjg1NfeKXHvOqgW65edd2+zTg2NBHYNaGO3zcNrBg+3z8v/BPvdyZaEuTh9thr810Om3nrz4yuXqKr3c95Uhf3JA503H+MN8/+iSPVCXe8zkwN187VK+P8hcdaKMLwgLRVdnmUa5NNaxX10ix0AOMnoVJwqrAvpfPQ=
From: "Chen, Jiqian" <Jiqian.Chen@amd.com>
To: Anthony PERARD <anthony.perard@cloud.com>
CC: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, Jan
 Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, George Dunlap <george.dunlap@citrix.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, Juergen Gross
	<jgross@suse.com>, "Daniel P . Smith" <dpsmith@apertussolutions.com>,
	"Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>, "Huang, Ray"
	<Ray.Huang@amd.com>, "Chen, Jiqian" <Jiqian.Chen@amd.com>
Subject: Re: [RFC XEN PATCH v5 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Topic: [RFC XEN PATCH v5 5/5] domctl: Add XEN_DOMCTL_gsi_permission to
 grant gsi
Thread-Index: AQHaRR65Xby346gk7ESzHNZtxrKssbEVI+YAgAGO+4CAAaWUAIAEltsA
Date: Mon, 26 Feb 2024 07:27:17 +0000
Message-ID:
 <BL1PR12MB584962AE7E0EBA1256C2047DE75A2@BL1PR12MB5849.namprd12.prod.outlook.com>
References: <20240112061317.418658-1-Jiqian.Chen@amd.com>
 <20240112061317.418658-6-Jiqian.Chen@amd.com>
 <09290ba4-6915-4f0d-8e16-3e14713d02ba@perard>
 <BL1PR12MB5849F3787ED9245E6986872CE7562@BL1PR12MB5849.namprd12.prod.outlook.com>
 <d0adfc76-5142-46b1-acaa-3a4f0331cfb0@perard>
In-Reply-To: <d0adfc76-5142-46b1-acaa-3a4f0331cfb0@perard>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-imapappendstamp: MN0PR12MB6150.namprd12.prod.outlook.com
 (15.20.7292.000)
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: BL1PR12MB5849:EE_|PH7PR12MB7355:EE_
x-ms-office365-filtering-correlation-id: 1a69e0f6-565c-4a4f-40ff-08dc369c5cb9
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 vdJz6DcJ1X2LlXVS+7Grmo7ORf0f7O07kf9JR6XGf9AswqNQ7P/eWHdPn/18EBj3Ps4PRnm0c26FJWepw9Mq6MN8aLBrvjA7WD4H5JvUlvbwXMw72O9z4ks81sZpe32hDLX469Gy5LdRZQtUDMTRc+ow0ins3dk9qYVqPjoh0PI4yLhfbDIFNyzGARwMMVtfdjEP2t5MM8L2WETWgdZB9MAyiRH7oFChKUV8hlwyzWBrhDivXH1P8awvJp1dx559QXTeKyMWN0RjLojVmkrQmY/jnJmYci2NmVVqtQWJ2UZMUfvu6AWVnNSguonPQ3drkmgz4p31pDb8d15Bsea1iH5ZMmKYdsBHCONPZA8ZVGwGmRrK68zcJYslWbeiASNlm/0BPkRsvcOEE7DcOILgjLeecfkPCaTQmeKhku77oJd9WGpC+rvAtILUB1dCIbELawIklmnxUJAdnx+3IaCRca28DTyJzowERfRA04A5W27iQmYZ4Zp3J5hS587lXMyc0XHQ92WHg7Zb/FOWNh3i2v6eqWQv3jRLT20S8ldcrFw2cXqXex8Q8/sYo49sh0dOFDvop+ZUbR5Pzqoz4wWTHZ42ftgste9kerhz7HjGi6Ht3fjm0cvDRjfLxKMXeLS1HvjOeM5XY78ggRNwEQ6MPr9A2qX4RNAYq5fuKP5Y7egGNqXR3NMR2dHMT61g07fL4E++PvmfErtMT9oPkAebpnkzrRJJWvnWH00ZbtGxPY8=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?UHBrUWN0bFB3NExJM1QzdUo0N0NuYnF4NnBUTzd5d2MyQTJBL0JrNi8yUnBU?=
 =?utf-8?B?ZzQwSEhqdmNGQWV0MktweXZtcWtwRXBHUCtvNVFRUFUrVklMYkduRW9teUVB?=
 =?utf-8?B?UmNaeWxPdENqV05oNk9DREZGSmZuSDdiZGhQdUlad21xaUhOU2VCUmRoYzhW?=
 =?utf-8?B?aGF2UE5Ibkg3Q25wVERISTVVUFR1TXFOS2JBUXlrTkJrTEVSRjZsYkhQSTMr?=
 =?utf-8?B?QlMrdlJtZjRnMHpnWFJuZ053K2IrRTF3aExaRk91OFRkNzNIcStPeUpsaXhy?=
 =?utf-8?B?dHhPSHZGZ2NXYi91NEVhVUpEdXNwMGdTdVZsTWVscm1XUzA0UUdGRjNyZkNK?=
 =?utf-8?B?NWlHQWt1SDh4cDlLeC9pZDJVR0ZZZ2wrY3RxdlZaTXRTb2RTc1VBeXRnNG4v?=
 =?utf-8?B?b0xsRE4zQTFkbzVHbktNbFN0WjNZMVZtVmlydjV1Wi91bm5HZWlOK0UvU1pC?=
 =?utf-8?B?S2FRbmhXSTY5WjBSUHRjSUFHS2pNc1p4SHRDRXc0OTZRWkRvSjhZa3V2SU9M?=
 =?utf-8?B?TGhNVnRkcU9QdlJOTkVhaUVHYWQ2M1FJTFJESEtNQzFpbGxnUDFPMHc4eGdS?=
 =?utf-8?B?ZUNxMy9lVlRmNGo5bWozUzZIU2hTak1NdXJzTldnTS93QVJFeGZ5SXZySXVH?=
 =?utf-8?B?cjJMbnhpZldVa2djU3l2T0RxOWlvYTBxb0tOdnRINlZSMmVOL2NBU1dGaS9t?=
 =?utf-8?B?UnFBSXFhR3FTS1p5WGV1eTFIbzM1K0NrSFZENVpvenI4UXVkdjljOGN3Yk1p?=
 =?utf-8?B?ZG9WMFlvbzg1OGJDWkV3MlJRV09pNlRML0NKc1hrcmFoZ3VvR1NIN1Z1OFpO?=
 =?utf-8?B?d2VUSjE3L2RFbXk4aVI2OXZrWGFtREd0ekl5NGpaWlJ3bWVzdk5OWlZhNWRq?=
 =?utf-8?B?dVFwKzB6RTlmNHZlMHZqTkxtdGtRbjF1NG9Bcmt4TTIveGVEOFcvMFF3NkRQ?=
 =?utf-8?B?KzlmQlZVYlM4SzFjalJpRjFvYkg2MjNLWGlhd2IwWk5KNnVGMGNlRm0renNY?=
 =?utf-8?B?STRNbVJhWXZPdXNoUStCMHFRamtTSFQzVmxvUDlRSWNvWGZ3ckRTSmpucGVU?=
 =?utf-8?B?ZFdkVGJWdFdOYzVxeWNwMzFJd0owZDhIZkRVbXFFUjF0OVFzNHBJdjlIM2xv?=
 =?utf-8?B?enc5NmxOTTBGRk9ZVlhxMjVlWFF5eFZoWU50ZE1FcUtra2xEOG00UWxHU3Rr?=
 =?utf-8?B?SFlUekJtcFVsb0FPYXcyWVdERnllYWMzc2lKTGx0cmZEVHJLRE1uOU5kUnRF?=
 =?utf-8?B?bFJqRWphTHJiaFJtSkQ2TEFMUTd1R2JKb25YejZCUEFMTnpRNThQRUU1b1hS?=
 =?utf-8?B?SjlqUTdzMWtvUUZlSlJGeVMzZTBiV3R0bWsvZ1NOTWlleElZU0x6Tjd6Sk9k?=
 =?utf-8?B?Ylk5blZSNW54SG9vKzlCOXJRaE5zYkxoN2tscmhuVjhqYXpnN0JuUG1WZGFs?=
 =?utf-8?B?TXJBYTJFQ2JwZG5yTit3UkQ1L0NlcllzZGV6aENPYk9hZkVwd0RoVFNQVnMw?=
 =?utf-8?B?Wk5PWWRWUEVRRU9MRFRkUnR5UEMzbGdIT0RaZFFjWGNRdXlFcngzSnJFano2?=
 =?utf-8?B?bGpLVy90TW8zaWtwZGxMVngrT0NZdFhXRHB3VDFvaHRzUloyYi9mRnVJeUo1?=
 =?utf-8?B?c0ZxRENFczAzWFMwTnM3Sm5LOGVuVlpYTjJvb0JVdEZjUE1QWndFWXh0d1Vm?=
 =?utf-8?B?cnBzZXpGbmkxMWU2VFhXY0xBMWJ2N2JhUlI3ZSthQU1DcjRTdlpzYVVpdSti?=
 =?utf-8?B?NkRWd0dpQTZZaWVkSURsMnIwcTFwb1Z5VlRuU0huM0xzUmEzbytXdGVXOURF?=
 =?utf-8?B?Y0IxMGpnNVdSekVsaTNIMzdwUmhGWHJmME1zTEI5d3pIV3c4UjgwOU0rTFhJ?=
 =?utf-8?B?SkIyVmM5OHdpMkx6OUVhSlBOQXVtYWlMc1kycElLdERkR1FFb0t4bzByWTlD?=
 =?utf-8?B?blBEMFFOWmM5OC9pa3BxRXBuOHFxZ1owckNzL3ZSd3k4QThFWmhmYnlNUnBv?=
 =?utf-8?B?YlVQakh3QWVnS1cwcThKQWdub3Z4SkIyRlM4dy9NOWNEbU9wbVhEeDJYdVgw?=
 =?utf-8?B?NHdCQjYvOUdFRFNwczNYWk9OV2pvQ1lNZ3Rsdz09?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <94326F390B9226478EF2516E73C81428@amdcloud.onmicrosoft.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a69e0f6-565c-4a4f-40ff-08dc369c5cb9
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Feb 2024 07:27:17.7963
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: +YXQAgn8cnEizGu74fr7lsJopjLJSwwVAbHqyCZ2n985WsAP+N63WxGpPVqjcfGXGF+yB3fXiRobngbRPmgxNg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7355

T24gMjAyNC8yLzIzIDIzOjU5LCBBbnRob255IFBFUkFSRCB3cm90ZToNCj4gT24gVGh1LCBGZWIg
MjIsIDIwMjQgYXQgMDc6MjI6NDVBTSArMDAwMCwgQ2hlbiwgSmlxaWFuIHdyb3RlOg0KPj4gT24g
MjAyNC8yLzIxIDIzOjAzLCBBbnRob255IFBFUkFSRCB3cm90ZToNCj4+PiBPbiBGcmksIEphbiAx
MiwgMjAyNCBhdCAwMjoxMzoxN1BNICswODAwLCBKaXFpYW4gQ2hlbiB3cm90ZToNCj4+Pj4gZGlm
ZiAtLWdpdCBhL3Rvb2xzL2xpYnMvbGlnaHQvbGlieGxfcGNpLmMgYi90b29scy9saWJzL2xpZ2h0
L2xpYnhsX3BjaS5jDQo+Pj4+IGluZGV4IGExYzZlODI2MzFlOS4uNDEzNmE4NjBhMDQ4IDEwMDY0
NA0KPj4+PiAtLS0gYS90b29scy9saWJzL2xpZ2h0L2xpYnhsX3BjaS5jDQo+Pj4+ICsrKyBiL3Rv
b2xzL2xpYnMvbGlnaHQvbGlieGxfcGNpLmMNCj4+Pj4gQEAgLTE0MjEsNiArMTQyMSw4IEBAIHN0
YXRpYyB2b2lkIHBjaV9hZGRfZG1fZG9uZShsaWJ4bF9fZWdjICplZ2MsDQo+Pj4+ICAgICAgdWlu
dDMyX3QgZmxhZyA9IFhFTl9ET01DVExfREVWX1JETV9SRUxBWEVEOw0KPj4+PiAgICAgIHVpbnQz
Ml90IGRvbWFpbmlkID0gZG9taWQ7DQo+Pj4+ICAgICAgYm9vbCBpc3N0dWJkb20gPSBsaWJ4bF9p
c19zdHViZG9tKGN0eCwgZG9taWQsICZkb21haW5pZCk7DQo+Pj4+ICsgICAgaW50IGdzaTsNCj4+
Pj4gKyAgICBib29sIGhhc19nc2kgPSB0cnVlOw0KPj4+DQo+Pj4gV2h5IGlzIHRoaXMgZnVuY3Rp
b24gaGFzICJnc2kiIHZhcmlhYmxlLCBhbmQgImdzaSA9IGlycSIgYnV0IHRoZSBuZXh0DQo+Pj4g
ZnVuY3Rpb24gcGNpX3JlbW92ZV9kZXRhY2hlZCgpIGRvZXMgb25seSBoYXZlICJpcnEiIHZhcmlh
YmxlPw0KPj4gQmVjYXVzZSBpbiBwY2lfYWRkX2RtX2RvbmUsIGl0IGNhbGxzICIgciA9IHhjX3Bo
eXNkZXZfbWFwX3BpcnEoY3R4LT54Y2gsIGRvbWlkLCBpcnEsICZpcnEpOyAiLCBpdCBwYXNzZXMg
dGhlIHBvaW50ZXIgb2YgaXJxIGluLCBhbmQgdGhlbiBpcnEgd2lsbCBiZSBjaGFuZ2VkLCBzbyBJ
IG5lZWQgdG8gdXNlIGdzaSB0byBzYXZlIHRoZSBvcmlnaW4gdmFsdWUuDQo+IA0KPiBJJ20gc29y
cnkgYnV0IHVuY29uZGl0aW9uYWwgImdzaSA9IGlycSIgbG9va3MgdmVyeSB3cm9uZywgdGhhdCBs
aW5lDQpNYWtlIHNlbnNlLCBJIHNob3VsZCBhZGQgYSBjb25kaXRpb24gYW5kIHNvbWUgY29tbWVu
dHMgaGVyZS4gTWF5YmU6DQoJLyogaWYgaGFzIGdzaSBzeXNmcyBub2RlLCB0aGVuIGZzY2FuZigp
IHJlYWQgZnJvbSB0aGUgZ3NpIHN5c2ZzIGFuZCBzdG9yZSBpdCBpbiBpcnEgdmFyaWFibGUsIHdl
IG5lZWQgdG8gc2F2ZSB0aGUgb3JpZ2luYWwgZ3NpIHZhbHVlLCBiZWNhdXNlIGlycSB3aWxsIGJl
IGNoYW5nZWQgaW4geGNfcGh5c2Rldl9tYXBfcGlycSAqLw0KCWlmIChoYXNfZ3NpKQ0KCQlnc2kg
PSBpcnE7DQoNCj4gbmVlZHMgdG8gYmUgcmVtb3ZlZCBvciBjaGFuZ2VkLCBvdGhlcndpc2Ugd2Ug
aGF2ZSB0d28gZnVuY3Rpb25zIGRvaW5nIHRoZQ0KPiBzYW1lIHRoaW5nIGp1c3QgYWZ0ZXIgdGhh
dCAoeGNfZG9tYWluX2lycV9wZXJtaXNzaW9uIGFuZA0KPiB4Y19kb21haW5fZ3NpX3Blcm1pc3Np
b24pLiBJbnN0ZWFkLCBteSBndWVzcyBpcyB0aGF0IHRoZQ0KPiBhcmd1bWVudHMgb2YgeGNfcGh5
c2Rldl9tYXBfcGlycSgpIG5lZWRzIHRvIGJlIGNoYW5nZXMuIFdoYXQgZG9lcw0KPiBjb250YWlu
IGBpcnFgIGFmdGVyIHRoZSBtYXBfcGlycSgpIGNhbGw/IEEgInBpcnEiIG9mIHNvbWUga2luZD8g
TWF5YmUNClllcywgcGlycS4NCg0KPiB4Y19waHlzZGV2X21hcF9waXJxKGN0eC0+eGNoLCBkb21p
ZCwgaXJxLCAmcGlycSkgd291bGQgbWFrZSB0aGluZ3MgbW9yZQ0KPiBjbGVhciBhYm91dCB3aGF0
J3MgZ29pbmcgb24uDQo+IA0KPiANCj4gQW5kIEJUVywgbWF5YmUgcmVuYW1pbmcgdGhlIHZhcmlh
YmxlICJoYXNfZ3NpIiB0byAiaXNfZ3NpIiBtaWdodCBtYWtlDQo+IHRoaW5ncyBhIGJpdCBjbGVh
cmVyIGFzIHdlbGwsIGFzIGluOiAidGhlIHZhcmlhYmxlICdpcnEnICRpc19nc2kiLA0KPiBpbnN0
ZWFkIG9mIGEgdmFyaWFibGUgdGhhdCBoYXZlIGEgbWVhbmluZyBvZiAidGhlIHN5c3RlbSAkaGFz
X2dzaSINCj4gd2l0aG91dCBuZWNlc3NhcmlseSBtZWFuaW5nIHRoYXQgdGhlIGNvZGUgaXMgdXNp
bmcgaXQuDQpHc2kgaXMgYSBuZXcgc3lzZnMgbm9kZSBhZGRlZCBieSBteSBrZXJuZWwgcGF0Y2gs
IGFuZCBpdCBpcyBzdGlsbCBpbiBkaXNjdXNzaW9uIHdpdGggUENJIE1haW50YWluZXIuDQpBbmQg
Zm9yIGNvbXBhdGlibGUgd2l0aCBvbGQgdmVyc2lvbiBvZiBrZXJuZWwgdGhhdCBkaWRuJ3QgaGFz
IGdzaSBzeXNmcyBub2RlLCB3ZSBzdGlsbCBuZWVkIHRvIHVzZSBpcnEgaGVyZS4NClNvLCBJIG5h
bWVkIHRoaXMgdmFyaWFibGUgdG8gaGFzX2dzaS4gSXMgaXQgY2xlYXJlciB0aGF0IGNoYW5naW5n
IGl0IHRvIGhhc19nc2lfc3lzZnNfbm9kZT8NCk1heWJlIEkgbmVlZCB0byBhZGQgc29tZSBjb21t
ZW50cyBpbiBjb2RlIHRvIG1ha2UgdGhlIHVzYWdlIG9mIGdzaSBjbGVhci4NCg0KPiANCj4gTWF5
YmUgdXNpbmcgKGFidXNpbmc/KSB0aGUgdmFyaWFibGUgbmFtZSAiaXJxIiBtaWdodCBiZSBhIGJp
dCB3cm9uZyBub3c/DQo+IFdoYXQgZG9lcyAiR1NJIiBzdGFuZCBmb3IgYW55d2F5PyBBbmQgYWJv
dXQgIlBJUlEiPyBUaGlzIGlzIGp1c3Qgc29tZQ0KR1NJIGlzIHg4NiBzcGVjaWZpYyBjb25jZXB0
LCBpdCBpcyByZWxhdGVkIHRvIElPQVBJQy1QSU4uIFBJUlEgaXMgdXNlZCB0byByb3V0ZSBpbnRl
cnJ1cHRzIGluIFhlbi4NCg0KPiBxdWVzdGlvbiB0byBmaWd1cmUgb3V0IGlmIHRoZXJlJ3MgcG90
ZW50aWFsbHkgYSBiZXR0ZXIgbmFtZSBmb3IgdGhlDQo+IHZhcmlhYmxlcyBuYW1lcy4NCj4gDQo+
IFRoYW5rcywNCj4gDQoNCi0tIA0KQmVzdCByZWdhcmRzLA0KSmlxaWFuIENoZW4uDQo=


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 07:33:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 07:33:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685269.1065721 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reVUV-0007xI-D7; Mon, 26 Feb 2024 07:33:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685269.1065721; Mon, 26 Feb 2024 07:33:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reVUV-0007xB-AV; Mon, 26 Feb 2024 07:33:35 +0000
Received: by outflank-mailman (input) for mailman id 685269;
 Mon, 26 Feb 2024 07:33:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reVUU-0007x5-1H
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 07:33:34 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 57d87cfe-d479-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 08:33:32 +0100 (CET)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-5640fef9fa6so3464072a12.0
 for <xen-devel@lists.xenproject.org>; Sun, 25 Feb 2024 23:33:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n18-20020a05640204d200b005649df0654asm2073502edw.21.2024.02.25.23.33.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 25 Feb 2024 23:33:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57d87cfe-d479-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708932812; x=1709537612; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=7IGT1ZD3A3SuyIsZBL9Lhns1VIFac8w5ou8WQZk+CGY=;
        b=M+fOpwjnP/jJNueTgs86ykkEeb/5fLwCHYyzIQ3fq6Y+z56VQcxU6O44/dJFvao6N1
         szYkTyV+PbBlTu+gE1VrTzPe0M10fD0auBpIjGbZ/ZBjoVOLR3FybUWLn915JRUcXLkp
         mAJAzoZrZvSP2RJssNHjMRsSarZcmHrqQ4S+f8hUN/tkUc0OiKFHtiRm59dnvTjwj3yR
         ARFlXGwT97Or3+zzUweIVnLEI8zlZ7cSnPWHCN36faac3itx7Ud40mMzr1rj9HeplULC
         cZN90KGUZFf5mqeCe4mdp1UjJ6OjcBFLkfk//4EHpVdjeSnU+sEaH0rdKE5S0CNKFsZK
         LaGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708932812; x=1709537612;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=7IGT1ZD3A3SuyIsZBL9Lhns1VIFac8w5ou8WQZk+CGY=;
        b=R3UftiIry8EtP7+1u6Fh/FBgWsiq1gTh8CBJiPImzBO17eMZuase9/BQqO0bj84BM6
         QY7lhoG0AKWwnSgUDWyF+xyRiwGIexHfDpuwGFzY95fZj1INYXQ7gKfXKoO6d07gHHR3
         Wchrudz0VuJ3wipEgb0ex3rcF3718P7z1Z4BHB+VG+1cicDPgloOINLPbSdRYNSFWST5
         hsbJSj4kJ8CS5x/KSsq3WkrcnRma7jVmoAIXbsDzhxBPzd/uR3n7UPs4q3lkeOXEQmZX
         G1I3q1l9C8NvQSLflGeJgo0ivGElWl092ZqSnQxTp8Ob19MD1kozqZIbfs9sOfgBmVBJ
         qz3g==
X-Forwarded-Encrypted: i=1; AJvYcCU1FgrBNp3I9NfaUJuYHigKaU97rbVG+NB1iDwlr/PVm0T9OYfwkGbI4Gs1LJakzrCpJeSlXescth7CZdinfT54wF6TMu2P8XrRz7lwOmE=
X-Gm-Message-State: AOJu0YxkSQR/sWA2fU9Xn1vqWBWXbttsd2tDTPtO9gKB66uRpsJiZut9
	8Waih65/8tXv2qvl8P6GK0CGk+ZWLJWQt4tR5GNYcByPMKFZXDlVh4cFao+BIA==
X-Google-Smtp-Source: AGHT+IG2zMtqh0WFcsIf0BRNOFZg20x5yl2hHGy0N6bHHVf9PgWhdaptam+4kn+EU/tuIZjsIzODEQ==
X-Received: by 2002:a05:6402:14d4:b0:565:dce3:670c with SMTP id f20-20020a05640214d400b00565dce3670cmr1566218edx.41.1708932812099;
        Sun, 25 Feb 2024 23:33:32 -0800 (PST)
Message-ID: <cb463469-51e1-4639-bf56-36fe9051d1b3@suse.com>
Date: Mon, 26 Feb 2024 08:33:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 3/4] xen/include: add pure and const attributes
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Simone Ballarin <simone.ballarin@bugseng.com>, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1697638210.git.simone.ballarin@bugseng.com>
 <fd5421162a00aa782e0776324ff6497193c1e3d3.1697638210.git.simone.ballarin@bugseng.com>
 <89778285-5cba-8fb5-70bc-710b6dd30a10@suse.com>
 <0032186f-80c7-4dba-b46e-10d4a8e2a8cb@bugseng.com>
 <e8bf9817-fd54-9bf4-4302-dcee682f9172@suse.com>
 <alpine.DEB.2.22.394.2310231417260.3516@ubuntu-linux-20-04-desktop>
 <8f8141e2-c55c-2e58-a078-d5d8764a02df@suse.com>
 <alpine.DEB.2.22.394.2402221730270.754277@ubuntu-linux-20-04-desktop>
 <6eb4dc7c-c6a5-496a-a37f-125b00e9aaa3@suse.com>
 <alpine.DEB.2.22.394.2402231432090.754277@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402231432090.754277@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.02.2024 23:36, Stefano Stabellini wrote:
> On Fri, 23 Feb 2024, Jan Beulich wrote:
>> On 23.02.2024 02:32, Stefano Stabellini wrote:
>>> On Tue, 24 Oct 2023, Jan Beulich wrote:
>>>> On 24.10.2023 00:05, Stefano Stabellini wrote:
>>>>> On Mon, 23 Oct 2023, Jan Beulich wrote:
>>>>>> On 23.10.2023 17:23, Simone Ballarin wrote:
>>>>>>> On 23/10/23 15:34, Jan Beulich wrote:
>>>>>>>> On 18.10.2023 16:18, Simone Ballarin wrote:
>>>>>>>>> --- a/xen/include/xen/pdx.h
>>>>>>>>> +++ b/xen/include/xen/pdx.h
>>>>>>>>> @@ -164,7 +164,7 @@ static inline unsigned long pfn_to_pdx(unsigned long pfn)
>>>>>>>>>    * @param pdx Page index
>>>>>>>>>    * @return Obtained pfn after decompressing the pdx
>>>>>>>>>    */
>>>>>>>>> -static inline unsigned long pdx_to_pfn(unsigned long pdx)
>>>>>>>>> +static inline __attribute_pure__ unsigned long pdx_to_pfn(unsigned long pdx)
>>>>>>>>>   {
>>>>>>>>>       return (pdx & pfn_pdx_bottom_mask) |
>>>>>>>>>              ((pdx << pfn_pdx_hole_shift) & pfn_top_mask);
>>>>>>>>
>>>>>>>> Taking this as an example for what I've said above: The compiler can't
>>>>>>>> know that the globals used by the functions won't change value. Even
>>>>>>>> within Xen it is only by convention that these variables are assigned
>>>>>>>> their values during boot, and then aren't changed anymore. Which makes
>>>>>>>> me wonder: Did you check carefully that around the time the variables
>>>>>>>> have their values established, no calls to the functions exist (which
>>>>>>>> might then be subject to folding)?
>>>>>>>
>>>>>>> There is no need to check that, the GCC documentation explicitly says:
>>>>>>>
>>>>>>> However, functions declared with the pure attribute *can safely read any 
>>>>>>> non-volatile objects*, and modify the value of objects in a way that 
>>>>>>> does not affect their return value or the observable state of the program.
>>>>>>
>>>>>> I did quote this same text in response to what Andrew has said, but I also
>>>>>> did note there that this needs to be taken with a grain of salt: The
>>>>>> compiler generally assumes a single-threaded environment, i.e. no changes
>>>>>> to globals behind the back of the code it is processing.
>>>>>
>>>>> Let's start from the beginning. The reason for Simone to add
>>>>> __attribute_pure__ to pdx_to_pfn and other functions is for
>>>>> documentation purposes. It is OK if it doesn't serve any purpose other
>>>>> than documentation.
>>>>>
>>>>> Andrew, for sure we do not want to lie to the compiler and introduce
>>>>> undefined behavior. If we think there is a risk of it, we should not do
>>>>> it.
>>>>>
>>>>> So, what do we want to document? We want to document that the function
>>>>> does not have side effects according to MISRA's definition of it, which
>>>>> might subtly differ from GCC's definition.
>>>>>
>>>>> Looking at GCC's definition of __attribute_pure__, with the
>>>>> clarification statement copy/pasted above by both Simone and Jan, it
>>>>> seems that __attribute_pure__ matches MISRA's definition of a function
>>>>> without side effects. It also seems that pdx_to_pfn abides to that
>>>>> definition.
>>>>>
>>>>> Jan has a point that GCC might be making other assumptions
>>>>> (single-thread execution) that might not hold true in our case. Given
>>>>> the way the GCC statement is written I think this is low risk. But maybe
>>>>> not all GCC versions we want to support in the project might have the
>>>>> same definition of __attribute_pure__. So we could end up using
>>>>> __attribute_pure__ correctly for the GCC version used for safety (GCC
>>>>> 12.1, see docs/misra/C-language-toolchain.rst) but it might actually
>>>>> break an older GCC version.
>>>>>
>>>>>
>>>>> So Option#1 is to use __attribute_pure__ taking the risk that a GCC or
>>>>> Clang version might interpret __attribute_pure__ differently and
>>>>> potentially misbehave.
>>>>>
>>>>> Option#2 is to avoid this risk, by not using __attribute_pure__.
>>>>> Instead, we can use SAF-xx-safe or deviations.rst to document that
>>>>> pdx_to_pfn and other functions like it are without side effects
>>>>> according to MISRA's definition.
>>>>>
>>>>>
>>>>> Both options have pros and cons. To me the most important factor is how
>>>>> many GCC versions come with the statement "pure attribute can safely
>>>>> read any non-volatile objects, and modify the value of objects in a way
>>>>> that does not affect their return value or the observable state of the
>>>>> program".
>>>>>
>>>>> I checked and these are the results:
>>>>> - gcc 4.0.2: no statement
>>>>> - gcc 5.1.0: no statement
>>>>> - gcc 6.1.0: no statement
>>>>> - gcc 7.1.0: no statement
>>>>> - gcc 8.1.0: alternative statement "The pure attribute imposes similar
>>>>>   but looser restrictions on a function’s definition than the const
>>>>>   attribute: it allows the function to read global variables."
>>>>> - gcc 9.1.0: yes statement
>>>>>
>>>>>
>>>>> So based on the above, __attribute_pure__ comes with its current
>>>>> definition only from gcc 9 onward. I don't know if as a Xen community we
>>>>> clearly declare a range of supported compilers, but I would imagine we
>>>>> would still want to support gcc versions older than 9? (Not to mention
>>>>> clang, which I haven't checked.)
>>>>>
>>>>> It doesn't seem to me that __attribute_pure__ could be correctly used on
>>>>> pdx_to_pfn with GCC 7.1.0 for example.
>>>>
>>>> The absence of documentation doesn't mean the attribute had different
>>>> (or even undefined) meaning in earlier versions. Instead it means one
>>>> would need to consult other places (source code?) to figure out whether
>>>> there was any behavioral difference (I don't think there was).
>>>>
>>>> That said, ...
>>>>
>>>>> So in conclusion, I think it is better to avoid __attribute_pure__ and
>>>>> use SAF-xx-safe or an alternative approach instead.
>>>>
>>>> ... I agree here. We just don't want to take chances.
>>>
>>> Let me resurrect this thread.
>>>
>>> Could we use something like "pure" that we #define as we want?
>>>
>>> Depending on the compiler version or other options we could #define pure
>>> to __attribute_pure__ or to nothing.
>>
>> While we can do about anything, I don't think it's a good idea to overload
>> a well known term with something having somewhat different meaning. If a
>> differently named custom attribute helps, that might be a possible option.
> 
> It doesn't have a different meaning. If it had a different meaning I'd
> agree with you.

Then we need to sort this aspect first: If there was no difference in
meaning, we ought to be using the real attribute, not a pseudo
surrogate. Yet the earlier discussion, according to my understanding,
has led to the understanding that for the given example the real
attribute cannot be applied entirely legitimately. Hence why the
thinking of alternatives actually started. What am I missing?

> The goal is for the #define to have exactly the same meaning as the gcc
> definition from gcc 9 onward. However, other versions of gcc or other
> compilers could have different semantics. Also we might not want to
> allow gcc to perform the optimizations that it might want to do if the
> attribute is passed.
> 
> So the definition would be clear and 100% aligned with the modern gcc
> definition. However we would be able to control the behavior better.

If we feared older gcc didn't implement "pure" suitably, we should
simply make __attribute_pure__ expand to nothing there. (Still use of
the attribute then would need limiting to cases where it can validly
be applied.)

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 07:48:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 07:48:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685275.1065731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reViR-0001Jq-KI; Mon, 26 Feb 2024 07:47:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685275.1065731; Mon, 26 Feb 2024 07:47:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reViR-0001Jj-HC; Mon, 26 Feb 2024 07:47:59 +0000
Received: by outflank-mailman (input) for mailman id 685275;
 Mon, 26 Feb 2024 07:47:58 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reViQ-0001Jd-2A
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 07:47:58 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5b05cb20-d47b-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 08:47:56 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a3e550ef31cso291214166b.3
 for <xen-devel@lists.xenproject.org>; Sun, 25 Feb 2024 23:47:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 tl18-20020a170907c31200b00a437d3e975esm101078ejc.210.2024.02.25.23.47.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 25 Feb 2024 23:47:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b05cb20-d47b-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708933676; x=1709538476; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ubgvHi/fNke8SNLpcolxIjm9OoEI4/HbtZuXFNgGEpw=;
        b=fwaolVc3ZezIYJ8+e4q7kNEbT3ez7+yQ+CBUvwRbzywOAYQHJ68/LIyjkjezmG1V10
         cJAUcxPaUrMTUBIbnHUIkeAGwlvUwVjnbQvQeACjaidggrDzF5YL5l3pUOFcBdEuZlS4
         gmP6zBhwXOYRbp1UTi0PQug8+kojcPyoxLJM8qrWnWJrOw8wnUwZg3sQlaQvUQ6lbHtd
         bACVlWQMt7vwoVmRsJCqqaaJgXxVOAZX8b2BcqJyJO+d8rZZD9Ef06v9md3nsw8DyBi9
         ApSCy3fKam+trmERnThr5/lIiLEXwdaZyCWyqhHcIj5Ga7AWSgfgfZAKXjl2/hxSNP+j
         EYkQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708933676; x=1709538476;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ubgvHi/fNke8SNLpcolxIjm9OoEI4/HbtZuXFNgGEpw=;
        b=gGRBgGZSaow1ez+jGsyk4hkhgUPMecH5jIu+QCm4biz02Pnaipk+BpwJZp/ohpVh8I
         CkvvW8m0JMbsoaYuP8zUzeidDoXYaYAuWCQKtwNYlBnq3QFRVz0nfKuy1bBWu7eDBnaI
         Q7vjXuNnSnVDgx8cozCnMIFjGzm8r2o6BO+GuepDHg2tdVt6AkeOeVWHn3IfpKe+7YzZ
         NjWF5//KQpGcBQRRtz6fO6lmwn32GLZArA93pUmxeRdkM8/b73B4kHBegCCiWafk7MbU
         IOPf2skU/n5LIlpJvEIbKAztgiu/vtjq0b+vqTCLMD+CevuCDV3wCbrupi2rgQi9Kzrg
         W9lA==
X-Forwarded-Encrypted: i=1; AJvYcCXq26AhT94LYmAh+XvQcj+6bYnd1PTkMYBoR3eof37QKJGHVuQlP4kL7i0acrqJpo4PeAuXxSg0NuGf1BghmVl9SeZRxc7V91QWZUTEcXI=
X-Gm-Message-State: AOJu0YwEgttsMzxM+x85DRwYdTT9/Qh1V83UZrWU38hvdWD3awAW7Y4C
	nQo+0o+wJnbtZ+sGkDPhjgO3MFDsL+IfDMnkmJHUrJ88/4B99Z7VD8Y2aXEfPA==
X-Google-Smtp-Source: AGHT+IHuAtwmynR1hxiZ3RodGJnTmG1kMnplqrT6Wr4OGCuxLmrqSGsLzM06YRHH/Zp38KZ6wpn/CA==
X-Received: by 2002:a17:906:fb87:b0:a43:793b:5b05 with SMTP id lr7-20020a170906fb8700b00a43793b5b05mr280562ejb.60.1708933676463;
        Sun, 25 Feb 2024 23:47:56 -0800 (PST)
Message-ID: <8f312718-dc6f-4fb5-a78c-39977921fe98@suse.com>
Date: Mon, 26 Feb 2024 08:47:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] x86/altcall: use an union as register type for
 function parameters on clang
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Matthew Grooms <mgrooms@shrew.net>
References: <20240223130354.73936-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240223130354.73936-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.02.2024 14:03, Roger Pau Monne wrote:
> The current code for alternative calls uses the caller parameter types as the
> types for the register variables that serve as function parameters:
> 
> uint8_t foo;
> [...]
> alternative_call(myfunc, foo);
> 
> Would expand roughly into:
> 
> register unint8_t a1_ asm("rdi") = foo;
> register unsigned long a2_ asm("rsi");
> [...]
> asm volatile ("call *%c[addr](%%rip)"...);
> 
> However with -O2 clang will generate incorrect code, given the following
> example:
> 
> unsigned int func(uint8_t t)
> {
>     return t;
> }
> 
> static void bar(uint8_t b)
> {
>     int ret_;
>     register uint8_t di asm("rdi") = b;
>     register unsigned long si asm("rsi");
>     register unsigned long dx asm("rdx");
>     register unsigned long cx asm("rcx");
>     register unsigned long r8 asm("r8");
>     register unsigned long r9 asm("r9");
>     register unsigned long r10 asm("r10");
>     register unsigned long r11 asm("r11");
> 
>     asm volatile ( "call %c[addr]"
>                    : "+r" (di), "=r" (si), "=r" (dx),
>                      "=r" (cx), "=r" (r8), "=r" (r9),
>                      "=r" (r10), "=r" (r11), "=a" (ret_)
>                    : [addr] "i" (&(func)), "g" (func)
>                    : "memory" );
> }
> 
> void foo(unsigned int a)
> {
>     bar(a);
> }
> 
> Clang generates the following assembly code:
> 
> func:                                   # @func
>         movl    %edi, %eax
>         retq
> foo:                                    # @foo
>         callq   func
>         retq
> 
> Note the truncation of the unsigned int parameter 'a' of foo() to uint8_t when
> passed into bar() is lost.  clang doesn't zero extend the parameters in the
> callee when required, as the psABI mandates.
> 
> The above can be worked around by using an union when defining the register
> variables, so that `di` becomes:
> 
> register union {
>     uint8_t e;
>     unsigned long r;
> } di asm("rdi") = { .e = b };
> 
> Which results in following code generated for `foo()`:
> 
> foo:                                    # @foo
>         movzbl  %dil, %edi
>         callq   func
>         retq
> 
> So the truncation is not longer lost.  Apply such workaround only when built
> with clang.
> 
> Reported-by: Matthew Grooms <mgrooms@shrew.net>
> Link: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277200
> Link: https://github.com/llvm/llvm-project/issues/12579
> Link: https://github.com/llvm/llvm-project/issues/82598
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
in the hopes that this isn't going to bite us later.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:00:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:00:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685291.1065746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reVug-00057J-Bt; Mon, 26 Feb 2024 08:00:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685291.1065746; Mon, 26 Feb 2024 08:00:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reVug-00057C-8y; Mon, 26 Feb 2024 08:00:38 +0000
Received: by outflank-mailman (input) for mailman id 685291;
 Mon, 26 Feb 2024 08:00:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reVuf-000576-DT
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:00:37 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1f977dc2-d47d-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 09:00:36 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a3e72ec566aso334260366b.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:00:36 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 z23-20020a170906241700b00a3fb62260e3sm2170726eja.72.2024.02.26.00.00.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 00:00:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f977dc2-d47d-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708934436; x=1709539236; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=EA1HOX6hhzoxW91ktlrtGSHXMNTU0IR+9VU1mSSBE+g=;
        b=ZBi3sd0LfZh2J1sedLOvEP6Z/DZexFcfBO9kTzNFDfLt5+k28SVckfVMMS0PEZayRk
         JJpSskA75NgHwpGItyeByLzB5Em2/TzFYxv5bnOjkxp4A3Jjd35RWbmHI/XwqCf9CZ9x
         onfsL6j55kmHPGvpZ8ibsU9aCzzyG4qKw8TBzJpmrHg4xg+OhEW9ApGvgwdhszoSc8Vz
         A0a9ZAnqJmtZP/t6epFpwW+y9vFw+FfW++Js/dV+vYoGA5VDsylaoVIIoIYYJejIM2ff
         fwJWqaJB/mKG+I2NXwNciOEo4g16Pd4TK7ZQj3tX8Qd5Zf3zc7vo6iwuV5cJStpR7cex
         Zw5g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708934436; x=1709539236;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=EA1HOX6hhzoxW91ktlrtGSHXMNTU0IR+9VU1mSSBE+g=;
        b=HYl9Tw9pJgcTL2OS2+/BtEfROM9ssR1vttT4dluVgCDYfUL6ni1qY9LwXhTsSroQ+p
         juDPUFppv9DOLrsQbOQE4RExNStx+CWzUOu6awock3rpQBxQV65cCAX/f5Y5qDOIe7sw
         5SZPKZJAhVw8X9V2XUe2SMswrab05L/w3dNKgbPRN2Q93X+OR8RUUmchE2dJhyNpYJqO
         05JICPmNxLZ/ydcPU3BaSawK8dfwqYkqHb9D/2Xmp3OPTmXWNL9Ang1t7uyTC0PR3NUp
         Bny8UMlWVe0341VE/pKVsn0lmzHD0akGVwTZiOrfUfvlrpTUbmEpwQzrN6obokscrtcW
         bQ/A==
X-Gm-Message-State: AOJu0YzqZ8UchjfeKK/zk99EJJfRYrEhSbyA0KdNOyd5USDX33Jo3r8b
	6REk9dWRxRLB0C95Hi5+wiJlgnbcqqK5QI2hekpQ7u4dRWAqDrays83cSaNiTQ==
X-Google-Smtp-Source: AGHT+IEpVN6+gaA1w7f2vk1TiHp24Pukl8qKgW9xRUA/aU99oaKVX/55/wuJQJAPTMf9s4l0GB0xMA==
X-Received: by 2002:a17:906:3515:b0:a43:6ce2:3836 with SMTP id r21-20020a170906351500b00a436ce23836mr480994eja.31.1708934435788;
        Mon, 26 Feb 2024 00:00:35 -0800 (PST)
Message-ID: <004877c8-9aea-44b9-aa93-65665b0d3a8d@suse.com>
Date: Mon, 26 Feb 2024 09:00:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 1/2] xen/console: drop return value from
 consoled_guest_rx/tx
To: Stefano Stabellini <sstabellini@kernel.org>,
 Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <4998ec735bd7e5a50a229507e2b92ae56ec1ba4b.1708680104.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2402231456350.754277@ubuntu-linux-20-04-desktop>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402231456350.754277@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.02.2024 23:56, Stefano Stabellini wrote:
> On Fri, 23 Feb 2024, Nicola Vetrini wrote:
>> These functions never saw a usage of their return value since
>> they were introduced, so it can be dropped since their usages
>> violate MISRA C Rule 17.7:
>> "The value returned by a function having non-void return type shall be used".
>>
>> No functional change.
>>
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>

The cleanup is certainly okay, but would one of you mind clarifying in how
far this code is relevant for certification? I don't expect there are plans
to run shim Xen in any projected production uses for which certification is
relevant? (The subject prefix is also unnecessarily wide here, when it's
only daemon code which is affected, not console code in general.)

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:06:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:06:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685294.1065756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reVzu-0005hl-Uu; Mon, 26 Feb 2024 08:06:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685294.1065756; Mon, 26 Feb 2024 08:06:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reVzu-0005he-S6; Mon, 26 Feb 2024 08:06:02 +0000
Received: by outflank-mailman (input) for mailman id 685294;
 Mon, 26 Feb 2024 08:06:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reVzt-0005hY-OA
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:06:01 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e0f33518-d47d-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 09:06:00 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a3f3d0d2787so259502066b.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:06:00 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l9-20020a170906078900b00a3d99dc3f3asm2152648ejc.67.2024.02.26.00.05.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 00:05:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0f33518-d47d-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708934760; x=1709539560; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1ULCwp2UA1ESsjpMBKq9RnRhKl0nq5NgpjAgXototj4=;
        b=CIiv8CtBIzkHqQB35B7nJuHPzWrcvUE9pVrrcMzhkklU7xnc9qXeO1fnu4sY0c+Pq+
         pGYP20R9r/AcH7IIGRJp7uyHH+DTQdHvjF7zIDYCgRhPUzbdRh2amc6tyed6CdTm+55i
         wxMaitrb4iOq/MwzJUWLqvvy94gglI54pDysMYLx6faW7JjjFzPUIxgCGbXRlZDcgBrf
         mmms+UbvpeH6AUEe+WBysEOS4GFfOGFHCLlAFY5vSWj2MynRF3YUxsNwiPseadOtkzKy
         5NiAwS8Q38oP9UB2zyNCV6o2bSMXfmBfqlSaFBCxmSpzCjy6doqEaj9iy8m6hvQsfhbw
         67aw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708934760; x=1709539560;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1ULCwp2UA1ESsjpMBKq9RnRhKl0nq5NgpjAgXototj4=;
        b=GgwGNLCBlBIPlRrKj3CwvCd+pQ8nalJmH7kXTc32EHkwHSNKQ9e6x+Mf94WdzakTtZ
         xDQvc1dOLyP+GIOnnR6ddHVOX4J3a3sEMYLds5rD5U5TsAwkYrpDJhwr1Ng420jH26Tu
         +WFsy/ZP36F84DUyeD07ds8muxiFY32xzysvimW66pNeSwPI5EGpRhcTNiZTCuRRIS9f
         xefHZV64uj73Qj5xYFY37JC+qamilPc//mRsE2RhuCvf7+Fy1T/K2XvzmfTqtZtqeXL+
         /ZeY8dQG068Nj6diRpDiz6WYJCD4A74SxiXEF+HBt19OYDk7zK/1LEDnV7QNAr5QTYLo
         5+RA==
X-Forwarded-Encrypted: i=1; AJvYcCX0D39w5/y3cn/kx/U5REWquS92uc7hCB3G+g/mmYblUweoWNGqAdGzTYh3NSp5dudQGcU9U4OsxarKnCPwvxaGGA+9WZb2vz6oIQp07ss=
X-Gm-Message-State: AOJu0Yxt2hT3Zmy0d1AKILqfNg103cE5LzdUsPLnyv8EFeYGEJkrgflx
	yqRl1vceBXgoNU2n8Zfa+4AilthaKq9xKQDydPqv2T+U4kWDULog5TAF5qZHPw==
X-Google-Smtp-Source: AGHT+IFCw9n6jKSb8+m4CC9/r9lqPHWUkaUESpWwkYw9PS5Gyq6SqLXFwj2sxPu3QDbdg9IvFVjgkQ==
X-Received: by 2002:a17:906:d145:b0:a43:7e63:6997 with SMTP id br5-20020a170906d14500b00a437e636997mr219703ejb.18.1708934760083;
        Mon, 26 Feb 2024 00:06:00 -0800 (PST)
Message-ID: <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
Date: Mon, 26 Feb 2024 09:06:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.02.2024 10:35, Nicola Vetrini wrote:
> Refactor cpu_notifier_call_chain into two functions:
> - the variant that is allowed to fail loses the nofail flag
> - the variant that shouldn't fail is encapsulated in a call
>   to the failing variant, with an additional check.
> 
> This prevents uses of the function that are not supposed to
> fail from ignoring the return value, thus violating Rule 17.7:
> "The value returned by a function having non-void return type shall
> be used".
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

I'm afraid I disagree with this kind of bifurcation. No matter what
Misra thinks or says, it is normal for return values of functions to
not always be relevant to check. To deal with the Misra rule imo
requires to first have an abstract plan of how to handle such
globally in the code base. Imo such a plan can't be to introduce
perhaps dozens of new wrapper functions like is done here.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:16:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:16:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685298.1065767 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWAF-0007dB-Tf; Mon, 26 Feb 2024 08:16:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685298.1065767; Mon, 26 Feb 2024 08:16:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWAF-0007d4-Px; Mon, 26 Feb 2024 08:16:43 +0000
Received: by outflank-mailman (input) for mailman id 685298;
 Mon, 26 Feb 2024 08:16:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reWAE-0007cy-Gl
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:16:42 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5e430e56-d47f-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 09:16:40 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-512a65cd2c7so3873876e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:16:40 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 a12-20020a0ce34c000000b0068fa7e73367sm2630439qvm.42.2024.02.26.00.16.38
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 00:16:39 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e430e56-d47f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708935400; x=1709540200; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Vs+kulZx0w00Y5wFeWg8vZZaaFGqzBJJwJkTF/Xc0/o=;
        b=T0PgZNSt9+ZHiRff0gbqZXR6UDLJ1xALRv7giJE3Yo7255vMNHgZ+wgcFoFZsyoNhL
         Db/HZgy6V2Y8LLSskdX+OqU8g2JpE60wPHwCjJcKEk8hAvV1M0I5gdfSkC2fXpi3ZMO6
         qqd6BdAfmsKV8A1wDvJVOjr/MPi0WQOhO+f2s=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708935400; x=1709540200;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Vs+kulZx0w00Y5wFeWg8vZZaaFGqzBJJwJkTF/Xc0/o=;
        b=WYk5Gz83DXr2gGZXnPXSEUkfasjVVfqU/+vP4N9jgbLa/j5N/MSsCJWFsMPteHr/rv
         lQzKmKZMbc831YQPFUMoZPBVNMZ4P/I3SPapyeeb7qHc2BNlG0RSYv4vYQnn3qR/EdQz
         dR09looIS4XA01VWE5LeIKX1h8W32ECY9nghcTTYtFB0U7J60K4AXPOXZ4ypFttoLs0G
         HDCQ+dDJvpP6NnNM/J6PVO4nM2yfTrjsaAKaFizj3l+xriuFXrowNz5MOulcMUToABSp
         iCm+FeuqgxX37FNQ8vN8Vp1acQkD5L/xLQTTE5v6FCKJtouKrWMo/em2blVYrytDQ/UQ
         BLkA==
X-Gm-Message-State: AOJu0YzKxop16jdyeQ8Y9OdlOsuG9+urQ18kB1uZN8vt/crd2dn+/1H5
	dLbFRA6EBTyeXZ8a4A89okm9ZcEgiL6Pj7YvhNm615Fck+c1JbVzIZXDGEOlUCI=
X-Google-Smtp-Source: AGHT+IGIJPy6uhZPQVjw/b+dE5KZwktCIodv3efAYz03UnPQ3OD2PgR9pgYE718A2NPYX4MkmJmBGQ==
X-Received: by 2002:a05:6512:3dac:b0:512:ed33:c16 with SMTP id k44-20020a0565123dac00b00512ed330c16mr4932678lfv.8.1708935399824;
        Mon, 26 Feb 2024 00:16:39 -0800 (PST)
Date: Mon, 26 Feb 2024 09:16:31 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Ross Lagerwall <ross.lagerwall@citrix.com>
Cc: xen-devel@lists.xenproject.org,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [PATCH 3/5] xen/livepatch: fix norevert test attempt to
 open-code revert
Message-ID: <ZdxI3z1LYUkk7WkY@macbook>
References: <20231130142944.46322-1-roger.pau@citrix.com>
 <20231130142944.46322-4-roger.pau@citrix.com>
 <CAG7k0ErX0yydgTyZQM=CU+7ZgWnrvqJGXRGmUtM8=O9_UH=12A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAG7k0ErX0yydgTyZQM=CU+7ZgWnrvqJGXRGmUtM8=O9_UH=12A@mail.gmail.com>

On Fri, Feb 23, 2024 at 10:10:38AM +0000, Ross Lagerwall wrote:
> On Thu, Nov 30, 2023 at 2:30 PM Roger Pau Monne <roger.pau@citrix.com> wrote:
> > diff --git a/xen/test/livepatch/xen_action_hooks_norevert.c b/xen/test/livepatch/xen_action_hooks_norevert.c
> > index 3e21ade6abfc..074f8e1d56ce 100644
> > --- a/xen/test/livepatch/xen_action_hooks_norevert.c
> > +++ b/xen/test/livepatch/xen_action_hooks_norevert.c
> > @@ -96,26 +96,14 @@ static int revert_hook(livepatch_payload_t *payload)
> >
> >  static void post_revert_hook(livepatch_payload_t *payload)
> >  {
> > -    int i;
> > +    unsigned long flags;
> >
> >      printk(KERN_DEBUG "%s: Hook starting.\n", __func__);
> >
> > -    for (i = 0; i < payload->nfuncs; i++)
> > -    {
> > -        const struct livepatch_func *func = &payload->funcs[i];
> > -        struct livepatch_fstate *fstate = &payload->fstate[i];
> > -
> > -        BUG_ON(revert_cnt != 1);
> > -        BUG_ON(fstate->applied != LIVEPATCH_FUNC_APPLIED);
> > -
> > -        /* Outside of quiesce zone: MAY TRIGGER HOST CRASH/UNDEFINED BEHAVIOR */
> > -        arch_livepatch_quiesce();
> > -        common_livepatch_revert(payload);
> > -        arch_livepatch_revive();
> > -        BUG_ON(fstate->applied == LIVEPATCH_FUNC_APPLIED);
> > -
> > -        printk(KERN_DEBUG "%s: post reverted: %s\n", __func__, func->name);
> > -    }
> > +    local_irq_save(flags);
> > +    revert_payload(payload);
> 
> Should this check or assert the return value of revert_payload()?

Hm, yes, why not.  I will put this inside of a BUG_ON(), as
post-revert hook doesn't return any value.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:23:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:23:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685301.1065777 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWGi-00015p-Hh; Mon, 26 Feb 2024 08:23:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685301.1065777; Mon, 26 Feb 2024 08:23:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWGi-00015i-Ek; Mon, 26 Feb 2024 08:23:24 +0000
Received: by outflank-mailman (input) for mailman id 685301;
 Mon, 26 Feb 2024 08:23:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=4NSY=KD=linaro.org=philmd@srs-se1.protection.inumbo.net>)
 id 1reWGg-00015c-SG
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:23:22 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4cf8fc38-d480-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 09:23:20 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a3e4765c86eso322359366b.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:23:20 -0800 (PST)
Received: from [192.168.69.100] ([176.176.164.69])
 by smtp.gmail.com with ESMTPSA id
 fj8-20020a1709069c8800b00a42ec98b9afsm2127370ejc.158.2024.02.26.00.23.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 00:23:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4cf8fc38-d480-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=linaro.org; s=google; t=1708935800; x=1709540600; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:from:references:cc:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IHkkhH3shB94Aa6njH/RF4O9pI/05EhqdX+P2NTRny8=;
        b=AfFyQewjQASx5azdB3zGaM7h0qgTHfvtQ2F/0gw7UlMsp3Pz+qYfY8Pwrb+5CHBgQq
         O8m1k41Eykei1b/qBj1TKqdxEc/LCoJOGniRzSv7ISaLZD/D2mixQ4We0Yk+HpqOc+Lc
         eJIYF4ciS6x4EXFIWM6glRTP5chuiEclzn2BGbYsgjV1GfcyPZgO3ByXAIecLTwd5oTn
         fy4sDv8Q+Ni9IfWNgJWgRECwxPQA9aydGA2dsoDh5adDWsXkbHQlenX/EtY7JP9FBUwm
         FeXSh3LOma6GvyK9C36uOpGDvriDutqVXvZCYSQd/sxtDDT4ZAzLrRXpMQ1giggTtuoO
         SwGw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708935800; x=1709540600;
        h=content-transfer-encoding:in-reply-to:from:references:cc:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IHkkhH3shB94Aa6njH/RF4O9pI/05EhqdX+P2NTRny8=;
        b=UKvvAzM9nVuLlis3dJO46qUyoOG7gSzC79r7BWL8U4jASswEN494jBMb/7b8lAjSIY
         2W67XScVG5QDqxQ9ulz8WgSPD95BloXEYlMk9OuuE6TwVmztMTL5506GQLspeVYMANjh
         JjHYLXztBF3+m4syIyNnW23iI6xpI9z51WbT0fk8T0ZPjtXwJ+2TJsQ5SZnErJwLuzfA
         uSTFiGo++7Tm1t1RCRFRYerP+wo3yxGfcopWnmS/n/aKf5NmPwiraqJLPwP5XfyWJfXO
         iH3iOwbCo65ud819rJE+52DTdmMH/BYfxY6V1/r6AiEgZnFlBl+naBpKUFjjTGRHeCN6
         6Gyw==
X-Forwarded-Encrypted: i=1; AJvYcCXgT5I8/bPGVc2H0MDe7M+Y8+YmYnb9z+0ScdSBBlBFgAyMGJGM0rySLozaCpGGBqbi1kKqHEzdmOkw2xe1aWMf0jJI8JQayQoLGJ+YD34=
X-Gm-Message-State: AOJu0YyyVTYNWL0JNiIpARPSfs8VXx82Nbm5gCxXjEFFaKTt5+CHwq2p
	PXDnXt+c15nBNzSxutCEdCUJH6OAvYQYHidfy+7iL+p7DA5QAnd7ih9DTeGm3jk=
X-Google-Smtp-Source: AGHT+IHOnMGtNqT1/asAZtr5MYy0RSUUVEKYMToG1GqpcICWSSVYwVWbEB7USOctPs4WSxY3+ruo5g==
X-Received: by 2002:a17:906:40d1:b0:a3e:93d0:3443 with SMTP id a17-20020a17090640d100b00a3e93d03443mr3955053ejk.34.1708935800385;
        Mon, 26 Feb 2024 00:23:20 -0800 (PST)
Message-ID: <601bae2e-4e42-4e88-aa97-e94bb4696ab8@linaro.org>
Date: Mon, 26 Feb 2024 09:23:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] hw/xen: detect when running inside stubdomain
Content-Language: en-US
To: =?UTF-8?Q?Marek_Marczykowski-G=C3=B3recki?=
 <marmarek@invisiblethingslab.com>, qemu-devel@nongnu.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
 "open list:X86 Xen CPUs" <xen-devel@lists.xenproject.org>
References: <20240219181627.282097-1-marmarek@invisiblethingslab.com>
From: =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>
In-Reply-To: <20240219181627.282097-1-marmarek@invisiblethingslab.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 19/2/24 19:16, Marek Marczykowski-Górecki wrote:
> Introduce global xen_is_stubdomain variable when qemu is running inside
> a stubdomain instead of dom0. This will be relevant for subsequent
> patches, as few things like accessing PCI config space need to be done
> differently.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
>   hw/xen/xen-legacy-backend.c | 15 +++++++++++++++
>   include/hw/xen/xen.h        |  1 +
>   system/globals.c            |  1 +
>   3 files changed, 17 insertions(+)


> +static bool xen_check_stubdomain(void)
> +{
> +    char *dm_path = g_strdup_printf("/local/domain/%d/image", xen_domid);
> +    uint32_t dm_domid;
> +    bool is_stubdom = false;
> +
> +    if (!xenstore_read_int(dm_path, "device-model-domid", &dm_domid))

BTW missing braces for QEMU coding style: {

> +        is_stubdom = dm_domid != 0;

}

> +
> +    g_free(dm_path);
> +    return is_stubdom;
> +}



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:23:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:23:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685302.1065787 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWH4-0001XK-PO; Mon, 26 Feb 2024 08:23:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685302.1065787; Mon, 26 Feb 2024 08:23:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWH4-0001XD-Mc; Mon, 26 Feb 2024 08:23:46 +0000
Received: by outflank-mailman (input) for mailman id 685302;
 Mon, 26 Feb 2024 08:23:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=HDYM=KD=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1reWH2-00015c-Vd
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:23:44 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5a338a19-d480-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 09:23:43 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id A270E4EE0742;
 Mon, 26 Feb 2024 09:23:42 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a338a19-d480-11ee-98f5-efadbce2ee36
MIME-Version: 1.0
Date: Mon, 26 Feb 2024 09:23:42 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org, consulting@bugseng.com, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH 1/2] xen/console: drop return value from
 consoled_guest_rx/tx
In-Reply-To: <004877c8-9aea-44b9-aa93-65665b0d3a8d@suse.com>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <4998ec735bd7e5a50a229507e2b92ae56ec1ba4b.1708680104.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2402231456350.754277@ubuntu-linux-20-04-desktop>
 <004877c8-9aea-44b9-aa93-65665b0d3a8d@suse.com>
Message-ID: <b417d7a330addd295b3cbbbac4bed2a4@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-26 09:00, Jan Beulich wrote:
> On 23.02.2024 23:56, Stefano Stabellini wrote:
>> On Fri, 23 Feb 2024, Nicola Vetrini wrote:
>>> These functions never saw a usage of their return value since
>>> they were introduced, so it can be dropped since their usages
>>> violate MISRA C Rule 17.7:
>>> "The value returned by a function having non-void return type shall 
>>> be used".
>>> 
>>> No functional change.
>>> 
>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>> 
>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> The cleanup is certainly okay, but would one of you mind clarifying in 
> how
> far this code is relevant for certification? I don't expect there are 
> plans
> to run shim Xen in any projected production uses for which 
> certification is
> relevant? (The subject prefix is also unnecessarily wide here, when 
> it's
> only daemon code which is affected, not console code in general.)
> 

I agree on the subject prefix being too wide. The configuration that 
uses consoled_guest_tx is #ifdef-ed for x86, so even in configurations 
that may never reach this condition this is relevant, unless its #ifdef 
is restricted to cases where the call may actually be reachable.
consoled_guest_rx was adjusted for uniformity, even though it's 
technically not included in the configuration under analysis at the 
moment.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:26:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:26:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685305.1065796 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWJD-00028b-4l; Mon, 26 Feb 2024 08:25:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685305.1065796; Mon, 26 Feb 2024 08:25:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWJD-00028U-25; Mon, 26 Feb 2024 08:25:59 +0000
Received: by outflank-mailman (input) for mailman id 685305;
 Mon, 26 Feb 2024 08:25:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reWJC-00028K-8H
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:25:58 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a98e9f3f-d480-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 09:25:56 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a3ed9cae56fso432353266b.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:25:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i19-20020a17090639d300b00a3f81200425sm2166049eje.122.2024.02.26.00.25.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 00:25:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a98e9f3f-d480-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708935956; x=1709540756; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=eUeYrdWPS0lIWnR9EDneERJUkpPmXqsRbAsuMqSX10c=;
        b=T/J64ZbnBGqi2XZfnsELPtYymXPjIWdQLnArSUNRJAXdsdMm6hxmOaoif37kzvi5wO
         /HOONfGxziMvn0flDE2UcZcq6v/ME+n8UtFZ+XQOOwKQfC0/i7FPi0mP+Sf8mMmcgCU3
         XXxltb3m0ppZ0ilPTbm19d+/t1hq1a2CwujWJOq8mKbJdB10CoocZO8p1nO4VcY1DR+g
         eWTi4iplfrE8IZVsmAaqdEBGbCobfP7Mt535Vak+PhrTdUeUWo6nOeWnbwGJM63PdscY
         omNo/Is/f2bWaqVGIHs1BNTUlQnY0XyTjyjrkG6nP8uZNd1IAVKrQX015zuPIFX9qTPb
         Hpfw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708935956; x=1709540756;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=eUeYrdWPS0lIWnR9EDneERJUkpPmXqsRbAsuMqSX10c=;
        b=NgZvlWIRWQ4yGwHZqXIhU79GAP5QjHAogoyX//Sdr/Rkz+gUe3FQWWEf00qT4ncm1H
         GBpbyR1+bKaUzW7g/l1HGitBUlkTNj8xnQd5IBuicVGZwK1Pdt9Rqxa5InI1T/tAxdM6
         /OGHxY2kF2JvcK8dW+mo3lvEvIjXPNHUWd7Z8X/CAamdv0I4s811Y2CHxg4I0oCj3XiA
         /vTBOS+CnHpc3C6nYWH0A4M2wD+VkNqX9jkwEXc8gltNFbGNIyG6DdqtsfL87TzQjV9a
         nC+l9RmKUYRJQVKPtXWq4cJc5Ft7DpYq6bg2QxFymyY4plFw6r8HBr8hrTB5EZXV4zI/
         WEsA==
X-Forwarded-Encrypted: i=1; AJvYcCX6UenN7Mj1NBoIKyMDUk3H8+y3InOVRnDZ/MUMN8FY2hA7AFj5rHcfuIy8G+FCql7zqPqxqdnN9SRC1AwZ02s/djHrJEB+S17TgGaV5iU=
X-Gm-Message-State: AOJu0YyDlB4zI/H1itJtUL3yjH5olSKr/1bBv7ri1kGSwESfCYhUY/7+
	MJqUaPnvgxAA62egQIUcUTM+XlUQpP7rLsVLxBmEuIHd6QvLH4v29dYYiqOmXw==
X-Google-Smtp-Source: AGHT+IGqtR6wy19YcGHLZpETATECuDGQuxEVIsnI6MUKjSanaFGrayR4faxLNXT73ARQHZypG66wPg==
X-Received: by 2002:a17:906:e298:b0:a42:f0b6:c2f1 with SMTP id gg24-20020a170906e29800b00a42f0b6c2f1mr4819760ejb.28.1708935955728;
        Mon, 26 Feb 2024 00:25:55 -0800 (PST)
Message-ID: <b41daccb-d402-4da8-bc88-933facd088a8@suse.com>
Date: Mon, 26 Feb 2024 09:25:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
Content-Language: en-US
To: Henry Wang <xin.wang2@amd.com>
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Alec Kwapis <alec.kwapis@medtronic.com>,
 xen-devel@lists.xenproject.org
References: <20240226011935.169462-1-xin.wang2@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226011935.169462-1-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 02:19, Henry Wang wrote:
> --- a/xen/arch/arm/include/asm/mm.h
> +++ b/xen/arch/arm/include/asm/mm.h
> @@ -428,6 +428,19 @@ static inline void page_set_xenheap_gfn(struct page_info *p, gfn_t gfn)
>      } while ( (y = cmpxchg(&p->u.inuse.type_info, x, nx)) != x );
>  }
>  
> +#define MAGIC_PAGE_N_GPFN(n)     ((GUEST_MAGIC_BASE >> PAGE_SHIFT) + n)
> +static inline bool is_magic_gpfn(xen_pfn_t gpfn)
> +{
> +    unsigned int i;
> +    for ( i = 0; i < NR_MAGIC_PAGES; i++ )

Nit: Blank line please between declaration(s) and statement(s).

> --- a/xen/arch/ppc/include/asm/mm.h
> +++ b/xen/arch/ppc/include/asm/mm.h
> @@ -256,4 +256,9 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
>      return true;
>  }
>  
> +static inline bool is_magic_gpfn(xen_pfn_t gpfn)
> +{
> +    return false;
> +}
> +
>  #endif /* _ASM_PPC_MM_H */
> --- a/xen/arch/riscv/include/asm/mm.h
> +++ b/xen/arch/riscv/include/asm/mm.h
> @@ -3,6 +3,7 @@
>  #ifndef _ASM_RISCV_MM_H
>  #define _ASM_RISCV_MM_H
>  
> +#include <public/xen.h>
>  #include <asm/page-bits.h>
>  
>  #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
> @@ -20,4 +21,9 @@ unsigned long calc_phys_offset(void);
>  
>  void turn_on_mmu(unsigned long ra);
>  
> +static inline bool is_magic_gpfn(xen_pfn_t gpfn)
> +{
> +    return false;
> +}
> +
>  #endif /* _ASM_RISCV_MM_H */
> --- a/xen/arch/x86/include/asm/mm.h
> +++ b/xen/arch/x86/include/asm/mm.h
> @@ -628,4 +628,9 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
>      return (mfn + nr) <= (virt_to_mfn(eva - 1) + 1);
>  }
>  
> +static inline bool is_magic_gpfn(xen_pfn_t gpfn)
> +{
> +    return false;
> +}

I don't think every arch should need to gain such a dummy function. Plus
the function name doesn't clarify at all what kind of "magic" this is
about. Plus I think the (being phased out) term "gpfn" would better not
be used in new functions anymore. Instead type-safe gfn_t would likely
better be used as parameter type.

> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>          }
>          else
>          {
> -            if ( is_domain_direct_mapped(d) )
> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
>              {
>                  mfn = _mfn(gpfn);
>  

I wonder whether is_domain_direct_mapped() shouldn't either be cloned
into e.g. is_gfn_direct_mapped(d, gfn), or be adjusted in-place to gain
such a (then optional) 2nd parameter. (Of course there again shouldn't be
a need for every domain to define a stub is_domain_direct_mapped() - if
not defined by an arch header, the stub can be supplied in a single
central place.)

> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -476,6 +476,12 @@ typedef uint64_t xen_callback_t;
>  #define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
>  #define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
>  
> +#define NR_MAGIC_PAGES 4
> +#define CONSOLE_PFN_OFFSET 0
> +#define XENSTORE_PFN_OFFSET 1
> +#define MEMACCESS_PFN_OFFSET 2
> +#define VUART_PFN_OFFSET 3
> +
>  #define GUEST_RAM_BANKS   2

Of these only NR_MAGIC_PAGES is really used in Xen, afaics.

Also while this is added to a tools-only section, I'm also concerned of
the ongoing additions here without suitable XEN_ prefixes. Any number
of kinds of magic pages may exist for other reasons in a platform; which
ones are meant would therefore better be sufficiently clear from the
identifier used.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:29:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:29:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685308.1065808 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWMC-0002mZ-Ng; Mon, 26 Feb 2024 08:29:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685308.1065808; Mon, 26 Feb 2024 08:29:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWMC-0002mS-JJ; Mon, 26 Feb 2024 08:29:04 +0000
Received: by outflank-mailman (input) for mailman id 685308;
 Mon, 26 Feb 2024 08:29:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reWMB-0002mM-4r
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:29:03 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 18350aae-d481-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 09:29:01 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-564fc495d83so3084520a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:29:01 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 tk6-20020a170907c28600b00a4306ac4c77sm1717747ejc.197.2024.02.26.00.29.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 00:29:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 18350aae-d481-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708936141; x=1709540941; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=cjTSGymniHHGW5tzXzYlfaZlNi2W4NqR8URzivEqBDU=;
        b=J/9U1/PGY/XvJnoOo5lmcIqZcxX3A+TPMqVHltDtCIuppVYNdHERRHoqYNBvpjad9k
         iVp46MOWXnxOS49R1eXwNsi5kdO017w5U0kMKKt+AqdjfiHVpkONqA8uJw4crfiDraEf
         V4vga8VfqXndP7QaV3S3qjnqFa0jI7ZWWVht8wgo5LCG1L2vWtJUwUnnYnvgQDuLfGFA
         4DQLlWksWjpaVwgNSwhjj6vicRD0IN24cNSoNAK0zKrVUx1Fk1wCTwWZlf3FLqhRttdG
         /J34Q58ClHxIoHW3x6GXQmj73e44b1IcFEuOH5oHS/lkXU7f1D57O7C4nz8mjdQH1lJV
         aH0w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708936141; x=1709540941;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=cjTSGymniHHGW5tzXzYlfaZlNi2W4NqR8URzivEqBDU=;
        b=gUT6J/X9dUPUve/qaFszRhhJZQg6rKRdOuRHZTINhhO0bdOH4HT6RrY7ummighS8Hc
         GjB4f8MvvoTbrgYx7fmGy1wsUfUKIQgotkMlA4JP0jqrpOGziGsergzKmJ2MJXRyXfn4
         uYgBMrYqNJJsljWL+TxO+QwFdaQCmWl2HDtTM9d8FJXN7sblq+BNo0HRlR3TpzdLGShY
         GuQ5MoMNwkTc6pbAUydPdaR0dAoL6IqYubAR6f0Dbdu8IOWuiscumGLxGnUa+SQHHZGk
         rVH1lHzh5P85oN3PVH8UPzfS9DyQMWeKyTJYBa1JvG2HGDqbzOn7UHehwpRv1iyWyb4i
         hhxw==
X-Forwarded-Encrypted: i=1; AJvYcCVyYe7smp6nrmcoFoKPRu6VqV4ju7i66J0rXDwzXrEMVaw9ySLJUP7T7oxElwRSQDiBEwMnDQWoJ+EvSkaC3+uxvZGd+gqAndd5Md5BG3I=
X-Gm-Message-State: AOJu0YzHd2BJ/Bo3XQzTugWbYVQSpLi3fWpZg/2eWgRO4Z9UwTfWE72q
	1qOi3l4+gITis1QRPr4lioGHJgYQ/jex1xmLFThw4iI+7GohXIumlNoFTpXB7w==
X-Google-Smtp-Source: AGHT+IH+KPONZ8u/2vRpsLBDicr5yT50J7zxWwhmIYa8GWV0d2jdz7Md6SDJ/qJhn1nIadHbvrG4Rw==
X-Received: by 2002:a17:906:fa07:b0:a3c:cbab:f3b with SMTP id lo7-20020a170906fa0700b00a3ccbab0f3bmr4925525ejb.23.1708936141400;
        Mon, 26 Feb 2024 00:29:01 -0800 (PST)
Message-ID: <3ed4e0e9-fec4-4e8b-a52d-fb4756ae50cd@suse.com>
Date: Mon, 26 Feb 2024 09:29:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/3] xen/cmdline: fix printf format specifier in
 no_config_param()
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240223120616.73503-1-roger.pau@citrix.com>
 <20240223120616.73503-2-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240223120616.73503-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.02.2024 13:06, Roger Pau Monne wrote:
> '*' sets the width field, which is the minimum number of characters to output,
> but what we want in no_config_param() is the precision instead, which is '.*'
> as it imposes a maximum limit on the output.
> 
> Fixes: 68d757df8dd2 ('x86/pv: Options to disable and/or compile out 32bit PV support')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:33:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:33:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685314.1065816 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWQB-0004g9-5e; Mon, 26 Feb 2024 08:33:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685314.1065816; Mon, 26 Feb 2024 08:33:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWQB-0004g2-2z; Mon, 26 Feb 2024 08:33:11 +0000
Received: by outflank-mailman (input) for mailman id 685314;
 Mon, 26 Feb 2024 08:33:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reWQ9-0004fw-Bq
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:33:09 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aaa9de41-d481-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 09:33:07 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a26fa294e56so479056166b.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:33:07 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 cw9-20020a170907160900b00a3fb7cafad8sm2204014ejd.39.2024.02.26.00.33.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 00:33:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aaa9de41-d481-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708936387; x=1709541187; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9AlK1QA3IkmQoJPu/OAS6Z5gNe9NMItGQZvQvtBNbik=;
        b=bbLvutCaKKSCNtnmcTOtiUFM9spTpNk/gmqef8RScUx4md0HkEasTwfaJWjRGMURkP
         E6fssPLDaLrGDhMxR4zV72M8GLIqvvKmeUGk099pS9yIsHevdr3RKwrYgNwKmVcmOqpV
         XT0RiMWhPc/K1U3P9zIdxIAJ9RJGiBMRX5PO7Z/o668Z9FC25T91IlF/Fs0bNE+lJca6
         GPzFLKxfhqAyxdmYs4aiAkpi7DAhtLVJBcXXPEeXbUib7jzUq8Utf162OmExaxdw2fsJ
         iKdT6MCW2517evvGHj9Hjx9Fww/taT32mH7WblZYRbNFV9ilPjMYmqA8R25XXb9fuZKB
         tH9w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708936387; x=1709541187;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9AlK1QA3IkmQoJPu/OAS6Z5gNe9NMItGQZvQvtBNbik=;
        b=FuX2sz3oZrjc0HhvtePmJfk7uLa6t2/2Da9nIQV7LHv2h4dwxpX8fssxFEbhcFWF8y
         EhuZaf6E4UnF1iria9mkQcfUjvR1whnaSE4He0N4Et7WHeN8GG9V4chAd99NnnpXtvF1
         q0RZHGQSkuAuEpA1JkLxpeb1ppPWqZmXG56ydkiiFyzv/dkxMDHA8iB0+e5bJbPuzioE
         GkIFpljeSUzybueo4whejtyHdisCTRDlnRmONXtUSxOzX8AdOEeQUpYkik/wjhF/bB3X
         z+lvNIaC302A1t2tym/FIJk+vaO4HD2fbGTfY/be4zcHJSkgm+poHaj9FPqtx6h/shRw
         6CQw==
X-Forwarded-Encrypted: i=1; AJvYcCWMrbkW0IfOcon1/9iAtq2x0bxGWttsQsx8ZghrKZOlqaL/yhjPe+L+w1NEOdLXSZjxjAvtqqi8to2tCnonDR2qtnmvGoeRKyxST1iqJ5U=
X-Gm-Message-State: AOJu0YyYMt3gW/17uRyqjWOauNmLpszlozmgSJjzsDuhh3rbp5SUyUlR
	oUlVh5vLVJtzpS6pjeYGJXWUWcFhhWNm8K6gWw1qElCfkatrMbhlBlQKAdEJWw==
X-Google-Smtp-Source: AGHT+IGsbUSL3lcsTh5mxgyZYAPlYqCNucjER0qehHR/o5ygudRx9tXvAyELpwOLdpYLH2pSq77PUQ==
X-Received: by 2002:a17:907:1707:b0:a42:f801:d664 with SMTP id le7-20020a170907170700b00a42f801d664mr3335006ejc.17.1708936387077;
        Mon, 26 Feb 2024 00:33:07 -0800 (PST)
Message-ID: <bf97563f-0354-43cc-9a3b-7e4a28afeceb@suse.com>
Date: Mon, 26 Feb 2024 09:33:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/3] x86/spec: fix BRANCH_HARDEN option to only be set
 when build-enabled
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240223120616.73503-1-roger.pau@citrix.com>
 <20240223120616.73503-3-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240223120616.73503-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 23.02.2024 13:06, Roger Pau Monne wrote:
> The current logic to handle the BRANCH_HARDEN option will report it as enabled
> even when build-time disabled. Fix this by only allowing the option to be set
> when support for it is built into Xen.
> 
> Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> Changes since v1:
>  - Use no_config_param().
> ---
>  xen/arch/x86/spec_ctrl.c | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
> index 421fe3f640df..4ce8a7a0b8ef 100644
> --- a/xen/arch/x86/spec_ctrl.c
> +++ b/xen/arch/x86/spec_ctrl.c
> @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
>  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
>  int8_t __read_mostly opt_eager_fpu = -1;
>  int8_t __read_mostly opt_l1d_flush = -1;
> -static bool __initdata opt_branch_harden = true;
> +static bool __initdata opt_branch_harden =
> +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
>  
>  bool __initdata bsp_delay_spec_ctrl;
>  uint8_t __read_mostly default_xen_spec_ctrl;
> @@ -268,7 +269,14 @@ static int __init cf_check parse_spec_ctrl(const char *s)
>          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
>              opt_l1d_flush = val;
>          else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> +        {
> +#ifdef CONFIG_SPECULATIVE_HARDEN_BRANCH
>              opt_branch_harden = val;
> +#else
> +            no_config_param("SPECULATIVE_HARDEN_BRANCH", "spec-ctrl", s, ss);
> +            rc = -EINVAL;
> +#endif
> +        }

If you use #ifdef (rather than IS_ENABLED()) here, the variable probably
shouldn't exist at all when the Kconfig option is off (albeit yet, another
#ifdef would then be needed higher up in the function). Question is - why
don't you use IS_ENABLED() here as well?

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:39:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:39:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685316.1065827 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWWG-0005Ux-RZ; Mon, 26 Feb 2024 08:39:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685316.1065827; Mon, 26 Feb 2024 08:39:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWWG-0005Uq-Nb; Mon, 26 Feb 2024 08:39:28 +0000
Received: by outflank-mailman (input) for mailman id 685316;
 Mon, 26 Feb 2024 08:39:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5KjY=KD=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1reWWF-0005TO-Ee
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:39:27 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8ab53c5a-d482-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 09:39:24 +0100 (CET)
Received: from CY5PR13CA0042.namprd13.prod.outlook.com (2603:10b6:930:11::10)
 by CY5PR12MB6646.namprd12.prod.outlook.com (2603:10b6:930:41::14)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.34; Mon, 26 Feb
 2024 08:39:21 +0000
Received: from CY4PEPF0000FCC1.namprd03.prod.outlook.com
 (2603:10b6:930:11:cafe::95) by CY5PR13CA0042.outlook.office365.com
 (2603:10b6:930:11::10) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.23 via Frontend
 Transport; Mon, 26 Feb 2024 08:39:21 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC1.mail.protection.outlook.com (10.167.242.103) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 26 Feb 2024 08:39:20 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb
 2024 02:39:20 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb
 2024 02:39:19 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 26 Feb 2024 02:39:17 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8ab53c5a-d482-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=XDbNHh2vSiG+rPtutxKR81viafXmipkzZLnjeAX3434cOkBjaD4AmEHryDsgwv49fa66IKr8pnlMFQ076mAQjcQPo5nht+I1LKs15LX4sYOQEv4EPftwy4E/FfhBliJgNAU5mEEu+Z4smMsY93bwe8uaJpg/EjO38tKr+pJlPcbOuO0ilAcvU7xgcti5PxRKmUdlHJ27VFT9N6cc0gSLy7gVxbQgNZzl4bBl/Lkd6YBRV34HpCQQIRG69fUe7YK6g9lAhLPakrWMVccGoDpuXOZm2e7IOkEnsUAjNd+Utf7FFeqEsiYbd5951tAbFE/AhVlYrEGVoKn8mUVbM3JDIA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=a/jCDJu3unxdFq/kDk7cAywEYK4eppDvAEGWf84PgjE=;
 b=UVrXM90R2u4jWG8JEA/XzRus95X18yj+dZRpdEZUklynsk850jQv4kpSuEBaV9B5QwTGRlO+UAdB05i86blgKiiNrlN+aw8Mj5ret41+JU4m4jOecigJE+QQgo6R4Qq0INNXB5GQHY8zmYLVFkaFyM25lQtNgVHJbMDIWZXNS6jRB1duvNqB5WhcheLvfPYvy/jEN5PkYfeWrgz/JeQ7imQx+4rcUIfHnRaXGVzOk9/FNF/996tjUWaX94Kqf7toXkyUvL+fyZuMYahbAhv1LhsEh7TuZXQCKw7ZlBUtj2HjROTCUBibozewocSIQgB/rnf1o42wsQLNIR2CiDK/6g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=a/jCDJu3unxdFq/kDk7cAywEYK4eppDvAEGWf84PgjE=;
 b=xQdQL4P46UHVJZ/dKIBMsze65Q4KJk0/VO3mjs7X3mH5BN5copglpmR5NWAVF3a1WghY3egNREw/SVB8ef7fPQ1sdf3VYA6KWIToDjlE2FcMMN+k5lruaZFYkBVO3tviptvoHTpcXpg8dULqUuk1Silbfbu4pVmHornZWH74UVY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <9579f84a-ec9e-44be-b614-b70ced221c47@amd.com>
Date: Mon, 26 Feb 2024 09:39:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v5 2/3] xen/arm: arm64: Add emulation of Debug Data Transfer
 Registers
Content-Language: en-US
To: Ayan Kumar Halder <ayankuma@amd.com>, Ayan Kumar Halder
	<ayan.kumar.halder@amd.com>, <xen-devel@lists.xenproject.org>
CC: <sstabellini@kernel.org>, <stefano.stabellini@amd.com>, <julien@xen.org>,
	<Volodymyr_Babchuk@epam.com>, <bertrand.marquis@arm.com>
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
 <20240220121743.3680715-3-ayan.kumar.halder@amd.com>
 <c5c0a46b-8793-481d-a678-da0929a0d6e9@amd.com>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <c5c0a46b-8793-481d-a678-da0929a0d6e9@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC1:EE_|CY5PR12MB6646:EE_
X-MS-Office365-Filtering-Correlation-Id: 3741897d-57cb-456c-dfe3-08dc36a66d53
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	9cdmHlN85BzdC1d0fKE8iaaaL3MydkF2kLzKqXUwLXhm2/lTK+/tAVBNhrbYkLkqs6y3BtauHKO/NI3KyVEnA4Yh+8cqp+8ODiRxZ4u8MAFBflf8ws2RuKvZEN9BATeMb9F3Uqw7HbaI9GNhQ+CLs2TFKrc6c5p0NqHr8RIJKkaSvJKmpd3RGe+LLjj1JZLcfdDT62RcW7VMcUnPBfkZWJvTYOluCyY5tsNVmiqbjtkWwMMbohsC6vBDSg+Z7u3f8j8DyAWSXhosl/wa6lQapOhdxidzvVje1PRXbt0EouEC1FpVMJfnQSDIyxs+xgH1drwYb6irk6JYC5i/K/vdqHbVYcYlZyr9ZfJPigGHdmimnteKQFKhLvbBPGgVO5Ga6HwhOsmrdy3ixR4RG6Gg5ARzoYbUpTyWmY2LKEbVyUptWTDRJIHhRDoQBqtZ2rD/a1lsRGvLO14HqnX4nm3JKAoANpVgVp0Az7a4wGI/dK0MQVbp87hntL/bUKd93mvRrQjlLCF5ZSfU4ActLf72Qc9RELmSASmc5V5C9aKgapEqGgFnnG2W8fk3rZLoYFlM7Qn+3SNXzn4nEuBkq8df3Rd3/YJc6AAn30zcCRIlCxCvXo2ESoU1LN4sqOpaqibn6T5dBXbJOT87wYJzSAQ03vrIgFjtpq9AwmEi1m4dKPdtVqlMcuvUhgOU/Kt/Tjxv2/l0p3qqU6ZEnULK55UTcxiUFxu4O7yKQj+jV2ykeCKGXiLXyichmITjf/UbOsOe
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 08:39:20.5358
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3741897d-57cb-456c-dfe3-08dc36a66d53
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC1.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6646

Hi Ayan,

On 23/02/2024 16:41, Ayan Kumar Halder wrote:
> Hi,
> 
> On 20/02/2024 12:17, Ayan Kumar Halder wrote:
>> From: Michal Orzel <michal.orzel@amd.com>
>>
>> Currently, if user enables HVC_DCC config option in Linux, it invokes access
>> to debug data transfer registers (i.e. DBGDTRTX_EL0 on arm64, DBGDTRTXINT on
>> arm32). As these registers are not emulated, Xen injects an undefined
>> exception to the guest and Linux crashes.
>>
>> To prevent this crash, introduce a partial emulation of DBGDTR[TR]X_EL0
>> (these registers share the same encoding) as RAZ/WI and MDCCSR_EL0 as TXfull.
>>
>> Refer ARM DDI 0487J.a ID042523, D19.3.8, DBGDTRTX_EL0
>> "If TXfull is set to 1, set DTRRX and DTRTX to UNKNOWN".
>>
>> Thus, any OS is expected to read MDCCSR_EL0 and check for TXfull before
>> using DBGDTRTX_EL0. Linux does it via hvc_dcc_init() ---> hvc_dcc_check(),
>> and returns -ENODEV in case TXfull bit is still set after writing a test
>> character. This way we prevent the guest from making use of HVC DCC as a
>> console.
>>
>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
>> Reviewed-by: Michal Orzel <michal.orzel@amd.com>
>> ---
>> Changes from
>>
>> v1 :- 1. DBGDTR_EL0 does not emulate RXfull. This is to avoid giving the OS any
>> indication that the RX buffer is full and is waiting to be read.
>>
>> 2. In Arm32, DBGOSLSR is emulated. Also DBGDTRTXINT is emulated at EL0 only.
>>
>> 3. Fixed the commit message and inline code comments.
>>
>> v2 :- 1. Split the patch into two (separate patches for arm64 and arm32).
>> 2. Removed the "fail" label.
>> 3. Fixed the commit message.
>>
>> v3 :- 1. "HSR_SYSREG_MDCCSR_EL0" emulation differs based on whether
>> partial_emulation_enabled is true or not.
>>
>> 2. If partial_emulation_enabled is false, then access to HSR_SYSREG_DBGDTR_EL0,
>> HSR_SYSREG_DBGDTRTX_EL0 would lead to undefined exception.
>>
>> v4 :- 1. Invoked "goto fail" from "default:" to ensure compliance with
>> MISRA 15.3.
>>
>>   xen/arch/arm/arm64/vsysreg.c         | 68 +++++++++++++++++++---------
>>   xen/arch/arm/include/asm/arm64/hsr.h |  3 ++
>>   2 files changed, 50 insertions(+), 21 deletions(-)
>>
>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>> index b5d54c569b..80918bc799 100644
>> --- a/xen/arch/arm/arm64/vsysreg.c
>> +++ b/xen/arch/arm/arm64/vsysreg.c
>> @@ -82,6 +82,7 @@ TVM_REG(CONTEXTIDR_EL1)
>>   void do_sysreg(struct cpu_user_regs *regs,
>>                  const union hsr hsr)
>>   {
>> +    const struct hsr_sysreg sysreg = hsr.sysreg;
>>       int regidx = hsr.sysreg.reg;
>>       struct vcpu *v = current;
>>   
>> @@ -159,9 +160,6 @@ void do_sysreg(struct cpu_user_regs *regs,
>>        *
>>        * Unhandled:
>>        *    MDCCINT_EL1
>> -     *    DBGDTR_EL0
>> -     *    DBGDTRRX_EL0
>> -     *    DBGDTRTX_EL0
>>        *    OSDTRRX_EL1
>>        *    OSDTRTX_EL1
>>        *    OSECCR_EL1
>> @@ -171,12 +169,42 @@ void do_sysreg(struct cpu_user_regs *regs,
>>        */
>>       case HSR_SYSREG_MDSCR_EL1:
>>           return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
>> +
>> +    /*
>> +     * Xen doesn't expose a real (or emulated) Debug Communications Channel
>> +     * (DCC) to a domain. Yet the Arm ARM implies this is not an optional
>> +     * feature. So some domains may start to probe it. For instance, the
>> +     * HVC_DCC driver in Linux (since f377775dc083 and at least up to v6.7),
>> +     * will try to write some characters and check if the transmit buffer
>> +     * has emptied.
>> +     */
>>       case HSR_SYSREG_MDCCSR_EL0:
>>           /*
>> +         * By setting TX status bit (only if partial emulation is enabled) to
>> +         * indicate the transmit buffer is full, we would hint the OS that the
>> +         * DCC is probably not working.
>> +         *
>> +         * Bit 29: TX full
>> +         *
>>            * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We emulate that
>>            * register as RAZ/WI above. So RO at both EL0 and EL1.
>>            */
>> -        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 0);
>> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 0,
>> +                                  partial_emulation ? (1U << 29) : 0);
>> +
>> +    case HSR_SYSREG_DBGDTR_EL0:
>> +    /* DBGDTR[TR]X_EL0 share the same encoding */
>> +    case HSR_SYSREG_DBGDTRTX_EL0:
>> +        /*
>> +         * Emulate as RAZ/WI (only if partial emulation is enabled) to prevent
>> +         * injecting undefined exception.
>> +         * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We emulate that
>> +         * register as RAZ/WI.
>> +         */
>> +        if ( !partial_emulation )
>> +            goto fail;
>> +        return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 0);
>> +
>>       HSR_SYSREG_DBG_CASES(DBGBVR):
>>       HSR_SYSREG_DBG_CASES(DBGBCR):
>>       HSR_SYSREG_DBG_CASES(DBGWVR):
>> @@ -394,26 +422,24 @@ void do_sysreg(struct cpu_user_regs *regs,
>>        * And all other unknown registers.
>>        */
>>       default:
>> -        {
>> -            const struct hsr_sysreg sysreg = hsr.sysreg;
>> -
>> -            gdprintk(XENLOG_ERR,
>> -                     "%s %d, %d, c%d, c%d, %d %s x%d @ 0x%"PRIregister"\n",
>> -                     sysreg.read ? "mrs" : "msr",
>> -                     sysreg.op0, sysreg.op1,
>> -                     sysreg.crn, sysreg.crm,
>> -                     sysreg.op2,
>> -                     sysreg.read ? "=>" : "<=",
>> -                     sysreg.reg, regs->pc);
>> -            gdprintk(XENLOG_ERR,
>> -                     "unhandled 64-bit sysreg access %#"PRIregister"\n",
>> -                     hsr.bits & HSR_SYSREG_REGS_MASK);
>> -            inject_undef_exception(regs, hsr);
>> -            return;
>> -        }
>> +        goto fail;
>>       }
>>   
>>       regs->pc += 4;
>> +    return;
>> +
>> + fail:
>> +
No need for this empty line.

>> +    gdprintk(XENLOG_ERR,
>> +             "%s %d, %d, c%d, c%d, %d %s x%d @ 0x%"PRIregister"\n",
>> +             sysreg.read ? "mrs" : "msr", sysreg.op0, sysreg.op1, sysreg.crn,
>> +             sysreg.crm, sysreg.op2, sysreg.read ? "=>" : "<=", sysreg.reg,
>> +             regs->pc);
The original formatting (i.e. placement of printk args) looked better. I'm not sure why you changed it.

>> +    gdprintk(XENLOG_ERR,
>> +             "unhandled 64-bit sysreg access %#"PRIregister"\n",
>> +             hsr.bits & HSR_SYSREG_REGS_MASK);
>> +    inject_undef_exception(regs, hsr);
>> +    return;
> 
> The last 'return' needs to be removed (spotted by Michal)
> 
~Michal



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:40:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:40:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685318.1065837 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWWo-0006Qq-3f; Mon, 26 Feb 2024 08:40:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685318.1065837; Mon, 26 Feb 2024 08:40:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWWn-0006QM-Vr; Mon, 26 Feb 2024 08:40:01 +0000
Received: by outflank-mailman (input) for mailman id 685318;
 Mon, 26 Feb 2024 08:40:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+ho1=KD=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1reWWm-0006Aw-NE
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:40:00 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a0098e7f-d482-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 09:40:00 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id C202E2252B;
 Mon, 26 Feb 2024 08:39:58 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 8FDA21329E;
 Mon, 26 Feb 2024 08:39:58 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id uF2hIV5O3GW5CQAAn2gu4w
 (envelope-from <jgross@suse.com>); Mon, 26 Feb 2024 08:39:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a0098e7f-d482-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708936798; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=3W5kxAVBkPCkGhzOvhislR9+9KRyj6IR2EuJkg2BVik=;
	b=mqJTJZyDA1u5I3Jd5LRNttTT+6MnG46+P0QcrEHZu0nypefAc9yQ6+EffLHsP2NnsHVhGE
	dqPLNYnK+5LlZvwR4aaHCY72R+HT0Yfr1IMOX08rQGQTkHLe0YTMYQrSu1HS9MAa/hsLXh
	hMLx8zHimU0inlLh9MYD2RgNDt4DiwI=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708936798; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=3W5kxAVBkPCkGhzOvhislR9+9KRyj6IR2EuJkg2BVik=;
	b=mqJTJZyDA1u5I3Jd5LRNttTT+6MnG46+P0QcrEHZu0nypefAc9yQ6+EffLHsP2NnsHVhGE
	dqPLNYnK+5LlZvwR4aaHCY72R+HT0Yfr1IMOX08rQGQTkHLe0YTMYQrSu1HS9MAa/hsLXh
	hMLx8zHimU0inlLh9MYD2RgNDt4DiwI=
From: Juergen Gross <jgross@suse.com>
To: minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org
Cc: samuel.thibault@ens-lyon.org,
	wl@xen.org,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH] Mini-OS: add symbol exports for xenstore stubdom
Date: Mon, 26 Feb 2024 09:39:55 +0100
Message-Id: <20240226083955.19350-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [4.90 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-0.00)[23.12%];
	 ARC_NA(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Level: ****
X-Spam-Score: 4.90
X-Spam-Flag: NO

Xenstore stubdom needs some more symbols exported.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 xenbus.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xenbus.c b/xenbus.c
index eb9af055..686428a4 100644
--- a/xenbus.c
+++ b/xenbus.c
@@ -45,6 +45,7 @@
 #endif
 
 struct xenstore_domain_interface *xenstore_buf;
+EXPORT_SYMBOL(xenstore_buf);
 static DECLARE_WAIT_QUEUE_HEAD(xb_waitq);
 DECLARE_WAIT_QUEUE_HEAD(xenbus_watch_queue);
 static __DECLARE_SEMAPHORE_GENERIC(xb_write_sem, 1);
@@ -70,6 +71,7 @@ static struct xenbus_req_info req_info[NR_REQS];
 static char *errmsg(struct xsd_sockmsg *rep);
 
 uint32_t xenbus_evtchn;
+EXPORT_SYMBOL(xenbus_evtchn);
 
 #ifdef CONFIG_PARAVIRT
 void get_xenbus(void *p)
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:40:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:40:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685320.1065850 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWWx-0007XQ-Gg; Mon, 26 Feb 2024 08:40:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685320.1065850; Mon, 26 Feb 2024 08:40:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWWx-0007XF-DM; Mon, 26 Feb 2024 08:40:11 +0000
Received: by outflank-mailman (input) for mailman id 685320;
 Mon, 26 Feb 2024 08:40:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reWWw-0006Aw-1F
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:40:10 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a60c76d4-d482-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 09:40:09 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-5655c7dd3b1so4358112a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:40:09 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 vx10-20020a170907a78a00b00a42eb84c7c2sm2177342ejc.142.2024.02.26.00.40.08
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 00:40:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a60c76d4-d482-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708936809; x=1709541609; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=38iolUXu8UDvXTv0mMIgiesYrJRJVFliE1KdkpJliH0=;
        b=dI220B4piRsi803pbq/Zun4ih4Or/rOorsrQMqDSrKJEKHNzGr6BhHqyL8WSUDtBbu
         3BSg39lxozxO6YENiKBKXo6DX/t0JY2aUngDe98P36o27BCQW1AyDGnj7Iig3guWwc7U
         pAWw86JN3bkpRfAQ4tFQacTIsU7HqetVTrl7jOLcgsst7z2uaKLAsPgNCfmviMDfHwE/
         vlbZAGuub1DG8PveViPK18znzblJo7iiPoSGHUNRpLDcCsnZwAAKm/PtjlLI3ibxwttv
         q/sSmfYcg38a/y1jyzj770CafMZvaw0M/M+cdNs/3JSXLZzWOmo0X2+OQoavSi0RQAQz
         64ow==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708936809; x=1709541609;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=38iolUXu8UDvXTv0mMIgiesYrJRJVFliE1KdkpJliH0=;
        b=blhLivizcErrHVyveBiGKUx0LK4Ycq4HZGNFxfs0kVnhrwek64DUGr8iQRVm3+nWQ+
         CT+KgS1//b0MitZ1I1Rt6agMKX6cWj+li7UD3C2CkeioTtRnHmQNpgCc8gCc8Mw8DMVU
         XayIJ5/E/ff2GSEAntuabwLUysiWPj9iU5gaSQQxRzU/BjKZqh6WR+O9tlW2h8W2s6Wj
         HR/L1urIx5iFTeAyHXtnmBKajvHaT3P1qiQuoceSKNOaDsjNowvHAYnzN1Q3R2ycz9qH
         gwssyDgPsI1/8Xva9MmeGVVMfdba+XdyR1VBskpHO5Czgi0hdn2WrStoRBwOsA5eqZNq
         r75A==
X-Forwarded-Encrypted: i=1; AJvYcCWtdYbWHyShoErIjGgR8t09Wv6upoFCVrmwzjmmSb9eY6RRddS+/ChJazMbEDa5ET5WKyEFbBql1+s06W5UUDRj391MZfQUQh0xUcPdsjI=
X-Gm-Message-State: AOJu0YwM8UXv5LVDBrX8sn9XcrJzTbD3Qpjtv61Q4YCnEDpl9EqvgucZ
	nNG7HfDgbtJR1ryN5zNpd7rycdXVeu1Ewn+P4TkYf0Sp0crxojlCGHTJV8PKVA==
X-Google-Smtp-Source: AGHT+IGScEBY6a7w8BdWiRyzxrjYi6TjnBY2cS+lpDf5hg9zCxwXfy0s3UrvcfElqm063CEeAcZTog==
X-Received: by 2002:a17:906:1159:b0:a3e:feee:3a5b with SMTP id i25-20020a170906115900b00a3efeee3a5bmr5300441eja.3.1708936808854;
        Mon, 26 Feb 2024 00:40:08 -0800 (PST)
Message-ID: <217c07b8-25af-4f7a-81d8-09485d72fb05@suse.com>
Date: Mon, 26 Feb 2024 09:40:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/3] x86/spec: fix INDIRECT_THUNK option to only be set
 when build-enabled
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240223120616.73503-1-roger.pau@citrix.com>
 <20240223120616.73503-4-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240223120616.73503-4-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.02.2024 13:06, Roger Pau Monne wrote:
> --- a/xen/arch/x86/spec_ctrl.c
> +++ b/xen/arch/x86/spec_ctrl.c
> @@ -239,6 +239,7 @@ static int __init cf_check parse_spec_ctrl(const char *s)
>          /* Xen's speculative sidechannel mitigation settings. */
>          else if ( !strncmp(s, "bti-thunk=", 10) )
>          {
> +#ifdef CONFIG_INDIRECT_THUNK
>              s += 10;
>  
>              if ( !cmdline_strcmp(s, "retpoline") )

              if ( !IS_ENABLED(CONFIG_INDIRECT_THUNK) )
              {
                  no_config_param("INDIRECT_THUNK", "spec-ctrl", s, ss);
                  rc = -EINVAL;
              }
              else if ( !cmdline_strcmp(s, "retpoline") )

?

Also, while touching INDIRECT_THUNK, could I talk you into also adjusting
documentation accordingly? It presently mentions INDIRECT_THUNK, when
really it means CONFIG_INDIRECT_THUNK.

Jan

> @@ -249,6 +250,10 @@ static int __init cf_check parse_spec_ctrl(const char *s)
>                  opt_thunk = THUNK_JMP;
>              else
>                  rc = -EINVAL;
> +#else
> +            no_config_param("INDIRECT_THUNK", "spec-ctrl", s, ss);
> +            rc = -EINVAL;
> +#endif
>          }
>  
>          /* Bits in MSR_SPEC_CTRL. */



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:40:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:40:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685327.1065861 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWXK-0008Gf-P8; Mon, 26 Feb 2024 08:40:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685327.1065861; Mon, 26 Feb 2024 08:40:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWXK-0008GY-Lu; Mon, 26 Feb 2024 08:40:34 +0000
Received: by outflank-mailman (input) for mailman id 685327;
 Mon, 26 Feb 2024 08:40:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reWXJ-0006Aw-GG
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:40:33 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b4265e3d-d482-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 09:40:32 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-512fce83ab5so747857e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:40:32 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 u23-20020a05620a121700b00787c6703539sm1911883qkj.90.2024.02.26.00.40.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 00:40:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b4265e3d-d482-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708936832; x=1709541632; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=5kyqQoWbLZsYMyfSMJ/kNXUWlgNoyKQjlC4kJIst9os=;
        b=B49y7yawvvSuUVx/gb899sgfC0Yz6DhcMWlWNWxyYTDGEtYy1VsoRxkDHh0oj5jytb
         Q2v7/zlxmMP0jSbE6wmVScM0YvXMCYqexiTM7y4kY/LErkGiLHYc7L3SrF2gPrZlbuIi
         GFlhnvZRpNbk+vfNOFn2icqxcjSeqxF6/iWUE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708936832; x=1709541632;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=5kyqQoWbLZsYMyfSMJ/kNXUWlgNoyKQjlC4kJIst9os=;
        b=TGIT4Ya1ZUSCurmM7dnoQL0HnHPp6xe/ku1fP+U0DpITNzyoE31GS2I7OfhFo/n8es
         BtMyLaX2aph7klMlj82TctdWaieX61iNLRwziu2BnZA1YYDwYvq9mLRKS6gjFSCF+mvL
         ahjfRp+UR7WFV8+gZNvBzj9KjJldCBgMh0dZCOK4dUNtd7zI5aSAtyL0DIdhNyl1taVQ
         c/DsC2xh3f3Owog89CItM/jNZ2XdOFjAA9EqmY1VE5T2Q4nUuVtcZV47Y+cpvrZqFfYE
         EJpdThnC7IStNjaWcmm9ZrQYQ4Z1MRHFQVXVtDggwBz5tr0CqNI05dzbPEQPjJOZIJxS
         WO8g==
X-Forwarded-Encrypted: i=1; AJvYcCXCadjba/3ukGIYp2qLRFXVFQu4m5SgZ0/9Rj2pECN3zhdOImISPjEU1dYF01aTZnqRnILMOqTgYPGCLFT6MPXvXdLBStaxf5tzGu2mpDM=
X-Gm-Message-State: AOJu0Yzr+819Pwo3kuMckaQ6kdeoKes3Y0R/bxgGrr+Y9ejkckXTyIir
	syYwetRGZWMO6aUIh5T3Xyg20N6h6ZJt7fTQ+viNH6M9nFfv9lRSBWOLarZcpsc=
X-Google-Smtp-Source: AGHT+IGYlrGjJgARVjSOmRn+owZS/BLi7nJzvRHrKvzbIiXhVhwvFkI6za/oBbroS3dcjufQP/ehwQ==
X-Received: by 2002:a05:6512:3a8e:b0:512:9c37:97fc with SMTP id q14-20020a0565123a8e00b005129c3797fcmr4564109lfu.57.1708936832336;
        Mon, 26 Feb 2024 00:40:32 -0800 (PST)
Date: Mon, 26 Feb 2024 09:40:17 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH 0/5] xen/livepatch: fixes for the pre-apply / post-revert
 hooks
Message-ID: <ZdxOcc1O35lkTS4L@macbook>
References: <20231130142944.46322-1-roger.pau@citrix.com>
 <6143d253-1d5c-48c8-81b9-3652b21dd9c0@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <6143d253-1d5c-48c8-81b9-3652b21dd9c0@suse.com>

On Fri, Feb 23, 2024 at 11:48:10AM +0100, Jan Beulich wrote:
> On 30.11.2023 15:29, Roger Pau Monne wrote:
> > Hello,
> > 
> > The follow series contain a misc of fixes mostly related to the usage of
> > the pre-apply / post-revert hooks.  The norevert test is also fixed to
> > work as I think was expected.  Finally both the no{apply,revert}
> > tests are fixed to build properly, as the files where previously
> > unhooked from the build system completely.
> > 
> > I'm unsure how useful the apply and revert hooks really are, as without
> > calling the internal apply/revert functions the state of the payload
> > structure is quite likely inconsistent with the code expectations.
> > 
> > Thanks, Roger.
> > 
> > Roger Pau Monne (5):
> >   xen/livepatch: register livepatch regions when loaded
> >   xen/livepatch: search for symbols in all loaded payloads
> >   xen/livepatch: fix norevert test attempt to open-code revert
> >   xen/livepatch: fix norevert test hook setup typo
> >   xen/livepatch: properly build the noapply and norevert tests
> 
> With the R-b-s that have arrived, could you clarify in how far these
> can be committed out of order?

All can be applied out of order, except for the last one, that's
"xen/livepatch: properly build the noapply and norevert tests" as
applying that will enable the tests in osstest, and those will fail
without all the previous adjustments.

I was about to send v2 with the requested changes, but will wait for
you to commit 2/5 and 4/5.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:43:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:43:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685335.1065871 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWZh-0000WU-3e; Mon, 26 Feb 2024 08:43:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685335.1065871; Mon, 26 Feb 2024 08:43:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWZh-0000WN-14; Mon, 26 Feb 2024 08:43:01 +0000
Received: by outflank-mailman (input) for mailman id 685335;
 Mon, 26 Feb 2024 08:42:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reWZf-0000WH-71
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:42:59 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0a2eab40-d483-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 09:42:57 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-5645960cd56so2998902a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:42:57 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g11-20020a056402428b00b00563c63e0a13sm2142158edc.49.2024.02.26.00.42.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 00:42:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0a2eab40-d483-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708936977; x=1709541777; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=n8XnJrbYiGVhwjiz9dh3tUKATEFLWFWO7YbGjz2i+8o=;
        b=LWpNWjlxeoegA0kuhmvKnCvmUDaot+mCmtbLlzd7ZdVs26txpItpyQCwlYsYcsrdDM
         Vu2iR6ZIRssyrAx8O3PBPWT0YP9LPKu7nS2cXJJ9S2Q9VThnJ73HHs1foA7ZYnQkdS2P
         x335/nUoRfJsN5XPChlesYtfPXygk8zFZiMYY9s8NIT3mFDJMMSFPcP4ynoVpstX3gWs
         WNnhWldwXG11J54aha4hVhSSYA+MPdkvulfvoKGolGNLZLFVbYM25qrpjfsErUNxrGfO
         yHufVc9/+sB1vIsUh6ubUccuVo2U68akqaV5HP7oczbfDu8upL1KB+9CxfiafYJN1LzN
         amYA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708936977; x=1709541777;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=n8XnJrbYiGVhwjiz9dh3tUKATEFLWFWO7YbGjz2i+8o=;
        b=KbL/eaYMHJPABMswCP1ikYd4UGK8YbqFaU/AQ+5sEX/QsduhB4lOqF+8AfihwgUq3j
         8JH05V6+mG2y5mQTqZOT/2y5GIRl8scQsgKov+9nukfsFhQc9aDdLaojN7fJ6AAI4p3a
         5lv6FoQjfGH+P5V1tEtQzUh27qZNZmbxhaHQcIfI5IcoESl8a2hCXXCKoHJrC7GZPD1X
         yC0bdDV6KwsfqWMdObjPkTmTLwc2G2N6uuXmRPbNSoJRJNjgPILoJp2v6eJQQlEBnqB8
         Hv2V/DFx4R7Ge+VquYh6s1ImiwXfNHSA5ou5KHcr3R/14hRC7je3kqygYcl2S3RVjQ81
         mlYA==
X-Forwarded-Encrypted: i=1; AJvYcCWtVbno0970fjgEG31fHr5jP5kny6eJ/1y6kHElUa2hhBfQYUHha3l4MCR66s6R++ykcQEOwS8p/QJETg5AdT8i+UUGUG0nWohDGN/70Vo=
X-Gm-Message-State: AOJu0Yz0q32iE6DwS47cmmArBjNnVUPmPGlq47GcYxntmxbpYqfZMjvg
	+mzhr0a/3TeDmThAZJVKX9UZiALO3Wqi2QgPeoSLNHPhD6+l0Ko5BKTxZ7rcsg==
X-Google-Smtp-Source: AGHT+IFRv4Y5Seek+nyh0tcj4ScD3OqFEvuSWi+Odih1gUvFMOSOg4nF7t/4ohr6O3FgE2DTc/DBZg==
X-Received: by 2002:aa7:d513:0:b0:564:aa72:78e1 with SMTP id y19-20020aa7d513000000b00564aa7278e1mr3839575edq.9.1708936976804;
        Mon, 26 Feb 2024 00:42:56 -0800 (PST)
Message-ID: <3dcb9afe-7f3f-495c-b182-b4242b4cb337@suse.com>
Date: Mon, 26 Feb 2024 09:42:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/3] x86/spec: fix BRANCH_HARDEN option to only be set
 when build-enabled
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240223120616.73503-1-roger.pau@citrix.com>
 <20240223120616.73503-3-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240223120616.73503-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.02.2024 13:06, Roger Pau Monne wrote:
> --- a/xen/arch/x86/spec_ctrl.c
> +++ b/xen/arch/x86/spec_ctrl.c
> @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
>  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
>  int8_t __read_mostly opt_eager_fpu = -1;
>  int8_t __read_mostly opt_l1d_flush = -1;
> -static bool __initdata opt_branch_harden = true;
> +static bool __initdata opt_branch_harden =
> +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
>  
>  bool __initdata bsp_delay_spec_ctrl;
>  uint8_t __read_mostly default_xen_spec_ctrl;
> @@ -268,7 +269,14 @@ static int __init cf_check parse_spec_ctrl(const char *s)
>          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
>              opt_l1d_flush = val;
>          else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> +        {
> +#ifdef CONFIG_SPECULATIVE_HARDEN_BRANCH
>              opt_branch_harden = val;
> +#else
> +            no_config_param("SPECULATIVE_HARDEN_BRANCH", "spec-ctrl", s, ss);
> +            rc = -EINVAL;
> +#endif
> +        }
>          else if ( (val = parse_boolean("srb-lock", s, ss)) >= 0 )
>              opt_srb_lock = val;
>          else if ( (val = parse_boolean("unpriv-mmio", s, ss)) >= 0 )

While looking at patch 3 I noticed another inconsistency: Wouldn't
"Compiled-in support:" better also enumerate HARDEN_BRANCH then, just
like for thunks both CONFIG_* state and actual runtime choice are
logged? Or alternatively, should logging of thunk runtime choice be
suppressed when the Kconfig setting is off?

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:44:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:44:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685337.1065881 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWao-00014O-Dr; Mon, 26 Feb 2024 08:44:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685337.1065881; Mon, 26 Feb 2024 08:44:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWao-00014H-Ai; Mon, 26 Feb 2024 08:44:10 +0000
Received: by outflank-mailman (input) for mailman id 685337;
 Mon, 26 Feb 2024 08:44:08 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reWam-000144-Ty
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:44:08 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 33c06fab-d483-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 09:44:07 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-565c6cf4819so1749832a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:44:07 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g11-20020a056402428b00b00563c63e0a13sm2142158edc.49.2024.02.26.00.44.06
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 00:44:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 33c06fab-d483-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708937046; x=1709541846; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OP4pR8KqizECVTvElPxzBJT/Os7DU3v48TUawUk6aGc=;
        b=JKckzQ4HuWg+Dj4P6rDLHl5hBK5JP6hsFbagH1umh1quX5GPHLOPuzSJJ4K0rrGVNQ
         UqeblC3XED1OgW5wYTtFUmrdTSATjTRWAg5hTkpxuBZpgjlybEMNKc/EdLi5kSOg/ZIq
         plB5kPJwveIIepuxaTMEi0x29Ta63uYt994157xvUbMCs1o5HFy75lXCRniqDFP5RnmM
         TEu/XJKyCQucoLavjKNEeCkWIXe/fIOcLWt08Tl7fU2w63qKwWo8WcecxJzFprAcFjR7
         /8MPmAHsMGIIPqCNTN4kjFgPEic/H/Q9HWe1XhIEaDBKg7Cv65f5lUQeiwey1/BP9opw
         I9lw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708937046; x=1709541846;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OP4pR8KqizECVTvElPxzBJT/Os7DU3v48TUawUk6aGc=;
        b=rF+h6rTwftTJsEP89qqOEH1ClGiwsc4h9OcV+grqRUuUR9BCqf+exWtzGpgRvj1psz
         QBZrMlX6sccyURW91mVDr+c6gWr5oNTUCyFRmKkmaft5ilknOwgEDq1A1hddFkuKMSXk
         P30S2Lo1fxz1oAPhUrQwBpjvAlMYWYvSxGpr+8ZIPwNXzNefAGhvV4ecAah996/rTkoH
         CiUKheu9c+eUznFislVxZMqov7rB7Uathws+/8SDJtbuEl2giLm+1nBXy8t9zrxCthNs
         zhR+ydPEf+K5qABq1IjQayKdm7bNb5XLrFJDyrYaSCgfj5S1jixaskXlb/qBWXUBwjm/
         a5fA==
X-Forwarded-Encrypted: i=1; AJvYcCV8P8eUFWb0S3BcRmz7SWFo0XLmWWJNmXlghiHKMWMxBnEIr2ln2+y3tRS4R1IPtXkIyqf/cXSwklRKQcD+qEwfS2yyQ7W5BLuIskeZJrA=
X-Gm-Message-State: AOJu0Yx8GTnMg3Z07DeGnm/wr+m8d1cFhlM1XQsFGenlYMW60CLlAC1/
	fLkpmM35ApDB7QD8QasA/Ivu9DThLwObhtHwDm3ep3CXne1UbR7ZtKFm/pOaQA==
X-Google-Smtp-Source: AGHT+IHqs+UyzSnNMpduXmnnZuCpApjmzvij2yXJjO68aOqEbXWcMHYoJJO2Ub2jEES3v6CLIwU7QQ==
X-Received: by 2002:a05:6402:5389:b0:563:c2e5:5289 with SMTP id ew9-20020a056402538900b00563c2e55289mr5010620edb.13.1708937046614;
        Mon, 26 Feb 2024 00:44:06 -0800 (PST)
Message-ID: <707666cc-914e-42da-b823-fdc5a38271be@suse.com>
Date: Mon, 26 Feb 2024 09:44:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: add symbol exports for xenstore stubdom
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: samuel.thibault@ens-lyon.org, wl@xen.org,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org
References: <20240226083955.19350-1-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226083955.19350-1-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 09:39, Juergen Gross wrote:
> Xenstore stubdom needs some more symbols exported.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Any Reported-by: possibly applicable here?

Jan

> --- a/xenbus.c
> +++ b/xenbus.c
> @@ -45,6 +45,7 @@
>  #endif
>  
>  struct xenstore_domain_interface *xenstore_buf;
> +EXPORT_SYMBOL(xenstore_buf);
>  static DECLARE_WAIT_QUEUE_HEAD(xb_waitq);
>  DECLARE_WAIT_QUEUE_HEAD(xenbus_watch_queue);
>  static __DECLARE_SEMAPHORE_GENERIC(xb_write_sem, 1);
> @@ -70,6 +71,7 @@ static struct xenbus_req_info req_info[NR_REQS];
>  static char *errmsg(struct xsd_sockmsg *rep);
>  
>  uint32_t xenbus_evtchn;
> +EXPORT_SYMBOL(xenbus_evtchn);
>  
>  #ifdef CONFIG_PARAVIRT
>  void get_xenbus(void *p)



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 08:56:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 08:56:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685346.1065894 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWmU-0003kS-Fp; Mon, 26 Feb 2024 08:56:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685346.1065894; Mon, 26 Feb 2024 08:56:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWmU-0003kL-DD; Mon, 26 Feb 2024 08:56:14 +0000
Received: by outflank-mailman (input) for mailman id 685346;
 Mon, 26 Feb 2024 08:56:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reWmT-0003kF-4M
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 08:56:13 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e3d08299-d484-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 09:56:12 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a3e6f79e83dso289690266b.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 00:56:12 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 gq9-20020a170906e24900b00a3fa275de38sm2191955ejb.162.2024.02.26.00.56.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 00:56:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e3d08299-d484-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708937771; x=1709542571; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=aaaGjeO8xh1J9ErIche9rNbsmQTIfaYVfGjYrUdriGA=;
        b=XFXxAX7tpilQAnnmXCFHVYWEb0bMOQTqTWA/wVUwG6LilO72vAUEyVWAedftK8qfpo
         gFpiTPbi0tY2lUAcHb7tqRE8k4+X0RPCnfxO4uNSAFJiYnMqKzSnfIc/If3eOsyKMqCF
         Et0V920vTQ7ozrrmDvK8Xg1nW2rGO/KIdPDv2EFd9NZw0P2JV+PtT82A4udJuTAfIKm+
         D/U4e+2o/E1Lpc/HSSEHIbjBJ/JKQOqqmaolazLomO+qYywAm5qYLAjJci0GD5nqJ1Vz
         J2cs+wq/LLlZKRvkY42I3OS+pWusRprwxjIL5M//im4c+5DQSFgINcImojs0nSnHoZH7
         g6CQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708937771; x=1709542571;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=aaaGjeO8xh1J9ErIche9rNbsmQTIfaYVfGjYrUdriGA=;
        b=c2S6T2sfTgU7N2xue99D7zcI5hQMeRw9vPB+HMFOJMDNjTXD4+16CUMULz1zHvI1QI
         eyhGpKbWJ3B9durI2JLoZBBvrEtQe/NCV4Af0aOsGFdZWuYUu9bpSn2febhJqK57SbKP
         U/C3larMKfbicJ7LILI4Y93rYFik0QIz14bDMcRXyqIFg3kfcbtdHZmQZIxhPBU8y6nO
         gL7Mla2SQSA3mEU1lrjN70+g8xh+I19ZMSBD9YNZiw2N0yJIBqGEnZEGfsGCc1FQOKzc
         Sj/E8lMMrqZtVWelAfaTm+5RZBK9pdKWq1dhVlt4vXZMoRGDysX8rbdss2pGxdIb816Q
         KL/g==
X-Gm-Message-State: AOJu0YydquSbYCqC1FebQRyN28k/VWrZozWpP9tNvQ0OLmu715TDFnV7
	2q/PkOB9B6Sg1J0VkrxgpPAj2Ar9J/wzVrqElbRfJOjRRcqIkPMD3NaoD7LiNw==
X-Google-Smtp-Source: AGHT+IFbsnxpH6q8bfHneMlQcWCOOQlhB9sc10k+rfJz7RCKL0Pc5AP37U43GKnoijWrpAAGDXZSWA==
X-Received: by 2002:a17:906:d923:b0:a41:30be:4a82 with SMTP id rn3-20020a170906d92300b00a4130be4a82mr3404498ejb.61.1708937771405;
        Mon, 26 Feb 2024 00:56:11 -0800 (PST)
Message-ID: <a8bab8eb-a735-4440-a075-7c62a12d4e4d@suse.com>
Date: Mon, 26 Feb 2024 09:56:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 1/2] xen/console: drop return value from
 consoled_guest_rx/tx
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <4998ec735bd7e5a50a229507e2b92ae56ec1ba4b.1708680104.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2402231456350.754277@ubuntu-linux-20-04-desktop>
 <004877c8-9aea-44b9-aa93-65665b0d3a8d@suse.com>
 <b417d7a330addd295b3cbbbac4bed2a4@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <b417d7a330addd295b3cbbbac4bed2a4@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 09:23, Nicola Vetrini wrote:
> On 2024-02-26 09:00, Jan Beulich wrote:
>> On 23.02.2024 23:56, Stefano Stabellini wrote:
>>> On Fri, 23 Feb 2024, Nicola Vetrini wrote:
>>>> These functions never saw a usage of their return value since
>>>> they were introduced, so it can be dropped since their usages
>>>> violate MISRA C Rule 17.7:
>>>> "The value returned by a function having non-void return type shall 
>>>> be used".
>>>>
>>>> No functional change.
>>>>
>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>>
>>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>>
>> The cleanup is certainly okay, but would one of you mind clarifying in 
>> how
>> far this code is relevant for certification? I don't expect there are 
>> plans
>> to run shim Xen in any projected production uses for which 
>> certification is
>> relevant? (The subject prefix is also unnecessarily wide here, when 
>> it's
>> only daemon code which is affected, not console code in general.)
>>
> 
> I agree on the subject prefix being too wide. The configuration that 
> uses consoled_guest_tx is #ifdef-ed for x86, so even in configurations 
> that may never reach this condition this is relevant, unless its #ifdef 
> is restricted to cases where the call may actually be reachable.

Hmm, I see. There are contradicting goals here then: It being just X86 is
to reduce the risk of someone overlooking a build breakage they may
introduce. Whereas for certification it's quite the other way around: We'd
like to "hide" as much code as possible.

Really I would have been inclined to suggest to drop the #ifdef, if
possible even without replacing by IS_ENABLED(), but instead leveraging
that pv_shim ought to be compile-time false whenever CONFIG_PV_SHIM=n.
After all that's a pattern we've been trying to follow. But with your
observation is becomes questionable whether extending use of IS_ENABLED()
is actually going to be helpful. Stefano - perhaps something to discuss
on one of the next meetings?

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:05:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:05:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685349.1065904 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWvb-0006Ki-EZ; Mon, 26 Feb 2024 09:05:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685349.1065904; Mon, 26 Feb 2024 09:05:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWvb-0006Kb-C2; Mon, 26 Feb 2024 09:05:39 +0000
Received: by outflank-mailman (input) for mailman id 685349;
 Mon, 26 Feb 2024 09:05:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reWva-0006KR-Jl; Mon, 26 Feb 2024 09:05:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reWva-000803-F4; Mon, 26 Feb 2024 09:05:38 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reWva-00041m-1e; Mon, 26 Feb 2024 09:05:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1reWva-0007WA-0l; Mon, 26 Feb 2024 09:05:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=jDLYjMdv/e6zw/Z3H0NVQIOyT9gWyTPO+8DTgJVYUb4=; b=4KBb23/Yp4gp6Rz0Y9100Dvzzz
	TVgF9JkPrtEu+xe9dP/wv8peESRgZ14jTXEXoQxtsBM4yTuduTjWp5QVSQl4X/jnzyJI+NQTmeH6h
	gZVLOzvBnLMrMubNyzMKnuHYu6jDwyccjLPNCFIZlk7/9+/xRkFShdnpsapVn2ULlvfM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184762-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184762: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=d206a76d7d2726f3b096037f2079ce0bd3ba329b
X-Osstest-Versions-That:
    linux=4ca0d9894fd517a2f2c0c10d26ebe99ab4396fe3
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 26 Feb 2024 09:05:38 +0000

flight 184762 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184762/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184760
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184760
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184760
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184760
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184760
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184760
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184760
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184760
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                d206a76d7d2726f3b096037f2079ce0bd3ba329b
baseline version:
 linux                4ca0d9894fd517a2f2c0c10d26ebe99ab4396fe3

Last test of basis   184760  2024-02-25 18:12:13 Z    0 days
Testing same since   184762  2024-02-26 01:13:54 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Aaro Koskinen <aaro.koskinen@iki.fi>
  Akira Yokosawa <akiyks@gmail.com>
  Akira Yokosawa <akiyks@gmail.com>  # Sphinx 4.3.2, 5.3.0 and 6.2.1
  Andreas Larsson <andreas@gaisler.com>
  Arnd Bergmann <arnd@arndb.de>
  Brian Foster <bfoster@redhat.com>
  Chen Jun <chenjun102@huawei.com>
  Dave Hansen <dave.hansen@linux.intel.com>
  Frank Li <Frank.Li@nxp.com>
  Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  Guenter Roeck <linux@roeck-us.net>
  Heikki Krogerus <heikki.krogerus@linux.intel.com>
  Jonathan Corbet <corbet@lwn.net>
  Kent Overstreet <kent.overstreet@linux.dev>
  Krishna Kurapati <quic_kriskura@quicinc.com>
  Lino Sanfilippo <l.sanfilippo@kunbus.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Marc Zyngier <maz@kernel.org>
  Marek Szyprowski <m.szyprowski@samsung.com>
  Mark Brown <broonie@kernel.org>
  Nam Cao <namcao@linutronix.de>
  Oliver Upton <oliver.upton@linux.dev>
  Ondrej Jirman <megi@xff.cz>
  Palmer Dabbelt <palmer@rivosinc.com>
  Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
  Pawel Laszczak <pawell@cadence.com>
  Peter Chen <peter.chen@kernel.org>
  Sean Christopherson <seanjc@google.com>
  Shanker Donthineni <sdonthineni@nvidia.com>
  Thinh Nguyen <Thinh.Nguyen@synopsys.com>
  Thomas Gleixner <tglx@linutronix.de>
  Vegard Nossum <vegard.nossum@oracle.com>
  Vidya Sagar <vidyas@nvidia.com>
  Xu Yang <xu.yang_2@nxp.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   4ca0d9894fd51..d206a76d7d272  d206a76d7d2726f3b096037f2079ce0bd3ba329b -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:09:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:09:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685355.1065914 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWzM-000738-Vn; Mon, 26 Feb 2024 09:09:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685355.1065914; Mon, 26 Feb 2024 09:09:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reWzM-000731-Sm; Mon, 26 Feb 2024 09:09:32 +0000
Received: by outflank-mailman (input) for mailman id 685355;
 Mon, 26 Feb 2024 09:09:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reWzL-00072v-N5
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:09:31 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bf4d85b2-d486-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 10:09:29 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-51197ca63f5so4261206e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 01:09:29 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 h11-20020ac8744b000000b0042e5e5795a7sm2249644qtr.51.2024.02.26.01.09.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 01:09:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf4d85b2-d486-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708938569; x=1709543369; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=6Ig/CUKmBIDvrweHGRl9Xe1rMFoZ+kaM5fAYRsxMzWU=;
        b=XkW14n5KhfOTJVP+MY3zfEqWb90vecd1G7pK5cykpQS6mJM10jQ/jQt/5tYQAd2H6V
         ci5zQ6UQnwP3HbG9cVVtXMpU+irDpP7ukQ5kd6VkgtSe42NnxnCSVH3Vptu8BbgdbQpf
         KGyquKro5S/DHBGo97AlYiaJ0aXiSVxCaKNAU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708938569; x=1709543369;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=6Ig/CUKmBIDvrweHGRl9Xe1rMFoZ+kaM5fAYRsxMzWU=;
        b=Yy3nolLwvbwnQyZUX5fOSjBtHKPDgFPAiLmv1d2dxlj01hXkA+/oQb262cIe5gmzit
         juzeFpbn1wO7ir+BqU0ffk9DETrlLynOLnUaLdsDOq2DXCNnK2g8C0/mJxQwormMpRw8
         WqzxrLDCwL0PyOr4oq1kG6R5wlNdFNyw4pffRiIXb5KjVZjHQwJiXmkI23BHdkwiYDUK
         Urrw3rhSnGRUkFsSajulJrcNxBMVNaF0+D4WUPwF3AF22sAxuFOqJO/LNjz3vel1GBHW
         05Uf1/SQIPEVAr3BsfffBL2lIhNIFlAjWcJFqwavtcQuQkoGZTu0u9QFBxTjeHu6wDRh
         fgeg==
X-Forwarded-Encrypted: i=1; AJvYcCXDbmUl4Ejg0TsiCtCYxmbGIelRu/6Ec70505qGFDOjwmJHS+47mBY1FjoJz9a2JgW0hvRA+XjxgfOJ2sg6NtXjrFXBKUkuCN5Pnzy+yZ4=
X-Gm-Message-State: AOJu0YxH23t9+WyQt8Mqx9m74DyY8k3IGs4dzjydyH66rNrKjjRIubZ4
	YokHUgHy5hZtxh0TSY+CXaaZ9/kb3IGs1cfRo70nkL+n+eMK35ZztlhdZoVpLho=
X-Google-Smtp-Source: AGHT+IHE5f9Kb6QXEV4K5Cf0DgQt5lHgPnS7mv5PDNrIK42ZcAz18bI9AOOlAkRnya67oXJDsg64Ow==
X-Received: by 2002:a05:6512:3c9e:b0:512:fe5c:4bae with SMTP id h30-20020a0565123c9e00b00512fe5c4baemr1358705lfv.31.1708938569151;
        Mon, 26 Feb 2024 01:09:29 -0800 (PST)
Date: Mon, 26 Feb 2024 10:09:21 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v2 2/3] x86/spec: fix BRANCH_HARDEN option to only be set
 when build-enabled
Message-ID: <ZdxVQRMRyQOEc581@macbook>
References: <20240223120616.73503-1-roger.pau@citrix.com>
 <20240223120616.73503-3-roger.pau@citrix.com>
 <3dcb9afe-7f3f-495c-b182-b4242b4cb337@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <3dcb9afe-7f3f-495c-b182-b4242b4cb337@suse.com>

On Mon, Feb 26, 2024 at 09:42:58AM +0100, Jan Beulich wrote:
> On 23.02.2024 13:06, Roger Pau Monne wrote:
> > --- a/xen/arch/x86/spec_ctrl.c
> > +++ b/xen/arch/x86/spec_ctrl.c
> > @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
> >  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
> >  int8_t __read_mostly opt_eager_fpu = -1;
> >  int8_t __read_mostly opt_l1d_flush = -1;
> > -static bool __initdata opt_branch_harden = true;
> > +static bool __initdata opt_branch_harden =
> > +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
> >  
> >  bool __initdata bsp_delay_spec_ctrl;
> >  uint8_t __read_mostly default_xen_spec_ctrl;
> > @@ -268,7 +269,14 @@ static int __init cf_check parse_spec_ctrl(const char *s)
> >          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
> >              opt_l1d_flush = val;
> >          else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> > +        {
> > +#ifdef CONFIG_SPECULATIVE_HARDEN_BRANCH
> >              opt_branch_harden = val;
> > +#else
> > +            no_config_param("SPECULATIVE_HARDEN_BRANCH", "spec-ctrl", s, ss);
> > +            rc = -EINVAL;
> > +#endif
> > +        }
> >          else if ( (val = parse_boolean("srb-lock", s, ss)) >= 0 )
> >              opt_srb_lock = val;
> >          else if ( (val = parse_boolean("unpriv-mmio", s, ss)) >= 0 )
> 
> While looking at patch 3 I noticed another inconsistency: Wouldn't
> "Compiled-in support:" better also enumerate HARDEN_BRANCH then, just
> like for thunks both CONFIG_* state and actual runtime choice are
> logged?

Yes, I guess we would also need to expand "Compiled-in support:" to
include HARDEN_ARRAY and HARDEN_GUEST_ACCESS.

> Or alternatively, should logging of thunk runtime choice be
> suppressed when the Kconfig setting is off?

Hm, I think printing "BTI-Thunk N/A" is good enough when the thunk has
been built time disabled.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:13:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:13:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685359.1065924 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reX3b-0000kC-Fw; Mon, 26 Feb 2024 09:13:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685359.1065924; Mon, 26 Feb 2024 09:13:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reX3b-0000k5-DT; Mon, 26 Feb 2024 09:13:55 +0000
Received: by outflank-mailman (input) for mailman id 685359;
 Mon, 26 Feb 2024 09:13:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1reX3Z-0000jz-UU
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:13:53 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1reX3Y-00087u-QB; Mon, 26 Feb 2024 09:13:52 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.240])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1reX3Y-0007rm-Ff; Mon, 26 Feb 2024 09:13:52 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=mBCccafeyLjXPKhV1rhlIp1nob8ZVADQEsxcj9gJQxY=; b=y0WPR6xzV+FnF46w89y9Qy+pbZ
	a99+tdYWDLvX0/gySOit11OxZJgI79EL+ORTAbWqvL6OflFxJSRAQudalYRAiN8yfnY4U7DQDxZvm
	eIyxmx5PZ2+KpFQBibmdJfxB5sNEGPNjN/3YJhmlCQxeucqqiSsJtg0Ye2j/ie6JGcVo=;
Message-ID: <d1518124-483c-4409-9b36-6a3392378911@xen.org>
Date: Mon, 26 Feb 2024 09:13:49 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Alec Kwapis <alec.kwapis@medtronic.com>
References: <20240226011935.169462-1-xin.wang2@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240226011935.169462-1-xin.wang2@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

Welcome back!

On 26/02/2024 01:19, Henry Wang wrote:
> An error message can seen from the init-dom0less application on
> direct-mapped 1:1 domains:
> ```
> Allocating magic pages
> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> Error on alloc magic pages
> ```
> 
> This is because populate_physmap() automatically assumes gfn == mfn
> for direct mapped domains. This cannot be true for the magic pages
> that are allocated later for Dom0less DomUs from the init-dom0less
> helper application executed in Dom0.
> 
> Force populate_physmap to take the "normal" memory allocation route for
> the magic pages even for 1:1 Dom0less DomUs. This should work as long
> as the 1:1 Dom0less DomU doesn't have anything else mapped at the same
> guest address as the magic pages:
> - gfn 0x39000 address 0x39000000
> - gfn 0x39001 address 0x39001000
> - gfn 0x39002 address 0x39002000
> - gfn 0x39003 address 0x39003000

This is very fragile. You are making the assumption that the magic pages 
are not clashing with any RAM region. The layout defined in arch-arm.h 
has been designed for guest where Xen is in full control of the layout. 
This is not the case for directmapped domain. I don't think it is 
correct to try to re-use part of the layout.

If you want to use 1:1 dom0less with xenstore & co, then you should find 
a different place in memory for the magic pages (TDB how to find that 
area). You will still have the problem of the 1:1 allocation, but I 
think this could be solved bty adding a flag to force a non-1:1 allocation.

> Create helper is_magic_gpfn() for Arm to assist this and stub helpers
> for non-Arm architectures to avoid #ifdef. Move the definition of the
> magic pages on Arm to a more common place.
> 
> Note that the init-dom0less application of the diffenent Xen version

s/diffenent/different/

> may allocate all or part of four magic pages for each DomU.
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> ---
>   tools/libs/guest/xg_dom_arm.c   |  6 ------
>   xen/arch/arm/include/asm/mm.h   | 13 +++++++++++++
>   xen/arch/ppc/include/asm/mm.h   |  5 +++++
>   xen/arch/riscv/include/asm/mm.h |  6 ++++++
>   xen/arch/x86/include/asm/mm.h   |  5 +++++
>   xen/common/memory.c             |  2 +-
>   xen/include/public/arch-arm.h   |  6 ++++++
>   7 files changed, 36 insertions(+), 7 deletions(-)
> 
> diff --git a/tools/libs/guest/xg_dom_arm.c b/tools/libs/guest/xg_dom_arm.c
> index 2fd8ee7ad4..8c579d7576 100644
> --- a/tools/libs/guest/xg_dom_arm.c
> +++ b/tools/libs/guest/xg_dom_arm.c
> @@ -25,12 +25,6 @@
>   
>   #include "xg_private.h"
>   
> -#define NR_MAGIC_PAGES 4
> -#define CONSOLE_PFN_OFFSET 0
> -#define XENSTORE_PFN_OFFSET 1
> -#define MEMACCESS_PFN_OFFSET 2
> -#define VUART_PFN_OFFSET 3
> -
>   #define LPAE_SHIFT 9
>   
>   #define PFN_4K_SHIFT  (0)
> diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h
> index cbcf3bf147..17149b4635 100644
> --- a/xen/arch/arm/include/asm/mm.h
> +++ b/xen/arch/arm/include/asm/mm.h
> @@ -428,6 +428,19 @@ static inline void page_set_xenheap_gfn(struct page_info *p, gfn_t gfn)
>       } while ( (y = cmpxchg(&p->u.inuse.type_info, x, nx)) != x );
>   }
>   
> +#define MAGIC_PAGE_N_GPFN(n)     ((GUEST_MAGIC_BASE >> PAGE_SHIFT) + n)
> +static inline bool is_magic_gpfn(xen_pfn_t gpfn)
> +{
> +    unsigned int i;
> +    for ( i = 0; i < NR_MAGIC_PAGES; i++ )
> +    {
> +        if ( gpfn == MAGIC_PAGE_N_GPFN(i) )
> +            return true;
> +    }
> +
> +    return false;
> +}
> +
>   #endif /*  __ARCH_ARM_MM__ */
>   /*
>    * Local variables:
> diff --git a/xen/arch/ppc/include/asm/mm.h b/xen/arch/ppc/include/asm/mm.h
> index a433936076..8ad81d9552 100644
> --- a/xen/arch/ppc/include/asm/mm.h
> +++ b/xen/arch/ppc/include/asm/mm.h
> @@ -256,4 +256,9 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
>       return true;
>   }
>   
> +static inline bool is_magic_gpfn(xen_pfn_t gpfn)
> +{
> +    return false;
> +}
> +
>   #endif /* _ASM_PPC_MM_H */
> diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
> index 07c7a0abba..a376a77e29 100644
> --- a/xen/arch/riscv/include/asm/mm.h
> +++ b/xen/arch/riscv/include/asm/mm.h
> @@ -3,6 +3,7 @@
>   #ifndef _ASM_RISCV_MM_H
>   #define _ASM_RISCV_MM_H
>   
> +#include <public/xen.h>
>   #include <asm/page-bits.h>
>   
>   #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
> @@ -20,4 +21,9 @@ unsigned long calc_phys_offset(void);
>   
>   void turn_on_mmu(unsigned long ra);
>   
> +static inline bool is_magic_gpfn(xen_pfn_t gpfn)
> +{
> +    return false;
> +}
> +
>   #endif /* _ASM_RISCV_MM_H */
> diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
> index 7d26d9cd2f..f385f36d78 100644
> --- a/xen/arch/x86/include/asm/mm.h
> +++ b/xen/arch/x86/include/asm/mm.h
> @@ -628,4 +628,9 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
>       return (mfn + nr) <= (virt_to_mfn(eva - 1) + 1);
>   }
>   
> +static inline bool is_magic_gpfn(xen_pfn_t gpfn)
> +{
> +    return false;
> +}
> +
>   #endif /* __ASM_X86_MM_H__ */
> diff --git a/xen/common/memory.c b/xen/common/memory.c
> index b3b05c2ec0..ab4bad79e2 100644
> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>           }
>           else
>           {
> -            if ( is_domain_direct_mapped(d) )
> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )

This path will also be reached by dom0. Effectively, this will prevent 
dom0 to allocate the memory 1:1 for the magic GPFN (which is guest 
specific).

Also, why are you only checking the first GFN? What if the caller pass 
an overlapped region?

>               {
>                   mfn = _mfn(gpfn);
>   
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index a25e87dbda..58aa6ff05b 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -476,6 +476,12 @@ typedef uint64_t xen_callback_t;
>   #define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
>   #define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
>   
> +#define NR_MAGIC_PAGES 4
> +#define CONSOLE_PFN_OFFSET 0
> +#define XENSTORE_PFN_OFFSET 1
> +#define MEMACCESS_PFN_OFFSET 2
> +#define VUART_PFN_OFFSET 3

Regardless of what I wrote above, it is not clear to me why you need to 
move these macros in public header. Just above, we are defining the 
magic region (see GUEST_MAGIC_BASE and GUEST_MAGIC_SIZE). This should be 
sufficient to detect whether a GFN belongs to the magic region.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:15:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:15:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685363.1065939 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reX4r-0001LS-0I; Mon, 26 Feb 2024 09:15:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685363.1065939; Mon, 26 Feb 2024 09:15:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reX4q-0001LB-So; Mon, 26 Feb 2024 09:15:12 +0000
Received: by outflank-mailman (input) for mailman id 685363;
 Mon, 26 Feb 2024 09:15:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=+ho1=KD=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1reX4p-0001J9-7V
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:15:11 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 89d9ba7c-d487-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 10:15:09 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id DDE2D1FB3E;
 Mon, 26 Feb 2024 09:15:08 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A74A013A58;
 Mon, 26 Feb 2024 09:15:08 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id sHifJpxW3GVkHQAAD6G6ig
 (envelope-from <jgross@suse.com>); Mon, 26 Feb 2024 09:15:08 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 89d9ba7c-d487-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708938908; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=HMjGI0U8bcjZJ5c6EYnbel/ON6czihwwfryCuQFsuxM=;
	b=L4iAxLX4HrgRtWe9oEdxAlGSf6b5MNv2tNfgIvBkEazqOQCYO6Q31FC/xq1+lj/04elTJR
	nyQvrwOixFA+YaNFe8w44Z4NtbX280ijutvB1dXfXJLNA4o6nLkhippHSMep5xeOOQKAvO
	lgHXEaEMU+9B9yRNLI4Qw+dIL0G2n/M=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1708938908; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=HMjGI0U8bcjZJ5c6EYnbel/ON6czihwwfryCuQFsuxM=;
	b=L4iAxLX4HrgRtWe9oEdxAlGSf6b5MNv2tNfgIvBkEazqOQCYO6Q31FC/xq1+lj/04elTJR
	nyQvrwOixFA+YaNFe8w44Z4NtbX280ijutvB1dXfXJLNA4o6nLkhippHSMep5xeOOQKAvO
	lgHXEaEMU+9B9yRNLI4Qw+dIL0G2n/M=
Message-ID: <d21b9f82-ddfd-4014-b653-42e52ef3abfa@suse.com>
Date: Mon, 26 Feb 2024 10:15:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] Mini-OS: add symbol exports for xenstore stubdom
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: samuel.thibault@ens-lyon.org, wl@xen.org,
 minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org
References: <20240226083955.19350-1-jgross@suse.com>
 <707666cc-914e-42da-b823-fdc5a38271be@suse.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <707666cc-914e-42da-b823-fdc5a38271be@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -1.47
X-Spamd-Result: default: False [-1.47 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-1.01)[87.29%];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCPT_COUNT_FIVE(0.00)[5];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.83)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Flag: NO

On 26.02.24 09:44, Jan Beulich wrote:
> On 26.02.2024 09:39, Juergen Gross wrote:
>> Xenstore stubdom needs some more symbols exported.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> 
> Any Reported-by: possibly applicable here?

If any, then Andrew.

He reported a CI loop failure.


Juergen



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:23:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:23:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685381.1065972 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXD1-00046Z-2i; Mon, 26 Feb 2024 09:23:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685381.1065972; Mon, 26 Feb 2024 09:23:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXD1-00046S-07; Mon, 26 Feb 2024 09:23:39 +0000
Received: by outflank-mailman (input) for mailman id 685381;
 Mon, 26 Feb 2024 09:23:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reXD0-000468-Hg
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:23:38 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b7de75b3-d488-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 10:23:36 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a4348aaa705so86503266b.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 01:23:36 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 vo1-20020a170907a80100b00a42ea946917sm2213752ejc.130.2024.02.26.01.23.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 01:23:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7de75b3-d488-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708939415; x=1709544215; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/XhYtCShDR93lDUGaWIA4sRv0Pa4Hm9U6nYsk/8VuPY=;
        b=GXNpM/h17BwPImWIcMM71tZwhZPhmP0Ur6wmd5XFPykOYgTG0qWiEj/lux8FFEJ6rC
         jVGhvVxdr5i1MXwfUfHSweNYRkw9mnmaEWEsyaznAQatHN3e4mGCj1dW/y5rbYvTcKs1
         GjZo6x2fmbihQNw8CQ4aR9v9+dJaTZOfu/njFXyuy0Xop5rPDKbPAnigEtdqMUDQ/4+T
         hz3kY7cwQJ1Mgx60iBoN5l/hdoxGpNlge+KS2gG+nNo7nKOux+/udGsheMvZlIBsifKB
         moAWUEi+PiqIWHveRL6eqj6ut/0eSWROkfWeFqOcwltjW96koahXAzITFiPGas+mSzj0
         Kz2w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708939416; x=1709544216;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/XhYtCShDR93lDUGaWIA4sRv0Pa4Hm9U6nYsk/8VuPY=;
        b=OEvC8dp6q7j2nbhHpSNbpFShI4TEsggN6SsZL91GBWMaifAxpUSooBw7TZDoAC+mUM
         CFunh/N/F5LQvd8iuf8epc4SzmExXO24cq5F+F5meoQ/ZD0he51zPwFOrFL5uiJYEV/8
         kTj2mR7bhtm5k7M/EjRMzm3haG2RvD0edaExW384bFXuymT0PEudFFDlbInUX/Tc+mMR
         9GHR5t5lDHJ2cUsBl5kWOhOZZQJzXHOGML02XDZQeZ5zjW01A4KASUE0NWoSsTnJqaga
         mXiQblPCqjGIHSKclAgHwD1eMEGSSavwjFW9N/ULN+NlYmHt3ViviWP4NahoZJ7w51RQ
         FatA==
X-Forwarded-Encrypted: i=1; AJvYcCVtkZOg1e5vmZL9oSYZnQRSSr5XwNNo0vMXQLONd6ZoWsQRQA67uV6lRIdIkNoaXjxPyIjj0njlBo4X9FuseuZchOf0rqvoagblrixXSts=
X-Gm-Message-State: AOJu0YzCWTA/VYq52Xu0wMLAmZ5g6hHKuDHXE5rFEIssFLgYvG56/4Ov
	D835ZG0muH7sqgAV4R9xjVYPQRWwLrIuLj1C/ddricHfuw71UD3P6GqkEXcyJQ==
X-Google-Smtp-Source: AGHT+IG1nahSdqZFD9/93euE3j9WPovmK3TPqq49DdnB3QeZtnSQRsuNgc82YteRoZznguXWzlqmJw==
X-Received: by 2002:a17:906:254b:b0:a3f:f73:411f with SMTP id j11-20020a170906254b00b00a3f0f73411fmr4066862ejb.45.1708939415701;
        Mon, 26 Feb 2024 01:23:35 -0800 (PST)
Message-ID: <e48c4fce-e7f3-4bce-bf7b-2cfcc0659426@suse.com>
Date: Mon, 26 Feb 2024 10:23:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation: Rework "build-each-commit-gcc" test
Content-Language: en-US
To: Anthony PERARD <anthony.perard@citrix.com>
Cc: Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <20240220140733.40980-1-anthony.perard@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240220140733.40980-1-anthony.perard@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 20.02.2024 15:07, Anthony PERARD wrote:
> Current issues with this test are:
> - when the job timeout, the log file is lost as there is no chance to
>   run the `mv` command.
> - GitLab job log is limited in size, so one usually have to download
>   the artifacts, which may be missing.
> 
> Use $GITLAB_CI to detect when the script is run as part of a GitLab
> pipeline. GitLab will add "GITLAB_CI=true" in the environment
> variables.
> 
> When run as part of $GITLAB_CI, ignore "dirty" worktree to allow to
> write "build-each-commit-gcc.log", which can then be grabbed as
> artifacts, even when the job timeout. The `git clean` command is
> changed to keep those build logs.
> 
> When run as part of $GITLAB_CI, we will also store the build output in
> a log file instead of writing it to stdout, because GitLab's job log
> is limited. But we will write the log to stdout in case of error, so
> we can find out more quickly why there's been an error.
> 
> This patch also make use of a GitLab feature, "log sections", which we
> will collapse by default. One section per commit been built.
> 
> There's a bit of colour added to the logs.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

I was trying to commit this, but ...

> --- a/automation/scripts/build-test.sh
> +++ b/automation/scripts/build-test.sh
> @@ -9,6 +9,37 @@
>  # Set NON_SYMBOLIC_REF=1 if you want to use this script in detached HEAD state.
>  # This is currently used by automated test system.
>  
> +# Colors with ANSI escape sequences
> +txt_info='[32m'
> +txt_err='[31m'
> +txt_clr='[0m'
> +
> +# $GITLAB_CI should be "true" or "false".
> +if [ "$GITLAB_CI" != true ]; then
> +    GITLAB_CI=false
> +fi
> +
> +gitlab_log_section() {
> +    if $GITLAB_CI; then
> +        echo -n "[0Ksection_$1:$(date +%s):$2
> [0K"

... there was either corruption on transmit here, or there's an embedded
newline that I don't know how to deal with.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:26:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:26:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685383.1065982 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXG3-0004fL-GP; Mon, 26 Feb 2024 09:26:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685383.1065982; Mon, 26 Feb 2024 09:26:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXG3-0004fE-Dl; Mon, 26 Feb 2024 09:26:47 +0000
Received: by outflank-mailman (input) for mailman id 685383;
 Mon, 26 Feb 2024 09:26:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reXG2-0004f8-4q
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:26:46 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2858ee9c-d489-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 10:26:44 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-564fd9eea75so3394075a12.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 01:26:44 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n9-20020a056402060900b0056200715130sm2181807edv.54.2024.02.26.01.26.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 01:26:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2858ee9c-d489-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708939604; x=1709544404; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8GNOaXDjZBzAVrce0EwOuJ4HRtW79oEeKI7dXw8Ja/4=;
        b=RV5F9IVc76QjMdRlZNOHtvTKJtu8zRANdTjaNBlt3GOYQwSEwnq+RGAYk/LDvu63H9
         E9GoUFJsMLP2oxDe/GSdt+Dx9ngqdsPBOqNqhmAadFkkvrPHv3my9AfJEyTZRWYqjBSj
         78NhYrIt0mlgRhOfNG13Oj3Txfwa0PJr95MUbDAvQyigArmIsdv47eF7hPDTsQDNYLNF
         A0lVCMJURBUeWneHU7m0g9TUxQMTVM0dRL67SfTFaDeZshyOrxUUjnftR9tnCUcDLIPK
         CSe5LuG8D7YC8a7hUCoruaLTM0QtKJNpzXrBNRpri1iBUMiTl0492TcO0ly0T1gQV6Ol
         YNGw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708939604; x=1709544404;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8GNOaXDjZBzAVrce0EwOuJ4HRtW79oEeKI7dXw8Ja/4=;
        b=IRSdN0+6OcXOpkj35/pqJryn/R9CnkJGs9j7UmWwis+83O9tT/DoBFYJnYfayJ8Lpk
         13rjpYcE/+YNjim/Yn4mb913+URqCeh0IUiu14e54aUc5nwnfN38lB4LByRISaN4/Qeb
         Z5KbEh5ftKOcNbkAWlxAy5fGhwvR8osSRI8gWQVRyt8ugPNs7oiAMNoNzE9OP5DIlFTw
         pEunv5JmTTeAMVQB1dyGfkAyug/5zV9767wYQBNCLUDxAa3WChEPLBDTVn/TL6Jd/tT7
         /HsyoTa1sCXuu4960GxoMFJjCVNERawlDe+k5POLaTMe2jsi8lCg8L77fSOUPrfst8u3
         xXEA==
X-Forwarded-Encrypted: i=1; AJvYcCUIgV5J+zmOJr82E1kqRgiznrsfcJ/1We3zpRBu+8+HCI0bguyJ+/0MNXPQ9E/2Yq1xxrtPlDVi5NMV/k6CDfxd8tv06SkmZeVnH///C6U=
X-Gm-Message-State: AOJu0YyOfjxjDnhNHWgnxWWD5uZgpyCuo4S/09CM8RjQaqqxyezY+I5W
	2fBtKvL2m3C7sv4THDWsyyUdgwyPvj3gZCrVnn+LD83q7LCqqvsce5iK5s3HDQ==
X-Google-Smtp-Source: AGHT+IHxy+fJot7CzELNq79H1rPR0J93G+cewhckX4vNwrN2lmqJX6FHpx4uwW2PXy0r22ixyaZmdQ==
X-Received: by 2002:a50:fc18:0:b0:563:d32f:5449 with SMTP id i24-20020a50fc18000000b00563d32f5449mr4231643edr.24.1708939604391;
        Mon, 26 Feb 2024 01:26:44 -0800 (PST)
Message-ID: <19742e01-bc8d-41ed-a12c-b7f7bf57f642@suse.com>
Date: Mon, 26 Feb 2024 10:26:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 0/5] xen/livepatch: fixes for the pre-apply / post-revert
 hooks
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231130142944.46322-1-roger.pau@citrix.com>
 <6143d253-1d5c-48c8-81b9-3652b21dd9c0@suse.com> <ZdxOcc1O35lkTS4L@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZdxOcc1O35lkTS4L@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 09:40, Roger Pau Monné wrote:
> On Fri, Feb 23, 2024 at 11:48:10AM +0100, Jan Beulich wrote:
>> On 30.11.2023 15:29, Roger Pau Monne wrote:
>>> Hello,
>>>
>>> The follow series contain a misc of fixes mostly related to the usage of
>>> the pre-apply / post-revert hooks.  The norevert test is also fixed to
>>> work as I think was expected.  Finally both the no{apply,revert}
>>> tests are fixed to build properly, as the files where previously
>>> unhooked from the build system completely.
>>>
>>> I'm unsure how useful the apply and revert hooks really are, as without
>>> calling the internal apply/revert functions the state of the payload
>>> structure is quite likely inconsistent with the code expectations.
>>>
>>> Thanks, Roger.
>>>
>>> Roger Pau Monne (5):
>>>   xen/livepatch: register livepatch regions when loaded
>>>   xen/livepatch: search for symbols in all loaded payloads
>>>   xen/livepatch: fix norevert test attempt to open-code revert
>>>   xen/livepatch: fix norevert test hook setup typo
>>>   xen/livepatch: properly build the noapply and norevert tests
>>
>> With the R-b-s that have arrived, could you clarify in how far these
>> can be committed out of order?
> 
> All can be applied out of order, except for the last one, that's
> "xen/livepatch: properly build the noapply and norevert tests" as
> applying that will enable the tests in osstest, and those will fail
> without all the previous adjustments.
> 
> I was about to send v2 with the requested changes, but will wait for
> you to commit 2/5 and 4/5.

Except that patch 2 had 3 failing hunks, and not just fuzz in any of them.
Which was too much for me to consider fixing up while committing.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:27:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:27:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685369.1065992 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXGr-00059y-PW; Mon, 26 Feb 2024 09:27:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685369.1065992; Mon, 26 Feb 2024 09:27:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXGr-00059r-MS; Mon, 26 Feb 2024 09:27:37 +0000
Received: by outflank-mailman (input) for mailman id 685369;
 Mon, 26 Feb 2024 09:16:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tEhT=KD=thalesgroup.com=francois.gourlot@srs-se1.protection.inumbo.net>)
 id 1reX5o-0001tH-Sg
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:16:13 +0000
Received: from esa.hc1631-21.eu.iphmx.com (esa.hc1631-21.eu.iphmx.com
 [23.90.123.40]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ad20b50f-d487-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 10:16:10 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ad20b50f-d487-11ee-98f5-efadbce2ee36
X-CSE-ConnectionGUID: sDeedq6pRLS72LzNAXNOgw==
X-CSE-MsgGUID: yoyst1LTRIe43KfDR/FzpA==
Authentication-Results: ob1.hc1631-21.eu.iphmx.com; dkim=pass (signature verified) header.i=@thalesgroup.com
X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="10687971"
X-IronPort-AV: E=Sophos;i="6.06,185,1705359600"; 
   d="scan'208,217";a="10687971"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=thalesgroup.com; i=@thalesgroup.com; s=bbmfo20230504;
  t=1708938968;
  h=from:to:cc:subject:date:message-id:references:
   in-reply-to:mime-version;
  bh=smISyhHC3Da/ebx5y/xZb5ahqk/wbLFphQoimWXdNMo=;
  b=LUzwLu/lGWbKzFhegJa9w8LFOiVPLbHMDnmTxrOxBN2wbtlHRmRg+vZ9
   S459P1bejl9kf8Ms98CLZXZefTJmmppx4Cav89LNC/PyxasuzKLKB4hVp
   wk662WbY772kJM+0hOfx6qo8mdBLZpaTVCCo3y/ygBG67YLLuXGxi5Ovs
   v4s/g3MRtF9l7OJ2jD7dpcI0CKNmt5/Ik5v5PBRo+JdzIif/roMYnUYzw
   g72PdyO3kEv0qBiCIs4a+Kg4VNqiLI53I0vrInjDsN1eANSigj2cdFFCN
   +Ui5w2VWggoa2H28OQaxz6zhe8fNLME9IYz0c27IJRCRbRMHQm7A8ahXT
   Q==;
X-CSE-ConnectionGUID: 3zFS9pc0RaKG8Ntrq4ra2A==
X-CSE-MsgGUID: 2v1Kx2PXTDG9hMup3R3Wng==
X-CSE-ConnectionGUID: cuA25v1jTLeqvsFcfLBfHw==
X-CSE-MsgGUID: +1r+ZW1qSz+zQgukquGwAg==
X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="18763490"
X-IronPort-AV: E=Sophos;i="6.06,185,1705359600"; 
   d="scan'208,217";a="18763490"
From: GOURLOT Francois <francois.gourlot@thalesgroup.com>
To: "Rob.Townley@gmail.com" <Rob.Townley@gmail.com>, Julien Grall
	<julien@xen.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Juergen Gross
	<jgross@suse.com>, Kelly Choi <kelly.choi@cloud.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, "LARRIEU
 Dominique" <dominique.larrieu@thalesgroup.com>
Subject: RE: IMPORTANT - : Need help on USB port virtualization with Xen
 hypervisor
Thread-Topic: IMPORTANT - : Need help on USB port virtualization with Xen
 hypervisor
Thread-Index: AdpmcWIlf+ro0u5NSE2ZDRGqfQbNXAArMvkAAC+VjYAALaPFYA==
Sensitivity: company-confidential
Date: Mon, 26 Feb 2024 09:15:24 +0000
Message-ID: <7d7062a0abf54fea890b38c545c459c3@thalesgroup.com>
References: <d2de4ae9ecb34efc962dea7f8b4e7cbd@thalesgroup.com>
 <bb2485e5-7818-435b-8d9c-dda88100979f@xen.org>
 <CA+VdTb8razTDCZdAssAf=HHbDhV+w8+k-XO+n6msv31izUfQEw@mail.gmail.com>
In-Reply-To: <CA+VdTb8razTDCZdAssAf=HHbDhV+w8+k-XO+n6msv31izUfQEw@mail.gmail.com>
Accept-Language: fr-FR, en-US
Content-Language: fr-FR
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-nodisclaimer: 0
Content-Type: multipart/alternative;
	boundary="_000_7d7062a0abf54fea890b38c545c459c3thalesgroupcom_"
MIME-Version: 1.0

--_000_7d7062a0abf54fea890b38c545c459c3thalesgroupcom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

RGVhciBSb2IgYW5kIEp1bGllbiwNCg0KVGhhbmsgeW91IGZvciB5b3VyIG1lc3NhZ2VzLg0KDQpE
b21pbmlxdWUgYW5kIHRoZSBUaGFsZXMgY3JldyB3aWxsIGdpdmUgeW91IG1vcmUgaW5mb3JtYXRp
b24gcXVpY2tseS4NCg0KQmVzdCByZWdhcmRzDQoNClJlc3BvbnNhYmxlIFByb2dyYW1tZQ0KDQpE
w6lwYXJ0ZW1lbnQgUkFGQUxFIEY0DQpUaGFsZXMgU0lYIEdUUyBGcmFuY2UNCg0KVDogKzMzKDAp
IDEgNDEgMzAgMjYgNjcNCk06ICszMyAoMCkgNiA4NSAyMyAxMiAzMA0KDQo0IEF2ZW51ZSBkZXMg
TG91dnJlc3Nlcw0KOTI2MjIgR2VubmV2aWxsaWVycw0KRnJhbmNlDQoNCg0KDQpEZSA6IFJvYiBU
b3dubGV5IDxyb2IudG93bmxleUBnbWFpbC5jb20+IERlDQpFbnZvecOpIDogZGltYW5jaGUgMjUg
ZsOpdnJpZXIgMjAyNCAxMzoyMA0Kw4AgOiBKdWxpZW4gR3JhbGwgPGp1bGllbkB4ZW4ub3JnPg0K
Q2MgOiBBbmRyZXcgQ29vcGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPjsgR09VUkxPVCBG
cmFuY29pcyA8ZnJhbmNvaXMuZ291cmxvdEB0aGFsZXNncm91cC5jb20+OyBHZW9yZ2UgRHVubGFw
IDxnZW9yZ2UuZHVubGFwQGNpdHJpeC5jb20+OyBKYW4gQmV1bGljaCA8amJldWxpY2hAc3VzZS5j
b20+OyBKdWVyZ2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+OyBLZWxseSBDaG9pIDxrZWxseS5j
aG9pQGNsb3VkLmNvbT47IFJvZ2VyIFBhdSBNb25uw6kgPHJvZ2VyLnBhdUBjaXRyaXguY29tPjsg
eGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnDQpPYmpldCA6IFJlOiBJTVBPUlRBTlQgLSA6
IE5lZWQgaGVscCBvbiBVU0IgcG9ydCB2aXJ0dWFsaXphdGlvbiB3aXRoIFhlbiBoeXBlcnZpc29y
DQoNCkluIHNob3J0LCBwdXJjaGFzZSBhbiBhZGRvbiBQQ0llIFVTQiBjYXJkIHRvIGlzb2xhdGUg
dHJhZmZpYy4gIFVTQiBpcyBhIHNoYXJlZCBtZWRpdW0gYW5kIGlzIHJvb3RlZCBpbiB0aGUgY2hl
YXBlc3QgZWxlY3Ryb25pY3MuICAgSXQgd2FzIG5ldmVyIGNvbnNpZGVyZWQgZW50ZXJwcmlzZSBn
cmFkZSBoYXJkd2FyZSBmb3IgbW9zdCBhbGwgb2YgaGlzdG9yeS4gICAgVGhlIGxldHRlciBCIGlu
ICBVU0IgIHN0YW5kcyBmb3IgQlVTIHdoaWNoIGFjdHVhbGx5IG1lYW5zIGEgc2hhcmVkIHJvYWR3
YXkgdGhhdCBvdGhlciBVU0IgZGV2aWNlcyBjYW4gYW5kIGRvIGltcGVkZSB0cmFmZmljLiAgIFdl
IGV4cGVyaWVuY2VkIGludGVyZmVyaW5nIHRyYWZmaWMgb24gaW5leHBlbnNpdmUgbGFwdG9wcyBy
dW5uaW5nICBYRU4gYmFzZWQgUXViZXMtT1MuICBUdXJuZWQgb3V0IHRoZSBXaUZpIGNhcmQgd2Fz
IGFjdHVhbGx5IHdpcmVkIG9udG8gdGhlIFVTQiBidXMuICAgVW5wbHVnZ2luZyBhbGwgZXh0ZXJu
YWwgVVNCIGVsZWN0cm9uaWNzIHJlZHVjZWQgaW5leHBsaWNhYmxlIFdpRmkgZGlzY29ubmVjdHMg
YW5kIHJlbGlhYmx5IGltcHJvdmVkIHRocm91Z2hwdXQuICAgIEhvd2V2ZXIsIHRoZXJlIHdhcyBu
byB3YXkgdG8gcmVtb3ZlIHRoZSBrZXlib2FyZCwgdG91Y2hwYWQsIGFuZCB0b3VjaHNjcmVlbiB3
aGljaCBhbGwgdXNlZCB0aGUgVVNCIHJvYWR3YXkuDQoNClRob3NlIHdlcmUgY2hlYXAgbGFwdG9w
cywgYnV0IGV2ZW4gZW50ZXJwcmlzZSBjbGFzcyBoYXJkd2FyZSBzdWNoIGFzIEhQRSAgYW5kIERl
bGwgU2VydmVycyBydW5uaW5nIFhDUC1ORyBkaWQgbm90IGhhdmUgZ3JlYXQgc3BlZWRzLiAgIEJ1
dCBpbiB0aGlzIHNjZW5hcmlvLCB3ZSBjb3VsZCBzaW1wbGlmeSB0aGUgcHJvYmxlbSBhbmQgIGlz
b2xhdGUgdGhlIGlzc3VlIGJ5IHB1cmNoYXNpbmcgYSAgbmV3IGFkZC1vbiBQQ0llIGNhcmQgd2l0
aCBVU0IgcG9ydHMgYW5kIHRoZW4gYXNzaWduIHRoZSBlbnRpcmUgUENJZSBjYXJkIHRvIHRoZSBW
TS4gICBUaGUgVk0gd291bGQgdGhlbiBoYXZlIGVudGlyZSBjb250cm9sIG9mIHRoZSBQQ0llIGNh
cmQgYW5kIGlzb2xhdGUgdHJhZmZpYyB0byBkZXZpY2VzIHBsdWdnZWQgaW50byB0aGF0IGNhcmQu
DQoNCkkgZGlkIG5vdCBrbm93IHVudGlsIGxhdGVyIHRoYXQgV2lyZXNoYXJrIGhhcyBwbHVnaW5z
IGZvciBzbmlmZmluZyBVU0IgdHJhZmZpYywgYnV0IG15IG5hbWUgaXMgIG5vdCBIZWlzZW5iZXJn
IGFuZCBub3Qgc3VyZSBpZiB0aGF0IHRyYWZmaWMgbW9uaXRvcmluZyBkb2VzIG5vdCBhbHNvIHNs
b3cgZG93biBvdGhlciBkZXZpY2VzLg0KDQpPbiBTYXQsIEZlYiAyNCwgMjAyNCBhdCA4OjM5IEFN
IEp1bGllbiBHcmFsbCA8anVsaWVuQHhlbi5vcmc8bWFpbHRvOmp1bGllbkB4ZW4ub3JnPj4gd3Jv
dGU6DQpIaSBEb21pbmlxdWUgYW5kIEZyYW5jb2lzLA0KDQpUaGFuayB5b3UgZm9yIHRoZSByZXBv
cnQuIEkgYW0gQ0Npbmcgc29tZSBwZXJzb24gd2hvIEkgdGhpbmsgY291bGQgaGVscC4NCg0KSSBh
bHNvIGhhdmUgc29tZSBxdWVzdGlvbnMgKHNlZSBiZWxvdykuDQoNCk9uIDIzLzAyLzIwMjQgMTY6
MDAsIEdPVVJMT1QgRnJhbmNvaXMgd3JvdGU6DQo+IERlYXIgQWxsLA0KPg0KPiBXZSBzZW5kIHlv
dSBhIG1lc3NhZ2UgZmV3IGRheXMgYWdvLg0KPg0KPiBXZSBoYXZlIG1ham9yIHBlcmZvcm1hbmNl
IHdpdGggWEVOIFVTQiBEcml2ZXJzLiBXZSB1c2UgeW91ciBkcml2ZXIgdG8NCj4gbG9hZCBkYXRh
IGluIGEgZGV2aWNlIGFuZCB3ZSBuZWVkIFVTQjMgSGlnaCBTcGVlZCBVU0IgcHJvdG9jb2xlDQo+
DQo+IERvIHlvdSBoYXZlIGEgbmV3IGlzc3VlIG9mIHRoZSBIQ0kgYW5kIE5FQyBVU0IgWGVuIERy
aXZlciA/DQpwDQpbLi4uXQ0KDQo+ICpEZSA6KkxBUlJJRVUgRG9taW5pcXVlIDxkb21pbmlxdWUu
bGFycmlldUB0aGFsZXNncm91cC5jb208bWFpbHRvOmRvbWluaXF1ZS5sYXJyaWV1QHRoYWxlc2dy
b3VwLmNvbT4+DQo+ICpFbnZvecOpIDoqIGpldWRpIDIyIGbDqXZyaWVyIDIwMjQgMTM6NTgNCj4g
KsOAIDoqIHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZzxtYWlsdG86eGVuLWRldmVsQGxp
c3RzLnhlbnByb2plY3Qub3JnPg0KPiAqQ2MgOiogV0lMTEVNUyBMb3VpcyA8bG91aXMud2lsbGVt
c0B0aGFsZXNncm91cC5jb208bWFpbHRvOmxvdWlzLndpbGxlbXNAdGhhbGVzZ3JvdXAuY29tPj47
IEdSVU8gTmljb2xhcw0KPiA8bmljb2xhcy1uLmdydW9AdGhhbGVzZ3JvdXAuY29tPG1haWx0bzpu
aWNvbGFzLW4uZ3J1b0B0aGFsZXNncm91cC5jb20+PjsgR09VUkxPVCBGcmFuY29pcw0KPiA8ZnJh
bmNvaXMuZ291cmxvdEB0aGFsZXNncm91cC5jb208bWFpbHRvOmZyYW5jb2lzLmdvdXJsb3RAdGhh
bGVzZ3JvdXAuY29tPj47IEtlbGx5IENob2kgPGtlbGx5LmNob2lAY2xvdWQuY29tPG1haWx0bzpr
ZWxseS5jaG9pQGNsb3VkLmNvbT4+DQo+ICpPYmpldCA6KiBOZWVkIGhlbHAgb24gVVNCIHBvcnQg
dmlydHVhbGl6YXRpb24gd2l0aCBYZW4gaHlwZXJ2aXNvcg0KPiAqQ3JpdMOocmUgZGUgZGlmZnVz
aW9uIDoqIENvbmZpZGVudGllbA0KPg0KPiAgICAgRGVhciBhbGwsDQo+DQo+ICAgICBXZSBhcmUg
ZGV0ZWN0aW5nc2V2ZXJhbCBpc3N1ZXMgd2l0aCBVU0IgcG9ydCB2aXJ0dWFsaXphdGlvbiB3aXRo
IHRoZQ0KPiAgICAgWGVuIGh5cGVydmlzb3IuDQoNCkp1c3QgdG8gY2xhcmlmeSwgeW91IGFyZSBh
c3NpZ25pbmcgdGhlIFBDSSBVU0IgYnVzIChyYXRoZXIgdGhhbiBqdXN0IHRoZQ0KVVNCIGRldmlj
ZSkgdG8gdGhlIGd1ZXN0LiBJcyB0aGF0IGNvcnJlY3Q/DQoNCklmIHNvLi4uDQoNCiA+DQogPiBE
byB5b3UgaGF2ZSBhIG5ldyBpc3N1ZSBvZiB0aGUgSENJIGFuZCBORUMgVVNCIFhlbiBEcml2ZXIg
Pw0KDQouLi4gSSBhbSBub3Qgc3VyZSB3aGF0IFhlbiBkcml2ZXIgeW91IGFyZSB0YWxraW5nIGFi
b3V0LiBJZiB0aGUgVVNCIGJ1cw0KaXMgYXNzaWduZWQgdG8gdGhlIGd1ZXN0LCB0aGVuIEkgd291
bGQgZXhwZWN0IG5vIHNwZWNpZmljIFhlbiBkcml2ZXIgYXJlDQpyZXF1aXJlZC4gRG8geW91IGhh
dmUgYW55IHBvaW50ZXIgdG8gdGhlIGV4aXN0aW5nIG9uZT8NCg0KPiAgICAgLSBXZSBjYW5ub3Qg
ZG8gUENJIHBhc3N0aHJvdWdoIG9mIHRoZSBQQ0kgdXNiIGJ1cyBvbiBhIFdpbmRvd3MgMTANCj4g
ICAgIDE2MDcgNjQtYml0IHZpcnR1YWwgbWFjaGluZS4gVGhlIGJhZCByZXN1bHQgaXMgYSBXaW5k
b3dzIGJsdWUgc2NyZWVuLg0KPg0KPiAgICAgLSBXaGVuIHdlIHVzZSB0aGUgcGFzc3Rocm91Z2gg
ZnVuY3Rpb25hbGl0eSBvbiBhIFdpbmRvd3MgMjFIMg0KPiAgICAgdmlydHVhbCBtYWNoaW5lLCB3
ZSBub3RpY2UgdGhhdHRoZSBzcGVlZCBvZiB0aGUgVVNCIHBvcnQgaXMgbm90IGhpZ2gNCj4gICAg
IHNwZWVkIGJ1dCBmdWxsIHNwZWVkIG9uIGEgVVNCIDMuMCBwb3J0DQo+DQo+ICAgICAtIFdlIG5v
dGljZSBpbnN0YWJpbGl0aWVzIHdoZW4gdXNpbmcgdGhlIG5lYy11c2IteGhjaSBkcml2ZXIsICBV
U0INCj4gICAgIDIuMCBrZXlzIGFyZSBub3QgcmVjb2duaXplZCBieSB0aGUgV2luZG93cyB2aXJ0
dWFsIG1hY2hpbmUNCj4gICAgIChpbmNvcnJlY3QgZGVzY3JpcHRvcikNCg0KRG8geW91IGhhdmUg
YW55IGxvZ3MgeW91IGNvdWxkIHNoYXJlIGZyb20gdGhlIGd1ZXN0IE9TPyBBbHNvLCB3b3VsZCB5
b3UNCmJlIGFibGUgdG8gc2hhcmUgbW9yZSBkZXRhaWxzIG9uIGhvdyB5b3UgY29uZmlndXJlIHRo
ZSBndWVzdD8gKGUuZy4geGwNCmNvbmZpZ3VyYXRpb24sIGxpYnZpcnQuLi4pPw0KDQo+DQo+ICAg
ICBXZSBuZWVkeW91ciBoZWxwdG8gZmluZCBhIHNvbHV0aW9uIGZvciB0aGVzZSBwcm9ibGVtcy4N
Cj4NCj4gICAgIFRoZSBTb2Z0d2FyZSB1c2VkIGFyZSA6DQoNCkNhbiB5b3Ugc2hhcmUgc29tZSBk
ZXRhaWxzIG9mIHRoZSBIVyB5b3UgYXJlIHVzaW5nPw0KDQo+DQo+ICAgICAtRGViaWFuIDExIHZl
cnNpb241LjEwLjAtMjANCj4NCj4gICAgIC1YZW4gdmVyc2lvbjQuMTQNCg0KQXJlIG1pbm9yIHZl
cnNpb24gb2YgWGVuIDQuMTQgYXJlIHlvdSB1c2luZz8gQW55IHBhdGNoZXMgb24gdG9wPw0KDQpY
ZW4gNC4xNCBoYXMgYWxzbyBiZWVuIG91dCBvZiBzdXBwb3J0IGZvciBhIGZldyBtb250aHMgbm93
LiBJZiB0aGF0J3MNCnBvc3NpYmxlLCBJIHdvdWxkIHJlY29tbWVuZCB0byB0cnkgYSBuZXcgWGVu
IHZlcnNpb24gKHRoZSBsYXN0IHN0YWJsZSBpcw0KNC4xOCkganVzdCBpbiBjYXNlIHlvdXIgaXNz
dWUgYXMgYmVlbiBmaXhlZCBpbiBuZXdlciByZWxlYXNlLg0KDQo+DQo+ICAgICAtV2luZG93cyAx
MCAxNjA3IGFuZCAyMUgyIGZvciB2aXJ0dWFsIG1hY2hpbmVzLiBWaXJ0dWFsIE1hY2hpbmUgSFZN
DQoNCkNoZWVycywNCg0KLS0NCkp1bGllbiBHcmFsbA0K

--_000_7d7062a0abf54fea890b38c545c459c3thalesgroupcom_
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64

PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy
bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt
YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj
cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv
VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg
Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv
ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl
PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6
IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1m
YWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAy
IDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpoZWx2ZXRpY2FuZXVlOw0KCXBhbm9zZS0x
OjAgMCAwIDAgMCAwIDAgMCAwIDA7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNvTm9y
bWFsLCBsaS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBjbTsNCgltYXJnaW4t
Ym90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGltZXMg
TmV3IFJvbWFuIixzZXJpZjt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe21zby1zdHls
ZS1wcmlvcml0eTo5OTsNCgljb2xvcjpibHVlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7
fQ0KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJe21zby1zdHlsZS1wcmlv
cml0eTo5OTsNCgljb2xvcjpwdXJwbGU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQpw
Lm1zb25vcm1hbDAsIGxpLm1zb25vcm1hbDAsIGRpdi5tc29ub3JtYWwwDQoJe21zby1zdHlsZS1u
YW1lOm1zb25vcm1hbDsNCgltc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzsNCgltYXJnaW4tcmlnaHQ6
MGNtOw0KCW1zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvOw0KCW1hcmdpbi1sZWZ0OjBjbTsNCglm
b250LXNpemU6MTIuMHB0Ow0KCWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4iLHNlcmlmO30N
CnNwYW4uRW1haWxTdHlsZTE4DQoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsLXJlcGx5Ow0KCWZv
bnQtZmFtaWx5OiJDYWxpYnJpIixzYW5zLXNlcmlmOw0KCWNvbG9yOiMxRjQ5N0Q7fQ0KLk1zb0No
cERlZmF1bHQNCgl7bXNvLXN0eWxlLXR5cGU6ZXhwb3J0LW9ubHk7DQoJZm9udC1mYW1pbHk6IkNh
bGlicmkiLHNhbnMtc2VyaWY7DQoJbXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVM7fQ0KQHBhZ2Ug
V29yZFNlY3Rpb24xDQoJe3NpemU6NjEyLjBwdCA3OTIuMHB0Ow0KCW1hcmdpbjo3MC44NXB0IDcw
Ljg1cHQgNzAuODVwdCA3MC44NXB0O30NCmRpdi5Xb3JkU2VjdGlvbjENCgl7cGFnZTpXb3JkU2Vj
dGlvbjE7fQ0KLS0+PC9zdHlsZT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlZGVm
YXVsdHMgdjpleHQ9ImVkaXQiIHNwaWRtYXg9IjEwMjYiIC8+DQo8L3htbD48IVtlbmRpZl0tLT48
IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlbGF5b3V0IHY6ZXh0PSJlZGl0Ij4NCjxv
OmlkbWFwIHY6ZXh0PSJlZGl0IiBkYXRhPSIxIiAvPg0KPC9vOnNoYXBlbGF5b3V0PjwveG1sPjwh
W2VuZGlmXS0tPg0KPC9oZWFkPg0KPGJvZHkgbGFuZz0iRlIiIGxpbms9ImJsdWUiIHZsaW5rPSJw
dXJwbGUiPg0KPGRpdiBjbGFzcz0iV29yZFNlY3Rpb24xIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi
PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkm
cXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEO21zby1mYXJlYXN0LWxhbmd1YWdlOkVOLVVT
Ij5EZWFyIFJvYiBhbmQgSnVsaWVuLDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90
O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEO21zby1mYXJlYXN0LWxhbmd1
YWdlOkVOLVVTIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9y
bWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1p
bHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVh
c3QtbGFuZ3VhZ2U6RU4tVVMiPlRoYW5rIHlvdSBmb3IgeW91ciBtZXNzYWdlcy48bzpwPjwvbzpw
Pjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5
bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5z
LXNlcmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPjxvOnA+Jm5i
c3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVO
LVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1
b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpFTi1VUyI+
RG9taW5pcXVlIGFuZCB0aGUgVGhhbGVzIGNyZXcgd2lsbCBnaXZlIHlvdSBtb3JlIGluZm9ybWF0
aW9uIHF1aWNrbHkuDQo8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6
JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3Qt
bGFuZ3VhZ2U6RU4tVVMiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250
LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RDttc28t
ZmFyZWFzdC1sYW5ndWFnZTpFTi1VUyI+QmVzdCByZWdhcmRzPG86cD48L286cD48L3NwYW4+PC9w
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNp
emU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xv
cjojMUY0OTdEO21zby1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTIj48bzpwPiZuYnNwOzwvbzpwPjwv
c3Bhbj48L3A+DQo8dGFibGUgY2xhc3M9Ik1zb05vcm1hbFRhYmxlIiBib3JkZXI9IjAiIGNlbGxz
cGFjaW5nPSIwIiBjZWxscGFkZGluZz0iMCIgc3R5bGU9ImJvcmRlci1jb2xsYXBzZTpjb2xsYXBz
ZSI+DQo8dGJvZHk+DQo8dHI+DQo8dGQgd2lkdGg9IjQ3MiIgdmFsaWduPSJ0b3AiIHN0eWxlPSJ3
aWR0aDozNjAuMHB0O3BhZGRpbmc6My4wcHQgMGNtIDEuNXB0IDBjbSI+DQo8cCBjbGFzcz0iTXNv
Tm9ybWFsIj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVv
dDtBcmlhbCZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiM3OTc5NzkiPlJlc3BvbnNhYmxlIFByb2dy
YW1tZTxvOnA+PC9vOnA+PC9zcGFuPjwvYj48L3A+DQo8L3RkPg0KPC90cj4NCjx0cj4NCjx0ZCB3
aWR0aD0iNDcyIiB2YWxpZ249InRvcCIgc3R5bGU9IndpZHRoOjM2MC4wcHQ7cGFkZGluZzouNzVw
dCAwY20gLjc1cHQgMGNtIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250
LXNpemU6MTAuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMtc2VyaWY7Y29s
b3I6Izc5Nzk3OSI+RMOpcGFydGVtZW50IFJBRkFMRSBGNDxiPjxvOnA+PC9vOnA+PC9iPjwvc3Bh
bj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48Yj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEw
LjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiM3OTc5
NzkiPlRoYWxlcyBTSVggR1RTIEZyYW5jZTxvOnA+PC9vOnA+PC9zcGFuPjwvYj48L3A+DQo8L3Rk
Pg0KPC90cj4NCjx0ciBzdHlsZT0iaGVpZ2h0OjIuNXB0Ij4NCjx0ZCB3aWR0aD0iNDcyIiB2YWxp
Z249InRvcCIgc3R5bGU9IndpZHRoOjM2MC4wcHQ7cGFkZGluZzouNzVwdCAwY20gNy41cHQgMGNt
O2hlaWdodDoyLjVwdCI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1z
aXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OyxzYW5zLXNlcmlmO2NvbG9y
OiM3OTc5NzkiPlQ6ICYjNDM7MzMoMCkgMSA0MSAzMCAyNiA2NzxvOnA+PC9vOnA+PC9zcGFuPjwv
cD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2Zv
bnQtZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6Izc5Nzk3OSI+TTog
JiM0MzszMyAoMCkgNiA4NSAyMyAxMiAzMDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjwvdGQ+DQo8
L3RyPg0KPHRyIHN0eWxlPSJoZWlnaHQ6My4wcHQiPg0KPHRkIHdpZHRoPSI0NzIiIHZhbGlnbj0i
dG9wIiBzdHlsZT0id2lkdGg6MzYwLjBwdDtib3JkZXI6bm9uZTtib3JkZXItdG9wOnNvbGlkICMw
MEJCREQgMi4yNXB0O3BhZGRpbmc6Ny41cHQgMGNtIDMuNzVwdCAwY207aGVpZ2h0OjMuMHB0Ij4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTAuMHB0O2ZvbnQt
ZmFtaWx5OiZxdW90O0FyaWFsJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6Izc5Nzk3OSI+NCBBdmVu
dWUgZGVzIExvdXZyZXNzZXM8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9y
bWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlh
bCZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiM3OTc5NzkiPjkyNjIyIEdlbm5ldmlsbGllcnM8bzpw
PjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9u
dC1zaXplOjEwLjBwdDtmb250LWZhbWlseTomcXVvdDtBcmlhbCZxdW90OyxzYW5zLXNlcmlmO2Nv
bG9yOiM3OTc5NzkiPkZyYW5jZTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjwvdGQ+DQo8L3RyPg0K
PC90Ym9keT4NCjwvdGFibGU+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9u
dC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7
Y29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpFTi1VUyI+Jm5ic3A7PG86cD48L286
cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6
ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9y
OiMxRjQ5N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPjxvOnA+Jm5ic3A7PC9vOnA+PC9z
cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6
MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZiI+RGUmbmJz
cDs6PC9zcGFuPjwvYj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTom
cXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPiBSb2IgVG93bmxleSAmbHQ7cm9iLnRvd25s
ZXlAZ21haWwuY29tJmd0OyBEZTxicj4NCjxiPkVudm95w6kmbmJzcDs6PC9iPiBkaW1hbmNoZSAy
NSBmw6l2cmllciAyMDI0IDEzOjIwPGJyPg0KPGI+w4AmbmJzcDs6PC9iPiBKdWxpZW4gR3JhbGwg
Jmx0O2p1bGllbkB4ZW4ub3JnJmd0Ozxicj4NCjxiPkNjJm5ic3A7OjwvYj4gQW5kcmV3IENvb3Bl
ciAmbHQ7YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbSZndDs7IEdPVVJMT1QgRnJhbmNvaXMgJmx0
O2ZyYW5jb2lzLmdvdXJsb3RAdGhhbGVzZ3JvdXAuY29tJmd0OzsgR2VvcmdlIER1bmxhcCAmbHQ7
Z2VvcmdlLmR1bmxhcEBjaXRyaXguY29tJmd0OzsgSmFuIEJldWxpY2ggJmx0O2piZXVsaWNoQHN1
c2UuY29tJmd0OzsgSnVlcmdlbiBHcm9zcyAmbHQ7amdyb3NzQHN1c2UuY29tJmd0OzsgS2VsbHkg
Q2hvaSAmbHQ7a2VsbHkuY2hvaUBjbG91ZC5jb20mZ3Q7OyBSb2dlcg0KIFBhdSBNb25uw6kgJmx0
O3JvZ2VyLnBhdUBjaXRyaXguY29tJmd0OzsgeGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3Jn
PGJyPg0KPGI+T2JqZXQmbmJzcDs6PC9iPiBSZTogSU1QT1JUQU5UIC0gOiBOZWVkIGhlbHAgb24g
VVNCIHBvcnQgdmlydHVhbGl6YXRpb24gd2l0aCBYZW4gaHlwZXJ2aXNvcjxvOnA+PC9vOnA+PC9z
cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPGRp
dj4NCjxkaXY+DQo8ZGl2IHN0eWxlPSJib3JkZXI6bm9uZSBibGFjayAxLjBwdDtwYWRkaW5nOjBj
bSAwY20gMGNtIDBjbTtiYWNrZ3JvdW5kLWNvbG9yOnJnYmEoMCwwLDAsMCk7d29yZC1zcGFjaW5n
OjBweCI+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6
ZToxMy41cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7aGVsdmV0aWNhbmV1ZSZxdW90OyxzZXJpZjtjb2xv
cjpibGFjaztib3JkZXI6bm9uZSB3aW5kb3d0ZXh0IDEuMHB0O3BhZGRpbmc6MGNtIj5JbiBzaG9y
dCwgcHVyY2hhc2UgYW4gYWRkb24gUENJZSBVU0IgY2FyZCB0byBpc29sYXRlIHRyYWZmaWMuJm5i
c3A7IFVTQiBpcyBhIHNoYXJlZCBtZWRpdW0gYW5kIGlzIHJvb3RlZCBpbiB0aGUgY2hlYXBlc3Qg
ZWxlY3Ryb25pY3MuDQogJm5ic3A7IEl0IHdhcyBuZXZlciBjb25zaWRlcmVkIGVudGVycHJpc2Ug
Z3JhZGUgaGFyZHdhcmUgZm9yIG1vc3QgYWxsIG9mIGhpc3RvcnkuICZuYnNwOyAmbmJzcDtUaGUg
bGV0dGVyIEIgaW4gJm5ic3A7VVNCICZuYnNwO3N0YW5kcyBmb3IgQlVTIHdoaWNoIGFjdHVhbGx5
IG1lYW5zIGEgc2hhcmVkIHJvYWR3YXkgdGhhdCBvdGhlciBVU0IgZGV2aWNlcyBjYW4gYW5kIGRv
IGltcGVkZSB0cmFmZmljLiAmbmJzcDsgV2UgZXhwZXJpZW5jZWQgaW50ZXJmZXJpbmcgdHJhZmZp
YyBvbiBpbmV4cGVuc2l2ZQ0KIGxhcHRvcHMgcnVubmluZyAmbmJzcDtYRU4gYmFzZWQgUXViZXMt
T1MuJm5ic3A7IFR1cm5lZCBvdXQgdGhlIFdpRmkgY2FyZCB3YXMgYWN0dWFsbHkgd2lyZWQgb250
byB0aGUgVVNCIGJ1cy4gJm5ic3A7IFVucGx1Z2dpbmcgYWxsIGV4dGVybmFsIFVTQiBlbGVjdHJv
bmljcyByZWR1Y2VkIGluZXhwbGljYWJsZSBXaUZpIGRpc2Nvbm5lY3RzIGFuZCByZWxpYWJseSBp
bXByb3ZlZCB0aHJvdWdocHV0LiAmbmJzcDsgJm5ic3A7SG93ZXZlciwgdGhlcmUgd2FzIG5vIHdh
eSB0byByZW1vdmUgdGhlDQoga2V5Ym9hcmQsIHRvdWNocGFkLCBhbmQgdG91Y2hzY3JlZW4gd2hp
Y2ggYWxsIHVzZWQgdGhlIFVTQiByb2Fkd2F5LiAmbmJzcDsmbmJzcDs8bzpwPjwvbzpwPjwvc3Bh
bj48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0i
Zm9udC1zaXplOjEzLjVwdDtmb250LWZhbWlseTomcXVvdDtoZWx2ZXRpY2FuZXVlJnF1b3Q7LHNl
cmlmO2NvbG9yOmJsYWNrO2JvcmRlcjpub25lIHdpbmRvd3RleHQgMS4wcHQ7cGFkZGluZzowY20i
PjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTMuNXB0O2ZvbnQtZmFtaWx5OiZxdW90
O2hlbHZldGljYW5ldWUmcXVvdDssc2VyaWY7Y29sb3I6YmxhY2s7Ym9yZGVyOm5vbmUgd2luZG93
dGV4dCAxLjBwdDtwYWRkaW5nOjBjbSI+VGhvc2Ugd2VyZSBjaGVhcCBsYXB0b3BzLCBidXQgZXZl
biZuYnNwO2VudGVycHJpc2UgY2xhc3MgaGFyZHdhcmUgc3VjaCBhcyBIUEUgJm5ic3A7YW5kIERl
bGwgU2VydmVycyBydW5uaW5nIFhDUC1ORyBkaWQgbm90IGhhdmUNCiBncmVhdCBzcGVlZHMuICZu
YnNwOyBCdXQgaW4gdGhpcyBzY2VuYXJpbywgd2UgY291bGQgc2ltcGxpZnkgdGhlIHByb2JsZW0g
YW5kICZuYnNwO2lzb2xhdGUgdGhlIGlzc3VlIGJ5Jm5ic3A7cHVyY2hhc2luZyBhICZuYnNwO25l
dyBhZGQtb24mbmJzcDtQQ0llIGNhcmQgd2l0aCBVU0IgcG9ydHMgYW5kIHRoZW4gYXNzaWduIHRo
ZSBlbnRpcmUgUENJZSBjYXJkIHRvIHRoZSBWTS4gJm5ic3A7IFRoZSBWTSB3b3VsZCB0aGVuIGhh
dmUgZW50aXJlIGNvbnRyb2wgb2YgdGhlIFBDSWUgY2FyZCBhbmQgaXNvbGF0ZQ0KIHRyYWZmaWMg
dG8gZGV2aWNlcyBwbHVnZ2VkIGludG8gdGhhdCBjYXJkLiAmbmJzcDs8bzpwPjwvbzpwPjwvc3Bh
bj48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0i
Zm9udC1zaXplOjEzLjVwdDtmb250LWZhbWlseTomcXVvdDtoZWx2ZXRpY2FuZXVlJnF1b3Q7LHNl
cmlmO2NvbG9yOmJsYWNrO2JvcmRlcjpub25lIHdpbmRvd3RleHQgMS4wcHQ7cGFkZGluZzowY20i
PjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTMuNXB0O2ZvbnQtZmFtaWx5OiZxdW90
O2hlbHZldGljYW5ldWUmcXVvdDssc2VyaWY7Y29sb3I6YmxhY2s7Ym9yZGVyOm5vbmUgd2luZG93
dGV4dCAxLjBwdDtwYWRkaW5nOjBjbSI+SSBkaWQgbm90IGtub3cgdW50aWwgbGF0ZXIgdGhhdCBX
aXJlc2hhcmsgaGFzIHBsdWdpbnMgZm9yIHNuaWZmaW5nIFVTQiB0cmFmZmljLCBidXQgbXkgbmFt
ZSBpcyAmbmJzcDtub3QgSGVpc2VuYmVyZyBhbmQgbm90DQogc3VyZSBpZiB0aGF0IHRyYWZmaWMg
bW9uaXRvcmluZyBkb2VzIG5vdCBhbHNvIHNsb3cgZG93biBvdGhlciBkZXZpY2VzLiAmbmJzcDs8
bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjxk
aXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxkaXY+DQo8
ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+T24gU2F0LCBGZWIgMjQsIDIwMjQgYXQgODozOSBB
TSBKdWxpZW4gR3JhbGwgJmx0OzxhIGhyZWY9Im1haWx0bzpqdWxpZW5AeGVuLm9yZyI+anVsaWVu
QHhlbi5vcmc8L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGJsb2NrcXVv
dGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFk
ZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tcmlnaHQ6MGNt
Ij4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtYXJnaW4tYm90dG9tOjEyLjBwdCI+SGkg
RG9taW5pcXVlIGFuZCBGcmFuY29pcyw8YnI+DQo8YnI+DQpUaGFuayB5b3UgZm9yIHRoZSByZXBv
cnQuIEkgYW0gQ0Npbmcgc29tZSBwZXJzb24gd2hvIEkgdGhpbmsgY291bGQgaGVscC48YnI+DQo8
YnI+DQpJIGFsc28gaGF2ZSBzb21lIHF1ZXN0aW9ucyAoc2VlIGJlbG93KS48YnI+DQo8YnI+DQpP
biAyMy8wMi8yMDI0IDE2OjAwLCBHT1VSTE9UIEZyYW5jb2lzIHdyb3RlOjxicj4NCiZndDsgRGVh
ciBBbGwsPGJyPg0KJmd0OyA8YnI+DQomZ3Q7IFdlIHNlbmQgeW91IGEgbWVzc2FnZSBmZXcgZGF5
cyBhZ28uPGJyPg0KJmd0OyA8YnI+DQomZ3Q7IFdlIGhhdmUgbWFqb3IgcGVyZm9ybWFuY2Ugd2l0
aCBYRU4gVVNCIERyaXZlcnMuIFdlIHVzZSB5b3VyIGRyaXZlciB0byA8YnI+DQomZ3Q7IGxvYWQg
ZGF0YSBpbiBhIGRldmljZSBhbmQgd2UgbmVlZCBVU0IzIEhpZ2ggU3BlZWQgVVNCIHByb3RvY29s
ZTxicj4NCiZndDsgPGJyPg0KJmd0OyBEbyB5b3UgaGF2ZSBhIG5ldyBpc3N1ZSBvZiB0aGUgSENJ
IGFuZCBORUMgVVNCIFhlbiBEcml2ZXIgPzxicj4NCnA8YnI+DQpbLi4uXTxicj4NCjxicj4NCiZn
dDsgKkRlJm5ic3A7OipMQVJSSUVVIERvbWluaXF1ZSAmbHQ7PGEgaHJlZj0ibWFpbHRvOmRvbWlu
aXF1ZS5sYXJyaWV1QHRoYWxlc2dyb3VwLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmRvbWluaXF1ZS5s
YXJyaWV1QHRoYWxlc2dyb3VwLmNvbTwvYT4mZ3Q7PGJyPg0KJmd0OyAqRW52b3nDqSZuYnNwOzoq
IGpldWRpIDIyIGbDqXZyaWVyIDIwMjQgMTM6NTg8YnI+DQomZ3Q7ICrDgCZuYnNwOzoqIDxhIGhy
ZWY9Im1haWx0bzp4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmciIHRhcmdldD0iX2JsYW5r
Ij54ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc8L2E+PGJyPg0KJmd0OyAqQ2MmbmJzcDs6
KiBXSUxMRU1TIExvdWlzICZsdDs8YSBocmVmPSJtYWlsdG86bG91aXMud2lsbGVtc0B0aGFsZXNn
cm91cC5jb20iIHRhcmdldD0iX2JsYW5rIj5sb3Vpcy53aWxsZW1zQHRoYWxlc2dyb3VwLmNvbTwv
YT4mZ3Q7OyBHUlVPIE5pY29sYXMNCjxicj4NCiZndDsgJmx0OzxhIGhyZWY9Im1haWx0bzpuaWNv
bGFzLW4uZ3J1b0B0aGFsZXNncm91cC5jb20iIHRhcmdldD0iX2JsYW5rIj5uaWNvbGFzLW4uZ3J1
b0B0aGFsZXNncm91cC5jb208L2E+Jmd0OzsgR09VUkxPVCBGcmFuY29pcw0KPGJyPg0KJmd0OyAm
bHQ7PGEgaHJlZj0ibWFpbHRvOmZyYW5jb2lzLmdvdXJsb3RAdGhhbGVzZ3JvdXAuY29tIiB0YXJn
ZXQ9Il9ibGFuayI+ZnJhbmNvaXMuZ291cmxvdEB0aGFsZXNncm91cC5jb208L2E+Jmd0OzsgS2Vs
bHkgQ2hvaSAmbHQ7PGEgaHJlZj0ibWFpbHRvOmtlbGx5LmNob2lAY2xvdWQuY29tIiB0YXJnZXQ9
Il9ibGFuayI+a2VsbHkuY2hvaUBjbG91ZC5jb208L2E+Jmd0Ozxicj4NCiZndDsgKk9iamV0Jm5i
c3A7OiogTmVlZCBoZWxwIG9uIFVTQiBwb3J0IHZpcnR1YWxpemF0aW9uIHdpdGggWGVuIGh5cGVy
dmlzb3I8YnI+DQomZ3Q7ICpDcml0w6hyZSBkZSBkaWZmdXNpb24mbmJzcDs6KiBDb25maWRlbnRp
ZWw8YnI+DQomZ3Q7IDxicj4NCiZndDsmbmJzcDsgJm5ic3A7ICZuYnNwO0RlYXIgYWxsLDxicj4N
CiZndDsgPGJyPg0KJmd0OyZuYnNwOyAmbmJzcDsgJm5ic3A7V2UgYXJlIGRldGVjdGluZ3NldmVy
YWwgaXNzdWVzIHdpdGggVVNCIHBvcnQgdmlydHVhbGl6YXRpb24gd2l0aCB0aGU8YnI+DQomZ3Q7
Jm5ic3A7ICZuYnNwOyAmbmJzcDtYZW4gaHlwZXJ2aXNvci48YnI+DQo8YnI+DQpKdXN0IHRvIGNs
YXJpZnksIHlvdSBhcmUgYXNzaWduaW5nIHRoZSBQQ0kgVVNCIGJ1cyAocmF0aGVyIHRoYW4ganVz
dCB0aGUgPGJyPg0KVVNCIGRldmljZSkgdG8gdGhlIGd1ZXN0LiBJcyB0aGF0IGNvcnJlY3Q/PGJy
Pg0KPGJyPg0KSWYgc28uLi48YnI+DQo8YnI+DQombmJzcDsmZ3Q7PGJyPg0KJm5ic3A7Jmd0OyBE
byB5b3UgaGF2ZSBhIG5ldyBpc3N1ZSBvZiB0aGUgSENJIGFuZCBORUMgVVNCIFhlbiBEcml2ZXIg
Pzxicj4NCjxicj4NCi4uLiBJIGFtIG5vdCBzdXJlIHdoYXQgWGVuIGRyaXZlciB5b3UgYXJlIHRh
bGtpbmcgYWJvdXQuIElmIHRoZSBVU0IgYnVzIDxicj4NCmlzIGFzc2lnbmVkIHRvIHRoZSBndWVz
dCwgdGhlbiBJIHdvdWxkIGV4cGVjdCBubyBzcGVjaWZpYyBYZW4gZHJpdmVyIGFyZSA8YnI+DQpy
ZXF1aXJlZC4gRG8geW91IGhhdmUgYW55IHBvaW50ZXIgdG8gdGhlIGV4aXN0aW5nIG9uZT88YnI+
DQo8YnI+DQomZ3Q7Jm5ic3A7ICZuYnNwOyAmbmJzcDstIFdlIGNhbm5vdCBkbyBQQ0kgcGFzc3Ro
cm91Z2ggb2YgdGhlIFBDSSB1c2IgYnVzIG9uIGEgV2luZG93cyAxMDxicj4NCiZndDsmbmJzcDsg
Jm5ic3A7ICZuYnNwOzE2MDcgNjQtYml0IHZpcnR1YWwgbWFjaGluZS4gVGhlIGJhZCByZXN1bHQg
aXMgYSBXaW5kb3dzIGJsdWUgc2NyZWVuLjxicj4NCiZndDsgPGJyPg0KJmd0OyZuYnNwOyAmbmJz
cDsgJm5ic3A7LSBXaGVuIHdlIHVzZSB0aGUgcGFzc3Rocm91Z2ggZnVuY3Rpb25hbGl0eSBvbiBh
IFdpbmRvd3MgMjFIMjxicj4NCiZndDsmbmJzcDsgJm5ic3A7ICZuYnNwO3ZpcnR1YWwgbWFjaGlu
ZSwgd2Ugbm90aWNlIHRoYXR0aGUgc3BlZWQgb2YgdGhlIFVTQiBwb3J0IGlzIG5vdCBoaWdoPGJy
Pg0KJmd0OyZuYnNwOyAmbmJzcDsgJm5ic3A7c3BlZWQgYnV0IGZ1bGwgc3BlZWQgb24gYSBVU0Ig
My4wIHBvcnQ8YnI+DQomZ3Q7IDxicj4NCiZndDsmbmJzcDsgJm5ic3A7ICZuYnNwOy0gV2Ugbm90
aWNlIGluc3RhYmlsaXRpZXMgd2hlbiB1c2luZyB0aGUgbmVjLXVzYi14aGNpIGRyaXZlciwgJm5i
c3A7VVNCPGJyPg0KJmd0OyZuYnNwOyAmbmJzcDsgJm5ic3A7Mi4wIGtleXMgYXJlIG5vdCByZWNv
Z25pemVkIGJ5IHRoZSBXaW5kb3dzIHZpcnR1YWwgbWFjaGluZTxicj4NCiZndDsmbmJzcDsgJm5i
c3A7ICZuYnNwOyhpbmNvcnJlY3QgZGVzY3JpcHRvcik8YnI+DQo8YnI+DQpEbyB5b3UgaGF2ZSBh
bnkgbG9ncyB5b3UgY291bGQgc2hhcmUgZnJvbSB0aGUgZ3Vlc3QgT1M/IEFsc28sIHdvdWxkIHlv
dSA8YnI+DQpiZSBhYmxlIHRvIHNoYXJlIG1vcmUgZGV0YWlscyBvbiBob3cgeW91IGNvbmZpZ3Vy
ZSB0aGUgZ3Vlc3Q/IChlLmcuIHhsIDxicj4NCmNvbmZpZ3VyYXRpb24sIGxpYnZpcnQuLi4pPzxi
cj4NCjxicj4NCiZndDsgPGJyPg0KJmd0OyZuYnNwOyAmbmJzcDsgJm5ic3A7V2UgbmVlZHlvdXIg
aGVscHRvIGZpbmQgYSBzb2x1dGlvbiBmb3IgdGhlc2UgcHJvYmxlbXMuPGJyPg0KJmd0OyA8YnI+
DQomZ3Q7Jm5ic3A7ICZuYnNwOyAmbmJzcDtUaGUgU29mdHdhcmUgdXNlZCBhcmUgOjxicj4NCjxi
cj4NCkNhbiB5b3Ugc2hhcmUgc29tZSBkZXRhaWxzIG9mIHRoZSBIVyB5b3UgYXJlIHVzaW5nPzxi
cj4NCjxicj4NCiZndDsgPGJyPg0KJmd0OyZuYnNwOyAmbmJzcDsgJm5ic3A7LURlYmlhbiAxMSB2
ZXJzaW9uNS4xMC4wLTIwPGJyPg0KJmd0OyA8YnI+DQomZ3Q7Jm5ic3A7ICZuYnNwOyAmbmJzcDst
WGVuIHZlcnNpb240LjE0PGJyPg0KPGJyPg0KQXJlIG1pbm9yIHZlcnNpb24gb2YgWGVuIDQuMTQg
YXJlIHlvdSB1c2luZz8gQW55IHBhdGNoZXMgb24gdG9wPzxicj4NCjxicj4NClhlbiA0LjE0IGhh
cyBhbHNvIGJlZW4gb3V0IG9mIHN1cHBvcnQgZm9yIGEgZmV3IG1vbnRocyBub3cuIElmIHRoYXQn
cyA8YnI+DQpwb3NzaWJsZSwgSSB3b3VsZCByZWNvbW1lbmQgdG8gdHJ5IGEgbmV3IFhlbiB2ZXJz
aW9uICh0aGUgbGFzdCBzdGFibGUgaXMgPGJyPg0KNC4xOCkganVzdCBpbiBjYXNlIHlvdXIgaXNz
dWUgYXMgYmVlbiBmaXhlZCBpbiBuZXdlciByZWxlYXNlLjxicj4NCjxicj4NCiZndDsgPGJyPg0K
Jmd0OyZuYnNwOyAmbmJzcDsgJm5ic3A7LVdpbmRvd3MgMTAgMTYwNyBhbmQgMjFIMiBmb3Igdmly
dHVhbCBtYWNoaW5lcy4gVmlydHVhbCBNYWNoaW5lIEhWTTxicj4NCjxicj4NCkNoZWVycyw8YnI+
DQo8YnI+DQotLSA8YnI+DQpKdWxpZW4gR3JhbGw8bzpwPjwvbzpwPjwvcD4NCjwvYmxvY2txdW90
ZT4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRtbD4NCg==

--_000_7d7062a0abf54fea890b38c545c459c3thalesgroupcom_--


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:29:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:29:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685387.1066002 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXIv-00064H-8j; Mon, 26 Feb 2024 09:29:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685387.1066002; Mon, 26 Feb 2024 09:29:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXIv-00064A-6C; Mon, 26 Feb 2024 09:29:45 +0000
Received: by outflank-mailman (input) for mailman id 685387;
 Mon, 26 Feb 2024 09:29:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reXIt-00062o-Rj
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:29:43 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 922eddaa-d489-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 10:29:42 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a3f829cde6dso312113066b.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 01:29:42 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 k18-20020a1709060cb200b00a413d1eda1bsm2239532ejh.49.2024.02.26.01.29.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 01:29:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 922eddaa-d489-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708939782; x=1709544582; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vdT5rVxCzzQei2lQdS7rNXeFjGJuJqcgwBCQqylrChU=;
        b=XJYlDagxohw9LlvRiNhwRqjT3ylWU+k+3it+fwSVYrNBKjbxX/DcmBGZDvpISryGRO
         HlMsl1+iNABwzxtOwCp0y1RoioTdBgV67eHB7t/jEq5tPU9k2A8gVyWADL56Nj5XZXEb
         idJcTfZdxnV2eKDz4z/WJ/hevrppf/4jUWJOu0vl4i8VHjeKOb50PeIxdVGA5ETdCH/x
         7g416DjNw5SWijL8iVdawgNdcROUy/OvaF2UnaOpdxzMydMP2eshQONoZXrk11LOF/Xy
         xpG0GhNWYf5pdVyQqFVMaqij8kXjdDRDbtyE/jtiFpLoWV2BSHDIILVPnkoSBj4/z3sT
         2m3A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708939782; x=1709544582;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vdT5rVxCzzQei2lQdS7rNXeFjGJuJqcgwBCQqylrChU=;
        b=LHgieyUZUhh8LWVqyBawDuFCeEttirsCF6UEgEG+mR8J70Huw/AUy7Hu18rAoGCgLw
         0oHvWrYiI7id8U7SQrEvi59oAfqJWVmrZb+tyv/6/7L3gJP5MsmTnTUEtCw5di6SxycW
         kf1RGbVtF5Yl3fd52tTnPE+0gaBBC8tX9NHlNpGv8doBerD6aZE1CgrFi98m+SpEgqzT
         glXceEsRDSwUoHJ4RJtVRI9n10YL/ZdQiZ8H4GbEut1EbYd+sPLDVQ3FshyFKKTrWCh4
         m5mwG9Zg4UDtuTO62ArJgRVNqp9r4jpRBZTU4dQa1mvI8OMq2esGOKg+/giXRsqGUEaX
         vi3A==
X-Forwarded-Encrypted: i=1; AJvYcCXLZV+KEZqPNvDbxdhgJUZj95E2miy3iuHN7YNk/UzRBUmLhGe0uUD4AOB0RvW5ZIFMCMZ4sWYQwIojBK7eo8DKgUmIy/uCwfztZAYtM4o=
X-Gm-Message-State: AOJu0Ywvpb4IcHtF/KST7iRP33u0vkM1keZM6fzPzYSoTPPbJVrtzejE
	uD1IiRFOsvbra05zYQkBx5uFpR0T5RGLJfYfMjuw/xVRkKRCI3eQdZxBERt/hg==
X-Google-Smtp-Source: AGHT+IFIkBqX5QJW2uWrAd6horu3G0GLtdsDsQXWsybY3H5M4MLkbl86e5StmM3T+09sIctF60Kgeg==
X-Received: by 2002:a17:906:39db:b0:a3f:173a:224c with SMTP id i27-20020a17090639db00b00a3f173a224cmr4629258eje.51.1708939781989;
        Mon, 26 Feb 2024 01:29:41 -0800 (PST)
Message-ID: <c3657c92-5e44-4919-9b95-4ce051b72d0b@suse.com>
Date: Mon, 26 Feb 2024 10:29:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/2] x86/hpet: use an atomic add instead of a cmpxchg loop
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-3-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240222090530.62530-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.02.2024 10:05, Roger Pau Monne wrote:
> The usage of a cmpxchg loop in hpet_get_channel() is unnecessary, as the same
> can be achieved with an atomic increment, which is both simpler to read, and
> avoid any need for a loop.
> 
> Note there can be a small divergence in the channel returned if next_channel
> overflows, but returned channel will always be in the [0, num_hpets_used)
> range, and that's fine for the purpose of balancing HPET channels across CPUs.
> This is also theoretical, as there's no system currently with 2^32 CPUs (as
> long as next_channel is 32bit width).
> 
> Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>

Hmm, I'm sorry - it's now me who is recorded as the author of the patch,
for my script not finding any Signed-off-by: here (note the typo).

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:33:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:33:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685393.1066013 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXMJ-0008L8-OM; Mon, 26 Feb 2024 09:33:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685393.1066013; Mon, 26 Feb 2024 09:33:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXMJ-0008Kz-KI; Mon, 26 Feb 2024 09:33:15 +0000
Received: by outflank-mailman (input) for mailman id 685393;
 Mon, 26 Feb 2024 09:33:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reXMI-0008K6-8u
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:33:14 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0fc61232-d48a-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 10:33:13 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a3ed9cae56fso443169666b.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 01:33:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 nb18-20020a1709071c9200b00a3e45a18a5asm2239124ejc.61.2024.02.26.01.33.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 01:33:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0fc61232-d48a-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708939992; x=1709544792; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=kYDgKBssLHkQ7Sh/vf+CUOrbYp1EF2ywVOlxW+i5b9M=;
        b=X30M8JO0atCMQeA0/7yqHlXucWLwW/+9wev4vgkMNLxzv9FdVXivMmF3QX9lTLhBFt
         bfjRVelluY5DCPdhJKFBlqNavEFUz28ILkoRuTX8zNM1CJUomi3mkmZPbPv9HN7iFBvz
         df6mw0tVLZ5yk8+8B1ek9MgEjb7sdops4aNlatYXmimmbl8H4Xo3VJEf9roqwFyvHzsi
         lMCkjgBTNs41+bii9SNctFIoQ3pTojzJthKoAlVLwa6hXAzUY55Yg7a4y7kRTJGIGtk8
         raB4M2RIvJHJpaofjzqeTdvKrGdz/7VZzkrPi39UBy084k7avgbdKmp7jwCNyJuqqUb3
         P5PA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708939992; x=1709544792;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kYDgKBssLHkQ7Sh/vf+CUOrbYp1EF2ywVOlxW+i5b9M=;
        b=H2kaOEpvfsDmO9jLsXrQhQDr+GypKRxUaA2/ZcOubXWsr+8o9ey79WLAKZklcJKWAw
         mBkE+qpLvXYee3CQUko7eB1+4bTgBacrIbExhJjf8ujzdoNSHP7AxSq95JtEb+hWT5py
         dzZi/SfZxCZNrbUrS6+E1uFiLSehztAcqxhvMJnz5nW726ihZ5j2q9i03pje1pCvQ1+c
         tsUh8CS1UjjhV5kmC1tp9uDJhx+SLaa1wTXKGSDmiuNKswV+hUfRsoS4EJsbYsHH5NQy
         1434WlfGZ4+p7hrMLiLDNxrG3vsoJhmUBNFw5dWaue0vafLOf1h/c6zJn6eD/A768sim
         rEGA==
X-Forwarded-Encrypted: i=1; AJvYcCV0gpUkRC7flRDHRk1hg040KhEP3UhM+lcTsUHkIuBKmXc4wDqUBDrfF9hdM647exY+PfPDSxpKrPWBMUal1ySCQtQ75bb7kscJ1anqozs=
X-Gm-Message-State: AOJu0YyIbBeaJ0skJEBZgtP3AwlqBhzbMOS16F3KwxjUUlC4u61bKL8W
	JThNNWV5EWmlUdJcJgxKcaJ+XEtJ3OT40xZwcte1+JgjNP0bEbhZHV6L75s8KQ==
X-Google-Smtp-Source: AGHT+IFNSKtUzNvkyQ8lVMaHPBT2fNKLD39TF/xfYVUTfKt0Eq1oWXdghil6rBJ9ZQ35gNJa/NNkdA==
X-Received: by 2002:a17:906:2844:b0:a3f:adad:9ce1 with SMTP id s4-20020a170906284400b00a3fadad9ce1mr5401009ejc.18.1708939992622;
        Mon, 26 Feb 2024 01:33:12 -0800 (PST)
Message-ID: <076cac3e-097a-4994-b68f-888b491c82a7@suse.com>
Date: Mon, 26 Feb 2024 10:33:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/3] x86/spec: fix BRANCH_HARDEN option to only be set
 when build-enabled
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240223120616.73503-1-roger.pau@citrix.com>
 <20240223120616.73503-3-roger.pau@citrix.com>
 <3dcb9afe-7f3f-495c-b182-b4242b4cb337@suse.com> <ZdxVQRMRyQOEc581@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZdxVQRMRyQOEc581@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 10:09, Roger Pau Monné wrote:
> On Mon, Feb 26, 2024 at 09:42:58AM +0100, Jan Beulich wrote:
>> On 23.02.2024 13:06, Roger Pau Monne wrote:
>>> --- a/xen/arch/x86/spec_ctrl.c
>>> +++ b/xen/arch/x86/spec_ctrl.c
>>> @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
>>>  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
>>>  int8_t __read_mostly opt_eager_fpu = -1;
>>>  int8_t __read_mostly opt_l1d_flush = -1;
>>> -static bool __initdata opt_branch_harden = true;
>>> +static bool __initdata opt_branch_harden =
>>> +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
>>>  
>>>  bool __initdata bsp_delay_spec_ctrl;
>>>  uint8_t __read_mostly default_xen_spec_ctrl;
>>> @@ -268,7 +269,14 @@ static int __init cf_check parse_spec_ctrl(const char *s)
>>>          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
>>>              opt_l1d_flush = val;
>>>          else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
>>> +        {
>>> +#ifdef CONFIG_SPECULATIVE_HARDEN_BRANCH
>>>              opt_branch_harden = val;
>>> +#else
>>> +            no_config_param("SPECULATIVE_HARDEN_BRANCH", "spec-ctrl", s, ss);
>>> +            rc = -EINVAL;
>>> +#endif
>>> +        }
>>>          else if ( (val = parse_boolean("srb-lock", s, ss)) >= 0 )
>>>              opt_srb_lock = val;
>>>          else if ( (val = parse_boolean("unpriv-mmio", s, ss)) >= 0 )
>>
>> While looking at patch 3 I noticed another inconsistency: Wouldn't
>> "Compiled-in support:" better also enumerate HARDEN_BRANCH then, just
>> like for thunks both CONFIG_* state and actual runtime choice are
>> logged?
> 
> Yes, I guess we would also need to expand "Compiled-in support:" to
> include HARDEN_ARRAY and HARDEN_GUEST_ACCESS.
> 
>> Or alternatively, should logging of thunk runtime choice be
>> suppressed when the Kconfig setting is off?
> 
> Hm, I think printing "BTI-Thunk N/A" is good enough when the thunk has
> been built time disabled.

Good enough - yes. But redundant with the other log line. And since all
of this output is getting longer and longer anyway, omitting whatever
can sensibly be omitted might be at least worth considering.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:36:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:36:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685395.1066023 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXPM-0000Te-5s; Mon, 26 Feb 2024 09:36:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685395.1066023; Mon, 26 Feb 2024 09:36:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXPM-0000TX-1h; Mon, 26 Feb 2024 09:36:24 +0000
Received: by outflank-mailman (input) for mailman id 685395;
 Mon, 26 Feb 2024 09:36:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5KjY=KD=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1reXPL-0000TR-3T
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:36:23 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on20629.outbound.protection.outlook.com
 [2a01:111:f400:7e88::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 801b2452-d48a-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 10:36:21 +0100 (CET)
Received: from DM5PR07CA0103.namprd07.prod.outlook.com (2603:10b6:4:ae::32) by
 CH3PR12MB8725.namprd12.prod.outlook.com (2603:10b6:610:170::14) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.34; Mon, 26 Feb
 2024 09:36:18 +0000
Received: from DS3PEPF000099E0.namprd04.prod.outlook.com
 (2603:10b6:4:ae:cafe::fb) by DM5PR07CA0103.outlook.office365.com
 (2603:10b6:4:ae::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend
 Transport; Mon, 26 Feb 2024 09:36:18 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS3PEPF000099E0.mail.protection.outlook.com (10.167.17.203) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 26 Feb 2024 09:36:18 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb
 2024 03:36:17 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb
 2024 03:36:17 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 26 Feb 2024 03:36:16 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 801b2452-d48a-11ee-8a58-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Jk00NATOWNnFmGXLhzoRlt3HkvcycrYCF7+ZGjHkkAaTrE/XwWVweVOo0m/OFZuNIyyXza+YlRUYtXIG51sYaEYSvEPHbXqXnPLbOJQkD/P2PKS7zbKkO1MnZHEsTEZBsoKIsiYTxkrLRJgaLHG5/7EHcF0R4gjMTG0QFp3C0l7W8v+U8m45ANLrWrtPPQRjuviriqLl86J9dQBweTyZMJmCvDFaCRKJG8Mf3UdftQ1nbSn9MF1kg9PIpq22susOZbWxCPZdRvjRko+3x2ih4taSxyIHIZqrBwsU3JHZYeYCxQtIWK+cqb7bxE5CVBNMOK7EAgqBoLKx5NS3hD6SaQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=taLG4QqoH00++xQxst8hZ3E49YmE4cpGqdAFRdF/X78=;
 b=l3074/+6gJoipYfZrIDKyF+jWUk9CP+CKo+HW5SUPYbQqG5q1EzxtsoERxDGG91vJlMF5091nEBGOUgbcUC57crF96T2BVteCrF0r1dCofYfMCsy/V5ByBxNw/CsCmlRfAvhTpMAmBKHJv+1PhnReiK9/qZGwRJIYuxOq2vrCpd3WCVNwoZI3d9ElCAJPtvsO5VbwOILcE1qWin/s3vH4uI3/6PmBpV6rk29KLSR4wdw1fitCYtJMQfhenmVHECVaVkv7JViwRHyqAYZrI8EoNhUMT/0oDuqLJI7kwrHuXGCt0T8Ge+Y+ulHAXN1p/Bkt199QtHI0R+66btK46MyEw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=taLG4QqoH00++xQxst8hZ3E49YmE4cpGqdAFRdF/X78=;
 b=ysVuEukFCu+amSa4A+uqGjbTQY6Q460G2R16eCaQzRMmO1ykAsYrWUS/UZTudzLx/YHuX66235/EyEV4UDMQooclngJwAA/MABEfKtAqfj0emQKxxxOI+HSAYTdAR1/iYjexWR+QCqi0LqVVcY3r2zF26Tf3+BWJ2m2wnezxi+A=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <de4c6988-7051-4d44-9b44-e850ad4b4cc1@amd.com>
Date: Mon, 26 Feb 2024 10:36:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Set correct per-cpu cpu_core_mask
Content-Language: en-US
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240226030146.179465-1-xin.wang2@amd.com>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240226030146.179465-1-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF000099E0:EE_|CH3PR12MB8725:EE_
X-MS-Office365-Filtering-Correlation-Id: 3c8199e3-997e-4961-7238-08dc36ae6253
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4JRKg8l5PnhvH0H37Fg9wevagcO3amLFsELLkfZQLwtiO1Qn0BjVjc10g9C5gxsh4QF1nucdiI0XCPW47aAUc4ZUTvbHnX8Kr2yxkTDY4jUfPaosetz+igepkEhUVaN0NDr28D/ZVEqjNj9271Y8mgKHioP8pbSsG9NwMJnYFZoMXUZew0Kbv9xLSRVeyrRPOE7GaV019dUEzuLCF8Ao+g34DoPBrrCsZFOWvLMwnoHa+HMZToESRrcPb2Qep1L3ZyGiC0aWkTGxsCw8u1mPD5AOYgTbXyMhP+LVYzP152LHRQ9mUOgtDiPpPujVnIRUixf8k06eQT8WfAEypUCU2io85iBWcVbjhAAQqSBYd5zmUNfW600NUmpflBjD3SNxI9nUKmaSBUtle6mzyXwoYLdSwSri3nFLJY1Y393vyptTQ34ri9f1HqpQCT247vilq2qEY3sTvnfHLLUXOiVUFcJ9XU//9WWFKSIZH5tQ26d7jR1WWA5eQr9DWzyzhnzqlttMF1oRUxF/rmeAWWyop0LwHMhk553FTLkDLx7dLxqH8jBDNaQAoEsW6QpNFOCpV+SZFsqYVDN8TgjYkuiaOyImT+j1C2EaNdasUd+vZLKXe9U6gjJLGwqIhRw+F27AsKrIx2BGtI9C5YKnjpRBS515iW5C/idnvhDua6vnMHuw7NdAcmtbaB/fuKDdISyyFC0Vp1mmDGJM+pb/RAFZvwB1sS9/YY3zIZAj31eGsws4nwUZ0DEApI89euiRkALb
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 09:36:18.0831
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3c8199e3-997e-4961-7238-08dc36ae6253
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF000099E0.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8725

Hi Henry,

On 26/02/2024 04:01, Henry Wang wrote:
> In the common sysctl command XEN_SYSCTL_physinfo, the value of
> cores_per_socket is calculated based on the cpu_core_mask of CPU0.
> Currently on Arm this is a fixed value 1 (can be checked via xl info),
> which is not correct. This is because during the Arm CPU online
> process at boot time, setup_cpu_sibling_map() only sets the per-cpu
> cpu_core_mask for itself.
> 
> cores_per_socket refers to the number of cores that belong to the same
> socket (NUMA node). Currently Xen on Arm does not support physical
> CPU hotplug and NUMA. Therefore if the MT bit (bit 24) in MPIDR_EL1
> is 0, cores_per_socket means all possible CPUs detected from the device
> tree. Setting the per-cpu cpu_core_mask in setup_cpu_sibling_map()
> accordingly. Drop the in-code comment which seems to be outdated.
> 
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
NIT: You first sent this patch as part of NUMA series:
https://lore.kernel.org/xen-devel/20231120025431.14845-16-Henry.Wang@arm.com/
Shouldn't you retain the Arm's authorship?

> ---
>  xen/arch/arm/smpboot.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index a84e706d77..d616778655 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -66,7 +66,6 @@ static bool cpu_is_dead;
>  
>  /* ID of the PCPU we're running on */
>  DEFINE_PER_CPU(unsigned int, cpu_id);
> -/* XXX these seem awfully x86ish... */
>  /* representing HT siblings of each logical CPU */
>  DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
>  /* representing HT and core siblings of each logical CPU */
> @@ -89,6 +88,11 @@ static int setup_cpu_sibling_map(int cpu)
>      cpumask_set_cpu(cpu, per_cpu(cpu_sibling_mask, cpu));
>      cpumask_set_cpu(cpu, per_cpu(cpu_core_mask, cpu));
>  
> +    /* PE not implemented using a multithreading type approach. */
> +    if ( system_cpuinfo.mpidr.mt == 0 )
Do we need this check? It mt was true, cpu_sibling_mask would be incorrect anyway (it would still be 1).

~Michal


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:45:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:45:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685400.1066032 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXY0-0003Ed-Vr; Mon, 26 Feb 2024 09:45:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685400.1066032; Mon, 26 Feb 2024 09:45:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXY0-0003EW-Sy; Mon, 26 Feb 2024 09:45:20 +0000
Received: by outflank-mailman (input) for mailman id 685400;
 Mon, 26 Feb 2024 09:45:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reXXz-0003EQ-Oh
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:45:19 +0000
Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com
 [2a00:1450:4864:20::634])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bf9b72d8-d48b-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 10:45:17 +0100 (CET)
Received: by mail-ej1-x634.google.com with SMTP id
 a640c23a62f3a-a3ed9cae56fso445226866b.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 01:45:17 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 bi7-20020a170907368700b00a3e9c4fa24esm2247064ejc.96.2024.02.26.01.45.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 01:45:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf9b72d8-d48b-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708940717; x=1709545517; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=WNtFkL+i/w+HtYgKZwh/iS0rpNu4sVr5A4aQuXUc1Ok=;
        b=QPH+Sq0AdEd7yUr4sLEFdQCz8jLlvvSRyTQR01ASR6lpVoX9M5jGjkLYuFlIp7Qaxv
         1tORZqB98qgpIYV49wWF8VHrbwOhrHApc+d/4qTz5wXPHM4KYqHrzEpzcwIFHkBCzZEn
         BT/ds9ISGTQVbeLnRZnGjixVf3W0nwmm7eSkRnRgxf3kXy1xouEnSwOqkBfY99C5LY1/
         gzP+tVF5HUbmV9DJ+f1HVIkmH+00M0/UP8idH8itbBRfezkFJYHkKraIQ9SdVMY7hkkh
         oyinlhbrml73kMHBwTfhhFVDO+TvEnpRELGzTYaXWV6iXmJDvXVx1VOCBU+RJLB8gDHP
         WqtQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708940717; x=1709545517;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=WNtFkL+i/w+HtYgKZwh/iS0rpNu4sVr5A4aQuXUc1Ok=;
        b=dQk7dm0x9o6C2IMo78DHrRDbzW3qvLzjLYXWQTtnOCy/fyrpIrTaTgP7hUVjoIVjx4
         cWPbl5nDAbZvgqbY1eIue50wWKP5re51xf6Y5jZD33bS1EV67cF1/uB5UGTEk8OUMpXx
         f8/WofcH3OVRn4AVwBpiZ9qBxwk2J3R0sjygVt+mKBgKRgF4uX4hROaGd2GvRzrq3Co6
         RIRxJz8emfCkABCHv+kqS1//10h7pM9VZvDBGAvmXI8nf3DMGTXJ/6NFGfP+Kqe+ETzA
         Snx3Dwu28ziS+pERf5mvCR44CjgxdYtnOdcTIqOXPdLkpBvXHYo0oxHC+AC1Coi6qKWS
         sYNA==
X-Forwarded-Encrypted: i=1; AJvYcCWmXhMa1fHERhH2VBt4oXvqefNZMaE9KWu4G6vJ0oOqPE9cXVYB7rqZoFgQKh5IivqfX/7WvAh53U0sGA5hP/nh/4pfVxZxgaEiqHABu3I=
X-Gm-Message-State: AOJu0YxzRoZydSib+74KuFTk6Z9OR9pSFOVzlluhw8PYMrSdh/dq5MO8
	t/9D8xbZyqtoWXfjUDGR8Khux4UD5+Xb8+mUFF2g+AwTybTGJoMXUvZnf34WlA==
X-Google-Smtp-Source: AGHT+IEXwvInGz35zW91JP62kDLg8JPNPVqOXQEdoZhafel6KVZMnMFF2gBo1YRFpB3IeiFEAcvhLg==
X-Received: by 2002:a17:906:57cf:b0:a3f:4fd8:53a4 with SMTP id u15-20020a17090657cf00b00a3f4fd853a4mr5351915ejr.9.1708940717081;
        Mon, 26 Feb 2024 01:45:17 -0800 (PST)
Message-ID: <44fd5092-7838-4d28-804b-bbfebfd44886@suse.com>
Date: Mon, 26 Feb 2024 10:45:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
 <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
 <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
 <1c53b52ee39161a8f59209d28af69fe997479dbc.camel@gmail.com>
 <3b38fe82-ee0f-4666-93e1-bd78fe69c534@suse.com>
 <5d2d35fe014094c991363e42c3e2ad9ca2af3938.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <5d2d35fe014094c991363e42c3e2ad9ca2af3938.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.02.2024 13:23, Oleksii wrote:
>>
>>>>> As 1- and 2-byte cases are emulated I decided that is not to
>>>>> provide
>>>>> sfx argument for emulation macros as it will not have to much
>>>>> affect on
>>>>> emulated types and just consume more performance on acquire and
>>>>> release
>>>>> version of sc/ld instructions.
>>>>
>>>> Question is whether the common case (4- and 8-byte accesses)
>>>> shouldn't
>>>> be valued higher, with 1- and 2-byte emulation being there just
>>>> to
>>>> allow things to not break altogether.
>>> If I understand you correctly, it would make sense to add the 'sfx'
>>> argument for the 1/2-byte access case, ensuring that all options
>>> are
>>> available for 1/2-byte access case as well.
>>
>> That's one of the possibilities. As said, I'm not overly worried
>> about
>> the emulated cases. For the initial implementation I'd recommend
>> going
>> with what is easiest there, yielding the best possible result for the
>> 4- and 8-byte cases. If later it turns out repeated acquire/release
>> accesses are a problem in the emulation loop, things can be changed
>> to explicit barriers, without touching the 4- and 8-byte cases.
> I am confused then a little bit if emulated case is not an issue.
> 
> For 4- and 8-byte cases for xchg .aqrl is used, for relaxed and aqcuire
> version of xchg barries are used.
> 
> The similar is done for cmpxchg.
> 
> If something will be needed to change in emulation loop it won't
> require to change 4- and 8-byte cases.

I'm afraid I don't understand your reply.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:54:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:54:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685404.1066043 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXgZ-0005xn-Ui; Mon, 26 Feb 2024 09:54:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685404.1066043; Mon, 26 Feb 2024 09:54:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXgZ-0005xg-RI; Mon, 26 Feb 2024 09:54:11 +0000
Received: by outflank-mailman (input) for mailman id 685404;
 Mon, 26 Feb 2024 09:54:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fECz=KD=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1reXgY-0005xa-MA
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:54:10 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fb6ba74c-d48c-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 10:54:08 +0100 (CET)
Received: from DS0PR12MB9448.namprd12.prod.outlook.com (2603:10b6:8:1bb::8) by
 SN7PR12MB7980.namprd12.prod.outlook.com (2603:10b6:806:341::22) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.34; Mon, 26 Feb
 2024 09:54:05 +0000
Received: from DS0PR12MB9448.namprd12.prod.outlook.com
 ([fe80::640c:7a82:230:eea4]) by DS0PR12MB9448.namprd12.prod.outlook.com
 ([fe80::640c:7a82:230:eea4%6]) with mapi id 15.20.7316.034; Mon, 26 Feb 2024
 09:54:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb6ba74c-d48c-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hZbInYWXd8GZELS2afBvr4sj04D4spKNoCsBw1YA8v3PqbSk/xZs/htxFD9kelijIEHcKsMxKX8ArTUM3KiCmFuxbw6Lq1Gy5V0aU4ROGbL13shYaF/D3wq8m5UZhpTdNV8Gr9M/yCaFWsJ1QkHVz2+VM7IIiKAEjvpqVY7MC0IYezAmb5g8reGCzwyBGhKpKEWzrqil5CTI3VaNA87kV3EBcVI6Prd+ljgCYGBmTTtT8PjOozR9RCmqoW110BmicigTwtZfI65SbEx1u0xou+O2zNCpNTZ6YAh19DDrVyC/ANZT4Y/di9n21OWNUIk1ErP/8D/FCZE4KWT/O3TQoA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=WY+M8Q39ASiwK9mzUUf8Hg0KQYd25uiWciIka+VsBCA=;
 b=a05YZhMUKLwJJ1QD53zFjWWP1rN6dUw2gmg48cRElOv4i/T8a1kVH2dmn0TtHKIS57Sx+xMEMjloo70bTtv0L5L4nG72cs2/vaU07qUyWbCRVLKZ6E0C5vnVMWKdIT3p5FDf66k7/v77VO/lftFZBJlBuK4EozcXmb4gMfytIGxL/4mNyd9DjPj7s1KmqZx/WGSU8wjte6ZoSGiOpps9w3DhpNIqOTacgZxEjWthDm3/0KJhJcdSqGUQB2bt+O9UMMX3Pm+kOuCONhCjUeAM12L+j/Z+kWcECv6zOFmt92pSK7WrVfwxjiNavGC8ku7yj2/fQHf6VfMsEQI20RHz4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=WY+M8Q39ASiwK9mzUUf8Hg0KQYd25uiWciIka+VsBCA=;
 b=YcSrINnW0biFpEx8i2mKfj0CD+kRyYphufkwfwn2RfhA6fafmItvfr6baQIku0SYJh7UgeZH+Izsw2bnvqIK8vaQhwZqSVdReQvnHClFTk8lPMjBCLSqytB3kipvC5MOXlhG0qD8V9G7rOjaUNbhSuSmZMz+Mvkw2BMOEFh5gVI=
From: "Wang, Henry" <Xin.Wang2@amd.com>
To: "Orzel, Michal" <Michal.Orzel@amd.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr
 Babchuk <Volodymyr_Babchuk@epam.com>
Subject: RE: [PATCH] xen/arm: Set correct per-cpu cpu_core_mask
Thread-Topic: [PATCH] xen/arm: Set correct per-cpu cpu_core_mask
Thread-Index: AQHaaGAnRoGRN1vAm06vXBNQgKmm5rEcXbUAgAAC7lA=
Date: Mon, 26 Feb 2024 09:54:04 +0000
Message-ID:
 <DS0PR12MB9448472A793F33228BE6A185A15A2@DS0PR12MB9448.namprd12.prod.outlook.com>
References: <20240226030146.179465-1-xin.wang2@amd.com>
 <de4c6988-7051-4d44-9b44-e850ad4b4cc1@amd.com>
In-Reply-To: <de4c6988-7051-4d44-9b44-e850ad4b4cc1@amd.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
msip_labels:
 MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_ActionId=b0971ed1-3c37-4d11-9197-15f40cb44849;MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_ContentBits=0;MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_Enabled=true;MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_Method=Standard;MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_Name=General;MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_SetDate=2024-02-26T09:46:39Z;MSIP_Label_4342314e-0df4-4b58-84bf-38bed6170a0f_SiteId=3dd8961f-e488-4e60-8e11-a82d994e183d;
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DS0PR12MB9448:EE_|SN7PR12MB7980:EE_
x-ms-office365-filtering-correlation-id: f4f50496-b6c8-449e-f46f-08dc36b0de30
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 /ZuqbD2troct+gaTDvnBtR+og50CO/DsJ3PYg7W2DPmBKO4zmnpNfXdOYNWGtZfcbqGzCHtjOHvSiDI7Uzz1UVWtIMPvT4BDw/vjIzN8t8KOJb+Ac1GB35+E0bs6gtw4qN8H0rcKXEQu4h6zSwZAQ3va/hbQnmrinfegYuOojlwCW8SIrQsz3zHgH3W/JT2OHRgogvsUlTP73xTfqgURvskT03N9kqDMLUe6r1/BiMa9Yi4VGlaPzhKgH2u0kP9WNpdssk6apuT/7lbS3UgNLBE/RKDEoEb3QFfSZx5GcQcuwnDATIrIqMVJ1Moh0xjIgln6slAvx1mfBuWcmVEiK5lLU4Gxyg63SCjX7BCT6elbF7MvR4UNu4abF1M8pcHETM5JZa3LbQ6+i2qBw1+v8nRWrnjBi2WQ3j9Sag4qZB/O3nBO7VWPX6kiiU2NZPzHRYnjgPNY7FHTvrzRAj0sJKAOZMyQVNHr5t58ZmoJslK+TsObAlDGa2XsyivkzOFjMq2HeTFQLbmktTN0cM0E61IjRK69VIihMTz5WuYdRc028bl1m0Lv9gCu4jdfuKrqPTL0YcFK14GM72EwA4U8xy8XN++39TKgLx9OGl2nDIq4uhGp1p29NBEdA7XUZhmMULV4ACYYmHc9GmLGLjZROT2FqCD0YhkrfS8jJQQzaGvLwK0Q7MIzT4ix4uMPJFnxp7HMOifnq1ZYDm8V7qBlnQ==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB9448.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?dmY0ZVd6eTVrbmpuU2ZNOFlwbkdtcEJKaWxjZ1ZhTlFPeHp1VnYrMFZWQU1U?=
 =?utf-8?B?WXJQbnYzM3BrckRISmNwc1poVFNPVFlHbmFpVU93TVFkMWhVeitOM0JYL2hl?=
 =?utf-8?B?dDBKTENzSDk1d09GdUgwclpwTmtaVkFkd3RwT3E2cWtuSnNreWE3Wjd2YjRu?=
 =?utf-8?B?eEZGd2pxNjc1dyt1Wm1id2gxN0plYVFSZU1uNnZ5RDlsNVFraFVhSmNtM1RR?=
 =?utf-8?B?VFhocHkreDlsUjhVUWg2V3QvNEJGaEcvMFRpbFVJUUhWeUcrOXFvcmFkNVlW?=
 =?utf-8?B?V2tuMDlVZUNSVzBzWmNPMnM4OHkrblNIQUdocUN3WGtrMlFEZlcveUM2SGNo?=
 =?utf-8?B?M1dPRWVYVjgvcXBCMDFKUGRxeWcrc3ROKy9sWEdqSGltckE1d3J5LytTd09h?=
 =?utf-8?B?VGg1ZVg4TldTMDIrWXhMcEVNU2pYWXZTTXpHNGtqUWFXWmYwdVlSTi9EZ05t?=
 =?utf-8?B?MDNBS2IxQ3FBODVCQm1RZVpFVmRza21mejdsamlJWHdUQkY5RkNZUmdtb0Yx?=
 =?utf-8?B?ZUVtNHE4dVd5MWZzdlRSenp0QXRRaWpXNGZRZk5ScGUrRG9mdWRCaWlhNFJi?=
 =?utf-8?B?Nks5R3Mwd3BFbXhaUHBqU1ljNVRvaE1FM2s2ZC8yazhyMnp1dFJrVEtoeEI3?=
 =?utf-8?B?RXIwNXkzVVZaQnNIZWdZbUZPRGRWb3JUbm9HU290N0d6RkxDN05qY1p5YldH?=
 =?utf-8?B?MFRNQVRmZEhiV2ZwNy8rSEpkcUlZbldOMHZPTjg4d2MrTEY5Y0M2bUQzaC9P?=
 =?utf-8?B?d2xweTBIM2lFY2VSUEFrWmkvb1lYcDFlN2lhNXBhY0RPeTZKUXNKQWVpWjd0?=
 =?utf-8?B?ckQyeXMvTWgyODJDcHFqUTdPeVFkN2FlcWZJZ3ZIejJNbXdrS2owZG5vTUVj?=
 =?utf-8?B?QzRkQWZBN01GS3V5VUdYN25Ld3lNZmZzRGJDQXA0OFg0QjNGTUdEazNVd0Jt?=
 =?utf-8?B?TytLNFZHaGtaTkJXMXlpNkE4cmFOMDh0T1JUc3FRT21sOHJGdEM5NXBaRkxu?=
 =?utf-8?B?Z1ZnaExjUVV1SDlWUk5RdDAzaVdhcVdNUXJaYlZDbTZLY2RzVWI3TVZFN0hh?=
 =?utf-8?B?OXdqY1ZXNkcvOHhlaXRWdHY5bzRrUTMwQWx4ZUYwQ3NvRVBzRFR6ODd3ZFZF?=
 =?utf-8?B?MmFFN1dBY1oyQ2RaWkUwYWlKb2ZJTWErT3RjUjdNcnRXTC9lYmpRL1REZU0w?=
 =?utf-8?B?ZTRwQS9tUHQxRUVscXdvTDZOZmo1UXdzRWpXa3hxN2ZreTF0R25zKzhtUGZT?=
 =?utf-8?B?dWNnUmRKRm9mRjhTWnRaeFYwdlNoZGRsY1Q4aVk4VnlRRmV0Q2hmOFRqSnl4?=
 =?utf-8?B?QjhtRWtpMUJWQUFUV3QwNExHWHhLYXJkSG5YVG1CZ0ZaZ1hjZ3M3UE5TblU2?=
 =?utf-8?B?THVjYlRCN1RCQVA2QURMbWt4REpvbnZUUVllbi9ISHoyZTNhUDJidkc1aE0w?=
 =?utf-8?B?dUlTeVUrR2Q1TVg3ZjhzZ3grTGxXVnRydmdPVkV2eWNqTkhhM3RaSFNvcmNL?=
 =?utf-8?B?aGt4T0NQQlJLUGxRbG8yOTNoRVdQdEFCNGVKeXhVcHdKU1BQenNSaDRoaVVl?=
 =?utf-8?B?c0IxYUsxeVIwSVR1dFpGRlNEdWFadndVQlBkdnpnWCtHcFFXK0dnWDJORENM?=
 =?utf-8?B?aXNDSHkwalZUWUZZNThrZUU5eDI0UXpQZTZlM0hLS3YxTXh2TDBQZjZmTDFJ?=
 =?utf-8?B?czZ0ODZ4YXVISWR1bmJtS2VCZWJBTmV1bHFiSjNsN0lZWTZYbTh4UEo5TGhF?=
 =?utf-8?B?eTJUOS9TTjErdmRscVBFM01uc1JDYzg2U2hhUkNQMG00YUFNK0wyQ1ptQmNH?=
 =?utf-8?B?Q0xsLzd0dE05SUtSQWNrVHJwNEs2YUduSmN2RjRvc3dSWmg0TGlwbGUwWEJW?=
 =?utf-8?B?eHVBMVN6ZEdWRVpLVWM1dVlvMGp2VnRSb05PdFRad0hndHhUYzhpTUk4Nnhs?=
 =?utf-8?B?OHhiMGd3Y2V4aVRXOFAzUk5YV0tGckZxTEQ0NGlJVWpjSTBSNUJ1V051ZzZo?=
 =?utf-8?B?Y1RCN1FGVkF5cEp0aHpIcFM0c2k3bzc5UDVvcUFTQWxjdWFLNmMwYk5yRGV6?=
 =?utf-8?B?MklRemRSS2NKYk5mcWxZTXRtWUE2WE5aYlljU1E3Q2pkb0VxTlNSWU1tNGln?=
 =?utf-8?Q?c8xA=3D?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB9448.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f4f50496-b6c8-449e-f46f-08dc36b0de30
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Feb 2024 09:54:04.9338
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: SA6itqHnv2xfh3w3qNNgz2m14x+rlTonoM/Uqw4j1UZSG3w+t18NCGKVDo9qIFbL/dChPAX9ylbJ/Wk7cOttFQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7980

W0FNRCBPZmZpY2lhbCBVc2UgT25seSAtIEdlbmVyYWxdDQoNCkhpIE1pY2hhbCwNCg0KPiAtLS0t
LU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBTdWJqZWN0OiBSZTogW1BBVENIXSB4ZW4vYXJtOiBT
ZXQgY29ycmVjdCBwZXItY3B1IGNwdV9jb3JlX21hc2sNCj4NCj4gSGkgSGVucnksDQo+DQo+IE9u
IDI2LzAyLzIwMjQgMDQ6MDEsIEhlbnJ5IFdhbmcgd3JvdGU6DQo+ID4gU2lnbmVkLW9mZi1ieTog
SGVucnkgV2FuZyA8eGluLndhbmcyQGFtZC5jb20+DQo+IE5JVDogWW91IGZpcnN0IHNlbnQgdGhp
cyBwYXRjaCBhcyBwYXJ0IG9mIE5VTUEgc2VyaWVzOg0KPiBodHRwczovL2xvcmUua2VybmVsLm9y
Zy94ZW4tZGV2ZWwvMjAyMzExMjAwMjU0MzEuMTQ4NDUtMTYtDQo+IEhlbnJ5LldhbmdAYXJtLmNv
bS8NCj4gU2hvdWxkbid0IHlvdSByZXRhaW4gdGhlIEFybSdzIGF1dGhvcnNoaXA/DQoNCkFoIGdv
b2QgcG9pbnQsIGluIGZhY3QgSSBkb24ndCByZWFsbHkga25vdywgc2luY2UgSSBiYXNpY2FsbHkg
cmV3cm90ZQ0KdGhlIHBhdGNoIEkgdGhvdWdodCBpdCBpcyBub3QgcmVhbGx5IG5lZWRlZC4gSSB3
aWxsIGFkZCBpdCBiYWNrIGluIHYyIHNpbmNlDQp5b3UgbWVudGlvbmVkIHRoaXMuDQoNCj4gPiAt
LS0NCj4gPiAgeGVuL2FyY2gvYXJtL3NtcGJvb3QuYyB8IDYgKysrKystDQo+ID4gIDEgZmlsZSBj
aGFuZ2VkLCA1IGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkNCj4gPg0KPiA+ICsgICAgLyog
UEUgbm90IGltcGxlbWVudGVkIHVzaW5nIGEgbXVsdGl0aHJlYWRpbmcgdHlwZSBhcHByb2FjaC4g
Ki8NCj4gPiArICAgIGlmICggc3lzdGVtX2NwdWluZm8ubXBpZHIubXQgPT0gMCApDQo+IERvIHdl
IG5lZWQgdGhpcyBjaGVjaz8gSXQgbXQgd2FzIHRydWUsIGNwdV9zaWJsaW5nX21hc2sgd291bGQg
YmUgaW5jb3JyZWN0DQo+IGFueXdheSAoaXQgd291bGQgc3RpbGwgYmUgMSkuDQoNCkkgYWRkZWQg
dGhpcyBjaGVjayBmb3IgcGxheWluZyBzYWZlLCBiZWNhdXNlIEkgb25seSB3YW50IHRvIGRvIHRo
ZSBjb3JyZWN0IHRoaW5nDQppbiB0aGlzIHBhdGNoIGFuZCBhdm9pZCBtYWtlIHRoaW5ncyB3b3Jz
ZSBmb3IgTVQgY2FzZS4gV2l0aCB0aGlzIHBhdGNoLCBub24tTVQNCmNhc2UgY2FuIGJlIGltcHJv
dmVkIGFuZCB0aGUgTVQgY2FzZSBpcyByZW1haW4gdW5jaGFuZ2VkLg0KDQpCdXQgSSBhZ3JlZSB3
aXRoIHlvdSwgYW5kIEkgd291bGQgYmUgbW9yZSB0aGFuIGhhcHB5IGlmIEkgY2FuIHJ1biBhIE1U
IHNldHVwIGFuZA0KZmluaXNoIHRoZSAiZWxzZSIgcGFydCB3aXRoIHRoaXMgcGF0Y2ggb3IgZm9s
bG93LXVwcy4gRG8geW91IGtub3cgbWF5YmUgcWVtdSBjYW4NCmFsbG93IG1lIHRvIGVtdWxhdGUg
YSBNVCBzZXR1cCBzbyB0aGF0IEkgY2FuIGZpeCBpdCBwcm9wZXJseSBpbiB2Mj8gVGhhbmtzIQ0K
DQpLaW5kIHJlZ2FyZHMsDQpIZW5yeQ0KDQo+DQo+IH5NaWNoYWwNCg==


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 09:57:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 09:57:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685406.1066052 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXjl-0006Wf-Br; Mon, 26 Feb 2024 09:57:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685406.1066052; Mon, 26 Feb 2024 09:57:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reXjl-0006WY-9D; Mon, 26 Feb 2024 09:57:29 +0000
Received: by outflank-mailman (input) for mailman id 685406;
 Mon, 26 Feb 2024 09:57:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reXjj-0006WP-8R
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 09:57:27 +0000
Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com
 [2a00:1450:4864:20::42a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 71720a62-d48d-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 10:57:25 +0100 (CET)
Received: by mail-wr1-x42a.google.com with SMTP id
 ffacd0b85a97d-33da51fd636so1766413f8f.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 01:57:25 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 bj20-20020a0560001e1400b0033db0c866f7sm7819401wrb.11.2024.02.26.01.57.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 01:57:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 71720a62-d48d-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708941445; x=1709546245; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=GoqpB2A1QWrYgNxFfnAU2BSKKVcn6++TCAwgRtuKPFI=;
        b=ZWjqtQSQYdjI8BFlgZhIzn9LcbBaJcHbPYF0aXizJPpidV3jV2IWuix0fn8Nbv1htt
         cXXrAmZsF510X5WCmrNTlwcRg5goK94uPLj02jmooksJarWAEk3fXAUBWGSeuSeflIVb
         RRgwXT19KQDBGY6nRPU+IFfiKVZUcwsq8HOUE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708941445; x=1709546245;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=GoqpB2A1QWrYgNxFfnAU2BSKKVcn6++TCAwgRtuKPFI=;
        b=dwzn4D2QZ0sv58vz3oyjZM7ahP7mhuzvMIFeuPCpgCKWKjxIENySpNqWD45SHaW348
         4YjmDVu6nX329YY8jLuMx83t9o5PiiG6Ez75fnl+ZSXOH/OkE8VZtTwYSl6PXXWwOvn4
         NzGH/+8qB2ClxpuhTl7PJhLjvlR+v7xRrqJRxXwOmJ8AEw4sWaG4cLKi1q9JnzRo5z7F
         pD1oytkYo3Do56V24z+1V2cBdNDyhXycLnf03KnqZYocgoULoA5vpCoj0rguGbHvGmkS
         wxBTTFCrf2OpHR8t+ZA38GKtJruZUfTJyi73uU1FcG7Lx+xgpXEuJGK+4n6vyVC1pxMF
         C6hA==
X-Forwarded-Encrypted: i=1; AJvYcCX5pI/l7kVAj52lyQHHzf+EsQd/bTROyXpO/3FEliVT8jLC3tY4GnXU6TZ4T2fkWD9K4BUpqa11xXZ0Ns1i80KUoS4MX24Ukhafw+4pG8A=
X-Gm-Message-State: AOJu0Yx+k0r62+hqion+AdUZbuBi8toWaFDloQC3NSt9wvIdz6zRu64P
	Y11Ujtkpxzs80TKdOrp5S2za2RQqMVDZYCr0OA8kdee6LW982OKZ/fp0T3OTRl4=
X-Google-Smtp-Source: AGHT+IEAXzD5v4SkZEtkdjU7xi6+CmPDp60BpErAFzdVl6kmYMbh6JL70Mf2KEVA24q5ufBcP6j6tQ==
X-Received: by 2002:adf:fcc6:0:b0:33d:9283:93b with SMTP id f6-20020adffcc6000000b0033d9283093bmr4348396wrs.47.1708941445001;
        Mon, 26 Feb 2024 01:57:25 -0800 (PST)
Date: Mon, 26 Feb 2024 10:57:19 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v2 2/3] x86/spec: fix BRANCH_HARDEN option to only be set
 when build-enabled
Message-ID: <Zdxgf-1BvPQsnTVb@macbook>
References: <20240223120616.73503-1-roger.pau@citrix.com>
 <20240223120616.73503-3-roger.pau@citrix.com>
 <3dcb9afe-7f3f-495c-b182-b4242b4cb337@suse.com>
 <ZdxVQRMRyQOEc581@macbook>
 <076cac3e-097a-4994-b68f-888b491c82a7@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <076cac3e-097a-4994-b68f-888b491c82a7@suse.com>

On Mon, Feb 26, 2024 at 10:33:14AM +0100, Jan Beulich wrote:
> On 26.02.2024 10:09, Roger Pau Monné wrote:
> > On Mon, Feb 26, 2024 at 09:42:58AM +0100, Jan Beulich wrote:
> >> On 23.02.2024 13:06, Roger Pau Monne wrote:
> >>> --- a/xen/arch/x86/spec_ctrl.c
> >>> +++ b/xen/arch/x86/spec_ctrl.c
> >>> @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
> >>>  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
> >>>  int8_t __read_mostly opt_eager_fpu = -1;
> >>>  int8_t __read_mostly opt_l1d_flush = -1;
> >>> -static bool __initdata opt_branch_harden = true;
> >>> +static bool __initdata opt_branch_harden =
> >>> +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
> >>>  
> >>>  bool __initdata bsp_delay_spec_ctrl;
> >>>  uint8_t __read_mostly default_xen_spec_ctrl;
> >>> @@ -268,7 +269,14 @@ static int __init cf_check parse_spec_ctrl(const char *s)
> >>>          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
> >>>              opt_l1d_flush = val;
> >>>          else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
> >>> +        {
> >>> +#ifdef CONFIG_SPECULATIVE_HARDEN_BRANCH
> >>>              opt_branch_harden = val;
> >>> +#else
> >>> +            no_config_param("SPECULATIVE_HARDEN_BRANCH", "spec-ctrl", s, ss);
> >>> +            rc = -EINVAL;
> >>> +#endif
> >>> +        }
> >>>          else if ( (val = parse_boolean("srb-lock", s, ss)) >= 0 )
> >>>              opt_srb_lock = val;
> >>>          else if ( (val = parse_boolean("unpriv-mmio", s, ss)) >= 0 )
> >>
> >> While looking at patch 3 I noticed another inconsistency: Wouldn't
> >> "Compiled-in support:" better also enumerate HARDEN_BRANCH then, just
> >> like for thunks both CONFIG_* state and actual runtime choice are
> >> logged?
> > 
> > Yes, I guess we would also need to expand "Compiled-in support:" to
> > include HARDEN_ARRAY and HARDEN_GUEST_ACCESS.
> > 
> >> Or alternatively, should logging of thunk runtime choice be
> >> suppressed when the Kconfig setting is off?
> > 
> > Hm, I think printing "BTI-Thunk N/A" is good enough when the thunk has
> > been built time disabled.
> 
> Good enough - yes. But redundant with the other log line. And since all
> of this output is getting longer and longer anyway, omitting whatever
> can sensibly be omitted might be at least worth considering.

I can add yet another patch to remove that when not built-in, this
cleanup is turning into a never-ending exercise I'm afraid.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:16:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:16:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685412.1066063 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reY1h-0002ls-RT; Mon, 26 Feb 2024 10:16:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685412.1066063; Mon, 26 Feb 2024 10:16:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reY1h-0002ll-Oh; Mon, 26 Feb 2024 10:16:01 +0000
Received: by outflank-mailman (input) for mailman id 685412;
 Mon, 26 Feb 2024 10:16:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reY1g-0002lf-Uy
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:16:00 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 098541fc-d490-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 11:15:59 +0100 (CET)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a26fa294e56so496715166b.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 02:15:59 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 n23-20020a170906089700b00a4272b24812sm2258419eje.215.2024.02.26.02.15.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 02:15:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 098541fc-d490-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708942559; x=1709547359; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MTvNZ3HGge1k6bKR+L3E3EUXp3IM01FdabSPSgnlCNY=;
        b=YtXogyHqBMvUD0f+gViNsGzY1h9Rpv+0El5ViVO0C6qp1HYLhSJONcWDiTR8qRECkq
         94rZo9jYIpKicqcuURXYLaLPcfZYngpgLhVhM+OZHSCTNWjUjrsK5561m+hCbkOf2+Z0
         FUlXfxoZSsHDw2TipJv4kCz+tHrNUBxziryA2noJEpx6/E95A16qpH0eFWia8BBUe1YA
         DS8G1Kh4dfrMMNmAZnwCfwwen6ZiKUQQ5XYHPOFE+ehsEUmSnQ1D5BZ6h4rcpWnKhOG+
         l24oOE+Hj9z8P9iERQG10eWt3hZKBZeqBv4p3Ajr7pcX/fqdeLdpFxI18m8v1SUlF/fR
         aecQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708942559; x=1709547359;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MTvNZ3HGge1k6bKR+L3E3EUXp3IM01FdabSPSgnlCNY=;
        b=O+f28OLIik6WC7ld99RH7i6PhhAI3K9ugktiz1y6cmOgipu7oo9933Dx28XQnwYF2r
         vtYb1KLci7pDfS7Mmm07cWTNdbf+YHUxNZqgtWr12g1ziaBXo23+QbZxwDto6VvYf6MZ
         wpb3MQcUPoWqsmAXHQ99Q7N5mNnm8Lg7Cg6ObvpB5DMFznDn7+d+AoM5C2nz+Yi+7iNa
         /l3Ym6d/hp0Ys+Ooln1ctrDwQEOeCNwGc4td6VTHzn9WeZ7A6Sm1hD+4a6PSrcMhbh0k
         jh2IRm7zzeoSEKQAYfdJji6ZcUQMivf2ynKQfz5JQ7qpaBpnWtPOGRwBC4TbPbilfm2X
         TY/g==
X-Forwarded-Encrypted: i=1; AJvYcCV5qRoWNLszvPFkFuTACVcCK4lzaMJZFWBgNaweE9qwL1LqcA12baF7UjqqnHGrHjQH0I5e/nhKljJAgs+3Vgkxyw+Sxc74Nlh6bmxTkuc=
X-Gm-Message-State: AOJu0YzM0dya1ulR2z9ERQ25hsL0HMiRZqTQ0OV8ObFoZpYiZaBbyNhO
	u50MO7carREB00e5Y7YjCLqnKe/EUAnJhoyzhq7s1llyEx6bgrzyXmAKTaSUPg==
X-Google-Smtp-Source: AGHT+IFSm/jfPK8Qnis1RqYbCgioJ8Mwp1ZgSrrjvebqu0GTTsIY22NL9I9oHFuUANZfjgnd5FdnIA==
X-Received: by 2002:a17:906:28d1:b0:a3d:6eb4:9769 with SMTP id p17-20020a17090628d100b00a3d6eb49769mr4556319ejd.15.1708942559115;
        Mon, 26 Feb 2024 02:15:59 -0800 (PST)
Message-ID: <d8666d36-cf27-4b4b-81e5-0eaba58f6669@suse.com>
Date: Mon, 26 Feb 2024 11:16:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/3] x86/spec: fix BRANCH_HARDEN option to only be set
 when build-enabled
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240223120616.73503-1-roger.pau@citrix.com>
 <20240223120616.73503-3-roger.pau@citrix.com>
 <3dcb9afe-7f3f-495c-b182-b4242b4cb337@suse.com> <ZdxVQRMRyQOEc581@macbook>
 <076cac3e-097a-4994-b68f-888b491c82a7@suse.com> <Zdxgf-1BvPQsnTVb@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zdxgf-1BvPQsnTVb@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 10:57, Roger Pau Monné wrote:
> On Mon, Feb 26, 2024 at 10:33:14AM +0100, Jan Beulich wrote:
>> On 26.02.2024 10:09, Roger Pau Monné wrote:
>>> On Mon, Feb 26, 2024 at 09:42:58AM +0100, Jan Beulich wrote:
>>>> On 23.02.2024 13:06, Roger Pau Monne wrote:
>>>>> --- a/xen/arch/x86/spec_ctrl.c
>>>>> +++ b/xen/arch/x86/spec_ctrl.c
>>>>> @@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
>>>>>  int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
>>>>>  int8_t __read_mostly opt_eager_fpu = -1;
>>>>>  int8_t __read_mostly opt_l1d_flush = -1;
>>>>> -static bool __initdata opt_branch_harden = true;
>>>>> +static bool __initdata opt_branch_harden =
>>>>> +    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
>>>>>  
>>>>>  bool __initdata bsp_delay_spec_ctrl;
>>>>>  uint8_t __read_mostly default_xen_spec_ctrl;
>>>>> @@ -268,7 +269,14 @@ static int __init cf_check parse_spec_ctrl(const char *s)
>>>>>          else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
>>>>>              opt_l1d_flush = val;
>>>>>          else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
>>>>> +        {
>>>>> +#ifdef CONFIG_SPECULATIVE_HARDEN_BRANCH
>>>>>              opt_branch_harden = val;
>>>>> +#else
>>>>> +            no_config_param("SPECULATIVE_HARDEN_BRANCH", "spec-ctrl", s, ss);
>>>>> +            rc = -EINVAL;
>>>>> +#endif
>>>>> +        }
>>>>>          else if ( (val = parse_boolean("srb-lock", s, ss)) >= 0 )
>>>>>              opt_srb_lock = val;
>>>>>          else if ( (val = parse_boolean("unpriv-mmio", s, ss)) >= 0 )
>>>>
>>>> While looking at patch 3 I noticed another inconsistency: Wouldn't
>>>> "Compiled-in support:" better also enumerate HARDEN_BRANCH then, just
>>>> like for thunks both CONFIG_* state and actual runtime choice are
>>>> logged?
>>>
>>> Yes, I guess we would also need to expand "Compiled-in support:" to
>>> include HARDEN_ARRAY and HARDEN_GUEST_ACCESS.
>>>
>>>> Or alternatively, should logging of thunk runtime choice be
>>>> suppressed when the Kconfig setting is off?
>>>
>>> Hm, I think printing "BTI-Thunk N/A" is good enough when the thunk has
>>> been built time disabled.
>>
>> Good enough - yes. But redundant with the other log line. And since all
>> of this output is getting longer and longer anyway, omitting whatever
>> can sensibly be omitted might be at least worth considering.
> 
> I can add yet another patch to remove that when not built-in, this
> cleanup is turning into a never-ending exercise I'm afraid.

Just to be clear: I'm not meaning to insist that you also clean up the
thunk part. But I still wanted to mention such as I noticed it.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:23:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:23:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685414.1066073 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reY9H-0004z1-Ij; Mon, 26 Feb 2024 10:23:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685414.1066073; Mon, 26 Feb 2024 10:23:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reY9H-0004yu-G3; Mon, 26 Feb 2024 10:23:51 +0000
Received: by outflank-mailman (input) for mailman id 685414;
 Mon, 26 Feb 2024 10:23:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=DoHO=KD=amd.com=ayan.kumar.halder@srs-se1.protection.inumbo.net>)
 id 1reY9F-0004yl-Nj
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:23:49 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 206ff1fb-d491-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 11:23:48 +0100 (CET)
Received: from PH8PR12MB7326.namprd12.prod.outlook.com (2603:10b6:510:216::7)
 by PH7PR12MB8037.namprd12.prod.outlook.com (2603:10b6:510:27d::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.20; Mon, 26 Feb
 2024 10:23:42 +0000
Received: from PH8PR12MB7326.namprd12.prod.outlook.com
 ([fe80::32a0:32b6:d5af:9ad0]) by PH8PR12MB7326.namprd12.prod.outlook.com
 ([fe80::32a0:32b6:d5af:9ad0%5]) with mapi id 15.20.7316.032; Mon, 26 Feb 2024
 10:23:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 206ff1fb-d491-11ee-8a58-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=eh478pjCqC6dGHpZAvkZB1x3CatZ+7x3w79FYYLKlCWhR+gxjpcuy12qnIhTWbWqVmuQGv9gi+hh3Blt4uXESytKHKjhPqc/f1v4jb6amJ6WiUC8QbbJ6jx+fQTFWrtW60wZIzS3gCYIsvtMhKsrYdrvPHZ2VHlCOxw++IcQqqGE2yjMSqK3xrs/yNEH2ZM7LmKWiwI091cR7G+WujXHYbXQ03Uup9Ng9I7BCOtae+gpIqfNQCrZmPBU0LXdvixQeWoFTdrLg00Mge0PZoLaFysvnXH+sB989I6hjmmndS5dH/Jt69BKI/A3Lj3MRsE/KyQ8NQ6Zna4kxrsiCMumWA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=YQP0lB8rP5JB9lBIJQVC7n9vyZDJqKcX0OpdlOQWSBI=;
 b=VKCeFtOGSqR/2nxnON/NWRzNayN4GBvb8J6Q7Zbs7t6S1xOrKkOZ0/DW/9IYiKTXY55As8VdI/+CPdiCq84aDsGH3Rl1WYJK1N/Je9MvuGsiSE9UGL8W2X84FSGoGCxAGwdKeJPwe7wzTuKNSnNa2iKCnF9OMQTVvAMzN4/vTm/CMr5s9yFDQVgOPfOI9rCs7W5DU18h6G6+xDXX+Xb13Vyo4v3s6PioS+t6i7fMTrD7kyiUX7oRnj8wXdiSJdRDCgbATxNPzNeeXAqW452clXcFY+QwOgF4OsexT3BKjDQct6OE820TIU0p/R9Xy+EdeQ5997Jlzs9sVnElyugJSA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
 header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=YQP0lB8rP5JB9lBIJQVC7n9vyZDJqKcX0OpdlOQWSBI=;
 b=cIFfw9jDStzyjITPy4jpayiIJh6XwcfmQYJx0YLCcBcDlWJDZUt7g2HPPWMIGoYLUJexNew97f9aHPsywYUbvKnT6hDxqO0auB6NryYR7TdACld9smg5jdXxlK/uGCyl1oSD9tp/5VraOqfchGvSIZ8Uf+TiTEHc7AkXA5jyXQ8=
Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=amd.com;
Message-ID: <ed611839-7704-4831-a34e-31bd6203f22e@amd.com>
Date: Mon, 26 Feb 2024 10:23:36 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN v5 2/3] xen/arm: arm64: Add emulation of Debug Data Transfer
 Registers
To: Michal Orzel <michal.orzel@amd.com>,
 Ayan Kumar Halder <ayan.kumar.halder@amd.com>, xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, stefano.stabellini@amd.com, julien@xen.org,
 Volodymyr_Babchuk@epam.com, bertrand.marquis@arm.com
References: <20240220121743.3680715-1-ayan.kumar.halder@amd.com>
 <20240220121743.3680715-3-ayan.kumar.halder@amd.com>
 <c5c0a46b-8793-481d-a678-da0929a0d6e9@amd.com>
 <9579f84a-ec9e-44be-b614-b70ced221c47@amd.com>
Content-Language: en-GB
From: Ayan Kumar Halder <ayankuma@amd.com>
In-Reply-To: <9579f84a-ec9e-44be-b614-b70ced221c47@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: CH2PR16CA0016.namprd16.prod.outlook.com
 (2603:10b6:610:50::26) To PH8PR12MB7326.namprd12.prod.outlook.com
 (2603:10b6:510:216::7)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: PH8PR12MB7326:EE_|PH7PR12MB8037:EE_
X-MS-Office365-Filtering-Correlation-Id: 9e1e8a09-ae81-4807-1a22-08dc36b501ac
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Kie0CdE4vPEyg/Ocl2SMlaJt6QGKMTIrMtRh7UAQttISA8YVGZ364GwIPHWeuDVF0r9b7bw9V4s934QHnl4HqDHls3DDpL6tddYbAoQ6oScWc3yhfNZq3JH6W87Yl1Aq14ZqVdWx2NOrvDbR+efEkA3jgk2yXbZ8Hi6QilFCkvATssRNBiuKrEm/P02epVtp8/qnc8mnBRj3ZHTubgENlA5av1H0TZikcDpopzF7KpHaFsi+7vsefGv9d/Ikq+WvwYM4u4d3NyNalkEQdR45+1EkcDwb8gx5E8ZyZMwcGlXiZgZzQMIOznw185EHTjRyxpae9CpXB1G1sACTHnXQoUvhX717FSsx9zVC53/hbvKcDEJbnwMN8FJRedQ1eECjrn65JxHsjlXZjkiNaAf/sgafjDfTpjzq4OjNna16BYZsVL1dMObWtSFor+BOAXt9b+xagfgbSoyToC1UJYWgJqL7K85HhqXVaUbc62Rkl3YicW++l8HgYYssM+/+DOyG3f3Vk9Z0fYhLRz3LBxM/CVMJkOONM7LQhcdkrpj/tdYxRNuea+StvLFqQH3/3mr6zKeZPkn4ouftNIFaLJwAj+NpFTIKcVEhXS1z8OcYc2tL2OjWbmfzhf06Nw9mfAotFVuWoEAIZJ5WwDLi47Dm+g==
X-Forefront-Antispam-Report:
	CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH8PR12MB7326.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Mk5HQlVtVFVjTWtKN3VRSGtUK1BkTmxEUy9aQjVFZTI1bE9wbzJTNEgvOGVK?=
 =?utf-8?B?V29VdTlxbHNqY1Rka1dhbnFFWEJHS1pSY3ViSE9NcHNEYXJJZHFqVC8vVzhp?=
 =?utf-8?B?TStGQWs0OXZDSzIySW1mWlVWR1ZJbFN3YlNQUWNBQ1pJTkgyRTFrR1E3c25B?=
 =?utf-8?B?b25QcVJWZXlNbnZxYmdpQkRVQ2NJeTNTbG1zSlFoUnFQcmE1dzk0YTNlNjZS?=
 =?utf-8?B?QUdWWDBzWDh6RzNHYi8vMnQramJjd2dOMklhb1BXbjltWS9sbGNxUmpTQVNi?=
 =?utf-8?B?bFVETDVTeGUrdkVNeW8xQVZrSHdBQ3JBOWdHb2pBMlFVc3gxeFpxeE5Hbk9m?=
 =?utf-8?B?cG52NVZ0MVVVSEdhWG9CVk9rdlNvRUwxQ1VtVW1qeWpISVBGSHUrdVkzajNS?=
 =?utf-8?B?aWNkQStrblB3MzNPTzgzYldjKzBybnpzd2FiOXZMQklyb2xveUVTMFVndzBw?=
 =?utf-8?B?RERicHp3UDB3ek9OdkxKUS9TOGVBQzdzUEdKb3lKUUVkOXBsbjRFa2NmUnhn?=
 =?utf-8?B?c0ZsQzFzalphSVZvWkdhc0NlL0ZoMXBPNk9vUFZWS1FuL3FJM2JYWEZnT0pD?=
 =?utf-8?B?eGFQbVVaMm96NHBaOHlheTJpcUdSNVVDbThkc0ViT0MxMSsvMFVQLzFtdWZo?=
 =?utf-8?B?cElhbEp1Wms1SDM4Y1ZyUU5CRGZidEUwOVBhWDJwd0xRMVc4MTJ0UnhxcUxW?=
 =?utf-8?B?K0hjSEdoVTVXWWNlZnVhL3FYV1JHU0tGODJVR1dqYWZjaTV6RW9iRmIwN21Q?=
 =?utf-8?B?WnYzUTZOSmZkWm1qZGUwbzlGcEVPajJDL1BFRmtoS05XQXcxSkNYTS8ra1k1?=
 =?utf-8?B?WkRDOWx3czNJcGUwNG12akE2NDU3ay8yU2E3ajNXcVpxUk1ObitSU0VId2V1?=
 =?utf-8?B?Zmh6L0ZGQ1BwSWp1bnhLc2NVR1l4U2lHNlc0Y28zT0g2ZFhldUlsK0Y4MVo2?=
 =?utf-8?B?SGpsUk5mT1RTVi9SME1Nb3ZTQlZqUWlZeXFmUFgzbXUxN25GUGJKMndSRmZs?=
 =?utf-8?B?amtTclVsSTVXWnFlL1d1eXd4M3lKN2lhS3NXcHFITC93WWlOZzRBL0psaEN6?=
 =?utf-8?B?UFlGeGlrWXBYWnc0Y3hPV3NuS3NZQ3VwU0ZiMm9sZmYwZTl3cHhabHV0WVFq?=
 =?utf-8?B?K0lhbysxOFJESmFESHRVeXRINXdQZG8xTGRqVitKUEo3MEhoM21XODNFb21w?=
 =?utf-8?B?MUFRemIzdDJlVXI3aEMwM1J2dHdIeUMxNWtTNHVYUkFEeFg2aXRHK0xJZXFR?=
 =?utf-8?B?TVorL0gwYndCQ0d5MURFYllZWnAxRFVTWC8xeXJ6SXhhMCtKTUNaejBQQmYv?=
 =?utf-8?B?Ymx1VDNxQ1BsdERkeUozblVtMjZwamtpa2RFRFlCdTF6bUV4V2xwVjhZTmFU?=
 =?utf-8?B?N1FxSWRDZkR5ZG9RdC9iVG9RQ2dNNzE0dHArUGNGdTNZamFZSm1qWWdCY3g1?=
 =?utf-8?B?TVYxOFMwb3VOd1kxZ2hkTzNQV1poMnZwYzRlOFZuWTZTUllZMlA3WUJDOE9E?=
 =?utf-8?B?MUkxUTl4a0NDQkhkNlVha3ZrN2VtbEo2Z2h2YW1uZDVhazFESGhhY1ZSK21y?=
 =?utf-8?B?OEdjSlczNHI1b0tMb01BazFxZmhNbmd1YzcyTXdMd1pJK0JKcEY4SEprSFVP?=
 =?utf-8?B?ZW9SZkVGaTcyRG53ejA4ZWE3TVo5UDc3NG5LbWxiRlVuT0llZ3phWmxGdVZL?=
 =?utf-8?B?dVFhSXlvREFIdEsyNXZ6c2hGdkhXRTR5VjQrVnY1R2FralhUdVhlVU1DL0l0?=
 =?utf-8?B?ZXJHeUJ1cWtPa1dQQitoVWNBYTExaE5pMzdsVS8wZDBKVVJnVUhrKzRrOFBo?=
 =?utf-8?B?QmVzcWVVWno0UzROUUdnakk0a2Vwa2ZERHNaQWd4STdYWXdWQjgwYnM2eTQv?=
 =?utf-8?B?Vk5MakpkdlBYekZjYnp3MWFiQzBJajlHeHh5aHU3dy9wVWhNWHhOSmNmdjIw?=
 =?utf-8?B?TXlEeDdoWklpNUFVdmUwWG10ajdkWUgwZkRTUE1CdUR1cUN4QWFraEMrd0cz?=
 =?utf-8?B?NzJ6dG9JWUlHY2s1RDg3YWU3Q2NCRmRPMXFiTURIRk9VRFlmVC9JOTdkSkVH?=
 =?utf-8?B?WUVDTVdjRENmcWVCcDJnQkptKzVES1ZhWkc2bktJNXpyd0l6UGdtVXNIMXg0?=
 =?utf-8?Q?Z67c/9QD/KaEjXA/Sj0hV48Zd?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9e1e8a09-ae81-4807-1a22-08dc36b501ac
X-MS-Exchange-CrossTenant-AuthSource: PH8PR12MB7326.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 10:23:42.6527
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BTOF2m/tdA2twPQ6gFLt/YJqgAFDoF3OgiUxBk63GzudpYlG941gqpOxCrj0rKcurfP2qUGta29KKGzeTquRfw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8037


On 26/02/2024 08:39, Michal Orzel wrote:
> Hi Ayan,
Hi,
>
> On 23/02/2024 16:41, Ayan Kumar Halder wrote:
>> Hi,
>>
>> On 20/02/2024 12:17, Ayan Kumar Halder wrote:
>>> From: Michal Orzel <michal.orzel@amd.com>
>>>
>>> Currently, if user enables HVC_DCC config option in Linux, it invokes access
>>> to debug data transfer registers (i.e. DBGDTRTX_EL0 on arm64, DBGDTRTXINT on
>>> arm32). As these registers are not emulated, Xen injects an undefined
>>> exception to the guest and Linux crashes.
>>>
>>> To prevent this crash, introduce a partial emulation of DBGDTR[TR]X_EL0
>>> (these registers share the same encoding) as RAZ/WI and MDCCSR_EL0 as TXfull.
>>>
>>> Refer ARM DDI 0487J.a ID042523, D19.3.8, DBGDTRTX_EL0
>>> "If TXfull is set to 1, set DTRRX and DTRTX to UNKNOWN".
>>>
>>> Thus, any OS is expected to read MDCCSR_EL0 and check for TXfull before
>>> using DBGDTRTX_EL0. Linux does it via hvc_dcc_init() ---> hvc_dcc_check(),
>>> and returns -ENODEV in case TXfull bit is still set after writing a test
>>> character. This way we prevent the guest from making use of HVC DCC as a
>>> console.
>>>
>>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
>>> Reviewed-by: Michal Orzel <michal.orzel@amd.com>
>>> ---
>>> Changes from
>>>
>>> v1 :- 1. DBGDTR_EL0 does not emulate RXfull. This is to avoid giving the OS any
>>> indication that the RX buffer is full and is waiting to be read.
>>>
>>> 2. In Arm32, DBGOSLSR is emulated. Also DBGDTRTXINT is emulated at EL0 only.
>>>
>>> 3. Fixed the commit message and inline code comments.
>>>
>>> v2 :- 1. Split the patch into two (separate patches for arm64 and arm32).
>>> 2. Removed the "fail" label.
>>> 3. Fixed the commit message.
>>>
>>> v3 :- 1. "HSR_SYSREG_MDCCSR_EL0" emulation differs based on whether
>>> partial_emulation_enabled is true or not.
>>>
>>> 2. If partial_emulation_enabled is false, then access to HSR_SYSREG_DBGDTR_EL0,
>>> HSR_SYSREG_DBGDTRTX_EL0 would lead to undefined exception.
>>>
>>> v4 :- 1. Invoked "goto fail" from "default:" to ensure compliance with
>>> MISRA 15.3.
>>>
>>>    xen/arch/arm/arm64/vsysreg.c         | 68 +++++++++++++++++++---------
>>>    xen/arch/arm/include/asm/arm64/hsr.h |  3 ++
>>>    2 files changed, 50 insertions(+), 21 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/arm64/vsysreg.c b/xen/arch/arm/arm64/vsysreg.c
>>> index b5d54c569b..80918bc799 100644
>>> --- a/xen/arch/arm/arm64/vsysreg.c
>>> +++ b/xen/arch/arm/arm64/vsysreg.c
>>> @@ -82,6 +82,7 @@ TVM_REG(CONTEXTIDR_EL1)
>>>    void do_sysreg(struct cpu_user_regs *regs,
>>>                   const union hsr hsr)
>>>    {
>>> +    const struct hsr_sysreg sysreg = hsr.sysreg;
>>>        int regidx = hsr.sysreg.reg;
>>>        struct vcpu *v = current;
>>>    
>>> @@ -159,9 +160,6 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>         *
>>>         * Unhandled:
>>>         *    MDCCINT_EL1
>>> -     *    DBGDTR_EL0
>>> -     *    DBGDTRRX_EL0
>>> -     *    DBGDTRTX_EL0
>>>         *    OSDTRRX_EL1
>>>         *    OSDTRTX_EL1
>>>         *    OSECCR_EL1
>>> @@ -171,12 +169,42 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>         */
>>>        case HSR_SYSREG_MDSCR_EL1:
>>>            return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 1);
>>> +
>>> +    /*
>>> +     * Xen doesn't expose a real (or emulated) Debug Communications Channel
>>> +     * (DCC) to a domain. Yet the Arm ARM implies this is not an optional
>>> +     * feature. So some domains may start to probe it. For instance, the
>>> +     * HVC_DCC driver in Linux (since f377775dc083 and at least up to v6.7),
>>> +     * will try to write some characters and check if the transmit buffer
>>> +     * has emptied.
>>> +     */
>>>        case HSR_SYSREG_MDCCSR_EL0:
>>>            /*
>>> +         * By setting TX status bit (only if partial emulation is enabled) to
>>> +         * indicate the transmit buffer is full, we would hint the OS that the
>>> +         * DCC is probably not working.
>>> +         *
>>> +         * Bit 29: TX full
>>> +         *
>>>             * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We emulate that
>>>             * register as RAZ/WI above. So RO at both EL0 and EL1.
>>>             */
>>> -        return handle_ro_raz(regs, regidx, hsr.sysreg.read, hsr, 0);
>>> +        return handle_ro_read_val(regs, regidx, hsr.sysreg.read, hsr, 0,
>>> +                                  partial_emulation ? (1U << 29) : 0);
>>> +
>>> +    case HSR_SYSREG_DBGDTR_EL0:
>>> +    /* DBGDTR[TR]X_EL0 share the same encoding */
>>> +    case HSR_SYSREG_DBGDTRTX_EL0:
>>> +        /*
>>> +         * Emulate as RAZ/WI (only if partial emulation is enabled) to prevent
>>> +         * injecting undefined exception.
>>> +         * Accessible at EL0 only if MDSCR_EL1.TDCC is set to 0. We emulate that
>>> +         * register as RAZ/WI.
>>> +         */
>>> +        if ( !partial_emulation )
>>> +            goto fail;
>>> +        return handle_raz_wi(regs, regidx, hsr.sysreg.read, hsr, 0);
>>> +
>>>        HSR_SYSREG_DBG_CASES(DBGBVR):
>>>        HSR_SYSREG_DBG_CASES(DBGBCR):
>>>        HSR_SYSREG_DBG_CASES(DBGWVR):
>>> @@ -394,26 +422,24 @@ void do_sysreg(struct cpu_user_regs *regs,
>>>         * And all other unknown registers.
>>>         */
>>>        default:
>>> -        {
>>> -            const struct hsr_sysreg sysreg = hsr.sysreg;
>>> -
>>> -            gdprintk(XENLOG_ERR,
>>> -                     "%s %d, %d, c%d, c%d, %d %s x%d @ 0x%"PRIregister"\n",
>>> -                     sysreg.read ? "mrs" : "msr",
>>> -                     sysreg.op0, sysreg.op1,
>>> -                     sysreg.crn, sysreg.crm,
>>> -                     sysreg.op2,
>>> -                     sysreg.read ? "=>" : "<=",
>>> -                     sysreg.reg, regs->pc);
>>> -            gdprintk(XENLOG_ERR,
>>> -                     "unhandled 64-bit sysreg access %#"PRIregister"\n",
>>> -                     hsr.bits & HSR_SYSREG_REGS_MASK);
>>> -            inject_undef_exception(regs, hsr);
>>> -            return;
>>> -        }
>>> +        goto fail;
>>>        }
>>>    
>>>        regs->pc += 4;
>>> +    return;
>>> +
>>> + fail:
>>> +
> No need for this empty line.
Ack
>
>>> +    gdprintk(XENLOG_ERR,
>>> +             "%s %d, %d, c%d, c%d, %d %s x%d @ 0x%"PRIregister"\n",
>>> +             sysreg.read ? "mrs" : "msr", sysreg.op0, sysreg.op1, sysreg.crn,
>>> +             sysreg.crm, sysreg.op2, sysreg.read ? "=>" : "<=", sysreg.reg,
>>> +             regs->pc);
> The original formatting (i.e. placement of printk args) looked better. I'm not sure why you changed it.
My bad. I changed it to align with 80 chars. I will revert it.
>
>>> +    gdprintk(XENLOG_ERR,
>>> +             "unhandled 64-bit sysreg access %#"PRIregister"\n",
>>> +             hsr.bits & HSR_SYSREG_REGS_MASK);
>>> +    inject_undef_exception(regs, hsr);
>>> +    return;
>> The last 'return' needs to be removed (spotted by Michal)

Ack

- Ayan

>>
> ~Michal
>


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:30:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:30:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685418.1066084 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYFR-0006uC-DC; Mon, 26 Feb 2024 10:30:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685418.1066084; Mon, 26 Feb 2024 10:30:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYFR-0006u5-8f; Mon, 26 Feb 2024 10:30:13 +0000
Received: by outflank-mailman (input) for mailman id 685418;
 Mon, 26 Feb 2024 10:30:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5KjY=KD=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1reYFP-0006tz-O4
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:30:11 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0385922f-d492-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 11:30:09 +0100 (CET)
Received: from BL1PR13CA0071.namprd13.prod.outlook.com (2603:10b6:208:2b8::16)
 by SJ0PR12MB6903.namprd12.prod.outlook.com (2603:10b6:a03:485::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.33; Mon, 26 Feb
 2024 10:30:05 +0000
Received: from MN1PEPF0000ECD4.namprd02.prod.outlook.com
 (2603:10b6:208:2b8:cafe::a6) by BL1PR13CA0071.outlook.office365.com
 (2603:10b6:208:2b8::16) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.25 via Frontend
 Transport; Mon, 26 Feb 2024 10:30:05 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 MN1PEPF0000ECD4.mail.protection.outlook.com (10.167.242.132) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 26 Feb 2024 10:30:04 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb
 2024 04:30:04 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb
 2024 04:30:04 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 26 Feb 2024 04:30:01 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0385922f-d492-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=UVkfqPzrUQ0GsX0vNNDv6ZoeOa5eB8gIDn7aFxo4Qlq8CZZHm2A65X9Ndv/ltbxMPhUQKLYHkSsstaVhRwIyZDHv55QLviM2TvMJ7tJI1cXyHqHNCwR+ElfXfVXT0G9lygaEs7xW8DhkFBBLt4v4o2OOP03r3PpXrZXGnNCWamD30T3QRc7s7fXxt6Oguk+ag0Znokcrt7h1GtAq8g1MSAceyRzH29AXoHdxBXj/MGBZnuFoSTM0y6bwSoNl5qqa5HSmOjPR+NRf1v0YPV5qENrd5Chpv8i5LsElnUJLL401G9vIH4eWRGzSO+oFJTzALb/hHCxTxteXr6DF+It7JA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=SeYIzwYxm9C6M/OnUbqDiodZbNcszOJVJa82PGgHvWE=;
 b=VmwZYRvW9GmgVoqwpJAmOu/hw6C8xVavHJ58hlW7/rzQjV9HWhXj0V7HqFViKxOKEfTStqV2kQOi4VJmmdLnOLcz5UHo8+E/H0NRovoP3RHP59eoEtF3gykfT5zQW7IyTF+6eaZIBcGhZdKmOpZkPPpS6fb0fx1WPYWEIsec+Kwc0ALP5LO5ipWRpWD7qiGEAEXRBu0KuxqQW8dB6s1qALjTGcDzrg5S+VhheqO1XVvJdoQcCQyE7C2hRkOgE594Ayhxm56b3xXfoCdlRRRGmPT8F0kp0FwxYP9K8oiES8a7Xd1+pLLYJmj3QbHhgUl/PeaRUNtOn6CRfLAiSldjIg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=SeYIzwYxm9C6M/OnUbqDiodZbNcszOJVJa82PGgHvWE=;
 b=nNIX+e1lRAjgmNc9eHtXe0gAQOl8jwA/Jk8jwy7ZejDRNcTfpkSe+EcM9YAO8u870cTKR1rA3s2A0+z3lhSMNv5o0b0a1USDnQb6R+Qvh7vwGQmZN+cmrZ9gH5DmoTGWaDc3bSRr4rz/IL4iP9WobyZ8p7rhkKMPyiXUBqJwPzg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <4c9bf719-7ad9-4739-9edc-eef06dcefb9f@amd.com>
Date: Mon, 26 Feb 2024 11:29:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
Content-Language: en-US
To: Henry Wang <xin.wang2@amd.com>, <xen-devel@lists.xenproject.org>
CC: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, "Juergen
 Gross" <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, "George
 Dunlap" <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Julien
 Grall" <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, Shawn Anastasio
	<sanastasio@raptorengineering.com>, Alistair Francis
	<alistair.francis@wdc.com>, Bob Eshleman <bobbyeshleman@gmail.com>, "Connor
 Davis" <connojdavis@gmail.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Alec Kwapis <alec.kwapis@medtronic.com>
References: <20240226011935.169462-1-xin.wang2@amd.com>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <20240226011935.169462-1-xin.wang2@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD4:EE_|SJ0PR12MB6903:EE_
X-MS-Office365-Filtering-Correlation-Id: 1a97944a-6874-4bb3-ed91-08dc36b5e580
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	XOqYjKkxg1eO5bbGnwgxm3WysZMh3fpO75aB/tO104uTnzG9zDdw+A1eire4jCC/kNBrK4XD0to/itL9sloj210Z41ewAH0AUMablW/pWNHcJGNJyCs3mekSZWNH9i8TkMdKDrhCpq/o0gIMDEko9mAP8GhAO0hdVdAcEzCgRzOA2IYJVgqDShOEjcYLZMNfE/Ksq3tQRvZNOzwIKJB1TNB4xGFQJcGvpn/04Lt7ps9WYM0W8i8jCbO7fun3izca2mwviBR/63t+miO2MR5Tgo66FHF0DLHnJGA4DpaFtY7Me3CXicf291q1ecuXChHmetpVU5AIlwFTzRCNyK9rvP48SXDQWCPBKrHZmGFpmODlZ6sT+5JOuzJmaRcwqFKUMXLFG02CCFcE2HgFpBnrDM/OMz5Guy9eJDeDlmG9IFXhhiiIsOZnvcM7pJ/EHSNL3n2UiYigvnbrPiggDFDWRrACp1PNCkB99auECKmjdKTNNMFvxhd3QOGLOhxGSkCmVHR2hQ211oo0e+dJDueWL7/4mClxVgwBng2znddNLlsedX+J6HY+P/RTawdX7Km6qDfTY9180iZunL29p4SI0BCxPNte1vJ7i5cz7cmfyt4QBj+9Gvt+Zb+tswPoML4Vx4fwP7UDsvxXrZyRlM5uWfDjrs51sMUfpm8pmTYQHfeCEqxumwJh0mxhxnPFWO5qJT6nfY23E08QGS/j15XuaIy38FRXnW2yvM0xBdx/SPB9PPf18iz1enK+ZHd4gGdH
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 10:30:04.6711
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a97944a-6874-4bb3-ed91-08dc36b5e580
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MN1PEPF0000ECD4.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6903

Hi Henry,

On 26/02/2024 02:19, Henry Wang wrote:
> An error message can seen from the init-dom0less application on
> direct-mapped 1:1 domains:
> ```
> Allocating magic pages
> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
> Error on alloc magic pages
> ```
> 
> This is because populate_physmap() automatically assumes gfn == mfn
> for direct mapped domains. This cannot be true for the magic pages
> that are allocated later for Dom0less DomUs from the init-dom0less
> helper application executed in Dom0.
> 
> Force populate_physmap to take the "normal" memory allocation route for
> the magic pages even for 1:1 Dom0less DomUs. This should work as long
> as the 1:1 Dom0less DomU doesn't have anything else mapped at the same
> guest address as the magic pages:
> - gfn 0x39000 address 0x39000000
> - gfn 0x39001 address 0x39001000
> - gfn 0x39002 address 0x39002000
> - gfn 0x39003 address 0x39003000
> Create helper is_magic_gpfn() for Arm to assist this and stub helpers
> for non-Arm architectures to avoid #ifdef. Move the definition of the
> magic pages on Arm to a more common place.
> 
> Note that the init-dom0less application of the diffenent Xen version
> may allocate all or part of four magic pages for each DomU.
> 
> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
> Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
> Signed-off-by: Henry Wang <xin.wang2@amd.com>
NIT: Generally, the first SOB is the same as author of the patch.

[...]
> diff --git a/xen/common/memory.c b/xen/common/memory.c
> index b3b05c2ec0..ab4bad79e2 100644
> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>          }
>          else
>          {
> -            if ( is_domain_direct_mapped(d) )
> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
I struggle to understand the goal of this patch and the proposed solution.
The problem with magic pages applies to static mem domUs in general.
A direct mapped domU is a static mem domU whose memory is 1:1 mapped.
Let's say we try to map a magic page for a direct mapped domU. That check will be false
and the execution will move to the next one i.e. is_domain_using_staticmem(d).
This check will be true and acquire_reserved_page() will fail instead (similar to the
static mem (no direct map) scenario). The only thing that has changed is the message.

~Michal


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:34:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:34:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685422.1066093 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYJs-0007kq-Sv; Mon, 26 Feb 2024 10:34:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685422.1066093; Mon, 26 Feb 2024 10:34:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYJs-0007kj-Q3; Mon, 26 Feb 2024 10:34:48 +0000
Received: by outflank-mailman (input) for mailman id 685422;
 Mon, 26 Feb 2024 10:34:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reYJq-0007kd-NN
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:34:46 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a8aa6587-d492-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 11:34:45 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-565c6cf4819so1935683a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 02:34:45 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 q32-20020a05640224a000b00564fa936b2bsm2237232eda.94.2024.02.26.02.34.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 02:34:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8aa6587-d492-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708943685; x=1709548485; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=N61N/9GC6VkqLeViNeOTGnAzYqt4hHA7mr7XqgOOGf4=;
        b=d5O0kELCEh88iSWsjPXYMXaGW2BwLPilcN1DPAVov29DVAe1Zc6QgaMH2XSndhL8Mk
         H4eGOSmOzaAbCHQV0E9EOMN87Ro8cfs2YJAAQFhlZ/3W6j79HdnZSyYz8mShC63ctxA3
         Okn1zALi8ib+ZAMirNYFHaq/VcWRpHEnY2ao3Vl5jcRSwm14wM9Wb9wAEVu/euGCfkXV
         LkHmXMSABVcUQBCcfMUeYOOwP3FoqX5mb1a+OpGOL2AhjcuvIZb4xdBsRLEwyQGK6VtD
         0oar2g5BR3Uo/aaGe+7gTe91VAMhD+wX8aSj378Dn/0NRiBLW1uZN+sLV3UfkUPOhSZh
         9I2A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708943685; x=1709548485;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=N61N/9GC6VkqLeViNeOTGnAzYqt4hHA7mr7XqgOOGf4=;
        b=GiesTM3MHrO9259knOjFKn5vM7G6/nifw5eIsmLJkL6wqVqwfPmkkEwlemZCw/Tw4G
         d7AJ/7+5YrbAAsYwG9tRq+iFrrNC3gjy2+nJ5j0S5ykcyUVxJD15FVjpifYX1OQFRfPX
         9sHyb/S1RO3/uZERwME+66BmK14gBdZShDo5ObAbemUKVEJ50c4VXhqj4GR3IE9wpWfh
         +UboBvRsM9cuvCzJqfnL+1ocDQuNu4Ri0rpHGmY1cRrya5pWXGEuRcYQ2tc0LJdUp479
         KxOjZMopWPwZ0rpSFKzrf8NpM1mxyS6CMP9Ol8Y17Gfbvg4rdx05dZscORWZZkdZDAK7
         GGiw==
X-Gm-Message-State: AOJu0YwqGQGFyGuChLvFzCHaoEYPfItLYz7lPTkVvUkV4z3U1YisvW7Z
	khM1kZ0VSel2MMhE3s+ninBpTxJZDuNNJk9eBTMO8y9xkBCSyw5xCtUI651QybfK1Z4BvhzHeJ8
	=
X-Google-Smtp-Source: AGHT+IHUMlb3j7Aa2H9FUCg1m1kk57GO3nMQ4PqtMNPACiNQ1huk0ClRyrTOvgLFxSLKo0/C8XCD7w==
X-Received: by 2002:a05:6402:3903:b0:561:ea0:e234 with SMTP id fe3-20020a056402390300b005610ea0e234mr5475337edb.16.1708943685158;
        Mon, 26 Feb 2024 02:34:45 -0800 (PST)
Message-ID: <239f2a0a-f1da-4c25-af43-dfac51bb1bd8@suse.com>
Date: Mon, 26 Feb 2024 11:34:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] arch: move array_index_mask_nospec()
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

At the time they were introduced, there were no asm/nospec.h yet, so
they were placed in system.h. Move them to nospec.h and drop
xen/nospec.h's including of asm/system.h; there's one unrelated #include
that needs adding in exchange, on x86.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- /dev/null
+++ b/xen/arch/arm/include/asm/arm32/nospec.h
@@ -0,0 +1,31 @@
+/* Portions taken from Linux arch arm */
+#ifndef __ASM_ARM32_NOSPEC_H
+#define __ASM_ARM32_NOSPEC_H
+
+#define CSDB    ".inst  0xe320f014"
+
+static inline unsigned long array_index_mask_nospec(unsigned long idx,
+                                                    unsigned long sz)
+{
+    unsigned long mask;
+
+    asm volatile( "cmp    %1, %2\n"
+                  "sbc    %0, %1, %1\n"
+                  CSDB
+                  : "=r" (mask)
+                  : "r" (idx), "Ir" (sz)
+                  : "cc" );
+
+    return mask;
+}
+#define array_index_mask_nospec array_index_mask_nospec
+
+#endif /* __ASM_ARM32_NOSPEC_H */
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--- a/xen/arch/arm/include/asm/arm32/system.h
+++ b/xen/arch/arm/include/asm/arm32/system.h
@@ -48,24 +48,6 @@ static inline int local_fiq_is_enabled(v
     return !(flags & PSR_FIQ_MASK);
 }
 
-#define CSDB    ".inst  0xe320f014"
-
-static inline unsigned long array_index_mask_nospec(unsigned long idx,
-                                                    unsigned long sz)
-{
-    unsigned long mask;
-
-    asm volatile( "cmp    %1, %2\n"
-                  "sbc    %0, %1, %1\n"
-                  CSDB
-                  : "=r" (mask)
-                  : "r" (idx), "Ir" (sz)
-                  : "cc" );
-
-    return mask;
-}
-#define array_index_mask_nospec array_index_mask_nospec
-
 #endif
 /*
  * Local variables:
--- /dev/null
+++ b/xen/arch/arm/include/asm/arm64/nospec.h
@@ -0,0 +1,35 @@
+/* Portions taken from Linux arch arm64 */
+#ifndef __ASM_ARM64_NOSPEC_H
+#define __ASM_ARM64_NOSPEC_H
+
+#define csdb()  asm volatile ( "hint #20" : : : "memory" )
+
+/*
+ * Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz
+ * and 0 otherwise.
+ */
+static inline unsigned long array_index_mask_nospec(unsigned long idx,
+                                                    unsigned long sz)
+{
+    unsigned long mask;
+
+    asm volatile ( "cmp     %1, %2\n"
+                   "sbc     %0, xzr, xzr\n"
+                   : "=r" (mask)
+                   : "r" (idx), "Ir" (sz)
+                   : "cc" );
+    csdb();
+
+    return mask;
+}
+#define array_index_mask_nospec array_index_mask_nospec
+
+#endif /* __ASM_ARM64_NOSPEC_H */
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--- a/xen/arch/arm/include/asm/arm64/system.h
+++ b/xen/arch/arm/include/asm/arm64/system.h
@@ -58,28 +58,6 @@ static inline int local_fiq_is_enabled(v
     return !(flags & PSR_FIQ_MASK);
 }
 
-#define csdb()  asm volatile ( "hint #20" : : : "memory" )
-
-/*
- * Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz
- * and 0 otherwise.
- */
-static inline unsigned long array_index_mask_nospec(unsigned long idx,
-                                                    unsigned long sz)
-{
-    unsigned long mask;
-
-    asm volatile ( "cmp     %1, %2\n"
-                   "sbc     %0, xzr, xzr\n"
-                   : "=r" (mask)
-                   : "r" (idx), "Ir" (sz)
-                   : "cc" );
-    csdb();
-
-    return mask;
-}
-#define array_index_mask_nospec array_index_mask_nospec
-
 #endif
 /*
  * Local variables:
--- a/xen/arch/arm/include/asm/nospec.h
+++ b/xen/arch/arm/include/asm/nospec.h
@@ -4,6 +4,14 @@
 #ifndef _ASM_ARM_NOSPEC_H
 #define _ASM_ARM_NOSPEC_H
 
+#if defined(CONFIG_ARM_32)
+# include <asm/arm32/nospec.h>
+#elif defined(CONFIG_ARM_64)
+# include <asm/arm64/nospec.h>
+#else
+# error "unknown ARM variant"
+#endif
+
 static inline bool evaluate_nospec(bool condition)
 {
     return condition;
--- a/xen/arch/x86/include/asm/nospec.h
+++ b/xen/arch/x86/include/asm/nospec.h
@@ -38,6 +38,30 @@ static always_inline void block_speculat
     barrier_nospec_true();
 }
 
+/**
+ * array_index_mask_nospec() - generate a mask that is ~0UL when the
+ *      bounds check succeeds and 0 otherwise
+ * @index: array element index
+ * @size: number of elements in array
+ *
+ * Returns:
+ *     0 - (index < size)
+ */
+static inline unsigned long array_index_mask_nospec(unsigned long index,
+                                                    unsigned long size)
+{
+    unsigned long mask;
+
+    asm volatile ( "cmp %[size], %[index]; sbb %[mask], %[mask];"
+                   : [mask] "=r" (mask)
+                   : [size] "g" (size), [index] "r" (index) );
+
+    return mask;
+}
+
+/* Override default implementation in nospec.h. */
+#define array_index_mask_nospec array_index_mask_nospec
+
 #endif /* _ASM_X86_NOSPEC_H */
 
 /*
--- a/xen/arch/x86/include/asm/system.h
+++ b/xen/arch/x86/include/asm/system.h
@@ -224,30 +224,6 @@ static always_inline unsigned long __xad
 #define smp_mb__before_atomic()    do { } while (0)
 #define smp_mb__after_atomic()     do { } while (0)
 
-/**
- * array_index_mask_nospec() - generate a mask that is ~0UL when the
- *      bounds check succeeds and 0 otherwise
- * @index: array element index
- * @size: number of elements in array
- *
- * Returns:
- *     0 - (index < size)
- */
-static inline unsigned long array_index_mask_nospec(unsigned long index,
-                                                    unsigned long size)
-{
-    unsigned long mask;
-
-    asm volatile ( "cmp %[size], %[index]; sbb %[mask], %[mask];"
-                   : [mask] "=r" (mask)
-                   : [size] "g" (size), [index] "r" (index) );
-
-    return mask;
-}
-
-/* Override default implementation in nospec.h. */
-#define array_index_mask_nospec array_index_mask_nospec
-
 #define local_irq_disable()     asm volatile ( "cli" : : : "memory" )
 #define local_irq_enable()      asm volatile ( "sti" : : : "memory" )
 
--- a/xen/arch/x86/include/asm/x86_emulate.h
+++ b/xen/arch/x86/include/asm/x86_emulate.h
@@ -15,6 +15,7 @@
 #include <xen/types.h>
 #include <xen/lib.h>
 #include <asm/regs.h>
+#include <asm/x86-defns.h>
 
 #include "../../x86_emulate/x86_emulate.h"
 
--- a/xen/include/xen/nospec.h
+++ b/xen/include/xen/nospec.h
@@ -7,7 +7,6 @@
 #ifndef XEN_NOSPEC_H
 #define XEN_NOSPEC_H
 
-#include <asm/system.h>
 #include <asm/nospec.h>
 
 /**


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:36:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:36:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685425.1066103 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYLq-0008HD-8i; Mon, 26 Feb 2024 10:36:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685425.1066103; Mon, 26 Feb 2024 10:36:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYLq-0008H6-4l; Mon, 26 Feb 2024 10:36:50 +0000
Received: by outflank-mailman (input) for mailman id 685425;
 Mon, 26 Feb 2024 10:36:49 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1reYLp-0008GS-4c
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:36:49 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f1f8085e-d492-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 11:36:48 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-512d19e2cb8so4314657e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 02:36:48 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 q7-20020ae9e407000000b00787ba4095dfsm2324216qkc.32.2024.02.26.02.36.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 02:36:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f1f8085e-d492-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708943808; x=1709548608; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=M9dwdsatRtu4h3pJta5iIlPVIMvAOIQKBR2N8KXJfCw=;
        b=QNLe7DS78ZzlH3VB5q9zTJnPZQuIXlzHNq4WQMKFw9eR1W+EgmCaN5Iu4+FH5Zlq8v
         OchXHdIym2irIl3JyAGybtx7h99eE8q7QT3J4UsdS3vNC0+BDZ+M1aChjwRarZisXSNW
         E+wqhoPrcgKSmgc7tzmrSITDYfrBUo/YJxdVc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708943808; x=1709548608;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=M9dwdsatRtu4h3pJta5iIlPVIMvAOIQKBR2N8KXJfCw=;
        b=sKw1W2seEKc+nM1wb1MMq4TDpkykQqQTd+EkzPs6FA5EWy44Z3KswleQqGG3P/qW5v
         1rCd2PNY4zfp4KlABV6gnabfq1D/jRz2yAnH6QxdDG4zrh6v3qOHdibrKS3sd2GWxLlQ
         HmkS4aapjKoOZSlEtxMItTBtRGA+Cv5oKS7OZSC4WQnLhv7LOgN9zlQSL2MhfLoQsucy
         7A7KtR7iLwlc+yLIJkWV7aoSPrEUoZ0tqxLrcgN7zeSCfwUHrDHfHSlWyZ9n6RkLr+mz
         5Gel7r1miCpuanmkW/ar0A90zHUa6RC+xfmsFrvrn5lm+mggVckCmQEeTq7tEtRRJzE9
         4mow==
X-Forwarded-Encrypted: i=1; AJvYcCWeEaH+bhF0cVOvGqrurKwcAmpATnOCsYcuYWIjoCrjJuoa5ak0B89IWJV+tHpWsT5MRET/mnnzNPV2iku2cjb0jhGPPSDsYRoMDI8M0Eg=
X-Gm-Message-State: AOJu0Ywr2gOswmAMwc1YAHgB1HVDPoeKOJ4quwqzr/+y3IvZ0YkFbmA2
	fJcop34VomDOM12xeLwMCQTqLh0mvw6Y4dxmb9klDw8C0+4fDtIIbKVMjunFqgM=
X-Google-Smtp-Source: AGHT+IHsYz0w3yuGvY/iVHSLc1a069UyeTmFg+2p8jXTUoXw45vfrcnbfRnYnp3qZAJbAXZtvLRsxg==
X-Received: by 2002:a05:6512:110e:b0:512:fd2a:872e with SMTP id l14-20020a056512110e00b00512fd2a872emr2033465lfg.18.1708943808008;
        Mon, 26 Feb 2024 02:36:48 -0800 (PST)
Message-ID: <16d40fe0-ff9e-4b11-be39-4eef472efe0b@citrix.com>
Date: Mon, 26 Feb 2024 10:36:45 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] arch: move array_index_mask_nospec()
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <volodymyr_babchuk@epam.com>
References: <239f2a0a-f1da-4c25-af43-dfac51bb1bd8@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <239f2a0a-f1da-4c25-af43-dfac51bb1bd8@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26/02/2024 10:34 am, Jan Beulich wrote:
> At the time they were introduced, there were no asm/nospec.h yet, so
> they were placed in system.h. Move them to nospec.h and drop
> xen/nospec.h's including of asm/system.h; there's one unrelated #include
> that needs adding in exchange, on x86.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:43:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:43:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685429.1066114 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYSF-000278-1Y; Mon, 26 Feb 2024 10:43:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685429.1066114; Mon, 26 Feb 2024 10:43:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYSE-000271-TT; Mon, 26 Feb 2024 10:43:26 +0000
Received: by outflank-mailman (input) for mailman id 685429;
 Mon, 26 Feb 2024 10:43:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=5KjY=KD=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1reYSE-00026v-A6
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:43:26 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dd56e116-d493-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 11:43:24 +0100 (CET)
Received: from CY5PR18CA0059.namprd18.prod.outlook.com (2603:10b6:930:13::22)
 by SA1PR12MB9245.namprd12.prod.outlook.com (2603:10b6:806:3a7::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.34; Mon, 26 Feb
 2024 10:43:21 +0000
Received: from CY4PEPF0000E9D7.namprd05.prod.outlook.com
 (2603:10b6:930:13:cafe::fd) by CY5PR18CA0059.outlook.office365.com
 (2603:10b6:930:13::22) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend
 Transport; Mon, 26 Feb 2024 10:43:21 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 CY4PEPF0000E9D7.mail.protection.outlook.com (10.167.241.78) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 26 Feb 2024 10:43:20 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb
 2024 04:43:18 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb
 2024 04:43:18 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 26 Feb 2024 04:43:17 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dd56e116-d493-11ee-8a58-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=KO3fNJYsXRFS8x329BGiiw3AUUFQ3RiCeQLDRHoODMv4f3Df8GfuV7eLIZXY1C6LsOpwyo8k0Eed3ZKTZBu+u0M2Vqcg4yEHcoocvj7mVGWB5qzEbSi37dyKau8LV47ovhqzdU2vW4kW1ls8HII9EeMw2FNSDOeoDo69temux9Z7NqPxqazovTR9qQj1hUe51PylDuEr3fx7Lq/Tewf1VUZ9K3+qpj5dB96KL4iHVt7eu5y0nIluz3Cv/HDLfVNrx79/H4lnhR7dUo/EjOD6rYsrozhO0T94r3srt8Mdy6zaZd1ODl0OzldI45huMydOIGH2klGHP/v572UVpwWf0A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=22VHKlc203k+dAjDW5lk4FTjC3jvnaX7PtHMAahlusc=;
 b=Vnm2NpUU/60YRcrtym9UFojaDwEHElmoEmBKEd3qHjcLh9VSlJtvLQLWlOuj/CxgXaY5vWTC9AJ+zs7DWmkUwmpxDa2EngrVKsaM6betzh+WjH9cxziCyKF42M0qwRcUMlcdHVYGjpzXjTZ56nlxOxEvAm41PmYTiWi72Sr5Yt4L/FrWO6i98UTXi9vcGT3rGmfhYLpGotWIiXND4cLIV7lMAQwu0fLZD39fp8txAXYYPiElxLxUGfkKfPnpkU+i2FbdJA3Q5l2zHUMNsCJPHJn50FMeLOWc0sPF6b+8T6DUiJwqaojVa/PnQz+QBgt5VOGpWEmF0gjnTgLYlojmHw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=22VHKlc203k+dAjDW5lk4FTjC3jvnaX7PtHMAahlusc=;
 b=rTMT/w/dRSsVz806O1wMgTKILrMCVkZXPG/NUqMTZfSZtNVWg7RwjzSdQMwLD8zkCqP4ki1glJTz4U0AwnP580KXZXHG/OvajtrV+KiXMGgyc014FRI+e7/uqWrMxKBq/I3bj17JFoHR4qdCvmPcsedqxBrofU9yzFzA9YhYI5E=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <e5cad59d-9c3f-4e4c-8b97-5e0fbd4eeda2@amd.com>
Date: Mon, 26 Feb 2024 11:43:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Set correct per-cpu cpu_core_mask
Content-Language: en-US
To: "Wang, Henry" <Xin.Wang2@amd.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240226030146.179465-1-xin.wang2@amd.com>
 <de4c6988-7051-4d44-9b44-e850ad4b4cc1@amd.com>
 <DS0PR12MB9448472A793F33228BE6A185A15A2@DS0PR12MB9448.namprd12.prod.outlook.com>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <DS0PR12MB9448472A793F33228BE6A185A15A2@DS0PR12MB9448.namprd12.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D7:EE_|SA1PR12MB9245:EE_
X-MS-Office365-Filtering-Correlation-Id: c490cd9f-2d75-48ea-ce85-08dc36b7bfde
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	Ssm7runJnLBYC2+svwPJH9RWno3NX661adpBIf4Q9+M21g+YaeD0sGi6DbpKkW+uaVWpDMvIAbxM37IAQVTfK67Q3M6SqylZ2v6IiRLWMMTRvBzHfpm/1/jESmaTWoOsaykFvFMhymZK2QYCIzJd0R9Bvv8SBnhYRG9dmE3M4NzE2Qj9LG9Z+KqjaTWKPF9jP+szuu/g7Blo1/2f/JD6I2ZlW+er1vIQ/yVsAQGCw5KOSA54nzQce0C+jjlRwEzCekIw/sYXT2r1U+DBt+uGdzsXfREDeLiM6i8WrE1SaJjmPZRat1rWUDxRoyOibblCmSGChrGiPBlL6O2txkjVGkJbarzATg59D9LQO3n2/ig2nILX32/aMiZiuO49NuTPJfKWdfgXiswegNn8btW5Q33effPnKq0soA0BRqlT/a1qC5afa3ZD7QMsSoPxfGQ8RNawrVY7+ipYpZRm+xZq+zsu4XkzxZ65upihar8k6g7C6DwZfCfVL+vbKZo5vAaSTQZ+3KEw6Ucmqld/ciuBHnOP+eICvnxqZwQygMeTfGbiThTC3w8qhTDhPm3Qyr3vwhBF2H7jpIJnSQgXmvk9yuR6jVWxjwlvK9KN8adg5iB30iras2pAAl7YWmhGhU1lFSCTSKnSJU2guZhSjeH1aYTjhXD8bVXVkzu6f5Sa9oPm2btXu20cY3aiDzEr81d9I+TJO8ttBWKyLEbF1/WBf0jEwT/mIyjby6O2H2htZKWgk5Gtt0FpT5fm6xTDDxcw
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 10:43:20.4639
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c490cd9f-2d75-48ea-ce85-08dc36b7bfde
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000E9D7.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB9245



On 26/02/2024 10:54, Wang, Henry wrote:
> [AMD Official Use Only - General]
> 
> Hi Michal,
> 
>> -----Original Message-----
>> Subject: Re: [PATCH] xen/arm: Set correct per-cpu cpu_core_mask
>>
>> Hi Henry,
>>
>> On 26/02/2024 04:01, Henry Wang wrote:
>>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
>> NIT: You first sent this patch as part of NUMA series:
>> https://lore.kernel.org/xen-devel/20231120025431.14845-16-
>> Henry.Wang@arm.com/
>> Shouldn't you retain the Arm's authorship?
> 
> Ah good point, in fact I don't really know, since I basically rewrote
> the patch I thought it is not really needed. I will add it back in v2 since
> you mentioned this.
> 
>>> ---
>>>  xen/arch/arm/smpboot.c | 6 +++++-
>>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>>
>>> +    /* PE not implemented using a multithreading type approach. */
>>> +    if ( system_cpuinfo.mpidr.mt == 0 )
>> Do we need this check? It mt was true, cpu_sibling_mask would be incorrect
>> anyway (it would still be 1).
> 
> I added this check for playing safe, because I only want to do the correct thing
> in this patch and avoid make things worse for MT case. With this patch, non-MT
> case can be improved and the MT case is remain unchanged.
> 
> But I agree with you, and I would be more than happy if I can run a MT setup and
> finish the "else" part with this patch or follow-ups. Do you know maybe qemu can
> allow me to emulate a MT setup so that I can fix it properly in v2? Thanks!
A65 is the only Arm CPU with SMT and I'm not aware of Qemu being able to emulate it.
AFAICT, in Xen on Arm we assume no SMT, hence my question about your check. With or without it,
some parts would still be incorrect (like cpu_sibling_mask), so what's the point in having a partial check.
I would keep your solution without the check. Others may have a different opinion though.

~Michal


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:43:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:43:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685430.1066123 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYSV-0002Qo-8m; Mon, 26 Feb 2024 10:43:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685430.1066123; Mon, 26 Feb 2024 10:43:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYSV-0002Qh-54; Mon, 26 Feb 2024 10:43:43 +0000
Received: by outflank-mailman (input) for mailman id 685430;
 Mon, 26 Feb 2024 10:43:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reYSU-00026v-5z
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:43:42 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e829e2ba-d493-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 11:43:41 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a28a6cef709so445091266b.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 02:43:41 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m14-20020a170906234e00b00a3d11feb32esm2278193eja.186.2024.02.26.02.43.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 02:43:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e829e2ba-d493-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708944221; x=1709549021; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=kkLFz815u90oFK+cyB0VCAdP704LGkuFGdIMHsiNoXk=;
        b=HnF3ghsTCLc/UnYNpRaNlDP1RVdz1RtDR+24LVfYIoWsVaqQwTCU/ZEh1O991jOb3/
         KiK6mGl5ojkIACwRuu5n2ldSTtU4DsLA8HvY2BDNVDXxYy7VcYIeCtAwpmVvPFSe192b
         pp+qcXpkbhQ5t3yUObnm6+PcIuAGpUusiDdmb01z39dMcoQebaC5lUp6riidBvSryBMV
         AckFzd++o7ljAtJsO8N4pIdmWGz6hdBkfPx9bC93YrX7fH86WD75wdTv3iLWA6D5VFkO
         +FQUTRvkHfPOUdcAQ8H3B3vMD/1/EyyH/5Vj0nopQpyuSJzsmwbnFV4+f6r4+sgDVfBP
         RwFw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708944221; x=1709549021;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=kkLFz815u90oFK+cyB0VCAdP704LGkuFGdIMHsiNoXk=;
        b=ZWAcS2G4PvVyEHKH9Q/qEB3B0Xv0X3PMihfyU6yjETQ8KkaQM6SzdfFK+LncmpOt94
         Lpk6wcAKXYDBM1loHqf28lk/WStGTk1VBIqMF1Job6DcQBmYmvSTXWG0tT0lh2AaBVpE
         Dr2c8YahpBvAt/6P+JIBEW2AiiA0irozLOmo9zgye0suoajqW5YNZT3dEJ+MRSdOzCEe
         K9fokBhIt+iaqCmhJKmFwCS/zueh5cZMS/r+c9ToyW3n3yBO02tLxljocYxV3vm/0a71
         VMeq/G7FD1cr6qgE8rD8NW++S8TAWWiUL7Eh4h4J6hanmgj+SJHx9fSewl/XUy/aupZ+
         xqNw==
X-Gm-Message-State: AOJu0Yw/W+JUpToq93dkcfxVC67PREDi227VfwqKQ5n+otiDTA1CbCN2
	jtoGXx8r04dFXjimslL/7+vt1kOJqf6S2lwvRFmmDGwzW4WPsOsZQn75dk1pd7uMWYK1NjKT68Y
	=
X-Google-Smtp-Source: AGHT+IHfbiA8xXwqgcMkUSmP5UyGdoPWknGwuDaATFEuuUAcGukTENPcOVenSG88zOwZmm7Wqi2ADA==
X-Received: by 2002:a17:906:3659:b0:a3e:da6:85f8 with SMTP id r25-20020a170906365900b00a3e0da685f8mr4317975ejb.30.1708944221200;
        Mon, 26 Feb 2024 02:43:41 -0800 (PST)
Message-ID: <94418e0e-1805-4c30-a4a7-4e9cbfaf984e@suse.com>
Date: Mon, 26 Feb 2024 11:43:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v2 0/2] gnttab: hypervisor side XSA-448 follow-up
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

1: almost fully ignore zero-size flush requests
2: restrict GNTTABOP_cache_flush to Arm

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:44:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:44:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685433.1066132 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYTf-00038G-HX; Mon, 26 Feb 2024 10:44:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685433.1066132; Mon, 26 Feb 2024 10:44:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYTf-000389-F0; Mon, 26 Feb 2024 10:44:55 +0000
Received: by outflank-mailman (input) for mailman id 685433;
 Mon, 26 Feb 2024 10:44:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reYTd-00037z-L7
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:44:53 +0000
Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com
 [2a00:1450:4864:20::536])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 128ca2d5-d494-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 11:44:52 +0100 (CET)
Received: by mail-ed1-x536.google.com with SMTP id
 4fb4d7f45d1cf-565a3910f86so2537903a12.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 02:44:52 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d6-20020a50ea86000000b005661badcccesm44605edo.87.2024.02.26.02.44.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 02:44:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 128ca2d5-d494-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708944292; x=1709549092; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=SV4cvth6dNGeBZBn9gXoqeVvBbQTC9ZF63evmEgB9hA=;
        b=MPudo0z3LnN5tgQ+S3EDj+RatihKperlWuF4FUli6GYzmzMz8nNkx/tSTIGU+JwIfk
         mndYXrzpMxrWEl4Vx8oAdnFJT85yk09QATeMu7oQC8EIgQBAUc0s+aSLTr2nbFZPPfyJ
         ymlusOEEAKlYS8e4YdWxnrSTSKhuUvQU9QGrutetmHB/YJhPIG8jtyJoTlXU0WSA+Wqr
         JlErmgRcli03Gmf4X67I/Aq0xf79NtMiIkk/Aodvai1I5Rlh0iUmDAvhoetEmkfBoPPn
         CtB21GpdmVEr6fPRsdoGeUh5h4SNQGOpzCFinxdPZdKd1YHtBvVljgRE56dQIp/adkJp
         NTWg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708944292; x=1709549092;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=SV4cvth6dNGeBZBn9gXoqeVvBbQTC9ZF63evmEgB9hA=;
        b=UqijV1cg9muUu8DLGhAfSJNUAxTPRfEsjUDVN6fwxIVmVF5UV0DB7IKVUqtnFg8WQc
         uP+3mjjIZ6sYJrqwQfQD874aoBHOLhTU88V5Jn2oRtlDgHoltD8rZCzbuoYowUWG5sT/
         IJmIhQOp7X7h1ZTqyEilY4Xm/C4XjfHRH029WESS0jclP8zunqZHSZ/NO11Be1b4qK6S
         hctMpNAN23JlRXf04009aTS2Std5oasMLJYuZPweLTlPT/7CZXOSK8B0mlP3qCnFWzLy
         zrPuC8PPx54zi92Qy55tg9KCQSQqFMju7Npmcik8EqGmBH4n/N1VNYG8wE9moAsJqOTJ
         3noQ==
X-Gm-Message-State: AOJu0Ywdjbvgq0DbsIcdxAYYLQVZ7dbD13a9IlJBXslyAfDYKBfo8Poc
	uYjJPJxFEw4/unToZhgJdEam2f4HLueNUe8LGvOT2RQIidMUy6uVdvVHNOYPQ1SrpJikLJ7REfc
	=
X-Google-Smtp-Source: AGHT+IH0I8RtAXffz31xuxeoGlz1JfuXlwAua+8YVei1OzZ4Frbb+0Qc59Er7I3BvpwBkyDcOhDJXg==
X-Received: by 2002:aa7:c0da:0:b0:565:cbd9:af39 with SMTP id j26-20020aa7c0da000000b00565cbd9af39mr2643097edp.16.1708944292290;
        Mon, 26 Feb 2024 02:44:52 -0800 (PST)
Message-ID: <e160ebb3-b313-4856-b161-2ac1ab0b61ca@suse.com>
Date: Mon, 26 Feb 2024 11:44:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 1/2] gnttab: almost fully ignore zero-size flush requests
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <94418e0e-1805-4c30-a4a7-4e9cbfaf984e@suse.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <94418e0e-1805-4c30-a4a7-4e9cbfaf984e@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Along the line with observations in the context of XSA-448, besides
"op" no field is relevant when the range to be flushed is empty, much
like e.g. the pointers passed to memcpy() are irrelevant (and would
never be "validated") when the passed length is zero. Split the existing
condition validating "op", "offset", and "length", leaving only the "op"
part ahead of the check for length being zero (or no flushing to be
performed).

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: Undo expression folding.

--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3528,15 +3528,17 @@ static int _cache_flush(const gnttab_cac
     void *v;
     int ret;
 
-    if ( (cflush->offset >= PAGE_SIZE) ||
-         (cflush->length > PAGE_SIZE) ||
-         (cflush->offset + cflush->length > PAGE_SIZE) ||
-         (cflush->op & ~(GNTTAB_CACHE_INVAL | GNTTAB_CACHE_CLEAN)) )
+    if ( cflush->op & ~(GNTTAB_CACHE_INVAL | GNTTAB_CACHE_CLEAN) )
         return -EINVAL;
 
     if ( cflush->length == 0 || cflush->op == 0 )
         return !*cur_ref ? 0 : -EILSEQ;
 
+    if ( (cflush->offset >= PAGE_SIZE) ||
+         (cflush->length > PAGE_SIZE) ||
+         cflush->offset + cflush->length > PAGE_SIZE )
+        return -EINVAL;
+
     /* currently unimplemented */
     if ( cflush->op & GNTTAB_CACHE_SOURCE_GREF )
         return -EOPNOTSUPP;



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:45:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:45:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685435.1066142 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYUL-0003d6-Pq; Mon, 26 Feb 2024 10:45:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685435.1066142; Mon, 26 Feb 2024 10:45:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYUL-0003cz-Mw; Mon, 26 Feb 2024 10:45:37 +0000
Received: by outflank-mailman (input) for mailman id 685435;
 Mon, 26 Feb 2024 10:45:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reYUJ-0003ck-TC
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:45:35 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2acc6136-d494-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 11:45:33 +0100 (CET)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-5643eccad0bso4514060a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 02:45:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d6-20020a50ea86000000b005661badcccesm44605edo.87.2024.02.26.02.45.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 02:45:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2acc6136-d494-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708944333; x=1709549133; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=1KdKJYo5Q7bYuv+aHSBf3jqyyInf5NNlws7iGpvocd8=;
        b=DHAAoJ0q/Yo1V4BNvkgI1/4fXDG8wpmuM6cQOEzWzEOVGomXKIIyLX//D7q+lpA2Io
         Gwv//a5053j+8JEnhsmFeWPEZBrJ/Qzad7t4PC94MZ8dUPF2+lcB+7ZxXtvMiYVn0dAi
         c5b0/obnqEc7tb9J68E+dN+U/pPYp0XQtFsUogaUcyEmjvFDnM8VsyYhNzRLdee6lEbp
         dKldaMywSTzD++vU8aYiJ9fIR+BOFmzFHmeMQ5f16mmLnNBMKR2Zmnh96Ovzv5JZLFxP
         8ff7dbpB4SJckYw39FutL4XtBIFvaaDBohrSWojxiEWMJtTQYVO1ReRJwIONlbxyv320
         rJZA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708944333; x=1709549133;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=1KdKJYo5Q7bYuv+aHSBf3jqyyInf5NNlws7iGpvocd8=;
        b=dsBtxpAD+EMdpocPwYVisXR9vJyB8YlczNS9kt4jXkc4IJEyr3EuiPQ5yBmQO/MX8k
         sKy2W2QaTj3tpqVYnzbcguJTD8AOERHvb2F6vytI9RVQzJJPHgUridsDqX2b1EQBoZG5
         LfJW+g4785TReWkV32Z6gwmiECnXNMtArKEyozHmcJAYQCOYm/4SVfb24st1q/uckARQ
         1G383vZISP6gQPP6uThOxCYoKthfZvSe409hdwAVZA4XfKbchHxzOQhjKSjdsDrotBMS
         IrdGoKEUclRbDcMKSG4LagdrP6khj6sxFaSdz0NAL40xRclGNblg/SdaB6HiWauMn06n
         0orw==
X-Gm-Message-State: AOJu0Yw1sNTTp8XD79RQzNFGvD1Dc0J8L8rNgDivBDeuQbbtlefqibAL
	wdMCHe0+xc6/KFHViE1GNZx6ukALMWyNPQ1dtkdfziiRbgevpDMJlSDZ75cOeQ9ariYZgxoyKDI
	=
X-Google-Smtp-Source: AGHT+IEHk7mKkhLGf0l55aNiymIDjGoQxhCcEYQLWbg+pKFgyX85fYEIa5wskm+8TG+wN6HFO0QYyg==
X-Received: by 2002:aa7:d416:0:b0:564:39b6:fe9 with SMTP id z22-20020aa7d416000000b0056439b60fe9mr4530036edq.12.1708944333355;
        Mon, 26 Feb 2024 02:45:33 -0800 (PST)
Message-ID: <0c2e7ded-45c2-485f-9184-c71a58f56109@suse.com>
Date: Mon, 26 Feb 2024 11:45:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v2 2/2] gnttab: restrict GNTTABOP_cache_flush to Arm
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <94418e0e-1805-4c30-a4a7-4e9cbfaf984e@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <94418e0e-1805-4c30-a4a7-4e9cbfaf984e@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

This special purpose operation possibly shouldn't have been a grant
table sub-op, but instead an arch-specific memory management one. No
users are known or supposed to exist outside of Arm. Along with adding
suitable #ifdef-s, move a helper function used solely here into the
first of these new #ifdef-s.

With the functionality restricted, questionable helper functions can
then also be purged from x86.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/include/asm/flushtlb.h
+++ b/xen/arch/x86/include/asm/flushtlb.h
@@ -182,21 +182,6 @@ void flush_area_mask(const cpumask_t *ma
 }
 
 static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache) {}
-static inline int invalidate_dcache_va_range(const void *p,
-                                             unsigned long size)
-{ return -EOPNOTSUPP; }
-static inline int clean_and_invalidate_dcache_va_range(const void *p,
-                                                       unsigned long size)
-{
-    unsigned int order = get_order_from_bytes(size);
-    /* sub-page granularity support needs to be added if necessary */
-    flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
-    return 0;
-}
-static inline int clean_dcache_va_range(const void *p, unsigned long size)
-{
-    return clean_and_invalidate_dcache_va_range(p, size);
-}
 
 unsigned int guest_flush_tlb_flags(const struct domain *d);
 void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask);
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -940,42 +940,6 @@ static void reduce_status_for_pin(struct
         gnttab_clear_flags(rd, clear_flags, status);
 }
 
-static struct active_grant_entry *grant_map_exists(const struct domain *ld,
-                                                   struct grant_table *rgt,
-                                                   mfn_t mfn,
-                                                   grant_ref_t *cur_ref)
-{
-    grant_ref_t ref, max_iter;
-
-    /*
-     * The remote grant table should be locked but the percpu rwlock
-     * cannot be checked for read lock without race conditions or high
-     * overhead so we cannot use an ASSERT
-     *
-     *   ASSERT(rw_is_locked(&rgt->lock));
-     */
-
-    max_iter = min(*cur_ref + (1 << GNTTABOP_CONTINUATION_ARG_SHIFT),
-                   nr_grant_entries(rgt));
-    for ( ref = *cur_ref; ref < max_iter; ref++ )
-    {
-        struct active_grant_entry *act = active_entry_acquire(rgt, ref);
-
-        if ( act->pin && act->domid == ld->domain_id &&
-             mfn_eq(act->mfn, mfn) )
-            return act;
-        active_entry_release(act);
-    }
-
-    if ( ref < nr_grant_entries(rgt) )
-    {
-        *cur_ref = ref;
-        return NULL;
-    }
-
-    return ERR_PTR(-EINVAL);
-}
-
 union maptrack_node {
     struct {
         /* Radix tree slot pointers use two of the bits. */
@@ -3519,6 +3483,44 @@ gnttab_swap_grant_ref(XEN_GUEST_HANDLE_P
     return 0;
 }
 
+#ifdef CONFIG_ARM
+
+static struct active_grant_entry *grant_map_exists(const struct domain *ld,
+                                                   struct grant_table *rgt,
+                                                   mfn_t mfn,
+                                                   grant_ref_t *cur_ref)
+{
+    grant_ref_t ref, max_iter;
+
+    /*
+     * The remote grant table should be locked but the percpu rwlock
+     * cannot be checked for read lock without race conditions or high
+     * overhead so we cannot use an ASSERT
+     *
+     *   ASSERT(rw_is_locked(&rgt->lock));
+     */
+
+    max_iter = min(*cur_ref + (1 << GNTTABOP_CONTINUATION_ARG_SHIFT),
+                   nr_grant_entries(rgt));
+    for ( ref = *cur_ref; ref < max_iter; ref++ )
+    {
+        struct active_grant_entry *act = active_entry_acquire(rgt, ref);
+
+        if ( act->pin && act->domid == ld->domain_id &&
+             mfn_eq(act->mfn, mfn) )
+            return act;
+        active_entry_release(act);
+    }
+
+    if ( ref < nr_grant_entries(rgt) )
+    {
+        *cur_ref = ref;
+        return NULL;
+    }
+
+    return ERR_PTR(-EINVAL);
+}
+
 static int _cache_flush(const gnttab_cache_flush_t *cflush, grant_ref_t *cur_ref)
 {
     struct domain *d, *owner;
@@ -3633,6 +3635,8 @@ gnttab_cache_flush(XEN_GUEST_HANDLE_PARA
     return 0;
 }
 
+#endif /* CONFIG_ARM */
+
 long do_grant_table_op(
     unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop, unsigned int count)
 {
@@ -3776,6 +3780,7 @@ long (do_grant_table_op)(
 
     case GNTTABOP_cache_flush:
     {
+#ifdef CONFIG_ARM
         XEN_GUEST_HANDLE_PARAM(gnttab_cache_flush_t) cflush =
             guest_handle_cast(uop, gnttab_cache_flush_t);
 
@@ -3788,6 +3793,9 @@ long (do_grant_table_op)(
             uop = guest_handle_cast(cflush, void);
             opaque_out = opaque_in;
         }
+#else
+        rc = -EOPNOTSUPP;
+#endif
         break;
     }
 



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:48:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:48:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685438.1066153 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYXD-0004Jv-AZ; Mon, 26 Feb 2024 10:48:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685438.1066153; Mon, 26 Feb 2024 10:48:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYXD-0004Jo-6l; Mon, 26 Feb 2024 10:48:35 +0000
Received: by outflank-mailman (input) for mailman id 685438;
 Mon, 26 Feb 2024 10:48:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1reYXB-0004Ji-Uq
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:48:33 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 94f8aea2-d494-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 11:48:31 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a3ddc13bbb3so425934466b.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 02:48:31 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 m16-20020a1709061ed000b00a3d99415705sm2294599ejj.73.2024.02.26.02.48.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 02:48:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 94f8aea2-d494-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708944511; x=1709549311; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Q8zVpDwchDEJOfDQ6FiF4/+UrPVWrsLuWj02A8p+uBQ=;
        b=Ejnf7BEZWFZ0eOLgrkikIDr8D0ceQ1+LwPKrNmic0Kp0JoQJL58s3QUl9J/Ry9V5XS
         hCzPhG3+rGyz1XUwt166RXTHcQMsFe4Lmfi4tAGcMWLYMEesdqc+Aw+fzTvSx0MbGbo9
         DLqTnEUMsuezY7BvBAXCPgZMf4+kuLezMo9kw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708944511; x=1709549311;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Q8zVpDwchDEJOfDQ6FiF4/+UrPVWrsLuWj02A8p+uBQ=;
        b=VedqTQKZ4DHSeByDmwCPqgZqToEwqSkk92ncOrjzRYcFX5DuyYnDBHhIQwQTPXVwBx
         RHvdfGBX8guWXbJLd81giUa2cdc85+fXc+hIeo3tttoAnMc7RWuetREuVWU9wEgkTx79
         FgAlZqkR/gTVzJI6fhHKJF1c0RLeBzsgYoo8/Rqb/o9tXiuFjvrbxWPiB94HbIcyqEjs
         5Hrgvkrsx83WXkSqZIXE+2F0JY0U6QgjA5UnUSuj4LGVuN4jRibYdMOOHxsjIlna0UT7
         34cUA4pUnJI8K6U0i0P8XX9PrY10K7HLI2HnSuNxtxsjXa+/Zktt/BBc+gIGOZOXkYHF
         YkWg==
X-Gm-Message-State: AOJu0YwXXRR2fG2EZk+zMW1L3L/oQ0EZ9AZdAwEMh39SmsrUDO5azXv+
	WMsBfaU0kGaMG7KFV1bZ//Y5fovMrSZQqsXRR/Ar20NubRIyiyE/hvUMw2rXoKgYZAfdESBd0CS
	H
X-Google-Smtp-Source: AGHT+IE0dfRmVXZQ1UyuFMvRkiWjqXqO1MUL/BEfX2Ra1lI8bXduWYRqnPBjMC6mdzYFg9S38B5W1w==
X-Received: by 2002:a17:906:bc54:b0:a3f:c736:526d with SMTP id s20-20020a170906bc5400b00a3fc736526dmr5285610ejv.31.1708944511338;
        Mon, 26 Feb 2024 02:48:31 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Christian Lindig <christian.lindig@citrix.com>,
	=?UTF-8?q?Edwin=20T=C3=B6r=C3=B6k?= <edwin.torok@cloud.com>,
	Rob Hoes <Rob.Hoes@citrix.com>
Subject: [PATCH] tools/oxenstored: Re-format
Date: Mon, 26 Feb 2024 10:48:29 +0000
Message-Id: <20240226104829.1044479-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Rerun make format.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Christian Lindig <christian.lindig@citrix.com>
CC: Edwin Török <edwin.torok@cloud.com>
CC: Rob Hoes <Rob.Hoes@citrix.com>
---
 tools/ocaml/xenstored/quota.ml | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/tools/ocaml/xenstored/quota.ml b/tools/ocaml/xenstored/quota.ml
index 1f652040d898..082cd25f26fc 100644
--- a/tools/ocaml/xenstored/quota.ml
+++ b/tools/ocaml/xenstored/quota.ml
@@ -55,13 +55,13 @@ let _check quota id size =
     raise Data_too_big
   );
   if id > 0 then
-  try
-    let entry = DomidMap.find id quota.cur in
-    if entry >= quota.maxent then (
-      warn "domain %u cannot create entry: quota reached" id;
-      raise Limit_reached
-    )
-  with Not_found -> ()
+    try
+      let entry = DomidMap.find id quota.cur in
+      if entry >= quota.maxent then (
+        warn "domain %u cannot create entry: quota reached" id;
+        raise Limit_reached
+      )
+    with Not_found -> ()
 
 let check quota id size =
   if !activate then
@@ -88,4 +88,4 @@ let merge orig_quota mod_quota dest_quota =
     | diff -> update_entry dest id diff (* update with [x=x+diff] *)
   in
   {dest_quota with cur = DomidMap.fold fold_merge mod_quota.cur dest_quota.cur}
-  (* dest_quota = dest_quota + (mod_quota - orig_quota) *)
+(* dest_quota = dest_quota + (mod_quota - orig_quota) *)

base-commit: 8de3afc0b402bc17f65093a53e5870862707a8c7
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:50:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:50:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685440.1066163 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYZ2-0006DY-Jy; Mon, 26 Feb 2024 10:50:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685440.1066163; Mon, 26 Feb 2024 10:50:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYZ2-0006DR-HN; Mon, 26 Feb 2024 10:50:28 +0000
Received: by outflank-mailman (input) for mailman id 685440;
 Mon, 26 Feb 2024 10:50:27 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YLHk=KD=bombadil.srs.infradead.org=BATV+3075c4dbfe213c24d6b2+7491+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1reYYy-0006Bw-Fl
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:50:27 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d6d9575c-d494-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 11:50:23 +0100 (CET)
Received: from hch by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red
 Hat Linux)) id 1reYYq-00000000C2e-1All;
 Mon, 26 Feb 2024 10:50:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d6d9575c-d494-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Type:MIME-Version
	:References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=8zXJnYkWvb7GCIP0mRpV6jKEXdPSRHoRUoq4Q4XvFso=; b=1FVpbdFGoDA69PZZUcL4HiCtMz
	S6WFk/vpfQAB/kguloupNrgoRgK4vjbvfGKDw6TZHXyCBf52luksnZyy+2wbqip9cqyCkauoU5RxA
	9Jgay49gicYkCypB2ID8yeiUIOoCgg9Ww6kzWEF+BI2PRQBUjmWf31Fba4biwbLFNimlj804lPzhB
	OatWMdfGH7woOtg6A0Wa5soW7v2mb8shmzzPAv/pPb1n3wNayVkic9AQ5Tn9aRJOFvCMEVDGlMx5e
	MxaIBCxza0I8HOVlkQli3oksCZa2jSLmj0y25Lqa3m4xble00cszPiVEb4gvbECDXlYq6xaz1b5MR
	wjjQo3tg==;
Date: Mon, 26 Feb 2024 02:50:16 -0800
From: Christoph Hellwig <hch@infradead.org>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org,
	torvalds@linux-foundation.org, brho@google.com, hannes@cmpxchg.org,
	lstoakes@gmail.com, akpm@linux-foundation.org, urezki@gmail.com,
	hch@infradead.org, boris.ostrovsky@oracle.com,
	sstabellini@kernel.org, jgross@suse.com, linux-mm@kvack.org,
	xen-devel@lists.xenproject.org, kernel-team@fb.com
Subject: Re: [PATCH v2 bpf-next 1/3] mm: Enforce VM_IOREMAP flag and range in
 ioremap_page_range.
Message-ID: <Zdxs6DDSVtjaqK-d@infradead.org>
References: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
 <20240223235728.13981-2-alexei.starovoitov@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240223235728.13981-2-alexei.starovoitov@gmail.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

On Fri, Feb 23, 2024 at 03:57:26PM -0800, Alexei Starovoitov wrote:
> From: Alexei Starovoitov <ast@kernel.org>
> 
> There are various users of get_vm_area() + ioremap_page_range() APIs.
> Enforce that get_vm_area() was requested as VM_IOREMAP type and range passed to
> ioremap_page_range() matches created vm_area to avoid accidentally ioremap-ing
> into wrong address range.

Nit: overly long lines in the commit message here.

Otherwise looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:51:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:51:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685445.1066172 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYa6-000762-ST; Mon, 26 Feb 2024 10:51:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685445.1066172; Mon, 26 Feb 2024 10:51:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYa6-00075v-Pm; Mon, 26 Feb 2024 10:51:34 +0000
Received: by outflank-mailman (input) for mailman id 685445;
 Mon, 26 Feb 2024 10:51:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reYa5-00073t-DV
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:51:33 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 00fdfbe8-d495-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 11:51:32 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a3e7f7b3d95so323930766b.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 02:51:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 mb2-20020a170906eb0200b00a4136d1899esm2298073ejb.102.2024.02.26.02.51.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 02:51:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00fdfbe8-d495-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708944692; x=1709549492; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jeY+cGEmZnIIAKXDOyKTG+h2GGkHjGO9G7ePqwDf/iw=;
        b=Kbr6AbWZYr1jX1Aaq37EHhzqgGfQSTF//ea65bh4ZzuqqecsZz2n16OyzFFULa+H6w
         1e3BurneHeMb7WmAxN7A/UIaNWf5neSt3IDCS2XxG8CVMqODoA6ljUBJRH4RjX85nv6b
         j16r5bvUS9DP9GzSzm2QB5DpEuHmXnOZbRmecKKTk8ta6ivdWod6MELmZw1EjwkknwYn
         aVq4rvhDi4yRMW9iLo0CH9QDjXrPL1VsttTCu4fBXv4LdmEb+SVHzR+hwhufLmRqehW2
         w/A+Z+gAUHYpZn0VB4n411GdW2JGuBiLSlnMI8+VIp2Y9CKuVwUoXMBizaZtcQrZlv4e
         30/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708944692; x=1709549492;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jeY+cGEmZnIIAKXDOyKTG+h2GGkHjGO9G7ePqwDf/iw=;
        b=aUZEG7icRfWFXXgMdqDst+oJv8vo/YF1O95cjzGIrMLdGU5BOy0W4nWS/Xx7jFcILO
         IRyP1yseHYxLFQZnzPXYY9Mq7BZpxyyeuzKWuUU61neurYM9qEwXF2qLmrbZ68ESiFv9
         XxcuMfz5w7H3VuO6MTLiK1DvllLpAcdwCVH7VUwXvy8+opDtZfqiwOOhLhHTlyHTESo/
         dU5/yzHmVVZXee34TMGWikxvEkl1/XXWjtSHt/pc6yuCCqZz0u8BxOEeQ51h18FTdDwE
         o8SMDwtcOuMTZS87eODkW6StyrnsMw17iVEYTIB6aOHkD51Uefw6GASysbL4J4yuAyLy
         3J6w==
X-Forwarded-Encrypted: i=1; AJvYcCVqQEJwIiVpC26bnebn/eHkrHptA/emV5xa0TaPIRB2lbBhVi9CKYM6TUVozMyFFzPTUC09UM5457Y8zps5iJZPsAzgjPeT+F0WRMwh1zU=
X-Gm-Message-State: AOJu0YwYQj4ma7ReHUXDcDADC0Q3PBTby8NrIF0u+m5guhSguKxyQMOs
	gR2pMeaU+YwYD8TQmOaDpglDQbojkPz8XU7Al5g2SsWNkFOPbN1xbOWn/IMpSg==
X-Google-Smtp-Source: AGHT+IERsG3lulKGVYn6CVq3pqUdaCJBcRLfAIhmYvwVCYgUaC6XlWNLQBBr0vsYCWz+a0fDqEziBg==
X-Received: by 2002:a17:906:6d50:b0:a43:4613:3689 with SMTP id a16-20020a1709066d5000b00a4346133689mr1830854ejt.49.1708944692360;
        Mon, 26 Feb 2024 02:51:32 -0800 (PST)
Message-ID: <b8bb4534-b11c-4fa6-a617-0460f494a626@suse.com>
Date: Mon, 26 Feb 2024 11:51:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
 <4f3b3d52a3ba20b05ecfa068b916b804@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4f3b3d52a3ba20b05ecfa068b916b804@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 23.02.2024 08:58, Nicola Vetrini wrote:
> On 2024-02-19 16:14, Nicola Vetrini wrote:
>> The cache clearing and invalidation helpers in x86 and Arm didn't
>> comply with MISRA C Rule 17.7: "The value returned by a function
>> having non-void return type shall be used". On Arm they
>> were always returning 0, while some in x86 returned -EOPNOTSUPP
>> and in common/grant_table the return value is saved.
>>
>> As a consequence, a common helper arch_grant_cache_flush that returns
>> an integer is introduced, so that each architecture can choose whether 
>> to
>> return an error value on certain conditions, and the helpers have 
>> either
>> been changed to return void (on Arm) or deleted entirely (on x86).
>>
>> Signed-off-by: Julien Grall <julien@xen.org>
>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>> ---
>> The original refactor idea came from Julien Grall in [1]; I edited that 
>> proposal
>> to fix build errors.
>>
>> I did introduce a cast to void for the call to flush_area_local on x86, 
>> because
>> even before this patch the return value of that function wasn't checked 
>> in all
>> but one use in x86/smp.c, and in this context the helper (perhaps 
>> incidentally)
>> ignored the return value of flush_area_local.
>>
>> [1] 
>> https://lore.kernel.org/xen-devel/09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org/
>> ---
>>  xen/arch/arm/include/asm/page.h     | 33 ++++++++++++++++++-----------
>>  xen/arch/x86/include/asm/flushtlb.h | 23 ++++++++++----------
>>  xen/common/grant_table.c            |  9 +-------
>>  3 files changed, 34 insertions(+), 31 deletions(-)
>>
> 
> I'll put this patch in the backlog at the moment: too many intricacies 
> while trying to untangle xen/flushtlb from xen/mm.h, and there are 
> easier cases that can be done faster. If someone is interested I can 
> post the partial work I've done so far, even though it doesn't
> build on x86.

This
https://lists.xen.org/archives/html/xen-devel/2024-02/msg01513.html
may be of interest to you in the context here.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:51:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:51:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685448.1066183 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYaS-0007dk-42; Mon, 26 Feb 2024 10:51:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685448.1066183; Mon, 26 Feb 2024 10:51:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYaS-0007dd-17; Mon, 26 Feb 2024 10:51:56 +0000
Received: by outflank-mailman (input) for mailman id 685448;
 Mon, 26 Feb 2024 10:51:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=YLHk=KD=bombadil.srs.infradead.org=BATV+3075c4dbfe213c24d6b2+7491+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1reYaQ-00073t-Dk
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:51:54 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0d255dad-d495-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 11:51:54 +0100 (CET)
Received: from hch by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red
 Hat Linux)) id 1reYaK-00000000CXP-1Ea2;
 Mon, 26 Feb 2024 10:51:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0d255dad-d495-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Type:MIME-Version
	:References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=8roVx31ufi19zVwomTMUR9WxKwNhqkCKOnoNzyb/69w=; b=P3+Hq4bmD5YRk1RKTqJit0XKn5
	edh2klMjQWEF1w5iz23iQZUvSSJmxb4CSHjtVtmQA8lMnrA3OE9mwCwkZqekhvR7l9wmK2QQirKzK
	zyedNXBLj78QUDEQE9qUuYheD0aSjuqCyv45SGF6zziygXDEi2xq7I8Gip/qKZ+8mPIPxQdbdXsD3
	OuUIwsR4wUfb2Vx9tqZKSvGcY6YfF1MJe3FpPeBK3caVnfapnLfBb9bY94uASzk1+gOmY9YTszZaN
	hBGBfzgqIv2mDJZKE8KiLS9XKG8zQ08KJ/PCXrl36Cbea6Y85IKYOiDZmvjqMGCo3eoCf+CsVRsNo
	Umn4nkiw==;
Date: Mon, 26 Feb 2024 02:51:48 -0800
From: Christoph Hellwig <hch@infradead.org>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org,
	torvalds@linux-foundation.org, brho@google.com, hannes@cmpxchg.org,
	lstoakes@gmail.com, akpm@linux-foundation.org, urezki@gmail.com,
	hch@infradead.org, boris.ostrovsky@oracle.com,
	sstabellini@kernel.org, jgross@suse.com, linux-mm@kvack.org,
	xen-devel@lists.xenproject.org, kernel-team@fb.com
Subject: Re: [PATCH v2 bpf-next 2/3] mm, xen: Separate xen use cases from
 ioremap.
Message-ID: <ZdxtRLw-PxYNkH1u@infradead.org>
References: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
 <20240223235728.13981-3-alexei.starovoitov@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240223235728.13981-3-alexei.starovoitov@gmail.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

On Fri, Feb 23, 2024 at 03:57:27PM -0800, Alexei Starovoitov wrote:
> From: Alexei Starovoitov <ast@kernel.org>
> 
> xen grant table and xenbus ring are not ioremap the way arch specific code is using it,
> so let's add VM_XEN flag to separate them from VM_IOREMAP users.
> xen will not and should not be calling ioremap_page_range() on that range.
> /proc/vmallocinfo will print such region as "xen" instead of "ioremap" as well.

Splitting this out is a good idea, but XEN seems a bit of a too
generit time.  Probably GRANT_TABLE or XEN_GRANT_TABLE if that isn't
too long would be better.  Maybe the Xen maintainers have an idea.

Also more overlong commit message lines here, I'm not going to complain
on the third patch if they show up again :)



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 10:55:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 10:55:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685452.1066193 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYdM-0008Ku-IB; Mon, 26 Feb 2024 10:54:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685452.1066193; Mon, 26 Feb 2024 10:54:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYdM-0008Kn-Et; Mon, 26 Feb 2024 10:54:56 +0000
Received: by outflank-mailman (input) for mailman id 685452;
 Mon, 26 Feb 2024 10:54:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fECz=KD=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1reYdK-0008Kd-E6
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 10:54:54 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2409::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 76f4bb0e-d495-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 11:54:51 +0100 (CET)
Received: from BL0PR0102CA0015.prod.exchangelabs.com (2603:10b6:207:18::28) by
 DS0PR12MB9060.namprd12.prod.outlook.com (2603:10b6:8:c4::12) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7316.34; Mon, 26 Feb 2024 10:54:49 +0000
Received: from BL6PEPF0001AB78.namprd02.prod.outlook.com
 (2603:10b6:207:18:cafe::73) by BL0PR0102CA0015.outlook.office365.com
 (2603:10b6:207:18::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend
 Transport; Mon, 26 Feb 2024 10:54:49 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL6PEPF0001AB78.mail.protection.outlook.com (10.167.242.171) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 26 Feb 2024 10:54:48 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb
 2024 04:54:47 -0600
Received: from [10.65.148.173] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Mon, 26 Feb 2024 04:54:46 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 76f4bb0e-d495-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=faUl4LFR8YwFVwKmcrgDE5FPKpzzi4tKyqhMrdxyi7Fqw1XABY8m6kxshSub0v4rgGQqGohOqKqZWa1H/hjiI5GQDVgISytxiVyRXfKMRDvitXY07+MMZDSFBEK+rYNQJyteVyL5Wg4Hrm/6N3c6+bBRK1TKqktuFCjWvCum9O3ThFJ8P5TpuIWed8/RwMGvm33TParFYzJWUkz0GZ4V52AiC0Uii53XsV2nsY1izwAaG+YB6FLk210zcFjqE6qppsba2Pq9/s56pI0ZPsXqGbMwpswA7MAUVGe+FufxotMOhyIJ/ORfJ1pkFZKjNBZA+N6QX88FV5HfFma7tu9XxQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=OTcY3MjSx4LU/IzmIoFgBTZJC2kZ2k0EUf/dwknBL7I=;
 b=YfpQYa5zZqQru/vswb/gHsFEgkLPrZzWurwsCuIjdNErJAZ7BANrh+8kMvwkQnwgElD8f/h6/+vwWD/3i0fOEc3Scc04rhGaHn1hWdnuo6j6Lpw+hq1uXKrON2hJhTID1so2YlsYZ3No5NdTJtE5/kGt81YAiFroMs9vWv+8oIQd5MQGRtBnxXRzTfieyIhIXGLbesI5ka6rtYDA4LJuXbFc5SiBgS6dEXitOdooAuwepL2fdX+7KWOLLy9c4EU5l+jFbpOnf1tC7Z1L8hwzX8C1J1PvSqVfftdJSzMHaOmYeUnTaVmwu87VI1+Ko7HxgVv8Yqx0s4IKeNAEql38zA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=OTcY3MjSx4LU/IzmIoFgBTZJC2kZ2k0EUf/dwknBL7I=;
 b=XFE/EhJ2lzHZHpA0vfpJE0GOIuk9O1oAaIfX1E3WmUmu1EWJf4m1Ro0IZxXYyyAi35Af66d6TpMzTY7pX8rNEEKEVcUptoS8r0E3xxFgnJjc/98QNP24xCGvHSphI8YsGpq3a/90mLV4fclbPTFbnSqf8q6YSYaHBgzOOdgkpJI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <61789c6a-c306-40fa-b6ef-9b5b5b32e1b7@amd.com>
Date: Mon, 26 Feb 2024 18:54:45 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Set correct per-cpu cpu_core_mask
Content-Language: en-US
To: Michal Orzel <michal.orzel@amd.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
	<julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>
References: <20240226030146.179465-1-xin.wang2@amd.com>
 <de4c6988-7051-4d44-9b44-e850ad4b4cc1@amd.com>
 <DS0PR12MB9448472A793F33228BE6A185A15A2@DS0PR12MB9448.namprd12.prod.outlook.com>
 <e5cad59d-9c3f-4e4c-8b97-5e0fbd4eeda2@amd.com>
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <e5cad59d-9c3f-4e4c-8b97-5e0fbd4eeda2@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB78:EE_|DS0PR12MB9060:EE_
X-MS-Office365-Filtering-Correlation-Id: b98cda1d-2062-4436-dbf0-08dc36b95a27
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	NifDSP/WtPoolJUJUQuya3ys3Q0eBaDFpatsSOgqvTYkhyguew3bq9atQVdOQTyx4nhhkl1zoyDpf9XvP3c8Yxx98aIL4rUkq11RDLeDxzKviPO49E7VP/UMcjS5WIAlbdjrJ2/cgO9ZIFhDxcvQygjKC3zzXrkkDddrdBVz4li/K14pqxAIY5ymCgSoYst1qakPkfAXn+zri8+o1XAQZqUpaVPgGz51GQzK8gvD91p2WCFDonqRACvAsaEyVb458vGYen9bomXRgALkQx/rG+gAza2P6EaP1MOcE2wDiISZLd4vhwEvtXFllVLVPxRP5kLuOtVKF+aTmpPoZQvlwZ86GRc+MyKIrRy7Yn6q7GXWV/jidu1bI8EaSV5/I13/+8jcY58SZQJa25NRXzdAEM35CLPuR1/W1sT/+JqxCaUQUMmbTnup0K2c7ODT2NQhgm1tC2Yu9WuDQ/P5ma+s0V7Cql5+yoZ565sjPqoj4yTQXZuR6PiqdXVXk+ZlC0Fc0ZQ1ObI/UYWhv71Yg4o5l0ddgCVNORxCEXG/i3S/dSrK6M6sRGwla4Rc8sUy5Bt7Fei4vidWk1wSRNcK5ZrWcC8JgjV2SjW7X8qAoJXsAgZ10v8AX8Sk4Pv98Z/+TdEWtt2Uqc9i58v6bFwDhwY85eWsikMhVm201DQcZrppAGBy5h5SDbc8tuCJE7GHtKPpn4z83tt1Rvrt6lO7T4+NYKXP2ziMSDvgDGO1fjewof8RVSgnaRuf47IrA44E9VzD
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 10:54:48.7616
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: b98cda1d-2062-4436-dbf0-08dc36b95a27
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB78.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB9060

Hi Michal,

On 2/26/2024 6:43 PM, Michal Orzel wrote:
> On 26/02/2024 10:54, Wang, Henry wrote:
>>>>   xen/arch/arm/smpboot.c | 6 +++++-
>>>>   1 file changed, 5 insertions(+), 1 deletion(-)
>>>>
>>>> +    /* PE not implemented using a multithreading type approach. */
>>>> +    if ( system_cpuinfo.mpidr.mt == 0 )
>>> Do we need this check? It mt was true, cpu_sibling_mask would be incorrect
>>> anyway (it would still be 1).
>> I added this check for playing safe, because I only want to do the correct thing
>> in this patch and avoid make things worse for MT case. With this patch, non-MT
>> case can be improved and the MT case is remain unchanged.
>>
>> But I agree with you, and I would be more than happy if I can run a MT setup and
>> finish the "else" part with this patch or follow-ups. Do you know maybe qemu can
>> allow me to emulate a MT setup so that I can fix it properly in v2? Thanks!
> A65 is the only Arm CPU with SMT and I'm not aware of Qemu being able to emulate it.
> AFAICT, in Xen on Arm we assume no SMT, hence my question about your check. With or without it,
> some parts would still be incorrect (like cpu_sibling_mask), so what's the point in having a partial check.
> I would keep your solution without the check. Others may have a different opinion though.

You made a point. So I think we can wait a bit more for others to share 
their thoughts and I will change accordingly after.

Just taking a note from the chat that we just had - maybe remove the if 
check and add an ASSERT() of the non-MT would also be a good idea (not 
sure if this change will break some existing HW setups though).

Kind regards,
Henry

> ~Michal



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 11:08:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 11:08:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685458.1066202 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYqA-0002Xx-PK; Mon, 26 Feb 2024 11:08:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685458.1066202; Mon, 26 Feb 2024 11:08:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYqA-0002Xq-MS; Mon, 26 Feb 2024 11:08:10 +0000
Received: by outflank-mailman (input) for mailman id 685458;
 Mon, 26 Feb 2024 11:08:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reYqA-0002Xk-1Y
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 11:08:10 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 51fd710a-d497-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 12:08:07 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-512ab55fde6so3604806e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 03:08:08 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 h11-20020a05620a21cb00b00787bc4a187dsm2316578qka.98.2024.02.26.03.08.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 03:08:06 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51fd710a-d497-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708945687; x=1709550487; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=5zfHZDEu0CL6ienvK3OVvGy8T3k8OJfLniKwloLIs7k=;
        b=mAkI/CMpSbEmYERyq5funrC0jkUXCjf6KUUotsySMW12qj8SHF9HTh7l16KSYVRqDu
         JUDG5/JIfOjqhc5EWAi+AD+YwwRAIhv6yH2j0qzorYzG2LqE/nDXZWCGXgnnUziQq31v
         5TpcZg1UM+szAuATQjdSF3b9EJACSpUDJMk9M=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708945687; x=1709550487;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=5zfHZDEu0CL6ienvK3OVvGy8T3k8OJfLniKwloLIs7k=;
        b=Zc3YV/ZLgix0X52J92WRCJw/blcsqOqScBBhFedIyCuC5ebzGhP2TpzM/pYBacIxiW
         FZRu8w3GQno6ID1REKAUF1MjBViQHYq7xelq1gnjhdUd0Oc7bc11pfe8xA7gV/AEAFK3
         o33IrFaF1asosbMSiN0ud2euCvQxBDa1Gsdxsr1BClumxbif/xEC05AlrjRqi07FN4v3
         oP2FPKNcb6gvRtMYGtcvOGAKCoo66EJSAh1ITARSaYpR7UW1PZ1bB3Gqdm1zfgUwy0P7
         cX2/iAxo9rVAH2PpqagkuIglzGQXtGs36KM5c4xQbwr/6dj7Fmq4MxElJdAs0vkdCNcD
         C5uw==
X-Gm-Message-State: AOJu0Yy2Hbg7HQ+x9W9ldPgvQutD6ZCDkpqyaDYEUx2Z40Y00rLv0hK+
	rdef2UuEyZ2/o+0A+4wo3+hfwWr4Kdf0QZuggh1nRwYH87y6v6t7bnOElseLjg7vWHYfYhHKGKI
	j
X-Google-Smtp-Source: AGHT+IH6mDiKIwMJ4XQDxfLlYBxRYjbuLxD1ZCs8N9zpvusZjApNntSdgYJCkhlxQxTIy1ZG8qxebg==
X-Received: by 2002:a19:f710:0:b0:512:bd51:a0ea with SMTP id z16-20020a19f710000000b00512bd51a0eamr3932997lfe.9.1708945686820;
        Mon, 26 Feb 2024 03:08:06 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v3 0/4] x86/spec: improve command line parsing
Date: Mon, 26 Feb 2024 12:07:44 +0100
Message-ID: <20240226110748.80254-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

A couple of improvements for the spec-ctrl command line parsing.

Thanks, Roger.

Roger Pau Monne (4):
  x86/spec: print the built-in SPECULATIVE_HARDEN_* options
  x86/spec: fix BRANCH_HARDEN option to only be set when build-enabled
  x86/spec: fix INDIRECT_THUNK option to only be set when build-enabled
  x86/spec: do not print thunk option selection if not built-in

 docs/misc/xen-command-line.pandoc | 10 +++----
 xen/arch/x86/spec_ctrl.c          | 46 +++++++++++++++++++++++++------
 2 files changed, 42 insertions(+), 14 deletions(-)

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 11:08:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 11:08:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685459.1066213 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYqC-0002mF-1W; Mon, 26 Feb 2024 11:08:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685459.1066213; Mon, 26 Feb 2024 11:08:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYqB-0002m8-Td; Mon, 26 Feb 2024 11:08:11 +0000
Received: by outflank-mailman (input) for mailman id 685459;
 Mon, 26 Feb 2024 11:08:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reYqB-0002Xk-7v
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 11:08:11 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 530829cc-d497-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 12:08:09 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-512aafb3ca8so2794424e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 03:08:09 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 ma6-20020a0562145b0600b0068fa5e5c245sm2789775qvb.84.2024.02.26.03.08.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 03:08:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 530829cc-d497-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708945689; x=1709550489; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=JwcqS4buxW/Jeygm8zvIbxwG10HttMJtYAWj5OZTXl0=;
        b=ZziypYdtto2mnD9xS444oMHwY/08n7H+rCSLTqnKJ7dlehlZZ6+Rtvr7AVUEngYHRK
         h5jElp6SQiyQqSioSZOST7oIiMO9yAXrbfU42ayN6Isoi+afYEaGqKKvtKMqHrdaNI+Q
         bYDWuPLzGzqI+02WXicdtzBeJfdCWHfJOyamU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708945689; x=1709550489;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=JwcqS4buxW/Jeygm8zvIbxwG10HttMJtYAWj5OZTXl0=;
        b=YTan8m1+VGzodByfiwu1A9slrzg7QmqsRvKpCWH4u+bO+FJTljcYRAyqntIQZilDTI
         Qk3SHMsM8WcIU7EP9L39faLR4QeQJVsYDdy0fc/MJivnZnj5oJSNpYImvVH5xmj62TLY
         U6csZJDJcVAxoNbzlGhB0mvrscvHoVY02ZGvTSx5Key0x/N1nLp2/FIju4U61hsI26a1
         5V+iBTJLlBNCtrkGYRubdjSy7oduodL9I9mSzu3fmrpr8eiEJEDBy7ZHKMItpesmMTaI
         6RyX93zLRJUpagsPVNJ4b6BZNI+qqXhrID6758aYEtPSRsq7ypf8aiRSb4j4aPOc09SO
         Oqiw==
X-Gm-Message-State: AOJu0Yz4Pdv2YQB7YVuJb8DudW9QjMZS+1cszMH8KW2b1AiDOkScCwuE
	wSIRMWJLFVIbU6CHzLMJvIWoaUENJC6sZbCoForJlAAa8zpeH0CMYZloBzkGacDt+7JPJnZtAu/
	A
X-Google-Smtp-Source: AGHT+IEckDKUWpABW+/YnVGbsKMeCnsqI4GOJxHp/u6i3Fib9QKC799WoK3U/L4+dpF6VaORefjJrw==
X-Received: by 2002:a05:6512:3f04:b0:512:d5c7:60d9 with SMTP id y4-20020a0565123f0400b00512d5c760d9mr5437819lfa.3.1708945689250;
        Mon, 26 Feb 2024 03:08:09 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 1/4] x86/spec: print the built-in SPECULATIVE_HARDEN_* options
Date: Mon, 26 Feb 2024 12:07:45 +0100
Message-ID: <20240226110748.80254-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240226110748.80254-1-roger.pau@citrix.com>
References: <20240226110748.80254-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Just like it's done for INDIRECT_THUNK and SHADOW_PAGING.

Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v2:
 - New in this version.
---
 xen/arch/x86/spec_ctrl.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 421fe3f640df..9f5ed8772533 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -466,13 +466,25 @@ static void __init print_details(enum ind_thunk thunk)
            (e21a & cpufeat_mask(X86_FEATURE_SBPB))           ? " SBPB"           : "");
 
     /* Compiled-in support which pertains to mitigations. */
-    if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) || IS_ENABLED(CONFIG_SHADOW_PAGING) )
+    if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) || IS_ENABLED(CONFIG_SHADOW_PAGING) ||
+         IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_ARRAY) ||
+         IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) ||
+         IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_GUEST_ACCESS) )
         printk("  Compiled-in support:"
 #ifdef CONFIG_INDIRECT_THUNK
                " INDIRECT_THUNK"
 #endif
 #ifdef CONFIG_SHADOW_PAGING
                " SHADOW_PAGING"
+#endif
+#ifdef CONFIG_SPECULATIVE_HARDEN_ARRAY
+               " SPECULATIVE_HARDEN_ARRAY"
+#endif
+#ifdef CONFIG_SPECULATIVE_HARDEN_BRANCH
+               " SPECULATIVE_HARDEN_BRANCH"
+#endif
+#ifdef CONFIG_SPECULATIVE_HARDEN_GUEST_ACCESS
+               " SPECULATIVE_HARDEN_GUEST_ACCESS"
 #endif
                "\n");
 
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 11:08:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 11:08:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685460.1066223 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYqK-00036l-8G; Mon, 26 Feb 2024 11:08:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685460.1066223; Mon, 26 Feb 2024 11:08:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYqK-00036c-4L; Mon, 26 Feb 2024 11:08:20 +0000
Received: by outflank-mailman (input) for mailman id 685460;
 Mon, 26 Feb 2024 11:08:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reYqI-00035d-UT
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 11:08:18 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 583520df-d497-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 12:08:18 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-5129cdae3c6so3760009e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 03:08:18 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 fu38-20020a05622a5da600b0042e09c6b82dsm2307513qtb.24.2024.02.26.03.08.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 03:08:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 583520df-d497-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708945697; x=1709550497; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=FaG0bjtk7lVLtBaV15USC2Ji0PDyjB7kZA+T0Bke/cE=;
        b=GjQqKov/KDH+XwcOwCVE/7RKe2qdo3GOuUFaqPVBNajMNRN7nNnkT71ZCH722hGP85
         e5B/sLGa7nGjqtlic5N3wkSNH7ISghBIwWe1vat0IUGDjA0qbAUD9Tntk2tipt6cAyuV
         f8gjviYzs52kHkIYmsVk5FCfjcJEyZyeYqJeE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708945697; x=1709550497;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=FaG0bjtk7lVLtBaV15USC2Ji0PDyjB7kZA+T0Bke/cE=;
        b=QOslzMYtuyC95td2cPoltgsqdPkzPUKnwhccYw8Y4uf/pv+jpj0lZMp5Gd/3RIG03H
         Ljta99vfh9lPjQguqW11XI7HuXRFI9F9a3bjkUFyOih6WEvk7x3acAN0BPF7NrvPlhy5
         Obzs4YXUrd6LWXfs5QEKfNHiAxfsSeG3fq/Ra4OqENl1x7VH3ehwCbNY5Vk/aEf2cMMV
         1sY6VJgO9i5q88Dj4RdRbkNH+qGfDYiqkv+w39X5ZGuWvMZ2eYorY29GhGFf5qPw0gBw
         0eTOng33W47YMMSrMLkqSMnI2ngE1haXD547D+45mwW4aztdg3sPQ1c1Fblf+c+xl1E5
         drhg==
X-Gm-Message-State: AOJu0YzuQIOIZIFaEmbIKQ+KDp9kUC+8w5AtXmWNJXbRNqy+/fBlWG8v
	MgHpu0cS2Ef27jsSIbeohPhU6TcnuvPxshMZnCqEyk5K++m4JNMgA9b0P2cFbG0Qm/qc8PRUkeK
	z
X-Google-Smtp-Source: AGHT+IFPk5TtQYJAUBadwwKI9KWtZ6T+v6n4ml3De9qHFqqBKDwnBmu3W5XhiQJ1B4vi/ShY2A3JTQ==
X-Received: by 2002:ac2:4c49:0:b0:512:ec53:5915 with SMTP id o9-20020ac24c49000000b00512ec535915mr4959845lfk.15.1708945697370;
        Mon, 26 Feb 2024 03:08:17 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 2/4] x86/spec: fix BRANCH_HARDEN option to only be set when build-enabled
Date: Mon, 26 Feb 2024 12:07:46 +0100
Message-ID: <20240226110748.80254-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240226110748.80254-1-roger.pau@citrix.com>
References: <20240226110748.80254-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The current logic to handle the BRANCH_HARDEN option will report it as enabled
even when build-time disabled. Fix this by only allowing the option to be set
when support for it is built into Xen.

Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v2:
 - Use IS_ENABLED() in the parser.

Changes since v1:
 - Use no_config_param().
---
 xen/arch/x86/spec_ctrl.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 9f5ed8772533..5fae80774519 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -50,7 +50,8 @@ static int8_t __initdata opt_psfd = -1;
 int8_t __ro_after_init opt_ibpb_ctxt_switch = -1;
 int8_t __read_mostly opt_eager_fpu = -1;
 int8_t __read_mostly opt_l1d_flush = -1;
-static bool __initdata opt_branch_harden = true;
+static bool __initdata opt_branch_harden =
+    IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH);
 
 bool __initdata bsp_delay_spec_ctrl;
 uint8_t __read_mostly default_xen_spec_ctrl;
@@ -268,7 +269,16 @@ static int __init cf_check parse_spec_ctrl(const char *s)
         else if ( (val = parse_boolean("l1d-flush", s, ss)) >= 0 )
             opt_l1d_flush = val;
         else if ( (val = parse_boolean("branch-harden", s, ss)) >= 0 )
-            opt_branch_harden = val;
+        {
+            if ( IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) )
+                opt_branch_harden = val;
+            else
+            {
+                no_config_param("SPECULATIVE_HARDEN_BRANCH", "spec-ctrl", s,
+                                ss);
+                rc = -EINVAL;
+            }
+        }
         else if ( (val = parse_boolean("srb-lock", s, ss)) >= 0 )
             opt_srb_lock = val;
         else if ( (val = parse_boolean("unpriv-mmio", s, ss)) >= 0 )
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 11:08:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 11:08:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685461.1066232 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYqP-0003SV-Ea; Mon, 26 Feb 2024 11:08:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685461.1066232; Mon, 26 Feb 2024 11:08:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYqP-0003SK-BW; Mon, 26 Feb 2024 11:08:25 +0000
Received: by outflank-mailman (input) for mailman id 685461;
 Mon, 26 Feb 2024 11:08:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reYqO-0002Xk-2U
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 11:08:24 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5a9cc593-d497-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 12:08:22 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-512f7332252so1722541e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 03:08:22 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 ld24-20020a056214419800b0068f455083fbsm2742951qvb.63.2024.02.26.03.08.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 03:08:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a9cc593-d497-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708945701; x=1709550501; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zWSZ24uDxhCSDRWaKYJOr2WehsNuND623TZwFKk5HHU=;
        b=TcMy4VK5VBR+/SP4mO7gEKd0UgW1hminCLShU0J2lu6yzQQoI6OZG8fN24GmoJqEhL
         6x8T+VK5oaghG/XcDPYjcfNfxaqWHKHuhQ51SaG/OyFurmpfHrVT3dFCWi3PTVtnX9Kh
         P2lKxxiV9xfYupovw/Q8BmdegVVpYokt1fYCE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708945701; x=1709550501;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=zWSZ24uDxhCSDRWaKYJOr2WehsNuND623TZwFKk5HHU=;
        b=ShBSMwGYPZOUtIcmBJffzCNfDOx3W2jL6rrqYWhiiEN4MqKawGcaXIrxABwg1jr92h
         3QMrwzpaUHHmVW89t9nPqt/PbjW9QFUqgMz+eJUrj5wdSqzMuXYfFuTE8Ue785gn8GSB
         Wwhg6VJRatzeyfWyjNLMDnpG+M8Ra1/QBKthX9tfN/qxeLac+oAaOMTDfn6G6cHguP5q
         gUlizzkcis40Pr5TaN6+wb0ysjOVTgvS2YAWBHDnpXP0i4AodIEaTIwKCEH0AtpdFU6F
         feRVOb9KgWx/dSBKtRxfDCvCHXNpk/9oux7nF4dtxOrMs7rFDv1mB8CLdvls6EEogAB5
         50mw==
X-Gm-Message-State: AOJu0YyMXC5qYg/p0nZxCIYnCsZAfp5rFiBG3sbzrsPLQ+/s2rTkzf7O
	DAGmVex/jRefkKMbOQ2+CrB2lrQyyitOZL5pRSIW+8RzOxyaJRcNfXcUWlU92ZqFg5fhe2f8icB
	+
X-Google-Smtp-Source: AGHT+IGqLUlMwqmEZc3pqD5QmoarnCv1FcAFbGiWLBw2Z/t+HGZkzTaCjkDsRXX7aggBuRmHYwlyWw==
X-Received: by 2002:a05:6512:3e0c:b0:512:b0a7:2943 with SMTP id i12-20020a0565123e0c00b00512b0a72943mr5349358lfv.5.1708945701487;
        Mon, 26 Feb 2024 03:08:21 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 3/4] x86/spec: fix INDIRECT_THUNK option to only be set when build-enabled
Date: Mon, 26 Feb 2024 12:07:47 +0100
Message-ID: <20240226110748.80254-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240226110748.80254-1-roger.pau@citrix.com>
References: <20240226110748.80254-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Attempt to provide a more helpful error message when the user attempts to set
spec-ctrl=bti-thunk option but the support is build-time disabled.

While there also adjust the command line documentation to mention
CONFIG_INDIRECT_THUNK instead of INDIRECT_THUNK.

Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v2:
 - Adjust documentation.
 - Use IS_ENABLED() instead of #ifdef.

Changes since v1:
 - New in this version.
---
 docs/misc/xen-command-line.pandoc | 10 +++++-----
 xen/arch/x86/spec_ctrl.c          |  7 ++++++-
 2 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index be76be8d5365..02896598df6f 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2417,11 +2417,11 @@ guests to use.
   performance reasons dom0 is unprotected by default.  If it is necessary to
   protect dom0 too, boot with `spec-ctrl=ibpb-entry`.
 
-If Xen was compiled with INDIRECT_THUNK support, `bti-thunk=` can be used to
-select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
-locations.  The default thunk is `retpoline` (generally preferred), with the
-alternatives being `jmp` (a `jmp *%reg` gadget, minimal overhead), and
-`lfence` (an `lfence; jmp *%reg` gadget).
+If Xen was compiled with `CONFIG_INDIRECT_THUNK` support, `bti-thunk=` can be
+used to select which of the thunks gets patched into the
+`__x86_indirect_thunk_%reg` locations.  The default thunk is `retpoline`
+(generally preferred), with the alternatives being `jmp` (a `jmp *%reg` gadget,
+minimal overhead), and `lfence` (an `lfence; jmp *%reg` gadget).
 
 On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
 `ibrs=` option can be used to force or prevent Xen using the feature itself.
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 5fae80774519..ca82b9e41ccd 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -241,7 +241,12 @@ static int __init cf_check parse_spec_ctrl(const char *s)
         {
             s += 10;
 
-            if ( !cmdline_strcmp(s, "retpoline") )
+            if ( !IS_ENABLED(CONFIG_INDIRECT_THUNK) )
+            {
+                no_config_param("INDIRECT_THUNK", "spec-ctrl=bti-thunk", s, ss);
+                rc = -EINVAL;
+            }
+            else if ( !cmdline_strcmp(s, "retpoline") )
                 opt_thunk = THUNK_RETPOLINE;
             else if ( !cmdline_strcmp(s, "lfence") )
                 opt_thunk = THUNK_LFENCE;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 11:08:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 11:08:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685462.1066242 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYqT-0003rF-Mk; Mon, 26 Feb 2024 11:08:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685462.1066242; Mon, 26 Feb 2024 11:08:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reYqT-0003qC-Ja; Mon, 26 Feb 2024 11:08:29 +0000
Received: by outflank-mailman (input) for mailman id 685462;
 Mon, 26 Feb 2024 11:08:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reYqR-0002Xk-Un
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 11:08:27 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5cfb0afe-d497-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 12:08:26 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-512f892500cso908892e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 03:08:26 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 bz8-20020ad44c08000000b0068fa89a5df6sm2764913qvb.124.2024.02.26.03.08.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 03:08:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5cfb0afe-d497-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708945706; x=1709550506; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=I2Df+PSqWP8Z0LteWDeQlgnr9scPhyhGzqXAiWsgtFg=;
        b=LYowHfAL46cSddsX8MFdRfo8+6nFN+ulhfWopCXTsEFjX4bzTxcS5BOJsQ5hAxuNp4
         G89C8XI0gywT8Zko3cQR69ZdMFUG+4wBtT8dJPfT8wyVACakHmA2ZvYsCOnmjXenyzPu
         R1FfkepzdA7BloLzrzJTC+hAWkFYERSy0Wi+k=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708945706; x=1709550506;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=I2Df+PSqWP8Z0LteWDeQlgnr9scPhyhGzqXAiWsgtFg=;
        b=GuszTtn1tQncMx8SCcb8hKcB79ix4u2EB1L3WXJd0GiG5Vprld0k9Ici9NhtEIHS8X
         VroDDQqxNUuys1RKgRL6kDHf5Z/K0PKXu2uyxjCcY1KSecuiOBBVNTvF2uTGpUn5epz1
         p1l2qsuIcgKOLhG94apshhtU2KkjfjxszLwXclpgIcVKxbX9GCojBrczNSJxS5VvQaf8
         h/WulDR6Qxk3mMM2Ils1i51TOJyDoyizcs523ncbNpLXZ8BhaObabm8bFTFnibMJ4D9P
         2kNUP2POhmweK13vZ4Q3r7MKZKwkpD0xtowK0YfXYXDI3LuOJxE3ESkhRd5vQCjfhgAH
         HowA==
X-Gm-Message-State: AOJu0YyHNDVfMS8wf4wiUMGNCzdFJrcxQb1WGzuTyfAZXlvHQkg9PPYh
	T+JBaPfpKbaDHDdnbG/nUBwgLXnnONr+dnSOh1A8fdkwU2aDtU+BXZ8xKQXZPQ5To+BX9+3eMhi
	0
X-Google-Smtp-Source: AGHT+IGo89SrYhBxTQ17xHUpFuNstG50fnkQUUi+pj40NDP//NHqEV1iJBMH9pNUOb5sFudl/c3IzQ==
X-Received: by 2002:a05:6512:2523:b0:512:d8ad:b454 with SMTP id be35-20020a056512252300b00512d8adb454mr4854028lfb.61.1708945705952;
        Mon, 26 Feb 2024 03:08:25 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v3 4/4] x86/spec: do not print thunk option selection if not built-in
Date: Mon, 26 Feb 2024 12:07:48 +0100
Message-ID: <20240226110748.80254-5-roger.pau@citrix.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240226110748.80254-1-roger.pau@citrix.com>
References: <20240226110748.80254-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Now that the thunk built-in enable is printed as part of the "Compiled-in
support:" line, avoid printing anything in "Xen settings:" if the thunk is
disabled at build time.

Note the BTI-Thunk option printing is also adjusted to print a colon in the
same way the other options on the line do.

Requested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v3:
 - New in this version.
---
 xen/arch/x86/spec_ctrl.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index ca82b9e41ccd..e8b0e62adba4 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -504,11 +504,12 @@ static void __init print_details(enum ind_thunk thunk)
                "\n");
 
     /* Settings for Xen's protection, irrespective of guests. */
-    printk("  Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s%s%s, Other:%s%s%s%s%s%s\n",
-           thunk == THUNK_NONE      ? "N/A" :
-           thunk == THUNK_RETPOLINE ? "RETPOLINE" :
-           thunk == THUNK_LFENCE    ? "LFENCE" :
-           thunk == THUNK_JMP       ? "JMP" : "?",
+    printk("  Xen settings: %s%sSPEC_CTRL: %s%s%s%s%s, Other:%s%s%s%s%s%s\n",
+           thunk != THUNK_NONE      ? "BTI-Thunk: " : "",
+           thunk == THUNK_NONE      ? "" :
+           thunk == THUNK_RETPOLINE ? "RETPOLINE, " :
+           thunk == THUNK_LFENCE    ? "LFENCE, " :
+           thunk == THUNK_JMP       ? "JMP, " : "?, ",
            (!boot_cpu_has(X86_FEATURE_IBRSB) &&
             !boot_cpu_has(X86_FEATURE_IBRS))         ? "No" :
            (default_xen_spec_ctrl & SPEC_CTRL_IBRS)  ? "IBRS+" :  "IBRS-",
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 11:18:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 11:18:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685474.1066253 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reZ0T-0007Rh-LG; Mon, 26 Feb 2024 11:18:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685474.1066253; Mon, 26 Feb 2024 11:18:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reZ0T-0007Ra-IT; Mon, 26 Feb 2024 11:18:49 +0000
Received: by outflank-mailman (input) for mailman id 685474;
 Mon, 26 Feb 2024 11:18:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reZ0S-0007RU-Ec
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 11:18:48 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ce8fe5ae-d498-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 12:18:46 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-512fd840142so937877e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 03:18:46 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 u11-20020a05651206cb00b00512f2b3f6afsm799618lff.13.2024.02.26.03.18.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 03:18:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ce8fe5ae-d498-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708946325; x=1709551125; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=6SssqkIv4Nwp63/ViK3ZjJrnf6N+m8jYq7CXWwIuyjA=;
        b=YsDoZioZuFa/JBw64jp0IlYlMHf6ldnKZTYiEmnBpJVRWyS52WmbN3RsctbHU9lH8a
         YxuC9RdUfCmUj09BntU5hcUvTN9hepUN4J8FsoKzpgpXqgnV15BUIpI0Iw3OMEdcOQme
         Tx9gjlf/3JhTRrAR3M16L+kh5e8CfpUJjPz/a1Mzye4zmHNVKeLGKIMbYA3HmzdpHSu9
         z/2BLUUNzxouJSzW5xgpRktaHzhvjuKT9fd2jWTbZm789BForfHnBUMMcqKXN6T66+If
         V+RTCTvuN8N//+TdzOgCSFrsRim0YCKdpEpt2lTVMwLyjoqqNPhyt4f0gbNI/X3mDbm6
         GKdQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708946325; x=1709551125;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6SssqkIv4Nwp63/ViK3ZjJrnf6N+m8jYq7CXWwIuyjA=;
        b=d8BAH5sfRZtxglFOSnuNyhMKNozVbtouT24dSjQ3zxkko0Wkuy6DOBhAVyeJzpxWXL
         NV+0Q6ZIFyU8LJjfUIWEquEidGKvO1hV7qTR/vatHD4SbiTeBiSHaJASXIoTnKgFcC2v
         VXwJ/v+rPR8r78+4Z2VUo/0s5ByFN7AVj3OLe3nVd9gYG8YFfan1idrd97J8GGF3ni3p
         OrTNf7QRMRF2Q1Zg8ySCAWKDyfUh19N+yTnzppkb1VqhAyPp07aTLwoVMry1K23VYFLr
         N2iqyQNsEOEDa5gOYCfStFnOKtPBkKiOfm7x3kd36QYEs0pCifuYcDR4FWzx/BQ2CpmI
         KosA==
X-Forwarded-Encrypted: i=1; AJvYcCXl8UW/STlDXMnQkUCrLnmxj0ANZ6oUrAnABnyroxuuHSgHby2cBScUNRzEErPbDCfSGlYiO18WYu01D6ZrqPaW0pbACUerfiyRHkaD24Y=
X-Gm-Message-State: AOJu0YxHCeqPduGRFHPLyZ3pLo4Ff/PotowT6jIDaCWrErQCxS42OQDJ
	y/tghFy5infbmoLQrWrB3z+uOAOwXLDB2w2ObIiiVMklHBFbJ/J/
X-Google-Smtp-Source: AGHT+IGvbDVQE/wPQVc0nEMy3yGMS/33aDInkcjgw+PsaUMLvfTxNux7al+kC/E0m/N8svmT7C30PA==
X-Received: by 2002:a05:6512:4005:b0:512:ae18:74a1 with SMTP id br5-20020a056512400500b00512ae1874a1mr4827765lfb.11.1708946325185;
        Mon, 26 Feb 2024 03:18:45 -0800 (PST)
Message-ID: <56ae27d003b8763af34864ae56433691685c3661.camel@gmail.com>
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Mon, 26 Feb 2024 12:18:44 +0100
In-Reply-To: <44fd5092-7838-4d28-804b-bbfebfd44886@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
	 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
	 <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
	 <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
	 <1c53b52ee39161a8f59209d28af69fe997479dbc.camel@gmail.com>
	 <3b38fe82-ee0f-4666-93e1-bd78fe69c534@suse.com>
	 <5d2d35fe014094c991363e42c3e2ad9ca2af3938.camel@gmail.com>
	 <44fd5092-7838-4d28-804b-bbfebfd44886@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-26 at 10:45 +0100, Jan Beulich wrote:
> On 23.02.2024 13:23, Oleksii wrote:
> > >=20
> > > > > > As 1- and 2-byte cases are emulated I decided that is not
> > > > > > to
> > > > > > provide
> > > > > > sfx argument for emulation macros as it will not have to
> > > > > > much
> > > > > > affect on
> > > > > > emulated types and just consume more performance on acquire
> > > > > > and
> > > > > > release
> > > > > > version of sc/ld instructions.
> > > > >=20
> > > > > Question is whether the common case (4- and 8-byte accesses)
> > > > > shouldn't
> > > > > be valued higher, with 1- and 2-byte emulation being there
> > > > > just
> > > > > to
> > > > > allow things to not break altogether.
> > > > If I understand you correctly, it would make sense to add the
> > > > 'sfx'
> > > > argument for the 1/2-byte access case, ensuring that all
> > > > options
> > > > are
> > > > available for 1/2-byte access case as well.
> > >=20
> > > That's one of the possibilities. As said, I'm not overly worried
> > > about
> > > the emulated cases. For the initial implementation I'd recommend
> > > going
> > > with what is easiest there, yielding the best possible result for
> > > the
> > > 4- and 8-byte cases. If later it turns out repeated
> > > acquire/release
> > > accesses are a problem in the emulation loop, things can be
> > > changed
> > > to explicit barriers, without touching the 4- and 8-byte cases.
> > I am confused then a little bit if emulated case is not an issue.
> >=20
> > For 4- and 8-byte cases for xchg .aqrl is used, for relaxed and
> > aqcuire
> > version of xchg barries are used.
> >=20
> > The similar is done for cmpxchg.
> >=20
> > If something will be needed to change in emulation loop it won't
> > require to change 4- and 8-byte cases.
>=20
> I'm afraid I don't understand your reply.
IIUC, emulated cases it is implemented correctly in terms of usage
barriers. And it also OK not to use sfx for lr/sc instructions and use
only barriers.

For 4- and 8-byte cases are used sfx + barrier depending on the
specific case ( relaxed, acquire, release, generic xchg/cmpxchg ).
What also looks to me correct. But you suggested to provide the best
possible result for 4- and 8-byte cases.=C2=A0

So I don't understand what the best possible result is as the current
one usage of __{cmp}xchg_generic for each specific case  ( relaxed,
acquire, release, generic xchg/cmpxchg ) looks correct to me:
xchg -> (..., ".aqrl", "", "") just suffix .aqrl suffix without
barriers.
xchg_release -> (..., "", RISCV_RELEASE_BARRIER, "" ) use only release
barrier
xchg_acquire -> (..., "", "", RISCV_ACQUIRE_BARRIER ), only acquire
barrier
xchg_relaxed ->  (..., "", "", "") - no barries, no sfx

The similar for cmpxchg().

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 11:28:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 11:28:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685478.1066263 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reZA6-0001io-Lb; Mon, 26 Feb 2024 11:28:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685478.1066263; Mon, 26 Feb 2024 11:28:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reZA6-0001ih-IC; Mon, 26 Feb 2024 11:28:46 +0000
Received: by outflank-mailman (input) for mailman id 685478;
 Mon, 26 Feb 2024 11:28:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reZA5-0001ib-Gk
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 11:28:45 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 327043c7-d49a-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 12:28:43 +0100 (CET)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-564a53b8133so2121824a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 03:28:43 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w14-20020a056402128e00b00563d03030e8sm2249831edv.55.2024.02.26.03.28.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 03:28:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 327043c7-d49a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708946923; x=1709551723; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=LcAAXH9D0FYnXXIjBHNn8QUKLCPqYFEsIPKcPDxSGrA=;
        b=QWsO0syq46AmxvRKLR51IVqc5Ky6jsTQHjbct0TCsb3rU/LwZ86xMmpbpOEvpo5UkE
         ipv1oHTie7+G6vI/zkGsrS2e9/N9J9kKqIfTNAekor7nG8iIxPXPVMumqZM2MzRlni1u
         GQcGkg0wbtno/rdrNwSzHMNepARS0KKcQz8i0qBBVB88L3RmiGi9lVl4m81QHk8ie4fa
         xS6Iaxc1BmQY7JqIdlbqVIILxWDoou3kNE0sh8y9AzFVSpQyymIvxBFizhlMHM2Jr/78
         1M+QKfvYxiB6PX71cJaSvk6/ggtzMTPKYSxouxgDm0sHXNI6gPw+Fy/xQrDBfqwfCWQf
         7oSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708946923; x=1709551723;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=LcAAXH9D0FYnXXIjBHNn8QUKLCPqYFEsIPKcPDxSGrA=;
        b=EEl4+rfTP4slBWD+EOLjhINqDO2oGCGAYPZXOgQ86B1Zo9S8kE57j9CdcTc4Vk4Wh8
         hxKe1QV0MjZ52waShSGomyqW60a06iuom0SZFwFqMAtp3ngCy4F0vLdrnlGnPoc1ePBj
         hsO3L9g3L0vu/eqRGo0T4ZDFlgn8PMwZV4gm1GFH94nEbIowVz6ch+0TiQrTBo0BvpdO
         qlF6Pvbdy4+n53I+bpuGt4xMwuzKZinfv9sfHI1Up5+p0NTtTGGScAXxftMTv2OdMOis
         BKAwJlkylZxM351bdpdWY/G4kd9WOPfIqmcoNAOZNNhLyOTDADCO0YkT/Qh0iSVUMK+c
         dTsA==
X-Forwarded-Encrypted: i=1; AJvYcCUup7osBBQOB0VTeqrFzVYD4uV5NFI/YEeXdfJvR7CvdeK/P2WxYqh801JXDJ3UwGQKylqYelH+OV3Gbvqi6HY/S5Kr4JdPIOKmxEn2Mag=
X-Gm-Message-State: AOJu0Yz26z0HUI4AlghhZ4gv60YCE0ly0DtwAjSBhDVZhuYWHjbSRAEb
	wwDV1RXVjLnzWTB7cHBMhM6t6EVmCrfih3P10eiQ2/wcGmoR8UYoReJ2RlfgMw==
X-Google-Smtp-Source: AGHT+IHLJLAbt1wQWjYza/EgrHSAgA+wlVcyBkFLny/N+2t1f8xi4gMlVgzuXgcDUDgvzySR2WTzLw==
X-Received: by 2002:a05:6402:1606:b0:565:ffa5:becd with SMTP id f6-20020a056402160600b00565ffa5becdmr1268102edv.37.1708946922791;
        Mon, 26 Feb 2024 03:28:42 -0800 (PST)
Message-ID: <7cab68d1-9bdf-4ea1-b49a-cce1e4af4692@suse.com>
Date: Mon, 26 Feb 2024 12:28:44 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
 <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
 <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
 <1c53b52ee39161a8f59209d28af69fe997479dbc.camel@gmail.com>
 <3b38fe82-ee0f-4666-93e1-bd78fe69c534@suse.com>
 <5d2d35fe014094c991363e42c3e2ad9ca2af3938.camel@gmail.com>
 <44fd5092-7838-4d28-804b-bbfebfd44886@suse.com>
 <56ae27d003b8763af34864ae56433691685c3661.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <56ae27d003b8763af34864ae56433691685c3661.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 12:18, Oleksii wrote:
> On Mon, 2024-02-26 at 10:45 +0100, Jan Beulich wrote:
>> On 23.02.2024 13:23, Oleksii wrote:
>>>>
>>>>>>> As 1- and 2-byte cases are emulated I decided that is not
>>>>>>> to
>>>>>>> provide
>>>>>>> sfx argument for emulation macros as it will not have to
>>>>>>> much
>>>>>>> affect on
>>>>>>> emulated types and just consume more performance on acquire
>>>>>>> and
>>>>>>> release
>>>>>>> version of sc/ld instructions.
>>>>>>
>>>>>> Question is whether the common case (4- and 8-byte accesses)
>>>>>> shouldn't
>>>>>> be valued higher, with 1- and 2-byte emulation being there
>>>>>> just
>>>>>> to
>>>>>> allow things to not break altogether.
>>>>> If I understand you correctly, it would make sense to add the
>>>>> 'sfx'
>>>>> argument for the 1/2-byte access case, ensuring that all
>>>>> options
>>>>> are
>>>>> available for 1/2-byte access case as well.
>>>>
>>>> That's one of the possibilities. As said, I'm not overly worried
>>>> about
>>>> the emulated cases. For the initial implementation I'd recommend
>>>> going
>>>> with what is easiest there, yielding the best possible result for
>>>> the
>>>> 4- and 8-byte cases. If later it turns out repeated
>>>> acquire/release
>>>> accesses are a problem in the emulation loop, things can be
>>>> changed
>>>> to explicit barriers, without touching the 4- and 8-byte cases.
>>> I am confused then a little bit if emulated case is not an issue.
>>>
>>> For 4- and 8-byte cases for xchg .aqrl is used, for relaxed and
>>> aqcuire
>>> version of xchg barries are used.
>>>
>>> The similar is done for cmpxchg.
>>>
>>> If something will be needed to change in emulation loop it won't
>>> require to change 4- and 8-byte cases.
>>
>> I'm afraid I don't understand your reply.
> IIUC, emulated cases it is implemented correctly in terms of usage
> barriers. And it also OK not to use sfx for lr/sc instructions and use
> only barriers.
> 
> For 4- and 8-byte cases are used sfx + barrier depending on the
> specific case ( relaxed, acquire, release, generic xchg/cmpxchg ).
> What also looks to me correct. But you suggested to provide the best
> possible result for 4- and 8-byte cases. 
> 
> So I don't understand what the best possible result is as the current
> one usage of __{cmp}xchg_generic for each specific case  ( relaxed,
> acquire, release, generic xchg/cmpxchg ) looks correct to me:
> xchg -> (..., ".aqrl", "", "") just suffix .aqrl suffix without
> barriers.
> xchg_release -> (..., "", RISCV_RELEASE_BARRIER, "" ) use only release
> barrier
> xchg_acquire -> (..., "", "", RISCV_ACQUIRE_BARRIER ), only acquire
> barrier
> xchg_relaxed ->  (..., "", "", "") - no barries, no sfx

So first: While explicit barriers are technically okay, I don't follow why
you insist on using them when you can achieve the same by suitably tagging
the actual insn doing the exchange. Then second: It's somewhat hard for me
to see the final effect on the emulation paths without you actually having
done the switch. Maybe no special handling is necessary there anymore
then. And as said, it may actually be acceptable for the emulation paths
to "only" be correct, but not be ideal in terms of performance. After all,
if you use the normal 4-byte primitive in there, more (non-explicit)
barriers than needed would occur if the involved loop has to take more
than one iteration. Which could (but imo doesn't need to be) avoided by
using a more relaxed 4-byte primitive there and an explicit barrier
outside of the loop.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 11:33:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 11:33:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685481.1066273 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reZEG-00047d-5j; Mon, 26 Feb 2024 11:33:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685481.1066273; Mon, 26 Feb 2024 11:33:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reZEG-00047W-1q; Mon, 26 Feb 2024 11:33:04 +0000
Received: by outflank-mailman (input) for mailman id 685481;
 Mon, 26 Feb 2024 11:33:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reZEF-00047Q-7K
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 11:33:03 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cc04036d-d49a-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 12:33:00 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-512d19e2cb8so4395793e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 03:33:00 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 f13-20020ac8464d000000b0042e5a324e6dsm2338354qto.76.2024.02.26.03.32.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 03:32:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cc04036d-d49a-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708947180; x=1709551980; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=qWRWxbqkGASpuBzi88vpArLnGKEm0KUJs7ZvZBR4hIs=;
        b=tgemmWm999Yshwv6wXy23OuoY0ervubsT1LX5sXb0Y8DlHKHiJVnoEFrkyL+BflFOL
         ieyxJiM0XG66OWJ/F5ao8oeCvVbfuEL9Szsm16K11F89ACkPwB/tjk3P4jJrCBpiKYMW
         7/WFCJzeB+51ZSMvl5wRiuCgfxxiPlhWp1rUo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708947180; x=1709551980;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=qWRWxbqkGASpuBzi88vpArLnGKEm0KUJs7ZvZBR4hIs=;
        b=hV3EqXRttg46HWMphZRnZenp9vttyZDGZDNgUmkd+EDoiqhERoryYlt6S8eb+t2m6g
         UR1AGQXz7MbQ0HYYP58mBU6dsJvEFyqN8uCDNwDzxKTcVluEzw6tOp/gpeJAIY16jWuT
         chjQOVG8SfemBKaoU8B88rSvjiZQNCgJstVaRT2xmNyAXmgQGCZKhuwBmQk4Zph5zmNJ
         bHL1D2scWxRUsZglCVv0ctFT9+VBNqQyhIZClpomIkEchIgt85+B9vGDuAmEb0oZRliO
         6PUMmRJjhf7NyXsYe1VqGMuxtuKH6dPgHIZ0A2zqyzyBeLXizIj/HCAkq1DkSDFH/4Tr
         d3OQ==
X-Forwarded-Encrypted: i=1; AJvYcCXfbjezsGvAw9RdqfYauWWGC2FXIYdZvmjl62xzBWfiGLWGZwE6UvKH45hPEIwDvDXTzIK4QhZp4AnsyIX7JuFlzwTmAMSL3y7QaxckZvw=
X-Gm-Message-State: AOJu0YyZEHQ1PoaA8Sr+yS81nKuBNrMYAY6Bhl3/Kl1eeOLd4HUjSXGS
	g8c1ex3oj3ev5Vx36jZMCkH1ZAgu6gHjA/cLUzPdFn/qFYD70r5xI9bVvj7sBOg=
X-Google-Smtp-Source: AGHT+IFz6qRvVq98SDkqeNWpzQY5MFpFKG/zvl850ycjpBpYZQLdZCCHpLWusRd29QdXMQsHKIbXVg==
X-Received: by 2002:a05:6512:3b85:b0:512:eeac:2463 with SMTP id g5-20020a0565123b8500b00512eeac2463mr4315162lfv.14.1708947180336;
        Mon, 26 Feb 2024 03:33:00 -0800 (PST)
Date: Mon, 26 Feb 2024 12:32:56 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v6 3/3] xen/livepatch: align functions to ensure minimal
 distance between entry points
Message-ID: <Zdx26DwTvcB2zcbw@macbook>
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-4-roger.pau@citrix.com>
 <670e4a5e-4eec-4a2c-9ed1-fcc164b1d76c@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <670e4a5e-4eec-4a2c-9ed1-fcc164b1d76c@suse.com>

On Tue, Feb 13, 2024 at 04:58:38PM +0100, Jan Beulich wrote:
> On 07.02.2024 15:55, Roger Pau Monne wrote:
> > The minimal function size requirements for an x86 livepatch are either 5 bytes
> > (for jmp) or 9 bytes (for endbr + jmp), and always 4 bytes on Arm.  Ensure that
> > distance between functions entry points is always at least of the minimal
> > required size for livepatch instruction replacement to be successful.
> > 
> > Add an additional align directive to the linker scripts, in order to ensure that
> > the next section placed after the .text.* (per-function sections) is also
> > aligned to the required boundary, so that the distance of the last function
> > entry point with the next symbol is also of minimal size.
> 
> Perhaps "... minimal required size"?

Yes.

> > --- a/xen/common/Kconfig
> > +++ b/xen/common/Kconfig
> > @@ -395,8 +395,11 @@ config CRYPTO
> >  config LIVEPATCH
> >  	bool "Live patching support"
> >  	default X86
> > -	depends on "$(XEN_HAS_BUILD_ID)" = "y"
> > +	depends on "$(XEN_HAS_BUILD_ID)" = "y" && CC_HAS_FUNCTION_ALIGNMENT
> >  	select CC_SPLIT_SECTIONS
> > +	select FUNCTION_ALIGNMENT_16B if XEN_IBT
> > +	select FUNCTION_ALIGNMENT_8B  if X86
> > +	select FUNCTION_ALIGNMENT_4B  if ARM
> 
> This isn't strictly needed, is it? Would be nice to avoid re-selection
> of what the default for an arch is anyway, as otherwise this will start
> looking clumsy when a couple more architectures are added.

My worry was that the default per-arch could change, ie: for example
x86 moving from 16 to 8 and then it would hamper livepatch support if
IBT is also enabled.  I however think it's very unlikely to reduce the
default alignment, and in any case we would hit a build time assert if
that ever happens.

So yes, I'm fine with dropping those.

> Preferably
> with that dropped (or it being clarified why it's still desirable to
> have):
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 11:33:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 11:33:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685483.1066283 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reZF3-0004ga-De; Mon, 26 Feb 2024 11:33:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685483.1066283; Mon, 26 Feb 2024 11:33:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reZF3-0004gT-Aj; Mon, 26 Feb 2024 11:33:53 +0000
Received: by outflank-mailman (input) for mailman id 685483;
 Mon, 26 Feb 2024 11:33:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=CwqN=KD=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reZF2-0004dR-4M
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 11:33:52 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ea4d2945-d49a-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 12:33:51 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-5129cdae3c6so3792506e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 03:33:51 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 qp15-20020a056214598f00b0068f4fae633csm2789794qvb.28.2024.02.26.03.33.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 03:33:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea4d2945-d49a-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708947231; x=1709552031; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=vLx1M/wdHuGW603vLItW5StklqX6rJE6dUZuu4WGVMo=;
        b=mCa75bmbIe3EfH3xSgcNCrq6/aIL2CPFfAr4cEUglMZZFkDV7fnUJo5I/zOb0vTZob
         b3EXVXl7E4laL8S6rYiyoHc3om40xqkyUyeAvHUjT8Z8jDNmExV5CarLFFlpT7E54C4a
         XtqHXc4xSFrivj7/EvKdQ3U+oAbqjEenafVZI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708947231; x=1709552031;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vLx1M/wdHuGW603vLItW5StklqX6rJE6dUZuu4WGVMo=;
        b=eaFOAtOb4cH2xXcO1xQXQNpnpkXE/+oI4lde1uS/1UHB15DMVmM+4BadaYBJXsDvek
         fdIwy1Xfy1W/fx1R2fMA2Z5FQzdTKOplNKHHlMAFQhLTN2j4KeTu1WukFJNXA5bkXDGV
         pLjWM53dPSwAJ4ID4Leo/f1P9zyMTodnwPrrkff8oJQaPfyjtvafK95lgXf+T8uDWQIE
         KyDEM6U5p4I5DV0lNsJeW2R2bL64S4RQQ76kz0ktNzWDM1j4S0so7Aw3/w3Ep6BRxXzd
         6bygUAjRhYM5zIVFlWGJLx+cuk47tCxDdYGUrCB1JaOXY3LJSpYQoCO/blfVCCoXOBZI
         izmg==
X-Forwarded-Encrypted: i=1; AJvYcCXwN7TImZH6t1GnkGJhAX+xnzTbRmpmyzYYxEjbH7rswGvcE6a9+wRWrci616DYZTG34kI4ik/7HbYygH8Ar5X275sLUEORXZFSDn0t3jw=
X-Gm-Message-State: AOJu0YzTpaR0/pDKU3ota37DR5j92fpnR+ywaUbqCMwqTUeOxzgywV6A
	1M7ZR7iEgM4aAJ3+fX/RgTBxV6BvghVEKxuAz/7sKjKAHCZ2iLK2yZmbhbm33sA=
X-Google-Smtp-Source: AGHT+IHHCddmGAQrvWY5Uv6hkoWew+ejXtOiHwSEC7TR8zupr1qeseoz7hlh8VCKKJDHF7MW8Jl68g==
X-Received: by 2002:a05:6512:a95:b0:513:a6:2f4 with SMTP id m21-20020a0565120a9500b0051300a602f4mr988082lfu.13.1708947231256;
        Mon, 26 Feb 2024 03:33:51 -0800 (PST)
Date: Mon, 26 Feb 2024 12:33:42 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v6 1/3] xen: introduce Kconfig function alignment option
Message-ID: <Zdx3FjaKKfF6R_U9@macbook>
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-2-roger.pau@citrix.com>
 <ab71e3e7-c032-454a-b9eb-4a8e9e064b8c@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <ab71e3e7-c032-454a-b9eb-4a8e9e064b8c@suse.com>

On Tue, Feb 13, 2024 at 04:51:13PM +0100, Jan Beulich wrote:
> On 07.02.2024 15:55, Roger Pau Monne wrote:
> > --- a/xen/arch/x86/Kconfig
> > +++ b/xen/arch/x86/Kconfig
> > @@ -29,6 +29,7 @@ config X86
> >  	select HAS_UBSAN
> >  	select HAS_VPCI if HVM
> >  	select NEEDS_LIBELF
> > +	select FUNCTION_ALIGNMENT_16B
> 
> With the insertion here as well as for Arm and PPC obeying alphabetic
> sorting:
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Would you like me to resend with that adjusted?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:27:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:27:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685502.1066297 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rea4Q-0005pr-Gg; Mon, 26 Feb 2024 12:26:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685502.1066297; Mon, 26 Feb 2024 12:26:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rea4Q-0005pk-E6; Mon, 26 Feb 2024 12:26:58 +0000
Received: by outflank-mailman (input) for mailman id 685502;
 Mon, 26 Feb 2024 12:26:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rea4P-0005pe-61
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:26:57 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 541d8175-d4a2-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 13:26:55 +0100 (CET)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a43037e40dcso204453666b.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:26:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 wb14-20020a170907d50e00b00a42fb41e108sm2144600ejc.90.2024.02.26.04.26.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 04:26:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 541d8175-d4a2-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708950415; x=1709555215; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pQaBybbjaIw+7Joa9mTe4+Zcunf/JJ188M54HzP/7h4=;
        b=XJQV6+yAW0VzwuCY/CU6cY0RuCP29kpkhsRHa1zQm4AuKsDCxQZ08/8TRY0URz+ALg
         QmMps8TcvgK8FQPIRTCX4w9RkjgZu4TfsebIyAPhUf0HlW+2irgIZ7qS4Sc/gxoyVBa4
         0B1oatxguysVxFr6eW335FHGaaICT1wIrSgttYNzG82xUrHqU03pAYfyUb+SCSyNIS6g
         5R9rIbKRRy604/cf4U5fEsIIJJ1ElGlN2e03f1OQXtJP5AoksKmz3CIPdCHUilOC/5Fc
         hJivL9FpiZMMu3hqVkA+51vzCRSbwCabtIHxYmND7+bsjZlh/emaFcyvno3cfx6nv01k
         silA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708950415; x=1709555215;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pQaBybbjaIw+7Joa9mTe4+Zcunf/JJ188M54HzP/7h4=;
        b=epefWsavajtQnT2x7RW1Y42Ev7AbG3jhtNcZDtOFd63Tckxq5RWf3S3MtVEY/miXOL
         A0sLekR+tkoSlkeoYi246jOldKfCx/gffW/MNDLJf9gydQXqzRo6+Sq/sQ60m5HHQLR4
         ocVlxlelr+HSBPEwrMK0oBt1pCqpcnT/wL4SnWL8aMeF0Dl2AVqq+gznkUUqtF7jXvgy
         XNnHg01JDAc5x2cuwYrtwtwystIYl48NTD5/IBXui0WAtB37V6vUTj+bn4RWWKB3wajp
         gS3JlPDigGnpe7SahsSed7EnTdnrXDOloRFEu8o9Li52BY+gbOOXnC3mnAT2Ci56Dei2
         NTbg==
X-Forwarded-Encrypted: i=1; AJvYcCV8jyNHTZkYZpIbBP6Mez3knoWwVF20VvbLw7tIc8TSX6LIPpfs80a0A8Md6CGLXypOc7h+uF108rYb24I/RVGsqgZFnyJX1JSgRVIHRM4=
X-Gm-Message-State: AOJu0Yz4+3iRwBKwwT6xZdMopc6IDZnnHDr64F2DMG3945cGjXo4CjGH
	PYIjpqkIPiMplX2drfC7CBfBL7kqurRDRMEDdAbErXOho3wFOGjW1+v+b+FeOQ==
X-Google-Smtp-Source: AGHT+IGpV3gcyTZgScw9lLFlZ4rIr2cQ165uGMYP8sZSDe5CJ53ShYYqRMcuSq3SGqfpGWLBNuUyGA==
X-Received: by 2002:a17:906:4717:b0:a3f:2247:bb59 with SMTP id y23-20020a170906471700b00a3f2247bb59mr4541799ejq.61.1708950415268;
        Mon, 26 Feb 2024 04:26:55 -0800 (PST)
Message-ID: <9bebbe9e-5b02-4f6b-bb9e-9c520099fa84@suse.com>
Date: Mon, 26 Feb 2024 13:26:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 1/3] xen: introduce Kconfig function alignment option
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-2-roger.pau@citrix.com>
 <ab71e3e7-c032-454a-b9eb-4a8e9e064b8c@suse.com> <Zdx3FjaKKfF6R_U9@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zdx3FjaKKfF6R_U9@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 12:33, Roger Pau Monné wrote:
> On Tue, Feb 13, 2024 at 04:51:13PM +0100, Jan Beulich wrote:
>> On 07.02.2024 15:55, Roger Pau Monne wrote:
>>> --- a/xen/arch/x86/Kconfig
>>> +++ b/xen/arch/x86/Kconfig
>>> @@ -29,6 +29,7 @@ config X86
>>>  	select HAS_UBSAN
>>>  	select HAS_VPCI if HVM
>>>  	select NEEDS_LIBELF
>>> +	select FUNCTION_ALIGNMENT_16B
>>
>> With the insertion here as well as for Arm and PPC obeying alphabetic
>> sorting:
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> Would you like me to resend with that adjusted?

I guess it can be taken care of while committing; I've taken note of
this. Sadly there is still at least one missing ack.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:28:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:28:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685504.1066308 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rea5p-0006M8-QK; Mon, 26 Feb 2024 12:28:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685504.1066308; Mon, 26 Feb 2024 12:28:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rea5p-0006M1-NN; Mon, 26 Feb 2024 12:28:25 +0000
Received: by outflank-mailman (input) for mailman id 685504;
 Mon, 26 Feb 2024 12:28:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rea5o-0006Lm-PU; Mon, 26 Feb 2024 12:28:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rea5o-0003d1-NF; Mon, 26 Feb 2024 12:28:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rea5o-0000VB-Bj; Mon, 26 Feb 2024 12:28:24 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rea5o-0004NH-B9; Mon, 26 Feb 2024 12:28:24 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/B1lyMDIvsdN6MNr+bxWEhFQzNQ5QoPdb6ZWOwDy+qg=; b=V+gDZDJifyzlvDjh5UvIvhSKV3
	mrYUjJhTAq32S0M3s87ONVCrdlsjbGDkCn4MF0B4/RU5kQCDFPN2PYquf7YiSm0fJEgI4+vJiZZ+v
	gwyZ39aSV8YR3NiS9ddq1Ler5sdfdbEoiRhrzytleLkf/Av4RdmswcJOnOX36ylitfYc=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184764-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184764: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=8de3afc0b402bc17f65093a53e5870862707a8c7
X-Osstest-Versions-That:
    xen=92babc88f67ed0ef3dc575a8b9534040274678ee
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 26 Feb 2024 12:28:24 +0000

flight 184764 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184764/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  8de3afc0b402bc17f65093a53e5870862707a8c7
baseline version:
 xen                  92babc88f67ed0ef3dc575a8b9534040274678ee

Last test of basis   184745  2024-02-23 23:03:51 Z    2 days
Testing same since   184764  2024-02-26 10:02:04 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   92babc88f6..8de3afc0b4  8de3afc0b402bc17f65093a53e5870862707a8c7 -> smoke


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:36:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:36:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685510.1066318 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaDk-0008Ge-K5; Mon, 26 Feb 2024 12:36:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685510.1066318; Mon, 26 Feb 2024 12:36:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaDk-0008GX-Gm; Mon, 26 Feb 2024 12:36:36 +0000
Received: by outflank-mailman (input) for mailman id 685510;
 Mon, 26 Feb 2024 12:36:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reaDj-0008GR-Ng
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:36:35 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ac88e8e5-d4a3-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 13:36:33 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-5640fef9fa6so3883422a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:36:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 f21-20020a056402195500b005648b000e91sm2311515edz.92.2024.02.26.04.36.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 04:36:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ac88e8e5-d4a3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708950993; x=1709555793; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=zisu0hMuB2wPFsEF8XASd5RZmHjK9uNXa/zjNVFsyZA=;
        b=CRgbLONljPsOgHPMZ6co+8/ozkvBQqc97QLrQbyzE30Oq2Tvcd84S9FEFsMUWl6L8H
         OqoCgrpvulWZ5hIewtBfmkfqv6VV5An8m8tZ+oywbnivtLFehhTSlDCrqnV7XJiVUl1p
         q2D5+wD0PAk5baSSdZbqwGx28VIyE4QHTN2JJtzLwIeuqnSIO4qaXqoJicmYa0+TVeh0
         cZUqYtIseOLZ3XgD/dgDrhvYXOexprTLs2X8Iut2P5amgNlFF6OHvFx7QvRSRFeH1iqe
         luLZsqMq4Px7ey3UwOVeGM++bkEOxIMVniRN16bgudkkDr5lO07BIfA3z76QPkxaq2tD
         WdaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708950993; x=1709555793;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=zisu0hMuB2wPFsEF8XASd5RZmHjK9uNXa/zjNVFsyZA=;
        b=TZ8A53XJEEuV4HzUvA/dZATboE2T78ND75CPvtjOwdC8moqKMIwp83knHCjXNeAhnx
         UhC7cs+i1gi3rla15PZz1OQf0s0K+LHcDx9yl48ihXoAdebrBz5OFN9uQkWX8cXJyWme
         tcHEiekkfyIZZiCagASfCtLgsyP0dXB7EaDf+Fov0dlcwboeyzM8YYAKI6UyvQEKsah7
         iasvL+iAmPQdJ2axvjArwSHmidOb87xNzMtlMHV0WPLYcUICy+2/3gBVtANrB1SFN89G
         dUHSx+tsLFuYDa8c6TV53+lyrs9qYvKOokDFSOey0LnD79ecUNSi/KLgWLCJDjnG799d
         plcg==
X-Forwarded-Encrypted: i=1; AJvYcCXkdqAzmWE+r7wttzQsIsD9dtSENwSkGcVf2sA12iQpy4pUT21KNEuEkMJLwjcxkwhbqhCCIaKqhnKXXW5g+6V8xvPbvdZTPli8ycHIUzA=
X-Gm-Message-State: AOJu0Yxn7505KlFmlBZely7oR60OjjwW9yKgm2Uj73y8vBTWZfUa4Pgv
	s8CFCixLp9MQDjvZZI7/gaCw8FAFLi76FFtfk+8+DoSXKsV5BiQLCCqOXvYBRw==
X-Google-Smtp-Source: AGHT+IFsrWDARmjctDxFfbnwHdMd3z1O7W8P6RbJM63IhabM4sntY2fNA/vNJrr5tUQ6McvNJHI1yg==
X-Received: by 2002:aa7:d912:0:b0:565:6c31:9a47 with SMTP id a18-20020aa7d912000000b005656c319a47mr5382340edr.39.1708950993157;
        Mon, 26 Feb 2024 04:36:33 -0800 (PST)
Message-ID: <116a9e07-4a6a-4170-bf0f-3a7ccbd68f0e@suse.com>
Date: Mon, 26 Feb 2024 13:36:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 3/3] xen/livepatch: align functions to ensure minimal
 distance between entry points
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-4-roger.pau@citrix.com>
 <670e4a5e-4eec-4a2c-9ed1-fcc164b1d76c@suse.com> <Zdx26DwTvcB2zcbw@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zdx26DwTvcB2zcbw@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 12:32, Roger Pau Monné wrote:
> On Tue, Feb 13, 2024 at 04:58:38PM +0100, Jan Beulich wrote:
>> On 07.02.2024 15:55, Roger Pau Monne wrote:
>>> The minimal function size requirements for an x86 livepatch are either 5 bytes
>>> (for jmp) or 9 bytes (for endbr + jmp), and always 4 bytes on Arm.  Ensure that
>>> distance between functions entry points is always at least of the minimal
>>> required size for livepatch instruction replacement to be successful.
>>>
>>> Add an additional align directive to the linker scripts, in order to ensure that
>>> the next section placed after the .text.* (per-function sections) is also
>>> aligned to the required boundary, so that the distance of the last function
>>> entry point with the next symbol is also of minimal size.
>>
>> Perhaps "... minimal required size"?
> 
> Yes.
> 
>>> --- a/xen/common/Kconfig
>>> +++ b/xen/common/Kconfig
>>> @@ -395,8 +395,11 @@ config CRYPTO
>>>  config LIVEPATCH
>>>  	bool "Live patching support"
>>>  	default X86
>>> -	depends on "$(XEN_HAS_BUILD_ID)" = "y"
>>> +	depends on "$(XEN_HAS_BUILD_ID)" = "y" && CC_HAS_FUNCTION_ALIGNMENT
>>>  	select CC_SPLIT_SECTIONS
>>> +	select FUNCTION_ALIGNMENT_16B if XEN_IBT
>>> +	select FUNCTION_ALIGNMENT_8B  if X86
>>> +	select FUNCTION_ALIGNMENT_4B  if ARM
>>
>> This isn't strictly needed, is it? Would be nice to avoid re-selection
>> of what the default for an arch is anyway, as otherwise this will start
>> looking clumsy when a couple more architectures are added.
> 
> My worry was that the default per-arch could change, ie: for example
> x86 moving from 16 to 8 and then it would hamper livepatch support if
> IBT is also enabled.  I however think it's very unlikely to reduce the
> default alignment, and in any case we would hit a build time assert if
> that ever happens.
> 
> So yes, I'm fine with dropping those.

Oh, no - not "those", only "that", i.e. only the last (Arm) one.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:39:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:39:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685512.1066328 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaGv-0000gw-2J; Mon, 26 Feb 2024 12:39:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685512.1066328; Mon, 26 Feb 2024 12:39:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaGu-0000gp-Vk; Mon, 26 Feb 2024 12:39:52 +0000
Received: by outflank-mailman (input) for mailman id 685512;
 Mon, 26 Feb 2024 12:39:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reaGt-0000fT-Uh
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:39:51 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 21e21c97-d4a4-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 13:39:50 +0100 (CET)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-565c6cf4819so2158333a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:39:50 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 qo7-20020a170907874700b00a4319de07c6sm1611214ejc.127.2024.02.26.04.39.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 04:39:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21e21c97-d4a4-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708951190; x=1709555990; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sU8B5Gxqw5IaqT1tDZu3OilAULQuRRdjJ1eRYjBRuEM=;
        b=bm7VYIXwiufEBlAOHYGq0Gz6QHbeRp/NW1MA9kc2yfWW/w24omxrTrYajj6KQgOq/F
         tE8kbUS0qhXEFoXEteh55FPeMJEyXvalv3V4k2Dh3HtJzTMPGqLT6nnAUkJIvDTP1LoB
         9YMV/vfuEH3XejDCIcs5qh14aDW8mn9NbY6shy7ycAWwQrwAA/AHJclUtrFkER1LzQx2
         lhVwBdfeINV5f9N7IiAmCzmKGhB3mjiQvUQzd/NIA253X1cgvo2yLoEf/onRIudc3PE6
         hNcgxZpstep4uYGJOKK6zJhRRgS/vy+i3RyPQgHkZ8m9+yTdjHNZ7GNKAV7wTh5tGGFf
         5Bnw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708951190; x=1709555990;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sU8B5Gxqw5IaqT1tDZu3OilAULQuRRdjJ1eRYjBRuEM=;
        b=NXmeAnyh/OSuRXLVXqjAoh7O6HtkaL8M5BGY8l07RImOyZ5/e9wThXSNJcm7Elq8W3
         f/aqktSkJgVS9GJBCtHw2MHfOYSdXR7iMoOgDSQBa3bB9+drG4Ztw5kPniDA2e7Qnvdd
         ue+14wUBCvHgJ9PpNWhtrCRIhvk+V2WfqUO1bwH3kTBLyw8Jo1ZK85IU9S0zE7uL4YLp
         WkBtUfvJI8U1Nocs4hA/cWD9I0ogzhx/XvxvWM6mysIbwrPMpIjXZLJEwrjGeFkB5oq3
         GencPV0/p9ZGoXvv9WGY+TDMIYOwP9kc6rbmjs5Sq6fhTsxGEem/yVhFl3uKPlz5JRV0
         WxAw==
X-Forwarded-Encrypted: i=1; AJvYcCXEdL7t5P24oDKgxPVky2J3VfhQqrNyTCZdjNRjgAPwCYcCDNLCEF91jAOEQdPsl0sC37QtxfkmpyCLL5OprZgEERPJOdU0jjuKoQw3yfM=
X-Gm-Message-State: AOJu0Yy9jdoucHrwB9rCwZGQR5MOs+ntPHVDd862osLV7V87jLvhua5w
	NXID2GXslAtjaxLjzCXklBntXTN4Gu6ADldlX5BegnhhsD4o3Ild70tP/k/1vQ==
X-Google-Smtp-Source: AGHT+IGviqNAWFMRUy56MLNnBWgnxb+aNO+7/z2Gk30tpKTZPOD5XvM7He51fmZJWW5sOzTy8R8uEA==
X-Received: by 2002:a17:906:e298:b0:a42:f0b6:c2f1 with SMTP id gg24-20020a170906e29800b00a42f0b6c2f1mr5408066ejb.28.1708951190005;
        Mon, 26 Feb 2024 04:39:50 -0800 (PST)
Message-ID: <85fa5e55-27c5-495d-bd7b-857260b1de20@suse.com>
Date: Mon, 26 Feb 2024 13:39:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/4] x86/spec: print the built-in SPECULATIVE_HARDEN_*
 options
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240226110748.80254-1-roger.pau@citrix.com>
 <20240226110748.80254-2-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226110748.80254-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 12:07, Roger Pau Monne wrote:
> Just like it's done for INDIRECT_THUNK and SHADOW_PAGING.
> 
> Reported-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

In principle
Reviewed-by: Jan Beulich <jbeulich@suse.com>
but ...

> --- a/xen/arch/x86/spec_ctrl.c
> +++ b/xen/arch/x86/spec_ctrl.c
> @@ -466,13 +466,25 @@ static void __init print_details(enum ind_thunk thunk)
>             (e21a & cpufeat_mask(X86_FEATURE_SBPB))           ? " SBPB"           : "");
>  
>      /* Compiled-in support which pertains to mitigations. */
> -    if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) || IS_ENABLED(CONFIG_SHADOW_PAGING) )
> +    if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) || IS_ENABLED(CONFIG_SHADOW_PAGING) ||
> +         IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_ARRAY) ||
> +         IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) ||
> +         IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_GUEST_ACCESS) )
>          printk("  Compiled-in support:"
>  #ifdef CONFIG_INDIRECT_THUNK
>                 " INDIRECT_THUNK"
>  #endif
>  #ifdef CONFIG_SHADOW_PAGING
>                 " SHADOW_PAGING"
> +#endif
> +#ifdef CONFIG_SPECULATIVE_HARDEN_ARRAY
> +               " SPECULATIVE_HARDEN_ARRAY"
> +#endif
> +#ifdef CONFIG_SPECULATIVE_HARDEN_BRANCH
> +               " SPECULATIVE_HARDEN_BRANCH"
> +#endif
> +#ifdef CONFIG_SPECULATIVE_HARDEN_GUEST_ACCESS
> +               " SPECULATIVE_HARDEN_GUEST_ACCESS"
>  #endif

... I'd like to suggest to drop the SPECULATIVE_ from the string literals.
They're relevant in the Kconfig identifiers, but they're imo redundant in
the context of these log messages. (Happy to adjust while committing, if
need be.)

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:42:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:42:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685516.1066337 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaJc-00029D-G5; Mon, 26 Feb 2024 12:42:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685516.1066337; Mon, 26 Feb 2024 12:42:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaJc-000296-DT; Mon, 26 Feb 2024 12:42:40 +0000
Received: by outflank-mailman (input) for mailman id 685516;
 Mon, 26 Feb 2024 12:42:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reaJa-000290-FD
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:42:38 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 85992e24-d4a4-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 13:42:37 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a3f893ad5f4so430379766b.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:42:37 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 x8-20020a170906710800b00a4138c3f065sm2386071ejj.56.2024.02.26.04.42.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 04:42:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85992e24-d4a4-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708951357; x=1709556157; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=zHplMMXsHba0zwdPMLcvoMdIb4TN6dCnVRyffeOQgUE=;
        b=Js9asqG7LPdtlHhS1/oflT/hgA6+96v4o96WQuDQag5nqOAKtOaiVi5pNVlgNKr5S9
         Z2SACrUPm33VYsDg01ROrE7ivAcK978S9NRmL51QlqEr3Gr9F0pDecz18y/Oa0OzbPTE
         sngD6hl+0GW62dYneaQl4Zkt1+A++WhT1+R2TrqS1RzcpNVKxjiL00pvWqoRvFo5o6lX
         kTghtx7RyAFDZTyoHIkZJ3FDv+MeD4P6ZmhDdv+MyFHRr3KOoHVVcUFAKo1hfY8xcdMr
         sMoz5YFoqhVyYtt9k/izEcvuwUPhHNBGudR+VD1xAjteevcgSJBXPtPqXPcFXop7D6ga
         7Dxw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708951357; x=1709556157;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zHplMMXsHba0zwdPMLcvoMdIb4TN6dCnVRyffeOQgUE=;
        b=sXrv/UhK2bJcSXbHhoZOAE4t0i+NljxjFwzXYNnltGfvveBZfBECD9kigj7/RVQk9I
         zvYnmC5XuvHW0uv1i1HFjUfmGrJyc42tgxP/AsqzVsMBDR4GjLpBrLf90SpUTG3/mckY
         BZpbxExOMFn1fsowTB/e0sjEZACROyleEXEGevM5e21OuY9O8I7zGrMZ9M2WcLPprDfI
         X7tsfb51VjR1p1AQuJuk8RkumzDpD5UfJDOjcE5Uo8kUjQC0ZN1u5EYxMj/pK4WCJ7PN
         TuQ7FSHyjjSVV3hRkK40v7n/SPNZ1NJfrTzgloOrxMwm7yWpH6GGVsQT0Py5kslrifLA
         4SUg==
X-Forwarded-Encrypted: i=1; AJvYcCXEXfVFBKkdqHk88EvAF6GOv5q++xw14XGyPKPjwOsfRzpy2YjICkI6i4zzZ7DwRpt1Td5Hk8cfNXjo1lMA1Rp2dFaNCzXvmi3ej/V7GLk=
X-Gm-Message-State: AOJu0YxyY54nyRD0JUYv7T6vT+z8lSoi38Vw9VSO0G8QlpfPHa6ImEyw
	hAhBoMlHfdda+FS/4IJKg/mWJCcRiMBpLXMkrxOYFH9nVEp8rYXtG/X2g3DqwQ==
X-Google-Smtp-Source: AGHT+IFQlm8me7Nta3WJCYKA2l+2lxnWL7UbR1sJvAdtEi7EVeDvXahYBBJYXPZTccw2wTw5DVd7+A==
X-Received: by 2002:a17:906:7c44:b0:a3e:90ec:2287 with SMTP id g4-20020a1709067c4400b00a3e90ec2287mr3965558ejp.26.1708951357283;
        Mon, 26 Feb 2024 04:42:37 -0800 (PST)
Message-ID: <767fbe0c-2c71-4f8b-a405-ae90a33e991b@suse.com>
Date: Mon, 26 Feb 2024 13:42:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 2/4] x86/spec: fix BRANCH_HARDEN option to only be set
 when build-enabled
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240226110748.80254-1-roger.pau@citrix.com>
 <20240226110748.80254-3-roger.pau@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226110748.80254-3-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 12:07, Roger Pau Monne wrote:
> The current logic to handle the BRANCH_HARDEN option will report it as enabled
> even when build-time disabled. Fix this by only allowing the option to be set
> when support for it is built into Xen.
> 
> Fixes: 2d6f36daa086 ('x86/nospec: Introduce CONFIG_SPECULATIVE_HARDEN_BRANCH')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:50:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:50:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685519.1066348 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaRY-00044Q-Ep; Mon, 26 Feb 2024 12:50:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685519.1066348; Mon, 26 Feb 2024 12:50:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaRY-00044J-B0; Mon, 26 Feb 2024 12:50:52 +0000
Received: by outflank-mailman (input) for mailman id 685519;
 Mon, 26 Feb 2024 12:50:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reaRX-00044D-Ug
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:50:51 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a9e4cfda-d4a5-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 13:50:48 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-5101cd91017so4533679e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:50:48 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 un6-20020a170907cb8600b00a4386852da5sm178964ejc.83.2024.02.26.04.50.47
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 04:50:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a9e4cfda-d4a5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708951848; x=1709556648; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=mBTas/3RpNzz+26rSLwd9G+9VGB7QiBP7Uhl8rFm+kI=;
        b=VKtialhinaMzqOuXI6bx2yaMS9i+pq1vu1S25WhqkEnkBCebmjPBNah3aYf6HpiR76
         ncjyYWbkQZcI87fq7BrsC2g+p0CI8I4Egnf9Rd01jt3U/ECmNERQjv5hC781gf+b4DTZ
         sD7LFzfb3NbTvR7wzsoZq/zuaWtqKaRhABZgz9eDiB7mBwn8L/qmNVAC3kDH1yf6pl2g
         atKACOAspN0mvcAFUWf6nt0sKruuymeF1Dhos6iW+0obPmmKiawYsH3ok6LpD9pGQ14E
         9KyYjunGXpsxv+qLh6lB/BZrSC4rY8CCx9rvju8ja9ZHhdWX+zLUVMajph+S3o57O1Sm
         mYqQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708951848; x=1709556648;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=mBTas/3RpNzz+26rSLwd9G+9VGB7QiBP7Uhl8rFm+kI=;
        b=N9mGdCs44VqTP7PtKnpTq4Jaiu0QRKwerlT31wfcCPhSctz05gAnL0PnG+cCQ39v/E
         ayS2U2X5PN/LAlHc51vgZRsa1/w4UQyGCHIN9BD7trjOON5iTXUO7MIBGslAee3rU1pN
         1nHelK//DiklJhj7fwMtsSoSQ7GZMHASRu7NM0YzRhxiN/3JA/uNTCElIg1qL4YBp+WB
         Ao1DT7aPgCm85ePLu/U/N4sUA/8sgIye5HJURW+OSRgCZYrVXNIaFHczg0dznckUPGZ0
         kjFlLfdYmKWVcmFAn2JjmdcO9qIH807/aSCTQwif6SLfiUdmzLDgxBzK9wuMso+rnnX9
         MOTw==
X-Forwarded-Encrypted: i=1; AJvYcCXdAAXzTGtvarbW4DZksj5yLNXHPx646C+CiyzQV0F9YJsEUQ4bOnaT/NHxNagW+Hk5e9GggK8WrSYx3KDQJbuAX+SW/dZKM+rQe3mia0A=
X-Gm-Message-State: AOJu0Yz4G72TwyivXUddLIvyKZ5o3ie8U821c2A251cZHGfNUayw/YhZ
	K2Tv1+NqoI38a1iZhNhssOkz05qivVnEDO0qqbahbL6XpfnXTq4I1aDWi5K0Rg==
X-Google-Smtp-Source: AGHT+IEbHhHtqh4/2TiRDB3MhcjqvVjDwBOa68IamUt1xVsH/8vQwazqYXB0+u3aCoYWxwkjS6p12Q==
X-Received: by 2002:a05:6512:a91:b0:512:ed78:a840 with SMTP id m17-20020a0565120a9100b00512ed78a840mr5425944lfu.61.1708951847802;
        Mon, 26 Feb 2024 04:50:47 -0800 (PST)
Message-ID: <81e32609-37cd-4c2d-ae27-6caafc7dab33@suse.com>
Date: Mon, 26 Feb 2024 13:50:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 3/4] x86/spec: fix INDIRECT_THUNK option to only be set
 when build-enabled
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240226110748.80254-1-roger.pau@citrix.com>
 <20240226110748.80254-4-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226110748.80254-4-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 12:07, Roger Pau Monne wrote:
> Attempt to provide a more helpful error message when the user attempts to set
> spec-ctrl=bti-thunk option but the support is build-time disabled.
> 
> While there also adjust the command line documentation to mention
> CONFIG_INDIRECT_THUNK instead of INDIRECT_THUNK.
> 
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with one minor remark:

> --- a/xen/arch/x86/spec_ctrl.c
> +++ b/xen/arch/x86/spec_ctrl.c
> @@ -241,7 +241,12 @@ static int __init cf_check parse_spec_ctrl(const char *s)
>          {
>              s += 10;
>  
> -            if ( !cmdline_strcmp(s, "retpoline") )
> +            if ( !IS_ENABLED(CONFIG_INDIRECT_THUNK) )
> +            {
> +                no_config_param("INDIRECT_THUNK", "spec-ctrl=bti-thunk", s, ss);
> +                rc = -EINVAL;
> +            }
> +            else if ( !cmdline_strcmp(s, "retpoline") )
>                  opt_thunk = THUNK_RETPOLINE;
>              else if ( !cmdline_strcmp(s, "lfence") )
>                  opt_thunk = THUNK_LFENCE;

How about

            if ( !IS_ENABLED(CONFIG_INDIRECT_THUNK) )
            {
                no_config_param("INDIRECT_THUNK", "spec-ctrl", s - 10, ss);
                rc = -EINVAL;
            }
            else if ( !cmdline_strcmp(s, "retpoline") )

or (likely less liked by you and Andrew) "s += 10;" dropped and then

            if ( !IS_ENABLED(CONFIG_INDIRECT_THUNK) )
            {
                no_config_param("INDIRECT_THUNK", "spec-ctrl", s, ss);
                rc = -EINVAL;
            }
            else if ( !cmdline_strcmp(s += 10, "retpoline") )

conserving a little on string literal space (sadly, despite the function
being __init, string literals remain post-init due to living in .rodata)?

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:54:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:54:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685523.1066358 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaVW-0004i0-Tz; Mon, 26 Feb 2024 12:54:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685523.1066358; Mon, 26 Feb 2024 12:54:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaVW-0004ht-RL; Mon, 26 Feb 2024 12:54:58 +0000
Received: by outflank-mailman (input) for mailman id 685523;
 Mon, 26 Feb 2024 12:54:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reaVV-0004hn-SF
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:54:57 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3d7e71a7-d4a6-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 13:54:55 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a3ed9cae56fso478361666b.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:54:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 x8-20020a170906710800b00a4138c3f065sm2396006ejj.56.2024.02.26.04.54.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 04:54:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d7e71a7-d4a6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708952095; x=1709556895; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ATeifoqIO8fF4lVNDNfylY1qFqEXX2S7rnGO0IvD8bw=;
        b=IhSFniePkpmyC7+13yLyPg1gqFYZ9YGZoiuL7frcpnh43IzTnvzjFO00krIc9ZmLfu
         F/PGaetN/BYCDVPLUgeyhAJiobQxz4xJmW3su7c9HB0j3CJ7LjQKhlzgP7cA/QI8GLDo
         UUVFOQH0d7ur1pEr28pX8tiiWMZMcJEBstRixrNhpKcs22oaSdGCKEq5CJjjgb5wz7hA
         Ir9SDPpfKFTmoRs5Dynx4O5Km8v1GRUGkalcyT/BP8Wn3ZM/rKIuVfYVHFOAOHxSbTtD
         iCvPAzUBC3K1kcNmhfe7BdyJmOXd8rkfCHND6MEL8seYWS1KHA4njdqzUE30Zn1igsyE
         n7Dw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708952095; x=1709556895;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ATeifoqIO8fF4lVNDNfylY1qFqEXX2S7rnGO0IvD8bw=;
        b=P0alUDYcIuSGjD7gtXWLsf9xGqA65vlAZR89oTmkwcszDbpowhGGVPwJXeBzhDFBot
         wmcw2sM84g4CMhjq3YJXCBxFpS8j8odrzXnmMitROzb07jLp6tyNcDiQWUCNfmoSVYjE
         eU772w86mGKI0hWipBJwv+/irBm/fH8PNiQ1AZwH4ix6kk5s246u327p2RCR6C5FDrJu
         RO45efkCNKkKtBPcUMTxpgmIHa1JZ/P14VuywsQZMoeFfE1XveNQvL0Iq/uLmS94FA9y
         hT5W0DK6bFiywySm4h9G7uuCLy1TzAwZneXS6Me2vVQM1YEfa2wCtT/44xg//Fvggdvk
         kAig==
X-Forwarded-Encrypted: i=1; AJvYcCU5KTDr0ETBncopjsewnVYk4cv86RLMKAj0gJRjKr+o3TGvPugRR2iqKQofN0VAuBTqaVrn/MPw0xTGSXhwb+oI1P4+ck3PiTXZGMQDeWs=
X-Gm-Message-State: AOJu0Yy5Gth8zVgdeMsUMpRIg6ymLbk2PJ7hDOO/28k36u/R5GnGFrxa
	vs/R6JcVN/FctiQrwOKRQAGyzlgBj/eqGsNYxB5+xISq8oJVPU3izRwa1mxQNw==
X-Google-Smtp-Source: AGHT+IGJIBBK0e+0epLPulSe2wNB/YhES14xkro/EGMVbfHkrunBhrRxlt5MoRcIaeL2/rke6b6pvA==
X-Received: by 2002:a17:906:490c:b0:a43:825c:7bde with SMTP id b12-20020a170906490c00b00a43825c7bdemr681364ejq.15.1708952095473;
        Mon, 26 Feb 2024 04:54:55 -0800 (PST)
Message-ID: <30e1bfea-d76b-4e3d-b148-f25d1c811f8d@suse.com>
Date: Mon, 26 Feb 2024 13:54:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 4/4] x86/spec: do not print thunk option selection if
 not built-in
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240226110748.80254-1-roger.pau@citrix.com>
 <20240226110748.80254-5-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226110748.80254-5-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 12:07, Roger Pau Monne wrote:
> Now that the thunk built-in enable is printed as part of the "Compiled-in
> support:" line, avoid printing anything in "Xen settings:" if the thunk is
> disabled at build time.

Why "Now that ..."? It's other logging the earlier patch adds there.

> Note the BTI-Thunk option printing is also adjusted to print a colon in the
> same way the other options on the line do.
> 
> Requested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

With either a clarification of what's meant or e.g. s/Now that/Since/
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:55:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:55:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685524.1066367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaVe-0004yy-4S; Mon, 26 Feb 2024 12:55:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685524.1066367; Mon, 26 Feb 2024 12:55:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaVe-0004yr-1o; Mon, 26 Feb 2024 12:55:06 +0000
Received: by outflank-mailman (input) for mailman id 685524;
 Mon, 26 Feb 2024 12:55:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1reaVc-0004yB-Ou
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:55:04 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 424aa120-d4a6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 13:55:03 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a3ee69976c9so380968166b.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:55:03 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ll9-20020a170907190900b00a42e4b5ab00sm2379645ejc.41.2024.02.26.04.55.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 04:55:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 424aa120-d4a6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708952103; x=1709556903; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=5w2NVQzEy0Ft7WjMPLYhygiLJCz4wsG8joT9atA1+T0=;
        b=hO9r8gen6CzVyADc0NrIgyyZkflG5+7qFTw+sHz425Qjga04nLlx6ulcglBkMGNXgs
         IRma+fEJ1U9h0he1Tb6EMnxuBVBBODpMHgT6bR0IPcR3bCgWcoUPvQQ8K9uADzfcJ24Z
         4yswxwlUtf07QHXz9KosETFyMKllIotwh+ATY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708952103; x=1709556903;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=5w2NVQzEy0Ft7WjMPLYhygiLJCz4wsG8joT9atA1+T0=;
        b=TEZQdRKjgssPu+JimXKQk65koO3BoH1aDA/b9UhHjQ8Ct8msciNcC7JKAg/59O53h1
         iGU0Idn6iAdYzTbTOshtalB15R5b7xr7rVD26KYDPBOEpkO6oo7YukTcog4xj7L10V3y
         RFDpx6zcYOJJnlV9fy/zvhaR8xa5ILSzCUhdF3EDgAn/7bnAk7ixTKUmmSyLkFnzL/gK
         63AlBChtvEx2sGBXq6ivdwQwQ8zIA+Aqkqri6s2Pzv5DmDF6oQlTcNer+kxkgd+Y4SV0
         R95wPSX0hJYwyCN6TG7LXNAZwxJpgxkZGc7NKm0oTB5GPUQXA+aI6KuzaJweWFRQ85xN
         BUsA==
X-Gm-Message-State: AOJu0YyDZlyW1qKqt9C1zJK+3t55iik81gac45kj2S8UFArVxdKCFd5c
	hGbptDvuC/Ivrjsfam3Pc1Ihgy67jsPLI6ceHaV0X1g6/4l992FzvcpjjUcyl2ZHRBww0nPDQF2
	c
X-Google-Smtp-Source: AGHT+IGeaEd9YSZ7F9GJ7i9W2ZNEQs6xzaQ1pXBI/vZpIn9E1CaFfrRiUdzlb2w6xdIdhxRn+1+Jtg==
X-Received: by 2002:a17:906:b115:b0:a3f:5b9b:a17b with SMTP id u21-20020a170906b11500b00a3f5b9ba17bmr4178004ejy.53.1708952103233;
        Mon, 26 Feb 2024 04:55:03 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 1/3] x86/entry: Adjustments to "reduce assembly code size of entry points"
Date: Mon, 26 Feb 2024 12:54:59 +0000
Message-Id: <20240226125501.1233599-2-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
References: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Some retroactive review, for if I'd got to the patch in time.

 * The new ASM-friendly BUILD_BUG_ON() should be in a header file.
 * entry_int82() wants the movl->movb treatment too.

Fixes: c144b9e32427 ("x86: Reduce assembly code size of entry points")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

Fixes just to link the two patches.
---
 xen/arch/x86/include/asm/asm_defns.h | 12 ++++++++++++
 xen/arch/x86/x86_64/compat/entry.S   |  2 +-
 xen/arch/x86/x86_64/entry.S          |  8 --------
 3 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/xen/arch/x86/include/asm/asm_defns.h b/xen/arch/x86/include/asm/asm_defns.h
index a9a6c21c76cd..f18a11b36198 100644
--- a/xen/arch/x86/include/asm/asm_defns.h
+++ b/xen/arch/x86/include/asm/asm_defns.h
@@ -56,6 +56,18 @@ register unsigned long current_stack_pointer asm("rsp");
 #define ASSERT_INTERRUPTS_DISABLED \
     ASSERT_INTERRUPT_STATUS(z, "INTERRUPTS DISABLED")
 
+#ifdef __ASSEMBLY__
+
+.macro BUILD_BUG_ON condstr cond:vararg
+        .if \cond
+        .error "Condition \"\condstr\" not satisfied"
+        .endif
+.endm
+/* preprocessor macro to make error message more user friendly */
+#define BUILD_BUG_ON(cond) BUILD_BUG_ON #cond cond
+
+#endif /* __ASSEMBLY__ */
+
 #ifdef __ASSEMBLY__
 # define _ASM_EX(p) p-.
 #else
diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S
index d4f0e4804090..93fbbeb4ae18 100644
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -15,7 +15,7 @@ FUNC(entry_int82)
         ENDBR64
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         pushq $0
-        movl  $HYPERCALL_VECTOR, 4(%rsp)
+        movb  $HYPERCALL_VECTOR, 4(%rsp)
         SAVE_ALL compat=1 /* DPL1 gate, restricted to 32bit PV guests only. */
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index a7bd8f0ca5b1..f8938b0b42fd 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -22,14 +22,6 @@
 #endif
 .endm
 
-.macro BUILD_BUG_ON condstr cond:vararg
-        .if \cond
-        .error "Condition \"\condstr\" not satisfied"
-        .endif
-.endm
-/* preprocessor macro to make error message more user friendly */
-#define BUILD_BUG_ON(cond) BUILD_BUG_ON #cond cond
-
 #ifdef CONFIG_PV
 /* %rbx: struct vcpu */
 FUNC_LOCAL(switch_to_kernel)
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:55:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:55:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685525.1066375 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaVe-00052c-HQ; Mon, 26 Feb 2024 12:55:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685525.1066375; Mon, 26 Feb 2024 12:55:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaVe-00051p-9s; Mon, 26 Feb 2024 12:55:06 +0000
Received: by outflank-mailman (input) for mailman id 685525;
 Mon, 26 Feb 2024 12:55:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1reaVd-0004hn-2S
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:55:05 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4205485a-d4a6-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 13:55:03 +0100 (CET)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a2f22bfb4e6so399546466b.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:55:03 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ll9-20020a170907190900b00a42e4b5ab00sm2379645ejc.41.2024.02.26.04.55.02
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 04:55:02 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4205485a-d4a6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708952103; x=1709556903; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=qpfXncMohFLZgpylZu3EmkAWeq+PuPeKADHYSj96UJs=;
        b=v6OAlLR89UxQ74ZgWloIpSq5qw8DW21I2fpyFq5fslg7UAzLPc/mgPh0B6H4Q5hgnT
         l9kEkuOnaoz5wUJ+DUEMdyStPY98e4S6M93A9sOXgAo7/dKzEV+naf461FsxVMkInT5K
         xvcXgqSyW8vM8RkXdj95+CmKkaBpU8UljVj3o=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708952103; x=1709556903;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=qpfXncMohFLZgpylZu3EmkAWeq+PuPeKADHYSj96UJs=;
        b=ACNUZxOBRqXHpiu5PHzasvRiI0zCB24cXLPAbK3N7nKysZoUSnMuJttJAMoS/USkfO
         P1r1f43f9rwdMcfSkli4wMApYhubeNNA0RYRHXfCVv6raS605lwthy7tkn3VdIgzN2O6
         gYQhs5OqkCM7VGdGnR4J2LgLBiVm+b7AIDTGL74WznEHbQ0fX/E6Gjbh/td4qDsDvPbU
         Qxwr5ChMWQTutj+gRhodZqYGzdsOlCyD4/k2QQ4tdny1JYbR4AzCrK7zAVxkkDrJAlOC
         lcSXREmSnzPs+7ZGzbOWGmz9H8M57e8OuMSwaQJj5iKctmWAW2A1kf/CpQOZAEbw3tlz
         9JSw==
X-Gm-Message-State: AOJu0YyrAH6mpvPSyIIicuHu/dC7f2D9f72lTqxX8Cy3CMgDDz6HU73k
	3sXX8E0x8jyEv/Wzo4M8nqRx+T3RYky5OQ6EeiYhCS4UobDmx1UortspQzULACwbm+KqZDNpPc2
	d
X-Google-Smtp-Source: AGHT+IHLDkpHncUByB7yvq733QUUtLrPIsQgcZFEgyD9njN+AB36pv1EjM9BRT+uXVSZK43w/9ebJA==
X-Received: by 2002:a17:906:3516:b0:a42:615:1395 with SMTP id r22-20020a170906351600b00a4206151395mr4149355eja.11.1708952102806;
        Mon, 26 Feb 2024 04:55:02 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 0/3] x86/entry: More cleanup
Date: Mon, 26 Feb 2024 12:54:58 +0000
Message-Id: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Misc adjustments and cleanup.  No functional change.

Andrew Cooper (3):
  x86/entry: Adjustments to "reduce assembly code size of entry points"
  x86/entry: Simplify expressions in compat_restore_all_guest()
  x86/entry: Introduce EFRAME_* constants

 xen/arch/x86/include/asm/asm_defns.h | 12 +++++
 xen/arch/x86/x86_64/asm-offsets.c    | 17 +++++++
 xen/arch/x86/x86_64/compat/entry.S   | 12 ++---
 xen/arch/x86/x86_64/entry.S          | 76 +++++++++++++---------------
 4 files changed, 69 insertions(+), 48 deletions(-)


base-commit: 8de3afc0b402bc17f65093a53e5870862707a8c7
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:55:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:55:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685526.1066388 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaVf-0005Sv-Kk; Mon, 26 Feb 2024 12:55:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685526.1066388; Mon, 26 Feb 2024 12:55:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaVf-0005Ro-GJ; Mon, 26 Feb 2024 12:55:07 +0000
Received: by outflank-mailman (input) for mailman id 685526;
 Mon, 26 Feb 2024 12:55:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1reaVe-0004hn-2h
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:55:06 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 429d66e0-d4a6-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 13:55:04 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a3e75e30d36so585120466b.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:55:04 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ll9-20020a170907190900b00a42e4b5ab00sm2379645ejc.41.2024.02.26.04.55.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 04:55:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 429d66e0-d4a6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708952104; x=1709556904; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=cUPkBSwvovweP2Vn5x5Slws9PAyzIz15Jh15k5P3vWY=;
        b=DfOoHuQR819ayTj+9RmgidFAecxkIKZqWBoD+g0No+0ZClGRsD/OkC7RDjf0bg4Imo
         cqx2WlKan6Op6JYOmuaUhp8nryRlGkdpqcPOl5COUX39U1FHCbJ3ZdJacfkxgDkwNvC8
         qMvphZGBcYp89r3ebuPjqz2CyyLzPvmvgS3fU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708952104; x=1709556904;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=cUPkBSwvovweP2Vn5x5Slws9PAyzIz15Jh15k5P3vWY=;
        b=iWuee0cxc7Sy0CluBeqV2wOCIFscROkh/oOkDHEkkmZaUQ/6SdlOE4Mh10UysvghJK
         NRHDYnXFHLoHY8XQOiMuWrirVQbE3E3hrRmf+pHNcLQKhn3b2GkX/2rFcwj4BF/VLqOd
         JOOTAaJKGAa+6l27zi/rS4xKpB3CakGxm2K6TsrMbySwQPYLSIjrp7s2tgaz7lhmZ43e
         ICKbNijrUz5Bc/Qu23A7fpgrtELkBa+/rcgicKuQy6EDRmBWWmauFXQOHweE2n3tTpMR
         VyoaIKGJ7uwOzZxxu6OG4J+4CYxoRRDhG5fBukzIj9sPWbwDcHYMCP8EWycMt/Ec0OsA
         N5Mw==
X-Gm-Message-State: AOJu0YzThXl2Tzk4UCuKUBBSMDo4LtOfusLrBn4BW50rbbnKGXq/cjs8
	/he2/JrW5H25DGMqOtT9gCtY/IFL9uo6ZVXZP5TvN7m5il8ZOlDbRT4MaGNxVTNMvk9ibuiZChD
	r
X-Google-Smtp-Source: AGHT+IEKNzT7MZ4rDTxGGiXsJK7wd2sxKAEBDCZBcMxCYN7prH6PCeoNPa15iBXbH092aJEeovGm2g==
X-Received: by 2002:a17:906:1159:b0:a3f:3d0d:30cc with SMTP id i25-20020a170906115900b00a3f3d0d30ccmr5855933eja.1.1708952103876;
        Mon, 26 Feb 2024 04:55:03 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 2/3] x86/entry: Simplify expressions in compat_restore_all_guest()
Date: Mon, 26 Feb 2024 12:55:00 +0000
Message-Id: <20240226125501.1233599-3-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
References: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

compat_restore_all_guest() already has SPEC_CTRL_EXIT_TO_PV with a documented
requirement for %rsp to be both regs and cpuinfo.

Use the now-normal annotations and simplify the expressions which happen to be
a subtraction of 0.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 xen/arch/x86/x86_64/compat/entry.S | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S
index 93fbbeb4ae18..727ab65290de 100644
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -117,19 +117,19 @@ compat_process_trap:
         jmp   compat_test_all_events
 END(compat_test_all_events)
 
-/* %rbx: struct vcpu, interrupts disabled */
+/* %rbx: struct vcpu, interrupts disabled, %rsp=regs/cpuinfo */
 FUNC(compat_restore_all_guest)
         ASSERT_INTERRUPTS_DISABLED
         mov   $~(X86_EFLAGS_IOPL | X86_EFLAGS_VM), %r11d
         and   UREGS_eflags(%rsp),%r11d
 
-.macro alt_cr4_pv32
+.macro alt_cr4_pv32 /* %rsp=regs/cpuinfo */
         testb $3,UREGS_cs(%rsp)
         jpe   2f
-        mov   CPUINFO_cr4-CPUINFO_guest_cpu_user_regs(%rsp), %rax
+        mov   CPUINFO_cr4(%rsp), %rax
         and   $~XEN_CR4_PV32_BITS, %rax
 1:
-        mov   %rax, CPUINFO_cr4-CPUINFO_guest_cpu_user_regs(%rsp)
+        mov   %rax, CPUINFO_cr4(%rsp)
         mov   %rax, %cr4
         /*
          * An NMI or MCE may have occurred between the previous two
@@ -141,7 +141,7 @@ FUNC(compat_restore_all_guest)
          * loop to cause a live lock: If NMIs/MCEs occurred at that
          * high a rate, we'd be live locked anyway.
          */
-        cmp   %rax, CPUINFO_cr4-CPUINFO_guest_cpu_user_regs(%rsp)
+        cmp   %rax, CPUINFO_cr4(%rsp)
         jne   1b
 2:
 .endm
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:55:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:55:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685527.1066394 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaVg-0005ae-5e; Mon, 26 Feb 2024 12:55:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685527.1066394; Mon, 26 Feb 2024 12:55:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaVf-0005Z4-Vw; Mon, 26 Feb 2024 12:55:07 +0000
Received: by outflank-mailman (input) for mailman id 685527;
 Mon, 26 Feb 2024 12:55:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1reaVf-0004hn-2l
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:55:07 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 430a8b83-d4a6-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 13:55:05 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a2f22bfb4e6so399551066b.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:55:05 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ll9-20020a170907190900b00a42e4b5ab00sm2379645ejc.41.2024.02.26.04.55.03
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 04:55:04 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 430a8b83-d4a6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708952104; x=1709556904; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=iwdL6fju3yL7qCFHqrnxqIG6VGZd2rsCO2B5E1ELuco=;
        b=FPXCLop5y5Fmug6EbeIqryKW/Hwk+uVDYAeSGE2Z5XAEsg6ni+Jdq3zMAHUv2UXqDf
         5g7rJ+j8PvxD+io7l2r32Y6bG0Q+GD0sks2HXvEbPaWK8ccT0yTnuEw4S3XoOnYocv0n
         FiKSZxDDtPMBL38gR3P+ebg5TBygJ4R80yVFg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708952104; x=1709556904;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=iwdL6fju3yL7qCFHqrnxqIG6VGZd2rsCO2B5E1ELuco=;
        b=OuEzg6hUpp8eDrSfnltJcr6oy5cq3GDWCgIO5dn+Kfw8KCFjIMAnOK6AGzgC6/hmDH
         DXvmhuzt62Js1lvwTIsoXiP0zPYKdAL8Sxt6qfHcHMzl5xt7Z5b905hpOYJ9PKqQEJcH
         29DDOx1UtU7WndN1Q12YoaEhiwuSl7OK0TAcisy82S77iKUvn+0KYu2MPDY/q/hU52mp
         a1dYLFdKrZnKBWqAefxdtW5sBqhluNX8m0phkr1OF+P2Iobh4uRuzGbfjH0jJ/kp0n5i
         7KNiP0+XlOO74JDqijV3NnKTrMC2R7Coget9XqEBT537C0nurPA9T0NHGJe/kiySMO+D
         lsjQ==
X-Gm-Message-State: AOJu0Ywohu4KyGCYDFFkiLYCj3Ce6E87uYPkJbj/0nnNewhKaHQUtFs6
	21dSPKTETW1MkG96a49MN6/gwjJnt1iT56TuWkUUMk9tcvzWmhLaFYtN4WYyEJE1DLzhcMNmBG3
	S
X-Google-Smtp-Source: AGHT+IEiO4/3u92iuWSWX1XLK6tyUrg8vJ3vqcOHRIZXVtosNyOvF+7YKfdvK7IrnDxa4yULQuLlIg==
X-Received: by 2002:a17:906:57d3:b0:a43:239c:3d0d with SMTP id u19-20020a17090657d300b00a43239c3d0dmr2556694ejr.37.1708952104635;
        Mon, 26 Feb 2024 04:55:04 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH 3/3] x86/entry: Introduce EFRAME_* constants
Date: Mon, 26 Feb 2024 12:55:01 +0000
Message-Id: <20240226125501.1233599-4-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
In-Reply-To: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
References: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

restore_all_guest() does a lot of manipulation of the stack after popping the
GPRs, and uses raw %rsp displacements to do so.  Also, almost all entrypaths
use raw %rsp displacements prior to pushing GPRs.

Provide better mnemonics, to aid readability and reduce the chance of errors
when editing.

No functional change.  The resulting binary is identical.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 xen/arch/x86/x86_64/asm-offsets.c  | 17 ++++++++
 xen/arch/x86/x86_64/compat/entry.S |  2 +-
 xen/arch/x86/x86_64/entry.S        | 68 +++++++++++++++---------------
 3 files changed, 52 insertions(+), 35 deletions(-)

diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index fee0edc61abb..4cc23cd032c1 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -51,6 +51,23 @@ void __dummy__(void)
     OFFSET(UREGS_kernel_sizeof, struct cpu_user_regs, es);
     BLANK();
 
+    /*
+     * EFRAME_* is for the entry/exit logic where %rsp is pointing at
+     * UREGS_error_code and GPRs are still guest values.
+     */
+#define OFFSET_EF(sym, mem)                                             \
+    DEFINE(sym, offsetof(struct cpu_user_regs, mem) -                   \
+                offsetof(struct cpu_user_regs, error_code))
+
+    OFFSET_EF(EFRAME_entry_vector,    entry_vector);
+    OFFSET_EF(EFRAME_rip,             rip);
+    OFFSET_EF(EFRAME_cs,              cs);
+    OFFSET_EF(EFRAME_eflags,          eflags);
+    OFFSET_EF(EFRAME_rsp,             rsp);
+    BLANK();
+
+#undef OFFSET_EF
+
     OFFSET(VCPU_processor, struct vcpu, processor);
     OFFSET(VCPU_domain, struct vcpu, domain);
     OFFSET(VCPU_vcpu_info, struct vcpu, vcpu_info_area.map);
diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S
index 727ab65290de..2f8fe5ebfbe4 100644
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -15,7 +15,7 @@ FUNC(entry_int82)
         ENDBR64
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         pushq $0
-        movb  $HYPERCALL_VECTOR, 4(%rsp)
+        movb  $HYPERCALL_VECTOR, EFRAME_entry_vector(%rsp)
         SAVE_ALL compat=1 /* DPL1 gate, restricted to 32bit PV guests only. */
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index f8938b0b42fd..1b846f3aaff0 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -188,15 +188,15 @@ FUNC_LOCAL(restore_all_guest)
 
         RESTORE_ALL
         BUILD_BUG_ON(TRAP_syscall & 0xff)
-        testb $TRAP_syscall >> 8, 4+1(%rsp)
+        testb $TRAP_syscall >> 8, EFRAME_entry_vector + 1(%rsp)
         jz    iret_exit_to_guest
 
-        movq  24(%rsp),%r11           # RFLAGS
+        mov   EFRAME_eflags(%rsp), %r11
         andq  $~(X86_EFLAGS_IOPL | X86_EFLAGS_VM), %r11
         orq   $X86_EFLAGS_IF,%r11
 
         /* Don't use SYSRET path if the return address is not canonical. */
-        movq  8(%rsp),%rcx
+        mov   EFRAME_rip(%rsp), %rcx
         sarq  $47,%rcx
         incl  %ecx
         cmpl  $1,%ecx
@@ -211,19 +211,19 @@ FUNC_LOCAL(restore_all_guest)
         ALTERNATIVE "", rag_clrssbsy, X86_FEATURE_XEN_SHSTK
 #endif
 
-        movq  8(%rsp), %rcx           # RIP
-        cmpw  $FLAT_USER_CS32,16(%rsp)# CS
-        movq  32(%rsp),%rsp           # RSP
+        mov   EFRAME_rip(%rsp), %rcx
+        cmpw  $FLAT_USER_CS32, EFRAME_cs(%rsp)
+        mov   EFRAME_rsp(%rsp), %rsp
         je    1f
         sysretq
 1:      sysretl
 
 LABEL_LOCAL(.Lrestore_rcx_iret_exit_to_guest)
-        movq  8(%rsp), %rcx           # RIP
+        mov   EFRAME_rip(%rsp), %rcx
 /* No special register assumptions. */
 iret_exit_to_guest:
-        andl  $~(X86_EFLAGS_IOPL | X86_EFLAGS_VM), 24(%rsp)
-        orl   $X86_EFLAGS_IF,24(%rsp)
+        andl  $~(X86_EFLAGS_IOPL | X86_EFLAGS_VM), EFRAME_eflags(%rsp)
+        orl   $X86_EFLAGS_IF, EFRAME_eflags(%rsp)
         addq  $8,%rsp
 .Lft0:  iretq
         _ASM_PRE_EXTABLE(.Lft0, handle_exception)
@@ -256,7 +256,7 @@ FUNC(lstar_enter)
         pushq %rcx
         pushq $0
         BUILD_BUG_ON(TRAP_syscall & 0xff)
-        movb  $TRAP_syscall >> 8, 4+1(%rsp)
+        movb  $TRAP_syscall >> 8, EFRAME_entry_vector + 1(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -295,7 +295,7 @@ FUNC(cstar_enter)
         pushq %rcx
         pushq $0
         BUILD_BUG_ON(TRAP_syscall & 0xff)
-        movb  $TRAP_syscall >> 8, 4+1(%rsp)
+        movb  $TRAP_syscall >> 8, EFRAME_entry_vector + 1(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -338,7 +338,7 @@ LABEL(sysenter_eflags_saved, 0)
         pushq $0 /* null rip */
         pushq $0
         BUILD_BUG_ON(TRAP_syscall & 0xff)
-        movb  $TRAP_syscall >> 8, 4+1(%rsp)
+        movb  $TRAP_syscall >> 8, EFRAME_entry_vector + 1(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -393,7 +393,7 @@ FUNC(entry_int80)
         ENDBR64
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         pushq $0
-        movb  $0x80, 4(%rsp)
+        movb  $0x80, EFRAME_entry_vector(%rsp)
         SAVE_ALL
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
@@ -658,7 +658,7 @@ END(ret_from_intr)
         .section .init.text, "ax", @progbits
 FUNC(early_page_fault)
         ENDBR64
-        movb  $X86_EXC_PF, 4(%rsp)
+        movb  $X86_EXC_PF, EFRAME_entry_vector(%rsp)
         SAVE_ALL
         movq  %rsp, %rdi
         call  do_early_page_fault
@@ -727,7 +727,7 @@ END(common_interrupt)
 
 FUNC(entry_PF)
         ENDBR64
-        movb  $X86_EXC_PF, 4(%rsp)
+        movb  $X86_EXC_PF, EFRAME_entry_vector(%rsp)
 END(entry_PF)
 /* No special register assumptions. */
 FUNC(handle_exception, 0)
@@ -911,98 +911,98 @@ END(entry_DE)
 FUNC(entry_MF)
         ENDBR64
         pushq $0
-        movb  $X86_EXC_MF, 4(%rsp)
+        movb  $X86_EXC_MF, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_MF)
 
 FUNC(entry_XM)
         ENDBR64
         pushq $0
-        movb  $X86_EXC_XM, 4(%rsp)
+        movb  $X86_EXC_XM, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_XM)
 
 FUNC(entry_NM)
         ENDBR64
         pushq $0
-        movb  $X86_EXC_NM, 4(%rsp)
+        movb  $X86_EXC_NM, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_NM)
 
 FUNC(entry_DB)
         ENDBR64
         pushq $0
-        movb  $X86_EXC_DB, 4(%rsp)
+        movb  $X86_EXC_DB, EFRAME_entry_vector(%rsp)
         jmp   handle_ist_exception
 END(entry_DB)
 
 FUNC(entry_BP)
         ENDBR64
         pushq $0
-        movb  $X86_EXC_BP, 4(%rsp)
+        movb  $X86_EXC_BP, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_BP)
 
 FUNC(entry_OF)
         ENDBR64
         pushq $0
-        movb  $X86_EXC_OF, 4(%rsp)
+        movb  $X86_EXC_OF, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_OF)
 
 FUNC(entry_BR)
         ENDBR64
         pushq $0
-        movb  $X86_EXC_BR, 4(%rsp)
+        movb  $X86_EXC_BR, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_BR)
 
 FUNC(entry_UD)
         ENDBR64
         pushq $0
-        movb  $X86_EXC_UD, 4(%rsp)
+        movb  $X86_EXC_UD, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_UD)
 
 FUNC(entry_TS)
         ENDBR64
-        movb  $X86_EXC_TS, 4(%rsp)
+        movb  $X86_EXC_TS, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_TS)
 
 FUNC(entry_NP)
         ENDBR64
-        movb  $X86_EXC_NP, 4(%rsp)
+        movb  $X86_EXC_NP, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_NP)
 
 FUNC(entry_SS)
         ENDBR64
-        movb  $X86_EXC_SS, 4(%rsp)
+        movb  $X86_EXC_SS, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_SS)
 
 FUNC(entry_GP)
         ENDBR64
-        movb  $X86_EXC_GP, 4(%rsp)
+        movb  $X86_EXC_GP, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_GP)
 
 FUNC(entry_AC)
         ENDBR64
-        movb  $X86_EXC_AC, 4(%rsp)
+        movb  $X86_EXC_AC, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_AC)
 
 FUNC(entry_CP)
         ENDBR64
-        movb  $X86_EXC_CP, 4(%rsp)
+        movb  $X86_EXC_CP, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 END(entry_CP)
 
 FUNC(entry_DF)
         ENDBR64
-        movb  $X86_EXC_DF, 4(%rsp)
+        movb  $X86_EXC_DF, EFRAME_entry_vector(%rsp)
         /* Set AC to reduce chance of further SMAP faults */
         ALTERNATIVE "", stac, X86_FEATURE_XEN_SMAP
         SAVE_ALL
@@ -1028,7 +1028,7 @@ END(entry_DF)
 FUNC(entry_NMI)
         ENDBR64
         pushq $0
-        movb  $X86_EXC_NMI, 4(%rsp)
+        movb  $X86_EXC_NMI, EFRAME_entry_vector(%rsp)
 END(entry_NMI)
 
 FUNC(handle_ist_exception)
@@ -1164,7 +1164,7 @@ END(handle_ist_exception)
 FUNC(entry_MC)
         ENDBR64
         pushq $0
-        movb  $X86_EXC_MC, 4(%rsp)
+        movb  $X86_EXC_MC, EFRAME_entry_vector(%rsp)
         jmp   handle_ist_exception
 END(entry_MC)
 
@@ -1203,7 +1203,7 @@ FUNC_LOCAL(autogen_stubs, 0) /* Automatically generated stubs. */
 1:
         ENDBR64
         pushq $0
-        movb  $vec,4(%rsp)
+        movb  $vec, EFRAME_entry_vector(%rsp)
         jmp   common_interrupt
 
         entrypoint 1b
@@ -1217,7 +1217,7 @@ FUNC_LOCAL(autogen_stubs, 0) /* Automatically generated stubs. */
         test  $8,%spl        /* 64bit exception frames are 16 byte aligned, but the word */
         jz    2f             /* size is 8 bytes.  Check whether the processor gave us an */
         pushq $0             /* error code, and insert an empty one if not.              */
-2:      movb  $vec,4(%rsp)
+2:      movb  $vec, EFRAME_entry_vector(%rsp)
         jmp   handle_exception
 
         entrypoint 1b
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:57:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:57:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685534.1066408 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaYK-0007Ou-Iw; Mon, 26 Feb 2024 12:57:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685534.1066408; Mon, 26 Feb 2024 12:57:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaYK-0007On-Fs; Mon, 26 Feb 2024 12:57:52 +0000
Received: by outflank-mailman (input) for mailman id 685534;
 Mon, 26 Feb 2024 12:57:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reaYJ-0007Ob-NO; Mon, 26 Feb 2024 12:57:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reaYJ-0004Ac-KN; Mon, 26 Feb 2024 12:57:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reaYJ-0001CP-D2; Mon, 26 Feb 2024 12:57:51 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1reaYJ-0004Zy-CQ; Mon, 26 Feb 2024 12:57:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=lDsnbzdXGpKAJwN8NF4Wyf/AHg6i8fCEX7Omfrjx9HM=; b=cqo5rzECupDUvFrgrj1FiYYutw
	JU2H+tUHg9OP0QHI847qTVWVkKUBwLpJwnTBxEfrxcDwjouj7ZgQSBu7Ais8IwJiX9uGKs6X1C6bJ
	z7XoW/9+sxiJ5gWa0RrO3QjRGxghtkRDCe+wHfMxbaZltlJOXOyyH4BcYN5jtY6aD57E=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184763-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184763: tolerable FAIL
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow:debian-hvm-install:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit2:xen-boot:fail:heisenbug
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=92babc88f67ed0ef3dc575a8b9534040274678ee
X-Osstest-Versions-That:
    xen=92babc88f67ed0ef3dc575a8b9534040274678ee
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 26 Feb 2024 12:57:51 +0000

flight 184763 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184763/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 12 debian-hvm-install fail in 184756 pass in 184763
 test-armhf-armhf-xl-credit2   8 xen-boot                   fail pass in 184756

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit2 15 migrate-support-check fail in 184756 never pass
 test-armhf-armhf-xl-credit2 16 saverestore-support-check fail in 184756 never pass
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184756
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184756
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184756
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184756
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184756
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184756
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184756
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184756
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184756
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184756
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184756
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184756
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  92babc88f67ed0ef3dc575a8b9534040274678ee
baseline version:
 xen                  92babc88f67ed0ef3dc575a8b9534040274678ee

Last test of basis   184763  2024-02-26 01:52:19 Z    0 days
Testing same since                          (not found)         0 attempts

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  fail    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Published tested tree is already up to date.



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 12:58:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 12:58:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685538.1066418 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaYf-0007qa-1O; Mon, 26 Feb 2024 12:58:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685538.1066418; Mon, 26 Feb 2024 12:58:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reaYe-0007qT-Ut; Mon, 26 Feb 2024 12:58:12 +0000
Received: by outflank-mailman (input) for mailman id 685538;
 Mon, 26 Feb 2024 12:58:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reaYe-0007q5-0o
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 12:58:12 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b1e9bbc5-d4a6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 13:58:11 +0100 (CET)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-512f54fc2dbso1660493e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 04:58:11 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 dw18-20020a0565122c9200b00513006c81e6sm182331lfb.227.2024.02.26.04.58.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 04:58:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1e9bbc5-d4a6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708952290; x=1709557090; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Ocrjv9MX/0j1ycgtaB2LxdkVc/truj/tAIo/gUlOhzk=;
        b=HlTzpWhMSd7ru6/cTRMt2wZnRW3uY8VEM/8cZUDxent5WKdNM5XAThEgeaCzrWN5se
         M7Os+P5LYtCM6T7HWdxxwyYFUYWXncVKrEo6dhHoILZcRbWk52KNvn5EffJa1RF375yN
         4jZtllIy4ZEZd59drifkT/Zsee2Kbw70G59Q5bbJP0/pNdXo3cJ5IhmXqIX3PCFHnW0s
         XlVoQHNLfA75Ppt8xxBYa3zU5ymhvpJIAhkC9ly8m/wpoOzjz4mPpj05ju5cMuPqYVoN
         iwWj1yVTjNBtBdsqtAwK8DwF7TDA6twyjHOf379I16yd8UjPy5gO4T+0JzUeND/0zEOR
         OuLw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708952290; x=1709557090;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Ocrjv9MX/0j1ycgtaB2LxdkVc/truj/tAIo/gUlOhzk=;
        b=wiayfxm3NaCDiNuOrFBZe0xGO0cZnHLFsMJ52hwEghXdtXHNOzwQ5SA5t8xfZJ7QM2
         fqlOsfVPquJKIN9ZbZaGLtQRyBD/Ae0sflfiFLMqbNZl3tulAeQuI+7qOb1/sbUl+fYm
         DfhE3u0vZIhaPD+hHhMLGrVriTlrhHYLg5ZxBgecz+UZu3dMjBlZ9uU9/66UlpfnZg/g
         FZNexa10wzbJJxm2TovNUqsq5F8/qmK5q+7saxCf5U5fumOgV3wDjifcmsr9UUaF6Aax
         TRCc40lfdzbXcS84F//2eXNzUoVToWIRweGJXsASH2rSrOi5vEXRJ4G568fV9UMnPVea
         gqwQ==
X-Forwarded-Encrypted: i=1; AJvYcCU4vmOqnPJj1FZx6OTFZ091dRgK9vykhOuijGqWJx5wteRe7Nmt+0dRMC46lenEaHoEv4yAKWUs0g4VWfDNm4zemNpk2nCkE0166/k8ZXk=
X-Gm-Message-State: AOJu0Yyg6JyTEjLd2MIi/v7iO/3UcCPCKphGhIWhodDcjW+LzVhPWNrw
	grEtJ+tzlY6HYacx2rdPFf39CVx0eXN9XdKujq9neXVAq40HsKih
X-Google-Smtp-Source: AGHT+IGV/dNISZxrid+ksVXLd70QU18jT9jgSGTlEX46q5o4utWWVPfElJaDau67OLjNPvSBS4zWTw==
X-Received: by 2002:a05:6512:110c:b0:512:f5b8:8912 with SMTP id l12-20020a056512110c00b00512f5b88912mr3721949lfg.16.1708952290141;
        Mon, 26 Feb 2024 04:58:10 -0800 (PST)
Message-ID: <4215fb9eb5be5898b3cc9c2977e55ac0e7270b4d.camel@gmail.com>
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Mon, 26 Feb 2024 13:58:09 +0100
In-Reply-To: <7cab68d1-9bdf-4ea1-b49a-cce1e4af4692@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
	 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
	 <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
	 <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
	 <1c53b52ee39161a8f59209d28af69fe997479dbc.camel@gmail.com>
	 <3b38fe82-ee0f-4666-93e1-bd78fe69c534@suse.com>
	 <5d2d35fe014094c991363e42c3e2ad9ca2af3938.camel@gmail.com>
	 <44fd5092-7838-4d28-804b-bbfebfd44886@suse.com>
	 <56ae27d003b8763af34864ae56433691685c3661.camel@gmail.com>
	 <7cab68d1-9bdf-4ea1-b49a-cce1e4af4692@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-26 at 12:28 +0100, Jan Beulich wrote:
> On 26.02.2024 12:18, Oleksii wrote:
> > On Mon, 2024-02-26 at 10:45 +0100, Jan Beulich wrote:
> > > On 23.02.2024 13:23, Oleksii wrote:
> > > > >=20
> > > > > > > > As 1- and 2-byte cases are emulated I decided that is
> > > > > > > > not
> > > > > > > > to
> > > > > > > > provide
> > > > > > > > sfx argument for emulation macros as it will not have
> > > > > > > > to
> > > > > > > > much
> > > > > > > > affect on
> > > > > > > > emulated types and just consume more performance on
> > > > > > > > acquire
> > > > > > > > and
> > > > > > > > release
> > > > > > > > version of sc/ld instructions.
> > > > > > >=20
> > > > > > > Question is whether the common case (4- and 8-byte
> > > > > > > accesses)
> > > > > > > shouldn't
> > > > > > > be valued higher, with 1- and 2-byte emulation being
> > > > > > > there
> > > > > > > just
> > > > > > > to
> > > > > > > allow things to not break altogether.
> > > > > > If I understand you correctly, it would make sense to add
> > > > > > the
> > > > > > 'sfx'
> > > > > > argument for the 1/2-byte access case, ensuring that all
> > > > > > options
> > > > > > are
> > > > > > available for 1/2-byte access case as well.
> > > > >=20
> > > > > That's one of the possibilities. As said, I'm not overly
> > > > > worried
> > > > > about
> > > > > the emulated cases. For the initial implementation I'd
> > > > > recommend
> > > > > going
> > > > > with what is easiest there, yielding the best possible result
> > > > > for
> > > > > the
> > > > > 4- and 8-byte cases. If later it turns out repeated
> > > > > acquire/release
> > > > > accesses are a problem in the emulation loop, things can be
> > > > > changed
> > > > > to explicit barriers, without touching the 4- and 8-byte
> > > > > cases.
> > > > I am confused then a little bit if emulated case is not an
> > > > issue.
> > > >=20
> > > > For 4- and 8-byte cases for xchg .aqrl is used, for relaxed and
> > > > aqcuire
> > > > version of xchg barries are used.
> > > >=20
> > > > The similar is done for cmpxchg.
> > > >=20
> > > > If something will be needed to change in emulation loop it
> > > > won't
> > > > require to change 4- and 8-byte cases.
> > >=20
> > > I'm afraid I don't understand your reply.
> > IIUC, emulated cases it is implemented correctly in terms of usage
> > barriers. And it also OK not to use sfx for lr/sc instructions and
> > use
> > only barriers.
> >=20
> > For 4- and 8-byte cases are used sfx + barrier depending on the
> > specific case ( relaxed, acquire, release, generic xchg/cmpxchg ).
> > What also looks to me correct. But you suggested to provide the
> > best
> > possible result for 4- and 8-byte cases.=C2=A0
> >=20
> > So I don't understand what the best possible result is as the
> > current
> > one usage of __{cmp}xchg_generic for each specific case=C2=A0 ( relaxed=
,
> > acquire, release, generic xchg/cmpxchg ) looks correct to me:
> > xchg -> (..., ".aqrl", "", "") just suffix .aqrl suffix without
> > barriers.
> > xchg_release -> (..., "", RISCV_RELEASE_BARRIER, "" ) use only
> > release
> > barrier
> > xchg_acquire -> (..., "", "", RISCV_ACQUIRE_BARRIER ), only acquire
> > barrier
> > xchg_relaxed ->=C2=A0 (..., "", "", "") - no barries, no sfx
>=20
> So first: While explicit barriers are technically okay, I don't
> follow why
> you insist on using them when you can achieve the same by suitably
> tagging
> the actual insn doing the exchange. Then second: It's somewhat hard
> for me
> to see the final effect on the emulation paths without you actually
> having
> done the switch. Maybe no special handling is necessary there anymore
> then. And as said, it may actually be acceptable for the emulation
> paths
> to "only" be correct, but not be ideal in terms of performance. After
> all,
> if you use the normal 4-byte primitive in there, more (non-explicit)
> barriers than needed would occur if the involved loop has to take
> more
> than one iteration. Which could (but imo doesn't need to be) avoided
> by
> using a more relaxed 4-byte primitive there and an explicit barrier
> outside of the loop.

According to the spec:
Table A.5 ( part of the table only I copied here )

Linux Construct          RVWMO Mapping
atomic <op> relaxed           amo<op>.{w|d}
atomic <op> acquire           amo<op>.{w|d}.aq
atomic <op> release           amo<op>.{w|d}.rl
atomic <op>                   amo<op>.{w|d}.aqrl

Linux Construct          RVWMO LR/SC Mapping
atomic <op> relaxed       loop: lr.{w|d}; <op>; sc.{w|d}; bnez loop
atomic <op> acquire       loop: lr.{w|d}.aq; <op>; sc.{w|d}; bnez loop
atomic <op> release       loop: lr.{w|d}; <op>; sc.{w|d}.aqrl=E2=88=97 ; bn=
ez=20
loop OR
                          fence.tso; loop: lr.{w|d}; <op>; sc.{w|d}=E2=88=
=97 ;
bnez loop
atomic <op>               loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez
loop

The Linux mappings for release operations may seem stronger than
necessary, but these mappings
are needed to cover some cases in which Linux requires stronger
orderings than the more intuitive
mappings would provide. In particular, as of the time this text is
being written, Linux is actively
debating whether to require load-load, load-store, and store-store
orderings between accesses in one
critical section and accesses in a subsequent critical section in the
same hart and protected by the
same synchronization object. Not all combinations of FENCE RW,W/FENCE
R,RW mappings
with aq/rl mappings combine to provide such orderings. There are a few
ways around this problem,
including:
1. Always use FENCE RW,W/FENCE R,RW, and never use aq/rl. This suffices
but is undesir-
able, as it defeats the purpose of the aq/rl modifiers.
2. Always use aq/rl, and never use FENCE RW,W/FENCE R,RW. This does not
currently work
due to the lack of load and store opcodes with aq and rl modifiers.
3. Strengthen the mappings of release operations such that they would
enforce sufficient order-
ings in the presence of either type of acquire mapping. This is the
currently-recommended
solution, and the one shown in Table A.5.


Based on this it is enough in our case use only suffixed istructions
(amo<op>.{w|d}{.aq, .rl, .aqrl, .aqrl }, lr.{w|d}.{.aq, .aqrl }.


But as far as I understand in Linux atomics were strengthen with
fences:
    Atomics present the same issue with locking: release and acquire
    variants need to be strengthened to meet the constraints defined
    by the Linux-kernel memory consistency model [1].
   =20
    Atomics present a further issue: implementations of atomics such
    as atomic_cmpxchg() and atomic_add_unless() rely on LR/SC pairs,
    which do not give full-ordering with .aqrl; for example, current
    implementations allow the "lr-sc-aqrl-pair-vs-full-barrier" test
    below to end up with the state indicated in the "exists" clause.
   =20
    In order to "synchronize" LKMM and RISC-V's implementation, this
    commit strengthens the implementations of the atomics operations
    by replacing .rl and .aq with the use of ("lightweigth") fences,
    and by replacing .aqrl LR/SC pairs in sequences such as:
   =20
      0:      lr.w.aqrl  %0, %addr
              bne        %0, %old, 1f
              ...
              sc.w.aqrl  %1, %new, %addr
              bnez       %1, 0b
      1:
   =20
    with sequences of the form:
   =20
      0:      lr.w       %0, %addr
              bne        %0, %old, 1f
              ...
              sc.w.rl    %1, %new, %addr   /* SC-release   */
              bnez       %1, 0b
              fence      rw, rw            /* "full" fence */
      1:
   =20
    following Daniel's suggestion.
   =20
    These modifications were validated with simulation of the RISC-V
    with sequences of the form:
   =20
      0:      lr.w       %0, %addr
              bne        %0, %old, 1f
              ...
              sc.w.rl    %1, %new, %addr   /* SC-release   */
              bnez       %1, 0b
              fence      rw, rw            /* "full" fence */
      1:
   =20
    following Daniel's suggestion.
   =20
    These modifications were validated with simulation of the RISC-V
    memory consistency model.
   =20
    C lr-sc-aqrl-pair-vs-full-barrier
   =20
    {}
   =20
    P0(int *x, int *y, atomic_t *u)
    {
            int r0;
            int r1;
   =20
            WRITE_ONCE(*x, 1);
            r0 =3D atomic_cmpxchg(u, 0, 1);
            r1 =3D READ_ONCE(*y);
    }
   =20
    P1(int *x, int *y, atomic_t *v)
    {
            int r0;
            int r1;
   =20
            WRITE_ONCE(*y, 1);
            r0 =3D atomic_cmpxchg(v, 0, 1);
            r1 =3D READ_ONCE(*x);
    }
   =20
    exists (u=3D1 /\ v=3D1 /\ 0:r1=3D0 /\ 1:r1=3D0)
   =20
    [1] https://marc.info/?l=3Dlinux-kernel&m=3D151930201102853&w=3D2
    =20
https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/hKywNHBk=
AXM
        https://marc.info/?l=3Dlinux-kernel&m=3D151633436614259&w=3D2


Thereby Linux kernel implementation seems to me more safe and it is a
reason why I want/wanted to be aligned with it.

~ Oleksii




From xen-devel-bounces@lists.xenproject.org Mon Feb 26 13:37:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 13:37:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685549.1066428 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebA8-0006sb-Uk; Mon, 26 Feb 2024 13:36:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685549.1066428; Mon, 26 Feb 2024 13:36:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebA8-0006sU-S8; Mon, 26 Feb 2024 13:36:56 +0000
Received: by outflank-mailman (input) for mailman id 685549;
 Mon, 26 Feb 2024 13:36:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rebA7-0006sO-HF
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 13:36:55 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1a1ad0f5-d4ac-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 14:36:53 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-512f7332252so1914379e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 05:36:53 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 bq34-20020a05620a46a200b007873c82f0easm2429708qkb.113.2024.02.26.05.36.51
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 05:36:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1a1ad0f5-d4ac-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708954613; x=1709559413; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=J4LsyS/JwWxmKvnclXrztXMRGI7d4Hu8lSZ6qOUJFiQ=;
        b=MymnbceQ/VKb/3KOL69KTmmT41ReD/Wu40XIpUFP0hosuKGkxc3x6HQLmcCRbo6RAM
         tUvdJN0zy3SWjqIdiCCDJZ25LWAXLCqjKzyDWsl+7hfSdlMC+a4MzGCGk+a55EuzMPpu
         MRM9rXDM2OWut2cuTemSo/0XIXOie5S+e4qok=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708954613; x=1709559413;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=J4LsyS/JwWxmKvnclXrztXMRGI7d4Hu8lSZ6qOUJFiQ=;
        b=va2TQn6hukEYZRAeur239Tsc0LUj36WAKzcNCIBFaiLc/9PzcT778YW4WseqBN36ui
         wLXAPG9tQxrZ55QIHAPE2QsI4UkhIBm/0fUoxnZG80ej+CHZI+wfFgPltLI96TFBlbc1
         d9IR7oYUoMveau9Xc9iw/IVyG3jmglG/Gz9KaoeeU/UUFWqAn83+Av1du80V861+Xp4h
         p4elBISDoYJ5DdcuH91MamAA/7p/mKdG0SV+1sjSvGnv14uHJmgpLrCv2A2BUJuqzKdr
         EIDUgnIjJOReqCwlB+N1v53U8NzzG5t23p7ZwOLJ8XykHbNpBlYmQY371HQ/+S7v9KkT
         6flQ==
X-Forwarded-Encrypted: i=1; AJvYcCWoO+Or0EJcPYin0QIF11fMn90zca8pXhAlXrNl6DboPZteFxCiDKywdBvkoQwJOLuDtKVloZ4UUpuWMITfGqaPn6It3lu/Wj0nPilPHIo=
X-Gm-Message-State: AOJu0YwMg/tJMwBCHiMRZgR7zRbBQYJqAVZmSQ2IL/UJl6XHhAJk/BI4
	bOvMeWb0Xtl2t//+HkrDn9FCbsUKoE+Fc8c2t5oNvlZS+KiPnFpESQsbZqC443A=
X-Google-Smtp-Source: AGHT+IGVAt0/tqEIqvf2BcI4SdfVjb+Y0HIov/oEetP2L1LIQfMJSwrQ1he4LxCcf/ns/cm/YbCLnQ==
X-Received: by 2002:ac2:5388:0:b0:512:e8eb:f978 with SMTP id g8-20020ac25388000000b00512e8ebf978mr4196530lfh.4.1708954612787;
        Mon, 26 Feb 2024 05:36:52 -0800 (PST)
Message-ID: <7365ac86-924c-4c9d-933f-87af6237de58@citrix.com>
Date: Mon, 26 Feb 2024 13:36:50 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86: Dom0 "broken ELF" reporting adjustments
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <7edea456-81ba-4e83-9441-0e82333ce168@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <7edea456-81ba-4e83-9441-0e82333ce168@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 22/01/2024 1:41 pm, Jan Beulich wrote:
> elf_load_binary() isn't the primary source of brokenness being
> indicated. Therefore make the respective PVH log message there
> conditional (much like PV has it), and add another instance when
> elf_xen_parse() failed (again matching behavior in the PV case).
>
> Make the PV side match the (new) use of %pd here.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 13:37:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 13:37:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685551.1066438 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebAg-0007L8-6H; Mon, 26 Feb 2024 13:37:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685551.1066438; Mon, 26 Feb 2024 13:37:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebAg-0007L1-3S; Mon, 26 Feb 2024 13:37:30 +0000
Received: by outflank-mailman (input) for mailman id 685551;
 Mon, 26 Feb 2024 13:37:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rebAe-0007Ki-Oz
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 13:37:28 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2e7fcd71-d4ac-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 14:37:27 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-512f7332252so1915147e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 05:37:27 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 bq34-20020a05620a46a200b007873c82f0easm2429708qkb.113.2024.02.26.05.37.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 05:37:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e7fcd71-d4ac-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708954647; x=1709559447; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6BIt+iE3foNAA53SQcPS2zvEaO/VpJHQbsJoTmUs+cI=;
        b=EbJ7w+/JGgUu1KeOFCV6MvO2PnUoBUZyTgvaj0VbR9MaoTD16z5SR5AJYmBlRinLTt
         zG5Nz4Kignzpwwx5WGFfvJidzjqVDKkMl1lexjs0DVyWHd273E/5axTAti8tWdE93QYs
         0m6NfChAKnJB9BeFKN4s1gtcI/RGLFwuFxRdU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708954647; x=1709559447;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6BIt+iE3foNAA53SQcPS2zvEaO/VpJHQbsJoTmUs+cI=;
        b=UVA392pgSI4e5ss1yQpjZQ3FPzh+KSyB/4UfRSeJNpydD/ZYQdLbmFoEFoIZ+Wa7+L
         HXm1tufjlc0wVO6JGUDJ1JZJykgj7quEK7w40fw7AaxhW2qt2bAkfhMKP2aOpmC/5TJi
         o3sRi+3v1lHmhv09Y8Md5U16HRZSG1k1i2bZ9TGWMHyF5jeNiqrXBX5a5m07F04wUgCF
         2n+bvNZlEjhXt8TyQDKiPnE+V7Qkf2/kjeqlVYwV0B2dUIsWdmU0uY7RjAVzPp2tEKH4
         6Aab1DYL1PIXjsRfl2f//SGTCX1UjWBQ1nFblonqJoXLRMAWE3yaKbvlGcRUG4vEcTi4
         b3Lw==
X-Forwarded-Encrypted: i=1; AJvYcCWOfySQ5/f9JEw5H7PqkDafvZlqainCLic83V7izvWCfpP2G5FiCt8z6ojjKXuHo0SxYJcNtIzQKOsvRk+GS6meT/XlQnVBqgClNKY9m2Y=
X-Gm-Message-State: AOJu0YyAzJ8cLc0mAVGkFqq6EMd7jBxPzCK7tHhu03Kfy1bQmI2sQ1xd
	PNoFpavAjaT34Sn3GlgY9IQV8I3K09qAE7JVyRMZAxoWAnnpXeY7NATYtVTexNw=
X-Google-Smtp-Source: AGHT+IFVYvLjon0QcWoxdORvXWFxHItbU1kIdVY7q+EJlYbOo4Gmu/QrkdsAJgwdAhyCCLRFRJuk/Q==
X-Received: by 2002:a05:6512:3e0c:b0:512:b0a7:2943 with SMTP id i12-20020a0565123e0c00b00512b0a72943mr5727371lfv.5.1708954647250;
        Mon, 26 Feb 2024 05:37:27 -0800 (PST)
Message-ID: <1d4b37b7-1afb-46ee-b845-af43b75ead61@citrix.com>
Date: Mon, 26 Feb 2024 13:37:26 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] x86/HVM: limit upcall vector related verbosity
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <89d77714-df09-4d0b-bc7d-7d773f8d4f4b@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <89d77714-df09-4d0b-bc7d-7d773f8d4f4b@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 18/12/2023 7:26 am, Jan Beulich wrote:
> Avoid logging all-identical messages for every vCPU, but make sure to
> log unusual events like the vector differing from vCPU 0's (note that
> the respective condition also makes sure vCPU 0 itself will have the
> vector setting logged), or it changing after it was once set. (Arguably
> a downside is that some vCPU not having its vector set would no longer
> be recognizable from the logs. But I think that's tolerable as
> sufficiently unlikely outside of people actively fiddling with related
> code.)
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:12:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:12:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685557.1066448 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebin-0005T1-3N; Mon, 26 Feb 2024 14:12:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685557.1066448; Mon, 26 Feb 2024 14:12:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebim-0005Su-W7; Mon, 26 Feb 2024 14:12:44 +0000
Received: by outflank-mailman (input) for mailman id 685557;
 Mon, 26 Feb 2024 14:12:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=w0W4=KD=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1rebim-0005SV-7x
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:12:44 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1b757002-d4b1-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 15:12:43 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a3e75e30d36so601344766b.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:12:43 -0800 (PST)
Received: from fhilly.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 k17-20020a17090666d100b00a3d8fb05c0csm2458644ejp.86.2024.02.26.06.12.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 06:12:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1b757002-d4b1-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708956762; x=1709561562; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=b8c4sqpQaxoDgLK2yQU0ZamZW0iKh8LR8huRc0FX160=;
        b=OL4fRUlpyYEIwPdFKnLnI61dxZtsoOMkpa7FejnhStv983tjos5/DSv1w10NcTIayt
         ia9lmNyXXLuy5474icXDpsZl7/Y4gQ/p9cOD84n5kKED77vBJ5D8AMufhmEGWR8DQgsO
         qvPYarMrVuqvQ9dfVLZapkhYu+kzVxhoE/caI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708956762; x=1709561562;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=b8c4sqpQaxoDgLK2yQU0ZamZW0iKh8LR8huRc0FX160=;
        b=A/cbqjWzNpQe4QjOwliaXtWBrOa4BroKDPi3b9skbUlrjjcbiXGsTP6cy9Hej38L0P
         QA8/rfYGk3QblvZJCn4/c2XkDsjuxKygx3qHhOOS1hmYQBCwOydT1MXuq6F22DImx2G8
         Z4+Vz/2KcLem6gyctTEeaKdUvlRSp24pKLA84EyUygkX/UjUWX2LDRxFQtHIU0tEugRb
         rX2/M1d1OLTV3BlKMHbzmKDK/5c21uFW4ZJxSBs1XeeZ/t8GRzOLPaow6gezfMlrINd9
         Y/DQ8WHaeGI9osDaMURgH275GDLtapDkGnDJ7J/z+0xDlsteVjRXy3SH2GWCJTN602LG
         lyVw==
X-Gm-Message-State: AOJu0YwR+E7SCbIJLnkDbPILwNpdYW5DJNs09SJ+SPVwY6dLyQSvw7d6
	LaFV5E1QG3g59tOby/IYGnfThNgVbKVVjCWoWOP8i2fn1S0JPVBQDk3zFEv2ZUDHgtER3gNanah
	8i8M=
X-Google-Smtp-Source: AGHT+IGXjCUcJfbNVmjJ/5WG9SVvIEYk1LGKAQL/Ld2vUcAzRwRKuy5tBbixYXjQPZ5UVYNjeDYCiQ==
X-Received: by 2002:a17:906:12d0:b0:a43:3066:ef13 with SMTP id l16-20020a17090612d000b00a433066ef13mr2834613ejb.37.1708956762411;
        Mon, 26 Feb 2024 06:12:42 -0800 (PST)
From: Fouad Hilly <fouad.hilly@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Fouad Hilly <fouad.hilly@cloud.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH] tools/xentop: Add VBD3 support to xentop
Date: Mon, 26 Feb 2024 14:12:11 +0000
Message-ID: <20240226141211.1416866-1-fouad.hilly@cloud.com>
X-Mailer: git-send-email 2.42.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Pritha Srivastava <pritha.srivastava@citrix.com>

xl now knows how to drive tapdisk, so modified libxenstat to
understand vbd3 statistics.

Signed-off-by: Jorge Martin <jorge.martin@citrix.com>
Signed-off-by: Pritha Srivastava <pritha.srivastava@citrix.com>
Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Juergen Gross <jgross@suse.com>
---
 tools/libs/stat/xenstat_linux.c | 69 ++++++++++++++++++++++++++++++++-
 tools/libs/stat/xenstat_priv.h  | 16 ++++++++
 tools/xentop/xentop.c           |  1 +
 3 files changed, 85 insertions(+), 1 deletion(-)

diff --git a/tools/libs/stat/xenstat_linux.c b/tools/libs/stat/xenstat_linux.c
index cbba54aa83ee..5a4a03634182 100644
--- a/tools/libs/stat/xenstat_linux.c
+++ b/tools/libs/stat/xenstat_linux.c
@@ -390,6 +390,39 @@ void xenstat_uninit_networks(xenstat_handle * handle)
 		fclose(priv->procnetdev);
 }
 
+static int read_attributes_vbd3(char *vbd3_path, xenstat_vbd *vbd)
+{
+	FILE *fp;
+	struct vbd3_stats vbd3_stats;
+
+	fp = fopen(vbd3_path, "rb");
+
+	if (fp == NULL)
+	{
+		return -1;
+	}
+
+	if (fread(&vbd3_stats, sizeof(struct vbd3_stats), 1, fp) != 1) {
+		fclose(fp);
+		return -1;
+	}
+
+	if (vbd3_stats.version != 1) {
+		fclose(fp);
+		return -1;
+	}
+
+	vbd->oo_reqs = vbd3_stats.oo_reqs;
+	vbd->rd_reqs = vbd3_stats.read_reqs_submitted;
+	vbd->rd_sects = vbd3_stats.read_sectors;
+	vbd->wr_reqs = vbd3_stats.write_reqs_submitted;
+	vbd->wr_sects = vbd3_stats.write_sectors;
+
+	fclose(fp);
+
+	return 0;
+}
+
 static int read_attributes_vbd(const char *vbd_directory, const char *what, char *ret, int cap)
 {
 	static char file_name[80];
@@ -438,7 +471,7 @@ int xenstat_collect_vbds(xenstat_node * node)
 		int ret;
 		char buf[256];
 
-		ret = sscanf(dp->d_name, "%3s-%u-%u", buf, &domid, &vbd.dev);
+		ret = sscanf(dp->d_name, "%255[^-]-%u-%u", buf, &domid, &vbd.dev);
 		if (ret != 3)
 			continue;
 		if (!(strstr(buf, "vbd")) && !(strstr(buf, "tap")))
@@ -448,6 +481,8 @@ int xenstat_collect_vbds(xenstat_node * node)
 			vbd.back_type = 1;
 		else if (strcmp(buf,"tap") == 0)
 			vbd.back_type = 2;
+		else if (strcmp(buf,"vbd3") == 0)
+			vbd.back_type = 3;
 		else
 			vbd.back_type = 0;
 
@@ -479,6 +514,38 @@ int xenstat_collect_vbds(xenstat_node * node)
 				vbd.error = 1;
 			}
 		}
+		else if (vbd.back_type == 3)
+		{
+
+			char *td3_pid;
+			char *path;
+
+			vbd.back_type = 3;
+			vbd.error = 0;
+
+			if (asprintf(&path, "/local/domain/0/backend/vbd3/%u/%u/kthread-pid", domid, vbd.dev) < 0)
+				continue;
+
+			td3_pid = xs_read(node->handle->xshandle, XBT_NULL, path, NULL);
+
+			if (td3_pid == NULL) {
+				free(path);
+				continue;
+			}
+
+			free(path);
+
+			if (asprintf(&path, "/dev/shm/td3-%s/vbd-%u-%u", td3_pid, domid, vbd.dev) < 0) {
+				free(td3_pid);
+				continue;
+			}
+
+			if (read_attributes_vbd3(path, &vbd) < 0)
+				vbd.error = 1;
+
+			free(td3_pid);
+			free(path);
+		}
 		else
 		{
 			vbd.error = 1;
diff --git a/tools/libs/stat/xenstat_priv.h b/tools/libs/stat/xenstat_priv.h
index 4eb44a8ebb84..c3a9635240e9 100644
--- a/tools/libs/stat/xenstat_priv.h
+++ b/tools/libs/stat/xenstat_priv.h
@@ -98,6 +98,22 @@ struct xenstat_vbd {
 	unsigned long long wr_sects;
 };
 
+struct vbd3_stats {
+	uint32_t version;
+	uint32_t __pad;
+	uint64_t oo_reqs;
+	uint64_t read_reqs_submitted;
+	uint64_t read_reqs_completed;
+	uint64_t read_sectors;
+	uint64_t read_total_ticks;
+	uint64_t write_reqs_submitted;
+	uint64_t write_reqs_completed;
+	uint64_t write_sectors;
+	uint64_t write_total_ticks;
+	uint64_t io_errors;
+	uint64_t flags;
+};
+
 extern int xenstat_collect_networks(xenstat_node * node);
 extern void xenstat_uninit_networks(xenstat_handle * handle);
 extern int xenstat_collect_vbds(xenstat_node * node);
diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
index 0a2fab7f15a3..f5a456fd4dfd 100644
--- a/tools/xentop/xentop.c
+++ b/tools/xentop/xentop.c
@@ -1124,6 +1124,7 @@ void do_vbd(xenstat_domain *domain)
 		"Unidentified",           /* number 0 */
 		"BlkBack",           /* number 1 */
 		"BlkTap",            /* number 2 */
+		"Tapdisk3"           /* number 3 */
 	};
 
 	num_vbds = xenstat_domain_num_vbds(domain);
-- 
2.42.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:20:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:20:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685561.1066458 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebq7-0007Zr-TY; Mon, 26 Feb 2024 14:20:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685561.1066458; Mon, 26 Feb 2024 14:20:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebq7-0007Zk-Ql; Mon, 26 Feb 2024 14:20:19 +0000
Received: by outflank-mailman (input) for mailman id 685561;
 Mon, 26 Feb 2024 14:20:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rebq6-0007YQ-Bg
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:20:18 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 294c1257-d4b2-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 15:20:15 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a26ed1e05c7so502497466b.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:20:15 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 lu2-20020a170906fac200b00a3d1c0a3d5dsm2480455ejb.63.2024.02.26.06.20.14
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 06:20:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 294c1257-d4b2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708957215; x=1709562015; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=qhOeWIZKnxkG91hGdjdz+br2BrR2hSoGpLpqsWeiUvg=;
        b=XgBtzANJ5uE3f4irkFrJXlkPFpov4LIOdAaJGaKsddzgeBDiPCOYE9nZwrTetbRcsY
         gvmHKSWsOBGApaNJJXteLFaYXmtCAIL+a8f05hPb5p4HxdlDudL1QoMM6Wi4aF8dzYnm
         0B+gi2fFno0QYjVQZW8DR6dDDviUfDmVXKJ2chSqBsmerNkcMiOgtQLjJTwy8n92Kwcb
         b4AW0jSr/3SJwxjgz8VVq95oDKmOVq1dUx0SQYKDUtECmwy6iXTkXtBgdxDfqkwLMgJb
         liBrCng3BECx+nysTcWJ8FmoJ629F3DSeEr2Q/DAeyjXkRkJlhOhoeZbHe9MF0qqOBOS
         gZyA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708957215; x=1709562015;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qhOeWIZKnxkG91hGdjdz+br2BrR2hSoGpLpqsWeiUvg=;
        b=T6npQxvPo5f9pfEh2XECI4nu/bCci7fdO7q6IUmuACTmbbED3qXE6nwvsiqCCJA1Bn
         xcyxT2vikrkpRbkqDfj71pdxuko3n0lcFTsY1ShriQ8spyUTx23iuJjHWUPJTODRVpdv
         2lsom8aTWL4V25T60uq1oRhPzIb67KUSLnPWQdFkt6ZVLvnaiS89AeF1vgOwaREUjMop
         A599WrWDx0dKevXPyPm3cfvqrXXukB5aJbZtMHzvxDORrn6aHs3KpRziMu8XQGvmdxTI
         1ufLBxf5NPRiqnFiY/zjgDlBIqJXbw6OOKEO613G18mv+rXm8zthY+XyDgMj5k3O8IFW
         VbDg==
X-Forwarded-Encrypted: i=1; AJvYcCXSXtMRQsug0pF5He1n10qk4zwoy/WM+8+nWscQxCw/3kRiXf53qsSxUToh30uepfDzKqBjDha4ZigW9avGymBtqpAmueytoifAZ62ALpA=
X-Gm-Message-State: AOJu0YyLh3rFwLrptE/00ickRZUVGFJ7g6tfzatMMU34FfpabKara4/u
	Gu6ARXtMXieN82TLSag0oRTi9kzBbt79Ys/Z3NoNf79UvvoLa5NS+pw/8f/4MA==
X-Google-Smtp-Source: AGHT+IG0gJ7ZFtUME12dZVyQhDkSa2kpsdrg0sqieBQCFDtppOQtaNZgCxfMXzYGYYWaJjHHTxNtvw==
X-Received: by 2002:a17:907:78d7:b0:a43:74fa:4d06 with SMTP id kv23-20020a17090778d700b00a4374fa4d06mr972956ejc.60.1708957215359;
        Mon, 26 Feb 2024 06:20:15 -0800 (PST)
Message-ID: <4f59127c-6a98-4672-9202-4d48817a1931@suse.com>
Date: Mon, 26 Feb 2024 15:20:14 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
 <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
 <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
 <1c53b52ee39161a8f59209d28af69fe997479dbc.camel@gmail.com>
 <3b38fe82-ee0f-4666-93e1-bd78fe69c534@suse.com>
 <5d2d35fe014094c991363e42c3e2ad9ca2af3938.camel@gmail.com>
 <44fd5092-7838-4d28-804b-bbfebfd44886@suse.com>
 <56ae27d003b8763af34864ae56433691685c3661.camel@gmail.com>
 <7cab68d1-9bdf-4ea1-b49a-cce1e4af4692@suse.com>
 <4215fb9eb5be5898b3cc9c2977e55ac0e7270b4d.camel@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4215fb9eb5be5898b3cc9c2977e55ac0e7270b4d.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26.02.2024 13:58, Oleksii wrote:
> On Mon, 2024-02-26 at 12:28 +0100, Jan Beulich wrote:
>> On 26.02.2024 12:18, Oleksii wrote:
>>> On Mon, 2024-02-26 at 10:45 +0100, Jan Beulich wrote:
>>>> On 23.02.2024 13:23, Oleksii wrote:
>>>>>>
>>>>>>>>> As 1- and 2-byte cases are emulated I decided that is
>>>>>>>>> not
>>>>>>>>> to
>>>>>>>>> provide
>>>>>>>>> sfx argument for emulation macros as it will not have
>>>>>>>>> to
>>>>>>>>> much
>>>>>>>>> affect on
>>>>>>>>> emulated types and just consume more performance on
>>>>>>>>> acquire
>>>>>>>>> and
>>>>>>>>> release
>>>>>>>>> version of sc/ld instructions.
>>>>>>>>
>>>>>>>> Question is whether the common case (4- and 8-byte
>>>>>>>> accesses)
>>>>>>>> shouldn't
>>>>>>>> be valued higher, with 1- and 2-byte emulation being
>>>>>>>> there
>>>>>>>> just
>>>>>>>> to
>>>>>>>> allow things to not break altogether.
>>>>>>> If I understand you correctly, it would make sense to add
>>>>>>> the
>>>>>>> 'sfx'
>>>>>>> argument for the 1/2-byte access case, ensuring that all
>>>>>>> options
>>>>>>> are
>>>>>>> available for 1/2-byte access case as well.
>>>>>>
>>>>>> That's one of the possibilities. As said, I'm not overly
>>>>>> worried
>>>>>> about
>>>>>> the emulated cases. For the initial implementation I'd
>>>>>> recommend
>>>>>> going
>>>>>> with what is easiest there, yielding the best possible result
>>>>>> for
>>>>>> the
>>>>>> 4- and 8-byte cases. If later it turns out repeated
>>>>>> acquire/release
>>>>>> accesses are a problem in the emulation loop, things can be
>>>>>> changed
>>>>>> to explicit barriers, without touching the 4- and 8-byte
>>>>>> cases.
>>>>> I am confused then a little bit if emulated case is not an
>>>>> issue.
>>>>>
>>>>> For 4- and 8-byte cases for xchg .aqrl is used, for relaxed and
>>>>> aqcuire
>>>>> version of xchg barries are used.
>>>>>
>>>>> The similar is done for cmpxchg.
>>>>>
>>>>> If something will be needed to change in emulation loop it
>>>>> won't
>>>>> require to change 4- and 8-byte cases.
>>>>
>>>> I'm afraid I don't understand your reply.
>>> IIUC, emulated cases it is implemented correctly in terms of usage
>>> barriers. And it also OK not to use sfx for lr/sc instructions and
>>> use
>>> only barriers.
>>>
>>> For 4- and 8-byte cases are used sfx + barrier depending on the
>>> specific case ( relaxed, acquire, release, generic xchg/cmpxchg ).
>>> What also looks to me correct. But you suggested to provide the
>>> best
>>> possible result for 4- and 8-byte cases. 
>>>
>>> So I don't understand what the best possible result is as the
>>> current
>>> one usage of __{cmp}xchg_generic for each specific case  ( relaxed,
>>> acquire, release, generic xchg/cmpxchg ) looks correct to me:
>>> xchg -> (..., ".aqrl", "", "") just suffix .aqrl suffix without
>>> barriers.
>>> xchg_release -> (..., "", RISCV_RELEASE_BARRIER, "" ) use only
>>> release
>>> barrier
>>> xchg_acquire -> (..., "", "", RISCV_ACQUIRE_BARRIER ), only acquire
>>> barrier
>>> xchg_relaxed ->  (..., "", "", "") - no barries, no sfx
>>
>> So first: While explicit barriers are technically okay, I don't
>> follow why
>> you insist on using them when you can achieve the same by suitably
>> tagging
>> the actual insn doing the exchange. Then second: It's somewhat hard
>> for me
>> to see the final effect on the emulation paths without you actually
>> having
>> done the switch. Maybe no special handling is necessary there anymore
>> then. And as said, it may actually be acceptable for the emulation
>> paths
>> to "only" be correct, but not be ideal in terms of performance. After
>> all,
>> if you use the normal 4-byte primitive in there, more (non-explicit)
>> barriers than needed would occur if the involved loop has to take
>> more
>> than one iteration. Which could (but imo doesn't need to be) avoided
>> by
>> using a more relaxed 4-byte primitive there and an explicit barrier
>> outside of the loop.
> 
> According to the spec:
> Table A.5 ( part of the table only I copied here )
> 
> Linux Construct          RVWMO Mapping
> atomic <op> relaxed           amo<op>.{w|d}
> atomic <op> acquire           amo<op>.{w|d}.aq
> atomic <op> release           amo<op>.{w|d}.rl
> atomic <op>                   amo<op>.{w|d}.aqrl
> 
> Linux Construct          RVWMO LR/SC Mapping
> atomic <op> relaxed       loop: lr.{w|d}; <op>; sc.{w|d}; bnez loop
> atomic <op> acquire       loop: lr.{w|d}.aq; <op>; sc.{w|d}; bnez loop
> atomic <op> release       loop: lr.{w|d}; <op>; sc.{w|d}.aqrl∗ ; bnez 
> loop OR
>                           fence.tso; loop: lr.{w|d}; <op>; sc.{w|d}∗ ;
> bnez loop
> atomic <op>               loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez
> loop

In your consideration what to implement you will want to limit
things to constructs we actually use. I can't find any use of the
relaxed, acquire, or release forms of atomics as mentioned above.

> The Linux mappings for release operations may seem stronger than
> necessary, but these mappings
> are needed to cover some cases in which Linux requires stronger
> orderings than the more intuitive
> mappings would provide. In particular, as of the time this text is
> being written, Linux is actively
> debating whether to require load-load, load-store, and store-store
> orderings between accesses in one
> critical section and accesses in a subsequent critical section in the
> same hart and protected by the
> same synchronization object. Not all combinations of FENCE RW,W/FENCE
> R,RW mappings
> with aq/rl mappings combine to provide such orderings. There are a few
> ways around this problem,
> including:
> 1. Always use FENCE RW,W/FENCE R,RW, and never use aq/rl. This suffices
> but is undesir-
> able, as it defeats the purpose of the aq/rl modifiers.
> 2. Always use aq/rl, and never use FENCE RW,W/FENCE R,RW. This does not
> currently work
> due to the lack of load and store opcodes with aq and rl modifiers.

I don't understand this point: Which specific load and/or store forms
are missing? According to my reading of the A extension spec all
combination of aq/rl exist with both lr and sc.

> 3. Strengthen the mappings of release operations such that they would
> enforce sufficient order-
> ings in the presence of either type of acquire mapping. This is the
> currently-recommended
> solution, and the one shown in Table A.5.
> 
> 
> Based on this it is enough in our case use only suffixed istructions
> (amo<op>.{w|d}{.aq, .rl, .aqrl, .aqrl }, lr.{w|d}.{.aq, .aqrl }.
> 
> 
> But as far as I understand in Linux atomics were strengthen with
> fences:
>     Atomics present the same issue with locking: release and acquire
>     variants need to be strengthened to meet the constraints defined
>     by the Linux-kernel memory consistency model [1].
>     
>     Atomics present a further issue: implementations of atomics such
>     as atomic_cmpxchg() and atomic_add_unless() rely on LR/SC pairs,
>     which do not give full-ordering with .aqrl; for example, current
>     implementations allow the "lr-sc-aqrl-pair-vs-full-barrier" test
>     below to end up with the state indicated in the "exists" clause.
>     
>     In order to "synchronize" LKMM and RISC-V's implementation, this
>     commit strengthens the implementations of the atomics operations
>     by replacing .rl and .aq with the use of ("lightweigth") fences,
>     and by replacing .aqrl LR/SC pairs in sequences such as:
>     
>       0:      lr.w.aqrl  %0, %addr
>               bne        %0, %old, 1f
>               ...
>               sc.w.aqrl  %1, %new, %addr
>               bnez       %1, 0b
>       1:
>     
>     with sequences of the form:
>     
>       0:      lr.w       %0, %addr
>               bne        %0, %old, 1f
>               ...
>               sc.w.rl    %1, %new, %addr   /* SC-release   */
>               bnez       %1, 0b
>               fence      rw, rw            /* "full" fence */
>       1:
>     
>     following Daniel's suggestion.

I'm likely missing something, yet as it looks it does help that the
code fragment above appears to be ...

>     These modifications were validated with simulation of the RISC-V
>     with sequences of the form:
>     
>       0:      lr.w       %0, %addr
>               bne        %0, %old, 1f
>               ...
>               sc.w.rl    %1, %new, %addr   /* SC-release   */
>               bnez       %1, 0b
>               fence      rw, rw            /* "full" fence */
>       1:
>     
>     following Daniel's suggestion.

... entirely the same as this one. Yet there's presumably a reason
for quoting it twice?

>     These modifications were validated with simulation of the RISC-V
>     memory consistency model.
>     
>     C lr-sc-aqrl-pair-vs-full-barrier
>     
>     {}
>     
>     P0(int *x, int *y, atomic_t *u)
>     {
>             int r0;
>             int r1;
>     
>             WRITE_ONCE(*x, 1);
>             r0 = atomic_cmpxchg(u, 0, 1);
>             r1 = READ_ONCE(*y);
>     }
>     
>     P1(int *x, int *y, atomic_t *v)
>     {
>             int r0;
>             int r1;
>     
>             WRITE_ONCE(*y, 1);
>             r0 = atomic_cmpxchg(v, 0, 1);
>             r1 = READ_ONCE(*x);
>     }
>     
>     exists (u=1 /\ v=1 /\ 0:r1=0 /\ 1:r1=0)
>     
>     [1] https://marc.info/?l=linux-kernel&m=151930201102853&w=2
>      
> https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/hKywNHBkAXM
>         https://marc.info/?l=linux-kernel&m=151633436614259&w=2
> 
> 
> Thereby Linux kernel implementation seems to me more safe and it is a
> reason why I want/wanted to be aligned with it.

Which may end up being okay. I hope you realize though that there's a
lot more explanation needed in the respective commits then compared to
what you've had so far. As a minimum, absolutely anything remotely
unexpected needs to be explained.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:22:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:22:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685564.1066468 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebsV-00086I-97; Mon, 26 Feb 2024 14:22:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685564.1066468; Mon, 26 Feb 2024 14:22:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebsV-00086B-5k; Mon, 26 Feb 2024 14:22:47 +0000
Received: by outflank-mailman (input) for mailman id 685564;
 Mon, 26 Feb 2024 14:22:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rebsT-00085k-89
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:22:45 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 810fbf38-d4b2-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 15:22:43 +0100 (CET)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-565a3910f86so2917582a12.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:22:43 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ds2-20020a0564021cc200b005644221a3desm2427671edb.3.2024.02.26.06.22.41
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 06:22:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 810fbf38-d4b2-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708957362; x=1709562162; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SzmLMSoqyq0ghO6dm13h0DHUVXokODno9z9BT+KWu24=;
        b=NWuv7ZLS/Dj6rPaBBRdqjUn6s2wNs/wbEyA70qshRfCRsXLfMwnc11Smh1VjtSYgYM
         fRgoL5dVHYN8lMrkd8m+UirUL8Nf6tZ1qHvFx5T7yJAZ9+AovsjQfiGnMMj+Eeta+Hu0
         iECqRrzw3/gTXHBIItYaqR6s/tQc6cwJZnn65mpeJh5JJAcwPQ/862IFssEYnasADYGo
         vCIoqNPHysffUe3QRHIMV9KoiNsb7tMSH8IACy9in6RU9UExVK7m2jcCxwL7IFinHDjA
         1qK7iV0C8IONLuPxAR670NEI3Bv3tIvMZhP0s5LRUO9EY1oFSN/KuK6tNZRNEe/n1fWJ
         ybIA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708957363; x=1709562163;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SzmLMSoqyq0ghO6dm13h0DHUVXokODno9z9BT+KWu24=;
        b=uezvyr1SdrAghWV2u1PkkXZGVg7tB6ZPGSjjmN9ZRS7uCZHSxMFBw+jf4xg82zmUR1
         KsecO9U1weFg0CuYP/Oy/LAZtoAZBRUFgRP1BlFrFNDglLoqqOknyHp/sH9+sF74nnNU
         RtLsbVDY1MUvy/ql7wuRgTzbIGF8/AQ6tjR+npNvgiVLUPPUrI9HWZHsA5iIFSFt/DvR
         ip0CZ9DDc+XLPPMh0cTVkspGjPn3hJ87bpGiAhPARlW14EainVMWNLfTH59cqu7qReMa
         lGtPaKxkmdV2rAaTBBjb8zNQlvRJKAUzD/iQt/14WgYm0SJlN+3MoIgQUkINgwNyq3KC
         xXFQ==
X-Forwarded-Encrypted: i=1; AJvYcCVXnN5PVW+yP1SjSfsGCyGKcyRXdK5lZrAP3zKns1PFMbiyFYjstI1U+IiDrBgvqe16huZu9jQjRh0rWuOxCg2IFo9LCgz+yoc+z2wJDt8=
X-Gm-Message-State: AOJu0YwVwLxj8i4gZB7zx7QZiyTPSIlH4Ndp0U9zEM+qDQG5SuH/DTeL
	6cdCig86lq4sdgC0xdo5aelXE8vVyZcgqNPy8yjdpmgwMTPS+Jydx7/F3g3EWQ==
X-Google-Smtp-Source: AGHT+IEgIv8Pa4H43yAeQkzvGskGLPhUvNljA//VLZFovPGlViWWCnn1SbbmhJokUAjRpaLIp9+XuA==
X-Received: by 2002:aa7:d5d5:0:b0:564:3392:e9db with SMTP id d21-20020aa7d5d5000000b005643392e9dbmr4353551eds.33.1708957362611;
        Mon, 26 Feb 2024 06:22:42 -0800 (PST)
Message-ID: <9468ccd8-44c3-4b50-b200-5a4ae1cca8ff@suse.com>
Date: Mon, 26 Feb 2024 15:22:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/xentop: Add VBD3 support to xentop
Content-Language: en-US
To: Fouad Hilly <fouad.hilly@cloud.com>
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240226141211.1416866-1-fouad.hilly@cloud.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226141211.1416866-1-fouad.hilly@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 15:12, Fouad Hilly wrote:
> From: Pritha Srivastava <pritha.srivastava@citrix.com>
> 
> xl now knows how to drive tapdisk, so modified libxenstat to
> understand vbd3 statistics.
> 
> Signed-off-by: Jorge Martin <jorge.martin@citrix.com>
> Signed-off-by: Pritha Srivastava <pritha.srivastava@citrix.com>

Just a formal question (I'm not really qualified to review this change):
With the above two S-o-b and the earlier From: - who is the original
author of this patch? The general expectation is for the 1st S-o-b to
be the author's.

Jan

> Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
> ---
> CC: Wei Liu <wl@xen.org>
> CC: Anthony PERARD <anthony.perard@citrix.com>
> CC: Juergen Gross <jgross@suse.com>
> ---
>  tools/libs/stat/xenstat_linux.c | 69 ++++++++++++++++++++++++++++++++-
>  tools/libs/stat/xenstat_priv.h  | 16 ++++++++
>  tools/xentop/xentop.c           |  1 +
>  3 files changed, 85 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/libs/stat/xenstat_linux.c b/tools/libs/stat/xenstat_linux.c
> index cbba54aa83ee..5a4a03634182 100644
> --- a/tools/libs/stat/xenstat_linux.c
> +++ b/tools/libs/stat/xenstat_linux.c
> @@ -390,6 +390,39 @@ void xenstat_uninit_networks(xenstat_handle * handle)
>  		fclose(priv->procnetdev);
>  }
>  
> +static int read_attributes_vbd3(char *vbd3_path, xenstat_vbd *vbd)
> +{
> +	FILE *fp;
> +	struct vbd3_stats vbd3_stats;
> +
> +	fp = fopen(vbd3_path, "rb");
> +
> +	if (fp == NULL)
> +	{
> +		return -1;
> +	}
> +
> +	if (fread(&vbd3_stats, sizeof(struct vbd3_stats), 1, fp) != 1) {
> +		fclose(fp);
> +		return -1;
> +	}
> +
> +	if (vbd3_stats.version != 1) {
> +		fclose(fp);
> +		return -1;
> +	}
> +
> +	vbd->oo_reqs = vbd3_stats.oo_reqs;
> +	vbd->rd_reqs = vbd3_stats.read_reqs_submitted;
> +	vbd->rd_sects = vbd3_stats.read_sectors;
> +	vbd->wr_reqs = vbd3_stats.write_reqs_submitted;
> +	vbd->wr_sects = vbd3_stats.write_sectors;
> +
> +	fclose(fp);
> +
> +	return 0;
> +}
> +
>  static int read_attributes_vbd(const char *vbd_directory, const char *what, char *ret, int cap)
>  {
>  	static char file_name[80];
> @@ -438,7 +471,7 @@ int xenstat_collect_vbds(xenstat_node * node)
>  		int ret;
>  		char buf[256];
>  
> -		ret = sscanf(dp->d_name, "%3s-%u-%u", buf, &domid, &vbd.dev);
> +		ret = sscanf(dp->d_name, "%255[^-]-%u-%u", buf, &domid, &vbd.dev);
>  		if (ret != 3)
>  			continue;
>  		if (!(strstr(buf, "vbd")) && !(strstr(buf, "tap")))
> @@ -448,6 +481,8 @@ int xenstat_collect_vbds(xenstat_node * node)
>  			vbd.back_type = 1;
>  		else if (strcmp(buf,"tap") == 0)
>  			vbd.back_type = 2;
> +		else if (strcmp(buf,"vbd3") == 0)
> +			vbd.back_type = 3;
>  		else
>  			vbd.back_type = 0;
>  
> @@ -479,6 +514,38 @@ int xenstat_collect_vbds(xenstat_node * node)
>  				vbd.error = 1;
>  			}
>  		}
> +		else if (vbd.back_type == 3)
> +		{
> +
> +			char *td3_pid;
> +			char *path;
> +
> +			vbd.back_type = 3;
> +			vbd.error = 0;
> +
> +			if (asprintf(&path, "/local/domain/0/backend/vbd3/%u/%u/kthread-pid", domid, vbd.dev) < 0)
> +				continue;
> +
> +			td3_pid = xs_read(node->handle->xshandle, XBT_NULL, path, NULL);
> +
> +			if (td3_pid == NULL) {
> +				free(path);
> +				continue;
> +			}
> +
> +			free(path);
> +
> +			if (asprintf(&path, "/dev/shm/td3-%s/vbd-%u-%u", td3_pid, domid, vbd.dev) < 0) {
> +				free(td3_pid);
> +				continue;
> +			}
> +
> +			if (read_attributes_vbd3(path, &vbd) < 0)
> +				vbd.error = 1;
> +
> +			free(td3_pid);
> +			free(path);
> +		}
>  		else
>  		{
>  			vbd.error = 1;
> diff --git a/tools/libs/stat/xenstat_priv.h b/tools/libs/stat/xenstat_priv.h
> index 4eb44a8ebb84..c3a9635240e9 100644
> --- a/tools/libs/stat/xenstat_priv.h
> +++ b/tools/libs/stat/xenstat_priv.h
> @@ -98,6 +98,22 @@ struct xenstat_vbd {
>  	unsigned long long wr_sects;
>  };
>  
> +struct vbd3_stats {
> +	uint32_t version;
> +	uint32_t __pad;
> +	uint64_t oo_reqs;
> +	uint64_t read_reqs_submitted;
> +	uint64_t read_reqs_completed;
> +	uint64_t read_sectors;
> +	uint64_t read_total_ticks;
> +	uint64_t write_reqs_submitted;
> +	uint64_t write_reqs_completed;
> +	uint64_t write_sectors;
> +	uint64_t write_total_ticks;
> +	uint64_t io_errors;
> +	uint64_t flags;
> +};
> +
>  extern int xenstat_collect_networks(xenstat_node * node);
>  extern void xenstat_uninit_networks(xenstat_handle * handle);
>  extern int xenstat_collect_vbds(xenstat_node * node);
> diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
> index 0a2fab7f15a3..f5a456fd4dfd 100644
> --- a/tools/xentop/xentop.c
> +++ b/tools/xentop/xentop.c
> @@ -1124,6 +1124,7 @@ void do_vbd(xenstat_domain *domain)
>  		"Unidentified",           /* number 0 */
>  		"BlkBack",           /* number 1 */
>  		"BlkTap",            /* number 2 */
> +		"Tapdisk3"           /* number 3 */
>  	};
>  
>  	num_vbds = xenstat_domain_num_vbds(domain);



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:25:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:25:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685566.1066478 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebur-0000Eo-Jq; Mon, 26 Feb 2024 14:25:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685566.1066478; Mon, 26 Feb 2024 14:25:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebur-0000Eh-H3; Mon, 26 Feb 2024 14:25:13 +0000
Received: by outflank-mailman (input) for mailman id 685566;
 Mon, 26 Feb 2024 14:25:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=x/J1=KD=cloud.com=frediano.ziglio@srs-se1.protection.inumbo.net>)
 id 1rebup-0000EW-Db
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:25:11 +0000
Received: from mail-oa1-x2a.google.com (mail-oa1-x2a.google.com
 [2001:4860:4864:20::2a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d8716594-d4b2-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 15:25:10 +0100 (CET)
Received: by mail-oa1-x2a.google.com with SMTP id
 586e51a60fabf-204235d0913so2289351fac.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:25:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8716594-d4b2-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708957509; x=1709562309; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=bzfpkFb4+SZoPluOOFxi152Tjd1q75L1p2cjJRfljzE=;
        b=TFK9aOIgH9kL1PPY6EPf5eNpTp08j8tQA0NHHI+xHzYDjxiUNsiDuUS3g9Y3jUd29y
         ziqJhu03FoQTvDDJ8KnlfV6Qu8LOPJ5oAT1f2dIkb7NUAjPQzcXng42kK7Jk36CxshLC
         g2LVYRPRSWfr5bMcYLsE4PvfcdM5Al4eWEB1c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708957509; x=1709562309;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=bzfpkFb4+SZoPluOOFxi152Tjd1q75L1p2cjJRfljzE=;
        b=epFKrM5YSVJtTZz8ldyUwBUDuzK/DI/SpWqUuD26Q6yv0Qx5u9ixXtDuJovXVgOwPf
         Rdak1hOhReb021EmUx64ldikpAq97AmW03VDOeJQpTsjX/WrMT87+K7pzqdgbxVBapKt
         fCm4IQPoWNmdLSxGMOPe20OW7ARJVFjet9fZrciF9J6YuExAvTtRMMn5XX7JzvdesTUl
         Wa6eMMfZbKZ3BFE4XYbpkTxi1PAh2gDFPVpajhG6J1FNGxZ70sglAHsVwjNpMrTSUWrs
         Rgqri0m/2sbqc0rSaCm3xzu4/MD2rGuFFo29tOBjOkMj5CNpzz5gSB3hrK2P/WtT6k4d
         O5+Q==
X-Gm-Message-State: AOJu0YzAe1g2C1YEeoYtxwfd4+e+a5UFJz1SAT1TVTTCwOCCEDiqkte6
	cRiyLHPja/mO6R0OgJ8OqV3VwDqAbC2MEsFqOoUwb8VBoPnzQHB+oOUX1S0Ryw6x26bwEmJDL43
	WnPe0MUbcrPmrQa3v6TeNOPI1SsQ1Vm0KIJoc0w==
X-Google-Smtp-Source: AGHT+IFQLS0Ii3kv0wfg5KnLFK0KwTSnvQVdriSoXE56bvCemxHLlBKffQl+yYSM/qX5M+27ENlO1BVg4ECoiLiJ2+k=
X-Received: by 2002:a05:6870:3450:b0:21e:7e5d:4fde with SMTP id
 i16-20020a056870345000b0021e7e5d4fdemr8047989oah.52.1708957509013; Mon, 26
 Feb 2024 06:25:09 -0800 (PST)
MIME-Version: 1.0
References: <20240226141211.1416866-1-fouad.hilly@cloud.com>
In-Reply-To: <20240226141211.1416866-1-fouad.hilly@cloud.com>
From: Frediano Ziglio <frediano.ziglio@cloud.com>
Date: Mon, 26 Feb 2024 14:24:51 +0000
Message-ID: <CACHz=Zi=qvEq9seMrTq0p-oiLTx_fzNCPtsaWXokfFdep8pxYQ@mail.gmail.com>
Subject: Re: [PATCH] tools/xentop: Add VBD3 support to xentop
To: Fouad Hilly <fouad.hilly@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, Wei Liu <wl@xen.org>, 
	Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross <jgross@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, Feb 26, 2024 at 2:12=E2=80=AFPM Fouad Hilly <fouad.hilly@cloud.com>=
 wrote:
>
> From: Pritha Srivastava <pritha.srivastava@citrix.com>
>
> xl now knows how to drive tapdisk, so modified libxenstat to
> understand vbd3 statistics.
>
> Signed-off-by: Jorge Martin <jorge.martin@citrix.com>
> Signed-off-by: Pritha Srivastava <pritha.srivastava@citrix.com>
> Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
> ---
> CC: Wei Liu <wl@xen.org>
> CC: Anthony PERARD <anthony.perard@citrix.com>
> CC: Juergen Gross <jgross@suse.com>
> ---
>  tools/libs/stat/xenstat_linux.c | 69 ++++++++++++++++++++++++++++++++-
>  tools/libs/stat/xenstat_priv.h  | 16 ++++++++
>  tools/xentop/xentop.c           |  1 +
>  3 files changed, 85 insertions(+), 1 deletion(-)
>
> diff --git a/tools/libs/stat/xenstat_linux.c b/tools/libs/stat/xenstat_li=
nux.c
> index cbba54aa83ee..5a4a03634182 100644
> --- a/tools/libs/stat/xenstat_linux.c
> +++ b/tools/libs/stat/xenstat_linux.c
> @@ -390,6 +390,39 @@ void xenstat_uninit_networks(xenstat_handle * handle=
)
>                 fclose(priv->procnetdev);
>  }
>
> +static int read_attributes_vbd3(char *vbd3_path, xenstat_vbd *vbd)
> +{
> +       FILE *fp;
> +       struct vbd3_stats vbd3_stats;
> +
> +       fp =3D fopen(vbd3_path, "rb");
> +
> +       if (fp =3D=3D NULL)
> +       {

The syntax of this if statement is different from the ones below.

> +               return -1;
> +       }
> +
> +       if (fread(&vbd3_stats, sizeof(struct vbd3_stats), 1, fp) !=3D 1) =
{
> +               fclose(fp);
> +               return -1;
> +       }
> +
> +       if (vbd3_stats.version !=3D 1) {
> +               fclose(fp);
> +               return -1;
> +       }
> +
> +       vbd->oo_reqs =3D vbd3_stats.oo_reqs;
> +       vbd->rd_reqs =3D vbd3_stats.read_reqs_submitted;
> +       vbd->rd_sects =3D vbd3_stats.read_sectors;
> +       vbd->wr_reqs =3D vbd3_stats.write_reqs_submitted;
> +       vbd->wr_sects =3D vbd3_stats.write_sectors;
> +
> +       fclose(fp);
> +
> +       return 0;
> +}
> +
>  static int read_attributes_vbd(const char *vbd_directory, const char *wh=
at, char *ret, int cap)
>  {
>         static char file_name[80];
> @@ -438,7 +471,7 @@ int xenstat_collect_vbds(xenstat_node * node)
>                 int ret;
>                 char buf[256];
>
> -               ret =3D sscanf(dp->d_name, "%3s-%u-%u", buf, &domid, &vbd=
.dev);
> +               ret =3D sscanf(dp->d_name, "%255[^-]-%u-%u", buf, &domid,=
 &vbd.dev);
>                 if (ret !=3D 3)
>                         continue;
>                 if (!(strstr(buf, "vbd")) && !(strstr(buf, "tap")))
> @@ -448,6 +481,8 @@ int xenstat_collect_vbds(xenstat_node * node)
>                         vbd.back_type =3D 1;
>                 else if (strcmp(buf,"tap") =3D=3D 0)
>                         vbd.back_type =3D 2;
> +               else if (strcmp(buf,"vbd3") =3D=3D 0)
> +                       vbd.back_type =3D 3;
>                 else
>                         vbd.back_type =3D 0;
>
> @@ -479,6 +514,38 @@ int xenstat_collect_vbds(xenstat_node * node)
>                                 vbd.error =3D 1;
>                         }
>                 }
> +               else if (vbd.back_type =3D=3D 3)
> +               {
> +
> +                       char *td3_pid;
> +                       char *path;
> +
> +                       vbd.back_type =3D 3;
> +                       vbd.error =3D 0;
> +
> +                       if (asprintf(&path, "/local/domain/0/backend/vbd3=
/%u/%u/kthread-pid", domid, vbd.dev) < 0)
> +                               continue;
> +
> +                       td3_pid =3D xs_read(node->handle->xshandle, XBT_N=
ULL, path, NULL);
> +
> +                       if (td3_pid =3D=3D NULL) {
> +                               free(path);
> +                               continue;
> +                       }
> +
> +                       free(path);

Why not freeing "path" unconditionally? We free it in all cases.

> +
> +                       if (asprintf(&path, "/dev/shm/td3-%s/vbd-%u-%u", =
td3_pid, domid, vbd.dev) < 0) {
> +                               free(td3_pid);
> +                               continue;
> +                       }
> +
> +                       if (read_attributes_vbd3(path, &vbd) < 0)
> +                               vbd.error =3D 1;
> +
> +                       free(td3_pid);
> +                       free(path);
> +               }
>                 else
>                 {
>                         vbd.error =3D 1;
> diff --git a/tools/libs/stat/xenstat_priv.h b/tools/libs/stat/xenstat_pri=
v.h
> index 4eb44a8ebb84..c3a9635240e9 100644
> --- a/tools/libs/stat/xenstat_priv.h
> +++ b/tools/libs/stat/xenstat_priv.h
> @@ -98,6 +98,22 @@ struct xenstat_vbd {
>         unsigned long long wr_sects;
>  };
>
> +struct vbd3_stats {
> +       uint32_t version;
> +       uint32_t __pad;
> +       uint64_t oo_reqs;
> +       uint64_t read_reqs_submitted;
> +       uint64_t read_reqs_completed;
> +       uint64_t read_sectors;
> +       uint64_t read_total_ticks;
> +       uint64_t write_reqs_submitted;
> +       uint64_t write_reqs_completed;
> +       uint64_t write_sectors;
> +       uint64_t write_total_ticks;
> +       uint64_t io_errors;
> +       uint64_t flags;
> +};
> +
>  extern int xenstat_collect_networks(xenstat_node * node);
>  extern void xenstat_uninit_networks(xenstat_handle * handle);
>  extern int xenstat_collect_vbds(xenstat_node * node);
> diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
> index 0a2fab7f15a3..f5a456fd4dfd 100644
> --- a/tools/xentop/xentop.c
> +++ b/tools/xentop/xentop.c
> @@ -1124,6 +1124,7 @@ void do_vbd(xenstat_domain *domain)
>                 "Unidentified",           /* number 0 */
>                 "BlkBack",           /* number 1 */
>                 "BlkTap",            /* number 2 */
> +               "Tapdisk3"           /* number 3 */
>         };
>
>         num_vbds =3D xenstat_domain_num_vbds(domain);
> --
> 2.42.0
>
>


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:29:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:29:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685570.1066488 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebyX-0000sS-9C; Mon, 26 Feb 2024 14:29:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685570.1066488; Mon, 26 Feb 2024 14:29:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebyX-0000sL-6J; Mon, 26 Feb 2024 14:29:01 +0000
Received: by outflank-mailman (input) for mailman id 685570;
 Mon, 26 Feb 2024 14:29:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rebyV-0000sF-VT
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:28:59 +0000
Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com
 [2a00:1450:4864:20::12a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6090f8bf-d4b3-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 15:28:58 +0100 (CET)
Received: by mail-lf1-x12a.google.com with SMTP id
 2adb3069b0e04-512e4f4e463so3653648e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:28:58 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l9-20020a170906078900b00a3d99dc3f3asm2448125ejc.67.2024.02.26.06.28.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 06:28:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6090f8bf-d4b3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708957737; x=1709562537; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=hECOQ2WXFS3Fb52EbsfN7R5bLqN4kYrXVtDndGZWRPc=;
        b=LyQAP1unyJ0OSCWGQkObugiodjxAyJMRKKvfq+vLNJN2c0Hp9MAXygHrQaqt1qw1as
         vltnqzNnq36UQr7hbLUtxxs/KxCajvNf6Wo+Y2LQJhfNXd6bz4qwqvwKu235hB9dV+J8
         KoenS6XZe+MYF75L0P9ED6wFoKO3251IzBzYL+NRqq25gAw5mnjgtXkgSGFPiB5EchnN
         yxAPISrXqIae/1aFECkLsHuGscSRXOhrZZs3d1rPkj6W15kgy542spGyNHSuQGkUvepD
         yQx9bj/AIrgLj3I9ys9ZUesMXU4xLGvhtuqjW6ZJfUjPXNW7V9eQJGXieizQspPiPOp6
         mHMw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708957737; x=1709562537;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=hECOQ2WXFS3Fb52EbsfN7R5bLqN4kYrXVtDndGZWRPc=;
        b=PESfMjNwkUVEj0mQqI4UpVMWgE8mZdwmjFQ6x7zGDz3xTznWV10JUZ+BWGLnQV1yv/
         akjlPGmPSjAxlKzzmawQl8JPKECmzx4C1xbZOps5lEKXxXKrEwtxpQWSAl2GfU4+GFVD
         ORqipako7OoINl0f0rkBfRavGkJA/JFDQBjP0+qGlCj6BYPZJOzRNxAtkCqZMENyJHjr
         dnNJVSabJiKN8Mbb1EMTpR61mOGtRWDxjrqM5cc1652dFVIkmb+M6Y5vLtUSRPLQBmVt
         KthDXALogzmBLL2TPnKwRqxavuBeYhBRSzBAl3MvidhuyZHb1PALQFD9RPvk28ZmhVJh
         C1xQ==
X-Forwarded-Encrypted: i=1; AJvYcCXkINoycJa9Nt9nfcsiSQqc0jEgWg7JbeY7IVW/c8lOkBOO50ZgyqtDgbsIeyygZkZbo7mTm53G4OW781CipYVXlefXcm+kpLJCj8+JVk4=
X-Gm-Message-State: AOJu0Yw+dmQAqh8VACUh2Qa4X0sG4eWYbqTiLcuaVPmjEYAGX8slgw6c
	2z0c5M3Upaf2FofyVJEScc3dmrIy0jiO2A1PkH8vfTTmaMsJrT0bpYsfyPlS8Q==
X-Google-Smtp-Source: AGHT+IGVpA4wJgqZn2yHrblWc5YuDqP6tbXe4V3StP7xiq9EMS5ZuLPDGPiKNla3bn9RAhTBFyF3IA==
X-Received: by 2002:ac2:4d87:0:b0:512:b04a:aa56 with SMTP id g7-20020ac24d87000000b00512b04aaa56mr4553891lfe.24.1708957737606;
        Mon, 26 Feb 2024 06:28:57 -0800 (PST)
Message-ID: <d68cc5ad-6dec-464c-a171-f3259a8fb70b@suse.com>
Date: Mon, 26 Feb 2024 15:28:56 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/3] x86/entry: Adjustments to "reduce assembly code size
 of entry points"
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
 <20240226125501.1233599-2-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226125501.1233599-2-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 13:54, Andrew Cooper wrote:
> Some retroactive review, for if I'd got to the patch in time.
> 
>  * The new ASM-friendly BUILD_BUG_ON() should be in a header file.
>  * entry_int82() wants the movl->movb treatment too.
> 
> Fixes: c144b9e32427 ("x86: Reduce assembly code size of entry points")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

> Fixes just to link the two patches.

Hmm, I see.

> --- a/xen/arch/x86/include/asm/asm_defns.h
> +++ b/xen/arch/x86/include/asm/asm_defns.h
> @@ -56,6 +56,18 @@ register unsigned long current_stack_pointer asm("rsp");
>  #define ASSERT_INTERRUPTS_DISABLED \
>      ASSERT_INTERRUPT_STATUS(z, "INTERRUPTS DISABLED")
>  
> +#ifdef __ASSEMBLY__
> +
> +.macro BUILD_BUG_ON condstr cond:vararg
> +        .if \cond
> +        .error "Condition \"\condstr\" not satisfied"
> +        .endif
> +.endm
> +/* preprocessor macro to make error message more user friendly */
> +#define BUILD_BUG_ON(cond) BUILD_BUG_ON #cond cond
> +
> +#endif /* __ASSEMBLY__ */
> +
>  #ifdef __ASSEMBLY__

Minor remark: Seeing these back to back is slightly odd. A little
higher up there's an "#ifndef __ASSEMBLY__" (producing e.g.
ASM_CALL_CONSTRAINT) in an #else of which this would imo be a
pretty good fit. Further down where SUBSECTION_LBL() is defined would
also look to be more suitable than adding yet another #ifdef.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:29:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:29:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685572.1066498 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebzB-0001X8-J9; Mon, 26 Feb 2024 14:29:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685572.1066498; Mon, 26 Feb 2024 14:29:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rebzB-0001X1-EN; Mon, 26 Feb 2024 14:29:41 +0000
Received: by outflank-mailman (input) for mailman id 685572;
 Mon, 26 Feb 2024 14:29:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rebz9-0000sF-O7
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:29:39 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 786cb726-d4b3-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 15:29:38 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-512bce554a5so3938520e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:29:38 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l9-20020a170906078900b00a3d99dc3f3asm2448125ejc.67.2024.02.26.06.29.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 06:29:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 786cb726-d4b3-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708957778; x=1709562578; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=BaaFxXwDVZJ2mUArreBYJzy2ZfGo1vhOiJJyArOICp8=;
        b=f8rgxSqPw5+MLVImmlisQp5aZh987JxRuoz/G16/HHUPwHg6tKd/60c/Uh9suIAS3P
         Ii2DzTEyLlKwILiXWOKwtkwmtg5UdnhbzKZqvgfBiI7NfZl4RtYNmI0moVVQ3cpq0Mm3
         Um/i3hGgpe79KiXOdIbdOCFC8YIW23hcjTkCdIbTS0U2FzlsNhyaKfycZ17+OsLbK4ER
         5Ac0xaTPMvd/ofykxvWmG1MumtWeA+Uit2eBQgBiTwh8mLMpizqDNPHm+r+NyzYAhB/M
         ccYsOsLWvK/+Tr1YushMKKCVjQ51hP3SudYjOmPjCvDUBLHfsI1ooqx7v3HTtkK2FmfI
         Heew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708957778; x=1709562578;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=BaaFxXwDVZJ2mUArreBYJzy2ZfGo1vhOiJJyArOICp8=;
        b=olUQicDR6Lp5YW6zCMJBpGIOTkxqQSMZWynNOISyLxTYaoFPWF/fYvfJuEkDHAupL0
         OXb0x0H0BJM5A2FYX6qlVxKSWqq4EuKUqeFeHpzcMIisnAcYr0iwM5NqblGqk14miMH/
         aF7k2q/fIsSXsZLrd+lGi5e6YfKTKe7o/r/er1+1XJEm0NJbPLaSISKxbbvGu5aNQQy7
         BpJYKPLzE9Ade31e6xg147wxjSBMvUBehbKWeqrz9YrnKbiTFfRLWBKSEpfp/e/qm/h/
         a6vSwvfnhctGwEMXtjU5lOxNZ12fMuGkgecD223nPnJeWOg3Ji0Nyq4qnWUOh5XHqeN9
         RBBQ==
X-Forwarded-Encrypted: i=1; AJvYcCV0S2Y26EaLroU3w3ooOl8qHa98OJzDrTKCfkD4GgG7KVwrHDgA19w9UjpshdWx9GHq+p1rTjKyULLbliOPgYUWQTs3jUJCoKvfMpSNcG4=
X-Gm-Message-State: AOJu0Yxhq0R3gxlx0MdEl0tvqy4WIcWOvYNiCYwX6+BYz3Xcia5hnyDm
	7DzQAuXQbZMTHIAUyJTyzuAyWthRSS1RI26SLW0iEBd+GELP76RRrEn7oaQaHA==
X-Google-Smtp-Source: AGHT+IE3X4a5BHCW2jViG//VrdrAL9LZrocslKZgqFXV8XXHANZxZLiMnw/tSWyni8lI903g1vpyBw==
X-Received: by 2002:ac2:5b1a:0:b0:512:eee2:d8d0 with SMTP id v26-20020ac25b1a000000b00512eee2d8d0mr4080251lfn.59.1708957777797;
        Mon, 26 Feb 2024 06:29:37 -0800 (PST)
Message-ID: <5f884442-df10-44eb-b748-c6b359b24910@suse.com>
Date: Mon, 26 Feb 2024 15:29:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/3] x86/entry: Simplify expressions in
 compat_restore_all_guest()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
 <20240226125501.1233599-3-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226125501.1233599-3-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 13:55, Andrew Cooper wrote:
> compat_restore_all_guest() already has SPEC_CTRL_EXIT_TO_PV with a documented
> requirement for %rsp to be both regs and cpuinfo.
> 
> Use the now-normal annotations and simplify the expressions which happen to be
> a subtraction of 0.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:32:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:32:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685578.1066507 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rec1z-0003Oh-UN; Mon, 26 Feb 2024 14:32:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685578.1066507; Mon, 26 Feb 2024 14:32:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rec1z-0003Oa-Ri; Mon, 26 Feb 2024 14:32:35 +0000
Received: by outflank-mailman (input) for mailman id 685578;
 Mon, 26 Feb 2024 14:32:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rec1y-0003OT-D2
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:32:34 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e0f23cc3-d4b3-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 15:32:33 +0100 (CET)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-55a035669d5so5582607a12.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:32:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m13-20020a1709062b8d00b00a3d4dc76454sm2456220ejg.159.2024.02.26.06.32.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 06:32:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e0f23cc3-d4b3-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708957953; x=1709562753; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=sq9Jn4HsxBCmv5YKEqHY9N3bvJT38sl32KjCdq1pe5M=;
        b=NqtVvyWtn/8qn4vJNwfa8YSA7i85MRk6sN6BUT/1sXYb+nK5NV6k/w2BmKGpsB3wIf
         A/4HqWMZ/eVpON6zFOL9un8D9Mcb4eZIBBdJGG+9iDRfZ/Hu42BtxrkwbAdBye4edBQK
         ACAuyQ88is8H/qqu6JeF0yUsxhudtMqY72RFCRAXP9sSrHlj1xQ25fUJQunsUMNWtaL4
         VNzhCmkoSKdxYHGLltMLymtY0Of14IAvbp2HuO0LQDuwHxLdhEwRfN8Z1Ao42NH0GZic
         eVp4OJpctZbGAniVRJz1LPpv1ZVPlK6Wc0R0Ecojg37w2B5bJJ9ph+2LU89LwMibw6Rn
         BoQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708957953; x=1709562753;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=sq9Jn4HsxBCmv5YKEqHY9N3bvJT38sl32KjCdq1pe5M=;
        b=DJc7gcvOQi/8/Gi4wOLgkFPC/vDHaZGmgvfp28YAaWTijzCDWwGVDSTJwMjv08goiq
         tn3UBHt2tJ1aPiiuWIQT5Ur2uL+RJsDSl+txfSCSdF9QN8FGgRdQjNPfcgjCS+BAZFiY
         lZDYRdNKpiwciarFe+hKRNspwz1G60g32Kd8EpPYHr3FNJ9T3DVseITOl1owNUOD+wja
         0R1eu87evg7afU+ZvEycWa9gUInJCHrCsp41shtZz9lT1zIEfHymIAoq4eqr2V9/zPnT
         hxFG+T3RIutn5HVMw/wiIbHJXmW3+kR/EwbB6bVFWb8jWYY19Hm0YBeTlTKAY89FWKqH
         iR/w==
X-Forwarded-Encrypted: i=1; AJvYcCWOVQuwQ9rJ2+pLNLT2eDsB5p4kUiMMJqMW/g4Z5RDUk0MmoPOLBXSqDjr6vt5nezHLWq5a0jAHUUPqXsCvtamOn4WMXG4NZdFGHErjkhQ=
X-Gm-Message-State: AOJu0YzdZRJa7Wj70r97bPH2L57+TzzmPNj8nw6WyOBkc8eYBmFEGh4q
	EU6iy4RxaHQH84CrZe/YvGGKIERQC4Jy9TNgNczjLesBrlMOkd3P0fX4YUiTOw==
X-Google-Smtp-Source: AGHT+IFeKgNGcO6S2yEzxIzMYZqw0QWQ9Ua0xke47tGX+sNRMZS+75Hck7WKiEiB0Div5i5dbfESjQ==
X-Received: by 2002:a17:906:d106:b0:a43:7ca4:a305 with SMTP id b6-20020a170906d10600b00a437ca4a305mr1018397ejz.43.1708957952983;
        Mon, 26 Feb 2024 06:32:32 -0800 (PST)
Message-ID: <01621599-7d1f-4cfb-9844-06fba32a62aa@suse.com>
Date: Mon, 26 Feb 2024 15:32:32 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/3] x86/entry: Introduce EFRAME_* constants
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
 <20240226125501.1233599-4-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226125501.1233599-4-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 13:55, Andrew Cooper wrote:
> restore_all_guest() does a lot of manipulation of the stack after popping the
> GPRs, and uses raw %rsp displacements to do so.  Also, almost all entrypaths
> use raw %rsp displacements prior to pushing GPRs.
> 
> Provide better mnemonics, to aid readability and reduce the chance of errors
> when editing.
> 
> No functional change.  The resulting binary is identical.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>
with one small request:

> --- a/xen/arch/x86/x86_64/asm-offsets.c
> +++ b/xen/arch/x86/x86_64/asm-offsets.c
> @@ -51,6 +51,23 @@ void __dummy__(void)
>      OFFSET(UREGS_kernel_sizeof, struct cpu_user_regs, es);
>      BLANK();
>  
> +    /*
> +     * EFRAME_* is for the entry/exit logic where %rsp is pointing at
> +     * UREGS_error_code and GPRs are still guest values.
> +     */

"still/already" or some such to match "entry/exit"?

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:38:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:38:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685580.1066517 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rec78-0003yq-HJ; Mon, 26 Feb 2024 14:37:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685580.1066517; Mon, 26 Feb 2024 14:37:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rec78-0003yj-Eh; Mon, 26 Feb 2024 14:37:54 +0000
Received: by outflank-mailman (input) for mailman id 685580;
 Mon, 26 Feb 2024 14:37:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rec77-0003yd-5F
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:37:53 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9e0f6f67-d4b4-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 15:37:50 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-512a65cd2c7so4385621e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:37:50 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 s10-20020a056512202a00b00512ef0a8f3asm863037lfs.15.2024.02.26.06.37.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 06:37:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e0f6f67-d4b4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708958270; x=1709563070; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=u1E0M73nbUP55/W60+PLeXgQRkWaZatj6ybuxc5UDrU=;
        b=QU7W078A52yWqTZZL5+9xlwnyzTdnIqNbhIDDm70sdChgrZbrPr1sElyXpg7KZv2ys
         Ce/MmiQp3MjcaoG8be88KqerDV5UOZGoQWap/apqBriIbOOJZO67y6yJt4JcJe8/lqXW
         4oWRH7gZ4D+i6noLj8AdlyfpHtMnjM7NUK5kC9wbnW7DS/wgtktBnxXAE90wkbpIk20W
         ShEWHViDLB+YvYw5OAoJx6aHBSDsVRNDzCCnAx64YTxmQw66skG05Monse2gX54JX5xa
         X7HfjJwrAA6towSJSWlK/IGoPQ2MlF6U9NLIz5OqTB9GoBN+fjmjNnT8dVPxCS3ChhuC
         5qjQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708958270; x=1709563070;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=u1E0M73nbUP55/W60+PLeXgQRkWaZatj6ybuxc5UDrU=;
        b=tsJys11oNodzBTgXyd+mpRMI4FoLSVSObDPo3XZUUpOg3fURsTWYS2ws/bsBxhLhCb
         SiZjUC/zcx8az24CQP++8IIF5LSkpVjSyx7+RJEJLC3+/eoGd6DaOSlUWY6/YX1kKpXB
         nZpeOQsJBMJjnw11iec6IdYB4lueUfVHmhjkN6zSz2y9EKj+U6stQNpCl7XRpmSivZkD
         z2xVUu/wZJbYw9EFKwjCaNkKPAVjB5RD/I+d13w1eKoA++PBIvyF3/9z2GdFF8oQAXPG
         FJ5KHK6P79lcu0XqZQG5oMF41TgWOA0C/xbZbYLnIQzFUd6RIxbPMbbb1SUUpfOnckyw
         v0EA==
X-Forwarded-Encrypted: i=1; AJvYcCUxM99/ujsgljoN26KIfLX9RSGsx8UgaGR0Rsxr3+OyR8LCp5ICPtKQct/xudgS12BNdoeG6Q4oi9odyqbYPL5jXaXCBKg3rsyikAg4ST8=
X-Gm-Message-State: AOJu0YzFdWA4+fcL+z5I/MfM7Lfi6slRz6VuiSitXabl9Hi0GN7aNHpx
	gVRc4pYGXIdG+qOsfpR9cHQf3nqHpE9TqscEUizBtg3dWWAA353I
X-Google-Smtp-Source: AGHT+IE0UOWJHNWn31KZ5iNnGSs+imdgvkHHDe+z5pd7bpRPXc4H+4LPs3eUeh7aPtGA7xzMK34Ddw==
X-Received: by 2002:a05:6512:2820:b0:512:b915:606a with SMTP id cf32-20020a056512282000b00512b915606amr5652410lfb.12.1708958269773;
        Mon, 26 Feb 2024 06:37:49 -0800 (PST)
Message-ID: <a5e4c87720ca781ef113d7ee575a97656350a114.camel@gmail.com>
Subject: Re: [PATCH v4 12/30] xen/riscv: introduce cmpxchg.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Mon, 26 Feb 2024 15:37:48 +0100
In-Reply-To: <4f59127c-6a98-4672-9202-4d48817a1931@suse.com>
References: <cover.1707146506.git.oleksii.kurochko@gmail.com>
	 <25315ca95baffc9b222fb0ae89375a94b01a9b46.1707146506.git.oleksii.kurochko@gmail.com>
	 <9e50ef30-8dc6-4380-aa65-724e5a376c10@suse.com>
	 <28844fdfcf5eea515497fb7b5fd8ea6fb1c5ebaa.camel@gmail.com>
	 <554a43e8-7d8d-45c4-936d-36f02c207531@suse.com>
	 <1c53b52ee39161a8f59209d28af69fe997479dbc.camel@gmail.com>
	 <3b38fe82-ee0f-4666-93e1-bd78fe69c534@suse.com>
	 <5d2d35fe014094c991363e42c3e2ad9ca2af3938.camel@gmail.com>
	 <44fd5092-7838-4d28-804b-bbfebfd44886@suse.com>
	 <56ae27d003b8763af34864ae56433691685c3661.camel@gmail.com>
	 <7cab68d1-9bdf-4ea1-b49a-cce1e4af4692@suse.com>
	 <4215fb9eb5be5898b3cc9c2977e55ac0e7270b4d.camel@gmail.com>
	 <4f59127c-6a98-4672-9202-4d48817a1931@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Mon, 2024-02-26 at 15:20 +0100, Jan Beulich wrote:
> On 26.02.2024 13:58, Oleksii wrote:
> > On Mon, 2024-02-26 at 12:28 +0100, Jan Beulich wrote:
> > > On 26.02.2024 12:18, Oleksii wrote:
> > > > On Mon, 2024-02-26 at 10:45 +0100, Jan Beulich wrote:
> > > > > On 23.02.2024 13:23, Oleksii wrote:
> > > > > > >=20
> > > > > > > > > > As 1- and 2-byte cases are emulated I decided that
> > > > > > > > > > is
> > > > > > > > > > not
> > > > > > > > > > to
> > > > > > > > > > provide
> > > > > > > > > > sfx argument for emulation macros as it will not
> > > > > > > > > > have
> > > > > > > > > > to
> > > > > > > > > > much
> > > > > > > > > > affect on
> > > > > > > > > > emulated types and just consume more performance on
> > > > > > > > > > acquire
> > > > > > > > > > and
> > > > > > > > > > release
> > > > > > > > > > version of sc/ld instructions.
> > > > > > > > >=20
> > > > > > > > > Question is whether the common case (4- and 8-byte
> > > > > > > > > accesses)
> > > > > > > > > shouldn't
> > > > > > > > > be valued higher, with 1- and 2-byte emulation being
> > > > > > > > > there
> > > > > > > > > just
> > > > > > > > > to
> > > > > > > > > allow things to not break altogether.
> > > > > > > > If I understand you correctly, it would make sense to
> > > > > > > > add
> > > > > > > > the
> > > > > > > > 'sfx'
> > > > > > > > argument for the 1/2-byte access case, ensuring that
> > > > > > > > all
> > > > > > > > options
> > > > > > > > are
> > > > > > > > available for 1/2-byte access case as well.
> > > > > > >=20
> > > > > > > That's one of the possibilities. As said, I'm not overly
> > > > > > > worried
> > > > > > > about
> > > > > > > the emulated cases. For the initial implementation I'd
> > > > > > > recommend
> > > > > > > going
> > > > > > > with what is easiest there, yielding the best possible
> > > > > > > result
> > > > > > > for
> > > > > > > the
> > > > > > > 4- and 8-byte cases. If later it turns out repeated
> > > > > > > acquire/release
> > > > > > > accesses are a problem in the emulation loop, things can
> > > > > > > be
> > > > > > > changed
> > > > > > > to explicit barriers, without touching the 4- and 8-byte
> > > > > > > cases.
> > > > > > I am confused then a little bit if emulated case is not an
> > > > > > issue.
> > > > > >=20
> > > > > > For 4- and 8-byte cases for xchg .aqrl is used, for relaxed
> > > > > > and
> > > > > > aqcuire
> > > > > > version of xchg barries are used.
> > > > > >=20
> > > > > > The similar is done for cmpxchg.
> > > > > >=20
> > > > > > If something will be needed to change in emulation loop it
> > > > > > won't
> > > > > > require to change 4- and 8-byte cases.
> > > > >=20
> > > > > I'm afraid I don't understand your reply.
> > > > IIUC, emulated cases it is implemented correctly in terms of
> > > > usage
> > > > barriers. And it also OK not to use sfx for lr/sc instructions
> > > > and
> > > > use
> > > > only barriers.
> > > >=20
> > > > For 4- and 8-byte cases are used sfx + barrier depending on the
> > > > specific case ( relaxed, acquire, release, generic xchg/cmpxchg
> > > > ).
> > > > What also looks to me correct. But you suggested to provide the
> > > > best
> > > > possible result for 4- and 8-byte cases.=C2=A0
> > > >=20
> > > > So I don't understand what the best possible result is as the
> > > > current
> > > > one usage of __{cmp}xchg_generic for each specific case=C2=A0 (
> > > > relaxed,
> > > > acquire, release, generic xchg/cmpxchg ) looks correct to me:
> > > > xchg -> (..., ".aqrl", "", "") just suffix .aqrl suffix without
> > > > barriers.
> > > > xchg_release -> (..., "", RISCV_RELEASE_BARRIER, "" ) use only
> > > > release
> > > > barrier
> > > > xchg_acquire -> (..., "", "", RISCV_ACQUIRE_BARRIER ), only
> > > > acquire
> > > > barrier
> > > > xchg_relaxed ->=C2=A0 (..., "", "", "") - no barries, no sfx
> > >=20
> > > So first: While explicit barriers are technically okay, I don't
> > > follow why
> > > you insist on using them when you can achieve the same by
> > > suitably
> > > tagging
> > > the actual insn doing the exchange. Then second: It's somewhat
> > > hard
> > > for me
> > > to see the final effect on the emulation paths without you
> > > actually
> > > having
> > > done the switch. Maybe no special handling is necessary there
> > > anymore
> > > then. And as said, it may actually be acceptable for the
> > > emulation
> > > paths
> > > to "only" be correct, but not be ideal in terms of performance.
> > > After
> > > all,
> > > if you use the normal 4-byte primitive in there, more (non-
> > > explicit)
> > > barriers than needed would occur if the involved loop has to take
> > > more
> > > than one iteration. Which could (but imo doesn't need to be)
> > > avoided
> > > by
> > > using a more relaxed 4-byte primitive there and an explicit
> > > barrier
> > > outside of the loop.
> >=20
> > According to the spec:
> > Table A.5 ( part of the table only I copied here )
> >=20
> > Linux Construct=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 R=
VWMO Mapping
> > atomic <op> relaxed=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 amo<op>.{w|d}
> > atomic <op> acquire=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 amo<op>.{w|d}.aq
> > atomic <op> release=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 amo<op>.{w|d}.rl
> > atomic <op>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 amo<op>.{w|d}.aqrl
> >=20
> > Linux Construct=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 R=
VWMO LR/SC Mapping
> > atomic <op> relaxed=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 loop: lr.{w|d};=
 <op>; sc.{w|d}; bnez loop
> > atomic <op> acquire=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 loop: lr.{w|d}.=
aq; <op>; sc.{w|d}; bnez
> > loop
> > atomic <op> release=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 loop: lr.{w|d};=
 <op>; sc.{w|d}.aqrl=E2=88=97 ;
> > bnez=20
> > loop OR
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 fence.tso; loop: lr.{w|d}; <op>;
> > sc.{w|d}=E2=88=97 ;
> > bnez loop
> > atomic <op>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl;
> > bnez
> > loop
>=20
> In your consideration what to implement you will want to limit
> things to constructs we actually use. I can't find any use of the
> relaxed, acquire, or release forms of atomics as mentioned above.
>=20
> > The Linux mappings for release operations may seem stronger than
> > necessary, but these mappings
> > are needed to cover some cases in which Linux requires stronger
> > orderings than the more intuitive
> > mappings would provide. In particular, as of the time this text is
> > being written, Linux is actively
> > debating whether to require load-load, load-store, and store-store
> > orderings between accesses in one
> > critical section and accesses in a subsequent critical section in
> > the
> > same hart and protected by the
> > same synchronization object. Not all combinations of FENCE
> > RW,W/FENCE
> > R,RW mappings
> > with aq/rl mappings combine to provide such orderings. There are a
> > few
> > ways around this problem,
> > including:
> > 1. Always use FENCE RW,W/FENCE R,RW, and never use aq/rl. This
> > suffices
> > but is undesir-
> > able, as it defeats the purpose of the aq/rl modifiers.
> > 2. Always use aq/rl, and never use FENCE RW,W/FENCE R,RW. This does
> > not
> > currently work
> > due to the lack of load and store opcodes with aq and rl modifiers.
>=20
> I don't understand this point: Which specific load and/or store forms
> are missing? According to my reading of the A extension spec all
> combination of aq/rl exist with both lr and sc.
I think this is not about lr and sc instructions.
It is about l{b|h|w|d} and s{b|h|w|d}, which should be used with fence
in case of acquire and seq_cst.

>=20
> > 3. Strengthen the mappings of release operations such that they
> > would
> > enforce sufficient order-
> > ings in the presence of either type of acquire mapping. This is the
> > currently-recommended
> > solution, and the one shown in Table A.5.
> >=20
> >=20
> > Based on this it is enough in our case use only suffixed
> > istructions
> > (amo<op>.{w|d}{.aq, .rl, .aqrl, .aqrl }, lr.{w|d}.{.aq, .aqrl }.
> >=20
> >=20
> > But as far as I understand in Linux atomics were strengthen with
> > fences:
> > =C2=A0=C2=A0=C2=A0 Atomics present the same issue with locking: release=
 and
> > acquire
> > =C2=A0=C2=A0=C2=A0 variants need to be strengthened to meet the constra=
ints
> > defined
> > =C2=A0=C2=A0=C2=A0 by the Linux-kernel memory consistency model [1].
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 Atomics present a further issue: implementations of =
atomics
> > such
> > =C2=A0=C2=A0=C2=A0 as atomic_cmpxchg() and atomic_add_unless() rely on =
LR/SC
> > pairs,
> > =C2=A0=C2=A0=C2=A0 which do not give full-ordering with .aqrl; for exam=
ple,
> > current
> > =C2=A0=C2=A0=C2=A0 implementations allow the "lr-sc-aqrl-pair-vs-full-b=
arrier"
> > test
> > =C2=A0=C2=A0=C2=A0 below to end up with the state indicated in the "exi=
sts"
> > clause.
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 In order to "synchronize" LKMM and RISC-V's implemen=
tation,
> > this
> > =C2=A0=C2=A0=C2=A0 commit strengthens the implementations of the atomic=
s
> > operations
> > =C2=A0=C2=A0=C2=A0 by replacing .rl and .aq with the use of ("lightweig=
th")
> > fences,
> > =C2=A0=C2=A0=C2=A0 and by replacing .aqrl LR/SC pairs in sequences such=
 as:
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lr.w.aq=
rl=C2=A0 %0, %addr
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 bne=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 %0, %old, 1f
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 ...
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 sc.w.aqrl=C2=A0 %1, %new, %addr
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 bnez=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 %1, 0b
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1:
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 with sequences of the form:
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lr.w=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 %0, %addr
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 bne=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 %0, %old, 1f
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 ...
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 sc.w.rl=C2=A0=C2=A0=C2=A0 %1, %new, %addr=C2=A0=C2=A0 /* SC-relea=
se=C2=A0=C2=A0 */
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 bnez=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 %1, 0b
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 fence=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rw, rw=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* "full" fence */
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1:
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 following Daniel's suggestion.
>=20
> I'm likely missing something, yet as it looks it does help that the
> code fragment above appears to be ...
>=20
> > =C2=A0=C2=A0=C2=A0 These modifications were validated with simulation o=
f the RISC-
> > V
> > =C2=A0=C2=A0=C2=A0 with sequences of the form:
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lr.w=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 %0, %addr
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 bne=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 %0, %old, 1f
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 ...
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 sc.w.rl=C2=A0=C2=A0=C2=A0 %1, %new, %addr=C2=A0=C2=A0 /* SC-relea=
se=C2=A0=C2=A0 */
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 bnez=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 %1, 0b
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0 fence=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rw, rw=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* "full" fence */
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1:
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 following Daniel's suggestion.
>=20
> ... entirely the same as this one. Yet there's presumably a reason
> for quoting it twice?
I think it was done by accident

~ Oleksii
>=20
> > =C2=A0=C2=A0=C2=A0 These modifications were validated with simulation o=
f the RISC-
> > V
> > =C2=A0=C2=A0=C2=A0 memory consistency model.
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 C lr-sc-aqrl-pair-vs-full-barrier
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 {}
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 P0(int *x, int *y, atomic_t *u)
> > =C2=A0=C2=A0=C2=A0 {
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int =
r0;
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int =
r1;
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 WRIT=
E_ONCE(*x, 1);
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 r0 =
=3D atomic_cmpxchg(u, 0, 1);
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 r1 =
=3D READ_ONCE(*y);
> > =C2=A0=C2=A0=C2=A0 }
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 P1(int *x, int *y, atomic_t *v)
> > =C2=A0=C2=A0=C2=A0 {
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int =
r0;
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int =
r1;
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 WRIT=
E_ONCE(*y, 1);
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 r0 =
=3D atomic_cmpxchg(v, 0, 1);
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 r1 =
=3D READ_ONCE(*x);
> > =C2=A0=C2=A0=C2=A0 }
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 exists (u=3D1 /\ v=3D1 /\ 0:r1=3D0 /\ 1:r1=3D0)
> > =C2=A0=C2=A0=C2=A0=20
> > =C2=A0=C2=A0=C2=A0 [1] https://marc.info/?l=3Dlinux-kernel&m=3D15193020=
1102853&w=3D2
> > =C2=A0=C2=A0=C2=A0=C2=A0=20
> > https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/hKyw=
NHBkAXM
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 https://marc.info/?l=3Dlinux=
-kernel&m=3D151633436614259&w=3D2
> >=20
> >=20
> > Thereby Linux kernel implementation seems to me more safe and it is
> > a
> > reason why I want/wanted to be aligned with it.
>=20
> Which may end up being okay. I hope you realize though that there's a
> lot more explanation needed in the respective commits then compared
> to
> what you've had so far. As a minimum, absolutely anything remotely
> unexpected needs to be explained.
>=20
> Jan



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:38:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:38:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685582.1066528 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rec7T-0004Nn-Sm; Mon, 26 Feb 2024 14:38:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685582.1066528; Mon, 26 Feb 2024 14:38:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rec7T-0004Ng-Q5; Mon, 26 Feb 2024 14:38:15 +0000
Received: by outflank-mailman (input) for mailman id 685582;
 Mon, 26 Feb 2024 14:38:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vKMM=KD=gmail.com=jiangshanlai@srs-se1.protection.inumbo.net>)
 id 1rec7S-0003yd-TM
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:38:14 +0000
Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com
 [2607:f8b0:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id aad8289b-d4b4-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 15:38:13 +0100 (CET)
Received: by mail-pl1-x631.google.com with SMTP id
 d9443c01a7336-1dc49b00bdbso26417105ad.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:38:13 -0800 (PST)
Received: from localhost ([198.11.176.14]) by smtp.gmail.com with ESMTPSA id
 d3-20020a170903230300b001d9edac54b1sm4015055plh.171.2024.02.26.06.38.10
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Mon, 26 Feb 2024 06:38:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: aad8289b-d4b4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708958291; x=1709563091; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=+gfvWhFbgTyzu48YJx+Ed8XWBq6XxX8rbNsTiG4trvc=;
        b=NWr8iBgzMx6GwoNN1MlI7HCe9dO04ZwaAKjtznu6PrYRna7SSUs9pZTmNLB2Fgt4RO
         O9iaQCEDRsWITgj5E99SzcSrc9HHIVCJ9b6NXkiXOvaCzseuRMzoLzb7CTKIWYvk+kbe
         4f8pBvvOC4ocpt4A5a4WxzFvraDS1hPp/J591/cdv3NSL0YPlPUKhHN/SI/wOQvY1RzS
         oqWAK9XNgp5mhrQpzYH6ng7it0KqQwV01/kPQ0GzZo9ia9y3tYd0Vl7eCxloBHI8oJXR
         grUiPyDpcUV17JoOxSNF3DWKei5BA8iCr2C4fx9+OSL8kTsBb3Ro4EuqDdzKu0Od825m
         OZTA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708958291; x=1709563091;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=+gfvWhFbgTyzu48YJx+Ed8XWBq6XxX8rbNsTiG4trvc=;
        b=VHpDVJaKsh1P4KYOuVEFEP1hnZJJ8DJwUu9AGd6nlJdu+gCNRXKfa+rl724Lien7x4
         1v3qBHeYFPnrPb3WdzGOD9FSI3mpzLode6N4E96jq6faWW+KA+aeGE7mqjy0PkxDnahQ
         227c9pyakRdzA/afb4w4Xb2MeCGbFQ03IGpOl8MYPhijtMcnedCNkKaLts5gyoCqL5GL
         CJZIKhCJsYKjqbrYxeQHVs8GPNFPG9guChL6E4yIMjBZ5a8eyeMkZvyC+rlQm/KvJvVw
         btzMR+xkkR5RArKoPdyaewRJGNLLSxfGioRC8yN1oQDbyJjcxqM7RpNFwoeNZ3ldnzYg
         +pxg==
X-Forwarded-Encrypted: i=1; AJvYcCU7DobnMkrjlGyfbPWBGkWbMNCV6JgIJt/uk5sQwS15DAE+/Qu7SGjd388ivxhuQtCTD7r6MMsOYHgYv8d3yROCimfHV+zWo6KAw1E1/O0=
X-Gm-Message-State: AOJu0Ywa8iPQYTUYm9J4iN2E3FlTqkS81qcSyIAqaS/W9/vZ4KlXDUld
	Hj/Bx+qE96xUs62tuAfdlhGwe8i2Y7DfTlyg2vNcNShaIceADXWj
X-Google-Smtp-Source: AGHT+IEIutnYKn12v5tvNLy/G9rTRg1goq2sEiyOH6n7oCM/wCUKEcvtJKvKWrDR0plMg8cjveFEyw==
X-Received: by 2002:a17:902:8349:b0:1dc:1fda:202e with SMTP id z9-20020a170902834900b001dc1fda202emr6685293pln.51.1708958291395;
        Mon, 26 Feb 2024 06:38:11 -0800 (PST)
From: Lai Jiangshan <jiangshanlai@gmail.com>
To: linux-kernel@vger.kernel.org
Cc: Hou Wenlong <houwenlong.hwl@antgroup.com>,
	Lai Jiangshan <jiangshan.ljs@antgroup.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Sean Christopherson <seanjc@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>,
	Ingo Molnar <mingo@redhat.com>,
	kvm@vger.kernel.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	x86@kernel.org,
	Kees Cook <keescook@chromium.org>,
	Juergen Gross <jgross@suse.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Darren Hart <dvhart@infradead.org>,
	Andy Shevchenko <andy@infradead.org>,
	xen-devel@lists.xenproject.org,
	platform-driver-x86@vger.kernel.org
Subject: [RFC PATCH 56/73] x86/pvm: Relocate kernel image early in PVH entry
Date: Mon, 26 Feb 2024 22:36:13 +0800
Message-Id: <20240226143630.33643-57-jiangshanlai@gmail.com>
X-Mailer: git-send-email 2.19.1.6.gb485710b
In-Reply-To: <20240226143630.33643-1-jiangshanlai@gmail.com>
References: <20240226143630.33643-1-jiangshanlai@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Hou Wenlong <houwenlong.hwl@antgroup.com>

For a PIE kernel, it runs in a high virtual address in the PVH entry, so
it needs to relocate the kernel image early in the PVH entry for the PVM
guest.

Signed-off-by: Hou Wenlong <houwenlong.hwl@antgroup.com>
Signed-off-by: Lai Jiangshan <jiangshan.ljs@antgroup.com>
---
 arch/x86/include/asm/init.h       |  5 +++++
 arch/x86/kernel/head64_identity.c |  5 -----
 arch/x86/platform/pvh/enlighten.c | 22 ++++++++++++++++++++++
 arch/x86/platform/pvh/head.S      |  4 ++++
 4 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/asm/init.h b/arch/x86/include/asm/init.h
index cc9ccf61b6bd..f78edef60253 100644
--- a/arch/x86/include/asm/init.h
+++ b/arch/x86/include/asm/init.h
@@ -4,6 +4,11 @@
 
 #define __head	__section(".head.text")
 
+#define SYM_ABS_VA(sym) ({					\
+	unsigned long __v;					\
+	asm("movabsq $" __stringify(sym) ", %0":"=r"(__v));	\
+	__v; })
+
 struct x86_mapping_info {
 	void *(*alloc_pgt_page)(void *); /* allocate buf for page table */
 	void *context;			 /* context for alloc_pgt_page */
diff --git a/arch/x86/kernel/head64_identity.c b/arch/x86/kernel/head64_identity.c
index 4e6a073d9e6c..f69f9904003c 100644
--- a/arch/x86/kernel/head64_identity.c
+++ b/arch/x86/kernel/head64_identity.c
@@ -82,11 +82,6 @@ static void __head set_kernel_map_base(unsigned long text_base)
 }
 #endif
 
-#define SYM_ABS_VA(sym) ({					\
-	unsigned long __v;					\
-	asm("movabsq $" __stringify(sym) ", %0":"=r"(__v));	\
-	__v; })
-
 static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd)
 {
 	unsigned long vaddr, vaddr_end;
diff --git a/arch/x86/platform/pvh/enlighten.c b/arch/x86/platform/pvh/enlighten.c
index 00a92cb2c814..8c64c31c971b 100644
--- a/arch/x86/platform/pvh/enlighten.c
+++ b/arch/x86/platform/pvh/enlighten.c
@@ -1,8 +1,10 @@
 // SPDX-License-Identifier: GPL-2.0
 #include <linux/acpi.h>
+#include <linux/pgtable.h>
 
 #include <xen/hvc-console.h>
 
+#include <asm/init.h>
 #include <asm/io_apic.h>
 #include <asm/hypervisor.h>
 #include <asm/e820/api.h>
@@ -113,6 +115,26 @@ static void __init hypervisor_specific_init(bool xen_guest)
 		xen_pvh_init(&pvh_bootparams);
 }
 
+#ifdef CONFIG_PVM_GUEST
+void pvm_relocate_kernel(unsigned long physbase);
+
+void __init pvm_update_pgtable(unsigned long physbase)
+{
+	pgdval_t *pgd;
+	pudval_t *pud;
+	unsigned long base;
+
+	pvm_relocate_kernel(physbase);
+
+	pgd = (pgdval_t *)init_top_pgt;
+	base = SYM_ABS_VA(_text);
+	pgd[pgd_index(base)] = pgd[0];
+	pgd[pgd_index(page_offset_base)] = pgd[0];
+	pud = (pudval_t *)level3_ident_pgt;
+	pud[pud_index(base)] = (unsigned long)level2_ident_pgt + _KERNPG_TABLE_NOENC;
+}
+#endif
+
 /*
  * This routine (and those that it might call) should not use
  * anything that lives in .bss since that segment will be cleared later.
diff --git a/arch/x86/platform/pvh/head.S b/arch/x86/platform/pvh/head.S
index baaa3fe34a00..127f297f7257 100644
--- a/arch/x86/platform/pvh/head.S
+++ b/arch/x86/platform/pvh/head.S
@@ -109,6 +109,10 @@ SYM_CODE_START_LOCAL(pvh_start_xen)
 	wrmsr
 
 #ifdef CONFIG_X86_PIE
+#ifdef CONFIG_PVM_GUEST
+	leaq	_text(%rip), %rdi
+	call	pvm_update_pgtable
+#endif
 	movabs  $2f, %rax
 	ANNOTATE_RETPOLINE_SAFE
 	jmp *%rax
-- 
2.19.1.6.gb485710b



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:39:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:39:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685585.1066537 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rec8g-0005Ad-5h; Mon, 26 Feb 2024 14:39:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685585.1066537; Mon, 26 Feb 2024 14:39:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rec8g-0005AW-37; Mon, 26 Feb 2024 14:39:30 +0000
Received: by outflank-mailman (input) for mailman id 685585;
 Mon, 26 Feb 2024 14:39:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=vKMM=KD=gmail.com=jiangshanlai@srs-se1.protection.inumbo.net>)
 id 1rec8f-0005AQ-Bg
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:39:29 +0000
Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com
 [2607:f8b0:4864:20::42f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d707ea55-d4b4-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 15:39:27 +0100 (CET)
Received: by mail-pf1-x42f.google.com with SMTP id
 d2e1a72fcca58-6e46dcd8feaso829831b3a.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:39:27 -0800 (PST)
Received: from localhost ([47.254.32.37]) by smtp.gmail.com with ESMTPSA id
 l3-20020a62be03000000b006e04553a4c5sm4068362pff.52.2024.02.26.06.39.24
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Mon, 26 Feb 2024 06:39:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d707ea55-d4b4-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708958365; x=1709563165; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=3obabIc0frWAPcp78SCT093DaVOW3OCMmYlsqxV6Jrg=;
        b=fTKn5SUFKdupT+67UvhtsrTfu7gAxDb6UDnjbvB3BL/XJorkFUjeZw4KbRdnLms16g
         u2K6SRVVgz/GzvAdxKeUjyeBzQp88npJ0ALb9O6GsqtAgRX/TFACWmeymrvyerseeZzG
         kMrxPp/NUcfmMRGTj0rb0BvrG4Xsqh4OWYTodVEpt9NOCPTFC9iSSg5KqmYrSgSA1A6H
         UstkUqJzOTgYlMR9duAxgKcapO+yMN4ruLS84D7h9zEGOqG97hqRK36WcVu69lN0uOc6
         s//KkMXTZO79qC8O/LVAFXj+1ssjJaDgoRsgZKCfUGvkw3SnwKBKQV8bKYOr4wM2G/hc
         lfHw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708958365; x=1709563165;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=3obabIc0frWAPcp78SCT093DaVOW3OCMmYlsqxV6Jrg=;
        b=OIgkP47wUEZQvUfE0rd0pOBIWGIEbVz9l0oyzKYnu8wSBtncBZMW/YAKZp4dGJ02of
         tv18qU9BGxYhMz/yAxc/TFvohrkw5mRHcjN5VTjrS3IGJnAS38F5iIIRSh2rb0KUSO6k
         P/P8jC53ipJnT/xgPmfxfHIrlWiMr/l7rFVwMXP9SDhV7Uw6gEaGgPUV6bmqH9GaM/FS
         x1CpH5gHqj6aF39NSJ6Ko/7yc73eEHfEpYC6QczNoUHKqX7khOa/qtvqK2TiNjIvG94M
         jDp6LngVpn+YtBhFkiPGYc6/KeaSRNXrKPDV+kmADihq3aZLTEWj/gQmiazbXyfHcN/k
         Jc3g==
X-Forwarded-Encrypted: i=1; AJvYcCXwaHz8QM/TO0nAXI6F/Rg2ug83SSuYdAkn/6A38mFUm5oYSTI/HlFxpC2YXnxTY10yCO5WjwrmjVdQz0c7nzw6tCBMTXbOgc67CaRektk=
X-Gm-Message-State: AOJu0YyUG6ck8K2plNlK7hd2IhliSy4xNyPjNoIAr+uk2ApUkr2khuka
	vXjtUXv+jVRuhQDWE1s0Rs+o1bPHVJT5MJnYiC8Tiz+l77X7y2DN
X-Google-Smtp-Source: AGHT+IHxB1SRDNUAQcM9eSd8jCrvT5j7mxZc3ehl/sTVGmSJcJcMTqd/bRM8vjNK9o0eC3pRQdhkvg==
X-Received: by 2002:a05:6a00:1817:b0:6e4:5a0f:b87a with SMTP id y23-20020a056a00181700b006e45a0fb87amr8639641pfa.12.1708958365514;
        Mon, 26 Feb 2024 06:39:25 -0800 (PST)
From: Lai Jiangshan <jiangshanlai@gmail.com>
To: linux-kernel@vger.kernel.org
Cc: Hou Wenlong <houwenlong.hwl@antgroup.com>,
	Lai Jiangshan <jiangshan.ljs@antgroup.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Sean Christopherson <seanjc@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>,
	Ingo Molnar <mingo@redhat.com>,
	kvm@vger.kernel.org,
	Paolo Bonzini <pbonzini@redhat.com>,
	x86@kernel.org,
	Kees Cook <keescook@chromium.org>,
	Juergen Gross <jgross@suse.com>,
	Andy Lutomirski <luto@kernel.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Ajay Kaher <akaher@vmware.com>,
	Alexey Makhalov <amakhalov@vmware.com>,
	VMware PV-Drivers Reviewers <pv-drivers@vmware.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	"Mike Rapoport (IBM)" <rppt@kernel.org>,
	Daniel Sneddon <daniel.sneddon@linux.intel.com>,
	Rick Edgecombe <rick.p.edgecombe@intel.com>,
	Alexey Kardashevskiy <aik@amd.com>,
	virtualization@lists.linux.dev,
	xen-devel@lists.xenproject.org
Subject: [RFC PATCH 66/73] x86/pvm: Use new cpu feature to describe XENPV and PVM
Date: Mon, 26 Feb 2024 22:36:23 +0800
Message-Id: <20240226143630.33643-67-jiangshanlai@gmail.com>
X-Mailer: git-send-email 2.19.1.6.gb485710b
In-Reply-To: <20240226143630.33643-1-jiangshanlai@gmail.com>
References: <20240226143630.33643-1-jiangshanlai@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Hou Wenlong <houwenlong.hwl@antgroup.com>

Some PVOPS are patched as the native version directly if the guest is
not a XENPV guest. However, this approach will not work after
introducing a PVM guest. To address this, use a new CPU feature to
describe XENPV and PVM, and ensure that those PVOPS are patched only
when it is not a paravirtual guest.

Signed-off-by: Hou Wenlong <houwenlong.hwl@antgroup.com>
Signed-off-by: Lai Jiangshan <jiangshan.ljs@antgroup.com>
---
 arch/x86/entry/entry_64.S          |  5 ++---
 arch/x86/include/asm/cpufeatures.h |  1 +
 arch/x86/include/asm/paravirt.h    | 14 +++++++-------
 arch/x86/kernel/pvm.c              |  1 +
 arch/x86/xen/enlighten_pv.c        |  1 +
 5 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index fe12605b3c05..6b41a1837698 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -127,9 +127,8 @@ SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL)
 	 * In the PVM guest case we must use eretu synthetic instruction.
 	 */
 
-	ALTERNATIVE_2 "testb %al, %al; jz swapgs_restore_regs_and_return_to_usermode", \
-		"jmp swapgs_restore_regs_and_return_to_usermode", X86_FEATURE_XENPV, \
-		"jmp swapgs_restore_regs_and_return_to_usermode", X86_FEATURE_KVM_PVM_GUEST
+	ALTERNATIVE "testb %al, %al; jz swapgs_restore_regs_and_return_to_usermode", \
+		"jmp swapgs_restore_regs_and_return_to_usermode", X86_FEATURE_PV_GUEST
 
 	/*
 	 * We win! This label is here just for ease of understanding
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index e17e72f13423..72ef58a2db19 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -238,6 +238,7 @@
 #define X86_FEATURE_VCPUPREEMPT		( 8*32+21) /* "" PV vcpu_is_preempted function */
 #define X86_FEATURE_TDX_GUEST		( 8*32+22) /* Intel Trust Domain Extensions Guest */
 #define X86_FEATURE_KVM_PVM_GUEST	( 8*32+23) /* KVM Pagetable-based Virtual Machine guest */
+#define X86_FEATURE_PV_GUEST		( 8*32+24) /* "" Paravirtual guest */
 
 /* Intel-defined CPU features, CPUID level 0x00000007:0 (EBX), word 9 */
 #define X86_FEATURE_FSGSBASE		( 9*32+ 0) /* RDFSBASE, WRFSBASE, RDGSBASE, WRGSBASE instructions*/
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index deaee9ec575e..a864ee481ca2 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -143,7 +143,7 @@ static __always_inline unsigned long read_cr2(void)
 {
 	return PVOP_ALT_CALLEE0(unsigned long, mmu.read_cr2,
 				"mov %%cr2, %%rax;",
-				ALT_NOT(X86_FEATURE_XENPV));
+				ALT_NOT(X86_FEATURE_PV_GUEST));
 }
 
 static __always_inline void write_cr2(unsigned long x)
@@ -154,13 +154,13 @@ static __always_inline void write_cr2(unsigned long x)
 static inline unsigned long __read_cr3(void)
 {
 	return PVOP_ALT_CALL0(unsigned long, mmu.read_cr3,
-			      "mov %%cr3, %%rax;", ALT_NOT(X86_FEATURE_XENPV));
+			      "mov %%cr3, %%rax;", ALT_NOT(X86_FEATURE_PV_GUEST));
 }
 
 static inline void write_cr3(unsigned long x)
 {
 	PVOP_ALT_VCALL1(mmu.write_cr3, x,
-			"mov %%rdi, %%cr3", ALT_NOT(X86_FEATURE_XENPV));
+			"mov %%rdi, %%cr3", ALT_NOT(X86_FEATURE_PV_GUEST));
 }
 
 static inline void __write_cr4(unsigned long x)
@@ -694,17 +694,17 @@ bool __raw_callee_save___native_vcpu_is_preempted(long cpu);
 static __always_inline unsigned long arch_local_save_flags(void)
 {
 	return PVOP_ALT_CALLEE0(unsigned long, irq.save_fl, "pushf; pop %%rax;",
-				ALT_NOT(X86_FEATURE_XENPV));
+				ALT_NOT(X86_FEATURE_PV_GUEST));
 }
 
 static __always_inline void arch_local_irq_disable(void)
 {
-	PVOP_ALT_VCALLEE0(irq.irq_disable, "cli;", ALT_NOT(X86_FEATURE_XENPV));
+	PVOP_ALT_VCALLEE0(irq.irq_disable, "cli;", ALT_NOT(X86_FEATURE_PV_GUEST));
 }
 
 static __always_inline void arch_local_irq_enable(void)
 {
-	PVOP_ALT_VCALLEE0(irq.irq_enable, "sti;", ALT_NOT(X86_FEATURE_XENPV));
+	PVOP_ALT_VCALLEE0(irq.irq_enable, "sti;", ALT_NOT(X86_FEATURE_PV_GUEST));
 }
 
 static __always_inline unsigned long arch_local_irq_save(void)
@@ -776,7 +776,7 @@ void native_pv_lock_init(void) __init;
 .endm
 
 #define SAVE_FLAGS	ALTERNATIVE "PARA_IRQ_save_fl;", "pushf; pop %rax;", \
-				    ALT_NOT(X86_FEATURE_XENPV)
+				    ALT_NOT(X86_FEATURE_PV_GUEST)
 #endif
 #endif /* CONFIG_PARAVIRT_XXL */
 #endif	/* CONFIG_X86_64 */
diff --git a/arch/x86/kernel/pvm.c b/arch/x86/kernel/pvm.c
index c38e46a96ad3..d39550a8159f 100644
--- a/arch/x86/kernel/pvm.c
+++ b/arch/x86/kernel/pvm.c
@@ -300,6 +300,7 @@ void __init pvm_early_setup(void)
 		return;
 
 	setup_force_cpu_cap(X86_FEATURE_KVM_PVM_GUEST);
+	setup_force_cpu_cap(X86_FEATURE_PV_GUEST);
 
 	wrmsrl(MSR_PVM_VCPU_STRUCT, __pa(this_cpu_ptr(&pvm_vcpu_struct)));
 	wrmsrl(MSR_PVM_EVENT_ENTRY, (unsigned long)(void *)pvm_early_kernel_event_entry - 256);
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index aeb33e0a3f76..c56483051528 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -335,6 +335,7 @@ static bool __init xen_check_xsave(void)
 static void __init xen_init_capabilities(void)
 {
 	setup_force_cpu_cap(X86_FEATURE_XENPV);
+	setup_force_cpu_cap(X86_FEATURE_PV_GUEST);
 	setup_clear_cpu_cap(X86_FEATURE_DCA);
 	setup_clear_cpu_cap(X86_FEATURE_APERFMPERF);
 	setup_clear_cpu_cap(X86_FEATURE_MTRR);
-- 
2.19.1.6.gb485710b



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:47:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:47:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685591.1066548 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1recGI-0007Y4-VC; Mon, 26 Feb 2024 14:47:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685591.1066548; Mon, 26 Feb 2024 14:47:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1recGI-0007Xx-RT; Mon, 26 Feb 2024 14:47:22 +0000
Received: by outflank-mailman (input) for mailman id 685591;
 Mon, 26 Feb 2024 14:47:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1recGH-0007Xr-LL
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:47:21 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f0e348d0-d4b5-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 15:47:19 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-5640fef9fa6so4087958a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:47:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g14-20020a056402320e00b00564e4debe29sm2423346eda.78.2024.02.26.06.47.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 06:47:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0e348d0-d4b5-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708958839; x=1709563639; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Z3XJgoMYHrTLdYZwssR39Cz6lXZjPT6Pt62MCOp6UDs=;
        b=ADqIoZzAJ3SHZkxM06M1zbxWuyy2VI1HSvqcfQcGLXWsu14U1HFtVTYZ732I0H0ViN
         wDXGhO28ahnE7nvruwUZQtF/1gKUrjSON6z4ON6Ws0OIgWiMjt4tdy5pYKtOE+yQbiTb
         TB1QL2IO7EAs/TxZqGifmnpExVbAjFpQdWL4Q5ZppjrDyqizbmXWVGqPPDgzSGVZC47P
         2vDVJfBjr1UxjcetkNG+bDfcsi7cnBUwo6LQlSyuS7k4MFPRkOLtNX47BdVPyfjA8wji
         IUeA9OGceyW2rjsPbj0qzWonmskod6YQcQUpw7xbfcm6DjfoIr94z1zXDJb32/Ye/eke
         emFw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708958839; x=1709563639;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Z3XJgoMYHrTLdYZwssR39Cz6lXZjPT6Pt62MCOp6UDs=;
        b=A4vk4SsmR2Eap7kfWCRQNOEfWDUysJNBXtoS6xrJSIwVTMbW0ANpU1v9SA4RakFQ73
         t11fra2oqWidUz0R3DsSyJF0Os223qngZh2zC5YW5JElmzfDReMii6Uzokx+4FtFkBbW
         0rrDYuAA00FWJpb8eqbXcJchiB1Zp/R30Vc1ikxPragy6QW07cygiW/1+uv4C+t3E2i0
         jWJIbfHE58m7GnUznm1OLlU+tBtcoo9bli/G4z4uSF3u6UdZkG2/CeoqBOY2k51s39Kd
         kgEN3iwAoVRIX9wJ4LmXpXM9mYITkRcfv9I1/3fVQz8GTgsMGPhB5SHLHdIjTnr5jN67
         Xfvg==
X-Forwarded-Encrypted: i=1; AJvYcCXlTbY2fF+ab9eyxVrBgyWrwPq9vhzUjG03GK+qG4dVWJHrYEmzNryD2LGJKxCVOcact1ivTY4g4xwjsDPdIg78Hzso7lMrkVDc5ZDQe9I=
X-Gm-Message-State: AOJu0Yx+I1faDd+RI39aOCPZGGsrBvR6b9Ii+TIZE6d+dwKzjgYRIzEl
	p196jrXFawOI72gpmNqpBTC5kZEhMrZAJ8Ey39kwK1iIZE2mG0LQUmMPaOzULA==
X-Google-Smtp-Source: AGHT+IEFZ7hgnAtH6j0a4SYKRjEADqXNEyuZQmLKR8RC1Q/hSqSQ2gc15lqkPXWwXjREi3cJJXW/7Q==
X-Received: by 2002:a05:6402:5246:b0:565:e282:33d8 with SMTP id t6-20020a056402524600b00565e28233d8mr3121954edd.38.1708958838669;
        Mon, 26 Feb 2024 06:47:18 -0800 (PST)
Message-ID: <6024f2c9-90e6-4ce9-a421-c0ec7441a926@suse.com>
Date: Mon, 26 Feb 2024 15:47:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13.3 01/14] vpci: use per-domain PCI lock to protect vpci
 structure
Content-Language: en-US
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Jun Nakajima <jun.nakajima@intel.com>, Kevin Tian <kevin.tian@intel.com>,
 Paul Durrant <paul@xen.org>, Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <20240221024504.357840-1-stewart.hildebrand@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240221024504.357840-1-stewart.hildebrand@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 21.02.2024 03:45, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Use the per-domain PCI read/write lock to protect the presence of the
> pci device vpci field. This lock can be used (and in a few cases is used
> right away) so that vpci removal can be performed while holding the lock
> in write mode. Previously such removal could race with vpci_read for
> example.
> 
> When taking both d->pci_lock and pdev->vpci->lock, they should be
> taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
> possible deadlock situations.
> 
> 1. Per-domain's pci_lock is used to protect pdev->vpci structure
> from being removed.
> 
> 2. Writing the command register and ROM BAR register may trigger
> modify_bars to run, which in turn may access multiple pdevs while
> checking for the existing BAR's overlap. The overlapping check, if
> done under the read lock, requires vpci->lock to be acquired on both
> devices being compared, which may produce a deadlock. It is not
> possible to upgrade read lock to write lock in such a case. So, in
> order to prevent the deadlock, use d->pci_lock in write mode instead.
> 
> All other code, which doesn't lead to pdev->vpci destruction and does
> not access multiple pdevs at the same time, can still use a
> combination of the read lock and pdev->vpci->lock.
> 
> 3. Drop const qualifier where the new rwlock is used and this is
> appropriate.
> 
> 4. Do not call process_pending_softirqs with any locks held. For that
> unlock prior the call and re-acquire the locks after. After
> re-acquiring the lock there is no need to check if pdev->vpci exists:
>  - in apply_map because of the context it is called (no race condition
>    possible)
>  - for MSI/MSI-X debug code because it is called at the end of
>    pdev->vpci access and no further access to pdev->vpci is made
> 
> 5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
> while accessing pdevs in vpci code.
> 
> 6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
> do not go away. The vPCI functions call several MSI-related functions
> which already have existing non-vPCI callers. Change those MSI-related
> functions to allow using either pcidevs_lock() or d->pci_lock for
> ensuring pdevs do not go away. Holding d->pci_lock in read mode is
> sufficient. Note that this pdev protection mechanism does not protect
> other state or critical sections. These MSI-related functions already
> have other race condition and state protection mechanims (e.g.
> d->event_lock and msixtbl RCU), so we deduce that the use of the global
> pcidevs_lock() is to ensure that pdevs do not go away.
> 
> 7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
> that pdevs do not go away. The purpose of this wrapper is to aid
> readability and document the intent of the pdev protection mechanism.
> 
> 8. When possible, the existing non-vPCI callers of these MSI-related
> functions haven't been switched to use the newly introduced per-domain
> pci_lock, and will continue to use the global pcidevs_lock(). This is
> done to reduce the risk of the new locking scheme introducing
> regressions. Those users will be adjusted in due time. One exception
> is where the pcidevs_lock() in allocate_and_map_msi_pirq() is moved to
> the caller, physdev_map_pirq(): this instance is switched to
> read_lock(&d->pci_lock) right away.
> 
> Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
with two small remaining remarks (below) and on the assumption that an
R-b from Roger in particular for the vPCI code is going to turn up
eventually.

> @@ -895,6 +891,15 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
>  {
>      unsigned int i;
>  
> +    /*
> +     * Assert that pdev_list doesn't change. ASSERT_PDEV_LIST_IS_READ_LOCKED
> +     * is not suitable here because it may allow either pcidevs_lock() or
> +     * pci_lock to be held, but here we rely on pci_lock being held, not
> +     * pcidevs_lock().
> +     */
> +    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
> +    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));

As to the comment, I think it's not really "may". I also think referral to
...

> @@ -913,13 +918,23 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
>              struct pci_dev *pdev = msix->pdev;
>  
>              spin_unlock(&msix->pdev->vpci->lock);
> +            read_unlock(&pdev->domain->pci_lock);
>              process_pending_softirqs();
> +
> +            if ( !read_trylock(&pdev->domain->pci_lock) )
> +                return -EBUSY;
> +
>              /* NB: we assume that pdev cannot go away for an alive domain. */
>              if ( !pdev->vpci || !spin_trylock(&pdev->vpci->lock) )
> +            {
> +                read_unlock(&pdev->domain->pci_lock);
>                  return -EBUSY;
> +            }
> +
>              if ( pdev->vpci->msix != msix )
>              {
>                  spin_unlock(&pdev->vpci->lock);
> +                read_unlock(&pdev->domain->pci_lock);
>                  return -EAGAIN;
>              }
>          }

... this machinery would be quite helpful (and iirc you even had such in an
earlier version).

> @@ -313,17 +316,31 @@ void vpci_dump_msi(void)
>                  {
>                      /*
>                       * On error vpci_msix_arch_print will always return without
> -                     * holding the lock.
> +                     * holding the locks.
>                       */
>                      printk("unable to print all MSI-X entries: %d\n", rc);
> -                    process_pending_softirqs();
> -                    continue;
> +                    goto pdev_done;
>                  }
>              }
>  
> +            /*
> +             * Unlock locks to process pending softirqs. This is
> +             * potentially unsafe, as d->pdev_list can be changed in
> +             * meantime.
> +             */
>              spin_unlock(&pdev->vpci->lock);
> +            read_unlock(&d->pci_lock);
> +        pdev_done:
>              process_pending_softirqs();
> +            if ( !read_trylock(&d->pci_lock) )
> +            {
> +                printk("unable to access other devices for the domain\n");
> +                goto domain_done;
> +            }
>          }
> +        read_unlock(&d->pci_lock);
> +    domain_done:
> +        ;

I think a blank line ahead of this label and perhaps also ahead of
"pdev_done" would be quite nice.

I guess respective adjustments could be done while committing, provided
there's not going to be any other reason for yet another revision.

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:49:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:49:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685594.1066558 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1recI5-00089m-CH; Mon, 26 Feb 2024 14:49:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685594.1066558; Mon, 26 Feb 2024 14:49:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1recI5-00089f-8x; Mon, 26 Feb 2024 14:49:13 +0000
Received: by outflank-mailman (input) for mailman id 685594;
 Mon, 26 Feb 2024 14:49:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fUdy=KD=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1recI3-00088I-NC
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:49:11 +0000
Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com
 [2a00:1450:4864:20::431])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 33199918-d4b6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 15:49:10 +0100 (CET)
Received: by mail-wr1-x431.google.com with SMTP id
 ffacd0b85a97d-33d6cc6d2fcso2028046f8f.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:49:10 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 a7-20020a056000188700b0033d926bf7b5sm8674905wri.76.2024.02.26.06.49.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 06:49:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 33199918-d4b6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1708958950; x=1709563750; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=dOUPYvVjo/EehxOKym7+j+5M3I7j7CMnd3AkYufTkp8=;
        b=RoukmiVf0NoYw2rT4JVHl9eGRmzGGcTtuD1wra2oMkDr6YZgwvfFxsRJ4g3S+z6iRD
         VHsotCXz8FK2O83Xz916TJJFV24rKhwgpuzXQzVmJXik4xxwvZnKLXiyhxYnCWw35/5J
         l6filgaETaAor4H9wu86VVdu0wl3ku68tLUxc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708958950; x=1709563750;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dOUPYvVjo/EehxOKym7+j+5M3I7j7CMnd3AkYufTkp8=;
        b=otlxloPOINlDY6s1d/TAh6IBFly7VGM6LWL/QiWJ1z5h6nwVP+ZKbACpgUFOr/Nnet
         4BgOJNYTkrifCdqNq/tqJZAfC8evPH28DXi3LERpzF4/OJZ4H5/SjDYSqNWFgB1YBRL7
         6gpFMpv+JZv2doprpI8H89HwCXFuQ+mxOU7KOTV5Db0gvGA7/ac63MilvXHank0B//XF
         XlvyyXkJ9fOLB1Vdjlb3VFqlM5hB+aQCVdMaDHzHJq+ulEv994IsnxalfKo2RSFyTq7P
         gQlYi5oiS3hzdXro4CydjzriCFeBXd0tHiCR/v+qzJuKEpGQiJGXP5aurIif+XllvWw6
         uIQg==
X-Forwarded-Encrypted: i=1; AJvYcCWUmozP8KJ+7RqQiOKHhZG19IhZdxjfoY5vSD71QVTy90w9+k88bLxPMH3rzrsDEtCh9yVLmF5UUEW/Q2oXba6ISZn2UHmg0p0VED4GEqY=
X-Gm-Message-State: AOJu0Yxshn5yvecfT2BrePcYfoP/QW6AbzCnkrGpZjJ+LvrJi0EFpY+2
	WfwA4fdmuIxk4uQ7d3aGjIFbbnHnbUkIQKUhyQE/n4byR7SzlC/paeLBeAStbec=
X-Google-Smtp-Source: AGHT+IF1XKTTgvrT37Xr+PlwAeQdI3WPvGyoP3L4v2lDvEB10hkYntEjsUVVek02bAzJDv0ucRQ4IA==
X-Received: by 2002:a5d:52cb:0:b0:33d:89a8:6b99 with SMTP id r11-20020a5d52cb000000b0033d89a86b99mr4198585wrv.70.1708958949758;
        Mon, 26 Feb 2024 06:49:09 -0800 (PST)
Date: Mon, 26 Feb 2024 14:49:08 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH] automation: Rework "build-each-commit-gcc" test
Message-ID: <d2cef00e-019d-44d6-9c12-a628455aea88@perard>
References: <20240220140733.40980-1-anthony.perard@citrix.com>
 <e48c4fce-e7f3-4bce-bf7b-2cfcc0659426@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <e48c4fce-e7f3-4bce-bf7b-2cfcc0659426@suse.com>

On Mon, Feb 26, 2024 at 10:23:37AM +0100, Jan Beulich wrote:
> On 20.02.2024 15:07, Anthony PERARD wrote:
> > --- a/automation/scripts/build-test.sh
> > +++ b/automation/scripts/build-test.sh
> > @@ -9,6 +9,37 @@
> >  # Set NON_SYMBOLIC_REF=3D1 if you want to use this script in detached =
HEAD state.
> >  # This is currently used by automated test system.
> > =20
> > +# Colors with ANSI escape sequences
> > +txt_info=3D'=1B[32m'
> > +txt_err=3D'=1B[31m'
> > +txt_clr=3D'=1B[0m'
> > +
> > +# $GITLAB_CI should be "true" or "false".
> > +if [ "$GITLAB_CI" !=3D true ]; then
> > +    GITLAB_CI=3Dfalse
> > +fi
> > +
> > +gitlab_log_section() {
> > +    if $GITLAB_CI; then
> > +        echo -n "=1B[0Ksection_$1:$(date +%s):$2
> > =1B[0K"
>=20
> ... there was either corruption on transmit here, or there's an embedded
> newline that I don't know how to deal with.

No corruption here, there is a \r in this string. There's also \e a few
times. Most tools can deal with these characters just fine, so I didn't
even think there could be an issue.

If that byte is really an issue, I could rewrite the patch to use
printf, and I think it would read as:

        printf "\e[0Ksection_$1:$(date +%s):$2\r\e[0K"

Thanks,

--=20
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 14:58:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 14:58:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685599.1066568 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1recQx-0002CM-5T; Mon, 26 Feb 2024 14:58:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685599.1066568; Mon, 26 Feb 2024 14:58:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1recQx-0002CF-2m; Mon, 26 Feb 2024 14:58:23 +0000
Received: by outflank-mailman (input) for mailman id 685599;
 Mon, 26 Feb 2024 14:58:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1recQv-0002C8-8e
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 14:58:21 +0000
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com
 [2a00:1450:4864:20::22f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7a59f019-d4b7-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 15:58:19 +0100 (CET)
Received: by mail-lj1-x22f.google.com with SMTP id
 38308e7fff4ca-2d27184197cso37617391fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 06:58:19 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 de38-20020a05620a372600b0078749c88a19sm2533729qkb.0.2024.02.26.06.58.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 06:58:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a59f019-d4b7-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708959499; x=1709564299; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KuHciCMiUtxsCMKWCTP1a/IkFfjhZRlK6gsKtMae7oY=;
        b=n6XYWJbraHviR0+108RVBAyw/dP7gVKly5Ear4vFWq6tDoOTPYAiLoR+2yxTfmKAx8
         qQhb1O4wLiZi8qIdjjDObcBUI5apllXBSNZj96mZ3u0n+qb1DsCV0YrwbAp9dLEmbrU5
         ImYX5M+r+uQEfCJjs1qfJqCiqcjURKnX2ScXI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708959499; x=1709564299;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KuHciCMiUtxsCMKWCTP1a/IkFfjhZRlK6gsKtMae7oY=;
        b=O3kJS1zOZ+ZHQqdhR/CegoA/JROPWNFav/s+QmrXeIQWaN6H0H7x8YzSUnM2e5TLTB
         ZNR9vKw9lBUDoych3D5mYLgUfGwq/VScWQJEmcYnGOeHoFxMHQJ9KdNFixwyNWTK3odu
         KIB3BG6M9/0xYCxcWUB/Kh638yLZTvxMJlw6JPq7ZowuG2w5iB+s7VDXsKR9WjRjct/c
         3a+RK52WpnlYT2ijOtx6xRlOm4ux4OM+biVU4Up/n3Xqj1tYnbCRjJrjKkzOtVyxiymv
         K6E4YV4HWtS7CCt3NLy8/83DHMS3DNxERILtVO3RF9uYmBB950I8o3F+OUUdE6D/Hop2
         j4tA==
X-Forwarded-Encrypted: i=1; AJvYcCUK2pv7DK2fFwoGOYkqKgPCkE5aH/iXJ1ZW+MSZA+OJx7pi8GX2bwOj5VfiQJygqkiUkT9tZ6kOXpc4eaj3bVcRA9a3nWfvf41SSoul8FQ=
X-Gm-Message-State: AOJu0YxAUAnGtqp+/NimNcvOUCgDCWsofnDBgFXnYBPEF5IpT1EPjuE7
	hoP/abAYf4di8Y+fWpzH5MoAsl11tXSK/ONvSvdYigp0ibKnWlVYEpuu5pzKeYM=
X-Google-Smtp-Source: AGHT+IHazFue5sC/kKfus/eRWOoVcf0/70OTD4Imy1YQYFTHv0AgoTSCiPRy8I94aqraHVuZxtqX3g==
X-Received: by 2002:a05:651c:211a:b0:2d2:8bbf:8fc2 with SMTP id a26-20020a05651c211a00b002d28bbf8fc2mr2205533ljq.34.1708959498811;
        Mon, 26 Feb 2024 06:58:18 -0800 (PST)
Message-ID: <7cc656d6-4bf0-4b53-ab95-5d49238cc876@citrix.com>
Date: Mon, 26 Feb 2024 14:58:16 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/3] x86/entry: Introduce EFRAME_* constants
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240226125501.1233599-1-andrew.cooper3@citrix.com>
 <20240226125501.1233599-4-andrew.cooper3@citrix.com>
 <01621599-7d1f-4cfb-9844-06fba32a62aa@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <01621599-7d1f-4cfb-9844-06fba32a62aa@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26/02/2024 2:32 pm, Jan Beulich wrote:
> On 26.02.2024 13:55, Andrew Cooper wrote:
>> restore_all_guest() does a lot of manipulation of the stack after popping the
>> GPRs, and uses raw %rsp displacements to do so.  Also, almost all entrypaths
>> use raw %rsp displacements prior to pushing GPRs.
>>
>> Provide better mnemonics, to aid readability and reduce the chance of errors
>> when editing.
>>
>> No functional change.  The resulting binary is identical.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> with one small request:
>
>> --- a/xen/arch/x86/x86_64/asm-offsets.c
>> +++ b/xen/arch/x86/x86_64/asm-offsets.c
>> @@ -51,6 +51,23 @@ void __dummy__(void)
>>      OFFSET(UREGS_kernel_sizeof, struct cpu_user_regs, es);
>>      BLANK();
>>  
>> +    /*
>> +     * EFRAME_* is for the entry/exit logic where %rsp is pointing at
>> +     * UREGS_error_code and GPRs are still guest values.
>> +     */
> "still/already" or some such to match "entry/exit"?

Ok.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 15:03:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 15:03:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685602.1066577 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1recWD-0004L6-Oz; Mon, 26 Feb 2024 15:03:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685602.1066577; Mon, 26 Feb 2024 15:03:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1recWD-0004Kz-MQ; Mon, 26 Feb 2024 15:03:49 +0000
Received: by outflank-mailman (input) for mailman id 685602;
 Mon, 26 Feb 2024 15:03:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1recWC-0004Kt-Tm
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 15:03:48 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3d85154d-d4b8-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 16:03:46 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-512ab55fde6so3980258e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 07:03:46 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 on33-20020a05621444a100b0068fcd643b9dsm2994029qvb.22.2024.02.26.07.03.45
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 07:03:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d85154d-d4b8-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708959826; x=1709564626; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rr6N2E4s3ZIVhqfK7Ixg4PBr82pi6VcdNcQD+020dwc=;
        b=pzMFHpzOwBaIY0g2SvEJA//FI6kHyX4r33L+gqeJiuQYvwcTF6UbYBSjrDYXkFQRVN
         YNf9/mIZE4KT+oqWqeXLDs01+mfh+S/05rXX44hz6OckmFnlGRirgL7gZTCmc1LUvHRJ
         c3h04y7ISuCbUImaQtETGFy078974h6nF801c=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708959826; x=1709564626;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rr6N2E4s3ZIVhqfK7Ixg4PBr82pi6VcdNcQD+020dwc=;
        b=YTqrZugViGjeVNbEVdyDhQh7by3/A8ozWjqwtLWRUs3fhdkPtp59axl68ETvzNIbPC
         XJ0YiJW8rlXCt732LNm4m2bq4q64nIBTymU/141xMBRITZ0dm+Ubu06JpX6gQJgDN0Bl
         Xy+7qGsm+1SP1978k+oAtkB2RsZSRciE59G3ICc8pnNYmECcGCxgWW/EbK3x0UjaYmQh
         CTI6qJLtvvLCnlqmiqgC3CRoR8xQkhYYetVPaREGHbO7Ib3cTiFvBZoUg5/mw9H80Lzp
         UBGrLbYFuC2ZyCvMH8KVx+DbFbthcFNJnyrMfIzJwd+Tydxs28v1FBb6WsrLy+V8FwsE
         aMuQ==
X-Forwarded-Encrypted: i=1; AJvYcCVzQQRFFt9mv4CN0Gva1UgWacdtP9Kb53K1euUxZq1XCNNmEoG2qcGC7+SbzNtiplPv1Ry4Kd/01mq5N6x1HDlQr8oxld4jZYIVent9j/M=
X-Gm-Message-State: AOJu0YyHtLFgD2BRDsZHiFzoEa1lCwSFReg0luxeKscrRgAlUweXI8a5
	+j73YcWZXRpNHwIaGxW5J26e9tWUHZXU42IsB2lTc9ym03xhXsfp+WRYNTQ1Qzc=
X-Google-Smtp-Source: AGHT+IHwkr0QNg4jlzXKXVUfUBASM6OMQOqvDwoDG0womIbrldHOThD8JHiGEYhgR1re4xV3o4e9lA==
X-Received: by 2002:a19:2d1d:0:b0:512:f73e:403b with SMTP id k29-20020a192d1d000000b00512f73e403bmr3000931lfj.19.1708959826323;
        Mon, 26 Feb 2024 07:03:46 -0800 (PST)
Message-ID: <e34cd78c-ce80-4a61-a1b9-8e3c9479df0f@citrix.com>
Date: Mon, 26 Feb 2024 15:03:43 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] tools/xentop: Add VBD3 support to xentop
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>, Fouad Hilly <fouad.hilly@cloud.com>
Cc: Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240226141211.1416866-1-fouad.hilly@cloud.com>
 <9468ccd8-44c3-4b50-b200-5a4ae1cca8ff@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <9468ccd8-44c3-4b50-b200-5a4ae1cca8ff@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26/02/2024 2:22 pm, Jan Beulich wrote:
> On 26.02.2024 15:12, Fouad Hilly wrote:
>> From: Pritha Srivastava <pritha.srivastava@citrix.com>
>>
>> xl now knows how to drive tapdisk, so modified libxenstat to
>> understand vbd3 statistics.
>>
>> Signed-off-by: Jorge Martin <jorge.martin@citrix.com>
>> Signed-off-by: Pritha Srivastava <pritha.srivastava@citrix.com>
> Just a formal question (I'm not really qualified to review this change):
> With the above two S-o-b and the earlier From: - who is the original
> author of this patch? The general expectation is for the 1st S-o-b to
> be the author's.

This patch has been sat in the XenServer patchqueue for a decade. 
Neither Pritha nor Jorge are with us any more.

Sadly the review system we used back then is also no longer with us. 
>From ticketing, I think it was co-developed at the same time.

This is the form the patch has existed in the patchqueue for that time,
so I'm tempted to say we reorder the SoB chain to make it match.  That's
the best I can figure out.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 15:10:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 15:10:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685604.1066588 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reccW-0006NO-F9; Mon, 26 Feb 2024 15:10:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685604.1066588; Mon, 26 Feb 2024 15:10:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reccW-0006NH-C1; Mon, 26 Feb 2024 15:10:20 +0000
Received: by outflank-mailman (input) for mailman id 685604;
 Mon, 26 Feb 2024 15:10:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reccV-0006NB-L8
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 15:10:19 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 260aa71b-d4b9-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 16:10:16 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-512f54c81d2so1667876e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 07:10:16 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 tk6-20020a170907c28600b00a4306ac4c77sm2027637ejc.197.2024.02.26.07.10.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 07:10:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 260aa71b-d4b9-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708960216; x=1709565016; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8tIbkpeNLAozugUEmgofrslaIe2aW51Xk6gGsC1waq4=;
        b=EBlxBlghvPN38b4dxtcpqYcFLLAu5DpQGXeK4PZSQt5ONcv3olrkDLHHiUdvjIPhOE
         G5zOowU8z7uNS5NKhy5Iic14NfLUpDfzBvLYJS7tWZCPf6Wbg7VaCmpKMvWP9Yv77T6U
         m4AspVvgz6XPa3WGZ2NL/mzAvE4qUXQKK6W06rpXuuxEW4vQ/EpVOgIC2Ho0h6u8njJY
         HEb3eutbBxlMVDSEuTuz0M/fbJBalINzN4Iwv4BDrj8OLYwyp6ipOL0cW+eWKQl69tWK
         fW3IpVim6GEPMlKcDdy2AC8inv1ikjTWqOBWDUjh/IJ17iDWrvfML4C3yFHqYFs91/Eb
         qWPQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708960216; x=1709565016;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8tIbkpeNLAozugUEmgofrslaIe2aW51Xk6gGsC1waq4=;
        b=tyY5K2yG7HZwTzGpfHz10PsFfJpfWUBztFAhfPtDuYvxVndT2FS+OcApTesRtN+SQc
         QOxHC+Xjp3wFcbN/f3aM10cQs3t5CTNwvTTOaVxxZbqqaL7vEwcVd5/NkLRlzuZvGYxa
         IwMWubfGl9oa3XaYlUQi3ygNfJxL78cTDVryqboM0nIpisicFNs0roEFEuZpUiyivOyi
         YrXuH3mShLk7zRQB3+Amd/nESPgnSG3W0PLpiKBKJhQlfz0jCu3S7WITEXrnn1nPtA3g
         zR2EAd0duk0U8W0bica63OoxKAcNw5HoXJSAGbPKnD83xrU2Lq1VhauRl7qLrUtQxzBi
         U+SQ==
X-Forwarded-Encrypted: i=1; AJvYcCVIrx7wJ8jq4HYvSGEaW6Hd92yWUdutTRTw8eSCgNGuuNjs6JrKviUZ/r8aE2jorzqfVzczh+ysA7XRwAN0LllHKbpozaYz+b9bubZEyUs=
X-Gm-Message-State: AOJu0Yw2G2aKcoNhH7hPDhzD1UhzDdbr/u6clsAKz3HIc3Q2kpOfNCu+
	rejR5sU+imX9cje5BYZy6SFuZVcxrUgODwMY8vN9ozhBXHIHW9qUexUnHEc5rA==
X-Google-Smtp-Source: AGHT+IE1qjn6qMl56PMhj+xQ/Y4Us0rvHI3X4iHVV2yuGB85Sa9OZtm+uCYZjZnNXSS65QQfd6REcg==
X-Received: by 2002:a05:6512:3da8:b0:512:f5b9:2e23 with SMTP id k40-20020a0565123da800b00512f5b92e23mr4368043lfv.11.1708960216359;
        Mon, 26 Feb 2024 07:10:16 -0800 (PST)
Message-ID: <38edbe43-307a-4f70-a099-d31e155df076@suse.com>
Date: Mon, 26 Feb 2024 16:10:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation: Rework "build-each-commit-gcc" test
Content-Language: en-US
To: Anthony PERARD <anthony.perard@cloud.com>
Cc: Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>, xen-devel@lists.xenproject.org
References: <20240220140733.40980-1-anthony.perard@citrix.com>
 <e48c4fce-e7f3-4bce-bf7b-2cfcc0659426@suse.com>
 <d2cef00e-019d-44d6-9c12-a628455aea88@perard>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d2cef00e-019d-44d6-9c12-a628455aea88@perard>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 15:49, Anthony PERARD wrote:
> On Mon, Feb 26, 2024 at 10:23:37AM +0100, Jan Beulich wrote:
>> On 20.02.2024 15:07, Anthony PERARD wrote:
>>> --- a/automation/scripts/build-test.sh
>>> +++ b/automation/scripts/build-test.sh
>>> @@ -9,6 +9,37 @@
>>>  # Set NON_SYMBOLIC_REF=1 if you want to use this script in detached HEAD state.
>>>  # This is currently used by automated test system.
>>>  
>>> +# Colors with ANSI escape sequences
>>> +txt_info='[32m'
>>> +txt_err='[31m'
>>> +txt_clr='[0m'
>>> +
>>> +# $GITLAB_CI should be "true" or "false".
>>> +if [ "$GITLAB_CI" != true ]; then
>>> +    GITLAB_CI=false
>>> +fi
>>> +
>>> +gitlab_log_section() {
>>> +    if $GITLAB_CI; then
>>> +        echo -n "[0Ksection_$1:$(date +%s):$2
>>> [0K"
>>
>> ... there was either corruption on transmit here, or there's an embedded
>> newline that I don't know how to deal with.
> 
> No corruption here, there is a \r in this string. There's also \e a few
> times. Most tools can deal with these characters just fine, so I didn't
> even think there could be an issue.

Okay, in an entirely different 2nd attempt I got it committed fine (I
hope). It's probably too old fashioned of me / my scripts that I
demand for every patch to pass a "patch --dry-run" first.

Jan

> If that byte is really an issue, I could rewrite the patch to use
> printf, and I think it would read as:
> 
>         printf "\e[0Ksection_$1:$(date +%s):$2\r\e[0K"
> 
> Thanks,
> 



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 15:44:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 15:44:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685620.1066613 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1red9N-0004YP-Ah; Mon, 26 Feb 2024 15:44:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685620.1066613; Mon, 26 Feb 2024 15:44:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1red9N-0004YI-8C; Mon, 26 Feb 2024 15:44:17 +0000
Received: by outflank-mailman (input) for mailman id 685620;
 Mon, 26 Feb 2024 15:44:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1red9M-0004YC-3W
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 15:44:16 +0000
Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com
 [2a00:1450:4864:20::435])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e41ad3c8-d4bd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 16:44:13 +0100 (CET)
Received: by mail-wr1-x435.google.com with SMTP id
 ffacd0b85a97d-33d90dfe73cso2214126f8f.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 07:44:13 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 b12-20020a05620a0f8c00b00787c1102df8sm2564765qkn.127.2024.02.26.07.44.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 07:44:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e41ad3c8-d4bd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708962253; x=1709567053; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=rbeISk6CLsumn+4LZjMFLRI8cm8qCnRIEsEzsFsiCgE=;
        b=QIJpKoSaund8E/11tsNdY3tI41fd+nv8ylpuJ2ogJPUTebtOCmHKvA/obIIgP3Thsz
         pIPw/N1TF4XQYTVp5s7z5K/Ny28hs22xULVpFqiiKpxBPSPlb5m1T/or08dJCc61XAIW
         /UikGA+PwyB73tkCOzbtNITAHGhtzPfjbmMTY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708962253; x=1709567053;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=rbeISk6CLsumn+4LZjMFLRI8cm8qCnRIEsEzsFsiCgE=;
        b=EiRmsX9tb+U4Vc3d2M0QrGFpWJrBOR/tiKi2vkYLweDAWTfBqaGQ87foKwTdGOLPxg
         IJ2toNP1fbSYHjzBcoES4pHifdkG6Kuyw2a4ZVzMhJ+IQZdK1PmgM7BPqEO0Y5A2dY9q
         sRvM2SjxD7plNFU+lO0Uhow3tPpxE6dzcpA0rRqn6k4oS/NMM8MykK5v5cJuzkKB3Dcu
         LUg4VkoDQX3Ay4GUa6dLE9/fqDkcRwY0xjWaQhR0AvBIyGTz2Uoe1N7e7copACe/Oqyj
         yEVYge40GEeFGm6U3Q1hvwEtESOsstEyQChYVhTUc7dyj9yteF3XtVIyZfUMU5AblbF6
         uAhw==
X-Forwarded-Encrypted: i=1; AJvYcCWjThjH+YtqkOhjtcDxKtND2Tn9DW6ZghxDpzwv/Ghnt7GTJJpCk37gXfJmOHo6sjJ0RPEZ4IKzFU1n1iybSWqArEKY2RY6Bz9KIS64abo=
X-Gm-Message-State: AOJu0YyBQRFkV2OXJS+Qt2Q+FRWwR36kVQyshbVTT2N4JgMq1p+nHfLb
	rUOOPX8QHywoW3MITrCUGOEIyE/T/PY6ynGEog7syAe4Oss2w4PPGZbofGl0698=
X-Google-Smtp-Source: AGHT+IG3ZgjP3oZEayxVy0ehnFNfIOUuZH9SltyvnNEikbNzr6Rx+ioIyftMziF1va9DEMToaZlgsQ==
X-Received: by 2002:a5d:58e5:0:b0:33d:9d76:2f82 with SMTP id f5-20020a5d58e5000000b0033d9d762f82mr6753723wrd.26.1708962253179;
        Mon, 26 Feb 2024 07:44:13 -0800 (PST)
Message-ID: <b549f8f3-b21d-4ca5-9ef8-e1c39856d691@citrix.com>
Date: Mon, 26 Feb 2024 15:44:10 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [REGRESSION] Re: [XEN PATCH v4 4/4] eclair: move function and macro
 properties outside ECLAIR
Content-Language: en-GB
To: Simone Ballarin <simone.ballarin@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>, George Dunlap
 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <387b160ae93c221c4bc2426605b96b432b26224e.1706886631.git.simone.ballarin@bugseng.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <387b160ae93c221c4bc2426605b96b432b26224e.1706886631.git.simone.ballarin@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 02/02/2024 3:16 pm, Simone Ballarin wrote:
> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
>
> Function and macro properties contained in ECLAIR/call_properties.ecl are of
> general interest: this patch moves these annotations in a generaric JSON file
> in docs. In this way, they can be exploited for other purposes (i.e. documentation,
> other tools).
>
> Add rst file containing explanation on how to update function_macro_properties.json.
> Add script to convert the JSON file in ECL configurations.
> Remove ECLAIR/call_properties.ecl: the file is now automatically generated from
> the JSON file.
>
> Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
>
> ---
> Changes in v4:
> - add missing script for converting the JSON file in ECL configurations;
> - improve commit message;
> - remove call_properties.ecs.
> ---
>  .../eclair_analysis/ECLAIR/analysis.ecl       |   1 +
>  .../ECLAIR/call_properties.ecl                | 128 ---
>  automation/eclair_analysis/prepare.sh         |   2 +
>  automation/eclair_analysis/propertyparser.py  |  37 +
>  docs/function_macro_properties.json           | 841 ++++++++++++++++++
>  docs/function_macro_properties.rst            |  58 ++
>  6 files changed, 939 insertions(+), 128 deletions(-)
>  delete mode 100644 automation/eclair_analysis/ECLAIR/call_properties.ecl
>  create mode 100644 automation/eclair_analysis/propertyparser.py
>  create mode 100644 docs/function_macro_properties.json
>  create mode 100644 docs/function_macro_properties.rst

This breaks the Sphinx build.

checking consistency...
/local/xen.git/docs/function_macro_properties.rst: WARNING: document
isn't included in any toctree

Also, the top level of docs really isn't an appropriate place to put
it.  Everything else is in docs/misra/.

When you've moved the files, you'll need to edit docs/misra/index.rst to
fix the build.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:25:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:25:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685631.1066639 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rednH-0003Dc-Mr; Mon, 26 Feb 2024 16:25:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685631.1066639; Mon, 26 Feb 2024 16:25:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rednH-0003DV-KL; Mon, 26 Feb 2024 16:25:31 +0000
Received: by outflank-mailman (input) for mailman id 685631;
 Mon, 26 Feb 2024 16:25:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rednG-0003DP-QA
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:25:30 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a7b513b3-d4c3-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:25:29 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a3d01a9a9a2so339525566b.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:25:29 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 lg25-20020a170907181900b00a43903e7a7csm170636ejc.51.2024.02.26.08.25.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 08:25:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a7b513b3-d4c3-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708964728; x=1709569528; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=+tmouwWJuYSCzk6pXS8mGFw3qvx7hu9XnjEfJTFEA2A=;
        b=GvcmF+5tXEsgfrOHgRi6vA0MTaxj3A9RVYrSK1FCno0C217aiod/QkmHGOKde00AKJ
         ysCSq4serYSOnkRV52nVZKY1ui/k34Uvvm7unKOu9hFofZemJ++kESldVFIb9XuITfum
         aR9qj8gRFor1bTYs5E8n6uLzmwjAJwW030I/g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708964728; x=1709569528;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=+tmouwWJuYSCzk6pXS8mGFw3qvx7hu9XnjEfJTFEA2A=;
        b=HhsRD2DKT9jwjmIBP4jfHQ8+rz+UR/w5so/6t+hfthlGkcoBD0ws+9y3KfV8WHBEju
         VWqAuHIExS8JJUa0xKhRU16UNRhbw+hglomj7CkWO/YDWdRo1blQbw0i3zI6hS732c5T
         nPGCKhmioDnGenK16PXkN2nkPMe7mILTFdpcGLNuwYTTqz6Csuw9rzHtiw8YO00gOfxD
         WeWe86ZssYt/WafpNRcUCa7ymoqHzEnELytDawDQOWnWMsM/Drv/ey55P3hCqj1OHeBX
         9HKoeq+OLJ9s8SzWvGKJQVpW7xgmYa0m3Qg9mVPuB4Ome1gIaHfN4pb3T6RpvpkA+hAS
         5jWQ==
X-Gm-Message-State: AOJu0YxkHxbksQEbHbOp55RVJdfN6n3IirlJNkMggUi3Jpxk5E0jELg7
	+QE/FPWjafzStvdEE83D9uHpoDLUiQxBa3uoabz2YRg3Q6dR2fIDXB5QW5bNuGR7vwc8CtXbH1v
	9
X-Google-Smtp-Source: AGHT+IHrm1Ok7e1tGMvLgtxKPn+GdKoxxK0rcGy5BpNkWAFsFVrJs1b0gEHgOo6/qf1GURM3PfmO9Q==
X-Received: by 2002:a17:906:f15a:b0:a3e:66b5:d65f with SMTP id gw26-20020a170906f15a00b00a3e66b5d65fmr4726665ejb.67.1708964728313;
        Mon, 26 Feb 2024 08:25:28 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>
Subject: [PATCH] docs/sphinx: Start an FAQ, and add Kconfig/CET details
Date: Mon, 26 Feb 2024 16:25:27 +0000
Message-Id: <20240226162527.1247421-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This is long overdue, and we need to start somewhere.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
---
 docs/faq.rst      | 71 +++++++++++++++++++++++++++++++++++++++++++++++
 docs/glossary.rst | 15 ++++++++++
 docs/index.rst    |  1 +
 3 files changed, 87 insertions(+)
 create mode 100644 docs/faq.rst

diff --git a/docs/faq.rst b/docs/faq.rst
new file mode 100644
index 000000000000..75cd70328a5c
--- /dev/null
+++ b/docs/faq.rst
@@ -0,0 +1,71 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Frequently Asked Questions
+==========================
+
+How do I...
+-----------
+
+... check whether a Kconfig option is active?
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+  Kconfig is a build time configuration system, combining inherent knowledge,
+  the capabilities of the toolchain, and explicit user choice to form a
+  configuration of a build of Xen.
+
+  A file, by default ``.config``, is produced by the build identifying the
+  configuration used.  Kconfig symbols all start with ``CONFIG_``, and come in
+  a variety of types including strings, integers and booleans.  Booleans are
+  the most common, and when active are expressed with ``...=y``.  e.g.::
+
+    xen.git/xen$ grep CONFIG_FOO .config
+    CONFIG_FOO_BOOLEAN=y
+    CONFIG_FOO_STRING="lorem ipsum"
+    CONFIG_FOO_INTEGER=42
+
+  Symbols which are either absent, or expressed as ``... is not set`` are
+  disabled.  e.g.::
+
+    xen.git/xen$ grep CONFIG_BAR .config
+    # CONFIG_BAR is not set
+
+  Builds of Xen configured with ``CONFIG_HYPFS_CONFIG=y`` embed their own
+  ``.config`` at build time, and can provide it to the :term:`control domain`
+  upon requested.  e.g.::
+
+    [root@host ~]# xenhypfs cat /buildinfo/config | grep -e FOO -e BAR
+    CONFIG_FOO=y
+    # CONFIG_BAR is not set
+
+
+... tell if CET is active?
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+  Control-flow Enforcement Technology support was added to Xen 4.14.  It is
+  build time conditional, dependent on both having a new-enough toolchain and
+  an explicit Kconfig option, and also requires capable hardware.  See
+  :term:`CET`.
+
+  For CET-SS, Shadow Stacks, the minimum toolchain requirements are ``binutils
+  >= 2.29`` or ``LLVM >= 6``.  No specific compiler support is required.
+  Check for ``CONFIG_XEN_SHSTK`` being active.
+
+  For CET-IBT, Indirect Branch Tracking, the minimum toolchain requirements
+  are ``GCC >= 9`` and ``binutils >= 2.29``.  Xen relies on a compiler feature
+  which is specific to GCC at the time of writing.  Check for
+  ``CONFIG_XEN_IBT`` being active.
+
+  If a capable Xen with booted on capable hardware, and CET is not disabled by
+  command line option or errata, Xen will print some details early on boot
+  about which CET facilities have been turned on::
+
+    ...
+    (XEN) CPU Vendor: Intel, Family 6 (0x6), Model 143 (0x8f), Stepping 8 (raw 000806f8)
+    (XEN) Enabling Supervisor Shadow Stacks
+    (XEN) Enabling Indirect Branch Tracking
+    (XEN)   - IBT disabled in UEFI Runtime Services
+    (XEN) EFI RAM map:
+    ...
+
+  This can be obtained from the control domain with ``xl dmesg``, but remember
+  to confirm that the console ring hasn't wrapped.
diff --git a/docs/glossary.rst b/docs/glossary.rst
index 8ddbdab160a1..6adeec77e14c 100644
--- a/docs/glossary.rst
+++ b/docs/glossary.rst
@@ -28,6 +28,21 @@ Glossary
      single instance of Xen, used as the identifier in various APIs, and is
      typically allocated sequentially from 0.
 
+   CET
+     Control-flow Enforcement Technology is a facility in x86 CPUs for
+     defending against memory safety vulnerabilities.  It is formed of two
+     independent features:
+
+     * CET-SS, Shadow Stacks, are designed to protect against Return Oriented
+       Programming (ROP) attacks.
+
+     * CET-IBT, Indirect Branch Tracking, is designed to protect against Call
+       or Jump Oriented Programming (COP/JOP) attacks.
+
+     Intel support CET-SS and CET-IBT from the Tiger Lake (Client, 2020) and
+     Sapphire Rapids (Server, 2023) CPUs.  AMD support only CET-SS, starting
+     with Zen3 (Both client and server, 2020) CPUs.
+
    guest
      The term 'guest' has two different meanings, depending on context, and
      should not be confused with :term:`domain`.
diff --git a/docs/index.rst b/docs/index.rst
index 22fdde80590c..ab051a0f3833 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -72,4 +72,5 @@ Miscellanea
 
 .. toctree::
 
+   faq
    glossary

base-commit: 60e00f77a5cc671d30c5ef3318f5b8e9b74e4aa3
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:40:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:40:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685634.1066650 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree1l-00068J-0W; Mon, 26 Feb 2024 16:40:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685634.1066650; Mon, 26 Feb 2024 16:40:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree1k-00068C-TD; Mon, 26 Feb 2024 16:40:28 +0000
Received: by outflank-mailman (input) for mailman id 685634;
 Mon, 26 Feb 2024 16:40:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree1k-000686-68
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:40:28 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bec5a5a9-d4c5-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:40:27 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-563c595f968so4331523a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:40:27 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 la22-20020a170906ad9600b00a3df13a4fe0sm2610636ejb.15.2024.02.26.08.40.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:40:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bec5a5a9-d4c5-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965626; x=1709570426; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=mAHNRAuYB35r/tg8UzZgPMbCFo1xfBZxNT28rnMxLh4=;
        b=MBwXxpTvKQdsecawDLdSJ2hmnz2gQFcsjy1V5YUdlYKz2ZDRof1y3gTOAnQxquYHf9
         gMiXwYVfKU2dSIIwLwlTL31I5zWaaRVfHi9tcWqwmdm/hLP9AHCMZv+7jS6uBVGsogHG
         b9bomRD9CnRNdMP67pJmQBMY1LBFlj6fw8Npf206GB95x1QMgBMM6brlRhZyQCs11Slr
         fXymeL+TyReOZAEzR4zl+jPNgejrg7fFDNxPUoXw8cMEE+GARJ+kxvl6U8Q9Wpp0SIVW
         njACv2Bq2jvKBYhoalwM9OyJZJmmBWXH0uAwKDoFxjqfkmpJMqn+O5C14X6lmLWIxBei
         T/ug==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965626; x=1709570426;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=mAHNRAuYB35r/tg8UzZgPMbCFo1xfBZxNT28rnMxLh4=;
        b=qpP9TW3rIHRTWFmEwj5TdTicdgfVJgy+7j9ZK18hfoKnGdrJAJcDXSkQX6KQzymyOr
         WbVRiPpy5qI0xMjVF1FVz+u2vib85DoJkqsNG1pbEEWczt/bhV8uoA2vex/4bWJgUOit
         W+R2KmKclnV5NbnzBZrAYcF8S1YdEnrr2YqR7CtaN/TGdfkR1YYmEBGxFakAIp1wbudJ
         WGV8mSvoBAmExyRczccHKlhADOEJcM2acxQGuSo4x6boXgcKPze7o3IFDtsCPR4toiyy
         mBADVItLSlYBihmxyBtYnHpM4MRYi5LTVMqIhEy5vOMizz4IfctImehLiPYjertNLi4r
         1DpQ==
X-Gm-Message-State: AOJu0YxWC3ZBDzgiMXjWYdydZHy7d9RRStW3V5K7fttG38oKmzYtsaD9
	HgjrqpfGVJhWfj2wzKQdb0ZLt0/wqm9XVy5tdCrnkh+ZJUdeoUUG0NOYTs9+Zjg2OmV2GTWYmEU
	=
X-Google-Smtp-Source: AGHT+IFzZTbfeA6/Gu2WGferDekLyB+goXbWrH3uPiwEeQgxxT0R26sUo5M6n6SjjzrYw9PUiHTYvw==
X-Received: by 2002:a17:906:69b:b0:a42:f3a6:9f7f with SMTP id u27-20020a170906069b00b00a42f3a69f7fmr3873194ejb.13.1708965626402;
        Mon, 26 Feb 2024 08:40:26 -0800 (PST)
Message-ID: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Date: Mon, 26 Feb 2024 17:40:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH v3 00/12] x86/HVM: misc tidying
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

01: VMX: don't run with CR4.VMXE set when VMX could not be enabled
02: x86/HVM: improve CET-IBT pruning of ENDBR
03: VMX: drop vmcs_revision_id
04: VMX: convert vmx_basic_msr
05: VMX: convert vmx_pin_based_exec_control
06: VMX: convert vmx_cpu_based_exec_control
07: VMX: convert vmx_secondary_exec_control
08: VMX: convert vmx_tertiary_exec_control
09: VMX: convert vmx_vmexit_control
10: VMX: convert vmx_vmentry_control
11: VMX: convert vmx_ept_vpid_cap
12: VMX: convert vmx_vmfunc

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:42:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:42:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685637.1066659 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree3L-0006er-9h; Mon, 26 Feb 2024 16:42:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685637.1066659; Mon, 26 Feb 2024 16:42:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree3L-0006ek-79; Mon, 26 Feb 2024 16:42:07 +0000
Received: by outflank-mailman (input) for mailman id 685637;
 Mon, 26 Feb 2024 16:42:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree3J-0006ea-GP
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:42:05 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f8c9f634-d4c5-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:42:04 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-563c595f968so4333920a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:42:04 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.42.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:42:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f8c9f634-d4c5-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965724; x=1709570524; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=6L5GGffCzRNgfzqvhyjL4MWez5VsJpAVC5R921GjUEE=;
        b=VVRBB1ynWbQeqlOy1mDrFYTUkp3QbwS2QrBa/75swhRLf5d5E78PYQXk8ZPiisxfcp
         UuGSLMK696rfC32SBH7FbGEQ8msGXU3WUed4s5/L1JmKhHnTZgDStqWnIvCXe/iH1vEG
         RGZl9sifExS5Mbh5JjHy0F0fIxrHahh0qbUpMbfhnUy4lAsDhEqtCThJVdxl3BeUEFD4
         u5W54zxWGn2mnRZEI7dSEY80dhfC32UgLW7VE7VfrgytZyw+Om7JU3EphngrjnUIBYnY
         iaoguAcIY06MeoLLatgAr1N3ecTvthkNf9ylDDN9DOWw4hHNmnEkeZGhh1E8Q5kUqe07
         aVbg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965724; x=1709570524;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=6L5GGffCzRNgfzqvhyjL4MWez5VsJpAVC5R921GjUEE=;
        b=qfiaO91j6/bERuGWoPDanph0UmDguDTQQowU4thqntOR7o3671QGnKq3BEx+ea+UeJ
         c0/tmZdY91+BvVuWCoistE+sn2uHe2qSTYX7ez//WuYmYTfBPwP2rJZFSPDCyx3fennM
         oqSJIl3MDuEHc+GdfDOTPSCcc8kGLBqpKa9Z5HnDMvOjWiw/XgUd2rwM9gHN/bQRQGFp
         F6SWpAW6bmpx3OXZt7a+G1RAH8L7zi9TNEGfROzhzlxJB922W86mcudV6d+JtiNACJi6
         mLAyV/4/w4M6yPKegKf4ofUzvMK8X66aNyPNHlaXqWI9N6urgdqNqA0RMlxGX6CaQRjV
         mULA==
X-Gm-Message-State: AOJu0Yz4XfqJDoxkyVVUeDvdNHEfGoe0rBTRlViLvtBb0EgNxKhPoAn+
	Jvod4cvdbDErMGBRvPEpvHQTL8OjywLYR0LkHkLgnj8wcFuetNM+yL+NbPtIpASgWviMFo+POQ4
	=
X-Google-Smtp-Source: AGHT+IFObpng2g0iR1LdcIGU+8vI61RJHJXBCIcZv6BTvuxx+tYrdIce4ZzeYalYq38YVjeDbHqyEA==
X-Received: by 2002:aa7:cb13:0:b0:566:2a1c:9c0f with SMTP id s19-20020aa7cb13000000b005662a1c9c0fmr187597edt.16.1708965723871;
        Mon, 26 Feb 2024 08:42:03 -0800 (PST)
Message-ID: <21e6cbec-dc29-452f-b6a2-6926245a8beb@suse.com>
Date: Mon, 26 Feb 2024 17:42:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 01/12] VMX: don't run with CR4.VMXE set when VMX could not
 be enabled
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

While generally benign, doing so is still at best misleading.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Using set_in_cr4() seems favorable over updating mmu_cr4_features
despite the resulting redundant CR4 update. But I certainly could be
talked into going the alternative route.
---
v2: Actually clear CR4.VMXE for the BSP on the error path.

--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -2961,14 +2961,18 @@ static bool __init has_if_pschange_mc(vo
 
 const struct hvm_function_table * __init start_vmx(void)
 {
-    set_in_cr4(X86_CR4_VMXE);
+    write_cr4(read_cr4() | X86_CR4_VMXE);
 
     if ( vmx_vmcs_init() )
     {
+        write_cr4(read_cr4() & ~X86_CR4_VMXE);
         printk("VMX: failed to initialise.\n");
         return NULL;
     }
 
+    /* Arrange for APs to have CR4.VMXE set early on. */
+    set_in_cr4(X86_CR4_VMXE);
+
     vmx_function_table.singlestep_supported = cpu_has_monitor_trap_flag;
 
     if ( cpu_has_vmx_dt_exiting )



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:42:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:42:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685638.1066669 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree3c-00073d-Gp; Mon, 26 Feb 2024 16:42:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685638.1066669; Mon, 26 Feb 2024 16:42:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree3c-00073W-E9; Mon, 26 Feb 2024 16:42:24 +0000
Received: by outflank-mailman (input) for mailman id 685638;
 Mon, 26 Feb 2024 16:42:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree3b-000739-FG
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:42:23 +0000
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
 [2a00:1450:4864:20::531])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 02e920d9-d4c6-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 17:42:21 +0100 (CET)
Received: by mail-ed1-x531.google.com with SMTP id
 4fb4d7f45d1cf-563fe793e1cso4459006a12.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:42:21 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.42.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:42:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02e920d9-d4c6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965741; x=1709570541; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=856hL3x+HrIg0hf4NEQ6xzQ16mC6OszsgQrB/VvbrOE=;
        b=NJdb4fk5w8RBnb14LfyQtixMKG9L3lJld6Mi5Hmjuql8QTj+rzqULWmUpHAlBpULR8
         loU5IyTvYF6dtu2k+h97Ajp4wVkyvixsuDvwz543GkphoDgEEoEGUj3ifCJ+9SfWsCHZ
         r7b29Oshu9M3eql7oFeJZXL+7h1q1Yajx5dvEYP4H8xcrO6d8dj6Z91LJO514v0US0kO
         BN2P7tQD9djO07vibb9oq1L93UuFCvV/ZVA8QxYx10P2mlsDpLRgs6hnC+XffC1MlNgV
         dgqo6LOC763mXGqZQSNHQa+MLsKdnV9HDuYO2NMFlPWvRuH9gbOZ9v4IldXYmitFGtKp
         eqxg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965741; x=1709570541;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=856hL3x+HrIg0hf4NEQ6xzQ16mC6OszsgQrB/VvbrOE=;
        b=fVhljMzqOXKDhNWjWnRIReKxwLtxw182Op7kz4nKV7n6qTDTaOZck0blpHz2JVpY29
         2swCLuTXLrufs8I3Xds83oGS9LARLd5eTi/PjGLtU3BCenG9B1Ffjo3aK+knSdlbNgYX
         UGkCXtq6zipk2z10yrRlTeJBce4IE2FnHwJODQEQocmSISvGRk4fXvprpbpl4Hw7kq84
         RAEV0xNFof+6qfnf+ak+2RHuqGF1uEvcWgUOJBM/PPLBMR3TxOV5JhCoyR6A0niKAKH8
         Zg6s0Tw84v12v1Y7Ahg7gYSEaNqg67ruFiK+mbl7D91RbkCgBEtb0LUCYo5A//LtgCqg
         +r8Q==
X-Gm-Message-State: AOJu0YzjdkQWLPBb1sI8DIXJgLAj83XaaitiVbq7QpQVlVzXBEuMVHJQ
	r9mUWsekZhSzhPp/HD2mQrl2KBcWpMvjjIIDagYsctmtNLkZwYjTA0WMPM8Ab6EdOiNU+rgLOYc
	=
X-Google-Smtp-Source: AGHT+IFX4tC3kWKlsoQOmrfXbRUbYfg6ORAJGlbqb7HSf3nXJ3JiVN1u1vNGBzF/EmN/XqFAYOBrQA==
X-Received: by 2002:aa7:d0d4:0:b0:565:76b6:920c with SMTP id u20-20020aa7d0d4000000b0056576b6920cmr4634899edo.9.1708965740965;
        Mon, 26 Feb 2024 08:42:20 -0800 (PST)
Message-ID: <537b0d9c-1936-4cf5-a012-d50b1097a22d@suse.com>
Date: Mon, 26 Feb 2024 17:42:20 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 02/12] x86/HVM: improve CET-IBT pruning of ENDBR
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

__init{const,data}_cf_clobber can have an effect only for pointers
actually populated in the respective tables. While not the case for SVM
right now, VMX installs a number of pointers only under certain
conditions. Hence the respective functions would have their ENDBR purged
only when those conditions are met. Invoke "pruning" functions after
having copied the respective tables, for them to install any "missing"
pointers.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
This is largely cosmetic for present hardware, which when supporting
CET-IBT likely also supports all of the advanced VMX features for which
hook pointers are installed conditionally. The only case this would make
a difference there is when use of respective features was suppressed via
command line option (where available). For future hooks it may end up
relevant even by default, and it also would be if AMD started supporting
CET-IBT; right now it matters only for .pi_update_irte, as iommu_intpost
continues to default to off.

Originally I had meant to put the SVM and VMX functions in presmp-
initcalls, but hvm/{svm,vmx}/built_in.o are linked into hvm/built_in.o
before hvm/hvm.o. And I don't think I want to fiddle with link order
here.
---
v3: Re-base.
v2: Use cpu_has_xen_ibt in prune_{svm,vmx}().

--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -161,10 +161,15 @@ static int __init cf_check hvm_enable(vo
     else if ( cpu_has_svm )
         fns = start_svm();
 
+    if ( fns )
+        hvm_funcs = *fns;
+
+    prune_vmx();
+    prune_svm();
+
     if ( fns == NULL )
         return 0;
 
-    hvm_funcs = *fns;
     hvm_enabled = 1;
 
     printk("HVM: %s enabled\n", fns->name);
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -2587,6 +2587,19 @@ const struct hvm_function_table * __init
     return &svm_function_table;
 }
 
+void __init prune_svm(void)
+{
+    /*
+     * Now that svm_function_table was copied, populate all function pointers
+     * which may have been left at NULL, for __initdata_cf_clobber to have as
+     * much of an effect as possible.
+     */
+    if ( !cpu_has_xen_ibt )
+        return;
+
+    /* Nothing at present. */
+}
+
 void asmlinkage svm_vmexit_handler(void)
 {
     struct cpu_user_regs *regs = guest_cpu_user_regs();
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -3042,6 +3042,30 @@ const struct hvm_function_table * __init
     return &vmx_function_table;
 }
 
+void __init prune_vmx(void)
+{
+    /*
+     * Now that vmx_function_table was copied, populate all function pointers
+     * which may have been left at NULL, for __initdata_cf_clobber to have as
+     * much of an effect as possible.
+     */
+    if ( !cpu_has_xen_ibt )
+        return;
+
+    vmx_function_table.set_descriptor_access_exiting =
+        vmx_set_descriptor_access_exiting;
+
+    vmx_function_table.update_eoi_exit_bitmap = vmx_update_eoi_exit_bitmap;
+    vmx_function_table.process_isr            = vmx_process_isr;
+    vmx_function_table.handle_eoi             = vmx_handle_eoi;
+
+    vmx_function_table.pi_update_irte = vmx_pi_update_irte;
+
+    vmx_function_table.deliver_posted_intr = vmx_deliver_posted_intr;
+    vmx_function_table.sync_pir_to_irr     = vmx_sync_pir_to_irr;
+    vmx_function_table.test_pir            = vmx_test_pir;
+}
+
 /*
  * Not all cases receive valid value in the VM-exit instruction length field.
  * Callers must know what they're doing!
--- a/xen/arch/x86/include/asm/hvm/hvm.h
+++ b/xen/arch/x86/include/asm/hvm/hvm.h
@@ -250,6 +250,9 @@ extern s8 hvm_port80_allowed;
 extern const struct hvm_function_table *start_svm(void);
 extern const struct hvm_function_table *start_vmx(void);
 
+void prune_svm(void);
+void prune_vmx(void);
+
 int hvm_domain_initialise(struct domain *d,
                           const struct xen_domctl_createdomain *config);
 void hvm_domain_relinquish_resources(struct domain *d);



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:42:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:42:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685642.1066679 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree44-0007gV-V3; Mon, 26 Feb 2024 16:42:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685642.1066679; Mon, 26 Feb 2024 16:42:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree44-0007gO-Rw; Mon, 26 Feb 2024 16:42:52 +0000
Received: by outflank-mailman (input) for mailman id 685642;
 Mon, 26 Feb 2024 16:42:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree44-0006ea-6t
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:42:52 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 14fa70fa-d4c6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:42:51 +0100 (CET)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-5129cdae3c6so4246468e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:42:51 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.42.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:42:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 14fa70fa-d4c6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965771; x=1709570571; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=lGEwsapTSmnGaMr1Jgff2JJNs1NGP6rlb8MtqM0Zg5M=;
        b=VVWNh8lM51dFkQXcYCAuVqphIixln+YbzyPvK2JsC+Qn7+qE1thSQiS7zY5d/shsgM
         VyoCKhBEmLLbQfUpHqSXg5F1t3lW4Pcy4bg4xDwb3e0oBnCzJGr4E19jYUq3ZbzDSLB+
         uXnt2NjeJ010cdADbUCC0iIZ0DDHpMkwx0Ls1mSiieI7i9K8DTMsTmNqlN9upnmUrV+h
         oCqh964V9sXYYoRbeRb0n1fKpLdE/nWLAFLShDQ/lheb/8jwUniTTmNRWXScihDLIcjl
         M8yOWcMTkZ+kp+qaxGEZVEtkvYtpZ7ud5MHWefWMbhHnT0PCbxghihlnOUCnMZy5gbfj
         OZdg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965771; x=1709570571;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=lGEwsapTSmnGaMr1Jgff2JJNs1NGP6rlb8MtqM0Zg5M=;
        b=SpBlZEqwvzRdvELvNEROf5CbBfPM8sRaduZ/UQpxDEVjkk73fUeLgBxACWWrrpQFBK
         ELjLasiHTsyeB25tVR2WvqKZSbT37SvXMgnOKp34+2tEkzOka/X24jtdeaVnv+ay43mr
         zyVtrID7TQwzti0llU6gV/+jnql6AHhfClhJ95UVsLraSsMI5yj0mV6/Wy0gNVg0EqEj
         G0oUK8AH5LYR8s3RSFOSFZmBk2JCRhTDKnP5QOAYwYMM8kPSQHISVab6BXt9bwDEQkup
         dXxyMFLq2r4a6as9ruEhRfOD7ZNzaTU+6yQNIYc/FruG6VWHU2E2jPV1IaeGrDcfeWP0
         5S0g==
X-Gm-Message-State: AOJu0YyLh3Sq3WOp7B7+/U2F+V87iXVfc9yATB8rqLw3GyExTsv57L+U
	F3FM4Kuztmeaim/Eljxn+IXbWBnKt+7lwN15Ve5lyF1PXy0KaA9jJHaLApF5NoDp7nJnt9q7wmI
	=
X-Google-Smtp-Source: AGHT+IFzOe9RPrj/tlAUmp2ftETyB287NeXrdceNxcU4UMcEgBERU+cMFdJrD1BufPsTF8baSgqG0A==
X-Received: by 2002:ac2:4e6e:0:b0:512:a939:3fcc with SMTP id y14-20020ac24e6e000000b00512a9393fccmr4269698lfs.32.1708965771184;
        Mon, 26 Feb 2024 08:42:51 -0800 (PST)
Message-ID: <7a4ec627-f801-409b-995e-42732970e47c@suse.com>
Date: Mon, 26 Feb 2024 17:42:50 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 03/12] VMX: drop vmcs_revision_id
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

It's effectively redundant with vmx_basic_msr. For the #define
replacement to work, struct vmcs_struct's respective field name also
needs to change: Drop the not really meaningful "vmcs_" prefix from it.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -175,7 +175,7 @@ static DEFINE_PER_CPU(paddr_t, current_v
 static DEFINE_PER_CPU(struct list_head, active_vmcs_list);
 DEFINE_PER_CPU(bool, vmxon);
 
-static u32 vmcs_revision_id __read_mostly;
+#define vmcs_revision_id (vmx_basic_msr & VMX_BASIC_REVISION_MASK)
 u64 __read_mostly vmx_basic_msr;
 
 static void __init vmx_display_features(void)
@@ -498,7 +498,6 @@ static int vmx_init_vmcs_config(bool bsp
     if ( !vmx_pin_based_exec_control )
     {
         /* First time through. */
-        vmcs_revision_id           = vmx_basic_msr_low & VMX_BASIC_REVISION_MASK;
         vmx_pin_based_exec_control = _vmx_pin_based_exec_control;
         vmx_cpu_based_exec_control = _vmx_cpu_based_exec_control;
         vmx_secondary_exec_control = _vmx_secondary_exec_control;
@@ -610,7 +609,7 @@ static paddr_t vmx_alloc_vmcs(void)
 
     vmcs = __map_domain_page(pg);
     clear_page(vmcs);
-    vmcs->vmcs_revision_id = vmcs_revision_id;
+    vmcs->revision_id = vmcs_revision_id;
     unmap_domain_page(vmcs);
 
     return page_to_maddr(pg);
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -1166,7 +1166,7 @@ static void nvmx_set_vmcs_pointer(struct
     paddr_t vvmcs_maddr = v->arch.hvm.vmx.vmcs_shadow_maddr;
 
     __vmpclear(vvmcs_maddr);
-    vvmcs->vmcs_revision_id |= VMCS_RID_TYPE_MASK;
+    vvmcs->revision_id |= VMCS_RID_TYPE_MASK;
     v->arch.hvm.vmx.secondary_exec_control |=
         SECONDARY_EXEC_ENABLE_VMCS_SHADOWING;
     __vmwrite(SECONDARY_VM_EXEC_CONTROL,
@@ -1181,7 +1181,7 @@ static void nvmx_clear_vmcs_pointer(stru
     paddr_t vvmcs_maddr = v->arch.hvm.vmx.vmcs_shadow_maddr;
 
     __vmpclear(vvmcs_maddr);
-    vvmcs->vmcs_revision_id &= ~VMCS_RID_TYPE_MASK;
+    vvmcs->revision_id &= ~VMCS_RID_TYPE_MASK;
     v->arch.hvm.vmx.secondary_exec_control &=
         ~SECONDARY_EXEC_ENABLE_VMCS_SHADOWING;
     __vmwrite(SECONDARY_VM_EXEC_CONTROL,
@@ -1799,10 +1799,10 @@ static int nvmx_handle_vmptrld(struct cp
             {
                 struct vmcs_struct *vvmcs = vvmcx;
 
-                if ( ((vvmcs->vmcs_revision_id ^ vmx_basic_msr) &
-                                         VMX_BASIC_REVISION_MASK) ||
+                if ( ((vvmcs->revision_id ^ vmx_basic_msr) &
+                      VMX_BASIC_REVISION_MASK) ||
                      (!cpu_has_vmx_vmcs_shadowing &&
-                      (vvmcs->vmcs_revision_id & ~VMX_BASIC_REVISION_MASK)) )
+                      (vvmcs->revision_id & ~VMX_BASIC_REVISION_MASK)) )
                 {
                     hvm_unmap_guest_frame(vvmcx, 1);
                     vmfail(regs, VMX_INSN_VMPTRLD_INCORRECT_VMCS_ID);
@@ -2214,7 +2214,7 @@ int nvmx_msr_read_intercept(unsigned int
             map_domain_page(_mfn(PFN_DOWN(v->arch.hvm.vmx.vmcs_pa)));
 
         data = (host_data & (~0ul << 32)) |
-               (vmcs->vmcs_revision_id & 0x7fffffff);
+               (vmcs->revision_id & 0x7fffffff);
         unmap_domain_page(vmcs);
 
         if ( !cpu_has_vmx_vmcs_shadowing )
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -17,7 +17,7 @@ int cf_check vmx_cpu_up(void);
 void cf_check vmx_cpu_down(void);
 
 struct vmcs_struct {
-    u32 vmcs_revision_id;
+    uint32_t revision_id;
     unsigned char data [0]; /* vmcs size is read from MSR */
 };
 



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:43:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:43:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685644.1066690 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree4b-0008Em-7C; Mon, 26 Feb 2024 16:43:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685644.1066690; Mon, 26 Feb 2024 16:43:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree4b-0008Ef-49; Mon, 26 Feb 2024 16:43:25 +0000
Received: by outflank-mailman (input) for mailman id 685644;
 Mon, 26 Feb 2024 16:43:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree4Z-000739-Tg
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:43:23 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2723a141-d4c6-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 17:43:22 +0100 (CET)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-55a179f5fa1so3890892a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:43:22 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.43.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:43:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2723a141-d4c6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965801; x=1709570601; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=LkkYKIciLpGXm67Xsu4H8V04SVbEw+0lUvEF+5IhLIk=;
        b=XXfDVeaXDw+1DjizdDZpyg46sz3YYfuN8o7Hvkuui0aj9Gm10sOgli6uE0v1FKCbvj
         Ry9EAne5Xu1V64WXrKaem62KKyVG83dIJvAIRS1kOXs5J2O62YwoncSvpm6nQ2MyBUbs
         TkPu4VG2sC5IqnhW22GYZz9Oo4TD4kQSFDhEWKXneiQ9rW8Bwy5QH7xTin5HRymkqKmj
         2FlpFYAo5k2MQtdubxQq8WA7Sk3OrYf6XrWBWM1Y6oGyE5TBhjvl6w8OFqG+PKCwFxif
         OzsPQgo/5EuYuAzCFoynsIxYTOf0ffB1AOHVS2Op+O2kMvSyjofVtNlYq37KCkmcTVDy
         OPjg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965801; x=1709570601;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=LkkYKIciLpGXm67Xsu4H8V04SVbEw+0lUvEF+5IhLIk=;
        b=E1QzGJt8vrJBRS8N7BPkKPom+sgndLRGyNium8Fh+Bdjy2XpJJ2vuvOgx/piPLSCZN
         Vg8eQ7mhRo1jzcfzHG0roQEFqSaBAvaWqKmSURu2NRO9pdg0ELFOUhUHJn4gmvGQ7miF
         EzgwRm/8I7bigBeeEbTWzBbOXlir603sFbINyzjYO/7l5enBsTAS4E2yorvG5m4qpUEe
         PWMYdMZotcMGQzgt3kBFwxP2wwAE8kdXKZGgTTORfiXm2bc/tYhO3rgq2EbMUqp11czU
         TfEwNShRGFknSrHIlABgivIehIA24ntYv2/w4RrvxwtV+Plaz5QnGflwxB2nFQnF1Qmj
         R73w==
X-Gm-Message-State: AOJu0Yz7pKaCklTI/oKDIYb9UwFNcSvqdmdAovQ8tWQP2pX7Fl7oS8/Z
	XerRsnh07eHgYhvde8SrzLjrxJtPEFU5Iprajz0i0yOF/A2LNSiKnsx5J1XQEqsAQbJvZoqar54
	=
X-Google-Smtp-Source: AGHT+IE66fn6sd7RGWxgxOc52YFINHv/3XxkEA1CzC+GF36RQh6W0nd1PBlnpyG6b+0X8mqAOeueZw==
X-Received: by 2002:aa7:d912:0:b0:564:a62e:dad with SMTP id a18-20020aa7d912000000b00564a62e0dadmr5156319edr.34.1708965801706;
        Mon, 26 Feb 2024 08:43:21 -0800 (PST)
Message-ID: <eb9874c7-a392-49c2-a610-cf7ff45a3e3b@suse.com>
Date: Mon, 26 Feb 2024 17:43:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 04/12] VMX: convert vmx_basic_msr
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

... to a struct field, which is then going to be accompanied by other
capability/control data presently living in individual variables. As
this structure isn't supposed to be altered post-boot, put it in
.data.ro_after_init right away.

Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -161,6 +161,7 @@ static int cf_check parse_ept_param_runt
 #endif
 
 /* Dynamic (run-time adjusted) execution control flags. */
+struct vmx_caps __ro_after_init vmx_caps;
 u32 vmx_pin_based_exec_control __read_mostly;
 u32 vmx_cpu_based_exec_control __read_mostly;
 u32 vmx_secondary_exec_control __read_mostly;
@@ -175,8 +176,7 @@ static DEFINE_PER_CPU(paddr_t, current_v
 static DEFINE_PER_CPU(struct list_head, active_vmcs_list);
 DEFINE_PER_CPU(bool, vmxon);
 
-#define vmcs_revision_id (vmx_basic_msr & VMX_BASIC_REVISION_MASK)
-u64 __read_mostly vmx_basic_msr;
+#define vmcs_revision_id (vmx_caps.basic_msr & VMX_BASIC_REVISION_MASK)
 
 static void __init vmx_display_features(void)
 {
@@ -505,8 +505,8 @@ static int vmx_init_vmcs_config(bool bsp
         vmx_ept_vpid_cap           = _vmx_ept_vpid_cap;
         vmx_vmexit_control         = _vmx_vmexit_control;
         vmx_vmentry_control        = _vmx_vmentry_control;
-        vmx_basic_msr              = ((u64)vmx_basic_msr_high << 32) |
-                                     vmx_basic_msr_low;
+        vmx_caps.basic_msr = ((uint64_t)vmx_basic_msr_high << 32) |
+                             vmx_basic_msr_low;
         vmx_vmfunc                 = _vmx_vmfunc;
 
         vmx_display_features();
@@ -560,7 +560,7 @@ static int vmx_init_vmcs_config(bool bsp
             mismatch = 1;
         }
         if ( (vmx_basic_msr_high & (VMX_BASIC_VMCS_SIZE_MASK >> 32)) !=
-             ((vmx_basic_msr & VMX_BASIC_VMCS_SIZE_MASK) >> 32) )
+             ((vmx_caps.basic_msr & VMX_BASIC_VMCS_SIZE_MASK) >> 32) )
         {
             printk("VMX: CPU%d unexpected VMCS size %Lu\n",
                    smp_processor_id(),
@@ -2214,7 +2214,7 @@ int __init vmx_vmcs_init(void)
          * _vmx_vcpu_up() may have made it past feature identification.
          * Make sure all dependent features are off as well.
          */
-        vmx_basic_msr              = 0;
+        memset(&vmx_caps, 0, sizeof(vmx_caps));
         vmx_pin_based_exec_control = 0;
         vmx_cpu_based_exec_control = 0;
         vmx_secondary_exec_control = 0;
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -294,6 +294,12 @@ extern u64 vmx_ept_vpid_cap;
 
 #define VMX_TSC_MULTIPLIER_MAX                  0xffffffffffffffffULL
 
+/* Capabilities and dynamic (run-time adjusted) execution control flags. */
+struct vmx_caps {
+    uint64_t basic_msr;
+};
+extern struct vmx_caps vmx_caps;
+
 #define cpu_has_wbinvd_exiting \
     (vmx_secondary_exec_control & SECONDARY_EXEC_WBINVD_EXITING)
 #define cpu_has_vmx_virtualize_apic_accesses \
@@ -379,9 +385,8 @@ extern u64 vmx_ept_vpid_cap;
  */
 #define VMX_BASIC_DEFAULT1_ZERO		(1ULL << 55)
 
-extern u64 vmx_basic_msr;
 #define cpu_has_vmx_ins_outs_instr_info \
-    (!!(vmx_basic_msr & VMX_BASIC_INS_OUT_INFO))
+    (!!(vmx_caps.basic_msr & VMX_BASIC_INS_OUT_INFO))
 
 /* Guest interrupt status */
 #define VMX_GUEST_INTR_STATUS_SUBFIELD_BITMASK  0x0FF
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -1561,7 +1561,7 @@ static int nvmx_handle_vmxon(struct cpu_
     rc = hvm_copy_from_guest_phys(&nvmcs_revid, gpa, sizeof(nvmcs_revid));
     if ( rc != HVMTRANS_okay ||
          (nvmcs_revid & ~VMX_BASIC_REVISION_MASK) ||
-         ((nvmcs_revid ^ vmx_basic_msr) & VMX_BASIC_REVISION_MASK) )
+         ((nvmcs_revid ^ vmx_caps.basic_msr) & VMX_BASIC_REVISION_MASK) )
     {
         vmfail_invalid(regs);
         return X86EMUL_OKAY;
@@ -1799,7 +1799,7 @@ static int nvmx_handle_vmptrld(struct cp
             {
                 struct vmcs_struct *vvmcs = vvmcx;
 
-                if ( ((vvmcs->revision_id ^ vmx_basic_msr) &
+                if ( ((vvmcs->revision_id ^ vmx_caps.basic_msr) &
                       VMX_BASIC_REVISION_MASK) ||
                      (!cpu_has_vmx_vmcs_shadowing &&
                       (vvmcs->revision_id & ~VMX_BASIC_REVISION_MASK)) )
@@ -2192,7 +2192,7 @@ int nvmx_msr_read_intercept(unsigned int
     case MSR_IA32_VMX_TRUE_PROCBASED_CTLS:
     case MSR_IA32_VMX_TRUE_EXIT_CTLS:
     case MSR_IA32_VMX_TRUE_ENTRY_CTLS:
-        if ( !(vmx_basic_msr & VMX_BASIC_DEFAULT1_ZERO) )
+        if ( !(vmx_caps.basic_msr & VMX_BASIC_DEFAULT1_ZERO) )
             return 0;
         break;
 



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:43:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:43:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685646.1066699 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree4t-0000F2-EW; Mon, 26 Feb 2024 16:43:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685646.1066699; Mon, 26 Feb 2024 16:43:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree4t-0000Ev-Bf; Mon, 26 Feb 2024 16:43:43 +0000
Received: by outflank-mailman (input) for mailman id 685646;
 Mon, 26 Feb 2024 16:43:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree4s-0000BK-Fu
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:43:42 +0000
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
 [2a00:1450:4864:20::131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 32db792f-d4c6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:43:41 +0100 (CET)
Received: by mail-lf1-x131.google.com with SMTP id
 2adb3069b0e04-512b13bf764so4022409e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:43:41 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.43.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:43:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 32db792f-d4c6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965821; x=1709570621; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=9n+bbgXBOrLiMHM6zz3tWMkdYfLaMnbCqmKV8XjOqSc=;
        b=LAHOrbhOX03bGkAeSyUIQCZqlxKaBHl+A0Lw8oZHYf/X82uwcPHnwUQWGqhW616bdI
         zjKpb3lGm+C1aaUyzNdjfTq0x7u+1ZF7yUQrRTsYgSNpagjoxOepww32ci7J6vUVaZ6T
         cfQi8ZJ7/0ye0rTSoyqVKR3MPEDKFLITh0hVZObNVPaGAna3l9hsMBeHSIJdgzqBisGV
         QdCZS05/CVz/DuwGlQzF5kS/dI89ttt7Wf4lTHpx7eO3O/cfLKNXWlta6ZW4jxAouVH6
         S6Ad9KHfuX2I99OZjju60X4Qkikia7jtHhY6fGbYGiItAQxNBtTmpBydlFJlx8R5b4Ss
         GZ4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965821; x=1709570621;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=9n+bbgXBOrLiMHM6zz3tWMkdYfLaMnbCqmKV8XjOqSc=;
        b=DIT5+O1PT3JblcDsg79dCEGLb/Nw+mLJfHRouGfMjf+LmdeGYKxG5O/2Jn6p05kSss
         ppkewOxJurlwYVqu825+05Evx9J03Hm/+TNViRH5cOF3aIIWlx5CFzamQM0F+d0C8rac
         NSeaCBy4WdjxlZVGjiu/UVNz8VJexHnDo51M5Q1iiHDw3e/LKaOEzaxHTFXDR1YiOOmK
         qHqlYIrpqO8YGnC7hOEU9J3tpICg3HBqs27BAn3NsDgIyduI+rcmLDKHQ8ciTkXIwb+h
         JjyTEV6m3E44uzjBmhZf2FnUNSQ5Ws/jo9jJtpZyWGOR/5FYUJLdGB03O/FwbrERVR+E
         eRtw==
X-Gm-Message-State: AOJu0YyU2k6hXNlTyJBiWFwUD3IsarHtLYPuC5b1NmkC1qb3oiFXNoAa
	f0CJeK4bEb+2IEKtNYi4h1UO43E1ROJRSqtwN1tirW+ZfAA328B2vznJHNt22zU5WfuRyIbjZuU
	=
X-Google-Smtp-Source: AGHT+IGNNeXW8Ss1pszPMFziokqO8jfdNI1FqfvC4Bl8vdK9qMAgCYc7W6+h/6lo68CWeWhxqVdjZw==
X-Received: by 2002:ac2:47fa:0:b0:512:cbf4:8f93 with SMTP id b26-20020ac247fa000000b00512cbf48f93mr4744980lfp.19.1708965821334;
        Mon, 26 Feb 2024 08:43:41 -0800 (PST)
Message-ID: <57a00d7b-1df1-4b74-b990-8b1e88784589@suse.com>
Date: Mon, 26 Feb 2024 17:43:40 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 05/12] VMX: convert vmx_pin_based_exec_control
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... to a field in the capability/controls struct. Use an instance of
that struct also in vmx_init_vmcs_config().

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v3: Add initializer to vmx_init_vmcs_config() new local var.
v2: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -162,7 +162,6 @@ static int cf_check parse_ept_param_runt
 
 /* Dynamic (run-time adjusted) execution control flags. */
 struct vmx_caps __ro_after_init vmx_caps;
-u32 vmx_pin_based_exec_control __read_mostly;
 u32 vmx_cpu_based_exec_control __read_mostly;
 u32 vmx_secondary_exec_control __read_mostly;
 uint64_t vmx_tertiary_exec_control __read_mostly;
@@ -261,7 +260,7 @@ static bool cap_check(
 static int vmx_init_vmcs_config(bool bsp)
 {
     u32 vmx_basic_msr_low, vmx_basic_msr_high, min, opt;
-    u32 _vmx_pin_based_exec_control;
+    struct vmx_caps caps = {};
     u32 _vmx_cpu_based_exec_control;
     u32 _vmx_secondary_exec_control = 0;
     uint64_t _vmx_tertiary_exec_control = 0;
@@ -278,7 +277,7 @@ static int vmx_init_vmcs_config(bool bsp
            PIN_BASED_NMI_EXITING);
     opt = (PIN_BASED_VIRTUAL_NMIS |
            PIN_BASED_POSTED_INTERRUPT);
-    _vmx_pin_based_exec_control = adjust_vmx_controls(
+    caps.pin_based_exec_control = adjust_vmx_controls(
         "Pin-Based Exec Control", min, opt,
         MSR_IA32_VMX_PINBASED_CTLS, &mismatch);
 
@@ -440,7 +439,7 @@ static int vmx_init_vmcs_config(bool bsp
     if ( (_vmx_secondary_exec_control & SECONDARY_EXEC_PAUSE_LOOP_EXITING) &&
           ple_gap == 0 )
     {
-        if ( !vmx_pin_based_exec_control )
+        if ( !vmx_caps.pin_based_exec_control )
             printk(XENLOG_INFO "Disable Pause-Loop Exiting.\n");
         _vmx_secondary_exec_control &= ~ SECONDARY_EXEC_PAUSE_LOOP_EXITING;
     }
@@ -458,10 +457,10 @@ static int vmx_init_vmcs_config(bool bsp
      * is a minimal requirement, only check the former, which is optional.
      */
     if ( !(_vmx_secondary_exec_control & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY) )
-        _vmx_pin_based_exec_control &= ~PIN_BASED_POSTED_INTERRUPT;
+        caps.pin_based_exec_control &= ~PIN_BASED_POSTED_INTERRUPT;
 
     if ( iommu_intpost &&
-         !(_vmx_pin_based_exec_control & PIN_BASED_POSTED_INTERRUPT) )
+         !(caps.pin_based_exec_control & PIN_BASED_POSTED_INTERRUPT) )
     {
         printk("Intel VT-d Posted Interrupt is disabled for CPU-side Posted "
                "Interrupt is not enabled\n");
@@ -495,10 +494,10 @@ static int vmx_init_vmcs_config(bool bsp
     if ( mismatch )
         return -EINVAL;
 
-    if ( !vmx_pin_based_exec_control )
+    if ( !vmx_caps.pin_based_exec_control )
     {
         /* First time through. */
-        vmx_pin_based_exec_control = _vmx_pin_based_exec_control;
+        vmx_caps = caps;
         vmx_cpu_based_exec_control = _vmx_cpu_based_exec_control;
         vmx_secondary_exec_control = _vmx_secondary_exec_control;
         vmx_tertiary_exec_control  = _vmx_tertiary_exec_control;
@@ -529,7 +528,7 @@ static int vmx_init_vmcs_config(bool bsp
             vmcs_revision_id, vmx_basic_msr_low & VMX_BASIC_REVISION_MASK);
         mismatch |= cap_check(
             "Pin-Based Exec Control",
-            vmx_pin_based_exec_control, _vmx_pin_based_exec_control);
+            vmx_caps.pin_based_exec_control, caps.pin_based_exec_control);
         mismatch |= cap_check(
             "CPU-Based Exec Control",
             vmx_cpu_based_exec_control, _vmx_cpu_based_exec_control);
@@ -1110,7 +1109,7 @@ static int construct_vmcs(struct vcpu *v
     vmx_vmcs_enter(v);
 
     /* VMCS controls. */
-    __vmwrite(PIN_BASED_VM_EXEC_CONTROL, vmx_pin_based_exec_control);
+    __vmwrite(PIN_BASED_VM_EXEC_CONTROL, vmx_caps.pin_based_exec_control);
 
     v->arch.hvm.vmx.exec_control = vmx_cpu_based_exec_control;
     if ( d->arch.vtsc && !cpu_has_vmx_tsc_scaling )
@@ -2136,7 +2135,7 @@ void vmcs_dump_vcpu(struct vcpu *v)
     printk("TSC Offset = 0x%016lx  TSC Multiplier = 0x%016lx\n",
            vmr(TSC_OFFSET), vmr(TSC_MULTIPLIER));
     if ( (v->arch.hvm.vmx.exec_control & CPU_BASED_TPR_SHADOW) ||
-         (vmx_pin_based_exec_control & PIN_BASED_POSTED_INTERRUPT) )
+         (vmx_caps.pin_based_exec_control & PIN_BASED_POSTED_INTERRUPT) )
         printk("TPR Threshold = 0x%02x  PostedIntrVec = 0x%02x\n",
                vmr32(TPR_THRESHOLD), vmr16(POSTED_INTR_NOTIFICATION_VECTOR));
     if ( (v->arch.hvm.vmx.secondary_exec_control &
@@ -2215,7 +2214,6 @@ int __init vmx_vmcs_init(void)
          * Make sure all dependent features are off as well.
          */
         memset(&vmx_caps, 0, sizeof(vmx_caps));
-        vmx_pin_based_exec_control = 0;
         vmx_cpu_based_exec_control = 0;
         vmx_secondary_exec_control = 0;
         vmx_tertiary_exec_control  = 0;
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -1057,7 +1057,7 @@ static void load_shadow_control(struct v
      * and EXCEPTION
      * Enforce the removed features
      */
-    nvmx_update_pin_control(v, vmx_pin_based_exec_control);
+    nvmx_update_pin_control(v, vmx_caps.pin_based_exec_control);
     vmx_update_cpu_exec_control(v);
     vmx_update_secondary_exec_control(v);
     nvmx_update_exit_control(v, vmx_vmexit_control);
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -217,7 +217,6 @@ extern u32 vmx_cpu_based_exec_control;
 #define PIN_BASED_VIRTUAL_NMIS          0x00000020
 #define PIN_BASED_PREEMPT_TIMER         0x00000040
 #define PIN_BASED_POSTED_INTERRUPT      0x00000080
-extern u32 vmx_pin_based_exec_control;
 
 #define VM_EXIT_SAVE_DEBUG_CNTRLS       0x00000004
 #define VM_EXIT_IA32E_MODE              0x00000200
@@ -297,6 +296,7 @@ extern u64 vmx_ept_vpid_cap;
 /* Capabilities and dynamic (run-time adjusted) execution control flags. */
 struct vmx_caps {
     uint64_t basic_msr;
+    uint32_t pin_based_exec_control;
 };
 extern struct vmx_caps vmx_caps;
 
@@ -307,7 +307,7 @@ extern struct vmx_caps vmx_caps;
 #define cpu_has_vmx_tpr_shadow \
     (vmx_cpu_based_exec_control & CPU_BASED_TPR_SHADOW)
 #define cpu_has_vmx_vnmi \
-    (vmx_pin_based_exec_control & PIN_BASED_VIRTUAL_NMIS)
+    (vmx_caps.pin_based_exec_control & PIN_BASED_VIRTUAL_NMIS)
 #define cpu_has_vmx_msr_bitmap \
     (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_MSR_BITMAP)
 #define cpu_has_vmx_secondary_exec_control \
@@ -344,7 +344,7 @@ extern struct vmx_caps vmx_caps;
 #define cpu_has_vmx_virtualize_x2apic_mode \
     (vmx_secondary_exec_control & SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE)
 #define cpu_has_vmx_posted_intr_processing \
-    (vmx_pin_based_exec_control & PIN_BASED_POSTED_INTERRUPT)
+    (vmx_caps.pin_based_exec_control & PIN_BASED_POSTED_INTERRUPT)
 #define cpu_has_vmx_vmcs_shadowing \
     (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VMCS_SHADOWING)
 #define cpu_has_vmx_vmfunc \



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:44:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:44:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685648.1066710 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree5B-0000pO-Sy; Mon, 26 Feb 2024 16:44:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685648.1066710; Mon, 26 Feb 2024 16:44:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree5B-0000pH-Pa; Mon, 26 Feb 2024 16:44:01 +0000
Received: by outflank-mailman (input) for mailman id 685648;
 Mon, 26 Feb 2024 16:43:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree59-0000BK-Ro
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:43:59 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3d3b3ca4-d4c6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:43:59 +0100 (CET)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2d09cf00214so51658461fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:43:59 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.43.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:43:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d3b3ca4-d4c6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965838; x=1709570638; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=I+hozS1nSCFWHKTN5uvAFxwcd3lo+nQd/gYn/2VyyrI=;
        b=QvpsZJUucOiHyY4YAqgv1YaVsvSIqt2sQbs0k5EWR8e6T/B/zSFij39iRRtNt0oUv8
         ENigmxjW3d6vM6cE4uLOhpPr8tV0ut4kdul4k7wHHIzDtqo7Frno6UOsBCokf1tGH9/j
         QWBEvlSqi+YTNC8b/oIIgmlNr8WLrOQLl1IBXYXyxt0UwoqtWShqCLS2TenxALylgYaD
         Ny+6UusEnvl/3t5BtkWcpIbq/slZvNXVBAXEfhuztAZxaoOakFY9MFyBbuW7IWlqLEPG
         Wu0yB/jL9KE5CfjH/Gsw/dTOTnhg7RIDrdc3XnLsN6uDwmWATR4VxZj/5aeIjlU90Iay
         IQVQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965838; x=1709570638;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=I+hozS1nSCFWHKTN5uvAFxwcd3lo+nQd/gYn/2VyyrI=;
        b=vG0FZTcKd8ISrB4va7io+aVP5eWJOT+GCQXeiSn5e7eLBbRYlcSLk7zwwwrTgmTB+s
         QtRmbwcNGJgS6RNrjKJUyY3PIiXHz3NqXsVyjQd9eJ4ODxSyajkRrhR9/ZbfC+GNVltW
         IEKzALt9dFoYYuoBwIdCZUvumVzDoLSe5PM/7TEUCIsfBu0fVhY1nx9BhvaAzFWlZShV
         /RK8t+Yfc2hqa12dEWUCLznAR1krU4jJ7+7g0Es6vfQVhU7kknHRWiKP50BScFNwlWkG
         2uDowlKVnhXQnEvx/hYk+c5qBoMw5yEAhtRbiJPYLbU0+Bjnm3hxgEHDL/fa6AJ0fgkx
         kFlA==
X-Gm-Message-State: AOJu0Yxd8sCGCZf6jHA1AZK+oSx24SxyOhdLQBUIxKXxlD1ljiUQuB4N
	GEt9VVqI+2zozihV/daRKtpOy9Vz4mwxG1pG+HUJzYn+wEIygI3zld8AoSGCWjexJ0Cvbq7NM0Y
	=
X-Google-Smtp-Source: AGHT+IGW/ceIqxVXPr0RzY6ctXnRWKp3aDNdb1/mUUv8JqciivLWvhcaIi3a9r4IsAjstRF9WGeotQ==
X-Received: by 2002:ac2:55a3:0:b0:513:31c:e54e with SMTP id y3-20020ac255a3000000b00513031ce54emr52954lfg.30.1708965838745;
        Mon, 26 Feb 2024 08:43:58 -0800 (PST)
Message-ID: <36dd8c5a-7b50-404e-ac96-3407c9d574c4@suse.com>
Date: Mon, 26 Feb 2024 17:43:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 06/12] VMX: convert vmx_cpu_based_exec_control
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... to a field in the capability/controls struct.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -162,7 +162,6 @@ static int cf_check parse_ept_param_runt
 
 /* Dynamic (run-time adjusted) execution control flags. */
 struct vmx_caps __ro_after_init vmx_caps;
-u32 vmx_cpu_based_exec_control __read_mostly;
 u32 vmx_secondary_exec_control __read_mostly;
 uint64_t vmx_tertiary_exec_control __read_mostly;
 u32 vmx_vmexit_control __read_mostly;
@@ -261,7 +260,6 @@ static int vmx_init_vmcs_config(bool bsp
 {
     u32 vmx_basic_msr_low, vmx_basic_msr_high, min, opt;
     struct vmx_caps caps = {};
-    u32 _vmx_cpu_based_exec_control;
     u32 _vmx_secondary_exec_control = 0;
     uint64_t _vmx_tertiary_exec_control = 0;
     u64 _vmx_ept_vpid_cap = 0;
@@ -299,12 +297,12 @@ static int vmx_init_vmcs_config(bool bsp
            CPU_BASED_MONITOR_TRAP_FLAG |
            CPU_BASED_ACTIVATE_SECONDARY_CONTROLS |
            CPU_BASED_ACTIVATE_TERTIARY_CONTROLS);
-    _vmx_cpu_based_exec_control = adjust_vmx_controls(
+    caps.cpu_based_exec_control = adjust_vmx_controls(
         "CPU-Based Exec Control", min, opt,
         MSR_IA32_VMX_PROCBASED_CTLS, &mismatch);
-    _vmx_cpu_based_exec_control &= ~CPU_BASED_RDTSC_EXITING;
-    if ( _vmx_cpu_based_exec_control & CPU_BASED_TPR_SHADOW )
-        _vmx_cpu_based_exec_control &=
+    caps.cpu_based_exec_control &= ~CPU_BASED_RDTSC_EXITING;
+    if ( caps.cpu_based_exec_control & CPU_BASED_TPR_SHADOW )
+        caps.cpu_based_exec_control &=
             ~(CPU_BASED_CR8_LOAD_EXITING | CPU_BASED_CR8_STORE_EXITING);
 
     rdmsrl(MSR_IA32_VMX_MISC, _vmx_misc_cap);
@@ -321,7 +319,7 @@ static int vmx_init_vmcs_config(bool bsp
         return -EINVAL;
     }
 
-    if ( _vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS )
+    if ( caps.cpu_based_exec_control & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS )
     {
         min = 0;
         opt = (SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES |
@@ -351,7 +349,7 @@ static int vmx_init_vmcs_config(bool bsp
          * "APIC Register Virtualization" and "Virtual Interrupt Delivery"
          * can be set only when "use TPR shadow" is set
          */
-        if ( (_vmx_cpu_based_exec_control & CPU_BASED_TPR_SHADOW) &&
+        if ( (caps.cpu_based_exec_control & CPU_BASED_TPR_SHADOW) &&
              opt_apicv_enabled )
             opt |= SECONDARY_EXEC_APIC_REGISTER_VIRT |
                    SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY |
@@ -362,7 +360,7 @@ static int vmx_init_vmcs_config(bool bsp
             MSR_IA32_VMX_PROCBASED_CTLS2, &mismatch);
     }
 
-    if ( _vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS )
+    if ( caps.cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS )
     {
         uint64_t opt = 0;
 
@@ -498,7 +496,6 @@ static int vmx_init_vmcs_config(bool bsp
     {
         /* First time through. */
         vmx_caps = caps;
-        vmx_cpu_based_exec_control = _vmx_cpu_based_exec_control;
         vmx_secondary_exec_control = _vmx_secondary_exec_control;
         vmx_tertiary_exec_control  = _vmx_tertiary_exec_control;
         vmx_ept_vpid_cap           = _vmx_ept_vpid_cap;
@@ -531,7 +528,7 @@ static int vmx_init_vmcs_config(bool bsp
             vmx_caps.pin_based_exec_control, caps.pin_based_exec_control);
         mismatch |= cap_check(
             "CPU-Based Exec Control",
-            vmx_cpu_based_exec_control, _vmx_cpu_based_exec_control);
+            vmx_caps.cpu_based_exec_control, caps.cpu_based_exec_control);
         mismatch |= cap_check(
             "Secondary Exec Control",
             vmx_secondary_exec_control, _vmx_secondary_exec_control);
@@ -1111,7 +1108,7 @@ static int construct_vmcs(struct vcpu *v
     /* VMCS controls. */
     __vmwrite(PIN_BASED_VM_EXEC_CONTROL, vmx_caps.pin_based_exec_control);
 
-    v->arch.hvm.vmx.exec_control = vmx_cpu_based_exec_control;
+    v->arch.hvm.vmx.exec_control = vmx_caps.cpu_based_exec_control;
     if ( d->arch.vtsc && !cpu_has_vmx_tsc_scaling )
         v->arch.hvm.vmx.exec_control |= CPU_BASED_RDTSC_EXITING;
 
@@ -2214,7 +2211,6 @@ int __init vmx_vmcs_init(void)
          * Make sure all dependent features are off as well.
          */
         memset(&vmx_caps, 0, sizeof(vmx_caps));
-        vmx_cpu_based_exec_control = 0;
         vmx_secondary_exec_control = 0;
         vmx_tertiary_exec_control  = 0;
         vmx_vmexit_control         = 0;
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -210,7 +210,6 @@ void vmx_vmcs_reload(struct vcpu *v);
 #define CPU_BASED_MONITOR_EXITING             0x20000000U
 #define CPU_BASED_PAUSE_EXITING               0x40000000U
 #define CPU_BASED_ACTIVATE_SECONDARY_CONTROLS 0x80000000U
-extern u32 vmx_cpu_based_exec_control;
 
 #define PIN_BASED_EXT_INTR_MASK         0x00000001
 #define PIN_BASED_NMI_EXITING           0x00000008
@@ -297,6 +296,7 @@ extern u64 vmx_ept_vpid_cap;
 struct vmx_caps {
     uint64_t basic_msr;
     uint32_t pin_based_exec_control;
+    uint32_t cpu_based_exec_control;
 };
 extern struct vmx_caps vmx_caps;
 
@@ -305,15 +305,15 @@ extern struct vmx_caps vmx_caps;
 #define cpu_has_vmx_virtualize_apic_accesses \
     (vmx_secondary_exec_control & SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES)
 #define cpu_has_vmx_tpr_shadow \
-    (vmx_cpu_based_exec_control & CPU_BASED_TPR_SHADOW)
+    (vmx_caps.cpu_based_exec_control & CPU_BASED_TPR_SHADOW)
 #define cpu_has_vmx_vnmi \
     (vmx_caps.pin_based_exec_control & PIN_BASED_VIRTUAL_NMIS)
 #define cpu_has_vmx_msr_bitmap \
-    (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_MSR_BITMAP)
+    (vmx_caps.cpu_based_exec_control & CPU_BASED_ACTIVATE_MSR_BITMAP)
 #define cpu_has_vmx_secondary_exec_control \
-    (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS)
+    (vmx_caps.cpu_based_exec_control & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS)
 #define cpu_has_vmx_tertiary_exec_control \
-    (vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS)
+    (vmx_caps.cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS)
 #define cpu_has_vmx_ept \
     (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_EPT)
 #define cpu_has_vmx_dt_exiting \
@@ -323,7 +323,7 @@ extern struct vmx_caps vmx_caps;
 #define cpu_has_vmx_vpid \
     (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VPID)
 #define cpu_has_monitor_trap_flag \
-    (vmx_cpu_based_exec_control & CPU_BASED_MONITOR_TRAP_FLAG)
+    (vmx_caps.cpu_based_exec_control & CPU_BASED_MONITOR_TRAP_FLAG)
 #define cpu_has_vmx_pat \
     (vmx_vmentry_control & VM_ENTRY_LOAD_GUEST_PAT)
 #define cpu_has_vmx_efer \



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:44:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:44:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685651.1066719 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree5X-0001MJ-3o; Mon, 26 Feb 2024 16:44:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685651.1066719; Mon, 26 Feb 2024 16:44:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree5X-0001MC-11; Mon, 26 Feb 2024 16:44:23 +0000
Received: by outflank-mailman (input) for mailman id 685651;
 Mon, 26 Feb 2024 16:44:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree5V-0000BK-Fg
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:44:21 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4a19299a-d4c6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:44:20 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-55a179f5fa1so3892560a12.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:44:20 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.44.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:44:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4a19299a-d4c6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965860; x=1709570660; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=SpUvRoudA/jtd2T5mUZMGFquMIbLtsovgL/QnQtxnRs=;
        b=apbkVH/4M0IOdvpR4ZB5WFHTCLWfO3zS5SVDHDfb52A7eFMlqb2EHUsc6z/gQLysbJ
         QJMos7nBWZLSgLHzyAQ6onKsD8hziFRNTdr2eJ12ATSnx4S4Pay8mrXActvxTqkMy62g
         zfyxgfQPgTvbUg4vd23P2dGFTqITLPSIY77f10DBUrxSqoZr2sg6KFxhhdTuqs9HjfSK
         z/GkXu2mKj9OHvQmiDs3DpIBGUN+2MkK9GcJaXTOmwHimlT1lDZ1RJb35SeRuRyhRwl2
         FZ69DhYN4xCTISW/DdXY5QvauRpk6Ef4zeozkCBpoLSULCj52LhXihqh4BD9O9n1mQEe
         xTRQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965860; x=1709570660;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=SpUvRoudA/jtd2T5mUZMGFquMIbLtsovgL/QnQtxnRs=;
        b=pXQBZ/mfvvE0By3+lZZ3ofhnHWNIvQDZBbNJ3iO3OCbzbQsrCzTG9kmtF2zd3rm/qQ
         4vWHUUbQuPRmqEfrOTH7ubbcxWbYdwyaGQn5hRXp3lmoG7FYfi+BHKb5WTwyjOd1+q4m
         /o1mVEnHnmoGxZ8ujtdir7fEb7YKf2roMV+mteV5dr3dkbGTGNThPtNrB+ADshN1AxCu
         i1FJei8lDk1ixyOrriaHzt452/mYqCyphPJcRlWcxMsXjIOaXEqPpkKIj2nlZNON8qu8
         1ONuv44TkNm48l0D4Fg0gTn2/vs/pMwRh3MI7vOQg5y3aLFkSwL+FN7MPKfTYH3i0Bx+
         zOog==
X-Gm-Message-State: AOJu0YyVakGcBU2YCefazkfPB/9MxSAepGOgq/5ftnjHKnaTbEP9Qtfu
	Mk7IpF+JcMCKXtXmJKb6je4CV6izxsXF6x6C5CWstyoAYMdW7rHla0VZ3ccKWU6zze4sAEBUODc
	=
X-Google-Smtp-Source: AGHT+IFT3FjGjnzj4OyiinDq6o0Iumj4WCl3H/OgMJ/gI0s9UyiW57ulZMbRpyF6Wtl+t2U1vETXnQ==
X-Received: by 2002:a05:6402:1c82:b0:565:9892:a74e with SMTP id cy2-20020a0564021c8200b005659892a74emr4105428edb.21.1708965860318;
        Mon, 26 Feb 2024 08:44:20 -0800 (PST)
Message-ID: <6fecfa2d-3af7-4241-b7ab-1b89817cdf0d@suse.com>
Date: Mon, 26 Feb 2024 17:44:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 07/12] VMX: convert vmx_secondary_exec_control
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... to a field in the capability/controls struct.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -162,7 +162,6 @@ static int cf_check parse_ept_param_runt
 
 /* Dynamic (run-time adjusted) execution control flags. */
 struct vmx_caps __ro_after_init vmx_caps;
-u32 vmx_secondary_exec_control __read_mostly;
 uint64_t vmx_tertiary_exec_control __read_mostly;
 u32 vmx_vmexit_control __read_mostly;
 u32 vmx_vmentry_control __read_mostly;
@@ -260,7 +259,6 @@ static int vmx_init_vmcs_config(bool bsp
 {
     u32 vmx_basic_msr_low, vmx_basic_msr_high, min, opt;
     struct vmx_caps caps = {};
-    u32 _vmx_secondary_exec_control = 0;
     uint64_t _vmx_tertiary_exec_control = 0;
     u64 _vmx_ept_vpid_cap = 0;
     u64 _vmx_misc_cap = 0;
@@ -355,7 +353,7 @@ static int vmx_init_vmcs_config(bool bsp
                    SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY |
                    SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE;
 
-        _vmx_secondary_exec_control = adjust_vmx_controls(
+        caps.secondary_exec_control = adjust_vmx_controls(
             "Secondary Exec Control", min, opt,
             MSR_IA32_VMX_PROCBASED_CTLS2, &mismatch);
     }
@@ -370,7 +368,7 @@ static int vmx_init_vmcs_config(bool bsp
     }
 
     /* The IA32_VMX_EPT_VPID_CAP MSR exists only when EPT or VPID available */
-    if ( _vmx_secondary_exec_control & (SECONDARY_EXEC_ENABLE_EPT |
+    if ( caps.secondary_exec_control & (SECONDARY_EXEC_ENABLE_EPT |
                                         SECONDARY_EXEC_ENABLE_VPID) )
     {
         rdmsrl(MSR_IA32_VMX_EPT_VPID_CAP, _vmx_ept_vpid_cap);
@@ -392,7 +390,7 @@ static int vmx_init_vmcs_config(bool bsp
         if ( !(_vmx_ept_vpid_cap & VMX_EPT_MEMORY_TYPE_WB) ||
              !(_vmx_ept_vpid_cap & VMX_EPT_WALK_LENGTH_4_SUPPORTED) ||
              !(_vmx_ept_vpid_cap & VMX_EPT_INVEPT_ALL_CONTEXT) )
-            _vmx_secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_EPT;
+            caps.secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_EPT;
 
         /*
          * the CPU must support INVVPID all context invalidation, because we
@@ -401,14 +399,14 @@ static int vmx_init_vmcs_config(bool bsp
          * Or we just don't use VPID.
          */
         if ( !(_vmx_ept_vpid_cap & VMX_VPID_INVVPID_ALL_CONTEXT) )
-            _vmx_secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_VPID;
+            caps.secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_VPID;
 
         /* EPT A/D bits is required for PML */
         if ( !(_vmx_ept_vpid_cap & VMX_EPT_AD_BIT) )
-            _vmx_secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_PML;
+            caps.secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_PML;
     }
 
-    if ( _vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_EPT )
+    if ( caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_EPT )
     {
         /*
          * To use EPT we expect to be able to clear certain intercepts.
@@ -421,25 +419,25 @@ static int vmx_init_vmcs_config(bool bsp
         if ( must_be_one & (CPU_BASED_INVLPG_EXITING |
                             CPU_BASED_CR3_LOAD_EXITING |
                             CPU_BASED_CR3_STORE_EXITING) )
-            _vmx_secondary_exec_control &=
+            caps.secondary_exec_control &=
                 ~(SECONDARY_EXEC_ENABLE_EPT |
                   SECONDARY_EXEC_UNRESTRICTED_GUEST);
     }
 
     /* PML cannot be supported if EPT is not used */
-    if ( !(_vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_EPT) )
-        _vmx_secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_PML;
+    if ( !(caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_EPT) )
+        caps.secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_PML;
 
     /* Turn off opt_ept_pml if PML feature is not present. */
-    if ( !(_vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_PML) )
+    if ( !(caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_PML) )
         opt_ept_pml = false;
 
-    if ( (_vmx_secondary_exec_control & SECONDARY_EXEC_PAUSE_LOOP_EXITING) &&
+    if ( (caps.secondary_exec_control & SECONDARY_EXEC_PAUSE_LOOP_EXITING) &&
           ple_gap == 0 )
     {
         if ( !vmx_caps.pin_based_exec_control )
             printk(XENLOG_INFO "Disable Pause-Loop Exiting.\n");
-        _vmx_secondary_exec_control &= ~ SECONDARY_EXEC_PAUSE_LOOP_EXITING;
+        caps.secondary_exec_control &= ~ SECONDARY_EXEC_PAUSE_LOOP_EXITING;
     }
 
     min = VM_EXIT_ACK_INTR_ON_EXIT;
@@ -454,7 +452,7 @@ static int vmx_init_vmcs_config(bool bsp
      * delivery" and "acknowledge interrupt on exit" is set. For the latter
      * is a minimal requirement, only check the former, which is optional.
      */
-    if ( !(_vmx_secondary_exec_control & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY) )
+    if ( !(caps.secondary_exec_control & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY) )
         caps.pin_based_exec_control &= ~PIN_BASED_POSTED_INTERRUPT;
 
     if ( iommu_intpost &&
@@ -466,7 +464,7 @@ static int vmx_init_vmcs_config(bool bsp
     }
 
     /* The IA32_VMX_VMFUNC MSR exists only when VMFUNC is available */
-    if ( _vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VM_FUNCTIONS )
+    if ( caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_VM_FUNCTIONS )
     {
         rdmsrl(MSR_IA32_VMX_VMFUNC, _vmx_vmfunc);
 
@@ -476,12 +474,12 @@ static int vmx_init_vmcs_config(bool bsp
          * Or we just don't use VMFUNC.
          */
         if ( !(_vmx_vmfunc & VMX_VMFUNC_EPTP_SWITCHING) )
-            _vmx_secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_VM_FUNCTIONS;
+            caps.secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_VM_FUNCTIONS;
     }
 
     /* Virtualization exceptions are only enabled if VMFUNC is enabled */
-    if ( !(_vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VM_FUNCTIONS) )
-        _vmx_secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS;
+    if ( !(caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_VM_FUNCTIONS) )
+        caps.secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS;
 
     min = 0;
     opt = (VM_ENTRY_LOAD_GUEST_PAT | VM_ENTRY_LOAD_GUEST_EFER |
@@ -496,7 +494,6 @@ static int vmx_init_vmcs_config(bool bsp
     {
         /* First time through. */
         vmx_caps = caps;
-        vmx_secondary_exec_control = _vmx_secondary_exec_control;
         vmx_tertiary_exec_control  = _vmx_tertiary_exec_control;
         vmx_ept_vpid_cap           = _vmx_ept_vpid_cap;
         vmx_vmexit_control         = _vmx_vmexit_control;
@@ -531,7 +528,7 @@ static int vmx_init_vmcs_config(bool bsp
             vmx_caps.cpu_based_exec_control, caps.cpu_based_exec_control);
         mismatch |= cap_check(
             "Secondary Exec Control",
-            vmx_secondary_exec_control, _vmx_secondary_exec_control);
+            vmx_caps.secondary_exec_control, caps.secondary_exec_control);
         mismatch |= cap_check(
             "Tertiary Exec Control",
             vmx_tertiary_exec_control, _vmx_tertiary_exec_control);
@@ -1112,7 +1109,7 @@ static int construct_vmcs(struct vcpu *v
     if ( d->arch.vtsc && !cpu_has_vmx_tsc_scaling )
         v->arch.hvm.vmx.exec_control |= CPU_BASED_RDTSC_EXITING;
 
-    v->arch.hvm.vmx.secondary_exec_control = vmx_secondary_exec_control;
+    v->arch.hvm.vmx.secondary_exec_control = vmx_caps.secondary_exec_control;
     v->arch.hvm.vmx.tertiary_exec_control  = vmx_tertiary_exec_control;
 
     /*
@@ -2211,7 +2208,6 @@ int __init vmx_vmcs_init(void)
          * Make sure all dependent features are off as well.
          */
         memset(&vmx_caps, 0, sizeof(vmx_caps));
-        vmx_secondary_exec_control = 0;
         vmx_tertiary_exec_control  = 0;
         vmx_vmexit_control         = 0;
         vmx_vmentry_control        = 0;
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -258,7 +258,6 @@ extern u32 vmx_vmentry_control;
 #define SECONDARY_EXEC_TSC_SCALING              0x02000000U
 #define SECONDARY_EXEC_BUS_LOCK_DETECTION       0x40000000U
 #define SECONDARY_EXEC_NOTIFY_VM_EXITING        0x80000000U
-extern u32 vmx_secondary_exec_control;
 
 #define TERTIARY_EXEC_LOADIWKEY_EXITING         BIT(0, UL)
 #define TERTIARY_EXEC_ENABLE_HLAT               BIT(1, UL)
@@ -297,13 +296,14 @@ struct vmx_caps {
     uint64_t basic_msr;
     uint32_t pin_based_exec_control;
     uint32_t cpu_based_exec_control;
+    uint32_t secondary_exec_control;
 };
 extern struct vmx_caps vmx_caps;
 
 #define cpu_has_wbinvd_exiting \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_WBINVD_EXITING)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_WBINVD_EXITING)
 #define cpu_has_vmx_virtualize_apic_accesses \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES)
 #define cpu_has_vmx_tpr_shadow \
     (vmx_caps.cpu_based_exec_control & CPU_BASED_TPR_SHADOW)
 #define cpu_has_vmx_vnmi \
@@ -315,13 +315,13 @@ extern struct vmx_caps vmx_caps;
 #define cpu_has_vmx_tertiary_exec_control \
     (vmx_caps.cpu_based_exec_control & CPU_BASED_ACTIVATE_TERTIARY_CONTROLS)
 #define cpu_has_vmx_ept \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_EPT)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_EPT)
 #define cpu_has_vmx_dt_exiting \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_DESCRIPTOR_TABLE_EXITING)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_DESCRIPTOR_TABLE_EXITING)
 #define cpu_has_vmx_rdtscp \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_RDTSCP)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_RDTSCP)
 #define cpu_has_vmx_vpid \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VPID)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_VPID)
 #define cpu_has_monitor_trap_flag \
     (vmx_caps.cpu_based_exec_control & CPU_BASED_MONITOR_TRAP_FLAG)
 #define cpu_has_vmx_pat \
@@ -329,41 +329,41 @@ extern struct vmx_caps vmx_caps;
 #define cpu_has_vmx_efer \
     (vmx_vmentry_control & VM_ENTRY_LOAD_GUEST_EFER)
 #define cpu_has_vmx_unrestricted_guest \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_UNRESTRICTED_GUEST)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_UNRESTRICTED_GUEST)
 #define vmx_unrestricted_guest(v)               \
     ((v)->arch.hvm.vmx.secondary_exec_control & \
      SECONDARY_EXEC_UNRESTRICTED_GUEST)
 #define cpu_has_vmx_ple \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_PAUSE_LOOP_EXITING)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_PAUSE_LOOP_EXITING)
 #define cpu_has_vmx_invpcid \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_INVPCID)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_INVPCID)
 #define cpu_has_vmx_apic_reg_virt \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_APIC_REGISTER_VIRT)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_APIC_REGISTER_VIRT)
 #define cpu_has_vmx_virtual_intr_delivery \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY)
 #define cpu_has_vmx_virtualize_x2apic_mode \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE)
 #define cpu_has_vmx_posted_intr_processing \
     (vmx_caps.pin_based_exec_control & PIN_BASED_POSTED_INTERRUPT)
 #define cpu_has_vmx_vmcs_shadowing \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VMCS_SHADOWING)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_VMCS_SHADOWING)
 #define cpu_has_vmx_vmfunc \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VM_FUNCTIONS)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_VM_FUNCTIONS)
 #define cpu_has_vmx_virt_exceptions \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS)
 #define cpu_has_vmx_pml \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_PML)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_PML)
 #define cpu_has_vmx_mpx \
     ((vmx_vmexit_control & VM_EXIT_CLEAR_BNDCFGS) && \
      (vmx_vmentry_control & VM_ENTRY_LOAD_BNDCFGS))
 #define cpu_has_vmx_xsaves \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_XSAVES)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_XSAVES)
 #define cpu_has_vmx_tsc_scaling \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_TSC_SCALING)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_TSC_SCALING)
 #define cpu_has_vmx_bus_lock_detection \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_BUS_LOCK_DETECTION)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_BUS_LOCK_DETECTION)
 #define cpu_has_vmx_notify_vm_exiting \
-    (vmx_secondary_exec_control & SECONDARY_EXEC_NOTIFY_VM_EXITING)
+    (vmx_caps.secondary_exec_control & SECONDARY_EXEC_NOTIFY_VM_EXITING)
 
 #define VMCS_RID_TYPE_MASK              0x80000000U
 



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:44:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:44:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685654.1066731 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree5s-0001v0-HP; Mon, 26 Feb 2024 16:44:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685654.1066731; Mon, 26 Feb 2024 16:44:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree5s-0001ut-Cm; Mon, 26 Feb 2024 16:44:44 +0000
Received: by outflank-mailman (input) for mailman id 685654;
 Mon, 26 Feb 2024 16:44:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree5r-0000BK-Qh
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:44:43 +0000
Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com
 [2a00:1450:4864:20::12c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 57887c6a-d4c6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:44:43 +0100 (CET)
Received: by mail-lf1-x12c.google.com with SMTP id
 2adb3069b0e04-512ed314881so3157563e87.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:44:43 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.44.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:44:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 57887c6a-d4c6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965883; x=1709570683; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=DpCtM4zMbfCyBbShI+77WKapeO/xVJwq7zqsqI2VmFg=;
        b=fm8GVAolL6nMFJaAUUU2MWPYmWxqQEPe6O0WQcAVCcQYu1br5shPzIu47AlPKTvaT4
         mHG0qJErEQF+VlikQu6TmFO8LZJ3uuah+ch/iYw/YLI14Gl4WO70ZAe9Iqz6/F985oLB
         69OA6tKSElezHC5AwcWDEQzxZhHy4QeaQe7bQJHQEjLMATkX4rx0deKUyLPYJRW1W3qf
         7x+SRWjvkqyTV1icDeBsvjlvtXOL7IWxXIEZSmBi9tJL29EODIiIIdpqYlHIQY56qzEB
         qkw/2e4ZECznMVR7RhMGKhhloY/3TvevQSfOgw5nqnOXTut6YvWQF6mdbEufVatDPoA2
         z8nA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965883; x=1709570683;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=DpCtM4zMbfCyBbShI+77WKapeO/xVJwq7zqsqI2VmFg=;
        b=thdkJ/v/ybchkbTjzzcEg+7s/h9OaBpvM7aA1waPRi450QbZUWcbTAIQ6dF1nJgz9l
         968H2QwGH+JFt04+q2M1AmutS8B1GLs/6uxBj3N/wwHysqvEsA0cOyYvuqWtgWIJEtqS
         PRnlorppMr9mlu/NkC8S9T5HetbxRuRyzNbsoS0K7YfH2iLK5TvWbKOnIoymeSXGNvBT
         QTxevarNvk7Gr1QeHuKdwKHEoNQBopfnGvACu5Po7YsySXH+ZA9vs+VXf5Vp8v4e2EF1
         JUIn8x5sVZsF6weG63Cqm4rbnfIoHfvdz7Tt0Wnnx9Qgm1DU3ghJpGzr3v/gC3wfROm+
         HuUA==
X-Gm-Message-State: AOJu0YxR3S+0+AFkqiECSbtL512rpyV72LusZvt7sk9XIJ5jsM8zpYvE
	xPjOaIlTgBqLp/YXlpmNN+9LBwVx/cUT3syLuRawVWGqWkcPDuTAUQfh+skYJL7k9SFNhuvGOCA
	=
X-Google-Smtp-Source: AGHT+IGWMVZE6sGhplj0N1999L5E25vxWGu25pGEjmTVpzLVMIlUPUrCU2rcZGKQHzeM/4seWwncuQ==
X-Received: by 2002:a05:6512:159e:b0:512:fda7:2789 with SMTP id bp30-20020a056512159e00b00512fda72789mr2686709lfb.51.1708965882899;
        Mon, 26 Feb 2024 08:44:42 -0800 (PST)
Message-ID: <2bdfb8c2-c5ef-4631-9754-147310f18507@suse.com>
Date: Mon, 26 Feb 2024 17:44:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 08/12] VMX: convert vmx_tertiary_exec_control
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... to a field in the capability/controls struct.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v3: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -162,7 +162,6 @@ static int cf_check parse_ept_param_runt
 
 /* Dynamic (run-time adjusted) execution control flags. */
 struct vmx_caps __ro_after_init vmx_caps;
-uint64_t vmx_tertiary_exec_control __read_mostly;
 u32 vmx_vmexit_control __read_mostly;
 u32 vmx_vmentry_control __read_mostly;
 u64 vmx_ept_vpid_cap __read_mostly;
@@ -259,7 +258,6 @@ static int vmx_init_vmcs_config(bool bsp
 {
     u32 vmx_basic_msr_low, vmx_basic_msr_high, min, opt;
     struct vmx_caps caps = {};
-    uint64_t _vmx_tertiary_exec_control = 0;
     u64 _vmx_ept_vpid_cap = 0;
     u64 _vmx_misc_cap = 0;
     u32 _vmx_vmexit_control;
@@ -362,7 +360,7 @@ static int vmx_init_vmcs_config(bool bsp
     {
         uint64_t opt = 0;
 
-        _vmx_tertiary_exec_control = adjust_vmx_controls2(
+        caps.tertiary_exec_control = adjust_vmx_controls2(
             "Tertiary Exec Control", 0, opt,
             MSR_IA32_VMX_PROCBASED_CTLS3, &mismatch);
     }
@@ -494,7 +492,6 @@ static int vmx_init_vmcs_config(bool bsp
     {
         /* First time through. */
         vmx_caps = caps;
-        vmx_tertiary_exec_control  = _vmx_tertiary_exec_control;
         vmx_ept_vpid_cap           = _vmx_ept_vpid_cap;
         vmx_vmexit_control         = _vmx_vmexit_control;
         vmx_vmentry_control        = _vmx_vmentry_control;
@@ -531,7 +528,7 @@ static int vmx_init_vmcs_config(bool bsp
             vmx_caps.secondary_exec_control, caps.secondary_exec_control);
         mismatch |= cap_check(
             "Tertiary Exec Control",
-            vmx_tertiary_exec_control, _vmx_tertiary_exec_control);
+            vmx_caps.tertiary_exec_control, caps.tertiary_exec_control);
         mismatch |= cap_check(
             "VMExit Control",
             vmx_vmexit_control, _vmx_vmexit_control);
@@ -1110,7 +1107,7 @@ static int construct_vmcs(struct vcpu *v
         v->arch.hvm.vmx.exec_control |= CPU_BASED_RDTSC_EXITING;
 
     v->arch.hvm.vmx.secondary_exec_control = vmx_caps.secondary_exec_control;
-    v->arch.hvm.vmx.tertiary_exec_control  = vmx_tertiary_exec_control;
+    v->arch.hvm.vmx.tertiary_exec_control  = vmx_caps.tertiary_exec_control;
 
     /*
      * Disable features which we don't want active by default:
@@ -2208,7 +2205,6 @@ int __init vmx_vmcs_init(void)
          * Make sure all dependent features are off as well.
          */
         memset(&vmx_caps, 0, sizeof(vmx_caps));
-        vmx_tertiary_exec_control  = 0;
         vmx_vmexit_control         = 0;
         vmx_vmentry_control        = 0;
         vmx_ept_vpid_cap           = 0;
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -265,7 +265,6 @@ extern u32 vmx_vmentry_control;
 #define TERTIARY_EXEC_GUEST_PAGING_VERIFY       BIT(3, UL)
 #define TERTIARY_EXEC_IPI_VIRT                  BIT(4, UL)
 #define TERTIARY_EXEC_VIRT_SPEC_CTRL            BIT(7, UL)
-extern uint64_t vmx_tertiary_exec_control;
 
 #define VMX_EPT_EXEC_ONLY_SUPPORTED                         0x00000001
 #define VMX_EPT_WALK_LENGTH_4_SUPPORTED                     0x00000040
@@ -297,6 +296,7 @@ struct vmx_caps {
     uint32_t pin_based_exec_control;
     uint32_t cpu_based_exec_control;
     uint32_t secondary_exec_control;
+    uint64_t tertiary_exec_control;
 };
 extern struct vmx_caps vmx_caps;
 



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:45:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:45:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685655.1066739 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree6C-0002Py-NB; Mon, 26 Feb 2024 16:45:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685655.1066739; Mon, 26 Feb 2024 16:45:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree6C-0002Pr-KP; Mon, 26 Feb 2024 16:45:04 +0000
Received: by outflank-mailman (input) for mailman id 685655;
 Mon, 26 Feb 2024 16:45:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree6B-0000BK-7r
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:45:03 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 630a19a5-d4c6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:45:02 +0100 (CET)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-55a8fd60af0so4313635a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:45:02 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.45.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:45:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 630a19a5-d4c6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965902; x=1709570702; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=qlvyaBxTJU8IjuZfhNaBPKvna3qwUHhOtqZ6G2oyic0=;
        b=Y2w1x6CUqI9wE/GfptBvw1f/ZbhfcI/IyaqOnCX1nQAwwKv0wRZzNcJ8m3sWZu5Vgl
         vOiNHprok6ftTbCBvJT1SxqhW1ALm0S2iPR12cIIJNuQWHRSf5f3CwqE7YVKE+9ohfP4
         T+m8pgDO/MErGLu/XnEKbPoQ4T8zUnVFREoTI50KXWSdbPxNyoVMxD4BAnX6HUwYHDyu
         zit1XrojLOLL+y6fQ5cuUUCe97h/rAd44CMMG2faPzILzckwGpsYVyoeKXE3bt7pJKGW
         tMKs6kK4dP7u+U1iAAJoEGkyXq/9IOp4Xft9OhIAJV7UhyDe3EW6DExiHPItd1CPNqiY
         PRLQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965902; x=1709570702;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=qlvyaBxTJU8IjuZfhNaBPKvna3qwUHhOtqZ6G2oyic0=;
        b=vGyXS1VCk1H69KgYsgKcF5iS6AHhzjAi4hbIeWm0NDG+lXlskgPRO9AGEk3X1xFsaV
         T1YkC2LWoFuIseP7u3s0JdjL+QWPL7ior4bdi8mieTjM3CKfKBzoTK/2FZ/DcTBIX9Sw
         Rz4t/BMjUqvommSc1rxSQV6eZXLMSPenxmq+IVnymn8HHGxquAsOnpCJQ9h/Fu87MGGN
         ljNG1rYR6RKcbpNfr3exlgJdN3IeLlXYmcfc7hsGSK3dn8t1gLmhZ92UusrX6sEytQhz
         9WDk4tkiFe1vF3Q+QobE9xvEEauwRzRwWBW17SCcprlsz7Z3iadTOchYPtijC90pOGlW
         ZywA==
X-Gm-Message-State: AOJu0YydpZS3XVmrsi8A2uu/tzVuKN99dqubFdOXl7jIG2vjom8lUfSD
	DqKd9JYidJr5LqNjiFbWaEqPp/3b4UFucND8rzijDPzXQ/nMaBToCoLEa4km7/B2I+3gzfGsSIM
	=
X-Google-Smtp-Source: AGHT+IGmLDwC13pcsL6fhS7ZmCc853YcZ4+1dBnx0uXrFWGkrq1vusQsnSjUr1bG8naqR66JU1KvCw==
X-Received: by 2002:aa7:df93:0:b0:565:e294:5119 with SMTP id b19-20020aa7df93000000b00565e2945119mr2370985edy.18.1708965902142;
        Mon, 26 Feb 2024 08:45:02 -0800 (PST)
Message-ID: <6c273891-8232-4de3-97e1-8f39a174ba32@suse.com>
Date: Mon, 26 Feb 2024 17:45:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 09/12] VMX: convert vmx_vmexit_control
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... to a field in the capability/controls struct.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -162,7 +162,6 @@ static int cf_check parse_ept_param_runt
 
 /* Dynamic (run-time adjusted) execution control flags. */
 struct vmx_caps __ro_after_init vmx_caps;
-u32 vmx_vmexit_control __read_mostly;
 u32 vmx_vmentry_control __read_mostly;
 u64 vmx_ept_vpid_cap __read_mostly;
 static uint64_t __read_mostly vmx_vmfunc;
@@ -260,7 +259,6 @@ static int vmx_init_vmcs_config(bool bsp
     struct vmx_caps caps = {};
     u64 _vmx_ept_vpid_cap = 0;
     u64 _vmx_misc_cap = 0;
-    u32 _vmx_vmexit_control;
     u32 _vmx_vmentry_control;
     u64 _vmx_vmfunc = 0;
     bool mismatch = false;
@@ -442,7 +440,7 @@ static int vmx_init_vmcs_config(bool bsp
     opt = (VM_EXIT_SAVE_GUEST_PAT | VM_EXIT_LOAD_HOST_PAT |
            VM_EXIT_LOAD_HOST_EFER | VM_EXIT_CLEAR_BNDCFGS);
     min |= VM_EXIT_IA32E_MODE;
-    _vmx_vmexit_control = adjust_vmx_controls(
+    caps.vmexit_control = adjust_vmx_controls(
         "VMExit Control", min, opt, MSR_IA32_VMX_EXIT_CTLS, &mismatch);
 
     /*
@@ -493,7 +491,6 @@ static int vmx_init_vmcs_config(bool bsp
         /* First time through. */
         vmx_caps = caps;
         vmx_ept_vpid_cap           = _vmx_ept_vpid_cap;
-        vmx_vmexit_control         = _vmx_vmexit_control;
         vmx_vmentry_control        = _vmx_vmentry_control;
         vmx_caps.basic_msr = ((uint64_t)vmx_basic_msr_high << 32) |
                              vmx_basic_msr_low;
@@ -531,7 +528,7 @@ static int vmx_init_vmcs_config(bool bsp
             vmx_caps.tertiary_exec_control, caps.tertiary_exec_control);
         mismatch |= cap_check(
             "VMExit Control",
-            vmx_vmexit_control, _vmx_vmexit_control);
+            vmx_caps.vmexit_control, caps.vmexit_control);
         mismatch |= cap_check(
             "VMEntry Control",
             vmx_vmentry_control, _vmx_vmentry_control);
@@ -1093,7 +1090,7 @@ void nocall vmx_asm_vmexit_handler(void)
 static int construct_vmcs(struct vcpu *v)
 {
     struct domain *d = v->domain;
-    u32 vmexit_ctl = vmx_vmexit_control;
+    uint32_t vmexit_ctl = vmx_caps.vmexit_control;
     u32 vmentry_ctl = vmx_vmentry_control;
     int rc = 0;
 
@@ -2205,7 +2202,6 @@ int __init vmx_vmcs_init(void)
          * Make sure all dependent features are off as well.
          */
         memset(&vmx_caps, 0, sizeof(vmx_caps));
-        vmx_vmexit_control         = 0;
         vmx_vmentry_control        = 0;
         vmx_ept_vpid_cap           = 0;
         vmx_vmfunc                 = 0;
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -1060,7 +1060,7 @@ static void load_shadow_control(struct v
     nvmx_update_pin_control(v, vmx_caps.pin_based_exec_control);
     vmx_update_cpu_exec_control(v);
     vmx_update_secondary_exec_control(v);
-    nvmx_update_exit_control(v, vmx_vmexit_control);
+    nvmx_update_exit_control(v, vmx_caps.vmexit_control);
     nvmx_update_entry_control(v);
     vmx_update_exception_bitmap(v);
     nvmx_update_apic_access_address(v);
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -227,7 +227,6 @@ void vmx_vmcs_reload(struct vcpu *v);
 #define VM_EXIT_LOAD_HOST_EFER          0x00200000
 #define VM_EXIT_SAVE_PREEMPT_TIMER      0x00400000
 #define VM_EXIT_CLEAR_BNDCFGS           0x00800000
-extern u32 vmx_vmexit_control;
 
 #define VM_ENTRY_IA32E_MODE             0x00000200
 #define VM_ENTRY_SMM                    0x00000400
@@ -297,6 +296,7 @@ struct vmx_caps {
     uint32_t cpu_based_exec_control;
     uint32_t secondary_exec_control;
     uint64_t tertiary_exec_control;
+    uint32_t vmexit_control;
 };
 extern struct vmx_caps vmx_caps;
 
@@ -354,7 +354,7 @@ extern struct vmx_caps vmx_caps;
 #define cpu_has_vmx_pml \
     (vmx_caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_PML)
 #define cpu_has_vmx_mpx \
-    ((vmx_vmexit_control & VM_EXIT_CLEAR_BNDCFGS) && \
+    ((vmx_caps.vmexit_control & VM_EXIT_CLEAR_BNDCFGS) && \
      (vmx_vmentry_control & VM_ENTRY_LOAD_BNDCFGS))
 #define cpu_has_vmx_xsaves \
     (vmx_caps.secondary_exec_control & SECONDARY_EXEC_XSAVES)



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:45:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:45:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685658.1066750 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree6o-00032g-W1; Mon, 26 Feb 2024 16:45:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685658.1066750; Mon, 26 Feb 2024 16:45:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree6o-00032Z-Sk; Mon, 26 Feb 2024 16:45:42 +0000
Received: by outflank-mailman (input) for mailman id 685658;
 Mon, 26 Feb 2024 16:45:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree6n-0002xA-GV
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:45:41 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 79128c5f-d4c6-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 17:45:39 +0100 (CET)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-563bb51c36eso3454948a12.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:45:39 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.45.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:45:38 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 79128c5f-d4c6-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965939; x=1709570739; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=MWVzS/z9C2M+MglkZ5k6DSCrcfyy95qeh7mUQUnz0es=;
        b=eYgL0tHH4HNv4aJwdyuBfYsA38P86jydDmCgdFI47eEc1mnmFPWbM7sTiBhh/CwQLC
         sXL1njilJl5sER618ME1FUNXzGuwgCX+d/uiUpI+1bhE74Zj/8DhQDyyAbp+/lWmU9Oq
         52R0QnKbpR4a6z8dGnIV19L8IWFdgX/Njht5KHnYgzVPXkEAjRi0huIGN8ybNhJVsZ8u
         Z8QDisAl7jGwGTLrICR13UAmf4ELgYWyEa/6Zht9E398CbvS4iuBPGCwKSqb7et3jKGx
         9oQwri9qqk1FMzVuBJVZs6vDCP/NeYL1kk1Nrik89FS0CoPuU4bN50Yni1Pqk62J6uIG
         ZvQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965939; x=1709570739;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=MWVzS/z9C2M+MglkZ5k6DSCrcfyy95qeh7mUQUnz0es=;
        b=JAdeL6H8tydfcayWVH1yMRjNS7982WJ0499qbM1j2vtsv7ntm8SWPV+myThNEXzm/d
         OmP96AZs0CcnS0gXkzuyOIKzWh5jqYVfgM0D3cdvx17Ef7nH/KDJR3kchK5MJyeFnYxi
         XtoObexJYbQ67SXm1aiZbJjuRyto4aCH3hk4f/rrfNW8LQLT2T90VymE3IJ1cRObs0z9
         9hOAISxDiYWJ4mL8oopmbEqUVpvdXGXel1C+SuTWv1qzuWK4KB9GSRFyXtf5r6NkeZ9c
         MEiV0IsAMUAkADAc3u0tnb9xfragYaPoH7Bs/wG5FVU7ASnMWGE7y0erxizDt6Uj7v3W
         /ARg==
X-Gm-Message-State: AOJu0YxyBphGkKOJsL3sNMzLoibwuILJ75Ju69kU+yO4+k3brGWJYPc0
	ULT31JcjbEyGQnHZloYAKJOYwAbemzGMzTG16qGi2IMhkviycxGwDFOZrYBxZDD1Gkbo+yNSlzc
	=
X-Google-Smtp-Source: AGHT+IFaiW5EwwFtgffnDVKX8BnLEM5/5qyGYEMeDX9Hsb4btZCx+5AHYu7RF9I4onhIURplq6pWLA==
X-Received: by 2002:aa7:c389:0:b0:566:137e:e076 with SMTP id k9-20020aa7c389000000b00566137ee076mr1201621edq.19.1708965939148;
        Mon, 26 Feb 2024 08:45:39 -0800 (PST)
Message-ID: <edce4800-e1fc-43d7-b7ed-ed7e1be43bcb@suse.com>
Date: Mon, 26 Feb 2024 17:45:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 11/12] VMX: convert vmx_ept_vpid_cap
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... to fields in the capability/controls struct: Take the opportunity
and split the two halves into separate EPT and VPID fields.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v3: Re-base.
v2: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -162,7 +162,6 @@ static int cf_check parse_ept_param_runt
 
 /* Dynamic (run-time adjusted) execution control flags. */
 struct vmx_caps __ro_after_init vmx_caps;
-u64 vmx_ept_vpid_cap __read_mostly;
 static uint64_t __read_mostly vmx_vmfunc;
 
 static DEFINE_PER_CPU_READ_MOSTLY(paddr_t, vmxon_region);
@@ -256,7 +255,6 @@ static int vmx_init_vmcs_config(bool bsp
 {
     u32 vmx_basic_msr_low, vmx_basic_msr_high, min, opt;
     struct vmx_caps caps = {};
-    u64 _vmx_ept_vpid_cap = 0;
     u64 _vmx_misc_cap = 0;
     u64 _vmx_vmfunc = 0;
     bool mismatch = false;
@@ -365,10 +363,10 @@ static int vmx_init_vmcs_config(bool bsp
     if ( caps.secondary_exec_control & (SECONDARY_EXEC_ENABLE_EPT |
                                         SECONDARY_EXEC_ENABLE_VPID) )
     {
-        rdmsrl(MSR_IA32_VMX_EPT_VPID_CAP, _vmx_ept_vpid_cap);
+        rdmsr(MSR_IA32_VMX_EPT_VPID_CAP, caps.ept, caps.vpid);
 
         if ( !opt_ept_ad )
-            _vmx_ept_vpid_cap &= ~VMX_EPT_AD_BIT;
+            caps.ept &= ~VMX_EPT_AD_BIT;
 
         /*
          * Additional sanity checking before using EPT:
@@ -381,9 +379,9 @@ static int vmx_init_vmcs_config(bool bsp
          *
          * Or we just don't use EPT.
          */
-        if ( !(_vmx_ept_vpid_cap & VMX_EPT_MEMORY_TYPE_WB) ||
-             !(_vmx_ept_vpid_cap & VMX_EPT_WALK_LENGTH_4_SUPPORTED) ||
-             !(_vmx_ept_vpid_cap & VMX_EPT_INVEPT_ALL_CONTEXT) )
+        if ( !(caps.ept & VMX_EPT_MEMORY_TYPE_WB) ||
+             !(caps.ept & VMX_EPT_WALK_LENGTH_4_SUPPORTED) ||
+             !(caps.ept & VMX_EPT_INVEPT_ALL_CONTEXT) )
             caps.secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_EPT;
 
         /*
@@ -392,11 +390,11 @@ static int vmx_init_vmcs_config(bool bsp
          *
          * Or we just don't use VPID.
          */
-        if ( !(_vmx_ept_vpid_cap & VMX_VPID_INVVPID_ALL_CONTEXT) )
+        if ( !(caps.vpid & VMX_VPID_INVVPID_ALL_CONTEXT) )
             caps.secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_VPID;
 
         /* EPT A/D bits is required for PML */
-        if ( !(_vmx_ept_vpid_cap & VMX_EPT_AD_BIT) )
+        if ( !(caps.ept & VMX_EPT_AD_BIT) )
             caps.secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_PML;
     }
 
@@ -488,7 +486,6 @@ static int vmx_init_vmcs_config(bool bsp
     {
         /* First time through. */
         vmx_caps = caps;
-        vmx_ept_vpid_cap           = _vmx_ept_vpid_cap;
         vmx_caps.basic_msr = ((uint64_t)vmx_basic_msr_high << 32) |
                              vmx_basic_msr_low;
         vmx_vmfunc                 = _vmx_vmfunc;
@@ -529,9 +526,8 @@ static int vmx_init_vmcs_config(bool bsp
         mismatch |= cap_check(
             "VMEntry Control",
             vmx_caps.vmentry_control, caps.vmentry_control);
-        mismatch |= cap_check(
-            "EPT and VPID Capability",
-            vmx_ept_vpid_cap, _vmx_ept_vpid_cap);
+        mismatch |= cap_check("EPT Capability", vmx_caps.ept, caps.ept);
+        mismatch |= cap_check("VPID Capability", vmx_caps.vpid, caps.vpid);
         mismatch |= cap_check(
             "VMFUNC Capability",
             vmx_vmfunc, _vmx_vmfunc);
@@ -2199,7 +2195,6 @@ int __init vmx_vmcs_init(void)
          * Make sure all dependent features are off as well.
          */
         memset(&vmx_caps, 0, sizeof(vmx_caps));
-        vmx_ept_vpid_cap           = 0;
         vmx_vmfunc                 = 0;
     }
 
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -274,12 +274,11 @@ void vmx_vmcs_reload(struct vcpu *v);
 #define VMX_EPT_AD_BIT                                      0x00200000
 #define VMX_EPT_INVEPT_SINGLE_CONTEXT                       0x02000000
 #define VMX_EPT_INVEPT_ALL_CONTEXT                          0x04000000
-#define VMX_VPID_INVVPID_INSTRUCTION                     0x00100000000ULL
-#define VMX_VPID_INVVPID_INDIVIDUAL_ADDR                 0x10000000000ULL
-#define VMX_VPID_INVVPID_SINGLE_CONTEXT                  0x20000000000ULL
-#define VMX_VPID_INVVPID_ALL_CONTEXT                     0x40000000000ULL
-#define VMX_VPID_INVVPID_SINGLE_CONTEXT_RETAINING_GLOBAL 0x80000000000ULL
-extern u64 vmx_ept_vpid_cap;
+#define VMX_VPID_INVVPID_INSTRUCTION                        0x00000001
+#define VMX_VPID_INVVPID_INDIVIDUAL_ADDR                    0x00000100
+#define VMX_VPID_INVVPID_SINGLE_CONTEXT                     0x00000200
+#define VMX_VPID_INVVPID_ALL_CONTEXT                        0x00000400
+#define VMX_VPID_INVVPID_SINGLE_CONTEXT_RETAINING_GLOBAL    0x00000800
 
 #define VMX_MISC_ACTIVITY_MASK                  0x000001c0
 #define VMX_MISC_PROC_TRACE                     0x00004000
@@ -297,6 +296,8 @@ struct vmx_caps {
     uint64_t tertiary_exec_control;
     uint32_t vmexit_control;
     uint32_t vmentry_control;
+    uint32_t ept;
+    uint32_t vpid;
 };
 extern struct vmx_caps vmx_caps;
 
--- a/xen/arch/x86/include/asm/hvm/vmx/vmx.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmx.h
@@ -279,17 +279,17 @@ typedef union cr_access_qual {
 extern uint8_t posted_intr_vector;
 
 #define cpu_has_vmx_ept_exec_only_supported        \
-    (vmx_ept_vpid_cap & VMX_EPT_EXEC_ONLY_SUPPORTED)
+    (vmx_caps.ept & VMX_EPT_EXEC_ONLY_SUPPORTED)
 
 #define cpu_has_vmx_ept_wl4_supported           \
-    (vmx_ept_vpid_cap & VMX_EPT_WALK_LENGTH_4_SUPPORTED)
-#define cpu_has_vmx_ept_mt_uc (vmx_ept_vpid_cap & VMX_EPT_MEMORY_TYPE_UC)
-#define cpu_has_vmx_ept_mt_wb (vmx_ept_vpid_cap & VMX_EPT_MEMORY_TYPE_WB)
-#define cpu_has_vmx_ept_2mb   (vmx_ept_vpid_cap & VMX_EPT_SUPERPAGE_2MB)
-#define cpu_has_vmx_ept_1gb   (vmx_ept_vpid_cap & VMX_EPT_SUPERPAGE_1GB)
-#define cpu_has_vmx_ept_ad    (vmx_ept_vpid_cap & VMX_EPT_AD_BIT)
+    (vmx_caps.ept & VMX_EPT_WALK_LENGTH_4_SUPPORTED)
+#define cpu_has_vmx_ept_mt_uc (vmx_caps.ept & VMX_EPT_MEMORY_TYPE_UC)
+#define cpu_has_vmx_ept_mt_wb (vmx_caps.ept & VMX_EPT_MEMORY_TYPE_WB)
+#define cpu_has_vmx_ept_2mb   (vmx_caps.ept & VMX_EPT_SUPERPAGE_2MB)
+#define cpu_has_vmx_ept_1gb   (vmx_caps.ept & VMX_EPT_SUPERPAGE_1GB)
+#define cpu_has_vmx_ept_ad    (vmx_caps.ept & VMX_EPT_AD_BIT)
 #define cpu_has_vmx_ept_invept_single_context   \
-    (vmx_ept_vpid_cap & VMX_EPT_INVEPT_SINGLE_CONTEXT)
+    (vmx_caps.ept & VMX_EPT_INVEPT_SINGLE_CONTEXT)
 
 #define EPT_2MB_SHIFT     16
 #define EPT_1GB_SHIFT     17
@@ -300,11 +300,11 @@ extern uint8_t posted_intr_vector;
 #define INVEPT_ALL_CONTEXT      2
 
 #define cpu_has_vmx_vpid_invvpid_individual_addr                    \
-    (vmx_ept_vpid_cap & VMX_VPID_INVVPID_INDIVIDUAL_ADDR)
+    (vmx_caps.vpid & VMX_VPID_INVVPID_INDIVIDUAL_ADDR)
 #define cpu_has_vmx_vpid_invvpid_single_context                     \
-    (vmx_ept_vpid_cap & VMX_VPID_INVVPID_SINGLE_CONTEXT)
+    (vmx_caps.vpid & VMX_VPID_INVVPID_SINGLE_CONTEXT)
 #define cpu_has_vmx_vpid_invvpid_single_context_retaining_global    \
-    (vmx_ept_vpid_cap & VMX_VPID_INVVPID_SINGLE_CONTEXT_RETAINING_GLOBAL)
+    (vmx_caps.vpid & VMX_VPID_INVVPID_SINGLE_CONTEXT_RETAINING_GLOBAL)
 
 #define INVVPID_INDIVIDUAL_ADDR                 0
 #define INVVPID_SINGLE_CONTEXT                  1



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:46:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:46:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685660.1066759 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree7A-0003Xp-B9; Mon, 26 Feb 2024 16:46:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685660.1066759; Mon, 26 Feb 2024 16:46:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree7A-0003Xi-8R; Mon, 26 Feb 2024 16:46:04 +0000
Received: by outflank-mailman (input) for mailman id 685660;
 Mon, 26 Feb 2024 16:46:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree78-0003OP-DE
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:46:02 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 865f386b-d4c6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:46:01 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-565b434f90aso2300408a12.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:46:01 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.46.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:46:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 865f386b-d4c6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965961; x=1709570761; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=dDYzTd9wYxsVXEBeurRAELomc9weD9fuwjmladT865s=;
        b=CUbO0BsfOIUCLnCKu9fQhKqjBKAtDo4QJZGwelMu14Z6cgPjCm3pFxv6DGqOpJRJsA
         bkCe/y6ejFmvKEiJ5NI4lw05NbXy/OCr0hksUPslS6TEB1QquiIaGjTIO9E15OGqUIWL
         CkrhLwic81YeZlQ40vOMzqZMGIx5n/3W7/06fGf+shv+8p+puN9oaqkWKz7sxmDvzzu0
         i3szoA/FvHiZXo2U5NIhb9t8JKhd//LGVP/zg/PJdo+tVuyW30IxSnpNWqd5XbrUDTFT
         QOhyiZL/v08MDIWDR9XqBIjPeXaLfrhhYshbP7jLEiH+V7+mD3jmXhyqFK28qdE0RRjD
         E0jA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965961; x=1709570761;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=dDYzTd9wYxsVXEBeurRAELomc9weD9fuwjmladT865s=;
        b=cG6gff/dtVbA7mtQK1iK/pk7CcFkT8fhdPtNYssRTlMr72hEG3zmpfm3NSp63MgctF
         E7xCSieNNV2jg6dWxt9t2eyuCphJRtQexUCTbY5GH60cWf0exbA/IKy0oxAN/MK4/TF3
         SZVO5JgHonl8kiAtew/Gm+XZVVq9Gxc8t+xvK3sQ+BH70m02gzb4wKUYRqPjHDBwDSga
         54DWrGO6m3y+DhgpQpzsDaGuECIpwvnbZTiirZcgvSs8jKd3C3LVD5+snNftev8Al8PG
         V4fZHA7QP6ot6A/D4xJS/mUi8rEeueRF6A4ABny2gkXJvEhIndtPaLuUilZND82PDN6H
         Rukw==
X-Gm-Message-State: AOJu0YxIjZ2n7gSJbz/mRYRRakGHS0S3bzKiy9s9cRsOW0FXoVq1J5Ht
	/rSEhsSjDwCC+9N54Vx9NbVOPOAXgAsONxNd2u0Ks8xeGhQbVDnXm6Qs9KMUUo0AqwMFfcCEiDA
	=
X-Google-Smtp-Source: AGHT+IEmtrBcVHu3UxTiqUGsoYFdIPDz0EMOfeYv3hTO8JhDlkEH7AAjpMwtSw/ojpcyq6wZQ/avBw==
X-Received: by 2002:a50:fb0e:0:b0:565:cda7:4b88 with SMTP id d14-20020a50fb0e000000b00565cda74b88mr3236704edq.37.1708965961529;
        Mon, 26 Feb 2024 08:46:01 -0800 (PST)
Message-ID: <1c94855b-884e-4207-aa52-6398c5b9393a@suse.com>
Date: Mon, 26 Feb 2024 17:46:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 12/12] VMX: convert vmx_vmfunc
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... to a field in the capability/controls struct.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -162,7 +162,6 @@ static int cf_check parse_ept_param_runt
 
 /* Dynamic (run-time adjusted) execution control flags. */
 struct vmx_caps __ro_after_init vmx_caps;
-static uint64_t __read_mostly vmx_vmfunc;
 
 static DEFINE_PER_CPU_READ_MOSTLY(paddr_t, vmxon_region);
 static DEFINE_PER_CPU(paddr_t, current_vmcs);
@@ -256,7 +255,6 @@ static int vmx_init_vmcs_config(bool bsp
     u32 vmx_basic_msr_low, vmx_basic_msr_high, min, opt;
     struct vmx_caps caps = {};
     u64 _vmx_misc_cap = 0;
-    u64 _vmx_vmfunc = 0;
     bool mismatch = false;
 
     rdmsr(MSR_IA32_VMX_BASIC, vmx_basic_msr_low, vmx_basic_msr_high);
@@ -458,14 +456,14 @@ static int vmx_init_vmcs_config(bool bsp
     /* The IA32_VMX_VMFUNC MSR exists only when VMFUNC is available */
     if ( caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_VM_FUNCTIONS )
     {
-        rdmsrl(MSR_IA32_VMX_VMFUNC, _vmx_vmfunc);
+        rdmsrl(MSR_IA32_VMX_VMFUNC, caps.vmfunc);
 
         /*
          * VMFUNC leaf 0 (EPTP switching) must be supported.
          *
          * Or we just don't use VMFUNC.
          */
-        if ( !(_vmx_vmfunc & VMX_VMFUNC_EPTP_SWITCHING) )
+        if ( !(caps.vmfunc & VMX_VMFUNC_EPTP_SWITCHING) )
             caps.secondary_exec_control &= ~SECONDARY_EXEC_ENABLE_VM_FUNCTIONS;
     }
 
@@ -488,7 +486,6 @@ static int vmx_init_vmcs_config(bool bsp
         vmx_caps = caps;
         vmx_caps.basic_msr = ((uint64_t)vmx_basic_msr_high << 32) |
                              vmx_basic_msr_low;
-        vmx_vmfunc                 = _vmx_vmfunc;
 
         vmx_display_features();
 
@@ -530,7 +527,7 @@ static int vmx_init_vmcs_config(bool bsp
         mismatch |= cap_check("VPID Capability", vmx_caps.vpid, caps.vpid);
         mismatch |= cap_check(
             "VMFUNC Capability",
-            vmx_vmfunc, _vmx_vmfunc);
+            vmx_caps.vmfunc, caps.vmfunc);
         if ( cpu_has_vmx_ins_outs_instr_info !=
              !!(vmx_basic_msr_high & (VMX_BASIC_INS_OUT_INFO >> 32)) )
         {
@@ -2195,7 +2192,6 @@ int __init vmx_vmcs_init(void)
          * Make sure all dependent features are off as well.
          */
         memset(&vmx_caps, 0, sizeof(vmx_caps));
-        vmx_vmfunc                 = 0;
     }
 
     return ret;
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -298,6 +298,7 @@ struct vmx_caps {
     uint32_t vmentry_control;
     uint32_t ept;
     uint32_t vpid;
+    uint64_t vmfunc;
 };
 extern struct vmx_caps vmx_caps;
 



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:47:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:47:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685673.1066770 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree8l-0004FF-M0; Mon, 26 Feb 2024 16:47:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685673.1066770; Mon, 26 Feb 2024 16:47:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ree8l-0004F8-IZ; Mon, 26 Feb 2024 16:47:43 +0000
Received: by outflank-mailman (input) for mailman id 685673;
 Mon, 26 Feb 2024 16:47:42 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1ree8k-0004Ey-S0
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:47:42 +0000
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com
 [2a00:1450:4864:20::12b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c21c1362-d4c6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:47:42 +0100 (CET)
Received: by mail-lf1-x12b.google.com with SMTP id
 2adb3069b0e04-512bce554a5so4193581e87.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:47:42 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 k14-20020a05620a0b8e00b00787b7732c0csm2638433qkh.4.2024.02.26.08.47.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:47:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c21c1362-d4c6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708966062; x=1709570862; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=FvRQNVsEjOsBL3aW6gixEzUas6FNPtLG7gqR3J12VzY=;
        b=d37MJqr9ykhr4KlmFRJh0Gfv/wqQT8H6ANeu6kvcqD+GR/yD9Y3ZwRsQXvPPzDqI0Z
         l6ZQpkv4uA1plvhpL6QtPUMZl8UNHpck2xj9XN6tnf2FxgSy0cpxbSKDx8wf9WsbxGZV
         IT2hzb3hODxKbRTcV2dMnB5EIVJwn/DpvIK3Q=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708966062; x=1709570862;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:to
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=FvRQNVsEjOsBL3aW6gixEzUas6FNPtLG7gqR3J12VzY=;
        b=J/zUp0WPzXq4L26L2NwzeK5B7rr0BGv7vrFN3ZIqrzBWjUlMTskLz1haqm4x1jEsHq
         6+0zLRFvHpuvv0cS+IbSCS2/zc0z0W8FYRW1ImITdIZ1Wcs6ReQHvh7sg0T5Mqfycp5k
         VFi4HOHDhxKQf6hA9/8WU+/SMCqjF+RYPE42U685OFehmrTvT/tw1acOUK6SXzKjJiHQ
         SX4FVp8TVEFQX3kTR+4rkK5HEajWGadtJr0N+Ym25Ykw5zp7bW/KD5DVY0UaXiOG9myN
         CkKUfrGaHEXezImCU74xzXd/TQdXbdib4WJlstELPxttLdPP9dnCxSqCSKZ5Ourjzj9Y
         swrw==
X-Forwarded-Encrypted: i=1; AJvYcCWj3Da6dID2RZcwmkq6THlymQB8TxBbjpiJOrNbdk8/QplC+3+Z6o6e6XOi3TC4cGkaYGa2J+tW+5Q9aJIN9Mqvy/NDSNt4FczWwbpafeU=
X-Gm-Message-State: AOJu0YzVEaT4Yfd8pm/D61tbuHk+ouGepsriz+m/2/TdgNXxAeVBgHJt
	5s3w7vtlK+L1mohbofP3X8K/DjXvmUFE5DTM9RS7N+8dJKujeoKC8+9Ht8OxFzDo7/1+vDernu9
	G
X-Google-Smtp-Source: AGHT+IF4fH+d/i/9AlJkCMBGxjqnicK1Kp13C43gdEdQBpl5OfUaWvqqKZMxq168vPVv+dlEcNb/HA==
X-Received: by 2002:a05:6512:3e9:b0:512:c9bd:2388 with SMTP id n9-20020a05651203e900b00512c9bd2388mr3913258lfq.67.1708966061668;
        Mon, 26 Feb 2024 08:47:41 -0800 (PST)
Message-ID: <03677752-2a39-4090-8d8b-12a82d82030d@citrix.com>
Date: Mon, 26 Feb 2024 16:47:39 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 5/6] nestedsvm: Remove bogus debug message from
 nestedsvm_check_intercepts
Content-Language: en-GB
To: George Dunlap <george.dunlap@cloud.com>, xen-devel@lists.xenproject.org
References: <20240206012051.3564035-1-george.dunlap@cloud.com>
 <20240206012051.3564035-6-george.dunlap@cloud.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240206012051.3564035-6-george.dunlap@cloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 06/02/2024 1:20 am, George Dunlap wrote:
> Changeset ef3e8db8068 ("x86/hvm: Corrections and improvements to
> unhandled vmexit logging") introduced a printk to the default path of
> the switch statement in nestedsvm_check_intercepts(), complaining of
> an unknown exit reason.
>
> Unfortunately, the "core" switch statement which is meant to handle
> all vmexit reasons is in nsvm_vmcb_guest_intercepts_exitcode(); the
> switch statement in nestedsvm_check_intercepts() is only meant to
> superimpose on top of that some special-casing for how to interaction
> between L1 and L0 vmexits.
>
> Remove the printk, and add a comment to prevent future confusion.
>
> Signed-off-by: George Dunlap <george.dunlap@cloud.com>

Erm...   The addition of this printk was very deliberate, to point out
where security fixes are needed.

It's not bogus in the slightest.  It is an error for exit reasons to not
be inspected for safety in this path.

Please revert this patch.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:49:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:49:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685675.1066779 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reeAV-00052W-Vq; Mon, 26 Feb 2024 16:49:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685675.1066779; Mon, 26 Feb 2024 16:49:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reeAV-00052P-TL; Mon, 26 Feb 2024 16:49:31 +0000
Received: by outflank-mailman (input) for mailman id 685675;
 Mon, 26 Feb 2024 16:49:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1ree6R-0000BK-U3
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:45:19 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6cfdc0e4-d4c6-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:45:19 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-512bd533be0so3923413e87.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:45:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d18-20020a056402401200b005640022af58sm2509237eda.83.2024.02.26.08.45.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:45:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6cfdc0e4-d4c6-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708965919; x=1709570719; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=XYZOa3J14PwMoF80jvxD0VhtxRYvyB0LOPQq6Y/apVY=;
        b=RsKSjMVR+mNCv97sPMaKUUhTvDzU+HokmPK7Qg98ElUbHU2RLXdd372r4sEhM1SZgK
         7uJsuvlvB/ePqvfF9QhaqFs2OkyJJBWLCajnagiAALsxnqPV/Nyb9j1dzGbP95Ym6aRh
         ooAxiDaA2vNlj6PbyF+wZ2G/+bIOMlv2Yi5LoTFbjt276IHbOA4PP5oT3MZCdxg/v8C1
         5G7kX1l6d3ZCxHHaKT+sqWMNv/pRL60Wzhu8V1EBV2udVpmb+YFn9Fsp8yn0u/EssWmC
         /BlxvIhko5sKL90XBrLpPAzESYuINEzrPX/yG4yBesQSgtE9NBentBdZKu9UXdQMlYyj
         mcVQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708965919; x=1709570719;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=XYZOa3J14PwMoF80jvxD0VhtxRYvyB0LOPQq6Y/apVY=;
        b=KMoSvHppyeC7uUCuX80xl2WNm+DmHUio/WzQHxwnVbJsvqeiRC2mpHZbhRf7n1eR1f
         Mg0v6Xjfl9cpDGz3waigNW/gijNSiOUchsBiNu4dGRX7HkF37OPYN/0LoMJ1Dqzm9dvx
         h1z8+O6TidLmatRk2eFckcjdjEWSD6CrSl5Uu29C21AchxJiBvctdZFjekEAyIDHejqp
         Ayh6UQVO4yeXQNp66+qLc0CyZ4nrz+xLnzpP8llo3fm9vaKx3luiiVdlOWDDMZylEoAo
         bXV72qD20QGhdc3BrD5BtwMYndaxtpT/tf1CFa3GQtU857GoxjBmE7CZbWbiXL2Q5YUA
         ImFA==
X-Gm-Message-State: AOJu0YwS30+2V3yknpPOPmmrgixh0K7XkFCO7f3pG6hhTOUwdyFQg/20
	HZ+2Au9vY39W8RDRKmn2pJT2BpEGNB7tV2r92uPZiFb7scHqC0okqj904P1a5u9+f4D8KaPnJSk
	=
X-Google-Smtp-Source: AGHT+IEJZL8vn/EMuwZRGM3/4zNNuLIusBD3O5kk1RqbEsr4YKFsolMj3+CB5jypecUGTBMBxffYIw==
X-Received: by 2002:a05:6512:20c7:b0:512:f69f:fc45 with SMTP id u7-20020a05651220c700b00512f69ffc45mr2912115lfr.27.1708965918966;
        Mon, 26 Feb 2024 08:45:18 -0800 (PST)
Message-ID: <a4db4772-f6d5-42d9-8d5f-be451b5639ff@suse.com>
Date: Mon, 26 Feb 2024 17:45:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH v3 10/12] VMX: convert vmx_vmentry_control
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Kevin Tian <kevin.tian@intel.com>, Jun Nakajima <jun.nakajima@intel.com>
References: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <293e5aef-8843-461c-bc96-709a605b2680@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

... to a field in the capability/controls struct.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v2: New.

--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -162,7 +162,6 @@ static int cf_check parse_ept_param_runt
 
 /* Dynamic (run-time adjusted) execution control flags. */
 struct vmx_caps __ro_after_init vmx_caps;
-u32 vmx_vmentry_control __read_mostly;
 u64 vmx_ept_vpid_cap __read_mostly;
 static uint64_t __read_mostly vmx_vmfunc;
 
@@ -259,7 +258,6 @@ static int vmx_init_vmcs_config(bool bsp
     struct vmx_caps caps = {};
     u64 _vmx_ept_vpid_cap = 0;
     u64 _vmx_misc_cap = 0;
-    u32 _vmx_vmentry_control;
     u64 _vmx_vmfunc = 0;
     bool mismatch = false;
 
@@ -480,7 +478,7 @@ static int vmx_init_vmcs_config(bool bsp
     min = 0;
     opt = (VM_ENTRY_LOAD_GUEST_PAT | VM_ENTRY_LOAD_GUEST_EFER |
            VM_ENTRY_LOAD_BNDCFGS);
-    _vmx_vmentry_control = adjust_vmx_controls(
+    caps.vmentry_control = adjust_vmx_controls(
         "VMEntry Control", min, opt, MSR_IA32_VMX_ENTRY_CTLS, &mismatch);
 
     if ( mismatch )
@@ -491,7 +489,6 @@ static int vmx_init_vmcs_config(bool bsp
         /* First time through. */
         vmx_caps = caps;
         vmx_ept_vpid_cap           = _vmx_ept_vpid_cap;
-        vmx_vmentry_control        = _vmx_vmentry_control;
         vmx_caps.basic_msr = ((uint64_t)vmx_basic_msr_high << 32) |
                              vmx_basic_msr_low;
         vmx_vmfunc                 = _vmx_vmfunc;
@@ -531,7 +528,7 @@ static int vmx_init_vmcs_config(bool bsp
             vmx_caps.vmexit_control, caps.vmexit_control);
         mismatch |= cap_check(
             "VMEntry Control",
-            vmx_vmentry_control, _vmx_vmentry_control);
+            vmx_caps.vmentry_control, caps.vmentry_control);
         mismatch |= cap_check(
             "EPT and VPID Capability",
             vmx_ept_vpid_cap, _vmx_ept_vpid_cap);
@@ -1091,7 +1088,7 @@ static int construct_vmcs(struct vcpu *v
 {
     struct domain *d = v->domain;
     uint32_t vmexit_ctl = vmx_caps.vmexit_control;
-    u32 vmentry_ctl = vmx_vmentry_control;
+    u32 vmentry_ctl = vmx_caps.vmentry_control;
     int rc = 0;
 
     vmx_vmcs_enter(v);
@@ -2202,7 +2199,6 @@ int __init vmx_vmcs_init(void)
          * Make sure all dependent features are off as well.
          */
         memset(&vmx_caps, 0, sizeof(vmx_caps));
-        vmx_vmentry_control        = 0;
         vmx_ept_vpid_cap           = 0;
         vmx_vmfunc                 = 0;
     }
--- a/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
+++ b/xen/arch/x86/include/asm/hvm/vmx/vmcs.h
@@ -235,7 +235,6 @@ void vmx_vmcs_reload(struct vcpu *v);
 #define VM_ENTRY_LOAD_GUEST_PAT         0x00004000
 #define VM_ENTRY_LOAD_GUEST_EFER        0x00008000
 #define VM_ENTRY_LOAD_BNDCFGS           0x00010000
-extern u32 vmx_vmentry_control;
 
 #define SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES 0x00000001U
 #define SECONDARY_EXEC_ENABLE_EPT               0x00000002U
@@ -297,6 +296,7 @@ struct vmx_caps {
     uint32_t secondary_exec_control;
     uint64_t tertiary_exec_control;
     uint32_t vmexit_control;
+    uint32_t vmentry_control;
 };
 extern struct vmx_caps vmx_caps;
 
@@ -325,9 +325,9 @@ extern struct vmx_caps vmx_caps;
 #define cpu_has_monitor_trap_flag \
     (vmx_caps.cpu_based_exec_control & CPU_BASED_MONITOR_TRAP_FLAG)
 #define cpu_has_vmx_pat \
-    (vmx_vmentry_control & VM_ENTRY_LOAD_GUEST_PAT)
+    (vmx_caps.vmentry_control & VM_ENTRY_LOAD_GUEST_PAT)
 #define cpu_has_vmx_efer \
-    (vmx_vmentry_control & VM_ENTRY_LOAD_GUEST_EFER)
+    (vmx_caps.vmentry_control & VM_ENTRY_LOAD_GUEST_EFER)
 #define cpu_has_vmx_unrestricted_guest \
     (vmx_caps.secondary_exec_control & SECONDARY_EXEC_UNRESTRICTED_GUEST)
 #define vmx_unrestricted_guest(v)               \
@@ -355,7 +355,7 @@ extern struct vmx_caps vmx_caps;
     (vmx_caps.secondary_exec_control & SECONDARY_EXEC_ENABLE_PML)
 #define cpu_has_vmx_mpx \
     ((vmx_caps.vmexit_control & VM_EXIT_CLEAR_BNDCFGS) && \
-     (vmx_vmentry_control & VM_ENTRY_LOAD_BNDCFGS))
+     (vmx_caps.vmentry_control & VM_ENTRY_LOAD_BNDCFGS))
 #define cpu_has_vmx_xsaves \
     (vmx_caps.secondary_exec_control & SECONDARY_EXEC_XSAVES)
 #define cpu_has_vmx_tsc_scaling \



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:53:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:53:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685677.1066790 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reeDt-0007PC-FK; Mon, 26 Feb 2024 16:53:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685677.1066790; Mon, 26 Feb 2024 16:53:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reeDt-0007P5-CW; Mon, 26 Feb 2024 16:53:01 +0000
Received: by outflank-mailman (input) for mailman id 685677;
 Mon, 26 Feb 2024 16:53:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=hPQ6=KD=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1reeDr-0007Oz-Uv
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:52:59 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7ee6b210-d4c7-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:52:58 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a3e552eff09so359939066b.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:52:58 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 r19-20020a170906365300b00a3d68aad90dsm2591068ejb.97.2024.02.26.08.52.57
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:52:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7ee6b210-d4c7-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1708966378; x=1709571178; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=zyFC6O9pVQ06Jhy/ge+QJAOQeVFoqSyVhea9OGY/icE=;
        b=WwRF9//g3m4qwp2Ue8H+8LbZcOLOtNPt8OOtUFa9PPUPqlQxmksKmhawA8AFnllwoF
         55fPE/kMe54J8Rja41CGIVoRwOl4kSA10AIi07mf+Ywe1vKBqFNFrVdVe/tArXO8H7kj
         oMO4NZLQ3USA1Kw37ZIEzWVE/6upambwYzcQHqzt3utn/epxv/K1grSFvJNfPeMJwU+z
         2xCSrUNQ7doptYvapcjdtY/koBjNvKg/N8H86wcWVgeWVKO3EIXgfk5j0XQDem5hwgJr
         akb1z/gxz0Lce/3+AOV/h3PmVM/6thX6UELTz1y79ElymcEg2aQ2/lYlI+Iw9Ow5OqyY
         AFUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708966378; x=1709571178;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zyFC6O9pVQ06Jhy/ge+QJAOQeVFoqSyVhea9OGY/icE=;
        b=YdE3GKpWjGbIzw+agsVyUm1T1HsVhUdLbJ/pFHFQ1R6FRq7LAPNXsf3V9s2Xx5l9Ft
         BCH6YRog66cGKx+ddnIHgcIbbNdnZO2YpWeqe8nrfOc2kYv9Dd5n4p4f5r2X3+06yg04
         +k2wTKJBEwjyYwKe+FHWgIt6qtA4vP2qyCegESE/TLEPEH+Wa4tXxEOqbXetA2vuEUnP
         zeGv0X6pwMYTydVJllhFvJ330EKXmBGc7NX7AWhLuli/nIQCHjloovSg3COmWNCiu09O
         N9eVUSILW8WLuM2QjMnOWfbSA4Cvtct+0EtgKQIdzBPlLJqKHSH5wFwjHFm18jOhoxZL
         QaEQ==
X-Forwarded-Encrypted: i=1; AJvYcCX/s+xBGdYarYoJ38nJ19MGErPSngNhcMuRA2BkvUtFTODytPBb/W2wit+vXBqjefCNcn/jIatezr4k8BpmNREAyN20T+krLMq5yMDDN14=
X-Gm-Message-State: AOJu0YyOdLtkd9purW5vJg7Yl9cRoHomtyaIIwPI5bA1zb2MwzdAtQW8
	+tGRPlX4Vr9sFhlQGvS1471g0vT3IuO0lcJfnknkPUbZfpFlZzsS+F/E1y+XuQ==
X-Google-Smtp-Source: AGHT+IG3aLTKib0FsS/a33AaUyViTnOvhl3GigJjoR9+U0zLbfY4g9JDPQWZD6/4wC/z4P1QxRWBQA==
X-Received: by 2002:a17:906:2794:b0:a43:72f6:723 with SMTP id j20-20020a170906279400b00a4372f60723mr1410817ejc.17.1708966378327;
        Mon, 26 Feb 2024 08:52:58 -0800 (PST)
Message-ID: <d026feda-1997-43fc-94e6-cb9da1ef807c@suse.com>
Date: Mon, 26 Feb 2024 17:52:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] docs/sphinx: Start an FAQ, and add Kconfig/CET details
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240226162527.1247421-1-andrew.cooper3@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240226162527.1247421-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 17:25, Andrew Cooper wrote:
> This is long overdue, and we need to start somewhere.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
perhaps (nit) with ...

> --- /dev/null
> +++ b/docs/faq.rst
> @@ -0,0 +1,71 @@
> +.. SPDX-License-Identifier: CC-BY-4.0
> +
> +Frequently Asked Questions
> +==========================
> +
> +How do I...
> +-----------
> +
> +... check whether a Kconfig option is active?
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +  Kconfig is a build time configuration system, combining inherent knowledge,
> +  the capabilities of the toolchain, and explicit user choice to form a
> +  configuration of a build of Xen.
> +
> +  A file, by default ``.config``, is produced by the build identifying the
> +  configuration used.  Kconfig symbols all start with ``CONFIG_``, and come in
> +  a variety of types including strings, integers and booleans.  Booleans are
> +  the most common, and when active are expressed with ``...=y``.  e.g.::
> +
> +    xen.git/xen$ grep CONFIG_FOO .config
> +    CONFIG_FOO_BOOLEAN=y
> +    CONFIG_FOO_STRING="lorem ipsum"
> +    CONFIG_FOO_INTEGER=42
> +
> +  Symbols which are either absent, or expressed as ``... is not set`` are
> +  disabled.  e.g.::
> +
> +    xen.git/xen$ grep CONFIG_BAR .config
> +    # CONFIG_BAR is not set
> +
> +  Builds of Xen configured with ``CONFIG_HYPFS_CONFIG=y`` embed their own
> +  ``.config`` at build time, and can provide it to the :term:`control domain`
> +  upon requested.  e.g.::
> +
> +    [root@host ~]# xenhypfs cat /buildinfo/config | grep -e FOO -e BAR
> +    CONFIG_FOO=y
> +    # CONFIG_BAR is not set
> +
> +
> +... tell if CET is active?
> +^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +  Control-flow Enforcement Technology support was added to Xen 4.14.  It is
> +  build time conditional, dependent on both having a new-enough toolchain and
> +  an explicit Kconfig option, and also requires capable hardware.  See
> +  :term:`CET`.
> +
> +  For CET-SS, Shadow Stacks, the minimum toolchain requirements are ``binutils
> +  >= 2.29`` or ``LLVM >= 6``.  No specific compiler support is required.
> +  Check for ``CONFIG_XEN_SHSTK`` being active.
> +
> +  For CET-IBT, Indirect Branch Tracking, the minimum toolchain requirements
> +  are ``GCC >= 9`` and ``binutils >= 2.29``.  Xen relies on a compiler feature
> +  which is specific to GCC at the time of writing.  Check for
> +  ``CONFIG_XEN_IBT`` being active.
> +
> +  If a capable Xen with booted on capable hardware, and CET is not disabled by

... s/with/is/ (or "was").

Jan


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 16:53:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 16:53:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685680.1066800 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reeEb-0007y8-SU; Mon, 26 Feb 2024 16:53:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685680.1066800; Mon, 26 Feb 2024 16:53:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reeEb-0007y1-PP; Mon, 26 Feb 2024 16:53:45 +0000
Received: by outflank-mailman (input) for mailman id 685680;
 Mon, 26 Feb 2024 16:53:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FdVt=KD=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1reeEa-0007Oz-Nk
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 16:53:44 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 99d7b438-d4c7-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 17:53:44 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-513031ce058so69840e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 08:53:44 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 eb20-20020ad44e54000000b0068f5781760bsm3134601qvb.51.2024.02.26.08.53.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 08:53:43 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 99d7b438-d4c7-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1708966423; x=1709571223; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=wg16Veg6CXyVGxxnjZbrEmDMVb7yNyPwrgdL/6tppWU=;
        b=Wn3areR2V8GFPboAugCbdF+ACiZ3Ja6poydwLIYgeEwrBcpoGmQmGPvUlmLJE3WhOq
         dBPrCOgLTInOlM7St/eSTXO1AQ6Y7suom1RTJyOjEnut/5WDOEwu3gCAvheQt+yRtJcf
         ruapDL/RQAJSKlmYulNLLuognpWH1iWtFSAlw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708966423; x=1709571223;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=wg16Veg6CXyVGxxnjZbrEmDMVb7yNyPwrgdL/6tppWU=;
        b=PzzJvoNN1D4ZJh+UKny3JzAORcd1Hh3W6AyUhhnxyc7PcXFOHjjmk+BY2IV88K4AUk
         zI737b463BYaQZpbjQXFnoIXcYLjm3RZSFurlYLsX+6wQ81H+WVgSTeHO47GfwtznZfW
         8Qoajt6blBUdQbs5Erkio2WSfD1ImKh02p6GZz+aVSSZZwzJB1MFOwdQLSf9IHtM+gpS
         n0YYNuZZQMb6s0Sys2YNSFglMdmoGHGA0hTSd7gid0PXNsd4Oobb1o8Ehgh8ZGTH/K3E
         MW48EYN0lMouK1VLC1tNX2I1e99BZIXYmz/O/NXAfapIVVmOHyTKHdzuk+18GX/aXpJN
         gl4g==
X-Forwarded-Encrypted: i=1; AJvYcCUIG7HAO42K32QlqhUtAzwiQjIqVxU9iHuN/YEH7KvCtoVUEnvJcJb9os3orCEPmzQWc0PjdsxDzm/1G6IC750xp9AgoFxVarmABdPC2tA=
X-Gm-Message-State: AOJu0YyoW3r4WcmpjjzbESwymVnuRlh7YZ93xne+oQfLUqPl9UFBISEv
	6/QDy8yVPkJF51rpoRtadueqaUrrMoz85EXeLL0DksnxbXcpWGUn/8UHZ2NfwDE=
X-Google-Smtp-Source: AGHT+IEazuURzTOTntMTOvDTv1OdHch4KV+G3BWyg56BbzmPMeCXJIwimJWQUNWi5agHiRL6KERADQ==
X-Received: by 2002:a05:6512:3f04:b0:512:d5c7:60d9 with SMTP id y4-20020a0565123f0400b00512d5c760d9mr6305207lfa.3.1708966423547;
        Mon, 26 Feb 2024 08:53:43 -0800 (PST)
Message-ID: <05b0f7a2-14b1-4cf9-bc70-fa106723e632@citrix.com>
Date: Mon, 26 Feb 2024 16:53:41 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] docs/sphinx: Start an FAQ, and add Kconfig/CET details
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <George.Dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240226162527.1247421-1-andrew.cooper3@citrix.com>
 <d026feda-1997-43fc-94e6-cb9da1ef807c@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <d026feda-1997-43fc-94e6-cb9da1ef807c@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26/02/2024 4:52 pm, Jan Beulich wrote:
> On 26.02.2024 17:25, Andrew Cooper wrote:
>> This is long overdue, and we need to start somewhere.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> perhaps (nit) with ...
>
>> --- /dev/null
>> +++ b/docs/faq.rst
>> @@ -0,0 +1,71 @@
>> +.. SPDX-License-Identifier: CC-BY-4.0
>> +
>> +Frequently Asked Questions
>> +==========================
>> +
>> +How do I...
>> +-----------
>> +
>> +... check whether a Kconfig option is active?
>> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> +
>> +  Kconfig is a build time configuration system, combining inherent knowledge,
>> +  the capabilities of the toolchain, and explicit user choice to form a
>> +  configuration of a build of Xen.
>> +
>> +  A file, by default ``.config``, is produced by the build identifying the
>> +  configuration used.  Kconfig symbols all start with ``CONFIG_``, and come in
>> +  a variety of types including strings, integers and booleans.  Booleans are
>> +  the most common, and when active are expressed with ``...=y``.  e.g.::
>> +
>> +    xen.git/xen$ grep CONFIG_FOO .config
>> +    CONFIG_FOO_BOOLEAN=y
>> +    CONFIG_FOO_STRING="lorem ipsum"
>> +    CONFIG_FOO_INTEGER=42
>> +
>> +  Symbols which are either absent, or expressed as ``... is not set`` are
>> +  disabled.  e.g.::
>> +
>> +    xen.git/xen$ grep CONFIG_BAR .config
>> +    # CONFIG_BAR is not set
>> +
>> +  Builds of Xen configured with ``CONFIG_HYPFS_CONFIG=y`` embed their own
>> +  ``.config`` at build time, and can provide it to the :term:`control domain`
>> +  upon requested.  e.g.::
>> +
>> +    [root@host ~]# xenhypfs cat /buildinfo/config | grep -e FOO -e BAR
>> +    CONFIG_FOO=y
>> +    # CONFIG_BAR is not set
>> +
>> +
>> +... tell if CET is active?
>> +^^^^^^^^^^^^^^^^^^^^^^^^^^
>> +
>> +  Control-flow Enforcement Technology support was added to Xen 4.14.  It is
>> +  build time conditional, dependent on both having a new-enough toolchain and
>> +  an explicit Kconfig option, and also requires capable hardware.  See
>> +  :term:`CET`.
>> +
>> +  For CET-SS, Shadow Stacks, the minimum toolchain requirements are ``binutils
>> +  >= 2.29`` or ``LLVM >= 6``.  No specific compiler support is required.
>> +  Check for ``CONFIG_XEN_SHSTK`` being active.
>> +
>> +  For CET-IBT, Indirect Branch Tracking, the minimum toolchain requirements
>> +  are ``GCC >= 9`` and ``binutils >= 2.29``.  Xen relies on a compiler feature
>> +  which is specific to GCC at the time of writing.  Check for
>> +  ``CONFIG_XEN_IBT`` being active.
>> +
>> +  If a capable Xen with booted on capable hardware, and CET is not disabled by
> ... s/with/is/ (or "was").

Oops yes.  Will fix.

~Andrew


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685690.1066810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewd-0007pp-8O; Mon, 26 Feb 2024 17:39:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685690.1066810; Mon, 26 Feb 2024 17:39:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewd-0007pe-5M; Mon, 26 Feb 2024 17:39:15 +0000
Received: by outflank-mailman (input) for mailman id 685690;
 Mon, 26 Feb 2024 17:39:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewb-0007o5-0J
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:13 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f3200a8c-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:10 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d27fef509eso26737691fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:10 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f3200a8c-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969149; x=1709573949; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=Em/qKiQQdW+4x5fl3Iv3jnukZezA0/bPJ5T9/Rqd5bY=;
        b=Ofjb7qwl3QRReWAg5ltfbo2+ZIosCYPNR/jh9YTVbvePbyM4s3RmuezXWhXuR0yNhA
         ATTx7uf/9JRU0ky4JSNw37gFghn2CLb8IUYafl8ATw8XBtIMUrrLs3nqICkco3CWmw31
         rSITbGUXRUOUdnst8PwDFGB4mG5ZbMkJV0X6tJnp4cfQXMvlHI7TmXojT1b/R7S+rEzY
         2LcnllWoDMdHeaeoNdxT/jljRn4DnZ8zuZcuFsSs3vSClh0ErxJJI4rSw0II4iBotElI
         uyCy0fMI8IqOzHc9l/CXv2TlbR44zxUJWgwYgH9clegckj3vecQwYgV+tNeqZITLvlXY
         SA7g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969149; x=1709573949;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=Em/qKiQQdW+4x5fl3Iv3jnukZezA0/bPJ5T9/Rqd5bY=;
        b=aPI+7MAz++9neMlOl8PXZlA9XxepHIw64ptbUEP2kmiTng+Rt1hXwcwyHC4yoN3thB
         r8pMdt/XtjIQm0FLdYD2oyt+OYpGVDbfxWZORnIZOHhtxDkK19RfP7TJh8A3mG8Paowv
         9ExwcCyTu7NMWBo7WZAmN92/bO4xoKSfvKalLi2jQouppaQWoDrycMWKUQ/tUQw+9m7X
         BP1SD8WU99F3pKtOPq8LJxdM0PWwtgJL+ItHzhsWuKOXS4El3bb8SLxY5v6f+s6K5B+2
         JZzN5fmMNrEUy1KLeBd+BFjcbyTfSHlQWewc3S/5wInplG3knAnTW8gMyO3QZS8Vmbsq
         BUNA==
X-Gm-Message-State: AOJu0YyOY0EXDN8KtKA8C16dcix3dxxt7C9dnjcoWs0SpSy2batRfmye
	Q8ORiFXxgpF3yyb/H89nZ7jzHt82YEhGQj8h4F1SvLe/DAaH93JAj78j+v5c
X-Google-Smtp-Source: AGHT+IEEzmg9fpq9zpc1oIs40IrTNM1Mv28s3DbGJ15V4HBez7INtD3UCq/lbbNlJ+HNOFSly/xYGg==
X-Received: by 2002:a2e:b0c5:0:b0:2d2:9345:78d8 with SMTP id g5-20020a2eb0c5000000b002d2934578d8mr339357ljl.31.1708969149238;
        Mon, 26 Feb 2024 09:39:09 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH v5 00/23] [PATCH v4 00/30]  Enable build of full Xen for RISC-V
Date: Mon, 26 Feb 2024 18:38:42 +0100
Message-ID: <cover.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch series performs all of the additions necessary to drop the
build overrides for RISCV and enable the full Xen build. Except in cases
where compatibile implementations already exist (e.g. atomic.h and
bitops.h), the newly added definitions are simple.

The patch series is based on the following patch series:
-	[PATCH v6 0/9] Introduce generic headers   [1]
- [PATCH] move __read_mostly to xen/cache.h  [2]
- [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE() [3]
- [PATCH] xen/lib: introduce generic find next bit operations [4]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.

[1] https://lore.kernel.org/xen-devel/cover.1703072575.git.oleksii.kurochko@gmail.com/
[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/
[3] https://lore.kernel.org/xen-devel/42fc6ae8d3eb802429d29c774502ff232340dc84.1706259490.git.federico.serafini@bugseng.com/
[4] https://lore.kernel.org/xen-devel/52730e6314210ba4164a9934a720c4fda201447b.1706266854.git.oleksii.kurochko@gmail.com/

---
Changes in V5:
 - Update the cover letter as one of the dependencies were merged to staging.
 - Was introduced asm-generic for atomic ops and separate patches for asm-generic bit ops
 - Moved fence.h to separate patch to deal with some patches dependecies on fence.h
 - Patches were dropped as they were merged to staging:
   * [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
   * [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
   * [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
   * [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
   * [PATCH v4 08/30] xen/riscv: introduce setup.h
   * [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
   * [PATCH v4 11/30] xen/riscv: introduce smp.h
   * [PATCH v4 15/30] xen/riscv: introduce irq.h
   * [PATCH v4 16/30] xen/riscv: introduce p2m.h
   * [PATCH v4 17/30] xen/riscv: introduce regs.h
   * [PATCH v4 18/30] xen/riscv: introduce time.h
   * [PATCH v4 19/30] xen/riscv: introduce event.h
   * [PATCH v4 22/30] xen/riscv: define an address of frame table
 - Other changes are specific to specific patches. please look at specific patch
---
Changes in V4:
 - Update the cover letter message: new patch series dependencies.
 - Some patches were merged to staging, so they were dropped in this patch series:
     [PATCH v3 09/34] xen/riscv: introduce system.h
     [PATCH v3 18/34] xen/riscv: introduce domain.h
     [PATCH v3 19/34] xen/riscv: introduce guest_access.h
 - Was sent out of this patch series:
     [PATCH v3 16/34] xen/lib: introduce generic find next bit operations
 - [PATCH v3 17/34] xen/riscv: add compilation of generic find-next-bit.c was
   droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
 - All other changes are specific to a specific patch.
---
Changes in V3:
 - Update the cover letter message
 - The following patches were dropped as they were merged to staging:
    [PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h
    [PATCH v2 04/39] xen/riscv: add public arch-riscv.h
    [PATCH v2 05/39] xen/riscv: introduce spinlock.h
    [PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S
    [PATCH v2 34/39] xen: add RISCV support for pmu.h
    [PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
 - Instead of the following patches were introduced new:
    [PATCH v2 10/39] xen/riscv: introduce asm/iommu.h
    [PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
 - remove "asm/"  for commit messages which start with "xen/riscv:"
 - code style updates.
 - add emulation of {cmp}xchg_* for 1 and 2 bytes types.
 - code style fixes.
 - add SPDX and footer for the newly added headers.
 - introduce generic find-next-bit.c.
 - some other mionor changes. ( details please find in a patch )
---
Changes in V2:
  - Drop the following patches as they are the part of [2]:
      [PATCH v1 06/57] xen/riscv: introduce paging.h
      [PATCH v1 08/57] xen/riscv: introduce asm/device.h
      [PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h
      [PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h
      [PATCH v1 13/57] xen/riscv: introduce asm/iocap.h
      [PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h
      [PATCH v1 18/57] xen/riscv: introduce asm/random.h
      [PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h
      [PATCH v1 24/57] xen/riscv: introduce asm/percpu.h
      [PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h
      [PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h
      [PATCH v1 38/57] xen/riscv: introduce asm/monitor.h
      [PATCH v1 39/57] xen/riscv: introduce asm/numa.h
      [PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
  - xen/lib.h in most of the cases were changed to xen/bug.h as
    mostly functionilty of bug.h is used.
  - align arch-riscv.h with Arm's version of it.
  - change the Author of commit with introduction of asm/atomic.h.
  - update some definition from spinlock.h.
  - code style changes.
---

Oleksii Kurochko (23):
  xen/riscv: disable unnecessary configs
  xen/riscv: use some asm-generic headers
  xen/riscv: introduce nospec.h
  xen/asm-generic: introduce generic fls() and flsl() functions
  xen/asm-generic: introduce generic find first set bit functions
  xen/asm-generic: introduce generic ffz()
  xen/asm-generic: introduce generic hweight64()
  xen/asm-generic: introduce generic non-atomic test_*bit()
  xen/riscv: introduce bitops.h
  xen/riscv: introduces acrquire, release and full barriers
  xen/riscv: introduce cmpxchg.h
  xen/riscv: introduce io.h
  xen/riscv: introduce atomic.h
  xen/riscv: introduce monitor.h
  xen/riscv: add definition of __read_mostly
  xen/riscv: add required things to current.h
  xen/riscv: add minimal stuff to page.h to build full Xen
  xen/riscv: add minimal stuff to processor.h to build full Xen
  xen/riscv: add minimal stuff to mm.h to build full Xen
  xen/riscv: introduce vm_event_*() functions
  xen/rirscv: add minimal amount of stubs to build full Xen
  xen/riscv: enable full Xen build
  xen/README: add compiler and binutils versions for RISC-V64

 README                                        |   9 +
 automation/gitlab-ci/build.yaml               |  24 +
 docs/misc/riscv/booting.txt                   |   8 +
 xen/arch/riscv/Makefile                       |  18 +-
 xen/arch/riscv/arch.mk                        |  12 +-
 xen/arch/riscv/configs/tiny64_defconfig       |  17 +
 xen/arch/riscv/early_printk.c                 | 168 -------
 xen/arch/riscv/include/asm/Makefile           |  12 +
 xen/arch/riscv/include/asm/atomic.h           | 296 +++++++++++++
 xen/arch/riscv/include/asm/bitops.h           | 152 +++++++
 xen/arch/riscv/include/asm/cache.h            |   2 +
 xen/arch/riscv/include/asm/cmpxchg.h          | 241 ++++++++++
 xen/arch/riscv/include/asm/config.h           |   2 +
 xen/arch/riscv/include/asm/current.h          |  19 +
 xen/arch/riscv/include/asm/fence.h            |   9 +
 xen/arch/riscv/include/asm/io.h               | 157 +++++++
 xen/arch/riscv/include/asm/mm.h               | 246 +++++++++++
 xen/arch/riscv/include/asm/monitor.h          |  26 ++
 xen/arch/riscv/include/asm/nospec.h           |  25 ++
 xen/arch/riscv/include/asm/page.h             |  19 +
 xen/arch/riscv/include/asm/processor.h        |  23 +
 xen/arch/riscv/mm.c                           |  52 ++-
 xen/arch/riscv/setup.c                        |  10 +-
 xen/arch/riscv/stubs.c                        | 415 ++++++++++++++++++
 xen/arch/riscv/traps.c                        |  25 ++
 xen/arch/riscv/vm_event.c                     |  19 +
 xen/include/asm-generic/atomic-ops.h          |  92 ++++
 xen/include/asm-generic/bitops/__ffs.h        |  47 ++
 xen/include/asm-generic/bitops/bitops-bits.h  |  21 +
 xen/include/asm-generic/bitops/ffs.h          |   9 +
 xen/include/asm-generic/bitops/ffsl.h         |  16 +
 xen/include/asm-generic/bitops/ffz.h          |  18 +
 .../asm-generic/bitops/find-first-set-bit.h   |  17 +
 xen/include/asm-generic/bitops/fls.h          |  18 +
 xen/include/asm-generic/bitops/flsl.h         |  10 +
 .../asm-generic/bitops/generic-non-atomic.h   |  89 ++++
 xen/include/asm-generic/bitops/hweight.h      |  13 +
 xen/include/asm-generic/bitops/test-bit.h     |  18 +
 38 files changed, 2198 insertions(+), 176 deletions(-)
 create mode 100644 docs/misc/riscv/booting.txt
 create mode 100644 xen/arch/riscv/include/asm/Makefile
 create mode 100644 xen/arch/riscv/include/asm/atomic.h
 create mode 100644 xen/arch/riscv/include/asm/bitops.h
 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h
 create mode 100644 xen/arch/riscv/include/asm/fence.h
 create mode 100644 xen/arch/riscv/include/asm/io.h
 create mode 100644 xen/arch/riscv/include/asm/monitor.h
 create mode 100644 xen/arch/riscv/include/asm/nospec.h
 create mode 100644 xen/arch/riscv/stubs.c
 create mode 100644 xen/arch/riscv/vm_event.c
 create mode 100644 xen/include/asm-generic/atomic-ops.h
 create mode 100644 xen/include/asm-generic/bitops/__ffs.h
 create mode 100644 xen/include/asm-generic/bitops/bitops-bits.h
 create mode 100644 xen/include/asm-generic/bitops/ffs.h
 create mode 100644 xen/include/asm-generic/bitops/ffsl.h
 create mode 100644 xen/include/asm-generic/bitops/ffz.h
 create mode 100644 xen/include/asm-generic/bitops/find-first-set-bit.h
 create mode 100644 xen/include/asm-generic/bitops/fls.h
 create mode 100644 xen/include/asm-generic/bitops/flsl.h
 create mode 100644 xen/include/asm-generic/bitops/generic-non-atomic.h
 create mode 100644 xen/include/asm-generic/bitops/hweight.h
 create mode 100644 xen/include/asm-generic/bitops/test-bit.h

-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685693.1066835 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewf-0008ND-5x; Mon, 26 Feb 2024 17:39:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685693.1066835; Mon, 26 Feb 2024 17:39:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewf-0008ME-0N; Mon, 26 Feb 2024 17:39:17 +0000
Received: by outflank-mailman (input) for mailman id 685693;
 Mon, 26 Feb 2024 17:39:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewd-0007pd-Pl
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:15 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f5bf0e89-d4cd-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 18:39:15 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d26227d508so37271271fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:15 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.12
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f5bf0e89-d4cd-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969154; x=1709573954; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=T7m6VMdKIKFz3mQdT8KKo2X5qy108374IX/REaCKQHg=;
        b=fce2LjTjTxSIJpJgG/+HYOOINmrc3gHzk5DijnZqchNRzLmrydFdf3lsn1UpaRXRCW
         nb4gykaDRtpR4sVsaw3FQhVNJGZjGFUOPVBb/P9U6HZvdD6m6lkWntUCHF3djSA/u/qk
         EYHkTgGiB1/qRXhGvW2xhekGE7rAwlbbPUZJ77IwWyIlludPbTEavGJiv+afCeJJKMos
         7niZ4dSETUUYASWYAfxybM/n/WsLo4QhkoO2t5nsMb7c3G10ykbWd8HWRdQSnXY8SKMC
         J7hjx2dS9xZk6DLuA5XwFvpp/xpPQ87kqh0UXKZSM2a2bCnSUjCaQrlA2vlrUNCtWtyt
         npHg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969154; x=1709573954;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=T7m6VMdKIKFz3mQdT8KKo2X5qy108374IX/REaCKQHg=;
        b=omh1gZYszTuH5eSVnKZMyMA231NYvDaJTXjnj9F13ooVV29tk7u7r3e1vG4khdxS7a
         /0lNw1GLLMwzdXIA7DNVfiTciV9p7JDdkB6D0nWpWdIRSWJKh57IL6oMchItjbKug7T4
         cdAGsoCobiDx+bQrgsXejKBV7Op14AcT/BOehNNda8UeUckVG3yR6zRouemCBWYcsqD+
         diqtoGLQcydUdbtuimj+IDqvgUFHLyfq1eDVfh4mPT60aC4pMASXsRtMbRbYrKuwjYvX
         BrITPMfR0HGm8AsRsPIIwR+wcxsWgHok4yfcUPRv5CmH87K1xnz+LQjrPndwjjWRPFig
         m1rQ==
X-Gm-Message-State: AOJu0Yx2dYJBEmgbOJKiYYZIaHfVYvwd7GEEAvlFdOazne42ejSkYMDs
	bUtx6I5VYUzuHOuDakaz+sSxVD4Uf3EtviKHJ7QcFhbgmPIurOqtDiQGULla
X-Google-Smtp-Source: AGHT+IH0gLIB5rqhmUwCgtqyn/EnJIYIP/ECs5zIRMjGJTzRJaySLpV4A2ck6fqAHspGdtqGSCq/jw==
X-Received: by 2002:a05:651c:105c:b0:2d2:3202:68c5 with SMTP id x28-20020a05651c105c00b002d2320268c5mr4276561ljm.32.1708969153933;
        Mon, 26 Feb 2024 09:39:13 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 03/23] xen/riscv: introduce nospec.h
Date: Mon, 26 Feb 2024 18:38:45 +0100
Message-ID: <16370492de5344ae8e1aab688261600491868f4e.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

>From the unpriviliged doc:
  No standard hints are presently defined.
  We anticipate standard hints to eventually include memory-system spatial
  and temporal locality hints, branch prediction hints, thread-scheduling
  hints, security tags, and instrumentation flags for simulation/emulation.

Also, there are no speculation execution barriers.

Therefore, functions evaluate_nospec() and block_speculation() should
remain empty until a specific platform has an extension to deal with
speculation execution.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Changes in V5:
   - new patch
---
 xen/arch/riscv/include/asm/nospec.h | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/nospec.h

diff --git a/xen/arch/riscv/include/asm/nospec.h b/xen/arch/riscv/include/asm/nospec.h
new file mode 100644
index 0000000000..4fb404a0a2
--- /dev/null
+++ b/xen/arch/riscv/include/asm/nospec.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2024 Vates */
+
+#ifndef _ASM_GENERIC_NOSPEC_H
+#define _ASM_GENERIC_NOSPEC_H
+
+static inline bool evaluate_nospec(bool condition)
+{
+    return condition;
+}
+
+static inline void block_speculation(void)
+{
+}
+
+#endif /* _ASM_GENERIC_NOSPEC_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685691.1066815 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewd-0007rc-Gt; Mon, 26 Feb 2024 17:39:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685691.1066815; Mon, 26 Feb 2024 17:39:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewd-0007rE-CQ; Mon, 26 Feb 2024 17:39:15 +0000
Received: by outflank-mailman (input) for mailman id 685691;
 Mon, 26 Feb 2024 17:39:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewb-0007o5-T6
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:13 +0000
Received: from mail-lj1-x230.google.com (mail-lj1-x230.google.com
 [2a00:1450:4864:20::230])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f40977c8-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:12 +0100 (CET)
Received: by mail-lj1-x230.google.com with SMTP id
 38308e7fff4ca-2d243797703so42728691fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:12 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f40977c8-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969151; x=1709573951; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/Cc1ctgpIyGSAwIswK/MOgWw8+E3f2g/putJbdmyaeg=;
        b=LaGBSFP6hdOW2rRGwOwK6ctOAzXq/KN9Bo+zGdZfXdGAL/hkcZjfy9RNEGn7sEJxAP
         ISPyxur+TTiQyEz+M2JYBw4EC6S/W1Dl9+qbljBO8u/pU+nhd5tjkjNEpV22Q3Q3L8vb
         AUdWFtgweGtqZxxQws9H+AKGDuYKmsBFN6kcvjzLPIjsmfrUlUhafrfjq/QXeE5W0+cN
         oBDtBAlgqYjr+HAxU2GDsPHYWT+B0Ys/wctR19O/AUGyHXcsqJOuCR14cQD+Y9O0qN3a
         QBNE0BXIS3SF7j+FAfRyawB/ZirXzSDSEgh6NCDmH5g04g9Azxka6NnPtxgkiQ4tOlCT
         GEww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969151; x=1709573951;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=/Cc1ctgpIyGSAwIswK/MOgWw8+E3f2g/putJbdmyaeg=;
        b=Yr5rSHkAIvjTTTLe+jjhPyHOrOMbT2e24WOzA1fTbIweWzAqkQQET+GXO/BVPD95QJ
         5O8ko0FNxUG1cy/6bVJvdt5Hhi4IJEU1Bzlx0ZtoYTR48dRX2ni2QPMW2H4SqZmcinUK
         dic/jp1BVuj780lg52gD1UnmCSKCVU+cmucKaIfaxg6cx+BPktEQMkxK9toQ+yhTSKEM
         kSEDxNvqh9Xjzce/CdoiP6Sj3DtCq0ctDq5Sh9jQlcuOrHXwDbYwKQW96sPLKWLlYF6+
         9Ia2lUu74SXGOqz8G0C0ut33akmJjGMkaos3jJQopLjRBNeBpM3yFbzO/wkFoTORnR6i
         Jmsw==
X-Gm-Message-State: AOJu0YxwvUg+g3hVUks0RFwiYnUMPXc6LkZeNKuPUq4/D1LAf4NAJFAL
	tUY87wVcfN2eNV4x5Mi02qc7hkZcQh4E2CFc6GiPkw0A/EGnjwfDorku631+
X-Google-Smtp-Source: AGHT+IHN+nYWAqmitVsteyEec6jSq5D0nVknyjk2VMGqGfbfY3ovdpOGSr9TSVHbVWHSp3sKaiqhqA==
X-Received: by 2002:a2e:80d3:0:b0:2d2:8c9a:c4e5 with SMTP id r19-20020a2e80d3000000b002d28c9ac4e5mr2078923ljg.18.1708969150810;
        Mon, 26 Feb 2024 09:39:10 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v5 01/23] xen/riscv: disable unnecessary configs
Date: Mon, 26 Feb 2024 18:38:43 +0100
Message-ID: <6aa24c4943b71592c3735cdb1881c268370e7811.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch disables unnecessary configs for two cases:
1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
2. By using tiny64_defconfig for non-randconfig builds.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5:
 - Rebase and drop duplicated configs in EXTRA_FIXED_RANDCONFIG list
 - Update the commit message
---
Changes in V4:
 - Nothing changed. Only rebase
---
Changes in V3:
 - Remove EXTRA_FIXED_RANDCONFIG for non-randconfig jobs.
   For non-randconfig jobs, it is sufficient to disable configs by using the defconfig.
 - Remove double blank lines in build.yaml file before archlinux-current-gcc-riscv64-debug
---
Changes in V2:
 - update the commit message.
 - remove xen/arch/riscv/Kconfig changes.
---
 automation/gitlab-ci/build.yaml         | 24 ++++++++++++++++++++++++
 xen/arch/riscv/configs/tiny64_defconfig | 17 +++++++++++++++++
 2 files changed, 41 insertions(+)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index aac29ee13a..3b3d2c47dc 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -519,6 +519,30 @@ alpine-3.18-gcc-debug-arm64-boot-cpupools:
       CONFIG_EXPERT=y
       CONFIG_GRANT_TABLE=n
       CONFIG_MEM_ACCESS=n
+      CONFIG_SCHED_CREDIT=n
+      CONFIG_SCHED_CREDIT2=n
+      CONFIG_SCHED_RTDS=n
+      CONFIG_SCHED_NULL=n
+      CONFIG_SCHED_ARINC653=n
+      CONFIG_TRACEBUFFER=n
+      CONFIG_HYPFS=n
+      CONFIG_SPECULATIVE_HARDEN_ARRAY=n
+      CONFIG_ARGO=n
+      CONFIG_HYPFS_CONFIG=n
+      CONFIG_CORE_PARKING=n
+      CONFIG_DEBUG_TRACE=n
+      CONFIG_IOREQ_SERVER=n
+      CONFIG_CRASH_DEBUG=n
+      CONFIG_KEXEC=n
+      CONFIG_LIVEPATCH=n
+      CONFIG_NUMA=n
+      CONFIG_PERF_COUNTERS=n
+      CONFIG_HAS_PMAP=n
+      CONFIG_XENOPROF=n
+      CONFIG_COMPAT=n
+      CONFIG_UBSAN=n
+      CONFIG_NEEDS_LIBELF=n
+      CONFIG_XSM=n
 
 archlinux-current-gcc-riscv64:
   extends: .gcc-riscv64-cross-build
diff --git a/xen/arch/riscv/configs/tiny64_defconfig b/xen/arch/riscv/configs/tiny64_defconfig
index 09defe236b..35915255e6 100644
--- a/xen/arch/riscv/configs/tiny64_defconfig
+++ b/xen/arch/riscv/configs/tiny64_defconfig
@@ -7,6 +7,23 @@
 # CONFIG_GRANT_TABLE is not set
 # CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
 # CONFIG_MEM_ACCESS is not set
+# CONFIG_ARGO is not set
+# CONFIG_HYPFS_CONFIG is not set
+# CONFIG_CORE_PARKING is not set
+# CONFIG_DEBUG_TRACE is not set
+# CONFIG_IOREQ_SERVER is not set
+# CONFIG_CRASH_DEBUG is not setz
+# CONFIG_KEXEC is not set
+# CONFIG_LIVEPATCH is not set
+# CONFIG_NUMA is not set
+# CONFIG_PERF_COUNTERS is not set
+# CONFIG_HAS_PMAP is not set
+# CONFIG_TRACEBUFFER is not set
+# CONFIG_XENOPROF is not set
+# CONFIG_COMPAT is not set
+# CONFIG_COVERAGE is not set
+# CONFIG_UBSAN is not set
+# CONFIG_NEEDS_LIBELF is not set
 
 CONFIG_RISCV_64=y
 CONFIG_DEBUG=y
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685692.1066830 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewe-0008Jk-RC; Mon, 26 Feb 2024 17:39:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685692.1066830; Mon, 26 Feb 2024 17:39:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewe-0008Jd-Nu; Mon, 26 Feb 2024 17:39:16 +0000
Received: by outflank-mailman (input) for mailman id 685692;
 Mon, 26 Feb 2024 17:39:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewd-0007pd-4Y
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:15 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f4ec6ccb-d4cd-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 18:39:14 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d2533089f6so40291731fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:13 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f4ec6ccb-d4cd-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969152; x=1709573952; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=z4jxrAoYCKDkhvwUTVgqE4mrg8XDUPc3gTT+Ybsf5Eg=;
        b=mJqi/cqb2UyVSHd15RNjWvsuwdYDnrYAWKOo0aY4N+xQpWP5Zh3CCbCB1/WElNxG+N
         I2ns+KxasUpKZNHpc8QQ883Nxv+Fnx797c+v3Es3XocJCGIcmYJT8l13tX8DKfd1ksuV
         Wh6GZrmpe5Fv0j0muk5eUgJW8fr1QLGMKPaAIgHRzP1UkkRJS02ACdK+BwL/HEmS79AW
         Mhzzd6/lgvyRa8kNE0JVh071eUl5eoXvDWXMf5piRsDDjP4ZqI21QHW0IV+q6plMvHfk
         TfK81RDUO7dYQRvSlNG+yKT16fUFrSQj0ed5Jj13AUjirf+9uTircfePVn/QeJaP1oYt
         YkCA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969152; x=1709573952;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=z4jxrAoYCKDkhvwUTVgqE4mrg8XDUPc3gTT+Ybsf5Eg=;
        b=IbxaL/j7TOBg68gB34SSum0roc+rZ4w0VIOty534rUX+8umQwZU4L3eGerQRRtJoV8
         fEAQDQIDZMHR+60AHxViywTc1EXkyy7HBbpCEGxj6DMWi6euTvpyDQNNAleEE2xgw6yQ
         M9GMZ98WNEugx6UO932Wv/T8vklYqFUgZhNkrizN4cGkleOmIgdmFCB4fCmHYAAAoi5y
         /vMSwfBazRKn7bGS1lObOXIaJnXexzpIjWd/wiAHtmzLGAJtfQKMTEB4240pjUIDVesp
         giGA8TlrsagVxilzIUC8CREGUPiPWDCHXRPMINo/T0gkvz1PZbxeWhm4H4GHr4ripncS
         Zbfg==
X-Gm-Message-State: AOJu0YwCB7+ZNo3PJPPEhGKe7Zs64WTB6Rfp+ZCj3s20SuiXkKnY3jhG
	hAdUa7ODh3gNSPcAI//ApvAUxaf7Cb2fqapgvfLdBCvstSlhzBtVoZlSrb87
X-Google-Smtp-Source: AGHT+IEibpaHUx8FcU8JFqu57+mlanlhlfIrSKsfNowGL2agu5Ar956bLzTD5O4HwptS0B9wTDOhPA==
X-Received: by 2002:a2e:b0c8:0:b0:2d2:7781:382e with SMTP id g8-20020a2eb0c8000000b002d27781382emr4528564ljl.32.1708969152357;
        Mon, 26 Feb 2024 09:39:12 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 02/23] xen/riscv: use some asm-generic headers
Date: Mon, 26 Feb 2024 18:38:44 +0100
Message-ID: <e0bf8689422066f5fcdf9e063717f6c748ea3af7.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The following headers end up the same as asm-generic's version:
* altp2m.h
* device.h
* div64.h
* hardirq.h
* hypercall.h
* iocap.h
* paging.h
* percpu.h
* random.h
* softirq.h
* vm_event.h

RISC-V should utilize the asm-generic's version of the mentioned
headers instead of introducing them in the arch-specific folder.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5:
 - Nothing changed. Only rebase.
 - update the commit message.
 - drop the message above revision log as there is no depenency for this patch
   from other patch series.
---
Changes in V4:
- removed numa.h from asm/include/Makefile because of the patch: [PATCH v2] NUMA: no need for asm/numa.h when !NUMA
- updated the commit message
---
Changes in V3:
 - remove monitor.h from the RISC-V asm/Makefile list.
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V2:
 - New commit introduced in V2.
---
 xen/arch/riscv/include/asm/Makefile | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/Makefile

diff --git a/xen/arch/riscv/include/asm/Makefile b/xen/arch/riscv/include/asm/Makefile
new file mode 100644
index 0000000000..ced02e26ed
--- /dev/null
+++ b/xen/arch/riscv/include/asm/Makefile
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+generic-y += altp2m.h
+generic-y += device.h
+generic-y += div64.h
+generic-y += hardirq.h
+generic-y += hypercall.h
+generic-y += iocap.h
+generic-y += paging.h
+generic-y += percpu.h
+generic-y += random.h
+generic-y += softirq.h
+generic-y += vm_event.h
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685694.1066849 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewi-0000Pq-E8; Mon, 26 Feb 2024 17:39:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685694.1066849; Mon, 26 Feb 2024 17:39:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewi-0000Pg-95; Mon, 26 Feb 2024 17:39:20 +0000
Received: by outflank-mailman (input) for mailman id 685694;
 Mon, 26 Feb 2024 17:39:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewg-0007o5-8k
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:18 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f6a85ce2-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:16 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d2505352e6so47244191fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:16 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6a85ce2-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969155; x=1709573955; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=5QXD9Bqte9tbBNNj19StZof4KTDsqaQLUFyEe8EONIE=;
        b=RYnO8C25oJP4vaPalPUe0Gywg5Q41kMBYG3+C6a3V09TZUfmOClrNZFuOAXcuClcL7
         nrn3webVJwDuIgPBD4EOAB8xloTxmHTfGx7ZUh2rxmRSmcxVUbiH05E4GGR4XV1hqgiO
         Uu1mgqKJ9qHGDdLVOeYQyMMrbXy8WScrIs5Wkn8nZxC2a/M5ntb6/8bAqDnfQc6rJsRK
         t0Ilh0CYxU4GPfYD6zByUlabdkZwG/SHfLdFbEtLJBff0Lr4mhePMiY/iVp90ntD08tX
         zTokcHF7MQFi4tZ+oFisd7RQeR77N4w7eCjW5462DZvl+lV2AqgJoHLYRsFBz6u1F9Ew
         AJvg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969155; x=1709573955;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=5QXD9Bqte9tbBNNj19StZof4KTDsqaQLUFyEe8EONIE=;
        b=QhInZbf+dmUjPLWmLrdkE/p8HUQQIrNNBUdxJ0lA6YLR/JwJ1JTQyptQ2eTGN5j3eU
         UkZHlnnCt8CGbd2Ve2Zkp44bI785zypTzSNLnbRyKCQxfqszavJxMW53NT+DKuyqkVPE
         ZjfjxYZtJvh+aAwc8mE8fBAuywklj3EZqMgtkol/zwtxoQ7jzDHDXJvQSyA0uK0pKWM1
         lBE8yR5YlxsWFQGleEkuKy0MPm6l3vZqXGhoiOeeZruzvSCeeUhBCisPjQrdY+TvAJmT
         gCGzSUHx3WQGNmN4nhouDtjShDwrxCFXCz3OzkqxW5hbA/EgYaRapunj467+ScsJzZSx
         HoFg==
X-Gm-Message-State: AOJu0YwcLmRAR7fL0EI7OOCKUD5drTwmBRFMlUrjM1N1FaT08LPHHyJI
	IOiZ//4VvyEAyxPTr/AoaTJXYfR1L4Qyg75rPA0cKFEU+YmjyLcJkX04QIr1
X-Google-Smtp-Source: AGHT+IFvsPxhN4C3ThBll7DvHcWDkL9leZhN0tUsmEUehjtHp8dOS7FUThlx2z3fbjBdgSnh26b3Yg==
X-Received: by 2002:a2e:be07:0:b0:2d2:86ca:c0d0 with SMTP id z7-20020a2ebe07000000b002d286cac0d0mr3368307ljq.16.1708969155350;
        Mon, 26 Feb 2024 09:39:15 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 04/23] xen/asm-generic: introduce generic fls() and flsl() functions
Date: Mon, 26 Feb 2024 18:38:46 +0100
Message-ID: <df7ab5055ef08fa595f913072302770a3f6a5c33.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

These functions can be useful for architectures that don't
have corresponding arch-specific instructions.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Changes in V5:
   - new patch
---
 xen/include/asm-generic/bitops/fls.h  | 18 ++++++++++++++++++
 xen/include/asm-generic/bitops/flsl.h | 10 ++++++++++
 2 files changed, 28 insertions(+)
 create mode 100644 xen/include/asm-generic/bitops/fls.h
 create mode 100644 xen/include/asm-generic/bitops/flsl.h

diff --git a/xen/include/asm-generic/bitops/fls.h b/xen/include/asm-generic/bitops/fls.h
new file mode 100644
index 0000000000..369a4c790c
--- /dev/null
+++ b/xen/include/asm-generic/bitops/fls.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FLS_H_
+#define _ASM_GENERIC_BITOPS_FLS_H_
+
+/**
+ * fls - find last (most-significant) bit set
+ * @x: the word to search
+ *
+ * This is defined the same way as ffs.
+ * Note fls(0) = 0, fls(1) = 1, fls(0x80000000) = 32.
+ */
+
+static inline int fls(unsigned int x)
+{
+    return generic_fls(x);
+}
+
+#endif /* _ASM_GENERIC_BITOPS_FLS_H_ */
diff --git a/xen/include/asm-generic/bitops/flsl.h b/xen/include/asm-generic/bitops/flsl.h
new file mode 100644
index 0000000000..d0a2e9c729
--- /dev/null
+++ b/xen/include/asm-generic/bitops/flsl.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FLSL_H_
+#define _ASM_GENERIC_BITOPS_FLSL_H_
+
+static inline int flsl(unsigned long x)
+{
+    return generic_flsl(x);
+}
+
+#endif /* _ASM_GENERIC_BITOPS_FLSL_H_ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685695.1066854 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewi-0000To-Qb; Mon, 26 Feb 2024 17:39:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685695.1066854; Mon, 26 Feb 2024 17:39:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewi-0000SY-L6; Mon, 26 Feb 2024 17:39:20 +0000
Received: by outflank-mailman (input) for mailman id 685695;
 Mon, 26 Feb 2024 17:39:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewh-0007o5-8r
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:19 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f71e570e-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:17 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d26227d508so37271881fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:17 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.15
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:15 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f71e570e-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969157; x=1709573957; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=cZDHTEej6Jv3SYqhJWcdCnlmv5/Saryrf0OJ+2bCraY=;
        b=f/BTuAOOL9lrH1hLeKUPooDvh2qzJ8i95BO0kUpxtORTrQW+t+1xPGBXirQuum6B8G
         XJgntsVAix3Z5bMWIgJblP31oSvyiRyMQuQTkHt6BeXU6EgPIcr8SZ9I2tcAgwq3HyIx
         chQBzR01FGqxnhiXX6OP2YSUJaXxm/l5OdE0sCw1K3e/7epV7oAPNeQ//JYpOq8ETPQZ
         saK4ZsTzRyY0KG1p1P0TIz94zRt28Pri4vKEukhiSYIO+lWGi4NXfIu/wJ0MI3kJO1I5
         Mzd4RwgNtEoz2Wd8PXO9q9aD4YOt8qJYXHbyiXi8wZC+qYcfi3LRmUvzlsRCSfbhID/L
         30Cw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969157; x=1709573957;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=cZDHTEej6Jv3SYqhJWcdCnlmv5/Saryrf0OJ+2bCraY=;
        b=j7p44ND+o0vi4ndD/kHTFYAOm+oEC/WVhJE7/Qmr9Jb0tj8Dr3m6a1jxYhtz+dS038
         FSedbsSIw2LZviCgB3g8/9gYqgnlw+tRyvy+tYZkSUdLNZ5+uiz+qYYl20ZxTxPfse1V
         15cxC91TL8SqGPLa8ADUTJbeuarOfw3KgdzetLZFkjNL5Xz1o4j1+sbCw+OwlOZsqiLH
         Q8EaWh0NopXdA4qx/Zb4IE79yQ5H4HKhhaB841RsWxq8TJgcKlx6OQZjZlqFbS0y1B4H
         2gaFNHpjP4VCftIyVMqqs7r1s0kHHBvFdk6a2Pl/7ME+W3gSyZ/b8LAV4G3m8hybrhcl
         aTxw==
X-Gm-Message-State: AOJu0YxzlJuf3WMo6o7rierF08/8j8jQHTnKQsujIcAz1wAjLIZ9vBKV
	GOzfZZYX+97NKWTpCADuqRryS9dvzRGpMkSdkDNbDwiL5x+BE1LuSKca3WfX
X-Google-Smtp-Source: AGHT+IHFR4+Ivxmy7gFlPOXPEnFVpLHuKhIHnvGtUwBdkWRa4mfpFeN/WIg0Q58ss7HZNSYz5/nM6Q==
X-Received: by 2002:a2e:bc12:0:b0:2d2:607e:8121 with SMTP id b18-20020a2ebc12000000b002d2607e8121mr5407859ljf.43.1708969156635;
        Mon, 26 Feb 2024 09:39:16 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 05/23] xen/asm-generic: introduce generic find first set bit functions
Date: Mon, 26 Feb 2024 18:38:47 +0100
Message-ID: <f4fce2215f629f5fe76f79557e1fe7a55876014c.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

These functions can be useful for architectures that don't
have corresponding arch-specific instructions.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Changes in V5:
   - new patch
---
 xen/include/asm-generic/bitops/__ffs.h        | 47 +++++++++++++++++++
 xen/include/asm-generic/bitops/ffs.h          |  9 ++++
 xen/include/asm-generic/bitops/ffsl.h         | 16 +++++++
 .../asm-generic/bitops/find-first-set-bit.h   | 17 +++++++
 4 files changed, 89 insertions(+)
 create mode 100644 xen/include/asm-generic/bitops/__ffs.h
 create mode 100644 xen/include/asm-generic/bitops/ffs.h
 create mode 100644 xen/include/asm-generic/bitops/ffsl.h
 create mode 100644 xen/include/asm-generic/bitops/find-first-set-bit.h

diff --git a/xen/include/asm-generic/bitops/__ffs.h b/xen/include/asm-generic/bitops/__ffs.h
new file mode 100644
index 0000000000..fecb4484d9
--- /dev/null
+++ b/xen/include/asm-generic/bitops/__ffs.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS___FFS_H_
+#define _ASM_GENERIC_BITOPS___FFS_H_
+
+/**
+ * ffs - find first bit in word.
+ * @word: The word to search
+ *
+ * Returns 0 if no bit exists, otherwise returns 1-indexed bit location.
+ */
+static inline unsigned int __ffs(unsigned long word)
+{
+    unsigned int num = 0;
+
+#if BITS_PER_LONG == 64
+    if ( (word & 0xffffffff) == 0 )
+    {
+        num += 32;
+        word >>= 32;
+    }
+#endif
+    if ( (word & 0xffff) == 0 )
+    {
+        num += 16;
+        word >>= 16;
+    }
+    if ( (word & 0xff) == 0 )
+    {
+        num += 8;
+        word >>= 8;
+    }
+    if ( (word & 0xf) == 0 )
+    {
+        num += 4;
+        word >>= 4;
+    }
+    if ( (word & 0x3) == 0 )
+    {
+        num += 2;
+        word >>= 2;
+    }
+    if ( (word & 0x1) == 0 )
+        num += 1;
+    return num;
+}
+
+#endif /* _ASM_GENERIC_BITOPS___FFS_H_ */
diff --git a/xen/include/asm-generic/bitops/ffs.h b/xen/include/asm-generic/bitops/ffs.h
new file mode 100644
index 0000000000..3f75fded14
--- /dev/null
+++ b/xen/include/asm-generic/bitops/ffs.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FFS_H_
+#define _ASM_GENERIC_BITOPS_FFS_H_
+
+#include <xen/macros.h>
+
+#define ffs(x) ({ unsigned int t_ = (x); fls(ISOLATE_LSB(t_)); })
+
+#endif /* _ASM_GENERIC_BITOPS_FFS_H_ */
diff --git a/xen/include/asm-generic/bitops/ffsl.h b/xen/include/asm-generic/bitops/ffsl.h
new file mode 100644
index 0000000000..d0996808f5
--- /dev/null
+++ b/xen/include/asm-generic/bitops/ffsl.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FFSL_H_
+#define _ASM_GENERIC_BITOPS_FFSL_H_
+
+/**
+ * ffsl - find first bit in long.
+ * @word: The word to search
+ *
+ * Returns 0 if no bit exists, otherwise returns 1-indexed bit location.
+ */
+static inline unsigned int ffsl(unsigned long word)
+{
+    return generic_ffsl(word);
+}
+
+#endif /* _ASM_GENERIC_BITOPS_FFSL_H_ */
diff --git a/xen/include/asm-generic/bitops/find-first-set-bit.h b/xen/include/asm-generic/bitops/find-first-set-bit.h
new file mode 100644
index 0000000000..7d28b8a89b
--- /dev/null
+++ b/xen/include/asm-generic/bitops/find-first-set-bit.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FIND_FIRST_SET_BIT_H_
+#define _ASM_GENERIC_BITOPS_FIND_FIRST_SET_BIT_H_
+
+/**
+ * find_first_set_bit - find the first set bit in @word
+ * @word: the word to search
+ *
+ * Returns the bit-number of the first set bit (first bit being 0).
+ * The input must *not* be zero.
+ */
+static inline unsigned int find_first_set_bit(unsigned long word)
+{
+        return ffsl(word) - 1;
+}
+
+#endif /* _ASM_GENERIC_BITOPS_FIND_FIRST_SET_BIT_H_ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685696.1066870 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewk-0000wQ-93; Mon, 26 Feb 2024 17:39:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685696.1066870; Mon, 26 Feb 2024 17:39:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewk-0000va-3N; Mon, 26 Feb 2024 17:39:22 +0000
Received: by outflank-mailman (input) for mailman id 685696;
 Mon, 26 Feb 2024 17:39:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewi-0007o5-9I
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:20 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f7e4da54-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:18 +0100 (CET)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2d21cdbc85bso53510311fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:18 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.16
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7e4da54-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969157; x=1709573957; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=eivVMumAvXMrEoExb3VSoypb50X29skE1zCwutaVn60=;
        b=NTbUmIcsaJYyu1/MgZgvqf3H96ZIsl3SJmbf6PNDTP7EiPVQ8WuXbIu1oOZGqHnbtT
         ed2DasKbbOT8f1rnW4zQzuRWI57LdGmiZJ6BcgniolSh/XgBpLflarszVZrpGEbt02RS
         ZUD3KKIzJHnN+AhAdONK84fV3H3arvuQ5RQgCTPBNhyOrXla0bC5kVWMv/2Wx2B+YJYX
         4tZHdo+aJXvsdq4NFoUJkotn2znMNsno0Q78jjYpxuN8ChjiCld53l0F+wN0VzaJcrhj
         tnS1I8ltFRnk/I2+FwzdP9A7grsKvtVRZISRzGiI0jJOBtP0+nYxC1hLHPA5WR9xvanx
         L0EQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969157; x=1709573957;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=eivVMumAvXMrEoExb3VSoypb50X29skE1zCwutaVn60=;
        b=tYR8jCVrq1Kfr7WFTmhdQUx2QXVCkGi9tGR4qHpW7bSOngbK70fBOf46E0H2w4+vW6
         +5vfvoN8C2VHvxqsTRLOT2mef6V2da0AT6shzvG4WGC/RT9VLyS7n9cxDUCIyK5Nx+o7
         8oziGvyUmvuofT0Tk39/kZuFkXYfd1vyK4Kt5R+tNV+zGRFVE1aq8SVtpzOFF61ddf8D
         lAtcZ916WnS8aErxuablTEFsbGOxCFVt2aIUPT1tZ3grK9O85AI13uAnBm9Ze76P+/Zn
         XCAPYFMeny4Tk3Un3eKmAATxTOJkWA/skvWbLF4cLdQBA7+MBA7/ceYZLxpHl0DGAjcO
         qW1A==
X-Gm-Message-State: AOJu0YwRUKoEpvq9kOFbRsr0qpPNsn/S9wRgxKEmAmAM2t1Lf2j7T7or
	BJxTYPx15Us2pMdahLhsqoSaa10Nzd1aqBMEVz5++V8yYuEDnZDoAW4b1h4l
X-Google-Smtp-Source: AGHT+IEeMZ/eHi/bYLV/U1FBIMclQJSUe6RCUzqCHjHTnw3w3bZD2rpAWXJhGJjEYOmFib6YAXHCwg==
X-Received: by 2002:a2e:8217:0:b0:2d2:6574:4011 with SMTP id w23-20020a2e8217000000b002d265744011mr4335694ljg.30.1708969157499;
        Mon, 26 Feb 2024 09:39:17 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 06/23] xen/asm-generic: introduce generic ffz()
Date: Mon, 26 Feb 2024 18:38:48 +0100
Message-ID: <a5b5c23122ee55cad72dad74b47cb7a539ebbc01.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The generic ffz() can be useful for architectures
that don't have corresponding arch-specific instruction.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Changes in V5:
   - new patch
---
 xen/include/asm-generic/bitops/ffz.h | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 xen/include/asm-generic/bitops/ffz.h

diff --git a/xen/include/asm-generic/bitops/ffz.h b/xen/include/asm-generic/bitops/ffz.h
new file mode 100644
index 0000000000..5932fe6695
--- /dev/null
+++ b/xen/include/asm-generic/bitops/ffz.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_FFZ_H_
+#define _ASM_GENERIC_BITOPS_FFZ_H_
+
+/*
+ * ffz - find first zero in word.
+ * @word: The word to search
+ *
+ * Undefined if no zero exists, so code should check against ~0UL first.
+ *
+ * ffz() is defined as __ffs() and not as ffs() as it is defined in such
+ * a way in Linux kernel (6.4.0 ) from where this header was taken, so this
+ * header is supposed to be aligned with Linux kernel version.
+ * Also, most architectures are defined in the same way in Xen.
+ */
+#define ffz(x)  __ffs(~(x))
+
+#endif /* _ASM_GENERIC_BITOPS_FFZ_H_ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685697.1066880 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewl-0001HB-Md; Mon, 26 Feb 2024 17:39:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685697.1066880; Mon, 26 Feb 2024 17:39:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewl-0001Gn-Ie; Mon, 26 Feb 2024 17:39:23 +0000
Received: by outflank-mailman (input) for mailman id 685697;
 Mon, 26 Feb 2024 17:39:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewj-0007o5-I7
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:21 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f89e9859-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:20 +0100 (CET)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2d28464c554so23731621fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:19 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.17
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f89e9859-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969159; x=1709573959; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=xlE/LVl89DqUSGdn7SyMPtS4yZCQebNAKpxbRc6e8bw=;
        b=MpHP7mRNqs6C2VufmAb9Js1Sangzi9czoJPmP3L7wgYCx8W+3tg63hzBc5b0NTe/if
         nob8Eidhp+OVOXkFaiKzhDtF2OBs3W2hSZTt6CEmgTPanK2gnF6Hvuthhp6xI3sPJR75
         vcmBkm1U+yP9V7L5nqSKrPhjmP00HJPVUTZ5fyH9ZFJPUEzTQDVKl4rzCyZbUUIvSN7j
         u8wZ6pu5aO7bW0EmDei4ubvHeBuynUDk3U7BWtqXfDcd91O6jLZt9TP+NKSR/OmTCp2/
         rZbowiNo8gEFdixa9HCRYN8rlSfMadHNQJAfTZyk3jVWEoWTnGs5eppTbdepFUHMHAM0
         nZVg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969159; x=1709573959;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=xlE/LVl89DqUSGdn7SyMPtS4yZCQebNAKpxbRc6e8bw=;
        b=TqP5iI4YqdSau1U8peTcS6yOIxOh59u/N2sDLXvaxG0tqIt3V1agWHeOOV0tiETG+z
         vlGaIR2dkdBuBMu549CiaFH4QPv6s9+w1hqKg/dS8b/PC8h45LUvu1MvoPTPMOJdZp2j
         8Jlg68WQIDl64lyQTquZ8DfepvFd7nJ3lM55WbsDnAKegl3F9+JhanmgIDIIIpmdyQ4d
         cTkS7UhqN1Y+gdRoeh+IF0oIN+HwymGZMfDxVROKJuVBNWXDzdUxNTrKFOEx6Zt92w6H
         dQl6ATojB7Byc4GQFlrRt1ZxCB8IgB32nYo6KiZ/LW6DP0uqoNXZBUuYemdqxhM9pWCQ
         kyOQ==
X-Gm-Message-State: AOJu0YxB6KPZ09mngZZcGqcX6JlEr4N5C7M7fqsmq8eTrVMr7OKGYeaO
	5gWPHpzeHl2LLSKKMj11I4CuDNUZtaikkImVIK5Q4npv+o326OteRQW8a5pH
X-Google-Smtp-Source: AGHT+IEjSQ8o3xX5dVo2wHraeEsZEaWY/KLvFJUb5rGHMzeij7VJlbzkgb8IFvpjotiU9+JEwAdSJw==
X-Received: by 2002:a2e:854f:0:b0:2d2:8cf7:ed07 with SMTP id u15-20020a2e854f000000b002d28cf7ed07mr1951027ljj.11.1708969158836;
        Mon, 26 Feb 2024 09:39:18 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 07/23] xen/asm-generic: introduce generic hweight64()
Date: Mon, 26 Feb 2024 18:38:49 +0100
Message-ID: <15dde1f7c5157321998b5518238b247a27d9fec4.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The generic hweight() function can be useful for architectures
that don't have corresponding arch-specific instructions.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Changes in V5:
   - new patch
---
 xen/include/asm-generic/bitops/hweight.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)
 create mode 100644 xen/include/asm-generic/bitops/hweight.h

diff --git a/xen/include/asm-generic/bitops/hweight.h b/xen/include/asm-generic/bitops/hweight.h
new file mode 100644
index 0000000000..0d7577054e
--- /dev/null
+++ b/xen/include/asm-generic/bitops/hweight.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_HWEIGHT_H_
+#define _ASM_GENERIC_BITOPS_HWEIGHT_H_
+
+/*
+ * hweightN - returns the hamming weight of a N-bit word
+ * @x: the word to weigh
+ *
+ * The Hamming Weight of a number is the total number of bits set in it.
+ */
+#define hweight64(x) generic_hweight64(x)
+
+#endif /* _ASM_GENERIC_BITOPS_HWEIGHT_H_ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685698.1066885 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewm-0001Ln-5H; Mon, 26 Feb 2024 17:39:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685698.1066885; Mon, 26 Feb 2024 17:39:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewl-0001Kh-Us; Mon, 26 Feb 2024 17:39:23 +0000
Received: by outflank-mailman (input) for mailman id 685698;
 Mon, 26 Feb 2024 17:39:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewk-0007o5-TR
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:22 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f962571f-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:21 +0100 (CET)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2d24a727f78so47697501fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:21 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.18
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f962571f-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969160; x=1709573960; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=uNrkylm2yFBJDlGcN/iXxdhzHVaAgu9JHW+hLkBgjOo=;
        b=Hi21yEtrZJvjjaCloV61QKBfjsVRXDnqEMyIiEJUHA3NO4b2i144e7vR/w1fxmnUNu
         toi9WoCDWHp0Zb2mkPhxFkcxWLqkGVQNsy9XHF+eAg3jRDQdbJF20SKh9AqKxeAXjzk+
         E3bvxjZ03VBs/nUrQQn63CgeeEfjkSdWjem+zj7929zNv3LQcVM4UFf+DUd3yw8ZegTg
         FMjmmMZGpE9QjyIAH/v32EyUifjdPB2DPCOkhvkMZ4wCsy1zhZjFLkSEdfG0m90aMnK/
         isEll0uj0owgSBjPONochx35SiQ8zFV8wt9Eufyr1sx/WunOjLJDPmuaE4lRTTyVqNLQ
         tv7g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969160; x=1709573960;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=uNrkylm2yFBJDlGcN/iXxdhzHVaAgu9JHW+hLkBgjOo=;
        b=K7jnBjjmSArRFlL3SGh6YZt+q9Dzac1+rLG7+GMg9kq43tAGy8D/Ds3f2tqEa/OiGN
         Pl5o6S47bDK6D5JuS2Ctk6iHiP8+4XcOrQDNLC52Cfj34pYBKy/cYaPpzUMHCdYLqlwJ
         RtrvVmkxDTN8hVOfaD7RrRIn9WIyMYZt8TAdnxxt6OpgZhmM4uKe72Nra5Xh2zT9hlJF
         LER4mO7Lp40BQoSW4/dp9qlrm001txgvboLHSqD8737C6oL/yWTkydgsNAMpi9Pr45/p
         jcyXpjah0syQ3J0deglNJ3pS7UDU+iMi0VaKni7itLV7fqo5Ig87mDCLGsASCE/YmhMb
         eSjQ==
X-Gm-Message-State: AOJu0Yxy6fTY6O8ZMjWlCSvTE0/xNozX5SMcc0pjX2NEbNHNSAl0X85O
	dLN3WRo2ISyVQmYEF1Le8pFbCUWt0prMrFrHgRiGSNF8PNUWyJQ+SNCQ6oi7
X-Google-Smtp-Source: AGHT+IHRuLqiYqHwAzLIXYX6Tjhc1JShcdSKACZaklZkja1toysaGTkLSSLcep1Bg/QWsWfPk/A67Q==
X-Received: by 2002:a05:651c:546:b0:2d2:5057:bbf4 with SMTP id q6-20020a05651c054600b002d25057bbf4mr6267478ljp.20.1708969160070;
        Mon, 26 Feb 2024 09:39:20 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 08/23] xen/asm-generic: introduce generic non-atomic test_*bit()
Date: Mon, 26 Feb 2024 18:38:50 +0100
Message-ID: <7cd2b77f2cce41a84ee7746833bb22bb99d94df1.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The patch introduces the following generic functions:
* test_bit
* generic___test_and_set_bit
* generic___test_and_clear_bit
* generic___test_and_change_bit

Also, the patch introduces the following generics which are
used by the functions mentioned above:
* BITOP_BITS_PER_WORD
* BITOP_MASK
* BITOP_WORD
* BITOP_TYPE

These functions and macros can be useful for architectures
that don't have corresponding arch-specific instructions.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Changes in V5:
   - new patch
---
 xen/include/asm-generic/bitops/bitops-bits.h  | 21 +++++
 .../asm-generic/bitops/generic-non-atomic.h   | 89 +++++++++++++++++++
 xen/include/asm-generic/bitops/test-bit.h     | 18 ++++
 3 files changed, 128 insertions(+)
 create mode 100644 xen/include/asm-generic/bitops/bitops-bits.h
 create mode 100644 xen/include/asm-generic/bitops/generic-non-atomic.h
 create mode 100644 xen/include/asm-generic/bitops/test-bit.h

diff --git a/xen/include/asm-generic/bitops/bitops-bits.h b/xen/include/asm-generic/bitops/bitops-bits.h
new file mode 100644
index 0000000000..4ece2affd6
--- /dev/null
+++ b/xen/include/asm-generic/bitops/bitops-bits.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_BITS_H_
+#define _ASM_GENERIC_BITOPS_BITS_H_
+
+#ifndef BITOP_BITS_PER_WORD
+#define BITOP_BITS_PER_WORD     32
+#endif
+
+#ifndef BITOP_MASK
+#define BITOP_MASK(nr)          (1U << ((nr) % BITOP_BITS_PER_WORD))
+#endif
+
+#ifndef BITOP_WORD
+#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
+#endif
+
+#ifndef BITOP_TYPE
+typedef uint32_t bitops_uint_t;
+#endif
+
+#endif /* _ASM_GENERIC_BITOPS_BITS_H_ */
diff --git a/xen/include/asm-generic/bitops/generic-non-atomic.h b/xen/include/asm-generic/bitops/generic-non-atomic.h
new file mode 100644
index 0000000000..42569d0d7c
--- /dev/null
+++ b/xen/include/asm-generic/bitops/generic-non-atomic.h
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * The file is based on Linux ( 6.4.0 ) header:
+ *   include/asm-generic/bitops/generic-non-atomic.h
+ * 
+ * Only functions that can be reused in Xen were left; others were removed.
+ * 
+ * Also, the following changes were done:
+ *  - it was updated the message inside #ifndef ... #endif.
+ *  - __always_inline -> always_inline to be align with definition in
+ *    xen/compiler.h.
+ *  - update function prototypes from
+ *    generic___test_and_*(unsigned long nr nr, volatile unsigned long *addr) to
+ *    generic___test_and_*(unsigned long nr, volatile void *addr) to be
+ *    consistent with other related macros/defines.
+ *  - convert identations from tabs to spaces.
+ *  - inside generic__test_and_* use 'bitops_uint_t' instead of 'unsigned long'
+ *    to be generic.
+ */
+
+#ifndef __ASM_GENERIC_BITOPS_GENERIC_NON_ATOMIC_H
+#define __ASM_GENERIC_BITOPS_GENERIC_NON_ATOMIC_H
+
+#include <xen/compiler.h>
+
+#include <asm-generic/bitops/bitops-bits.h>
+
+#ifndef _LINUX_BITOPS_H
+#error only <xen/bitops.h> can be included directly
+#endif
+
+/*
+ * Generic definitions for bit operations, should not be used in regular code
+ * directly.
+ */
+
+/**
+ * generic___test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic___test_and_set_bit(unsigned long nr, volatile void *addr)
+{
+    bitops_uint_t mask = BITOP_MASK(nr);
+    bitops_uint_t *p = ((bitops_uint_t *)addr) + BITOP_WORD(nr);
+    bitops_uint_t old = *p;
+
+    *p = old | mask;
+    return (old & mask) != 0;
+}
+
+/**
+ * generic___test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ *
+ * This operation is non-atomic and can be reordered.
+ * If two examples of this operation race, one can appear to succeed
+ * but actually fail.  You must protect multiple accesses with a lock.
+ */
+static always_inline bool
+generic___test_and_clear_bit(bitops_uint_t nr, volatile void *addr)
+{
+    bitops_uint_t mask = BITOP_MASK(nr);
+    bitops_uint_t *p = ((bitops_uint_t *)addr) + BITOP_WORD(nr);
+    bitops_uint_t old = *p;
+
+    *p = old & ~mask;
+    return (old & mask) != 0;
+}
+
+/* WARNING: non atomic and it can be reordered! */
+static always_inline bool
+generic___test_and_change_bit(unsigned long nr, volatile void *addr)
+{
+    bitops_uint_t mask = BITOP_MASK(nr);
+    bitops_uint_t *p = ((bitops_uint_t *)addr) + BITOP_WORD(nr);
+    bitops_uint_t old = *p;
+
+    *p = old ^ mask;
+    return (old & mask) != 0;
+}
+
+#endif /* __ASM_GENERIC_BITOPS_GENERIC_NON_ATOMIC_H */
diff --git a/xen/include/asm-generic/bitops/test-bit.h b/xen/include/asm-generic/bitops/test-bit.h
new file mode 100644
index 0000000000..6fb414d808
--- /dev/null
+++ b/xen/include/asm-generic/bitops/test-bit.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_BITOPS_TESTBIT_H_
+#define _ASM_GENERIC_BITOPS_TESTBIT_H_
+
+#include <asm-generic/bitops/bitops-bits.h>
+
+/**
+ * test_bit - Determine whether a bit is set
+ * @nr: bit number to test
+ * @addr: Address to start counting from
+ */
+static inline int test_bit(int nr, const volatile void *addr)
+{
+    const volatile bitops_uint_t *p = addr;
+    return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD - 1)));
+}
+
+#endif /* _ASM_GENERIC_BITOPS_TESTBIT_H_ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685699.1066900 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewo-0001v6-J5; Mon, 26 Feb 2024 17:39:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685699.1066900; Mon, 26 Feb 2024 17:39:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewo-0001ur-EG; Mon, 26 Feb 2024 17:39:26 +0000
Received: by outflank-mailman (input) for mailman id 685699;
 Mon, 26 Feb 2024 17:39:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewm-0007o5-Gi
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:24 +0000
Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com
 [2a00:1450:4864:20::22d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fa32c1b7-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:22 +0100 (CET)
Received: by mail-lj1-x22d.google.com with SMTP id
 38308e7fff4ca-2d243797703so42731401fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:22 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa32c1b7-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969161; x=1709573961; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/rCykoQ35RmlaQov+EPt/Ynaxeb1jOtf3tAtx9aL5IQ=;
        b=U/QvG8zMJLTu70KPdl9e9qFLq527Zdd8s5mYtJvyi/qmj7wS0Nx05mEH/rifPSFkMX
         DMSX9mOwzdZizlIn9u45pMopOZFguEhrytbCfxmlZzIh0XRlgHCGFVA3uWUmF90xRllR
         mK7aeR5sNpmUVEzkZCCFGWjJvzBBwgwP3YEcFeAesMoTjXS9HaqbAF+hisVGVUGxisY5
         txjTN3BOjQVaGvZp/GXHHaQGaTzY4ZDj4J8Ymfoa/EGOzHVcyqshkg+2uAB4sA47EbZM
         RegFgZorAholpHI3Jk/RkVm705eKT2IHLr/oxz2lWfN4MtVTzHnjAFRQTyEy5Ws9zEi/
         FAag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969161; x=1709573961;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=/rCykoQ35RmlaQov+EPt/Ynaxeb1jOtf3tAtx9aL5IQ=;
        b=U99dIuXeIooOfPKAszEnkNk00iIZpnZoZStO7Bawpsq3pcJojgantDcc1rFzMR4Fuw
         aH1Ye2zLS4zC6quDCeCfYltQlD4zUfhU8hysoqgoSSB7QMiBLDD6xAeHoZtxhkEVeVVZ
         g9Jv1cCo1RXkZaPrg1A72E4G7qMKPZdDuB8mXpvly+z3tssW/wzgU67eBKFNJWx0QLb9
         KDteZYW78h3vsUMedgHgK2K+FXTYpq51m8b4Qs9fqQDaPhV6zabURi5PCks65RbUJ90l
         PXRjuPxde9ylPONDv5NjSKj2Z1OCS9L+IEfwa+AAOjnjn4mxMZ2ttNijwQWwPKFL9M/B
         DMhg==
X-Gm-Message-State: AOJu0Yx6FIXDqh4gsenDe+yhjonR7XdHEJ4QfvMUtm3OM9B7Zgp1SxoT
	UN7xk6b7tkJgoNEsE3GNn6YFK79QVjEYk+LgNISPbuD7j5mW8zGrlvw82qhl
X-Google-Smtp-Source: AGHT+IFMwP8JNvbJcnskJ9n/DVf+muacLYkTr75j+QsuDPbLQVjkeynnb54mht0+ZxbixksRGIQ79Q==
X-Received: by 2002:a2e:a9a6:0:b0:2d2:9314:3ac3 with SMTP id x38-20020a2ea9a6000000b002d293143ac3mr592316ljq.4.1708969161359;
        Mon, 26 Feb 2024 09:39:21 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 09/23] xen/riscv: introduce bitops.h
Date: Mon, 26 Feb 2024 18:38:51 +0100
Message-ID: <d06f0d90199e83ed6d1a410dfd3ebb6afc5e391f.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Taken from Linux-6.4.0-rc1

Xen's bitops.h consists of several Linux's headers:
* linux/arch/include/asm/bitops.h:
  * The following function were removed as they aren't used in Xen:
        * test_and_set_bit_lock
        * clear_bit_unlock
        * __clear_bit_unlock
  * The following functions were renamed in the way how they are
    used by common code:
        * __test_and_set_bit
        * __test_and_clear_bit
  * The declaration and implementation of the following functios
    were updated to make Xen build happy:
        * clear_bit
        * set_bit
        * __test_and_clear_bit
        * __test_and_set_bit
  * linux/include/asm-generic/bitops/generic-non-atomic.h with the
    following changes:
     * Only functions that can be reused in Xen were left;
       others were removed.
     * it was updated the message inside #ifndef ... #endif.
     * __always_inline -> always_inline to be align with definition in
       xen/compiler.h.
     * update function prototypes from
       generic___test_and_*(unsigned long nr nr, volatile unsigned long *addr)
       to
       generic___test_and_*(unsigned long nr, volatile void *addr) to be
       consistent with other related macros/defines.
     * convert identations from tabs to spaces.
     * inside generic__test_and_* use 'bitops_uint_t' instead of 'unsigned long'
        to be generic.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
  Patches 04 - 08 of this patch series are prerequisite for this patch.
---
Changes in V5:
 - Code style fixes
 - s/__NOP/NOP/g
 - s/__NOT/NOT/g
 - update the comments above functions: test_and_set_bit, test_and_clear_bit, set_bit, clear_bit as all of them
   are using atomic operation and a memory barrier, so the operation in it cannot be reordered.
 - s/volatile uint32_t/volatile bitops_uint_t in  test_and_set_bit, test_and_clear_bit, set_bit, clear_bit.
 - update the commit message
 - split introduction of asm-generic functions to separate patches:
   Patches 04 - 08 of this patch series are prerequisite for this patch.
---
Changes in V4:
  - updated the commit message: dropped the message about what was taken from linux/include/asm-generic/bitops/find.h
    as related changes now are located in xen/bitops.h. Also these changes were removed from riscv/bitops.h
  - switch tabs to spaces.
  - update return type of __ffs function, format __ffs according to Xen code style. Move the function to
    respective asm-generic header.
  - format ffsl() according to Xen code style, update the type of num: int -> unsigned to be align with
    return type of the function. Move the function to respective asm-generic header.
  - add new line for the files:
      asm-generic/bitops-bits.h
      asm-generic/ffz.h
      asm-generic/find-first-bit-set.h
      asm-generic/fls.h
      asm-generic/flsl.h
      asm-generic/test-bit.h
  - rename asm-generic/find-first-bit-set.h to asm-generic/find-first-set-bit.h to be aligned with the function
    name implemented inside.
  - introduce generic___test_and*() operation for non-atomic bitops.
  - rename current __test_and_*() -> test_and_*() as their implementation are atomic aware.
  - define __test_and_*() to generic___test_and_*().
  - introduce test_and_change_bit().
  - update asm-generic/bitops/bitops-bits.h to give possoibility to change BITOP_*() macros by architecture.
    Also, it was introduced bitops_uint_t type to make generic___test_and_*() generic.
  - "include asm-generic/bitops/bitops-bits.h" to files which use its definitions.
  - add comment why generic ffz is defined as __ffs().
  - update the commit message.
  - swtich ffsl() to generic_ffsl().
---
Changes in V3:
 - update the commit message
 - Introduce the following asm-generic bitops headers:
	create mode 100644 xen/arch/riscv/include/asm/bitops.h
	create mode 100644 xen/include/asm-generic/bitops/bitops-bits.h
	create mode 100644 xen/include/asm-generic/bitops/ffs.h
	create mode 100644 xen/include/asm-generic/bitops/ffz.h
	create mode 100644 xen/include/asm-generic/bitops/find-first-bit-set.h
	create mode 100644 xen/include/asm-generic/bitops/fls.h
	create mode 100644 xen/include/asm-generic/bitops/flsl.h
	create mode 100644 xen/include/asm-generic/bitops/hweight.h
	create mode 100644 xen/include/asm-generic/bitops/test-bit.h
 - switch some bitops functions to asm-generic's versions.
 - re-sync some macros with Linux kernel version mentioned in the commit message.
 - Xen code style fixes.
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/bitops.h | 152 ++++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/config.h |   2 +
 2 files changed, 154 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h

diff --git a/xen/arch/riscv/include/asm/bitops.h b/xen/arch/riscv/include/asm/bitops.h
new file mode 100644
index 0000000000..17b3cf5be5
--- /dev/null
+++ b/xen/arch/riscv/include/asm/bitops.h
@@ -0,0 +1,152 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#define BITOP_BITS_PER_WORD BITS_PER_LONG
+
+#define BITOP_TYPE
+typedef uint64_t bitops_uint_t;
+
+#include <asm-generic/bitops/bitops-bits.h>
+
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+#if BITS_PER_LONG == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITS_PER_LONG == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITS_PER_LONG"
+#endif
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    unsigned long res, mask;                            \
+    mask = BITOP_MASK(nr);                              \
+    __asm__ __volatile__ (                              \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define __op_bit_ord(op, mod, nr, addr, ord)    \
+    __asm__ __volatile__ (                      \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define __op_bit(op, mod, nr, addr) \
+    __op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))
+
+/**
+ * test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ */
+static inline int test_and_set_bit(int nr, volatile void *p)
+{
+    volatile bitops_uint_t *addr = p;
+
+    return test_and_op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ */
+static inline int test_and_clear_bit(int nr, volatile void *p)
+{
+    volatile bitops_uint_t *addr = p;
+
+    return test_and_op_bit(and, NOT, nr, addr);
+}
+
+/**
+ * set_bit - Atomically set a bit in memory
+ * @nr: the bit to set
+ * @addr: the address to start counting from
+ *
+ * Note that @nr may be almost arbitrarily large; this function is not
+ * restricted to acting on a single-word quantity.
+ */
+static inline void set_bit(int nr, volatile void *p)
+{
+    volatile bitops_uint_t *addr = p;
+
+    __op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * clear_bit - Clears a bit in memory
+ * @nr: Bit to clear
+ * @addr: Address to start counting from
+ */
+static inline void clear_bit(int nr, volatile void *p)
+{
+    volatile bitops_uint_t *addr = p;
+
+    __op_bit(and, NOT, nr, addr);
+}
+
+/**
+ * test_and_change_bit - Toggle (change) a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is atomic and cannot be reordered.
+ * It also implies a memory barrier.
+ */
+static inline int test_and_change_bit(int nr, volatile unsigned long *addr)
+{
+	return test_and_op_bit(xor, NOP, nr, addr);
+}
+
+#undef test_and_op_bit
+#undef __op_bit
+#undef NOP
+#undef NOT
+#undef __AMO
+
+#include <asm-generic/bitops/generic-non-atomic.h>
+
+#define __test_and_set_bit generic___test_and_set_bit
+#define __test_and_clear_bit generic___test_and_clear_bit
+#define __test_and_change_bit generic___test_and_change_bit
+
+#include <asm-generic/bitops/fls.h>
+#include <asm-generic/bitops/flsl.h>
+#include <asm-generic/bitops/__ffs.h>
+#include <asm-generic/bitops/ffs.h>
+#include <asm-generic/bitops/ffsl.h>
+#include <asm-generic/bitops/ffz.h>
+#include <asm-generic/bitops/find-first-set-bit.h>
+#include <asm-generic/bitops/hweight.h>
+#include <asm-generic/bitops/test-bit.h>
+
+#endif /* _ASM_RISCV_BITOPS_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/riscv/include/asm/config.h b/xen/arch/riscv/include/asm/config.h
index 2c7f2b1ff9..479da15782 100644
--- a/xen/arch/riscv/include/asm/config.h
+++ b/xen/arch/riscv/include/asm/config.h
@@ -113,6 +113,8 @@
 # error "Unsupported RISCV variant"
 #endif
 
+#define BITS_PER_BYTE 8
+
 #define BYTES_PER_LONG (1 << LONG_BYTEORDER)
 #define BITS_PER_LONG  (BYTES_PER_LONG << 3)
 #define POINTER_ALIGN  BYTES_PER_LONG
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685700.1066904 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewp-0001zW-7F; Mon, 26 Feb 2024 17:39:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685700.1066904; Mon, 26 Feb 2024 17:39:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewo-0001yk-Ss; Mon, 26 Feb 2024 17:39:26 +0000
Received: by outflank-mailman (input) for mailman id 685700;
 Mon, 26 Feb 2024 17:39:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewn-0007o5-Cg
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:25 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id faf19dce-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:23 +0100 (CET)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2d28387db09so20490801fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:23 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.21
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:22 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: faf19dce-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969163; x=1709573963; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=PGlTx0l/7wzx/Zv16vmoEO2TfVyCaXBuPMB/Z0fPp6g=;
        b=HRKgY2YjSz5fSZrl9lAYw2hZ0drFmb5vy/csCvf8EqQmgwI/fQZ/PneMLY/8Rbu9GB
         7JnxfqvjLyLkxSYvUWEwL2UtlHWQsbTMi93bvQJHjSEgaDQjwom9LhcmXxH6/ReqLPAh
         T7d5eyDJfewGw7LlBNhvhnNKqDmmu+fLa5MCLT0tK7doH5yP6snVsBphLW7zDb7yzQ9k
         MXNqwwnZfuNPXJrrk2HGxGE9946Dgz16lawsmI4b8/KcgCa/GtjEt6BhpC1tzMbsa9tj
         9WahmJ2e3IjMxOX+Tvshkbg1S6M5M8FmNtOWbmRZo9uHA6A3TkeTQcmAs4bZlD7xQMMe
         67bg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969163; x=1709573963;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=PGlTx0l/7wzx/Zv16vmoEO2TfVyCaXBuPMB/Z0fPp6g=;
        b=PwLDIFcEizCrBN01OAV1Dhv/QTIuq6jBlvhc1HBRotSo9x/eiKwtCqI7xKFe7K7UGH
         xZn+ZbjLwFgTi+0XpeRUfGMDyFSJaJFiewq50R25U+SO2Mwn4mWOmVFq1sxOwRUznb6E
         BdVxyHzKdUdHCoYesNd82I+JuepjrsTl7UiQ54WuvuVP+L8BSGY4sdOVVlhb6pjddUKj
         7mlexGQLgsJWiPKiBTIJMWnsZbNzKTIqYgp3SvzTX9rwMbd4UNfS/g3qcGhTX2LTSYy1
         MKwKe0F2NsJCgfrTDd8sheynQnnGSUUQGmRCQ7HopxD35tgl7JEN3x+gLEn+HYR8C/j5
         vZ3g==
X-Gm-Message-State: AOJu0YxGnTXvxwOF8Bj9ZnYAFpm1Fgl6m2VoQl0X3nQaixrU+a5qaR3u
	2g90Zgn8Vvw0ZHx4L79tsqDqHaLnTpzlLeLAwidzOWp3IWU24WMkPeo1ka8L
X-Google-Smtp-Source: AGHT+IGfvvy6IL4qNFWijrM1136jAJtzr6Qi7RCs8sZOJg5TyImx7CwaLDqSR9VkmFL8jvs2UOtDHw==
X-Received: by 2002:a2e:86cf:0:b0:2d2:50bc:99d6 with SMTP id n15-20020a2e86cf000000b002d250bc99d6mr5020428ljj.35.1708969162707;
        Mon, 26 Feb 2024 09:39:22 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 10/23] xen/riscv: introduces acrquire, release and full barriers
Date: Mon, 26 Feb 2024 18:38:52 +0100
Message-ID: <85eb894608120a05eb616cea721d24e02212a5cc.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Changes in V5:
   - new patch
---
 xen/arch/riscv/include/asm/fence.h | 9 +++++++++
 1 file changed, 9 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/fence.h

diff --git a/xen/arch/riscv/include/asm/fence.h b/xen/arch/riscv/include/asm/fence.h
new file mode 100644
index 0000000000..27f46fa897
--- /dev/null
+++ b/xen/arch/riscv/include/asm/fence.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef _ASM_RISCV_FENCE_H
+#define _ASM_RISCV_FENCE_H
+
+#define RISCV_ACQUIRE_BARRIER   "\tfence r , rw\n"
+#define RISCV_RELEASE_BARRIER   "\tfence rw, w\n"
+#define RISCV_FULL_BARRIER      "\tfence rw, rw\n"
+
+#endif	/* _ASM_RISCV_FENCE_H */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685701.1066912 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewq-00027s-C7; Mon, 26 Feb 2024 17:39:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685701.1066912; Mon, 26 Feb 2024 17:39:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewp-00025R-Ml; Mon, 26 Feb 2024 17:39:27 +0000
Received: by outflank-mailman (input) for mailman id 685701;
 Mon, 26 Feb 2024 17:39:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewo-0007pd-3E
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:26 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fbbcd017-d4cd-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 18:39:25 +0100 (CET)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2d2533089f6so40294501fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:25 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fbbcd017-d4cd-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969164; x=1709573964; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GBB2G5OMKwNGN2xc04uv75wIpktlEVDQCB1awKCqGdw=;
        b=KxPibwD2dcRoZjXQrh2NrCmabM/HpeYTICUS9iwKOB4y08pjHu9/nt9/VlLfrTNv4s
         N1jVkPLOQS58fYRcg0sX7M4GqnWmKMaqBln2TLJCXBNTrYg2NTRvNllG5MIPHGWbphj/
         jhDOqXKlWEdBjMmmi/RIF/g9bTDufyWm/xV+avTwTVD/w+nh5dG2sUHrAgBQiNyqoHTp
         2gZJm6Z0fRoY6EI5xSFgAFzex/L4CyxyrxRk+Ch5REOtfnhKn0Rw/Vm4pE14RfuOffk6
         SFIca4mEKGzTbdBlDRy/pBwT/qxG62RyQ+AsC27uLVdd9/RI48E2COBdRlpDSbCS8d7X
         jfJQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969164; x=1709573964;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GBB2G5OMKwNGN2xc04uv75wIpktlEVDQCB1awKCqGdw=;
        b=vFBiFAlRdopVu+q4eeH08543NsNmB/IJ/hj9N2VCeORhuEzlWl7sEI5B7HQJsOmHww
         0SyXZA5RGsduvdzs/ZgADA+qk+L/m+lHWYgWFpmvNcqMDzFR22z905koz9QOpGZGLRH/
         kUyoVHgPtEaNWeZzVsC/reTGlHaH1rvNWu5uxv/8w4E+tby043Gh3+hLPR1bVwxdtxS0
         ue/vBICEU21zkSAOrNoyr3g3uiZNDp+HIy0Eu2TWCjAT79EYuCwBXZwoumIu7TgKT9ef
         yzoA+y/7MWJOCPzC+Iovli9+ZIOrgHH4ut1EHmE+lJwxqNXNnn2BSkyl04h3V1LY5OyV
         8sRA==
X-Gm-Message-State: AOJu0YyDvHHTckgcotKNGMbqvZduBuzfxysU3elAB6UkQQsT0xg/19c7
	ULoTVi2NYJkE8xS3dU2/enBOnhJI1vrMtMrct6hDSP4/deWY4iH/3EA+Rna4
X-Google-Smtp-Source: AGHT+IFyJAoWwLlzN+kV/Q01rxcp40BP1AB0+Ii2OKw476b0K0hDY+zjP1cUaMgvDNkhXRX/cLndWQ==
X-Received: by 2002:a2e:a23c:0:b0:2d2:91ef:51c8 with SMTP id i28-20020a2ea23c000000b002d291ef51c8mr709339ljm.47.1708969164004;
        Mon, 26 Feb 2024 09:39:24 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 11/23] xen/riscv: introduce cmpxchg.h
Date: Mon, 26 Feb 2024 18:38:53 +0100
Message-ID: <ce7604de39b3480553eeaeafc11138494016983f.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The header was taken from Linux kernl 6.4.0-rc1.

Addionally, were updated:
* add emulation of {cmp}xchg for 1/2 byte types using 32-bit atomic
  access.
* replace tabs with spaces
* replace __* variale with *__
* introduce generic version of xchg_* and cmpxchg_*.

Implementation of 4- and 8-byte cases were left as it is done in
Linux kernel as according to the RISC-V spec:
```
Table A.5 ( only part of the table was copied here )

Linux Construct       RVWMO Mapping
atomic <op> relaxed    amo<op>.{w|d}
atomic <op> acquire    amo<op>.{w|d}.aq
atomic <op> release    amo<op>.{w|d}.rl
atomic <op>            amo<op>.{w|d}.aqrl

Linux Construct       RVWMO LR/SC Mapping
atomic <op> relaxed    loop: lr.{w|d}; <op>; sc.{w|d}; bnez loop
atomic <op> acquire    loop: lr.{w|d}.aq; <op>; sc.{w|d}; bnez loop
atomic <op> release    loop: lr.{w|d}; <op>; sc.{w|d}.aqrl∗ ; bnez loop OR
                       fence.tso; loop: lr.{w|d}; <op>; sc.{w|d}∗ ; bnez loop
atomic <op>            loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez loop

The Linux mappings for release operations may seem stronger than necessary,
but these mappings are needed to cover some cases in which Linux requires
stronger orderings than the more intuitive mappings would provide.
In particular, as of the time this text is being written, Linux is actively
debating whether to require load-load, load-store, and store-store orderings
between accesses in one critical section and accesses in a subsequent critical
section in the same hart and protected by the same synchronization object.
Not all combinations of FENCE RW,W/FENCE R,RW mappings with aq/rl mappings
combine to provide such orderings.
There are a few ways around this problem, including:
1. Always use FENCE RW,W/FENCE R,RW, and never use aq/rl. This suffices
   but is undesirable, as it defeats the purpose of the aq/rl modifiers.
2. Always use aq/rl, and never use FENCE RW,W/FENCE R,RW. This does not
   currently work due to the lack of load and store opcodes with aq and rl
   modifiers.
3. Strengthen the mappings of release operations such that they would
   enforce sufficient orderings in the presence of either type of acquire mapping.
   This is the currently-recommended solution, and the one shown in Table A.5.
```

But in Linux kenrel atomics were strengthen with fences:
```
Atomics present the same issue with locking: release and acquire
variants need to be strengthened to meet the constraints defined
by the Linux-kernel memory consistency model [1].

Atomics present a further issue: implementations of atomics such
as atomic_cmpxchg() and atomic_add_unless() rely on LR/SC pairs,
which do not give full-ordering with .aqrl; for example, current
implementations allow the "lr-sc-aqrl-pair-vs-full-barrier" test
below to end up with the state indicated in the "exists" clause.

In order to "synchronize" LKMM and RISC-V's implementation, this
commit strengthens the implementations of the atomics operations
by replacing .rl and .aq with the use of ("lightweigth") fences,
and by replacing .aqrl LR/SC pairs in sequences such as:

0:      lr.w.aqrl  %0, %addr
        bne        %0, %old, 1f
        ...
        sc.w.aqrl  %1, %new, %addr
        bnez       %1, 0b
1:

with sequences of the form:

0:      lr.w       %0, %addr
        bne        %0, %old, 1f
              ...
        sc.w.rl    %1, %new, %addr   /* SC-release   */
        bnez       %1, 0b
        fence      rw, rw            /* "full" fence */
1:

following Daniel's suggestion.

These modifications were validated with simulation of the RISC-V
memory consistency model.

C lr-sc-aqrl-pair-vs-full-barrier

{}

P0(int *x, int *y, atomic_t *u)
{
        int r0;
        int r1;

        WRITE_ONCE(*x, 1);
        r0 = atomic_cmpxchg(u, 0, 1);
        r1 = READ_ONCE(*y);
}

P1(int *x, int *y, atomic_t *v)
{
        int r0;
        int r1;

        WRITE_ONCE(*y, 1);
        r0 = atomic_cmpxchg(v, 0, 1);
        r1 = READ_ONCE(*x);
}

exists (u=1 /\ v=1 /\ 0:r1=0 /\ 1:r1=0)

[1] https://marc.info/?l=linux-kernel&m=151930201102853&w=2
https://groups.google.com/a/groups.riscv.org/forum/#!topic/isa-dev/hKywNHBkAXM
https://marc.info/?l=linux-kernel&m=151633436614259&w=2
```

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5:
 - update the commit message.
 - drop ALIGN_DOWN().
 - update the definition of emulate_xchg_1_2(): 
   - lr.d -> lr.w, sc.d -> sc.w.
   - drop ret argument.
   - code style fixes around asm volatile.
   - update prototype.
   - use asm named operands.
   - rename local variables.
   - add comment above the macros
 - update the definition of __xchg_generic:
   - drop local ptr__ variable.
   - code style fixes around switch()
   - update prototype.
 - introduce RISCV_FULL_BARRIES.
 - redefine cmpxchg()
 - update emulate_cmpxchg_1_2():
   - update prototype
   - update local variables names and usage of them
   - use name asm operands.
   - add comment above the macros
---
Changes in V4:
 - Code style fixes.
 - enforce in __xchg_*() has the same type for new and *ptr, also "\n"
   was removed at the end of asm instruction.
 - dependency from https://lore.kernel.org/xen-devel/cover.1706259490.git.federico.serafini@bugseng.com/
 - switch from ASSERT_UNREACHABLE to STATIC_ASSERT_UNREACHABLE().
 - drop xchg32(ptr, x) and xchg64(ptr, x) as they aren't used.
 - drop cmpxcg{32,64}_{local} as they aren't used.
 - introduce generic version of xchg_* and cmpxchg_*.
 - update the commit message.
---
Changes in V3:
 - update the commit message
 - add emulation of {cmp}xchg_... for 1 and 2 bytes types
---
Changes in V2:
 - update the comment at the top of the header.
 - change xen/lib.h to xen/bug.h.
 - sort inclusion of headers properly.
---
 xen/arch/riscv/include/asm/cmpxchg.h | 258 +++++++++++++++++++++++++++
 1 file changed, 258 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h

diff --git a/xen/arch/riscv/include/asm/cmpxchg.h b/xen/arch/riscv/include/asm/cmpxchg.h
new file mode 100644
index 0000000000..66cbe26737
--- /dev/null
+++ b/xen/arch/riscv/include/asm/cmpxchg.h
@@ -0,0 +1,258 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (C) 2014 Regents of the University of California */
+
+#ifndef _ASM_RISCV_CMPXCHG_H
+#define _ASM_RISCV_CMPXCHG_H
+
+#include <xen/compiler.h>
+#include <xen/lib.h>
+
+#include <asm/fence.h>
+#include <asm/io.h>
+#include <asm/system.h>
+
+#define __amoswap_generic(ptr, new, ret, sfx, pre, post) \
+({ \
+    asm volatile( \
+        pre \
+        " amoswap" sfx " %0, %2, %1\n" \
+        post \
+        : "=r" (ret), "+A" (*ptr) \
+        : "r" (new) \
+        : "memory" ); \
+})
+
+/*
+ * For LR and SC, the A extension requires that the address held in rs1 be
+ * naturally aligned to the size of the operand (i.e., eight-byte aligned
+ * for 64-bit words and four-byte aligned for 32-bit words).
+ * If the address is not naturally aligned, an address-misaligned exception
+ * or an access-fault exception will be generated.
+ * 
+ * Thereby:
+ * - for 1-byte xchg access the containing word by clearing low two bits
+ * - for 2-byte xchg ccess the containing word by clearing first bit.
+ * 
+ * If resulting 4-byte access is still misalgined, it will fault just as
+ * non-emulated 4-byte access would.
+ */
+#define emulate_xchg_1_2(ptr, new, sc_sfx, pre, post) \
+({ \
+    uint32_t *aligned_ptr = (uint32_t *)((unsigned long)ptr & ~(0x4 - sizeof(*ptr))); \
+    uint8_t new_val_pos = ((unsigned long)(ptr) & (0x4 - sizeof(*ptr))) * BITS_PER_BYTE; \
+    unsigned long mask = GENMASK(((sizeof(*ptr)) * BITS_PER_BYTE) - 1, 0) << new_val_pos; \
+    unsigned int new_ = new << new_val_pos; \
+    unsigned int old_val; \
+    unsigned int xchged_val; \
+    \
+    asm volatile ( \
+        pre \
+        "0: lr.w %[op_oldval], %[op_aligned_ptr]\n" \
+        "   and  %[op_xchged_val], %[op_oldval], %z[op_nmask]\n" \
+        "   or   %[op_xchged_val], %[op_xchged_val], %z[op_new]\n" \
+        "   sc.w" sc_sfx " %[op_xchged_val], %[op_xchged_val], %[op_aligned_ptr]\n" \
+        "   bnez %[op_xchged_val], 0b\n" \
+        post \
+        : [op_oldval] "=&r" (old_val), [op_xchged_val] "=&r" (xchged_val), [op_aligned_ptr]"+A" (*aligned_ptr) \
+        : [op_new] "rJ" (new_), [op_nmask] "rJ" (~mask) \
+        : "memory" ); \
+    \
+    (__typeof__(*(ptr)))((old_val & mask) >> new_val_pos); \
+})
+
+#define __xchg_generic(ptr, new, size, sfx, pre, post) \
+({ \
+    __typeof__(*(ptr)) new__ = (new); \
+    __typeof__(*(ptr)) ret__; \
+    switch ( size ) \
+    { \
+    case 1: \
+    case 2: \
+        ret__ = emulate_xchg_1_2(ptr, new__, sfx, pre, post); \
+        break; \
+    case 4: \
+        __amoswap_generic(ptr, new__, ret__,\
+                          ".w" sfx,  pre, post); \
+        break; \
+    case 8: \
+        __amoswap_generic(ptr, new__, ret__,\
+                          ".d" sfx,  pre, post); \
+        break; \
+    default: \
+        STATIC_ASSERT_UNREACHABLE(); \
+    } \
+    ret__; \
+})
+
+#define xchg_relaxed(ptr, x) \
+({ \
+    __typeof__(*(ptr)) x_ = (x); \
+    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)), "", "", ""); \
+})
+
+#define xchg_acquire(ptr, x) \
+({ \
+    __typeof__(*(ptr)) x_ = (x); \
+    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)), \
+                                       "", "", RISCV_ACQUIRE_BARRIER); \
+})
+
+#define xchg_release(ptr, x) \
+({ \
+    __typeof__(*(ptr)) x_ = (x); \
+    (__typeof__(*(ptr)))__xchg_generic(ptr, x_, sizeof(*(ptr)),\
+                                       "", RISCV_RELEASE_BARRIER, ""); \
+})
+
+#define xchg(ptr, x) __xchg_generic(ptr, (unsigned long)(x), sizeof(*(ptr)), \
+                                    ".aqrl", "", "")
+
+#define __generic_cmpxchg(ptr, old, new, ret, lr_sfx, sc_sfx, pre, post)	\
+ ({ \
+    register unsigned int rc; \
+    asm volatile( \
+        pre \
+        "0: lr" lr_sfx " %0, %2\n" \
+        "   bne  %0, %z3, 1f\n" \
+        "   sc" sc_sfx " %1, %z4, %2\n" \
+        "   bnez %1, 0b\n" \
+        post \
+        "1:\n" \
+        : "=&r" (ret), "=&r" (rc), "+A" (*ptr) \
+        : "rJ" (old), "rJ" (new) \
+        : "memory"); \
+ })
+
+/*
+ * For LR and SC, the A extension requires that the address held in rs1 be
+ * naturally aligned to the size of the operand (i.e., eight-byte aligned
+ * for 64-bit words and four-byte aligned for 32-bit words).
+ * If the address is not naturally aligned, an address-misaligned exception
+ * or an access-fault exception will be generated.
+ * 
+ * Thereby:
+ * - for 1-byte xchg access the containing word by clearing low two bits
+ * - for 2-byte xchg ccess the containing word by clearing first bit.
+ * 
+ * If resulting 4-byte access is still misalgined, it will fault just as
+ * non-emulated 4-byte access would.
+ *
+ * old_val was casted to unsigned long at the end of the define because of
+ * the following issue:
+ * ./arch/riscv/include/asm/cmpxchg.h:166:5: error: cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
+ * 166 |     (__typeof__(*(ptr)))(old_val >> new_val_pos); \
+ *     |     ^
+ * ./arch/riscv/include/asm/cmpxchg.h:184:17: note: in expansion of macro 'emulate_cmpxchg_1_2'
+ * 184 |         ret__ = emulate_cmpxchg_1_2(ptr, old, new, \
+ *     |                 ^~~~~~~~~~~~~~~~~~~
+ * ./arch/riscv/include/asm/cmpxchg.h:227:5: note: in expansion of macro '__cmpxchg_generic'
+ * 227 |     __cmpxchg_generic(ptr, (unsigned long)(o), (unsigned long)(n), \
+ *     |     ^~~~~~~~~~~~~~~~~
+ * ./include/xen/lib.h:141:26: note: in expansion of macro '__cmpxchg'
+ * 141 |     ((__typeof__(*(ptr)))__cmpxchg(ptr, (unsigned long)o_,              \
+ *     |                          ^~~~~~~~~
+ * common/event_channel.c:109:13: note: in expansion of macro 'cmpxchgptr'
+ * 109 |             cmpxchgptr(&xen_consumers[i], NULL, fn);
+ */
+#define emulate_cmpxchg_1_2(ptr, old, new, sc_sfx, pre, post) \
+({ \
+    uint32_t *aligned_ptr = (uint32_t *)((unsigned long)ptr & ~(0x4 - sizeof(*ptr))); \
+    uint8_t new_val_pos = ((unsigned long)(ptr) & (0x4 - sizeof(*ptr))) * BITS_PER_BYTE; \
+    unsigned long mask = GENMASK(((sizeof(*ptr)) * BITS_PER_BYTE) - 1, 0) << new_val_pos; \
+    unsigned int old_ = old << new_val_pos; \
+    unsigned int new_ = new << new_val_pos; \
+    unsigned int old_val; \
+    unsigned int xchged_val; \
+    \
+    __asm__ __volatile__ ( \
+        pre \
+        "0: lr.w %[op_xchged_val], %[op_aligned_ptr]\n" \
+        "   and  %[op_oldval], %[op_xchged_val], %z[op_mask]\n" \
+        "   bne  %[op_oldval], %z[op_old], 1f\n" \
+        "   xor  %[op_xchged_val], %[op_oldval], %[op_xchged_val]\n" \
+        "   or   %[op_xchged_val], %[op_xchged_val], %z[op_new]\n" \
+        "   sc.w" sc_sfx " %[op_xchged_val], %[op_xchged_val], %[op_aligned_ptr]\n" \
+        "   bnez %[op_xchged_val], 0b\n" \
+        post \
+        "1:\n" \
+        : [op_oldval] "=&r" (old_val), [op_xchged_val] "=&r" (xchged_val), [op_aligned_ptr] "+A" (*aligned_ptr) \
+        : [op_old] "rJ" (old_), [op_new] "rJ" (new_), \
+          [op_mask] "rJ" (mask) \
+        : "memory" ); \
+    \
+    (__typeof__(*(ptr)))((unsigned long)old_val >> new_val_pos); \
+})
+
+/*
+ * Atomic compare and exchange.  Compare OLD with MEM, if identical,
+ * store NEW in MEM.  Return the initial value in MEM.  Success is
+ * indicated by comparing RETURN with OLD.
+ */
+#define __cmpxchg_generic(ptr, old, new, size, sc_sfx, pre, post) \
+({ \
+    __typeof__(ptr) ptr__ = (ptr); \
+    __typeof__(*(ptr)) old__ = (__typeof__(*(ptr)))(old); \
+    __typeof__(*(ptr)) new__ = (__typeof__(*(ptr)))(new); \
+    __typeof__(*(ptr)) ret__; \
+    switch ( size ) \
+    { \
+    case 1: \
+    case 2: \
+        ret__ = emulate_cmpxchg_1_2(ptr, old, new, \
+                            sc_sfx, pre, post); \
+        break; \
+    case 4: \
+        __generic_cmpxchg(ptr__, old__, new__, ret__, \
+                          ".w", ".w"sc_sfx, pre, post); \
+        break; \
+    case 8: \
+        __generic_cmpxchg(ptr__, old__, new__, ret__, \
+                          ".d", ".d"sc_sfx, pre, post); \
+        break; \
+    default: \
+        STATIC_ASSERT_UNREACHABLE(); \
+    } \
+    ret__; \
+})
+
+#define cmpxchg_relaxed(ptr, o, n) \
+({ \
+    __typeof__(*(ptr)) o_ = (o); \
+    __typeof__(*(ptr)) n_ = (n); \
+    (__typeof__(*(ptr)))__cmpxchg_generic(ptr, \
+                    o_, n_, sizeof(*(ptr)), "", "", ""); \
+})
+
+#define cmpxchg_acquire(ptr, o, n) \
+({ \
+    __typeof__(*(ptr)) o_ = (o); \
+    __typeof__(*(ptr)) n_ = (n); \
+    (__typeof__(*(ptr)))__cmpxchg_generic(ptr, o_, n_, sizeof(*(ptr)), \
+                                          "", "", RISCV_ACQUIRE_BARRIER); \
+})
+
+#define cmpxchg_release(ptr, o, n) \
+({ \
+    __typeof__(*(ptr)) o_ = (o); \
+    __typeof__(*(ptr)) n_ = (n); \
+    (__typeof__(*(ptr)))__cmpxchg_release(ptr, o_, n_, sizeof(*(ptr)), \
+                                          "", RISCV_RELEASE_BARRIER, ""); \
+})
+
+#define __cmpxchg(ptr, o, n, s) \
+    (__typeof__(*(ptr))) \
+    __cmpxchg_generic(ptr, (unsigned long)(o), (unsigned long)(n), \
+                      s, ".rl", "", RISCV_FULL_BARRIER)
+
+#define cmpxchg(ptr, o, n) __cmpxchg(ptr, o, n, sizeof(*(ptr)))
+
+#endif /* _ASM_RISCV_CMPXCHG_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685702.1066928 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reews-0002oj-U0; Mon, 26 Feb 2024 17:39:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685702.1066928; Mon, 26 Feb 2024 17:39:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reews-0002nU-H3; Mon, 26 Feb 2024 17:39:30 +0000
Received: by outflank-mailman (input) for mailman id 685702;
 Mon, 26 Feb 2024 17:39:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewq-0007o5-Bj
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:28 +0000
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com
 [2a00:1450:4864:20::22f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fc939793-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:26 +0100 (CET)
Received: by mail-lj1-x22f.google.com with SMTP id
 38308e7fff4ca-2d29111272eso8909541fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:26 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.24
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fc939793-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969165; x=1709573965; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=tKStfHZFEkzLaa1NeaeyMF3odmESAg9+t6rwdA0aLto=;
        b=K477ykmKO5tphm1r2rRytAMIQh6WCRrerI/iaZgKgDoi5cX7X6xXe0Zuu9fDxT5E/E
         0bVeDnhmjdtCbJYT5q0cV0BdNgOEDmihxDWmJje2zIRIp+LVKtOToflB0VEpJUecdmyu
         NCCKkYcRahQ8ItWNFj1unaC3M77ykvk5j7GJNTp5jInpFnlfW3Tz4aApoGyNMkhKhDaJ
         c7M3ngGHtRugUcavAzb/bDQ6NwnShMD3kT4TCciLasP/2OOli3Nfai/nrll6Y3bn5hjq
         4ZqMcW5y//iNmuRQla6pzpzXahK1PrENW47nFrSHYAlkQHL3QFCXlur3vvhbWzgNAMJ3
         CqIw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969165; x=1709573965;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=tKStfHZFEkzLaa1NeaeyMF3odmESAg9+t6rwdA0aLto=;
        b=ADgV0ELmLDklMDcNaPk+raPnoe+D3Tb4GJZze8jPwQvcFKbOIzgzYRlx6Me1yAUS1w
         Uwy3WPJEBHeDd58Ay8cFphAtt50Lln1NkBY3EPjh9+k3HDgtqLG4yXQhs/12a/ITRnQv
         EXNXvVdJXvFBv/WUTe+pk0CxJGuhhm9YyboHPZrp8yoyV/fBeKdRvYbRe56fkfKMNYkt
         SciPKOocuTtluQNnQgdYSQ8uZImFeT7VcHNYL/eLrHFhEb2FSeviXerlQnO+D3AsEuDz
         9owe1gaR10b1FtsGn9bYZPZ31LU74Bg/eC52Mw8/Xc3+WURKDjHCc7wC/WbB7VdrhGyf
         N+fA==
X-Gm-Message-State: AOJu0YxcSNVw37fQIkgpk9ZcxVcav60XD9rrOfBbCv6Z73SwlwX0QFPG
	rskZJVtXGnVHxFRaXrp4mxmtVIwFAegvkIcTdkWBzFatQYE/arZ7wxIwZ7vv
X-Google-Smtp-Source: AGHT+IF8j64m05lU0O7eEs9CTJOqRPca6Py6Y/cfMXmv+mN7jFGosdpTMOiLH6kiqBIixDV6f3aLsA==
X-Received: by 2002:a2e:b8c7:0:b0:2d2:4ee4:c8a5 with SMTP id s7-20020a2eb8c7000000b002d24ee4c8a5mr5110208ljp.50.1708969165395;
        Mon, 26 Feb 2024 09:39:25 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 12/23] xen/riscv: introduce io.h
Date: Mon, 26 Feb 2024 18:38:54 +0100
Message-ID: <dd7c95b5197dfd0cca0edf9c0ada631336eb60d7.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The header taken form Linux 6.4.0-rc1 and is based on
arch/riscv/include/asm/mmio.h with the following changes:
- drop forcing of endianess for read*(), write*() functions as
  no matter what CPU endianness, what endianness a particular device
  (and hence its MMIO region(s)) is using is entirely independent.
  Hence conversion, where necessary, needs to occur at a layer up.
  Another one reason to drop endianess conversion here is:
  https://patchwork.kernel.org/project/linux-riscv/patch/20190411115623.5749-3-hch@lst.de/
  One of the answers of the author of the commit:
    And we don't know if Linux will be around if that ever changes.
    The point is:
     a) the current RISC-V spec is LE only
     b) the current linux port is LE only except for this little bit
    There is no point in leaving just this bitrotting code around.  It
    just confuses developers, (very very slightly) slows down compiles
    and will bitrot.  It also won't be any significant help to a future
    developer down the road doing a hypothetical BE RISC-V Linux port.
- drop unused argument of __io_ar() macros.
- drop "#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}"
  as they are unnessary.
- Adopt the Xen code style for this header, considering that significant changes
  are not anticipated in the future.
  In the event of any issues, adapting them to Xen style should be easily
  manageable.
- drop unnessary __r variables in macros read*_cpu()

Addionally, to the header was added definions of ioremap_*().

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5:
 - Xen code style related fixes
 - drop #define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}
 - drop cpu_to_le16()
 - remove unuused argument in _io_ar()
 - update the commit message 
 - drop unnessary __r variables in macros read*_cpu()
 - update the comments at the top of the header.
---
Changes in V4:
 - delete inner parentheses in macros.
 - s/u<N>/uint<N>.
---
Changes in V3:
 - re-sync with linux kernel
 - update the commit message
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/io.h | 157 ++++++++++++++++++++++++++++++++
 1 file changed, 157 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/io.h

diff --git a/xen/arch/riscv/include/asm/io.h b/xen/arch/riscv/include/asm/io.h
new file mode 100644
index 0000000000..95a459432c
--- /dev/null
+++ b/xen/arch/riscv/include/asm/io.h
@@ -0,0 +1,157 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ *  The header taken form Linux 6.4.0-rc1 and is based on
+ *  arch/riscv/include/asm/mmio.h with the following changes:
+ *   - drop forcing of endianess for read*(), write*() functions as
+ *     no matter what CPU endianness, what endianness a particular device
+ *     (and hence its MMIO region(s)) is using is entirely independent.
+ *     Hence conversion, where necessary, needs to occur at a layer up.
+ *     Another one reason to drop endianess conversion is:
+ *     https://patchwork.kernel.org/project/linux-riscv/patch/20190411115623.5749-3-hch@lst.de/
+ *     One of the answers of the author of the commit:
+ *       And we don't know if Linux will be around if that ever changes.
+ *       The point is:
+ *        a) the current RISC-V spec is LE only
+ *        b) the current linux port is LE only except for this little bit
+ *       There is no point in leaving just this bitrotting code around.  It
+ *       just confuses developers, (very very slightly) slows down compiles
+  *      and will bitrot.  It also won't be any significant help to a future
+ *       developer down the road doing a hypothetical BE RISC-V Linux port.
+ *   - drop unused argument of __io_ar() macros.
+ *   - drop "#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}"
+ *     as they are unnessary.
+ *   - Adopt the Xen code style for this header, considering that significant changes
+ *     are not anticipated in the future.
+ *     In the event of any issues, adapting them to Xen style should be easily
+ *     manageable.
+ *   - drop unnessary __r variables in macros read*_cpu()
+ *
+ * Copyright (C) 1996-2000 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ * Copyright (C) 2014 Regents of the University of California
+ * Copyright (C) 2024 Vates
+ */
+
+#ifndef _ASM_RISCV_IO_H
+#define _ASM_RISCV_IO_H
+
+#include <asm/byteorder.h>
+
+/*
+ * The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't
+ * change the properties of memory regions.  This should be fixed by the
+ * upcoming platform spec.
+ */
+#define ioremap_nocache(addr, size) ioremap(addr, size)
+#define ioremap_wc(addr, size) ioremap(addr, size)
+#define ioremap_wt(addr, size) ioremap(addr, size)
+
+/* Generic IO read/write.  These perform native-endian accesses. */
+static inline void __raw_writeb(uint8_t val, volatile void __iomem *addr)
+{
+    asm volatile ( "sb %0, 0(%1)" : : "r" (val), "r" (addr) );
+}
+
+static inline void __raw_writew(uint16_t val, volatile void __iomem *addr)
+{
+    asm volatile ( "sh %0, 0(%1)" : : "r" (val), "r" (addr) );
+}
+
+static inline void __raw_writel(uint32_t val, volatile void __iomem *addr)
+{
+    asm volatile ( "sw %0, 0(%1)" : : "r" (val), "r" (addr) );
+}
+
+#ifdef CONFIG_64BIT
+static inline void __raw_writeq(u64 val, volatile void __iomem *addr)
+{
+    asm volatile ( "sd %0, 0(%1)" : : "r" (val), "r" (addr) );
+}
+#endif
+
+static inline uint8_t __raw_readb(const volatile void __iomem *addr)
+{
+    uint8_t val;
+
+    asm volatile ( "lb %0, 0(%1)" : "=r" (val) : "r" (addr) );
+    return val;
+}
+
+static inline uint16_t __raw_readw(const volatile void __iomem *addr)
+{
+    uint16_t val;
+
+    asm volatile ( "lh %0, 0(%1)" : "=r" (val) : "r" (addr) );
+    return val;
+}
+
+static inline uint32_t __raw_readl(const volatile void __iomem *addr)
+{
+    uint32_t val;
+
+    asm volatile ( "lw %0, 0(%1)" : "=r" (val) : "r" (addr) );
+    return val;
+}
+
+#ifdef CONFIG_64BIT
+static inline u64 __raw_readq(const volatile void __iomem *addr)
+{
+    u64 val;
+
+    asm volatile ( "ld %0, 0(%1)" : "=r" (val) : "r" (addr) );
+    return val;
+}
+#endif
+
+/*
+ * Unordered I/O memory access primitives.  These are even more relaxed than
+ * the relaxed versions, as they don't even order accesses between successive
+ * operations to the I/O regions.
+ */
+#define readb_cpu(c)        __raw_readb(c)
+#define readw_cpu(c)        __raw_readw(c)
+#define readl_cpu(c)        __raw_readl(c)
+
+#define writeb_cpu(v, c)    __raw_writeb(v, c)
+#define writew_cpu(v, c)    __raw_writew(v, c)
+#define writel_cpu(v, c)    __raw_writel(v, c)
+
+#ifdef CONFIG_64BIT
+#define readq_cpu(c)        __raw_readq(c)
+#define writeq_cpu(v, c)    __raw_writeq(v, c)
+#endif
+
+/*
+ * I/O memory access primitives. Reads are ordered relative to any
+ * following Normal memory access. Writes are ordered relative to any prior
+ * Normal memory access.  The memory barriers here are necessary as RISC-V
+ * doesn't define any ordering between the memory space and the I/O space.
+ */
+#define __io_br()   do { } while (0)
+#define __io_ar()   asm volatile ( "fence i,r" : : : "memory" );
+#define __io_bw()   asm volatile ( "fence w,o" : : : "memory" );
+#define __io_aw()   do { } while (0)
+
+#define readb(c)    ({ uint8_t  v; __io_br(); v = readb_cpu(c); __io_ar(); v; })
+#define readw(c)    ({ uint16_t v; __io_br(); v = readw_cpu(c); __io_ar(); v; })
+#define readl(c)    ({ uint32_t v; __io_br(); v = readl_cpu(c); __io_ar(); v; })
+
+#define writeb(v, c)    ({ __io_bw(); writeb_cpu(v, c); __io_aw(); })
+#define writew(v, c)    ({ __io_bw(); writew_cpu(v, c); __io_aw(); })
+#define writel(v, c)    ({ __io_bw(); writel_cpu(v, c); __io_aw(); })
+
+#ifdef CONFIG_64BIT
+#define readq(c)        ({ uint64_t v; __io_br(); v = readq_cpu(c); __io_ar(); v; })
+#define writeq(v, c)    ({ __io_bw(); writeq_cpu(v, c); __io_aw(); })
+#endif
+
+#endif /* _ASM_RISCV_IO_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685703.1066936 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewv-0003Nb-48; Mon, 26 Feb 2024 17:39:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685703.1066936; Mon, 26 Feb 2024 17:39:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewu-0003Mq-TZ; Mon, 26 Feb 2024 17:39:32 +0000
Received: by outflank-mailman (input) for mailman id 685703;
 Mon, 26 Feb 2024 17:39:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reews-0007pd-4H
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:30 +0000
Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com
 [2a00:1450:4864:20::22b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fe6b737e-d4cd-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 18:39:29 +0100 (CET)
Received: by mail-lj1-x22b.google.com with SMTP id
 38308e7fff4ca-2d21a68dd3bso45216901fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:29 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fe6b737e-d4cd-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969169; x=1709573969; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=aoDhsePwwepX/r5KY2WQ6UW+JdGX15fzkDHjAkqmcRc=;
        b=bZbt/OTLWtcB9bi1/HajYiz6rpc4h+T00s1ludaBa16/Jj4gH6c+UfmmdR7xa/nG9g
         bNfwTRTxYODpgaiH7/fJXsZjFIUxY30IW/OAED+TyH5TXTRDxTMgZvybfYTZdr3vE5A9
         QRLRMlXknHX0i7A9Mqcyw+0UQodGj3rpwxobRJvvxiP13c62MLNsGCPoVsFvRMvrb8XY
         MkdgZvUSVVAFaqttgI5jbxGJOS+ro9H31zeRSh5kfTm3hQCnvL3qXpSOF+ZoFjLICIXv
         qpDCleEshq2NH5sLk9/4dwNM7ffdwiays3CRlfEYkwEbvjHZr3f5VIhFBelmdz6KgB5a
         /VxA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969169; x=1709573969;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=aoDhsePwwepX/r5KY2WQ6UW+JdGX15fzkDHjAkqmcRc=;
        b=rrO5+t4sc63ZrefTqKpql4AesUHgUo4VBpAWvFwckn3zIIKHGHmlniYqcXXtO7ty1Y
         jVzZGkG7kVs7JmUAyBPExMraJ011cfFbhT7DjyKTxUlKOTQxKDx7prSPbP5l0A32myRm
         96y2Y9h8PulxDMLVMroppjJ/JkTlhOeaadypfeLjtN8Y87mUPs7YijCMZCaalQKG0uoF
         2cPq6GgQ02tTbNiqdcAgHZdMs41ludMbwgMbZuBsI3dKwmBK1hwoIuAqXb+XwqIoDE5f
         lpwXVjh/YrRBC/NxC4/qYUPmM418kK4xyo1rUr+PN+su9YYEuhEWMFeZbohfLszxkX0g
         tsQg==
X-Gm-Message-State: AOJu0YzX16N8ZpC6EO0dxqElJLu6Sm3QmtRrhc7RLjztq6BVt2yO2qUT
	EA0tXCaS99hPk6uyIkqqwllpX1reEP8w7P/mj8mVJFq+TqHEYg11MCL4T6LP
X-Google-Smtp-Source: AGHT+IGurxaq8CewGzRaeGMOflWCYSSwy2LYoKIQHP54CIvRNl7MBbPlrKlyP0rswHRCHwOHQQyptQ==
X-Received: by 2002:a2e:a498:0:b0:2d2:38ff:8b6 with SMTP id h24-20020a2ea498000000b002d238ff08b6mr4136045lji.49.1708969169135;
        Mon, 26 Feb 2024 09:39:29 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 15/23] xen/riscv: add definition of __read_mostly
Date: Mon, 26 Feb 2024 18:38:57 +0100
Message-ID: <4e089eff445421192fcedc5f04fd6cee25090677.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The definition of __read_mostly should be removed in:
https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/

The patch introduces it in arch-specific header to not
block enabling of full Xen build for RISC-V.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
- [PATCH] move __read_mostly to xen/cache.h  [2]

Right now, the patch series doesn't have a direct dependency on [2] and it
provides __read_mostly in the patch:
    [PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the
final version of the patch [2] is provided.

[2] https://lore.kernel.org/xen-devel/f25eb5c9-7c14-6e23-8535-2c66772b333e@suse.com/
---
Changes in V4-V6:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/cache.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/riscv/include/asm/cache.h b/xen/arch/riscv/include/asm/cache.h
index 69573eb051..94bd94db53 100644
--- a/xen/arch/riscv/include/asm/cache.h
+++ b/xen/arch/riscv/include/asm/cache.h
@@ -3,4 +3,6 @@
 #ifndef _ASM_RISCV_CACHE_H
 #define _ASM_RISCV_CACHE_H
 
+#define __read_mostly __section(".data.read_mostly")
+
 #endif /* _ASM_RISCV_CACHE_H */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685704.1066944 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reeww-0003WP-9D; Mon, 26 Feb 2024 17:39:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685704.1066944; Mon, 26 Feb 2024 17:39:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewv-0003TN-Iu; Mon, 26 Feb 2024 17:39:33 +0000
Received: by outflank-mailman (input) for mailman id 685704;
 Mon, 26 Feb 2024 17:39:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewt-0007pd-8r
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:31 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ff0b1b4b-d4cd-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 18:39:30 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d220e39907so54921291fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:30 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff0b1b4b-d4cd-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969170; x=1709573970; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Tb07bjP5MT5URv/T6UfEIfVo4wn4HOEqDnf1WKrmicA=;
        b=dB3Xa/tesInuzjYOh7NNr804nKk56Y06LxnsSHzAssXH+r/uQ75rSXRvhYishB/wPm
         Yp57tUmTK25mBVbbydn0EXJMTS0iwe8C56Mp7lYFPtuvJ4ZugZ7I1SOw0bMB+FmgwaO1
         dSj+uzXPlB8uegjaYvY8p+5KXoGWaJ5ThpKdv5pcvTalXCil2Hydqa7NP2nT+Dj5T8Et
         XqKkqZfKkfKqMMKi/ir583KJc+Ut0P4HUlcMbx+DEApQpWmUkhP064Ssnb3WI25RuBQo
         Fr5593N4Y2RIO2EGxbMB5NJiYc4T35cM4h3xjCdlp7MBKDCc7lVcfUaXWbUMzGAqNGSc
         94cQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969170; x=1709573970;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=Tb07bjP5MT5URv/T6UfEIfVo4wn4HOEqDnf1WKrmicA=;
        b=ObCBA4e8aAXgI5YeYOB43BP0znwT1sbdz5RgoaXro0XA71OxAB0y+0rtWpJjKa8mvL
         2C7IidvuC6Z8+H8TvgyMJQSZEWy6yNF+J53Owl5jQdzOH99cW4Yur2ZfMso+pYBHFnqA
         df36JZngCjioo3tsoaNw2k9eI3WhfsdpbFiSJFrZCSZRRzLSJ4riJWkWseyxt5dpgjPw
         /PCtiAC15KDY0qp1fPH+axnJAiz1vzk/fV5/i4ZJJUUXfS/940P+hcOfL88EztX4v6y7
         DPvwyDU+7WOeYeDvEX2PdATWDNAeHgCEixJK4S0NS6Hmcm30pdGRBL2ZsB3JULgiqJEf
         KteQ==
X-Gm-Message-State: AOJu0Yytyf3SIEVBXtgmzzJ8FF8sSNBI0qBBm3MzgUzFYTOSHe/O3VjW
	KhOqhnE2BrdEt8VWB5TjNj1rxdYjbxv+wyaEcdnkkGAb0/jtKGowNFlygYW3
X-Google-Smtp-Source: AGHT+IGuW0ldyVHoN5ow3oTKDG66b+FmJr3b8VDMZKhzGzBWZ2dvIriEHzA+7YRr9PKsNjEnYncSoQ==
X-Received: by 2002:a05:651c:224:b0:2d2:5123:4d03 with SMTP id z4-20020a05651c022400b002d251234d03mr5091015ljn.22.1708969170163;
        Mon, 26 Feb 2024 09:39:30 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 16/23] xen/riscv: add required things to current.h
Date: Mon, 26 Feb 2024 18:38:58 +0100
Message-ID: <9063614bd46592b323e7bde3c6be6727eb922312.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Add minimal requied things to be able to build full Xen.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5:
 - Nothing changed. Only rebase.
---
Changes in V4:
 - BUG() was changed to BUG_ON("unimplemented");
 - Change "xen/bug.h" to "xen/lib.h" as BUG_ON is defined in xen/lib.h.
 - Add Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V3:
 - add SPDX
 - drop a forward declaration of struct vcpu;
 - update guest_cpu_user_regs() macros
 - replace get_processor_id with smp_processor_id
 - update the commit message
 - code style fixes
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/current.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/xen/arch/riscv/include/asm/current.h b/xen/arch/riscv/include/asm/current.h
index d84f15dc50..aedb6dc732 100644
--- a/xen/arch/riscv/include/asm/current.h
+++ b/xen/arch/riscv/include/asm/current.h
@@ -3,6 +3,21 @@
 #ifndef __ASM_CURRENT_H
 #define __ASM_CURRENT_H
 
+#include <xen/lib.h>
+#include <xen/percpu.h>
+#include <asm/processor.h>
+
+#ifndef __ASSEMBLY__
+
+/* Which VCPU is "current" on this PCPU. */
+DECLARE_PER_CPU(struct vcpu *, curr_vcpu);
+
+#define current            this_cpu(curr_vcpu)
+#define set_current(vcpu)  do { current = (vcpu); } while (0)
+#define get_cpu_current(cpu)  per_cpu(curr_vcpu, cpu)
+
+#define guest_cpu_user_regs() ({ BUG_ON("unimplemented"); NULL; })
+
 #define switch_stack_and_jump(stack, fn) do {               \
     asm volatile (                                          \
             "mv sp, %0\n"                                   \
@@ -10,4 +25,8 @@
     unreachable();                                          \
 } while ( false )
 
+#define get_per_cpu_offset() __per_cpu_offset[smp_processor_id()]
+
+#endif /* __ASSEMBLY__ */
+
 #endif /* __ASM_CURRENT_H */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685706.1066949 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewx-0003xt-Mv; Mon, 26 Feb 2024 17:39:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685706.1066949; Mon, 26 Feb 2024 17:39:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reewx-0003vl-CI; Mon, 26 Feb 2024 17:39:35 +0000
Received: by outflank-mailman (input) for mailman id 685706;
 Mon, 26 Feb 2024 17:39:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewv-0007pd-5F
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:33 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fff24c6b-d4cd-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 18:39:32 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d29111272eso8910981fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:32 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.30
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fff24c6b-d4cd-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969171; x=1709573971; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=9b077YYDybYOLF6nhirG91sM1YPk52VLw2WhqDJoTFw=;
        b=ijh3xok3akSCLVg4cU/MXGrI/auLteGf0gukJGmG2KwbRx1ZUBOgw5A50F/f5szUjN
         LjLjR6n+iHRo/jjkWUz7s16VSzqkqbFO+phtsC19lTn87KaV6fYZXavUNdtaKOQrRva8
         K/wC6z3iKiKk6ebG8UWYSTT9VOyT/ve+kZ4dupB6xRggE+XnGn9YQPHJzijy/diaZt8G
         RUr1DQaR1OgosvGgOt0E809jS+tbv14Zttr3Fi+Iu9TRiPOOB6VeDIAm2UYBHyhUZfNP
         aBR/xuJN8W5M+qVT4IGUohqegizcQCtz6Q2xayvNln5IoFXrOX6e0Ly268oisJHbKrmP
         T5eQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969171; x=1709573971;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=9b077YYDybYOLF6nhirG91sM1YPk52VLw2WhqDJoTFw=;
        b=XrgqekOL4cb7REZlyYNhAhGqmoxp/Btz/NIkNrGcewqVGZN7hBdd0X2E3ZpHNweu7l
         UctFgzJmeT5l5A6WRYHWSzaCqWMLXecWRPnvuA7V9Afr/vBvn3h2Qu5ti2+wqOm6uyUs
         rAoyfSB4G3qjUJCISpbVE6z0LNBVYPhd+1ETWeAmxDNSJgYNXnEp+l7qyENx3Rf8kMFb
         hMlMSPx5US1JYGeqfjowD2yq8SuYC6gks0xokhj/hMkV8lzCngAMHMhbOYFU3pB/YDIE
         zqSt9WMZH5DTRjHogL4GsHOiyhajWh7KXwDexBnK4ChhZ5to1y2JxZL1Uvza1LB+Fo4T
         tj8g==
X-Gm-Message-State: AOJu0YyZ89sKIj2H35LDopc0tIKCyu67dYU4ER4dgF0s78ZmBSN0OKLC
	6cBqfh2iMi+oumedXhHZ+AeM3jVzmddMJQ3DqjRI6Jb6oeSRjcWl/Th6uu6b
X-Google-Smtp-Source: AGHT+IExD9nzqU4wd2o0hcsdcr2pP41prdAcVvY7g24eNfXaFQKHUfBCLoBlJhVtxI2wDJcq5z5S6A==
X-Received: by 2002:a2e:8507:0:b0:2d2:3e88:7c4f with SMTP id j7-20020a2e8507000000b002d23e887c4fmr4362942lji.43.1708969171177;
        Mon, 26 Feb 2024 09:39:31 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 17/23] xen/riscv: add minimal stuff to page.h to build full Xen
Date: Mon, 26 Feb 2024 18:38:59 +0100
Message-ID: <ee58261d95c60f98527026f157eec5d73c4f0959.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5:
 - Nothing changed. Only rebase.
---
Changes in V4:
---
 - Change message -> subject in "Changes in V3"
 - s/BUG/BUG_ON("...")
 - Do proper rebase ( pfn_to_paddr() and paddr_to_pfn() aren't removed ).
---
Changes in V3:
 - update the commit subject
 - add implemetation of PAGE_HYPERVISOR macros
 - add Acked-by: Jan Beulich <jbeulich@suse.com>
 - drop definition of pfn_to_addr, and paddr_to_pfn in <asm/mm.h>
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/include/asm/page.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/xen/arch/riscv/include/asm/page.h b/xen/arch/riscv/include/asm/page.h
index 95074e29b3..c831e16417 100644
--- a/xen/arch/riscv/include/asm/page.h
+++ b/xen/arch/riscv/include/asm/page.h
@@ -6,6 +6,7 @@
 #ifndef __ASSEMBLY__
 
 #include <xen/const.h>
+#include <xen/bug.h>
 #include <xen/types.h>
 
 #include <asm/mm.h>
@@ -32,6 +33,10 @@
 #define PTE_LEAF_DEFAULT            (PTE_VALID | PTE_READABLE | PTE_WRITABLE)
 #define PTE_TABLE                   (PTE_VALID)
 
+#define PAGE_HYPERVISOR_RW          (PTE_VALID | PTE_READABLE | PTE_WRITABLE)
+
+#define PAGE_HYPERVISOR             PAGE_HYPERVISOR_RW
+
 /* Calculate the offsets into the pagetables for a given VA */
 #define pt_linear_offset(lvl, va)   ((va) >> XEN_PT_LEVEL_SHIFT(lvl))
 
@@ -62,6 +67,20 @@ static inline bool pte_is_valid(pte_t p)
     return p.pte & PTE_VALID;
 }
 
+static inline void invalidate_icache(void)
+{
+    BUG_ON("unimplemented");
+}
+
+#define clear_page(page) memset((void *)(page), 0, PAGE_SIZE)
+#define copy_page(dp, sp) memcpy(dp, sp, PAGE_SIZE)
+
+/* TODO: Flush the dcache for an entire page. */
+static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache)
+{
+    BUG_ON("unimplemented");
+}
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_RISCV_PAGE_H */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685708.1066963 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reex0-0004or-J0; Mon, 26 Feb 2024 17:39:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685708.1066963; Mon, 26 Feb 2024 17:39:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reex0-0004le-4g; Mon, 26 Feb 2024 17:39:38 +0000
Received: by outflank-mailman (input) for mailman id 685708;
 Mon, 26 Feb 2024 17:39:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewx-0007pd-5o
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:35 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 010a9754-d4ce-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 18:39:34 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d09cf00214so52747141fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:34 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 010a9754-d4ce-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969173; x=1709573973; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=bX4BHdNJNDukdQPM6DZY3F0YnW+xw9T4wfmHKLd07cI=;
        b=af+RPiRKwqxL62dZnGgYva2vRdebEjTsd9doxB1H5dJC9Wsj1rkyR+Ud/YFzXr6Qa7
         x7cKhUpMwHi2XXTbhoNhlKLdzCIhmVng44TbLi40455qFbdQE84HexMAfT3AX6RnRi4C
         fRjLSvwzySutHEh0QhchQduhjgMl3EsywX+hrs3GxvcmufnDEWD0eXLqVO6Wwgfu72gL
         Cx1NaMrvTot6QuvFsH7aKaFcYrFhprL91aKRKaw5Vver0SD+W+gh54b6eip8451hohXo
         Nxj4JV5b7s1TDJcrMfqcpuVb5ZfjBtpf2rQcnN5NXxDfeZ/oHIxKEwRUyKym6M7ZQONx
         JREA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969173; x=1709573973;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=bX4BHdNJNDukdQPM6DZY3F0YnW+xw9T4wfmHKLd07cI=;
        b=OtU+aufgrCf0aJD0XN0iNk7yfZsP4tR/8wwgdc6K9bu29UJrE6dKCxr6QY9e1hGbJE
         1zi6w5DSN3yBDORFAw2Byuxx7bN8g6dmp5FKRLQ4se+kD/w4SbE4DEIzL057UQNhTBxW
         4az2ejZ/zHw5DGC5zZRWpG7qrk2YnKTy9jC3ydBYa4DzQ0jty5wEH1uMKUIl5Oad1Y3a
         Ke1N2uU/XGQHWiBgoc8xYJNiubBKMwfuqfBDqoXG8I2AG3DsSKNU8Bz15gnF1EW9KdG8
         21+G+TFx3hg9U7WPno09brZkxwb/Tl+kyNzEubCIEbqaqsfhWxrny2yz6nG3dv/lpXew
         BdKA==
X-Gm-Message-State: AOJu0YwNRW+RlemNtzedKJ4Le8Mhu9mxSnlJiE+hTmofyKhei0cuKXqs
	NujmS7MGI/icFZXVhpRbaawMaraFMeu24o55g+1450bu5UcR9839E+VUepcv
X-Google-Smtp-Source: AGHT+IHpgtqEx8CRyxZf9Vbd76Uot9LETD0MPHRKWefXtusBVmjqx9sL/ix9P2FyNRF4bjkDHs4OAA==
X-Received: by 2002:a05:651c:10a7:b0:2d2:724d:f651 with SMTP id k7-20020a05651c10a700b002d2724df651mr4473277ljn.38.1708969173405;
        Mon, 26 Feb 2024 09:39:33 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 19/23] xen/riscv: add minimal stuff to mm.h to build full Xen
Date: Mon, 26 Feb 2024 18:39:01 +0100
Message-ID: <7f5d859b4dbc0593447ade0b221ece17eda68d1e.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5:
 - update the comment around "struct domain *domain;" : zero -> NULL
 - fix ident. for unsigned long val;
 - put page_to_virt() and virt_to_page() close to each other.
 - drop unnessary leading underscore
 - drop a space before the comment: /* Count of uses of this frame as its current type. */
 - drop comment about a page 'not as a shadow'. it is not necessary for RISC-V
---
Changes in V4:
 - update an argument name of PFN_ORDERN macros.
 - drop pad at the end of 'struct page_info'.
 - Change message -> subject in "Changes in V3"
 - delete duplicated macros from riscv/mm.h
 - fix identation in struct page_info
 - align comment for PGC_ macros
 - update definitions of domain_set_alloc_bitsize() and domain_clamp_alloc_bitsize()
 - drop unnessary comments.
 - s/BUG/BUG_ON("...")
 - define __virt_to_maddr, __maddr_to_virt as stubs
 - add inclusion of xen/mm-frame.h for mfn_x and others
 - include "xen/mm.h" instead of "asm/mm.h" to fix compilation issues:
	 In file included from arch/riscv/setup.c:7:
	./arch/riscv/include/asm/mm.h:60:28: error: field 'list' has incomplete type
	   60 |     struct page_list_entry list;
	      |                            ^~~~
	./arch/riscv/include/asm/mm.h:81:43: error: 'MAX_ORDER' undeclared here (not in a function)
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
	      |                                           ^~~~~~~~~
	./arch/riscv/include/asm/mm.h:81:31: error: bit-field 'first_dirty' width not an integer constant
	   81 |                 unsigned long first_dirty:MAX_ORDER + 1;
 - Define __virt_to_mfn() and __mfn_to_virt() using maddr_to_mfn() and mfn_to_maddr().
---
Changes in V3:
 - update the commit title
 - introduce DIRECTMAP_VIRT_START.
 - drop changes related pfn_to_paddr() and paddr_to_pfn as they were remvoe in
   [PATCH v2 32/39] xen/riscv: add minimal stuff to asm/page.h to build full Xen
 - code style fixes.
 - drop get_page_nr  and put_page_nr as they don't need for time being
 - drop CONFIG_STATIC_MEMORY related things
 - code style fixes
---
Changes in V2:
 - define stub for arch_get_dma_bitsize(void)
---
 xen/arch/riscv/include/asm/mm.h | 246 ++++++++++++++++++++++++++++++++
 xen/arch/riscv/mm.c             |   2 +-
 xen/arch/riscv/setup.c          |   2 +-
 3 files changed, 248 insertions(+), 2 deletions(-)

diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
index 07c7a0abba..2f13c1c3c2 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -3,11 +3,252 @@
 #ifndef _ASM_RISCV_MM_H
 #define _ASM_RISCV_MM_H
 
+#include <public/xen.h>
+#include <xen/bug.h>
+#include <xen/mm-frame.h>
+#include <xen/pdx.h>
+#include <xen/types.h>
+
 #include <asm/page-bits.h>
 
 #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
 #define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
 
+#define paddr_to_pdx(pa)    mfn_to_pdx(maddr_to_mfn(pa))
+#define gfn_to_gaddr(gfn)   pfn_to_paddr(gfn_x(gfn))
+#define gaddr_to_gfn(ga)    _gfn(paddr_to_pfn(ga))
+#define mfn_to_maddr(mfn)   pfn_to_paddr(mfn_x(mfn))
+#define maddr_to_mfn(ma)    _mfn(paddr_to_pfn(ma))
+#define vmap_to_mfn(va)     maddr_to_mfn(virt_to_maddr((vaddr_t)va))
+#define vmap_to_page(va)    mfn_to_page(vmap_to_mfn(va))
+
+static inline unsigned long __virt_to_maddr(unsigned long va)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline void *__maddr_to_virt(unsigned long ma)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+#define virt_to_maddr(va) __virt_to_maddr((unsigned long)(va))
+#define maddr_to_virt(pa) __maddr_to_virt((unsigned long)(pa))
+
+/* Convert between Xen-heap virtual addresses and machine frame numbers. */
+#define __virt_to_mfn(va)  mfn_x(maddr_to_mfn(virt_to_maddr(va)))
+#define __mfn_to_virt(mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn)))
+
+/*
+ * We define non-underscored wrappers for above conversion functions.
+ * These are overriden in various source files while underscored version
+ * remain intact.
+ */
+#define virt_to_mfn(va)     __virt_to_mfn(va)
+#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+
+struct page_info
+{
+    /* Each frame can be threaded onto a doubly-linked list. */
+    struct page_list_entry list;
+
+    /* Reference count and various PGC_xxx flags and fields. */
+    unsigned long count_info;
+
+    /* Context-dependent fields follow... */
+    union {
+        /* Page is in use: ((count_info & PGC_count_mask) != 0). */
+        struct {
+            /* Type reference count and various PGT_xxx flags and fields. */
+            unsigned long type_info;
+        } inuse;
+        /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
+        union {
+            struct {
+                /*
+                 * Index of the first *possibly* unscrubbed page in the buddy.
+                 * One more bit than maximum possible order to accommodate
+                 * INVALID_DIRTY_IDX.
+                 */
+#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1)
+                unsigned long first_dirty:MAX_ORDER + 1;
+
+                /* Do TLBs need flushing for safety before next page use? */
+                bool need_tlbflush:1;
+
+#define BUDDY_NOT_SCRUBBING    0
+#define BUDDY_SCRUBBING        1
+#define BUDDY_SCRUB_ABORT      2
+                unsigned long scrub_state:2;
+            };
+
+            unsigned long val;
+        } free;
+    } u;
+
+    union {
+        /* Page is in use */
+        struct {
+            /* Owner of this page (NULL if page is anonymous). */
+            struct domain *domain;
+        } inuse;
+
+        /* Page is on a free list. */
+        struct {
+            /* Order-size of the free chunk this page is the head of. */
+            unsigned int order;
+        } free;
+    } v;
+
+    union {
+        /*
+         * Timestamp from 'TLB clock', used to avoid extra safety flushes.
+         * Only valid for: a) free pages, and b) pages with zero type count
+         */
+        uint32_t tlbflush_timestamp;
+    };
+};
+
+#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
+
+/* PDX of the first page in the frame table. */
+extern unsigned long frametable_base_pdx;
+
+/* Convert between machine frame numbers and page-info structures. */
+#define mfn_to_page(mfn)                                            \
+    (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
+#define page_to_mfn(pg)                                             \
+    pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)
+
+static inline void *page_to_virt(const struct page_info *pg)
+{
+    return mfn_to_virt(mfn_x(page_to_mfn(pg)));
+}
+
+/* Convert between Xen-heap virtual addresses and page-info structures. */
+static inline struct page_info *virt_to_page(const void *v)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+/*
+ * Common code requires get_page_type and put_page_type.
+ * We don't care about typecounts so we just do the minimum to make it
+ * happy.
+ */
+static inline int get_page_type(struct page_info *page, unsigned long type)
+{
+    return 1;
+}
+
+static inline void put_page_type(struct page_info *page)
+{
+}
+
+static inline void put_page_and_type(struct page_info *page)
+{
+    put_page_type(page);
+    put_page(page);
+}
+
+/*
+ * RISC-V does not have an M2P, but common code expects a handful of
+ * M2P-related defines and functions. Provide dummy versions of these.
+ */
+#define INVALID_M2P_ENTRY        (~0UL)
+#define SHARED_M2P_ENTRY         (~0UL - 1UL)
+#define SHARED_M2P(_e)           ((_e) == SHARED_M2P_ENTRY)
+
+#define set_gpfn_from_mfn(mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0)
+#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
+
+#define PDX_GROUP_SHIFT (PAGE_SHIFT + VPN_BITS)
+
+static inline unsigned long domain_get_maximum_gpfn(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+/*
+ * On RISCV, all the RAM is currently direct mapped in Xen.
+ * Hence return always true.
+ */
+static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
+{
+    return true;
+}
+
+#define PG_shift(idx)   (BITS_PER_LONG - (idx))
+#define PG_mask(x, idx) (x ## UL << PG_shift(idx))
+
+#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
+#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
+#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
+
+/* Count of uses of this frame as its current type. */
+#define PGT_count_width   PG_shift(2)
+#define PGT_count_mask    ((1UL << PGT_count_width) - 1)
+
+/*
+ * Page needs to be scrubbed. Since this bit can only be set on a page that is
+ * free (i.e. in PGC_state_free) we can reuse PGC_allocated bit.
+ */
+#define _PGC_need_scrub   _PGC_allocated
+#define PGC_need_scrub    PGC_allocated
+
+/* Cleared when the owning guest 'frees' this page. */
+#define _PGC_allocated    PG_shift(1)
+#define PGC_allocated     PG_mask(1, 1)
+/* Page is Xen heap? */
+#define _PGC_xen_heap     PG_shift(2)
+#define PGC_xen_heap      PG_mask(1, 2)
+/* Page is broken? */
+#define _PGC_broken       PG_shift(7)
+#define PGC_broken        PG_mask(1, 7)
+/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */
+#define PGC_state         PG_mask(3, 9)
+#define PGC_state_inuse   PG_mask(0, 9)
+#define PGC_state_offlining PG_mask(1, 9)
+#define PGC_state_offlined PG_mask(2, 9)
+#define PGC_state_free    PG_mask(3, 9)
+#define page_state_is(pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st)
+
+/* Count of references to this frame. */
+#define PGC_count_width   PG_shift(9)
+#define PGC_count_mask    ((1UL << PGC_count_width) - 1)
+
+#define _PGC_extra        PG_shift(10)
+#define PGC_extra         PG_mask(1, 10)
+
+#define is_xen_heap_page(page) ((page)->count_info & PGC_xen_heap)
+#define is_xen_heap_mfn(mfn) \
+    (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn)))
+
+#define is_xen_fixed_mfn(mfn)                                   \
+    ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) &&   \
+     (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1)))
+
+#define page_get_owner(p)    (p)->v.inuse.domain
+#define page_set_owner(p, d) ((p)->v.inuse.domain = (d))
+
+/* TODO: implement */
+#define mfn_valid(mfn) ({ (void)(mfn); 0; })
+
+#define domain_set_alloc_bitsize(d) ((void)(d))
+#define domain_clamp_alloc_bitsize(d, b) ((void)(d), (b))
+
+#define PFN_ORDER(pfn) ((pfn)->v.free.order)
+
 extern unsigned char cpu0_boot_stack[];
 
 void setup_initial_pagetables(void);
@@ -20,4 +261,9 @@ unsigned long calc_phys_offset(void);
 
 void turn_on_mmu(unsigned long ra);
 
+static inline unsigned int arch_get_dma_bitsize(void)
+{
+    return 32; /* TODO */
+}
+
 #endif /* _ASM_RISCV_MM_H */
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index 053f043a3d..fe3a43be20 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -5,12 +5,12 @@
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/macros.h>
+#include <xen/mm.h>
 #include <xen/pfn.h>
 
 #include <asm/early_printk.h>
 #include <asm/csr.h>
 #include <asm/current.h>
-#include <asm/mm.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 6593f601c1..98a94c4c48 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -2,9 +2,9 @@
 
 #include <xen/compile.h>
 #include <xen/init.h>
+#include <xen/mm.h>
 
 #include <asm/early_printk.h>
-#include <asm/mm.h>
 
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:39:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:39:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685709.1066971 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reex2-0005Br-4M; Mon, 26 Feb 2024 17:39:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685709.1066971; Mon, 26 Feb 2024 17:39:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reex1-000549-II; Mon, 26 Feb 2024 17:39:39 +0000
Received: by outflank-mailman (input) for mailman id 685709;
 Mon, 26 Feb 2024 17:39:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewy-0007pd-5s
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:36 +0000
Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com
 [2a00:1450:4864:20::236])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 01d71309-d4ce-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 18:39:35 +0100 (CET)
Received: by mail-lj1-x236.google.com with SMTP id
 38308e7fff4ca-2d240d8baf6so42591771fa.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:35 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.33
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 01d71309-d4ce-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969174; x=1709573974; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=5RsympPuEgBknw/9wTxPuYHDwH2qWezPZnr8TKX/wj4=;
        b=GhBiZmPA4LegGkMI6WGCgiHYs3cuDsk3EtTnQblQ4kHV9n3/FtYP2X/rrDWyWwcXHO
         5SjcyAMocnVVch2xVLxq6EDHL2W8/CdIPkbpINRLxH+55QNLX02qy4gDfxY1KglVY5e4
         dIQN4726227lMqWl4WBfq4NpiyK1d+cXy5x1EEb/Ao7XEVqoqb9xNW44sd9vN5fA1G8G
         8VnRRp9zxR055DYsUHyYk0odJLMy1kRe0hSIRf3U49bfy4KI8XFq7UUnzu2ZwRq91Fvk
         uKOR2TZLcGXKGnh9jH4fxBmk+Ea08k060Rfsao4qcaIWPex+zIaFqVv4FxKW378tYtn0
         h9yA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969174; x=1709573974;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=5RsympPuEgBknw/9wTxPuYHDwH2qWezPZnr8TKX/wj4=;
        b=TRaLM0BP/R8s+sC8s9xZciGQ7xfg8Fpbca8GZMNeGUU775qiR2peoD/U+nQNcCSzHF
         2jXfVcnQ8o+sncPHlCQbowm8FhSbLqri+nwrymd8+e27UZnAfdv4WHO1c3Y7Oacsan/J
         DOGdTxkhOhJDDhxl0vnWRMY01td9PQY2sqBl62Y7s0+okenWo9dGJptBAWfnz4da70x5
         V6x14cuEmJ7V9XpxcK8rJ9NZWnXVLd3otsqhbDmc4Dg7XnQDqBm8P01R5gy5anWgkLqs
         fG/b4S399HXBCttpJYxEpHzvXbpU1Od6eu+ROw5WhqSY5ExdnicPhlc54iWOBv53i4ce
         m2Uw==
X-Gm-Message-State: AOJu0YxIjd4dBWrlt83kKBWd5A61Cxwl/sANs9S380hkdF7KnCHdjkzx
	95InGvitl9aL4kDDTp0Tv9Ml1dBNKc4OemTCj6bnZR2Ni4nBPodTw0CVMJz6
X-Google-Smtp-Source: AGHT+IELBKLqzsNxDEfYRgyCkeSZDD+7XeWu4l9xHCHuL97TqolRL4k1BVXLLvSsCsFGw1fN3waNeA==
X-Received: by 2002:a2e:ba14:0:b0:2d0:b2f7:84c5 with SMTP id p20-20020a2eba14000000b002d0b2f784c5mr3819090lja.53.1708969174356;
        Mon, 26 Feb 2024 09:39:34 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>
Subject: [PATCH v5 20/23] xen/riscv: introduce vm_event_*() functions
Date: Mon, 26 Feb 2024 18:39:02 +0100
Message-ID: <e80322d7613f45fbd806e8314b530dc564f20ee5.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5:
 - Only rebase was done.
---
Changes in V4:
  - New patch.
---
 xen/arch/riscv/Makefile   |  1 +
 xen/arch/riscv/vm_event.c | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+)
 create mode 100644 xen/arch/riscv/vm_event.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 2fefe14e7c..1ed1a8369b 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
 obj-y += traps.o
+obj-y += vm_event.o
 
 $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
diff --git a/xen/arch/riscv/vm_event.c b/xen/arch/riscv/vm_event.c
new file mode 100644
index 0000000000..bb1fc73bc1
--- /dev/null
+++ b/xen/arch/riscv/vm_event.c
@@ -0,0 +1,19 @@
+#include <xen/bug.h>
+
+struct vm_event_st;
+struct vcpu;
+
+void vm_event_fill_regs(struct vm_event_st *req)
+{
+    BUG_ON("unimplemented");
+}
+
+void vm_event_set_registers(struct vcpu *v, struct vm_event_st *rsp)
+{
+    BUG_ON("unimplemented");
+}
+
+void vm_event_monitor_next_interrupt(struct vcpu *v)
+{
+    /* Not supported on RISCV. */
+}
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:49:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:49:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685730.1066990 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref6B-0002Dx-4D; Mon, 26 Feb 2024 17:49:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685730.1066990; Mon, 26 Feb 2024 17:49:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref6B-0002Dq-10; Mon, 26 Feb 2024 17:49:07 +0000
Received: by outflank-mailman (input) for mailman id 685730;
 Mon, 26 Feb 2024 17:49:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reex2-0007o5-50
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:40 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 03afc76f-d4ce-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:38 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d29111272eso8912891fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:38 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 03afc76f-d4ce-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969178; x=1709573978; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=R2XUH+C4ZG4qf/BanYnmK67T40KUDC1N5sZMm+w9Bf4=;
        b=GUEKD30ChXDxmdq/2hXdnnnVPJBCoSeK2Nn1maZvGEHVTKNRVB+KtgX3RshplxutWs
         L6/w9R/i1U/nPoCiOg+wVBLcMUq2XiMrqo4Udfy3XG6BYiEP6Nh79S48/C3SWX4qP0D4
         HN4Rhzre9eA8DYyAIeCKfsIICBV1vH5XoOqKde1hE3AP9TRR3dmk6tbGhSulJSg21mPm
         4Oe9k4n0mWs30+AB4K8KMGrJV35aBDGWi+lcjEb2lno1gVn64i8ECdtll6Oqp6H+KuW2
         tgG+IfxYH6LHbrCoUUra6sjSCOSSRJW+SGMDSMu4mBPOyST6w9vck9bxRehJ3+k1zVRh
         ZEyw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969178; x=1709573978;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=R2XUH+C4ZG4qf/BanYnmK67T40KUDC1N5sZMm+w9Bf4=;
        b=T926Zcg8r7CmjswqvfglQxE48A07EqwVtbJmP5ZvZD7I2o2wHJQGF9R0g2wevNkKuI
         kG5EOaabf/VXQw3YaWqgrMx0U3D317GtnsbqGR0TMOdA7p2mup81VnqOMS/WH1kecFbY
         nHsM6JdGFDuDSJOtiWJ43T3mSfWlQKHmyWTZAQ4yY61OWDuDAjsvGMdDn5lR5x2k73fX
         6w/oU0aj/54SmDpNa2u7Z01PC3COmLmhT9C9eUWjh+zpkqMd/SrurmPCLp0KoV32SvuQ
         NcLVj9Hr3zZV+/hPxTS8HHgnx+DzIS/yHFu4msktoQc8sHiyEqqb9Czy9J84K9CCtEYN
         jVtQ==
X-Gm-Message-State: AOJu0YzqZIux+Fq5cOStjduiBzTNNT8ga9J5leb7vtHXyxSWX5u4V18U
	GQ4HDroFL8v79BmZiI45ZN1LqgTzAg9MmcGLeFKQTp95b5yyahE2M5rv6ZPq
X-Google-Smtp-Source: AGHT+IEQjQP0gKPYEK4Porh0kK/iQ1HDsiBoHURpoxYmiVeWR7OPIxhJTMHmI8Qz+6GglUYmUeFQ8A==
X-Received: by 2002:a2e:860b:0:b0:2d2:2fe9:2896 with SMTP id a11-20020a2e860b000000b002d22fe92896mr4232359lji.42.1708969177793;
        Mon, 26 Feb 2024 09:39:37 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 23/23] xen/README: add compiler and binutils versions for RISC-V64
Date: Mon, 26 Feb 2024 18:39:05 +0100
Message-ID: <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

This patch doesn't represent a strict lower bound for GCC and
GNU Binutils; rather, these versions are specifically employed by
the Xen RISC-V container and are anticipated to undergo continuous
testing.

While it is feasible to utilize Clang, it's important to note that,
currently, there is no Xen RISC-V CI job in place to verify the
seamless functioning of the build with Clang.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Changes in V5:
  - update the commit message and README file with additional explanation about GCC and
    GNU Binutils version. Additionally, it was added information about Clang.
---
 Changes in V4:
  - Update version of GCC (12.2) and GNU Binutils (2.39) to the version
    which are in Xen's contrainter for RISC-V
---
 Changes in V3:
  - new patch
---
 README | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/README b/README
index c8a108449e..7fd4173743 100644
--- a/README
+++ b/README
@@ -48,6 +48,15 @@ provided by your OS distributor:
       - For ARM 64-bit:
         - GCC 5.1 or later
         - GNU Binutils 2.24 or later
+      - For RISC-V 64-bit:
+        - GCC 12.2 or later
+        - GNU Binutils 2.39 or later
+        This doesn't represent a strict lower bound for GCC and GNU Binutils;
+        rather, these versions are specifically employed by the Xen RISC-V
+        container and are anticipated to undergo continuous testing.
+        While it is feasible to utilize Clang, it's important to note that,
+        currently, there is no Xen RISC-V CI job in place to verify the
+        seamless functioning of the build with Clang.
     * POSIX compatible awk
     * Development install of zlib (e.g., zlib-dev)
     * Development install of Python 2.7 or later (e.g., python-dev)
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:49:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:49:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685732.1066996 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref6B-0002Gz-EX; Mon, 26 Feb 2024 17:49:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685732.1066996; Mon, 26 Feb 2024 17:49:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref6B-0002GQ-8C; Mon, 26 Feb 2024 17:49:07 +0000
Received: by outflank-mailman (input) for mailman id 685732;
 Mon, 26 Feb 2024 17:49:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewr-0007o5-U0
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:30 +0000
Received: from mail-lj1-x235.google.com (mail-lj1-x235.google.com
 [2a00:1450:4864:20::235])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fd6ea638-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:28 +0100 (CET)
Received: by mail-lj1-x235.google.com with SMTP id
 38308e7fff4ca-2d24a727f78so47698731fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:28 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fd6ea638-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969167; x=1709573967; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=SUqcPrIhWe0Vk4v1sB7d9+i8qhCirbfaZ3hk8sBz3cE=;
        b=MjrTnmlokNSCB+mpY7bQraWTco6yjA8xr35swExpUhrctW27PpLK8LOi5nZ1tW3oYz
         0IVvWv4R8AU3a5gR9FVkB3+UEHn2ZkTMY+HKeE0bNmgs5UtGnMKKzvxwuhXTqq7bTfk4
         +AjSJh7d8Nc6Q+kyrZsSrWPbKp+vVGQsRB5m5yLflLBbF98RAtV44mqj4rDBCzXql33j
         2uKOPxOkqJouVIpJmqV7jB2cbCQ0la2k8smvqevH0CiPF8ZEvcsqhQlaJhqhkSorQUUH
         ED4Ykue5YBDHsn7ONrc1jQJvGHDgIUbWOVojWNoHexFHPaCRmeuO83+nenGePXxCsO7/
         kGCw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969167; x=1709573967;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=SUqcPrIhWe0Vk4v1sB7d9+i8qhCirbfaZ3hk8sBz3cE=;
        b=PTIDupHoo9xCN5F9k3XnO+vGDmuupAyJjFzgJU2S/Gb3dusBrLJYycrnfk9F8KzcTb
         s0srM42akEHPv3CnbWdcUoh63lWTCJzQfXlChjLMvHH3qBJksZYhe31FvfH5EY+EgjPS
         qDGAlN5NWRvWSrLjVCzO6FkMObNCD+eJf9gAiPgEE7LHDHm37UvvsJFRYQkY7fYFU9Nr
         uAiNJnlyvzw59kyXraE+o3ogepo+A9GD+IdImIBvQz28TXvmzmoh4W+XB1bMTaqG4eIN
         creIJx60C5OYgtqDslN8dcBciKESC1Z70QlSZmV0a4Pxfl32WNWyz9RRqPXU/c4KW42P
         iDGg==
X-Gm-Message-State: AOJu0Yyu3gnkHqrlkGx0PYjwtAL0YKrHYKUGCG6Njs+IbekPLFira6uO
	RB2Mk1ypwkSKjIqpmoaz4TxP8SKeEEwkuaD9NwtnbGlMUIZCGxkhejecz3by
X-Google-Smtp-Source: AGHT+IE2jjYrQGrLtX7+yNNjBIujy1867FvMwRnqSl90csYLZUvko1M+Hz4cnzgF6OT31oX4csk5Pw==
X-Received: by 2002:a2e:8784:0:b0:2d2:3c10:4b6c with SMTP id n4-20020a2e8784000000b002d23c104b6cmr4490012lji.24.1708969166768;
        Mon, 26 Feb 2024 09:39:26 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 13/23] xen/riscv: introduce atomic.h
Date: Mon, 26 Feb 2024 18:38:55 +0100
Message-ID: <85ad8c86901d045beed228947d4c3faf277af3ca.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Initially the patch was introduced by Bobby, who takes the header from
Linux kernel.

The following changes were done on top of Linux kernel header:
 - atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated
     to use__*xchg_generic()
 - drop casts in write_atomic() as they are unnecessary
 - drop introduction of WRITE_ONCE() and READ_ONCE().
   Xen provides ACCESS_ONCE()
 - remove zero-length array access in read_atomic()
 - drop defines similar to pattern
 - #define atomic_add_return_relaxed   atomic_add_return_relaxed
 - move not RISC-V specific functions to asm-generic/atomics-ops.h

Signed-off-by: Bobby Eshleman <bobbyeshleman@gmail.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5:
 - fence.h changes were moved to separate patch as patches related to io.h and cmpxchg.h,
   which are dependecies for this patch, also needed changes in fence.h
 - remove accessing of zero-length array
 - drops cast in write_atomic()
 - drop introduction of WRITE_ONCE() and READ_ONCE().
 - drop defines similar to pattern #define atomic_add_return_relaxed   atomic_add_return_relaxed
 - Xen code style fixes
 - move not RISC-V specific functions to asm-generic/atomics-ops.h
---
Changes in V4:
 - do changes related to the updates of [PATCH v3 13/34] xen/riscv: introduce cmpxchg.h
 - drop casts in read_atomic_size(), write_atomic(), add_sized()
 - tabs -> spaces
 - drop #ifdef CONFIG_SMP ... #endif in fence.ha as it is simpler to handle NR_CPUS=1
   the same as NR_CPUS>1 with accepting less than ideal performance.
---
Changes in V3:
  - update the commit message
  - add SPDX for fence.h
  - code style fixes
  - Remove /* TODO: ... */ for add_sized macros. It looks correct to me.
  - re-order the patch
  - merge to this patch fence.h
---
Changes in V2:
 - Change an author of commit. I got this header from Bobby's old repo.
---
 xen/arch/riscv/include/asm/atomic.h  | 296 +++++++++++++++++++++++++++
 xen/include/asm-generic/atomic-ops.h |  92 +++++++++
 2 files changed, 388 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/atomic.h
 create mode 100644 xen/include/asm-generic/atomic-ops.h

diff --git a/xen/arch/riscv/include/asm/atomic.h b/xen/arch/riscv/include/asm/atomic.h
new file mode 100644
index 0000000000..8007ae4c90
--- /dev/null
+++ b/xen/arch/riscv/include/asm/atomic.h
@@ -0,0 +1,296 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Taken and modified from Linux.
+ *
+ * The following changes were done:
+ * - * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated
+ *     to use__*xchg_generic()
+ * - drop casts in write_atomic() as they are unnecessary
+ * - drop introduction of WRITE_ONCE() and READ_ONCE().
+ *   Xen provides ACCESS_ONCE()
+ * - remove zero-length array access in read_atomic()
+ * - drop defines similar to pattern
+ *   #define atomic_add_return_relaxed   atomic_add_return_relaxed
+ * - move not RISC-V specific functions to asm-generic/atomics-ops.h
+ * 
+ * Copyright (C) 2007 Red Hat, Inc. All Rights Reserved.
+ * Copyright (C) 2012 Regents of the University of California
+ * Copyright (C) 2017 SiFive
+ * Copyright (C) 2024 Vates SAS
+ */
+
+#ifndef _ASM_RISCV_ATOMIC_H
+#define _ASM_RISCV_ATOMIC_H
+
+#include <xen/atomic.h>
+
+#include <asm/cmpxchg.h>
+#include <asm/fence.h>
+#include <asm/io.h>
+#include <asm/system.h>
+
+#include <asm-generic/atomic-ops.h>
+
+void __bad_atomic_size(void);
+
+/*
+ * Legacy from Linux kernel. For some reason they wanted to have ordered
+ * read/write access. Thereby read* is used instead of read<X>_cpu()
+ */
+static always_inline void read_atomic_size(const volatile void *p,
+                                           void *res,
+                                           unsigned int size)
+{
+    switch ( size )
+    {
+    case 1: *(uint8_t *)res = readb(p); break;
+    case 2: *(uint16_t *)res = readw(p); break;
+    case 4: *(uint32_t *)res = readl(p); break;
+    case 8: *(uint32_t *)res  = readq(p); break;
+    default: __bad_atomic_size(); break;
+    }
+}
+
+#define read_atomic(p) ({                               \
+    union { typeof(*p) val; char c[sizeof(*p)]; } x_;   \
+    read_atomic_size(p, x_.c, sizeof(*p));              \
+    x_.val;                                             \
+})
+
+#define write_atomic(p, x)                              \
+({                                                      \
+    typeof(*p) x__ = (x);                               \
+    switch ( sizeof(*p) )                               \
+    {                                                   \
+    case 1: writeb(x__,  p); break;                     \
+    case 2: writew(x__, p); break;                      \
+    case 4: writel(x__, p); break;                      \
+    case 8: writeq(x__, p); break;                      \
+    default: __bad_atomic_size(); break;                \
+    }                                                   \
+    x__;                                                \
+})
+
+#define add_sized(p, x)                                 \
+({                                                      \
+    typeof(*(p)) x__ = (x);                             \
+    switch ( sizeof(*(p)) )                             \
+    {                                                   \
+    case 1: writeb(read_atomic(p) + x__, p); break;     \
+    case 2: writew(read_atomic(p) + x__, p); break;     \
+    case 4: writel(read_atomic(p) + x__, p); break;     \
+    default: __bad_atomic_size(); break;                \
+    }                                                   \
+})
+
+#define __atomic_acquire_fence() \
+    __asm__ __volatile__ ( RISCV_ACQUIRE_BARRIER "" ::: "memory" )
+
+#define __atomic_release_fence() \
+    __asm__ __volatile__ ( RISCV_RELEASE_BARRIER "" ::: "memory" )
+
+/*
+ * First, the atomic ops that have no ordering constraints and therefor don't
+ * have the AQ or RL bits set.  These don't return anything, so there's only
+ * one version to worry about.
+ */
+#define ATOMIC_OP(op, asm_op, I, asm_type, c_type, prefix)  \
+static inline                                               \
+void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \
+{                                                           \
+    __asm__ __volatile__ (                                  \
+        "   amo" #asm_op "." #asm_type " zero, %1, %0"      \
+        : "+A" (v->counter)                                 \
+        : "r" (I)                                           \
+        : "memory" );                                       \
+}                                                           \
+
+#define ATOMIC_OPS(op, asm_op, I)                           \
+        ATOMIC_OP (op, asm_op, I, w, int,   )
+
+ATOMIC_OPS(add, add,  i)
+ATOMIC_OPS(sub, add, -i)
+ATOMIC_OPS(and, and,  i)
+ATOMIC_OPS( or,  or,  i)
+ATOMIC_OPS(xor, xor,  i)
+
+#undef ATOMIC_OP
+#undef ATOMIC_OPS
+
+/*
+ * Atomic ops that have ordered, relaxed, acquire, and release variants.
+ * There's two flavors of these: the arithmatic ops have both fetch and return
+ * versions, while the logical ops only have fetch versions.
+ */
+#define ATOMIC_FETCH_OP(op, asm_op, I, asm_type, c_type, prefix)    \
+static inline                                                       \
+c_type atomic##prefix##_fetch_##op##_relaxed(c_type i,              \
+                         atomic##prefix##_t *v)                     \
+{                                                                   \
+    register c_type ret;                                            \
+    __asm__ __volatile__ (                                          \
+        "   amo" #asm_op "." #asm_type " %1, %2, %0"                \
+        : "+A" (v->counter), "=r" (ret)                             \
+        : "r" (I)                                                   \
+        : "memory" );                                               \
+    return ret;                                                     \
+}                                                                   \
+static inline                                                       \
+c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t *v) \
+{                                                                   \
+    register c_type ret;                                            \
+    __asm__ __volatile__ (                                          \
+        "   amo" #asm_op "." #asm_type ".aqrl  %1, %2, %0"          \
+        : "+A" (v->counter), "=r" (ret)                             \
+        : "r" (I)                                                   \
+        : "memory" );                                               \
+    return ret;                                                     \
+}
+
+#define ATOMIC_OP_RETURN(op, asm_op, c_op, I, asm_type, c_type, prefix) \
+static inline                                                           \
+c_type atomic##prefix##_##op##_return_relaxed(c_type i,                 \
+                          atomic##prefix##_t *v)                        \
+{                                                                       \
+        return atomic##prefix##_fetch_##op##_relaxed(i, v) c_op I;      \
+}                                                                       \
+static inline                                                           \
+c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t *v)  \
+{                                                                       \
+        return atomic##prefix##_fetch_##op(i, v) c_op I;                \
+}
+
+#define ATOMIC_OPS(op, asm_op, c_op, I)                                 \
+        ATOMIC_FETCH_OP( op, asm_op,       I, w, int,   )               \
+        ATOMIC_OP_RETURN(op, asm_op, c_op, I, w, int,   )
+
+ATOMIC_OPS(add, add, +,  i)
+ATOMIC_OPS(sub, add, +, -i)
+
+#undef ATOMIC_OPS
+
+#define ATOMIC_OPS(op, asm_op, I) \
+        ATOMIC_FETCH_OP(op, asm_op, I, w, int,   )
+
+ATOMIC_OPS(and, and, i)
+ATOMIC_OPS( or,  or, i)
+ATOMIC_OPS(xor, xor, i)
+
+#undef ATOMIC_OPS
+
+#undef ATOMIC_FETCH_OP
+#undef ATOMIC_OP_RETURN
+
+/* This is required to provide a full barrier on success. */
+static inline int atomic_add_unless(atomic_t *v, int a, int u)
+{
+       int prev, rc;
+
+    __asm__ __volatile__ (
+        "0: lr.w     %[p],  %[c]\n"
+        "   beq      %[p],  %[u], 1f\n"
+        "   add      %[rc], %[p], %[a]\n"
+        "   sc.w.rl  %[rc], %[rc], %[c]\n"
+        "   bnez     %[rc], 0b\n"
+        RISCV_FULL_BARRIER
+        "1:\n"
+        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
+        : [a] "r" (a), [u] "r" (u)
+        : "memory");
+    return prev;
+}
+
+/*
+ * atomic_{cmp,}xchg is required to have exactly the same ordering semantics as
+ * {cmp,}xchg and the operations that return, so they need a full barrier.
+ */
+#define ATOMIC_OP(c_t, prefix, size)                            \
+static inline                                                   \
+c_t atomic##prefix##_xchg_relaxed(atomic##prefix##_t *v, c_t n) \
+{                                                               \
+    return __xchg_generic(&(v->counter), n, size, "", "", "");  \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_xchg_acquire(atomic##prefix##_t *v, c_t n) \
+{                                                               \
+    return __xchg_generic(&(v->counter), n, size,               \
+                          "", "", RISCV_ACQUIRE_BARRIER);       \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_xchg_release(atomic##prefix##_t *v, c_t n) \
+{                                                               \
+    return __xchg_generic(&(v->counter), n, size,               \
+                          "", RISCV_RELEASE_BARRIER, "");       \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n)         \
+{                                                               \
+    return __xchg_generic(&(v->counter), n, size,               \
+                          ".aqrl", "", "");                     \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg_relaxed(atomic##prefix##_t *v,     \
+                     c_t o, c_t n)                              \
+{                                                               \
+    return __cmpxchg_generic(&(v->counter), o, n, size,         \
+                             "", "", "");                       \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg_acquire(atomic##prefix##_t *v,     \
+                     c_t o, c_t n)                              \
+{                                                               \
+    return __cmpxchg_generic(&(v->counter), o, n, size,         \
+                             "", "", RISCV_ACQUIRE_BARRIER);    \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg_release(atomic##prefix##_t *v,     \
+                     c_t o, c_t n)                              \
+{	                                                            \
+    return __cmpxchg_generic(&(v->counter), o, n, size,         \
+                             "", RISCV_RELEASE_BARRIER, "");    \
+}                                                               \
+static inline                                                   \
+c_t atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n) \
+{                                                               \
+    return __cmpxchg_generic(&(v->counter), o, n, size,         \
+                             ".rl", "", " fence rw, rw\n");     \
+}
+
+#define ATOMIC_OPS() \
+    ATOMIC_OP(int,   , 4)
+
+ATOMIC_OPS()
+
+#undef ATOMIC_OPS
+#undef ATOMIC_OP
+
+static inline int atomic_sub_if_positive(atomic_t *v, int offset)
+{
+       int prev, rc;
+
+    __asm__ __volatile__ (
+        "0: lr.w     %[p],  %[c]\n"
+        "   sub      %[rc], %[p], %[o]\n"
+        "   bltz     %[rc], 1f\n"
+        "   sc.w.rl  %[rc], %[rc], %[c]\n"
+        "   bnez     %[rc], 0b\n"
+        "   fence    rw, rw\n"
+        "1:\n"
+        : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter)
+        : [o] "r" (offset)
+        : "memory" );
+    return prev - offset;
+}
+
+#define atomic_dec_if_positive(v)	atomic_sub_if_positive(v, 1)
+
+#endif /* _ASM_RISCV_ATOMIC_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-generic/atomic-ops.h b/xen/include/asm-generic/atomic-ops.h
new file mode 100644
index 0000000000..fdd5a93ed8
--- /dev/null
+++ b/xen/include/asm-generic/atomic-ops.h
@@ -0,0 +1,92 @@
+#/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_ATOMIC_OPS_H_
+#define _ASM_GENERIC_ATOMIC_OPS_H_
+
+#include <xen/atomic.h>
+#include <xen/lib.h>
+
+#ifndef ATOMIC_READ
+static inline int atomic_read(const atomic_t *v)
+{
+    return ACCESS_ONCE(v->counter);
+}
+#endif
+
+#ifndef _ATOMIC_READ
+static inline int _atomic_read(atomic_t v)
+{
+    return v.counter;
+}
+#endif
+
+#ifndef ATOMIC_SET
+static inline void atomic_set(atomic_t *v, int i)
+{
+    ACCESS_ONCE(v->counter) = i;
+}
+#endif
+
+#ifndef _ATOMIC_SET
+static inline void _atomic_set(atomic_t *v, int i)
+{
+    v->counter = i;
+}
+#endif
+
+#ifndef ATOMIC_SUB_AND_TEST
+static inline int atomic_sub_and_test(int i, atomic_t *v)
+{
+    return atomic_sub_return(i, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_INC
+static inline void atomic_inc(atomic_t *v)
+{
+    atomic_add(1, v);
+}
+#endif
+
+#ifndef ATOMIC_INC_RETURN
+static inline int atomic_inc_return(atomic_t *v)
+{
+    return atomic_add_return(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC
+static inline void atomic_dec(atomic_t *v)
+{
+    atomic_sub(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC_RETURN
+static inline int atomic_dec_return(atomic_t *v)
+{
+    return atomic_sub_return(1, v);
+}
+#endif
+
+#ifndef ATOMIC_DEC_AND_TEST
+static inline int atomic_dec_and_test(atomic_t *v)
+{
+    return atomic_sub_return(1, v) == 0;
+}
+#endif
+
+#ifndef ATOMIC_ADD_NEGATIVE
+static inline int atomic_add_negative(int i, atomic_t *v)
+{
+    return atomic_add_return(i, v) < 0;
+}
+#endif
+
+#ifndef ATOMIC_INC_AND_TEST
+static inline int atomic_inc_and_test(atomic_t *v)
+{
+    return atomic_add_return(1, v) == 0;
+}
+#endif
+
+#endif /* _ASM_GENERIC_ATOMIC_OPS_H_ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:49:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:49:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685739.1067010 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref6w-0003RK-Rx; Mon, 26 Feb 2024 17:49:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685739.1067010; Mon, 26 Feb 2024 17:49:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref6w-0003RD-Nc; Mon, 26 Feb 2024 17:49:54 +0000
Received: by outflank-mailman (input) for mailman id 685739;
 Mon, 26 Feb 2024 17:49:53 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ref6v-0003PS-HU; Mon, 26 Feb 2024 17:49:53 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ref6v-0001rf-GT; Mon, 26 Feb 2024 17:49:53 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1ref6v-0007PK-4U; Mon, 26 Feb 2024 17:49:53 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1ref6v-0006xZ-40; Mon, 26 Feb 2024 17:49:53 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ECoDogrL+WMZx4JaUR0nddJsk+2oRolJcmGDs2L80p0=; b=n3dUk4HGhGQdEx7pQ4b+fUOxJi
	uuh4OXhWoDIVwvKGUyt/JyWMtTP8sXmsxOXCgf82fQNPLmYc4dwZOe0QTUWhxhMwPV88sXC7rt+xz
	L8OQeKDScTor4l+7VflbyCMNon4IXfBT1Qrr+gA+KySXMgOtPdnvrYMyNrV3x8WXzZfo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184768-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184768: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=44fdc4f3983be77dda709d7a0c3fb8905fbf920b
X-Osstest-Versions-That:
    ovmf=ba96acd963e794e86633ffd5e7f96cbc03db673c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 26 Feb 2024 17:49:53 +0000

flight 184768 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184768/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 44fdc4f3983be77dda709d7a0c3fb8905fbf920b
baseline version:
 ovmf                 ba96acd963e794e86633ffd5e7f96cbc03db673c

Last test of basis   184761  2024-02-25 20:12:59 Z    0 days
Testing same since   184768  2024-02-26 15:43:07 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Yi Li <yi1.li@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   ba96acd963..44fdc4f398  44fdc4f3983be77dda709d7a0c3fb8905fbf920b -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:50:11 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:50:11 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685742.1067020 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref7D-0004sl-4H; Mon, 26 Feb 2024 17:50:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685742.1067020; Mon, 26 Feb 2024 17:50:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref7D-0004sa-0g; Mon, 26 Feb 2024 17:50:11 +0000
Received: by outflank-mailman (input) for mailman id 685742;
 Mon, 26 Feb 2024 17:50:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewz-0007pd-GZ
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:37 +0000
Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com
 [2a00:1450:4864:20::233])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 02a1ac2b-d4ce-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 18:39:36 +0100 (CET)
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2d29111272eso8912501fa.0
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:36 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.34
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02a1ac2b-d4ce-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969176; x=1709573976; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=5WTYbeaI/8M//O8Mho5D4ujQilkp7OeZ0ER9cjWpZiI=;
        b=mgQlPVZK5diHUVyhaIerK27PW0Ozc7AiIwWT30ftyAY9+7FFfvj7eZwM/0lTGXsh5C
         GjbN599pQcokL040x2jWR9X6BgTmz7lQ9K5hG4lwu0UdgJYyMcDqrSKDZZLFAsaMa3V9
         gSbidA0ZrujeRIJGUzMl+gKBZhYuQ+DsWA73o3zZ9pZA7F4/EU4mEb54R7qU4spUVAJ0
         u+78fEpvTtjYL8QTDaOnlVznf5StfavIZjGDAXtN6huH4jjHpJ+ewuAZ3qLP6Ie/toqo
         0z+g9FS5Tb1eWQR5KTyqFB3kTAfBhrlNJltXgnxSn6v/yHTVjvSPJkErfe1EK/nv2vd+
         66AA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969176; x=1709573976;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=5WTYbeaI/8M//O8Mho5D4ujQilkp7OeZ0ER9cjWpZiI=;
        b=iZZ3SU3w03Z3/ErXQVkOrHEwtR5Ay8sGDzkUVgHQc64OPb1UCSgwIn4fJAzeuooc1U
         7dBAnJ1d8DF0ayMqgf+vNigHaC3eDEfV7Oh1yIIczx8laFy3N1zW3yeuUhaAbbpgivTy
         bzFOG9d36aKrj3OR8vhJQelbBrcnEcoGRK6/PERAAuzmJFZggDI+qUSijJyTcj87k8/n
         1ytg+Mw4kXonuTAR1foqk0uNSSkpIpvpZE32lawmM6CRx4Szsi4cWFhUgiyXms7hFqSN
         wT123RHAU97aO9nYk1jAQoGuLamKx/ot1eHpUD1iPl1S6sKU5VxUXB7mFcZjZQ0mAMpG
         EtLA==
X-Gm-Message-State: AOJu0YxH6SdhzBlFhnK2V27d6iffLgZPtZgNiDk7Bv/e8S49I4rtbQjz
	fYcwC4WA3MPkxz81SQAEXtwOVeFOxvJ43eWoKHUMfLGMnSrusGElelCyc/6x
X-Google-Smtp-Source: AGHT+IEKghU9aySWrWNACGG41Xu0HX93hwXV4vOXCuaUYiEAbb+Js2iTq61v1wKCgAZf6Wd1ZxyGQg==
X-Received: by 2002:a2e:a54a:0:b0:2d2:3915:cfc4 with SMTP id e10-20020a2ea54a000000b002d23915cfc4mr6142362ljn.4.1708969175646;
        Mon, 26 Feb 2024 09:39:35 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 21/23] xen/rirscv: add minimal amount of stubs to build full Xen
Date: Mon, 26 Feb 2024 18:39:03 +0100
Message-ID: <fd3dce5ff76e88e2389df9c0e3a910f735ddfb84.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5:
 - drop unrelated changes
 - assert_failed("unimplmented...") change to BUG_ON()
---
Changes in V4:
  - added new stubs which are necessary for compilation after rebase: __cpu_up(), __cpu_disable(), __cpu_die()
    from smpboot.c
  - back changes related to printk() in early_printk() as they should be removed in the next patch to avoid
    compilation error.
  - update definition of cpu_khz: __read_mostly -> __ro_after_init.
  - drop vm_event_reset_vmtrace(). It is defibed in asm-generic/vm_event.h.
  - move vm_event_*() functions from stubs.c to riscv/vm_event.c.
  - s/BUG/BUG_ON("unimplemented") in stubs.c
  - back irq_actor_none() and irq_actor_none() as common/irq.c isn't compiled at this moment,
    so this function are needed to avoid compilation error.
  - defined max_page to avoid compilation error, it will be removed as soon as common/page_alloc.c will
    be compiled.
---
Changes in V3:
 - code style fixes.
 - update attribute for frametable_base_pdx  and frametable_virt_end to __ro_after_init.
   insteaf of read_mostly.
 - use BUG() instead of assert_failed/WARN for newly introduced stubs.
 - drop "#include <public/vm_event.h>" in stubs.c and use forward declaration instead.
 - drop ack_node() and end_node() as they aren't used now.
---
Changes in V2:
 - define udelay stub
 - remove 'select HAS_PDX' from RISC-V Kconfig because of
   https://lore.kernel.org/xen-devel/20231006144405.1078260-1-andrew.cooper3@citrix.com/
---
 xen/arch/riscv/Makefile |   1 +
 xen/arch/riscv/mm.c     |  50 +++++
 xen/arch/riscv/setup.c  |   8 +
 xen/arch/riscv/stubs.c  | 438 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/riscv/traps.c  |  25 +++
 5 files changed, 522 insertions(+)
 create mode 100644 xen/arch/riscv/stubs.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 1ed1a8369b..60afbc0ad9 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -4,6 +4,7 @@ obj-y += mm.o
 obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
+obj-y += stubs.o
 obj-y += traps.o
 obj-y += vm_event.o
 
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index fe3a43be20..2c3fb7d72e 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/cache.h>
 #include <xen/compiler.h>
 #include <xen/init.h>
@@ -14,6 +15,9 @@
 #include <asm/page.h>
 #include <asm/processor.h>
 
+unsigned long __ro_after_init frametable_base_pdx;
+unsigned long __ro_after_init frametable_virt_end;
+
 struct mmu_desc {
     unsigned int num_levels;
     unsigned int pgtbl_count;
@@ -294,3 +298,49 @@ unsigned long __init calc_phys_offset(void)
     phys_offset = load_start - XEN_VIRT_START;
     return phys_offset;
 }
+
+void put_page(struct page_info *page)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long get_upper_mfn_bound(void)
+{
+    /* No memory hotplug yet, so current memory limit is the final one. */
+    return max_page - 1;
+}
+
+void arch_dump_shared_mem_info(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int populate_pt_range(unsigned long virt, unsigned long nr_mfns)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int xenmem_add_to_physmap_one(struct domain *d, unsigned int space,
+                              union add_to_physmap_extra extra,
+                              unsigned long idx, gfn_t gfn)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
+
+int destroy_xen_mappings(unsigned long s, unsigned long e)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
+
+int map_pages_to_xen(unsigned long virt,
+                     mfn_t mfn,
+                     unsigned long nr_mfns,
+                     unsigned int flags)
+{
+    BUG_ON("unimplemented");
+    return -1;
+}
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 98a94c4c48..8bb5bdb2ae 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -1,11 +1,19 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/compile.h>
 #include <xen/init.h>
 #include <xen/mm.h>
 
+#include <public/version.h>
+
 #include <asm/early_printk.h>
 
+void arch_get_xen_caps(xen_capabilities_info_t *info)
+{
+    BUG_ON("unimplemented");
+}
+
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
     __aligned(STACK_SIZE);
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
new file mode 100644
index 0000000000..529f1dbe52
--- /dev/null
+++ b/xen/arch/riscv/stubs.c
@@ -0,0 +1,438 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#include <xen/cpumask.h>
+#include <xen/domain.h>
+#include <xen/irq.h>
+#include <xen/nodemask.h>
+#include <xen/time.h>
+#include <public/domctl.h>
+
+#include <asm/current.h>
+
+/* smpboot.c */
+
+cpumask_t cpu_online_map;
+cpumask_t cpu_present_map;
+cpumask_t cpu_possible_map;
+
+/* ID of the PCPU we're running on */
+DEFINE_PER_CPU(unsigned int, cpu_id);
+/* XXX these seem awfully x86ish... */
+/* representing HT siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
+/* representing HT and core siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
+
+nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
+
+/*
+ * max_page is defined in page_alloc.c which isn't complied for now.
+ * definition of max_page will be remove as soon as page_alloc is built.
+ */
+unsigned long __read_mostly max_page;
+
+/* time.c */
+
+unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
+
+s_time_t get_s_time(void)
+{
+    BUG_ON("unimplemented");
+}
+
+int reprogram_timer(s_time_t timeout)
+{
+    BUG_ON("unimplemented");
+}
+
+void send_timer_event(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void domain_set_time_offset(struct domain *d, int64_t time_offset_seconds)
+{
+    BUG_ON("unimplemented");
+}
+
+/* shutdown.c */
+
+void machine_restart(unsigned int delay_millisecs)
+{
+    BUG_ON("unimplemented");
+}
+
+void machine_halt(void)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domctl.c */
+
+long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
+                    XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_domain_info(const struct domain *d,
+                          struct xen_domctl_getdomaininfo *info)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_get_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+/* monitor.c */
+
+int arch_monitor_domctl_event(struct domain *d,
+                              struct xen_domctl_monitor_op *mop)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smp.c */
+
+void arch_flush_tlb_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_event_check_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_call_function_mask(const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+/* irq.c */
+
+struct pirq *alloc_pirq_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int pirq_guest_bind(struct vcpu *v, struct pirq *pirq, int will_share)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_guest_unbind(struct domain *d, struct pirq *pirq)
+{
+    BUG_ON("unimplemented");
+}
+
+void pirq_set_affinity(struct domain *d, int pirq, const cpumask_t *mask)
+{
+    BUG_ON("unimplemented");
+}
+
+hw_irq_controller no_irq_type = {
+    .typename = "none",
+    .startup = irq_startup_none,
+    .shutdown = irq_shutdown_none,
+    .enable = irq_enable_none,
+    .disable = irq_disable_none,
+};
+
+int arch_init_one_irq_desc(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+void smp_send_state_dump(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/* domain.c */
+
+DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
+unsigned long __per_cpu_offset[NR_CPUS];
+
+void context_switch(struct vcpu *prev, struct vcpu *next)
+{
+    BUG_ON("unimplemented");
+}
+
+void continue_running(struct vcpu *same)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_local_execstate(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void sync_vcpu_execstate(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void startup_cpu_idle_loop(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_domain_struct(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void dump_pageframe_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void free_vcpu_struct(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_create(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_vcpu_destroy(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_switch_to_aarch64_mode(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_create(struct domain *d,
+                       struct xen_domctl_createdomain *config,
+                       unsigned int flags)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_teardown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_destroy(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_shutdown(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_pause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_unpause(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_domain_soft_reset(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_domain_creation_finished(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_set_info_guest(struct vcpu *v, vcpu_guest_context_u c)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_initialise_vcpu(struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+}
+
+int arch_vcpu_reset(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+int domain_relinquish_resources(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_domain_info(struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_dump_vcpu_info(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_mark_events_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_update_evtchn_irq(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_block_unless_event_pending(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void vcpu_kick(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+struct domain *alloc_domain_struct(void)
+{
+    BUG_ON("unimplemented");
+}
+
+struct vcpu *alloc_vcpu_struct(const struct domain *d)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long
+hypercall_create_continuation(unsigned int op, const char *format, ...)
+{
+    BUG_ON("unimplemented");
+}
+
+int __init parse_arch_dom0_param(const char *s, const char *e)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guestcopy.c */
+
+unsigned long raw_copy_to_guest(void *to, const void *from, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned long raw_copy_from_guest(void *to, const void __user *from,
+                                  unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* sysctl.c */
+
+long arch_do_sysctl(struct xen_sysctl *sysctl,
+                    XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
+{
+    BUG_ON("unimplemented");
+}
+
+void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
+{
+    BUG_ON("unimplemented");
+}
+
+/* p2m.c */
+
+int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
+{
+    BUG_ON("unimplemented");
+}
+
+int unmap_mmio_regions(struct domain *d,
+                       gfn_t start_gfn,
+                       unsigned long nr,
+                       mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int map_mmio_regions(struct domain *d,
+                     gfn_t start_gfn,
+                     unsigned long nr,
+                     mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+                          unsigned long gfn, mfn_t mfn)
+{
+    BUG_ON("unimplemented");
+}
+
+/* Return the size of the pool, in bytes. */
+int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
+{
+    BUG_ON("unimplemented");
+}
+
+/* delay.c */
+
+void udelay(unsigned long usecs)
+{
+    BUG_ON("unimplemented");
+}
+
+/* guest_access.h */ 
+
+static inline unsigned long raw_clear_guest(void *to, unsigned int len)
+{
+    BUG_ON("unimplemented");
+}
+
+/* smpboot.c */
+
+int __cpu_up(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_disable(void)
+{
+    BUG_ON("unimplemented");
+}
+
+void __cpu_die(unsigned int cpu)
+{
+    BUG_ON("unimplemented");
+}
+
+/*
+ * The following functions are defined in common/irq.c, which will be built in
+ * the next commit, so these changes will be removed there.
+ */
+
+void cf_check irq_actor_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+}
+
+unsigned int cf_check irq_startup_none(struct irq_desc *desc)
+{
+    BUG_ON("unimplemented");
+
+    return 0;
+}
diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c
index ccd3593f5a..5415cf8d90 100644
--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -4,6 +4,10 @@
  *
  * RISC-V Trap handlers
  */
+
+#include <xen/lib.h>
+#include <xen/sched.h>
+
 #include <asm/processor.h>
 #include <asm/traps.h>
 
@@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs)
 {
     die();
 }
+
+void vcpu_show_execution_state(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void show_execution_state(const struct cpu_user_regs *regs)
+{
+    printk("implement show_execution_state(regs)\n");
+}
+
+void arch_hypercall_tasklet_result(struct vcpu *v, long res)
+{
+    BUG_ON("unimplemented");
+}
+
+enum mc_disposition arch_do_multicall_call(struct mc_state *state)
+{
+    BUG_ON("unimplemented");
+    return mc_continue;
+}
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:51:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:51:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685749.1067030 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref8B-0005x9-CG; Mon, 26 Feb 2024 17:51:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685749.1067030; Mon, 26 Feb 2024 17:51:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref8B-0005wj-9c; Mon, 26 Feb 2024 17:51:11 +0000
Received: by outflank-mailman (input) for mailman id 685749;
 Mon, 26 Feb 2024 17:51:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reews-0007o5-CD
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:30 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fddff295-d4cd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:28 +0100 (CET)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2d27184197cso40171481fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:28 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fddff295-d4cd-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969168; x=1709573968; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=PMMEpTcUg6wPnIKq2yrDzwzPHhvyzoLsQ8c7Qj98SWo=;
        b=c0FT6eTaXjzQ78GLtCEwOl7zWAE/f/k3yYVIb7up8c6TWFjicVHGoTHeNDc+6hbSBK
         yGceaXy9jphdSSYozIGz6xgp76J924Ll2AuR44skAHZOf3ESE5mH1Y9wOIV0xScoHX8/
         m+yMIGPmBLQBRJF2aig9+wMTGoWGn6Zz0qf56Si5Oj+EU1k9r20tbjtKIYYHN5hdssFA
         sDsl/B97wYxh8O45x5wnfGuf373xSBU54aTMJ1LzTJLW6pqNcaOLuJYNsnUw4SDF3oi/
         YISWmJzWTi7XKYj2VkDY71Dq6Wxlc6DAuVjmI8DxEMzfB99SSr1yc+rsqI+bl2e5IMe7
         KsEA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969168; x=1709573968;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=PMMEpTcUg6wPnIKq2yrDzwzPHhvyzoLsQ8c7Qj98SWo=;
        b=tyAQA6G4xtdnOaWXO6GBUU40rPQYKIlveonpaHgWZzQgdfo7zRcP01hCMA4VCwtcPh
         7W5tCJ1k8/sCCyU1XA8yO1ADqgkY9FPd6WY477InItUBBs0foh8GchY+s+dwwzkpoE+E
         KtlpLZMxPXfTa5Go3rFIBkOPsWRYZfc2phtBCIU4kCubsjNJC8kI5Awn8o0RR2Gy1MIV
         RvK1ZQ73XVcsIwEzj4LUXrYjOSCk9h/wobbfzcte6TVu3xy+A2QuCBOeUn6k3yLgmRBX
         KtJXGZg3xGaLV2xTdQ28p8KJw5Z5TxMSQuz0FIs5hwPps7iYHFAVuUrOeSY7KJllaEKZ
         1YRg==
X-Gm-Message-State: AOJu0YxOPvXOxvJKMqvSQjOU9WSKjKCj4z4dZllaBjJTFgwSVnhLxn8l
	sztZjkcjQSl8kPsezqo9aE3TnaGN8WqKA3nSgmlByNIRB1j4ql1IDoLqUVLN
X-Google-Smtp-Source: AGHT+IExOW9mAsdMV/2cYLS5seStBsmA8LC26c77mh0Eaj4vnYKuGRK2m6T9kx3k/B8KDLf7AvR1sg==
X-Received: by 2002:a05:651c:1994:b0:2d2:7cb9:8d21 with SMTP id bx20-20020a05651c199400b002d27cb98d21mr4945367ljb.16.1708969168133;
        Mon, 26 Feb 2024 09:39:28 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Tamas K Lengyel <tamas@tklengyel.com>,
	Alexandru Isaila <aisaila@bitdefender.com>,
	Petre Pircalabu <ppircalabu@bitdefender.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v5 14/23] xen/riscv: introduce monitor.h
Date: Mon, 26 Feb 2024 18:38:56 +0100
Message-ID: <bf4b47a16997628e5a45290dd38a8e1f70fd8fc8.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 Waiting for dependency to be merged: [PATCH v6 0/9] Introduce generic headers
 (https://lore.kernel.org/xen-devel/84568b0c24a5ec96244f3f34537e9a148367facf.1707499278.git.oleksii.kurochko@gmail.com/)
---
Changes in V4/V5:
 - Nothing changed. Only rebase.
---
Changes in V3:
 - new patch.
---
 xen/arch/riscv/include/asm/monitor.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/monitor.h

diff --git a/xen/arch/riscv/include/asm/monitor.h b/xen/arch/riscv/include/asm/monitor.h
new file mode 100644
index 0000000000..f4fe2c0690
--- /dev/null
+++ b/xen/arch/riscv/include/asm/monitor.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_RISCV_MONITOR_H__
+#define __ASM_RISCV_MONITOR_H__
+
+#include <xen/bug.h>
+
+#include <asm-generic/monitor.h>
+
+struct domain;
+
+static inline uint32_t arch_monitor_get_capabilities(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+#endif /* __ASM_RISCV_MONITOR_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:52:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:52:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685754.1067039 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref8y-0006l3-Li; Mon, 26 Feb 2024 17:52:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685754.1067039; Mon, 26 Feb 2024 17:52:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1ref8y-0006kw-J1; Mon, 26 Feb 2024 17:52:00 +0000
Received: by outflank-mailman (input) for mailman id 685754;
 Mon, 26 Feb 2024 17:51:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reewx-0007o5-Sk
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:35 +0000
Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com
 [2a00:1450:4864:20::232])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 00810cc9-d4ce-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:33 +0100 (CET)
Received: by mail-lj1-x232.google.com with SMTP id
 38308e7fff4ca-2d180d6bd32so49108011fa.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:33 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.31
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 00810cc9-d4ce-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969172; x=1709573972; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ZCwX2BCVLI3lXZG2aWqhyu9yA4jHT6KSlCqY/yiMy5E=;
        b=K/kySB9bH365FzkmKtciJtjl/bChluXtD5NzaZUBJOnrbnsA13m1v/u0lm3yFwmVdT
         cI8MzwEZDkBcSS2shR7UwfxYY2iilMxrx4aegYCn1sIJRU9Nt9y/sAYAE/DT6G/FJxyX
         pVryFglifIHpUKh2cioHqHHS2KCPgZHxxuCAHwUZ8a9oGoQQ562d3kGLcddz5otTujyZ
         mYamKUxar86TzD4ESrMbBgSNrnvaPxcnJ1PkaTWAH1VvrJQsnnABpQDBVvlDZs4xcHtY
         R/m6BRuoufVY6/fDHvzmkfVT+zu8HrZF5RAnmtP5uxCOD/Bbq13a5/AqZvGVL+bE6h2/
         AxlQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969172; x=1709573972;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ZCwX2BCVLI3lXZG2aWqhyu9yA4jHT6KSlCqY/yiMy5E=;
        b=XWQrz3KhfBvE03ETxFDFUqPzOUAaN1RM7/vEGftFCc3VO/KSdOv8iEvdS5F6p/orft
         NAD/sONDJT8RnE8Zh8e0XM0004OH4V9HLH4dfEsG0DA+3G1VApTZd+07AUGrZMUKTC1F
         Sf7vw4jVNhO9zFCMOg5ihq+EyMGxs+vr8k0EpmGtpd86eG4gvPV98gzw+28xFcEBhWi1
         GH4pxjKEEtTO7UXbJeV+yAiq7rYc5jdve0TsW+9SCCeDW/e9R7WJgiuMbasXo1xfuUk+
         DDpsbAa5vQRlIC3baJWA4F7b/VtHYs5Mlguiq9WwnANhBPoSQHj2KWbrRJYuJeGAswos
         jc0g==
X-Gm-Message-State: AOJu0YzJs+LhGhInKPNcXf1m+Hj9/1DkmFYaCsXsKvwZs6D27Sq+czz2
	74QCbwnSYvKhzbIh2S7Xq+SriFeUzG9yX+tQBPluqjg98H/qhytkx05lWfSX
X-Google-Smtp-Source: AGHT+IEahFH9d60CghpPN5UMUuIMNVwUJdv5YNlsA9M0tIHLvik4uJbaD6X9+1X5KU3flXH9J58e1g==
X-Received: by 2002:a05:651c:2019:b0:2d2:29c2:e7ea with SMTP id s25-20020a05651c201900b002d229c2e7eamr4101847ljo.24.1708969172528;
        Mon, 26 Feb 2024 09:39:32 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>
Subject: [PATCH v5 18/23] xen/riscv: add minimal stuff to processor.h to build full Xen
Date: Mon, 26 Feb 2024 18:39:00 +0100
Message-ID: <4e1ee99a9ad71015b5e8860d20b63337b526d0e9.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in V5:
 - Code style fixes.
 - drop introduced TOOLCHAIN_HAS_ZIHINTPAUSE and use as-insn instead and use
   as-insn istead.
---
Changes in V4:
 - Change message -> subject in "Changes in V3"
 - Documentation about system requirement was added. In the future, it can be checked if the extension is supported
   by system __riscv_isa_extension_available() ( https://gitlab.com/xen-project/people/olkur/xen/-/commit/737998e89ed305eb92059300c374dfa53d2143fa )
 - update cpu_relax() function to check if __riscv_zihintpause is supported by a toolchain
 - add conditional _zihintpause to -march if it is supported by a toolchain
Changes in V3:
 - update the commit subject
 - rename get_processor_id to smp_processor_id
 - code style fixes
 - update the cpu_relax instruction: use pause instruction instead of div %0, %0, zero
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 docs/misc/riscv/booting.txt            |  8 ++++++++
 xen/arch/riscv/arch.mk                 |  8 +++++++-
 xen/arch/riscv/include/asm/processor.h | 23 +++++++++++++++++++++++
 3 files changed, 38 insertions(+), 1 deletion(-)
 create mode 100644 docs/misc/riscv/booting.txt

diff --git a/docs/misc/riscv/booting.txt b/docs/misc/riscv/booting.txt
new file mode 100644
index 0000000000..38fad74956
--- /dev/null
+++ b/docs/misc/riscv/booting.txt
@@ -0,0 +1,8 @@
+System requirements
+===================
+
+The following extensions are expected to be supported by a system on which
+Xen is run:
+- Zihintpause:
+  On a system that doesn't have this extension, cpu_relax() should be
+  implemented properly. Otherwise, an illegal instruction exception will arise.
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index 8403f96b6f..fabe323ec5 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -5,6 +5,12 @@ $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
 
 CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64
 
+ifeq ($(CONFIG_RISCV_64),y)
+has_zihintpause = $(call as-insn,$(CC) -mabi=lp64 -march=rv64i_zihintpause, "pause",_zihintpause,)
+else
+has_zihintpause = $(call as-insn,$(CC) -mabi=ilp32 -march=rv32i_zihintpause, "pause",_zihintpause,)
+endif
+
 riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g
 riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
 
@@ -12,7 +18,7 @@ riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
 # into the upper half _or_ the lower half of the address space.
 # -mcmodel=medlow would force Xen into the lower half.
 
-CFLAGS += -march=$(riscv-march-y) -mstrict-align -mcmodel=medany
+CFLAGS += -march=$(riscv-march-y)$(has_zihintpause) -mstrict-align -mcmodel=medany
 
 # TODO: Drop override when more of the build is working
 override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
diff --git a/xen/arch/riscv/include/asm/processor.h b/xen/arch/riscv/include/asm/processor.h
index 6db681d805..b96af07660 100644
--- a/xen/arch/riscv/include/asm/processor.h
+++ b/xen/arch/riscv/include/asm/processor.h
@@ -12,6 +12,9 @@
 
 #ifndef __ASSEMBLY__
 
+/* TODO: need to be implemeted */
+#define smp_processor_id() 0
+
 /* On stack VCPU state */
 struct cpu_user_regs
 {
@@ -53,6 +56,26 @@ struct cpu_user_regs
     unsigned long pregs;
 };
 
+/* TODO: need to implement */
+#define cpu_to_core(cpu)   (0)
+#define cpu_to_socket(cpu) (0)
+
+static inline void cpu_relax(void)
+{
+#ifdef __riscv_zihintpause
+    /*
+     * Reduce instruction retirement.
+     * This assumes the PC changes.
+     */
+    __asm__ __volatile__ ( "pause" );
+#else
+    /* Encoding of the pause instruction */
+    __asm__ __volatile__ ( ".insn 0x100000F" );
+#endif
+
+    barrier();
+}
+
 static inline void wfi(void)
 {
     __asm__ __volatile__ ("wfi");
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 17:55:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 17:55:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685766.1067049 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1refBu-0008RN-8v; Mon, 26 Feb 2024 17:55:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685766.1067049; Mon, 26 Feb 2024 17:55:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1refBu-0008RG-63; Mon, 26 Feb 2024 17:55:02 +0000
Received: by outflank-mailman (input) for mailman id 685766;
 Mon, 26 Feb 2024 17:55:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=OiHA=KD=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1reex1-0007o5-HT
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 17:39:39 +0000
Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com
 [2a00:1450:4864:20::22f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 032d632f-d4ce-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 18:39:37 +0100 (CET)
Received: by mail-lj1-x22f.google.com with SMTP id
 38308e7fff4ca-2d208be133bso52656281fa.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 09:39:37 -0800 (PST)
Received: from fedora.. ([94.75.70.14]) by smtp.gmail.com with ESMTPSA id
 z8-20020a2e8e88000000b002d0ae22ff6fsm920883ljk.60.2024.02.26.09.39.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 26 Feb 2024 09:39:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 032d632f-d4ce-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1708969177; x=1709573977; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=sQrmbyqRAD9OqhWHU9VBJdf59WULWSiYOjY5GOmOiP8=;
        b=MrS8Ph52DBAHpkf+7+utIfKTV58otVKRrSZHxS49lTkpSNTTdxeW2qoH4olBwOsMab
         4HrQUKef6xImykIcMQlhsnwC4h3Ic8hx8UJB1AHe5T6usge16/XW2N5B096FHLAju5Qk
         FLAUJLR83WgbNCNzVbHxmysXBG71VyLW7YPcUk1BrESAx2SOr+0wa+wj4Q5JDYQoQhGd
         N4emcrTLXgyebyhUualV3a8af94uyu3h8FlAalvEclgB9w44WBUwEB/wkkwBx/aEbhXL
         /RrMKrF3o6riZ6ryiM82XjGhr6nvK7S3t3av9kp+6E2CaRNiT3MPBSOufxL+W35lXrmq
         gyJQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1708969177; x=1709573977;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=sQrmbyqRAD9OqhWHU9VBJdf59WULWSiYOjY5GOmOiP8=;
        b=Cbq7ToOuU/+byghycJmiLCvFlt6pFhrymPNQzsu4wT/8+vOldeUbico4/+d8XG84ef
         Vz0JXcErYzRzswF5W8T9RJZ8Dh1kIkXopqg5Xz8tVQWguYxqRyt3dIUVpOk0iR65MsET
         2x13ZtXSLc/BOJP80kw+8NwE2Jn7U2paIA3yD4eCphAWpjYVhRrcAR5+HIbqODNH9m1F
         GumEB9/zHWGpQYwZYawa6Wh7t6cUqdEyISMR/o26M40MT2/VLl+vcXF9Ojhtkk8P5hxb
         +02c1/r5FjoSTTcY+Tut9DxTQJOebi2JgRc6vyLOwTmf/kESbZ7pE0YrgR+BmzO88R64
         qA+w==
X-Gm-Message-State: AOJu0YzApondEHlW4GzpKZ8LWBCvwLj/08Hhl+uBpLn34PBBc/5dvMIL
	B5+zoWBiuCjuf7O9Y76AHPLgTUSxfSGkueLOIkeaqZMe2F/4tm7+CvBhxMD9
X-Google-Smtp-Source: AGHT+IF/tg5wpS5mpbCbON3DvBO+MEIBxev4Pe1ujGURD/7Ny4Hgny6UeixhjDTHKwLHMISElxqFzg==
X-Received: by 2002:a2e:9010:0:b0:2d2:8fc7:aa2d with SMTP id h16-20020a2e9010000000b002d28fc7aa2dmr1368651ljg.47.1708969176944;
        Mon, 26 Feb 2024 09:39:36 -0800 (PST)
From: Oleksii Kurochko <oleksii.kurochko@gmail.com>
To: xen-devel@lists.xenproject.org
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v5 22/23] xen/riscv: enable full Xen build
Date: Mon, 26 Feb 2024 18:39:04 +0100
Message-ID: <4ee60959772504f7bfaad5e1d5d504e1346bf217.1708962629.git.oleksii.kurochko@gmail.com>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <cover.1708962629.git.oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes in V5:
 - Nothing changed. Only rebase.
---
Changes in V4:
 - drop stubs for irq_actor_none() and irq_actor_none() as common/irq.c is compiled now.
 - drop defintion of max_page in stubs.c as common/page_alloc.c is compiled now.
 - drop printk() related changes in riscv/early_printk.c as common version will be used.
---
Changes in V3:
 - Reviewed-by: Jan Beulich <jbeulich@suse.com>
 - unrealted change dropped in tiny64_defconfig
---
Changes in V2:
 - Nothing changed. Only rebase.
---
 xen/arch/riscv/Makefile       |  16 +++-
 xen/arch/riscv/arch.mk        |   4 -
 xen/arch/riscv/early_printk.c | 168 ----------------------------------
 xen/arch/riscv/stubs.c        |  23 -----
 4 files changed, 15 insertions(+), 196 deletions(-)

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 60afbc0ad9..81b77b13d6 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -12,10 +12,24 @@ $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
 
 $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
-	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(objtree)/common/symbols-dummy.o -o $(dot-target).0
+	$(NM) -pa --format=sysv $(dot-target).0 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).0.S
+	$(MAKE) $(build)=$(@D) $(dot-target).0.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(dot-target).0.o -o $(dot-target).1
+	$(NM) -pa --format=sysv $(dot-target).1 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).1.S
+	$(MAKE) $(build)=$(@D) $(dot-target).1.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
+	    $(dot-target).1.o -o $@
 	$(NM) -pa --format=sysv $@ \
 		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		> $@.map
+	rm -f $(@D)/.$(@F).[0-9]*
 
 $(obj)/xen.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed_dep,cpp_lds_S)
diff --git a/xen/arch/riscv/arch.mk b/xen/arch/riscv/arch.mk
index fabe323ec5..197d5e1893 100644
--- a/xen/arch/riscv/arch.mk
+++ b/xen/arch/riscv/arch.mk
@@ -19,7 +19,3 @@ riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += -march=$(riscv-march-y)$(has_zihintpause) -mstrict-align -mcmodel=medany
-
-# TODO: Drop override when more of the build is working
-override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
-override ALL_LIBS-y =
diff --git a/xen/arch/riscv/early_printk.c b/xen/arch/riscv/early_printk.c
index 60742a042d..610c814f54 100644
--- a/xen/arch/riscv/early_printk.c
+++ b/xen/arch/riscv/early_printk.c
@@ -40,171 +40,3 @@ void early_printk(const char *str)
         str++;
     }
 }
-
-/*
- * The following #if 1 ... #endif should be removed after printk
- * and related stuff are ready.
- */
-#if 1
-
-#include <xen/stdarg.h>
-#include <xen/string.h>
-
-/**
- * strlen - Find the length of a string
- * @s: The string to be sized
- */
-size_t (strlen)(const char * s)
-{
-    const char *sc;
-
-    for (sc = s; *sc != '\0'; ++sc)
-        /* nothing */;
-    return sc - s;
-}
-
-/**
- * memcpy - Copy one area of memory to another
- * @dest: Where to copy to
- * @src: Where to copy from
- * @count: The size of the area.
- *
- * You should not use this function to access IO space, use memcpy_toio()
- * or memcpy_fromio() instead.
- */
-void *(memcpy)(void *dest, const void *src, size_t count)
-{
-    char *tmp = (char *) dest, *s = (char *) src;
-
-    while (count--)
-        *tmp++ = *s++;
-
-    return dest;
-}
-
-int vsnprintf(char* str, size_t size, const char* format, va_list args)
-{
-    size_t i = 0; /* Current position in the output string */
-    size_t written = 0; /* Total number of characters written */
-    char* dest = str;
-
-    while ( format[i] != '\0' && written < size - 1 )
-    {
-        if ( format[i] == '%' )
-        {
-            i++;
-
-            if ( format[i] == '\0' )
-                break;
-
-            if ( format[i] == '%' )
-            {
-                if ( written < size - 1 )
-                {
-                    dest[written] = '%';
-                    written++;
-                }
-                i++;
-                continue;
-            }
-
-            /*
-             * Handle format specifiers.
-             * For simplicity, only %s and %d are implemented here.
-             */
-
-            if ( format[i] == 's' )
-            {
-                char* arg = va_arg(args, char*);
-                size_t arglen = strlen(arg);
-
-                size_t remaining = size - written - 1;
-
-                if ( arglen > remaining )
-                    arglen = remaining;
-
-                memcpy(dest + written, arg, arglen);
-
-                written += arglen;
-                i++;
-            }
-            else if ( format[i] == 'd' )
-            {
-                int arg = va_arg(args, int);
-
-                /* Convert the integer to string representation */
-                char numstr[32]; /* Assumes a maximum of 32 digits */
-                int numlen = 0;
-                int num = arg;
-                size_t remaining;
-
-                if ( arg < 0 )
-                {
-                    if ( written < size - 1 )
-                    {
-                        dest[written] = '-';
-                        written++;
-                    }
-
-                    num = -arg;
-                }
-
-                do
-                {
-                    numstr[numlen] = '0' + num % 10;
-                    num = num / 10;
-                    numlen++;
-                } while ( num > 0 );
-
-                /* Reverse the string */
-                for (int j = 0; j < numlen / 2; j++)
-                {
-                    char tmp = numstr[j];
-                    numstr[j] = numstr[numlen - 1 - j];
-                    numstr[numlen - 1 - j] = tmp;
-                }
-
-                remaining = size - written - 1;
-
-                if ( numlen > remaining )
-                    numlen = remaining;
-
-                memcpy(dest + written, numstr, numlen);
-
-                written += numlen;
-                i++;
-            }
-        }
-        else
-        {
-            if ( written < size - 1 )
-            {
-                dest[written] = format[i];
-                written++;
-            }
-            i++;
-        }
-    }
-
-    if ( size > 0 )
-        dest[written] = '\0';
-
-    return written;
-}
-
-void printk(const char *format, ...)
-{
-    static char buf[1024];
-
-    va_list args;
-    va_start(args, format);
-
-    (void)vsnprintf(buf, sizeof(buf), format, args);
-
-    early_printk(buf);
-
-    va_end(args);
-}
-
-#endif
-
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
index 529f1dbe52..bda35fc347 100644
--- a/xen/arch/riscv/stubs.c
+++ b/xen/arch/riscv/stubs.c
@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
 
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
 
-/*
- * max_page is defined in page_alloc.c which isn't complied for now.
- * definition of max_page will be remove as soon as page_alloc is built.
- */
-unsigned long __read_mostly max_page;
-
 /* time.c */
 
 unsigned long __ro_after_init cpu_khz;  /* CPU clock frequency in kHz. */
@@ -419,20 +413,3 @@ void __cpu_die(unsigned int cpu)
 {
     BUG_ON("unimplemented");
 }
-
-/*
- * The following functions are defined in common/irq.c, which will be built in
- * the next commit, so these changes will be removed there.
- */
-
-void cf_check irq_actor_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-}
-
-unsigned int cf_check irq_startup_none(struct irq_desc *desc)
-{
-    BUG_ON("unimplemented");
-
-    return 0;
-}
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Mon Feb 26 18:30:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 18:30:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685780.1067060 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1refk8-00036X-Ub; Mon, 26 Feb 2024 18:30:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685780.1067060; Mon, 26 Feb 2024 18:30:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1refk8-00036Q-RM; Mon, 26 Feb 2024 18:30:24 +0000
Received: by outflank-mailman (input) for mailman id 685780;
 Mon, 26 Feb 2024 18:30:23 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1refk7-00036G-Py; Mon, 26 Feb 2024 18:30:23 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1refk7-0002jJ-MO; Mon, 26 Feb 2024 18:30:23 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1refk7-0000Fe-Ar; Mon, 26 Feb 2024 18:30:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1refk7-00064n-AM; Mon, 26 Feb 2024 18:30:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=SkBsCW8g5FwsOhFye+k+bMrLpE1zM9QpkVHqv9uojgg=; b=wUu2QcNsbnibhx2Xg9ciZM1Skn
	u5pTHogNnS3HhGozPKV2YInwMHt7S4VmAWY1kYyT+ZO3EGO7Ovc9ZsgRJra0+xSQgSv0eUYr7gvv2
	7kiairuPzmV4rM9vdTQh9RFJ5BZwKiwI9LQ4zKg0MFgZRYbcSCbYpTUZzDYlATdpSHEo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184769-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184769: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=03fb5f503cb5faa0556b23d594496719ced6a11b
X-Osstest-Versions-That:
    xen=8de3afc0b402bc17f65093a53e5870862707a8c7
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 26 Feb 2024 18:30:23 +0000

flight 184769 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184769/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  03fb5f503cb5faa0556b23d594496719ced6a11b
baseline version:
 xen                  8de3afc0b402bc17f65093a53e5870862707a8c7

Last test of basis   184764  2024-02-26 10:02:04 Z    0 days
Testing same since   184769  2024-02-26 16:00:26 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  George Dunlap <george.dunlap@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   8de3afc0b4..03fb5f503c  03fb5f503cb5faa0556b23d594496719ced6a11b -> smoke


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 19:34:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 19:34:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685786.1067069 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1regjc-0008VR-9r; Mon, 26 Feb 2024 19:33:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685786.1067069; Mon, 26 Feb 2024 19:33:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1regjc-0008VK-7H; Mon, 26 Feb 2024 19:33:56 +0000
Received: by outflank-mailman (input) for mailman id 685786;
 Mon, 26 Feb 2024 19:33:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kdE/=KD=raptorengineering.com=sanastasio@srs-se1.protection.inumbo.net>)
 id 1regja-0008VB-EU
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 19:33:54 +0000
Received: from raptorengineering.com (mail.raptorengineering.com
 [23.155.224.40]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f6838935-d4dd-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 20:33:49 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by mail.rptsys.com (Postfix) with ESMTP id BBBBC8285534;
 Mon, 26 Feb 2024 13:33:47 -0600 (CST)
Received: from mail.rptsys.com ([127.0.0.1])
 by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032)
 with ESMTP id lBIrBmDgm9TK; Mon, 26 Feb 2024 13:33:46 -0600 (CST)
Received: from localhost (localhost [127.0.0.1])
 by mail.rptsys.com (Postfix) with ESMTP id 69E7C82856C2;
 Mon, 26 Feb 2024 13:33:46 -0600 (CST)
Received: from mail.rptsys.com ([127.0.0.1])
 by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id UsSaXHAZACpY; Mon, 26 Feb 2024 13:33:46 -0600 (CST)
Received: from [10.11.0.3] (5.edge.rptsys.com [23.155.224.38])
 by mail.rptsys.com (Postfix) with ESMTPSA id C18AB8285534;
 Mon, 26 Feb 2024 13:33:45 -0600 (CST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f6838935-d4dd-11ee-98f5-efadbce2ee36
DKIM-Filter: OpenDKIM Filter v2.10.3 mail.rptsys.com 69E7C82856C2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD;
	t=1708976026; bh=gynGyrftQROr1vQf3qgFIDI8jMyRF6PVZXguwa8Svi0=;
	h=Message-ID:Date:MIME-Version:To:From;
	b=mTEy0A4KKW3QGVAnTvDkZUhcmWy1lyDDH8gl7elEGL2fGbHMarzvBDbdJmv4z15Y6
	 slgvsCfpfsA+27swzJ2qZqN5JLA3gTBMe9VbdSU5WPEwfIzDy5sN/BTZsLOAy6pdg5
	 04hv5dT2CzuRM2FpznZzgsWzv4zj18iTjyTF1GP0=
X-Virus-Scanned: amavisd-new at rptsys.com
Message-ID: <95824bd5-63e5-4759-bd2f-c903543b450f@raptorengineering.com>
Date: Mon, 26 Feb 2024 13:33:44 -0600
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 2/6] xen/ppc: address violations of MISRA C:2012 Rule
 11.8.
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>,
 Simone Ballarin <simone.ballarin@bugseng.com>
Cc: xen-devel@lists.xenproject.org, consulting@bugseng.com,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
References: <cover.1702982442.git.maria.celeste.cesario@bugseng.com>
 <9270a4fe1712cff6a99e60c7862de1c1b2dde3d6.1702982442.git.maria.celeste.cesario@bugseng.com>
 <alpine.DEB.2.22.394.2402231518390.754277@ubuntu-linux-20-04-desktop>
From: Shawn Anastasio <sanastasio@raptorengineering.com>
In-Reply-To: <alpine.DEB.2.22.394.2402231518390.754277@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Hi Stefano,

Sorry for the delay on this.

Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com>

Best,
Shawn

On 2/23/24 5:19 PM, Stefano Stabellini wrote:
> Shawn,
> 
> I am thinking of committing this, if you disagree please say something
> in the next couple of days
> 
> 
> On Tue, 19 Dec 2023, Simone Ballarin wrote:
>> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
>>
>> The xen sources contain violations of MISRA C:2012 Rule 11.8 whose
>> headline states:
>> "A conversion shall not remove any const, volatile or _Atomic qualification
>> from the type pointed to by a pointer".
>>
>> Fix violation by adding missing const qualifier in cast.
>>
>> Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
>> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>> ---
>> Adaptation requested by the community to make the code more consistent.
>> ---
>>  xen/arch/ppc/include/asm/atomic.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/xen/arch/ppc/include/asm/atomic.h b/xen/arch/ppc/include/asm/atomic.h
>> index 64168aa3f1..fe778579fb 100644
>> --- a/xen/arch/ppc/include/asm/atomic.h
>> +++ b/xen/arch/ppc/include/asm/atomic.h
>> @@ -16,7 +16,7 @@
>>  
>>  static inline int atomic_read(const atomic_t *v)
>>  {
>> -    return *(volatile int *)&v->counter;
>> +    return *(const volatile int *)&v->counter;
>>  }
>>  
>>  static inline int _atomic_read(atomic_t v)
>> -- 
>> 2.40.0
>>


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 19:36:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 19:36:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685788.1067080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1regli-0000bM-Kz; Mon, 26 Feb 2024 19:36:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685788.1067080; Mon, 26 Feb 2024 19:36:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1regli-0000bF-IG; Mon, 26 Feb 2024 19:36:06 +0000
Received: by outflank-mailman (input) for mailman id 685788;
 Mon, 26 Feb 2024 19:36:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kdE/=KD=raptorengineering.com=sanastasio@srs-se1.protection.inumbo.net>)
 id 1reglg-0000b8-TF
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 19:36:04 +0000
Received: from raptorengineering.com (mail.raptorengineering.com
 [23.155.224.40]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 466f51ae-d4de-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 20:36:03 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by mail.rptsys.com (Postfix) with ESMTP id 0FA3B8285AB2;
 Mon, 26 Feb 2024 13:36:02 -0600 (CST)
Received: from mail.rptsys.com ([127.0.0.1])
 by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032)
 with ESMTP id PhloNTDn7NEj; Mon, 26 Feb 2024 13:36:00 -0600 (CST)
Received: from localhost (localhost [127.0.0.1])
 by mail.rptsys.com (Postfix) with ESMTP id D14AE82856C2;
 Mon, 26 Feb 2024 13:36:00 -0600 (CST)
Received: from mail.rptsys.com ([127.0.0.1])
 by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id zPYB3xai3P0j; Mon, 26 Feb 2024 13:36:00 -0600 (CST)
Received: from [10.11.0.3] (5.edge.rptsys.com [23.155.224.38])
 by mail.rptsys.com (Postfix) with ESMTPSA id C72098285534;
 Mon, 26 Feb 2024 13:35:59 -0600 (CST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 466f51ae-d4de-11ee-98f5-efadbce2ee36
DKIM-Filter: OpenDKIM Filter v2.10.3 mail.rptsys.com D14AE82856C2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD;
	t=1708976160; bh=bqCP0ZUNmVGkR+SUgOYwxxRk4p0Tqhhp0n0FaC8rdWo=;
	h=Message-ID:Date:MIME-Version:To:From;
	b=PYKRlDqwMUyvKSxn9w01LOJMfPXBtnSDADdVncQ1Znjy/JejgDBH+Fpo1oaRgYkeh
	 is6JB20TdzALh34YzE2nna4ZwSwinkyfXCVhs1vlmohK3XuYfw+pnqHnp4mfKObNDz
	 gmk8fV9zMUmW0SqwfbC1P20xzLP1Y2uFjcxm/BHo=
X-Virus-Scanned: amavisd-new at rptsys.com
Message-ID: <3f3c350a-0cc5-4899-aadb-fa3c9f3ef1dd@raptorengineering.com>
Date: Mon, 26 Feb 2024 13:35:59 -0600
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 1/3] xen: introduce Kconfig function alignment option
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-2-roger.pau@citrix.com>
From: Shawn Anastasio <sanastasio@raptorengineering.com>
In-Reply-To: <20240207145547.89689-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Roger,

On 2/7/24 8:55 AM, Roger Pau Monne wrote:
> And use it to replace CODE_ALIGN in assembly.  This allows to generaliz=
e the
> way the code alignment gets set across all architectures.
>=20
> No functional change intended.
>=20
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>

Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com>

Thanks,
Shawn


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 20:16:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 20:16:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685792.1067090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rehOp-0007LP-Ho; Mon, 26 Feb 2024 20:16:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685792.1067090; Mon, 26 Feb 2024 20:16:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rehOp-0007LI-Eg; Mon, 26 Feb 2024 20:16:31 +0000
Received: by outflank-mailman (input) for mailman id 685792;
 Mon, 26 Feb 2024 20:16:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=jmZe=KD=epam.com=prvs=8786cae52a=oleksandr_tyshchenko@srs-se1.protection.inumbo.net>)
 id 1rehOn-0007LB-A7
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 20:16:29 +0000
Received: from mx0a-0039f301.pphosted.com (mx0a-0039f301.pphosted.com
 [148.163.133.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ea2d8211-d4e3-11ee-98f5-efadbce2ee36;
 Mon, 26 Feb 2024 21:16:26 +0100 (CET)
Received: from pps.filterd (m0174677.ppops.net [127.0.0.1])
 by mx0a-0039f301.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id
 41QGjnTo017029; Mon, 26 Feb 2024 20:16:21 GMT
Received: from eur05-db8-obe.outbound.protection.outlook.com
 (mail-db8eur05lp2104.outbound.protection.outlook.com [104.47.17.104])
 by mx0a-0039f301.pphosted.com (PPS) with ESMTPS id 3wgq3r2ekj-1
 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
 Mon, 26 Feb 2024 20:16:20 +0000 (GMT)
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com (2603:10a6:10:ed::15)
 by AM9PR03MB7012.eurprd03.prod.outlook.com (2603:10a6:20b:2d4::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.34; Mon, 26 Feb
 2024 20:16:16 +0000
Received: from DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167]) by DB8PR03MB6108.eurprd03.prod.outlook.com
 ([fe80::82af:59a5:4446:9167%4]) with mapi id 15.20.7316.034; Mon, 26 Feb 2024
 20:16:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ea2d8211-d4e3-11ee-98f5-efadbce2ee36
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ZJardXiJnu5oc1atOU31Y0EEXVHIKGlTJPZJOY1vPds0iBghZkcWX6PFxOcHRIvxq6EnooUGIlehYlnS7rg8909PMhA8QrKsfQ4J6W6uXVysLfuacpQ8zV4rBntvtVgc9fkvJ0cNVz7tyHpUBfnVkkiVD1rMovYTMt06pabEd3XkevNxneyztobDYgo/+0TuWnP6TPJ+zJ2XhFxQSp4/dTPrDXGwtWCscL5ZSyv8O8p26iO2HT2jWBf0HbrVwVVP6myPNfWBgnRrerEqW3z97PAuhwwxG54jO6aMn/nDB7bbcjfFeR1z8ddUaPJsvLT3ttJ0DOqxDqDdIHRBw16j/g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=UMiuHiAdxoalaMQOpYHkmbGCocU+YMo5+uL0+5vZYMI=;
 b=Q9SdyKJJH+c1ekahQrudJlBCc2Ws23lQFOyYuoxTsXCb7T0zd/p1TC/xxh+/iLqRfbOr0Ao0zquvY7ao1/GsU3ZKzKzJYHOTV1SPyyZMm5YAOMN0aP1qTxTz64tNEQp58QWKpkTiYMdk1kVrnHXcAzGxQ9/n/Mp/rw+zK2s3RmAexbK2eK22HvJruzAqRrbhVwg93+ItClnhNwmTk4FFgDIXYbN5sCeDFRie+zWnBak3PswFAZgUKPOdsmKK02D5QPbZbmhpzE3RFHJbl2KbHA54PKmFhpgDLP0iamNxZlTGpw2MQ9Lj9naVw3QHcxFtKeFcS1/AKy3Y+ea/wAiyOw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com;
 dkim=pass header.d=epam.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UMiuHiAdxoalaMQOpYHkmbGCocU+YMo5+uL0+5vZYMI=;
 b=NGeiceIB4VBUep93d6J5h7vvZmEXD1q/yjfg2qDhoYalVFgmTLsMRhmzudNlIaSqZwQ3VySq/BSwrvFQsQzivfIxrr+SCtgL4qE9YYvFzOS/V9v0NTEff1VYH1Tfu9viKuUr0lehnWUYCtYj847C+ul+2r8FkhNgDtuCUvJ6FYPZag4P3N9vamfnFgIUDQFy2T8OzMKBGt4teUGiqUGNrLK4g3uGeckg7SfXkvMUTguNmEAkc+xRQnTFcnk1WgrAon/APQ9GDvviOPaXXBsNZey/hJoUBijAYisF5BjY58j9nfGZDRnOCDEka5LVit8ABvBf1zLwZg2/shrTQQYDGw==
From: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
To: Peng Fan <peng.fan@nxp.com>
CC: Julien Grall <julien@xen.org>,
        "vikram.garhwal@amd.com"
	<vikram.garhwal@amd.com>,
        Leo Yan <leo.yan@linaro.org>, Viresh Kumar
	<viresh.kumar@linaro.org>,
        Stewart Hildebrand <stewart.hildebrand@amd.com>,
        "sgarzare@redhat.com" <sgarzare@redhat.com>,
        "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>,
        Stefano Stabellini
	<stefano.stabellini@amd.com>
Subject: Re: question about virtio-vsock on xen
Thread-Topic: question about virtio-vsock on xen
Thread-Index: AdpmUSXKZvg8Aft6RRu9smUwoggtbwAUAHAAACuPlYAARFRfYAAj+0EA
Date: Mon, 26 Feb 2024 20:16:16 +0000
Message-ID: <e391b7fc-bf6f-4c3a-b444-5b320466294c@epam.com>
References: 
 <DU0PR04MB941734DA793D87B7FF3A491488552@DU0PR04MB9417.eurprd04.prod.outlook.com>
 <alpine.DEB.2.22.394.2402231332580.754277@ubuntu-linux-20-04-desktop>
 <ee69684a-5aad-417a-8522-1cd1b5322bbe@epam.com>
 <DU0PR04MB94175684F8B7678A7B885559885A2@DU0PR04MB9417.eurprd04.prod.outlook.com>
In-Reply-To: 
 <DU0PR04MB94175684F8B7678A7B885559885A2@DU0PR04MB9417.eurprd04.prod.outlook.com>
Accept-Language: en-US, ru-RU
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DB8PR03MB6108:EE_|AM9PR03MB7012:EE_
x-ms-office365-filtering-correlation-id: a9308b06-fcc4-4eb2-15a2-08dc3707c989
x-ld-processed: b41b72d0-4e9f-4c26-8a69-f949f367c91d,ExtAddr
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 
 X+nKJUQP5rK/GBOHDQyWT7mcZ/pO6XbwxFtZ+8PbC161/FgJ3IaDB3dT4ilul6fhwG9aBbwr4TSIPHttpWA99Dx1ACyAw2DkDrM2mIdAk+JjpAH2z+Fj68TtZrpIJP/pnoDuxVNTYOezyry1RPqd/AbR7Vrswxu4NBWeQQ27KpAM04vKjv2l7LhIJfI6Xe8qCOn5caed629AkewlELRLK3yJvysDNdi/2A1ROLXg1dl22S/rO0G4ztu1IiIfiHzu2bs2evxxd37wj/4iaSpCIklpPvd86NzuAQsbBhXyPYLBh6CHu6nzzbiMT8Gh2cDNWrkkPWPUqg0Ddw+t7pp9rgmngVJ7B9GMlNAywmu4HPx4dPeV/iLXyuM3g2xXyQoXyryTpkFUo8S6kzOsnmEK5YPxPRfIucIoiVb7GRCi9wTlLrhcSMIJ1fvcqOdvYfcCR3sA75ReuHHy2w//VQnQ9l/ekX3mFVQn0TUamhCKcZul0Wht5i6W1up8F7HNbmDtWG1tloiu9B1DxczQO3sf1Yi4/7e/tf/wev5KVr8vE//vWxeB1AexVFu/PC1vgk+p1srDk/ju61oaVZemzG+kZrNRDvNsGBQFrKEr4ZGyWMNAcq7r3tdvEi9kTRIH6xtDRt7ykvinHjJl9rfZ26LIeB4F9mXbMZ0ijaz/6msVy2GW4pIQnl4rGjSdLerS2E5rhVcGpTtimrmtN1vTEvcwGQ==
x-forefront-antispam-report: 
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR03MB6108.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?utf-8?B?eEkwVDNoM1RFU21MRjFjNEFBVUozNkZtcld4ZTVBRXZmMU5IY3ZEbkdGUGVT?=
 =?utf-8?B?TVhONDlSaWFkSUNFTXFpcWdCR0s4cDVSK3hPd2kyWUxaWmRuRXFuVHBQN3Bp?=
 =?utf-8?B?QmlNRUd5MSs5bFZLSUlOYU9uYVdwKzlHUm51eXNXTDZUUzRWYy9NZjhTMGtS?=
 =?utf-8?B?eXp2TnZtVEo4T2Y5ZHZjNUhGZlhqOFZtUUJrQVhmTVlwOXlnRUQvZzJFNEZt?=
 =?utf-8?B?Y0tWTExnT2FtVGpJNTdMZERKTkpMSUtLZ3pPY0hGUklqbEw4UXBhRkdtOUxH?=
 =?utf-8?B?cVNrL2xTV2gxZk9palBKWkpoVjkvL0NHOHA0bUFibjE3R0VUeW4xYUVpbm1S?=
 =?utf-8?B?QXNnUGNWZ0Q3SCt3ZVo2VUZ1cDNlTnZkYkh6eFdoZzJQOXQyenVtOFd1b0Ra?=
 =?utf-8?B?clY0WVZoNU53U3JLQ1MxdXorY0hSTS9QYkp2aVJFUGtoUVp5YmtzdzNMNDIw?=
 =?utf-8?B?RlFSVllRT3Z3VG9BcTdkL2h1d1NVczBVVmEveWpwOThFNmxsTzRrRlV4M1E2?=
 =?utf-8?B?T2pYM1RrNmFqWEx5amZwRjk0a2xVYmR2amlydzFqSHpIUUNpanFQcXhEbExH?=
 =?utf-8?B?MElqWUFHVTRNazk2VEhtWHVRV0RqblY5MENaWWEyR3JHNVEyS3lwSkE5UGNC?=
 =?utf-8?B?UXgyb01rMThUL3NKZE5CQ1VXSkJ0bk5RdWhvOVVpeDJ3UXcyaEtrYnRpU2tl?=
 =?utf-8?B?Y0JjazhYZXFxUnRBQXFReHBMS0hKbk9lWTNsaWlRVnFXUm9WRG1ncjdobXVj?=
 =?utf-8?B?NlMzM2N6L1R0ZnJlcm5NTUNnS2NsZ21lM0gzVHl0d2VhbWhjZEJOUE95NHhk?=
 =?utf-8?B?OHQ0Q29ac1hJalBwSkRDNUlwbEJxQVlaS1ZVN0tFbmFEMTMvK0pDbmtwLzYr?=
 =?utf-8?B?WlFmUlIxZ2RIYXVydkptZEFXczBoeE15ZFU4Qml1WTcwSUhUMCtPbnRPOW1l?=
 =?utf-8?B?WUpaUDRVOUNOMzBxWWlKV29SZm0yUW5LYUVjYVlzMWo2TGx1cG1YV1lvbXF3?=
 =?utf-8?B?cDBvL2VCZGZFek9YMGZoejBCUlU0UE9oS2lpV012WXVuMjVBamhKRGd0TGZj?=
 =?utf-8?B?dVVCZ3JmTFdsOU90RzdQbHdTNko0MytkSGcxTHRSd0ZDVFdONnRuWGhBRlFJ?=
 =?utf-8?B?ZmtMaklTQmZ6U1J0Ykh2WkpCMEtuK1M5a1d3V3lCQVc0YkZQWUdHakFBeDNE?=
 =?utf-8?B?Wk42RGY4WkNUY1JKcDZiT3RHYk9zcFlTZW5HY3hDczM4SHFqOEE5bmhYTWhl?=
 =?utf-8?B?RmMzT3d2L011Yks4SUN1dnNGcnE4Mk10NXhPaThmS01YL1FSRFVRUzhhYm1P?=
 =?utf-8?B?Y0IveFE4MkZMcnFPcENtUFB0Y1crb3l1KzdvR1Eyb1hXOWI3OEZENkxYZWg0?=
 =?utf-8?B?WHZsY0w4bEUzV25JcktpaGxMbXZEQlhYVSszamdQZTZwYzk3RE1oSUR2TVZs?=
 =?utf-8?B?RzUvOEVpZncveGpBN0o0R3dwNEV4VEZvUzhGbnNhdGZjY010bjM4TDh5aDFw?=
 =?utf-8?B?U0JQUkwxWG5DeXJRWTh2THI2dks3b0xWWXZMOEdHcEd6SHJQSVlaUFRKS2tk?=
 =?utf-8?B?eEVhVElsQmNPQ3hBR292T1dQLzVLK2dqYTVEa1htSitCVmYrbjRxMGRKRHVY?=
 =?utf-8?B?THpQZk1FWUJ6bWlsY0sxaldvT2Zlalg4UE11V25IWEp2WThZSDV1SEpyZnBY?=
 =?utf-8?B?d2JzV2pTSGE2NHhDR2NLQlpibXhCdGJpaHVrbGVvVEN5b1hBWHh2cEVnQmhv?=
 =?utf-8?B?M1hkeC9EU1ZhNEpRT0hxQUI0SzlTeUdjdzNHYlFmRDU0WmI1RmR3MjBDQ2ZH?=
 =?utf-8?B?c1lrVngxSWlmbmx3STNxcWQwaFBhT2cvS3h5OUxuWE5YaVBIQzVGa3NSMkNY?=
 =?utf-8?B?QlZwdVZEOFVlOWxXamRPY21DckI4NzhqQk1Jc3ZsYnl6M3lsaVBEdStjUS9q?=
 =?utf-8?B?dUFINnVFVkQ0Wm5lNlJjOU10Y0xCcFM2U0tCMzZZNE1kc1FYNFdwbnFWK1VT?=
 =?utf-8?B?OVZNMVlCNDgwa3RtM3dpMWErV0Mzb1h5bnp3NGhxcEIvKzFsdERXMS9SQS9F?=
 =?utf-8?B?WjZFbmR2K0ZhR2dpdmtOWkR0cXM5QjZnMCswcFBwV01sWE81SXVNN3RKaWRx?=
 =?utf-8?B?bE5jd1l0YTZFNDI3bjNGWm0vaXFMdnY5S0kyRDZGSDBjREJnKzRVOTNCbEVC?=
 =?utf-8?Q?kb3rT+mnF2TX/t9zFzWoBoM=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <42A4B907A6A6B842BEC9107627589EFA@eurprd03.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB8PR03MB6108.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a9308b06-fcc4-4eb2-15a2-08dc3707c989
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Feb 2024 20:16:16.5046
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: AqGVEsvIryxCkectFi1h2eGepB0RvpwDLzxMykB1hYX5bLRNCjgwemaelccAG+o/Jf+lKnmmA5b/mSjSZHKXy/SfOcN339iFHEmrOXOrxwY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR03MB7012
X-Proofpoint-GUID: urlTz-QlCRM3X0TNka4jqFp0Pg3JWxpK
X-Proofpoint-ORIG-GUID: urlTz-QlCRM3X0TNka4jqFp0Pg3JWxpK
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-26_11,2024-02-26_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 phishscore=0
 lowpriorityscore=0 clxscore=1015 spamscore=0 impostorscore=0
 malwarescore=0 mlxscore=0 adultscore=0 priorityscore=1501 mlxlogscore=934
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.19.0-2402120000 definitions=main-2402260154

DQoNCk9uIDI2LjAyLjI0IDA1OjA5LCBQZW5nIEZhbiB3cm90ZToNCj4gSGkgT2xla3NhbmRyLA0K
DQpIZWxsbyBQZW5nDQoNCg0KW3NuaXBdDQoNCj4+DQo+PiAgICAgLi4uIFBlbmcsIHdlIGhhdmUg
dmhvc3QtdnNvY2sgKGFuZCB2aG9zdC1uZXQpIFhlbiBQb0MuIEFsdGhvdWdoIGl0IGlzIG5vbi0N
Cj4+IHVwc3RyZWFtYWJsZSBpbiBpdHMgY3VycmVudCBzaGFwZSAoYmFzZWQgb24gb2xkIExpbnV4
IHZlcnNpb24sIHJlcXVpcmVzIHNvbWUNCj4+IHJld29yayBhbmQgcHJvcGVyIGludGVncmF0aW9u
LCBtb3N0IGxpa2VseSByZXF1aXJlcyBpbnZvbHZpbmcgUWVtdSBhbmQNCj4+IHByb3RvY29sIGNo
YW5nZXMgdG8gcGFzcyBhbiBhZGRpdGlvbmFsIGluZm8gdG8gdmhvc3QpLCBpdCB3b3JrcyB3aXRo
IExpbnV4DQo+PiB2NS4xMCArIHBhdGNoZWQgUWVtdSB2Ny4wLCBzbyB5b3UgY2FuIHJlZmVyIHRv
IHRoZSBZb2N0byBtZXRhIGxheWVyIHdoaWNoDQo+PiBjb250YWlucyBrZXJuZWwgcGF0Y2hlcyBm
b3IgdGhlIGRldGFpbHMgWzFdLg0KPiANCj4gVGhhbmtzIGZvciB0aGUgcG9pbnRlciwgSSBhbSBy
ZWFkaW5nIHRoZSBjb2RlLg0KPiANCj4+DQo+PiBJbiBhIG51dHNoZWxsLCBiZWZvcmUgYWNjZXNz
aW5nIHRoZSBndWVzdCBkYXRhIHRoZSBob3N0IG1vZHVsZSBuZWVkcyB0byBtYXANCj4+IGRlc2Ny
aXB0b3JzIGluIHZpcnRpbyByaW5ncyB3aGljaCBjb250YWluIGVpdGhlciBndWVzdCBncmFudCBi
YXNlZCBETUENCj4+IGFkZHJlc3NlcyAoYnkgdXNpbmcgWGVuIGdyYW50IG1hcHBpbmdzKSBvciBn
dWVzdCBwc2V1ZG8tcGh5c2ljYWwgYWRkcmVzc2VzDQo+PiAoYnkgdXNpbmcgWGVuIGZvcmVpZ24g
bWFwcGluZ3MpLiBBZnRlciBhY2Nlc3NpbmcgdGhlIGd1ZXN0IGRhdGEgdGhlIGhvc3QNCj4+IG1v
ZHVsZSBuZWVkcyB0byB1bm1hcCB0aGVtLg0KPiANCj4gT2ssIEkgdGhvdWdodCAgdGhlIGN1cnJl
bnQgeGVuIHZpcnRpbyBjb2RlIGFscmVhZHkgbWFwIGV2ZXJ5IHJlYWR5Lg0KPiANCg0KSXQgZG9l
cywgYXMgeW91IHNhaWQgdGhlIHZpcnRpby1ibGstcGNpIHdvcmtlZCBpbiB5b3VyIGVudmlyb25t
ZW50LiBCdXQgDQp2aG9zdCgtdnNvY2spIGlzIGEgc3BlY2lhbCBjYXNlLCB1bmxpa2UgZm9yIHZp
cnRpby1ibGstcGNpIHdoZXJlIHRoZSANCndob2xlIGJhY2tlbmQgcmVzaWRlcyBpbiBRZW11LCBo
ZXJlIHdlIGhhdmUgYSBzcGxpdCBtb2RlbC4gQXMgSSANCnVuZGVyc3RhbmQgdGhlIFFlbXUgcGVy
Zm9ybXMgb25seSBpbml0aWFsIHNldHVwL2NvbmZpZ3VyYXRpb24gdGhlbiANCm9mZmxvYWRzIHRo
ZSBJL08gcHJvY2Vzc2luZyB0byBhIHNlcGFyYXRlIGVudGl0eSB3aGljaCBpcyB0aGUgTGludXgg
DQptb2R1bGUgaW4gdGhhdCBwYXJ0aWN1bGFyIGNhc2Uu


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 22:01:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 22:01:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685800.1067116 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rej26-00057x-AN; Mon, 26 Feb 2024 22:01:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685800.1067116; Mon, 26 Feb 2024 22:01:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rej26-00057q-7E; Mon, 26 Feb 2024 22:01:10 +0000
Received: by outflank-mailman (input) for mailman id 685800;
 Mon, 26 Feb 2024 22:01:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cwyd=KD=amd.com=stefano.stabellini@srs-se1.protection.inumbo.net>)
 id 1rej24-00057j-VZ
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 22:01:09 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2415::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 886ffd5b-d4f2-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 23:01:04 +0100 (CET)
Received: from DS7PR03CA0211.namprd03.prod.outlook.com (2603:10b6:5:3ba::6) by
 DM4PR12MB6110.namprd12.prod.outlook.com (2603:10b6:8:ad::7) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7316.34; Mon, 26 Feb 2024 22:01:00 +0000
Received: from DS1PEPF0001708F.namprd03.prod.outlook.com
 (2603:10b6:5:3ba:cafe::69) by DS7PR03CA0211.outlook.office365.com
 (2603:10b6:5:3ba::6) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend
 Transport; Mon, 26 Feb 2024 22:01:00 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS1PEPF0001708F.mail.protection.outlook.com (10.167.17.139) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Mon, 26 Feb 2024 22:01:00 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 26 Feb
 2024 16:00:59 -0600
Received: from ubuntu-20.04.2-arm64.shared (10.180.168.240) by
 SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2507.35 via Frontend Transport; Mon, 26 Feb 2024 16:00:59 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 886ffd5b-d4f2-11ee-8a58-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Dsjj/4A+NE80GSPot01tzEdEbDpPu+Ypnml1i6RBf3s7n3lPA507lJ0ev79u+CukmH5fFRqhS7QJdaTP+OBfC2eVDQX+ZFQ6FuxrqtICtVO/c7FwQxY3OVRtIcLFivXSwAbuND9v37d9Ts79eoXfEoZLljkNiyXRpB8s3YA3CkbAGOZXs3gqRfBDKGoPndTW/YzUun08j/hsu6hQKLESzr46Bb4uKgxkH5QAHLYOaDgFkkle55PDhiosptT/pmZcCglVfM7e5lHSUcjkljptbLZmuEWMwqacH9Pqw0Fl9UpSog3NDJXL7HVZC1OYlHVr+tPJ0c8ReeMGiDbxakOjmQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=4/5bQQxxeS8lpi2svK66Drb1FT6L3WH3kQkmcgWzcsU=;
 b=nIcHJsi9E+CFI8OFaQ0apwTiaC8abrTrTFdVrA9WFt/4S4WKOV056VQUHzZgHtkCXzkbIAYXEqhj6Rbnswr9uDE4q3H+q59drgyZro4UrkPQjnIEWvd4pg1FO108VPtZ9X6XxKZAuUiV/H52Lj7cxIJTDla3gPHniqGVwxprLgUiwFxp1s40566zbHejsygobGyNBO6ijA8PnBT28datSVCK5MepndGV4lW7FLVrXXCeSJsMaoDLG2IieoIfHYUCXmUJKr8sO+3tTsEzNb98Ga/RwAEP51ZtCZwpGUiM2i8Xo1kd4WNPvmTOIUYiCag/K+g+3ir4Qb1qR0X4yIh59Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=nxp.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4/5bQQxxeS8lpi2svK66Drb1FT6L3WH3kQkmcgWzcsU=;
 b=yBf/rJn6iqoqMkQLqspR3so2V6ntkCtU/eRXp76gWYMHsU4spUcAA4a64iYuA5Uz3o+vpTbhudCjVIfV0gh140pBxIeboxxeo7hCq1BLD8D9i3WoP8dtQXCxd4aFtUF7/sRUml9sbl1xScDjrDUOMBNnyT3xr5g0opXfEWSItH8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Date: Mon, 26 Feb 2024 14:00:58 -0800
From: Stefano Stabellini <stefano.stabellini@amd.com>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Peng Fan <peng.fan@nxp.com>
CC: Stefano Stabellini <stefano.stabellini@amd.com>, Julien Grall
	<julien@xen.org>, "vikram.garhwal@amd.com" <vikram.garhwal@amd.com>,
	"oleksandr_tyshchenko@epam.com" <oleksandr_tyshchenko@epam.com>, Leo Yan
	<leo.yan@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>,
	"stewart.hildebrand@amd.com" <stewart.hildebrand@amd.com>,
	"sgarzare@redhat.com" <sgarzare@redhat.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>
Subject: RE: question about virtio-vsock on xen
In-Reply-To: <DU0PR04MB941746B1F385B6A514704F2B885A2@DU0PR04MB9417.eurprd04.prod.outlook.com>
Message-ID: <alpine.DEB.2.22.394.2402261359280.247676@ubuntu-linux-20-04-desktop>
References: <DU0PR04MB941734DA793D87B7FF3A491488552@DU0PR04MB9417.eurprd04.prod.outlook.com> <alpine.DEB.2.22.394.2402231332580.754277@ubuntu-linux-20-04-desktop> <DU0PR04MB941746B1F385B6A514704F2B885A2@DU0PR04MB9417.eurprd04.prod.outlook.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Received-SPF: None (SATLEXMB03.amd.com: stefano.stabellini@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS1PEPF0001708F:EE_|DM4PR12MB6110:EE_
X-MS-Office365-Filtering-Correlation-Id: 24dee0e3-db02-4e59-e044-08dc37166b39
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	HnvfMCvHALZy7fJ2jNIzd67kEsPDgSb3uS9EZGZ4JKqAEpdrz7+AkKKc3wBtXXev+E9xujUDyZMtWYH+3qQhq4EdG2vtuAvv9zjH8CVhuSJYJKuZf97xS1rFlVmSTaCvK9IQlia56c/yos0qyx5xSorWxyH0q9a4TEdx774Rt3oRALcqAWJUDChfh3J0Vmu8yGBHreJuKaBDE2MUDH2Fdxv9rIbcBuRotsg2FvMJEHayM/9or1QzW7jtK94MQrmV2WaPuIwYvoTO9sTyWNGX/KLIXJFI885vMdNgbWSgTgghvJ8cgUZHTsglhkJ2RQl4Vjn/eyeYaCyrrY+5wJX22Rw81u6QD9cIlSPTrsMwQHRDWUxh5hlxuuq11f9pF7euD4B25UdVXOTCYspZx6RDhpNqYMImB5WZanHl/1xrzREwXE9/6+IdQhzyQZ5sjZzNC9Gp0vCwf6QhLGlcBN399Ifr2o1+8g55fpzvC3cgLat5dHPm3eggrie/M0jAqxTbYpUCuRUM+3vtQuJw080d/t46RsAmudh06jgfR4oUKrdB+Z2ko2NC79eaR/KTHuO/UHSJrcuETZmkTW1k5PW63NZ9GC2sluJEtcqzY7u2olpLhwWNjjzZaAUUeiSmmivz+9hY00RMHctoiprgOgZe6LQZoOv4+I4jNnTHhXlTzx8Fw0oR0z+6aNYbmPsQj1YcJSHooHEEAKcn5JYU3T6/htFKGxXFqJUFb0P3FEc2O29VU/PfBYwiFksvmDdQL9Jf
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2024 22:01:00.6749
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 24dee0e3-db02-4e59-e044-08dc37166b39
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS1PEPF0001708F.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6110

On Mon, 26 Feb 2024, Peng Fan wrote:
> Hi Stefano,
> 
> > Subject: Re: question about virtio-vsock on xen
> > 
> > Hi Peng,
> > 
> > We haven't tried to setup virtio-vsock yet.
> > 
> > In general, I am very supportive of using QEMU for virtio backends. We use
> > QEMU to provide virtio-net, virtio-block, virtio-console and more.
> 
> Would you mind share how to setup virtio-console using qemu + xen?

Vikram (CCed) has been working on it and should be able to share more
details

 
> > However, typically virtio-vsock comes into play for VM-to-VM
> > communication, which is different. Going via QEMU in Dom0 just to have 1
> > VM communicate with another VM is not an ideal design: it adds latency and
> > uses resources in Dom0 when actually we could do without it.
> > 
> > A better model for VM-to-VM communication would be to have the VM talk
> > to each other directly via grant table or pre-shared memory (see the static
> > shared memory feature) or via Xen hypercalls (see Argo.)
> 
> The goal is to make android trout VM run with XEN + i.MX95, so need vsock.

I am not familiar with the details of Android Trout... Where is vsock
used? Just asking for my own understanding.


> > For a good Xen design, I think the virtio-vsock backend would need to be in
> > Xen itself (the hypervisor).
> > 
> > Of course that is more work and it doesn't help you with the specific question
> > you had below :-)
> > 
> > For that, I don't have a pointer to help you but maybe others in CC have.
> > 
> > Cheers,
> > 
> > Stefano
> > 
> > 
> > On Fri, 23 Feb 2024, Peng Fan wrote:
> > > Hi All,
> > >
> > > Has anyone make virtio-vsock on xen work? My dm args as below:
> > >
> > > virtio = [
> > >
> > 'backend=0,type=virtio,device,transport=pci,bdf=05:00.0,backend_type=qem
> > u,grant_usage=true'
> > > ]
> > > device_model_args = [
> > > '-D', '/home/root/qemu_log.txt',
> > > '-d',
> > > 'trace:*vsock*,trace:*vhost*,trace:*virtio*,trace:*pci_update*,trace:*
> > > pci_route*,trace:*handle_ioreq*,trace:*xen*',
> > > '-device',
> > > 'vhost-vsock-pci,iommu_platform=false,id=vhost-vsock-pci0,bus=pcie.0,a
> > > ddr=5.0,guest-cid=3']
> > >
> > > During my test, it always return failure in dom0 kernel in below code:
> > >
> > > vhost_transport_do_send_pkt {
> > > ...
> > >                nbytes = copy_to_iter(hdr, sizeof(*hdr), &iov_iter);
> > >                 if (nbytes != sizeof(*hdr)) {
> > >                         vq_err(vq, "Faulted on copying pkt hdr %x %x %x %px\n",
> > nbytes, sizeof(*hdr),
> > > __builtin_object_size(hdr, 0), &iov_iter);
> > >                         kfree_skb(skb);
> > >                         break;
> > >                 }
> > > }
> > >
> > > I checked copy_to_iter, it is copy data to __user addr, but it never
> > > pass, the copy to __user addr always return 0 bytes copied.
> > >
> > > The asm code "sttr x7, [x6]" will trigger data abort, the kernel will
> > > run into do_page_fault, but lock_mm_and_find_vma report it is
> > > VM_FAULT_BADMAP, that means the __user addr is not mapped, no vma
> > has this addr.
> > >
> > > I am not sure what may cause this. Appreciate if any comments.
> > >
> > > BTW: I tested blk pci, it works, so the virtio pci should work on my setup.
> > >
> > > Thanks,
> > > Peng.
> > >
> 


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 22:27:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 22:27:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685806.1067126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rejRV-00087x-7d; Mon, 26 Feb 2024 22:27:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685806.1067126; Mon, 26 Feb 2024 22:27:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rejRV-00087q-4K; Mon, 26 Feb 2024 22:27:25 +0000
Received: by outflank-mailman (input) for mailman id 685806;
 Mon, 26 Feb 2024 22:27:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rejRU-00087g-BA; Mon, 26 Feb 2024 22:27:24 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rejRU-0007b0-9V; Mon, 26 Feb 2024 22:27:24 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rejRT-0001D4-SD; Mon, 26 Feb 2024 22:27:23 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rejRT-0000Bw-Rj; Mon, 26 Feb 2024 22:27:23 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=VO1n+Nh3aOZ/YVYqiXcGTEEaki/2+/fpq02KbDdCv3U=; b=ITwZn3o4CiO3G1LfoW2QjTPgxj
	yQ95T1u1EQ+ruhBgsJ7hC4iqfD6fIKZV0+jPNsZttXhFvbU6tkshFfqpILB9z4zwKe79URLbVkeXv
	ShKXB9JDJycdCawg587DemfDEYNso3rGqXRdDDrgz8NYtEoyw8HtvDHBj00obwd7P0RU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184772-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184772: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=68238d4f948069fc2c6b9cc13863bdced52a84d0
X-Osstest-Versions-That:
    ovmf=44fdc4f3983be77dda709d7a0c3fb8905fbf920b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 26 Feb 2024 22:27:23 +0000

flight 184772 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184772/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 68238d4f948069fc2c6b9cc13863bdced52a84d0
baseline version:
 ovmf                 44fdc4f3983be77dda709d7a0c3fb8905fbf920b

Last test of basis   184768  2024-02-26 15:43:07 Z    0 days
Testing same since   184772  2024-02-26 19:43:08 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Junfeng Guan <junfengx.guan@intel.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   44fdc4f398..68238d4f94  68238d4f948069fc2c6b9cc13863bdced52a84d0 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 22:50:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 22:50:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685811.1067136 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rejnJ-0002U3-Uw; Mon, 26 Feb 2024 22:49:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685811.1067136; Mon, 26 Feb 2024 22:49:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rejnJ-0002Tw-SG; Mon, 26 Feb 2024 22:49:57 +0000
Received: by outflank-mailman (input) for mailman id 685811;
 Mon, 26 Feb 2024 22:49:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gV/7=KD=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rejnJ-0002Tq-0b
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 22:49:57 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5b0324d9-d4f9-11ee-8a58-1f161083a0e0;
 Mon, 26 Feb 2024 23:49:55 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 48AC5CE130D;
 Mon, 26 Feb 2024 22:49:49 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6A311C433C7;
 Mon, 26 Feb 2024 22:49:47 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b0324d9-d4f9-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708987788;
	bh=PdsaxXnClbW/2yke9ROjV1q7f3C3QN3so1Jxm111Ncw=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=oxGBDbE8diFp73R5jsD9mgzCuAy0l6HveyBJgqM5DUjJtk98OVf3UfbkDQu0E0q6W
	 SHqNmUmGhIsyTIJzQ0V/l1zmvgKVgh4T7S/m7e7C+CRWxDcnBtSXlDqZYL0W1mOKdn
	 d8C0ccgM9A6UO13/UL9tpRDFTLIrnrrF540fJW2A94ANMqYR3hmSpbzswzZG1h0AGH
	 Hf16d/SP5FNtbalUY9wInRoHlz8pPo066/WmUphNNqo25GIeepanbCF8U5GspzUhpe
	 ndS7qWiF3kNB4JLj0bA018Ise3xVClFHQmP4/2igFcJ4ubqKvM0lwUatiuRseO0hfp
	 sL0nJs23d53LQ==
Date: Mon, 26 Feb 2024 14:49:46 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Nicola Vetrini <nicola.vetrini@bugseng.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH 1/2] xen/console: drop return value from
 consoled_guest_rx/tx
In-Reply-To: <a8bab8eb-a735-4440-a075-7c62a12d4e4d@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402261448170.247676@ubuntu-linux-20-04-desktop>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com> <4998ec735bd7e5a50a229507e2b92ae56ec1ba4b.1708680104.git.nicola.vetrini@bugseng.com> <alpine.DEB.2.22.394.2402231456350.754277@ubuntu-linux-20-04-desktop> <004877c8-9aea-44b9-aa93-65665b0d3a8d@suse.com>
 <b417d7a330addd295b3cbbbac4bed2a4@bugseng.com> <a8bab8eb-a735-4440-a075-7c62a12d4e4d@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 26 Feb 2024, Jan Beulich wrote:
> On 26.02.2024 09:23, Nicola Vetrini wrote:
> > On 2024-02-26 09:00, Jan Beulich wrote:
> >> On 23.02.2024 23:56, Stefano Stabellini wrote:
> >>> On Fri, 23 Feb 2024, Nicola Vetrini wrote:
> >>>> These functions never saw a usage of their return value since
> >>>> they were introduced, so it can be dropped since their usages
> >>>> violate MISRA C Rule 17.7:
> >>>> "The value returned by a function having non-void return type shall 
> >>>> be used".
> >>>>
> >>>> No functional change.
> >>>>
> >>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> >>>
> >>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> >>
> >> The cleanup is certainly okay, but would one of you mind clarifying in 
> >> how
> >> far this code is relevant for certification? I don't expect there are 
> >> plans
> >> to run shim Xen in any projected production uses for which 
> >> certification is
> >> relevant? (The subject prefix is also unnecessarily wide here, when 
> >> it's
> >> only daemon code which is affected, not console code in general.)
> >>
> > 
> > I agree on the subject prefix being too wide. The configuration that 
> > uses consoled_guest_tx is #ifdef-ed for x86, so even in configurations 
> > that may never reach this condition this is relevant, unless its #ifdef 
> > is restricted to cases where the call may actually be reachable.
> 
> Hmm, I see. There are contradicting goals here then: It being just X86 is
> to reduce the risk of someone overlooking a build breakage they may
> introduce. Whereas for certification it's quite the other way around: We'd
> like to "hide" as much code as possible.
> 
> Really I would have been inclined to suggest to drop the #ifdef, if
> possible even without replacing by IS_ENABLED(), but instead leveraging
> that pv_shim ought to be compile-time false whenever CONFIG_PV_SHIM=n.

This is OK


> After all that's a pattern we've been trying to follow. But with your
> observation is becomes questionable whether extending use of IS_ENABLED()
> is actually going to be helpful. Stefano - perhaps something to discuss
> on one of the next meetings?

Yes. I checked with the safety manager and his opinion is that
IS_ENABLED() is OK to use as a way to disable code from a safety
perspective.


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 22:57:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 22:57:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685820.1067169 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rejuk-0004D0-Vj; Mon, 26 Feb 2024 22:57:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685820.1067169; Mon, 26 Feb 2024 22:57:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rejuk-0004Ct-Sm; Mon, 26 Feb 2024 22:57:38 +0000
Received: by outflank-mailman (input) for mailman id 685820;
 Mon, 26 Feb 2024 22:57:37 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rejuj-0004Ch-2T; Mon, 26 Feb 2024 22:57:37 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rejuj-0008Eb-0E; Mon, 26 Feb 2024 22:57:37 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rejui-00021R-Ij; Mon, 26 Feb 2024 22:57:36 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rejui-0004Q0-ID; Mon, 26 Feb 2024 22:57:36 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=TS8PstvXLzb40f3hssKwEpZNGyjxxuSRhrM4JD2TB7E=; b=N+giPyUYMi0lCSPBksvCgAvjdt
	AkStTQRuJo1jaYurtmGkfG1mIxoKo5+NcGM6pxWNYhQEZDysC0J06TQ6+II5tPxwxgfbva1mKVGwL
	S2x3t4fbPcEGXFnmoff76f7IIwoxfQrZecb4Qrr7aIh0zVv6TFk3buoVXSLR6lGsMZu0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184767-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184767: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=8de3afc0b402bc17f65093a53e5870862707a8c7
X-Osstest-Versions-That:
    xen=92babc88f67ed0ef3dc575a8b9534040274678ee
From: osstest service owner <osstest-admin@xenproject.org>
Date: Mon, 26 Feb 2024 22:57:36 +0000

flight 184767 xen-unstable real [real]
flight 184773 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184767/
http://logs.test-lab.xenproject.org/osstest/logs/184773/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 12 windows-install fail pass in 184773-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop  fail in 184773 like 184763
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184763
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184763
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184763
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184763
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184763
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184763
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184763
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184763
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184763
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184763
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184763
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  8de3afc0b402bc17f65093a53e5870862707a8c7
baseline version:
 xen                  92babc88f67ed0ef3dc575a8b9534040274678ee

Last test of basis   184763  2024-02-26 01:52:19 Z    0 days
Testing same since   184767  2024-02-26 13:08:57 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Daniel P. Smith <dpsmith@apertussolutions.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Nicola Vetrini <nicola.vetrini@bugseng.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   92babc88f6..8de3afc0b4  8de3afc0b402bc17f65093a53e5870862707a8c7 -> master


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 23:07:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 23:07:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685828.1067180 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rek4D-0005uv-1f; Mon, 26 Feb 2024 23:07:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685828.1067180; Mon, 26 Feb 2024 23:07:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rek4C-0005uo-U2; Mon, 26 Feb 2024 23:07:24 +0000
Received: by outflank-mailman (input) for mailman id 685828;
 Mon, 26 Feb 2024 23:07:23 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gV/7=KD=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rek4B-0005ui-BK
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 23:07:23 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cb00a9a7-d4fb-11ee-8a58-1f161083a0e0;
 Tue, 27 Feb 2024 00:07:22 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id DF163CE0E03;
 Mon, 26 Feb 2024 23:07:15 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8FFC9C433C7;
 Mon, 26 Feb 2024 23:07:14 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb00a9a7-d4fb-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708988835;
	bh=MfBfvvSQBzN8cnVm0q01rEP9iQuXcny525SJfVlZ1SU=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=WvTWzoocnBvyNuHdXdQ4al4hofFIqvtMoqMKVO3iBj7yMuNObvOkKEcFPQysRKLWv
	 3FgrA5rjSR6E9U8SKDvlQJzQFoZWY7irZLhNKeH/UaIdsCAv7bjHCQFhwdu2KAZ272
	 LFEacbAlTQIz8cjMOVr9NDdIIJhI6ZBQbry00pEkXSfprvUHUFNfu7Vxx6CQmmaet7
	 9vwG8QbKeTR9QCdLxSIkTM1UG82GjtMRWoyo0uwCBWghsM9+mxBzRCaoSfnLAMBisc
	 vkR8INXWgT9NbCnqgK9ZSpRkld8IOjNDRMjGt9EQ+RsmQRGNfaio4Shg4ZzLizPeg+
	 4kENyH6lYkJYQ==
Date: Mon, 26 Feb 2024 15:07:13 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Olaf Hering <olaf@aepfle.de>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, Doug Goldstein <cardoe@cardoe.com>
Subject: Re: [PATCH v1] automation: remove bin86/dev86 from tumbleweed
 image
In-Reply-To: <20240224064055.69dc7fb8.olaf@aepfle.de>
Message-ID: <alpine.DEB.2.22.394.2402261507040.247676@ubuntu-linux-20-04-desktop>
References: <20231213202515.30587-1-olaf@aepfle.de> <alpine.DEB.2.22.394.2402231522020.754277@ubuntu-linux-20-04-desktop> <20240224064055.69dc7fb8.olaf@aepfle.de>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Sat, 24 Feb 2024, Olaf Hering wrote:
> Fri, 23 Feb 2024 15:22:38 -0800 (PST) Stefano Stabellini <sstabellini@kernel.org>:
> 
> > Do you have a successful gitlab pipeline with this patch applied that
> > you can give me as proof of testing and success?
> 
> Yes, all of them since the patch went out.

Great thanks!

Acked-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Mon Feb 26 23:49:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Mon, 26 Feb 2024 23:49:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685832.1067190 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rekib-0002S2-2u; Mon, 26 Feb 2024 23:49:09 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685832.1067190; Mon, 26 Feb 2024 23:49:09 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rekib-0002Rv-02; Mon, 26 Feb 2024 23:49:09 +0000
Received: by outflank-mailman (input) for mailman id 685832;
 Mon, 26 Feb 2024 23:49:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=gV/7=KD=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rekiY-0002Rl-Uh
 for xen-devel@lists.xenproject.org; Mon, 26 Feb 2024 23:49:06 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9e006ce1-d501-11ee-98f5-efadbce2ee36;
 Tue, 27 Feb 2024 00:49:02 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id EC2E061241;
 Mon, 26 Feb 2024 23:49:00 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 57138C433F1;
 Mon, 26 Feb 2024 23:48:59 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e006ce1-d501-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708991340;
	bh=PjO7PQQo3rEX1iWh2Ea74qiVJOGbFJIdkegOczGU+Ts=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JbuhtV30nPy3DGuY/NyGUWKIkDUTh2mymmgu9EpChSkfZEkit5MBphuM1ftmQkuAB
	 A2djkv9QnzgqV+bgnucNnCJjVcv/3pyuT3V8HJWd8y+ru8bsqcJBJSIG3kQ2jktoJV
	 Q+ZbvrM2vClEH3jHSyQr6+wPXK9PuH7s0ON5WSShexZCzDc5RRkkb2yZL+PSof/0As
	 78iH0yuNOwwAPLOOw42cRPHoIKG+K+Dvo3qiARPHQ8LhLtrlDqt5VZ/LtjUhDXEWsR
	 myu31qTL3ZOeuF3ZP0lMN2hQFq9hTsUgUuWVDmPCAkSWmMwujIiumMbXyJrBE6HRRN
	 SQOehvyCtEnVw==
Date: Mon, 26 Feb 2024 15:48:57 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Simone Ballarin <simone.ballarin@bugseng.com>, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 3/4] xen/include: add pure and const attributes
In-Reply-To: <cb463469-51e1-4639-bf56-36fe9051d1b3@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402261511010.247676@ubuntu-linux-20-04-desktop>
References: <cover.1697638210.git.simone.ballarin@bugseng.com> <fd5421162a00aa782e0776324ff6497193c1e3d3.1697638210.git.simone.ballarin@bugseng.com> <89778285-5cba-8fb5-70bc-710b6dd30a10@suse.com> <0032186f-80c7-4dba-b46e-10d4a8e2a8cb@bugseng.com>
 <e8bf9817-fd54-9bf4-4302-dcee682f9172@suse.com> <alpine.DEB.2.22.394.2310231417260.3516@ubuntu-linux-20-04-desktop> <8f8141e2-c55c-2e58-a078-d5d8764a02df@suse.com> <alpine.DEB.2.22.394.2402221730270.754277@ubuntu-linux-20-04-desktop>
 <6eb4dc7c-c6a5-496a-a37f-125b00e9aaa3@suse.com> <alpine.DEB.2.22.394.2402231432090.754277@ubuntu-linux-20-04-desktop> <cb463469-51e1-4639-bf56-36fe9051d1b3@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-329351275-1708989087=:247676"
Content-ID: <alpine.DEB.2.22.394.2402261533040.247676@ubuntu-linux-20-04-desktop>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-329351275-1708989087=:247676
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.22.394.2402261533041.247676@ubuntu-linux-20-04-desktop>

On Mon, 26 Feb 2024, Jan Beulich wrote:
> On 23.02.2024 23:36, Stefano Stabellini wrote:
> > On Fri, 23 Feb 2024, Jan Beulich wrote:
> >> On 23.02.2024 02:32, Stefano Stabellini wrote:
> >>> On Tue, 24 Oct 2023, Jan Beulich wrote:
> >>>> On 24.10.2023 00:05, Stefano Stabellini wrote:
> >>>>> On Mon, 23 Oct 2023, Jan Beulich wrote:
> >>>>>> On 23.10.2023 17:23, Simone Ballarin wrote:
> >>>>>>> On 23/10/23 15:34, Jan Beulich wrote:
> >>>>>>>> On 18.10.2023 16:18, Simone Ballarin wrote:
> >>>>>>>>> --- a/xen/include/xen/pdx.h
> >>>>>>>>> +++ b/xen/include/xen/pdx.h
> >>>>>>>>> @@ -164,7 +164,7 @@ static inline unsigned long pfn_to_pdx(unsigned long pfn)
> >>>>>>>>>    * @param pdx Page index
> >>>>>>>>>    * @return Obtained pfn after decompressing the pdx
> >>>>>>>>>    */
> >>>>>>>>> -static inline unsigned long pdx_to_pfn(unsigned long pdx)
> >>>>>>>>> +static inline __attribute_pure__ unsigned long pdx_to_pfn(unsigned long pdx)
> >>>>>>>>>   {
> >>>>>>>>>       return (pdx & pfn_pdx_bottom_mask) |
> >>>>>>>>>              ((pdx << pfn_pdx_hole_shift) & pfn_top_mask);
> >>>>>>>>
> >>>>>>>> Taking this as an example for what I've said above: The compiler can't
> >>>>>>>> know that the globals used by the functions won't change value. Even
> >>>>>>>> within Xen it is only by convention that these variables are assigned
> >>>>>>>> their values during boot, and then aren't changed anymore. Which makes
> >>>>>>>> me wonder: Did you check carefully that around the time the variables
> >>>>>>>> have their values established, no calls to the functions exist (which
> >>>>>>>> might then be subject to folding)?
> >>>>>>>
> >>>>>>> There is no need to check that, the GCC documentation explicitly says:
> >>>>>>>
> >>>>>>> However, functions declared with the pure attribute *can safely read any 
> >>>>>>> non-volatile objects*, and modify the value of objects in a way that 
> >>>>>>> does not affect their return value or the observable state of the program.
> >>>>>>
> >>>>>> I did quote this same text in response to what Andrew has said, but I also
> >>>>>> did note there that this needs to be taken with a grain of salt: The
> >>>>>> compiler generally assumes a single-threaded environment, i.e. no changes
> >>>>>> to globals behind the back of the code it is processing.
> >>>>>
> >>>>> Let's start from the beginning. The reason for Simone to add
> >>>>> __attribute_pure__ to pdx_to_pfn and other functions is for
> >>>>> documentation purposes. It is OK if it doesn't serve any purpose other
> >>>>> than documentation.
> >>>>>
> >>>>> Andrew, for sure we do not want to lie to the compiler and introduce
> >>>>> undefined behavior. If we think there is a risk of it, we should not do
> >>>>> it.
> >>>>>
> >>>>> So, what do we want to document? We want to document that the function
> >>>>> does not have side effects according to MISRA's definition of it, which
> >>>>> might subtly differ from GCC's definition.
> >>>>>
> >>>>> Looking at GCC's definition of __attribute_pure__, with the
> >>>>> clarification statement copy/pasted above by both Simone and Jan, it
> >>>>> seems that __attribute_pure__ matches MISRA's definition of a function
> >>>>> without side effects. It also seems that pdx_to_pfn abides to that
> >>>>> definition.
> >>>>>
> >>>>> Jan has a point that GCC might be making other assumptions
> >>>>> (single-thread execution) that might not hold true in our case. Given
> >>>>> the way the GCC statement is written I think this is low risk. But maybe
> >>>>> not all GCC versions we want to support in the project might have the
> >>>>> same definition of __attribute_pure__. So we could end up using
> >>>>> __attribute_pure__ correctly for the GCC version used for safety (GCC
> >>>>> 12.1, see docs/misra/C-language-toolchain.rst) but it might actually
> >>>>> break an older GCC version.
> >>>>>
> >>>>>
> >>>>> So Option#1 is to use __attribute_pure__ taking the risk that a GCC or
> >>>>> Clang version might interpret __attribute_pure__ differently and
> >>>>> potentially misbehave.
> >>>>>
> >>>>> Option#2 is to avoid this risk, by not using __attribute_pure__.
> >>>>> Instead, we can use SAF-xx-safe or deviations.rst to document that
> >>>>> pdx_to_pfn and other functions like it are without side effects
> >>>>> according to MISRA's definition.
> >>>>>
> >>>>>
> >>>>> Both options have pros and cons. To me the most important factor is how
> >>>>> many GCC versions come with the statement "pure attribute can safely
> >>>>> read any non-volatile objects, and modify the value of objects in a way
> >>>>> that does not affect their return value or the observable state of the
> >>>>> program".
> >>>>>
> >>>>> I checked and these are the results:
> >>>>> - gcc 4.0.2: no statement
> >>>>> - gcc 5.1.0: no statement
> >>>>> - gcc 6.1.0: no statement
> >>>>> - gcc 7.1.0: no statement
> >>>>> - gcc 8.1.0: alternative statement "The pure attribute imposes similar
> >>>>>   but looser restrictions on a function’s definition than the const
> >>>>>   attribute: it allows the function to read global variables."
> >>>>> - gcc 9.1.0: yes statement
> >>>>>
> >>>>>
> >>>>> So based on the above, __attribute_pure__ comes with its current
> >>>>> definition only from gcc 9 onward. I don't know if as a Xen community we
> >>>>> clearly declare a range of supported compilers, but I would imagine we
> >>>>> would still want to support gcc versions older than 9? (Not to mention
> >>>>> clang, which I haven't checked.)
> >>>>>
> >>>>> It doesn't seem to me that __attribute_pure__ could be correctly used on
> >>>>> pdx_to_pfn with GCC 7.1.0 for example.
> >>>>
> >>>> The absence of documentation doesn't mean the attribute had different
> >>>> (or even undefined) meaning in earlier versions. Instead it means one
> >>>> would need to consult other places (source code?) to figure out whether
> >>>> there was any behavioral difference (I don't think there was).
> >>>>
> >>>> That said, ...
> >>>>
> >>>>> So in conclusion, I think it is better to avoid __attribute_pure__ and
> >>>>> use SAF-xx-safe or an alternative approach instead.
> >>>>
> >>>> ... I agree here. We just don't want to take chances.
> >>>
> >>> Let me resurrect this thread.
> >>>
> >>> Could we use something like "pure" that we #define as we want?
> >>>
> >>> Depending on the compiler version or other options we could #define pure
> >>> to __attribute_pure__ or to nothing.
> >>
> >> While we can do about anything, I don't think it's a good idea to overload
> >> a well known term with something having somewhat different meaning. If a
> >> differently named custom attribute helps, that might be a possible option.
> > 
> > It doesn't have a different meaning. If it had a different meaning I'd
> > agree with you.
> 
> Then we need to sort this aspect first: If there was no difference in
> meaning, we ought to be using the real attribute, not a pseudo
> surrogate. Yet the earlier discussion, according to my understanding,
> has led to the understanding that for the given example the real
> attribute cannot be applied entirely legitimately. Hence why the
> thinking of alternatives actually started. What am I missing?

There are two different questions:
1) using __attribute_pure__ in general when appropriate
2) using __attribute_pure__ in pdx_to_pfn as this patch does


I was talking about 1): as a general approach it looks like a good idea
to use __attribute_pure__ when possible and appropriate.

Now let's talk about 2). The latest definition of __attribute_pure__ is:

"""
The pure attribute prohibits a function from modifying the state of the program that is observable by means other than inspecting the function’s return value. However, functions declared with the pure attribute can safely read any non-volatile objects, and modify the value of objects in a way that does not affect their return value or the observable state of the program.
"""

So there are two interesting issues:

a) While this documentation explicitly allows for reading global vars,
older versions of the docs are less clear. What do we do about them?

b) Jan wrote that he interprets the statements above to be only valid in
a single-threaded environment


To be honest, I am not convinced by b). Jan, is there a statement in the
GCC docs that says that all the attributes (pure being one of them) only
apply to a single-thread environment? That would be extremely limiting
for something like __attribute_pure__. I think we should take the
documentation of attribute pure at face value. To me, it clearly applies
to pdx_to_pfn. Roberto and the team at Bugseng came to the same
conclusion.

On the other end, I think a) is important. Older version of GCC don't
clarify the behavior toward global variables. From the documentation, I
would use __attribute_pure__ only with GCC 9 or later. Which is why we
need the #define.


> > The goal is for the #define to have exactly the same meaning as the gcc
> > definition from gcc 9 onward. However, other versions of gcc or other
> > compilers could have different semantics. Also we might not want to
> > allow gcc to perform the optimizations that it might want to do if the
> > attribute is passed.
> > 
> > So the definition would be clear and 100% aligned with the modern gcc
> > definition. However we would be able to control the behavior better.
> 
> If we feared older gcc didn't implement "pure" suitably, we should
> simply make __attribute_pure__ expand to nothing there. (Still use of
> the attribute then would need limiting to cases where it can validly
> be applied.)

That's fine by me
--8323329-329351275-1708989087=:247676--


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 00:02:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 00:02:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685835.1067200 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rekv0-0005dB-Ln; Tue, 27 Feb 2024 00:01:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685835.1067200; Tue, 27 Feb 2024 00:01:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rekv0-0005d4-If; Tue, 27 Feb 2024 00:01:58 +0000
Received: by outflank-mailman (input) for mailman id 685835;
 Tue, 27 Feb 2024 00:01:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=15Q7=KE=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rekuy-0005cy-Lv
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 00:01:56 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6a3690d4-d503-11ee-8a58-1f161083a0e0;
 Tue, 27 Feb 2024 01:01:54 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 14D536136C;
 Tue, 27 Feb 2024 00:01:53 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id DC7EFC433C7;
 Tue, 27 Feb 2024 00:01:50 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6a3690d4-d503-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708992112;
	bh=WHm2jyxlCXyjW+R4esnXtFh6rU+brq2dUnlQPZLjxg0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=gK5jRHNkSpsvb7pcLqCMKPMG9jVFhJg68gmgFHnCMWAPdnSoergDzLRWKA6ljO1DP
	 ut5Gb+VXxDfyXCf22DBEn8zTbv63se4YJpDDWWj59Uxu3loUITwV79+drAMwcD8b6J
	 faJ0scQTqKaNfXhfNDWGVcenIyNatSmRgviwYq7JvAf6J16+IEi/jJ2kVNkWBrK0Aa
	 WWoeQmWjRIceOu7PfeZ6QVGm3Fd8ZfYv4G8WfvPzbDkgmfwBDG2GhHrYVwLGTpaBOf
	 tFkzGHm+PsgFlNfHQK3RlKS/OpEIN0w1YOpZ+I71LXE8p5JVLjMtVx5Q1milJYKp/D
	 pUkmnku5GArVA==
Date: Mon, 26 Feb 2024 16:01:49 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, michal.orzel@amd.com, 
    xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, bertrand.marquis@arm.com, julien@xen.org, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
In-Reply-To: <fb460b92d758891c1de42f7e588d5ffb@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402261553140.247676@ubuntu-linux-20-04-desktop>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com> <4f3b3d52a3ba20b05ecfa068b916b804@bugseng.com> <alpine.DEB.2.22.394.2402231502540.754277@ubuntu-linux-20-04-desktop>
 <fb460b92d758891c1de42f7e588d5ffb@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Sat, 24 Feb 2024, Nicola Vetrini wrote:
> Hi Stefano,
> 
> On 2024-02-24 00:05, Stefano Stabellini wrote:
> > On Fri, 23 Feb 2024, Nicola Vetrini wrote:
> > > On 2024-02-19 16:14, Nicola Vetrini wrote:
> > > > The cache clearing and invalidation helpers in x86 and Arm didn't
> > > > comply with MISRA C Rule 17.7: "The value returned by a function
> > > > having non-void return type shall be used". On Arm they
> > > > were always returning 0, while some in x86 returned -EOPNOTSUPP
> > > > and in common/grant_table the return value is saved.
> > > >
> > > > As a consequence, a common helper arch_grant_cache_flush that returns
> > > > an integer is introduced, so that each architecture can choose whether
> > > to
> > > > return an error value on certain conditions, and the helpers have either
> > > > been changed to return void (on Arm) or deleted entirely (on x86).
> > > >
> > > > Signed-off-by: Julien Grall <julien@xen.org>
> > > > Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> > > > ---
> > > > The original refactor idea came from Julien Grall in [1]; I edited that
> > > > proposal
> > > > to fix build errors.
> > > >
> > > > I did introduce a cast to void for the call to flush_area_local on x86,
> > > > because
> > > > even before this patch the return value of that function wasn't checked
> > > in
> > > > all
> > > > but one use in x86/smp.c, and in this context the helper (perhaps
> > > > incidentally)
> > > > ignored the return value of flush_area_local.
> > > >
> > > > [1]
> > > >
> > > https://lore.kernel.org/xen-devel/09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org/
> > > > ---
> > > >  xen/arch/arm/include/asm/page.h     | 33 ++++++++++++++++++-----------
> > > >  xen/arch/x86/include/asm/flushtlb.h | 23 ++++++++++----------
> > > >  xen/common/grant_table.c            |  9 +-------
> > > >  3 files changed, 34 insertions(+), 31 deletions(-)
> > > >
> > > 
> > > I'll put this patch in the backlog at the moment: too many intricacies
> > > while
> > > trying to untangle xen/flushtlb from xen/mm.h, and there are easier cases
> > > that
> > > can be done faster. If someone is interested I can post the partial work
> > > I've
> > > done so far, even though it doesn't
> > > build on x86.
> > 
> > I understand that the blocker is:
> > 
> > diff --git a/xen/arch/arm/include/asm/page.h
> > b/xen/arch/arm/include/asm/page.h
> > index 69f817d1e6..e90c9de361 100644
> > --- a/xen/arch/arm/include/asm/page.h
> > +++ b/xen/arch/arm/include/asm/page.h
> > @@ -123,6 +123,7 @@
> > 
> >  #ifndef __ASSEMBLY__
> > 
> > +#include <public/grant_table.h>
> >  #include <xen/errno.h>
> >  #include <xen/types.h>
> >  #include <xen/lib.h>
> > 
> > 
> > And the headers disentagling required to solve it, right?
> > 
> > 
> > Let me ask a silly question. public/grant_table.h seems needed by
> > arch_grant_cache_flush. Can we move arch_grant_cache_flush somewhere
> > else? It is not like page.h is a perfect fit for it anyway.
> > 
> > For instance, can we move it to
> > 
> > xen/arch/arm/include/asm/grant_table.h
> > 
> > ?
> 
> Yes, this is what was suggested and what I was trying to accomplish.
> Basically my plan is:
> 
> 1. move the arch_grant_cache_flush helper to asm/grant_table.h for both
> architectures
> 2. pull out of xen/mm.h this hunk (note the inclusion of asm/flushtlb in the
> middle of the file) because there is a build error on tlbflush_current_time()
> induced in some .c file (don't remember which) by the earlier movement

It looks like it would be easier to resolve the build error on
tlbflush_current_time() in another way. What's the build error exactly?
Looking at the implementation of tlbflush_current_time on the various
arches I couldn't find any potential issues.

I just moved the implementation of arch_grant_cache_flush to arch
specific headers and it seemed to have worked for me. See below. Maybe I
am building with a different kconfig.


diff --git a/xen/arch/arm/include/asm/grant_table.h b/xen/arch/arm/include/asm/grant_table.h
index d3c518a926..2c5c07e061 100644
--- a/xen/arch/arm/include/asm/grant_table.h
+++ b/xen/arch/arm/include/asm/grant_table.h
@@ -76,6 +76,20 @@ int replace_grant_host_mapping(uint64_t gpaddr, mfn_t frame,
 #define gnttab_need_iommu_mapping(d)                    \
     (is_domain_direct_mapped(d) && is_iommu_enabled(d))
 
+static inline int arch_grant_cache_flush(unsigned int op, const void *p,
+                                         unsigned long size)
+{
+    if ( (op & GNTTAB_CACHE_INVAL) && (op & GNTTAB_CACHE_CLEAN) )
+        clean_and_invalidate_dcache_va_range(p, size);
+    else if ( op & GNTTAB_CACHE_INVAL )
+        invalidate_dcache_va_range(p, size);
+    else if ( op & GNTTAB_CACHE_CLEAN )
+        clean_dcache_va_range(p, size);
+
+    /* ARM callers assume that dcache_* functions cannot fail. */
+    return 0;
+}
+
 #endif /* __ASM_GRANT_TABLE_H__ */
 /*
  * Local variables:
diff --git a/xen/arch/arm/include/asm/page.h b/xen/arch/arm/include/asm/page.h
index 69f817d1e6..aea692a24d 100644
--- a/xen/arch/arm/include/asm/page.h
+++ b/xen/arch/arm/include/asm/page.h
@@ -159,13 +159,13 @@ static inline size_t read_dcache_line_bytes(void)
  * if 'range' is large enough we might want to use model-specific
  * full-cache flushes. */
 
-static inline int invalidate_dcache_va_range(const void *p, unsigned long size)
+static inline void invalidate_dcache_va_range(const void *p, unsigned long size)
 {
     size_t cacheline_mask = dcache_line_bytes - 1;
     unsigned long idx = 0;
 
     if ( !size )
-        return 0;
+        return;
 
     /* Passing a region that wraps around is illegal */
     ASSERT(((uintptr_t)p + size - 1) >= (uintptr_t)p);
@@ -188,17 +188,15 @@ static inline int invalidate_dcache_va_range(const void *p, unsigned long size)
         asm volatile (__clean_and_invalidate_dcache_one(0) : : "r" (p + idx));
 
     dsb(sy);           /* So we know the flushes happen before continuing */
-
-    return 0;
 }
 
-static inline int clean_dcache_va_range(const void *p, unsigned long size)
+static inline void clean_dcache_va_range(const void *p, unsigned long size)
 {
     size_t cacheline_mask = dcache_line_bytes - 1;
     unsigned long idx = 0;
 
     if ( !size )
-        return 0;
+        return;
 
     /* Passing a region that wraps around is illegal */
     ASSERT(((uintptr_t)p + size - 1) >= (uintptr_t)p);
@@ -211,18 +209,16 @@ static inline int clean_dcache_va_range(const void *p, unsigned long size)
             idx += dcache_line_bytes, size -= dcache_line_bytes )
         asm volatile (__clean_dcache_one(0) : : "r" (p + idx));
     dsb(sy);           /* So we know the flushes happen before continuing */
-    /* ARM callers assume that dcache_* functions cannot fail. */
-    return 0;
 }
 
-static inline int clean_and_invalidate_dcache_va_range
+static inline void clean_and_invalidate_dcache_va_range
     (const void *p, unsigned long size)
 {
     size_t cacheline_mask = dcache_line_bytes - 1;
     unsigned long idx = 0;
 
     if ( !size )
-        return 0;
+        return;
 
     /* Passing a region that wraps around is illegal */
     ASSERT(((uintptr_t)p + size - 1) >= (uintptr_t)p);
@@ -235,8 +231,6 @@ static inline int clean_and_invalidate_dcache_va_range
             idx += dcache_line_bytes, size -= dcache_line_bytes )
         asm volatile (__clean_and_invalidate_dcache_one(0) : : "r" (p + idx));
     dsb(sy);         /* So we know the flushes happen before continuing */
-    /* ARM callers assume that dcache_* functions cannot fail. */
-    return 0;
 }
 
 /* Macros for flushing a single small item.  The predicate is always
diff --git a/xen/arch/x86/include/asm/flushtlb.h b/xen/arch/x86/include/asm/flushtlb.h
index bb0ad58db4..d0c9120b5f 100644
--- a/xen/arch/x86/include/asm/flushtlb.h
+++ b/xen/arch/x86/include/asm/flushtlb.h
@@ -182,21 +182,6 @@ void flush_area_mask(const cpumask_t *mask, const void *va,
 }
 
 static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache) {}
-static inline int invalidate_dcache_va_range(const void *p,
-                                             unsigned long size)
-{ return -EOPNOTSUPP; }
-static inline int clean_and_invalidate_dcache_va_range(const void *p,
-                                                       unsigned long size)
-{
-    unsigned int order = get_order_from_bytes(size);
-    /* sub-page granularity support needs to be added if necessary */
-    flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
-    return 0;
-}
-static inline int clean_dcache_va_range(const void *p, unsigned long size)
-{
-    return clean_and_invalidate_dcache_va_range(p, size);
-}
 
 unsigned int guest_flush_tlb_flags(const struct domain *d);
 void guest_flush_tlb_mask(const struct domain *d, const cpumask_t *mask);
diff --git a/xen/arch/x86/include/asm/grant_table.h b/xen/arch/x86/include/asm/grant_table.h
index 5c23cec90c..60a6dbb231 100644
--- a/xen/arch/x86/include/asm/grant_table.h
+++ b/xen/arch/x86/include/asm/grant_table.h
@@ -72,4 +72,20 @@ static inline void gnttab_clear_flags(struct domain *d,
 #define gnttab_need_iommu_mapping(d)                \
     (!paging_mode_translate(d) && need_iommu_pt_sync(d))
 
+static inline int arch_grant_cache_flush(unsigned int op, const void *p,
+                                     unsigned long size)
+{
+    unsigned int order;
+
+    if ( !(op & GNTTAB_CACHE_CLEAN) )
+        return -EOPNOTSUPP;
+
+    order = get_order_from_bytes(size);
+    /* sub-page granularity support needs to be added if necessary */
+    (void) flush_area_local(p, FLUSH_CACHE|FLUSH_ORDER(order));
+
+    return 0;
+}
+
+
 #endif /* __ASM_GRANT_TABLE_H__ */
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 37b178a67b..0df663944f 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3575,14 +3575,7 @@ static int _cache_flush(const gnttab_cache_flush_t *cflush, grant_ref_t *cur_ref
     v = map_domain_page(mfn);
     v += cflush->offset;
 
-    if ( (cflush->op & GNTTAB_CACHE_INVAL) && (cflush->op & GNTTAB_CACHE_CLEAN) )
-        ret = clean_and_invalidate_dcache_va_range(v, cflush->length);
-    else if ( cflush->op & GNTTAB_CACHE_INVAL )
-        ret = invalidate_dcache_va_range(v, cflush->length);
-    else if ( cflush->op & GNTTAB_CACHE_CLEAN )
-        ret = clean_dcache_va_range(v, cflush->length);
-    else
-        ret = 0;
+    ret = arch_grant_cache_flush(cflush->op, v, cflush->length);
 
     if ( d != owner )
     {


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 00:26:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 00:26:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685839.1067210 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1relIy-00008k-Pg; Tue, 27 Feb 2024 00:26:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685839.1067210; Tue, 27 Feb 2024 00:26:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1relIy-00008d-MZ; Tue, 27 Feb 2024 00:26:44 +0000
Received: by outflank-mailman (input) for mailman id 685839;
 Tue, 27 Feb 2024 00:26:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=15Q7=KE=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1relIx-00008X-Ig
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 00:26:43 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e02f4abc-d506-11ee-98f5-efadbce2ee36;
 Tue, 27 Feb 2024 01:26:41 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id D715C61211;
 Tue, 27 Feb 2024 00:26:39 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 66A69C433F1;
 Tue, 27 Feb 2024 00:26:38 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e02f4abc-d506-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1708993599;
	bh=s0M4+fYC1LgvU+prBk8DsjbwuoIHnSbxoJJ/ByJ02Ek=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=oLonez6frDis/k6Ll9oO2ph6fzj+SYbtcl37q/OcRXMHFYOAB7wpoJdQoQqd/KmOW
	 yBe+vIGLtmU+4NDr0cNZnHC+29Pd0qzTGydFC7589QOr8i3tmWVlgjwauqGZUI+U0q
	 B7iwJRrXJnwOmyiGxwVtnaAl9vKSy2e4QpUkNLXvIGm1Q+LoZXdHwH5ta8ZukyIKnp
	 oyvtww/BL6fhgtpz80h1duS7WWZJVsspR5yZZC53ptFefuZc+bXhLmsZicjaSZg3bA
	 O4UTzW2QxVOMPkf+hMLpDafcqjXaw0vJ18UCMvEsOY2w8PuJXxn/y+9njHsEsByHC6
	 ciXyb0k75W3CA==
Date: Mon, 26 Feb 2024 16:26:36 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Nicola Vetrini <nicola.vetrini@bugseng.com>, sstabellini@kernel.org, 
    consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
In-Reply-To: <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com> <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com> <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Mon, 26 Feb 2024, Jan Beulich wrote:
> On 23.02.2024 10:35, Nicola Vetrini wrote:
> > Refactor cpu_notifier_call_chain into two functions:
> > - the variant that is allowed to fail loses the nofail flag
> > - the variant that shouldn't fail is encapsulated in a call
> >   to the failing variant, with an additional check.
> > 
> > This prevents uses of the function that are not supposed to
> > fail from ignoring the return value, thus violating Rule 17.7:
> > "The value returned by a function having non-void return type shall
> > be used".
> > 
> > No functional change.
> > 
> > Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> 
> I'm afraid I disagree with this kind of bifurcation. No matter what
> Misra thinks or says, it is normal for return values of functions to
> not always be relevant to check.

Hi Jan, I disagree.

Regardless of MISRA, I really think return values need to be checked.
Moreover, we decided as a group to honor MISRA Rule 17.7, which requires
return values to be checked. This patch is a good step forward.


> To deal with the Misra rule imo requires to first have an abstract
> plan of how to handle such globally in the code base. Imo such a plan
> can't be to introduce perhaps dozens of new wrapper functions like is
> done here.

This patch is following the right pattern, one we already follow with
the _locked suffix.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 00:38:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 00:38:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685842.1067219 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1relUI-0001tI-QQ; Tue, 27 Feb 2024 00:38:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685842.1067219; Tue, 27 Feb 2024 00:38:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1relUI-0001tB-Nj; Tue, 27 Feb 2024 00:38:26 +0000
Received: by outflank-mailman (input) for mailman id 685842;
 Tue, 27 Feb 2024 00:38:25 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1relUH-0001t1-CB; Tue, 27 Feb 2024 00:38:25 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1relUH-0002Yl-9S; Tue, 27 Feb 2024 00:38:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1relUH-0005m9-1d; Tue, 27 Feb 2024 00:38:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1relUH-00068Y-13; Tue, 27 Feb 2024 00:38:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=UpMXMUmKsx0/rwHsyJVI6H4iVuBcEfDFgnFdTZlYKvM=; b=7HO5UafcHxLB5GWQ+9nsvMZUJM
	qJcYQYth+zxgQZD3X6aMOhchQ00lxs3c5AohkXkeK4VHknGp2IZTqGQqeat0J1mYNL64l9S/FKnu7
	+5S/kONVEXjanzYTECWwWx017QCl9fAiMIbs3zwWHB3A2bODaYxDquO0gUuhVAfMK7as=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184774-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184774: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=37541208f119a9c552c6c6c3246ea61be0d44035
X-Osstest-Versions-That:
    xen=03fb5f503cb5faa0556b23d594496719ced6a11b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 00:38:25 +0000

flight 184774 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184774/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  37541208f119a9c552c6c6c3246ea61be0d44035
baseline version:
 xen                  03fb5f503cb5faa0556b23d594496719ced6a11b

Last test of basis   184769  2024-02-26 16:00:26 Z    0 days
Testing same since   184774  2024-02-26 22:00:26 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   03fb5f503c..37541208f1  37541208f119a9c552c6c6c3246ea61be0d44035 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 00:38:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 00:38:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685845.1067231 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1relUO-00028q-3N; Tue, 27 Feb 2024 00:38:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685845.1067231; Tue, 27 Feb 2024 00:38:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1relUN-00028j-VD; Tue, 27 Feb 2024 00:38:31 +0000
Received: by outflank-mailman (input) for mailman id 685845;
 Tue, 27 Feb 2024 00:38:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=uNB2=KE=nxp.com=peng.fan@srs-se1.protection.inumbo.net>)
 id 1relUM-00027y-Cs
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 00:38:30 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2611::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 865ab8bd-d508-11ee-8a58-1f161083a0e0;
 Tue, 27 Feb 2024 01:38:29 +0100 (CET)
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com (2603:10a6:10:358::11)
 by GVXPR04MB9734.eurprd04.prod.outlook.com (2603:10a6:150:112::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.34; Tue, 27 Feb
 2024 00:38:26 +0000
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::1232:ed97:118f:72fd]) by DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::1232:ed97:118f:72fd%4]) with mapi id 15.20.7316.034; Tue, 27 Feb 2024
 00:38:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 865ab8bd-d508-11ee-8a58-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BvB1OMEAMRGEzGeYcguzntm56Zsby+CdnlNFZKo7QQUSlMmLNqfSsjqF8f3Ge0yEoOiaVsSOSVZKdn2JRhR9hPe8sRMcduGAsTSe4yJlYPW+yWQZh6JfBflJTKbM/bmjq9FWIrE1pxcMdXYB+X77edkHLgaU7k8VOWhdfQK+6GR3HPhuM7RCe+X03y3NBgc9JXPU2aIp38CSW5L0QVK9G/6xPdCWy88G3zAu/kFdkabN4In6J+Hqiu0pc31gUibiQiDTxn17gOQKblAABojU4rsIZfEsVBDVzeirLPDWNAgDz7wB9IA38Mcn49QlFLErftmqBoaT5topReijort2Ng==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=3z97EjzGtu9/xxGGajgA60u5lrmI1O9k7D6WquivLoc=;
 b=K/MWDVzwx+EQX7NIyRCUCWkMM6jl7CrrEVTyFAfivUcKdSSpRxackFJwDpZ5vIJHVN2KS3SgM8c0UBbUKCiJduAaS4DifVCTNhuEcqwhT9Qbrr8e1VSGKd2FnMUc3sva/ooxS+sXTfSHi7vkgTgrjoxNwEYLPHTUt6aHQZ7YhLrqqH3WCEr8LOqcS43PXYoohCmHO8TZEMbvJDn0UewohKT2v6zuUjh7xOp3ZmX8O2WG3MIX2Ec5WQW+ZHucSjrgxnCe66WUis+tmszrHWoKo+dKrI72rvdVwPSpZFcqSE3AhX8LWvpRl9NXh02J6IfFu5I7H2jXjuV+Dj38wLUF+g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass
 header.d=nxp.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3z97EjzGtu9/xxGGajgA60u5lrmI1O9k7D6WquivLoc=;
 b=mPp8wqxhTgnPRMrwGc/X7AianILchKsp58k+m61unkZnrFkQf/je99fBcWLXuj2R1Aa20RedTBSml1QdBITmrxSfRwxDfEKlOZtZc6Gx7BqvR4ZAzIUhT5wcABZUCgb4u8CsZVMg0WDimT8IkgPzX+CUIn4bPH/DbHl59RjY/XI=
From: Peng Fan <peng.fan@nxp.com>
To: Stefano Stabellini <stefano.stabellini@amd.com>
CC: Julien Grall <julien@xen.org>, "vikram.garhwal@amd.com"
	<vikram.garhwal@amd.com>, "oleksandr_tyshchenko@epam.com"
	<oleksandr_tyshchenko@epam.com>, Leo Yan <leo.yan@linaro.org>, Viresh Kumar
	<viresh.kumar@linaro.org>, "stewart.hildebrand@amd.com"
	<stewart.hildebrand@amd.com>, "sgarzare@redhat.com" <sgarzare@redhat.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: RE: question about virtio-vsock on xen
Thread-Topic: question about virtio-vsock on xen
Thread-Index: AdpmUSXKZvg8Aft6RRu9smUwoggtbwAUAHAAAG+7FYAAJ8zQAAAFc0Og
Date: Tue, 27 Feb 2024 00:38:26 +0000
Message-ID:
 <DU0PR04MB94178062C441EBBAF5D0BF2888592@DU0PR04MB9417.eurprd04.prod.outlook.com>
References:
 <DU0PR04MB941734DA793D87B7FF3A491488552@DU0PR04MB9417.eurprd04.prod.outlook.com>
 <alpine.DEB.2.22.394.2402231332580.754277@ubuntu-linux-20-04-desktop>
 <DU0PR04MB941746B1F385B6A514704F2B885A2@DU0PR04MB9417.eurprd04.prod.outlook.com>
 <alpine.DEB.2.22.394.2402261359280.247676@ubuntu-linux-20-04-desktop>
In-Reply-To:
 <alpine.DEB.2.22.394.2402261359280.247676@ubuntu-linux-20-04-desktop>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=nxp.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DU0PR04MB9417:EE_|GVXPR04MB9734:EE_
x-ms-office365-filtering-correlation-id: c92ece35-f0ed-42cf-58c8-08dc372c6914
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 UQW1x/Ly6oh9pO0phiA+7b8Kps6Fp2rpW7mOMy4r41a0HYiNgAJGIP0mzlJvpNIMCGLnBY5dzfxqcRjRn7+VnEedYs9F45mm5EmVwGCC3Kszbt4Poa4rMzvP+C+psH/LioHEhyXxIubsrzEVVZdI66a2z2I3zHmuUHtiRfidxQZFL7AjqfrzUzw1ny6uom+6roNATLKAqnj4O32przE8mvKObbB0jGZ/CCABjuxY5u3whvXUe07QgBeWfaSlJEkQbxjTyGLbGvTQQxYkNvz58EY7y0GYnVTpLeQPcKV9aktp6AvCHZpruzL8pWPqFAIOBYvxkhek7YiUaJGrMjaDIZ8WyRrdWEiC5n4ObAPH4po4i5HTARHc533YGq9Cc3ICmpNmByW3VU0SajVn3dmg+QJjlchVdkcU5OMDsrj4QlD+eSMeHeCL32WCFnA41/L1eEUkdbFD/c/SfZezCOYh/hTd2p1QgIRevDe8TBs4jp68y0lumOLzJg18o4tKtg2oXY/7FE2oTSKtBWiADklOOJnPedPOHLTyvF0WoCjujCerFyQ13e3EfPYkF4l6wzdtJY/7p9+gDZv0X+Ji2rQuA6+eSh6aWOOgiozjp4lWvqC8cRVGULRX9wg+qlR3s8n8eqdlROrJ/VdkFE+myIdnOf6ZNo1RjpFnJeGaVSSRGnbOXB6l9RyZ0M8dluW411kWNiX9m+1K60qCOxaOBbe/lA==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU0PR04MB9417.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?us-ascii?Q?w5+wLzuzHhhWiVx33Z0QuTlbuKxX5oP1WREYooLeqt6zM+UBw61361jOCESO?=
 =?us-ascii?Q?r4NsTik0/2gxwN6nV2KTKpuY+nWywtbRu2ZORZEGYJ8cvhzOigOZqAqZyJJH?=
 =?us-ascii?Q?jqCXu38uN9GNPlYSCmSZMroXOjmdefsLyu9VG1o5lLYueyVB7kDQuFyRRW1I?=
 =?us-ascii?Q?pduFZ+YKMnKt5XmQBNJ6sGYJzFfcltCwTVeRXAINVStXS39JINzEjoVnbzN5?=
 =?us-ascii?Q?Hz8F51uVhgouS1sBa0xOy9LRHryD2qY51Vd8UYpDfyrlqfB4npoBlwJD5sDi?=
 =?us-ascii?Q?3nE0hoSkGOd6qCcpYmEwnD9BUKPbuUZb7BxRKyijT9Cr/Tg6CRcsEhdZSVWI?=
 =?us-ascii?Q?56KpcmPxHh1n8Qdoja+F3MEln9mNEJqgUm6ynefY4wbcemDq2rs7UdURT5BP?=
 =?us-ascii?Q?RaRV7lcTh361dP6Z4n0Y/7Ra0bHhFWWDDuZYufLT7GzPHKyaU6vd5fpevTDp?=
 =?us-ascii?Q?vrdKfJ3Hay0gejPtCbsklyPPIRwMhY6S8lPn7Dvg001HZixeIJbbm3hes4Nm?=
 =?us-ascii?Q?KfbpRlaz99BBXFX3Ny2kdAHHtg0Un3ISKTiT8M2Inwq11bo3iVL0EOnuw9u9?=
 =?us-ascii?Q?/1jlgf7SGVEMcnL0K+JmZld7nTS+xK586qB9vCq3EFRReBpwMRnW1nMsWWlv?=
 =?us-ascii?Q?jUmVHpA0a04t2TeTs4LFhnPxWBRWmT/UQsnhQqU6NAHUDs5LHzspP9OAlkRy?=
 =?us-ascii?Q?h1GU+iTMUF6wBHnwqnSje8dHuv6qdJZfCLvY3IC6Knt2KX80nTDcDjq9NxXW?=
 =?us-ascii?Q?SUv1OK57MxsuAXwRrxy5JZejw33eFB3t88Yh4uv8GVbK78GeyxQsyodMq+CX?=
 =?us-ascii?Q?+/ACn3hqnmIi1zHgireMvD1OD6M2VB7hT34csMetjaIuc/AS+65a2bbD6tLZ?=
 =?us-ascii?Q?dMWmeQNyrrnixfbE7lslNoLJz//F72HGk0oFxxKgovhtxp+yAbtAHMvUxput?=
 =?us-ascii?Q?pGlt7rGD7WWgaiZk55VLM+0kVV4SQF25WWSNdLW0fkU5wRrIrAmbyK4mUWs7?=
 =?us-ascii?Q?BESKV2oDKDQJiXgYYZ+o7r2iPD0EdrLmSlOFfXOIKRycr1xoguYNA+C1+zmo?=
 =?us-ascii?Q?zhOFJsamctEKnftNzS05DBUxsg/lCP11KPtMtUp/V+/WWm2jwJte9XQIYjme?=
 =?us-ascii?Q?T6qw2W97I49zC+4CDUDL5Q3nLrKJbJaYGC0bxLPC//Ibolrl4VobWQH53tiG?=
 =?us-ascii?Q?ZSbd+7GNPf/a9PGoqMRqjdUE/B8Mr6+VAth3TxEPoQp13dmUuXB/ml+JCNdV?=
 =?us-ascii?Q?fRGBr3ZEmw6DFFwIn5YBeDMO+TXGsGOCevULJhY40WVneZ46a5NaMEzzWThq?=
 =?us-ascii?Q?h/6jDtVAT4fPgMUfb3fR84QkwDRb5wAsEgDszBAhmwguatTroWWRfR3QatDI?=
 =?us-ascii?Q?mKmu4ZhDByy0jjsyMZIh8q7kz3gzpnW6dDZvpxn/OCUOhYKPkB80cAYwV2gA?=
 =?us-ascii?Q?/VWt6SgkNP5yIPVUNqCDmq+uvl9yfTN0MeIdeOsAlEnJRBay3EIObnxFBm+p?=
 =?us-ascii?Q?Ck2dA8aX2gA5K/Dq+eZqVMpHlnj7yDtjn4dctHwfqheKLdbroCU6cxt2U6eQ?=
 =?us-ascii?Q?8nbt2nAQgg+b+h4hI8o=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DU0PR04MB9417.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c92ece35-f0ed-42cf-58c8-08dc372c6914
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Feb 2024 00:38:26.0405
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: MyBlp6xPqZsinYfZqGM5hJYN5wQBrB8MpCmrmoCIvCHq3JJxayJb4nb/JA/zq7vOqKhZTo+0Gg0MlzALG4DRdw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR04MB9734

> Subject: RE: question about virtio-vsock on xen
>=20
> On Mon, 26 Feb 2024, Peng Fan wrote:
> > Hi Stefano,
> >
> > > Subject: Re: question about virtio-vsock on xen
> > >
> > > Hi Peng,
> > >
> > > We haven't tried to setup virtio-vsock yet.
> > >
> > > In general, I am very supportive of using QEMU for virtio backends.
> > > We use QEMU to provide virtio-net, virtio-block, virtio-console and m=
ore.
> >
> > Would you mind share how to setup virtio-console using qemu + xen?
>=20
> Vikram (CCed) has been working on it and should be able to share more
> details
>=20
>=20
> > > However, typically virtio-vsock comes into play for VM-to-VM
> > > communication, which is different. Going via QEMU in Dom0 just to
> > > have 1 VM communicate with another VM is not an ideal design: it
> > > adds latency and uses resources in Dom0 when actually we could do
> without it.
> > >
> > > A better model for VM-to-VM communication would be to have the VM
> > > talk to each other directly via grant table or pre-shared memory
> > > (see the static shared memory feature) or via Xen hypercalls (see
> > > Argo.)
> >
> > The goal is to make android trout VM run with XEN + i.MX95, so need vso=
ck.
>=20
> I am not familiar with the details of Android Trout... Where is vsock use=
d?
> Just asking for my own understanding.

Here is the link
https://source.android.com/docs/automotive/virtualization/reference_platfor=
m

vsock is used for audio control hal, dumpstate HAL, vehicle HAL.

Regards,
Peng.

>=20
>=20
> > > For a good Xen design, I think the virtio-vsock backend would need
> > > to be in Xen itself (the hypervisor).
> > >
> > > Of course that is more work and it doesn't help you with the
> > > specific question you had below :-)
> > >
> > > For that, I don't have a pointer to help you but maybe others in CC h=
ave.
> > >
> > > Cheers,
> > >
> > > Stefano
> > >
> > >
> > > On Fri, 23 Feb 2024, Peng Fan wrote:
> > > > Hi All,
> > > >
> > > > Has anyone make virtio-vsock on xen work? My dm args as below:
> > > >
> > > > virtio =3D [
> > > >
> > > 'backend=3D0,type=3Dvirtio,device,transport=3Dpci,bdf=3D05:00.0,backe=
nd_type
> > > =3Dqem
> > > u,grant_usage=3Dtrue'
> > > > ]
> > > > device_model_args =3D [
> > > > '-D', '/home/root/qemu_log.txt',
> > > > '-d',
> > > > 'trace:*vsock*,trace:*vhost*,trace:*virtio*,trace:*pci_update*,tra
> > > > ce:* pci_route*,trace:*handle_ioreq*,trace:*xen*',
> > > > '-device',
> > > > 'vhost-vsock-pci,iommu_platform=3Dfalse,id=3Dvhost-vsock-pci0,bus=
=3Dpcie
> > > > .0,a
> > > > ddr=3D5.0,guest-cid=3D3']
> > > >
> > > > During my test, it always return failure in dom0 kernel in below co=
de:
> > > >
> > > > vhost_transport_do_send_pkt {
> > > > ...
> > > >                nbytes =3D copy_to_iter(hdr, sizeof(*hdr), &iov_iter=
);
> > > >                 if (nbytes !=3D sizeof(*hdr)) {
> > > >                         vq_err(vq, "Faulted on copying pkt hdr %x
> > > > %x %x %px\n",
> > > nbytes, sizeof(*hdr),
> > > > __builtin_object_size(hdr, 0), &iov_iter);
> > > >                         kfree_skb(skb);
> > > >                         break;
> > > >                 }
> > > > }
> > > >
> > > > I checked copy_to_iter, it is copy data to __user addr, but it
> > > > never pass, the copy to __user addr always return 0 bytes copied.
> > > >
> > > > The asm code "sttr x7, [x6]" will trigger data abort, the kernel
> > > > will run into do_page_fault, but lock_mm_and_find_vma report it is
> > > > VM_FAULT_BADMAP, that means the __user addr is not mapped, no
> vma
> > > has this addr.
> > > >
> > > > I am not sure what may cause this. Appreciate if any comments.
> > > >
> > > > BTW: I tested blk pci, it works, so the virtio pci should work on m=
y setup.
> > > >
> > > > Thanks,
> > > > Peng.
> > > >
> >


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 00:40:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 00:40:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685848.1067241 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1relWD-00045x-FX; Tue, 27 Feb 2024 00:40:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685848.1067241; Tue, 27 Feb 2024 00:40:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1relWD-00045q-9y; Tue, 27 Feb 2024 00:40:25 +0000
Received: by outflank-mailman (input) for mailman id 685848;
 Tue, 27 Feb 2024 00:40:24 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=uNB2=KE=nxp.com=peng.fan@srs-se1.protection.inumbo.net>)
 id 1relWC-00045i-DN
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 00:40:24 +0000
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2611::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id caa8da27-d508-11ee-8a58-1f161083a0e0;
 Tue, 27 Feb 2024 01:40:23 +0100 (CET)
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com (2603:10a6:10:358::11)
 by GVXPR04MB9734.eurprd04.prod.outlook.com (2603:10a6:150:112::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.34; Tue, 27 Feb
 2024 00:40:20 +0000
Received: from DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::1232:ed97:118f:72fd]) by DU0PR04MB9417.eurprd04.prod.outlook.com
 ([fe80::1232:ed97:118f:72fd%4]) with mapi id 15.20.7316.034; Tue, 27 Feb 2024
 00:40:20 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: caa8da27-d508-11ee-8a58-1f161083a0e0
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=k3L2n/B8cLHJ0tKhxHg2UXa912ftH/ZnXQN6WES+P6zqrzDdT8PSXoDFMKjDoPPExxxmDnpblVp6dH45z938NmBbM81bc1Dgi+f4pJuxon0yRj2oWZLY5FrVeY28/ML7eHfKra/2MKP04cFtjsBV86PFZHuYaZN5QYhJWnsU2I4T265JTylHdv7CBv+ehcedvTV3e5yuomAFf6nWhEbzHiQQNA7AjrQpYK8AGZpOLTkcXUJkljOZEJ4xGVaoJ+rmZmZn8gYWUASt2++y886eMj9G/iLQPVVY3oRJAiXQX2iplGna8KQ2nGV8hJrUPwp4lflOZswtsoXmtqJ8FzvpkQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=pd5dneSFqnwjQ7kRPh0WYjn0rt78AoXtKsG++ZtjS8k=;
 b=SaTLsxLvfCPzFKpUZH1qGZDEl76ADsYq5RMXjPG15kAR5H/5jTqIYlDVJwFZAhO5679SjuVlyM2uXSMPMIT6/JR3z64G0DsZIASScbGAjNIOs7xU6KTSIxQFz71EGViYEfWNgYKaCEStZcpP1qPHbafsGJ9Obbr4ouQHmW/28WEyTk6SY8x/JCRclbGgFhqaKjdmFeSiOuX0Ems+kLXnRDqwV8oz5SQ9fdVO8pllqNHJmMwBN4R/OUnwF/tpchhtueiGJwZNE8zprQ5tFD4JxbxzzLsEPwaZ6obQ00SFyWkSOCko2lNX5CvGI5RAOgj/8oDLZ0LSSXEGPMOQ8uDvbA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass
 header.d=nxp.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=pd5dneSFqnwjQ7kRPh0WYjn0rt78AoXtKsG++ZtjS8k=;
 b=cvTpzS9aftJrdUfcDGCSEwfiZVy1J5/5F4dLsx0JWdyy4IoxEvzQHrOf212Y/2w47w15Z2BJGwidkHbxBau/OXl8ypUpp3b2aEdayOQM3vyXcm2F1rCR4/8U1wZblujhCRbGtKAxh39Exd7ucP7HXHy+4fsl090LoLcrkQzy7ek=
From: Peng Fan <peng.fan@nxp.com>
To: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>
CC: Julien Grall <julien@xen.org>, "vikram.garhwal@amd.com"
	<vikram.garhwal@amd.com>, Leo Yan <leo.yan@linaro.org>, Viresh Kumar
	<viresh.kumar@linaro.org>, Stewart Hildebrand <stewart.hildebrand@amd.com>,
	"sgarzare@redhat.com" <sgarzare@redhat.com>, "xen-devel@lists.xenproject.org"
	<xen-devel@lists.xenproject.org>, Stefano Stabellini
	<stefano.stabellini@amd.com>
Subject: RE: question about virtio-vsock on xen
Thread-Topic: question about virtio-vsock on xen
Thread-Index: AdpmUSXKZvg8Aft6RRu9smUwoggtbwAUAHAAACuPlYAARFRfYAAj+0EAAAkpkYA=
Date: Tue, 27 Feb 2024 00:40:20 +0000
Message-ID:
 <DU0PR04MB941733E93504EA7EA3A20B6788592@DU0PR04MB9417.eurprd04.prod.outlook.com>
References:
 <DU0PR04MB941734DA793D87B7FF3A491488552@DU0PR04MB9417.eurprd04.prod.outlook.com>
 <alpine.DEB.2.22.394.2402231332580.754277@ubuntu-linux-20-04-desktop>
 <ee69684a-5aad-417a-8522-1cd1b5322bbe@epam.com>
 <DU0PR04MB94175684F8B7678A7B885559885A2@DU0PR04MB9417.eurprd04.prod.outlook.com>
 <e391b7fc-bf6f-4c3a-b444-5b320466294c@epam.com>
In-Reply-To: <e391b7fc-bf6f-4c3a-b444-5b320466294c@epam.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=nxp.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: DU0PR04MB9417:EE_|GVXPR04MB9734:EE_
x-ms-office365-filtering-correlation-id: 3c03585d-7c05-43cc-5f5a-08dc372cad05
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 rKn4p94G5Q5Wl9Xm0gCLgK2ajT5eF8iyF6wlEIq21VirItOK6avs+JafkShPPkPrEton/uW5zjbLE8/O0zKQvNDHzZj/5QZ4JJCmdcYC7QIXNsBEq36Sn8UG31fcY274VwX0yY5p/sg73mEydq8ySAmF4yFJD9FggVQ5eEgrsa3ia1TEiv2idOpjLfa+ic9qhK7jcjff8UQUh+wPI6qg1HaIfvEvHoz68qYjt4iWOdFOYihmcT2Q67qSgs08TRMmk9tmsjXo4/1eQuihvCrExVpS+aUX8VOBPEJdKaSsMOiqFmzHHTWInDyTokFJXrgUoIj1++WVaWpwFFzwwjzZnoiwEH8kcDc6UlrnvgJimhq/iuBB+VzJ3tvMedxT8r4fq4cdzDnDfEwP0cktzBc5VFV9C5xAhjMfdZbV9jSXWRnPXhI72czFwG3EqRJ7Ec1pFd+7onBo8bvSdII4ApFy4dgozcd7749D1xe0w50NtCEv82eUuIcnCkeYOatJLtxDpkU2fZBrVywRsF7x2skYG/SLH4oY470KFRjNZONh4XRVxjqW9+CSEaKKEMHUWQRFCANw3IAI0NHADvia2pUed802sJ0AkGwqysldHwVRao0f0up4TRVgA9AOxbMOCj9cHQmIkqcqdDNViCZM9xv6ZvpW7yr+6OCoYyGCtAJPv6iJF7YKs6iR0/3+VBAN9xUcBVQOYOBMimaMe6kaPKaobg==
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU0PR04MB9417.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?utf-8?B?L0dTdG1yL2d0b0IzeUFza1V2NENXTmhublNzVDRlcUFkcU4vWUc3OUt5Sk9P?=
 =?utf-8?B?S2NJQ2R4K1ZLQlpGSjRlRE1MNDI0bmU5R3NLRW5SYmVzanVWUmRuWVVFZ1J5?=
 =?utf-8?B?bFNLazMwU3Y3aWdGK1lJVjhIZXJjSUt1ZHVaeW9JQTMrajdxQTFYVFFzcDRP?=
 =?utf-8?B?R2VWNHJ1QXh4ZVZvZkpPdWxKWnloYWgxT3N3dTQxZFBiRjU3VVRNK0RDeHhm?=
 =?utf-8?B?WVluNnI2aExtN00yZW5MY3hiRG16TittZ2ZoSFlic1pjajNtcVAzNU9ib1pS?=
 =?utf-8?B?TGFZUUZTYW1Ca1Z4Snl3aWJZK3Y2QURiUXNLNVhzMEVUMitHM0VTSDE5bVow?=
 =?utf-8?B?aWtVUVNPaVo0Ymh4S3pKYzlNaEhrYkFjQXI3N2xraDV6RzAxQVlWL1lvQUhI?=
 =?utf-8?B?WGFhY3FIR2cycjlUM2tTUnI5bzd0anE0c1JNcEs5UFU1OEhsRFpPQ1NhOTYv?=
 =?utf-8?B?eWpNTVFHOFUrU1Q5cUdsa0pBTTVTMy9wQ282WHFqSzRzeXBiajJTOEgyNG5o?=
 =?utf-8?B?eGtmOVovbitPSXE1NWZyZzgyK2FTeUVDb1BSZCsyRTJvK0laYWNGeGova1l2?=
 =?utf-8?B?UGo1UGc2cTVLL1NYM0tKK0lDcGgwN1kwRWh2akMrUVVpTDV2NHBKZ3E4OXd5?=
 =?utf-8?B?aEZOaXFPd0Q3dTQyU25BOXl4WmxtcnNON24rM3lhTzNRTXVWMWgzdmV6QnRC?=
 =?utf-8?B?MXBYcUIwOWo3ekhaZ1B2dURsR3ZHUEk4RjMxUTh0dnBCSXBqUHladXMwMStH?=
 =?utf-8?B?UXBFbzVVVlM2NStjejFHT2NraWx4azR1U2U1OXNZa1UrN1ZjYXJjQms2S1E0?=
 =?utf-8?B?OUhuTTBaUTBJWGx0Y0FqS2xqOXZkNTB0ZDE1R0pHeDVpbGprNzh6TU0rbzNh?=
 =?utf-8?B?ZkJ6VUZzRzdjaWV0V29zSEFEYzZYQzhIQVRoNTZFVnQxUHlhUVdNNVZ2bks3?=
 =?utf-8?B?cE9lSXdTL0Y1Zmd2UUxVay9TSDFGeHBaVDBlNnVhdGFjVk9iVVFKUHlxS0hH?=
 =?utf-8?B?dmdqMlVjbjA0ajFWQTVXcXpHMmQ2TFJzYmZHL1JGNjFiWXNHbTVXMHNrVFly?=
 =?utf-8?B?TUlkbStOWW4vRkZwdGk4SUFjVVZvOHRwN3hZNFQvVitWenhGbFNsTWNNNjAr?=
 =?utf-8?B?aGNsT2E5VitOSVlwNU55WDhLOEdMUmFVUXdPdlU2NHBSSld2dkI4d3MxNzFV?=
 =?utf-8?B?NGJCV0ZoZXF0TE13NXhxdUN0TUY2cFJWR1pDM3hZZHZ3ejB0SlgvSzVFNHMw?=
 =?utf-8?B?RUkxRFlSMWlRbis3Nk1udW5ERWdHSVhaejh6ejg0NXdFT1dKRVRsWEFham9S?=
 =?utf-8?B?SEExVWNjZVZWV0VDcFNibVMzL0M1aW1sN0l2Z2FaSjdtU3AzZHV4WFFwRjBB?=
 =?utf-8?B?U3psZkpEOWRQTWhYMEE2cUY2cWdTYWs3M2kxb1AxNTRCR2FXaXM0Sm5xQmw3?=
 =?utf-8?B?MFV4R091eUx4WWMvUFY2ZXhBSTBmbFlLejBRSWp4NHdFcitNR09kbng4Zll0?=
 =?utf-8?B?TWxIYzN3b3FINEhrYWlhZG94UnZySllXV2YzeDVJeTgzdk5MUEVWNHAyY09I?=
 =?utf-8?B?Y1J4Ri9HRTNlZmRuSUR3T1EvL1R3V2ZndzZPeTFmdEtaNUk2WGRPbWR0UmJy?=
 =?utf-8?B?Z0pCN2lQaHpwR1crbGUyT2Q5WDA0NU9UVEtQRlIvMmRBMWJXZlpwT3FJV2tk?=
 =?utf-8?B?dFZ5NmhoQ3owcjd6UHpDZ2gxSzkyUDdpVlF0QURBOHN6dGlNYVNwQWlSVUdT?=
 =?utf-8?B?OE10TnZuTjJCWjRJQlRJQUVEK0V1aUwyUTZrYXg2WXM3cFA4VFZ0WkhKaWU4?=
 =?utf-8?B?QnphZEE3NUN1WUQzSjNHWXRGU1dYcUM5RDgzTE4zTFl2dzBOS2FCS2pQU282?=
 =?utf-8?B?Qk82bHlqbkx2QUNTUEVOS2hJNElOZ3hDa3lQeDBnWEdyYzBjd0VUdmpUUWEz?=
 =?utf-8?B?U2lUb290NlNsWU5UQVZjcFc4ZEdLcTlyY0x6YS84dXdLYjV1bVJvcnFhVk9F?=
 =?utf-8?B?TTVRbkUzZXlXUjlvMFc2Zm1MMHJTYVl2UWFtYVYwTTc3V3FYWmR2cHlRWDJL?=
 =?utf-8?B?VE9DUHVtL3AxTHZXNkdYZmg2b1lzNHBrc3ZYbkQvajN0RTBZRzNSeHJKK0xw?=
 =?utf-8?Q?c+TM=3D?=
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DU0PR04MB9417.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3c03585d-7c05-43cc-5f5a-08dc372cad05
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Feb 2024 00:40:20.0394
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 0b1afxBe6YQsTJ0Cyfpv4ft9vDGt5j+6NWH3JwjohYfSLepwSIKT8h0M0VC6H7oSHTQPN6RErr12iRnYDJxx1w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR04MB9734

PiBTdWJqZWN0OiBSZTogcXVlc3Rpb24gYWJvdXQgdmlydGlvLXZzb2NrIG9uIHhlbg0KPiANCj4g
DQo+IA0KPiBPbiAyNi4wMi4yNCAwNTowOSwgUGVuZyBGYW4gd3JvdGU6DQo+ID4gSGkgT2xla3Nh
bmRyLA0KPiANCj4gSGVsbG8gUGVuZw0KPiANCj4gDQo+IFtzbmlwXQ0KPiANCj4gPj4NCj4gPj4g
ICAgIC4uLiBQZW5nLCB3ZSBoYXZlIHZob3N0LXZzb2NrIChhbmQgdmhvc3QtbmV0KSBYZW4gUG9D
LiBBbHRob3VnaA0KPiA+PiBpdCBpcyBub24tIHVwc3RyZWFtYWJsZSBpbiBpdHMgY3VycmVudCBz
aGFwZSAoYmFzZWQgb24gb2xkIExpbnV4DQo+ID4+IHZlcnNpb24sIHJlcXVpcmVzIHNvbWUgcmV3
b3JrIGFuZCBwcm9wZXIgaW50ZWdyYXRpb24sIG1vc3QgbGlrZWx5DQo+ID4+IHJlcXVpcmVzIGlu
dm9sdmluZyBRZW11IGFuZCBwcm90b2NvbCBjaGFuZ2VzIHRvIHBhc3MgYW4gYWRkaXRpb25hbA0K
PiA+PiBpbmZvIHRvIHZob3N0KSwgaXQgd29ya3Mgd2l0aCBMaW51eA0KPiA+PiB2NS4xMCArIHBh
dGNoZWQgUWVtdSB2Ny4wLCBzbyB5b3UgY2FuIHJlZmVyIHRvIHRoZSBZb2N0byBtZXRhIGxheWVy
DQo+ID4+IHdoaWNoIGNvbnRhaW5zIGtlcm5lbCBwYXRjaGVzIGZvciB0aGUgZGV0YWlscyBbMV0u
DQo+ID4NCj4gPiBUaGFua3MgZm9yIHRoZSBwb2ludGVyLCBJIGFtIHJlYWRpbmcgdGhlIGNvZGUu
DQo+ID4NCj4gPj4NCj4gPj4gSW4gYSBudXRzaGVsbCwgYmVmb3JlIGFjY2Vzc2luZyB0aGUgZ3Vl
c3QgZGF0YSB0aGUgaG9zdCBtb2R1bGUgbmVlZHMNCj4gPj4gdG8gbWFwIGRlc2NyaXB0b3JzIGlu
IHZpcnRpbyByaW5ncyB3aGljaCBjb250YWluIGVpdGhlciBndWVzdCBncmFudA0KPiA+PiBiYXNl
ZCBETUEgYWRkcmVzc2VzIChieSB1c2luZyBYZW4gZ3JhbnQgbWFwcGluZ3MpIG9yIGd1ZXN0DQo+
ID4+IHBzZXVkby1waHlzaWNhbCBhZGRyZXNzZXMgKGJ5IHVzaW5nIFhlbiBmb3JlaWduIG1hcHBp
bmdzKS4gQWZ0ZXINCj4gPj4gYWNjZXNzaW5nIHRoZSBndWVzdCBkYXRhIHRoZSBob3N0IG1vZHVs
ZSBuZWVkcyB0byB1bm1hcCB0aGVtLg0KPiA+DQo+ID4gT2ssIEkgdGhvdWdodCAgdGhlIGN1cnJl
bnQgeGVuIHZpcnRpbyBjb2RlIGFscmVhZHkgbWFwIGV2ZXJ5IHJlYWR5Lg0KPiA+DQo+IA0KPiBJ
dCBkb2VzLCBhcyB5b3Ugc2FpZCB0aGUgdmlydGlvLWJsay1wY2kgd29ya2VkIGluIHlvdXIgZW52
aXJvbm1lbnQuIEJ1dA0KPiB2aG9zdCgtdnNvY2spIGlzIGEgc3BlY2lhbCBjYXNlLCB1bmxpa2Ug
Zm9yIHZpcnRpby1ibGstcGNpIHdoZXJlIHRoZSB3aG9sZQ0KPiBiYWNrZW5kIHJlc2lkZXMgaW4g
UWVtdSwgaGVyZSB3ZSBoYXZlIGEgc3BsaXQgbW9kZWwuIEFzIEkgdW5kZXJzdGFuZCB0aGUNCj4g
UWVtdSBwZXJmb3JtcyBvbmx5IGluaXRpYWwgc2V0dXAvY29uZmlndXJhdGlvbiB0aGVuIG9mZmxv
YWRzIHRoZSBJL08NCj4gcHJvY2Vzc2luZyB0byBhIHNlcGFyYXRlIGVudGl0eSB3aGljaCBpcyB0
aGUgTGludXggbW9kdWxlIGluIHRoYXQgcGFydGljdWxhcg0KPiBjYXNlLg0KDQpUaGFua3MgZm9y
IHNoYXJpbmcgbWUgdGhlIGluZm9ybWF0aW9uLiBJIG5lZWQgdG8gbGVhcm4gbW9yZSBzdHVmZjop
DQoNClRoYW5rcywNClBlbmcuDQo=


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 00:57:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 00:57:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685854.1067249 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1relmV-00068u-S6; Tue, 27 Feb 2024 00:57:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685854.1067249; Tue, 27 Feb 2024 00:57:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1relmV-00068n-PF; Tue, 27 Feb 2024 00:57:15 +0000
Received: by outflank-mailman (input) for mailman id 685854;
 Tue, 27 Feb 2024 00:57:13 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1relmT-00068d-Sw; Tue, 27 Feb 2024 00:57:13 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1relmT-0002xF-Pb; Tue, 27 Feb 2024 00:57:13 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1relmT-0006TK-G8; Tue, 27 Feb 2024 00:57:13 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1relmT-0004n5-FW; Tue, 27 Feb 2024 00:57:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=INLM7nRsD98S3WVP0RaMg1uIk6Fy8XF6ylq09s+Lkh4=; b=yUXP6dqcztydmFREEtp9eebERI
	wxS1JTsytOak+KG8kXfkGFFeqECoTUMark9OxPbxNEeI1758iTzOoR7xpCWhJsfeEl5Z32V3JpFMi
	wNBdu8V8ztDhuCeimkm5eSXr8VjOoK1ho3+/sq893wHuMcNZmTfH7vJgJuPylNrgvvCs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184775-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184775: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=d25421d0d8cd2493b30215ef80d2424ecb19c870
X-Osstest-Versions-That:
    ovmf=68238d4f948069fc2c6b9cc13863bdced52a84d0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 00:57:13 +0000

flight 184775 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184775/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 d25421d0d8cd2493b30215ef80d2424ecb19c870
baseline version:
 ovmf                 68238d4f948069fc2c6b9cc13863bdced52a84d0

Last test of basis   184772  2024-02-26 19:43:08 Z    0 days
Testing same since   184775  2024-02-26 22:43:02 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Gerd Hoffmann <kraxel@redhat.com>
  Michael Kubacki <michael.kubacki@microsoft.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   68238d4f94..d25421d0d8  d25421d0d8cd2493b30215ef80d2424ecb19c870 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 03:08:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 03:08:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685861.1067259 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1renpW-000365-6I; Tue, 27 Feb 2024 03:08:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685861.1067259; Tue, 27 Feb 2024 03:08:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1renpW-00035y-3h; Tue, 27 Feb 2024 03:08:30 +0000
Received: by outflank-mailman (input) for mailman id 685861;
 Tue, 27 Feb 2024 03:08:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1renpU-00035o-3q; Tue, 27 Feb 2024 03:08:28 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1renpT-0004Gk-Uf; Tue, 27 Feb 2024 03:08:27 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1renpT-0005c7-JN; Tue, 27 Feb 2024 03:08:27 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1renpT-00042J-I7; Tue, 27 Feb 2024 03:08:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=QpG7mZvzVb6MktB8dobM61ZDNq7j5nY+WITyy9ExhBg=; b=awysW0QR8UY8kkGiPn0kpebI9C
	8q/B6FLvK6lbRxpaHxBRA7mqNOG1hBKmaAmtitd8DwhPe8Vf0HKM+zutB7EdAnWgPkrgM1ebht13b
	eBLp04vHGpAD3SvUL/rbG6pH8OWo2myiMwQDUzc+opYT/hpdm2JMTFRSwSqFqnWSsZWk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184771-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184771: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=45ec2f5f6ed3ec3a79ba1329ad585497cdcbe663
X-Osstest-Versions-That:
    linux=d206a76d7d2726f3b096037f2079ce0bd3ba329b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 03:08:27 +0000

flight 184771 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184771/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184762
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184762
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184762
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184762
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184762
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184762
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184762
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184762
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                45ec2f5f6ed3ec3a79ba1329ad585497cdcbe663
baseline version:
 linux                d206a76d7d2726f3b096037f2079ce0bd3ba329b

Last test of basis   184762  2024-02-26 01:13:54 Z    1 days
Testing same since   184771  2024-02-26 19:42:20 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Chris Clayton <chris2553@googlemail.com>
  David Sterba <dsterba@suse.com>
  Elad Nachman <enachman@marvell.com>
  Filipe Manana <fdmanana@suse.com>
  Han Xu <han.xu@nxp.com>
  Johannes Thumshirn <johannes.thumshirn@wdc.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Mark O'Donovan <shiftee@posteo.net>
  Miquel Raynal <miquel.raynal@bootlin.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   d206a76d7d272..45ec2f5f6ed3e  45ec2f5f6ed3ec3a79ba1329ad585497cdcbe663 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 03:52:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 03:52:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685866.1067269 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reoVd-0008W5-6c; Tue, 27 Feb 2024 03:52:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685866.1067269; Tue, 27 Feb 2024 03:52:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reoVd-0008Vy-3r; Tue, 27 Feb 2024 03:52:01 +0000
Received: by outflank-mailman (input) for mailman id 685866;
 Tue, 27 Feb 2024 03:51:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reoVb-0008Vo-8T; Tue, 27 Feb 2024 03:51:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reoVb-00056L-3Q; Tue, 27 Feb 2024 03:51:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reoVa-0007RX-QR; Tue, 27 Feb 2024 03:51:58 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1reoVa-0006Re-Py; Tue, 27 Feb 2024 03:51:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=vNonQWisEAKDprxHaSscN8OUaWN8WbOS7Noz/yA5CpU=; b=CsdoCZO3M6xZNisnbiJuP/LSVW
	nMYBFvubeRbCBnMxdLFLAOmqDQ1TY+Ad2Dy/UKUYrV8igMCwvAkPCxFDYKU2Ce/W+Ehn141Dvkq6m
	pQe1EFgCbEtCveXyOda0QeTit4rmT0xlFeKvi7GYv697bc/GpI1Qb8BrsDC/QvRvglOo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184777-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184777: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=33c81c25bbd55141ad395af89160b4ed4b703cdf
X-Osstest-Versions-That:
    ovmf=d25421d0d8cd2493b30215ef80d2424ecb19c870
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 03:51:58 +0000

flight 184777 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184777/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 33c81c25bbd55141ad395af89160b4ed4b703cdf
baseline version:
 ovmf                 d25421d0d8cd2493b30215ef80d2424ecb19c870

Last test of basis   184775  2024-02-26 22:43:02 Z    0 days
Testing same since   184777  2024-02-27 01:11:23 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Michael Kubacki <michael.kubacki@microsoft.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   d25421d0d8..33c81c25bb  33c81c25bbd55141ad395af89160b4ed4b703cdf -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 07:09:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 07:09:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685871.1067279 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rerZx-0003uf-S6; Tue, 27 Feb 2024 07:08:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685871.1067279; Tue, 27 Feb 2024 07:08:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rerZx-0003uY-PN; Tue, 27 Feb 2024 07:08:41 +0000
Received: by outflank-mailman (input) for mailman id 685871;
 Tue, 27 Feb 2024 07:08:40 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rerZv-0003uS-VF
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 07:08:40 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 069bbf77-d53f-11ee-98f5-efadbce2ee36;
 Tue, 27 Feb 2024 08:08:36 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-512e4f4e463so4659680e87.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 23:08:36 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p2-20020a50c942000000b005657eefa8e9sm470760edh.4.2024.02.26.23.08.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 23:08:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 069bbf77-d53f-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709017716; x=1709622516; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6DiqyyY8kH1DhXKFE72nnr45e75sN0m/Xzifq4aTt88=;
        b=CegvHst+cRPeHD6DTPGU5SIbhG8UB5VNzeFJ3sEVZbs+2KrG40/LukpChZi3qesHAF
         r2iKZLdL0+Kn8oHaPAbfkdV1l/T7zNiVJ80m5TJ33A9cmvj57pYVn8RPn1J1FbTGhAOd
         77fisg+tKkWhcgFWRIsJcP+5sXrnUu+0KFrwYsZ8lW+QPURuy39AdVi2c0WhFDe7zK1o
         MXxMZzD2PjbhfJPiQVnFJnl16/yVtm1zak4f6E/zkFY0wgv9yp+QbCb0/SFxElgcfVuQ
         q9QRgw3+6ESZ3YoLKPYngdM6fJLAUFoH1pSYxhBEUIO5xiNia7NRNfC2uxnUXcvW0Wot
         9r8A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709017716; x=1709622516;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6DiqyyY8kH1DhXKFE72nnr45e75sN0m/Xzifq4aTt88=;
        b=mV4lMtCvHnz3CynicY5KYnxZSIpDSVa6JByMse2FK6XDglGVinBWanB1E8dSzwf2M5
         qpY7XW/8fYPhQcvt30YonzVyWeqLmJAv+XUaFPTDlwTOlPbHrrtgjbZDenX+Eyg2r/ed
         /8LzexcTMKscxs2SMQoQ3gSi1Bhx2oozFb1C9yGyCtpfikJVywhjzepiHduBrAH4f66b
         pO/hOfkDDxGOfkBkZfsVCPcFWKoCARE97CCnFjeGTqHrsnaujHunRdTDZL7U4L9fLJDs
         65NuAXkW73w0mn2b1XzvrrzXKp2edajPrKw90bUKmEywP74YqgW6uQqe0qkkbVJZY+yZ
         MufQ==
X-Forwarded-Encrypted: i=1; AJvYcCWZVfEbK32kzlJ3LflATTMEYjCqLUa81b8sGM1fDaZYLimPs+LH6T91neoqbNjPq5CEaK+GQX9vFWBIlCXftvBmUYswkAdmN3rlcnqvgIw=
X-Gm-Message-State: AOJu0YzbU+WR5cSob4rerlkwoQMv8X0eS97Q1er8cE/5mab1awGMltmU
	nf98nHHRitiFpyixkGl9Gm+abSr41rJebVju2PrWOakB8qNbWNfqWVx9kEp+Og==
X-Google-Smtp-Source: AGHT+IFCWGZHenn+WrfpOmvRrRI2L5wJT0U/zuBZZmpyqx4Bj9DjOkbZ6vAlOJqZDnAh5Q/xPNolvA==
X-Received: by 2002:ac2:4d04:0:b0:513:1330:3700 with SMTP id r4-20020ac24d04000000b0051313303700mr94659lfi.20.1709017716096;
        Mon, 26 Feb 2024 23:08:36 -0800 (PST)
Message-ID: <9fce0592-8b10-4b82-bcc6-38d6b32010a1@suse.com>
Date: Tue, 27 Feb 2024 08:08:35 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 1/2] xen/console: drop return value from
 consoled_guest_rx/tx
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 xen-devel@lists.xenproject.org, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <4998ec735bd7e5a50a229507e2b92ae56ec1ba4b.1708680104.git.nicola.vetrini@bugseng.com>
 <alpine.DEB.2.22.394.2402231456350.754277@ubuntu-linux-20-04-desktop>
 <004877c8-9aea-44b9-aa93-65665b0d3a8d@suse.com>
 <b417d7a330addd295b3cbbbac4bed2a4@bugseng.com>
 <a8bab8eb-a735-4440-a075-7c62a12d4e4d@suse.com>
 <alpine.DEB.2.22.394.2402261448170.247676@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402261448170.247676@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 23:49, Stefano Stabellini wrote:
> On Mon, 26 Feb 2024, Jan Beulich wrote:
>> On 26.02.2024 09:23, Nicola Vetrini wrote:
>>> On 2024-02-26 09:00, Jan Beulich wrote:
>>>> On 23.02.2024 23:56, Stefano Stabellini wrote:
>>>>> On Fri, 23 Feb 2024, Nicola Vetrini wrote:
>>>>>> These functions never saw a usage of their return value since
>>>>>> they were introduced, so it can be dropped since their usages
>>>>>> violate MISRA C Rule 17.7:
>>>>>> "The value returned by a function having non-void return type shall 
>>>>>> be used".
>>>>>>
>>>>>> No functional change.
>>>>>>
>>>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>>>>
>>>>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>>>>
>>>> The cleanup is certainly okay, but would one of you mind clarifying in 
>>>> how
>>>> far this code is relevant for certification? I don't expect there are 
>>>> plans
>>>> to run shim Xen in any projected production uses for which 
>>>> certification is
>>>> relevant? (The subject prefix is also unnecessarily wide here, when 
>>>> it's
>>>> only daemon code which is affected, not console code in general.)
>>>>
>>>
>>> I agree on the subject prefix being too wide. The configuration that 
>>> uses consoled_guest_tx is #ifdef-ed for x86, so even in configurations 
>>> that may never reach this condition this is relevant, unless its #ifdef 
>>> is restricted to cases where the call may actually be reachable.
>>
>> Hmm, I see. There are contradicting goals here then: It being just X86 is
>> to reduce the risk of someone overlooking a build breakage they may
>> introduce. Whereas for certification it's quite the other way around: We'd
>> like to "hide" as much code as possible.
>>
>> Really I would have been inclined to suggest to drop the #ifdef, if
>> possible even without replacing by IS_ENABLED(), but instead leveraging
>> that pv_shim ought to be compile-time false whenever CONFIG_PV_SHIM=n.
> 
> This is OK
> 
> 
>> After all that's a pattern we've been trying to follow. But with your
>> observation is becomes questionable whether extending use of IS_ENABLED()
>> is actually going to be helpful. Stefano - perhaps something to discuss
>> on one of the next meetings?
> 
> Yes. I checked with the safety manager and his opinion is that
> IS_ENABLED() is OK to use as a way to disable code from a safety
> perspective.

Yet unlike when #ifdef is used, such code would remain visible to e.g.
Eclair even after the preprocessing step. Note the context in which
I'm bringing this up - if IS_ENABLED() was properly used here (and as
tightly as possible), the tool would still have complained, aiui.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 07:23:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 07:23:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685875.1067290 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rernt-0006U7-5f; Tue, 27 Feb 2024 07:23:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685875.1067290; Tue, 27 Feb 2024 07:23:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rernt-0006U0-2f; Tue, 27 Feb 2024 07:23:05 +0000
Received: by outflank-mailman (input) for mailman id 685875;
 Tue, 27 Feb 2024 07:23:04 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rerns-0006Tu-48
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 07:23:04 +0000
Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com
 [2a00:1450:4864:20::62c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0aac285b-d541-11ee-8a58-1f161083a0e0;
 Tue, 27 Feb 2024 08:23:02 +0100 (CET)
Received: by mail-ej1-x62c.google.com with SMTP id
 a640c23a62f3a-a3d01a9a9a2so395706166b.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 23:23:02 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h16-20020a170906261000b00a3ca56e9bcfsm469823ejc.187.2024.02.26.23.23.01
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 23:23:01 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0aac285b-d541-11ee-8a58-1f161083a0e0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709018582; x=1709623382; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3rH9419YQ4l6iuXA5GaUW0ySnSMzOnuMS2xgFJL6omw=;
        b=ST9PmWmxsp1IBWWBd3kEzesAbI/k0iPQlKE7aDfuYNHA1aJz2Angb9pdOJDZ/9YyeQ
         BrhHUuaRPaAWXPj9OmKZscSve3J0MXcOJdlbMLmKclj1d+gmj1mKbOVv5MNWOhxQ3cu/
         GUfts7EM1d6HeQgqJAsK+ikTX4puAuxMtCwm+xoEn7mxRqa+cVzCVIb5BJqu1Ow4k92e
         Ta/rVdZASuqpFF9cGlmSCt+NL27bjJ+F9CcN2sdX4L/eLro2ONehHPRZgD908y3dArrv
         gm266BUKjk20J23YJBQpCRcrVmQ/5w2ifXNgHv+kxaB/wKgyGmolNwBTbzP6RaUNTkcM
         hNlQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709018582; x=1709623382;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3rH9419YQ4l6iuXA5GaUW0ySnSMzOnuMS2xgFJL6omw=;
        b=eZWerYZ/lqx1asl5Pgy6EcPitYp+7eQ84/ZrFH+6qc4yWhMU//dbkzsInxtJuZGPb6
         VqibmgQXG/3Ztg2VchKGYFBqYg8NjpGvBwFfI398ExzUb3uruKuv8xAo75ymG7ogbY34
         Z9QwoY1vB9cmkobazq/ycG2gwzME9fuShWVdH2vJXfz4Js7KUXKYKOTJB8vzmarI55si
         nlSRtfFcbmQqCbny9y7y3P3baHoveVvGSzyxGtW5ZBEukxRlq4JW7B+4+zoCjyDGK8ES
         CrMXwhkYK064dS4GzzuO7tLuIHzmYMTs7QjhFwDXTuJe99DzINgtQlc0SoTMiNx+XgKB
         1JpA==
X-Forwarded-Encrypted: i=1; AJvYcCWY20avm+Mb+Fq+1mRNX2bBMSjRoatu8veGGQDW3qC4iOapCY6sDCgLhQRNq5nNSsDVJH++8rTcAxeR+zBDGhFJLJz3k43ZIYcL9/JfKZs=
X-Gm-Message-State: AOJu0Ywixs6HMOJnjTjJNHAa6dpmEH6tcZc1N92fwuD+LAxaYIP9ON/S
	MxfXoj4c52bExBjmbG1lz8n/akfEYQIBmxOku0xoDbhSC1syutbeM2Dfrf6Obw==
X-Google-Smtp-Source: AGHT+IHbxk67kUb80kjH6zAOo1nrkS1CiZXg7lgtWJ3NJqRqhvXTOM+3Zk/KB9RlNLtaFf+cQlJW5A==
X-Received: by 2002:a17:906:150b:b0:a3f:20b9:2b68 with SMTP id b11-20020a170906150b00b00a3f20b92b68mr5717193ejd.10.1709018581919;
        Mon, 26 Feb 2024 23:23:01 -0800 (PST)
Message-ID: <4e3e949d-5714-4c3b-8514-50a3674c94d8@suse.com>
Date: Tue, 27 Feb 2024 08:23:00 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 3/4] xen/include: add pure and const attributes
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Simone Ballarin <simone.ballarin@bugseng.com>, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1697638210.git.simone.ballarin@bugseng.com>
 <fd5421162a00aa782e0776324ff6497193c1e3d3.1697638210.git.simone.ballarin@bugseng.com>
 <89778285-5cba-8fb5-70bc-710b6dd30a10@suse.com>
 <0032186f-80c7-4dba-b46e-10d4a8e2a8cb@bugseng.com>
 <e8bf9817-fd54-9bf4-4302-dcee682f9172@suse.com>
 <alpine.DEB.2.22.394.2310231417260.3516@ubuntu-linux-20-04-desktop>
 <8f8141e2-c55c-2e58-a078-d5d8764a02df@suse.com>
 <alpine.DEB.2.22.394.2402221730270.754277@ubuntu-linux-20-04-desktop>
 <6eb4dc7c-c6a5-496a-a37f-125b00e9aaa3@suse.com>
 <alpine.DEB.2.22.394.2402231432090.754277@ubuntu-linux-20-04-desktop>
 <cb463469-51e1-4639-bf56-36fe9051d1b3@suse.com>
 <alpine.DEB.2.22.394.2402261511010.247676@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402261511010.247676@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27.02.2024 00:48, Stefano Stabellini wrote:
> On Mon, 26 Feb 2024, Jan Beulich wrote:
>> On 23.02.2024 23:36, Stefano Stabellini wrote:
>>> On Fri, 23 Feb 2024, Jan Beulich wrote:
>>>> On 23.02.2024 02:32, Stefano Stabellini wrote:
>>>>> On Tue, 24 Oct 2023, Jan Beulich wrote:
>>>>>> On 24.10.2023 00:05, Stefano Stabellini wrote:
>>>>>>> On Mon, 23 Oct 2023, Jan Beulich wrote:
>>>>>>>> On 23.10.2023 17:23, Simone Ballarin wrote:
>>>>>>>>> On 23/10/23 15:34, Jan Beulich wrote:
>>>>>>>>>> On 18.10.2023 16:18, Simone Ballarin wrote:
>>>>>>>>>>> --- a/xen/include/xen/pdx.h
>>>>>>>>>>> +++ b/xen/include/xen/pdx.h
>>>>>>>>>>> @@ -164,7 +164,7 @@ static inline unsigned long pfn_to_pdx(unsigned long pfn)
>>>>>>>>>>>    * @param pdx Page index
>>>>>>>>>>>    * @return Obtained pfn after decompressing the pdx
>>>>>>>>>>>    */
>>>>>>>>>>> -static inline unsigned long pdx_to_pfn(unsigned long pdx)
>>>>>>>>>>> +static inline __attribute_pure__ unsigned long pdx_to_pfn(unsigned long pdx)
>>>>>>>>>>>   {
>>>>>>>>>>>       return (pdx & pfn_pdx_bottom_mask) |
>>>>>>>>>>>              ((pdx << pfn_pdx_hole_shift) & pfn_top_mask);
>>>>>>>>>>
>>>>>>>>>> Taking this as an example for what I've said above: The compiler can't
>>>>>>>>>> know that the globals used by the functions won't change value. Even
>>>>>>>>>> within Xen it is only by convention that these variables are assigned
>>>>>>>>>> their values during boot, and then aren't changed anymore. Which makes
>>>>>>>>>> me wonder: Did you check carefully that around the time the variables
>>>>>>>>>> have their values established, no calls to the functions exist (which
>>>>>>>>>> might then be subject to folding)?
>>>>>>>>>
>>>>>>>>> There is no need to check that, the GCC documentation explicitly says:
>>>>>>>>>
>>>>>>>>> However, functions declared with the pure attribute *can safely read any 
>>>>>>>>> non-volatile objects*, and modify the value of objects in a way that 
>>>>>>>>> does not affect their return value or the observable state of the program.
>>>>>>>>
>>>>>>>> I did quote this same text in response to what Andrew has said, but I also
>>>>>>>> did note there that this needs to be taken with a grain of salt: The
>>>>>>>> compiler generally assumes a single-threaded environment, i.e. no changes
>>>>>>>> to globals behind the back of the code it is processing.
>>>>>>>
>>>>>>> Let's start from the beginning. The reason for Simone to add
>>>>>>> __attribute_pure__ to pdx_to_pfn and other functions is for
>>>>>>> documentation purposes. It is OK if it doesn't serve any purpose other
>>>>>>> than documentation.
>>>>>>>
>>>>>>> Andrew, for sure we do not want to lie to the compiler and introduce
>>>>>>> undefined behavior. If we think there is a risk of it, we should not do
>>>>>>> it.
>>>>>>>
>>>>>>> So, what do we want to document? We want to document that the function
>>>>>>> does not have side effects according to MISRA's definition of it, which
>>>>>>> might subtly differ from GCC's definition.
>>>>>>>
>>>>>>> Looking at GCC's definition of __attribute_pure__, with the
>>>>>>> clarification statement copy/pasted above by both Simone and Jan, it
>>>>>>> seems that __attribute_pure__ matches MISRA's definition of a function
>>>>>>> without side effects. It also seems that pdx_to_pfn abides to that
>>>>>>> definition.
>>>>>>>
>>>>>>> Jan has a point that GCC might be making other assumptions
>>>>>>> (single-thread execution) that might not hold true in our case. Given
>>>>>>> the way the GCC statement is written I think this is low risk. But maybe
>>>>>>> not all GCC versions we want to support in the project might have the
>>>>>>> same definition of __attribute_pure__. So we could end up using
>>>>>>> __attribute_pure__ correctly for the GCC version used for safety (GCC
>>>>>>> 12.1, see docs/misra/C-language-toolchain.rst) but it might actually
>>>>>>> break an older GCC version.
>>>>>>>
>>>>>>>
>>>>>>> So Option#1 is to use __attribute_pure__ taking the risk that a GCC or
>>>>>>> Clang version might interpret __attribute_pure__ differently and
>>>>>>> potentially misbehave.
>>>>>>>
>>>>>>> Option#2 is to avoid this risk, by not using __attribute_pure__.
>>>>>>> Instead, we can use SAF-xx-safe or deviations.rst to document that
>>>>>>> pdx_to_pfn and other functions like it are without side effects
>>>>>>> according to MISRA's definition.
>>>>>>>
>>>>>>>
>>>>>>> Both options have pros and cons. To me the most important factor is how
>>>>>>> many GCC versions come with the statement "pure attribute can safely
>>>>>>> read any non-volatile objects, and modify the value of objects in a way
>>>>>>> that does not affect their return value or the observable state of the
>>>>>>> program".
>>>>>>>
>>>>>>> I checked and these are the results:
>>>>>>> - gcc 4.0.2: no statement
>>>>>>> - gcc 5.1.0: no statement
>>>>>>> - gcc 6.1.0: no statement
>>>>>>> - gcc 7.1.0: no statement
>>>>>>> - gcc 8.1.0: alternative statement "The pure attribute imposes similar
>>>>>>>   but looser restrictions on a function’s definition than the const
>>>>>>>   attribute: it allows the function to read global variables."
>>>>>>> - gcc 9.1.0: yes statement
>>>>>>>
>>>>>>>
>>>>>>> So based on the above, __attribute_pure__ comes with its current
>>>>>>> definition only from gcc 9 onward. I don't know if as a Xen community we
>>>>>>> clearly declare a range of supported compilers, but I would imagine we
>>>>>>> would still want to support gcc versions older than 9? (Not to mention
>>>>>>> clang, which I haven't checked.)
>>>>>>>
>>>>>>> It doesn't seem to me that __attribute_pure__ could be correctly used on
>>>>>>> pdx_to_pfn with GCC 7.1.0 for example.
>>>>>>
>>>>>> The absence of documentation doesn't mean the attribute had different
>>>>>> (or even undefined) meaning in earlier versions. Instead it means one
>>>>>> would need to consult other places (source code?) to figure out whether
>>>>>> there was any behavioral difference (I don't think there was).
>>>>>>
>>>>>> That said, ...
>>>>>>
>>>>>>> So in conclusion, I think it is better to avoid __attribute_pure__ and
>>>>>>> use SAF-xx-safe or an alternative approach instead.
>>>>>>
>>>>>> ... I agree here. We just don't want to take chances.
>>>>>
>>>>> Let me resurrect this thread.
>>>>>
>>>>> Could we use something like "pure" that we #define as we want?
>>>>>
>>>>> Depending on the compiler version or other options we could #define pure
>>>>> to __attribute_pure__ or to nothing.
>>>>
>>>> While we can do about anything, I don't think it's a good idea to overload
>>>> a well known term with something having somewhat different meaning. If a
>>>> differently named custom attribute helps, that might be a possible option.
>>>
>>> It doesn't have a different meaning. If it had a different meaning I'd
>>> agree with you.
>>
>> Then we need to sort this aspect first: If there was no difference in
>> meaning, we ought to be using the real attribute, not a pseudo
>> surrogate. Yet the earlier discussion, according to my understanding,
>> has led to the understanding that for the given example the real
>> attribute cannot be applied entirely legitimately. Hence why the
>> thinking of alternatives actually started. What am I missing?
> 
> There are two different questions:
> 1) using __attribute_pure__ in general when appropriate
> 2) using __attribute_pure__ in pdx_to_pfn as this patch does
> 
> 
> I was talking about 1): as a general approach it looks like a good idea
> to use __attribute_pure__ when possible and appropriate.
> 
> Now let's talk about 2). The latest definition of __attribute_pure__ is:
> 
> """
> The pure attribute prohibits a function from modifying the state of the program that is observable by means other than inspecting the function’s return value. However, functions declared with the pure attribute can safely read any non-volatile objects, and modify the value of objects in a way that does not affect their return value or the observable state of the program.
> """
> 
> So there are two interesting issues:
> 
> a) While this documentation explicitly allows for reading global vars,
> older versions of the docs are less clear. What do we do about them?
> 
> b) Jan wrote that he interprets the statements above to be only valid in
> a single-threaded environment
> 
> 
> To be honest, I am not convinced by b). Jan, is there a statement in the
> GCC docs that says that all the attributes (pure being one of them) only
> apply to a single-thread environment?

It would need to be the other way around, I'm afraid: C99 defines its
abstract machine in a way not even considering possible parallel
execution (except for other external agents, e.g. when "volatile" is
necessary for memory accesses when that memory may also be modified
by such an external agent, e.g. a device on the bus). Hence we'd need
to find an explicit statement in gcc docs which relaxes that globally
or for certain aspects.

> That would be extremely limiting
> for something like __attribute_pure__. I think we should take the
> documentation of attribute pure at face value. To me, it clearly applies
> to pdx_to_pfn. Roberto and the team at Bugseng came to the same
> conclusion.
> 
> On the other end, I think a) is important. Older version of GCC don't
> clarify the behavior toward global variables. From the documentation, I
> would use __attribute_pure__ only with GCC 9 or later. Which is why we
> need the #define.

Right, this is a position we can take. As said, I think we'd then limit
ourselves more than necessary. Otoh the number of people using gcc8 or
older to build up-to-date Xen should be constantly decreasing ...

Jan

>>> The goal is for the #define to have exactly the same meaning as the gcc
>>> definition from gcc 9 onward. However, other versions of gcc or other
>>> compilers could have different semantics. Also we might not want to
>>> allow gcc to perform the optimizations that it might want to do if the
>>> attribute is passed.
>>>
>>> So the definition would be clear and 100% aligned with the modern gcc
>>> definition. However we would be able to control the behavior better.
>>
>> If we feared older gcc didn't implement "pure" suitably, we should
>> simply make __attribute_pure__ expand to nothing there. (Still use of
>> the attribute then would need limiting to cases where it can validly
>> be applied.)
> 
> That's fine by me



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 07:28:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 07:28:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685880.1067300 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rertR-00076R-Od; Tue, 27 Feb 2024 07:28:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685880.1067300; Tue, 27 Feb 2024 07:28:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rertR-00076K-Ly; Tue, 27 Feb 2024 07:28:49 +0000
Received: by outflank-mailman (input) for mailman id 685880;
 Tue, 27 Feb 2024 07:28:48 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rertQ-000766-Sy
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 07:28:48 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d7aad9da-d541-11ee-98f5-efadbce2ee36;
 Tue, 27 Feb 2024 08:28:46 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a3e550ef31cso437771166b.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 23:28:47 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 hg13-20020a170906f34d00b00a3d8fb05c0csm479038ejb.86.2024.02.26.23.28.46
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 23:28:46 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d7aad9da-d541-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709018926; x=1709623726; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4yGTMfEzs3kic1uFPt1x3+Vtw5Q24sXPBBHyvwGV154=;
        b=cWiUlF3FtkIBLOvpcd9r3ZIQrJ0mt8zBbPl7X0jGnDpYbU0P6rkKZht8SAvM2bFo6a
         GjNnO9VRX435uPXtadj474Y75AMjVsXn1FRWgctHTb/xZzC6h08SLnRLLZMPJDf01JoC
         c2Ma4VuJllAfdYOhvIq/i+zhd2FYq6IejUXwnXpBpY16a4i/U4zOeqSdvOfPqnsZuSw5
         +BRzQlqf/ErJhEpC4L3JIj+mUk6xN8f++PKfC3Zf/u/AAiCguJaGb6G7NnXcWwQ7IkwU
         1w/VjF2clj1Mp910ypd1x5zbbB606A7ri6/20vpIJ0iaUOBIwiengG1vvDnDtjf670Q5
         u0jg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709018926; x=1709623726;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=4yGTMfEzs3kic1uFPt1x3+Vtw5Q24sXPBBHyvwGV154=;
        b=bwuhd7vGplQgCPLkUqrKHgUtEUWoocsMWiBHT1ziHgzfO8WvLN0g0DSPoeyjdLLVPl
         JR/x3bw8VNYVYsh9//nKAaNhjCynHb/70NOosXmH7erAZBoWxCvw6d5LRDJat2CxVSK7
         mfpUnirTyE2JwhCEtK2pKW0UE976lkMDY+QPw7c1QD4X3GI9c0VbKelRiQesjVlcxN5U
         xJCyB+Pu5Qlg2phRwqzk+R0yVRh03ZPgHLIkzNlUs0DEGCSNPjM8+cJDtaXEEmKpQUT5
         Zqfo04xRvxE90BN/6Zrpf/NdlqzkSAiSrHigBJbM646bBgR/+IYeBK5mIiheu+CxXUqh
         4wVg==
X-Forwarded-Encrypted: i=1; AJvYcCUwA5RR1uAjclCyCvLosgGKlcCu2iqoTBGuoJ2CZsX3hmITu02g2ERV/f+3iG0sVtYTWXt6Ga9Gcjq67dA+ASrzjH5PejkInExrqz65V2w=
X-Gm-Message-State: AOJu0YzBPEBzZz0UTFWIZirzyG0t7UpOI1YmshZcEFmHO2uS/IxTKsPp
	BsRtBB3WSBT2/NhsUecESjiE/vJSeCFEFB1kHiXt7/c3iY95jWaZ01X6R5aOKg==
X-Google-Smtp-Source: AGHT+IHvQ5mS/YyTT8P9i1C68SFs6y08ylywe3LRbqMvH8oMNTQZndrJ137GDorhL4agtpD2ptZtJw==
X-Received: by 2002:a17:906:150b:b0:a3f:20b9:2b68 with SMTP id b11-20020a170906150b00b00a3f20b92b68mr5726365ejd.10.1709018926631;
        Mon, 26 Feb 2024 23:28:46 -0800 (PST)
Message-ID: <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com>
Date: Tue, 27 Feb 2024 08:28:45 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Nicola Vetrini <nicola.vetrini@bugseng.com>, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
 <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
 <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.02.2024 01:26, Stefano Stabellini wrote:
> On Mon, 26 Feb 2024, Jan Beulich wrote:
>> On 23.02.2024 10:35, Nicola Vetrini wrote:
>>> Refactor cpu_notifier_call_chain into two functions:
>>> - the variant that is allowed to fail loses the nofail flag
>>> - the variant that shouldn't fail is encapsulated in a call
>>>   to the failing variant, with an additional check.
>>>
>>> This prevents uses of the function that are not supposed to
>>> fail from ignoring the return value, thus violating Rule 17.7:
>>> "The value returned by a function having non-void return type shall
>>> be used".
>>>
>>> No functional change.
>>>
>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>
>> I'm afraid I disagree with this kind of bifurcation. No matter what
>> Misra thinks or says, it is normal for return values of functions to
>> not always be relevant to check.
> 
> Hi Jan, I disagree.
> 
> Regardless of MISRA, I really think return values need to be checked.
> Moreover, we decided as a group to honor MISRA Rule 17.7, which requires
> return values to be checked. This patch is a good step forward.

Yet splitting functions isn't the only way to deal with Misra's
requirements, I suppose. After all there are functions where the
return value is purely courtesy for perhaps just one of its callers.

Splitting simply doesn't scale very well, imo.

>> To deal with the Misra rule imo requires to first have an abstract
>> plan of how to handle such globally in the code base. Imo such a plan
>> can't be to introduce perhaps dozens of new wrapper functions like is
>> done here.
> 
> This patch is following the right pattern, one we already follow with
> the _locked suffix.

Right, and - just to mention it - one which I similarly dislike, albeit
to a lesser degree.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 07:35:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 07:35:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685882.1067309 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rerzg-0000SD-DK; Tue, 27 Feb 2024 07:35:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685882.1067309; Tue, 27 Feb 2024 07:35:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rerzg-0000S6-AV; Tue, 27 Feb 2024 07:35:16 +0000
Received: by outflank-mailman (input) for mailman id 685882;
 Tue, 27 Feb 2024 07:35:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rerzf-0000S0-33
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 07:35:15 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bd644fb8-d542-11ee-98f5-efadbce2ee36;
 Tue, 27 Feb 2024 08:35:11 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a3e552eff09so425652266b.3
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 23:35:12 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 c19-20020a170906695300b00a3d8a76a92esm488022ejs.175.2024.02.26.23.35.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 23:35:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd644fb8-d542-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709019312; x=1709624112; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Myf3nBv3rrva9CpYrxE4KlHrZZDdK1HrDwj1b+jGTRs=;
        b=dEdCgdO+8P4C7e1irqlblZBOQddW2ef6hbVo8NsR6K9E8EIOkjg7k2KdQ87lBVvFCN
         cfBS9sacIv+vI8OMb6h4+q+ml2oxe+cMVAWtMBczJ6IBrrUtGY2m6mRmUfhxGyLX+9/K
         bwZg/zFO7oDHwHYOLuJJmN2867X6Lhk73DX8d6kWskZaeB3SIQ1vtSB+MIu9/Bkztlq4
         eqspbqyDxXfc5fxNfQ7zF4EsjWaf02XPum26Mq1P3ypGn1d0izdqZBUc2lrM9XkRLJkq
         sctPjonC0EZz/MQp8H3H47zWjBHpyefdAmK6+GN2/yWcxm+owW342mfPrjJP4VLEAC+g
         61Hw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709019312; x=1709624112;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Myf3nBv3rrva9CpYrxE4KlHrZZDdK1HrDwj1b+jGTRs=;
        b=MTZhZDcc8b8iNpHLQJ8GnMFCy0qEldjyJwnmtRNvtSDA8lrmMjYcV181A+2ROvRUlF
         MAcEJDLkEMvziTIHdw7XyxT79UrpusvAxukp0USkjzo8yW6imBSxzwYd6ZBh9xVo3lZu
         mgza7/YwWUrIuTZLUptGdRIg6JK0baB27QTEy48jXZR2G0j0FH63/rt51Avw18ip6gtH
         KdmYCbLHnRRDpWaCQVqb67G9A8bJyRP/gZXwpdedJvRSGo05VFmXQq8scu/4Ejl1TgPI
         6Zpi1DJR+WLtHqmHIdJ7Q9iFvThYRz4bqCXraqPU3Mc9t86SPeXOo7p+XNdXYF5tICmU
         lgKA==
X-Forwarded-Encrypted: i=1; AJvYcCUBlPq2ox5rC9UOia7i2qOfeiYlaRFYwvlY59kMmRLVGi+LiDm7IPFwojYJerm6FZqwE6tbwKIZGD3t7dv37Ff/bzxFwOuaRPtlHmxoWn4=
X-Gm-Message-State: AOJu0Yz3kvrbO1SSd0Sd8OEjJIqwQSXYA67Vx76fBimasl4pZnCMBSVg
	+OpKV4ihPNWys75ub/cixfSiL2Eps2etT57gKas5xDfrT7aAyv78IEQUGISe8g==
X-Google-Smtp-Source: AGHT+IGR5NTsRPSvhjGb36rlS/1mETpMkxqrWaw/mFx5Mq/BSWcZ0AKgX70egt3bQcoH9Or9uQT9aQ==
X-Received: by 2002:a17:906:a142:b0:a41:315b:94d9 with SMTP id bu2-20020a170906a14200b00a41315b94d9mr5063818ejb.71.1709019312229;
        Mon, 26 Feb 2024 23:35:12 -0800 (PST)
Message-ID: <160f665e-0f06-45a0-bc29-80240306884a@suse.com>
Date: Tue, 27 Feb 2024 08:35:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 02/23] xen/riscv: use some asm-generic headers
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <e0bf8689422066f5fcdf9e063717f6c748ea3af7.1708962629.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e0bf8689422066f5fcdf9e063717f6c748ea3af7.1708962629.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 18:38, Oleksii Kurochko wrote:
> The following headers end up the same as asm-generic's version:
> * altp2m.h
> * device.h
> * div64.h
> * hardirq.h
> * hypercall.h
> * iocap.h
> * paging.h
> * percpu.h
> * random.h
> * softirq.h
> * vm_event.h
> 
> RISC-V should utilize the asm-generic's version of the mentioned
> headers instead of introducing them in the arch-specific folder.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> ---
> Changes in V5:
>  - Nothing changed. Only rebase.
>  - update the commit message.
>  - drop the message above revision log as there is no depenency for this patch
>    from other patch series.

Please can you make sure you submit patches against a sufficiently up-to-
date version of the staging branch? I committed v4 of this patch a couple
of days ago already, upon your own confirmation that it was okay to go in
ahead of others.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 07:38:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 07:38:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685888.1067319 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1res2a-00014G-Uz; Tue, 27 Feb 2024 07:38:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685888.1067319; Tue, 27 Feb 2024 07:38:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1res2a-000149-SD; Tue, 27 Feb 2024 07:38:16 +0000
Received: by outflank-mailman (input) for mailman id 685888;
 Tue, 27 Feb 2024 07:38:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1res2Z-00013E-OY
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 07:38:15 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 29401359-d543-11ee-98f5-efadbce2ee36;
 Tue, 27 Feb 2024 08:38:12 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a3e72ec566aso497415966b.2
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 23:38:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 lu17-20020a170906fad100b00a3e45a18a5asm489474ejb.61.2024.02.26.23.38.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 23:38:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 29401359-d543-11ee-98f5-efadbce2ee36
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709019493; x=1709624293; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=t8rr3Yu1kppMh7d6qm1M4HurjPsU/I8DuPlwD+6HbtQ=;
        b=Lrx235eoubCcjYFPKB9e4/tJUC0lhF3JHpaUGvnmHzXHjBniRehrljiRxiUC9HOxlx
         6Yb8kMf4J8NP8w5jl6hnD0HPuIctlPBwUszYkfbKJfGPptVN2RhgSKwXNJUN7qM9QW3+
         GO3YXweOSYAUFhoTlw7yHle4MbdFUbmgo9M1xI2b0eJiUC98lbekO38UBPJupDjbYmPQ
         3bSvSrosD3admS97Q/a1e54O8jQI+sFPbbR7Th4USGuSVrI1iHwQnb63RynswEklLS/Y
         Qx50B6fBpFatslbqd/Bo2DT+rn+m4jdq8h65/hvE3TcdsU+NpiAoRvcuV7W675W4Kryz
         AmuQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709019493; x=1709624293;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=t8rr3Yu1kppMh7d6qm1M4HurjPsU/I8DuPlwD+6HbtQ=;
        b=do30untyf7a2aRnonkK43tPZEa5ktSGh2TyXhTEaemVU+dY1zOL8Tz31AzCPM7Qq7R
         cYEpfPjCp/ikip++IiKk6vtZ2VDlu1UZTr9/HvU23kZ2vpViOmT5mFGHgdD5Yxk+Yz2x
         MLHqV0ufaYql3NngxSpfkisbYtvpcVEBcRkacqOndOjaxU88xjiPC54uu7pRjwL2nq1G
         qlVOfC2NuJQjT9YB61OFL6YPALwu0f7Dfv7/R9/Krv3IpPx3DT77QWMsZGczzgVRzqdk
         lMfZxbJYHu3zomKXBvAvqIGhANcdsexaByvJ/GKG26Ae3USvAQ0OLLbTCyWUSMaG7j1s
         MIFA==
X-Forwarded-Encrypted: i=1; AJvYcCU0VTDIrgqIyMnfEVVkuDxNVZLa4qD8ZFEP2tspEl2a5StBwWrlBp1HurFmt5RrKQd8ad3I4MMV9MHOugTtWOWBruZG5QdZHXTcn7BnM9c=
X-Gm-Message-State: AOJu0YxZVUfx6qrUF3zkdwDaJX+ws1KmGqRh/WZd7VGJQr+aJb3JtvU7
	dTZJlQVPZlHoZ625FxBaNjLsBU+7JwGwAxO2dbeBRNbnPj2tQZqC3n+Iv5VD6A==
X-Google-Smtp-Source: AGHT+IFCgvsCIJDrbcrG1XQKZuxdSNCmiqPMYqmqG6zl2wy4ypIH/5+a83IAd5W1O7yRyYVQiPBksg==
X-Received: by 2002:a17:906:48c9:b0:a43:3f37:4d88 with SMTP id d9-20020a17090648c900b00a433f374d88mr3941008ejt.71.1709019493296;
        Mon, 26 Feb 2024 23:38:13 -0800 (PST)
Message-ID: <293c6236-1ad0-4a48-853f-afec419656e9@suse.com>
Date: Tue, 27 Feb 2024 08:38:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 03/23] xen/riscv: introduce nospec.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <16370492de5344ae8e1aab688261600491868f4e.1708962629.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <16370492de5344ae8e1aab688261600491868f4e.1708962629.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 18:38, Oleksii Kurochko wrote:
> From the unpriviliged doc:
>   No standard hints are presently defined.
>   We anticipate standard hints to eventually include memory-system spatial
>   and temporal locality hints, branch prediction hints, thread-scheduling
>   hints, security tags, and instrumentation flags for simulation/emulation.
> 
> Also, there are no speculation execution barriers.
> 
> Therefore, functions evaluate_nospec() and block_speculation() should
> remain empty until a specific platform has an extension to deal with
> speculation execution.

What about array_index_mask_nospec(), though? No custom implementation,
meaning the generic one will be used there? If that's the intention,
then ...

> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 07:55:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 07:55:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685892.1067330 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1resIz-0004JI-B8; Tue, 27 Feb 2024 07:55:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685892.1067330; Tue, 27 Feb 2024 07:55:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1resIz-0004JB-7f; Tue, 27 Feb 2024 07:55:13 +0000
Received: by outflank-mailman (input) for mailman id 685892;
 Tue, 27 Feb 2024 07:55:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1resIx-0004J5-HE
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 07:55:11 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 87c49ef1-d545-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 08:55:10 +0100 (CET)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-5648d92919dso5152922a12.1
 for <xen-devel@lists.xenproject.org>; Mon, 26 Feb 2024 23:55:10 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s21-20020aa7cb15000000b005645961ad39sm506626edt.47.2024.02.26.23.55.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 26 Feb 2024 23:55:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87c49ef1-d545-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709020510; x=1709625310; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6N5s7n+fNWhEu6tGZmRmFlf3iWQVPeoCf9N7x/6a50k=;
        b=AJ7b0b4TwaUHCYp6l9ssfS7U7QWrMXnfcce/kIpqqyarjD27omTu0OXT0VLRKxIIwL
         vYy9nzXbkqLJqhEIffSaMIla7IVIGjI6fMaMKPbXK9Eg6kpjvABLhjdsvlwe9iF3S+3y
         FwC5Nfaw+kitZ319E/J3AgveFwgqyk8Muvzpjy4ySO82a3xoPsZc6F20tM6L1y3N2QO2
         iwIc7/cOQggoCcpjSAkheXOItjxtS8HI+dgC8dZdiMnTaL2vLWEp20ZbQ7Q9CHzcyOAP
         OiKmPrNA0bCumihKol0jrueo6kbmtlpGxhA93LfUJhS9n5Z2hrwDVDc1WcMJMOjeq/Ue
         6pEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709020510; x=1709625310;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6N5s7n+fNWhEu6tGZmRmFlf3iWQVPeoCf9N7x/6a50k=;
        b=Cc3pVMpn6M9zF22lwFMaTwFwKfyTsELCszZ8zB89zx/VNOOE93ShWdCqYk2vOxkW3h
         hfLLcapwd4EP/+GRkthZZtK3AjyK1vCfRiV5XaF4uaBFTAFtN4C6VWsw6dWgbwvBzi4X
         Y3rMEh8aVqFzh4i/Ndxt5/ZDBvQT9YkePcqXHWTKyxnWBWqz7dyZP1i12vVh79x+oF/i
         DK/hx4ZZadTytfZ8eew4zqNkDpsJVHX1vHweKX8sG1xiZFY6HEekNi9K6fMn2SlIDnOY
         3BUz/JebDwNcBGRKmHvc69bJlf6SpwpueYWUsfGThJddP6MVNBw55CMu/bTu1kvMe0dc
         Icpg==
X-Forwarded-Encrypted: i=1; AJvYcCVZZfcTxEgaiacERwavyRxhwPwHw42H0OeJyDYQCJOEb8ryr6H8EktDp1iuNHGLS9faCDhCtylCA3mef2XDjhgPTSDD4cfvUrItz3hvEgk=
X-Gm-Message-State: AOJu0YyYAEi06MkdKoqz79O2p1uGHKZkz+DDb8GCmVuihK3Q73NczcLM
	UbLjeqqmGMLG3qNkPIxCwkxcNHLGMRyu6ccaHB867d2KdShyudew6ivkBoemUQ==
X-Google-Smtp-Source: AGHT+IFKRVeK8SvrAhq3ASQZICmD3fh3JiyTBAXrcpr3+fhE8dX6gfEC/847gbL30z/nurq3f3gnTw==
X-Received: by 2002:a05:6402:1646:b0:566:2ee3:4993 with SMTP id s6-20020a056402164600b005662ee34993mr936079edx.26.1709020509806;
        Mon, 26 Feb 2024 23:55:09 -0800 (PST)
Message-ID: <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
Date: Tue, 27 Feb 2024 08:55:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 18:39, Oleksii Kurochko wrote:
> This patch doesn't represent a strict lower bound for GCC and
> GNU Binutils; rather, these versions are specifically employed by
> the Xen RISC-V container and are anticipated to undergo continuous
> testing.

Up and until that container would be updated to a newer gcc. I'm
afraid I view this as too weak a criteria, but I'm also not meaning to
stand in the way if somebody else wants to ack this patch in this form;
my bare minimum requirement is now met.

> --- a/README
> +++ b/README
> @@ -48,6 +48,15 @@ provided by your OS distributor:
>        - For ARM 64-bit:
>          - GCC 5.1 or later
>          - GNU Binutils 2.24 or later
> +      - For RISC-V 64-bit:
> +        - GCC 12.2 or later
> +        - GNU Binutils 2.39 or later
> +        This doesn't represent a strict lower bound for GCC and GNU Binutils;
> +        rather, these versions are specifically employed by the Xen RISC-V
> +        container and are anticipated to undergo continuous testing.

As per above, I think here it really needs saying "at the time of writing"
or recording a concrete date. Furthermore I expect "these versions" relates
to the specifically named versions and particularly _not_ to "or later":
With the criteria you apply, using later versions (or in fact any version
other than the very specific ones used in the container) would be similarly
untested. Much like x86 and Arm don't have the full range of permitted
tool chain versions continuously tested. Plus don't forget that distros may
apply their own selection of patches on top of what they take from upstream
(and they may also take random snapshots rather than released versions).

IOW it is hard for me to see why RISC-V needs stronger restrictions here
than other architectures. It ought to be possible to determine a baseline
version. Even if taking the desire to have "pause" available as a
requirement, gas (and presumably gld) 2.36.1 would already suffice.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 08:15:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 08:15:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685899.1067340 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rescf-00086c-6M; Tue, 27 Feb 2024 08:15:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685899.1067340; Tue, 27 Feb 2024 08:15:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rescf-00086V-2V; Tue, 27 Feb 2024 08:15:33 +0000
Received: by outflank-mailman (input) for mailman id 685899;
 Tue, 27 Feb 2024 08:15:31 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rescd-00086P-J9
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 08:15:31 +0000
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com
 [2a00:1450:4864:20::336])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5e267ab4-d548-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 09:15:29 +0100 (CET)
Received: by mail-wm1-x336.google.com with SMTP id
 5b1f17b1804b1-412a3903586so14957975e9.3
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 00:15:29 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 i13-20020adfe48d000000b0033ae7d768b2sm10503193wrm.117.2024.02.27.00.15.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 00:15:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5e267ab4-d548-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709021729; x=1709626529; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=f1Lu3qJeQL0CxcRwHeBftGmjM7NsyeZOURl2FrMqWrk=;
        b=rUnh+wUiQvlw9uJvI8paqqb8qt1vV4F/DsEtzn07+EGh+Bmvu65ARkbviaPR+3im4n
         oYcEpDttNZO7Eb5RMEHJbTL3PNVLk62wSehgl4CRgSG2e7rcsl70YBxzv6LzXP2kkdL2
         IaEFuNyQ6b27FfwzTYrrrT4+7/hKh6X+Ey/AI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709021729; x=1709626529;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=f1Lu3qJeQL0CxcRwHeBftGmjM7NsyeZOURl2FrMqWrk=;
        b=gYg2tIQCYD2L/VUyGGZcKjxvNq6eKjn7kkVu9ekbZ6jf8mqKNVu2W4uH69ElVTiOFo
         FjZt+FnaqrKtIye8MNsAEDY3bKo5F/B8lAQpa6brr3FSyFu0bHIGcuPJYt8E233rBmIA
         Mrq5ijYJ6JJ9eMEnSKAz5t8GUYD+hWh0QhaU25IM9FE4XzdD072nSTfvUNt/fCGykjC9
         6Uxw3ryN6w0raNVLOIzlHu7R1iTwJ81zQLmfSVVPyvW/UUs/vlrBuhp/Md+8ixog07XI
         eKsh2LMOT52WTyjplIvZ/xhwWWQJCNHkrSP+VwBmY9WlJ4f+I/IFOe7YEzVFnz1q2i+I
         KrOg==
X-Forwarded-Encrypted: i=1; AJvYcCVNRK/kkffddy519BBYBIFD7cT7qvF2NEkelceRgq8QVJ4H52ffO138ms2aG8oBrtWphM2QENeWISQzTPzD7S0FvtzU3yc/1ksBc5sjUjY=
X-Gm-Message-State: AOJu0YxnLoXlRxJAei8Y+yCTjsGBjKBJ+LrCfXa0MeTK3Tc6cKKGMzuV
	YMU+HZByb6lwo1ArQOSXHYqy19iZe7mzUiVgFbOGcyJWlkxRLeuOSLfL/M+KkOA=
X-Google-Smtp-Source: AGHT+IHGbHjJ6KGAWio/eOr5jbHB7oxw6OfECvx/T5iqVMSzrmqlk/zZINYpuIbt5bavRPXUb+uKKw==
X-Received: by 2002:a05:600c:1c8d:b0:412:afa6:cf3a with SMTP id k13-20020a05600c1c8d00b00412afa6cf3amr149487wms.32.1709021728807;
        Tue, 27 Feb 2024 00:15:28 -0800 (PST)
Date: Tue, 27 Feb 2024 09:15:27 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v6 3/3] xen/livepatch: align functions to ensure minimal
 distance between entry points
Message-ID: <Zd2aH_1yNe8XzlKx@macbook>
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-4-roger.pau@citrix.com>
 <670e4a5e-4eec-4a2c-9ed1-fcc164b1d76c@suse.com>
 <Zdx26DwTvcB2zcbw@macbook>
 <116a9e07-4a6a-4170-bf0f-3a7ccbd68f0e@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <116a9e07-4a6a-4170-bf0f-3a7ccbd68f0e@suse.com>

On Mon, Feb 26, 2024 at 01:36:32PM +0100, Jan Beulich wrote:
> On 26.02.2024 12:32, Roger Pau Monné wrote:
> > On Tue, Feb 13, 2024 at 04:58:38PM +0100, Jan Beulich wrote:
> >> On 07.02.2024 15:55, Roger Pau Monne wrote:
> >>> The minimal function size requirements for an x86 livepatch are either 5 bytes
> >>> (for jmp) or 9 bytes (for endbr + jmp), and always 4 bytes on Arm.  Ensure that
> >>> distance between functions entry points is always at least of the minimal
> >>> required size for livepatch instruction replacement to be successful.
> >>>
> >>> Add an additional align directive to the linker scripts, in order to ensure that
> >>> the next section placed after the .text.* (per-function sections) is also
> >>> aligned to the required boundary, so that the distance of the last function
> >>> entry point with the next symbol is also of minimal size.
> >>
> >> Perhaps "... minimal required size"?
> > 
> > Yes.
> > 
> >>> --- a/xen/common/Kconfig
> >>> +++ b/xen/common/Kconfig
> >>> @@ -395,8 +395,11 @@ config CRYPTO
> >>>  config LIVEPATCH
> >>>  	bool "Live patching support"
> >>>  	default X86
> >>> -	depends on "$(XEN_HAS_BUILD_ID)" = "y"
> >>> +	depends on "$(XEN_HAS_BUILD_ID)" = "y" && CC_HAS_FUNCTION_ALIGNMENT
> >>>  	select CC_SPLIT_SECTIONS
> >>> +	select FUNCTION_ALIGNMENT_16B if XEN_IBT
> >>> +	select FUNCTION_ALIGNMENT_8B  if X86
> >>> +	select FUNCTION_ALIGNMENT_4B  if ARM
> >>
> >> This isn't strictly needed, is it? Would be nice to avoid re-selection
> >> of what the default for an arch is anyway, as otherwise this will start
> >> looking clumsy when a couple more architectures are added.
> > 
> > My worry was that the default per-arch could change, ie: for example
> > x86 moving from 16 to 8 and then it would hamper livepatch support if
> > IBT is also enabled.  I however think it's very unlikely to reduce the
> > default alignment, and in any case we would hit a build time assert if
> > that ever happens.
> > 
> > So yes, I'm fine with dropping those.
> 
> Oh, no - not "those", only "that", i.e. only the last (Arm) one.

Oh, I see what you mean, even x86 selects the default one when IBT is
enabled, and when not the requirement for livepatch is < than the
default anyway.  That's why I said that we could even drop all of them
and just rely on the build time assert to catch any changes here.

Feel free to drop the ARM one.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 08:53:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 08:53:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685910.1067349 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1retDO-0005DE-VQ; Tue, 27 Feb 2024 08:53:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685910.1067349; Tue, 27 Feb 2024 08:53:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1retDO-0005D7-So; Tue, 27 Feb 2024 08:53:30 +0000
Received: by outflank-mailman (input) for mailman id 685910;
 Tue, 27 Feb 2024 08:53:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1retDN-0005Cz-7n
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 08:53:29 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id abc7632e-d54d-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 09:53:26 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-5656e5754ccso4957799a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 00:53:26 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d42-20020a056402402a00b00565a2c48a30sm559452eda.31.2024.02.27.00.53.25
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 27 Feb 2024 00:53:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: abc7632e-d54d-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709024006; x=1709628806; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=xNwkY3QBeqLDJtBywx0Tsk9fY9oZ9ZcnbZMnppJ+DJs=;
        b=C9jbUtnqrJxsQG/l15EEARZNbCQq+aVOgYrGJAtZlAoIgYxWCUdjrp/+YproxqZKVm
         gTehAlFNgYL44546VSPCU397mxmtoVkdXGqlIXATniK0wcL6k1HPP6OLLFZoMZcNZZcx
         QLhFvLMlMzwIr8bpP9Sk68FlLFcVs2kyFzB5MNev8wtk3Xs9SfmFDJjJu62IS6tgHvlB
         unbALf/m42awPz/Pl2f5540JWgD87O71gMGh09n7Fy2RRCp4sBtJ3+EZ5U2N8rtdmkca
         iXhHxjZxbBWMHSHnDziuwqX/cY4u1V0BjmnVI5QXfmbQAIymi9XdM7Rp/MdQKSn8K2w0
         g5mQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709024006; x=1709628806;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=xNwkY3QBeqLDJtBywx0Tsk9fY9oZ9ZcnbZMnppJ+DJs=;
        b=VgkLZ/fmx162zHPFM8dcXfDOv0BjGIo0LRWHCbU3FVkebI5JXWgG9UyQ8QY66/3tl6
         SmnQ+/nvFMXNvtEZTL2V3FjXfQjZ2vskMN8mjA93v4aUOHC3nRUljVCREEkOC5D8BlQ7
         MWSVsv+uV6rrs4OtsiMicE39fKntEe7mEmIeDf1eZgkBFqsCdMAiZyDwSFzFNW/RhAhK
         fpMFuC8Va1+LfLocdzd3lQGIky/xXXcuGlj2pwQE0nYtX/kUdgtNoguQMcH5AXCw99Om
         FUZUn1nT+GlLWkO02Us4azlqaxcZ7wj+W7z4nJxtE7QC2Fz88jicgo5qEcqEMz2dkxI6
         RyxQ==
X-Forwarded-Encrypted: i=1; AJvYcCVtw+Yauku7Z4//6lojsDtTgXL6afBLhenQ/ijA0SUIt8zmZDFAGfVxbjlQJrAzyfzziad1snhPREfPHBTtUKhGWFhp3BoiGAUCIzQOJf0=
X-Gm-Message-State: AOJu0YxuKsGoq8nL6j+QEcpCHptGm03MipRt1BDwKIfRey0/l9VZHdyo
	+IDWFWnn+Ux4UtlkkjOKcgu8jXcsTyEkiGok+HDrj3g4iaOImZJacrTPxIniaQ==
X-Google-Smtp-Source: AGHT+IE2ZJZxrA2WYl2Xq0SVC4OcizetLxAa63RLla+2sLa10mtMAXjAQ8dMjDDN0BP8nINp7WBP3g==
X-Received: by 2002:a05:6402:340d:b0:564:901b:edee with SMTP id k13-20020a056402340d00b00564901bedeemr7017355edc.25.1709024006217;
        Tue, 27 Feb 2024 00:53:26 -0800 (PST)
Message-ID: <ce9f469e-50d1-44ab-9914-f3c65ec91f30@suse.com>
Date: Tue, 27 Feb 2024 09:53:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v6 3/3] xen/livepatch: align functions to ensure minimal
 distance between entry points
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Julien Grall <julien@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 Shawn Anastasio <sanastasio@raptorengineering.com>,
 Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, xen-devel@lists.xenproject.org
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-4-roger.pau@citrix.com>
 <670e4a5e-4eec-4a2c-9ed1-fcc164b1d76c@suse.com> <Zdx26DwTvcB2zcbw@macbook>
 <116a9e07-4a6a-4170-bf0f-3a7ccbd68f0e@suse.com> <Zd2aH_1yNe8XzlKx@macbook>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zd2aH_1yNe8XzlKx@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27.02.2024 09:15, Roger Pau Monné wrote:
> On Mon, Feb 26, 2024 at 01:36:32PM +0100, Jan Beulich wrote:
>> On 26.02.2024 12:32, Roger Pau Monné wrote:
>>> On Tue, Feb 13, 2024 at 04:58:38PM +0100, Jan Beulich wrote:
>>>> On 07.02.2024 15:55, Roger Pau Monne wrote:
>>>>> The minimal function size requirements for an x86 livepatch are either 5 bytes
>>>>> (for jmp) or 9 bytes (for endbr + jmp), and always 4 bytes on Arm.  Ensure that
>>>>> distance between functions entry points is always at least of the minimal
>>>>> required size for livepatch instruction replacement to be successful.
>>>>>
>>>>> Add an additional align directive to the linker scripts, in order to ensure that
>>>>> the next section placed after the .text.* (per-function sections) is also
>>>>> aligned to the required boundary, so that the distance of the last function
>>>>> entry point with the next symbol is also of minimal size.
>>>>
>>>> Perhaps "... minimal required size"?
>>>
>>> Yes.
>>>
>>>>> --- a/xen/common/Kconfig
>>>>> +++ b/xen/common/Kconfig
>>>>> @@ -395,8 +395,11 @@ config CRYPTO
>>>>>  config LIVEPATCH
>>>>>  	bool "Live patching support"
>>>>>  	default X86
>>>>> -	depends on "$(XEN_HAS_BUILD_ID)" = "y"
>>>>> +	depends on "$(XEN_HAS_BUILD_ID)" = "y" && CC_HAS_FUNCTION_ALIGNMENT
>>>>>  	select CC_SPLIT_SECTIONS
>>>>> +	select FUNCTION_ALIGNMENT_16B if XEN_IBT
>>>>> +	select FUNCTION_ALIGNMENT_8B  if X86
>>>>> +	select FUNCTION_ALIGNMENT_4B  if ARM
>>>>
>>>> This isn't strictly needed, is it? Would be nice to avoid re-selection
>>>> of what the default for an arch is anyway, as otherwise this will start
>>>> looking clumsy when a couple more architectures are added.
>>>
>>> My worry was that the default per-arch could change, ie: for example
>>> x86 moving from 16 to 8 and then it would hamper livepatch support if
>>> IBT is also enabled.  I however think it's very unlikely to reduce the
>>> default alignment, and in any case we would hit a build time assert if
>>> that ever happens.
>>>
>>> So yes, I'm fine with dropping those.
>>
>> Oh, no - not "those", only "that", i.e. only the last (Arm) one.
> 
> Oh, I see what you mean, even x86 selects the default one when IBT is
> enabled, and when not the requirement for livepatch is < than the
> default anyway.  That's why I said that we could even drop all of them
> and just rely on the build time assert to catch any changes here.

Just to clarify: The default I mean is the architecture imposed one.
Leaving aside Thumb mode, Arm instructions are all 32-bit words, and
hence less than 4-byte alignment makes no sense (and may even be
disallowed by the architecture). Whereas for x86 what you're talking
about is just a compiler default, which isn't really guaranteed to
never be lower (with -Os for example I'd expect it to be perhaps as
low as 1).

Jan

> Feel free to drop the ARM one.
> 
> Thanks, Roger.



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 09:26:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 09:26:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685923.1067371 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1retii-0001C4-L2; Tue, 27 Feb 2024 09:25:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685923.1067371; Tue, 27 Feb 2024 09:25:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1retii-0001Bx-I5; Tue, 27 Feb 2024 09:25:52 +0000
Received: by outflank-mailman (input) for mailman id 685923;
 Tue, 27 Feb 2024 09:25:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1retig-0001Br-KA
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 09:25:50 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 31adba96-d552-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 10:25:49 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-512b3b04995so4101320e87.3
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 01:25:49 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 bz8-20020ad44c08000000b0068fa89a5df6sm3945326qvb.124.2024.02.27.01.25.47
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 01:25:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31adba96-d552-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709025949; x=1709630749; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=pg2sukypEsunK/w9zk0F/rvlrwUsh5HyxJOjDlTANmE=;
        b=Dn+B4LRi/BScYdFydx5HD8SeeGtvACxPNags7iJqwPC7LNtn6xAMTiCGulU9iRQs2G
         1Znz4r+Pm2oftWnu3X4jB3X2c5zZxuB86+Xku2uTL9pqB4Go+R58yP+4VYnV8PcIY8dS
         3hlS1uLrRHl4ntXYTe7vvwhwg+MbIOkebqnjw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709025949; x=1709630749;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pg2sukypEsunK/w9zk0F/rvlrwUsh5HyxJOjDlTANmE=;
        b=a2p47O0p/zXZ47TP6SBRfoBnccv756tw7E+DNH7LRJ/vnVPZ91v4pB7ii7b9Oe2dNh
         3eeRHE+ZlnyYl/xMWoc6C7VCegPV6U1KnRx1/0oE3fOC4yfJHT0dmNdKlNgdgxBQeCcq
         Ak+YDNWwV7pdBrqx3m3CBiTVRfBjGtH3siH2rnn352rnBrNEfl6JbPtt8ZdpBl4hGAAn
         glbm/FUG3FTC3Voy8VmHIYKPDVDZuT38oRF7nD+B5oNd+FcbxLIKwFOLDSckoOoSbhe+
         6FAsV/wbybUaLi+uFBqL0cIYTmP/bOzDIdN3/lxxwZ+vpYcKMQ9zP8FebEqISi/kFxqX
         FTEQ==
X-Forwarded-Encrypted: i=1; AJvYcCVZhfWuhrhe1XtbyP6L5Q0mrnKZZcyA88cIsmd2ed5ldH8SXdcuk86xi38uICfhssIWUgwGwWfH2uNjwoZ13E/UBBjBNSMSYbReBQY6uJM=
X-Gm-Message-State: AOJu0Yyu0gs6afrPjkQj9VUDarlTpJBkpEucaxLkC70WtnQlmAr8o/9E
	+AqWWmvCtPzhbrnP5MgY9NGKpCDu8Nb0gAz77jSqJjA/Z3Aah+qThLqdirloWLY=
X-Google-Smtp-Source: AGHT+IFRvMSwlItMxjbfid+D3ha/nqlIya3CneQRT2JRXSk7zhZw1dyZXf7xL9A08dDifuN2Utnsvw==
X-Received: by 2002:a05:6512:3a8f:b0:512:fb30:aade with SMTP id q15-20020a0565123a8f00b00512fb30aademr5099385lfu.3.1709025948747;
        Tue, 27 Feb 2024 01:25:48 -0800 (PST)
Date: Tue, 27 Feb 2024 10:25:46 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Julien Grall <julien@xen.org>,
	Bertrand Marquis <bertrand.marquis@arm.com>,
	Michal Orzel <michal.orzel@amd.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
	Shawn Anastasio <sanastasio@raptorengineering.com>,
	Alistair Francis <alistair.francis@wdc.com>,
	Bob Eshleman <bobbyeshleman@gmail.com>,
	Connor Davis <connojdavis@gmail.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v6 3/3] xen/livepatch: align functions to ensure minimal
 distance between entry points
Message-ID: <Zd2qmrXTJb9NBdUW@macbook>
References: <20240207145547.89689-1-roger.pau@citrix.com>
 <20240207145547.89689-4-roger.pau@citrix.com>
 <670e4a5e-4eec-4a2c-9ed1-fcc164b1d76c@suse.com>
 <Zdx26DwTvcB2zcbw@macbook>
 <116a9e07-4a6a-4170-bf0f-3a7ccbd68f0e@suse.com>
 <Zd2aH_1yNe8XzlKx@macbook>
 <ce9f469e-50d1-44ab-9914-f3c65ec91f30@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ce9f469e-50d1-44ab-9914-f3c65ec91f30@suse.com>

On Tue, Feb 27, 2024 at 09:53:24AM +0100, Jan Beulich wrote:
> On 27.02.2024 09:15, Roger Pau Monné wrote:
> > On Mon, Feb 26, 2024 at 01:36:32PM +0100, Jan Beulich wrote:
> >> On 26.02.2024 12:32, Roger Pau Monné wrote:
> >>> On Tue, Feb 13, 2024 at 04:58:38PM +0100, Jan Beulich wrote:
> >>>> On 07.02.2024 15:55, Roger Pau Monne wrote:
> >>>>> The minimal function size requirements for an x86 livepatch are either 5 bytes
> >>>>> (for jmp) or 9 bytes (for endbr + jmp), and always 4 bytes on Arm.  Ensure that
> >>>>> distance between functions entry points is always at least of the minimal
> >>>>> required size for livepatch instruction replacement to be successful.
> >>>>>
> >>>>> Add an additional align directive to the linker scripts, in order to ensure that
> >>>>> the next section placed after the .text.* (per-function sections) is also
> >>>>> aligned to the required boundary, so that the distance of the last function
> >>>>> entry point with the next symbol is also of minimal size.
> >>>>
> >>>> Perhaps "... minimal required size"?
> >>>
> >>> Yes.
> >>>
> >>>>> --- a/xen/common/Kconfig
> >>>>> +++ b/xen/common/Kconfig
> >>>>> @@ -395,8 +395,11 @@ config CRYPTO
> >>>>>  config LIVEPATCH
> >>>>>  	bool "Live patching support"
> >>>>>  	default X86
> >>>>> -	depends on "$(XEN_HAS_BUILD_ID)" = "y"
> >>>>> +	depends on "$(XEN_HAS_BUILD_ID)" = "y" && CC_HAS_FUNCTION_ALIGNMENT
> >>>>>  	select CC_SPLIT_SECTIONS
> >>>>> +	select FUNCTION_ALIGNMENT_16B if XEN_IBT
> >>>>> +	select FUNCTION_ALIGNMENT_8B  if X86
> >>>>> +	select FUNCTION_ALIGNMENT_4B  if ARM
> >>>>
> >>>> This isn't strictly needed, is it? Would be nice to avoid re-selection
> >>>> of what the default for an arch is anyway, as otherwise this will start
> >>>> looking clumsy when a couple more architectures are added.
> >>>
> >>> My worry was that the default per-arch could change, ie: for example
> >>> x86 moving from 16 to 8 and then it would hamper livepatch support if
> >>> IBT is also enabled.  I however think it's very unlikely to reduce the
> >>> default alignment, and in any case we would hit a build time assert if
> >>> that ever happens.
> >>>
> >>> So yes, I'm fine with dropping those.
> >>
> >> Oh, no - not "those", only "that", i.e. only the last (Arm) one.
> > 
> > Oh, I see what you mean, even x86 selects the default one when IBT is
> > enabled, and when not the requirement for livepatch is < than the
> > default anyway.  That's why I said that we could even drop all of them
> > and just rely on the build time assert to catch any changes here.
> 
> Just to clarify: The default I mean is the architecture imposed one.
> Leaving aside Thumb mode, Arm instructions are all 32-bit words, and
> hence less than 4-byte alignment makes no sense (and may even be
> disallowed by the architecture). Whereas for x86 what you're talking
> about is just a compiler default, which isn't really guaranteed to
> never be lower (with -Os for example I'd expect it to be perhaps as
> low as 1).

Right, it's a compiler default, but in patch 1 we already set the
default alignment for x86 to be 16 bytes.

When in your first comment you mentioned "... default for an arch is
anyway" I assumed you mean the default in the arch Kconfig file, not
what the ISA mandates.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 09:45:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 09:45:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685934.1067409 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reu1H-0004RQ-Gv; Tue, 27 Feb 2024 09:45:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685934.1067409; Tue, 27 Feb 2024 09:45:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reu1H-0004R0-C4; Tue, 27 Feb 2024 09:45:03 +0000
Received: by outflank-mailman (input) for mailman id 685934;
 Tue, 27 Feb 2024 09:45:02 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reu1G-0004QU-Jl; Tue, 27 Feb 2024 09:45:02 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reu1G-0004eP-HU; Tue, 27 Feb 2024 09:45:02 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reu1G-0000To-6K; Tue, 27 Feb 2024 09:45:02 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1reu1G-0005Oq-5n; Tue, 27 Feb 2024 09:45:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=i3CXqPIh+29uHPt7Az6EbWLDNq3CuXS09IPx2U0cfwY=; b=vnpkMSQ+47zu6ogxG2Q8BKtsHs
	3TPOMxzuLqQU2Ha6Pu2JQ1SWFBP6RyiavtGa14s+oZw8CtcLfgYcKsH9KFMqzC6pQO2H3Id/yWjNW
	0NNTYGBP9mNkuFMPd3/+CH7ZrR5dLVf8uCrkuyR38CbMZ87zqtlAicv1wfc2GgGKCGZo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184776-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184776: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-amd64-i386-qemut-rhel6hvm-intel:guest-start/redhat.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-dom0pvh-xl-amd:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=03fb5f503cb5faa0556b23d594496719ced6a11b
X-Osstest-Versions-That:
    xen=8de3afc0b402bc17f65093a53e5870862707a8c7
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 09:45:02 +0000

flight 184776 xen-unstable real [real]
flight 184779 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184776/
http://logs.test-lab.xenproject.org/osstest/logs/184779/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-qemut-rhel6hvm-intel 14 guest-start/redhat.repeat fail pass in 184779-retest
 test-amd64-amd64-dom0pvh-xl-amd 22 guest-start/debian.repeat fail pass in 184779-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop      fail blocked in 184767
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184767
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184767
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184767
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184767
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184767
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184767
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184767
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184767
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184767
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184767
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184767
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  03fb5f503cb5faa0556b23d594496719ced6a11b
baseline version:
 xen                  8de3afc0b402bc17f65093a53e5870862707a8c7

Last test of basis   184767  2024-02-26 13:08:57 Z    0 days
Testing same since   184776  2024-02-26 23:10:57 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  George Dunlap <george.dunlap@cloud.com>
  Jan Beulich <jbeulich@suse.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              fail    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   8de3afc0b4..03fb5f503c  03fb5f503cb5faa0556b23d594496719ced6a11b -> master


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 09:57:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 09:57:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685940.1067424 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reuDI-0006JE-Hg; Tue, 27 Feb 2024 09:57:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685940.1067424; Tue, 27 Feb 2024 09:57:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reuDI-0006J7-Dw; Tue, 27 Feb 2024 09:57:28 +0000
Received: by outflank-mailman (input) for mailman id 685940;
 Tue, 27 Feb 2024 09:57:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reuDH-0006Ix-Ja; Tue, 27 Feb 2024 09:57:27 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reuDH-0004ru-Gz; Tue, 27 Feb 2024 09:57:27 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1reuDH-0000mq-86; Tue, 27 Feb 2024 09:57:27 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1reuDH-000239-7V; Tue, 27 Feb 2024 09:57:27 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=/pqsm+5zaz8odu8fNjv1bquHPKZYRct6Ub24Ux+kZ4I=; b=J5XwiB27NvUXJ+vTgYSL76d40D
	sLpjnIc706uYY3ZRlLlyEtVa+jqoTrq0JnULiBEzo5/ic1E8i797QVFsXItkb4QKLAtbZtM2HU4Iq
	HyKO0CwS8q4LPgCgk/wGjVbD+kaK5ioYh/dTy9XP9mw1zzqQW6OMtNDFW+WrmvTqLU54=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184778-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184778: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=74b5309da9fb7a919bec5a8b5a63d1ede5eb6745
X-Osstest-Versions-That:
    ovmf=33c81c25bbd55141ad395af89160b4ed4b703cdf
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 09:57:27 +0000

flight 184778 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184778/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 74b5309da9fb7a919bec5a8b5a63d1ede5eb6745
baseline version:
 ovmf                 33c81c25bbd55141ad395af89160b4ed4b703cdf

Last test of basis   184777  2024-02-27 01:11:23 Z    0 days
Testing same since   184778  2024-02-27 07:44:39 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Abner Chang <abner.chang@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   33c81c25bb..74b5309da9  74b5309da9fb7a919bec5a8b5a63d1ede5eb6745 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 10:23:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 10:23:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685947.1067434 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reucC-0001vK-K4; Tue, 27 Feb 2024 10:23:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685947.1067434; Tue, 27 Feb 2024 10:23:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reucC-0001vD-Hc; Tue, 27 Feb 2024 10:23:12 +0000
Received: by outflank-mailman (input) for mailman id 685947;
 Tue, 27 Feb 2024 10:23:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Avpg=KE=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1reucA-0001uu-Jj
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 10:23:10 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 33c0813c-d55a-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 11:23:08 +0100 (CET)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-565d1656c12so4109817a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 02:23:08 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 33c0813c-d55a-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709029388; x=1709634188; darn=lists.xenproject.org;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=gs1bJQGyFtyT9dDiE+4oMh1B/DXPh7iVC2jtxolcNZ4=;
        b=QsMHoHotgYd2z+6mA6sRWFdqtXlNFehMroNdXlKiYkuHYTrNq1QAA3EOPYtpkdkQdR
         OpQV6NlGZI3BcMgrYiDffQGhYWLhJPTaesR6xI6zf7vDDD6yuycktNBvxRvpcEOKDTtp
         unm6kA+dkVVrBNsh17Rn2evC/jSfD0vMIydUo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709029388; x=1709634188;
        h=to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gs1bJQGyFtyT9dDiE+4oMh1B/DXPh7iVC2jtxolcNZ4=;
        b=GopuQuGxnJnssjtRibdBhO9KTyp/cF5lmwVDkTep99l9J4CJeg0XfrIqjKQFrNoiuc
         EMwsmVaxpl/vSf+U893iRVUO32e/uGMZI8DbWQwGp7ojyabGCj6uJJZCP3S13WoL/LMg
         bbH6sIfJPij5dUUsgfX3wuE6JIYFEaxwdL1g54GA5DwPCsCZVXTVFOpdqgdS7RqVisuS
         2N5T8Vzmk4hYwx8h1fnhVNFwU/fRnV84O1fqU0lVbA4B39xnRvo0izzBL+ikmxUFU/T5
         Dpr+7IsCVspAu6HJfwFVwpfqUdeP7Oj8n6QQ2qm2jLkxbByYhDsZBUkzigTQxxVvkQWY
         XF3g==
X-Gm-Message-State: AOJu0YxoCugl1yxAi/ivhJ8tmYb0qQvBjIdyGUETH9zRSeA0GMz9dq74
	YeeD/5FCGnOwftIYmaNOXUbrWMZgW/Kao9h/c7rCuzeZnP5BM8cg4U1ZMelPyRgvsG9aOyCo42G
	nBOLTjPHSbl09xYMave0ZkWW2J7Mmj6Rn8KKfZPKhXXFAPhZF+/k=
X-Google-Smtp-Source: AGHT+IECtribn+1k2KDdX1D+PWMwwsLUXlAjrwvib0bNLfuDzqO1SZmdodQ7GhZY2sp0Rm+V3USV/AmPs3PuXHhA3pE=
X-Received: by 2002:a05:6402:530c:b0:561:3b53:d0af with SMTP id
 eo12-20020a056402530c00b005613b53d0afmr8766028edb.12.1709029387829; Tue, 27
 Feb 2024 02:23:07 -0800 (PST)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Tue, 27 Feb 2024 10:22:31 +0000
Message-ID: <CAO-mL=zCVa+bbp3ft9GrnaJ0nVHa-9NbjJP_0uR7c4xGjAkn=A@mail.gmail.com>
Subject: XEN SUMMIT CFP DEADLINE - END OF THIS WEEK!
To: xen-devel@lists.xenproject.org, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Content-Type: multipart/alternative; boundary="00000000000092a4ac06125a6928"

--00000000000092a4ac06125a6928
Content-Type: text/plain; charset="UTF-8"

Hi everyone,

*Just a reminder that our CFP for Xen Summit 2024 is at the end of this
week! *

Please submit your talks before then:
https://events.linuxfoundation.org/xen-project-summit/program/cfp/

We look forward to seeing you.

Many thanks,
Kelly Choi

Community Manager
Xen Project

--00000000000092a4ac06125a6928
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi everyone,<div><br></div><div><b><font color=3D"#ff0000"=
>Just a reminder that our CFP for Xen Summit 2024 is at the end of this wee=
k!=C2=A0</font></b></div><div><b><font color=3D"#ff0000"><br></font></b></d=
iv><div>Please submit your talks before then:</div><div><a href=3D"https://=
events.linuxfoundation.org/xen-project-summit/program/cfp/">https://events.=
linuxfoundation.org/xen-project-summit/program/cfp/</a><br></div><div><br><=
/div><div>We look=C2=A0forward to seeing you.</div><div><br></div><div><div=
><div dir=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_signatu=
re"><div dir=3D"ltr"><div>Many thanks,</div><div>Kelly Choi</div><div><br><=
/div><div><div style=3D"color:rgb(136,136,136)">Community Manager</div><div=
 style=3D"color:rgb(136,136,136)">Xen Project=C2=A0<br></div></div></div></=
div></div></div></div>

--00000000000092a4ac06125a6928--


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:07:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:07:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685997.1067474 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revIc-0001GQ-IF; Tue, 27 Feb 2024 11:07:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685997.1067474; Tue, 27 Feb 2024 11:07:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revIc-0001GJ-D2; Tue, 27 Feb 2024 11:07:02 +0000
Received: by outflank-mailman (input) for mailman id 685997;
 Tue, 27 Feb 2024 11:07:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1revIb-0001GD-4W
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:07:01 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 535d9570-d560-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 12:06:58 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-512b29f82d1so5652519e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:06:58 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 h11-20020a05620a21cb00b00787bc4a187dsm3461692qka.98.2024.02.27.03.06.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:06:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 535d9570-d560-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709032018; x=1709636818; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=P56KoYlKn+q6+EBgOUVJxkt0jtKwzfqfiZxHZSHtIfE=;
        b=nmqx87OQJGwqMjwwxXJpNw+1hZ0M83eBbF/bWLPfRbpTHgIgQb0d0vgJjLis0gEbh7
         IKtMbMKKptmGZx4bnqD5k80390ghy0D6NHwhNpgUzWDKsAZNXLR0zFO3iSDaso9/uBVV
         w5EQfMeKIUoOyW+aM5gK8qtiRhQGQ0KSwrg/A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709032018; x=1709636818;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=P56KoYlKn+q6+EBgOUVJxkt0jtKwzfqfiZxHZSHtIfE=;
        b=cA2WGjPSATvOAi2POPqqTIdJ2uhsI6QuXwRqTVQyf4vcKzy+kBbjuyIdng5DjCRC3r
         tx4SXV+j/b0irZFybFGSdXnYPAtHgovWh+qkfrEiJu+Et3R3msB/kAIvIQAbP0zrgEds
         d9gv6ptP+X4bhq64CJf3jVEB4T6M02EkKblBSDm/JclZWkGQyxEviqcuRUSp6FG+MQMJ
         +6aTx4WXBOFWOmpZxgP6a0S0DKi7mleiAh+ekKTJfLBpdkrAjn/fbahGQuj6yZWmxaD6
         BH7FY8hyGA5b2+LJT98pDohILyPn65u6uGiIMK7axs4Up4+yQK52kf3IESn83YUGPfr9
         NiFw==
X-Gm-Message-State: AOJu0Yzo+0rkEL5VQv9bAHSNtjpjD4bXtFa2pOr4Bv3Bo1gdFthox6KY
	UsfrEEnzYnnGsEgnQ1gv/4y0vCGu6l3cI2YWB3IMSBGu4VWk4JQgNqO9RYCFL6A=
X-Google-Smtp-Source: AGHT+IE2DH+fWiPGdbX1ZcQRr0ouT2AENOBZBeiWlI14QRzc9Ilp8XTeQWHpErGElXAVuvDrwyYk/Q==
X-Received: by 2002:a05:6512:1053:b0:512:bf09:624a with SMTP id c19-20020a056512105300b00512bf09624amr6724437lfb.66.1709032018099;
        Tue, 27 Feb 2024 03:06:58 -0800 (PST)
Date: Tue, 27 Feb 2024 12:06:55 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Subject: Re: [PATCH v13.3 01/14] vpci: use per-domain PCI lock to protect
 vpci structure
Message-ID: <Zd3CT37FrXU14qFa@macbook>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <20240221024504.357840-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240221024504.357840-1-stewart.hildebrand@amd.com>

On Tue, Feb 20, 2024 at 09:45:03PM -0500, Stewart Hildebrand wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> 
> Use the per-domain PCI read/write lock to protect the presence of the
> pci device vpci field. This lock can be used (and in a few cases is used
> right away) so that vpci removal can be performed while holding the lock
> in write mode. Previously such removal could race with vpci_read for
> example.
> 
> When taking both d->pci_lock and pdev->vpci->lock, they should be
> taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
> possible deadlock situations.
> 
> 1. Per-domain's pci_lock is used to protect pdev->vpci structure
> from being removed.
> 
> 2. Writing the command register and ROM BAR register may trigger
> modify_bars to run, which in turn may access multiple pdevs while
> checking for the existing BAR's overlap. The overlapping check, if
> done under the read lock, requires vpci->lock to be acquired on both
> devices being compared, which may produce a deadlock. It is not
> possible to upgrade read lock to write lock in such a case. So, in
> order to prevent the deadlock, use d->pci_lock in write mode instead.
> 
> All other code, which doesn't lead to pdev->vpci destruction and does
> not access multiple pdevs at the same time, can still use a
> combination of the read lock and pdev->vpci->lock.
> 
> 3. Drop const qualifier where the new rwlock is used and this is
> appropriate.
> 
> 4. Do not call process_pending_softirqs with any locks held. For that
> unlock prior the call and re-acquire the locks after. After
> re-acquiring the lock there is no need to check if pdev->vpci exists:
>  - in apply_map because of the context it is called (no race condition
>    possible)
>  - for MSI/MSI-X debug code because it is called at the end of
>    pdev->vpci access and no further access to pdev->vpci is made
> 
> 5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
> while accessing pdevs in vpci code.
> 
> 6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
> do not go away. The vPCI functions call several MSI-related functions
> which already have existing non-vPCI callers. Change those MSI-related
> functions to allow using either pcidevs_lock() or d->pci_lock for
> ensuring pdevs do not go away. Holding d->pci_lock in read mode is
> sufficient. Note that this pdev protection mechanism does not protect
> other state or critical sections. These MSI-related functions already
> have other race condition and state protection mechanims (e.g.
> d->event_lock and msixtbl RCU), so we deduce that the use of the global
> pcidevs_lock() is to ensure that pdevs do not go away.
> 
> 7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
> that pdevs do not go away. The purpose of this wrapper is to aid
> readability and document the intent of the pdev protection mechanism.
> 
> 8. When possible, the existing non-vPCI callers of these MSI-related
> functions haven't been switched to use the newly introduced per-domain
> pci_lock, and will continue to use the global pcidevs_lock(). This is
> done to reduce the risk of the new locking scheme introducing
> regressions. Those users will be adjusted in due time. One exception
> is where the pcidevs_lock() in allocate_and_map_msi_pirq() is moved to
> the caller, physdev_map_pirq(): this instance is switched to
> read_lock(&d->pci_lock) right away.
> 
> Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:08:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:08:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.685999.1067483 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revJu-0001mw-Pn; Tue, 27 Feb 2024 11:08:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 685999.1067483; Tue, 27 Feb 2024 11:08:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revJu-0001mp-Mj; Tue, 27 Feb 2024 11:08:22 +0000
Received: by outflank-mailman (input) for mailman id 685999;
 Tue, 27 Feb 2024 11:08:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1revJt-0001mh-UB
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:08:21 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 843471ab-d560-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 12:08:20 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-412949fd224so21383325e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:08:20 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 qj24-20020a056214321800b0068f9fd1d688sm4006600qvb.21.2024.02.27.03.08.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:08:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 843471ab-d560-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709032100; x=1709636900; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=hjCem2GDc226H2NfOzwHUZjIVg4IVbP1kpH+LPAyKbA=;
        b=Px8Vr+3Sx4hwz+rTdalbfzBLk3YNA3NKViXsCjF5tOq01NsIWemac4M3t/Hi2FnY9X
         dHj/NyfFoDdJgUZH4bIgRqbRbbqouSnbErx918PNDIVsTmHxMcZeqBkOag6YahZh19KQ
         sYiEUa25qPu+yqegEvSGo3EmweAfsMIzEvOyc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709032100; x=1709636900;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=hjCem2GDc226H2NfOzwHUZjIVg4IVbP1kpH+LPAyKbA=;
        b=ezbza01Yb+wCf9KpO58qKZpomNF7BmDpJn6QKLJHEb/9VOO1TC1okrVsy+UeWKrAMq
         vAkB1t0sXH45A5ZhxGF79aNZWNJIKJOCMXFjVr7XH+jgOJjKsXbRfGKPmOGYHokvmXHL
         nvGo/Rd4x8SWLAX6tRbOckuMiaEG4Qk/7A1mfvdjrIE86aSXa4ZGdFosWBZb1zLhqA+n
         GiPF5dHuf7Vp9vqHLZhdwzGMmG5mE2RFe/qkZSAccrzNFuzGEQhXnhtKUJHQDNlEymql
         5rP9xYzivioKTXDe6dJ7dS36CHy8IgUExjwm7d1/UpeEdA6q3jqpMewvIikEBMZX2KP4
         WEFw==
X-Forwarded-Encrypted: i=1; AJvYcCWwYLCUbHVJ77kn5zPnOJDY35UQ8/ildOhSBI0xFsMaHMJXK6Po2X5IQ3atHNMHGx65FxWC9oaIZopFri/b6f85GN4dAKDIUSyHCze/h84=
X-Gm-Message-State: AOJu0YxkmNgAxYk9gKdXsXL1oz/+Z5ed/EiBAm7eS/7tohNNzRlLpnJm
	bqq300ly7+eZ/PVex+zLdHdV7iK2zY3B1+Q9sZOqdqQ9DeM8lLqJnvEdOEvWfxM=
X-Google-Smtp-Source: AGHT+IH1qo2rkVtqEz0aVpOgQQsjmoKQJH4cxoh8N4/CsAV0YwaJkBHoa7Cqwy/6qxobU9SVSZ7WKA==
X-Received: by 2002:adf:cd07:0:b0:33d:f1d4:37e0 with SMTP id w7-20020adfcd07000000b0033df1d437e0mr1096690wrm.4.1709032100237;
        Tue, 27 Feb 2024 03:08:20 -0800 (PST)
Date: Tue, 27 Feb 2024 12:08:17 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stewart Hildebrand <stewart.hildebrand@amd.com>,
	Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Kevin Tian <kevin.tian@intel.com>, Paul Durrant <paul@xen.org>,
	Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v13.3 01/14] vpci: use per-domain PCI lock to protect
 vpci structure
Message-ID: <Zd3CoaSHtU59PjZc@macbook>
References: <20240202213321.1920347-1-stewart.hildebrand@amd.com>
 <20240202213321.1920347-2-stewart.hildebrand@amd.com>
 <20240221024504.357840-1-stewart.hildebrand@amd.com>
 <6024f2c9-90e6-4ce9-a421-c0ec7441a926@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <6024f2c9-90e6-4ce9-a421-c0ec7441a926@suse.com>

On Mon, Feb 26, 2024 at 03:47:17PM +0100, Jan Beulich wrote:
> On 21.02.2024 03:45, Stewart Hildebrand wrote:
> > From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > 
> > Use the per-domain PCI read/write lock to protect the presence of the
> > pci device vpci field. This lock can be used (and in a few cases is used
> > right away) so that vpci removal can be performed while holding the lock
> > in write mode. Previously such removal could race with vpci_read for
> > example.
> > 
> > When taking both d->pci_lock and pdev->vpci->lock, they should be
> > taken in this exact order: d->pci_lock then pdev->vpci->lock to avoid
> > possible deadlock situations.
> > 
> > 1. Per-domain's pci_lock is used to protect pdev->vpci structure
> > from being removed.
> > 
> > 2. Writing the command register and ROM BAR register may trigger
> > modify_bars to run, which in turn may access multiple pdevs while
> > checking for the existing BAR's overlap. The overlapping check, if
> > done under the read lock, requires vpci->lock to be acquired on both
> > devices being compared, which may produce a deadlock. It is not
> > possible to upgrade read lock to write lock in such a case. So, in
> > order to prevent the deadlock, use d->pci_lock in write mode instead.
> > 
> > All other code, which doesn't lead to pdev->vpci destruction and does
> > not access multiple pdevs at the same time, can still use a
> > combination of the read lock and pdev->vpci->lock.
> > 
> > 3. Drop const qualifier where the new rwlock is used and this is
> > appropriate.
> > 
> > 4. Do not call process_pending_softirqs with any locks held. For that
> > unlock prior the call and re-acquire the locks after. After
> > re-acquiring the lock there is no need to check if pdev->vpci exists:
> >  - in apply_map because of the context it is called (no race condition
> >    possible)
> >  - for MSI/MSI-X debug code because it is called at the end of
> >    pdev->vpci access and no further access to pdev->vpci is made
> > 
> > 5. Use d->pci_lock around for_each_pdev and pci_get_pdev()
> > while accessing pdevs in vpci code.
> > 
> > 6. Switch vPCI functions to use per-domain pci_lock for ensuring pdevs
> > do not go away. The vPCI functions call several MSI-related functions
> > which already have existing non-vPCI callers. Change those MSI-related
> > functions to allow using either pcidevs_lock() or d->pci_lock for
> > ensuring pdevs do not go away. Holding d->pci_lock in read mode is
> > sufficient. Note that this pdev protection mechanism does not protect
> > other state or critical sections. These MSI-related functions already
> > have other race condition and state protection mechanims (e.g.
> > d->event_lock and msixtbl RCU), so we deduce that the use of the global
> > pcidevs_lock() is to ensure that pdevs do not go away.
> > 
> > 7. Introduce wrapper construct, pdev_list_is_read_locked(), for checking
> > that pdevs do not go away. The purpose of this wrapper is to aid
> > readability and document the intent of the pdev protection mechanism.
> > 
> > 8. When possible, the existing non-vPCI callers of these MSI-related
> > functions haven't been switched to use the newly introduced per-domain
> > pci_lock, and will continue to use the global pcidevs_lock(). This is
> > done to reduce the risk of the new locking scheme introducing
> > regressions. Those users will be adjusted in due time. One exception
> > is where the pcidevs_lock() in allocate_and_map_msi_pirq() is moved to
> > the caller, physdev_map_pirq(): this instance is switched to
> > read_lock(&d->pci_lock) right away.
> > 
> > Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
> > Suggested-by: Jan Beulich <jbeulich@suse.com>
> > Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> > Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> > Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> 
> Acked-by: Jan Beulich <jbeulich@suse.com>
> with two small remaining remarks (below) and on the assumption that an
> R-b from Roger in particular for the vPCI code is going to turn up
> eventually.
> 
> > @@ -895,6 +891,15 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
> >  {
> >      unsigned int i;
> >  
> > +    /*
> > +     * Assert that pdev_list doesn't change. ASSERT_PDEV_LIST_IS_READ_LOCKED
> > +     * is not suitable here because it may allow either pcidevs_lock() or
> > +     * pci_lock to be held, but here we rely on pci_lock being held, not
> > +     * pcidevs_lock().
> > +     */
> > +    ASSERT(rw_is_locked(&msix->pdev->domain->pci_lock));
> > +    ASSERT(spin_is_locked(&msix->pdev->vpci->lock));
> 
> As to the comment, I think it's not really "may". I also think referral to
> ...

+1 to dropping 'may'.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:13:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:13:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686005.1067492 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revOc-0003m4-AZ; Tue, 27 Feb 2024 11:13:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686005.1067492; Tue, 27 Feb 2024 11:13:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revOc-0003lx-7K; Tue, 27 Feb 2024 11:13:14 +0000
Received: by outflank-mailman (input) for mailman id 686005;
 Tue, 27 Feb 2024 11:13:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1revOb-0003lr-Cz
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:13:13 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 321358a2-d561-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 12:13:12 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-512ed314881so4198507e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:13:12 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 xy23-20020a05620a5dd700b00787bdbeeaa3sm3486876qkn.107.2024.02.27.03.13.11
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:13:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 321358a2-d561-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709032392; x=1709637192; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=jH8zQMVgDn4lne2+zjTbSXco8BxL7Aa6h7rnmCxKkVM=;
        b=hlXUaQCZxWRErFEqjbjALyGwF6sDVJr8EGRMzlsTxkJrBPV1XusxRKrkS/mTEdBzgX
         53Wt1iKPZlblm8LHtoek4TdoCMtLBDRsoqdFoBHbx1g7vCc+ewbPIxP22idhB7trpoBO
         fL+Z/LfodvVyIS/ly+IhpMMtgXz2UF8saQl/Q=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709032392; x=1709637192;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jH8zQMVgDn4lne2+zjTbSXco8BxL7Aa6h7rnmCxKkVM=;
        b=w/rbTphb1Ya+YgkqheuSjLO62ZDjbVnFoMwtKaszpTQXKDly7WUH7EOfpJQlX/Qzfe
         yebdBTdNOoE+FBKdpy4MSMbAHZyauzYBPl/COKDr9Ma7rvE/49EZQbEbH30ckancy+Q3
         bme9htJ5M38ZkyNP5lave2MKJzqnguIhhLCd71rGKD1X49nYd3+WqvnQp/CFhmqgb/El
         6QmvKlInomNDMa3ZLRIlwzRFEc4oq7Elm5qavz4psxRXBxU7jASCbyzDVBar83g0zqjw
         p9TEkNEJOptIFtdUxAIksWHMDXJM4d79KhSU6hT9tR/Ab0eUWWke54Wdf7zmFvrao7IS
         JYrA==
X-Forwarded-Encrypted: i=1; AJvYcCW7dz641fiHthJUJaaaKzpqkL9mQmYhSpayZD8MNrIOAYe8lORTaquEOp5iLi8eCqVhIVHhb72xyjcoJpcb7GYasN85xkrsNSzcWPlFCo8=
X-Gm-Message-State: AOJu0YyEtaFEAmCC4Y31azsnZfWtxyaFVdPDFs23ybJfsqdQuAMbXDkY
	0eH54aFK/NsK72+MrkOHIBsfy5ClYzwByxHQiXkSg6kJ06N5Ng1k8odIjJDtyPo=
X-Google-Smtp-Source: AGHT+IH2HTQrKv2RDrlqLsIV6Tn4RRVahK7yQzdNazFa0k6AtGIubOh4b0s3yBEugww1BRQRVwq2eA==
X-Received: by 2002:a19:f719:0:b0:512:ac3a:7f27 with SMTP id z25-20020a19f719000000b00512ac3a7f27mr5078816lfe.66.1709032391999;
        Tue, 27 Feb 2024 03:13:11 -0800 (PST)
Date: Tue, 27 Feb 2024 12:13:09 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 1/4] x86/spec: print the built-in SPECULATIVE_HARDEN_*
 options
Message-ID: <Zd3DxW88JoyUSU4m@macbook>
References: <20240226110748.80254-1-roger.pau@citrix.com>
 <20240226110748.80254-2-roger.pau@citrix.com>
 <85fa5e55-27c5-495d-bd7b-857260b1de20@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <85fa5e55-27c5-495d-bd7b-857260b1de20@suse.com>

On Mon, Feb 26, 2024 at 01:39:49PM +0100, Jan Beulich wrote:
> On 26.02.2024 12:07, Roger Pau Monne wrote:
> > Just like it's done for INDIRECT_THUNK and SHADOW_PAGING.
> > 
> > Reported-by: Jan Beulich <jbeulich@suse.com>
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> In principle
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> but ...
> 
> > --- a/xen/arch/x86/spec_ctrl.c
> > +++ b/xen/arch/x86/spec_ctrl.c
> > @@ -466,13 +466,25 @@ static void __init print_details(enum ind_thunk thunk)
> >             (e21a & cpufeat_mask(X86_FEATURE_SBPB))           ? " SBPB"           : "");
> >  
> >      /* Compiled-in support which pertains to mitigations. */
> > -    if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) || IS_ENABLED(CONFIG_SHADOW_PAGING) )
> > +    if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) || IS_ENABLED(CONFIG_SHADOW_PAGING) ||
> > +         IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_ARRAY) ||
> > +         IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_BRANCH) ||
> > +         IS_ENABLED(CONFIG_SPECULATIVE_HARDEN_GUEST_ACCESS) )
> >          printk("  Compiled-in support:"
> >  #ifdef CONFIG_INDIRECT_THUNK
> >                 " INDIRECT_THUNK"
> >  #endif
> >  #ifdef CONFIG_SHADOW_PAGING
> >                 " SHADOW_PAGING"
> > +#endif
> > +#ifdef CONFIG_SPECULATIVE_HARDEN_ARRAY
> > +               " SPECULATIVE_HARDEN_ARRAY"
> > +#endif
> > +#ifdef CONFIG_SPECULATIVE_HARDEN_BRANCH
> > +               " SPECULATIVE_HARDEN_BRANCH"
> > +#endif
> > +#ifdef CONFIG_SPECULATIVE_HARDEN_GUEST_ACCESS
> > +               " SPECULATIVE_HARDEN_GUEST_ACCESS"
> >  #endif
> 
> ... I'd like to suggest to drop the SPECULATIVE_ from the string literals.
> They're relevant in the Kconfig identifiers, but they're imo redundant in
> the context of these log messages. (Happy to adjust while committing, if
> need be.)

Oh, yes, indeed, we already print HARDEN_BRANCH instead of
SPECULATIVE_HARDEN_BRANCH.  Please adjust at commit if you don't mind.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:18:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:18:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686008.1067503 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revTy-0004PB-TQ; Tue, 27 Feb 2024 11:18:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686008.1067503; Tue, 27 Feb 2024 11:18:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revTy-0004P4-Qc; Tue, 27 Feb 2024 11:18:46 +0000
Received: by outflank-mailman (input) for mailman id 686008;
 Tue, 27 Feb 2024 11:18:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1revTx-0004Oy-Dz
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:18:45 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f7561ae6-d561-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 12:18:43 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-512f5484a37so3669889e87.3
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:18:43 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 kr3-20020a0562142b8300b0068fd10638c0sm4012956qvb.20.2024.02.27.03.18.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:18:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f7561ae6-d561-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709032723; x=1709637523; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=H6XqcWk1QCsXWbFTmS8LaYKzCZDVJ7qxVs8gVsVfhhQ=;
        b=ug/hxxkmC0FtyCIKRZA/+g2e0w9scwRRXIXILvEWPVP2YkfTylkbz+jjsKWV7wDfnu
         mNQwyyCIrImG6TO97u97A55vuJZwyN9QsE5vSPsn9OqWB2bj9OlHiu3wDqPsvr1/9Hiu
         /r9PSCcuoXPv/g5rulE7r8vlw2+qyUI2IVGIs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709032723; x=1709637523;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=H6XqcWk1QCsXWbFTmS8LaYKzCZDVJ7qxVs8gVsVfhhQ=;
        b=BR73BRIxxnez+wK1X6oCNpYXs3XE1AazyOjQGvMBLmlXkKrPPdaOfXTKFb985IKJ3K
         Jpmb/Drdj26GCcfMPQn3tqfA26EmpgynI1pdrF9zKnm44Prcq9C56NdDCcZl3vZwbrLr
         scz5Kjtx7VYV++7rwFU18YnDw6fWQxawaow7vOTwmUaBzv0L9xCmQ5FcWXhPWbsCUrf9
         2eBoelhxHQqZxBj+xry9v8y0mLPA4PsWE/wWKzcETcE7q/wfZuwgKwMpp6fBDktqITDk
         H/bx6O1Z+8re0G79pxCTwRs4dreZ1DH25XSpLw8WcHzB9S12iJqJ94bOahTyGZqh8TGm
         1kfg==
X-Forwarded-Encrypted: i=1; AJvYcCVf5RAb4detp5rwL+GuGnlLIlvzwdJ3kJv4a+gpobBf8d+2INSx2JfDMvKYph50i/sVadT9KYr0Y3IocYwVLyeRAf75kyf61MTS0j5xIsc=
X-Gm-Message-State: AOJu0YxpMXmvCVOdLFy5Nr5H7vkTEvuSE/k0lRjVjjKEGsd6/tfmSnZa
	QwV5L2sm+IKgIAYxS2nrFdhmlIz5ga2C5vFdZxbjlVwZP2vurd64OFPv4QClpjk=
X-Google-Smtp-Source: AGHT+IFj2vRVJy1b+0pBQ6MQxpNJsw740qXrbNVmfYTIYHSqQpkEFdKwcdJJ8t5yN3KayHtPErGFAA==
X-Received: by 2002:a05:6512:3da9:b0:512:bd65:860a with SMTP id k41-20020a0565123da900b00512bd65860amr8523131lfv.5.1709032722845;
        Tue, 27 Feb 2024 03:18:42 -0800 (PST)
Date: Tue, 27 Feb 2024 12:18:40 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 3/4] x86/spec: fix INDIRECT_THUNK option to only be
 set when build-enabled
Message-ID: <Zd3FEA_tt4GJexSC@macbook>
References: <20240226110748.80254-1-roger.pau@citrix.com>
 <20240226110748.80254-4-roger.pau@citrix.com>
 <81e32609-37cd-4c2d-ae27-6caafc7dab33@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <81e32609-37cd-4c2d-ae27-6caafc7dab33@suse.com>

On Mon, Feb 26, 2024 at 01:50:46PM +0100, Jan Beulich wrote:
> On 26.02.2024 12:07, Roger Pau Monne wrote:
> > Attempt to provide a more helpful error message when the user attempts to set
> > spec-ctrl=bti-thunk option but the support is build-time disabled.
> > 
> > While there also adjust the command line documentation to mention
> > CONFIG_INDIRECT_THUNK instead of INDIRECT_THUNK.
> > 
> > Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> with one minor remark:
> 
> > --- a/xen/arch/x86/spec_ctrl.c
> > +++ b/xen/arch/x86/spec_ctrl.c
> > @@ -241,7 +241,12 @@ static int __init cf_check parse_spec_ctrl(const char *s)
> >          {
> >              s += 10;
> >  
> > -            if ( !cmdline_strcmp(s, "retpoline") )
> > +            if ( !IS_ENABLED(CONFIG_INDIRECT_THUNK) )
> > +            {
> > +                no_config_param("INDIRECT_THUNK", "spec-ctrl=bti-thunk", s, ss);
> > +                rc = -EINVAL;
> > +            }
> > +            else if ( !cmdline_strcmp(s, "retpoline") )
> >                  opt_thunk = THUNK_RETPOLINE;
> >              else if ( !cmdline_strcmp(s, "lfence") )
> >                  opt_thunk = THUNK_LFENCE;
> 
> How about
> 
>             if ( !IS_ENABLED(CONFIG_INDIRECT_THUNK) )
>             {
>                 no_config_param("INDIRECT_THUNK", "spec-ctrl", s - 10, ss);
>                 rc = -EINVAL;
>             }
>             else if ( !cmdline_strcmp(s, "retpoline") )
> 
> or (likely less liked by you and Andrew) "s += 10;" dropped and then
> 
>             if ( !IS_ENABLED(CONFIG_INDIRECT_THUNK) )
>             {
>                 no_config_param("INDIRECT_THUNK", "spec-ctrl", s, ss);
>                 rc = -EINVAL;
>             }
>             else if ( !cmdline_strcmp(s += 10, "retpoline") )
> 
> conserving a little on string literal space (sadly, despite the function
> being __init, string literals remain post-init due to living in .rodata)?

TBH I'm not overly worried about those 10 extra characters, but if I
had to choose I prefer the first option (so `s - 10`).

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:21:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:21:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686011.1067512 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revWA-0006BY-C5; Tue, 27 Feb 2024 11:21:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686011.1067512; Tue, 27 Feb 2024 11:21:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revWA-0006BR-9O; Tue, 27 Feb 2024 11:21:02 +0000
Received: by outflank-mailman (input) for mailman id 686011;
 Tue, 27 Feb 2024 11:21:00 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1revW8-0006BL-Dc
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:21:00 +0000
Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com
 [2a00:1450:4864:20::334])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 47f65cba-d562-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 12:20:58 +0100 (CET)
Received: by mail-wm1-x334.google.com with SMTP id
 5b1f17b1804b1-412949fd224so21471725e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:20:58 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 x9-20020ae9f809000000b00787b95c91d3sm3491227qkh.62.2024.02.27.03.20.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:20:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47f65cba-d562-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709032858; x=1709637658; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=i1Cub/YqSvCVVFtSPnLaVSR7dkwlBhVODg+6zhlKa/I=;
        b=t2Lv543V+2rL3uFjeRuJNExN9duTwwJTPtzL9UNofmzFYpvi07+JX8DmUEPW+Fgtg8
         V0y0Y/99ey+ejpdrZSHcwBvD9GPLcrXo2QyayfIyDPBp2h+1VVEkLlWINlVNHyQx+l3o
         bIwv8azBO/myA9PcOVhLPaIZZfz7WvRM3UpbU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709032858; x=1709637658;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=i1Cub/YqSvCVVFtSPnLaVSR7dkwlBhVODg+6zhlKa/I=;
        b=SUFty4WxWM44keB4kderracbCojHx68hVS7ANB5TSYS1gdoEs6Ce5Gw0l67EgKWR4d
         QVEy6zzekfl+/O0pAlVOPA6ekYSKueJPjGugOKoJu0Lb+5rg6KQ2Vn2uowD5wJGzyeQC
         ESQJzlOulW8CgOaejyKwIBEE4sCpHhB1FyV1lEkHtyzcD6WBhXWuJn9se73jO1K9ZIPN
         ZIB1I7iyAq/8keuOsVKUPbEWBz2OyKNC2xTXTOg72QD7ckLm6eflcdlT05+LhZ4YzSBT
         6OllXN+7n3gvC/k99HxcDsI8r+0tsUfFUk2C1DTgMJdMDT05kPHTkWwiuwI7b9hmHNKX
         aRyA==
X-Forwarded-Encrypted: i=1; AJvYcCXwi1oSLibWCj6u6kT2zVRFOEsmG0ui5cCXDdkFDDSxekU9BBe3e1Ey2qZVSYTaVxcH5iL3+ujRZkSsxk/na4wrbNp+pEaS2Rp1uQDLkGw=
X-Gm-Message-State: AOJu0Yyij/uQM6wChz8g2loqhF7kvc2GOfW+vqTr5VPp3ZynbQ8x9bac
	OPuXTnddSod9OGjcCg4PKw022BKbVZZKKIeZbEL0qb2EZco6TEdLnCuzLVcZwfw=
X-Google-Smtp-Source: AGHT+IEw7clDteVMfPNJHdusSpmUghVqFMUpYTerFucgttEF8M+ih7uJaB1UCbtl74wm0gooEeEL1w==
X-Received: by 2002:a5d:4701:0:b0:33d:9d49:cfe2 with SMTP id y1-20020a5d4701000000b0033d9d49cfe2mr9040748wrq.21.1709032858007;
        Tue, 27 Feb 2024 03:20:58 -0800 (PST)
Date: Tue, 27 Feb 2024 12:20:55 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH v3 4/4] x86/spec: do not print thunk option selection if
 not built-in
Message-ID: <Zd3Fl6nx-R4aq2wR@macbook>
References: <20240226110748.80254-1-roger.pau@citrix.com>
 <20240226110748.80254-5-roger.pau@citrix.com>
 <30e1bfea-d76b-4e3d-b148-f25d1c811f8d@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <30e1bfea-d76b-4e3d-b148-f25d1c811f8d@suse.com>

On Mon, Feb 26, 2024 at 01:54:54PM +0100, Jan Beulich wrote:
> On 26.02.2024 12:07, Roger Pau Monne wrote:
> > Now that the thunk built-in enable is printed as part of the "Compiled-in
> > support:" line, avoid printing anything in "Xen settings:" if the thunk is
> > disabled at build time.
> 
> Why "Now that ..."? It's other logging the earlier patch adds there.

'Now that ...'  would refer to patch 1/4.

> > Note the BTI-Thunk option printing is also adjusted to print a colon in the
> > same way the other options on the line do.
> > 
> > Requested-by: Jan Beulich <jbeulich@suse.com>
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> With either a clarification of what's meant or e.g. s/Now that/Since/
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

'Since' is likely more appropriate.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:24:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:24:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686015.1067526 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revZE-0006p8-TR; Tue, 27 Feb 2024 11:24:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686015.1067526; Tue, 27 Feb 2024 11:24:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revZE-0006p1-QD; Tue, 27 Feb 2024 11:24:12 +0000
Received: by outflank-mailman (input) for mailman id 686015;
 Tue, 27 Feb 2024 11:24:10 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1revZC-0006ov-Mg
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:24:10 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1revZC-0006ez-CP; Tue, 27 Feb 2024 11:24:10 +0000
Received: from [15.248.2.14] (helo=[10.45.19.66])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1revZC-0001v2-62; Tue, 27 Feb 2024 11:24:10 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=PC1ttbj9W64G47kcVHzRW1GrwvQsq11zrRsFg1UCMr4=; b=kCcQdViTkVNDkIfx8A6dzwL2Br
	y8wjKxk0lukrCM0TG7GJ81RGS8aabgANqq2TBbgxKoV5PM1f65J9Sfq2V58Yl/k/irtR0tZVNnYwC
	yrmM10K4Z6+DL0f8ItPs4MGbFs3rLBgFsanN2SP3OThADeZVjsvx87UD3xNWSWp+ANRA=;
Message-ID: <bbd0a238-05f3-4d2b-8450-d431570e31b4@xen.org>
Date: Tue, 27 Feb 2024 11:24:08 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH v5] xen/arm: ffa: reclaim shared memory on guest
 destroy
Content-Language: en-GB
To: Jens Wiklander <jens.wiklander@linaro.org>, xen-devel@lists.xenproject.org
Cc: patches@linaro.org, Volodymyr Babchuk <volodymyr_babchuk@epam.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>
References: <20240213111336.881934-1-jens.wiklander@linaro.org>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240213111336.881934-1-jens.wiklander@linaro.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 13/02/2024 11:13, Jens Wiklander wrote:
> When an FF-A enabled guest is destroyed it may leave behind memory
> shared with SPs. This memory must be reclaimed before it's reused or an
> SP may make changes to memory used by a new unrelated guest. So when the
> domain is teared down add FF-A requests to reclaim all remaining shared
> memory.
> 
> SPs in the secure world are notified using VM_DESTROYED that a guest has
> been destroyed. An SP is supposed to relinquish all shared memory to allow
> reclaiming the memory. The relinquish operation may need to be delayed if
> the shared memory is for instance part of a DMA operation.
> 
> The domain reference counter is increased when the first FF-A shared
> memory is registered and the counter is decreased again when the last
> shared memory is reclaimed. If FF-A shared memory registrations remain
> at the end of of ffa_domain_teardown() a timer is set to try to reclaim
> the shared memory every second until the memory is reclaimed.
> 
> A few minor style fixes with a removed empty line here and an added new
> line there.
> 
> Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>

This is now committed.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:25:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:25:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686017.1067537 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revaf-0007WT-7J; Tue, 27 Feb 2024 11:25:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686017.1067537; Tue, 27 Feb 2024 11:25:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revaf-0007WM-4R; Tue, 27 Feb 2024 11:25:41 +0000
Received: by outflank-mailman (input) for mailman id 686017;
 Tue, 27 Feb 2024 11:25:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1revae-0007WG-MM
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:25:40 +0000
Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com
 [2a00:1450:4864:20::129])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ef19acba-d562-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 12:25:39 +0100 (CET)
Received: by mail-lf1-x129.google.com with SMTP id
 2adb3069b0e04-5101cd91017so6097244e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:25:39 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 t19-20020a05622a181300b0042e56fb8e0bsm3451425qtc.93.2024.02.27.03.25.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:25:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ef19acba-d562-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709033138; x=1709637938; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=S+zlHOY3NrHjR8jd8VF14b6wXZRMhCsLjnfVmMAKV4Y=;
        b=QiF3t8uTt3ZnA930rxKmRnMMXVWl15EsCVnpGyUo0ZYXN1kKW0b+rxbXZ0BTXbg9I5
         oY+uouKkUj6xOhpFuEiwM8e/LlyNRYISr9faCzqF4Em1cWgQCXd5RxsHTfLSMSq2eEji
         TMAWUJGwijoWQ/ThNzc+D8L3LMvI0vqABu2zU=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709033138; x=1709637938;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=S+zlHOY3NrHjR8jd8VF14b6wXZRMhCsLjnfVmMAKV4Y=;
        b=wk1WmakoZXmh+6u4jiBr9+F0wPpeMk40rVl48o9zxlTg8Yzu6or8XjXzNWkV5lcZoM
         8rE9UktlvMNhb25rAS4zLxDgSS/mmnScxe8yaKL2Ull6GgJ4OvRvFob/xUzIhkU120LE
         1/KQWgsYCb7aUJjXZL9yrHbAiPblyfRlS4uZ+QEf2DzyeU/CXOOU4+iqjg+PHXAXcobt
         n77f1PlmDxdNmKU9euCAmrk0eqyl4d2Fi7H3LCNOitIQz9wj2hMC1yTA1rQL1RJaiUF/
         21J0lEUD8SqhYZ83+ARuAxhBHJQ4vbRLzx0MJgaMefiEOiOj19Xn85hYzUvMxnfYQygJ
         QlPA==
X-Gm-Message-State: AOJu0YykxeV1SwlGVREIHL3+Q2CGIXJh6zXoq/93vZlc8kS3ldFxeimB
	9fReUODlOZZzuIiWVw6+8OiMsU4eO4HjzohbwbjFQZoM0/pZ7Qx07kBKwFGQbhJRVMxKb4hqotH
	m
X-Google-Smtp-Source: AGHT+IGF2bw8FPDMYaFcjGbBwmbWyrti/TTWBrM5mh/itTDXZZJVI/edUAjEpi3mtT6h0loWW2VS+w==
X-Received: by 2002:a05:6512:acf:b0:512:ec7d:7831 with SMTP id n15-20020a0565120acf00b00512ec7d7831mr6753187lfu.26.1709033138410;
        Tue, 27 Feb 2024 03:25:38 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2 0/5] xen/livepatch: fixes for the pre-apply / post-revert hooks
Date: Tue, 27 Feb 2024 12:25:23 +0100
Message-ID: <20240227112528.4540-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

The follow series contain a misc of fixes mostly related to the usage of
the pre-apply / post-revert hooks.  The norevert test is also fixed to
work as I think was expected.  Finally both the no{apply,revert}
tests are fixed to build properly, as the files where previously
unhooked from the build system completely.

I'm unsure how useful the apply and revert hooks really are, as without
calling the internal apply/revert functions the state of the payload
structure is quite likely inconsistent with the code expectations.

Thanks, Roger.

Roger Pau Monne (5):
  xen/livepatch: register livepatch regions when loaded
  xen/livepatch: search for symbols in all loaded payloads
  xen/livepatch: fix norevert test attempt to open-code revert
  xen/livepatch: properly build the noapply and norevert tests
  xen/livepatch: group and document payload hooks

 xen/common/livepatch.c                        | 94 +++++++++++--------
 xen/common/virtual_region.c                   | 42 ++++-----
 xen/include/xen/livepatch.h                   | 32 +------
 xen/include/xen/livepatch_payload.h           | 37 ++++++--
 xen/test/livepatch/Makefile                   |  4 +-
 .../livepatch/xen_action_hooks_norevert.c     | 22 +----
 6 files changed, 112 insertions(+), 119 deletions(-)

-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:25:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:25:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686018.1067546 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revaj-0007mO-FJ; Tue, 27 Feb 2024 11:25:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686018.1067546; Tue, 27 Feb 2024 11:25:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revaj-0007mH-CB; Tue, 27 Feb 2024 11:25:45 +0000
Received: by outflank-mailman (input) for mailman id 686018;
 Tue, 27 Feb 2024 11:25:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1revai-0007l8-83
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:25:44 +0000
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com
 [2a00:1450:4864:20::136])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f0a095c5-d562-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 12:25:41 +0100 (CET)
Received: by mail-lf1-x136.google.com with SMTP id
 2adb3069b0e04-5101cd91017so6097290e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:25:41 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 ow34-20020a05620a822200b00783f8693df1sm3495000qkn.37.2024.02.27.03.25.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:25:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f0a095c5-d562-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709033141; x=1709637941; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=zbRmLbHcypo3JwhtnaEO0O2QVQTm0Ebx3zCXiOOR5V4=;
        b=c3GlLnhuurlM5WjFBDmPEqJWlVF8ym2CfaFqjLPRoDl+7hSVrX/klAj4/4Lc1JMS9L
         tHtvfepIxuGKDZJC+N2hGLvkGU/9AgF4yC8sCbq2qjBuwh8jaWBQtZw/fdZ8ygzhMMCF
         KEpetiMEQdcH2Kte/IoBlsGsQgUVpYN8SwgQ8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709033141; x=1709637941;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=zbRmLbHcypo3JwhtnaEO0O2QVQTm0Ebx3zCXiOOR5V4=;
        b=JHyEho1WmQEpza3H/ylJLVsz72AjQ8njhCBpR7kxVELvOQhRvF/bob+g3g1L8NJLRp
         IVSu/5sN8kWgjEwHY1CXff4KnZHO+gdpIM5Hj+TR6Zm6B+yxDCezPnVwmSnYMeQFUEuX
         OzAyG1qTMWYPiq63pqWZQFEcLtab+a793GRCIMRkqWpuOd/wWh+1CaCz8+m3QNcAcnu8
         eLIhv50481G9iiqgpAsnfVKwV5/06wRJ6niI0la1NatK2/ReVobsQjfwouUwrollZ8Vh
         RZwVfhyBLgvTUh/KMMwCpEAaNkg7forGudThc+Fb776gYlVLZwNungtD6Yp/sY1m1LmL
         0tJA==
X-Gm-Message-State: AOJu0YwZIKJ39vp1j5qGtx1ItdsswihkFWu1WEM9+JfAjgooj82NN3u6
	X+bruyDNujcmAm7I1ZJGcWUbvjUSHqtGsATROTfSW0B+9UqfDgTc9oXiep4EkKJBQDGLFQ+++vt
	v
X-Google-Smtp-Source: AGHT+IHPcPdLXOxgviX0WjqxhQH7g/pIM3M7XqU6ugPtHAfWzob4fpoIF5FsyyX+Mazt8klqsRoXZg==
X-Received: by 2002:a19:6405:0:b0:512:acf1:6970 with SMTP id y5-20020a196405000000b00512acf16970mr5150205lfb.35.1709033140736;
        Tue, 27 Feb 2024 03:25:40 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v2 1/5] xen/livepatch: register livepatch regions when loaded
Date: Tue, 27 Feb 2024 12:25:24 +0100
Message-ID: <20240227112528.4540-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240227112528.4540-1-roger.pau@citrix.com>
References: <20240227112528.4540-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Currently livepatch regions are registered as virtual regions only after the
livepatch has been applied.

This can lead to issues when using the pre-apply or post-revert hooks, as at
that point the livepatch is not in the virtual regions list.  If a livepatch
pre-apply hook contains a WARN() it would trigger an hypervisor crash, as the
code to handle the bug frame won't be able to find the instruction pointer that
triggered the #UD in any of the registered virtual regions, and hence crash.

Fix this by adding the livepatch payloads as virtual regions as soon as loaded,
and only remove them once the payload is unloaded.  This requires some changes
to the virtual regions code, as the removal of the virtual regions is no longer
done in stop machine context, and hence an RCU barrier is added in order to
make sure there are no users of the virtual region after it's been removed from
the list.

Fixes: 8313c864fa95 ('livepatch: Implement pre-|post- apply|revert hooks')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Add the virtual region in livepatch_upload().
 - Make unregister_virtual_region() depend on CONFIG_LIVEPATCH.
---
 xen/common/livepatch.c      |  4 ++--
 xen/common/virtual_region.c | 42 ++++++++++++++-----------------------
 2 files changed, 18 insertions(+), 28 deletions(-)

diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index 2c4b84382798..d7f50e101858 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -1071,6 +1071,7 @@ static int build_symbol_table(struct payload *payload,
 static void free_payload(struct payload *data)
 {
     ASSERT(spin_is_locked(&payload_lock));
+    unregister_virtual_region(&data->region);
     list_del(&data->list);
     payload_cnt--;
     payload_version++;
@@ -1170,6 +1171,7 @@ static int livepatch_upload(struct xen_sysctl_livepatch_upload *upload)
         INIT_LIST_HEAD(&data->list);
         INIT_LIST_HEAD(&data->applied_list);
 
+        register_virtual_region(&data->region);
         list_add_tail(&data->list, &payload_list);
         payload_cnt++;
         payload_version++;
@@ -1386,7 +1388,6 @@ static inline void apply_payload_tail(struct payload *data)
      * The applied_list is iterated by the trap code.
      */
     list_add_tail_rcu(&data->applied_list, &applied_list);
-    register_virtual_region(&data->region);
 
     data->state = LIVEPATCH_STATE_APPLIED;
 }
@@ -1432,7 +1433,6 @@ static inline void revert_payload_tail(struct payload *data)
      * The applied_list is iterated by the trap code.
      */
     list_del_rcu(&data->applied_list);
-    unregister_virtual_region(&data->region);
 
     data->reverted = true;
     data->state = LIVEPATCH_STATE_CHECKED;
diff --git a/xen/common/virtual_region.c b/xen/common/virtual_region.c
index ddac5c9147e5..e3a4dc8540df 100644
--- a/xen/common/virtual_region.c
+++ b/xen/common/virtual_region.c
@@ -23,14 +23,8 @@ static struct virtual_region core_init __initdata = {
 };
 
 /*
- * RCU locking. Additions are done either at startup (when there is only
- * one CPU) or when all CPUs are running without IRQs.
- *
- * Deletions are bit tricky. We do it when Live Patch (all CPUs running
- * without IRQs) or during bootup (when clearing the init).
- *
- * Hence we use list_del_rcu (which sports an memory fence) and a spinlock
- * on deletion.
+ * RCU locking. Modifications to the list must be done in exclusive mode, and
+ * hence need to hold the spinlock.
  *
  * All readers of virtual_region_list MUST use list_for_each_entry_rcu.
  */
@@ -58,41 +52,36 @@ const struct virtual_region *find_text_region(unsigned long addr)
 
 void register_virtual_region(struct virtual_region *r)
 {
-    ASSERT(!local_irq_is_enabled());
+    unsigned long flags;
 
+    spin_lock_irqsave(&virtual_region_lock, flags);
     list_add_tail_rcu(&r->list, &virtual_region_list);
+    spin_unlock_irqrestore(&virtual_region_lock, flags);
 }
 
-static void remove_virtual_region(struct virtual_region *r)
+/*
+ * Suggest inline so when !CONFIG_LIVEPATCH the function is not left
+ * unreachable after init code is removed.
+ */
+static void inline remove_virtual_region(struct virtual_region *r)
 {
     unsigned long flags;
 
     spin_lock_irqsave(&virtual_region_lock, flags);
     list_del_rcu(&r->list);
     spin_unlock_irqrestore(&virtual_region_lock, flags);
-    /*
-     * We do not need to invoke call_rcu.
-     *
-     * This is due to the fact that on the deletion we have made sure
-     * to use spinlocks (to guard against somebody else calling
-     * unregister_virtual_region) and list_deletion spiced with
-     * memory barrier.
-     *
-     * That protects us from corrupting the list as the readers all
-     * use list_for_each_entry_rcu which is safe against concurrent
-     * deletions.
-     */
 }
 
+#ifdef CONFIG_LIVEPATCH
 void unregister_virtual_region(struct virtual_region *r)
 {
-    /* Expected to be called from Live Patch - which has IRQs disabled. */
-    ASSERT(!local_irq_is_enabled());
-
     remove_virtual_region(r);
+
+    /* Assert that no CPU might be using the removed region. */
+    rcu_barrier();
 }
 
-#if defined(CONFIG_LIVEPATCH) && defined(CONFIG_X86)
+#ifdef CONFIG_X86
 void relax_virtual_region_perms(void)
 {
     const struct virtual_region *region;
@@ -117,6 +106,7 @@ void tighten_virtual_region_perms(void)
     rcu_read_unlock(&rcu_virtual_region_lock);
 }
 #endif
+#endif
 
 void __init unregister_init_virtual_region(void)
 {
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:25:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:25:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686019.1067558 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revak-00083A-PQ; Tue, 27 Feb 2024 11:25:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686019.1067558; Tue, 27 Feb 2024 11:25:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revak-000833-Kk; Tue, 27 Feb 2024 11:25:46 +0000
Received: by outflank-mailman (input) for mailman id 686019;
 Tue, 27 Feb 2024 11:25:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1revaj-0007l8-Ih
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:25:45 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f21e9684-d562-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 12:25:44 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-512e39226efso5576425e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:25:44 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 on33-20020a05621444a100b0068fcd643b9dsm4026531qvb.22.2024.02.27.03.25.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:25:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f21e9684-d562-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709033143; x=1709637943; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=adt3YzhrFcYBWzR8wX43amlG22Ma23x7ZH5fofejSpQ=;
        b=AvxkxG2156hQiKaZqE94OZXt9Z/SOQkbKp05QHlbelojlxbS+IYbJDz6ORDOujv51G
         +Wxqwr1F08v2J50ydEsmk/Xx/Gq5BFqs668lUfAv0CtNU5hGZdCo8k00Vi2KGJkdSgct
         hNdy0MtI4nGAaKv0rjtCGiHiptkDSdSeI8EoM=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709033143; x=1709637943;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=adt3YzhrFcYBWzR8wX43amlG22Ma23x7ZH5fofejSpQ=;
        b=jE+SQ20eku5AEKdICf+lRtq0AZGWdNfGom8LEHt4CwKE2RTSstz7Kc0VAEHuHaIAdY
         UnuHt2RWKPkW7M3CSsQbSl4CFJP+Gl3GasHyO9h4fhsG9wb+ZZEiY+qpCOb1zNKMNfV6
         HE2xrwrXDHLOKy+B/59MkO7Xcrdvz3AbXhvwULpC9JjrcvYQBzGO+NvM4UrMdZfX1TIT
         GqV0Kat4wtG0cg89zNW0hI4AnrKqWQPbJf/Ti1wrZ3NICeBeJc5R8vW5yGTDH29nX2AG
         S+jYoy0wkkAUs8dlcL+shGLWksYLT23hOtG3j0WN8VLsQAgI4ehA2a7gVtVz8oijHtm0
         nDCQ==
X-Gm-Message-State: AOJu0Yw/r6V2W1q5ytZYg9Ji0kF5n0v5PjIaCDRFb36umnXaj/H1tbqy
	kPuvPjpU6v5kotacfCYzOYTlEEkxhS7sv1mporB47Et+SJYnXzxwlo/rGUVpdpJTZHYRCJQtIZc
	B
X-Google-Smtp-Source: AGHT+IFrcnfBqjubkVqz3lZabcZvN3Ua4viJGKpy6BgXTXfWXX5iv7XXj2buKAEP1g7vqVfsLqFjNQ==
X-Received: by 2002:ac2:5e85:0:b0:512:ed5f:3bf0 with SMTP id b5-20020ac25e85000000b00512ed5f3bf0mr5608169lfq.28.1709033143015;
        Tue, 27 Feb 2024 03:25:43 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>
Subject: [PATCH v2 2/5] xen/livepatch: search for symbols in all loaded payloads
Date: Tue, 27 Feb 2024 12:25:25 +0100
Message-ID: <20240227112528.4540-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240227112528.4540-1-roger.pau@citrix.com>
References: <20240227112528.4540-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When checking if an address belongs to a patch, or when resolving a symbol,
take into account all loaded livepatch payloads, even if not applied.

This is required in order for the pre-apply and post-revert hooks to work
properly, or else Xen won't detect the instruction pointer belonging to those
hooks as being part of the currently active text.

Move the RCU handling to be used for payload_list instead of applied_list, as
now the calls from trap code will iterate over the payload_list.

Fixes: 8313c864fa95 ('livepatch: Implement pre-|post- apply|revert hooks')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>
---
 xen/common/livepatch.c | 49 +++++++++++++++---------------------------
 1 file changed, 17 insertions(+), 32 deletions(-)

diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index d7f50e101858..14295bae8704 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -36,13 +36,14 @@
  * caller in schedule_work.
  */
 static DEFINE_SPINLOCK(payload_lock);
-static LIST_HEAD(payload_list);
-
 /*
- * Patches which have been applied. Need RCU in case we crash (and then
- * traps code would iterate via applied_list) when adding entries on the list.
+ * Need RCU in case we crash (and then traps code would iterate via
+ * payload_list) when adding entries on the list.
  */
-static DEFINE_RCU_READ_LOCK(rcu_applied_lock);
+static DEFINE_RCU_READ_LOCK(rcu_payload_lock);
+static LIST_HEAD(payload_list);
+
+/* Patches which have been applied. Only modified from stop machine context. */
 static LIST_HEAD(applied_list);
 
 static unsigned int payload_cnt;
@@ -111,12 +112,8 @@ bool is_patch(const void *ptr)
     const struct payload *data;
     bool r = false;
 
-    /*
-     * Only RCU locking since this list is only ever changed during apply
-     * or revert context. And in case it dies there we need an safe list.
-     */
-    rcu_read_lock(&rcu_applied_lock);
-    list_for_each_entry_rcu ( data, &applied_list, applied_list )
+    rcu_read_lock(&rcu_payload_lock);
+    list_for_each_entry_rcu ( data, &payload_list, list )
     {
         if ( (ptr >= data->rw_addr &&
               ptr < (data->rw_addr + data->rw_size)) ||
@@ -130,7 +127,7 @@ bool is_patch(const void *ptr)
         }
 
     }
-    rcu_read_unlock(&rcu_applied_lock);
+    rcu_read_unlock(&rcu_payload_lock);
 
     return r;
 }
@@ -166,12 +163,8 @@ static const char *cf_check livepatch_symbols_lookup(
     const void *va = (const void *)addr;
     const char *n = NULL;
 
-    /*
-     * Only RCU locking since this list is only ever changed during apply
-     * or revert context. And in case it dies there we need an safe list.
-     */
-    rcu_read_lock(&rcu_applied_lock);
-    list_for_each_entry_rcu ( data, &applied_list, applied_list )
+    rcu_read_lock(&rcu_payload_lock);
+    list_for_each_entry_rcu ( data, &payload_list, list )
     {
         if ( va < data->text_addr ||
              va >= (data->text_addr + data->text_size) )
@@ -200,7 +193,7 @@ static const char *cf_check livepatch_symbols_lookup(
         n = data->symtab[best].name;
         break;
     }
-    rcu_read_unlock(&rcu_applied_lock);
+    rcu_read_unlock(&rcu_payload_lock);
 
     return n;
 }
@@ -1072,7 +1065,8 @@ static void free_payload(struct payload *data)
 {
     ASSERT(spin_is_locked(&payload_lock));
     unregister_virtual_region(&data->region);
-    list_del(&data->list);
+    list_del_rcu(&data->list);
+    rcu_barrier();
     payload_cnt--;
     payload_version++;
     free_payload_data(data);
@@ -1172,7 +1166,7 @@ static int livepatch_upload(struct xen_sysctl_livepatch_upload *upload)
         INIT_LIST_HEAD(&data->applied_list);
 
         register_virtual_region(&data->region);
-        list_add_tail(&data->list, &payload_list);
+        list_add_tail_rcu(&data->list, &payload_list);
         payload_cnt++;
         payload_version++;
     }
@@ -1383,11 +1377,7 @@ static int apply_payload(struct payload *data)
 
 static inline void apply_payload_tail(struct payload *data)
 {
-    /*
-     * We need RCU variant (which has barriers) in case we crash here.
-     * The applied_list is iterated by the trap code.
-     */
-    list_add_tail_rcu(&data->applied_list, &applied_list);
+    list_add_tail(&data->applied_list, &applied_list);
 
     data->state = LIVEPATCH_STATE_APPLIED;
 }
@@ -1427,12 +1417,7 @@ static int revert_payload(struct payload *data)
 
 static inline void revert_payload_tail(struct payload *data)
 {
-
-    /*
-     * We need RCU variant (which has barriers) in case we crash here.
-     * The applied_list is iterated by the trap code.
-     */
-    list_del_rcu(&data->applied_list);
+    list_del(&data->applied_list);
 
     data->reverted = true;
     data->state = LIVEPATCH_STATE_CHECKED;
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:25:49 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:25:49 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686020.1067567 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revan-0008M8-4V; Tue, 27 Feb 2024 11:25:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686020.1067567; Tue, 27 Feb 2024 11:25:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revan-0008M1-1T; Tue, 27 Feb 2024 11:25:49 +0000
Received: by outflank-mailman (input) for mailman id 686020;
 Tue, 27 Feb 2024 11:25:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1reval-0007WG-7S
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:25:47 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f391e6f1-d562-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 12:25:46 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-512fd840142so2473743e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:25:46 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 qj24-20020a056214321800b0068f9fd1d688sm4018394qvb.21.2024.02.27.03.25.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:25:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f391e6f1-d562-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709033145; x=1709637945; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=gyNyY8wN07OAJqlH/Vju+7fTgYfs6X4zDGd5JPLgEbI=;
        b=D6XIvQCJrAglvQtlVm3VzruKoSa+wJGOY9iSuMSRfO75lRyZ9ATsdfD8YVibjFi3TQ
         neKlcbMLLmCUhoEgDIDYJyam/VmhBtwrbN5LbAemBjh5oVx4ys/etJdsDqVll09WeCtb
         2ZMpka2SHsQtnqVUg16L8PXLvag95N6Dobe90=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709033145; x=1709637945;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=gyNyY8wN07OAJqlH/Vju+7fTgYfs6X4zDGd5JPLgEbI=;
        b=qmP30Fc9FXXsCpCNhkXWf7JcrIEecUg9J4WHY3wHfgl7eoPfnoqZlTu9fMdJaX8XGM
         sZjd1ABpG3Yf/yDf/05mq4BJydppb+w6GzuMQeeuHQ5mKv9uA89eodcz081Iyw1/2vgl
         Vc2iqqkYT4ONY3K/EVt5Fw0qg05rkS/PEVshaYzKmA2oD///6bNE0i1cNBj/Jr0FWHNW
         46E02tXPZ2TySLGhsF6rJb5sKPvyWkS1ffGzGivluO0nU8kyohYAB5CwdILWTreTm9ZU
         J/h/HiG6Ml0UJUJkOxbPyAZXOIkAQ+ELWQtVS6UEGZpTQvOWBlliBfONGTqd2jU3HoFp
         5llg==
X-Gm-Message-State: AOJu0Yxa9j5ebesyGK3dLJKvmkUOtbWn08c/kIN0VpxbUv7geL6h70Uo
	/ZguVoqVXeQ3nTwJi6TQKc6XlZT/0zs6p8D/VuqBQpJl7UdXNhBGYHDoHG3BnJZk82mrNOabaT3
	H
X-Google-Smtp-Source: AGHT+IFK2BifGhYTEE4PkQH9bmj7jJsbCai+7Q5HGiCe5i5wxZUV6bppAL1KpT0Cm+eNWRY2nGA/Sg==
X-Received: by 2002:ac2:4a9e:0:b0:512:fb31:1ad1 with SMTP id l30-20020ac24a9e000000b00512fb311ad1mr3967371lfp.57.1709033145436;
        Tue, 27 Feb 2024 03:25:45 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>
Subject: [PATCH v2 3/5] xen/livepatch: fix norevert test attempt to open-code revert
Date: Tue, 27 Feb 2024 12:25:26 +0100
Message-ID: <20240227112528.4540-4-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240227112528.4540-1-roger.pau@citrix.com>
References: <20240227112528.4540-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The purpose of the norevert test is to install a dummy handler that replaces
the internal Xen revert code, and then perform the revert in the post-revert
hook.  For that purpose the usage of the previous common_livepatch_revert() is
not enough, as that just reverts specific functions, but not the whole state of
the payload.

Remove both common_livepatch_{apply,revert}() and instead expose
revert_payload{,_tail}() in order to perform the patch revert from the
post-revert hook.

Fixes: 6047104c3ccc ('livepatch: Add per-function applied/reverted state tracking marker')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - Check return value of revert_payload().
---
 xen/common/livepatch.c                        | 41 +++++++++++++++++--
 xen/include/xen/livepatch.h                   | 32 ++-------------
 .../livepatch/xen_action_hooks_norevert.c     | 22 +++-------
 3 files changed, 46 insertions(+), 49 deletions(-)

diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index 14295bae8704..5a7d5b7be0ad 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -1366,7 +1366,22 @@ static int apply_payload(struct payload *data)
     ASSERT(!local_irq_is_enabled());
 
     for ( i = 0; i < data->nfuncs; i++ )
-        common_livepatch_apply(&data->funcs[i], &data->fstate[i]);
+    {
+        const struct livepatch_func *func = &data->funcs[i];
+        struct livepatch_fstate *state = &data->fstate[i];
+
+        /* If the action has been already executed on this function, do nothing. */
+        if ( state->applied == LIVEPATCH_FUNC_APPLIED )
+        {
+            printk(XENLOG_WARNING LIVEPATCH
+                   "%s: %s has been already applied before\n",
+                   __func__, func->name);
+            continue;
+        }
+
+        arch_livepatch_apply(func, state);
+        state->applied = LIVEPATCH_FUNC_APPLIED;
+    }
 
     arch_livepatch_revive();
 
@@ -1382,7 +1397,7 @@ static inline void apply_payload_tail(struct payload *data)
     data->state = LIVEPATCH_STATE_APPLIED;
 }
 
-static int revert_payload(struct payload *data)
+int revert_payload(struct payload *data)
 {
     unsigned int i;
     int rc;
@@ -1397,7 +1412,25 @@ static int revert_payload(struct payload *data)
     }
 
     for ( i = 0; i < data->nfuncs; i++ )
-        common_livepatch_revert(&data->funcs[i], &data->fstate[i]);
+    {
+        const struct livepatch_func *func = &data->funcs[i];
+        struct livepatch_fstate *state = &data->fstate[i];
+
+        /*
+         * If the apply action hasn't been executed on this function, do
+         * nothing.
+         */
+        if ( !func->old_addr || state->applied == LIVEPATCH_FUNC_NOT_APPLIED )
+        {
+            printk(XENLOG_WARNING LIVEPATCH
+                   "%s: %s has not been applied before\n",
+                   __func__, func->name);
+            continue;
+        }
+
+        arch_livepatch_revert(func, state);
+        state->applied = LIVEPATCH_FUNC_NOT_APPLIED;
+    }
 
     /*
      * Since we are running with IRQs disabled and the hooks may call common
@@ -1415,7 +1448,7 @@ static int revert_payload(struct payload *data)
     return 0;
 }
 
-static inline void revert_payload_tail(struct payload *data)
+void revert_payload_tail(struct payload *data)
 {
     list_del(&data->applied_list);
 
diff --git a/xen/include/xen/livepatch.h b/xen/include/xen/livepatch.h
index ad0eae28bd0d..d074a5bebecc 100644
--- a/xen/include/xen/livepatch.h
+++ b/xen/include/xen/livepatch.h
@@ -138,35 +138,11 @@ void arch_livepatch_post_action(void);
 void arch_livepatch_mask(void);
 void arch_livepatch_unmask(void);
 
-static inline void common_livepatch_apply(const struct livepatch_func *func,
-                                          struct livepatch_fstate *state)
-{
-    /* If the action has been already executed on this function, do nothing. */
-    if ( state->applied == LIVEPATCH_FUNC_APPLIED )
-    {
-        printk(XENLOG_WARNING LIVEPATCH "%s: %s has been already applied before\n",
-                __func__, func->name);
-        return;
-    }
-
-    arch_livepatch_apply(func, state);
-    state->applied = LIVEPATCH_FUNC_APPLIED;
-}
+/* Only for testing purposes. */
+struct payload;
+int revert_payload(struct payload *data);
+void revert_payload_tail(struct payload *data);
 
-static inline void common_livepatch_revert(const struct livepatch_func *func,
-                                           struct livepatch_fstate *state)
-{
-    /* If the apply action hasn't been executed on this function, do nothing. */
-    if ( !func->old_addr || state->applied == LIVEPATCH_FUNC_NOT_APPLIED )
-    {
-        printk(XENLOG_WARNING LIVEPATCH "%s: %s has not been applied before\n",
-                __func__, func->name);
-        return;
-    }
-
-    arch_livepatch_revert(func, state);
-    state->applied = LIVEPATCH_FUNC_NOT_APPLIED;
-}
 #else
 
 /*
diff --git a/xen/test/livepatch/xen_action_hooks_norevert.c b/xen/test/livepatch/xen_action_hooks_norevert.c
index c17385519263..c5fbab174680 100644
--- a/xen/test/livepatch/xen_action_hooks_norevert.c
+++ b/xen/test/livepatch/xen_action_hooks_norevert.c
@@ -96,26 +96,14 @@ static int revert_hook(livepatch_payload_t *payload)
 
 static void post_revert_hook(livepatch_payload_t *payload)
 {
-    int i;
+    unsigned long flags;
 
     printk(KERN_DEBUG "%s: Hook starting.\n", __func__);
 
-    for (i = 0; i < payload->nfuncs; i++)
-    {
-        const struct livepatch_func *func = &payload->funcs[i];
-        struct livepatch_fstate *fstate = &payload->fstate[i];
-
-        BUG_ON(revert_cnt != 1);
-        BUG_ON(fstate->applied != LIVEPATCH_FUNC_APPLIED);
-
-        /* Outside of quiesce zone: MAY TRIGGER HOST CRASH/UNDEFINED BEHAVIOR */
-        arch_livepatch_quiesce();
-        common_livepatch_revert(payload);
-        arch_livepatch_revive();
-        BUG_ON(fstate->applied == LIVEPATCH_FUNC_APPLIED);
-
-        printk(KERN_DEBUG "%s: post reverted: %s\n", __func__, func->name);
-    }
+    local_irq_save(flags);
+    BUG_ON(revert_payload(payload));
+    revert_payload_tail(payload);
+    local_irq_restore(flags);
 
     printk(KERN_DEBUG "%s: Hook done.\n", __func__);
 }
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:25:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:25:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686021.1067577 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revap-0000DS-Eq; Tue, 27 Feb 2024 11:25:51 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686021.1067577; Tue, 27 Feb 2024 11:25:51 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revap-0000DJ-Ao; Tue, 27 Feb 2024 11:25:51 +0000
Received: by outflank-mailman (input) for mailman id 686021;
 Tue, 27 Feb 2024 11:25:50 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1revao-0007l8-Ag
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:25:50 +0000
Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com
 [2a00:1450:4864:20::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f4f9fad6-d562-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 12:25:48 +0100 (CET)
Received: by mail-lf1-x133.google.com with SMTP id
 2adb3069b0e04-512ab55fde6so5392930e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:25:48 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 w25-20020ae9e519000000b00787a7e9c3e8sm3458446qkf.56.2024.02.27.03.25.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:25:47 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f4f9fad6-d562-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709033148; x=1709637948; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=dfnDRWSsRLdZRck0ImeXNurgPdtQnzpZXT2sgnYgWu4=;
        b=uM1hH/OJnt121jcozicLyDoip+o+9jCeI9n6TkYjqbT+Xdqotq6z5nfHAfpa83iEvn
         o2SG3u9zI5cPlzxuY7PcFyERPZZLJDt2A81P8maKG6RHAhKxIEuItn3Yxf+oJl0QdBvl
         3ILjgD3tKiErlSe58n1dRTIJ8UowV3DGpVe/g=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709033148; x=1709637948;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=dfnDRWSsRLdZRck0ImeXNurgPdtQnzpZXT2sgnYgWu4=;
        b=oPmEf4634BNmUaRj8S/9xcZ9C4OnfSgzAzH7HETiBWsVfg8J8PAMAGA+0Ji1ANClVY
         muiqdNTI3xU16rqynxwzKZlxJGKEebCnJOYi6h3dCO9Fyy9dUtHggO4SqwEqlrVGGvTz
         lrnjvpQgCzmSf8xJfszFtzN4CJRwFsPQcGgyM8D0k39DLyhFrljwmH88Oa5t9dhAkO3u
         9Gd1wKlZ6Np8x40d2hXBlpGqqgHmhjTCl/y0FRKD+hGIfauDQ4pt+Ka/RR63btUkbjOg
         yQ58HqvrrpPHmKyJh+691FhZDjnElSg876LoUSZsbQOKZ1oSdQ5b78J3rSby8s5jGngC
         yUuw==
X-Gm-Message-State: AOJu0YzdfccXPmRDkURVR0p4Lq23HTZ8HyoIVJIG2PdFfHlnh+e7scjP
	VF+OXHPFDUZQhIaZRrCMzcNZTl8BH0dEHI2mtNZLdZGqsp/n+uPupAXnqZ07VOIE6T1Nnhf5YWu
	o
X-Google-Smtp-Source: AGHT+IFeyZ15xptCBWdoSu4rws/k7G4zdbWzKZJdYq04/EcuMOsIn9zH2PJQuPkeumcjTGJS1tHSOg==
X-Received: by 2002:a05:6512:3da9:b0:511:5f38:76e with SMTP id k41-20020a0565123da900b005115f38076emr7669601lfv.1.1709033147736;
        Tue, 27 Feb 2024 03:25:47 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>
Subject: [PATCH v2 4/5] xen/livepatch: properly build the noapply and norevert tests
Date: Tue, 27 Feb 2024 12:25:27 +0100
Message-ID: <20240227112528.4540-5-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240227112528.4540-1-roger.pau@citrix.com>
References: <20240227112528.4540-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

It seems the build variables for those tests where copy-pasted from
xen_action_hooks_marker-objs and not adjusted to use the correct source files.

Fixes: 6047104c3ccc ('livepatch: Add per-function applied/reverted state tracking marker')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>
---
 xen/test/livepatch/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/test/livepatch/Makefile b/xen/test/livepatch/Makefile
index c258ab0b5940..d987a8367f15 100644
--- a/xen/test/livepatch/Makefile
+++ b/xen/test/livepatch/Makefile
@@ -118,12 +118,12 @@ xen_action_hooks_marker-objs := xen_action_hooks_marker.o xen_hello_world_func.o
 $(obj)/xen_action_hooks_noapply.o: $(obj)/config.h
 
 extra-y += xen_action_hooks_noapply.livepatch
-xen_action_hooks_noapply-objs := xen_action_hooks_marker.o xen_hello_world_func.o note.o xen_note.o
+xen_action_hooks_noapply-objs := xen_action_hooks_noapply.o xen_hello_world_func.o note.o xen_note.o
 
 $(obj)/xen_action_hooks_norevert.o: $(obj)/config.h
 
 extra-y += xen_action_hooks_norevert.livepatch
-xen_action_hooks_norevert-objs := xen_action_hooks_marker.o xen_hello_world_func.o note.o xen_note.o
+xen_action_hooks_norevert-objs := xen_action_hooks_norevert.o xen_hello_world_func.o note.o xen_note.o
 
 EXPECT_BYTES_COUNT := 8
 CODE_GET_EXPECT=$(shell $(OBJDUMP) -d --insn-width=1 $(1) | sed -n -e '/<'$(2)'>:$$/,/^$$/ p' | tail -n +2 | head -n $(EXPECT_BYTES_COUNT) | awk '{$$0=$$2; printf "%s", substr($$0,length-1)}' | sed 's/.\{2\}/0x&,/g' | sed 's/^/{/;s/,$$/}/g')
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:25:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:25:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686022.1067586 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revaq-0000UZ-Oj; Tue, 27 Feb 2024 11:25:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686022.1067586; Tue, 27 Feb 2024 11:25:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1revaq-0000Tk-KK; Tue, 27 Feb 2024 11:25:52 +0000
Received: by outflank-mailman (input) for mailman id 686022;
 Tue, 27 Feb 2024 11:25:51 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1revap-0007WG-Kx
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:25:51 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f63b87b1-d562-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 12:25:51 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-512be9194b7so4262243e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 03:25:51 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 jv2-20020a05622aa08200b0042e2002ed12sm3470912qtb.57.2024.02.27.03.25.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 03:25:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f63b87b1-d562-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709033150; x=1709637950; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=xmVYxznNchZhLSEAgmjblx5ppkOHBquqKgG3mrrDbSE=;
        b=dZI/PR5/KCLcfnBdWWtfQzqiaSa8azfG2+hkv8d4pMAggF+oqPbrdsyGO5rOkK80xY
         xeJlNBGvO5ij2M8yGlhi2jWOTlsK3CVjZygfDw+psbGzC7tpowAdoeKRk2y5WcW6uNJ3
         YUo/akddZIqh6vcifk4q9djZPXZff3mKDcbIY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709033150; x=1709637950;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=xmVYxznNchZhLSEAgmjblx5ppkOHBquqKgG3mrrDbSE=;
        b=gkB8amPkAedFVIiGKRVYzPFgESYmYUhSfnqbPjdGiebPxFaRUN+EObVkcpv7XZTLig
         hKaMwDcNGEioEd6vAU25IpbzC9e5CPyxwTvsOD17YC36Q/5+K264zjhyKmPngCNkkXkX
         gZGbuSapTU25Ke7THDfbvBkF+esq5lY/ivsdHJi2We0yw7AZ1MUj30ulJnB3nlSFm/6J
         08vLaMWOYAnqIWBc7SMU7CVdumcTs3k1JxyDMwMVBx1z6wJ01NclZfCCpFn1WAft5zXL
         ZXbarjwJNj2DEH3fMGRsMc3al8+7NyBUnzIol72MRkdH6lDkT6b3WuyW6VPKVibebrhK
         VKzw==
X-Gm-Message-State: AOJu0YyxALu1GPR35EdH3dNNrCKvdFBHNmi714DPdYp6d7Knk9cF+q4X
	jWez9KlfP5Yc+28wOgMPNBrjUAn2m5xlw1IYT5CvrnpNMVabhLWfs9S9h6xJlnTJRqXS0WWWM/4
	e
X-Google-Smtp-Source: AGHT+IFqfmLFdna5wALovZCB6WfEyZOIy2tNf0bS6nbC4SyizxjX2j1beaVKH42Pb2NkuS8nqDeZMA==
X-Received: by 2002:a05:6512:6d0:b0:512:fd8a:d0e1 with SMTP id u16-20020a05651206d000b00512fd8ad0e1mr4295594lff.45.1709033150026;
        Tue, 27 Feb 2024 03:25:50 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>
Subject: [PATCH v2 5/5] xen/livepatch: group and document payload hooks
Date: Tue, 27 Feb 2024 12:25:28 +0100
Message-ID: <20240227112528.4540-6-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240227112528.4540-1-roger.pau@citrix.com>
References: <20240227112528.4540-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Group the payload hooks between the pre/post handlers, and the apply/revert
replacements.  Also attempt to comment the context in which the hooks are
executed.

No functional change.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - New in this version.
---
 xen/include/xen/livepatch_payload.h | 37 ++++++++++++++++++++++-------
 1 file changed, 29 insertions(+), 8 deletions(-)

diff --git a/xen/include/xen/livepatch_payload.h b/xen/include/xen/livepatch_payload.h
index b9cd4f209670..472d6a4a63c1 100644
--- a/xen/include/xen/livepatch_payload.h
+++ b/xen/include/xen/livepatch_payload.h
@@ -82,6 +82,8 @@ struct payload {
  * collision.  Since multiple hooks can be registered, the
  * .livepatch.hook.load section is a table of functions that will be
  * executed in series by the livepatch infrastructure at patch load time.
+ *
+ * Note the load hook is executed in quiesced context.
  */
 #define LIVEPATCH_LOAD_HOOK(_fn) \
     livepatch_loadcall_t *__weak \
@@ -96,14 +98,20 @@ struct payload {
      livepatch_unloadcall_t *__weak \
         const livepatch_unload_data_##_fn __section(".livepatch.hooks.unload") = _fn;
 
+/*
+ * Pre/Post action hooks.
+ *
+ * This hooks are executed before or after the livepatch application. Pre hooks
+ * can veto the application/revert of the livepatch.  They are not executed in
+ * quiesced context.  All of pre and post hooks are considered vetoing, and
+ * hence filling any of those will block the usage of the REPLACE action.
+ *
+ * Each of the hooks below can only be set once per livepatch payload.
+ */
 #define LIVEPATCH_PREAPPLY_HOOK(_fn) \
     livepatch_precall_t *__attribute__((weak, used)) \
         const livepatch_preapply_data_##_fn __section(".livepatch.hooks.preapply") = _fn;
 
-#define LIVEPATCH_APPLY_HOOK(_fn) \
-    livepatch_actioncall_t *__attribute__((weak, used)) \
-        const livepatch_apply_data_##_fn __section(".livepatch.hooks.apply") = _fn;
-
 #define LIVEPATCH_POSTAPPLY_HOOK(_fn) \
     livepatch_postcall_t *__attribute__((weak, used)) \
         const livepatch_postapply_data_##_fn __section(".livepatch.hooks.postapply") = _fn;
@@ -112,14 +120,27 @@ struct payload {
     livepatch_precall_t *__attribute__((weak, used)) \
         const livepatch_prerevert_data_##_fn __section(".livepatch.hooks.prerevert") = _fn;
 
-#define LIVEPATCH_REVERT_HOOK(_fn) \
-    livepatch_actioncall_t *__attribute__((weak, used)) \
-        const livepatch_revert_data_##_fn __section(".livepatch.hooks.revert") = _fn;
-
 #define LIVEPATCH_POSTREVERT_HOOK(_fn) \
     livepatch_postcall_t *__attribute__((weak, used)) \
         const livepatch_postrevert_data_##_fn __section(".livepatch.hooks.postrevert") = _fn;
 
+/*
+ * Action replacement hooks.
+ *
+ * The following hooks replace the hypervisor implementation for the livepatch
+ * application and revert routines.  When filling the hooks below the native
+ * apply and revert routines will not be executed, so the provided hooks need
+ * to make sure the state of the payload after apply or revert is as expected
+ * by the livepatch logic.
+ */
+#define LIVEPATCH_APPLY_HOOK(_fn) \
+    livepatch_actioncall_t *__attribute__((weak, used)) \
+        const livepatch_apply_data_##_fn __section(".livepatch.hooks.apply") = _fn;
+
+#define LIVEPATCH_REVERT_HOOK(_fn) \
+    livepatch_actioncall_t *__attribute__((weak, used)) \
+        const livepatch_revert_data_##_fn __section(".livepatch.hooks.revert") = _fn;
+
 #endif /* __XEN_LIVEPATCH_PAYLOAD_H__ */
 
 /*
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 11:53:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 11:53:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686041.1067598 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rew11-0008J2-V0; Tue, 27 Feb 2024 11:52:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686041.1067598; Tue, 27 Feb 2024 11:52:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rew11-0008Iv-Qt; Tue, 27 Feb 2024 11:52:55 +0000
Received: by outflank-mailman (input) for mailman id 686041;
 Tue, 27 Feb 2024 11:52:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rew10-0008In-KQ
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 11:52:54 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rew0x-0007F8-A2; Tue, 27 Feb 2024 11:52:51 +0000
Received: from [15.248.2.14] (helo=[10.45.19.66])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rew0x-0003ZW-1H; Tue, 27 Feb 2024 11:52:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=kmabYmYV7w/DwdNxModdPKfiurpHr2ETQgn0Vl9O2rc=; b=Wj+TyPM/rwmzSKS191oBA9wFUG
	NHmZKnQJAUHMuTmsKiY+Lsr0cxrPh0aM3KX02kagaorxhMSOoauW0CR7wV5z1J3tGF8eX3xw0v30Z
	DBKDOoE1L/hqwkS8/kDfJzxEZdhspQg0fOJSimdU2rotOjy0jhl8jOvPY5QOeu8efvO8=;
Message-ID: <7a8e610e-913e-4a56-8ce1-6dd6abd894f4@xen.org>
Date: Tue, 27 Feb 2024 11:52:49 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: Nicola Vetrini <nicola.vetrini@bugseng.com>, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
 <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
 <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
 <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 27/02/2024 07:28, Jan Beulich wrote:
> On 27.02.2024 01:26, Stefano Stabellini wrote:
>> On Mon, 26 Feb 2024, Jan Beulich wrote:
>>> On 23.02.2024 10:35, Nicola Vetrini wrote:
>>>> Refactor cpu_notifier_call_chain into two functions:
>>>> - the variant that is allowed to fail loses the nofail flag
>>>> - the variant that shouldn't fail is encapsulated in a call
>>>>    to the failing variant, with an additional check.
>>>>
>>>> This prevents uses of the function that are not supposed to
>>>> fail from ignoring the return value, thus violating Rule 17.7:
>>>> "The value returned by a function having non-void return type shall
>>>> be used".
>>>>
>>>> No functional change.
>>>>
>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>>
>>> I'm afraid I disagree with this kind of bifurcation. No matter what
>>> Misra thinks or says, it is normal for return values of functions to
>>> not always be relevant to check.
>>
>> Hi Jan, I disagree.
>>
>> Regardless of MISRA, I really think return values need to be checked.
>> Moreover, we decided as a group to honor MISRA Rule 17.7, which requires
>> return values to be checked. This patch is a good step forward.
> 
> Yet splitting functions isn't the only way to deal with Misra's
> requirements, I suppose. After all there are functions where the
> return value is purely courtesy for perhaps just one of its callers.

You are right that we have some places where one caller care about the 
return value. But the problem is how do you tell whether the return was 
ignored on purpose or not?

We had at least one XSA because the return value of a function was not 
checked (see XSA-222). We also had plenty of smaller patches to check 
returns.

So far, we added __must_check when we believed return values should be 
checked. But usually at the point we notice, this is far too late.

To me the goal should be that we enforce __must_check everywhere. We are 
probably going to detect places where we forgot to check the return. For 
thoses that are on purpose, we can document them.

> 
> Splitting simply doesn't scale very well, imo.

Do you have another proposal? As Stefano said, we adopted the rule 17.7. 
So we know need a solution to address it.

> 
>>> To deal with the Misra rule imo requires to first have an abstract
>>> plan of how to handle such globally in the code base. Imo such a plan
>>> can't be to introduce perhaps dozens of new wrapper functions like is
>>> done here.
>>
>> This patch is following the right pattern, one we already follow with
>> the _locked suffix.
> 
> Right, and - just to mention it - one which I similarly dislike, albeit
> to a lesser degree.

AFAIU, we are debating between having a boolean indicating if a function 
doesn't fail or adding a wrapper.

While I understand this requires to add more code, the advantage of the 
suffix is this is more obvious for the reader (including the reviewer) 
that the call is not supposed to fail. I agree this is a matter of taste 
here...

In this case, what would be your suggestion to address the problem?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 12:01:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 12:01:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686055.1067632 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rew9N-0003b9-77; Tue, 27 Feb 2024 12:01:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686055.1067632; Tue, 27 Feb 2024 12:01:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rew9M-0003Zh-VD; Tue, 27 Feb 2024 12:01:32 +0000
Received: by outflank-mailman (input) for mailman id 686055;
 Tue, 27 Feb 2024 12:01:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rTLY=KE=xenbits.xen.org=andrewcoop@srs-se1.protection.inumbo.net>)
 id 1rew9L-0003B3-KJ
 for xen-devel@lists.xen.org; Tue, 27 Feb 2024 12:01:31 +0000
Received: from mail.xenproject.org (mail.xenproject.org [104.130.215.37])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ed50bc92-d567-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 13:01:25 +0100 (CET)
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <andrewcoop@xenbits.xen.org>)
 id 1rew95-0007T9-5d; Tue, 27 Feb 2024 12:01:15 +0000
Received: from andrewcoop by xenbits.xenproject.org with local (Exim 4.92)
 (envelope-from <andrewcoop@xenbits.xen.org>)
 id 1rew95-0004EA-2k; Tue, 27 Feb 2024 12:01:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ed50bc92-d567-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Date:Message-Id:Subject:CC:From:To:MIME-Version:
	Content-Transfer-Encoding:Content-Type;
	bh=RZUb5Iyay6kEEsHWSiJR9Q+yyaDlMBNXCpg1C7pZ3bU=; b=Fif1T0SQd1x4NwmKLzabvl/RBT
	CYqnPeqEwPLvwXtOYU7W3xtJO/Hwm2C5fNtguL+71Sa7ttne0LoWFgxpW4x4qKuspm8DPHuRwsO4Q
	vqMzZ5oy8+Ok/ajBUu00dze7fMAAKfLZbNzKgQ1AtziZ0Sd7967oXeGcCSRXtkFNcOoU=;
Content-Type: multipart/mixed; boundary="=separator"; charset="utf-8"
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.509 (Entity 5.509)
To: xen-announce@lists.xen.org, xen-devel@lists.xen.org,
 xen-users@lists.xen.org, oss-security@lists.openwall.com
From: Xen.org security team <security@xen.org>
CC: Xen.org security team <security-team-members@xen.org>
Subject: Xen Security Advisory 451 v2 (CVE-2023-46841) - x86: shadow stack
 vs exceptions from emulation stubs
Message-Id: <E1rew95-0004EA-2k@xenbits.xenproject.org>
Date: Tue, 27 Feb 2024 12:01:15 +0000

--=separator
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

            Xen Security Advisory CVE-2023-46841 / XSA-451
                               version 2

         x86: shadow stack vs exceptions from emulation stubs

UPDATES IN VERSION 2
====================

Largely cosmetic adjustment in patches.

Public release.

ISSUE DESCRIPTION
=================

Recent x86 CPUs offer functionality named Control-flow Enforcement
Technology (CET).  A sub-feature of this are Shadow Stacks (CET-SS).
CET-SS is a hardware feature designed to protect against Return Oriented
Programming attacks. When enabled, traditional stacks holding both data
and return addresses are accompanied by so called "shadow stacks",
holding little more than return addresses.  Shadow stacks aren't
writable by normal instructions, and upon function returns their
contents are used to check for possible manipulation of a return address
coming from the traditional stack.

In particular certain memory accesses need intercepting by Xen.  In
various cases the necessary emulation involves kind of replaying of
the instruction.  Such replaying typically involves filling and then
invoking of a stub.  Such a replayed instruction may raise an
exceptions, which is expected and dealt with accordingly.

Unfortunately the interaction of both of the above wasn't right:
Recovery involves removal of a call frame from the (traditional) stack.
The counterpart of this operation for the shadow stack was missing.

IMPACT
======

An unprivileged guest can cause a hypervisor crash, causing a Denial of
Service (DoS) of the entire host.

VULNERABLE SYSTEMS
==================

Xen 4.14 and onwards are vulnerable.  Xen 4.13 and older are not
vulnerable.

Only x86 systems with CET-SS enabled are vulnerable.  x86 systems with
CET-SS unavailable or disabled are not vulnerable.  Arm systems are not
vulnerable.  See
https://xenbits.xen.org/docs/latest/faq.html#tell-if-cet-is-active
for how to determine whether CET-SS is active.

Only HVM or PVH guests can leverage the vulnerability.  PV guests cannot
leverage the vulnerability.

MITIGATION
==========

While in principle it is possible to disable use of CET on capable
systems using the "cet=no-shstk" command line option, doing so disables
an important security feature and may therefore not be advisable.

CREDITS
=======

This issue was discovered by Jan Beulich of SUSE.

RESOLUTION
==========

Applying the appropriate (set of) attached patch(es) resolves this issue.

Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball.  Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.

xsa451-?.patch         xen-unstable
xsa451-4.18.patch      Xen 4.18.x
xsa451-4.17.patch      Xen 4.17.x
xsa451-4.16.patch      Xen 4.16.x
xsa451-4.15.patch      Xen 4.15.x

$ sha256sum xsa451*
446178a9a37646e62622988efffa3d1ffa0b579fc089ab79138507acfd3440c0  xsa451-1.patch
614ab6925ea60f36212f0cd01929f3a97161de1828040770792e146c170bfea2  xsa451-2.patch
ad529273d7dc97bff239f1727a9702eb24d41b723d2a3077a1fecc4684900f91  xsa451-3.patch
2c68480657220cfab92fe9821ce201ff7c9e0b541619a1add541f3d66fa13e9d  xsa451-4.15.patch
fa8ab72e61fae0130fb81b0a7ce508fdb3bcb3c800b0ab7684aa6595cbad88ea  xsa451-4.16.patch
e41cab6471586a5f50e10eb26895fec624cc6d8fd3b4ff71495466df8aaa19e5  xsa451-4.17.patch
d6b76a8db6c80c0684fc94becc2e23091c8f1dcbebc726438dbb1a6cde543335  xsa451-4.18.patch
$

DEPLOYMENT DURING EMBARGO
=========================

Deployment of the patches and/or mitigations described above (or
others which are substantially similar) is permitted during the
embargo, even on public-facing systems with untrusted guest users and
administrators.

But: Distribution of updated software is prohibited (except to other
members of the predisclosure list).

Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.

(Note: this during-embargo deployment notice is retained in
post-embargo publicly released Xen Project advisories, even though it
is then no longer applicable.  This is to enable the community to have
oversight of the Xen Project Security Team's decisionmaking.)

For more information about permissible uses of embargoed information,
consult the Xen Project community's agreed Security Policy:
  http://www.xenproject.org/security-policy.html
-----BEGIN PGP SIGNATURE-----

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmXdu4UMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZApoIAMmKsIAqbt/QlUZFUXYx+DAW20Bl7DUGjJlFv6kx
pBDSxW3a2evYo+CTTapVeRfosI+/kI61pcyFd19EGdthVcgufPQOC7yVxmu8j7Wi
s6lb/h0b6vKFOUubKN+EtaVRR34acqmQwSq668AjcyL8M5xIdWfYDpKHVft29x8i
QwKdKnvsWwaFrUathVTlspqcHLkNWf7+nsTVapMG2O15UrqYdJPErhL/Bh+iwSih
exc/fRFyQuqFL7qHnvPXz+AhajjHmDO+1Z3OCir9MleyZ3JJvIq6Vnje75+DFHeT
n9kFt29LJMvRzlDzIdfUy9R98h0r3WIQBaicFO2pBKlp6i8=
=JJb5
-----END PGP SIGNATURE-----

--=separator
Content-Type: application/octet-stream; name="xsa451-1.patch"
Content-Disposition: attachment; filename="xsa451-1.patch"
Content-Transfer-Encoding: base64

RnJvbTogQW5kcmV3IENvb3BlciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNv
bT4KU3ViamVjdDogeDg2OiBkb2N1bWVudCBob3cgc3R1YiBleGNlcHRpb24g
cmVjb3Zlcnkgd29ya3MKCkRlc2NyaWJlIGhvdyBpdCBpcyBtZWFudCB0byB3
b3JrLCBldmVuIGlmIG9uZSBhc3BlY3Qgb2YgaXQgd2lsbCBvbmx5IGJlCnRh
a2VuIGNhcmUgb2Ygc3Vic2VxdWVudGx5LgoKU2lnbmVkLW9mZi1ieTogQW5k
cmV3IENvb3BlciA8YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbT4KUmV2aWV3
ZWQtYnk6IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNvbT4KCi0tLSBh
L3hlbi9hcmNoL3g4Ni9leHRhYmxlLmMKKysrIGIveGVuL2FyY2gveDg2L2V4
dGFibGUuYwpAQCAtOTQsNiArOTQsMjIgQEAgc2VhcmNoX2V4Y2VwdGlvbl90
YWJsZShjb25zdCBzdHJ1Y3QgY3B1XwogICAgIGlmICggcmVnaW9uICYmIHJl
Z2lvbi0+ZXggKQogICAgICAgICByZXR1cm4gc2VhcmNoX29uZV9leHRhYmxl
KHJlZ2lvbi0+ZXgsIHJlZ2lvbi0+ZXhfZW5kLCByZWdzLT5yaXApOwogCisg
ICAgLyoKKyAgICAgKiBFbXVsYXRpb24gc3R1YnMgKHdoaWNoIGFyZSBwZXIt
Q1BVKSBhcmUgY29uc3RydWN0ZWQgd2l0aCBhIFJFVCBhdCB0aGUKKyAgICAg
KiBlbmQsIGFuZCBhcmUgQ0FMTGVkIGJ5IHRoZSBpbnZva2luZyBjb2RlLgor
ICAgICAqCisgICAgICogQW4gZXhjZXB0aW9uIGluIHRoZSBzdHVicyBtYXkg
b2NjdXIgYW55d2hlcmUsIHNvIHdlIGZpcnN0IG1hdGNoIGFueQorICAgICAq
ICVyaXAgaW4gdGhlIGNvcnJlY3Qgc3R1Yiwgd2l0aCBhIHNhbml0eSBjaGVj
ayBvbiAlcnNwIHRvby4gIEJ1dCwgYW4KKyAgICAgKiBlbnRyeSBpbiBleF90
YWJsZVtdIG5lZWRzIHRvIGJlIGNvbXBpbGUtdGltZSBjb25zdGFudCwgc28g
d2UgcmVnaXN0ZXIKKyAgICAgKiB0aGUgZml4dXAgYWRkcmVzcyB1c2luZyB0
aGUgaW52b2tpbmcgQ0FMTCdzIHJldHVybiBhZGRyZXNzLgorICAgICAqCisg
ICAgICogVG8gcmVjb3Zlciwgd2U6CisgICAgICogMSkgRW11bGF0ZSBhIHBz
ZXVkby1SRVQgdG8gZ2V0IG91dCBvZiB0aGUgc3R1Yi4gIFdlIFBPUCB0aGUg
cmV0dXJuCisgICAgICogICAgYWRkcmVzcyBvZmYgdGhlIHN0YWNrKHMpLCB1
c2UgaXQgdG8gbG9vayB1cCB0aGUgZml4dXAgYWRkcmVzcywgYW5kCisgICAg
ICogICAgSk1QIHRoZXJlLCB0aGVuCisgICAgICogMikgRW11bGF0ZSBhIFBV
U0ggb2YgJ3Rva2VuJyBvbnRvIHRoZSBkYXRhIHN0YWNrIHRvIHBhc3MgaW5m
b3JtYXRpb24KKyAgICAgKiAgICBhYm91dCB0aGUgZXhjZXB0aW9uIGJhY2sg
dG8gdGhlIGludm9raW5nIGNvZGUuCisgICAgICovCiAgICAgaWYgKCByZWdz
LT5yaXAgPj0gc3R1YiArIFNUVUJfQlVGX1NJWkUgLyAyICYmCiAgICAgICAg
ICByZWdzLT5yaXAgPCBzdHViICsgU1RVQl9CVUZfU0laRSAmJgogICAgICAg
ICAgcmVncy0+cnNwID4gKHVuc2lnbmVkIGxvbmcpcmVncyAmJgo=

--=separator
Content-Type: application/octet-stream; name="xsa451-2.patch"
Content-Disposition: attachment; filename="xsa451-2.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODY6IGFjY291bnQgZm9yIHNoYWRvdyBzdGFjayBpbiBleGNlcHRpb24t
ZnJvbS1zdHViIHJlY292ZXJ5CgpEZWFsaW5nIHdpdGggZXhjZXB0aW9ucyBy
YWlzZWQgZnJvbSB3aXRoaW4gZW11bGF0aW9uIHN0dWJzIGludm9sdmVzCmRp
c2NhcmRpbmcgcmV0dXJuIGFkZHJlc3MgKHJlcGxhY2VkIGJ5IGV4Y2VwdGlv
biByZWxhdGVkIGluZm9ybWF0aW9uKS4KU3VjaCBkaXNjYXJkaW5nIG9mIGNv
dXJzZSBhbHNvIHJlcXVpcmVzIHJlbW92aW5nIHRoZSBjb3JyZXNwb25kaW5n
IGVudHJ5CmZyb20gdGhlIHNoYWRvdyBzdGFjay4KCkFsc28gYW1lbmQgdGhl
IGNvbW1lbnQgaW4gZml4dXBfZXhjZXB0aW9uX3JldHVybigpLCB0byBmdXJ0
aGVyIGNsYXJpZnkKd2h5IHVzZSBvZiBwdHJbMV0gY2FuJ3QgYmUgYW4gb3V0
LW9mLWJvdW5kcyBhY2Nlc3MuCgpXaGlsZSB0b3VjaGluZyBkb19pbnZhbGlk
X29wKCkgYWxzbyBhZGQgYSBtaXNzaW5nIGZhbGwtdGhyb3VnaAphbm5vdGF0
aW9uLgoKVGhpcyBpcyBDVkUtMjAyMy00Njg0MSAvIFhTQS00NTEuCgpGaXhl
czogMjA5ZmI5OTE5YjUwICgieDg2L2V4dGFibGU6IEFkanVzdCBleHRhYmxl
IGhhbmRsaW5nIHRvIGJlIHNoYWRvdyBzdGFjayBjb21wYXRpYmxlIikKU2ln
bmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpS
ZXZpZXdlZC1ieTogQW5kcmV3IENvb3BlciA8YW5kcmV3LmNvb3BlcjNAY2l0
cml4LmNvbT4KCi0tLSBhL3hlbi9hcmNoL3g4Ni9leHRhYmxlLmMKKysrIGIv
eGVuL2FyY2gveDg2L2V4dGFibGUuYwpAQCAtODYsMTMgKzg2LDE2IEBAIHNl
YXJjaF9vbmVfZXh0YWJsZShjb25zdCBzdHJ1Y3QgZXhjZXB0aW8KIH0KIAog
dW5zaWduZWQgbG9uZwotc2VhcmNoX2V4Y2VwdGlvbl90YWJsZShjb25zdCBz
dHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncykKK3NlYXJjaF9leGNlcHRpb25f
dGFibGUoY29uc3Qgc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsIHVuc2ln
bmVkIGxvbmcgKnN0dWJfcmEpCiB7CiAgICAgY29uc3Qgc3RydWN0IHZpcnR1
YWxfcmVnaW9uICpyZWdpb24gPSBmaW5kX3RleHRfcmVnaW9uKHJlZ3MtPnJp
cCk7CiAgICAgdW5zaWduZWQgbG9uZyBzdHViID0gdGhpc19jcHUoc3R1YnMu
YWRkcik7CiAKICAgICBpZiAoIHJlZ2lvbiAmJiByZWdpb24tPmV4ICkKKyAg
ICB7CisgICAgICAgICpzdHViX3JhID0gMDsKICAgICAgICAgcmV0dXJuIHNl
YXJjaF9vbmVfZXh0YWJsZShyZWdpb24tPmV4LCByZWdpb24tPmV4X2VuZCwg
cmVncy0+cmlwKTsKKyAgICB9CiAKICAgICAvKgogICAgICAqIEVtdWxhdGlv
biBzdHVicyAod2hpY2ggYXJlIHBlci1DUFUpIGFyZSBjb25zdHJ1Y3RlZCB3
aXRoIGEgUkVUIGF0IHRoZQpAQCAtMTE1LDEzICsxMTgsMTMgQEAgc2VhcmNo
X2V4Y2VwdGlvbl90YWJsZShjb25zdCBzdHJ1Y3QgY3B1XwogICAgICAgICAg
cmVncy0+cnNwID4gKHVuc2lnbmVkIGxvbmcpcmVncyAmJgogICAgICAgICAg
cmVncy0+cnNwIDwgKHVuc2lnbmVkIGxvbmcpZ2V0X2NwdV9pbmZvKCkgKQog
ICAgIHsKLSAgICAgICAgdW5zaWduZWQgbG9uZyByZXRwdHIgPSAqKHVuc2ln
bmVkIGxvbmcgKilyZWdzLT5yc3A7CisgICAgICAgIHVuc2lnbmVkIGxvbmcg
cmV0YWRkciA9ICoodW5zaWduZWQgbG9uZyAqKXJlZ3MtPnJzcCwgZml4dXA7
CiAKLSAgICAgICAgcmVnaW9uID0gZmluZF90ZXh0X3JlZ2lvbihyZXRwdHIp
OwotICAgICAgICByZXRwdHIgPSByZWdpb24gJiYgcmVnaW9uLT5leAotICAg
ICAgICAgICAgICAgICA/IHNlYXJjaF9vbmVfZXh0YWJsZShyZWdpb24tPmV4
LCByZWdpb24tPmV4X2VuZCwgcmV0cHRyKQotICAgICAgICAgICAgICAgICA6
IDA7Ci0gICAgICAgIGlmICggcmV0cHRyICkKKyAgICAgICAgcmVnaW9uID0g
ZmluZF90ZXh0X3JlZ2lvbihyZXRhZGRyKTsKKyAgICAgICAgZml4dXAgPSBy
ZWdpb24gJiYgcmVnaW9uLT5leAorICAgICAgICAgICAgICAgID8gc2VhcmNo
X29uZV9leHRhYmxlKHJlZ2lvbi0+ZXgsIHJlZ2lvbi0+ZXhfZW5kLCByZXRh
ZGRyKQorICAgICAgICAgICAgICAgIDogMDsKKyAgICAgICAgaWYgKCBmaXh1
cCApCiAgICAgICAgIHsKICAgICAgICAgICAgIC8qCiAgICAgICAgICAgICAg
KiBQdXQgdHJhcCBudW1iZXIgYW5kIGVycm9yIGNvZGUgb24gdGhlIHN0YWNr
IChpbiBwbGFjZSBvZiB0aGUKQEAgLTEzMyw3ICsxMzYsOCBAQCBzZWFyY2hf
ZXhjZXB0aW9uX3RhYmxlKGNvbnN0IHN0cnVjdCBjcHVfCiAgICAgICAgICAg
ICB9OwogCiAgICAgICAgICAgICAqKHVuc2lnbmVkIGxvbmcgKilyZWdzLT5y
c3AgPSB0b2tlbi5yYXc7Ci0gICAgICAgICAgICByZXR1cm4gcmV0cHRyOwor
ICAgICAgICAgICAgKnN0dWJfcmEgPSByZXRhZGRyOworICAgICAgICAgICAg
cmV0dXJuIGZpeHVwOwogICAgICAgICB9CiAgICAgfQogCi0tLSBhL3hlbi9h
cmNoL3g4Ni9pbmNsdWRlL2FzbS91YWNjZXNzLmgKKysrIGIveGVuL2FyY2gv
eDg2L2luY2x1ZGUvYXNtL3VhY2Nlc3MuaApAQCAtNDIxLDcgKzQyMSw4IEBA
IHVuaW9uIHN0dWJfZXhjZXB0aW9uX3Rva2VuIHsKICAgICB1bnNpZ25lZCBs
b25nIHJhdzsKIH07CiAKLWV4dGVybiB1bnNpZ25lZCBsb25nIHNlYXJjaF9l
eGNlcHRpb25fdGFibGUoY29uc3Qgc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJl
Z3MpOworZXh0ZXJuIHVuc2lnbmVkIGxvbmcgc2VhcmNoX2V4Y2VwdGlvbl90
YWJsZShjb25zdCBzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywKKyAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWdu
ZWQgbG9uZyAqc3R1Yl9yYSk7CiBleHRlcm4gdm9pZCBzb3J0X2V4Y2VwdGlv
bl90YWJsZXModm9pZCk7CiBleHRlcm4gdm9pZCBzb3J0X2V4Y2VwdGlvbl90
YWJsZShzdHJ1Y3QgZXhjZXB0aW9uX3RhYmxlX2VudHJ5ICpzdGFydCwKICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0cnVjdCBl
eGNlcHRpb25fdGFibGVfZW50cnkgKnN0b3ApOwotLS0gYS94ZW4vYXJjaC94
ODYvdHJhcHMuYworKysgYi94ZW4vYXJjaC94ODYvdHJhcHMuYwpAQCAtODM4
LDcgKzgzOCw3IEBAIHZvaWQgYXNtbGlua2FnZSBkb191bmhhbmRsZWRfdHJh
cChzdHJ1Y3QKIH0KIAogc3RhdGljIHZvaWQgZml4dXBfZXhjZXB0aW9uX3Jl
dHVybihzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywKLSAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBmaXh1cCkK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
bG9uZyBmaXh1cCwgdW5zaWduZWQgbG9uZyBzdHViX3JhKQogewogICAgIGlm
ICggSVNfRU5BQkxFRChDT05GSUdfWEVOX1NIU1RLKSApCiAgICAgewpAQCAt
ODU1LDcgKzg1NSw4IEBAIHN0YXRpYyB2b2lkIGZpeHVwX2V4Y2VwdGlvbl9y
ZXR1cm4oc3RydWMKICAgICAgICAgICAgIC8qCiAgICAgICAgICAgICAgKiBT
ZWFyY2ggZm9yICVyaXAuICBUaGUgc2hzdGsgY3VycmVudGx5IGxvb2tzIGxp
a2UgdGhpczoKICAgICAgICAgICAgICAqCi0gICAgICAgICAgICAgKiAgIC4u
LiAgW0xpa2VseSBwb2ludGVkIHRvIGJ5IFNTUF0KKyAgICAgICAgICAgICAq
ICAgdG9rICBbU3VwZXJ2aXNvciB0b2tlbiwgPT0gJnRvayB8IEJVU1ksIG9u
bHkgd2l0aCBGUkVEIGluYWN0aXZlXQorICAgICAgICAgICAgICogICAuLi4g
IFtQb2ludGVkIHRvIGJ5IFNTUCBmb3IgbW9zdCBleGNlcHRpb25zLCBlbXB0
eSBpbiBJU1QgY2FzZXNdCiAgICAgICAgICAgICAgKiAgICVjcyAgWz09IHJl
Z3MtPmNzXQogICAgICAgICAgICAgICogICAlcmlwIFs9PSByZWdzLT5yaXBd
CiAgICAgICAgICAgICAgKiAgIFNTUCAgW0xpa2VseSBwb2ludHMgdG8gMyBz
bG90cyBoaWdoZXIsIGFib3ZlICVjc10KQEAgLTg3Myw3ICs4NzQsNTYgQEAg
c3RhdGljIHZvaWQgZml4dXBfZXhjZXB0aW9uX3JldHVybihzdHJ1YwogICAg
ICAgICAgICAgICovCiAgICAgICAgICAgICBpZiAoIHB0clswXSA9PSByZWdz
LT5yaXAgJiYgcHRyWzFdID09IHJlZ3MtPmNzICkKICAgICAgICAgICAgIHsK
KyAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nIHByaW1hcnlfc2hzdGsg
PQorICAgICAgICAgICAgICAgICAgICAoc3NwICYgfihTVEFDS19TSVpFIC0g
MSkpICsKKyAgICAgICAgICAgICAgICAgICAgKFBSSU1BUllfU0hTVEtfU0xP
VCArIDEpICogUEFHRV9TSVpFIC0gODsKKwogICAgICAgICAgICAgICAgIHdy
c3MoZml4dXAsIHB0cik7CisKKyAgICAgICAgICAgICAgICBpZiAoICFzdHVi
X3JhICkKKyAgICAgICAgICAgICAgICAgICAgZ290byBzaHN0a19kb25lOwor
CisgICAgICAgICAgICAgICAgLyoKKyAgICAgICAgICAgICAgICAgKiBTdHVi
IHJlY292ZXJ5IG91Z2h0IHRvIGhhcHBlbiBvbmx5IHdoZW4gdGhlIG91dGVy
IGNvbnRleHQKKyAgICAgICAgICAgICAgICAgKiB3YXMgb24gdGhlIG1haW4g
c2hhZG93IHN0YWNrLiAgV2UgbmVlZCB0byBhbHNvICJwb3AiIHRoZQorICAg
ICAgICAgICAgICAgICAqIHN0dWIncyByZXR1cm4gYWRkcmVzcyBmcm9tIHRo
ZSBpbnRlcnJ1cHRlZCBjb250ZXh0J3Mgc2hhZG93CisgICAgICAgICAgICAg
ICAgICogc3RhY2suICBUaGF0IGlzLAorICAgICAgICAgICAgICAgICAqIC0g
aWYgd2UncmUgc3RpbGwgb24gdGhlIG1haW4gc3RhY2ssIHdlIG5lZWQgdG8g
bW92ZSB0aGUKKyAgICAgICAgICAgICAgICAgKiAgIGVudGlyZSBzdGFjayAo
dXAgdG8gYW5kIGluY2x1ZGluZyB0aGUgZXhjZXB0aW9uIGZyYW1lKQorICAg
ICAgICAgICAgICAgICAqICAgdXAgYnkgb25lIHNsb3QsIGluY3JlbWVudGlu
ZyB0aGUgb3JpZ2luYWwgU1NQIGluIHRoZQorICAgICAgICAgICAgICAgICAq
ICAgZXhjZXB0aW9uIGZyYW1lLAorICAgICAgICAgICAgICAgICAqIC0gaWYg
d2UncmUgb24gYW4gSVNUIHN0YWNrLCB3ZSBuZWVkIHRvIGluY3JlbWVudCB0
aGUKKyAgICAgICAgICAgICAgICAgKiAgIG9yaWdpbmFsIFNTUC4KKyAgICAg
ICAgICAgICAgICAgKi8KKyAgICAgICAgICAgICAgICBCVUdfT04oKHB0clst
MV0gXiBwcmltYXJ5X3Noc3RrKSA+PiBQQUdFX1NISUZUKTsKKworICAgICAg
ICAgICAgICAgIGlmICggKHNzcCBeIHByaW1hcnlfc2hzdGspID4+IFBBR0Vf
U0hJRlQgKQorICAgICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICAg
ICAgLyoKKyAgICAgICAgICAgICAgICAgICAgICogV2UncmUgb24gYW4gSVNU
IHN0YWNrLiAgRmlyc3QgbWFrZSBzdXJlIHRoZSB0d28gcmV0dXJuCisgICAg
ICAgICAgICAgICAgICAgICAqIGFkZHJlc3NlcyBhY3R1YWxseSBtYXRjaC4g
IFRoZW4gaW5jcmVtZW50IHRoZSBpbnRlcnJ1cHRlZAorICAgICAgICAgICAg
ICAgICAgICAgKiBjb250ZXh0J3MgU1NQLgorICAgICAgICAgICAgICAgICAg
ICAgKi8KKyAgICAgICAgICAgICAgICAgICAgQlVHX09OKHN0dWJfcmEgIT0g
Kih1bnNpZ25lZCBsb25nKilwdHJbLTFdKTsKKyAgICAgICAgICAgICAgICAg
ICAgd3JzcyhwdHJbLTFdICsgOCwgJnB0clstMV0pOworICAgICAgICAgICAg
ICAgICAgICBnb3RvIHNoc3RrX2RvbmU7CisgICAgICAgICAgICAgICAgfQor
CisgICAgICAgICAgICAgICAgLyogTWFrZSBzdXJlIHRoZSB0d28gcmV0dXJu
IGFkZHJlc3NlcyBhY3R1YWxseSBtYXRjaC4gKi8KKyAgICAgICAgICAgICAg
ICBCVUdfT04oc3R1Yl9yYSAhPSBwdHJbMl0pOworCisgICAgICAgICAgICAg
ICAgLyogTW92ZSBleGNlcHRpb24gZnJhbWUsIHVwZGF0aW5nIFNTUCB0aGVy
ZS4gKi8KKyAgICAgICAgICAgICAgICB3cnNzKHB0clsxXSwgJnB0clsyXSk7
IC8qICVjcyAqLworICAgICAgICAgICAgICAgIHdyc3MocHRyWzBdLCAmcHRy
WzFdKTsgLyogJXJpcCAqLworICAgICAgICAgICAgICAgIHdyc3MocHRyWy0x
XSArIDgsICZwdHJbMF0pOyAvKiBTU1AgKi8KKworICAgICAgICAgICAgICAg
IC8qIE1vdmUgYWxsIG5ld2VyIGVudHJpZXMuICovCisgICAgICAgICAgICAg
ICAgd2hpbGUgKCAtLXB0ciAhPSBfcChzc3ApICkKKyAgICAgICAgICAgICAg
ICAgICAgd3JzcyhwdHJbLTFdLCAmcHRyWzBdKTsKKworICAgICAgICAgICAg
ICAgIC8qIEZpbmFsbHkgYWNjb3VudCBmb3Igb3VyIG93biBzdGFjayBoYXZp
bmcgc2hpZnRlZCB1cC4gKi8KKyAgICAgICAgICAgICAgICBhc20gdm9sYXRp
bGUgKCAiaW5jc3NwZCAlMCIgOjogInIiICgyKSApOworCiAgICAgICAgICAg
ICAgICAgZ290byBzaHN0a19kb25lOwogICAgICAgICAgICAgfQogICAgICAg
ICB9CkBAIC04OTQsNyArOTQ0LDggQEAgc3RhdGljIHZvaWQgZml4dXBfZXhj
ZXB0aW9uX3JldHVybihzdHJ1YwogCiBzdGF0aWMgYm9vbCBleHRhYmxlX2Zp
eHVwKHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzLCBib29sIHByaW50KQog
ewotICAgIHVuc2lnbmVkIGxvbmcgZml4dXAgPSBzZWFyY2hfZXhjZXB0aW9u
X3RhYmxlKHJlZ3MpOworICAgIHVuc2lnbmVkIGxvbmcgc3R1Yl9yYSA9IDA7
CisgICAgdW5zaWduZWQgbG9uZyBmaXh1cCA9IHNlYXJjaF9leGNlcHRpb25f
dGFibGUocmVncywgJnN0dWJfcmEpOwogCiAgICAgaWYgKCB1bmxpa2VseShm
aXh1cCA9PSAwKSApCiAgICAgICAgIHJldHVybiBmYWxzZTsKQEAgLTkwOCw3
ICs5NTksNyBAQCBzdGF0aWMgYm9vbCBleHRhYmxlX2ZpeHVwKHN0cnVjdCBj
cHVfdXNlCiAgICAgICAgICAgICAgICB2ZWN0b3JfbmFtZShyZWdzLT5lbnRy
eV92ZWN0b3IpLCByZWdzLT5lcnJvcl9jb2RlLAogICAgICAgICAgICAgICAg
X3AocmVncy0+cmlwKSwgX3AocmVncy0+cmlwKSwgX3AoZml4dXApKTsKIAot
ICAgIGZpeHVwX2V4Y2VwdGlvbl9yZXR1cm4ocmVncywgZml4dXApOworICAg
IGZpeHVwX2V4Y2VwdGlvbl9yZXR1cm4ocmVncywgZml4dXAsIHN0dWJfcmEp
OwogICAgIHRoaXNfY3B1KGxhc3RfZXh0YWJsZV9hZGRyKSA9IHJlZ3MtPnJp
cDsKIAogICAgIHJldHVybiB0cnVlOwpAQCAtMTE3MSw3ICsxMjIyLDggQEAg
dm9pZCBhc21saW5rYWdlIGRvX2ludmFsaWRfb3Aoc3RydWN0IGNwdQogICAg
IHsKICAgICBjYXNlIEJVR0ZSQU1FX3J1bl9mbjoKICAgICBjYXNlIEJVR0ZS
QU1FX3dhcm46Ci0gICAgICAgIGZpeHVwX2V4Y2VwdGlvbl9yZXR1cm4ocmVn
cywgKHVuc2lnbmVkIGxvbmcpZWlwKTsKKyAgICAgICAgZml4dXBfZXhjZXB0
aW9uX3JldHVybihyZWdzLCAodW5zaWduZWQgbG9uZyllaXAsIDApOworICAg
ICAgICBmYWxsdGhyb3VnaDsKICAgICBjYXNlIEJVR0ZSQU1FX2J1ZzoKICAg
ICBjYXNlIEJVR0ZSQU1FX2Fzc2VydDoKICAgICAgICAgcmV0dXJuOwo=

--=separator
Content-Type: application/octet-stream; name="xsa451-3.patch"
Content-Disposition: attachment; filename="xsa451-3.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODY6IHJlLXJ1biBleGNlcHRpb24tZnJvbS1zdHViIHJlY292ZXJ5IHNl
bGZ0ZXN0cyB3aXRoIENFVC1TUyBlbmFibGVkCgpPbiB0aGUgQlNQLCBzaGFk
b3cgc3RhY2tzIGFyZSBlbmFibGVkIG9ubHkgcmVsYXRpdmVseSBsYXRlIGlu
IHRoZQpib290aW5nIHByb2Nlc3MuIFRoZXkgaW4gcGFydGljdWxhciBhcmVu
J3QgYWN0aXZlIHlldCB3aGVuIGluaXRjYWxscyBhcmUKcnVuLiBLZWVwIHRo
ZSB0ZXN0aW5nIHRoZXJlLCBidXQgaW52b2tlIHRoYXQgdGVzdGluZyBhIDJu
ZCB0aW1lIHdoZW4Kc2hhZG93IHN0YWNrcyBhcmUgYWN0aXZlLCB0byBtYWtl
IHN1cmUgd2Ugd29uJ3QgcmVncmVzcyB0aGF0IGNhc2UgYWZ0ZXIKYWRkcmVz
c2luZyBYU0EtNDUxLgoKV2hpbGUgdG91Y2hpbmcgdGhpcyBjb2RlLCBzd2l0
Y2ggdGhlIGd1YXJkIGZyb20gTkRFQlVHIHRvIENPTkZJR19ERUJVRywKc3Vj
aCB0aGF0IElTX0VOQUJMRUQoKSBjYW4gdmFsaWRseSBiZSB1c2VkIGF0IHRo
ZSBuZXcgY2FsbCBzaXRlLgoKU2lnbmVkLW9mZi1ieTogSmFuIEJldWxpY2gg
PGpiZXVsaWNoQHN1c2UuY29tPgpBY2tlZC1ieTogQW5kcmV3IENvb3BlciA8
YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbT4KCi0tLSBhL3hlbi9hcmNoL3g4
Ni9leHRhYmxlLmMKKysrIGIveGVuL2FyY2gveDg2L2V4dGFibGUuYwpAQCAt
MTQ0LDEwICsxNDQsMTEgQEAgc2VhcmNoX2V4Y2VwdGlvbl90YWJsZShjb25z
dCBzdHJ1Y3QgY3B1XwogICAgIHJldHVybiAwOwogfQogCi0jaWZuZGVmIE5E
RUJVRworI2lmZGVmIENPTkZJR19ERUJVRworI2luY2x1ZGUgPGFzbS9zZXR1
cC5oPgogI2luY2x1ZGUgPGFzbS90cmFwcy5oPgogCi1zdGF0aWMgaW50IF9f
aW5pdCBjZl9jaGVjayBzdHViX3NlbGZ0ZXN0KHZvaWQpCitpbnQgX19pbml0
IGNmX2NoZWNrIHN0dWJfc2VsZnRlc3Qodm9pZCkKIHsKICAgICBzdGF0aWMg
Y29uc3Qgc3RydWN0IHsKICAgICAgICAgdWludDhfdCBvcGNbOF07CkBAIC0x
NzEsNyArMTcyLDggQEAgc3RhdGljIGludCBfX2luaXQgY2ZfY2hlY2sgc3R1
Yl9zZWxmdGVzdAogICAgIHVuc2lnbmVkIGludCBpOwogICAgIGJvb2wgZmFp
bCA9IGZhbHNlOwogCi0gICAgcHJpbnRrKCJSdW5uaW5nIHN0dWIgcmVjb3Zl
cnkgc2VsZnRlc3RzLi4uXG4iKTsKKyAgICBwcmludGsoIiVzIHN0dWIgcmVj
b3Zlcnkgc2VsZnRlc3RzLi4uXG4iLAorICAgICAgICAgICBzeXN0ZW1fc3Rh
dGUgPCBTWVNfU1RBVEVfYWN0aXZlID8gIlJ1bm5pbmciIDogIlJlLXJ1bm5p
bmciKTsKIAogICAgIGZvciAoIGkgPSAwOyBpIDwgQVJSQVlfU0laRSh0ZXN0
cyk7ICsraSApCiAgICAgewotLS0gYS94ZW4vYXJjaC94ODYvaW5jbHVkZS9h
c20vc2V0dXAuaAorKysgYi94ZW4vYXJjaC94ODYvaW5jbHVkZS9hc20vc2V0
dXAuaApAQCAtMzgsNiArMzgsOCBAQCB2b2lkICpib290c3RyYXBfbWFwKGNv
bnN0IG1vZHVsZV90ICptb2QpCiAKIGludCByZW1vdmVfeGVuX3Jhbmdlcyhz
dHJ1Y3QgcmFuZ2VzZXQgKnIpOwogCitpbnQgY2ZfY2hlY2sgc3R1Yl9zZWxm
dGVzdCh2b2lkKTsKKwogZXh0ZXJuIHVpbnQ4X3Qga2JkX3NoaWZ0X2ZsYWdz
OwogCiAjaWZkZWYgTkRFQlVHCi0tLSBhL3hlbi9hcmNoL3g4Ni9zZXR1cC5j
CisrKyBiL3hlbi9hcmNoL3g4Ni9zZXR1cC5jCkBAIC03NDAsNiArNzQwLDEw
IEBAIHN0YXRpYyB2b2lkIG5vcmV0dXJuIGluaXRfZG9uZSh2b2lkKQogCiAg
ICAgc3lzdGVtX3N0YXRlID0gU1lTX1NUQVRFX2FjdGl2ZTsKIAorICAgIC8q
IFJlLXJ1biBzdHViIHJlY292ZXJ5IHNlbGYtdGVzdHMgd2l0aCBDRVQtU1Mg
YWN0aXZlLiAqLworICAgIGlmICggSVNfRU5BQkxFRChDT05GSUdfREVCVUcp
ICYmIGNwdV9oYXNfeGVuX3Noc3RrICkKKyAgICAgICAgc3R1Yl9zZWxmdGVz
dCgpOworCiAgICAgZG9tYWluX3VucGF1c2VfYnlfc3lzdGVtY29udHJvbGxl
cihkb20wKTsKIAogICAgIC8qIE1VU1QgYmUgZG9uZSBwcmlvciB0byByZW1v
dmluZyAuaW5pdCBkYXRhLiAqLwo=

--=separator
Content-Type: application/octet-stream; name="xsa451-4.15.patch"
Content-Disposition: attachment; filename="xsa451-4.15.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODY6IGFjY291bnQgZm9yIHNoYWRvdyBzdGFjayBpbiBleGNlcHRpb24t
ZnJvbS1zdHViIHJlY292ZXJ5CgpEZWFsaW5nIHdpdGggZXhjZXB0aW9ucyBy
YWlzZWQgZnJvbSB3aXRoaW4gZW11bGF0aW9uIHN0dWJzIGludm9sdmVzCmRp
c2NhcmRpbmcgcmV0dXJuIGFkZHJlc3MgKHJlcGxhY2VkIGJ5IGV4Y2VwdGlv
biByZWxhdGVkIGluZm9ybWF0aW9uKS4KU3VjaCBkaXNjYXJkaW5nIG9mIGNv
dXJzZSBhbHNvIHJlcXVpcmVzIHJlbW92aW5nIHRoZSBjb3JyZXNwb25kaW5n
IGVudHJ5CmZyb20gdGhlIHNoYWRvdyBzdGFjay4KCkFsc28gYW1lbmQgdGhl
IGNvbW1lbnQgaW4gZml4dXBfZXhjZXB0aW9uX3JldHVybigpLCB0byBmdXJ0
aGVyIGNsYXJpZnkKd2h5IHVzZSBvZiBwdHJbMV0gY2FuJ3QgYmUgYW4gb3V0
LW9mLWJvdW5kcyBhY2Nlc3MuCgpUaGlzIGlzIENWRS0yMDIzLTQ2ODQxIC8g
WFNBLTQ1MS4KCkZpeGVzOiAyMDlmYjk5MTliNTAgKCJ4ODYvZXh0YWJsZTog
QWRqdXN0IGV4dGFibGUgaGFuZGxpbmcgdG8gYmUgc2hhZG93IHN0YWNrIGNv
bXBhdGlibGUiKQpTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxp
Y2hAc3VzZS5jb20+ClJldmlld2VkLWJ5OiBBbmRyZXcgQ29vcGVyIDxhbmRy
ZXcuY29vcGVyM0BjaXRyaXguY29tPgoKLS0tIGEveGVuL2FyY2gveDg2L2V4
dGFibGUuYworKysgYi94ZW4vYXJjaC94ODYvZXh0YWJsZS5jCkBAIC04NSwy
NiArODUsMjkgQEAgc2VhcmNoX29uZV9leHRhYmxlKGNvbnN0IHN0cnVjdCBl
eGNlcHRpbwogfQogCiB1bnNpZ25lZCBsb25nCi1zZWFyY2hfZXhjZXB0aW9u
X3RhYmxlKGNvbnN0IHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzKQorc2Vh
cmNoX2V4Y2VwdGlvbl90YWJsZShjb25zdCBzdHJ1Y3QgY3B1X3VzZXJfcmVn
cyAqcmVncywgdW5zaWduZWQgbG9uZyAqc3R1Yl9yYSkKIHsKICAgICBjb25z
dCBzdHJ1Y3QgdmlydHVhbF9yZWdpb24gKnJlZ2lvbiA9IGZpbmRfdGV4dF9y
ZWdpb24ocmVncy0+cmlwKTsKICAgICB1bnNpZ25lZCBsb25nIHN0dWIgPSB0
aGlzX2NwdShzdHVicy5hZGRyKTsKIAogICAgIGlmICggcmVnaW9uICYmIHJl
Z2lvbi0+ZXggKQorICAgIHsKKyAgICAgICAgKnN0dWJfcmEgPSAwOwogICAg
ICAgICByZXR1cm4gc2VhcmNoX29uZV9leHRhYmxlKHJlZ2lvbi0+ZXgsIHJl
Z2lvbi0+ZXhfZW5kIC0gMSwgcmVncy0+cmlwKTsKKyAgICB9CiAKICAgICBp
ZiAoIHJlZ3MtPnJpcCA+PSBzdHViICsgU1RVQl9CVUZfU0laRSAvIDIgJiYK
ICAgICAgICAgIHJlZ3MtPnJpcCA8IHN0dWIgKyBTVFVCX0JVRl9TSVpFICYm
CiAgICAgICAgICByZWdzLT5yc3AgPiAodW5zaWduZWQgbG9uZylyZWdzICYm
CiAgICAgICAgICByZWdzLT5yc3AgPCAodW5zaWduZWQgbG9uZylnZXRfY3B1
X2luZm8oKSApCiAgICAgewotICAgICAgICB1bnNpZ25lZCBsb25nIHJldHB0
ciA9ICoodW5zaWduZWQgbG9uZyAqKXJlZ3MtPnJzcDsKKyAgICAgICAgdW5z
aWduZWQgbG9uZyByZXRhZGRyID0gKih1bnNpZ25lZCBsb25nICopcmVncy0+
cnNwLCBmaXh1cDsKIAotICAgICAgICByZWdpb24gPSBmaW5kX3RleHRfcmVn
aW9uKHJldHB0cik7Ci0gICAgICAgIHJldHB0ciA9IHJlZ2lvbiAmJiByZWdp
b24tPmV4Ci0gICAgICAgICAgICAgICAgID8gc2VhcmNoX29uZV9leHRhYmxl
KHJlZ2lvbi0+ZXgsIHJlZ2lvbi0+ZXhfZW5kIC0gMSwgcmV0cHRyKQotICAg
ICAgICAgICAgICAgICA6IDA7Ci0gICAgICAgIGlmICggcmV0cHRyICkKKyAg
ICAgICAgcmVnaW9uID0gZmluZF90ZXh0X3JlZ2lvbihyZXRhZGRyKTsKKyAg
ICAgICAgZml4dXAgPSByZWdpb24gJiYgcmVnaW9uLT5leAorICAgICAgICAg
ICAgICAgID8gc2VhcmNoX29uZV9leHRhYmxlKHJlZ2lvbi0+ZXgsIHJlZ2lv
bi0+ZXhfZW5kIC0gMSwgcmV0YWRkcikKKyAgICAgICAgICAgICAgICA6IDA7
CisgICAgICAgIGlmICggZml4dXAgKQogICAgICAgICB7CiAgICAgICAgICAg
ICAvKgogICAgICAgICAgICAgICogUHV0IHRyYXAgbnVtYmVyIGFuZCBlcnJv
ciBjb2RlIG9uIHRoZSBzdGFjayAoaW4gcGxhY2Ugb2YgdGhlCkBAIC0xMTYs
NyArMTE5LDggQEAgc2VhcmNoX2V4Y2VwdGlvbl90YWJsZShjb25zdCBzdHJ1
Y3QgY3B1XwogICAgICAgICAgICAgfTsKIAogICAgICAgICAgICAgKih1bnNp
Z25lZCBsb25nICopcmVncy0+cnNwID0gdG9rZW4ucmF3OwotICAgICAgICAg
ICAgcmV0dXJuIHJldHB0cjsKKyAgICAgICAgICAgICpzdHViX3JhID0gcmV0
YWRkcjsKKyAgICAgICAgICAgIHJldHVybiBmaXh1cDsKICAgICAgICAgfQog
ICAgIH0KIAotLS0gYS94ZW4vYXJjaC94ODYvdHJhcHMuYworKysgYi94ZW4v
YXJjaC94ODYvdHJhcHMuYwpAQCAtNzgzLDcgKzc4Myw3IEBAIHN0YXRpYyB2
b2lkIGRvX3Jlc2VydmVkX3RyYXAoc3RydWN0IGNwdV8KIH0KIAogc3RhdGlj
IHZvaWQgZml4dXBfZXhjZXB0aW9uX3JldHVybihzdHJ1Y3QgY3B1X3VzZXJf
cmVncyAqcmVncywKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgbG9uZyBmaXh1cCkKKyAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBmaXh1cCwgdW5zaWduZWQg
bG9uZyBzdHViX3JhKQogewogICAgIGlmICggSVNfRU5BQkxFRChDT05GSUdf
WEVOX1NIU1RLKSApCiAgICAgewpAQCAtODAwLDcgKzgwMCw4IEBAIHN0YXRp
YyB2b2lkIGZpeHVwX2V4Y2VwdGlvbl9yZXR1cm4oc3RydWMKICAgICAgICAg
ICAgIC8qCiAgICAgICAgICAgICAgKiBTZWFyY2ggZm9yICVyaXAuICBUaGUg
c2hzdGsgY3VycmVudGx5IGxvb2tzIGxpa2UgdGhpczoKICAgICAgICAgICAg
ICAqCi0gICAgICAgICAgICAgKiAgIC4uLiAgW0xpa2VseSBwb2ludGVkIHRv
IGJ5IFNTUF0KKyAgICAgICAgICAgICAqICAgdG9rICBbU3VwZXJ2aXNvciB0
b2tlbiwgPT0gJnRvayB8IEJVU1ksIG9ubHkgd2l0aCBGUkVEIGluYWN0aXZl
XQorICAgICAgICAgICAgICogICAuLi4gIFtQb2ludGVkIHRvIGJ5IFNTUCBm
b3IgbW9zdCBleGNlcHRpb25zLCBlbXB0eSBpbiBJU1QgY2FzZXNdCiAgICAg
ICAgICAgICAgKiAgICVjcyAgWz09IHJlZ3MtPmNzXQogICAgICAgICAgICAg
ICogICAlcmlwIFs9PSByZWdzLT5yaXBdCiAgICAgICAgICAgICAgKiAgIFNT
UCAgW0xpa2VseSBwb2ludHMgdG8gMyBzbG90cyBoaWdoZXIsIGFib3ZlICVj
c10KQEAgLTgxOCw3ICs4MTksNTYgQEAgc3RhdGljIHZvaWQgZml4dXBfZXhj
ZXB0aW9uX3JldHVybihzdHJ1YwogICAgICAgICAgICAgICovCiAgICAgICAg
ICAgICBpZiAoIHB0clswXSA9PSByZWdzLT5yaXAgJiYgcHRyWzFdID09IHJl
Z3MtPmNzICkKICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICB1bnNp
Z25lZCBsb25nIHByaW1hcnlfc2hzdGsgPQorICAgICAgICAgICAgICAgICAg
ICAoc3NwICYgfihTVEFDS19TSVpFIC0gMSkpICsKKyAgICAgICAgICAgICAg
ICAgICAgKFBSSU1BUllfU0hTVEtfU0xPVCArIDEpICogUEFHRV9TSVpFIC0g
ODsKKwogICAgICAgICAgICAgICAgIHdyc3MoZml4dXAsIHB0cik7CisKKyAg
ICAgICAgICAgICAgICBpZiAoICFzdHViX3JhICkKKyAgICAgICAgICAgICAg
ICAgICAgZ290byBzaHN0a19kb25lOworCisgICAgICAgICAgICAgICAgLyoK
KyAgICAgICAgICAgICAgICAgKiBTdHViIHJlY292ZXJ5IG91Z2h0IHRvIGhh
cHBlbiBvbmx5IHdoZW4gdGhlIG91dGVyIGNvbnRleHQKKyAgICAgICAgICAg
ICAgICAgKiB3YXMgb24gdGhlIG1haW4gc2hhZG93IHN0YWNrLiAgV2UgbmVl
ZCB0byBhbHNvICJwb3AiIHRoZQorICAgICAgICAgICAgICAgICAqIHN0dWIn
cyByZXR1cm4gYWRkcmVzcyBmcm9tIHRoZSBpbnRlcnJ1cHRlZCBjb250ZXh0
J3Mgc2hhZG93CisgICAgICAgICAgICAgICAgICogc3RhY2suICBUaGF0IGlz
LAorICAgICAgICAgICAgICAgICAqIC0gaWYgd2UncmUgc3RpbGwgb24gdGhl
IG1haW4gc3RhY2ssIHdlIG5lZWQgdG8gbW92ZSB0aGUKKyAgICAgICAgICAg
ICAgICAgKiAgIGVudGlyZSBzdGFjayAodXAgdG8gYW5kIGluY2x1ZGluZyB0
aGUgZXhjZXB0aW9uIGZyYW1lKQorICAgICAgICAgICAgICAgICAqICAgdXAg
Ynkgb25lIHNsb3QsIGluY3JlbWVudGluZyB0aGUgb3JpZ2luYWwgU1NQIGlu
IHRoZQorICAgICAgICAgICAgICAgICAqICAgZXhjZXB0aW9uIGZyYW1lLAor
ICAgICAgICAgICAgICAgICAqIC0gaWYgd2UncmUgb24gYW4gSVNUIHN0YWNr
LCB3ZSBuZWVkIHRvIGluY3JlbWVudCB0aGUKKyAgICAgICAgICAgICAgICAg
KiAgIG9yaWdpbmFsIFNTUC4KKyAgICAgICAgICAgICAgICAgKi8KKyAgICAg
ICAgICAgICAgICBCVUdfT04oKHB0clstMV0gXiBwcmltYXJ5X3Noc3RrKSA+
PiBQQUdFX1NISUZUKTsKKworICAgICAgICAgICAgICAgIGlmICggKHNzcCBe
IHByaW1hcnlfc2hzdGspID4+IFBBR0VfU0hJRlQgKQorICAgICAgICAgICAg
ICAgIHsKKyAgICAgICAgICAgICAgICAgICAgLyoKKyAgICAgICAgICAgICAg
ICAgICAgICogV2UncmUgb24gYW4gSVNUIHN0YWNrLiAgRmlyc3QgbWFrZSBz
dXJlIHRoZSB0d28gcmV0dXJuCisgICAgICAgICAgICAgICAgICAgICAqIGFk
ZHJlc3NlcyBhY3R1YWxseSBtYXRjaC4gIFRoZW4gaW5jcmVtZW50IHRoZSBp
bnRlcnJ1cHRlZAorICAgICAgICAgICAgICAgICAgICAgKiBjb250ZXh0J3Mg
U1NQLgorICAgICAgICAgICAgICAgICAgICAgKi8KKyAgICAgICAgICAgICAg
ICAgICAgQlVHX09OKHN0dWJfcmEgIT0gKih1bnNpZ25lZCBsb25nKilwdHJb
LTFdKTsKKyAgICAgICAgICAgICAgICAgICAgd3JzcyhwdHJbLTFdICsgOCwg
JnB0clstMV0pOworICAgICAgICAgICAgICAgICAgICBnb3RvIHNoc3RrX2Rv
bmU7CisgICAgICAgICAgICAgICAgfQorCisgICAgICAgICAgICAgICAgLyog
TWFrZSBzdXJlIHRoZSB0d28gcmV0dXJuIGFkZHJlc3NlcyBhY3R1YWxseSBt
YXRjaC4gKi8KKyAgICAgICAgICAgICAgICBCVUdfT04oc3R1Yl9yYSAhPSBw
dHJbMl0pOworCisgICAgICAgICAgICAgICAgLyogTW92ZSBleGNlcHRpb24g
ZnJhbWUsIHVwZGF0aW5nIFNTUCB0aGVyZS4gKi8KKyAgICAgICAgICAgICAg
ICB3cnNzKHB0clsxXSwgJnB0clsyXSk7IC8qICVjcyAqLworICAgICAgICAg
ICAgICAgIHdyc3MocHRyWzBdLCAmcHRyWzFdKTsgLyogJXJpcCAqLworICAg
ICAgICAgICAgICAgIHdyc3MocHRyWy0xXSArIDgsICZwdHJbMF0pOyAvKiBT
U1AgKi8KKworICAgICAgICAgICAgICAgIC8qIE1vdmUgYWxsIG5ld2VyIGVu
dHJpZXMuICovCisgICAgICAgICAgICAgICAgd2hpbGUgKCAtLXB0ciAhPSBf
cChzc3ApICkKKyAgICAgICAgICAgICAgICAgICAgd3JzcyhwdHJbLTFdLCAm
cHRyWzBdKTsKKworICAgICAgICAgICAgICAgIC8qIEZpbmFsbHkgYWNjb3Vu
dCBmb3Igb3VyIG93biBzdGFjayBoYXZpbmcgc2hpZnRlZCB1cC4gKi8KKyAg
ICAgICAgICAgICAgICBhc20gdm9sYXRpbGUgKCAiaW5jc3NwZCAlMCIgOjog
InIiICgyKSApOworCiAgICAgICAgICAgICAgICAgZ290byBzaHN0a19kb25l
OwogICAgICAgICAgICAgfQogICAgICAgICB9CkBAIC04MzksNyArODg5LDgg
QEAgc3RhdGljIHZvaWQgZml4dXBfZXhjZXB0aW9uX3JldHVybihzdHJ1Ywog
CiBzdGF0aWMgYm9vbCBleHRhYmxlX2ZpeHVwKHN0cnVjdCBjcHVfdXNlcl9y
ZWdzICpyZWdzLCBib29sIHByaW50KQogewotICAgIHVuc2lnbmVkIGxvbmcg
Zml4dXAgPSBzZWFyY2hfZXhjZXB0aW9uX3RhYmxlKHJlZ3MpOworICAgIHVu
c2lnbmVkIGxvbmcgc3R1Yl9yYSA9IDA7CisgICAgdW5zaWduZWQgbG9uZyBm
aXh1cCA9IHNlYXJjaF9leGNlcHRpb25fdGFibGUocmVncywgJnN0dWJfcmEp
OwogCiAgICAgaWYgKCB1bmxpa2VseShmaXh1cCA9PSAwKSApCiAgICAgICAg
IHJldHVybiBmYWxzZTsKQEAgLTg1Myw3ICs5MDQsNyBAQCBzdGF0aWMgYm9v
bCBleHRhYmxlX2ZpeHVwKHN0cnVjdCBjcHVfdXNlCiAgICAgICAgICAgICAg
ICB2ZWNfbmFtZShyZWdzLT5lbnRyeV92ZWN0b3IpLCByZWdzLT5lcnJvcl9j
b2RlLAogICAgICAgICAgICAgICAgX3AocmVncy0+cmlwKSwgX3AocmVncy0+
cmlwKSwgX3AoZml4dXApKTsKIAotICAgIGZpeHVwX2V4Y2VwdGlvbl9yZXR1
cm4ocmVncywgZml4dXApOworICAgIGZpeHVwX2V4Y2VwdGlvbl9yZXR1cm4o
cmVncywgZml4dXAsIHN0dWJfcmEpOwogICAgIHRoaXNfY3B1KGxhc3RfZXh0
YWJsZV9hZGRyKSA9IHJlZ3MtPnJpcDsKIAogICAgIHJldHVybiB0cnVlOwpA
QCAtMTE0NCw3ICsxMTk1LDcgQEAgdm9pZCBkb19pbnZhbGlkX29wKHN0cnVj
dCBjcHVfdXNlcl9yZWdzCiAgICAgICAgIHZvaWQgKCpmbikoc3RydWN0IGNw
dV91c2VyX3JlZ3MgKikgPSBidWdfcHRyKGJ1Zyk7CiAKICAgICAgICAgZm4o
cmVncyk7Ci0gICAgICAgIGZpeHVwX2V4Y2VwdGlvbl9yZXR1cm4ocmVncywg
KHVuc2lnbmVkIGxvbmcpZWlwKTsKKyAgICAgICAgZml4dXBfZXhjZXB0aW9u
X3JldHVybihyZWdzLCAodW5zaWduZWQgbG9uZyllaXAsIDApOwogICAgICAg
ICByZXR1cm47CiAgICAgfQogCkBAIC0xMTY1LDcgKzEyMTYsNyBAQCB2b2lk
IGRvX2ludmFsaWRfb3Aoc3RydWN0IGNwdV91c2VyX3JlZ3MKICAgICBjYXNl
IEJVR0ZSQU1FX3dhcm46CiAgICAgICAgIHByaW50aygiWGVuIFdBUk4gYXQg
JXMlczolZFxuIiwgcHJlZml4LCBmaWxlbmFtZSwgbGluZW5vKTsKICAgICAg
ICAgc2hvd19leGVjdXRpb25fc3RhdGUocmVncyk7Ci0gICAgICAgIGZpeHVw
X2V4Y2VwdGlvbl9yZXR1cm4ocmVncywgKHVuc2lnbmVkIGxvbmcpZWlwKTsK
KyAgICAgICAgZml4dXBfZXhjZXB0aW9uX3JldHVybihyZWdzLCAodW5zaWdu
ZWQgbG9uZyllaXAsIDApOwogICAgICAgICByZXR1cm47CiAKICAgICBjYXNl
IEJVR0ZSQU1FX2J1ZzoKLS0tIGEveGVuL2luY2x1ZGUvYXNtLXg4Ni91YWNj
ZXNzLmgKKysrIGIveGVuL2luY2x1ZGUvYXNtLXg4Ni91YWNjZXNzLmgKQEAg
LTQyMSw3ICs0MjEsOCBAQCB1bmlvbiBzdHViX2V4Y2VwdGlvbl90b2tlbiB7
CiAgICAgdW5zaWduZWQgbG9uZyByYXc7CiB9OwogCi1leHRlcm4gdW5zaWdu
ZWQgbG9uZyBzZWFyY2hfZXhjZXB0aW9uX3RhYmxlKGNvbnN0IHN0cnVjdCBj
cHVfdXNlcl9yZWdzICpyZWdzKTsKK2V4dGVybiB1bnNpZ25lZCBsb25nIHNl
YXJjaF9leGNlcHRpb25fdGFibGUoY29uc3Qgc3RydWN0IGNwdV91c2VyX3Jl
Z3MgKnJlZ3MsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGxvbmcgKnN0dWJfcmEpOwogZXh0ZXJuIHZv
aWQgc29ydF9leGNlcHRpb25fdGFibGVzKHZvaWQpOwogZXh0ZXJuIHZvaWQg
c29ydF9leGNlcHRpb25fdGFibGUoc3RydWN0IGV4Y2VwdGlvbl90YWJsZV9l
bnRyeSAqc3RhcnQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICBjb25zdCBzdHJ1Y3QgZXhjZXB0aW9uX3RhYmxlX2VudHJ5ICpzdG9wKTsK

--=separator
Content-Type: application/octet-stream; name="xsa451-4.16.patch"
Content-Disposition: attachment; filename="xsa451-4.16.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODY6IGFjY291bnQgZm9yIHNoYWRvdyBzdGFjayBpbiBleGNlcHRpb24t
ZnJvbS1zdHViIHJlY292ZXJ5CgpEZWFsaW5nIHdpdGggZXhjZXB0aW9ucyBy
YWlzZWQgZnJvbSB3aXRoaW4gZW11bGF0aW9uIHN0dWJzIGludm9sdmVzCmRp
c2NhcmRpbmcgcmV0dXJuIGFkZHJlc3MgKHJlcGxhY2VkIGJ5IGV4Y2VwdGlv
biByZWxhdGVkIGluZm9ybWF0aW9uKS4KU3VjaCBkaXNjYXJkaW5nIG9mIGNv
dXJzZSBhbHNvIHJlcXVpcmVzIHJlbW92aW5nIHRoZSBjb3JyZXNwb25kaW5n
IGVudHJ5CmZyb20gdGhlIHNoYWRvdyBzdGFjay4KCkFsc28gYW1lbmQgdGhl
IGNvbW1lbnQgaW4gZml4dXBfZXhjZXB0aW9uX3JldHVybigpLCB0byBmdXJ0
aGVyIGNsYXJpZnkKd2h5IHVzZSBvZiBwdHJbMV0gY2FuJ3QgYmUgYW4gb3V0
LW9mLWJvdW5kcyBhY2Nlc3MuCgpUaGlzIGlzIENWRS0yMDIzLTQ2ODQxIC8g
WFNBLTQ1MS4KCkZpeGVzOiAyMDlmYjk5MTliNTAgKCJ4ODYvZXh0YWJsZTog
QWRqdXN0IGV4dGFibGUgaGFuZGxpbmcgdG8gYmUgc2hhZG93IHN0YWNrIGNv
bXBhdGlibGUiKQpTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxp
Y2hAc3VzZS5jb20+ClJldmlld2VkLWJ5OiBBbmRyZXcgQ29vcGVyIDxhbmRy
ZXcuY29vcGVyM0BjaXRyaXguY29tPgoKLS0tIGEveGVuL2FyY2gveDg2L2V4
dGFibGUuYworKysgYi94ZW4vYXJjaC94ODYvZXh0YWJsZS5jCkBAIC04Niwy
NiArODYsMjkgQEAgc2VhcmNoX29uZV9leHRhYmxlKGNvbnN0IHN0cnVjdCBl
eGNlcHRpbwogfQogCiB1bnNpZ25lZCBsb25nCi1zZWFyY2hfZXhjZXB0aW9u
X3RhYmxlKGNvbnN0IHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzKQorc2Vh
cmNoX2V4Y2VwdGlvbl90YWJsZShjb25zdCBzdHJ1Y3QgY3B1X3VzZXJfcmVn
cyAqcmVncywgdW5zaWduZWQgbG9uZyAqc3R1Yl9yYSkKIHsKICAgICBjb25z
dCBzdHJ1Y3QgdmlydHVhbF9yZWdpb24gKnJlZ2lvbiA9IGZpbmRfdGV4dF9y
ZWdpb24ocmVncy0+cmlwKTsKICAgICB1bnNpZ25lZCBsb25nIHN0dWIgPSB0
aGlzX2NwdShzdHVicy5hZGRyKTsKIAogICAgIGlmICggcmVnaW9uICYmIHJl
Z2lvbi0+ZXggKQorICAgIHsKKyAgICAgICAgKnN0dWJfcmEgPSAwOwogICAg
ICAgICByZXR1cm4gc2VhcmNoX29uZV9leHRhYmxlKHJlZ2lvbi0+ZXgsIHJl
Z2lvbi0+ZXhfZW5kLCByZWdzLT5yaXApOworICAgIH0KIAogICAgIGlmICgg
cmVncy0+cmlwID49IHN0dWIgKyBTVFVCX0JVRl9TSVpFIC8gMiAmJgogICAg
ICAgICAgcmVncy0+cmlwIDwgc3R1YiArIFNUVUJfQlVGX1NJWkUgJiYKICAg
ICAgICAgIHJlZ3MtPnJzcCA+ICh1bnNpZ25lZCBsb25nKXJlZ3MgJiYKICAg
ICAgICAgIHJlZ3MtPnJzcCA8ICh1bnNpZ25lZCBsb25nKWdldF9jcHVfaW5m
bygpICkKICAgICB7Ci0gICAgICAgIHVuc2lnbmVkIGxvbmcgcmV0cHRyID0g
Kih1bnNpZ25lZCBsb25nICopcmVncy0+cnNwOworICAgICAgICB1bnNpZ25l
ZCBsb25nIHJldGFkZHIgPSAqKHVuc2lnbmVkIGxvbmcgKilyZWdzLT5yc3As
IGZpeHVwOwogCi0gICAgICAgIHJlZ2lvbiA9IGZpbmRfdGV4dF9yZWdpb24o
cmV0cHRyKTsKLSAgICAgICAgcmV0cHRyID0gcmVnaW9uICYmIHJlZ2lvbi0+
ZXgKLSAgICAgICAgICAgICAgICAgPyBzZWFyY2hfb25lX2V4dGFibGUocmVn
aW9uLT5leCwgcmVnaW9uLT5leF9lbmQsIHJldHB0cikKLSAgICAgICAgICAg
ICAgICAgOiAwOwotICAgICAgICBpZiAoIHJldHB0ciApCisgICAgICAgIHJl
Z2lvbiA9IGZpbmRfdGV4dF9yZWdpb24ocmV0YWRkcik7CisgICAgICAgIGZp
eHVwID0gcmVnaW9uICYmIHJlZ2lvbi0+ZXgKKyAgICAgICAgICAgICAgICA/
IHNlYXJjaF9vbmVfZXh0YWJsZShyZWdpb24tPmV4LCByZWdpb24tPmV4X2Vu
ZCwgcmV0YWRkcikKKyAgICAgICAgICAgICAgICA6IDA7CisgICAgICAgIGlm
ICggZml4dXAgKQogICAgICAgICB7CiAgICAgICAgICAgICAvKgogICAgICAg
ICAgICAgICogUHV0IHRyYXAgbnVtYmVyIGFuZCBlcnJvciBjb2RlIG9uIHRo
ZSBzdGFjayAoaW4gcGxhY2Ugb2YgdGhlCkBAIC0xMTcsNyArMTIwLDggQEAg
c2VhcmNoX2V4Y2VwdGlvbl90YWJsZShjb25zdCBzdHJ1Y3QgY3B1XwogICAg
ICAgICAgICAgfTsKIAogICAgICAgICAgICAgKih1bnNpZ25lZCBsb25nICop
cmVncy0+cnNwID0gdG9rZW4ucmF3OwotICAgICAgICAgICAgcmV0dXJuIHJl
dHB0cjsKKyAgICAgICAgICAgICpzdHViX3JhID0gcmV0YWRkcjsKKyAgICAg
ICAgICAgIHJldHVybiBmaXh1cDsKICAgICAgICAgfQogICAgIH0KIAotLS0g
YS94ZW4vYXJjaC94ODYvdHJhcHMuYworKysgYi94ZW4vYXJjaC94ODYvdHJh
cHMuYwpAQCAtODk1LDcgKzg5NSw3IEBAIHN0YXRpYyB2b2lkIGRvX3Jlc2Vy
dmVkX3RyYXAoc3RydWN0IGNwdV8KIH0KIAogc3RhdGljIHZvaWQgZml4dXBf
ZXhjZXB0aW9uX3JldHVybihzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywK
LSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg
bG9uZyBmaXh1cCkKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgdW5zaWduZWQgbG9uZyBmaXh1cCwgdW5zaWduZWQgbG9uZyBzdHViX3Jh
KQogewogICAgIGlmICggSVNfRU5BQkxFRChDT05GSUdfWEVOX1NIU1RLKSAp
CiAgICAgewpAQCAtOTEyLDcgKzkxMiw4IEBAIHN0YXRpYyB2b2lkIGZpeHVw
X2V4Y2VwdGlvbl9yZXR1cm4oc3RydWMKICAgICAgICAgICAgIC8qCiAgICAg
ICAgICAgICAgKiBTZWFyY2ggZm9yICVyaXAuICBUaGUgc2hzdGsgY3VycmVu
dGx5IGxvb2tzIGxpa2UgdGhpczoKICAgICAgICAgICAgICAqCi0gICAgICAg
ICAgICAgKiAgIC4uLiAgW0xpa2VseSBwb2ludGVkIHRvIGJ5IFNTUF0KKyAg
ICAgICAgICAgICAqICAgdG9rICBbU3VwZXJ2aXNvciB0b2tlbiwgPT0gJnRv
ayB8IEJVU1ksIG9ubHkgd2l0aCBGUkVEIGluYWN0aXZlXQorICAgICAgICAg
ICAgICogICAuLi4gIFtQb2ludGVkIHRvIGJ5IFNTUCBmb3IgbW9zdCBleGNl
cHRpb25zLCBlbXB0eSBpbiBJU1QgY2FzZXNdCiAgICAgICAgICAgICAgKiAg
ICVjcyAgWz09IHJlZ3MtPmNzXQogICAgICAgICAgICAgICogICAlcmlwIFs9
PSByZWdzLT5yaXBdCiAgICAgICAgICAgICAgKiAgIFNTUCAgW0xpa2VseSBw
b2ludHMgdG8gMyBzbG90cyBoaWdoZXIsIGFib3ZlICVjc10KQEAgLTkzMCw3
ICs5MzEsNTYgQEAgc3RhdGljIHZvaWQgZml4dXBfZXhjZXB0aW9uX3JldHVy
bihzdHJ1YwogICAgICAgICAgICAgICovCiAgICAgICAgICAgICBpZiAoIHB0
clswXSA9PSByZWdzLT5yaXAgJiYgcHRyWzFdID09IHJlZ3MtPmNzICkKICAg
ICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nIHBy
aW1hcnlfc2hzdGsgPQorICAgICAgICAgICAgICAgICAgICAoc3NwICYgfihT
VEFDS19TSVpFIC0gMSkpICsKKyAgICAgICAgICAgICAgICAgICAgKFBSSU1B
UllfU0hTVEtfU0xPVCArIDEpICogUEFHRV9TSVpFIC0gODsKKwogICAgICAg
ICAgICAgICAgIHdyc3MoZml4dXAsIHB0cik7CisKKyAgICAgICAgICAgICAg
ICBpZiAoICFzdHViX3JhICkKKyAgICAgICAgICAgICAgICAgICAgZ290byBz
aHN0a19kb25lOworCisgICAgICAgICAgICAgICAgLyoKKyAgICAgICAgICAg
ICAgICAgKiBTdHViIHJlY292ZXJ5IG91Z2h0IHRvIGhhcHBlbiBvbmx5IHdo
ZW4gdGhlIG91dGVyIGNvbnRleHQKKyAgICAgICAgICAgICAgICAgKiB3YXMg
b24gdGhlIG1haW4gc2hhZG93IHN0YWNrLiAgV2UgbmVlZCB0byBhbHNvICJw
b3AiIHRoZQorICAgICAgICAgICAgICAgICAqIHN0dWIncyByZXR1cm4gYWRk
cmVzcyBmcm9tIHRoZSBpbnRlcnJ1cHRlZCBjb250ZXh0J3Mgc2hhZG93Cisg
ICAgICAgICAgICAgICAgICogc3RhY2suICBUaGF0IGlzLAorICAgICAgICAg
ICAgICAgICAqIC0gaWYgd2UncmUgc3RpbGwgb24gdGhlIG1haW4gc3RhY2ss
IHdlIG5lZWQgdG8gbW92ZSB0aGUKKyAgICAgICAgICAgICAgICAgKiAgIGVu
dGlyZSBzdGFjayAodXAgdG8gYW5kIGluY2x1ZGluZyB0aGUgZXhjZXB0aW9u
IGZyYW1lKQorICAgICAgICAgICAgICAgICAqICAgdXAgYnkgb25lIHNsb3Qs
IGluY3JlbWVudGluZyB0aGUgb3JpZ2luYWwgU1NQIGluIHRoZQorICAgICAg
ICAgICAgICAgICAqICAgZXhjZXB0aW9uIGZyYW1lLAorICAgICAgICAgICAg
ICAgICAqIC0gaWYgd2UncmUgb24gYW4gSVNUIHN0YWNrLCB3ZSBuZWVkIHRv
IGluY3JlbWVudCB0aGUKKyAgICAgICAgICAgICAgICAgKiAgIG9yaWdpbmFs
IFNTUC4KKyAgICAgICAgICAgICAgICAgKi8KKyAgICAgICAgICAgICAgICBC
VUdfT04oKHB0clstMV0gXiBwcmltYXJ5X3Noc3RrKSA+PiBQQUdFX1NISUZU
KTsKKworICAgICAgICAgICAgICAgIGlmICggKHNzcCBeIHByaW1hcnlfc2hz
dGspID4+IFBBR0VfU0hJRlQgKQorICAgICAgICAgICAgICAgIHsKKyAgICAg
ICAgICAgICAgICAgICAgLyoKKyAgICAgICAgICAgICAgICAgICAgICogV2Un
cmUgb24gYW4gSVNUIHN0YWNrLiAgRmlyc3QgbWFrZSBzdXJlIHRoZSB0d28g
cmV0dXJuCisgICAgICAgICAgICAgICAgICAgICAqIGFkZHJlc3NlcyBhY3R1
YWxseSBtYXRjaC4gIFRoZW4gaW5jcmVtZW50IHRoZSBpbnRlcnJ1cHRlZAor
ICAgICAgICAgICAgICAgICAgICAgKiBjb250ZXh0J3MgU1NQLgorICAgICAg
ICAgICAgICAgICAgICAgKi8KKyAgICAgICAgICAgICAgICAgICAgQlVHX09O
KHN0dWJfcmEgIT0gKih1bnNpZ25lZCBsb25nKilwdHJbLTFdKTsKKyAgICAg
ICAgICAgICAgICAgICAgd3JzcyhwdHJbLTFdICsgOCwgJnB0clstMV0pOwor
ICAgICAgICAgICAgICAgICAgICBnb3RvIHNoc3RrX2RvbmU7CisgICAgICAg
ICAgICAgICAgfQorCisgICAgICAgICAgICAgICAgLyogTWFrZSBzdXJlIHRo
ZSB0d28gcmV0dXJuIGFkZHJlc3NlcyBhY3R1YWxseSBtYXRjaC4gKi8KKyAg
ICAgICAgICAgICAgICBCVUdfT04oc3R1Yl9yYSAhPSBwdHJbMl0pOworCisg
ICAgICAgICAgICAgICAgLyogTW92ZSBleGNlcHRpb24gZnJhbWUsIHVwZGF0
aW5nIFNTUCB0aGVyZS4gKi8KKyAgICAgICAgICAgICAgICB3cnNzKHB0clsx
XSwgJnB0clsyXSk7IC8qICVjcyAqLworICAgICAgICAgICAgICAgIHdyc3Mo
cHRyWzBdLCAmcHRyWzFdKTsgLyogJXJpcCAqLworICAgICAgICAgICAgICAg
IHdyc3MocHRyWy0xXSArIDgsICZwdHJbMF0pOyAvKiBTU1AgKi8KKworICAg
ICAgICAgICAgICAgIC8qIE1vdmUgYWxsIG5ld2VyIGVudHJpZXMuICovCisg
ICAgICAgICAgICAgICAgd2hpbGUgKCAtLXB0ciAhPSBfcChzc3ApICkKKyAg
ICAgICAgICAgICAgICAgICAgd3JzcyhwdHJbLTFdLCAmcHRyWzBdKTsKKwor
ICAgICAgICAgICAgICAgIC8qIEZpbmFsbHkgYWNjb3VudCBmb3Igb3VyIG93
biBzdGFjayBoYXZpbmcgc2hpZnRlZCB1cC4gKi8KKyAgICAgICAgICAgICAg
ICBhc20gdm9sYXRpbGUgKCAiaW5jc3NwZCAlMCIgOjogInIiICgyKSApOwor
CiAgICAgICAgICAgICAgICAgZ290byBzaHN0a19kb25lOwogICAgICAgICAg
ICAgfQogICAgICAgICB9CkBAIC05NTEsNyArMTAwMSw4IEBAIHN0YXRpYyB2
b2lkIGZpeHVwX2V4Y2VwdGlvbl9yZXR1cm4oc3RydWMKIAogc3RhdGljIGJv
b2wgZXh0YWJsZV9maXh1cChzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywg
Ym9vbCBwcmludCkKIHsKLSAgICB1bnNpZ25lZCBsb25nIGZpeHVwID0gc2Vh
cmNoX2V4Y2VwdGlvbl90YWJsZShyZWdzKTsKKyAgICB1bnNpZ25lZCBsb25n
IHN0dWJfcmEgPSAwOworICAgIHVuc2lnbmVkIGxvbmcgZml4dXAgPSBzZWFy
Y2hfZXhjZXB0aW9uX3RhYmxlKHJlZ3MsICZzdHViX3JhKTsKIAogICAgIGlm
ICggdW5saWtlbHkoZml4dXAgPT0gMCkgKQogICAgICAgICByZXR1cm4gZmFs
c2U7CkBAIC05NjUsNyArMTAxNiw3IEBAIHN0YXRpYyBib29sIGV4dGFibGVf
Zml4dXAoc3RydWN0IGNwdV91c2UKICAgICAgICAgICAgICAgIHZlY19uYW1l
KHJlZ3MtPmVudHJ5X3ZlY3RvciksIHJlZ3MtPmVycm9yX2NvZGUsCiAgICAg
ICAgICAgICAgICBfcChyZWdzLT5yaXApLCBfcChyZWdzLT5yaXApLCBfcChm
aXh1cCkpOwogCi0gICAgZml4dXBfZXhjZXB0aW9uX3JldHVybihyZWdzLCBm
aXh1cCk7CisgICAgZml4dXBfZXhjZXB0aW9uX3JldHVybihyZWdzLCBmaXh1
cCwgc3R1Yl9yYSk7CiAgICAgdGhpc19jcHUobGFzdF9leHRhYmxlX2FkZHIp
ID0gcmVncy0+cmlwOwogCiAgICAgcmV0dXJuIHRydWU7CkBAIC0xMjU2LDcg
KzEzMDcsNyBAQCB2b2lkIGRvX2ludmFsaWRfb3Aoc3RydWN0IGNwdV91c2Vy
X3JlZ3MKICAgICAgICAgdm9pZCAoKmZuKShzdHJ1Y3QgY3B1X3VzZXJfcmVn
cyAqKSA9IGJ1Z19wdHIoYnVnKTsKIAogICAgICAgICBmbihyZWdzKTsKLSAg
ICAgICAgZml4dXBfZXhjZXB0aW9uX3JldHVybihyZWdzLCAodW5zaWduZWQg
bG9uZyllaXApOworICAgICAgICBmaXh1cF9leGNlcHRpb25fcmV0dXJuKHJl
Z3MsICh1bnNpZ25lZCBsb25nKWVpcCwgMCk7CiAgICAgICAgIHJldHVybjsK
ICAgICB9CiAKQEAgLTEyNzcsNyArMTMyOCw3IEBAIHZvaWQgZG9faW52YWxp
ZF9vcChzdHJ1Y3QgY3B1X3VzZXJfcmVncwogICAgIGNhc2UgQlVHRlJBTUVf
d2FybjoKICAgICAgICAgcHJpbnRrKCJYZW4gV0FSTiBhdCAlcyVzOiVkXG4i
LCBwcmVmaXgsIGZpbGVuYW1lLCBsaW5lbm8pOwogICAgICAgICBzaG93X2V4
ZWN1dGlvbl9zdGF0ZShyZWdzKTsKLSAgICAgICAgZml4dXBfZXhjZXB0aW9u
X3JldHVybihyZWdzLCAodW5zaWduZWQgbG9uZyllaXApOworICAgICAgICBm
aXh1cF9leGNlcHRpb25fcmV0dXJuKHJlZ3MsICh1bnNpZ25lZCBsb25nKWVp
cCwgMCk7CiAgICAgICAgIHJldHVybjsKIAogICAgIGNhc2UgQlVHRlJBTUVf
YnVnOgotLS0gYS94ZW4vaW5jbHVkZS9hc20teDg2L3VhY2Nlc3MuaAorKysg
Yi94ZW4vaW5jbHVkZS9hc20teDg2L3VhY2Nlc3MuaApAQCAtNDIxLDcgKzQy
MSw4IEBAIHVuaW9uIHN0dWJfZXhjZXB0aW9uX3Rva2VuIHsKICAgICB1bnNp
Z25lZCBsb25nIHJhdzsKIH07CiAKLWV4dGVybiB1bnNpZ25lZCBsb25nIHNl
YXJjaF9leGNlcHRpb25fdGFibGUoY29uc3Qgc3RydWN0IGNwdV91c2VyX3Jl
Z3MgKnJlZ3MpOworZXh0ZXJuIHVuc2lnbmVkIGxvbmcgc2VhcmNoX2V4Y2Vw
dGlvbl90YWJsZShjb25zdCBzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywK
KyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
dW5zaWduZWQgbG9uZyAqc3R1Yl9yYSk7CiBleHRlcm4gdm9pZCBzb3J0X2V4
Y2VwdGlvbl90YWJsZXModm9pZCk7CiBleHRlcm4gdm9pZCBzb3J0X2V4Y2Vw
dGlvbl90YWJsZShzdHJ1Y3QgZXhjZXB0aW9uX3RhYmxlX2VudHJ5ICpzdGFy
dCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnN0IHN0
cnVjdCBleGNlcHRpb25fdGFibGVfZW50cnkgKnN0b3ApOwo=

--=separator
Content-Type: application/octet-stream; name="xsa451-4.17.patch"
Content-Disposition: attachment; filename="xsa451-4.17.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODY6IGFjY291bnQgZm9yIHNoYWRvdyBzdGFjayBpbiBleGNlcHRpb24t
ZnJvbS1zdHViIHJlY292ZXJ5CgpEZWFsaW5nIHdpdGggZXhjZXB0aW9ucyBy
YWlzZWQgZnJvbSB3aXRoaW4gZW11bGF0aW9uIHN0dWJzIGludm9sdmVzCmRp
c2NhcmRpbmcgcmV0dXJuIGFkZHJlc3MgKHJlcGxhY2VkIGJ5IGV4Y2VwdGlv
biByZWxhdGVkIGluZm9ybWF0aW9uKS4KU3VjaCBkaXNjYXJkaW5nIG9mIGNv
dXJzZSBhbHNvIHJlcXVpcmVzIHJlbW92aW5nIHRoZSBjb3JyZXNwb25kaW5n
IGVudHJ5CmZyb20gdGhlIHNoYWRvdyBzdGFjay4KCkFsc28gYW1lbmQgdGhl
IGNvbW1lbnQgaW4gZml4dXBfZXhjZXB0aW9uX3JldHVybigpLCB0byBmdXJ0
aGVyIGNsYXJpZnkKd2h5IHVzZSBvZiBwdHJbMV0gY2FuJ3QgYmUgYW4gb3V0
LW9mLWJvdW5kcyBhY2Nlc3MuCgpUaGlzIGlzIENWRS0yMDIzLTQ2ODQxIC8g
WFNBLTQ1MS4KCkZpeGVzOiAyMDlmYjk5MTliNTAgKCJ4ODYvZXh0YWJsZTog
QWRqdXN0IGV4dGFibGUgaGFuZGxpbmcgdG8gYmUgc2hhZG93IHN0YWNrIGNv
bXBhdGlibGUiKQpTaWduZWQtb2ZmLWJ5OiBKYW4gQmV1bGljaCA8amJldWxp
Y2hAc3VzZS5jb20+ClJldmlld2VkLWJ5OiBBbmRyZXcgQ29vcGVyIDxhbmRy
ZXcuY29vcGVyM0BjaXRyaXguY29tPgoKLS0tIGEveGVuL2FyY2gveDg2L2V4
dGFibGUuYworKysgYi94ZW4vYXJjaC94ODYvZXh0YWJsZS5jCkBAIC04Niwy
NiArODYsMjkgQEAgc2VhcmNoX29uZV9leHRhYmxlKGNvbnN0IHN0cnVjdCBl
eGNlcHRpbwogfQogCiB1bnNpZ25lZCBsb25nCi1zZWFyY2hfZXhjZXB0aW9u
X3RhYmxlKGNvbnN0IHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzKQorc2Vh
cmNoX2V4Y2VwdGlvbl90YWJsZShjb25zdCBzdHJ1Y3QgY3B1X3VzZXJfcmVn
cyAqcmVncywgdW5zaWduZWQgbG9uZyAqc3R1Yl9yYSkKIHsKICAgICBjb25z
dCBzdHJ1Y3QgdmlydHVhbF9yZWdpb24gKnJlZ2lvbiA9IGZpbmRfdGV4dF9y
ZWdpb24ocmVncy0+cmlwKTsKICAgICB1bnNpZ25lZCBsb25nIHN0dWIgPSB0
aGlzX2NwdShzdHVicy5hZGRyKTsKIAogICAgIGlmICggcmVnaW9uICYmIHJl
Z2lvbi0+ZXggKQorICAgIHsKKyAgICAgICAgKnN0dWJfcmEgPSAwOwogICAg
ICAgICByZXR1cm4gc2VhcmNoX29uZV9leHRhYmxlKHJlZ2lvbi0+ZXgsIHJl
Z2lvbi0+ZXhfZW5kLCByZWdzLT5yaXApOworICAgIH0KIAogICAgIGlmICgg
cmVncy0+cmlwID49IHN0dWIgKyBTVFVCX0JVRl9TSVpFIC8gMiAmJgogICAg
ICAgICAgcmVncy0+cmlwIDwgc3R1YiArIFNUVUJfQlVGX1NJWkUgJiYKICAg
ICAgICAgIHJlZ3MtPnJzcCA+ICh1bnNpZ25lZCBsb25nKXJlZ3MgJiYKICAg
ICAgICAgIHJlZ3MtPnJzcCA8ICh1bnNpZ25lZCBsb25nKWdldF9jcHVfaW5m
bygpICkKICAgICB7Ci0gICAgICAgIHVuc2lnbmVkIGxvbmcgcmV0cHRyID0g
Kih1bnNpZ25lZCBsb25nICopcmVncy0+cnNwOworICAgICAgICB1bnNpZ25l
ZCBsb25nIHJldGFkZHIgPSAqKHVuc2lnbmVkIGxvbmcgKilyZWdzLT5yc3As
IGZpeHVwOwogCi0gICAgICAgIHJlZ2lvbiA9IGZpbmRfdGV4dF9yZWdpb24o
cmV0cHRyKTsKLSAgICAgICAgcmV0cHRyID0gcmVnaW9uICYmIHJlZ2lvbi0+
ZXgKLSAgICAgICAgICAgICAgICAgPyBzZWFyY2hfb25lX2V4dGFibGUocmVn
aW9uLT5leCwgcmVnaW9uLT5leF9lbmQsIHJldHB0cikKLSAgICAgICAgICAg
ICAgICAgOiAwOwotICAgICAgICBpZiAoIHJldHB0ciApCisgICAgICAgIHJl
Z2lvbiA9IGZpbmRfdGV4dF9yZWdpb24ocmV0YWRkcik7CisgICAgICAgIGZp
eHVwID0gcmVnaW9uICYmIHJlZ2lvbi0+ZXgKKyAgICAgICAgICAgICAgICA/
IHNlYXJjaF9vbmVfZXh0YWJsZShyZWdpb24tPmV4LCByZWdpb24tPmV4X2Vu
ZCwgcmV0YWRkcikKKyAgICAgICAgICAgICAgICA6IDA7CisgICAgICAgIGlm
ICggZml4dXAgKQogICAgICAgICB7CiAgICAgICAgICAgICAvKgogICAgICAg
ICAgICAgICogUHV0IHRyYXAgbnVtYmVyIGFuZCBlcnJvciBjb2RlIG9uIHRo
ZSBzdGFjayAoaW4gcGxhY2Ugb2YgdGhlCkBAIC0xMTcsNyArMTIwLDggQEAg
c2VhcmNoX2V4Y2VwdGlvbl90YWJsZShjb25zdCBzdHJ1Y3QgY3B1XwogICAg
ICAgICAgICAgfTsKIAogICAgICAgICAgICAgKih1bnNpZ25lZCBsb25nICop
cmVncy0+cnNwID0gdG9rZW4ucmF3OwotICAgICAgICAgICAgcmV0dXJuIHJl
dHB0cjsKKyAgICAgICAgICAgICpzdHViX3JhID0gcmV0YWRkcjsKKyAgICAg
ICAgICAgIHJldHVybiBmaXh1cDsKICAgICAgICAgfQogICAgIH0KIAotLS0g
YS94ZW4vYXJjaC94ODYvaW5jbHVkZS9hc20vdWFjY2Vzcy5oCisrKyBiL3hl
bi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS91YWNjZXNzLmgKQEAgLTQyMSw3ICs0
MjEsOCBAQCB1bmlvbiBzdHViX2V4Y2VwdGlvbl90b2tlbiB7CiAgICAgdW5z
aWduZWQgbG9uZyByYXc7CiB9OwogCi1leHRlcm4gdW5zaWduZWQgbG9uZyBz
ZWFyY2hfZXhjZXB0aW9uX3RhYmxlKGNvbnN0IHN0cnVjdCBjcHVfdXNlcl9y
ZWdzICpyZWdzKTsKK2V4dGVybiB1bnNpZ25lZCBsb25nIHNlYXJjaF9leGNl
cHRpb25fdGFibGUoY29uc3Qgc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3Ms
CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IHVuc2lnbmVkIGxvbmcgKnN0dWJfcmEpOwogZXh0ZXJuIHZvaWQgc29ydF9l
eGNlcHRpb25fdGFibGVzKHZvaWQpOwogZXh0ZXJuIHZvaWQgc29ydF9leGNl
cHRpb25fdGFibGUoc3RydWN0IGV4Y2VwdGlvbl90YWJsZV9lbnRyeSAqc3Rh
cnQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBz
dHJ1Y3QgZXhjZXB0aW9uX3RhYmxlX2VudHJ5ICpzdG9wKTsKLS0tIGEveGVu
L2FyY2gveDg2L3RyYXBzLmMKKysrIGIveGVuL2FyY2gveDg2L3RyYXBzLmMK
QEAgLTg1Niw3ICs4NTYsNyBAQCB2b2lkIGRvX3VuaGFuZGxlZF90cmFwKHN0
cnVjdCBjcHVfdXNlcl9yCiB9CiAKIHN0YXRpYyB2b2lkIGZpeHVwX2V4Y2Vw
dGlvbl9yZXR1cm4oc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsCi0gICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVuc2lnbmVkIGxvbmcg
Zml4dXApCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHVu
c2lnbmVkIGxvbmcgZml4dXAsIHVuc2lnbmVkIGxvbmcgc3R1Yl9yYSkKIHsK
ICAgICBpZiAoIElTX0VOQUJMRUQoQ09ORklHX1hFTl9TSFNUSykgKQogICAg
IHsKQEAgLTg3Myw3ICs4NzMsOCBAQCBzdGF0aWMgdm9pZCBmaXh1cF9leGNl
cHRpb25fcmV0dXJuKHN0cnVjCiAgICAgICAgICAgICAvKgogICAgICAgICAg
ICAgICogU2VhcmNoIGZvciAlcmlwLiAgVGhlIHNoc3RrIGN1cnJlbnRseSBs
b29rcyBsaWtlIHRoaXM6CiAgICAgICAgICAgICAgKgotICAgICAgICAgICAg
ICogICAuLi4gIFtMaWtlbHkgcG9pbnRlZCB0byBieSBTU1BdCisgICAgICAg
ICAgICAgKiAgIHRvayAgW1N1cGVydmlzb3IgdG9rZW4sID09ICZ0b2sgfCBC
VVNZLCBvbmx5IHdpdGggRlJFRCBpbmFjdGl2ZV0KKyAgICAgICAgICAgICAq
ICAgLi4uICBbUG9pbnRlZCB0byBieSBTU1AgZm9yIG1vc3QgZXhjZXB0aW9u
cywgZW1wdHkgaW4gSVNUIGNhc2VzXQogICAgICAgICAgICAgICogICAlY3Mg
IFs9PSByZWdzLT5jc10KICAgICAgICAgICAgICAqICAgJXJpcCBbPT0gcmVn
cy0+cmlwXQogICAgICAgICAgICAgICogICBTU1AgIFtMaWtlbHkgcG9pbnRz
IHRvIDMgc2xvdHMgaGlnaGVyLCBhYm92ZSAlY3NdCkBAIC04OTEsNyArODky
LDU2IEBAIHN0YXRpYyB2b2lkIGZpeHVwX2V4Y2VwdGlvbl9yZXR1cm4oc3Ry
dWMKICAgICAgICAgICAgICAqLwogICAgICAgICAgICAgaWYgKCBwdHJbMF0g
PT0gcmVncy0+cmlwICYmIHB0clsxXSA9PSByZWdzLT5jcyApCiAgICAgICAg
ICAgICB7CisgICAgICAgICAgICAgICAgdW5zaWduZWQgbG9uZyBwcmltYXJ5
X3Noc3RrID0KKyAgICAgICAgICAgICAgICAgICAgKHNzcCAmIH4oU1RBQ0tf
U0laRSAtIDEpKSArCisgICAgICAgICAgICAgICAgICAgIChQUklNQVJZX1NI
U1RLX1NMT1QgKyAxKSAqIFBBR0VfU0laRSAtIDg7CisKICAgICAgICAgICAg
ICAgICB3cnNzKGZpeHVwLCBwdHIpOworCisgICAgICAgICAgICAgICAgaWYg
KCAhc3R1Yl9yYSApCisgICAgICAgICAgICAgICAgICAgIGdvdG8gc2hzdGtf
ZG9uZTsKKworICAgICAgICAgICAgICAgIC8qCisgICAgICAgICAgICAgICAg
ICogU3R1YiByZWNvdmVyeSBvdWdodCB0byBoYXBwZW4gb25seSB3aGVuIHRo
ZSBvdXRlciBjb250ZXh0CisgICAgICAgICAgICAgICAgICogd2FzIG9uIHRo
ZSBtYWluIHNoYWRvdyBzdGFjay4gIFdlIG5lZWQgdG8gYWxzbyAicG9wIiB0
aGUKKyAgICAgICAgICAgICAgICAgKiBzdHViJ3MgcmV0dXJuIGFkZHJlc3Mg
ZnJvbSB0aGUgaW50ZXJydXB0ZWQgY29udGV4dCdzIHNoYWRvdworICAgICAg
ICAgICAgICAgICAqIHN0YWNrLiAgVGhhdCBpcywKKyAgICAgICAgICAgICAg
ICAgKiAtIGlmIHdlJ3JlIHN0aWxsIG9uIHRoZSBtYWluIHN0YWNrLCB3ZSBu
ZWVkIHRvIG1vdmUgdGhlCisgICAgICAgICAgICAgICAgICogICBlbnRpcmUg
c3RhY2sgKHVwIHRvIGFuZCBpbmNsdWRpbmcgdGhlIGV4Y2VwdGlvbiBmcmFt
ZSkKKyAgICAgICAgICAgICAgICAgKiAgIHVwIGJ5IG9uZSBzbG90LCBpbmNy
ZW1lbnRpbmcgdGhlIG9yaWdpbmFsIFNTUCBpbiB0aGUKKyAgICAgICAgICAg
ICAgICAgKiAgIGV4Y2VwdGlvbiBmcmFtZSwKKyAgICAgICAgICAgICAgICAg
KiAtIGlmIHdlJ3JlIG9uIGFuIElTVCBzdGFjaywgd2UgbmVlZCB0byBpbmNy
ZW1lbnQgdGhlCisgICAgICAgICAgICAgICAgICogICBvcmlnaW5hbCBTU1Au
CisgICAgICAgICAgICAgICAgICovCisgICAgICAgICAgICAgICAgQlVHX09O
KChwdHJbLTFdIF4gcHJpbWFyeV9zaHN0aykgPj4gUEFHRV9TSElGVCk7CisK
KyAgICAgICAgICAgICAgICBpZiAoIChzc3AgXiBwcmltYXJ5X3Noc3RrKSA+
PiBQQUdFX1NISUZUICkKKyAgICAgICAgICAgICAgICB7CisgICAgICAgICAg
ICAgICAgICAgIC8qCisgICAgICAgICAgICAgICAgICAgICAqIFdlJ3JlIG9u
IGFuIElTVCBzdGFjay4gIEZpcnN0IG1ha2Ugc3VyZSB0aGUgdHdvIHJldHVy
bgorICAgICAgICAgICAgICAgICAgICAgKiBhZGRyZXNzZXMgYWN0dWFsbHkg
bWF0Y2guICBUaGVuIGluY3JlbWVudCB0aGUgaW50ZXJydXB0ZWQKKyAgICAg
ICAgICAgICAgICAgICAgICogY29udGV4dCdzIFNTUC4KKyAgICAgICAgICAg
ICAgICAgICAgICovCisgICAgICAgICAgICAgICAgICAgIEJVR19PTihzdHVi
X3JhICE9ICoodW5zaWduZWQgbG9uZyopcHRyWy0xXSk7CisgICAgICAgICAg
ICAgICAgICAgIHdyc3MocHRyWy0xXSArIDgsICZwdHJbLTFdKTsKKyAgICAg
ICAgICAgICAgICAgICAgZ290byBzaHN0a19kb25lOworICAgICAgICAgICAg
ICAgIH0KKworICAgICAgICAgICAgICAgIC8qIE1ha2Ugc3VyZSB0aGUgdHdv
IHJldHVybiBhZGRyZXNzZXMgYWN0dWFsbHkgbWF0Y2guICovCisgICAgICAg
ICAgICAgICAgQlVHX09OKHN0dWJfcmEgIT0gcHRyWzJdKTsKKworICAgICAg
ICAgICAgICAgIC8qIE1vdmUgZXhjZXB0aW9uIGZyYW1lLCB1cGRhdGluZyBT
U1AgdGhlcmUuICovCisgICAgICAgICAgICAgICAgd3JzcyhwdHJbMV0sICZw
dHJbMl0pOyAvKiAlY3MgKi8KKyAgICAgICAgICAgICAgICB3cnNzKHB0clsw
XSwgJnB0clsxXSk7IC8qICVyaXAgKi8KKyAgICAgICAgICAgICAgICB3cnNz
KHB0clstMV0gKyA4LCAmcHRyWzBdKTsgLyogU1NQICovCisKKyAgICAgICAg
ICAgICAgICAvKiBNb3ZlIGFsbCBuZXdlciBlbnRyaWVzLiAqLworICAgICAg
ICAgICAgICAgIHdoaWxlICggLS1wdHIgIT0gX3Aoc3NwKSApCisgICAgICAg
ICAgICAgICAgICAgIHdyc3MocHRyWy0xXSwgJnB0clswXSk7CisKKyAgICAg
ICAgICAgICAgICAvKiBGaW5hbGx5IGFjY291bnQgZm9yIG91ciBvd24gc3Rh
Y2sgaGF2aW5nIHNoaWZ0ZWQgdXAuICovCisgICAgICAgICAgICAgICAgYXNt
IHZvbGF0aWxlICggImluY3NzcGQgJTAiIDo6ICJyIiAoMikgKTsKKwogICAg
ICAgICAgICAgICAgIGdvdG8gc2hzdGtfZG9uZTsKICAgICAgICAgICAgIH0K
ICAgICAgICAgfQpAQCAtOTEyLDcgKzk2Miw4IEBAIHN0YXRpYyB2b2lkIGZp
eHVwX2V4Y2VwdGlvbl9yZXR1cm4oc3RydWMKIAogc3RhdGljIGJvb2wgZXh0
YWJsZV9maXh1cChzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncywgYm9vbCBw
cmludCkKIHsKLSAgICB1bnNpZ25lZCBsb25nIGZpeHVwID0gc2VhcmNoX2V4
Y2VwdGlvbl90YWJsZShyZWdzKTsKKyAgICB1bnNpZ25lZCBsb25nIHN0dWJf
cmEgPSAwOworICAgIHVuc2lnbmVkIGxvbmcgZml4dXAgPSBzZWFyY2hfZXhj
ZXB0aW9uX3RhYmxlKHJlZ3MsICZzdHViX3JhKTsKIAogICAgIGlmICggdW5s
aWtlbHkoZml4dXAgPT0gMCkgKQogICAgICAgICByZXR1cm4gZmFsc2U7CkBA
IC05MjYsNyArOTc3LDcgQEAgc3RhdGljIGJvb2wgZXh0YWJsZV9maXh1cChz
dHJ1Y3QgY3B1X3VzZQogICAgICAgICAgICAgICAgdmVjdG9yX25hbWUocmVn
cy0+ZW50cnlfdmVjdG9yKSwgcmVncy0+ZXJyb3JfY29kZSwKICAgICAgICAg
ICAgICAgIF9wKHJlZ3MtPnJpcCksIF9wKHJlZ3MtPnJpcCksIF9wKGZpeHVw
KSk7CiAKLSAgICBmaXh1cF9leGNlcHRpb25fcmV0dXJuKHJlZ3MsIGZpeHVw
KTsKKyAgICBmaXh1cF9leGNlcHRpb25fcmV0dXJuKHJlZ3MsIGZpeHVwLCBz
dHViX3JhKTsKICAgICB0aGlzX2NwdShsYXN0X2V4dGFibGVfYWRkcikgPSBy
ZWdzLT5yaXA7CiAKICAgICByZXR1cm4gdHJ1ZTsKQEAgLTEyMTQsNyArMTI2
NSw3IEBAIHZvaWQgZG9faW52YWxpZF9vcChzdHJ1Y3QgY3B1X3VzZXJfcmVn
cwogICAgICAgICB2b2lkICgqZm4pKHN0cnVjdCBjcHVfdXNlcl9yZWdzICop
ID0gYnVnX3B0cihidWcpOwogCiAgICAgICAgIGZuKHJlZ3MpOwotICAgICAg
ICBmaXh1cF9leGNlcHRpb25fcmV0dXJuKHJlZ3MsICh1bnNpZ25lZCBsb25n
KWVpcCk7CisgICAgICAgIGZpeHVwX2V4Y2VwdGlvbl9yZXR1cm4ocmVncywg
KHVuc2lnbmVkIGxvbmcpZWlwLCAwKTsKICAgICAgICAgcmV0dXJuOwogICAg
IH0KIApAQCAtMTIzNSw3ICsxMjg2LDcgQEAgdm9pZCBkb19pbnZhbGlkX29w
KHN0cnVjdCBjcHVfdXNlcl9yZWdzCiAgICAgY2FzZSBCVUdGUkFNRV93YXJu
OgogICAgICAgICBwcmludGsoIlhlbiBXQVJOIGF0ICVzJXM6JWRcbiIsIHBy
ZWZpeCwgZmlsZW5hbWUsIGxpbmVubyk7CiAgICAgICAgIHNob3dfZXhlY3V0
aW9uX3N0YXRlKHJlZ3MpOwotICAgICAgICBmaXh1cF9leGNlcHRpb25fcmV0
dXJuKHJlZ3MsICh1bnNpZ25lZCBsb25nKWVpcCk7CisgICAgICAgIGZpeHVw
X2V4Y2VwdGlvbl9yZXR1cm4ocmVncywgKHVuc2lnbmVkIGxvbmcpZWlwLCAw
KTsKICAgICAgICAgcmV0dXJuOwogCiAgICAgY2FzZSBCVUdGUkFNRV9idWc6
Cg==

--=separator
Content-Type: application/octet-stream; name="xsa451-4.18.patch"
Content-Disposition: attachment; filename="xsa451-4.18.patch"
Content-Transfer-Encoding: base64

RnJvbTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpTdWJqZWN0
OiB4ODY6IGFjY291bnQgZm9yIHNoYWRvdyBzdGFjayBpbiBleGNlcHRpb24t
ZnJvbS1zdHViIHJlY292ZXJ5CgpEZWFsaW5nIHdpdGggZXhjZXB0aW9ucyBy
YWlzZWQgZnJvbSB3aXRoaW4gZW11bGF0aW9uIHN0dWJzIGludm9sdmVzCmRp
c2NhcmRpbmcgcmV0dXJuIGFkZHJlc3MgKHJlcGxhY2VkIGJ5IGV4Y2VwdGlv
biByZWxhdGVkIGluZm9ybWF0aW9uKS4KU3VjaCBkaXNjYXJkaW5nIG9mIGNv
dXJzZSBhbHNvIHJlcXVpcmVzIHJlbW92aW5nIHRoZSBjb3JyZXNwb25kaW5n
IGVudHJ5CmZyb20gdGhlIHNoYWRvdyBzdGFjay4KCkFsc28gYW1lbmQgdGhl
IGNvbW1lbnQgaW4gZml4dXBfZXhjZXB0aW9uX3JldHVybigpLCB0byBmdXJ0
aGVyIGNsYXJpZnkKd2h5IHVzZSBvZiBwdHJbMV0gY2FuJ3QgYmUgYW4gb3V0
LW9mLWJvdW5kcyBhY2Nlc3MuCgpXaGlsZSB0b3VjaGluZyBkb19pbnZhbGlk
X29wKCkgYWxzbyBhZGQgYSBtaXNzaW5nIGZhbGwtdGhyb3VnaAphbm5vdGF0
aW9uLgoKVGhpcyBpcyBDVkUtMjAyMy00Njg0MSAvIFhTQS00NTEuCgpGaXhl
czogMjA5ZmI5OTE5YjUwICgieDg2L2V4dGFibGU6IEFkanVzdCBleHRhYmxl
IGhhbmRsaW5nIHRvIGJlIHNoYWRvdyBzdGFjayBjb21wYXRpYmxlIikKU2ln
bmVkLW9mZi1ieTogSmFuIEJldWxpY2ggPGpiZXVsaWNoQHN1c2UuY29tPgpS
ZXZpZXdlZC1ieTogQW5kcmV3IENvb3BlciA8YW5kcmV3LmNvb3BlcjNAY2l0
cml4LmNvbT4KCi0tLSBhL3hlbi9hcmNoL3g4Ni9leHRhYmxlLmMKKysrIGIv
eGVuL2FyY2gveDg2L2V4dGFibGUuYwpAQCAtODYsMjYgKzg2LDI5IEBAIHNl
YXJjaF9vbmVfZXh0YWJsZShjb25zdCBzdHJ1Y3QgZXhjZXB0aW8KIH0KIAog
dW5zaWduZWQgbG9uZwotc2VhcmNoX2V4Y2VwdGlvbl90YWJsZShjb25zdCBz
dHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncykKK3NlYXJjaF9leGNlcHRpb25f
dGFibGUoY29uc3Qgc3RydWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsIHVuc2ln
bmVkIGxvbmcgKnN0dWJfcmEpCiB7CiAgICAgY29uc3Qgc3RydWN0IHZpcnR1
YWxfcmVnaW9uICpyZWdpb24gPSBmaW5kX3RleHRfcmVnaW9uKHJlZ3MtPnJp
cCk7CiAgICAgdW5zaWduZWQgbG9uZyBzdHViID0gdGhpc19jcHUoc3R1YnMu
YWRkcik7CiAKICAgICBpZiAoIHJlZ2lvbiAmJiByZWdpb24tPmV4ICkKKyAg
ICB7CisgICAgICAgICpzdHViX3JhID0gMDsKICAgICAgICAgcmV0dXJuIHNl
YXJjaF9vbmVfZXh0YWJsZShyZWdpb24tPmV4LCByZWdpb24tPmV4X2VuZCwg
cmVncy0+cmlwKTsKKyAgICB9CiAKICAgICBpZiAoIHJlZ3MtPnJpcCA+PSBz
dHViICsgU1RVQl9CVUZfU0laRSAvIDIgJiYKICAgICAgICAgIHJlZ3MtPnJp
cCA8IHN0dWIgKyBTVFVCX0JVRl9TSVpFICYmCiAgICAgICAgICByZWdzLT5y
c3AgPiAodW5zaWduZWQgbG9uZylyZWdzICYmCiAgICAgICAgICByZWdzLT5y
c3AgPCAodW5zaWduZWQgbG9uZylnZXRfY3B1X2luZm8oKSApCiAgICAgewot
ICAgICAgICB1bnNpZ25lZCBsb25nIHJldHB0ciA9ICoodW5zaWduZWQgbG9u
ZyAqKXJlZ3MtPnJzcDsKKyAgICAgICAgdW5zaWduZWQgbG9uZyByZXRhZGRy
ID0gKih1bnNpZ25lZCBsb25nICopcmVncy0+cnNwLCBmaXh1cDsKIAotICAg
ICAgICByZWdpb24gPSBmaW5kX3RleHRfcmVnaW9uKHJldHB0cik7Ci0gICAg
ICAgIHJldHB0ciA9IHJlZ2lvbiAmJiByZWdpb24tPmV4Ci0gICAgICAgICAg
ICAgICAgID8gc2VhcmNoX29uZV9leHRhYmxlKHJlZ2lvbi0+ZXgsIHJlZ2lv
bi0+ZXhfZW5kLCByZXRwdHIpCi0gICAgICAgICAgICAgICAgIDogMDsKLSAg
ICAgICAgaWYgKCByZXRwdHIgKQorICAgICAgICByZWdpb24gPSBmaW5kX3Rl
eHRfcmVnaW9uKHJldGFkZHIpOworICAgICAgICBmaXh1cCA9IHJlZ2lvbiAm
JiByZWdpb24tPmV4CisgICAgICAgICAgICAgICAgPyBzZWFyY2hfb25lX2V4
dGFibGUocmVnaW9uLT5leCwgcmVnaW9uLT5leF9lbmQsIHJldGFkZHIpCisg
ICAgICAgICAgICAgICAgOiAwOworICAgICAgICBpZiAoIGZpeHVwICkKICAg
ICAgICAgewogICAgICAgICAgICAgLyoKICAgICAgICAgICAgICAqIFB1dCB0
cmFwIG51bWJlciBhbmQgZXJyb3IgY29kZSBvbiB0aGUgc3RhY2sgKGluIHBs
YWNlIG9mIHRoZQpAQCAtMTE3LDcgKzEyMCw4IEBAIHNlYXJjaF9leGNlcHRp
b25fdGFibGUoY29uc3Qgc3RydWN0IGNwdV8KICAgICAgICAgICAgIH07CiAK
ICAgICAgICAgICAgICoodW5zaWduZWQgbG9uZyAqKXJlZ3MtPnJzcCA9IHRv
a2VuLnJhdzsKLSAgICAgICAgICAgIHJldHVybiByZXRwdHI7CisgICAgICAg
ICAgICAqc3R1Yl9yYSA9IHJldGFkZHI7CisgICAgICAgICAgICByZXR1cm4g
Zml4dXA7CiAgICAgICAgIH0KICAgICB9CiAKLS0tIGEveGVuL2FyY2gveDg2
L2luY2x1ZGUvYXNtL3VhY2Nlc3MuaAorKysgYi94ZW4vYXJjaC94ODYvaW5j
bHVkZS9hc20vdWFjY2Vzcy5oCkBAIC00MjEsNyArNDIxLDggQEAgdW5pb24g
c3R1Yl9leGNlcHRpb25fdG9rZW4gewogICAgIHVuc2lnbmVkIGxvbmcgcmF3
OwogfTsKIAotZXh0ZXJuIHVuc2lnbmVkIGxvbmcgc2VhcmNoX2V4Y2VwdGlv
bl90YWJsZShjb25zdCBzdHJ1Y3QgY3B1X3VzZXJfcmVncyAqcmVncyk7Citl
eHRlcm4gdW5zaWduZWQgbG9uZyBzZWFyY2hfZXhjZXB0aW9uX3RhYmxlKGNv
bnN0IHN0cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzLAorICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25n
ICpzdHViX3JhKTsKIGV4dGVybiB2b2lkIHNvcnRfZXhjZXB0aW9uX3RhYmxl
cyh2b2lkKTsKIGV4dGVybiB2b2lkIHNvcnRfZXhjZXB0aW9uX3RhYmxlKHN0
cnVjdCBleGNlcHRpb25fdGFibGVfZW50cnkgKnN0YXJ0LAogICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IGV4Y2VwdGlv
bl90YWJsZV9lbnRyeSAqc3RvcCk7Ci0tLSBhL3hlbi9hcmNoL3g4Ni90cmFw
cy5jCisrKyBiL3hlbi9hcmNoL3g4Ni90cmFwcy5jCkBAIC04NDUsNyArODQ1
LDcgQEAgdm9pZCBkb191bmhhbmRsZWRfdHJhcChzdHJ1Y3QgY3B1X3VzZXJf
cgogfQogCiBzdGF0aWMgdm9pZCBmaXh1cF9leGNlcHRpb25fcmV0dXJuKHN0
cnVjdCBjcHVfdXNlcl9yZWdzICpyZWdzLAotICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nIGZpeHVwKQorICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nIGZp
eHVwLCB1bnNpZ25lZCBsb25nIHN0dWJfcmEpCiB7CiAgICAgaWYgKCBJU19F
TkFCTEVEKENPTkZJR19YRU5fU0hTVEspICkKICAgICB7CkBAIC04NjIsNyAr
ODYyLDggQEAgc3RhdGljIHZvaWQgZml4dXBfZXhjZXB0aW9uX3JldHVybihz
dHJ1YwogICAgICAgICAgICAgLyoKICAgICAgICAgICAgICAqIFNlYXJjaCBm
b3IgJXJpcC4gIFRoZSBzaHN0ayBjdXJyZW50bHkgbG9va3MgbGlrZSB0aGlz
OgogICAgICAgICAgICAgICoKLSAgICAgICAgICAgICAqICAgLi4uICBbTGlr
ZWx5IHBvaW50ZWQgdG8gYnkgU1NQXQorICAgICAgICAgICAgICogICB0b2sg
IFtTdXBlcnZpc29yIHRva2VuLCA9PSAmdG9rIHwgQlVTWSwgb25seSB3aXRo
IEZSRUQgaW5hY3RpdmVdCisgICAgICAgICAgICAgKiAgIC4uLiAgW1BvaW50
ZWQgdG8gYnkgU1NQIGZvciBtb3N0IGV4Y2VwdGlvbnMsIGVtcHR5IGluIElT
VCBjYXNlc10KICAgICAgICAgICAgICAqICAgJWNzICBbPT0gcmVncy0+Y3Nd
CiAgICAgICAgICAgICAgKiAgICVyaXAgWz09IHJlZ3MtPnJpcF0KICAgICAg
ICAgICAgICAqICAgU1NQICBbTGlrZWx5IHBvaW50cyB0byAzIHNsb3RzIGhp
Z2hlciwgYWJvdmUgJWNzXQpAQCAtODgwLDcgKzg4MSw1NiBAQCBzdGF0aWMg
dm9pZCBmaXh1cF9leGNlcHRpb25fcmV0dXJuKHN0cnVjCiAgICAgICAgICAg
ICAgKi8KICAgICAgICAgICAgIGlmICggcHRyWzBdID09IHJlZ3MtPnJpcCAm
JiBwdHJbMV0gPT0gcmVncy0+Y3MgKQogICAgICAgICAgICAgeworICAgICAg
ICAgICAgICAgIHVuc2lnbmVkIGxvbmcgcHJpbWFyeV9zaHN0ayA9CisgICAg
ICAgICAgICAgICAgICAgIChzc3AgJiB+KFNUQUNLX1NJWkUgLSAxKSkgKwor
ICAgICAgICAgICAgICAgICAgICAoUFJJTUFSWV9TSFNUS19TTE9UICsgMSkg
KiBQQUdFX1NJWkUgLSA4OworCiAgICAgICAgICAgICAgICAgd3JzcyhmaXh1
cCwgcHRyKTsKKworICAgICAgICAgICAgICAgIGlmICggIXN0dWJfcmEgKQor
ICAgICAgICAgICAgICAgICAgICBnb3RvIHNoc3RrX2RvbmU7CisKKyAgICAg
ICAgICAgICAgICAvKgorICAgICAgICAgICAgICAgICAqIFN0dWIgcmVjb3Zl
cnkgb3VnaHQgdG8gaGFwcGVuIG9ubHkgd2hlbiB0aGUgb3V0ZXIgY29udGV4
dAorICAgICAgICAgICAgICAgICAqIHdhcyBvbiB0aGUgbWFpbiBzaGFkb3cg
c3RhY2suICBXZSBuZWVkIHRvIGFsc28gInBvcCIgdGhlCisgICAgICAgICAg
ICAgICAgICogc3R1YidzIHJldHVybiBhZGRyZXNzIGZyb20gdGhlIGludGVy
cnVwdGVkIGNvbnRleHQncyBzaGFkb3cKKyAgICAgICAgICAgICAgICAgKiBz
dGFjay4gIFRoYXQgaXMsCisgICAgICAgICAgICAgICAgICogLSBpZiB3ZSdy
ZSBzdGlsbCBvbiB0aGUgbWFpbiBzdGFjaywgd2UgbmVlZCB0byBtb3ZlIHRo
ZQorICAgICAgICAgICAgICAgICAqICAgZW50aXJlIHN0YWNrICh1cCB0byBh
bmQgaW5jbHVkaW5nIHRoZSBleGNlcHRpb24gZnJhbWUpCisgICAgICAgICAg
ICAgICAgICogICB1cCBieSBvbmUgc2xvdCwgaW5jcmVtZW50aW5nIHRoZSBv
cmlnaW5hbCBTU1AgaW4gdGhlCisgICAgICAgICAgICAgICAgICogICBleGNl
cHRpb24gZnJhbWUsCisgICAgICAgICAgICAgICAgICogLSBpZiB3ZSdyZSBv
biBhbiBJU1Qgc3RhY2ssIHdlIG5lZWQgdG8gaW5jcmVtZW50IHRoZQorICAg
ICAgICAgICAgICAgICAqICAgb3JpZ2luYWwgU1NQLgorICAgICAgICAgICAg
ICAgICAqLworICAgICAgICAgICAgICAgIEJVR19PTigocHRyWy0xXSBeIHBy
aW1hcnlfc2hzdGspID4+IFBBR0VfU0hJRlQpOworCisgICAgICAgICAgICAg
ICAgaWYgKCAoc3NwIF4gcHJpbWFyeV9zaHN0aykgPj4gUEFHRV9TSElGVCAp
CisgICAgICAgICAgICAgICAgeworICAgICAgICAgICAgICAgICAgICAvKgor
ICAgICAgICAgICAgICAgICAgICAgKiBXZSdyZSBvbiBhbiBJU1Qgc3RhY2su
ICBGaXJzdCBtYWtlIHN1cmUgdGhlIHR3byByZXR1cm4KKyAgICAgICAgICAg
ICAgICAgICAgICogYWRkcmVzc2VzIGFjdHVhbGx5IG1hdGNoLiAgVGhlbiBp
bmNyZW1lbnQgdGhlIGludGVycnVwdGVkCisgICAgICAgICAgICAgICAgICAg
ICAqIGNvbnRleHQncyBTU1AuCisgICAgICAgICAgICAgICAgICAgICAqLwor
ICAgICAgICAgICAgICAgICAgICBCVUdfT04oc3R1Yl9yYSAhPSAqKHVuc2ln
bmVkIGxvbmcqKXB0clstMV0pOworICAgICAgICAgICAgICAgICAgICB3cnNz
KHB0clstMV0gKyA4LCAmcHRyWy0xXSk7CisgICAgICAgICAgICAgICAgICAg
IGdvdG8gc2hzdGtfZG9uZTsKKyAgICAgICAgICAgICAgICB9CisKKyAgICAg
ICAgICAgICAgICAvKiBNYWtlIHN1cmUgdGhlIHR3byByZXR1cm4gYWRkcmVz
c2VzIGFjdHVhbGx5IG1hdGNoLiAqLworICAgICAgICAgICAgICAgIEJVR19P
TihzdHViX3JhICE9IHB0clsyXSk7CisKKyAgICAgICAgICAgICAgICAvKiBN
b3ZlIGV4Y2VwdGlvbiBmcmFtZSwgdXBkYXRpbmcgU1NQIHRoZXJlLiAqLwor
ICAgICAgICAgICAgICAgIHdyc3MocHRyWzFdLCAmcHRyWzJdKTsgLyogJWNz
ICovCisgICAgICAgICAgICAgICAgd3JzcyhwdHJbMF0sICZwdHJbMV0pOyAv
KiAlcmlwICovCisgICAgICAgICAgICAgICAgd3JzcyhwdHJbLTFdICsgOCwg
JnB0clswXSk7IC8qIFNTUCAqLworCisgICAgICAgICAgICAgICAgLyogTW92
ZSBhbGwgbmV3ZXIgZW50cmllcy4gKi8KKyAgICAgICAgICAgICAgICB3aGls
ZSAoIC0tcHRyICE9IF9wKHNzcCkgKQorICAgICAgICAgICAgICAgICAgICB3
cnNzKHB0clstMV0sICZwdHJbMF0pOworCisgICAgICAgICAgICAgICAgLyog
RmluYWxseSBhY2NvdW50IGZvciBvdXIgb3duIHN0YWNrIGhhdmluZyBzaGlm
dGVkIHVwLiAqLworICAgICAgICAgICAgICAgIGFzbSB2b2xhdGlsZSAoICJp
bmNzc3BkICUwIiA6OiAiciIgKDIpICk7CisKICAgICAgICAgICAgICAgICBn
b3RvIHNoc3RrX2RvbmU7CiAgICAgICAgICAgICB9CiAgICAgICAgIH0KQEAg
LTkwMSw3ICs5NTEsOCBAQCBzdGF0aWMgdm9pZCBmaXh1cF9leGNlcHRpb25f
cmV0dXJuKHN0cnVjCiAKIHN0YXRpYyBib29sIGV4dGFibGVfZml4dXAoc3Ry
dWN0IGNwdV91c2VyX3JlZ3MgKnJlZ3MsIGJvb2wgcHJpbnQpCiB7Ci0gICAg
dW5zaWduZWQgbG9uZyBmaXh1cCA9IHNlYXJjaF9leGNlcHRpb25fdGFibGUo
cmVncyk7CisgICAgdW5zaWduZWQgbG9uZyBzdHViX3JhID0gMDsKKyAgICB1
bnNpZ25lZCBsb25nIGZpeHVwID0gc2VhcmNoX2V4Y2VwdGlvbl90YWJsZShy
ZWdzLCAmc3R1Yl9yYSk7CiAKICAgICBpZiAoIHVubGlrZWx5KGZpeHVwID09
IDApICkKICAgICAgICAgcmV0dXJuIGZhbHNlOwpAQCAtOTE1LDcgKzk2Niw3
IEBAIHN0YXRpYyBib29sIGV4dGFibGVfZml4dXAoc3RydWN0IGNwdV91c2UK
ICAgICAgICAgICAgICAgIHZlY3Rvcl9uYW1lKHJlZ3MtPmVudHJ5X3ZlY3Rv
ciksIHJlZ3MtPmVycm9yX2NvZGUsCiAgICAgICAgICAgICAgICBfcChyZWdz
LT5yaXApLCBfcChyZWdzLT5yaXApLCBfcChmaXh1cCkpOwogCi0gICAgZml4
dXBfZXhjZXB0aW9uX3JldHVybihyZWdzLCBmaXh1cCk7CisgICAgZml4dXBf
ZXhjZXB0aW9uX3JldHVybihyZWdzLCBmaXh1cCwgc3R1Yl9yYSk7CiAgICAg
dGhpc19jcHUobGFzdF9leHRhYmxlX2FkZHIpID0gcmVncy0+cmlwOwogCiAg
ICAgcmV0dXJuIHRydWU7CkBAIC0xMTgzLDcgKzEyMzQsOCBAQCB2b2lkIGRv
X2ludmFsaWRfb3Aoc3RydWN0IGNwdV91c2VyX3JlZ3MKICAgICB7CiAgICAg
Y2FzZSBCVUdGUkFNRV9ydW5fZm46CiAgICAgY2FzZSBCVUdGUkFNRV93YXJu
OgotICAgICAgICBmaXh1cF9leGNlcHRpb25fcmV0dXJuKHJlZ3MsICh1bnNp
Z25lZCBsb25nKWVpcCk7CisgICAgICAgIGZpeHVwX2V4Y2VwdGlvbl9yZXR1
cm4ocmVncywgKHVuc2lnbmVkIGxvbmcpZWlwLCAwKTsKKyAgICAgICAgZmFs
bHRocm91Z2g7CiAgICAgY2FzZSBCVUdGUkFNRV9idWc6CiAgICAgY2FzZSBC
VUdGUkFNRV9hc3NlcnQ6CiAgICAgICAgIHJldHVybjsK

--=separator--


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 12:48:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 12:48:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686121.1067649 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rewsI-00065V-Jj; Tue, 27 Feb 2024 12:47:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686121.1067649; Tue, 27 Feb 2024 12:47:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rewsI-00065O-H2; Tue, 27 Feb 2024 12:47:58 +0000
Received: by outflank-mailman (input) for mailman id 686121;
 Tue, 27 Feb 2024 12:47:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rewsH-00065I-Lf
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 12:47:57 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6d651725-d56e-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 13:47:55 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-563c595f968so5542547a12.0
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 04:47:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 vg8-20020a170907d30800b00a43a4e405bbsm666196ejc.115.2024.02.27.04.47.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 27 Feb 2024 04:47:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6d651725-d56e-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709038075; x=1709642875; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Z1QqbI5mtmSOu0DPqiyO616vQ7jNr/cWLNVJDh2On38=;
        b=VWQTt0JFYUYLIilM8cULjy/aKzfBOx9haYBTLHY2uUW1bz/W+NYoY6FNayp2X01rp1
         twKjoWYROHKjAByAxB45+GxEQDMKfDOT7pLDM0lqM5RJPBXs2BqaY48lgStbUI2/tu72
         nX9kIKrDXmcp9My/or+AJ0NMT9QQ99w1KEI6DUeJLrA/UC9vILHAexcitbAFk3iJnnWm
         goz42HlZeWW8PW8kzuXKxWVhFJCk8UYXhqy0jXrg20wzws6qiKKEL4mlhtJz0+JBPRh6
         5gGmfMGJH6xxMo7VMG1ffS7LLbc0kDw/2wrytEezPyE5Zgr9HuqaxMDArRFa2+3puHVw
         0+2w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709038075; x=1709642875;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Z1QqbI5mtmSOu0DPqiyO616vQ7jNr/cWLNVJDh2On38=;
        b=E3SMedkIWM8ICJ6AYnBBwJibSA8QxiFZTfE/NCKCfcu2dbHwRQ0BpEuHOIFb3qYvK+
         nrelcdhnqV23pPVdJo0b5eNvm/4JcY1ipbp/HQYKsbaNYBsgh2PHPPfvfao+VyIgFyeQ
         jRev50fCF6qeqtibCoQLEHLbNEyLQZ6wxpXLUxDjALVjK49ERloBO38RKg/J8gX1ZLYK
         mneeWDnoF2KCoupcRKgXUoMkY3L3p4wifVroJKxIXE7QIfAAkFND8u05qSilXzBk3rh9
         YRrEE3af8LmK96Tvb0n6Eyt2I6FR+rr75AJvuKn2/iS6COcMSMIVDwClwLy3InyZAyVV
         4A+w==
X-Forwarded-Encrypted: i=1; AJvYcCWhEJE1OtrLWAY39Hz1hsZU4+2b9AiDdthUJaQ4uCXtO0fG1J28EU+LsuMVa2T0Pl8c6zNhy8zS8dlryvB9StMAZACRvVoW5dHgj9CMrfc=
X-Gm-Message-State: AOJu0YwN/t/IP4JO14pfN0pEnx6pQGClz17PlXiJAb06cK+RE/k0rfHm
	jylKKZCTiKQea5O92yiO4WfCj8BSE4sYO27MV0+wTuq4z41rzZbrx5nhTRoQSg==
X-Google-Smtp-Source: AGHT+IEW5gWWHZA3E4nolNaxAaQC570yEkJ8SZkia87fnHUE8gYjqmuAwxs35II9g6BG9/jVAiuSnw==
X-Received: by 2002:a17:906:3685:b0:a3f:98c1:3ed8 with SMTP id a5-20020a170906368500b00a3f98c13ed8mr5701889ejc.29.1709038074958;
        Tue, 27 Feb 2024 04:47:54 -0800 (PST)
Message-ID: <4bee79ca-7a7e-4bcc-ac97-5a5a57ec2c91@suse.com>
Date: Tue, 27 Feb 2024 13:47:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: Nicola Vetrini <nicola.vetrini@bugseng.com>, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
 <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
 <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
 <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com>
 <7a8e610e-913e-4a56-8ce1-6dd6abd894f4@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <7a8e610e-913e-4a56-8ce1-6dd6abd894f4@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.02.2024 12:52, Julien Grall wrote:
> Hi Jan,
> 
> On 27/02/2024 07:28, Jan Beulich wrote:
>> On 27.02.2024 01:26, Stefano Stabellini wrote:
>>> On Mon, 26 Feb 2024, Jan Beulich wrote:
>>>> On 23.02.2024 10:35, Nicola Vetrini wrote:
>>>>> Refactor cpu_notifier_call_chain into two functions:
>>>>> - the variant that is allowed to fail loses the nofail flag
>>>>> - the variant that shouldn't fail is encapsulated in a call
>>>>>    to the failing variant, with an additional check.
>>>>>
>>>>> This prevents uses of the function that are not supposed to
>>>>> fail from ignoring the return value, thus violating Rule 17.7:
>>>>> "The value returned by a function having non-void return type shall
>>>>> be used".
>>>>>
>>>>> No functional change.
>>>>>
>>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>>>
>>>> I'm afraid I disagree with this kind of bifurcation. No matter what
>>>> Misra thinks or says, it is normal for return values of functions to
>>>> not always be relevant to check.
>>>
>>> Hi Jan, I disagree.
>>>
>>> Regardless of MISRA, I really think return values need to be checked.
>>> Moreover, we decided as a group to honor MISRA Rule 17.7, which requires
>>> return values to be checked. This patch is a good step forward.
>>
>> Yet splitting functions isn't the only way to deal with Misra's
>> requirements, I suppose. After all there are functions where the
>> return value is purely courtesy for perhaps just one of its callers.
> 
> You are right that we have some places where one caller care about the 
> return value. But the problem is how do you tell whether the return was 
> ignored on purpose or not?
> 
> We had at least one XSA because the return value of a function was not 
> checked (see XSA-222). We also had plenty of smaller patches to check 
> returns.
> 
> So far, we added __must_check when we believed return values should be 
> checked. But usually at the point we notice, this is far too late.
> 
> To me the goal should be that we enforce __must_check everywhere. We are 
> probably going to detect places where we forgot to check the return. For 
> thoses that are on purpose, we can document them.
> 
>>
>> Splitting simply doesn't scale very well, imo.
> 
> Do you have another proposal? As Stefano said, we adopted the rule 17.7. 
> So we know need a solution to address it.

One possibility that was circulated while discussing was to add (void)
casts. I'm not a huge fan of those, but between the two options that
might be the lesser evil. We also use funny (should I say ugly)
workarounds in a few cases where we have __must_check but still want
to not really handle the return value in certain cases. Given there are
example in the code base, extending use of such constructs is certainly
also something that may want considering.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 12:53:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 12:53:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686123.1067658 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rewxk-00008J-6m; Tue, 27 Feb 2024 12:53:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686123.1067658; Tue, 27 Feb 2024 12:53:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rewxk-00008B-4I; Tue, 27 Feb 2024 12:53:36 +0000
Received: by outflank-mailman (input) for mailman id 686123;
 Tue, 27 Feb 2024 12:53:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rewxi-000085-L7
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 12:53:34 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rewxi-0008Q7-1j; Tue, 27 Feb 2024 12:53:34 +0000
Received: from [15.248.2.14] (helo=[10.45.19.66])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rewxh-0000m6-O2; Tue, 27 Feb 2024 12:53:33 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=b6e7ixJF5w1pI1Ywu47yw51uOYI+anWHRmhRIskW2KU=; b=RGjkJmfP+oYsxYxR75SC1wgFnu
	eXjhYMe0TG3hxPAr36FzVu6pJi9c/UjFue+f+XNSkN73zx4myLxWY27czgp5za/xyEfGaMW5PHiWF
	nDR9XTw2XcqGB1D5nkZ5euXaKLcLbPx69/oWkghNzMVhZ620jOeKo5jzXc44/lTvzucc=;
Message-ID: <9c5e0fdd-e306-46fc-9f9c-45b81163a0c0@xen.org>
Date: Tue, 27 Feb 2024 12:53:31 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/3] docs/misra/rules.rst: add rule 16.6 and 20.12
Content-Language: en-GB
To: Stefano Stabellini <stefano.stabellini@amd.com>,
 xen-devel@lists.xenproject.org
Cc: andrew.cooper3@citrix.com, jbeulich@suse.com, george.dunlap@citrix.com,
 bertrannd.marquis@arm.com, roger.pau@citrix.com,
 roberto.bagnara@bugseng.com, federico.serafini@bugseng.com
References: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop>
 <20240213223334.3693410-1-stefano.stabellini@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240213223334.3693410-1-stefano.stabellini@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Stefano,

On 13/02/2024 22:33, Stefano Stabellini wrote:
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:08:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:08:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686130.1067681 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexC2-0002RF-Hs; Tue, 27 Feb 2024 13:08:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686130.1067681; Tue, 27 Feb 2024 13:08:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexC2-0002R8-Ew; Tue, 27 Feb 2024 13:08:22 +0000
Received: by outflank-mailman (input) for mailman id 686130;
 Tue, 27 Feb 2024 13:08:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t50j=KE=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rexC2-0002R2-0H
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:08:22 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 47d842a4-d571-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 14:08:20 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-5101cd91017so6257032e87.2
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 05:08:20 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 g3-20020ac870c3000000b0042e390c9804sm3558972qtp.6.2024.02.27.05.08.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 27 Feb 2024 05:08:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 47d842a4-d571-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709039300; x=1709644100; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dkaJ7r516jVfdhjRuJVqlCumeVhhVzku00X2uXxLS2M=;
        b=hoKQpBivWdprS+wx6ZnIPABM0I9RlldUxuTQ9aI2uCrrdyXEKJkEJkHdqwU45FdTD6
         iPj4jHVKFwgeDOZ4tppwVvs6/uaNUiuL/VY3ekA4dqDIhxryMsRvDfnIrfL4WrKx/GXp
         /8jKIi2tC8tpvRx6juN9V8t6YZWju/Xq6BVko=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709039300; x=1709644100;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dkaJ7r516jVfdhjRuJVqlCumeVhhVzku00X2uXxLS2M=;
        b=S24JDJ5voBF49bTHNlntj1dlp8fKfStWDxhaOpkUEDMzyqB3B1tFqKEPXWUkR5bR6x
         6XuW0MrDP7KJ25eilADcYji9Y+Lt27Yd8ZNMdt6UCEuuCraEl8pgpjMysqSbT2mef0rn
         P9jXWiEmdXYUeOiZzrQeZJBdcyuoYqQyqpTbPLjPQH0Sn110b2aKA6FLBu64UbHD8WDM
         Xohj4vEwtVPfKqRC83nxd74s97tK6cV9UgNPFe43LSe+9Fv5ylqrvfLwsk5CfIoBQeF/
         3pJ35qlsRDatTJrGxng+oj6tiJpq/TUJ/jC5uzSBwI7OAwG4VxqG+vJ9F6c6kVNh4HDj
         ML9w==
X-Forwarded-Encrypted: i=1; AJvYcCUNHp75YNIXFOM6wiqArH31qgkMOgBP9EL78K9CX6qHFNKIJ4U+HrhzFnWZOWtQ5gugg3DRcJeZV175E/HxcFRNrajG9/m3yfLSHNW+GWk=
X-Gm-Message-State: AOJu0Yw7QYHjoNajI2kE+RmOc/t5Qf1Tcebf1TP/UhQ+zPSjY/+65Y6+
	lNr8Z6sP+pEYu+YAJMvlZN1xjU9vD0jZB0jUhX7BVrFLZ9R/S2YsRj2j6VI5YyM=
X-Google-Smtp-Source: AGHT+IHqflKF5cpkhYDSsLM9AxUlYDbC4toFRRN2pDfdKGUxSwtG9RmVlgkl1ifH4xFJ5aq2VPVh9A==
X-Received: by 2002:a05:6512:10d2:b0:512:e00b:8da4 with SMTP id k18-20020a05651210d200b00512e00b8da4mr7363773lfg.53.1709039300335;
        Tue, 27 Feb 2024 05:08:20 -0800 (PST)
Message-ID: <e8ffdaa2-0786-4f9b-b4c4-03fe88e5c72b@citrix.com>
Date: Tue, 27 Feb 2024 13:08:17 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/5] xen/livepatch: register livepatch regions when
 loaded
Content-Language: en-GB
To: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
 Ross Lagerwall <ross.lagerwall@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>
References: <20240227112528.4540-1-roger.pau@citrix.com>
 <20240227112528.4540-2-roger.pau@citrix.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <20240227112528.4540-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27/02/2024 11:25 am, Roger Pau Monne wrote:
> diff --git a/xen/common/virtual_region.c b/xen/common/virtual_region.c
> index ddac5c9147e5..e3a4dc8540df 100644
> --- a/xen/common/virtual_region.c
> +++ b/xen/common/virtual_region.c
> <snip>
>  
> +#ifdef CONFIG_LIVEPATCH
>  void unregister_virtual_region(struct virtual_region *r)
>  {
> -    /* Expected to be called from Live Patch - which has IRQs disabled. */
> -    ASSERT(!local_irq_is_enabled());
> -
>      remove_virtual_region(r);
> +
> +    /* Assert that no CPU might be using the removed region. */
> +    rcu_barrier();
>  }
>  
> -#if defined(CONFIG_LIVEPATCH) && defined(CONFIG_X86)
> +#ifdef CONFIG_X86
>  void relax_virtual_region_perms(void)
>  {
>      const struct virtual_region *region;
> @@ -117,6 +106,7 @@ void tighten_virtual_region_perms(void)
>      rcu_read_unlock(&rcu_virtual_region_lock);
>  }
>  #endif
> +#endif

Can we please have:

#endif /* CONFIG_X86 */
#endif /* CONFIG_LIVEPATCH */

before this gets any more complicated to follow.  Can be fixed on commit.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:17:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:17:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686147.1067746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexKk-0005yl-0G; Tue, 27 Feb 2024 13:17:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686147.1067746; Tue, 27 Feb 2024 13:17:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexKj-0005ye-TL; Tue, 27 Feb 2024 13:17:21 +0000
Received: by outflank-mailman (input) for mailman id 686147;
 Tue, 27 Feb 2024 13:17:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fto1=KE=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1rexKi-0005yY-1a
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:17:20 +0000
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2062b.outbound.protection.outlook.com
 [2a01:111:f400:7e88::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 877e6e22-d572-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 14:17:18 +0100 (CET)
Received: from BL1PR13CA0152.namprd13.prod.outlook.com (2603:10b6:208:2bd::7)
 by SA1PR12MB8859.namprd12.prod.outlook.com (2603:10b6:806:37c::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Tue, 27 Feb
 2024 13:17:13 +0000
Received: from BL6PEPF0001AB59.namprd02.prod.outlook.com
 (2603:10b6:208:2bd:cafe::5e) by BL1PR13CA0152.outlook.office365.com
 (2603:10b6:208:2bd::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.25 via Frontend
 Transport; Tue, 27 Feb 2024 13:17:13 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BL6PEPF0001AB59.mail.protection.outlook.com (10.167.241.11) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 27 Feb 2024 13:17:12 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 07:17:10 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 07:17:10 -0600
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 27 Feb 2024 07:17:07 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 877e6e22-d572-11ee-afd6-a90da7624cb6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=W+qkmM/ZRNcE5ghQLAXSYGGP0BOuVkDvfPm6V+Ch9WYGSdNulONqYHRJz9OLN3eM1YlTUGNWwC46Ogt1UoMs02pFNZaFepnfCROn+lk+9XNVu676baqpGadREbXQhUKFXhvMDa29Q4QyMii29WZXraPGWyYdxoPM0EXgO0wQ1N20IkhN9FuY1RBvuo7h6UDuXDDqYXSQf7S/lUlGUWlS5otIpVa5Dh5uo96rf/BbEj9MXB3E9qCTRRwkxQuqTCa0Ndsbe1fdXx1R7lJFlE1cZt2bgKEwpvukxA1fJjGb9NpmP6+0Xukqg830D3LWVtjDToEW98UCSbJEXbYfC4QSAQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=3jfcnZbpTwT0k2X+HU8w9stp1k0F2Txx7SkiWNsvG7U=;
 b=EXfakeevXh9mdjEhzMotmhTMIViLDxEL9bpFS0rwx7K4yFKg3gq2U5nUO2+bUOy0MKrqSP8MX2EuG+R7q2n3LrPfHgX4Eq6j3Pjp4oywrQ38cTC4BtP5lOAna4V3KxXDvDtEsXfs2hTIfC/3k5t2C7CnZg6Ft3QZT4MNVCOwmm7thxerxGYzTM2MgnPnvXHGGd5h3bE9FWf1Vz3XzqSiRSZBvjCNfPKUVy19mJcE1qQfYE3uVuGrjDZQ7XuPBEEiIyoA5hvJwJiz6mhrZDXIT0RU4ZmYwWELoKPVyvR5HRhJfl3Bc0SiGnrrB44LE6grW2fHAyosqwvhGmfEIy3hcw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=3jfcnZbpTwT0k2X+HU8w9stp1k0F2Txx7SkiWNsvG7U=;
 b=zlHZTCYz9pRnkpEv29Eez+8WnKnDnB75FtMO7LBkdoiUCjw/0lSjxrk7dMmVAbe+Zqhh11edCjUqrOzNJ7i9nbbE4v2FU3Gyme5B/pofHcOw0zVJcmsLJJeRRfPft7+TiWoT+fq1c7uUVCvlhnJGPuMCicjo2lzNidcnikWP/WA=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <a84aeb87-17e8-4195-90cb-7b0123064106@amd.com>
Date: Tue, 27 Feb 2024 21:17:06 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Alec Kwapis <alec.kwapis@medtronic.com>
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <d1518124-483c-4409-9b36-6a3392378911@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <d1518124-483c-4409-9b36-6a3392378911@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB59:EE_|SA1PR12MB8859:EE_
X-MS-Office365-Filtering-Correlation-Id: c99bcef4-40a5-45d0-4ab0-08dc37966922
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4XTDCSDomYt4bnaCAK9HZJC4dH8pl9A3jR0GFmsNUYcqauQcHsT32q3ppPIBd+yRa/zUdmxWClqTnFwuXwp2BCkN1wbmHNZ64RCWkTnzqUX/X4lcjZG75BJzdNX8NVhZCNGymKRj77yapbppyCrOgqi94bQ9a2eRzA6o1cPtWA0zlDeIAHxgK5FnyOrODDxC7EBSyYz3RatKUsd4FLc39B6aCw30YsMY4yDqPNOoASRGjWHZsOfA9ZeMcC9kj3DwI/ZvbNDAm/j+5djbb722ygNcYu11V5d/3c5dJjdCtlXzRtHA5FHtG7/hoj3YdnWB0Z0gRANZKxfyzG4LBabyTHWrsfGGF2dbkDAdnbWxLtCyPd931lQZS1AxTMAUoG4lfyJKyXkkc07JR8f2S3BpvU0LI2Hv3l/5z+lOa28Nh9LurMXEBdcuS9ClyURD/m/5qK8tEzPj5+Doi9X8Bd0XZhHobTfd3v9e2RWDNmJ3OTpcS3qGE9Ns41ZQCw3WwHVYqurX2V7SGctij170w/oRs09vTdvZrKtwjtvTYARkRa7LytdH5bwnofzjrTH9EVCkpJN4q+51tEcrv0XKy0A99e6gwJq5vSuzdXKHUra8gglz6R6aFgPHAX+qdCVBGt6a/Iht7izDAw5cScJaBtZeEsC0x+WeWrFreIJQBo1YMAcA6djrh6WgkLS3CsUSAALMhzID+/i4WXDnBZIvP6g8///yHLv2OdTqMTdZz++OO8k2LLbN4gWqK1Za+1L/fi/I
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 13:17:12.7776
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: c99bcef4-40a5-45d0-4ab0-08dc37966922
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB59.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8859

(-RISC-V and PPC people to avoid spamming their inbox as this is quite 
Arm specific)

Hi Julien,

On 2/26/2024 5:13 PM, Julien Grall wrote:
> Hi Henry,
>
> Welcome back!

Thanks!

> On 26/02/2024 01:19, Henry Wang wrote:
>> An error message can seen from the init-dom0less application on
>> direct-mapped 1:1 domains:
>> ```
>> Allocating magic pages
>> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
>> Error on alloc magic pages
>> ```
>>
>> This is because populate_physmap() automatically assumes gfn == mfn
>> for direct mapped domains. This cannot be true for the magic pages
>> that are allocated later for Dom0less DomUs from the init-dom0less
>> helper application executed in Dom0.
>>
>> Force populate_physmap to take the "normal" memory allocation route for
>> the magic pages even for 1:1 Dom0less DomUs. This should work as long
>> as the 1:1 Dom0less DomU doesn't have anything else mapped at the same
>> guest address as the magic pages:
>> - gfn 0x39000 address 0x39000000
>> - gfn 0x39001 address 0x39001000
>> - gfn 0x39002 address 0x39002000
>> - gfn 0x39003 address 0x39003000
>
> This is very fragile. You are making the assumption that the magic 
> pages are not clashing with any RAM region. The layout defined in 
> arch-arm.h has been designed for guest where Xen is in full control of 
> the layout. This is not the case for directmapped domain. I don't 
> think it is correct to try to re-use part of the layout.

Apologies for the (a bit) late reply, it took a bit longer for me to 
understand the story about directmap stuff, and yes, now I agree with 
you, for those directmapped domains we should not reuse the guest layout 
directly.

> If you want to use 1:1 dom0less with xenstore & co, then you should 
> find a different place in memory for the magic pages (TDB how to find 
> that area). 

Yes, and maybe we can use similar strategy in find_unallocated_memory() 
or find_domU_holes() to do that.

> You will still have the problem of the 1:1 allocation, but I think 
> this could be solved bty adding a flag to force a non-1:1 allocation.

After checking the code flow, below rough plan came to my mind, I think 
what we need to do is:

(1) Find a range of un-used memory using similar method in 
find_unallocated_memory()/find_domU_holes()

(2) Change the base address, i.e. GUEST_MAGIC_BASE in alloc_xs_page() in 
init-dom0less.c to point to the address in (1) if static mem or 11 
directmap. (I think this is a bit tricky though, do you have any method 
that in your mind?)

(3) Use a flag or combination of existing flags (CDF_staticmem + 
CDF_directmap) in populate_physmap() to force the allocation of these 
magic pages using alloc_domheap_pages() - i.e. the "else" condition in 
the bottom

Would you mind sharing some thoughts on that? Thanks!

>> Create helper is_magic_gpfn() for Arm to assist this and stub helpers
>> for non-Arm architectures to avoid #ifdef. Move the definition of the
>> magic pages on Arm to a more common place.
>>
>> Note that the init-dom0less application of the diffenent Xen version
>
> s/diffenent/different/

Oops, will correct this in v2, thanks for spotting it.

>> +
>>   #endif /* __ASM_X86_MM_H__ */
>> diff --git a/xen/common/memory.c b/xen/common/memory.c
>> index b3b05c2ec0..ab4bad79e2 100644
>> --- a/xen/common/memory.c
>> +++ b/xen/common/memory.c
>> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>>           }
>>           else
>>           {
>> -            if ( is_domain_direct_mapped(d) )
>> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
>
> This path will also be reached by dom0. Effectively, this will prevent 
> dom0 to allocate the memory 1:1 for the magic GPFN (which is guest 
> specific).

I think above proposal will solve this issue.

> Also, why are you only checking the first GFN? What if the caller pass 
> an overlapped region?

I am a bit confused. My understanding is at this point we are handling 
one page at a time.

>>               {
>>                   mfn = _mfn(gpfn);
>>   diff --git a/xen/include/public/arch-arm.h 
>> b/xen/include/public/arch-arm.h
>> index a25e87dbda..58aa6ff05b 100644
>> --- a/xen/include/public/arch-arm.h
>> +++ b/xen/include/public/arch-arm.h
>> @@ -476,6 +476,12 @@ typedef uint64_t xen_callback_t;
>>   #define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
>>   #define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
>>   +#define NR_MAGIC_PAGES 4
>> +#define CONSOLE_PFN_OFFSET 0
>> +#define XENSTORE_PFN_OFFSET 1
>> +#define MEMACCESS_PFN_OFFSET 2
>> +#define VUART_PFN_OFFSET 3
>
> Regardless of what I wrote above, it is not clear to me why you need 
> to move these macros in public header. Just above, we are defining the 
> magic region (see GUEST_MAGIC_BASE and GUEST_MAGIC_SIZE). This should 
> be sufficient to detect whether a GFN belongs to the magic region.

You are correct, I will undo the code movement in v2.

Kind regards,
Henry



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:19:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:19:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686151.1067757 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexMc-0006gH-Em; Tue, 27 Feb 2024 13:19:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686151.1067757; Tue, 27 Feb 2024 13:19:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexMc-0006gA-BU; Tue, 27 Feb 2024 13:19:18 +0000
Received: by outflank-mailman (input) for mailman id 686151;
 Tue, 27 Feb 2024 13:19:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rexMa-0006ei-DD
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:19:16 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cd48ec8c-d572-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 14:19:14 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a3f5808b0dfso582998366b.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 05:19:14 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 cw6-20020a170907160600b00a43a79b10absm666563ejd.140.2024.02.27.05.19.13
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 27 Feb 2024 05:19:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cd48ec8c-d572-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709039954; x=1709644754; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=Fgopwm/HIJBjbpnU+nFyXJh5v7ox1imyRzjBVkdWtSA=;
        b=TomO2vR41/hsIJht/AmojO2RqtHZgjjUkwN85jLhRwARdV+l+jeVvsUCn+Gia8UcgJ
         AwKIYr77EFWmGCKRVksl4QYgyXegmcnPdHR+gq5IfXBwpOicxXW3M/smIftuwy3mcmD8
         kCRynf6hH+VY1O7MeCvany3Vp37P2vpzJHWmZZjp2k5Kn8CizoBCosjfAw4vow0o/k+0
         aOCoIpu1cdv0xuoapT4kAeLKGTtTe71bETmCemOicHkUQVkaCmPcCVgo07xp+b9kfCep
         b9NXRYp4IPVNi4kor/7CGLcaj2IhiQAxzQFd7C14f5KlmbaHJ9YTS9vGIo4qUL2h1p5w
         FObQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709039954; x=1709644754;
        h=content-transfer-encoding:autocrypt:content-language:cc:to:subject
         :from:user-agent:mime-version:date:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Fgopwm/HIJBjbpnU+nFyXJh5v7ox1imyRzjBVkdWtSA=;
        b=vTMNocMZcUSdJVLYBTygrU61RredFmJu7mfrfhnZYJbH3sDZFdwAW9/W2d6Y6mzFVr
         xXxeI3Dz0eeDlFvs27YKA/2Xfxu2uafY8o6UR7vsRahhgeJheAC4ZxDPS9KvsUyU/8KH
         AdfCbT46tKxOzWcU37op0COOFjrVW3xyiRrd0M9im3gJl/Nu9H8kI/TczZ64R1K6JX8c
         9fDH1Hj7/F+9hZ/JPoZkv8+DINs1e3aLxo0/Y/ZuiUlFVlhiwOEzw95gwHSCHz1ixHj2
         qsWwgOj+WiHOXmNZV6SAB2AMdmG9davsj8j67LLfUgW/BSbj+OR4oTskM0lM9lxEGorY
         5FFA==
X-Gm-Message-State: AOJu0YxDx4i3AxzZo0e4gkp/2bPRfKHW9O+xZkfa73wKrQn8X5J0GBQk
	YVoGou+j738cgGU1KEuJRvp5FfIcHcY371ui/5hNRnvfs35bTIkwRk2l0JDjmX/2aBqQYQzj1kY
	=
X-Google-Smtp-Source: AGHT+IEczKgYAh3hzFqIIYSJW3Z4MQ/vNOY15LI8GlZpCfTM6l2S8A22UAf2XXxbPMtZ08HUJS4Hmg==
X-Received: by 2002:a17:906:3590:b0:a43:4156:96d2 with SMTP id o16-20020a170906359000b00a43415696d2mr4316545ejb.27.1709039953884;
        Tue, 27 Feb 2024 05:19:13 -0800 (PST)
Message-ID: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com>
Date: Tue, 27 Feb 2024 14:19:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Jan Beulich <jbeulich@suse.com>
Subject: preparations for 4.18.1
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Anthony Perard <anthony.perard@citrix.com>, Julien Grall <julien@xen.org>,
 Kelly Choi <kelly.choi@cloud.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

All,

the release is due in two to three weeks. Please point out backports you find
missing from the respective staging branch, but which you consider relevant.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:24:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:24:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686165.1067815 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexR8-0000yg-GL; Tue, 27 Feb 2024 13:23:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686165.1067815; Tue, 27 Feb 2024 13:23:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexR8-0000yZ-Ci; Tue, 27 Feb 2024 13:23:58 +0000
Received: by outflank-mailman (input) for mailman id 686165;
 Tue, 27 Feb 2024 13:23:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rexR7-0000yG-Fs
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:23:57 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7566d18a-d573-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 14:23:56 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a3e75e30d36so790477066b.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 05:23:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a21-20020a170906191500b00a413d1eda1bsm764840eje.49.2024.02.27.05.23.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 27 Feb 2024 05:23:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7566d18a-d573-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709040236; x=1709645036; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=jRT+YB1dAgC5Diu8qeFxjOe6cGLNCgY1w7oc1ceuW84=;
        b=W3FUAY7ZHXFbcVfwDwtR3IeNrHYOpVpN4CLbbBhLjU2dH+wm+5KI0kbLrFsJiM9kPO
         eitW/RElpWXqCY6YCYYQDlm029rlR9xreKLKXk+2VhxPdNdCT+L5JtFDlmAupt56xeL3
         LySIQmcS/OSKzBYJhi7qYNfBYpexCMj+wdebJdJpTL+7iP9E7IQ5zrdB/eNtcFARBnqE
         hW5/7LWcqcgjr34KxbDO5LSnzvPtZjDoztEt8v1ZYaYk5GRic+d8tLTE10lfOnSoEbj9
         /TEIcZ8IYOzZRoXt+c1M/hAD0Msk9MS3zfHe4yYB5p7WjLVko7WRstqqDNoVr0vlbQCL
         TdxQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709040236; x=1709645036;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=jRT+YB1dAgC5Diu8qeFxjOe6cGLNCgY1w7oc1ceuW84=;
        b=KRepukMtdGJFgW+V+Wrmu2X0r4IghTmSO6T1OoMqFXcxBB4BEolkU1MoBKLo/ykWvD
         U5HwpEjro/Eq3OkZjKtW0l0F41X6paQuhS80/1W3/VzpD4VdQU8wY/CiRTTnyzA3GvJe
         Ry9h+QLw496ad0lxejQHzvIu0CLDFnlcSHTH2TocZOAmlxJdK/jXo+ICgMt37akIhi/o
         2Y4pmRvBwO3y+i/icgUaoC98fK5TKZCWNh8zvE/aVwNRDZKZeIFswvvuUaxGWMeOmgwU
         smIVca03T3sobjDnjFCLoGEpxi/o0jh2MQDjIb83stqE1Gx47N73LtsWNkDJSmlTed2e
         e0CQ==
X-Forwarded-Encrypted: i=1; AJvYcCWbIIFTf4RiO0BPcG1jXxDWjTdTAttd2ksSmMXZdXPS/TPwGOo0PzCFuf2y77RCxNMYbirjt9QEHy5lt4FQvaUq3rxiiYGxjVOlWcPPHmk=
X-Gm-Message-State: AOJu0Yx44RCVR8hnYR/Ddccl57P+73hi4Ic4c5Pr70coA09dyYcix8C6
	F3Yz2fGpymNClaly5ZKNy0vuLA+5SyuGOlkSaFoXq0D1qCUyAGW9iX8Ivfyrhw==
X-Google-Smtp-Source: AGHT+IFocQw1h4E6mhak6vG9tpYvmwnht6yfbGkoFpEktBxn4s/YEVlTlHL+3YoFK8NTGZMq7EBQDQ==
X-Received: by 2002:a17:906:52cd:b0:a3f:c32:7b0c with SMTP id w13-20020a17090652cd00b00a3f0c327b0cmr9382909ejn.22.1709040235951;
        Tue, 27 Feb 2024 05:23:55 -0800 (PST)
Message-ID: <04a8bc24-0cb9-4eb9-9d68-e1e1defaaba4@suse.com>
Date: Tue, 27 Feb 2024 14:23:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 4/4] x86/spec: do not print thunk option selection if
 not built-in
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240226110748.80254-1-roger.pau@citrix.com>
 <20240226110748.80254-5-roger.pau@citrix.com>
 <30e1bfea-d76b-4e3d-b148-f25d1c811f8d@suse.com> <Zd3Fl6nx-R4aq2wR@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zd3Fl6nx-R4aq2wR@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27.02.2024 12:20, Roger Pau Monné wrote:
> On Mon, Feb 26, 2024 at 01:54:54PM +0100, Jan Beulich wrote:
>> On 26.02.2024 12:07, Roger Pau Monne wrote:
>>> Now that the thunk built-in enable is printed as part of the "Compiled-in
>>> support:" line, avoid printing anything in "Xen settings:" if the thunk is
>>> disabled at build time.
>>
>> Why "Now that ..."? It's other logging the earlier patch adds there.
> 
> 'Now that ...'  would refer to patch 1/4.

But that's why I'm asking: Patch 1 does not add any printing for thunk
usage; it extends that and shadow-paging's printing by the three
hardening ones. But anyway ...

>>> Note the BTI-Thunk option printing is also adjusted to print a colon in the
>>> same way the other options on the line do.
>>>
>>> Requested-by: Jan Beulich <jbeulich@suse.com>
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>
>> With either a clarification of what's meant or e.g. s/Now that/Since/
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> 'Since' is likely more appropriate.

... I'll switch to this then.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:24:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:24:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686170.1067837 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexRe-0001b5-TZ; Tue, 27 Feb 2024 13:24:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686170.1067837; Tue, 27 Feb 2024 13:24:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexRe-0001au-QZ; Tue, 27 Feb 2024 13:24:30 +0000
Received: by outflank-mailman (input) for mailman id 686170;
 Tue, 27 Feb 2024 13:24:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fto1=KE=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1rexRd-0001aI-KX
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:24:29 +0000
Received: from NAM04-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2408::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 878b1bf5-d573-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 14:24:27 +0100 (CET)
Received: from BYAPR06CA0012.namprd06.prod.outlook.com (2603:10b6:a03:d4::25)
 by IA0PR12MB8304.namprd12.prod.outlook.com (2603:10b6:208:3dc::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.36; Tue, 27 Feb
 2024 13:24:22 +0000
Received: from SJ1PEPF00001CE9.namprd03.prod.outlook.com
 (2603:10b6:a03:d4:cafe::ce) by BYAPR06CA0012.outlook.office365.com
 (2603:10b6:a03:d4::25) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend
 Transport; Tue, 27 Feb 2024 13:24:22 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SJ1PEPF00001CE9.mail.protection.outlook.com (10.167.242.25) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 27 Feb 2024 13:24:22 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 07:24:21 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 07:24:21 -0600
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 27 Feb 2024 07:24:18 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 878b1bf5-d573-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jyLxjzTldQQEazV08EdsnjoNw4HzhX+HfsWhaavy281yB5GWgLU3ZqDqk3Vr38rAwZfjA4M9p+VjGjtRytRgKrFTZLxiYViaxGYS/YxA8t2UVotzyua6DsZuTvtR5qYc3XWD+fb/ieJBrzYYr2McS42tp7BrB1yuWd49X2qiW3wLAhfL8AR571z12dUvqFeqVkrosxtX1PBmlU8SV33Wd6VZ47LKYaUvqP7q4axhG0tQaW/Dekhfb61Bdzy2rhG282cE6g1AvEPSBw+dhE3XrteNJ0djJPxMHvX+IgcZS5qcYd29Qh3YzZxZnitujVc8sTk73AChEoij1fYO/zfWlg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=PrzLqgCioMChk63CLo+D5m7J1D016EnpPjTTKejBi8I=;
 b=ZEEk5+SXQh0GKSrPqwkpbGPlJjriquDUf89VusHnZe/1nEsUAVYmYNIo9idvC9eQIbX0MhB8xhvNzOzh4sJBPgfuPyeRZaEczwgURLHjyB8gOT3Z6YR1lBFBj88IpwKoCHGsx+9nVubvBGuRHuJKqD75CbQ6j8OglBUGJDdJYJUTvlQImtvVKDfGcBQgB0+kLaRDrjwUzEGgXBSlfOiMXn0CovJU5I0SZ6zLHOYQ/tOohVXG9CiSzno3L3Ad6zyjkVTDFmxomq42KURLkYhemOK9so+PZR4MP5ZOT3IGwq0IfX+c1OkoLYsrKeQNhu5Er0khQkaqdmPtfnA+C+kwYA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PrzLqgCioMChk63CLo+D5m7J1D016EnpPjTTKejBi8I=;
 b=MZMGGWHzZWxNe9gkwbRHBlE7UwEkE0MwnZE1Vkj6wfsMxnQap5SNS+u7ZtFcRdW2w0b+ebXJNhksWBKOQNg9hVunCg36R6ybnVWAEP0Y0bv7qoE86PUjtmf4jOdHkMjxBkp1DXW2PEhnRgXd6SyY+FZ1jAI/bSebHLiQRdKckQI=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <c3edf84d-3177-4125-bdd6-ba99190340a3@amd.com>
Date: Tue, 27 Feb 2024 21:24:12 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
CC: Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Alec Kwapis
	<alec.kwapis@medtronic.com>, <xen-devel@lists.xenproject.org>
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <b41daccb-d402-4da8-bc88-933facd088a8@suse.com>
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <b41daccb-d402-4da8-bc88-933facd088a8@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CE9:EE_|IA0PR12MB8304:EE_
X-MS-Office365-Filtering-Correlation-Id: 13af9ca9-5b1c-4ced-2cd9-08dc3797691c
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	bt6SAEwumedJDIynDD0jkCObIsEeoslWjm15FUz9Fz16UFhUlFqyrSaRMhmjrDp0AzATermYRyjOI+mXGmI5AE4CP4eKXYzpU39e6CHwo7RIzSToJsrzA0Nj5c3BOPfqNagFE/pRPrEs93ETGjU1YDeloRtuMCqPk1XSRRIhEGXkHm+uN3n1xXrbZvJ5lEEt/Y2/iGiEBpPw+obpVMtdkE+gcMe/EyecuVnd9rrH8+wMCIvk9jPdmP1o5uz0Gb6xYp7LOx+B735RPjHMtdQPRugSv/0EDf9UaT8mxNOjCMgWa3SQ77ezLNvTMfxxsmKMPdABOI+h9NRABBl88YOAEA9DbvZraQPw2TBReQrznh3PRVmXGqLjHGp6ajmf/6rRRhXzfvNCA9jg1sc7DkvBW/aDVQ9YCWWsAvad/nbMxRzvSUBST2lOaSrNav9f3USIhI9PkNFLnM8a6I8LjgrM8+R+YUL+7ZimeSnutyY7sR5Kmxhr/f5x4RYwuJ3VdyT1u0kJ0llvYiM8HgdvJhbPRZXwTIS7b92sMf9tcxknotHgneeOf4Z/tpTluFm0wrn+6eFDjRVnk6kK7Frk0uo0LEhzwbeketXxvu3tgpulypIomN1AElCADuZ/DSc2gpVcD8F1zHWrYKBn93PxsbJS12OiCZBjSLTpCteee9y1cQhAgtSNkPxd8tGR9C7tHriyfsyWLqE4cpVfxtfOvf9ka5SAOl1WskpgEIcXBz9mpQKKfvICD9wtWDXfZf7a/fmK
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 13:24:22.1561
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 13af9ca9-5b1c-4ced-2cd9-08dc3797691c
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CE9.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8304

Hi Jan,

On 2/26/2024 4:25 PM, Jan Beulich wrote:
> On 26.02.2024 02:19, Henry Wang wrote:
>> --- a/xen/arch/arm/include/asm/mm.h
>> +++ b/xen/arch/arm/include/asm/mm.h
>> @@ -428,6 +428,19 @@ static inline void page_set_xenheap_gfn(struct page_info *p, gfn_t gfn)
>>       } while ( (y = cmpxchg(&p->u.inuse.type_info, x, nx)) != x );
>>   }
>>   
>> +#define MAGIC_PAGE_N_GPFN(n)     ((GUEST_MAGIC_BASE >> PAGE_SHIFT) + n)
>> +static inline bool is_magic_gpfn(xen_pfn_t gpfn)
>> +{
>> +    unsigned int i;
>> +    for ( i = 0; i < NR_MAGIC_PAGES; i++ )
> Nit: Blank line please between declaration(s) and statement(s).

Thanks, will correct in next version.

>> --- a/xen/arch/x86/include/asm/mm.h
>> +++ b/xen/arch/x86/include/asm/mm.h
>> @@ -628,4 +628,9 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
>>       return (mfn + nr) <= (virt_to_mfn(eva - 1) + 1);
>>   }
>>   
>> +static inline bool is_magic_gpfn(xen_pfn_t gpfn)
>> +{
>> +    return false;
>> +}
> I don't think every arch should need to gain such a dummy function.

Thanks for raising the concern here and about the 
is_domain_direct_mapped(), I will try to do some clean-ups if necessary 
in next version.

> Plus
> the function name doesn't clarify at all what kind of "magic" this is
> about. Plus I think the (being phased out) term "gpfn" would better not
> be used in new functions anymore. Instead type-safe gfn_t would likely
> better be used as parameter type.

Sure, I will use gfn_t in the next version.

>> --- a/xen/common/memory.c
>> +++ b/xen/common/memory.c
>> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>>           }
>>           else
>>           {
>> -            if ( is_domain_direct_mapped(d) )
>> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
>>               {
>>                   mfn = _mfn(gpfn);
>>   
> I wonder whether is_domain_direct_mapped() shouldn't either be cloned
> into e.g. is_gfn_direct_mapped(d, gfn), or be adjusted in-place to gain
> such a (then optional) 2nd parameter. (Of course there again shouldn't be
> a need for every domain to define a stub is_domain_direct_mapped() - if
> not defined by an arch header, the stub can be supplied in a single
> central place.)

Same here, it looks like you prefer the centralized 
is_domain_direct_mapped() now, as we are having more archs. I can do the 
clean-up when sending v2. Just out of curiosity, do you think it is a 
good practice to place the is_domain_direct_mapped() implementation in 
xen/domain.h with proper arch #ifdefs? If not do you have any better 
ideas? Thanks!

>> --- a/xen/include/public/arch-arm.h
>> +++ b/xen/include/public/arch-arm.h
>> @@ -476,6 +476,12 @@ typedef uint64_t xen_callback_t;
>>   #define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
>>   #define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
>>   
>> +#define NR_MAGIC_PAGES 4
>> +#define CONSOLE_PFN_OFFSET 0
>> +#define XENSTORE_PFN_OFFSET 1
>> +#define MEMACCESS_PFN_OFFSET 2
>> +#define VUART_PFN_OFFSET 3
>> +
>>   #define GUEST_RAM_BANKS   2
> Of these only NR_MAGIC_PAGES is really used in Xen, afaics.
> Also while this is added to a tools-only section, I'm also concerned of
> the ongoing additions here without suitable XEN_ prefixes. Any number
> of kinds of magic pages may exist for other reasons in a platform; which
> ones are meant would therefore better be sufficiently clear from the
> identifier used.

Yes you are correct, like I replied in another thread, I will undo the 
changes in next version.

Kind regards,
Henry
> Jan



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:27:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:27:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686176.1067846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexUI-0002Do-98; Tue, 27 Feb 2024 13:27:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686176.1067846; Tue, 27 Feb 2024 13:27:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexUI-0002Dh-6S; Tue, 27 Feb 2024 13:27:14 +0000
Received: by outflank-mailman (input) for mailman id 686176;
 Tue, 27 Feb 2024 13:27:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rexUH-0002DZ-Sj
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:27:13 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e9f04651-d573-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 14:27:11 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a43488745bcso326168766b.3
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 05:27:11 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 kf5-20020a17090776c500b00a438d594d3esm760275ejc.217.2024.02.27.05.27.10
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 27 Feb 2024 05:27:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e9f04651-d573-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709040431; x=1709645231; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=1PP71nXAl22kstrYPkbKOLLUjfFtEG8yr+uCm8pr9jg=;
        b=NNKXUPLQSgoWE2ZkgTHAwrz/W8+f/vxRGkHfpiklHuqZDK+57vRQQnDtYa8PbQm5qA
         lbPTA3E+Bgb0VIy4hg5Y+DnAWvd8hm5Njodncp91CylvfsNij1OVsrbhVFmp8x9gSGnf
         Lm4+q/hi0U4XtnwGvnhUrQCPPFD1+jireJrwV4REsCq9LeZzpKOPeQeLsNXWfklK2BSi
         5qw2g3aCLrevrHVqaGQdtB3tyNsUSzU8xpNpnu2zK+yBX2Oa7Qt8fz6GPZ8xyZtZjNL9
         kUuzP4ttZnlgtzIwfhy92bV4tJ6qmply6sD7Gqba9ZZ6AzDxAlXr2/3jqMLBMGc18ZFv
         BAwQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709040431; x=1709645231;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=1PP71nXAl22kstrYPkbKOLLUjfFtEG8yr+uCm8pr9jg=;
        b=NOIzKpbUcwMtLr78rKl3bRB6CTx8N/1h0Sq7AhZUzkGby5B7IzxTSmvLEmBht3T4YB
         DpgtBsXIUb3nVnGXTdRWiqd8+M+xYwQRY2c1QQTv5rUs/sRGI+h8SmUd/bognbGrM38j
         JU0G4qqH8HgRovvtaBtpX7cKu4G/kQJ5vWML+qdgk4tgUp/qo2J8L9AKojloEps3bkXk
         OMGTAuEI6VbX54NU0b4fSEtHbdQUkmLssSuC+i5sXTKrDF+ML+nyvNC9uyA/meo/W1uy
         0P/XaVQ6v7J4Z8c74/2+UjJu6NIuy3+ia6Llh2y5/Z9YXe7ESeYZ5pN0L2OcmyxlYzh+
         GoDQ==
X-Forwarded-Encrypted: i=1; AJvYcCUbU0/rkHWal62EAYYbkh1u/eFLamsBGHM4CFNKN9tUzl3+xwog3hcYFO1k81A9twgoC/SFCe23KMpdvqcBdqeFlmXnJhQ5GelPDgQqI04=
X-Gm-Message-State: AOJu0Yw0k9slmqniEYv6/iKmGySuvc22UOIgjM8KjNn5DDM5uBbLDpVL
	9JdyA4UO5MsqWDsAVTk1b9NoGAcqfpBrikbuAHRdehD2++jHdAbVDWqC1PVdQQ==
X-Google-Smtp-Source: AGHT+IFAmF91kqqVuwaegLa4ryhgF0hkMr/LeUACD6mvMV7cBBgVve8H4LrMQTrRp6JxtJYFFl/F3Q==
X-Received: by 2002:a17:906:80d4:b0:a43:10b6:40f6 with SMTP id a20-20020a17090680d400b00a4310b640f6mr4603307ejx.1.1709040431432;
        Tue, 27 Feb 2024 05:27:11 -0800 (PST)
Message-ID: <23dc102e-e827-4f1e-876d-76a7df4c4927@suse.com>
Date: Tue, 27 Feb 2024 14:27:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
Content-Language: en-US
To: Henry Wang <xin.wang2@amd.com>
Cc: Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>, xen-devel@lists.xenproject.org
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <b41daccb-d402-4da8-bc88-933facd088a8@suse.com>
 <c3edf84d-3177-4125-bdd6-ba99190340a3@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <c3edf84d-3177-4125-bdd6-ba99190340a3@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.02.2024 14:24, Henry Wang wrote:
> On 2/26/2024 4:25 PM, Jan Beulich wrote:
>> On 26.02.2024 02:19, Henry Wang wrote:
>>> --- a/xen/common/memory.c
>>> +++ b/xen/common/memory.c
>>> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>>>           }
>>>           else
>>>           {
>>> -            if ( is_domain_direct_mapped(d) )
>>> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
>>>               {
>>>                   mfn = _mfn(gpfn);
>>>   
>> I wonder whether is_domain_direct_mapped() shouldn't either be cloned
>> into e.g. is_gfn_direct_mapped(d, gfn), or be adjusted in-place to gain
>> such a (then optional) 2nd parameter. (Of course there again shouldn't be
>> a need for every domain to define a stub is_domain_direct_mapped() - if
>> not defined by an arch header, the stub can be supplied in a single
>> central place.)
> 
> Same here, it looks like you prefer the centralized 
> is_domain_direct_mapped() now, as we are having more archs. I can do the 
> clean-up when sending v2. Just out of curiosity, do you think it is a 
> good practice to place the is_domain_direct_mapped() implementation in 
> xen/domain.h with proper arch #ifdefs?

arch #ifdefs? I'd like to avoid such, if at all possible. Generic fallbacks
generally ought to key their conditionals to the very identifier not
(already) being defined.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:27:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:27:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686179.1067857 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexUh-0002iR-Ky; Tue, 27 Feb 2024 13:27:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686179.1067857; Tue, 27 Feb 2024 13:27:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexUh-0002iK-Ho; Tue, 27 Feb 2024 13:27:39 +0000
Received: by outflank-mailman (input) for mailman id 686179;
 Tue, 27 Feb 2024 13:27:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fto1=KE=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1rexUg-0002bP-9N
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:27:38 +0000
Received: from NAM04-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam04on20600.outbound.protection.outlook.com
 [2a01:111:f403:2408::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f8b6cca6-d573-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 14:27:37 +0100 (CET)
Received: from MW4P221CA0013.NAMP221.PROD.OUTLOOK.COM (2603:10b6:303:8b::18)
 by BL1PR12MB5126.namprd12.prod.outlook.com (2603:10b6:208:312::8) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Tue, 27 Feb
 2024 13:27:34 +0000
Received: from MWH0EPF000989E7.namprd02.prod.outlook.com
 (2603:10b6:303:8b:cafe::7b) by MW4P221CA0013.outlook.office365.com
 (2603:10b6:303:8b::18) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.26 via Frontend
 Transport; Tue, 27 Feb 2024 13:27:33 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 MWH0EPF000989E7.mail.protection.outlook.com (10.167.241.134) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 27 Feb 2024 13:27:33 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 07:27:31 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 07:27:31 -0600
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 27 Feb 2024 07:27:28 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f8b6cca6-d573-11ee-afd6-a90da7624cb6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ilBTWxFnGU2PV0RgxG/LfsGp3Gr7LHnLdMcXAI+abyI4Z/13deZjwvpmrrjSr7gGM6lEgi1kj53gjn9vgRnhqM6hnbycgVXJQEuOZVaXSlvYgfVALOM5Mo02s1iY+l2F2S1Zl+/mrIwEhwa8UAblFbMdAdMQINcDWWGGS1HRkjMYmYv5XtXQdVu40d7c7P4/YpNVfN1rSeoPa7MrZ0AMapwFCOpIZXRkc+UE58c+1Tte1oZjKvKLWnNCoyHSsP48wGEOyFPZZ8XVwpwuIYULn7i5ZRoXJKoXWzszeyMurFIZ+gASMLufPkR1V2NSN1NwQIuSWAfDS2cQbuplFFTPSg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=7fG32DN/8O+e4OkIL0FSBLZGXZRyL1cyzCIz9t1QoEI=;
 b=cpqgITKuYRJKWCFU0MKAFUg4koIOkp7hU2Z1xfxG5GUyV+N51p20Hj24HNE8dtMXfvNufoHCzAURPQ884IThZ+3pP0gJAiWf0ND6ikg1Y2IffdqhU2P7PSy8PMMVZL2XkGUp8wNiJBo/hD2L7zClgFP2gD48/G+6MvKqYTy+xxpLowLf2+18wMNU0aN7g5a3O9T2GrNLfMsXkyPN/bzvwXD6+MzrKU9QVTKbgp7+M5DLpMQVET5yVDQIOdRnSR+VeesL0+5IZzJD9u6O5f9Fhj9/y8ZgNsIzDsfVPILZ9+DeLI7RfadPdzdVfjr8e409Rdwb5TbBwTSdjkczLHSfVg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7fG32DN/8O+e4OkIL0FSBLZGXZRyL1cyzCIz9t1QoEI=;
 b=P7gPVF5fy0s2VcjdKs/qONxdbbSpTJr2s8Ur6Va+cu+WRCmhdIhK/3JWZH6iVKbT+k4UL5IUuZJ5C1LzLUBA8YojO9PKpWxgxelGUF5Dm8/KbFmghynjcSIKTPet8gEPjN04pIc97j5VJzWhntmJDtjWyvi0oF1hAHj1XCJoLTo=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <f3ff21bc-55f9-465d-95a2-269101b3d71e@amd.com>
Date: Tue, 27 Feb 2024 21:27:27 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
Content-Language: en-US
To: Michal Orzel <michal.orzel@amd.com>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Julien Grall
	<julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, "Bertrand
 Marquis" <bertrand.marquis@arm.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Alec Kwapis <alec.kwapis@medtronic.com>
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <4c9bf719-7ad9-4739-9edc-eef06dcefb9f@amd.com>
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <4c9bf719-7ad9-4739-9edc-eef06dcefb9f@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: MWH0EPF000989E7:EE_|BL1PR12MB5126:EE_
X-MS-Office365-Filtering-Correlation-Id: d31b6b61-7903-4b56-d62d-08dc3797db16
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	xa6KAPn2iQsh92daJHsl0ArZT3UqkL15Gc4AFfdG+n3BoYn2kWIH9VSAul+u7lRagmglkECd6kjcHCltcC1dDULuA/6dNKgTDNh9UtbD8NL3plrmPXr6nPnAeSoWATUyjLw686oYb3h0uuH7cXbtDyt2YNBLFEJA85TJ74pKfc6W69r+7TgOiElXEGcmwYA7muViB2YDG8pgDb2aghCAw/mXTVHRestDewgeekG6L/kUP6WlcQlooPEbepmASqY3bpKA3NpDqe7VzRM0Nczrz3Y70LUEhMcjbuEIstUzs5ys/WfgKoPSJNpJlVBB5sFeHiiT51LIE7gTZq8ffFmu/4FZOub81fUvV8pe7Pm9QeOsxF4saXElQE3qij9rRehh5qwqqkc724u+Qpxg348d3UzpZF1vb/eXyR6g6Ii2NXYD/8/i9NGbzqltA6e4d7kSQB1T9yyaPjA+L5Ou6DbH6QNfJw13GJJmq5L4yBof8RjpbqHTcYXC0QOeB3hM86oA6iov15+iWSXI/w1hkNoeDiBdCSvMulhTtjhP02+M9t7ngW+b8p1fRp9rez6IKxYnWYZuam3MgCZB84ImSLTXye9tQUvMsfjEqOUtIZeFtCW6y5kt/UOM/GaZK7aauQh1uou+PquaobcqT9sj1yNgMyYemytasmBCoorhRnWL10VEzbmx+8CAJGqo1q3NnFED6uKK7x/jqNivG8yvhyKhmgie9ky22axx/kGd6WjBvT7lhEnU4UrEKqgbgGfuSI6L
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400014)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 13:27:33.3345
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d31b6b61-7903-4b56-d62d-08dc3797db16
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	MWH0EPF000989E7.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5126

Hi Michal,

On 2/26/2024 6:29 PM, Michal Orzel wrote:
> Hi Henry,
>
> On 26/02/2024 02:19, Henry Wang wrote:
>> An error message can seen from the init-dom0less application on
>> direct-mapped 1:1 domains:
>> ```
>> Allocating magic pages
>> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
>> Error on alloc magic pages
>> ```
>>
>> This is because populate_physmap() automatically assumes gfn == mfn
>> for direct mapped domains. This cannot be true for the magic pages
>> that are allocated later for Dom0less DomUs from the init-dom0less
>> helper application executed in Dom0.
>>
>> Force populate_physmap to take the "normal" memory allocation route for
>> the magic pages even for 1:1 Dom0less DomUs. This should work as long
>> as the 1:1 Dom0less DomU doesn't have anything else mapped at the same
>> guest address as the magic pages:
>> - gfn 0x39000 address 0x39000000
>> - gfn 0x39001 address 0x39001000
>> - gfn 0x39002 address 0x39002000
>> - gfn 0x39003 address 0x39003000
>> Create helper is_magic_gpfn() for Arm to assist this and stub helpers
>> for non-Arm architectures to avoid #ifdef. Move the definition of the
>> magic pages on Arm to a more common place.
>>
>> Note that the init-dom0less application of the diffenent Xen version
>> may allocate all or part of four magic pages for each DomU.
>>
>> Reported-by: Alec Kwapis <alec.kwapis@medtronic.com>
>> Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
>> Signed-off-by: Henry Wang <xin.wang2@amd.com>
> NIT: Generally, the first SOB is the same as author of the patch.

I will fix in the next version. Thanks.

> [...]
>> diff --git a/xen/common/memory.c b/xen/common/memory.c
>> index b3b05c2ec0..ab4bad79e2 100644
>> --- a/xen/common/memory.c
>> +++ b/xen/common/memory.c
>> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>>           }
>>           else
>>           {
>> -            if ( is_domain_direct_mapped(d) )
>> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
> I struggle to understand the goal of this patch and the proposed solution.
> The problem with magic pages applies to static mem domUs in general.
> A direct mapped domU is a static mem domU whose memory is 1:1 mapped.
> Let's say we try to map a magic page for a direct mapped domU. That check will be false
> and the execution will move to the next one i.e. is_domain_using_staticmem(d).
> This check will be true and acquire_reserved_page() will fail instead (similar to the
> static mem (no direct map) scenario). The only thing that has changed is the message.

Yes you are definitely correct, I missed the check for static mem 
domains. Will fix in v2 once the proposal in the other thread is agreed. 
Thanks for spotting this issue!

Kind regards,
Henry

> ~Michal



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:29:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:29:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686181.1067866 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexWF-0003Qd-Ul; Tue, 27 Feb 2024 13:29:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686181.1067866; Tue, 27 Feb 2024 13:29:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexWF-0003QW-Re; Tue, 27 Feb 2024 13:29:15 +0000
Received: by outflank-mailman (input) for mailman id 686181;
 Tue, 27 Feb 2024 13:29:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=rG8r=KE=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1rexWE-0003ON-6T
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:29:14 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 31c49571-d574-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 14:29:12 +0100 (CET)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-55a5e7fa471so5747459a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 05:29:12 -0800 (PST)
Received: from fhilly.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 f8-20020a056402194800b005648d0eebdbsm739946edz.96.2024.02.27.05.29.10
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 05:29:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 31c49571-d574-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709040551; x=1709645351; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=O8OPP4zUnBp0qZtUz97nx+VGwT/g+6InQ+dRne4geAI=;
        b=cQA64swXFCmnMaXsAGqlVmEelAPV+1PFyN5xNK3QrSqrAURKVe/QoreIkfhzwVVT8k
         A9L3qcPDgqviDFnuQcAxsloBILbNVAWaTlqNsYshp4E5jjn5/v40EpgY6stSNIgBMjgf
         +MzLIwMTmZLY+U9/qFiyPiaOQzFSCLoVUG7bE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709040551; x=1709645351;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=O8OPP4zUnBp0qZtUz97nx+VGwT/g+6InQ+dRne4geAI=;
        b=o1g8mzIjAyA6F0cBDdOVnYWM9wy8Mh6KC4Q1Z/swaRBP8HTm0jFNRIFqt61xd+s5HM
         +mw7ZUVRsBNdn183EBFLHgmVP8BQn/wND+IRo3vZ4XVBH/EVnpFu5kfFZeTvPokOQDM1
         v8qsG8+xnwOkrTIdRvV6p29LpyiPBJ1hHltqK534ycpvGrHEqnas9HXyajQH+8ib7Alw
         hGpc7lfth3d4iCX41ybBDn9cOHcvk4PuDOqc7L/+jah+ggOxcVeGqicz1ULtaIesJMGi
         v1xxYjA52kLrq/UsyS9w2KFuSnfaTu9fFlplaNP40T3qeDOLk98PBJtIxAY6E1ZsRNca
         MUVw==
X-Gm-Message-State: AOJu0YyO6rVwuMboUJcSrUUOVPkRhBNcloVIlgrw3htZVqXJXyDefE/3
	7qvzYqP9iMbLpTjO8brY+49eEZPKwP4hYOJgBGaANMosjImnZK2pji36L8LoSkH/NxFmpFt4IOO
	OYi4=
X-Google-Smtp-Source: AGHT+IGMGnZx1EQaEb4du/db3axhqXjvwFvuH3yFWHOii2P0hvfLukl9yJ5Lq4s3jlKQ94tRWdea2w==
X-Received: by 2002:a05:6402:612:b0:566:1fbd:22bb with SMTP id n18-20020a056402061200b005661fbd22bbmr2249049edv.9.1709040551191;
        Tue, 27 Feb 2024 05:29:11 -0800 (PST)
From: Fouad Hilly <fouad.hilly@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Fouad Hilly <fouad.hilly@cloud.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v2] tools/xentop: Add VBD3 support to xentop
Date: Tue, 27 Feb 2024 13:26:28 +0000
Message-ID: <20240227132628.2157031-1-fouad.hilly@cloud.com>
X-Mailer: git-send-email 2.42.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Pritha Srivastava <pritha.srivastava@citrix.com>

xl now knows how to drive tapdisk, so modify libxenstat to
understand vbd3 statistics.

Signed-off-by: Pritha Srivastava <pritha.srivastava@citrix.com>
Signed-off-by: Jorge Martin <jorge.martin@citrix.com>
Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Juergen Gross <jgross@suse.com>

v2:
- Fix order of SoB
- Fix Syntax
- Re-order free(path)
---
 tools/libs/stat/xenstat_linux.c | 65 ++++++++++++++++++++++++++++++++-
 tools/libs/stat/xenstat_priv.h  | 16 ++++++++
 tools/xentop/xentop.c           |  1 +
 3 files changed, 81 insertions(+), 1 deletion(-)

diff --git a/tools/libs/stat/xenstat_linux.c b/tools/libs/stat/xenstat_linux.c
index cbba54aa83ee..6d82e204aad4 100644
--- a/tools/libs/stat/xenstat_linux.c
+++ b/tools/libs/stat/xenstat_linux.c
@@ -390,6 +390,38 @@ void xenstat_uninit_networks(xenstat_handle * handle)
 		fclose(priv->procnetdev);
 }
 
+static int read_attributes_vbd3(char *vbd3_path, xenstat_vbd *vbd)
+{
+	FILE *fp;
+	struct vbd3_stats vbd3_stats;
+
+	fp = fopen(vbd3_path, "rb");
+
+	if (fp == NULL) {
+		return -1;
+	}
+
+	if (fread(&vbd3_stats, sizeof(struct vbd3_stats), 1, fp) != 1) {
+		fclose(fp);
+		return -1;
+	}
+
+	if (vbd3_stats.version != 1) {
+		fclose(fp);
+		return -1;
+	}
+
+	vbd->oo_reqs = vbd3_stats.oo_reqs;
+	vbd->rd_reqs = vbd3_stats.read_reqs_submitted;
+	vbd->rd_sects = vbd3_stats.read_sectors;
+	vbd->wr_reqs = vbd3_stats.write_reqs_submitted;
+	vbd->wr_sects = vbd3_stats.write_sectors;
+
+	fclose(fp);
+
+	return 0;
+}
+
 static int read_attributes_vbd(const char *vbd_directory, const char *what, char *ret, int cap)
 {
 	static char file_name[80];
@@ -438,7 +470,7 @@ int xenstat_collect_vbds(xenstat_node * node)
 		int ret;
 		char buf[256];
 
-		ret = sscanf(dp->d_name, "%3s-%u-%u", buf, &domid, &vbd.dev);
+		ret = sscanf(dp->d_name, "%255[^-]-%u-%u", buf, &domid, &vbd.dev);
 		if (ret != 3)
 			continue;
 		if (!(strstr(buf, "vbd")) && !(strstr(buf, "tap")))
@@ -448,6 +480,8 @@ int xenstat_collect_vbds(xenstat_node * node)
 			vbd.back_type = 1;
 		else if (strcmp(buf,"tap") == 0)
 			vbd.back_type = 2;
+		else if (strcmp(buf,"vbd3") == 0)
+			vbd.back_type = 3;
 		else
 			vbd.back_type = 0;
 
@@ -479,6 +513,35 @@ int xenstat_collect_vbds(xenstat_node * node)
 				vbd.error = 1;
 			}
 		}
+		else if (vbd.back_type == 3)
+		{
+			char *td3_pid;
+			char *path;
+
+			vbd.back_type = 3;
+			vbd.error = 0;
+
+			if (asprintf(&path, "/local/domain/0/backend/vbd3/%u/%u/kthread-pid", domid, vbd.dev) < 0)
+				continue;
+
+			td3_pid = xs_read(node->handle->xshandle, XBT_NULL, path, NULL);
+
+			free(path);
+
+			if (td3_pid == NULL)
+				continue;
+
+			if (asprintf(&path, "/dev/shm/td3-%s/vbd-%u-%u", td3_pid, domid, vbd.dev) < 0) {
+				free(td3_pid);
+				continue;
+			}
+
+			if (read_attributes_vbd3(path, &vbd) < 0)
+				vbd.error = 1;
+
+			free(td3_pid);
+			free(path);
+		}
 		else
 		{
 			vbd.error = 1;
diff --git a/tools/libs/stat/xenstat_priv.h b/tools/libs/stat/xenstat_priv.h
index 4eb44a8ebb84..c3a9635240e9 100644
--- a/tools/libs/stat/xenstat_priv.h
+++ b/tools/libs/stat/xenstat_priv.h
@@ -98,6 +98,22 @@ struct xenstat_vbd {
 	unsigned long long wr_sects;
 };
 
+struct vbd3_stats {
+	uint32_t version;
+	uint32_t __pad;
+	uint64_t oo_reqs;
+	uint64_t read_reqs_submitted;
+	uint64_t read_reqs_completed;
+	uint64_t read_sectors;
+	uint64_t read_total_ticks;
+	uint64_t write_reqs_submitted;
+	uint64_t write_reqs_completed;
+	uint64_t write_sectors;
+	uint64_t write_total_ticks;
+	uint64_t io_errors;
+	uint64_t flags;
+};
+
 extern int xenstat_collect_networks(xenstat_node * node);
 extern void xenstat_uninit_networks(xenstat_handle * handle);
 extern int xenstat_collect_vbds(xenstat_node * node);
diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
index 0a2fab7f15a3..f5a456fd4dfd 100644
--- a/tools/xentop/xentop.c
+++ b/tools/xentop/xentop.c
@@ -1124,6 +1124,7 @@ void do_vbd(xenstat_domain *domain)
 		"Unidentified",           /* number 0 */
 		"BlkBack",           /* number 1 */
 		"BlkTap",            /* number 2 */
+		"Tapdisk3"           /* number 3 */
 	};
 
 	num_vbds = xenstat_domain_num_vbds(domain);
-- 
2.42.0



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:32:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:32:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686183.1067877 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexZE-0005zi-Bd; Tue, 27 Feb 2024 13:32:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686183.1067877; Tue, 27 Feb 2024 13:32:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexZE-0005zb-8f; Tue, 27 Feb 2024 13:32:20 +0000
Received: by outflank-mailman (input) for mailman id 686183;
 Tue, 27 Feb 2024 13:32:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=FeT6=KE=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1rexZC-0005zV-SJ
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:32:18 +0000
Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com
 [2a00:1450:4864:20::22c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a063d41e-d574-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 14:32:17 +0100 (CET)
Received: by mail-lj1-x22c.google.com with SMTP id
 38308e7fff4ca-2d2ab9c5e83so1171611fa.2
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 05:32:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a063d41e-d574-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709040737; x=1709645537; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=wlfFfCKXh6eF1WcOCQlyirhSLscGtf3veXkT7yBHVjk=;
        b=NZ+XsX0CkIKLFloaTjKq8bK0rQUsHrdDwXp+183wEhoU9MmF3EdFCJsZFOB5QHlauB
         SnJVkQCgiXLnPp7rruacJKyPu2rKJUMnBqMdOaxYflVGfXkUNf9pWesRCHPaOd9GWBnV
         0WG92QnxCMT5NddlyzjEuqhe+ZxPI/dXBU9ng=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709040737; x=1709645537;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=wlfFfCKXh6eF1WcOCQlyirhSLscGtf3veXkT7yBHVjk=;
        b=ECe7CtlOB89nFP3NCMPD1W8MFiig0oWje7s0xSHCg23iAKUFZY/kJf/+dPEUxYQmCI
         S77h8iZsK2FMHrmA8792P+TgZkxN0luZqbQwMsqjOU1CDeqPETuZoYlT1Z1SN8GI7dg0
         8sMNV+WclJCtuhJG7DzLv0LdUVmSKNhIfXu6IojpoAPqiZJVb5ibfYceFZBHYgIt0F4D
         oBxt74ODht0o+ltOb0RXTg8/gg9QJr6ND/5iH1H4PNde9qmUGt1XU+O7wr4VOo1RUSFW
         awz/epj7L1O6D7o5BwgzNZpqr3HfmGJVj+bQ5sc9Z82pCC4JJ0ijV1Z17sBwDf2CGI8V
         rrOg==
X-Gm-Message-State: AOJu0YzkR8u9xc/LtJCVzVPT7Dbe0CkdQ769LGfNQ1INbk5ffNjGW9aT
	oh8i+IrCjKQZtUq9zFc3yK3ujDtsSM2s3E6CNfsvHKI6a5CDcgtOSMhMxnHA4Y3zWgGkJsiyD/0
	uxUdmrhYRNAlr03cMiiYFn3+uM8/1b45hoOqL
X-Google-Smtp-Source: AGHT+IERoOUncLhTrpIy++4f2blcDkUt2umlcfRtBoX9WwR/Q0vuRVOXt/pHKCIltgv2iWsIlsQ4nyyXkYc6o6qKQiM=
X-Received: by 2002:a2e:9881:0:b0:2d2:7392:5057 with SMTP id
 b1-20020a2e9881000000b002d273925057mr6004309ljj.30.1709040737512; Tue, 27 Feb
 2024 05:32:17 -0800 (PST)
MIME-Version: 1.0
References: <20240131165816.58762-1-roger.pau@citrix.com>
In-Reply-To: <20240131165816.58762-1-roger.pau@citrix.com>
From: Ross Lagerwall <ross.lagerwall@cloud.com>
Date: Tue, 27 Feb 2024 13:32:05 +0000
Message-ID: <CAG7k0EqfMecUGQJSqMLxmR7vOjushAYQuG-r+3eAhs4a-UhuTw@mail.gmail.com>
Subject: Re: [PATCH] livepatch-build-tools: allow patch file name sizes up to
 128 characters
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Jan 31, 2024 at 4:58=E2=80=AFPM Roger Pau Monne <roger.pau@citrix.c=
om> wrote:
>
> XenServer uses quite long Xen version names, and encode such in the livep=
atch
> filename, and it's currently running out of space in the file name.
>
> Bump max filename size to 128, so it also matches the patch name length i=
n the
> hypervisor interface.
>
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
>  livepatch-build | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/livepatch-build b/livepatch-build
> index cdb852cc7fea..78dd2d801048 100755
> --- a/livepatch-build
> +++ b/livepatch-build
> @@ -72,8 +72,8 @@ function make_patch_name()
>      fi
>
>      # Only allow alphanumerics and '_' and '-' in the patch name.  Every=
thing
> -    # else is replaced with '-'.  Truncate to 48 chars.
> -    echo ${PATCHNAME//[^a-zA-Z0-9_-]/-} |cut -c 1-48
> +    # else is replaced with '-'.  Truncate to 128 chars.
> +    echo ${PATCHNAME//[^a-zA-Z0-9_-]/-} |cut -c -128
>  }

I think it should be XEN_LIVEPATCH_NAME_SIZE - 1 to match the hypervisor
since AFAICT the hypervisor expects the last character to be NUL.

In get_name(), it checks:

    if ( !name->size || name->size > XEN_LIVEPATCH_NAME_SIZE )
        return -EINVAL;

... and:

    if ( n[name->size - 1] )
        return -EINVAL;

Do you agree with that?

Ross


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:36:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:36:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686185.1067887 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexcu-0006Zk-S3; Tue, 27 Feb 2024 13:36:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686185.1067887; Tue, 27 Feb 2024 13:36:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexcu-0006Zd-OI; Tue, 27 Feb 2024 13:36:08 +0000
Received: by outflank-mailman (input) for mailman id 686185;
 Tue, 27 Feb 2024 13:36:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fto1=KE=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1rexct-0006ZX-CN
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:36:07 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 26cda1ba-d575-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 14:36:04 +0100 (CET)
Received: from BN0PR02CA0004.namprd02.prod.outlook.com (2603:10b6:408:e4::9)
 by DS7PR12MB6288.namprd12.prod.outlook.com (2603:10b6:8:93::7) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7316.36; Tue, 27 Feb 2024 13:36:01 +0000
Received: from BN1PEPF0000468B.namprd05.prod.outlook.com
 (2603:10b6:408:e4:cafe::9f) by BN0PR02CA0004.outlook.office365.com
 (2603:10b6:408:e4::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.50 via Frontend
 Transport; Tue, 27 Feb 2024 13:36:00 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 BN1PEPF0000468B.mail.protection.outlook.com (10.167.243.136) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 27 Feb 2024 13:36:00 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 07:36:00 -0600
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 27 Feb 2024 07:35:57 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 26cda1ba-d575-11ee-afd6-a90da7624cb6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hIOhsWxuaW5JvRh8Bfubdk3s3KP+ONk6cOvYRDrEcEWG8prmJPIlqQCKc4wp5UrHJ4U50L++zBfAEZUcJTHVkTM/6lPyj0q/FV17R50UWdmzbdnRxD0C2p0nSEF3biMTB5GVaCQTtMIE+GWX0VZvQkIqLIhQGLPggwldC9ddD21e1IrrmldNKFhR3R7WRVwlXeg3eMGFYVS3k7JIfyIW3mc99TfqAoYFzKuHyHD837oQ9VcrHHHj5DWKKlr64/nmok1If7dbN8X/RcDs2jxork5CzR4Ot6KExKBFJt0susL0KfzjxeQRhekuEq22bK4i6Q7K/pintt5emi0+V/Hwtg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=zSQk1YnS2Ht4wfo1Q19hBrTpkLntDCtVYNW8+lCGlPA=;
 b=fS6ddMSZUW48YXNdQIzdcGf575I9qCqWh/s1Or+RoOhzQxFTUiTcHITNlfDPqpPIJZoeSCvPCeO6ZyU2uXKUdAZjafzAiWNmHtNBmrs73ZlpCsNIvl7QrxiJvmTPSNHBDK19Jfrk0jIEZFHPNnNNCxuX9ERxVOX/VXjUCTv6ix5z2NMeJVWsXUb7sREDvqxSh1sY9J5aENuStggq1EM2ZpxpJvy4a6Npe0KyKaY+rm2nDB3oYcDh0dthAsSEteCmJ53jvB8YgTn15aCuSvsH8p2YHi5E4Qm/A8iTtPWpuXdUAjLRcjAYo+dVppjVK2eS8upMGpxVlrR2ssp6/krySg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=zSQk1YnS2Ht4wfo1Q19hBrTpkLntDCtVYNW8+lCGlPA=;
 b=Jq2X5PgAioO1nQx/qYMS4LEqmjRcQKAZf/XR+0gBII7HzMkL7vLys/50D4aaC30nNJ5ul5wU5lJ09QiFa2ZMdeFTYxspK81RhAUfJGVrXANXnWY6yUOJLczwXK8T27HgnvG6ijLsRPzD5nDt5rntV6VSBoOtNRM8411JkgQKO40=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <ca749e1e-569e-4d7d-8009-63a9469d3ffd@amd.com>
Date: Tue, 27 Feb 2024 21:35:56 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
CC: Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
	"Michal Orzel" <michal.orzel@amd.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Alec Kwapis <alec.kwapis@medtronic.com>,
	<xen-devel@lists.xenproject.org>
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <b41daccb-d402-4da8-bc88-933facd088a8@suse.com>
 <c3edf84d-3177-4125-bdd6-ba99190340a3@amd.com>
 <23dc102e-e827-4f1e-876d-76a7df4c4927@suse.com>
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <23dc102e-e827-4f1e-876d-76a7df4c4927@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN1PEPF0000468B:EE_|DS7PR12MB6288:EE_
X-MS-Office365-Filtering-Correlation-Id: bbc0fff0-87e8-4a75-f8c1-08dc37990980
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yhBM9GpSEmjS6lv/8rTR7i8p8qqxlOGi7wANJeAO7ZUtvEyHplbuD+IYzGnX4eWVrUc6CDpxtUifdG5765vPVEW4DkHT4ReaLalxRkmHftkq8i/x+/yelX1rESvVDpB32tkNgU/bnoOXEfPOs+Z6N530BjRAO8ky6gqGJsWSulfvLSNnWgBnyZ86iYh7d+4nsubSF6X2djX/Rz9RqKF1LrDjkZzyicrdLyEkKm/ADP3PjKavl0o9G93mz62Z9+5XvSW/wLqAbIprUGLapKPGyCCWnGMDSiR/uJWTBHxh5AYZ/4FI/FeJ/6lIlpHwCR8jsdzhdQ/jst0HVcT27jD8EEeRjjmUpAalTWcnGUxEPbivSf+L0v4MRtndqQLjwfz1FT3EeSbtICAe4i9fNQUwG7JcXw/hAK54yFY1VEA1ZdlaXIYahbm33ClviHgtSfXYONzl6Pb9KhD7e2fbz96ef09cG0eOljrXb8KNfmSruxu681fnZUfmAbgN5qvmBatUmyX5vM139H9Msw8bUCavlv/oE8Geq3o+cZA+bNKh/LTiJxcEf8SDw/P1bkzelu3azDHtp2VMIBHXaHVbJZEwfXtLG5S4MducBP4FsKm3sPdsADZJkjh1I3xW275LHwv8cepspP2GXmxPdnD+TDsyRmd1R3qFP0pyjeh64E2qsyG7D2TaXpSVvsGwatMJjbD+YSIweoy+zUvtvNBdVS4UpLa9Kz2gWCMra+px4axbPzHI277JBHEylPIVrTmISszH
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400014)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 13:36:00.8270
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: bbc0fff0-87e8-4a75-f8c1-08dc37990980
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN1PEPF0000468B.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6288

Hi Jan,

On 2/27/2024 9:27 PM, Jan Beulich wrote:
> On 27.02.2024 14:24, Henry Wang wrote:
>> On 2/26/2024 4:25 PM, Jan Beulich wrote:
>>> On 26.02.2024 02:19, Henry Wang wrote:
>>>> --- a/xen/common/memory.c
>>>> +++ b/xen/common/memory.c
>>>> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>>>>            }
>>>>            else
>>>>            {
>>>> -            if ( is_domain_direct_mapped(d) )
>>>> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
>>>>                {
>>>>                    mfn = _mfn(gpfn);
>>>>    
>>> I wonder whether is_domain_direct_mapped() shouldn't either be cloned
>>> into e.g. is_gfn_direct_mapped(d, gfn), or be adjusted in-place to gain
>>> such a (then optional) 2nd parameter. (Of course there again shouldn't be
>>> a need for every domain to define a stub is_domain_direct_mapped() - if
>>> not defined by an arch header, the stub can be supplied in a single
>>> central place.)
>> Same here, it looks like you prefer the centralized
>> is_domain_direct_mapped() now, as we are having more archs. I can do the
>> clean-up when sending v2. Just out of curiosity, do you think it is a
>> good practice to place the is_domain_direct_mapped() implementation in
>> xen/domain.h with proper arch #ifdefs?
> arch #ifdefs? I'd like to avoid such, if at all possible. Generic fallbacks
> generally ought to key their conditionals to the very identifier not
> (already) being defined.

I meant something like this (as I saw CDF_directmap is currently gated 
in this way in xen/domain.h):

#ifdef CONFIG_ARM
#define is_domain_direct_mapped(d) ((d)->cdf & CDF_directmap)
#else
#define is_domain_direct_mapped(d) ((void)(d), 0)
#endif

I am having trouble to think of another way to keep the function in a 
centralized place while
avoiding the #ifdefs. Would you mind elaborating a bit? Thanks!

Kind regards,
Henry


> Jan



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:47:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:47:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686194.1067896 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexnx-00018W-Va; Tue, 27 Feb 2024 13:47:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686194.1067896; Tue, 27 Feb 2024 13:47:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexnx-00018P-SL; Tue, 27 Feb 2024 13:47:33 +0000
Received: by outflank-mailman (input) for mailman id 686194;
 Tue, 27 Feb 2024 13:47:32 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rexnw-00018D-Kr; Tue, 27 Feb 2024 13:47:32 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rexnw-00013I-Jm; Tue, 27 Feb 2024 13:47:32 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rexnw-00046S-61; Tue, 27 Feb 2024 13:47:32 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rexnw-000341-5C; Tue, 27 Feb 2024 13:47:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=7pNffAEBNzsODK8S/YWb2MkWiX3VcSoE2XyN9XQSAIg=; b=koa4xB5HAVxPVTo1wq5s0ezQdF
	N5wqL43/iRdGeSv5/nfjju5+BB5um7DDNOZpsuevgQR9IHf2TSR0+FrR06Yrc203xWI6EB6Rfm7co
	FBQZPO2eeu8ka3c8IbWJLj7WEcF1JJ+ms7UuwdUDohKj/i1T43H9t73Sw36aBjqwf9ak=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184781-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184781: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87
X-Osstest-Versions-That:
    xen=37541208f119a9c552c6c6c3246ea61be0d44035
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 13:47:32 +0000

flight 184781 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184781/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87
baseline version:
 xen                  37541208f119a9c552c6c6c3246ea61be0d44035

Last test of basis   184774  2024-02-26 22:00:26 Z    0 days
Testing same since   184781  2024-02-27 10:00:25 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
  Olaf Hering <olaf@aepfle.de>
  Roger Pau Monné <roger.pau@citrix.com>
  Shawn Anastasio <sanastasio@raptorengineering.com>
  Simone Ballarin <simone.ballarin@bugseng.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   37541208f1..ca7c872c70  ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 13:51:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 13:51:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686198.1067907 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexrX-0003AC-F7; Tue, 27 Feb 2024 13:51:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686198.1067907; Tue, 27 Feb 2024 13:51:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rexrX-0003A5-Bt; Tue, 27 Feb 2024 13:51:15 +0000
Received: by outflank-mailman (input) for mailman id 686198;
 Tue, 27 Feb 2024 13:51:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=HnYt=KE=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rexrW-00038n-3m
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 13:51:14 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 43d069d5-d577-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 14:51:11 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a3ed9cae56fso707621066b.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 05:51:11 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 qu24-20020a170907111800b00a3e559aaff9sm793030ejb.29.2024.02.27.05.51.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 27 Feb 2024 05:51:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 43d069d5-d577-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709041871; x=1709646671; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=Yok1+WIrlSpKjzylNjv4/P+Vhf513Inyq/FWByxKCmE=;
        b=Rii8jCE1bm5TUyf79sgOXULtgfNEFUexvRa4IIouvvEVQvMxpp+V1FQQnWGHfXG19C
         OlnCEw2u7avchLCEn23e5gdLzJT4iiOPqXriLdw8VbnXbI1mBCUqth1243c6LYOQfxYc
         jaLxDiqOBZ4jibrZ/zHz1bFEMjUaxbp0dxu4fayN2VI+Y6oEjhdSBhr0V/uL88+aC9KZ
         /4yp4RQCRPjbeVJB2qfl3Z8pW/uzEnkBuydBSXEAum9Doyg5p4+2RMzHs1NgrFIIrnRZ
         xi4DiMyw2P8VrBF/6EBTI5JFFyLMS0T3pnMEaW1GUaJR/S5vh99NJEKbpi/T7bPueJ2o
         ykfg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709041871; x=1709646671;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=Yok1+WIrlSpKjzylNjv4/P+Vhf513Inyq/FWByxKCmE=;
        b=YCFFSRghLcKgO56fHILtdGjY1poANoRYmAE41Nh+Li74xmnv2/j4j8vb70SbuCR55g
         y40A+e/xt/vo8KqSNPKQjAGRsCuwNJXUiRLLVPBDt8L+N9kmCBhkFeVLAafVNx9sA9sr
         Dnq2lCDKAycy3SuK19XlHuuVpeaDxrnwdstlwIOMdmOmLZtri9/rVb35CthSnahlToxC
         A0ysj1lrCsfCplQHJEG8p3NDqlffLUO+z8ZMwuxaSPuqIUX/9ofGnn2jjD3O8Lk8hCSN
         SrbMSMWBu0hudYNHRePCaS0a45aw6mKgKsOHonJVUoGgI1mvpdIIkep+mdBzXq7/vD7D
         O+mg==
X-Forwarded-Encrypted: i=1; AJvYcCX0l3P7iBXR4u1tphDgRJLJByqm0Yoenc/V0mWPLBWWEUvflwiPKW9FJOzjzza3SH2ZHVeF/GxjZowDuYQbDtxQ88LfGk3pJeNZzf8bjQc=
X-Gm-Message-State: AOJu0Yy2MoDU9DgYhFr6wUPiZBU/y09OW7K2PApGHLM3opWJd8P8Xe0b
	m9iDv1Sq2rFHefbYh1ouJdwoHdmpd2edJS+SnDCH5u/qpc9/s9K42hup1V3yzw==
X-Google-Smtp-Source: AGHT+IH8VLsGz+Y4Ea4tBem0tNCKa+IINLd1n31PrY+W9qxCYbu4aJWOjdPT+W4dAPf/rcUB5m4Q1g==
X-Received: by 2002:a17:906:2844:b0:a3f:adad:9ce1 with SMTP id s4-20020a170906284400b00a3fadad9ce1mr8545201ejc.18.1709041870637;
        Tue, 27 Feb 2024 05:51:10 -0800 (PST)
Message-ID: <0acb5dc2-e665-4c8e-a10b-2c7664823bf7@suse.com>
Date: Tue, 27 Feb 2024 14:51:09 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
To: Henry Wang <xin.wang2@amd.com>
Cc: Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>, xen-devel@lists.xenproject.org
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <b41daccb-d402-4da8-bc88-933facd088a8@suse.com>
 <c3edf84d-3177-4125-bdd6-ba99190340a3@amd.com>
 <23dc102e-e827-4f1e-876d-76a7df4c4927@suse.com>
 <ca749e1e-569e-4d7d-8009-63a9469d3ffd@amd.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ca749e1e-569e-4d7d-8009-63a9469d3ffd@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.02.2024 14:35, Henry Wang wrote:
> Hi Jan,
> 
> On 2/27/2024 9:27 PM, Jan Beulich wrote:
>> On 27.02.2024 14:24, Henry Wang wrote:
>>> On 2/26/2024 4:25 PM, Jan Beulich wrote:
>>>> On 26.02.2024 02:19, Henry Wang wrote:
>>>>> --- a/xen/common/memory.c
>>>>> +++ b/xen/common/memory.c
>>>>> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>>>>>            }
>>>>>            else
>>>>>            {
>>>>> -            if ( is_domain_direct_mapped(d) )
>>>>> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
>>>>>                {
>>>>>                    mfn = _mfn(gpfn);
>>>>>    
>>>> I wonder whether is_domain_direct_mapped() shouldn't either be cloned
>>>> into e.g. is_gfn_direct_mapped(d, gfn), or be adjusted in-place to gain
>>>> such a (then optional) 2nd parameter. (Of course there again shouldn't be
>>>> a need for every domain to define a stub is_domain_direct_mapped() - if
>>>> not defined by an arch header, the stub can be supplied in a single
>>>> central place.)
>>> Same here, it looks like you prefer the centralized
>>> is_domain_direct_mapped() now, as we are having more archs. I can do the
>>> clean-up when sending v2. Just out of curiosity, do you think it is a
>>> good practice to place the is_domain_direct_mapped() implementation in
>>> xen/domain.h with proper arch #ifdefs?
>> arch #ifdefs? I'd like to avoid such, if at all possible. Generic fallbacks
>> generally ought to key their conditionals to the very identifier not
>> (already) being defined.
> 
> I meant something like this (as I saw CDF_directmap is currently gated 
> in this way in xen/domain.h):
> 
> #ifdef CONFIG_ARM
> #define is_domain_direct_mapped(d) ((d)->cdf & CDF_directmap)
> #else
> #define is_domain_direct_mapped(d) ((void)(d), 0)
> #endif
> 
> I am having trouble to think of another way to keep the function in a 
> centralized place while
> avoiding the #ifdefs. Would you mind elaborating a bit? Thanks!

What is already there is fine to change. I took your earlier reply to
mean that you want to add an "#ifndef CONFIG_ARM" to put in place some
new fallback handler.

Of course the above could also be done without any direct CONFIG_ARM
dependency. For example, CDF_directmap could simply evaluate to zero
when direct mapped memory isn't supported.

Jan


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 14:22:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 14:22:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686214.1067964 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reyLL-0002fh-E0; Tue, 27 Feb 2024 14:22:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686214.1067964; Tue, 27 Feb 2024 14:22:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reyLL-0002fa-BK; Tue, 27 Feb 2024 14:22:03 +0000
Received: by outflank-mailman (input) for mailman id 686214;
 Tue, 27 Feb 2024 14:22:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=fto1=KE=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1reyLK-0002fE-13
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 14:22:02 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8fc78099-d57b-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 15:21:57 +0100 (CET)
Received: from BL1PR13CA0195.namprd13.prod.outlook.com (2603:10b6:208:2be::20)
 by MN6PR12MB8470.namprd12.prod.outlook.com (2603:10b6:208:46d::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Tue, 27 Feb
 2024 14:21:51 +0000
Received: from BL6PEPF0001AB51.namprd04.prod.outlook.com
 (2603:10b6:208:2be:cafe::19) by BL1PR13CA0195.outlook.office365.com
 (2603:10b6:208:2be::20) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.25 via Frontend
 Transport; Tue, 27 Feb 2024 14:21:51 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL6PEPF0001AB51.mail.protection.outlook.com (10.167.242.75) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 27 Feb 2024 14:21:51 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 08:21:51 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 08:21:51 -0600
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 27 Feb 2024 08:21:47 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8fc78099-d57b-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=MlP+jSREOZJzhnUqe2YTwttopRWtFs+KAKPPMZoUSoaYVqRtRdGExR/FOfjAlmjkZMh7/qQ+5GCAbmjJGvd1BIldGzZaWkDraUuHz0XVv8zeBoE6A9evzYvvJCSibYFaXoyNXR8bfgww8kmpSKFgD3k2h1zsGVO/WSzoeh+2CJAUP3B1V8KPWBbaDAGVLD3l9iDdwo/Jekl/xvEgLB1oIzIn/DTzouiuLXi3z+rd7Z3uwJlftRcX8DQuEvpP9PWNl1YezUYpWOfXxcXvQmQycr6taKmPEo9jsNyVjVyCMgkDuuWcRj7bXqedc/AX68elPpWmAth6rFpGdhLa92lcJw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=50gZi4HX0UroOWiE1CHbPHFkXdBR0q1Po8JppdrMdfo=;
 b=jL/STKQNEeO4313iUcvOkaT4gfTnuPvCoeEYEjn/4udOXlQLd2X2ukoPFMfxL2QaEIXCmbjlzr25RNpWC1fGjCl74h3OiVAfYnuyUBNpFojeWJmlVCXwBQepLksgbmEOADRI+e/ie02qn+Mv6rHkvKg6xX0PXG8Zc6J8kbFtduu9H7dCtMFk6pIHjqJ7RNuCpndhlcIHXL686GsBX0k8G1L3kwoTriJP1TU13flC5Jm7O0sXY5GJ4Uw53LnrytQ1MisPdGjrCU8hxZEAhjDs1Sh7Iwr3vYDD+XmL5EriZ1htoIIRusy7HvJ4pcahQCrTlnblMwX/piUtM7cLxoXtNQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=50gZi4HX0UroOWiE1CHbPHFkXdBR0q1Po8JppdrMdfo=;
 b=OC/hLt+Zt9Rfl4nzlwQIRcdGTMmgFqR5iJMxhUoVilIaKnHF/Hn6IyXAYRK2fuwNp/+B7VXHMrHgDOgRqmicdLzcjodFj0DAaLHYVdzUGqIGA/IhKk+pc9DpzbbZcx8rfms5/yec0lqw+Fw7mHY6WydKhy9sMqoXJfZM29GkwNY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <ab851734-0ad9-4eef-bcb3-dbe1aed08e2c@amd.com>
Date: Tue, 27 Feb 2024 22:21:47 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
CC: Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano Stabellini
	<sstabellini@kernel.org>, Bertrand Marquis <bertrand.marquis@arm.com>, Michal
 Orzel <michal.orzel@amd.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	=?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Alec Kwapis
	<alec.kwapis@medtronic.com>, <xen-devel@lists.xenproject.org>
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <b41daccb-d402-4da8-bc88-933facd088a8@suse.com>
 <c3edf84d-3177-4125-bdd6-ba99190340a3@amd.com>
 <23dc102e-e827-4f1e-876d-76a7df4c4927@suse.com>
 <ca749e1e-569e-4d7d-8009-63a9469d3ffd@amd.com>
 <0acb5dc2-e665-4c8e-a10b-2c7664823bf7@suse.com>
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <0acb5dc2-e665-4c8e-a10b-2c7664823bf7@suse.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB51:EE_|MN6PR12MB8470:EE_
X-MS-Office365-Filtering-Correlation-Id: 20dd8c0b-90a6-458a-36d4-08dc379f7111
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	yorn6CSUx2upAMPpbSvty9GXuEJ7/7t3csBPbv6NgYvZU+5py/6XYMq2wfn1GsFC9kUgNn/gyxLbl5IHH+16m5aE2Ux/UB0ObnKqb5ohnmQlA9ibOP606x8q4mEY7Cd8lUBS2Tpi05kjfHgdIAjo3jjuBTdJDlN5/7elcpOfEdPmUh0h9xQ/owfHAEQr3bdSCcRBqOwO6ARA53qSK5/hhNMOKZHQez2147UH4Nk8mOSBpNhWATQjCl3jZfHfR/q4Grup8Gat22/DMWDi6akEjJywcr+U9x2g57KGrZ6W0Z6lTOWf39VAqcdCzwc7mPySfxgIbuzR32DQcP2+aCGKj2uy6RIAEJuNS4b/qyiLQMI8zyE9g8MRUx1Cj30+EGkEZzYQu7KCNkcqiqfsJ2pa5tzR6G40qusJgnn5YW2JnZxE0brMHB8ftAenQ+3sL/8JAThAZsfNVmNTdiQjNAJvE5AQTHk9yv/xPj49Kvzjk7cAkLqFV18R5m0btE+oURHHMnfRPq2+OrIJbCwdN8wo5jyCfTWY1hcyboLQ8HDuanxO9ImqAK6Hag1oSiPUmuyU+Zy8vdZ51dnJb4kdsbQnchSbE7sRm6pN/M3A8ldllFuF5PFCNbKv58Ub1GQu7AV8pJpb+7DAeqziq6XPHEacrgwxnJFlqfT6ynArJUG0D2TQUJM8bsfcxV5qimLxwkPC8XkVS0CI32RTqc7TN1Zrkth9g2yWZCbzPkn6Ex6sNnyxUxs0sucmL/6gLWVHYtMT
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 14:21:51.5580
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 20dd8c0b-90a6-458a-36d4-08dc379f7111
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL6PEPF0001AB51.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN6PR12MB8470

Hi Jan,

On 2/27/2024 9:51 PM, Jan Beulich wrote:
> On 27.02.2024 14:35, Henry Wang wrote:
>> Hi Jan,
>>
>> On 2/27/2024 9:27 PM, Jan Beulich wrote:
>>> On 27.02.2024 14:24, Henry Wang wrote:
>>>> On 2/26/2024 4:25 PM, Jan Beulich wrote:
>>>>> On 26.02.2024 02:19, Henry Wang wrote:
>>>>>> --- a/xen/common/memory.c
>>>>>> +++ b/xen/common/memory.c
>>>>>> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>>>>>>             }
>>>>>>             else
>>>>>>             {
>>>>>> -            if ( is_domain_direct_mapped(d) )
>>>>>> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
>>>>>>                 {
>>>>>>                     mfn = _mfn(gpfn);
>>>>>>     
>>>>> I wonder whether is_domain_direct_mapped() shouldn't either be cloned
>>>>> into e.g. is_gfn_direct_mapped(d, gfn), or be adjusted in-place to gain
>>>>> such a (then optional) 2nd parameter. (Of course there again shouldn't be
>>>>> a need for every domain to define a stub is_domain_direct_mapped() - if
>>>>> not defined by an arch header, the stub can be supplied in a single
>>>>> central place.)
>>>> Same here, it looks like you prefer the centralized
>>>> is_domain_direct_mapped() now, as we are having more archs. I can do the
>>>> clean-up when sending v2. Just out of curiosity, do you think it is a
>>>> good practice to place the is_domain_direct_mapped() implementation in
>>>> xen/domain.h with proper arch #ifdefs?
>>> arch #ifdefs? I'd like to avoid such, if at all possible. Generic fallbacks
>>> generally ought to key their conditionals to the very identifier not
>>> (already) being defined.
>> I meant something like this (as I saw CDF_directmap is currently gated
>> in this way in xen/domain.h):
>>
>> #ifdef CONFIG_ARM
>> #define is_domain_direct_mapped(d) ((d)->cdf & CDF_directmap)
>> #else
>> #define is_domain_direct_mapped(d) ((void)(d), 0)
>> #endif
>>
>> I am having trouble to think of another way to keep the function in a
>> centralized place while
>> avoiding the #ifdefs. Would you mind elaborating a bit? Thanks!
> What is already there is fine to change. I took your earlier reply to
> mean that you want to add an "#ifndef CONFIG_ARM" to put in place some
> new fallback handler.
>
> Of course the above could also be done without any direct CONFIG_ARM
> dependency. For example, CDF_directmap could simply evaluate to zero
> when direct mapped memory isn't supported.

Yes correct, that will indeed simplify the code a lot. I can do the 
change accordingly in follow-up versions.

Kind regards,
Henry

> Jan



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 14:57:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 14:57:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686221.1067978 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reytd-0000Ie-2Q; Tue, 27 Feb 2024 14:57:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686221.1067978; Tue, 27 Feb 2024 14:57:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reytc-0000IX-Uq; Tue, 27 Feb 2024 14:57:28 +0000
Received: by outflank-mailman (input) for mailman id 686221;
 Tue, 27 Feb 2024 14:57:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t50j=KE=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1reytb-0000IQ-7V
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 14:57:27 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 83ca4268-d580-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 15:57:24 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a431324e84cso306583166b.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 06:57:23 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ot14-20020a170906ccce00b00a3eeb10acb4sm831739ejb.185.2024.02.27.06.57.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 27 Feb 2024 06:57:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 83ca4268-d580-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709045843; x=1709650643; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=VbI/aw2A61gCRV7MAhTKmzdWUbLF4UdyloWgSaqoifg=;
        b=sfZIAMT6fgVhmLFZOWInBPpPUg9u9GdKq++l4KK2qrcWRM6tNJKZdvjcBvy0ZmDAfl
         sarcQQRCY9c33kgtWX46NJ3OtTDeqoSqV06W3aKJkN7Y5H0TxC64QWYsfbNkFsjU8Thw
         b9OxdDqaLF1P/G0q/Ct7qDFC7TrXOxh/P4IL8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709045843; x=1709650643;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=VbI/aw2A61gCRV7MAhTKmzdWUbLF4UdyloWgSaqoifg=;
        b=PW2MkulEYTMJREeNPFouf26ba616hzISumMVPynuoh9Nyp4VvWp2dsBF7KuDePvtO3
         1pbV/ezUMa6ZXCBF24eJP0TYrnL51f8ztIBR8VwKBJpxUpLbxWqgGYBlfKcIe92sJ4Bx
         TluCz9DlKkaWIm75X7dMo9Ms3b1H/zJ9B5LmkD+r/A311n4MBUsxRRoXEMBrEu5Uy/23
         Ajtdo1c/4Fy46I0B+xlNJy8QFDpVo+FG3PIOR0aUi3ydpRLEVACdYvcwEm0p0P1klwt9
         UrJRDPFqskdC1TJOV8zFwgxhMoki+Xf1uribmXNZj1OeRJshhrKqkBp/XnW9FFjT3cH0
         1spw==
X-Forwarded-Encrypted: i=1; AJvYcCX4oJTBEJhmZ5B/aS66tAVvATvszaXP9wgqR88KKuxfV5sAryTnxbrP8T0he/cWkXalwAvcAxyTgqyEL0fYsddoAtWmG6z5+xchfnQuL7I=
X-Gm-Message-State: AOJu0Ywo55kyjtagW1FCV7qPTfXiijEuwhEytjRkyZrfpxtHqWxXNS6d
	S/ZloQ4leapQc9OmhA+FhMBSEfTG+Wa6URNY/tneAV5qwbku6CNNy/MJoP9sM5I=
X-Google-Smtp-Source: AGHT+IE++3HltYANMGnr5DsiVYmqCSJs99lmT+iq6xGpcElKBK8C3+9N6tIoWNmoss2ALRODFdpAlw==
X-Received: by 2002:a17:906:e52:b0:a43:d853:98f1 with SMTP id q18-20020a1709060e5200b00a43d85398f1mr451293eji.17.1709045843463;
        Tue, 27 Feb 2024 06:57:23 -0800 (PST)
Message-ID: <ca4fe67a-3efd-4ce5-b37d-e68b1bf4c1a7@citrix.com>
Date: Tue, 27 Feb 2024 14:57:22 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: preparations for 4.18.1
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Anthony Perard <anthony.perard@citrix.com>, Julien Grall <julien@xen.org>,
 Kelly Choi <kelly.choi@cloud.com>
References: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27/02/2024 1:19 pm, Jan Beulich wrote:
> All,
>
> the release is due in two to three weeks. Please point out backports you find
> missing from the respective staging branch, but which you consider relevant.
>
> Jan
>

Looking at the XenServer patchqueue, a couple to consider but nothing
jumps out as critically urgent.

b6cf604207fd and 098d868e52ac as oxenstored perf fixes, although there's
one incremental (non-functional) fix I'm still waiting on an ack on. 
The backport to 4.18 is trivial.  The backport to 4.17 is miserable but
I've already done it.

ae6d4fd87676 which is a UBSAN fix, guest reachable but we believe not an
issue in non-UBSAN builds.

cf7fe8b72dea which is the CPUID rescan on microcode load.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 14:58:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 14:58:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686224.1067987 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reyu8-0000l7-DJ; Tue, 27 Feb 2024 14:58:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686224.1067987; Tue, 27 Feb 2024 14:58:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1reyu8-0000l0-AN; Tue, 27 Feb 2024 14:58:00 +0000
Received: by outflank-mailman (input) for mailman id 686224;
 Tue, 27 Feb 2024 14:57:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=qLI9=KE=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1reyu6-0000IQ-Vn
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 14:57:59 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20601.outbound.protection.outlook.com
 [2a01:111:f403:2417::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 96de4a4a-d580-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 15:57:57 +0100 (CET)
Received: from BN7PR02CA0010.namprd02.prod.outlook.com (2603:10b6:408:20::23)
 by DS0PR12MB8317.namprd12.prod.outlook.com (2603:10b6:8:f4::10) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.36; Tue, 27 Feb
 2024 14:57:53 +0000
Received: from SN1PEPF0002636C.namprd02.prod.outlook.com
 (2603:10b6:408:20:cafe::4b) by BN7PR02CA0010.outlook.office365.com
 (2603:10b6:408:20::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.50 via Frontend
 Transport; Tue, 27 Feb 2024 14:57:53 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SN1PEPF0002636C.mail.protection.outlook.com (10.167.241.137) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 27 Feb 2024 14:57:52 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 08:57:52 -0600
Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 27 Feb 2024 08:57:51 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 96de4a4a-d580-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=HGFri9EeGhHjGxvxu7ykHF97PigITRmmPBRvXXJL7LIKm2Yahor9SoN8OaXgtCpDg6GeqvlLPMS0B+Ll5vWiZNsEPDRi+ddnMZQEVYGl1kjjXp2VWoyp/vySOD7lSfPO+X6ZO+cf+cr2YvAqcvCIMYfmbFBWyOvBRKG1IKWoRKWjBRHrmxlTTCPwuo5JcY/KYYeexxi8syeGssyFEp9XIgm+15R2D3A0itCz9n97za/KeMAKu7xrmWWJUI5ZN0fQumeAU9PHy8JSh7KSGYsbvirGhXNQYKnIjeYeGSq1YNpT3jaEYJqgTnermod52umtzf0kZYY9g+DwY7VFrhItrw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=vvj0LEigc/XintbNW+DN4Ey7AN3YiDC/XrVKTpDiUTE=;
 b=ZZVk1eUn5Jtsd7tFfcd4v+q8zJVmzhzGzYfW8AmlwxhkcwKvZ/Cq21PinAH30leIyHVwIiMOv2mZtJSzJ/0FJQoAXcqmM6w73mh69IncBQvhlmFFJlDoeRigrQ6sDQwxrZFYr3CnCSOcQk+a/LMfhd7wtQuco8+yRgHwR+6WHe3fRgX7dp8vqzFN6E+yNQKAaAV5u89p5Qx3+jc/dEB1h6ZdSGho8i9eeDSWzsTOiufS7FOguFBYd1SVPABD6clR07yiZkM78oDXIzk9BSobXp32oWO9CHhaLo+kz67psGzQjiUlZEnaMS8K3Cv70a1AJCeg7aipq1v/X3DETUfnEw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=vvj0LEigc/XintbNW+DN4Ey7AN3YiDC/XrVKTpDiUTE=;
 b=c/bIhHWlCH7vbJvGouWPwH+ZJlF9zHBfJueKm8N9H/n6z/vu2yqeAjRcXpF9uZHXESRb/TEoc8QwxVt19T2x+1EE6qb5EzW7PevRDLCncNs0p5Ky3chVfBoRjM76BggD0q3iWOpzCmBjWV8yGwHlCPPRXFGcceDmZ9NbcwYhKPk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Stewart Hildebrand <stewart.hildebrand@amd.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
	<wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>
Subject: [PATCH] tests/vpci: fix unit tests after locking change
Date: Tue, 27 Feb 2024 09:57:43 -0500
Message-ID: <20240227145745.1114780-1-stewart.hildebrand@amd.com>
X-Mailer: git-send-email 2.43.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: stewart.hildebrand@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF0002636C:EE_|DS0PR12MB8317:EE_
X-MS-Office365-Filtering-Correlation-Id: 424b2944-2860-4a4d-a90e-08dc37a47950
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	FnjFvrK+MLx1CCp6uHI7K4H0WRURtY9Dz5xEv2iHZR+ARaAZu3pU/CijYi2Pw9ywp2OPCgU4Rn+GLR6DSZ9KwEmbUGXlxsggMivYJCl4zTvmxSAGXCgd6Q4XaTw//B8mTebMPzXzOzWOetT8xvrzxGaSDJ5YR3ftFF7htiAuR9X8PPGmAm6JIypu/vy+XV2FidyE0zbAxcU5k+Z8tSo8xT1MA5pMdId0YItMG8XNbKnIeGVlBTmhJuVelnSNGc/ORmGPrcEe0E19LzxCnAdCCApxT6mBzFqPFhs1FwnVUdUgz/7WCbfveinumMCxtFgcyvqTSho2O0bXPFd5Fut6KShqGcytBSSrp61w2zlFNkx8SxX4epO82dlSvcQjNTHy0HvqGdhueDEzxKLA/nT/Fow5aSUjk7QWxluUuiaFPtFEAyq4WCV/vk07gfCszdBrsmJ8dSqjCVzhH5mMcmyC1d98np5ICy9XiGJAFAD9qhxochyBhTBUPhkZ9eOU7nsS7oEPGtyMNrTFU14jdpzsy9BpbS9eTtQ7640FspuZmFAwMUFCRVm9DgpvzRP/5h28UEpSfbotT5TOc+GPVjlKISQbw2+czlKLgvq1KDAXTgC4lwZ8mmjs8ZtDUzqHPzHLFuDp3zzAt3XkMz4OLGrdHriP/wPsdWTH+dREapcPIHCACGwKSZG41kJPL7f/QvCVOQlToHszISKQb8aQNv3ZKbL5Z0WIxpWhMJIQgnq15T7WKUX592t4kT9j2MH/a5Gj
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400014)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 14:57:52.8792
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 424b2944-2860-4a4d-a90e-08dc37a47950
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF0002636C.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8317

The recent vPCI locking broke the vPCI unit tests. Fix it to unblock CI.

Fixes: 4f78438b45e2 ("vpci: use per-domain PCI lock to protect vpci structure")
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
---
 tools/tests/vpci/emul.h | 9 ++++++++-
 tools/tests/vpci/main.c | 2 +-
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/tools/tests/vpci/emul.h b/tools/tests/vpci/emul.h
index 3c2f66a18f13..da446bba86b4 100644
--- a/tools/tests/vpci/emul.h
+++ b/tools/tests/vpci/emul.h
@@ -37,7 +37,10 @@
 
 #include "list.h"
 
+typedef bool rwlock_t;
+
 struct domain {
+    rwlock_t pci_lock;
 };
 
 struct pci_dev {
@@ -46,7 +49,7 @@ struct pci_dev {
 
 struct vcpu
 {
-    const struct domain *domain;
+    struct domain *domain;
 };
 
 extern const struct vcpu *current;
@@ -56,6 +59,10 @@ typedef bool spinlock_t;
 #define spin_lock_init(l) (*(l) = false)
 #define spin_lock(l) (*(l) = true)
 #define spin_unlock(l) (*(l) = false)
+#define read_lock(l) (*(l) = true)
+#define read_unlock(l) (*(l) = false)
+#define write_lock(l) (*(l) = true)
+#define write_unlock(l) (*(l) = false)
 
 typedef union {
     uint32_t sbdf;
diff --git a/tools/tests/vpci/main.c b/tools/tests/vpci/main.c
index 64d4552936c7..33223db3eb77 100644
--- a/tools/tests/vpci/main.c
+++ b/tools/tests/vpci/main.c
@@ -21,7 +21,7 @@
 /* Single vcpu (current), and single domain with a single PCI device. */
 static struct vpci vpci;
 
-const static struct domain d;
+static struct domain d;
 
 const struct pci_dev test_pdev = {
     .vpci = &vpci,

base-commit: 576528a2a742069af203e90c613c5c93e23c9755
-- 
2.43.2



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 15:29:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 15:29:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686229.1067997 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rezOR-0006bB-Rp; Tue, 27 Feb 2024 15:29:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686229.1067997; Tue, 27 Feb 2024 15:29:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rezOR-0006b4-P4; Tue, 27 Feb 2024 15:29:19 +0000
Received: by outflank-mailman (input) for mailman id 686229;
 Tue, 27 Feb 2024 15:29:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rezOQ-0006au-Rm; Tue, 27 Feb 2024 15:29:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rezOQ-0002zc-Nf; Tue, 27 Feb 2024 15:29:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rezOQ-0007tT-EO; Tue, 27 Feb 2024 15:29:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rezOQ-0002Ni-DT; Tue, 27 Feb 2024 15:29:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Ypc9GRfZYKp6iIPJub2U+ONDc3Y02U310AnTu2XvGJw=; b=qF9GegJoUQ3SYaeczRUbCWSLBy
	yt9PGHfZDXniNVnMay5m6xnUvEq+wdgo2u1Qf7lqvW4AmtVmCaa5PpdQKaizLTMAQ59vWRB3VB0sR
	RkVYUA+AhNP7wlAGrl0HSashN3/UTMs6b4d2FhRq/f6jZwf2zt0AfiW91j72zCMkHM8s=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184782-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184782: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=ba9c3ceaf83d3ee448b2fd38bc9ad28b690c697a
X-Osstest-Versions-That:
    ovmf=74b5309da9fb7a919bec5a8b5a63d1ede5eb6745
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 15:29:18 +0000

flight 184782 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184782/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 ba9c3ceaf83d3ee448b2fd38bc9ad28b690c697a
baseline version:
 ovmf                 74b5309da9fb7a919bec5a8b5a63d1ede5eb6745

Last test of basis   184778  2024-02-27 07:44:39 Z    0 days
Testing same since   184782  2024-02-27 12:43:39 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Tuan Phan <tphan@ventanamicro.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   74b5309da9..ba9c3ceaf8  ba9c3ceaf83d3ee448b2fd38bc9ad28b690c697a -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 15:37:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 15:37:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686235.1068008 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rezWM-000080-Le; Tue, 27 Feb 2024 15:37:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686235.1068008; Tue, 27 Feb 2024 15:37:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rezWM-00007l-J7; Tue, 27 Feb 2024 15:37:30 +0000
Received: by outflank-mailman (input) for mailman id 686235;
 Tue, 27 Feb 2024 15:37:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NOTz=KE=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rezWL-00007f-Gc
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 15:37:29 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1c4478d5-d586-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 16:37:27 +0100 (CET)
Received: from beta.station (net-93-144-106-196.cust.dsl.teletu.it
 [93.144.106.196])
 by support.bugseng.com (Postfix) with ESMTPSA id 7B29A4EE0742;
 Tue, 27 Feb 2024 16:37:26 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1c4478d5-d586-11ee-a1ee-f123f15fe8a2
From: Simone Ballarin <simone.ballarin@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH] docs: Move function and macro properties in docs/misra
Date: Tue, 27 Feb 2024 16:36:30 +0100
Message-Id: <ee28804796625835688069997a06b719d0739b9b.1709048153.git.simone.ballarin@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

The list of function/macro properties is not MISRA-specific documentation.
Their addition was directly motivated to address MISRA findings and they
are not used elsewhere. For this reason, this patch moves these properties
in docs/misra.

This patch also fixes a Sphinx warning caused by the missing inclusion
of the rst file in the toctree.

Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Fixes: f4519ee8 ("eclair: move function and macro properties outside ECLAIR")
Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
---
 automation/eclair_analysis/propertyparser.py                    | 2 +-
 .../function-macro-properties.json}                             | 0
 .../function-macro-properties.rst}                              | 0
 docs/misra/index.rst                                            | 1 +
 4 files changed, 2 insertions(+), 1 deletion(-)
 rename docs/{function_macro_properties.json => misra/function-macro-properties.json} (100%)
 rename docs/{function_macro_properties.rst => misra/function-macro-properties.rst} (100%)

diff --git a/automation/eclair_analysis/propertyparser.py b/automation/eclair_analysis/propertyparser.py
index 0d02f505a6..5059a68fec 100644
--- a/automation/eclair_analysis/propertyparser.py
+++ b/automation/eclair_analysis/propertyparser.py
@@ -2,7 +2,7 @@ import json
 import os
 
 script_dir = os.path.dirname(__file__)
-properties_path = os.path.join(script_dir, "../../docs/function_macro_properties.json")
+properties_path = os.path.join(script_dir, "../../docs/misra/function-macro-properties.json")
 output_path   = os.path.join(script_dir, "ECLAIR/call_properties.ecl")
 
 with open(properties_path) as fp:
diff --git a/docs/function_macro_properties.json b/docs/misra/function-macro-properties.json
similarity index 100%
rename from docs/function_macro_properties.json
rename to docs/misra/function-macro-properties.json
diff --git a/docs/function_macro_properties.rst b/docs/misra/function-macro-properties.rst
similarity index 100%
rename from docs/function_macro_properties.rst
rename to docs/misra/function-macro-properties.rst
diff --git a/docs/misra/index.rst b/docs/misra/index.rst
index 5068ee5aef..bde9d9cffb 100644
--- a/docs/misra/index.rst
+++ b/docs/misra/index.rst
@@ -16,3 +16,4 @@ Xen hypervisor code and related documents.
    documenting-violations
    exclude-list
    xen-static-analysis
+   function-macro-properties
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 15:39:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 15:39:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686237.1068018 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rezYE-0000lu-0g; Tue, 27 Feb 2024 15:39:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686237.1068018; Tue, 27 Feb 2024 15:39:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rezYD-0000ln-TY; Tue, 27 Feb 2024 15:39:25 +0000
Received: by outflank-mailman (input) for mailman id 686237;
 Tue, 27 Feb 2024 15:39:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NOTz=KE=bugseng.com=simone.ballarin@srs-se1.protection.inumbo.net>)
 id 1rezYD-0000lh-2F
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 15:39:25 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 61e299f8-d586-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 16:39:24 +0100 (CET)
Received: from [192.168.1.9] (net-93-144-106-196.cust.dsl.teletu.it
 [93.144.106.196])
 by support.bugseng.com (Postfix) with ESMTPSA id 6AB524EE0742;
 Tue, 27 Feb 2024 16:39:23 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61e299f8-d586-11ee-afd6-a90da7624cb6
Message-ID: <ea2f8c01-35d9-4c4f-9c57-1691a9d66fca@bugseng.com>
Date: Tue, 27 Feb 2024 16:39:22 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [REGRESSION] Re: [XEN PATCH v4 4/4] eclair: move function and
 macro properties outside ECLAIR
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, sstabellini@kernel.org,
 Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>, George Dunlap
 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>
References: <cover.1706886631.git.simone.ballarin@bugseng.com>
 <387b160ae93c221c4bc2426605b96b432b26224e.1706886631.git.simone.ballarin@bugseng.com>
 <b549f8f3-b21d-4ca5-9ef8-e1c39856d691@citrix.com>
From: Simone Ballarin <simone.ballarin@bugseng.com>
Organization: BUGSENG
In-Reply-To: <b549f8f3-b21d-4ca5-9ef8-e1c39856d691@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 26/02/24 16:44, Andrew Cooper wrote:
> On 02/02/2024 3:16 pm, Simone Ballarin wrote:
>> From: Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
>>
>> Function and macro properties contained in ECLAIR/call_properties.ecl are of
>> general interest: this patch moves these annotations in a generaric JSON file
>> in docs. In this way, they can be exploited for other purposes (i.e. documentation,
>> other tools).
>>
>> Add rst file containing explanation on how to update function_macro_properties.json.
>> Add script to convert the JSON file in ECL configurations.
>> Remove ECLAIR/call_properties.ecl: the file is now automatically generated from
>> the JSON file.
>>
>> Signed-off-by: Maria Celeste Cesario  <maria.celeste.cesario@bugseng.com>
>> Signed-off-by: Simone Ballarin  <simone.ballarin@bugseng.com>
>>
>> ---
>> Changes in v4:
>> - add missing script for converting the JSON file in ECL configurations;
>> - improve commit message;
>> - remove call_properties.ecs.
>> ---
>>   .../eclair_analysis/ECLAIR/analysis.ecl       |   1 +
>>   .../ECLAIR/call_properties.ecl                | 128 ---
>>   automation/eclair_analysis/prepare.sh         |   2 +
>>   automation/eclair_analysis/propertyparser.py  |  37 +
>>   docs/function_macro_properties.json           | 841 ++++++++++++++++++
>>   docs/function_macro_properties.rst            |  58 ++
>>   6 files changed, 939 insertions(+), 128 deletions(-)
>>   delete mode 100644 automation/eclair_analysis/ECLAIR/call_properties.ecl
>>   create mode 100644 automation/eclair_analysis/propertyparser.py
>>   create mode 100644 docs/function_macro_properties.json
>>   create mode 100644 docs/function_macro_properties.rst
> 
> This breaks the Sphinx build.
> 
> checking consistency...
> /local/xen.git/docs/function_macro_properties.rst: WARNING: document
> isn't included in any toctree
> 
> Also, the top level of docs really isn't an appropriate place to put
> it.  Everything else is in docs/misra/.
> 
> When you've moved the files, you'll need to edit docs/misra/index.rst to
> fix the build.
> 
> ~Andrew
> 

I've submitted a new patch moving these files in docs/misra
and adding the rst file in the docs/misra/index.rst's toctree.

-- 
Simone Ballarin, M.Sc.

Field Application Engineer, BUGSENG (https://bugseng.com)



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 15:40:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 15:40:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686239.1068028 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rezZ6-0002DD-AY; Tue, 27 Feb 2024 15:40:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686239.1068028; Tue, 27 Feb 2024 15:40:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rezZ6-0002D6-6S; Tue, 27 Feb 2024 15:40:20 +0000
Received: by outflank-mailman (input) for mailman id 686239;
 Tue, 27 Feb 2024 15:40:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t50j=KE=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rezZ5-0002Cv-6x
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 15:40:19 +0000
Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com
 [2a00:1450:4864:20::130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 81c2194b-d586-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 16:40:17 +0100 (CET)
Received: by mail-lf1-x130.google.com with SMTP id
 2adb3069b0e04-512d19e2cb8so6529430e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 07:40:17 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 h6-20020ac87146000000b0042c5a47df18sm3653919qtp.55.2024.02.27.07.40.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Tue, 27 Feb 2024 07:40:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81c2194b-d586-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709048417; x=1709653217; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=BlkqbhJtmw3BGt75/n83egbkYsfuFT4QE61QJDngrQE=;
        b=rPHNGvK8YFT/vCerS1WkolTM32uTQtDl0+pvLK0XiaDBK07562Js2kPA45Ik0Opc1V
         +2UBuhAUfaFsTKhB/Fo2bx90lzZOt8tKAPX+19HvjU+RG+o+Z6e25tvQhG6Zw1+XAQzu
         hbgdzA66JoszSJn220a9CYA1yFONBuvyIeMgQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709048417; x=1709653217;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=BlkqbhJtmw3BGt75/n83egbkYsfuFT4QE61QJDngrQE=;
        b=C9CGZYYfvThqZFbB0VnAZcjVU9gBV6cHOo14NRiH1HH5ISJxkTkMGh3JpBGyskg6Mh
         NUYoL3BrwO1Af3pVLYPa3HvE2oMqEHCJs2boEWAsr1xbHx4p0pNl5XK9ZSKRDdsBtFfD
         7vjDUukjf4FBIR0ssYVA6PKR24wTxSoCk5PkcGfILKCvRgmr51+NpIob0YGEkb12rRub
         F7u1Pti9uWvIawefbk1MZxf1k5wGrMEVQ0KR6WPh4zpigaYfIqFo0PPIpPYHUGKLufy5
         pmpoi7IrBVk6nESsRd3Ro3xJJP3/5oR5VuzFElhRE/KcV3v8tZdzp5YQvVRnzoVsldYs
         833Q==
X-Forwarded-Encrypted: i=1; AJvYcCWGgIDYQDgDfQnWE++VKJmgu6ppL1JpD8SEbsaXcvEN4vq5Zid0eGgBcWbpGWeDbbDKACT1m9V4Z2Tgjd9D1yWa54cWUODnrEiKvC21XeA=
X-Gm-Message-State: AOJu0YxgDfdfKDFETkou4SH8C7aquim7sBB4UKG2fQHp387YPtSdeQdO
	Fy3yv8m5QsJ8K1Cd52Fq/9BncUS68NzQHjUXXDdEkAWp4YghMGqB+WnvdbFs1RI=
X-Google-Smtp-Source: AGHT+IGjZYoGTBQ7m74E1rM6WOr/SjYUaldRa24W24FAtT+U7ftReN55MbznqNrevn+Ogh7nDyblmA==
X-Received: by 2002:a05:6512:318e:b0:512:fe1f:d3c1 with SMTP id i14-20020a056512318e00b00512fe1fd3c1mr4550477lfe.58.1709048417018;
        Tue, 27 Feb 2024 07:40:17 -0800 (PST)
Message-ID: <7aaf2925-277f-413f-be75-00e52a208c98@citrix.com>
Date: Tue, 27 Feb 2024 15:40:14 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] docs: Move function and macro properties in
 docs/misra
Content-Language: en-GB
To: Simone Ballarin <simone.ballarin@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com, Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Wei Liu <wl@xen.org>
References: <ee28804796625835688069997a06b719d0739b9b.1709048153.git.simone.ballarin@bugseng.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ee28804796625835688069997a06b719d0739b9b.1709048153.git.simone.ballarin@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27/02/2024 3:36 pm, Simone Ballarin wrote:
> The list of function/macro properties is not MISRA-specific documentation.
> Their addition was directly motivated to address MISRA findings and they
> are not used elsewhere. For this reason, this patch moves these properties
> in docs/misra.
>
> This patch also fixes a Sphinx warning caused by the missing inclusion
> of the rst file in the toctree.
>
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Fixes: f4519ee8 ("eclair: move function and macro properties outside ECLAIR")
> Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

Thanks for fixing this.

~Andrew


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 15:46:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 15:46:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686244.1068038 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rezf6-00034l-2p; Tue, 27 Feb 2024 15:46:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686244.1068038; Tue, 27 Feb 2024 15:46:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rezf6-00034e-04; Tue, 27 Feb 2024 15:46:32 +0000
Received: by outflank-mailman (input) for mailman id 686244;
 Tue, 27 Feb 2024 15:46:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=KGmn=KE=lst.de=hch@srs-se1.protection.inumbo.net>)
 id 1rezf4-00033c-TI
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 15:46:30 +0000
Received: from verein.lst.de (verein.lst.de [213.95.11.211])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5f843f12-d587-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 16:46:29 +0100 (CET)
Received: by verein.lst.de (Postfix, from userid 2407)
 id 9F0FA68C4E; Tue, 27 Feb 2024 16:46:27 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f843f12-d587-11ee-afd6-a90da7624cb6
Date: Tue, 27 Feb 2024 16:46:27 +0100
From: Christoph Hellwig <hch@lst.de>
To: Juergen Gross <jgross@suse.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
	Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com>,
	Jens Axboe <axboe@kernel.dk>, xen-devel@lists.xenproject.org,
	linux-block@vger.kernel.org
Subject: Re: convert xen-blkfront to atomic queue limit updates v2
Message-ID: <20240227154627.GA15446@lst.de>
References: <20240221125845.3610668-1-hch@lst.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240221125845.3610668-1-hch@lst.de>
User-Agent: Mutt/1.5.17 (2007-11-01)

Jens,

can you pick this up with the Xen maintainer review in place?



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 16:09:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 16:09:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686249.1068048 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf01Z-0007NN-Tb; Tue, 27 Feb 2024 16:09:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686249.1068048; Tue, 27 Feb 2024 16:09:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf01Z-0007NG-Qy; Tue, 27 Feb 2024 16:09:45 +0000
Received: by outflank-mailman (input) for mailman id 686249;
 Tue, 27 Feb 2024 16:09:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=t50j=KE=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rf01Y-0007N8-MX
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 16:09:44 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9e6e5eaf-d58a-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 17:09:43 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a3f3d0d2787so459640766b.3
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 08:09:43 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 vb5-20020a170907d04500b00a3e9c4fa24esm898416ejc.96.2024.02.27.08.09.41
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 08:09:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9e6e5eaf-d58a-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709050182; x=1709654982; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=0qRD6B3/O73clnI5MGy+C8q94KAo6+M7vgkh5nipZSY=;
        b=d0Ovq7gVaaaXgr3/BFHP3amA8uXah1Wlpp42vqJeLaS19Nyh3jXtgZx4wnsDZOMyjf
         6WW9Xl8Lgu22yCEyY0nTea+KAUEZCSot2ksUo9y7Ddi1/0HzpX8KWwDNqZR1huxHbXcW
         hkfP2OJY7MW/jWuzeyv+lNJVfTA9Jo0Rj/obI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709050182; x=1709654982;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=0qRD6B3/O73clnI5MGy+C8q94KAo6+M7vgkh5nipZSY=;
        b=Nlt9wAanNUjhhu6LIXYC3fAOMK/Xc6axPusRXerNLQ429FH40V3VwRMGvjUSwvSLLX
         SzhPGtZWG82gm5WtWhD3PzDTLsB5xNlVDuAMQAhn0gyqCuz4P20Rb5HW+dvj02fYmGRY
         1T+NkJrchFS/++s5OYR4+E7OwcXgioQD/5j2N7Dh+1xC+fPCthZQXv3vikiGDfLHO/A/
         5Rk8KSPXAzz6P5ezeDhNwAcvs9oe9ztKGlufsfCe9i1qHH44WIQ+8Vh3y+eiejwwT4HZ
         v7ya+iRpoOyOVpg6QCIBRUlVTozlSc9xZp9vpMEuv5hDG/uDIRiyWjp5pgdxN4fu8y2e
         4Pvw==
X-Gm-Message-State: AOJu0Yw2DNzkows7/HYGBnj7LJ97B25PRCtklIy79tnrMuPf2faMheD1
	g0ZaatLMNqueX18CVtAhLrh4CXR3xVbr080ckZU7NDpqE15dAgER4lDKJRg/eol1QvraFkbMzPb
	3
X-Google-Smtp-Source: AGHT+IE2CWDx4prxdmfBEaf9CCawCpjLujAhz/xRlfEWQGeDttTwTlQuwWSQ72NlxAEa2l0CdMzMOg==
X-Received: by 2002:a17:906:140f:b0:a3f:c932:9c67 with SMTP id p15-20020a170906140f00b00a3fc9329c67mr7054390ejc.68.1709050182478;
        Tue, 27 Feb 2024 08:09:42 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86: Resync intel-family.h from Linux
Date: Tue, 27 Feb 2024 16:09:40 +0000
Message-Id: <20240227160940.2054111-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

>From v6.8-rc8

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 xen/arch/x86/include/asm/intel-family.h | 38 ++++++++++++++++++++++---
 1 file changed, 34 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/include/asm/intel-family.h b/xen/arch/x86/include/asm/intel-family.h
index ffc49151befe..b65e9c46b922 100644
--- a/xen/arch/x86/include/asm/intel-family.h
+++ b/xen/arch/x86/include/asm/intel-family.h
@@ -26,6 +26,9 @@
  *		_G	- parts with extra graphics on
  *		_X	- regular server parts
  *		_D	- micro server parts
+ *		_N,_P	- other mobile parts
+ *		_H	- premium mobile parts
+ *		_S	- other client parts
  *
  *		Historical OPTDIFFs:
  *
@@ -37,6 +40,9 @@
  * their own names :-(
  */
 
+/* Wildcard match for FAM6 so X86_MATCH_INTEL_FAM6_MODEL(ANY) works */
+#define INTEL_FAM6_ANY			X86_MODEL_ANY
+
 #define INTEL_FAM6_CORE_YONAH		0x0E
 
 #define INTEL_FAM6_CORE2_MEROM		0x0F
@@ -93,8 +99,6 @@
 #define INTEL_FAM6_ICELAKE_L		0x7E	/* Sunny Cove */
 #define INTEL_FAM6_ICELAKE_NNPI		0x9D	/* Sunny Cove */
 
-#define INTEL_FAM6_LAKEFIELD		0x8A	/* Sunny Cove / Tremont */
-
 #define INTEL_FAM6_ROCKETLAKE		0xA7	/* Cypress Cove */
 
 #define INTEL_FAM6_TIGERLAKE_L		0x8C	/* Willow Cove */
@@ -102,12 +106,31 @@
 
 #define INTEL_FAM6_SAPPHIRERAPIDS_X	0x8F	/* Golden Cove */
 
+#define INTEL_FAM6_EMERALDRAPIDS_X	0xCF
+
+#define INTEL_FAM6_GRANITERAPIDS_X	0xAD
+#define INTEL_FAM6_GRANITERAPIDS_D	0xAE
+
+/* "Hybrid" Processors (P-Core/E-Core) */
+
+#define INTEL_FAM6_LAKEFIELD		0x8A	/* Sunny Cove / Tremont */
+
 #define INTEL_FAM6_ALDERLAKE		0x97	/* Golden Cove / Gracemont */
 #define INTEL_FAM6_ALDERLAKE_L		0x9A	/* Golden Cove / Gracemont */
 
-#define INTEL_FAM6_RAPTORLAKE		0xB7
+#define INTEL_FAM6_RAPTORLAKE		0xB7	/* Raptor Cove / Enhanced Gracemont */
+#define INTEL_FAM6_RAPTORLAKE_P		0xBA
+#define INTEL_FAM6_RAPTORLAKE_S		0xBF
+
+#define INTEL_FAM6_METEORLAKE		0xAC
+#define INTEL_FAM6_METEORLAKE_L		0xAA
+
+#define INTEL_FAM6_ARROWLAKE_H		0xC5
+#define INTEL_FAM6_ARROWLAKE		0xC6
+
+#define INTEL_FAM6_LUNARLAKE_M		0xBD
 
-/* "Small Core" Processors (Atom) */
+/* "Small Core" Processors (Atom/E-Core) */
 
 #define INTEL_FAM6_ATOM_BONNELL		0x1C /* Diamondville, Pineview */
 #define INTEL_FAM6_ATOM_BONNELL_MID	0x26 /* Silverthorne, Lincroft */
@@ -134,6 +157,13 @@
 #define INTEL_FAM6_ATOM_TREMONT		0x96 /* Elkhart Lake */
 #define INTEL_FAM6_ATOM_TREMONT_L	0x9C /* Jasper Lake */
 
+#define INTEL_FAM6_ATOM_GRACEMONT	0xBE /* Alderlake N */
+
+#define INTEL_FAM6_ATOM_CRESTMONT_X	0xAF /* Sierra Forest */
+#define INTEL_FAM6_ATOM_CRESTMONT	0xB6 /* Grand Ridge */
+
+#define INTEL_FAM6_ATOM_DARKMONT_X	0xDD /* Clearwater Forest */
+
 /* Xeon Phi */
 
 #define INTEL_FAM6_XEON_PHI_KNL		0x57 /* Knights Landing */

base-commit: cfe3ad67127b86e1b1c06993b86422673a51b050
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Tue Feb 27 16:11:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 16:11:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686251.1068058 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf03c-0000fR-7d; Tue, 27 Feb 2024 16:11:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686251.1068058; Tue, 27 Feb 2024 16:11:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf03c-0000fK-55; Tue, 27 Feb 2024 16:11:52 +0000
Received: by outflank-mailman (input) for mailman id 686251;
 Tue, 27 Feb 2024 16:11:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf03b-0000f4-7X; Tue, 27 Feb 2024 16:11:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf03b-0004Iw-4x; Tue, 27 Feb 2024 16:11:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf03a-0001bP-Sp; Tue, 27 Feb 2024 16:11:50 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rf03a-00011Q-SD; Tue, 27 Feb 2024 16:11:50 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Wc1cuEu6f3apdS71x2wbF4u9siL5zK10IdFdkCh8LRs=; b=3SckXf5ena6p67+XIjnoeQ42al
	piNh+/YcZF+qEJsSatS7dN+vYtCfiO+u92vOh0tvfs0i0wLnWE2K5gb7sfbJ982QLyVC48zsBwc6T
	bmeDi4O237kP+q2Z/GGUC2YnB2q7tFe1rPDmDReYM8nvsw8EASiv2Ro7chrP7HDD+BE0=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184787-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184787: regressions - trouble: blocked/fail
X-Osstest-Failures:
    xen-unstable-smoke:build-amd64:xen-build:fail:regression
    xen-unstable-smoke:build-arm64-xsm:xen-build:fail:regression
    xen-unstable-smoke:build-armhf:xen-build:fail:regression
    xen-unstable-smoke:build-amd64-libvirt:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    xen=576528a2a742069af203e90c613c5c93e23c9755
X-Osstest-Versions-That:
    xen=ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 16:11:50 +0000

flight 184787 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184787/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 184781
 build-arm64-xsm               6 xen-build                fail REGR. vs. 184781
 build-armhf                   6 xen-build                fail REGR. vs. 184781

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a

version targeted for testing:
 xen                  576528a2a742069af203e90c613c5c93e23c9755
baseline version:
 xen                  ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87

Last test of basis   184781  2024-02-27 10:00:25 Z    0 days
Testing same since   184787  2024-02-27 14:02:12 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jens Wiklander <jens.wiklander@linaro.org>
  Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stewart Hildebrand <stewart.hildebrand@amd.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>

jobs:
 build-arm64-xsm                                              fail    
 build-amd64                                                  fail    
 build-armhf                                                  fail    
 build-amd64-libvirt                                          blocked 
 test-armhf-armhf-xl                                          blocked 
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    blocked 
 test-amd64-amd64-libvirt                                     blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 365 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 16:24:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 16:24:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686260.1068070 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf0G7-0002lz-Ef; Tue, 27 Feb 2024 16:24:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686260.1068070; Tue, 27 Feb 2024 16:24:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf0G7-0002ls-Bm; Tue, 27 Feb 2024 16:24:47 +0000
Received: by outflank-mailman (input) for mailman id 686260;
 Tue, 27 Feb 2024 16:24:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rf0G6-0002lm-AG
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 16:24:46 +0000
Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com
 [2a00:1450:4864:20::12e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b7e11201-d58c-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 17:24:45 +0100 (CET)
Received: by mail-lf1-x12e.google.com with SMTP id
 2adb3069b0e04-512e568607aso4572256e87.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 08:24:45 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 vq18-20020a05620a559200b00787c4a07c41sm3533217qkn.42.2024.02.27.08.24.43
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 08:24:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b7e11201-d58c-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709051085; x=1709655885; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=Ddr4nhzWC8oBBn+4YSPEj/IHR+Nkh7G4QtnH4b4ZEQw=;
        b=ruY56H2ralUnNj6FlSJ0GFqSDojsBSBvp7M1H/5NumCh6+/Vtj4RZIJXnWVAJ9o0UV
         sZ99SFU3A/KFmZQ9Ia0982mdtzlS2Y4o6oW6oB4WoHEpJDoF7AJlTRyUrmxbolCd6lT3
         16APYnkE8CIpZWZkePcTr8jzT6m2mM7QnHoAs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709051085; x=1709655885;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Ddr4nhzWC8oBBn+4YSPEj/IHR+Nkh7G4QtnH4b4ZEQw=;
        b=IIom+jKzCDDYfhyP7uK1hAZRnNaVHyzuiyvNohdJSGtNKTtepDWcxZb1+1YdXT8TEM
         fUBnKxSZJlyNWKfTti/7P2N2S9kVit5/2+3kYcyUoamyV7jhUFw0PvYVf5xkA5NQfTrt
         1lsOUADXUly95FyK7k7NjI3zC4FXELl07JUe9iyWeWqncilCgGmSuFlMIFUjgWYr89tz
         cJKb/Ojd/3Kn7g7XHbbqY8SlSS1GRqRSJQzdx/bnwIXTX54F5SpGns+oT8nVJlg380Ry
         l+Jp+jskSCiYcloO0zHgYPbPI6YQIxxT2Pum+XWdjbWAb3FGJw5VIQOlmyKiQ+f/NeFR
         H1zQ==
X-Gm-Message-State: AOJu0Yx6hXbJrR/BUaFlKVlv12YoTbAv336M/V+pGsM5GH++rzFBV+MD
	WvBniCoMhrEoRm4I1UF/4nNu32Xq1nGm8TTCPQA0KAW76zTU6brBeWftEDcyyEQ=
X-Google-Smtp-Source: AGHT+IF0l17cD4knqW30nSs5kjrFRUzfVaIlGtj3g/7hljvUXAJ5ziaCnwyHCh/JDbs5NT76tJck0A==
X-Received: by 2002:a05:6512:6d0:b0:512:fc60:38e4 with SMTP id u16-20020a05651206d000b00512fc6038e4mr5042904lff.54.1709051084702;
        Tue, 27 Feb 2024 08:24:44 -0800 (PST)
Date: Tue, 27 Feb 2024 17:24:42 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Stewart Hildebrand <stewart.hildebrand@amd.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH] tests/vpci: fix unit tests after locking change
Message-ID: <Zd4MyuVC8jcdpH6r@macbook>
References: <20240227145745.1114780-1-stewart.hildebrand@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240227145745.1114780-1-stewart.hildebrand@amd.com>

On Tue, Feb 27, 2024 at 09:57:43AM -0500, Stewart Hildebrand wrote:
> The recent vPCI locking broke the vPCI unit tests. Fix it to unblock CI.
> 
> Fixes: 4f78438b45e2 ("vpci: use per-domain PCI lock to protect vpci structure")
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
> ---
>  tools/tests/vpci/emul.h | 9 ++++++++-
>  tools/tests/vpci/main.c | 2 +-
>  2 files changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/tests/vpci/emul.h b/tools/tests/vpci/emul.h
> index 3c2f66a18f13..da446bba86b4 100644
> --- a/tools/tests/vpci/emul.h
> +++ b/tools/tests/vpci/emul.h
> @@ -37,7 +37,10 @@
>  
>  #include "list.h"
>  
> +typedef bool rwlock_t;
> +
>  struct domain {
> +    rwlock_t pci_lock;
>  };
>  
>  struct pci_dev {
> @@ -46,7 +49,7 @@ struct pci_dev {
>  
>  struct vcpu
>  {
> -    const struct domain *domain;
> +    struct domain *domain;
>  };
>  
>  extern const struct vcpu *current;
> @@ -56,6 +59,10 @@ typedef bool spinlock_t;
>  #define spin_lock_init(l) (*(l) = false)
>  #define spin_lock(l) (*(l) = true)
>  #define spin_unlock(l) (*(l) = false)
> +#define read_lock(l) (*(l) = true)
> +#define read_unlock(l) (*(l) = false)
> +#define write_lock(l) (*(l) = true)
> +#define write_unlock(l) (*(l) = false)

I now realize the spinlock handlers are pointless to set a boolean,
and the same applies to the rwlocks, but anyway, it's not going to
affect the testing.

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 16:32:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 16:32:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686263.1068080 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf0NQ-0004nt-64; Tue, 27 Feb 2024 16:32:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686263.1068080; Tue, 27 Feb 2024 16:32:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf0NQ-0004nm-3C; Tue, 27 Feb 2024 16:32:20 +0000
Received: by outflank-mailman (input) for mailman id 686263;
 Tue, 27 Feb 2024 16:32:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=7gEZ=KE=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rf0NP-0004nc-Ah
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 16:32:19 +0000
Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com
 [2a00:1450:4864:20::132])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c3fae9e1-d58d-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 17:32:15 +0100 (CET)
Received: by mail-lf1-x132.google.com with SMTP id
 2adb3069b0e04-5131316693cso838071e87.0
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 08:32:15 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 d20-20020a05620a159400b0078790117ebasm3740724qkk.82.2024.02.27.08.32.13
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 08:32:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c3fae9e1-d58d-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709051534; x=1709656334; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=q4HCKVjMj5LF9ZFLEW+4sb388RZ6ORvNancXxdYsAyY=;
        b=pKgBL7RClljJuO+YzIjPFChffvC9vf8lboPith7++pI3JMhoRPMy3nM3hf9TWbh5NR
         /6BhT5GFcWR6IjB7PnQOuEM0sg46yEFYXNje5IU3T/w/asQy7bCnle1nkRVkG/GQdPPj
         VDfgfVrjCiSlvOzuFpAWjI4j+JhQ9y4skmNGQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709051534; x=1709656334;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=q4HCKVjMj5LF9ZFLEW+4sb388RZ6ORvNancXxdYsAyY=;
        b=KqCCPaKf11l6ZLIsMSEcwbQMwjphbQNV9Tp5Ii/elD3fSh2yzOMGscKgycCS8yHDHL
         3mb63GE7MWmdGiyoETvUhB1zYvbZdlYF3qeRL75M+RRd6sGrg9jAEAZVt6D8b232PXGz
         fmnOHSyBABeYJI1sYaqs5pBkt6J/h9AkSXYnMr0Q/6k034miXpFL0QJPgWF6UA/rU77Q
         KRWuBE6aNx6aMG2v99rm7QZKM9ERW7X+kghE2KdWI15TYV5+beRoTZRo7HjGSjH649O6
         paOVE4kFcuGr38XFVQE4JbvNS9/ryfhLhF4ty76TKakMRa6TVU4Z/xLUy2CjrttB4+tk
         Oq+A==
X-Gm-Message-State: AOJu0Yy6h4CXp5p0VDhyBEqumWw71BA4a6JP3xn37DTushAqYVftXjh8
	CU0xRBI4UuScI00xWBJ1lRkSUZOGaEsG+npq1ng2R6IVK1VXCapfLfvhsyxTWVU=
X-Google-Smtp-Source: AGHT+IHU6GJ9wAI/IUbYSy+65zScx4s84XXxrlPCHMgkAGebFG2ZsJlLWWCd1825NccN1uB5ObG05Q==
X-Received: by 2002:ac2:488b:0:b0:513:14dc:97bf with SMTP id x11-20020ac2488b000000b0051314dc97bfmr689767lfc.23.1709051534506;
        Tue, 27 Feb 2024 08:32:14 -0800 (PST)
Date: Tue, 27 Feb 2024 17:32:12 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Ross Lagerwall <ross.lagerwall@cloud.com>
Cc: xen-devel@lists.xenproject.org,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [PATCH] livepatch-build-tools: allow patch file name sizes up to
 128 characters
Message-ID: <Zd4OjBv5jEt0Uzha@macbook>
References: <20240131165816.58762-1-roger.pau@citrix.com>
 <CAG7k0EqfMecUGQJSqMLxmR7vOjushAYQuG-r+3eAhs4a-UhuTw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAG7k0EqfMecUGQJSqMLxmR7vOjushAYQuG-r+3eAhs4a-UhuTw@mail.gmail.com>

On Tue, Feb 27, 2024 at 01:32:05PM +0000, Ross Lagerwall wrote:
> On Wed, Jan 31, 2024 at 4:58 PM Roger Pau Monne <roger.pau@citrix.com> wrote:
> >
> > XenServer uses quite long Xen version names, and encode such in the livepatch
> > filename, and it's currently running out of space in the file name.
> >
> > Bump max filename size to 128, so it also matches the patch name length in the
> > hypervisor interface.
> >
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > ---
> >  livepatch-build | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/livepatch-build b/livepatch-build
> > index cdb852cc7fea..78dd2d801048 100755
> > --- a/livepatch-build
> > +++ b/livepatch-build
> > @@ -72,8 +72,8 @@ function make_patch_name()
> >      fi
> >
> >      # Only allow alphanumerics and '_' and '-' in the patch name.  Everything
> > -    # else is replaced with '-'.  Truncate to 48 chars.
> > -    echo ${PATCHNAME//[^a-zA-Z0-9_-]/-} |cut -c 1-48
> > +    # else is replaced with '-'.  Truncate to 128 chars.
> > +    echo ${PATCHNAME//[^a-zA-Z0-9_-]/-} |cut -c -128
> >  }
> 
> I think it should be XEN_LIVEPATCH_NAME_SIZE - 1 to match the hypervisor
> since AFAICT the hypervisor expects the last character to be NUL.
> 
> In get_name(), it checks:
> 
>     if ( !name->size || name->size > XEN_LIVEPATCH_NAME_SIZE )
>         return -EINVAL;
> 
> ... and:
> 
>     if ( n[name->size - 1] )
>         return -EINVAL;
> 
> Do you agree with that?

Oh, I didn't catch that one.  xen-livepatch should also be adjusted
then, as get_name() there does:

static int get_name(int argc, char *argv[], char *name)
{
    ssize_t len = strlen(argv[0]);
    if ( len > XEN_LIVEPATCH_NAME_SIZE )

Which needs to be adjusted to len >= instead, and the error printed
should also mention that ID must be no more than
XEN_LIVEPATCH_NAME_SIZE - 1.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 16:47:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 16:47:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686270.1068090 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf0bw-00075S-DT; Tue, 27 Feb 2024 16:47:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686270.1068090; Tue, 27 Feb 2024 16:47:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf0bw-00075L-At; Tue, 27 Feb 2024 16:47:20 +0000
Received: by outflank-mailman (input) for mailman id 686270;
 Tue, 27 Feb 2024 16:47:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=U1D7=KE=raptorengineering.com=sanastasio@srs-se1.protection.inumbo.net>)
 id 1rf0bu-00075F-Om
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 16:47:18 +0000
Received: from raptorengineering.com (mail.raptorengineering.com
 [23.155.224.40]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ddb1516c-d58f-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 17:47:17 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by mail.rptsys.com (Postfix) with ESMTP id 813E382855C7;
 Tue, 27 Feb 2024 10:47:16 -0600 (CST)
Received: from mail.rptsys.com ([127.0.0.1])
 by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032)
 with ESMTP id 1kkwr_FxUiRv; Tue, 27 Feb 2024 10:47:15 -0600 (CST)
Received: from localhost (localhost [127.0.0.1])
 by mail.rptsys.com (Postfix) with ESMTP id 0C45B82856C4;
 Tue, 27 Feb 2024 10:47:15 -0600 (CST)
Received: from mail.rptsys.com ([127.0.0.1])
 by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id C-yOD_MQEb8v; Tue, 27 Feb 2024 10:47:14 -0600 (CST)
Received: from [10.11.0.3] (5.edge.rptsys.com [23.155.224.38])
 by mail.rptsys.com (Postfix) with ESMTPSA id 6123C82855C7;
 Tue, 27 Feb 2024 10:47:14 -0600 (CST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ddb1516c-d58f-11ee-afd6-a90da7624cb6
DKIM-Filter: OpenDKIM Filter v2.10.3 mail.rptsys.com 0C45B82856C4
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD;
	t=1709052435; bh=BRkDsG3bTz7IfsDQLF6YdjxxW9JxgbwPIfTDzGmy4Vc=;
	h=Message-ID:Date:MIME-Version:To:From;
	b=COqxvmP0PAmUV4NAI6wO6PyUhkVFBz2iU29ebkO+hYHFdMbXI5OKFVd3JBp3nOxZv
	 bKwdckzg4sCqj3Yx7cXG5a6QfySCjGR/iFwJeoFKSFgU+NnmH/vwZvB27UqyYgx0bY
	 rq8S3Nu0Y3LUkqlsQRQbE2ifpVyUZ8zdAoSKNQ4Y=
X-Virus-Scanned: amavisd-new at rptsys.com
Message-ID: <3bacf6df-cb9f-4435-ae0a-6a228bc03aa8@raptorengineering.com>
Date: Tue, 27 Feb 2024 10:47:13 -0600
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] xen/lib: introduce generic find next bit operations
Content-Language: en-US
To: Oleksii <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
 <0d10bc590e3b6574eea35f72f324c509b4824d69.camel@gmail.com>
From: Shawn Anastasio <sanastasio@raptorengineering.com>
In-Reply-To: <0d10bc590e3b6574eea35f72f324c509b4824d69.camel@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 2/20/24 9:20 AM, Oleksii wrote:
> Hi Shawn,
> 

> Could you please take a look at the patch and PPC-related changes (
> xen/arch/ppc/include/asm/bitops.h )?
>

Hi Oleksii,

Sorry for the delay. This all looks good to me:

Acked-by: Shawn Anastasio <sanastasio@raptorengineering.com>

> Thanks in advance.
> 
> ~ Oleksii

Thanks,
Shawn


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 17:39:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 17:39:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686278.1068110 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf1QV-0007JB-BF; Tue, 27 Feb 2024 17:39:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686278.1068110; Tue, 27 Feb 2024 17:39:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf1QV-0007J4-7i; Tue, 27 Feb 2024 17:39:35 +0000
Received: by outflank-mailman (input) for mailman id 686278;
 Tue, 27 Feb 2024 17:39:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf1QU-0007GE-O3; Tue, 27 Feb 2024 17:39:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf1QU-0005tX-JE; Tue, 27 Feb 2024 17:39:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf1QU-0007CK-8y; Tue, 27 Feb 2024 17:39:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rf1QU-0001GL-8G; Tue, 27 Feb 2024 17:39:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=iXkpuc2ruNL4pviSitezW1Ab60+Diy4oYqqCyPWYtno=; b=0ant0EvVt6VKbY7I0WCg7j4aEj
	mVx6l9LADGkc2bca+zob6MAOu3fc8M4JrLnNhFzXpCAt7+BEfJkwmcGfXEHTAJPKLKruvF9Gt2uKy
	bBobuE75g8ermuBFHBezF1Zo7DHDg13AbLzM1BzMh8IaowL8463bX6U80reaNO6wUfnw=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184780-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184780: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=37541208f119a9c552c6c6c3246ea61be0d44035
X-Osstest-Versions-That:
    xen=03fb5f503cb5faa0556b23d594496719ced6a11b
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 17:39:34 +0000

flight 184780 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184780/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184776
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184776
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184776
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184776
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184776
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184776
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184776
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184776
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184776
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184776
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184776
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184776
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  37541208f119a9c552c6c6c3246ea61be0d44035
baseline version:
 xen                  03fb5f503cb5faa0556b23d594496719ced6a11b

Last test of basis   184776  2024-02-26 23:10:57 Z    0 days
Testing same since   184780  2024-02-27 09:47:29 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   03fb5f503c..37541208f1  37541208f119a9c552c6c6c3246ea61be0d44035 -> master


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 18:00:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 18:00:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686296.1068143 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf1kV-0003Ef-71; Tue, 27 Feb 2024 18:00:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686296.1068143; Tue, 27 Feb 2024 18:00:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf1kV-0003EY-3t; Tue, 27 Feb 2024 18:00:15 +0000
Received: by outflank-mailman (input) for mailman id 686296;
 Tue, 27 Feb 2024 18:00:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=L8US=KE=bombadil.srs.infradead.org=BATV+5d0d8640c4fce96ce78f+7492+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rf1kU-0003ES-0h
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 18:00:14 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0c9d5293-d59a-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 19:00:11 +0100 (CET)
Received: from hch by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red
 Hat Linux)) id 1rf1kA-00000006L78-3JBT;
 Tue, 27 Feb 2024 17:59:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c9d5293-d59a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Type:MIME-Version
	:References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=GukGcLj9PQJkkJdIQj2GR9q/JvN/F9bhdGOfMYM4O/U=; b=XwrNH63ffZ7GWr5zMD/MMNx9XI
	v9aKaslPO74VTtTivC5ZzlDmyM/6RnQ6cWIobn9vQYSp0ptp30DwyT/kCVaXr+GrAXmk1GUUxWnS8
	DyBGtLgUkEI4F5RnePXoVclfk6mgvIZ2f97YaRsMb0qfH+scl3eoThCPCW7OHCHjk3Ips9pyendm4
	7Qy6XJfkHoF79fYInWSFYVun7XXy+OYAuvZQT8ctwRRbJGmiwvXPaP6PpCZdc832OybgiyC8SAnJv
	rjm5Ybep2LF4gvmEhBTgaZR8l9m7fJGKIne09MLk5dj2tDRxjYnE3qEHrtHixqQ2CRTi0UmWmpsah
	oSGmJfgw==;
Date: Tue, 27 Feb 2024 09:59:54 -0800
From: Christoph Hellwig <hch@infradead.org>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org,
	torvalds@linux-foundation.org, brho@google.com, hannes@cmpxchg.org,
	lstoakes@gmail.com, akpm@linux-foundation.org, urezki@gmail.com,
	hch@infradead.org, boris.ostrovsky@oracle.com,
	sstabellini@kernel.org, jgross@suse.com, linux-mm@kvack.org,
	xen-devel@lists.xenproject.org, kernel-team@fb.com
Subject: Re: [PATCH v2 bpf-next 3/3] mm: Introduce VM_SPARSE kind and
 vm_area_[un]map_pages().
Message-ID: <Zd4jGhvb-Utdo2jU@infradead.org>
References: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
 <20240223235728.13981-4-alexei.starovoitov@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240223235728.13981-4-alexei.starovoitov@gmail.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

> privately-managed pages into a sparse vm area with the following steps:
> 
>   area = get_vm_area(area_size, VM_SPARSE);  // at bpf prog verification time
>   vm_area_map_pages(area, kaddr, 1, page);   // on demand
>                     // it will return an error if kaddr is out of range
>   vm_area_unmap_pages(area, kaddr, 1);
>   free_vm_area(area);                        // after bpf prog is unloaded

I'm still wondering if this should just use an opaque cookie instead
of exposing the vm_area.  But otherwise this mostly looks fine to me.

> +	if (addr < (unsigned long)area->addr || (void *)end > area->addr + area->size)
> +		return -ERANGE;

This check is duplicated so many times that it really begs for a helper.

> +int vm_area_unmap_pages(struct vm_struct *area, unsigned long addr, unsigned int count)
> +{
> +	unsigned long size = ((unsigned long)count) * PAGE_SIZE;
> +	unsigned long end = addr + size;
> +
> +	if (WARN_ON_ONCE(!(area->flags & VM_SPARSE)))
> +		return -EINVAL;
> +	if (addr < (unsigned long)area->addr || (void *)end > area->addr + area->size)
> +		return -ERANGE;
> +
> +	vunmap_range(addr, end);
> +	return 0;

Does it make much sense to have an error return here vs just debug
checks?  It's not like the caller can do much if it violates these
basic invariants.


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 18:33:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 18:33:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686301.1068159 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf2GC-0007bh-NR; Tue, 27 Feb 2024 18:33:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686301.1068159; Tue, 27 Feb 2024 18:33:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf2GC-0007ba-Ks; Tue, 27 Feb 2024 18:33:00 +0000
Received: by outflank-mailman (input) for mailman id 686301;
 Tue, 27 Feb 2024 18:32:59 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=tHoq=KE=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rf2GB-0007bU-NS
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 18:32:59 +0000
Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com
 [2a00:1450:4864:20::335])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a1428eea-d59e-11ee-afd6-a90da7624cb6;
 Tue, 27 Feb 2024 19:32:58 +0100 (CET)
Received: by mail-wm1-x335.google.com with SMTP id
 5b1f17b1804b1-412949fd224so283285e9.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 10:32:58 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 q14-20020a05600c46ce00b00411a6ce0f99sm12029549wmo.24.2024.02.27.10.32.57
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 10:32:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a1428eea-d59e-11ee-afd6-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709058778; x=1709663578; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=6fWIBDBkp/IOrhGUXdzpoJf5SNpN+zqrsYGJcvpSxLw=;
        b=UJt11bwYBaEZBoR4MEHLFLtiPGrvWagDyTzUTjLgcSX01kIpEjmcaPIvQV70+G1nOs
         CG39yDfArNXM2AsQ5dz8X8nlJnMxBkXrmONjgJEfRnnq/xu4sty5KBglbFClJs/+Lbpu
         612KjPSIG8CcW5vGjxexE+17qKZJX8Zox+HAg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709058778; x=1709663578;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=6fWIBDBkp/IOrhGUXdzpoJf5SNpN+zqrsYGJcvpSxLw=;
        b=JTCjxj6cAJ9XKEkZKU10ByrnjCCVhT1JH9gFrHAr3+kTZ9w8AiNCar6oOdTpgAI4mJ
         X5WmqvKmDBETLxqZ+tzY8o7+WePCzr03pcP5BkyDB/Ac7dnimr/xgh67DPQPYYaa+/iq
         J767OmOCrYeCXA1aTG6NFclAiDrnrusIMedexsVx6g9yVW7fT8B+bs8url+LL0BIvYpl
         2ktTsndLnXVDVUIpH6kFtM/BigzFc/R1eKEqZBB6eYMch3jLpqoPQltiZJS5qrJ6N4eM
         42UafyDQA5CMXbdu0rhHC4ZvSh1m1oUEu5aa3Q/TIi68VR1+efJE81vp9rM5/aVh4RH3
         NB2g==
X-Gm-Message-State: AOJu0YztmLbi1uac9glQ/JucibksznryaAmWvYhmmOV6GAQeJJ8h/HS6
	7xeAK4nqUeoZE4BJQvRizN8PXoWraJelBk6JX5vj5KoBbCoyadh150IQ/wD9dVnt2zcnjapG6HW
	b
X-Google-Smtp-Source: AGHT+IEHtPX8kRyydQMiolCNd3RZbqr3tRifZF1ZBav4F/ZeadDK0TvXLmn+/huJ05RSW3kc+PCpxA==
X-Received: by 2002:a05:600c:4f4e:b0:410:78fb:bed2 with SMTP id m14-20020a05600c4f4e00b0041078fbbed2mr152750wmq.19.1709058777781;
        Tue, 27 Feb 2024 10:32:57 -0800 (PST)
Date: Tue, 27 Feb 2024 18:32:56 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Fouad Hilly <fouad.hilly@cloud.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>, Wei Liu <wl@xen.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH v2] tools/xentop: Add VBD3 support to xentop
Message-ID: <fc59e215-2aef-4e07-83df-b27a5cb79313@perard>
References: <20240227132628.2157031-1-fouad.hilly@cloud.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240227132628.2157031-1-fouad.hilly@cloud.com>

On Tue, Feb 27, 2024 at 01:26:28PM +0000, Fouad Hilly wrote:
> diff --git a/tools/libs/stat/xenstat_linux.c b/tools/libs/stat/xenstat_linux.c
> index cbba54aa83ee..6d82e204aad4 100644
> --- a/tools/libs/stat/xenstat_linux.c
> +++ b/tools/libs/stat/xenstat_linux.c
> @@ -390,6 +390,38 @@ void xenstat_uninit_networks(xenstat_handle * handle)
>  		fclose(priv->procnetdev);
>  }
>  
> +static int read_attributes_vbd3(char *vbd3_path, xenstat_vbd *vbd)

"const char *vbd3_path"?


>  static int read_attributes_vbd(const char *vbd_directory, const char *what, char *ret, int cap)
>  {
>  	static char file_name[80];
> @@ -438,7 +470,7 @@ int xenstat_collect_vbds(xenstat_node * node)
>  		int ret;
>  		char buf[256];
>  
> -		ret = sscanf(dp->d_name, "%3s-%u-%u", buf, &domid, &vbd.dev);
> +		ret = sscanf(dp->d_name, "%255[^-]-%u-%u", buf, &domid, &vbd.dev);

255 is overly ambitious, but it match the size of buf, so I guess it's
kind of ok, even if unnecessary.

>  		if (ret != 3)
>  			continue;
>  		if (!(strstr(buf, "vbd")) && !(strstr(buf, "tap")))
> @@ -448,6 +480,8 @@ int xenstat_collect_vbds(xenstat_node * node)
>  			vbd.back_type = 1;
>  		else if (strcmp(buf,"tap") == 0)
>  			vbd.back_type = 2;
> +		else if (strcmp(buf,"vbd3") == 0)
> +			vbd.back_type = 3;

Yay for magic number... Do you think you could introduce an enum or
define to replace this "3" by a meaningful? Maybe something like
XENSTAT_VBD_TYPE_VBD3, (name derived from the existing function
xenstat_vbd_type()).

I'd like at least to replace the "3". But if you feel like having
another patch to replace the "2" and "1", that would be a plus.

>  		else
>  			vbd.back_type = 0;
>  
> @@ -479,6 +513,35 @@ int xenstat_collect_vbds(xenstat_node * node)
>  				vbd.error = 1;
>  			}
>  		}
> +		else if (vbd.back_type == 3)
> +		{
> +			char *td3_pid;
> +			char *path;
> +
> +			vbd.back_type = 3;

`back_type` should already be 3 ;-).

> +			vbd.error = 0;
> +
> +			if (asprintf(&path, "/local/domain/0/backend/vbd3/%u/%u/kthread-pid", domid, vbd.dev) < 0)
> +				continue;
> +
> +			td3_pid = xs_read(node->handle->xshandle, XBT_NULL, path, NULL);
> +
> +			free(path);
> +
> +			if (td3_pid == NULL)
> +				continue;
> +
> +			if (asprintf(&path, "/dev/shm/td3-%s/vbd-%u-%u", td3_pid, domid, vbd.dev) < 0) {
> +				free(td3_pid);
> +				continue;
> +			}
> +
> +			if (read_attributes_vbd3(path, &vbd) < 0)
> +				vbd.error = 1;

Why sometime we do "continue" and sometime we do "vbd.error=1"?

> +
> +			free(td3_pid);
> +			free(path);
> +		}
>  		else
>  		{
>  			vbd.error = 1;
> diff --git a/tools/libs/stat/xenstat_priv.h b/tools/libs/stat/xenstat_priv.h
> index 4eb44a8ebb84..c3a9635240e9 100644
> --- a/tools/libs/stat/xenstat_priv.h
> +++ b/tools/libs/stat/xenstat_priv.h
> @@ -98,6 +98,22 @@ struct xenstat_vbd {
>  	unsigned long long wr_sects;
>  };
>  
> +struct vbd3_stats {
> +	uint32_t version;
> +	uint32_t __pad;
> +	uint64_t oo_reqs;
> +	uint64_t read_reqs_submitted;
> +	uint64_t read_reqs_completed;
> +	uint64_t read_sectors;
> +	uint64_t read_total_ticks;
> +	uint64_t write_reqs_submitted;
> +	uint64_t write_reqs_completed;
> +	uint64_t write_sectors;
> +	uint64_t write_total_ticks;
> +	uint64_t io_errors;
> +	uint64_t flags;
> +};

Is that a binary interface for an external project? I'm asking because
`__pad` would seems useless otherwise.
If it is part of an interface, please add a comment about it, add a link
to the project/source where this interface is described, and where is
the canonical location? Also, is there an header we could maybe just
use if it's in the system or an header we could import into the
repository?

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 18:52:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 18:52:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686306.1068168 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf2ZE-0002Ed-Dc; Tue, 27 Feb 2024 18:52:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686306.1068168; Tue, 27 Feb 2024 18:52:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf2ZE-0002EW-AY; Tue, 27 Feb 2024 18:52:40 +0000
Received: by outflank-mailman (input) for mailman id 686306;
 Tue, 27 Feb 2024 18:52:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf2ZC-0002EM-OD; Tue, 27 Feb 2024 18:52:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf2ZC-0007Jn-Kc; Tue, 27 Feb 2024 18:52:38 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf2ZC-0003An-Ab; Tue, 27 Feb 2024 18:52:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rf2ZC-00081b-AA; Tue, 27 Feb 2024 18:52:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=oyURZeiZcGFZABRZl8ciN3cd+PfELEnBxJ8IYObGrOk=; b=oWHKl9Z2UYJ1LOh1P8dRo6n82p
	UbWQ5SbKgl3sN86XDxWHKw6Bgpj7XVMXh3JSxydmZa1ryGbbMJmiIWuoRNXKJS6IiV2rk4pYNKDCV
	nJ0k+lxTkxQQQIo+4X2QtdTedDJYwMUs97H4FO6noQWZa6tM7tC5Xen3KKpFEm505gyg=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184793-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184793: regressions - trouble: blocked/fail
X-Osstest-Failures:
    xen-unstable-smoke:build-amd64:xen-build:fail:regression
    xen-unstable-smoke:build-arm64-xsm:xen-build:fail:regression
    xen-unstable-smoke:build-armhf:xen-build:fail:regression
    xen-unstable-smoke:build-amd64-libvirt:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-amd64-amd64-libvirt:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-amd64-amd64-xl-qemuu-debianhvm-amd64:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:build-check(1):blocked:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:build-check(1):blocked:nonblocking
X-Osstest-Versions-This:
    xen=576528a2a742069af203e90c613c5c93e23c9755
X-Osstest-Versions-That:
    xen=ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 18:52:38 +0000

flight 184793 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184793/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64                   6 xen-build                fail REGR. vs. 184781
 build-arm64-xsm               6 xen-build                fail REGR. vs. 184781
 build-armhf                   6 xen-build                fail REGR. vs. 184781

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt           1 build-check(1)               blocked  n/a
 test-amd64-amd64-libvirt      1 build-check(1)               blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)        blocked n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               blocked  n/a
 test-armhf-armhf-xl           1 build-check(1)               blocked  n/a

version targeted for testing:
 xen                  576528a2a742069af203e90c613c5c93e23c9755
baseline version:
 xen                  ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87

Last test of basis   184781  2024-02-27 10:00:25 Z    0 days
Testing same since   184787  2024-02-27 14:02:12 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jens Wiklander <jens.wiklander@linaro.org>
  Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stewart Hildebrand <stewart.hildebrand@amd.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>

jobs:
 build-arm64-xsm                                              fail    
 build-amd64                                                  fail    
 build-armhf                                                  fail    
 build-amd64-libvirt                                          blocked 
 test-armhf-armhf-xl                                          blocked 
 test-arm64-arm64-xl-xsm                                      blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    blocked 
 test-amd64-amd64-libvirt                                     blocked 


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 365 lines long.)


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 21:22:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 21:22:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686317.1068191 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf4tj-0002g8-2e; Tue, 27 Feb 2024 21:21:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686317.1068191; Tue, 27 Feb 2024 21:21:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf4ti-0002g1-VW; Tue, 27 Feb 2024 21:21:58 +0000
Received: by outflank-mailman (input) for mailman id 686317;
 Tue, 27 Feb 2024 21:21:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=PYXm=KE=kernel.dk=axboe@srs-se1.protection.inumbo.net>)
 id 1rf4tg-0002fv-TL
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 21:21:57 +0000
Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com
 [2607:f8b0:4864:20::529])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3a8a7896-d5b6-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 22:21:54 +0100 (CET)
Received: by mail-pg1-x529.google.com with SMTP id
 41be03b00d2f7-5d862e8b163so1802684a12.1
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 13:21:54 -0800 (PST)
Received: from [127.0.0.1] ([198.8.77.194]) by smtp.gmail.com with ESMTPSA id
 r10-20020a635d0a000000b005dc36279d6dsm6125145pgb.73.2024.02.27.13.21.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 27 Feb 2024 13:21:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3a8a7896-d5b6-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1709068913; x=1709673713; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:date:message-id:subject
         :references:in-reply-to:to:from:from:to:cc:subject:date:message-id
         :reply-to;
        bh=UANm1dgS5T0WOHSiSrzCuBbRubRxDNMtxNP78En/ilQ=;
        b=uqyWEx2lFUaEwblYV8FlU1y84VKAHaWPVI+4r8CRg1wvGxhQVkUZajtcwoCFyUrBVg
         o3cRb7WvREtbZOXo1mJTpHcl2EfysmdTBgEVq5q/yl3jUn+7IneBuaLoEI6eUqhjf3Ai
         6uKrekVqOzNuP9qKms3meWBvZCy+cfX6XS9Sln6RIjoRTcb3CNDs1EmfrlkLtq7uzdGs
         G9Nyr7aOrpc9JT1HqdiCp1vgZ02DIbFUu4rfMzVzzhtdO68Ep4eBFryU6cbMGmBJwYp/
         A7zyPQod6Ly9Y4/RxCzUffKIU9055z6fvZV5L8OVehzS8+p9uAt9X6iLi2/APOBvbWqe
         fr5g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709068913; x=1709673713;
        h=content-transfer-encoding:mime-version:date:message-id:subject
         :references:in-reply-to:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=UANm1dgS5T0WOHSiSrzCuBbRubRxDNMtxNP78En/ilQ=;
        b=g85Z9U5HTctJG/rZOAmdU6jtDjFtjdLqPIm6ACS6PE+Zv84CgfrIfmELeUSjKSe2/E
         DsLQaKmneHtrkSDkFkqWWCY+gO1WR6aHGHECr69+GKxwgTZg9DuihRi1kKiGeWWxqeRG
         x+iBm4KjvkOO1OOjzE/ShF/gsg/y0RGtPV2ANYQxkUiKUr3qIMCt5zqi8XaB5xOCezUM
         vHFb7P7zi4cOr8/0QhBK3C/5ekAizVtUSgtoL3hDHybaNhYGPT7atrKqbJVaLgKFWeOu
         7qgcTi869kgrW9wHO1SgsMC+3STfg4TAWJuqoCOk0JD6i1K2wGP7k+un69Tom1M9ayaN
         ukEQ==
X-Forwarded-Encrypted: i=1; AJvYcCUabo+ssVHoFmDoJr2yj9a8lPLkTpyc91X1QA2/814nEk40NqDdGJBlFQ//imBJgood6dvwip9hOQjyqx05yxJPXTT88s2XAvLnfAYf+g4=
X-Gm-Message-State: AOJu0YzXaEwHHzriQe3w/KE5DZrsDBBLrQAVA1/Df29BQKREBg+QBoc8
	R8iQDmYHSTWG+TIcPVoWT/6yTXsGfGe2bQxJuUZ/O+ZYqpXsKCG7AL4m4lJ77Mg=
X-Google-Smtp-Source: AGHT+IF8ppKk3qSRiiw1IL2DaiUC6GURziSEFqA+jG8dpM8Ewv32Uofv42ldLmpmg5FLD3yIW5RJVg==
X-Received: by 2002:a05:6a20:4283:b0:1a0:e557:7ec3 with SMTP id o3-20020a056a20428300b001a0e5577ec3mr14434749pzj.1.1709068913195;
        Tue, 27 Feb 2024 13:21:53 -0800 (PST)
From: Jens Axboe <axboe@kernel.dk>
To: Juergen Gross <jgross@suse.com>, 
 Stefano Stabellini <sstabellini@kernel.org>, 
 Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>, 
 =?utf-8?q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, 
 xen-devel@lists.xenproject.org, linux-block@vger.kernel.org, 
 Christoph Hellwig <hch@lst.de>
In-Reply-To: <20240221125845.3610668-1-hch@lst.de>
References: <20240221125845.3610668-1-hch@lst.de>
Subject: Re: convert xen-blkfront to atomic queue limit updates v2
Message-Id: <170906891213.1104664.4203607989260212614.b4-ty@kernel.dk>
Date: Tue, 27 Feb 2024 14:21:52 -0700
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
X-Mailer: b4 0.12.5-dev-2aabd


On Wed, 21 Feb 2024 13:58:41 +0100, Christoph Hellwig wrote:
> this series converts xen-blkfront to the new atomic queue limits update
> API in the block tree.  I don't have a Xen setup so this is compile
> tested only.
> 
> Changes since v1:
>  - constify the info argument to blkif_set_queue_limits
>  - remove a spurious word from a commit message
> 
> [...]

Applied, thanks!

[1/4] xen-blkfront: set max_discard/secure erase limits to UINT_MAX
      commit: 4a718d7dbab873bc24034fc865d3a5442632d1fd
[2/4] xen-blkfront: rely on the default discard granularity
      commit: 738be136327a56e5a67e1942a2c318fb91914a3f
[3/4] xen-blkfront: don't redundantly set max_sements in blkif_recover
      commit: 4f81b87d91be2a00195f85847d040c2276cac2ae
[4/4] xen-blkfront: atomically update queue limits
      commit: ba3f67c1163812b5d7ec33705c31edaa30ce6c51

Best regards,
-- 
Jens Axboe





From xen-devel-bounces@lists.xenproject.org Tue Feb 27 22:51:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 22:51:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686331.1068210 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf6Hn-0004LU-J3; Tue, 27 Feb 2024 22:50:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686331.1068210; Tue, 27 Feb 2024 22:50:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf6Hn-0004LN-GK; Tue, 27 Feb 2024 22:50:55 +0000
Received: by outflank-mailman (input) for mailman id 686331;
 Tue, 27 Feb 2024 22:50:54 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf6Hm-0004LD-MJ; Tue, 27 Feb 2024 22:50:54 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf6Hm-0003FK-Jg; Tue, 27 Feb 2024 22:50:54 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf6Hm-0001xM-A8; Tue, 27 Feb 2024 22:50:54 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rf6Hm-0002mr-9e; Tue, 27 Feb 2024 22:50:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=QphL8yFsnrpmvHyRj9bF6DmAqnrdJA8v9Rle2mN31vo=; b=o2Bi4MUsmwc8Dshvun05cLgwTA
	oM4qWA5XE533DFw1HTT3RR9vmhOo64agc/t2BSnl9PZ7ITR2Soh5INKIsbB3pputCnqGZl0HLVQg1
	GzDs/uUQeLHowqZweKv3it485nDE/Q65184D370ssevg2N4h23CUPQrIYZPkjSEt5iXM=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184798-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184798: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=801c4e72621d5541d249af1938a77466f0dcfe88
X-Osstest-Versions-That:
    xen=ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 22:50:54 +0000

flight 184798 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184798/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  801c4e72621d5541d249af1938a77466f0dcfe88
baseline version:
 xen                  ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87

Last test of basis   184781  2024-02-27 10:00:25 Z    0 days
Failing since        184787  2024-02-27 14:02:12 Z    0 days    3 attempts
Testing same since   184798  2024-02-27 19:00:30 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jens Wiklander <jens.wiklander@linaro.org>
  Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Stewart Hildebrand <stewart.hildebrand@amd.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ca7c872c70..801c4e7262  801c4e72621d5541d249af1938a77466f0dcfe88 -> smoke


From xen-devel-bounces@lists.xenproject.org Tue Feb 27 23:43:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Tue, 27 Feb 2024 23:43:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686341.1068224 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf76p-0001WU-IJ; Tue, 27 Feb 2024 23:43:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686341.1068224; Tue, 27 Feb 2024 23:43:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf76p-0001WN-D1; Tue, 27 Feb 2024 23:43:39 +0000
Received: by outflank-mailman (input) for mailman id 686341;
 Tue, 27 Feb 2024 23:43:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf76o-0001WD-O4; Tue, 27 Feb 2024 23:43:38 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf76o-0004Ab-KE; Tue, 27 Feb 2024 23:43:38 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rf76o-0004mc-7P; Tue, 27 Feb 2024 23:43:38 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rf76o-0005hu-6x; Tue, 27 Feb 2024 23:43:38 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=st88U/HaJfNMkOlnRxjKgxT+zsj3cNl/F/e57Qr6TWE=; b=skNvFuKWGqiOEEN48MIDJ7jwYu
	jgW6wxBnwpmlPlyu0Q++gCr5hmkh1tIQLq7bCVxUppSBxAE73oziOQfP7iva6pBs5N/4RTgXszt8i
	ceYNLzMh/mJXjJ4TKF55dJOP8a1SOLdHMcXlyR//67zds8hmlpNvptK+dXrQR6yuKPss=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184783-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.18-testing test] 184783: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.18-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=58bb8115104c9fca749ee4cfcd3579ac1ed644db
X-Osstest-Versions-That:
    xen=b1fdd7d0e47e0831ac7a99d0417385fc10d3068c
From: osstest service owner <osstest-admin@xenproject.org>
Date: Tue, 27 Feb 2024 23:43:38 +0000

flight 184783 xen-4.18-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184783/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184553
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184553
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184553
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184553
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184553
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184553
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184553
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184553
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184553
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184553
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184553
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184553
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  58bb8115104c9fca749ee4cfcd3579ac1ed644db
baseline version:
 xen                  b1fdd7d0e47e0831ac7a99d0417385fc10d3068c

Last test of basis   184553  2024-02-01 17:08:47 Z   26 days
Testing same since   184783  2024-02-27 13:07:15 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Cyril Rébert (zithro) <slack@rabbit.lu>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  oxjo <oxjo@proton.me>
  Paul Durrant <paul@xen.org>
  Petr Beneš <w1benny@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   b1fdd7d0e4..58bb811510  58bb8115104c9fca749ee4cfcd3579ac1ed644db -> stable-4.18


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 01:31:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 01:31:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686368.1068289 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf8nP-00048g-DT; Wed, 28 Feb 2024 01:31:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686368.1068289; Wed, 28 Feb 2024 01:31:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf8nP-00048Y-6n; Wed, 28 Feb 2024 01:31:43 +0000
Received: by outflank-mailman (input) for mailman id 686368;
 Wed, 28 Feb 2024 01:31:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=fDsi=KF=gmail.com=alexei.starovoitov@srs-se1.protection.inumbo.net>)
 id 1rf8nO-00048S-GS
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 01:31:42 +0000
Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com
 [2a00:1450:4864:20::32b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1f155abc-d5d9-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 02:31:40 +0100 (CET)
Received: by mail-wm1-x32b.google.com with SMTP id
 5b1f17b1804b1-41241f64c6bso38314935e9.0
 for <xen-devel@lists.xenproject.org>; Tue, 27 Feb 2024 17:31:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1f155abc-d5d9-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709083900; x=1709688700; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=tQB+JZ1G68ICyO1N3p0oqOVpl3RnCmkeQUi97bJMYlw=;
        b=lpV8t09eHyhd9CiSOkeN4xvqZ0whcw0CYYOaIZFc8JF3KInWBVEbCapI+QvC1x3EE4
         TTWRscfveLLgchiK/HMULXkwQHIhEvHfE9sGT4Ktsy9e6OgotyhiCHRC01/rRLB6sVlT
         DHFSixqz8XBhpwff9iNRraxQ99/bz0I8JeAKos+gDUtr6KGsFXG00kEM93HdQBZ/h55F
         bXbLkDyNdQba8ISStrEuaec3iZ98OgrrJURLiVJMHfyE8D4lfEp2kjzoh2ZccreOXRKt
         DwlBMvJYD7DH070lZ5IJa/OWS0OF1dgEIvJyvW5T73YYrI3BIOZqeFUzLp/bdfkWKA5G
         3ZDw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709083900; x=1709688700;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=tQB+JZ1G68ICyO1N3p0oqOVpl3RnCmkeQUi97bJMYlw=;
        b=DFmsnXOxJ8jrJoRqXDWo+RV8nJHSaVQyx7hdoFBoqgkg85XmQDD7kOaT1fmxqZ7unq
         7jY4lT/qVSePf0hYpYdhS6ZzgCwx5aVyGJidlJUCh/phYmFFOpgyw3Q7VNQJSQ8bHshP
         f3AKZesSC40veAniBh9tL6F3GLW49TNCeDd7Kp71XvHApxH1Anz1GbfYbdPHrFxmz/pL
         w8hxb/AGEkX82fPIfJER25Ii/jQPBzdbUwb4xFIrie4bTt4TV0qMadRSV05tKElfXRhx
         NPt8UMcrHVjE3oq8z2UBMgFeB8zKFBLBkfI+4ZFQOJ1rISGJ93oZOyYSAqo/LhjxzEwF
         iIGQ==
X-Forwarded-Encrypted: i=1; AJvYcCXWt5Xief1EUIZ5umDEric2nQdlC8ojxKB4okUpNlSMB/PER397vU2NiTaZ99ojCAwDw++pKBnQGWnLePrYjIkU64Mb28H6Fw9Vhv1XwY8=
X-Gm-Message-State: AOJu0YyZS6k0f9enH+Uv7yyip329PTacK+WRhPbCJYkYxF0hGs9VZJ6N
	FbKUKTbsW72sjekf+UGLg4s3Odmqx3X9YB+i2uF2ItWHyiUTsIuTSoZvPhKizMqghqLDYYkRwa4
	zP7db/mvLDX3GoTH/3PJ3lxHFqWY=
X-Google-Smtp-Source: AGHT+IHW0RvW1Yw8eRC97o+L4oxzxSjZSar1xugDudti2CXKZy7764qOr6NgwPLSJiP97CBaYnawADxltOlqy8AvltM=
X-Received: by 2002:adf:db07:0:b0:33d:82ae:67f8 with SMTP id
 s7-20020adfdb07000000b0033d82ae67f8mr8131519wri.50.1709083899508; Tue, 27 Feb
 2024 17:31:39 -0800 (PST)
MIME-Version: 1.0
References: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
 <20240223235728.13981-4-alexei.starovoitov@gmail.com> <Zd4jGhvb-Utdo2jU@infradead.org>
In-Reply-To: <Zd4jGhvb-Utdo2jU@infradead.org>
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Date: Tue, 27 Feb 2024 17:31:28 -0800
Message-ID: <CAADnVQ+f06b1hDrAyLM-OrzDfEEa=jtamJOKfEnEo4ewKPV0cA@mail.gmail.com>
Subject: Re: [PATCH v2 bpf-next 3/3] mm: Introduce VM_SPARSE kind and vm_area_[un]map_pages().
To: Christoph Hellwig <hch@infradead.org>
Cc: bpf <bpf@vger.kernel.org>, Daniel Borkmann <daniel@iogearbox.net>, 
	Andrii Nakryiko <andrii@kernel.org>, Linus Torvalds <torvalds@linux-foundation.org>, 
	Barret Rhoden <brho@google.com>, Johannes Weiner <hannes@cmpxchg.org>, 
	Lorenzo Stoakes <lstoakes@gmail.com>, Andrew Morton <akpm@linux-foundation.org>, 
	Uladzislau Rezki <urezki@gmail.com>, Boris Ostrovsky <boris.ostrovsky@oracle.com>, sstabellini@kernel.org, 
	Juergen Gross <jgross@suse.com>, linux-mm <linux-mm@kvack.org>, xen-devel@lists.xenproject.org, 
	Kernel Team <kernel-team@fb.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 27, 2024 at 9:59=E2=80=AFAM Christoph Hellwig <hch@infradead.or=
g> wrote:
>
> > privately-managed pages into a sparse vm area with the following steps:
> >
> >   area =3D get_vm_area(area_size, VM_SPARSE);  // at bpf prog verificat=
ion time
> >   vm_area_map_pages(area, kaddr, 1, page);   // on demand
> >                     // it will return an error if kaddr is out of range
> >   vm_area_unmap_pages(area, kaddr, 1);
> >   free_vm_area(area);                        // after bpf prog is unloa=
ded
>
> I'm still wondering if this should just use an opaque cookie instead
> of exposing the vm_area.  But otherwise this mostly looks fine to me.

What would it look like with a cookie?
A static inline wrapper around get_vm_area() that returns area->addr ?
And the start address of vmap range will be such a cookie?

Then vm_area_map_pages() will be doing find_vm_area() for kaddr
to check that vm_area->flag & VM_SPARSE ?
That's fine,
but what would be an equivalent of void free_vm_area(struct vm_struct *area=
) ?
Another static inline wrapper similar to remove_vm_area()
that also does kfree(area); ?

Fine by me, but api isn't user friendly with such obfuscation.

I guess I don't understand the motivation to hide 'struct vm_struct *'.

> > +     if (addr < (unsigned long)area->addr || (void *)end > area->addr =
+ area->size)
> > +             return -ERANGE;
>
> This check is duplicated so many times that it really begs for a helper.

ok. will do.

> > +int vm_area_unmap_pages(struct vm_struct *area, unsigned long addr, un=
signed int count)
> > +{
> > +     unsigned long size =3D ((unsigned long)count) * PAGE_SIZE;
> > +     unsigned long end =3D addr + size;
> > +
> > +     if (WARN_ON_ONCE(!(area->flags & VM_SPARSE)))
> > +             return -EINVAL;
> > +     if (addr < (unsigned long)area->addr || (void *)end > area->addr =
+ area->size)
> > +             return -ERANGE;
> > +
> > +     vunmap_range(addr, end);
> > +     return 0;
>
> Does it make much sense to have an error return here vs just debug
> checks?  It's not like the caller can do much if it violates these
> basic invariants.

Ok. Will switch to void return.

Will reduce commit line logs to 75 chars in all patches as suggested.

re: VM_GRANT_TABLE or VM_XEN_GRANT_TABLE suggestion for patch 2.

I'm not sure it fits, since only one of get_vm_area() in xen code
is a grant table related. The other one is for xenbus that
creates a shared memory ring between domains.
So I'm planning to keep it as VM_XEN in the next revision unless
folks come up with a better name.

Thanks for the reviews.


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 01:59:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 01:59:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686371.1068299 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf9Dg-00072H-Cq; Wed, 28 Feb 2024 01:58:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686371.1068299; Wed, 28 Feb 2024 01:58:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf9Dg-00072A-8s; Wed, 28 Feb 2024 01:58:52 +0000
Received: by outflank-mailman (input) for mailman id 686371;
 Wed, 28 Feb 2024 01:58:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Gbmv=KF=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1rf9De-000724-Py
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 01:58:50 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e936becd-d5dc-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 02:58:49 +0100 (CET)
Received: from BN0PR02CA0060.namprd02.prod.outlook.com (2603:10b6:408:e5::35)
 by BY5PR12MB4292.namprd12.prod.outlook.com (2603:10b6:a03:212::12)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.36; Wed, 28 Feb
 2024 01:58:43 +0000
Received: from BN3PEPF0000B078.namprd04.prod.outlook.com
 (2603:10b6:408:e5:cafe::15) by BN0PR02CA0060.outlook.office365.com
 (2603:10b6:408:e5::35) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.49 via Frontend
 Transport; Wed, 28 Feb 2024 01:58:42 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN3PEPF0000B078.mail.protection.outlook.com (10.167.243.123) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 01:58:42 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 19:58:41 -0600
Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 27 Feb 2024 19:58:39 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e936becd-d5dc-11ee-afd7-a90da7624cb6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=H6H+WbmxVg3p/IEnPkklGZ2IpZZy2Nces89eg20oviLGWv6opACTRzh2fN3Sa1oFgAmO1L4yH6VGgXINsPI4AA+O4mEpI4jLU5y0sbZk8ThTeeturMf/R5ioxF44y6Sjtg4QYW7cuZRVr1k4Af5jlkTPKBSCeaFHZ//Pi+A6ROb2IZqOvNM0z49OD/VwuA8bJFilkIpChQ9z/9Jf9xHtqsgxlOtlGQEtOimur6emVoCMF6IkwrX7lbLFtOeYpIiG4Ke0G8F+5vkBEJXhTn7W19y98+9okJA04sGX5qRzItX/nPdz0yexT+jsith1SDmZZ4J0NR96kRjnpTfp8GNaeQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=7st0nfvYqfoUDlhUdhHj6eV1a5dpb/v+ibFIxsOD8gg=;
 b=nHJa4MeMmjA2/OEv736MOubOOqWEwPPRfVsnghcPZjzeReQz05Hp/gAOuYflQuwRD9NYGmIP6QYtTJF1Kp04+NSo61uuUcyImgk2wZASVnMxyDC54f15Ek5DCJCDtqJxusTioRwnUgfdNoKOV3mPGDdnfsPNwZnd9Vn0GMtxHrQ0yAz0l29F9dnSDtw3N4EvKq31iyDmHfs7EwRQ/m8ZHrWNzdyatyXD73W9UzKXM+h/jZkqaSjw7jF2m8dFLiB7FMnJoEQIJsEh2TQaqpFr5pz4MkJTqysHrUJDbxqEx9wLs2ldItN+wB9F8Xj82/tXSWJKCpTnjgCZVexxvQspjg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=7st0nfvYqfoUDlhUdhHj6eV1a5dpb/v+ibFIxsOD8gg=;
 b=oO8LAzeeI3TOnkXGBH342y/ddQ0Z5F/BIsC0xXm+PVGJyj44iQyK0VB1mqLiWDvozDgEU7PMbZs/HYzsa2mDkwk63ZKziWuthyC9R3kyQ6VhLSekV8zauCMbDC3C1FKKgwFAzpX6XaSYOcpzucnEFwcJW+gwSjQL3S6kp8koE1A=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Henry Wang <xin.wang2@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Henry Wang <xin.wang2@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, Henry Wang <Henry.Wang@arm.com>
Subject: [PATCH v2] xen/arm: Set correct per-cpu cpu_core_mask
Date: Wed, 28 Feb 2024 09:58:22 +0800
Message-ID: <20240228015822.56108-1-xin.wang2@amd.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN3PEPF0000B078:EE_|BY5PR12MB4292:EE_
X-MS-Office365-Filtering-Correlation-Id: 06f777dd-2c20-4af4-3de7-08dc3800ca35
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	h0uB01p654lLevUXFjF2SYWz1p05XSx98E7ANH+bjkAQztnYzmiFBnr26sQVOKp5VNgX6daSE+tHoSxJq96U9XGpe1xpZo5uGwHe6Y9aBGqYgNGLYFPTbDDZBdbya0J2bZaaNAtot5AZr/3IOse5+gzt5rvTIsOf5gOMJYmt5MmDLZQvn9wRTRRIqJo4FJng203EVYJ7KCO+1CBHuAidL8rsqvoQq/OarG45lbgfJXMssuZK94feD55Dulskaim4JXfnRoJ/LQe+ewgET6fXfKDrYqx945s70A05qIJ04VPXymwAviatZKDWMHIokwwWHQx+ntn0l8x0RiKC2WD808rSp9QxeZngMUfdzwwL9gikHzrCMs5A5S+GnXrZzaCb45EHlO8xpsDjIHPRHdVQd8Y3epw4bqs4u0B85w9AKO3nt9EV7aZybdCrRQAhbvRUJ0Xji+7FH83ls8kkLeIgZ0w9oJC2LjzcPvtHRKxwweuEtH5cTo3CusZ1yXJHCRM0qc8OwDHQzhPO8sWN/o0fLc39ZfKeqj6txpgDWj36tPO3vWI000r7OcxJQe7FZQmu5Kfg+hvLYzE+QreHBV2NxDH7+i56jCYHfFgf7FEqFDABqRmgiKHVzsfVzfb8CL7D3Zi7xEDlRPBDC0IAV0z7csSTBFHqOHu7/gCheM4ZEYghlbv1m8jBlRruRGKMPL1h9+JTOElUXgDSY7FB+xxwsXBSRDKzSKlVxBbdbWNum6HGOqt1dlo84QCMfhCNPkcq
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 01:58:42.2941
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 06f777dd-2c20-4af4-3de7-08dc3800ca35
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN3PEPF0000B078.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4292

In the common sysctl command XEN_SYSCTL_physinfo, the value of
cores_per_socket is calculated based on the cpu_core_mask of CPU0.
Currently on Arm this is a fixed value 1 (can be checked via xl info),
which is not correct. This is because during the Arm CPU online
process at boot time, setup_cpu_sibling_map() only sets the per-cpu
cpu_core_mask for itself.

cores_per_socket refers to the number of cores that belong to the same
socket (NUMA node). Currently Xen on Arm does not support physical
CPU hotplug and NUMA, also we assume there is no multithread. Therefore
cores_per_socket means all possible CPUs detected from the device
tree. Setting the per-cpu cpu_core_mask in setup_cpu_sibling_map()
accordingly. Drop the in-code comment which seems to be outdated.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Signed-off-by: Henry Wang <xin.wang2@amd.com>
---
v2:
- Do not do the multithread check.
---
 xen/arch/arm/smpboot.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index a84e706d77..d9ebd55d4a 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -66,7 +66,6 @@ static bool cpu_is_dead;
 
 /* ID of the PCPU we're running on */
 DEFINE_PER_CPU(unsigned int, cpu_id);
-/* XXX these seem awfully x86ish... */
 /* representing HT siblings of each logical CPU */
 DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
 /* representing HT and core siblings of each logical CPU */
@@ -89,6 +88,10 @@ static int setup_cpu_sibling_map(int cpu)
     cpumask_set_cpu(cpu, per_cpu(cpu_sibling_mask, cpu));
     cpumask_set_cpu(cpu, per_cpu(cpu_core_mask, cpu));
 
+    /* Currently we assume there is no multithread. */
+    cpumask_or(per_cpu(cpu_core_mask, cpu),
+               per_cpu(cpu_core_mask, cpu), &cpu_possible_map);
+
     return 0;
 }
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 02:01:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 02:01:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686373.1068308 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf9G3-0000Xm-O0; Wed, 28 Feb 2024 02:01:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686373.1068308; Wed, 28 Feb 2024 02:01:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf9G3-0000Xf-LB; Wed, 28 Feb 2024 02:01:19 +0000
Received: by outflank-mailman (input) for mailman id 686373;
 Wed, 28 Feb 2024 02:01:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=msSf=KF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rf9G2-0000XY-1a
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 02:01:18 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 409830df-d5dd-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 03:01:15 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 9AD8261321;
 Wed, 28 Feb 2024 02:01:13 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 137FEC433F1;
 Wed, 28 Feb 2024 02:01:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 409830df-d5dd-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709085673;
	bh=e4ZteuGmXoaqpCbvsXp1fdtBhk7Px+vs4YIh/BOFNo8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=OLPn2fmwpfLFZ/M8wqVqdSavLYlPVXa0oS1FIBKP4MNXDgYVO6r5D9SxntbRLfx1M
	 xrt3vO01vIUPbvArfbkB4oPvgJNmDZaY8aQd0UDI0Q3HF+I7oxTwSuUFEZ6uSb6PcO
	 zT4o2qwaIzk1OWAE1CryHYCTEcOy93tmf+lOE15HjS9d95qApVHOIqInPHSg8VVFvW
	 e+DnH6Umgrea/Jna5orNwIWaDQzJp0qEUsY+LdZ6WoOBbvd8IprdU6l3F1LuA/PDZN
	 K5kzN6tGVUZ/TjP1DPSLzfkUpghSWCgQqGXoE3u0W9F6gBHcTdTa2NC7/oLKZHoKEn
	 DsfYSbF5Gno7w==
Date: Tue, 27 Feb 2024 18:01:07 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Nicola Vetrini <nicola.vetrini@bugseng.com>, 
    xen-devel@lists.xenproject.org, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH 1/2] xen/console: drop return value from
 consoled_guest_rx/tx
In-Reply-To: <9fce0592-8b10-4b82-bcc6-38d6b32010a1@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402271800390.575685@ubuntu-linux-20-04-desktop>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com> <4998ec735bd7e5a50a229507e2b92ae56ec1ba4b.1708680104.git.nicola.vetrini@bugseng.com> <alpine.DEB.2.22.394.2402231456350.754277@ubuntu-linux-20-04-desktop> <004877c8-9aea-44b9-aa93-65665b0d3a8d@suse.com>
 <b417d7a330addd295b3cbbbac4bed2a4@bugseng.com> <a8bab8eb-a735-4440-a075-7c62a12d4e4d@suse.com> <alpine.DEB.2.22.394.2402261448170.247676@ubuntu-linux-20-04-desktop> <9fce0592-8b10-4b82-bcc6-38d6b32010a1@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 27 Feb 2024, Jan Beulich wrote:
> On 26.02.2024 23:49, Stefano Stabellini wrote:
> > On Mon, 26 Feb 2024, Jan Beulich wrote:
> >> On 26.02.2024 09:23, Nicola Vetrini wrote:
> >>> On 2024-02-26 09:00, Jan Beulich wrote:
> >>>> On 23.02.2024 23:56, Stefano Stabellini wrote:
> >>>>> On Fri, 23 Feb 2024, Nicola Vetrini wrote:
> >>>>>> These functions never saw a usage of their return value since
> >>>>>> they were introduced, so it can be dropped since their usages
> >>>>>> violate MISRA C Rule 17.7:
> >>>>>> "The value returned by a function having non-void return type shall 
> >>>>>> be used".
> >>>>>>
> >>>>>> No functional change.
> >>>>>>
> >>>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> >>>>>
> >>>>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> >>>>
> >>>> The cleanup is certainly okay, but would one of you mind clarifying in 
> >>>> how
> >>>> far this code is relevant for certification? I don't expect there are 
> >>>> plans
> >>>> to run shim Xen in any projected production uses for which 
> >>>> certification is
> >>>> relevant? (The subject prefix is also unnecessarily wide here, when 
> >>>> it's
> >>>> only daemon code which is affected, not console code in general.)
> >>>>
> >>>
> >>> I agree on the subject prefix being too wide. The configuration that 
> >>> uses consoled_guest_tx is #ifdef-ed for x86, so even in configurations 
> >>> that may never reach this condition this is relevant, unless its #ifdef 
> >>> is restricted to cases where the call may actually be reachable.
> >>
> >> Hmm, I see. There are contradicting goals here then: It being just X86 is
> >> to reduce the risk of someone overlooking a build breakage they may
> >> introduce. Whereas for certification it's quite the other way around: We'd
> >> like to "hide" as much code as possible.
> >>
> >> Really I would have been inclined to suggest to drop the #ifdef, if
> >> possible even without replacing by IS_ENABLED(), but instead leveraging
> >> that pv_shim ought to be compile-time false whenever CONFIG_PV_SHIM=n.
> > 
> > This is OK
> > 
> > 
> >> After all that's a pattern we've been trying to follow. But with your
> >> observation is becomes questionable whether extending use of IS_ENABLED()
> >> is actually going to be helpful. Stefano - perhaps something to discuss
> >> on one of the next meetings?
> > 
> > Yes. I checked with the safety manager and his opinion is that
> > IS_ENABLED() is OK to use as a way to disable code from a safety
> > perspective.
> 
> Yet unlike when #ifdef is used, such code would remain visible to e.g.
> Eclair even after the preprocessing step. Note the context in which
> I'm bringing this up - if IS_ENABLED() was properly used here (and as
> tightly as possible), the tool would still have complained, aiui.

Let me check with Roberto about this.


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 02:10:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 02:10:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686377.1068318 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf9On-0002NF-Ho; Wed, 28 Feb 2024 02:10:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686377.1068318; Wed, 28 Feb 2024 02:10:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf9On-0002N8-F0; Wed, 28 Feb 2024 02:10:21 +0000
Received: by outflank-mailman (input) for mailman id 686377;
 Wed, 28 Feb 2024 02:10:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=msSf=KF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rf9Om-0002N0-Jb
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 02:10:20 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8390fdc2-d5de-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 03:10:18 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 39F8DCE1F8F;
 Wed, 28 Feb 2024 02:10:13 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 55B29C433C7;
 Wed, 28 Feb 2024 02:10:11 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8390fdc2-d5de-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709086212;
	bh=k7uotmsW7zomBtopWFDRzvX/L8saNOM2jRg3oDb+lu4=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=EqIWoNj9D+3lC+3+DwZVoIjZ7Q7FePP2UKhNOLnYgvgaXI2KY6n6085hMth5j9u42
	 AP4fbkstI9mR5Rniy6BJTWkH54/EYXCgiIwCclUMTna5q/pB6f3UHFA3c/oo7yfS84
	 7BDtV2Swnvo7pVsr3BNmAD3Fdy5cswgOl1CMDx5oWbZcWI27Qz8kI1SiumR8mRTM6/
	 PzqGTX8Zs4adOCEy7Baw6HOjksb7xmLJfiZYL+IDiOHDNnBtyl9htEK9MUJsyBd83i
	 OZJrZ5R9oFM3GaXVfaEWEzfNkhvBhoeR0H5YplhNUZUW/1JTtXpuN9eKP7u4no5bWQ
	 H33XRxUUR/vJA==
Date: Tue, 27 Feb 2024 18:10:09 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Julien Grall <julien@xen.org>, Nicola Vetrini <nicola.vetrini@bugseng.com>, 
    consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, 
    xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
In-Reply-To: <4bee79ca-7a7e-4bcc-ac97-5a5a57ec2c91@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402271808410.575685@ubuntu-linux-20-04-desktop>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com> <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com> <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com> <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
 <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com> <7a8e610e-913e-4a56-8ce1-6dd6abd894f4@xen.org> <4bee79ca-7a7e-4bcc-ac97-5a5a57ec2c91@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 27 Feb 2024, Jan Beulich wrote:
> On 27.02.2024 12:52, Julien Grall wrote:
> > Hi Jan,
> > 
> > On 27/02/2024 07:28, Jan Beulich wrote:
> >> On 27.02.2024 01:26, Stefano Stabellini wrote:
> >>> On Mon, 26 Feb 2024, Jan Beulich wrote:
> >>>> On 23.02.2024 10:35, Nicola Vetrini wrote:
> >>>>> Refactor cpu_notifier_call_chain into two functions:
> >>>>> - the variant that is allowed to fail loses the nofail flag
> >>>>> - the variant that shouldn't fail is encapsulated in a call
> >>>>>    to the failing variant, with an additional check.
> >>>>>
> >>>>> This prevents uses of the function that are not supposed to
> >>>>> fail from ignoring the return value, thus violating Rule 17.7:
> >>>>> "The value returned by a function having non-void return type shall
> >>>>> be used".
> >>>>>
> >>>>> No functional change.
> >>>>>
> >>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> >>>>
> >>>> I'm afraid I disagree with this kind of bifurcation. No matter what
> >>>> Misra thinks or says, it is normal for return values of functions to
> >>>> not always be relevant to check.
> >>>
> >>> Hi Jan, I disagree.
> >>>
> >>> Regardless of MISRA, I really think return values need to be checked.
> >>> Moreover, we decided as a group to honor MISRA Rule 17.7, which requires
> >>> return values to be checked. This patch is a good step forward.
> >>
> >> Yet splitting functions isn't the only way to deal with Misra's
> >> requirements, I suppose. After all there are functions where the
> >> return value is purely courtesy for perhaps just one of its callers.
> > 
> > You are right that we have some places where one caller care about the 
> > return value. But the problem is how do you tell whether the return was 
> > ignored on purpose or not?
> > 
> > We had at least one XSA because the return value of a function was not 
> > checked (see XSA-222). We also had plenty of smaller patches to check 
> > returns.
> > 
> > So far, we added __must_check when we believed return values should be 
> > checked. But usually at the point we notice, this is far too late.
> > 
> > To me the goal should be that we enforce __must_check everywhere. We are 
> > probably going to detect places where we forgot to check the return. For 
> > thoses that are on purpose, we can document them.
> > 
> >>
> >> Splitting simply doesn't scale very well, imo.
> > 
> > Do you have another proposal? As Stefano said, we adopted the rule 17.7. 
> > So we know need a solution to address it.
> 
> One possibility that was circulated while discussing was to add (void)
> casts. I'm not a huge fan of those, but between the two options that
> might be the lesser evil. We also use funny (should I say ugly)
> workarounds in a few cases where we have __must_check but still want
> to not really handle the return value in certain cases. Given there are
> example in the code base, extending use of such constructs is certainly
> also something that may want considering.

I asked Roberto if void casts are an option for compliance.

In any case, I don't think we should use void casts in the specific
cases this patch is dealing with. Void casts (if anything) should be a
last resort while this patch fixes the issue in a better way.


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 02:14:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 02:14:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686379.1068328 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf9Sr-0002xo-24; Wed, 28 Feb 2024 02:14:33 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686379.1068328; Wed, 28 Feb 2024 02:14:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rf9Sq-0002xh-Vm; Wed, 28 Feb 2024 02:14:32 +0000
Received: by outflank-mailman (input) for mailman id 686379;
 Wed, 28 Feb 2024 02:14:31 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=msSf=KF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rf9Sp-0002xb-TC
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 02:14:31 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1a140b44-d5df-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 03:14:29 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 040F06133A;
 Wed, 28 Feb 2024 02:14:28 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0C78CC433C7;
 Wed, 28 Feb 2024 02:14:25 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1a140b44-d5df-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709086467;
	bh=OLoNRKwF25WpTLRj1j7m8QF5oKKCpZpWgANAuj0OFWM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=FDlGRPgoKU7kd1eEEf1bI/eTMK2ED6bVUobFb+UzAm/EvtUKB50JUymw/HBQiHLNR
	 tE89dyYnU0Et8ON6qTAwntSfO64UyjkTDhgw3+VLZEuNX+ybwyhbe+uZfZhnQtVd1N
	 lwT6PxiiiuoMz53vDE6qe/hC7nJcqzi7DH565H7r3MczRvyeEBbWbuYsOksAr0gCvl
	 sug99Sm9cEK0eNkF8DXHkf0t5z4qdRWBShoNnYlpXisMicXUNMOGa6TN6c9AwYNOnu
	 xNv0EUGRFKc9S1Au/cZfCzQBY8BS4e0QY2x4j84rqdmjM4b21W34B5lF65sIxd+6zK
	 bznRjsywNWDcA==
Date: Tue, 27 Feb 2024 18:14:24 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, 
    Simone Ballarin <simone.ballarin@bugseng.com>, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org, 
    roberto.bagnara@bugseng.com, Bertrand.Marquis@arm.com, 
    roger.pau@citrix.com, michal.orzel@amd.com
Subject: Re: [XEN PATCH 3/4] xen/include: add pure and const attributes
In-Reply-To: <4e3e949d-5714-4c3b-8514-50a3674c94d8@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402271810170.575685@ubuntu-linux-20-04-desktop>
References: <cover.1697638210.git.simone.ballarin@bugseng.com> <fd5421162a00aa782e0776324ff6497193c1e3d3.1697638210.git.simone.ballarin@bugseng.com> <89778285-5cba-8fb5-70bc-710b6dd30a10@suse.com> <0032186f-80c7-4dba-b46e-10d4a8e2a8cb@bugseng.com>
 <e8bf9817-fd54-9bf4-4302-dcee682f9172@suse.com> <alpine.DEB.2.22.394.2310231417260.3516@ubuntu-linux-20-04-desktop> <8f8141e2-c55c-2e58-a078-d5d8764a02df@suse.com> <alpine.DEB.2.22.394.2402221730270.754277@ubuntu-linux-20-04-desktop>
 <6eb4dc7c-c6a5-496a-a37f-125b00e9aaa3@suse.com> <alpine.DEB.2.22.394.2402231432090.754277@ubuntu-linux-20-04-desktop> <cb463469-51e1-4639-bf56-36fe9051d1b3@suse.com> <alpine.DEB.2.22.394.2402261511010.247676@ubuntu-linux-20-04-desktop>
 <4e3e949d-5714-4c3b-8514-50a3674c94d8@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-867894101-1709086347=:575685"
Content-ID: <alpine.DEB.2.22.394.2402271812340.575685@ubuntu-linux-20-04-desktop>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-867894101-1709086347=:575685
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.22.394.2402271812341.575685@ubuntu-linux-20-04-desktop>

On Tue, 27 Feb 2024, Jan Beulich wrote:
> On 27.02.2024 00:48, Stefano Stabellini wrote:
> > On Mon, 26 Feb 2024, Jan Beulich wrote:
> >> On 23.02.2024 23:36, Stefano Stabellini wrote:
> >>> On Fri, 23 Feb 2024, Jan Beulich wrote:
> >>>> On 23.02.2024 02:32, Stefano Stabellini wrote:
> >>>>> On Tue, 24 Oct 2023, Jan Beulich wrote:
> >>>>>> On 24.10.2023 00:05, Stefano Stabellini wrote:
> >>>>>>> On Mon, 23 Oct 2023, Jan Beulich wrote:
> >>>>>>>> On 23.10.2023 17:23, Simone Ballarin wrote:
> >>>>>>>>> On 23/10/23 15:34, Jan Beulich wrote:
> >>>>>>>>>> On 18.10.2023 16:18, Simone Ballarin wrote:
> >>>>>>>>>>> --- a/xen/include/xen/pdx.h
> >>>>>>>>>>> +++ b/xen/include/xen/pdx.h
> >>>>>>>>>>> @@ -164,7 +164,7 @@ static inline unsigned long pfn_to_pdx(unsigned long pfn)
> >>>>>>>>>>>    * @param pdx Page index
> >>>>>>>>>>>    * @return Obtained pfn after decompressing the pdx
> >>>>>>>>>>>    */
> >>>>>>>>>>> -static inline unsigned long pdx_to_pfn(unsigned long pdx)
> >>>>>>>>>>> +static inline __attribute_pure__ unsigned long pdx_to_pfn(unsigned long pdx)
> >>>>>>>>>>>   {
> >>>>>>>>>>>       return (pdx & pfn_pdx_bottom_mask) |
> >>>>>>>>>>>              ((pdx << pfn_pdx_hole_shift) & pfn_top_mask);
> >>>>>>>>>>
> >>>>>>>>>> Taking this as an example for what I've said above: The compiler can't
> >>>>>>>>>> know that the globals used by the functions won't change value. Even
> >>>>>>>>>> within Xen it is only by convention that these variables are assigned
> >>>>>>>>>> their values during boot, and then aren't changed anymore. Which makes
> >>>>>>>>>> me wonder: Did you check carefully that around the time the variables
> >>>>>>>>>> have their values established, no calls to the functions exist (which
> >>>>>>>>>> might then be subject to folding)?
> >>>>>>>>>
> >>>>>>>>> There is no need to check that, the GCC documentation explicitly says:
> >>>>>>>>>
> >>>>>>>>> However, functions declared with the pure attribute *can safely read any 
> >>>>>>>>> non-volatile objects*, and modify the value of objects in a way that 
> >>>>>>>>> does not affect their return value or the observable state of the program.
> >>>>>>>>
> >>>>>>>> I did quote this same text in response to what Andrew has said, but I also
> >>>>>>>> did note there that this needs to be taken with a grain of salt: The
> >>>>>>>> compiler generally assumes a single-threaded environment, i.e. no changes
> >>>>>>>> to globals behind the back of the code it is processing.
> >>>>>>>
> >>>>>>> Let's start from the beginning. The reason for Simone to add
> >>>>>>> __attribute_pure__ to pdx_to_pfn and other functions is for
> >>>>>>> documentation purposes. It is OK if it doesn't serve any purpose other
> >>>>>>> than documentation.
> >>>>>>>
> >>>>>>> Andrew, for sure we do not want to lie to the compiler and introduce
> >>>>>>> undefined behavior. If we think there is a risk of it, we should not do
> >>>>>>> it.
> >>>>>>>
> >>>>>>> So, what do we want to document? We want to document that the function
> >>>>>>> does not have side effects according to MISRA's definition of it, which
> >>>>>>> might subtly differ from GCC's definition.
> >>>>>>>
> >>>>>>> Looking at GCC's definition of __attribute_pure__, with the
> >>>>>>> clarification statement copy/pasted above by both Simone and Jan, it
> >>>>>>> seems that __attribute_pure__ matches MISRA's definition of a function
> >>>>>>> without side effects. It also seems that pdx_to_pfn abides to that
> >>>>>>> definition.
> >>>>>>>
> >>>>>>> Jan has a point that GCC might be making other assumptions
> >>>>>>> (single-thread execution) that might not hold true in our case. Given
> >>>>>>> the way the GCC statement is written I think this is low risk. But maybe
> >>>>>>> not all GCC versions we want to support in the project might have the
> >>>>>>> same definition of __attribute_pure__. So we could end up using
> >>>>>>> __attribute_pure__ correctly for the GCC version used for safety (GCC
> >>>>>>> 12.1, see docs/misra/C-language-toolchain.rst) but it might actually
> >>>>>>> break an older GCC version.
> >>>>>>>
> >>>>>>>
> >>>>>>> So Option#1 is to use __attribute_pure__ taking the risk that a GCC or
> >>>>>>> Clang version might interpret __attribute_pure__ differently and
> >>>>>>> potentially misbehave.
> >>>>>>>
> >>>>>>> Option#2 is to avoid this risk, by not using __attribute_pure__.
> >>>>>>> Instead, we can use SAF-xx-safe or deviations.rst to document that
> >>>>>>> pdx_to_pfn and other functions like it are without side effects
> >>>>>>> according to MISRA's definition.
> >>>>>>>
> >>>>>>>
> >>>>>>> Both options have pros and cons. To me the most important factor is how
> >>>>>>> many GCC versions come with the statement "pure attribute can safely
> >>>>>>> read any non-volatile objects, and modify the value of objects in a way
> >>>>>>> that does not affect their return value or the observable state of the
> >>>>>>> program".
> >>>>>>>
> >>>>>>> I checked and these are the results:
> >>>>>>> - gcc 4.0.2: no statement
> >>>>>>> - gcc 5.1.0: no statement
> >>>>>>> - gcc 6.1.0: no statement
> >>>>>>> - gcc 7.1.0: no statement
> >>>>>>> - gcc 8.1.0: alternative statement "The pure attribute imposes similar
> >>>>>>>   but looser restrictions on a function’s definition than the const
> >>>>>>>   attribute: it allows the function to read global variables."
> >>>>>>> - gcc 9.1.0: yes statement
> >>>>>>>
> >>>>>>>
> >>>>>>> So based on the above, __attribute_pure__ comes with its current
> >>>>>>> definition only from gcc 9 onward. I don't know if as a Xen community we
> >>>>>>> clearly declare a range of supported compilers, but I would imagine we
> >>>>>>> would still want to support gcc versions older than 9? (Not to mention
> >>>>>>> clang, which I haven't checked.)
> >>>>>>>
> >>>>>>> It doesn't seem to me that __attribute_pure__ could be correctly used on
> >>>>>>> pdx_to_pfn with GCC 7.1.0 for example.
> >>>>>>
> >>>>>> The absence of documentation doesn't mean the attribute had different
> >>>>>> (or even undefined) meaning in earlier versions. Instead it means one
> >>>>>> would need to consult other places (source code?) to figure out whether
> >>>>>> there was any behavioral difference (I don't think there was).
> >>>>>>
> >>>>>> That said, ...
> >>>>>>
> >>>>>>> So in conclusion, I think it is better to avoid __attribute_pure__ and
> >>>>>>> use SAF-xx-safe or an alternative approach instead.
> >>>>>>
> >>>>>> ... I agree here. We just don't want to take chances.
> >>>>>
> >>>>> Let me resurrect this thread.
> >>>>>
> >>>>> Could we use something like "pure" that we #define as we want?
> >>>>>
> >>>>> Depending on the compiler version or other options we could #define pure
> >>>>> to __attribute_pure__ or to nothing.
> >>>>
> >>>> While we can do about anything, I don't think it's a good idea to overload
> >>>> a well known term with something having somewhat different meaning. If a
> >>>> differently named custom attribute helps, that might be a possible option.
> >>>
> >>> It doesn't have a different meaning. If it had a different meaning I'd
> >>> agree with you.
> >>
> >> Then we need to sort this aspect first: If there was no difference in
> >> meaning, we ought to be using the real attribute, not a pseudo
> >> surrogate. Yet the earlier discussion, according to my understanding,
> >> has led to the understanding that for the given example the real
> >> attribute cannot be applied entirely legitimately. Hence why the
> >> thinking of alternatives actually started. What am I missing?
> > 
> > There are two different questions:
> > 1) using __attribute_pure__ in general when appropriate
> > 2) using __attribute_pure__ in pdx_to_pfn as this patch does
> > 
> > 
> > I was talking about 1): as a general approach it looks like a good idea
> > to use __attribute_pure__ when possible and appropriate.
> > 
> > Now let's talk about 2). The latest definition of __attribute_pure__ is:
> > 
> > """
> > The pure attribute prohibits a function from modifying the state of the program that is observable by means other than inspecting the function’s return value. However, functions declared with the pure attribute can safely read any non-volatile objects, and modify the value of objects in a way that does not affect their return value or the observable state of the program.
> > """
> > 
> > So there are two interesting issues:
> > 
> > a) While this documentation explicitly allows for reading global vars,
> > older versions of the docs are less clear. What do we do about them?
> > 
> > b) Jan wrote that he interprets the statements above to be only valid in
> > a single-threaded environment
> > 
> > 
> > To be honest, I am not convinced by b). Jan, is there a statement in the
> > GCC docs that says that all the attributes (pure being one of them) only
> > apply to a single-thread environment?
> 
> It would need to be the other way around, I'm afraid: C99 defines its
> abstract machine in a way not even considering possible parallel
> execution (except for other external agents, e.g. when "volatile" is
> necessary for memory accesses when that memory may also be modified
> by such an external agent, e.g. a device on the bus). Hence we'd need
> to find an explicit statement in gcc docs which relaxes that globally
> or for certain aspects.

I don't think so: the way C99 defines its abstract machine is not
tightly coupled with the way gcc extensions are defined.

Roberto, you have worked with gcc extensions quite a bit, what's your
take on this?

Other maintainers, please express your view.


> > That would be extremely limiting
> > for something like __attribute_pure__. I think we should take the
> > documentation of attribute pure at face value. To me, it clearly applies
> > to pdx_to_pfn. Roberto and the team at Bugseng came to the same
> > conclusion.
> > 
> > On the other end, I think a) is important. Older version of GCC don't
> > clarify the behavior toward global variables. From the documentation, I
> > would use __attribute_pure__ only with GCC 9 or later. Which is why we
> > need the #define.
> 
> Right, this is a position we can take. As said, I think we'd then limit
> ourselves more than necessary. Otoh the number of people using gcc8 or
> older to build up-to-date Xen should be constantly decreasing ...

That is true, but I think it is an OK limitation to have.
--8323329-867894101-1709086347=:575685--


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 02:50:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 02:50:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686386.1068338 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfA1e-0000HH-Vj; Wed, 28 Feb 2024 02:50:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686386.1068338; Wed, 28 Feb 2024 02:50:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfA1e-0000HA-T1; Wed, 28 Feb 2024 02:50:30 +0000
Received: by outflank-mailman (input) for mailman id 686386;
 Wed, 28 Feb 2024 02:50:29 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfA1d-0000H0-LJ; Wed, 28 Feb 2024 02:50:29 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfA1d-0006xg-Fo; Wed, 28 Feb 2024 02:50:29 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfA1d-0008Aj-1I; Wed, 28 Feb 2024 02:50:29 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfA1c-00057P-V2; Wed, 28 Feb 2024 02:50:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=LBd7+UPwTgYp9ddbC4JUeZhM4xf8R4gZqQbhnjoZn4M=; b=zWFMyz0onR8gkqzhSANSFuLty5
	Vsw0X0lf7izKJStsE9DiYhI0E8HtDa8cbLlIks0O9Zbres74LdJkN/Jrk8Eg0qoSFwup2tP+D8J8U
	Mx3/vUcyWWmOsi781Vs978TIzNr/k2Sl+9/yFiBpg7fWKwgYixNvF7B2TIc3Z2hzyd78=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184784-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.15-testing test] 184784: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.15-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.15-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=2484803074fc1f93942912a82e4906f9e441bf4e
X-Osstest-Versions-That:
    xen=dfafb3ec959b20307d4c640a48b3b55a2896ac30
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 02:50:28 +0000

flight 184784 xen-4.15-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184784/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184529
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184529
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184529
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184529
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184529
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184529
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184529
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184529
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184529
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184529
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184529
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184529
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  2484803074fc1f93942912a82e4906f9e441bf4e
baseline version:
 xen                  dfafb3ec959b20307d4c640a48b3b55a2896ac30

Last test of basis   184529  2024-01-30 14:06:47 Z   28 days
Testing same since   184784  2024-02-27 13:36:49 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   dfafb3ec95..2484803074  2484803074fc1f93942912a82e4906f9e441bf4e -> stable-4.15


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 03:42:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 03:42:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686392.1068353 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfApo-0006f6-Py; Wed, 28 Feb 2024 03:42:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686392.1068353; Wed, 28 Feb 2024 03:42:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfApo-0006ez-Mb; Wed, 28 Feb 2024 03:42:20 +0000
Received: by outflank-mailman (input) for mailman id 686392;
 Wed, 28 Feb 2024 03:42:18 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfApm-0006ep-Hv; Wed, 28 Feb 2024 03:42:18 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfApm-0007sD-Ef; Wed, 28 Feb 2024 03:42:18 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfApm-0002Bd-43; Wed, 28 Feb 2024 03:42:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfApm-0002iC-3j; Wed, 28 Feb 2024 03:42:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=fPajU2ElPmgjasgNwYun/KiFJzyjYbcYVlRgxY1ogLU=; b=vwCXhhU++IfLIfx3/C9KLyQqnE
	yQA3yekSbWJlXz7s69VglBrG4Fp1IYg2eKnLTQZvaRHWN4tC8kYMRmjFwj8QCtevfJs+giqlD5Oul
	2JymfI9S0sWSR+prknzsSc5kBR0eD10Qdl+K5bYY81tYw9i1s7FmxNaIBzUb3iM73VFQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184790-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184790: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=aceb3490a2a350b128156fd4e36e53fc19739e4e
X-Osstest-Versions-That:
    ovmf=ba9c3ceaf83d3ee448b2fd38bc9ad28b690c697a
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 03:42:18 +0000

flight 184790 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184790/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 aceb3490a2a350b128156fd4e36e53fc19739e4e
baseline version:
 ovmf                 ba9c3ceaf83d3ee448b2fd38bc9ad28b690c697a

Last test of basis   184782  2024-02-27 12:43:39 Z    0 days
Testing same since   184790  2024-02-27 15:45:00 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Gerd Hoffmann <kraxel@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   ba9c3ceaf8..aceb3490a2  aceb3490a2a350b128156fd4e36e53fc19739e4e -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 06:12:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 06:12:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686324.1068362 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfDB3-0006wh-75; Wed, 28 Feb 2024 06:12:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686324.1068362; Wed, 28 Feb 2024 06:12:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfDB3-0006wa-4R; Wed, 28 Feb 2024 06:12:25 +0000
Received: by outflank-mailman (input) for mailman id 686324;
 Tue, 27 Feb 2024 22:35:15 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SVdi=KE=amd.com=vikram.garhwal@srs-se1.protection.inumbo.net>)
 id 1rf62d-0002I9-SQ
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 22:35:15 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 78cdcbbc-d5c0-11ee-afd7-a90da7624cb6;
 Tue, 27 Feb 2024 23:35:14 +0100 (CET)
Received: from BN0PR04CA0070.namprd04.prod.outlook.com (2603:10b6:408:ea::15)
 by IA1PR12MB8079.namprd12.prod.outlook.com (2603:10b6:208:3fb::5)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.36; Tue, 27 Feb
 2024 22:35:10 +0000
Received: from BN3PEPF0000B071.namprd04.prod.outlook.com
 (2603:10b6:408:ea:cafe::59) by BN0PR04CA0070.outlook.office365.com
 (2603:10b6:408:ea::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.50 via Frontend
 Transport; Tue, 27 Feb 2024 22:35:10 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN3PEPF0000B071.mail.protection.outlook.com (10.167.243.116) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 27 Feb 2024 22:35:09 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 16:35:07 -0600
Received: from xsjfnuv50.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 27 Feb 2024 16:35:07 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78cdcbbc-d5c0-11ee-afd7-a90da7624cb6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=kNkGdTpq4mOcI6s0v5uOOXjqQk7yjs4ceNE+401QBpouUakJ56Y1A7hDuqEnzqSslC7H3OQTlsD3pWkOYaUoVItYio5NTnxbWA16GBP5TCAQOEn3EKKJ1S5TS7G7Hajn3auARhreT0/eronl4nKDaFMDWS/zoF2YtMzf7uTzVsHZD6fSQ797439vZj5xKZAvFqyeeLxVm5puKIFyydeCyLk6hhwiVSUM899kxuttTTuvgvP+DPiIM7sF0J+5iey+W8ZVpKk/Ou+ETbM0+2kAUx6AhzbXDDBohGXi1j7sAgm63XIVUh0hmMbH3lbVZ6wqKuzcyrCPKPBx1UupKj70PQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=q791cF6GA8d6F2oGognOVh2s9YFO/NpyQ1CcNhKOZRI=;
 b=QIm90RTZCwD8PKKjnZsk7QB7/kU50z6wtefufGf5RMF3Jq1V/Ws2I2xJiYEAvwJZDMn1BLFCDiqVTAdgHNgljhXCTt3S5ZMT2ItDJU+IV9/01x3ndHxvkNiPld7Zd0+aVctsQDTKOZiEIB3QWrp1ON/ASCSB/xlGR84q5xZ/Vrh1m7YBnZJQ6PStdDLQZCNSpV/84oa6zpq/a4LUN3PwSUlQxUZYJ0bg4vk0epPvdafKEdhEjBG8BwNBn8G0Ix54wzqyb6LYncfs+A3hEJztryNOLWLlUGk7crq2p/gFP2HKUhIV8DZVn1O6lnH+ijuZJH0yLKW98ILWuLpN1W6VMg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=nongnu.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=q791cF6GA8d6F2oGognOVh2s9YFO/NpyQ1CcNhKOZRI=;
 b=iYXjovZwOrcfwWf61bLbpzFMWdsCVSGRNZ+KYKQtDYwpWxcE8AbQAwABWZ74dtFoB/WKYcHeZvdWt6+IvLiOrdWNbrkGq07CytLwh9xHAG4pAX7oD3bRaQ2i6xEHvgZSnte7sjTaE1ugGKuFaSafjcU0q0KPyMjHhC3n+wBat88=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Vikram Garhwal <vikram.garhwal@amd.com>
To: <qemu-devel@nongnu.org>
CC: <sstabellini@kernel.org>, <vikram.garhwal@amd.com>, <jgross@suse.com>,
	Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
	Paolo Bonzini <pbonzini@redhat.com>, Richard Henderson
	<richard.henderson@linaro.org>, Eduardo Habkost <eduardo@habkost.net>,
	"Michael S. Tsirkin" <mst@redhat.com>, Marcel Apfelbaum
	<marcel.apfelbaum@gmail.com>, "open list:X86 Xen CPUs"
	<xen-devel@lists.xenproject.org>
Subject: [QEMU][PATCH v3 2/7] xen: add pseudo RAM region for grant mappings
Date: Tue, 27 Feb 2024 14:34:56 -0800
Message-ID: <20240227223501.28475-3-vikram.garhwal@amd.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20240227223501.28475-1-vikram.garhwal@amd.com>
References: <20240227223501.28475-1-vikram.garhwal@amd.com>
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: vikram.garhwal@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN3PEPF0000B071:EE_|IA1PR12MB8079:EE_
X-MS-Office365-Filtering-Correlation-Id: 7551176a-e61d-47ca-558e-08dc37e45b0e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	3WoNnATOV76Ns0xdkiLkepu0ykyJmiUe37sILWm+XYzpTfXhqkE/3LDmmdZlyJnT3RpkAqyb9WDN/VlzA9933f8wlDZ4HYRuqvNSvr+BykqLwr/gfGcokiKl7g7UXybObN+GUW4j6794TvadSBW7sx2YmKol+MomtElGEyrHr4M4dpydDfyb/6eVWhD01rLN/SyloBEC5wmExkKcJ/UOTe2oSHbfJtGscEie5V5OU/fw7lH03j0Hw+Y3hvJOn0rrElGRbf9Rl7e96M/0teTnFOE4R7LSxnpr8O1Bcii+sMltrhMSbdgXcGkvzmj6qj3A4OUgOHRWZtDycfw6j1EYhuBuinozHUHsxgYkiSC4TkJ0xs9gl0Ju8KqsZb3nv5bHK9HnA15luNz/czr0r8WapHCFu0/aTV5SK4IlyTCbPu+iZiemYEXN9Oxpsv+RjZaiulTvyZiOVO1Bs00JNflgJI6i5H1eaU4Ch8KAnM0y/Qb+lvv02I59YDptmFeUgHfn1IWeXWcHpPOsFwE7LhrJNNiBq/KQycYoag+C4B9jz5P6M0LdN6tbVPCNxR1w6OyE5Sc+R6/lQVuhQ8/RuikbuUjgcqyaUPhdl8XydGhOy2nwt6Bksg11IFGrUuu7Wf4o8d9qenjvEqPFGejQQhuCUzPrGYxMZnWKmhv17vtz1AzXZnGrWWH/ePOwucrIRzkndHWtZrNvFwI6iq/kC6a7AnJ05YCTJbPLekFSjIrUn93sZXuy5KmakgoS+A32YBpW
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400014)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 22:35:09.9036
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 7551176a-e61d-47ca-558e-08dc37e45b0e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN3PEPF0000B071.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8079

From: Juergen Gross <jgross@suse.com>

Add a memory region which can be used to automatically map granted
memory. It is starting at 0x8000000000000000ULL in order to be able to
distinguish it from normal RAM.

For this reason the xen.ram memory region is expanded, which has no
further impact as it is used just as a container of the real RAM
regions and now the grant region.

Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/i386/xen/xen-hvm.c           |  3 +++
 hw/xen/xen-hvm-common.c         |  4 ++--
 hw/xen/xen-mapcache.c           | 27 +++++++++++++++++++++++++++
 include/hw/xen/xen-hvm-common.h |  2 ++
 include/hw/xen/xen_pvdev.h      |  3 +++
 include/sysemu/xen-mapcache.h   |  3 +++
 6 files changed, 40 insertions(+), 2 deletions(-)

diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index f42621e674..67a55558a6 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -172,6 +172,9 @@ static void xen_ram_init(PCMachineState *pcms,
                                  x86ms->above_4g_mem_size);
         memory_region_add_subregion(sysmem, 0x100000000ULL, &ram_hi);
     }
+
+    /* Add grant mappings as a pseudo RAM region. */
+    ram_grants = *xen_init_grant_ram();
 }
 
 static XenPhysmap *get_physmapping(hwaddr start_addr, ram_addr_t size)
diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index baa1adb9f2..6e53d3bf81 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -9,7 +9,7 @@
 #include "hw/boards.h"
 #include "hw/xen/arch_hvm.h"
 
-MemoryRegion ram_memory;
+MemoryRegion ram_memory, ram_grants;
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
                    Error **errp)
@@ -26,7 +26,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
         return;
     }
 
-    if (mr == &ram_memory) {
+    if (mr == &ram_memory || mr == &ram_grants) {
         return;
     }
 
diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 4f956d048e..dfc412d138 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -14,7 +14,9 @@
 
 #include <sys/resource.h>
 
+#include "hw/xen/xen-hvm-common.h"
 #include "hw/xen/xen_native.h"
+#include "hw/xen/xen_pvdev.h"
 #include "qemu/bitmap.h"
 
 #include "sysemu/runstate.h"
@@ -590,3 +592,28 @@ uint8_t *xen_replace_cache_entry(hwaddr old_phys_addr,
     mapcache_unlock();
     return p;
 }
+
+MemoryRegion *xen_init_grant_ram(void)
+{
+    RAMBlock *block;
+
+    memory_region_init(&ram_grants, NULL, "xen.grants",
+                       XEN_MAX_VIRTIO_GRANTS * XC_PAGE_SIZE);
+    block = g_malloc0(sizeof(*block));
+    block->mr = &ram_grants;
+    block->used_length = XEN_MAX_VIRTIO_GRANTS * XC_PAGE_SIZE;
+    block->max_length = XEN_MAX_VIRTIO_GRANTS * XC_PAGE_SIZE;
+    block->fd = -1;
+    block->page_size = XC_PAGE_SIZE;
+    block->host = (void *)XEN_GRANT_ADDR_OFF;
+    block->offset = XEN_GRANT_ADDR_OFF;
+    block->flags = RAM_PREALLOC;
+    ram_grants.ram_block = block;
+    ram_grants.ram = true;
+    ram_grants.terminates = true;
+    ram_block_add_list(block);
+    memory_region_add_subregion(get_system_memory(), XEN_GRANT_ADDR_OFF,
+                                &ram_grants);
+
+    return &ram_grants;
+}
diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-common.h
index 4b1d728f35..8deeff6bcf 100644
--- a/include/hw/xen/xen-hvm-common.h
+++ b/include/hw/xen/xen-hvm-common.h
@@ -16,6 +16,8 @@
 #include <xen/hvm/ioreq.h>
 
 extern MemoryRegion ram_memory;
+
+extern MemoryRegion ram_grants;
 extern MemoryListener xen_io_listener;
 extern DeviceListener xen_device_listener;
 
diff --git a/include/hw/xen/xen_pvdev.h b/include/hw/xen/xen_pvdev.h
index ddad4b9f36..0f1b5edfa9 100644
--- a/include/hw/xen/xen_pvdev.h
+++ b/include/hw/xen/xen_pvdev.h
@@ -80,4 +80,7 @@ int xen_pv_send_notify(struct XenLegacyDevice *xendev);
 void xen_pv_printf(struct XenLegacyDevice *xendev, int msg_level,
                    const char *fmt, ...)  G_GNUC_PRINTF(3, 4);
 
+#define XEN_GRANT_ADDR_OFF    0x8000000000000000ULL
+#define XEN_MAX_VIRTIO_GRANTS 65536
+
 #endif /* QEMU_HW_XEN_PVDEV_H */
diff --git a/include/sysemu/xen-mapcache.h b/include/sysemu/xen-mapcache.h
index c8e7c2f6cf..f4bedb1c11 100644
--- a/include/sysemu/xen-mapcache.h
+++ b/include/sysemu/xen-mapcache.h
@@ -10,6 +10,7 @@
 #define XEN_MAPCACHE_H
 
 #include "exec/cpu-common.h"
+#include "exec/ram_addr.h"
 
 typedef hwaddr (*phys_offset_to_gaddr_t)(hwaddr phys_offset,
                                          ram_addr_t size);
@@ -25,6 +26,8 @@ void xen_invalidate_map_cache(void);
 uint8_t *xen_replace_cache_entry(hwaddr old_phys_addr,
                                  hwaddr new_phys_addr,
                                  hwaddr size);
+MemoryRegion *xen_init_grant_ram(void);
+
 #else
 
 static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 06:12:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 06:12:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686327.1068376 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfDB3-00076l-Ss; Wed, 28 Feb 2024 06:12:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686327.1068376; Wed, 28 Feb 2024 06:12:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfDB3-00074C-KL; Wed, 28 Feb 2024 06:12:25 +0000
Received: by outflank-mailman (input) for mailman id 686327;
 Tue, 27 Feb 2024 22:35:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SVdi=KE=amd.com=vikram.garhwal@srs-se1.protection.inumbo.net>)
 id 1rf62p-0002In-FL
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 22:35:27 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7e77f67a-d5c0-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 23:35:23 +0100 (CET)
Received: from DS7PR05CA0050.namprd05.prod.outlook.com (2603:10b6:8:2f::9) by
 DM4PR12MB5085.namprd12.prod.outlook.com (2603:10b6:5:388::11) with
 Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7316.36; Tue, 27 Feb 2024 22:35:19 +0000
Received: from CY4PEPF0000FCC3.namprd03.prod.outlook.com
 (2603:10b6:8:2f:cafe::7f) by DS7PR05CA0050.outlook.office365.com
 (2603:10b6:8:2f::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.26 via Frontend
 Transport; Tue, 27 Feb 2024 22:35:19 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC3.mail.protection.outlook.com (10.167.242.105) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 27 Feb 2024 22:35:19 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 16:35:09 -0600
Received: from xsjfnuv50.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 27 Feb 2024 16:35:09 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e77f67a-d5c0-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=n+wvoNjvHGV6H4AXYaKQcu5+HXvA/qX04A7HgEiw8Ee1G3W1xTPqU9uGvNJAjgdBwrMwhgC0v1cSHNelFOmlQgVrK2V8J//HnMQkb9ZLHUKnfqVlEA+r39CaCTEIfJdPs0G+1oDpNHlyzLIOzSQcA7QgdI0dNgfQzVe/CIdRPoKiJXZKAPLBYPoTucJ0YwRqjbBeYku4YfTLVs4vxFw85ybrYsOr/GrelOVm70hu05RdBteKot7Ys6QJG2y4KSF7jIFRTwYi/391JRhn8+exQYDf0PGv/ohrHKqqgawWPlWeMtqm+6Yp+h7+g6ggJlhvz7DbrLdkSaTZoYG/FRZ1Fw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=5OZlucp+PoJdXp0H7/yZMTJ3c3snBOa8bo8Wnac6SL0=;
 b=aSZg+KfQynp/yG5GSBkWOE3vSOROC10yCZ/0LKp/HhoiZvSe3s7b8wSk7onKSGISECBVXBW+YFhTANBLWFVmKyDW+dqUd2Uxo3ZIPohC9NVUUQHlBq1PNYhJHDL4+ImcgLnv+Y5CbMtmd9L3zC0qmj4Umqw7de8Ee5oLYDrgibEWzSDTbi9sjMFt0uKd+qh52gSuDpogCvMRL/XMEiaP7UzyTGHjuCIg/1OrkKMXgn6Z0hzOtEKBHKAmK2N1D7m+CC4RuiK9aABTTyRIVYBYRMF6imTKM08RCDIozrSzoqKhx1cPWkpV51jlrX7szVLHZpshuNHQywKxvApbR0D3fw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=nongnu.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=5OZlucp+PoJdXp0H7/yZMTJ3c3snBOa8bo8Wnac6SL0=;
 b=0k4dvU1NkQs9St7CQukJZRnyMZHs+mx8khDu3pNBnV1RVyNznnuwMmS/1J50ds7aa+/xzEDUcRrBbHno7F3DjOmVETfUpj9W5hxHO+8pa14et1iuSq8Q/d9pMvUFPi/Wd1gekkhYyU5+vx9bQt1XRDxKFWqHbNNRAnjztShBpzE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Vikram Garhwal <vikram.garhwal@amd.com>
To: <qemu-devel@nongnu.org>
CC: <sstabellini@kernel.org>, <vikram.garhwal@amd.com>, <jgross@suse.com>,
	Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
	"open list:X86 Xen CPUs" <xen-devel@lists.xenproject.org>
Subject: [QEMU][PATCH v3 4/7] xen: let xen_ram_addr_from_mapcache() return -1 in case of not found entry
Date: Tue, 27 Feb 2024 14:34:58 -0800
Message-ID: <20240227223501.28475-5-vikram.garhwal@amd.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20240227223501.28475-1-vikram.garhwal@amd.com>
References: <20240227223501.28475-1-vikram.garhwal@amd.com>
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: vikram.garhwal@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC3:EE_|DM4PR12MB5085:EE_
X-MS-Office365-Filtering-Correlation-Id: f800fa24-9cb5-4900-4018-08dc37e460c2
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	ttUogOJd0iTIkqEQCd/nzbRBJYmOmeXxMNF3sPtwOTMiZB2RJ1iFEO5+Y4HxhBup+iPRbYAveHsIyKzCtkzLEuq8CReCndcXSrYV6xXSoNReTT01XTwawYlGO0/HEFwp6YTpMI7qFUuZO87nVn95QlBH5KwYehdA6b4Y4TH7oU0efT0gu1A8YyBj5PhM4MG3+pVjqYDouXJj+vzsOUwIt4R8fWyy++0QyEzDn8WkYDB4MQx4O+Jcz4cPsjuXwF1vbN1n8x0Kp++ULGQEoy+e5upFQxfLiuDJMKTAFyGbwnLbp5BbJh/gujrOFqLDTKRrk4Z9TCCp/SA6mArFcP4w9kzDiRLyW0Q2KXaOAmvyzbiasAkzdvHH64T2P4oNPXAG+D2eICaTUtji5o7b1YuaO35oBkFWL/Jeoiw9a2+jaYywe7houWkbOaf8XTPD+bTsreb35Go/mcAPpOGMc13uK7XeTx7TGNHanDj8JO656LmMFZNoQ4cMrqKqFSh4hw/YKigI2kJjmKqQcj5HMOdTacBbBI/Hw3CId9CVO0tn2/5smT1AXjnzrZQa/e4DxSAiPSI62Xr2bHQJYAsmw3ZeVwr4RZ8T3GW7VNAOPnZe/VMS8Ae9I7roiPU9Z+EWk4PEwpSOEOoe81ZQ0hrd9EIWiEV51e/h8PFG4LuSdE+T7q9p1hGABHhfaoWSv2MBrr7CTDFdjE6OyVPhzC74G3htX8EUIbE2pOD1i+Gzy75p0tNLQTLyMTd7540SqUw8MlaK
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400014)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 22:35:19.4088
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f800fa24-9cb5-4900-4018-08dc37e460c2
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC3.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5085

From: Juergen Gross <jgross@suse.com>

Today xen_ram_addr_from_mapcache() will either abort() or return 0 in
case it can't find a matching entry for a pointer value. Both cases
are bad, so change that to return an invalid address instead.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 hw/xen/xen-mapcache.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index dfc412d138..179b7e95b2 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -396,13 +396,8 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
         }
     }
     if (!found) {
-        trace_xen_ram_addr_from_mapcache_not_found(ptr);
-        QTAILQ_FOREACH(reventry, &mapcache->locked_entries, next) {
-            trace_xen_ram_addr_from_mapcache_found(reventry->paddr_index,
-                                                   reventry->vaddr_req);
-        }
-        abort();
-        return 0;
+        mapcache_unlock();
+        return RAM_ADDR_INVALID;
     }
 
     entry = &mapcache->entry[paddr_index % mapcache->nr_buckets];
@@ -411,7 +406,7 @@ ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
     }
     if (!entry) {
         trace_xen_ram_addr_from_mapcache_not_in_cache(ptr);
-        raddr = 0;
+        raddr = RAM_ADDR_INVALID;
     } else {
         raddr = (reventry->paddr_index << MCACHE_BUCKET_SHIFT) +
              ((unsigned long) ptr - (unsigned long) entry->vaddr_base);
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 06:12:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 06:12:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686326.1068367 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfDB3-0006zc-Fx; Wed, 28 Feb 2024 06:12:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686326.1068367; Wed, 28 Feb 2024 06:12:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfDB3-0006z5-BO; Wed, 28 Feb 2024 06:12:25 +0000
Received: by outflank-mailman (input) for mailman id 686326;
 Tue, 27 Feb 2024 22:35:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=SVdi=KE=amd.com=vikram.garhwal@srs-se1.protection.inumbo.net>)
 id 1rf62p-0002In-8Y
 for xen-devel@lists.xenproject.org; Tue, 27 Feb 2024 22:35:27 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2414::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7e6d83e2-d5c0-11ee-a1ee-f123f15fe8a2;
 Tue, 27 Feb 2024 23:35:23 +0100 (CET)
Received: from DS7PR05CA0059.namprd05.prod.outlook.com (2603:10b6:8:2f::32) by
 SN7PR12MB6958.namprd12.prod.outlook.com (2603:10b6:806:262::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Tue, 27 Feb
 2024 22:35:20 +0000
Received: from CY4PEPF0000FCC3.namprd03.prod.outlook.com
 (2603:10b6:8:2f:cafe::23) by DS7PR05CA0059.outlook.office365.com
 (2603:10b6:8:2f::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.25 via Frontend
 Transport; Tue, 27 Feb 2024 22:35:20 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 CY4PEPF0000FCC3.mail.protection.outlook.com (10.167.242.105) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Tue, 27 Feb 2024 22:35:20 +0000
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 27 Feb
 2024 16:35:11 -0600
Received: from xsjfnuv50.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Tue, 27 Feb 2024 16:35:11 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7e6d83e2-d5c0-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=DqrLQcdXHv/CjdR7VN/GoH4OP9IDQXk/v5FepoDm26pfAticCtxhQFi+fPPBB03EOijIbxiG/8LHK9hoetuKspl2QPNCwUim3+60QSZdE6xgm/YAMg359oBZ779RmMZpM4OYYxBhoFttf/kjkjap7i6emLLo15dKeLjYKTsUMYHV1QZvb/7Nl2V1REt7Vxd3l/OqaIQ41REoeZSxM8nrNYQpE3q1jNCurIQdt/suhAMqhjz+n8xjJmrgUs30RaeZIiAo0IrFXneGcj+l8nCf3aiHsNo9igQNMz1A6/q/YMDPQpCDng+wb4KbvNX5YBBJSCeleuOD3RxEsNsGpFX3zw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=4EUODE8Te6vqif3czPfW2F2AieavOD9cQNiVAhzYX3Q=;
 b=gWR3sCNOzlxD0EDklAYI5OIAZIBD0oVL0AV96BZa0H6iuznLIS3n2ZfRWRO3pTIMG35RUUZDzhnUA1IVZmF/sX3j2E9t0xJqzrMoqOMDRCfBrh0EZ3wFcYgRADhp/DwsO/+Rv/Cn9BGb6zWU4gFKVYdL/KCOJ3XuhAnS+gPVy2usQbxDHqCnGh7Zz1dTmeIu75sO9xP/nI+Rd4gKlAaC2exnto3NQ44uyMh2vz+FVgH/1Gg8bByuLmx7qUhuC7EEEji3XnJKQkVCiUT7vgC7+BOpfaTVMSQEC9Unq5gPNqs54a/jZqXDzEntw8z9f4SCU5YZ2KeHWJNihnqOY+4TsQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=nongnu.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=4EUODE8Te6vqif3czPfW2F2AieavOD9cQNiVAhzYX3Q=;
 b=2fSb/J66bYWR+R761ie20WlgsZFVcQyTTMzttpi+yVoo2E41zDv7J4FWUaWt9Zb9BnXxCkaFvn77/mdo7rLPACaQoJBb/5rbdKYchOMYqjDBZGHhAuxyoPsoq7Eykm26eiU5aH3M7v16Zkr3Aw3Gi+TtOlCjMJaPolDfPNVS/9A=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Vikram Garhwal <vikram.garhwal@amd.com>
To: <qemu-devel@nongnu.org>
CC: <sstabellini@kernel.org>, <vikram.garhwal@amd.com>, <jgross@suse.com>,
	Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>,
	Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, "David
 Hildenbrand" <david@redhat.com>, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?=
	<philmd@linaro.org>, "open list:X86 Xen CPUs"
	<xen-devel@lists.xenproject.org>
Subject: [QEMU][PATCH v3 6/7] xen: add map and unmap callbacks for grant region
Date: Tue, 27 Feb 2024 14:35:00 -0800
Message-ID: <20240227223501.28475-7-vikram.garhwal@amd.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20240227223501.28475-1-vikram.garhwal@amd.com>
References: <20240227223501.28475-1-vikram.garhwal@amd.com>
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: None (SATLEXMB04.amd.com: vikram.garhwal@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC3:EE_|SN7PR12MB6958:EE_
X-MS-Office365-Filtering-Correlation-Id: 23d5ea88-1602-40cc-631f-08dc37e46126
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	62q64bDcZSgQSaqGlQ9flbe+pzxZmlJSnIu4UO2fS1KGw3KhDoflniJQBoloCX9p22lvQ1KCLEGCLMdxS1UxoDZXxeUn7xrHm9QiFGg89IkUio8Op3ZfnxXel4GvBt+0idrGQ8XAiaYkLarHL+RWEmL0Iqn50YzdcWdf05DPcAK02lu7tG1JtuQlVQ0PFSOmlr+QWcsgyz1ldMqGva34sVv9CsyKqiC1+hX79YQvbtG06DLMiZ9xaVJi2J8+t2GsvVmAvGr86gk1vApbZdA55mBDF+mJ+xlPZgN+dlD5sLlS9spAWcdyfy3hgrGwqmXf3BBQGz1NNOb4vHyNUEDEduxEhW4WyBieKfMWWOni61DlZ04QmPi75GvwO91L2viq966nwRRe/Gq82qfVzjhOTb9zddPgp+ekVqW/wP4AcWPEnL399yHFZxONjbs0HBeh1/jwD7ozy0qPNwKbJpDoljzgi+mkkD9DpdIaDGizPlPJ20NpsHynDXlCo7n4NqghQuTNCgYMgZqmrnw5VqiHwc3evCUilampLRjoWTr4eBl9nknimwWJ4XmQ7bNUKpnLnBRVaRAGKongZaVBJB/HEb7d4ECDfoWiJOxWb/ssSpU/zpBeRqMCCZ9e6Hx28VJCUrDBzA98lMwH6TIUnn22auzWBYFaoqWP/dl3tKrOyXSS5F2KoU7m27reXVbynrW/SeePlMJUUg/ioXGCc09vJaQPJ2LDJvMMPmu1mZB6n+xABsYDKlrOOYK8TRrfyU9H
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 22:35:20.0650
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 23d5ea88-1602-40cc-631f-08dc37e46126
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	CY4PEPF0000FCC3.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB6958

From: Juergen Gross <jgross@suse.com>

Add the callbacks for mapping/unmapping guest memory via grants to the
special grant memory region.

Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
---
 hw/xen/xen-mapcache.c | 176 +++++++++++++++++++++++++++++++++++++++++-
 system/physmem.c      |  11 ++-
 2 files changed, 182 insertions(+), 5 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 179b7e95b2..2e4c9b4947 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -9,6 +9,8 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/queue.h"
+#include "qemu/thread.h"
 #include "qemu/units.h"
 #include "qemu/error-report.h"
 
@@ -23,6 +25,8 @@
 #include "sysemu/xen-mapcache.h"
 #include "trace.h"
 
+#include <xenevtchn.h>
+#include <xengnttab.h>
 
 #if HOST_LONG_BITS == 32
 #  define MCACHE_BUCKET_SHIFT 16
@@ -377,7 +381,7 @@ uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
     return p;
 }
 
-ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
+static ram_addr_t xen_ram_addr_from_mapcache_try(void *ptr)
 {
     MapCacheEntry *entry = NULL;
     MapCacheRev *reventry;
@@ -588,10 +592,179 @@ uint8_t *xen_replace_cache_entry(hwaddr old_phys_addr,
     return p;
 }
 
+struct XENMappedGrantRegion {
+    void *addr;
+    unsigned int pages;
+    unsigned int refs;
+    unsigned int prot;
+    uint32_t idx;
+    QLIST_ENTRY(XENMappedGrantRegion) list;
+};
+
+static xengnttab_handle *xen_region_gnttabdev;
+static QLIST_HEAD(GrantRegionList, XENMappedGrantRegion) xen_grant_mappings =
+    QLIST_HEAD_INITIALIZER(xen_grant_mappings);
+static QemuMutex xen_map_mutex;
+
+static void *xen_map_grant_dyn(MemoryRegion **mr, hwaddr addr, hwaddr *plen,
+                               bool is_write, MemTxAttrs attrs)
+{
+    unsigned int page_off = addr & (XC_PAGE_SIZE - 1);
+    unsigned int i;
+    unsigned int total_grants = 0;
+    unsigned int nrefs = (page_off + *plen + XC_PAGE_SIZE - 1) >> XC_PAGE_SHIFT;
+    uint32_t ref = (addr - XEN_GRANT_ADDR_OFF) >> XC_PAGE_SHIFT;
+    uint32_t *refs = NULL;
+    unsigned int prot = PROT_READ;
+    struct XENMappedGrantRegion *mgr = NULL;
+
+    if (is_write) {
+        prot |= PROT_WRITE;
+    }
+
+    qemu_mutex_lock(&xen_map_mutex);
+
+    QLIST_FOREACH(mgr, &xen_grant_mappings, list) {
+        if (mgr->idx == ref &&
+            mgr->pages == nrefs &&
+            (mgr->prot & prot) == prot) {
+            break;
+        }
+
+        total_grants += mgr->pages;
+    }
+
+    if (!mgr) {
+        if (nrefs + total_grants >= XEN_MAX_VIRTIO_GRANTS) {
+            qemu_mutex_unlock(&xen_map_mutex);
+            return NULL;
+        }
+
+        mgr = g_new(struct XENMappedGrantRegion, 1);
+
+        if (nrefs == 1) {
+            refs = &ref;
+        } else {
+            refs = g_new(uint32_t, nrefs);
+            for (i = 0; i < nrefs; i++) {
+                refs[i] = ref + i;
+            }
+        }
+        mgr->addr = xengnttab_map_domain_grant_refs(xen_region_gnttabdev, nrefs,
+                                                    xen_domid, refs, prot);
+        if (mgr->addr) {
+            mgr->pages = nrefs;
+            mgr->refs = 1;
+            mgr->prot = prot;
+            mgr->idx = ref;
+
+            QLIST_INSERT_HEAD(&xen_grant_mappings, mgr, list);
+        } else {
+            g_free(mgr);
+            mgr = NULL;
+        }
+    } else {
+        mgr->refs++;
+    }
+
+    qemu_mutex_unlock(&xen_map_mutex);
+
+    if (nrefs > 1) {
+        g_free(refs);
+    }
+
+    return mgr ? mgr->addr + page_off : NULL;
+}
+
+static void xen_unmap_grant_dyn(MemoryRegion *mr, void *buffer, ram_addr_t addr,
+                                hwaddr len, bool is_write, hwaddr access_len)
+{
+    unsigned int page_off = (unsigned long)buffer & (XC_PAGE_SIZE - 1);
+    unsigned int nrefs = (page_off + len + XC_PAGE_SIZE - 1) >> XC_PAGE_SHIFT;
+    unsigned int prot = PROT_READ;
+    struct XENMappedGrantRegion *mgr = NULL;
+
+    if (is_write) {
+        prot |= PROT_WRITE;
+    }
+
+    qemu_mutex_lock(&xen_map_mutex);
+
+    QLIST_FOREACH(mgr, &xen_grant_mappings, list) {
+        if (mgr->addr == buffer - page_off &&
+            mgr->pages == nrefs &&
+            (mgr->prot & prot) == prot) {
+            break;
+        }
+    }
+    if (mgr) {
+        mgr->refs--;
+        if (!mgr->refs) {
+            xengnttab_unmap(xen_region_gnttabdev, mgr->addr, nrefs);
+
+            QLIST_REMOVE(mgr, list);
+            g_free(mgr);
+        }
+    } else {
+        error_report("xen_unmap_grant_dyn() trying to unmap unknown buffer");
+    }
+
+    qemu_mutex_unlock(&xen_map_mutex);
+}
+
+static ram_addr_t xen_ram_addr_from_grant_cache(void *ptr)
+{
+    unsigned int page_off = (unsigned long)ptr & (XC_PAGE_SIZE - 1);
+    struct XENMappedGrantRegion *mgr = NULL;
+    ram_addr_t raddr = RAM_ADDR_INVALID;
+
+    qemu_mutex_lock(&xen_map_mutex);
+
+    QLIST_FOREACH(mgr, &xen_grant_mappings, list) {
+        if (mgr->addr == ptr - page_off) {
+            break;
+        }
+    }
+
+    if (mgr) {
+        raddr = (mgr->idx << XC_PAGE_SHIFT) + page_off + XEN_GRANT_ADDR_OFF;
+    }
+
+    qemu_mutex_unlock(&xen_map_mutex);
+
+    return raddr;
+}
+
+ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
+{
+    ram_addr_t raddr;
+
+    raddr = xen_ram_addr_from_mapcache_try(ptr);
+    if (raddr == RAM_ADDR_INVALID) {
+        raddr = xen_ram_addr_from_grant_cache(ptr);
+    }
+
+    return raddr;
+}
+
+static const struct MemoryRegionOps xen_grant_mr_ops = {
+    .map = xen_map_grant_dyn,
+    .unmap = xen_unmap_grant_dyn,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
 MemoryRegion *xen_init_grant_ram(void)
 {
     RAMBlock *block;
 
+    qemu_mutex_init(&xen_map_mutex);
+
+    xen_region_gnttabdev = xengnttab_open(NULL, 0);
+    if (xen_region_gnttabdev == NULL) {
+        fprintf(stderr, "can't open gnttab device\n");
+        return NULL;
+    }
+
     memory_region_init(&ram_grants, NULL, "xen.grants",
                        XEN_MAX_VIRTIO_GRANTS * XC_PAGE_SIZE);
     block = g_malloc0(sizeof(*block));
@@ -606,6 +779,7 @@ MemoryRegion *xen_init_grant_ram(void)
     ram_grants.ram_block = block;
     ram_grants.ram = true;
     ram_grants.terminates = true;
+    ram_grants.ops = &xen_grant_mr_ops;
     ram_block_add_list(block);
     memory_region_add_subregion(get_system_memory(), XEN_GRANT_ADDR_OFF,
                                 &ram_grants);
diff --git a/system/physmem.c b/system/physmem.c
index d989e9fc1f..e6fc075d8f 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2232,13 +2232,16 @@ RAMBlock *qemu_ram_block_from_host(void *ptr, bool round_offset,
 
     if (xen_enabled()) {
         ram_addr_t ram_addr;
+
         RCU_READ_LOCK_GUARD();
         ram_addr = xen_ram_addr_from_mapcache(ptr);
-        block = qemu_get_ram_block(ram_addr);
-        if (block) {
-            *offset = ram_addr - block->offset;
+        if (ram_addr != RAM_ADDR_INVALID) {
+            block = qemu_get_ram_block(ram_addr);
+            if (block) {
+                *offset = ram_addr - block->offset;
+            }
+            return block;
         }
-        return block;
     }
 
     RCU_READ_LOCK_GUARD();
-- 
2.17.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 06:38:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 06:38:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686404.1068392 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfDZw-0002ja-Uv; Wed, 28 Feb 2024 06:38:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686404.1068392; Wed, 28 Feb 2024 06:38:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfDZw-0002jT-Ru; Wed, 28 Feb 2024 06:38:08 +0000
Received: by outflank-mailman (input) for mailman id 686404;
 Wed, 28 Feb 2024 06:38:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfDZv-0002jJ-PR; Wed, 28 Feb 2024 06:38:07 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfDZv-0003NR-Ln; Wed, 28 Feb 2024 06:38:07 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfDZv-0005LK-BA; Wed, 28 Feb 2024 06:38:07 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfDZv-0004Dx-AQ; Wed, 28 Feb 2024 06:38:07 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=CqyYpvKb0BCXU6Nq+3a0e4wnpicFQEsbLYKWid3p51g=; b=Wo6hBl1KV2K1auIgaV6Hp8JGuf
	fAvDQi4Kcd0vORpYGmrKyMw9SSEbH5iYxyALKEDPLU7+yhMHiT1t0ewgQuNN4iq8edvZ/QDftFcDx
	hpERz/sQpcb/3G0927VhsB6CuAmmHjLASVipa8YZIJa4z7y7BOos2Ggyf6JvaCi+k9/I=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184785-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.16-testing test] 184785: regressions - trouble: broken/fail/pass
X-Osstest-Failures:
    xen-4.16-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:<job status>:broken:regression
    xen-4.16-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:host-install(5):broken:regression
    xen-4.16-testing:test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm:debian-hvm-install:fail:regression
    xen-4.16-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ffd75b161de76e43fd54d751ab70d2fdf6e2e87d
X-Osstest-Versions-That:
    xen=e481fc9f32339ebf9ddd171a3995a3e44527d148
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 06:38:07 +0000

flight 184785 xen-4.16-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184785/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm    <job status>      broken
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 5 host-install(5) broken REGR. vs. 184530
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. vs. 184530

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184530
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184530
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184530
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184530
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184530
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184530
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184530
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184530
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184530
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184530
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184530
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184530
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 xen                  ffd75b161de76e43fd54d751ab70d2fdf6e2e87d
baseline version:
 xen                  e481fc9f32339ebf9ddd171a3995a3e44527d148

Last test of basis   184530  2024-01-30 14:07:07 Z   28 days
Testing same since   184785  2024-02-27 13:36:59 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           broken  
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 fail    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm broken
broken-step test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm host-install(5)

Not pushing.

------------------------------------------------------------
commit ffd75b161de76e43fd54d751ab70d2fdf6e2e87d
Author: Jan Beulich <jbeulich@suse.com>
Date:   Tue Feb 27 14:13:21 2024 +0100

    x86: account for shadow stack in exception-from-stub recovery
    
    Dealing with exceptions raised from within emulation stubs involves
    discarding return address (replaced by exception related information).
    Such discarding of course also requires removing the corresponding entry
    from the shadow stack.
    
    Also amend the comment in fixup_exception_return(), to further clarify
    why use of ptr[1] can't be an out-of-bounds access.
    
    This is CVE-2023-46841 / XSA-451.
    
    Fixes: 209fb9919b50 ("x86/extable: Adjust extable handling to be shadow stack compatible")
    Signed-off-by: Jan Beulich <jbeulich@suse.com>
    Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
    master commit: 91f5f7a9154919a765c3933521760acffeddbf28
    master date: 2024-02-27 13:49:22 +0100
(qemu changes not included)


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 08:28:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 08:28:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686418.1068403 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfFIB-0007Te-5i; Wed, 28 Feb 2024 08:27:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686418.1068403; Wed, 28 Feb 2024 08:27:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfFIB-0007TX-1L; Wed, 28 Feb 2024 08:27:55 +0000
Received: by outflank-mailman (input) for mailman id 686418;
 Wed, 28 Feb 2024 08:27:54 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfFIA-0007TR-0J
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 08:27:54 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 432007f3-d613-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 09:27:51 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-5643ae47cd3so6728543a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 00:27:51 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 oy29-20020a170907105d00b00a42fb41e108sm1592011ejb.90.2024.02.28.00.27.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 00:27:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 432007f3-d613-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709108871; x=1709713671; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=tRJIBNBk85fUEukKDpIzLWxzV99t0JunP9xFi8z07Sk=;
        b=I+Vi8nzqicydyHvdAvq8iMllQtxk7Eu1WUd1fW+ir4ZR4oL90RQXASwzzlpMjZPKSf
         myjKnmcdG2J0+syunwksqe1krxtqv4FVCRgg6VqzMeoEb5BKr/2BwOLSuEUI1LeFFJWv
         voZBmbkSnyzqp4nrSZtuPlmrf68nfcrzGs3v0+AG0MqOquowMbxw37KkjPKDS/anUzN8
         WWIj6B6jeZlb/BpiDMp+aJTGluQAFn3cur0NxoWcD9GxbjuxEjtQw9gkNhUROHUwkRV4
         MXkyV4Gy3ieliyUYn7Vd20YZ9j9ukvu2JhePngEkWXNCaSCg75K7+yK+PEqXTU6FfD5q
         0WFA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709108871; x=1709713671;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=tRJIBNBk85fUEukKDpIzLWxzV99t0JunP9xFi8z07Sk=;
        b=akL9WpRc8a5biL6V4TmBCzrz3UYtcluqfNf07Tj4X4/JR9cJR64AVFzg2WhhrBAEPs
         Vi+PPEW2JIqC0/yItv2/iA42zDs+c5PwsgvrBddiVh355ml65/5On3l64bASdomMirwN
         /GcKa7NtYoMVqgS2QZ8MTnP8mZ0CxMhl+hh8Q+EiA6kie0Vuk96F18lfWxztOJ1vHcxy
         y1Jqp/oPJvi099RQrmuRCiBUaxyJSWbZAV5M1sd6qoBPQ1xR49EDbLvQo4LR4hjpteYw
         MBCZQKBJTbFameFAFebyD4XWTxLlev6jbxduBLks3OdcB9NkYLJ8bKnemjB1dIq0uwsn
         DM3A==
X-Forwarded-Encrypted: i=1; AJvYcCW7eqHu5sGP8pQ33R6DI41sllABZSC6VSEl9+1tvR05rt0u4+HErbeXI+b7ooP+WUn5xhCNkgRLnI+un3JamYxEtX5CVpHcBRS/VfSAxOQ=
X-Gm-Message-State: AOJu0YzB1fWoDz60a7mkq+iZcjayRdKaL/ABeqD4aZpg3DXL1oaX25br
	zGP+JrDI/j1LjkMmFRaLf5FhUgVZWnkecV1NnEdhWJEmVt64WeF/CRz91RasoQ==
X-Google-Smtp-Source: AGHT+IEjjhlehty8AlkfLKG31jBEbHvsivHHTTIfpVZadeN6yxSlIoco9e1fVfd9s16S1vJZONQTww==
X-Received: by 2002:a17:906:7c44:b0:a3e:90ec:2287 with SMTP id g4-20020a1709067c4400b00a3e90ec2287mr7915808ejp.26.1709108871060;
        Wed, 28 Feb 2024 00:27:51 -0800 (PST)
Message-ID: <3e131a4d-652a-4759-bf8a-2564b6c3859e@suse.com>
Date: Wed, 28 Feb 2024 09:27:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: preparations for 4.18.1
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Anthony Perard <anthony.perard@citrix.com>, Julien Grall <julien@xen.org>,
 Kelly Choi <kelly.choi@cloud.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott <dave@recoil.org>
References: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com>
 <ca4fe67a-3efd-4ce5-b37d-e68b1bf4c1a7@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ca4fe67a-3efd-4ce5-b37d-e68b1bf4c1a7@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 27.02.2024 15:57, Andrew Cooper wrote:
> Looking at the XenServer patchqueue, a couple to consider but nothing
> jumps out as critically urgent.
> 
> b6cf604207fd and 098d868e52ac as oxenstored perf fixes, although there's
> one incremental (non-functional) fix I'm still waiting on an ack on. 
> The backport to 4.18 is trivial.  The backport to 4.17 is miserable but
> I've already done it.

I wouldn't normally consider such for backport, but I'm not opposed if the
oxenstore maintainers agree (now Cc-ed) and if I can leave putting in these
two backports to you.

> ae6d4fd87676 which is a UBSAN fix, guest reachable but we believe not an
> issue in non-UBSAN builds.

Hmm, yes - no idea why I missed this one.

> cf7fe8b72dea which is the CPUID rescan on microcode load.

This is already there on the 4.18 branch, and it isn't applicable to the
upstream 4.17 one (I assume you did backport the commit referenced in the
Fixes: tag to your XenServer tree).

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 08:53:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 08:53:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686421.1068413 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfFgu-0002ie-44; Wed, 28 Feb 2024 08:53:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686421.1068413; Wed, 28 Feb 2024 08:53:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfFgu-0002iX-0A; Wed, 28 Feb 2024 08:53:28 +0000
Received: by outflank-mailman (input) for mailman id 686421;
 Wed, 28 Feb 2024 08:53:26 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=QaFI=KF=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rfFgs-0002iR-KM
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 08:53:26 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d50fc090-d616-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 09:53:25 +0100 (CET)
Received: from Dell.bugseng.com (unknown [37.163.36.84])
 by support.bugseng.com (Postfix) with ESMTPSA id 2019A4EE0737;
 Wed, 28 Feb 2024 09:53:23 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d50fc090-d616-11ee-afd7-a90da7624cb6
From: Federico Serafini <federico.serafini@bugseng.com>
To: xen-devel@lists.xenproject.org
Cc: consulting@bugseng.com,
	Federico Serafini <federico.serafini@bugseng.com>,
	Simone Ballarin <simone.ballarin@bugseng.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH] automation/eclair: extend deviations of MISRA C:2012 Rule 16.3
Date: Wed, 28 Feb 2024 09:53:12 +0100
Message-Id: <6c96f889ae3fbdf516f0edfe4a5fb207540b6862.1709109662.git.federico.serafini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Update ECLAIR configuration to deviate more cases where an
unintentional fallthrough cannot happen.

Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
---
 .../eclair_analysis/ECLAIR/deviations.ecl     | 15 +++++++++++++--
 docs/misra/deviations.rst                     | 19 ++++++++++++++++++-
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl b/automation/eclair_analysis/ECLAIR/deviations.ecl
index fd32ff8a9c..28ce251dc1 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -349,11 +349,14 @@ therefore have the same behavior of a boolean"
 # Series 16.
 #
 
--doc_begin="Switch clauses ending with continue, goto, return statements are
-safe."
+-doc_begin="Switch clauses ending with an unconditional flow control statement (i.e. continue, goto or return) are safe."
 -config=MC3R1.R16.3,terminals+={safe, "node(continue_stmt||goto_stmt||return_stmt)"}
 -doc_end
 
+-doc_begin="Switch clauses ending with an if-else statement are safe if both branches consist of a flow control statement (i.e., break, continue, goto, return)."
+-config=MC3R1.R16.3,terminals+={safe, "node(if_stmt)&&child(then,node(break_stmt||continue_stmt||goto_stmt||return_stmt))&&child(else,node(break_stmt||continue_stmt||goto_stmt||return_stmt))"}
+-doc_end
+
 -doc_begin="Switch clauses ending with a call to a function that does not give
 the control back (i.e., a function with attribute noreturn) are safe."
 -config=MC3R1.R16.3,terminals+={safe, "call(property(noreturn))"}
@@ -368,6 +371,14 @@ safe."
 -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(/BUG\\(\\);/))))"}
 -doc_end
 
+-doc_begin="On X86, switch clauses ending generating an exception through \"generate_exception()\" are safe."
+-config=MC3R1.R16.3,reports+={safe, "any_area(any_loc(file(^xen/arch/x86/.*$)&&any_exp(text(/generate_exception(.*?)/))))"}
+-doc_end
+
+-doc_begin="Switch clauses ending generating a parse error through \"PARSE_ERR_RET()\" are safe."
+-config=MC3R1.R16.3,reports+={safe, "any_area(any_loc(any_exp(text(/PARSE_ERR_RET(.*?)/))))"}
+-doc_end
+
 -doc_begin="Switch clauses not ending with the break statement are safe if an
 explicit comment indicating the fallthrough intention is present."
 -config=MC3R1.R16.3,reports+={safe, "any_area(end_loc(any_exp(text(^(?s).*/\\* [fF]all ?through.? \\*/.*$,0..1))))"}
diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
index 123c78e20a..dbff17f493 100644
--- a/docs/misra/deviations.rst
+++ b/docs/misra/deviations.rst
@@ -291,7 +291,14 @@ Deviations related to MISRA C:2012 Rules:
      - Project-wide deviation; tagged as `deliberate` for ECLAIR.
 
    * - R16.3
-     - Switch clauses ending with continue, goto, return statements are safe.
+     - Switch clauses ending with an unconditional flow control statement
+       (i.e., continue, goto, or return) are safe.
+     - Tagged as `safe` for ECLAIR.
+
+   * - R16.3
+     - Switch clauses ending with an if-else statement are safe if both
+       branches consist of a flow control statement (i.e., continue, break,
+       goto, return).
      - Tagged as `safe` for ECLAIR.
 
    * - R16.3
@@ -307,6 +314,16 @@ Deviations related to MISRA C:2012 Rules:
      - Switch clauses ending with failure method \"BUG()\" are safe.
      - Tagged as `safe` for ECLAIR.
 
+   * - R16.3
+     - On X86, switch clauses ending generating an exception through
+       \"generate_exception()\" are safe.
+     - Tagged as `safe` for ECLAIR.
+
+   * - R16.3
+     - Switch clauses ending generating a parse error through
+       \"PARSE_ERR_RET()\" are safe.
+     - Tagged as `safe` for ECLAIR.
+
    * - R16.3
      - Existing switch clauses not ending with the break statement are safe if
        an explicit comment indicating the fallthrough intention is present.
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 09:01:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 09:01:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686424.1068422 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfFox-0004Hh-Sc; Wed, 28 Feb 2024 09:01:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686424.1068422; Wed, 28 Feb 2024 09:01:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfFox-0004Ha-Q3; Wed, 28 Feb 2024 09:01:47 +0000
Received: by outflank-mailman (input) for mailman id 686424;
 Wed, 28 Feb 2024 09:01:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfFow-0004HO-RA; Wed, 28 Feb 2024 09:01:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfFow-0006St-P4; Wed, 28 Feb 2024 09:01:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfFow-0005GJ-HV; Wed, 28 Feb 2024 09:01:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfFow-0003aN-H1; Wed, 28 Feb 2024 09:01:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ntbnRwuIT13mG6Ba1lQWj6DSx2Hvot2BPv5ZevvbGTc=; b=IVzPMDjt77X0FmRvdRGCeMX67P
	88f09jat/BDkWKn15gyKfPa7wJvDvyV1vzQqaqYAgWxjI+FhOnF7RTV+U1TRF4IZR9KZkFcI6U81w
	PBJUzTA9lWc9Xk/sjXsklXdlcjr8L3O9UIY4DkL/nDC6cOL9gkUK9W3DFxh2KWs7WnMk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184803-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184803: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=3e91e421365027ee3e655feab33c67a4f544c777
X-Osstest-Versions-That:
    ovmf=aceb3490a2a350b128156fd4e36e53fc19739e4e
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 09:01:46 +0000

flight 184803 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184803/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 3e91e421365027ee3e655feab33c67a4f544c777
baseline version:
 ovmf                 aceb3490a2a350b128156fd4e36e53fc19739e4e

Last test of basis   184790  2024-02-27 15:45:00 Z    0 days
Testing same since   184803  2024-02-28 03:46:46 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jayaprakash N <n.jayaprakash@intel.com>
  Jayaprakash, N <n.jayaprakash@intel.com>
  Michael D Kinney <michael.d.kinney@intel.com>
  Rebecca Cran <rebecca@os.amperecomputing.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   aceb3490a2..3e91e42136  3e91e421365027ee3e655feab33c67a4f544c777 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 09:07:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 09:07:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686428.1068433 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfFtt-0004sf-Dh; Wed, 28 Feb 2024 09:06:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686428.1068433; Wed, 28 Feb 2024 09:06:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfFtt-0004sY-BE; Wed, 28 Feb 2024 09:06:53 +0000
Received: by outflank-mailman (input) for mailman id 686428;
 Wed, 28 Feb 2024 09:06:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfFts-0004sS-Jo
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 09:06:52 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b4b7fe38-d618-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 10:06:50 +0100 (CET)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-55a035669d5so9201334a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 01:06:49 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 t10-20020aa7d70a000000b00566626a355bsm535541edq.27.2024.02.28.01.06.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 01:06:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b4b7fe38-d618-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709111209; x=1709716009; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=dBTQme72G1D1WHjl6s4FaCExq4+OBKBcEuq5q+muxj0=;
        b=RMxEQPS5aduAZRDjTlqU/Sl3p7cptdbc1uT4w2jdPbKy9wgrwCG0IVFbKqp44Ele0x
         MxNAf+/09tCnaD9i2dY/zhlpuHvfpxm+utS8GmE3/n5HCFNkfvBpzcLFnCz6/pILRlhE
         rq2iE8CHwpKAKPNYWciMrSHFEmnFrQp3LMR25A7YWGO1dDcHZmTTDOVyku9Feu4jzm3D
         KeoQcrtMQKHmGDOq1eIRpm7VzQuOVp+v3P1R2AMzMEzygbq4Oh+T0kd5aIFs286w7SNk
         a/CUnpsclaWOJ2kHi0DLfeHp1nY2CFiU5YUStIMRuGPG8baeLJSRshQ07EjA5XOMcJEW
         nIKw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709111209; x=1709716009;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=dBTQme72G1D1WHjl6s4FaCExq4+OBKBcEuq5q+muxj0=;
        b=h7roNA7FiEYwglkY3QCxa6wg1/PFYijcsQ5bI+dh1MIMPqrbA83bHDby+zgC5eOqVu
         cC5Ml5z63SqKuzDA5R0p37bakHhELSlEf5MPe3b0MiJAxxNfROYkprk/luo5cIvdBfWG
         nDBstKTBSUtzF2+umatEbbD/OuPYlGgGkEjCanSFzWfRXEADbz7sYHpiLm6htmffSRR4
         PU77Au37cJyNEXJp3ttJAwPtOkw5nIx08+zuC3w84zqEphEnqjfW7/0OUJ7WMJraVLzN
         oGm62HL3DLBcB1iareGfmNZdE3HEm43OcALQrgn38IJx4nj6sYyitQII3/mTplLSWFeD
         lJxQ==
X-Forwarded-Encrypted: i=1; AJvYcCWUxjCyFs7WIEp4d84ipNdrjvgOi7y9Qgm3MxTX8FZE6mLswkT8pSM/R4OIQgDIoUJ3eiItNLOBHcmk4BSycUP4PyqXf3vK/KL6lPNVSqs=
X-Gm-Message-State: AOJu0Yx1b5MFg+ebCQyTvyDTBVH6bFNxFG5/4OogJanJ1ojXA9nJADM9
	Ow1cdQcx9a1c/41Nthvi+A1G7xdYpuHGLqGlz3caBFTC1aiCKWg0tJPl2D0rcA==
X-Google-Smtp-Source: AGHT+IFGcWHX5vCqXQ32/fcePpSmxTM2nndWl/lu4VRKWx2TC2uOWDn8t8KdPtLZDDkamBmOvyzOxg==
X-Received: by 2002:aa7:d958:0:b0:565:5924:9d8b with SMTP id l24-20020aa7d958000000b0056559249d8bmr8521051eds.35.1709111209179;
        Wed, 28 Feb 2024 01:06:49 -0800 (PST)
Message-ID: <1a00c221-ce1c-45be-9ab3-e8f1a0e5678d@suse.com>
Date: Wed, 28 Feb 2024 10:06:47 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair: extend deviations of MISRA C:2012
 Rule 16.3
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <6c96f889ae3fbdf516f0edfe4a5fb207540b6862.1709109662.git.federico.serafini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <6c96f889ae3fbdf516f0edfe4a5fb207540b6862.1709109662.git.federico.serafini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 28.02.2024 09:53, Federico Serafini wrote:
> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl

Comments below apply similarly to text added to this file.

> --- a/docs/misra/deviations.rst
> +++ b/docs/misra/deviations.rst
> @@ -291,7 +291,14 @@ Deviations related to MISRA C:2012 Rules:
>       - Project-wide deviation; tagged as `deliberate` for ECLAIR.
>  
>     * - R16.3
> -     - Switch clauses ending with continue, goto, return statements are safe.
> +     - Switch clauses ending with an unconditional flow control statement
> +       (i.e., continue, goto, or return) are safe.
> +     - Tagged as `safe` for ECLAIR.

With this edit (unmentioned in the description, btw) ...

> +   * - R16.3
> +     - Switch clauses ending with an if-else statement are safe if both
> +       branches consist of a flow control statement (i.e., continue, break,
> +       goto, return).

... why is it not also "ending with" here?

Also what about either situation ending with a call to a noreturn function?

> @@ -307,6 +314,16 @@ Deviations related to MISRA C:2012 Rules:
>       - Switch clauses ending with failure method \"BUG()\" are safe.
>       - Tagged as `safe` for ECLAIR.
>  
> +   * - R16.3
> +     - On X86, switch clauses ending generating an exception through
> +       \"generate_exception()\" are safe.
> +     - Tagged as `safe` for ECLAIR.

This macro is limited to the emulator, so shouldn't be deviated globally.
Furthermore - why does the special case need mentioning here? Shouldn't
it be the underlying pattern which is deviated (along the lines of the
earlier ones):

    if ( true )
    {
        ...
        goto ...; /* Or break / continue / return */
    }

> +   * - R16.3
> +     - Switch clauses ending generating a parse error through
> +       \"PARSE_ERR_RET()\" are safe.
> +     - Tagged as `safe` for ECLAIR.

Again this isn't a global scope macro, so shouldn't be deviated globally.
Plus it ends in "return", so ought to be covered by the earlier clause.
The fact that the return is in a body of do {} while(0) shouldn't matter
at all - that's purely syntactic sugar.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 09:10:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 09:10:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686430.1068443 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfFx9-0006Sh-U8; Wed, 28 Feb 2024 09:10:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686430.1068443; Wed, 28 Feb 2024 09:10:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfFx9-0006Sa-Pn; Wed, 28 Feb 2024 09:10:15 +0000
Received: by outflank-mailman (input) for mailman id 686430;
 Wed, 28 Feb 2024 09:10:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfFx8-0006SS-Rw
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 09:10:14 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2e672aba-d619-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 10:10:13 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a3e6f79e83dso613775866b.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 01:10:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 he37-20020a1709073da500b00a3e51df2280sm1598788ejc.223.2024.02.28.01.10.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 01:10:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2e672aba-d619-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709111413; x=1709716213; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=6BxTp3fIYjYk9mhDu7zbmBjT4KnrNQ7URUXMzdcEneg=;
        b=OLJfUW85JnZvyiIRKWbaP2be0eecXfQfhIJ28gOqKA951N4beu9rSaxA3mEkxVk6iX
         3PgNUwI3n/CF00KkK5qqUTAm+3eIUfHKDsUrNO8bLJDigkwO6I6QYqpoo5r46Hlehjot
         z8ke9yznb6ibS7M1N3FMUUo3HOOoI+xbFH+Hm7/Z6jetvetkxxJ34aJv7puN+dKqRpFp
         K8DtogaMLwm6KX6oLs0BiqyuL+HbA63Wul1IUtptuVZmbIhMflFSw33JJV8sEAdTPJx2
         NjVVbyb7931awtUaoM8mI2deS4Mbm1ifMJrWfJDlRP/P8GyrSpfO5lpXtkxpwlqu80b+
         6+CQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709111413; x=1709716213;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6BxTp3fIYjYk9mhDu7zbmBjT4KnrNQ7URUXMzdcEneg=;
        b=cEvhAcUSTXBvKpAGMRvPkdsQVycAakfoe4e6An9ZYGfBx2142RIn2lJlW/W78rQmHM
         VtCPvEjYriKdT/o+BHPu1iJ33cooWjKlwNL2bY9pPuP5XrwL264SyxM3POK1DvxJNHjX
         Pi/SpxNCr6TTVfombS5bo0igBtIRVnh7z3BDpQ/7CiwBH7IOPwj6eTzr6bN+dmf1lGNI
         55nBMSGYkzjKCCiD6G5l7WNze32yeF3L1eMf9r/n336tP8AXoW8uq23vmuwXQXdlX8Jd
         YRDjLqSmc/L4yINwn7YRHX4KNrkupM3q8gVWvV+kDv2o2Oy2NP+At7qfUtftZ1mFPnMD
         HnLQ==
X-Forwarded-Encrypted: i=1; AJvYcCXNCeo53QiCGgAajyJTumPfWmhA9FBvLux39rtoYplx3DnGXCP5KMbijSZRLptJu5z8xE6tQinp0JTsbhper/3++NJ/diMEBlfTuL2+THQ=
X-Gm-Message-State: AOJu0Yyiy56piklB8PKUVWyJswuTV4yolq5mf70JQKmzzdGy+OvDuoFs
	AYdMY0GlqdmxYgnlL0tPteZYjPJugv1qbxTtqlLwLvM2QUmUZbJ5/aJ23PdVTA==
X-Google-Smtp-Source: AGHT+IG6YRx/4CTeETPale057kTXHxFTbbgMM0y7NPCOG2ytcYGW/Px+F7LNY0QM0llYr5ivFsfRxg==
X-Received: by 2002:a17:906:34cb:b0:a3f:7e1:5166 with SMTP id h11-20020a17090634cb00b00a3f07e15166mr8086118ejb.10.1709111413181;
        Wed, 28 Feb 2024 01:10:13 -0800 (PST)
Message-ID: <654dc3e7-b7c3-439a-b784-38834989bd43@suse.com>
Date: Wed, 28 Feb 2024 10:10:12 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86: Resync intel-family.h from Linux
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240227160940.2054111-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240227160940.2054111-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 27.02.2024 17:09, Andrew Cooper wrote:
> From v6.8-rc8

Typo or time leap? There wasn't even rc7 yet ...

> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Wed Feb 28 09:59:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 09:59:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686439.1068452 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfGiY-0003L9-G9; Wed, 28 Feb 2024 09:59:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686439.1068452; Wed, 28 Feb 2024 09:59:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfGiY-0003L2-Da; Wed, 28 Feb 2024 09:59:14 +0000
Received: by outflank-mailman (input) for mailman id 686439;
 Wed, 28 Feb 2024 09:59:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M6mx=KF=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rfGiX-0003Kd-9D
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 09:59:13 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 052f5193-d620-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 10:59:11 +0100 (CET)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-513181719easo720399e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 01:59:11 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 o26-20020ac2435a000000b00512d747866asm1478764lfl.127.2024.02.28.01.59.09
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 01:59:09 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 052f5193-d620-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709114350; x=1709719150; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=uP8g1BKObLvdPj3GeR3kbFtD7/85sFX7fNf7mAGODJY=;
        b=D0XeA98httkw+tKaf8G/SeQalf8hDGHdkhOzzDljQkf+03iFiBAXYYFgcwB/BMPhWE
         NDKgCwhF/003m2Los0kf9q00ykeS+e5vbb6PcrMforZ5CxwV/AZ9aK4/RJZIm3MvLdEi
         732y7JTAly7QlKWfeiyGkl13E/jm5k4HhysTZP+ym83VRgaS+HFTuFqwV3LfhcIpCk6X
         Gb1G0/fH2fNjeb/JMoyPVaEE4IL4fzcfmbIfjqI8fY57W7AEvwkWhNmxGfbL9XQeBkRv
         aHaTWEbF0Jvz60ybFJ140iOGVkpvuA9Y5lEjkPbZv1lfDPVENytQXUVbj00Mlqwdr3d6
         L+3A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709114350; x=1709719150;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=uP8g1BKObLvdPj3GeR3kbFtD7/85sFX7fNf7mAGODJY=;
        b=Uv5nnYB1kTxVzXkFKPwNbmWOyLA5THW+VxWPQsSFPh5Nc9CNuLJIFwrvRM3q9He3+3
         FGOayzSbUHfZan09HcuBKvh0z/vQUiO0Tq3ZH4utfozzbBOZ1X8Y7kYZlrNJFcglYz1i
         wZC0ep6kqOj4Qx5y8sB5U6mBNvaqACnc8K75xcSrjBUDhs/Mk+QAVXDpc6Qi1BXImHGy
         VGyOD7hNjWCi4M/XL1U2oAUhAVjzkjZt/8PiQHlxQbEcqL8Zf+C8FZ4G0Uw0me1WFMq+
         CvV0WPYraL844yRa8hpJ/CfDa0PwGwIlcOxPHADEEpScStVBmx6QjAAvceBITv9g0zZh
         6k5A==
X-Forwarded-Encrypted: i=1; AJvYcCWQJx/htcvnzGxS99/2ka9K4wM26RCFrvS5hlRdLF/wQuHEm+IRZ1UnDP01MddDaBjObasM8VS7BoHxLn58iO1azelOJvuubvbKgM2baBg=
X-Gm-Message-State: AOJu0YwxANCiZAjR1WmZcG0GbDRZd1SM/RcaX6VLBUA8gu/L8VKhMo1u
	3/t8tocVMJV2fgyzhABNl4W/5MtP1o/x+LgNziYa8mADTmRYR1l2
X-Google-Smtp-Source: AGHT+IHmWKmpOZ9MM/CGGUSiULBpzWz/jGboiIzRrB7SF6fRiiBgLSKtb96rZ9tbbZtmTQa2kHbhyw==
X-Received: by 2002:a05:6512:3d1a:b0:513:43e:e22d with SMTP id d26-20020a0565123d1a00b00513043ee22dmr3603754lfv.35.1709114350294;
        Wed, 28 Feb 2024 01:59:10 -0800 (PST)
Message-ID: <3637804a9dde10871b4f4a354cced82548114893.camel@gmail.com>
Subject: Re: [PATCH v5 03/23] xen/riscv: introduce nospec.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
 xen-devel@lists.xenproject.org
Date: Wed, 28 Feb 2024 10:59:09 +0100
In-Reply-To: <293c6236-1ad0-4a48-853f-afec419656e9@suse.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
	 <16370492de5344ae8e1aab688261600491868f4e.1708962629.git.oleksii.kurochko@gmail.com>
	 <293c6236-1ad0-4a48-853f-afec419656e9@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Tue, 2024-02-27 at 08:38 +0100, Jan Beulich wrote:
> On 26.02.2024 18:38, Oleksii Kurochko wrote:
> > From the unpriviliged doc:
> > =C2=A0 No standard hints are presently defined.
> > =C2=A0 We anticipate standard hints to eventually include memory-system
> > spatial
> > =C2=A0 and temporal locality hints, branch prediction hints, thread-
> > scheduling
> > =C2=A0 hints, security tags, and instrumentation flags for
> > simulation/emulation.
> >=20
> > Also, there are no speculation execution barriers.
> >=20
> > Therefore, functions evaluate_nospec() and block_speculation()
> > should
> > remain empty until a specific platform has an extension to deal
> > with
> > speculation execution.
>=20
> What about array_index_mask_nospec(), though? No custom
> implementation,
> meaning the generic one will be used there? If that's the intention,
> then ...
Yes, the generic one will be used.

~ Oleksii
>=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>=20
> Acked-by: Jan Beulich <jbeulich@suse.com>
>=20
> Jan
>=20



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 10:15:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 10:15:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686455.1068515 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfGy4-0006gt-BZ; Wed, 28 Feb 2024 10:15:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686455.1068515; Wed, 28 Feb 2024 10:15:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfGy4-0006gm-8c; Wed, 28 Feb 2024 10:15:16 +0000
Received: by outflank-mailman (input) for mailman id 686455;
 Wed, 28 Feb 2024 10:15:14 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfGy2-0006gc-PH; Wed, 28 Feb 2024 10:15:14 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfGy2-0007re-Lx; Wed, 28 Feb 2024 10:15:14 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfGy2-0001fJ-AD; Wed, 28 Feb 2024 10:15:14 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfGy2-0007jG-9g; Wed, 28 Feb 2024 10:15:14 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=UO9kdt9E5ctXIi0KAk2/pD8+oAl6CnppQk409YYu+/k=; b=ViQfD3P31F6siRwVOfmlrQQT+6
	9wKgSjl3nv5LNnOp41Ak7N6ujbOAzLmt2+jpl90YJfcdYWK6DY8ee1VT7WoC4GjWZ7vS/2aPgHwzk
	/0puoVKh1xW8jQGe0r12qxJzcdljor4eNw24IWluUoEYytSGgSWuzMeG/iQkyJKhJc/4=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184786-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.17-testing test] 184786: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-4.17-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.17-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=49f77602373b58b7bbdb40cea2b49d2f88d4003d
X-Osstest-Versions-That:
    xen=091466ba55d1e2e75738f751818ace2e3ed08ccf
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 10:15:14 +0000

flight 184786 xen-4.17-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184786/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184564
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184564
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184564
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184564
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184564
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184564
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184564
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184564
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184564
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184564
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184564
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184564
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass

version targeted for testing:
 xen                  49f77602373b58b7bbdb40cea2b49d2f88d4003d
baseline version:
 xen                  091466ba55d1e2e75738f751818ace2e3ed08ccf

Last test of basis   184564  2024-02-02 07:07:04 Z   26 days
Testing same since   184786  2024-02-27 13:37:10 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Cyril Rébert (zithro) <slack@rabbit.lu>
  Jan Beulich <jbeulich@suse.com>
  Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
  oxjo <oxjo@proton.me>
  Paul Durrant <paul@xen.org>
  Petr Beneš <w1benny@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   091466ba55..49f7760237  49f77602373b58b7bbdb40cea2b49d2f88d4003d -> stable-4.17


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 10:35:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 10:35:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686460.1068524 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHHq-0000wu-VU; Wed, 28 Feb 2024 10:35:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686460.1068524; Wed, 28 Feb 2024 10:35:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHHq-0000wn-SK; Wed, 28 Feb 2024 10:35:42 +0000
Received: by outflank-mailman (input) for mailman id 686460;
 Wed, 28 Feb 2024 10:35:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfHHp-0000wf-GG
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 10:35:41 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfHHo-0008Co-Rr; Wed, 28 Feb 2024 10:35:40 +0000
Received: from [15.248.2.225] (helo=[10.45.19.69])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfHHo-0003U9-JW; Wed, 28 Feb 2024 10:35:40 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=VblhwjmdAP0Edj1ztR7piXC5kmWcaYAOHgs0OAj9eVA=; b=f0o4rKtK1rZf+PBT9lWhR422v1
	vKR4w9J+HZd55IIkq8roGPY/MYIOTqSbGeIiGAcG5AVDEVfb5YYR+QEKEIWLBdarEb11PsWSpbzKF
	5IH+X9bKbFpwU/J++pI/OIumDUQ12oEBPm/iZTeY4LX/8pBKkjeZwe86XdE7AF64Ep+g=;
Message-ID: <3982ba47-6709-47e3-a9c2-e2d3b4a2d8e3@xen.org>
Date: Wed, 28 Feb 2024 10:35:37 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <d1518124-483c-4409-9b36-6a3392378911@xen.org>
 <a84aeb87-17e8-4195-90cb-7b0123064106@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <a84aeb87-17e8-4195-90cb-7b0123064106@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Henry,

On 27/02/2024 13:17, Henry Wang wrote:
> (-RISC-V and PPC people to avoid spamming their inbox as this is quite 
> Arm specific)
> 
> Hi Julien,
> 
> On 2/26/2024 5:13 PM, Julien Grall wrote:
>> Hi Henry,
>>
>> Welcome back!
> 
> Thanks!
> 
>> On 26/02/2024 01:19, Henry Wang wrote:
>>> An error message can seen from the init-dom0less application on
>>> direct-mapped 1:1 domains:
>>> ```
>>> Allocating magic pages
>>> memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
>>> Error on alloc magic pages
>>> ```
>>>
>>> This is because populate_physmap() automatically assumes gfn == mfn
>>> for direct mapped domains. This cannot be true for the magic pages
>>> that are allocated later for Dom0less DomUs from the init-dom0less
>>> helper application executed in Dom0.
>>>
>>> Force populate_physmap to take the "normal" memory allocation route for
>>> the magic pages even for 1:1 Dom0less DomUs. This should work as long
>>> as the 1:1 Dom0less DomU doesn't have anything else mapped at the same
>>> guest address as the magic pages:
>>> - gfn 0x39000 address 0x39000000
>>> - gfn 0x39001 address 0x39001000
>>> - gfn 0x39002 address 0x39002000
>>> - gfn 0x39003 address 0x39003000
>>
>> This is very fragile. You are making the assumption that the magic 
>> pages are not clashing with any RAM region. The layout defined in 
>> arch-arm.h has been designed for guest where Xen is in full control of 
>> the layout. This is not the case for directmapped domain. I don't 
>> think it is correct to try to re-use part of the layout.
> 
> Apologies for the (a bit) late reply, it took a bit longer for me to 
> understand the story about directmap stuff, and yes, now I agree with 
> you, for those directmapped domains we should not reuse the guest layout 
> directly.
> 
>> If you want to use 1:1 dom0less with xenstore & co, then you should 
>> find a different place in memory for the magic pages (TDB how to find 
>> that area). 
> 
> Yes, and maybe we can use similar strategy in find_unallocated_memory() 
> or find_domU_holes() to do that.
> 
>> You will still have the problem of the 1:1 allocation, but I think 
>> this could be solved bty adding a flag to force a non-1:1 allocation.
> 
> After checking the code flow, below rough plan came to my mind, I think 
> what we need to do is:
> 
> (1) Find a range of un-used memory using similar method in 
> find_unallocated_memory()/find_domU_holes()

AFAIK, the toolstack doesn't have any knowledge of the memeory layout 
for dom0less domUs today. We would need to expose it first.

Then the region could either be statically allocated (i.e. the admin 
provides it in the DTB) or dynamically.

> 
> (2) Change the base address, i.e. GUEST_MAGIC_BASE in alloc_xs_page() in 
> init-dom0less.c to point to the address in (1) if static mem or 11 
> directmap. (I think this is a bit tricky though, do you have any method 
> that in your mind?)

AFAIK, the toolstack doesn't know whether a domain is direct mapped or 
using static mem.

I think this ties with what I just wrote above. For dom0less domUs, we 
probably want Xen to prepare a memory layout (similar to the e820) that 
will then be retrieved by the toolstack.

> 
> (3) Use a flag or combination of existing flags (CDF_staticmem + 
> CDF_directmap) in populate_physmap() to force the allocation of these 
> magic pages using alloc_domheap_pages() - i.e. the "else" condition in 
> the bottom

If I am not mistaken, CDF_* are per-domain. So we would want to use MEMF_*.

>>> +
>>>   #endif /* __ASM_X86_MM_H__ */
>>> diff --git a/xen/common/memory.c b/xen/common/memory.c
>>> index b3b05c2ec0..ab4bad79e2 100644
>>> --- a/xen/common/memory.c
>>> +++ b/xen/common/memory.c
>>> @@ -219,7 +219,7 @@ static void populate_physmap(struct memop_args *a)
>>>           }
>>>           else
>>>           {
>>> -            if ( is_domain_direct_mapped(d) )
>>> +            if ( is_domain_direct_mapped(d) && !is_magic_gpfn(gpfn) )
>>
>> This path will also be reached by dom0. Effectively, this will prevent 
>> dom0 to allocate the memory 1:1 for the magic GPFN (which is guest 
>> specific).
> 
> I think above proposal will solve this issue.
> 
>> Also, why are you only checking the first GFN? What if the caller pass 
>> an overlapped region?
> 
> I am a bit confused. My understanding is at this point we are handling 
> one page at a time.

We are handling one "extent" at the time. This could be one or multiple 
pages (see extent_order).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 10:36:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 10:36:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686462.1068534 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHII-0001N9-BN; Wed, 28 Feb 2024 10:36:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686462.1068534; Wed, 28 Feb 2024 10:36:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHII-0001N2-8d; Wed, 28 Feb 2024 10:36:10 +0000
Received: by outflank-mailman (input) for mailman id 686462;
 Wed, 28 Feb 2024 10:36:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NMA4=KF=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rfHIH-0001Ml-8H
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 10:36:09 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2cb73f51-d625-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 11:36:05 +0100 (CET)
Received: from BN8PR16CA0032.namprd16.prod.outlook.com (2603:10b6:408:4c::45)
 by DS7PR12MB8274.namprd12.prod.outlook.com (2603:10b6:8:da::13) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Wed, 28 Feb
 2024 10:36:02 +0000
Received: from BN2PEPF000044A2.namprd02.prod.outlook.com
 (2603:10b6:408:4c:cafe::44) by BN8PR16CA0032.outlook.office365.com
 (2603:10b6:408:4c::45) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.28 via Frontend
 Transport; Wed, 28 Feb 2024 10:36:02 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN2PEPF000044A2.mail.protection.outlook.com (10.167.243.153) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 10:36:01 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 28 Feb
 2024 04:36:01 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 28 Feb
 2024 04:36:01 -0600
Received: from XIR-MICHALO-L1.xilinx.com (10.180.168.240) by
 SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35
 via Frontend Transport; Wed, 28 Feb 2024 04:36:00 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2cb73f51-d625-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PQvhUsjMCn5OFne3T09KQ6Gu9y5CLQeYkWHWAlyTyWk9hv2i0MpwWtDBniQlqzniPElmLqrNxv5/CUD1nD2j8K5ting89EmBzisswyjnQmYUfDHeWzv2ceIjCsnrbeZoz/gwZTh+o91IxnGCmGZZvVIrgiS0YYeZCeOEycebOr2n+qy11jYShU5IdIiNzph6kTgbCLZa0bk4XwBUBNfSij60RghMBN6oMoHJ6UFCTxVYA5/eE08tN3h1ArLi/Wy9Yh02PzGpFQInFGOwuN9f2l6RFkrEP2SoEMTYfazebOXgWd2x7wda2VYHyXK7QnoCLheZfdaym2xCdJFFvnBpEQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=UJETay2uUVAJ5SihwmOLSlS2wrmi7o0ZhePbQ5L7+O4=;
 b=W+Yq9sLfkSpVfHFkqxzhDxcQ51REd81eE2zHq15U26rwxuc5441TUnIfURRQYQGxMSRQ1rTLrN2zgDTWpBKDB8UN3B6SAKnmcb+NQlE1jeSa3zoZ1rNakvc4C3sXT1VRIDpanPiZKREqBF4FRDBemPo4OEHT3LSBiI9GJr45Tkq+lXOys6yVvZs9EQ2exQD9Zkgs+D2/6Ux4mX2XoEVixcfvITG+zeZPZlu+W0oyeDbNiF+klIjzNZeGjjgCHLajzVAAYwVUeTPsA5xeI4agC1y4cp7lekPmbuaIKgLh1pjaM8vqcWWerNb/N9pbu+HCnVPVfSYd0v7/ABhcWDZirQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=UJETay2uUVAJ5SihwmOLSlS2wrmi7o0ZhePbQ5L7+O4=;
 b=GFFdZqwsOPGlqobLh0qOO8x6LR3F4sHQcXRD1ESQfu9rlipD2ZfbcdDchN+IchHTh2NWDeZ08qvJxJehiUkWRCkbvj7vUULRZYv/h7xZP5ebJkEApDJpSa3PpO9GqrrUhFUK6Hk1nMsTgY9Trl+8U545j61lxvOJcvcgBkHwBDE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Michal Orzel <michal.orzel@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Michal Orzel <michal.orzel@amd.com>, Stefano Stabellini
	<sstabellini@kernel.org>, Julien Grall <julien@xen.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [PATCH] xen/arm: Fix arm32 build failure when early printk is enabled
Date: Wed, 28 Feb 2024 11:35:55 +0100
Message-ID: <20240228103555.172101-1-michal.orzel@amd.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF000044A2:EE_|DS7PR12MB8274:EE_
X-MS-Office365-Filtering-Correlation-Id: f79f49a2-41b8-4e27-1b29-08dc38490f43
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	4zCMVFiEf2wW+hvfBrN5IKQoyT09ngaqz8oCRW9sH1+RNR4h/0ji/l8CLDi6ZYyOyjDCuQtVAsec8joYtXcGirtdSMB196ifu7MaBB420VonrkUdUlLrpQZQeXrJmUev5TWzXiACvGPEynmO9Jv+OR3zf93XRkl9q1iJIybVnX0mgp8u/KF1FzU7BPsjrRQXxsYVqeAr+QxtFRpRkt+0AWsOoneFLG+d4d2rqc2MlnlnFgg1pKLIvzDUjxgKwB4fy6oOf0KB6tOYg0KGf+rTw17zqMnpuqNNUz/A9oag0KL7txY/oZc2UviiSoy5x8YwwEpPwYybRD6rAje9VruZ15ktLGv0+oHisCeqqtDh0JgqaAFh5YWm3nj+9eBZLrZYwhccop5wG5NHoh25o2AqDFWnSfIYSYPoFsyxr3T3z3qyZL56+sCKpKj4HpVGZc2mEaZnn2aWTWjPzWph8RSUNdAynjacn6G3Hgua2pGr70DChj1QsgfG3yiHOcq37z4vZUVqkQg0AHCdfumhuGd08UqmG9GzuH9xRv05N08YDmTJtzH0SP0EyhumRA//xKobd7VP30eW5VN22z0LuG0B8Nq6tUP5vIh28y66+Hm8d2JELKedtVcwywInrL6cYRzYfptgQdMZ8lJvcBglYdfYpOczVyp/oM4smlBYnBUHUrWXYLyL9kZOdRL/twp8aR/cFHIeE30psXg6o3fyZ0uQRJ1q/vQkYJzZMJCTBJPuo/ibbS7ht2OrjRSU/XHDAcAy
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 10:36:01.9123
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: f79f49a2-41b8-4e27-1b29-08dc38490f43
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF000044A2.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8274

Commit 0441c3acc7e9 forgot to rename FIXMAP_CONSOLE to FIX_CONSOLE in
TEMPORARY_EARLY_UART_VIRTUAL_ADDRESS macro. This results in a build
failure on arm32, when early printk is enabled:
arch/arm/arm32/mmu/head.S:311: Error: invalid operands (*UND* and *ABS* sections) for `*'

Fixes: 0441c3acc7e9 ("xen/arm: fixmap: Rename the fixmap slots to follow the x86 convention")
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
 xen/arch/arm/include/asm/early_printk.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/include/asm/early_printk.h b/xen/arch/arm/include/asm/early_printk.h
index f444e89a8638..46a5e562dd83 100644
--- a/xen/arch/arm/include/asm/early_printk.h
+++ b/xen/arch/arm/include/asm/early_printk.h
@@ -20,7 +20,7 @@
     (FIXMAP_ADDR(FIX_CONSOLE) + (CONFIG_EARLY_UART_BASE_ADDRESS & ~PAGE_MASK))
 
 #define TEMPORARY_EARLY_UART_VIRTUAL_ADDRESS \
-    (TEMPORARY_FIXMAP_ADDR(FIXMAP_CONSOLE) + (CONFIG_EARLY_UART_BASE_ADDRESS & ~PAGE_MASK))
+    (TEMPORARY_FIXMAP_ADDR(FIX_CONSOLE) + (CONFIG_EARLY_UART_BASE_ADDRESS & ~PAGE_MASK))
 
 #endif /* !CONFIG_EARLY_PRINTK */
 
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 10:54:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 10:54:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686467.1068545 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHZK-0004Sq-QI; Wed, 28 Feb 2024 10:53:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686467.1068545; Wed, 28 Feb 2024 10:53:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHZK-0004Sj-Mb; Wed, 28 Feb 2024 10:53:46 +0000
Received: by outflank-mailman (input) for mailman id 686467;
 Wed, 28 Feb 2024 10:53:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KqYS=KF=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rfHZJ-0004Sb-Co
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 10:53:45 +0000
Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com
 [2607:f8b0:4864:20::82f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a3d461a8-d627-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 11:53:44 +0100 (CET)
Received: by mail-qt1-x82f.google.com with SMTP id
 d75a77b69052e-42e75559043so23467441cf.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 02:53:44 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 o6-20020ac85546000000b0042e637083b7sm4406140qtr.26.2024.02.28.02.53.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 02:53:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a3d461a8-d627-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709117623; x=1709722423; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=IELvhF5UbL2ab4kEVnUWT0FKk9lvIAUI7sawg12kAUI=;
        b=qUB6m/S7rf0mTYki+XwBUzFrfhxRhzFsmS+M4aD7iFbwl8S8LP5v1LO84YwWeK9Ge3
         84q7E9U05OcXXubUzoNzKD4LqoqJRK/CKii1jbwE1GCEJcFLrVMBtFDrqKBaqCr95SUI
         FWMVt64X/hgK4wr6eqp/vGPunK+1E3HaaIUnw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709117623; x=1709722423;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IELvhF5UbL2ab4kEVnUWT0FKk9lvIAUI7sawg12kAUI=;
        b=mKUGzSgCtf7UsO+Apo3WFqbhOHliPeozkujlVR5pnS3rUEUr2A0TX0TiH9pDrah3mC
         e80Yh1APKFZ1D5OC0YftUlDPUKR7eo7mB5j6P5ETf+gbrdfjdHsKZPiFr1Kj/bk2GFEO
         LAS/tPWCZ7K9QzRpHt75PXYDJShHltM8erakNzL5MmaB1SBJatNQZNbTPdgLblFvbMLl
         h4n4++rWsBaPMvtT44VAe3HpUS+XNxRL2fiZ5U0GRwk33Bk/9mhDDY1WFGr4a3vT3Ggm
         7kaqRbUhhiEOlUyzrvOiMyRYPe7j8ebvcBCCCqwZFky9BskbVxNkAlRSJXnqBKrBjUnS
         MwIQ==
X-Forwarded-Encrypted: i=1; AJvYcCXwVMK0zGFDFpnkyVMXu8h5x8FdCM9Y4QdHZ+LhM7yCOSs3zym6cZclY6fusRiiGSgtUh0ZYmnGK8i6vemU1zUuYCpOea0FDSMrM0VSA1A=
X-Gm-Message-State: AOJu0YwAlj+fiI4d1XGeS0d824zQRVnE5voDp6oAgIlwAcSc4yDZqscG
	rMqYb+E+nRR54wEL9tYeaRMh+iaiiC+ngeH57VuU3sGoy6yHjNGHLnz0xAgI11E=
X-Google-Smtp-Source: AGHT+IHTCZPI5CFIphc/e97qp66O1hKmxtfW9gMoc6vkiYwH3cAhC427f812NDtWw3NCcnSLvnl6IQ==
X-Received: by 2002:ac8:7490:0:b0:42e:7036:9bef with SMTP id v16-20020ac87490000000b0042e70369befmr11259645qtq.38.1709117622975;
        Wed, 28 Feb 2024 02:53:42 -0800 (PST)
Date: Wed, 28 Feb 2024 11:53:40 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Tamas K Lengyel <tamas@tklengyel.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH 1/2] x86/memsharing: use an atomic add instead of a
 cmpxchg loop
Message-ID: <Zd8QtCwp75Phjluc@macbook>
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-2-roger.pau@citrix.com>
 <d8939069-a1cb-4794-a6aa-75a3e4d97884@suse.com>
 <CABfawhkhH7Pakkq+RgjfmW-f60CbOKxr6wWxvBSr2AkN+m1OdQ@mail.gmail.com>
 <2f68276d-0b10-4cec-8636-5456314c194e@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <2f68276d-0b10-4cec-8636-5456314c194e@suse.com>

On Fri, Feb 23, 2024 at 08:43:24AM +0100, Jan Beulich wrote:
> On 22.02.2024 19:03, Tamas K Lengyel wrote:
> > On Thu, Feb 22, 2024 at 5:06 AM Jan Beulich <jbeulich@suse.com> wrote:
> >> On 22.02.2024 10:05, Roger Pau Monne wrote:
> >>> The usage of a cmpxchg loop in get_next_handle() is unnecessary, as the same
> >>> can be achieved with an atomic increment, which is both simpler to read, and
> >>> avoid any need for a loop.
> >>>
> >>> The cmpxchg usage is likely a remnant of 32bit support, which didn't have an
> >>> instruction to do an atomic 64bit add, and instead a cmpxchg had to be used.
> >>>
> >>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >>> Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>
> >>
> >> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> >> albeit ...
> >>
> >>> --- a/xen/arch/x86/mm/mem_sharing.c
> >>> +++ b/xen/arch/x86/mm/mem_sharing.c
> >>> @@ -179,13 +179,7 @@ static void mem_sharing_page_unlock(struct page_info *pg)
> >>>
> >>>  static shr_handle_t get_next_handle(void)
> >>>  {
> >>> -    /* Get the next handle get_page style */
> >>> -    uint64_t x, y = next_handle;
> >>> -    do {
> >>> -        x = y;
> >>> -    }
> >>> -    while ( (y = cmpxchg(&next_handle, x, x + 1)) != x );
> >>> -    return x + 1;
> >>> +    return arch_fetch_and_add(&next_handle, 1) + 1;
> >>>  }
> >>
> >> ... the adding of 1 here is a little odd when taken together with
> >> next_handle's initializer. Tamas, you've not written that code, but do
> >> you have any thoughts towards the possible removal of either the
> >> initializer or the adding here? Plus that variable of course could
> >> very well do with moving into this function.
> > 
> > I have to say I find the existing logic here hard to parse but by the
> > looks I don't think we need the + 1 once we switch to
> > arch_fetch_and_add. Also could go without initializing next_handle to
> > 1. Moving it into the function would not really accomplish anything
> > other than style AFAICT?
> 
> Well, limiting scope of things can be viewed as purely style, but I
> think it's more than that: It makes intentions more clear and reduces
> the chance of abuse (deliberate or unintentional).

I'm afraid that whatever is the outcome here, I will defer it to a
further commit, since the purpose here is to be a non-functional
change.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:06:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:06:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686470.1068555 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHlL-0006Gb-N0; Wed, 28 Feb 2024 11:06:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686470.1068555; Wed, 28 Feb 2024 11:06:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHlL-0006GU-Jz; Wed, 28 Feb 2024 11:06:11 +0000
Received: by outflank-mailman (input) for mailman id 686470;
 Wed, 28 Feb 2024 11:06:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+A+G=KF=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfHlJ-0006GO-UY
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:06:09 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5f600c67-d629-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 12:06:08 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 641BD4EE0737;
 Wed, 28 Feb 2024 12:06:07 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5f600c67-d629-11ee-afd7-a90da7624cb6
MIME-Version: 1.0
Date: Wed, 28 Feb 2024 12:06:07 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>, consulting@bugseng.com, Andrew Cooper
 <andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, Wei
 Liu <wl@xen.org>, xen-devel@lists.xenproject.org, Stefano Stabellini
 <sstabellini@kernel.org>
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
In-Reply-To: <4bee79ca-7a7e-4bcc-ac97-5a5a57ec2c91@suse.com>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
 <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
 <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
 <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com>
 <7a8e610e-913e-4a56-8ce1-6dd6abd894f4@xen.org>
 <4bee79ca-7a7e-4bcc-ac97-5a5a57ec2c91@suse.com>
Message-ID: <61bdcfc51c4d6ad81949f1f4d84593e1@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-27 13:47, Jan Beulich wrote:
> On 27.02.2024 12:52, Julien Grall wrote:
>> Hi Jan,
>> 
>> On 27/02/2024 07:28, Jan Beulich wrote:
>>> On 27.02.2024 01:26, Stefano Stabellini wrote:
>>>> On Mon, 26 Feb 2024, Jan Beulich wrote:
>>>>> On 23.02.2024 10:35, Nicola Vetrini wrote:
>>>>>> Refactor cpu_notifier_call_chain into two functions:
>>>>>> - the variant that is allowed to fail loses the nofail flag
>>>>>> - the variant that shouldn't fail is encapsulated in a call
>>>>>>    to the failing variant, with an additional check.
>>>>>> 
>>>>>> This prevents uses of the function that are not supposed to
>>>>>> fail from ignoring the return value, thus violating Rule 17.7:
>>>>>> "The value returned by a function having non-void return type 
>>>>>> shall
>>>>>> be used".
>>>>>> 
>>>>>> No functional change.
>>>>>> 
>>>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>>>> 
>>>>> I'm afraid I disagree with this kind of bifurcation. No matter what
>>>>> Misra thinks or says, it is normal for return values of functions 
>>>>> to
>>>>> not always be relevant to check.
>>>> 
>>>> Hi Jan, I disagree.
>>>> 
>>>> Regardless of MISRA, I really think return values need to be 
>>>> checked.
>>>> Moreover, we decided as a group to honor MISRA Rule 17.7, which 
>>>> requires
>>>> return values to be checked. This patch is a good step forward.
>>> 
>>> Yet splitting functions isn't the only way to deal with Misra's
>>> requirements, I suppose. After all there are functions where the
>>> return value is purely courtesy for perhaps just one of its callers.
>> 
>> You are right that we have some places where one caller care about the
>> return value. But the problem is how do you tell whether the return 
>> was
>> ignored on purpose or not?
>> 
>> We had at least one XSA because the return value of a function was not
>> checked (see XSA-222). We also had plenty of smaller patches to check
>> returns.
>> 
>> So far, we added __must_check when we believed return values should be
>> checked. But usually at the point we notice, this is far too late.
>> 
>> To me the goal should be that we enforce __must_check everywhere. We 
>> are
>> probably going to detect places where we forgot to check the return. 
>> For
>> thoses that are on purpose, we can document them.
>> 
>>> 
>>> Splitting simply doesn't scale very well, imo.
>> 

Yes. I certainly don't plan to do much splitting either. I saw this one 
as a low-hanging fruit.

>> Do you have another proposal? As Stefano said, we adopted the rule 
>> 17.7.
>> So we know need a solution to address it.
> 
> One possibility that was circulated while discussing was to add (void)
> casts. I'm not a huge fan of those, but between the two options that
> might be the lesser evil. We also use funny (should I say ugly)
> workarounds in a few cases where we have __must_check but still want
> to not really handle the return value in certain cases. Given there are
> example in the code base, extending use of such constructs is certainly
> also something that may want considering.
> 

Can you point out some of these constructs, just to get an idea of what 
that might look like?

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:09:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:09:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686473.1068565 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHoR-0006s9-4C; Wed, 28 Feb 2024 11:09:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686473.1068565; Wed, 28 Feb 2024 11:09:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHoR-0006s2-1R; Wed, 28 Feb 2024 11:09:23 +0000
Received: by outflank-mailman (input) for mailman id 686473;
 Wed, 28 Feb 2024 11:09:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+A+G=KF=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfHoP-0006rw-22
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:09:21 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d1bba512-d629-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 12:09:19 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 733944EE0737;
 Wed, 28 Feb 2024 12:09:19 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d1bba512-d629-11ee-afd7-a90da7624cb6
MIME-Version: 1.0
Date: Wed, 28 Feb 2024 12:09:19 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>, George
 Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
In-Reply-To: <alpine.DEB.2.22.394.2402271808410.575685@ubuntu-linux-20-04-desktop>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
 <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
 <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
 <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com>
 <7a8e610e-913e-4a56-8ce1-6dd6abd894f4@xen.org>
 <4bee79ca-7a7e-4bcc-ac97-5a5a57ec2c91@suse.com>
 <alpine.DEB.2.22.394.2402271808410.575685@ubuntu-linux-20-04-desktop>
Message-ID: <6af04933659178b3ccabc5caf646273c@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-28 03:10, Stefano Stabellini wrote:
> On Tue, 27 Feb 2024, Jan Beulich wrote:
>> On 27.02.2024 12:52, Julien Grall wrote:
>> > Hi Jan,
>> >
>> > On 27/02/2024 07:28, Jan Beulich wrote:
>> >> On 27.02.2024 01:26, Stefano Stabellini wrote:
>> >>> On Mon, 26 Feb 2024, Jan Beulich wrote:
>> >>>> On 23.02.2024 10:35, Nicola Vetrini wrote:
>> >>>>> Refactor cpu_notifier_call_chain into two functions:
>> >>>>> - the variant that is allowed to fail loses the nofail flag
>> >>>>> - the variant that shouldn't fail is encapsulated in a call
>> >>>>>    to the failing variant, with an additional check.
>> >>>>>
>> >>>>> This prevents uses of the function that are not supposed to
>> >>>>> fail from ignoring the return value, thus violating Rule 17.7:
>> >>>>> "The value returned by a function having non-void return type shall
>> >>>>> be used".
>> >>>>>
>> >>>>> No functional change.
>> >>>>>
>> >>>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>> >>>>
>> >>>> I'm afraid I disagree with this kind of bifurcation. No matter what
>> >>>> Misra thinks or says, it is normal for return values of functions to
>> >>>> not always be relevant to check.
>> >>>
>> >>> Hi Jan, I disagree.
>> >>>
>> >>> Regardless of MISRA, I really think return values need to be checked.
>> >>> Moreover, we decided as a group to honor MISRA Rule 17.7, which requires
>> >>> return values to be checked. This patch is a good step forward.
>> >>
>> >> Yet splitting functions isn't the only way to deal with Misra's
>> >> requirements, I suppose. After all there are functions where the
>> >> return value is purely courtesy for perhaps just one of its callers.
>> >
>> > You are right that we have some places where one caller care about the
>> > return value. But the problem is how do you tell whether the return was
>> > ignored on purpose or not?
>> >
>> > We had at least one XSA because the return value of a function was not
>> > checked (see XSA-222). We also had plenty of smaller patches to check
>> > returns.
>> >
>> > So far, we added __must_check when we believed return values should be
>> > checked. But usually at the point we notice, this is far too late.
>> >
>> > To me the goal should be that we enforce __must_check everywhere. We are
>> > probably going to detect places where we forgot to check the return. For
>> > thoses that are on purpose, we can document them.
>> >
>> >>
>> >> Splitting simply doesn't scale very well, imo.
>> >
>> > Do you have another proposal? As Stefano said, we adopted the rule 17.7.
>> > So we know need a solution to address it.
>> 
>> One possibility that was circulated while discussing was to add (void)
>> casts. I'm not a huge fan of those, but between the two options that
>> might be the lesser evil. We also use funny (should I say ugly)
>> workarounds in a few cases where we have __must_check but still want
>> to not really handle the return value in certain cases. Given there 
>> are
>> example in the code base, extending use of such constructs is 
>> certainly
>> also something that may want considering.
> 
> I asked Roberto if void casts are an option for compliance.
> 

void casts are an option for sure. The rationale for the rule explicitly 
lists them as a compliance mechanism. An interesting aspect is what 
would be the consensus around void casts on functions whose return value 
is always ignored vs. functions whose return value is sometimes ignored.

> In any case, I don't think we should use void casts in the specific
> cases this patch is dealing with. Void casts (if anything) should be a
> last resort while this patch fixes the issue in a better way.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:18:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:18:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686477.1068575 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHxL-00006X-VB; Wed, 28 Feb 2024 11:18:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686477.1068575; Wed, 28 Feb 2024 11:18:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfHxL-00006Q-SB; Wed, 28 Feb 2024 11:18:35 +0000
Received: by outflank-mailman (input) for mailman id 686477;
 Wed, 28 Feb 2024 11:18:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfHxL-00006K-6y
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:18:35 +0000
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com
 [2a00:1450:4864:20::134])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1bf9f421-d62b-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 12:18:33 +0100 (CET)
Received: by mail-lf1-x134.google.com with SMTP id
 2adb3069b0e04-512ab55fde6so7208488e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 03:18:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 u24-20020a170906409800b00a3cfd838f32sm1762879ejj.178.2024.02.28.03.18.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 03:18:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1bf9f421-d62b-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709119113; x=1709723913; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4uh43WYyos/QNMgImDchNX+Q4uLC6PkPBT72EL0leEI=;
        b=anX5e6hxZhLquR1PN3R0xDI9z/JuNNrIdLtBsVmmL5iGjbXi8XrAlKZ8zjo+BbFa4Q
         AX4b7dft4eDXEXcyTPOSAkyfxyTNf8acrM3z5SWxF9FXrQaDEK41GtK6YJQT/FFQmXRK
         E3sIUOXVgI50OXytkju7K7yy4aPzIu8E+GenOZTcwQDjyg071o3mwQ4SiqzCguumL5g5
         GCzSNgUSi3eHXSEk7mE+BqbJHsgdBDS0T4zOiJ9Zy3Ia4Bb3PJS/a5b/FZgw4xXwRH0c
         dN3ZQuO9vK6f6mkvvkDgobMGMiWszgmQ//P584LIUi99z1qZzOYJXwdMbORI7TGYO76b
         deag==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709119113; x=1709723913;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=4uh43WYyos/QNMgImDchNX+Q4uLC6PkPBT72EL0leEI=;
        b=kA5KV6P5aTJvf8K9W2MNVOjPZrjmREYupt6Nx6LkJhvEg2MBEeE4byNoRbDbyJo6K+
         dVgKnaeWHZZNgg3UqGpvZE9K+PYeQHQdU4SDYOxt2OdR14F2SdHKA8wXTJKXWeyqU/gP
         bbdpzLFeOpR7BpA7rfPlbdNn7Nv0PX/Dr2FvNPHFmIaJhfQCsLbjj8rxDWqtuF3beqOv
         XS2kUy0x0yBoqwoi9RbfGAkE/ApIxSC33tZ7iC1YJMHGLReqTiqDgiTumHkokH2fuhKt
         SvrdtvPmfi2E2Llrobw7SflMdzVNwUMfMMS78eYXdc0fFDoLldef3BfOKiGpjgZNVcqa
         +yTQ==
X-Forwarded-Encrypted: i=1; AJvYcCXiNMk2uFSiR8sQ0teOo/WCLO7ccUPwzmkJSIwN1kXgWwGJpIG0RUaSq+5VBYYd0mVg0EI0UJ4sqxb8kM5HjIsGkRQ6x4P+Qs9H2UVQ2OI=
X-Gm-Message-State: AOJu0YwHBrsrGdR5qwuXut5aBLBAdKMgY7en/lq7Ess0MTKCmOT67G1c
	QcGuViF3o8Z7Q+PRapBgnArx5o63MCo9V6UMvUnbBMyvUPt3hWwonw3/AbaKVg==
X-Google-Smtp-Source: AGHT+IFDwO14G023nwh5yMKiO/3bGVUn4uxx31tIhARZJWTjAVgomiIy+Zh0n6sb5dOWFYMUxbLjYw==
X-Received: by 2002:ac2:5309:0:b0:513:17fb:24a1 with SMTP id c9-20020ac25309000000b0051317fb24a1mr1370119lfh.64.1709119113162;
        Wed, 28 Feb 2024 03:18:33 -0800 (PST)
Message-ID: <de7c8de8-2373-4d78-8aec-c633bb253660@suse.com>
Date: Wed, 28 Feb 2024 12:18:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] x86/memsharing: use an atomic add instead of a
 cmpxchg loop
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Tamas K Lengyel <tamas@tklengyel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-2-roger.pau@citrix.com>
 <d8939069-a1cb-4794-a6aa-75a3e4d97884@suse.com>
 <CABfawhkhH7Pakkq+RgjfmW-f60CbOKxr6wWxvBSr2AkN+m1OdQ@mail.gmail.com>
 <2f68276d-0b10-4cec-8636-5456314c194e@suse.com> <Zd8QtCwp75Phjluc@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zd8QtCwp75Phjluc@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28.02.2024 11:53, Roger Pau Monné wrote:
> On Fri, Feb 23, 2024 at 08:43:24AM +0100, Jan Beulich wrote:
>> On 22.02.2024 19:03, Tamas K Lengyel wrote:
>>> On Thu, Feb 22, 2024 at 5:06 AM Jan Beulich <jbeulich@suse.com> wrote:
>>>> On 22.02.2024 10:05, Roger Pau Monne wrote:
>>>>> The usage of a cmpxchg loop in get_next_handle() is unnecessary, as the same
>>>>> can be achieved with an atomic increment, which is both simpler to read, and
>>>>> avoid any need for a loop.
>>>>>
>>>>> The cmpxchg usage is likely a remnant of 32bit support, which didn't have an
>>>>> instruction to do an atomic 64bit add, and instead a cmpxchg had to be used.
>>>>>
>>>>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>>> Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>
>>>>
>>>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>>> albeit ...
>>>>
>>>>> --- a/xen/arch/x86/mm/mem_sharing.c
>>>>> +++ b/xen/arch/x86/mm/mem_sharing.c
>>>>> @@ -179,13 +179,7 @@ static void mem_sharing_page_unlock(struct page_info *pg)
>>>>>
>>>>>  static shr_handle_t get_next_handle(void)
>>>>>  {
>>>>> -    /* Get the next handle get_page style */
>>>>> -    uint64_t x, y = next_handle;
>>>>> -    do {
>>>>> -        x = y;
>>>>> -    }
>>>>> -    while ( (y = cmpxchg(&next_handle, x, x + 1)) != x );
>>>>> -    return x + 1;
>>>>> +    return arch_fetch_and_add(&next_handle, 1) + 1;
>>>>>  }
>>>>
>>>> ... the adding of 1 here is a little odd when taken together with
>>>> next_handle's initializer. Tamas, you've not written that code, but do
>>>> you have any thoughts towards the possible removal of either the
>>>> initializer or the adding here? Plus that variable of course could
>>>> very well do with moving into this function.
>>>
>>> I have to say I find the existing logic here hard to parse but by the
>>> looks I don't think we need the + 1 once we switch to
>>> arch_fetch_and_add. Also could go without initializing next_handle to
>>> 1. Moving it into the function would not really accomplish anything
>>> other than style AFAICT?
>>
>> Well, limiting scope of things can be viewed as purely style, but I
>> think it's more than that: It makes intentions more clear and reduces
>> the chance of abuse (deliberate or unintentional).
> 
> I'm afraid that whatever is the outcome here, I will defer it to a
> further commit, since the purpose here is to be a non-functional
> change.

That's fine with me, but an ack from Tamas is still pending, unless I
missed something somewhere.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:22:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:22:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686481.1068585 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfI14-0001pB-Go; Wed, 28 Feb 2024 11:22:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686481.1068585; Wed, 28 Feb 2024 11:22:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfI14-0001p4-Do; Wed, 28 Feb 2024 11:22:26 +0000
Received: by outflank-mailman (input) for mailman id 686481;
 Wed, 28 Feb 2024 11:22:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfI13-0001oy-2Z
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:22:25 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a4a9c252-d62b-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 12:22:23 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a28a6cef709so802009866b.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 03:22:23 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 qh16-20020a170906ecb000b00a432777eb77sm1741836ejb.60.2024.02.28.03.22.22
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 03:22:22 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a4a9c252-d62b-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709119342; x=1709724142; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=pzRSwln9kYqqayC627UbGURKm6ipG9CKJ/VFEG1KIPs=;
        b=bDPO6lm/TxcU1uXQI/O1kcN1HfVkG8ItVCt2PtuKnDiaC8riwEaekrasVKjcKAhWNp
         OS5apRcPfNFFBglldbfsQqyf0Wtfwf3zL81eL/BM/kn8Kt9gRhF1VL/Fg7QMrJjpTQCJ
         k7wRlFClmO8bj88mn6LPI438cBWnoSRDLiUmV1cfmAtVW6xDyve84dhuSj36JyPi0LBo
         alLcGoQIQswARt1BkNhEAD4IOwkUwQKzAGz5rx+bVBJ7hZ+KupnQvQqF1VH54HKjn/c8
         hAw0Z1EAMCXKYilG/B6dSuwup9Nz5/Vp8+b7Gq+UJC0lmznPteiXFckANdFehqO/AARV
         hhSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709119342; x=1709724142;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=pzRSwln9kYqqayC627UbGURKm6ipG9CKJ/VFEG1KIPs=;
        b=oNnT12wN6HWNL1eFfCDjB/Ajv4fV2ILIu/rUgAVqOt0QDb1uktZ47yM2KIgWwZ4w9V
         JvDUWUQi2mpROOimjfqDpNB0g54v6moJC3NyYNRtq5zC2RE0r3dxnPsGIgykPBLmn1Dw
         rwb+5uSeEZ17CG+KbHvQH8teg7tYk+/N/4x0j7djod8WZ3REeeMZS6AwqQVXPRKivJg5
         j5vri7pzJNNqsr2GcgFdO34BRXe7vAuGlcSwtthbzqeQc4JBNpwvTkmJidikvnGiPmz+
         2Gr3md7FyUYt5e1689+EvGzw9jONQXa/3+ywc7TzgsplEdVGG7oufe7XBUawfI3nGbsd
         2zRQ==
X-Forwarded-Encrypted: i=1; AJvYcCXOTvGEsewcJaWMXxiaDIjBvZXC+1Z6wPLXCjhbNLE08maQTuTjoyWhfu1mU8cmG9GfTQCwC7fldapfs2RJOW3Y3SCbx/m0Ah133HPhYJc=
X-Gm-Message-State: AOJu0YxJsCkQdHLybp+qhwGeLhFFXeYDk5b6wIHPaw8PF5QV7tFXba2x
	5BEHN6H29Y0Bc1Fe9+4ntXlyXzkQ4TjhQK/YAUWJoS957bXyYqVOZ7df4b8KoA==
X-Google-Smtp-Source: AGHT+IG10w+skdRKBQ2QKEtAWWQCWmBWo23p9FB0pkd1PJo6V8FxSZFSE757BUrQ/y7+61VFkEATaw==
X-Received: by 2002:a17:907:119b:b0:a38:916e:a4fe with SMTP id uz27-20020a170907119b00b00a38916ea4femr8551978ejb.39.1709119342655;
        Wed, 28 Feb 2024 03:22:22 -0800 (PST)
Message-ID: <52f536db-782d-4805-906a-2934b289b997@suse.com>
Date: Wed, 28 Feb 2024 12:22:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: Julien Grall <julien@xen.org>, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Stefano Stabellini <sstabellini@kernel.org>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
 <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
 <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
 <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com>
 <7a8e610e-913e-4a56-8ce1-6dd6abd894f4@xen.org>
 <4bee79ca-7a7e-4bcc-ac97-5a5a57ec2c91@suse.com>
 <61bdcfc51c4d6ad81949f1f4d84593e1@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <61bdcfc51c4d6ad81949f1f4d84593e1@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 28.02.2024 12:06, Nicola Vetrini wrote:
> On 2024-02-27 13:47, Jan Beulich wrote:
>> On 27.02.2024 12:52, Julien Grall wrote:
>>> Do you have another proposal? As Stefano said, we adopted the rule 
>>> 17.7.
>>> So we know need a solution to address it.
>>
>> One possibility that was circulated while discussing was to add (void)
>> casts. I'm not a huge fan of those, but between the two options that
>> might be the lesser evil. We also use funny (should I say ugly)
>> workarounds in a few cases where we have __must_check but still want
>> to not really handle the return value in certain cases. Given there are
>> example in the code base, extending use of such constructs is certainly
>> also something that may want considering.
> 
> Can you point out some of these constructs, just to get an idea of what 
> that might look like?

Grep for __must_check used in comments.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:33:02 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:33:02 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686486.1068595 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIB4-0003dl-FH; Wed, 28 Feb 2024 11:32:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686486.1068595; Wed, 28 Feb 2024 11:32:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIB4-0003de-B5; Wed, 28 Feb 2024 11:32:46 +0000
Received: by outflank-mailman (input) for mailman id 686486;
 Wed, 28 Feb 2024 11:32:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eIhW=KF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfIB3-0003dY-6K
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:32:45 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 16d48837-d62d-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 12:32:44 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a441d7c6125so10554266b.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 03:32:44 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 vi3-20020a170907d40300b00a43eb697337sm734611ejc.49.2024.02.28.03.32.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 03:32:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 16d48837-d62d-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709119963; x=1709724763; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OGUX/xuM0orMXlAXQ0whhEheAcbUc5oBmQHJYNDcRUk=;
        b=SCuqL9SDn8R89e9U2g8YgiJ+gQzuT2t6xbR6GOwLlzEmRUVktuTlHQNB47juW1QdWg
         VcCLO9egz0I1Oh6OP9JTk9nmRyOqIq9WCs0QDouK4GzH+1EOS3cQw+i1TvyfrorTUgMK
         TbkFrIbe03QX1bv7IbTLcik5XFtsYirmvwY6Y=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709119963; x=1709724763;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OGUX/xuM0orMXlAXQ0whhEheAcbUc5oBmQHJYNDcRUk=;
        b=lb+BWiRaBi/Xi9rVGono17hjBG595jfmryiCeYWJwxWRPzr58kiL/c/Ge5SN4WVkN9
         XGYrswNN5jwXwEOGDCGVHuVIflBiDaj/m4tWIco/zL0DMirojMbuhwGB/8TurSyy2K3S
         Emb+frU3DkLWPEERcmS/N3xu5AnWg6ku94Y+KgbHEf4QaLxnz6aY97fbSZrcMZN7z4cc
         SUwAcdy9EnrzPrVh5WtaWtKiyYaomQiuiFKQpHRPxvq/1lIfZS07wDTbrgKmQLSaWXSX
         ADBOssqbDYdHPZ12gJMk6Cw6vu3RljTKyTVgDxSmBHA8dc++ycQsuZP0wYqOW+USY3jf
         Uf6w==
X-Forwarded-Encrypted: i=1; AJvYcCU1Vl1e/zxLIr0gk2EKA0FbWIHAgzvPx8xnm2uDxY6BND58qEqgA1K7JqYexgH06Mc8W0LZ+k/32FP8bhjQWHo/C583+YeNUx67H81PJFc=
X-Gm-Message-State: AOJu0YxfMs60F3rRdUPElU0mBKoUNxBZz7Qvrz+/dRMivPQRnSZ69aog
	lchYzxxLhrFHERIRswjLz3t9ILRgqGJeqq1/n9iOcmXjqBMwdOldaV94ILvs7uU=
X-Google-Smtp-Source: AGHT+IEuGfdUX8Zde3l7c4seSihkqtpl/WESA5vn+N1Ku1e/AdlRDCBCtbTQH4KbLc9tVwo13X+Yyg==
X-Received: by 2002:a17:906:26ca:b0:a3f:c345:f295 with SMTP id u10-20020a17090626ca00b00a3fc345f295mr9170838ejc.23.1709119963402;
        Wed, 28 Feb 2024 03:32:43 -0800 (PST)
Message-ID: <22ab5021-7113-40de-a4ce-e2f7113df618@citrix.com>
Date: Wed, 28 Feb 2024 11:32:41 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86: Resync intel-family.h from Linux
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240227160940.2054111-1-andrew.cooper3@citrix.com>
 <654dc3e7-b7c3-439a-b784-38834989bd43@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <654dc3e7-b7c3-439a-b784-38834989bd43@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/02/2024 9:10 am, Jan Beulich wrote:
> On 27.02.2024 17:09, Andrew Cooper wrote:
>> From v6.8-rc8
> Typo or time leap? There wasn't even rc7 yet ...

Oops yes.  A typo.

d206a76d7d27 - (tag: v6.8-rc6) Linux 6.8-rc6 (3 days ago) <Linus Torvalds>

>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:36:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:36:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686488.1068604 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIEK-0004DO-Rr; Wed, 28 Feb 2024 11:36:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686488.1068604; Wed, 28 Feb 2024 11:36:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIEK-0004DH-PB; Wed, 28 Feb 2024 11:36:08 +0000
Received: by outflank-mailman (input) for mailman id 686488;
 Wed, 28 Feb 2024 11:36:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfIEJ-0004DB-6y
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:36:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfIEF-0000rW-Qh; Wed, 28 Feb 2024 11:36:03 +0000
Received: from [15.248.2.225] (helo=[10.45.19.69])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfIEF-0006pc-2X; Wed, 28 Feb 2024 11:36:03 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=CJg+MwqUmZhwkaRKDSIwjIPui01t1mOwbNVFxzVp/Ao=; b=KzWXJDAUcX/8jwgkiFAoVGRcBI
	LFKPNPvjP01jceFIPRFlNX2mVJMbQH9wCQ0YAddDvGJmfzLP4cByDg3gY7mkJI1n7fnv3hK9uc4FY
	ewy7o358Q1YleZq3EkhcKt1gZG7Sr2wxNMp0FQxU+RldPFMbMUwlWck+QBu1PihwAM1M=;
Message-ID: <bcae31fb-856d-4737-b780-f41b3e24cfa7@xen.org>
Date: Wed, 28 Feb 2024 11:36:01 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
Content-Language: en-GB
To: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: Jan Beulich <jbeulich@suse.com>, consulting@bugseng.com,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
 <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
 <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
 <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com>
 <7a8e610e-913e-4a56-8ce1-6dd6abd894f4@xen.org>
 <4bee79ca-7a7e-4bcc-ac97-5a5a57ec2c91@suse.com>
 <alpine.DEB.2.22.394.2402271808410.575685@ubuntu-linux-20-04-desktop>
 <6af04933659178b3ccabc5caf646273c@bugseng.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <6af04933659178b3ccabc5caf646273c@bugseng.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Nicola,

On 28/02/2024 11:09, Nicola Vetrini wrote:
>> I asked Roberto if void casts are an option for compliance.
>>
> 
> void casts are an option for sure. The rationale for the rule explicitly 
> lists them as a compliance mechanism. An interesting aspect is what 
> would be the consensus around void casts on functions whose return value 
> is always ignored vs. functions whose return value is sometimes ignored.

If a return is always ignored, then the function should return void. For 
the second case, I think it will be on the case by case basis.

> 
>> In any case, I don't think we should use void casts in the specific
>> cases this patch is dealing with. Void casts (if anything) should be a
>> last resort while this patch fixes the issue in a better way.

+1.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:40:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:40:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686490.1068614 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIIV-00067c-Aq; Wed, 28 Feb 2024 11:40:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686490.1068614; Wed, 28 Feb 2024 11:40:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIIV-00067V-8G; Wed, 28 Feb 2024 11:40:27 +0000
Received: by outflank-mailman (input) for mailman id 686490;
 Wed, 28 Feb 2024 11:40:26 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=29qp=KF=cloud.com=fouad.hilly@srs-se1.protection.inumbo.net>)
 id 1rfIIU-00067M-07
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:40:26 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 276c0b9b-d62e-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 12:40:21 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-56648955ac5so1762943a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 03:40:21 -0800 (PST)
Received: from fhilly.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 f8-20020a170906c08800b00a434b5fcab6sm1763161ejz.221.2024.02.28.03.40.19
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 03:40:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 276c0b9b-d62e-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709120420; x=1709725220; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=nNLtdVIFvfiAKfPmHNGfLLU4fprtE7gWwzTpMhmZat4=;
        b=UMJjXx73Tqso9CvH4hqbo+Ks+oHlgTT5ZlqjVOSxcq101nYlHjodsZ1Ty1CxwxLgOR
         NsQp8sdYY8dYV/Unrx90ofkfk55FsXBkPSojALDqTvVUadETBAVvpHiQzQotVkxLQOy2
         HvFnHltXETuaryEfhyrqfnSdkw58CPgr5ZfGk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709120420; x=1709725220;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=nNLtdVIFvfiAKfPmHNGfLLU4fprtE7gWwzTpMhmZat4=;
        b=Mrx/bW/A8WRlt2VEDN7VEfPKrFeF7SAQCXNVtFcuvpFOhr36agfuxOKRWpueJQmeo3
         q9nRkFyigwMD8rczIdcyDKQdrDY6S4cTpISed5wk0Rltei65csVKv8y+ab23M429ehhi
         Go5EIvhNX0L90rzowYfcaCUWGJYqKvNsDvoVg3PwJ4P3bdxaKYX9642L9uEwY8CVddf4
         CJ4oQ0J7WTDG2YVZID4vcRWb7VQ0+nZDcLwLfdEIFXia9FiWqYxQLJsM8WefJP9SD8wr
         5gRQ3NICIKkSMCJs0VTg7SD4+sruMCPg84vtBDzTRQN0rUO7R6+vY5kOwy5syZ/kTnIH
         P0hA==
X-Gm-Message-State: AOJu0YyZGNj9iQvQIa6dnXsX/8Fs1OTzhVUCcwfWNzZ6vNQrzlXKwiXv
	W6If9t9oc/qh8ntJJoh1+tn3A1QYvg5gtTzYykfkdTzo22mguuniThDOTtl7LFInHd4KZdqS/Q2
	IzD0=
X-Google-Smtp-Source: AGHT+IHN0TNxsygBBurhKaAfUrRwLbCYzfsps2JeFiXVrS9eF2ZkY6PYRUlAdCJZpyDtcQqsk4DTww==
X-Received: by 2002:a17:906:e52:b0:a43:d853:98f1 with SMTP id q18-20020a1709060e5200b00a43d85398f1mr2214401eji.17.1709120420171;
        Wed, 28 Feb 2024 03:40:20 -0800 (PST)
From: Fouad Hilly <fouad.hilly@cloud.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Fouad Hilly <fouad.hilly@cloud.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Juergen Gross <jgross@suse.com>
Subject: [PATCH v3] tools/xentop: Add VBD3 support to xentop
Date: Wed, 28 Feb 2024 11:39:47 +0000
Message-ID: <20240228113947.57053-1-fouad.hilly@cloud.com>
X-Mailer: git-send-email 2.42.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Pritha Srivastava <pritha.srivastava@citrix.com>

xl now knows how to drive tapdisk, so modify libxenstat to
understand vbd3 statistics.

Signed-off-by: Pritha Srivastava <pritha.srivastava@citrix.com>
Signed-off-by: Jorge Martin <jorge.martin@citrix.com>
Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
---
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
CC: Juergen Gross <jgross@suse.com>

v2:
- Fix order of SoB
- Fix Syntax
- Re-order free(path)

v3:
- Add const to char *vbd3_path
- Replace VBD3 type magic number with define
- Remove setting vbd.back_type
- Add comment with link to external interface
---
 tools/libs/stat/xenstat_linux.c | 65 ++++++++++++++++++++++++++++++++-
 tools/libs/stat/xenstat_priv.h  | 17 +++++++++
 tools/xentop/xentop.c           |  1 +
 3 files changed, 82 insertions(+), 1 deletion(-)

diff --git a/tools/libs/stat/xenstat_linux.c b/tools/libs/stat/xenstat_linux.c
index cbba54aa83ee..b60e3fdcab3e 100644
--- a/tools/libs/stat/xenstat_linux.c
+++ b/tools/libs/stat/xenstat_linux.c
@@ -35,6 +35,7 @@
 #include "xenstat_priv.h"
 
 #define SYSFS_VBD_PATH "/sys/bus/xen-backend/devices"
+#define XENSTAT_VBD_TYPE_VBD3 3
 
 struct priv_data {
 	FILE *procnetdev;
@@ -390,6 +391,38 @@ void xenstat_uninit_networks(xenstat_handle * handle)
 		fclose(priv->procnetdev);
 }
 
+static int read_attributes_vbd3(const char *vbd3_path, xenstat_vbd *vbd)
+{
+	FILE *fp;
+	struct vbd3_stats vbd3_stats;
+
+	fp = fopen(vbd3_path, "rb");
+
+	if (fp == NULL) {
+		return -1;
+	}
+
+	if (fread(&vbd3_stats, sizeof(struct vbd3_stats), 1, fp) != 1) {
+		fclose(fp);
+		return -1;
+	}
+
+	if (vbd3_stats.version != 1) {
+		fclose(fp);
+		return -1;
+	}
+
+	vbd->oo_reqs = vbd3_stats.oo_reqs;
+	vbd->rd_reqs = vbd3_stats.read_reqs_submitted;
+	vbd->rd_sects = vbd3_stats.read_sectors;
+	vbd->wr_reqs = vbd3_stats.write_reqs_submitted;
+	vbd->wr_sects = vbd3_stats.write_sectors;
+
+	fclose(fp);
+
+	return 0;
+}
+
 static int read_attributes_vbd(const char *vbd_directory, const char *what, char *ret, int cap)
 {
 	static char file_name[80];
@@ -438,7 +471,7 @@ int xenstat_collect_vbds(xenstat_node * node)
 		int ret;
 		char buf[256];
 
-		ret = sscanf(dp->d_name, "%3s-%u-%u", buf, &domid, &vbd.dev);
+		ret = sscanf(dp->d_name, "%255[^-]-%u-%u", buf, &domid, &vbd.dev);
 		if (ret != 3)
 			continue;
 		if (!(strstr(buf, "vbd")) && !(strstr(buf, "tap")))
@@ -448,6 +481,8 @@ int xenstat_collect_vbds(xenstat_node * node)
 			vbd.back_type = 1;
 		else if (strcmp(buf,"tap") == 0)
 			vbd.back_type = 2;
+		else if (strcmp(buf,"vbd3") == 0)
+			vbd.back_type = XENSTAT_VBD_TYPE_VBD3;
 		else
 			vbd.back_type = 0;
 
@@ -479,6 +514,34 @@ int xenstat_collect_vbds(xenstat_node * node)
 				vbd.error = 1;
 			}
 		}
+		else if (vbd.back_type == XENSTAT_VBD_TYPE_VBD3)
+		{
+			char *td3_pid;
+			char *path;
+
+			vbd.error = 0;
+
+			if (asprintf(&path, "/local/domain/0/backend/vbd3/%u/%u/kthread-pid", domid, vbd.dev) < 0)
+				continue;
+
+			td3_pid = xs_read(node->handle->xshandle, XBT_NULL, path, NULL);
+
+			free(path);
+
+			if (td3_pid == NULL)
+				continue;
+
+			if (asprintf(&path, "/dev/shm/td3-%s/vbd-%u-%u", td3_pid, domid, vbd.dev) < 0) {
+				free(td3_pid);
+				continue;
+			}
+
+			if (read_attributes_vbd3(path, &vbd) < 0)
+				vbd.error = 1;
+
+			free(td3_pid);
+			free(path);
+		}
 		else
 		{
 			vbd.error = 1;
diff --git a/tools/libs/stat/xenstat_priv.h b/tools/libs/stat/xenstat_priv.h
index 4eb44a8ebb84..51ff6862cf49 100644
--- a/tools/libs/stat/xenstat_priv.h
+++ b/tools/libs/stat/xenstat_priv.h
@@ -98,6 +98,23 @@ struct xenstat_vbd {
 	unsigned long long wr_sects;
 };
 
+/* Interface provided by https://github.com/xapi-project/blktap */
+struct vbd3_stats {
+	uint32_t version;
+	uint32_t __pad;
+	uint64_t oo_reqs;
+	uint64_t read_reqs_submitted;
+	uint64_t read_reqs_completed;
+	uint64_t read_sectors;
+	uint64_t read_total_ticks;
+	uint64_t write_reqs_submitted;
+	uint64_t write_reqs_completed;
+	uint64_t write_sectors;
+	uint64_t write_total_ticks;
+	uint64_t io_errors;
+	uint64_t flags;
+};
+
 extern int xenstat_collect_networks(xenstat_node * node);
 extern void xenstat_uninit_networks(xenstat_handle * handle);
 extern int xenstat_collect_vbds(xenstat_node * node);
diff --git a/tools/xentop/xentop.c b/tools/xentop/xentop.c
index 0a2fab7f15a3..f5a456fd4dfd 100644
--- a/tools/xentop/xentop.c
+++ b/tools/xentop/xentop.c
@@ -1124,6 +1124,7 @@ void do_vbd(xenstat_domain *domain)
 		"Unidentified",           /* number 0 */
 		"BlkBack",           /* number 1 */
 		"BlkTap",            /* number 2 */
+		"Tapdisk3"           /* number 3 */
 	};
 
 	num_vbds = xenstat_domain_num_vbds(domain);
-- 
2.42.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:42:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:42:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686494.1068625 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIKm-0006e9-MW; Wed, 28 Feb 2024 11:42:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686494.1068625; Wed, 28 Feb 2024 11:42:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIKm-0006e2-Jz; Wed, 28 Feb 2024 11:42:48 +0000
Received: by outflank-mailman (input) for mailman id 686494;
 Wed, 28 Feb 2024 11:42:47 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfIKl-0006dw-U0
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:42:47 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfIKl-0000zH-Ib; Wed, 28 Feb 2024 11:42:47 +0000
Received: from [15.248.2.225] (helo=[10.45.19.69])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfIKl-0007AV-BW; Wed, 28 Feb 2024 11:42:47 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=6i4AXTXAi3TiGxl1nXtnGb2DoF+BXykICIRa/wpPzR4=; b=CFsTonCwnZree/CPNiAkRH6HEc
	fPKwKYDJWniWq7XkBF+pGv6roShZJHsZ9MbQ+r2mVIcMwAaGCatuvMqmDCaixasjSr41Hx1Bv+RK3
	kk6PaEXhqNYORU/E4+iW4I226JOCGZZ26zbSp02QhG506CE/ausetR2Fw8K7sxUqn/EU=;
Message-ID: <9201bdc5-2e4c-48b7-8c92-f5063fde90b6@xen.org>
Date: Wed, 28 Feb 2024 11:42:45 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Fix arm32 build failure when early printk is
 enabled
Content-Language: en-GB
To: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240228103555.172101-1-michal.orzel@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240228103555.172101-1-michal.orzel@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Michal,

On 28/02/2024 10:35, Michal Orzel wrote:
> Commit 0441c3acc7e9 forgot to rename FIXMAP_CONSOLE to FIX_CONSOLE in
> TEMPORARY_EARLY_UART_VIRTUAL_ADDRESS macro. This results in a build
> failure on arm32, when early printk is enabled:
> arch/arm/arm32/mmu/head.S:311: Error: invalid operands (*UND* and *ABS* sections) for `*'

Good catch! Somewhat related I wonder whether we should add earlyprintk 
testing in gitlab?

> 
> Fixes: 0441c3acc7e9 ("xen/arm: fixmap: Rename the fixmap slots to follow the x86 convention")
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:45:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:45:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686497.1068635 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIMx-0007Gi-66; Wed, 28 Feb 2024 11:45:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686497.1068635; Wed, 28 Feb 2024 11:45:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIMx-0007Gb-2m; Wed, 28 Feb 2024 11:45:03 +0000
Received: by outflank-mailman (input) for mailman id 686497;
 Wed, 28 Feb 2024 11:45:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eIhW=KF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfIMw-0007GQ-DA
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:45:02 +0000
Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com
 [2607:f8b0:4864:20::730])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cdffc24d-d62e-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 12:45:01 +0100 (CET)
Received: by mail-qk1-x730.google.com with SMTP id
 af79cd13be357-787bb0d85eeso336684685a.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 03:45:01 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 j12-20020a37c24c000000b007872b549a89sm4499431qkm.41.2024.02.28.03.44.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 03:44:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cdffc24d-d62e-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709120700; x=1709725500; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=T2AITKqOVUg1v9b8sdlxfW9BrUJXH6ZaTM0gzMJSIFY=;
        b=fOEEAsu3gkHsNG7ALuyihsIiq5WHLxEap3xg2N2RhG+WycokRrwpcTvnigh6fREsAY
         h3PO85VZucH8rXGrLiVpiZCh6ly/f2r0RSLY61pTTYAd84dP++yG1MSOQQuHfbtD79Eq
         oq/qW2q18xloLS2f5ZGJaO235P8ASOfv08nAY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709120700; x=1709725500;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=T2AITKqOVUg1v9b8sdlxfW9BrUJXH6ZaTM0gzMJSIFY=;
        b=FYLu1q7zEIwMvfvTiGWQg+2aZmUq+Olg52uunpIYRMJ/IGMYGlUh8G3L93r0kmP1st
         TlTIhIuhYrrAiyz96cE2mCuY+8k7kt0uHu1nTh046vKF+SL38kcr4aVaGCuYWtOFmVAV
         hcGRcNhm54OxD5CzP+9JohSOzmZ3JtAzJ1eAZoCAaKRT+e/gTJoQbsn3YR463+5VA3mJ
         dsh6lP1suaLsrFL5n+6NZXr+7jjrDo09eGAxhPYP+Bhkm26VXiVw8wQMMEqnuei86lOT
         JpM49ANUfmqV1AG0IQ5ljQgBThfUc09Yq7iGZwVEwQPys98hegLYVR5uwUOfq5x0yinG
         KqGg==
X-Forwarded-Encrypted: i=1; AJvYcCXm4j7omhN0gaf6H6FktepI6K4c21jmqggLIrLqtGojNt5YYJR0lA0iTN55GDjPtTbymzYd7yKO3mD1OHPYaTDMYo58BmZH9v9Ty6bHH7k=
X-Gm-Message-State: AOJu0Yx5kqbN6eeRLZMQdCZyV0qILEqN7GSxONEv8k2OEt7PIywYIbGe
	eoywv67qINCHXlBpazK2JWofcoxjYG5Eh2Uyq5nzj+cwFl+TD0xeQuqGoU7rd/k=
X-Google-Smtp-Source: AGHT+IH8epyiw6KAhOoQIKwNU9v/4Vp0yj8aCewbC6fKssBADbv3N7892yX7ldqlHs8onHgVibFDIg==
X-Received: by 2002:a05:620a:ec4:b0:787:c4bd:2460 with SMTP id x4-20020a05620a0ec400b00787c4bd2460mr4511405qkm.36.1709120700324;
        Wed, 28 Feb 2024 03:45:00 -0800 (PST)
Message-ID: <48b48de3-b0e6-4294-91c5-cade0a72c3d1@citrix.com>
Date: Wed, 28 Feb 2024 11:44:56 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: preparations for 4.18.1
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Anthony Perard <anthony.perard@citrix.com>, Julien Grall <julien@xen.org>,
 Kelly Choi <kelly.choi@cloud.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Christian Lindig <christian.lindig@citrix.com>, David Scott <dave@recoil.org>
References: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com>
 <ca4fe67a-3efd-4ce5-b37d-e68b1bf4c1a7@citrix.com>
 <3e131a4d-652a-4759-bf8a-2564b6c3859e@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <3e131a4d-652a-4759-bf8a-2564b6c3859e@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/02/2024 8:27 am, Jan Beulich wrote:
> On 27.02.2024 15:57, Andrew Cooper wrote:
>> Looking at the XenServer patchqueue, a couple to consider but nothing
>> jumps out as critically urgent.
>>
>> b6cf604207fd and 098d868e52ac as oxenstored perf fixes, although there's
>> one incremental (non-functional) fix I'm still waiting on an ack on. 
>> The backport to 4.18 is trivial.  The backport to 4.17 is miserable but
>> I've already done it.
> I wouldn't normally consider such for backport, but I'm not opposed if the
> oxenstore maintainers agree (now Cc-ed) and if I can leave putting in these
> two backports to you.

I'm not overly fussed.  On our side, it came in as a bug, but we also
have scalability limits specified where upstream Xen doesn't.

>> cf7fe8b72dea which is the CPUID rescan on microcode load.
> This is already there on the 4.18 branch, and it isn't applicable to the
> upstream 4.17 one (I assume you did backport the commit referenced in the
> Fixes: tag to your XenServer tree).

Ah yes.  The rescan raw patch has been in our patchqueue since 2018, and
I keep on forgetting how poor we are at getting that work complete.

Sorry for the noise.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:50:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:50:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686500.1068645 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfISC-0000mp-Nu; Wed, 28 Feb 2024 11:50:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686500.1068645; Wed, 28 Feb 2024 11:50:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfISC-0000mi-LI; Wed, 28 Feb 2024 11:50:28 +0000
Received: by outflank-mailman (input) for mailman id 686500;
 Wed, 28 Feb 2024 11:50:27 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfISB-0000mc-4Z
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:50:27 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfISA-00018t-Bv; Wed, 28 Feb 2024 11:50:26 +0000
Received: from [15.248.2.225] (helo=[10.45.19.69])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfISA-0007Xt-44; Wed, 28 Feb 2024 11:50:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=UBDafR9o0aAJTyKn0caO3T1elLEyJKlLU4oAOuabf/8=; b=n72+OB1KOyQws+4P19dwoCY31/
	r4ZYUtpGUNLpEjXHwp6CmghdiMaDq715km170Ut6saSF/yv85qoGXBdt1772+gIOKgqpCV03Ub2ll
	SGemPpEVh69A0WOiDXAjnVtJ74I3jFd1HL0mPiveRxyEsJBe0nH2BrTMAC9Few6JIMYg=;
Message-ID: <1f251f2e-91de-4f81-a93c-dcb95746d2e5@xen.org>
Date: Wed, 28 Feb 2024 11:50:24 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: preparations for 4.18.1
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Anthony Perard <anthony.perard@citrix.com>, Kelly Choi <kelly.choi@cloud.com>
References: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 27/02/2024 13:19, Jan Beulich wrote:
> All,
> 
> the release is due in two to three weeks. Please point out backports you find
> missing from the respective staging branch, but which you consider relevant.

For Arm:

e11f576650 ("xen/arm: Fix UBSAN failure in start_xen()")

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:53:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:53:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686505.1068655 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIVJ-0001Vx-6i; Wed, 28 Feb 2024 11:53:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686505.1068655; Wed, 28 Feb 2024 11:53:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIVJ-0001Vq-2a; Wed, 28 Feb 2024 11:53:41 +0000
Received: by outflank-mailman (input) for mailman id 686505;
 Wed, 28 Feb 2024 11:53:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Gbmv=KF=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1rfIVH-0001Vk-Jl
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:53:39 +0000
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
 (mail-bn7nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2009::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0172dd0d-d630-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 12:53:37 +0100 (CET)
Received: from DS7PR05CA0022.namprd05.prod.outlook.com (2603:10b6:5:3b9::27)
 by PH7PR12MB6740.namprd12.prod.outlook.com (2603:10b6:510:1ab::7) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.41; Wed, 28 Feb
 2024 11:53:34 +0000
Received: from DS1PEPF00017094.namprd03.prod.outlook.com
 (2603:10b6:5:3b9:cafe::2) by DS7PR05CA0022.outlook.office365.com
 (2603:10b6:5:3b9::27) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.26 via Frontend
 Transport; Wed, 28 Feb 2024 11:53:34 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 DS1PEPF00017094.mail.protection.outlook.com (10.167.17.137) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 11:53:33 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 28 Feb
 2024 05:53:33 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 28 Feb
 2024 05:53:32 -0600
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 28 Feb 2024 05:53:30 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0172dd0d-d630-11ee-afd7-a90da7624cb6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=hiiClwfXiE9FqsAEP/8Z/JWgDQbt/+sawICxsBqkrvirvDCF0fG/iYgtRxl9BL8k46fbolzDKLPhBnr4/sTn1ygj5NPhteUnoaQC0SrSChVYwek6qhXWC5xLJWWbps1LdTTdhtkxspUThIlrbtsjdx+uytbjfk+aJgSTEjN/5yAVfcANHoAOjiAcfbnWP7Hac9UBcOGy6Yb7xybkLkEZwPAfYXKjq/4GWE+RLxKKpkj8zbOZMeHsatHScwj/iKFYRWRS0PHbqGvlL95OFXzUmPeGzGv63aHNScVDpGuoSu0qBlKBqDbA3QMSelUe95txb9BzBCqd7URIgF8I3sTuZA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=FLzq84tn4lCka/GQbxFPBE0tifvRbGlwJ8Ye+duneBo=;
 b=WgLfJUJk2qADch331FrRyzCT3R0WocIkELN/hN+P5mE3R22ltRomh4rs4WpEDRwk0BFSXesvE4ucdy+A+77cpR8YZuvskjAbiCxV3gdwVBseo07284TsvZ+qRLrAbJXtMdHCJQiG/lzXnfoQJVky+BIGNZsP70uDpiqX17mWZppW/8WSsLTNxt4Qc+sqCUucM1KowhsB2s7yK61p2WvIyk5j+RanlstciwM0UiMqm9Tpyr36wY0brMZO78U34ZAQl7fF5hFAUg5KS6kTuTLFYKvsItaqLwHD14CEVO+8ckx+C5/z2c82rjaK/29QomhGcNzEGYFgacPy4vfr/T/OpQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FLzq84tn4lCka/GQbxFPBE0tifvRbGlwJ8Ye+duneBo=;
 b=cZGHTTSZaDY7X29iCbEm9hwRZFjpXK76cGY07SGOvvAfKpqZVCUycl1L6FDFZVif7jQuPWO4ZaFB1Yu4x7qUzVOl8GHI2Y1EFiN5eAVtTr8urz9Y/Ns1lrEtfC4f4YCkow3D2IWV1kaagN95x+Tpc0n2bHLAd4Xkn9HZuEIUC24=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <8e56b75d-10a2-4764-8244-3e596f9fc695@amd.com>
Date: Wed, 28 Feb 2024 19:53:29 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Alec Kwapis <alec.kwapis@medtronic.com>
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <d1518124-483c-4409-9b36-6a3392378911@xen.org>
 <a84aeb87-17e8-4195-90cb-7b0123064106@amd.com>
 <3982ba47-6709-47e3-a9c2-e2d3b4a2d8e3@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <3982ba47-6709-47e3-a9c2-e2d3b4a2d8e3@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS1PEPF00017094:EE_|PH7PR12MB6740:EE_
X-MS-Office365-Filtering-Correlation-Id: 95ede8c5-6dfe-45a8-7b59-08dc3853e3ed
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	eYk3eLP2bThrkSYqPInrhxGhr3EjIvbamtv/ThJEZuXN7ff+y21SrK50rYRmfYvvOqNnuLReqMc1udZXiyjEZVYjIL+48YnxMM5mH5u2grBBM8Zms2bk2MEFGrPn6X8Tyh3ZTbRhR+M+AtKsjVsEAV60SAP46Lzjnaf3Z75eLkh23HwyX3R8SjKoIIWr/wdcXkRtAhTVUeUrNu0nfLioZwej8ranEZipn7n9MCjjo3huuwi8ZuWPK1iqt+i6H9qe4hxxtkJ3E2ANkuJJa7CMbwi576Z+dnrBytp299pBNUHvZAjs7i/KYLtwtmbEXvz3ARv3poZPIvs4rtIs0ySRYLaD2XOKMWG8up3SGOYY6xZm21nww3pusjTzwu0RluJO3Tpm5t1Awy9VN/LYJobBKzypcMMf9hM4f1yPI37k8uP7HB4w3uZ6phLo03t3oQpjZE1sFNvQso95u1IfygtgcCVoaxCHrDdhb7YBl99igLHwFAKwoVDgGo6cz0kJ4Xb8En/kEh4vei8FzpvPCNooO9pR4HcwTN/TVACdiZlGWJz2vaANtxPyigU3Pis6E2nGjEgqqNnA/OXfvv9f7RK0BN2jPquSVFSANHIoy6BrszF7ePFdpfQuy7PafZQ8tZ1PR1VN77VdxlQG4KWtaTT9eYG3i4QWPtKVdDfMK74z5R+WQmQELvfzqS/ONrKdrpHQav4hZLhMq2jSqwfUsqTv9F/NVVuGaVEv1HtOa5PSoQg4iyXhOzsYiygTENmGpZuu
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400014)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 11:53:33.6419
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 95ede8c5-6dfe-45a8-7b59-08dc3853e3ed
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS1PEPF00017094.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6740

Hi Julien,

On 2/28/2024 6:35 PM, Julien Grall wrote:
> Hi Henry,
>>>>
>>>> Force populate_physmap to take the "normal" memory allocation route 
>>>> for
>>>> the magic pages even for 1:1 Dom0less DomUs. This should work as long
>>>> as the 1:1 Dom0less DomU doesn't have anything else mapped at the same
>>>> guest address as the magic pages:
>>>> - gfn 0x39000 address 0x39000000
>>>> - gfn 0x39001 address 0x39001000
>>>> - gfn 0x39002 address 0x39002000
>>>> - gfn 0x39003 address 0x39003000
>>>
>>> This is very fragile. You are making the assumption that the magic 
>>> pages are not clashing with any RAM region. The layout defined in 
>>> arch-arm.h has been designed for guest where Xen is in full control 
>>> of the layout. This is not the case for directmapped domain. I don't 
>>> think it is correct to try to re-use part of the layout.
>>
>> Apologies for the (a bit) late reply, it took a bit longer for me to 
>> understand the story about directmap stuff, and yes, now I agree with 
>> you, for those directmapped domains we should not reuse the guest 
>> layout directly.
>>
>>> If you want to use 1:1 dom0less with xenstore & co, then you should 
>>> find a different place in memory for the magic pages (TDB how to 
>>> find that area). 
>>
>> Yes, and maybe we can use similar strategy in 
>> find_unallocated_memory() or find_domU_holes() to do that.
>>
>>> You will still have the problem of the 1:1 allocation, but I think 
>>> this could be solved bty adding a flag to force a non-1:1 allocation.
>>
>> After checking the code flow, below rough plan came to my mind, I 
>> think what we need to do is:
>>
>> (1) Find a range of un-used memory using similar method in 
>> find_unallocated_memory()/find_domU_holes()
>
> AFAIK, the toolstack doesn't have any knowledge of the memeory layout 
> for dom0less domUs today. We would need to expose it first.

If I understand correctly, I think the issue you mentioned here and ...

> Then the region could either be statically allocated (i.e. the admin 
> provides it in the DTB) or dynamically.
>
>> (2) Change the base address, i.e. GUEST_MAGIC_BASE in alloc_xs_page() 
>> in init-dom0less.c to point to the address in (1) if static mem or 11 
>> directmap. (I think this is a bit tricky though, do you have any 
>> method that in your mind?)
>
> AFAIK, the toolstack doesn't know whether a domain is direct mapped or 
> using static mem.

...here basically means we want to do the finding of the unused region 
in toolstack. Since currently what we care about is only a couple of 
pages instead of the whole memory map, could it be possible that we do 
the opposite: in alloc_xs_page(), we issue a domctl hypercall to Xen and 
do the finding work in Xen and return with the found gfn? Then the page 
can be mapped by populate_physmap() from alloc_xs_page() and used for 
XenStore.

If above approach makes sense to you, I have a further question: Since I 
understand that the extended region is basically for safely foreign 
mapping pages, and init_dom0less.c uses foreign memory map for this 
XenStore page, should we find the wanted page in the extended region? or 
even extended region should be excluded?

> I think this ties with what I just wrote above. For dom0less domUs, we 
> probably want Xen to prepare a memory layout (similar to the e820) 
> that will then be retrieved by the toolstack.
>
>>
>> (3) Use a flag or combination of existing flags (CDF_staticmem + 
>> CDF_directmap) in populate_physmap() to force the allocation of these 
>> magic pages using alloc_domheap_pages() - i.e. the "else" condition 
>> in the bottom
>
> If I am not mistaken, CDF_* are per-domain. So we would want to use 
> MEMF_*.

Ah yes you are correct, I indeed missed MEMF_*

>>> Also, why are you only checking the first GFN? What if the caller 
>>> pass an overlapped region?
>>
>> I am a bit confused. My understanding is at this point we are 
>> handling one page at a time.
>
> We are handling one "extent" at the time. This could be one or 
> multiple pages (see extent_order).

I agree, sorry I didn't express myself well. For this specific XenStore 
page, I think the extent_order is
fixed as 0 so there is only 1 page. But you made a good point and I will 
remember to check if there are
multiple pages or an overlapped region. Thanks!

Kind regards,
Henry



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:56:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:56:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686507.1068665 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIXx-00023r-II; Wed, 28 Feb 2024 11:56:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686507.1068665; Wed, 28 Feb 2024 11:56:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIXx-00023k-Et; Wed, 28 Feb 2024 11:56:25 +0000
Received: by outflank-mailman (input) for mailman id 686507;
 Wed, 28 Feb 2024 11:56:24 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfIXw-00023e-8K
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:56:24 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfIXu-0001FV-T5; Wed, 28 Feb 2024 11:56:22 +0000
Received: from [15.248.2.225] (helo=[10.45.19.69])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfIXu-0007tk-Kf; Wed, 28 Feb 2024 11:56:22 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=gfEVWc96OXcQlDHIuriBFoWP7JUHlvoN9aHFiQ2GFq8=; b=eOcw56uSNjYRldIdFSqhO6YSdO
	z3YqgMweNjaQAReHKIX6BfiANOURMwXwGqIcIjlBxLmS8bFpi44KCxd6mLNqM8vm8X5iO6FSC7dfC
	T2oBTxsyaPYZBFhj2Hf2JIOaTuFAf/StdFinBGMBdogIZgKe6wCgiqF8hlGelayM9JxQ=;
Message-ID: <361faf56-6cc0-444d-837e-5538555eef76@xen.org>
Date: Wed, 28 Feb 2024 11:56:20 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] xen/lib: introduce generic find next bit operations
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Shawn Anastasio <sanastasio@raptorengineering.com>
References: <fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <fea2e65768457adcfedbfcc294004b1d5c2e86ea.1707495704.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 09/02/2024 17:58, Oleksii Kurochko wrote:
> find-next-bit.c is common for Arm64, PPC and RISCV64,
> so it is moved to xen/lib.
> 
> PPC has been transitioned to generic functions from find-next-bit.c
> since it now shares the same implementation as the PPC-specific code.
> 
> The MISRA exclude list has been updated to verify
> lib/find-next-bit.c instead of Arm's find_next_bit.c,
> as Arm's find_next_bit.c has been relocated to xen/lib/.
> 
> Despite CONFIG_GENERIC_FIND_FIRST_BIT not currently being used in
> Xen and being removed from the Linux kernel [1], it could theoretically
> prove useful for Xen. This is because the Linux kernel transitioned
> Arm to the generic version of find_first_bit() and
> find_first_zero_bit() due to improvements [1] in both performance
> and .text size.
> It would be prudent to investigate whether [1] is applicable to Xen
> as well and, if so, implement the necessary changes in a separate patch.
> 
> [1] https://lore.kernel.org/linux-arch/20211005054059.475634-5-yury.norov@gmail.com/
> 
> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

This is now committed.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 11:57:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 11:57:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686509.1068675 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIYg-0002Xs-Qk; Wed, 28 Feb 2024 11:57:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686509.1068675; Wed, 28 Feb 2024 11:57:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIYg-0002Xl-Ne; Wed, 28 Feb 2024 11:57:10 +0000
Received: by outflank-mailman (input) for mailman id 686509;
 Wed, 28 Feb 2024 11:57:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfIYf-0002Xd-On
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 11:57:09 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfIYf-0001GP-GN; Wed, 28 Feb 2024 11:57:09 +0000
Received: from [15.248.2.225] (helo=[10.45.19.69])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfIYf-0007tk-B4; Wed, 28 Feb 2024 11:57:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:
	References:Cc:To:From:Subject:MIME-Version:Date:Message-ID;
	bh=ZH/YCG1g8b9nhcrPE9WEwVLQcY/BdV2qCyFooCB0B1A=; b=VaV+CsAjS9B7AwtvsPOwHXgrKz
	MfMld6hhj1GdMCiATdCak0DiEAHGYsdeNhM8OeR8N8DJ5tWE0q7z89FAam3JeuI1wd3CbmxdROqcs
	LOFB/xQmxgfN51okTTQ2IugpUzXKzJRHEBHqPz+kL1X/fRRSkKgHxjUXcUh/qB+a0DKY=;
Message-ID: <f4f94437-9bcb-4eb7-8f80-9e11019a53bb@xen.org>
Date: Wed, 28 Feb 2024 11:57:08 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Fix arm32 build failure when early printk is
 enabled
Content-Language: en-GB
From: Julien Grall <julien@xen.org>
To: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240228103555.172101-1-michal.orzel@amd.com>
 <9201bdc5-2e4c-48b7-8c92-f5063fde90b6@xen.org>
In-Reply-To: <9201bdc5-2e4c-48b7-8c92-f5063fde90b6@xen.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 28/02/2024 11:42, Julien Grall wrote:
> Hi Michal,
> 
> On 28/02/2024 10:35, Michal Orzel wrote:
>> Commit 0441c3acc7e9 forgot to rename FIXMAP_CONSOLE to FIX_CONSOLE in
>> TEMPORARY_EARLY_UART_VIRTUAL_ADDRESS macro. This results in a build
>> failure on arm32, when early printk is enabled:
>> arch/arm/arm32/mmu/head.S:311: Error: invalid operands (*UND* and 
>> *ABS* sections) for `*'
> 
> Good catch! Somewhat related I wonder whether we should add earlyprintk 
> testing in gitlab?
> 
>>
>> Fixes: 0441c3acc7e9 ("xen/arm: fixmap: Rename the fixmap slots to 
>> follow the x86 convention")
>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> 
> Acked-by: Julien Grall <jgrall@amazon.com>

And committed.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 12:18:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 12:18:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686527.1068692 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIsZ-0006yS-Uv; Wed, 28 Feb 2024 12:17:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686527.1068692; Wed, 28 Feb 2024 12:17:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIsZ-0006yL-S7; Wed, 28 Feb 2024 12:17:43 +0000
Received: by outflank-mailman (input) for mailman id 686527;
 Wed, 28 Feb 2024 12:17:42 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eIhW=KF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfIsY-0006yF-DA
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 12:17:42 +0000
Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com
 [2607:f8b0:4864:20::82f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5d945d8d-d633-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 13:17:40 +0100 (CET)
Received: by mail-qt1-x82f.google.com with SMTP id
 d75a77b69052e-42a31b90edcso36905401cf.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 04:17:40 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 w20-20020ac86b14000000b0042eb46d15bbsm51704qts.88.2024.02.28.04.17.36
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 04:17:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5d945d8d-d633-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709122658; x=1709727458; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:to
         :content-language:references:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=WW1MfJP14IrIt+y46I6jINaufgP5xttrEAT53WmYt4I=;
        b=lIgVPA8C2APnMtc4sDBlvNtbFhoJ5ThQ8QRuLsSL4JVD/DUW884SQOFS1lePfvx8gz
         ekTqE0NNGNDDQnZ5inHxLL7ZaFMnq6BQ4YK8O5xtbrSMu1OTxhgBBeOQt2zBqarqfGHB
         AinsUCef6xWWTLTwPKWgZ5YCUpQISocI3ojMA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709122658; x=1709727458;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:to
         :content-language:references:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=WW1MfJP14IrIt+y46I6jINaufgP5xttrEAT53WmYt4I=;
        b=OS2OueFwQFKygAGPzaB9JAF3C8hMhfJK57cnKirc0A0L25U5g7b5EIBeFiw5ZMSrZE
         LcEWiTNKdGQWWJcOvBOvrT/oixO3/TvSxwR7rVAC/sQxEHjmTQhGajAwxe2YhE6GTAkD
         PbfeYhDrh5/BSWZhUkJEDqC4eMj4Sdy8WUTOD7ebMEEQVamfvK8Xxbl2g4obUzkUJvDl
         LRwzOwko9ekPe6qs/tZNAeGExQbDFJQU3SljUKx5lHkuuEMHQ10BUfwasyu7DHDhuGse
         hZcE6NRCMQfb8HsZVdCbqwGrmBEj7x4ANdk4GZJq1MEgK8YXATjhg3InLVt6VkOasDlw
         lLlg==
X-Gm-Message-State: AOJu0YxWvKZehPguwunyQBn2Jwoff0d4peBc3ymjWS2btynyLwvVW84s
	gfBGDeXpOW6hIY0BciiCkDKBiGI+lnRvrEu6GayAcJbaSxJ69NZ2at/qTr0qCsMLDrUWRrmzW0C
	1
X-Google-Smtp-Source: AGHT+IEqxJFWaANY4p8xru6+bC6DV9LIVjgPpcpnLxP67wF/5K8msFb7Eqk2d8scPy4T0cNK6Pk8Zw==
X-Received: by 2002:a05:622a:2ca:b0:42e:9745:376d with SMTP id a10-20020a05622a02ca00b0042e9745376dmr6672689qtx.19.1709122658103;
        Wed, 28 Feb 2024 04:17:38 -0800 (PST)
Message-ID: <27fad7c4-905b-4181-9882-f383eb194032@citrix.com>
Date: Wed, 28 Feb 2024 12:17:35 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Fwd: New Defects reported by Coverity Scan for XenProject
References: <65df054dd791_2895d52c84469bd9989838e@prd-scan-dashboard-0.mail>
Content-Language: en-GB
To: xen-devel <xen-devel@lists.xenproject.org>,
 Stewart Hildebrand <stewart.hildebrand@dornerworks.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jan Beulich <jbeulich@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <65df054dd791_2895d52c84469bd9989838e@prd-scan-dashboard-0.mail>
X-Forwarded-Message-Id: <65df054dd791_2895d52c84469bd9989838e@prd-scan-dashboard-0.mail>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Not sure how well this is going to be formatted, but there's one new and
potentially interesting issue found by Coverity.

~Andrew

----8<----

New defect(s) Reported-by: Coverity Scan
Showing 1 of 1 defect(s)


** CID 1592633: (LOCK_EVASION)
/xen/drivers/vpci/header.c: 229 in vpci_process_pending()
/xen/drivers/vpci/header.c: 189 in vpci_process_pending()
/xen/drivers/vpci/header.c: 239 in vpci_process_pending()


________________________________________________________________________________________________________
*** CID 1592633: (LOCK_EVASION)
/xen/drivers/vpci/header.c: 229 in vpci_process_pending()
223 224 /* Clean all the rangesets */
225 for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
226 if ( !rangeset_is_empty(header->bars[i].mem) )
227 rangeset_purge(header->bars[i].mem);
228
>>> CID 1592633: (LOCK_EVASION)
>>> Thread1 sets "pdev" to a new value. Now the two threads have an
>>> inconsistent view of "pdev" and updates to fields of "pdev" or
>>> fields correlated with "pdev" may be lost.
229 v->vpci.pdev = NULL;
230 231 read_unlock(&v->domain->pci_lock);
232 233 if ( !is_hardware_domain(v->domain) )
234 domain_crash(v->domain);
/xen/drivers/vpci/header.c: 189 in vpci_process_pending()
183 return false;
184 185 read_lock(&v->domain->pci_lock);
186 187 if ( !pdev->vpci || (v->domain != pdev->domain) )
188 {
>>> CID 1592633: (LOCK_EVASION)
>>> Thread1 sets "pdev" to a new value. Now the two threads have an
>>> inconsistent view of "pdev" and updates to fields of "pdev" or
>>> fields correlated with "pdev" may be lost.
189 v->vpci.pdev = NULL;
190 read_unlock(&v->domain->pci_lock);
191 return false;
192 }
193 194 header = &pdev->vpci->header;
/xen/drivers/vpci/header.c: 239 in vpci_process_pending()
233 if ( !is_hardware_domain(v->domain) )
234 domain_crash(v->domain);
235 236 return false;
237 }
238 }
>>> CID 1592633: (LOCK_EVASION)
>>> Thread1 sets "pdev" to a new value. Now the two threads have an
>>> inconsistent view of "pdev" and updates to fields of "pdev" or
>>> fields correlated with "pdev" may be lost.
239 v->vpci.pdev = NULL;
240 241 spin_lock(&pdev->vpci->lock);
242 modify_decoding(pdev, v->vpci.cmd, v->vpci.rom_only);
243 spin_unlock(&pdev->vpci->lock);
244



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 12:19:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 12:19:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686530.1068703 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfItk-0007UJ-A3; Wed, 28 Feb 2024 12:18:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686530.1068703; Wed, 28 Feb 2024 12:18:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfItk-0007UC-5p; Wed, 28 Feb 2024 12:18:56 +0000
Received: by outflank-mailman (input) for mailman id 686530;
 Wed, 28 Feb 2024 12:18:54 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Epip=KF=cloud.com=kelly.choi@srs-se1.protection.inumbo.net>)
 id 1rfIti-0007TS-74
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 12:18:54 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 88f70205-d633-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 13:18:52 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a3f4464c48dso649149566b.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 04:18:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 88f70205-d633-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709122731; x=1709727531; darn=lists.xenproject.org;
        h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=4taUMyx+sVUhovAipqeBrqCwHbuFWWOytz2rfh5lTew=;
        b=VTz4RXzPO7fKFvz2hBd9nWRLiEccSYxb9uivOdhKvs0Ba4vCTjGMAFhy1ayAo0A2IH
         LJMh03PBbTi0s+jGjTaLe7S/AeRAlFrvVD6ZJVfVpjMt4OfUEZF5NQlugxmzKsuqEY/3
         6ckWW0ePTnNvpn3HbHZcYrpBommx4J5veQVGc=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709122731; x=1709727531;
        h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=4taUMyx+sVUhovAipqeBrqCwHbuFWWOytz2rfh5lTew=;
        b=dm1C/JroBqN7c07de/d16mTb2+pvxzNKTKxBHJnfbQHTBdQBUrd3kIFiqTgJIsp/b3
         8Eft7mYAP2xeNbs0HxLl0+XdT3YEyU715ckMJCuOtuLDTw9egyUIO+rr7VGZR24VxyYi
         mCCs7GB/n6EiMmESZB+s76nv+toxpIk6pYZ2dEnqS6h3ZVbs/NCy27jkK417rrlKx5/N
         MolpYQxgDCwjbw7GXFUo1eaOPbmevLYDjtZcCTbrfUeR4oCZqTpySrOglOu/gf5fu/rb
         L+8mw1iFTAxIX/ZCc2yoo8eUTQ0269iwfeoUxAK6pOJLrAC0Gni2hhj4+DSo+lIBNgsW
         2kjg==
X-Gm-Message-State: AOJu0YyY7YAE6jq2o/Scc94B9nOzSiFU24nFT8e+qChaTcP83k+FK+G2
	KYMBKkjg4L2SivSZHpCdYmwtNROpNEI2ieenzFlMoTzNu4Prjn9wgRs3UT/yVqDURTvNPn4RXxQ
	eENhMy/AWQcaZIkXq7MWG16IEBK/I9G1R7sXLCn1jXnoxqbSI
X-Google-Smtp-Source: AGHT+IGDvjOVbM4/z/sUM5gtt+xtbdqOdtx9QNtkSKsbbDqHVDD0O7XabBJ5GUWocwKH8h72OAXEPK7rdf8neJkqfHQ=
X-Received: by 2002:a17:906:3683:b0:a3e:7f54:12c1 with SMTP id
 a3-20020a170906368300b00a3e7f5412c1mr9188021ejc.72.1709122731589; Wed, 28 Feb
 2024 04:18:51 -0800 (PST)
MIME-Version: 1.0
From: Kelly Choi <kelly.choi@cloud.com>
Date: Wed, 28 Feb 2024 12:18:15 +0000
Message-ID: <CAO-mL=y60vBUtrWrYQeVpiR79MeMR8SkgAtt5MZdt9msNB+mXA@mail.gmail.com>
Subject: Xen Survey Results
To: xen-devel@lists.xenproject.org, xen-users@lists.xenproject.org, 
	xen-announce@lists.xenproject.org
Cc: committers@xenproject.org, advisory-board@lists.xenproject.org
Content-Type: multipart/alternative; boundary="0000000000004b616a06127025e2"

--0000000000004b616a06127025e2
Content-Type: text/plain; charset="UTF-8"

Hi everyone,

Thank you for contributing your feedback to the recent survey.

A lot of you wrote detailed responses and gave specific feedback as to what
the community was doing well on, and what we could improve.

I've spent a significant amount of time collating these responses and would
like to present to you the results. I aim to run this annually to track our
community health, so I encourage you all to participate.

*VIEW SURVEY RESULTS HERE*
<https://cryptpad.fr/file/#/2/file/nvDsPv6i+ojtA7UKXiFm8i2R/>

Many thanks,
Kelly Choi

Community Manager
Xen Project

--0000000000004b616a06127025e2
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi everyone,=C2=A0<div><br></div><div>Thank you for contri=
buting your feedback to the recent survey.=C2=A0</div><div><br></div><div>A=
 lot of you wrote detailed responses and gave specific feedback as to what =
the community was doing well on, and what we could improve.=C2=A0</div><div=
><br></div><div>I&#39;ve spent a significant amount of time collating these=
 responses and=C2=A0would like to present to you the results. I aim to run =
this annually to track our community health, so I encourage you all to part=
icipate.=C2=A0</div><div><br></div><div><a href=3D"https://cryptpad.fr/file=
/#/2/file/nvDsPv6i+ojtA7UKXiFm8i2R/"><b>VIEW SURVEY RESULTS HERE</b></a></d=
iv><div><br clear=3D"all"><div><div dir=3D"ltr" class=3D"gmail_signature" d=
ata-smartmail=3D"gmail_signature"><div dir=3D"ltr"><div>Many thanks,</div><=
div>Kelly Choi</div><div><br></div><div><div style=3D"color:rgb(136,136,136=
)">Community Manager</div><div style=3D"color:rgb(136,136,136)">Xen Project=
=C2=A0<br></div></div></div></div></div></div></div>

--0000000000004b616a06127025e2--


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 12:19:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 12:19:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686547.1068732 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIuU-0000Px-WB; Wed, 28 Feb 2024 12:19:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686547.1068732; Wed, 28 Feb 2024 12:19:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfIuU-0000Pq-T7; Wed, 28 Feb 2024 12:19:42 +0000
Received: by outflank-mailman (input) for mailman id 686547;
 Wed, 28 Feb 2024 12:19:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eIhW=KF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfIuT-0007Te-DS
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 12:19:41 +0000
Received: from mail-yw1-x112f.google.com (mail-yw1-x112f.google.com
 [2607:f8b0:4864:20::112f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a4c460b0-d633-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 13:19:39 +0100 (CET)
Received: by mail-yw1-x112f.google.com with SMTP id
 00721157ae682-609408d4b31so4020757b3.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 04:19:39 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 r7-20020a05620a03c700b00787f695c487sm18635qkm.98.2024.02.28.04.19.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 04:19:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a4c460b0-d633-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709122777; x=1709727577; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:to:from
         :content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=zIqV97Yzso96j6r8pPlx9zxMzC9ALupR+ISIpoZks0M=;
        b=IOM8LoIy/T+Lavsu1YYBmX/W6juFF4UYfyJEHbn7Upwfi8JMH8lbv/YDX56M7vCNh2
         k/+QdYzY2Pz7s3yNFMhktRjozBoQmnEMX1kyhXC5CM2ugEo3AxFe3kp52hFSzDV54b39
         hjge2C7sw5HD3kXBZddmxuXu2DuxNs1b15XJg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709122777; x=1709727577;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:to:from
         :content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=zIqV97Yzso96j6r8pPlx9zxMzC9ALupR+ISIpoZks0M=;
        b=exZlIDpbqDhvXx32Ei9hqa2hNvqVBqzOMUCVmWD4+tFeMC6cYySnti5Xau+p30K7o6
         4amRAtYDPFHClm8p+cWivOYekLjEmH9YlB5nk0OflifOheYzBMU8LF4xlFWkKkkBNCAy
         5hKYnkHSPhBSBrYOBhl0rDLfxP4L8aYYNtHFs/IsU/lA8mbjqTPAWLZv54Kx/Tw14rTf
         UMVb7c5b8YSQVJD+UY4axoIbBcmJC0Towny3ue3FJ7QTVC3AW6wNUMUBqSlPTHH3Qvv8
         XqMbqb/xTLfPmB6JX1kYQIGajSfU15/PleGBMePKfXmQErb6/VX4Nveo06wIzkiATN20
         zAZQ==
X-Gm-Message-State: AOJu0Yz4/IdWN/zT2CnXEa8jOMjQokpEYVLHfQ/zP4kYL8L8jQKeROhE
	Orp2qM8GKhveHWnudq3t1tNRLdc/JdfeJ8RnaxE5GQjkEeJWGydBBkQZOVpFzyxSKQIVlSvDPtv
	E
X-Google-Smtp-Source: AGHT+IGP72SvZHhHUPfQYaMtamT0Ld3mmJiNJIBtF+NnVQ3Qw/zpgYIjvYtpPAJrGQmiGc7sVsnFgg==
X-Received: by 2002:a81:b547:0:b0:609:344b:edf7 with SMTP id c7-20020a81b547000000b00609344bedf7mr3259043ywk.2.1709122777585;
        Wed, 28 Feb 2024 04:19:37 -0800 (PST)
Message-ID: <d81f78a8-3e04-4fba-95fc-3b602ee3a740@citrix.com>
Date: Wed, 28 Feb 2024 12:19:35 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Fwd: New Defects reported by Coverity Scan for XenProject
Content-Language: en-GB
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: xen-devel <xen-devel@lists.xenproject.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jan Beulich <jbeulich@suse.com>,
 Stewart Hildebrand <Stewart.Hildebrand@amd.com>
References: <65df054dd791_2895d52c84469bd9989838e@prd-scan-dashboard-0.mail>
 <27fad7c4-905b-4181-9882-f383eb194032@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <27fad7c4-905b-4181-9882-f383eb194032@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Take 2, hopefully with Stewart's correct email address this time.

~Andrew

On 28/02/2024 12:17 pm, Andrew Cooper wrote:
> Not sure how well this is going to be formatted, but there's one new and
> potentially interesting issue found by Coverity.
>
> ~Andrew
>
> ----8<----
>
> New defect(s) Reported-by: Coverity Scan
> Showing 1 of 1 defect(s)
>
>
> ** CID 1592633: (LOCK_EVASION)
> /xen/drivers/vpci/header.c: 229 in vpci_process_pending()
> /xen/drivers/vpci/header.c: 189 in vpci_process_pending()
> /xen/drivers/vpci/header.c: 239 in vpci_process_pending()
>
>
> ________________________________________________________________________________________________________
> *** CID 1592633: (LOCK_EVASION)
> /xen/drivers/vpci/header.c: 229 in vpci_process_pending()
> 223 224 /* Clean all the rangesets */
> 225 for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
> 226 if ( !rangeset_is_empty(header->bars[i].mem) )
> 227 rangeset_purge(header->bars[i].mem);
> 228
>>>> CID 1592633: (LOCK_EVASION)
>>>> Thread1 sets "pdev" to a new value. Now the two threads have an
>>>> inconsistent view of "pdev" and updates to fields of "pdev" or
>>>> fields correlated with "pdev" may be lost.
> 229 v->vpci.pdev = NULL;
> 230 231 read_unlock(&v->domain->pci_lock);
> 232 233 if ( !is_hardware_domain(v->domain) )
> 234 domain_crash(v->domain);
> /xen/drivers/vpci/header.c: 189 in vpci_process_pending()
> 183 return false;
> 184 185 read_lock(&v->domain->pci_lock);
> 186 187 if ( !pdev->vpci || (v->domain != pdev->domain) )
> 188 {
>>>> CID 1592633: (LOCK_EVASION)
>>>> Thread1 sets "pdev" to a new value. Now the two threads have an
>>>> inconsistent view of "pdev" and updates to fields of "pdev" or
>>>> fields correlated with "pdev" may be lost.
> 189 v->vpci.pdev = NULL;
> 190 read_unlock(&v->domain->pci_lock);
> 191 return false;
> 192 }
> 193 194 header = &pdev->vpci->header;
> /xen/drivers/vpci/header.c: 239 in vpci_process_pending()
> 233 if ( !is_hardware_domain(v->domain) )
> 234 domain_crash(v->domain);
> 235 236 return false;
> 237 }
> 238 }
>>>> CID 1592633: (LOCK_EVASION)
>>>> Thread1 sets "pdev" to a new value. Now the two threads have an
>>>> inconsistent view of "pdev" and updates to fields of "pdev" or
>>>> fields correlated with "pdev" may be lost.
> 239 v->vpci.pdev = NULL;
> 240 241 spin_lock(&pdev->vpci->lock);
> 242 modify_decoding(pdev, v->vpci.cmd, v->vpci.rom_only);
> 243 spin_unlock(&pdev->vpci->lock);
> 244
>



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 12:27:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 12:27:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686565.1068754 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJ1g-0003Iv-Te; Wed, 28 Feb 2024 12:27:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686565.1068754; Wed, 28 Feb 2024 12:27:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJ1g-0003Io-R6; Wed, 28 Feb 2024 12:27:08 +0000
Received: by outflank-mailman (input) for mailman id 686565;
 Wed, 28 Feb 2024 12:27:07 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfJ1f-0003IO-3y
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 12:27:07 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfJ1e-0001ox-Fy; Wed, 28 Feb 2024 12:27:06 +0000
Received: from [15.248.2.225] (helo=[10.45.19.69])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfJ1e-0001F1-7r; Wed, 28 Feb 2024 12:27:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=JJthXOk0MMslCQCxJJh8RnkNyO9qWm5ioQIP1MDFZdk=; b=qex29y0Uezi09O5zsyFn+4Ytzb
	XvKePckSIDQgEiDcuxzFhKNUuRYpHLJp2kECyJ51gAOeXx4px0bwV4oMv32+8U/M1amRYWEoT3mCF
	d6BvgOnLdDlfQZ3TbS0+ni9kx1QHCCjnnWfXJDfNuUej7Pfs622TasqVbG8aHsaC4JCg=;
Message-ID: <98739f8d-dc00-4db8-a0b2-07f054107acf@xen.org>
Date: Wed, 28 Feb 2024 12:27:03 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
Content-Language: en-GB
To: Henry Wang <xin.wang2@amd.com>, xen-devel@lists.xenproject.org
Cc: Anthony PERARD <anthony.perard@citrix.com>,
 Juergen Gross <jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Alec Kwapis <alec.kwapis@medtronic.com>
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <d1518124-483c-4409-9b36-6a3392378911@xen.org>
 <a84aeb87-17e8-4195-90cb-7b0123064106@amd.com>
 <3982ba47-6709-47e3-a9c2-e2d3b4a2d8e3@xen.org>
 <8e56b75d-10a2-4764-8244-3e596f9fc695@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <8e56b75d-10a2-4764-8244-3e596f9fc695@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Henry,

On 28/02/2024 11:53, Henry Wang wrote:
> On 2/28/2024 6:35 PM, Julien Grall wrote:
>> Hi Henry,
>>>>>
>>>>> Force populate_physmap to take the "normal" memory allocation route 
>>>>> for
>>>>> the magic pages even for 1:1 Dom0less DomUs. This should work as long
>>>>> as the 1:1 Dom0less DomU doesn't have anything else mapped at the same
>>>>> guest address as the magic pages:
>>>>> - gfn 0x39000 address 0x39000000
>>>>> - gfn 0x39001 address 0x39001000
>>>>> - gfn 0x39002 address 0x39002000
>>>>> - gfn 0x39003 address 0x39003000
>>>>
>>>> This is very fragile. You are making the assumption that the magic 
>>>> pages are not clashing with any RAM region. The layout defined in 
>>>> arch-arm.h has been designed for guest where Xen is in full control 
>>>> of the layout. This is not the case for directmapped domain. I don't 
>>>> think it is correct to try to re-use part of the layout.
>>>
>>> Apologies for the (a bit) late reply, it took a bit longer for me to 
>>> understand the story about directmap stuff, and yes, now I agree with 
>>> you, for those directmapped domains we should not reuse the guest 
>>> layout directly.
>>>
>>>> If you want to use 1:1 dom0less with xenstore & co, then you should 
>>>> find a different place in memory for the magic pages (TDB how to 
>>>> find that area). 
>>>
>>> Yes, and maybe we can use similar strategy in 
>>> find_unallocated_memory() or find_domU_holes() to do that.
>>>
>>>> You will still have the problem of the 1:1 allocation, but I think 
>>>> this could be solved bty adding a flag to force a non-1:1 allocation.
>>>
>>> After checking the code flow, below rough plan came to my mind, I 
>>> think what we need to do is:
>>>
>>> (1) Find a range of un-used memory using similar method in 
>>> find_unallocated_memory()/find_domU_holes()
>>
>> AFAIK, the toolstack doesn't have any knowledge of the memeory layout 
>> for dom0less domUs today. We would need to expose it first.
> 
> If I understand correctly, I think the issue you mentioned here and ...
> 
>> Then the region could either be statically allocated (i.e. the admin 
>> provides it in the DTB) or dynamically.
>>
>>> (2) Change the base address, i.e. GUEST_MAGIC_BASE in alloc_xs_page() 
>>> in init-dom0less.c to point to the address in (1) if static mem or 11 
>>> directmap. (I think this is a bit tricky though, do you have any 
>>> method that in your mind?)
>>
>> AFAIK, the toolstack doesn't know whether a domain is direct mapped or 
>> using static mem.
> 
> ...here basically means we want to do the finding of the unused region 
> in toolstack. Since currently what we care about is only a couple of 
> pages instead of the whole memory map, could it be possible that we do 
> the opposite: in alloc_xs_page(), we issue a domctl hypercall to Xen and 
> do the finding work in Xen and return with the found gfn? Then the page 
> can be mapped by populate_physmap() from alloc_xs_page() and used for 
> XenStore.

I know that DOMCTL hypercalls are not stable. But I am not overly happy 
with creating an hypercall which is just "fetch the magic regions". I 
think it need to be a more general one that would expose all the regions.

Also, you can't really find the magic regions when the hypercall is done 
as we don't have the guest memory layout. This will need to be done in 
advance.

Overall, I think it would be better if we provide an hypercall listing 
the regions currently occupied (similar to e820). One will have the type 
"magic pages".

> 
> If above approach makes sense to you, I have a further question: Since I 
> understand that the extended region is basically for safely foreign 
> mapping pages

This is not about safety. The extended region is optional. It was 
introduced so it is easy for Linux to find an unallocated region to map 
external pages (e.g. vCPU shared info) so it doesn't waste RAM pages.

, and init_dom0less.c uses foreign memory map for this
> XenStore page, 
> should we find the wanted page in the extended region? or 
> even extended region should be excluded?

How is the extended region found for dom0less domUs today? It would be 
fine to steal some part of the extended regions for the magic pages. But 
they would need to be reserved *before* the guest is first unpaused.

>> I think this ties with what I just wrote above. For dom0less domUs, we 
>> probably want Xen to prepare a memory layout (similar to the e820) 
>> that will then be retrieved by the toolstack.
>>
>>>
>>> (3) Use a flag or combination of existing flags (CDF_staticmem + 
>>> CDF_directmap) in populate_physmap() to force the allocation of these 
>>> magic pages using alloc_domheap_pages() - i.e. the "else" condition 
>>> in the bottom
>>
>> If I am not mistaken, CDF_* are per-domain. So we would want to use 
>> MEMF_*.
> 
> Ah yes you are correct, I indeed missed MEMF_*
> 
>>>> Also, why are you only checking the first GFN? What if the caller 
>>>> pass an overlapped region?
>>>
>>> I am a bit confused. My understanding is at this point we are 
>>> handling one page at a time.
>>
>> We are handling one "extent" at the time. This could be one or 
>> multiple pages (see extent_order).
> 
> I agree, sorry I didn't express myself well. For this specific XenStore 
> page, I think the extent_order is
> fixed as 0 so there is only 1 page.

Correct. But you should not rely on this :).

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 12:43:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 12:43:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686584.1068767 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJGw-00086d-9v; Wed, 28 Feb 2024 12:42:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686584.1068767; Wed, 28 Feb 2024 12:42:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJGw-00086W-7M; Wed, 28 Feb 2024 12:42:54 +0000
Received: by outflank-mailman (input) for mailman id 686584;
 Wed, 28 Feb 2024 12:42:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=NMA4=KF=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rfJGv-00086N-Lt
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 12:42:53 +0000
Received: from NAM12-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:2417::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e147e9e1-d636-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 13:42:49 +0100 (CET)
Received: from BN9PR03CA0473.namprd03.prod.outlook.com (2603:10b6:408:139::28)
 by CH0PR12MB5218.namprd12.prod.outlook.com (2603:10b6:610:d1::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Wed, 28 Feb
 2024 12:42:46 +0000
Received: from BN2PEPF000044A3.namprd02.prod.outlook.com
 (2603:10b6:408:139:cafe::e7) by BN9PR03CA0473.outlook.office365.com
 (2603:10b6:408:139::28) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.27 via Frontend
 Transport; Wed, 28 Feb 2024 12:42:45 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN2PEPF000044A3.mail.protection.outlook.com (10.167.243.154) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 12:42:45 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 28 Feb
 2024 06:42:45 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 28 Feb
 2024 04:42:45 -0800
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 28 Feb 2024 06:42:44 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e147e9e1-d636-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=jZfL+M8sdAzOUxjx8DxbA2DPaxfI2rj/m6leLZmOt9RD6DQJ/esSVbmzt7208b31M3B++xZYogVuTDNn9krVMprv9pVEw7Jswz+0Z7Hb3ySAwJQ+viBf+aUFTVs1xB/hC3VP7aGZ9/xG8DGFanIFU3Dqk4z/axjuZw8s+L9I8eRjzeQjCEIrgeMMQMe0BjA6LEP572HS5ZkOjl4ARnXrQ3X1LZARjDOMrmVKgQ37UI2/FsmcLS3CIW8SeTehxMVvKvqwNrTzcbbDh7iO1ReHSOU2cwy7RSgY6CzcFd19UvECLI6EEyH4aaA2aWUppZegqdyLBARPwb9GEKTFoGNZ9w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=B1IXjfxMjHnGO9oplCFMYfK64TTR/fPk9KjMyeZlPPk=;
 b=JaJfTA+DhItzQP/lqXBF3zkND4JbpBnbweervfYiY+ufxmNffOpRy2ISIT6ip2ELCmgK+kHimAF6UkT+Cw52fy1hvgoXjGWDIMS7zVqk9yYOwhi0ueZWfh6NCdQQ6gzudxBbBA9szEWWlMLH7h/DUfTNmsC4T58lAdgXwYu4wd+7L02v9LcJZdAlDYzK1ljy9oYgVtWYa2568tj55c4tgHV6ieHNQBehytlQMb8DNRLG+OJLFUsDl1CO4WStvcVQ2O3fMzZ9TRsZ92xerJfaJ0+u+lDltShxn39HW2JLQPv7y9UpWs9lx/sKoJB6i1SpF9EHWQvh/nFoEpF7EXjzYg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=B1IXjfxMjHnGO9oplCFMYfK64TTR/fPk9KjMyeZlPPk=;
 b=mmsZaw4TJv1ksYgNDmqwDSADI2uDbYk3B/bNUrujwNi+xJtwdTcRi2isRTzi2PtNqFlY8EiQhK1Px/ZwoPGrUo0+bVFTZik/YiFmV5oxyzR5BkYC0RQ1lxe8FgF8SCOnZ9kJvQzpyGzgXQdfS/ulZb+47PStlfeUI27N1JovFEE=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <af11620a-ec16-4677-b968-d8e36c4555d2@amd.com>
Date: Wed, 28 Feb 2024 13:42:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Fix arm32 build failure when early printk is
 enabled
Content-Language: en-US
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Stefano Stabellini <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240228103555.172101-1-michal.orzel@amd.com>
 <9201bdc5-2e4c-48b7-8c92-f5063fde90b6@xen.org>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <9201bdc5-2e4c-48b7-8c92-f5063fde90b6@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN2PEPF000044A3:EE_|CH0PR12MB5218:EE_
X-MS-Office365-Filtering-Correlation-Id: 3df9e65d-ff25-40c7-0559-08dc385ac364
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	YrwJwok5rjpqoSvVkDM3eRMOmM6unxWll9I8b8lbPUu4tgzgJ81fpUKjke6qoqv0RD88t14gS1he+IbTBxuvkaOG7FsuJrW3XGjPZBfea0BenJCSFNrMICX5AkXx71W2/5FNSawJkRMt719FqPOEdO5x1uGXtIxZKGCES26/a+ZNoO0AYL9vP6YT0JYTYBigDAK/LviOa3/c2bWyBZ9aQoiqO0lzaODJ45YsOQR5+Je2ng8X5Pb0eOlYEHtfrXkf0RnIGw7txR5iU32yYsedxe4dCqG28sAroLcVjK98hxd6gPW7TKeqSsn9Pifbcn9rrtamQi1GwT9HWdkSgkq/he5nPmQermTQ1nXmY1TGfhrm4ONm4fgGYyvzFVkTIXXfKUTEfnF+LvEN/P3oMQRIchAMQGqc9yq+nWEZnCkq9gdoq5wb9cZpxTX1BIQH1zTGwD0DdWhAo67C2BuGKLQgMs8NCYBpF+lU/8bdysAPjw9Xzy+1vBQnH0eiips5X1GVpxAV8xQVru+V7XIbjIFjMQYt4xAFqkdQfmfmgo4YmJ3m0wM1Jatopv8KJABkYyYUHVFQIWcJRaEeKaaj3o8aUAlQvwetI0kDsZwDR4glHCaaVPq/quX92AyJTBxJ8AirIViuW9eb6bhcOptLaALfNnFdnfRsOViVPpwIvDBkIg3VDiZrf7VcfBV0IjMnmUHFjuSNQIyf8YdxCO3c0IKexxS56jQ00wb9B3AHJozt/zC1NMa5tzybXfXf9A+xIwzC
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400014)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 12:42:45.5623
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 3df9e65d-ff25-40c7-0559-08dc385ac364
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BN2PEPF000044A3.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5218

Hi Julien,

On 28/02/2024 12:42, Julien Grall wrote:
> 
> 
> Hi Michal,
> 
> On 28/02/2024 10:35, Michal Orzel wrote:
>> Commit 0441c3acc7e9 forgot to rename FIXMAP_CONSOLE to FIX_CONSOLE in
>> TEMPORARY_EARLY_UART_VIRTUAL_ADDRESS macro. This results in a build
>> failure on arm32, when early printk is enabled:
>> arch/arm/arm32/mmu/head.S:311: Error: invalid operands (*UND* and *ABS* sections) for `*'
> 
> Good catch! Somewhat related I wonder whether we should add earlyprintk
> testing in gitlab?
I thought about adding this and I think we should at least have build jobs (hypervisor only, no toolstack)
selecting early printk. When it comes to testing if early printk works, I'm not sure. It'd be nice
but FWIR we have limited bandwidth.

@Stefano, what's your opinion?

~Michal


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 12:50:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 12:50:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686588.1068777 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJO8-0001vc-T7; Wed, 28 Feb 2024 12:50:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686588.1068777; Wed, 28 Feb 2024 12:50:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJO8-0001vV-QV; Wed, 28 Feb 2024 12:50:20 +0000
Received: by outflank-mailman (input) for mailman id 686588;
 Wed, 28 Feb 2024 12:50:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=Gbmv=KF=amd.com=Xin.Wang2@srs-se1.protection.inumbo.net>)
 id 1rfJO8-0001u1-7u
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 12:50:20 +0000
Received: from NAM02-DM3-obe.outbound.protection.outlook.com
 (mail-dm3nam02on20600.outbound.protection.outlook.com
 [2a01:111:f403:2405::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ec0d981c-d637-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 13:50:17 +0100 (CET)
Received: from CH2PR05CA0010.namprd05.prod.outlook.com (2603:10b6:610::23) by
 SA1PR12MB5660.namprd12.prod.outlook.com (2603:10b6:806:238::15) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Wed, 28 Feb
 2024 12:50:13 +0000
Received: from SN1PEPF0002BA4D.namprd03.prod.outlook.com
 (2603:10b6:610:0:cafe::86) by CH2PR05CA0010.outlook.office365.com
 (2603:10b6:610::23) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7362.12 via Frontend
 Transport; Wed, 28 Feb 2024 12:50:12 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SN1PEPF0002BA4D.mail.protection.outlook.com (10.167.242.70) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Wed, 28 Feb 2024 12:50:12 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 28 Feb
 2024 06:50:11 -0600
Received: from [172.31.100.92] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Wed, 28 Feb 2024 06:50:08 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ec0d981c-d637-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Sx1AL4LBO1GwZJBx6cakI37KTIJSuD3QJhwB7yfzKUOfSeOOgugBdPrLV2R+3PJ6hMCC/5f/QKQQ6eekEw8VPfSLaKWbAZKEO6Lq1FKtCTWXWTgFT25R4BafUKizW2blLBJvLuP1FHnRn57ogfAykTy/tRoeMUhyhGQcO6RyVZwA4HxVjAf2f6G56vnkDNHwjoNBzcHMr4FkNGyrJUTXMLoFWz8ezGr0JqU7obx0WDy93/wlDB7AncRUXSZxTr+C72et3nZTMh+7gABQf+6HpYTkj7ftLJbscB4QM9P4JXUu+9lJCYmHlwKpL7JEYx0JisTFta/V0XHh8I0O0J+u3w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=uybtuNqwicTm0XbF9JMzhofiQJwwSLgnTfRwNgDzpwk=;
 b=PhzZBH9/c8yE3VslDgr7mV25YmBZ7X9XrIjiP1GA/yHelf2fQHwpkpCiCnojbcNJl/vgxuPQAxivpldfulb5b/unCkR9WDvJtSAbWeoN6f6U1Fnaf75uWOn51oM6vxc4W0h2j7rFw98o6PMa5ubFMJHQhuXTYH/V+DPDfwpPdB8QoPliCyTDmz7wY6cbGzVBxAvFK/+jcX8k/71WXHj1xJ8fAgkSFUQEkY9kXf6032/fQgy+QhWiabqRCZsswyVLpL707eZ0lZ4X+FnkuGD3UZ/A9628zfyT6l2STqv2lQ/N+QvedLpArr6YSoCYv/ClskeSXaMZCGjWvNSSZUtPlw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=uybtuNqwicTm0XbF9JMzhofiQJwwSLgnTfRwNgDzpwk=;
 b=FztXMqvYJnXHy+HtMeraZ/ie+ABHFoI1/ZMkEkG8UPDGhnCPkfQHJOYCzB/q7ZbmSd718brPX02CyfDP8/mlkXc3Vegjcy+ILhax+8GPrhIZU0YL3h7fA3Iw9s9NELyY5GFK+cyoyzlP+1gTizquFaYdjbASHUWw8OSIgEWiLU8=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <e7960c4c-8fe8-44c0-8bff-3349fe274e90@amd.com>
Date: Wed, 28 Feb 2024 20:50:08 +0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/common: Do not allocate magic pages 1:1 for direct
 mapped domains
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Anthony PERARD <anthony.perard@citrix.com>, Juergen Gross
	<jgross@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Michal Orzel <michal.orzel@amd.com>, "Volodymyr
 Babchuk" <Volodymyr_Babchuk@epam.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
	<roger.pau@citrix.com>, Alec Kwapis <alec.kwapis@medtronic.com>
References: <20240226011935.169462-1-xin.wang2@amd.com>
 <d1518124-483c-4409-9b36-6a3392378911@xen.org>
 <a84aeb87-17e8-4195-90cb-7b0123064106@amd.com>
 <3982ba47-6709-47e3-a9c2-e2d3b4a2d8e3@xen.org>
 <8e56b75d-10a2-4764-8244-3e596f9fc695@amd.com>
 <98739f8d-dc00-4db8-a0b2-07f054107acf@xen.org>
Content-Language: en-US
From: Henry Wang <xin.wang2@amd.com>
In-Reply-To: <98739f8d-dc00-4db8-a0b2-07f054107acf@xen.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate
 permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA4D:EE_|SA1PR12MB5660:EE_
X-MS-Office365-Filtering-Correlation-Id: 00f58296-07e1-44be-3d4d-08dc385bcd92
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	692EEDhQGkK7u0POqEe/vU6Ia1TxHBVuIcFb6TGjk3uPn8bsDlcywrjO0DNLVlRvVKJfdCIZPrc+lN5I320KQz7/4X+j1Kivrtc2+3D8xxO53L7m/gaXEm88IMuRDWCiTuVoeanjRnRVejDbcXzt5vUnMBlvG2fbpjw/KGhhDXSg6Sc/iJICoBXXLQP73jOl4jUka6uLIS5CizeH/ZSwFLUu/1omXNNgcFo9l0lynU8wmnvF7t0c+tbnXA5xlFdUhuUfx8vVN5UvxwSdgBucyZl6+Yxdok7zyn6URhoQCTCHKY6Gzzck5GwQlokI6KMd9ZfJny8uSCNOYM1HiwR1J2DExKJvWUpsc/4fhG1TscVkIV4SNRHKjm2xvVwK7uc+e7iwhnG0Gvo1iPOF2lX0qBjPUIkh8Akkm0vpjDtVwruxbmSOc5b9JmiKv/gCAQscy6XbiAP+lF9+x5bjlXiatOdRWCPTzqDCuQFPskmsyDhUTVVoZvYx5Wl1DQCvdJhhrQWPGcS3e6R3kVxuOYe8IyffmElWDPrAwFyRBzi1uOkyMe2mzp3uw4CT50aeUWpPjUR2rvsz1NTLAVKsAOUT0er7ddZSkbaYubFhetthnMfxpdp/PlkARLPBsNmwYIFyy9UMxMR5Ay8IgaqYTxlDUmB3/Loc9twnGDbap06OBI8NeK8p8orjy/nohPYEwQC7z4TpOmzhd0i/4dQ/UZpW9jSOc0gFMiA5zNgID9Yf3MMTv7DStg8qzdqClRq0GMIa
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(230273577357003)(82310400014)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2024 12:50:12.1391
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 00f58296-07e1-44be-3d4d-08dc385bcd92
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF0002BA4D.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB5660

Hi Julien,

On 2/28/2024 8:27 PM, Julien Grall wrote:
> Hi Henry,
>>>> After checking the code flow, below rough plan came to my mind, I 
>>>> think what we need to do is:
>>>>
>>>> (1) Find a range of un-used memory using similar method in 
>>>> find_unallocated_memory()/find_domU_holes()
>>>
>>> AFAIK, the toolstack doesn't have any knowledge of the memeory 
>>> layout for dom0less domUs today. We would need to expose it first.
>>
>> If I understand correctly, I think the issue you mentioned here and ...
>>
>>> Then the region could either be statically allocated (i.e. the admin 
>>> provides it in the DTB) or dynamically.
>>>
>>>> (2) Change the base address, i.e. GUEST_MAGIC_BASE in 
>>>> alloc_xs_page() in init-dom0less.c to point to the address in (1) 
>>>> if static mem or 11 directmap. (I think this is a bit tricky 
>>>> though, do you have any method that in your mind?)
>>>
>>> AFAIK, the toolstack doesn't know whether a domain is direct mapped 
>>> or using static mem.
>>
>> ...here basically means we want to do the finding of the unused 
>> region in toolstack. Since currently what we care about is only a 
>> couple of pages instead of the whole memory map, could it be possible 
>> that we do the opposite: in alloc_xs_page(), we issue a domctl 
>> hypercall to Xen and do the finding work in Xen and return with the 
>> found gfn? Then the page can be mapped by populate_physmap() from 
>> alloc_xs_page() and used for XenStore.
>
> I know that DOMCTL hypercalls are not stable. But I am not overly 
> happy with creating an hypercall which is just "fetch the magic 
> regions". I think it need to be a more general one that would expose 
> all the regions.
>
> Also, you can't really find the magic regions when the hypercall is 
> done as we don't have the guest memory layout. This will need to be 
> done in advance.
>
> Overall, I think it would be better if we provide an hypercall listing 
> the regions currently occupied (similar to e820). One will have the 
> type "magic pages".

Yeah now it is more clear. I agree your approach is indeed a lot better. 
I will check how e820 works and see if I can do something similar. Also 
it might not be related, I think we somehow had a similar discussion in 
[1] when I do static heap.

>> If above approach makes sense to you, I have a further question: 
>> Since I understand that the extended region is basically for safely 
>> foreign mapping pages
>
> This is not about safety. The extended region is optional. It was 
> introduced so it is easy for Linux to find an unallocated region to 
> map external pages (e.g. vCPU shared info) so it doesn't waste RAM pages.
>
> , and init_dom0less.c uses foreign memory map for this
>> XenStore page, should we find the wanted page in the extended region? 
>> or even extended region should be excluded?
>
> How is the extended region found for dom0less domUs today? 

The extended regions for dom0less domUs are found by function 
find_domU_holes() introduced by commit 2a24477 xen/arm: implement domU 
extended regions. I think this commit basically followed the original 
algorithm introduced by commit 57f8785 libxl/arm: Add handling of 
extended regions for DomU.

> It would be fine to steal some part of the extended regions for the 
> magic pages. But they would need to be reserved *before* the guest is 
> first unpaused.

I also thought this today, as I think writing a function basically doing 
the same as what we do for extended regions is probably too much, so 
stealing some part of the extended region is easier. What I worry about 
is that: Once the extended regions are allocated and written to the 
"reg" property of the device tree hypervisor node, the data structures 
to record these extended regions are freed. So we kind of "lost" the 
information about these extended regions if we want to get them later 
(for example my use case). Also, if we still some part of memory from 
the extended regions, should we carve out the "stolen" parts from the 
device tree as well?

>>>>> Also, why are you only checking the first GFN? What if the caller 
>>>>> pass an overlapped region?
>>>>
>>>> I am a bit confused. My understanding is at this point we are 
>>>> handling one page at a time.
>>>
>>> We are handling one "extent" at the time. This could be one or 
>>> multiple pages (see extent_order).
>>
>> I agree, sorry I didn't express myself well. For this specific 
>> XenStore page, I think the extent_order is
>> fixed as 0 so there is only 1 page.
>
> Correct. But you should not rely on this :).

Yeah definitely :)

[1] 
https://lore.kernel.org/xen-devel/e53601a1-a5ac-897a-334d-de45d96e9863@xen.org/

Kind regards,
Henry



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 12:58:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 12:58:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686594.1068788 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJW0-0002mh-Lq; Wed, 28 Feb 2024 12:58:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686594.1068788; Wed, 28 Feb 2024 12:58:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJW0-0002ma-IR; Wed, 28 Feb 2024 12:58:28 +0000
Received: by outflank-mailman (input) for mailman id 686594;
 Wed, 28 Feb 2024 12:58:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfJW0-0002mR-0G
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 12:58:28 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0f78c292-d639-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 13:58:25 +0100 (CET)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-56647babfe6so2314123a12.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 04:58:25 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ij13-20020a056402158d00b00565ba75a739sm1726443edb.95.2024.02.28.04.58.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 04:58:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f78c292-d639-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709125105; x=1709729905; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=5A/wWjF3vVLCALEoYlfcnrvEaYFldWBjh5ULoDsqBgA=;
        b=AdphRkkmjky7+Ch94nalOgWjt6rz+FHlYruHRzMIz+TpEepoUmzAW1MxifpA6S0zd2
         39Dy7K03dLkG/CbCH6Z2GruOu8kxQnuHY660A1TwNq0GK/sruZ7FZ0MgbfDSkCai0rM0
         2QPA7xOjRMo/BTvp+zWhpCW9pR7dC7NQIkp5c1foSoAMH4XHZQi50kuLvm6r+4pj4zAC
         b9RxI3PGgfykcHw7bZv1reDwS1PwKhXJNgJvBwio0yTqHzcW1gE59nOhS4twyhSmV3Ye
         izu6x5q8sgQ0duUUoAHso/kirDwoKy4sAhwkilqvC7wpZJqlZ9F9/ejXDJ+dXUT1MQSw
         I9og==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709125105; x=1709729905;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5A/wWjF3vVLCALEoYlfcnrvEaYFldWBjh5ULoDsqBgA=;
        b=PpL1Zx8JGBxranXIYcVVsB29P5uBN6tR0azYzBSlhLt0yfs1dY/tQascsSNm1Ss4JG
         fnfO8KsCSbHsUkzXLJ7nuCC00bN1rHqQNPz91EtLjG2uqunnbEWJAIScxM+HeqY+Reiv
         n4Z4QF1mUYc+M44oZl908QR8WM4CzqJ8xE7OEAV4U2YJI/zeStR0g+1euICgdTBTDHhp
         nZ6pKu1595VdCOP3CTFFh1cEJhg6bME4S9H9qFOp0bjTGoJsO2KBpeyKMRyTONL3GMVo
         Fo8LGSlu9mPf1gmVGePRx15XFXg93DYywNJg5+FL1bvCknSVRKE9TxsdRSMwqBa1Xrow
         T5SQ==
X-Forwarded-Encrypted: i=1; AJvYcCUqIn3JIAhgdAiUj/xfP9QnfUIieMYJNq6NM7GyzB87oPNduja7yS7YIwrfdAHpU3CLlOAda2Tgx0+uph3at67VOWtt3YiYZMzq6FvX1Yc=
X-Gm-Message-State: AOJu0YxfXeCWsEXpyD9NejOA3s3zoMagnBCqs7C1UkSsvhA8iFhAoGme
	4bNVylyZsclCE2c7aqaoRNQ/N6Q6VCtc0sxB0aGtj5QdDz8sCVPZRx73F/H4Hw==
X-Google-Smtp-Source: AGHT+IFMFeP6iaUs4q//IP3HDemMlPP+qy5KQbMyMvrtv40xQGp/eYwdFZAyYdIRDCARVyij+itmzA==
X-Received: by 2002:a05:6402:1487:b0:565:7ce5:abd5 with SMTP id e7-20020a056402148700b005657ce5abd5mr8378817edv.18.1709125105251;
        Wed, 28 Feb 2024 04:58:25 -0800 (PST)
Message-ID: <5a4e8583-7747-4f24-94f0-d040dabb5b04@suse.com>
Date: Wed, 28 Feb 2024 13:58:24 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: preparations for 4.18.1
Content-Language: en-US
To: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
Cc: Wei Liu <wl@xen.org>, Anthony Perard <anthony.perard@citrix.com>,
 Kelly Choi <kelly.choi@cloud.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com>
 <1f251f2e-91de-4f81-a93c-dcb95746d2e5@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <1f251f2e-91de-4f81-a93c-dcb95746d2e5@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 28.02.2024 12:50, Julien Grall wrote:
> On 27/02/2024 13:19, Jan Beulich wrote:
>> All,
>>
>> the release is due in two to three weeks. Please point out backports you find
>> missing from the respective staging branch, but which you consider relevant.
> 
> For Arm:
> 
> e11f576650 ("xen/arm: Fix UBSAN failure in start_xen()")

Which I assume you or Stefano will take care of?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:04:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:04:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686597.1068798 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJbo-0004lF-AX; Wed, 28 Feb 2024 13:04:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686597.1068798; Wed, 28 Feb 2024 13:04:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJbo-0004l8-7h; Wed, 28 Feb 2024 13:04:28 +0000
Received: by outflank-mailman (input) for mailman id 686597;
 Wed, 28 Feb 2024 13:04:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfJbn-0004l2-Fz
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:04:27 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e5ddefb1-d639-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 14:04:25 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-5655c7dd3b1so1590576a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:04:25 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 jx11-20020a170906ca4b00b00a3f480154a3sm1828200ejb.65.2024.02.28.05.04.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 05:04:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e5ddefb1-d639-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709125465; x=1709730265; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=F4c1l7EwRkTAAhcVJbRO5x9wRcKnN4wAaQmkchFWJoQ=;
        b=IUWM8bN8myae8/2B1LLExvev0bgEpagLqxLr7HEAdUnhY4YpYX7+Ec2s808gnlFd/I
         BOOU/NI19UtgYVc4r+K9MRnTHj/sOQ77YBs8QU3XNRKDCLtzLqhkUSxgffc05KkWNZ82
         HVPViJfOrB/JnPaAijAogHtpgSlJrpgPzuesKJt4QdG5nYjk4jcjKAtqYP0pmnfvyMrD
         sdP8N/bnGveihHwdW64af1LU5eAjHDKVjiDN3M5n0PEawCer12Zyv3BndAjAWghsr4CX
         M7B8I3/RM1c8DtAsDROeXsw5kMUGQ1EVDPxKNYrBpyGzgo9bHUYgPAa5LNpjPoZ9X6S1
         394Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709125465; x=1709730265;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:cc:references
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=F4c1l7EwRkTAAhcVJbRO5x9wRcKnN4wAaQmkchFWJoQ=;
        b=WEEE3cHt1Ob8FSotDLGZ1hy7Kfb118NSeWxBaZ4XiTeMAs/yck4u8LVPEcuE3EHtJR
         Ci/H1OzCVX4qXW7N5qKFrAcUU/aNDiYrUD4G+3g8fuRTudv2ngTNolATzTru+dry4bqV
         X1kC8NW9P35OB85RoBZPTgkGSL8HDcPEZbBdjdwfUyhDOU6O0esb2u6KkGT/JQpTwoAr
         gG0qxeofK1hIkE1XieHgxZnGeco7Ak0DMvUJ8iEyKYPuWJKUzIm/gku+4oh6ekTcL1Lq
         tmkr+LcD1SE/EhOmKOv+WVJNc8rDKQupKY5pTQUz4ZVCMDMx1Ntozxh+4LSvzbog0g+V
         ejig==
X-Gm-Message-State: AOJu0YygD8TBpNxmSgYp71NbnMkkobMOdfnDIL3b/WvGO8B42KV0FiSi
	rom9HYkxYJE+Awr6cBagKY+eR7jcfPxKdWhh7grHxD9a64g/nNEAVrZitzFSGQ==
X-Google-Smtp-Source: AGHT+IFJ5qXRpw9LURj7SzE9CzWlF5Db2zX+HB/mIyRJvu85ptcaDkmOUAEoF5LAyypvRjqlQ4buBg==
X-Received: by 2002:a17:906:a05a:b0:a3e:69ff:141f with SMTP id bg26-20020a170906a05a00b00a3e69ff141fmr2127953ejb.33.1709125464856;
        Wed, 28 Feb 2024 05:04:24 -0800 (PST)
Message-ID: <b174757c-8463-4561-ae95-a5791d33080b@suse.com>
Date: Wed, 28 Feb 2024 14:04:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Fwd: New Defects reported by Coverity Scan for XenProject
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
References: <65df054dd791_2895d52c84469bd9989838e@prd-scan-dashboard-0.mail>
 <27fad7c4-905b-4181-9882-f383eb194032@citrix.com>
 <d81f78a8-3e04-4fba-95fc-3b602ee3a740@citrix.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Stewart Hildebrand <Stewart.Hildebrand@amd.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <d81f78a8-3e04-4fba-95fc-3b602ee3a740@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 28.02.2024 13:19, Andrew Cooper wrote:
> Take 2, hopefully with Stewart's correct email address this time.
> 
> ~Andrew
> 
> On 28/02/2024 12:17 pm, Andrew Cooper wrote:
>> Not sure how well this is going to be formatted, but there's one new and
>> potentially interesting issue found by Coverity.

To be honest I didn't consider this interesting at all, but instead a false
positive due to limited insight that the tool has. But maybe I was wrong
and you see something I didn't see? vpci_process_pending() is vCPU-local
(run from the guest resume path), and hence there simply are no two threads
who want to look at the field. Storing NULL into it is merely a kind of
progress indicator, relevant to the given vCPU only.

Jan

>> ----8<----
>>
>> New defect(s) Reported-by: Coverity Scan
>> Showing 1 of 1 defect(s)
>>
>>
>> ** CID 1592633: (LOCK_EVASION)
>> /xen/drivers/vpci/header.c: 229 in vpci_process_pending()
>> /xen/drivers/vpci/header.c: 189 in vpci_process_pending()
>> /xen/drivers/vpci/header.c: 239 in vpci_process_pending()
>>
>>
>> ________________________________________________________________________________________________________
>> *** CID 1592633: (LOCK_EVASION)
>> /xen/drivers/vpci/header.c: 229 in vpci_process_pending()
>> 223 224 /* Clean all the rangesets */
>> 225 for ( i = 0; i < ARRAY_SIZE(header->bars); i++ )
>> 226 if ( !rangeset_is_empty(header->bars[i].mem) )
>> 227 rangeset_purge(header->bars[i].mem);
>> 228
>>>>> CID 1592633: (LOCK_EVASION)
>>>>> Thread1 sets "pdev" to a new value. Now the two threads have an
>>>>> inconsistent view of "pdev" and updates to fields of "pdev" or
>>>>> fields correlated with "pdev" may be lost.
>> 229 v->vpci.pdev = NULL;
>> 230 231 read_unlock(&v->domain->pci_lock);
>> 232 233 if ( !is_hardware_domain(v->domain) )
>> 234 domain_crash(v->domain);
>> /xen/drivers/vpci/header.c: 189 in vpci_process_pending()
>> 183 return false;
>> 184 185 read_lock(&v->domain->pci_lock);
>> 186 187 if ( !pdev->vpci || (v->domain != pdev->domain) )
>> 188 {
>>>>> CID 1592633: (LOCK_EVASION)
>>>>> Thread1 sets "pdev" to a new value. Now the two threads have an
>>>>> inconsistent view of "pdev" and updates to fields of "pdev" or
>>>>> fields correlated with "pdev" may be lost.
>> 189 v->vpci.pdev = NULL;
>> 190 read_unlock(&v->domain->pci_lock);
>> 191 return false;
>> 192 }
>> 193 194 header = &pdev->vpci->header;
>> /xen/drivers/vpci/header.c: 239 in vpci_process_pending()
>> 233 if ( !is_hardware_domain(v->domain) )
>> 234 domain_crash(v->domain);
>> 235 236 return false;
>> 237 }
>> 238 }
>>>>> CID 1592633: (LOCK_EVASION)
>>>>> Thread1 sets "pdev" to a new value. Now the two threads have an
>>>>> inconsistent view of "pdev" and updates to fields of "pdev" or
>>>>> fields correlated with "pdev" may be lost.
>> 239 v->vpci.pdev = NULL;
>> 240 241 spin_lock(&pdev->vpci->lock);
>> 242 modify_decoding(pdev, v->vpci.cmd, v->vpci.rom_only);
>> 243 spin_unlock(&pdev->vpci->lock);
>> 244
>>
> 



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:27:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:27:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686609.1068810 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJxZ-00016U-87; Wed, 28 Feb 2024 13:26:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686609.1068810; Wed, 28 Feb 2024 13:26:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJxZ-00016N-5c; Wed, 28 Feb 2024 13:26:57 +0000
Received: by outflank-mailman (input) for mailman id 686609;
 Wed, 28 Feb 2024 13:26:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KqYS=KF=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rfJxY-00016H-1j
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:26:56 +0000
Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com
 [2607:f8b0:4864:20::f31])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 09f36609-d63d-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 14:26:55 +0100 (CET)
Received: by mail-qv1-xf31.google.com with SMTP id
 6a1803df08f44-6901114742bso9477176d6.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:26:55 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 mf2-20020a0562145d8200b0068f752195b5sm5245824qvb.86.2024.02.28.05.26.52
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 05:26:53 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09f36609-d63d-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709126814; x=1709731614; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=lCMMSK93bM8/azXpNgcJi8z64clOQqCgYjwRGy5nd+I=;
        b=ZP2DQkG8cWG0fIEGEH5nvn4x2oyn2j6EJJUcIV11NsyH/8YdJDP0KY/lXmdWAyrVof
         e+KCRRJ6PMCUltUiNKaxDGjec7Kpp5yAV2LpOX9jdEdC9dJkYOMWJhp85uHPUq8yVH/f
         dydFZqskfCaOcr7K/R9AVXlo23rcCSL4ilslg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709126814; x=1709731614;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=lCMMSK93bM8/azXpNgcJi8z64clOQqCgYjwRGy5nd+I=;
        b=pirJcD7qUYwC+nTZ5a6wpgjGmrZJDfqStpjxhYIc7h43jVzBLCdssIDAcKLJp7OPuV
         H7V3cNUlRLnUOwtbxDm+vccz17ctR4jkEJBFzps5Izy+/8QudsH16SXSLlvQZTvo+5+l
         iHlc/DhuzPz0mCVASyItKwE7IlOorTTJt6dij25YQpyBLfT9kKCvm5PFIBoWNEFYGfe6
         N6rrnC0vKdJVM7IXQuEUBhPhqz2Om7LikeyT75x5rCFUYEPVNQgCt+3BtBfyB9DEvZ01
         DCVImnrjKYZK4wrrpF0a2lzSfUvNplD7ShW03GfU/YQa7XfTdvttrDQBDNOXWtwJsRuR
         1L5w==
X-Forwarded-Encrypted: i=1; AJvYcCVV9fTB2xGwjcfhsSWeORvFn2bSVo0VtgqAg9Vbe9Ccu7k+4Y1yLpopirfGHJQwaWg1+P7JRknxXoL34yGpJWGVzQb5FAK9FJ9P0UEDxlo=
X-Gm-Message-State: AOJu0YyZ1q8Ix1yNm9vwlGlCnZ7v3oPPpKZsRUpjJ/JmTIvLj5dp0UvK
	+yNOeXaZBDL01qxvX8vOtDkWbHzx0VnhT6vAFCuHRCzb0ufuvt++wx+YuKXitRA=
X-Google-Smtp-Source: AGHT+IGlQSdmNSEnpcRaosxZdKOAXmTYrGh5veoT5xTz+bzJp6Zr+ZENrB1//t1TL+zWdmGI5oodcw==
X-Received: by 2002:a0c:9a08:0:b0:68f:441b:66ea with SMTP id p8-20020a0c9a08000000b0068f441b66eamr4306024qvd.26.1709126813874;
        Wed, 28 Feb 2024 05:26:53 -0800 (PST)
Date: Wed, 28 Feb 2024 14:26:51 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	xen-devel <xen-devel@lists.xenproject.org>,
	Stewart Hildebrand <stewart.hildebrand@amd.com>
Subject: Re: Fwd: New Defects reported by Coverity Scan for XenProject
Message-ID: <Zd80m6-4YNasktjg@macbook>
References: <65df054dd791_2895d52c84469bd9989838e@prd-scan-dashboard-0.mail>
 <27fad7c4-905b-4181-9882-f383eb194032@citrix.com>
 <d81f78a8-3e04-4fba-95fc-3b602ee3a740@citrix.com>
 <b174757c-8463-4561-ae95-a5791d33080b@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <b174757c-8463-4561-ae95-a5791d33080b@suse.com>

On Wed, Feb 28, 2024 at 02:04:23PM +0100, Jan Beulich wrote:
> On 28.02.2024 13:19, Andrew Cooper wrote:
> > Take 2, hopefully with Stewart's correct email address this time.
> > 
> > ~Andrew
> > 
> > On 28/02/2024 12:17 pm, Andrew Cooper wrote:
> >> Not sure how well this is going to be formatted, but there's one new and
> >> potentially interesting issue found by Coverity.
> 
> To be honest I didn't consider this interesting at all, but instead a false
> positive due to limited insight that the tool has. But maybe I was wrong
> and you see something I didn't see? vpci_process_pending() is vCPU-local
> (run from the guest resume path), and hence there simply are no two threads
> who want to look at the field. Storing NULL into it is merely a kind of
> progress indicator, relevant to the given vCPU only.

Indeed, there's no (intended) way for vpci_process_pending() to be
executed concurrently against the same vcpu parameter, and hence there
should be no concurrency hazards.  Setting it to NULL is a mere
indication there's no further work to be done, and the vCPU can resume
guest context execution.

defer_map() (the function that queues the work to be done) can only be
reached as a result of a guest accessing the PCI config space, so if
the vCPU is not running defer_map() can't be called either for that
vCPU.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:27:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:27:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686611.1068821 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJyW-0001a0-ID; Wed, 28 Feb 2024 13:27:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686611.1068821; Wed, 28 Feb 2024 13:27:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJyW-0001Zt-FC; Wed, 28 Feb 2024 13:27:56 +0000
Received: by outflank-mailman (input) for mailman id 686611;
 Wed, 28 Feb 2024 13:27:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eIhW=KF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfJyW-0001Zl-0f
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:27:56 +0000
Received: from mail-yb1-xb30.google.com (mail-yb1-xb30.google.com
 [2607:f8b0:4864:20::b30])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2db55e91-d63d-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 14:27:55 +0100 (CET)
Received: by mail-yb1-xb30.google.com with SMTP id
 3f1490d57ef6-dc6d9a8815fso5461187276.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:27:54 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 14-20020ac84e8e000000b0042e93e97957sm2103401qtp.34.2024.02.28.05.27.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 05:27:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2db55e91-d63d-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709126874; x=1709731674; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/x1UVeM+Spn+Uw0ix5uhbhckWEtPjuGgGbqoE9ALHhA=;
        b=RK/mXX+GmostfwdQksKhpNrnJYYQClVpGW8YfkWGp/9q7ST9QshbGImBgk/FHGX0XE
         2SSIz/51y6Tu47LoLXTlRwl3XDSxOiqoKAY3nLG0dL3v2zWApQaPFfLfw1K79sI/JAs9
         JTy9qu8qea2R2zEJ60hzJJI+qhDnDEveEqX8A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709126874; x=1709731674;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/x1UVeM+Spn+Uw0ix5uhbhckWEtPjuGgGbqoE9ALHhA=;
        b=EU/9bhK+FFvBqxOIQs6M2xvq9c4n9uBvpSzya7Vvc14a6CVxFdYGxkiNDlCL+9z8bx
         b32lGV784PbPsbsm/TKBaHz0wya8GT4suJ15qtTAR9meAugfevg4Z68ZnGFzGIwc3RjO
         Nu+B6x/ZHxoxUBXAuM18NQOqgrLgAkf2EzKxAVemDaA6gTWbJYmq4o5xxpySkCDxEfz4
         9K6zE2gt5ZLxkHbaSr8bHw9xONLD8hq6JuA38KO2huW0eNvQwAdlAj4EqhbftlP4sej5
         MVWM6xzkvunyo8pNuNg4fObZUGyd5uC3hCu83ox/Da85bkSm6808EYXX1EyxBHqXHpX7
         ljqg==
X-Gm-Message-State: AOJu0YwlToZLP//hl3fQ9XypcDKIwsIDr8T6kiadBzEP38epjVohLDs4
	IJzww/2pRvBkQBa3DOZp7NUqWxxwsw4DdogFkxRAZWW83fTFeYiN+6s4xkEVLu0=
X-Google-Smtp-Source: AGHT+IHCjEIULgMEYLdOP0vFGEkvTd5GcPJpU89aTVwslm6VhKqryDBxW2eBjW2dy7sYeelgp+Y2zQ==
X-Received: by 2002:a25:6b11:0:b0:dc6:d738:1fa6 with SMTP id g17-20020a256b11000000b00dc6d7381fa6mr2665967ybc.6.1709126873836;
        Wed, 28 Feb 2024 05:27:53 -0800 (PST)
Message-ID: <2acbc687-26fd-40bd-a74e-f17b2a125082@citrix.com>
Date: Wed, 28 Feb 2024 13:27:51 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Fwd: New Defects reported by Coverity Scan for XenProject
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Jan Beulich <jbeulich@suse.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>,
 Stewart Hildebrand <stewart.hildebrand@amd.com>
References: <65df054dd791_2895d52c84469bd9989838e@prd-scan-dashboard-0.mail>
 <27fad7c4-905b-4181-9882-f383eb194032@citrix.com>
 <d81f78a8-3e04-4fba-95fc-3b602ee3a740@citrix.com>
 <b174757c-8463-4561-ae95-a5791d33080b@suse.com> <Zd80m6-4YNasktjg@macbook>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <Zd80m6-4YNasktjg@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/02/2024 1:26 pm, Roger Pau Monné wrote:
> On Wed, Feb 28, 2024 at 02:04:23PM +0100, Jan Beulich wrote:
>> On 28.02.2024 13:19, Andrew Cooper wrote:
>>> Take 2, hopefully with Stewart's correct email address this time.
>>>
>>> ~Andrew
>>>
>>> On 28/02/2024 12:17 pm, Andrew Cooper wrote:
>>>> Not sure how well this is going to be formatted, but there's one new and
>>>> potentially interesting issue found by Coverity.
>> To be honest I didn't consider this interesting at all, but instead a false
>> positive due to limited insight that the tool has. But maybe I was wrong
>> and you see something I didn't see? vpci_process_pending() is vCPU-local
>> (run from the guest resume path), and hence there simply are no two threads
>> who want to look at the field. Storing NULL into it is merely a kind of
>> progress indicator, relevant to the given vCPU only.
> Indeed, there's no (intended) way for vpci_process_pending() to be
> executed concurrently against the same vcpu parameter, and hence there
> should be no concurrency hazards.  Setting it to NULL is a mere
> indication there's no further work to be done, and the vCPU can resume
> guest context execution.
>
> defer_map() (the function that queues the work to be done) can only be
> reached as a result of a guest accessing the PCI config space, so if
> the vCPU is not running defer_map() can't be called either for that
> vCPU.

Fine.  So long as someone has looked into it and thinks we're fine.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:28:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:28:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686613.1068831 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJz3-00026Y-R8; Wed, 28 Feb 2024 13:28:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686613.1068831; Wed, 28 Feb 2024 13:28:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfJz3-00026Q-OG; Wed, 28 Feb 2024 13:28:29 +0000
Received: by outflank-mailman (input) for mailman id 686613;
 Wed, 28 Feb 2024 13:28:28 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KqYS=KF=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rfJz2-00026I-AK
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:28:28 +0000
Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com
 [2607:f8b0:4864:20::730])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4106a726-d63d-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 14:28:27 +0100 (CET)
Received: by mail-qk1-x730.google.com with SMTP id
 af79cd13be357-787bb013422so379896285a.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:28:27 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 f3-20020a05620a12e300b00787f6064a9fsm129781qkl.108.2024.02.28.05.28.25
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 05:28:25 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4106a726-d63d-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709126906; x=1709731706; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=0j5MyD9s/GyIrST96PYBQhHX3gFeuvn0TYPadHUmof8=;
        b=m4WIwGzTbAawjF/fdxE5tnTXqlLhGUZa0KT0iBSJ9s1sFxZax0T59fDCsmz7TD6u29
         b6KtMU1UtL8tSk0RpnynS4SWPW4JjFx5A+rzorgerG/W8zVeBiEJ/rBGt1H9IBlH9nwx
         D7jfhm5QBdo6nQlRhtv9Lg/ANwlBePUz7aVHs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709126906; x=1709731706;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=0j5MyD9s/GyIrST96PYBQhHX3gFeuvn0TYPadHUmof8=;
        b=IjJMBVhkEEyfbrLLHcrOXzBgw2/MzV2BGB66fzh0hUXVBTbKwWyrLTfGvw3BH8UWct
         rmJxccq1RCP3UyNLHekV8Xj15TCbhvOQL7bYcErh4PAXhVVXkC5kjLlPHAA2HmmHC7Ki
         c1NMzCLPQyWh7n0z8t0P6ub88Rfr4y2AmDxVWYD7/6inexE18XNxs4gGcKgOrbyu8E8x
         FGyuxK3K3OX2PYogrSarxnuZspQMEcvocG+FvzbNLE85UsNwZ94st2sresqNdE/dZCMH
         hCw0SXcLefS0i9y4ynAXtcsYt8IFGcUz2tO6ff0+ipSiIpdTEVxJFbP+xXRUGuo16UWu
         JYfQ==
X-Forwarded-Encrypted: i=1; AJvYcCUe0tqdev402uAnzwck8fAgff0fNNV2l9Yq1FRs7AWrxygm+gNZ8H+V4GwqCewu7I8NxxDiM47EZMDJP8HdnjA0vd28v6J7rrs58UddaBw=
X-Gm-Message-State: AOJu0YzzyyWoVB8AnrbBsDqXR6oKAkn7Joy3NdbOyqwkqX2QxsQZ9QMf
	WVQSyzxWahxGKzEZCMBX3Ti8KCiqV1vJtV76pY0bxJXtOydlsV41OKs2azKVZ+A=
X-Google-Smtp-Source: AGHT+IEY+IvmPSEnzKDGh0+zU9pOU1Ev1g4pS0B9K9aQYRcoHO0Q07QU/LgXvtvoAOcOZ6Wz1A20hw==
X-Received: by 2002:a05:620a:2f2:b0:787:a7c1:9d03 with SMTP id a18-20020a05620a02f200b00787a7c19d03mr4635217qko.69.1709126906248;
        Wed, 28 Feb 2024 05:28:26 -0800 (PST)
Date: Wed, 28 Feb 2024 14:28:24 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Tamas K Lengyel <tamas@tklengyel.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH 1/2] x86/memsharing: use an atomic add instead of a
 cmpxchg loop
Message-ID: <Zd80-IGn13aThDaQ@macbook>
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-2-roger.pau@citrix.com>
 <d8939069-a1cb-4794-a6aa-75a3e4d97884@suse.com>
 <CABfawhkhH7Pakkq+RgjfmW-f60CbOKxr6wWxvBSr2AkN+m1OdQ@mail.gmail.com>
 <2f68276d-0b10-4cec-8636-5456314c194e@suse.com>
 <Zd8QtCwp75Phjluc@macbook>
 <de7c8de8-2373-4d78-8aec-c633bb253660@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <de7c8de8-2373-4d78-8aec-c633bb253660@suse.com>

On Wed, Feb 28, 2024 at 12:18:31PM +0100, Jan Beulich wrote:
> On 28.02.2024 11:53, Roger Pau Monné wrote:
> > On Fri, Feb 23, 2024 at 08:43:24AM +0100, Jan Beulich wrote:
> >> On 22.02.2024 19:03, Tamas K Lengyel wrote:
> >>> On Thu, Feb 22, 2024 at 5:06 AM Jan Beulich <jbeulich@suse.com> wrote:
> >>>> On 22.02.2024 10:05, Roger Pau Monne wrote:
> >>>>> The usage of a cmpxchg loop in get_next_handle() is unnecessary, as the same
> >>>>> can be achieved with an atomic increment, which is both simpler to read, and
> >>>>> avoid any need for a loop.
> >>>>>
> >>>>> The cmpxchg usage is likely a remnant of 32bit support, which didn't have an
> >>>>> instruction to do an atomic 64bit add, and instead a cmpxchg had to be used.
> >>>>>
> >>>>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >>>>> Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>
> >>>>
> >>>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> >>>> albeit ...
> >>>>
> >>>>> --- a/xen/arch/x86/mm/mem_sharing.c
> >>>>> +++ b/xen/arch/x86/mm/mem_sharing.c
> >>>>> @@ -179,13 +179,7 @@ static void mem_sharing_page_unlock(struct page_info *pg)
> >>>>>
> >>>>>  static shr_handle_t get_next_handle(void)
> >>>>>  {
> >>>>> -    /* Get the next handle get_page style */
> >>>>> -    uint64_t x, y = next_handle;
> >>>>> -    do {
> >>>>> -        x = y;
> >>>>> -    }
> >>>>> -    while ( (y = cmpxchg(&next_handle, x, x + 1)) != x );
> >>>>> -    return x + 1;
> >>>>> +    return arch_fetch_and_add(&next_handle, 1) + 1;
> >>>>>  }
> >>>>
> >>>> ... the adding of 1 here is a little odd when taken together with
> >>>> next_handle's initializer. Tamas, you've not written that code, but do
> >>>> you have any thoughts towards the possible removal of either the
> >>>> initializer or the adding here? Plus that variable of course could
> >>>> very well do with moving into this function.
> >>>
> >>> I have to say I find the existing logic here hard to parse but by the
> >>> looks I don't think we need the + 1 once we switch to
> >>> arch_fetch_and_add. Also could go without initializing next_handle to
> >>> 1. Moving it into the function would not really accomplish anything
> >>> other than style AFAICT?
> >>
> >> Well, limiting scope of things can be viewed as purely style, but I
> >> think it's more than that: It makes intentions more clear and reduces
> >> the chance of abuse (deliberate or unintentional).
> > 
> > I'm afraid that whatever is the outcome here, I will defer it to a
> > further commit, since the purpose here is to be a non-functional
> > change.
> 
> That's fine with me, but an ack from Tamas is still pending, unless I
> missed something somewhere.

No, just wanted to clarify that I wasn't expecting to do further
changes here, FTAOD.  Not sure if Tamas was expecting me to further
adjust the code.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:39:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:39:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686622.1068844 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfK9O-0004iZ-RZ; Wed, 28 Feb 2024 13:39:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686622.1068844; Wed, 28 Feb 2024 13:39:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfK9O-0004iS-O1; Wed, 28 Feb 2024 13:39:10 +0000
Received: by outflank-mailman (input) for mailman id 686622;
 Wed, 28 Feb 2024 13:39:09 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=Darz=KF=tklengyel.com=tamas@srs-se1.protection.inumbo.net>)
 id 1rfK9N-0004h0-4y
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:39:09 +0000
Received: from sender4-op-o12.zoho.com (sender4-op-o12.zoho.com
 [136.143.188.12]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bd585abe-d63e-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 14:39:06 +0100 (CET)
Received: from mail-yb1-f178.google.com (mail-yb1-f178.google.com
 [209.85.219.178]) by mx.zohomail.com
 with SMTPS id 1709127540597642.1284075319935;
 Wed, 28 Feb 2024 05:39:00 -0800 (PST)
Received: by mail-yb1-f178.google.com with SMTP id
 3f1490d57ef6-dc6d9a8815fso5471275276.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:39:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bd585abe-d63e-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; t=1709127542; cv=none; 
	d=zohomail.com; s=zohoarc; 
	b=LRyqE3j19UwntIfaJ0HJtX/y9o9/XX4vm0GOuH/jnMgSp+V2WpvcX4NpOqFDZ5AVZR4T5hFPmJAy7Xx+sS/rJTkJeLp8ljTiwVPnUpN0Zf428NU1ikNiYfWkesv0rpOZcifPZA14bXQ5P0FvuHqaLOEQJEhadBEqAMccj2wYWN8=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; 
	t=1709127542; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; 
	bh=60avIusnchtcXE++c0JOpCawy+/7NK5IEamagVfQABI=; 
	b=fWBnqHRsOOxuf9STGEkr4WqxyVAsvb5sjftd2jnO9HNpjQwS40lHZVkOJ4FhcnF3ALOIaJjCzNNGblTsAqveK/gM2k0iacA+00IDejzkhmoj32XoCmVqza0NEQfrGJlGj+bewycFl/ay4dUrV6TXI4DweZJkQR5dSFf+IBfCGpk=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass  header.i=tklengyel.com;
	spf=pass  smtp.mailfrom=tamas@tklengyel.com;
	dmarc=pass header.from=<tamas@tklengyel.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1709127542;
	s=zmail; d=tklengyel.com; i=tamas@tklengyel.com;
	h=MIME-Version:References:In-Reply-To:From:From:Date:Date:Message-ID:Subject:Subject:To:To:Cc:Cc:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To;
	bh=60avIusnchtcXE++c0JOpCawy+/7NK5IEamagVfQABI=;
	b=O5a/xxDkSbJO1DdruZjFXBNTTSqpN+YyO4WdrY/wdysa8zqifamfssGKDSLD86sM
	VnsXuwTEt//UEkUk5Af/4EJzQBN9qMYulD5lw+mxRLj1oKfzZ5dtypJbOE+qUXHB5Qm
	vHgD+M3HxfraFtykVSXaqymNyFPxSog2/t1UIiJs=
X-Forwarded-Encrypted: i=1; AJvYcCW63K9G9FwKWpIex+ZmKoX0JbaIZB/20mXSnqb/Ww6oYqpxGBbH3h0jRoBoVYZbJyEIj7YSNxrryv5g02MK2a45lq7fo7PsUN9OFwXW6mM=
X-Gm-Message-State: AOJu0YzlnaAeZIiO2VNE+eiTk3+gCZFiZc8z+9sPHa0cA5dc9QcRT9J6
	QmlsYP1Maj9qMRID3HDmIt6GgdiVniJX86A5dV8gcj4j3X3MedexkNNb5cEjfAgibd6Oq4CPCIL
	qSgpuDDDHAr8RW2jlcnhtqTX+JCI=
X-Google-Smtp-Source: AGHT+IHKWmkwUHqD2Slm7SyWIP+2fb3JCb6sAGlxQhpm3o3Ec5tJkSmXIjZ7DFZSzGZsvxBpRagGlBYz7zbpW02wXtY=
X-Received: by 2002:a5b:54a:0:b0:dcf:30dc:127c with SMTP id
 r10-20020a5b054a000000b00dcf30dc127cmr2543195ybp.18.1709127539658; Wed, 28
 Feb 2024 05:38:59 -0800 (PST)
MIME-Version: 1.0
References: <20240222090530.62530-1-roger.pau@citrix.com> <20240222090530.62530-2-roger.pau@citrix.com>
 <d8939069-a1cb-4794-a6aa-75a3e4d97884@suse.com> <CABfawhkhH7Pakkq+RgjfmW-f60CbOKxr6wWxvBSr2AkN+m1OdQ@mail.gmail.com>
 <2f68276d-0b10-4cec-8636-5456314c194e@suse.com> <Zd8QtCwp75Phjluc@macbook>
 <de7c8de8-2373-4d78-8aec-c633bb253660@suse.com> <Zd80-IGn13aThDaQ@macbook>
In-Reply-To: <Zd80-IGn13aThDaQ@macbook>
From: Tamas K Lengyel <tamas@tklengyel.com>
Date: Wed, 28 Feb 2024 08:38:23 -0500
X-Gmail-Original-Message-ID: <CABfawhnhT2ppikq9E9xVUgrXw6+DgmYTfBTRBi8b3-i4ZNRG=w@mail.gmail.com>
Message-ID: <CABfawhnhT2ppikq9E9xVUgrXw6+DgmYTfBTRBi8b3-i4ZNRG=w@mail.gmail.com>
Subject: Re: [PATCH 1/2] x86/memsharing: use an atomic add instead of a
 cmpxchg loop
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>, 
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 28, 2024 at 8:28=E2=80=AFAM Roger Pau Monn=C3=A9 <roger.pau@cit=
rix.com> wrote:
>
> On Wed, Feb 28, 2024 at 12:18:31PM +0100, Jan Beulich wrote:
> > On 28.02.2024 11:53, Roger Pau Monn=C3=A9 wrote:
> > > On Fri, Feb 23, 2024 at 08:43:24AM +0100, Jan Beulich wrote:
> > >> On 22.02.2024 19:03, Tamas K Lengyel wrote:
> > >>> On Thu, Feb 22, 2024 at 5:06=E2=80=AFAM Jan Beulich <jbeulich@suse.=
com> wrote:
> > >>>> On 22.02.2024 10:05, Roger Pau Monne wrote:
> > >>>>> The usage of a cmpxchg loop in get_next_handle() is unnecessary, =
as the same
> > >>>>> can be achieved with an atomic increment, which is both simpler t=
o read, and
> > >>>>> avoid any need for a loop.
> > >>>>>
> > >>>>> The cmpxchg usage is likely a remnant of 32bit support, which did=
n't have an
> > >>>>> instruction to do an atomic 64bit add, and instead a cmpxchg had =
to be used.
> > >>>>>
> > >>>>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > >>>>> Signed-of-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> > >>>>
> > >>>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> > >>>> albeit ...
> > >>>>
> > >>>>> --- a/xen/arch/x86/mm/mem_sharing.c
> > >>>>> +++ b/xen/arch/x86/mm/mem_sharing.c
> > >>>>> @@ -179,13 +179,7 @@ static void mem_sharing_page_unlock(struct p=
age_info *pg)
> > >>>>>
> > >>>>>  static shr_handle_t get_next_handle(void)
> > >>>>>  {
> > >>>>> -    /* Get the next handle get_page style */
> > >>>>> -    uint64_t x, y =3D next_handle;
> > >>>>> -    do {
> > >>>>> -        x =3D y;
> > >>>>> -    }
> > >>>>> -    while ( (y =3D cmpxchg(&next_handle, x, x + 1)) !=3D x );
> > >>>>> -    return x + 1;
> > >>>>> +    return arch_fetch_and_add(&next_handle, 1) + 1;
> > >>>>>  }
> > >>>>
> > >>>> ... the adding of 1 here is a little odd when taken together with
> > >>>> next_handle's initializer. Tamas, you've not written that code, bu=
t do
> > >>>> you have any thoughts towards the possible removal of either the
> > >>>> initializer or the adding here? Plus that variable of course could
> > >>>> very well do with moving into this function.
> > >>>
> > >>> I have to say I find the existing logic here hard to parse but by t=
he
> > >>> looks I don't think we need the + 1 once we switch to
> > >>> arch_fetch_and_add. Also could go without initializing next_handle =
to
> > >>> 1. Moving it into the function would not really accomplish anything
> > >>> other than style AFAICT?
> > >>
> > >> Well, limiting scope of things can be viewed as purely style, but I
> > >> think it's more than that: It makes intentions more clear and reduce=
s
> > >> the chance of abuse (deliberate or unintentional).
> > >
> > > I'm afraid that whatever is the outcome here, I will defer it to a
> > > further commit, since the purpose here is to be a non-functional
> > > change.
> >
> > That's fine with me, but an ack from Tamas is still pending, unless I
> > missed something somewhere.
>
> No, just wanted to clarify that I wasn't expecting to do further
> changes here, FTAOD.  Not sure if Tamas was expecting me to further
> adjust the code.

Acked-by: Tamas K Lengyel <tamas@tklengyel.com>


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:41:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:41:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686626.1068855 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKB4-0006fY-5c; Wed, 28 Feb 2024 13:40:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686626.1068855; Wed, 28 Feb 2024 13:40:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKB4-0006fR-2Z; Wed, 28 Feb 2024 13:40:54 +0000
Received: by outflank-mailman (input) for mailman id 686626;
 Wed, 28 Feb 2024 13:40:52 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfKB2-0006eF-IY
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:40:52 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfKAz-00039Y-UJ; Wed, 28 Feb 2024 13:40:49 +0000
Received: from [15.248.2.225] (helo=[10.45.19.69])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfKAz-0005Jn-MM; Wed, 28 Feb 2024 13:40:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=SVUINBDj6t3rhaNpNflDWIemUXsHKdOAr0yeyGKyma4=; b=nOgVCfS4SAJ8BQDGglm7YG8mVK
	sbUI4mhhl3I8QvhwwH6cKqK/tfTDOI/mdm7kFykVy0W9PUfGyochlfCT33aWoiT7cH5Gxj/2DCzrP
	HTNHDAOvscui+tulEa1Ln5V4v/wznt3vgH7WMR28R6v7SjiC3c338OvCKX87l5Ks46nw=;
Message-ID: <521a4d3b-8ef1-4449-97bd-1a3d3e0e35ec@xen.org>
Date: Wed, 28 Feb 2024 13:40:48 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: preparations for 4.18.1
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: Wei Liu <wl@xen.org>, Anthony Perard <anthony.perard@citrix.com>,
 Kelly Choi <kelly.choi@cloud.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com>
 <1f251f2e-91de-4f81-a93c-dcb95746d2e5@xen.org>
 <5a4e8583-7747-4f24-94f0-d040dabb5b04@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <5a4e8583-7747-4f24-94f0-d040dabb5b04@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 28/02/2024 12:58, Jan Beulich wrote:
> On 28.02.2024 12:50, Julien Grall wrote:
>> On 27/02/2024 13:19, Jan Beulich wrote:
>>> All,
>>>
>>> the release is due in two to three weeks. Please point out backports you find
>>> missing from the respective staging branch, but which you consider relevant.
>>
>> For Arm:
>>
>> e11f576650 ("xen/arm: Fix UBSAN failure in start_xen()")
> 
> Which I assume you or Stefano will take care of?

I was expecting Stefano would do it as he did the backports in the past.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:51:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:51:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686634.1068865 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKKj-00009S-72; Wed, 28 Feb 2024 13:50:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686634.1068865; Wed, 28 Feb 2024 13:50:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKKj-00009L-3n; Wed, 28 Feb 2024 13:50:53 +0000
Received: by outflank-mailman (input) for mailman id 686634;
 Wed, 28 Feb 2024 13:50:52 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfKKi-00009D-9W
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:50:52 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 61de7fb9-d640-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 14:50:50 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-563c403719cso6918096a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:50:51 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 i2-20020a0564020f0200b00565efe074f4sm1794127eda.85.2024.02.28.05.50.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 05:50:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 61de7fb9-d640-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709128250; x=1709733050; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=6gLhTWaeA6K8LvtOWvt7viO6z/8krLAbtiOifZaThhE=;
        b=TVi2a1bxTWJgJ5+853KE5vL/R53OsK7MLIsilsohma0pQhiXxVb605Z+7myFDByo35
         55QSpGGJGkxo7ETDgu82XdKTNoqa08fnIPsyXaOX/yRY/DVmb7QbzTcFUUAqiObd7Uyq
         B2l78WwSW/z3yD/fqVWK/lwcBdbYIA/DP3yCRQjsNf1AbbCDw/O6Flx//3kYmrr2xoQw
         hktRIupX+nOqyPhjVDCYn7m4zr8TK6R/YGIT4p33IVpF/4vsarPP5ug2hagh9raOzDpT
         d8eZfDcQZnEGgXSY07dYskZDsgJwYiHsF2FIkQ/e99Mdxj0Zzkq9cXtx6V3Zkk04JdGM
         swMg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709128250; x=1709733050;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=6gLhTWaeA6K8LvtOWvt7viO6z/8krLAbtiOifZaThhE=;
        b=NS/LYuF56kAUQEoinQ/UFrAEKh5oy+NBDsfEtxbYb4eO94zXzRZcmqERzuiVXCLp6X
         MCPIKyCVz6MWaZKULaWNbzoT6jHFXJKgjGkSjArkT+8pm+eBJOus0RlSn3SUsP1sXVV8
         ogh9g/Y7Dub7peN4i1IbY084+hJFV2FXl64E0Yu+Jh/b9vnqRkBRUHIQKIaQGsT7/pxI
         /NhBwyspYSSeJcnoogAFtYbw5fe90FXYG7DAD/jkxs9fnqwTNIpcoQ/9bFaSODMIw6WK
         vIIc+1RuWTvXa4vp4Sl46vb2j4TfFQTki5g0ucsjl/+NrFwi4hc9UOOcGxQYmUhYqFwW
         7aRw==
X-Gm-Message-State: AOJu0YybPog6921a70FNvnQ0gKY5KTJ0FIeC4GO8lpnhsQrBJt5Gqhdg
	PJtVIG/9Q4dwqv6yYhjbjAywN7V+rGkoRWCYe/Fc7s/y9yDdwlB1yRpir0VbsMrEnDY5iXHPf2c
	=
X-Google-Smtp-Source: AGHT+IGzeq78e3YxEG9nZClAs6SJQ65DIRxb3NLzC5C3F+8ZUOyPNLllO+WuGMaN3l/lc8lAscKSnA==
X-Received: by 2002:aa7:d694:0:b0:566:ba0:414c with SMTP id d20-20020aa7d694000000b005660ba0414cmr5519206edr.26.1709128250605;
        Wed, 28 Feb 2024 05:50:50 -0800 (PST)
Message-ID: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
Date: Wed, 28 Feb 2024 14:50:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH 0/4] x86: CET-SS related adjustments
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

One might think of this as follow-on to XSA-451, but that's not quite
the right order of events.

There are a few open aspects; see individual patches.

1: remove redundant XEN_SHSTK check from reinit_bsp_stack()
2: record SSP at non-guest entry points
3: traps: use entry_ssp in fixup_exception_return()
4: prefer shadow stack for producing call traces

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:51:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:51:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686637.1068875 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKLi-0000es-Hg; Wed, 28 Feb 2024 13:51:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686637.1068875; Wed, 28 Feb 2024 13:51:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKLi-0000el-EN; Wed, 28 Feb 2024 13:51:54 +0000
Received: by outflank-mailman (input) for mailman id 686637;
 Wed, 28 Feb 2024 13:51:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfKLh-0000eZ-Iy
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:51:53 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8656b08a-d640-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 14:51:51 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-5640fef9fa6so7149996a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:51:51 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s24-20020a170906bc5800b00a3efa4e033asm1870713ejv.151.2024.02.28.05.51.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 05:51:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8656b08a-d640-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709128311; x=1709733111; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=w+KEn+hcWSFN2S28B1lVdeEMQiVbpABucnYzaxqMQm8=;
        b=Qux/xgWvGZjujzYDSY9JylKPOdCk9vfXcDbtCRNcu/qhlMpWSRfq/YKSiostT2hCqL
         ENy3V6+6Xzot+bPS/zjeWFY+MqpUHaBwkkISI9qBZcdMo0SLjCOAhExVbYKs8x0BRu7u
         B1/8Q2gv8+2ntl+CBI23f2cXMWbXHr6Q+Hzb2pgf3mlpaBTKVWHu/SUmh982igB9+uBo
         +latjEivpfsyuZMqYROVfk+jKmUPbZi5oSTDMAyXYGsqTaslaG/izfx5Th52IemrIGOn
         A6X/YSgK11Hg1TSPqKlAKlGR3BBZ2iQZGkjvkEym7KriRmBFed9wxi3B9Rstz1Z7QB1f
         T/eg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709128311; x=1709733111;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=w+KEn+hcWSFN2S28B1lVdeEMQiVbpABucnYzaxqMQm8=;
        b=xClN4FF4yGscz0hTU+E0GgX9MEu8xab0dtU3VLJJtHXZ+Av4lUxkjCUA6dtp28Ojz0
         4gt89nlgM/npsL+HKSY2xZ1rIFCRREavpeg+8i/DXgdMRLEqAWnVGIWs0Q/daoB546i0
         4i8EcJWSqYabUQAVUZvqHBVDk5ISCjKn90o0W2qB+9JNg05oTsa+5isTs+Qj+adKGiZT
         YaQM0mKmcCUQefDuJPfQFrD64L2ythphe/qBtl7ejPgn2mwWI2e3IEpNJDi8zJNkYk5e
         uz7zKTx7N45IIVhNhzQDhMDoAal3gprc9aghyhqeLA1L21FKblUX7BZGwu4JzbR3gUch
         Edig==
X-Gm-Message-State: AOJu0YxC2X+kkMDIJmlauChEoMrqqaPhrBWQWVCFoVwPKjfsjlzkSw4E
	TIfycmr6rZxzt8/MCEts6SJSIInDZlStu69Lsxb6DRe2Cw7JrAnuyQW3UU5p1uDANtpgjU1d2r8
	=
X-Google-Smtp-Source: AGHT+IGEAk3oZI5VJ2IkQIs92O6kgtOshVr0bXPkF4PYNTxa8IWW377FE7w/kvyHwVWECxGBjJJyOw==
X-Received: by 2002:a17:907:bb99:b0:a44:15c3:c8e9 with SMTP id xo25-20020a170907bb9900b00a4415c3c8e9mr714367ejc.28.1709128311157;
        Wed, 28 Feb 2024 05:51:51 -0800 (PST)
Message-ID: <b1cde647-3319-4363-962e-35f716b5c03b@suse.com>
Date: Wed, 28 Feb 2024 14:51:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 1/4] x86: remove redundant XEN_SHSTK check from
 reinit_bsp_stack()
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

As of 72d51813d631 ("x86: amend cpu_has_xen_{ibt,shstk}") this has been
integrated into cpu_has_xen_shstk.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -810,7 +810,7 @@ static void __init noreturn reinit_bsp_s
     if ( rc )
         panic("Error %d setting up PV root page table\n", rc);
 
-    if ( IS_ENABLED(CONFIG_XEN_SHSTK) && cpu_has_xen_shstk )
+    if ( cpu_has_xen_shstk )
     {
         wrmsrl(MSR_PL0_SSP,
                (unsigned long)stack + (PRIMARY_SHSTK_SLOT + 1) * PAGE_SIZE - 8);



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:52:12 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:52:12 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686639.1068885 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKLz-00011v-Of; Wed, 28 Feb 2024 13:52:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686639.1068885; Wed, 28 Feb 2024 13:52:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKLz-00011m-Ls; Wed, 28 Feb 2024 13:52:11 +0000
Received: by outflank-mailman (input) for mailman id 686639;
 Wed, 28 Feb 2024 13:52:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfKLy-0000eZ-Ax
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:52:10 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 90714bbd-d640-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 14:52:08 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a26ed1e05c7so878413366b.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:52:08 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s24-20020a170906bc5800b00a3efa4e033asm1870713ejv.151.2024.02.28.05.52.07
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 05:52:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 90714bbd-d640-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709128328; x=1709733128; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=gaX7a4NCHOeGdWCkg8kUSL8PRj5hzXXfo73pbxAntJ8=;
        b=bwZLe610hPexq3V1s2hT6jb5Wkg6RJ3QLjqeyoR1QAVC3Wqd5Wm/IFnMS5QDEAFuNl
         yG9t1lqEYfusKKMRT+RV8e3I0AVK0vUVjtFUCJUOBUdFVb9kJNqrpbD+vJMF8iBCS+z4
         NKKwV15nWYLVfnbIsmE3EhHoJthM9ReuJA4hhlrttgu0tEwX2xzjnlKRSfeiDXuRxkAu
         mO2vgO81Mj0o9M1TZOCs7HLcKyYLwLQie59Q323tJbmQEvC8kTmxjVj2iD7I7TJut/Ub
         Jl3+47gHgBzqUtOSobDuiQQcdUrhWjHgvR+GMWMUYqNTMY9F54sFZjS1fem5QN5T3S1Z
         te3A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709128328; x=1709733128;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=gaX7a4NCHOeGdWCkg8kUSL8PRj5hzXXfo73pbxAntJ8=;
        b=JJIAvTxvRsMagmKhooOXwBP/QKIuprxgrijJO9tcWX0PbKdJ9PhnR57gTXg7nnSNmJ
         0mDy5LQ9z7EjYmvxfP+7QI+LEbMx1k64LEd4OJjS7jffnVrnfvLoNAsKTo+fcU47pbsA
         sd3EvJUWbuS42AZ3NnVdgwvBuVwZD6bIxiydjB9vgarnQzkiXKNtbY1vBbucarAFvfsy
         uhobAyy0kB2qg5P8YO6s4B6NoOeNYlPqY96WJOK7Bhhdkv8dm4feu9tqz+EZIZJep6x3
         ZMpyo2aPPcXUAF1Ogm9KjrYAvawk/Ia9ojY8+1GJuH76PWnlnYGaBR/HJLka5qQ+0RVb
         bKgg==
X-Gm-Message-State: AOJu0YwtlNmfkPuILH0lbhRLtKDcwMWT2BBr1lYY5cfhZ2fRsbQYF1tf
	rpkFnxUMtqScaWl6qKDD6BXgO6ipPmgjYESCP+sknPqYxPryp3NlWFbhaNLdYxrjb3XX1FITZuM
	=
X-Google-Smtp-Source: AGHT+IF962mt3eRDy21WIUNAu+Y3BfGUsdDO1T8wRDs+WGYmPyqGL5DdVNU2BOS3piXY0SkY/N1xRw==
X-Received: by 2002:a17:906:70c:b0:a40:189d:c5bb with SMTP id y12-20020a170906070c00b00a40189dc5bbmr8228552ejb.0.1709128328039;
        Wed, 28 Feb 2024 05:52:08 -0800 (PST)
Message-ID: <0ad4543b-8eed-4147-b32d-b68d21fade98@suse.com>
Date: Wed, 28 Feb 2024 14:52:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 2/4] x86: record SSP at non-guest entry points
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

We will want to use that value for call trace generation, and likely
also to eliminate the somewhat fragile shadow stack searching done in
fixup_exception_return(). For those purposes, guest-only entry points do
not need to record that value.

To keep the saving code simple, record our own SSP that corresponds to
an exception frame, pointing to the top of the shadow stack counterpart
of what the CPU has saved on the regular stack. Consuming code can then
work its way from there.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
To record the full 64-bit value, some of the unused bits in the %cs slot
could be used. Sadly that slot has saved_upcall_mask in an unhelpful
location, otherwise simply storing low and high 32 bits in those two
separate half-slots would be a pretty obvious choice. As long as
"entry_ssp" is used in non-guest-entry frames only, we could of course
put half of it into a union with saved_upcall_mask ...

Else may want to put a BUILD_BUG_ON(VADDR_BITS > 48) somewhere, but I'm
afraid I can't really identify a good place for such to live.

Leveraging that the CPU stores zero in the upper bits of the selector
register slots, the save sequence could also be

        shl   $16, %rcx
        or    %rcx, UREGS_entry_ssp-2(%rsp)

That's shorter and avoids a 16-bit operation, but may be less desirable,
for being a read-modify-write access.

--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1354,6 +1354,7 @@ void arch_get_info_guest(struct vcpu *v,
     if ( !compat )
     {
         memcpy(&c.nat->user_regs, &v->arch.user_regs, sizeof(c.nat->user_regs));
+        c.nat->user_regs.entry_ssp = 0;
         if ( is_pv_domain(d) )
             memcpy(c.nat->trap_ctxt, v->arch.pv.trap_ctxt,
                    sizeof(c.nat->trap_ctxt));
--- a/xen/arch/x86/hvm/svm/entry.S
+++ b/xen/arch/x86/hvm/svm/entry.S
@@ -94,7 +94,7 @@ __UNLIKELY_END(nsvm_hap)
         sti
         vmrun
 
-        SAVE_ALL
+        SAVE_ALL ssp=0
 
         GET_CURRENT(bx)
 
--- a/xen/arch/x86/hvm/vmx/entry.S
+++ b/xen/arch/x86/hvm/vmx/entry.S
@@ -25,7 +25,7 @@
 #define VMLAUNCH     .byte 0x0f,0x01,0xc2
 
 ENTRY(vmx_asm_vmexit_handler)
-        SAVE_ALL
+        SAVE_ALL ssp=0
 
         mov  %cr2,%rax
         GET_CURRENT(bx)
@@ -119,7 +119,7 @@ UNLIKELY_END(realmode)
 
 .Lvmx_vmentry_fail:
         sti
-        SAVE_ALL
+        SAVE_ALL ssp=0
 
         /*
          * SPEC_CTRL_ENTRY notes
--- a/xen/arch/x86/include/asm/asm_defns.h
+++ b/xen/arch/x86/include/asm/asm_defns.h
@@ -221,7 +221,7 @@ static always_inline void stac(void)
 #endif
 
 #ifdef __ASSEMBLY__
-.macro SAVE_ALL compat=0
+.macro SAVE_ALL compat=0 ssp=IS_ENABLED(CONFIG_XEN_SHSTK)
         addq  $-(UREGS_error_code-UREGS_r15), %rsp
         cld
         movq  %rdi,UREGS_rdi(%rsp)
@@ -235,6 +235,9 @@ static always_inline void stac(void)
         movq  %rax,UREGS_rax(%rsp)
         xor   %eax, %eax
 .if !\compat
+.if \ssp
+        rdsspq %rcx
+.endif
         movq  %r8,UREGS_r8(%rsp)
         movq  %r9,UREGS_r9(%rsp)
         movq  %r10,UREGS_r10(%rsp)
@@ -264,6 +267,11 @@ static always_inline void stac(void)
         xor   %r13d, %r13d
         xor   %r14d, %r14d
         xor   %r15d, %r15d
+.if \ssp && !\compat
+        mov   %cx, UREGS_entry_ssp(%rsp)
+        shr   $16, %rcx
+        mov   %ecx, UREGS_entry_ssp+2(%rsp)
+.endif
 .endm
 
 #define LOAD_ONE_REG(reg, compat) \
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -48,6 +48,7 @@ void __dummy__(void)
     OFFSET(UREGS_eflags, struct cpu_user_regs, rflags);
     OFFSET(UREGS_rsp, struct cpu_user_regs, rsp);
     OFFSET(UREGS_ss, struct cpu_user_regs, ss);
+    OFFSET(UREGS_entry_ssp, struct cpu_user_regs, entry_ssp);
     OFFSET(UREGS_kernel_sizeof, struct cpu_user_regs, es);
     BLANK();
 
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -257,7 +257,7 @@ FUNC(lstar_enter)
         pushq $0
         BUILD_BUG_ON(TRAP_syscall & 0xff)
         movb  $TRAP_syscall >> 8, EFRAME_entry_vector + 1(%rsp)
-        SAVE_ALL
+        SAVE_ALL ssp=0
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
         /* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
@@ -296,7 +296,7 @@ FUNC(cstar_enter)
         pushq $0
         BUILD_BUG_ON(TRAP_syscall & 0xff)
         movb  $TRAP_syscall >> 8, EFRAME_entry_vector + 1(%rsp)
-        SAVE_ALL
+        SAVE_ALL ssp=0
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
         /* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
@@ -339,7 +339,7 @@ LABEL(sysenter_eflags_saved, 0)
         pushq $0
         BUILD_BUG_ON(TRAP_syscall & 0xff)
         movb  $TRAP_syscall >> 8, EFRAME_entry_vector + 1(%rsp)
-        SAVE_ALL
+        SAVE_ALL ssp=0
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
         /* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
@@ -394,7 +394,7 @@ FUNC(entry_int80)
         ALTERNATIVE "", clac, X86_FEATURE_XEN_SMAP
         pushq $0
         movb  $0x80, EFRAME_entry_vector(%rsp)
-        SAVE_ALL
+        SAVE_ALL ssp=0
 
         SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
         /* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
--- a/xen/include/public/arch-x86/xen-x86_64.h
+++ b/xen/include/public/arch-x86/xen-x86_64.h
@@ -183,7 +183,19 @@ struct cpu_user_regs {
     uint8_t  _pad1[3];
     __DECL_REG_LO16(flags); /* rflags.IF == !saved_upcall_mask */
     __DECL_REG_LO8(sp);
-    uint16_t ss, _pad2[3];
+    uint16_t ss;
+#if !defined(__XEN__)
+    uint16_t _pad2[3];
+#elif defined(COMPILE_OFFSETS)
+    uint16_t entry_ssp[3];
+#else
+    /*
+     * This points _at_ the corresponding shadow stack frame; it is _not_ the
+     * outer context's SSP.  That, if the outer context has CET-SS enabled,
+     * is stored in the top slot of the pointed to shadow stack frame.
+     */
+    signed long entry_ssp:48;
+#endif
     uint16_t es, _pad3[3];
     uint16_t ds, _pad4[3];
     uint16_t fs, _pad5[3];



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:52:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:52:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686641.1068895 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKMO-0001cE-1J; Wed, 28 Feb 2024 13:52:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686641.1068895; Wed, 28 Feb 2024 13:52:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKMN-0001c7-Tm; Wed, 28 Feb 2024 13:52:35 +0000
Received: by outflank-mailman (input) for mailman id 686641;
 Wed, 28 Feb 2024 13:52:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfKMM-0000eZ-49
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:52:34 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9ea85554-d640-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 14:52:32 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a43dba50bb7so162533766b.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:52:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s24-20020a170906bc5800b00a3efa4e033asm1870713ejv.151.2024.02.28.05.52.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 05:52:31 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9ea85554-d640-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709128352; x=1709733152; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=q/QgX3d2GmmEaeKazC02JMPPR+cdwuaWvIKU6akMN0U=;
        b=UHGSrD3sxscb+9I8SDvCFpl7XcCK7zxkzdQgPy5rNcF8VEV7/pEbatmfEsco78Lm6F
         gdQjtxHkE+8R6FX4KTW/leivPFH8Spc+6Gd+awyy1ksoy3Qn7SCxa3XqZJqplshqyf4N
         5P7P2kSZp0nehIXtHqCQTJCBBlhdfQDlm/S2vgT6FcKd5hsCNuKyvbk1Ko5QQnKxSAfF
         tcw+fvBCSxtZwYzQ8SQa5zqqNgsjq6AGCyDa1LAyP9XJinpSaSarWKx4TiNitohq8yhG
         2y7xJ+leIpWL2kPeIC+jMhY8ZB5NIBLa4tHokIlGt3RGEeYnp2Q3a0oDPPQgohbrBk9E
         L+IQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709128352; x=1709733152;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=q/QgX3d2GmmEaeKazC02JMPPR+cdwuaWvIKU6akMN0U=;
        b=Ogdh2pCWGWaoeu2mafflGFdmYi/VVCebOGldbCbJhO9yrH72X1sJH43KPTw8q+r0Ac
         oWzxFDwAJP7zye9fCWtfhHTOdgavqBH8TGGO0/MRkaQb99a5LfWETfq1mvfyviAZWNbe
         rRkA59wmol1iGRVbugbOs0W0TzwwRfo3/9CQcA1DIEix5fTT00qBEUJfusycwCwC/4uw
         UuMyTdsUPFCSFsXo29ooTCL70BAKJArVMXk3URk3ejxTLY89f4GFBo7t+bw8gZrEM5XJ
         jigr9VEUggXJ6Iacr7dGBRX7L/EM3Pcj31IA5lDtK6t2brc8zrSFP9wFGIYgTzGIHrXn
         lfPg==
X-Gm-Message-State: AOJu0YzWLSHogjE3LhPLJ3sEpVwnjphp/nm/Y/Ev054AuJV4Deo2v1SP
	5QgpsezdR7TdDI9u52gjLso0LcCVM/nCfTqlZ83h9MOQ3IvzPMOe3OAKtPNfRHE9NBBozuQ5r54
	=
X-Google-Smtp-Source: AGHT+IHw+/IuBLaeeo8x6NbzlhNGLECq9A1xf3dIcqlbJd62TTy6LJ4i9ZBlmf4YPKGTOwnwHCbpJQ==
X-Received: by 2002:a17:906:6d45:b0:a43:4c0a:5360 with SMTP id a5-20020a1709066d4500b00a434c0a5360mr5894721ejt.32.1709128351933;
        Wed, 28 Feb 2024 05:52:31 -0800 (PST)
Message-ID: <45f89d82-bdc6-44b6-a784-bcdfdcca403d@suse.com>
Date: Wed, 28 Feb 2024 14:52:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 3/4] x86/traps: use entry_ssp in fixup_exception_return()
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

With the value recorded on entry there's no need anymore to go hunt for
the respective exception frame on the shadow stack. By deriving "ptr"
from that field (without any offset), it then ends up pointin one slot
lower than before. Therefore all array indexes need incrementing, nicely
doing away with all the negative ones.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
Indentation of the prior inner (but not innermost) if()'s body is
deliberately left untouched, to aid review. It'll be adjusted in a
separate follow-on patch.

--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -434,18 +434,6 @@ unsigned long get_stack_trace_bottom(uns
     }
 }
 
-static unsigned long get_shstk_bottom(unsigned long sp)
-{
-    switch ( get_stack_page(sp) )
-    {
-#ifdef CONFIG_XEN_SHSTK
-    case 0:  return ROUNDUP(sp, IST_SHSTK_SIZE) - sizeof(unsigned long);
-    case 5:  return ROUNDUP(sp, PAGE_SIZE)      - sizeof(unsigned long);
-#endif
-    default: return sp - sizeof(unsigned long);
-    }
-}
-
 unsigned long get_stack_dump_bottom(unsigned long sp)
 {
     switch ( get_stack_page(sp) )
@@ -837,24 +825,26 @@ static void fixup_exception_return(struc
 {
     if ( IS_ENABLED(CONFIG_XEN_SHSTK) )
     {
-        unsigned long ssp, *ptr, *base;
+        unsigned long ssp = rdssp();
 
-        if ( (ssp = rdssp()) == SSP_NO_SHSTK )
-            goto shstk_done;
+        if ( ssp != SSP_NO_SHSTK )
+        {
+            unsigned long *ptr = _p(regs->entry_ssp);
+            unsigned long primary_shstk =
+                (ssp & ~(STACK_SIZE - 1)) +
+                (PRIMARY_SHSTK_SLOT + 1) * PAGE_SIZE - 8;
 
-        ptr = _p(ssp);
-        base = _p(get_shstk_bottom(ssp));
+            BUG_ON((regs->entry_ssp ^ primary_shstk) >> PAGE_SHIFT);
 
-        for ( ; ptr < base; ++ptr )
-        {
             /*
-             * Search for %rip.  The shstk currently looks like this:
+             * The shstk currently looks like this:
              *
              *   tok  [Supervisor token, == &tok | BUSY, only with FRED inactive]
              *   ...  [Pointed to by SSP for most exceptions, empty in IST cases]
              *   %cs  [== regs->cs]
              *   %rip [== regs->rip]
-             *   SSP  [Likely points to 3 slots higher, above %cs]
+             *   SSP  [Pointed to by entry_ssp; Likely points to 3 slots
+             *         higher, above %cs]
              *   ...  [call tree to this function, likely 2/3 slots]
              *
              * and we want to overwrite %rip with fixup.  There are two
@@ -867,13 +857,10 @@ static void fixup_exception_return(struc
              *
              * Check for both regs->rip and regs->cs matching.
              */
-            if ( ptr[0] == regs->rip && ptr[1] == regs->cs )
-            {
-                unsigned long primary_shstk =
-                    (ssp & ~(STACK_SIZE - 1)) +
-                    (PRIMARY_SHSTK_SLOT + 1) * PAGE_SIZE - 8;
+            BUG_ON(ptr[1] != regs->rip || ptr[2] != regs->cs);
 
-                wrss(fixup, ptr);
+            {
+                wrss(fixup, &ptr[1]);
 
                 if ( !stub_ra )
                     goto shstk_done;
@@ -890,7 +877,7 @@ static void fixup_exception_return(struc
                  * - if we're on an IST stack, we need to increment the
                  *   original SSP.
                  */
-                BUG_ON((ptr[-1] ^ primary_shstk) >> PAGE_SHIFT);
+                BUG_ON((ptr[0] ^ primary_shstk) >> PAGE_SHIFT);
 
                 if ( (ssp ^ primary_shstk) >> PAGE_SHIFT )
                 {
@@ -899,37 +886,27 @@ static void fixup_exception_return(struc
                      * addresses actually match.  Then increment the interrupted
                      * context's SSP.
                      */
-                    BUG_ON(stub_ra != *(unsigned long*)ptr[-1]);
-                    wrss(ptr[-1] + 8, &ptr[-1]);
+                    BUG_ON(stub_ra != *(unsigned long*)ptr[0]);
+                    wrss(ptr[0] + 8, &ptr[0]);
                     goto shstk_done;
                 }
 
                 /* Make sure the two return addresses actually match. */
-                BUG_ON(stub_ra != ptr[2]);
+                BUG_ON(stub_ra != ptr[3]);
 
                 /* Move exception frame, updating SSP there. */
-                wrss(ptr[1], &ptr[2]); /* %cs */
-                wrss(ptr[0], &ptr[1]); /* %rip */
-                wrss(ptr[-1] + 8, &ptr[0]); /* SSP */
+                wrss(ptr[2], &ptr[3]); /* %cs */
+                wrss(ptr[1], &ptr[2]); /* %rip */
+                wrss(ptr[0] + 8, &ptr[1]); /* SSP */
 
                 /* Move all newer entries. */
-                while ( --ptr != _p(ssp) )
-                    wrss(ptr[-1], &ptr[0]);
+                while ( ptr-- != _p(ssp) )
+                    wrss(ptr[0], &ptr[1]);
 
                 /* Finally account for our own stack having shifted up. */
                 asm volatile ( "incsspd %0" :: "r" (2) );
-
-                goto shstk_done;
             }
         }
-
-        /*
-         * We failed to locate and fix up the shadow IRET frame.  This could
-         * be due to shadow stack corruption, or bad logic above.  We cannot
-         * continue executing the interrupted context.
-         */
-        BUG();
-
     }
  shstk_done:
 



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:53:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:53:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686644.1068904 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKNC-0002LE-DA; Wed, 28 Feb 2024 13:53:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686644.1068904; Wed, 28 Feb 2024 13:53:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKNC-0002L7-Ad; Wed, 28 Feb 2024 13:53:26 +0000
Received: by outflank-mailman (input) for mailman id 686644;
 Wed, 28 Feb 2024 13:53:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfKNA-0002Kr-CK
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:53:24 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bc70ed05-d640-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 14:53:22 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a3122b70439so694660466b.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:53:22 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s24-20020a170906bc5800b00a3efa4e033asm1870713ejv.151.2024.02.28.05.53.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 05:53:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bc70ed05-d640-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709128402; x=1709733202; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=8+WqFSaalzUAe+nFKzuTogcSanw3XdVfQLlmS0YufqE=;
        b=WcbTao1+xDJZDr5rHAtpwwYDm/khjGu5DwDNyCOdvR/GrfV7WbmsLwBGPjl2XXYhUu
         8pOFe+yKlNhdSh8k9CViBC/EchGht2418mfV0MkvZvPApGsTbmakJkpDbcm2J0N1TaCm
         jQeFCyT4Ep1Wt/4MOI8XeeomF7v4gYndEoWK1YQ66uRsHz1xnbysNMxrs8EQoFfShDcg
         Zy9A9QrmF6MAn1ZOQUS/pP71bQMg4iai0xB1tcoisliXMSRonUoAPdnnFdioD9S4Qvga
         UVttXBH81PXxJIyC4zUXY+uMU6cGll8XqUHZHaRhFAYzb2kGE9L54nIQGyTFsGDgMle1
         2gQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709128402; x=1709733202;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=8+WqFSaalzUAe+nFKzuTogcSanw3XdVfQLlmS0YufqE=;
        b=sQ2Wlqf4kpv+2DRT0oWjjSVNIbqvr69yaRbGmJVoHsquR07D+0APbe8DmSHvpvcQZ/
         Daab6jF3NvnilWEssuQ69EDhV8kqMdoyRVG75mcSSuPnDOY191QA5xI2n66cJ7L+dTFu
         LIBBY8gMmrHmJgT7MdYexinOV5qhXGE1VJADPUkw+ZVKBFyhjLTPYWn61x1pSD1J49iZ
         toURCU0aFhEYmE64FpBYAQ0WiAZVQodr6A5ZOiuaxHbTW6nt1CtaNvlJjcT+1uEzj7JG
         KYxZH2Rr6QK6QZQxgeOz3f2d30xsLp1x6XZYUy0QHFAtBwtFbA/+CIx64VN4GVBGDImh
         D85w==
X-Gm-Message-State: AOJu0Ywg2G7WgmHy7x2xAfRCEfTZpHsYy6RtXNLO30OISlnAMBXDEQwP
	ulitziYCE1iTZwKWloZMHsSMqfV6/kYW4f00bWaHqzK4G4F0PZqI8++aEMEPqdxjAjkv0XtAMGk
	=
X-Google-Smtp-Source: AGHT+IFXBj6+nUIuDVTPACCxjqiwBVHNhpXEO8GKSg4nnFMo8PfQlrLEa79p1XwGxbyquvxdvyeu3A==
X-Received: by 2002:a17:906:1996:b0:a43:f22e:57a6 with SMTP id g22-20020a170906199600b00a43f22e57a6mr1415733ejd.67.1709128401845;
        Wed, 28 Feb 2024 05:53:21 -0800 (PST)
Message-ID: <fdacbb8b-5a58-4d6b-89b2-ea98b4fed895@suse.com>
Date: Wed, 28 Feb 2024 14:53:21 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: [PATCH 4/4] x86: prefer shadow stack for producing call traces
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Shadow stacks contain little more than return addresses, and they in
particular allow precise call traces also without FRAME_POINTER.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
While the 'E' for exception frames is probably okay, I'm not overly
happy with the 'C' (for CET). I would have preferred 'S' (for shadow),
but we use that character already.

As an alternative to suppressing output for the top level exception
frame, adding the new code ahead of the 'R' output line (and then also
ahead of the stack top read) could be considered.

Perhaps having a printk() for the PV entry case is meaningless, for
- no frame being pushed when entered from CPL=3 (64-bit PV),
- no entry possible from CPL<3 (32-bit PV disabled when CET is active)?
In which case the comment probably should just be "Bogus." and the code
merely be "break;".

Quite likely a number of other uses of is_active_kernel_text() also want
amending with in_stub().

--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -449,6 +449,11 @@ unsigned long get_stack_dump_bottom(unsi
     }
 }
 
+static bool in_stub(unsigned long addr)
+{
+    return !((this_cpu(stubs.addr) ^ addr) >> STUB_BUF_SHIFT);
+}
+
 #if !defined(CONFIG_FRAME_POINTER)
 
 /*
@@ -539,6 +544,50 @@ static void show_trace(const struct cpu_
          !is_active_kernel_text(tos) )
         printk("   [<%p>] R %pS\n", _p(regs->rip), _p(regs->rip));
 
+    if ( IS_ENABLED(CONFIG_XEN_SHSTK) && rdssp() != SSP_NO_SHSTK )
+    {
+        const unsigned long *ptr = _p(regs->entry_ssp);
+        unsigned int n;
+
+        for ( n = 0; (unsigned long)ptr & (PAGE_SIZE - sizeof(*ptr)); ++n )
+        {
+            unsigned long val = *ptr;
+
+            if ( is_active_kernel_text(val) || in_stub(val) )
+            {
+                /* Normal return address entry.  */
+                printk("   [<%p>] C %pS\n", _p(val), _p(val));
+                ++ptr;
+            }
+            else if ( !((val ^ *ptr) >> (PAGE_SHIFT + STACK_ORDER)) )
+            {
+                if ( val & (sizeof(val) - 1) )
+                {
+                    /* Most likely a supervisor token. */
+                    break;
+                }
+
+                /*
+                 * Ought to be a hypervisor interruption frame.  But don't
+                 * (re)log the current frame's %rip.
+                 */
+                if ( n || ptr[1] != regs->rip )
+                    printk("   [<%p>] E %pS\n", _p(ptr[1]), _p(ptr[1]));
+                ptr = _p(val);
+            }
+            else
+            {
+                /* Ought to be a PV guest hypercall/interruption frame.  */
+                printk("   %04lx:[<%p>] E\n", ptr[2], _p(ptr[1]));
+                ptr = 0;
+            }
+        }
+
+        /* Fall back to legacy stack trace if nothing was logged at all. */
+        if ( n )
+            return;
+    }
+
     if ( fault )
     {
         printk("   [Fault on access]\n");



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:54:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:54:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686646.1068915 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKNy-0002yA-Nk; Wed, 28 Feb 2024 13:54:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686646.1068915; Wed, 28 Feb 2024 13:54:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKNy-0002y3-Kt; Wed, 28 Feb 2024 13:54:14 +0000
Received: by outflank-mailman (input) for mailman id 686646;
 Wed, 28 Feb 2024 13:54:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eIhW=KF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfKNx-0002xq-Jb
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:54:13 +0000
Received: from mail-oa1-x36.google.com (mail-oa1-x36.google.com
 [2001:4860:4864:20::36])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d9a58331-d640-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 14:54:12 +0100 (CET)
Received: by mail-oa1-x36.google.com with SMTP id
 586e51a60fabf-21f70f72fb5so3662461fac.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:54:12 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 wl12-20020a05620a57cc00b007874e64c879sm4585542qkn.116.2024.02.28.05.54.09
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 05:54:10 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d9a58331-d640-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709128451; x=1709733251; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=RpEai/TZdpTFYPyWBBuomxZ211kLnNsS/6XnU5iJ+HU=;
        b=U6PPKYEMRTG0XWlXu8nAiSMgZ1dIkmxzCuMeRGs5Er+z2toKJaiBklr0EbNpUzajbk
         Iyci69TRYDSA0bqHCCIINU2YLOsd/ZmuCWpl7NpbUlmh2bVPQ9uvF74pJiNbY1zKukJa
         5RYULvUkJiuDmMzMR5fQqXisPgJDJLuWUbTTA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709128451; x=1709733251;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=RpEai/TZdpTFYPyWBBuomxZ211kLnNsS/6XnU5iJ+HU=;
        b=ZvSIP85+p773mfKHodmjQKvI2zYbzB2zhxeTZ74WG1Q40rnjwKQ65vHFufv7TjFDsC
         2OtIKmDAkswbt1CTDA21a9Yhi771RCr6r+V1VSZ05B9HS/GMGbUwfiBS1L1Wv/hPYVAm
         mUA9XleFT2k7i9s1o4e+7Z7nEAvoF0CxzwrNn0luW9Qju/tsTy+2kP2rYQyM/38uqXgY
         Ilkrk80ACejqUWIddHCE8pyZXUfYFet9gN5yJinYL72MF8svdf1wRAzFuDtIRnJCii6v
         TysyBPEQO3i9E8T6HNhfzl1DLOrBt6EWcHo3ACtgHKRFRzemwhcfx3k1iJrwHrTR0cmN
         NVEQ==
X-Forwarded-Encrypted: i=1; AJvYcCWO3hx66C9asmyAlhFmFzM7VjjaUOYzKHHCnuRYxWrp1121wl3Mj+tLBc8YkSRvxUp/29rSRh4DX+bNjmBaxcL6szBDDd9AdoGiA0xl/6o=
X-Gm-Message-State: AOJu0YzklO8AWhQKQmDp2O4rSYOJv9AwUrsq94fzLL1q6SBEvoy2TS44
	1SXqizr3gtWt6R4683AxEVTvbnqEfQRgJ6dJaznhfhb0be4gVHOco3RGCmcgw9U=
X-Google-Smtp-Source: AGHT+IHoqH/F7cqnXdsLLHeBgAg5uC3v8i68FE2xFR5UEECETFNNEGW7lpoJ6XvlswFFOWFyRFDTuQ==
X-Received: by 2002:a05:6871:7429:b0:21e:74d8:8f46 with SMTP id nw41-20020a056871742900b0021e74d88f46mr14890715oac.21.1709128451369;
        Wed, 28 Feb 2024 05:54:11 -0800 (PST)
Message-ID: <637e1b89-e55f-43ff-a9b4-44a6723c9074@citrix.com>
Date: Wed, 28 Feb 2024 13:54:08 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/4] x86: remove redundant XEN_SHSTK check from
 reinit_bsp_stack()
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
 <b1cde647-3319-4363-962e-35f716b5c03b@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <b1cde647-3319-4363-962e-35f716b5c03b@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 28/02/2024 1:51 pm, Jan Beulich wrote:
> As of 72d51813d631 ("x86: amend cpu_has_xen_{ibt,shstk}") this has been
> integrated into cpu_has_xen_shstk.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

I'm vaguely recall noticing that after your original patch, but never
got around to sending a fix.


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 13:59:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 13:59:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686652.1068925 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKSt-0004Ge-8Z; Wed, 28 Feb 2024 13:59:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686652.1068925; Wed, 28 Feb 2024 13:59:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKSt-0004GX-5w; Wed, 28 Feb 2024 13:59:19 +0000
Received: by outflank-mailman (input) for mailman id 686652;
 Wed, 28 Feb 2024 13:59:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KqYS=KF=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rfKSr-0004GR-Vj
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 13:59:17 +0000
Received: from mail-oa1-x30.google.com (mail-oa1-x30.google.com
 [2001:4860:4864:20::30])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8f0e7510-d641-11ee-afd7-a90da7624cb6;
 Wed, 28 Feb 2024 14:59:16 +0100 (CET)
Received: by mail-oa1-x30.google.com with SMTP id
 586e51a60fabf-21ffac15528so1945337fac.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 05:59:16 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 y8-20020ac87088000000b0042e6ad88de0sm4541777qto.73.2024.02.28.05.59.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 05:59:14 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8f0e7510-d641-11ee-afd7-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709128755; x=1709733555; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=os3HgQ8AeYU2sQX2v+d85sqmjd8DP68ruVnpK6fu8HM=;
        b=VsMrFVMwHQbxeX82oQz7YWdgJ/vZZm4D6K3kiiqxcCMx97iqGM5nCgq2+jPbmKdKWK
         mYxoEYmXnOIpIRHPzDzacPSdEIUyCzT87zxL9Vsgw/4KoUtaDyPsW3cFNqaxTKv5vhTv
         XLdc0dce6NTwDay31jTc6O9AW7fqmuk4yqIMQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709128755; x=1709733555;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=os3HgQ8AeYU2sQX2v+d85sqmjd8DP68ruVnpK6fu8HM=;
        b=rnWaGqYFCQrtHgg9GVQ+2eibWPKGY4cmGObf4KhTIs030DXeo45/ofMB2af24UtiGu
         fF+R6y1h0cd4FjMcrueViLRE6ATxNspHjFmvFY58nonrcMI5PzOtHjHAw86DZ3rmEuVW
         H2JQ4V1Um9uA/aYnpbCsGxDDuskctQZq+lDjoj59S9D+uEmyy3nRmpIzEI/FQfIqhXTu
         tXYUDUhXw37RZE0xAGBIgPd/UNHHKDzF+DMPG4aj2NloNtcAkpIjvzM+OOMmrTuzyDH+
         5GnL66mhOLtba9RNIsTrgj2PJ5YndwTgFJBJfC4PovLqRp5YuuZ6hNQd6+3svo5/65ZT
         zU6w==
X-Gm-Message-State: AOJu0YxvoV3SNjiy3fiMv4wo+m3XizLBW3N8O7D1GuoTHQuprFQej8gt
	P/hQbv9v6Wsv+f+64ZHmnIs/YD75cvIgbmfX7H8or4ZsxP7z663GcsmITGFGWlTColCcuCXQ8Lr
	z
X-Google-Smtp-Source: AGHT+IEHnPPzPBUqjZiM/VDJhnbwmqY4z8BNRWRpslfDq6RWy82cxvphMLpvVrohfA9Zc6J9wCs3ag==
X-Received: by 2002:a05:6870:164d:b0:21e:a94b:e80 with SMTP id c13-20020a056870164d00b0021ea94b0e80mr14668408oae.18.1709128754933;
        Wed, 28 Feb 2024 05:59:14 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/altcall: always use a temporary parameter stashing variable
Date: Wed, 28 Feb 2024 14:59:08 +0100
Message-ID: <20240228135908.13319-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The usage in ALT_CALL_ARG() on clang of:

register union {
    typeof(arg) e;
    const unsigned long r;
} ...

When `arg` is the first argument to alternative_{,v}call() and
const_vlapic_vcpu() is used results in clang 3.5.0 complaining with:

arch/x86/hvm/vlapic.c:141:47: error: non-const static data member must be initialized out of line
         alternative_call(hvm_funcs.test_pir, const_vlapic_vcpu(vlapic), vec) )

Workaround this by pulling `arg1` into a local variable, like it's done for
further arguments (arg2, arg3...)

Originally arg1 wasn't pulled into a variable because for the a1_ register
local variable the possible clobbering as a result of operators on other
variables don't matter:

https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html#Local-Register-Variables

Note clang version 3.8.1 seems to already be fixed and don't require the
workaround, but since it's harmless do it uniformly everywhere.

Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Fixes: 2ce562b2a413 ('x86/altcall: use a union as register type for function parameters on clang')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Gitlab CI seems OK on both the 4.17 and staging branches with this applied:

4.17:    https://gitlab.com/xen-project/people/royger/xen/-/pipelines/1193801183
staging: https://gitlab.com/xen-project/people/royger/xen/-/pipelines/1193801881
---
 xen/arch/x86/include/asm/alternative.h | 36 +++++++++++++++++---------
 1 file changed, 24 insertions(+), 12 deletions(-)

diff --git a/xen/arch/x86/include/asm/alternative.h b/xen/arch/x86/include/asm/alternative.h
index 3c14db5078ba..0d3697f1de49 100644
--- a/xen/arch/x86/include/asm/alternative.h
+++ b/xen/arch/x86/include/asm/alternative.h
@@ -253,21 +253,24 @@ extern void alternative_branches(void);
 })
 
 #define alternative_vcall1(func, arg) ({           \
-    ALT_CALL_ARG(arg, 1);                          \
+    typeof(arg) v1_ = (arg);                       \
+    ALT_CALL_ARG(v1_, 1);                          \
     ALT_CALL_NO_ARG2;                              \
     (void)sizeof(func(arg));                       \
     (void)alternative_callN(1, int, func);         \
 })
 
 #define alternative_call1(func, arg) ({            \
-    ALT_CALL_ARG(arg, 1);                          \
+    typeof(arg) v1_ = (arg);                       \
+    ALT_CALL_ARG(v1_, 1);                          \
     ALT_CALL_NO_ARG2;                              \
     alternative_callN(1, typeof(func(arg)), func); \
 })
 
 #define alternative_vcall2(func, arg1, arg2) ({           \
+    typeof(arg1) v1_ = (arg1);                            \
     typeof(arg2) v2_ = (arg2);                            \
-    ALT_CALL_ARG(arg1, 1);                                \
+    ALT_CALL_ARG(v1_, 1);                                 \
     ALT_CALL_ARG(v2_, 2);                                 \
     ALT_CALL_NO_ARG3;                                     \
     (void)sizeof(func(arg1, arg2));                       \
@@ -275,17 +278,19 @@ extern void alternative_branches(void);
 })
 
 #define alternative_call2(func, arg1, arg2) ({            \
+    typeof(arg1) v1_ = (arg1);                            \
     typeof(arg2) v2_ = (arg2);                            \
-    ALT_CALL_ARG(arg1, 1);                                \
+    ALT_CALL_ARG(v1_, 1);                                 \
     ALT_CALL_ARG(v2_, 2);                                 \
     ALT_CALL_NO_ARG3;                                     \
     alternative_callN(2, typeof(func(arg1, arg2)), func); \
 })
 
 #define alternative_vcall3(func, arg1, arg2, arg3) ({    \
+    typeof(arg1) v1_ = (arg1);                           \
     typeof(arg2) v2_ = (arg2);                           \
     typeof(arg3) v3_ = (arg3);                           \
-    ALT_CALL_ARG(arg1, 1);                               \
+    ALT_CALL_ARG(v1_, 1);                                \
     ALT_CALL_ARG(v2_, 2);                                \
     ALT_CALL_ARG(v3_, 3);                                \
     ALT_CALL_NO_ARG4;                                    \
@@ -294,9 +299,10 @@ extern void alternative_branches(void);
 })
 
 #define alternative_call3(func, arg1, arg2, arg3) ({     \
+    typeof(arg1) v1_ = (arg1);                            \
     typeof(arg2) v2_ = (arg2);                           \
     typeof(arg3) v3_ = (arg3);                           \
-    ALT_CALL_ARG(arg1, 1);                               \
+    ALT_CALL_ARG(v1_, 1);                                \
     ALT_CALL_ARG(v2_, 2);                                \
     ALT_CALL_ARG(v3_, 3);                                \
     ALT_CALL_NO_ARG4;                                    \
@@ -305,10 +311,11 @@ extern void alternative_branches(void);
 })
 
 #define alternative_vcall4(func, arg1, arg2, arg3, arg4) ({ \
+    typeof(arg1) v1_ = (arg1);                              \
     typeof(arg2) v2_ = (arg2);                              \
     typeof(arg3) v3_ = (arg3);                              \
     typeof(arg4) v4_ = (arg4);                              \
-    ALT_CALL_ARG(arg1, 1);                                  \
+    ALT_CALL_ARG(v1_, 1);                                   \
     ALT_CALL_ARG(v2_, 2);                                   \
     ALT_CALL_ARG(v3_, 3);                                   \
     ALT_CALL_ARG(v4_, 4);                                   \
@@ -318,10 +325,11 @@ extern void alternative_branches(void);
 })
 
 #define alternative_call4(func, arg1, arg2, arg3, arg4) ({  \
+    typeof(arg1) v1_ = (arg1);                              \
     typeof(arg2) v2_ = (arg2);                              \
     typeof(arg3) v3_ = (arg3);                              \
     typeof(arg4) v4_ = (arg4);                              \
-    ALT_CALL_ARG(arg1, 1);                                  \
+    ALT_CALL_ARG(v1_, 1);                                   \
     ALT_CALL_ARG(v2_, 2);                                   \
     ALT_CALL_ARG(v3_, 3);                                   \
     ALT_CALL_ARG(v4_, 4);                                   \
@@ -332,11 +340,12 @@ extern void alternative_branches(void);
 })
 
 #define alternative_vcall5(func, arg1, arg2, arg3, arg4, arg5) ({ \
+    typeof(arg1) v1_ = (arg1);                                    \
     typeof(arg2) v2_ = (arg2);                                    \
     typeof(arg3) v3_ = (arg3);                                    \
     typeof(arg4) v4_ = (arg4);                                    \
     typeof(arg5) v5_ = (arg5);                                    \
-    ALT_CALL_ARG(arg1, 1);                                        \
+    ALT_CALL_ARG(v1_, 1);                                         \
     ALT_CALL_ARG(v2_, 2);                                         \
     ALT_CALL_ARG(v3_, 3);                                         \
     ALT_CALL_ARG(v4_, 4);                                         \
@@ -347,11 +356,12 @@ extern void alternative_branches(void);
 })
 
 #define alternative_call5(func, arg1, arg2, arg3, arg4, arg5) ({  \
+    typeof(arg1) v1_ = (arg1);                                    \
     typeof(arg2) v2_ = (arg2);                                    \
     typeof(arg3) v3_ = (arg3);                                    \
     typeof(arg4) v4_ = (arg4);                                    \
     typeof(arg5) v5_ = (arg5);                                    \
-    ALT_CALL_ARG(arg1, 1);                                        \
+    ALT_CALL_ARG(v1_, 1);                                         \
     ALT_CALL_ARG(v2_, 2);                                         \
     ALT_CALL_ARG(v3_, 3);                                         \
     ALT_CALL_ARG(v4_, 4);                                         \
@@ -363,12 +373,13 @@ extern void alternative_branches(void);
 })
 
 #define alternative_vcall6(func, arg1, arg2, arg3, arg4, arg5, arg6) ({ \
+    typeof(arg1) v1_ = (arg1);                                          \
     typeof(arg2) v2_ = (arg2);                                          \
     typeof(arg3) v3_ = (arg3);                                          \
     typeof(arg4) v4_ = (arg4);                                          \
     typeof(arg5) v5_ = (arg5);                                          \
     typeof(arg6) v6_ = (arg6);                                          \
-    ALT_CALL_ARG(arg1, 1);                                              \
+    ALT_CALL_ARG(v1_, 1);                                               \
     ALT_CALL_ARG(v2_, 2);                                               \
     ALT_CALL_ARG(v3_, 3);                                               \
     ALT_CALL_ARG(v4_, 4);                                               \
@@ -379,12 +390,13 @@ extern void alternative_branches(void);
 })
 
 #define alternative_call6(func, arg1, arg2, arg3, arg4, arg5, arg6) ({  \
+    typeof(arg1) v1_ = (arg1);                                          \
     typeof(arg2) v2_ = (arg2);                                          \
     typeof(arg3) v3_ = (arg3);                                          \
     typeof(arg4) v4_ = (arg4);                                          \
     typeof(arg5) v5_ = (arg5);                                          \
     typeof(arg6) v6_ = (arg6);                                          \
-    ALT_CALL_ARG(arg1, 1);                                              \
+    ALT_CALL_ARG(v1_, 1);                                               \
     ALT_CALL_ARG(v2_, 2);                                               \
     ALT_CALL_ARG(v3_, 3);                                               \
     ALT_CALL_ARG(v4_, 4);                                               \
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 14:28:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 14:28:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686663.1068935 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKv9-0002h6-H1; Wed, 28 Feb 2024 14:28:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686663.1068935; Wed, 28 Feb 2024 14:28:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKv9-0002gz-Df; Wed, 28 Feb 2024 14:28:31 +0000
Received: by outflank-mailman (input) for mailman id 686663;
 Wed, 28 Feb 2024 14:28:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfKv7-0002gt-Qs
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 14:28:29 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a333572b-d645-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 15:28:27 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-564fc495d83so6579228a12.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 06:28:27 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 a6-20020a1709063e8600b00a4138c3f054sm1892286ejj.32.2024.02.28.06.28.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 06:28:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a333572b-d645-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709130507; x=1709735307; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ldZhWo/Cuxns8dCwcA1Jo6eZxJbm0ahkIXNWTlT4PtU=;
        b=KVc0Vl5TnOgMJHQOckRLmZk67BaIi0r2gxRQda7nNGIa0oEHMvsqy1B73h0ZC1t2Wh
         5i0yJEAle+lb0Ud0/V8Wf3ZEEq8Lxap/UylIDyIpIYIsirs1cHzurxL/NpA4refx+tLW
         PB/SHSnRuwNyvkng2lGXfLUkrEF2jA7X/I/OjfNmBwlgdhK/JHVQZqCBdl1cVpV4v2pG
         jwzqlJqdA72/dKjUtXW888aCOXDKiZobLqgjos0rHJEcM7uMa966c8oELiqtvjZ+Nu2W
         Fk+kshGFq5MYLlCXGRJCZMIYkpP8vrhJV3ziYAWeBDlzqy3HgEKSMvxWKpKR6aNjf9sb
         kxIA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709130507; x=1709735307;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ldZhWo/Cuxns8dCwcA1Jo6eZxJbm0ahkIXNWTlT4PtU=;
        b=P2Jq+rvl4Ln/U/mzewyDVDBJ/tkkvsZsjEhkGMBMPBFII+lVhEOBr8bw9VR0Uchdcl
         IznSvHhu2g4aqdwjVvMv2JHozPqJjN1EWjMdFemUDtzj2cNQfrKPRt6WF9YiT1DzOfSp
         l+JOJXL20ybvRTCp994zZr3yWU90IqR6wA6MK/Be5sE3gMoMXUOqvNgnuQ9ZzpYmDgJv
         egTLx3wQ1hujOolneMFds9k8O/V0B/QWxpZW/51HbyPzXNlSpB9IH/5ooGeoYNdGMSiU
         9jCAm2HnjtDHZoytkxH3MFRcjqmgAUFRT5fE0xeh4x4x6e2ozvUyzswvmvNmQLyS7BJ9
         kpnQ==
X-Forwarded-Encrypted: i=1; AJvYcCWTm1PjNANMXDlF2hIfNDouvrRYsNd/afdmx3mEVyqcf0h2NhoLgsyFgxQbcIIR3PrwCvRGHGZ/f5GHfbjQhYkWiWdjqvYoAsUF2L/RhkQ=
X-Gm-Message-State: AOJu0YyJjLC5/HD9/hIWr0/j2eQycKVj+Re7mrYNqRD29PzEUEAANTr/
	74zIqUSw4rJW01GgnVke5dFauB0GwvimpIcAvs3IBOiBTSpHDy7hk5bUkvksAA==
X-Google-Smtp-Source: AGHT+IHBUm/jGZFOfR7MgLSGeBCXN5WpKcQilJuwQ9wl8rslvwMPiGJwu59f/D2oCPxdXwOBlh/uyw==
X-Received: by 2002:a17:906:8313:b0:a44:1103:ede3 with SMTP id j19-20020a170906831300b00a441103ede3mr925691ejx.31.1709130506976;
        Wed, 28 Feb 2024 06:28:26 -0800 (PST)
Message-ID: <11f0e1dd-01b4-4d13-8137-e86b0975ad80@suse.com>
Date: Wed, 28 Feb 2024 15:28:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/altcall: always use a temporary parameter stashing
 variable
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240228135908.13319-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240228135908.13319-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28.02.2024 14:59, Roger Pau Monne wrote:
> The usage in ALT_CALL_ARG() on clang of:
> 
> register union {
>     typeof(arg) e;
>     const unsigned long r;
> } ...
> 
> When `arg` is the first argument to alternative_{,v}call() and
> const_vlapic_vcpu() is used results in clang 3.5.0 complaining with:
> 
> arch/x86/hvm/vlapic.c:141:47: error: non-const static data member must be initialized out of line
>          alternative_call(hvm_funcs.test_pir, const_vlapic_vcpu(vlapic), vec) )
> 
> Workaround this by pulling `arg1` into a local variable, like it's done for
> further arguments (arg2, arg3...)
> 
> Originally arg1 wasn't pulled into a variable because for the a1_ register
> local variable the possible clobbering as a result of operators on other
> variables don't matter:
> 
> https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html#Local-Register-Variables
> 
> Note clang version 3.8.1 seems to already be fixed and don't require the
> workaround, but since it's harmless do it uniformly everywhere.
> 
> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Fixes: 2ce562b2a413 ('x86/altcall: use a union as register type for function parameters on clang')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

I'm okay with this change, but since you don't mention anything in this
regard: Did you look at whether / how generated code (with gcc) changes?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 14:30:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 14:30:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686667.1068950 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKx8-0004JG-31; Wed, 28 Feb 2024 14:30:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686667.1068950; Wed, 28 Feb 2024 14:30:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKx7-0004Ix-Tv; Wed, 28 Feb 2024 14:30:33 +0000
Received: by outflank-mailman (input) for mailman id 686667;
 Wed, 28 Feb 2024 14:30:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=LpFc=KF=ens-lyon.org=samuel.thibault@bounce.ens-lyon.org>)
 id 1rfKx6-0004IV-Od
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 14:30:32 +0000
Received: from sonata.ens-lyon.org (sonata.ens-lyon.org [140.77.166.138])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id eb2008fc-d645-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 15:30:29 +0100 (CET)
Received: from localhost (localhost [127.0.0.1])
 by sonata.ens-lyon.org (Postfix) with ESMTP id 898A7A0337;
 Wed, 28 Feb 2024 15:30:27 +0100 (CET)
Received: from sonata.ens-lyon.org ([127.0.0.1])
 by localhost (sonata.ens-lyon.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id cN4d9Sti-1jD; Wed, 28 Feb 2024 15:30:27 +0100 (CET)
Received: from begin (aamiens-653-1-111-57.w83-192.abo.wanadoo.fr
 [83.192.234.57])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest
 SHA256) (No client certificate requested)
 by sonata.ens-lyon.org (Postfix) with ESMTPSA id 3D3A4A032E;
 Wed, 28 Feb 2024 15:30:27 +0100 (CET)
Received: from samy by begin with local (Exim 4.97)
 (envelope-from <samuel.thibault@ens-lyon.org>)
 id 1rfKx0-000000029w5-2xGp; Wed, 28 Feb 2024 15:30:26 +0100
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: eb2008fc-d645-11ee-a1ee-f123f15fe8a2
Date: Wed, 28 Feb 2024 15:30:26 +0100
From: Samuel Thibault <samuel.thibault@ens-lyon.org>
To: Juergen Gross <jgross@suse.com>
Cc: minios-devel@lists.xenproject.org, xen-devel@lists.xenproject.org,
	wl@xen.org
Subject: Re: [PATCH] Mini-OS: add symbol exports for xenstore stubdom
Message-ID: <20240228143026.sen3xzazvpqbmggp@begin>
Mail-Followup-To: Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Juergen Gross <jgross@suse.com>, minios-devel@lists.xenproject.org,
	xen-devel@lists.xenproject.org, wl@xen.org
References: <20240226083955.19350-1-jgross@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240226083955.19350-1-jgross@suse.com>
Organization: I am not organized
User-Agent: NeoMutt/20170609 (1.8.3)

Juergen Gross, le lun. 26 févr. 2024 09:39:55 +0100, a ecrit:
> Xenstore stubdom needs some more symbols exported.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Thanks!

> ---
>  xenbus.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/xenbus.c b/xenbus.c
> index eb9af055..686428a4 100644
> --- a/xenbus.c
> +++ b/xenbus.c
> @@ -45,6 +45,7 @@
>  #endif
>  
>  struct xenstore_domain_interface *xenstore_buf;
> +EXPORT_SYMBOL(xenstore_buf);
>  static DECLARE_WAIT_QUEUE_HEAD(xb_waitq);
>  DECLARE_WAIT_QUEUE_HEAD(xenbus_watch_queue);
>  static __DECLARE_SEMAPHORE_GENERIC(xb_write_sem, 1);
> @@ -70,6 +71,7 @@ static struct xenbus_req_info req_info[NR_REQS];
>  static char *errmsg(struct xsd_sockmsg *rep);
>  
>  uint32_t xenbus_evtchn;
> +EXPORT_SYMBOL(xenbus_evtchn);
>  
>  #ifdef CONFIG_PARAVIRT
>  void get_xenbus(void *p)
> -- 
> 2.35.3
> 

-- 
Samuel
---
Pour une évaluation indépendante, transparente et rigoureuse !
Je soutiens la Commission d'Évaluation de l'Inria.


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 14:31:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 14:31:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686671.1068959 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKxV-0004nZ-7z; Wed, 28 Feb 2024 14:30:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686671.1068959; Wed, 28 Feb 2024 14:30:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfKxV-0004nS-4Z; Wed, 28 Feb 2024 14:30:57 +0000
Received: by outflank-mailman (input) for mailman id 686671;
 Wed, 28 Feb 2024 14:30:56 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=n4Dg=KF=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1rfKxU-0004IV-Rd
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 14:30:56 +0000
Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com
 [2607:f8b0:4864:20::1034])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id fa97262d-d645-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 15:30:54 +0100 (CET)
Received: by mail-pj1-x1034.google.com with SMTP id
 98e67ed59e1d1-29996cc6382so3973408a91.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 06:30:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fa97262d-d645-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709130653; x=1709735453; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=E0VlonLqhBdA+xA9i99/449MrxyCIN089k/qvTq0NHE=;
        b=IAPIqYVaaYnl5e4E7ffH7GVQFpvzMMfPaIwup1d6chqmGGCCJMTwwKN5A71/geDwXj
         646oBqL1ielIde77ivSxnBGd6rcM+MmM75z3r9RUsVLSo4Pej5YBpEpZeFpbq9ooGAUl
         BywV4JvU/8u3Hn8ObWtxq05tJ+Ut3dJCWP47A=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709130653; x=1709735453;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=E0VlonLqhBdA+xA9i99/449MrxyCIN089k/qvTq0NHE=;
        b=cy2zTao13s/TLvRhXGHDwSg0DCDi8SY/LYWkUQBcn2ERdMzCmpFpwpm6T5novMz7Mm
         N0vjJ/qzCbIQ0phsuvh54xmIxxWeoE3XqfcVgxdqfM4HQ1Lb/0By7MpyAWgNtEEdoQHi
         X3YGzdgB8a7Zy23aal8n3verI+cEpHxT6DhEFQVMBk6IHihV9F/BvH3/duSwrgQ0VOO5
         wUndz3umdTdhGy7OMBfGN65kzdPhpDtH+tHBbUFJdO0gxwqFrpBhYQxOFG2LGwDkwcIj
         DIY+AlMrzgNnqpW1j/GvrEosKZ4gaIkphsEcUKl3jO0ir63c+ARzUyIbBgvXsyI3SnDr
         TtIQ==
X-Gm-Message-State: AOJu0YzhLNvXt2pmrJYV6JNigw0nGhW1XlphF032HP0j0gk/QaKmCMZC
	Z4/8veZ5FH9291Up3gv0YZNf7pEkwdDy0u6AD/SGgAQJFlVn7N0MkSMKs/50h/Q+smeeKib+Q+f
	IX8y+/EOmZQADFJ9F7i2LtA75fxYY+Uwr1iL8
X-Google-Smtp-Source: AGHT+IEJNCr2dmwBrf8wBcx3mE+U/a/HOse1dc+CY8inY2XU3sKpglxxscJnJiJuklSL3Yvur0zcl1pkoPfcZLkTANU=
X-Received: by 2002:a17:90a:88c:b0:299:3f10:20ba with SMTP id
 v12-20020a17090a088c00b002993f1020bamr11035825pjc.27.1709130653320; Wed, 28
 Feb 2024 06:30:53 -0800 (PST)
MIME-Version: 1.0
References: <20240227112528.4540-1-roger.pau@citrix.com> <20240227112528.4540-6-roger.pau@citrix.com>
In-Reply-To: <20240227112528.4540-6-roger.pau@citrix.com>
From: Ross Lagerwall <ross.lagerwall@citrix.com>
Date: Wed, 28 Feb 2024 14:30:43 +0000
Message-ID: <CAG7k0EqQuqU5eyGRoZ2r4uwUuXWKZynN09ePmnVXH_32cWE_og@mail.gmail.com>
Subject: Re: [PATCH v2 5/5] xen/livepatch: group and document payload hooks
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 27, 2024 at 11:25=E2=80=AFAM Roger Pau Monne <roger.pau@citrix.=
com> wrote:
>
> Group the payload hooks between the pre/post handlers, and the apply/reve=
rt
> replacements.  Also attempt to comment the context in which the hooks are
> executed.
>
> No functional change.
>
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
> Changes since v1:
>  - New in this version.
> ---
>  xen/include/xen/livepatch_payload.h | 37 ++++++++++++++++++++++-------
>  1 file changed, 29 insertions(+), 8 deletions(-)
>
> diff --git a/xen/include/xen/livepatch_payload.h b/xen/include/xen/livepa=
tch_payload.h
> index b9cd4f209670..472d6a4a63c1 100644
> --- a/xen/include/xen/livepatch_payload.h
> +++ b/xen/include/xen/livepatch_payload.h
> @@ -82,6 +82,8 @@ struct payload {
>   * collision.  Since multiple hooks can be registered, the
>   * .livepatch.hook.load section is a table of functions that will be
>   * executed in series by the livepatch infrastructure at patch load time=
.
> + *
> + * Note the load hook is executed in quiesced context.
>   */
>  #define LIVEPATCH_LOAD_HOOK(_fn) \
>      livepatch_loadcall_t *__weak \
> @@ -96,14 +98,20 @@ struct payload {
>       livepatch_unloadcall_t *__weak \
>          const livepatch_unload_data_##_fn __section(".livepatch.hooks.un=
load") =3D _fn;
>
> +/*
> + * Pre/Post action hooks.
> + *
> + * This hooks are executed before or after the livepatch application. Pr=
e hooks
> + * can veto the application/revert of the livepatch.  They are not execu=
ted in
> + * quiesced context.  All of pre and post hooks are considered vetoing, =
and
> + * hence filling any of those will block the usage of the REPLACE action=
.
> + *
> + * Each of the hooks below can only be set once per livepatch payload.
> + */
>  #define LIVEPATCH_PREAPPLY_HOOK(_fn) \
>      livepatch_precall_t *__attribute__((weak, used)) \
>          const livepatch_preapply_data_##_fn __section(".livepatch.hooks.=
preapply") =3D _fn;
>
> -#define LIVEPATCH_APPLY_HOOK(_fn) \
> -    livepatch_actioncall_t *__attribute__((weak, used)) \
> -        const livepatch_apply_data_##_fn __section(".livepatch.hooks.app=
ly") =3D _fn;
> -
>  #define LIVEPATCH_POSTAPPLY_HOOK(_fn) \
>      livepatch_postcall_t *__attribute__((weak, used)) \
>          const livepatch_postapply_data_##_fn __section(".livepatch.hooks=
.postapply") =3D _fn;
> @@ -112,14 +120,27 @@ struct payload {
>      livepatch_precall_t *__attribute__((weak, used)) \
>          const livepatch_prerevert_data_##_fn __section(".livepatch.hooks=
.prerevert") =3D _fn;
>
> -#define LIVEPATCH_REVERT_HOOK(_fn) \
> -    livepatch_actioncall_t *__attribute__((weak, used)) \
> -        const livepatch_revert_data_##_fn __section(".livepatch.hooks.re=
vert") =3D _fn;
> -
>  #define LIVEPATCH_POSTREVERT_HOOK(_fn) \
>      livepatch_postcall_t *__attribute__((weak, used)) \
>          const livepatch_postrevert_data_##_fn __section(".livepatch.hook=
s.postrevert") =3D _fn;
>
> +/*
> + * Action replacement hooks.
> + *
> + * The following hooks replace the hypervisor implementation for the liv=
epatch
> + * application and revert routines.  When filling the hooks below the na=
tive
> + * apply and revert routines will not be executed, so the provided hooks=
 need
> + * to make sure the state of the payload after apply or revert is as exp=
ected
> + * by the livepatch logic.
> + */
> +#define LIVEPATCH_APPLY_HOOK(_fn) \
> +    livepatch_actioncall_t *__attribute__((weak, used)) \
> +        const livepatch_apply_data_##_fn __section(".livepatch.hooks.app=
ly") =3D _fn;
> +
> +#define LIVEPATCH_REVERT_HOOK(_fn) \
> +    livepatch_actioncall_t *__attribute__((weak, used)) \
> +        const livepatch_revert_data_##_fn __section(".livepatch.hooks.re=
vert") =3D _fn;
> +
>  #endif /* __XEN_LIVEPATCH_PAYLOAD_H__ */
>

LGTM, thanks.

Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 14:33:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 14:33:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686677.1068969 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfL04-0005we-Jq; Wed, 28 Feb 2024 14:33:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686677.1068969; Wed, 28 Feb 2024 14:33:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfL04-0005wX-HC; Wed, 28 Feb 2024 14:33:36 +0000
Received: by outflank-mailman (input) for mailman id 686677;
 Wed, 28 Feb 2024 14:33:35 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=+A+G=KF=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfL03-0005wR-Lq
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 14:33:35 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 59f297c6-d646-11ee-afd8-a90da7624cb6;
 Wed, 28 Feb 2024 15:33:34 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id CE10D4EE0737;
 Wed, 28 Feb 2024 15:33:33 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 59f297c6-d646-11ee-afd8-a90da7624cb6
MIME-Version: 1.0
Date: Wed, 28 Feb 2024 15:33:33 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>, George
 Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH] xen: cache clearing and invalidation helpers
 refactoring
In-Reply-To: <b8bb4534-b11c-4fa6-a617-0460f494a626@suse.com>
References: <cc6bf44701c808645c69bacaf4463295e2cb0fba.1708354388.git.nicola.vetrini@bugseng.com>
 <4f3b3d52a3ba20b05ecfa068b916b804@bugseng.com>
 <b8bb4534-b11c-4fa6-a617-0460f494a626@suse.com>
Message-ID: <4b8015c51f93ae05ad0748832e06cf8b@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-26 11:51, Jan Beulich wrote:
> On 23.02.2024 08:58, Nicola Vetrini wrote:
>> On 2024-02-19 16:14, Nicola Vetrini wrote:
>>> The cache clearing and invalidation helpers in x86 and Arm didn't
>>> comply with MISRA C Rule 17.7: "The value returned by a function
>>> having non-void return type shall be used". On Arm they
>>> were always returning 0, while some in x86 returned -EOPNOTSUPP
>>> and in common/grant_table the return value is saved.
>>> 
>>> As a consequence, a common helper arch_grant_cache_flush that returns
>>> an integer is introduced, so that each architecture can choose 
>>> whether
>>> to
>>> return an error value on certain conditions, and the helpers have
>>> either
>>> been changed to return void (on Arm) or deleted entirely (on x86).
>>> 
>>> Signed-off-by: Julien Grall <julien@xen.org>
>>> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
>>> ---
>>> The original refactor idea came from Julien Grall in [1]; I edited 
>>> that
>>> proposal
>>> to fix build errors.
>>> 
>>> I did introduce a cast to void for the call to flush_area_local on 
>>> x86,
>>> because
>>> even before this patch the return value of that function wasn't 
>>> checked
>>> in all
>>> but one use in x86/smp.c, and in this context the helper (perhaps
>>> incidentally)
>>> ignored the return value of flush_area_local.
>>> 
>>> [1]
>>> https://lore.kernel.org/xen-devel/09589e8f-77b6-47f7-b5bd-cf485e4b60f1@xen.org/
>>> ---
>>>  xen/arch/arm/include/asm/page.h     | 33 
>>> ++++++++++++++++++-----------
>>>  xen/arch/x86/include/asm/flushtlb.h | 23 ++++++++++----------
>>>  xen/common/grant_table.c            |  9 +-------
>>>  3 files changed, 34 insertions(+), 31 deletions(-)
>>> 
>> 
>> I'll put this patch in the backlog at the moment: too many intricacies
>> while trying to untangle xen/flushtlb from xen/mm.h, and there are
>> easier cases that can be done faster. If someone is interested I can
>> post the partial work I've done so far, even though it doesn't
>> build on x86.
> 
> This
> https://lists.xen.org/archives/html/xen-devel/2024-02/msg01513.html
> may be of interest to you in the context here.
> 

I'll take a look, thanks.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 14:34:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 14:34:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686679.1068980 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfL1A-0006hc-Ue; Wed, 28 Feb 2024 14:34:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686679.1068980; Wed, 28 Feb 2024 14:34:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfL1A-0006hV-Pq; Wed, 28 Feb 2024 14:34:44 +0000
Received: by outflank-mailman (input) for mailman id 686679;
 Wed, 28 Feb 2024 14:34:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=n4Dg=KF=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1rfL19-0006hL-U7
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 14:34:43 +0000
Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com
 [2607:f8b0:4864:20::429])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 81fa8732-d646-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 15:34:42 +0100 (CET)
Received: by mail-pf1-x429.google.com with SMTP id
 d2e1a72fcca58-6e56d594b31so219702b3a.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 06:34:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 81fa8732-d646-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709130880; x=1709735680; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ULSlAjwWXGhSHuHX6N2FXR6xOtwfEQECei5WCk+WRwc=;
        b=PFszrlCbC1PvdFzYSVPlbJ+oRm/nBemKSIwuAsucUC/NgzSSbU1uOEmNn2eeTYiOSW
         dh78zl6v4DHVnn2jfJCtwF/QUK0MnRXpprmRJRZHNh7bOWHtKCOjvOTD1IeuhxFY4YS6
         PiRqQTLqQDrL40z6rd+a7hvKMvr68SD3pIjUg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709130880; x=1709735680;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ULSlAjwWXGhSHuHX6N2FXR6xOtwfEQECei5WCk+WRwc=;
        b=PYD1Xw/LCwKEpXmC3FrjK1K3CXhYzTlFzP1Xe5kGHflGJOKiNAH++W5tt41IAzXaim
         1yP6WcmYhWSOzqgywLTRrK3nKQSoWoXuyG9Jy6WiJjkrYox8RrWYdyW65VFI80V9DuQH
         c+Bx1XuFLlpOiuMvB/+g3ufk85HKK1u/41xWMc/l+KwZp22r0flNFSxNTQocWoz84nIg
         0JvW8DiMzXw+nQQd3W9+IqyRAwo5w8wzUkf/BMpMMwhU1y4dBLxxAevgk+MQJROLhpRL
         CQkGtEE/KD/E8YCKOwyychnTUEhE/lBzqxQHBpHMuaatwfCJjXro9oAfCGSNuhC1vOeV
         8Jwg==
X-Forwarded-Encrypted: i=1; AJvYcCV6Sh/9kalGGD95MuvSBKILPGD5qnkmp+TLH5EQt3mtFlBzvd4SgmYeYlkPXb9SYwId3VQ+XWr35Myxi74TzxC8T/7RDEQRo6tToZLTWPk=
X-Gm-Message-State: AOJu0YyFRmyET8J1EJi0PLDHJriny9K4i6Tq7s89+w+pUm7OrA/90gNZ
	anJhjsjtSQOjTvyF94sx3N2feEZBY01cimCjaoEgiGJLt9692SPJ/0adPxHHaJnJmWElpjKl2QC
	blRE6PJRuqlzA2Srod+zv1T1Tx66Op9dkK93s2RFeoxYCRdxCIXRt
X-Google-Smtp-Source: AGHT+IF1r0nIc/BifAyhN/P28DHFrRL6E2dBy5ugOhljUKrfcMZYVvHTIwO3zE8vsqmhBOTbll6cVVlYAZCa3u7pa1k=
X-Received: by 2002:a17:90b:14a:b0:29a:6c13:f0a with SMTP id
 em10-20020a17090b014a00b0029a6c130f0amr10654397pjb.16.1709130880490; Wed, 28
 Feb 2024 06:34:40 -0800 (PST)
MIME-Version: 1.0
References: <20240227112528.4540-1-roger.pau@citrix.com> <20240227112528.4540-2-roger.pau@citrix.com>
 <e8ffdaa2-0786-4f9b-b4c4-03fe88e5c72b@citrix.com>
In-Reply-To: <e8ffdaa2-0786-4f9b-b4c4-03fe88e5c72b@citrix.com>
From: Ross Lagerwall <ross.lagerwall@citrix.com>
Date: Wed, 28 Feb 2024 14:34:29 +0000
Message-ID: <CAG7k0Eq8V3v6Jc3Z5d=mxzWuJ6DSHW5XZ1=D5MCw67W2LDY3sw@mail.gmail.com>
Subject: Re: [PATCH v2 1/5] xen/livepatch: register livepatch regions when loaded
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Roger Pau Monne <roger.pau@citrix.com>, xen-devel@lists.xenproject.org, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, George Dunlap <george.dunlap@citrix.com>, 
	Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>, 
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 27, 2024 at 1:08=E2=80=AFPM Andrew Cooper <andrew.cooper3@citri=
x.com> wrote:
>
> On 27/02/2024 11:25 am, Roger Pau Monne wrote:
> > diff --git a/xen/common/virtual_region.c b/xen/common/virtual_region.c
> > index ddac5c9147e5..e3a4dc8540df 100644
> > --- a/xen/common/virtual_region.c
> > +++ b/xen/common/virtual_region.c
> > <snip>
> >
> > +#ifdef CONFIG_LIVEPATCH
> >  void unregister_virtual_region(struct virtual_region *r)
> >  {
> > -    /* Expected to be called from Live Patch - which has IRQs disabled=
. */
> > -    ASSERT(!local_irq_is_enabled());
> > -
> >      remove_virtual_region(r);
> > +
> > +    /* Assert that no CPU might be using the removed region. */
> > +    rcu_barrier();
> >  }
> >
> > -#if defined(CONFIG_LIVEPATCH) && defined(CONFIG_X86)
> > +#ifdef CONFIG_X86
> >  void relax_virtual_region_perms(void)
> >  {
> >      const struct virtual_region *region;
> > @@ -117,6 +106,7 @@ void tighten_virtual_region_perms(void)
> >      rcu_read_unlock(&rcu_virtual_region_lock);
> >  }
> >  #endif
> > +#endif
>
> Can we please have:
>
> #endif /* CONFIG_X86 */
> #endif /* CONFIG_LIVEPATCH */
>
> before this gets any more complicated to follow.  Can be fixed on commit.
>

LGTM with this suggested change.

Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 14:48:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 14:48:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686684.1068989 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLEF-000151-1Z; Wed, 28 Feb 2024 14:48:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686684.1068989; Wed, 28 Feb 2024 14:48:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLEE-00014u-Ua; Wed, 28 Feb 2024 14:48:14 +0000
Received: by outflank-mailman (input) for mailman id 686684;
 Wed, 28 Feb 2024 14:48:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfLED-00014o-Ox
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 14:48:13 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 654e27e3-d648-11ee-afd8-a90da7624cb6;
 Wed, 28 Feb 2024 15:48:12 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a3d484a58f6so791806466b.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 06:48:12 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 lf7-20020a170907174700b00a4316384159sm1912315ejc.224.2024.02.28.06.48.11
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 06:48:11 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 654e27e3-d648-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709131692; x=1709736492; darn=lists.xenproject.org;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8T6SqAr8zRjYU0xw/FLOnzZ8QZ6ZQnxq79TKnjkGfqk=;
        b=YgWsPuF26+YbI64fTVyvvx/9Pe6LnYS23k80Qsow+zVmHxm05bduQ64tKZzBUWJkyp
         7/qhXjYLbQO2kxjpe2pe716WGp9mWQhOqpSbodx8iIY+zAZuh/SxOH1RcIUiLc04B6+f
         SSGFPhXX2WIU4msB0lkz6FngwD0fVMUVBMKqg/XUD8rhcpH7RWWrV69dpDGmknegQ44R
         51kKb8xQES0nY9Mp/a/r2cvE/aLDcukFiyB09To6ORcBzijMaN3p6HME9FgF3yMHdZ60
         0yTc0sPZp3J/TihWDB/MWvIQmQeWyJ4ruFMlINnIy6aTEkGgQXQsBP01QCvDzypJ89xi
         S6Cw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709131692; x=1709736492;
        h=content-transfer-encoding:autocrypt:subject:from:cc:to
         :content-language:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8T6SqAr8zRjYU0xw/FLOnzZ8QZ6ZQnxq79TKnjkGfqk=;
        b=jBDqDTwW/7cCwlZd5GheFpKAVSsFUoUjg68U/uD/hK+RXwjWaNchPOqZ6PEMKpazbO
         AGv1WwVQSWbRh7QCGHJsdGYfiETLRoDwQhIM+TEcjHPbE/iRrOzg7Is6/tqXBIFbUevN
         Z5njAz1yAoV7aZ4+dPUqNRSPGT4baV+tMVzIijFIGat050RuXwQ1JQourRfi/Tv9XctY
         ehm//29X+f0PndY8f+P+NzvMTN+ICyBBZTQyn3ZM1eXh30P1TnTLcCBM7wrobl9ha1zL
         AC8k29IU9ftS5JqgPa7hB6HmQFQn3qlXpMYsCy1tHkwXfwGeW/XJLml+mSdDLTpXi8cW
         irBQ==
X-Gm-Message-State: AOJu0YwA/DdM2C/9u/mBORDyziuDclSGUtT/7vgJTP7wgLMbZklOsoR/
	kj1/b7K6Vdtql7kCDM9RT1NRS08wFM+D2omLo6RiJs9SHCDUBJrjSTvB+9ovf9+ebBzQKcIeCaU
	=
X-Google-Smtp-Source: AGHT+IHIftzain54YhWj9ebHX9KxhqYls7nqAQPwBaAoCPecTYqGy4pwmjZoTEt7VRxpARLXJEwdgA==
X-Received: by 2002:a17:906:ce30:b0:a3f:bd94:4d80 with SMTP id sd16-20020a170906ce3000b00a3fbd944d80mr8515992ejb.76.1709131691832;
        Wed, 28 Feb 2024 06:48:11 -0800 (PST)
Message-ID: <58656398-2d64-48b8-9ddc-c6836847a586@suse.com>
Date: Wed, 28 Feb 2024 15:48:10 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Subject: [PATCH] x86: serializing of non-serializing MSR writes
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Linux commit 25a068b8e9a4e ("x86/apic: Add extra serialization for non-
serializing MSRs") explains why an MFENCE+LFENCE pair is generally
needed ahead of ICR writes in x2APIC mode, and also why at least in
theory such is also needed ahead of TSC_DEADLINE writes. A comment of
our own in send_IPI_mask_x2apic_phys() further explains a condition
under which the LFENCE can be avoided.

Further Linux commit 04c3024560d3 ("x86/barrier: Do not serialize MSR
accesses on AMD") explains that this barrier isn't needed on AMD or
Hygon, and is in fact hampering performance in a measurable way.

Introduce a similarly named helper function, but with a parameter
allowing callers to specify whether a memory access will follow, thus
permitting the LFENCE to be omitted.

Putting an instance in apic_wait_icr_idle() is to be on the safe side.
The one case where it was clearly missing is in send_IPI_shortcut(),
which is also used in x2APIC mode when called from send_IPI_mask().

Function comment shamelessly borrowed (but adapted) from Linux.

Fixes: 5500d265a2a8 ("x86/smp: use APIC ALLBUT destination shorthand when possible")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
I question the need for a fence ahead of writing TSC_DEADLINE: The Linux
commit message talks about LVT being accessed via MMIO in xAPIC mode,
but that should not be relevant here: It's all the local CPU, so there
ought to not be visibility concerns (much like for a self-IPI no fence
is needed ahead of the ICR write). If that wasn't needed, we could
further use alternatives patching to remove the fence also from
apic_wait_icr_idle() when in xAPIC mode. (And only then would I agree to
have APIC in the feature identifier, like Linux has it.)

A number of apic_write() may better be turned into apic_mem_write(), in
particular e.g. the ones in send_IPI_mask_{flat,phys}(). That way it
would be quite a bit easier to spot paths taken only in xAPIC mode.

The INIT-INIT-SIPI sequence for AP startup doesn't use any barrier, also
not in Linux afaics. I can't explain the lack thereof, though.

--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -1309,6 +1309,12 @@ int reprogram_timer(s_time_t timeout)
 
     if ( tdt_enabled )
     {
+        /*
+         * WRMSR to TSC_DEADLINE is not serializing.  We could pass @timeout
+         * here, but the passed value is preferably compile-time-constant.
+         */
+        weak_wrmsr_fence(false);
+
         wrmsrl(MSR_IA32_TSC_DEADLINE, timeout ? stime2tsc(timeout) : 0);
         return 1;
     }
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -675,8 +675,12 @@ void amd_log_freq(const struct cpuinfo_x
 
 void cf_check early_init_amd(struct cpuinfo_x86 *c)
 {
-	if (c == &boot_cpu_data)
+	if (c == &boot_cpu_data) {
+		/* No fencing needed ahead of certain MSR writes. */
+		setup_force_cpu_cap(X86_FEATURE_NO_WRMSR_FENCE);
+
 		amd_init_levelling();
+	}
 
 	ctxt_switch_levelling(NULL);
 }
--- a/xen/arch/x86/genapic/x2apic.c
+++ b/xen/arch/x86/genapic/x2apic.c
@@ -97,15 +97,15 @@ static void cf_check send_IPI_mask_x2api
 
     /*
      * Ensure that any synchronisation data written in program order by this
-     * CPU is seen by notified remote CPUs. The WRMSR contained within
-     * apic_icr_write() can otherwise be executed early.
+     * CPU is seen by notified remote CPUs. The WRMSR contained in the loop
+     * below can otherwise be executed early.
      * 
-     * The reason smp_mb() is sufficient here is subtle: the register arguments
+     * The reason MFENCE is sufficient here is subtle: the register arguments
      * to WRMSR must depend on a memory read executed after the barrier. This
      * is guaranteed by cpu_physical_id(), which reads from a global array (and
      * so cannot be hoisted above the barrier even by a clever compiler).
      */
-    smp_mb();
+    weak_wrmsr_fence(true);
 
     local_irq_save(flags);
 
@@ -130,7 +130,7 @@ static void cf_check send_IPI_mask_x2api
     const cpumask_t *cluster_cpus;
     unsigned long flags;
 
-    smp_mb(); /* See above for an explanation. */
+    weak_wrmsr_fence(true); /* See above for an explanation. */
 
     local_irq_save(flags);
 
--- a/xen/arch/x86/include/asm/cpufeatures.h
+++ b/xen/arch/x86/include/asm/cpufeatures.h
@@ -24,7 +24,7 @@ XEN_CPUFEATURE(APERFMPERF,        X86_SY
 XEN_CPUFEATURE(MFENCE_RDTSC,      X86_SYNTH( 9)) /* MFENCE synchronizes RDTSC */
 XEN_CPUFEATURE(XEN_SMEP,          X86_SYNTH(10)) /* SMEP gets used by Xen itself */
 XEN_CPUFEATURE(XEN_SMAP,          X86_SYNTH(11)) /* SMAP gets used by Xen itself */
-/* Bit 12 unused. */
+XEN_CPUFEATURE(NO_WRMSR_FENCE,    X86_SYNTH(12)) /* No MFENCE{,+LFENCE} ahead of certain WRMSR. */
 XEN_CPUFEATURE(IND_THUNK_LFENCE,  X86_SYNTH(13)) /* Use IND_THUNK_LFENCE */
 XEN_CPUFEATURE(IND_THUNK_JMP,     X86_SYNTH(14)) /* Use IND_THUNK_JMP */
 XEN_CPUFEATURE(SC_NO_BRANCH_HARDEN, X86_SYNTH(15)) /* (Disable) Conditional branch hardening */
--- a/xen/arch/x86/include/asm/msr.h
+++ b/xen/arch/x86/include/asm/msr.h
@@ -97,6 +97,25 @@ static inline void msr_split(struct cpu_
     regs->rax = (uint32_t)val;
 }
 
+/*
+ * Make previous memory operations globally visible before a WRMSR.  Most
+ * WRMSRs are full serializing instructions themselves and do not require this
+ * barrier.  This may only be required for the TSC_DEADLINE and x2APIC MSRs.
+ *
+ * MFENCE makes writes visible, but only affects load/store instructions.
+ * WRMSR is unfortunately not a load/store instruction and is unaffected by
+ * MFENCE.  The LFENCE ensures that the WRMSR is not reordered, but callers
+ * can indicate to avoid it when they have a suitable memory access between
+ * the invocation of this function and the WRMSR in question.
+ */
+static inline void weak_wrmsr_fence(bool have_mem_access)
+{
+    alternative("mfence", "", X86_FEATURE_NO_WRMSR_FENCE);
+
+    if ( !have_mem_access )
+        alternative("lfence", "", X86_FEATURE_NO_WRMSR_FENCE);
+}
+
 static inline uint64_t rdtsc(void)
 {
     uint32_t low, high;
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -39,7 +39,10 @@ static unsigned int prepare_ICR2(unsigne
 void apic_wait_icr_idle(void)
 {
     if ( x2apic_enabled )
+    {
+        weak_wrmsr_fence(false);
         return;
+    }
 
     while ( apic_read(APIC_ICR) & APIC_ICR_BUSY )
         cpu_relax();


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 14:52:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 14:52:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686688.1069000 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLIk-0003ZR-ND; Wed, 28 Feb 2024 14:52:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686688.1069000; Wed, 28 Feb 2024 14:52:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLIk-0003ZK-J5; Wed, 28 Feb 2024 14:52:54 +0000
Received: by outflank-mailman (input) for mailman id 686688;
 Wed, 28 Feb 2024 14:52:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfLIi-0003ZE-Ko
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 14:52:52 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0b58672b-d649-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 15:52:50 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a440b1c445eso101594066b.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 06:52:50 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 p4-20020a170906614400b00a3f4bafa6fbsm1903065ejl.168.2024.02.28.06.52.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 06:52:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b58672b-d649-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709131970; x=1709736770; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=0cAx2wtEdscxtqJw6koZOCpPdjA45RO8nsZSMPNeIa4=;
        b=Y4JDYgWjghHr8NPR0z+9GHFBo2I5wcNNLJUk9LnJQ4dSCzQZPC55h4Mdkpe0lReEC8
         utvO5GKrXoLPSwcgdLMPQ85h4E/Xhd7tBerD3N+2EvSlyIxlZmOUYKSa+LJ89sYaPpjH
         ECGSL1218xVli8NNaH3yAnudoMj9kvdgFMfkIEw4jitcDOyTHVLcn8kLScMl4WCOvf9P
         DFlqHb8O0f5iLFJnMpXDkE4MfD297OETaCp2uTNz6BDFASSpVaFWKLc+jOi1+71eGFQO
         iLW+Im683FoWw17Axmd04fwWo5PXoz+Jxbk2NTxvomsAMjPsK5lNClMXhOyqd0hyPkcH
         3LaA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709131970; x=1709736770;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=0cAx2wtEdscxtqJw6koZOCpPdjA45RO8nsZSMPNeIa4=;
        b=YGX8zvm3VW34GPhc9MLCRJjj5noJLQckGrYPCjWYhWfPbVF/IUA6g7QJa/zFsJLB+5
         uvYHn+jpoYdd3S7wLKCr8y5LoHCrVW35jXnf/0tvkjDkokXpjl2gGdHApocTae1/nrya
         hIEd15dE0595ztgRFeb/8KCoZ0SBIzM8YGqdCfDiHQAxCW+HMSwz/VzLbTwFREG5Si1L
         3ziTY2gNPt0Rw07XacjqwlvETGuFvrcRO1EuCO4HSO8s0uvRhzNLz21DedYT+6IKBM9O
         om9LFMqYfs/ML06T9laFzGybuxKXngFktOfOG2MGbmyEIkaC/PBsw5kZZTUlUGnX5o7y
         OhrA==
X-Forwarded-Encrypted: i=1; AJvYcCXmCPK9D7EmcY1FQk+NH9RGroIXcN9zYItBmUoXK2bjerw+Xb5AwA+eNDeQkzbMzcKk9tZqyuwoN8vyFgx6Z6UvGB/D/aGaGkcAzkeJsd0=
X-Gm-Message-State: AOJu0YxyfTdyoZUImbnRG4DTWnHihMYRE0+VatIxktDcBLX/sY4tnEK8
	krvNjeuTkDSpGF0i0ewxm5PriNQ+JeWyQsHIMc1owoNqS4H7oaqy5Jio9TrdDFemD0mt/8luZxg
	=
X-Google-Smtp-Source: AGHT+IHUQXZ+VeT9gde4R5AX9cm3TiU/uXgIrEDpepJ8GUJ86S29iSHWw8Xj7K8C6jydPVSgcgKxZg==
X-Received: by 2002:a17:906:792:b0:a43:fec3:b648 with SMTP id l18-20020a170906079200b00a43fec3b648mr1736086ejc.24.1709131970236;
        Wed, 28 Feb 2024 06:52:50 -0800 (PST)
Message-ID: <271ae402-7400-4dc5-9222-88523b9a2922@suse.com>
Date: Wed, 28 Feb 2024 15:52:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Ping: [PATCH v2] Argo: don't obtain excess page references
To: Christopher Clark <christopher.w.clark@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Julien Grall <julien@xen.org>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <79ed633d-b0bd-4a7d-a0c6-37a034e1ee96@suse.com>
 <0374516c-b5fa-4880-a07a-0b788f491e9a@xen.org>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <0374516c-b5fa-4880-a07a-0b788f491e9a@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 18.02.2024 19:01, Julien Grall wrote:
> On 14/02/2024 10:12, Jan Beulich wrote:
>> find_ring_mfn() already holds a page reference when trying to obtain a
>> writable type reference. We shouldn't make assumptions on the general
>> reference count limit being effectively "infinity". Obtain merely a type
>> ref, re-using the general ref by only dropping the previously acquired
>> one in the case of an error.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Julien Grall <jgrall@amazon.com>

I'll give it till the end of the week for an ack to arrive (or a substantial
objection), and commit some time next week in the absence of any response.

Jan



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 14:59:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 14:59:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686690.1069009 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLOy-0004Iy-BP; Wed, 28 Feb 2024 14:59:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686690.1069009; Wed, 28 Feb 2024 14:59:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLOy-0004Ir-8S; Wed, 28 Feb 2024 14:59:20 +0000
Received: by outflank-mailman (input) for mailman id 686690;
 Wed, 28 Feb 2024 14:59:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfLOx-0004Il-Le
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 14:59:19 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f237fd0c-d649-11ee-afd8-a90da7624cb6;
 Wed, 28 Feb 2024 15:59:18 +0100 (CET)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a3ed9cae56fso183180966b.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 06:59:18 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m2-20020a1709062b8200b00a3e72a6bf1csm1959326ejg.14.2024.02.28.06.59.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 06:59:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f237fd0c-d649-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709132358; x=1709737158; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=h1T+kpF3CbFMISAXr63wAGZxjagtLjgT6UxuFBQu/WU=;
        b=UF6THkb5dcZX90Up7/Y2Z2TV9BYWbM9N6wfolqiwQofeuFNxD3ecipa4FNZaPtwkGU
         9gZ7qyHJ32UF5+Soby6WGNvKZruIgi3CgYgfl+dIm7V89INfK0Wb5ZgXv5o73FOTsyz9
         vHvdom3CbBGkWrGbUXO8YNb6SWrZDXOttrHO3OxYPQtSKg07744pU0RLeHmNlJe3W5HT
         oQzWYcrxTq1dUBVAZn+FB+INhHumlD6APYnC68ymlCv0ETlk4rXy3l5lygs9eXhqFDD4
         993qdb+ocqO2lrAp9VsfoDq0sFWSp7USUXXw+QOgR64OpbNtOA65lq5+jD3AECOfBLWb
         tI5A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709132358; x=1709737158;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=h1T+kpF3CbFMISAXr63wAGZxjagtLjgT6UxuFBQu/WU=;
        b=HvU4zim9BhPq84OVWx5cvlp5kNJt6RQGXhlL6VmCrzpG7Sy0N2/bTuvBnmyFAGLMwP
         NSAo+x7+IFc1aeWdlEUfMfJ+/XEJb/wlN/bpiuwaT7MIyCF7bD2Ze3AkFPya9FtfjOBU
         LNGLfJO5FfDjrkG246ZA5oTGEFwN3G4xlebX/+EoJULDlD5wyln1TmAPy6ba5Gkj3/TK
         xwN6EiHCdn7NrMMpyrhBcOpJROGI0GX52X1hS4GRluirjjSGHjXpVR+QAMFVibA7rVk+
         Zq5stk5BBfS11lb0CcB5tkSLtFd4i7ZAGLpscRoEm7aTtTqGUEjit7O1WXGvJcZGvY4K
         5Vyg==
X-Forwarded-Encrypted: i=1; AJvYcCX/1RqE5PaHYzNbz99orWaC6307grkjJRQfQpu3j8ONCYvqr3s6N+chtE2U5viEYN7kMJLRXu3LIdri8ejmtVN4XO7ReegzHpL8s7sjfQQ=
X-Gm-Message-State: AOJu0Yw2zr1dqU4Se52WerECyeTqOhG3IEZVbaTPtCNKe3OuGHO2IzHi
	yc/tmlnvgGlxaCcl/wg65iqsE3JayrkrdZllnFXdP956UkKbPAsbPTmAi/RYcA==
X-Google-Smtp-Source: AGHT+IFjX2hJKYVcJCSl97VfJgRz81X5z73sZGIW9iPKstXm7X/S6N930FllUv9NApy7unvJpQU8DQ==
X-Received: by 2002:a17:907:1b10:b0:a3e:57ed:8b93 with SMTP id mp16-20020a1709071b1000b00a3e57ed8b93mr3301857ejc.19.1709132357937;
        Wed, 28 Feb 2024 06:59:17 -0800 (PST)
Message-ID: <5175ecb0-e6d9-418c-a73c-266d325406a4@suse.com>
Date: Wed, 28 Feb 2024 15:59:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 04/12] xen/spinlock: rename recursive lock functions
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-5-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20231212094725.22184-5-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 12.12.2023 10:47, Juergen Gross wrote:
> Rename the recursive spin_lock() functions by replacing the trailing
> "_recursive" with a leading "r".
> 
> Switch the parameter to be a pointer to rspinlock_t.
> 
> Remove the indirection through a macro, as it is adding only complexity
> without any gain.

Considering we aren't aware of any leveraging of this, doing so is
probably okay. Still I think it was done that way for a reason. Plus
of course if we undo the indirection here, sooner or later we should
also undo similar indirection elsewhere.

> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Jan



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 15:09:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 15:09:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686694.1069019 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLYf-0006CQ-8K; Wed, 28 Feb 2024 15:09:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686694.1069019; Wed, 28 Feb 2024 15:09:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLYf-0006CJ-4e; Wed, 28 Feb 2024 15:09:21 +0000
Received: by outflank-mailman (input) for mailman id 686694;
 Wed, 28 Feb 2024 15:09:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfLYe-0006CD-7M
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 15:09:20 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 583ebc9e-d64b-11ee-afd8-a90da7624cb6;
 Wed, 28 Feb 2024 16:09:18 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a28a6cef709so839164766b.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 07:09:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 tl26-20020a170907c31a00b00a3fb7cafad8sm1950097ejc.39.2024.02.28.07.09.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 07:09:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 583ebc9e-d64b-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709132958; x=1709737758; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=YhZYBZtincd42tPqrvkY4hq/5948OK8ztponalyQ6jU=;
        b=Frhw5RUBd9Tn8lhhSCcBeo+cjTVmYmDiuGj2Iwz68oPmqHOrkKv1plvtxkpj0cUGjL
         T6xt3aHrqvDIhUiy2312cBdY+ySow+fRCi12agNjxAULOIEL+wHbHeBOawRKf5WeFmtX
         0dQZVGFKx8maan6oMjZCoOxaUwxfilmO8Nez1UqQx60WNctW51thKSW1NM3YxHS10e/t
         2afM7LagjivzSHJjLHWVL3nJfdqz0lad5Kv5Tr6zA73Nt92IALnX/0bZpVwlWl/xjw4W
         rdaDd10nMCUgs0iqVCwD39/ZkmuXrAZQ/2RbfqB7FI87oJmc9oQDW7EaGFXH3D8LSMxC
         u0Cw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709132958; x=1709737758;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=YhZYBZtincd42tPqrvkY4hq/5948OK8ztponalyQ6jU=;
        b=iVogyMErvXDF1Uvo4o1PSvv9POT/TLgj/0hDeCaSSIoHjF7p/na4+HJd4s6Dn3zwUb
         2M9rp+qBcqkW+Y30OIknIPT3SMr/O4feg8oM27uKdMLcRRBdNdm+Pf5O9NLJBAavQcPJ
         HNCk5VLz5yA8zxwp8C0Mp79zrMNORZANokmKGZn/Htmtn3/XHArwUVhmVwlkoNjtHrnC
         lcVmt/1ZkoQL16DqUvx1CeOIo2vPju+mzpBB6q829gtOZafXy4RAfHc4DG1mobWiqUpW
         v3UzbUGdn9bbwENbcUyuTnNjCiJRAByii1cZQSYdpBOyEdQrDPtlH+U7VfE22FvVOyA/
         tZEQ==
X-Forwarded-Encrypted: i=1; AJvYcCUu4tOXWYfFPH7ImVGn6GHqLX6PF4C0WLhuQ6Gh2J+EIjB82oEPAM93t7hDmBxghKO2n4oHyNwWG81KJwoaIhK/WDvWtowH6ADHqSOSc3k=
X-Gm-Message-State: AOJu0Yxz7PSiq5bbrp4p/QSJglgRX05KQ0N6u+IgjfzsIDEQEPrVnTb8
	PBX0EChL0Iz5aJJJ21OixOpF4HGqTuZD8M77IXdsjN0iLGT6UqDTlY2IaEksFg==
X-Google-Smtp-Source: AGHT+IHHh5meT+s6XrienMEnMAIQbHI5+pawTXEro+uy6nwugh3YlOXbsgdMWeFaZSp+eVXRciaFpA==
X-Received: by 2002:a17:906:d204:b0:a43:da8a:9ec9 with SMTP id w4-20020a170906d20400b00a43da8a9ec9mr2739154ejz.68.1709132958618;
        Wed, 28 Feb 2024 07:09:18 -0800 (PST)
Message-ID: <411c0f54-67cd-4b12-83ec-218703045707@suse.com>
Date: Wed, 28 Feb 2024 16:09:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 05/12] xen/spinlock: add
 rspin_[un]lock_irq[save|restore]()
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-6-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20231212094725.22184-6-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 12.12.2023 10:47, Juergen Gross wrote:
> Instead of special casing rspin_lock_irqsave() and
> rspin_unlock_irqrestore() for the console lock, add those functions
> to spinlock handling and use them where needed.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
> V2:
> - new patch

In how far is this a necessary part of the series?

> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -647,13 +647,15 @@ void show_stack_overflow(unsigned int cpu, const struct cpu_user_regs *regs)
>  void show_execution_state(const struct cpu_user_regs *regs)
>  {
>      /* Prevent interleaving of output. */
> -    unsigned long flags = console_lock_recursive_irqsave();
> +    unsigned long flags;
> +
> +    rspin_lock_irqsave(&console_lock, flags);
>  
>      show_registers(regs);
>      show_code(regs);
>      show_stack(regs);
>  
> -    console_unlock_recursive_irqrestore(flags);
> +    rspin_unlock_irqrestore(&console_lock, flags);
>  }
>  
>  void cf_check show_execution_state_nonconst(struct cpu_user_regs *regs)
> @@ -663,7 +665,7 @@ void cf_check show_execution_state_nonconst(struct cpu_user_regs *regs)
>  
>  void vcpu_show_execution_state(struct vcpu *v)
>  {
> -    unsigned long flags = 0;
> +    unsigned long flags;
>  
>      if ( test_bit(_VPF_down, &v->pause_flags) )
>      {
> @@ -698,7 +700,7 @@ void vcpu_show_execution_state(struct vcpu *v)
>  #endif
>  
>      /* Prevent interleaving of output. */
> -    flags = console_lock_recursive_irqsave();
> +    rspin_lock_irqsave(&console_lock, flags);
>  
>      vcpu_show_registers(v);
>  
> @@ -708,7 +710,7 @@ void vcpu_show_execution_state(struct vcpu *v)
>           * Stop interleaving prevention: The necessary P2M lookups involve
>           * locking, which has to occur with IRQs enabled.
>           */
> -        console_unlock_recursive_irqrestore(flags);
> +        rspin_unlock_irqrestore(&console_lock, flags);
>  
>          show_hvm_stack(v, &v->arch.user_regs);
>      }
> @@ -717,7 +719,7 @@ void vcpu_show_execution_state(struct vcpu *v)
>          if ( guest_kernel_mode(v, &v->arch.user_regs) )
>              show_guest_stack(v, &v->arch.user_regs);
>  
> -        console_unlock_recursive_irqrestore(flags);
> +        rspin_unlock_irqrestore(&console_lock, flags);
>      }
>  

I view these as layering violations; ...

> --- a/xen/drivers/char/console.c
> +++ b/xen/drivers/char/console.c
> @@ -120,7 +120,7 @@ static int __read_mostly sercon_handle = -1;
>  int8_t __read_mostly opt_console_xen; /* console=xen */
>  #endif
>  
> -static DEFINE_RSPINLOCK(console_lock);
> +DEFINE_RSPINLOCK(console_lock);

... this should remain static. The question therefore just is whether
to omit this patch or ...

> @@ -1158,22 +1158,6 @@ void console_end_log_everything(void)
>      atomic_dec(&print_everything);
>  }
>  
> -unsigned long console_lock_recursive_irqsave(void)
> -{
> -    unsigned long flags;
> -
> -    local_irq_save(flags);
> -    rspin_lock(&console_lock);
> -
> -    return flags;
> -}
> -
> -void console_unlock_recursive_irqrestore(unsigned long flags)
> -{
> -    rspin_unlock(&console_lock);
> -    local_irq_restore(flags);
> -}

... whether to retain these two functions as thin wrappers around the
new, more generic construct.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 15:11:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 15:11:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686697.1069029 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLah-0007lh-JG; Wed, 28 Feb 2024 15:11:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686697.1069029; Wed, 28 Feb 2024 15:11:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLah-0007la-G8; Wed, 28 Feb 2024 15:11:27 +0000
Received: by outflank-mailman (input) for mailman id 686697;
 Wed, 28 Feb 2024 15:11:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfLag-0007lO-RW; Wed, 28 Feb 2024 15:11:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfLag-0004s9-M5; Wed, 28 Feb 2024 15:11:26 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfLag-0004ZN-Al; Wed, 28 Feb 2024 15:11:26 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfLag-0005cN-AF; Wed, 28 Feb 2024 15:11:26 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=JFYkKqNk05KVMgmjbl+fExob/h1hYL28mfFK0kANCRE=; b=GUyDRW9GRapzxFZtGcAUjxDMOc
	iY0CRN4nU+Yi2tsJSZ9x8nUtrt/JhXRVYlKHJfOZBriotFZW/CyqOM/Q8nQfuNDcVbNiSozq5qWwB
	KNuORCNPejq74UrJc/BMWWFiFep/bT54pNtvXHL4QPhBHxoiyBHOAa9KoXwFMNPoyC9U=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184805-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184805: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=b2b3d40f9a030ba79ce3abadccb5da485af33589
X-Osstest-Versions-That:
    xen=801c4e72621d5541d249af1938a77466f0dcfe88
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 15:11:26 +0000

flight 184805 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184805/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  b2b3d40f9a030ba79ce3abadccb5da485af33589
baseline version:
 xen                  801c4e72621d5541d249af1938a77466f0dcfe88

Last test of basis   184798  2024-02-27 19:00:30 Z    0 days
Testing same since   184805  2024-02-28 12:04:00 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Julien Grall <jgrall@amazon.com>
  Michal Orzel <michal.orzel@amd.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Shawn Anastasio <sanastasio@raptorengineering.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   801c4e7262..b2b3d40f9a  b2b3d40f9a030ba79ce3abadccb5da485af33589 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 15:16:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 15:16:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686701.1069039 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLfj-0000oW-5F; Wed, 28 Feb 2024 15:16:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686701.1069039; Wed, 28 Feb 2024 15:16:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLfj-0000oP-20; Wed, 28 Feb 2024 15:16:39 +0000
Received: by outflank-mailman (input) for mailman id 686701;
 Wed, 28 Feb 2024 15:16:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eIhW=KF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfLfi-0000oJ-CI
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 15:16:38 +0000
Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com
 [2607:f8b0:4864:20::72b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5cbbbfec-d64c-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 16:16:36 +0100 (CET)
Received: by mail-qk1-x72b.google.com with SMTP id
 af79cd13be357-787c0f5eaa2so275079185a.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 07:16:36 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 ow7-20020a05620a820700b00787b8e49d9fsm4659816qkn.14.2024.02.28.07.16.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 07:16:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5cbbbfec-d64c-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709133395; x=1709738195; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=8RvzjC1RKZCdRo7Bz7AsdUZ6IYrXYuGXtUjIBEhg3wU=;
        b=LVyvl23vzVkrg4QYEkkUJT24BXV4TBNKWoV0lm8rQXP/GYWNAnHDtLruew8509QV+E
         23VoGDmyZY3CJgwXeLU6zk0sK7IkgH+anv+wQjk1ulpxcla2CsKQjjOzpHzTJxQh58wj
         kWhkrqaJa1sXDegxvPPVuzpW/wRZAsB2XK4sA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709133395; x=1709738195;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=8RvzjC1RKZCdRo7Bz7AsdUZ6IYrXYuGXtUjIBEhg3wU=;
        b=XJWCPglZTGeH1lZ4P4VC4r1fKgdsgsdxkaOe+Q7jkVNzgljlWxJy20b00ZmuVBhSG8
         cIaJWeHaU0iKPaeBQt3kLWLfns2K6zzrVwz5cQhjL15eTl3nmDC4neNxXdedeCvwzuCi
         zcrp1PyYo/qZn5hAxe581X0lqdJHl0pPO5tdeaoKChizeVFc3aypq/vsxx+KL6oXNOJC
         Af9wd9S85UCbRhp1HNlYQ0P41xCjGp+C6m5sU6KKXJQzY9U4R8J+GpdTvvFC4iLNL3Wh
         d4togPdfIUAnTIADGu3CTD+6IoeuyHjS13Hvx78sotfhWOUaAbuzyB1o/8Ghswy+sqrr
         NeuA==
X-Forwarded-Encrypted: i=1; AJvYcCXjwqf9BjiQ5QKuscZHO1/EJReKCmELsAMY3yvuToZeT62MpNwQbJ5/zLp7fuPvztJ/vjMpgDf1tyd+FYPqKn5MMNcWhrbJTJgPtLSyiHs=
X-Gm-Message-State: AOJu0YyNZnzOJHN5v7AKh58SmwhvFEPzvwtQXawPY16vd7XCtW2fktUB
	YZo9VtRV/ZgvsZ1b2bOXo4k7b2Y0vCDkGilt5yiUC28nJuoYqDj+2MQIHLCiDxA=
X-Google-Smtp-Source: AGHT+IHZaWXYDNFOLkkZneUGN0O+s//Jead9SHjdCT+6ouFWFgs2aavSzQ6Uheavv48CMGVtBkRhoQ==
X-Received: by 2002:a05:620a:5641:b0:787:a0f4:dd3 with SMTP id vw1-20020a05620a564100b00787a0f40dd3mr5703793qkn.4.1709133395108;
        Wed, 28 Feb 2024 07:16:35 -0800 (PST)
Message-ID: <320ca33d-2c40-417e-874c-8af447c43734@citrix.com>
Date: Wed, 28 Feb 2024 15:16:32 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/4] x86: record SSP at non-guest entry points
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
 <0ad4543b-8eed-4147-b32d-b68d21fade98@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <0ad4543b-8eed-4147-b32d-b68d21fade98@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/02/2024 1:52 pm, Jan Beulich wrote:
> We will want to use that value for call trace generation, and likely
> also to eliminate the somewhat fragile shadow stack searching done in
> fixup_exception_return(). For those purposes, guest-only entry points do
> not need to record that value.
>
> To keep the saving code simple, record our own SSP that corresponds to
> an exception frame, pointing to the top of the shadow stack counterpart
> of what the CPU has saved on the regular stack. Consuming code can then
> work its way from there.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> To record the full 64-bit value, some of the unused bits in the %cs slot
> could be used. Sadly that slot has saved_upcall_mask in an unhelpful
> location, otherwise simply storing low and high 32 bits in those two
> separate half-slots would be a pretty obvious choice. As long as
> "entry_ssp" is used in non-guest-entry frames only, we could of course
> put half of it into a union with saved_upcall_mask ...
>
> Else may want to put a BUILD_BUG_ON(VADDR_BITS > 48) somewhere, but I'm
> afraid I can't really identify a good place for such to live.

Perhaps in reinit_bsp_stack() when we enable SHSTK on the BSP?

Having it anywhere vaguely relevant is better than not having it.

>
> Leveraging that the CPU stores zero in the upper bits of the selector
> register slots, the save sequence could also be
>
>         shl   $16, %rcx
>         or    %rcx, UREGS_entry_ssp-2(%rsp)
>
> That's shorter and avoids a 16-bit operation, but may be less desirable,
> for being a read-modify-write access.

I doubt you'll be able to measure a difference between the two options
(it's into the active stack, after all), but the two stores is probably
marginally better.  When shstks are active, we're taking a large hit
from the busy token handling.

I was concerned by the misaligned access, but it's not misaligned, its
it?  It's the start of entry_ssp which is misaligned and the -2 brings
it back to being properly aligned.

> --- a/xen/arch/x86/include/asm/asm_defns.h
> +++ b/xen/arch/x86/include/asm/asm_defns.h
> @@ -221,7 +221,7 @@ static always_inline void stac(void)
>  #endif
>  
>  #ifdef __ASSEMBLY__
> -.macro SAVE_ALL compat=0
> +.macro SAVE_ALL compat=0 ssp=IS_ENABLED(CONFIG_XEN_SHSTK)

I'm not sure this is what you want to do.  Because it's only the
default, we'll still....

>          addq  $-(UREGS_error_code-UREGS_r15), %rsp
>          cld
>          movq  %rdi,UREGS_rdi(%rsp)
> @@ -235,6 +235,9 @@ static always_inline void stac(void)
>          movq  %rax,UREGS_rax(%rsp)
>          xor   %eax, %eax
>  .if !\compat
> +.if \ssp
> +        rdsspq %rcx

... pick this up even in !CONFIG_XEN_SHSTK builds, and ...

> +.endif
>          movq  %r8,UREGS_r8(%rsp)
>          movq  %r9,UREGS_r9(%rsp)
>          movq  %r10,UREGS_r10(%rsp)
> @@ -264,6 +267,11 @@ static always_inline void stac(void)
>          xor   %r13d, %r13d
>          xor   %r14d, %r14d
>          xor   %r15d, %r15d
> +.if \ssp && !\compat
> +        mov   %cx, UREGS_entry_ssp(%rsp)
> +        shr   $16, %rcx
> +        mov   %ecx, UREGS_entry_ssp+2(%rsp)

... store it here.

I think you need to use ssp=1 by default, and

#ifdef CONFIG_XEN_SHSTK
.if
    ...

for these two blocks, so they disappear properly in !SHSTK builds.

But for the rest of the behaviour, there are two overlapping things,
because you end up getting entry_ssp=0 in SHSTK builds running on
hardware without shstk active.

And with that in mind, to confirm, the RDSSP block depends on the xor
%ecx,%ecx between the two hunks in order to function as intended?

> --- a/xen/include/public/arch-x86/xen-x86_64.h
> +++ b/xen/include/public/arch-x86/xen-x86_64.h
> @@ -183,7 +183,19 @@ struct cpu_user_regs {
>      uint8_t  _pad1[3];
>      __DECL_REG_LO16(flags); /* rflags.IF == !saved_upcall_mask */
>      __DECL_REG_LO8(sp);
> -    uint16_t ss, _pad2[3];
> +    uint16_t ss;
> +#if !defined(__XEN__)
> +    uint16_t _pad2[3];
> +#elif defined(COMPILE_OFFSETS)
> +    uint16_t entry_ssp[3];
> +#else
> +    /*
> +     * This points _at_ the corresponding shadow stack frame; it is _not_ the
> +     * outer context's SSP.  That, if the outer context has CET-SS enabled,
> +     * is stored in the top slot of the pointed to shadow stack frame.
> +     */
> +    signed long entry_ssp:48;
> +#endif

I have to admit that I dislike this.  (And only some of that is because
it's work I'm going to have to revert in order to make FRED support work...)

We desperately need to break our use of this structure to start with,
and with that done, we don't need to play games about hiding SSP in a
spare 6 bytes in an ABI dubiously made public nearly two decades ago.

How hard would it be just:

#define cpu_user_regs abi_cpu_user_regs
#include <public/xen.h>
#undef cpu_user_regs

and make a copy of cpu_user_regs that we can really put an SSP field into?

It would make this patch more simple, and we could finally get the vm86
block off the stack (which also fixes OoB accesses in the #DF handler -
cant remember if I finished the bodge around that or not.)

Irrespective of anything else, why do we have COMPILE_OFFSETS getting in
here?

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 15:17:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 15:17:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686703.1069049 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLg7-0001EP-Ge; Wed, 28 Feb 2024 15:17:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686703.1069049; Wed, 28 Feb 2024 15:17:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLg7-0001EI-DQ; Wed, 28 Feb 2024 15:17:03 +0000
Received: by outflank-mailman (input) for mailman id 686703;
 Wed, 28 Feb 2024 15:17:02 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=n4Dg=KF=cloud.com=ross.lagerwall@srs-se1.protection.inumbo.net>)
 id 1rfLg6-0000oJ-EZ
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 15:17:02 +0000
Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com
 [2607:f8b0:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6b1871f3-d64c-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 16:17:00 +0100 (CET)
Received: by mail-pg1-x533.google.com with SMTP id
 41be03b00d2f7-5ce2aada130so4923430a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 07:17:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b1871f3-d64c-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709133419; x=1709738219; darn=lists.xenproject.org;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:from:to:cc:subject:date
         :message-id:reply-to;
        bh=qrT0N1SHcSGgmq3jhOcG8IZZuotgNdCyfC2K7dJOpNk=;
        b=nqxbq6G35+CdQI0xez/hR5+yRG+o9mdHXSJy4wicNbM5CELmHTv6NomsrH7G29UGgd
         DCl3qPIX1em3UYtZ2ZJ6VsNKEiaseOjING6mUNEGzHBTEZLz5+mK1IWi3k8NR4lrTdYa
         4Cpi682W9hnoe4Ec262tc4OnFHZg9/qilXdHw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709133419; x=1709738219;
        h=content-transfer-encoding:cc:to:subject:message-id:date:from
         :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=qrT0N1SHcSGgmq3jhOcG8IZZuotgNdCyfC2K7dJOpNk=;
        b=rc5+TgHfo7NhEh6yfVEgoS0lwEqD3O4lmzPuqqkcXl+dDitdS9ngqKb0W9KFdDd6BP
         xhag0ZMiUdK3FvkJ1F29UZJXmN0lN8aD/ty/rT0k0luPiVxxNcwFz1S0TJUD8bdp6UIR
         uXy2d2H6xmyagdtQrFTE6GjQBm2ESrK0K4OFySvGvGVKdFU4FwqRuZ2FChMtEXDEnPk0
         yE69rabjeZY1SHVQU8RVk/Ua62KGIN4j1+Ga4EgY1NQZb7xwkxngSXKUBfS99zIBn5F2
         rZP4/K+AYfa9kS0Lb/VW4wk0ZLa5O0c0A1VZvzyRZVqGJa4Wkx6Z/QHSQ3UFFh0qO7Y3
         0Ktw==
X-Gm-Message-State: AOJu0YwLgSyk8fXWk+A7MbgpeCcO3v3iAnesqEzZyDj5wWybTX+/LVpA
	O6HJ/RDOeakfuyT59/2ffMOagQvHVAnHn/f55yKpnxZNUMp65IJDSpOFl8VqRNbP9OvWS+NTcZ4
	e+bb21wSUdMC4jlvVg00xH0y+rWy1T/CodRCK
X-Google-Smtp-Source: AGHT+IFWwzl+hBfPN29ObMt6zGcgwWDtn6WAz+7As9vgfjg+YYO+YhyaefJ/G+eTUsAwRJ2/BgaUlTJAMX1U+fbMKUw=
X-Received: by 2002:a17:90a:eb12:b0:29a:c6fa:4223 with SMTP id
 j18-20020a17090aeb1200b0029ac6fa4223mr8198560pjz.43.1709133419060; Wed, 28
 Feb 2024 07:16:59 -0800 (PST)
MIME-Version: 1.0
References: <20240227112528.4540-1-roger.pau@citrix.com> <20240227112528.4540-4-roger.pau@citrix.com>
In-Reply-To: <20240227112528.4540-4-roger.pau@citrix.com>
From: Ross Lagerwall <ross.lagerwall@citrix.com>
Date: Wed, 28 Feb 2024 15:16:48 +0000
Message-ID: <CAG7k0ErUtQGJ0X9gxib9jH3hpksJ4X8R1Hy5+ANzYYB=Vo2+CQ@mail.gmail.com>
Subject: Re: [PATCH v2 3/5] xen/livepatch: fix norevert test attempt to
 open-code revert
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: xen-devel@lists.xenproject.org, 
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 27, 2024 at 11:25=E2=80=AFAM Roger Pau Monne <roger.pau@citrix.=
com> wrote:
>
> The purpose of the norevert test is to install a dummy handler that repla=
ces
> the internal Xen revert code, and then perform the revert in the post-rev=
ert
> hook.  For that purpose the usage of the previous common_livepatch_revert=
() is
> not enough, as that just reverts specific functions, but not the whole st=
ate of
> the payload.
>
> Remove both common_livepatch_{apply,revert}() and instead expose
> revert_payload{,_tail}() in order to perform the patch revert from the
> post-revert hook.
>
> Fixes: 6047104c3ccc ('livepatch: Add per-function applied/reverted state =
tracking marker')
> Signed-off-by: Roger Pau Monn=C3=A9 <roger.pau@citrix.com>
> ---
> Changes since v1:
>  - Check return value of revert_payload().
> ---
>  xen/common/livepatch.c                        | 41 +++++++++++++++++--
>  xen/include/xen/livepatch.h                   | 32 ++-------------
>  .../livepatch/xen_action_hooks_norevert.c     | 22 +++-------
>  3 files changed, 46 insertions(+), 49 deletions(-)
>
> diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
> index 14295bae8704..5a7d5b7be0ad 100644
> --- a/xen/common/livepatch.c
> +++ b/xen/common/livepatch.c
> @@ -1366,7 +1366,22 @@ static int apply_payload(struct payload *data)
>      ASSERT(!local_irq_is_enabled());
>
>      for ( i =3D 0; i < data->nfuncs; i++ )
> -        common_livepatch_apply(&data->funcs[i], &data->fstate[i]);
> +    {
> +        const struct livepatch_func *func =3D &data->funcs[i];
> +        struct livepatch_fstate *state =3D &data->fstate[i];
> +
> +        /* If the action has been already executed on this function, do =
nothing. */
> +        if ( state->applied =3D=3D LIVEPATCH_FUNC_APPLIED )
> +        {
> +            printk(XENLOG_WARNING LIVEPATCH
> +                   "%s: %s has been already applied before\n",
> +                   __func__, func->name);
> +            continue;
> +        }
> +
> +        arch_livepatch_apply(func, state);
> +        state->applied =3D LIVEPATCH_FUNC_APPLIED;
> +    }
>
>      arch_livepatch_revive();
>
> @@ -1382,7 +1397,7 @@ static inline void apply_payload_tail(struct payloa=
d *data)
>      data->state =3D LIVEPATCH_STATE_APPLIED;
>  }
>
> -static int revert_payload(struct payload *data)
> +int revert_payload(struct payload *data)
>  {
>      unsigned int i;
>      int rc;
> @@ -1397,7 +1412,25 @@ static int revert_payload(struct payload *data)
>      }
>
>      for ( i =3D 0; i < data->nfuncs; i++ )
> -        common_livepatch_revert(&data->funcs[i], &data->fstate[i]);
> +    {
> +        const struct livepatch_func *func =3D &data->funcs[i];
> +        struct livepatch_fstate *state =3D &data->fstate[i];
> +
> +        /*
> +         * If the apply action hasn't been executed on this function, do
> +         * nothing.
> +         */
> +        if ( !func->old_addr || state->applied =3D=3D LIVEPATCH_FUNC_NOT=
_APPLIED )
> +        {
> +            printk(XENLOG_WARNING LIVEPATCH
> +                   "%s: %s has not been applied before\n",
> +                   __func__, func->name);
> +            continue;
> +        }
> +
> +        arch_livepatch_revert(func, state);
> +        state->applied =3D LIVEPATCH_FUNC_NOT_APPLIED;
> +    }
>
>      /*
>       * Since we are running with IRQs disabled and the hooks may call co=
mmon
> @@ -1415,7 +1448,7 @@ static int revert_payload(struct payload *data)
>      return 0;
>  }
>
> -static inline void revert_payload_tail(struct payload *data)
> +void revert_payload_tail(struct payload *data)
>  {
>      list_del(&data->applied_list);
>
> diff --git a/xen/include/xen/livepatch.h b/xen/include/xen/livepatch.h
> index ad0eae28bd0d..d074a5bebecc 100644
> --- a/xen/include/xen/livepatch.h
> +++ b/xen/include/xen/livepatch.h
> @@ -138,35 +138,11 @@ void arch_livepatch_post_action(void);
>  void arch_livepatch_mask(void);
>  void arch_livepatch_unmask(void);
>
> -static inline void common_livepatch_apply(const struct livepatch_func *f=
unc,
> -                                          struct livepatch_fstate *state=
)
> -{
> -    /* If the action has been already executed on this function, do noth=
ing. */
> -    if ( state->applied =3D=3D LIVEPATCH_FUNC_APPLIED )
> -    {
> -        printk(XENLOG_WARNING LIVEPATCH "%s: %s has been already applied=
 before\n",
> -                __func__, func->name);
> -        return;
> -    }
> -
> -    arch_livepatch_apply(func, state);
> -    state->applied =3D LIVEPATCH_FUNC_APPLIED;
> -}
> +/* Only for testing purposes. */
> +struct payload;
> +int revert_payload(struct payload *data);
> +void revert_payload_tail(struct payload *data);
>
> -static inline void common_livepatch_revert(const struct livepatch_func *=
func,
> -                                           struct livepatch_fstate *stat=
e)
> -{
> -    /* If the apply action hasn't been executed on this function, do not=
hing. */
> -    if ( !func->old_addr || state->applied =3D=3D LIVEPATCH_FUNC_NOT_APP=
LIED )
> -    {
> -        printk(XENLOG_WARNING LIVEPATCH "%s: %s has not been applied bef=
ore\n",
> -                __func__, func->name);
> -        return;
> -    }
> -
> -    arch_livepatch_revert(func, state);
> -    state->applied =3D LIVEPATCH_FUNC_NOT_APPLIED;
> -}
>  #else
>
>  /*
> diff --git a/xen/test/livepatch/xen_action_hooks_norevert.c b/xen/test/li=
vepatch/xen_action_hooks_norevert.c
> index c17385519263..c5fbab174680 100644
> --- a/xen/test/livepatch/xen_action_hooks_norevert.c
> +++ b/xen/test/livepatch/xen_action_hooks_norevert.c
> @@ -96,26 +96,14 @@ static int revert_hook(livepatch_payload_t *payload)
>
>  static void post_revert_hook(livepatch_payload_t *payload)
>  {
> -    int i;
> +    unsigned long flags;
>
>      printk(KERN_DEBUG "%s: Hook starting.\n", __func__);
>
> -    for (i =3D 0; i < payload->nfuncs; i++)
> -    {
> -        const struct livepatch_func *func =3D &payload->funcs[i];
> -        struct livepatch_fstate *fstate =3D &payload->fstate[i];
> -
> -        BUG_ON(revert_cnt !=3D 1);
> -        BUG_ON(fstate->applied !=3D LIVEPATCH_FUNC_APPLIED);
> -
> -        /* Outside of quiesce zone: MAY TRIGGER HOST CRASH/UNDEFINED BEH=
AVIOR */
> -        arch_livepatch_quiesce();
> -        common_livepatch_revert(payload);
> -        arch_livepatch_revive();
> -        BUG_ON(fstate->applied =3D=3D LIVEPATCH_FUNC_APPLIED);
> -
> -        printk(KERN_DEBUG "%s: post reverted: %s\n", __func__, func->nam=
e);
> -    }
> +    local_irq_save(flags);
> +    BUG_ON(revert_payload(payload));
> +    revert_payload_tail(payload);
> +    local_irq_restore(flags);
>
>      printk(KERN_DEBUG "%s: Hook done.\n", __func__);
>  }
> --
> 2.44.0
>

Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 15:19:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 15:19:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686706.1069059 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLiS-0002Sg-Rl; Wed, 28 Feb 2024 15:19:28 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686706.1069059; Wed, 28 Feb 2024 15:19:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLiS-0002SZ-Op; Wed, 28 Feb 2024 15:19:28 +0000
Received: by outflank-mailman (input) for mailman id 686706;
 Wed, 28 Feb 2024 15:19:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfLiR-0002R9-9h
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 15:19:27 +0000
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com
 [2a00:1450:4864:20::12d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c1bf193f-d64c-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 16:19:25 +0100 (CET)
Received: by mail-lf1-x12d.google.com with SMTP id
 2adb3069b0e04-51320536bddso137794e87.3
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 07:19:25 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 vx2-20020a170907a78200b00a43af1ea069sm1759193ejc.118.2024.02.28.07.19.24
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 07:19:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c1bf193f-d64c-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709133565; x=1709738365; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=yZ+PpHwHuxyIijc08+YFX/whxYqZCaXqTIzosDoQE3U=;
        b=bfDizt9qKF2qdiTrBnPZoYvLAtvhfKlIDqZ7jT2aBcWTKDvOevEeA85p9wZKDFfDP1
         X7vdSjbd48oejjpmlYOj1+4sz3iFJqG2pFM96pQRb2N+aBz0mt5W6+qgpammOKr8mUtj
         6cZU/JGgp7moBGy7YYXpuJLVWC7fi14YqgaoVdNHPvfswACF9krmSeoKCoxyOp7SahXQ
         +tmwNsssN2BcgJ3m55TrktmMR6UOiJzASm4YLMKwWH9bmltxDQZpp1tSAXWFKulMT6AS
         9gNEw8j265n3P2BxHZHG024drHF2GrS4MpBjNRHe9efsdRpkjDFiM2gPPFnCAGcKL12/
         AbEA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709133565; x=1709738365;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=yZ+PpHwHuxyIijc08+YFX/whxYqZCaXqTIzosDoQE3U=;
        b=iXsFfDhEo/zHXe/43mbjfIr7dWMiXgKGJPRRhyXSSvSYvnwuAJdnufVM+1RcoEt9+A
         MZnqDwDLprmoDefPY6HXKKMMBfN1Qzv/VSFusUbIcrvAirzYFFrpEKVo/h2VcBmfUaNA
         N32XrvDoo/ZeTiPUni3kNb1QJyAWNWMWABUmiUs7/JAa3mk0GFp1WnspqoD5cUeC4qOc
         r+pNlFhkFrMT1fMVRD07zHEBAuN2NVmprJEajxgfnZDV9BbvYo88AZUwCWUh76FqTArC
         Z+UzUSReLNQbtIPPAyw7GDLALd69L4ML4c7LGRgyoKVYck/0T7BZFelbUl945poWMISy
         kS2A==
X-Forwarded-Encrypted: i=1; AJvYcCU9ehFhku+FQCJeryKmaDd9w7SkhLOYs6YZqhdinpkph6w1g7E5CkZNQ0RPXEkIyrrj8+mGeROl9Hk7F8Dz+o5ml/skwD6VhPCHFkaVEHQ=
X-Gm-Message-State: AOJu0Yw7bwUCq6ssBk0CUSBvX79J8iGf7ldXC4CPkYFlCLACjVy2xkkV
	Nnp19fO78EohH5wl53KT2vyhcGDBie3dX1rUyqc9SjR+HYoaywutOBCyEl+2VQ==
X-Google-Smtp-Source: AGHT+IFVk2Bp7m0sbV9PRMR3aOQVvbTjLBNpY3sBEoF4kecksBYzga+ZJjMaHKxPyvbIFwGmLbdL4A==
X-Received: by 2002:a19:ac4b:0:b0:513:202b:9001 with SMTP id r11-20020a19ac4b000000b00513202b9001mr12174lfc.6.1709133564656;
        Wed, 28 Feb 2024 07:19:24 -0800 (PST)
Message-ID: <b76bff63-80de-4227-bcd9-8d3aa1275858@suse.com>
Date: Wed, 28 Feb 2024 16:19:23 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 06/12] xen/spinlock: make struct lock_profile
 rspinlock_t aware
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Alejandro Vallejo <alejandro.vallejo@cloud.com>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-7-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20231212094725.22184-7-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 12.12.2023 10:47, Juergen Gross wrote:
> --- a/xen/common/spinlock.c
> +++ b/xen/common/spinlock.c
> @@ -538,19 +538,31 @@ static void spinlock_profile_iterate(lock_profile_subfunc *sub, void *par)
>  static void cf_check spinlock_profile_print_elem(struct lock_profile *data,
>      int32_t type, int32_t idx, void *par)
>  {
> -    struct spinlock *lock = data->lock;
> +    unsigned int cpu;
> +    uint32_t lockval;

Any reason for this not being unsigned int as well? The more that ...

> +    if ( data->is_rlock )
> +    {
> +        cpu = data->rlock->debug.cpu;
> +        lockval = data->rlock->tickets.head_tail;
> +    }
> +    else
> +    {
> +        cpu = data->lock->debug.cpu;
> +        lockval = data->lock->tickets.head_tail;
> +    }
>  
>      printk("%s ", lock_profile_ancs[type].name);
>      if ( type != LOCKPROF_TYPE_GLOBAL )
>          printk("%d ", idx);
> -    printk("%s: addr=%p, lockval=%08x, ", data->name, lock,
> -           lock->tickets.head_tail);
> -    if ( lock->debug.cpu == SPINLOCK_NO_CPU )
> +    printk("%s: addr=%p, lockval=%08x, ", data->name, data->lock, lockval);

... it's then printed with plain x as the format char.

> +    if ( cpu == SPINLOCK_NO_CPU )
>          printk("not locked\n");
>      else
> -        printk("cpu=%d\n", lock->debug.cpu);
> -    printk("  lock:%" PRId64 "(%" PRI_stime "), block:%" PRId64 "(%" PRI_stime ")\n",
> -           data->lock_cnt, data->time_hold, data->block_cnt, data->time_block);
> +        printk("cpu=%u\n", cpu);
> +    printk("  lock:%" PRIu64 "(%" PRI_stime "), block:%" PRIu64 "(%" PRI_stime ")\n",
> +           data->lock_cnt, data->time_hold, (uint64_t)data->block_cnt,

I think I know why the cast is suddenly / unexpectedly needed, but imo
such wants stating in the description, when generally we aim at avoiding
casts where possible.

> --- a/xen/include/xen/spinlock.h
> +++ b/xen/include/xen/spinlock.h
> @@ -76,13 +76,19 @@ union lock_debug { };
>  */
>  
>  struct spinlock;
> +/* Temporary hack until a dedicated struct rspinlock is existing. */
> +#define rspinlock spinlock
>  
>  struct lock_profile {
>      struct lock_profile *next;       /* forward link */
>      const char          *name;       /* lock name */
> -    struct spinlock     *lock;       /* the lock itself */
> +    union {
> +        struct spinlock *lock;       /* the lock itself */
> +        struct rspinlock *rlock;     /* the recursive lock itself */
> +    };

_LOCK_PROFILE() wants to initialize this field, unconditionally using
.lock. While I expect that problem to be taken care of in one of the
later patches, use of the macro won't work anymore with this union in
use with very old gcc that formally we still support. While a road to
generally raising the baseline requirements is still pretty unclear to
me, an option might be to require (and document) that to enable
DEBUG_LOCK_PROFILE somewhat newer gcc needs using.

>      uint64_t            lock_cnt;    /* # of complete locking ops */
> -    uint64_t            block_cnt;   /* # of complete wait for lock */
> +    uint64_t            block_cnt:63; /* # of complete wait for lock */
> +    uint64_t            is_rlock:1;  /* use rlock pointer */

bool?

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 15:21:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 15:21:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686710.1069069 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLkE-00047P-5U; Wed, 28 Feb 2024 15:21:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686710.1069069; Wed, 28 Feb 2024 15:21:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLkE-00047I-2a; Wed, 28 Feb 2024 15:21:18 +0000
Received: by outflank-mailman (input) for mailman id 686710;
 Wed, 28 Feb 2024 15:21:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eIhW=KF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfLkC-00046f-8I
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 15:21:16 +0000
Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com
 [2607:f8b0:4864:20::f2d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 02ada04e-d64d-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 16:21:14 +0100 (CET)
Received: by mail-qv1-xf2d.google.com with SMTP id
 6a1803df08f44-68facd7ea4aso7932696d6.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 07:21:14 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 od1-20020a0562142f0100b0068f9b20fb0bsm5268569qvb.100.2024.02.28.07.21.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 07:21:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02ada04e-d64d-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709133673; x=1709738473; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=j8QTheCBKVUBKIDTYuNTqQKn7bHopMG2hMs7YpvwL3U=;
        b=db7jyLT/TtxwD250EOvy1G/mgPbssl/1RbR5s645p+DWO8E1vYtreDXncA/1yQn6BY
         /JOQ0EIEe9cvFDJD03fEyae/jPt4/YvcNxvH6fUclqvIsVCiztpFhNsZBuRz1cQCGq2p
         nrFpqG/SiirxKh1S4hcxwqgMJr+09ZvJH0LY0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709133673; x=1709738473;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=j8QTheCBKVUBKIDTYuNTqQKn7bHopMG2hMs7YpvwL3U=;
        b=P4GmKbRDWDno4wfOKnGJTBXsG/aFyP6OZV9vXMkgHmQwyyBtuTe247MRDQEQWaHFaA
         Z1+gXPPztRYv4mSE6xbk4Ww77seg1ELAcOQu/Qpr9GWjUKTnEk787hHlCshbdpSPLTy6
         adtqEeYv3gcyFswMHFSsvBeY0AV/qtr3HzZnkqwVaEY5MSHCcp5KVpgjv+Rmf7npB4zK
         5ftswTYh9380xB5MsoSOV1oxXAZ9+57eR/HvtF6qrq8+iJdavG/PG/pLYCFQwVguiRAT
         NONC5rbYnEQFrYz/nsXPVIZDJcwnaF+QnQErXJ4oVEaOLkGxnaYHjQQRs53dJE84Tuhs
         tFHA==
X-Forwarded-Encrypted: i=1; AJvYcCWCQZA5mDFIHoRO4Tu5FkDzFbLRdmlAY78PFVoBRGGrR6RLev6Atrg7OcwWLironn4adrG48CO36qOFVpzdoFrySjSmHS60Crci4poHRtg=
X-Gm-Message-State: AOJu0YyAH20lETt1so9Vny9iHKz2S+NheISEvW0dHX8wP1EdRnR8TDcL
	d/NjA6X8kjdL6O/sBu3ZXHbGFfqplEWU2fMbovlg8IGRKnkRIcTEtYKQDnZsAzs=
X-Google-Smtp-Source: AGHT+IHgPadCOuUfGjl3pP8WzlQeO8n82cN1iJCQ/5y3YWVMXyvauzK4uwFaGn4xem11JmxI/lgP2w==
X-Received: by 2002:a0c:df85:0:b0:68f:edbb:f317 with SMTP id w5-20020a0cdf85000000b0068fedbbf317mr2710300qvl.30.1709133673589;
        Wed, 28 Feb 2024 07:21:13 -0800 (PST)
Message-ID: <aebc274a-31ea-41f2-abb7-e45493ada436@citrix.com>
Date: Wed, 28 Feb 2024 15:21:10 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/4] x86/traps: use entry_ssp in fixup_exception_return()
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
 <45f89d82-bdc6-44b6-a784-bcdfdcca403d@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <45f89d82-bdc6-44b6-a784-bcdfdcca403d@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/02/2024 1:52 pm, Jan Beulich wrote:
> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -837,24 +825,26 @@ static void fixup_exception_return(struc
>  {
>      if ( IS_ENABLED(CONFIG_XEN_SHSTK) )
>      {
> -        unsigned long ssp, *ptr, *base;
> +        unsigned long ssp = rdssp();
>  
> -        if ( (ssp = rdssp()) == SSP_NO_SHSTK )
> -            goto shstk_done;
> +        if ( ssp != SSP_NO_SHSTK )
> +        {
> +            unsigned long *ptr = _p(regs->entry_ssp);

To double check, this works by the magic of:

    signed long entry_ssp:48;

getting sign extended back into a canonical address?

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 15:22:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 15:22:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686714.1069079 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLkx-0004ev-EZ; Wed, 28 Feb 2024 15:22:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686714.1069079; Wed, 28 Feb 2024 15:22:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLkx-0004eo-AQ; Wed, 28 Feb 2024 15:22:03 +0000
Received: by outflank-mailman (input) for mailman id 686714;
 Wed, 28 Feb 2024 15:22:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EdaL=KF=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rfLkv-00046f-TJ
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 15:22:01 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1df411c1-d64d-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 16:22:00 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 41504225C1;
 Wed, 28 Feb 2024 15:21:59 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id F322413A72;
 Wed, 28 Feb 2024 15:21:58 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id L9PGOZZP32UzAwAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 28 Feb 2024 15:21:58 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1df411c1-d64d-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709133719; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=aqKuQZZeNejtV5JojVQkZFqH25IA5L/7ebKWxrvjISY=;
	b=V+tcPXcCGXrwo/CQyYNMzmDUZhdkSSaPBwq0yVz/eVRz8ElQtmNMsrdMvgf8K3nHL98NxH
	K4fSTCi3O8GutUaLUVE3SRZuApDo8pTTjDBIPAyc0AXFCE112I7gidjiFe6B/rcqz1i00Q
	eIvQwMvuAQti8sOpxNVNHyc9IGc3jjA=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709133719; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=aqKuQZZeNejtV5JojVQkZFqH25IA5L/7ebKWxrvjISY=;
	b=V+tcPXcCGXrwo/CQyYNMzmDUZhdkSSaPBwq0yVz/eVRz8ElQtmNMsrdMvgf8K3nHL98NxH
	K4fSTCi3O8GutUaLUVE3SRZuApDo8pTTjDBIPAyc0AXFCE112I7gidjiFe6B/rcqz1i00Q
	eIvQwMvuAQti8sOpxNVNHyc9IGc3jjA=
Message-ID: <318818a3-4d7c-45ff-a656-067d12af555a@suse.com>
Date: Wed, 28 Feb 2024 16:21:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 05/12] xen/spinlock: add
 rspin_[un]lock_irq[save|restore]()
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, George Dunlap <george.dunlap@citrix.com>,
 Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-6-jgross@suse.com>
 <411c0f54-67cd-4b12-83ec-218703045707@suse.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <411c0f54-67cd-4b12-83ec-218703045707@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Level: ****
X-Spamd-Bar: ++++
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=V+tcPXcC
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [4.27 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 MIME_GOOD(-0.10)[text/plain];
	 BAYES_SPAM(5.10)[100.00%];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.67)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Score: 4.27
X-Rspamd-Queue-Id: 41504225C1
X-Spam-Flag: NO

On 28.02.24 16:09, Jan Beulich wrote:
> On 12.12.2023 10:47, Juergen Gross wrote:
>> Instead of special casing rspin_lock_irqsave() and
>> rspin_unlock_irqrestore() for the console lock, add those functions
>> to spinlock handling and use them where needed.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>> V2:
>> - new patch
> 
> In how far is this a necessary part of the series?

Not really necessary. It just seemed wrong to have an open coded
variant of rspin_lock_irqsave() and rspin_unlock_irqrestore().

> 
>> --- a/xen/arch/x86/traps.c
>> +++ b/xen/arch/x86/traps.c
>> @@ -647,13 +647,15 @@ void show_stack_overflow(unsigned int cpu, const struct cpu_user_regs *regs)
>>   void show_execution_state(const struct cpu_user_regs *regs)
>>   {
>>       /* Prevent interleaving of output. */
>> -    unsigned long flags = console_lock_recursive_irqsave();
>> +    unsigned long flags;
>> +
>> +    rspin_lock_irqsave(&console_lock, flags);
>>   
>>       show_registers(regs);
>>       show_code(regs);
>>       show_stack(regs);
>>   
>> -    console_unlock_recursive_irqrestore(flags);
>> +    rspin_unlock_irqrestore(&console_lock, flags);
>>   }
>>   
>>   void cf_check show_execution_state_nonconst(struct cpu_user_regs *regs)
>> @@ -663,7 +665,7 @@ void cf_check show_execution_state_nonconst(struct cpu_user_regs *regs)
>>   
>>   void vcpu_show_execution_state(struct vcpu *v)
>>   {
>> -    unsigned long flags = 0;
>> +    unsigned long flags;
>>   
>>       if ( test_bit(_VPF_down, &v->pause_flags) )
>>       {
>> @@ -698,7 +700,7 @@ void vcpu_show_execution_state(struct vcpu *v)
>>   #endif
>>   
>>       /* Prevent interleaving of output. */
>> -    flags = console_lock_recursive_irqsave();
>> +    rspin_lock_irqsave(&console_lock, flags);
>>   
>>       vcpu_show_registers(v);
>>   
>> @@ -708,7 +710,7 @@ void vcpu_show_execution_state(struct vcpu *v)
>>            * Stop interleaving prevention: The necessary P2M lookups involve
>>            * locking, which has to occur with IRQs enabled.
>>            */
>> -        console_unlock_recursive_irqrestore(flags);
>> +        rspin_unlock_irqrestore(&console_lock, flags);
>>   
>>           show_hvm_stack(v, &v->arch.user_regs);
>>       }
>> @@ -717,7 +719,7 @@ void vcpu_show_execution_state(struct vcpu *v)
>>           if ( guest_kernel_mode(v, &v->arch.user_regs) )
>>               show_guest_stack(v, &v->arch.user_regs);
>>   
>> -        console_unlock_recursive_irqrestore(flags);
>> +        rspin_unlock_irqrestore(&console_lock, flags);
>>       }
>>   
> 
> I view these as layering violations; ...
> 
>> --- a/xen/drivers/char/console.c
>> +++ b/xen/drivers/char/console.c
>> @@ -120,7 +120,7 @@ static int __read_mostly sercon_handle = -1;
>>   int8_t __read_mostly opt_console_xen; /* console=xen */
>>   #endif
>>   
>> -static DEFINE_RSPINLOCK(console_lock);
>> +DEFINE_RSPINLOCK(console_lock);
> 
> ... this should remain static. The question therefore just is whether
> to omit this patch or ...
> 
>> @@ -1158,22 +1158,6 @@ void console_end_log_everything(void)
>>       atomic_dec(&print_everything);
>>   }
>>   
>> -unsigned long console_lock_recursive_irqsave(void)
>> -{
>> -    unsigned long flags;
>> -
>> -    local_irq_save(flags);
>> -    rspin_lock(&console_lock);
>> -
>> -    return flags;
>> -}
>> -
>> -void console_unlock_recursive_irqrestore(unsigned long flags)
>> -{
>> -    rspin_unlock(&console_lock);
>> -    local_irq_restore(flags);
>> -}
> 
> ... whether to retain these two functions as thin wrappers around the
> new, more generic construct.

I'd vote for the latter.


Juergen


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 15:26:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 15:26:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686718.1069088 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLpK-0005gi-2p; Wed, 28 Feb 2024 15:26:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686718.1069088; Wed, 28 Feb 2024 15:26:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLpJ-0005gb-Vz; Wed, 28 Feb 2024 15:26:33 +0000
Received: by outflank-mailman (input) for mailman id 686718;
 Wed, 28 Feb 2024 15:26:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfLpJ-0005gV-GD
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 15:26:33 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bfdc3e67-d64d-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 16:26:31 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a440b1c445eso108937866b.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 07:26:31 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 w18-20020a170906481200b00a440ec600e3sm517437ejq.121.2024.02.28.07.26.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 07:26:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bfdc3e67-d64d-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709133991; x=1709738791; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=qBCQAmcAwbz+4BjecfA3CxB4ZRhe3irb7v10lQK7hPE=;
        b=a0A8s6YsCX/notlfJ+KNjSi2SQ1StL1/9wAGvtbyXLi5UV+PaBHxzyojY1NPxxG5RE
         GK3Sv0XY0bgrC8o831HABc0+J4ZsJ9D5TwjU1/Ho4B0TvNGmYaCkCF7W60RtSaPidqzI
         O+wAviAutC6+BAs5OmGruBjFRdhUqGB/HWr+jOExoM5vrBZq1rdW9L3WK52XQrglkO4n
         e+PFtjnG2oR7kdrubTGfZY9wHu0NyjUGJpIm8ObOJSKSjYznEtgsXwWmNHZA7JAvXX2L
         JI928e1M+zLS6t164eVcUHGEbFOnM4QK02+m6fLLepopODDz0z5UUL9DKDpzM+MR9Neu
         45SA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709133991; x=1709738791;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=qBCQAmcAwbz+4BjecfA3CxB4ZRhe3irb7v10lQK7hPE=;
        b=eGcJW2jVVPuLbvBXfaGSNHpK3ta4l9OH31a73gXmhY1fd/fPn97zxjJ4ZMbeGiZFoc
         w+mGmhY2eGMWZfHNqaoE0beYMyVck2u48PipdKojhJXSS7+Z2F3nb0KVIJMqpxGVCShw
         emWUmvZpOmwUtoKsj1aortO58N8Fm277LXTpbF0djK4KfYg4SNEiCJR8A+8cF7j973R4
         +oPS6qqfd2OD4UDhAn6OAu62iUiOzUGXb8aw7ouI4mDs7nci2nQJKB9dfYMHCG2FmYAX
         1/st9VZ4nqIUxqpLYwWQ64aDZjzdAtDgH1rHvtLROJZKAccEaxNOddzuqqAIgRumi6wc
         k/8Q==
X-Forwarded-Encrypted: i=1; AJvYcCUJga2uQO0hEwLCSrEP5TX/fvJPPZx1OWZupIO7J1yvb7f9w/q7vhYFzPGDPB4pUQ9OxWz0dIoOJ/NdT2DDEpbQIzJow3Os8Z+KfQxihSU=
X-Gm-Message-State: AOJu0YyvJTbbgKjNXw/DLKJF7zQm3tYMVN5uLrNY76EiiK0q/HrYgQgk
	hz04CpET5exAbY8NhIAvOpq1PkyVgFX1i3GMTXtjC4lHq+7eNOUf1cypUUOEbQ==
X-Google-Smtp-Source: AGHT+IHd1v/CSRpIrXzfYayj6GEQtN4EaQiXBkNxAe/PQh6vZVkrCEKg2JbiRXOKtZzCVMI5Oxq7PA==
X-Received: by 2002:a17:906:4a17:b0:a43:f924:d65 with SMTP id w23-20020a1709064a1700b00a43f9240d65mr1823858eju.26.1709133991068;
        Wed, 28 Feb 2024 07:26:31 -0800 (PST)
Message-ID: <8d08be46-f847-431b-86c8-7cae3d68574f@suse.com>
Date: Wed, 28 Feb 2024 16:26:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/4] x86/traps: use entry_ssp in fixup_exception_return()
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
 <45f89d82-bdc6-44b6-a784-bcdfdcca403d@suse.com>
 <aebc274a-31ea-41f2-abb7-e45493ada436@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <aebc274a-31ea-41f2-abb7-e45493ada436@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28.02.2024 16:21, Andrew Cooper wrote:
> On 28/02/2024 1:52 pm, Jan Beulich wrote:
>> --- a/xen/arch/x86/traps.c
>> +++ b/xen/arch/x86/traps.c
>> @@ -837,24 +825,26 @@ static void fixup_exception_return(struc
>>  {
>>      if ( IS_ENABLED(CONFIG_XEN_SHSTK) )
>>      {
>> -        unsigned long ssp, *ptr, *base;
>> +        unsigned long ssp = rdssp();
>>  
>> -        if ( (ssp = rdssp()) == SSP_NO_SHSTK )
>> -            goto shstk_done;
>> +        if ( ssp != SSP_NO_SHSTK )
>> +        {
>> +            unsigned long *ptr = _p(regs->entry_ssp);
> 
> To double check, this works by the magic of:
> 
>     signed long entry_ssp:48;
> 
> getting sign extended back into a canonical address?

That's the very reason for using an explicitly signed bitfield there,
yes.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 15:36:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 15:36:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686722.1069099 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLz5-0007yq-Vg; Wed, 28 Feb 2024 15:36:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686722.1069099; Wed, 28 Feb 2024 15:36:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfLz5-0007yj-Sq; Wed, 28 Feb 2024 15:36:39 +0000
Received: by outflank-mailman (input) for mailman id 686722;
 Wed, 28 Feb 2024 15:36:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGB+=KF=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rfLz5-0007yd-1r
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 15:36:39 +0000
Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com
 [2a00:1450:4864:20::32f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 291c1f5a-d64f-11ee-afd8-a90da7624cb6;
 Wed, 28 Feb 2024 16:36:37 +0100 (CET)
Received: by mail-wm1-x32f.google.com with SMTP id
 5b1f17b1804b1-412b7baa9c5so2063155e9.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 07:36:38 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 z9-20020a05600c114900b00410bca333b7sm2394015wmz.27.2024.02.28.07.36.36
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 07:36:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 291c1f5a-d64f-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709134597; x=1709739397; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=vc3MR6wqRwFjdZ59y43EE7o5MZp5u8KpNsnAusaH2zI=;
        b=EGw5yP+FlmaJ7iK783LR48AXUg8qkLhiHux167XWbnZE666EFIKwoaIQ8yxJ4We4hH
         gXn9+Asf+lxCK/JT2LAEAJvCbjKtLlaQhRd3lLKB4NmNCU0rrDr8W7zNs8JfzbDEC01G
         wHbDX3nxypo7qKwTfXmm6z6fEzFFLiYUN85iI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709134597; x=1709739397;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=vc3MR6wqRwFjdZ59y43EE7o5MZp5u8KpNsnAusaH2zI=;
        b=OR8YgusG5F8m3wKj0Pa9xl6ftpll5cqu+sa9U42aCAxfEooe3ZOXdoXmV4iuOUdBhz
         WJPX3XKYG/KQWA9rD3zo2ICG9Bkp86wOCHaA1nCVRtd9wc0fgSXP5tszkOvu5zHKmtbC
         mVpjCMT34QeFr5TNJQMj/3kHMUIAAr2q4TYIJc6PMDtqS7VgXr0w36+8b6FAtSsYTtrr
         kKwdAiae+PO0/lr3SmZARot24kFVCD7jF8rONFJQH7FwpcBL9nDWNGgup+PjcIvhtnIX
         291A4bXN6/CEITbRyx9ZddxgQPBKqo8JLIY/YBRlb7nheuygoVXkatZ4BP8B/9q1ArdN
         qxdg==
X-Gm-Message-State: AOJu0YzYbjKoaU7zJzdvZUvna0TCxJC95GhIakJTpmZjJPXehG3GRQ27
	RieCqqxC9TxgRe7OLlSRyr8AM6G79n316YV0n0yDJ5kzPXQMyAbeISVn+CudNeM=
X-Google-Smtp-Source: AGHT+IHhEwHeHosI3c6mcjtPP0oInY2niCQa0bxlDl2wUDFEW5rSqsPBjjLgbWyUcG0u8SvLx27tTw==
X-Received: by 2002:a05:600c:4f09:b0:412:ae70:992e with SMTP id l9-20020a05600c4f0900b00412ae70992emr3901658wmq.21.1709134597489;
        Wed, 28 Feb 2024 07:36:37 -0800 (PST)
Date: Wed, 28 Feb 2024 15:36:35 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 1/2] libxl: Fix segfault in device_model_spawn_outcome
Message-ID: <f477d391-4676-4cf3-b87a-8f8ee83e7db6@perard>
References: <20240109170540.38239-1-jandryuk@gmail.com>
 <20240109170540.38239-2-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240109170540.38239-2-jandryuk@gmail.com>

On Tue, Jan 09, 2024 at 12:05:39PM -0500, Jason Andryuk wrote:
> libxl__spawn_qdisk_backend() explicitly sets guest_config to NULL when
> starting QEMU (the usual launch through libxl__spawn_local_dm() has a
> guest_config though).
> 
> Bail early on a NULL guest_config/d_config.  This skips the QMP queries
> for chardevs and VNC, but this xenpv QEMU instance isn't expected to
> provide those - only qdisk (or 9pfs backends after an upcoming change).
> 
> Signed-off-by: Jason Andryuk <jandryuk@gmail.com>

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 15:43:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 15:43:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686725.1069109 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfM5M-0002Do-L0; Wed, 28 Feb 2024 15:43:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686725.1069109; Wed, 28 Feb 2024 15:43:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfM5M-0002Dh-I4; Wed, 28 Feb 2024 15:43:08 +0000
Received: by outflank-mailman (input) for mailman id 686725;
 Wed, 28 Feb 2024 15:43:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EdaL=KF=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rfM5L-0002Db-Je
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 15:43:07 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1098a6d5-d650-11ee-afd8-a90da7624cb6;
 Wed, 28 Feb 2024 16:43:06 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id EA8201F7C5;
 Wed, 28 Feb 2024 15:43:05 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A738B13A5D;
 Wed, 28 Feb 2024 15:43:05 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 3N8iJ4lU32W6CAAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 28 Feb 2024 15:43:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1098a6d5-d650-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709134986; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=aMI8Uu9+chYTcBUcNm9qfVC5FSLSs/YUdASWkUTPQqY=;
	b=tHhfoRiWdiLUhQaOfCPZYXwj2bPKgmyOU43rZrp8xKJ0/UJXzfzY2NrkjHxsTlwQXgZG1V
	vm1Be0wkOW3BCveeyywta4Y0MJlA0Kl1lpsJ2ZmCgibJs9WXqcvKAjS47LsAo0eqNuX5xe
	IybchevhopihY/VlRvQT1sDqIbd6v2M=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709134985; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=aMI8Uu9+chYTcBUcNm9qfVC5FSLSs/YUdASWkUTPQqY=;
	b=c5Xl4xiWxPRPoo0bVB1PhPXti+AHDX0q7GkJkWMG97ggp7tdQKB33Ayd4Pq9XcyEOZbHOe
	H/GywQaWKPQ40G1e90ZwhRA8LX/RNTHGLfgNhJpOvA+RTRtDPi3ujSPp/FsnEVgnB96P4p
	nFcYU2V16kILwjjbFh7xiC2u5/rX4/g=
Message-ID: <99aca5d7-87f1-4cce-a23d-3e1468ba3918@suse.com>
Date: Wed, 28 Feb 2024 16:43:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 06/12] xen/spinlock: make struct lock_profile
 rspinlock_t aware
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Alejandro Vallejo <alejandro.vallejo@cloud.com>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-7-jgross@suse.com>
 <b76bff63-80de-4227-bcd9-8d3aa1275858@suse.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <b76bff63-80de-4227-bcd9-8d3aa1275858@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=c5Xl4xiW
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-3.79 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_GOOD(-0.10)[text/plain];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.71)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Score: -3.79
X-Rspamd-Queue-Id: EA8201F7C5
X-Spam-Flag: NO

On 28.02.24 16:19, Jan Beulich wrote:
> On 12.12.2023 10:47, Juergen Gross wrote:
>> --- a/xen/common/spinlock.c
>> +++ b/xen/common/spinlock.c
>> @@ -538,19 +538,31 @@ static void spinlock_profile_iterate(lock_profile_subfunc *sub, void *par)
>>   static void cf_check spinlock_profile_print_elem(struct lock_profile *data,
>>       int32_t type, int32_t idx, void *par)
>>   {
>> -    struct spinlock *lock = data->lock;
>> +    unsigned int cpu;
>> +    uint32_t lockval;
> 
> Any reason for this not being unsigned int as well? The more that ...
> 
>> +    if ( data->is_rlock )
>> +    {
>> +        cpu = data->rlock->debug.cpu;
>> +        lockval = data->rlock->tickets.head_tail;
>> +    }
>> +    else
>> +    {
>> +        cpu = data->lock->debug.cpu;
>> +        lockval = data->lock->tickets.head_tail;
>> +    }

I've used the same type as tickets.head_tail.

>>   
>>       printk("%s ", lock_profile_ancs[type].name);
>>       if ( type != LOCKPROF_TYPE_GLOBAL )
>>           printk("%d ", idx);
>> -    printk("%s: addr=%p, lockval=%08x, ", data->name, lock,
>> -           lock->tickets.head_tail);
>> -    if ( lock->debug.cpu == SPINLOCK_NO_CPU )
>> +    printk("%s: addr=%p, lockval=%08x, ", data->name, data->lock, lockval);
> 
> ... it's then printed with plain x as the format char.

Which hasn't been changed by the patch. I can change it to PRIx32 if you want.

> 
>> +    if ( cpu == SPINLOCK_NO_CPU )
>>           printk("not locked\n");
>>       else
>> -        printk("cpu=%d\n", lock->debug.cpu);
>> -    printk("  lock:%" PRId64 "(%" PRI_stime "), block:%" PRId64 "(%" PRI_stime ")\n",
>> -           data->lock_cnt, data->time_hold, data->block_cnt, data->time_block);
>> +        printk("cpu=%u\n", cpu);
>> +    printk("  lock:%" PRIu64 "(%" PRI_stime "), block:%" PRIu64 "(%" PRI_stime ")\n",
>> +           data->lock_cnt, data->time_hold, (uint64_t)data->block_cnt,
> 
> I think I know why the cast is suddenly / unexpectedly needed, but imo
> such wants stating in the description, when generally we aim at avoiding
> casts where possible.

Okay, will add a sentence.

> 
>> --- a/xen/include/xen/spinlock.h
>> +++ b/xen/include/xen/spinlock.h
>> @@ -76,13 +76,19 @@ union lock_debug { };
>>   */
>>   
>>   struct spinlock;
>> +/* Temporary hack until a dedicated struct rspinlock is existing. */
>> +#define rspinlock spinlock
>>   
>>   struct lock_profile {
>>       struct lock_profile *next;       /* forward link */
>>       const char          *name;       /* lock name */
>> -    struct spinlock     *lock;       /* the lock itself */
>> +    union {
>> +        struct spinlock *lock;       /* the lock itself */
>> +        struct rspinlock *rlock;     /* the recursive lock itself */
>> +    };
> 
> _LOCK_PROFILE() wants to initialize this field, unconditionally using
> .lock. While I expect that problem to be taken care of in one of the
> later patches, use of the macro won't work anymore with this union in
> use with very old gcc that formally we still support. While a road to
> generally raising the baseline requirements is still pretty unclear to
> me, an option might be to require (and document) that to enable
> DEBUG_LOCK_PROFILE somewhat newer gcc needs using.

Patch 8 is using either .lock or .rlock depending on the lock type.

What is the problem with the old gcc version? Static initializers of
anonymous union members?

> 
>>       uint64_t            lock_cnt;    /* # of complete locking ops */
>> -    uint64_t            block_cnt;   /* # of complete wait for lock */
>> +    uint64_t            block_cnt:63; /* # of complete wait for lock */
>> +    uint64_t            is_rlock:1;  /* use rlock pointer */
> 
> bool?

Yes.


Juergen


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 16:02:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 16:02:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686729.1069118 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfMNc-0006w7-4w; Wed, 28 Feb 2024 16:02:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686729.1069118; Wed, 28 Feb 2024 16:02:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfMNc-0006w0-2K; Wed, 28 Feb 2024 16:02:00 +0000
Received: by outflank-mailman (input) for mailman id 686729;
 Wed, 28 Feb 2024 16:01:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGB+=KF=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rfMNa-0006vu-EV
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 16:01:58 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b24944a0-d652-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 17:01:56 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a2f22bfb4e6so808727966b.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 08:01:56 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 lc15-20020a170906f90f00b00a4406929637sm648282ejb.162.2024.02.28.08.01.54
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 08:01:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b24944a0-d652-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709136116; x=1709740916; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=6vCy/QJlmrqeXeGtfl6MixF44IvLE3ydHnEt5/a/0is=;
        b=gbnwB1/X4pBlaGk8Z3nVcZWd8eQQoCPCXlOqh6UWGNYLtZtCN8h6TlO0ODBSDY9EvR
         ACTT0eRiCoVZNklK9tHmYj0Puo1zPsoNjjuJ7qv/FMwFR2lzavxReFcAiW94I65X0QqM
         xnN5W4+mJWbyHyjcuGHaJz0YUAjkCwrxSQFmo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709136116; x=1709740916;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=6vCy/QJlmrqeXeGtfl6MixF44IvLE3ydHnEt5/a/0is=;
        b=cNZFOpnjGFuvHxH/t23qOmmLG0+vZjkGREuxtFBCkRFc1PJW1chhSMyLcfIWsvJkqY
         nSTDDJ+FGGAHnkUj5peiSNgbdVcCLZ5xBPsdrk1sHU3eF2Vv2P/dCkSq7UaWMSeXivqE
         u4KfM+E0CCEoOYJ9oMtveLCSzSrqeeviSzbUbCt8klv6BiErx8oz9oFByFh0FCO1tcY6
         iPx5wWrEGDTa9aJO3OFOIKP6KX5DUSJq4TgLWpop5mIPP3BIgoDrsZMDLcm3NEQ+Iarx
         UuIUJw92ZB7/rCmFLBiZgIC4l/nIkETk2jmAjEKitGoYirh+/3fSBFGDL8qyVNHzkngC
         wU8A==
X-Gm-Message-State: AOJu0Yy7Hbz0FI5A6zZfcK5z6pIOGqns+AxDeop544kxP+/EllXmOdnK
	PYSMVWTBzqUGEGnrX93ArekBYVpI3uASwUhHDxwt23gb7T80H/NjbRhoVgcBbwI=
X-Google-Smtp-Source: AGHT+IFqLHTY/Uj7ignJTAgwYaxpWjsmZiSboUWzOFwnOD+Vot8CQ7Gjr/iGk+gnv57+A9xeg7oNrQ==
X-Received: by 2002:a17:906:44f:b0:a43:ef3d:e42e with SMTP id e15-20020a170906044f00b00a43ef3de42emr87688eja.18.1709136115773;
        Wed, 28 Feb 2024 08:01:55 -0800 (PST)
Date: Wed, 28 Feb 2024 16:01:54 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Jason Andryuk <jandryuk@gmail.com>
Cc: xen-devel@lists.xenproject.org, Wei Liu <wl@xen.org>,
	Juergen Gross <jgross@suse.com>
Subject: Re: [PATCH 2/2] libxl: devd: Spawn QEMU for 9pfs
Message-ID: <3d849b15-cfad-4d85-b2b8-11cd19027720@perard>
References: <20240109170540.38239-1-jandryuk@gmail.com>
 <20240109170540.38239-3-jandryuk@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240109170540.38239-3-jandryuk@gmail.com>

On Tue, Jan 09, 2024 at 12:05:40PM -0500, Jason Andryuk wrote:
> @@ -3430,7 +3431,7 @@ void libxl__spawn_qdisk_backend(libxl__egc *egc, libxl__dm_spawn_state *dmss)
>       * because we will call this from unprivileged driver domains,
>       * so save it in the current domain libxl private dir.
>       */
> -    dmss->spawn.pidpath = GCSPRINTF("libxl/%u/qdisk-backend-pid", domid);
> +    dmss->spawn.pidpath = GCSPRINTF("libxl/%u/qemu-xenpv-backend-pid", domid);

This path is documented in "docs/misc/xenstore-paths.pandoc", I'm not
sure it's such a good idea to change it.

But maybe it's ok to change the path because xl devd is expected add
then remove this path? And that pid isn't going to be useful to anything
else?

I'd rather not change the path, but if you still think it's ok, we can
go for it. Thought?

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 16:02:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 16:02:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686732.1069128 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfMOY-0007V8-EC; Wed, 28 Feb 2024 16:02:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686732.1069128; Wed, 28 Feb 2024 16:02:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfMOY-0007V1-Bj; Wed, 28 Feb 2024 16:02:58 +0000
Received: by outflank-mailman (input) for mailman id 686732;
 Wed, 28 Feb 2024 16:02:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=iHog=KF=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfMOX-0007Uv-NY
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 16:02:57 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d5ca900c-d652-11ee-afd8-a90da7624cb6;
 Wed, 28 Feb 2024 17:02:56 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a3e85a76fa8so567346066b.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 08:02:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 cx11-20020a170907168b00b00a43e8430190sm1014758ejd.47.2024.02.28.08.02.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 08:02:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d5ca900c-d652-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709136176; x=1709740976; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=99Fim99KC2lLNEyLs9RhWoU4a6GouEolWcAdgTKNHZk=;
        b=Ang745JPIc+laPjYfG4E1y8jl2iXLsiGmMnYVhwON/EqRTBsAjJ85qCeNNTDeKAULX
         h7FFha4VGE0nfPGkZ5ErlrqBFs2UEKjm1kRbiwXnFTFSTl2Ljn4A/EvgloGIcFMj3kzk
         sUJpWM99z9Z+7roizhuiHMlw36z139TgXsGkVcJrlh9DHTm5KXGkQgrNN0qo4pUyufqi
         qQ9J6C5GhNantDzgQfVzEO/ilghtihbSciw5Z0CczpdVtci80WSf5UzXSpBvr63AK0im
         L2mQzPUojWyJbzrPX+cEuHF0DC1SgReZEGDYlhY9i2whuX30WC7TinG/ua+9i1YasJr6
         NDPw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709136176; x=1709740976;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=99Fim99KC2lLNEyLs9RhWoU4a6GouEolWcAdgTKNHZk=;
        b=eBsfmavsbUoeP4KXFWA/EH+ojziJcom0zbiH5ZVdWrFV1346c8DVu+r/UcYEUA8IeP
         SJXOR4lFdxfeGsqnNIn1YUugyyYRS4Ech3PfrNhDmWAoNvbYfxeOae6KIm6V01KNxkal
         IMe1QUOyEmy1xV3+GZBEiYnFrjaKg7kTfNa6xIxzfFsC/aeFmA1K+9Zg7u+uqSuxMq7Y
         FZekN48TVrppe60KShLjlQH1A6KNXHEFFRaxkQTw8Vo/G2WIE2jlXLedAm10y6sHozsU
         IYGcYq9FKp0Tm2zAjhRTa9sdC2NhCndDlcEfTsUhkxKGr1lacAryn3lVyGVaUNh17BKF
         VYxQ==
X-Forwarded-Encrypted: i=1; AJvYcCWXqkFGVBHM27PhHTgnfX/B+bUFe2qA/BDkzdcyKTl0S/7GnzDRsiVJ6gntzAYzNfWvWjoUyLSqDNYLEQ15QSM8TqzeEcfHtaDsYeadoy8=
X-Gm-Message-State: AOJu0YzfPlseROvvIc0vZMlnrzYefnbGzTEvl4fro0WRRFG751VZSvYO
	XHZ1R615Vad5p/yRwSXYumY4odxlAHLoI3l8e7qDHOVmyEBFZyeP2N4MACOg4w==
X-Google-Smtp-Source: AGHT+IEm4HL9oqEV+XnwVlzsWzgPYFpapdFctp4xzn0B6GqI9GsN/pkMdxxbVqww5SNmo7vLkHkdgw==
X-Received: by 2002:a17:906:7c45:b0:a43:bf24:41aa with SMTP id g5-20020a1709067c4500b00a43bf2441aamr106182ejp.37.1709136175635;
        Wed, 28 Feb 2024 08:02:55 -0800 (PST)
Message-ID: <8a0397c8-4ce6-4f4b-8a50-b0deaa30f291@suse.com>
Date: Wed, 28 Feb 2024 17:02:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 06/12] xen/spinlock: make struct lock_profile
 rspinlock_t aware
Content-Language: en-US
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Alejandro Vallejo <alejandro.vallejo@cloud.com>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-7-jgross@suse.com>
 <b76bff63-80de-4227-bcd9-8d3aa1275858@suse.com>
 <99aca5d7-87f1-4cce-a23d-3e1468ba3918@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <99aca5d7-87f1-4cce-a23d-3e1468ba3918@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28.02.2024 16:43, Jürgen Groß wrote:
> On 28.02.24 16:19, Jan Beulich wrote:
>> On 12.12.2023 10:47, Juergen Gross wrote:
>>> --- a/xen/common/spinlock.c
>>> +++ b/xen/common/spinlock.c
>>> @@ -538,19 +538,31 @@ static void spinlock_profile_iterate(lock_profile_subfunc *sub, void *par)
>>>   static void cf_check spinlock_profile_print_elem(struct lock_profile *data,
>>>       int32_t type, int32_t idx, void *par)
>>>   {
>>> -    struct spinlock *lock = data->lock;
>>> +    unsigned int cpu;
>>> +    uint32_t lockval;
>>
>> Any reason for this not being unsigned int as well? The more that ...
>>
>>> +    if ( data->is_rlock )
>>> +    {
>>> +        cpu = data->rlock->debug.cpu;
>>> +        lockval = data->rlock->tickets.head_tail;
>>> +    }
>>> +    else
>>> +    {
>>> +        cpu = data->lock->debug.cpu;
>>> +        lockval = data->lock->tickets.head_tail;
>>> +    }
> 
> I've used the same type as tickets.head_tail.
> 
>>>   
>>>       printk("%s ", lock_profile_ancs[type].name);
>>>       if ( type != LOCKPROF_TYPE_GLOBAL )
>>>           printk("%d ", idx);
>>> -    printk("%s: addr=%p, lockval=%08x, ", data->name, lock,
>>> -           lock->tickets.head_tail);
>>> -    if ( lock->debug.cpu == SPINLOCK_NO_CPU )
>>> +    printk("%s: addr=%p, lockval=%08x, ", data->name, data->lock, lockval);
>>
>> ... it's then printed with plain x as the format char.
> 
> Which hasn't been changed by the patch. I can change it to PRIx32 if you want.

As per ./CODING_STYLE unsigned int is preferred.

>>> --- a/xen/include/xen/spinlock.h
>>> +++ b/xen/include/xen/spinlock.h
>>> @@ -76,13 +76,19 @@ union lock_debug { };
>>>   */
>>>   
>>>   struct spinlock;
>>> +/* Temporary hack until a dedicated struct rspinlock is existing. */
>>> +#define rspinlock spinlock
>>>   
>>>   struct lock_profile {
>>>       struct lock_profile *next;       /* forward link */
>>>       const char          *name;       /* lock name */
>>> -    struct spinlock     *lock;       /* the lock itself */
>>> +    union {
>>> +        struct spinlock *lock;       /* the lock itself */
>>> +        struct rspinlock *rlock;     /* the recursive lock itself */
>>> +    };
>>
>> _LOCK_PROFILE() wants to initialize this field, unconditionally using
>> .lock. While I expect that problem to be taken care of in one of the
>> later patches, use of the macro won't work anymore with this union in
>> use with very old gcc that formally we still support. While a road to
>> generally raising the baseline requirements is still pretty unclear to
>> me, an option might be to require (and document) that to enable
>> DEBUG_LOCK_PROFILE somewhat newer gcc needs using.
> 
> Patch 8 is using either .lock or .rlock depending on the lock type.
> 
> What is the problem with the old gcc version? Static initializers of
> anonymous union members?

Yes.

Jan


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 16:15:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 16:15:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686737.1069139 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfMaU-0002LH-Jh; Wed, 28 Feb 2024 16:15:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686737.1069139; Wed, 28 Feb 2024 16:15:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfMaU-0002LA-Gn; Wed, 28 Feb 2024 16:15:18 +0000
Received: by outflank-mailman (input) for mailman id 686737;
 Wed, 28 Feb 2024 16:15:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eIhW=KF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfMaT-0002Kk-2S
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 16:15:17 +0000
Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com
 [2607:f8b0:4864:20::732])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8e12c59a-d654-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 17:15:15 +0100 (CET)
Received: by mail-qk1-x732.google.com with SMTP id
 af79cd13be357-787b4d1393aso361789685a.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 08:15:15 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 nf9-20020a0562143b8900b0068fe4669e71sm5360241qvb.91.2024.02.28.08.15.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 08:15:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8e12c59a-d654-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709136914; x=1709741714; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=f7WVomFudO0WGDDK8vul+YJ5ZvWnuYzorq7iIax68lQ=;
        b=cMpMkwUqtFlTk+wAP5pEnJfM7qUnHMAAmmaBYZnd3zTipl9AjB47bn/oegseya4e+w
         mY1Aj1RgR0vyaxqphyNlZp62W11TSdmsIKTS2bg9wz37+Ivdh8TJHbRAEOAMeuLdwgSE
         +6tfu+/yVOPunBkoGgNBfDfD6ApgPwv6p5Tio=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709136914; x=1709741714;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=f7WVomFudO0WGDDK8vul+YJ5ZvWnuYzorq7iIax68lQ=;
        b=a5XaZ/Peyym0tvJGN+kz1LNGfYEqDIhId4PR8jk94XsfcGXt9+EMyGzsKJnOQCrVn6
         7AsiqBgdDasxr2wtsHmlJoPYHzqY2TMrFjJpX4GRWxJdhrohmYXd4KscAFyritbyN2t9
         lYVB4znHSB0P0MfML1PF6pTHsZKKFuqc1cAm0zuaQ2LDusCE8PiPRQOIXKb7HNs1//g7
         bvoOwDA+Ct91WVGam1ThPsldswTa6HLIGZ4q0+sWjeJhDRT0Ml8+vDZZs6AKJS/vZjmb
         Q9KiReGvuzNExyoZrruR8ctLxgh8pIb9DqcDunPfb0jBVzYGMr3H5BhqVchCxgy/gp3p
         7eNw==
X-Forwarded-Encrypted: i=1; AJvYcCUtg2X30Rqlt60Bw2PA2SUprxzKfwT+SYSaggDAF3vOlIP5YoGBmnA5joeZoXoauDku5AvEIgUaCnUm/Q2wH1EXgnnIQ2Fj6gA9Yr13GuA=
X-Gm-Message-State: AOJu0YyTiT6TmeVOObhGbgZsAACHB/XLaODH56ClyM1W3dnO5DyH/xXD
	rxqiRQz3eGOUJBqmjJnyXXdLfkl0tfpcwQZx8FyH+HV5Bp2QZbB1vS59KTSzbhw=
X-Google-Smtp-Source: AGHT+IEE4SRhJkZWjecOzMgxJq0h0zvWsNwoibDK6OyuXeXYiHEDXSCSXagx/FectCLqgzVFl8RXaA==
X-Received: by 2002:ad4:5c4b:0:b0:68f:7891:ef2e with SMTP id a11-20020ad45c4b000000b0068f7891ef2emr6463089qva.53.1709136913854;
        Wed, 28 Feb 2024 08:15:13 -0800 (PST)
Message-ID: <e687d4a1-efd8-4d7f-9c5d-8796bb99014a@citrix.com>
Date: Wed, 28 Feb 2024 16:15:11 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/4] x86: prefer shadow stack for producing call traces
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
 <fdacbb8b-5a58-4d6b-89b2-ea98b4fed895@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <fdacbb8b-5a58-4d6b-89b2-ea98b4fed895@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/02/2024 1:53 pm, Jan Beulich wrote:
> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -539,6 +544,50 @@ static void show_trace(const struct cpu_
>           !is_active_kernel_text(tos) )
>          printk("   [<%p>] R %pS\n", _p(regs->rip), _p(regs->rip));
>  
> +    if ( IS_ENABLED(CONFIG_XEN_SHSTK) && rdssp() != SSP_NO_SHSTK )
> +    {
> +        const unsigned long *ptr = _p(regs->entry_ssp);
> +        unsigned int n;
> +
> +        for ( n = 0; (unsigned long)ptr & (PAGE_SIZE - sizeof(*ptr)); ++n )
> +        {
> +            unsigned long val = *ptr;
> +
> +            if ( is_active_kernel_text(val) || in_stub(val) )
> +            {
> +                /* Normal return address entry.  */
> +                printk("   [<%p>] C %pS\n", _p(val), _p(val));
> +                ++ptr;
> +            }
> +            else if ( !((val ^ *ptr) >> (PAGE_SHIFT + STACK_ORDER)) )
> +            {
> +                if ( val & (sizeof(val) - 1) )
> +                {
> +                    /* Most likely a supervisor token. */
> +                    break;
> +                }

Tokens are their own linear address, with metadata in the bottom two
bits.  I think it would be better to check that explicitly, rather than
assuming anything nonzero in the upper bits is a token.

> +
> +                /*
> +                 * Ought to be a hypervisor interruption frame.  But don't
> +                 * (re)log the current frame's %rip.
> +                 */
> +                if ( n || ptr[1] != regs->rip )
> +                    printk("   [<%p>] E %pS\n", _p(ptr[1]), _p(ptr[1]));
> +                ptr = _p(val);
> +            }
> +            else
> +            {
> +                /* Ought to be a PV guest hypercall/interruption frame.  */
> +                printk("   %04lx:[<%p>] E\n", ptr[2], _p(ptr[1]));
> +                ptr = 0;

On a CPL3 -> CPL0 transition, the guest's SSP is written back into
MSR_PL3_SSP.  The supervisor token on MSR_PL0_SSP is marked busy (either
automatically, or by SETSSBY), but nothing pertaining to CPL3 is pushed
onto the supervisor shadow stack.

This is why we can move off an IST stack onto the primary stack when
interrupting CPL3 with only a CLEARSSBSY/SETSSBSY pair, and no memmove()
loop of WRSS's.

In other words, I'm pretty sure this is a dead codeapth.  (Or worse, if
it happens not to  be dead, then the comment is misleading.)

A CPL1 -> CPL0 transition does push an shstk interrupt frame, and not
wanting to memmove() the shstk by 3 slots on a context switch is part of
why I just disallowed PV32 guests when CET was active.

~Andrew


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 16:22:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 16:22:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686741.1069148 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfMhj-00045e-9I; Wed, 28 Feb 2024 16:22:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686741.1069148; Wed, 28 Feb 2024 16:22:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfMhj-00045X-6n; Wed, 28 Feb 2024 16:22:47 +0000
Received: by outflank-mailman (input) for mailman id 686741;
 Wed, 28 Feb 2024 16:22:45 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=EdaL=KF=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rfMhh-000444-Ry
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 16:22:45 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9a1adc8c-d655-11ee-afd8-a90da7624cb6;
 Wed, 28 Feb 2024 17:22:44 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 2BB421F796;
 Wed, 28 Feb 2024 16:22:44 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id E092513A58;
 Wed, 28 Feb 2024 16:22:43 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id HEg/NdNd32UIEgAAD6G6ig
 (envelope-from <jgross@suse.com>); Wed, 28 Feb 2024 16:22:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9a1adc8c-d655-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709137364; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pasyg7Lr8XSdWfcKR2Tf73gW2fkWyLHtMJIArEbeWac=;
	b=ICqeFfEEp8iC1NByTOzdUmMN2tSf1EhwDT0Daj01lLsKfcTVd/ydT+OTQNj7U/UdRrWJJP
	CYjJb8AgDZEoHJdxTSGykw7+/b+Q+pTYkFQuXVpUBWAxAsIUnl13AjRP3mn5A8AMtdHJ5P
	ClyikKu0wVzemWGhGBP7aFCDDw7LLEY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709137364; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=pasyg7Lr8XSdWfcKR2Tf73gW2fkWyLHtMJIArEbeWac=;
	b=ICqeFfEEp8iC1NByTOzdUmMN2tSf1EhwDT0Daj01lLsKfcTVd/ydT+OTQNj7U/UdRrWJJP
	CYjJb8AgDZEoHJdxTSGykw7+/b+Q+pTYkFQuXVpUBWAxAsIUnl13AjRP3mn5A8AMtdHJ5P
	ClyikKu0wVzemWGhGBP7aFCDDw7LLEY=
Message-ID: <9256b285-22b5-4c0c-8577-2b8832fb6e19@suse.com>
Date: Wed, 28 Feb 2024 17:22:43 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 06/12] xen/spinlock: make struct lock_profile
 rspinlock_t aware
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Alejandro Vallejo <alejandro.vallejo@cloud.com>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-7-jgross@suse.com>
 <b76bff63-80de-4227-bcd9-8d3aa1275858@suse.com>
 <99aca5d7-87f1-4cce-a23d-3e1468ba3918@suse.com>
 <8a0397c8-4ce6-4f4b-8a50-b0deaa30f291@suse.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <8a0397c8-4ce6-4f4b-8a50-b0deaa30f291@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.58
X-Spamd-Result: default: False [-3.58 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.71)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Flag: NO

On 28.02.24 17:02, Jan Beulich wrote:
> On 28.02.2024 16:43, Jürgen Groß wrote:
>> On 28.02.24 16:19, Jan Beulich wrote:
>>> On 12.12.2023 10:47, Juergen Gross wrote:
>>>> --- a/xen/common/spinlock.c
>>>> +++ b/xen/common/spinlock.c
>>>> @@ -538,19 +538,31 @@ static void spinlock_profile_iterate(lock_profile_subfunc *sub, void *par)
>>>>    static void cf_check spinlock_profile_print_elem(struct lock_profile *data,
>>>>        int32_t type, int32_t idx, void *par)
>>>>    {
>>>> -    struct spinlock *lock = data->lock;
>>>> +    unsigned int cpu;
>>>> +    uint32_t lockval;
>>>
>>> Any reason for this not being unsigned int as well? The more that ...
>>>
>>>> +    if ( data->is_rlock )
>>>> +    {
>>>> +        cpu = data->rlock->debug.cpu;
>>>> +        lockval = data->rlock->tickets.head_tail;
>>>> +    }
>>>> +    else
>>>> +    {
>>>> +        cpu = data->lock->debug.cpu;
>>>> +        lockval = data->lock->tickets.head_tail;
>>>> +    }
>>
>> I've used the same type as tickets.head_tail.
>>
>>>>    
>>>>        printk("%s ", lock_profile_ancs[type].name);
>>>>        if ( type != LOCKPROF_TYPE_GLOBAL )
>>>>            printk("%d ", idx);
>>>> -    printk("%s: addr=%p, lockval=%08x, ", data->name, lock,
>>>> -           lock->tickets.head_tail);
>>>> -    if ( lock->debug.cpu == SPINLOCK_NO_CPU )
>>>> +    printk("%s: addr=%p, lockval=%08x, ", data->name, data->lock, lockval);
>>>
>>> ... it's then printed with plain x as the format char.
>>
>> Which hasn't been changed by the patch. I can change it to PRIx32 if you want.
> 
> As per ./CODING_STYLE unsigned int is preferred.

Okay, I'll switch to unsigned int then.

> 
>>>> --- a/xen/include/xen/spinlock.h
>>>> +++ b/xen/include/xen/spinlock.h
>>>> @@ -76,13 +76,19 @@ union lock_debug { };
>>>>    */
>>>>    
>>>>    struct spinlock;
>>>> +/* Temporary hack until a dedicated struct rspinlock is existing. */
>>>> +#define rspinlock spinlock
>>>>    
>>>>    struct lock_profile {
>>>>        struct lock_profile *next;       /* forward link */
>>>>        const char          *name;       /* lock name */
>>>> -    struct spinlock     *lock;       /* the lock itself */
>>>> +    union {
>>>> +        struct spinlock *lock;       /* the lock itself */
>>>> +        struct rspinlock *rlock;     /* the recursive lock itself */
>>>> +    };
>>>
>>> _LOCK_PROFILE() wants to initialize this field, unconditionally using
>>> .lock. While I expect that problem to be taken care of in one of the
>>> later patches, use of the macro won't work anymore with this union in
>>> use with very old gcc that formally we still support. While a road to
>>> generally raising the baseline requirements is still pretty unclear to
>>> me, an option might be to require (and document) that to enable
>>> DEBUG_LOCK_PROFILE somewhat newer gcc needs using.
>>
>> Patch 8 is using either .lock or .rlock depending on the lock type.
>>
>> What is the problem with the old gcc version? Static initializers of
>> anonymous union members?
> 
> Yes.

The easiest solution might then be to give the union a name.


Juergen



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 16:57:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 16:57:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686745.1069159 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfNES-0001ip-QX; Wed, 28 Feb 2024 16:56:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686745.1069159; Wed, 28 Feb 2024 16:56:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfNES-0001ii-Nd; Wed, 28 Feb 2024 16:56:36 +0000
Received: by outflank-mailman (input) for mailman id 686745;
 Wed, 28 Feb 2024 16:56:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=KqYS=KF=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rfNER-0001ic-Im
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 16:56:35 +0000
Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com
 [2607:f8b0:4864:20::72a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5264dfe3-d65a-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 17:56:31 +0100 (CET)
Received: by mail-qk1-x72a.google.com with SMTP id
 af79cd13be357-787ef8b194dso73776185a.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 08:56:31 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 h8-20020a05620a10a800b00785da717d64sm4719424qkk.111.2024.02.28.08.56.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 08:56:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5264dfe3-d65a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709139391; x=1709744191; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=N0xKBzokyOF7L7x5XSmIhKlw17iIgKyEqx8u4319JZU=;
        b=Q3nXSKD1xdG0jg3PlRCUjqGwHozv80w90F99EDTnRS6NCGrpjI8PTlIcY5h03Sra8U
         NfVvGimwto7UauPvff0vFX8q+5YaCXiZGg5RZJqoT9JsSwFFWZlHak5oFSAMUFDjq6EY
         TAyCp1cVFY165dUrBgil1hJXFWAwyzjAxNpl4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709139391; x=1709744191;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=N0xKBzokyOF7L7x5XSmIhKlw17iIgKyEqx8u4319JZU=;
        b=Wkv4RHBu5+RlbKS9g63ACeiihS5L6nqneshrYihuXugns1CaXXb1voFRVWY97OoPTX
         NJCzZAayJywxcJ+OnxR4uaRPnrfnA0nGIDUYRBLdWDfViFGLXecPAPisZE3AO/aq0YPI
         nDju6ps7FcwMKq4783b+JMPDfwRilN/q2A5uD+e0ICGxpcDlDd57kIiTmo7+8L7ELSqY
         Z9FxyOYZUlhmXOSFKhNRJmkIqZpvLFMGcZW+tlTpKKy2QK5IZQBpSyt5KOfskwJNV6S+
         TxMlzU7phlg2iD4kSvj009sw6ZhnnXFhY3ptDN6bRkOJEcX07wMBPA+gWcuof/hPn01U
         vmxA==
X-Forwarded-Encrypted: i=1; AJvYcCUHJvNuO8fA99te2HfVbvEu6r7NgrLrFS2AOZghyL1OF9b0qbCprI9fM6BbbYoB/VF/WwDzw00FZ660FdGy4MvZ2AVJp1x/zF2jVgDFprI=
X-Gm-Message-State: AOJu0YyPWAY6ko24liy9LGrfyg8053qCUSF/Ty67tq962mgBs3ja5Fnn
	6l2gE1FGSMF9FQCmN7DeGS+VIbxTqH9fOdMWs4tgogb1Umuu9tYts+ghQzc1yXk=
X-Google-Smtp-Source: AGHT+IEn0V/qQECJjKpcNHMt9o6EukjJed5x4DbQ/LCrvJUbFIyxu9hpQU8SAgxSwIJFKCPCR73saA==
X-Received: by 2002:a05:620a:24d4:b0:787:edf7:b178 with SMTP id m20-20020a05620a24d400b00787edf7b178mr4669985qkn.27.1709139390715;
        Wed, 28 Feb 2024 08:56:30 -0800 (PST)
Date: Wed, 28 Feb 2024 17:56:28 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
	xen-devel@lists.xenproject.org
Subject: Re: [PATCH] x86/altcall: always use a temporary parameter stashing
 variable
Message-ID: <Zd9lvN2tUrmEt2pg@macbook>
References: <20240228135908.13319-1-roger.pau@citrix.com>
 <11f0e1dd-01b4-4d13-8137-e86b0975ad80@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <11f0e1dd-01b4-4d13-8137-e86b0975ad80@suse.com>

On Wed, Feb 28, 2024 at 03:28:25PM +0100, Jan Beulich wrote:
> On 28.02.2024 14:59, Roger Pau Monne wrote:
> > The usage in ALT_CALL_ARG() on clang of:
> > 
> > register union {
> >     typeof(arg) e;
> >     const unsigned long r;
> > } ...
> > 
> > When `arg` is the first argument to alternative_{,v}call() and
> > const_vlapic_vcpu() is used results in clang 3.5.0 complaining with:
> > 
> > arch/x86/hvm/vlapic.c:141:47: error: non-const static data member must be initialized out of line
> >          alternative_call(hvm_funcs.test_pir, const_vlapic_vcpu(vlapic), vec) )
> > 
> > Workaround this by pulling `arg1` into a local variable, like it's done for
> > further arguments (arg2, arg3...)
> > 
> > Originally arg1 wasn't pulled into a variable because for the a1_ register
> > local variable the possible clobbering as a result of operators on other
> > variables don't matter:
> > 
> > https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html#Local-Register-Variables
> > 
> > Note clang version 3.8.1 seems to already be fixed and don't require the
> > workaround, but since it's harmless do it uniformly everywhere.
> > 
> > Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > Fixes: 2ce562b2a413 ('x86/altcall: use a union as register type for function parameters on clang')
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> I'm okay with this change, but since you don't mention anything in this
> regard: Did you look at whether / how generated code (with gcc) changes?

So the specific example of vlapic_test_irq() shows no changes to the
generated code. bloat-o-meter shows a 0 delta:

add/remove: 0/0 grow/shrink: 0/0 up/down: 0/0 (0)
Function                                     old     new   delta
Total: Before=3570098, After=3570098, chg +0.00%

I assume there can be some reordering of instructions, as we now force
arg1 temporary variable (v1_) to be initialized ahead of the rest of
the arguments.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 17:03:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 17:03:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686753.1069189 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfNKo-0003u3-N5; Wed, 28 Feb 2024 17:03:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686753.1069189; Wed, 28 Feb 2024 17:03:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfNKo-0003tw-KC; Wed, 28 Feb 2024 17:03:10 +0000
Received: by outflank-mailman (input) for mailman id 686753;
 Wed, 28 Feb 2024 17:03:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=M6mx=KF=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rfNKn-0003tq-DJ
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 17:03:09 +0000
Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com
 [2a00:1450:4864:20::12f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3ef3352f-d65b-11ee-afd8-a90da7624cb6;
 Wed, 28 Feb 2024 18:03:08 +0100 (CET)
Received: by mail-lf1-x12f.google.com with SMTP id
 2adb3069b0e04-512f3e75391so4933495e87.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 09:03:08 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 i10-20020a198c4a000000b00512dbd9cf32sm274786lfj.54.2024.02.28.09.03.06
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 09:03:07 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3ef3352f-d65b-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709139788; x=1709744588; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=30nIXP+od+UEsjVHmY9H6m9mbcUi/8rjFTWK7dcCxL4=;
        b=Aflv6hxGy0Z8tArcQ6DuuRwsDbCSSx4rH4hSOlJ03egMjJPHD0JsUFVhCbVg6W7ELT
         WLsCj/SJKerKar+dLTXDZoy18MbbXUQWDbQY9B8jEXe7slowYHdgCanV3iRlMBUfgXSG
         O/gHmNYQV6CGz1aLAbJHc8o7WHlpNFGwxrN/4Rm5ViCTfsDAKy3819SxH8+SePNvD/1a
         WQOL7nwS9pdGt7BZyAWOW2Mm7bgmoySK5kiLiNi5cVg/OwV+ZafkL6W1D69gwgyyOj3z
         lavMTgb8B4u061WfEMyLTB7wDvFHmq08AA23iVhahPfmuVsxXBv/tC+ZuWjrVAIjJw7R
         MHvw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709139788; x=1709744588;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=30nIXP+od+UEsjVHmY9H6m9mbcUi/8rjFTWK7dcCxL4=;
        b=vHOROFeF6LXpXRpahm3Ubsv3ChMfHgTnar4MAR/BZ1lg55XSzn348jx1IftL/PlY83
         WlVcJUgJuXDFZvuj952/zXpqcm99E5VVpr1dX3CQcFWB8JeCVIcUhq5qZo3fsJyoKvd/
         4JxxjmPjJWtwI3IMDaXu0L5rIx933EdfE825OFanM9OCanpkgEwOsAhmlRZBk1vqYMjd
         xQnQoLEBd2ipxWhYgd9H5Sz/lTLuwRA47P40XI8TNbQir87juj6HWMgDrI6S1WOq6tv+
         d8ekVDPSLO+SGIFGMJDipp7s9RJM5gn9sI5IW6ykWMMXa7hZPToftHhy21IWKUh8IP56
         wrXw==
X-Forwarded-Encrypted: i=1; AJvYcCU9HWK5f5b3k+K++oh0U2sqCTMCo8ihzNUOskTXI/xj8FRineqGs4diHh32E3D+YLUz6udTUKArLQL1161ME31vQdj+83ZIQLf7VEau9MQ=
X-Gm-Message-State: AOJu0Yy9z52ivFYdpjyoJtsvk7KiNg4MEzJuzzwwC6JlHPdFpSKoPNAe
	6Sym7m0N/H6Fe1xG5WY3C4ehiBtkI9DmPhPKO9wyXVzhcBTdahrg
X-Google-Smtp-Source: AGHT+IHXlJZqNm+PSjjEfTml7jOtdStwEmV7hJWTn+x6bByrM3DEgf2BSaWdZZ/Pq/YS0Z6AzvKvvQ==
X-Received: by 2002:ac2:43d5:0:b0:512:bda2:7bd9 with SMTP id u21-20020ac243d5000000b00512bda27bd9mr126627lfl.64.1709139787434;
        Wed, 28 Feb 2024 09:03:07 -0800 (PST)
Message-ID: <b76526510f7ef080ddd970e85c3a88f1a7b58e0d.camel@gmail.com>
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Date: Wed, 28 Feb 2024 18:03:06 +0100
In-Reply-To: <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
	 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
	 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Tue, 2024-02-27 at 08:55 +0100, Jan Beulich wrote:
> On 26.02.2024 18:39, Oleksii Kurochko wrote:
> > This patch doesn't represent a strict lower bound for GCC and
> > GNU Binutils; rather, these versions are specifically employed by
> > the Xen RISC-V container and are anticipated to undergo continuous
> > testing.
>=20
> Up and until that container would be updated to a newer gcc. I'm
> afraid I view this as too weak a criteria, but I'm also not meaning
> to
> stand in the way if somebody else wants to ack this patch in this
> form;
> my bare minimum requirement is now met.
>=20
> > --- a/README
> > +++ b/README
> > @@ -48,6 +48,15 @@ provided by your OS distributor:
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - For ARM 64-bit:
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GCC 5.1 or later
> > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GNU Binutils 2.24 or=
 later
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - For RISC-V 64-bit:
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GCC 12.2 or later
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - GNU Binutils 2.39 or late=
r
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 This doesn't represent a st=
rict lower bound for GCC and
> > GNU Binutils;
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rather, these versions are =
specifically employed by the
> > Xen RISC-V
> > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 container and are anticipat=
ed to undergo continuous
> > testing.
>=20
> As per above, I think here it really needs saying "at the time of
> writing"
> or recording a concrete date. Furthermore I expect "these versions"
> relates
> to the specifically named versions and particularly _not_ to "or
> later":
> With the criteria you apply, using later versions (or in fact any
> version
> other than the very specific ones used in the container) would be
> similarly
> untested. Much like x86 and Arm don't have the full range of
> permitted
> tool chain versions continuously tested. Plus don't forget that
> distros may
> apply their own selection of patches on top of what they take from
> upstream
> (and they may also take random snapshots rather than released
> versions).
>=20
> IOW it is hard for me to see why RISC-V needs stronger restrictions
> here
> than other architectures. It ought to be possible to determine a
> baseline
> version. Even if taking the desire to have "pause" available as a
> requirement, gas (and presumably gld) 2.36.1 would already suffice.
I'll be happy to determine a baseline version and RISC-V doesn't need
stronger restriction that why I wrote: "This patch doesn't represent a
strict lower bound for GCC and GNU Binutils".

Would it be good to use for GCC -> "12.2 or later" and for Binutils ->
"2.36.1 or later"?

I missed that I've pushed RISC-V contrainer without fixing version of
archlinux, so you are right that after container update what I wrote
won't be true, as compiler version might be changed.

Just for clarifying when the version will be agreed, does it mean that
I should use a toolchain with mentioned version in this file and each
time to verify that everything still working with this versions?

~ Oleksii



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 18:06:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 18:06:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686761.1069198 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfOJI-0004TO-Aw; Wed, 28 Feb 2024 18:05:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686761.1069198; Wed, 28 Feb 2024 18:05:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfOJI-0004TH-83; Wed, 28 Feb 2024 18:05:40 +0000
Received: by outflank-mailman (input) for mailman id 686761;
 Wed, 28 Feb 2024 18:05:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=WGB+=KF=cloud.com=anthony.perard@srs-se1.protection.inumbo.net>)
 id 1rfOJG-0004TB-Pt
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 18:05:38 +0000
Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com
 [2a00:1450:4864:20::434])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f92f2db3-d663-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 19:05:36 +0100 (CET)
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-33d153254b7so53298f8f.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 10:05:36 -0800 (PST)
Received: from perard.uk.xensource.com (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 e12-20020a5d530c000000b0033e0019b814sm2381870wrv.34.2024.02.28.10.05.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 10:05:35 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f92f2db3-d663-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=cloud.com; s=cloud; t=1709143536; x=1709748336; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=F2FqYtTnxWdlln7CeeFml8oacs7aq896YTM/LaVKHR0=;
        b=fWafAnCZKHEc3J1J7fmhW21eJGySagWYHPLRB0iCD3Ulsfm9x9WhwTfTvHx1NBz0kE
         HRN/KJCbepy65Zx36xszK6GFYMCaV1HRuIHvsu/+kUFazvQVJMkU7fmUowQ0IbJ7uV+x
         YzGBptaYhWrtX6REd7NOP3WSm+1unpuVkxWdI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709143536; x=1709748336;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=F2FqYtTnxWdlln7CeeFml8oacs7aq896YTM/LaVKHR0=;
        b=XL7R8z/KZAFS6nU5aE1qDf2ID+sDsojAYVEqybNgUVgpvay9qsCcxunbQw3h6Q53oX
         Ki1TNJBpwoYFfUYhnA2TrqYYv1TeS0rMJxvvKHWy2RUgBXdrw6eT85jnBwCdNrG0S+Gg
         Ov4bK+enLydmmlNv8n1REPrP9uC3VRDNuZNPS3mQXd9Nt/EWVUvRdVrU1m7ojCOPpB1Y
         I86M12w9VGaL4V41/R9rm8DO5zafDYmUc37AYmEcpKmUEioeUlYiRmTdpov1XBebVUlr
         6jeJhrLKglUoEoQZCSPKnQDN0Aq6wWHh1sTt2Y/YMOBrXfFi06u1omAyphWN7PG13Ppr
         +27w==
X-Gm-Message-State: AOJu0Yz+w6hZw5vZcXOV4BZ46HzAYY9cl/WzYKvQENbsNCAN2k5c0QTW
	K/7UN0erNly4fmF5fn+idHk8tIKxjzl9ebLvZTaWbLKyNIcBTaYbG/1aKuGgsTXHOitg1DU0Dpt
	H
X-Google-Smtp-Source: AGHT+IG6qbC4qJxHw4sNh9WE0/0tdvk+Yd4xV8KXQ7a/KZC0WYT3XV7xrpbPpjbC3sqhxiDcIaPAgg==
X-Received: by 2002:a5d:58e6:0:b0:33d:efbd:517d with SMTP id f6-20020a5d58e6000000b0033defbd517dmr231577wrd.20.1709143536164;
        Wed, 28 Feb 2024 10:05:36 -0800 (PST)
Date: Wed, 28 Feb 2024 18:05:34 +0000
From: Anthony PERARD <anthony.perard@cloud.com>
To: Carlo Nonato <carlo.nonato@minervasys.tech>
Cc: xen-devel@lists.xenproject.org, andrea.bastoni@minervasys.tech,
	Wei Liu <wl@xen.org>, Juergen Gross <jgross@suse.com>,
	Marco Solieri <marco.solieri@minervasys.tech>
Subject: Re: [PATCH v6 06/15] tools: add support for cache coloring
 configuration
Message-ID: <c8e1da62-eea0-4103-b9f7-f5bd05b90e30@perard>
References: <20240129171811.21382-1-carlo.nonato@minervasys.tech>
 <20240129171811.21382-7-carlo.nonato@minervasys.tech>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240129171811.21382-7-carlo.nonato@minervasys.tech>

On Mon, Jan 29, 2024 at 06:18:02PM +0100, Carlo Nonato wrote:
> diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
> index 2ef8b4e054..4b541fffd2 100644
> --- a/tools/include/xenctrl.h
> +++ b/tools/include/xenctrl.h
> @@ -2653,6 +2653,15 @@ int xc_livepatch_replace(xc_interface *xch, char *name, uint32_t timeout, uint32
>  int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
>                           xen_pfn_t start_pfn, xen_pfn_t nr_pfns);
>  
> +/*
> + * Set LLC colors for a domain.
> + * This is an internal hypercall. It can only be used directly after domain

What is an "internal hypercall"? Can those even exist?

> + * creation. An attempt to use it afterwards will result in an error.
> + */
> +int xc_domain_set_llc_colors(xc_interface *xch, uint32_t domid,
> +                             const unsigned int *llc_colors,
> +                             unsigned int num_llc_colors);
> +
>  #if defined(__arm__) || defined(__aarch64__)
>  int xc_dt_overlay(xc_interface *xch, void *overlay_fdt,
>                    uint32_t overlay_fdt_size, uint8_t overlay_op);
> diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c
> index f2d9d14b4d..ad02288659 100644
> --- a/tools/libs/ctrl/xc_domain.c
> +++ b/tools/libs/ctrl/xc_domain.c
> @@ -2180,6 +2180,40 @@ int xc_domain_soft_reset(xc_interface *xch,
>      domctl.domain = domid;
>      return do_domctl(xch, &domctl);
>  }
> +
> +int xc_domain_set_llc_colors(xc_interface *xch, uint32_t domid,
> +                             const unsigned int *llc_colors,
> +                             unsigned int num_llc_colors)
> +{
> +    struct xen_domctl domctl = {};
> +    DECLARE_HYPERCALL_BUFFER(uint32_t, local);
> +    int ret = -1;
> +
> +    if ( num_llc_colors )
> +    {
> +        size_t bytes = sizeof(uint32_t) * num_llc_colors;

Isn't there a risk of overflow, maybe only on 32bit platform? Or maybe
that doesn't matter because the hypervisor should be able to find out if
the buffer is too short, right?

> +        local = xc_hypercall_buffer_alloc(xch, local, bytes);
> +        if ( local == NULL )
> +        {
> +            PERROR("Could not allocate LLC colors for set_llc_colors");
> +            return -ENOMEM;
> +        }
> +        memcpy(local, llc_colors, bytes);
> +        set_xen_guest_handle(domctl.u.set_llc_colors.llc_colors, local);
> +    }
> +
> +    domctl.cmd = XEN_DOMCTL_set_llc_colors;
> +    domctl.domain = domid;
> +    domctl.u.set_llc_colors.num_llc_colors = num_llc_colors;
> +
> +    ret = do_domctl(xch, &domctl);
> +
> +    if ( local )
> +        xc_hypercall_buffer_free(xch, local);

It doesn't looks like you need to check if "local != NULL" before
calling xc_hypercall_buffer_free(), it should work even with
local==NULL. This is even used multiple time in xc_kexec.

> +
> +    return ret;
> +}

Thanks,

-- 
Anthony PERARD


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 18:48:43 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 18:48:43 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686764.1069209 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfOyh-0001ec-FL; Wed, 28 Feb 2024 18:48:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686764.1069209; Wed, 28 Feb 2024 18:48:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfOyh-0001eV-BU; Wed, 28 Feb 2024 18:48:27 +0000
Received: by outflank-mailman (input) for mailman id 686764;
 Wed, 28 Feb 2024 18:48:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfOyg-0001eL-0Z; Wed, 28 Feb 2024 18:48:26 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfOyf-0000oW-Rz; Wed, 28 Feb 2024 18:48:25 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfOyf-00024E-IE; Wed, 28 Feb 2024 18:48:25 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfOyf-0000Kw-Hq; Wed, 28 Feb 2024 18:48:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=bIamcWNpKoCOXwUcWvqzytgnWROjemccM+2ti8BffFM=; b=Y5AbboNVs4k/j0yX83tHJdqv7o
	I4LBlw0Erdi5pd2GSiuUBtGlIHdR5CEzk39ksCHIguA29hohX2Gcs8H9fV74XyuDKV0+mw/M6/ykQ
	Y7Ui3DnRauBLXDN41qH5UpKD/JTlmscDEFI2C3IOJ2gH5jI71FXETgzAaiOelXJ7llUs=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184806-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184806: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=195e75371b13c4f7ecdf7b5c50aed0d02f2d7ce8
X-Osstest-Versions-That:
    xen=b2b3d40f9a030ba79ce3abadccb5da485af33589
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 18:48:25 +0000

flight 184806 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184806/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  195e75371b13c4f7ecdf7b5c50aed0d02f2d7ce8
baseline version:
 xen                  b2b3d40f9a030ba79ce3abadccb5da485af33589

Last test of basis   184805  2024-02-28 12:04:00 Z    0 days
Testing same since   184806  2024-02-28 16:00:24 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   b2b3d40f9a..195e75371b  195e75371b13c4f7ecdf7b5c50aed0d02f2d7ce8 -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 19:25:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 19:25:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686774.1069234 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfPYF-00074O-AN; Wed, 28 Feb 2024 19:25:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686774.1069234; Wed, 28 Feb 2024 19:25:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfPYF-00074H-7g; Wed, 28 Feb 2024 19:25:11 +0000
Received: by outflank-mailman (input) for mailman id 686774;
 Wed, 28 Feb 2024 19:25:09 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfPYD-000747-JY; Wed, 28 Feb 2024 19:25:09 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfPYD-0001UU-HI; Wed, 28 Feb 2024 19:25:09 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfPYD-0004SC-2S; Wed, 28 Feb 2024 19:25:09 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfPYD-0006WX-1s; Wed, 28 Feb 2024 19:25:09 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Y6MPJl+dC2YaLWsIg64lOb2f8ZX/+2omI8b26/Lk0Yo=; b=draFRh7r9xZY2UOkaOQhFoxd/1
	I5WDV4EpM14QtzYoETYSArxSNV/D/FiSwpqMi5PY+QeN1xTX19UMW55zEv3q8UMVgiRK03d68ZuP6
	Q0oyJIMkGcwBiK9oV1fLPxI/bONpp1f3OJgSvABGiKJ0qVwZbRcsZ+Xf57JBPg8f/Aj8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184796-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184796: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-libvirt-raw:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87
X-Osstest-Versions-That:
    xen=37541208f119a9c552c6c6c3246ea61be0d44035
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 19:25:09 +0000

flight 184796 xen-unstable real [real]
flight 184808 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184796/
http://logs.test-lab.xenproject.org/osstest/logs/184808/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-raw 17 guest-start/debian.repeat fail pass in 184808-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184780
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184780
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184780
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184780
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184780
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184780
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184780
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184780
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184780
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184780
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184780
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184780
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87
baseline version:
 xen                  37541208f119a9c552c6c6c3246ea61be0d44035

Last test of basis   184780  2024-02-27 09:47:29 Z    1 days
Testing same since   184796  2024-02-27 18:09:07 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Maria Celeste Cesario <maria.celeste.cesario@bugseng.com>
  Olaf Hering <olaf@aepfle.de>
  Roger Pau Monné <roger.pau@citrix.com>
  Shawn Anastasio <sanastasio@raptorengineering.com>
  Simone Ballarin <simone.ballarin@bugseng.com>
  Stefano Stabellini <sstabellini@kernel.org>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 fail    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   37541208f1..ca7c872c70  ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87 -> master


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 19:49:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 19:49:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686780.1069243 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfPv6-0001Sy-8G; Wed, 28 Feb 2024 19:48:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686780.1069243; Wed, 28 Feb 2024 19:48:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfPv6-0001Sr-5d; Wed, 28 Feb 2024 19:48:48 +0000
Received: by outflank-mailman (input) for mailman id 686780;
 Wed, 28 Feb 2024 19:48:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfPv4-0001Se-PI; Wed, 28 Feb 2024 19:48:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfPv4-0001sC-MA; Wed, 28 Feb 2024 19:48:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfPv4-0005lS-9c; Wed, 28 Feb 2024 19:48:46 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfPv4-0004eN-98; Wed, 28 Feb 2024 19:48:46 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=ddw7VDKEadJ/sG7VtdKIq8T1cjxGDRo7iO85PccL0sI=; b=rp5O10KdCfLAUxze2Y8Fu2xF5q
	2H8GqtXRDCyCbocCGZ+3uoNTnoWgyOgcP8rIIYhWs4kTF7jFbqhPp9T57NXob2i0IJfoao+WDQMCu
	F6s6pkWQLcDK4fThs/SMoAc37TJegi+P94dpzZ7J36IX1ymRml31W/NaNEwWb5zp/lwA=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184809-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xtf test] 184809: all pass - PUSHED
X-Osstest-Versions-This:
    xtf=ad98fbacc7ed6b709f87bf0ef64b81eafb557cce
X-Osstest-Versions-That:
    xtf=d1de27cf787e9c86610b11f6fe13d90f5ca91e27
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 19:48:46 +0000

flight 184809 xtf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184809/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xtf                  ad98fbacc7ed6b709f87bf0ef64b81eafb557cce
baseline version:
 xtf                  d1de27cf787e9c86610b11f6fe13d90f5ca91e27

Last test of basis   184556  2024-02-01 19:13:08 Z   27 days
Testing same since   184809  2024-02-28 17:41:19 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>

jobs:
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-amd64-pvops                                            pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xtf.git
   d1de27c..ad98fba  ad98fbacc7ed6b709f87bf0ef64b81eafb557cce -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 21:03:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 21:03:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686786.1069257 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfR5h-0002HX-MG; Wed, 28 Feb 2024 21:03:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686786.1069257; Wed, 28 Feb 2024 21:03:49 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfR5h-0002HQ-Jg; Wed, 28 Feb 2024 21:03:49 +0000
Received: by outflank-mailman (input) for mailman id 686786;
 Wed, 28 Feb 2024 21:03:48 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfR5g-0002HG-KH; Wed, 28 Feb 2024 21:03:48 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfR5g-0003Iq-IZ; Wed, 28 Feb 2024 21:03:48 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfR5g-0000tG-1B; Wed, 28 Feb 2024 21:03:48 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfR5g-0003MU-0g; Wed, 28 Feb 2024 21:03:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=fm0BaeVBm9HlRqPF4ODn4UTW2SXT6LpYHlf+EC+7MMM=; b=JFnNHNce7jypYUNzrqP6OrdRQh
	QANSS4n2tDR8Kc67kmSaJczr3cDU9PoTB/ZKewUrJzuTmABZPhPRR7iuzChZhz2g815Vq4FPWdrSC
	g1rVThWi69m7zobzzU69EJ/Iff18oNkZlq0Ty4VAH+z3lvXuJzQ2E1RJyaibn/v9tSvo=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184804-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.16-testing test] 184804: tolerable trouble: fail/pass/starved - PUSHED
X-Osstest-Failures:
    xen-4.16-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.16-testing:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.16-testing:test-arm64-arm64-libvirt-raw:build-check(1):starved:nonblocking
    xen-4.16-testing:test-arm64-arm64-libvirt-xsm:build-check(1):starved:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl:build-check(1):starved:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-credit1:build-check(1):starved:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-credit2:build-check(1):starved:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-thunderx:build-check(1):starved:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-vhd:build-check(1):starved:nonblocking
    xen-4.16-testing:test-arm64-arm64-xl-xsm:build-check(1):starved:nonblocking
    xen-4.16-testing:build-arm64-pvops:hosts-allocate:starved:nonblocking
X-Osstest-Versions-This:
    xen=ffd75b161de76e43fd54d751ab70d2fdf6e2e87d
X-Osstest-Versions-That:
    xen=e481fc9f32339ebf9ddd171a3995a3e44527d148
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 21:03:48 +0000

flight 184804 xen-4.16-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184804/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184530
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184530
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184530
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184530
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184530
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184530
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184530
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184530
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184530
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184530
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184530
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184530
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-arm64-arm64-libvirt-raw  1 build-check(1)               starved  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)               starved  n/a
 test-arm64-arm64-xl           1 build-check(1)               starved  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)               starved  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)               starved  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)               starved  n/a
 test-arm64-arm64-xl-vhd       1 build-check(1)               starved  n/a
 test-arm64-arm64-xl-xsm       1 build-check(1)               starved  n/a
 build-arm64-pvops             2 hosts-allocate               starved  n/a

version targeted for testing:
 xen                  ffd75b161de76e43fd54d751ab70d2fdf6e2e87d
baseline version:
 xen                  e481fc9f32339ebf9ddd171a3995a3e44527d148

Last test of basis   184530  2024-01-30 14:07:07 Z   29 days
Testing same since   184785  2024-02-27 13:36:59 Z    1 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Jan Beulich <jbeulich@suse.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            starved 
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          starved 
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 starved 
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      starved 
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  starved 
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  starved 
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 starved 
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 starved 
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      starved 
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   e481fc9f32..ffd75b161d  ffd75b161de76e43fd54d751ab70d2fdf6e2e87d -> stable-4.16


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 21:28:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 21:28:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686791.1069268 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfRTC-0004wX-KC; Wed, 28 Feb 2024 21:28:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686791.1069268; Wed, 28 Feb 2024 21:28:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfRTC-0004wQ-GS; Wed, 28 Feb 2024 21:28:06 +0000
Received: by outflank-mailman (input) for mailman id 686791;
 Wed, 28 Feb 2024 21:28:05 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfRTB-0004wG-OU; Wed, 28 Feb 2024 21:28:05 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfRTB-0003iT-KR; Wed, 28 Feb 2024 21:28:05 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfRTB-0001xq-C4; Wed, 28 Feb 2024 21:28:05 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfRTB-000558-Bd; Wed, 28 Feb 2024 21:28:05 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=G55izSiOAJwOUwuNRxi8g3zrt9Vd+S6ivAgxEsO7vZ0=; b=XKS5nlVu9NuPUt+KQN5NnVOg+4
	HJ3ubOxNiC9eUjrqz8eyZlzjzaE04xrBPKaKu7BHMcfILZsjck8L/D70K75m84mzvivWlZXWuvMv6
	c8twHzao1seV7sc/UMIftbvqkr/bedKQyEuv1KlFeXO2EIqzF9X9zFswALmIc4gx+i1g=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184810-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184810: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=54fd7b997470e6686667ca8e18f9ba6139efcdea
X-Osstest-Versions-That:
    xen=195e75371b13c4f7ecdf7b5c50aed0d02f2d7ce8
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 21:28:05 +0000

flight 184810 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184810/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  54fd7b997470e6686667ca8e18f9ba6139efcdea
baseline version:
 xen                  195e75371b13c4f7ecdf7b5c50aed0d02f2d7ce8

Last test of basis   184806  2024-02-28 16:00:24 Z    0 days
Testing same since   184810  2024-02-28 19:02:05 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Roger Pau Monné <roger.pau@citrix.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   195e75371b..54fd7b9974  54fd7b997470e6686667ca8e18f9ba6139efcdea -> smoke


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 21:32:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 21:32:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686796.1069278 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfRXf-0006P9-6h; Wed, 28 Feb 2024 21:32:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686796.1069278; Wed, 28 Feb 2024 21:32:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfRXf-0006P2-2R; Wed, 28 Feb 2024 21:32:43 +0000
Received: by outflank-mailman (input) for mailman id 686796;
 Wed, 28 Feb 2024 21:32:41 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfRXd-0006Os-KU; Wed, 28 Feb 2024 21:32:41 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfRXd-0003nU-JQ; Wed, 28 Feb 2024 21:32:41 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfRXd-00029f-9d; Wed, 28 Feb 2024 21:32:41 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfRXd-0007SK-98; Wed, 28 Feb 2024 21:32:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hWPH5iqhQnzSyRA38YAaYE0XuYVxIi2R76RTE8JY5G0=; b=2KTuOj2EioBtdMSYue7TWvGJ23
	poudmkk1i145QmeDkjE9A2GT9U8zmDp3isnbcHSd1+2kskKYDlz4IRv9KgAKANc2OwYVg+zz/RDS6
	+T0w9bv9wrb8SNm0etG7gC7SZNrB1I7OhAjnrZrZNleRal7aBHusNFWbQN6F8Cg+23GY=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184812-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xtf test] 184812: all pass - PUSHED
X-Osstest-Versions-This:
    xtf=5c37e0b719c925207db50e89a5b11d7ce78cf1fb
X-Osstest-Versions-That:
    xtf=ad98fbacc7ed6b709f87bf0ef64b81eafb557cce
From: osstest service owner <osstest-admin@xenproject.org>
Date: Wed, 28 Feb 2024 21:32:41 +0000

flight 184812 xtf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184812/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xtf                  5c37e0b719c925207db50e89a5b11d7ce78cf1fb
baseline version:
 xtf                  ad98fbacc7ed6b709f87bf0ef64b81eafb557cce

Last test of basis   184809  2024-02-28 17:41:19 Z    0 days
Testing same since   184812  2024-02-28 20:13:14 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>

jobs:
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-amd64-pvops                                            pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xtf.git
   ad98fba..5c37e0b  5c37e0b719c925207db50e89a5b11d7ce78cf1fb -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 22:27:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 22:27:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686804.1069289 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfSOk-0003hq-68; Wed, 28 Feb 2024 22:27:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686804.1069289; Wed, 28 Feb 2024 22:27:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfSOk-0003hj-1h; Wed, 28 Feb 2024 22:27:34 +0000
Received: by outflank-mailman (input) for mailman id 686804;
 Wed, 28 Feb 2024 22:27:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=msSf=KF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfSOj-0003hd-BS
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 22:27:33 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8d748a69-d688-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 23:27:29 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 06884CE2267;
 Wed, 28 Feb 2024 22:27:26 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6A210C433F1;
 Wed, 28 Feb 2024 22:27:24 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8d748a69-d688-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709159245;
	bh=WQPApiHH6A9D93VDTMCaZkmgN6V7GL8Od7WxVYedFXY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=kO/lmmJZijVTPHovlIo/QvNkCTUHJorrWPLhyHY0GMBd8dOPme1lNeh9AVjxMSPG5
	 7E0MNj3BeqXxhuRJRkip9SueiqOk3Yw8lgqVq+jnbPTLuFtwqf/kKORBHntvBddtIH
	 AjzzOdmcAjjZzZBH+qDfAzGUA2Ryi0p+waniHuiIVgjVr4CcQxaAMJ/s7pE+LnXVao
	 9Wz4geqZ+kcjWPH8frwKuAlsUXkznEd0qMFaCNpWRNT6KbgVPxtOCU9IPNTxSeQEm2
	 MC0HEUvzJefxiBIn9kdyWx44O53zHMsoZCPZXUtylMPmFPAnPZTYrOamSpz1CzOh5U
	 gy6kZNg4QgZjg==
Date: Wed, 28 Feb 2024 14:27:22 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Michal Orzel <michal.orzel@amd.com>
cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH] xen/arm: Fix arm32 build failure when early printk is
 enabled
In-Reply-To: <af11620a-ec16-4677-b968-d8e36c4555d2@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402281425260.575685@ubuntu-linux-20-04-desktop>
References: <20240228103555.172101-1-michal.orzel@amd.com> <9201bdc5-2e4c-48b7-8c92-f5063fde90b6@xen.org> <af11620a-ec16-4677-b968-d8e36c4555d2@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 28 Feb 2024, Michal Orzel wrote:
> Hi Julien,
> 
> On 28/02/2024 12:42, Julien Grall wrote:
> > 
> > 
> > Hi Michal,
> > 
> > On 28/02/2024 10:35, Michal Orzel wrote:
> >> Commit 0441c3acc7e9 forgot to rename FIXMAP_CONSOLE to FIX_CONSOLE in
> >> TEMPORARY_EARLY_UART_VIRTUAL_ADDRESS macro. This results in a build
> >> failure on arm32, when early printk is enabled:
> >> arch/arm/arm32/mmu/head.S:311: Error: invalid operands (*UND* and *ABS* sections) for `*'
> > 
> > Good catch! Somewhat related I wonder whether we should add earlyprintk
> > testing in gitlab?
> I thought about adding this and I think we should at least have build jobs (hypervisor only, no toolstack)
> selecting early printk. When it comes to testing if early printk works, I'm not sure. It'd be nice
> but FWIR we have limited bandwidth.
> 
> @Stefano, what's your opinion?

I think it would be a good and quick test to have. To save testing
bandwidth I think we should reduce the amount of debug/non-debug
variations of the same tests that we have.


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 22:38:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 22:38:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686807.1069298 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfSZ3-0005Kj-3T; Wed, 28 Feb 2024 22:38:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686807.1069298; Wed, 28 Feb 2024 22:38:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfSZ3-0005Kc-0E; Wed, 28 Feb 2024 22:38:13 +0000
Received: by outflank-mailman (input) for mailman id 686807;
 Wed, 28 Feb 2024 22:38:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=msSf=KF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfSZ1-0005KW-Kl
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 22:38:11 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0b1ed263-d68a-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 23:38:09 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 5BC63CE1FD6;
 Wed, 28 Feb 2024 22:38:04 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 69594C433F1;
 Wed, 28 Feb 2024 22:38:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0b1ed263-d68a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709159883;
	bh=5WaCi+6gVTyfUU0Op00ytWhMO9xw7BQScrZbELM1l1Q=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=bg8Ho0ICp8cDupFShhOwIsLfSkS7QOhJ/uVHZnNKL8wA2xoiYLFTqYGSRy04es9Ux
	 1MvrpRmYQqMDvUhBbxN1rs6ZTbg8r2yw0IhLrk3JxV3hMDOEIGFHY7aJ1RmiJhjWNB
	 TXJgjtkbaUsRCP03JR2hMTTrC8qODPHdhgoKGqK0MuW34+9mc7OPR4YHe1+PsOVsAE
	 nIpmgY2h9UAamQsCWR3aRntrzWN/6Ak/e8rXzcxzG0GcJjAXjk3U/m9LZ9j5W2iFYr
	 Ov7NC8Jk8LCadrSiPP1k54I02S3NqmdciY1TWzlb5aiVUfNDKZmwnH6PDFMdKxEJ1s
	 ECaiLPzD27TQw==
Date: Wed, 28 Feb 2024 14:38:00 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Nicola Vetrini <nicola.vetrini@bugseng.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Jan Beulich <jbeulich@suse.com>, consulting@bugseng.com, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
In-Reply-To: <bcae31fb-856d-4737-b780-f41b3e24cfa7@xen.org>
Message-ID: <alpine.DEB.2.22.394.2402281437470.575685@ubuntu-linux-20-04-desktop>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com> <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com> <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com> <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
 <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com> <7a8e610e-913e-4a56-8ce1-6dd6abd894f4@xen.org> <4bee79ca-7a7e-4bcc-ac97-5a5a57ec2c91@suse.com> <alpine.DEB.2.22.394.2402271808410.575685@ubuntu-linux-20-04-desktop> <6af04933659178b3ccabc5caf646273c@bugseng.com>
 <bcae31fb-856d-4737-b780-f41b3e24cfa7@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 28 Feb 2024, Julien Grall wrote:
> Hi Nicola,
> 
> On 28/02/2024 11:09, Nicola Vetrini wrote:
> > > I asked Roberto if void casts are an option for compliance.
> > > 
> > 
> > void casts are an option for sure. The rationale for the rule explicitly
> > lists them as a compliance mechanism. An interesting aspect is what would be
> > the consensus around void casts on functions whose return value is always
> > ignored vs. functions whose return value is sometimes ignored.
> 
> If a return is always ignored, then the function should return void. For the
> second case, I think it will be on the case by case basis.

+1

 
> > > In any case, I don't think we should use void casts in the specific
> > > cases this patch is dealing with. Void casts (if anything) should be a
> > > last resort while this patch fixes the issue in a better way.
> 
> +1.



From xen-devel-bounces@lists.xenproject.org Wed Feb 28 22:46:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 22:46:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686809.1069308 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfSgZ-00073u-RU; Wed, 28 Feb 2024 22:45:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686809.1069308; Wed, 28 Feb 2024 22:45:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfSgZ-00073n-Os; Wed, 28 Feb 2024 22:45:59 +0000
Received: by outflank-mailman (input) for mailman id 686809;
 Wed, 28 Feb 2024 22:45:58 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=msSf=KF=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfSgY-00073h-Cw
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 22:45:58 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 21ebd94a-d68b-11ee-a1ee-f123f15fe8a2;
 Wed, 28 Feb 2024 23:45:56 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id B76C7614CA;
 Wed, 28 Feb 2024 22:45:54 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 80F19C433C7;
 Wed, 28 Feb 2024 22:45:53 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 21ebd94a-d68b-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709160354;
	bh=cjtlg/QVrYzErKBipwK6r+KHc9Hh6O43ER2xx5rOxfY=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=tdThU2SnCSD5Hl6IvVakztb48od9ZdR0amVvZDF14jKzbS73hPeryNCJnFnPPkXd6
	 QHa45DzbR+daS/Jpj1l4mk8Y+EzeRRpq7G+uUppRCnYWc4sb1l+OaDGrtJn7FWjSjL
	 zGDDjRAoQp5Gp1YD2nZzJ17FWf8z1WMYshPFSAMlF1o6vRFLYbk7D8Hz+uAQnwazjK
	 gD2X40kpp94zkoraEhJnQPVR1JyzXx4s6UghdZ8Ykxw9KRJFS9gw8Vgsc+DFv/LsD9
	 nAV9b+K9Tw0oVc5WdxbwZutsLRdDvW/gL4RtcU1N1IkeR4ONNdqqvvDcBnoVx9StX5
	 jpytTpV86ZKrw==
Date: Wed, 28 Feb 2024 14:45:51 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    Anthony Perard <anthony.perard@citrix.com>, 
    Kelly Choi <kelly.choi@cloud.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: preparations for 4.18.1
In-Reply-To: <521a4d3b-8ef1-4449-97bd-1a3d3e0e35ec@xen.org>
Message-ID: <alpine.DEB.2.22.394.2402281445450.575685@ubuntu-linux-20-04-desktop>
References: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com> <1f251f2e-91de-4f81-a93c-dcb95746d2e5@xen.org> <5a4e8583-7747-4f24-94f0-d040dabb5b04@suse.com> <521a4d3b-8ef1-4449-97bd-1a3d3e0e35ec@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 28 Feb 2024, Julien Grall wrote:
> Hi Jan,
> 
> On 28/02/2024 12:58, Jan Beulich wrote:
> > On 28.02.2024 12:50, Julien Grall wrote:
> > > On 27/02/2024 13:19, Jan Beulich wrote:
> > > > All,
> > > > 
> > > > the release is due in two to three weeks. Please point out backports you
> > > > find
> > > > missing from the respective staging branch, but which you consider
> > > > relevant.
> > > 
> > > For Arm:
> > > 
> > > e11f576650 ("xen/arm: Fix UBSAN failure in start_xen()")
> > 
> > Which I assume you or Stefano will take care of?
> 
> I was expecting Stefano would do it as he did the backports in the past.

Done


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 22:59:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 22:59:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686814.1069322 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfStA-0000bL-Vd; Wed, 28 Feb 2024 22:59:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686814.1069322; Wed, 28 Feb 2024 22:59:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfStA-0000bE-Rj; Wed, 28 Feb 2024 22:59:00 +0000
Received: by outflank-mailman (input) for mailman id 686814;
 Wed, 28 Feb 2024 22:58:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfSt9-0000b8-Cc
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 22:58:59 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfSt7-0005Gd-G6; Wed, 28 Feb 2024 22:58:57 +0000
Received: from gw1.octic.net ([88.97.20.152] helo=[10.0.1.193])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfSt7-0005dJ-AV; Wed, 28 Feb 2024 22:58:57 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=53vrm0Ud7rmFbogaJsCW2UgFz1QgmxpZIY5PWAqI8+k=; b=UvoyBCqogSbdWea3gCbFosm03P
	Gz+vPW8bSs3rW3aVUHTj+7hW1rNdZEJtnyt+MJDvOecdwNaHzES0xVigWt3YJwkfPoP6UnEONTVIE
	WFdXce5BWUi1PZJrNrYnP7w/DWwCm5bFWckG/bENPG9T4LFF0qP/31CIXEswFdYs/c+4=;
Message-ID: <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
Date: Wed, 28 Feb 2024 22:58:55 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 27/02/2024 07:55, Jan Beulich wrote:
> On 26.02.2024 18:39, Oleksii Kurochko wrote:
>> This patch doesn't represent a strict lower bound for GCC and
>> GNU Binutils; rather, these versions are specifically employed by
>> the Xen RISC-V container and are anticipated to undergo continuous
>> testing.
> 
> Up and until that container would be updated to a newer gcc. I'm
> afraid I view this as too weak a criteria,

I disagree. We have to decide a limit at some point. It is sensible to 
say that we are only supporting what we can tests. AFAIK, this is what 
QEMU has been doing.

> but I'm also not meaning to
> stand in the way if somebody else wants to ack this patch in this form;
> my bare minimum requirement is now met.
> 
>> --- a/README
>> +++ b/README
>> @@ -48,6 +48,15 @@ provided by your OS distributor:
>>         - For ARM 64-bit:
>>           - GCC 5.1 or later
>>           - GNU Binutils 2.24 or later
>> +      - For RISC-V 64-bit:
>> +        - GCC 12.2 or later
>> +        - GNU Binutils 2.39 or later
>> +        This doesn't represent a strict lower bound for GCC and GNU Binutils;
>> +        rather, these versions are specifically employed by the Xen RISC-V
>> +        container and are anticipated to undergo continuous testing.
> 
> As per above, I think here it really needs saying "at the time of writing"
> or recording a concrete date. Furthermore I expect "these versions" relates
> to the specifically named versions and particularly _not_ to "or later":
> With the criteria you apply, using later versions (or in fact any version
> other than the very specific ones used in the container) would be similarly
> untested. Much like x86 and Arm don't have the full range of permitted
> tool chain versions continuously tested. Plus don't forget that distros may
> apply their own selection of patches on top of what they take from upstream
> (and they may also take random snapshots rather than released versions).

TBH, I think this should be dropped from the README. With the wording, 
it implies that older GCC would work, but this is not a guarantee.

The same for Arm, I suspect some revision of GCC below 5.1 that may 
work. But that's just convenience to list a lower limit.

With the sentence dropped, I would be happy to ack this patch.

> 
> IOW it is hard for me to see why RISC-V needs stronger restrictions here
> than other architectures. It ought to be possible to determine a baseline
> version. Even if taking the desire to have "pause" available as a
> requirement, gas (and presumably gld) 2.36.1 would already suffice.

I think we want to bump it on Arm. There are zero reasons to try to keep 
a lower versions if nobody tests/use it in production.

I would suggest to do the same on x86. What's the point of try to 
support Xen with a 15+ years old compiler?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Wed Feb 28 23:11:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Wed, 28 Feb 2024 23:11:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686817.1069332 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfT5F-0003VX-16; Wed, 28 Feb 2024 23:11:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686817.1069332; Wed, 28 Feb 2024 23:11:28 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfT5E-0003VQ-UC; Wed, 28 Feb 2024 23:11:28 +0000
Received: by outflank-mailman (input) for mailman id 686817;
 Wed, 28 Feb 2024 23:11:27 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=eIhW=KF=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfT5D-0003VK-Ms
 for xen-devel@lists.xenproject.org; Wed, 28 Feb 2024 23:11:27 +0000
Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com
 [2a00:1450:4864:20::22a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b1dc3a82-d68e-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 00:11:25 +0100 (CET)
Received: by mail-lj1-x22a.google.com with SMTP id
 38308e7fff4ca-2d28e465655so3514761fa.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 15:11:25 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 f18-20020a05600c155200b00412b68cc169sm246212wmg.16.2024.02.28.15.11.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 15:11:23 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b1dc3a82-d68e-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709161885; x=1709766685; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=eb9sSmXWyyVHX0TuZl+mGrxr1zi5wA1dX/FkBS2Rw0Q=;
        b=Lkm6hmXJRWs4eh9ve1wjX8pZWl5s/i7wQ8d2fRiyVz7n4p7yprfABZVLeoIokcGy9l
         Hk+gbWn8+KpyTF6TiL4jaKyy/aAb5bKuCRwreV7w8+K/InooHSti3JhRbBxXXMamx9Tq
         u2ncvBsgeuzoUA+C7b3JME8bmL+zUNE9wzRgo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709161885; x=1709766685;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=eb9sSmXWyyVHX0TuZl+mGrxr1zi5wA1dX/FkBS2Rw0Q=;
        b=r+p7fvXVAKZcyp81CBN9WUYWNn3l6XAbNiy9xjPY1KQciv3t6BIOhANmaEc+2dvKSu
         sWm1ZVYpeXSul/Oh41r83GeR66XqrNpMTi9jF4f4sw10V0CafKyRK2QlXLMZJZFLVp42
         vpOmtAu4u+oGn/CBRUXJ+3Of4zTHmxPhwolUpH3UV04e4RbsnIHcT88iwxjBdAf3+Es4
         fLbjaW3IpP/ekPWX20PybVe64Vv7rG78ar5aBZ68qZaoyrJ1vsQf7yJK0kuCiQZnxguw
         q3JlaJMFTM3XBYMgijpqssYS8nlnPQJh7X/LHBED2WbvZ0jAODB0F7ewB03qG0hzYwnM
         YKFQ==
X-Forwarded-Encrypted: i=1; AJvYcCVofu8KB6qUIvlItgb+g+GGhIwfML84RViZ/d11++db4DZmECxKF6kZ0S4vqkme91V8pjhs0c+/OEYhJP69XEs5nQpvLdHqNtZcQsGcKPs=
X-Gm-Message-State: AOJu0YwdzR+lFpwBF2PxKVjnous5/rvCOADhp3V7L6BggXWPoiSQlv6J
	xdrlNsDCd/LvVvIGUSOwuRdF0oqRtmo0LnnDCshw19JE4e/eG1fuBbULRZE/aZE=
X-Google-Smtp-Source: AGHT+IEyB8O85gk5mxNzLVkTwjQdUUN4mlnYV2jcrna5eO2lSipyl35cklvEFtd1vm/rxi8sAMKCQQ==
X-Received: by 2002:a2e:781a:0:b0:2d2:6ed5:e45a with SMTP id t26-20020a2e781a000000b002d26ed5e45amr150543ljc.12.1709161884751;
        Wed, 28 Feb 2024 15:11:24 -0800 (PST)
Message-ID: <d0afb4d6-9fbc-49de-9f07-f76e4aeda8c1@citrix.com>
Date: Wed, 28 Feb 2024 23:11:22 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-GB
To: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/02/2024 10:58 pm, Julien Grall wrote:
> Hi Jan,
>
> On 27/02/2024 07:55, Jan Beulich wrote:
>> On 26.02.2024 18:39, Oleksii Kurochko wrote:
>>> This patch doesn't represent a strict lower bound for GCC and
>>> GNU Binutils; rather, these versions are specifically employed by
>>> the Xen RISC-V container and are anticipated to undergo continuous
>>> testing.
>>
>> Up and until that container would be updated to a newer gcc. I'm
>> afraid I view this as too weak a criteria,
>
> I disagree. We have to decide a limit at some point. It is sensible to
> say that we are only supporting what we can tests. AFAIK, this is what
> QEMU has been doing.
>
>> but I'm also not meaning to
>> stand in the way if somebody else wants to ack this patch in this form;
>> my bare minimum requirement is now met.
>>
>>> --- a/README
>>> +++ b/README
>>> @@ -48,6 +48,15 @@ provided by your OS distributor:
>>>         - For ARM 64-bit:
>>>           - GCC 5.1 or later
>>>           - GNU Binutils 2.24 or later
>>> +      - For RISC-V 64-bit:
>>> +        - GCC 12.2 or later
>>> +        - GNU Binutils 2.39 or later
>>> +        This doesn't represent a strict lower bound for GCC and GNU
>>> Binutils;
>>> +        rather, these versions are specifically employed by the Xen
>>> RISC-V
>>> +        container and are anticipated to undergo continuous testing.
>>
>> As per above, I think here it really needs saying "at the time of
>> writing"
>> or recording a concrete date. Furthermore I expect "these versions"
>> relates
>> to the specifically named versions and particularly _not_ to "or later":
>> With the criteria you apply, using later versions (or in fact any
>> version
>> other than the very specific ones used in the container) would be
>> similarly
>> untested. Much like x86 and Arm don't have the full range of permitted
>> tool chain versions continuously tested. Plus don't forget that
>> distros may
>> apply their own selection of patches on top of what they take from
>> upstream
>> (and they may also take random snapshots rather than released versions).
>
> TBH, I think this should be dropped from the README. With the wording,
> it implies that older GCC would work, but this is not a guarantee.
>
> The same for Arm, I suspect some revision of GCC below 5.1 that may
> work. But that's just convenience to list a lower limit.
>
> With the sentence dropped, I would be happy to ack this patch.
>
>>
>> IOW it is hard for me to see why RISC-V needs stronger restrictions here
>> than other architectures. It ought to be possible to determine a
>> baseline
>> version. Even if taking the desire to have "pause" available as a
>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
>
> I think we want to bump it on Arm. There are zero reasons to try to
> keep a lower versions if nobody tests/use it in production.
>
> I would suggest to do the same on x86. What's the point of try to
> support Xen with a 15+ years old compiler?

There is a material cost to supporting ancient toolchains.  I'm
increasingly unwilling to keep paying.

I'm also bored of needing to support versions of binutils which don't
know the Virt instructions, which are approaching 2 decades old now.

There are very good reasons to move to GCC 5.1 minimum across the board,
because that gets us several features (__has_include(), asm goto and
_Generic()) which will make material improvements to our code.  And I'd
like to start using other bits of C11/gnu11.

The choice of minimum toolchains should definitely be per-arch, and
"this is the oldest one we regularly test" is an entirely fine reason to
set the minimum bar.

Furthermore, Linux has regularly been bumping minimum toolchain versions
due to code generation issues, and we'd be foolish not pay attention.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 01:10:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 01:10:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686827.1069341 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfUwE-0007ut-Jk; Thu, 29 Feb 2024 01:10:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686827.1069341; Thu, 29 Feb 2024 01:10:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfUwE-0007um-H6; Thu, 29 Feb 2024 01:10:18 +0000
Received: by outflank-mailman (input) for mailman id 686827;
 Thu, 29 Feb 2024 01:10:17 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfUwD-0007ud-Q5
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 01:10:17 +0000
Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com
 [2607:f8b0:4864:20::f2a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4b9fcd7b-d69f-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 02:10:16 +0100 (CET)
Received: by mail-qv1-xf2a.google.com with SMTP id
 6a1803df08f44-686a92a8661so1427736d6.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 17:10:15 -0800 (PST)
Received: from [192.168.1.10] (host-92-3-248-192.as13285.net. [92.3.248.192])
 by smtp.gmail.com with ESMTPSA id
 mf15-20020a0562145d8f00b0068cc0b46682sm171047qvb.4.2024.02.28.17.10.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 17:10:13 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4b9fcd7b-d69f-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709169014; x=1709773814; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=zigJQi4wIFSrs+HGgYuL+wnmWC7wTVrxj7zxCQOvnJk=;
        b=CgPXXa19FGXATcFrbDQbJ4qzeiLcK5QRs1hlGbSYWW9a0h2NmhxfNoiCfVNqDSU0Y2
         UtMNZ2+O4ChaHkaZA3qcP4dTkkp+HjbDt9uoc7WeRJficvQ0suL0sNiZjAdDGHEAzb3z
         ++CRosLv/HPxJ4rBM8bt8pPda6joMgmme9xqY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709169014; x=1709773814;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=zigJQi4wIFSrs+HGgYuL+wnmWC7wTVrxj7zxCQOvnJk=;
        b=uSGQ0r67EneZJBCA7h54D/8YtEuuw64NS2qtVL1yENFckjsf6kVEqnVEQ+g4tOK+vb
         QSzdfAmclbp1UF0dl6MPOzAfxDVctSaUc/meL6i9BHtS5wEZ2uwC3Iz/wKNAa1nZ87Nx
         cApSVQaCMKEJKqFhyoHiDUS6a3vsuXF21+TmvE64/jM6XTbutbdBOvO4/cU094xbBruz
         penFW+T/w2APlD6CCIXUsjT2CfbNEV8II861vOy9RPDm+5Vszk18NqP5GVENnidpnaAr
         FgHqHD3gdcf4DUVhQ3qVM/XNrX/7z2V7xDPcmAAwkvKaOH3ooRlMIK7MFGisDPR8UgEA
         98pg==
X-Forwarded-Encrypted: i=1; AJvYcCWHfuzMi59d7tW64gk6dDeymUNRzZoFoBioH87tOQrJNPqT5HF8kb5dv+YZfB7a2BINbEOudxXTEGE/6cOUyE30xMyMxQlPirGPoH8c/3U=
X-Gm-Message-State: AOJu0YyznLqbMzv5sllFQPoYdfTagkTRncdKHXBeio1CUWSaPiSpcl9a
	UqXsLczrrI5Ylcy672O+KZbGcCyCDR6z+bVZuAr4BOIkTcBkGkN/im6zSYJVZD0=
X-Google-Smtp-Source: AGHT+IEA5AtkoR0X4zpaCMVeZsWvIwRV0mkiA2sJfSNX+rt1LcO47BcjG5FGEflT3JCZHtNEg5gj8w==
X-Received: by 2002:a0c:da03:0:b0:68f:2c18:3b8a with SMTP id x3-20020a0cda03000000b0068f2c183b8amr749829qvj.61.1709169014634;
        Wed, 28 Feb 2024 17:10:14 -0800 (PST)
Message-ID: <3bc3a319-45a8-4b22-b225-a2b19ee04c28@citrix.com>
Date: Thu, 29 Feb 2024 01:10:11 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86: serializing of non-serializing MSR writes
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Dave Hansen <dave.hansen@intel.com>
References: <58656398-2d64-48b8-9ddc-c6836847a586@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <58656398-2d64-48b8-9ddc-c6836847a586@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28/02/2024 2:48 pm, Jan Beulich wrote:
> Linux commit 25a068b8e9a4e ("x86/apic: Add extra serialization for non-
> serializing MSRs") explains why an MFENCE+LFENCE pair is generally
> needed ahead of ICR writes in x2APIC mode, and also why at least in
> theory such is also needed ahead of TSC_DEADLINE writes. A comment of
> our own in send_IPI_mask_x2apic_phys() further explains a condition
> under which the LFENCE can be avoided.
>
> Further Linux commit 04c3024560d3 ("x86/barrier: Do not serialize MSR
> accesses on AMD") explains that this barrier isn't needed on AMD or
> Hygon, and is in fact hampering performance in a measurable way.
>
> Introduce a similarly named helper function, but with a parameter
> allowing callers to specify whether a memory access will follow, thus
> permitting the LFENCE to be omitted.
>
> Putting an instance in apic_wait_icr_idle() is to be on the safe side.
> The one case where it was clearly missing is in send_IPI_shortcut(),
> which is also used in x2APIC mode when called from send_IPI_mask().
>
> Function comment shamelessly borrowed (but adapted) from Linux.
>
> Fixes: 5500d265a2a8 ("x86/smp: use APIC ALLBUT destination shorthand when possible")
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> I question the need for a fence ahead of writing TSC_DEADLINE: The Linux
> commit message talks about LVT being accessed via MMIO in xAPIC mode,
> but that should not be relevant here: It's all the local CPU, so there
> ought to not be visibility concerns (much like for a self-IPI no fence
> is needed ahead of the ICR write). If that wasn't needed, we could
> further use alternatives patching to remove the fence also from
> apic_wait_icr_idle() when in xAPIC mode. (And only then would I agree to
> have APIC in the feature identifier, like Linux has it.)
>
> A number of apic_write() may better be turned into apic_mem_write(), in
> particular e.g. the ones in send_IPI_mask_{flat,phys}(). That way it
> would be quite a bit easier to spot paths taken only in xAPIC mode.
>
> The INIT-INIT-SIPI sequence for AP startup doesn't use any barrier, also
> not in Linux afaics. I can't explain the lack thereof, though.

I have some opinions about what Linux did here...  I don't think a
single vendor/arch-neutral helper can possibly be right.

It is vendor and uarch dependent which WRMSR's transparently degrade to
WRMSRNS, and it is vendor dependent which serialising sequence to use
(if any).  e.g. AMD have recently (Zen2 uarch I believe) retroactively
defined FS/GS_BASE to be non-serialising.  (And this is another CPUID
bit we need to OR backwards in time.)

Furthermore, IIRC AMD still owe us an update to the APM; the APM
currently says that a serialising sequence is needed for ICR.  I'm told
this isn't actually true, but I'm also very wary making an adjustment
which is directly contradicted by the docs.


The Linux change explains why in principle the IPI can be emitted before
the stores are visible.

This does actually explain TSC_DEADLINE too.  Setting a deadline in the
past gets you an interrupt immediately, and if you combine that with a
WRMSR being ordered ahead of an MFENCE, then causality is violated. 
You'll take the interrupt on whichever instruction boundary has most
recently retired, which will can be the wrong side of the WRMSR
triggering the interrupts, at which point you'll livelock taking timer
interrupts and re-arming the timer in the past.


Now, for fencing, things are more complicated.  AMD define MFENCE as
architecturally serialising.  Intel do not, hence why apparently a WRMSR
can possibly move across it.  The LFENCE is added for it's new
speculative property of dispatch serialising.

We don't actually care about architecturally serialising.  If someone is
interacting with these MSRs with relevant data in WC memory, then they
get to keep all resulting pieces.

However, we do care about plain stores, and for that we do need an
MFENCE;LFENCE on Intel  (The jury is out on whether a single
SERIALIZE[sic] would be better, but it should have the correct semantics
architecturally speaking.)

In particular, ...


>
> --- a/xen/arch/x86/apic.c
> +++ b/xen/arch/x86/apic.c
> @@ -1309,6 +1309,12 @@ int reprogram_timer(s_time_t timeout)
>  
>      if ( tdt_enabled )
>      {
> +        /*
> +         * WRMSR to TSC_DEADLINE is not serializing.  We could pass @timeout
> +         * here, but the passed value is preferably compile-time-constant.
> +         */
> +        weak_wrmsr_fence(false);
> +
>          wrmsrl(MSR_IA32_TSC_DEADLINE, timeout ? stime2tsc(timeout) : 0);
>          return 1;
>      }
> --- a/xen/arch/x86/cpu/amd.c
> +++ b/xen/arch/x86/cpu/amd.c
> @@ -675,8 +675,12 @@ void amd_log_freq(const struct cpuinfo_x
>  
>  void cf_check early_init_amd(struct cpuinfo_x86 *c)
>  {
> -	if (c == &boot_cpu_data)
> +	if (c == &boot_cpu_data) {
> +		/* No fencing needed ahead of certain MSR writes. */
> +		setup_force_cpu_cap(X86_FEATURE_NO_WRMSR_FENCE);
> +
>  		amd_init_levelling();
> +	}
>  
>  	ctxt_switch_levelling(NULL);
>  }
> --- a/xen/arch/x86/genapic/x2apic.c
> +++ b/xen/arch/x86/genapic/x2apic.c
> @@ -97,15 +97,15 @@ static void cf_check send_IPI_mask_x2api
>  
>      /*
>       * Ensure that any synchronisation data written in program order by this
> -     * CPU is seen by notified remote CPUs. The WRMSR contained within
> -     * apic_icr_write() can otherwise be executed early.
> +     * CPU is seen by notified remote CPUs. The WRMSR contained in the loop
> +     * below can otherwise be executed early.
>       * 
> -     * The reason smp_mb() is sufficient here is subtle: the register arguments
> +     * The reason MFENCE is sufficient here is subtle: the register arguments
>       * to WRMSR must depend on a memory read executed after the barrier. This
>       * is guaranteed by cpu_physical_id(), which reads from a global array (and
>       * so cannot be hoisted above the barrier even by a clever compiler).
>       */
> -    smp_mb();
> +    weak_wrmsr_fence(true);
>  
>      local_irq_save(flags);
>  
> @@ -130,7 +130,7 @@ static void cf_check send_IPI_mask_x2api
>      const cpumask_t *cluster_cpus;
>      unsigned long flags;
>  
> -    smp_mb(); /* See above for an explanation. */
> +    weak_wrmsr_fence(true); /* See above for an explanation. */
>  
>      local_irq_save(flags);
>  
> --- a/xen/arch/x86/include/asm/cpufeatures.h
> +++ b/xen/arch/x86/include/asm/cpufeatures.h
> @@ -24,7 +24,7 @@ XEN_CPUFEATURE(APERFMPERF,        X86_SY
>  XEN_CPUFEATURE(MFENCE_RDTSC,      X86_SYNTH( 9)) /* MFENCE synchronizes RDTSC */
>  XEN_CPUFEATURE(XEN_SMEP,          X86_SYNTH(10)) /* SMEP gets used by Xen itself */
>  XEN_CPUFEATURE(XEN_SMAP,          X86_SYNTH(11)) /* SMAP gets used by Xen itself */
> -/* Bit 12 unused. */
> +XEN_CPUFEATURE(NO_WRMSR_FENCE,    X86_SYNTH(12)) /* No MFENCE{,+LFENCE} ahead of certain WRMSR. */
>  XEN_CPUFEATURE(IND_THUNK_LFENCE,  X86_SYNTH(13)) /* Use IND_THUNK_LFENCE */
>  XEN_CPUFEATURE(IND_THUNK_JMP,     X86_SYNTH(14)) /* Use IND_THUNK_JMP */
>  XEN_CPUFEATURE(SC_NO_BRANCH_HARDEN, X86_SYNTH(15)) /* (Disable) Conditional branch hardening */
> --- a/xen/arch/x86/include/asm/msr.h
> +++ b/xen/arch/x86/include/asm/msr.h
> @@ -97,6 +97,25 @@ static inline void msr_split(struct cpu_
>      regs->rax = (uint32_t)val;
>  }
>  
> +/*
> + * Make previous memory operations globally visible before a WRMSR.  Most
> + * WRMSRs are full serializing instructions themselves and do not require this
> + * barrier.  This may only be required for the TSC_DEADLINE and x2APIC MSRs.
> + *
> + * MFENCE makes writes visible, but only affects load/store instructions.
> + * WRMSR is unfortunately not a load/store instruction and is unaffected by
> + * MFENCE.

[ On Intel.   AMD didn't end up with this (mis)behaviour. ]

>   The LFENCE ensures that the WRMSR is not reordered, but callers
> + * can indicate to avoid it when they have a suitable memory access between
> + * the invocation of this function and the WRMSR in question.

... this makes no sense.

We need the LFENCE for dispatch serialising properties, not it's load
ordering properties.  What use will other memory have, when the entire
problem is that WRMSR doesn't interact with them?

Worse, it's a Spectre-v1 gadget and we're now acutely familiar with how
the CPU will find its way around these.  So even expressing "I
critically need the LFENCE" still gets you pot luck on whether it has
any effect against a causality-violating WRMSR.

~Andrew

> + */
> +static inline void weak_wrmsr_fence(bool have_mem_access)
> +{
> +    alternative("mfence", "", X86_FEATURE_NO_WRMSR_FENCE);
> +
> +    if ( !have_mem_access )
> +        alternative("lfence", "", X86_FEATURE_NO_WRMSR_FENCE);
> +}
> +
>  static inline uint64_t rdtsc(void)
>  {
>      uint32_t low, high;
> --- a/xen/arch/x86/smp.c
> +++ b/xen/arch/x86/smp.c
> @@ -39,7 +39,10 @@ static unsigned int prepare_ICR2(unsigne
>  void apic_wait_icr_idle(void)
>  {
>      if ( x2apic_enabled )
> +    {
> +        weak_wrmsr_fence(false);
>          return;
> +    }
>  
>      while ( apic_read(APIC_ICR) & APIC_ICR_BUSY )
>          cpu_relax();



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 05:17:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 05:17:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686836.1069351 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfYnI-0000ef-BR; Thu, 29 Feb 2024 05:17:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686836.1069351; Thu, 29 Feb 2024 05:17:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfYnI-0000eY-8v; Thu, 29 Feb 2024 05:17:20 +0000
Received: by outflank-mailman (input) for mailman id 686836;
 Thu, 29 Feb 2024 05:17:19 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfYnH-0000eM-6h; Thu, 29 Feb 2024 05:17:19 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfYnH-0003HT-3z; Thu, 29 Feb 2024 05:17:19 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfYnG-0002LS-Mp; Thu, 29 Feb 2024 05:17:18 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfYnG-0000qH-MR; Thu, 29 Feb 2024 05:17:18 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=HYx8CrrU3onFanTC2L/Te9el4p/BDXTPPZmVr5EJE+g=; b=fuEnYsrIk2yoD99SyKM+zI8AYB
	YvhdNxxGULjyN0k+aPIKZB9ZaQcWctC2mCuwlygSlYHDclLlqWct1AV1LF4A4nv5+KQjniG3Pm1Vw
	P9E8JKNBW2I/4PmCTLQ4Ovm0in3AjUSguhcEnW551l92sshywA9uAtrV4QvMEqTDo3q8=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184802-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184802: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:xen-boot:fail:heisenbug
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=cf1182944c7cc9f1c21a8a44e0d29abe12527412
X-Osstest-Versions-That:
    linux=45ec2f5f6ed3ec3a79ba1329ad585497cdcbe663
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 29 Feb 2024 05:17:18 +0000

flight 184802 linux-linus real [real]
flight 184814 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184802/
http://logs.test-lab.xenproject.org/osstest/logs/184814/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt      8 xen-boot            fail pass in 184814-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-check fail in 184814 like 184771
 test-armhf-armhf-libvirt    15 migrate-support-check fail in 184814 never pass
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184771
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184771
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184771
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184771
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184771
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184771
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184771
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                cf1182944c7cc9f1c21a8a44e0d29abe12527412
baseline version:
 linux                45ec2f5f6ed3ec3a79ba1329ad585497cdcbe663

Last test of basis   184771  2024-02-26 19:42:20 Z    2 days
Testing same since   184802  2024-02-28 01:14:44 Z    1 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Morton <akpm@linux-foundation.org>
  Aneesh Kumar K.V (IBM) <aneesh.kumar@kernel.org>
  Byungchul Park <byungchul@sk.com>
  Jann Horn <jannh@google.com>
  Johannes Weiner <hannes@cmpxchg.org>
  Liam R. Howlett <Liam.Howlett@oracle.com>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lorenzo Stoakes <lstoakes@gmail.com>
  Marco Elver <elver@google.com>
  Mickaël Salaün <mic@digikod.net>
  Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
  Nhat Pham <nphamcs@gmail.com>
  Paul Moore <paul@paul-moore.com>
  Vlastimil Babka <vbabka@suse.cz>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   45ec2f5f6ed3e..cf1182944c7cc  cf1182944c7cc9f1c21a8a44e0d29abe12527412 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 05:28:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 05:28:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686841.1069362 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfYyB-0002Ea-EG; Thu, 29 Feb 2024 05:28:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686841.1069362; Thu, 29 Feb 2024 05:28:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfYyB-0002ET-A6; Thu, 29 Feb 2024 05:28:35 +0000
Received: by outflank-mailman (input) for mailman id 686841;
 Thu, 29 Feb 2024 05:28:34 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfYyA-0002EJ-Hh; Thu, 29 Feb 2024 05:28:34 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfYyA-0003S6-GD; Thu, 29 Feb 2024 05:28:34 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfYyA-0002dh-5N; Thu, 29 Feb 2024 05:28:34 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfYyA-0007D6-4v; Thu, 29 Feb 2024 05:28:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=iKqAC/DXn74LSOD7cOPqXJN54nxki18uHGfSRpPZT8E=; b=VklQEuakFzx6sisq56+lHrPQDG
	ZHzrVlmidyvwvsvRGsWIqSy5ppZJsieMOpHaLmG1UkWjabDLiA5TXyAICzdD1zSp/dMvDa2TMNgve
	LV420b9wJlJSUHKw2w5OoVbY3god+mmJRfk6exzrQXC/1Dr14+i5XW+e2h9aMV5AE/HQ=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184815-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184815: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=d9a6e7b0b8a67392a57788d97634256546207a64
X-Osstest-Versions-That:
    ovmf=3e91e421365027ee3e655feab33c67a4f544c777
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 29 Feb 2024 05:28:34 +0000

flight 184815 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184815/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 d9a6e7b0b8a67392a57788d97634256546207a64
baseline version:
 ovmf                 3e91e421365027ee3e655feab33c67a4f544c777

Last test of basis   184803  2024-02-28 03:46:46 Z    1 days
Testing same since   184815  2024-02-29 03:41:16 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Nickle Wang <nicklew@nvidia.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   3e91e42136..d9a6e7b0b8  d9a6e7b0b8a67392a57788d97634256546207a64 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 06:52:40 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 06:52:40 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686848.1069371 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfaHH-0003Oc-AA; Thu, 29 Feb 2024 06:52:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686848.1069371; Thu, 29 Feb 2024 06:52:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfaHH-0003OV-7H; Thu, 29 Feb 2024 06:52:23 +0000
Received: by outflank-mailman (input) for mailman id 686848;
 Thu, 29 Feb 2024 06:52:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfaHG-0003OP-MT
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 06:52:22 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 15bdb294-d6cf-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 07:52:20 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-55a5e7fa471so916307a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 22:52:20 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l10-20020a170906794a00b00a3d8a76a92esm355081ejo.175.2024.02.28.22.52.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 22:52:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 15bdb294-d6cf-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709189540; x=1709794340; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HfhBD+zsE1vHXkJemddi5dsD6z61IxsMSWlOltOFLzU=;
        b=TbIvVt4gB+UYaQF7cC3EqtwaIc0H+erM1P/Bg89OjE4DjJvef1x44sp2lHuHPnIvQR
         ckIwgCl7KNXBgE03HNpay0xWYKN1ydTXU/YMSJXTMqJIqqM2UEE11wO304289Iw/9LEg
         NoiTazXUqW4uzpvRpVE2cqrO605ujbDdaEb9WWvl871Ug7itpv2cjFLU95nrbBkPiTMI
         s0cPyHVowRj30Khyu3ulctAY4UjSBNE2IFRIvP8FRQL1j4SHtQIINO0aAtAyJovRFap4
         DNLLSokyHxymrjoK9ZLdM0EDAMLAgLoYTwWl1mLFncJkjkdBeqxPBGSZPaXxQs+6ELkN
         lHbA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709189540; x=1709794340;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HfhBD+zsE1vHXkJemddi5dsD6z61IxsMSWlOltOFLzU=;
        b=GWr1FQ9hjuimfQRFulTfBBo+OlsNyTeWMD1Ij3P9uYIECLHATcPBI4JNPrHgBY4Yhv
         Qf6Pu6nNFigpXgUvFvH8EdNUzjax8s7NyeeXEdn4X4Ykd4lPiHgy9+hOS6/oo5dlBjJd
         AVcmgqcGn2iQPc4emq20l6s2xgBxRUYiC9PoJgRrLUeSG3ZqIlUbcanOSfoj9g9RRDMX
         ePD/g4Qm6QUjaWqTZkOHZVfLWJIfKp9AqDfZbT0idw7+Rb5PS0T9hCKHnFQod96XfYif
         ZdpWP0FNqAHkGhR7iVZirzeqrTCf3XXEyFjBER6V4d2W+O/xMzKylxIMNeZQcS+WnZde
         0uUg==
X-Forwarded-Encrypted: i=1; AJvYcCXTwRZjIhwG4WKJfMYR41kZpzbZBBoiqTiy87oYy2Ie9W10dZb9YNon9gnai5/R2aorlfMrnsNqUZYNDJOLNzgG017a9bgzcGqXvdCiSGc=
X-Gm-Message-State: AOJu0YxqnKXPiLMgbm2nbNNLvWVOiz3VMN0ncT+n4Rwou2MmZrlSGH5V
	D54OBwQhbOx0cZxmYRITzSt+sXgAt0Ff61PmE5wcGFMTj6rZhx9GRm2pWaGQkg==
X-Google-Smtp-Source: AGHT+IE1pIao1LomA1koX1ZEnF7eS33jfqer77o36Ksv1cn0L/o/bhnQEG16EaiU3vm7C1U2L979qw==
X-Received: by 2002:a17:906:6b85:b0:a44:2636:2c68 with SMTP id l5-20020a1709066b8500b00a4426362c68mr817762ejr.13.1709189540277;
        Wed, 28 Feb 2024 22:52:20 -0800 (PST)
Message-ID: <4b6d34a7-38ec-4be4-bb7d-3e83137bd704@suse.com>
Date: Thu, 29 Feb 2024 07:52:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/altcall: always use a temporary parameter stashing
 variable
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240228135908.13319-1-roger.pau@citrix.com>
 <11f0e1dd-01b4-4d13-8137-e86b0975ad80@suse.com> <Zd9lvN2tUrmEt2pg@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <Zd9lvN2tUrmEt2pg@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28.02.2024 17:56, Roger Pau Monné wrote:
> On Wed, Feb 28, 2024 at 03:28:25PM +0100, Jan Beulich wrote:
>> On 28.02.2024 14:59, Roger Pau Monne wrote:
>>> The usage in ALT_CALL_ARG() on clang of:
>>>
>>> register union {
>>>     typeof(arg) e;
>>>     const unsigned long r;
>>> } ...
>>>
>>> When `arg` is the first argument to alternative_{,v}call() and
>>> const_vlapic_vcpu() is used results in clang 3.5.0 complaining with:
>>>
>>> arch/x86/hvm/vlapic.c:141:47: error: non-const static data member must be initialized out of line
>>>          alternative_call(hvm_funcs.test_pir, const_vlapic_vcpu(vlapic), vec) )
>>>
>>> Workaround this by pulling `arg1` into a local variable, like it's done for
>>> further arguments (arg2, arg3...)
>>>
>>> Originally arg1 wasn't pulled into a variable because for the a1_ register
>>> local variable the possible clobbering as a result of operators on other
>>> variables don't matter:
>>>
>>> https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html#Local-Register-Variables
>>>
>>> Note clang version 3.8.1 seems to already be fixed and don't require the
>>> workaround, but since it's harmless do it uniformly everywhere.
>>>
>>> Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> Fixes: 2ce562b2a413 ('x86/altcall: use a union as register type for function parameters on clang')
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>
>> I'm okay with this change, but since you don't mention anything in this
>> regard: Did you look at whether / how generated code (with gcc) changes?
> 
> So the specific example of vlapic_test_irq() shows no changes to the
> generated code. bloat-o-meter shows a 0 delta:
> 
> add/remove: 0/0 grow/shrink: 0/0 up/down: 0/0 (0)
> Function                                     old     new   delta
> Total: Before=3570098, After=3570098, chg +0.00%
> 
> I assume there can be some reordering of instructions, as we now force
> arg1 temporary variable (v1_) to be initialized ahead of the rest of
> the arguments.

Yeah, re-ordering is okay. I was more worried of new unnecessary register
copying, for example. Thanks for confirming.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 07:01:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 07:01:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686852.1069382 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfaQI-00058r-CL; Thu, 29 Feb 2024 07:01:42 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686852.1069382; Thu, 29 Feb 2024 07:01:42 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfaQI-00058k-95; Thu, 29 Feb 2024 07:01:42 +0000
Received: by outflank-mailman (input) for mailman id 686852;
 Thu, 29 Feb 2024 07:01:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=MycN=KG=rabbit.lu=slack@srs-se1.protection.inumbo.net>)
 id 1rfaQH-00058c-CO
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 07:01:41 +0000
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com
 [2a00:1450:4864:20::32c])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 62005e69-d6d0-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 08:01:38 +0100 (CET)
Received: by mail-wm1-x32c.google.com with SMTP id
 5b1f17b1804b1-412b5a3d2e3so3764265e9.0
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 23:01:38 -0800 (PST)
Received: from localhost.localdomain (82-64-138-184.subs.proxad.net.
 [82.64.138.184]) by smtp.googlemail.com with ESMTPSA id
 q11-20020a05600c46cb00b00412b4dca795sm1112700wmo.7.2024.02.28.23.01.35
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 28 Feb 2024 23:01:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 62005e69-d6d0-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=rabbit-lu.20230601.gappssmtp.com; s=20230601; t=1709190098; x=1709794898; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=N+XvJ1Fl3pHzHwaInzg8gm5Lb2VaZTApN77ZDCzckBU=;
        b=fIrPaXIHGkz4qgZ1MptkHkIhiYDZoam308A+FgpRjVvKB4fw2MPq+M4ezaneZ9Jgym
         baJ1VWx8PFU5/35KHf6rEHL7p1VVRvtmxkuHNGlc8d+04PQAipGZ/BKTe8P53qYSBRKH
         no7uLgRC0iOlGE6fnWIBqrs38rT5JiwDhEyL2ZPV3UjVXoYm1GsWwKfcA3szls3rOg31
         nbfFQpENhM+Y4Hr7AjP+R+9Vya0YoOdOd5P3q76YWyn+vPvT2X8W73FfdmFfdRdMn9st
         EzAGsh2wJVPVfUs6ZliusB7M+w/IF452zcjmrS11zokVEbXvaIg54z8IXhKpgctWv0pV
         v/CQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709190098; x=1709794898;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=N+XvJ1Fl3pHzHwaInzg8gm5Lb2VaZTApN77ZDCzckBU=;
        b=axqsM8s2fzHYecAk0O5AUMnBsnDAiFkUINNpk0YS7X2NZJe8XfYEdMItHzSp4PAPHF
         uFJ96eZWs8cxIZTy5DrD43mPGBqlYQN8WKeUKZFBIPx2AQj3rEndEmZQRGXtPBqCpQZc
         TVv71aM0alzGFCZXLa7RqCkyke0klLHQACscTt6qh+bzsPIEllUEPPKQAzgoauriAaTV
         +isq6YMESHFTDp2swT/pb9Cx8oT1nChPOR7jivNBBjCUg2KY0+ycf8SJZ2aZEF/FkkkI
         uoNRHlTpN0uqhRs8eIy1p+TwSk3Cuo6YgcdO2sCZp4xXl2DEQwWbafdQy4eAwPuMzzub
         8oAA==
X-Gm-Message-State: AOJu0YwRfGWxD7seDLy8lLT2pBd0K/mMesmp/pjfhYo1dck1TK+rm0K7
	WtXRYifRg1fm/f3exjXCLUObcqHQVkoacjcfNtGbMOYZX85ADHfcQHdoAyJta3aIQDIT/0pi4LZ
	62htbVw==
X-Google-Smtp-Source: AGHT+IFZfT6xfkRkUXf4s8EmWrprK9vrXBueCfy1deQZH/oAhpSdos9Ld1d/h9tG1HGqp/f1oPyspw==
X-Received: by 2002:a05:600c:1c02:b0:412:65a5:c9f4 with SMTP id j2-20020a05600c1c0200b0041265a5c9f4mr1102154wms.28.1709190097579;
        Wed, 28 Feb 2024 23:01:37 -0800 (PST)
From: =?UTF-8?q?zithro=20/=20Cyril=20R=C3=A9bert?= <slack@rabbit.lu>
To: xen-devel@lists.xenproject.org
Cc: =?UTF-8?q?zithro=20/=20Cyril=20R=C3=A9bert?= <slack@rabbit.lu>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	=?UTF-8?q?Marek=20Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH] tools/xl: add suspend-to-ram and resume subcommands
Date: Thu, 29 Feb 2024 08:00:10 +0100
Message-Id: <4e9f3590be9f19edb87e05c7b7c2efeae8863109.1709190010.git.slack@rabbit.lu>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The xl command doesn't provide suspend/resume, so add them :
  xl suspend-to-ram <Domain>
  xl resume <Domain>

This patch follows a discussion on XenDevel: when you want the
virtualized equivalent of "sleep"-ing a host, it's better to
suspend/resume than to pause/unpause a domain.

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Suggested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

---
- Tested on v4.17, x86
- the function "libxl_domain_resume" is called like libvirt does, so
  using a "co-operative resume", but I don't know what it means (:
- there may be a problem with the words resume <-> restore, like
  for "LIBXL_HAVE_NO_SUSPEND_RESUME"
- for the docs, I only slightly adapted a copy/paste from xl pause ...
---
---
 docs/man/xl.1.pod.in    | 10 ++++++++++
 tools/xl/xl.h           |  2 ++
 tools/xl/xl_cmdtable.c  | 10 ++++++++++
 tools/xl/xl_vmcontrol.c | 36 ++++++++++++++++++++++++++++++++++++
 4 files changed, 58 insertions(+)

diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in
index bed8393473..63d80f1449 100644
--- a/docs/man/xl.1.pod.in
+++ b/docs/man/xl.1.pod.in
@@ -682,6 +682,10 @@ Pass the VNC password to vncviewer via stdin.
 
 =back
 
+=item B<resume> I<domain-id>
+
+Resume a domain, after having been suspended to RAM.
+
 =item B<save> [I<OPTIONS>] I<domain-id> I<checkpointfile> [I<configfile>]
 
 Saves a running domain to a state file so that it can be restored
@@ -760,6 +764,12 @@ in response to this event.
 
 =back
 
+=item B<suspend-to-ram> I<domain-id>
+
+Suspend a domain to RAM.  When in a suspended state the domain will still
+consume allocated resources (such as memory), but will not be eligible for
+scheduling by the Xen hypervisor. It is in a shutdown state, but not dying.
+
 =item B<sysrq> I<domain-id> I<letter>
 
 Send a <Magic System Request> to the domain, each type of request is
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index 9c86bb1d98..716ad32423 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -133,6 +133,8 @@ int main_migrate(int argc, char **argv);
 int main_dump_core(int argc, char **argv);
 int main_pause(int argc, char **argv);
 int main_unpause(int argc, char **argv);
+int main_suspendtoram(int argc, char **argv);
+int main_resume(int argc, char **argv);
 int main_destroy(int argc, char **argv);
 int main_shutdown(int argc, char **argv);
 int main_reboot(int argc, char **argv);
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 62bdb2aeaa..1382282252 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -136,6 +136,16 @@ const struct cmd_spec cmd_table[] = {
       "Unpause a paused domain",
       "<Domain>",
     },
+    { "suspend-to-ram",
+      &main_suspendtoram, 0, 1,
+      "Suspend a domain to RAM",
+      "<Domain>",
+    },
+    { "resume",
+      &main_resume, 0, 1,
+      "Resume a domain from RAM",
+      "<Domain>",
+    },
     { "console",
       &main_console, 0, 0,
       "Attach to domain's console",
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index 98f6bd2e76..ba45f89c5a 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -42,6 +42,16 @@ static void unpause_domain(uint32_t domid)
     libxl_domain_unpause(ctx, domid, NULL);
 }
 
+static void suspend_domain_toram(uint32_t domid)
+{
+    libxl_domain_suspend_only(ctx, domid, NULL);
+}
+
+static void resume_domain(uint32_t domid)
+{
+    libxl_domain_resume(ctx, domid, 1, NULL);
+}
+
 static void destroy_domain(uint32_t domid, int force)
 {
     int rc;
@@ -82,6 +92,32 @@ int main_unpause(int argc, char **argv)
     return EXIT_SUCCESS;
 }
 
+int main_suspendtoram(int argc, char **argv)
+{
+    int opt;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "suspend-to-ram", 1) {
+        /* No options */
+    }
+
+    suspend_domain_toram(find_domain(argv[optind]));
+
+    return EXIT_SUCCESS;
+}
+
+int main_resume(int argc, char **argv)
+{
+    int opt;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "resume", 1) {
+        /* No options */
+    }
+
+    resume_domain(find_domain(argv[optind]));
+
+    return EXIT_SUCCESS;
+}
+
 int main_destroy(int argc, char **argv)
 {
     int opt;
-- 
2.39.2



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 07:03:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 07:03:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686854.1069391 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfaRs-0005g1-M0; Thu, 29 Feb 2024 07:03:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686854.1069391; Thu, 29 Feb 2024 07:03:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfaRs-0005fu-JP; Thu, 29 Feb 2024 07:03:20 +0000
Received: by outflank-mailman (input) for mailman id 686854;
 Thu, 29 Feb 2024 07:03:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfaRq-0005fm-VG
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 07:03:18 +0000
Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com
 [2a00:1450:4864:20::535])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9d45939a-d6d0-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 08:03:17 +0100 (CET)
Received: by mail-ed1-x535.google.com with SMTP id
 4fb4d7f45d1cf-565d1656c12so963416a12.1
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 23:03:17 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s12-20020a056402164c00b005667516866dsm321384edx.33.2024.02.28.23.03.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 23:03:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d45939a-d6d0-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709190197; x=1709794997; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=UcIUT9y/WAluqHtUy+fvfzc42TETOeVyLkUfy04xi6Q=;
        b=UPzAimFsgzhW1jbhrO2U+9id/yP6u8RSfDPB7t6NtGuZBi+W39KDzXXwMZ3eNTbiD2
         G/7KDV2JEuyQFbbaXSCSj5MnlXiT2CdeJY8J6OzIaMYsDiVg881J4YIQWxMK4+gSLqr3
         BJYasbnz0FAHsyt/2vxXbPqWrp1HHBKHHssIktpRWBGHAK6qGv4nW4C8ejZJOTbZD8nb
         Q7EiZsC+tY4oBok+vKuPjnUlU0b1FVLnjHhEZNY7gFckjC3qAM5uZ2S8I6DjidA5HPLV
         D+nbzvTOugV+WpqEahKyWJBL6JGOS8hvf9H7NqbXWfSJpXoztHp7tssmmtNJ/QaYZwCx
         roKA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709190197; x=1709794997;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=UcIUT9y/WAluqHtUy+fvfzc42TETOeVyLkUfy04xi6Q=;
        b=D+9b3XfN+TIE//ln1RE7YY6zldHGjSUDe8T/DMxtPnmSGS1ufFWJPXuWhKRM0YEUsf
         QhmGETWyVYJ4OZtn2pjdLHZ+9C2OJs9J6jR51TDlkZUWUykvgfR6WiyfSitDBqVZwmZI
         myba9Tl8SFpaLCVWLe+2y75htitqHGXKvuAiFWbzhJo4PQh1cXIk8hYnFgVMnEitBwuC
         oZxEuFtOVgAoss5yWsMeACf8OJo/ckobLkirZuewsOg6j4tugHnOLy8td8y5ysBdmMDl
         UA53+8OH+X9nsj2yVwMWqRrE54ysfhYjevjofnG91QWYiJhVaEDFCBezZimUy0MaaZ4R
         wHXQ==
X-Forwarded-Encrypted: i=1; AJvYcCVXyt59EKMvnrTUZpCP7Wzn4xTuF6H+e5CEEOyOY1Pu/0psb9w5MyjoeQQhqrwjduzk2ytDiCdhghAKvRjjJ/G9blUdJn+Zzz+ZK8Q2FJw=
X-Gm-Message-State: AOJu0YxK8kINaPIhuiJysFEqmkbop2MpLhd9RasPM9/8AoUzrORigNf7
	ztHP5RcUOY7433+nHBb+h7KcE4m95rJTPCjjHmEkiQowXnI+9Ygt6OgF2ANKyA==
X-Google-Smtp-Source: AGHT+IFsBMNtwxkE5x7L8PMc+f5Fod468HKv26dQEXHBijGuPM8Pm2OTsvBTIglOvJbntJEQrdHiMw==
X-Received: by 2002:a50:cc08:0:b0:566:13f8:bcc with SMTP id m8-20020a50cc08000000b0056613f80bccmr761373edi.0.1709190197024;
        Wed, 28 Feb 2024 23:03:17 -0800 (PST)
Message-ID: <00881c6e-22a5-4283-b071-a9b92fbaf89a@suse.com>
Date: Thu, 29 Feb 2024 08:03:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: preparations for 4.18.1
Content-Language: en-US
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Wei Liu <wl@xen.org>, Anthony Perard <anthony.perard@citrix.com>,
 Kelly Choi <kelly.choi@cloud.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
 Julien Grall <julien@xen.org>
References: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com>
 <1f251f2e-91de-4f81-a93c-dcb95746d2e5@xen.org>
 <5a4e8583-7747-4f24-94f0-d040dabb5b04@suse.com>
 <521a4d3b-8ef1-4449-97bd-1a3d3e0e35ec@xen.org>
 <alpine.DEB.2.22.394.2402281445450.575685@ubuntu-linux-20-04-desktop>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <alpine.DEB.2.22.394.2402281445450.575685@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 28.02.2024 23:45, Stefano Stabellini wrote:
> On Wed, 28 Feb 2024, Julien Grall wrote:
>> On 28/02/2024 12:58, Jan Beulich wrote:
>>> On 28.02.2024 12:50, Julien Grall wrote:
>>>> On 27/02/2024 13:19, Jan Beulich wrote:
>>>>> the release is due in two to three weeks. Please point out backports you
>>>>> find
>>>>> missing from the respective staging branch, but which you consider
>>>>> relevant.
>>>>
>>>> For Arm:
>>>>
>>>> e11f576650 ("xen/arm: Fix UBSAN failure in start_xen()")
>>>
>>> Which I assume you or Stefano will take care of?
>>
>> I was expecting Stefano would do it as he did the backports in the past.

Was it deliberate that you did not also put it on the 4.17 branch?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 07:38:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 07:38:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686866.1069426 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfazP-0001ll-JZ; Thu, 29 Feb 2024 07:37:59 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686866.1069426; Thu, 29 Feb 2024 07:37:59 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfazP-0001le-Ez; Thu, 29 Feb 2024 07:37:59 +0000
Received: by outflank-mailman (input) for mailman id 686866;
 Thu, 29 Feb 2024 07:37:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfazN-0001lW-Lk
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 07:37:57 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 73b99fe3-d6d5-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 08:37:55 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a44360a8b9dso56336066b.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 23:37:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 lb24-20020a170906add800b00a4131367204sm391444ejb.80.2024.02.28.23.37.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 23:37:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 73b99fe3-d6d5-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709192275; x=1709797075; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=TxUzgkmubUb3J27gLbgz86cIAhX78Zc6oPVUOiI1wZg=;
        b=ed4JsGKOV7Ra49DqgXUQcwY4s+5Pkvyvr9WDp6Uo7I88mP+au/R5qIPFE2ViQvMrhV
         yw1PVmek9X5VgtLYjrjaIcox8aVCh5mbLYkMCnU3z1qm2t6NSKH9X7ht1/7GCAcNdAxf
         JLIWFeVoCR2nZgK/yOjPhDpVq2HHzDnzJZJ8BD/VdV5vtsjBTQ1qmWzrln8F5orMXvgA
         l8OFlIfDtS5CfoD8igDYctKJVXd2mc5btjzEk4gEqkDRh8cfx6c+xefcArdgQqLWLbiP
         IujqA6eiE/r7E5eRU/jGtdfFpVMD4z6rbSOpGvQdNtJAS/r5V+QcH6XUPK/jpZtbGD9H
         XZ0A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709192275; x=1709797075;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=TxUzgkmubUb3J27gLbgz86cIAhX78Zc6oPVUOiI1wZg=;
        b=hyUKn2w1+1ZDuXlbIr+St4XxUdyN17hDcUr9spIsD1VzLiD+lLX5w6ucK4aCI8CR64
         H+e7xm7cT1vyZmmrGHblSOHo96NmrIkJ8G7tFPbr+4/xAOZ4Do1JocTfeDkXuGX9Ah78
         RA+5OhXr33tTnSXsVGEdzrr6YvxK7UqVwi8SbPgGChxcDjIoQEM5UGCn4hWI0gihLtBt
         Kxrcr6iuBlfsJF2rocIoilHTg1RpmikhcxiUf1emFjj1PfXIgtaV3TJB68Juuw19ts/O
         jVPowV281yU0a6bB/OsTdELAwRQ5HoCpH4bs77ahxpaj+KzeeKZl1DYk1zzyKtYoOY9I
         qzTw==
X-Forwarded-Encrypted: i=1; AJvYcCXX7Org4Q1TMTWeWDJ7sy8bo+scNMbHtDyJvmMh8sszKNzFE7/wkAaAUHJElbQhtTZ7LzKHhrC8Z27Sm4aMUpABx/NW8xbwL/lQ7w5SX+8=
X-Gm-Message-State: AOJu0YzrSbQlTwyq0tWq3oSjno1jTw87zmvBBnh/9wGJJtKfmFqqSmWn
	BpgfBp0HVk2JhJ2liGcdURESlNDCgDv+YnVDxwzVlbfh6XMs36BooopEmKdouw==
X-Google-Smtp-Source: AGHT+IHe39N40kSTHn0ZakWxc6IYudPegU4lQ7t1DTRAqUGWBh43lajHEKJUCVHQIbYIQR7Lw/qPLA==
X-Received: by 2002:a17:906:fc1e:b0:a43:d6a9:ced8 with SMTP id ov30-20020a170906fc1e00b00a43d6a9ced8mr855517ejb.56.1709192274861;
        Wed, 28 Feb 2024 23:37:54 -0800 (PST)
Message-ID: <19c837ce-534c-4b76-b8fd-f5baa9f328bd@suse.com>
Date: Thu, 29 Feb 2024 08:37:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] x86/memsharing: use an atomic add instead of a
 cmpxchg loop
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Tamas K Lengyel <tamas@tklengyel.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240222090530.62530-1-roger.pau@citrix.com>
 <20240222090530.62530-2-roger.pau@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240222090530.62530-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 22.02.2024 10:05, Roger Pau Monne wrote:
> The usage of a cmpxchg loop in get_next_handle() is unnecessary, as the same
> can be achieved with an atomic increment, which is both simpler to read, and
> avoid any need for a loop.
> 
> The cmpxchg usage is likely a remnant of 32bit support, which didn't have an
> instruction to do an atomic 64bit add, and instead a cmpxchg had to be used.
> 
> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-of-by: Roger Pau Monné <roger.pau@citrix.com>

Hmm, same typo again, so my commit script again got the author wrong.
I'm sorry again. I would consider adding an explicit confirmation step
when there's no (well-formed) S-o-b, but I use this script elsewhere
as well, and in e.g. binutils and gcc S-o-b are pretty uncommon. (And
to avoid the question: I'm specifically avoiding From: itself in the
common case, as for you and other people with non-ASCII characters in
their names those typically don't properly show up there.)

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 07:58:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 07:58:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686869.1069436 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfbJM-0004p1-4U; Thu, 29 Feb 2024 07:58:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686869.1069436; Thu, 29 Feb 2024 07:58:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfbJM-0004ou-1H; Thu, 29 Feb 2024 07:58:36 +0000
Received: by outflank-mailman (input) for mailman id 686869;
 Thu, 29 Feb 2024 07:58:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfbJL-0004oo-GS
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 07:58:35 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 55446d58-d6d8-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 08:58:32 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-563d56ee65cso901372a12.2
 for <xen-devel@lists.xenproject.org>; Wed, 28 Feb 2024 23:58:32 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s25-20020a05640217d900b005661a50b7c5sm363398edy.13.2024.02.28.23.58.31
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 28 Feb 2024 23:58:32 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 55446d58-d6d8-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709193512; x=1709798312; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Z0ZPYVhutgapvoG0oXtXKA7FLhQPQCscjuBl1qW0rxw=;
        b=ciQP1TL4UTM9a0e38bKhxRMoBnSGzD8G/lXjWRj1XSwLrw+uiklAB5Xt+l0Z75PGeZ
         sLgi0RCLNJyTLHAtO4GH+zpiv0ygs2vxuPiWz7vIE0GeVJviMFbPdxx+m50ZvhBEQl5Q
         pYyN5eBwMg3ZXIOt43ypeKBM3O/9P1sMSiRNTWYKgQYgpVhnhM9nkofLjtZb+xINNGAm
         Nx7umTw2bCl5R6vCnjYso/csEOsCGv96fte/xHlnzZ7zMR1IwWKHN74EFaojHcCBsxbJ
         T785ajSh2znloviP9OWrmb4TOixRUxuigVxOALzrS0k4d0+lcSVgVXEQkBDp/Q1kXWy7
         i1Gg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709193512; x=1709798312;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Z0ZPYVhutgapvoG0oXtXKA7FLhQPQCscjuBl1qW0rxw=;
        b=PPR2HNV8CcFg5m8B4kQpGcsbzdtjqJMiPObPfDQkuiLbFszkTkXirikagewE242FEe
         19ZTFb1DkxvIkfOzmntwNmSIJExcXZhDghydggUqOKdCpdlb+IJi+SHTwOshnxKD8x2G
         2bVHUa/x/emWYILuaQz0JbQg6scE+PbzV9DkEjBCG9KQXMgiAx6d1Ausoal5nFxpepQI
         LVhQa5MarOLya6M1R3V80+1+ok1qwk80y5JllbKTgtYdvDjQSmKKIwNzSE2fbilka+8e
         7r/AWFEyFqmWhbQLbAk+MULMURa30bcM+j+OoD9u1PPmb3UwrGd+U1Xx+fl861vWilfu
         UxBA==
X-Forwarded-Encrypted: i=1; AJvYcCX98K6fDq9IwEE4YmuS2BAnKJ9ueXskqI9yjG4yA5oa6If5t6eG5eqSyw2OaeEBAkBI75tzruXua0BHyHVZd1xqFgF+A0iBtZ2cvT3vKbA=
X-Gm-Message-State: AOJu0YwtcwEYKO2ng+yOlTSzCvLvB4uiPEFl0dvFPR9kZc3qgDqSCyzh
	xPBpQhCwIr2/eRDqW1myHoi1M0wvMh9vnTpGUbxNoI7qrUqKF0pobJ3EnZCXyg==
X-Google-Smtp-Source: AGHT+IHKjtqdCUKNSMQ7XwVBupCzypDcAZ9ITc9ihPdzjObPek+h8iVmRwmanCt6ZCtHZl1rAvfjwg==
X-Received: by 2002:a05:6402:520c:b0:565:6911:a82b with SMTP id s12-20020a056402520c00b005656911a82bmr881174edd.40.1709193512312;
        Wed, 28 Feb 2024 23:58:32 -0800 (PST)
Message-ID: <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
Date: Thu, 29 Feb 2024 08:58:30 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 28.02.2024 23:58, Julien Grall wrote:
> On 27/02/2024 07:55, Jan Beulich wrote:
>> On 26.02.2024 18:39, Oleksii Kurochko wrote:
>>> This patch doesn't represent a strict lower bound for GCC and
>>> GNU Binutils; rather, these versions are specifically employed by
>>> the Xen RISC-V container and are anticipated to undergo continuous
>>> testing.
>>
>> Up and until that container would be updated to a newer gcc. I'm
>> afraid I view this as too weak a criteria,
> 
> I disagree. We have to decide a limit at some point. It is sensible to 
> say that we are only supporting what we can tests. AFAIK, this is what 
> QEMU has been doing.

I view qemu as a particularly bad example. They raise their baselines
far too aggressively for my taste.

>> IOW it is hard for me to see why RISC-V needs stronger restrictions here
>> than other architectures. It ought to be possible to determine a baseline
>> version. Even if taking the desire to have "pause" available as a
>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
> 
> I think we want to bump it on Arm. There are zero reasons to try to keep 
> a lower versions if nobody tests/use it in production.
> 
> I would suggest to do the same on x86. What's the point of try to 
> support Xen with a 15+ years old compiler?

It could have long been bumped if only a proper scheme to follow for
this and future bumping would have been put forward by anyone keen on
such bumping, like - see his reply - e.g. Andrew. You may recall that
this was discussed more than once on meetings, with no real outcome.
I'm personally not meaning to stand in the way of such bumping as long
as it's done in a predictable manner, but I'm not keen on doing so and
hence I don't view it as my obligation to try to invent a reasonable
scheme. (My personal view is that basic functionality should be
possible to have virtually everywhere, whereas for advanced stuff it
is fine to require a more modern tool chain.)

The one additional concern I've raised in the past is that in the end
it's not just minimal tool chain versions we rely on, but also other
core system tools (see the recent move from "which" to "command -v"
for an example of such a dependency, where luckily it turned out to
not be an issue that the -v had only become a standard thing at some
point). While for the tool chain I can arrange for making newer
versions available, for core system tools I can't. Therefore being too
eager there would mean I can't really / easily (smoke) test Xen
anymore on ancient hardware every once in a while. When afaict we do
too little of such testing already anyway, despite not having any
lower bound on hardware that formally we support running Xen on. (And
no, upgrading the ancient distros on that ancient hardware is not an
option for me.)

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 08:01:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 08:01:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686876.1069445 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfbME-00076N-09; Thu, 29 Feb 2024 08:01:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686876.1069445; Thu, 29 Feb 2024 08:01:33 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfbMD-00076G-Tw; Thu, 29 Feb 2024 08:01:33 +0000
Received: by outflank-mailman (input) for mailman id 686876;
 Thu, 29 Feb 2024 08:01:32 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=wRyR=KG=bugseng.com=federico.serafini@srs-se1.protection.inumbo.net>)
 id 1rfbMC-00075c-6A
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 08:01:32 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id beedb57f-d6d8-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 09:01:30 +0100 (CET)
Received: from [192.168.1.229] (93-36-216-194.ip62.fastwebnet.it
 [93.36.216.194])
 by support.bugseng.com (Postfix) with ESMTPSA id 57B194EE0737;
 Thu, 29 Feb 2024 09:01:29 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: beedb57f-d6d8-11ee-a1ee-f123f15fe8a2
Message-ID: <98d82c40-a81a-4c75-8cdf-aaef640382b6@bugseng.com>
Date: Thu, 29 Feb 2024 09:01:28 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair: extend deviations of MISRA C:2012
 Rule 16.3
Content-Language: en-US, it
To: Jan Beulich <jbeulich@suse.com>
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <6c96f889ae3fbdf516f0edfe4a5fb207540b6862.1709109662.git.federico.serafini@bugseng.com>
 <1a00c221-ce1c-45be-9ab3-e8f1a0e5678d@suse.com>
From: Federico Serafini <federico.serafini@bugseng.com>
Organization: BUGSENG srl
In-Reply-To: <1a00c221-ce1c-45be-9ab3-e8f1a0e5678d@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 28/02/24 10:06, Jan Beulich wrote:
> On 28.02.2024 09:53, Federico Serafini wrote:
>> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
>> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> 
> Comments below apply similarly to text added to this file.
> 
>> --- a/docs/misra/deviations.rst
>> +++ b/docs/misra/deviations.rst
>> @@ -291,7 +291,14 @@ Deviations related to MISRA C:2012 Rules:
>>        - Project-wide deviation; tagged as `deliberate` for ECLAIR.
>>   
>>      * - R16.3
>> -     - Switch clauses ending with continue, goto, return statements are safe.
>> +     - Switch clauses ending with an unconditional flow control statement
>> +       (i.e., continue, goto, or return) are safe.
>> +     - Tagged as `safe` for ECLAIR.
> 
> With this edit (unmentioned in the description, btw) ...
> 
>> +   * - R16.3
>> +     - Switch clauses ending with an if-else statement are safe if both
>> +       branches consist of a flow control statement (i.e., continue, break,
>> +       goto, return).
> 
> ... why is it not also "ending with" here?

Because the allowed pattern is:

if ( cond )
  return; /* Or continue / break / goto */
else
  break;  /* Or continue / goto / return */

See below for more information.

> 
> Also what about either situation ending with a call to a noreturn function?

This can be added.

> 
>> @@ -307,6 +314,16 @@ Deviations related to MISRA C:2012 Rules:
>>        - Switch clauses ending with failure method \"BUG()\" are safe.
>>        - Tagged as `safe` for ECLAIR.
>>   
>> +   * - R16.3
>> +     - On X86, switch clauses ending generating an exception through
>> +       \"generate_exception()\" are safe.
>> +     - Tagged as `safe` for ECLAIR.
> 
> This macro is limited to the emulator, so shouldn't be deviated globally.

Noted.

> Furthermore - why does the special case need mentioning here? Shouldn't
> it be the underlying pattern which is deviated (along the lines of the
> earlier ones):
> 
>      if ( true )
>      {
>          ...
>          goto ...; /* Or break / continue / return */
>      }

This pattern that involves a compound statement for the true branch
is not deviated by this configuration.

See below for more information.

> 
>> +   * - R16.3
>> +     - Switch clauses ending generating a parse error through
>> +       \"PARSE_ERR_RET()\" are safe.
>> +     - Tagged as `safe` for ECLAIR.
> 
> Again this isn't a global scope macro, so shouldn't be deviated globally.

Noted.

> Plus it ends in "return", so ought to be covered by the earlier clause.
> The fact that the return is in a body of do {} while(0) shouldn't matter
> at all - that's purely syntactic sugar.

I gather from your comments/questions that you would like to deviate
*all* the patterns where an unintentional fall through can not happen.

Rule 16.3 is a purely syntactic rule, and, as a consequence,
in the current version of ECLAIR additional "allowed pattern" (aka
deviations) for that rule need to be described through AST nodes,
meaning that all what you consider as syntactic sugar cannot be ignored.

A deviation that covers all the pattern you are asking for could be
done, but it will result in a complex and quite long expression
(not easy to read and justify in front of an assessor).

Hence, what I am proposing is to deviate only the the simplest and
most readable cases, such as:

if ( cond )
   return x;
else
   return y;

without involving compound statements, fake do-wile and fake if
statements but rather deviating the macro inside of which are used
(as I did).

-- 
Federico Serafini, M.Sc.

Software Engineer, BUGSENG (http://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 08:28:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 08:28:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686882.1069455 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfbmS-0001ss-3o; Thu, 29 Feb 2024 08:28:40 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686882.1069455; Thu, 29 Feb 2024 08:28:40 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfbmS-0001sl-1G; Thu, 29 Feb 2024 08:28:40 +0000
Received: by outflank-mailman (input) for mailman id 686882;
 Thu, 29 Feb 2024 08:28:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfbmQ-0001sf-37
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 08:28:38 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 876c283e-d6dc-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 09:28:35 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a3e552eff09so80037566b.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 00:28:34 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ch14-20020a170906c2ce00b00a42ea946917sm430382ejb.130.2024.02.29.00.28.33
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 00:28:34 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 876c283e-d6dc-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709195314; x=1709800114; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=HHze7WSwz4Ru+J/UVMxo4r1+7P5s2XyBorYTfUVw/V8=;
        b=CX00drdmtr6Zq1E48ZBwlh6nI4wJTDoO29ltkPTzZIMZb/n7urLqpISosulUpel0Xx
         EZKyBj+mCYHGKQ3wJTyOL9K8ahtXrflLIhgGCLATPuzApByUSuzSH9goJ4M/R2xOZWJV
         yVp8Q0Lp6HHfybbo1/CoMtXySFJulmcveR1eLOdjSwPeak341mHxYnb4YhKkckiNcWjP
         ZrzbgFSFAdhAakfuVmLjd9HDvziwOanyEaNQamNoq8AN1gE30R3Lpages3imDjTl93t8
         VP0EKOrZUkZiNuCTHaTI4QNaSzIY/3gjmLWm8DQn3U0eI6wTtqcBn1pn72cy2ASokkIN
         CR9A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709195314; x=1709800114;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=HHze7WSwz4Ru+J/UVMxo4r1+7P5s2XyBorYTfUVw/V8=;
        b=UdZFmIMJpBYRcjbyEI2H8xLLnsrA7bXFtWhSdOoHI5bjzg66FUrETiWMjZulSWAXfL
         qAQpuhClzHVbGCryezaIF/dvHxfTEbmwC0VxrY3R49yedBTY23fo1MhNjfi7tFOKrSs+
         olvs50q0OUexd9UO9Xd2JqCEJ5rc+vmrAwoNq8ZYiRWNRKEuXHqU+IyHJ9ugvY3aNpvk
         HDwaIj4pBfAhuknf/hlarYMH98W+OSstTEA+4iDk6+jh6S2CgqHCl6D0T8aaKZHUvk79
         Z4lp3anPGMkLBSAJkf32qsgbrSoOwdOcHr1yD7wugmePA0E0jTbV+gfXHt15eWnWPyJJ
         dM3w==
X-Forwarded-Encrypted: i=1; AJvYcCUg+naYDFHi2XFEB+uAxERk4NtWgz22kdyhZbl+fS3jWpUgo76vi2T8vc9xOBE5oPLKGzD5MdOICkm8ohBbEXvwrpYnLT2wQ4Hz5HpIGDc=
X-Gm-Message-State: AOJu0YyA9TpzVr0wdAFJyQXWu6ZU/pivY+4zFMUGN98gHDud516qlVyX
	8kykvf75h6Bmm7mjpwhrOq7GZY0zRMTbUDO/wDJyWGKVrqWiBE1kCo/o0lXrOw==
X-Google-Smtp-Source: AGHT+IGDxMA8iN85HxphBrIKIO8VuFZSBfVVSQj8jsBlTz+YBLJf5FnkpiPXdVXTd9E/mGcGt22IAg==
X-Received: by 2002:a17:906:a88:b0:a3e:473d:e391 with SMTP id y8-20020a1709060a8800b00a3e473de391mr906719ejf.22.1709195314303;
        Thu, 29 Feb 2024 00:28:34 -0800 (PST)
Message-ID: <054d2605-c626-4ea2-b0e4-c0e50190e8e0@suse.com>
Date: Thu, 29 Feb 2024 09:28:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 4/4] x86: prefer shadow stack for producing call traces
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
 <fdacbb8b-5a58-4d6b-89b2-ea98b4fed895@suse.com>
 <e687d4a1-efd8-4d7f-9c5d-8796bb99014a@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e687d4a1-efd8-4d7f-9c5d-8796bb99014a@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28.02.2024 17:15, Andrew Cooper wrote:
> On 28/02/2024 1:53 pm, Jan Beulich wrote:
>> --- a/xen/arch/x86/traps.c
>> +++ b/xen/arch/x86/traps.c
>> @@ -539,6 +544,50 @@ static void show_trace(const struct cpu_
>>           !is_active_kernel_text(tos) )
>>          printk("   [<%p>] R %pS\n", _p(regs->rip), _p(regs->rip));
>>  
>> +    if ( IS_ENABLED(CONFIG_XEN_SHSTK) && rdssp() != SSP_NO_SHSTK )
>> +    {
>> +        const unsigned long *ptr = _p(regs->entry_ssp);
>> +        unsigned int n;
>> +
>> +        for ( n = 0; (unsigned long)ptr & (PAGE_SIZE - sizeof(*ptr)); ++n )
>> +        {
>> +            unsigned long val = *ptr;
>> +
>> +            if ( is_active_kernel_text(val) || in_stub(val) )
>> +            {
>> +                /* Normal return address entry.  */
>> +                printk("   [<%p>] C %pS\n", _p(val), _p(val));
>> +                ++ptr;
>> +            }
>> +            else if ( !((val ^ *ptr) >> (PAGE_SHIFT + STACK_ORDER)) )
>> +            {
>> +                if ( val & (sizeof(val) - 1) )
>> +                {
>> +                    /* Most likely a supervisor token. */
>> +                    break;
>> +                }
> 
> Tokens are their own linear address, with metadata in the bottom two
> bits.  I think it would be better to check that explicitly, rather than
> assuming anything nonzero in the upper bits is a token.

Well, yes. What you don't say though is what to do in such an event
(other than simply breaking out of the loop). The lack of a clear route
here is why I've done it the "wider" way. And why hence the comment says
"likely".

>> +
>> +                /*
>> +                 * Ought to be a hypervisor interruption frame.  But don't
>> +                 * (re)log the current frame's %rip.
>> +                 */
>> +                if ( n || ptr[1] != regs->rip )
>> +                    printk("   [<%p>] E %pS\n", _p(ptr[1]), _p(ptr[1]));
>> +                ptr = _p(val);
>> +            }
>> +            else
>> +            {
>> +                /* Ought to be a PV guest hypercall/interruption frame.  */
>> +                printk("   %04lx:[<%p>] E\n", ptr[2], _p(ptr[1]));
>> +                ptr = 0;
> 
> On a CPL3 -> CPL0 transition, the guest's SSP is written back into
> MSR_PL3_SSP.  The supervisor token on MSR_PL0_SSP is marked busy (either
> automatically, or by SETSSBY), but nothing pertaining to CPL3 is pushed
> onto the supervisor shadow stack.
> 
> This is why we can move off an IST stack onto the primary stack when
> interrupting CPL3 with only a CLEARSSBSY/SETSSBSY pair, and no memmove()
> loop of WRSS's.
> 
> In other words, I'm pretty sure this is a dead codeapth.  (Or worse, if
> it happens not to  be dead, then the comment is misleading.)

IOW you're confirming the respective post-commit-message remark. Then, like
above, the question here similarly is: What's the most reasonable thing to
do in the final "else"?

> A CPL1 -> CPL0 transition does push an shstk interrupt frame, and not
> wanting to memmove() the shstk by 3 slots on a context switch is part of
> why I just disallowed PV32 guests when CET was active.

Hmm, yes, such would be needed when switching to PV32. When switching from
PV32, such moving wouldn't be necessary, would it? Upon exiting to ring3
the ring0 shadow stack isn't consulted (beyond an alignment check on SSP),
and upon next entry (from ring3) the proper base ring0 would be used
again. For VMX it would apparently be a matter of using the "load CET"
VM-exit control to get SSP similarly reset to its base value. For SVM I'm
afraid I can't find any explicit information in the PM as to what happens
during #VMEXIT (nor what, if anything, is saved by VMRUN).

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 08:32:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 08:32:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686884.1069465 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfbq5-0003aG-Iv; Thu, 29 Feb 2024 08:32:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686884.1069465; Thu, 29 Feb 2024 08:32:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfbq5-0003a9-Fu; Thu, 29 Feb 2024 08:32:25 +0000
Received: by outflank-mailman (input) for mailman id 686884;
 Thu, 29 Feb 2024 08:32:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfbq4-0003a1-Rg
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 08:32:24 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0f267d62-d6dd-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 09:32:22 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a43488745bcso100772466b.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 00:32:22 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 sa24-20020a1709076d1800b00a440019ac45sm438328ejc.69.2024.02.29.00.32.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 00:32:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0f267d62-d6dd-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709195542; x=1709800342; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=CLMf8eDTNHw6Alw0RWohgl+4qC6dnJ/q/Vt6MEOnodc=;
        b=DvDYzl9/ym25Uhk1VKateXDJHX3Vz9wvBhhJ/keOi3xIOtq2NCM1ZM5KXhrH5ZJNvr
         qjiA1r7V9+i3uGGXKWkFDYyGdiLEmEAHmvvP/pITCPgi/Fxt1xnoKidK/kf1ur6fxssm
         7WX9PdWFG+UJ+LDIlLh9EVBdY/YuQX5zcL13q8wJq9Tz9janYYEh1o4+U5fsKYrvTDJt
         57SSg6B9bHZCYIO18SYcQX/W81+HKyeLGSrSdjSBMAq54lioTu0seLB9TrG33o8X5+6Z
         yyr2fVT3Z6OB9a0bOVMPEJ8skBHTR8+tJ9fV24XxG++sB3jSg8tA6X2NfrT3LZbTPIq8
         sDFQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709195542; x=1709800342;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=CLMf8eDTNHw6Alw0RWohgl+4qC6dnJ/q/Vt6MEOnodc=;
        b=A1QosMo8Bl2IplwDjNAuMGDLHG41RFiV6e+aUnm/3LbPcgB0Z/ZyGCT6JP3/5OKBOS
         EesX60zga7xIq4CcEVV7oU0QD3sEiPwyx77miXBsAmmpcsRjUDG/WgssS7z5CGmBzCZ2
         67DRJ2zWgx9EH6w9F8i8K2e1g3rcJW0mflq3T8VCBuiywophntGWQr/kpdom2VH83ZoD
         CjyAFjYULI6Fj4zdCRBIEZKoGNqWpqx+RMlqZyZE1/TNWxWh4yR6qm+lbSSJoeiTkUwx
         IWA9ZRk9C0BtsC2E3iLSs1bYa1WVsuCTlI4c5g+unJ8OPMjhn8xK0u1IYP0M0mGrVgcV
         oeYA==
X-Forwarded-Encrypted: i=1; AJvYcCX7wQ6hYvuc20aSnX21Uq3x+D3kpjc28ByUr0W12uRPSPBlXus50WyoC7SXQ42bZm/Vrvvd9Sc01I6yvzQC7QaZs3v8euRvC3HkjxEGps8=
X-Gm-Message-State: AOJu0YyUUjtHqnTDrQsH7Ju8FbxrLHjp14invXm9/0rA/PQIBUJrUTAQ
	07mCym6fvP1YJfg24+//+44Cm1sn8qngfWZbQx6+A7BdnSZTe+DvazuNKdDV4A==
X-Google-Smtp-Source: AGHT+IFve4bD7L4jKEawJbWIMMsrD0cTVvPZ5QfJl0TsjKBifJu8LQS0Ho96E+jbYAfPUaEAb/La5g==
X-Received: by 2002:a17:906:3bc9:b0:a43:f949:8e8d with SMTP id v9-20020a1709063bc900b00a43f9498e8dmr973821ejf.67.1709195542159;
        Thu, 29 Feb 2024 00:32:22 -0800 (PST)
Message-ID: <0e2f5394-6322-4845-954f-ff080626523f@suse.com>
Date: Thu, 29 Feb 2024 09:32:20 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH] automation/eclair: extend deviations of MISRA C:2012
 Rule 16.3
Content-Language: en-US
To: Federico Serafini <federico.serafini@bugseng.com>
Cc: consulting@bugseng.com, Simone Ballarin <simone.ballarin@bugseng.com>,
 Doug Goldstein <cardoe@cardoe.com>,
 Stefano Stabellini <sstabellini@kernel.org>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <6c96f889ae3fbdf516f0edfe4a5fb207540b6862.1709109662.git.federico.serafini@bugseng.com>
 <1a00c221-ce1c-45be-9ab3-e8f1a0e5678d@suse.com>
 <98d82c40-a81a-4c75-8cdf-aaef640382b6@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <98d82c40-a81a-4c75-8cdf-aaef640382b6@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 09:01, Federico Serafini wrote:
> On 28/02/24 10:06, Jan Beulich wrote:
>> On 28.02.2024 09:53, Federico Serafini wrote:
>>> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
>>> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
>>
>> Comments below apply similarly to text added to this file.
>>
>>> --- a/docs/misra/deviations.rst
>>> +++ b/docs/misra/deviations.rst
>>> @@ -291,7 +291,14 @@ Deviations related to MISRA C:2012 Rules:
>>>        - Project-wide deviation; tagged as `deliberate` for ECLAIR.
>>>   
>>>      * - R16.3
>>> -     - Switch clauses ending with continue, goto, return statements are safe.
>>> +     - Switch clauses ending with an unconditional flow control statement
>>> +       (i.e., continue, goto, or return) are safe.
>>> +     - Tagged as `safe` for ECLAIR.
>>
>> With this edit (unmentioned in the description, btw) ...
>>
>>> +   * - R16.3
>>> +     - Switch clauses ending with an if-else statement are safe if both
>>> +       branches consist of a flow control statement (i.e., continue, break,
>>> +       goto, return).
>>
>> ... why is it not also "ending with" here?
> 
> Because the allowed pattern is:
> 
> if ( cond )
>   return; /* Or continue / break / goto */
> else
>   break;  /* Or continue / goto / return */
> 
> See below for more information.
> 
>>
>> Also what about either situation ending with a call to a noreturn function?
> 
> This can be added.
> 
>>
>>> @@ -307,6 +314,16 @@ Deviations related to MISRA C:2012 Rules:
>>>        - Switch clauses ending with failure method \"BUG()\" are safe.
>>>        - Tagged as `safe` for ECLAIR.
>>>   
>>> +   * - R16.3
>>> +     - On X86, switch clauses ending generating an exception through
>>> +       \"generate_exception()\" are safe.
>>> +     - Tagged as `safe` for ECLAIR.
>>
>> This macro is limited to the emulator, so shouldn't be deviated globally.
> 
> Noted.
> 
>> Furthermore - why does the special case need mentioning here? Shouldn't
>> it be the underlying pattern which is deviated (along the lines of the
>> earlier ones):
>>
>>      if ( true )
>>      {
>>          ...
>>          goto ...; /* Or break / continue / return */
>>      }
> 
> This pattern that involves a compound statement for the true branch
> is not deviated by this configuration.
> 
> See below for more information.
> 
>>
>>> +   * - R16.3
>>> +     - Switch clauses ending generating a parse error through
>>> +       \"PARSE_ERR_RET()\" are safe.
>>> +     - Tagged as `safe` for ECLAIR.
>>
>> Again this isn't a global scope macro, so shouldn't be deviated globally.
> 
> Noted.
> 
>> Plus it ends in "return", so ought to be covered by the earlier clause.
>> The fact that the return is in a body of do {} while(0) shouldn't matter
>> at all - that's purely syntactic sugar.
> 
> I gather from your comments/questions that you would like to deviate
> *all* the patterns where an unintentional fall through can not happen.
> 
> Rule 16.3 is a purely syntactic rule, and, as a consequence,
> in the current version of ECLAIR additional "allowed pattern" (aka
> deviations) for that rule need to be described through AST nodes,
> meaning that all what you consider as syntactic sugar cannot be ignored.
> 
> A deviation that covers all the pattern you are asking for could be
> done, but it will result in a complex and quite long expression
> (not easy to read and justify in front of an assessor).
> 
> Hence, what I am proposing is to deviate only the the simplest and
> most readable cases, such as:
> 
> if ( cond )
>    return x;
> else
>    return y;
> 
> without involving compound statements, fake do-wile and fake if
> statements but rather deviating the macro inside of which are used
> (as I did).

I see. Problem is that this isn't sufficient for the code we have, and
the seemingly random deviation of certain constructs by name looks to
me as pretty undesirable.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 09:02:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 09:02:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686896.1069475 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfcJE-00086V-Tg; Thu, 29 Feb 2024 09:02:32 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686896.1069475; Thu, 29 Feb 2024 09:02:32 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfcJE-00086O-Qz; Thu, 29 Feb 2024 09:02:32 +0000
Received: by outflank-mailman (input) for mailman id 686896;
 Thu, 29 Feb 2024 09:02:32 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfcJD-00086G-SS
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 09:02:31 +0000
Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com
 [2a00:1450:4864:20::52b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 442d1994-d6e1-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 10:02:30 +0100 (CET)
Received: by mail-ed1-x52b.google.com with SMTP id
 4fb4d7f45d1cf-565a3910f86so987544a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 01:02:29 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s12-20020a056402520c00b0056691924615sm414578edd.2.2024.02.29.01.02.28
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 01:02:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 442d1994-d6e1-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709197349; x=1709802149; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=A440DB1rpePBv7q7I/YQhlrb420DWDiyJv6Tx+CZ5uA=;
        b=LhWVjAGytZvsTdpaZlaY+BFkmvYjdfu6FJ7YOUvahGT7Vgd57zoMjwwnct8ew3aMWI
         7j4unoyj8N1TfVHLLqMH+T/HG/L+VeK8MkajG3StKnZtRiciCzINRqNjHjb1xJUfbiR7
         TmTwXTIpr1CoVAFt/Xn3lQJPw1yeY+siUfhMJ6+pw9NSd4+VcLczhVVWxynwv6Vo0OiF
         wIH5hW1TVPPsl7heG9tOdVybrUHvD1tzvSEeEWRTSUAHQOptksfEZfNAJubz8s59d3VO
         FK+aMmKCaM/bSok/F02MKtdlt8CW/MBeRqJJl2od0koIn9TsBQ8+vUKzCmGG3SepGrLo
         BXhw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709197349; x=1709802149;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=A440DB1rpePBv7q7I/YQhlrb420DWDiyJv6Tx+CZ5uA=;
        b=wRNjLTWddP8n515vTH0mhV8EEDqmyOER3KUW9EiyjP5ooFX2ZkxaAlYLJbnE/kZu5R
         ItA26z0OW90Vwhmqh97V7y3fF0XlaqN5BYHPUQiC3/MOZWb+FyT2uLZRN/99App46eMB
         QMjqhDL2ra0D2mZJl7jMLt+W+5TsujqPpxOLMdDt8ub7c5PiOO4up6vowMk4YwoL1cFc
         2hhdixQhL7eLn/QHNvReYno+4l1tZqfVY6vGqWF1ITywteUhkzY3S7sB5Ch9cyrDdWPY
         3KABf11Gg5PME+uWPboBdk6leOUmkI7rQC0riUMEbhE14CSGAfn2+PrJRZ+sSYMsLMFB
         zdCA==
X-Forwarded-Encrypted: i=1; AJvYcCVkzDLP4VpZ+0fX1MTA6qpFp7sD5w/IkmtWSkmMvBHCmWeQuJAsPevrN5wXg7hGya2NNO7f6dd2vEgmBMvFCafKRKEzNwVSljAoWk8yCf0=
X-Gm-Message-State: AOJu0Yxs0iC7Fk4nH57I6slM3mqR7GmcsxjNTMHxBxq8bTQvlbsHOA+j
	VCUlmyvh1tYBbrwwLXJwFzSsL5y+u0LsZwGCt2WLcWMiFipzrYZKqDU8DYGNJQ==
X-Google-Smtp-Source: AGHT+IEQ35IA/S9f+UX5V7UcXXAkVypu3ElqFMIJRrUFLmS7faCc3K40/tciZ8nMfraRgVEPVn1mug==
X-Received: by 2002:a05:6402:2153:b0:565:bb25:bb7b with SMTP id bq19-20020a056402215300b00565bb25bb7bmr1031236edb.6.1709197349201;
        Thu, 29 Feb 2024 01:02:29 -0800 (PST)
Message-ID: <e9140136-3446-4d0f-a977-549d5aed11a1@suse.com>
Date: Thu, 29 Feb 2024 10:02:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86: serializing of non-serializing MSR writes
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Dave Hansen <dave.hansen@intel.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <58656398-2d64-48b8-9ddc-c6836847a586@suse.com>
 <3bc3a319-45a8-4b22-b225-a2b19ee04c28@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3bc3a319-45a8-4b22-b225-a2b19ee04c28@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 02:10, Andrew Cooper wrote:
> On 28/02/2024 2:48 pm, Jan Beulich wrote:
>> Linux commit 25a068b8e9a4e ("x86/apic: Add extra serialization for non-
>> serializing MSRs") explains why an MFENCE+LFENCE pair is generally
>> needed ahead of ICR writes in x2APIC mode, and also why at least in
>> theory such is also needed ahead of TSC_DEADLINE writes. A comment of
>> our own in send_IPI_mask_x2apic_phys() further explains a condition
>> under which the LFENCE can be avoided.
>>
>> Further Linux commit 04c3024560d3 ("x86/barrier: Do not serialize MSR
>> accesses on AMD") explains that this barrier isn't needed on AMD or
>> Hygon, and is in fact hampering performance in a measurable way.
>>
>> Introduce a similarly named helper function, but with a parameter
>> allowing callers to specify whether a memory access will follow, thus
>> permitting the LFENCE to be omitted.
>>
>> Putting an instance in apic_wait_icr_idle() is to be on the safe side.
>> The one case where it was clearly missing is in send_IPI_shortcut(),
>> which is also used in x2APIC mode when called from send_IPI_mask().
>>
>> Function comment shamelessly borrowed (but adapted) from Linux.
>>
>> Fixes: 5500d265a2a8 ("x86/smp: use APIC ALLBUT destination shorthand when possible")
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> I question the need for a fence ahead of writing TSC_DEADLINE: The Linux
>> commit message talks about LVT being accessed via MMIO in xAPIC mode,
>> but that should not be relevant here: It's all the local CPU, so there
>> ought to not be visibility concerns (much like for a self-IPI no fence
>> is needed ahead of the ICR write). If that wasn't needed, we could
>> further use alternatives patching to remove the fence also from
>> apic_wait_icr_idle() when in xAPIC mode. (And only then would I agree to
>> have APIC in the feature identifier, like Linux has it.)
>>
>> A number of apic_write() may better be turned into apic_mem_write(), in
>> particular e.g. the ones in send_IPI_mask_{flat,phys}(). That way it
>> would be quite a bit easier to spot paths taken only in xAPIC mode.
>>
>> The INIT-INIT-SIPI sequence for AP startup doesn't use any barrier, also
>> not in Linux afaics. I can't explain the lack thereof, though.
> 
> I have some opinions about what Linux did here...  I don't think a
> single vendor/arch-neutral helper can possibly be right.
> 
> It is vendor and uarch dependent which WRMSR's transparently degrade to
> WRMSRNS, and it is vendor dependent which serialising sequence to use
> (if any).  e.g. AMD have recently (Zen2 uarch I believe) retroactively
> defined FS/GS_BASE to be non-serialising.  (And this is another CPUID
> bit we need to OR backwards in time.)
> 
> Furthermore, IIRC AMD still owe us an update to the APM; the APM
> currently says that a serialising sequence is needed for ICR.  I'm told
> this isn't actually true, but I'm also very wary making an adjustment
> which is directly contradicted by the docs.

I can see you wanting the doc to be corrected. What I'm having trouble
with is you having indicated (long ago) that we can avoid this fence on
AMD, just to now effectively object to me (finally) getting around to
actually doing so?

> The Linux change explains why in principle the IPI can be emitted before
> the stores are visible.
> 
> This does actually explain TSC_DEADLINE too.  Setting a deadline in the
> past gets you an interrupt immediately, and if you combine that with a
> WRMSR being ordered ahead of an MFENCE, then causality is violated. 
> You'll take the interrupt on whichever instruction boundary has most
> recently retired, which will can be the wrong side of the WRMSR
> triggering the interrupts, at which point you'll livelock taking timer
> interrupts and re-arming the timer in the past.

Are you saying the interrupt is raised ahead of the insn retiring? That
would be concerning, imo.

Irrespective of that, as to live-locking: If that would really be a
possible issue, moving lapic_timer_on() ahead of local_irq_enable() in
acpi_processor_idle() and mwait_idle() would avoid that; the sole other
use of reprogram_timer() already runs with IRQs off.

> Now, for fencing, things are more complicated.  AMD define MFENCE as
> architecturally serialising.  Intel do not, hence why apparently a WRMSR
> can possibly move across it.  The LFENCE is added for it's new
> speculative property of dispatch serialising.
> 
> We don't actually care about architecturally serialising.  If someone is
> interacting with these MSRs with relevant data in WC memory, then they
> get to keep all resulting pieces.
> 
> However, we do care about plain stores, and for that we do need an
> MFENCE;LFENCE on Intel  (The jury is out on whether a single
> SERIALIZE[sic] would be better, but it should have the correct semantics
> architecturally speaking.)
> 
> In particular, ...
> 
>> --- a/xen/arch/x86/genapic/x2apic.c
>> +++ b/xen/arch/x86/genapic/x2apic.c
>> @@ -97,15 +97,15 @@ static void cf_check send_IPI_mask_x2api
>>  
>>      /*
>>       * Ensure that any synchronisation data written in program order by this
>> -     * CPU is seen by notified remote CPUs. The WRMSR contained within
>> -     * apic_icr_write() can otherwise be executed early.
>> +     * CPU is seen by notified remote CPUs. The WRMSR contained in the loop
>> +     * below can otherwise be executed early.
>>       * 
>> -     * The reason smp_mb() is sufficient here is subtle: the register arguments
>> +     * The reason MFENCE is sufficient here is subtle: the register arguments
>>       * to WRMSR must depend on a memory read executed after the barrier. This
>>       * is guaranteed by cpu_physical_id(), which reads from a global array (and
>>       * so cannot be hoisted above the barrier even by a clever compiler).
>>       */
>> -    smp_mb();
>> +    weak_wrmsr_fence(true);
>>  
>>      local_irq_save(flags);
>>  
>> @@ -130,7 +130,7 @@ static void cf_check send_IPI_mask_x2api
>>      const cpumask_t *cluster_cpus;
>>      unsigned long flags;
>>  
>> -    smp_mb(); /* See above for an explanation. */
>> +    weak_wrmsr_fence(true); /* See above for an explanation. */
>>  
>>      local_irq_save(flags);
>>  
>> --- a/xen/arch/x86/include/asm/cpufeatures.h
>> +++ b/xen/arch/x86/include/asm/cpufeatures.h
>> @@ -24,7 +24,7 @@ XEN_CPUFEATURE(APERFMPERF,        X86_SY
>>  XEN_CPUFEATURE(MFENCE_RDTSC,      X86_SYNTH( 9)) /* MFENCE synchronizes RDTSC */
>>  XEN_CPUFEATURE(XEN_SMEP,          X86_SYNTH(10)) /* SMEP gets used by Xen itself */
>>  XEN_CPUFEATURE(XEN_SMAP,          X86_SYNTH(11)) /* SMAP gets used by Xen itself */
>> -/* Bit 12 unused. */
>> +XEN_CPUFEATURE(NO_WRMSR_FENCE,    X86_SYNTH(12)) /* No MFENCE{,+LFENCE} ahead of certain WRMSR. */
>>  XEN_CPUFEATURE(IND_THUNK_LFENCE,  X86_SYNTH(13)) /* Use IND_THUNK_LFENCE */
>>  XEN_CPUFEATURE(IND_THUNK_JMP,     X86_SYNTH(14)) /* Use IND_THUNK_JMP */
>>  XEN_CPUFEATURE(SC_NO_BRANCH_HARDEN, X86_SYNTH(15)) /* (Disable) Conditional branch hardening */
>> --- a/xen/arch/x86/include/asm/msr.h
>> +++ b/xen/arch/x86/include/asm/msr.h
>> @@ -97,6 +97,25 @@ static inline void msr_split(struct cpu_
>>      regs->rax = (uint32_t)val;
>>  }
>>  
>> +/*
>> + * Make previous memory operations globally visible before a WRMSR.  Most
>> + * WRMSRs are full serializing instructions themselves and do not require this
>> + * barrier.  This may only be required for the TSC_DEADLINE and x2APIC MSRs.
>> + *
>> + * MFENCE makes writes visible, but only affects load/store instructions.
>> + * WRMSR is unfortunately not a load/store instruction and is unaffected by
>> + * MFENCE.
> 
> [ On Intel.   AMD didn't end up with this (mis)behaviour. ]
> 
>>   The LFENCE ensures that the WRMSR is not reordered, but callers
>> + * can indicate to avoid it when they have a suitable memory access between
>> + * the invocation of this function and the WRMSR in question.
> 
> ... this makes no sense.
> 
> We need the LFENCE for dispatch serialising properties, not it's load
> ordering properties.  What use will other memory have, when the entire
> problem is that WRMSR doesn't interact with them?

Are you suggesting the comment (and code) in send_IPI_mask_x2apic_*()
(left visible in context further up) are wrong then? I consider it
correct (looking forward to see you prove it wrong), and with that
having a way to avoid the LFENCE looks correct to me. Plus the comment
here doesn't say "load ordering" anywhere. It's strictly execution
ordering, guaranteed by a memory access the WRMSR input is dependent
upon. For load ordering, MFENCE alone would be enough.

> Worse, it's a Spectre-v1 gadget and we're now acutely familiar with how
> the CPU will find its way around these.  So even expressing "I
> critically need the LFENCE" still gets you pot luck on whether it has
> any effect against a causality-violating WRMSR.

Hmm, besides me possibly taking this as "drop this patch" (which could
do with making explicit, if that was meant), I'm afraid I can't view
this remark as actionable in any way. Yet I firmly expect an IPI to not
be raised speculatively (and, as said above, neither an LAPIC timer
interrupt), so I'm even having trouble seeing how this would form a
Spectre-v1 gadget.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 09:39:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 09:39:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686906.1069485 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfct5-0004Mi-Qt; Thu, 29 Feb 2024 09:39:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686906.1069485; Thu, 29 Feb 2024 09:39:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfct5-0004Mb-Nc; Thu, 29 Feb 2024 09:39:35 +0000
Received: by outflank-mailman (input) for mailman id 686906;
 Thu, 29 Feb 2024 09:39:34 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfct4-0004MV-6j
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 09:39:34 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 70b718b2-d6e6-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 10:39:31 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a26ed1e05c7so127104566b.2
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 01:39:31 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 t9-20020a17090616c900b00a44218b5defsm498611ejd.60.2024.02.29.01.39.30
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 01:39:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 70b718b2-d6e6-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709199571; x=1709804371; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=PNUxzD4V7HMu9zXzs5ZGoPtX0o3RxKaDv/Vsb8iSies=;
        b=aDeTU0xUG0mHzkZ6xeY6LZ7/uON90t/mpfaZHbcz58abx8TuUyC6/lQHfKYkXxoNtK
         7pXmQYrd272XtGMlKJevKmKORY8+BqkrV5YbKQh40zqXM0YEyVFV7s5xyW42AOUW3Rp8
         267SEv2IFDDVmXOW0hv1pHiQ69hIAuIzMgJrkM9NMzbrjY8F9ufeYt9c5+d5eGwBiwqW
         8vfsvovMz8fap0wRTnuwXU0I5p9Oo8SYsgQHdMwWquIleNMtyzM1wdq9MrfY2oU4mSLb
         906QksXF8Yhwg3U2ooQIFW+PNnZE64ZCvfDrWjp/gFdBGo1tFHpq88gPv3ds64PJRspD
         n8Dg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709199571; x=1709804371;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=PNUxzD4V7HMu9zXzs5ZGoPtX0o3RxKaDv/Vsb8iSies=;
        b=X0iED/shNHxy7KsCFTuJnq6+mUwogPBjv8653cDIFuv8VNKQwcCYFE6toR3vUNZNcM
         dUrDURYoGhkAkpyAqwrsZRcu1x0QivLGbeNv832x0B+WLW6BTUITpsVy3AMqJEX6e7hN
         OZyE6nW0cDOjDoNmpF2sx3QPh8WptyClfZj34y66+Jk8ejq6cOmOD8xjAKAUNyGvC/Rk
         uuGa8Hq0LEZUmFAs/g5b+7CK+mvG8OGgkG+pt0ydn4c/q8CSb4jaEA9g/r9fTGDMhZV7
         LFGKPR+Zo+Kn12BLavmKYvDH+jOE1VSrGGJr6Sqk8Iojd0pN4/pT5zCmtaywdTLVEWY2
         QETw==
X-Forwarded-Encrypted: i=1; AJvYcCU/87FNTCBVjZhk1PVRjgO+cv0GU01OvGpRi+uXe6yrY1cIoj05PeItORnoBOPF3Ca+ORXEG3nhhc4agzvvR1eWSiagCowRElMcwyEtZfg=
X-Gm-Message-State: AOJu0YwWiHhWL8Csv2/gZPUwTCiHlCNz2KOAThIWAbtnSaAQeIcq65MF
	L8KuOs0xpHKXHbeDh3g1Yt1sOeR/2ayRwpRpXoP5WAej897VLBrhxhhim9BKMg==
X-Google-Smtp-Source: AGHT+IFnnk4xvYiZxMh+MdDBIxZ/RVzDeUAyI5uls0PhX7Umq239vSnEH/tilvVdrClX/eQXrI770w==
X-Received: by 2002:a17:906:a28c:b0:a43:d853:98f1 with SMTP id i12-20020a170906a28c00b00a43d85398f1mr970099ejz.17.1709199571250;
        Thu, 29 Feb 2024 01:39:31 -0800 (PST)
Message-ID: <8579a6d3-6860-4a7b-848f-1240da768f33@suse.com>
Date: Thu, 29 Feb 2024 10:39:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 2/4] x86: record SSP at non-guest entry points
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>,
 "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <33da6f8b-af22-48c8-acce-3aa55c0e0414@suse.com>
 <0ad4543b-8eed-4147-b32d-b68d21fade98@suse.com>
 <320ca33d-2c40-417e-874c-8af447c43734@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <320ca33d-2c40-417e-874c-8af447c43734@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 28.02.2024 16:16, Andrew Cooper wrote:
> On 28/02/2024 1:52 pm, Jan Beulich wrote:
>> We will want to use that value for call trace generation, and likely
>> also to eliminate the somewhat fragile shadow stack searching done in
>> fixup_exception_return(). For those purposes, guest-only entry points do
>> not need to record that value.
>>
>> To keep the saving code simple, record our own SSP that corresponds to
>> an exception frame, pointing to the top of the shadow stack counterpart
>> of what the CPU has saved on the regular stack. Consuming code can then
>> work its way from there.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> To record the full 64-bit value, some of the unused bits in the %cs slot
>> could be used. Sadly that slot has saved_upcall_mask in an unhelpful
>> location, otherwise simply storing low and high 32 bits in those two
>> separate half-slots would be a pretty obvious choice. As long as
>> "entry_ssp" is used in non-guest-entry frames only, we could of course
>> put half of it into a union with saved_upcall_mask ...
>>
>> Else may want to put a BUILD_BUG_ON(VADDR_BITS > 48) somewhere, but I'm
>> afraid I can't really identify a good place for such to live.
> 
> Perhaps in reinit_bsp_stack() when we enable SHSTK on the BSP?
> 
> Having it anywhere vaguely relevant is better than not having it.

Okay.

>> Leveraging that the CPU stores zero in the upper bits of the selector
>> register slots, the save sequence could also be
>>
>>         shl   $16, %rcx
>>         or    %rcx, UREGS_entry_ssp-2(%rsp)
>>
>> That's shorter and avoids a 16-bit operation, but may be less desirable,
>> for being a read-modify-write access.
> 
> I doubt you'll be able to measure a difference between the two options
> (it's into the active stack, after all), but the two stores is probably
> marginally better.  When shstks are active, we're taking a large hit
> from the busy token handling.
> 
> I was concerned by the misaligned access, but it's not misaligned, its
> it?  It's the start of entry_ssp which is misaligned and the -2 brings
> it back to being properly aligned.

Correct.

>> --- a/xen/arch/x86/include/asm/asm_defns.h
>> +++ b/xen/arch/x86/include/asm/asm_defns.h
>> @@ -221,7 +221,7 @@ static always_inline void stac(void)
>>  #endif
>>  
>>  #ifdef __ASSEMBLY__
>> -.macro SAVE_ALL compat=0
>> +.macro SAVE_ALL compat=0 ssp=IS_ENABLED(CONFIG_XEN_SHSTK)
> 
> I'm not sure this is what you want to do.  Because it's only the
> default, we'll still....
> 
>>          addq  $-(UREGS_error_code-UREGS_r15), %rsp
>>          cld
>>          movq  %rdi,UREGS_rdi(%rsp)
>> @@ -235,6 +235,9 @@ static always_inline void stac(void)
>>          movq  %rax,UREGS_rax(%rsp)
>>          xor   %eax, %eax
>>  .if !\compat
>> +.if \ssp
>> +        rdsspq %rcx
> 
> ... pick this up even in !CONFIG_XEN_SHSTK builds, and ...
> 
>> +.endif
>>          movq  %r8,UREGS_r8(%rsp)
>>          movq  %r9,UREGS_r9(%rsp)
>>          movq  %r10,UREGS_r10(%rsp)
>> @@ -264,6 +267,11 @@ static always_inline void stac(void)
>>          xor   %r13d, %r13d
>>          xor   %r14d, %r14d
>>          xor   %r15d, %r15d
>> +.if \ssp && !\compat
>> +        mov   %cx, UREGS_entry_ssp(%rsp)
>> +        shr   $16, %rcx
>> +        mov   %ecx, UREGS_entry_ssp+2(%rsp)
> 
> ... store it here.
> 
> I think you need to use ssp=1 by default, and
> 
> #ifdef CONFIG_XEN_SHSTK
> .if
>     ...
> 
> for these two blocks, so they disappear properly in !SHSTK builds.

I'm afraid I don't follow: The macro parameter exists for use sites
to pass 0 even in SHSTK builds, for the entry-from-guest paths where
its recording is of no interest. Non-zero should never be passed
explicitly. Perhaps I ought to add a comment to this effect:

/* Use sites may override ssp to 0. It should never be overridden to 1. */

If that doesn't address your concern, then I'm afraid I'm not fully
understanding your comments, or I'm overlooking something crucial.

> But for the rest of the behaviour, there are two overlapping things,
> because you end up getting entry_ssp=0 in SHSTK builds running on
> hardware without shstk active.

Yes. I guess I don't understand what you're trying to indicate to
me.

> And with that in mind, to confirm, the RDSSP block depends on the xor
> %ecx,%ecx between the two hunks in order to function as intended?

Yes. In fact initially I had used %edx, with "interesting" effects.
It's not said anywhere in the macro that %edx needs to be zero by
the point the macro completes; comments to this effect exist only
past several of the use sites of the macro.

>> --- a/xen/include/public/arch-x86/xen-x86_64.h
>> +++ b/xen/include/public/arch-x86/xen-x86_64.h
>> @@ -183,7 +183,19 @@ struct cpu_user_regs {
>>      uint8_t  _pad1[3];
>>      __DECL_REG_LO16(flags); /* rflags.IF == !saved_upcall_mask */
>>      __DECL_REG_LO8(sp);
>> -    uint16_t ss, _pad2[3];
>> +    uint16_t ss;
>> +#if !defined(__XEN__)
>> +    uint16_t _pad2[3];
>> +#elif defined(COMPILE_OFFSETS)
>> +    uint16_t entry_ssp[3];
>> +#else
>> +    /*
>> +     * This points _at_ the corresponding shadow stack frame; it is _not_ the
>> +     * outer context's SSP.  That, if the outer context has CET-SS enabled,
>> +     * is stored in the top slot of the pointed to shadow stack frame.
>> +     */
>> +    signed long entry_ssp:48;
>> +#endif
> 
> I have to admit that I dislike this.  (And only some of that is because
> it's work I'm going to have to revert in order to make FRED support work...)

Right, some of the bits in the various slots which have space available
are used there. But the frame layout there is different anyway, so by
that point we won't get away without re-working our exception frame
layout. Taking care of the new extra field then is, I expect, not going
to make much of a difference, when without doing the transformation now
we can have some immediate gain.

> We desperately need to break our use of this structure to start with,
> and with that done, we don't need to play games about hiding SSP in a
> spare 6 bytes in an ABI dubiously made public nearly two decades ago.
> 
> How hard would it be just:
> 
> #define cpu_user_regs abi_cpu_user_regs
> #include <public/xen.h>
> #undef cpu_user_regs
> 
> and make a copy of cpu_user_regs that we can really put an SSP field into?

Well, that's easy to write here, but this pattern would then need
repeating in a few dozen places. Even abstracting it by way of a
helper header would seem problematic to me: We can't very well
forbid common code to include public/xen.h directly. We also have
a couple of direct inclusions of public/arch-x86/xen.h, which would
similarly need taking care of.

A better approach might be an #ifdef __XEN__ inside the header
itself. I could try that, but I'd stop as soon as I consider knock-
on effects too heavy for the simple purpose here.

I know you've been pointing out that we want to change how the stack
is organized. This not having happened yet was, for me, largely
because of quite likely not being straightforward to achieve.

> It would make this patch more simple, and we could finally get the vm86
> block off the stack (which also fixes OoB accesses in the #DF handler -
> cant remember if I finished the bodge around that or not.)
> 
> Irrespective of anything else, why do we have COMPILE_OFFSETS getting in
> here?

Because offsetof() implies determining the address of a field, which
is not allowed for bitfields.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 09:56:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 09:56:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686910.1069505 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfd93-0007Il-Cx; Thu, 29 Feb 2024 09:56:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686910.1069505; Thu, 29 Feb 2024 09:56:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfd93-0007Ie-AJ; Thu, 29 Feb 2024 09:56:05 +0000
Received: by outflank-mailman (input) for mailman id 686910;
 Thu, 29 Feb 2024 09:56:04 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xh/y=KG=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rfd92-0007IO-BJ
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 09:56:04 +0000
Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com
 [2607:f8b0:4864:20::730])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bed128b0-d6e8-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 10:56:02 +0100 (CET)
Received: by mail-qk1-x730.google.com with SMTP id
 af79cd13be357-787df45e513so43875785a.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 01:56:02 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 vv3-20020a05620a562300b00787ad324b83sm505124qkn.120.2024.02.29.01.55.59
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 01:56:00 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bed128b0-d6e8-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709200560; x=1709805360; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ichlPTccQ4535UGjZfXAC8i6aK0BxI1NGty2U9n+z6Q=;
        b=KyHOkarHbqmZVNWGBNjj0hIpDC1XeNHz4/87eu/QfSEsCZ12EiiwrFX4p9lO6HfFea
         WU8vl2Kn7WCvYXHvfqEHKRZzO9ZQJb7k9juOu/7wQomm5PJRTPP9eiC95s2qgcqdW0xO
         aq5yk2HkCiYJDRnz16JAI4PKH87ZHUDFLw9/8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709200560; x=1709805360;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ichlPTccQ4535UGjZfXAC8i6aK0BxI1NGty2U9n+z6Q=;
        b=Ir1teeMrnFcROO2FGpf4tiaKCp9TlQdi5C3nk2td42CEjAa35ulAHq1Vw0S4u+XGwY
         phZo9uLoY+5+cEceqWA5PIJ5wjwBlR7z+jdraricy04PaWV+20XR3HqkHezSSK10idWT
         5e8eui3M0fgv6yU4dfceAsR7kU2VBYo/slGVLeJ25/ebA2gBirj3jlFhNitpGJJEgB//
         oTBTY43eU1P3YYPoZ+TQ2ez2s1WVRKGTw7mGX1KduSaGXvZqLF6DtX4ybaJIXfdlwvT1
         lfT3tpYuF+dyZkYVeksfKzAc4WZtrGz1u0/mooivRPyOgViVQqlEeMZS4u2bz8AhP7yu
         18GA==
X-Gm-Message-State: AOJu0Yy4F7WeLdm682Kybl2BI8Hz7WNsBtHKmQo0A9Z0riz+8g4OCPM0
	dLhA769xUYMRkJA2XWtF8LF1s3H6XNPzjDEgPxV9swICXAb9Ih978lYImkweaefjc7zCAvlixh0
	L
X-Google-Smtp-Source: AGHT+IFCTo2madUZMCyQcfvHbsHEVw0kDEKX9gsO3v8hankNjc86gX9tbq2ZjgRZGIVy4CyCSez8Ig==
X-Received: by 2002:a05:620a:24d2:b0:787:f8fd:8ed with SMTP id m18-20020a05620a24d200b00787f8fd08edmr1812421qkn.55.1709200560358;
        Thu, 29 Feb 2024 01:56:00 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [PATCH 1/2] README: bump minimum required clang/llvm version
Date: Thu, 29 Feb 2024 10:55:28 +0100
Message-ID: <20240229095529.17723-2-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240229095529.17723-1-roger.pau@citrix.com>
References: <20240229095529.17723-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We no longer have a way to build with the minimum required clang/llvm version
stated in the README on the gitlab CI loop, since we dropped the Debian Jessie
container that had Clang 3.5.0.

Bump the minimum required Clang/LLVM to the one used in the oldest production
FreeBSD version (13.2 currently), as that's the main reason I care to maintain
Clang/LLVM support, and as far as I know FreeBSD is the only production
deployment of Xen built with Clang/LLVM.

Purge the build jobs for non-supported Clang versions from Gitlab CI.  Note the
.dockerfiles for the respective distros are explicitly not adjusted to drop the
install of the clang packages, or else those jobs would start to fail on older
Xen branches.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
I'm willing to consider older versions, but there needs to be a reason (iow:
use-case) for considering those, as maintaining support for older toolchains is
a burden.
---
 README                          |  2 +-
 automation/gitlab-ci/build.yaml | 45 ---------------------------------
 2 files changed, 1 insertion(+), 46 deletions(-)

diff --git a/README b/README
index c8a108449e29..5fe52cc7a932 100644
--- a/README
+++ b/README
@@ -41,7 +41,7 @@ provided by your OS distributor:
         - GCC 4.1.2_20070115 or later
         - GNU Binutils 2.16.91.0.5 or later
         or
-        - Clang/LLVM 3.5 or later
+        - Clang/LLVM 14.0.0 or later
       - For ARM 32-bit:
         - GCC 4.9 or later
         - GNU Binutils 2.24 or later
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 6d2cb18b8883..347fe1b5a8db 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -638,21 +638,6 @@ debian-stretch-gcc:
   variables:
     CONTAINER: debian:stretch
 
-debian-stretch-clang:
-  extends: .clang-x86-64-build
-  variables:
-    CONTAINER: debian:stretch
-
-debian-stretch-clang-debug:
-  extends: .clang-x86-64-build-debug
-  variables:
-    CONTAINER: debian:stretch
-
-debian-stretch-32-clang-debug:
-  extends: .clang-x86-32-build-debug
-  variables:
-    CONTAINER: debian:stretch-i386
-
 debian-stretch-32-gcc-debug:
   extends: .gcc-x86-32-build-debug
   variables:
@@ -725,16 +710,6 @@ ubuntu-trusty-gcc-debug:
   variables:
     CONTAINER: ubuntu:trusty
 
-ubuntu-xenial-clang:
-  extends: .clang-x86-64-build
-  variables:
-    CONTAINER: ubuntu:xenial
-
-ubuntu-xenial-clang-debug:
-  extends: .clang-x86-64-build-debug
-  variables:
-    CONTAINER: ubuntu:xenial
-
 ubuntu-xenial-gcc:
   extends: .gcc-x86-64-build
   variables:
@@ -745,16 +720,6 @@ ubuntu-xenial-gcc-debug:
   variables:
     CONTAINER: ubuntu:xenial
 
-ubuntu-bionic-clang:
-  extends: .clang-x86-64-build
-  variables:
-    CONTAINER: ubuntu:bionic
-
-ubuntu-bionic-clang-debug:
-  extends: .clang-x86-64-build-debug
-  variables:
-    CONTAINER: ubuntu:bionic
-
 ubuntu-bionic-gcc:
   extends: .gcc-x86-64-build
   variables:
@@ -775,16 +740,6 @@ ubuntu-focal-gcc-debug:
   variables:
     CONTAINER: ubuntu:focal
 
-ubuntu-focal-clang:
-  extends: .clang-x86-64-build
-  variables:
-    CONTAINER: ubuntu:focal
-
-ubuntu-focal-clang-debug:
-  extends: .clang-x86-64-build-debug
-  variables:
-    CONTAINER: ubuntu:focal
-
 opensuse-leap-clang:
   extends: .clang-x86-64-build
   variables:
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 09:56:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 09:56:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686909.1069495 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfd90-00074F-5n; Thu, 29 Feb 2024 09:56:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686909.1069495; Thu, 29 Feb 2024 09:56:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfd90-000748-3D; Thu, 29 Feb 2024 09:56:02 +0000
Received: by outflank-mailman (input) for mailman id 686909;
 Thu, 29 Feb 2024 09:56:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xh/y=KG=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rfd8y-000742-O5
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 09:56:00 +0000
Received: from mail-qv1-xf2f.google.com (mail-qv1-xf2f.google.com
 [2607:f8b0:4864:20::f2f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id bcf2dba3-d6e8-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 10:55:59 +0100 (CET)
Received: by mail-qv1-xf2f.google.com with SMTP id
 6a1803df08f44-68fb3a3f1c5so3513646d6.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 01:55:59 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 nh7-20020a056214390700b0068f35e9e9a2sm562125qvb.8.2024.02.29.01.55.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 01:55:57 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bcf2dba3-d6e8-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709200558; x=1709805358; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=09Gvp0FqPrs4OCB/d8XxpIF80agD7996UJLIpBx9qPk=;
        b=tOFAvGdwonq+A4FdFr2+Nl5Clqc6vQlp1o3iy66Pg8YFZlPkXQFlzsz4ukyq9s6da/
         4eUQa2m7pKzfANsUg8pa0zOppMVhRZgVVNfls1kEf8WzkFOLswLl41HiEXOCHElfzPti
         94h4rHBlu1mwANbsIJE6dgKRSKoiRmndcftn8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709200558; x=1709805358;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=09Gvp0FqPrs4OCB/d8XxpIF80agD7996UJLIpBx9qPk=;
        b=XHhbLnt41Bc1RD78WzhVUw8q+Aj2kjkc2PhqEduTqlHOcGXHR8BxouvGUsOVUsuEg2
         6wpsOwzxSAfOqur+BpBBSNRaMl6ZiuKLLKS+KtQjoDTfF/2ZBGRSb/Z48kmBJtJOIaNB
         2IOqV9k6i+DwBFMJBz3l37MdAqmC4/VbixVGzqqS0DbKh/r138+eG6CKXZsWTWam9I2S
         KbIE2vi28qgt2zTzfnPfv5QeR51OSQLsRdfS2vDTlw6+JQtQZeJOE4tUj8IBHlQcO64i
         L/Vkhwe8joWWpely4rJZXfeM1eiaREzJH0FCBKJQ1lVDEbbNdDoZMu/tVUuMzgWfiW64
         ifKA==
X-Gm-Message-State: AOJu0Yyp/GieesrNH2u7gNa4JnbdLTv69pKv/UBbKDxk5Mw6Sk0kyNSJ
	xpDPSwEpQbuV7Wu79SB3/4m3GMS02SF9gXZlCI2RWwTiGXDQlGn1x+0jlQv+Rtp5AJfQW0QfRsK
	I
X-Google-Smtp-Source: AGHT+IGRHRg4kDIvqPMEQtw2h50mzs4HISfZldU38u09+ifRS3hv+VLMcpw/n5LNL405fsibNskB+w==
X-Received: by 2002:a0c:9a9a:0:b0:68f:e80f:a27c with SMTP id y26-20020a0c9a9a000000b0068fe80fa27cmr1545620qvd.47.1709200557783;
        Thu, 29 Feb 2024 01:55:57 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Doug Goldstein <cardoe@cardoe.com>
Subject: [PATCH 0/2] Restrict Clang/LLVM supported versions
Date: Thu, 29 Feb 2024 10:55:27 +0100
Message-ID: <20240229095529.17723-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hello,

The following series limits the supported versions of Clang to what I
actually care about, as I seem to be the only one that actively uses
Clang/LLVM builds of Xen.

Patch 2 adds non-debug tests for -clang builds.

Thanks, Roger.

Roger Pau Monne (2):
  README: bump minimum required clang/llvm version
  automation: introduce non debug clang based tests

 README                          |  2 +-
 automation/gitlab-ci/build.yaml | 45 ---------------------------------
 automation/gitlab-ci/test.yaml  | 18 +++++++++++--
 3 files changed, 17 insertions(+), 48 deletions(-)

-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 09:56:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 09:56:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686911.1069516 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfd96-0007Zb-Ko; Thu, 29 Feb 2024 09:56:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686911.1069516; Thu, 29 Feb 2024 09:56:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfd96-0007ZS-I8; Thu, 29 Feb 2024 09:56:08 +0000
Received: by outflank-mailman (input) for mailman id 686911;
 Thu, 29 Feb 2024 09:56:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xh/y=KG=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rfd95-000742-SE
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 09:56:07 +0000
Received: from mail-qv1-xf35.google.com (mail-qv1-xf35.google.com
 [2607:f8b0:4864:20::f35])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c1718812-d6e8-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 10:56:06 +0100 (CET)
Received: by mail-qv1-xf35.google.com with SMTP id
 6a1803df08f44-68fb7928970so2866986d6.2
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 01:56:07 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 nf8-20020a0562143b8800b0068fb74ec7c9sm558588qvb.34.2024.02.29.01.56.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 01:56:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c1718812-d6e8-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709200565; x=1709805365; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=I+Jq4VgjlztXV9LuPFURoS9J4NOCZCFBiZ63KHy9B7E=;
        b=A4qVVgyWOmT2UrFCOuIHZ7lDc473YXT1AVc4psgKVylAjFb9QmCCYWZi1VKDguRvq2
         hyYF9bojV3c/PDcGj6Z8lKv65obaJr6CvXFD+Yjcd0lLqG7AmAzMP0SxK9Gw5a9c34FA
         +X60l4n9JXne/klOEczAmrClCfkooWvL5vOCY=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709200565; x=1709805365;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=I+Jq4VgjlztXV9LuPFURoS9J4NOCZCFBiZ63KHy9B7E=;
        b=Ef5gCu+SBEOHh34qoKSqfISVJAclcVN0I/mm+QJzv1tPivHbn/Wia3G9riKy+pLchi
         xzKSEMULCciSfHSN0hnLGiPU8rQmPR24wC8zJRgoQw3dgEnZLjNtpXoxvMYr0Ms0unRW
         i6Zh4vz0IaKYFUVMVhzOddCBBLqIfIaXY2OL6wE3yhgz3tXwn1m/+PEUc2mgAW5JoC9t
         u+FRhCXpLrSd/I+rISSeUd92eDQ1R22qxz7l3SGFuE0/aGGPbre6+ooZ4kRrGYHzRo6a
         REXnnFSYLv8nsMe+lUKPjFIz01o1l9qZjx0UZdhbXoss4348P4EfhNdlk7IoYuHzHzz5
         zJ/w==
X-Gm-Message-State: AOJu0Yx5SPqlLkHZcF0u2XZJNaFNp176n9HKkigA4+bDzX9veDo0veql
	iVxvxvgNx6tjRajcyX6+slpM0KCbeygWko6g5U8AlTHFRXw/uj1qliuuH7Kvb+uYSdO3XSqhQYM
	+
X-Google-Smtp-Source: AGHT+IHpM2bPpu8mHD/WLp7i0gIrB8L7CHvbPXk6xReTWVVWbDqpncTL+Uacrdl0HbI2WS0x4N0k8w==
X-Received: by 2002:a05:6214:32a:b0:68f:fe88:cc4c with SMTP id j10-20020a056214032a00b0068ffe88cc4cmr1794865qvu.41.1709200565411;
        Thu, 29 Feb 2024 01:56:05 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH 2/2] automation: introduce non debug clang based tests
Date: Thu, 29 Feb 2024 10:55:29 +0100
Message-ID: <20240229095529.17723-3-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
In-Reply-To: <20240229095529.17723-1-roger.pau@citrix.com>
References: <20240229095529.17723-1-roger.pau@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The generated code between the debug and release builds can be quite
different, as a note 2ce562b2a413 only manifested in non-debug builds due to
the usage of -O2.

Duplicate the clang based test in order to test with both debug and non-debug
Xen builds.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 automation/gitlab-ci/test.yaml | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 8b7b2e4da92d..dedca794b257 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -422,13 +422,20 @@ qemu-smoke-x86-64-gcc:
   needs:
     - debian-stretch-gcc-debug
 
-qemu-smoke-x86-64-clang:
+qemu-smoke-x86-64-clang-debug:
   extends: .qemu-x86-64
   script:
     - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE}
   needs:
     - debian-bookworm-clang-debug
 
+qemu-smoke-x86-64-clang:
+  extends: .qemu-x86-64
+  script:
+    - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE}
+  needs:
+    - debian-bookworm-clang
+
 qemu-smoke-x86-64-gcc-pvh:
   extends: .qemu-x86-64
   script:
@@ -436,13 +443,20 @@ qemu-smoke-x86-64-gcc-pvh:
   needs:
     - debian-stretch-gcc-debug
 
-qemu-smoke-x86-64-clang-pvh:
+qemu-smoke-x86-64-clang-debug-pvh:
   extends: .qemu-x86-64
   script:
     - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE}
   needs:
     - debian-bookworm-clang-debug
 
+qemu-smoke-x86-64-clang-pvh:
+  extends: .qemu-x86-64
+  script:
+    - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE}
+  needs:
+    - debian-bookworm-clang
+
 qemu-smoke-riscv64-gcc:
   extends: .qemu-riscv64
   script:
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 09:56:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 09:56:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686914.1069527 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfd9O-0008MW-4z; Thu, 29 Feb 2024 09:56:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686914.1069527; Thu, 29 Feb 2024 09:56:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfd9N-0008Lg-VK; Thu, 29 Feb 2024 09:56:25 +0000
Received: by outflank-mailman (input) for mailman id 686914;
 Thu, 29 Feb 2024 09:56:25 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfd9M-0007IO-Vi
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 09:56:24 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id cb959490-d6e8-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 10:56:23 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id CBA3A4EE0737;
 Thu, 29 Feb 2024 10:56:22 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cb959490-d6e8-11ee-a1ee-f123f15fe8a2
MIME-Version: 1.0
Date: Thu, 29 Feb 2024 10:56:22 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Stefano Stabellini <sstabellini@kernel.org>
Cc: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>,
 consulting@bugseng.com, Andrew Cooper <andrew.cooper3@citrix.com>, George
 Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 2/2] xen/cpu: address MISRA C Rule 17.7
In-Reply-To: <alpine.DEB.2.22.394.2402281437470.575685@ubuntu-linux-20-04-desktop>
References: <cover.1708680104.git.nicola.vetrini@bugseng.com>
 <dd4ac0e670a2ad7ecb5eb435e5e3b4b313b1e0b6.1708680104.git.nicola.vetrini@bugseng.com>
 <33342a17-e71c-4752-a16f-da5c0ef77b51@suse.com>
 <alpine.DEB.2.22.394.2402261619210.247676@ubuntu-linux-20-04-desktop>
 <2178731a-ec81-4505-ba8a-2f945bf85133@suse.com>
 <7a8e610e-913e-4a56-8ce1-6dd6abd894f4@xen.org>
 <4bee79ca-7a7e-4bcc-ac97-5a5a57ec2c91@suse.com>
 <alpine.DEB.2.22.394.2402271808410.575685@ubuntu-linux-20-04-desktop>
 <6af04933659178b3ccabc5caf646273c@bugseng.com>
 <bcae31fb-856d-4737-b780-f41b3e24cfa7@xen.org>
 <alpine.DEB.2.22.394.2402281437470.575685@ubuntu-linux-20-04-desktop>
Message-ID: <ee30028d5c53f3b90acad005147b56f1@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-28 23:38, Stefano Stabellini wrote:
> On Wed, 28 Feb 2024, Julien Grall wrote:
>> Hi Nicola,
>> 
>> On 28/02/2024 11:09, Nicola Vetrini wrote:
>> > > I asked Roberto if void casts are an option for compliance.
>> > >
>> >
>> > void casts are an option for sure. The rationale for the rule explicitly
>> > lists them as a compliance mechanism. An interesting aspect is what would be
>> > the consensus around void casts on functions whose return value is always
>> > ignored vs. functions whose return value is sometimes ignored.
>> 
>> If a return is always ignored, then the function should return void. 
>> For the
>> second case, I think it will be on the case by case basis.
> 
> +1
> 

Ok, noted.

> 
>> > > In any case, I don't think we should use void casts in the specific
>> > > cases this patch is dealing with. Void casts (if anything) should be a
>> > > last resort while this patch fixes the issue in a better way.
>> 
>> +1.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 10:07:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 10:07:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686923.1069536 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdJw-0002DO-1k; Thu, 29 Feb 2024 10:07:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686923.1069536; Thu, 29 Feb 2024 10:07:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdJv-0002DH-UR; Thu, 29 Feb 2024 10:07:19 +0000
Received: by outflank-mailman (input) for mailman id 686923;
 Thu, 29 Feb 2024 10:07:18 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aH8z=KG=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rfdJu-0002DB-LY
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 10:07:18 +0000
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam10on20601.outbound.protection.outlook.com
 [2a01:111:f403:2412::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 500a51e9-d6ea-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 11:07:16 +0100 (CET)
Received: from PR1P264CA0094.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:345::19)
 by CH2PR12MB4055.namprd12.prod.outlook.com (2603:10b6:610:78::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.41; Thu, 29 Feb
 2024 10:07:10 +0000
Received: from SA2PEPF000015CB.namprd03.prod.outlook.com
 (2603:10a6:102:345:cafe::44) by PR1P264CA0094.outlook.office365.com
 (2603:10a6:102:345::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.31 via Frontend
 Transport; Thu, 29 Feb 2024 10:07:09 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SA2PEPF000015CB.mail.protection.outlook.com (10.167.241.201) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 29 Feb 2024 10:07:08 +0000
Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 29 Feb
 2024 04:07:07 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com
 (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 29 Feb
 2024 04:07:07 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 29 Feb 2024 04:07:06 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 500a51e9-d6ea-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=OLHOJfkeThvy+wjs4tOSNKWnp5XJhKR5svN6Gm58zE3XbkTjTmo/MiG4vTe8cyRLF1xnTddxeH5sLGnKB0YRRJUF9w7+9mT8fwXW9npqDJciFI19BMPB4yaLCYrQ5sA2h9ckA9j3SPv0ZtkHp9b2JTN6yh8QSVg3xj8BarP3krhGCKGRYjpdWRsrH+Xaahaj/zlGLC0lXRXJ1k67aO/IQK6llXko2KbSS3/kO9b8rO+nVZXLZ+8vy8DaVXY48ELNofcdWTX48fksTQgGA0l2X1OUGEPbX/q2/P1HA6maYceXXjSo0cKj6hB8/WBFANa9CRgh7ZbZfILXpyrzMPW7bA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=aM7XNxdFsKX0tite25WaThEAVbSQuAhgx4TeZHguDqQ=;
 b=B0JiKpFaJZCbWpeQYgC1KfmOa5UeRzFDFjbezova2CnGSgIVhVUODrnvtqrEyS1qxZKt4UUtPe+ICMyAC+uT7RN3xXVs8P4MsALogTs10kO8GAPjlM3+f3gBko8NB35iVpEpn0BqSk0NCgrAHFHVPKLrhxnOxQIMOpuJc3BxftT6/78CGaZj+fkP2ldOAt428UL+6i2IxV35dsgd/goYJvGiPuohrNN0Agnh1nuwqlGoke4MJbS3YGAlbdehlJS7mV07hTXSHSV89QqdQvDyfu33X6crxFAFvZ9Bw0F2wqAK5CIAf/5l4sLo4Q0vcEXzOr+1LOcz0bPDVko/oRMcOQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=aM7XNxdFsKX0tite25WaThEAVbSQuAhgx4TeZHguDqQ=;
 b=Y2X0E3TQ0CTQB78boxTcLfJY5zsIQGGNfIWmfNecDGTuogxMK02pvF8H4N0rcLeGlm3tHNd8F1256rgeoW8sN95SBRFjWD1VuB77VbZwkLp44NCqoI2sCP2icM0Sd/OsFFRrOEi3I8ukZ8s6jTsCq42R9m7FLFV1ns66LFNmPSY=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <8228a48f-566c-4931-9f24-27bdaa1f0f28@amd.com>
Date: Thu, 29 Feb 2024 11:07:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Fix arm32 build failure when early printk is
 enabled
To: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>, "Bertrand
 Marquis" <bertrand.marquis@arm.com>, Volodymyr Babchuk
	<Volodymyr_Babchuk@epam.com>
References: <20240228103555.172101-1-michal.orzel@amd.com>
 <9201bdc5-2e4c-48b7-8c92-f5063fde90b6@xen.org>
 <af11620a-ec16-4677-b968-d8e36c4555d2@amd.com>
 <alpine.DEB.2.22.394.2402281425260.575685@ubuntu-linux-20-04-desktop>
Content-Language: en-US
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <alpine.DEB.2.22.394.2402281425260.575685@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB05.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF000015CB:EE_|CH2PR12MB4055:EE_
X-MS-Office365-Filtering-Correlation-Id: 81f105e1-e218-4d5c-feea-08dc390e304f
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	iAyQt6fZqSO2ZY/iEq3bmmjQcUBtZdvVkEUoEoucgN1W/D6xIXG8lASsAc8m+xGKCnhtlmlZKzoJGO5Ll9ldpwhUBiDMvF4BfJWE7KeDTEhIpOvRC/YEV5U0jPRlZAI4ovY6OLqbDkW8bYeq0vnADIFtqhqs1B88/8TPwbrKsax44/A8WJRAPD98xfz+LXnbfAPeBitDlPCRKwVxtHYYLIuOaLoFT42j2j9Ru2GiLqFi8vAqNeK7peH7atWiGCGIM8JhvDtniQ4qWmUGZwE6dOX2/e/fisbDWpvynkEsAOssc+IKdlaTj5o0Y1yBnL5hrgiXKfg5ST7ciAHsTUlwT8qhkXCHzl1pTawHeGjJIDZt4TxuL6GpB9XeQx5VOGMBN3sqyfiL7f2wsPt1cGVbP0xDYPerUpsmRWx4CPyeiw3+YjIChL9WKuoPZumVHRZJGF0xA/eNcOWjbGeZhQ1pfYxnTPLfBDV/dJSt/Wux+pimzCSaHVbC/T05lZgppMwloU2nz0a1OinJPcuifLall3/wPML6PCy/ayHmX9boAftusDeOsNa89LXwVOciVn4EyyqvvRRIm+OeqhZB/uhMpS/X2JRSkNO/IgcRII9Naehhye7kXhsEYQz70A3NZXkvgD5Mq64NfNHFxoCdo3lre7iwTgZ3ASHp8JJp0x/6FzWJPYLQ/DCxT9QCtRkGJNSeOrQDqZSUewirq1j3VXJxN1tQpnmLXjzgKnVfvyrTrpaHBtD5ybflirTZDSeUv/c8
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400014)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 10:07:08.2017
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 81f105e1-e218-4d5c-feea-08dc390e304f
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SA2PEPF000015CB.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4055



On 28/02/2024 23:27, Stefano Stabellini wrote:
> 
> 
> On Wed, 28 Feb 2024, Michal Orzel wrote:
>> Hi Julien,
>>
>> On 28/02/2024 12:42, Julien Grall wrote:
>>>
>>>
>>> Hi Michal,
>>>
>>> On 28/02/2024 10:35, Michal Orzel wrote:
>>>> Commit 0441c3acc7e9 forgot to rename FIXMAP_CONSOLE to FIX_CONSOLE in
>>>> TEMPORARY_EARLY_UART_VIRTUAL_ADDRESS macro. This results in a build
>>>> failure on arm32, when early printk is enabled:
>>>> arch/arm/arm32/mmu/head.S:311: Error: invalid operands (*UND* and *ABS* sections) for `*'
>>>
>>> Good catch! Somewhat related I wonder whether we should add earlyprintk
>>> testing in gitlab?
>> I thought about adding this and I think we should at least have build jobs (hypervisor only, no toolstack)
>> selecting early printk. When it comes to testing if early printk works, I'm not sure. It'd be nice
>> but FWIR we have limited bandwidth.
>>
>> @Stefano, what's your opinion?
> 
> I think it would be a good and quick test to have. To save testing
> bandwidth I think we should reduce the amount of debug/non-debug
> variations of the same tests that we have.
Yes, I suggested that some time ago. We could keep both versions for generic tests,
but remove the non-debug version (unless you prefer to do the opposite) for:
Arm64:
- staticmem
- staticheap
- shared-mem
- boot-cpupools
Arm32:
- gzip
- without-dom0

This would save us some bandwidth that we could use for testing other features (e.g. early printk).

~Michal



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 10:11:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 10:11:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686925.1069546 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdNZ-0003kX-GG; Thu, 29 Feb 2024 10:11:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686925.1069546; Thu, 29 Feb 2024 10:11:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdNZ-0003kQ-D2; Thu, 29 Feb 2024 10:11:05 +0000
Received: by outflank-mailman (input) for mailman id 686925;
 Thu, 29 Feb 2024 10:11:04 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfdNY-0003kK-FD
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 10:11:04 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfdNV-0000nA-CB; Thu, 29 Feb 2024 10:11:01 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfdNV-00013q-6d; Thu, 29 Feb 2024 10:11:01 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=uWACTYlld6o6URRybhcGiwQ1PW6iTyXiVTWdw714I/8=; b=UBW4EbDnygSuhjl10+ULICtncE
	qhoa9rJ9ovKKwnvcLhEbiF5q5vQ7EHSECn6uA/b/UlfjOuDANoexeRuNNCdUey3AFNDXRMLF+qKKc
	ziT21KeIuDwF5Fep16KOcq82xFQNoFC8mA0EFYCu4McT7F3Y2MOyWVUodBM7zB025qLY=;
Message-ID: <99b74dda-225b-4402-abb9-fcccfda0adbc@xen.org>
Date: Thu, 29 Feb 2024 10:10:58 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Fix arm32 build failure when early printk is
 enabled
Content-Language: en-GB
To: Michal Orzel <michal.orzel@amd.com>,
 Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org,
 Bertrand Marquis <bertrand.marquis@arm.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240228103555.172101-1-michal.orzel@amd.com>
 <9201bdc5-2e4c-48b7-8c92-f5063fde90b6@xen.org>
 <af11620a-ec16-4677-b968-d8e36c4555d2@amd.com>
 <alpine.DEB.2.22.394.2402281425260.575685@ubuntu-linux-20-04-desktop>
 <8228a48f-566c-4931-9f24-27bdaa1f0f28@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <8228a48f-566c-4931-9f24-27bdaa1f0f28@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 29/02/2024 10:07, Michal Orzel wrote:
> 
> 
> On 28/02/2024 23:27, Stefano Stabellini wrote:
>>
>>
>> On Wed, 28 Feb 2024, Michal Orzel wrote:
>>> Hi Julien,
>>>
>>> On 28/02/2024 12:42, Julien Grall wrote:
>>>>
>>>>
>>>> Hi Michal,
>>>>
>>>> On 28/02/2024 10:35, Michal Orzel wrote:
>>>>> Commit 0441c3acc7e9 forgot to rename FIXMAP_CONSOLE to FIX_CONSOLE in
>>>>> TEMPORARY_EARLY_UART_VIRTUAL_ADDRESS macro. This results in a build
>>>>> failure on arm32, when early printk is enabled:
>>>>> arch/arm/arm32/mmu/head.S:311: Error: invalid operands (*UND* and *ABS* sections) for `*'
>>>>
>>>> Good catch! Somewhat related I wonder whether we should add earlyprintk
>>>> testing in gitlab?
>>> I thought about adding this and I think we should at least have build jobs (hypervisor only, no toolstack)
>>> selecting early printk. When it comes to testing if early printk works, I'm not sure. It'd be nice
>>> but FWIR we have limited bandwidth.
>>>
>>> @Stefano, what's your opinion?
>>
>> I think it would be a good and quick test to have. To save testing
>> bandwidth I think we should reduce the amount of debug/non-debug
>> variations of the same tests that we have.
> Yes, I suggested that some time ago. We could keep both versions for generic tests,
> but remove the non-debug version (unless you prefer to do the opposite) for:

I think it makes sense during development window to use the debug 
version. However, I think we want some non-debug testing during the 
hardening phase.

Can gitlab read CONFIG_DEBUG from Config.mk?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 10:23:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 10:23:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686932.1069555 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdZj-00066z-H2; Thu, 29 Feb 2024 10:23:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686932.1069555; Thu, 29 Feb 2024 10:23:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdZj-00066s-EY; Thu, 29 Feb 2024 10:23:39 +0000
Received: by outflank-mailman (input) for mailman id 686932;
 Thu, 29 Feb 2024 10:23:38 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfdZi-00066m-RI
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 10:23:38 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfdZh-00011S-8f; Thu, 29 Feb 2024 10:23:37 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfdZh-0002xv-2L; Thu, 29 Feb 2024 10:23:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=VlKEEr7nzJ/vC+nohN07hS03u5u0GKXZwV5f7Ew6S0A=; b=ayeEl4vGB4m4Xj4E9l4llzVVhH
	kvg/yKbC7XX0JSmnmOmXNfHrt5j0rwYVQl5CYVHFKgBKJc/A68enMfxDuv5IS9LAfG98HXotwc7cA
	1cKMwntwq2sIfLsK98L1z6cP+Dq3lGZsA/6todYA2aRRwGec/8sfH3R+lQG7ZydQcb2U=;
Message-ID: <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
Date: Thu, 29 Feb 2024 10:23:35 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 29/02/2024 07:58, Jan Beulich wrote:
> On 28.02.2024 23:58, Julien Grall wrote:
>> On 27/02/2024 07:55, Jan Beulich wrote:
>>> On 26.02.2024 18:39, Oleksii Kurochko wrote:
>>>> This patch doesn't represent a strict lower bound for GCC and
>>>> GNU Binutils; rather, these versions are specifically employed by
>>>> the Xen RISC-V container and are anticipated to undergo continuous
>>>> testing.
>>>
>>> Up and until that container would be updated to a newer gcc. I'm
>>> afraid I view this as too weak a criteria,
>>
>> I disagree. We have to decide a limit at some point. It is sensible to
>> say that we are only supporting what we can tests. AFAIK, this is what
>> QEMU has been doing.
> 
> I view qemu as a particularly bad example. They raise their baselines
> far too aggressively for my taste.

AFAICT, the decision was based on the supported distros at the time. 
Which makes sense to me (even though I got recently caught because of 
this check). They also seem to be open to relax the check if there are 
any use cases.

Why would we want to support build Xen on non-supported distros?

> 
>>> IOW it is hard for me to see why RISC-V needs stronger restrictions here
>>> than other architectures. It ought to be possible to determine a baseline
>>> version. Even if taking the desire to have "pause" available as a
>>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
>>
>> I think we want to bump it on Arm. There are zero reasons to try to keep
>> a lower versions if nobody tests/use it in production.
>>
>> I would suggest to do the same on x86. What's the point of try to
>> support Xen with a 15+ years old compiler?
> 
> It could have long been bumped if only a proper scheme to follow for
> this and future bumping would have been put forward by anyone keen on
> such bumping, like - see his reply - e.g. Andrew. You may recall that
> this was discussed more than once on meetings, with no real outcome.
> I'm personally not meaning to stand in the way of such bumping as long
> as it's done in a predictable manner, but I'm not keen on doing so and
> hence I don't view it as my obligation to try to invent a reasonable
> scheme. (My personal view is that basic functionality should be
> possible to have virtually everywhere, whereas for advanced stuff it
> is fine to require a more modern tool chain.)

That's one way to see it. The problem with this statement is a user 
today is mislead to think you can build Xen with any GCC versions since 
4.1. I don't believe we can guarantee that and we are exposing our users 
to unnecessary risk.

In addition to that, I agree with Andrew. This is preventing us to 
improve our code base and we have to carry hacks for older compilers.

> 
> The one additional concern I've raised in the past is that in the end
> it's not just minimal tool chain versions we rely on, but also other
> core system tools (see the recent move from "which" to "command -v"
> for an example of such a dependency, where luckily it turned out to
> not be an issue that the -v had only become a standard thing at some
> point). While for the tool chain I can arrange for making newer
> versions available, for core system tools I can't.

I agree we probably want to clarify the minimum version of the 
coretools. However, I think we need to separate the two. Otherwise, we 
will be forever in the statu quo on x86.

> Therefore being too
> eager there would mean I can't really / easily (smoke) test Xen
> anymore on ancient hardware every once in a while. When afaict we do
> too little of such testing already anyway, despite not having any
> lower bound on hardware that formally we support running Xen on.

Can you provide more details of what you mean by "ancient"?

> (And
> no, upgrading the ancient distros on that ancient hardware is not an
> option for me.)

May I ask why? Is it because newer distros don't support your HW?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 10:31:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 10:31:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686936.1069565 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdhL-0007av-8M; Thu, 29 Feb 2024 10:31:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686936.1069565; Thu, 29 Feb 2024 10:31:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdhL-0007ao-5r; Thu, 29 Feb 2024 10:31:31 +0000
Received: by outflank-mailman (input) for mailman id 686936;
 Thu, 29 Feb 2024 10:31:30 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfdhK-0007ad-Dr; Thu, 29 Feb 2024 10:31:30 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfdhK-0001Bi-AU; Thu, 29 Feb 2024 10:31:30 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfdhJ-0006G9-Vi; Thu, 29 Feb 2024 10:31:30 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfdhJ-0007x1-VI; Thu, 29 Feb 2024 10:31:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=dxIDtgwBIcmVEkK87cp4dQfZLn9Pklq6Gn53wXhX4nk=; b=jCCUSu6aAKNmc1kaASQygG6E0L
	G567e/7W1Ax1QN8j1NiEO5+/MrL812LHpTr13lqGHS+gf/emxqrEf+Hs5IDrAk7E08BNZAin/XKf2
	8Ih0rh8+qgAhnc8+0YRmz+8AynCOMk6sj+2pdpsB2kC70578IOhGuORT+iJFL/9PfInk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184817-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184817: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=c20850540ad6a32f4fc17bde9b01c92b0df18bf0
X-Osstest-Versions-That:
    xen=54fd7b997470e6686667ca8e18f9ba6139efcdea
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 29 Feb 2024 10:31:29 +0000

flight 184817 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184817/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  c20850540ad6a32f4fc17bde9b01c92b0df18bf0
baseline version:
 xen                  54fd7b997470e6686667ca8e18f9ba6139efcdea

Last test of basis   184810  2024-02-28 19:02:05 Z    0 days
Testing same since   184817  2024-02-29 08:00:28 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Anthony PERARD <anthony.perard@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jason Andryuk <jandryuk@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Simone Ballarin <simone.ballarin@bugseng.com>
  Tamas K Lengyel <tamas@tklengyel.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   54fd7b9974..c20850540a  c20850540ad6a32f4fc17bde9b01c92b0df18bf0 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 10:34:06 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 10:34:06 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686940.1069576 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdjo-00006h-Mt; Thu, 29 Feb 2024 10:34:04 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686940.1069576; Thu, 29 Feb 2024 10:34:04 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdjo-00006a-JA; Thu, 29 Feb 2024 10:34:04 +0000
Received: by outflank-mailman (input) for mailman id 686940;
 Thu, 29 Feb 2024 10:34:03 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=r6LD=KG=thalesgroup.com=dominique.larrieu@srs-se1.protection.inumbo.net>)
 id 1rfdjm-00006U-V7
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 10:34:03 +0000
Received: from esa.hc1631-21.eu.iphmx.com (esa.hc1631-21.eu.iphmx.com
 [23.90.122.185]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 0c56844f-d6ee-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 11:34:00 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0c56844f-d6ee-11ee-a1ee-f123f15fe8a2
X-CSE-ConnectionGUID: q+8AHSfxTuC1nbWXnvVspg==
X-CSE-MsgGUID: Bn8yrphaSSGXuKhmSjbiCg==
Authentication-Results: ob1.hc1631-21.eu.iphmx.com; dkim=pass (signature verified) header.i=@thalesgroup.com
X-IronPort-AV: E=McAfee;i="6600,9927,10998"; a="10261770"
X-IronPort-AV: E=Sophos;i="6.06,194,1705359600"; 
   d="scan'208,217";a="10261770"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=thalesgroup.com; i=@thalesgroup.com; s=bbmfo20230504;
  t=1709202839;
  h=from:to:cc:subject:date:message-id:references:
   in-reply-to:mime-version;
  bh=nmT2jA0pwzkjRVk3wbtLpkBIt5do8afyGM2uaNplqPQ=;
  b=fJFdWgWBVn55jOIDx7uiDnYY7FqF82XE4LH1NOadAtRA5w/TK0XHJDly
   8C4Va5Bzd2gJ8+nujYTc4t6O7I9Dw8aqgpaojTAcb+jy8erAwagD2Qnwr
   SVT+j9n0UEvhyID+lmoAPutgvdMZh7eZX5RhB69uE+EyHX6KpvVA4EQDz
   0/abQtyAKcjmIGzc/vTOlqDXmH2+VTRYqB0SzOFNS0UW7XFlYbBKKZA3O
   et8diYYhzKSzMG5r8HSNMzMC2UYGObGJbQE0AR0frZseJS4a51mgvPtIj
   dMUZICAX1QPws2FTwI3gZBvwqpAENb8nFZ5xupcqphxNAEfLm4yACo9fM
   A==;
X-CSE-ConnectionGUID: tx5r8i3gT0GHq/kQ1s1lAA==
X-CSE-MsgGUID: 1AN6wz0JRrqfSaAwEvDOkA==
X-CSE-ConnectionGUID: DRQMvSU+SO+mh9RCzlWQ/w==
X-CSE-MsgGUID: kdQhD2ufSamb/5RbpPBS9w==
X-IronPort-AV: E=McAfee;i="6600,9927,10998"; a="18919783"
X-IronPort-AV: E=Sophos;i="6.06,194,1705359600"; 
   d="scan'208,217";a="18919783"
From: LARRIEU Dominique <dominique.larrieu@thalesgroup.com>
To: Julien Grall <julien@xen.org>
CC: GOURLOT Francois <francois.gourlot@thalesgroup.com>, GRUO Nicolas
	<nicolas-n.gruo@thalesgroup.com>, Cc <xen-devel@lists.xenproject.org>, "Kelly
 Choi" <kelly.choi@cloud.com>, Jan Beulich <jbeulich@suse.com>,
	=?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= <roger.pau@citrix.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Juergen Gross <jgross@suse.com>, WILLEMS Louis
	<louis.willems@thalesgroup.com>
Subject: RE: IMPORTANT - : Need help on USB port virtualization with Xen
 hypervisor
Thread-Topic: IMPORTANT - : Need help on USB port virtualization with Xen
 hypervisor
Thread-Index: AdpmcWIlf+ro0u5NSE2ZDRGqfQbNXAArMvkAAFwqbLAAdf8JUAAgWMTwAAKsSXA=
Sensitivity: company-confidential
Date: Thu, 29 Feb 2024 10:33:57 +0000
Message-ID: <3ee3659afff645cabed86bcc22c44686@thalesgroup.com>
References: <d2de4ae9ecb34efc962dea7f8b4e7cbd@thalesgroup.com>
 <bb2485e5-7818-435b-8d9c-dda88100979f@xen.org>
 <a46ac2e14fa2410eafc26a37a00a442f@thalesgroup.com>
 <427d2ecb865648b7a459c592c208c0be@thalesgroup.com>
 <673b2bc630d748e8af0a15d4b553906e@thalesgroup.com>
In-Reply-To: <673b2bc630d748e8af0a15d4b553906e@thalesgroup.com>
Accept-Language: fr-FR, en-US
Content-Language: fr-FR
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-nodisclaimer: 0
Content-Type: multipart/alternative;
	boundary="_000_3ee3659afff645cabed86bcc22c44686thalesgroupcom_"
MIME-Version: 1.0

--_000_3ee3659afff645cabed86bcc22c44686thalesgroupcom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

SGkgSnVsaWVuLA0KDQpUaGFuayB5b3UgZm9yIHlvdXIgcXVpY2sgYW5zd2VyLg0KDQpZb3Ugd2ls
bCBmaW5kIGJlbG93IG91ciBhbnN3ZXJzIChpbiByZWQpIHRvIHlvdXIgcXVlc3Rpb25zLg0KDQpU
byBzdW1tYXJpemUgb3VyIHJlcXVlc3QgOiB3aGF0IHdlIHdvdWxkIGxpa2UgaXMgdG8gdXNlIFVT
QiAzLjAgZHJpdmVyIHdpdGggaGlnaCBzcGVlZCBjb25maWd1cmF0aW9uLg0KVG9kYXksIGl0IGlz
IG5vdCBwb3NzaWJsZSB0byBkbyB0aGF0Lg0KVGhlIGRyaXZlciBzdGF5IGluIGZ1bGwgc3BlZWQg
bW9kZSwgYW5kIG1vcmUgb2Z0ZW4gaW4gVVNCIDEuMCBvciAyLjAgY29uZmlndXJhdGlvbi4NCg0K
SXMgaXQgcG9zc2libGUgdG8gY29uZmlndXJlIHRoZSBYRU4gVVNCIDMuMCBkcml2ZXIgd2l0aCB0
aGUgaGlnaCBzcGVlZCBtb2RlID8NCihyZWFkIGluIG91ciBhbnN3ZXJzIGJlbG93IHRoZSByZWZl
cmVuY2UgdG8gdGhlIGRyaXZlcikNCg0KQmVzdCByZWdhcmRzLA0KRG9taW5pcXVlDQoNCg0KDQoN
Ci0tLS0tTWVzc2FnZSBkJ29yaWdpbmUtLS0tLQ0KDQpEZSA6IEp1bGllbiBHcmFsbCA8anVsaWVu
QHhlbi5vcmc8bWFpbHRvOmp1bGllbkB4ZW4ub3JnPj4NCg0KRW52b3nDqSA6IHNhbWVkaSAyNCBm
w6l2cmllciAyMDI0IDE0OjM3DQoNCsOAIDogR09VUkxPVCBGcmFuY29pcyA8ZnJhbmNvaXMuZ291
cmxvdEB0aGFsZXNncm91cC5jb208bWFpbHRvOmZyYW5jb2lzLmdvdXJsb3RAdGhhbGVzZ3JvdXAu
Y29tPj47IHhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZzxtYWlsdG86eGVuLWRldmVsQGxp
c3RzLnhlbnByb2plY3Qub3JnPiBDYyA6IEtlbGx5IENob2kgPGtlbGx5LmNob2lAY2xvdWQuY29t
PG1haWx0bzprZWxseS5jaG9pQGNsb3VkLmNvbT4+OyBKYW4gQmV1bGljaCA8amJldWxpY2hAc3Vz
ZS5jb208bWFpbHRvOmpiZXVsaWNoQHN1c2UuY29tPj47IFJvZ2VyIFBhdSBNb25uw6kgPHJvZ2Vy
LnBhdUBjaXRyaXguY29tPG1haWx0bzpyb2dlci5wYXVAY2l0cml4LmNvbT4+OyBBbmRyZXcgQ29v
cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPG1haWx0bzphbmRyZXcuY29vcGVyM0BjaXRy
aXguY29tPj47IEdlb3JnZSBEdW5sYXAgPGdlb3JnZS5kdW5sYXBAY2l0cml4LmNvbTxtYWlsdG86
Z2VvcmdlLmR1bmxhcEBjaXRyaXguY29tPj47IEphbiBCZXVsaWNoIDxqYmV1bGljaEBzdXNlLmNv
bTxtYWlsdG86amJldWxpY2hAc3VzZS5jb20+PjsgSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2Uu
Y29tPG1haWx0bzpqZ3Jvc3NAc3VzZS5jb20+PiBPYmpldCA6IFJlOiBJTVBPUlRBTlQgLSA6IE5l
ZWQgaGVscCBvbiBVU0IgcG9ydCB2aXJ0dWFsaXphdGlvbiB3aXRoIFhlbiBoeXBlcnZpc29yDQoN
Cg0KDQpIaSBEb21pbmlxdWUgYW5kIEZyYW5jb2lzLA0KDQoNCg0KVGhhbmsgeW91IGZvciB0aGUg
cmVwb3J0LiBJIGFtIENDaW5nIHNvbWUgcGVyc29uIHdobyBJIHRoaW5rIGNvdWxkIGhlbHAuDQoN
Cg0KDQpJIGFsc28gaGF2ZSBzb21lIHF1ZXN0aW9ucyAoc2VlIGJlbG93KS4NCg0KDQoNCk9uIDIz
LzAyLzIwMjQgMTY6MDAsIEdPVVJMT1QgRnJhbmNvaXMgd3JvdGU6DQoNCj4gRGVhciBBbGwsDQoN
Cj4NCg0KPiBXZSBzZW5kIHlvdSBhIG1lc3NhZ2UgZmV3IGRheXMgYWdvLg0KDQo+DQoNCj4gV2Ug
aGF2ZSBtYWpvciBwZXJmb3JtYW5jZSB3aXRoIFhFTiBVU0IgRHJpdmVycy4gV2UgdXNlIHlvdXIg
ZHJpdmVyIHRvDQoNCj4gbG9hZCBkYXRhIGluIGEgZGV2aWNlIGFuZCB3ZSBuZWVkIFVTQjMgSGln
aCBTcGVlZCBVU0IgcHJvdG9jb2xlDQoNCj4NCg0KPiBEbyB5b3UgaGF2ZSBhIG5ldyBpc3N1ZSBv
ZiB0aGUgSENJIGFuZCBORUMgVVNCIFhlbiBEcml2ZXIgPw0KDQoNCg0KWy4uLl0NCg0KDQoNCj4g
KkRlIDoqTEFSUklFVSBEb21pbmlxdWUgPGRvbWluaXF1ZS5sYXJyaWV1QHRoYWxlc2dyb3VwLmNv
bTxtYWlsdG86ZG9taW5pcXVlLmxhcnJpZXVAdGhhbGVzZ3JvdXAuY29tPj4NCg0KPiAqRW52b3nD
qSA6KiBqZXVkaSAyMiBmw6l2cmllciAyMDI0IDEzOjU4ICrDgCA6Kg0KDQo+IHhlbi1kZXZlbEBs
aXN0cy54ZW5wcm9qZWN0Lm9yZzxtYWlsdG86eGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3Jn
PiAqQ2MgOiogV0lMTEVNUyBMb3Vpcw0KDQo+IDxsb3Vpcy53aWxsZW1zQHRoYWxlc2dyb3VwLmNv
bTxtYWlsdG86bG91aXMud2lsbGVtc0B0aGFsZXNncm91cC5jb20+PjsgR1JVTyBOaWNvbGFzDQoN
Cj4gPG5pY29sYXMtbi5ncnVvQHRoYWxlc2dyb3VwLmNvbTxtYWlsdG86bmljb2xhcy1uLmdydW9A
dGhhbGVzZ3JvdXAuY29tPj47IEdPVVJMT1QgRnJhbmNvaXMNCg0KPiA8ZnJhbmNvaXMuZ291cmxv
dEB0aGFsZXNncm91cC5jb208bWFpbHRvOmZyYW5jb2lzLmdvdXJsb3RAdGhhbGVzZ3JvdXAuY29t
Pj47IEtlbGx5IENob2kgPGtlbGx5LmNob2lAY2xvdWQuY29tPG1haWx0bzprZWxseS5jaG9pQGNs
b3VkLmNvbT4+DQoNCj4gKk9iamV0IDoqIE5lZWQgaGVscCBvbiBVU0IgcG9ydCB2aXJ0dWFsaXph
dGlvbiB3aXRoIFhlbiBoeXBlcnZpc29yDQoNCj4gKkNyaXTDqHJlIGRlIGRpZmZ1c2lvbiA6KiBD
b25maWRlbnRpZWwNCg0KPg0KDQo+ICAgICBEZWFyIGFsbCwNCg0KPg0KDQo+ICAgICBXZSBhcmUg
ZGV0ZWN0aW5nIHNldmVyYWwgaXNzdWVzIHdpdGggVVNCIHBvcnQgdmlydHVhbGl6YXRpb24gd2l0
aCB0aGUNCg0KPiAgICAgWGVuIGh5cGVydmlzb3IuDQoNCg0KDQpKdXN0IHRvIGNsYXJpZnksIHlv
dSBhcmUgYXNzaWduaW5nIHRoZSBQQ0kgVVNCIGJ1cyAocmF0aGVyIHRoYW4ganVzdCB0aGUgVVNC
IGRldmljZSkgdG8gdGhlIGd1ZXN0LiBJcyB0aGF0IGNvcnJlY3Q/DQoNCg0KDQpJZiBzby4uLg0K
DQoNCg0KW1RoYWxlc10gOiB3ZSBhcmUgdXNpbmcgdGhlIFVTQiBjaGlwc2V0IG9mIHRoZSBtb3Ro
ZXIgYm9hcmQgKHNlZSBiZWxvdyB0aGUgcmVmZXJlbmNlcykNCg0KDQoNCj4NCg0KPiBEbyB5b3Ug
aGF2ZSBhIG5ldyBpc3N1ZSBvZiB0aGUgSENJIGFuZCBORUMgVVNCIFhlbiBEcml2ZXIgPw0KDQoN
Cg0KLi4uIEkgYW0gbm90IHN1cmUgd2hhdCBYZW4gZHJpdmVyIHlvdSBhcmUgdGFsa2luZyBhYm91
dC4gSWYgdGhlIFVTQiBidXMgaXMgYXNzaWduZWQgdG8gdGhlIGd1ZXN0LCB0aGVuIEkgd291bGQg
ZXhwZWN0IG5vIHNwZWNpZmljIFhlbiBkcml2ZXIgYXJlIHJlcXVpcmVkLiBEbyB5b3UgaGF2ZSBh
bnkgcG9pbnRlciB0byB0aGUgZXhpc3Rpbmcgb25lPw0KDQoNCg0KW1RoYWxlc10gOiBzZWUgcmVm
ZXJlbmNlcyB0byBIVyBhbmQgU1cgYmVsb3cNCg0KDQoNCj4gICAgIC0gV2UgY2Fubm90IGRvIFBD
SSBwYXNzdGhyb3VnaCBvZiB0aGUgUENJIHVzYiBidXMgb24gYSBXaW5kb3dzIDEwDQoNCj4gICAg
IDE2MDcgNjQtYml0IHZpcnR1YWwgbWFjaGluZS4gVGhlIGJhZCByZXN1bHQgaXMgYSBXaW5kb3dz
IGJsdWUgc2NyZWVuLg0KDQo+DQoNCj4gICAgIC0gV2hlbiB3ZSB1c2UgdGhlIHBhc3N0aHJvdWdo
IGZ1bmN0aW9uYWxpdHkgb24gYSBXaW5kb3dzIDIxSDINCg0KPiAgICAgdmlydHVhbCBtYWNoaW5l
LCB3ZSBub3RpY2UgdGhhdHRoZSBzcGVlZCBvZiB0aGUgVVNCIHBvcnQgaXMgbm90IGhpZ2gNCg0K
PiAgICAgc3BlZWQgYnV0IGZ1bGwgc3BlZWQgb24gYSBVU0IgMy4wIHBvcnQNCg0KPg0KDQo+ICAg
ICAtIFdlIG5vdGljZSBpbnN0YWJpbGl0aWVzIHdoZW4gdXNpbmcgdGhlIG5lYy11c2IteGhjaSBk
cml2ZXIsICBVU0INCg0KPiAgICAgMi4wIGtleXMgYXJlIG5vdCByZWNvZ25pemVkIGJ5IHRoZSBX
aW5kb3dzIHZpcnR1YWwgbWFjaGluZQ0KDQo+ICAgICAoaW5jb3JyZWN0IGRlc2NyaXB0b3IpDQoN
Cg0KDQpEbyB5b3UgaGF2ZSBhbnkgbG9ncyB5b3UgY291bGQgc2hhcmUgZnJvbSB0aGUgZ3Vlc3Qg
T1M/IEFsc28sIHdvdWxkIHlvdSBiZSBhYmxlIHRvIHNoYXJlIG1vcmUgZGV0YWlscyBvbiBob3cg
eW91IGNvbmZpZ3VyZSB0aGUgZ3Vlc3Q/IChlLmcuIHhsIGNvbmZpZ3VyYXRpb24sIGxpYnZpcnQu
Li4pPw0KDQoNCg0KPg0KDQo+ICAgICBXZSBuZWVkeW91ciBoZWxwdG8gZmluZCBhIHNvbHV0aW9u
IGZvciB0aGVzZSBwcm9ibGVtcy4NCg0KPg0KDQo+ICAgICBUaGUgU29mdHdhcmUgdXNlZCBhcmUg
Og0KDQoNCg0KQ2FuIHlvdSBzaGFyZSBzb21lIGRldGFpbHMgb2YgdGhlIEhXIHlvdSBhcmUgdXNp
bmcgPw0KDQoNCg0KW1RoYWxlc10gOiBIVyA6DQoNCltUaGFsZXNdIDogV2UgYXJlIHVzaW5nIGEg
R0VUQUMgWDUwMCB1c2luZyB0aGUgTW9iaWxlIEludGVsIFFNMTc1IENoaXBzZXQNCg0KW1RoYWxl
c10gOiBVU0IgY29udHJvbGxlcjogSW50ZWwgQ29ycG9yYXRpb24gMTAwIFNlcmllcy9DMjMwIFNl
cmllcyBDaGlwc2V0IEZhbWlseSBVU0IgMy4wIHhIQ0kgQ29udHJvbGxlciAocmV2IDMxKQ0KDQpb
VGhhbGVzXSA6IFN1YnN5c3RlbTogTWl0YWMgMTAwIFNlcmllcy9DMjMwIFNlcmllcyBDaGlwc2V0
IEZhbWlseSBVU0IgMy4wIHhIQ0kgQ29udHJvbGxlcg0KDQoNCg0KW1RoYWxlc10gOiBTVyA6DQoN
CltUaGFsZXNdIDogS2VybmVsIGRyaXZlciBpbiB1c2U6IHhoY2lfaGNkDQoNCltUaGFsZXNdIDog
S2VybmVsIG1vZHVsZXMgOiB4aGNpX3BjaQ0KDQoNCg0KPg0KDQo+ICAgICAtRGViaWFuIDExIHZl
cnNpb241LjEwLjAtMjANCg0KPg0KDQo+ICAgICAtWGVuIHZlcnNpb240LjE0DQoNCg0KDQpBcmUg
bWlub3IgdmVyc2lvbiBvZiBYZW4gNC4xNCBhcmUgeW91IHVzaW5nID8gQW55IHBhdGNoZXMgb24g
dG9wID8NCg0KDQoNClhlbiA0LjE0IGhhcyBhbHNvIGJlZW4gb3V0IG9mIHN1cHBvcnQgZm9yIGEg
ZmV3IG1vbnRocyBub3cuIElmIHRoYXTigJlzIHBvc3NpYmxlLCBJIHdvdWxkIHJlY29tbWVuZCB0
byB0cnkgYSBuZXcgWGVuIHZlcnNpb24gKHRoZSBsYXN0IHN0YWJsZSBpcw0KDQo0LjE4KSBqdXN0
IGluIGNhc2UgeW91ciBpc3N1ZSBhcyBiZWVuIGZpeGVkIGluIG5ld2VyIHJlbGVhc2UuDQoNCg0K
DQpbVGhhbGVzXSA6IHdlIGhhdmUgdG8gdXNlIFhlbiByZWxlYXNlIHRoYXQgYXJlIGRvd25sb2Fk
ZWQgZnJvbSB0aGUgRGViaWFuIG9mZmljaWFsIHJlcG9zaXRvcnkgIChyZXF1ZXN0ZWQgYnkgb3Vy
IGN1c3RvbWVyKQ0KDQpbVGhhbGVzXSA6IHRoZSBsYXN0IG9uZSBpcyB2ZXJzaW9uIDQuMTQsIGJ1
dCB3ZSB3b3VsZCBsaWtlIHRvIGtub3cgaWYgdGhlcmUgaXMgb25lIG1vcmUgcmVjZW50IHRoYXQg
aXMgY29tcGF0aWJsZSB3aXRoIExpbnV4IERlYmlhbiAxMSA/DQoNCg0KDQo+DQoNCj4gICAgIC1X
aW5kb3dzIDEwIDE2MDcgYW5kIDIxSDIgZm9yIHZpcnR1YWwgbWFjaGluZXMuIFZpcnR1YWwgTWFj
aGluZQ0KDQo+IEhWTQ0KDQoNCg0KQ2hlZXJzLA0KDQoNCg0KLS0NCg0KSnVsaWVuIEdyYWxsDQo=

--_000_3ee3659afff645cabed86bcc22c44686thalesgroupcom_
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64

PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy
bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt
YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj
cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv
VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg
Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv
ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl
PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6
IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1m
YWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAy
IDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNvTm9ybWFsLCBsaS5Nc29Ob3JtYWws
IGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBjbTsNCgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJ
Zm9udC1zaXplOjExLjBwdDsNCglmb250LWZhbWlseToiQ2FsaWJyaSIsc2Fucy1zZXJpZjsNCglt
c28tZmFyZWFzdC1sYW5ndWFnZTpFTi1VUzt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJ
e21zby1zdHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjojMDU2M0MxOw0KCXRleHQtZGVjb3JhdGlv
bjp1bmRlcmxpbmU7fQ0KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJe21z
by1zdHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjojOTU0RjcyOw0KCXRleHQtZGVjb3JhdGlvbjp1
bmRlcmxpbmU7fQ0KcC5Nc29QbGFpblRleHQsIGxpLk1zb1BsYWluVGV4dCwgZGl2Lk1zb1BsYWlu
VGV4dA0KCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7DQoJbXNvLXN0eWxlLWxpbms6IlRleHRlIGJy
dXQgQ2FyIjsNCgltYXJnaW46MGNtOw0KCW1hcmdpbi1ib3R0b206LjAwMDFwdDsNCglmb250LXNp
emU6MTEuMHB0Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIixzYW5zLXNlcmlmOw0KCW1zby1mYXJl
YXN0LWxhbmd1YWdlOkVOLVVTO30NCnAubXNvbm9ybWFsMCwgbGkubXNvbm9ybWFsMCwgZGl2Lm1z
b25vcm1hbDANCgl7bXNvLXN0eWxlLW5hbWU6bXNvbm9ybWFsOw0KCW1zby1tYXJnaW4tdG9wLWFs
dDphdXRvOw0KCW1hcmdpbi1yaWdodDowY207DQoJbXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG87
DQoJbWFyZ2luLWxlZnQ6MGNtOw0KCWZvbnQtc2l6ZToxMi4wcHQ7DQoJZm9udC1mYW1pbHk6IlRp
bWVzIE5ldyBSb21hbiIsc2VyaWY7fQ0Kc3Bhbi5UZXh0ZWJydXRDYXINCgl7bXNvLXN0eWxlLW5h
bWU6IlRleHRlIGJydXQgQ2FyIjsNCgltc28tc3R5bGUtcHJpb3JpdHk6OTk7DQoJbXNvLXN0eWxl
LWxpbms6IlRleHRlIGJydXQiOw0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIixzYW5zLXNlcmlmO30N
CnNwYW4uRW1haWxTdHlsZTIwDQoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsOw0KCWZvbnQtZmFt
aWx5OiJDYWxpYnJpIixzYW5zLXNlcmlmOw0KCWNvbG9yOiMxRjQ5N0Q7fQ0Kc3Bhbi5FbWFpbFN0
eWxlMjENCgl7bXNvLXN0eWxlLXR5cGU6cGVyc29uYWwtcmVwbHk7DQoJZm9udC1mYW1pbHk6IkNh
bGlicmkiLHNhbnMtc2VyaWY7DQoJY29sb3I6IzFGNDk3RDt9DQouTXNvQ2hwRGVmYXVsdA0KCXtt
c28tc3R5bGUtdHlwZTpleHBvcnQtb25seTsNCglmb250LXNpemU6MTAuMHB0O30NCkBwYWdlIFdv
cmRTZWN0aW9uMQ0KCXtzaXplOjYxMi4wcHQgNzkyLjBwdDsNCgltYXJnaW46NzAuODVwdCA3MC44
NXB0IDcwLjg1cHQgNzAuODVwdDt9DQpkaXYuV29yZFNlY3Rpb24xDQoJe3BhZ2U6V29yZFNlY3Rp
b24xO30NCi0tPjwvc3R5bGU+PCEtLVtpZiBndGUgbXNvIDldPjx4bWw+DQo8bzpzaGFwZWRlZmF1
bHRzIHY6ZXh0PSJlZGl0IiBzcGlkbWF4PSIxMDI2IiAvPg0KPC94bWw+PCFbZW5kaWZdLS0+PCEt
LVtpZiBndGUgbXNvIDldPjx4bWw+DQo8bzpzaGFwZWxheW91dCB2OmV4dD0iZWRpdCI+DQo8bzpp
ZG1hcCB2OmV4dD0iZWRpdCIgZGF0YT0iMSIgLz4NCjwvbzpzaGFwZWxheW91dD48L3htbD48IVtl
bmRpZl0tLT4NCjwvaGVhZD4NCjxib2R5IGxhbmc9IkZSIiBsaW5rPSIjMDU2M0MxIiB2bGluaz0i
Izk1NEY3MiI+DQo8ZGl2IGNsYXNzPSJXb3JkU2VjdGlvbjEiPg0KPHAgY2xhc3M9Ik1zb05vcm1h
bCI+PHNwYW4gc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPkhpIEp1bGllbiw8bzpwPjwvbzpwPjwvc3Bh
bj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+
PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4g
bGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdEIj5UaGFuayB5b3UgZm9yIHlvdXIgcXVp
Y2sgYW5zd2VyLjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxz
cGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48
L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxl
PSJjb2xvcjojMUY0OTdEIj5Zb3Ugd2lsbCBmaW5kIGJlbG93IG91ciBhbnN3ZXJzDQo8L3NwYW4+
PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjpyZWQiPihpbiByZWQpPC9zcGFuPjxzcGFu
IGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+IHRvIHlvdXIgcXVlc3Rpb25zLjxv
OnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVO
LVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0K
PHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0
OTdEIj5UbyBzdW1tYXJpemUgb3VyIHJlcXVlc3QgOiB3aGF0IHdlIHdvdWxkIGxpa2UgaXMgdG8g
dXNlIFVTQiAzLjAgZHJpdmVyIHdpdGggaGlnaCBzcGVlZCBjb25maWd1cmF0aW9uLjxvOnA+PC9v
OnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBz
dHlsZT0iY29sb3I6IzFGNDk3RCI+VG9kYXksIGl0IGlzIG5vdCBwb3NzaWJsZSB0byBkbyB0aGF0
LjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9
IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+VGhlIGRyaXZlciBzdGF5IGluIGZ1bGwgc3Bl
ZWQgbW9kZSwgYW5kIG1vcmUgb2Z0ZW4gaW4gVVNCIDEuMCBvciAyLjAgY29uZmlndXJhdGlvbi48
bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJF
Ti1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFG
NDk3RCI+SXMgaXQgcG9zc2libGUgdG8gY29uZmlndXJlIHRoZSBYRU4gVVNCIDMuMCBkcml2ZXIg
d2l0aCB0aGUgaGlnaCBzcGVlZCBtb2RlID88bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPihy
ZWFkIGluIG91ciBhbnN3ZXJzIGJlbG93IHRoZSByZWZlcmVuY2UgdG8gdGhlIGRyaXZlcik8bzpw
PjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBsYW5nPSJFTi1V
UyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxw
IGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3
RCI+QmVzdCByZWdhcmRzLDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3Jt
YWwiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+RG9taW5pcXVlPG86
cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gbGFuZz0iRU4t
VVMiIHN0eWxlPSJjb2xvcjojMUY0OTdEIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8
cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij48c3BhbiBsYW5nPSJFTi1VUyI+PG86cD4mbmJzcDs8L286
cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+PHNwYW4gbGFuZz0iRU4tVVMi
Pi0tLS0tTWVzc2FnZSBkJ29yaWdpbmUtLS0tLTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNs
YXNzPSJNc29QbGFpblRleHQiPkRlJm5ic3A7OiBKdWxpZW4gR3JhbGwgJmx0OzxhIGhyZWY9Im1h
aWx0bzpqdWxpZW5AeGVuLm9yZyI+PHNwYW4gc3R5bGU9ImNvbG9yOndpbmRvd3RleHQ7dGV4dC1k
ZWNvcmF0aW9uOm5vbmUiPmp1bGllbkB4ZW4ub3JnPC9zcGFuPjwvYT4mZ3Q7PG86cD48L286cD48
L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij5FbnZvecOpJm5ic3A7OiBzYW1lZGkgMjQgZsOp
dnJpZXIgMjAyNCAxNDozNzxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+
w4AmbmJzcDs6IEdPVVJMT1QgRnJhbmNvaXMgJmx0OzxhIGhyZWY9Im1haWx0bzpmcmFuY29pcy5n
b3VybG90QHRoYWxlc2dyb3VwLmNvbSI+PHNwYW4gc3R5bGU9ImNvbG9yOndpbmRvd3RleHQ7dGV4
dC1kZWNvcmF0aW9uOm5vbmUiPmZyYW5jb2lzLmdvdXJsb3RAdGhhbGVzZ3JvdXAuY29tPC9zcGFu
PjwvYT4mZ3Q7Ow0KPGEgaHJlZj0ibWFpbHRvOnhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9y
ZyI+PHNwYW4gc3R5bGU9ImNvbG9yOndpbmRvd3RleHQ7dGV4dC1kZWNvcmF0aW9uOm5vbmUiPnhl
bi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZzwvc3Bhbj48L2E+IENjJm5ic3A7OiBLZWxseSBD
aG9pICZsdDs8YSBocmVmPSJtYWlsdG86a2VsbHkuY2hvaUBjbG91ZC5jb20iPjxzcGFuIHN0eWxl
PSJjb2xvcjp3aW5kb3d0ZXh0O3RleHQtZGVjb3JhdGlvbjpub25lIj5rZWxseS5jaG9pQGNsb3Vk
LmNvbTwvc3Bhbj48L2E+Jmd0OzsNCiBKYW4gQmV1bGljaCAmbHQ7PGEgaHJlZj0ibWFpbHRvOmpi
ZXVsaWNoQHN1c2UuY29tIj48c3BhbiBzdHlsZT0iY29sb3I6d2luZG93dGV4dDt0ZXh0LWRlY29y
YXRpb246bm9uZSI+amJldWxpY2hAc3VzZS5jb208L3NwYW4+PC9hPiZndDs7IFJvZ2VyIFBhdSBN
b25uw6kgJmx0OzxhIGhyZWY9Im1haWx0bzpyb2dlci5wYXVAY2l0cml4LmNvbSI+PHNwYW4gc3R5
bGU9ImNvbG9yOndpbmRvd3RleHQ7dGV4dC1kZWNvcmF0aW9uOm5vbmUiPnJvZ2VyLnBhdUBjaXRy
aXguY29tPC9zcGFuPjwvYT4mZ3Q7Ow0KIEFuZHJldyBDb29wZXIgJmx0OzxhIGhyZWY9Im1haWx0
bzphbmRyZXcuY29vcGVyM0BjaXRyaXguY29tIj48c3BhbiBzdHlsZT0iY29sb3I6d2luZG93dGV4
dDt0ZXh0LWRlY29yYXRpb246bm9uZSI+YW5kcmV3LmNvb3BlcjNAY2l0cml4LmNvbTwvc3Bhbj48
L2E+Jmd0OzsgR2VvcmdlIER1bmxhcCAmbHQ7PGEgaHJlZj0ibWFpbHRvOmdlb3JnZS5kdW5sYXBA
Y2l0cml4LmNvbSI+PHNwYW4gc3R5bGU9ImNvbG9yOndpbmRvd3RleHQ7dGV4dC1kZWNvcmF0aW9u
Om5vbmUiPmdlb3JnZS5kdW5sYXBAY2l0cml4LmNvbTwvc3Bhbj48L2E+Jmd0OzsNCiBKYW4gQmV1
bGljaCAmbHQ7PGEgaHJlZj0ibWFpbHRvOmpiZXVsaWNoQHN1c2UuY29tIj48c3BhbiBzdHlsZT0i
Y29sb3I6d2luZG93dGV4dDt0ZXh0LWRlY29yYXRpb246bm9uZSI+amJldWxpY2hAc3VzZS5jb208
L3NwYW4+PC9hPiZndDs7IEp1ZXJnZW4gR3Jvc3MgJmx0OzxhIGhyZWY9Im1haWx0bzpqZ3Jvc3NA
c3VzZS5jb20iPjxzcGFuIHN0eWxlPSJjb2xvcjp3aW5kb3d0ZXh0O3RleHQtZGVjb3JhdGlvbjpu
b25lIj5qZ3Jvc3NAc3VzZS5jb208L3NwYW4+PC9hPiZndDsNCiBPYmpldCZuYnNwOzogUmU6IElN
UE9SVEFOVCAtIDogTmVlZCBoZWxwIG9uIFVTQiBwb3J0IHZpcnR1YWxpemF0aW9uIHdpdGggWGVu
IGh5cGVydmlzb3I8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPjxvOnA+
Jm5ic3A7PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+SGkgRG9taW5pcXVlIGFu
ZCBGcmFuY29pcyw8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPjxvOnA+
Jm5ic3A7PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+VGhhbmsgeW91IGZvciB0
aGUgcmVwb3J0LiBJIGFtIENDaW5nIHNvbWUgcGVyc29uIHdobyBJIHRoaW5rIGNvdWxkIGhlbHAu
PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij48bzpwPiZuYnNwOzwvbzpw
PjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPkkgYWxzbyBoYXZlIHNvbWUgcXVlc3Rpb25z
IChzZWUgYmVsb3cpLjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+PG86
cD4mbmJzcDs8L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij5PbiAyMy8wMi8yMDI0
IDE2OjAwLCBHT1VSTE9UIEZyYW5jb2lzIHdyb3RlOjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9
Ik1zb1BsYWluVGV4dCI+Jmd0OyBEZWFyIEFsbCw8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJN
c29QbGFpblRleHQiPiZndDsgPG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0
Ij4mZ3Q7IFdlIHNlbmQgeW91IGEgbWVzc2FnZSBmZXcgZGF5cyBhZ28uPG86cD48L286cD48L3A+
DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij4mZ3Q7IDxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9
Ik1zb1BsYWluVGV4dCI+Jmd0OyBXZSBoYXZlIG1ham9yIHBlcmZvcm1hbmNlIHdpdGggWEVOIFVT
QiBEcml2ZXJzLiBXZSB1c2UgeW91ciBkcml2ZXIgdG8NCjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xh
c3M9Ik1zb1BsYWluVGV4dCI+Jmd0OyBsb2FkIGRhdGEgaW4gYSBkZXZpY2UgYW5kIHdlIG5lZWQg
VVNCMyBIaWdoIFNwZWVkIFVTQiBwcm90b2NvbGU8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJN
c29QbGFpblRleHQiPiZndDsgPG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0
Ij4mZ3Q7IERvIHlvdSBoYXZlIGEgbmV3IGlzc3VlIG9mIHRoZSBIQ0kgYW5kIE5FQyBVU0IgWGVu
IERyaXZlciA/PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij48bzpwPiZu
YnNwOzwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPlsuLi5dPG86cD48L286cD48
L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwIGNs
YXNzPSJNc29QbGFpblRleHQiPiZndDsgKkRlJm5ic3A7OipMQVJSSUVVIERvbWluaXF1ZSAmbHQ7
PGEgaHJlZj0ibWFpbHRvOmRvbWluaXF1ZS5sYXJyaWV1QHRoYWxlc2dyb3VwLmNvbSI+PHNwYW4g
c3R5bGU9ImNvbG9yOndpbmRvd3RleHQ7dGV4dC1kZWNvcmF0aW9uOm5vbmUiPmRvbWluaXF1ZS5s
YXJyaWV1QHRoYWxlc2dyb3VwLmNvbTwvc3Bhbj48L2E+Jmd0OzxvOnA+PC9vOnA+PC9wPg0KPHAg
Y2xhc3M9Ik1zb1BsYWluVGV4dCI+Jmd0OyAqRW52b3nDqSZuYnNwOzoqIGpldWRpIDIyIGbDqXZy
aWVyIDIwMjQgMTM6NTggKsOAJm5ic3A7OiogPG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNv
UGxhaW5UZXh0Ij4mZ3Q7IDxhIGhyZWY9Im1haWx0bzp4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVj
dC5vcmciPjxzcGFuIHN0eWxlPSJjb2xvcjp3aW5kb3d0ZXh0O3RleHQtZGVjb3JhdGlvbjpub25l
Ij54ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc8L3NwYW4+PC9hPiAqQ2MmbmJzcDs6KiBX
SUxMRU1TIExvdWlzDQo8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPiZn
dDsgJmx0OzxhIGhyZWY9Im1haWx0bzpsb3Vpcy53aWxsZW1zQHRoYWxlc2dyb3VwLmNvbSI+PHNw
YW4gc3R5bGU9ImNvbG9yOndpbmRvd3RleHQ7dGV4dC1kZWNvcmF0aW9uOm5vbmUiPmxvdWlzLndp
bGxlbXNAdGhhbGVzZ3JvdXAuY29tPC9zcGFuPjwvYT4mZ3Q7OyBHUlVPIE5pY29sYXMNCjxvOnA+
PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+Jmd0OyAmbHQ7PGEgaHJlZj0ibWFp
bHRvOm5pY29sYXMtbi5ncnVvQHRoYWxlc2dyb3VwLmNvbSI+PHNwYW4gc3R5bGU9ImNvbG9yOndp
bmRvd3RleHQ7dGV4dC1kZWNvcmF0aW9uOm5vbmUiPm5pY29sYXMtbi5ncnVvQHRoYWxlc2dyb3Vw
LmNvbTwvc3Bhbj48L2E+Jmd0OzsgR09VUkxPVCBGcmFuY29pcw0KPG86cD48L286cD48L3A+DQo8
cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij4mZ3Q7ICZsdDs8YSBocmVmPSJtYWlsdG86ZnJhbmNvaXMu
Z291cmxvdEB0aGFsZXNncm91cC5jb20iPjxzcGFuIHN0eWxlPSJjb2xvcjp3aW5kb3d0ZXh0O3Rl
eHQtZGVjb3JhdGlvbjpub25lIj5mcmFuY29pcy5nb3VybG90QHRoYWxlc2dyb3VwLmNvbTwvc3Bh
bj48L2E+Jmd0OzsgS2VsbHkgQ2hvaSAmbHQ7PGEgaHJlZj0ibWFpbHRvOmtlbGx5LmNob2lAY2xv
dWQuY29tIj48c3BhbiBzdHlsZT0iY29sb3I6d2luZG93dGV4dDt0ZXh0LWRlY29yYXRpb246bm9u
ZSI+a2VsbHkuY2hvaUBjbG91ZC5jb208L3NwYW4+PC9hPiZndDsNCjxvOnA+PC9vOnA+PC9wPg0K
PHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+Jmd0OyAqT2JqZXQmbmJzcDs6KiBOZWVkIGhlbHAgb24g
VVNCIHBvcnQgdmlydHVhbGl6YXRpb24gd2l0aCBYZW4gaHlwZXJ2aXNvcg0KPG86cD48L286cD48
L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij4mZ3Q7ICpDcml0w6hyZSBkZSBkaWZmdXNpb24m
bmJzcDs6KiBDb25maWRlbnRpZWw8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRl
eHQiPiZndDsgPG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij4mZ3Q7Jm5i
c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IERlYXIgYWxsLDxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9
Ik1zb1BsYWluVGV4dCI+Jmd0OyA8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRl
eHQiPjxzcGFuIGxhbmc9IkVOLVVTIj4mZ3Q7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IFdlIGFy
ZSBkZXRlY3Rpbmc8c3BhbiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+DQo8L3NwYW4+c2V2ZXJhbCBp
c3N1ZXMgd2l0aCBVU0IgcG9ydCB2aXJ0dWFsaXphdGlvbiB3aXRoIHRoZTxvOnA+PC9vOnA+PC9z
cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPiZndDsmbmJzcDsmbmJzcDsmbmJzcDsm
bmJzcDsgWGVuIGh5cGVydmlzb3IuPG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5U
ZXh0Ij48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPkp1c3Qg
dG8gY2xhcmlmeSwgeW91IGFyZSBhc3NpZ25pbmcgdGhlIFBDSSBVU0IgYnVzIChyYXRoZXIgdGhh
biBqdXN0IHRoZSBVU0IgZGV2aWNlKSB0byB0aGUgZ3Vlc3QuIElzIHRoYXQgY29ycmVjdD88bzpw
PjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPjxvOnA+Jm5ic3A7PC9vOnA+PC9w
Pg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+SWYgc28uLi48bzpwPjwvbzpwPjwvcD4NCjxwIGNs
YXNzPSJNc29QbGFpblRleHQiPjxzcGFuIHN0eWxlPSJjb2xvcjpibGFjayI+PG86cD4mbmJzcDs8
L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+PHNwYW4gbGFuZz0iRU4t
VVMiIHN0eWxlPSJjb2xvcjpyZWQiPltUaGFsZXNdIDogd2UgYXJlIHVzaW5nIHRoZSBVU0IgY2hp
cHNldCBvZiB0aGUgbW90aGVyIGJvYXJkIChzZWUgYmVsb3cgdGhlIHJlZmVyZW5jZXMpPG86cD48
L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+PHNwYW4gbGFuZz0iRU4t
VVMiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQi
PiZndDs8bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPiZndDsg
RG8geW91IGhhdmUgYSBuZXcgaXNzdWUgb2YgdGhlIEhDSSBhbmQgTkVDIFVTQiBYZW4gRHJpdmVy
ID88bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPjxvOnA+Jm5ic3A7PC9v
OnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+Li4uIEkgYW0gbm90IHN1cmUgd2hhdCBY
ZW4gZHJpdmVyIHlvdSBhcmUgdGFsa2luZyBhYm91dC4gSWYgdGhlIFVTQiBidXMgaXMgYXNzaWdu
ZWQgdG8gdGhlIGd1ZXN0LCB0aGVuIEkgd291bGQgZXhwZWN0IG5vIHNwZWNpZmljIFhlbiBkcml2
ZXIgYXJlIHJlcXVpcmVkLiBEbyB5b3UgaGF2ZSBhbnkgcG9pbnRlciB0byB0aGUgZXhpc3Rpbmcg
b25lPzxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+PHNwYW4gc3R5bGU9
ImNvbG9yOmJsYWNrIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNv
UGxhaW5UZXh0Ij48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOnJlZCI+W1RoYWxlc10g
OiBzZWUgcmVmZXJlbmNlcyB0byBIVyBhbmQgU1cgYmVsb3c8bzpwPjwvbzpwPjwvc3Bhbj48L3A+
DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij48c3BhbiBsYW5nPSJFTi1VUyI+PG86cD4mbmJzcDs8
L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+PHNwYW4gbGFuZz0iRU4t
VVMiPiZndDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgLSBXZSBjYW5ub3QgZG8gUENJIHBhc3N0
aHJvdWdoIG9mIHRoZSBQQ0kgdXNiIGJ1cyBvbiBhIFdpbmRvd3MgMTA8bzpwPjwvbzpwPjwvc3Bh
bj48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij4mZ3Q7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i
c3A7IDE2MDcgNjQtYml0IHZpcnR1YWwgbWFjaGluZS4gVGhlIGJhZCByZXN1bHQgaXMgYSBXaW5k
b3dzIGJsdWUgc2NyZWVuLjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+
Jmd0OyA8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPiZndDsmbmJzcDsm
bmJzcDsmbmJzcDsmbmJzcDsgLSBXaGVuIHdlIHVzZSB0aGUgcGFzc3Rocm91Z2ggZnVuY3Rpb25h
bGl0eSBvbiBhIFdpbmRvd3MgMjFIMjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWlu
VGV4dCI+Jmd0OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyB2aXJ0dWFsIG1hY2hpbmUsIHdlIG5v
dGljZSB0aGF0dGhlIHNwZWVkIG9mIHRoZSBVU0IgcG9ydCBpcyBub3QgaGlnaDxvOnA+PC9vOnA+
PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+Jmd0OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNw
OyBzcGVlZCBidXQgZnVsbCBzcGVlZCBvbiBhIFVTQiAzLjAgcG9ydDxvOnA+PC9vOnA+PC9wPg0K
PHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+Jmd0OyA8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJN
c29QbGFpblRleHQiPiZndDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgLSBXZSBub3RpY2UgaW5z
dGFiaWxpdGllcyB3aGVuIHVzaW5nIHRoZSBuZWMtdXNiLXhoY2kgZHJpdmVyLCAmbmJzcDtVU0I8
bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPiZndDsmbmJzcDsmbmJzcDsm
bmJzcDsmbmJzcDsgMi4wIGtleXMgYXJlIG5vdCByZWNvZ25pemVkIGJ5IHRoZSBXaW5kb3dzIHZp
cnR1YWwgbWFjaGluZTxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+Jmd0
OyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyAoaW5jb3JyZWN0IGRlc2NyaXB0b3IpPG86cD48L286
cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxw
IGNsYXNzPSJNc29QbGFpblRleHQiPkRvIHlvdSBoYXZlIGFueSBsb2dzIHlvdSBjb3VsZCBzaGFy
ZSBmcm9tIHRoZSBndWVzdCBPUz8gQWxzbywgd291bGQgeW91IGJlIGFibGUgdG8gc2hhcmUgbW9y
ZSBkZXRhaWxzIG9uIGhvdyB5b3UgY29uZmlndXJlIHRoZSBndWVzdD8gKGUuZy4geGwgY29uZmln
dXJhdGlvbiwgbGlidmlydC4uLik/PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5U
ZXh0Ij48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPiZndDsg
PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij4mZ3Q7Jm5ic3A7Jm5ic3A7
Jm5ic3A7Jm5ic3A7IFdlIG5lZWR5b3VyIGhlbHB0byBmaW5kIGEgc29sdXRpb24gZm9yIHRoZXNl
IHByb2JsZW1zLjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+Jmd0OyA8
bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPjxzcGFuIGxhbmc9IkVOLVVT
Ij4mZ3Q7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7IFRoZSBTb2Z0d2FyZSB1c2VkIGFyZSZuYnNw
Ozo8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij48c3BhbiBs
YW5nPSJFTi1VUyI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb1Bs
YWluVGV4dCI+PHNwYW4gbGFuZz0iRU4tVVMiPkNhbiB5b3Ugc2hhcmUgc29tZSBkZXRhaWxzIG9m
IHRoZSBIVyB5b3UgYXJlIHVzaW5nJm5ic3A7PzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNs
YXNzPSJNc29QbGFpblRleHQiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3
RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+
PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjpyZWQiPltUaGFsZXNdIDogSFcgOjwvc3Bh
bj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPjxvOnA+PC9vOnA+PC9z
cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHls
ZT0iY29sb3I6cmVkIj5bVGhhbGVzXSA6IDwvc3Bhbj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9
ImNvbG9yOnJlZCI+V2UgYXJlIHVzaW5nIGEgR0VUQUMgWDUwMCB1c2luZyB0aGUgTW9iaWxlIElu
dGVsIFFNMTc1IENoaXBzZXQ8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjoj
MUY0OTdEIj48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij48
c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOnJlZCI+W1RoYWxlc10gOiBVU0IgY29udHJv
bGxlcjogSW50ZWwgQ29ycG9yYXRpb24gMTAwIFNlcmllcy9DMjMwIFNlcmllcyBDaGlwc2V0IEZh
bWlseSBVU0IgMy4wIHhIQ0kgQ29udHJvbGxlciAocmV2IDMxKTxvOnA+PC9vOnA+PC9zcGFuPjwv
cD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29s
b3I6cmVkIj5bVGhhbGVzXSA6IFN1YnN5c3RlbTogTWl0YWMgMTAwIFNlcmllcy9DMjMwIFNlcmll
cyBDaGlwc2V0IEZhbWlseSBVU0IgMy4wIHhIQ0kgQ29udHJvbGxlcjxvOnA+PC9vOnA+PC9zcGFu
PjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0i
Y29sb3I6IzFGNDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1z
b1BsYWluVGV4dCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjpyZWQiPltUaGFsZXNd
IDogU1cgOjwvc3Bhbj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMxRjQ5N0QiPjxv
OnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPjxzcGFuIGxhbmc9
IkVOLVVTIiBzdHlsZT0iY29sb3I6cmVkIj5bVGhhbGVzXSA6IEtlcm5lbCBkcml2ZXIgaW4gdXNl
OiB4aGNpX2hjZDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQi
PjxzcGFuIHN0eWxlPSJjb2xvcjpyZWQiPltUaGFsZXNdJm5ic3A7OiBLZXJuZWwgbW9kdWxlcyZu
YnNwOzogeGhjaV9wY2k8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5U
ZXh0Ij48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPiZndDsg
PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij4mZ3Q7Jm5ic3A7Jm5ic3A7
Jm5ic3A7Jm5ic3A7IC1EZWJpYW4gMTEgdmVyc2lvbjUuMTAuMC0yMDxvOnA+PC9vOnA+PC9wPg0K
PHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+Jmd0OyA8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJN
c29QbGFpblRleHQiPiZndDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgLVhlbiB2ZXJzaW9uNC4x
NDxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+PG86cD4mbmJzcDs8L286
cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5UZXh0Ij48c3BhbiBsYW5nPSJFTi1VUyI+QXJlIG1p
bm9yIHZlcnNpb24gb2YgWGVuIDQuMTQgYXJlIHlvdSB1c2luZyZuYnNwOz8NCjwvc3Bhbj5Bbnkg
cGF0Y2hlcyBvbiB0b3AmbmJzcDs/PG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvUGxhaW5U
ZXh0Ij48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPlhlbiA0
LjE0IGhhcyBhbHNvIGJlZW4gb3V0IG9mIHN1cHBvcnQgZm9yIGEgZmV3IG1vbnRocyBub3cuDQo8
c3BhbiBsYW5nPSJFTi1VUyI+SWYgdGhhdOKAmXMgcG9zc2libGUsIEkgd291bGQgcmVjb21tZW5k
IHRvIHRyeSBhIG5ldyBYZW4gdmVyc2lvbiAodGhlIGxhc3Qgc3RhYmxlIGlzPG86cD48L286cD48
L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+NC4xOCkganVzdCBpbiBjYXNlIHlv
dXIgaXNzdWUgYXMgYmVlbiBmaXhlZCBpbiBuZXdlciByZWxlYXNlLjxvOnA+PC9vOnA+PC9wPg0K
PHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjpy
ZWQiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQi
PjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6cmVkIj5bVGhhbGVzXSA6PC9zcGFuPjxz
cGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6IzFGNDk3RCI+DQo8L3NwYW4+PHNwYW4gbGFu
Zz0iRU4tVVMiIHN0eWxlPSJjb2xvcjpyZWQiPndlIGhhdmUgdG8gdXNlIFhlbiByZWxlYXNlIHRo
YXQgYXJlIGRvd25sb2FkZWQgZnJvbSB0aGUgRGViaWFuIG9mZmljaWFsIHJlcG9zaXRvcnk8L3Nw
YW4+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJjb2xvcjojMUY0OTdEIj4NCjwvc3Bhbj48c3Bh
biBsYW5nPSJFTi1VUyIgc3R5bGU9ImNvbG9yOnJlZCI+Jm5ic3A7KHJlcXVlc3RlZCBieSBvdXIg
Y3VzdG9tZXIpPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iY29sb3I6YmxhY2siPjxv
OnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPjxzcGFuIGxhbmc9
IkVOLVVTIiBzdHlsZT0iY29sb3I6cmVkIj5bVGhhbGVzXSA6IHRoZSBsYXN0IG9uZSBpcyB2ZXJz
aW9uIDQuMTQsIGJ1dCB3ZSB3b3VsZCBsaWtlIHRvIGtub3cgaWYgdGhlcmUgaXMgb25lIG1vcmUg
cmVjZW50IHRoYXQgaXMgY29tcGF0aWJsZSB3aXRoIExpbnV4IERlYmlhbiAxMSA/PG86cD48L286
cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+PHNwYW4gbGFuZz0iRU4tVVMi
IHN0eWxlPSJjb2xvcjpibGFjayI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xh
c3M9Ik1zb1BsYWluVGV4dCI+Jmd0OyA8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFp
blRleHQiPiZndDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgLVdpbmRvd3MgMTAgMTYwNyBhbmQg
MjFIMiBmb3IgdmlydHVhbCBtYWNoaW5lcy4gVmlydHVhbCBNYWNoaW5lDQo8bzpwPjwvbzpwPjwv
cD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPiZndDsgSFZNPG86cD48L286cD48L3A+DQo8cCBj
bGFzcz0iTXNvUGxhaW5UZXh0Ij48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Q
bGFpblRleHQiPkNoZWVycyw8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQi
PjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb1BsYWluVGV4dCI+LS08bzpwPjwv
bzpwPjwvcD4NCjxwIGNsYXNzPSJNc29QbGFpblRleHQiPkp1bGllbiBHcmFsbDxvOnA+PC9vOnA+
PC9wPg0KPC9kaXY+DQo8L2JvZHk+DQo8L2h0bWw+DQo=

--_000_3ee3659afff645cabed86bcc22c44686thalesgroupcom_--


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 10:43:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 10:43:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686945.1069585 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdsd-00028z-Mu; Thu, 29 Feb 2024 10:43:11 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686945.1069585; Thu, 29 Feb 2024 10:43:11 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfdsd-00028s-Jv; Thu, 29 Feb 2024 10:43:11 +0000
Received: by outflank-mailman (input) for mailman id 686945;
 Thu, 29 Feb 2024 10:43:10 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfdsc-00028m-FQ
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 10:43:10 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5392571d-d6ef-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 11:43:08 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a444205f764so62699466b.2
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 02:43:08 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 i8-20020a170906698800b00a43e8e76825sm551417ejr.149.2024.02.29.02.43.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 02:43:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5392571d-d6ef-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709203387; x=1709808187; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=e+97K3Va0wILi/TEk43C6BEvjMMCdN17c8jlE9FRLRQ=;
        b=bNGXjVvYKTT1dEoRg+I1jILw8OXjZ/ay6mbMwfk4oAZnLBOgmHlcD+HEAcies2KeO6
         +AVhbsPdCTb980khCBNan51ZHzlsOejKJdjEpqnAq641v3QZQLFzXUKYnUC8MlIQnq0Q
         iEyi/B77YuqxdDrHNXFz5h39OW2KbI58tVSEI=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709203387; x=1709808187;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=e+97K3Va0wILi/TEk43C6BEvjMMCdN17c8jlE9FRLRQ=;
        b=ap35cE5dMwgPLNZq+UkDd1U+RA5gdIL4TOsJhRX3pvqJHDJMJr90RlwWFuckl2+JLr
         24tuuYOpnnibaPMU42Q9kcWbN9BzaaDFnvE3GcRM6GXk/kLe5+LsQi2iUv6UGXkvFeIe
         SV2PECRGCAFo7MsoEMam0EvmXVpQ65GQDYLpaCA34Tv9EDaBrPFPheY3psthG0HrMg+r
         hIMZs5FYzebEBvMByOuCoyCD6q2tzDxRdboB/AcuDv4VYHJg/ogh2/GA7RzrBCnnImxE
         yPF+TpFROOqXN2Qj5s40Qe4/LroD1BQ/4MbsMcbDh675crn1JCsy1ADNIQeOddWpWQcE
         wDgA==
X-Gm-Message-State: AOJu0YzGai7f25vBhIZdW5Bre1ZYdAnoOeW5nbSnSdYkth7Pww+mDEpC
	VSXGHUtERxbkPwEuLWaUUH2E8QSbj1tEWJouIlhueFLbImPrVUtBzouaKYsWxzq9X9eA0/IydIs
	H
X-Google-Smtp-Source: AGHT+IHkUWf2Dj6ASdr/bVBK9V8FQhh8mjfGb7b58yDSpuaHu6tR9Oe3f9op0BTW12MJNBN/oPwXbA==
X-Received: by 2002:a17:906:d146:b0:a44:1e32:a503 with SMTP id br6-20020a170906d14600b00a441e32a503mr1162316ejb.22.1709203386741;
        Thu, 29 Feb 2024 02:43:06 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/cpu-policy: Fix x2APIC visibility for PV guests
Date: Thu, 29 Feb 2024 10:43:04 +0000
Message-Id: <20240229104304.2478614-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Right now, the host x2APIC setting filters into the PV max and default
policies, yet PV guests cannot set MSR_APIC_BASE.EXTD or access any of the
x2APIC MSR range.  Therefore they absolutely shouldn't see the x2APIC bit.

Linux has workarounds for the collateral damage caused by this leakage; it
unconditionally filters out the x2APIC CPUID bit, and EXTD when reading
MSR_APIC_BASE.

Hide the x2APIC bit in the PV default policy, but for compatibility, tolerate
incoming VMs which already saw the bit.  This is logic from before the
default/max split in Xen 4.14 which wasn't correctly adjusted at the time.

Update the annotation from !A to !S which slightly better describes that it
doesn't really exist in PV guests.  HVM guests, for which x2APIC can be
emulated completely, already has it unconditionally set in the max policy.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

This wants backporting as far as people can tollerate, but it's really not
obvious which commit in 4.14 should be referenced in a Fixes: tag.
---
 xen/arch/x86/cpu-policy.c                   | 19 +++++++++++++++++--
 xen/include/public/arch-x86/cpufeatureset.h |  2 +-
 2 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index 10079c26ae24..a0205672428d 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -534,6 +534,14 @@ static void __init calculate_pv_max_policy(void)
     *p = host_cpu_policy;
     x86_cpu_policy_to_featureset(p, fs);
 
+    /*
+     * Xen at the time of writing (Feb 2024, 4.19 dev cycle) used to leak the
+     * host x2APIC capability into PV guests, but never supported the guest
+     * trying to turn x2APIC mode on.  Tolerate an incoming VM which saw the
+     * x2APIC CPUID bit.
+     */
+    __set_bit(X86_FEATURE_X2APIC, fs);
+
     for ( i = 0; i < ARRAY_SIZE(fs); ++i )
         fs[i] &= pv_max_featuremask[i];
 
@@ -566,6 +574,14 @@ static void __init calculate_pv_def_policy(void)
     *p = pv_max_cpu_policy;
     x86_cpu_policy_to_featureset(p, fs);
 
+    /*
+     * PV guests have never been able to use x2APIC mode, but at the time of
+     * writing (Feb 2024, 4.19 dev cycle), the host value used to leak into
+     * guests.  Hide it by default so new guests don't get mislead into
+     * thinking that they can use x2APIC.
+     */
+    __clear_bit(X86_FEATURE_X2APIC, fs);
+
     for ( i = 0; i < ARRAY_SIZE(fs); ++i )
         fs[i] &= pv_def_featuremask[i];
 
@@ -830,11 +846,10 @@ void recalculate_cpuid_policy(struct domain *d)
     }
 
     /*
-     * Allow the toolstack to set HTT, X2APIC and CMP_LEGACY.  These bits
+     * Allow the toolstack to set HTT and CMP_LEGACY.  These bits
      * affect how to interpret topology information in other cpuid leaves.
      */
     __set_bit(X86_FEATURE_HTT, max_fs);
-    __set_bit(X86_FEATURE_X2APIC, max_fs);
     __set_bit(X86_FEATURE_CMP_LEGACY, max_fs);
 
     /*
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index be5c1b748e27..b230d3a6907d 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -123,7 +123,7 @@ XEN_CPUFEATURE(PCID,          1*32+17) /*H  Process Context ID */
 XEN_CPUFEATURE(DCA,           1*32+18) /*   Direct Cache Access */
 XEN_CPUFEATURE(SSE4_1,        1*32+19) /*A  Streaming SIMD Extensions 4.1 */
 XEN_CPUFEATURE(SSE4_2,        1*32+20) /*A  Streaming SIMD Extensions 4.2 */
-XEN_CPUFEATURE(X2APIC,        1*32+21) /*!A Extended xAPIC */
+XEN_CPUFEATURE(X2APIC,        1*32+21) /*!S Extended xAPIC */
 XEN_CPUFEATURE(MOVBE,         1*32+22) /*A  movbe instruction */
 XEN_CPUFEATURE(POPCNT,        1*32+23) /*A  POPCNT instruction */
 XEN_CPUFEATURE(TSC_DEADLINE,  1*32+24) /*S  TSC Deadline Timer */
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 11:51:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 11:51:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686957.1069596 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfewy-0003JB-Do; Thu, 29 Feb 2024 11:51:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686957.1069596; Thu, 29 Feb 2024 11:51:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfewy-0003J4-Ax; Thu, 29 Feb 2024 11:51:44 +0000
Received: by outflank-mailman (input) for mailman id 686957;
 Thu, 29 Feb 2024 11:51:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfewx-0003Iy-52
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 11:51:43 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfeww-0002oJ-LQ; Thu, 29 Feb 2024 11:51:42 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfeww-0007UP-Dh; Thu, 29 Feb 2024 11:51:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=K6VjbTW6Jv+NExibT1D+zXR6Evxh9PaZtfdIcQv1kss=; b=XHRuc1fSjBKi0UE1ALQKQT9XDO
	DW7bxB4Ys6L4EhfZPbeP0Icz9bMpHWf2F8SqOzk+iT46uWCnuWJ78KvReWSzbUhn8jxGF47W7IBg8
	BdTG+/5uR3z6b5pZ0yx38zWs98zsmuXLt4HTGrf39d+OV7Z4RMEccj00vBpCp/1RguD4=;
Message-ID: <4778822e-582f-4e0d-9933-86d8d49ea3a5@xen.org>
Date: Thu, 29 Feb 2024 11:51:39 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: IMPORTANT - : Need help on USB port virtualization with Xen
 hypervisor
Content-Language: en-GB
To: LARRIEU Dominique <dominique.larrieu@thalesgroup.com>
Cc: GOURLOT Francois <francois.gourlot@thalesgroup.com>,
 GRUO Nicolas <nicolas-n.gruo@thalesgroup.com>,
 Cc <xen-devel@lists.xenproject.org>, Kelly Choi <kelly.choi@cloud.com>,
 Jan Beulich <jbeulich@suse.com>, =?UTF-8?Q?Roger_Pau_Monn=C3=A9?=
 <roger.pau@citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Juergen Gross <jgross@suse.com>,
 WILLEMS Louis <louis.willems@thalesgroup.com>
References: <d2de4ae9ecb34efc962dea7f8b4e7cbd@thalesgroup.com>
 <bb2485e5-7818-435b-8d9c-dda88100979f@xen.org>
 <a46ac2e14fa2410eafc26a37a00a442f@thalesgroup.com>
 <427d2ecb865648b7a459c592c208c0be@thalesgroup.com>
 <673b2bc630d748e8af0a15d4b553906e@thalesgroup.com>
 <3ee3659afff645cabed86bcc22c44686@thalesgroup.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <3ee3659afff645cabed86bcc22c44686@thalesgroup.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Dominique,

On 29/02/2024 10:33, LARRIEU Dominique wrote:
> Thank you for your quick answer.
> 
> You will find below our answers (in red) to your questions.
> 
> To summarize our request : what we would like is to use USB 3.0 driver with high speed configuration.
> Today, it is not possible to do that.
> The driver stay in full speed mode, and more often in USB 1.0 or 2.0 configuration.
> 
> Is it possible to configure the XEN USB 3.0 driver with the high speed mode ?
> (read in our answers below the reference to the driver)

Looking at what you provided below, it is still unclear what you mean by 
"Xen USB 3.0 driver". By any chance, are you confused with the "x" in xhci?

The drivers "xhci_hcd" and "xhci_pci" are generic and not 
provided/maintained by Xen. So far, with the information provided, it is 
unclear whether the problem is actually in the driver itself. I have a 
few more questions. See below.

>>      We are detecting several issues with USB port virtualization with the
> 
>>      Xen hypervisor.
> 
> 
> 
> Just to clarify, you are assigning the PCI USB bus (rather than just the USB device) to the guest. Is that correct?
> 
> 
> 
> If so...
> 
> 
> 
> [Thales] : we are using the USB chipset of the mother board (see below the references)

Thanks for the clarification. IIUC, you said you are having problem in 
the guest itself. Have you tried to access the same USB device from 
dom0? Or better without any Xen involved (i.e. booting Debian on baremetal).

This would help to narrow down the issue.

>>
> 
>>      We needyour helpto find a solution for these problems.
> 
>>
> 
>>      The Software used are :
> 
> 
> 
> Can you share some details of the HW you are using ?
> 
> 
> 
> [Thales] : HW :
> 
> [Thales] : We are using a GETAC X500 using the Mobile Intel QM175 Chipset
> 
> [Thales] : USB controller: Intel Corporation 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller (rev 31)
> 
> [Thales] : Subsystem: Mitac 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller
> 
> 
> 
> [Thales] : SW :
> 
> [Thales] : Kernel driver in use: xhci_hcd
> 
> [Thales] : Kernel modules : xhci_pci
> 
> 
> 
>>
> 
>>      -Debian 11 version5.10.0-20

IIUC, before you said you had problem with Windows. So is this the 
software in dom0? Or did you actually try with Debian and still see the 
same issue?

Regarding the configuration, given you are using the official Debian 
package. Are you using 'xl' to create your guest?

The ideal would be if you can paste the full configuration. If you can't 
can you tell us if you are using any of these options: 'pci', 'usbdev', 
'usbctrl'?

[...]

>>      -Xen version4.14
> 
> 
> 
> Are minor version of Xen 4.14 are you using ? Any patches on top ?
> 
> 
> 
> Xen 4.14 has also been out of support for a few months now. If that’s possible, I would recommend to try a new Xen version (the last stable is
> 
> 4.18) just in case your issue as been fixed in newer release.
> 
> 
> 
> [Thales] : we have to use Xen release that are downloaded from the Debian official repository  (requested by our customer)
> 
> [Thales] : the last one is version 4.14, but we would like to know if there is one more recent that is compatible with Linux Debian 11 ?

Newer Xen releases should be compatible with Debian 11. But I am not 
aware of any "official" package for that specific version.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 11:56:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 11:56:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686960.1069605 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rff1Q-0003tV-UL; Thu, 29 Feb 2024 11:56:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686960.1069605; Thu, 29 Feb 2024 11:56:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rff1Q-0003tO-RP; Thu, 29 Feb 2024 11:56:20 +0000
Received: by outflank-mailman (input) for mailman id 686960;
 Thu, 29 Feb 2024 11:56:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rff1O-0003tD-Td
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 11:56:18 +0000
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com
 [2a00:1450:4864:20::62e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 8bd33da1-d6f9-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 12:56:17 +0100 (CET)
Received: by mail-ej1-x62e.google.com with SMTP id
 a640c23a62f3a-a3122b70439so147499966b.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 03:56:17 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 j12-20020a170906094c00b00a3e5adf11c7sm609513ejd.157.2024.02.29.03.56.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 03:56:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8bd33da1-d6f9-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709207777; x=1709812577; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KKTlU9l00RReVLNiskghv+Ya2Ums+IDx8IJVD/f+928=;
        b=TqjDNyE/OvOLzEdmUSKZj5Hpiv5W0Nx8/530DPr4Fy7lgzqBGR8ocRmKd+Qu6RepRa
         YkG0nml4bcGwFgl59Kc9Jcl2orUbvAzGAYll8IGPH+0pEfHUlaHmT8e/MrReolLl+G0x
         gZpjySW/ZNvAy/ySbeezvCPbpZpUCZROJ0g3dXERuRBKQhs7v0qlg5ZlPc57xiPbVBVU
         6QhET0YXw8Tmmfl3S3vUUqHSb1lLc0do8AU1AXKN40MXjlTvyAPVPryQ9/TwLqTBEupw
         N4UisxJZTAVV4fsOoJLXfEXtJDs1/Xsi+3ANoPSeXrgu0bku5t03Yqal6gvDCDJWnhZm
         tCIA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709207777; x=1709812577;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KKTlU9l00RReVLNiskghv+Ya2Ums+IDx8IJVD/f+928=;
        b=d9ylIjgQ7SaHTLQlpdbOXm9Hqz64VGRn2YKUmq8pVqIsAOZ4zm/BEYL45K/2pCju2i
         ZT8uWM7coXXoWlR/FpqnIOAgSgePLkF5zdrBRdJAtgVOO9Li70vVIEhDdoqdJgnpzPcu
         8jBhWamy0f5keRd06qU8JWAslzAJjhkqhM2o+BUvAx/OxJexnHzFiSHw35PdWNZ0PmTx
         lzBJcWknYGGcPUdn1JmmdnmSadN0CWyQg4VTYyYt2pQi5cxdzWV7+yBtaYP//S32xvkO
         FajLAM/oJBBQe2p25TQCQ39aQjqKyHVIpEk+3RtY7vufRC+ju1Zgn4VuHDHJ7y2EQbMw
         nNkQ==
X-Forwarded-Encrypted: i=1; AJvYcCWey0ATX4NU4shoIc1pRq6Mh++Z4e1zAiRlkXU8OHvHWEqXIrHj4TIYrVhkrme71ebfeDMXCCTd2RTdHCOP15NrLXVvY4uSwXTLvvCWFp8=
X-Gm-Message-State: AOJu0Yx7WDgmkJPeucjuruRA/yJM11UF61vbRS4UXYjbxkAP53p5p/st
	wd8MaM1CcPyYbzJUdEaPA99tE0WKY0CgfPOkLqCIgt90cVwW+Da/HP4Pkg9Ujw==
X-Google-Smtp-Source: AGHT+IFOlm8N5jbpiJOBoY38ro1ICF4a6Ffl6wdTjvhU+ee0WYi+g8SBg7eEaoO9/38EN8k5W6bFLw==
X-Received: by 2002:a17:907:20d5:b0:a44:752:ad4e with SMTP id qq21-20020a17090720d500b00a440752ad4emr1265641ejb.75.1709207777262;
        Thu, 29 Feb 2024 03:56:17 -0800 (PST)
Message-ID: <751bc87b-c8a4-4905-92bf-fc50f9102d4a@suse.com>
Date: Thu, 29 Feb 2024 12:56:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
 <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 11:23, Julien Grall wrote:
> On 29/02/2024 07:58, Jan Beulich wrote:
>> Therefore being too
>> eager there would mean I can't really / easily (smoke) test Xen
>> anymore on ancient hardware every once in a while. When afaict we do
>> too little of such testing already anyway, despite not having any
>> lower bound on hardware that formally we support running Xen on.
> 
> Can you provide more details of what you mean by "ancient"?

Formally we support running Xen on any x86 hardware supporting 64-bit
mode. I don't think I have any 1st gen systems left, but I think a
2nd gen SVM and a 2nd gen VMX one is what I still have around.

>> (And
>> no, upgrading the ancient distros on that ancient hardware is not an
>> option for me.)
> 
> May I ask why? Is it because newer distros don't support your HW?

Because as part of my job I also need to support ancient versions of
Xen on ancient distros. Since I need to keep those around, it makes
sense to me to then also test modern Xen there (every now and then, as
said, and not really extensively).

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 11:56:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 11:56:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686961.1069615 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rff1p-0004J1-54; Thu, 29 Feb 2024 11:56:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686961.1069615; Thu, 29 Feb 2024 11:56:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rff1p-0004Iu-26; Thu, 29 Feb 2024 11:56:45 +0000
Received: by outflank-mailman (input) for mailman id 686961;
 Thu, 29 Feb 2024 11:56:44 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xh/y=KG=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rff1o-0004Ik-Fe
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 11:56:44 +0000
Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com
 [2607:f8b0:4864:20::736])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 9a4421ad-d6f9-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 12:56:42 +0100 (CET)
Received: by mail-qk1-x736.google.com with SMTP id
 af79cd13be357-787c900af02so59357785a.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 03:56:42 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 b6-20020a05620a04e600b00787f6064a9fsm584569qkh.108.2024.02.29.03.56.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 03:56:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9a4421ad-d6f9-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709207801; x=1709812601; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=TJAuCfUOEyK5EMPgJRPdwSJV9B95LJ1m5JcD53wqlYU=;
        b=dHMaUIUIMaUMVng8RhKl4pK1mxGjWaQberU+r2PZFrzRg60av+nefFP6fByJY7RUzp
         to6WDxgDw5Kdny4AXqRr9ADjCIfUVZ2xP24/Gw2WLOuzfsPcNGBZlbbnAh5tGIRsasX/
         hyGMGM4KNJ4v32qyoEtwqCxUlDDa/8RWAGdFo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709207801; x=1709812601;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=TJAuCfUOEyK5EMPgJRPdwSJV9B95LJ1m5JcD53wqlYU=;
        b=Voq1MVGddIOVrDpTSLVf82e9+deFribdJ84o4y7V+VdxIpTIg0ZK0hUuut9r/a0+nX
         6sKZEXgkf7VkfYJwpERh1afX6UHQwXLh/MlsUVp6rDrOVAUYvMoMcS1Ft/BiNSP2yVQd
         9JnnyYnrPvOMmqKVdfcOvH7gtrcw9sKNsBpNOXNZRqUTLhDMRQ3M03Va1SJdmZ2N6TE8
         pz0CfW9C5zCyfByuJz62kIvYJMMHdh+7gz54uJiDB7NcBWzVG5bhBptejqz/XUkumaf2
         9dK0/j4fw2/uv/VMRqRXjdiwG1D6YMSr2/Lhhy025+I2f/zwuWHkO81/lpOQF93be3u6
         HU6w==
X-Gm-Message-State: AOJu0YySboX1LgEfKtEmhShqSLnpoellqT5DlaDzw+WkacI5au8r3qjT
	TPpu1dKPGsJbEuLQuXaPHC0kUtdN/OZXCszpA4C5sXB2/6kI67RP70L3+aRv4UE=
X-Google-Smtp-Source: AGHT+IGs/GPmQQD0vqpZHE01EdVrRrPw/zKlleZ9A1O0xRrniiI8jDVkiE95nZsN42u73WUuAZ5kAg==
X-Received: by 2002:a05:620a:8f02:b0:787:fd85:26e8 with SMTP id rh2-20020a05620a8f0200b00787fd8526e8mr1233330qkn.19.1709207801349;
        Thu, 29 Feb 2024 03:56:41 -0800 (PST)
Date: Thu, 29 Feb 2024 12:56:39 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
	Jan Beulich <JBeulich@suse.com>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] x86/cpu-policy: Fix x2APIC visibility for PV guests
Message-ID: <ZeBw96VzCVeF0-6T@macbook>
References: <20240229104304.2478614-1-andrew.cooper3@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240229104304.2478614-1-andrew.cooper3@citrix.com>

On Thu, Feb 29, 2024 at 10:43:04AM +0000, Andrew Cooper wrote:
> Right now, the host x2APIC setting filters into the PV max and default
> policies, yet PV guests cannot set MSR_APIC_BASE.EXTD or access any of the
> x2APIC MSR range.  Therefore they absolutely shouldn't see the x2APIC bit.
> 
> Linux has workarounds for the collateral damage caused by this leakage; it
> unconditionally filters out the x2APIC CPUID bit, and EXTD when reading
> MSR_APIC_BASE.
> 
> Hide the x2APIC bit in the PV default policy, but for compatibility, tolerate
> incoming VMs which already saw the bit.  This is logic from before the
> default/max split in Xen 4.14 which wasn't correctly adjusted at the time.
> 
> Update the annotation from !A to !S which slightly better describes that it
> doesn't really exist in PV guests.  HVM guests, for which x2APIC can be
> emulated completely, already has it unconditionally set in the max policy.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Wei Liu <wl@xen.org>
> 
> This wants backporting as far as people can tollerate, but it's really not
> obvious which commit in 4.14 should be referenced in a Fixes: tag.

Oh, so we didn't use to expose x2APIC in Xen < 4.14 for PV at all?

I think this need mentioning in the commit message, as it's not clear
whether x2APIC has always been advertised to guests.

If it's indeed only Xen 4.14 that started exposing the flag, it's IMO
less dangerous to stop exposing it.  My main concern would be OSes
having grow some dependency on it, and us no longer exposing it
causing collateral damage (which would be an OS bug anyway).

> ---
>  xen/arch/x86/cpu-policy.c                   | 19 +++++++++++++++++--
>  xen/include/public/arch-x86/cpufeatureset.h |  2 +-
>  2 files changed, 18 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
> index 10079c26ae24..a0205672428d 100644
> --- a/xen/arch/x86/cpu-policy.c
> +++ b/xen/arch/x86/cpu-policy.c
> @@ -534,6 +534,14 @@ static void __init calculate_pv_max_policy(void)
>      *p = host_cpu_policy;
>      x86_cpu_policy_to_featureset(p, fs);
>  
> +    /*
> +     * Xen at the time of writing (Feb 2024, 4.19 dev cycle) used to leak the
> +     * host x2APIC capability into PV guests, but never supported the guest
> +     * trying to turn x2APIC mode on.  Tolerate an incoming VM which saw the
> +     * x2APIC CPUID bit.
> +     */
> +    __set_bit(X86_FEATURE_X2APIC, fs);
> +
>      for ( i = 0; i < ARRAY_SIZE(fs); ++i )
>          fs[i] &= pv_max_featuremask[i];
>  
> @@ -566,6 +574,14 @@ static void __init calculate_pv_def_policy(void)
>      *p = pv_max_cpu_policy;
>      x86_cpu_policy_to_featureset(p, fs);
>  
> +    /*
> +     * PV guests have never been able to use x2APIC mode, but at the time of
> +     * writing (Feb 2024, 4.19 dev cycle), the host value used to leak into
> +     * guests.  Hide it by default so new guests don't get mislead into
> +     * thinking that they can use x2APIC.
> +     */
> +    __clear_bit(X86_FEATURE_X2APIC, fs);

IIRC if you use the 'S' tag it won't be added to the default PV policy
already, so there should be nothing to clear?  pv_def_featuremask
shouldn't contain the bit in the first place.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 11:59:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 11:59:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686964.1069625 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rff4G-0005Gg-HC; Thu, 29 Feb 2024 11:59:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686964.1069625; Thu, 29 Feb 2024 11:59:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rff4G-0005GZ-Ef; Thu, 29 Feb 2024 11:59:16 +0000
Received: by outflank-mailman (input) for mailman id 686964;
 Thu, 29 Feb 2024 11:59:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rff4F-0005GT-HV
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 11:59:15 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f48df371-d6f9-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 12:59:13 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a293f2280c7so166239166b.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 03:59:13 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 kw11-20020a170907770b00b00a43f170ad9asm606233ejc.152.2024.02.29.03.59.12
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 03:59:12 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f48df371-d6f9-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709207953; x=1709812753; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=pPg34hk/0SnBk16LHps3PKPgFPhMSPoHpr+UeKsvq6E=;
        b=MqISVuuskMA79+MaEj2RqtRQcIYSlPbazp6ojGvvcUz0N3lGGCQB7VJyFm+a1ROrWf
         V3KozavhGwWNOWysHx5KD9MpHmyG2MC3ofEjvopWAom1hWpvEO7Q1ShYSCMegflYc/dH
         cS/hhhbOJHajHGAoz/Ynr/F66hLPa36f5fP7bElPk2XB3W9I3L6UwgY965k/y8t/dpVs
         w9dCRh9tBvsgcd7oKcx8/LdQbB2a5OTo7bvVhvSAhh7mgHV0uWEjJLRqmR6RbatNP4CZ
         FcaOYZV1WLiGfQ9D4rcSqW0PEfMpbgfhp0/XzXbTX06XsFYF+hPsS3T0Z4CCGYLwkFYc
         NUDA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709207953; x=1709812753;
        h=content-transfer-encoding:in-reply-to:autocrypt:references:cc:to
         :from:content-language:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=pPg34hk/0SnBk16LHps3PKPgFPhMSPoHpr+UeKsvq6E=;
        b=Fabw0okTuz7SB53l+AGn7iBVAe0wp9ChfYE4ItLMEERCK49F+h3DZXnhmEaKMgsEXI
         2jwg9txPbsVy/ATWFZcwEtQJ4ZWKTgDUI8fu09z/LD7xOk0/SQEnlWBcl2o5VNI9lWoP
         /nyrZNb+C6Xew6W+5kRljAaRVr2vBeSyB9X2xN6R3I4VadEfhNF8O9QBAF+sCQK+NMQB
         qaoVIh4EcZ+T99u84SvUuv0PmORuRIUJNJ45RVxfmwo3uMURSYkJhS8ZiceIsjlJBHz5
         WSLPyjyzOf2wxoyHZel3zawbQ4qmUqfBrmtdPkEs3Plvlo3be7V9aYXVEmjyXmtvXjQT
         gR7w==
X-Forwarded-Encrypted: i=1; AJvYcCUcM2TkxYI6FeXtlI9H+qXSHzckV26ZBxE4gguBm9XZQMT4nHa3WoV31B0jbMb9U/rMe8EvnrWEMWr1nmX7H5k/I4B/nVyMbTBb8IEAQuw=
X-Gm-Message-State: AOJu0YxPzH9mlqTsFA5DCfhr61HsfCJvh/CsjmvA2hFGDDa1rA2EG0Pq
	9RSs1zuGjgD9uijqcQlMs+QAEmZCjsO/w8d8hKvPaxYkZ4sTg/t2lU8vTNgzt6/aIFJ6JleoKvg
	=
X-Google-Smtp-Source: AGHT+IEekvDeYV00WxtnEDYvGVwqoMwnIDmz7LLro2Tg0kBdcQCtY6RJYXmUybdih3H5m8tSdAL8HQ==
X-Received: by 2002:a17:907:174b:b0:a44:1f3:e474 with SMTP id lf11-20020a170907174b00b00a4401f3e474mr1203974ejc.23.1709207953021;
        Thu, 29 Feb 2024 03:59:13 -0800 (PST)
Message-ID: <d3d18aff-64e1-4620-b538-e026afd4af17@suse.com>
Date: Thu, 29 Feb 2024 12:59:11 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
 <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
 <751bc87b-c8a4-4905-92bf-fc50f9102d4a@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <751bc87b-c8a4-4905-92bf-fc50f9102d4a@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 12:56, Jan Beulich wrote:
> On 29.02.2024 11:23, Julien Grall wrote:
>> On 29/02/2024 07:58, Jan Beulich wrote:
>>> (And
>>> no, upgrading the ancient distros on that ancient hardware is not an
>>> option for me.)
>>
>> May I ask why? Is it because newer distros don't support your HW?
> 
> Because as part of my job I also need to support ancient versions of
> Xen on ancient distros. Since I need to keep those around, it makes
> sense to me to then also test modern Xen there (every now and then, as
> said, and not really extensively).

Oh, and - because upgrading has proven to take quite a bit of time, if
also accounting for all the follow-on work that needs doing when parts
of the upgrade didn't quite go as intended. Whether newer distros
formally don't support such old hardware is just another possible
factor.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:05:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:05:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686986.1069635 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffAY-00076o-OO; Thu, 29 Feb 2024 12:05:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686986.1069635; Thu, 29 Feb 2024 12:05:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffAY-00076h-Li; Thu, 29 Feb 2024 12:05:46 +0000
Received: by outflank-mailman (input) for mailman id 686986;
 Thu, 29 Feb 2024 12:05:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rffAX-00076b-IB
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:05:45 +0000
Received: from mail-oa1-x2b.google.com (mail-oa1-x2b.google.com
 [2001:4860:4864:20::2b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id dcab9feb-d6fa-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 13:05:43 +0100 (CET)
Received: by mail-oa1-x2b.google.com with SMTP id
 586e51a60fabf-204235d0913so433753fac.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 04:05:43 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 e3-20020ac84e43000000b0042a9d64396esm636220qtw.74.2024.02.29.04.05.40
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 04:05:41 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dcab9feb-d6fa-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709208342; x=1709813142; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Q9JxPPytdh+qLVt/LIrS/j0sQiHi3Cz+7vBE29Vl4fg=;
        b=QnvQxzCYyUuXqJkPj6s4x15U/WQ1Y/SBuT52YYIkTr43TjqtBXEANjfFw7+0WVymyr
         S1Tc5UZpqtmX/K0WSY5p8YRJokve82NGaspDUmmH2iuuZIW37YIAMCRHRafG+ql4DBPZ
         qGPPR7M+JlA32V0Im2ncQnXRceOCRuzBpN1yE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709208342; x=1709813142;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Q9JxPPytdh+qLVt/LIrS/j0sQiHi3Cz+7vBE29Vl4fg=;
        b=I5wO/Jivepu/iKWqoLHTgLM0On0q7IMkIa+Q73j689FgF6snsf1jgUNU8qzrD1BC1H
         AIVPnSD9Xoz7PlCZ+9dNvo/9P2iKGWd4jiuCosfJyF8bv8WaDVf3CzHbEBVUrwQ6Rftq
         nVXOI26lugfArw5lgQ9ki1TtYgdHQAo7EZeJBzvNeX/lhXB/ACowTlBN0VfSfrTjJTKp
         GwsAsQrlrFnzv8A5/aoPYLlTmYr/OiWVFc5LJShYhf+UAHrb9mCfEjVh/h2CJNZE2g06
         2YtBy64wFj5MIJBMsRGeqpOyVmalM3nnG9SsqLCFkcj9OYQoWpXTaN4GNCJB73PMWVY6
         RP7g==
X-Forwarded-Encrypted: i=1; AJvYcCVd3/F9rGPuxLbMQxmfHmfcFRLC+tMKPv6olaIy4DJ+uKAt8GnUbB4Alg0hRdcP+m3SjB0cOfzYRlCTSxpWfIyKcptgfuizqspsA+s2/OY=
X-Gm-Message-State: AOJu0YyifIh5rQTLyKitQO0uljqddYHwGrCL449j5Q3I1K2PGUb855yQ
	3c6mPZDqNv27W2yu0rXshMtkHDTdLrThNbNGYgL3i9lHrO0btDJOpqF+vGnRr5o=
X-Google-Smtp-Source: AGHT+IEd9smskYDZOioEk/gETWsOQPv5Mj2T2sHYMXtuob9LdKK8GNnvUgl97N0HnmBoeW9q0I+BNg==
X-Received: by 2002:a05:6871:5828:b0:21f:17b4:3842 with SMTP id oj40-20020a056871582800b0021f17b43842mr1682168oac.45.1709208342195;
        Thu, 29 Feb 2024 04:05:42 -0800 (PST)
Message-ID: <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com>
Date: Thu, 29 Feb 2024 12:05:38 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-GB
To: Julien Grall <julien@xen.org>, Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
 <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/02/2024 10:23 am, Julien Grall wrote:
>>>> IOW it is hard for me to see why RISC-V needs stronger restrictions
>>>> here
>>>> than other architectures. It ought to be possible to determine a
>>>> baseline
>>>> version. Even if taking the desire to have "pause" available as a
>>>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
>>>
>>> I think we want to bump it on Arm. There are zero reasons to try to
>>> keep
>>> a lower versions if nobody tests/use it in production.
>>>
>>> I would suggest to do the same on x86. What's the point of try to
>>> support Xen with a 15+ years old compiler?
>>
>> It could have long been bumped if only a proper scheme to follow for
>> this and future bumping would have been put forward by anyone keen on
>> such bumping, like - see his reply - e.g. Andrew. You may recall that
>> this was discussed more than once on meetings, with no real outcome.
>> I'm personally not meaning to stand in the way of such bumping as long
>> as it's done in a predictable manner, but I'm not keen on doing so and
>> hence I don't view it as my obligation to try to invent a reasonable
>> scheme. (My personal view is that basic functionality should be
>> possible to have virtually everywhere, whereas for advanced stuff it
>> is fine to require a more modern tool chain.)
>
> That's one way to see it. The problem with this statement is a user
> today is mislead to think you can build Xen with any GCC versions
> since 4.1. I don't believe we can guarantee that and we are exposing
> our users to unnecessary risk.
>
> In addition to that, I agree with Andrew. This is preventing us to
> improve our code base and we have to carry hacks for older compilers.

I don't think anyone here is suggesting that we switch to a
bleeding-edge-only policy.  But 15y of support is extreme in the
opposite direction.

Xen ought to be buildable in the contemporary distros of the day, and I
don't think anyone is going to credibly argue otherwise.

But, it's also fine for new things to have newer requirements.

Take CET for example.  I know we have disagreements on exactly how it's
toolchain-conditionalness is implemented, but the basic principle of "If
you want shiny new optional feature $X, you need newer toolchain $Y" is
entirely fine.

A brand new architecture is exactly the same.  Saying "this is the
minimum, because it's what we test" doesn't preclude someone coming
along and saying "can we use $N-1 ?  See here it works, and here's a
change to CI test it".


Anyway, its clear we need to write some policy on this, before making
specific adjustments.  To get started, is there going to be any
objection whatsoever on some principles which begin as follows:

* For established architectures, we expect Xen to be buildable on the
common contemporary distros.  (i.e. minima is not newer than what's
available in contemporary distros, without a good reason)

* Optional features explicitly may have newer minima, generally chosen
by when toolchain support landed and/or was bugfixed suitably to be usable.

* Xen won't expect to update minima "just because".  But updates
across-the-board will be considered periodically where it doesn't
conflict with point 1, and where changing the minima allows us to use a
new feature to have a positive impact on the codebase.

* We always reserve the right to update minima to e.g. avoid crippling
code generation bugs, even if it conflicts with point 1.  Where
workarounds can reasonably be done, they ought to be preferred, but this
is ultimately at the discretion of the relevant architecture maintainers.

?

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:07:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:07:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686991.1069645 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffCY-0007dG-31; Thu, 29 Feb 2024 12:07:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686991.1069645; Thu, 29 Feb 2024 12:07:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffCY-0007d9-08; Thu, 29 Feb 2024 12:07:50 +0000
Received: by outflank-mailman (input) for mailman id 686991;
 Thu, 29 Feb 2024 12:07:49 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rffCX-0007cz-Cm; Thu, 29 Feb 2024 12:07:49 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rffCX-0003AV-AR; Thu, 29 Feb 2024 12:07:49 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rffCW-0000Jv-U6; Thu, 29 Feb 2024 12:07:49 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rffCW-000084-Tb; Thu, 29 Feb 2024 12:07:48 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=yPPhfimd4iuXzQnW/0qmbjTybdHpCtS6YNnCcUoH6mI=; b=iyZvDbCYxJZGwdXY/c3kDw4E9a
	gixvKrZS8ClWXiadv3XIh4Z6D/DzVHECakZI/v4xnL5QVNDkiqZFMENNWcHCMcBS3Kt4kRcK30Xm9
	0UOQS24QYuJtNdsfbcT8BZtnHkh043wAECxqzuPPXePLcYCeZdGLI6+fdxB1bFvPUm7s=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184818-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [ovmf test] 184818: all pass - PUSHED
X-Osstest-Versions-This:
    ovmf=dc7cfa9bab7487aa0cec02d13aa8c34ff24b37a8
X-Osstest-Versions-That:
    ovmf=d9a6e7b0b8a67392a57788d97634256546207a64
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 29 Feb 2024 12:07:48 +0000

flight 184818 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184818/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf                 dc7cfa9bab7487aa0cec02d13aa8c34ff24b37a8
baseline version:
 ovmf                 d9a6e7b0b8a67392a57788d97634256546207a64

Last test of basis   184815  2024-02-29 03:41:16 Z    0 days
Testing same since   184818  2024-02-29 10:11:15 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ard Biesheuvel <ardb@kernel.org>
  Gerd Hoffmann <kraxel@redhat.com>
  Laszlo Ersek <lersek@redhat.com>

jobs:
 build-amd64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   d9a6e7b0b8..dc7cfa9bab  dc7cfa9bab7487aa0cec02d13aa8c34ff24b37a8 -> xen-tested-master


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:12:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:12:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686995.1069655 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffGa-00017P-Jx; Thu, 29 Feb 2024 12:12:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686995.1069655; Thu, 29 Feb 2024 12:12:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffGa-00017I-HD; Thu, 29 Feb 2024 12:12:00 +0000
Received: by outflank-mailman (input) for mailman id 686995;
 Thu, 29 Feb 2024 12:11:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rffGZ-00017C-Hh
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:11:59 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bbebad56-d6fb-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 13:11:57 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a443d2b78caso95604766b.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 04:11:57 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 s22-20020a170906501600b00a42e2bc82dbsm615537ejj.169.2024.02.29.04.11.56
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 04:11:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bbebad56-d6fb-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709208717; x=1709813517; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=z4ltvyyAJimZey8Ue4oGEaTFAdTeP4wAJEl5/2woNww=;
        b=VM26dmoBq/Fqt7gVYycUAB92OUnlESRrDMMpE4NWZqzQ0smwfRkovjf3wWbCRpDZff
         b8xkzRiEc1YfJxy0PbODQLGYCev4JAE+cp4aJdZegf5JglBnoVv8uDd/rniaNlFlKjm3
         rAdPTw6w7XwW1926P29/Y7Cg5ooNxnD8VyFRr6yA1DivFeXDX/S7LJ1G5BoT9q9R7d+R
         yKoZICvDjQlHOfAnLuKjXgJ6qvFhf8MGZOKGezpg7mboNd3pYaTXHvQ2MjxomzEWZnd/
         NwH6Knda4VTsrWUcXl9e9K1NrtGrNsUOlw98iNeONXoxUumii3Q5SY1BJgSSWnKAg/y0
         g+qA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709208717; x=1709813517;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=z4ltvyyAJimZey8Ue4oGEaTFAdTeP4wAJEl5/2woNww=;
        b=nA+yFas0vUeHj5EC4hccK5bcmu9ZpftGU559mM3yhJ9NYznUKusGQBFtMrlN4YaPq8
         c/GaknMrY9FzEU4fS3zmVfxsZgwdaG/HMnK+XlH7WXGj0rPpYmGPwN3r1rmPSqra3O4N
         fYgGEi1V+oYe7BmGTL7gN0tbQnCb/Hfjk644x0CamNa+tFuQ4kM8H33qnQpbEXQwMNwe
         aZ4r1RAcI5B3HGESUSz98PUG/YO1C+k9+7uovgu1eNFb1uc8YPOvownSnoRE7sn3mM0t
         wW6TxyY9N5Gt9/BJBCRacpXyPJIMUwH9v7xQ3cuuYjqZ5QdIMPG3C3uTl2MheAw6b/xd
         osdQ==
X-Forwarded-Encrypted: i=1; AJvYcCWM79rkYtATVIPBxgxatyO7dRjq5lC/bmFzn8YgRGv6Y4GCqMebzUjlkqYccso7LibT3lc/3WT0mErCOhfar8l8XAZtfqNxnkwgOHuXWZE=
X-Gm-Message-State: AOJu0Yx+nnabMm3Lw2uw0nlLSDTxEXcwcVjuzwWBB9z7+qS/PDq6jA6s
	0Gy8BygdWR/sNjXagq4mxo7Z1e+nyP459+P+AXODboO0RetRZ7fR9zq1s2rgHg==
X-Google-Smtp-Source: AGHT+IFxY79TZMnXmb/qmm/11L4GNwAD0PVU9w4Rk91vfWE/xz5j4SIcLt9hg2KONAIIWFzaDvEFCA==
X-Received: by 2002:a17:906:e091:b0:a44:4fbd:a8fe with SMTP id gh17-20020a170906e09100b00a444fbda8femr847931ejb.1.1709208716924;
        Thu, 29 Feb 2024 04:11:56 -0800 (PST)
Message-ID: <780ec3ab-08e3-4fd8-a85f-0e89ea75dcc8@suse.com>
Date: Thu, 29 Feb 2024 13:11:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] README: bump minimum required clang/llvm version
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Doug Goldstein <cardoe@cardoe.com>, xen-devel@lists.xenproject.org
References: <20240229095529.17723-1-roger.pau@citrix.com>
 <20240229095529.17723-2-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240229095529.17723-2-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 10:55, Roger Pau Monne wrote:
> --- a/README
> +++ b/README
> @@ -41,7 +41,7 @@ provided by your OS distributor:
>          - GCC 4.1.2_20070115 or later
>          - GNU Binutils 2.16.91.0.5 or later
>          or
> -        - Clang/LLVM 3.5 or later
> +        - Clang/LLVM 14.0.0 or later

Wow, that's a big step. I'm build-testing with Clang7 on one system and
with Clang5 on another (and the latter more frequently than the former).
If any real dependency on this new a version (about 3 years old?) was
introduced, I would then no longer be able to locally test any Clang
builds (and hence the risk would again increase that I introduce issues
that affect just Clang builds).

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:14:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:14:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.686997.1069666 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffIc-0001ga-0T; Thu, 29 Feb 2024 12:14:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 686997.1069666; Thu, 29 Feb 2024 12:14:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffIb-0001gT-SC; Thu, 29 Feb 2024 12:14:05 +0000
Received: by outflank-mailman (input) for mailman id 686997;
 Thu, 29 Feb 2024 12:14:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aH8z=KG=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rffIb-0001gN-1r
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:14:05 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 065b7ef6-d6fc-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 13:14:03 +0100 (CET)
Received: from SN7PR04CA0220.namprd04.prod.outlook.com (2603:10b6:806:127::15)
 by SA3PR12MB7950.namprd12.prod.outlook.com (2603:10b6:806:31c::13)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.41; Thu, 29 Feb
 2024 12:13:59 +0000
Received: from SN1PEPF0002636E.namprd02.prod.outlook.com
 (2603:10b6:806:127:cafe::e2) by SN7PR04CA0220.outlook.office365.com
 (2603:10b6:806:127::15) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.30 via Frontend
 Transport; Thu, 29 Feb 2024 12:13:59 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SN1PEPF0002636E.mail.protection.outlook.com (10.167.241.139) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 29 Feb 2024 12:13:59 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 29 Feb
 2024 06:13:58 -0600
Received: from XIR-MICHALO-L1.xilinx.com (10.180.168.240) by
 SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35
 via Frontend Transport; Thu, 29 Feb 2024 06:13:56 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 065b7ef6-d6fc-11ee-afd8-a90da7624cb6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=fgwZGtiCcMMge7xnJQO0JWYuDE0pnjXoI99iBYkXlOy/zjT7exzHx6TiKkOMCoC3b8ulJt0FtBVqrrRdRC8bSv+B4C3kOFqa96wuBnh4o5VC3NeML8Y1EUxB5Kife+9hgqWpjgV/pBgYLOE+ZQVOI12XpEu9RwOrSvaujpckkCTUS8YlOWahCtqJ6HdkiJi2NmzxbmhNTGBeD2gfENpriJb8ZkEF445geIaSWbLLp3/uaHaHaG7Q9r0OiMlj0tO2HeIyyzj21T/mTbx4uBcDX2f1BRmGoXKjmaqACEtXODyZ3agMNWs+BLjpI7ttiuhijJsPM/2FfaLT1PnHBTBVcg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=hVr6b9i3p7tYsLB7Do102HaoN7SytS39uVfDOSQGWQw=;
 b=QF7zADPpF6Z9gGT7Jhi0sczmqHZGTIF/iybq9mK8Vtb1KinV25TxY7RO5vfVkQjmblrnlRuH/R3vfUN3xpdg3SD1ubZvPcrFlFNlRclnNUy4NOWjeVeKLxYmJ32Lc6WCpSqR3INMo72l7Sx/tk7m1G4bOpyFfu26eFkrkTt3F6hfuTWJTV3cAM4AnzsgLb1mg91QzzBB2Y9bZsP4XbS0fpuYwIVljHalmA6SWYFGy4Oo86H2gPqpUP5vXoeuHs7JY1w1OEtN4+FqFJuunzx4yLPufjbtixFo5L5Iv85n1McAw8tlFoxaJrbq8hs1+w6i2F5hXh9aJs0ay7yI5VRcAg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com;
 dmarc=pass (p=quarantine sp=quarantine pct=100) action=none
 header.from=amd.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=hVr6b9i3p7tYsLB7Do102HaoN7SytS39uVfDOSQGWQw=;
 b=q+KT7md55yI9DSjzJK8VuCh0S0vcDUhpfVtSQRItbmi9+28tv7r5VMHS0J3hOJdwz1ZqeHZrRu0e5kIHWKrB0p08rxbe/ycBmwN2D9kP3EZsMaBllKWIQCX/cQf7Mm4sOawhSp3mX8eMSaiN4MMafeCKT3UvSmq+eAsPfJrCK7A=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
From: Michal Orzel <michal.orzel@amd.com>
To: <xen-devel@lists.xenproject.org>
CC: Michal Orzel <michal.orzel@amd.com>, Andrew Cooper
	<andrew.cooper3@citrix.com>, George Dunlap <george.dunlap@citrix.com>, "Jan
 Beulich" <jbeulich@suse.com>, Julien Grall <julien@xen.org>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: [PATCH] SUPPORT.md: clarify support of booting 32-bit Xen on ARMv8
Date: Thu, 29 Feb 2024 13:13:52 +0100
Message-ID: <20240229121352.11264-1-michal.orzel@amd.com>
X-Mailer: git-send-email 2.25.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SN1PEPF0002636E:EE_|SA3PR12MB7950:EE_
X-MS-Office365-Filtering-Correlation-Id: 2915a05b-b973-4cee-05e5-08dc391fe8e8
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	15x22nPmqGnGVLv2hFZ9761BTe3iGCjzUF/OyawdJnP/7OAoC8k0Xi09erYYUlC6Y1ISUE1KEcSYh42KbXsBCYcEV0ZbwW7c99L0O9xwspuJ8RRYZRgiNeA1VgfXgVWikYCvdzzVJgvrKwIY5K4pAuyonmluSPMxmsuYVKgPohsYRGnIt+0yCPd5BJkPHMW680D0ItLUKD7tSzF4sFgRDYlRUFJeVJwPXj0YbA81eGOZpbOYaKmTMZiCUvHGmwAVw1+t8coIoX1F0SUa2BBB0EZ3nMqjJamT9jc4LIXhwHfiBRS9GpeDkzOO4Sc5VHBBNh3V3+I+EvhKK6U5A8Whe9Y9u++RB5+QRuqhDjK/Dv4I5pdhI7l+oPQZW+7P2+1aqrnDT9bXuazYVI3swTGkUq40XxeX5A6FU09r6fVmwZl3mKCNpkYaFQrfTeNRMYBj+6x2TnonNmYP1XvoLlzEVON+fo/UsVZjpZsltm3Rwqg76+UrdMwjVP6tstzFBWtgMSI/Ay8UjNr7NVJc0w3o17hVnp1QLPftqQhtyQx7prceaFHi1Mpz6KTJOjQUREvOPxYTzNLLbxeJJAvzkPJ0EzMLkL/j35r4q++JbYxDUOjIlFqA3t4qpChezSPEvK7hMZW1nW5CxrKVxjDGwPVmDc/Z14FDC6Y2iIWmpbwVZMBPnAH0m2HYg25qFwXXDzsFpnTAn3knzQvaP0AXW/2qdpGqUoy/+vyn0tegEWx2FEqI0nJcaLVmC2pI7v2iwbyL
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 12:13:59.3432
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2915a05b-b973-4cee-05e5-08dc391fe8e8
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF0002636E.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB7950

Since commit bd1001db0af1 ("xen/arm: arm32: Allow Xen to boot on
unidentified CPUs"), it's been possible to boot 32-bit Xen on ARMv8A CPUs
in AArch32 state (assuming HW supports EL2 execution in AArch32). Clarify
the support statement and mark it as Tech Preview, as this use case is
uncommon and hasn't really been tested/hardened.

Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
 SUPPORT.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/SUPPORT.md b/SUPPORT.md
index a90d1108c9d9..acc61230bb5e 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -40,6 +40,7 @@ supported in this document.
     Status: Supported
     Status, Cortex A57 r0p0-r1p1: Supported, not security supported
     Status, Cortex A77 r0p0-r1p0: Supported, not security supported
+    Status, Xen in AArch32 mode: Tech Preview
 
 For the Cortex A57 r0p0 - r1p1, see Errata 832075.
 For the Cortex A77 r0p0 - r1p0, see Errata 1508412.
-- 
2.25.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:17:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:17:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687003.1069675 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffLj-0002JC-HH; Thu, 29 Feb 2024 12:17:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687003.1069675; Thu, 29 Feb 2024 12:17:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffLj-0002J5-Ei; Thu, 29 Feb 2024 12:17:19 +0000
Received: by outflank-mailman (input) for mailman id 687003;
 Thu, 29 Feb 2024 12:17:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rffLi-0002Iz-8x
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:17:18 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 7a743c9d-d6fc-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 13:17:17 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a34c5ca2537so140613866b.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 04:17:17 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 x11-20020a170906710b00b00a4434e9938asm631439ejj.84.2024.02.29.04.17.15
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 04:17:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7a743c9d-d6fc-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709209036; x=1709813836; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=E29zZS9OS38WsEG/fXh9BRCCU3ulttk+8HpykdOIMtE=;
        b=fn2oOYiTqEA5G8qQuXI+PmD7xstpxPmRBP+AjVUz2yfrowma/vIA/AQJiLN2pWnXwp
         X7TZJPCXtAZbJpL6VuW56is9iYV17QqT6VxePeCutRF6QJcO0xoVNEHde+zU040jVDGu
         JfhdCtALnyOExGS50a5rZ0wF/JWN7ZGKZV2cSm/Xi1ByxaH0VqfJTIjgzDpwwt2YgEc2
         sqLej0x1sUsfIbIDQGVr9BhXWIOPlXIY9/Ns3sXaB/SDGVA+CK/GygyJiURwPY/WH2Uw
         PnE+7I7Dc/Oz9OpMboejmA2UK6j0TVCyH24DbWBixLWUglVR4oMqMwdsOvdf78kmBr3F
         US2g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709209036; x=1709813836;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=E29zZS9OS38WsEG/fXh9BRCCU3ulttk+8HpykdOIMtE=;
        b=aQ+6AG2R83DACKxK75tDYWc8HsIsHfTJZYlPsq2eEifen9YJasz4Yu3DI5Pymas548
         gULXJJPTvFmlZIv82jf94awMhHcs93RS48YzSCRZa59m2+FglYH+zQ2NDWDgizx/aOZV
         NQLS9uI6YfNX9InkoWRF9u6Uf5nlB3BHoluPbWkdXcU7/OW5ShhLgBt+6gTmtKbHNVOz
         d1qOrc4k8a+a2cd7aHlRFs9NtIxv4+zTO+wWax7MgDs5ONvrtx9VQclmrQqaFchYvBFT
         sdJs8492bfkotKYcg3rUxLsdX6ldUgXL3JOVzDmPwXszwHeZUf3GTZRIeorTaRmkrbU7
         AkxA==
X-Forwarded-Encrypted: i=1; AJvYcCXxU3wwAYEaqLtRBLLupl8yGpUzmbfVlXbePufhqf7os5g7Ro3E9GLV6JwBxvVM0sSVK/WZh9U37P8xI3TP8VYN/2j/nvbxU0bOQEtVI6U=
X-Gm-Message-State: AOJu0YyTm46ijo6HgdsUUJiqb3fBfOHSFNDIdK7nLshJz2Y0xORL9o65
	YRfUfRvNIftYzOQVvclzCPDc5RP0FfFRDN/aJIWygUePLgk26hKfvnkKUwxDzg==
X-Google-Smtp-Source: AGHT+IGChSSNgIfoxpE2Rfg6iyfYc8ePJzIsPPi0fU0xTE83tsXfRRSQpwvJB9Cbf7RIaxI8C8uxWg==
X-Received: by 2002:a17:907:118d:b0:a3e:5726:c12 with SMTP id uz13-20020a170907118d00b00a3e57260c12mr1343701ejb.37.1709209036572;
        Thu, 29 Feb 2024 04:17:16 -0800 (PST)
Message-ID: <2a112321-f4fd-4ab4-a431-a0bb4cb6eaa2@suse.com>
Date: Thu, 29 Feb 2024 13:17:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko
 <oleksii.kurochko@gmail.com>, Julien Grall <julien@xen.org>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
 <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
 <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 13:05, Andrew Cooper wrote:
> On 29/02/2024 10:23 am, Julien Grall wrote:
>>>>> IOW it is hard for me to see why RISC-V needs stronger restrictions
>>>>> here
>>>>> than other architectures. It ought to be possible to determine a
>>>>> baseline
>>>>> version. Even if taking the desire to have "pause" available as a
>>>>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
>>>>
>>>> I think we want to bump it on Arm. There are zero reasons to try to
>>>> keep
>>>> a lower versions if nobody tests/use it in production.
>>>>
>>>> I would suggest to do the same on x86. What's the point of try to
>>>> support Xen with a 15+ years old compiler?
>>>
>>> It could have long been bumped if only a proper scheme to follow for
>>> this and future bumping would have been put forward by anyone keen on
>>> such bumping, like - see his reply - e.g. Andrew. You may recall that
>>> this was discussed more than once on meetings, with no real outcome.
>>> I'm personally not meaning to stand in the way of such bumping as long
>>> as it's done in a predictable manner, but I'm not keen on doing so and
>>> hence I don't view it as my obligation to try to invent a reasonable
>>> scheme. (My personal view is that basic functionality should be
>>> possible to have virtually everywhere, whereas for advanced stuff it
>>> is fine to require a more modern tool chain.)
>>
>> That's one way to see it. The problem with this statement is a user
>> today is mislead to think you can build Xen with any GCC versions
>> since 4.1. I don't believe we can guarantee that and we are exposing
>> our users to unnecessary risk.
>>
>> In addition to that, I agree with Andrew. This is preventing us to
>> improve our code base and we have to carry hacks for older compilers.
> 
> I don't think anyone here is suggesting that we switch to a
> bleeding-edge-only policy.  But 15y of support is extreme in the
> opposite direction.
> 
> Xen ought to be buildable in the contemporary distros of the day, and I
> don't think anyone is going to credibly argue otherwise.
> 
> But, it's also fine for new things to have newer requirements.
> 
> Take CET for example.  I know we have disagreements on exactly how it's
> toolchain-conditionalness is implemented, but the basic principle of "If
> you want shiny new optional feature $X, you need newer toolchain $Y" is
> entirely fine.
> 
> A brand new architecture is exactly the same.  Saying "this is the
> minimum, because it's what we test" doesn't preclude someone coming
> along and saying "can we use $N-1 ?  See here it works, and here's a
> change to CI test it".
> 
> 
> Anyway, its clear we need to write some policy on this, before making
> specific adjustments.  To get started, is there going to be any
> objection whatsoever on some principles which begin as follows:

Largely not, but one aspect needs clarifying up front:

> * For established architectures, we expect Xen to be buildable on the
> common contemporary distros.  (i.e. minima is not newer than what's
> available in contemporary distros, without a good reason)

What counts as contemporary distro? Still in normal support? LTS? Yet
more extreme forms?

Plus - whose distros would we consider?

Jan

> * Optional features explicitly may have newer minima, generally chosen
> by when toolchain support landed and/or was bugfixed suitably to be usable.
> 
> * Xen won't expect to update minima "just because".  But updates
> across-the-board will be considered periodically where it doesn't
> conflict with point 1, and where changing the minima allows us to use a
> new feature to have a positive impact on the codebase.
> 
> * We always reserve the right to update minima to e.g. avoid crippling
> code generation bugs, even if it conflicts with point 1.  Where
> workarounds can reasonably be done, they ought to be preferred, but this
> is ultimately at the discretion of the relevant architecture maintainers.
> 
> ?
> 
> ~Andrew



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:27:54 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:27:54 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687007.1069685 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffVn-0004OQ-Eq; Thu, 29 Feb 2024 12:27:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687007.1069685; Thu, 29 Feb 2024 12:27:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffVn-0004OJ-CG; Thu, 29 Feb 2024 12:27:43 +0000
Received: by outflank-mailman (input) for mailman id 687007;
 Thu, 29 Feb 2024 12:27:42 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rffVm-0004OD-0z
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:27:42 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rffVk-0003cl-4f; Thu, 29 Feb 2024 12:27:40 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rffVj-00011q-S4; Thu, 29 Feb 2024 12:27:39 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=kFYCJO0UODbwDXJhr/9AMLUzqRfQrzD7DIO3d2rmI6Q=; b=CO6fVbwGIPI/kJy6wzgrurSqoy
	M9WlLvSahHQ71pv7tUXPuSEKihNPC5k3Vpj2eI6nXX0QrGgW3kNlWt7wHoUyLV9ejNAMiyNZOn3Vi
	t6GZQUZ5RmAYHA0u2K+YPdKXVH8oAXyrLuLhsw2PsuzJNv+1T7kGd/4Gyohjoe1ykeVw=;
Message-ID: <e817be8b-bef7-4702-87d3-3950cf176cf6@xen.org>
Date: Thu, 29 Feb 2024 12:27:38 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-GB
To: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
 <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
 <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Andrew,

On 29/02/2024 12:05, Andrew Cooper wrote:
> On 29/02/2024 10:23 am, Julien Grall wrote:
>>>>> IOW it is hard for me to see why RISC-V needs stronger restrictions
>>>>> here
>>>>> than other architectures. It ought to be possible to determine a
>>>>> baseline
>>>>> version. Even if taking the desire to have "pause" available as a
>>>>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
>>>>
>>>> I think we want to bump it on Arm. There are zero reasons to try to
>>>> keep
>>>> a lower versions if nobody tests/use it in production.
>>>>
>>>> I would suggest to do the same on x86. What's the point of try to
>>>> support Xen with a 15+ years old compiler?
>>>
>>> It could have long been bumped if only a proper scheme to follow for
>>> this and future bumping would have been put forward by anyone keen on
>>> such bumping, like - see his reply - e.g. Andrew. You may recall that
>>> this was discussed more than once on meetings, with no real outcome.
>>> I'm personally not meaning to stand in the way of such bumping as long
>>> as it's done in a predictable manner, but I'm not keen on doing so and
>>> hence I don't view it as my obligation to try to invent a reasonable
>>> scheme. (My personal view is that basic functionality should be
>>> possible to have virtually everywhere, whereas for advanced stuff it
>>> is fine to require a more modern tool chain.)
>>
>> That's one way to see it. The problem with this statement is a user
>> today is mislead to think you can build Xen with any GCC versions
>> since 4.1. I don't believe we can guarantee that and we are exposing
>> our users to unnecessary risk.
>>
>> In addition to that, I agree with Andrew. This is preventing us to
>> improve our code base and we have to carry hacks for older compilers.
> 
> I don't think anyone here is suggesting that we switch to a
> bleeding-edge-only policy.  But 15y of support is extreme in the
> opposite direction.
> 
> Xen ought to be buildable in the contemporary distros of the day, and I
> don't think anyone is going to credibly argue otherwise.
> 
> But, it's also fine for new things to have newer requirements.
> 
> Take CET for example.  I know we have disagreements on exactly how it's
> toolchain-conditionalness is implemented, but the basic principle of "If
> you want shiny new optional feature $X, you need newer toolchain $Y" is
> entirely fine.
> 
> A brand new architecture is exactly the same.  Saying "this is the
> minimum, because it's what we test" doesn't preclude someone coming
> along and saying "can we use $N-1 ?  See here it works, and here's a
> change to CI test it".
> 
> 
> Anyway, its clear we need to write some policy on this, before making
> specific adjustments.  To get started, is there going to be any
> objection whatsoever on some principles which begin as follows:

No objections.

> 
> * For established architectures, we expect Xen to be buildable on the
> common contemporary distros.  (i.e. minima is not newer than what's
> available in contemporary distros, without a good reason)

I think we would need to list the distros we are taking into account. 
Reading the rest of the principles, I am assuming you would be ok if new 
distros are added if there is a use case.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:32:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:32:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687012.1069696 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffaI-0006M2-WD; Thu, 29 Feb 2024 12:32:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687012.1069696; Thu, 29 Feb 2024 12:32:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffaI-0006Lv-TK; Thu, 29 Feb 2024 12:32:22 +0000
Received: by outflank-mailman (input) for mailman id 687012;
 Thu, 29 Feb 2024 12:32:21 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rffaH-0006Lp-00
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:32:21 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rffaF-0003jf-Nb; Thu, 29 Feb 2024 12:32:19 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rffaF-0001An-GY; Thu, 29 Feb 2024 12:32:19 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=yi26Zld1whDZBg0WiyiEQrqeOGpI97uU3msBEwI+Unc=; b=ZsncASid21OMOX2sfq/p0sr+/Q
	YQR3Noe4KtcbmfKr3OOhi87CXzfzmM0grFlY8E7cAikTBha8ycFglAQn4Mzrn30/fLqHzyN4pJktv
	QzMCO6kkpMTJNyOsGn50PjaYouvGhLgyMC8YyOom3y/2F81Q7Isd38cZ4omi3uhfH9Ps=;
Message-ID: <9b212dec-59ef-4cbb-bea7-add3867e4599@xen.org>
Date: Thu, 29 Feb 2024 12:32:17 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>, Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko <oleksii.kurochko@gmail.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
 <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
 <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com>
 <2a112321-f4fd-4ab4-a431-a0bb4cb6eaa2@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <2a112321-f4fd-4ab4-a431-a0bb4cb6eaa2@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

On 29/02/2024 12:17, Jan Beulich wrote:
> On 29.02.2024 13:05, Andrew Cooper wrote:
>> On 29/02/2024 10:23 am, Julien Grall wrote:
>>>>>> IOW it is hard for me to see why RISC-V needs stronger restrictions
>>>>>> here
>>>>>> than other architectures. It ought to be possible to determine a
>>>>>> baseline
>>>>>> version. Even if taking the desire to have "pause" available as a
>>>>>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
>>>>>
>>>>> I think we want to bump it on Arm. There are zero reasons to try to
>>>>> keep
>>>>> a lower versions if nobody tests/use it in production.
>>>>>
>>>>> I would suggest to do the same on x86. What's the point of try to
>>>>> support Xen with a 15+ years old compiler?
>>>>
>>>> It could have long been bumped if only a proper scheme to follow for
>>>> this and future bumping would have been put forward by anyone keen on
>>>> such bumping, like - see his reply - e.g. Andrew. You may recall that
>>>> this was discussed more than once on meetings, with no real outcome.
>>>> I'm personally not meaning to stand in the way of such bumping as long
>>>> as it's done in a predictable manner, but I'm not keen on doing so and
>>>> hence I don't view it as my obligation to try to invent a reasonable
>>>> scheme. (My personal view is that basic functionality should be
>>>> possible to have virtually everywhere, whereas for advanced stuff it
>>>> is fine to require a more modern tool chain.)
>>>
>>> That's one way to see it. The problem with this statement is a user
>>> today is mislead to think you can build Xen with any GCC versions
>>> since 4.1. I don't believe we can guarantee that and we are exposing
>>> our users to unnecessary risk.
>>>
>>> In addition to that, I agree with Andrew. This is preventing us to
>>> improve our code base and we have to carry hacks for older compilers.
>>
>> I don't think anyone here is suggesting that we switch to a
>> bleeding-edge-only policy.  But 15y of support is extreme in the
>> opposite direction.
>>
>> Xen ought to be buildable in the contemporary distros of the day, and I
>> don't think anyone is going to credibly argue otherwise.
>>
>> But, it's also fine for new things to have newer requirements.
>>
>> Take CET for example.  I know we have disagreements on exactly how it's
>> toolchain-conditionalness is implemented, but the basic principle of "If
>> you want shiny new optional feature $X, you need newer toolchain $Y" is
>> entirely fine.
>>
>> A brand new architecture is exactly the same.  Saying "this is the
>> minimum, because it's what we test" doesn't preclude someone coming
>> along and saying "can we use $N-1 ?  See here it works, and here's a
>> change to CI test it".
>>
>>
>> Anyway, its clear we need to write some policy on this, before making
>> specific adjustments.  To get started, is there going to be any
>> objection whatsoever on some principles which begin as follows:
> 
> Largely not, but one aspect needs clarifying up front:
> 
>> * For established architectures, we expect Xen to be buildable on the
>> common contemporary distros.  (i.e. minima is not newer than what's
>> available in contemporary distros, without a good reason)
> 
> What counts as contemporary distro? Still in normal support? LTS? Yet
> more extreme forms?

LTS makes sense. More I am not sure. I am under the impression that 
people using older distros are those that wants a stable system. So they 
would unlikely try to upgrade the hypervisor.

Even for LTS, I would argue that if it has been released 5 years ago, 
then you probably want to update it at the same time as moving to a 
newer Xen version.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:32:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:32:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687014.1069705 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffae-0006la-7d; Thu, 29 Feb 2024 12:32:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687014.1069705; Thu, 29 Feb 2024 12:32:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffae-0006lT-4j; Thu, 29 Feb 2024 12:32:44 +0000
Received: by outflank-mailman (input) for mailman id 687014;
 Thu, 29 Feb 2024 12:32:43 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aH8z=KG=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rffad-0006jA-9o
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:32:43 +0000
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
 (mail-bn8nam11on20601.outbound.protection.outlook.com
 [2a01:111:f403:2414::601])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a132d30c-d6fe-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 13:32:42 +0100 (CET)
Received: from SJ0PR03CA0179.namprd03.prod.outlook.com (2603:10b6:a03:338::34)
 by SJ0PR12MB5676.namprd12.prod.outlook.com (2603:10b6:a03:42e::8)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.24; Thu, 29 Feb
 2024 12:32:37 +0000
Received: from SJ1PEPF00001CE4.namprd03.prod.outlook.com
 (2603:10b6:a03:338:cafe::9d) by SJ0PR03CA0179.outlook.office365.com
 (2603:10b6:a03:338::34) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.32 via Frontend
 Transport; Thu, 29 Feb 2024 12:32:37 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 SJ1PEPF00001CE4.mail.protection.outlook.com (10.167.242.20) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 29 Feb 2024 12:32:37 +0000
Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 29 Feb
 2024 06:32:36 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB06.amd.com
 (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 29 Feb
 2024 06:32:36 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 29 Feb 2024 06:32:35 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a132d30c-d6fe-11ee-afd8-a90da7624cb6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=EIltx6Sz8sr5t+Ht/MdXpzOkYwyw1DsQv63EumAQWepxxj3rZIqQw4QjzZ2t+P13y/3nOKLoiNQ6cfszM3F2Cwyf0jEXK3js+srylU3jrFwfHNJjjYl1B+X02fPxqRyyCgREzHr5t/C3rFNgl5B2GT6BrnbU1mfdSSygpDfoUeoCSCC3uLax6XgTx5ofRobqLqhlpcyg4XeBy1K2pC8tck08lczomg91yE77C+QrwzlXR4FoC91qZyjdGpkHzl3HGwn1oehWP0yyebwlBGb56WPxpkbxrJYYfJ36x5eLFnII6Njen79TzEQVH3nKf8QI8DEcdnded+WpYx5gOXwB7Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=FlaPRp4dxBv1LE1vxL6Yk9Nptb5zzoJhHzbFlvkzh90=;
 b=isXVQ4P7z0sCDJm16JN5bhh/Nl6XJt+d1/mEadGmk3f1LfXQAJrYN1TjN3nmLDvtf7vm2tf26UJogrC7KT1s8FWGGvMEFAcH7OZ0KbNVXQfwMItkW7jiz+TBVT56rY8hkDk56wYRj7gMmQ+ezcHhkLE2vSwaLa0ohhI7qtQUhbwuv+yEkohf5RACJg6ppk1h6gpDBaRCu2BqlJQyIoWdRnEynGB/6L7tJKPcRChL36vWjGLXk9miGRjXez9PKkDHxpA4ZinE5evOHwG0SmmEHWOdBVJ9N1hXTGH55STZsvrZBD7mwMDH/vH0jjGxeP5RZCAaFiOma1rjguQC3YAHHw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=FlaPRp4dxBv1LE1vxL6Yk9Nptb5zzoJhHzbFlvkzh90=;
 b=lMgoikC1ZogQmNyGOk6Q988FRRnSNplF9i8BAX/jrdE5kJ/avgQt8TxlFwU2izhNEQRFUIo1XJI8cpbv+b4NaMLne6la8TiA/dcyYb20xi4LmFKhWd/uRccdUUSxPEGrQ78p3dOxb5Ouuze+8M4hv8XaFBV6ItgnqInsn8+TNXk=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <5671ee78-eeba-4bfc-bd95-dadb07c55cbd@amd.com>
Date: Thu, 29 Feb 2024 13:32:34 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] xen/arm: Fix arm32 build failure when early printk is
 enabled
Content-Language: en-US
To: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>
CC: <xen-devel@lists.xenproject.org>, Bertrand Marquis
	<bertrand.marquis@arm.com>, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
References: <20240228103555.172101-1-michal.orzel@amd.com>
 <9201bdc5-2e4c-48b7-8c92-f5063fde90b6@xen.org>
 <af11620a-ec16-4677-b968-d8e36c4555d2@amd.com>
 <alpine.DEB.2.22.394.2402281425260.575685@ubuntu-linux-20-04-desktop>
 <8228a48f-566c-4931-9f24-27bdaa1f0f28@amd.com>
 <99b74dda-225b-4402-abb9-fcccfda0adbc@xen.org>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <99b74dda-225b-4402-abb9-fcccfda0adbc@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CE4:EE_|SJ0PR12MB5676:EE_
X-MS-Office365-Filtering-Correlation-Id: 97b597ad-9f57-43a6-0417-08dc39228348
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	TMBcVLd3UonZt1BdDdjNqUswPmmSlpqSzLZe+M/EUx+zG5gDOa31FXxbrH4+c0HwsemQa39u9Yw9eFfexRKGgD4KmLbXz+02TgeZ9mAd9iY/5kenp9BydJLGdWJzbcz9gPNO0KbTDFkfFWF7Yl4TVHfl/ZPEjmroRaHy8hTUk2QmLlPqWr0vfGwAGAjWmL0ic21Av3RpmuKHbom160mWKXB8mr6AUDRH86Y+HxkS1D6d6tkE5UamWijMvhFwBb0I+ICCxhNrzLQgasLHrhs3RRg2JJIiHeqa+gVnEyi8gKLVAOnp4ntfRzOw5z+fcODfYQY2g/4rACagJ7z1Njs9AkPGBXcHYxKI2WV3A+OFN1NNkdPJhfrgg8BUsDjObra8DLCXw2RGCJN1XcOShkD1Rv9ideLmkKRpTteNGDisoqNMn7OIOaLtQMewlybm26Cz+IKH7XoEI+L5oUMpI2mh75e3vcH5JXkArnxXHt/I1Cf37sHX+Iz14QQlq7mI3b8w3ClrthY0y43yhfMCfI8ef11gPKum39khxoGjmrislBTKyTDfBa9CCvuQOJqZcVRaEUu/DxHiTb27WI7cdVok4SBgZfdilM6R2i8BCW5S5lBood5GHWmTF0uekghcOeevSu8zfhsOhrvMz6xOrDttzFHLdwPazyQSO/F1/xefaxRQgdHa3uPwqh4KHwHZcqdODy4b/aDSJheJY+c8TI/pk5sOwHY6z8IxdFhtG8aV7bq96XP3fsOatOsifQGxY30R
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 12:32:37.2880
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 97b597ad-9f57-43a6-0417-08dc39228348
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CE4.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB5676



On 29/02/2024 11:10, Julien Grall wrote:
> 
> 
> Hi,
> 
> On 29/02/2024 10:07, Michal Orzel wrote:
>>
>>
>> On 28/02/2024 23:27, Stefano Stabellini wrote:
>>>
>>>
>>> On Wed, 28 Feb 2024, Michal Orzel wrote:
>>>> Hi Julien,
>>>>
>>>> On 28/02/2024 12:42, Julien Grall wrote:
>>>>>
>>>>>
>>>>> Hi Michal,
>>>>>
>>>>> On 28/02/2024 10:35, Michal Orzel wrote:
>>>>>> Commit 0441c3acc7e9 forgot to rename FIXMAP_CONSOLE to FIX_CONSOLE in
>>>>>> TEMPORARY_EARLY_UART_VIRTUAL_ADDRESS macro. This results in a build
>>>>>> failure on arm32, when early printk is enabled:
>>>>>> arch/arm/arm32/mmu/head.S:311: Error: invalid operands (*UND* and *ABS* sections) for `*'
>>>>>
>>>>> Good catch! Somewhat related I wonder whether we should add earlyprintk
>>>>> testing in gitlab?
>>>> I thought about adding this and I think we should at least have build jobs (hypervisor only, no toolstack)
>>>> selecting early printk. When it comes to testing if early printk works, I'm not sure. It'd be nice
>>>> but FWIR we have limited bandwidth.
>>>>
>>>> @Stefano, what's your opinion?
>>>
>>> I think it would be a good and quick test to have. To save testing
>>> bandwidth I think we should reduce the amount of debug/non-debug
>>> variations of the same tests that we have.
>> Yes, I suggested that some time ago. We could keep both versions for generic tests,
>> but remove the non-debug version (unless you prefer to do the opposite) for:
> 
> I think it makes sense during development window to use the debug
> version. However, I think we want some non-debug testing during the
> hardening phase.
> 
> Can gitlab read CONFIG_DEBUG from Config.mk?
At the moment, we have 2 types of jobs - non debug and debug (with -debug suffix).
They set "debug" variable accordingly, which is used later on to modify .config:
echo "CONFIG_DEBUG=${debug}" >> xen/.config

Without this line, Xen would be built according to default value of CONFIG_DEBUG.
That said, I don't think we want to get back to this behavior.

If we want to save some bandwidth, we should make a decision whether to keep debug or non-debug versions.
x86 has both versions for build jobs and mostly debug test jobs.

~Michal


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:36:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:36:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687017.1069715 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffdk-0007V8-Qv; Thu, 29 Feb 2024 12:35:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687017.1069715; Thu, 29 Feb 2024 12:35:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffdk-0007V1-O1; Thu, 29 Feb 2024 12:35:56 +0000
Received: by outflank-mailman (input) for mailman id 687017;
 Thu, 29 Feb 2024 12:35:55 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rffdj-0007Uv-GE
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:35:55 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rffdi-0003ot-PV; Thu, 29 Feb 2024 12:35:54 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rffdi-0001VP-HS; Thu, 29 Feb 2024 12:35:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=aUOhniEb0JI2MdNf7CXOXR/davOYKdGJYIkfnl88fVA=; b=PrR6BIG3V486vnKd2ge07ezX16
	2Z/1LSUC1dH2Mj4SzFAwZMA1lDPjQ9zG+pparGIIEzSga0hrM+mbFmBTTibrYa/6wg7oHWqi6iDC5
	Fp0d1bjqgnWJKoshO3SVczn0+KeFZCodyEjENUqQ1HyKRKLPMbC4rzbGhBrFO/ZssYeA=;
Message-ID: <d1b7a4fd-0754-45f7-8c58-db36f0e94cfc@xen.org>
Date: Thu, 29 Feb 2024 12:35:52 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] SUPPORT.md: clarify support of booting 32-bit Xen on
 ARMv8
Content-Language: en-GB
To: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20240229121352.11264-1-michal.orzel@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <20240229121352.11264-1-michal.orzel@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Michal,

On 29/02/2024 12:13, Michal Orzel wrote:
> Since commit bd1001db0af1 ("xen/arm: arm32: Allow Xen to boot on
> unidentified CPUs"), it's been possible to boot 32-bit Xen on ARMv8A CPUs
> in AArch32 state (assuming HW supports EL2 execution in AArch32). Clarify
> the support statement and mark it as Tech Preview, as this use case is
> uncommon and hasn't really been tested/hardened.
> 
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> ---
>   SUPPORT.md | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/SUPPORT.md b/SUPPORT.md
> index a90d1108c9d9..acc61230bb5e 100644
> --- a/SUPPORT.md
> +++ b/SUPPORT.md
> @@ -40,6 +40,7 @@ supported in this document.
>       Status: Supported
I would consider to use 'Status, Xen in aarch64 mode: Supported' and then...

>       Status, Cortex A57 r0p0-r1p1: Supported, not security supported
>       Status, Cortex A77 r0p0-r1p0: Supported, not security supported
> +    Status, Xen in AArch32 mode: Tech Preview

... move this line closer. What do you think?

In any case, I am happy with the idea to clarify the SUPPORT state.

Acked-by: Julien Grall <jgrall@amazon.com>

>   
>   For the Cortex A57 r0p0 - r1p1, see Errata 832075.
>   For the Cortex A77 r0p0 - r1p0, see Errata 1508412.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:38:16 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:38:16 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687022.1069726 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfffy-00082o-6D; Thu, 29 Feb 2024 12:38:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687022.1069726; Thu, 29 Feb 2024 12:38:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfffy-00082h-3O; Thu, 29 Feb 2024 12:38:14 +0000
Received: by outflank-mailman (input) for mailman id 687022;
 Thu, 29 Feb 2024 12:38:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aH8z=KG=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rfffw-00082Z-CP
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:38:12 +0000
Received: from NAM04-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam04on20601.outbound.protection.outlook.com
 [2a01:111:f403:2409::601])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 64e2400a-d6ff-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 13:38:10 +0100 (CET)
Received: from SJ0PR03CA0231.namprd03.prod.outlook.com (2603:10b6:a03:39f::26)
 by CH3PR12MB9024.namprd12.prod.outlook.com (2603:10b6:610:176::9)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Thu, 29 Feb
 2024 12:38:07 +0000
Received: from SJ1PEPF00001CEA.namprd03.prod.outlook.com
 (2603:10b6:a03:39f:cafe::ea) by SJ0PR03CA0231.outlook.office365.com
 (2603:10b6:a03:39f::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.28 via Frontend
 Transport; Thu, 29 Feb 2024 12:38:06 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 SJ1PEPF00001CEA.mail.protection.outlook.com (10.167.242.26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 29 Feb 2024 12:38:06 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 29 Feb
 2024 06:38:05 -0600
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 29 Feb 2024 06:38:04 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 64e2400a-d6ff-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=dJQDSkbilx3/m8h3vsI6VPJKPgveIW/1/FMGBLqKTbVKWbNWu75X9TVM/9TE5Cx4FGQGEpc0jc3M273hqDF9KI/e4ojSPFXcyqmlWQamh5xQpFdO1Cu8hst1+PWOQy+9SqCnH2DG+tE8XVxUIBWFJND7QrjH7zhFoj+uT5KHaeTYRhVNSUt8DijhEKAbP5DVejcTDVuV7hmeqmb7fBdqgHGPmmRxTFg+2v4Z0B0FTVCtjUA4+N4pBk3FyTWN17f6dwc36SYlh+j1vjjZk/0xmbekoaSNKCZXHeQQXcGEtZby8lZyS4CD+KGPWSCuxaaHVVWZ1F4s3T6MKLA6fBQd9g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=GCDT5YnQejQia952pd3v+RqJGlrl5xNRXEoQQK1M5ek=;
 b=V5eUqHnDdH4BQGQijHO+pyU/DsX9nljgXeiaUFDBjw7+Ka71xml6xsO6nvKcbYtkSyEOqEumnfBU3fjSMLwW9ZxdF0ieWRIZVBw/os6uAMZDKuTb7VDhyDUPrZ1XUHUUOSKxcpkVwC+Wx2B7R5SSDJseU+Pl2oWasy4f3j32ioQOiP98h0jTU787fckM8+9rsMJkfpnPQvW6k74QN3AZSPsosNBS4FY4gSHIoORc1QXs3DevO0y3/xUtTLfQhk+o5EyAOfFwiXzFm484+MRvqf1md/SQ+56KMIZgKP5U8n3fUxHF52kV46E/Ibw+3jdEwx1IrarSSVeSgdB8izo95Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GCDT5YnQejQia952pd3v+RqJGlrl5xNRXEoQQK1M5ek=;
 b=Oql5uLoO+GcQRKPWyJ1huKvYjlEsi0yeYDBDCr9HA07ZHC/dMWel3nKp7vf8goXEMR0wRrU2jbtF5mAXzCwHme0r1qnqrmn9ULCcBMOoCoxum9HncRpIVgzVkZWRUSZlBN5H3+TxgoETlH8NZnGgu287lIiZ4row4XcNsaXEuyg=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <94176748-8438-460d-9434-6d18add55cef@amd.com>
Date: Thu, 29 Feb 2024 13:37:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] SUPPORT.md: clarify support of booting 32-bit Xen on
 ARMv8
Content-Language: en-US
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20240229121352.11264-1-michal.orzel@amd.com>
 <d1b7a4fd-0754-45f7-8c58-db36f0e94cfc@xen.org>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <d1b7a4fd-0754-45f7-8c58-db36f0e94cfc@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Received-SPF: None (SATLEXMB03.amd.com: michal.orzel@amd.com does not
 designate permitted sender hosts)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CEA:EE_|CH3PR12MB9024:EE_
X-MS-Office365-Filtering-Correlation-Id: 900772e1-165a-4917-b4d9-08dc3923474e
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	dsQtToHyy0ikv7aOZE5uYK4gz4QYnkY5kbG4Xeg0U+lYUmw5sY5gW8Ze8pLZhsS+Bc9KyxVCzpKL/IP4bvLaHe54dFEcSOw4D47XY6jwTdxCvxyl8G1Pkihgi9f/vf1l8ZBjqQMAXJF3mI0r4URFGdVIHqkOubHNSGD2lgWQJB+j0AWACfjiWosaYlrK4bZJoZuqSI9D20HD60VyJ2mvz3D13o7sdol/Rxi+PlXmSYQ1jPqxAMtSriFDNNbPfgps7j0td38v+xmw3Vl5+keJueHnGY+IFm7gT5xPFGk25tjFO9EqQXnlB4cMcVCy21ZnpD4elesLGBnrw8jY+Ap9qzwuZsh7fZylQrNr1XREz6Ua1j9qLT1jUoVSlRvMpDJCBYArws61czYeWuPn7gzbtkSGv3iZuLWgN+JSJDASaR7M6At0JmNtoy3Mz/pG9sJd2qB2T0CaWGi2Jyxpk1lPHJN4DiIRcbZxO1LFb7nwwWd+TLNb0scFWxsRSY+eJ/i1B0DBXVPTgS9ffHStKaOgbAoXykVLpHJ+HfUUxs4ysYBxnf0NMaZ9bPlazDi3q6yfcYYdU+ppDoRKjAgyVDDs7CdvbeMXXasxmcvZkk/pLxg0r98BjTrTF9040mHlt/2ccuE9Q/OJZJ4YuWaAdY9krZglQo5rHVz8RmR3sQ7l8gd0ZRa4LbfRBUZ0Grpf4QLlMsqGJ73oFiLJlj40Z3xY37gVXTspUSYiMSya8nNQQjHMXFsR7rAw1Y2yP/LGtVBp
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 12:38:06.1653
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 900772e1-165a-4917-b4d9-08dc3923474e
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SJ1PEPF00001CEA.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9024

Hi Julien,

On 29/02/2024 13:35, Julien Grall wrote:
> 
> 
> Hi Michal,
> 
> On 29/02/2024 12:13, Michal Orzel wrote:
>> Since commit bd1001db0af1 ("xen/arm: arm32: Allow Xen to boot on
>> unidentified CPUs"), it's been possible to boot 32-bit Xen on ARMv8A CPUs
>> in AArch32 state (assuming HW supports EL2 execution in AArch32). Clarify
>> the support statement and mark it as Tech Preview, as this use case is
>> uncommon and hasn't really been tested/hardened.
>>
>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>> ---
>>   SUPPORT.md | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/SUPPORT.md b/SUPPORT.md
>> index a90d1108c9d9..acc61230bb5e 100644
>> --- a/SUPPORT.md
>> +++ b/SUPPORT.md
>> @@ -40,6 +40,7 @@ supported in this document.
>>       Status: Supported
> I would consider to use 'Status, Xen in aarch64 mode: Supported' and then...
> 
>>       Status, Cortex A57 r0p0-r1p1: Supported, not security supported
>>       Status, Cortex A77 r0p0-r1p0: Supported, not security supported
>> +    Status, Xen in AArch32 mode: Tech Preview
> 
> ... move this line closer. What do you think?
That works for me too (+AArch64 instead of aarch64). Shall I respin the patch?

~Michal


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:40:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:40:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687024.1069736 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffi7-0001jU-I2; Thu, 29 Feb 2024 12:40:27 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687024.1069736; Thu, 29 Feb 2024 12:40:27 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffi7-0001jN-Ex; Thu, 29 Feb 2024 12:40:27 +0000
Received: by outflank-mailman (input) for mailman id 687024;
 Thu, 29 Feb 2024 12:40:26 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rffi6-0001i5-1D
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:40:26 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rffi5-0003ux-Ae; Thu, 29 Feb 2024 12:40:25 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rffi5-0001jj-4p; Thu, 29 Feb 2024 12:40:25 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=FWW3gYXOl+wmWx1lEhk6ME+giyRl+lKbMJzNjbterbA=; b=ul8WlqLLKH8HtlvXU9waR+euBt
	9aP49BAdgBHihohTH6wXebomLeXGzjaArv3isiZkUTYNzYyEHb4bUbGV8gCzUR7qu7eDDx4YOwknV
	jakGQ1DY6qfJJOLOlo0AYHrcxhnvMj6ZZEvGSnRkY4RJoRcbI2UG0Jpa6VmxgQyyt3iQ=;
Message-ID: <544718e1-4855-4af1-a450-6362a8cd08f0@xen.org>
Date: Thu, 29 Feb 2024 12:40:22 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] SUPPORT.md: clarify support of booting 32-bit Xen on
 ARMv8
Content-Language: en-GB
To: Michal Orzel <michal.orzel@amd.com>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20240229121352.11264-1-michal.orzel@amd.com>
 <d1b7a4fd-0754-45f7-8c58-db36f0e94cfc@xen.org>
 <94176748-8438-460d-9434-6d18add55cef@amd.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <94176748-8438-460d-9434-6d18add55cef@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit



On 29/02/2024 12:37, Michal Orzel wrote:
> Hi Julien,

Hi Michal,

> 
> On 29/02/2024 13:35, Julien Grall wrote:
>> On 29/02/2024 12:13, Michal Orzel wrote:
>>> Since commit bd1001db0af1 ("xen/arm: arm32: Allow Xen to boot on
>>> unidentified CPUs"), it's been possible to boot 32-bit Xen on ARMv8A CPUs
>>> in AArch32 state (assuming HW supports EL2 execution in AArch32). Clarify
>>> the support statement and mark it as Tech Preview, as this use case is
>>> uncommon and hasn't really been tested/hardened.
>>>
>>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>>> ---
>>>    SUPPORT.md | 1 +
>>>    1 file changed, 1 insertion(+)
>>>
>>> diff --git a/SUPPORT.md b/SUPPORT.md
>>> index a90d1108c9d9..acc61230bb5e 100644
>>> --- a/SUPPORT.md
>>> +++ b/SUPPORT.md
>>> @@ -40,6 +40,7 @@ supported in this document.
>>>        Status: Supported
>> I would consider to use 'Status, Xen in aarch64 mode: Supported' and then...
>>
>>>        Status, Cortex A57 r0p0-r1p1: Supported, not security supported
>>>        Status, Cortex A77 r0p0-r1p0: Supported, not security supported
>>> +    Status, Xen in AArch32 mode: Tech Preview
>>
>> ... move this line closer. What do you think?
> That works for me too (+AArch64 instead of aarch64).

Ah yes. I keep forgetting capitalizing properly :).

> Shall I respin the patch?

Up to you. I am happy to fix it. But I will wait a day or two just to 
give a chance for the others to comment.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:40:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:40:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687026.1069746 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffiS-0002AB-Qv; Thu, 29 Feb 2024 12:40:48 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687026.1069746; Thu, 29 Feb 2024 12:40:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffiS-0002A4-NE; Thu, 29 Feb 2024 12:40:48 +0000
Received: by outflank-mailman (input) for mailman id 687026;
 Thu, 29 Feb 2024 12:40:46 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xh/y=KG=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rffiQ-0001yM-Rm
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:40:46 +0000
Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com
 [2607:f8b0:4864:20::733])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c22036aa-d6ff-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 13:40:46 +0100 (CET)
Received: by mail-qk1-x733.google.com with SMTP id
 af79cd13be357-787be5d999aso50871485a.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 04:40:46 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 h12-20020a37c44c000000b00787d46d606csm622722qkm.128.2024.02.29.04.40.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 04:40:44 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c22036aa-d6ff-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709210445; x=1709815245; darn=lists.xenproject.org;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;
        bh=U8O5dFQWIYfvrKnM0mHF87nY6TM28+0Hl6PaX6zmGOw=;
        b=kk/fpJs97f51bvYqqYguv4Og1m1KsCG0NtLs4IAix/nm4VxROL2aNdoqxUZOEsA+QS
         PuqB+IlACf661sPB9SchKg+4hWx3xfPzXGtgLaoD8B7Sm2j53rx6bffKPUJUA2+U4gf5
         CmwAX1Q++/XeFa5fNPSWEl61U41hya7R17Uu8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709210445; x=1709815245;
        h=in-reply-to:content-disposition:mime-version:references:message-id
         :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=U8O5dFQWIYfvrKnM0mHF87nY6TM28+0Hl6PaX6zmGOw=;
        b=hsLNmAsAstLK/emsh1DZ5u6tH3o7Wvsfz3nZcvFz8oU9M7JUXOczztk0MTRy1St4ng
         l6v8CfMdH7+9c7A9e1FRn1nARg9iCo/L6m7uLac4hnXc0JHZc0ro8WjpRZ2N8EZH2Ulm
         pDSua1PHzsf/W0i8cEEiWHuctiafUIMoo0uRINiN7T6ZXPVwZ2abE2K5NcS85hoEeklm
         FtHO5NF2xvmPe+PDl7aOPyoV1V/1toKfdwjeW5PImbrSwarGvKSb7OPmBAJj1IL3HnsA
         7RoFnMCFV9EjKVcR0r5Y43xVGmNppImT5xI8BOsHaxwaqArwXRUVlZDGMA4+Jcaqu+IA
         VJlg==
X-Forwarded-Encrypted: i=1; AJvYcCW6ZHXBVKSQPdNEp89A6SiBxPOlm6ivd9XpWZ1lm+o+L3b2V/dnwkBo9bc2uu+CHCkFQ4SNb39OOjErPhP64/QVOYhOqPHkbVy1czstg7o=
X-Gm-Message-State: AOJu0YwbU5J/6f3zoqH7C8vhiR7m8m+GsN5z3PdZ0D+uP6zooNrrYwS0
	PKYzr7KkHsjEOHVF7+n2v63Ik9Hk/PMdFyNgkeXtLUI9DoOAp2sEKPmsXpcMKMU=
X-Google-Smtp-Source: AGHT+IELKHvvM+bwREt9wRpL0aZI6wZFfoHOHxe2+gqO/dLkIkGock/P+FkCwoG2sHRIrHNqiDD3Qg==
X-Received: by 2002:a05:620a:16b3:b0:787:8eca:dd13 with SMTP id s19-20020a05620a16b300b007878ecadd13mr1885971qkj.69.1709210445217;
        Thu, 29 Feb 2024 04:40:45 -0800 (PST)
Date: Thu, 29 Feb 2024 13:40:43 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	Doug Goldstein <cardoe@cardoe.com>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH 1/2] README: bump minimum required clang/llvm version
Message-ID: <ZeB7S6i7pIwzkUNE@macbook>
References: <20240229095529.17723-1-roger.pau@citrix.com>
 <20240229095529.17723-2-roger.pau@citrix.com>
 <780ec3ab-08e3-4fd8-a85f-0e89ea75dcc8@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <780ec3ab-08e3-4fd8-a85f-0e89ea75dcc8@suse.com>

On Thu, Feb 29, 2024 at 01:11:55PM +0100, Jan Beulich wrote:
> On 29.02.2024 10:55, Roger Pau Monne wrote:
> > --- a/README
> > +++ b/README
> > @@ -41,7 +41,7 @@ provided by your OS distributor:
> >          - GCC 4.1.2_20070115 or later
> >          - GNU Binutils 2.16.91.0.5 or later
> >          or
> > -        - Clang/LLVM 3.5 or later
> > +        - Clang/LLVM 14.0.0 or later
> 
> Wow, that's a big step. I'm build-testing with Clang7 on one system and
> with Clang5 on another (and the latter more frequently than the former).
> If any real dependency on this new a version (about 3 years old?) was
> introduced, I would then no longer be able to locally test any Clang
> builds (and hence the risk would again increase that I introduce issues
> that affect just Clang builds).

Would it be possible for you to update to a newer version?  I see both
the OpenSUSE containers in Gitlab have newer versions of Clang.

The reasoning for selecting such versions is in the commit message, I
simply see no point in maintaining support for older Clang versions if
there's no-one that actually runs such builds.  Just listing them as
supported for being able to output a binary that's never run tested is
not helpful.

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:42:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:42:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687029.1069756 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffju-0002lW-5N; Thu, 29 Feb 2024 12:42:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687029.1069756; Thu, 29 Feb 2024 12:42:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffju-0002lP-1m; Thu, 29 Feb 2024 12:42:18 +0000
Received: by outflank-mailman (input) for mailman id 687029;
 Thu, 29 Feb 2024 12:42:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=aH8z=KG=amd.com=Michal.Orzel@srs-se1.protection.inumbo.net>)
 id 1rffjs-0002lH-Tn
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:42:16 +0000
Received: from NAM12-MW2-obe.outbound.protection.outlook.com
 (mail-mw2nam12on20600.outbound.protection.outlook.com
 [2a01:111:f403:200a::600])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id f64b7f02-d6ff-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 13:42:14 +0100 (CET)
Received: from BL0PR01CA0029.prod.exchangelabs.com (2603:10b6:208:71::42) by
 IA1PR12MB6649.namprd12.prod.outlook.com (2603:10b6:208:3a2::6) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.7316.39; Thu, 29 Feb 2024 12:42:10 +0000
Received: from BL02EPF0001A108.namprd05.prod.outlook.com
 (2603:10b6:208:71:cafe::6b) by BL0PR01CA0029.outlook.office365.com
 (2603:10b6:208:71::42) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.32 via Frontend
 Transport; Thu, 29 Feb 2024 12:42:10 +0000
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BL02EPF0001A108.mail.protection.outlook.com (10.167.241.138) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 29 Feb 2024 12:42:09 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 29 Feb
 2024 06:42:09 -0600
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 29 Feb
 2024 04:42:09 -0800
Received: from [10.252.147.188] (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 29 Feb 2024 06:42:07 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f64b7f02-d6ff-11ee-a1ee-f123f15fe8a2
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ffE9xrQH2Klfl4B00RouhcSdKwYBg0LPc4CVtV1yQz75N+P+6p/r1U3f3eY9A5FOTAbPAirZzUn1tFSSpjMnL70sINR2mai3u+eXYLA72qp05q9kkEYo9GKrn56C3KW4x3ocIgsMDD7ErcsJKaR0s2HgOrC4ZIa982INX4067hNX1H24q49mm0FiYhxqCJMECEGCfDlNDZ6TCO/qyAwsNvYQOZfxVJdrQ+7g2PyGE2Mw3+2k82/h781IdFCBAJmWAuyv8D82RgAV7p3IkqHj9buZPbs/y/7d/reBvtA/VdkHae/6npkV9Pyc9RrK/nOQVe5yLB1hQRHNetTCdFl0+g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=aR+/1AD3CcTgZburua/5KcSDzg4uv6mZ0FR95VNhb44=;
 b=EArdR3jfmAx+05yN78+0QkkaDcHKiLxzIwydiJ9JScYRXoTn+lo/6vT2XFGuotoucgnq9WB6z/PADw03wogO/sGJNBwHHqMcgAHsfQ8i8ZNPMQoDY4YIhQ9CBS9QDPqEdQCUHxFLMXgOFiHbUgso/6oZejC82/XHhPlPdEc+G+ZgH3+7Qz6xZ4WLhxsB7qfxF467ICwkluj+w4v1AvzFkMI1y7G+VcEdvXbhraV6noj1qvKqfRGF9T46+rp8mkE5OZNnjUqlPnoo6MitwWzFQBxL2x++d0rVZ850fDC/f6/viu0j1Sb/geCzIORwlFABCy5lAbMz2k6BOWmoxulruQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=xen.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=aR+/1AD3CcTgZburua/5KcSDzg4uv6mZ0FR95VNhb44=;
 b=r2+9K8rriAZT0OrNS+Urj57+vONvp2mIjMaYjvK5UgsAuporIz1nOdpXcwXFb1M/iXnRe/soEcv0CYFyRILUr0QcEybt5hQyxCMeXdOk16irrjcyNp9mzroNn6YSSxs02phXS1YICDvpsIQewXw+5bLYQ1us05VYFee+bidfV5c=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Message-ID: <216e9119-a7f9-43f3-80f9-f1c15685cad3@amd.com>
Date: Thu, 29 Feb 2024 13:42:07 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] SUPPORT.md: clarify support of booting 32-bit Xen on
 ARMv8
Content-Language: en-US
To: Julien Grall <julien@xen.org>, <xen-devel@lists.xenproject.org>
CC: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
	<george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, "Stefano
 Stabellini" <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <20240229121352.11264-1-michal.orzel@amd.com>
 <d1b7a4fd-0754-45f7-8c58-db36f0e94cfc@xen.org>
 <94176748-8438-460d-9434-6d18add55cef@amd.com>
 <544718e1-4855-4af1-a450-6362a8cd08f0@xen.org>
From: Michal Orzel <michal.orzel@amd.com>
In-Reply-To: <544718e1-4855-4af1-a450-6362a8cd08f0@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BL02EPF0001A108:EE_|IA1PR12MB6649:EE_
X-MS-Office365-Filtering-Correlation-Id: ced1f797-014b-45f6-ef40-08dc3923d886
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	8gdnVajq/gKzVfQ+e5PcmVlV3Xhca8+SXXRiHlEBYAoBOqbLno3a16n28KzAEBq6WqMIGfq9vydhFUc/Xvpu16k5Q2gYzZnG1IcUEQjqRRh3Udbnr/ZgRoSAUzUpBRQjoBCUqNU4NxH9EEpA3l62Xo6aiiSU1XK6dxg/aQ4NSeqeoLUy1RIeNhwbnVeWd5DiU6R5b9vxpjxn5yW3fNI/ETQ3FxzaOSpmgwtPzDP60Mwv+HWhWOlV1g76eMkuv79caykK4aLMyoQPlk/v3gsDnBvP57hPO7j9lR1CJGcavE1a3JoFT50K6gOdWbXmLU5lEzz3Nd4kN8DQ5j4poJro73q0XyWc6H2mG7ThsUqC+yH5vzWKBUQ0ZSXtA4hmSyb1BkHteZRlLxieFh6MmXOFssvcQIy/mrieVyQC5P6XjeQo1CFGFcMtJ1FJUG3NBT7We0jhqRq7Uq5RU7YAQlVk+28kY4fWt3fhs+6DYsYBFiCO8/WHz7nZZQHyvP1r8xSjXoCuz+HJA/8mojUNP0sRWJ/Fo0ShRTEVVd0dl0nzd7zIBRtK4Trj33Rx0EGFNPCTpu7TYprXNgxqVJPSVxXQc69UPGOsp97EGEVfNYjoqYNfNxi1wojaRyRf3QMxOzuziJHEgHghmWYAo2T2VB0hXAzYGC7gTYz326+R/8/9Pj3vS/2526di4NZq3sE3umdh1q30cmS+1x4VjF+bWthao+vVYHCazMlSb9kqxVCDciOrqjvdbjobZLCfGRHz83yA
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 12:42:09.8815
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ced1f797-014b-45f6-ef40-08dc3923d886
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	BL02EPF0001A108.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6649



On 29/02/2024 13:40, Julien Grall wrote:
> 
> 
> On 29/02/2024 12:37, Michal Orzel wrote:
>> Hi Julien,
> 
> Hi Michal,
> 
>>
>> On 29/02/2024 13:35, Julien Grall wrote:
>>> On 29/02/2024 12:13, Michal Orzel wrote:
>>>> Since commit bd1001db0af1 ("xen/arm: arm32: Allow Xen to boot on
>>>> unidentified CPUs"), it's been possible to boot 32-bit Xen on ARMv8A CPUs
>>>> in AArch32 state (assuming HW supports EL2 execution in AArch32). Clarify
>>>> the support statement and mark it as Tech Preview, as this use case is
>>>> uncommon and hasn't really been tested/hardened.
>>>>
>>>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
>>>> ---
>>>>    SUPPORT.md | 1 +
>>>>    1 file changed, 1 insertion(+)
>>>>
>>>> diff --git a/SUPPORT.md b/SUPPORT.md
>>>> index a90d1108c9d9..acc61230bb5e 100644
>>>> --- a/SUPPORT.md
>>>> +++ b/SUPPORT.md
>>>> @@ -40,6 +40,7 @@ supported in this document.
>>>>        Status: Supported
>>> I would consider to use 'Status, Xen in aarch64 mode: Supported' and then...
>>>
>>>>        Status, Cortex A57 r0p0-r1p1: Supported, not security supported
>>>>        Status, Cortex A77 r0p0-r1p0: Supported, not security supported
>>>> +    Status, Xen in AArch32 mode: Tech Preview
>>>
>>> ... move this line closer. What do you think?
>> That works for me too (+AArch64 instead of aarch64).
> 
> Ah yes. I keep forgetting capitalizing properly :).
> 
>> Shall I respin the patch?
> 
> Up to you. I am happy to fix it. But I will wait a day or two just to
> give a chance for the others to comment.
That works for me, let's wait for Bertrand and Stefano.

~Michal


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:47:57 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:47:57 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687037.1069766 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffpJ-0003T0-SU; Thu, 29 Feb 2024 12:47:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687037.1069766; Thu, 29 Feb 2024 12:47:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffpJ-0003St-Pj; Thu, 29 Feb 2024 12:47:53 +0000
Received: by outflank-mailman (input) for mailman id 687037;
 Thu, 29 Feb 2024 12:47:52 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rffpI-0003Sn-Qd
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:47:52 +0000
Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com
 [2a00:1450:4864:20::62f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id bf60a643-d700-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 13:47:50 +0100 (CET)
Received: by mail-ej1-x62f.google.com with SMTP id
 a640c23a62f3a-a44628725e3so22434466b.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 04:47:50 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h4-20020a1709063b4400b00a433f470cf1sm648666ejf.138.2024.02.29.04.47.49
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 04:47:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: bf60a643-d700-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709210870; x=1709815670; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=5Ze8Gl+c5m0bhaEc2tOK4/IUPw2m7dVfUmIkxjX/BKw=;
        b=XYfguade4+qeZFAfM0OMGR1/X3R7QiD4I7DjlQ8r3Njip3lwpmK17+J5Dy6fx2AUN8
         4vCqdqzIffNCvZmygelP8ODow2N2Ab374I7MIVNVlaQRSW5GKVg6MFAhLZUd2++JXyRf
         XR5Lpl14ivNZbJxxRFbm7GZaeqfbI7ZGkfmuvvJodzougmmf44LYzB4lM6wXQslhyCj+
         1aI85vKnYcPkvnO1leklw7dGexWTVmcPXrKNQmexqfmmjWzqu4DsilAMlvZerEyDmS8h
         XYz45zq/ej2FeEfwdoItbTcLIPG3cQlRISQTC/NfI63lCJgGDYc8nmUpZilR0ru59g1e
         KkKA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709210870; x=1709815670;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=5Ze8Gl+c5m0bhaEc2tOK4/IUPw2m7dVfUmIkxjX/BKw=;
        b=lXMpfJ9CYevgJD2KIQ2Yjl7aJ8hWBEDG2uEIhEqxHaIMsyFlEbDkm7Vxu1MlJn7krD
         KMHZzu4ryODg5lyp4PIDsiAwJCnW90y1gGuRWTmsgzQNa0PK4lSrCSu6mLRtoGmKHm0w
         tFcJdttvWuMkvbOuaNCsR6fdI2cMPekhuFFkTXEPpsHQ+u+vydYPdc1aEYe0sty0GTD8
         PB81++ZzYwkPYb9x7zhs6eKTpG+rTogXxl1Xg9pCvO3wld8LO9pqOCBeJaAmc4SaB/ZS
         kNm+37LnNyBVIBNFJAzAI6MbVOC6J+4iCqkpEjmYM3aHDdi0l6bJjurU8NcARdsZ5WVR
         5Flg==
X-Forwarded-Encrypted: i=1; AJvYcCXFtfOSoYvyreiEb9LwFpQuK3rurQZw3VxJHdebIWM7K0pNJVuuBtgylhBi7p6WlXJ/9Jp7OIRUhlSI5og3gVJeVghfZz5dFxZKGZeo5b4=
X-Gm-Message-State: AOJu0Yxt2C+3jAAlbyeyR4MVAE7fkz+ZUZs/3yFUNtHeNx0BvzvESlWq
	rJOMtwkZXmHmz/ldv+lSeddGdEnX3FgnpzeiXs3Cx62FopLlIzXI67OeOiiM5v2r6aKpR4B4pE4
	=
X-Google-Smtp-Source: AGHT+IFHn201zo35Usxx1CqEBgm4c8z9VNJgcf4hmP7t39TBbHGV2rQCACoTcW5zSsbZYppIjwZ6Eg==
X-Received: by 2002:a17:906:aeca:b0:a43:4e75:21ac with SMTP id me10-20020a170906aeca00b00a434e7521acmr1299893ejb.52.1709210870132;
        Thu, 29 Feb 2024 04:47:50 -0800 (PST)
Message-ID: <707ff486-b448-4bc0-be7e-1c692b360734@suse.com>
Date: Thu, 29 Feb 2024 13:47:48 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpu-policy: Fix x2APIC visibility for PV guests
Content-Language: en-US
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240229104304.2478614-1-andrew.cooper3@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240229104304.2478614-1-andrew.cooper3@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 11:43, Andrew Cooper wrote:
> Right now, the host x2APIC setting filters into the PV max and default
> policies, yet PV guests cannot set MSR_APIC_BASE.EXTD or access any of the
> x2APIC MSR range.  Therefore they absolutely shouldn't see the x2APIC bit.
> 
> Linux has workarounds for the collateral damage caused by this leakage; it
> unconditionally filters out the x2APIC CPUID bit, and EXTD when reading
> MSR_APIC_BASE.
> 
> Hide the x2APIC bit in the PV default policy, but for compatibility, tolerate
> incoming VMs which already saw the bit.  This is logic from before the
> default/max split in Xen 4.14 which wasn't correctly adjusted at the time.

What about guest_cpuid()'s handling of leaf 0xb then? The %edx value
will change once a guest is rebooted, aiui. The comment in
recalculate_cpuid_policy() that you update refers to that.

> Update the annotation from !A to !S which slightly better describes that it
> doesn't really exist in PV guests.  HVM guests, for which x2APIC can be
> emulated completely, already has it unconditionally set in the max policy.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Wei Liu <wl@xen.org>
> 
> This wants backporting as far as people can tollerate, but it's really not
> obvious which commit in 4.14 should be referenced in a Fixes: tag.

Why 4.14? In 4.7.0 I see ...

> @@ -830,11 +846,10 @@ void recalculate_cpuid_policy(struct domain *d)
>      }
>  
>      /*
> -     * Allow the toolstack to set HTT, X2APIC and CMP_LEGACY.  These bits
> +     * Allow the toolstack to set HTT and CMP_LEGACY.  These bits
>       * affect how to interpret topology information in other cpuid leaves.
>       */
>      __set_bit(X86_FEATURE_HTT, max_fs);
> -    __set_bit(X86_FEATURE_X2APIC, max_fs);
>      __set_bit(X86_FEATURE_CMP_LEGACY, max_fs);
>  
>      /*

... these adjustments, just still in calculate_pv_featureset(). I
haven't gone further backwards to check if/when this exposure has
really appeared. I wouldn't be surprised if it's been like that
for all the time since we gained x2APIC support in the hypervisor.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:48:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:48:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687039.1069776 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffq1-0003yW-4K; Thu, 29 Feb 2024 12:48:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687039.1069776; Thu, 29 Feb 2024 12:48:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffq1-0003yP-1N; Thu, 29 Feb 2024 12:48:37 +0000
Received: by outflank-mailman (input) for mailman id 687039;
 Thu, 29 Feb 2024 12:48:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rffpz-0003yF-6c
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:48:35 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d8929e30-d700-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 13:48:33 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 625361F7E8;
 Thu, 29 Feb 2024 12:48:32 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id F3ACF1329E;
 Thu, 29 Feb 2024 12:48:31 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id jHdROR994GXvIQAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 12:48:31 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d8929e30-d700-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210912; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=3LpVg4VR4Av9lRaWgZwCKN2sTIaBmuVNnNx2pKsXnAQ=;
	b=ez5EV45Q4D9+xFRUXrbBc1zEca1qn8clECUCYT05wNlagrpYoljdhUa8Qfr4g77XhJRTBU
	ZEjitWKtItm/a6py8tGoS36Yw/bI48w9MgfDpop+RLme9Y4oBxfZUNeqmxbiOhir9NzLGj
	encU29x7/3KusKz1S2TowZi82GxkX4c=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210912; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:  content-transfer-encoding:content-transfer-encoding;
	bh=3LpVg4VR4Av9lRaWgZwCKN2sTIaBmuVNnNx2pKsXnAQ=;
	b=ez5EV45Q4D9+xFRUXrbBc1zEca1qn8clECUCYT05wNlagrpYoljdhUa8Qfr4g77XhJRTBU
	ZEjitWKtItm/a6py8tGoS36Yw/bI48w9MgfDpop+RLme9Y4oBxfZUNeqmxbiOhir9NzLGj
	encU29x7/3KusKz1S2TowZi82GxkX4c=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Anthony PERARD <anthony.perard@citrix.com>
Subject: [PATCH v9 0/6] tools: enable xenstore-stubdom to use 9pfs
Date: Thu, 29 Feb 2024 13:48:24 +0100
Message-Id: <20240229124830.13680-1-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=ez5EV45Q
X-Spamd-Result: default: False [-0.31 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[10];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: -0.31
X-Rspamd-Queue-Id: 625361F7E8
X-Spam-Level: 
X-Spam-Flag: NO
X-Spamd-Bar: /

This series is adding 9pfs support to Xenstore-stubdom, enabling it
to do logging to a dom0 directory.

This is a prerequisite for the final goal to add live update support
to Xenstore-stubdom, as it enables the stubdom to store its state in
a dom0 file.

Reposting the rest series. CI-test has passed.

Changes in V9:
- new patch 1
- patches 1+2 and 4 of V8 have been applied

Changes in V8:
- patches 1-13 of V7 have been applied

Changes in V7:
- fixed V6 bugs

Changes in V6:
- patch 1 of V5 has been applied
- rebase
- addressed comments

Changes in V5:
- 10 patches have been applied already
- rename source directory to tools/9pfsd
- addressed comments

Changes in V4:
- patch 2 of V3 was applied
- added support of reading directories
- addressed review comments

Changes in V3:
- new patches 1, 23-25
- addressed review comments

Changes in V2:
- support of multiple rings per device
- xenlogd->xen-9pfsd rename
- addressed review comments
- fixed some bugs

Juergen Gross (6):
  config: update Mini-OS commit
  stubdom: extend xenstore stubdom configs
  tools/xenstored: mount 9pfs device in stubdom
  tools/xenstored: add helpers for filename handling
  tools/xenstored: support complete log capabilities in stubdom
  tools/xenstored: have a single do_control_memreport()

 Config.mk                                     |  2 +-
 stubdom/xenstore-minios.cfg                   |  2 +-
 stubdom/xenstorepvh-minios.cfg                |  2 +-
 .../Linux/init.d/sysconfig.xencommons.in      |  1 -
 tools/hotplug/Linux/launch-xenstore.in        |  1 +
 tools/xenstored/control.c                     | 29 +++------
 tools/xenstored/core.c                        | 15 ++++-
 tools/xenstored/core.h                        | 11 +++-
 tools/xenstored/domain.c                      |  2 +
 tools/xenstored/lu_daemon.c                   |  4 +-
 tools/xenstored/minios.c                      | 62 +++++++++++++++++++
 tools/xenstored/posix.c                       |  8 ++-
 12 files changed, 108 insertions(+), 31 deletions(-)

-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:48:41 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:48:41 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687040.1069786 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffq5-0004E6-AO; Thu, 29 Feb 2024 12:48:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687040.1069786; Thu, 29 Feb 2024 12:48:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffq5-0004Dw-7N; Thu, 29 Feb 2024 12:48:41 +0000
Received: by outflank-mailman (input) for mailman id 687040;
 Thu, 29 Feb 2024 12:48:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rffq3-0003qK-B0
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:48:39 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dbe42f33-d700-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 13:48:38 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 2A3911F7DD;
 Thu, 29 Feb 2024 12:48:38 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id C44851329E;
 Thu, 29 Feb 2024 12:48:37 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id SwVzLiV94GULIgAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 12:48:37 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dbe42f33-d700-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210918; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=IKydSM4pbIaFxQw0VTkeodepd55HC/WaytggrgVsJrc=;
	b=o1wAnnmiRwJMwM2zCwT0uQlWyz01/ktW0S7yxx4D7FuoMRPp7fqDVlFT6lmWsgbGHIzu7B
	x3u2LDmmdXrq1WRpNVc+c8w5GOvquv1/aSsCtVJdiRDRxwhsvQb5SqPe0AvzsvSFuujZV/
	W96xrsoJRllA4yrwWqL2nYctUGwAfRg=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210918; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=IKydSM4pbIaFxQw0VTkeodepd55HC/WaytggrgVsJrc=;
	b=o1wAnnmiRwJMwM2zCwT0uQlWyz01/ktW0S7yxx4D7FuoMRPp7fqDVlFT6lmWsgbGHIzu7B
	x3u2LDmmdXrq1WRpNVc+c8w5GOvquv1/aSsCtVJdiRDRxwhsvQb5SqPe0AvzsvSFuujZV/
	W96xrsoJRllA4yrwWqL2nYctUGwAfRg=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>
Subject: [PATCH v9 1/6] config: update Mini-OS commit
Date: Thu, 29 Feb 2024 13:48:25 +0100
Message-Id: <20240229124830.13680-2-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240229124830.13680-1-jgross@suse.com>
References: <20240229124830.13680-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spamd-Result: default: False [4.90 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_SEVEN(0.00)[8];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.00)[34.07%]
X-Spam-Level: ****
X-Spam-Score: 4.90
X-Spam-Flag: NO

Update the Mini-OS upstream revision.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V9:
- new patch
---
 Config.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Config.mk b/Config.mk
index 6f6e0425ba..a962f095ca 100644
--- a/Config.mk
+++ b/Config.mk
@@ -224,7 +224,7 @@ QEMU_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/qemu-xen.git
 QEMU_UPSTREAM_REVISION ?= master
 
 MINIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/mini-os.git
-MINIOS_UPSTREAM_REVISION ?= 2bc8dbb9b6111113d27455cbca318ea337309c04
+MINIOS_UPSTREAM_REVISION ?= b6a5b4d72b88e5c4faed01f5a44505de022860fc
 
 SEABIOS_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/seabios.git
 SEABIOS_UPSTREAM_REVISION ?= rel-1.16.3
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:48:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:48:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687041.1069796 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffqA-0004X5-Gi; Thu, 29 Feb 2024 12:48:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687041.1069796; Thu, 29 Feb 2024 12:48:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffqA-0004Ww-DX; Thu, 29 Feb 2024 12:48:46 +0000
Received: by outflank-mailman (input) for mailman id 687041;
 Thu, 29 Feb 2024 12:48:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rffq8-0003qK-Ma
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:48:44 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de
 [2a07:de40:b251:101:10:150:64:1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id df2c7477-d700-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 13:48:44 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id B150C21CE4;
 Thu, 29 Feb 2024 12:48:43 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 865691329E;
 Thu, 29 Feb 2024 12:48:43 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id aaZVHyt94GUPIgAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 12:48:43 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df2c7477-d700-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210923; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/eJaaQzJFr9iaY84hbFZ6eF6VnkmvJwoY+5FWl4+J1Y=;
	b=rbS1Gdn1tpCoIIdJ7yCl1awe0yg1l8LML+1+2rg913jASv3h1+p20upRpaOOzRfFxNGuP2
	LdqI0IxiSDYwzsCLxwHktWNvJrLrqCVeu8vE7vRJ+b+wjrQ5OSHJAgjwDxKYtTIXOIy1ur
	8MJBOFPolqYbmY/DF9r+dxlBWKn8U5E=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210923; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=/eJaaQzJFr9iaY84hbFZ6eF6VnkmvJwoY+5FWl4+J1Y=;
	b=rbS1Gdn1tpCoIIdJ7yCl1awe0yg1l8LML+1+2rg913jASv3h1+p20upRpaOOzRfFxNGuP2
	LdqI0IxiSDYwzsCLxwHktWNvJrLrqCVeu8vE7vRJ+b+wjrQ5OSHJAgjwDxKYtTIXOIy1ur
	8MJBOFPolqYbmY/DF9r+dxlBWKn8U5E=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Samuel Thibault <samuel.thibault@ens-lyon.org>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v9 2/6] stubdom: extend xenstore stubdom configs
Date: Thu, 29 Feb 2024 13:48:26 +0100
Message-Id: <20240229124830.13680-3-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240229124830.13680-1-jgross@suse.com>
References: <20240229124830.13680-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
X-Spamd-Bar: /
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=rbS1Gdn1
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [0.67 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-0.82)[85.02%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,ens-lyon.org,gmail.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: 0.67
X-Rspamd-Queue-Id: B150C21CE4
X-Spam-Flag: NO

Extend the config files of the Xenstore stubdoms to include XENBUS
and 9PFRONT items in order to support file based logging.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 stubdom/xenstore-minios.cfg    | 2 +-
 stubdom/xenstorepvh-minios.cfg | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/stubdom/xenstore-minios.cfg b/stubdom/xenstore-minios.cfg
index a41704bb6b..239da519b9 100644
--- a/stubdom/xenstore-minios.cfg
+++ b/stubdom/xenstore-minios.cfg
@@ -3,7 +3,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
diff --git a/stubdom/xenstorepvh-minios.cfg b/stubdom/xenstorepvh-minios.cfg
index 6af51f5753..752b90d7d3 100644
--- a/stubdom/xenstorepvh-minios.cfg
+++ b/stubdom/xenstorepvh-minios.cfg
@@ -4,7 +4,7 @@ CONFIG_NETFRONT=n
 CONFIG_FBFRONT=n
 CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
-CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:48:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:48:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687043.1069806 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffqG-0004xL-NA; Thu, 29 Feb 2024 12:48:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687043.1069806; Thu, 29 Feb 2024 12:48:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffqG-0004x7-KW; Thu, 29 Feb 2024 12:48:52 +0000
Received: by outflank-mailman (input) for mailman id 687043;
 Thu, 29 Feb 2024 12:48:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rffqF-0003yF-HH
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:48:51 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e29561d0-d700-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 13:48:49 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 6FD6D21CDF;
 Thu, 29 Feb 2024 12:48:49 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 2C8FD1329E;
 Thu, 29 Feb 2024 12:48:49 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id YZw1CTF94GUSIgAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 12:48:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e29561d0-d700-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210929; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GuKS5JMiiFYV782dbcVdzQvPQKcEtbDic1ZOTvla/ok=;
	b=ZD2gnEYcMgdt7jCW2U6rJ0U15CXK0vP/hOrFZU8b3+mgf79YW4aD/0UVqZ2cBzkkqwueCG
	4FMa3v2mMaHA+3wgBMO+cT6p27tmrOT9A2cyhFxShd0vZ7WZwNVXHjTUHvDmQJ0u4dRvra
	5Mn5wVCFQM+8AG4Gm5a6LJEvP0hVZTM=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210929; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GuKS5JMiiFYV782dbcVdzQvPQKcEtbDic1ZOTvla/ok=;
	b=ZD2gnEYcMgdt7jCW2U6rJ0U15CXK0vP/hOrFZU8b3+mgf79YW4aD/0UVqZ2cBzkkqwueCG
	4FMa3v2mMaHA+3wgBMO+cT6p27tmrOT9A2cyhFxShd0vZ7WZwNVXHjTUHvDmQJ0u4dRvra
	5Mn5wVCFQM+8AG4Gm5a6LJEvP0hVZTM=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v9 3/6] tools/xenstored: mount 9pfs device in stubdom
Date: Thu, 29 Feb 2024 13:48:27 +0100
Message-Id: <20240229124830.13680-4-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240229124830.13680-1-jgross@suse.com>
References: <20240229124830.13680-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=ZD2gnEYc
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-1.51 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: -1.51
X-Rspamd-Queue-Id: 6FD6D21CDF
X-Spam-Flag: NO

Mount the 9pfs device in stubdom enabling it to use files.

This has to happen in a worker thread in order to allow the main thread
handling the required Xenstore accesses in parallel.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- add logging in case of errors (Julien Grall)
---
 tools/xenstored/core.h   |  6 +++++
 tools/xenstored/domain.c |  2 ++
 tools/xenstored/minios.c | 54 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+)

diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index f6af086f01..fe0ee90581 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -36,6 +36,8 @@
 #include "list.h"
 #include "hashtable.h"
 
+#define XENSTORE_LIB_DIR	XEN_LIB_DIR "/xenstore"
+
 #ifndef O_CLOEXEC
 #define O_CLOEXEC 0
 /* O_CLOEXEC support is needed for Live Update in the daemon case. */
@@ -399,6 +401,10 @@ void handle_special_fds(void);
 int get_socket_fd(void);
 void set_socket_fd(int fd);
 
+#ifdef __MINIOS__
+void mount_9pfs(void);
+#endif
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index 1a7d5e9756..64c8fd0cc3 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -1236,6 +1236,8 @@ void stubdom_init(void)
 		barf_perror("Failed to initialize stubdom");
 
 	xenevtchn_notify(xce_handle, stubdom->port);
+
+	mount_9pfs();
 #endif
 }
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 22ac8defbd..562a9b4972 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -17,10 +17,20 @@
 */
 #include <sys/types.h>
 #include <sys/mman.h>
+#include <syslog.h>
+#include "talloc.h"
 #include "core.h"
 #include "utils.h"
 #include <xen/grant_table.h>
 #include <mini-os/lib.h>
+#include <mini-os/9pfront.h>
+#include <mini-os/sched.h>
+#include <mini-os/xenbus.h>
+#include <mini-os/xmalloc.h>
+
+#define P9_STATE_PATH	"device/9pfs/0/state"
+
+static void *p9_device;
 
 void finish_daemonize(void)
 {
@@ -74,3 +84,47 @@ int get_socket_fd(void)
 void set_socket_fd(int fd)
 {
 }
+
+static void mount_thread(void *p)
+{
+	xenbus_event_queue events = NULL;
+	char *err;
+	char *dummy;
+
+	err = xenbus_watch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs", &events);
+	if (err) {
+		log("error \"%s\" when setting watch on \"%s\"\n", err,
+		    P9_STATE_PATH);
+		free(err);
+		return;
+	}
+
+	for (;;) {
+		xenbus_wait_for_watch(&events);
+
+		/*
+		 * We only care for existence of the state node.
+		 * State changes are handled in init_9pfront().
+		 */
+		err = xenbus_read(XBT_NIL, P9_STATE_PATH, &dummy);
+		if (!err)
+			break;
+		free(err);
+	}
+
+	free(dummy);
+
+	err = xenbus_unwatch_path_token(XBT_NIL, P9_STATE_PATH, "9pfs");
+	if (err) {
+		log("error \"%s\" when unwatching \"%s\", leaking watch\n",
+		    err, P9_STATE_PATH);
+		free(err);
+	}
+
+	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+}
+
+void mount_9pfs(void)
+{
+	create_thread("mount-9pfs", mount_thread, NULL);
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:49:00 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:49:00 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687044.1069816 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffqL-0005Ni-4X; Thu, 29 Feb 2024 12:48:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687044.1069816; Thu, 29 Feb 2024 12:48:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffqL-0005NX-1M; Thu, 29 Feb 2024 12:48:57 +0000
Received: by outflank-mailman (input) for mailman id 687044;
 Thu, 29 Feb 2024 12:48:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rffqK-0003qK-6P
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:48:56 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e6021e57-d700-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 13:48:55 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 2E4A51F7DD;
 Thu, 29 Feb 2024 12:48:55 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id DCE251329E;
 Thu, 29 Feb 2024 12:48:54 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id om5ENDZ94GUaIgAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 12:48:54 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e6021e57-d700-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210935; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xUq3AE+SsnVI0DjnPSJQwNriZRE6cVmY47vrUqUrQuc=;
	b=Zhx+SPnTxks/JXFIrtG+IJisVmMcPJAIKk/4Bp0F8OUE6nXWEAjIGQz0RkQW1w80MSktmD
	tFiupbVk/jkU0bXmfHx4BfuLTfdNozVywRPM2jrE0BodbQyNGR3yI2GDWWuC2pwpKTHl6+
	ibTmJ5wp2vCKsC1y2vmL4ZOjMxNafmE=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210935; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=xUq3AE+SsnVI0DjnPSJQwNriZRE6cVmY47vrUqUrQuc=;
	b=Zhx+SPnTxks/JXFIrtG+IJisVmMcPJAIKk/4Bp0F8OUE6nXWEAjIGQz0RkQW1w80MSktmD
	tFiupbVk/jkU0bXmfHx4BfuLTfdNozVywRPM2jrE0BodbQyNGR3yI2GDWWuC2pwpKTHl6+
	ibTmJ5wp2vCKsC1y2vmL4ZOjMxNafmE=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v9 4/6] tools/xenstored: add helpers for filename handling
Date: Thu, 29 Feb 2024 13:48:28 +0100
Message-Id: <20240229124830.13680-5-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240229124830.13680-1-jgross@suse.com>
References: <20240229124830.13680-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=Zhx+SPnT
X-Spamd-Result: default: False [-0.31 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Spam-Score: -0.31
X-Rspamd-Queue-Id: 2E4A51F7DD
X-Spam-Level: 
X-Spam-Flag: NO
X-Spamd-Bar: /

Add some helpers for handling filenames which might need different
implementations between stubdom and daemon environments:

- expansion of relative filenames (those are not really defined today,
  just expand them to be relative to /var/lib/xen/xenstore)
- expansion of xenstore_daemon_rundir() (used e.g. for saving the state
  file in case of live update - needs to be unchanged in the daemon
  case, but should result in /var/lib/xen/xenstore for stubdom)

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
---
V3:
- make absolute_filename() return a pointer to const (Julien Grall)
---
 tools/xenstored/core.c      | 15 +++++++++++++--
 tools/xenstored/core.h      |  5 ++++-
 tools/xenstored/lu_daemon.c |  4 ++--
 tools/xenstored/minios.c    |  5 +++++
 tools/xenstored/posix.c     |  8 +++++++-
 5 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 48fc787ac1..bada1ad9a2 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -63,7 +63,7 @@ char **orig_argv;
 LIST_HEAD(connections);
 int tracefd = -1;
 bool keep_orphans = false;
-char *tracefile = NULL;
+const char *tracefile = NULL;
 static struct hashtable *nodes;
 unsigned int trace_flags = TRACE_OBJ | TRACE_IO;
 
@@ -137,6 +137,17 @@ void trace_destroy(const void *data, const char *type)
 		trace("obj: DESTROY %s %p\n", type, data);
 }
 
+/*
+ * Return an absolute filename.
+ * In case of a relative filename given as input, prepend XENSTORE_LIB_DIR.
+ */
+const char *absolute_filename(const void *ctx, const char *filename)
+{
+	if (filename[0] != '/')
+		return talloc_asprintf(ctx, XENSTORE_LIB_DIR "/%s", filename);
+	return talloc_strdup(ctx, filename);
+}
+
 void close_log(void)
 {
 	if (tracefd >= 0)
@@ -2759,7 +2770,7 @@ int main(int argc, char *argv[])
 #endif
 
 	if (tracefile)
-		tracefile = talloc_strdup(NULL, tracefile);
+		tracefile = absolute_filename(NULL, tracefile);
 
 #ifndef NO_LIVE_UPDATE
 	/* Read state in case of live update. */
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index fe0ee90581..e58779e88c 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -341,7 +341,7 @@ void close_log(void);
 extern int orig_argc;
 extern char **orig_argv;
 
-extern char *tracefile;
+extern const char *tracefile;
 extern int tracefd;
 
 /* Trace flag values must be kept in sync with trace_switches[] contents. */
@@ -405,6 +405,9 @@ void set_socket_fd(int fd);
 void mount_9pfs(void);
 #endif
 
+const char *xenstore_rundir(void);
+const char *absolute_filename(const void *ctx, const char *filename);
+
 /* Close stdin/stdout/stderr to complete daemonize */
 void finish_daemonize(void);
 
diff --git a/tools/xenstored/lu_daemon.c b/tools/xenstored/lu_daemon.c
index 71bcabadd3..6351111ab0 100644
--- a/tools/xenstored/lu_daemon.c
+++ b/tools/xenstored/lu_daemon.c
@@ -24,7 +24,7 @@ void lu_get_dump_state(struct lu_dump_state *state)
 	state->size = 0;
 
 	state->filename = talloc_asprintf(NULL, "%s/state_dump",
-					  xenstore_daemon_rundir());
+					  xenstore_rundir());
 	if (!state->filename)
 		barf("Allocation failure");
 
@@ -65,7 +65,7 @@ FILE *lu_dump_open(const void *ctx)
 	int fd;
 
 	filename = talloc_asprintf(ctx, "%s/state_dump",
-				   xenstore_daemon_rundir());
+				   xenstore_rundir());
 	if (!filename)
 		return NULL;
 
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index 562a9b4972..e70386f8c7 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -128,3 +128,8 @@ void mount_9pfs(void)
 {
 	create_thread("mount-9pfs", mount_thread, NULL);
 }
+
+const char *xenstore_rundir(void)
+{
+	return XENSTORE_LIB_DIR;
+}
diff --git a/tools/xenstored/posix.c b/tools/xenstored/posix.c
index 496329dfd1..d88c82d972 100644
--- a/tools/xenstored/posix.c
+++ b/tools/xenstored/posix.c
@@ -326,9 +326,10 @@ void early_init(bool live_update, bool dofork, const char *pidfile)
 {
 	reopen_log();
 
-	/* Make sure xenstored directory exists. */
+	/* Make sure xenstored directories exist. */
 	/* Errors ignored here, will be reported when we open files */
 	mkdir(xenstore_daemon_rundir(), 0755);
+	mkdir(XENSTORE_LIB_DIR, 0755);
 
 	if (dofork) {
 		openlog("xenstored", 0, LOG_DAEMON);
@@ -406,3 +407,8 @@ void set_socket_fd(int fd)
 {
 	sock = fd;
 }
+
+const char *xenstore_rundir(void)
+{
+	return xenstore_daemon_rundir();
+}
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:49:03 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:49:03 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687047.1069827 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffqR-0005x5-Ds; Thu, 29 Feb 2024 12:49:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687047.1069827; Thu, 29 Feb 2024 12:49:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffqR-0005ww-92; Thu, 29 Feb 2024 12:49:03 +0000
Received: by outflank-mailman (input) for mailman id 687047;
 Thu, 29 Feb 2024 12:49:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rffqQ-0003qK-0N
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:49:02 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e9762d9a-d700-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 13:49:01 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id D33A221CE4;
 Thu, 29 Feb 2024 12:49:00 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 944E21329E;
 Thu, 29 Feb 2024 12:49:00 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id 0f6/Ijx94GUfIgAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 12:49:00 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e9762d9a-d700-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210941; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uDTztpSOFaUYt0rzH/ToFpRY71aqRBUTIhIi8A11O7I=;
	b=pb5K+HL9dVuzE/s9oQuxXLiXfFwD849xJb4pBIUjMjq6K7mloQmnVlB53nbhgvNGaG1ISY
	4UtRTbEVhSgwHVag13ETb2DysPYsfTWhw5LTXcvqKpX7m+OT1Lke2+62DzBI8+YaT7ud42
	TlJ/P87S2fn5kibGtqSsICOMenbyAz0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210940; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=uDTztpSOFaUYt0rzH/ToFpRY71aqRBUTIhIi8A11O7I=;
	b=nv5NwGW3cC2LqIPemYRpUrqDjloMvH+MU5IYKLaQUmhudOJSqRXznYnQMd1IF8zbr9z2Cg
	i+WUVxlLBq4+DIEOZDQO5J0s6wYTtXg+4q2+6H2SV9GDGWRGtNd605oOgP4XYRpTmnGHOZ
	ZT9U+27dSv26K20zpVPG7qsEnitTVI4=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Julien Grall <julien@xen.org>,
	Jason Andryuk <jandryuk@gmail.com>,
	Julien Grall <jgrall@amazon.com>
Subject: [PATCH v9 5/6] tools/xenstored: support complete log capabilities in stubdom
Date: Thu, 29 Feb 2024 13:48:29 +0100
Message-Id: <20240229124830.13680-6-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240229124830.13680-1-jgross@suse.com>
References: <20240229124830.13680-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Spam-Level: 
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=nv5NwGW3
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-1.51 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from];
	 TO_DN_SOME(0.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 BAYES_HAM(-3.00)[100.00%];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 DWL_DNSWL_MED(-2.00)[suse.com:dkim];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com,amazon.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: -1.51
X-Rspamd-Queue-Id: D33A221CE4
X-Spam-Flag: NO

With 9pfs being fully available in Xenstore-stubdom now, there is no
reason to not fully support all logging capabilities in stubdom.

Open the logfile on stubdom only after the 9pfs file system has been
mounted.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
---
V3:
- remove now stale comment in sysconfig.xencommons.in (Julien Grall)
---
 .../Linux/init.d/sysconfig.xencommons.in      |  1 -
 tools/hotplug/Linux/launch-xenstore.in        |  1 +
 tools/xenstored/control.c                     | 30 +++++++++----------
 tools/xenstored/minios.c                      |  3 ++
 4 files changed, 19 insertions(+), 16 deletions(-)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index 433e4849af..1bdd830d8a 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -58,7 +58,6 @@ XENSTORED_ARGS=
 ## Default: Not defined, tracing off
 #
 # Log xenstored messages
-# Only evaluated if XENSTORETYPE is "daemon".
 #XENSTORED_TRACE=[yes|on|1]
 
 ## Type: integer
diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/launch-xenstore.in
index e854ca1eb8..da4eeca7c5 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -98,6 +98,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @CONFIG_DIR@/@CONFIG_LEAF
 	[ -z "$XENSTORE_DOMAIN_SIZE" ] && XENSTORE_DOMAIN_SIZE=8
 	XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory $XENSTORE_DOMAIN_SIZE"
 	[ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem $XENSTORE_MAX_DOMAIN_SIZE"
+	[ -z "$XENSTORED_TRACE" ] || XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS -T xenstored-trace.log"
 
 	echo -n Starting $XENSTORE_DOMAIN_KERNEL...
 	${LIBEXEC_BIN}/init-xenstore-domain $XENSTORE_DOMAIN_ARGS || exit 1
diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index b2f64d674f..dae23a5ac0 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -201,19 +201,6 @@ static int do_control_quota_s(const void *ctx, struct connection *conn,
 	return EINVAL;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_logfile(const void *ctx, struct connection *conn,
 			      const char **vec, int num)
 {
@@ -222,13 +209,26 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 
 	close_log();
 	talloc_free(tracefile);
-	tracefile = talloc_strdup(NULL, vec[0]);
+	tracefile = absolute_filename(NULL, vec[0]);
 	reopen_log();
 
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
 
+#ifdef __MINIOS__
+static int do_control_memreport(const void *ctx, struct connection *conn,
+				const char **vec, int num)
+{
+	if (num)
+		return EINVAL;
+
+	talloc_report_full(NULL, stdout);
+
+	send_ack(conn, XS_CONTROL);
+	return 0;
+}
+#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
@@ -309,10 +309,10 @@ static struct cmd_s cmds[] = {
 		"[-c <cmdline>] [-F] [-t <timeout>] <file>\n"
 		"    Default timeout is 60 seconds.", 5 },
 #endif
+	{ "logfile", do_control_logfile, "<file>" },
 #ifdef __MINIOS__
 	{ "memreport", do_control_memreport, "" },
 #else
-	{ "logfile", do_control_logfile, "<file>" },
 	{ "memreport", do_control_memreport, "[<file>]" },
 #endif
 	{ "print", do_control_print, "<string>" },
diff --git a/tools/xenstored/minios.c b/tools/xenstored/minios.c
index e70386f8c7..a229954cf4 100644
--- a/tools/xenstored/minios.c
+++ b/tools/xenstored/minios.c
@@ -122,6 +122,9 @@ static void mount_thread(void *p)
 	}
 
 	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+
+	/* Start logging if selected. */
+	reopen_log();
 }
 
 void mount_9pfs(void)
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:49:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:49:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687048.1069836 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffqW-0006NI-Jj; Thu, 29 Feb 2024 12:49:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687048.1069836; Thu, 29 Feb 2024 12:49:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffqW-0006NB-Gy; Thu, 29 Feb 2024 12:49:08 +0000
Received: by outflank-mailman (input) for mailman id 687048;
 Thu, 29 Feb 2024 12:49:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rffqV-0003qK-E3
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:49:07 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ecba23be-d700-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 13:49:06 +0100 (CET)
Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [10.150.64.98])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 7AED821CE4;
 Thu, 29 Feb 2024 12:49:06 +0000 (UTC)
Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 431351329E;
 Thu, 29 Feb 2024 12:49:06 +0000 (UTC)
Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167])
 by imap2.dmz-prg2.suse.org with ESMTPSA id fCTdDkJ94GUnIgAAn2gu4w
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 12:49:06 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ecba23be-d700-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210946; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GGQ2M6ZDlC8Q8t5jjlaB/u1rTlT60Df0gemVV+Z+fCo=;
	b=edTSX6cs5QDWPPRpMv1IUgCzx8+cyhAx6s4yms/R3cnzLnkGloU9kkkrozDmdhivG/b3vj
	9sO/xt44uyr30u3BHbBZ/6GepJB+r3hJl4eU2VF92FNyehGkchIgJIADo74yvFWS0ZMY1i
	U6w/oKmdEo62ez9Ocb7P6XUEVbT0Yfs=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709210946; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=GGQ2M6ZDlC8Q8t5jjlaB/u1rTlT60Df0gemVV+Z+fCo=;
	b=edTSX6cs5QDWPPRpMv1IUgCzx8+cyhAx6s4yms/R3cnzLnkGloU9kkkrozDmdhivG/b3vj
	9sO/xt44uyr30u3BHbBZ/6GepJB+r3hJl4eU2VF92FNyehGkchIgJIADo74yvFWS0ZMY1i
	U6w/oKmdEo62ez9Ocb7P6XUEVbT0Yfs=
From: Juergen Gross <jgross@suse.com>
To: xen-devel@lists.xenproject.org
Cc: Juergen Gross <jgross@suse.com>,
	Wei Liu <wl@xen.org>,
	Julien Grall <julien@xen.org>,
	Anthony PERARD <anthony.perard@citrix.com>,
	Jason Andryuk <jandryuk@gmail.com>
Subject: [PATCH v9 6/6] tools/xenstored: have a single do_control_memreport()
Date: Thu, 29 Feb 2024 13:48:30 +0100
Message-Id: <20240229124830.13680-7-jgross@suse.com>
X-Mailer: git-send-email 2.35.3
In-Reply-To: <20240229124830.13680-1-jgross@suse.com>
References: <20240229124830.13680-1-jgross@suse.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [4.89 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 R_MISSING_CHARSET(2.50)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 RCPT_COUNT_FIVE(0.00)[6];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 MID_CONTAINS_FROM(1.00)[];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 FREEMAIL_CC(0.00)[suse.com,xen.org,citrix.com,gmail.com];
	 RCVD_TLS_ALL(0.00)[];
	 BAYES_HAM(-0.01)[48.12%]
X-Spam-Level: ****
X-Spam-Score: 4.89
X-Spam-Flag: NO

With 9pfs now available in Xenstore-stubdom, there is no reason to
have distinct do_control_memreport() variants for the daemon and the
stubdom implementations.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
---
 tools/xenstored/control.c | 27 +++++++--------------------
 1 file changed, 7 insertions(+), 20 deletions(-)

diff --git a/tools/xenstored/control.c b/tools/xenstored/control.c
index dae23a5ac0..9561289179 100644
--- a/tools/xenstored/control.c
+++ b/tools/xenstored/control.c
@@ -216,23 +216,11 @@ static int do_control_logfile(const void *ctx, struct connection *conn,
 	return 0;
 }
 
-#ifdef __MINIOS__
-static int do_control_memreport(const void *ctx, struct connection *conn,
-				const char **vec, int num)
-{
-	if (num)
-		return EINVAL;
-
-	talloc_report_full(NULL, stdout);
-
-	send_ack(conn, XS_CONTROL);
-	return 0;
-}
-#else
 static int do_control_memreport(const void *ctx, struct connection *conn,
 				const char **vec, int num)
 {
 	FILE *fp;
+	const char *filename;
 	int fd;
 
 	if (num > 1)
@@ -255,8 +243,12 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 			if (!fp)
 				close(fd);
 		}
-	} else
-		fp = fopen(vec[0], "a");
+	} else {
+		filename = absolute_filename(ctx, vec[0]);
+		if (!filename)
+			return ENOMEM;
+		fp = fopen(filename, "a");
+	}
 
 	if (!fp)
 		return EBADF;
@@ -267,7 +259,6 @@ static int do_control_memreport(const void *ctx, struct connection *conn,
 	send_ack(conn, XS_CONTROL);
 	return 0;
 }
-#endif
 
 static int do_control_print(const void *ctx, struct connection *conn,
 			    const char **vec, int num)
@@ -310,11 +301,7 @@ static struct cmd_s cmds[] = {
 		"    Default timeout is 60 seconds.", 5 },
 #endif
 	{ "logfile", do_control_logfile, "<file>" },
-#ifdef __MINIOS__
-	{ "memreport", do_control_memreport, "" },
-#else
 	{ "memreport", do_control_memreport, "[<file>]" },
-#endif
 	{ "print", do_control_print, "<string>" },
 	{ "quota", do_control_quota,
 		"[set <name> <val>|<domid>|max [-r]]" },
-- 
2.35.3



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 12:51:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 12:51:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687054.1069846 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffsj-00015C-1j; Thu, 29 Feb 2024 12:51:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687054.1069846; Thu, 29 Feb 2024 12:51:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rffsi-000155-UP; Thu, 29 Feb 2024 12:51:24 +0000
Received: by outflank-mailman (input) for mailman id 687054;
 Thu, 29 Feb 2024 12:51:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rffsi-00014v-7M
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 12:51:24 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 3d61f8b9-d701-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 13:51:22 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a4429c556efso152963966b.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 04:51:22 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ti12-20020a170907c20c00b00a4138c3f054sm656409ejc.32.2024.02.29.04.51.21
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 04:51:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3d61f8b9-d701-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709211081; x=1709815881; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MyrzNvu2lYus1WJ92zu6GB+N2NdJKl6Xa1em8kZLN7M=;
        b=RG+orhGZfxz/gKkaPjcvB8bAdnHBhKv1AqnPIrOBkHZCChTsH+jmSM8d9umHKgZuw4
         GtcOxDvlmQ50IGSGr/CcrO8Nl7sJ0ThsnzyPBfCeCL3qomH1lpyEBkk2c9BsOFNdFD0j
         H85OSO19ttRwDivQ0O8GDtcer9oRMMsC6305QXXZX79tYekxBvmXEVwyKUX5yHcXKIN1
         3LmusXNAvowuxjjOCQhGmTwVsBp+Yec3nXjzRMkGSzILefdkeY3Eqb22KVlUtxLL4XkF
         V1xTuofJ21y+xdYvLhst7PH1zR/hOSK2c2ZpRNMRJ0hOicql33n2LgTc5JAVLSqOOpi8
         02ew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709211081; x=1709815881;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MyrzNvu2lYus1WJ92zu6GB+N2NdJKl6Xa1em8kZLN7M=;
        b=eWhlT0GoFOMqzLuyfuQiOZRK5EXS3hp8p8bJ9kxpZVIwcLpnGnhARgjSpJql/kJYyl
         6Iu9egak94m/F1tRU6Fs9GF8cOoSXMqqPmCi80m9AHgXkHBdp8obn1WLMCOqfYQkIGSq
         JDkiU+SuZ8RAQJd4V1FWXE7FNG5mhEXUXGeWCfK+l0n8hHbMvDVlTGtT3SwRFHoSpNUb
         b0aO2qGbFdVy5rGFxsVnjUvWo95mG6Qojz7Uc//spGoct0VgQh8i0J5ucE928bYX8YME
         gE0MXv2qOEVx2z2gdCa/5uzfIL6v+9OrHkcmVa3eX8pANzL/5JqysBRWrhajFEDxt5Jx
         Z8qg==
X-Forwarded-Encrypted: i=1; AJvYcCXHHhYJ1KEeIMp+wNKrWEaTxI1nMZneW0vW3yvhuEORwNEmUF3X3DWgu/jlUWVgxx1RHyOKfjqwT5iaUhKtgAKe6bJt3ORHhbk6wRk/D4k=
X-Gm-Message-State: AOJu0YxbZi8ynNQ6RUOVs/p6LCSQhAxwqvfPZTlMM2W8CS/CuQ+ShHTm
	tnxv59ZUQIijtQKq33uEeHbJRD3x7yv+SKlyDHwHn6KjBdDUcHjL5Vq4fltcrOG1wcY9+HdFDWQ
	=
X-Google-Smtp-Source: AGHT+IEnRbyK82of9AakUkrx+DncXYF2uUriUbtIw4sKaunI3VBVmF36OqTuo/gcwo9zmszoAHjWow==
X-Received: by 2002:a17:906:3c47:b0:a44:1ace:2186 with SMTP id i7-20020a1709063c4700b00a441ace2186mr1446459ejg.22.1709211081573;
        Thu, 29 Feb 2024 04:51:21 -0800 (PST)
Message-ID: <aa3fbfca-4675-4f92-9037-a4a7c5eca4dd@suse.com>
Date: Thu, 29 Feb 2024 13:51:20 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko
 <oleksii.kurochko@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
 <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
 <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com>
 <2a112321-f4fd-4ab4-a431-a0bb4cb6eaa2@suse.com>
 <9b212dec-59ef-4cbb-bea7-add3867e4599@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <9b212dec-59ef-4cbb-bea7-add3867e4599@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 13:32, Julien Grall wrote:
> On 29/02/2024 12:17, Jan Beulich wrote:
>> On 29.02.2024 13:05, Andrew Cooper wrote:
>>> On 29/02/2024 10:23 am, Julien Grall wrote:
>>>>>>> IOW it is hard for me to see why RISC-V needs stronger restrictions
>>>>>>> here
>>>>>>> than other architectures. It ought to be possible to determine a
>>>>>>> baseline
>>>>>>> version. Even if taking the desire to have "pause" available as a
>>>>>>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
>>>>>>
>>>>>> I think we want to bump it on Arm. There are zero reasons to try to
>>>>>> keep
>>>>>> a lower versions if nobody tests/use it in production.
>>>>>>
>>>>>> I would suggest to do the same on x86. What's the point of try to
>>>>>> support Xen with a 15+ years old compiler?
>>>>>
>>>>> It could have long been bumped if only a proper scheme to follow for
>>>>> this and future bumping would have been put forward by anyone keen on
>>>>> such bumping, like - see his reply - e.g. Andrew. You may recall that
>>>>> this was discussed more than once on meetings, with no real outcome.
>>>>> I'm personally not meaning to stand in the way of such bumping as long
>>>>> as it's done in a predictable manner, but I'm not keen on doing so and
>>>>> hence I don't view it as my obligation to try to invent a reasonable
>>>>> scheme. (My personal view is that basic functionality should be
>>>>> possible to have virtually everywhere, whereas for advanced stuff it
>>>>> is fine to require a more modern tool chain.)
>>>>
>>>> That's one way to see it. The problem with this statement is a user
>>>> today is mislead to think you can build Xen with any GCC versions
>>>> since 4.1. I don't believe we can guarantee that and we are exposing
>>>> our users to unnecessary risk.
>>>>
>>>> In addition to that, I agree with Andrew. This is preventing us to
>>>> improve our code base and we have to carry hacks for older compilers.
>>>
>>> I don't think anyone here is suggesting that we switch to a
>>> bleeding-edge-only policy.  But 15y of support is extreme in the
>>> opposite direction.
>>>
>>> Xen ought to be buildable in the contemporary distros of the day, and I
>>> don't think anyone is going to credibly argue otherwise.
>>>
>>> But, it's also fine for new things to have newer requirements.
>>>
>>> Take CET for example.  I know we have disagreements on exactly how it's
>>> toolchain-conditionalness is implemented, but the basic principle of "If
>>> you want shiny new optional feature $X, you need newer toolchain $Y" is
>>> entirely fine.
>>>
>>> A brand new architecture is exactly the same.  Saying "this is the
>>> minimum, because it's what we test" doesn't preclude someone coming
>>> along and saying "can we use $N-1 ?  See here it works, and here's a
>>> change to CI test it".
>>>
>>>
>>> Anyway, its clear we need to write some policy on this, before making
>>> specific adjustments.  To get started, is there going to be any
>>> objection whatsoever on some principles which begin as follows:
>>
>> Largely not, but one aspect needs clarifying up front:
>>
>>> * For established architectures, we expect Xen to be buildable on the
>>> common contemporary distros.  (i.e. minima is not newer than what's
>>> available in contemporary distros, without a good reason)
>>
>> What counts as contemporary distro? Still in normal support? LTS? Yet
>> more extreme forms?
> 
> LTS makes sense. More I am not sure. I am under the impression that 
> people using older distros are those that wants a stable system. So they 
> would unlikely try to upgrade the hypervisor.
> 
> Even for LTS, I would argue that if it has been released 5 years ago, 
> then you probably want to update it at the same time as moving to a 
> newer Xen version.

For the purposes of distros I agree. For the purposes of individuals
I don't: What's wrong with running a newer hypervisor and/or kernel
underneath an older distro?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:01:18 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:01:18 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687066.1069855 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfg26-0003Tl-0Z; Thu, 29 Feb 2024 13:01:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687066.1069855; Thu, 29 Feb 2024 13:01:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfg25-0003Te-UE; Thu, 29 Feb 2024 13:01:05 +0000
Received: by outflank-mailman (input) for mailman id 687066;
 Thu, 29 Feb 2024 13:01:05 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfg25-0003TY-1Z
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:01:05 +0000
Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com
 [2a00:1450:4864:20::633])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9824c9f9-d702-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 14:01:03 +0100 (CET)
Received: by mail-ej1-x633.google.com with SMTP id
 a640c23a62f3a-a3d484a58f6so146069466b.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:01:03 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 gg20-20020a170906e29400b00a4426352d8asm659377ejb.193.2024.02.29.05.01.02
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 05:01:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9824c9f9-d702-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709211663; x=1709816463; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=QRP+weW0GjLngKlURrimZf/CrgVi3wASdZaM9BCrpwQ=;
        b=Kh87GuLkC7m9JeF3LwiSKBs7ZE23MpcPNgCIwqFkye5vAXd5HXk7BGLMGrpv56IDPf
         vmhA7Gm9jujcUi8K9Fl1zpoBW/k8fu6syPtcasW738e86K88/roICzvDY3oML+oQh4VQ
         Cy5MBtCvyG90AClywgBHQiCrpVvSKXPCA2lVKy7k51tAjTod5S+vT8/B6ZvH6d00rPyU
         i8i86tJT/gzCzQhPtSSR9/WiG+Y6I39H+Y9RkTUHAbHY1AtiqL8FVxGOARV5PKqr8C6F
         ujXuPhnfIrrXRhjlYPW/XIUle2TvEN4yPZmLK+Oi1PBn7k/3LptCCTw6UcouclgAYakJ
         d90w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709211663; x=1709816463;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=QRP+weW0GjLngKlURrimZf/CrgVi3wASdZaM9BCrpwQ=;
        b=rGZqmoWWN5jJN0N2oylEiJKVnrdLn2MMR7/Ds+flC0fYWO1IgLKxOd+qpMAOJXsrei
         ShCfRnzjZ6vtXqD7KxGNYSnwzGDBwaqFpe0S7hrql2oFEblhLfJlsB8dol5OjrgusOVC
         mDOjnshhHO5uK3DDp9km3SOQheB61LZjKiqF3QNxsL89FdFjg/I9Rw/qV5O0ApMmcJ6O
         NMhPnGCIqavVvXREL/KEx76z1PdXaSn7W3rPBrSvaEel0r44f4DXmU61WU/HIzhb6twS
         coqY7B1cWA/dPJv4cx/b1xHgDrK9bYZVZpR7XmBI4sVDaq3El7wVAYF2w/jBDpBQnNWe
         n4Cw==
X-Forwarded-Encrypted: i=1; AJvYcCWG3B1nFM97GAYxRSa0OrARXxS6makhNm5OMerRb2+lofG9/FZ8R48prZryRkkZQwJqQAv+KdTLciZ6VpgeWJmG4HoJVJsHp0Vw4YaLpV8=
X-Gm-Message-State: AOJu0Ywa+TJTbQyTVUZCmCwPDygN+r5Lb8pJ+HRcuulf3TKjdeKdRUcK
	J8rVu9H+QX4NXrTJoBojOBAtWr5VLlBFj9pSW3nxtD3HYOXuecVLbJo6ZrzcPXuCz+ydLSprMNw
	=
X-Google-Smtp-Source: AGHT+IHd80tWvIKGtqBK92bIcwTFX+EBeB7krGTS4Vfhu1Yndi22o1yNYWgVYSDWQTCZkscnYH4pvg==
X-Received: by 2002:a17:906:3414:b0:a44:311a:3ec9 with SMTP id c20-20020a170906341400b00a44311a3ec9mr1404569ejb.51.1709211663378;
        Thu, 29 Feb 2024 05:01:03 -0800 (PST)
Message-ID: <78032daf-9052-4ab8-ba05-456fa4aee17b@suse.com>
Date: Thu, 29 Feb 2024 14:01:01 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] README: bump minimum required clang/llvm version
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Doug Goldstein <cardoe@cardoe.com>, xen-devel@lists.xenproject.org
References: <20240229095529.17723-1-roger.pau@citrix.com>
 <20240229095529.17723-2-roger.pau@citrix.com>
 <780ec3ab-08e3-4fd8-a85f-0e89ea75dcc8@suse.com> <ZeB7S6i7pIwzkUNE@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZeB7S6i7pIwzkUNE@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 13:40, Roger Pau Monné wrote:
> On Thu, Feb 29, 2024 at 01:11:55PM +0100, Jan Beulich wrote:
>> On 29.02.2024 10:55, Roger Pau Monne wrote:
>>> --- a/README
>>> +++ b/README
>>> @@ -41,7 +41,7 @@ provided by your OS distributor:
>>>          - GCC 4.1.2_20070115 or later
>>>          - GNU Binutils 2.16.91.0.5 or later
>>>          or
>>> -        - Clang/LLVM 3.5 or later
>>> +        - Clang/LLVM 14.0.0 or later
>>
>> Wow, that's a big step. I'm build-testing with Clang7 on one system and
>> with Clang5 on another (and the latter more frequently than the former).
>> If any real dependency on this new a version (about 3 years old?) was
>> introduced, I would then no longer be able to locally test any Clang
>> builds (and hence the risk would again increase that I introduce issues
>> that affect just Clang builds).
> 
> Would it be possible for you to update to a newer version?  I see both
> the OpenSUSE containers in Gitlab have newer versions of Clang.

No. These are SLES versions which I'm not intending to touch. See

https://lists.xen.org/archives/html/xen-devel/2024-02/msg01793.html

and

https://lists.xen.org/archives/html/xen-devel/2024-02/msg01795.html

for why. The most recent piece of hardware I've installed a fresh (but
not exactly new, yet still fully supported) SLES version on would
apparently offer Clang7 only, either.

If anything I could see about building a newer Clang version myself;
no idea how involved that is compared to the building of gcc (which I
consider pretty straightforward, but that's biased by me being used to
doing so).

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:03:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:03:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687069.1069866 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfg4I-0004aY-Eg; Thu, 29 Feb 2024 13:03:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687069.1069866; Thu, 29 Feb 2024 13:03:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfg4I-0004aR-Aj; Thu, 29 Feb 2024 13:03:22 +0000
Received: by outflank-mailman (input) for mailman id 687069;
 Thu, 29 Feb 2024 13:03:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfg4H-0004aL-8Y
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:03:21 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id e9872243-d702-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 14:03:20 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a446478b04bso18725966b.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:03:20 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 mp7-20020a1709071b0700b00a440ceb4110sm670082ejc.183.2024.02.29.05.03.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 05:03:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e9872243-d702-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709211800; x=1709816600; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=L+eXhvCDuJb+v4y6mj8ABLnYe6afYa26GLF55SyHyBI=;
        b=GOKdjT+QpS2DZCzErYIoDT/Mlxd8eAVJWnnc8a0VE1XtfAq05QjqvCPsIG0+cykDY5
         piZxrUeOnIjqDlyMA0YJEy+M3scP8YAv1JHb2QiQPFE6QFwU142LvMcEOKK1mauH9Rco
         tjf6u2E07m/If0rhWrkLfY6sqNunibHNljmf1nbztQzGL8s9FykGtE/sM9JSj4yvOMTx
         0zIjhNceTGqhbRM2GC57rUZyknCSPobWi5G/ffRE4qw7R9l1YDO1zSmoY/57Z9SRzoLf
         mVLnEKDTP0NU5mi7oVuLQL2GfdPeY3c0+f417A1lnP/hJzi0LQEdnLVCLJqv9Lk9zfJa
         1v5w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709211800; x=1709816600;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=L+eXhvCDuJb+v4y6mj8ABLnYe6afYa26GLF55SyHyBI=;
        b=xMC2TFXBTqGNH2TkgoESGdoU3iXg0SyLs3eqvaNT7OfP8rgWQY1rqxhq9BSyEKz4sv
         XQq+1JHByXGa49eXhlAXaOG/TVJn6rs/Upc2y5EK3Y1HpNv4sUjSTF8Gk4u+MpfzQ00A
         SEYjdc7i0DE9HuK0J1CZnrffrUMmiN5yWyzEqHKiFWnrb8unFF1zEOn8Nf4T6BSMqDAs
         uEesWX/vl4S9XSLYlhuYDcNSWBBBmP8csHi2qGHzblfIC3RUuD9Js11232akP8YcMnmy
         PzvmifZoDQa/ZKIZWmE36aJvgbgcBbybb+PyUE2tuE5IMOOuBiYKcrv0Egja4bKll4dH
         EPUQ==
X-Forwarded-Encrypted: i=1; AJvYcCUd1o1I83lW+/sR1r6bTT1BfJpNLTKCGdiy+OKriTv10dpi4uplDNBq+RXGT2YwkaUlRZTESef9aw1DJf/tu3xBkB8bDvs6q63Qx0pDLzI=
X-Gm-Message-State: AOJu0Yyb08VzBGM9p8dhG1quWlv74UmAnGox/Pzxm/IpMHCCS7tGa8vC
	efueYChJIXthpWfnT6W43SoRASYVxfxJTYIp7ujTTVswiBRYmNH/V1n909CUZQ==
X-Google-Smtp-Source: AGHT+IEVsQfR53UbMtrZML6OoUJTkPNNEOV5Xa4PKNBjk6nYEnBrbwKhyRMQ1+RYzMsxGIj0hVNvVw==
X-Received: by 2002:a17:906:d145:b0:a3e:7f54:12c1 with SMTP id br5-20020a170906d14500b00a3e7f5412c1mr1552311ejb.72.1709211799908;
        Thu, 29 Feb 2024 05:03:19 -0800 (PST)
Message-ID: <52ea04ab-9a88-4288-b8d3-7af49e67c88f@suse.com>
Date: Thu, 29 Feb 2024 14:03:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 1/6] config: update Mini-OS commit
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20240229124830.13680-1-jgross@suse.com>
 <20240229124830.13680-2-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240229124830.13680-2-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 13:48, Juergen Gross wrote:
> Update the Mini-OS upstream revision.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Jan Beulich <jbeulich@suse.com>




From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:05:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:05:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687071.1069876 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfg6T-0005QE-Q4; Thu, 29 Feb 2024 13:05:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687071.1069876; Thu, 29 Feb 2024 13:05:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfg6T-0005Q7-MK; Thu, 29 Feb 2024 13:05:37 +0000
Received: by outflank-mailman (input) for mailman id 687071;
 Thu, 29 Feb 2024 13:05:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfg6R-0005Q0-TG
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:05:35 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 39205687-d703-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 14:05:33 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a3e7f7b3d95so133795766b.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:05:33 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 vk5-20020a170907cbc500b00a445188e29bsm402834ejc.148.2024.02.29.05.05.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 05:05:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 39205687-d703-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709211933; x=1709816733; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2bUEHBz94I+H8EA8+6kL1Yh/CNUpXKRmo9pEpsgXqKE=;
        b=fnnQV77s2HiCUd5X8LE2DiF50hLcMMnTtFXh5DlNh+aWG1FWYOcYV0F+2h6yVIINqv
         lc4yc1tJDFIjYgdB3cfL/DeBFXYz5YK3xrptX6r2A62CrbM/vPFdNCbubNlP4tOe27qT
         gt264zjix/4CpfdAYSiv25MHH4d2207I/yO1fgSffjLh6bAfoRdjVCH5Yf0RzfBfWzIQ
         HC2EGeX/27EP6nVTE7+TBYOqazNpR0cqI+74wS/cndUsaMFt+cGOqc5Sh1uyqARcWBMN
         vqWScGaSmg3q3XgdV8vYNMgAQly4ymPy75Uvj1X4wLIlFM/zYKGRO6BhG1qSgXHVwMPI
         VCkQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709211933; x=1709816733;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2bUEHBz94I+H8EA8+6kL1Yh/CNUpXKRmo9pEpsgXqKE=;
        b=oozP+E1X0hMxkKr5fkXrlwt0C/qzmjmwkBfYLJtP4tUrqWaTAjDSAzX/TZHqS7Jogs
         p0yCANHMZmy4wV8c1NImSYufI4bEwueHUpK5QTdNswuEm+zzXOjVlSIebIFt9m3xpswh
         e8WX57lGv2YuhxSRF+fX0QXWc852Bchej/Rv2gmUS9jRwLYeFtbRPc6fMm8iQvSHEM4Z
         4ApfT8W+ZSvFEr4wQi1g1SmOxPea5na7jE1+aCpXDaVMwVmTQ3LqfvVVHKJIil4sNDd9
         0DBM7xdRiK4x776ruOt3UV8qg/U63mPUcI2RdFMtBFV1RV1vyXN9B+dspqiH5QcE2OqB
         M/PQ==
X-Forwarded-Encrypted: i=1; AJvYcCXC5193jO12N8jQA25Hn1eBtgX/XYvIX8fMA+NRSrW1Wja6C9b5l83IKaIkZlUuxyJ/yHpZvjbqNdJviD3t0FxUiICoyLTe4rTlWXE8Zec=
X-Gm-Message-State: AOJu0YyBXyxk88NkSWrt/0rLX8DSltGLZh4DHY5B8AuQqbpP32sWFksm
	W3e7iJXEGYCS/vMOywmPovroRJE4XxgBj+/3W4E4rqCj/BM9hj2/CMCv73B5NA==
X-Google-Smtp-Source: AGHT+IHaoldRU4mz3gQK6FFTgzMYjHN0b7XXz4xomWa93SyhmtxIGxP6MB+XAF0YC8uOFz3ap9XuUw==
X-Received: by 2002:a17:906:a899:b0:a3e:8809:525c with SMTP id ha25-20020a170906a89900b00a3e8809525cmr1531488ejb.37.1709211933441;
        Thu, 29 Feb 2024 05:05:33 -0800 (PST)
Message-ID: <6378d51b-ab4d-4019-970c-b8879c75b7d9@suse.com>
Date: Thu, 29 Feb 2024 14:05:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 2/6] stubdom: extend xenstore stubdom configs
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 Jason Andryuk <jandryuk@gmail.com>, xen-devel@lists.xenproject.org
References: <20240229124830.13680-1-jgross@suse.com>
 <20240229124830.13680-3-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240229124830.13680-3-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 13:48, Juergen Gross wrote:
> Extend the config files of the Xenstore stubdoms to include XENBUS
> and 9PFRONT items in order to support file based logging.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

Was an ack from Samuel lost here? Or was it dropped on purpose?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:12:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:12:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687076.1069886 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgD9-0007QA-EN; Thu, 29 Feb 2024 13:12:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687076.1069886; Thu, 29 Feb 2024 13:12:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgD9-0007Q3-BU; Thu, 29 Feb 2024 13:12:31 +0000
Received: by outflank-mailman (input) for mailman id 687076;
 Thu, 29 Feb 2024 13:12:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfgD7-0007Pv-RI
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:12:29 +0000
Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com
 [2a00:1450:4864:20::635])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 304cccf5-d704-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 14:12:28 +0100 (CET)
Received: by mail-ej1-x635.google.com with SMTP id
 a640c23a62f3a-a26fa294e56so169988866b.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:12:28 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 qw16-20020a1709066a1000b00a3edb758561sm669721ejc.129.2024.02.29.05.12.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 05:12:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 304cccf5-d704-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709212348; x=1709817148; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=qIc5HKpjtJ58Pd/LBVD3zNyzRmZRR7kT8YdzWRqMUAs=;
        b=TiGvaPBirO4LZ1294IxpLbDnNvoyo+9ATS/6NacaoLv4gyJ1NekvLgZtNIJWFoaeI6
         OBsce8bdVoQAlXTZQeO3dICiXiREcdWRb/5oi3a59VsaVLf+73OdkbYeGt+W+ULiW/iS
         y2ToBdEkPpjoJmS0txnwzTJimkzT5gLrhv+g4D09DMYH5j4xFXhMgAhdFf/LHF2rjS+w
         AL2XsCKiuaG0mnm1z6VgEL4qF3WcpoaNgAl1o85IobMQDOQNvMsIjVCmkGVhn24Sqa91
         RO5KmxbMHQQ02zQoyEWjMNnVqKQKU3csW5iH2gBjWZyuR9m4zhmZrHpUfbZ67bSZTzds
         +AmQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709212348; x=1709817148;
        h=content-transfer-encoding:in-reply-to:autocrypt:content-language
         :references:cc:to:from:subject:user-agent:mime-version:date
         :message-id:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=qIc5HKpjtJ58Pd/LBVD3zNyzRmZRR7kT8YdzWRqMUAs=;
        b=HOXJmPxa523INWJs49Ds696uiGEeYkDvv/F7We0sZFAsBr82Az4t94a+6wgQxRedla
         AabHeNMgVvrBjDji/212kzal0bWVf+/6eU2YfPC1rsK9FIFecEybO2hFvhDIH2QGEXly
         ziTGBIO8GQzvjuGHPOPCshdapZdNW0M28kUn0tNKwtQZn1X3RynNiqnUbK8nmIZ1cWSv
         F5jUZ0i2y9ittz+XfFDsgicsTc4cDrk2uFmNzx9jNjidwsoRT8DJcSxQLfBx1lKosdGn
         3C1krmnAzZvEj7dc4u9E4TxtDB5SsBzaSNAz+XJJguEHCZgyyKd7c9U6AALWAPVuBf72
         Y1PQ==
X-Forwarded-Encrypted: i=1; AJvYcCWcUuIjsdEY2wMW6uL8RcMSnSqVW621kH23IE+ZqSLCOJm2MRTjxHeLa8FYtWWoQ6bK+oMYYXvq04SSrzO2HjLZ2J1DWhye5iVlxzsiIOo=
X-Gm-Message-State: AOJu0Ywr+oDwm19oNUxWo4z1QCxZ1TBUfdPix7/Toy5jpKRY6dy1RPeG
	YtoJ9cY8hb3jQ8k322uIy7Gp3VKxGWsPjzkJRHCYzaAuNwXLE+doCMeujh3v7A==
X-Google-Smtp-Source: AGHT+IHaiXUCa1XkIsS2y9ZR9e7FKOW/OtGQChhqAjmIarvSs5TUnSBA4Qo8Z2Awor/GtNhZs11e1A==
X-Received: by 2002:a17:907:11cb:b0:a43:af0b:c1c0 with SMTP id va11-20020a17090711cb00b00a43af0bc1c0mr1551603ejb.16.1709212348150;
        Thu, 29 Feb 2024 05:12:28 -0800 (PST)
Message-ID: <132dde96-d7d3-4823-8a0e-7b4a62526f2e@suse.com>
Date: Thu, 29 Feb 2024 14:12:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] README: bump minimum required clang/llvm version
From: Jan Beulich <jbeulich@suse.com>
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Doug Goldstein <cardoe@cardoe.com>, xen-devel@lists.xenproject.org
References: <20240229095529.17723-1-roger.pau@citrix.com>
 <20240229095529.17723-2-roger.pau@citrix.com>
 <780ec3ab-08e3-4fd8-a85f-0e89ea75dcc8@suse.com> <ZeB7S6i7pIwzkUNE@macbook>
 <78032daf-9052-4ab8-ba05-456fa4aee17b@suse.com>
Content-Language: en-US
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <78032daf-9052-4ab8-ba05-456fa4aee17b@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 14:01, Jan Beulich wrote:
> On 29.02.2024 13:40, Roger Pau Monné wrote:
>> On Thu, Feb 29, 2024 at 01:11:55PM +0100, Jan Beulich wrote:
>>> On 29.02.2024 10:55, Roger Pau Monne wrote:
>>>> --- a/README
>>>> +++ b/README
>>>> @@ -41,7 +41,7 @@ provided by your OS distributor:
>>>>          - GCC 4.1.2_20070115 or later
>>>>          - GNU Binutils 2.16.91.0.5 or later
>>>>          or
>>>> -        - Clang/LLVM 3.5 or later
>>>> +        - Clang/LLVM 14.0.0 or later
>>>
>>> Wow, that's a big step. I'm build-testing with Clang7 on one system and
>>> with Clang5 on another (and the latter more frequently than the former).
>>> If any real dependency on this new a version (about 3 years old?) was
>>> introduced, I would then no longer be able to locally test any Clang
>>> builds (and hence the risk would again increase that I introduce issues
>>> that affect just Clang builds).
>>
>> Would it be possible for you to update to a newer version?  I see both
>> the OpenSUSE containers in Gitlab have newer versions of Clang.
> 
> No. These are SLES versions which I'm not intending to touch. See
> 
> https://lists.xen.org/archives/html/xen-devel/2024-02/msg01793.html
> 
> and
> 
> https://lists.xen.org/archives/html/xen-devel/2024-02/msg01795.html
> 
> for why. The most recent piece of hardware I've installed a fresh (but
> not exactly new, yet still fully supported) SLES version on would
> apparently offer Clang7 only, either.

Bah, that's not even Clang, only LLVM.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:14:05 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:14:05 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687078.1069896 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgEd-0008AB-OE; Thu, 29 Feb 2024 13:14:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687078.1069896; Thu, 29 Feb 2024 13:14:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgEd-0008A4-LS; Thu, 29 Feb 2024 13:14:03 +0000
Received: by outflank-mailman (input) for mailman id 687078;
 Thu, 29 Feb 2024 13:14:02 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfgEc-00089y-Fd
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:14:02 +0000
Received: from mail-qv1-xf30.google.com (mail-qv1-xf30.google.com
 [2607:f8b0:4864:20::f30])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6750bbc3-d704-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 14:14:01 +0100 (CET)
Received: by mail-qv1-xf30.google.com with SMTP id
 6a1803df08f44-68f41af71ebso6129136d6.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:14:01 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 lx12-20020a0562145f0c00b0068fef90f4cesm716527qvb.99.2024.02.29.05.13.58
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 05:13:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6750bbc3-d704-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709212440; x=1709817240; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=+ZP/l18B1Z6kSWg8B7up6I1124bGIubuSaBLfSxaWMc=;
        b=fFV/XYyBzdKAs87dwx/K2xoUi2jXCcLLA9LfPiynz7hkf/r6BGlXlk8BnqjMQ85aLn
         kpinpW/nahDd7ori4vOnBncqYN0tWOnUamJhuoMrva4BiFEO0qLP4SzbxmCTzvk45yk2
         uRh8LcnJMl/lg5rTRyFtlGGlE1pj18jciDYnE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709212440; x=1709817240;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=+ZP/l18B1Z6kSWg8B7up6I1124bGIubuSaBLfSxaWMc=;
        b=SW9iqgnHaU/hLeYjyfXcmqEzKU9o96hSV+8N13ctPM2QxyVRkRFoD1d4OacAHFvQmx
         wUtwG/Kc7oMnJCHn9tqCyZWy1FmDcjKCSWfIpLsIK1XRmP7Y/DEXU2iol9FN36WDIOvv
         9WCUfgrwyjGCT6lxtZ5sYKWOg9D1eVlX/RzsxwRuvjNpCL+071G7iccBzfqDO7PHJFjg
         K413QbmKPzhbXXTF6Y6if78a8UHqV1ylm+Ds+HbDB+/Q07AbbNuChb1ECK47qUsV0/jR
         qb3VUDepRi/BEQcIRAIP2CjjvfpGpAx5IsROnpE+ewfkwa0y2XvQj+5JSftiJQBFi85V
         GZPA==
X-Gm-Message-State: AOJu0Yw2/phLQwE7a1Xfu1mV0OlCtewJv2rjFsFuGGcbuNotqOP6+Phe
	ZiQnpbfG/4zUpx8P6+gMf7b1+k/nYB+YM4KNUATQ7T0EfBtviIw2otCIfwOkhPM=
X-Google-Smtp-Source: AGHT+IFsDo+XLzrg0BFPXr09FpCao10hokXFJkFI0RdFI3TPcO/FB93pS0Af0SyRT+UOgAHpnfsGEQ==
X-Received: by 2002:a05:6214:4959:b0:68f:3dce:7b94 with SMTP id pe25-20020a056214495900b0068f3dce7b94mr2330660qvb.4.1709212440311;
        Thu, 29 Feb 2024 05:14:00 -0800 (PST)
Message-ID: <e5225318-9914-4264-90ec-b7e8c2c0d6e3@citrix.com>
Date: Thu, 29 Feb 2024 13:13:57 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpu-policy: Fix x2APIC visibility for PV guests
Content-Language: en-GB
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Xen-devel <xen-devel@lists.xenproject.org>,
 Jan Beulich <JBeulich@suse.com>, Wei Liu <wl@xen.org>,
 Juergen Gross <jgross@suse.com>
References: <20240229104304.2478614-1-andrew.cooper3@citrix.com>
 <ZeBw96VzCVeF0-6T@macbook>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <ZeBw96VzCVeF0-6T@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/02/2024 11:56 am, Roger Pau Monné wrote:
> On Thu, Feb 29, 2024 at 10:43:04AM +0000, Andrew Cooper wrote:
>> Right now, the host x2APIC setting filters into the PV max and default
>> policies, yet PV guests cannot set MSR_APIC_BASE.EXTD or access any of the
>> x2APIC MSR range.  Therefore they absolutely shouldn't see the x2APIC bit.
>>
>> Linux has workarounds for the collateral damage caused by this leakage; it
>> unconditionally filters out the x2APIC CPUID bit, and EXTD when reading
>> MSR_APIC_BASE.
>>
>> Hide the x2APIC bit in the PV default policy, but for compatibility, tolerate
>> incoming VMs which already saw the bit.  This is logic from before the
>> default/max split in Xen 4.14 which wasn't correctly adjusted at the time.
>>
>> Update the annotation from !A to !S which slightly better describes that it
>> doesn't really exist in PV guests.  HVM guests, for which x2APIC can be
>> emulated completely, already has it unconditionally set in the max policy.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>> CC: Jan Beulich <JBeulich@suse.com>
>> CC: Roger Pau Monné <roger.pau@citrix.com>
>> CC: Wei Liu <wl@xen.org>
>>
>> This wants backporting as far as people can tollerate, but it's really not
>> obvious which commit in 4.14 should be referenced in a Fixes: tag.
> Oh, so we didn't use to expose x2APIC in Xen < 4.14 for PV at all?

We did conditionally expose x2APIC to PV guests prior to that.

What we didn't have in Xen < 4.14 was a split between max and default
policies.  Everything before 4.14 is far harder to reason about.

>
> I think this need mentioning in the commit message, as it's not clear
> whether x2APIC has always been advertised to guests.
>
> If it's indeed only Xen 4.14 that started exposing the flag, it's IMO
> less dangerous to stop exposing it.  My main concern would be OSes
> having grow some dependency on it, and us no longer exposing it
> causing collateral damage (which would be an OS bug anyway).

As I said, Linux explicitly self-hides the cap, because at one point it
tried turning x2APIC on and got unhappy at getting a #GP back.

Juergen may remember better.  IIRC it was fallout from making WRMSR not
always-silently-safe.AFAICT NetBSD doesn't explode because it's
x2APIC-enablement logic is inside #ifndef XENPV.
>> ---
>>  xen/arch/x86/cpu-policy.c                   | 19 +++++++++++++++++--
>>  xen/include/public/arch-x86/cpufeatureset.h |  2 +-
>>  2 files changed, 18 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
>> index 10079c26ae24..a0205672428d 100644
>> --- a/xen/arch/x86/cpu-policy.c
>> +++ b/xen/arch/x86/cpu-policy.c
>> @@ -534,6 +534,14 @@ static void __init calculate_pv_max_policy(void)
>>      *p = host_cpu_policy;
>>      x86_cpu_policy_to_featureset(p, fs);
>>  
>> +    /*
>> +     * Xen at the time of writing (Feb 2024, 4.19 dev cycle) used to leak the
>> +     * host x2APIC capability into PV guests, but never supported the guest
>> +     * trying to turn x2APIC mode on.  Tolerate an incoming VM which saw the
>> +     * x2APIC CPUID bit.
>> +     */
>> +    __set_bit(X86_FEATURE_X2APIC, fs);
>> +
>>      for ( i = 0; i < ARRAY_SIZE(fs); ++i )
>>          fs[i] &= pv_max_featuremask[i];
>>  
>> @@ -566,6 +574,14 @@ static void __init calculate_pv_def_policy(void)
>>      *p = pv_max_cpu_policy;
>>      x86_cpu_policy_to_featureset(p, fs);
>>  
>> +    /*
>> +     * PV guests have never been able to use x2APIC mode, but at the time of
>> +     * writing (Feb 2024, 4.19 dev cycle), the host value used to leak into
>> +     * guests.  Hide it by default so new guests don't get mislead into
>> +     * thinking that they can use x2APIC.
>> +     */
>> +    __clear_bit(X86_FEATURE_X2APIC, fs);
> IIRC if you use the 'S' tag it won't be added to the default PV policy
> already, so there should be nothing to clear?  pv_def_featuremask
> shouldn't contain the bit in the first place.

Bah.  That means there's a bug visible in context.  Serves me right for
last minute clean-up...

I need to set the bit after applying pv_max_featuremask[].

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:15:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:15:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687081.1069906 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgG6-0000Xn-5x; Thu, 29 Feb 2024 13:15:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687081.1069906; Thu, 29 Feb 2024 13:15:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgG6-0000Xg-2S; Thu, 29 Feb 2024 13:15:34 +0000
Received: by outflank-mailman (input) for mailman id 687081;
 Thu, 29 Feb 2024 13:15:33 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xh/y=KG=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rfgG5-0000XX-6G
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:15:33 +0000
Received: from mail-oo1-xc33.google.com (mail-oo1-xc33.google.com
 [2607:f8b0:4864:20::c33])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9d6ae6cd-d704-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 14:15:32 +0100 (CET)
Received: by mail-oo1-xc33.google.com with SMTP id
 006d021491bc7-5a0e5f083f1so340292eaf.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:15:32 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 w26-20020a05620a149a00b00787af8b5c02sm657685qkj.39.2024.02.29.05.15.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 05:15:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9d6ae6cd-d704-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709212530; x=1709817330; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=R+jK1WuNiZJxzm9wGU5l+tCGwClKbWQ4eO1w6yhByD4=;
        b=ZAligymMs4k+0qdeApLX4R/m/C6MI12dqj2kmYFZpYdHea82ch9J3xpkZtAHEOZLEm
         CLNp2orSsROdz4uGoHYf93A2+t6nAQzz5ak9fUZIk+Wq2jssc13Inhho9R+6wfzAEkN5
         GtEITJftDt76G1g3tmFnR1MtIT2+6+Lcs9GQQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709212530; x=1709817330;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=R+jK1WuNiZJxzm9wGU5l+tCGwClKbWQ4eO1w6yhByD4=;
        b=Zhv21SGNlOfbrcFK/5hPrFk2HCqoLxEYPVb6VXvFq2V6euBU6CSFgAOURSQp3vpJGe
         Oc7qnermMdlwrNqupAg6FSiNq5wLRRqaq2fFdZxBwnvJS3zkF62qS7YYdArWfEPDPSDF
         lL7KL9liXgLcABVDx2R1kc10QggTw7Jesz354JYoXfQ4m4gUmyZpnM0aN0DaolVeanTB
         xgqQ2V8H+LjkbtuLV5SV+TqDa7yWWiCVq7f0rq927cVmdpIY/1uP0wPUAWS+hYduxXX1
         777B1MicFcfJu0juVcnYoJKvPa6L4NfEQGcNOOgD048yv3nrHGFZPtdoI9uJ8UWeYrVh
         yqPg==
X-Gm-Message-State: AOJu0YyRzwJZuR9GOL1WIucyYrX1NynS1ScPqz33nB5H0EoGhRKCEr2n
	FoAS1Q7+uwd3VPJSqLAvSQFZC/bplLLGQrZhzuKNX6kAoXQj1bA9bYc8YLn1iTf9VmdlBm8vXtF
	e
X-Google-Smtp-Source: AGHT+IFm+od/7YzI5/UCoJ7VKQ8UEdeF+iHN7TtG2XUW07j1auYpDJebl860+/4zkt5ps5FV6CbOKw==
X-Received: by 2002:a05:6358:8414:b0:17b:78f1:b4ea with SMTP id b20-20020a056358841400b0017b78f1b4eamr2186438rwk.28.1709212530508;
        Thu, 29 Feb 2024 05:15:30 -0800 (PST)
From: Roger Pau Monne <roger.pau@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Roger Pau Monne <roger.pau@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Paul Durrant <paul@xen.org>,
	Stewart Hildebrand <stewart.hildebrand@amd.com>
Subject: [PATCH] pci: fix locking around vPCI removal in pci_remove_device()
Date: Thu, 29 Feb 2024 14:15:25 +0100
Message-ID: <20240229131525.19099-1-roger.pau@citrix.com>
X-Mailer: git-send-email 2.44.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Currently vpci_deassign_device() is called without holding the per-domain
pci_lock in pci_remove_device(), which leads to:

Assertion 'rw_is_write_locked(&pdev->domain->pci_lock)' failed at ../drivers/vpci/vpci.c:47
[...]
Xen call trace:
   [<ffff82d040260eac>] R vpci_deassign_device+0x10d/0x1b9
   [<ffff82d04027932f>] S pci_remove_device+0x2b1/0x380
   [<ffff82d040260bd0>] F pci_physdev_op+0x197/0x19e
   [<ffff82d04032272d>] F do_physdev_op+0x342/0x12aa
   [<ffff82d0402f067a>] F pv_hypercall+0x58e/0x62b
   [<ffff82d0402012ba>] F lstar_enter+0x13a/0x140

Move the existing block that removes the device from the domain pdev_list ahead
and also issue the call to vpci_deassign_device() there.  It's fine to remove
the device from the domain list of assigned devices, as further functions only
care that the pdev domain field is correctly set to the owner of the device
about to be removed.

Moving the vpci_deassign_device() past the pci_cleanup_msi() call can be
dangerous, as doing the MSI cleanup ahead of having removed the vPCI handlers
could lead to stale data in vPCI MSI(-X) internal structures.

Fixes: 4f78438b45e2 ('vpci: use per-domain PCI lock to protect vpci structure')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/drivers/passthrough/pci.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 4c0a836486ec..194701c9137d 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -817,15 +817,15 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
     list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
         if ( pdev->bus == bus && pdev->devfn == devfn )
         {
-            vpci_deassign_device(pdev);
-            pci_cleanup_msi(pdev);
-            ret = iommu_remove_device(pdev);
             if ( pdev->domain )
             {
                 write_lock(&pdev->domain->pci_lock);
+                vpci_deassign_device(pdev);
                 list_del(&pdev->domain_list);
                 write_unlock(&pdev->domain->pci_lock);
             }
+            pci_cleanup_msi(pdev);
+            ret = iommu_remove_device(pdev);
             printk(XENLOG_DEBUG "PCI remove device %pp\n", &pdev->sbdf);
             free_pdev(pseg, pdev);
             break;
-- 
2.44.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:23:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:23:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687094.1069948 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgNd-0003pS-8t; Thu, 29 Feb 2024 13:23:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687094.1069948; Thu, 29 Feb 2024 13:23:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgNd-0003pL-6C; Thu, 29 Feb 2024 13:23:21 +0000
Received: by outflank-mailman (input) for mailman id 687094;
 Thu, 29 Feb 2024 13:23:20 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfgNc-0003p3-3v
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:23:20 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b353b0e2-d705-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 14:23:18 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a36126ee41eso154084666b.2
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:23:17 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 gb18-20020a170907961200b00a3efa4e033asm680118ejc.151.2024.02.29.05.23.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 05:23:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b353b0e2-d705-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709212997; x=1709817797; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=OIlpSUibrCGxxNEFFk2NHMnPuwzvvO9KjQXdlo3fam4=;
        b=X7FkwMaxR0PMwcv98lRIHSBWaKmp24VkO6+M2hYvGhEwIxm0evJa75fRWOMiZ/Gx1n
         sCcBvlk9kA2MS9sC4FEJ1zHrWjpqC4V94qqJAvc1bkadfI9o7D/sLzTlBxA1+DObuE9W
         ZnW2V7gyoVd0aMqSsjkxeFEhBnSsGIj7L+PqjRctVwpz3TdRRCpZ8UFG6nDnapep+Oo4
         nnN3VwJnkXvjstY1Y6Z0TXQquzaXC/O50JoPs5NSaYCJBHiQCxYpIGMgNYTaUqVdsUPh
         1XUnpp68B2cz8ADhMOrmcrkjUI1CuAFkZE94f0Hub3MsDKTI9PdTcmpuqhjspeto0J88
         zYgA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709212997; x=1709817797;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=OIlpSUibrCGxxNEFFk2NHMnPuwzvvO9KjQXdlo3fam4=;
        b=nDYfJIS4IhHWxUdRtePW1eCx9qyXDXtN1ImrBk/+tNoFIxgRWLzs0Agng3XIq4vi4N
         ea9/TsoOnQUtdFyctiq60Onsks+IA3X2uXaXPuosd5tw6RgqCFLrMRk0NmDQWbdbRSpD
         anTruA0QnccyTU+PZvx1m8cyle9f50PP8GlK20O/ejsaOeQb0LfFWaV64CiZpibupeDh
         jt5xYWIvxlrzaeYMvgj7nSy1LvpdBgxf0xq9AsRnlWJNlcDP0xFZ8eMoZ77oAK36faaT
         takkqh8errHKxsmUnPoX6hj7AOKXKmtP/U4cgrbHG9p9VIufx8kapGRfQ+zX5xRtfs31
         LlCQ==
X-Forwarded-Encrypted: i=1; AJvYcCVCt9b5s1P5yNMECvHiHu9YwoxgbwCp83Nd6n6vxYnw11ewY1Z269dZ/XUpM7cIOcBozVFMVzNkKmieLP+eE1auYESVbRxWs/KZbJB8hQA=
X-Gm-Message-State: AOJu0YwlfnvGsYrukwaOUQrDYU7u34mJe0P7Jcw034nmdwoeqtCGcivC
	pqqUYqA+E+lKfayGJrWffs9u0CfvdTtMhV5XcK6KVgsNnZPItdTouNbUyYBy8Q==
X-Google-Smtp-Source: AGHT+IF4VjaypXJweyqY+HBHnsTuyLsGPbWapDIRwUVXGLrurLzWm+JSjMgWDtKOXPkTcN3C3a+XrQ==
X-Received: by 2002:a17:906:d8b4:b0:a3f:2ffd:c68b with SMTP id qc20-20020a170906d8b400b00a3f2ffdc68bmr1641034ejb.6.1709212997377;
        Thu, 29 Feb 2024 05:23:17 -0800 (PST)
Message-ID: <29bebf83-a78b-440f-a642-0ec9aa40c0d7@suse.com>
Date: Thu, 29 Feb 2024 14:23:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] pci: fix locking around vPCI removal in
 pci_remove_device()
Content-Language: en-US
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Paul Durrant <paul@xen.org>,
 Stewart Hildebrand <stewart.hildebrand@amd.com>,
 xen-devel@lists.xenproject.org
References: <20240229131525.19099-1-roger.pau@citrix.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20240229131525.19099-1-roger.pau@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 14:15, Roger Pau Monne wrote:
> Currently vpci_deassign_device() is called without holding the per-domain
> pci_lock in pci_remove_device(), which leads to:
> 
> Assertion 'rw_is_write_locked(&pdev->domain->pci_lock)' failed at ../drivers/vpci/vpci.c:47
> [...]
> Xen call trace:
>    [<ffff82d040260eac>] R vpci_deassign_device+0x10d/0x1b9
>    [<ffff82d04027932f>] S pci_remove_device+0x2b1/0x380
>    [<ffff82d040260bd0>] F pci_physdev_op+0x197/0x19e
>    [<ffff82d04032272d>] F do_physdev_op+0x342/0x12aa
>    [<ffff82d0402f067a>] F pv_hypercall+0x58e/0x62b
>    [<ffff82d0402012ba>] F lstar_enter+0x13a/0x140
> 
> Move the existing block that removes the device from the domain pdev_list ahead
> and also issue the call to vpci_deassign_device() there.  It's fine to remove
> the device from the domain list of assigned devices, as further functions only
> care that the pdev domain field is correctly set to the owner of the device
> about to be removed.
> 
> Moving the vpci_deassign_device() past the pci_cleanup_msi() call can be
> dangerous, as doing the MSI cleanup ahead of having removed the vPCI handlers
> could lead to stale data in vPCI MSI(-X) internal structures.
> 
> Fixes: 4f78438b45e2 ('vpci: use per-domain PCI lock to protect vpci structure')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -817,15 +817,15 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
>      list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list )
>          if ( pdev->bus == bus && pdev->devfn == devfn )
>          {
> -            vpci_deassign_device(pdev);
> -            pci_cleanup_msi(pdev);
> -            ret = iommu_remove_device(pdev);
>              if ( pdev->domain )
>              {
>                  write_lock(&pdev->domain->pci_lock);
> +                vpci_deassign_device(pdev);
>                  list_del(&pdev->domain_list);
>                  write_unlock(&pdev->domain->pci_lock);

This then also is properly symmetric with what pci_add_device() has.

Jan

>              }
> +            pci_cleanup_msi(pdev);
> +            ret = iommu_remove_device(pdev);
>              printk(XENLOG_DEBUG "PCI remove device %pp\n", &pdev->sbdf);
>              free_pdev(pseg, pdev);
>              break;



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:23:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:23:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687099.1069969 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgO3-0004VH-L8; Thu, 29 Feb 2024 13:23:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687099.1069969; Thu, 29 Feb 2024 13:23:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgO3-0004UQ-IX; Thu, 29 Feb 2024 13:23:47 +0000
Received: by outflank-mailman (input) for mailman id 687099;
 Thu, 29 Feb 2024 13:23:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfgO2-0003p3-Hv
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:23:46 +0000
Received: from mail-qv1-xf33.google.com (mail-qv1-xf33.google.com
 [2607:f8b0:4864:20::f33])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id c30fbea2-d705-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 14:23:44 +0100 (CET)
Received: by mail-qv1-xf33.google.com with SMTP id
 6a1803df08f44-690494d2e97so3353316d6.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:23:44 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 b19-20020a0cf053000000b0068f6130e80dsm717623qvl.69.2024.02.29.05.23.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 05:23:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c30fbea2-d705-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709213024; x=1709817824; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=3xkkDsIfnCjDOGW9TDsukCtuFF7czac82KzI4GF5Lho=;
        b=I0qtFJyrapZDHK3lpBoUnB9P/q8ZDY1FrmMFNb0nDLZM+5a1or8gGMST31kmfLGjAD
         ByFO1omm6LLX3TKApim1hpW1UTHInR26UmhnU2Z7KCPtPxN0b8v4Tm51P23qYUyz6LxF
         783SESktBtmrAnTcGi66sVj5EPzLQZa9wiulA=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709213024; x=1709817824;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=3xkkDsIfnCjDOGW9TDsukCtuFF7czac82KzI4GF5Lho=;
        b=Dc58xzdlCZK+rGP5KjLDgY3AuUqlmk+lxaCewuMS4sH6VGzFirdD4jUhiSqPmmsTaT
         QwwMQcYu9G1N30cnZ/pE70vo16psh1dviyNaw5ruuubn0lRSMi9tJXaf4XOCRDLgwHeD
         RoLJ7Nbj6Fy149I1DPnwETOBkZuBVc4h3bBE91GZt65Tsn8fJ/Lp1H5fNTHtPx+UsjNR
         YSgJZaiRFtmqVs+Wo2BShq4L4Ie5HPTAIhmSEM6KjICnhAMvfvWCjoWvX9U7OClAlkal
         xwN4h5zvjPFYJ1cSt2jPudP6+NwEUAzCQQ7RhICdSwVxVuC/yneXrMAX2xpiF8M4AYzS
         hIxA==
X-Forwarded-Encrypted: i=1; AJvYcCXi9kF9lypOn71IgHZAgN/2s0fZGEOMoVg8xXwKgXafujJQQeBsRVJMabWczBAzemr7RXqfCxxcM5yYG/7AJFQcaE4bIxCN41JcaMMiDk0=
X-Gm-Message-State: AOJu0Yygk7CcrYRIjUtnfQGyp+rDO9IMULVp6mUJVS0mmK+BpLR72OkY
	p+9BmgYLP1NZlDqs5rqw4QyymsRg9LHb+dvhB3tkQ8h+Xy+u/GcpDtnw8GpmkbQ=
X-Google-Smtp-Source: AGHT+IGz3kZmMvSQwBxgEr/slFaR0FwkaGOqs8g3kEUGGBfz6plBAVAlpbUpY7HFfj+rvqUZ7E4Geg==
X-Received: by 2002:a05:6214:d69:b0:68f:a0c2:7341 with SMTP id 9-20020a0562140d6900b0068fa0c27341mr2857308qvs.13.1709213023778;
        Thu, 29 Feb 2024 05:23:43 -0800 (PST)
Message-ID: <623e9c0e-f05f-41e7-a800-0bba01cba7f4@citrix.com>
Date: Thu, 29 Feb 2024 13:23:41 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpu-policy: Fix x2APIC visibility for PV guests
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240229104304.2478614-1-andrew.cooper3@citrix.com>
 <707ff486-b448-4bc0-be7e-1c692b360734@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <707ff486-b448-4bc0-be7e-1c692b360734@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/02/2024 12:47 pm, Jan Beulich wrote:
> On 29.02.2024 11:43, Andrew Cooper wrote:
>> Right now, the host x2APIC setting filters into the PV max and default
>> policies, yet PV guests cannot set MSR_APIC_BASE.EXTD or access any of the
>> x2APIC MSR range.  Therefore they absolutely shouldn't see the x2APIC bit.
>>
>> Linux has workarounds for the collateral damage caused by this leakage; it
>> unconditionally filters out the x2APIC CPUID bit, and EXTD when reading
>> MSR_APIC_BASE.
>>
>> Hide the x2APIC bit in the PV default policy, but for compatibility, tolerate
>> incoming VMs which already saw the bit.  This is logic from before the
>> default/max split in Xen 4.14 which wasn't correctly adjusted at the time.
> What about guest_cpuid()'s handling of leaf 0xb then? The %edx value
> will change once a guest is rebooted, aiui. The comment in
> recalculate_cpuid_policy() that you update refers to that.

That comment is going in the next patch irrespective.

But yes - this will change leaf 0xb from being
host-conditionally-visible to always hidden.

PV guests don't have any coherent idea of topology.  Linux (with the
topo fixes) now explicitly ignores everything it can see and just fakes
up a flat non-SMT topology in a single package.

For PV dom0, where it sees the host MADT, the values in leaf 0xb are
bogus as they don't come from the same enumeration space.
>> Update the annotation from !A to !S which slightly better describes that it
>> doesn't really exist in PV guests.  HVM guests, for which x2APIC can be
>> emulated completely, already has it unconditionally set in the max policy.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>> CC: Jan Beulich <JBeulich@suse.com>
>> CC: Roger Pau Monné <roger.pau@citrix.com>
>> CC: Wei Liu <wl@xen.org>
>>
>> This wants backporting as far as people can tollerate, but it's really not
>> obvious which commit in 4.14 should be referenced in a Fixes: tag.
> Why 4.14? In 4.7.0 I see ...
>
>> @@ -830,11 +846,10 @@ void recalculate_cpuid_policy(struct domain *d)
>>      }
>>  
>>      /*
>> -     * Allow the toolstack to set HTT, X2APIC and CMP_LEGACY.  These bits
>> +     * Allow the toolstack to set HTT and CMP_LEGACY.  These bits
>>       * affect how to interpret topology information in other cpuid leaves.
>>       */
>>      __set_bit(X86_FEATURE_HTT, max_fs);
>> -    __set_bit(X86_FEATURE_X2APIC, max_fs);
>>      __set_bit(X86_FEATURE_CMP_LEGACY, max_fs);
>>  
>>      /*
> ... these adjustments, just still in calculate_pv_featureset(). I
> haven't gone further backwards to check if/when this exposure has
> really appeared. I wouldn't be surprised if it's been like that
> for all the time since we gained x2APIC support in the hypervisor.

4.14 was when we got the proper default vs max split.  Before then, this
block of logic was an opencoded "max(ish) for tookstacks which know
about it" kind of thing.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:25:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:25:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687110.1070016 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgPy-0005kk-Eq; Thu, 29 Feb 2024 13:25:46 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687110.1070016; Thu, 29 Feb 2024 13:25:46 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgPy-0005kb-Be; Thu, 29 Feb 2024 13:25:46 +0000
Received: by outflank-mailman (input) for mailman id 687110;
 Thu, 29 Feb 2024 13:25:45 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rfgPx-0005kU-7k
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:25:45 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 09e920ca-d706-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 14:25:43 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id BA2F52201C;
 Thu, 29 Feb 2024 13:25:42 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 916A413503;
 Thu, 29 Feb 2024 13:25:42 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id vgX/IdaF4GWQEQAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 13:25:42 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 09e920ca-d706-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709213142; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=LghDOadVy2nZN2AlqhQaXyGtoxv8ZZ6/rYxKpGZkLgY=;
	b=GSQNsrZPtQDL0wUzNBNWvkIfrEhxvvlgeiwSu6NO9nrtgxq7TdmCJeqOXDht+1JjeReNBK
	XzsvqqODfGhZLz+9BA/jrnIpYysrHjdStdjZyUo4gkby1svW/WBHrpVX/avLoLTAHPGYaO
	azzzqqL4Whty9LT+S8aKqV4+/qm1qDU=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709213142; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=LghDOadVy2nZN2AlqhQaXyGtoxv8ZZ6/rYxKpGZkLgY=;
	b=GSQNsrZPtQDL0wUzNBNWvkIfrEhxvvlgeiwSu6NO9nrtgxq7TdmCJeqOXDht+1JjeReNBK
	XzsvqqODfGhZLz+9BA/jrnIpYysrHjdStdjZyUo4gkby1svW/WBHrpVX/avLoLTAHPGYaO
	azzzqqL4Whty9LT+S8aKqV4+/qm1qDU=
Message-ID: <5e755279-2c77-4542-852d-7d4acb4ccd76@suse.com>
Date: Thu, 29 Feb 2024 14:25:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v9 2/6] stubdom: extend xenstore stubdom configs
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: Samuel Thibault <samuel.thibault@ens-lyon.org>,
 Jason Andryuk <jandryuk@gmail.com>, xen-devel@lists.xenproject.org
References: <20240229124830.13680-1-jgross@suse.com>
 <20240229124830.13680-3-jgross@suse.com>
 <6378d51b-ab4d-4019-970c-b8879c75b7d9@suse.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <6378d51b-ab4d-4019-970c-b8879c75b7d9@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Authentication-Results: smtp-out1.suse.de;
	none
X-Spamd-Result: default: False [-1.45 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 RCPT_COUNT_THREE(0.00)[4];
	 FREEMAIL_ENVRCPT(0.00)[gmail.com];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 TO_DN_SOME(0.00)[];
	 BAYES_HAM(-2.20)[96.17%];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.83)[subject];
	 FREEMAIL_CC(0.00)[ens-lyon.org,gmail.com,lists.xenproject.org];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Level: 
X-Spam-Flag: NO
X-Spam-Score: -1.45

On 29.02.24 14:05, Jan Beulich wrote:
> On 29.02.2024 13:48, Juergen Gross wrote:
>> Extend the config files of the Xenstore stubdoms to include XENBUS
>> and 9PFRONT items in order to support file based logging.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
> 
> Was an ack from Samuel lost here? Or was it dropped on purpose?

Oh, I think I lost it.


Juergen



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:25:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:25:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687111.1070026 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgQ5-00061t-La; Thu, 29 Feb 2024 13:25:53 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687111.1070026; Thu, 29 Feb 2024 13:25:53 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgQ5-00061m-IR; Thu, 29 Feb 2024 13:25:53 +0000
Received: by outflank-mailman (input) for mailman id 687111;
 Thu, 29 Feb 2024 13:25:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfgQ3-00060p-Lo; Thu, 29 Feb 2024 13:25:51 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfgQ3-0004wY-Jf; Thu, 29 Feb 2024 13:25:51 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfgQ3-0002DO-7u; Thu, 29 Feb 2024 13:25:51 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfgQ3-0002jB-7L; Thu, 29 Feb 2024 13:25:51 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=X4gvCxnTM0+zlCZTGQjz8kC5O5z/XY7xEcu/SuPOVMw=; b=oSZaqcPnnYfZOBy3AAKGPIrkxj
	w6UuVbnVaI5eSWq1k+T1e/1OQ/2XhFcj4wJsveSsYN0JfD15AsnXOp8/5HePq6SlcA6nei7ntDCkG
	KLZvDGummxXwTuA8AG66fOI5ho1cvKE4xUpwy+3k+fZALZzcZv6t38lEtcpQe1cMrw90=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184811-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable test] 184811: tolerable FAIL - PUSHED
X-Osstest-Failures:
    xen-unstable:test-armhf-armhf-xl-vhd:guest-start/debian.repeat:fail:heisenbug
    xen-unstable:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-unstable:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-unstable:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=195e75371b13c4f7ecdf7b5c50aed0d02f2d7ce8
X-Osstest-Versions-That:
    xen=ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 29 Feb 2024 13:25:51 +0000

flight 184811 xen-unstable real [real]
flight 184819 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184811/
http://logs.test-lab.xenproject.org/osstest/logs/184819/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-vhd 17 guest-start/debian.repeat fail pass in 184819-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184796
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184796
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop             fail like 184796
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184796
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184796
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184796
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184796
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184796
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184796
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184796
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184796
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184796
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  195e75371b13c4f7ecdf7b5c50aed0d02f2d7ce8
baseline version:
 xen                  ca7c872c70cb58f1e4b59f8eb619ffa0ecc1ed87

Last test of basis   184796  2024-02-27 18:09:07 Z    1 days
Testing same since   184811  2024-02-28 19:28:00 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Jens Wiklander <jens.wiklander@linaro.org>
  Julien Grall <jgrall@amazon.com>
  Michal Orzel <michal.orzel@amd.com>
  Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Shawn Anastasio <sanastasio@raptorengineering.com>
  Stewart Hildebrand <stewart.hildebrand@amd.com>
  Volodymyr Babchuk <volodymyr_babchuk@epam.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-i386-examine-bios                                 pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-i386-examine                                      pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-i386-examine-uefi                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      fail    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   ca7c872c70..195e75371b  195e75371b13c4f7ecdf7b5c50aed0d02f2d7ce8 -> master


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:30:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:30:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687122.1070035 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgU7-0007nK-BM; Thu, 29 Feb 2024 13:30:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687122.1070035; Thu, 29 Feb 2024 13:30:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgU7-0007mh-8G; Thu, 29 Feb 2024 13:30:03 +0000
Received: by outflank-mailman (input) for mailman id 687122;
 Thu, 29 Feb 2024 13:30:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfgU5-0007Q0-CW
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:30:01 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a2fe5cef-d706-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 14:30:00 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a28a6cef709so144555666b.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:30:00 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 ts6-20020a170907c5c600b00a443858cf78sm688122ejc.185.2024.02.29.05.29.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 05:29:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a2fe5cef-d706-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709213400; x=1709818200; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=YvmJUgixnktpYQND47KyjYjCa+vl6iYdDcZMASTpmUc=;
        b=b5ATq1egUInjkFXydlHftmM/M88DXM+PsSuk4XPJl6HQVfxV4pW+adUyt5sV/Ptd7c
         hFzczgt4sQ7qEo8KOH0gLFSP7HFTTu+REqISuBeIOv80+GCaY2RKhBfmwmSKtiQBnXgv
         +ZOtuqtXLV1DRFd5wVICkLH6gprLlwHlHTzWa2w9ZBOCpAnb4SdBldylIb+8Yze2AFvn
         u3fen3jfoK+Z8Jl8LpgGSejZjzbVHYoxBShqwzOJIcBGxnMAMjxeQUx4YRRsa+ZflaW1
         10q7MP68P9ZR4sPaxlDpN5stH4eYMIkcXYd8LhH/ADIzazRsrJLgEzEeKPO+61Bf0BkQ
         3I2Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709213400; x=1709818200;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=YvmJUgixnktpYQND47KyjYjCa+vl6iYdDcZMASTpmUc=;
        b=c2jnPhV+ejYKd2OdiVDpvzVTPPDD34yNoca4vXwCUXhyU8rNBt7nLb3cGycpYQjsF3
         cfJ1f8iRzRLq+dH1GD8EwMDCaFVmxAeoFJTVyoMy77iVb0Ml7qHSopphED8rjdoMUHPU
         jb7r33bnHOG5cmNqbWh1o5JLeVjOdUSd2IkOODq5lG0WaFX/Y58DYLVd4rSCgPAE1qpp
         N/t3lRiDWKJv5/Za9O6pO+jbrdb7RUTtDdB+lFOXhPYd7uuVdASAWdar5OQwjmQHWDsF
         3+myVMY2TwYl7R2RzknWbiFste9NMroY01PqVlyRwWgs4BtXvOl548sZ4mgoz7HgsaAM
         QcQA==
X-Forwarded-Encrypted: i=1; AJvYcCULOqQ0VRmKmzr47vVajXgNKTqHiOsZnepopsruezSC3fae6n+l5fCyziLXZ4rj8P6SfyoHW2xSk1ksNBB6W7V1tAcYIrYO82kqPk8yIQA=
X-Gm-Message-State: AOJu0Yxd91ntD2YlJ1pqkqV7mbvKZSwUJ6h7+v36sc0OynzVaNeE42TL
	8tPEJRjt3BKWgsizWfiyYrS3697SjixjL8MJIEZmaP7o0CkluYFGdT41MmduhQ==
X-Google-Smtp-Source: AGHT+IFNpNMgIZSB5VcDdo6p03Y2wvWPnxLy46yfdLaHDyaL2fqYFY0VqMpeaPi8YrhuhQ+jBe22Lw==
X-Received: by 2002:a17:906:48b:b0:a43:1e2b:b069 with SMTP id f11-20020a170906048b00b00a431e2bb069mr1501461eja.4.1709213399705;
        Thu, 29 Feb 2024 05:29:59 -0800 (PST)
Message-ID: <e811b3b0-a3d6-488d-b93c-e24c416477bf@suse.com>
Date: Thu, 29 Feb 2024 14:29:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpu-policy: Fix x2APIC visibility for PV guests
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240229104304.2478614-1-andrew.cooper3@citrix.com>
 <707ff486-b448-4bc0-be7e-1c692b360734@suse.com>
 <623e9c0e-f05f-41e7-a800-0bba01cba7f4@citrix.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <623e9c0e-f05f-41e7-a800-0bba01cba7f4@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 14:23, Andrew Cooper wrote:
> On 29/02/2024 12:47 pm, Jan Beulich wrote:
>> On 29.02.2024 11:43, Andrew Cooper wrote:
>>> Right now, the host x2APIC setting filters into the PV max and default
>>> policies, yet PV guests cannot set MSR_APIC_BASE.EXTD or access any of the
>>> x2APIC MSR range.  Therefore they absolutely shouldn't see the x2APIC bit.
>>>
>>> Linux has workarounds for the collateral damage caused by this leakage; it
>>> unconditionally filters out the x2APIC CPUID bit, and EXTD when reading
>>> MSR_APIC_BASE.
>>>
>>> Hide the x2APIC bit in the PV default policy, but for compatibility, tolerate
>>> incoming VMs which already saw the bit.  This is logic from before the
>>> default/max split in Xen 4.14 which wasn't correctly adjusted at the time.
>> What about guest_cpuid()'s handling of leaf 0xb then? The %edx value
>> will change once a guest is rebooted, aiui. The comment in
>> recalculate_cpuid_policy() that you update refers to that.
> 
> That comment is going in the next patch irrespective.
> 
> But yes - this will change leaf 0xb from being
> host-conditionally-visible to always hidden.

Imo this wants saying explicitly, including why that's okay to do,
especially since ...

> PV guests don't have any coherent idea of topology.  Linux (with the
> topo fixes) now explicitly ignores everything it can see and just fakes
> up a flat non-SMT topology in a single package.

... you validly use "now" here. Plus Linux isn't the only PV guest we
need to care about.

What's wrong (more wrong than the present putting of vCPU ID * 2 there)
with retaining the population of that leaf (by dropping the x2apic
dependency there)?

>>> This wants backporting as far as people can tollerate, but it's really not
>>> obvious which commit in 4.14 should be referenced in a Fixes: tag.
>> Why 4.14? In 4.7.0 I see ...
>>
>>> @@ -830,11 +846,10 @@ void recalculate_cpuid_policy(struct domain *d)
>>>      }
>>>  
>>>      /*
>>> -     * Allow the toolstack to set HTT, X2APIC and CMP_LEGACY.  These bits
>>> +     * Allow the toolstack to set HTT and CMP_LEGACY.  These bits
>>>       * affect how to interpret topology information in other cpuid leaves.
>>>       */
>>>      __set_bit(X86_FEATURE_HTT, max_fs);
>>> -    __set_bit(X86_FEATURE_X2APIC, max_fs);
>>>      __set_bit(X86_FEATURE_CMP_LEGACY, max_fs);
>>>  
>>>      /*
>> ... these adjustments, just still in calculate_pv_featureset(). I
>> haven't gone further backwards to check if/when this exposure has
>> really appeared. I wouldn't be surprised if it's been like that
>> for all the time since we gained x2APIC support in the hypervisor.
> 
> 4.14 was when we got the proper default vs max split.  Before then, this
> block of logic was an opencoded "max(ish) for tookstacks which know
> about it" kind of thing.

Except it was also affecting what guests get to see, afaict.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:44:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:44:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687125.1070046 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfghq-0003X9-Hx; Thu, 29 Feb 2024 13:44:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687125.1070046; Thu, 29 Feb 2024 13:44:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfghq-0003X2-Ee; Thu, 29 Feb 2024 13:44:14 +0000
Received: by outflank-mailman (input) for mailman id 687125;
 Thu, 29 Feb 2024 13:44:12 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfgho-0003Ww-QA
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:44:12 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfghn-0005JO-Ly; Thu, 29 Feb 2024 13:44:11 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfghn-0005nE-D7; Thu, 29 Feb 2024 13:44:11 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=o3SxjlVNtj9SkhQGFSMFDYXEZDnh9/s8sp4lHtTa3Fg=; b=MSF/BfBta2gS/W3uXbyTwHcVx8
	dM/JuNuXyij1OSXebKRb8eOGSAEOUHtbwZoP7TzvrAfdLX3c2+YoG2xclqjs3QIgA/W5PDtbuF3BZ
	WPI8fJWLdkxy94M95iY0MzdPZ7zqWQObOPhzjS1dDK/EstnV1h1fDfZHA3jKP5pHVXR4=;
Message-ID: <8eed0200-eabf-4024-b018-74be45ef95db@xen.org>
Date: Thu, 29 Feb 2024 13:44:09 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko
 <oleksii.kurochko@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
 <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
 <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com>
 <2a112321-f4fd-4ab4-a431-a0bb4cb6eaa2@suse.com>
 <9b212dec-59ef-4cbb-bea7-add3867e4599@xen.org>
 <aa3fbfca-4675-4f92-9037-a4a7c5eca4dd@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <aa3fbfca-4675-4f92-9037-a4a7c5eca4dd@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi Jan,

On 29/02/2024 12:51, Jan Beulich wrote:
> On 29.02.2024 13:32, Julien Grall wrote:
>> On 29/02/2024 12:17, Jan Beulich wrote:
>>> On 29.02.2024 13:05, Andrew Cooper wrote:
>>>> On 29/02/2024 10:23 am, Julien Grall wrote:
>>>>>>>> IOW it is hard for me to see why RISC-V needs stronger restrictions
>>>>>>>> here
>>>>>>>> than other architectures. It ought to be possible to determine a
>>>>>>>> baseline
>>>>>>>> version. Even if taking the desire to have "pause" available as a
>>>>>>>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
>>>>>>>
>>>>>>> I think we want to bump it on Arm. There are zero reasons to try to
>>>>>>> keep
>>>>>>> a lower versions if nobody tests/use it in production.
>>>>>>>
>>>>>>> I would suggest to do the same on x86. What's the point of try to
>>>>>>> support Xen with a 15+ years old compiler?
>>>>>>
>>>>>> It could have long been bumped if only a proper scheme to follow for
>>>>>> this and future bumping would have been put forward by anyone keen on
>>>>>> such bumping, like - see his reply - e.g. Andrew. You may recall that
>>>>>> this was discussed more than once on meetings, with no real outcome.
>>>>>> I'm personally not meaning to stand in the way of such bumping as long
>>>>>> as it's done in a predictable manner, but I'm not keen on doing so and
>>>>>> hence I don't view it as my obligation to try to invent a reasonable
>>>>>> scheme. (My personal view is that basic functionality should be
>>>>>> possible to have virtually everywhere, whereas for advanced stuff it
>>>>>> is fine to require a more modern tool chain.)
>>>>>
>>>>> That's one way to see it. The problem with this statement is a user
>>>>> today is mislead to think you can build Xen with any GCC versions
>>>>> since 4.1. I don't believe we can guarantee that and we are exposing
>>>>> our users to unnecessary risk.
>>>>>
>>>>> In addition to that, I agree with Andrew. This is preventing us to
>>>>> improve our code base and we have to carry hacks for older compilers.
>>>>
>>>> I don't think anyone here is suggesting that we switch to a
>>>> bleeding-edge-only policy.  But 15y of support is extreme in the
>>>> opposite direction.
>>>>
>>>> Xen ought to be buildable in the contemporary distros of the day, and I
>>>> don't think anyone is going to credibly argue otherwise.
>>>>
>>>> But, it's also fine for new things to have newer requirements.
>>>>
>>>> Take CET for example.  I know we have disagreements on exactly how it's
>>>> toolchain-conditionalness is implemented, but the basic principle of "If
>>>> you want shiny new optional feature $X, you need newer toolchain $Y" is
>>>> entirely fine.
>>>>
>>>> A brand new architecture is exactly the same.  Saying "this is the
>>>> minimum, because it's what we test" doesn't preclude someone coming
>>>> along and saying "can we use $N-1 ?  See here it works, and here's a
>>>> change to CI test it".
>>>>
>>>>
>>>> Anyway, its clear we need to write some policy on this, before making
>>>> specific adjustments.  To get started, is there going to be any
>>>> objection whatsoever on some principles which begin as follows:
>>>
>>> Largely not, but one aspect needs clarifying up front:
>>>
>>>> * For established architectures, we expect Xen to be buildable on the
>>>> common contemporary distros.  (i.e. minima is not newer than what's
>>>> available in contemporary distros, without a good reason)
>>>
>>> What counts as contemporary distro? Still in normal support? LTS? Yet
>>> more extreme forms?
>>
>> LTS makes sense. More I am not sure. I am under the impression that
>> people using older distros are those that wants a stable system. So they
>> would unlikely try to upgrade the hypervisor.
>>
>> Even for LTS, I would argue that if it has been released 5 years ago,
>> then you probably want to update it at the same time as moving to a
>> newer Xen version.
> 
> For the purposes of distros I agree. For the purposes of individuals
> I don't: What's wrong with running a newer hypervisor and/or kernel
> underneath an older distro?

There is nothing wrong. I just don't understand the benefits for us to 
support that use case. To me there are two sorts of individuals:
  1. The ones that are using distro packages. They will unlikely want to 
switch to a newer hypervisor
  2. The ones that are happy to compile and hack their system. Fairly 
likely they will use a more distros and/or would not be put up by 
upgrading it.

What individuals do you have in mind?

Also, for me, the minimum doesn't prevent anyone to try to compile with 
an older compiler. It is only here to say that as a community we will 
not investigate or trying to workaround bugs in those compilers.

I don't see the problem with that if someone decide to use an older dom0 
distros with a newer hypervisor. It would likely not be the only issue 
they will have.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:49:29 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:49:29 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687128.1070056 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgmp-0004Bh-2p; Thu, 29 Feb 2024 13:49:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687128.1070056; Thu, 29 Feb 2024 13:49:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgmp-0004Ba-07; Thu, 29 Feb 2024 13:49:23 +0000
Received: by outflank-mailman (input) for mailman id 687128;
 Thu, 29 Feb 2024 13:49:21 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfgmn-0004BU-KR
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:49:21 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 561c55a9-d709-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 14:49:19 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a3e7f7b3d95so140875666b.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:49:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 pv25-20020a170907209900b00a3ee9305b02sm705750ejb.20.2024.02.29.05.49.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 05:49:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 561c55a9-d709-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709214559; x=1709819359; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=Tmw1rCRlcBdk+SpLJM+j+TD/uGvXzBehuhQ32pC4rsc=;
        b=Jm06GsEKQ2N8vDM/sBnI3gsupy9K8/SDhq4cuK8xMZ5z+HJs5eVSk+gR0WYHQfj3Ay
         f8/PlpsTChrYKksF5FIngCUmgsD3hDalRbDUaYsFXtFoBdF+cD4f6CB3FVJmuBbpMOYo
         2hbNNheWqrtHbQLrdwQo0tpaQ4PCIwJGyh4ceTMZIuaK9SKXH17cl/qfln7m4U8Bkyko
         GlZuNSahOYht1FJJ3Jc2eTB7JYgZy4hT+muKsh7hHmuJ1oL/R8zkTfxbQ8LLWut2vywD
         sI6VP6PzXP2XqtRYWLhGhPIhJoNGg2qDtEgXZQj2YmQDwxjWXDunFGSaCIEA8DwHn49s
         4arQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709214559; x=1709819359;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=Tmw1rCRlcBdk+SpLJM+j+TD/uGvXzBehuhQ32pC4rsc=;
        b=raua1tCB2sWApheOsTkJfdvyOEflipgUOPYIp3DFs1YpAJJYrGixN3lqDYizgvAvgM
         YKogL/5d2hLmW4oOTb1E1bBfeMCCTQT7iWFSxd8RknEd0RT/gmDrrsMRqvNbdusCCOec
         5wRzVgkgm7tN6fFdzO78krm7gpBj8r6CSGa6g8iRfdqtxJf4e2gUbQUcmwtzTfyUaIa6
         sHEqtTjb7W+f98j1VbEWcaI/pP/A8+PwjCj37p7/z6rcvmemXX41xxP+unexzprq7+0Z
         FNJiyzq9zMmDCGCjKcbeQB+Mq5YBZFGWHZ3XE3gkNDdnZ33i5SCLmp45SA2UV80N32Sr
         pZJQ==
X-Forwarded-Encrypted: i=1; AJvYcCWrT+cL0bsWYA6JzBcfz3f/w9tcjO0rqWoz0S85/biqTR8zcJLaS14DCOj6khk+4hXvY6U2PHLT5Y8UCLLLyRekdgZ3zpO9l7kGTMpzwQM=
X-Gm-Message-State: AOJu0Ywzl5FzBXcA3juCfJrjJKd+WR/ZgCQhhl1cUiZ3ub0FVRL8SuJ2
	u0xAukX9xoI8lYq+BOEDLnrr6q7n9ebzMsN00GlpkkOn9/nMvCW0zB1/P61dmw==
X-Google-Smtp-Source: AGHT+IG2fV8aLvsB7VYAQh101YPygnqihyfjDEkAvqylgVXyKtfgD5dNl0nuo+U5anxRapgaRMnO8g==
X-Received: by 2002:a17:906:e2d4:b0:a44:296a:5e4c with SMTP id gr20-20020a170906e2d400b00a44296a5e4cmr1592354ejb.26.1709214559103;
        Thu, 29 Feb 2024 05:49:19 -0800 (PST)
Message-ID: <d9b1a07c-d1b6-4bc1-a12f-396620ea7d24@suse.com>
Date: Thu, 29 Feb 2024 14:49:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 07/12] xen/spinlock: add explicit non-recursive locking
 functions
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, Lukasz Hawrylko <lukasz@hawrylko.pl>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 =?UTF-8?Q?Mateusz_M=C3=B3wka?= <mateusz.mowka@intel.com>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-8-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20231212094725.22184-8-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 12.12.2023 10:47, Juergen Gross wrote:
> In order to prepare a type-safe recursive spinlock structure, add
> explicitly non-recursive locking functions to be used for non-recursive
> locking of spinlocks, which are used recursively, too.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
preferably with ...

> --- a/xen/include/xen/spinlock.h
> +++ b/xen/include/xen/spinlock.h
> @@ -101,6 +101,8 @@ struct lock_profile_qhead {
>  };
>  
>  #define _LOCK_PROFILE(lockname) { .name = #lockname, .lock = &lockname, }
> +#define _RLOCK_PROFILE(lockname) { .name = #lockname, .rlock = &lockname,     \
> +    .is_rlock = 1, }

... "true" used here, ...

> @@ -133,13 +135,16 @@ struct lock_profile_qhead {
>              break;                                                            \
>          }                                                                     \
>          prof->name = #l;                                                      \
> -        prof->lock = &(s)->l;                                                 \
> +        prof->lockptr = &(s)->l;                                              \
> +        prof->is_rlock = isr;                                                 \
>          prof->next = (s)->profile_head.elem_q;                                \
>          (s)->profile_head.elem_q = prof;                                      \
>      } while( 0 )
>  
> -#define spin_lock_init_prof(s, l) __spin_lock_init_prof(s, l, spinlock_t)
> -#define rspin_lock_init_prof(s, l) __spin_lock_init_prof(s, l, rspinlock_t)
> +#define spin_lock_init_prof(s, l)                                             \
> +    __spin_lock_init_prof(s, l, lock, spinlock_t, 0)

... "false" here, ...

> +#define rspin_lock_init_prof(s, l)                                            \
> +    __spin_lock_init_prof(s, l, rlock, rspinlock_t, 1)

... "true" again here, and ...

> @@ -174,6 +179,7 @@ struct lock_profile_qhead { };
>  
>  #endif
>  
> +
>  typedef union {
>      uint32_t head_tail;
>      struct {

... definitely with this hunk dropped.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:49:30 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:49:30 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687129.1070066 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgmw-0004TJ-9b; Thu, 29 Feb 2024 13:49:30 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687129.1070066; Thu, 29 Feb 2024 13:49:30 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgmw-0004TC-6H; Thu, 29 Feb 2024 13:49:30 +0000
Received: by outflank-mailman (input) for mailman id 687129;
 Thu, 29 Feb 2024 13:49:28 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfgmu-0004SY-Md
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:49:28 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfgmu-0005RW-D9; Thu, 29 Feb 2024 13:49:28 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfgmu-0005vb-5G; Thu, 29 Feb 2024 13:49:28 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=b4vchcwnMvTdcQ1H7CWGgZ5zgDInDgGoaXp1bDf4v+4=; b=el/PSrQO3sXTfSy4NOkzaxOHG/
	6UFOcxJOtM7iWYq8HQD5SKsFOIgx5TOZ/DhdYp6oBmfCikE5wVxYqRZoqU9i2fzn8nSD9UljOWhMe
	xVoYPHlJn9pOUx5xXaR3yDo3bi+pP9DTuRF5Oh08sC7PIsXBYoYY8kRc8SiT6Mtgw3HM=;
Message-ID: <e44a8bcf-a037-441c-b85c-651784df020a@xen.org>
Date: Thu, 29 Feb 2024 13:49:26 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 03/23] xen/riscv: introduce nospec.h
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <16370492de5344ae8e1aab688261600491868f4e.1708962629.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <16370492de5344ae8e1aab688261600491868f4e.1708962629.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 26/02/2024 17:38, Oleksii Kurochko wrote:
>  From the unpriviliged doc:
>    No standard hints are presently defined.
>    We anticipate standard hints to eventually include memory-system spatial
>    and temporal locality hints, branch prediction hints, thread-scheduling
>    hints, security tags, and instrumentation flags for simulation/emulation.
> 
> Also, there are no speculation execution barriers.
> 
> Therefore, functions evaluate_nospec() and block_speculation() should
> remain empty until a specific platform has an extension to deal with
> speculation execution.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>   Changes in V5:
>     - new patch
> ---
>   xen/arch/riscv/include/asm/nospec.h | 25 +++++++++++++++++++++++++
>   1 file changed, 25 insertions(+)
>   create mode 100644 xen/arch/riscv/include/asm/nospec.h
> 
> diff --git a/xen/arch/riscv/include/asm/nospec.h b/xen/arch/riscv/include/asm/nospec.h
> new file mode 100644
> index 0000000000..4fb404a0a2
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/nospec.h
> @@ -0,0 +1,25 @@
> +/* SPDX-License-Identifier: GPL-2.0 */

New file should use the SPDX tag GPL-2.0-only. I guess this could be 
fixed on commit?

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:54:21 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:54:21 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687134.1070076 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgrZ-0007D7-Ri; Thu, 29 Feb 2024 13:54:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687134.1070076; Thu, 29 Feb 2024 13:54:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgrZ-0007D0-P9; Thu, 29 Feb 2024 13:54:17 +0000
Received: by outflank-mailman (input) for mailman id 687134;
 Thu, 29 Feb 2024 13:54:16 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfgrY-0007Cu-85
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:54:16 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfgrX-0005Y2-Gj; Thu, 29 Feb 2024 13:54:15 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfgrX-00063I-84; Thu, 29 Feb 2024 13:54:15 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=fHqAgmIJ16aenmat+zXd3xoW6+g/MT2fpdWqfvojaDE=; b=cxHel2HXzG68DIfZQbKO8r5WAa
	RDv5OmmtA27MenFzw0IpTFW7FER3NlkTxTn6hGM+1EbjTOYTHsXbOJ1C1YA7vyJx4XFG6RRb23kuK
	hmZBkT71qthyJGUMwF0dUazPGD08kAXUPNcl1oC3CtDIDUxi3KYsH3IYobl+qvIfSIc0=;
Message-ID: <e22ca712-b8cf-4bc9-87df-22cbd7819733@xen.org>
Date: Thu, 29 Feb 2024 13:54:13 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 04/23] xen/asm-generic: introduce generic fls() and
 flsl() functions
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <df7ab5055ef08fa595f913072302770a3f6a5c33.1708962629.git.oleksii.kurochko@gmail.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <df7ab5055ef08fa595f913072302770a3f6a5c33.1708962629.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Oleksii,

On 26/02/2024 17:38, Oleksii Kurochko wrote:
> These functions can be useful for architectures that don't
> have corresponding arch-specific instructions.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>   Changes in V5:
>     - new patch
> ---
>   xen/include/asm-generic/bitops/fls.h  | 18 ++++++++++++++++++
>   xen/include/asm-generic/bitops/flsl.h | 10 ++++++++++

One header per function seems a little bit excessive to me. Do you have 
any pointer where this request is coming from?

Why not using the pattern.

In arch implementation:

#define fls
static inline ...

In the generic header (asm-generic or xen/):

#ifndef fls
static inline ...
#endif

>   2 files changed, 28 insertions(+)
>   create mode 100644 xen/include/asm-generic/bitops/fls.h
>   create mode 100644 xen/include/asm-generic/bitops/flsl.h
> 
> diff --git a/xen/include/asm-generic/bitops/fls.h b/xen/include/asm-generic/bitops/fls.h
> new file mode 100644
> index 0000000000..369a4c790c
> --- /dev/null
> +++ b/xen/include/asm-generic/bitops/fls.h
> @@ -0,0 +1,18 @@
> +/* SPDX-License-Identifier: GPL-2.0 */

You should use GPL-2.0-only.

> +#ifndef _ASM_GENERIC_BITOPS_FLS_H_
> +#define _ASM_GENERIC_BITOPS_FLS_H_
> +
> +/**
> + * fls - find last (most-significant) bit set
> + * @x: the word to search
> + *
> + * This is defined the same way as ffs.
> + * Note fls(0) = 0, fls(1) = 1, fls(0x80000000) = 32.
> + */
> +
> +static inline int fls(unsigned int x)
> +{
> +    return generic_fls(x);
> +}
> +
> +#endif /* _ASM_GENERIC_BITOPS_FLS_H_ */

Missing emacs magic. I am probably not going to repeat this remark and 
the one above again. So please have a look.

> diff --git a/xen/include/asm-generic/bitops/flsl.h b/xen/include/asm-generic/bitops/flsl.h
> new file mode 100644
> index 0000000000..d0a2e9c729
> --- /dev/null
> +++ b/xen/include/asm-generic/bitops/flsl.h
> @@ -0,0 +1,10 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _ASM_GENERIC_BITOPS_FLSL_H_
> +#define _ASM_GENERIC_BITOPS_FLSL_H_
> +
> +static inline int flsl(unsigned long x)
> +{
> +    return generic_flsl(x);
> +}
> +
> +#endif /* _ASM_GENERIC_BITOPS_FLSL_H_ */

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:56:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:56:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687140.1070086 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgtq-0007og-Cg; Thu, 29 Feb 2024 13:56:38 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687140.1070086; Thu, 29 Feb 2024 13:56:38 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgtq-0007oZ-9D; Thu, 29 Feb 2024 13:56:38 +0000
Received: by outflank-mailman (input) for mailman id 687140;
 Thu, 29 Feb 2024 13:56:37 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rfgto-0007nf-T1
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:56:37 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5a1abfe3-d70a-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 14:56:35 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 163241F7E9;
 Thu, 29 Feb 2024 13:56:35 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8867713503;
 Thu, 29 Feb 2024 13:56:34 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 0Ki5HxKN4GWlGAAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 13:56:34 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a1abfe3-d70a-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709214995; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=NK+sPAoplNtwgn2K6tYI+HQfXfK23WDATXWG9Cmn1cI=;
	b=KAIRMexNlJhuE5wRYxaYQylkZDdRk5LuqCyFPAIB8sENP9+grHVUnptEysRahSSimTAk14
	hSrmvXTl0c689++DfmiMrIcbVmOMKyPCBZe8UhcxGIEx+nSlkFb76LpG9HnfR/yq/rd5vR
	FY+4dGqkuuJPRO5qh6ewRg7ny3qVIII=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709214995; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=NK+sPAoplNtwgn2K6tYI+HQfXfK23WDATXWG9Cmn1cI=;
	b=KAIRMexNlJhuE5wRYxaYQylkZDdRk5LuqCyFPAIB8sENP9+grHVUnptEysRahSSimTAk14
	hSrmvXTl0c689++DfmiMrIcbVmOMKyPCBZe8UhcxGIEx+nSlkFb76LpG9HnfR/yq/rd5vR
	FY+4dGqkuuJPRO5qh6ewRg7ny3qVIII=
Message-ID: <2605cc82-dec2-42af-acba-e7170c2929d9@suse.com>
Date: Thu, 29 Feb 2024 14:56:33 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 07/12] xen/spinlock: add explicit non-recursive locking
 functions
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, Lukasz Hawrylko <lukasz@hawrylko.pl>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 =?UTF-8?Q?Mateusz_M=C3=B3wka?= <mateusz.mowka@intel.com>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-8-jgross@suse.com>
 <d9b1a07c-d1b6-4bc1-a12f-396620ea7d24@suse.com>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <d9b1a07c-d1b6-4bc1-a12f-396620ea7d24@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------yEbwmL2dYTXqUfPBHj0P1TFZ"
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=KAIRMexN
X-Spamd-Result: default: False [-6.70 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 TO_DN_SOME(0.00)[];
	 HAS_ATTACHMENT(0.00)[];
	 MIME_BASE64_TEXT_BOGUS(1.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 MID_RHS_MATCH_FROM(0.00)[];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 RCVD_DKIM_ARC_DNSWL_HI(-1.00)[];
	 FROM_HAS_DN(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 DWL_DNSWL_LOW(-1.00)[suse.com:dkim];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 RCPT_COUNT_TWELVE(0.00)[15];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 RCVD_IN_DNSWL_HI(-0.50)[2a07:de40:b281:104:10:150:64:97:from];
	 RCVD_TLS_ALL(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 163241F7E9
X-Spam-Level: 
X-Spam-Score: -6.70
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------yEbwmL2dYTXqUfPBHj0P1TFZ
Content-Type: multipart/mixed; boundary="------------d2tNLeOOzGMQN1wlZkUxq066";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Julien Grall
 <julien@xen.org>, Bertrand Marquis <bertrand.marquis@arm.com>,
 Michal Orzel <michal.orzel@amd.com>,
 Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Tamas K Lengyel <tamas@tklengyel.com>, Lukasz Hawrylko <lukasz@hawrylko.pl>,
 "Daniel P. Smith" <dpsmith@apertussolutions.com>,
 =?UTF-8?Q?Mateusz_M=C3=B3wka?= <mateusz.mowka@intel.com>,
 xen-devel@lists.xenproject.org
Message-ID: <2605cc82-dec2-42af-acba-e7170c2929d9@suse.com>
Subject: Re: [PATCH v4 07/12] xen/spinlock: add explicit non-recursive locking
 functions
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-8-jgross@suse.com>
 <d9b1a07c-d1b6-4bc1-a12f-396620ea7d24@suse.com>
In-Reply-To: <d9b1a07c-d1b6-4bc1-a12f-396620ea7d24@suse.com>
Autocrypt-Gossip: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJ3BBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AAIQkQoDSui/t3IH4WIQQ+pJkfkcoLMCa4X6CgNK6L+3cgfgn7AJ9DmMd0SMJE
 ePbc7/m22D2v04iu7ACffXTdZQhNl557tJuDXZSBxDmW/tLOwU0EWTecRBAIAIK5OMKMU5R2
 Lk2bbjgX7vyQuCFFyKf9rC/4itNwhYWFSlKzVj3WJBDsoi2KvPm7AI+XB6NIkNAkshL5C0kd
 pcNd5Xo0jRR5/WE/bT7LyrJ0OJWS/qUit5eNNvsO+SxGAk28KRa1ieVLeZi9D03NL0+HIAtZ
 tecfqwgl3Y72UpLUyt+r7LQhcI/XR5IUUaD4C/chB4Vq2QkDKO7Q8+2HJOrFIjiVli4lU+Sf
 OBp64m//Y1xys++Z4ODoKh7tkh5DxiO3QBHG7bHK0CSQsJ6XUvPVYubAuy1XfSDzSeSBl//C
 v78Fclb+gi9GWidSTG/4hsEzd1fY5XwCZG/XJJY9M/sAAwUH/09Ar9W2U1Qm+DwZeP2ii3Ou
 14Z9VlVVPhcEmR/AFykL9dw/OV2O/7cdi52+l00reUu6Nd4Dl8s4f5n8b1YFzmkVVIyhwjvU
 jxtPyUgDOt6DRa+RaDlXZZmxQyWcMv2anAgYWGVszeB8Myzsw8y7xhBEVV1S+1KloCzw4V8Z
 DSJrcsZlyMDoiTb7FyqxwQnM0f6qHxWbmOOnbzJmBqpNpFuDcz/4xNsymJylm6oXiucHQBAP
 Xb/cE1YNHpuaH4SRhIxwQilCYEznWowQphNAbJtEKOmcocY7EbSt8VjXTzmYENkIfkrHRyXQ
 dUm5AoL51XZljkCqNwrADGkTvkwsWSvCSQQYEQIACQUCWTecRAIbDAAKCRCgNK6L+3cgfuef
 AJ9wlZQNQUp0KwEf8Tl37RmcxCL4bQCcC5alCSMzUBJ5DBIcR4BY+CyQFAs=

--------------d2tNLeOOzGMQN1wlZkUxq066
Content-Type: multipart/mixed; boundary="------------2negF6jxm6H0mufd0Ddf72UJ"

--------------2negF6jxm6H0mufd0Ddf72UJ
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMjkuMDIuMjQgMTQ6NDksIEphbiBCZXVsaWNoIHdyb3RlOg0KPiBPbiAxMi4xMi4yMDIz
IDEwOjQ3LCBKdWVyZ2VuIEdyb3NzIHdyb3RlOg0KPj4gSW4gb3JkZXIgdG8gcHJlcGFyZSBh
IHR5cGUtc2FmZSByZWN1cnNpdmUgc3BpbmxvY2sgc3RydWN0dXJlLCBhZGQNCj4+IGV4cGxp
Y2l0bHkgbm9uLXJlY3Vyc2l2ZSBsb2NraW5nIGZ1bmN0aW9ucyB0byBiZSB1c2VkIGZvciBu
b24tcmVjdXJzaXZlDQo+PiBsb2NraW5nIG9mIHNwaW5sb2Nrcywgd2hpY2ggYXJlIHVzZWQg
cmVjdXJzaXZlbHksIHRvby4NCj4+DQo+PiBTaWduZWQtb2ZmLWJ5OiBKdWVyZ2VuIEdyb3Nz
IDxqZ3Jvc3NAc3VzZS5jb20+DQo+IA0KPiBBY2tlZC1ieTogSmFuIEJldWxpY2ggPGpiZXVs
aWNoQHN1c2UuY29tPg0KPiBwcmVmZXJhYmx5IHdpdGggLi4uDQo+IA0KPj4gLS0tIGEveGVu
L2luY2x1ZGUveGVuL3NwaW5sb2NrLmgNCj4+ICsrKyBiL3hlbi9pbmNsdWRlL3hlbi9zcGlu
bG9jay5oDQo+PiBAQCAtMTAxLDYgKzEwMSw4IEBAIHN0cnVjdCBsb2NrX3Byb2ZpbGVfcWhl
YWQgew0KPj4gICB9Ow0KPj4gICANCj4+ICAgI2RlZmluZSBfTE9DS19QUk9GSUxFKGxvY2tu
YW1lKSB7IC5uYW1lID0gI2xvY2tuYW1lLCAubG9jayA9ICZsb2NrbmFtZSwgfQ0KPj4gKyNk
ZWZpbmUgX1JMT0NLX1BST0ZJTEUobG9ja25hbWUpIHsgLm5hbWUgPSAjbG9ja25hbWUsIC5y
bG9jayA9ICZsb2NrbmFtZSwgICAgIFwNCj4+ICsgICAgLmlzX3Jsb2NrID0gMSwgfQ0KPiAN
Cj4gLi4uICJ0cnVlIiB1c2VkIGhlcmUsIC4uLg0KPiANCj4+IEBAIC0xMzMsMTMgKzEzNSwx
NiBAQCBzdHJ1Y3QgbG9ja19wcm9maWxlX3FoZWFkIHsNCj4+ICAgICAgICAgICAgICAgYnJl
YWs7ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgXA0KPj4gICAgICAgICAgIH0gICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBcDQo+PiAgICAgICAg
ICAgcHJvZi0+bmFtZSA9ICNsOyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIFwNCj4+IC0gICAgICAgIHByb2YtPmxvY2sgPSAmKHMpLT5s
OyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBcDQo+
PiArICAgICAgICBwcm9mLT5sb2NrcHRyID0gJihzKS0+bDsgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgXA0KPj4gKyAgICAgICAgcHJvZi0+aXNfcmxv
Y2sgPSBpc3I7ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIFwNCj4+ICAgICAgICAgICBwcm9mLT5uZXh0ID0gKHMpLT5wcm9maWxlX2hlYWQuZWxl
bV9xOyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXA0KPj4gICAgICAgICAgIChz
KS0+cHJvZmlsZV9oZWFkLmVsZW1fcSA9IHByb2Y7ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICBcDQo+PiAgICAgICB9IHdoaWxlKCAwICkNCj4+ICAgDQo+PiAtI2Rl
ZmluZSBzcGluX2xvY2tfaW5pdF9wcm9mKHMsIGwpIF9fc3Bpbl9sb2NrX2luaXRfcHJvZihz
LCBsLCBzcGlubG9ja190KQ0KPj4gLSNkZWZpbmUgcnNwaW5fbG9ja19pbml0X3Byb2Yocywg
bCkgX19zcGluX2xvY2tfaW5pdF9wcm9mKHMsIGwsIHJzcGlubG9ja190KQ0KPj4gKyNkZWZp
bmUgc3Bpbl9sb2NrX2luaXRfcHJvZihzLCBsKSAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgIFwNCj4+ICsgICAgX19zcGluX2xvY2tfaW5pdF9wcm9mKHMs
IGwsIGxvY2ssIHNwaW5sb2NrX3QsIDApDQo+IA0KPiAuLi4gImZhbHNlIiBoZXJlLCAuLi4N
Cj4gDQo+PiArI2RlZmluZSByc3Bpbl9sb2NrX2luaXRfcHJvZihzLCBsKSAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXA0KPj4gKyAgICBfX3NwaW5fbG9j
a19pbml0X3Byb2YocywgbCwgcmxvY2ssIHJzcGlubG9ja190LCAxKQ0KPiANCj4gLi4uICJ0
cnVlIiBhZ2FpbiBoZXJlLCBhbmQgLi4uDQo+IA0KPj4gQEAgLTE3NCw2ICsxNzksNyBAQCBz
dHJ1Y3QgbG9ja19wcm9maWxlX3FoZWFkIHsgfTsNCj4+ICAgDQo+PiAgICNlbmRpZg0KPj4g
ICANCj4+ICsNCj4+ICAgdHlwZWRlZiB1bmlvbiB7DQo+PiAgICAgICB1aW50MzJfdCBoZWFk
X3RhaWw7DQo+PiAgICAgICBzdHJ1Y3Qgew0KPiANCj4gLi4uIGRlZmluaXRlbHkgd2l0aCB0
aGlzIGh1bmsgZHJvcHBlZC4NCg0KSSdtIGZpbmUgd2l0aCBhbGwgb2YgYWJvdmUuDQoNCg0K
SnVlcmdlbg0KDQo=
--------------2negF6jxm6H0mufd0Ddf72UJ
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------2negF6jxm6H0mufd0Ddf72UJ--

--------------d2tNLeOOzGMQN1wlZkUxq066--

--------------yEbwmL2dYTXqUfPBHj0P1TFZ
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXgjREFAwAAAAAACgkQsN6d1ii/Ey+O
pwf8CqziVk6M590Gsq9+9fptOswnyu7igDR/m635N8fty3skUNArPQLMQ54vqHG6604031ot1y7b
sNng+tzpflIs09O8EiakD5gXehoBJxbtX0LD/MRiONjAcpL6vs8L0eZRWMOd/tB0+aURg2ZYPG1D
qCGwqXBWJfuDGRmoHsvuAnVdXjpcpPus3pzQz5PBIysSk3I9JuBGCjZD01iaR8IlSy8MTgej8kHc
xMY5vE292EC+b74vRbLkYqVSEILhFrvDsJwC3fnA3lNLh4NnS/Zpr+WLf2VZ2weukfLvdk0MXpr2
FTkE+Ja9DA/sKC/aLIwtkaBiPTpleVCZpvXvR3lTtQ==
=wXqe
-----END PGP SIGNATURE-----

--------------yEbwmL2dYTXqUfPBHj0P1TFZ--


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 13:59:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 13:59:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687142.1070096 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgwF-0008N9-Q2; Thu, 29 Feb 2024 13:59:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687142.1070096; Thu, 29 Feb 2024 13:59:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgwF-0008N2-MR; Thu, 29 Feb 2024 13:59:07 +0000
Received: by outflank-mailman (input) for mailman id 687142;
 Thu, 29 Feb 2024 13:59:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfgwE-0008Mw-Nc
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 13:59:06 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id b2e3ddab-d70a-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 14:59:04 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a441d7c6125so122467766b.2
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 05:59:04 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 he44-20020a1709073dac00b00a4323d1b18fsm718395ejc.34.2024.02.29.05.59.03
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 05:59:03 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b2e3ddab-d70a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709215144; x=1709819944; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=03vZQBuOcZqMdZomF1eNdi4uFh8LQI+u7elTQbC+Uf8=;
        b=EZTtIH3mhvApCCq7cTa/oed5ztqrkBRsoEHwDwFxrzcbP+KXmCrh/CahUEW21UE6+M
         3f5ENJMnn5xzJ+97E+hoitrUFyTJQt6WERqifJCAl7FL7pTMn8N+q3redLV8Cv5vAwNY
         uExOMZtpJuZF0D6sxv6x2bHXJGKb8TdVJpdvYHsXm+nHBg02s8lptAmKMYJv5oAo4S6N
         Lo1+GqFALjfiyhlNmgJrvxcRF/i3zdqHl7nsoIOzO0+lFgYUSb6PjDBx4Ymt8NWcdQ4A
         xdYfq6zFMxxfblssTC9FkL3EBHiBb/VkJpWWYcPKVE1s23c16Sawt6IpTf9RPSQhKX82
         qxog==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709215144; x=1709819944;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=03vZQBuOcZqMdZomF1eNdi4uFh8LQI+u7elTQbC+Uf8=;
        b=rHu0TOwbMDnm9hDJBfuA9WE5MAy5lfqx1ARrJOOGE7bHpyBhq7+GR30ltHP3BPuzQg
         ZW/4FATzrihhxW0kPc4nW4ul9W0kBlaPiWu+QKgEi0u7h85annLSWZ/knMAGd/YupaGW
         rXewvgC3YTcpJqAeEU10pVVLQjRHzzNF7UcOHx4cG68yx9fZoEGD3UaQSmimD2lGmER7
         K7Ko1Z9K/IWtjRHSqVMZo6QsVu7NS5IaIkPpFnoASCCSoKZPWqRx+2Xy8V+7/8qGN6FJ
         3pLj1M4DjEZvYUZUDyohciXqUOZhsZ6e1o7rIKl/4zs/gdX6CQTDXbno7EnYTJJoU7MG
         fqxA==
X-Forwarded-Encrypted: i=1; AJvYcCXUPKjT43LQug+zsVjqjl8BwDw3VX1DNlVdPCnoXO5nLxBbJQXpIoAGnF2F4FrGqgAcTcf7PL8JEqPuLzki8aN7FfgcdRi2P4hsLc5NgK4=
X-Gm-Message-State: AOJu0YxdLKz/9NkZrVnjxdsm0oHonKaM+Lc5HUvdILay8dOphs8TogGb
	rA7xNPtttPh7UUETTkvqhxbAIdUbiXrIsg3Tttgq8DqsMNfoZsogVQ91zXqSPg==
X-Google-Smtp-Source: AGHT+IFC5RIldXD6RnWcf/o1/0EykYtVyJcx4Yr6xTB9ZHBjHcUiWyilNULT8KS60NSmVoO4UuroxQ==
X-Received: by 2002:a17:906:a291:b0:a3e:8972:4404 with SMTP id i17-20020a170906a29100b00a3e89724404mr1517953ejz.9.1709215144177;
        Thu, 29 Feb 2024 05:59:04 -0800 (PST)
Message-ID: <9eab8dd5-266a-432e-805b-d2479f7e1579@suse.com>
Date: Thu, 29 Feb 2024 14:59:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 08/12] xen/spinlock: add another function level
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-9-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20231212094725.22184-9-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 12.12.2023 10:47, Juergen Gross wrote:
> @@ -377,25 +388,25 @@ void _spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags)
>      local_irq_restore(flags);
>  }
>  
> +static int always_inline spin_is_locked_common(const spinlock_tickets_t *t)
> +{
> +    return t->head != t->tail;
> +}
> +
>  int _spin_is_locked(const spinlock_t *lock)
>  {
> -    /*
> -     * Recursive locks may be locked by another CPU, yet we return
> -     * "false" here, making this function suitable only for use in
> -     * ASSERT()s and alike.
> -     */
> -    return lock->recurse_cpu == SPINLOCK_NO_CPU
> -           ? lock->tickets.head != lock->tickets.tail
> -           : lock->recurse_cpu == smp_processor_id();
> +    return spin_is_locked_common(&lock->tickets);
>  }

This looks like a functional change. I haven't spotted an adjustment in an
earlier patch that would make the lost case unnecessary, but even if there
was one, the removal thereof would then also want doing there, I think.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:01:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:01:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687145.1070105 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgyR-0001ps-4T; Thu, 29 Feb 2024 14:01:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687145.1070105; Thu, 29 Feb 2024 14:01:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfgyR-0001pl-1o; Thu, 29 Feb 2024 14:01:23 +0000
Received: by outflank-mailman (input) for mailman id 687145;
 Thu, 29 Feb 2024 14:01:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfgyQ-0001pf-Cx
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:01:22 +0000
Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com
 [2a00:1450:4864:20::629])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 0453e300-d70b-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 15:01:21 +0100 (CET)
Received: by mail-ej1-x629.google.com with SMTP id
 a640c23a62f3a-a2f22bfb4e6so174145466b.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 06:01:21 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 vi18-20020a170907d41200b00a4455153b45sm391373ejc.141.2024.02.29.06.01.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 06:01:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 0453e300-d70b-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709215281; x=1709820081; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gbJFnp9Dr+ZWHKrFGCv+4WB3le3vsgd13uQiLdxmE2s=;
        b=KbbTjTXD5O8+m5arlloKPrvJN1Zb0n2cah/m60iskRjjD6typ1GxQ1oepB8SKRw8hW
         1UlKZD/egFh+HsEvYG2BeOhX12VJ6sRZ17DpxYRxSNclYxAbxefoVKOljq68bX2aKkuF
         036aDtOFUWlCP9yrVJEkLDlcPg6p68B4+naQHkYKKBgd3ii+GeA2pkY/DJPsXmHQ0lab
         NlxH4BL9POwPhUM32Z92GQn4qa2gaSxvGA61VQwbdRE/SY1dMpdQga1hS+9ypLzK6QAg
         Q4f/LrE/b0fgM0zhavvUwU0/N0kGrdaD5ILQach4tg8OiJwoAdtIPyTkKmJO1cNjZxUx
         wAzg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709215281; x=1709820081;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gbJFnp9Dr+ZWHKrFGCv+4WB3le3vsgd13uQiLdxmE2s=;
        b=JcxFVZ7DKDT8sRPHFbZPYiavlXnIthiHO3pN/ViKiVmqSRua/QRYmruYBfyXFk4nl3
         PIzOWk4GeSoFnuxR4XFxEwE98voOSq+Ekvm770zI1rC9i1z0+leJ43jnM75NCmmxhOOE
         rezej35dd8HuNzv8j/p2CrsmGPiYwZqjOjfnLWNrB1Jwv7P1ksE6u967mMSQyCJ9TpMT
         e6U6odvDfitJGbzf5ZUkHXsvwOv7V86fODU7y03VgIBx/KtUxL9Qxx5/fbND4xZTfZVN
         iQzAigJxNVAaYHumyV78Wo8bSdojJNUOAHQTxct9XS9NN215cKRK4fyzV8Gjl7Pk3qzU
         FYWQ==
X-Forwarded-Encrypted: i=1; AJvYcCU8eEFuJ8RFV4D6I4ZpPLwsRZdQ7+U9Y6VzJBpH2GcbVPHdfQHbM6nIgE1msNUB9+ItR31QwBXhfZacB1DW8XThQ7xSODm3vUDKaw8r93U=
X-Gm-Message-State: AOJu0YzHy4fhMKCdAyhR0ohHU+ZkFbFX5Q7uQaTbfdZVRpzW/nrLbXjC
	O20EHuk82DKWDOky5JgCISBErYvrpvC/Vd17nBqYV5rp86IxIyyXY9KZzgIv3A==
X-Google-Smtp-Source: AGHT+IFoaPFmDKlblRlg9RHUq6IfIuSw8U3z+9ri8Zp057gN6Q+5BHfF48u7+mq91QHxCStAcVuHjg==
X-Received: by 2002:a17:906:b349:b0:a43:2255:2241 with SMTP id cd9-20020a170906b34900b00a4322552241mr1351453ejb.53.1709215280938;
        Thu, 29 Feb 2024 06:01:20 -0800 (PST)
Message-ID: <7d9a7a89-2ad1-49e3-9cac-7cf8d6b5e6c5@suse.com>
Date: Thu, 29 Feb 2024 15:01:19 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 03/23] xen/riscv: introduce nospec.h
Content-Language: en-US
To: Julien Grall <julien@xen.org>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <16370492de5344ae8e1aab688261600491868f4e.1708962629.git.oleksii.kurochko@gmail.com>
 <e44a8bcf-a037-441c-b85c-651784df020a@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e44a8bcf-a037-441c-b85c-651784df020a@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 14:49, Julien Grall wrote:
> On 26/02/2024 17:38, Oleksii Kurochko wrote:
>> --- /dev/null
>> +++ b/xen/arch/riscv/include/asm/nospec.h
>> @@ -0,0 +1,25 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
> 
> New file should use the SPDX tag GPL-2.0-only. I guess this could be 
> fixed on commit?

I wouldn't mind doing so.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:03:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:03:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687148.1070116 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfh0f-0002fg-KS; Thu, 29 Feb 2024 14:03:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687148.1070116; Thu, 29 Feb 2024 14:03:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfh0f-0002fZ-Hq; Thu, 29 Feb 2024 14:03:41 +0000
Received: by outflank-mailman (input) for mailman id 687148;
 Thu, 29 Feb 2024 14:03:40 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kRMo=KG=amd.com=Stewart.Hildebrand@srs-se1.protection.inumbo.net>)
 id 1rfh0e-0002eU-7c
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:03:40 +0000
Received: from NAM11-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam11on20600.outbound.protection.outlook.com
 [2a01:111:f403:2415::600])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 55c2817e-d70b-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 15:03:39 +0100 (CET)
Received: from CH2PR03CA0022.namprd03.prod.outlook.com (2603:10b6:610:59::32)
 by MN2PR12MB4319.namprd12.prod.outlook.com (2603:10b6:208:1dc::10)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.39; Thu, 29 Feb
 2024 14:03:35 +0000
Received: from DS3PEPF000099DF.namprd04.prod.outlook.com
 (2603:10b6:610:59:cafe::6a) by CH2PR03CA0022.outlook.office365.com
 (2603:10b6:610:59::32) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.28 via Frontend
 Transport; Thu, 29 Feb 2024 14:03:35 +0000
Received: from SATLEXMB03.amd.com (165.204.84.17) by
 DS3PEPF000099DF.mail.protection.outlook.com (10.167.17.202) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.7292.25 via Frontend Transport; Thu, 29 Feb 2024 14:03:35 +0000
Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 29 Feb
 2024 08:03:34 -0600
Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com
 (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 29 Feb
 2024 06:03:34 -0800
Received: from [172.30.63.250] (10.180.168.240) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend
 Transport; Thu, 29 Feb 2024 08:03:34 -0600
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 55c2817e-d70b-11ee-afd8-a90da7624cb6
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=FRmk7iOfuR2wDiQuGeGvxtgSHJ4JxVRhW47EL6bgvciIDwXC4Hxy7Wip9Q+GJu83mwiD32VkuYetesYs04Xkdy9pDh28GYkXK2Q/B0j6XUZ0pwXnoyI0PYFGAA1JytmzGOXty6g5uxAvCZxcJOdW9ibnKrSy1qGFsOG6n/VVsgqdBZCTze1x/wTdUex8YnlrDuMAOXxKXA+WI+61LIn/c70xcpGo/8McPIlbCNfeNHNCCnhMQEShZ5He9Nfi+DKYUtPmnzqNzSlI6jb8VLbja8LjOFwPhUWL/bnF+hR3syXPt1aqMMkkr2UvKurkqERneDYQuXEGb1VkGCYZ8GH+Fg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=/4J5McUmXEV6SeAVg8/yzDPAv8aLMQpiOom1ClSKdag=;
 b=YxMY7tSbl7bmedebburTNXmjjKPnl6Wmsr6Zx5oI8mUL8ia5WZjpdvHGF3JiOCPtrB/Inh3mW7V49oszazLOHXEpdwt6VkvAzF8ocszEhpbTOpkq/cWKGeLdHQ0QGBoolO4d/xwoh0+sXz2S5+E27mcrue4hFLdXcI+DH/4o6iIqewCzGkDFDakG8sOYPv0z9cSPJMFHP37f0f5RmwDN0Xu1VIulzm7vUFn3T8U2gEEbPjezxHjbAgizSmLMDxjr/+4NiJZ4FzzqfmK4IVPJUg45CbBeaGPo5cM0bSYQyBRLHCO8Ptd/7CfE8BmY3XoAyq8DcOOqALi3XGl6A6i+LA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=suse.com smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=/4J5McUmXEV6SeAVg8/yzDPAv8aLMQpiOom1ClSKdag=;
 b=4qVAKkvUXf4hNTZYfutMC+iWRXL2BlcPR/1brC3UWdJY7oq7UK0XPG177UySpLf8VRlt9+iviLqC+Iow7XcWl0TU4r28He2eAdsWRITCKLtZLL3DKd0I0J394ddPBLz+0YuSBQ3YnisM9H3JUWoPe1NuFUF0GNJFebE1Ky6icak=
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C
Message-ID: <80a5de29-0215-4100-b5a3-85ed4639f00e@amd.com>
Date: Thu, 29 Feb 2024 09:03:33 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] pci: fix locking around vPCI removal in
 pci_remove_device()
To: Jan Beulich <jbeulich@suse.com>, Roger Pau Monne <roger.pau@citrix.com>
CC: Paul Durrant <paul@xen.org>, <xen-devel@lists.xenproject.org>
References: <20240229131525.19099-1-roger.pau@citrix.com>
 <29bebf83-a78b-440f-a642-0ec9aa40c0d7@suse.com>
Content-Language: en-US
From: Stewart Hildebrand <stewart.hildebrand@amd.com>
In-Reply-To: <29bebf83-a78b-440f-a642-0ec9aa40c0d7@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DS3PEPF000099DF:EE_|MN2PR12MB4319:EE_
X-MS-Office365-Filtering-Correlation-Id: 24d8f64c-6ece-44d7-bc7b-08dc392f38a2
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	kH9EJXhdtRmEnfIbvmQm0aYq53BpCv+0MAWyU+6tNL+PzdX36YHaVsIBCIq0vIkyr8tvwv3jKFZQBjYY4mW1ijXLqKwffbg9Nlswh4snf7pzhA9rktozhryFEhVQzh/Zb0H24/4cYaeniJxSnB43wHnmvgctMoMpFnhbyUMBa61ROOddN9I1P7i+DxI8FFhnmz35C06uw4E+gntc5U/TiUdpoLY8ktMF2ceFFdN41d2QdqsZ0UMdaf4AGY0eQiT84Vx3vtngGLNM3KqkMobCxQcl/R+MYtshcDJucZuah2v3g7f9cng5f9RVhlvetM/+DbNwzYdIio0KHeJX1KSCmAgKbwtad32crbcNC9yAjknP7crNHIBl1rZhgrPOxEfh69XXOx0vj1sx57vE5AmEA6an9LVMosxhKrBlpKpBoPe9rL28uz+Ca62BKT1MLCit5rKbF8Tk3h0Io1ltVRonzVR1xHJ9viZL9TvOQPuaOXSYnWHuCjk4RjKHcAhVAesIggSK2A83pCWBScsBKH1kX4zfbK2rFiL7+BDu4hI9tFjeoFAza/+UxuXgmKlM1pULUMsSDGs1EvIZRBpqxvxEi4uSW2afYOBITZsHsIzp7xmigAnfIqCKVlJmHHaNbJCa9zSxuDT9VCfdXik5C7nx0Wh48Oxb9txWDlwvqEEGDmFKFA2yiNdDZz3ZNswcrEGPUmFa3+RZIvsCnHnlhjZBG4ZahyNM1HA36Xcj/g9Td6mIYk1TS1nxw7KhhrT+vH0z
X-Forefront-Antispam-Report:
	CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400014)(36860700004);DIR:OUT;SFP:1101;
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Feb 2024 14:03:35.5421
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 24d8f64c-6ece-44d7-bc7b-08dc392f38a2
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DS3PEPF000099DF.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4319

On 2/29/24 08:23, Jan Beulich wrote:
> On 29.02.2024 14:15, Roger Pau Monne wrote:
>> Currently vpci_deassign_device() is called without holding the per-domain
>> pci_lock in pci_remove_device(), which leads to:
>>
>> Assertion 'rw_is_write_locked(&pdev->domain->pci_lock)' failed at ../drivers/vpci/vpci.c:47
>> [...]
>> Xen call trace:
>>    [<ffff82d040260eac>] R vpci_deassign_device+0x10d/0x1b9
>>    [<ffff82d04027932f>] S pci_remove_device+0x2b1/0x380
>>    [<ffff82d040260bd0>] F pci_physdev_op+0x197/0x19e
>>    [<ffff82d04032272d>] F do_physdev_op+0x342/0x12aa
>>    [<ffff82d0402f067a>] F pv_hypercall+0x58e/0x62b
>>    [<ffff82d0402012ba>] F lstar_enter+0x13a/0x140
>>
>> Move the existing block that removes the device from the domain pdev_list ahead
>> and also issue the call to vpci_deassign_device() there.  It's fine to remove
>> the device from the domain list of assigned devices, as further functions only
>> care that the pdev domain field is correctly set to the owner of the device
>> about to be removed.
>>
>> Moving the vpci_deassign_device() past the pci_cleanup_msi() call can be
>> dangerous, as doing the MSI cleanup ahead of having removed the vPCI handlers
>> could lead to stale data in vPCI MSI(-X) internal structures.
>>
>> Fixes: 4f78438b45e2 ('vpci: use per-domain PCI lock to protect vpci structure')
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>

Thanks


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:03:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:03:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687149.1070126 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfh0w-0002zv-UD; Thu, 29 Feb 2024 14:03:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687149.1070126; Thu, 29 Feb 2024 14:03:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfh0w-0002zo-Qs; Thu, 29 Feb 2024 14:03:58 +0000
Received: by outflank-mailman (input) for mailman id 687149;
 Thu, 29 Feb 2024 14:03:57 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfh0v-0002eU-93
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:03:57 +0000
Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com
 [2a00:1450:4864:20::530])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 60ce10d3-d70b-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 15:03:56 +0100 (CET)
Received: by mail-ed1-x530.google.com with SMTP id
 4fb4d7f45d1cf-565c6cf4819so3460039a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 06:03:56 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 l26-20020a1709065a9a00b00a3db46018a4sm723780ejq.71.2024.02.29.06.03.55
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 06:03:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 60ce10d3-d70b-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709215436; x=1709820236; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=bYLy3yIzkFdLXiS8CK5ZH/TzODR3Ws3E9a/i5G7l7xA=;
        b=DXKxyP9kfswRJnh5xj4idIbch1ntgFA7+cxgFEDdbPJK1VhlH4Hsefb8BlpvFyoCqU
         HGNXE3oVbTW47SzRct/4WNXdI+kvZt4uOj8s6GYEXKd8ZE87MR1yxPi70RtNi2Zpw6nx
         KAP+TJaYql7K8t3sx5W8VKbCyh/clwHLMuFoeNoi71QHmnYFOvZ8MAvPn0c+S+BUm2Oy
         AZQ3cE13AndmcdJM701MfoTUPk7u7nYXqkbYjenD0iRsd1wMKWC7fIPadcU0UAnCj9cY
         KHwQdb9ql1GIn2MoJowXiGw+TK4FnlqYUt54j3ANM+cgpf2LgkGhl9txB7fhXV+n79At
         3vFQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709215436; x=1709820236;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=bYLy3yIzkFdLXiS8CK5ZH/TzODR3Ws3E9a/i5G7l7xA=;
        b=IF5ORAcvfM20X5RwOGR6au9H5pYmxTODcfI/zDQhVSKnNJho9nH+TXpVrZIC8b6k9A
         s0cy0UeYK8ReyRHsBFvewS03Q5Y4KII4zgoekSoY8uESSao9/1A8X5pEmhWQ/6x4q8X6
         WNni2MckoKl/7L+szNdMHlobS7DGhSKdKCXc0BND6hVge7GnOAau0nkC4sDFJ9Xqx/7G
         Fe00pgmtQICCG9Rj3G+6UCmSd140wzUKp++sDR6CG1d4b423t1W2WwekyeEvgKQzko1I
         9IKwUM/gPgD2JILR19pEMHVnEWHSoiu+UYlnuzZYdpPZvpaGfTJC1T3xTwTlu+lE5GjW
         XhMQ==
X-Forwarded-Encrypted: i=1; AJvYcCUk6ykjZq3DuEd4Hreou9YFLA3tkvhCDQ6epeevgLYW5PyoRub2Z9n4GUXlw+duiRYa9OYJOnRTBlvkymGIbYGzDueWU1aLoO2+14PGjFU=
X-Gm-Message-State: AOJu0YxL8obXH87E6eUnqXd4xW6RJVWNJ6ne9eaT4Rlr3Ukc/orTmMWc
	ca9MoHIAAbgEDeXAM8y6ZHM5g9f+IwCwuR5z+PljPK/VPp0WD7jauWkeT67Qcw==
X-Google-Smtp-Source: AGHT+IFDeViwNblwwnIC1ntEIbotxN431pB7zBuYPAXQxxZZ2fxwS1Al0fKSa2fxze3chI0xFgu72A==
X-Received: by 2002:a17:907:97d4:b0:a3f:d2a8:a5c4 with SMTP id js20-20020a17090797d400b00a3fd2a8a5c4mr1593847ejc.35.1709215436171;
        Thu, 29 Feb 2024 06:03:56 -0800 (PST)
Message-ID: <654620c6-731a-4a91-8f1c-72961191dff1@suse.com>
Date: Thu, 29 Feb 2024 15:03:54 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 04/23] xen/asm-generic: introduce generic fls() and
 flsl() functions
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <df7ab5055ef08fa595f913072302770a3f6a5c33.1708962629.git.oleksii.kurochko@gmail.com>
 <e22ca712-b8cf-4bc9-87df-22cbd7819733@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <e22ca712-b8cf-4bc9-87df-22cbd7819733@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 14:54, Julien Grall wrote:
> On 26/02/2024 17:38, Oleksii Kurochko wrote:
>> These functions can be useful for architectures that don't
>> have corresponding arch-specific instructions.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> ---
>>   Changes in V5:
>>     - new patch
>> ---
>>   xen/include/asm-generic/bitops/fls.h  | 18 ++++++++++++++++++
>>   xen/include/asm-generic/bitops/flsl.h | 10 ++++++++++
> 
> One header per function seems a little bit excessive to me. Do you have 
> any pointer where this request is coming from?

That's in an attempt to follow Linux'es way of having this, aiui. This way
an arch can mix and match header inclusions and private definitions without
any #ifdef-ary.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:07:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:07:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687156.1070135 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfh4L-00042a-DY; Thu, 29 Feb 2024 14:07:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687156.1070135; Thu, 29 Feb 2024 14:07:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfh4L-00042T-Al; Thu, 29 Feb 2024 14:07:29 +0000
Received: by outflank-mailman (input) for mailman id 687156;
 Thu, 29 Feb 2024 14:07:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfh4L-00042L-39
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:07:29 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id dea89ef1-d70b-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 15:07:27 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a34c5ca2537so161120266b.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 06:07:27 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 vb7-20020a170907d04700b00a44446be9b1sm646937ejc.221.2024.02.29.06.07.26
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 06:07:26 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: dea89ef1-d70b-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709215647; x=1709820447; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=gKR0/WJL3fRI7ZkzCQNJoGNwMEdBYJr9FzJpQyaDC8A=;
        b=IYYevIsjdff33b96aXdZK2yDrI5lbHg6igtRcW0zavUnKsYOby8Anb4b0Glk1XhyUv
         2WUCemFbkWEtuRKUVSC4cub2cCQDKkNZe85aJGv9/d6fP8l/7/XLGISLVMt6PKLIIxXf
         ah8Gsm0UzxWafeQgBGBtzJ/FII5EFYIaK2doTxtFkmVGOwIAIGd3QX2PsCh+/Kysfbm1
         cMw701xupn8e20fBg1yKRSOAbWCPjOKqHf6CMFNc4rskUTwAd2epr6QHfJeX7yHJIK2W
         q7+JQzV9WCMTrdmmuPOPpH3BZRXhsVoGT6DDRS0lzZWa+JjZIraIPOlWUHldyqcRhUYj
         XYeQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709215647; x=1709820447;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=gKR0/WJL3fRI7ZkzCQNJoGNwMEdBYJr9FzJpQyaDC8A=;
        b=t8KU8bSlOOdMUfGeBAY85zjeP7AWvKgzind6/HiNAbcbWQEmEzVNJpWMwsXeMBEk2N
         KEQ+l3spfDoinnoF5upF19CYIIoGaqY2wVsqJTLxErlunzTCz7TDdqOOzbpCyFY2dGLA
         PYEt2nm+4jqL4fnnQJp2pjpPsxIo77iOp+M7LKkU3ft6VU9ygEYbOp9CMP3mUhkYZKKD
         WZ6lWd2z4Wz9nhqobQMkgoptZnzVGrgkERmfvZmCIyIaGOrlighH6Ul6rb2/CziMYy9U
         kSvHGidsSnYvgeEX4S1Zf+b+UXA3pGQfPgveBZlbykmJpMpkfghe3rU5pPqz/MehN+tS
         gZcQ==
X-Forwarded-Encrypted: i=1; AJvYcCV9h2a+1Bthkv6p+lLgF943YBmx9AY9fHXEG1fpmTrJX+lpjwBnOF5OMCC/2iNTvdJ0Mv7aK8t3n2PyVs3DQx/kkPu0T3PzrgLymNIj3+w=
X-Gm-Message-State: AOJu0Yz1yhTfvwMEJbVA76IZfQSZqi2RE2Evn0Kn1DA5SwV9j4EYLOp4
	dQO7UKVeg6yuB2tKeeCJ9aGYvvRI7ubrPrtNh64UIJx4GBhgp214dnTCml7Awg==
X-Google-Smtp-Source: AGHT+IHbziOv4HWhIKyjfCxxOQBRAmow1lQXYAvgvqGy/Ud01/QUGPbwtOtNGx9GreMBhELVv+MM/Q==
X-Received: by 2002:a17:907:20ae:b0:a3f:ce8:1234 with SMTP id pw14-20020a17090720ae00b00a3f0ce81234mr1298144ejb.68.1709215647219;
        Thu, 29 Feb 2024 06:07:27 -0800 (PST)
Message-ID: <f28e22b7-2657-40d9-8a97-e747c689c2b0@suse.com>
Date: Thu, 29 Feb 2024 15:07:25 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-US
To: Julien Grall <julien@xen.org>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko
 <oleksii.kurochko@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
 <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
 <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com>
 <2a112321-f4fd-4ab4-a431-a0bb4cb6eaa2@suse.com>
 <9b212dec-59ef-4cbb-bea7-add3867e4599@xen.org>
 <aa3fbfca-4675-4f92-9037-a4a7c5eca4dd@suse.com>
 <8eed0200-eabf-4024-b018-74be45ef95db@xen.org>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <8eed0200-eabf-4024-b018-74be45ef95db@xen.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 14:44, Julien Grall wrote:
> Hi Jan,
> 
> On 29/02/2024 12:51, Jan Beulich wrote:
>> On 29.02.2024 13:32, Julien Grall wrote:
>>> On 29/02/2024 12:17, Jan Beulich wrote:
>>>> On 29.02.2024 13:05, Andrew Cooper wrote:
>>>>> On 29/02/2024 10:23 am, Julien Grall wrote:
>>>>>>>>> IOW it is hard for me to see why RISC-V needs stronger restrictions
>>>>>>>>> here
>>>>>>>>> than other architectures. It ought to be possible to determine a
>>>>>>>>> baseline
>>>>>>>>> version. Even if taking the desire to have "pause" available as a
>>>>>>>>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
>>>>>>>>
>>>>>>>> I think we want to bump it on Arm. There are zero reasons to try to
>>>>>>>> keep
>>>>>>>> a lower versions if nobody tests/use it in production.
>>>>>>>>
>>>>>>>> I would suggest to do the same on x86. What's the point of try to
>>>>>>>> support Xen with a 15+ years old compiler?
>>>>>>>
>>>>>>> It could have long been bumped if only a proper scheme to follow for
>>>>>>> this and future bumping would have been put forward by anyone keen on
>>>>>>> such bumping, like - see his reply - e.g. Andrew. You may recall that
>>>>>>> this was discussed more than once on meetings, with no real outcome.
>>>>>>> I'm personally not meaning to stand in the way of such bumping as long
>>>>>>> as it's done in a predictable manner, but I'm not keen on doing so and
>>>>>>> hence I don't view it as my obligation to try to invent a reasonable
>>>>>>> scheme. (My personal view is that basic functionality should be
>>>>>>> possible to have virtually everywhere, whereas for advanced stuff it
>>>>>>> is fine to require a more modern tool chain.)
>>>>>>
>>>>>> That's one way to see it. The problem with this statement is a user
>>>>>> today is mislead to think you can build Xen with any GCC versions
>>>>>> since 4.1. I don't believe we can guarantee that and we are exposing
>>>>>> our users to unnecessary risk.
>>>>>>
>>>>>> In addition to that, I agree with Andrew. This is preventing us to
>>>>>> improve our code base and we have to carry hacks for older compilers.
>>>>>
>>>>> I don't think anyone here is suggesting that we switch to a
>>>>> bleeding-edge-only policy.  But 15y of support is extreme in the
>>>>> opposite direction.
>>>>>
>>>>> Xen ought to be buildable in the contemporary distros of the day, and I
>>>>> don't think anyone is going to credibly argue otherwise.
>>>>>
>>>>> But, it's also fine for new things to have newer requirements.
>>>>>
>>>>> Take CET for example.  I know we have disagreements on exactly how it's
>>>>> toolchain-conditionalness is implemented, but the basic principle of "If
>>>>> you want shiny new optional feature $X, you need newer toolchain $Y" is
>>>>> entirely fine.
>>>>>
>>>>> A brand new architecture is exactly the same.  Saying "this is the
>>>>> minimum, because it's what we test" doesn't preclude someone coming
>>>>> along and saying "can we use $N-1 ?  See here it works, and here's a
>>>>> change to CI test it".
>>>>>
>>>>>
>>>>> Anyway, its clear we need to write some policy on this, before making
>>>>> specific adjustments.  To get started, is there going to be any
>>>>> objection whatsoever on some principles which begin as follows:
>>>>
>>>> Largely not, but one aspect needs clarifying up front:
>>>>
>>>>> * For established architectures, we expect Xen to be buildable on the
>>>>> common contemporary distros.  (i.e. minima is not newer than what's
>>>>> available in contemporary distros, without a good reason)
>>>>
>>>> What counts as contemporary distro? Still in normal support? LTS? Yet
>>>> more extreme forms?
>>>
>>> LTS makes sense. More I am not sure. I am under the impression that
>>> people using older distros are those that wants a stable system. So they
>>> would unlikely try to upgrade the hypervisor.
>>>
>>> Even for LTS, I would argue that if it has been released 5 years ago,
>>> then you probably want to update it at the same time as moving to a
>>> newer Xen version.
>>
>> For the purposes of distros I agree. For the purposes of individuals
>> I don't: What's wrong with running a newer hypervisor and/or kernel
>> underneath an older distro?
> 
> There is nothing wrong. I just don't understand the benefits for us to 
> support that use case. To me there are two sorts of individuals:
>   1. The ones that are using distro packages. They will unlikely want to 
> switch to a newer hypervisor
>   2. The ones that are happy to compile and hack their system. Fairly 
> likely they will use a more distros and/or would not be put up by 
> upgrading it.
> 
> What individuals do you have in mind?

People like me.

> Also, for me, the minimum doesn't prevent anyone to try to compile with 
> an older compiler. It is only here to say that as a community we will 
> not investigate or trying to workaround bugs in those compilers.

Besides this also allowing to use functionality you won't have an easy
way of replacing, what you say also doesn't make clear whether - for
cases where the issue can be (reasonably easily) worked around - patches
would be accepted, or rejected on the basis of only helping a below-the-
line compiler.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:08:48 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:08:48 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687159.1070146 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfh5X-0004ch-Na; Thu, 29 Feb 2024 14:08:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687159.1070146; Thu, 29 Feb 2024 14:08:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfh5X-0004ca-Ke; Thu, 29 Feb 2024 14:08:43 +0000
Received: by outflank-mailman (input) for mailman id 687159;
 Thu, 29 Feb 2024 14:08:43 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfh5X-0004cU-57
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:08:43 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfh5V-0005vn-3H; Thu, 29 Feb 2024 14:08:41 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfh5U-00070p-Tj; Thu, 29 Feb 2024 14:08:41 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=ZhEVV07QhjmW4WPBcBjiExgQDTWwxdqTGeVErWPEgWw=; b=niGBEgUc7WGTscn4OC4DnaF0H3
	0PZeqX5NLFQyMJ1ueqrRzm3jaf13I/aT6+EdrLIuTHQowux3/9EIgkI8uLVmaOX1kU4UIPPuuZtmO
	D2wx4FiAIAKK+BHJZ+ddrgeGQjI+C0lKjDsP6STUuksqcxnAPkmk0JPMtb0xu6cQcMec=;
Message-ID: <d8e29575-9cca-4a08-9deb-03c0b1ade9d7@xen.org>
Date: Thu, 29 Feb 2024 14:08:39 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 04/23] xen/asm-generic: introduce generic fls() and
 flsl() functions
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Oleksii Kurochko <oleksii.kurochko@gmail.com>, xen-devel@lists.xenproject.org
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <df7ab5055ef08fa595f913072302770a3f6a5c33.1708962629.git.oleksii.kurochko@gmail.com>
 <e22ca712-b8cf-4bc9-87df-22cbd7819733@xen.org>
 <654620c6-731a-4a91-8f1c-72961191dff1@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <654620c6-731a-4a91-8f1c-72961191dff1@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Jan,

On 29/02/2024 14:03, Jan Beulich wrote:
> On 29.02.2024 14:54, Julien Grall wrote:
>> On 26/02/2024 17:38, Oleksii Kurochko wrote:
>>> These functions can be useful for architectures that don't
>>> have corresponding arch-specific instructions.
>>>
>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>> ---
>>>    Changes in V5:
>>>      - new patch
>>> ---
>>>    xen/include/asm-generic/bitops/fls.h  | 18 ++++++++++++++++++
>>>    xen/include/asm-generic/bitops/flsl.h | 10 ++++++++++
>>
>> One header per function seems a little bit excessive to me. Do you have
>> any pointer where this request is coming from?
> 
> That's in an attempt to follow Linux'es way of having this, aiui. This way
> an arch can mix and match header inclusions and private definitions without
> any #ifdef-ary.

Ok. I am not going to oppose it if the goal is to keep the headers in 
sync with Linux.

Although, it might have been useful to mention it in the commit message.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:14:35 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:14:35 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687164.1070156 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhB9-0006tY-E0; Thu, 29 Feb 2024 14:14:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687164.1070156; Thu, 29 Feb 2024 14:14:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhB9-0006tR-BJ; Thu, 29 Feb 2024 14:14:31 +0000
Received: by outflank-mailman (input) for mailman id 687164;
 Thu, 29 Feb 2024 14:14:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfhB7-0006s5-SB
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:14:29 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id d99185ca-d70c-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 15:14:28 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a446b5a08f0so1934866b.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 06:14:28 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 gx9-20020a170906f1c900b00a42ec389486sm722879ejb.207.2024.02.29.06.14.27
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 06:14:27 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d99185ca-d70c-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709216068; x=1709820868; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=FMxTJoPxKdLtIXOcRWfGkvIL94Wu4mm3XYfDwuO8Ruo=;
        b=JTT3r0TJiDi6AMacoUfF+bGoOW40sdMbo0+8VKmO+K2fiGAT7Nc5HqHpQY0EzQm8y1
         8hkdWyryE7+t5rCmbj/02dciwSxoYOKswO9qzNiKNoUnOZ2O336HXXoHetsvWE1UEhMF
         FkuG2+r52sZIdqPFd5MK/oN6FNSGl3DXxCmuceMitNeQDDGvyopTEAwS/4BFc/2qNFcx
         UzayHgy+owRZoWXZ7jhTvfIaDtcHLsN+zGlwlKvP+jjaj1cm889vWlKF4qtpXnTAaNoV
         2JeJBXDai9B0N20Y/Ok+KuciF5HBVrSwdr14rHPf1Fc6s0g4H+bXvTLAlKuvOYz9oMAh
         cjDQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709216068; x=1709820868;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=FMxTJoPxKdLtIXOcRWfGkvIL94Wu4mm3XYfDwuO8Ruo=;
        b=Wpz+qVN6tNv1F6KyUmUlBzTwnKONTxz28zYEX1tK+dhWFpDj/GSdtak2gWVuKP9RD4
         5oZB7geb8H/WKdAqD65Lx6NtMj5YAuQMZ1RiSerjIdY8WygvB8S9sUFzqZAMEcU2LqCA
         iwnlTttKCuuQuv5pshJO6NhExeoaGLfp9aczJoZxkg//XaUMV9YyNnbWM62VpspseLFn
         pMpCkq2Zm4lfwkLO6ZADSwumynRqJ/8/VdpqeNPns7C+0EMi825hSBY7tz+egbGoYhhP
         c5xVXm6+tsNJeRVdv3bNOJUydO2tjfFW/kIMoEnLHP/bgfqcf9tc6epa58qYQG3SkgBL
         Xm2Q==
X-Forwarded-Encrypted: i=1; AJvYcCW3FbnF6r2y7f058BW3x7W5qc2FvLmhdVcxoXwDh0cxgf6L4xY9yD14K7OAdBuRhEr/VKrypUmbuOOnmhMqKscpxXbhetYMxhIV3siG3J4=
X-Gm-Message-State: AOJu0Yy8lzWJdBgEUfqTNsAGGIr1WbG5f5NnOoEG4QcIG8e2XXk57uud
	GiRrU6wetLMojKl0plVRErqKU7R8+n6c6uZzuwrH5ojsusxfWnftVQWQZyg/jA==
X-Google-Smtp-Source: AGHT+IFfr8izmvpp9Hh4ziCV8USLe7+E0TqUZq/CsJsRjICkXSWqCZgtM/Kl8KNb68JPwCEs3uyP4Q==
X-Received: by 2002:a17:906:2354:b0:a3f:c32:7b0c with SMTP id m20-20020a170906235400b00a3f0c327b0cmr1569047eja.22.1709216068237;
        Thu, 29 Feb 2024 06:14:28 -0800 (PST)
Message-ID: <1517146f-8bd9-4c8c-95d6-10b754e85d75@suse.com>
Date: Thu, 29 Feb 2024 15:14:26 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 09/12] xen/spinlock: add missing rspin_is_locked() and
 rspin_barrier()
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-10-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20231212094725.22184-10-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 12.12.2023 10:47, Juergen Gross wrote:
> --- a/xen/common/spinlock.c
> +++ b/xen/common/spinlock.c
> @@ -458,6 +458,23 @@ void _spin_barrier(spinlock_t *lock)
>      spin_barrier_common(&lock->tickets, &lock->debug, LOCK_PROFILE_PAR);
>  }
>  
> +int rspin_is_locked(const rspinlock_t *lock)
> +{
> +    /*
> +     * Recursive locks may be locked by another CPU, yet we return
> +     * "false" here, making this function suitable only for use in
> +     * ASSERT()s and alike.
> +     */
> +    return lock->recurse_cpu == SPINLOCK_NO_CPU
> +           ? spin_is_locked_common(&lock->tickets)
> +           : lock->recurse_cpu == smp_processor_id();
> +}
> +
> +void rspin_barrier(rspinlock_t *lock)
> +{
> +    spin_barrier_common(&lock->tickets, &lock->debug, LOCK_PROFILE_PAR);
> +}

Ah, here we go. Looks all okay to me, but needs re-ordering such that the
earlier patch won't transiently introduce a regression.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:14:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:14:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687165.1070166 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhBU-0007Pw-M8; Thu, 29 Feb 2024 14:14:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687165.1070166; Thu, 29 Feb 2024 14:14:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhBU-0007Pp-Ip; Thu, 29 Feb 2024 14:14:52 +0000
Received: by outflank-mailman (input) for mailman id 687165;
 Thu, 29 Feb 2024 14:14:51 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>) id 1rfhBS-0007PW-Vn
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:14:50 +0000
Received: from xenbits.xenproject.org ([104.239.192.120])
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfhBR-00064j-U2; Thu, 29 Feb 2024 14:14:49 +0000
Received: from [15.248.3.86] (helo=[10.45.19.50])
 by xenbits.xenproject.org with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92)
 (envelope-from <julien@xen.org>)
 id 1rfhBR-0007Tv-Ng; Thu, 29 Feb 2024 14:14:49 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org;
	s=20200302mail; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:
	References:Cc:To:Subject:MIME-Version:Date:Message-ID;
	bh=bmwGVTfkjIMYE3GpyYTpwmNMOuHBCDVt2ZBFy8r5+FE=; b=fCcqv+PBonuyddsny68kHe+wU5
	PtlPxuoBSz+jt3ELaKHZ7D4w7rbOHqVbqzoG/o+cvYfMbi8adKMlC61vkHZRW0Doj4JUnjHmrmCO0
	BEluY/MzqKrnxQTtZdT15PBQU7R5vlIl6NZslPt0HwlkF7KzuZs6nXtU8rrtOftTesIQ=;
Message-ID: <b86733f5-e61e-43ab-8375-8a3c7480c8d3@xen.org>
Date: Thu, 29 Feb 2024 14:14:47 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <george.dunlap@citrix.com>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org, Oleksii Kurochko
 <oleksii.kurochko@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
 <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org>
 <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com>
 <2a112321-f4fd-4ab4-a431-a0bb4cb6eaa2@suse.com>
 <9b212dec-59ef-4cbb-bea7-add3867e4599@xen.org>
 <aa3fbfca-4675-4f92-9037-a4a7c5eca4dd@suse.com>
 <8eed0200-eabf-4024-b018-74be45ef95db@xen.org>
 <f28e22b7-2657-40d9-8a97-e747c689c2b0@suse.com>
From: Julien Grall <julien@xen.org>
In-Reply-To: <f28e22b7-2657-40d9-8a97-e747c689c2b0@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit



On 29/02/2024 14:07, Jan Beulich wrote:
> On 29.02.2024 14:44, Julien Grall wrote:
>> Hi Jan,
>>
>> On 29/02/2024 12:51, Jan Beulich wrote:
>>> On 29.02.2024 13:32, Julien Grall wrote:
>>>> On 29/02/2024 12:17, Jan Beulich wrote:
>>>>> On 29.02.2024 13:05, Andrew Cooper wrote:
>>>>>> On 29/02/2024 10:23 am, Julien Grall wrote:
>>>>>>>>>> IOW it is hard for me to see why RISC-V needs stronger restrictions
>>>>>>>>>> here
>>>>>>>>>> than other architectures. It ought to be possible to determine a
>>>>>>>>>> baseline
>>>>>>>>>> version. Even if taking the desire to have "pause" available as a
>>>>>>>>>> requirement, gas (and presumably gld) 2.36.1 would already suffice.
>>>>>>>>>
>>>>>>>>> I think we want to bump it on Arm. There are zero reasons to try to
>>>>>>>>> keep
>>>>>>>>> a lower versions if nobody tests/use it in production.
>>>>>>>>>
>>>>>>>>> I would suggest to do the same on x86. What's the point of try to
>>>>>>>>> support Xen with a 15+ years old compiler?
>>>>>>>>
>>>>>>>> It could have long been bumped if only a proper scheme to follow for
>>>>>>>> this and future bumping would have been put forward by anyone keen on
>>>>>>>> such bumping, like - see his reply - e.g. Andrew. You may recall that
>>>>>>>> this was discussed more than once on meetings, with no real outcome.
>>>>>>>> I'm personally not meaning to stand in the way of such bumping as long
>>>>>>>> as it's done in a predictable manner, but I'm not keen on doing so and
>>>>>>>> hence I don't view it as my obligation to try to invent a reasonable
>>>>>>>> scheme. (My personal view is that basic functionality should be
>>>>>>>> possible to have virtually everywhere, whereas for advanced stuff it
>>>>>>>> is fine to require a more modern tool chain.)
>>>>>>>
>>>>>>> That's one way to see it. The problem with this statement is a user
>>>>>>> today is mislead to think you can build Xen with any GCC versions
>>>>>>> since 4.1. I don't believe we can guarantee that and we are exposing
>>>>>>> our users to unnecessary risk.
>>>>>>>
>>>>>>> In addition to that, I agree with Andrew. This is preventing us to
>>>>>>> improve our code base and we have to carry hacks for older compilers.
>>>>>>
>>>>>> I don't think anyone here is suggesting that we switch to a
>>>>>> bleeding-edge-only policy.  But 15y of support is extreme in the
>>>>>> opposite direction.
>>>>>>
>>>>>> Xen ought to be buildable in the contemporary distros of the day, and I
>>>>>> don't think anyone is going to credibly argue otherwise.
>>>>>>
>>>>>> But, it's also fine for new things to have newer requirements.
>>>>>>
>>>>>> Take CET for example.  I know we have disagreements on exactly how it's
>>>>>> toolchain-conditionalness is implemented, but the basic principle of "If
>>>>>> you want shiny new optional feature $X, you need newer toolchain $Y" is
>>>>>> entirely fine.
>>>>>>
>>>>>> A brand new architecture is exactly the same.  Saying "this is the
>>>>>> minimum, because it's what we test" doesn't preclude someone coming
>>>>>> along and saying "can we use $N-1 ?  See here it works, and here's a
>>>>>> change to CI test it".
>>>>>>
>>>>>>
>>>>>> Anyway, its clear we need to write some policy on this, before making
>>>>>> specific adjustments.  To get started, is there going to be any
>>>>>> objection whatsoever on some principles which begin as follows:
>>>>>
>>>>> Largely not, but one aspect needs clarifying up front:
>>>>>
>>>>>> * For established architectures, we expect Xen to be buildable on the
>>>>>> common contemporary distros.  (i.e. minima is not newer than what's
>>>>>> available in contemporary distros, without a good reason)
>>>>>
>>>>> What counts as contemporary distro? Still in normal support? LTS? Yet
>>>>> more extreme forms?
>>>>
>>>> LTS makes sense. More I am not sure. I am under the impression that
>>>> people using older distros are those that wants a stable system. So they
>>>> would unlikely try to upgrade the hypervisor.
>>>>
>>>> Even for LTS, I would argue that if it has been released 5 years ago,
>>>> then you probably want to update it at the same time as moving to a
>>>> newer Xen version.
>>>
>>> For the purposes of distros I agree. For the purposes of individuals
>>> I don't: What's wrong with running a newer hypervisor and/or kernel
>>> underneath an older distro?
>>
>> There is nothing wrong. I just don't understand the benefits for us to
>> support that use case. To me there are two sorts of individuals:
>>    1. The ones that are using distro packages. They will unlikely want to
>> switch to a newer hypervisor
>>    2. The ones that are happy to compile and hack their system. Fairly
>> likely they will use a more distros and/or would not be put up by
>> upgrading it.
>>
>> What individuals do you have in mind?
> 
> People like me.

Which means? From what I read you mostly use an older distros for smoke 
testing/convenience.

>> Also, for me, the minimum doesn't prevent anyone to try to compile with
>> an older compiler. It is only here to say that as a community we will
>> not investigate or trying to workaround bugs in those compilers.
> 
> Besides this also allowing to use functionality you won't have an easy
> way of replacing, what you say also doesn't make clear whether - for
> cases where the issue can be (reasonably easily) worked around - patches
> would be accepted, or rejected on the basis of only helping a below-the-
> line compiler.

I would not accept them.

Cheers,

-- 
Julien Grall


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:18:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:18:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687171.1070175 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhF6-0000V5-5i; Thu, 29 Feb 2024 14:18:36 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687171.1070175; Thu, 29 Feb 2024 14:18:36 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhF6-0000Uy-37; Thu, 29 Feb 2024 14:18:36 +0000
Received: by outflank-mailman (input) for mailman id 687171;
 Thu, 29 Feb 2024 14:18:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rfhF4-0000Sl-B6
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:18:34 +0000
Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 6b40fab7-d70d-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 15:18:33 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out1.suse.de (Postfix) with ESMTPS id 816F72236C;
 Thu, 29 Feb 2024 14:18:32 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 10DB213503;
 Thu, 29 Feb 2024 14:18:32 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id bh2hATiS4GUxHgAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 14:18:32 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6b40fab7-d70d-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709216312; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=cwFpXNyFH3FHcysLv87lCl1U+yDO/aQTyRzGoVPXfk0=;
	b=LXHHPEpeRIX7Jjc7n2+skF3WJ90LfFrf27VWd6Klmmc+Igh/hwuU4/QACuzB3zG7KGFRfx
	OQs3FDBzTBZBWVKmAHmz2aqWIyLUB9dcSFF/sOr1FKqJzZWYCspwi1kF0vR7TO0CS750Q2
	W1dcVeXNWB45aGgDkmL0Jk1oJUU6/GI=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709216312; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=cwFpXNyFH3FHcysLv87lCl1U+yDO/aQTyRzGoVPXfk0=;
	b=LXHHPEpeRIX7Jjc7n2+skF3WJ90LfFrf27VWd6Klmmc+Igh/hwuU4/QACuzB3zG7KGFRfx
	OQs3FDBzTBZBWVKmAHmz2aqWIyLUB9dcSFF/sOr1FKqJzZWYCspwi1kF0vR7TO0CS750Q2
	W1dcVeXNWB45aGgDkmL0Jk1oJUU6/GI=
Message-ID: <b9865ed2-694a-4c88-963c-c9bb41727f1d@suse.com>
Date: Thu, 29 Feb 2024 15:18:31 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 09/12] xen/spinlock: add missing rspin_is_locked() and
 rspin_barrier()
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>,
 =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>, Wei Liu
 <wl@xen.org>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Paul Durrant <paul@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-10-jgross@suse.com>
 <1517146f-8bd9-4c8c-95d6-10b754e85d75@suse.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <1517146f-8bd9-4c8c-95d6-10b754e85d75@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Authentication-Results: smtp-out1.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=LXHHPEpe
X-Spamd-Result: default: False [-1.49 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-0.90)[86.07%];
	 MIME_GOOD(-0.10)[text/plain];
	 DWL_DNSWL_LOW(-1.00)[suse.com:dkim];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[9];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.71)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 816F72236C
X-Spam-Level: 
X-Spam-Score: -1.49
X-Spam-Flag: NO

On 29.02.24 15:14, Jan Beulich wrote:
> On 12.12.2023 10:47, Juergen Gross wrote:
>> --- a/xen/common/spinlock.c
>> +++ b/xen/common/spinlock.c
>> @@ -458,6 +458,23 @@ void _spin_barrier(spinlock_t *lock)
>>       spin_barrier_common(&lock->tickets, &lock->debug, LOCK_PROFILE_PAR);
>>   }
>>   
>> +int rspin_is_locked(const rspinlock_t *lock)
>> +{
>> +    /*
>> +     * Recursive locks may be locked by another CPU, yet we return
>> +     * "false" here, making this function suitable only for use in
>> +     * ASSERT()s and alike.
>> +     */
>> +    return lock->recurse_cpu == SPINLOCK_NO_CPU
>> +           ? spin_is_locked_common(&lock->tickets)
>> +           : lock->recurse_cpu == smp_processor_id();
>> +}
>> +
>> +void rspin_barrier(rspinlock_t *lock)
>> +{
>> +    spin_barrier_common(&lock->tickets, &lock->debug, LOCK_PROFILE_PAR);
>> +}
> 
> Ah, here we go. Looks all okay to me, but needs re-ordering such that the
> earlier patch won't transiently introduce a regression.

Yes, just wanted to answer something similar to your remark on patch 8.


Juergen


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:21:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:21:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687173.1070186 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhHa-0003CS-JJ; Thu, 29 Feb 2024 14:21:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687173.1070186; Thu, 29 Feb 2024 14:21:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhHa-0003CL-FX; Thu, 29 Feb 2024 14:21:10 +0000
Received: by outflank-mailman (input) for mailman id 687173;
 Thu, 29 Feb 2024 14:21:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=xh/y=KG=cloud.com=roger.pau@srs-se1.protection.inumbo.net>)
 id 1rfhHZ-0003CD-62
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:21:09 +0000
Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com
 [2a00:1450:4864:20::32a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c649d33c-d70d-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 15:21:05 +0100 (CET)
Received: by mail-wm1-x32a.google.com with SMTP id
 5b1f17b1804b1-412c23551e2so2086695e9.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 06:21:06 -0800 (PST)
Received: from localhost ([213.195.118.74]) by smtp.gmail.com with ESMTPSA id
 ba16-20020a0560001c1000b0033d9c7eb63csm1979160wrb.84.2024.02.29.06.21.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 06:21:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c649d33c-d70d-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709216465; x=1709821265; darn=lists.xenproject.org;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date:from:to
         :cc:subject:date:message-id:reply-to;
        bh=XyayGrmlCYqAjt5gP4PYhjhCQgqwvx5Smhda8T+WU1g=;
        b=s5wpTronZrfzsy5srvQbd7bUVHQ3KWjX9dH985iPaBgmWr4JI1HOorUCu75FLORNzk
         1M6zgoQWdIexETOTJATR+Sa6sXQc1O2GAzBOTKM0yLgidFELVYyf6aBffhYekqifZSTB
         Th7+Ngj6M/U/zTwpQMjJVTgfegVhnP8cqllFk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709216465; x=1709821265;
        h=in-reply-to:content-transfer-encoding:content-disposition
         :mime-version:references:message-id:subject:cc:to:from:date
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=XyayGrmlCYqAjt5gP4PYhjhCQgqwvx5Smhda8T+WU1g=;
        b=lKBiPC3K002HwNv54xhWlmg1yQxpM3AsZxTPpoS15egUHSQxBNmufPfCOehX3N55+d
         8N9aAe050KmmqqEdfFtg1csZXMuMAdUtjzgbl3HTJOJWk0kLnsd05iPdTr6iAXoYqGgq
         9Rr7g8iHZHQoRScNWsCwrREkx+nhCT7FHrVLgxvuYNnfRp1iR6mtieV1orBduo+TpZoe
         sEolpZmMk9pvxasl8mnu79IiFLTSiasJ5Ov4549ObCFHIGXiDH03Ag95U4KYXippPG5U
         24V82VuJV13n/RNxWLAvfvTHem9AcCkj9GHz9/W5b/eokp7vFLzhIdhPTr6aEkY6W8AJ
         HUeA==
X-Forwarded-Encrypted: i=1; AJvYcCVDEtBTBogO5TB2lBHv3/bTzsbdCxlC1C2/MGsIpbqV0saaWYvFgN9MhDuileUKnemae9WrBQXfRy0YLHUUcmK37CKN/CnIqHdug19fEW0=
X-Gm-Message-State: AOJu0YxnkE6jBdHujxUSC40cGBOpgWKUBGCwDfLjICFz+kn7/ddBcSgi
	vTmUmcOwg8Mr22RFynrvA/3x1lTaQrKirjmYkahclZZLTA3IeXZe4r0d9xAIybs=
X-Google-Smtp-Source: AGHT+IHqo2TyVjn3Os5bPfWfibGbr6SfPjXd0ekwCoRyERAQ53fsLWLMdnXT9nbbVyvY9U2Nxln+Xw==
X-Received: by 2002:a05:600c:4f84:b0:412:c1d4:dd0d with SMTP id n4-20020a05600c4f8400b00412c1d4dd0dmr1173256wmq.3.1709216465376;
        Thu, 29 Feb 2024 06:21:05 -0800 (PST)
Date: Thu, 29 Feb 2024 15:21:04 +0100
From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	George Dunlap <george.dunlap@citrix.com>,
	Julien Grall <julien@xen.org>,
	Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	Doug Goldstein <cardoe@cardoe.com>, xen-devel@lists.xenproject.org
Subject: Re: [PATCH 1/2] README: bump minimum required clang/llvm version
Message-ID: <ZeCS0OC4FsbEXo8S@macbook>
References: <20240229095529.17723-1-roger.pau@citrix.com>
 <20240229095529.17723-2-roger.pau@citrix.com>
 <780ec3ab-08e3-4fd8-a85f-0e89ea75dcc8@suse.com>
 <ZeB7S6i7pIwzkUNE@macbook>
 <78032daf-9052-4ab8-ba05-456fa4aee17b@suse.com>
 <132dde96-d7d3-4823-8a0e-7b4a62526f2e@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <132dde96-d7d3-4823-8a0e-7b4a62526f2e@suse.com>

On Thu, Feb 29, 2024 at 02:12:26PM +0100, Jan Beulich wrote:
> On 29.02.2024 14:01, Jan Beulich wrote:
> > On 29.02.2024 13:40, Roger Pau Monné wrote:
> >> On Thu, Feb 29, 2024 at 01:11:55PM +0100, Jan Beulich wrote:
> >>> On 29.02.2024 10:55, Roger Pau Monne wrote:
> >>>> --- a/README
> >>>> +++ b/README
> >>>> @@ -41,7 +41,7 @@ provided by your OS distributor:
> >>>>          - GCC 4.1.2_20070115 or later
> >>>>          - GNU Binutils 2.16.91.0.5 or later
> >>>>          or
> >>>> -        - Clang/LLVM 3.5 or later
> >>>> +        - Clang/LLVM 14.0.0 or later
> >>>
> >>> Wow, that's a big step. I'm build-testing with Clang7 on one system and
> >>> with Clang5 on another (and the latter more frequently than the former).
> >>> If any real dependency on this new a version (about 3 years old?) was
> >>> introduced, I would then no longer be able to locally test any Clang
> >>> builds (and hence the risk would again increase that I introduce issues
> >>> that affect just Clang builds).
> >>
> >> Would it be possible for you to update to a newer version?  I see both
> >> the OpenSUSE containers in Gitlab have newer versions of Clang.
> > 
> > No. These are SLES versions which I'm not intending to touch. See
> > 
> > https://lists.xen.org/archives/html/xen-devel/2024-02/msg01793.html
> > 
> > and
> > 
> > https://lists.xen.org/archives/html/xen-devel/2024-02/msg01795.html
> > 
> > for why. The most recent piece of hardware I've installed a fresh (but
> > not exactly new, yet still fully supported) SLES version on would
> > apparently offer Clang7 only, either.

Hm, OK.  Would containers be an option?

I don't know for gcc, but for clang I'm quite sure the only run time
tested builds are the ones used by FreeBSD.  I'm hesitant to claim
clang versions as supported when they are only tested to generate an
output without triggering any errors, but there's no testing at all
that the generated code is functional.

> Bah, that's not even Clang, only LLVM.

I'm confused by this, doesn't your llvm package include clang?

Thanks, Roger.


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:22:58 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:22:58 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687175.1070195 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhJI-0003jJ-TF; Thu, 29 Feb 2024 14:22:56 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687175.1070195; Thu, 29 Feb 2024 14:22:56 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhJI-0003jC-QI; Thu, 29 Feb 2024 14:22:56 +0000
Received: by outflank-mailman (input) for mailman id 687175;
 Thu, 29 Feb 2024 14:22:55 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfhJH-0003j2-Lk
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:22:55 +0000
Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com
 [2607:f8b0:4864:20::833])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 064393e0-d70e-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 15:22:53 +0100 (CET)
Received: by mail-qt1-x833.google.com with SMTP id
 d75a77b69052e-42eb3821a2dso7723621cf.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 06:22:53 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 f6-20020a05620a12e600b00787ec1797cbsm713892qkl.68.2024.02.29.06.22.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 06:22:51 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 064393e0-d70e-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709216572; x=1709821372; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=2Iqk1NTdTbbbDKkIL8Peddq7htr0rfl6qGGtw8YL4cE=;
        b=CcoL7+FcG0vh2xo41ZUE1lvorYNrMT7bkKGj2ZEe+qzrhaXsQ6lNEM98RlZYD6GTvy
         piHB4hveu598AYF2Hs9HAtIrYjEwqrbyhilI5qaCbYmAkcDWS7+F2YmLoK4Ib4muivM3
         TlITCtR6U+uf6cwbOT0U7DKRCfW/C/AOYUveE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709216572; x=1709821372;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=2Iqk1NTdTbbbDKkIL8Peddq7htr0rfl6qGGtw8YL4cE=;
        b=edfssi0mfkRRD64/K3mo0vftg8GMJ+cW3/KnlcJLrWTF4tLhCWKBhxzliesw9q/LrL
         qJu1Evgc6Mv5DvT017cDTqDWggXEl2C10VMneybTkU//b3tCYuwRZfDAKmouOjxGDA2h
         h03HGykcpBKR1kaPyC3gooRdLq9k/jhWvP/FK9Y6roC5dm0nx+rIEu0TEKhoF4+FMEyi
         Cf4msAf5UJEXdE1KHIIAcHG6P9tOC4b/f418Lc03j6d4pG15LSZODwgaHjawqEToxmI4
         Id2JIpTzAQYScvr1SRSNlKLJ6Nz745YwGhdnOpdJKvCk52h/G/2Eb0g8UZKCEFOviu9I
         IQPA==
X-Forwarded-Encrypted: i=1; AJvYcCU5TBNW/saChbG3EbO1dFZxBklxuwrwRWWpZxJ1tIr81c/Vh7Ooukjfaxdrx1y8M7aI+D5BsZ/XLkikwqIyCc5NHlMVHBEfISMx93wiOA8=
X-Gm-Message-State: AOJu0YxHl7yFVDpTNbKoNeNgVjyaXH5RhvaJVLo86TMK0ywlAbPwePmK
	EvVK6Dx53OieAKhFCzZa1cHq5F5VKVUP/TgACwNRsgJF6zLrNX30g6l59MdEFp8=
X-Google-Smtp-Source: AGHT+IENh6dkCAZ0dL6/MjaL7xIjoIYPlcRF/XTmZaXAUsA/uz392W8Qmr3XlRd87KdjslfOmhNRrw==
X-Received: by 2002:ac8:5c10:0:b0:42d:ff6f:d68a with SMTP id i16-20020ac85c10000000b0042dff6fd68amr2133872qti.24.1709216572341;
        Thu, 29 Feb 2024 06:22:52 -0800 (PST)
Message-ID: <c4013d29-8820-4c8d-80ec-1015edfa8820@citrix.com>
Date: Thu, 29 Feb 2024 14:22:49 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpu-policy: Fix x2APIC visibility for PV guests
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240229104304.2478614-1-andrew.cooper3@citrix.com>
 <707ff486-b448-4bc0-be7e-1c692b360734@suse.com>
 <623e9c0e-f05f-41e7-a800-0bba01cba7f4@citrix.com>
 <e811b3b0-a3d6-488d-b93c-e24c416477bf@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <e811b3b0-a3d6-488d-b93c-e24c416477bf@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/02/2024 1:29 pm, Jan Beulich wrote:
> On 29.02.2024 14:23, Andrew Cooper wrote:
>> On 29/02/2024 12:47 pm, Jan Beulich wrote:
>>> On 29.02.2024 11:43, Andrew Cooper wrote:
>>>> Right now, the host x2APIC setting filters into the PV max and default
>>>> policies, yet PV guests cannot set MSR_APIC_BASE.EXTD or access any of the
>>>> x2APIC MSR range.  Therefore they absolutely shouldn't see the x2APIC bit.
>>>>
>>>> Linux has workarounds for the collateral damage caused by this leakage; it
>>>> unconditionally filters out the x2APIC CPUID bit, and EXTD when reading
>>>> MSR_APIC_BASE.
>>>>
>>>> Hide the x2APIC bit in the PV default policy, but for compatibility, tolerate
>>>> incoming VMs which already saw the bit.  This is logic from before the
>>>> default/max split in Xen 4.14 which wasn't correctly adjusted at the time.
>>> What about guest_cpuid()'s handling of leaf 0xb then? The %edx value
>>> will change once a guest is rebooted, aiui. The comment in
>>> recalculate_cpuid_policy() that you update refers to that.
>> That comment is going in the next patch irrespective.
>>
>> But yes - this will change leaf 0xb from being
>> host-conditionally-visible to always hidden.
> Imo this wants saying explicitly,

Yeah - already started that for v2.

>  including why that's okay to do,
> especially since ...
>
>> PV guests don't have any coherent idea of topology.  Linux (with the
>> topo fixes) now explicitly ignores everything it can see and just fakes
>> up a flat non-SMT topology in a single package.
> ... you validly use "now" here. Plus Linux isn't the only PV guest we
> need to care about.

As I said on the other thread, NetBSD works in the spirit in which PV
guests were intended and completely ignores x2APIC in XENPV builds.

>
> What's wrong (more wrong than the present putting of vCPU ID * 2 there)
> with retaining the population of that leaf (by dropping the x2apic
> dependency there)?

Without an MADT it's meaningless.   For PV dom0, it's actively wrong
because there is an MADT and it's in the wrong space.

libxc has never written anything coherent in here, because it's never
had any coherent idea about topology.  Alejandro is working on that, and
I believe one of the prep series has been posted.  There's a lot more to go.

Even today, we end up overlaying the host's APIC_ID space layout over
the blind vCPU_ID * 2, which makes the result still nonsense.

Various versions of Xen have tried playing with this, without
understanding properly what they're doing, and XenServer still has a
revert of a Xen 3.4 patch in the patch, as it broken migration of guests
at the time...


There is going to be a future (hopefully soon) where HVM guests get to
see something which conforms to the architectural specs, and is sane.

But doing the same for PV guests is more complicated, because of the
conflicting requirements between PV guests not really having an APIC,
but APIC being the x86 architectural expression of topology.


>>>> This wants backporting as far as people can tollerate, but it's really not
>>>> obvious which commit in 4.14 should be referenced in a Fixes: tag.
>>> Why 4.14? In 4.7.0 I see ...
>>>
>>>> @@ -830,11 +846,10 @@ void recalculate_cpuid_policy(struct domain *d)
>>>>      }
>>>>  
>>>>      /*
>>>> -     * Allow the toolstack to set HTT, X2APIC and CMP_LEGACY.  These bits
>>>> +     * Allow the toolstack to set HTT and CMP_LEGACY.  These bits
>>>>       * affect how to interpret topology information in other cpuid leaves.
>>>>       */
>>>>      __set_bit(X86_FEATURE_HTT, max_fs);
>>>> -    __set_bit(X86_FEATURE_X2APIC, max_fs);
>>>>      __set_bit(X86_FEATURE_CMP_LEGACY, max_fs);
>>>>  
>>>>      /*
>>> ... these adjustments, just still in calculate_pv_featureset(). I
>>> haven't gone further backwards to check if/when this exposure has
>>> really appeared. I wouldn't be surprised if it's been like that
>>> for all the time since we gained x2APIC support in the hypervisor.
>> 4.14 was when we got the proper default vs max split.  Before then, this
>> block of logic was an opencoded "max(ish) for tookstacks which know
>> about it" kind of thing.
> Except it was also affecting what guests get to see, afaict.

No - this hunk explicitly doesn't.

What this hunk says is "don't override the toolstack's choice based on
what the host can see".

Because even today, Xen is still blindly zeroing toolstack settings it
doesn't like, because my series fixing this is still sat on the mailing
list from years and years ago over and argument over whether a function
lives in libx86 or elsewhere...

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 14:28:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 14:28:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687181.1070206 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhOV-0004QJ-Jn; Thu, 29 Feb 2024 14:28:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687181.1070206; Thu, 29 Feb 2024 14:28:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfhOV-0004QC-G7; Thu, 29 Feb 2024 14:28:19 +0000
Received: by outflank-mailman (input) for mailman id 687181;
 Thu, 29 Feb 2024 14:28:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfhOU-0004Q6-IW
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 14:28:18 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c7af538a-d70e-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 15:28:17 +0100 (CET)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-565a3910f86so1564538a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 06:28:17 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 h2-20020a0564020e8200b005653439cadcsm662894eda.25.2024.02.29.06.28.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 06:28:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c7af538a-d70e-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709216897; x=1709821697; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=T3W/0jISBokMHOR0jW03nKtLF0I9Wu2tJsHJKhuUaHk=;
        b=cwJniDIy+jz5jXaY4PX+wKRxVWb7OnslZSWeY3cmLDkh7PygaRblnDmXv8YUoZ5HHN
         r2hBBPxPZWkrffSe6zzbZKuRGgfuEwvR+mgu5cSSPVDMFF03pvsVj6GAoMbp9j+/t99v
         3cCPdLrGNvXPlyhfL8rMIrmfFX5mAay84JTAIHhMSbXgJgLu7M8iSsqB1wTGB+kZPqTj
         kpa++4W1pLakI7HLQzWJ0XvMS9iD6mABS3ihsonDGhGGxRiBg92Wa3BAOf7D0zE9ueUz
         O72Q1CV7+rP1v+xfa3b/pPgF+4WxApt77daSAOfEwZNjhygGIaor2szmF9C5pFyW53R0
         OQkQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709216897; x=1709821697;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=T3W/0jISBokMHOR0jW03nKtLF0I9Wu2tJsHJKhuUaHk=;
        b=auiCyzpiFUWw9fYdyhtdAmX8PTGiPENNVw0gpk6ed0ziSIPftT3EvvF0jsZu/TklpD
         2SL9PwZ51Cce/EZxXmBJRdrG5lLPtBmIen7gYvCg6QAG/ud2wvukbneXKMh6jvDnqzYC
         PYMv7MAjJVB2KKMPlznlTydSuqd9zTWecsm2FTtHDO0vupXK/B2iU6gSkfTi/HrmEYB5
         IYaxPIB3/HT/+VB3GNb6LL7yezqZtz87vtia08K3BdADYkMAQTFhH+qsCayYWLVaFoJk
         oR0+Ra1kxd/VOn7+c/rsm1xOa+eRl6Sw64bbUayadu/t4s0+IhW0EkVZFPuYc2GPvnjn
         ROEw==
X-Forwarded-Encrypted: i=1; AJvYcCXKna5bAUdzlJzPY5NGK0mfRWgMjtsB97QKWYB2GHcI2Xz1gDtc3fCZrcL2imw75qSeqfoiCLB78wZusmzl40Hm5J6ULxcpVgcfYGJfrK8=
X-Gm-Message-State: AOJu0Yx7W/01efItOqBDGEeIf3BUt8CjrDCRhrCRZM1nnl3DlWoCVsvc
	yhmDld/0xbBnaEXZo/79J377OG+0K1jGMjttBrzX8xswwX/BPKq3TGIgQVnrUQ==
X-Google-Smtp-Source: AGHT+IHxz2PKqhrTMzysxz0ryH9Zq3DbFrfJHpUto2xJBRZsGkcKIbprSpxkR4m05ppcA+wv04P6iw==
X-Received: by 2002:a50:cc46:0:b0:565:1815:dfc5 with SMTP id n6-20020a50cc46000000b005651815dfc5mr1555654edi.0.1709216897152;
        Thu, 29 Feb 2024 06:28:17 -0800 (PST)
Message-ID: <51df7369-b77b-4e48-8759-b736358cd06a@suse.com>
Date: Thu, 29 Feb 2024 15:28:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] README: bump minimum required clang/llvm version
Content-Language: en-US
To: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 Doug Goldstein <cardoe@cardoe.com>, xen-devel@lists.xenproject.org
References: <20240229095529.17723-1-roger.pau@citrix.com>
 <20240229095529.17723-2-roger.pau@citrix.com>
 <780ec3ab-08e3-4fd8-a85f-0e89ea75dcc8@suse.com> <ZeB7S6i7pIwzkUNE@macbook>
 <78032daf-9052-4ab8-ba05-456fa4aee17b@suse.com>
 <132dde96-d7d3-4823-8a0e-7b4a62526f2e@suse.com> <ZeCS0OC4FsbEXo8S@macbook>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <ZeCS0OC4FsbEXo8S@macbook>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 15:21, Roger Pau Monné wrote:
> On Thu, Feb 29, 2024 at 02:12:26PM +0100, Jan Beulich wrote:
>> Bah, that's not even Clang, only LLVM.
> 
> I'm confused by this, doesn't your llvm package include clang?

No, there are quite a few RPMs in general in SLES to cover everything,
yet on the distro in question the clang* one(s) are (is) missing.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687194.1070251 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKT-0000QV-Vx; Thu, 29 Feb 2024 15:28:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687194.1070251; Thu, 29 Feb 2024 15:28:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKT-0000QD-Ry; Thu, 29 Feb 2024 15:28:13 +0000
Received: by outflank-mailman (input) for mailman id 687194;
 Thu, 29 Feb 2024 15:28:12 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKS-0008Ql-PM
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:12 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 2576eed4-d717-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:28:10 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id 3B1364EE074C;
 Thu, 29 Feb 2024 16:28:10 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2576eed4-d717-11ee-a1ee-f123f15fe8a2
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH 03/10] x86: address some violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:27:55 +0100
Message-Id: <3c9e90aaf5dde769b689468fc818e4ae61fa11f3.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1709219010.git.nicola.vetrini@bugseng.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

GUARD(1) is also amended to avoid modifying UA_KEEP or its definition.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
I wasn't very sure whether touching the definition of UA_KEEP would be a good
idea, so I added parentheses in the only user I've seen so far that causes a
violation.
---
 xen/arch/x86/include/asm/irq.h | 6 +++---
 xen/arch/x86/usercopy.c        | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/include/asm/irq.h b/xen/arch/x86/include/asm/irq.h
index 082a3d6bbc6a..5c722848e8ce 100644
--- a/xen/arch/x86/include/asm/irq.h
+++ b/xen/arch/x86/include/asm/irq.h
@@ -179,9 +179,9 @@ void cleanup_domain_irq_mapping(struct domain *d);
     void *__ret = radix_tree_lookup(&(d)->arch.hvm.emuirq_pirq, emuirq);\
     __ret ? radix_tree_ptr_to_int(__ret) : IRQ_UNBOUND;                 \
 })
-#define IRQ_UNBOUND -1
-#define IRQ_PT -2
-#define IRQ_MSI_EMU -3
+#define IRQ_UNBOUND (-1)
+#define IRQ_PT      (-2)
+#define IRQ_MSI_EMU (-3)
 
 bool cpu_has_pending_apic_eoi(void);
 
diff --git a/xen/arch/x86/usercopy.c b/xen/arch/x86/usercopy.c
index b8c2d1cc0bed..b0b55398e968 100644
--- a/xen/arch/x86/usercopy.c
+++ b/xen/arch/x86/usercopy.c
@@ -106,7 +106,7 @@ unsigned int copy_from_guest_ll(void *to, const void __user *from, unsigned int
     return n;
 }
 
-#if GUARD(1) + 0
+#if GUARD((1)) + 0
 
 /**
  * copy_to_guest_pv: - Copy a block of data into PV guest space.
-- 
2.34.1


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687198.1070291 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKX-0001Rx-IP; Thu, 29 Feb 2024 15:28:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687198.1070291; Thu, 29 Feb 2024 15:28:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKX-0001RX-BU; Thu, 29 Feb 2024 15:28:17 +0000
Received: by outflank-mailman (input) for mailman id 687198;
 Thu, 29 Feb 2024 15:28:15 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKV-0008Ql-QY
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:15 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 277a35f8-d717-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:28:14 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id 7B5DA4EE0751;
 Thu, 29 Feb 2024 16:28:13 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 277a35f8-d717-11ee-a1ee-f123f15fe8a2
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH 08/10] xen/errno: address violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:28:00 +0100
Message-Id: <92dbcd9e5aa2ab45c50f3f567ddece59108568ad.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1709219010.git.nicola.vetrini@bugseng.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/include/xen/errno.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/xen/errno.h b/xen/include/xen/errno.h
index 69b28dd3c6c5..506674701fae 100644
--- a/xen/include/xen/errno.h
+++ b/xen/include/xen/errno.h
@@ -3,7 +3,7 @@
 
 #ifndef __ASSEMBLY__
 
-#define XEN_ERRNO(name, value) name = value,
+#define XEN_ERRNO(name, value) name = (value),
 enum {
 #include <public/errno.h>
 };
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687199.1070296 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKY-0001Xr-5R; Thu, 29 Feb 2024 15:28:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687199.1070296; Thu, 29 Feb 2024 15:28:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKX-0001Vj-Qj; Thu, 29 Feb 2024 15:28:17 +0000
Received: by outflank-mailman (input) for mailman id 687199;
 Thu, 29 Feb 2024 15:28:16 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKW-0008Ql-Nu
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:16 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 27e30321-d717-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:28:15 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id 2F2B24EE0750;
 Thu, 29 Feb 2024 16:28:14 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 27e30321-d717-11ee-a1ee-f123f15fe8a2
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH 09/10] xen/include: tasklet: address violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:28:01 +0100
Message-Id: <952d13bdb830cb1d8d3282e49333323bc8755cd5.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1709219010.git.nicola.vetrini@bugseng.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/include/xen/tasklet.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/xen/tasklet.h b/xen/include/xen/tasklet.h
index 593d6a2400fb..78760b694a39 100644
--- a/xen/include/xen/tasklet.h
+++ b/xen/include/xen/tasklet.h
@@ -27,7 +27,7 @@ struct tasklet
 
 #define _DECLARE_TASKLET(name, func, data, softirq)                     \
     struct tasklet name = {                                             \
-        LIST_HEAD_INIT(name.list), -1, softirq, 0, 0, func, data }
+        LIST_HEAD_INIT((name).list), -1, softirq, 0, 0, func, data }
 #define DECLARE_TASKLET(name, func, data)               \
     _DECLARE_TASKLET(name, func, data, 0)
 #define DECLARE_SOFTIRQ_TASKLET(name, func, data)       \
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687196.1070271 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKV-0000wt-O0; Thu, 29 Feb 2024 15:28:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687196.1070271; Thu, 29 Feb 2024 15:28:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKV-0000vn-Gq; Thu, 29 Feb 2024 15:28:15 +0000
Received: by outflank-mailman (input) for mailman id 687196;
 Thu, 29 Feb 2024 15:28:13 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKT-00087e-KT
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:13 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 26a97bd9-d717-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:28:13 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id 2112C4EE074F;
 Thu, 29 Feb 2024 16:28:12 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 26a97bd9-d717-11ee-afd8-a90da7624cb6
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	Rahul Singh <rahul.singh@arm.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [XEN PATCH 06/10] arm/smmu: address some violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:27:58 +0100
Message-Id: <8bbf294d832b973648e84e5d837b1878a84f7645.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1709219010.git.nicola.vetrini@bugseng.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/drivers/passthrough/arm/smmu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index 625ed0e41961..83196057a937 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -242,7 +242,7 @@ struct arm_smmu_xen_device {
 	struct iommu_group *group;
 };
 
-#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->iommu)
+#define dev_archdata(dev) ((struct arm_smmu_xen_device *)(dev)->iommu)
 #define dev_iommu_domain(dev) (dev_archdata(dev)->domain)
 #define dev_iommu_group(dev) (dev_archdata(dev)->group)
 
@@ -627,7 +627,7 @@ struct arm_smmu_master_cfg {
 };
 #define INVALID_SMENDX			-1
 #define for_each_cfg_sme(cfg, i, idx, num) \
-	for (i = 0; idx = cfg->smendx[i], i < num; ++i)
+	for (i = 0; idx = (cfg)->smendx[i], (i) < (num); ++(i))
 
 struct arm_smmu_master {
 	struct device_node		*of_node;
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687195.1070266 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKV-0000rj-Bx; Thu, 29 Feb 2024 15:28:15 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687195.1070266; Thu, 29 Feb 2024 15:28:15 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKV-0000rA-6I; Thu, 29 Feb 2024 15:28:15 +0000
Received: by outflank-mailman (input) for mailman id 687195;
 Thu, 29 Feb 2024 15:28:13 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKT-0008Ql-F2
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:13 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 25de8189-d717-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:28:11 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id C5A944EE074D;
 Thu, 29 Feb 2024 16:28:10 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25de8189-d717-11ee-a1ee-f123f15fe8a2
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH 04/10] xen/public: address violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:27:56 +0100
Message-Id: <0cdc4dc2fcad699a2274277b32de3ee0207d5a2d.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1709219010.git.nicola.vetrini@bugseng.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore
the macro XEN_DEFINE_UUID_ should wrap its parameters in parentheses.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/include/public/xen.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index b47d48d0e2d6..fa23080bd7af 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -988,7 +988,7 @@ typedef struct {
       ((b) >>  8) & 0xFF, ((b) >>  0) & 0xFF,                           \
       ((c) >>  8) & 0xFF, ((c) >>  0) & 0xFF,                           \
       ((d) >>  8) & 0xFF, ((d) >>  0) & 0xFF,                           \
-                e1, e2, e3, e4, e5, e6}}
+                (e1), (e2), (e3), (e4), (e5), (e6)}}
 
 #if defined(__STDC_VERSION__) ? __STDC_VERSION__ >= 199901L : defined(__GNUC__)
 #define XEN_DEFINE_UUID(a, b, c, d, e1, e2, e3, e4, e5, e6)             \
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687197.1070277 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKW-00012z-74; Thu, 29 Feb 2024 15:28:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687197.1070277; Thu, 29 Feb 2024 15:28:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKV-00010D-Tv; Thu, 29 Feb 2024 15:28:15 +0000
Received: by outflank-mailman (input) for mailman id 687197;
 Thu, 29 Feb 2024 15:28:14 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKU-00087e-2r
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:14 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2711c01f-d717-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:28:13 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id C4A8C4EE0737;
 Thu, 29 Feb 2024 16:28:12 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2711c01f-d717-11ee-afd8-a90da7624cb6
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	Rahul Singh <rahul.singh@arm.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [XEN PATCH 07/10] xen/arm: smmuv3: address violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:27:59 +0100
Message-Id: <a11cd3f9c3dc64d67df14d8b7527b7467d46f573.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1709219010.git.nicola.vetrini@bugseng.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/drivers/passthrough/arm/smmu-v3.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index c3ac6d17d1c8..b1c40c2c0ae7 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -111,7 +111,7 @@
 #define GFP_KERNEL		0
 
 /* Device logger functions */
-#define dev_name(dev)	dt_node_full_name(dev->of_node)
+#define dev_name(dev)	dt_node_full_name((dev)->of_node)
 #define dev_dbg(dev, fmt, ...)			\
 	printk(XENLOG_DEBUG "SMMUv3: %s: " fmt, dev_name(dev), ## __VA_ARGS__)
 #define dev_notice(dev, fmt, ...)		\
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687200.1070302 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKY-0001fA-In; Thu, 29 Feb 2024 15:28:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687200.1070302; Thu, 29 Feb 2024 15:28:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKY-0001cQ-6M; Thu, 29 Feb 2024 15:28:18 +0000
Received: by outflank-mailman (input) for mailman id 687200;
 Thu, 29 Feb 2024 15:28:17 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKX-0008Ql-4w
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:17 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 28499bfa-d717-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:28:15 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id D53774EE0752;
 Thu, 29 Feb 2024 16:28:14 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 28499bfa-d717-11ee-a1ee-f123f15fe8a2
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH 10/10] xen/keyhandler: address violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:28:02 +0100
Message-Id: <2bc4a964f0f2f47488e72237678e944dbdbd7bb7.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1709219010.git.nicola.vetrini@bugseng.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/common/keyhandler.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/common/keyhandler.c b/xen/common/keyhandler.c
index 127ca506965c..4c1ce007870f 100644
--- a/xen/common/keyhandler.c
+++ b/xen/common/keyhandler.c
@@ -42,10 +42,10 @@ static struct keyhandler {
 } key_table[128] __read_mostly =
 {
 #define KEYHANDLER(k, f, desc, diag)            \
-    [k] = { { .fn = (f) }, desc, 0, diag }
+    [k] = { { .fn = (f) }, (desc), 0, (diag) }
 
 #define IRQ_KEYHANDLER(k, f, desc, diag)        \
-    [k] = { { .irq_fn = (f) }, desc, 1, diag }
+    [k] = { { .irq_fn = (f) }, (desc), 1, (diag) }
 
     IRQ_KEYHANDLER('A', do_toggle_alt_key, "toggle alternative key handling", 0),
     IRQ_KEYHANDLER('d', dump_registers, "dump registers", 1),
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687192.1070230 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKS-0008Or-DT; Thu, 29 Feb 2024 15:28:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687192.1070230; Thu, 29 Feb 2024 15:28:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKS-0008Nf-9I; Thu, 29 Feb 2024 15:28:12 +0000
Received: by outflank-mailman (input) for mailman id 687192;
 Thu, 29 Feb 2024 15:28:11 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKR-00087e-0N
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:11 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 25227046-d717-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:28:10 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id 852CA4EE074B;
 Thu, 29 Feb 2024 16:28:09 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 25227046-d717-11ee-afd8-a90da7624cb6
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: [XEN PATCH 02/10] xen/arm: address some violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:27:54 +0100
Message-Id: <efdeb0e013c36e18b5545fbdb33a43bb3f87039c.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1709219010.git.nicola.vetrini@bugseng.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
Style in arm64/cpufeature.c has not been amended, because this file seems
to be kept in sync with its Linux counterpart.
---
 xen/arch/arm/arm64/cpufeature.c          | 14 +++++++-------
 xen/arch/arm/cpuerrata.c                 |  4 ++--
 xen/arch/arm/include/asm/arm64/sysregs.h |  2 +-
 xen/arch/arm/include/asm/guest_atomics.h |  4 ++--
 xen/arch/arm/include/asm/mm.h            |  2 +-
 xen/arch/arm/include/asm/smccc.h         |  8 ++++----
 xen/arch/arm/include/asm/vgic-emul.h     |  8 ++++----
 xen/arch/arm/vcpreg.c                    |  5 +++--
 8 files changed, 24 insertions(+), 23 deletions(-)

diff --git a/xen/arch/arm/arm64/cpufeature.c b/xen/arch/arm/arm64/cpufeature.c
index 864413d9cc03..6fb8974ade7f 100644
--- a/xen/arch/arm/arm64/cpufeature.c
+++ b/xen/arch/arm/arm64/cpufeature.c
@@ -78,13 +78,13 @@
 
 #define __ARM64_FTR_BITS(SIGNED, VISIBLE, STRICT, TYPE, SHIFT, WIDTH, SAFE_VAL) \
 	{						\
-		.sign = SIGNED,				\
-		.visible = VISIBLE,			\
-		.strict = STRICT,			\
-		.type = TYPE,				\
-		.shift = SHIFT,				\
-		.width = WIDTH,				\
-		.safe_val = SAFE_VAL,			\
+		.sign = (SIGNED),				\
+		.visible = (VISIBLE),			\
+		.strict = (STRICT),			\
+		.type = (TYPE),				\
+		.shift = (SHIFT),				\
+		.width = (WIDTH),				\
+		.safe_val = (SAFE_VAL),			\
 	}
 
 /* Define a feature with unsigned values */
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index a28fa6ac78cc..c678a555910f 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -462,8 +462,8 @@ static bool has_ssbd_mitigation(const struct arm_cpu_capabilities *entry)
 #define MIDR_RANGE(model, min, max)     \
     .matches = is_affected_midr_range,  \
     .midr_model = model,                \
-    .midr_range_min = min,              \
-    .midr_range_max = max
+    .midr_range_min = (min),            \
+    .midr_range_max = (max)
 
 #define MIDR_ALL_VERSIONS(model)        \
     .matches = is_affected_midr_range,  \
diff --git a/xen/arch/arm/include/asm/arm64/sysregs.h b/xen/arch/arm/include/asm/arm64/sysregs.h
index 3fdeb9d8cdef..b593e4028b53 100644
--- a/xen/arch/arm/include/asm/arm64/sysregs.h
+++ b/xen/arch/arm/include/asm/arm64/sysregs.h
@@ -465,7 +465,7 @@
 /* Access to system registers */
 
 #define WRITE_SYSREG64(v, name) do {                    \
-    uint64_t _r = v;                                    \
+    uint64_t _r = (v);                                  \
     asm volatile("msr "__stringify(name)", %0" : : "r" (_r));       \
 } while (0)
 #define READ_SYSREG64(name) ({                          \
diff --git a/xen/arch/arm/include/asm/guest_atomics.h b/xen/arch/arm/include/asm/guest_atomics.h
index a1745f8613f6..8893eb9a55d7 100644
--- a/xen/arch/arm/include/asm/guest_atomics.h
+++ b/xen/arch/arm/include/asm/guest_atomics.h
@@ -32,7 +32,7 @@ static inline void guest_##name(struct domain *d, int nr, volatile void *p) \
     perfc_incr(atomics_guest_paused);                                       \
                                                                             \
     domain_pause_nosync(d);                                                 \
-    name(nr, p);                                                            \
+    (name)(nr, p);                                                          \
     domain_unpause(d);                                                      \
 }
 
@@ -52,7 +52,7 @@ static inline int guest_##name(struct domain *d, int nr, volatile void *p)  \
     perfc_incr(atomics_guest_paused);                                       \
                                                                             \
     domain_pause_nosync(d);                                                 \
-    oldbit = name(nr, p);                                                   \
+    oldbit = (name)(nr, p);                                                 \
     domain_unpause(d);                                                      \
                                                                             \
     return oldbit;                                                          \
diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h
index cbcf3bf14767..48538b5337aa 100644
--- a/xen/arch/arm/include/asm/mm.h
+++ b/xen/arch/arm/include/asm/mm.h
@@ -250,7 +250,7 @@ static inline void __iomem *ioremap_wc(paddr_t start, size_t len)
 #define gaddr_to_gfn(ga)    _gfn(paddr_to_pfn(ga))
 #define mfn_to_maddr(mfn)   pfn_to_paddr(mfn_x(mfn))
 #define maddr_to_mfn(ma)    _mfn(paddr_to_pfn(ma))
-#define vmap_to_mfn(va)     maddr_to_mfn(virt_to_maddr((vaddr_t)va))
+#define vmap_to_mfn(va)     maddr_to_mfn(virt_to_maddr((vaddr_t)(va)))
 #define vmap_to_page(va)    mfn_to_page(vmap_to_mfn(va))
 
 /* Page-align address and convert to frame number format */
diff --git a/xen/arch/arm/include/asm/smccc.h b/xen/arch/arm/include/asm/smccc.h
index 1adcd37443c7..a1f309eea45a 100644
--- a/xen/arch/arm/include/asm/smccc.h
+++ b/xen/arch/arm/include/asm/smccc.h
@@ -122,7 +122,7 @@ struct arm_smccc_res {
 #define __constraint_read_7 __constraint_read_6, "r" (r7)
 
 #define __declare_arg_0(a0, res)                            \
-    struct arm_smccc_res    *___res = res;                  \
+    struct arm_smccc_res    *___res = (res);                \
     register unsigned long  r0 ASM_REG(0) = (uint32_t)a0;   \
     register unsigned long  r1 ASM_REG(1);                  \
     register unsigned long  r2 ASM_REG(2);                  \
@@ -130,7 +130,7 @@ struct arm_smccc_res {
 
 #define __declare_arg_1(a0, a1, res)                        \
     typeof(a1) __a1 = a1;                                   \
-    struct arm_smccc_res    *___res = res;                  \
+    struct arm_smccc_res    *___res = (res);                \
     register unsigned long  r0 ASM_REG(0) = (uint32_t)a0;   \
     register unsigned long  r1 ASM_REG(1) = __a1;           \
     register unsigned long  r2 ASM_REG(2);                  \
@@ -139,7 +139,7 @@ struct arm_smccc_res {
 #define __declare_arg_2(a0, a1, a2, res)                    \
     typeof(a1) __a1 = a1;                                   \
     typeof(a2) __a2 = a2;                                   \
-    struct arm_smccc_res    *___res = res;				    \
+    struct arm_smccc_res    *___res = (res);                \
     register unsigned long  r0 ASM_REG(0) = (uint32_t)a0;   \
     register unsigned long  r1 ASM_REG(1) = __a1;           \
     register unsigned long  r2 ASM_REG(2) = __a2;           \
@@ -149,7 +149,7 @@ struct arm_smccc_res {
     typeof(a1) __a1 = a1;                                   \
     typeof(a2) __a2 = a2;                                   \
     typeof(a3) __a3 = a3;                                   \
-    struct arm_smccc_res    *___res = res;                  \
+    struct arm_smccc_res    *___res = (res);                \
     register unsigned long  r0 ASM_REG(0) = (uint32_t)a0;   \
     register unsigned long  r1 ASM_REG(1) = __a1;           \
     register unsigned long  r2 ASM_REG(2) = __a2;           \
diff --git a/xen/arch/arm/include/asm/vgic-emul.h b/xen/arch/arm/include/asm/vgic-emul.h
index e52fbaa3ec04..e2afb52498a8 100644
--- a/xen/arch/arm/include/asm/vgic-emul.h
+++ b/xen/arch/arm/include/asm/vgic-emul.h
@@ -6,11 +6,11 @@
  * a range of registers
  */
 
-#define VREG32(reg) reg ... reg + 3
-#define VREG64(reg) reg ... reg + 7
+#define VREG32(reg) (reg) ... (reg) + 3
+#define VREG64(reg) (reg) ... (reg) + 7
 
-#define VRANGE32(start, end) start ... end + 3
-#define VRANGE64(start, end) start ... end + 7
+#define VRANGE32(start, end) (start) ... (end) + 3
+#define VRANGE64(start, end) (start) ... (end) + 7
 
 /*
  * 64 bits registers can be accessible using 32-bit and 64-bit unless
diff --git a/xen/arch/arm/vcpreg.c b/xen/arch/arm/vcpreg.c
index a2d050070473..019cf34f003a 100644
--- a/xen/arch/arm/vcpreg.c
+++ b/xen/arch/arm/vcpreg.c
@@ -39,7 +39,8 @@
  */
 
 #ifdef CONFIG_ARM_64
-#define WRITE_SYSREG_SZ(sz, val, sysreg) WRITE_SYSREG((uint##sz##_t)val, sysreg)
+#define WRITE_SYSREG_SZ(sz, val, sysreg) \
+    WRITE_SYSREG((uint##sz##_t)(val), sysreg)
 #else
 /*
  * WRITE_SYSREG{32/64} on arm32 is defined as variadic macro which imposes
@@ -64,7 +65,7 @@ static bool func(struct cpu_user_regs *regs, type##sz##_t *r, bool read)    \
     bool cache_enabled = vcpu_has_cache_enabled(v);                         \
                                                                             \
     GUEST_BUG_ON(read);                                                     \
-    WRITE_SYSREG_SZ(sz, *r, reg);                                           \
+    WRITE_SYSREG_SZ(sz, *(r), reg);                                         \
                                                                             \
     p2m_toggle_cache(v, cache_enabled);                                     \
                                                                             \
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687191.1070226 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKS-0008MB-5V; Thu, 29 Feb 2024 15:28:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687191.1070226; Thu, 29 Feb 2024 15:28:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKS-0008M4-2J; Thu, 29 Feb 2024 15:28:12 +0000
Received: by outflank-mailman (input) for mailman id 687191;
 Thu, 29 Feb 2024 15:28:10 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKQ-00087e-9P
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:10 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 24a431a6-d717-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:28:09 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id ABBBC4EE074A;
 Thu, 29 Feb 2024 16:28:07 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 24a431a6-d717-11ee-afd8-a90da7624cb6
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH 01/10] xen/include: address violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:27:53 +0100
Message-Id: <15d6e4fb5c873e7ea42cfcee2faa0bf33c10d101.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1709219010.git.nicola.vetrini@bugseng.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/include/xen/bug.h      |  2 +-
 xen/include/xen/init.h     |  4 +--
 xen/include/xen/kconfig.h  |  2 +-
 xen/include/xen/list.h     | 59 +++++++++++++++++++-------------------
 xen/include/xen/param.h    | 22 +++++++-------
 xen/include/xen/spinlock.h |  2 +-
 6 files changed, 45 insertions(+), 46 deletions(-)

diff --git a/xen/include/xen/bug.h b/xen/include/xen/bug.h
index 2c45c462fc63..77fe1e1ba840 100644
--- a/xen/include/xen/bug.h
+++ b/xen/include/xen/bug.h
@@ -80,7 +80,7 @@ struct bug_frame {
     [bf_type]    "i" (type),                                                 \
     [bf_ptr]     "i" (ptr),                                                  \
     [bf_msg]     "i" (msg),                                                  \
-    [bf_line_lo] "i" ((line & ((1 << BUG_LINE_LO_WIDTH) - 1))                \
+    [bf_line_lo] "i" (((line) & ((1 << BUG_LINE_LO_WIDTH) - 1))              \
                       << BUG_DISP_WIDTH),                                    \
     [bf_line_hi] "i" (((line) >> BUG_LINE_LO_WIDTH) << BUG_DISP_WIDTH)
 
diff --git a/xen/include/xen/init.h b/xen/include/xen/init.h
index 1d7c0216bc80..0a4223833755 100644
--- a/xen/include/xen/init.h
+++ b/xen/include/xen/init.h
@@ -63,9 +63,9 @@ typedef int (*initcall_t)(void);
 typedef void (*exitcall_t)(void);
 
 #define presmp_initcall(fn) \
-    const static initcall_t __initcall_##fn __init_call("presmp") = fn
+    const static initcall_t __initcall_##fn __init_call("presmp") = (fn)
 #define __initcall(fn) \
-    const static initcall_t __initcall_##fn __init_call("1") = fn
+    const static initcall_t __initcall_##fn __init_call("1") = (fn)
 #define __exitcall(fn) \
     static exitcall_t __exitcall_##fn __exit_call = fn
 
diff --git a/xen/include/xen/kconfig.h b/xen/include/xen/kconfig.h
index c25dc0f6c2a9..b7e70289737b 100644
--- a/xen/include/xen/kconfig.h
+++ b/xen/include/xen/kconfig.h
@@ -25,7 +25,7 @@
 #define __ARG_PLACEHOLDER_1 0,
 #define config_enabled(cfg) _config_enabled(cfg)
 #define _config_enabled(value) __config_enabled(__ARG_PLACEHOLDER_##value)
-#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk 1, 0)
+#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk (1), (0))
 #define ___config_enabled(__ignored, val, ...) val
 
 /*
diff --git a/xen/include/xen/list.h b/xen/include/xen/list.h
index b5eab3a1eb6c..d803e7848cad 100644
--- a/xen/include/xen/list.h
+++ b/xen/include/xen/list.h
@@ -490,9 +490,9 @@ static inline void list_splice_init(struct list_head *list,
  * @member: the name of the list_struct within the struct.
  */
 #define list_for_each_entry(pos, head, member)                          \
-    for (pos = list_entry((head)->next, typeof(*pos), member);          \
-         &pos->member != (head);                                        \
-         pos = list_entry(pos->member.next, typeof(*pos), member))
+    for (pos = list_entry((head)->next, typeof(*(pos)), member);          \
+         &(pos)->member != (head);                                      \
+         pos = list_entry((pos)->member.next, typeof(*(pos)), member))
 
 /**
  * list_for_each_entry_reverse - iterate backwards over list of given type.
@@ -501,9 +501,9 @@ static inline void list_splice_init(struct list_head *list,
  * @member: the name of the list_struct within the struct.
  */
 #define list_for_each_entry_reverse(pos, head, member)                  \
-    for (pos = list_entry((head)->prev, typeof(*pos), member);          \
-         &pos->member != (head);                                        \
-         pos = list_entry(pos->member.prev, typeof(*pos), member))
+    for (pos = list_entry((head)->prev, typeof(*(pos)), member);          \
+         &(pos)->member != (head);                                      \
+         pos = list_entry((pos)->member.prev, typeof(*(pos)), member))
 
 /**
  * list_prepare_entry - prepare a pos entry for use in
@@ -516,7 +516,7 @@ static inline void list_splice_init(struct list_head *list,
  * list_for_each_entry_continue.
  */
 #define list_prepare_entry(pos, head, member)           \
-    ((pos) ? : list_entry(head, typeof(*pos), member))
+    ((pos) ? : list_entry(head, typeof(*(pos)), member))
 
 /**
  * list_for_each_entry_continue - continue iteration over list of given type
@@ -528,9 +528,9 @@ static inline void list_splice_init(struct list_head *list,
  * the current position.
  */
 #define list_for_each_entry_continue(pos, head, member)                 \
-    for (pos = list_entry(pos->member.next, typeof(*pos), member);      \
-         &pos->member != (head);                                        \
-         pos = list_entry(pos->member.next, typeof(*pos), member))
+    for (pos = list_entry((pos)->member.next, typeof(*(pos)), member);  \
+         &(pos)->member != (head);                                      \
+         pos = list_entry((pos)->member.next, typeof(*(pos)), member))
 
 /**
  * list_for_each_entry_from - iterate over list of given type from the
@@ -542,8 +542,8 @@ static inline void list_splice_init(struct list_head *list,
  * Iterate over list of given type, continuing from current position.
  */
 #define list_for_each_entry_from(pos, head, member)                     \
-    for (; &pos->member != (head);                                      \
-         pos = list_entry(pos->member.next, typeof(*pos), member))
+    for (; &(pos)->member != (head);                                    \
+         pos = list_entry((pos)->member.next, typeof(*(pos)), member))
 
 /**
  * list_for_each_entry_safe - iterate over list of given type safe
@@ -554,10 +554,10 @@ static inline void list_splice_init(struct list_head *list,
  * @member: the name of the list_struct within the struct.
  */
 #define list_for_each_entry_safe(pos, n, head, member)                  \
-    for (pos = list_entry((head)->next, typeof(*pos), member),          \
-         n = list_entry(pos->member.next, typeof(*pos), member);        \
-         &pos->member != (head);                                        \
-         pos = n, n = list_entry(n->member.next, typeof(*n), member))
+    for (pos = list_entry((head)->next, typeof(*(pos)), member),        \
+         n = list_entry((pos)->member.next, typeof(*(pos)), member);    \
+         &(pos)->member != (head);                                      \
+         pos = (n), n = list_entry((n)->member.next, typeof(*(n)), member))
 
 /**
  * list_for_each_entry_safe_continue
@@ -570,10 +570,10 @@ static inline void list_splice_init(struct list_head *list,
  * safe against removal of list entry.
  */
 #define list_for_each_entry_safe_continue(pos, n, head, member)         \
-    for (pos = list_entry(pos->member.next, typeof(*pos), member),      \
-         n = list_entry(pos->member.next, typeof(*pos), member);        \
-         &pos->member != (head);                                        \
-         pos = n, n = list_entry(n->member.next, typeof(*n), member))
+    for (pos = list_entry((pos)->member.next, typeof(*(pos)), member),  \
+         n = list_entry((pos)->member.next, typeof(*(pos)), member);    \
+         &(pos)->member != (head);                                      \
+         pos = (n), n = list_entry((n)->member.next, typeof(*(n)), member))
 
 /**
  * list_for_each_entry_safe_from
@@ -586,9 +586,9 @@ static inline void list_splice_init(struct list_head *list,
  * removal of list entry.
  */
 #define list_for_each_entry_safe_from(pos, n, head, member)             \
-    for (n = list_entry(pos->member.next, typeof(*pos), member);        \
-         &pos->member != (head);                                        \
-         pos = n, n = list_entry(n->member.next, typeof(*n), member))
+    for (n = list_entry((pos)->member.next, typeof(*(pos)), member);    \
+         &(pos)->member != (head);                                      \
+         pos = (n), n = list_entry((n)->member.next, typeof(*(n)), member))
 
 /**
  * list_for_each_entry_safe_reverse
@@ -601,10 +601,10 @@ static inline void list_splice_init(struct list_head *list,
  * of list entry.
  */
 #define list_for_each_entry_safe_reverse(pos, n, head, member)          \
-    for (pos = list_entry((head)->prev, typeof(*pos), member),          \
-         n = list_entry(pos->member.prev, typeof(*pos), member);        \
-         &pos->member != (head);                                        \
-         pos = n, n = list_entry(n->member.prev, typeof(*n), member))
+    for (pos = list_entry((head)->prev, typeof(*(pos)), member),        \
+         n = list_entry((pos)->member.prev, typeof(*(pos)), member);    \
+         &(pos)->member != (head);                                      \
+         pos = (n), n = list_entry((n)->member.prev, typeof(*(n)), member))
 
 /**
  * list_for_each_rcu - iterate over an rcu-protected list
@@ -653,9 +653,9 @@ static inline void list_splice_init(struct list_head *list,
  * as long as the traversal is guarded by rcu_read_lock().
  */
 #define list_for_each_entry_rcu(pos, head, member)                      \
-    for (pos = list_entry((head)->next, typeof(*pos), member);          \
+    for (pos = list_entry((head)->next, typeof(*(pos)), member);        \
          &rcu_dereference(pos)->member != (head);                       \
-         pos = list_entry(pos->member.next, typeof(*pos), member))
+         pos = list_entry((pos)->member.next, typeof(*(pos)), member))
 
 /**
  * list_for_each_continue_rcu
@@ -977,4 +977,3 @@ static inline void hlist_add_after_rcu(struct hlist_node *prev,
           pos = pos->next)
 
 #endif /* __XEN_LIST_H__ */
-
diff --git a/xen/include/xen/param.h b/xen/include/xen/param.h
index 13607e0e50e0..1bdbab34ab1f 100644
--- a/xen/include/xen/param.h
+++ b/xen/include/xen/param.h
@@ -45,42 +45,42 @@ extern const struct kernel_param __setup_start[], __setup_end[];
 #define TEMP_NAME(base) _TEMP_NAME(base, __LINE__)
 
 #define custom_param(_name, _var) \
-    __setup_str __setup_str_##_var[] = _name; \
+    __setup_str __setup_str_##_var[] = (_name); \
     __kparam __setup_##_var = \
         { .name = __setup_str_##_var, \
           .type = OPT_CUSTOM, \
-          .par.func = _var }
+          .par.func = (_var) }
 #define boolean_param(_name, _var) \
-    __setup_str __setup_str_##_var[] = _name; \
+    __setup_str __setup_str_##_var[] = (_name); \
     __kparam __setup_##_var = \
         { .name = __setup_str_##_var, \
           .type = OPT_BOOL, \
           .len = sizeof(_var) + \
                  BUILD_BUG_ON_ZERO(sizeof(_var) != sizeof(bool)), \
-          .par.var = &_var }
+          .par.var = &(_var) }
 #define integer_param(_name, _var) \
-    __setup_str __setup_str_##_var[] = _name; \
+    __setup_str __setup_str_##_var[] = (_name); \
     __kparam __setup_##_var = \
         { .name = __setup_str_##_var, \
           .type = OPT_UINT, \
           .len = sizeof(_var), \
-          .par.var = &_var }
+          .par.var = &(_var) }
 #define size_param(_name, _var) \
-    __setup_str __setup_str_##_var[] = _name; \
+    __setup_str __setup_str_##_var[] = (_name); \
     __kparam __setup_##_var = \
         { .name = __setup_str_##_var, \
           .type = OPT_SIZE, \
           .len = sizeof(_var), \
-          .par.var = &_var }
+          .par.var = &(_var) }
 #define string_param(_name, _var) \
-    __setup_str __setup_str_##_var[] = _name; \
+    __setup_str __setup_str_##_var[] = (_name); \
     __kparam __setup_##_var = \
         { .name = __setup_str_##_var, \
           .type = OPT_STR, \
           .len = sizeof(_var), \
-          .par.var = &_var }
+          .par.var = &(_var) }
 #define ignore_param(_name)                 \
-    __setup_str TEMP_NAME(__setup_str_ign)[] = _name;    \
+    __setup_str TEMP_NAME(__setup_str_ign)[] = (_name);  \
     __kparam TEMP_NAME(__setup_ign) =                    \
         { .name = TEMP_NAME(__setup_str_ign),            \
           .type = OPT_IGNORE }
diff --git a/xen/include/xen/spinlock.h b/xen/include/xen/spinlock.h
index 1cd9120eac7a..0e6a083dfb9e 100644
--- a/xen/include/xen/spinlock.h
+++ b/xen/include/xen/spinlock.h
@@ -94,7 +94,7 @@ struct lock_profile_qhead {
     int32_t                   idx;     /* index for printout */
 };
 
-#define _LOCK_PROFILE(lockname) { .name = #lockname, .lock = &lockname, }
+#define _LOCK_PROFILE(lockname) { .name = #lockname, .lock = &(lockname), }
 #define _LOCK_PROFILE_PTR(name)                                               \
     static struct lock_profile * const __lock_profile_##name                  \
     __used_section(".lockprofile.data") =                                     \
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687193.1070246 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKT-0000Ob-NY; Thu, 29 Feb 2024 15:28:13 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687193.1070246; Thu, 29 Feb 2024 15:28:13 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKT-0000OP-KM; Thu, 29 Feb 2024 15:28:13 +0000
Received: by outflank-mailman (input) for mailman id 687193;
 Thu, 29 Feb 2024 15:28:12 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKS-00087e-Nk
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:12 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 26432195-d717-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:28:12 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id 7886A4EE074E;
 Thu, 29 Feb 2024 16:28:11 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 26432195-d717-11ee-afd8-a90da7624cb6
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [XEN PATCH 05/10] xen/perfc: address violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:27:57 +0100
Message-Id: <3418c09bb8c4ceeceb66334191adfad190954f56.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <cover.1709219010.git.nicola.vetrini@bugseng.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.

No functional change.

Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
---
 xen/common/perfc.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/common/perfc.c b/xen/common/perfc.c
index 7400667bf0c4..02f4fc8fe032 100644
--- a/xen/common/perfc.c
+++ b/xen/common/perfc.c
@@ -10,10 +10,10 @@
 #include <public/sysctl.h>
 #include <asm/perfc.h>
 
-#define PERFCOUNTER( var, name )              { name, TYPE_SINGLE, 0 },
-#define PERFCOUNTER_ARRAY( var, name, size )  { name, TYPE_ARRAY,  size },
-#define PERFSTATUS( var, name )               { name, TYPE_S_SINGLE, 0 },
-#define PERFSTATUS_ARRAY( var, name, size )   { name, TYPE_S_ARRAY,  size },
+#define PERFCOUNTER( var, name )              { (name), TYPE_SINGLE, 0 },
+#define PERFCOUNTER_ARRAY( var, name, size )  { (name), TYPE_ARRAY,  (size) },
+#define PERFSTATUS( var, name )               { (name), TYPE_S_SINGLE, 0 },
+#define PERFSTATUS_ARRAY( var, name, size )   { (name), TYPE_S_ARRAY,  (size) },
 static const struct {
     const char *name;
     enum { TYPE_SINGLE, TYPE_ARRAY,
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:28:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:28:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687190.1070216 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKQ-00087w-V5; Thu, 29 Feb 2024 15:28:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687190.1070216; Thu, 29 Feb 2024 15:28:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiKQ-00087p-Rt; Thu, 29 Feb 2024 15:28:10 +0000
Received: by outflank-mailman (input) for mailman id 687190;
 Thu, 29 Feb 2024 15:28:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfiKP-00087e-2B
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:28:09 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 239fe54f-d717-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:28:07 +0100 (CET)
Received: from nico.bugseng.com (unknown [46.228.253.196])
 by support.bugseng.com (Postfix) with ESMTPSA id 87D254EE0737;
 Thu, 29 Feb 2024 16:28:05 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 239fe54f-d717-11ee-afd8-a90da7624cb6
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: nicola.vetrini@bugseng.com,
	xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org,
	michal.orzel@amd.com,
	xenia.ragiadakou@amd.com,
	ayan.kumar.halder@amd.com,
	consulting@bugseng.com,
	jbeulich@suse.com,
	andrew.cooper3@citrix.com,
	roger.pau@citrix.com,
	bertrand.marquis@arm.com,
	julien@xen.org,
	George Dunlap <george.dunlap@citrix.com>,
	Wei Liu <wl@xen.org>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Rahul Singh <rahul.singh@arm.com>
Subject: [XEN PATCH 00/10] address some violations of MISRA C Rule 20.7
Date: Thu, 29 Feb 2024 16:27:52 +0100
Message-Id: <cover.1709219010.git.nicola.vetrini@bugseng.com>
X-Mailer: git-send-email 2.34.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Hi all,

this series aims to refactor some macros that cause violations of MISRA C Rule
20.7 ("Expressions resulting from the expansion of macro parameters shall be
enclosed in parentheses"). All the macros touched by these patches are in some
way involved in violations, and the strategy adopted to bring them into
compliance is to add parentheses around macro arguments where needed.

Given that the community has previously requested a deviation from the rule, as
stated in docs/misra/deviations.rst, and reported below for convenience [1],
some macro parameters do not need any adjusting (e.g., when used as lhs to
an assignment in statement expressions).

This series fixes a significant portion of the violations on Arm
(from ~14000 to ~2500). On x86, though there is one patch touching it, there are
still many more; they will be part of a later series.

[1] - Code violating Rule 20.7 is safe when macro parameters are used:
       (1) as function arguments;
       (2) as macro arguments;
       (3) as array indices;
       (4) as lhs in assignments.

Nicola Vetrini (10):
  xen/include: address violations of MISRA C Rule 20.7
  xen/arm: address some violations of MISRA C Rule 20.7
  x86: address some violations of MISRA C Rule 20.7
  xen/public: address violations of MISRA C Rule 20.7
  xen/perfc: address violations of MISRA C Rule 20.7
  arm/smmu: address some violations of MISRA C Rule 20.7
  xen/arm: smmuv3: address violations of MISRA C Rule 20.7
  xen/errno: address violations of MISRA C Rule 20.7
  xen/include: tasklet: address violations of MISRA C Rule 20.7
  xen/keyhandler: address violations of MISRA C Rule 20.7

 xen/arch/arm/arm64/cpufeature.c          | 14 +++---
 xen/arch/arm/cpuerrata.c                 |  4 +-
 xen/arch/arm/include/asm/arm64/sysregs.h |  2 +-
 xen/arch/arm/include/asm/guest_atomics.h |  4 +-
 xen/arch/arm/include/asm/mm.h            |  2 +-
 xen/arch/arm/include/asm/smccc.h         |  8 ++--
 xen/arch/arm/include/asm/vgic-emul.h     |  8 ++--
 xen/arch/arm/vcpreg.c                    |  5 +-
 xen/arch/x86/include/asm/irq.h           |  6 +--
 xen/arch/x86/usercopy.c                  |  2 +-
 xen/common/keyhandler.c                  |  4 +-
 xen/common/perfc.c                       |  8 ++--
 xen/drivers/passthrough/arm/smmu-v3.c    |  2 +-
 xen/drivers/passthrough/arm/smmu.c       |  4 +-
 xen/include/public/xen.h                 |  2 +-
 xen/include/xen/bug.h                    |  2 +-
 xen/include/xen/errno.h                  |  2 +-
 xen/include/xen/init.h                   |  4 +-
 xen/include/xen/kconfig.h                |  2 +-
 xen/include/xen/list.h                   | 59 ++++++++++++------------
 xen/include/xen/param.h                  | 22 ++++-----
 xen/include/xen/spinlock.h               |  2 +-
 xen/include/xen/tasklet.h                |  2 +-
 23 files changed, 85 insertions(+), 85 deletions(-)

-- 
2.34.1


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:32:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:32:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687216.1070327 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiOT-0006p6-0W; Thu, 29 Feb 2024 15:32:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687216.1070327; Thu, 29 Feb 2024 15:32:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiOS-0006oz-SW; Thu, 29 Feb 2024 15:32:20 +0000
Received: by outflank-mailman (input) for mailman id 687216;
 Thu, 29 Feb 2024 15:32:19 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfiOR-0006ot-KF
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:32:19 +0000
Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com
 [2a00:1450:4864:20::632])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b908e2de-d717-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:32:18 +0100 (CET)
Received: by mail-ej1-x632.google.com with SMTP id
 a640c23a62f3a-a3122b70439so191127266b.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 07:32:18 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 tj7-20020a170907c24700b00a413d1eda4bsm789972ejc.87.2024.02.29.07.32.17
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 07:32:17 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b908e2de-d717-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709220738; x=1709825538; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=xrC7QPOJE9dPLFEXGY/QD3AwNupVDWNAcYL77MydVgI=;
        b=Ej4+ridFvusv1lWXL8rPuXY0FBrdNNl7AJW1kzbbo9D7Ld49zRu8luvcTSjqnc001t
         cq/zUvhhm8YcB/zFlyrmFLF/8vxeouSjLaq5V3e0hOyNzYLPXP3jsxR+fW0d3jKlskfT
         vrmrCsQVuPPhnV1PZJU3zPS3I+RqOm0Nshbe9pwtwl/MFdJ93cGC+Lq24FJMikL9wF2X
         O2Kv0so8PM40NSY4SQ130lDdmZQWO2HXsTvB+uEuX3PH73cjsE5wwIktFccd4uOWUDRc
         Fr4/wfMaZ1PQuunCCIy1hcwP8j4Az7YDMmunMXetHpBx2c5vjVOjFjiex/Li5eqxkd2E
         hP/Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709220738; x=1709825538;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=xrC7QPOJE9dPLFEXGY/QD3AwNupVDWNAcYL77MydVgI=;
        b=izpx3yndlV90SUg8i3CcPR8LL61p3f6ai+i/I6ljGS+Isdr4SvEO2uIpqlCYd95lJq
         Ok1E5l9mcxvRp9IIDUOF2UPKmxJ4qYeAZL4ONMFvQAZE2DJndCf/h3QbujTD/BzQ0dvn
         7driV+M/NCcQbxN0+XnE/4EujjreTUGXIbXEYT0+iSba2hKunONA7BpQYdRpz76Gd/T9
         3zQ4rb67izolwJoNhdeJa/iysIyNI+d3tsraylt+AH+4UWwi7kjQwZVmx+g3HydzH+uS
         mdWH7zoTwq6kK3Xcz/fBW/IYdUDk/0idiASi3ZHWIHV4IHYeVBHhMo/Om3nHCQVgZ5XW
         IESA==
X-Forwarded-Encrypted: i=1; AJvYcCVFZMxlKWpzzV86uUwNSmaN1AfA5mo94s8IgnOWbdpwIyZZpq35ObGx59i6i3x1meglXIyj7SD27h+9a9D+XO/fT3mtV5MCgF74y8GUTJk=
X-Gm-Message-State: AOJu0YwNUveRBsuSg4MUiX6eFpGdo5X9AU+9+jYUWF6njs75x9I6wEvK
	wvZmWvXM7kOZmudSFwih5pb1OAiXW3Q9t8ounNStohPnUuhQDkZZ7YalmCqCww==
X-Google-Smtp-Source: AGHT+IGARFwoa5UH0HPSKNdPrSOO4KJU7cEPAU/hvgniIIUms646HVf396WwLqeEeunP1DRnH/pHiA==
X-Received: by 2002:a17:906:d8b4:b0:a42:615:1395 with SMTP id qc20-20020a170906d8b400b00a4206151395mr1930469ejb.11.1709220737946;
        Thu, 29 Feb 2024 07:32:17 -0800 (PST)
Message-ID: <a037facf-c4d9-491f-b39f-56ed4221abf1@suse.com>
Date: Thu, 29 Feb 2024 16:32:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 10/12] xen/spinlock: split recursive spinlocks from
 normal ones
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-11-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20231212094725.22184-11-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 12.12.2023 10:47, Juergen Gross wrote:
> --- a/xen/common/spinlock.c
> +++ b/xen/common/spinlock.c
> @@ -541,6 +541,55 @@ void rspin_unlock_irqrestore(rspinlock_t *lock, unsigned long flags)
>      local_irq_restore(flags);
>  }
>  
> +int nrspin_trylock(rspinlock_t *lock)
> +{
> +    check_lock(&lock->debug, true);
> +
> +    if ( unlikely(lock->recurse_cpu != SPINLOCK_NO_CPU) )
> +        return 0;
> +
> +    return spin_trylock_common(&lock->tickets, &lock->debug, LOCK_PROFILE_PAR);
> +}

I wonder if we shouldn't take the opportunity and stop having trylock
functions have "int" return type. Perhaps already spin_trylock_common()
when introduced could use "bool" instead, then here following suit.

> +void nrspin_lock(rspinlock_t *lock)
> +{
> +    spin_lock_common(&lock->tickets, &lock->debug, LOCK_PROFILE_PAR, NULL,
> +                     NULL);
> +}
> +
> +void nrspin_unlock(rspinlock_t *lock)
> +{
> +    spin_unlock_common(&lock->tickets, &lock->debug, LOCK_PROFILE_PAR);
> +}
> +
> +void nrspin_lock_irq(rspinlock_t *lock)
> +{
> +    ASSERT(local_irq_is_enabled());
> +    local_irq_disable();
> +    nrspin_lock(lock);
> +}
> +
> +void nrspin_unlock_irq(rspinlock_t *lock)
> +{
> +    nrspin_unlock(lock);
> +    local_irq_enable();
> +}
> +
> +unsigned long __nrspin_lock_irqsave(rspinlock_t *lock)
> +{
> +    unsigned long flags;
> +
> +    local_irq_save(flags);
> +    nrspin_lock(lock);
> +    return flags;

Nit: Strictly speaking we want a blank line ahead of this "return".

> @@ -166,11 +172,15 @@ struct lock_profile_qhead { };
>  struct lock_profile { };
>  
>  #define SPIN_LOCK_UNLOCKED {                                                  \
> +    .debug =_LOCK_DEBUG,                                                      \
> +}
> +#define RSPIN_LOCK_UNLOCKED {                                                 \
> +    .debug =_LOCK_DEBUG,                                                      \
>      .recurse_cpu = SPINLOCK_NO_CPU,                                           \
>      .debug =_LOCK_DEBUG,                                                      \
>  }

Initializing .debug twice?

> @@ -180,7 +190,6 @@ struct lock_profile { };
>  
>  #endif
>  
> -
>  typedef union {
>      uint32_t head_tail;
>      struct {

Looks like this might be undoing what the earlier patch isn't going to
do anymore?

> @@ -242,6 +257,19 @@ void rspin_unlock_irqrestore(rspinlock_t *lock, unsigned long flags);
>  int rspin_is_locked(const rspinlock_t *lock);
>  void rspin_barrier(rspinlock_t *lock);
>  
> +int nrspin_trylock(rspinlock_t *lock);
> +void nrspin_lock(rspinlock_t *lock);
> +void nrspin_unlock(rspinlock_t *lock);
> +void nrspin_lock_irq(rspinlock_t *lock);
> +void nrspin_unlock_irq(rspinlock_t *lock);
> +#define nrspin_lock_irqsave(l, f)                               \
> +    ({                                                          \
> +        BUILD_BUG_ON(sizeof(f) != sizeof(unsigned long));       \
> +        ((f) = __nrspin_lock_irqsave(l));                       \

I don't think the outer pair of parentheses is needed here. As to the
leading double underscores, see comments elsewhere.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:35:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:35:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687219.1070336 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiRN-0007Rn-Ft; Thu, 29 Feb 2024 15:35:21 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687219.1070336; Thu, 29 Feb 2024 15:35:21 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiRN-0007Rg-D8; Thu, 29 Feb 2024 15:35:21 +0000
Received: by outflank-mailman (input) for mailman id 687219;
 Thu, 29 Feb 2024 15:35:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfiRM-0007Ra-Lz
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:35:20 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 250e24df-d718-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:35:19 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a44628725e3so54629666b.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 07:35:19 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 vb9-20020a170907d04900b00a4446655ceasm682001ejc.110.2024.02.29.07.35.18
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 07:35:18 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 250e24df-d718-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709220919; x=1709825719; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MDGlgUYloOERxBLTOm/aU3tqCwar96Q0mrxOXuvrUGo=;
        b=E1TbkBBgMhi3EyDjEBIGurWgZ7T3suHp4xloIb3lOVi8o22SDAtMBrqWfNbfYV4L82
         s9grj54F3M8HZ842yBaeFP/kAI10aX2JHpAlQGosp595NGmol3svwhLn1+Nfs47wdeRM
         6aaNktoXeG/YZMvA/Zgkn6V1OS5ZHR1OrAzAVHd/4Wq8fMkfdwvj1ZItHMrZ29ybynf4
         LbDdj5xviCiSlYmv2bjPGPj/2xyu+3jFBJRzw5oBvozusDF0CKNOuqeml5FgN56q+O0K
         iR4Woq3fr1w6LqHex35fqoQpJFd5W4TNuGaOmy9b73LVuSMP2D1qoyG4p8vtpL36ur+0
         8qyw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709220919; x=1709825719;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=MDGlgUYloOERxBLTOm/aU3tqCwar96Q0mrxOXuvrUGo=;
        b=WOPzY5BK1Vo35Ix91CSt5BVesEoBLmatJJ9JpPYWbwY50QRCPi+DDsQnHHhWzRqF1b
         MMXIOCOeG++5Ao3xEYBtPL7K6S0eywPhEg+vdt5X9Ao0ekU50SWGlV/ZyzMxsPFnSjWF
         Kd/4AmRwB0gMwYRAoHOkWt9sQCZVsL99kGPJEmMRqOv8lDsq1mpJS2Rt4BH/RB9EAIHl
         2Tn76e1Qp8pPaS5xy/iMA4QXDcZkBAaCqc2FwZLL8QG6n1BwXIFvDps6P/1zuqQAxLoB
         Oa4DKhkvTnP6y6glu3jCYsrK19ugS5TCgkg1D+Y0WHzGT58PKEjxbcjgEaW1sNrtOcSY
         767A==
X-Forwarded-Encrypted: i=1; AJvYcCXBwEWYGDZd2xdEllFHSbRi6LdTNSgodrZtYIRLWbkaya+ZXCebOrbUQ0WuxNziXYPK1BnaYumlf+x6HTZXpHs3REKNFU0Eh3gYM9l6Xpk=
X-Gm-Message-State: AOJu0YxGTUEH7gF9LxdM8RC82hOsjWKA+0w1eIll5C8SZXMNHEVIkcK7
	AYsoHsXpnzXBXgljalZqq1RH8RLtxsL5l7or3MmuuRFRs+3Gl0XjiEgigeHc5w==
X-Google-Smtp-Source: AGHT+IGSl9a1D1i2NsuYJEoevvYwoVL1zN+fJ70zZOS/ljAtRtiVXgp9mjvV8jMoTDVXRltjmIw/Ug==
X-Received: by 2002:a17:906:3bd2:b0:a41:3e39:b918 with SMTP id v18-20020a1709063bd200b00a413e39b918mr1811293ejf.24.1709220919196;
        Thu, 29 Feb 2024 07:35:19 -0800 (PST)
Message-ID: <d7a49ea9-cb32-4d7b-b093-a86dda43333d@suse.com>
Date: Thu, 29 Feb 2024 16:35:17 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 11/12] xen/spinlock: remove indirection through macros
 for spin_*() functions
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-12-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20231212094725.22184-12-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 12.12.2023 10:47, Juergen Gross wrote:
> In reality all spin_*() functions are macros which are defined to just
> call a related real function.
> 
> Remove this macro layer, as it is adding complexity without any gain.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
with the same remark as on patch 04.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:45:27 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:45:27 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687231.1070345 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiaz-0000wh-BX; Thu, 29 Feb 2024 15:45:17 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687231.1070345; Thu, 29 Feb 2024 15:45:17 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiaz-0000wa-8n; Thu, 29 Feb 2024 15:45:17 +0000
Received: by outflank-mailman (input) for mailman id 687231;
 Thu, 29 Feb 2024 15:45:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rfiay-0000wS-9t
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:45:16 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 87bdf8c1-d719-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:45:15 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 1E03C1F7E9;
 Thu, 29 Feb 2024 15:45:14 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D7A8613503;
 Thu, 29 Feb 2024 15:45:13 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id Tx85M4mm4GX+MwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 15:45:13 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 87bdf8c1-d719-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709221514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Z8SfY5jVnFw4kZflsv6bRIFUWCtymoR6L22v1i1XNxM=;
	b=hmZjsJa2y8K2UkiwOz92Zu7CVi9X7WL89xGgl+jg3c0s+0b2xXzDa8uOCB2Y5tFywnME6/
	3e9NDIsppA7G4XYQNrh5lUESO1bzuM/TQdd5yoaEvy+PCogiyOJsS6sDF+lIYct1GsvlVQ
	USPxgHoEAHS4XN/pZGcPzqKiF5v84Og=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709221514; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=Z8SfY5jVnFw4kZflsv6bRIFUWCtymoR6L22v1i1XNxM=;
	b=hmZjsJa2y8K2UkiwOz92Zu7CVi9X7WL89xGgl+jg3c0s+0b2xXzDa8uOCB2Y5tFywnME6/
	3e9NDIsppA7G4XYQNrh5lUESO1bzuM/TQdd5yoaEvy+PCogiyOJsS6sDF+lIYct1GsvlVQ
	USPxgHoEAHS4XN/pZGcPzqKiF5v84Og=
Message-ID: <d8f15460-8d58-4096-9e4f-b6ac307863db@suse.com>
Date: Thu, 29 Feb 2024 16:45:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 10/12] xen/spinlock: split recursive spinlocks from
 normal ones
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-11-jgross@suse.com>
 <a037facf-c4d9-491f-b39f-56ed4221abf1@suse.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <a037facf-c4d9-491f-b39f-56ed4221abf1@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=hmZjsJa2
X-Spamd-Result: default: False [-3.13 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 DWL_DNSWL_HI(-3.50)[suse.com:dkim];
	 SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MX_GOOD(-0.01)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.67)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Rspamd-Server: rspamd1.dmz-prg2.suse.org
X-Rspamd-Queue-Id: 1E03C1F7E9
X-Spam-Level: 
X-Spam-Score: -3.13
X-Spam-Flag: NO

On 29.02.24 16:32, Jan Beulich wrote:
> On 12.12.2023 10:47, Juergen Gross wrote:
>> --- a/xen/common/spinlock.c
>> +++ b/xen/common/spinlock.c
>> @@ -541,6 +541,55 @@ void rspin_unlock_irqrestore(rspinlock_t *lock, unsigned long flags)
>>       local_irq_restore(flags);
>>   }
>>   
>> +int nrspin_trylock(rspinlock_t *lock)
>> +{
>> +    check_lock(&lock->debug, true);
>> +
>> +    if ( unlikely(lock->recurse_cpu != SPINLOCK_NO_CPU) )
>> +        return 0;
>> +
>> +    return spin_trylock_common(&lock->tickets, &lock->debug, LOCK_PROFILE_PAR);
>> +}
> 
> I wonder if we shouldn't take the opportunity and stop having trylock
> functions have "int" return type. Perhaps already spin_trylock_common()
> when introduced could use "bool" instead, then here following suit.

Fine with me.

> 
>> +void nrspin_lock(rspinlock_t *lock)
>> +{
>> +    spin_lock_common(&lock->tickets, &lock->debug, LOCK_PROFILE_PAR, NULL,
>> +                     NULL);
>> +}
>> +
>> +void nrspin_unlock(rspinlock_t *lock)
>> +{
>> +    spin_unlock_common(&lock->tickets, &lock->debug, LOCK_PROFILE_PAR);
>> +}
>> +
>> +void nrspin_lock_irq(rspinlock_t *lock)
>> +{
>> +    ASSERT(local_irq_is_enabled());
>> +    local_irq_disable();
>> +    nrspin_lock(lock);
>> +}
>> +
>> +void nrspin_unlock_irq(rspinlock_t *lock)
>> +{
>> +    nrspin_unlock(lock);
>> +    local_irq_enable();
>> +}
>> +
>> +unsigned long __nrspin_lock_irqsave(rspinlock_t *lock)
>> +{
>> +    unsigned long flags;
>> +
>> +    local_irq_save(flags);
>> +    nrspin_lock(lock);
>> +    return flags;
> 
> Nit: Strictly speaking we want a blank line ahead of this "return".

Okay, will add it.

> 
>> @@ -166,11 +172,15 @@ struct lock_profile_qhead { };
>>   struct lock_profile { };
>>   
>>   #define SPIN_LOCK_UNLOCKED {                                                  \
>> +    .debug =_LOCK_DEBUG,                                                      \
>> +}
>> +#define RSPIN_LOCK_UNLOCKED {                                                 \
>> +    .debug =_LOCK_DEBUG,                                                      \
>>       .recurse_cpu = SPINLOCK_NO_CPU,                                           \
>>       .debug =_LOCK_DEBUG,                                                      \
>>   }
> 
> Initializing .debug twice?

Oh, right. Will drop one.

> 
>> @@ -180,7 +190,6 @@ struct lock_profile { };
>>   
>>   #endif
>>   
>> -
>>   typedef union {
>>       uint32_t head_tail;
>>       struct {
> 
> Looks like this might be undoing what the earlier patch isn't going to
> do anymore?

Yes, seen it already.

> 
>> @@ -242,6 +257,19 @@ void rspin_unlock_irqrestore(rspinlock_t *lock, unsigned long flags);
>>   int rspin_is_locked(const rspinlock_t *lock);
>>   void rspin_barrier(rspinlock_t *lock);
>>   
>> +int nrspin_trylock(rspinlock_t *lock);
>> +void nrspin_lock(rspinlock_t *lock);
>> +void nrspin_unlock(rspinlock_t *lock);
>> +void nrspin_lock_irq(rspinlock_t *lock);
>> +void nrspin_unlock_irq(rspinlock_t *lock);
>> +#define nrspin_lock_irqsave(l, f)                               \
>> +    ({                                                          \
>> +        BUILD_BUG_ON(sizeof(f) != sizeof(unsigned long));       \
>> +        ((f) = __nrspin_lock_irqsave(l));                       \
> 
> I don't think the outer pair of parentheses is needed here. As to the
> leading double underscores, see comments elsewhere.

Okay.


Juergen


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:46:19 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:46:19 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687233.1070356 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfibz-0001R8-LH; Thu, 29 Feb 2024 15:46:19 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687233.1070356; Thu, 29 Feb 2024 15:46:19 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfibz-0001R1-I5; Thu, 29 Feb 2024 15:46:19 +0000
Received: by outflank-mailman (input) for mailman id 687233;
 Thu, 29 Feb 2024 15:46:18 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfiby-0001Qt-6y
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:46:18 +0000
Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com
 [2a00:1450:4864:20::52a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id acf19f64-d719-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:46:17 +0100 (CET)
Received: by mail-ed1-x52a.google.com with SMTP id
 4fb4d7f45d1cf-56533e30887so1800276a12.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 07:46:17 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 he44-20020a1709073dac00b00a4323d1b18fsm804181ejc.34.2024.02.29.07.46.16
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 07:46:16 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: acf19f64-d719-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709221576; x=1709826376; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=yvStt6RpT6Q49zFDLwqwelFVZVbqpoF3M0TQaFXVjFk=;
        b=aJgXDQGi3dkrOiIiocS9QnZwsB4s36KgC0e3XZ2jTpEJZRnfOwkSCm1OWmX/rmkQ3A
         3GbGsVGN9A9Bi1IY0qhLd0qHaVh8143OMtBfLP3Z3iGcgi4MjlU3FtecQ+T3wsS1aLQ0
         uWUmZhYfLISi92N1LgtmQZOLMelC0ZSJDQuwZ3NfbQIuNusGkJqkONg7bQmS7dye0uNz
         fI6kg0rWEHZc1OaRTQPQEOsadqu47SMjTuhiIc5ovMGObIo0DaKYMPgnWIvbeWpqbcR4
         Fjq+13NCgDw73VqH5kInaZy+90oEDKRljG+t5/x5CjSMuquVLqTtu/Dv6Pti6joZMQGf
         4TUw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709221576; x=1709826376;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=yvStt6RpT6Q49zFDLwqwelFVZVbqpoF3M0TQaFXVjFk=;
        b=q+jhEUHzkdQ312Ea+m46kxL8SwgAm/lLBifAahQPw8ZZlsrnFCf3hdPblITULzor3j
         cqjdrIAvA3DfuJUsxPbg/m+15Ygz8O5lkRTZsgCjxhdjuFhWf19QZopLhyOJ8QifGmA/
         CZDKLi3wh7q7Qf5dpGwhh7DbNFra/ApiGox1aENLszTbtmjEV2xcE6am9feFj3Lq552Z
         zAn5h/D62duDkxogr6nxyaxEuyzgC+i/Cb1rH2O2nRfVYWMpz2wmjNOqFmVHgp16b0+U
         cm7ksEsrn5Za0gnirU/NybP9oVs7msDnlf6X6kY6zaCR00Y6BoaWAKinUwLA31BKHRco
         w+LA==
X-Forwarded-Encrypted: i=1; AJvYcCWy32Qnq7Z/o2Oe0uli+4wP/7HiBP8K0YHSwiPbo7wsmWB1hw2uH9nh3hYdgzMoCgD+U90ffIz4EsaQ574zqUTe1pxPEXNNKW9TU9ECds0=
X-Gm-Message-State: AOJu0YzCyxXNWtdvrd0qY3eovt7mfTjP0ezBaxjh/TppL3E4khq1o7Q9
	noE3ZT0dsrsxtG5aA5sCNI2vVdxXfnZZFLM8TPMMxNg30ydTvYuXqzhfx3DBPQ==
X-Google-Smtp-Source: AGHT+IHGX/jJLHhP0Gk2ekkO5WT9x5v3oH3WPAGQiTxxufBleiRuvX5yk4vx+Ae3mcMKcS3LCnhLcw==
X-Received: by 2002:a17:906:46d9:b0:a44:4ee9:c560 with SMTP id k25-20020a17090646d900b00a444ee9c560mr1162166ejs.53.1709221576655;
        Thu, 29 Feb 2024 07:46:16 -0800 (PST)
Message-ID: <c96542eb-daf4-4ce2-915e-86ca8e1848b5@suse.com>
Date: Thu, 29 Feb 2024 16:46:15 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/12] xen/spinlock: support higher number of cpus
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-13-jgross@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <20231212094725.22184-13-jgross@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 12.12.2023 10:47, Juergen Gross wrote:
> Allow 16 bits per cpu number, which is the limit imposed by
> spinlock_tickets_t.
> 
> This will allow up to 65535 cpus, while increasing only the size of
> recursive spinlocks in debug builds from 8 to 12 bytes.

I think we want to be more conservative here, for the case of there
being bugs: The CPU holding a lock may wrongly try to acquire it a
2nd time. That's the 65536th ticket then, wrapping the value.
Therefore my suggestion would be to only (mention) go(ing) up to 32k.

> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  xen/common/spinlock.c      |  1 +
>  xen/include/xen/spinlock.h | 18 +++++++++---------
>  2 files changed, 10 insertions(+), 9 deletions(-)

Shouldn't this also bump the upper bound of the NR_CPUS range then
in xen/arch/Kconfig?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:49:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:49:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687236.1070365 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfif5-0002EU-2E; Thu, 29 Feb 2024 15:49:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687236.1070365; Thu, 29 Feb 2024 15:49:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfif4-0002EN-Vs; Thu, 29 Feb 2024 15:49:30 +0000
Received: by outflank-mailman (input) for mailman id 687236;
 Thu, 29 Feb 2024 15:49:29 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=CJxE=KG=kernel.org=sashal@srs-se1.protection.inumbo.net>)
 id 1rfif3-0002EH-Ms
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:49:29 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1de7406b-d71a-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:49:27 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 049A46118F;
 Thu, 29 Feb 2024 15:49:26 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 23133C433F1;
 Thu, 29 Feb 2024 15:49:23 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1de7406b-d71a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709221765;
	bh=8b7DfpvkFhJ07ZMBds5WMnK3M2mYX6QhEVWSf43j/K4=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=SYecIYPBfgm2t/4TqRQUjQnGt/AEfhnkzY1mSZyS5N0p7LilltiLz6j7XWLa6EYU6
	 6Q/EXXpQIjQ/2K6umnUHKpZ4fBhEX/f7gOdt4VwWiyqG7bmuUc11rh//85GxaSpEe9
	 XGI9HP1nRlf/Tv+Lm1pjhF/CueR42WLe1j4ke8Y0ip5MzTXrO3fxnZ8pQlj/SSy1AU
	 /38jJoeUyQtc4GhIii9OnetlwuFrJXJA93fXOhxv0DwozOtWvEPfS+EyBxSTBYjqCB
	 PTO+EBasDwL8XREScSOAdfNYy3PIsl6h664qbXmSjJ4KTeCbaMNa+wdWErrpo0PHLz
	 E8GcX21G08GvA==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Kunwu Chan <chentao@kylinos.cn>,
	kernel test robot <lkp@intel.com>,
	Markus Elfring <Markus.Elfring@web.de>,
	Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	dave.hansen@linux.intel.com,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 6.7 18/26] x86/xen: Add some null pointer checking to smp.c
Date: Thu, 29 Feb 2024 10:48:37 -0500
Message-ID: <20240229154851.2849367-18-sashal@kernel.org>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240229154851.2849367-1-sashal@kernel.org>
References: <20240229154851.2849367-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
X-stable-base: Linux 6.7.6
Content-Transfer-Encoding: 8bit

From: Kunwu Chan <chentao@kylinos.cn>

[ Upstream commit 3693bb4465e6e32a204a5b86d3ec7e6b9f7e67c2 ]

kasprintf() returns a pointer to dynamically allocated memory
which can be NULL upon failure. Ensure the allocation was successful
by checking the pointer validity.

Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202401161119.iof6BQsf-lkp@intel.com/
Suggested-by: Markus Elfring <Markus.Elfring@web.de>
Reviewed-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20240119094948.275390-1-chentao@kylinos.cn
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 4b0d6fff88de5..1fb9a1644d944 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -65,6 +65,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	char *resched_name, *callfunc_name, *debug_name;
 
 	resched_name = kasprintf(GFP_KERNEL, "resched%d", cpu);
+	if (!resched_name)
+		goto fail_mem;
 	per_cpu(xen_resched_irq, cpu).name = resched_name;
 	rc = bind_ipi_to_irqhandler(XEN_RESCHEDULE_VECTOR,
 				    cpu,
@@ -77,6 +79,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	per_cpu(xen_resched_irq, cpu).irq = rc;
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfunc%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
 	per_cpu(xen_callfunc_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_VECTOR,
 				    cpu,
@@ -90,6 +94,9 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	if (!xen_fifo_events) {
 		debug_name = kasprintf(GFP_KERNEL, "debug%d", cpu);
+		if (!debug_name)
+			goto fail_mem;
+
 		per_cpu(xen_debug_irq, cpu).name = debug_name;
 		rc = bind_virq_to_irqhandler(VIRQ_DEBUG, cpu,
 					     xen_debug_interrupt,
@@ -101,6 +108,9 @@ int xen_smp_intr_init(unsigned int cpu)
 	}
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfuncsingle%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
+
 	per_cpu(xen_callfuncsingle_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_SINGLE_VECTOR,
 				    cpu,
@@ -114,6 +124,8 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	return 0;
 
+ fail_mem:
+	rc = -ENOMEM;
  fail:
 	xen_smp_intr_free(cpu);
 	return rc;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:50:23 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:50:23 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687238.1070376 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfifu-0003nQ-CI; Thu, 29 Feb 2024 15:50:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687238.1070376; Thu, 29 Feb 2024 15:50:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfifu-0003nJ-9J; Thu, 29 Feb 2024 15:50:22 +0000
Received: by outflank-mailman (input) for mailman id 687238;
 Thu, 29 Feb 2024 15:50:20 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=CJxE=KG=kernel.org=sashal@srs-se1.protection.inumbo.net>)
 id 1rfifs-0003n2-Eb
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:50:20 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org
 [2604:1380:4641:c500::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 3ca9e7be-d71a-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:50:19 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id D30DC611AE;
 Thu, 29 Feb 2024 15:50:17 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id E6209C433B1;
 Thu, 29 Feb 2024 15:50:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 3ca9e7be-d71a-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709221817;
	bh=8b7DfpvkFhJ07ZMBds5WMnK3M2mYX6QhEVWSf43j/K4=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=f5G2clpUy0vR8RIAKU3RgU+VZZDBDBrodtCWEPP3IZ8Elm7+6uQpChU4bZ27zWc2/
	 SPefgt8BCCNyryP5bm9nun4RDqN1bkOab1z4nej6sZge43SFDdyCiQ4wXOA6pgNZJc
	 KceQptfd8xIe5CkNPmhmgOmGbNxd7dhTKBpIPZfjJj/HJ0voQrvZECAQt/XQHWXAme
	 Nx0qKrIOWWkaO0+hvadXfU9qwYgIBqkbExdY9rq3LwAhhHIpjs8nM9e5F/q8UReXH/
	 kGKUjo5GiglJc7Y+FoOJaSCVyGbHGqy6YqvKd9G8w3FS4wb2O+8k3yv403tzAAnJmf
	 0OVF4wJTuGF7A==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Kunwu Chan <chentao@kylinos.cn>,
	kernel test robot <lkp@intel.com>,
	Markus Elfring <Markus.Elfring@web.de>,
	Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	dave.hansen@linux.intel.com,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 6.6 16/21] x86/xen: Add some null pointer checking to smp.c
Date: Thu, 29 Feb 2024 10:49:36 -0500
Message-ID: <20240229154946.2850012-16-sashal@kernel.org>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240229154946.2850012-1-sashal@kernel.org>
References: <20240229154946.2850012-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
X-stable-base: Linux 6.6.18
Content-Transfer-Encoding: 8bit

From: Kunwu Chan <chentao@kylinos.cn>

[ Upstream commit 3693bb4465e6e32a204a5b86d3ec7e6b9f7e67c2 ]

kasprintf() returns a pointer to dynamically allocated memory
which can be NULL upon failure. Ensure the allocation was successful
by checking the pointer validity.

Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202401161119.iof6BQsf-lkp@intel.com/
Suggested-by: Markus Elfring <Markus.Elfring@web.de>
Reviewed-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20240119094948.275390-1-chentao@kylinos.cn
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 4b0d6fff88de5..1fb9a1644d944 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -65,6 +65,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	char *resched_name, *callfunc_name, *debug_name;
 
 	resched_name = kasprintf(GFP_KERNEL, "resched%d", cpu);
+	if (!resched_name)
+		goto fail_mem;
 	per_cpu(xen_resched_irq, cpu).name = resched_name;
 	rc = bind_ipi_to_irqhandler(XEN_RESCHEDULE_VECTOR,
 				    cpu,
@@ -77,6 +79,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	per_cpu(xen_resched_irq, cpu).irq = rc;
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfunc%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
 	per_cpu(xen_callfunc_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_VECTOR,
 				    cpu,
@@ -90,6 +94,9 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	if (!xen_fifo_events) {
 		debug_name = kasprintf(GFP_KERNEL, "debug%d", cpu);
+		if (!debug_name)
+			goto fail_mem;
+
 		per_cpu(xen_debug_irq, cpu).name = debug_name;
 		rc = bind_virq_to_irqhandler(VIRQ_DEBUG, cpu,
 					     xen_debug_interrupt,
@@ -101,6 +108,9 @@ int xen_smp_intr_init(unsigned int cpu)
 	}
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfuncsingle%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
+
 	per_cpu(xen_callfuncsingle_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_SINGLE_VECTOR,
 				    cpu,
@@ -114,6 +124,8 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	return 0;
 
+ fail_mem:
+	rc = -ENOMEM;
  fail:
 	xen_smp_intr_free(cpu);
 	return rc;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:50:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:50:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687241.1070386 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfigM-0004Ok-Mq; Thu, 29 Feb 2024 15:50:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687241.1070386; Thu, 29 Feb 2024 15:50:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfigM-0004Od-JL; Thu, 29 Feb 2024 15:50:50 +0000
Received: by outflank-mailman (input) for mailman id 687241;
 Thu, 29 Feb 2024 15:50:50 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=CJxE=KG=kernel.org=sashal@srs-se1.protection.inumbo.net>)
 id 1rfigM-0003n2-5M
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:50:50 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4eb06412-d71a-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:50:49 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 1982760B2D;
 Thu, 29 Feb 2024 15:50:48 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 40F81C433A6;
 Thu, 29 Feb 2024 15:50:46 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4eb06412-d71a-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709221847;
	bh=8b7DfpvkFhJ07ZMBds5WMnK3M2mYX6QhEVWSf43j/K4=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=A/JwljoA+H0b+5cMRWeidCvo6HML6IL1/ttJed4GCWFeaATiLqs4TW8haB5g2gDNE
	 5r7y5t+2OYFGwHWPV9257gCaivkgHUbGMsI0BUpwsk28wJmXpPZbSDR9R63alxezl0
	 lFaNRBItzmyXHce5eY0oDZED9v9KCjBsrNsqtsi/VrIkDbwa8BEJHhK9pXWidhFVdC
	 YZhSKl+FuED7k7vWJiA8UimhG16L1LozN0rFtfoHDtZhoX3n1KTOpew0XHfHa14U5d
	 uwx2Lczj32NP8pUvZkY8geg3modySxzFbhY+XVjDqvhEvUhRoX+hBia+k1VNqzQhMn
	 opLtNGRgRJQpQ==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Kunwu Chan <chentao@kylinos.cn>,
	kernel test robot <lkp@intel.com>,
	Markus Elfring <Markus.Elfring@web.de>,
	Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	dave.hansen@linux.intel.com,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 6.1 08/12] x86/xen: Add some null pointer checking to smp.c
Date: Thu, 29 Feb 2024 10:50:25 -0500
Message-ID: <20240229155032.2850566-8-sashal@kernel.org>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240229155032.2850566-1-sashal@kernel.org>
References: <20240229155032.2850566-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
X-stable-base: Linux 6.1.79
Content-Transfer-Encoding: 8bit

From: Kunwu Chan <chentao@kylinos.cn>

[ Upstream commit 3693bb4465e6e32a204a5b86d3ec7e6b9f7e67c2 ]

kasprintf() returns a pointer to dynamically allocated memory
which can be NULL upon failure. Ensure the allocation was successful
by checking the pointer validity.

Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202401161119.iof6BQsf-lkp@intel.com/
Suggested-by: Markus Elfring <Markus.Elfring@web.de>
Reviewed-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20240119094948.275390-1-chentao@kylinos.cn
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 4b0d6fff88de5..1fb9a1644d944 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -65,6 +65,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	char *resched_name, *callfunc_name, *debug_name;
 
 	resched_name = kasprintf(GFP_KERNEL, "resched%d", cpu);
+	if (!resched_name)
+		goto fail_mem;
 	per_cpu(xen_resched_irq, cpu).name = resched_name;
 	rc = bind_ipi_to_irqhandler(XEN_RESCHEDULE_VECTOR,
 				    cpu,
@@ -77,6 +79,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	per_cpu(xen_resched_irq, cpu).irq = rc;
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfunc%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
 	per_cpu(xen_callfunc_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_VECTOR,
 				    cpu,
@@ -90,6 +94,9 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	if (!xen_fifo_events) {
 		debug_name = kasprintf(GFP_KERNEL, "debug%d", cpu);
+		if (!debug_name)
+			goto fail_mem;
+
 		per_cpu(xen_debug_irq, cpu).name = debug_name;
 		rc = bind_virq_to_irqhandler(VIRQ_DEBUG, cpu,
 					     xen_debug_interrupt,
@@ -101,6 +108,9 @@ int xen_smp_intr_init(unsigned int cpu)
 	}
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfuncsingle%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
+
 	per_cpu(xen_callfuncsingle_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_SINGLE_VECTOR,
 				    cpu,
@@ -114,6 +124,8 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	return 0;
 
+ fail_mem:
+	rc = -ENOMEM;
  fail:
 	xen_smp_intr_free(cpu);
 	return rc;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:51:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:51:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687243.1070395 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfigk-0004uz-V1; Thu, 29 Feb 2024 15:51:14 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687243.1070395; Thu, 29 Feb 2024 15:51:14 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfigk-0004us-Ro; Thu, 29 Feb 2024 15:51:14 +0000
Received: by outflank-mailman (input) for mailman id 687243;
 Thu, 29 Feb 2024 15:51:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=CJxE=KG=kernel.org=sashal@srs-se1.protection.inumbo.net>)
 id 1rfigk-0004uf-81
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:51:14 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5b33144c-d71a-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:51:11 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 11687CE1B29;
 Thu, 29 Feb 2024 15:51:05 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id A0C1CC4166C;
 Thu, 29 Feb 2024 15:51:02 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5b33144c-d71a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709221864;
	bh=tjjI2nsFG1aGxXNU8gC7cWP86c+GUyG67wihdRCmajA=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=fQl6jY/mvN7HDbo1qFcVRtiCZ7Mp44U9PTKuXE3P7ne5bb3QNV+vpYpu0K9FMnW8/
	 z8XQyuc3768o0VJAhmpMISxIPYyNHTVdWD4ic/LBMFB0zvf/wlDQy7Zps/M3dGU/Xg
	 C5eCmz7MP8us0T1nZc/i0rQxnTTcC3Jj5//omdT51nkmgvXfcGPRxUmMsJDWByfPxh
	 byZatFfgz4I2s3fHCSDij/9kDnGhrU7BXXdr163eO6HyeEZnzSY8wMuccICabmtlVy
	 ss+cmpQ6cYENbjfVfM+YYtfUTkH9+ppKujIIbM/n/3tiFj6v01cOL8jhtCmU1q/uHK
	 SW/zocYIP+mmw==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Kunwu Chan <chentao@kylinos.cn>,
	kernel test robot <lkp@intel.com>,
	Markus Elfring <Markus.Elfring@web.de>,
	Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	dave.hansen@linux.intel.com,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 5.15 4/7] x86/xen: Add some null pointer checking to smp.c
Date: Thu, 29 Feb 2024 10:50:53 -0500
Message-ID: <20240229155057.2850873-4-sashal@kernel.org>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240229155057.2850873-1-sashal@kernel.org>
References: <20240229155057.2850873-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
X-stable-base: Linux 5.15.149
Content-Transfer-Encoding: 8bit

From: Kunwu Chan <chentao@kylinos.cn>

[ Upstream commit 3693bb4465e6e32a204a5b86d3ec7e6b9f7e67c2 ]

kasprintf() returns a pointer to dynamically allocated memory
which can be NULL upon failure. Ensure the allocation was successful
by checking the pointer validity.

Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202401161119.iof6BQsf-lkp@intel.com/
Suggested-by: Markus Elfring <Markus.Elfring@web.de>
Reviewed-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20240119094948.275390-1-chentao@kylinos.cn
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index cdec892b28e2e..a641e0d452194 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -65,6 +65,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	char *resched_name, *callfunc_name, *debug_name;
 
 	resched_name = kasprintf(GFP_KERNEL, "resched%d", cpu);
+	if (!resched_name)
+		goto fail_mem;
 	per_cpu(xen_resched_irq, cpu).name = resched_name;
 	rc = bind_ipi_to_irqhandler(XEN_RESCHEDULE_VECTOR,
 				    cpu,
@@ -77,6 +79,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	per_cpu(xen_resched_irq, cpu).irq = rc;
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfunc%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
 	per_cpu(xen_callfunc_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_VECTOR,
 				    cpu,
@@ -90,6 +94,9 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	if (!xen_fifo_events) {
 		debug_name = kasprintf(GFP_KERNEL, "debug%d", cpu);
+		if (!debug_name)
+			goto fail_mem;
+
 		per_cpu(xen_debug_irq, cpu).name = debug_name;
 		rc = bind_virq_to_irqhandler(VIRQ_DEBUG, cpu,
 					     xen_debug_interrupt,
@@ -101,6 +108,9 @@ int xen_smp_intr_init(unsigned int cpu)
 	}
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfuncsingle%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
+
 	per_cpu(xen_callfuncsingle_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_SINGLE_VECTOR,
 				    cpu,
@@ -114,6 +124,8 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	return 0;
 
+ fail_mem:
+	rc = -ENOMEM;
  fail:
 	xen_smp_intr_free(cpu);
 	return rc;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:51:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:51:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687244.1070406 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfigv-0005Kk-52; Thu, 29 Feb 2024 15:51:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687244.1070406; Thu, 29 Feb 2024 15:51:25 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfigv-0005Kd-1x; Thu, 29 Feb 2024 15:51:25 +0000
Received: by outflank-mailman (input) for mailman id 687244;
 Thu, 29 Feb 2024 15:51:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=CJxE=KG=kernel.org=sashal@srs-se1.protection.inumbo.net>)
 id 1rfigt-0004uf-Ob
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:51:23 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6229885a-d71a-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:51:21 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id CC0C560C05;
 Thu, 29 Feb 2024 15:51:20 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 164DCC433C7;
 Thu, 29 Feb 2024 15:51:18 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6229885a-d71a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709221880;
	bh=tjjI2nsFG1aGxXNU8gC7cWP86c+GUyG67wihdRCmajA=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=TNgLyCZAP5SabKhrY69Yh/FMxMXc8w/meTe6JT/b5VvB6yoxI+RL6zcQVMZk9ad0j
	 eeODhydE1IjagsFlaWxP56CN70+igf+sa3jBiGfW+JCdm0ejtmIN4L+ACOnuG9jy+Z
	 istEkgVzmHux6sv5riDPneGTS8VgUg/2ueh4XLGxwM3Gni4+fL5AFA4V1Kgo5qXBcn
	 WeOYczAKXiJkwTo90oLOkwuJAk574qJZyNILenGIbGKZpicQ/UgkxwSap/yWcrCgM9
	 +WcIkQ5Y5Fs8DKWMRtqqhrZkIW5OR1oMTZSnMFut8BSgl47x6LGxAAu5XYUkyCOzi7
	 imGlwczYqMYPA==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Kunwu Chan <chentao@kylinos.cn>,
	kernel test robot <lkp@intel.com>,
	Markus Elfring <Markus.Elfring@web.de>,
	Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	dave.hansen@linux.intel.com,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 5.10 4/7] x86/xen: Add some null pointer checking to smp.c
Date: Thu, 29 Feb 2024 10:51:08 -0500
Message-ID: <20240229155112.2851155-4-sashal@kernel.org>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240229155112.2851155-1-sashal@kernel.org>
References: <20240229155112.2851155-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
X-stable-base: Linux 5.10.210
Content-Transfer-Encoding: 8bit

From: Kunwu Chan <chentao@kylinos.cn>

[ Upstream commit 3693bb4465e6e32a204a5b86d3ec7e6b9f7e67c2 ]

kasprintf() returns a pointer to dynamically allocated memory
which can be NULL upon failure. Ensure the allocation was successful
by checking the pointer validity.

Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202401161119.iof6BQsf-lkp@intel.com/
Suggested-by: Markus Elfring <Markus.Elfring@web.de>
Reviewed-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20240119094948.275390-1-chentao@kylinos.cn
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index cdec892b28e2e..a641e0d452194 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -65,6 +65,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	char *resched_name, *callfunc_name, *debug_name;
 
 	resched_name = kasprintf(GFP_KERNEL, "resched%d", cpu);
+	if (!resched_name)
+		goto fail_mem;
 	per_cpu(xen_resched_irq, cpu).name = resched_name;
 	rc = bind_ipi_to_irqhandler(XEN_RESCHEDULE_VECTOR,
 				    cpu,
@@ -77,6 +79,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	per_cpu(xen_resched_irq, cpu).irq = rc;
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfunc%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
 	per_cpu(xen_callfunc_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_VECTOR,
 				    cpu,
@@ -90,6 +94,9 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	if (!xen_fifo_events) {
 		debug_name = kasprintf(GFP_KERNEL, "debug%d", cpu);
+		if (!debug_name)
+			goto fail_mem;
+
 		per_cpu(xen_debug_irq, cpu).name = debug_name;
 		rc = bind_virq_to_irqhandler(VIRQ_DEBUG, cpu,
 					     xen_debug_interrupt,
@@ -101,6 +108,9 @@ int xen_smp_intr_init(unsigned int cpu)
 	}
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfuncsingle%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
+
 	per_cpu(xen_callfuncsingle_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_SINGLE_VECTOR,
 				    cpu,
@@ -114,6 +124,8 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	return 0;
 
+ fail_mem:
+	rc = -ENOMEM;
  fail:
 	xen_smp_intr_free(cpu);
 	return rc;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:51:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:51:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687245.1070415 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfih7-0005nh-C4; Thu, 29 Feb 2024 15:51:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687245.1070415; Thu, 29 Feb 2024 15:51:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfih7-0005na-92; Thu, 29 Feb 2024 15:51:37 +0000
Received: by outflank-mailman (input) for mailman id 687245;
 Thu, 29 Feb 2024 15:51:35 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=CJxE=KG=kernel.org=sashal@srs-se1.protection.inumbo.net>)
 id 1rfih5-0004uf-TT
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:51:35 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6967c0e7-d71a-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:51:34 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id F180E61186;
 Thu, 29 Feb 2024 15:51:32 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 31CA8C433A6;
 Thu, 29 Feb 2024 15:51:31 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6967c0e7-d71a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709221892;
	bh=KXobGiOoIaXzXOXJJJNhZJEIoAnZhnUZnhGA9Ii+OtU=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=uKEOwQXkaRM50JayCtjr3hO4eGqUvyr9Ne1xck4BFiZQRM0WsTp/H9RzlRQlEnOrU
	 ompXaRzIoW6acWogxvjVgA4/P9DFYyG2FBb0LcQhPFPl9SIjRxoKo/AM8f6GmFwc1k
	 Jfj6gatFeFfJOedRfcs6ZxphlTqRKLXOsEUzisDkbxt1/eUDw6CX4JjSvIH6UB5ljb
	 MXMrB3aPsIAk/wx5Eqpofm0S3VTMmqGRwBgzDI4bkgi12V8DS5amhqrUqJUHi8DQNG
	 8oroIu3rRmOBSsljROGLHccYCPdc/gLFipLJD5fKdFv1xXNMYmSoaR/n3Gd7t0IajP
	 5pO6lqvmZUTsA==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Kunwu Chan <chentao@kylinos.cn>,
	kernel test robot <lkp@intel.com>,
	Markus Elfring <Markus.Elfring@web.de>,
	Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	dave.hansen@linux.intel.com,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 5.4 3/5] x86/xen: Add some null pointer checking to smp.c
Date: Thu, 29 Feb 2024 10:51:25 -0500
Message-ID: <20240229155127.2851327-3-sashal@kernel.org>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240229155127.2851327-1-sashal@kernel.org>
References: <20240229155127.2851327-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
X-stable-base: Linux 5.4.269
Content-Transfer-Encoding: 8bit

From: Kunwu Chan <chentao@kylinos.cn>

[ Upstream commit 3693bb4465e6e32a204a5b86d3ec7e6b9f7e67c2 ]

kasprintf() returns a pointer to dynamically allocated memory
which can be NULL upon failure. Ensure the allocation was successful
by checking the pointer validity.

Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202401161119.iof6BQsf-lkp@intel.com/
Suggested-by: Markus Elfring <Markus.Elfring@web.de>
Reviewed-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20240119094948.275390-1-chentao@kylinos.cn
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index a1cc855c539c1..a76ba342a6695 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -65,6 +65,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	char *resched_name, *callfunc_name, *debug_name;
 
 	resched_name = kasprintf(GFP_KERNEL, "resched%d", cpu);
+	if (!resched_name)
+		goto fail_mem;
 	per_cpu(xen_resched_irq, cpu).name = resched_name;
 	rc = bind_ipi_to_irqhandler(XEN_RESCHEDULE_VECTOR,
 				    cpu,
@@ -77,6 +79,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	per_cpu(xen_resched_irq, cpu).irq = rc;
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfunc%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
 	per_cpu(xen_callfunc_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_VECTOR,
 				    cpu,
@@ -90,6 +94,9 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	if (!xen_fifo_events) {
 		debug_name = kasprintf(GFP_KERNEL, "debug%d", cpu);
+		if (!debug_name)
+			goto fail_mem;
+
 		per_cpu(xen_debug_irq, cpu).name = debug_name;
 		rc = bind_virq_to_irqhandler(VIRQ_DEBUG, cpu,
 					     xen_debug_interrupt,
@@ -101,6 +108,9 @@ int xen_smp_intr_init(unsigned int cpu)
 	}
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfuncsingle%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
+
 	per_cpu(xen_callfuncsingle_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_SINGLE_VECTOR,
 				    cpu,
@@ -114,6 +124,8 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	return 0;
 
+ fail_mem:
+	rc = -ENOMEM;
  fail:
 	xen_smp_intr_free(cpu);
 	return rc;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:51:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:51:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687248.1070425 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfihH-0006IR-J9; Thu, 29 Feb 2024 15:51:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687248.1070425; Thu, 29 Feb 2024 15:51:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfihH-0006IK-GP; Thu, 29 Feb 2024 15:51:47 +0000
Received: by outflank-mailman (input) for mailman id 687248;
 Thu, 29 Feb 2024 15:51:46 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=CJxE=KG=kernel.org=sashal@srs-se1.protection.inumbo.net>)
 id 1rfihG-0004uf-Ew
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:51:46 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 6f1ffc22-d71a-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:51:44 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 49A84CE069E;
 Thu, 29 Feb 2024 15:51:42 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C1AF0C43390;
 Thu, 29 Feb 2024 15:51:39 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 6f1ffc22-d71a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709221901;
	bh=KXobGiOoIaXzXOXJJJNhZJEIoAnZhnUZnhGA9Ii+OtU=;
	h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
	b=qvBKADrrmuExjeB+SVbNlSFKgfL0OSYS54DpXS6dNcfTQe/U3WDufQQdPs0fDDf7Z
	 R9i2zIFwZ2QfSw7nKhd9O2c4gJ9qMycBJvbIJFt+B8cCBEoSkwn6xeGzyidyFoXUSs
	 tgioE3CkLMQgF1Bb5RmYHoUOXlFTSWSvcpurApGhuxl2tH3N+BbJEq1q5MLA/N5LXj
	 LYtK5Nm46drdxAP++nB1VogR9uBhU+6hIkDkgwADi5QZPbkTYdYMfC+l7oqxZWhBK3
	 7mFgrmSKM64u8ahjgKsX8a7olU1VMs2fecN9Er+4tO8O5pXqoLIoFxrmR3mqbprbfd
	 5WrgItUqwEaXA==
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Cc: Kunwu Chan <chentao@kylinos.cn>,
	kernel test robot <lkp@intel.com>,
	Markus Elfring <Markus.Elfring@web.de>,
	Juergen Gross <jgross@suse.com>,
	Sasha Levin <sashal@kernel.org>,
	tglx@linutronix.de,
	mingo@redhat.com,
	bp@alien8.de,
	dave.hansen@linux.intel.com,
	x86@kernel.org,
	xen-devel@lists.xenproject.org
Subject: [PATCH AUTOSEL 4.19 2/3] x86/xen: Add some null pointer checking to smp.c
Date: Thu, 29 Feb 2024 10:51:36 -0500
Message-ID: <20240229155138.2851458-2-sashal@kernel.org>
X-Mailer: git-send-email 2.43.0
In-Reply-To: <20240229155138.2851458-1-sashal@kernel.org>
References: <20240229155138.2851458-1-sashal@kernel.org>
MIME-Version: 1.0
X-stable: review
X-Patchwork-Hint: Ignore
X-stable-base: Linux 4.19.307
Content-Transfer-Encoding: 8bit

From: Kunwu Chan <chentao@kylinos.cn>

[ Upstream commit 3693bb4465e6e32a204a5b86d3ec7e6b9f7e67c2 ]

kasprintf() returns a pointer to dynamically allocated memory
which can be NULL upon failure. Ensure the allocation was successful
by checking the pointer validity.

Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202401161119.iof6BQsf-lkp@intel.com/
Suggested-by: Markus Elfring <Markus.Elfring@web.de>
Reviewed-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20240119094948.275390-1-chentao@kylinos.cn
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/xen/smp.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index a1cc855c539c1..a76ba342a6695 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -65,6 +65,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	char *resched_name, *callfunc_name, *debug_name;
 
 	resched_name = kasprintf(GFP_KERNEL, "resched%d", cpu);
+	if (!resched_name)
+		goto fail_mem;
 	per_cpu(xen_resched_irq, cpu).name = resched_name;
 	rc = bind_ipi_to_irqhandler(XEN_RESCHEDULE_VECTOR,
 				    cpu,
@@ -77,6 +79,8 @@ int xen_smp_intr_init(unsigned int cpu)
 	per_cpu(xen_resched_irq, cpu).irq = rc;
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfunc%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
 	per_cpu(xen_callfunc_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_VECTOR,
 				    cpu,
@@ -90,6 +94,9 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	if (!xen_fifo_events) {
 		debug_name = kasprintf(GFP_KERNEL, "debug%d", cpu);
+		if (!debug_name)
+			goto fail_mem;
+
 		per_cpu(xen_debug_irq, cpu).name = debug_name;
 		rc = bind_virq_to_irqhandler(VIRQ_DEBUG, cpu,
 					     xen_debug_interrupt,
@@ -101,6 +108,9 @@ int xen_smp_intr_init(unsigned int cpu)
 	}
 
 	callfunc_name = kasprintf(GFP_KERNEL, "callfuncsingle%d", cpu);
+	if (!callfunc_name)
+		goto fail_mem;
+
 	per_cpu(xen_callfuncsingle_irq, cpu).name = callfunc_name;
 	rc = bind_ipi_to_irqhandler(XEN_CALL_FUNCTION_SINGLE_VECTOR,
 				    cpu,
@@ -114,6 +124,8 @@ int xen_smp_intr_init(unsigned int cpu)
 
 	return 0;
 
+ fail_mem:
+	rc = -ENOMEM;
  fail:
 	xen_smp_intr_free(cpu);
 	return rc;
-- 
2.43.0



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:52:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:52:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687251.1070435 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfihq-000731-SK; Thu, 29 Feb 2024 15:52:22 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687251.1070435; Thu, 29 Feb 2024 15:52:22 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfihq-00072u-P7; Thu, 29 Feb 2024 15:52:22 +0000
Received: by outflank-mailman (input) for mailman id 687251;
 Thu, 29 Feb 2024 15:52:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfihp-0006Ea-4r
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:52:21 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 85879405-d71a-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 16:52:20 +0100 (CET)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-56648955ac5so1484277a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 07:52:20 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 e20-20020a056402105400b0056518035195sm713202edu.69.2024.02.29.07.52.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 07:52:19 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 85879405-d71a-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709221940; x=1709826740; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SVzVZ/oy9DLrxSYVu+pp2bLxrUV0JY0FX+M/mynUhX8=;
        b=AYzGuqvE7GX3BR8XiwHnukl8uemoAs+3NTv4JZsGMwzVL4e/x3a77NQvr/FzAD95Ky
         /j32yB1YROEA3vxE6GXurDVdYKcW0a0BQrKdIbY18F/tHkaSUe5svqz8xz1GkDZ/dbVj
         v5wcIlV6GCTd7hOIvQRArplq6XTuF884yYgWNtnYJ5Lfz7fi7XKfOIPEbpN/+9MT0qHY
         89ogUZX2EkRsZYMZhR/dOcP6Ap0S/S+1UjlD6tS8Po1+KoE86S7r8KU3Y7RrEVZdXl9M
         pgxaEYxQy7NIVEiw9eUnubHfnE3tDONeYEQjyxOoJL6OjodoIxavK042UDIdglJ4mo1T
         EDnQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709221940; x=1709826740;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SVzVZ/oy9DLrxSYVu+pp2bLxrUV0JY0FX+M/mynUhX8=;
        b=dqqJVu13mNyKLcGmMnILcFMW4PGglB8zT8RFKuEttc5HmixztkOO9hp54NgFkJqUFx
         ukh0lHSI5KntTTIVf9WJw6y7sVXr7uja4LBV0NtA8RtbmspUiiajOB7G9L4XQB+hQ+au
         ynJJf/OkP9iEit1qjpM3WDH9Qc0ZwDc/QBqk/4Hx4MBVWeq1OIFBXO6FqiShQLVuQBOl
         Tb1CaCLJWJlG7eOCVk9OXLi4kPaB0LGPsBnM6FKESK0yiN7EQirdpABhjAIhNrgVDvLf
         Cj8A85WarHO+fjfFVdVCoh3S7sS1ZJjt5IaK5VdSZfXbs8xMIZhtwtRxMEUdGU7jS712
         bq7w==
X-Forwarded-Encrypted: i=1; AJvYcCUGSCM+xEOQujiYbMkBfHuAij9ZPfI17dB4vDGWrXGUhOGjYFVgzRtjd7HNqXbp/jXWS4AKZvmvTUxIa+PZpGJRbJAOE5yR6aogWK+Agbs=
X-Gm-Message-State: AOJu0YxHNqBYn9W6K+mCugBMC78aUuPQ9sFNL9BlppprSPN9HicvBj9Y
	KKGaPaZA1aieOeKBxxPeepzOjiMut1DCMmmDRBtJQ1X/QoOhhPKpG8Y6D+1mSg==
X-Google-Smtp-Source: AGHT+IEI6liXWThAakrsd/J5jlIdLgYzZwRT1hc1i/v5CzIJJqvbqNjbeqyEabxRBLMYPjyOGq3Sfw==
X-Received: by 2002:a05:6402:903:b0:566:3f3e:3a23 with SMTP id g3-20020a056402090300b005663f3e3a23mr1696214edz.1.1709221939925;
        Thu, 29 Feb 2024 07:52:19 -0800 (PST)
Message-ID: <4c2ca7b7-0e21-4866-8aaf-1987cf3306b5@suse.com>
Date: Thu, 29 Feb 2024 16:52:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 04/23] xen/asm-generic: introduce generic fls() and
 flsl() functions
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <df7ab5055ef08fa595f913072302770a3f6a5c33.1708962629.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <df7ab5055ef08fa595f913072302770a3f6a5c33.1708962629.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 18:38, Oleksii Kurochko wrote:
> --- /dev/null
> +++ b/xen/include/asm-generic/bitops/fls.h
> @@ -0,0 +1,18 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _ASM_GENERIC_BITOPS_FLS_H_
> +#define _ASM_GENERIC_BITOPS_FLS_H_
> +
> +/**
> + * fls - find last (most-significant) bit set
> + * @x: the word to search
> + *
> + * This is defined the same way as ffs.
> + * Note fls(0) = 0, fls(1) = 1, fls(0x80000000) = 32.
> + */
> +
> +static inline int fls(unsigned int x)
> +{
> +    return generic_fls(x);
> +}

This being an inline function, it requires generic_fls() to be declared.
Yet there's no other header included here. I think these headers would
better be self-contained. Or else (e.g. because of this leading to an
#include cycle) something needs saying somewhere.

The other thing here that worries me is the use of plain int as return
type. Yes, generic_fls() is declared like that, too. But no, the return
value there or here cannot be negative.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 15:56:45 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 15:56:45 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687263.1070446 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfilz-0007sB-Hv; Thu, 29 Feb 2024 15:56:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687263.1070446; Thu, 29 Feb 2024 15:56:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfilz-0007s4-EA; Thu, 29 Feb 2024 15:56:39 +0000
Received: by outflank-mailman (input) for mailman id 687263;
 Thu, 29 Feb 2024 15:56:38 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=cziD=KG=bombadil.srs.infradead.org=BATV+ba4d12e182ac9dd248a8+7494+infradead.org+hch@srs-se1.protection.inumbo.net>)
 id 1rfily-0007qy-8Y
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 15:56:38 +0000
Received: from bombadil.infradead.org (bombadil.infradead.org
 [2607:7c80:54:3::133])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1d90169d-d71b-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 16:56:36 +0100 (CET)
Received: from hch by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red
 Hat Linux)) id 1rfilp-0000000E9xh-2JaC;
 Thu, 29 Feb 2024 15:56:29 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1d90169d-d71b-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Type:MIME-Version
	:References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:
	Content-Transfer-Encoding:Content-ID:Content-Description;
	bh=ZZeeQhzP/DHpKF5QdT9LDfZVh1KeLzxu5kAiu8X0IuI=; b=MyDA82W9Ybj19i6XFkxtLJHcIj
	DIdV0B551cX9Of1vbLiE7wQyZ+QZ5s8/rx8PcMT7ZPqE5MxTVE1PqdqAGpoLtaEP7Z4GODcSqBYpf
	+5PUuQblRp6R9otKcB2rgpT/TckK30wW/x9kWHfBUBOEYLAv9ENYNYjr74JbVv+4HkHvDmFDcyos1
	L5Gi5pVl5ETAKTSF6e0VODoCUz2lHFEdWX18oSiBxUvU0VJHDtUoU+XqIK7UWxX26b+RFFj8q7AE8
	7UYQl/Ts9ehrXAidDb+ChLkR4jo9tWvXsBvM8DXn+wWbhjnOn95vr0rgW5lmvfOKAFxXuYbK3wwqj
	Ze2WUX/Q==;
Date: Thu, 29 Feb 2024 07:56:29 -0800
From: Christoph Hellwig <hch@infradead.org>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Christoph Hellwig <hch@infradead.org>, bpf <bpf@vger.kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Barret Rhoden <brho@google.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Lorenzo Stoakes <lstoakes@gmail.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Uladzislau Rezki <urezki@gmail.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	sstabellini@kernel.org, Juergen Gross <jgross@suse.com>,
	linux-mm <linux-mm@kvack.org>, xen-devel@lists.xenproject.org,
	Kernel Team <kernel-team@fb.com>
Subject: Re: [PATCH v2 bpf-next 3/3] mm: Introduce VM_SPARSE kind and
 vm_area_[un]map_pages().
Message-ID: <ZeCpLW25Fn6Di3Gu@infradead.org>
References: <20240223235728.13981-1-alexei.starovoitov@gmail.com>
 <20240223235728.13981-4-alexei.starovoitov@gmail.com>
 <Zd4jGhvb-Utdo2jU@infradead.org>
 <CAADnVQ+f06b1hDrAyLM-OrzDfEEa=jtamJOKfEnEo4ewKPV0cA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAADnVQ+f06b1hDrAyLM-OrzDfEEa=jtamJOKfEnEo4ewKPV0cA@mail.gmail.com>
X-SRS-Rewrite: SMTP reverse-path rewritten from <hch@infradead.org> by bombadil.infradead.org. See http://www.infradead.org/rpr.html

On Tue, Feb 27, 2024 at 05:31:28PM -0800, Alexei Starovoitov wrote:
> What would it look like with a cookie?
> A static inline wrapper around get_vm_area() that returns area->addr ?
> And the start address of vmap range will be such a cookie?

Hmm, just making the kernel virtual address the cookie actually
sounds pretty neat indeed even if I did not have that in mind.

> I guess I don't understand the motivation to hide 'struct vm_struct *'.

The prime reason is that then people will try to start random APIs that
work on it.  But let's give it a try without the wrappers and see how
things go.



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:09:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:09:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687267.1070455 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiyf-0002ge-KO; Thu, 29 Feb 2024 16:09:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687267.1070455; Thu, 29 Feb 2024 16:09:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfiyf-0002gX-Hd; Thu, 29 Feb 2024 16:09:45 +0000
Received: by outflank-mailman (input) for mailman id 687267;
 Thu, 29 Feb 2024 16:09:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygh/=KG=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rfiye-0002gR-Nm
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:09:44 +0000
Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com
 [2a00:1450:4864:20::135])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f3763e91-d71c-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:09:43 +0100 (CET)
Received: by mail-lf1-x135.google.com with SMTP id
 2adb3069b0e04-5132c455250so328729e87.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:09:43 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 m12-20020a19434c000000b005129994f013sm300138lfj.184.2024.02.29.08.09.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 08:09:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f3763e91-d71c-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709222983; x=1709827783; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=ElSet3YFfblPGyJ+kvjm8n5Jw+L91jCWvs9T39e2+Rw=;
        b=jKR+fxnCCY6+hUvtx7iubxmSozYegjbholoCcKN0YSz6LJdQ+rk16aL3p1Pd82BFsW
         A6JL1JhvbFRe3RaqNpSt/rzxFgS0Gcz2UNNWTzKyZmYXA/6a16w6rBt2mRZSZPJzOW7f
         acxukOX8viTDMNPqdy94TAzhhUrnSKI4WFk36RjvYIxN9BP0TUdFEXVgDhtIZ9KqDER4
         vM1ROxgk44I0ZHzhNjMOC1M2ple/4iDcsoYkaweCh++8O9CWJkFHu0M/+gbQas2Yn2v2
         cVDJVsJUk/E6LsJSrnhc8/a8TopDoAu4SR5/49uju22tyfUlNoRlmQU3K0kqpWqB1bTe
         Rieg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709222983; x=1709827783;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ElSet3YFfblPGyJ+kvjm8n5Jw+L91jCWvs9T39e2+Rw=;
        b=WE6+MQsM85R6IWjFan1Y1ry+Lmi1z1+HXSjvhvr/EnQV1qP46p4SOTVAnzPRgmAiLh
         /zh03qEV6KRlgHQLUAK5WVFGjQ+TOcpk2qm29TqMKLC/BHDScGrcjfUr5lqAYSyELE8Y
         JoBskxc8483wLDtViClIOlFJC74cw9RZFQRT/x8AsX672PneCGozLdcx8o9ahoxyFSj0
         6rwL+k9/Xif83H0umMmsNI6SsRUqkYdRkac+mruiMvkyunL6AjDMc5zCSAHnDCKiOyaM
         Ay7+9IZ8xAmyRh5uRKJGjXHED7aJ8K854EeoYl/G24e7r/KLGZRWki8vOJiIjMAsbzNw
         4lAw==
X-Forwarded-Encrypted: i=1; AJvYcCUjPe4AStu+fkn8xgfdjnrDqjGREzUkJDUBKkWgbrX0Fs21V7fSqo5xxQk9R6j+2IR/on+gUeDRd3nGS9fEeSibYvkpYA3ziOP/KarpSNk=
X-Gm-Message-State: AOJu0YxMyOqc/comlSv2lK3J8PKsEv2p81Mwq4Ue9JLxdR7TwBSdrR67
	aGkMQ8WJe0eVDCtDTvsqSxc6otxFxTsMeXx/MK18vT/KlVkxRJl0
X-Google-Smtp-Source: AGHT+IHqO1oocMSOs0rIl8nqDfyz/DM0iyjZjETz/z8lOeBNJtf6BzozFNa0r0iWXRkKjJmcV+4RRg==
X-Received: by 2002:a19:5e18:0:b0:513:2be9:7a8e with SMTP id s24-20020a195e18000000b005132be97a8emr554472lfb.62.1709222982968;
        Thu, 29 Feb 2024 08:09:42 -0800 (PST)
Message-ID: <6150e4780fadf81e70b56da9e63112047945e249.camel@gmail.com>
Subject: Re: [PATCH v5 03/23] xen/riscv: introduce nospec.h
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>
Cc: Alistair Francis <alistair.francis@wdc.com>, Bob Eshleman
 <bobbyeshleman@gmail.com>, Connor Davis <connojdavis@gmail.com>, Andrew
 Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>,  Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Date: Thu, 29 Feb 2024 17:09:42 +0100
In-Reply-To: <7d9a7a89-2ad1-49e3-9cac-7cf8d6b5e6c5@suse.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
	 <16370492de5344ae8e1aab688261600491868f4e.1708962629.git.oleksii.kurochko@gmail.com>
	 <e44a8bcf-a037-441c-b85c-651784df020a@xen.org>
	 <7d9a7a89-2ad1-49e3-9cac-7cf8d6b5e6c5@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Thu, 2024-02-29 at 15:01 +0100, Jan Beulich wrote:
> On 29.02.2024 14:49, Julien Grall wrote:
> > On 26/02/2024 17:38, Oleksii Kurochko wrote:
> > > --- /dev/null
> > > +++ b/xen/arch/riscv/include/asm/nospec.h
> > > @@ -0,0 +1,25 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> >=20
> > New file should use the SPDX tag GPL-2.0-only. I guess this could
> > be=20
> > fixed on commit?
>=20
> I wouldn't mind doing so.
I would happy with that. Thanks.

~ Oleksii


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:10:51 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:10:51 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687269.1070466 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfizi-0004BY-Ue; Thu, 29 Feb 2024 16:10:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687269.1070466; Thu, 29 Feb 2024 16:10:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfizi-0004BR-RS; Thu, 29 Feb 2024 16:10:50 +0000
Received: by outflank-mailman (input) for mailman id 687269;
 Thu, 29 Feb 2024 16:10:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfizh-0004BF-2K
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:10:49 +0000
Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com
 [2a00:1450:4864:20::636])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 1913589a-d71d-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:10:47 +0100 (CET)
Received: by mail-ej1-x636.google.com with SMTP id
 a640c23a62f3a-a3fb8b0b7acso165666666b.2
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:10:47 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 gs41-20020a1709072d2900b00a3d9e6e9983sm824860ejc.174.2024.02.29.08.10.44
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:10:45 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1913589a-d71d-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709223046; x=1709827846; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=a3Z+PfadN5bd277EOL7z+3fzaDA/n/PXl/UtTOHkuNY=;
        b=BUHE4YTnJtIoRQeurtDjQ1MskyVzqsMPI8oPV3MNHw+OCmKdGNojzZppoTH2pqNq+o
         obsmaMg2lQGaF6Xet19h6AIsXXfQ06bhMcIMWEX84D1eiKMOiWsw3g2itjxiJKyiuv8J
         o8WPeaQyFrUmJp5vuWYpGE+z7JCeiaRKYV1TE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709223046; x=1709827846;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=a3Z+PfadN5bd277EOL7z+3fzaDA/n/PXl/UtTOHkuNY=;
        b=NFKjvDA6cNnxdVx9mJGbvliVdGOWEhaO9zWmqntvNTANgnE8l6MUqK0rgAcKBa/ihZ
         F9vv5sfNEnsfomzC2UwbVjrZlOPNHqoLzVBHRuCNzNsroswCd+vDCaxVw1NUl1qiqpkt
         v7ovsY8fRp1qdVM3cH//CLKaZqFjhFGc/jnfflMyRb37TNRj3jb+3ZvRG4n9OJBef+ta
         fXDGwToTfHjvU7xXLyR/OWOJZ3aP10sLU1Ie1KzCgfgM/2HQgS64YH6dNWxFHunN1Ovv
         u6Lnx5S2kiBx6+v4RR9TXQd7equ8GDqbX7UJr5dm2E0sNAmBmtUgHviN9daEcRc8q4Jr
         /hHQ==
X-Forwarded-Encrypted: i=1; AJvYcCWDlqISR8jV/vhTzosUEiBCMHO8OEwsinx42md7NnNZCIKe+M965p8xPriw3Bhe5H8wwFAd1Yr5Onw02Ytmt4jIT7N7JgkuOwDSgpLn1Io=
X-Gm-Message-State: AOJu0YzeNeL9p652mIQpiUVaroEusrT8WE5v1H9g8yDIUw0eD5B9T7Xy
	VW5S8O5nHWOOZY4x0ya5N5gpDPj6Ic4oNC9z1phy3FtFc/47M/PDQzi0TbgnsO4=
X-Google-Smtp-Source: AGHT+IFH9JfVmsYdZH7GCC/a9MqcXTvC7ub7/fVF7nkmLYFyEeFPiKHPdOaFUgKqAfgrhmjT5tznTw==
X-Received: by 2002:a17:906:397:b0:a3e:792f:3955 with SMTP id b23-20020a170906039700b00a3e792f3955mr1729741eja.62.1709223046479;
        Thu, 29 Feb 2024 08:10:46 -0800 (PST)
Message-ID: <b844e28f-a53b-49c0-b4b9-297367546bd3@citrix.com>
Date: Thu, 29 Feb 2024 16:10:44 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 01/10] xen/include: address violations of MISRA C Rule
 20.7
Content-Language: en-GB
To: Nicola Vetrini <nicola.vetrini@bugseng.com>,
 xen-devel@lists.xenproject.org
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com, jbeulich@suse.com,
 roger.pau@citrix.com, bertrand.marquis@arm.com, julien@xen.org,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <15d6e4fb5c873e7ea42cfcee2faa0bf33c10d101.1709219010.git.nicola.vetrini@bugseng.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <15d6e4fb5c873e7ea42cfcee2faa0bf33c10d101.1709219010.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/02/2024 3:27 pm, Nicola Vetrini wrote:
> diff --git a/xen/include/xen/kconfig.h b/xen/include/xen/kconfig.h
> index c25dc0f6c2a9..b7e70289737b 100644
> --- a/xen/include/xen/kconfig.h
> +++ b/xen/include/xen/kconfig.h
> @@ -25,7 +25,7 @@
>  #define __ARG_PLACEHOLDER_1 0,
>  #define config_enabled(cfg) _config_enabled(cfg)
>  #define _config_enabled(value) __config_enabled(__ARG_PLACEHOLDER_##value)
> -#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk 1, 0)
> +#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk (1), (0))
>  #define ___config_enabled(__ignored, val, ...) val

This one hunk I suggest we deviate rather than adjust.  You've subtly
broken it, and it's extreme preprocessor magic in the first place to
turn an absent symbol into a 0.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:17:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:17:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687281.1070508 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfj5o-0005mU-UE; Thu, 29 Feb 2024 16:17:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687281.1070508; Thu, 29 Feb 2024 16:17:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfj5o-0005mN-RT; Thu, 29 Feb 2024 16:17:08 +0000
Received: by outflank-mailman (input) for mailman id 687281;
 Thu, 29 Feb 2024 16:17:07 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygh/=KG=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rfj5n-0005mF-Nv
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:17:07 +0000
Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com
 [2a00:1450:4864:20::234])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id fb890447-d71d-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:17:06 +0100 (CET)
Received: by mail-lj1-x234.google.com with SMTP id
 38308e7fff4ca-2d2628e81b8so20852071fa.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:17:06 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 h10-20020a19700a000000b00511ad9e3f25sm296596lfc.286.2024.02.29.08.17.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 08:17:05 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: fb890447-d71d-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709223426; x=1709828226; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=lDvdrrKupf26n1kpUObIgUkeaJWeXR3TDF1KarxNepM=;
        b=DaKOM4kxBSfmz8JPvULRlVEPHkA0Gq07SnkNHSu1/aNywVlc7+4WK4L5n0ZqZoPHBB
         4uDQY7gffaxE6d8C/8EwZL07+LJly6PKhQvRR64WgBfAmeQnnhW3m/alOokV6rSBXEal
         GWBZ7biGSLW/3VZyVE/ke07cztOiCZPYvYma/ltusJNHlxUilBB3gS/Jao3f/wTTYvrv
         YF7zRu2+oWjoqpBDfICQ7fU1aEbmGTd3NBXHCwyRZI8MvKFR5s7cbpj5omyc7As/gVZ3
         w4mREvkoV9YCOgR5CK4SrZccFVG1aS9srD9cMwDB77CxqY5ICUZIbJIKZAlbZs0gpBn3
         HjCg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709223426; x=1709828226;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=lDvdrrKupf26n1kpUObIgUkeaJWeXR3TDF1KarxNepM=;
        b=YFT/DzqljMcT58lPc5ufh3bTYri/jlc4JCH/pgXcAKxASigB5R47qVqGY2EOtakAdk
         TEDnr4NN+XbMqaRijH37GKoERJ7yjxd8C8lsFCYvuIzuj6dHl0xl59Yhaaw+ijgI3G3o
         2ZYS24r08G6IPh1gGTn3kTGzcvZlUGADySjbKtW1TVPB/pxQiFdzdgzRJyFwZy6FmHrE
         QiFKOYad8Pa/mDEDXn+MWgZywWCGaXxcmxs1wO4/c+q9kOqw4nUgqp62wrmTK9jHwdfh
         J31k0yoDDaJkyKocGInCOgNV6VL6YGCPczNtl0AuM5piIiGqJrSiMbl4uxVNAYTsVF2h
         7gKA==
X-Forwarded-Encrypted: i=1; AJvYcCVLDEkL93OR6ELVF9NmE3XwXJ+RTRClzXzS7p5T2sm/wFeMaF42lULuTm/TzU5FrRjlY3A3VeOZXyCwEgm/o6eMpS6NqcU3uGRC811/ac8=
X-Gm-Message-State: AOJu0YwVIpZ+OfhMWMtOdbGoQhNtNq6l9lgj8zEyg07iygfUvN6e9G/M
	HU03g1HsOMaA6BuEU/ZUd9wavGW3kVy50PzqtgqgXt7lNIYDDev68gWS9WUu
X-Google-Smtp-Source: AGHT+IFXXTqppu0ZXr20LcNnWEM2+s2FEA+KOjKmnnH6EUf7YEIrcBBAjaFLWoC8vkKGjon5p5Xpbw==
X-Received: by 2002:a05:6512:40b:b0:512:db3c:702f with SMTP id u11-20020a056512040b00b00512db3c702fmr944258lfk.9.1709223426035;
        Thu, 29 Feb 2024 08:17:06 -0800 (PST)
Message-ID: <da22899de22aebf092103801b2b0421c1f86aaf4.camel@gmail.com>
Subject: Re: [PATCH v5 04/23] xen/asm-generic: introduce generic fls() and
 flsl() functions
From: Oleksii <oleksii.kurochko@gmail.com>
To: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, Stefano
 Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Date: Thu, 29 Feb 2024 17:17:05 +0100
In-Reply-To: <e22ca712-b8cf-4bc9-87df-22cbd7819733@xen.org>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
	 <df7ab5055ef08fa595f913072302770a3f6a5c33.1708962629.git.oleksii.kurochko@gmail.com>
	 <e22ca712-b8cf-4bc9-87df-22cbd7819733@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

Hi Julien,

On Thu, 2024-02-29 at 13:54 +0000, Julien Grall wrote:
> Hi Oleksii,
>=20
> On 26/02/2024 17:38, Oleksii Kurochko wrote:
> > These functions can be useful for architectures that don't
> > have corresponding arch-specific instructions.
> >=20
> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > =C2=A0 Changes in V5:
> > =C2=A0=C2=A0=C2=A0 - new patch
> > ---
> > =C2=A0 xen/include/asm-generic/bitops/fls.h=C2=A0 | 18 ++++++++++++++++=
++
> > =C2=A0 xen/include/asm-generic/bitops/flsl.h | 10 ++++++++++
>=20
> One header per function seems a little bit excessive to me. Do you
> have=20
> any pointer where this request is coming from?
The goal was to be in sync with Linux kernel as Jan mentioned.
I will update the commit message as you suggested in one of replies.

>=20
> Why not using the pattern.
>=20
> In arch implementation:
>=20
> #define fls
> static inline ...
>=20
> In the generic header (asm-generic or xen/):
>=20
> #ifndef fls
> static inline ...
> #endif
>=20
> > =C2=A0 2 files changed, 28 insertions(+)
> > =C2=A0 create mode 100644 xen/include/asm-generic/bitops/fls.h
> > =C2=A0 create mode 100644 xen/include/asm-generic/bitops/flsl.h
> >=20
> > diff --git a/xen/include/asm-generic/bitops/fls.h
> > b/xen/include/asm-generic/bitops/fls.h
> > new file mode 100644
> > index 0000000000..369a4c790c
> > --- /dev/null
> > +++ b/xen/include/asm-generic/bitops/fls.h
> > @@ -0,0 +1,18 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
>=20
> You should use GPL-2.0-only.
Sure, I'll update the license here and in other files. I automatically
copied this SPDX from Linux kernel.

>=20
> > +#ifndef _ASM_GENERIC_BITOPS_FLS_H_
> > +#define _ASM_GENERIC_BITOPS_FLS_H_
> > +
> > +/**
> > + * fls - find last (most-significant) bit set
> > + * @x: the word to search
> > + *
> > + * This is defined the same way as ffs.
> > + * Note fls(0) =3D 0, fls(1) =3D 1, fls(0x80000000) =3D 32.
> > + */
> > +
> > +static inline int fls(unsigned int x)
> > +{
> > +=C2=A0=C2=A0=C2=A0 return generic_fls(x);
> > +}
> > +
> > +#endif /* _ASM_GENERIC_BITOPS_FLS_H_ */
>=20
> Missing emacs magic. I am probably not going to repeat this remark
> and=20
> the one above again. So please have a look.
Sure, I'll update files with emacs magic.

~ Oleksii
>=20
> > diff --git a/xen/include/asm-generic/bitops/flsl.h
> > b/xen/include/asm-generic/bitops/flsl.h
> > new file mode 100644
> > index 0000000000..d0a2e9c729
> > --- /dev/null
> > +++ b/xen/include/asm-generic/bitops/flsl.h
> > @@ -0,0 +1,10 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef _ASM_GENERIC_BITOPS_FLSL_H_
> > +#define _ASM_GENERIC_BITOPS_FLSL_H_
> > +
> > +static inline int flsl(unsigned long x)
> > +{
> > +=C2=A0=C2=A0=C2=A0 return generic_flsl(x);
> > +}
> > +
> > +#endif /* _ASM_GENERIC_BITOPS_FLSL_H_ */
>=20
> Cheers,
>=20



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:21:25 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:21:25 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687284.1070517 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfj9w-0007eu-F6; Thu, 29 Feb 2024 16:21:24 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687284.1070517; Thu, 29 Feb 2024 16:21:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfj9w-0007en-CD; Thu, 29 Feb 2024 16:21:24 +0000
Received: by outflank-mailman (input) for mailman id 687284;
 Thu, 29 Feb 2024 16:21:23 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfj9v-0007eh-QW
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:21:23 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 936535ef-d71e-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:21:21 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 4E10D4EE0737;
 Thu, 29 Feb 2024 17:21:21 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 936535ef-d71e-11ee-a1ee-f123f15fe8a2
MIME-Version: 1.0
Date: Thu, 29 Feb 2024 17:21:21 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org,
 michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com,
 consulting@bugseng.com, jbeulich@suse.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, julien@xen.org, George Dunlap
 <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH 01/10] xen/include: address violations of MISRA C Rule
 20.7
In-Reply-To: <b844e28f-a53b-49c0-b4b9-297367546bd3@citrix.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <15d6e4fb5c873e7ea42cfcee2faa0bf33c10d101.1709219010.git.nicola.vetrini@bugseng.com>
 <b844e28f-a53b-49c0-b4b9-297367546bd3@citrix.com>
Message-ID: <d9674a6e2ef00457a90d570fc3866c81@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

On 2024-02-29 17:10, Andrew Cooper wrote:
> On 29/02/2024 3:27 pm, Nicola Vetrini wrote:
>> diff --git a/xen/include/xen/kconfig.h b/xen/include/xen/kconfig.h
>> index c25dc0f6c2a9..b7e70289737b 100644
>> --- a/xen/include/xen/kconfig.h
>> +++ b/xen/include/xen/kconfig.h
>> @@ -25,7 +25,7 @@
>>  #define __ARG_PLACEHOLDER_1 0,
>>  #define config_enabled(cfg) _config_enabled(cfg)
>>  #define _config_enabled(value) 
>> __config_enabled(__ARG_PLACEHOLDER_##value)
>> -#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk 
>> 1, 0)
>> +#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk 
>> (1), (0))
>>  #define ___config_enabled(__ignored, val, ...) val
> 
> This one hunk I suggest we deviate rather than adjust.  You've subtly
> broken it, and it's extreme preprocessor magic in the first place to
> turn an absent symbol into a 0.
> 

How so? I did test this because I was very wary of it, but it seemed to 
expand fine (either if ((0)) or if ((1)) ). I may of course be wrong, 
and it could be deviated regardless.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:25:44 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:25:44 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687286.1070527 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjE3-0000DY-Ue; Thu, 29 Feb 2024 16:25:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687286.1070527; Thu, 29 Feb 2024 16:25:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjE3-0000DR-Rq; Thu, 29 Feb 2024 16:25:39 +0000
Received: by outflank-mailman (input) for mailman id 687286;
 Thu, 29 Feb 2024 16:25:39 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfjE2-0000DL-Ue
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:25:38 +0000
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com
 [2a00:1450:4864:20::62b])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2c323a2e-d71f-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:25:38 +0100 (CET)
Received: by mail-ej1-x62b.google.com with SMTP id
 a640c23a62f3a-a3566c0309fso161669666b.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:25:38 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 i2-20020a1709064ec200b00a3eb1b1896bsm829356ejv.58.2024.02.29.08.25.35
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:25:36 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2c323a2e-d71f-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709223937; x=1709828737; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=9UXKK7taF2IuAOWtnYqFL0txKPc2o4FeSg+6PljMdfc=;
        b=AzTPy7xrQX9ek+f+NczjXbFfA9oJ28GO5dtBAZV9FlNTxbgoFq+Nd1+t4BEbNun258
         fDFI82/5RgPeoLiBj3GBx0Eg9kilPGrIAAG9XFHHQ4N8moam/ApL09KRsWP8qcwu0YQe
         d6Qu38iMKjRlboV2QYpgxgawH0bkbh4p4y1Hk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709223937; x=1709828737;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=9UXKK7taF2IuAOWtnYqFL0txKPc2o4FeSg+6PljMdfc=;
        b=N1GWQmlwZ+gmQTNlADsNlF4O/w+RwSSpdnVSCiq4xlMd7HLp4URfylz6WawLY9BK0w
         BiWKcMHeG7PyV0kBmZW80cNF/QFU95L6Igvb/wBcLPkfUBAUaGp9meF1Syx93pvIUgmQ
         OlUNko53uLtCjOWQqLgtrS+L7LGY0pJ9QKVXsCeU4qppyHcQ3QM+Y26DG8eIO9EBBNGP
         M1CHHzJrEuyb9s2W5lfCa0R1W6gbB0ldfT5wccYJsIQZmJOd9OkperF/TlM///mhwlgU
         tc6mi5PG82eV1JP//6+u5K1I4zgwoEfh4hTS2mMSoHth0OThg1Sg7O3ccOpiiUji4Z+P
         fpug==
X-Forwarded-Encrypted: i=1; AJvYcCWJi0kcZrjwe4hT9d/ee+OVx4VZC1JJTXiUuJTsT/MfjrhoTMx4jWVObJixww6wLgsdKTK0N6dTHudzts14AXX1ro/WA9KCFOpDgm/sFJ8=
X-Gm-Message-State: AOJu0YzUFzDos7OV+KSg1M8OpVJnL6YmnxtLRDE2szlbz/v6MGbELapx
	DRU4GxQMFq+mQEXSRPPtnrgsaTIaLka6gpCP34HtJV39eoY/rYv8MsNtnOeZ0yU=
X-Google-Smtp-Source: AGHT+IF1ZV9e+kn+sjn7bLId5b2WUuV9Mi+QdWmGTjSKUtT1pDREd3ZQkOBKSvy9TaPrlv4JOI8RKw==
X-Received: by 2002:a17:906:4152:b0:a3e:e0b7:90d0 with SMTP id l18-20020a170906415200b00a3ee0b790d0mr1913406ejk.41.1709223937563;
        Thu, 29 Feb 2024 08:25:37 -0800 (PST)
Message-ID: <c4332e38-9977-4ad0-9a07-eadac7045802@citrix.com>
Date: Thu, 29 Feb 2024 16:25:35 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 04/23] xen/asm-generic: introduce generic fls() and
 flsl() functions
Content-Language: en-GB
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>,
 xen-devel@lists.xenproject.org
Cc: George Dunlap <george.dunlap@citrix.com>, Jan Beulich
 <jbeulich@suse.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <df7ab5055ef08fa595f913072302770a3f6a5c33.1708962629.git.oleksii.kurochko@gmail.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <df7ab5055ef08fa595f913072302770a3f6a5c33.1708962629.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 26/02/2024 5:38 pm, Oleksii Kurochko wrote:
> These functions can be useful for architectures that don't
> have corresponding arch-specific instructions.
>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  Changes in V5:
>    - new patch
> ---
>  xen/include/asm-generic/bitops/fls.h  | 18 ++++++++++++++++++
>  xen/include/asm-generic/bitops/flsl.h | 10 ++++++++++
>  2 files changed, 28 insertions(+)
>  create mode 100644 xen/include/asm-generic/bitops/fls.h
>  create mode 100644 xen/include/asm-generic/bitops/flsl.h
>
> diff --git a/xen/include/asm-generic/bitops/fls.h b/xen/include/asm-generic/bitops/fls.h
> new file mode 100644
> index 0000000000..369a4c790c
> --- /dev/null
> +++ b/xen/include/asm-generic/bitops/fls.h
> @@ -0,0 +1,18 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _ASM_GENERIC_BITOPS_FLS_H_
> +#define _ASM_GENERIC_BITOPS_FLS_H_
> +
> +/**
> + * fls - find last (most-significant) bit set
> + * @x: the word to search
> + *
> + * This is defined the same way as ffs.
> + * Note fls(0) = 0, fls(1) = 1, fls(0x80000000) = 32.
> + */
> +
> +static inline int fls(unsigned int x)
> +{
> +    return generic_fls(x);
> +}
> +
> +#endif /* _ASM_GENERIC_BITOPS_FLS_H_ */
> diff --git a/xen/include/asm-generic/bitops/flsl.h b/xen/include/asm-generic/bitops/flsl.h
> new file mode 100644
> index 0000000000..d0a2e9c729
> --- /dev/null
> +++ b/xen/include/asm-generic/bitops/flsl.h
> @@ -0,0 +1,10 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _ASM_GENERIC_BITOPS_FLSL_H_
> +#define _ASM_GENERIC_BITOPS_FLSL_H_
> +
> +static inline int flsl(unsigned long x)
> +{
> +    return generic_flsl(x);
> +}
> +
> +#endif /* _ASM_GENERIC_BITOPS_FLSL_H_ */

Please don't do this.  It's compounding existing problems we have with
bitops, and there's a way to simplify things instead.

If you can wait a couple of days, I'll see about finishing and posting
my prototype demonstrating a simplification across all architectures,
and a reduction of code overall.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:25:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:25:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687287.1070537 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjEJ-0000Z9-89; Thu, 29 Feb 2024 16:25:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687287.1070537; Thu, 29 Feb 2024 16:25:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjEJ-0000Z2-5I; Thu, 29 Feb 2024 16:25:55 +0000
Received: by outflank-mailman (input) for mailman id 687287;
 Thu, 29 Feb 2024 16:25:53 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjEH-0000WZ-BD
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:25:53 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 331132ca-d71f-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:25:49 +0100 (CET)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-563cb3ba9daso1563324a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:25:49 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 et5-20020a056402378500b00566a66b7409sm601971edb.16.2024.02.29.08.25.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:25:48 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 331132ca-d71f-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709223949; x=1709828749; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=m5uwW5Gs8BNeP8m0OIGGJgypUgHyrMfzDBMuxqGO/mM=;
        b=PoMvTrT0wVf1Jtk2EEuZa46OshXdMSYDUI7urEJh3D3s8dyXIY1ML9NL8kYsH0Xslg
         3MXaWQJLm3vWLQWY9zeVfnxyl1+StFSCTSn0pkz6CTJi9fGFqCBDnVI0FE2Z1xkzWUZD
         PQ7VMaHzCLosUSr1sxcU7D+Mk03oxMrOSg1eLzG6vVNej/dOqITr6OHrVcoEBW3/lEqa
         7j25VjEPhJrD9BuPoOlANJe2PCPr3KAVBl/mcd9Ek/kyVY+winqWOXZPvWS8qh4PUxql
         QdqREMqhARhND3pPMJmF/QO4cANBqRai8/ELxABsrTaqzSq5+EOcJW+hq0YZqRkvuHjE
         d9yw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709223949; x=1709828749;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=m5uwW5Gs8BNeP8m0OIGGJgypUgHyrMfzDBMuxqGO/mM=;
        b=ocStEa1Nlgi6V92dT+IzdqJX5D+dAQHm56aPMyRFI4rng2PHZpjwKCA3e6cQivCuiJ
         ZzBHDTxtfJRb8yo9HXmcFdksfCHXGCWH2vaQ8mIxoyXAceZs3Hyi/Q6KHgIGa4mW03bS
         4rDydjVa5PL8pqo4DbWQBP7oVbCS7LFX3YdbujTr240xX9R0ed/2s/Pa9bsaajzmh3Bo
         P3GGer8l3uWLNIGYQx0A0FmYpeaMmjnB+004PFEX4PWRhVb4hWH33cifwfjzdUBEckF2
         l4a51YMt5VHYBl5oMkf8DECa2DoqAUPfw/DBwpxVXMIGL0cB/o2DaFPv2FaOA/c+aeqH
         03fg==
X-Forwarded-Encrypted: i=1; AJvYcCX6NSqstkiELiEpts/dVprT3vxyTSZXZWrk0jglLTRcRGg1gObbricD8ygmMol0Bqw/eOfiVHI8gRZA0Rz1dGiLzs7a8x5CK9DjCU6h1y0=
X-Gm-Message-State: AOJu0YxzkYRg4p3bCgTCqUaND/x26uzYROx+xVAo591ShSrnf/KsFJRl
	RVehiadYNGfKMR4RWANWTn+y2RExA7p3regHH1aTNzf32PQjrzpSYYKUpuQkXA==
X-Google-Smtp-Source: AGHT+IEwimYARGtTXDS8xxyeHSDO65Ehn0VSIKYV8ZFLXZJa0tE4RgdrnlhL9vxhBmyUjuN262iO2A==
X-Received: by 2002:a50:ef07:0:b0:566:2a1a:fe18 with SMTP id m7-20020a50ef07000000b005662a1afe18mr1953136eds.36.1709223949170;
        Thu, 29 Feb 2024 08:25:49 -0800 (PST)
Message-ID: <26be05f7-7361-40d9-92f2-cf2e22da9d4e@suse.com>
Date: Thu, 29 Feb 2024 17:25:47 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 01/10] xen/include: address violations of MISRA C Rule
 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <15d6e4fb5c873e7ea42cfcee2faa0bf33c10d101.1709219010.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <15d6e4fb5c873e7ea42cfcee2faa0bf33c10d101.1709219010.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 16:27, Nicola Vetrini wrote:
> --- a/xen/include/xen/kconfig.h
> +++ b/xen/include/xen/kconfig.h
> @@ -25,7 +25,7 @@
>  #define __ARG_PLACEHOLDER_1 0,
>  #define config_enabled(cfg) _config_enabled(cfg)
>  #define _config_enabled(value) __config_enabled(__ARG_PLACEHOLDER_##value)
> -#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk 1, 0)
> +#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk (1), (0))
>  #define ___config_enabled(__ignored, val, ...) val

In addition to what Andrew said, would you mind clarifying what exactly the
violation is here? I find it questionable that numeric literals need
parenthesizing; they don't normally need to, aynwhere.

> --- a/xen/include/xen/list.h
> +++ b/xen/include/xen/list.h
> @@ -490,9 +490,9 @@ static inline void list_splice_init(struct list_head *list,
>   * @member: the name of the list_struct within the struct.
>   */
>  #define list_for_each_entry(pos, head, member)                          \
> -    for (pos = list_entry((head)->next, typeof(*pos), member);          \
> -         &pos->member != (head);                                        \
> -         pos = list_entry(pos->member.next, typeof(*pos), member))
> +    for (pos = list_entry((head)->next, typeof(*(pos)), member);          \
> +         &(pos)->member != (head);                                      \
> +         pos = list_entry((pos)->member.next, typeof(*(pos)), member))

this ends up inconsistent, which I think isn't nice: Some uses of "pos"
are now parenthesized, while others aren't. Applies further down as well.

You may also want to take this as a strong suggestion to split dissimilar
changes, so uncontroversial parts can go in.

> @@ -977,4 +977,3 @@ static inline void hlist_add_after_rcu(struct hlist_node *prev,
>            pos = pos->next)
>  
>  #endif /* __XEN_LIST_H__ */
> -

Unrelated change?

> --- a/xen/include/xen/spinlock.h
> +++ b/xen/include/xen/spinlock.h
> @@ -94,7 +94,7 @@ struct lock_profile_qhead {
>      int32_t                   idx;     /* index for printout */
>  };
>  
> -#define _LOCK_PROFILE(lockname) { .name = #lockname, .lock = &lockname, }
> +#define _LOCK_PROFILE(lockname) { .name = #lockname, .lock = &(lockname), }

This also may be viewed as falling in the same category, but is less
problematic because the other use is stringification, when in principle
some kind of expression would be passed in (albeit in practice I don't
expect anyone would do that).

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:28:04 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:28:04 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687294.1070548 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjGN-0001Jf-JC; Thu, 29 Feb 2024 16:28:03 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687294.1070548; Thu, 29 Feb 2024 16:28:03 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjGN-0001JY-GK; Thu, 29 Feb 2024 16:28:03 +0000
Received: by outflank-mailman (input) for mailman id 687294;
 Thu, 29 Feb 2024 16:28:01 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjGL-0001JM-Ga
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:28:01 +0000
Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com
 [2a00:1450:4864:20::62a])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 811e2fdd-d71f-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:28:00 +0100 (CET)
Received: by mail-ej1-x62a.google.com with SMTP id
 a640c23a62f3a-a3d5e77cfbeso207059066b.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:28:00 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 d25-20020a17090648d900b00a4339b8b1bbsm825064ejt.212.2024.02.29.08.27.59
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:27:59 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 811e2fdd-d71f-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709224080; x=1709828880; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=/gpsXEOg1wVyzd7WtANsuYmOpKxJtCLX7vfBXtObqbA=;
        b=ALm2q60QEwGw0TBrlrvnxafqAP0AR3bk1mwIuIBf+pJhpz5YXesy/+ON96qVFAKiuA
         E2WgcYc/3ThuUNkX78nktBlJ69ro4Upf4OWJmUyT2SDiMA6ct7fgo3UGLrNblMNATruh
         DyKebWj0xZ+quOSnRe8nmLS/2Q0YTA4N2DSPt+4iP2d0eIllDunNIgpQcDDfWH/2LBQv
         fuJ/HEMORjZr0Ytevn+ZTTtGfmSON12x/TZODMHWmCNbSblZtdx12ghZrLx5Z7CfItbJ
         Y9ZTldNvNBD1o+mk9iO4V7G3jj6jGs5M4kfZrw135qG80SpiZKz+rXi5yuvEzkt+tC/Q
         6AXw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709224080; x=1709828880;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=/gpsXEOg1wVyzd7WtANsuYmOpKxJtCLX7vfBXtObqbA=;
        b=tilc3T40SOebv4xMYvGJv6uoKTaTMaq1Cm62xUbvp0z51W1voWWV9x2QXAUzIynp+1
         N4/mauAHjQarWMm0eSgbVYbL+eSSzzF/J6UukE/CW7Lz0QTwq2zLOOrVC4hzXmg0N+f3
         hJYBIuwJPbJZ9GLwclFApDhqrEG4fSY1WCN+T18zpJFL4GuqgIgg324gsZXuRwM0CnTT
         DZ69gDEPNdlOQchQbLGOUo0gffhrfMnK7d8RncDpjC0LGqmiua6vjhizs8cSwmjv0wbm
         b5PK/hheCm/Lk4kpT5LV596W2rKcsmaGPP8smWGunEaHFgz6rAs22cvko1sWvvz/R1gM
         OilQ==
X-Forwarded-Encrypted: i=1; AJvYcCV0wcQolo5sw/3SLGA3SnuJLfVrCW1BXMgl5ZZHyi59mpVp276xNm+9u8yZr03LZjfgua2IFezIHXggt9N9teikFqkC8ru3tVyfcLukFAA=
X-Gm-Message-State: AOJu0YxwQymIquPE5vqZHQygmLy61kYqnH50djSFImbRejPZvkEmLJMi
	Oeg+cosL9x/nFALVK/mzNB5cGU1vels3DbrO0UBCvd9pm2bSgghOkOdFbss9UA==
X-Google-Smtp-Source: AGHT+IFsiKEkfrYuyIrR0wHTeBE5tIhC7xBIJeyqGrSnNonDwVwfvSYtclrBhEuQmN6IKOBVl88qoA==
X-Received: by 2002:a17:906:eb10:b0:a44:4e64:d12c with SMTP id mb16-20020a170906eb1000b00a444e64d12cmr1273591ejb.35.1709224080061;
        Thu, 29 Feb 2024 08:28:00 -0800 (PST)
Message-ID: <48fe3e5b-13a4-40cf-8c4a-37f2d0eaddc9@suse.com>
Date: Thu, 29 Feb 2024 17:27:58 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v5 03/23] xen/riscv: introduce nospec.h
Content-Language: en-US
To: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alistair Francis <alistair.francis@wdc.com>,
 Bob Eshleman <bobbyeshleman@gmail.com>, Connor Davis
 <connojdavis@gmail.com>, Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
 <16370492de5344ae8e1aab688261600491868f4e.1708962629.git.oleksii.kurochko@gmail.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <16370492de5344ae8e1aab688261600491868f4e.1708962629.git.oleksii.kurochko@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 26.02.2024 18:38, Oleksii Kurochko wrote:
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/nospec.h
> @@ -0,0 +1,25 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/* Copyright (C) 2024 Vates */
> +
> +#ifndef _ASM_GENERIC_NOSPEC_H
> +#define _ASM_GENERIC_NOSPEC_H

Btw, at the very last second I noticed the GENERIC in here, which I
took the liberty to replace. But please be more careful when moving
files around in the tree.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:29:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:29:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687296.1070558 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjHO-0001sZ-TD; Thu, 29 Feb 2024 16:29:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687296.1070558; Thu, 29 Feb 2024 16:29:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjHO-0001sS-PI; Thu, 29 Feb 2024 16:29:06 +0000
Received: by outflank-mailman (input) for mailman id 687296;
 Thu, 29 Feb 2024 16:29:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rfjHN-0001sI-Uc
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:29:05 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id a6ad33fa-d71f-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:29:03 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id 169191F7FB;
 Thu, 29 Feb 2024 16:29:03 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id C7A4013A4B;
 Thu, 29 Feb 2024 16:29:02 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id Y1jZLs6w4GXTPgAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 16:29:02 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a6ad33fa-d71f-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709224143; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=sKdLELKEgs3KqopRitC+wX3U/F20nUCJT7pBkAQ+XZ4=;
	b=NrI+tDjouDihNdoeIn2r+m/tOxVlzCx16gStSxEX8mdUYgkIYd3//IYFVJSMyo9jp2c2Br
	aa0pDe7G8d7KR/7LDJZ+BwdosKbX/199JAiA7twvLs45bEQSgAXCe9zizA7hyieAcUndvP
	Dddlh+efRi1Z84ZeEUg/bd2MwtcZc7I=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709224143; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=sKdLELKEgs3KqopRitC+wX3U/F20nUCJT7pBkAQ+XZ4=;
	b=NrI+tDjouDihNdoeIn2r+m/tOxVlzCx16gStSxEX8mdUYgkIYd3//IYFVJSMyo9jp2c2Br
	aa0pDe7G8d7KR/7LDJZ+BwdosKbX/199JAiA7twvLs45bEQSgAXCe9zizA7hyieAcUndvP
	Dddlh+efRi1Z84ZeEUg/bd2MwtcZc7I=
Message-ID: <f6658df0-13f2-4129-afcd-0f122c87132b@suse.com>
Date: Thu, 29 Feb 2024 17:29:02 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/12] xen/spinlock: support higher number of cpus
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-13-jgross@suse.com>
 <c96542eb-daf4-4ce2-915e-86ca8e1848b5@suse.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <c96542eb-daf4-4ce2-915e-86ca8e1848b5@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -0.59
X-Spamd-Result: default: False [-0.59 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-0.01)[51.54%];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.71)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Flag: NO

On 29.02.24 16:46, Jan Beulich wrote:
> On 12.12.2023 10:47, Juergen Gross wrote:
>> Allow 16 bits per cpu number, which is the limit imposed by
>> spinlock_tickets_t.
>>
>> This will allow up to 65535 cpus, while increasing only the size of
>> recursive spinlocks in debug builds from 8 to 12 bytes.
> 
> I think we want to be more conservative here, for the case of there
> being bugs: The CPU holding a lock may wrongly try to acquire it a
> 2nd time. That's the 65536th ticket then, wrapping the value.

Is this really a problem? There will be no other cpu left seeing the lock
as "free" in this case, as all others will be waiting for the head to reach
their private tail value.

> Therefore my suggestion would be to only (mention) go(ing) up to 32k.
> 
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   xen/common/spinlock.c      |  1 +
>>   xen/include/xen/spinlock.h | 18 +++++++++---------
>>   2 files changed, 10 insertions(+), 9 deletions(-)
> 
> Shouldn't this also bump the upper bound of the NR_CPUS range then
> in xen/arch/Kconfig?

Fine with me, I can add another patch to the series doing that.


Juergen


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:31:47 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:31:47 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687300.1070568 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjJt-0003rU-8X; Thu, 29 Feb 2024 16:31:41 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687300.1070568; Thu, 29 Feb 2024 16:31:41 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjJt-0003rN-5K; Thu, 29 Feb 2024 16:31:41 +0000
Received: by outflank-mailman (input) for mailman id 687300;
 Thu, 29 Feb 2024 16:31:39 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjJr-0003rC-QK
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:31:39 +0000
Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com
 [2a00:1450:4864:20::52e])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 02ba1a52-d720-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:31:38 +0100 (CET)
Received: by mail-ed1-x52e.google.com with SMTP id
 4fb4d7f45d1cf-564a53b8133so1648771a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:31:38 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 5-20020a0564021f4500b00564d7d23919sm745468edz.67.2024.02.29.08.31.37
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:31:37 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 02ba1a52-d720-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709224297; x=1709829097; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:from:to:cc:subject:date:message-id:reply-to;
        bh=RGy2hlIVhrmudPHpZlR5MZzLrNNufU61H/AK1P74Fbo=;
        b=JPrTM6y9C8AwM1nrUdWET/zk3U7s/NiqF3JQEhd3u/QODI5dqQ+51dPZGqz7hMrScr
         g7Tvu+Wd/+PKWXzvhG4wQyTCBfYrSs2JtH6TbZDb1tjbLSDtj2lPyLapY2RQ7r3DM2I7
         V87Wf1rtEl6GNYn7uLMSqO/5WRUSWtT9CkfH/mZTQcGM3IUOV0Z4FlfJZAMMMdW0XFvx
         YlNwvCA8pL0qFXUPfJPiQGziNeUfidJKGP3OLYDWWvS/m3KI6hCZde241ysCqKSDhsnN
         B3GkalRCPHZ+2qudp40DBbMbElNQqacmNIhovP20A7gTmzB+G8Mc2BF/Px8/rQuupPK2
         KDwg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709224297; x=1709829097;
        h=content-transfer-encoding:in-reply-to:autocrypt:from
         :content-language:references:cc:to:subject:user-agent:mime-version
         :date:message-id:x-gm-message-state:from:to:cc:subject:date
         :message-id:reply-to;
        bh=RGy2hlIVhrmudPHpZlR5MZzLrNNufU61H/AK1P74Fbo=;
        b=ZqRYt4kFsJad2UQzmVmGVVEfGeB+HFNMPsh4zBiQfwDdVZ4hV6ld03wf1PtOV/urR3
         17E7LhaLRUDH7n/AXckyolKoVWxCq2gWB7HRwgGmf6fEksAGB2OhA/jkPu6klYwJ7zNm
         sVCNRIzrXR0TaXX2BM20yxItYzw43FXgzvGX9njL7x99L5Supc9x7Y9ZgNcIEnI/Yvfe
         Hp5kXQY+fEOyD458j9WksKtDavggskeZxSEcKycWGpwMjIoxoQkg/5Tg52MS+fBrnuqy
         133jLjrrodPtrP3qYN9bDeEmX5gGLDJfhg326cPgadeInuRS4JDI5G90jdGtYTcnCBLA
         Cdbw==
X-Forwarded-Encrypted: i=1; AJvYcCXojGxHE+CuiZwFNH3kxoALCO9nWYCOvUdsDqPq+epIM5NizlJYRgmq9fsKlUQ2joS3prNYYMKSGtCnIsd2qG4+1S9IvXTBcodWn2y4avQ=
X-Gm-Message-State: AOJu0Yy8sFUmoU/bz9If9bgTs/MMiXVtd6Z9T47hhOozQkc2cqywATP8
	ov5NAmiIOb6sLL5kI80q0JaFW7kn2W2xfpHZNkdJJKThAZgnqTBsGoAAabEs6AxH+4EVRltCK6E
	=
X-Google-Smtp-Source: AGHT+IHPKDmd2Vdci4CGTmANuYERFwwOK7TAPV8KVTqpJO/13tZRWuIcYaCFtrvyU0tXZz7pGL7oRw==
X-Received: by 2002:a50:c88c:0:b0:565:7d4a:1d4a with SMTP id d12-20020a50c88c000000b005657d4a1d4amr1647414edh.8.1709224297588;
        Thu, 29 Feb 2024 08:31:37 -0800 (PST)
Message-ID: <f2d107a1-0bcf-4d71-89e8-819ee38e0651@suse.com>
Date: Thu, 29 Feb 2024 17:31:36 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/12] xen/spinlock: support higher number of cpus
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-13-jgross@suse.com>
 <c96542eb-daf4-4ce2-915e-86ca8e1848b5@suse.com>
 <f6658df0-13f2-4129-afcd-0f122c87132b@suse.com>
Content-Language: en-US
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <f6658df0-13f2-4129-afcd-0f122c87132b@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 17:29, Jürgen Groß wrote:
> On 29.02.24 16:46, Jan Beulich wrote:
>> On 12.12.2023 10:47, Juergen Gross wrote:
>>> Allow 16 bits per cpu number, which is the limit imposed by
>>> spinlock_tickets_t.
>>>
>>> This will allow up to 65535 cpus, while increasing only the size of
>>> recursive spinlocks in debug builds from 8 to 12 bytes.
>>
>> I think we want to be more conservative here, for the case of there
>> being bugs: The CPU holding a lock may wrongly try to acquire it a
>> 2nd time. That's the 65536th ticket then, wrapping the value.
> 
> Is this really a problem? There will be no other cpu left seeing the lock
> as "free" in this case, as all others will be waiting for the head to reach
> their private tail value.

But isn't said CPU then going to make progress, rather than indefinitely
spinning on the lock?

>> Therefore my suggestion would be to only (mention) go(ing) up to 32k.
>>
>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>> ---
>>>   xen/common/spinlock.c      |  1 +
>>>   xen/include/xen/spinlock.h | 18 +++++++++---------
>>>   2 files changed, 10 insertions(+), 9 deletions(-)
>>
>> Shouldn't this also bump the upper bound of the NR_CPUS range then
>> in xen/arch/Kconfig?
> 
> Fine with me, I can add another patch to the series doing that.

Why not do it right here? The upper bound there is like it is only
because of the restriction that's lifted here.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:34:59 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:34:59 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687303.1070578 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjN4-0004x4-MT; Thu, 29 Feb 2024 16:34:58 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687303.1070578; Thu, 29 Feb 2024 16:34:58 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjN4-0004wx-Jv; Thu, 29 Feb 2024 16:34:58 +0000
Received: by outflank-mailman (input) for mailman id 687303;
 Thu, 29 Feb 2024 16:34:57 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjN3-0004wp-5v
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:34:57 +0000
Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com
 [2a00:1450:4864:20::630])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 78392d6d-d720-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:34:55 +0100 (CET)
Received: by mail-ej1-x630.google.com with SMTP id
 a640c23a62f3a-a3fb8b0b7acso170568966b.2
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:34:55 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 y1-20020a1709060bc100b00a43aa27b5f2sm840849ejg.94.2024.02.29.08.34.54
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:34:54 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 78392d6d-d720-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709224495; x=1709829295; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=ddklw9Fup78ZYsuy+7Gip+a/kwQoVC/SFYGdun8k6mI=;
        b=UMVFfXppou8cdvZSTZZNRIiPbBo6Y0bkvGQe5/GrNwfYlqajNv1oz6/09Mk/U5iMic
         AAFbImTDLqWNn0ptzOPcIEexGcXnyms22E/qsToSJJ3BFVw7nWmF4Bz30ke9QlsoFD0V
         +Qd9JIk2YGSJYf42njMthqPpDCIal/N3o/COr5ouagzFJCEv2H8gw8XwbYuQhV3MtmDm
         wnUnFNlK0atMm5ZUoUFIkEZj+kyMYH3POmopxChgu9yd5H+V7Xb6Aop069BaoBIwJFHS
         jjeLNBEwBoUbi0zvag4lzpKrWOvlndNaWohM0qZ+8FSGZHH6eHE1+bqfxgoY5bTk7AID
         MlXQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709224495; x=1709829295;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=ddklw9Fup78ZYsuy+7Gip+a/kwQoVC/SFYGdun8k6mI=;
        b=G8uQPXUvm5d8TouHnfzXbVVnYeZZc6R4OAr13P4kFMGnQoNRt+idKg4rXYBMA64nNZ
         V1CPXNJ6I35ZOmuNSLxbDN3LfgsYgvgaQQ/ngmdILFSZF/1TALwxCAqCvEYiEB6i3tlb
         6VU5EhewnghR8GyFOq9vEReae73bx4JpqQ3pBztMvWurj1Y4x1/XjzoPbhZYSzQUZP60
         yGXHHMxJuXoQ9ja89TjApuwtYi+yOdZUnim9v7eZJdiEv38Mxbc+UggTK0fOpGBsTfad
         K7RE1+WvcSoFsxOoxKjkXD6NXysVI+Jly3tTEbhExILzyOBpIbRc0GwYILPGGzFLLeK4
         7b9w==
X-Forwarded-Encrypted: i=1; AJvYcCW/7cUfGKv6ZXnmOjCko30zEJ2f3t8Eo3UIEH3/hN9x1CkQnkovgWd+ysZ4qhfucBdOIj+rKcJb2mBOsAnmwmW0ynOozJgVR2tp7+ZnMn4=
X-Gm-Message-State: AOJu0YwxplsnrMtmI7YlRYFFrECY4YyoY72aO+ouwD8kLa5FN6bfERdM
	dlImD7ERz08ZoeXwT5UBLzpyztlTyb0ZfXetQ+pJMSJ4UmPpX+AHk+rT6ze7GA==
X-Google-Smtp-Source: AGHT+IHDYl+4GMETDUgdjWma9SC4pD46U6M1GZiv/Rujt98Jl8UYYqdMOcIJH0lYvJUY1kLtDrjNIw==
X-Received: by 2002:a17:906:3709:b0:a44:5258:ff3c with SMTP id d9-20020a170906370900b00a445258ff3cmr1299883ejc.1.1709224494674;
        Thu, 29 Feb 2024 08:34:54 -0800 (PST)
Message-ID: <905119be-8731-4669-ac7f-c21aed6845dc@suse.com>
Date: Thu, 29 Feb 2024 17:34:53 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 02/10] xen/arm: address some violations of MISRA C
 Rule 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
 xen-devel@lists.xenproject.org
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <efdeb0e013c36e18b5545fbdb33a43bb3f87039c.1709219010.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <efdeb0e013c36e18b5545fbdb33a43bb3f87039c.1709219010.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 16:27, Nicola Vetrini wrote:
> --- a/xen/arch/arm/cpuerrata.c
> +++ b/xen/arch/arm/cpuerrata.c
> @@ -462,8 +462,8 @@ static bool has_ssbd_mitigation(const struct arm_cpu_capabilities *entry)
>  #define MIDR_RANGE(model, min, max)     \
>      .matches = is_affected_midr_range,  \
>      .midr_model = model,                \
> -    .midr_range_min = min,              \
> -    .midr_range_max = max
> +    .midr_range_min = (min),            \
> +    .midr_range_max = (max)

Why min and max, but not model?

> --- a/xen/arch/arm/include/asm/smccc.h
> +++ b/xen/arch/arm/include/asm/smccc.h
> @@ -122,7 +122,7 @@ struct arm_smccc_res {
>  #define __constraint_read_7 __constraint_read_6, "r" (r7)
>  
>  #define __declare_arg_0(a0, res)                            \
> -    struct arm_smccc_res    *___res = res;                  \
> +    struct arm_smccc_res    *___res = (res);                \
>      register unsigned long  r0 ASM_REG(0) = (uint32_t)a0;   \

Why res but not a0?

> --- a/xen/arch/arm/include/asm/vgic-emul.h
> +++ b/xen/arch/arm/include/asm/vgic-emul.h
> @@ -6,11 +6,11 @@
>   * a range of registers
>   */
>  
> -#define VREG32(reg) reg ... reg + 3
> -#define VREG64(reg) reg ... reg + 7
> +#define VREG32(reg) (reg) ... (reg) + 3
> +#define VREG64(reg) (reg) ... (reg) + 7

#define VREG32(reg) (reg) ... ((reg) + 3)
#define VREG64(reg) (reg) ... ((reg) + 7)

?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:37:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:37:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687306.1070588 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjPj-0005Xv-7E; Thu, 29 Feb 2024 16:37:43 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687306.1070588; Thu, 29 Feb 2024 16:37:43 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjPj-0005Xk-4f; Thu, 29 Feb 2024 16:37:43 +0000
Received: by outflank-mailman (input) for mailman id 687306;
 Thu, 29 Feb 2024 16:37:41 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjPh-0005Xa-CV
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:37:41 +0000
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com
 [2a00:1450:4864:20::534])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id da15868e-d720-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:37:39 +0100 (CET)
Received: by mail-ed1-x534.google.com with SMTP id
 4fb4d7f45d1cf-564fc495d83so1561556a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:37:39 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 v8-20020a1709067d8800b00a3d68aad90dsm847012ejo.97.2024.02.29.08.37.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:37:38 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: da15868e-d720-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709224659; x=1709829459; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=KqA/Dje2EInne6YGsYiMVHRKN2X20+ifwXC7sFOgaqA=;
        b=Fq6KWZ9iB/NEkWIeaQaDZbbvy6LLfg2nZ2hcTelNJG6YWVDotASLr4yRacQDKjbL0i
         OWDer4gjacjrgc5OIW9R/wb6fUaKu5CMypD6KygxycCnBnZgI5ik73uHdZjuAhPIFRPu
         h/0GIKKaBBrPAexWzhW4S6VnIk09YuyzRCsPBtN5WIBRDy6jeZqxU1oDclxTunCl7dxo
         dS/Jqd6+oQjicm3YpLma1OjBP/mBCql8I4rPpe4TOoyPpWzW7B8UyoANYh5JF44/mMbZ
         CcGYOx4z98I1d2S8v81H84D/kXlcpMUsuh3emG8ZHicWhR97oZ2GQm5hwkTWe3Mu4i39
         nqww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709224659; x=1709829459;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=KqA/Dje2EInne6YGsYiMVHRKN2X20+ifwXC7sFOgaqA=;
        b=ETZ7iCUtInJKdJRZNNx2lv4VKDDFHWlZH8fvqV1sai2cACVcFrxEpB5rVEDQ2/qd4Z
         e+uNO5Y1IOLPVLrZlPr6zb+o6Q/PgmPlydlBD5waoCpZy3tHaX3i8ghQLUxU/LVZZMjZ
         YQIDwnKdbcm6B8pbl+LPYPsyJhQqcERysjcOozMAsvpocfvoCsv/VUuvxu6Ey+4sI41l
         PeoEj+cfv4yhW2/Ff+zSHyNU3T1Lb8V6LeyDdUOM4D2ASwNviKbcd5KnHVLeTclUobuu
         sEPBOutVu0uOt4jH8Ox8d+rKuTvCKelIyJLbBcq9x1OcFHfFtLfhGT345W3d4ryiGv1e
         Wy/w==
X-Forwarded-Encrypted: i=1; AJvYcCXioJVjO0Br9lZM4pTZykG87UH+hhvqpC08maAhuDpUMyZMaR/Qt7K6GNTA6OuX883GqCWFg28PPoeQWkBzAR+IlUdmGuCBDExmkJqCSHo=
X-Gm-Message-State: AOJu0YyNliDJbdIcj2A72cTJI7SUeOJUZ7PP79WacINB4FGUv/L8OEgd
	5BczuZapdpoqkfqFEEHiUzXsXTe8pNBczNAnMiscnLq2zdrSAdWnFZpVinTeuQ==
X-Google-Smtp-Source: AGHT+IEx4oa3ceXc86TvI1ftL3nMtWsjrbF6PzqJW5o9aijhBmwOL5YPH4FVQKBopipEscUwMTvokQ==
X-Received: by 2002:a17:906:5fd3:b0:a44:48dd:bcb2 with SMTP id k19-20020a1709065fd300b00a4448ddbcb2mr1678026ejv.25.1709224658901;
        Thu, 29 Feb 2024 08:37:38 -0800 (PST)
Message-ID: <14d94c7e-7cd4-4508-a74e-eddd6a592219@suse.com>
Date: Thu, 29 Feb 2024 17:37:37 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 03/10] x86: address some violations of MISRA C Rule
 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <3c9e90aaf5dde769b689468fc818e4ae61fa11f3.1709219010.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3c9e90aaf5dde769b689468fc818e4ae61fa11f3.1709219010.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 16:27, Nicola Vetrini wrote:
> --- a/xen/arch/x86/include/asm/irq.h
> +++ b/xen/arch/x86/include/asm/irq.h
> @@ -179,9 +179,9 @@ void cleanup_domain_irq_mapping(struct domain *d);
>      void *__ret = radix_tree_lookup(&(d)->arch.hvm.emuirq_pirq, emuirq);\
>      __ret ? radix_tree_ptr_to_int(__ret) : IRQ_UNBOUND;                 \
>  })
> -#define IRQ_UNBOUND -1
> -#define IRQ_PT -2
> -#define IRQ_MSI_EMU -3
> +#define IRQ_UNBOUND (-1)
> +#define IRQ_PT      (-2)
> +#define IRQ_MSI_EMU (-3)
>  
>  bool cpu_has_pending_apic_eoi(void);
>  

I'd be happy to ack this change right away.

> --- a/xen/arch/x86/usercopy.c
> +++ b/xen/arch/x86/usercopy.c
> @@ -106,7 +106,7 @@ unsigned int copy_from_guest_ll(void *to, const void __user *from, unsigned int
>      return n;
>  }
>  
> -#if GUARD(1) + 0
> +#if GUARD((1)) + 0

I don't even understand the need for this one, and nothing is said in
the description in that regard. Generally I'm afraid I'm averse to
such (seemingly) redundant parentheses in macro invocations.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:40:07 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:40:07 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687308.1070598 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjS1-0007bZ-JT; Thu, 29 Feb 2024 16:40:05 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687308.1070598; Thu, 29 Feb 2024 16:40:05 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjS1-0007bS-Gx; Thu, 29 Feb 2024 16:40:05 +0000
Received: by outflank-mailman (input) for mailman id 687308;
 Thu, 29 Feb 2024 16:40:03 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfjRz-0007DF-RO
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:40:03 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 2f3c2ac7-d721-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:40:02 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id AA2404EE0737;
 Thu, 29 Feb 2024 17:40:01 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 2f3c2ac7-d721-11ee-afd8-a90da7624cb6
MIME-Version: 1.0
Date: Thu, 29 Feb 2024 17:40:01 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 01/10] xen/include: address violations of MISRA C Rule
 20.7
In-Reply-To: <26be05f7-7361-40d9-92f2-cf2e22da9d4e@suse.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <15d6e4fb5c873e7ea42cfcee2faa0bf33c10d101.1709219010.git.nicola.vetrini@bugseng.com>
 <26be05f7-7361-40d9-92f2-cf2e22da9d4e@suse.com>
Message-ID: <4143df23c3d1f8dcb7fcc2d97e077d01@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

On 2024-02-29 17:25, Jan Beulich wrote:
> On 29.02.2024 16:27, Nicola Vetrini wrote:
>> --- a/xen/include/xen/kconfig.h
>> +++ b/xen/include/xen/kconfig.h
>> @@ -25,7 +25,7 @@
>>  #define __ARG_PLACEHOLDER_1 0,
>>  #define config_enabled(cfg) _config_enabled(cfg)
>>  #define _config_enabled(value) 
>> __config_enabled(__ARG_PLACEHOLDER_##value)
>> -#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk 
>> 1, 0)
>> +#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk 
>> (1), (0))
>>  #define ___config_enabled(__ignored, val, ...) val
> 
> In addition to what Andrew said, would you mind clarifying what exactly 
> the
> violation is here? I find it questionable that numeric literals need
> parenthesizing; they don't normally need to, aynwhere.
> 

This one's a little special. I couldn't parenthesize val further down, 
because then it would break the build:

In file included from ././include/xen/config.h:14,
                  from <command-line>:
./include/xen/kconfig.h:29:52: error: expected identifier or ‘(’ before 
‘)’ token
    29 | #define ___config_enabled(__ignored, val, ...) (val)
       |                                                    ^
./include/xen/kconfig.h:39:34: note: in expansion of macro 
‘___config_enabled’
    39 | #define _static_if(arg1_or_junk) ___config_enabled(arg1_or_junk 
static,)
       |                                  ^~~~~~~~~~~~~~~~~
./include/xen/kconfig.h:38:26: note: in expansion of macro ‘_static_if’
    38 | #define static_if(value) _static_if(__ARG_PLACEHOLDER_##value)
       |                          ^~~~~~~~~~
./include/xen/kconfig.h:41:27: note: in expansion of macro ‘static_if’
    41 | #define STATIC_IF(option) static_if(option)
       |                           ^~~~~~~~~
common/page_alloc.c:260:1: note: in expansion of macro ‘STATIC_IF’
   260 | STATIC_IF(CONFIG_NUMA) mfn_t first_valid_mfn = 
INVALID_MFN_INITIALIZER;
       | ^~~~~~~~~
make[2]: *** [Rules.mk:249: common/page_alloc.o] Error 1


>> --- a/xen/include/xen/list.h
>> +++ b/xen/include/xen/list.h
>> @@ -490,9 +490,9 @@ static inline void list_splice_init(struct 
>> list_head *list,
>>   * @member: the name of the list_struct within the struct.
>>   */
>>  #define list_for_each_entry(pos, head, member)                        
>>   \
>> -    for (pos = list_entry((head)->next, typeof(*pos), member);        
>>   \
>> -         &pos->member != (head);                                      
>>   \
>> -         pos = list_entry(pos->member.next, typeof(*pos), member))
>> +    for (pos = list_entry((head)->next, typeof(*(pos)), member);      
>>     \
>> +         &(pos)->member != (head);                                    
>>   \
>> +         pos = list_entry((pos)->member.next, typeof(*(pos)), 
>> member))
> 
> this ends up inconsistent, which I think isn't nice: Some uses of "pos"
> are now parenthesized, while others aren't. Applies further down as 
> well.
> 

Yeah, the inconsistency is due to the fact that a non-parenthesized 
parameter as lhs is already deviated. To keep it consistent here I can 
add parentheses, but then the deviation would be kind of pointless, 
wouldn't it?

> You may also want to take this as a strong suggestion to split 
> dissimilar
> changes, so uncontroversial parts can go in.
> 

Ok, that was an oversight.

>> @@ -977,4 +977,3 @@ static inline void hlist_add_after_rcu(struct 
>> hlist_node *prev,
>>            pos = pos->next)
>> 
>>  #endif /* __XEN_LIST_H__ */
>> -
> 
> Unrelated change?
> 

Oh, yes. Will drop it.

>> --- a/xen/include/xen/spinlock.h
>> +++ b/xen/include/xen/spinlock.h
>> @@ -94,7 +94,7 @@ struct lock_profile_qhead {
>>      int32_t                   idx;     /* index for printout */
>>  };
>> 
>> -#define _LOCK_PROFILE(lockname) { .name = #lockname, .lock = 
>> &lockname, }
>> +#define _LOCK_PROFILE(lockname) { .name = #lockname, .lock = 
>> &(lockname), }
> 
> This also may be viewed as falling in the same category, but is less
> problematic because the other use is stringification, when in principle
> some kind of expression would be passed in (albeit in practice I don't
> expect anyone would do that).
> 

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:40:53 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:40:53 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687310.1070607 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjSm-0000A0-Rd; Thu, 29 Feb 2024 16:40:52 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687310.1070607; Thu, 29 Feb 2024 16:40:52 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjSm-00009r-Ov; Thu, 29 Feb 2024 16:40:52 +0000
Received: by outflank-mailman (input) for mailman id 687310;
 Thu, 29 Feb 2024 16:40:51 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjSl-00009d-UW
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:40:51 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4ba24ef8-d721-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:40:49 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-55a179f5fa1so1933921a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:40:49 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 wb6-20020a170907d50600b00a3f0dbdf106sm838118ejc.105.2024.02.29.08.40.48
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:40:49 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4ba24ef8-d721-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709224849; x=1709829649; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=66ED0Hu2okQ0qkhX0QsEVkNRBgSv+C3kCbNUGkI/Wxk=;
        b=PQ8j9O51NkBz772vYiGkxjJ5KBoIvvXf7xY4yw1j/YcM6R+oI4X3QnsvzjDu/O/iJI
         qxOgYeyYnOIoMonWEM8gS6uEv3FhNL4mER209V76STPnE+jRd8WJMt/kI0IpDkNBGY0N
         bs41ESfCRwyXAy05tj5QmBFr3ntE6FFuahlK0jiDU+s+8VXGKJ3FuK4bLVogv8W7XMp6
         wsiPAis425tboJ+xeEk8bZeMgCaVn/Y+jh+nvCKDr7o3huZYw65E5CzdE5CaHsvIu9pd
         Ja8whnNVL99cGs0BbOgzIaYFvXRBFu4OW3m2avdhUjuONXLwWn9tEbv0WhoOJcoFibVl
         t6ww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709224849; x=1709829649;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=66ED0Hu2okQ0qkhX0QsEVkNRBgSv+C3kCbNUGkI/Wxk=;
        b=Kl3YDvK0KWwTob/QtW5dV1HKlA8cdCsRFqsW6MlLIcQrYLWqhmc7oAmbF4oKNJ02wK
         E3RpVgn8yQnOqLbYehh/JxUPm/NsX0MWXTIobNEQsQusaP40zAe2xTTNhiKg1+Erl+Ed
         +z0VG0AtFBlogTqMe97wbus68KxKKvpcsRM66X5FNjeQIbl2dLLVQxdPDPRQLxulMjOA
         R3ckYLUBOTUn5Iau6IwG3rfs4EmgdB6ETCRxgVS17xOmQhaczf893DoQ9GlDfMpSiODm
         7G7VyEsJCiUqXcV4pTQNZnzIF4VDa5MFANE/ouNdaxVjJTzYH1US3ZLD7OnQhLRFA496
         6uLA==
X-Forwarded-Encrypted: i=1; AJvYcCXKoaCnYPDowz7hUD2xBjQUkEsC0JaiLsxssot4VjwPjOfllCNrvq+tioqBRaTe1IhPBrZWmPHoLqjL6LZnCO42myvF5RmUC8Lcz10HoUQ=
X-Gm-Message-State: AOJu0YzCYE0V0a8P9NP9rsJRKJJVJFKj22m9bnJddRuYafz0diyHuoW/
	MNa8N3hSMpJTiSDlYprUuNgsFSmHbQSS1qtWZGKZX5zb6zA54QY1JTYbomqYrg==
X-Google-Smtp-Source: AGHT+IE9JgcOaTa+Q5MtayXrrNJgm7NLFr1BPYcaXBZ9t9AeNi0bY8sKYd4C+CfyFgonxQK1bsPAuw==
X-Received: by 2002:a17:906:46d9:b0:a44:4ee9:c560 with SMTP id k25-20020a17090646d900b00a444ee9c560mr1267938ejs.53.1709224849435;
        Thu, 29 Feb 2024 08:40:49 -0800 (PST)
Message-ID: <bd95193c-522a-4c74-98e0-68fa088a5b49@suse.com>
Date: Thu, 29 Feb 2024 17:40:47 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 04/10] xen/public: address violations of MISRA C Rule
 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <0cdc4dc2fcad699a2274277b32de3ee0207d5a2d.1709219010.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <0cdc4dc2fcad699a2274277b32de3ee0207d5a2d.1709219010.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 16:27, Nicola Vetrini wrote:
> --- a/xen/include/public/xen.h
> +++ b/xen/include/public/xen.h
> @@ -988,7 +988,7 @@ typedef struct {
>        ((b) >>  8) & 0xFF, ((b) >>  0) & 0xFF,                           \
>        ((c) >>  8) & 0xFF, ((c) >>  0) & 0xFF,                           \
>        ((d) >>  8) & 0xFF, ((d) >>  0) & 0xFF,                           \
> -                e1, e2, e3, e4, e5, e6}}
> +                (e1), (e2), (e3), (e4), (e5), (e6)}}

Why? Wasn't it agreed already that long macro arguments passed on
(no matter whether to a function, a macro, or like used here) don't
need parenthesizing?

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:42:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:42:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687317.1070618 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjUa-0000jk-6N; Thu, 29 Feb 2024 16:42:44 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687317.1070618; Thu, 29 Feb 2024 16:42:44 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjUa-0000jd-2s; Thu, 29 Feb 2024 16:42:44 +0000
Received: by outflank-mailman (input) for mailman id 687317;
 Thu, 29 Feb 2024 16:42:43 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjUZ-0000jV-4K
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:42:43 +0000
Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com
 [2a00:1450:4864:20::533])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 8dbc9f41-d721-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:42:40 +0100 (CET)
Received: by mail-ed1-x533.google.com with SMTP id
 4fb4d7f45d1cf-563b7b3e3ecso1904252a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:42:40 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 m5-20020a509305000000b0056676b80a38sm792808eda.3.2024.02.29.08.42.39
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:42:40 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 8dbc9f41-d721-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709224960; x=1709829760; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=vNZxO+maZ/8cU1NVm8QqlPDl3pXTDuL601hEP/qUtHI=;
        b=f7xUX8ZlpInN0K3betHm9pC5JNXXtSwseq2KVVb5xHuzXj10Bexy0CY2ZxiPM8Mk3U
         x/dJ8eCsDp/d3eTdTbQJE0LPQX6/vhot/FEpUwINocXSanePiS6zTjJvL03wo3iUnahU
         uJpDY13nmmgnFfiUK8F6JgpNxb6DReZuPLuYUqOKFv7OyIrPWwJiiZ9iQ8ckb8h1qTB9
         PhaxupZ677RAQ2ROT02+jJ6Go5j0t87WzxPaR1p9fSKM/gjU/Ur880+3yDUZQGJQulTZ
         yuGuI4rwbcvO3tOGfVqEq5qtcEOix/8Uj5E9fRd9T3LYJCglz3o1iR8CjnhdSzkjI9Ho
         e/uw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709224960; x=1709829760;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=vNZxO+maZ/8cU1NVm8QqlPDl3pXTDuL601hEP/qUtHI=;
        b=pSTJ+/rBUCNcJK5wrG6MrcgHxRqu52ksgJTREq0jJXqnrT0ZPrfn1tof/XuUXWeeLF
         Cd1XcLXUt86b46vNfbjQpco3zQcdeCRAynvoP6sgI/hHl4Zh2PmneqHeWh2o8tURlqVP
         Y6IcaeBdUeLPhX4rIBhEX+FsUY4j216IjnFlqVkVONDdVoMtF7AdFEzg9FdvAYRhAGJ5
         lH1MRoTzne/tQcO/l2LDBm6vjm8lxxcZNRhJwC7Z3OU0F6QKRZHDf6f0NV+DGiS+QoF0
         ZXm5HpGFJD6v9kraLznT2FWcobbA5We5nIGHNnmN86w9v9wP2hYskzruVrcKPZeVCOxF
         IRIw==
X-Forwarded-Encrypted: i=1; AJvYcCVliOgto5H6pDeZqk1hvV7lq1JYXXt8++mJRssZv+6R4t1HpRKhw9Pv2/afEtLq77msZ137eHhR/2x/WyCuBKU+cdApNwFiweLwNVMQwgw=
X-Gm-Message-State: AOJu0YytQOldhaMpahBG5a3HygNay4c5wSRUTr7zGDaX2lH1cZn/YNHw
	B7C8MehuwgS0nM9vUdCxz2g/S4+GYYTeuZZf5jeykDKub5z8uPd4ZdvAOZjwEQ==
X-Google-Smtp-Source: AGHT+IG5ddQt4hC4mKS5cvgDWe0fw61PGHx4/ht5iJnK2UBcC+G87TFo7LID5pxTXvQ7LwAPJWGtAQ==
X-Received: by 2002:a50:cc46:0:b0:565:3aa7:565f with SMTP id n6-20020a50cc46000000b005653aa7565fmr1767025edi.8.1709224960317;
        Thu, 29 Feb 2024 08:42:40 -0800 (PST)
Message-ID: <dca418e7-d863-4fa9-9740-aada11313c33@suse.com>
Date: Thu, 29 Feb 2024 17:42:38 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 05/10] xen/perfc: address violations of MISRA C Rule
 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <3418c09bb8c4ceeceb66334191adfad190954f56.1709219010.git.nicola.vetrini@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <3418c09bb8c4ceeceb66334191adfad190954f56.1709219010.git.nicola.vetrini@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 16:27, Nicola Vetrini wrote:
> --- a/xen/common/perfc.c
> +++ b/xen/common/perfc.c
> @@ -10,10 +10,10 @@
>  #include <public/sysctl.h>
>  #include <asm/perfc.h>
>  
> -#define PERFCOUNTER( var, name )              { name, TYPE_SINGLE, 0 },
> -#define PERFCOUNTER_ARRAY( var, name, size )  { name, TYPE_ARRAY,  size },
> -#define PERFSTATUS( var, name )               { name, TYPE_S_SINGLE, 0 },
> -#define PERFSTATUS_ARRAY( var, name, size )   { name, TYPE_S_ARRAY,  size },
> +#define PERFCOUNTER( var, name )              { (name), TYPE_SINGLE, 0 },
> +#define PERFCOUNTER_ARRAY( var, name, size )  { (name), TYPE_ARRAY,  (size) },
> +#define PERFSTATUS( var, name )               { (name), TYPE_S_SINGLE, 0 },
> +#define PERFSTATUS_ARRAY( var, name, size )   { (name), TYPE_S_ARRAY,  (size) },

Same question as for patch 4. Plus if this needed touching, then the stray
blanks immediately inside the parentheses would please be dropped as well.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:45:09 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:45:09 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687321.1070627 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjWs-0001Jb-HV; Thu, 29 Feb 2024 16:45:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687321.1070627; Thu, 29 Feb 2024 16:45:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjWs-0001JU-Er; Thu, 29 Feb 2024 16:45:06 +0000
Received: by outflank-mailman (input) for mailman id 687321;
 Thu, 29 Feb 2024 16:45:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfjWr-0001Ia-Hr
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:45:05 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id e31a322c-d721-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:45:04 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id 6C7014EE0737;
 Thu, 29 Feb 2024 17:45:03 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: e31a322c-d721-11ee-a1ee-f123f15fe8a2
MIME-Version: 1.0
Date: Thu, 29 Feb 2024 17:45:03 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 03/10] x86: address some violations of MISRA C Rule
 20.7
In-Reply-To: <14d94c7e-7cd4-4508-a74e-eddd6a592219@suse.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <3c9e90aaf5dde769b689468fc818e4ae61fa11f3.1709219010.git.nicola.vetrini@bugseng.com>
 <14d94c7e-7cd4-4508-a74e-eddd6a592219@suse.com>
Message-ID: <85c54b2492a87f0538831c8baf5a8c76@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-29 17:37, Jan Beulich wrote:
> On 29.02.2024 16:27, Nicola Vetrini wrote:
>> --- a/xen/arch/x86/include/asm/irq.h
>> +++ b/xen/arch/x86/include/asm/irq.h
>> @@ -179,9 +179,9 @@ void cleanup_domain_irq_mapping(struct domain *d);
>>      void *__ret = radix_tree_lookup(&(d)->arch.hvm.emuirq_pirq, 
>> emuirq);\
>>      __ret ? radix_tree_ptr_to_int(__ret) : IRQ_UNBOUND;               
>>   \
>>  })
>> -#define IRQ_UNBOUND -1
>> -#define IRQ_PT -2
>> -#define IRQ_MSI_EMU -3
>> +#define IRQ_UNBOUND (-1)
>> +#define IRQ_PT      (-2)
>> +#define IRQ_MSI_EMU (-3)
>> 
>>  bool cpu_has_pending_apic_eoi(void);
>> 
> 
> I'd be happy to ack this change right away.
> 
>> --- a/xen/arch/x86/usercopy.c
>> +++ b/xen/arch/x86/usercopy.c
>> @@ -106,7 +106,7 @@ unsigned int copy_from_guest_ll(void *to, const 
>> void __user *from, unsigned int
>>      return n;
>>  }
>> 
>> -#if GUARD(1) + 0
>> +#if GUARD((1)) + 0
> 
> I don't even understand the need for this one, and nothing is said in
> the description in that regard. Generally I'm afraid I'm averse to
> such (seemingly) redundant parentheses in macro invocations.
> 

It's because
#define UA_KEEP(args...) args
#define GUARD UA_KEEP

which would expand to #if 1 + 0, while the rule demands #if (1) + 0
I did note in the message after --- that I didn't wanna touch UA_KEEP so 
I did this instead, which I'm not particularly happy about either. I can 
remove this and deviate, there is no other issue with GUARD.
-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:46:01 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:46:01 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687326.1070638 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjXk-0001sg-UK; Thu, 29 Feb 2024 16:46:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687326.1070638; Thu, 29 Feb 2024 16:46:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjXk-0001sZ-RD; Thu, 29 Feb 2024 16:46:00 +0000
Received: by outflank-mailman (input) for mailman id 687326;
 Thu, 29 Feb 2024 16:45:59 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rfjXj-0001sN-MV
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:45:59 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de
 [2a07:de40:b251:101:10:150:64:2])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 030267d7-d722-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:45:57 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org
 [IPv6:2a07:de40:b281:104:10:150:64:97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id E31CC1F7FB;
 Thu, 29 Feb 2024 16:45:56 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 954FC13A4B;
 Thu, 29 Feb 2024 16:45:56 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id 5rmoIsS04GVZQwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 16:45:56 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 030267d7-d722-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709225157; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=LlM0EOHTz0YBL4jjRL1JziDUdOXTFKw3iRSHPbmFw40=;
	b=tuxqjgy93bBc3WM/a17BBaj+9lIaO27kjqaMuR/Uzj1S5QgKcncydRVuUKcTKuZ+6FO5XA
	2O4Gkc0iTcDp5SNi6StAFKKWxeUfjulOM0IWDOfEd56NADSCpKrgPJuofWnsflCCaMsuMN
	+cBTj1H9ICAmx1nYKw/H06heAI4nmrc=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709225156; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 in-reply-to:in-reply-to:references:references:autocrypt:autocrypt;
	bh=LlM0EOHTz0YBL4jjRL1JziDUdOXTFKw3iRSHPbmFw40=;
	b=azoJiFhX3fK1R5K+SsYSOr+NgI3e5z4mG/6Zh7VsukFsGwTyCSf6e/2+t8RvVZjO1yToD4
	y0+WQ15hD/AWar5d9+znamZ2pLVjqJZijjepEVx8neu/DuvA5F7w1N0fByyN7uPvN5KaP8
	LctoVDCdpIFaWLTUnK7QSoFFN/v2E5E=
Message-ID: <58241a32-6671-41e5-8403-5d1b1c4236d2@suse.com>
Date: Thu, 29 Feb 2024 17:45:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/12] xen/spinlock: support higher number of cpus
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-13-jgross@suse.com>
 <c96542eb-daf4-4ce2-915e-86ca8e1848b5@suse.com>
 <f6658df0-13f2-4129-afcd-0f122c87132b@suse.com>
 <f2d107a1-0bcf-4d71-89e8-819ee38e0651@suse.com>
From: Juergen Gross <jgross@suse.com>
Autocrypt: addr=jgross@suse.com; keydata=
 xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB
 ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve
 dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ
 NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx
 XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB
 AAHNH0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT7CwHkEEwECACMFAlOMcK8CGwMH
 CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh
 mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0
 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH
 kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc
 Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO
 RoVBYuiocc51872tRGywc03xaQydB+9R7BHPzsBNBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy
 vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn
 sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa
 aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg
 w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS
 auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAHCwF8EGAECAAkFAlOMcBYCGwwACgkQsN6d
 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn
 fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g
 HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk
 QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6
 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHfw==
In-Reply-To: <f2d107a1-0bcf-4d71-89e8-819ee38e0651@suse.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="------------DP0p9WkeEYzKlrjK0Gnww01y"
X-Spam-Level: 
Authentication-Results: smtp-out2.suse.de;
	dkim=pass header.d=suse.com header.s=susede1 header.b=azoJiFhX
X-Rspamd-Server: rspamd2.dmz-prg2.suse.org
X-Spamd-Result: default: False [-3.63 / 50.00];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 TO_DN_SOME(0.00)[];
	 HAS_ATTACHMENT(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_TRACE(0.00)[suse.com:+];
	 MIME_BASE64_TEXT(0.10)[];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 MX_GOOD(-0.01)[];
	 SIGNED_PGP(-2.00)[];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~];
	 BAYES_HAM(-3.00)[100.00%];
	 MID_RHS_MATCH_FROM(0.00)[];
	 MIME_UNKNOWN(0.10)[application/pgp-keys];
	 ARC_NA(0.00)[];
	 R_DKIM_ALLOW(-0.20)[suse.com:s=susede1];
	 FROM_HAS_DN(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain];
	 NEURAL_SPAM_SHORT(2.57)[0.857];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 RCVD_TLS_ALL(0.00)[]
X-Spam-Score: -3.63
X-Rspamd-Queue-Id: E31CC1F7FB
X-Spam-Flag: NO

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------DP0p9WkeEYzKlrjK0Gnww01y
Content-Type: multipart/mixed; boundary="------------Zleu4NbOFIntqkN7rnvygxxB";
 protected-headers="v1"
From: Juergen Gross <jgross@suse.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
Message-ID: <58241a32-6671-41e5-8403-5d1b1c4236d2@suse.com>
Subject: Re: [PATCH v4 12/12] xen/spinlock: support higher number of cpus
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-13-jgross@suse.com>
 <c96542eb-daf4-4ce2-915e-86ca8e1848b5@suse.com>
 <f6658df0-13f2-4129-afcd-0f122c87132b@suse.com>
 <f2d107a1-0bcf-4d71-89e8-819ee38e0651@suse.com>
In-Reply-To: <f2d107a1-0bcf-4d71-89e8-819ee38e0651@suse.com>
Autocrypt-Gossip: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJ3BBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AAIQkQoDSui/t3IH4WIQQ+pJkfkcoLMCa4X6CgNK6L+3cgfgn7AJ9DmMd0SMJE
 ePbc7/m22D2v04iu7ACffXTdZQhNl557tJuDXZSBxDmW/tLOwU0EWTecRBAIAIK5OMKMU5R2
 Lk2bbjgX7vyQuCFFyKf9rC/4itNwhYWFSlKzVj3WJBDsoi2KvPm7AI+XB6NIkNAkshL5C0kd
 pcNd5Xo0jRR5/WE/bT7LyrJ0OJWS/qUit5eNNvsO+SxGAk28KRa1ieVLeZi9D03NL0+HIAtZ
 tecfqwgl3Y72UpLUyt+r7LQhcI/XR5IUUaD4C/chB4Vq2QkDKO7Q8+2HJOrFIjiVli4lU+Sf
 OBp64m//Y1xys++Z4ODoKh7tkh5DxiO3QBHG7bHK0CSQsJ6XUvPVYubAuy1XfSDzSeSBl//C
 v78Fclb+gi9GWidSTG/4hsEzd1fY5XwCZG/XJJY9M/sAAwUH/09Ar9W2U1Qm+DwZeP2ii3Ou
 14Z9VlVVPhcEmR/AFykL9dw/OV2O/7cdi52+l00reUu6Nd4Dl8s4f5n8b1YFzmkVVIyhwjvU
 jxtPyUgDOt6DRa+RaDlXZZmxQyWcMv2anAgYWGVszeB8Myzsw8y7xhBEVV1S+1KloCzw4V8Z
 DSJrcsZlyMDoiTb7FyqxwQnM0f6qHxWbmOOnbzJmBqpNpFuDcz/4xNsymJylm6oXiucHQBAP
 Xb/cE1YNHpuaH4SRhIxwQilCYEznWowQphNAbJtEKOmcocY7EbSt8VjXTzmYENkIfkrHRyXQ
 dUm5AoL51XZljkCqNwrADGkTvkwsWSvCSQQYEQIACQUCWTecRAIbDAAKCRCgNK6L+3cgfuef
 AJ9wlZQNQUp0KwEf8Tl37RmcxCL4bQCcC5alCSMzUBJ5DBIcR4BY+CyQFAs=

--------------Zleu4NbOFIntqkN7rnvygxxB
Content-Type: multipart/mixed; boundary="------------9MX0jtEcuI90lOWnocHV3Y73"

--------------9MX0jtEcuI90lOWnocHV3Y73
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64

T24gMjkuMDIuMjQgMTc6MzEsIEphbiBCZXVsaWNoIHdyb3RlOg0KPiBPbiAyOS4wMi4yMDI0
IDE3OjI5LCBKw7xyZ2VuIEdyb8OfIHdyb3RlOg0KPj4gT24gMjkuMDIuMjQgMTY6NDYsIEph
biBCZXVsaWNoIHdyb3RlOg0KPj4+IE9uIDEyLjEyLjIwMjMgMTA6NDcsIEp1ZXJnZW4gR3Jv
c3Mgd3JvdGU6DQo+Pj4+IEFsbG93IDE2IGJpdHMgcGVyIGNwdSBudW1iZXIsIHdoaWNoIGlz
IHRoZSBsaW1pdCBpbXBvc2VkIGJ5DQo+Pj4+IHNwaW5sb2NrX3RpY2tldHNfdC4NCj4+Pj4N
Cj4+Pj4gVGhpcyB3aWxsIGFsbG93IHVwIHRvIDY1NTM1IGNwdXMsIHdoaWxlIGluY3JlYXNp
bmcgb25seSB0aGUgc2l6ZSBvZg0KPj4+PiByZWN1cnNpdmUgc3BpbmxvY2tzIGluIGRlYnVn
IGJ1aWxkcyBmcm9tIDggdG8gMTIgYnl0ZXMuDQo+Pj4NCj4+PiBJIHRoaW5rIHdlIHdhbnQg
dG8gYmUgbW9yZSBjb25zZXJ2YXRpdmUgaGVyZSwgZm9yIHRoZSBjYXNlIG9mIHRoZXJlDQo+
Pj4gYmVpbmcgYnVnczogVGhlIENQVSBob2xkaW5nIGEgbG9jayBtYXkgd3JvbmdseSB0cnkg
dG8gYWNxdWlyZSBpdCBhDQo+Pj4gMm5kIHRpbWUuIFRoYXQncyB0aGUgNjU1MzZ0aCB0aWNr
ZXQgdGhlbiwgd3JhcHBpbmcgdGhlIHZhbHVlLg0KPj4NCj4+IElzIHRoaXMgcmVhbGx5IGEg
cHJvYmxlbT8gVGhlcmUgd2lsbCBiZSBubyBvdGhlciBjcHUgbGVmdCBzZWVpbmcgdGhlIGxv
Y2sNCj4+IGFzICJmcmVlIiBpbiB0aGlzIGNhc2UsIGFzIGFsbCBvdGhlcnMgd2lsbCBiZSB3
YWl0aW5nIGZvciB0aGUgaGVhZCB0byByZWFjaA0KPj4gdGhlaXIgcHJpdmF0ZSB0YWlsIHZh
bHVlLg0KPiANCj4gQnV0IGlzbid0IHNhaWQgQ1BVIHRoZW4gZ29pbmcgdG8gbWFrZSBwcm9n
cmVzcywgcmF0aGVyIHRoYW4gaW5kZWZpbml0ZWx5DQo+IHNwaW5uaW5nIG9uIHRoZSBsb2Nr
Pw0KDQpObywgSSBkb24ndCB0aGluayBzby4NCg0KVGhlIGxpbWl0IGlzbid0IDY1NTM1IGJl
Y2F1c2Ugb2YgdGhlIHRpY2tldCBtZWNoYW5pc20sIGJ1dCBiZWNhdXNlIG9mIHRoZQ0KcnNw
aW4gbWVjaGFuaXNtLCB3aGVyZSB3ZSBuZWVkIGEgIm5vIGNwdSBpcyBvd25pbmcgdGhlIGxv
Y2siIHZhbHVlLiBXaXRob3V0DQp0aGUgcmVjdXJzaXZlIGxvY2tzIHRoZSBsaW1pdCB3b3Vs
ZCBiZSA2NTUzNiAob3IgNDA5NiB0b2RheSkuDQoNCj4gDQo+Pj4gVGhlcmVmb3JlIG15IHN1
Z2dlc3Rpb24gd291bGQgYmUgdG8gb25seSAobWVudGlvbikgZ28oaW5nKSB1cCB0byAzMmsu
DQo+Pj4NCj4+Pj4gU2lnbmVkLW9mZi1ieTogSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1c2Uu
Y29tPg0KPj4+PiAtLS0NCj4+Pj4gICAgeGVuL2NvbW1vbi9zcGlubG9jay5jICAgICAgfCAg
MSArDQo+Pj4+ICAgIHhlbi9pbmNsdWRlL3hlbi9zcGlubG9jay5oIHwgMTggKysrKysrKysr
LS0tLS0tLS0tDQo+Pj4+ICAgIDIgZmlsZXMgY2hhbmdlZCwgMTAgaW5zZXJ0aW9ucygrKSwg
OSBkZWxldGlvbnMoLSkNCj4+Pg0KPj4+IFNob3VsZG4ndCB0aGlzIGFsc28gYnVtcCB0aGUg
dXBwZXIgYm91bmQgb2YgdGhlIE5SX0NQVVMgcmFuZ2UgdGhlbg0KPj4+IGluIHhlbi9hcmNo
L0tjb25maWc/DQo+Pg0KPj4gRmluZSB3aXRoIG1lLCBJIGNhbiBhZGQgYW5vdGhlciBwYXRj
aCB0byB0aGUgc2VyaWVzIGRvaW5nIHRoYXQuDQo+IA0KPiBXaHkgbm90IGRvIGl0IHJpZ2h0
IGhlcmU/IFRoZSB1cHBlciBib3VuZCB0aGVyZSBpcyBsaWtlIGl0IGlzIG9ubHkNCj4gYmVj
YXVzZSBvZiB0aGUgcmVzdHJpY3Rpb24gdGhhdCdzIGxpZnRlZCBoZXJlLg0KDQpJJ2QgcHJl
ZmVyIHNwbGl0dGluZyB0aGUgdHdvIGluc3RhbmNlcywgYnV0IGlmIHlvdSBwcmVmZXIgaXQg
dG8gYmUgaW4gYQ0Kc2luZ2xlIHBhdGNoLCBzbyBiZSBpdC4NCg0KDQpKdWVyZ2VuDQoNCg==

--------------9MX0jtEcuI90lOWnocHV3Y73
Content-Type: application/pgp-keys; name="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Disposition: attachment; filename="OpenPGP_0xB0DE9DD628BF132F.asc"
Content-Description: OpenPGP public key
Content-Transfer-Encoding: quoted-printable

-----BEGIN PGP PUBLIC KEY BLOCK-----

xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjri
oyspZKOBycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2
kaV2KL9650I1SJvedYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i
1TXkH09XSSI8mEQ/ouNcMvIJNwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/B
BLUVbDa4+gmzDC9ezlZkTZG2t14zWPvxXP3FAp2pkW0xqG7/377qptDmrk42GlSK
N4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEBAAHNHEp1ZXJnZW4gR3Jvc3Mg
PGpnQHBmdXBmLm5ldD7CwHkEEwECACMFAlOMcBYCGwMHCwkIBwMCAQYVCAIJCgsE
FgIDAQIeAQIXgAAKCRCw3p3WKL8TL0KdB/93FcIZ3GCNwFU0u3EjNbNjmXBKDY4F
UGNQH2lvWAUy+dnyThpwdtF/jQ6j9RwE8VP0+NXcYpGJDWlNb9/JmYqLiX2Q3Tye
vpB0CA3dbBQp0OW0fgCetToGIQrg0MbD1C/sEOv8Mr4NAfbauXjZlvTj30H2jO0u
+6WGM6nHwbh2l5O8ZiHkH32iaSTfN7Eu5RnNVUJbvoPHZ8SlM4KWm8rG+lIkGurq
qu5gu8q8ZMKdsdGC4bBxdQKDKHEFExLJK/nRPFmAuGlId1E3fe10v5QL+qHI3EIP
tyfE7i9Hz6rVwi7lWKgh7pe0ZvatAudZ+JNIlBKptb64FaiIOAWDCx1SzR9KdWVy
Z2VuIEdyb3NzIDxqZ3Jvc3NAc3VzZS5jb20+wsB5BBMBAgAjBQJTjHCvAhsDBwsJ
CAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/Ey/HmQf/RtI7kv5A2PS4
RF7HoZhPVPogNVbC4YA6lW7DrWf0teC0RR3MzXfy6pJ+7KLgkqMlrAbN/8Dvjoz7
8X+5vhH/rDLa9BuZQlhFmvcGtCF8eR0T1v0nC/nuAFVGy+67q2DH8As3KPu0344T
BDpAvr2uYM4tSqxK4DURx5INz4ZZ0WNFHcqsfvlGJALDeE0LhITTd9jLzdDad1pQ
SToCnLl6SBJZjDOX9QQcyUigZFtCXFst4dlsvddrxyqT1f17+2cFSdu7+ynLmXBK
7abQ3rwJY8SbRO2iRulogc5vr/RLMMlscDAiDkaFQWLoqHHOdfO9rURssHNN8WkM
nQfvUewRz80hSnVlcmdlbiBHcm9zcyA8amdyb3NzQG5vdmVsbC5jb20+wsB5BBMB
AgAjBQJTjHDXAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQsN6d1ii/
Ey8PUQf/ehmgCI9jB9hlgexLvgOtf7PJnFOXgMLdBQgBlVPO3/D9R8LtF9DBAFPN
hlrsfIG/SqICoRCqUcJ96Pn3P7UUinFG/I0ECGF4EvTE1jnDkfJZr6jrbjgyoZHi
w/4BNwSTL9rWASyLgqlA8u1mf+c2yUwcGhgkRAd1gOwungxcwzwqgljf0N51N5Jf
VRHRtyfwq/ge+YEkDGcTU6Y0sPOuj4Dyfm8fJzdfHNQsWq3PnczLVELStJNdapwP
OoE+lotufe3AM2vAEYJ9rTz3Cki4JFUsgLkHFqGZarrPGi1eyQcXeluldO3m91NK
/1xMI3/+8jbO0tsn1tqSEUGIJi7ox80eSnVlcmdlbiBHcm9zcyA8amdyb3NzQHN1
c2UuZGU+wsB5BBMBAgAjBQJTjHDrAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgEC
F4AACgkQsN6d1ii/Ey+LhQf9GL45eU5vOowA2u5N3g3OZUEBmDHVVbqMtzwlmNC4
k9Kx39r5s2vcFl4tXqW7g9/ViXYuiDXb0RfUpZiIUW89siKrkzmQ5dM7wRqzgJpJ
wK8Bn2MIxAKArekWpiCKvBOB/Cc+3EXE78XdlxLyOi/NrmSGRIov0karw2RzMNOu
5D+jLRZQd1Sv27AR+IP3I8U4aqnhLpwhK7MEy9oCILlgZ1QZe49kpcumcZKORmzB
TNh30FVKK1EvmV2xAKDoaEOgQB4iFQLhJCdP1I5aSgM5IVFdn7v5YgEYuJYx37Io
N1EblHI//x/e2AaIHpzK5h88NEawQsaNRpNSrcfbFmAg987ATQRTjHAWAQgAyzH6
AOODMBjgfWE9VeCgsrwH3exNAU32gLq2xvjpWnHIs98ndPUDpnoxWQugJ6MpMncr
0xSwFmHEgnSEjK/PAjppgmyc57BwKII3sV4on+gDVFJR6Y8ZRwgnBC5mVM6JjQ5x
Dk8WRXljExRfUX9pNhdE5eBOZJrDRoLUmmjDtKzWaDhIg/+1Hzz93X4fCQkNVbVF
LELU9bMaLPBG/x5q4iYZ2k2ex6d47YE1ZFdMm6YBYMOljGkZKwYde5ldM9mo45mm
we0icXKLkpEdIXKTZeKDO+Hdv1aqFuAcccTg9RXDQjmwhC3yEmrmcfl0+rPghO0I
v3OOImwTEe4co3c1mwARAQABwsBfBBgBAgAJBQJTjHAWAhsMAAoJELDendYovxMv
Q/gH/1ha96vm4P/L+bQpJwrZ/dneZcmEwTbe8YFsw2V/Buv6Z4Mysln3nQK5ZadD
534CF7TDVft7fC4tU4PONxF5D+/tvgkPfDAfF77zy2AH1vJzQ1fOU8lYFpZXTXIH
b+559UqvIB8AdgR3SAJGHHt4RKA0F7f5ipYBBrC6cyXJyyoprT10EMvU8VGiwXvT
yJz3fjoYsdFzpWPlJEBRMedCot60g5dmbdrZ5DWClAr0yau47zpWj3enf1tLWaqc
suylWsviuGjKGw7KHQd3bxALOknAp4dN3QwBYCKuZ7AddY9yjynVaD5X7nF9nO5B
jR/i1DG86lem3iBDXzXsZDn8R3/CwO0EGAEIACAWIQSFEmdy6PYElKXQl/ew3p3W
KL8TLwUCWt3w0AIbAgCBCRCw3p3WKL8TL3YgBBkWCAAdFiEEUy2wekH2OPMeOLge
gFxhu0/YY74FAlrd8NAACgkQgFxhu0/YY75NiwD/fQf/RXpyv9ZX4n8UJrKDq422
bcwkujisT6jix2mOOwYBAKiip9+mAD6W5NPXdhk1XraECcIspcf2ff5kCAlG0DIN
aTUH/RIwNWzXDG58yQoLdD/UPcFgi8GWtNUp0Fhc/GeBxGipXYnvuWxwS+Qs1Qay
7/Nbal/v4/eZZaWs8wl2VtrHTS96/IF6q2o0qMey0dq2AxnZbQIULiEndgR625EF
RFg+IbO4ldSkB3trsF2ypYLij4ZObm2casLIP7iB8NKmQ5PndL8Y07TtiQ+Sb/wn
g4GgV+BJoKdDWLPCAlCMilwbZ88Ijb+HF/aipc9hsqvW/hnXC2GajJSAY3Qs9Mib
4Hm91jzbAjmp7243pQ4bJMfYHemFFBRaoLC7ayqQjcsttN2ufINlqLFPZPR/i3IX
kt+z4drzFUyEjLM1vVvIMjkUoJs=3D
=3DeeAB
-----END PGP PUBLIC KEY BLOCK-----

--------------9MX0jtEcuI90lOWnocHV3Y73--

--------------Zleu4NbOFIntqkN7rnvygxxB--

--------------DP0p9WkeEYzKlrjK0Gnww01y
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature.asc"

-----BEGIN PGP SIGNATURE-----

wsB5BAABCAAjFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAmXgtMQFAwAAAAAACgkQsN6d1ii/Ey/i
2Qf/ZO2A4sEgznSs4CVuNJ1Cv8PBjmYnzxB99fqaeWXlxiAsHDyrQT+XCHvBRiLEvarvhTUJawAl
R+uaNRpBgqfeEtnFw7apRiaJT0Zgqjg/aTpkQGcN+1Q63hT5IbSQWcPbEKY/l6IacBwwK8ym9TjP
I/LUyP6I0CxPno5kOsklGJnEBQncvtwRhw9gNfUvnmOXT1yZaMrv01npoSJUpeHVr7olnIgct115
ZxfZfbM0cPcfywYesM96GKPOYQk0T5fVNn7E4eqROgpj5/hUIW28DOxFrRAnIqprVSPvW0O+tJ6W
8GRXG7GZgZyxmeTmFueDN+m+YegAyuEED3CQpSJVkQ==
=z9QL
-----END PGP SIGNATURE-----

--------------DP0p9WkeEYzKlrjK0Gnww01y--


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:47:24 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:47:24 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687331.1070649 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjZ5-0002Q1-9W; Thu, 29 Feb 2024 16:47:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687331.1070649; Thu, 29 Feb 2024 16:47:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjZ5-0002Pu-4v; Thu, 29 Feb 2024 16:47:23 +0000
Received: by outflank-mailman (input) for mailman id 687331;
 Thu, 29 Feb 2024 16:47:21 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjZ3-0002Pm-RB
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:47:21 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 34a77297-d722-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:47:20 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-563d32ee33aso1664877a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:47:20 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 fe1-20020a056402390100b00566a1b8614esm704231edb.58.2024.02.29.08.47.19
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:47:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 34a77297-d722-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709225240; x=1709830040; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=J9B+z67McwWOw6JQKYvIA0eTo1/FoaYxnTLvrl1OPJI=;
        b=EipfeJCmjBa3PcsmS0R0uhIoCQZBdG/zwD5TKfGzbHR5S7UsCBv2ykPtGkQV+M4ar3
         7PNV00Qn1TPkGGI3ALaPGkDW3DZYjTd/gLQf8Dn/iIWMbhYbwopP746V7ZvQmTMKcodz
         NpBXbmr8Iurqu6TfBaTIoOEdph4v32os9IKRxJ+3AtpMcRrpniJdf0vmGbw1yZA9z9up
         5ROyBXjyPY/pMqtdxBw9WWQA3b0y4WmEzzcONpgR6RaF3zmB9btivYTZKHs/fKLpPi40
         54KFnnRE10KQH/78pQaxvNM77cTyQYYiCaRbZAgSR4O5IDhju3Xuoix/IyhThNdtOAXK
         AC+w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709225240; x=1709830040;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=J9B+z67McwWOw6JQKYvIA0eTo1/FoaYxnTLvrl1OPJI=;
        b=oPEaCv3DnTCGe3BxVzIt4QJMBBSqlpbljjy/xR/Z+MxctRp9kfHSFHoYKQTYgKbXKx
         VzCCW//d1V0azLDjMZ8wBZjDVmCvlujcWwjCveKP+yulJ5S3hmwHVCkMIHDJLaQTnxyk
         jIAKhZ2LSS8+M4TJVj5wIJEhixpY2hxqXe7S6mcDbrmVAHiBByAwzANNQxrV6sNB2JPJ
         KcK5RzAe4qk2iZC/oD5J7i3VpgXFHiU9BbcKbjiHj/asSVXDQCbgSzpbN7xe5AN/xkJx
         Obsq7BTChgEz4dL7rRyoqj678qrMQZyP6FR71inbR3tWfn6mZFiGHVIl/CZwAb8emreW
         50yw==
X-Forwarded-Encrypted: i=1; AJvYcCUQRWF7cqeMt9CEwPI6xXmnbXKSph+K8Yv/sZFXDZy9H/vW9oRWAL/Au+K92p2KiaqzAQJ7nSQkna9mbnTGG0QjFBjplzu3NEXTwlfiRqQ=
X-Gm-Message-State: AOJu0YzJvXGiXkA5BWScowo0N5Oc+5OFr0R5gpmFH+cFItDeHEg3ioOc
	bOYiWXU1Lk/+MlDskKgl+/CuMT14gR6V/3KmMl/3phgmqIiy/tXC/NtZ+yXg5g==
X-Google-Smtp-Source: AGHT+IEdnCKBhbzvM76uMG8xc6/KhdoMOmk2PPPvu8+qjniUKS0SddbtbpbPFv/DkmqashrwvkVh2A==
X-Received: by 2002:a05:6402:3119:b0:564:f6d5:f291 with SMTP id dc25-20020a056402311900b00564f6d5f291mr1664887edb.34.1709225240244;
        Thu, 29 Feb 2024 08:47:20 -0800 (PST)
Message-ID: <3b7af122-2970-4f66-a6ea-bfb242302639@suse.com>
Date: Thu, 29 Feb 2024 17:47:18 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 01/10] xen/include: address violations of MISRA C Rule
 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <15d6e4fb5c873e7ea42cfcee2faa0bf33c10d101.1709219010.git.nicola.vetrini@bugseng.com>
 <26be05f7-7361-40d9-92f2-cf2e22da9d4e@suse.com>
 <4143df23c3d1f8dcb7fcc2d97e077d01@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <4143df23c3d1f8dcb7fcc2d97e077d01@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 17:40, Nicola Vetrini wrote:
> On 2024-02-29 17:25, Jan Beulich wrote:
>> On 29.02.2024 16:27, Nicola Vetrini wrote:
>>> --- a/xen/include/xen/kconfig.h
>>> +++ b/xen/include/xen/kconfig.h
>>> @@ -25,7 +25,7 @@
>>>  #define __ARG_PLACEHOLDER_1 0,
>>>  #define config_enabled(cfg) _config_enabled(cfg)
>>>  #define _config_enabled(value) 
>>> __config_enabled(__ARG_PLACEHOLDER_##value)
>>> -#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk 
>>> 1, 0)
>>> +#define __config_enabled(arg1_or_junk) ___config_enabled(arg1_or_junk 
>>> (1), (0))
>>>  #define ___config_enabled(__ignored, val, ...) val
>>
>> In addition to what Andrew said, would you mind clarifying what exactly 
>> the
>> violation is here? I find it questionable that numeric literals need
>> parenthesizing; they don't normally need to, aynwhere.
>>
> 
> This one's a little special. I couldn't parenthesize val further down, 
> because then it would break the build:
> 
> In file included from ././include/xen/config.h:14,
>                   from <command-line>:
> ./include/xen/kconfig.h:29:52: error: expected identifier or ‘(’ before 
> ‘)’ token
>     29 | #define ___config_enabled(__ignored, val, ...) (val)
>        |                                                    ^
> ./include/xen/kconfig.h:39:34: note: in expansion of macro 
> ‘___config_enabled’
>     39 | #define _static_if(arg1_or_junk) ___config_enabled(arg1_or_junk 
> static,)
>        |                                  ^~~~~~~~~~~~~~~~~
> ./include/xen/kconfig.h:38:26: note: in expansion of macro ‘_static_if’
>     38 | #define static_if(value) _static_if(__ARG_PLACEHOLDER_##value)
>        |                          ^~~~~~~~~~
> ./include/xen/kconfig.h:41:27: note: in expansion of macro ‘static_if’
>     41 | #define STATIC_IF(option) static_if(option)
>        |                           ^~~~~~~~~
> common/page_alloc.c:260:1: note: in expansion of macro ‘STATIC_IF’
>    260 | STATIC_IF(CONFIG_NUMA) mfn_t first_valid_mfn = 
> INVALID_MFN_INITIALIZER;
>        | ^~~~~~~~~
> make[2]: *** [Rules.mk:249: common/page_alloc.o] Error 1

So if we're not gong to deviate the construct, then this change needs to
come entirely on its own, with a really good description.

>>> --- a/xen/include/xen/list.h
>>> +++ b/xen/include/xen/list.h
>>> @@ -490,9 +490,9 @@ static inline void list_splice_init(struct 
>>> list_head *list,
>>>   * @member: the name of the list_struct within the struct.
>>>   */
>>>  #define list_for_each_entry(pos, head, member)                        
>>>   \
>>> -    for (pos = list_entry((head)->next, typeof(*pos), member);        
>>>   \
>>> -         &pos->member != (head);                                      
>>>   \
>>> -         pos = list_entry(pos->member.next, typeof(*pos), member))
>>> +    for (pos = list_entry((head)->next, typeof(*(pos)), member);      
>>>     \
>>> +         &(pos)->member != (head);                                    
>>>   \
>>> +         pos = list_entry((pos)->member.next, typeof(*(pos)), 
>>> member))
>>
>> this ends up inconsistent, which I think isn't nice: Some uses of "pos"
>> are now parenthesized, while others aren't. Applies further down as 
>> well.
>>
> 
> Yeah, the inconsistency is due to the fact that a non-parenthesized 
> parameter as lhs is already deviated. To keep it consistent here I can 
> add parentheses, but then the deviation would be kind of pointless, 
> wouldn't it?

I don't think so: It would still be useful for cases where a macro
parameter is used solely as the lhs of some kind of assignment
operator. But yes, before making adjustments you will want to wait
for possible further comments, especially from e.g. Julien, who iirc
was primarily against this kind of parenthesization.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:47:46 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:47:46 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687333.1070658 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjZR-0002qX-Fr; Thu, 29 Feb 2024 16:47:45 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687333.1070658; Thu, 29 Feb 2024 16:47:45 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjZR-0002qQ-Cm; Thu, 29 Feb 2024 16:47:45 +0000
Received: by outflank-mailman (input) for mailman id 687333;
 Thu, 29 Feb 2024 16:47:44 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfjZQ-0002Pm-SU
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:47:44 +0000
Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com
 [2a00:1450:4864:20::62d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 429e609a-d722-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:47:44 +0100 (CET)
Received: by mail-ej1-x62d.google.com with SMTP id
 a640c23a62f3a-a44282a09bdso181664966b.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:47:44 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 d25-20020a1709067f1900b00a440e2ada28sm839750ejr.201.2024.02.29.08.47.42
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:47:42 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 429e609a-d722-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709225263; x=1709830063; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=XdUnGZlU6VtasExL7xFxpE+W1ikXD8je+ONl8osCxEs=;
        b=P4CLJnNYwPsGOAy9qvBI4laVNJrWEOKfwEa1uCIOXZifpLDNwXnLaNO9IWorTeWaGC
         Xj8SO/F/t9snHNhANE1aE3rlsWn4DDO3vdIJNbs0QJRLUiL3fRlceZMoPDCjkPj+J/yb
         YcbNwWuOJwwVaArs2+IHub5+qFWS6aDY41nNE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709225263; x=1709830063;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=XdUnGZlU6VtasExL7xFxpE+W1ikXD8je+ONl8osCxEs=;
        b=EnmajzNUPsLv/QZTffcurY7GUAC6PuhCUQV3/DDS8FW34haXouoJqqdqOCBcaqxcua
         IEcBDlUa6uuYUsDfgxKL/lyXtZ5I/sqnAWE3NJknQJtNfeMbSAJNwCVmx4vJdm1oUCfF
         pZXigb5Q+KD0DPKAFZKkROxyosuQB3kV3jHt8CEskESjkyBqgoe4hRTRyVO6o3jl1dY1
         72PnwYyaXAOdZ5K25aKFigE1j9FUoGvHPhu6ilsz9dwNO7k6BO92A58JIiu0vrvfFcGG
         MOzPYVLPV22e0FuIANePy++eA+OSk/9NlOuhIQ0xWUW5pKG/VvC4x2hOrnX+2afv4/Ys
         tKgA==
X-Gm-Message-State: AOJu0YyOz2cw9Q8hb1omYJiV0VgnaHUrSw89aGNpb/6UvGoTVCS0z7tp
	aVc5PwILPgdsyaZ3r7pATahE1sYC0z2sWDNg1auDQRmZguZ+Lhc95XIa2HlEMCJvDZ/dKWUohBt
	k
X-Google-Smtp-Source: AGHT+IHIzeCeOGBP4kLlARZ9iNHdebiu0J582qb0P65U/EEM7LJDmxYfmdWcAoYbaO75TQ4MuymClA==
X-Received: by 2002:a17:906:a88:b0:a3e:473d:e391 with SMTP id y8-20020a1709060a8800b00a3e473de391mr1953878ejf.22.1709225263690;
        Thu, 29 Feb 2024 08:47:43 -0800 (PST)
Message-ID: <dd361525-be1a-4d11-bbc8-4f5eedd381e0@citrix.com>
Date: Thu, 29 Feb 2024 16:47:41 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 01/10] xen/include: address violations of MISRA C Rule
 20.7
Content-Language: en-GB
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org,
 michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com,
 consulting@bugseng.com, jbeulich@suse.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, julien@xen.org,
 George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <15d6e4fb5c873e7ea42cfcee2faa0bf33c10d101.1709219010.git.nicola.vetrini@bugseng.com>
 <b844e28f-a53b-49c0-b4b9-297367546bd3@citrix.com>
 <d9674a6e2ef00457a90d570fc3866c81@bugseng.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <d9674a6e2ef00457a90d570fc3866c81@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/02/2024 4:21 pm, Nicola Vetrini wrote:
> On 2024-02-29 17:10, Andrew Cooper wrote:
>> On 29/02/2024 3:27 pm, Nicola Vetrini wrote:
>>> diff --git a/xen/include/xen/kconfig.h b/xen/include/xen/kconfig.h
>>> index c25dc0f6c2a9..b7e70289737b 100644
>>> --- a/xen/include/xen/kconfig.h
>>> +++ b/xen/include/xen/kconfig.h
>>> @@ -25,7 +25,7 @@
>>>  #define __ARG_PLACEHOLDER_1 0,
>>>  #define config_enabled(cfg) _config_enabled(cfg)
>>>  #define _config_enabled(value)
>>> __config_enabled(__ARG_PLACEHOLDER_##value)
>>> -#define __config_enabled(arg1_or_junk)
>>> ___config_enabled(arg1_or_junk 1, 0)
>>> +#define __config_enabled(arg1_or_junk)
>>> ___config_enabled(arg1_or_junk (1), (0))
>>>  #define ___config_enabled(__ignored, val, ...) val
>>
>> This one hunk I suggest we deviate rather than adjust.  You've subtly
>> broken it, and it's extreme preprocessor magic in the first place to
>> turn an absent symbol into a 0.
>>
>
> How so? I did test this because I was very wary of it, but it seemed
> to expand fine (either if ((0)) or if ((1)) ). I may of course be
> wrong, and it could be deviated regardless.
>

arg1_or_junk(1) can now be a function or macro expansion depending on
context, where previously it couldn't be.

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:49:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:49:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687337.1070668 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjam-0003YU-UA; Thu, 29 Feb 2024 16:49:08 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687337.1070668; Thu, 29 Feb 2024 16:49:08 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjam-0003YN-RV; Thu, 29 Feb 2024 16:49:08 +0000
Received: by outflank-mailman (input) for mailman id 687337;
 Thu, 29 Feb 2024 16:49:07 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfjal-0003YF-Sw
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:49:07 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 735ab007-d722-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 17:49:06 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id C5C914EE0737;
 Thu, 29 Feb 2024 17:49:05 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 735ab007-d722-11ee-a1ee-f123f15fe8a2
MIME-Version: 1.0
Date: Thu, 29 Feb 2024 17:49:05 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 04/10] xen/public: address violations of MISRA C Rule
 20.7
In-Reply-To: <bd95193c-522a-4c74-98e0-68fa088a5b49@suse.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <0cdc4dc2fcad699a2274277b32de3ee0207d5a2d.1709219010.git.nicola.vetrini@bugseng.com>
 <bd95193c-522a-4c74-98e0-68fa088a5b49@suse.com>
Message-ID: <82940f688e5a5eee274fa579991ebd15@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-29 17:40, Jan Beulich wrote:
> On 29.02.2024 16:27, Nicola Vetrini wrote:
>> --- a/xen/include/public/xen.h
>> +++ b/xen/include/public/xen.h
>> @@ -988,7 +988,7 @@ typedef struct {
>>        ((b) >>  8) & 0xFF, ((b) >>  0) & 0xFF,                         
>>   \
>>        ((c) >>  8) & 0xFF, ((c) >>  0) & 0xFF,                         
>>   \
>>        ((d) >>  8) & 0xFF, ((d) >>  0) & 0xFF,                         
>>   \
>> -                e1, e2, e3, e4, e5, e6}}
>> +                (e1), (e2), (e3), (e4), (e5), (e6)}}
> 
> Why? Wasn't it agreed already that long macro arguments passed on
> (no matter whether to a function, a macro, or like used here) don't
> need parenthesizing?
> 

That applies to all outermost macro invocations, but not to the 
innermost one. If you want also aggregate initalizers to be deviated, 
that could be done (provided that the macro arg is not included in some 
expression, such as "{..., e1 + 1, ...}"

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:50:36 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:50:36 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687339.1070677 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjcB-0005Ou-60; Thu, 29 Feb 2024 16:50:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687339.1070677; Thu, 29 Feb 2024 16:50:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjcB-0005On-3R; Thu, 29 Feb 2024 16:50:35 +0000
Received: by outflank-mailman (input) for mailman id 687339;
 Thu, 29 Feb 2024 16:50:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfjcA-0005OR-BR
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:50:34 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a77307e8-d722-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:50:33 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id CD5974EE0737;
 Thu, 29 Feb 2024 17:50:32 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a77307e8-d722-11ee-afd8-a90da7624cb6
MIME-Version: 1.0
Date: Thu, 29 Feb 2024 17:50:32 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Jan Beulich <jbeulich@suse.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, George Dunlap <george.dunlap@citrix.com>, Wei Liu
 <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 05/10] xen/perfc: address violations of MISRA C Rule
 20.7
In-Reply-To: <dca418e7-d863-4fa9-9740-aada11313c33@suse.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <3418c09bb8c4ceeceb66334191adfad190954f56.1709219010.git.nicola.vetrini@bugseng.com>
 <dca418e7-d863-4fa9-9740-aada11313c33@suse.com>
Message-ID: <c272200b83065adf6adac2a464d30a40@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit

On 2024-02-29 17:42, Jan Beulich wrote:
> On 29.02.2024 16:27, Nicola Vetrini wrote:
>> --- a/xen/common/perfc.c
>> +++ b/xen/common/perfc.c
>> @@ -10,10 +10,10 @@
>>  #include <public/sysctl.h>
>>  #include <asm/perfc.h>
>> 
>> -#define PERFCOUNTER( var, name )              { name, TYPE_SINGLE, 0 
>> },
>> -#define PERFCOUNTER_ARRAY( var, name, size )  { name, TYPE_ARRAY,  
>> size },
>> -#define PERFSTATUS( var, name )               { name, TYPE_S_SINGLE, 
>> 0 },
>> -#define PERFSTATUS_ARRAY( var, name, size )   { name, TYPE_S_ARRAY,  
>> size },
>> +#define PERFCOUNTER( var, name )              { (name), TYPE_SINGLE, 
>> 0 },
>> +#define PERFCOUNTER_ARRAY( var, name, size )  { (name), TYPE_ARRAY,  
>> (size) },
>> +#define PERFSTATUS( var, name )               { (name), 
>> TYPE_S_SINGLE, 0 },
>> +#define PERFSTATUS_ARRAY( var, name, size )   { (name), TYPE_S_ARRAY, 
>>  (size) },
> 
> Same question as for patch 4. Plus if this needed touching, then the 
> stray
> blanks immediately inside the parentheses would please be dropped as 
> well.
> 

Noted

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:53:38 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:53:38 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687342.1070687 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjf5-0006na-KY; Thu, 29 Feb 2024 16:53:35 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687342.1070687; Thu, 29 Feb 2024 16:53:35 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjf5-0006nT-HD; Thu, 29 Feb 2024 16:53:35 +0000
Received: by outflank-mailman (input) for mailman id 687342;
 Thu, 29 Feb 2024 16:53:34 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ot4I=KG=bugseng.com=nicola.vetrini@srs-se1.protection.inumbo.net>)
 id 1rfjf4-0006n4-6Z
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:53:34 +0000
Received: from support.bugseng.com (mail.bugseng.com [162.55.131.47])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 129fd58f-d723-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:53:33 +0100 (CET)
Received: from support.bugseng.com (support.bugseng.com [162.55.131.47])
 by support.bugseng.com (Postfix) with ESMTPA id D61804EE0737;
 Thu, 29 Feb 2024 17:53:32 +0100 (CET)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 129fd58f-d723-11ee-afd8-a90da7624cb6
MIME-Version: 1.0
Date: Thu, 29 Feb 2024 17:53:32 +0100
From: Nicola Vetrini <nicola.vetrini@bugseng.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org,
 michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com,
 consulting@bugseng.com, jbeulich@suse.com, roger.pau@citrix.com,
 bertrand.marquis@arm.com, julien@xen.org, George Dunlap
 <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH 01/10] xen/include: address violations of MISRA C Rule
 20.7
In-Reply-To: <dd361525-be1a-4d11-bbc8-4f5eedd381e0@citrix.com>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <15d6e4fb5c873e7ea42cfcee2faa0bf33c10d101.1709219010.git.nicola.vetrini@bugseng.com>
 <b844e28f-a53b-49c0-b4b9-297367546bd3@citrix.com>
 <d9674a6e2ef00457a90d570fc3866c81@bugseng.com>
 <dd361525-be1a-4d11-bbc8-4f5eedd381e0@citrix.com>
Message-ID: <95c49f91b6a2b4d7766db7b0152969af@bugseng.com>
X-Sender: nicola.vetrini@bugseng.com
Organization: BUGSENG s.r.l.
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit

On 2024-02-29 17:47, Andrew Cooper wrote:
> On 29/02/2024 4:21 pm, Nicola Vetrini wrote:
>> On 2024-02-29 17:10, Andrew Cooper wrote:
>>> On 29/02/2024 3:27 pm, Nicola Vetrini wrote:
>>>> diff --git a/xen/include/xen/kconfig.h b/xen/include/xen/kconfig.h
>>>> index c25dc0f6c2a9..b7e70289737b 100644
>>>> --- a/xen/include/xen/kconfig.h
>>>> +++ b/xen/include/xen/kconfig.h
>>>> @@ -25,7 +25,7 @@
>>>>  #define __ARG_PLACEHOLDER_1 0,
>>>>  #define config_enabled(cfg) _config_enabled(cfg)
>>>>  #define _config_enabled(value)
>>>> __config_enabled(__ARG_PLACEHOLDER_##value)
>>>> -#define __config_enabled(arg1_or_junk)
>>>> ___config_enabled(arg1_or_junk 1, 0)
>>>> +#define __config_enabled(arg1_or_junk)
>>>> ___config_enabled(arg1_or_junk (1), (0))
>>>>  #define ___config_enabled(__ignored, val, ...) val
>>> 
>>> This one hunk I suggest we deviate rather than adjust.  You've subtly
>>> broken it, and it's extreme preprocessor magic in the first place to
>>> turn an absent symbol into a 0.
>>> 
>> 
>> How so? I did test this because I was very wary of it, but it seemed
>> to expand fine (either if ((0)) or if ((1)) ). I may of course be
>> wrong, and it could be deviated regardless.
>> 
> 
> arg1_or_junk(1) can now be a function or macro expansion depending on
> context, where previously it couldn't be.
> 

I see, that would be a latent bug. I do agree on deviating then.

-- 
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:54:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:54:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687344.1070698 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjfz-0007VP-Rj; Thu, 29 Feb 2024 16:54:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687344.1070698; Thu, 29 Feb 2024 16:54:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjfz-0007VI-Ov; Thu, 29 Feb 2024 16:54:31 +0000
Received: by outflank-mailman (input) for mailman id 687344;
 Thu, 29 Feb 2024 16:54:30 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygh/=KG=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rfjfy-0007VC-1D
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:54:30 +0000
Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com
 [2a00:1450:4864:20::22e])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 340c21f3-d723-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:54:29 +0100 (CET)
Received: by mail-lj1-x22e.google.com with SMTP id
 38308e7fff4ca-2d2533089f6so13836681fa.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:54:29 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 k8-20020a05651c10a800b002d31953bc30sm97443ljn.55.2024.02.29.08.54.27
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 08:54:28 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 340c21f3-d723-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709225669; x=1709830469; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=uHTRH7mtyCb6DPkEgHpCUaZWQ7YxVbG0pvJNtrNBTYY=;
        b=hyz+4lsqYgiyqLXN9GBQ8p1EQ6UZaBOyJSUdagJLQDmrQ+6ATrToRAip2vuv7NKirh
         HI2yxYZzr02nLC8e28kgBS3++GILYsqlSoqXaZgUgYirGHRlACoAHWDvtL4z6FjBTQFV
         00gGXRzShGq4xtLpvvXGy0eljz3+CazAL/1D+9fash9S60hLBc+JvuuOEmRVcyJbnsmb
         196/tMJsd9Laxq5On7BMO3+qbtZb43ZDtQ38OdW3YiHo7XxrARhlyfo+rwJ64kC7TdDI
         cqRLw547gj48iOgOlCn02Kht6gDbz+Wx8CPRoKMAIjjSu8UgW6JfNc+ex3DWEZ+Cu0z4
         ZSEA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709225669; x=1709830469;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=uHTRH7mtyCb6DPkEgHpCUaZWQ7YxVbG0pvJNtrNBTYY=;
        b=f/fqsgABie8a5WXjCqggPTkFicrc9g1NDqArnFm8ZNCfBJB+T6UPdpje97CNYO9nGx
         5kQFnYGqylsZbWC8gJMGibVGwdEprLb4hnoEwYX9E0DZv/Kg+UW9kpIErdSOLNLeV94/
         /8GoxRrUmntMxKxkyISow+p33kQ8rF93tM9Zoq99FzPIDfBI9RpSzMofsRGjmWo/7NQK
         pnoYB+NYFOm/PGpXVY13GCPkKkdHKtb1TMe3FxY8HsLdH5ryqpAezSW9tdURST+68Fiy
         pQFxORWvLPaWBYexVl4lKEAo3eapfdjf64dSjudmtocUlmwmfTzaCEu4ngLQ0bQuln+e
         CllQ==
X-Forwarded-Encrypted: i=1; AJvYcCU0agBEN4c/S9+dzhwLA4tmQtJdq4GV2xfThWysSGolWgn4HXmd3mPvSZ2pZGUw4qPWSSUSWLXm3fFBlIx7lBxBg5LvI73DF54XaPpvqGM=
X-Gm-Message-State: AOJu0YzIvYim+07iO8YMixiy2pvw963chPFwh/hhhkTlrwVJIfia3fov
	7ZM3EKUopZTGx1G503mT5VebparRlZywUEYXEuojoY26xr3MX/S3
X-Google-Smtp-Source: AGHT+IEGr+CMp/XbhGtq5l8EVs29eDLGMgbvTWEA1p7c7aOKBJ3JpZmFYgQra/r66I4uRYiwxbYEdQ==
X-Received: by 2002:a2e:a40d:0:b0:2d3:9b4:4363 with SMTP id p13-20020a2ea40d000000b002d309b44363mr1632852ljn.23.1709225668496;
        Thu, 29 Feb 2024 08:54:28 -0800 (PST)
Message-ID: <7febc1be78f19ef9b5c5e76ebb47ec3883cf2a2e.camel@gmail.com>
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
From: Oleksii <oleksii.kurochko@gmail.com>
To: Jan Beulich <jbeulich@suse.com>, Julien Grall <julien@xen.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, George Dunlap
 <george.dunlap@citrix.com>, Stefano Stabellini <sstabellini@kernel.org>,
 Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Date: Thu, 29 Feb 2024 17:54:27 +0100
In-Reply-To: <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
	 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
	 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
	 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
	 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Thu, 2024-02-29 at 08:58 +0100, Jan Beulich wrote:
> On 28.02.2024 23:58, Julien Grall wrote:
> > On 27/02/2024 07:55, Jan Beulich wrote:
> > > On 26.02.2024 18:39, Oleksii Kurochko wrote:
> > > > This patch doesn't represent a strict lower bound for GCC and
> > > > GNU Binutils; rather, these versions are specifically employed
> > > > by
> > > > the Xen RISC-V container and are anticipated to undergo
> > > > continuous
> > > > testing.
> > >=20
> > > Up and until that container would be updated to a newer gcc. I'm
> > > afraid I view this as too weak a criteria,
> >=20
> > I disagree. We have to decide a limit at some point. It is sensible
> > to=20
> > say that we are only supporting what we can tests. AFAIK, this is
> > what=20
> > QEMU has been doing.
>=20
> I view qemu as a particularly bad example. They raise their baselines
> far too aggressively for my taste.
>=20
> > > IOW it is hard for me to see why RISC-V needs stronger
> > > restrictions here
> > > than other architectures. It ought to be possible to determine a
> > > baseline
> > > version. Even if taking the desire to have "pause" available as a
> > > requirement, gas (and presumably gld) 2.36.1 would already
> > > suffice.
> >=20
> > I think we want to bump it on Arm. There are zero reasons to try to
> > keep=20
> > a lower versions if nobody tests/use it in production.
> >=20
> > I would suggest to do the same on x86. What's the point of try to=20
> > support Xen with a 15+ years old compiler?
>=20
> It could have long been bumped if only a proper scheme to follow for
> this and future bumping would have been put forward by anyone keen on
> such bumping, like - see his reply - e.g. Andrew. You may recall that
> this was discussed more than once on meetings, with no real outcome.
> I'm personally not meaning to stand in the way of such bumping as
> long
> as it's done in a predictable manner, but I'm not keen on doing so
> and
> hence I don't view it as my obligation to try to invent a reasonable
> scheme. (My personal view is that basic functionality should be
> possible to have virtually everywhere, whereas for advanced stuff it
> is fine to require a more modern tool chain.)
>=20
> The one additional concern I've raised in the past is that in the end
> it's not just minimal tool chain versions we rely on, but also other
> core system tools (see the recent move from "which" to "command -v"
> for an example of such a dependency, where luckily it turned out to
> not be an issue that the -v had only become a standard thing at some
> point). While for the tool chain I can arrange for making newer
> versions available, for core system tools I can't.=C2=A0
Can't we identify the top X most popular Linux distributions ( LTS
versions ) and align Xen's minimal toolchain version with the selected
distributions' minimum toolchain versions?

> Therefore being too
> eager there would mean I can't really / easily (smoke) test Xen
> anymore on ancient hardware every once in a while. When afaict we do
> too little of such testing already anyway, despite not having any
> lower bound on hardware that formally we support running Xen on. (And
> no, upgrading the ancient distros on that ancient hardware is not an
> option for me.)
It seems there is no room for upgrading the toolchain version. This
leads to the question of determining the threshold between maintaining
the version as minimal as possible and deciding to upgrade it.
I understand your situation where you have an ancient hardware that
necessitates the use of older Linux distributions. However, is this a
common use case?

~ Oleksii







From xen-devel-bounces@lists.xenproject.org Thu Feb 29 16:54:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 16:54:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687346.1070708 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjgN-0007y1-3E; Thu, 29 Feb 2024 16:54:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687346.1070708; Thu, 29 Feb 2024 16:54:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjgN-0007xu-0O; Thu, 29 Feb 2024 16:54:55 +0000
Received: by outflank-mailman (input) for mailman id 687346;
 Thu, 29 Feb 2024 16:54:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjgL-0007VC-PY
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 16:54:53 +0000
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
 [2a00:1450:4864:20::529])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 4242dfc1-d723-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 17:54:53 +0100 (CET)
Received: by mail-ed1-x529.google.com with SMTP id
 4fb4d7f45d1cf-5656e5754ccso1629296a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 08:54:53 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 fj10-20020a0564022b8a00b00563f8233ba8sm771540edb.7.2024.02.29.08.54.52
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 08:54:52 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4242dfc1-d723-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709225692; x=1709830492; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=L+CWK2Zjsq0ZCpSJAGUrwHduyq7jVSXLK9awLW+BQvA=;
        b=Zd/0g0qEcMCWqlt9qU6YM/w7MSbRsfB5sAVIO1qYPxKACxjZkLBiZzjUqMNA/0bnhj
         S7Hsw8rG8ufSYdJx4NxYGkaqzJAHLoqu/NFS6skPJx5wXtyQXJ4j25ynuowTDruRLxZb
         JsGDvmuwt8sSSrkVZMaUjAadWKW9mJ+GYy/IYIShHLvylnkt1QqvQrak0VyUSYYGOxkt
         y4K4kmTv/7IgRu/dSgwKOZnUsuYzmEq0AqpH4X51o8UgQ8SLch0rIawAldW2boZALhNY
         jRo84QuoMgrdWVyiQbOR8N7w38CT//SRwEO5+vIuN7TqJCqk77AzwvJpG0lwCtTXpubt
         Ls9A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709225692; x=1709830492;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=L+CWK2Zjsq0ZCpSJAGUrwHduyq7jVSXLK9awLW+BQvA=;
        b=JUiL6qth3JdywFunFTyVj6alXPJFxmYWRWUBfSWPS2T+oa24eqXnb7zdvbzcYlDAZl
         FNIiQVKhfbDvDGDXIaKzdckYY0PbhOK6A1sO2yFkChgsHFEXuxQZwfsqLMxAGxXn9GV1
         x16hPycCNG4GnNcf/dEYD3+vWG1nCPnuLgUUvsiEC2x9W1g4Es8SFhh9LWhbSXeC8vJR
         qtYAlEFb+1K4+tepQeubDDWVK6FFEedg3EQA0wXpkAotwJgWORPTk6CQLOUWc6/iBfKQ
         H8SNuoBgI8GZZizEVoN0s6AFe4b8txoRYbc35+rSKv4POetgQYWkkYwrC1VKetxGp28Y
         WqIA==
X-Forwarded-Encrypted: i=1; AJvYcCUSvMehxe6yq7+GqOYnZXflMTpV103o0M4RRy44ZzH/zLtw1vEqJbUTQhI+tE1B2v1piMkXY7OWZqhn2y+0cz3rMUHqXkKGgrcuWcOzOFw=
X-Gm-Message-State: AOJu0YydZRl/8UJuVM0RDemu4+NlTxDS7VzWAE6AWjtmIKDBaymX9gJ3
	DJEkS/ccWp2/pAWpgX1rovndZDsrUdpr5owvdowYNO+VGQR/7VvqosD+ehOL6A==
X-Google-Smtp-Source: AGHT+IGeEWmiJepmBQyjXed6QksG/+l/ow1Mmbq9CijcJKzRKlunTZJIm4ddXb6DR1aEk9gkc0X1Zg==
X-Received: by 2002:a05:6402:6cf:b0:566:c167:bf18 with SMTP id n15-20020a05640206cf00b00566c167bf18mr240270edy.35.1709225692573;
        Thu, 29 Feb 2024 08:54:52 -0800 (PST)
Message-ID: <305dcff3-b37b-4f19-bd79-6e8c71c6d850@suse.com>
Date: Thu, 29 Feb 2024 17:54:51 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/12] xen/spinlock: support higher number of cpus
Content-Language: en-US
To: Juergen Gross <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-13-jgross@suse.com>
 <c96542eb-daf4-4ce2-915e-86ca8e1848b5@suse.com>
 <f6658df0-13f2-4129-afcd-0f122c87132b@suse.com>
 <f2d107a1-0bcf-4d71-89e8-819ee38e0651@suse.com>
 <58241a32-6671-41e5-8403-5d1b1c4236d2@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <58241a32-6671-41e5-8403-5d1b1c4236d2@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 17:45, Juergen Gross wrote:
> On 29.02.24 17:31, Jan Beulich wrote:
>> On 29.02.2024 17:29, Jürgen Groß wrote:
>>> On 29.02.24 16:46, Jan Beulich wrote:
>>>> On 12.12.2023 10:47, Juergen Gross wrote:
>>>>> Allow 16 bits per cpu number, which is the limit imposed by
>>>>> spinlock_tickets_t.
>>>>>
>>>>> This will allow up to 65535 cpus, while increasing only the size of
>>>>> recursive spinlocks in debug builds from 8 to 12 bytes.
>>>>
>>>> I think we want to be more conservative here, for the case of there
>>>> being bugs: The CPU holding a lock may wrongly try to acquire it a
>>>> 2nd time. That's the 65536th ticket then, wrapping the value.
>>>
>>> Is this really a problem? There will be no other cpu left seeing the lock
>>> as "free" in this case, as all others will be waiting for the head to reach
>>> their private tail value.
>>
>> But isn't said CPU then going to make progress, rather than indefinitely
>> spinning on the lock?
> 
> No, I don't think so.

Hmm. If CPU0 takes a pristine lock, it'll get ticket 0x0000. All other
CPUs will get tickets 0x0001 ... 0xffff. Then CPU0 trying to take the lock
again will get ticket 0x0000 again, which equals what .head still has (no
unlocks so far), thus signalling the lock to be free when it isn't.

> The limit isn't 65535 because of the ticket mechanism, but because of the
> rspin mechanism, where we need a "no cpu is owning the lock" value. Without
> the recursive locks the limit would be 65536 (or 4096 today).

I think this rather belongs ...

>>>> Therefore my suggestion would be to only (mention) go(ing) up to 32k.
>>>>
>>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>> ---
>>>>>    xen/common/spinlock.c      |  1 +
>>>>>    xen/include/xen/spinlock.h | 18 +++++++++---------
>>>>>    2 files changed, 10 insertions(+), 9 deletions(-)
>>>>
>>>> Shouldn't this also bump the upper bound of the NR_CPUS range then
>>>> in xen/arch/Kconfig?
>>>
>>> Fine with me, I can add another patch to the series doing that.
>>
>> Why not do it right here? The upper bound there is like it is only
>> because of the restriction that's lifted here.

... here (for having nothing to do with the supposed lack of hanging
that I'm seeing)?

> I'd prefer splitting the two instances, but if you prefer it to be in a
> single patch, so be it.

I'm not going to insist - if want to do it separately, please do.
Perhaps others would actually prefer it that way ...

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 17:01:08 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 17:01:08 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687354.1070717 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjmH-0001s3-PK; Thu, 29 Feb 2024 17:01:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687354.1070717; Thu, 29 Feb 2024 17:01:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjmH-0001rw-MN; Thu, 29 Feb 2024 17:01:01 +0000
Received: by outflank-mailman (input) for mailman id 687354;
 Thu, 29 Feb 2024 17:01:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=ygh/=KG=gmail.com=oleksii.kurochko@srs-se1.protection.inumbo.net>)
 id 1rfjmG-0001rq-LD
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 17:01:00 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 1cdb7e6d-d724-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 18:00:59 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-565a2c4cc1aso1794793a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 09:00:59 -0800 (PST)
Received: from [192.168.206.239] ([94.75.70.14])
 by smtp.gmail.com with ESMTPSA id
 fi12-20020a056402550c00b005661badcccesm756429edb.87.2024.02.29.09.00.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 09:00:58 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 1cdb7e6d-d724-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709226059; x=1709830859; darn=lists.xenproject.org;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:from:to:cc:subject
         :date:message-id:reply-to;
        bh=MlQ9zW/t7TnZhS7Gp5cDrlNdJ1O1m19wbjUu/KGR9Nk=;
        b=etsHvKbDubXOrzIuAupMpu2TlQrzFkUiYLNYso/+7uQ1xU0WV8ntrq4VT9Kxi7KDnY
         w8yzX+B98nPPeDSYGZnn/21vgQay/pzjt6wAP1Tn7CWZ174Xz1J3jS6t6K3qC1RRAUtg
         WPVfsm1WDZzHsLIsbMIRcBi6fOJxk0cfOKDZjCHnRBZjeGGMV8u9z8sm+kbex9BKSxSx
         UTZtoJ0DKOcrfyLcvePKkytbAGNsVzFz9wssIfH3EgjRon7pKd4UmbI9l+VI3CGF6WPF
         ckoFDHicj/nyYWqxKV8WYO83eeyGZ29kyJKvNqZ6XiPC1fLO+J2HehnB4L3wkadQWoBG
         yuqA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709226059; x=1709830859;
        h=mime-version:user-agent:content-transfer-encoding:references
         :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state
         :from:to:cc:subject:date:message-id:reply-to;
        bh=MlQ9zW/t7TnZhS7Gp5cDrlNdJ1O1m19wbjUu/KGR9Nk=;
        b=EjyyLKK/oqhqIwq3EiZDS9Fq/epfvTIuY5EZYVALnn9kcy37OoONShMAog6x/zGLpb
         U9BlouKJIop8vQBkBksboIqW/hbcCPloINHu49sFbiJC7FnCTTwVpBytdUSBmhMSvpRK
         UoI5gtol3o4wOF0bkm6jmjvBAdcshdDf8HQm4kOMV0GwJwCilpKCb31mqfNPpOVoatbs
         +J1s7MTNa5LTOi4voj5//4TOkpoM7OsyXu/DIWMB7PC/lyNSE0seyvzWITPvLLzBRLVM
         C6wZZalNyuxwOaDdezviZ907Cw3nnkuZIRI8v+35QgELM7gTLP5um6+kseZ4UMyA5eZh
         cZ6g==
X-Forwarded-Encrypted: i=1; AJvYcCUGeBnYBbarW9m6i8W4TQ4QOuXw6AXTp/3oXYsa8BFZIDemxyxpjlx16xnzrkEVyWTEeMZYtKZU/qIhuqULEeotMH9PrwSRq/pOcICO9aw=
X-Gm-Message-State: AOJu0YzKMN1dUxB7aXaxCszf/5KBULMNdYBTDHUZXOOl8AqN/8rWipW6
	Dk1l3c6q8880MKwxmLfuM4IOEoByanT0KWQB7vHCi3BqEJlKYgWv
X-Google-Smtp-Source: AGHT+IH7hqSC+NL8tElsWBARAoEY4Atamd/8E+rTNpdbOmpEL5zMO6i3C/bamFWzJjDtFGrKAPUoVg==
X-Received: by 2002:a05:6402:b59:b0:566:ab45:e6c5 with SMTP id bx25-20020a0564020b5900b00566ab45e6c5mr1361625edb.28.1709226059069;
        Thu, 29 Feb 2024 09:00:59 -0800 (PST)
Message-ID: <93854992e2828f692d1d50c19d1f19a4e065d6cb.camel@gmail.com>
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
From: Oleksii <oleksii.kurochko@gmail.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>, Julien Grall
 <julien@xen.org>,  Jan Beulich <jbeulich@suse.com>
Cc: George Dunlap <george.dunlap@citrix.com>, Stefano Stabellini
 <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org,  Roger Pau =?ISO-8859-1?Q?Monn=E9?=
 <roger.pau@citrix.com>
Date: Thu, 29 Feb 2024 18:00:57 +0100
In-Reply-To: <d0afb4d6-9fbc-49de-9f07-f76e4aeda8c1@citrix.com>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com>
	 <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com>
	 <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com>
	 <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
	 <d0afb4d6-9fbc-49de-9f07-f76e4aeda8c1@citrix.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.50.2 (3.50.2-1.fc39) 
MIME-Version: 1.0

On Wed, 2024-02-28 at 23:11 +0000, Andrew Cooper wrote:
> Furthermore, Linux has regularly been bumping minimum toolchain
> versions
> due to code generation issues, and we'd be foolish not pay attention.
Do they document that?

It looks like their doc is pretty old, because in Documentation/Changes
it is mentioned:
   GNU C                  5.1              gcc --version

And RISC-V support in GCC wad added after 7.0 or so...

But there is also the following words:

   This document is originally based on my "Changes" file for 2.0.x
   kernels
   and therefore owes credit to the same people as that file (Jared
   Mauch,
   Axel Boldt, Alessandro Sigala, and countless other users all over
   the
   'net).

Probably the doc wasn't updated for a long time, but at the same time
there is a line with Rust:
  =20
   Rust (optional)        1.62.0           rustc --version
  =20
~ Oleksii





From xen-devel-bounces@lists.xenproject.org Thu Feb 29 17:04:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 17:04:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687361.1070728 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjpU-00036K-By; Thu, 29 Feb 2024 17:04:20 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687361.1070728; Thu, 29 Feb 2024 17:04:20 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjpU-00036D-8m; Thu, 29 Feb 2024 17:04:20 +0000
Received: by outflank-mailman (input) for mailman id 687361;
 Thu, 29 Feb 2024 17:04:19 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=OAsO=KG=suse.com=jgross@srs-se1.protection.inumbo.net>)
 id 1rfjpT-000362-Ek
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 17:04:19 +0000
Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 92bc3eca-d724-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 18:04:17 +0100 (CET)
Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by smtp-out2.suse.de (Postfix) with ESMTPS id D94611F7FB;
 Thu, 29 Feb 2024 17:04:16 +0000 (UTC)
Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
 (No client certificate requested)
 by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9DB0413A4B;
 Thu, 29 Feb 2024 17:04:16 +0000 (UTC)
Received: from dovecot-director2.suse.de ([10.150.64.162])
 by imap1.dmz-prg2.suse.org with ESMTPSA id h8m9JBC54GWuRwAAD6G6ig
 (envelope-from <jgross@suse.com>); Thu, 29 Feb 2024 17:04:16 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 92bc3eca-d724-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709226256; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CpIhu/fNvYnfU69bJy54gkBsvNjkEKe9ROl0WvT9ts8=;
	b=A3D+twB/qA4CJ/Jy5isozdiOBE2GrHUj0OfSrBG+tY8luFOc3WZCE8Jpy8dD2nJxu6r39H
	w9JQ0hdtS/1NR3IGxQyJQA9XZs6eyaWmhcXi/e3hVU0d0g6Iu0eZTlci6uoQr8EtBuBb8J
	O+g5JRYGxpMd5JioUbWhuUZNiRZnplY=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1;
	t=1709226256; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc:
	 mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=CpIhu/fNvYnfU69bJy54gkBsvNjkEKe9ROl0WvT9ts8=;
	b=A3D+twB/qA4CJ/Jy5isozdiOBE2GrHUj0OfSrBG+tY8luFOc3WZCE8Jpy8dD2nJxu6r39H
	w9JQ0hdtS/1NR3IGxQyJQA9XZs6eyaWmhcXi/e3hVU0d0g6Iu0eZTlci6uoQr8EtBuBb8J
	O+g5JRYGxpMd5JioUbWhuUZNiRZnplY=
Message-ID: <84f700a6-1a98-41a9-bb29-4da2c2f802b0@suse.com>
Date: Thu, 29 Feb 2024 18:04:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/12] xen/spinlock: support higher number of cpus
Content-Language: en-US
To: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-13-jgross@suse.com>
 <c96542eb-daf4-4ce2-915e-86ca8e1848b5@suse.com>
 <f6658df0-13f2-4129-afcd-0f122c87132b@suse.com>
 <f2d107a1-0bcf-4d71-89e8-819ee38e0651@suse.com>
 <58241a32-6671-41e5-8403-5d1b1c4236d2@suse.com>
 <305dcff3-b37b-4f19-bd79-6e8c71c6d850@suse.com>
From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
In-Reply-To: <305dcff3-b37b-4f19-bd79-6e8c71c6d850@suse.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Authentication-Results: smtp-out2.suse.de;
	none
X-Spam-Level: 
X-Spam-Score: -3.58
X-Spamd-Result: default: False [-3.58 / 50.00];
	 ARC_NA(0.00)[];
	 RCVD_VIA_SMTP_AUTH(0.00)[];
	 XM_UA_NO_VERSION(0.01)[];
	 FROM_HAS_DN(0.00)[];
	 TO_DN_SOME(0.00)[];
	 TO_MATCH_ENVRCPT_ALL(0.00)[];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_GOOD(-0.10)[text/plain];
	 NEURAL_HAM_LONG(-1.00)[-1.000];
	 RCVD_COUNT_THREE(0.00)[3];
	 DKIM_SIGNED(0.00)[suse.com:s=susede1];
	 NEURAL_HAM_SHORT(-0.20)[-1.000];
	 RCPT_COUNT_SEVEN(0.00)[7];
	 DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email];
	 FUZZY_BLOCKED(0.00)[rspamd.com];
	 FROM_EQ_ENVFROM(0.00)[];
	 MIME_TRACE(0.00)[0:+];
	 R_MIXED_CHARSET(0.71)[subject];
	 RCVD_TLS_ALL(0.00)[];
	 MID_RHS_MATCH_FROM(0.00)[]
X-Spam-Flag: NO

On 29.02.24 17:54, Jan Beulich wrote:
> On 29.02.2024 17:45, Juergen Gross wrote:
>> On 29.02.24 17:31, Jan Beulich wrote:
>>> On 29.02.2024 17:29, Jürgen Groß wrote:
>>>> On 29.02.24 16:46, Jan Beulich wrote:
>>>>> On 12.12.2023 10:47, Juergen Gross wrote:
>>>>>> Allow 16 bits per cpu number, which is the limit imposed by
>>>>>> spinlock_tickets_t.
>>>>>>
>>>>>> This will allow up to 65535 cpus, while increasing only the size of
>>>>>> recursive spinlocks in debug builds from 8 to 12 bytes.
>>>>>
>>>>> I think we want to be more conservative here, for the case of there
>>>>> being bugs: The CPU holding a lock may wrongly try to acquire it a
>>>>> 2nd time. That's the 65536th ticket then, wrapping the value.
>>>>
>>>> Is this really a problem? There will be no other cpu left seeing the lock
>>>> as "free" in this case, as all others will be waiting for the head to reach
>>>> their private tail value.
>>>
>>> But isn't said CPU then going to make progress, rather than indefinitely
>>> spinning on the lock?
>>
>> No, I don't think so.
> 
> Hmm. If CPU0 takes a pristine lock, it'll get ticket 0x0000. All other
> CPUs will get tickets 0x0001 ... 0xffff. Then CPU0 trying to take the lock

No, they'll get 0x0001 ... 0xfffe (only 65535 cpus are supported).

> again will get ticket 0x0000 again, which equals what .head still has (no

And the first cpu will get 0xffff when trying to get the lock again.

> unlocks so far), thus signalling the lock to be free when it isn't.
> 
>> The limit isn't 65535 because of the ticket mechanism, but because of the
>> rspin mechanism, where we need a "no cpu is owning the lock" value. Without
>> the recursive locks the limit would be 65536 (or 4096 today).
> 
> I think this rather belongs ...

No, that was meant to tell you that without programming bug 65536 cpus would
be perfectly fine for the ticket mechanism, and with bug 65535 cpus are fine.

> 
>>>>> Therefore my suggestion would be to only (mention) go(ing) up to 32k.
>>>>>
>>>>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>>>>> ---
>>>>>>     xen/common/spinlock.c      |  1 +
>>>>>>     xen/include/xen/spinlock.h | 18 +++++++++---------
>>>>>>     2 files changed, 10 insertions(+), 9 deletions(-)
>>>>>
>>>>> Shouldn't this also bump the upper bound of the NR_CPUS range then
>>>>> in xen/arch/Kconfig?
>>>>
>>>> Fine with me, I can add another patch to the series doing that.
>>>
>>> Why not do it right here? The upper bound there is like it is only
>>> because of the restriction that's lifted here.
> 
> ... here (for having nothing to do with the supposed lack of hanging
> that I'm seeing)?
> 
>> I'd prefer splitting the two instances, but if you prefer it to be in a
>> single patch, so be it.
> 
> I'm not going to insist - if want to do it separately, please do.
> Perhaps others would actually prefer it that way ...

Okay. I'll do it in another patch.


Juergen


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 17:05:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 17:05:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687364.1070737 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjr0-0003cJ-Lj; Thu, 29 Feb 2024 17:05:54 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687364.1070737; Thu, 29 Feb 2024 17:05:54 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjr0-0003cC-Is; Thu, 29 Feb 2024 17:05:54 +0000
Received: by outflank-mailman (input) for mailman id 687364;
 Thu, 29 Feb 2024 17:05:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjqz-0003c2-2U
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 17:05:53 +0000
Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com
 [2a00:1450:4864:20::52d])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id cad8ee49-d724-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 18:05:51 +0100 (CET)
Received: by mail-ed1-x52d.google.com with SMTP id
 4fb4d7f45d1cf-563d32ee33aso1695016a12.2
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 09:05:51 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g26-20020a056402321a00b0055fba4996d9sm779204eda.71.2024.02.29.09.05.50
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 09:05:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: cad8ee49-d724-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709226351; x=1709831151; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=SQ057896F41b8xKHo3i3EmInyNgHYs5xAmXc0tMAJw0=;
        b=Qlr9rXjbyilwC9MFqwa59YlDiMwSB7i5NCBEWS5gXkRFeinRebUhby7EDhtkSOEn2+
         EiiIOnh8iPb/dtmm6TM9iaAqgZzkwzcBddcU5Isbs/LK+T/8TvTgVaSG43jQPdA25FKj
         6sWMFfBv4y6bd4Yt/ItnSJOFfkKAt66nk0UYBLSD76Fho19zImvEZq1Uhf/XUnua5t9S
         f4l+W/FxE6Ji6fXIjStCsjQDPubqoNIvZxwL23+AUREzroyxFZiiLAcQ5UM9uHz675Uj
         qXusIkAeXyu9KS+iWj90q9lu6ATAZpEuJcJiwGqjyAkbDsKpj14qkMXF6Djv/KVhD7wq
         nE3A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709226351; x=1709831151;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=SQ057896F41b8xKHo3i3EmInyNgHYs5xAmXc0tMAJw0=;
        b=ne8XfD2zC2mnGBHHjD79hczkquIMQs2zN7oYBu9tu0gkEaE1LvSQ2Ydv+xw06MHDm6
         7nmC6fGmb5j56rHOYI34aTLNFocsCzdj8vvnSIv+stwmueFUrApjayuC24/3H4HBxxL8
         JH0qd2FA7Z6MNsJjYehATgWXmAJ3Vt/0j39D0GbXmV53sA8GwK7TNY+tHLHAuMO+/MuK
         8JB+y8OVNgtol8NvCMEjzkKvp+lJZ6UYzr4kmMVzQHbODxcqGvTxALVBwXL9SHusQzOO
         QP4+2ItEfSqy5Z/miGWdlk0UUIK1XAA6WETvMhhY8Txzp7GBbx+f4VlUUjkXkdp8lTjt
         TrfA==
X-Forwarded-Encrypted: i=1; AJvYcCU5LAnoQnq+XPANu+QXlluvHHqZj8UqzpOg8oKooOILtKfEACiQD3+4RIOlLpVDMSygiYz8wXAPYTyCVvyK6DhEE7QjUUy7OPXc3EGHnKQ=
X-Gm-Message-State: AOJu0YwMpDziOjeLzlHAzVLNxgw7JjXLfgL5YkNE559u60b/Wjrofcx+
	RhkMwUtbfksKfdFB3cwZH8kUOElUECFHCoHkFoMDp7Z3WXEQoR8GRC7cK1cPxw==
X-Google-Smtp-Source: AGHT+IGhsGq3Rcr1toRtnu606f48JbBE+Yt/jiGxXn3NRWm3QKckBRKQe1SGjZlxNZgyloyOHRrEJg==
X-Received: by 2002:a05:6402:95a:b0:566:b326:d105 with SMTP id h26-20020a056402095a00b00566b326d105mr815879edz.24.1709226351191;
        Thu, 29 Feb 2024 09:05:51 -0800 (PST)
Message-ID: <f32720e1-294d-453f-b830-ea524cd27e97@suse.com>
Date: Thu, 29 Feb 2024 18:05:49 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [XEN PATCH 03/10] x86: address some violations of MISRA C Rule
 20.7
Content-Language: en-US
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
Cc: sstabellini@kernel.org, michal.orzel@amd.com, xenia.ragiadakou@amd.com,
 ayan.kumar.halder@amd.com, consulting@bugseng.com,
 andrew.cooper3@citrix.com, roger.pau@citrix.com, bertrand.marquis@arm.com,
 julien@xen.org, Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
References: <cover.1709219010.git.nicola.vetrini@bugseng.com>
 <3c9e90aaf5dde769b689468fc818e4ae61fa11f3.1709219010.git.nicola.vetrini@bugseng.com>
 <14d94c7e-7cd4-4508-a74e-eddd6a592219@suse.com>
 <85c54b2492a87f0538831c8baf5a8c76@bugseng.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <85c54b2492a87f0538831c8baf5a8c76@bugseng.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 29.02.2024 17:45, Nicola Vetrini wrote:
> On 2024-02-29 17:37, Jan Beulich wrote:
>> On 29.02.2024 16:27, Nicola Vetrini wrote:
>>> --- a/xen/arch/x86/include/asm/irq.h
>>> +++ b/xen/arch/x86/include/asm/irq.h
>>> @@ -179,9 +179,9 @@ void cleanup_domain_irq_mapping(struct domain *d);
>>>      void *__ret = radix_tree_lookup(&(d)->arch.hvm.emuirq_pirq, 
>>> emuirq);\
>>>      __ret ? radix_tree_ptr_to_int(__ret) : IRQ_UNBOUND;               
>>>   \
>>>  })
>>> -#define IRQ_UNBOUND -1
>>> -#define IRQ_PT -2
>>> -#define IRQ_MSI_EMU -3
>>> +#define IRQ_UNBOUND (-1)
>>> +#define IRQ_PT      (-2)
>>> +#define IRQ_MSI_EMU (-3)
>>>
>>>  bool cpu_has_pending_apic_eoi(void);
>>>
>>
>> I'd be happy to ack this change right away.
>>
>>> --- a/xen/arch/x86/usercopy.c
>>> +++ b/xen/arch/x86/usercopy.c
>>> @@ -106,7 +106,7 @@ unsigned int copy_from_guest_ll(void *to, const 
>>> void __user *from, unsigned int
>>>      return n;
>>>  }
>>>
>>> -#if GUARD(1) + 0
>>> +#if GUARD((1)) + 0
>>
>> I don't even understand the need for this one, and nothing is said in
>> the description in that regard. Generally I'm afraid I'm averse to
>> such (seemingly) redundant parentheses in macro invocations.
>>
> 
> It's because
> #define UA_KEEP(args...) args
> #define GUARD UA_KEEP
> 
> which would expand to #if 1 + 0, while the rule demands #if (1) + 0
> I did note in the message after --- that I didn't wanna touch UA_KEEP so 
> I did this instead, which I'm not particularly happy about either. I can 
> remove this and deviate, there is no other issue with GUARD.

Or

#if (GUARD(1) + 0)

? To me at least that's quite a bit less odd. But I guess that still
wouldn't satisfy the rule. Perhaps even

#if (GUARD(1)) + 0

would be a little less odd, albeit there I'd already be on the edge.

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 17:07:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 17:07:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687368.1070748 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjsT-00049Q-0T; Thu, 29 Feb 2024 17:07:25 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687368.1070748; Thu, 29 Feb 2024 17:07:24 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfjsS-00049H-T5; Thu, 29 Feb 2024 17:07:24 +0000
Received: by outflank-mailman (input) for mailman id 687368;
 Thu, 29 Feb 2024 17:07:22 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <SRS0=x8AV=KG=suse.com=jbeulich@srs-se1.protection.inumbo.net>)
 id 1rfjsQ-00049B-QC
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 17:07:22 +0000
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
 [2a00:1450:4864:20::532])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 009bec0b-d725-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 18:07:22 +0100 (CET)
Received: by mail-ed1-x532.google.com with SMTP id
 4fb4d7f45d1cf-563b7b3e3ecso1950862a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 09:07:21 -0800 (PST)
Received: from [10.156.60.236] (ip-037-024-206-209.um08.pools.vodafone-ip.de.
 [37.24.206.209]) by smtp.gmail.com with ESMTPSA id
 g26-20020a056402321a00b0055fba4996d9sm779204eda.71.2024.02.29.09.07.20
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 09:07:21 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 009bec0b-d725-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=suse.com; s=google; t=1709226441; x=1709831241; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=kLh97JT7fMjf3Avpvtfftiu9BtIYDYw0gwqX9vMvwxE=;
        b=T7Zc4cvKGBdd/dEvJhrnAXHQ9OKJWChQ/um/1X8hg59j63MtVGyxW2qHR+Jz9Sk8qC
         30V9DaGWPObW8bcu50bW6uEKCSFwBX/Xhbf+KBJpfh5nC5W5u4QwfxYylfO8HWmbgF+2
         mHzHA57TQfJVjYWRxUSoKaWvLO57JUjrdw7xDhRkp0thwO4Ad+eYBbQxUF1U/u9HDz3X
         YphcP9u3H5y+dCzAvw2bANI+eCaqOYrXqOs03/CXWcJjGq+Q8QVhciliY1fbNHSYKbec
         8pY1Dc/ycdEcBTJrbn713f38uENmhr/Wh4je7QUYj5jc3eVN9Bw1QoiRtk0X6uFb3YBv
         UZ8Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709226441; x=1709831241;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=kLh97JT7fMjf3Avpvtfftiu9BtIYDYw0gwqX9vMvwxE=;
        b=rffc3Ot30fd/7JGvPo/odnZpApy4i7bGIfS992kqS9gKlFQiDEwzf8g98KtkP+U33c
         80e0Zq1ZRAzR3GMx6YF5ujjCh9ziSoXsxUlofgQrF/cuDnOytcw9LOH2u9I3cUdETWFS
         XpfewHVTCuOtwrBnE4my9d2sU2vT9ZIdy3aZZfUTCZENruu86yOEXU6rMa+EpQNjyOBY
         Ypn/GsAe0fAvhO/psliTQnX/OGZhfCCPsOe159we8E+7fgzV6fY1m+tpUsKh61M+Kx6v
         l6smNcoCdw0UBrr811km3krit7Aj07DTlycZd7JEPy92NaIxKVL0Cq8TVAW84UqAVCHu
         EMMQ==
X-Forwarded-Encrypted: i=1; AJvYcCUUcjievGPZuER/xibrsxfeT3IKT2rpX8W4nwNE8qQUp6ecgsQIHwqrgIzt5acV2up6LB64xJgU5RrX9rqeR9QiSW/YFB/nddNbqwbMlHU=
X-Gm-Message-State: AOJu0Ywg15IeL/PtzDZ/uvTtaX3ELcrLPBnkbV8RqHIrDduEPDywK0qr
	R6GX96tpxzHqubtSw2P/V/oFa5Tx7N1AnotPAb2KeFsxOtk665S67H6aoYn1Ow==
X-Google-Smtp-Source: AGHT+IHNc/8P03vKpmXdXHcI+Amqx0oXOh1Ibf8PHri3kqk6zBZ4WtxhxkoaGDNcyHbCBihoBE+oaw==
X-Received: by 2002:a05:6402:b62:b0:566:131a:26ee with SMTP id cb2-20020a0564020b6200b00566131a26eemr1996403edb.35.1709226441461;
        Thu, 29 Feb 2024 09:07:21 -0800 (PST)
Message-ID: <fc2dea79-c1de-44c4-97e2-f41610759ccc@suse.com>
Date: Thu, 29 Feb 2024 18:07:20 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 12/12] xen/spinlock: support higher number of cpus
Content-Language: en-US
To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= <jgross@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
 George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>,
 Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
 xen-devel@lists.xenproject.org
References: <20231212094725.22184-1-jgross@suse.com>
 <20231212094725.22184-13-jgross@suse.com>
 <c96542eb-daf4-4ce2-915e-86ca8e1848b5@suse.com>
 <f6658df0-13f2-4129-afcd-0f122c87132b@suse.com>
 <f2d107a1-0bcf-4d71-89e8-819ee38e0651@suse.com>
 <58241a32-6671-41e5-8403-5d1b1c4236d2@suse.com>
 <305dcff3-b37b-4f19-bd79-6e8c71c6d850@suse.com>
 <84f700a6-1a98-41a9-bb29-4da2c2f802b0@suse.com>
From: Jan Beulich <jbeulich@suse.com>
Autocrypt: addr=jbeulich@suse.com; keydata=
 xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk
 hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK
 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD
 /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py
 O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl
 MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP
 nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo
 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp
 Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC
 AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee
 e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF
 hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l
 IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS
 FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj
 t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8
 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3
 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9
 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V
 m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM
 EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr
 wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A
 nAuWpQkjM1ASeQwSHEeAWPgskBQL
In-Reply-To: <84f700a6-1a98-41a9-bb29-4da2c2f802b0@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29.02.2024 18:04, Jürgen Groß wrote:
> On 29.02.24 17:54, Jan Beulich wrote:
>> On 29.02.2024 17:45, Juergen Gross wrote:
>>> On 29.02.24 17:31, Jan Beulich wrote:
>>>> On 29.02.2024 17:29, Jürgen Groß wrote:
>>>>> On 29.02.24 16:46, Jan Beulich wrote:
>>>>>> On 12.12.2023 10:47, Juergen Gross wrote:
>>>>>>> Allow 16 bits per cpu number, which is the limit imposed by
>>>>>>> spinlock_tickets_t.
>>>>>>>
>>>>>>> This will allow up to 65535 cpus, while increasing only the size of
>>>>>>> recursive spinlocks in debug builds from 8 to 12 bytes.
>>>>>>
>>>>>> I think we want to be more conservative here, for the case of there
>>>>>> being bugs: The CPU holding a lock may wrongly try to acquire it a
>>>>>> 2nd time. That's the 65536th ticket then, wrapping the value.
>>>>>
>>>>> Is this really a problem? There will be no other cpu left seeing the lock
>>>>> as "free" in this case, as all others will be waiting for the head to reach
>>>>> their private tail value.
>>>>
>>>> But isn't said CPU then going to make progress, rather than indefinitely
>>>> spinning on the lock?
>>>
>>> No, I don't think so.
>>
>> Hmm. If CPU0 takes a pristine lock, it'll get ticket 0x0000. All other
>> CPUs will get tickets 0x0001 ... 0xffff. Then CPU0 trying to take the lock
> 
> No, they'll get 0x0001 ... 0xfffe (only 65535 cpus are supported).
> 
>> again will get ticket 0x0000 again, which equals what .head still has (no
> 
> And the first cpu will get 0xffff when trying to get the lock again.

Oh, right. Still a little too close to the boundary for my taste ...

Jan


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 17:32:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 17:32:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687374.1070758 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfkGG-0002fx-SF; Thu, 29 Feb 2024 17:32:00 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687374.1070758; Thu, 29 Feb 2024 17:32:00 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfkGG-0002fq-P3; Thu, 29 Feb 2024 17:32:00 +0000
Received: by outflank-mailman (input) for mailman id 687374;
 Thu, 29 Feb 2024 17:31:59 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfkGF-0002fg-Ru; Thu, 29 Feb 2024 17:31:59 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfkGF-0002F6-Oj; Thu, 29 Feb 2024 17:31:59 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfkGF-0005Ea-Bo; Thu, 29 Feb 2024 17:31:59 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfkGF-0002Ok-BI; Thu, 29 Feb 2024 17:31:59 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=qHfXMuR77GOobju0pYLmKAX7sK5PVVA/bD+S8isUuNQ=; b=z/jbLvuM9E0ESSQZB5XiQ4NDAP
	RairX7nbvfEwSJOcN2UFRUYOnRmyjiJ1XsrAKhFKuE5Z1Nt9yAu+Xf4k49R8TsAKLvTiexowaTMdZ
	94Lvfk4BYE+O5V693cIyjoIvspGfzBIOf//m6njZrnXbhn/jVBcx8dyqO2kuKZqVV8Ck=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184813-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-4.18-testing test] 184813: tolerable trouble: fail/pass/starved - PUSHED
X-Osstest-Failures:
    xen-4.18-testing:test-amd64-i386-qemuu-rhel6hvm-amd:redhat-install:fail:heisenbug
    xen-4.18-testing:test-amd64-i386-xl-qemut-debianhvm-i386-xsm:debian-hvm-install:fail:heisenbug
    xen-4.18-testing:test-amd64-i386-xl-qemuu-win7-amd64:windows-install:fail:heisenbug
    xen-4.18-testing:test-amd64-i386-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-xl-pvshim:guest-start:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-amd64-libvirt-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-amd64-i386-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    xen-4.18-testing:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    xen-4.18-testing:test-arm64-arm64-xl-xsm:hosts-allocate:starved:nonblocking
X-Osstest-Versions-This:
    xen=498b3624d0ecc1267773e6482fd0b732e90c4511
X-Osstest-Versions-That:
    xen=58bb8115104c9fca749ee4cfcd3579ac1ed644db
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 29 Feb 2024 17:31:59 +0000

flight 184813 xen-4.18-testing real [real]
flight 184821 xen-4.18-testing real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/184813/
http://logs.test-lab.xenproject.org/osstest/logs/184821/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-i386-qemuu-rhel6hvm-amd 12 redhat-install fail pass in 184821-retest
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass in 184821-retest
 test-amd64-i386-xl-qemuu-win7-amd64 12 windows-install fail pass in 184821-retest

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop   fail in 184821 like 184783
 test-armhf-armhf-libvirt     16 saverestore-support-check    fail  like 184783
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184783
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184783
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184783
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184783
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184783
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop             fail like 184783
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop             fail like 184783
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184783
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184783
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop             fail like 184783
 test-amd64-i386-libvirt-xsm  15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt      15 migrate-support-check        fail   never pass
 test-amd64-i386-xl-pvshim    14 guest-start                  fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-check        fail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm       3 hosts-allocate               starved  n/a

version targeted for testing:
 xen                  498b3624d0ecc1267773e6482fd0b732e90c4511
baseline version:
 xen                  58bb8115104c9fca749ee4cfcd3579ac1ed644db

Last test of basis   184783  2024-02-27 13:07:15 Z    2 days
Testing same since   184813  2024-02-28 23:10:45 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Julien Grall <jgrall@amazon.com>
  Luca Fancellu <luca.fancellu@arm.com>
  Michal Orzel <michal.orzel@amd.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64-xtf                                              pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-prev                                             pass    
 build-i386-prev                                              pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-xtf-amd64-amd64-1                                       pass    
 test-xtf-amd64-amd64-2                                       pass    
 test-xtf-amd64-amd64-3                                       pass    
 test-xtf-amd64-amd64-4                                       pass    
 test-xtf-amd64-amd64-5                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-coresched-i386-xl                                 pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm            pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm         pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm                  fail    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm                  pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-i386-libvirt-xsm                                  pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      starved 
 test-amd64-i386-xl-xsm                                       pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-qemuu-freebsd11-amd64                       pass    
 test-amd64-amd64-qemuu-freebsd12-amd64                       pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-i386-xl-qemut-ws16-amd64                          fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-amd64-i386-xl-qemuu-ws16-amd64                          fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict         pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-i386-libvirt                                      pass    
 test-amd64-amd64-livepatch                                   pass    
 test-amd64-i386-livepatch                                    pass    
 test-amd64-amd64-migrupgrade                                 pass    
 test-amd64-i386-migrupgrade                                  pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-i386-libvirt-pair                                 pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-i386-xl-pvshim                                    fail    
 test-amd64-amd64-pygrub                                      pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-xl-qcow2                                    pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-i386-libvirt-raw                                  pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow              pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-amd64-i386-xl-shadow                                    pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-libvirt-vhd                                 pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    
 test-amd64-i386-xl-vhd                                       pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   58bb811510..498b3624d0  498b3624d0ecc1267773e6482fd0b732e90c4511 -> stable-4.18


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 17:43:15 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 17:43:15 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687387.1070772 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfkR4-0005Cj-2l; Thu, 29 Feb 2024 17:43:10 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687387.1070772; Thu, 29 Feb 2024 17:43:10 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfkR3-0005Cc-UY; Thu, 29 Feb 2024 17:43:09 +0000
Received: by outflank-mailman (input) for mailman id 687387;
 Thu, 29 Feb 2024 17:43:09 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfkR3-0005CW-4X
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 17:43:09 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id ff1da872-d729-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 18:43:07 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 064D5612B4;
 Thu, 29 Feb 2024 17:43:06 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 99010C433F1;
 Thu, 29 Feb 2024 17:43:04 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ff1da872-d729-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709228585;
	bh=laGnU43JNDV9gD0iueA78YHAUkpfIV5vXOftx9ZRrNs=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=f4HUGy+juqqcJnWvc7vRlVpuYsAqiFyXW7q++y5FqW+GbGBZ4vnE1qDvHZMWFEHjD
	 KvUAz67XUYwVya9TeMdPRGM2jvh3qV4mGihXDVc1Nnzw8MMxSfeDqDZgbtdwA35vxe
	 L0FNZMZQP7HnqvVM0pUTfQCMRmqMQ+lQYAPrVygWgj2B7sYA5SlxNHEY+dTNxvBdu4
	 2uRUEB7IlJDSu5LXoslGZwNkIS6RTxO0GQHr9ZJFP3xs/5zUK2o5OdMDZZuBXtWX0X
	 /DVbCN/zFWB9RBxbN+vp5Kl6CtjVBfthwAqYIRhREEYpQ9XmoTvM0HTF4ZThx9lzas
	 W4WJ8QQ8j4mog==
Date: Thu, 29 Feb 2024 09:43:00 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Julien Grall <julien@xen.org>
cc: Jan Beulich <jbeulich@suse.com>, George Dunlap <george.dunlap@citrix.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    xen-devel@lists.xenproject.org, 
    Oleksii Kurochko <oleksii.kurochko@gmail.com>, 
    Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v5 23/23] xen/README: add compiler and binutils versions
 for RISC-V64
In-Reply-To: <b86733f5-e61e-43ab-8375-8a3c7480c8d3@xen.org>
Message-ID: <alpine.DEB.2.22.394.2402290936320.853156@ubuntu-linux-20-04-desktop>
References: <cover.1708962629.git.oleksii.kurochko@gmail.com> <ebaa05ded3cfddb1e3f7c87cd806b866312878fb.1708962629.git.oleksii.kurochko@gmail.com> <73a38d85-0798-4cfa-8625-0f8c9e05b2a3@suse.com> <80fb8091-9a26-40a4-b2e7-371df9411861@xen.org>
 <889b132b-b92e-465b-94af-4e4133e4c297@suse.com> <52188e88-374c-449f-a71c-bfce9b6d07f4@xen.org> <cd52114b-c4f1-4bb9-96a4-d802dc50bade@citrix.com> <2a112321-f4fd-4ab4-a431-a0bb4cb6eaa2@suse.com> <9b212dec-59ef-4cbb-bea7-add3867e4599@xen.org>
 <aa3fbfca-4675-4f92-9037-a4a7c5eca4dd@suse.com> <8eed0200-eabf-4024-b018-74be45ef95db@xen.org> <f28e22b7-2657-40d9-8a97-e747c689c2b0@suse.com> <b86733f5-e61e-43ab-8375-8a3c7480c8d3@xen.org>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: multipart/mixed; BOUNDARY="8323329-760596596-1709228480=:853156"
Content-ID: <alpine.DEB.2.22.394.2402290941260.853156@ubuntu-linux-20-04-desktop>

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-760596596-1709228480=:853156
Content-Type: text/plain; CHARSET=UTF-8
Content-Transfer-Encoding: 8BIT
Content-ID: <alpine.DEB.2.22.394.2402290941261.853156@ubuntu-linux-20-04-desktop>

On Thu, 29 Feb 2024, Julien Grall wrote:
> On 29/02/2024 14:07, Jan Beulich wrote:
> > On 29.02.2024 14:44, Julien Grall wrote:
> > > Hi Jan,
> > > 
> > > On 29/02/2024 12:51, Jan Beulich wrote:
> > > > On 29.02.2024 13:32, Julien Grall wrote:
> > > > > On 29/02/2024 12:17, Jan Beulich wrote:
> > > > > > On 29.02.2024 13:05, Andrew Cooper wrote:
> > > > > > > On 29/02/2024 10:23 am, Julien Grall wrote:
> > > > > > > > > > > IOW it is hard for me to see why RISC-V needs stronger
> > > > > > > > > > > restrictions
> > > > > > > > > > > here
> > > > > > > > > > > than other architectures. It ought to be possible to
> > > > > > > > > > > determine a
> > > > > > > > > > > baseline
> > > > > > > > > > > version. Even if taking the desire to have "pause"
> > > > > > > > > > > available as a
> > > > > > > > > > > requirement, gas (and presumably gld) 2.36.1 would already
> > > > > > > > > > > suffice.
> > > > > > > > > > 
> > > > > > > > > > I think we want to bump it on Arm. There are zero reasons to
> > > > > > > > > > try to
> > > > > > > > > > keep
> > > > > > > > > > a lower versions if nobody tests/use it in production.
> > > > > > > > > > 
> > > > > > > > > > I would suggest to do the same on x86. What's the point of
> > > > > > > > > > try to
> > > > > > > > > > support Xen with a 15+ years old compiler?
> > > > > > > > > 
> > > > > > > > > It could have long been bumped if only a proper scheme to
> > > > > > > > > follow for
> > > > > > > > > this and future bumping would have been put forward by anyone
> > > > > > > > > keen on
> > > > > > > > > such bumping, like - see his reply - e.g. Andrew. You may
> > > > > > > > > recall that
> > > > > > > > > this was discussed more than once on meetings, with no real
> > > > > > > > > outcome.
> > > > > > > > > I'm personally not meaning to stand in the way of such bumping
> > > > > > > > > as long
> > > > > > > > > as it's done in a predictable manner, but I'm not keen on
> > > > > > > > > doing so and
> > > > > > > > > hence I don't view it as my obligation to try to invent a
> > > > > > > > > reasonable
> > > > > > > > > scheme. (My personal view is that basic functionality should
> > > > > > > > > be
> > > > > > > > > possible to have virtually everywhere, whereas for advanced
> > > > > > > > > stuff it
> > > > > > > > > is fine to require a more modern tool chain.)
> > > > > > > > 
> > > > > > > > That's one way to see it. The problem with this statement is a
> > > > > > > > user
> > > > > > > > today is mislead to think you can build Xen with any GCC
> > > > > > > > versions
> > > > > > > > since 4.1. I don't believe we can guarantee that and we are
> > > > > > > > exposing
> > > > > > > > our users to unnecessary risk.
> > > > > > > > 
> > > > > > > > In addition to that, I agree with Andrew. This is preventing us
> > > > > > > > to
> > > > > > > > improve our code base and we have to carry hacks for older
> > > > > > > > compilers.
> > > > > > > 
> > > > > > > I don't think anyone here is suggesting that we switch to a
> > > > > > > bleeding-edge-only policy.  But 15y of support is extreme in the
> > > > > > > opposite direction.
> > > > > > > 
> > > > > > > Xen ought to be buildable in the contemporary distros of the day,
> > > > > > > and I
> > > > > > > don't think anyone is going to credibly argue otherwise.
> > > > > > > 
> > > > > > > But, it's also fine for new things to have newer requirements.
> > > > > > > 
> > > > > > > Take CET for example.  I know we have disagreements on exactly how
> > > > > > > it's
> > > > > > > toolchain-conditionalness is implemented, but the basic principle
> > > > > > > of "If
> > > > > > > you want shiny new optional feature $X, you need newer toolchain
> > > > > > > $Y" is
> > > > > > > entirely fine.
> > > > > > > 
> > > > > > > A brand new architecture is exactly the same.  Saying "this is the
> > > > > > > minimum, because it's what we test" doesn't preclude someone
> > > > > > > coming
> > > > > > > along and saying "can we use $N-1 ?  See here it works, and here's
> > > > > > > a
> > > > > > > change to CI test it".
> > > > > > > 
> > > > > > > 
> > > > > > > Anyway, its clear we need to write some policy on this, before
> > > > > > > making
> > > > > > > specific adjustments.  To get started, is there going to be any
> > > > > > > objection whatsoever on some principles which begin as follows:
> > > > > > 
> > > > > > Largely not, but one aspect needs clarifying up front:
> > > > > > 
> > > > > > > * For established architectures, we expect Xen to be buildable on
> > > > > > > the
> > > > > > > common contemporary distros.  (i.e. minima is not newer than
> > > > > > > what's
> > > > > > > available in contemporary distros, without a good reason)
> > > > > > 
> > > > > > What counts as contemporary distro? Still in normal support? LTS?
> > > > > > Yet
> > > > > > more extreme forms?
> > > > > 
> > > > > LTS makes sense. More I am not sure. I am under the impression that
> > > > > people using older distros are those that wants a stable system. So
> > > > > they
> > > > > would unlikely try to upgrade the hypervisor.
> > > > > 
> > > > > Even for LTS, I would argue that if it has been released 5 years ago,
> > > > > then you probably want to update it at the same time as moving to a
> > > > > newer Xen version.
> > > > 
> > > > For the purposes of distros I agree. For the purposes of individuals
> > > > I don't: What's wrong with running a newer hypervisor and/or kernel
> > > > underneath an older distro?
> > > 
> > > There is nothing wrong. I just don't understand the benefits for us to
> > > support that use case. To me there are two sorts of individuals:
> > >    1. The ones that are using distro packages. They will unlikely want to
> > > switch to a newer hypervisor
> > >    2. The ones that are happy to compile and hack their system. Fairly
> > > likely they will use a more distros and/or would not be put up by
> > > upgrading it.
> > > 
> > > What individuals do you have in mind?
> > 
> > People like me.
> 
> Which means? From what I read you mostly use an older distros for smoke
> testing/convenience.

This is a cost/benefit decision. Supporting new Xen on ancient distros
has a cost for maintainers and contributors. Is it worth paying this
cost for the benefits it provides?

It is natural that different people are going to have different opinions
on this, because their use-case and situation are different.

We need to define exactly what "ancient" distro means, but from what I
see my answer is "no, it is not worth it".

For example, for ARM I would raise the minimum GCC version to something
around GCC5.
--8323329-760596596-1709228480=:853156--


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 17:47:42 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 17:47:42 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687389.1070782 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfkVP-0005xk-IW; Thu, 29 Feb 2024 17:47:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687389.1070782; Thu, 29 Feb 2024 17:47:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfkVP-0005xd-Fl; Thu, 29 Feb 2024 17:47:39 +0000
Received: by outflank-mailman (input) for mailman id 687389;
 Thu, 29 Feb 2024 17:47:38 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfkVO-0005xX-DJ
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 17:47:38 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 9f895295-d72a-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 18:47:37 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 000E6CE17A3;
 Thu, 29 Feb 2024 17:47:34 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3B603C433F1;
 Thu, 29 Feb 2024 17:47:33 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 9f895295-d72a-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709228854;
	bh=h+725gTgwrWW08pKog7k0AVSNurxSbMcVAWqMnf9LZc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=dwYdLPT7rlis1mT3+QBiGR/3MWL9Sr3zpjfp5qLrGntbU5FyxMsZ9JQPZgPlnhbXf
	 JqblHLMqTr+xpXbbUUgw2gTs4+PpufxZPqU8VTDQLiBYZ6c74ETPPQIScZ3mBxIR0d
	 Gyh05sVzIMTYQWGPfkN3AO4Bm3kdopL0MvC0ElI5+PfGoaQGH9t0IvkpeUj94jl0Zq
	 H+2zcxEzBgUotVaSJUdx27/kdix9CZsSEXhBFUZ94JA2q+E7g0X+8PdfuiEW30JNNL
	 i/8ppn5yTJK7/BwpNYLdC7xVbeDJTyXSzryei6Zyy2fFNZbW+Zqf0kF0GkL04ia7F8
	 1wP5XnU9eJxQA==
Date: Thu, 29 Feb 2024 09:47:31 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>, 
    Anthony Perard <anthony.perard@citrix.com>, 
    Kelly Choi <kelly.choi@cloud.com>, 
    "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>, 
    Julien Grall <julien@xen.org>, michal.orzel@amd.com
Subject: Re: preparations for 4.18.1
In-Reply-To: <00881c6e-22a5-4283-b071-a9b92fbaf89a@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402290945210.853156@ubuntu-linux-20-04-desktop>
References: <7dc621a0-924c-499c-86c1-c35ec1f34ec2@suse.com> <1f251f2e-91de-4f81-a93c-dcb95746d2e5@xen.org> <5a4e8583-7747-4f24-94f0-d040dabb5b04@suse.com> <521a4d3b-8ef1-4449-97bd-1a3d3e0e35ec@xen.org> <alpine.DEB.2.22.394.2402281445450.575685@ubuntu-linux-20-04-desktop>
 <00881c6e-22a5-4283-b071-a9b92fbaf89a@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 29 Feb 2024, Jan Beulich wrote:
> On 28.02.2024 23:45, Stefano Stabellini wrote:
> > On Wed, 28 Feb 2024, Julien Grall wrote:
> >> On 28/02/2024 12:58, Jan Beulich wrote:
> >>> On 28.02.2024 12:50, Julien Grall wrote:
> >>>> On 27/02/2024 13:19, Jan Beulich wrote:
> >>>>> the release is due in two to three weeks. Please point out backports you
> >>>>> find
> >>>>> missing from the respective staging branch, but which you consider
> >>>>> relevant.
> >>>>
> >>>> For Arm:
> >>>>
> >>>> e11f576650 ("xen/arm: Fix UBSAN failure in start_xen()")
> >>>
> >>> Which I assume you or Stefano will take care of?
> >>
> >> I was expecting Stefano would do it as he did the backports in the past.
> 
> Was it deliberate that you did not also put it on the 4.17 branch?

Yes, I wasn't sure how far back we want to backport so I thought it
would be better to check with Julien and Michal.


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 18:14:10 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 18:14:10 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687394.1070792 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfkuw-0003Yb-Ji; Thu, 29 Feb 2024 18:14:02 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687394.1070792; Thu, 29 Feb 2024 18:14:02 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfkuw-0003YU-Gr; Thu, 29 Feb 2024 18:14:02 +0000
Received: by outflank-mailman (input) for mailman id 687394;
 Thu, 29 Feb 2024 18:14:01 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfkuv-0003YO-Jr
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 18:14:01 +0000
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com
 [2a00:1450:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4f3bdb7b-d72e-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 19:13:59 +0100 (CET)
Received: by mail-ej1-x631.google.com with SMTP id
 a640c23a62f3a-a293f2280c7so251959766b.1
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 10:13:59 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 gs41-20020a1709072d2900b00a3d9e6e9983sm920149ejc.174.2024.02.29.10.13.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 10:13:56 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4f3bdb7b-d72e-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709230438; x=1709835238; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=F5eq2gDanKu26sYGf0JeKb4Y5I3SBwePWVmjX/aO+oM=;
        b=jH4YBtPcHT9+B+6qmxEDXR8hMW6ZtgVQFG4Ec82Q9zCxZaYI85mi6DfGCX6F4qebDh
         qtnKd2kzL+fL84/LtCZ/krQwaS+RiwnZCjDL2GVeei+liGjUGU6lQG5r2U05sfhEBMrq
         xro/VmcKpVc3nS3xS4thWob6xuI2h78sGOEUQ=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709230438; x=1709835238;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=F5eq2gDanKu26sYGf0JeKb4Y5I3SBwePWVmjX/aO+oM=;
        b=GCKw8k8TJ6pMZdtZOCq4yVmrAJpwFNmSg3woxHZpZy3lgxfpSmOLU7308HfjFX0+S7
         /DmVBNhuompN+hPGk36QPN3HjQDYggvZy+91N74QKM7qgoBcqxwCVetTmm+xQjoajg9V
         LbsKuTAVCshWMbhoc+V3gkZ9V4YpzCvgXwbbuVsLq+1TCVDOp2eogukNhHzsUvM4hqO3
         qlETwTmnfUbsWkL2uvuYD/QZtL4L7+qpDEv7ZcHEBRTcp/DoPowy3nAWaFqNaycJgvdW
         t8NfAtEUrxx0CjW1fXq8oPyhcV77iqIKpxsW60/xlDzQcBv+mco57NCkrqCcoa/BCaMJ
         WziQ==
X-Gm-Message-State: AOJu0YzM1JIh7pNuI71PYBr97cpfbg86V3gHNx9b2XSefhrbHmupsbj6
	vcpLwIZPDRke7QA/Tq57eir5sIKvBBy5JE85n3JY192mi9zX7pWMIe1kf73Ly4hMUWdQ2gRyCXL
	S
X-Google-Smtp-Source: AGHT+IH8BKZ8PMWYkZ046yAG3mTPMFd4XNAvPL9mFFkvxwVI30tPXCmRMFgRW4itFlHsp1Uqisnstg==
X-Received: by 2002:a17:906:6956:b0:a44:5a3d:7883 with SMTP id c22-20020a170906695600b00a445a3d7883mr1316934ejs.10.1709230437735;
        Thu, 29 Feb 2024 10:13:57 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/cpu-policy: Allow for levelling of VERW side effects
Date: Thu, 29 Feb 2024 18:13:54 +0000
Message-Id: <20240229181354.2560819-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

MD_CLEAR and FB_CLEAR need OR-ing across a migrate pool.  Allow this, by
having them unconditinally set in max, with the host values reflected in
default.  Annotate the bits as having special properies.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 xen/arch/x86/cpu-policy.c                   | 24 +++++++++++++++++++++
 xen/include/public/arch-x86/cpufeatureset.h |  4 ++--
 2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index f3ed2d3a3227..41123e6cf778 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -442,6 +442,16 @@ static void __init guest_common_max_feature_adjustments(uint32_t *fs)
         __set_bit(X86_FEATURE_RSBA, fs);
         __set_bit(X86_FEATURE_RRSBA, fs);
 
+        /*
+         * These bits indicate that the VERW instruction may have gained
+         * scrubbing side effects.  With pooling, they mean "you might migrate
+         * somewhere where scrubbing is necessary", and may need exposing on
+         * unaffected hardware.  This is fine, because the VERW instruction
+         * has been around since the 286.
+         */
+        __set_bit(X86_FEATURE_MD_CLEAR, fs);
+        __set_bit(X86_FEATURE_FB_CLEAR, fs);
+
         /*
          * The Gather Data Sampling microcode mitigation (August 2023) has an
          * adverse performance impact on the CLWB instruction on SKX/CLX/CPX.
@@ -476,6 +486,20 @@ static void __init guest_common_default_feature_adjustments(uint32_t *fs)
              cpu_has_rdrand && !is_forced_cpu_cap(X86_FEATURE_RDRAND) )
             __clear_bit(X86_FEATURE_RDRAND, fs);
 
+        /*
+         * These bits indicate that the VERW instruction may have gained
+         * scrubbing side effects.  The max policy has them set for migration
+         * reasons, so reset the default policy back to the host values in
+         * case we're unaffected.
+         */
+        fs[FEATURESET_7d0]   &= ~cpufeat_mask(X86_FEATURE_MD_CLEAR);
+        fs[FEATURESET_m10Al] &= ~cpufeat_mask(X86_FEATURE_FB_CLEAR);
+
+        fs[FEATURESET_7d0]   |= (boot_cpu_data.x86_capability[FEATURESET_7d0] &
+                                 cpufeat_mask(X86_FEATURE_MD_CLEAR));
+        fs[FEATURESET_m10Al] |= (boot_cpu_data.x86_capability[FEATURESET_m10Al] &
+                                 cpufeat_mask(X86_FEATURE_FB_CLEAR));
+
         /*
          * The Gather Data Sampling microcode mitigation (August 2023) has an
          * adverse performance impact on the CLWB instruction on SKX/CLX/CPX.
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index b230d3a6907d..0374cec3a2af 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -262,7 +262,7 @@ XEN_CPUFEATURE(AVX512_4FMAPS, 9*32+ 3) /*A  AVX512 Multiply Accumulation Single
 XEN_CPUFEATURE(FSRM,          9*32+ 4) /*A  Fast Short REP MOVS */
 XEN_CPUFEATURE(AVX512_VP2INTERSECT, 9*32+8) /*a  VP2INTERSECT{D,Q} insns */
 XEN_CPUFEATURE(SRBDS_CTRL,    9*32+ 9) /*   MSR_MCU_OPT_CTRL and RNGDS_MITG_DIS. */
-XEN_CPUFEATURE(MD_CLEAR,      9*32+10) /*A  VERW clears microarchitectural buffers */
+XEN_CPUFEATURE(MD_CLEAR,      9*32+10) /*!A VERW clears microarchitectural buffers */
 XEN_CPUFEATURE(RTM_ALWAYS_ABORT, 9*32+11) /*! June 2021 TSX defeaturing in microcode. */
 XEN_CPUFEATURE(TSX_FORCE_ABORT, 9*32+13) /* MSR_TSX_FORCE_ABORT.RTM_ABORT */
 XEN_CPUFEATURE(SERIALIZE,     9*32+14) /*A  SERIALIZE insn */
@@ -334,7 +334,7 @@ XEN_CPUFEATURE(DOITM,              16*32+12) /*   Data Operand Invariant Timing
 XEN_CPUFEATURE(SBDR_SSDP_NO,       16*32+13) /*A  No Shared Buffer Data Read or Sideband Stale Data Propagation */
 XEN_CPUFEATURE(FBSDP_NO,           16*32+14) /*A  No Fill Buffer Stale Data Propagation */
 XEN_CPUFEATURE(PSDP_NO,            16*32+15) /*A  No Primary Stale Data Propagation */
-XEN_CPUFEATURE(FB_CLEAR,           16*32+17) /*A  Fill Buffers cleared by VERW */
+XEN_CPUFEATURE(FB_CLEAR,           16*32+17) /*!A Fill Buffers cleared by VERW */
 XEN_CPUFEATURE(FB_CLEAR_CTRL,      16*32+18) /*   MSR_OPT_CPU_CTRL.FB_CLEAR_DIS */
 XEN_CPUFEATURE(RRSBA,              16*32+19) /*!  Restricted RSB Alternative */
 XEN_CPUFEATURE(BHI_NO,             16*32+20) /*A  No Branch History Injection  */

base-commit: 54fd7b997470e6686667ca8e18f9ba6139efcdea
prerequisite-patch-id: d2cbc8f341e98ccfd66016f19532df3ddbfc68a4
prerequisite-patch-id: 4b4799fae62b5f41b9b0d2078e8b081605341a0a
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 19:09:20 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 19:09:20 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687397.1070802 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rflmF-0002Xj-KB; Thu, 29 Feb 2024 19:09:07 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687397.1070802; Thu, 29 Feb 2024 19:09:07 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rflmF-0002Xc-GI; Thu, 29 Feb 2024 19:09:07 +0000
Received: by outflank-mailman (input) for mailman id 687397;
 Thu, 29 Feb 2024 19:09:06 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rflmD-0002XP-UO
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 19:09:06 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id ffab313f-d735-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 20:09:02 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 3B9AC61325;
 Thu, 29 Feb 2024 19:09:01 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 293EBC433B2;
 Thu, 29 Feb 2024 19:09:00 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: ffab313f-d735-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709233741;
	bh=/8j+WRa1L45Y9Ru5UpgPpTU8E8b9p2u+DaXfN8JZnnc=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=CYABx7hpOmAtU6q1UqKnOOpd3j1lsWjbrR/7SmoMic+PtgVQiobk8mmbft3bqZKiZ
	 c5pZzSaliyPj8OiqHdAPlSCVef60ivWYVJ595ubmIGi/UW8LPBQ8BqbEKDXz8SbQo2
	 +pyQf/uaK2N67v5xDtdvy+Xe51ljGplBra/AP/aD1XqulvCozwKTSOyTXNtV5Prl1u
	 wSN4lb7DmKSd7gVKPWE97cYbB801RoT8ewuj5ZPRMJifZxHLeoqxIng5flZIhGS/Wb
	 nNgaXhiluvNJrYm9GaFRDLOW+0/4zyOcc5P0g3XPs6snzDjfR2v1J0xWMqQoWX7L/t
	 /yekTvXDyHYkQ==
Date: Thu, 29 Feb 2024 11:08:58 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Michal Orzel <michal.orzel@amd.com>
cc: Julien Grall <julien@xen.org>, Stefano Stabellini <sstabellini@kernel.org>, 
    xen-devel@lists.xenproject.org, 
    Bertrand Marquis <bertrand.marquis@arm.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [PATCH] xen/arm: Fix arm32 build failure when early printk is
 enabled
In-Reply-To: <5671ee78-eeba-4bfc-bd95-dadb07c55cbd@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402291108290.853156@ubuntu-linux-20-04-desktop>
References: <20240228103555.172101-1-michal.orzel@amd.com> <9201bdc5-2e4c-48b7-8c92-f5063fde90b6@xen.org> <af11620a-ec16-4677-b968-d8e36c4555d2@amd.com> <alpine.DEB.2.22.394.2402281425260.575685@ubuntu-linux-20-04-desktop> <8228a48f-566c-4931-9f24-27bdaa1f0f28@amd.com>
 <99b74dda-225b-4402-abb9-fcccfda0adbc@xen.org> <5671ee78-eeba-4bfc-bd95-dadb07c55cbd@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 29 Feb 2024, Michal Orzel wrote:
> On 29/02/2024 11:10, Julien Grall wrote:
> > 
> > 
> > Hi,
> > 
> > On 29/02/2024 10:07, Michal Orzel wrote:
> >>
> >>
> >> On 28/02/2024 23:27, Stefano Stabellini wrote:
> >>>
> >>>
> >>> On Wed, 28 Feb 2024, Michal Orzel wrote:
> >>>> Hi Julien,
> >>>>
> >>>> On 28/02/2024 12:42, Julien Grall wrote:
> >>>>>
> >>>>>
> >>>>> Hi Michal,
> >>>>>
> >>>>> On 28/02/2024 10:35, Michal Orzel wrote:
> >>>>>> Commit 0441c3acc7e9 forgot to rename FIXMAP_CONSOLE to FIX_CONSOLE in
> >>>>>> TEMPORARY_EARLY_UART_VIRTUAL_ADDRESS macro. This results in a build
> >>>>>> failure on arm32, when early printk is enabled:
> >>>>>> arch/arm/arm32/mmu/head.S:311: Error: invalid operands (*UND* and *ABS* sections) for `*'
> >>>>>
> >>>>> Good catch! Somewhat related I wonder whether we should add earlyprintk
> >>>>> testing in gitlab?
> >>>> I thought about adding this and I think we should at least have build jobs (hypervisor only, no toolstack)
> >>>> selecting early printk. When it comes to testing if early printk works, I'm not sure. It'd be nice
> >>>> but FWIR we have limited bandwidth.
> >>>>
> >>>> @Stefano, what's your opinion?
> >>>
> >>> I think it would be a good and quick test to have. To save testing
> >>> bandwidth I think we should reduce the amount of debug/non-debug
> >>> variations of the same tests that we have.
> >> Yes, I suggested that some time ago. We could keep both versions for generic tests,
> >> but remove the non-debug version (unless you prefer to do the opposite) for:
> > 
> > I think it makes sense during development window to use the debug
> > version. However, I think we want some non-debug testing during the
> > hardening phase.
> > 
> > Can gitlab read CONFIG_DEBUG from Config.mk?
> At the moment, we have 2 types of jobs - non debug and debug (with -debug suffix).
> They set "debug" variable accordingly, which is used later on to modify .config:
> echo "CONFIG_DEBUG=${debug}" >> xen/.config
> 
> Without this line, Xen would be built according to default value of CONFIG_DEBUG.
> That said, I don't think we want to get back to this behavior.
> 
> If we want to save some bandwidth, we should make a decision whether to keep debug or non-debug versions.
> x86 has both versions for build jobs and mostly debug test jobs.

It is good to have some debug and non-debug jobs, but we probably don't
need both versions of every job.


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 19:09:37 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 19:09:37 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687398.1070812 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rflmj-00030D-RT; Thu, 29 Feb 2024 19:09:37 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687398.1070812; Thu, 29 Feb 2024 19:09:37 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rflmj-000304-OB; Thu, 29 Feb 2024 19:09:37 +0000
Received: by outflank-mailman (input) for mailman id 687398;
 Thu, 29 Feb 2024 19:09:36 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rflmi-0002zk-DE
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 19:09:36 +0000
Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com
 [2607:f8b0:4864:20::736])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 13604487-d736-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 20:09:35 +0100 (CET)
Received: by mail-qk1-x736.google.com with SMTP id
 af79cd13be357-787f27e634cso93929485a.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 11:09:35 -0800 (PST)
Received: from [10.80.67.149] (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 i4-20020a37c204000000b00787548de251sm946454qkm.23.2024.02.29.11.09.32
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Thu, 29 Feb 2024 11:09:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 13604487-d736-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709233774; x=1709838574; darn=lists.xenproject.org;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :from:to:cc:subject:date:message-id:reply-to;
        bh=IGBPVXm+xoX/IFviVhInKWzNzminbH6fIphLnWqkNDU=;
        b=Fe5rZ99/3taDd3EgW1ZHcDs/rdsZYB4lNt1UpadjsGguDqBjt6JfRO+U0nWu5lIFOr
         WngktVbhUl9vxRc7AJahdzLX4ccJHE3ptULqcmzlmKDtQQJEaXQTwKLR6W01u9M4RwMp
         zZIzH9ELf6RvoIqtxZSQTY4z6TkGn+brbk/Jk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709233774; x=1709838574;
        h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc
         :to:content-language:subject:user-agent:mime-version:date:message-id
         :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
        bh=IGBPVXm+xoX/IFviVhInKWzNzminbH6fIphLnWqkNDU=;
        b=nFze1HWYcEAUw6lJkVYRbGRvospXRCK83fyDyynteE+sQ6QpJQQJxOHfYTPnTArdZR
         d6+buhrBcWEBAbMWmlxR82fVKZew5FYPCPrA6c25NzONhiJqq+japkhxYdCRWBlSazAT
         pY+z63DQSJhNH4gIELjul4beQisPPug8et1IHrUoV7puPgMjDw8EfLI8GyFRpcPQzh+S
         c1jtGHIf75nAuV/RwF2M/2SHk1PPacvbEj1jRQqaWmaESngGfAlM2DXdm8FybwnbrhLx
         aTsZP5WtoALsdumR5kYxwp9sYjT5N7LQMhspwWY+6s+VY0iWz1xKAFCQf7F5aqsKnqHH
         paOg==
X-Forwarded-Encrypted: i=1; AJvYcCXVoPhHuxs28jlvozYqUF5Ae/Sf8LQ1l6u9HWYALFIISXrCbGCCsO5EtlHRTXgxUoUBch/N1lMXGk1PhMbZClJk4e3K6UX2bu+pxgWR7uE=
X-Gm-Message-State: AOJu0YzfnktXWnHzwl4Eexmap8CMKwrk/cxwIuqebwPWGWLQdZopxSmv
	ASE8NKvypG+VfC0axL35rKlegwRySxs1oyjUJAFJy3jPr/R2M/1bD83mNBgOK5k=
X-Google-Smtp-Source: AGHT+IHFLVA/LtYlBwoJfvAcH+al1gCFiKz0szThJmLCGXSFUwBuFcSfn12b+U0kCb5+JQybfGuMIA==
X-Received: by 2002:a05:620a:148e:b0:784:93d:c905 with SMTP id w14-20020a05620a148e00b00784093dc905mr3655151qkj.25.1709233774287;
        Thu, 29 Feb 2024 11:09:34 -0800 (PST)
Message-ID: <0252e902-1c40-41a9-813d-51aa0c71ebb2@citrix.com>
Date: Thu, 29 Feb 2024 19:09:31 +0000
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/cpu-policy: Fix x2APIC visibility for PV guests
Content-Language: en-GB
To: Jan Beulich <jbeulich@suse.com>
Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= <roger.pau@citrix.com>,
 Wei Liu <wl@xen.org>, Xen-devel <xen-devel@lists.xenproject.org>
References: <20240229104304.2478614-1-andrew.cooper3@citrix.com>
 <707ff486-b448-4bc0-be7e-1c692b360734@suse.com>
From: Andrew Cooper <andrew.cooper3@citrix.com>
Autocrypt: addr=andrew.cooper3@citrix.com; keydata=
 xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp
 VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn
 srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR
 Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E
 ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5
 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe
 LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV
 e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5
 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ
 ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v
 cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI
 CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO
 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh
 IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4
 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z
 JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK
 mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET
 ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy
 RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi
 dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF
 /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt
 TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4
 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn
 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p
 vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU
 g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy
 wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd
 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i
 kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1
 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk
 uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB
 XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ
 HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd
 pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA
 vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk
 b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg
 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP
 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i
 nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ
 B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo
 d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs
 6+ahAA==
In-Reply-To: <707ff486-b448-4bc0-be7e-1c692b360734@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 29/02/2024 12:47 pm, Jan Beulich wrote:
>> @@ -830,11 +846,10 @@ void recalculate_cpuid_policy(struct domain *d)
>>      }
>>  
>>      /*
>> -     * Allow the toolstack to set HTT, X2APIC and CMP_LEGACY.  These bits
>> +     * Allow the toolstack to set HTT and CMP_LEGACY.  These bits
>>       * affect how to interpret topology information in other cpuid leaves.
>>       */
>>      __set_bit(X86_FEATURE_HTT, max_fs);
>> -    __set_bit(X86_FEATURE_X2APIC, max_fs);
>>      __set_bit(X86_FEATURE_CMP_LEGACY, max_fs);
>>  
>>      /*
> ... these adjustments, just still in calculate_pv_featureset(). I
> haven't gone further backwards to check if/when this exposure has
> really appeared. I wouldn't be surprised if it's been like that
> for all the time since we gained x2APIC support in the hypervisor.

4.7 has:

cpufeatureset.h:140:XEN_CPUFEATURE(X2APIC,        1*32+21) /*!A Extended
xAPIC */

so will expose it to PV guests.

4.6 has this gem I'd forgotten, in pv_cpuid():

        if ( !cpu_has_apic )
           __clear_bit(X86_FEATURE_X2APIC % 32, &c);

but I don't see any (sensible) logic to hide x2APIC from the view of a
PV guest.

So PV guests really have see x2APIC from the host forever (and
irrespective of Xen's x2APIC support).

~Andrew


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 19:26:52 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 19:26:52 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687405.1070822 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfm3L-00060j-D2; Thu, 29 Feb 2024 19:26:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687405.1070822; Thu, 29 Feb 2024 19:26:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfm3L-00060c-9z; Thu, 29 Feb 2024 19:26:47 +0000
Received: by outflank-mailman (input) for mailman id 687405;
 Thu, 29 Feb 2024 19:26:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfm3K-00060S-9x; Thu, 29 Feb 2024 19:26:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfm3K-0004MD-8B; Thu, 29 Feb 2024 19:26:46 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfm3J-00016R-U5; Thu, 29 Feb 2024 19:26:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfm3J-0005ge-Tb; Thu, 29 Feb 2024 19:26:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=hLyDFIh6cKiH2iBxPHRsg5o/wa6BjmTFuVGyJrNZI14=; b=uNHZ+g2b053ylZE+Z3SnlEgAAs
	sg0Dbf4xOR9A1BhrcFdAag72RAuTFRV8A2qYgMRs0E3AwqdvFs8CUS9ichmuJZiTP8bQ90QOyG9wE
	HoVh+HVLHdAiBdiq2fBwIaHYt1NzNHI4JvkXQIdIc3UDObj1EalVzXwADWwI2gMn6ZMU=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184823-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [xen-unstable-smoke test] 184823: tolerable all pass - PUSHED
X-Osstest-Failures:
    xen-unstable-smoke:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    xen-unstable-smoke:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
X-Osstest-Versions-This:
    xen=635dd1120a01961a39dce6ad3f09692681379378
X-Osstest-Versions-That:
    xen=c20850540ad6a32f4fc17bde9b01c92b0df18bf0
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 29 Feb 2024 19:26:45 +0000

flight 184823 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184823/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  635dd1120a01961a39dce6ad3f09692681379378
baseline version:
 xen                  c20850540ad6a32f4fc17bde9b01c92b0df18bf0

Last test of basis   184817  2024-02-29 08:00:28 Z    0 days
Testing same since   184823  2024-02-29 17:00:27 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Juergen Gross <jgross@suse.com>
  Oleksii Kurochko <oleksii.kurochko@gmail.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Samuel Thibault <samuel.thibault@ens-lyon.org>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   c20850540a..635dd1120a  635dd1120a01961a39dce6ad3f09692681379378 -> smoke


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 20:54:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 20:54:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687411.1070832 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfnPl-00086Q-Kj; Thu, 29 Feb 2024 20:54:01 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687411.1070832; Thu, 29 Feb 2024 20:54:01 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfnPl-00086J-I9; Thu, 29 Feb 2024 20:54:01 +0000
Received: by outflank-mailman (input) for mailman id 687411;
 Thu, 29 Feb 2024 20:54:00 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfnPk-00086A-6X
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 20:54:00 +0000
Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com
 [2a00:1450:4864:20::52f])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id a8fb1c35-d744-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 21:53:59 +0100 (CET)
Received: by mail-ed1-x52f.google.com with SMTP id
 4fb4d7f45d1cf-55a179f5fa1so2328647a12.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 12:53:59 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 q12-20020a056402518c00b00563ec73bbafsm927632edd.46.2024.02.29.12.53.55
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 12:53:55 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: a8fb1c35-d744-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709240037; x=1709844837; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=OzsVoEciUKquuzr3WMreLrvOgFe/hTg/KGXFrUh70oQ=;
        b=iCLbPLY17sDP0n4KHjca7MHq7CuEmrqXhqykpM2wTItePWVcumyy+qNVo3qRSguv76
         JnhtW5qgLv2cO3dJ8oYblBIX3inK5T6Rtn53A4mOd7CqUPvE5A5ZFF/fwLv/9rsgVt6F
         +TqWnp6PcWWWL8npPbSp+0asFqusu/1Rz7rm8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709240037; x=1709844837;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=OzsVoEciUKquuzr3WMreLrvOgFe/hTg/KGXFrUh70oQ=;
        b=Z8ffKKuOuRfwAgI5wu1ukqSsgQYwWC0DWCapIT2Z5fjYfQBJw5PLaHJ0NS65LvjkFC
         lluhqWOKG406E/ROgokE18+diJ8tStYwcpBAe1Jz73C9GAAvtiI+Oy/rSCTuCPsbjav6
         fZJo9TRHq546Hql+5FrFiXg0AM947KfKLS3z3fRR/VRgEduMfgoGZ3TEbDiej7mIGsnH
         btjqYgw8iFVossA3vssatJahLSbQFUuavWjtGwF5fhxkyRMxl287iCHAli7wLJI8pXT4
         XaD/BzC26UeeQhIWAH1xh1/kVnzmBu6mFX4Y1Aezdrv2n/2A3rAdU/ZAR54U+ImgYqPm
         Mh8w==
X-Gm-Message-State: AOJu0YyXDn85xwA8wNU08sUaCjXzyyqyiZ/LElhjqoOcxfF4Kq2bWWh5
	70QF6Af+0+D+XuW9Tbrp6cSq2Xp/NZk7o++r9eq3OhGhJyE15fBk7AkKALkQV7msLNUgXxWogI4
	l
X-Google-Smtp-Source: AGHT+IHABz3esixvShfI7MGbBO4iHpkL6yqv/TqMTE8Lr3+pBY+8klK0r+fLUQhwBm1eg+5Jl1u9Zg==
X-Received: by 2002:a05:6402:5203:b0:566:a4ca:2a3b with SMTP id s3-20020a056402520300b00566a4ca2a3bmr76307edd.15.1709240037267;
        Thu, 29 Feb 2024 12:53:57 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] tests/resource: Fix HVM guest in !SHADOW builds
Date: Thu, 29 Feb 2024 20:53:54 +0000
Message-Id: <20240229205354.2574207-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Right now, test-resource always creates HVM Shadow guests.  But if Xen has
SHADOW compiled out, running the test yields:

  $./test-resource
  XENMEM_acquire_resource tests
  Test x86 PV
    Created d1
    Test grant table
  Test x86 PVH
    Skip: 95 - Operation not supported

and doesn't really test HVM guests, but doesn't fail either.

There's nothing paging-mode-specific about this test, so default to HAP if
possible and provide a more specific message if neither HAP or Shadow are
available.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 tools/tests/resource/test-resource.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/tools/tests/resource/test-resource.c b/tools/tests/resource/test-resource.c
index 7ae88ea34807..2796053588d3 100644
--- a/tools/tests/resource/test-resource.c
+++ b/tools/tests/resource/test-resource.c
@@ -20,6 +20,8 @@ static xc_interface *xch;
 static xenforeignmemory_handle *fh;
 static xengnttab_handle *gh;
 
+static xc_physinfo_t physinfo;
+
 static void test_gnttab(uint32_t domid, unsigned int nr_frames,
                         unsigned long gfn)
 {
@@ -172,6 +174,23 @@ static void test_domain_configurations(void)
 
         printf("Test %s\n", t->name);
 
+#if defined(__x86_64__) || defined(__i386__)
+        /*
+         * On x86, use HAP guests if possible, but skip if neither HAP nor
+         * SHADOW is available.
+         */
+        if ( t->create.flags & XEN_DOMCTL_CDF_hvm )
+        {
+            if ( physinfo.capabilities & XEN_SYSCTL_PHYSCAP_hap )
+                t->create.flags |= XEN_DOMCTL_CDF_hap;
+            else if ( !(physinfo.capabilities & XEN_SYSCTL_PHYSCAP_shadow) )
+            {
+                printf("  Skip: Neither HAP or SHADOW available\n");
+                continue;
+            }
+        }
+#endif
+
         rc = xc_domain_create(xch, &domid, &t->create);
         if ( rc )
         {
@@ -214,6 +233,8 @@ static void test_domain_configurations(void)
 
 int main(int argc, char **argv)
 {
+    int rc;
+
     printf("XENMEM_acquire_resource tests\n");
 
     xch = xc_interface_open(NULL, NULL, 0);
@@ -227,6 +248,10 @@ int main(int argc, char **argv)
     if ( !gh )
         err(1, "xengnttab_open");
 
+    rc = xc_physinfo(xch, &physinfo);
+    if ( rc )
+        err(1, "Failed to obtain physinfo");
+
     test_domain_configurations();
 
     return !!nr_failures;

base-commit: 635dd1120a01961a39dce6ad3f09692681379378
prerequisite-patch-id: 1fddefae616fa9a57ddc85bbf770e3c5bdb47b1f
prerequisite-patch-id: 50c4a201d8fa10c1797a1b17f3f2729b5787da84
prerequisite-patch-id: 4b4799fae62b5f41b9b0d2078e8b081605341a0a
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 20:58:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 20:58:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687414.1070841 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfnUN-0000Ek-4J; Thu, 29 Feb 2024 20:58:47 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687414.1070841; Thu, 29 Feb 2024 20:58:47 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfnUN-0000Ed-1l; Thu, 29 Feb 2024 20:58:47 +0000
Received: by outflank-mailman (input) for mailman id 687414;
 Thu, 29 Feb 2024 20:58:46 +0000
Received: from mail.xenproject.org ([104.130.215.37])
 by lists.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfnUM-0000ET-0a; Thu, 29 Feb 2024 20:58:46 +0000
Received: from host146.205.237.98.conversent.net ([205.237.98.146]
 helo=infra.test-lab.xenproject.org)
 by mail.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfnUL-00063J-SB; Thu, 29 Feb 2024 20:58:45 +0000
Received: from [172.16.148.1] (helo=osstest.test-lab.xenproject.org)
 by infra.test-lab.xenproject.org with esmtp (Exim 4.92)
 (envelope-from <osstest-admin@xenproject.org>)
 id 1rfnUL-0004CX-GP; Thu, 29 Feb 2024 20:58:45 +0000
Received: from osstest by osstest.test-lab.xenproject.org with local (Exim
 4.92) (envelope-from <osstest-admin@xenproject.org>)
 id 1rfnUL-0002Tz-Fj; Thu, 29 Feb 2024 20:58:45 +0000
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
	d=xenproject.org; s=20200302mail; h=Date:From:Subject:MIME-Version:
	Content-Transfer-Encoding:Content-Type:Message-ID:To;
	bh=Hb9z7J2jcq8FzY+ADR3/ASpsTCvxgTYhcc4rb7RBwBs=; b=wtkPiEXcCewX/V4wFYbw6ab0Sb
	mZwgvGSMrOQYJiOxL+x6AhIPpyKUUvobLtDLzGevDRBA5sU/8TUjJpT0qAqC7ejFePFur3pYQqYA3
	0cPU7v1G192KKotQmlHN/Kj/tiuJ3GV6ixwkf3pfw8URXlFy8Xa2RGeNxHutxsG1g9Fk=;
To: xen-devel@lists.xenproject.org
Message-ID: <osstest-184816-mainreport@xen.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
Subject: [linux-linus test] 184816: tolerable FAIL - PUSHED
X-Osstest-Failures:
    linux-linus:test-armhf-armhf-libvirt:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-win7-amd64:guest-stop:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemuu-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-xl-qemut-ws16-amd64:guest-stop:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-qemuu-nested-amd:debian-hvm-install/l1/l2:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-thunderx:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit2:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-xsm:saverestore-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-qcow2:migrate-support-check:fail:nonblocking
    linux-linus:test-amd64-amd64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-libvirt-raw:saverestore-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-arm64-arm64-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-vhd:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-raw:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-arndale:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-multivcpu:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-rtds:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-xl-credit1:saverestore-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt:migrate-support-check:fail:nonblocking
    linux-linus:test-armhf-armhf-libvirt-qcow2:migrate-support-check:fail:nonblocking
X-Osstest-Versions-This:
    linux=805d849d7c3cc1f38efefd48b2480d62b7b5dcb7
X-Osstest-Versions-That:
    linux=cf1182944c7cc9f1c21a8a44e0d29abe12527412
From: osstest service owner <osstest-admin@xenproject.org>
Date: Thu, 29 Feb 2024 20:58:45 +0000

flight 184816 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/184816/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt   16 saverestore-support-check fail blocked in 184802
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stop            fail like 184802
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stop            fail like 184802
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stop            fail like 184802
 test-armhf-armhf-libvirt-raw 15 saverestore-support-check    fail  like 184802
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stop            fail like 184802
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 184802
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 184802
 test-amd64-amd64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl          16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check fail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-xsm      15 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-xsm      16 saverestore-support-check    fail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-check        fail never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-check        fail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-check    fail   never pass
 test-arm64-arm64-xl-vhd      14 migrate-support-check        fail   never pass
 test-arm64-arm64-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-vhd      14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-vhd      15 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-check        fail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-check    fail  never pass
 test-armhf-armhf-xl-rtds     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-rtds     16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-check    fail   never pass
 test-armhf-armhf-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-check        fail never pass

version targeted for testing:
 linux                805d849d7c3cc1f38efefd48b2480d62b7b5dcb7
baseline version:
 linux                cf1182944c7cc9f1c21a8a44e0d29abe12527412

Last test of basis   184802  2024-02-28 01:14:44 Z    1 days
Testing same since   184816  2024-02-29 05:22:18 Z    0 days    1 attempts

------------------------------------------------------------
People who touched revisions under test:
  Ard Biesheuvel <ardb@kernel.org>
  Doug Smythies <dsmythies@telus.net>
  Herbert Xu <herbert@gondor.apana.org.au>
  Linus Torvalds <torvalds@linux-foundation.org>
  Lukas Bulwahn <lukas.bulwahn@gmail.com>
  Mark Brown <broonie@kernel.org>
  Naresh Solanki <naresh.solanki@9elements.com>
  Rafael J. Wysocki <rafael.j.wysocki@intel.com>
  Théo Lebrun <theo.lebrun@bootlin.com>

jobs:
 build-amd64-xsm                                              pass    
 build-arm64-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-arm64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-arm64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-arm64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-coresched-amd64-xl                                pass    
 test-arm64-arm64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm           pass    
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm        pass    
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm                 pass    
 test-amd64-amd64-libvirt-xsm                                 pass    
 test-arm64-arm64-libvirt-xsm                                 pass    
 test-amd64-amd64-xl-xsm                                      pass    
 test-arm64-arm64-xl-xsm                                      pass    
 test-amd64-amd64-qemuu-nested-amd                            fail    
 test-amd64-amd64-xl-pvhv2-amd                                pass    
 test-amd64-amd64-dom0pvh-xl-amd                              pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-freebsd11-amd64                             pass    
 test-amd64-amd64-freebsd12-amd64                             pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-qemut-ws16-amd64                         fail    
 test-amd64-amd64-xl-qemuu-ws16-amd64                         fail    
 test-armhf-armhf-xl-arndale                                  pass    
 test-amd64-amd64-examine-bios                                pass    
 test-amd64-amd64-xl-credit1                                  pass    
 test-arm64-arm64-xl-credit1                                  pass    
 test-armhf-armhf-xl-credit1                                  pass    
 test-amd64-amd64-xl-credit2                                  pass    
 test-arm64-arm64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  pass    
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict        pass    
 test-amd64-amd64-examine                                     pass    
 test-arm64-arm64-examine                                     pass    
 test-armhf-armhf-examine                                     pass    
 test-amd64-amd64-qemuu-nested-intel                          pass    
 test-amd64-amd64-xl-pvhv2-intel                              pass    
 test-amd64-amd64-dom0pvh-xl-intel                            pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     pass    
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-amd64-libvirt-pair                                pass    
 test-amd64-amd64-xl-pvshim                                   pass    
 test-amd64-amd64-pygrub                                      pass    
 test-amd64-amd64-libvirt-qcow2                               pass    
 test-armhf-armhf-libvirt-qcow2                               pass    
 test-amd64-amd64-libvirt-raw                                 pass    
 test-arm64-arm64-libvirt-raw                                 pass    
 test-armhf-armhf-libvirt-raw                                 pass    
 test-amd64-amd64-xl-rtds                                     pass    
 test-armhf-armhf-xl-rtds                                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow             pass    
 test-amd64-amd64-xl-shadow                                   pass    
 test-arm64-arm64-xl-thunderx                                 pass    
 test-amd64-amd64-examine-uefi                                pass    
 test-amd64-amd64-xl-vhd                                      pass    
 test-arm64-arm64-xl-vhd                                      pass    
 test-armhf-armhf-xl-vhd                                      pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

hint: The 'hooks/update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-receive' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
hint: The 'hooks/post-update' hook was ignored because it's not set as executable.
hint: You can disable this warning with `git config advice.ignoredHook false`.
To xenbits.xen.org:/home/xen/git/linux-pvops.git
   cf1182944c7cc..805d849d7c3cc  805d849d7c3cc1f38efefd48b2480d62b7b5dcb7 -> tested/linux-linus


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 21:07:55 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 21:07:55 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687420.1070851 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfnd7-0001uj-0G; Thu, 29 Feb 2024 21:07:49 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687420.1070851; Thu, 29 Feb 2024 21:07:48 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfnd6-0001uc-Tf; Thu, 29 Feb 2024 21:07:48 +0000
Received: by outflank-mailman (input) for mailman id 687420;
 Thu, 29 Feb 2024 21:07:47 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfnd5-0001uW-NJ
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 21:07:47 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 958bb036-d746-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 22:07:46 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 9EDA6613F7;
 Thu, 29 Feb 2024 21:07:44 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 49002C433F1;
 Thu, 29 Feb 2024 21:07:43 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 958bb036-d746-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709240864;
	bh=ox19t7QmUxyU/JAUL6B4sxCskmAr0rEXAveXqSTUgZI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=id0CLQNPaRPC9x14fxKtOvRW7yC43G/HztQPC9L7bGPHgFNoPHuROHRpBin3Y66dP
	 3qm3NFlxhiRGXsvebpIcQGRzUjNKvkB4lZt1F6ZzUV3a1IYeFzei8X+1M4byd4Fb1d
	 6vTTRv+98G2ZSP3bYRVKSf+0BpieblSwYYcyp58qPIoiBjGxd18ytIZgz9EQ0e0QU2
	 Jc5diSzgAUZ5gBaAPPInVpC+xDjdSMgh1rVNYHLaerKfHNidVHNFDTfhGBHC7+8X0c
	 NmZMOK1VA6uHtaOiydmdgEjnKJMsdlncVJaiNDeXo7rFi9G/ehULRxltTgkWdTSb5P
	 MvNxKVG2xfo9g==
Date: Thu, 29 Feb 2024 13:07:41 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Michal Orzel <michal.orzel@amd.com>
cc: Julien Grall <julien@xen.org>, xen-devel@lists.xenproject.org, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Jan Beulich <jbeulich@suse.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>
Subject: Re: [PATCH] SUPPORT.md: clarify support of booting 32-bit Xen on
 ARMv8
In-Reply-To: <216e9119-a7f9-43f3-80f9-f1c15685cad3@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402291307300.853156@ubuntu-linux-20-04-desktop>
References: <20240229121352.11264-1-michal.orzel@amd.com> <d1b7a4fd-0754-45f7-8c58-db36f0e94cfc@xen.org> <94176748-8438-460d-9434-6d18add55cef@amd.com> <544718e1-4855-4af1-a450-6362a8cd08f0@xen.org> <216e9119-a7f9-43f3-80f9-f1c15685cad3@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 29 Feb 2024, Michal Orzel wrote:
> 
> On 29/02/2024 13:40, Julien Grall wrote:
> > 
> > 
> > On 29/02/2024 12:37, Michal Orzel wrote:
> >> Hi Julien,
> > 
> > Hi Michal,
> > 
> >>
> >> On 29/02/2024 13:35, Julien Grall wrote:
> >>> On 29/02/2024 12:13, Michal Orzel wrote:
> >>>> Since commit bd1001db0af1 ("xen/arm: arm32: Allow Xen to boot on
> >>>> unidentified CPUs"), it's been possible to boot 32-bit Xen on ARMv8A CPUs
> >>>> in AArch32 state (assuming HW supports EL2 execution in AArch32). Clarify
> >>>> the support statement and mark it as Tech Preview, as this use case is
> >>>> uncommon and hasn't really been tested/hardened.
> >>>>
> >>>> Signed-off-by: Michal Orzel <michal.orzel@amd.com>
> >>>> ---
> >>>>    SUPPORT.md | 1 +
> >>>>    1 file changed, 1 insertion(+)
> >>>>
> >>>> diff --git a/SUPPORT.md b/SUPPORT.md
> >>>> index a90d1108c9d9..acc61230bb5e 100644
> >>>> --- a/SUPPORT.md
> >>>> +++ b/SUPPORT.md
> >>>> @@ -40,6 +40,7 @@ supported in this document.
> >>>>        Status: Supported
> >>> I would consider to use 'Status, Xen in aarch64 mode: Supported' and then...
> >>>
> >>>>        Status, Cortex A57 r0p0-r1p1: Supported, not security supported
> >>>>        Status, Cortex A77 r0p0-r1p0: Supported, not security supported
> >>>> +    Status, Xen in AArch32 mode: Tech Preview
> >>>
> >>> ... move this line closer. What do you think?
> >> That works for me too (+AArch64 instead of aarch64).
> > 
> > Ah yes. I keep forgetting capitalizing properly :).
> > 
> >> Shall I respin the patch?
> > 
> > Up to you. I am happy to fix it. But I will wait a day or two just to
> > give a chance for the others to comment.
> That works for me, let's wait for Bertrand and Stefano.

I am fine with it


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 22:15:13 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 22:15:13 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687426.1070862 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfog5-0002D1-QM; Thu, 29 Feb 2024 22:14:57 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687426.1070862; Thu, 29 Feb 2024 22:14:57 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfog5-0002Cu-Ng; Thu, 29 Feb 2024 22:14:57 +0000
Received: by outflank-mailman (input) for mailman id 687426;
 Thu, 29 Feb 2024 22:14:56 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=kPQN=KG=cloud.com=andrew.cooper@srs-se1.protection.inumbo.net>)
 id 1rfog4-0002Co-J2
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 22:14:56 +0000
Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com
 [2a00:1450:4864:20::52c])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id f71cc6c7-d74f-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 23:14:54 +0100 (CET)
Received: by mail-ed1-x52c.google.com with SMTP id
 4fb4d7f45d1cf-56648955ac5so2041538a12.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 14:14:54 -0800 (PST)
Received: from andrewcoop.citrite.net (default-46-102-197-194.interdsl.co.uk.
 [46.102.197.194]) by smtp.gmail.com with ESMTPSA id
 vo14-20020a170907a80e00b00a44223c2c87sm1077619ejc.30.2024.02.29.14.14.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 29 Feb 2024 14:14:50 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: f71cc6c7-d74f-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=citrix.com; s=google; t=1709244893; x=1709849693; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=9YWh8kcKPGDNRrSVVwkO2zWFYbs9veRI7tHYSK9K2FI=;
        b=g/AL2txnnoRcjGy8u3zigktjNZ9JRM7moC27YdX9pgW9uR3mmz2q3d0PLDgMgoc5sl
         pSqfKxSdwAw7xvz9GLxELwN2dioqUYrhe2b0uJzN2kOnZ35XLuw9HtFbk5IfY045D3nZ
         usK2hWuPOHqgLuw7EwE4XrJ6f2O1lr9JybJZg=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709244893; x=1709849693;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=9YWh8kcKPGDNRrSVVwkO2zWFYbs9veRI7tHYSK9K2FI=;
        b=ANeDZ+tuOhM9YRl3I8yi1VmQ3iS6URnGVR3rz0emuIMKCQawy90IBbczj+J64dxU1g
         UjLp+PZYF0U2cRbmy08pNp+sscgTdhFYbCj8VvKTWvQh2mevHOdRVroGolHesHGpTdZG
         2Itpvv6AKtJ128Hjv93zoRYZflEogpzdUkDBi327GPlwws68N/zMmhazwsSa6ZlZO6Jb
         WNykMV+0FZO9//teVjVb5o0M+ACKZ92znSaTNBsSjYPztF4pcy21zjjDdvHNmUNZOSXJ
         29apzg+lcBVsEMm6ELspOQxuWBrzqFWlNTHBKS8DA9Sh7bQJ3FDTqH8k7HBBRGuF8LIS
         vTMQ==
X-Gm-Message-State: AOJu0YxIAJErr9YM49SEEQcZ/P7ihUrrhlyoNjrP4zqSO73wrvMCSChn
	+vqgfsCuBxDWn5m/2PmtZGZUcC+Cw+sNIiT7/VY//CMTELam2l2RuNht7cVwRK3pWIL6SeMuZJu
	K
X-Google-Smtp-Source: AGHT+IHqGZ/jLWjVRR72h6d0eXydWfJ6YLNQO8K+lurvOfG/OEIFALUm0Va4hpBj9LIyTb9m5mQT2w==
X-Received: by 2002:a17:906:ca46:b0:a3f:2ef9:598a with SMTP id jx6-20020a170906ca4600b00a3f2ef9598amr5284ejb.36.1709244892820;
        Thu, 29 Feb 2024 14:14:52 -0800 (PST)
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	=?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= <roger.pau@citrix.com>,
	Wei Liu <wl@xen.org>
Subject: [PATCH] x86/cpu-policy: Hide x2APIC from PV guests
Date: Thu, 29 Feb 2024 22:14:48 +0000
Message-Id: <20240229221448.2593171-1-andrew.cooper3@citrix.com>
X-Mailer: git-send-email 2.30.2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

PV guests can't write to MSR_APIC_BASE (in order to set EXTD), nor can they
access any of the x2APIC MSR range.  Therefore they mustn't see the x2APIC
CPUID bit saying that they can.

Right now, the host x2APIC flag filters into PV guests, meaning that PV guests
generally see x2APIC except on Zen1-and-older AMD systems.

Linux works around this by explicitly hiding the bit itself, and filtering
EXTD out of MSR_APIC_BASE reads.  NetBSD behaves more in the spirit of PV
guests, and entirely ignores the APIC when built as a PV guest.

Change the annotation from !A to !S.  This has a consequence of stripping it
out of both PV featuremasks.  However, as existing guests may have seen the
bit, set it back into the PV Max policy; a VM which saw the bit and is alive
enough to migrate will have ignored it one way or another.

Hiding x2APIC does also change the contents of leaf 0xb, but as the
information is nonsense to begin with, this is likely an improvement on the
status quo.  The blind reporting of APIC_ID = vCPU_ID * 2 isn't interlinked
with the host's topology structure, and the APIC_IDs are useless without an
MADT to start with.  Dom0 is the only PV VM to get an MADT but it's the host
one, meaning the two sets of APIC_IDs are from different address spaces.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

v2:
 * Set x2APIC in PV max after applying the featuremask.
 * Drop the hunk for the default policy as it's handled by the A->S transform.
 * Rewrite the commit message.
---
 xen/arch/x86/cpu-policy.c                   | 11 +++++++++--
 xen/include/public/arch-x86/cpufeatureset.h |  2 +-
 2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index 15b49048fd55..c9b32bc17849 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -561,6 +561,14 @@ static void __init calculate_pv_max_policy(void)
     for ( i = 0; i < ARRAY_SIZE(fs); ++i )
         fs[i] &= pv_max_featuremask[i];
 
+    /*
+     * Xen at the time of writing (Feb 2024, 4.19 dev cycle) used to leak the
+     * host x2APIC capability into PV guests, but never supported the guest
+     * trying to turn x2APIC mode on.  Tolerate an incoming VM which saw the
+     * x2APIC CPUID bit and is alive enough to migrate.
+     */
+    __set_bit(X86_FEATURE_X2APIC, fs);
+
     /*
      * If Xen isn't virtualising MSR_SPEC_CTRL for PV guests (functional
      * availability, or admin choice), hide the feature.
@@ -854,11 +862,10 @@ void recalculate_cpuid_policy(struct domain *d)
     }
 
     /*
-     * Allow the toolstack to set HTT, X2APIC and CMP_LEGACY.  These bits
+     * Allow the toolstack to set HTT and CMP_LEGACY.  These bits
      * affect how to interpret topology information in other cpuid leaves.
      */
     __set_bit(X86_FEATURE_HTT, max_fs);
-    __set_bit(X86_FEATURE_X2APIC, max_fs);
     __set_bit(X86_FEATURE_CMP_LEGACY, max_fs);
 
     /*
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index 7e184ce0e3f4..0374cec3a2af 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -123,7 +123,7 @@ XEN_CPUFEATURE(PCID,          1*32+17) /*H  Process Context ID */
 XEN_CPUFEATURE(DCA,           1*32+18) /*   Direct Cache Access */
 XEN_CPUFEATURE(SSE4_1,        1*32+19) /*A  Streaming SIMD Extensions 4.1 */
 XEN_CPUFEATURE(SSE4_2,        1*32+20) /*A  Streaming SIMD Extensions 4.2 */
-XEN_CPUFEATURE(X2APIC,        1*32+21) /*!A Extended xAPIC */
+XEN_CPUFEATURE(X2APIC,        1*32+21) /*!S Extended xAPIC */
 XEN_CPUFEATURE(MOVBE,         1*32+22) /*A  movbe instruction */
 XEN_CPUFEATURE(POPCNT,        1*32+23) /*A  POPCNT instruction */
 XEN_CPUFEATURE(TSC_DEADLINE,  1*32+24) /*S  TSC Deadline Timer */
-- 
2.30.2



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 22:49:26 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 22:49:26 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687431.1070872 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpDK-0006FD-C8; Thu, 29 Feb 2024 22:49:18 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687431.1070872; Thu, 29 Feb 2024 22:49:18 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpDK-0006F6-9K; Thu, 29 Feb 2024 22:49:18 +0000
Received: by outflank-mailman (input) for mailman id 687431;
 Thu, 29 Feb 2024 22:49:16 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfpDI-0006F0-JL
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 22:49:16 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id c27e56bb-d754-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 23:49:14 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 04ECE61197;
 Thu, 29 Feb 2024 22:49:13 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id F236DC433C7;
 Thu, 29 Feb 2024 22:49:10 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: c27e56bb-d754-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709246952;
	bh=dvqARpEbAbkdXCQ+z4XZTxRY0PLDuQrcwcZwXk0/jhM=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=XxVzseJjjzuXv8iupFUshoUHyTTfY7IPaE+xFO8zfndhKAq3GHSRsfpyjeLOs66OJ
	 WnPmK9MEuNZVRlf2rawyVat9oMMYiE92bwGqZdldj5/aeOaeDarjTiISpDKhWNoVy3
	 p1JUL55rPfDxHynnDMB9LMjUpSP7wAXxmSJu+wgbAefzAXONEXjP+I2fztwA/TNFK+
	 px/GkP3gpiVer8hBroU2GHWL7R2Nh2V55rH74UkkjkxtInVKzcveb8/WIkuwfkwJLC
	 UOeG+L7w3UM7pW08SZyfVBfufLL6jrSVJN0W1R9cFrNOkx+URD/95fEdoxhgm2MUL2
	 LKVhGFcQcewWg==
Date: Thu, 29 Feb 2024 14:49:09 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: Jan Beulich <jbeulich@suse.com>, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, andrew.cooper3@citrix.com, roger.pau@citrix.com, 
    bertrand.marquis@arm.com, julien@xen.org, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>, 
    xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH 04/10] xen/public: address violations of MISRA C Rule
 20.7
In-Reply-To: <82940f688e5a5eee274fa579991ebd15@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402291447590.853156@ubuntu-linux-20-04-desktop>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com> <0cdc4dc2fcad699a2274277b32de3ee0207d5a2d.1709219010.git.nicola.vetrini@bugseng.com> <bd95193c-522a-4c74-98e0-68fa088a5b49@suse.com> <82940f688e5a5eee274fa579991ebd15@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 29 Feb 2024, Nicola Vetrini wrote:
> On 2024-02-29 17:40, Jan Beulich wrote:
> > On 29.02.2024 16:27, Nicola Vetrini wrote:
> > > --- a/xen/include/public/xen.h
> > > +++ b/xen/include/public/xen.h
> > > @@ -988,7 +988,7 @@ typedef struct {
> > >        ((b) >>  8) & 0xFF, ((b) >>  0) & 0xFF,                           \
> > >        ((c) >>  8) & 0xFF, ((c) >>  0) & 0xFF,                           \
> > >        ((d) >>  8) & 0xFF, ((d) >>  0) & 0xFF,                           \
> > > -                e1, e2, e3, e4, e5, e6}}
> > > +                (e1), (e2), (e3), (e4), (e5), (e6)}}
> > 
> > Why? Wasn't it agreed already that long macro arguments passed on
> > (no matter whether to a function, a macro, or like used here) don't
> > need parenthesizing?
> > 
> 
> That applies to all outermost macro invocations, but not to the innermost one.

I don't understand what you mean. Maybe a couple of trivial examples
would help.


> If you want also aggregate initalizers to be deviated, that could be done
> (provided that the macro arg is not included in some expression, such as
> "{..., e1 + 1, ...}"

My gut feeling tells me that probably this is what we want but I'd
rather first understand exactly what you meant above


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 22:53:14 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 22:53:14 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687436.1070882 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpH6-0007dm-RR; Thu, 29 Feb 2024 22:53:12 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687436.1070882; Thu, 29 Feb 2024 22:53:12 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpH6-0007df-ON; Thu, 29 Feb 2024 22:53:12 +0000
Received: by outflank-mailman (input) for mailman id 687436;
 Thu, 29 Feb 2024 22:53:11 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfpH5-0007dZ-Ha
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 22:53:11 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4e8cebce-d755-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 23:53:09 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 28C6161197;
 Thu, 29 Feb 2024 22:53:08 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20759C433C7;
 Thu, 29 Feb 2024 22:53:06 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e8cebce-d755-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709247187;
	bh=NVZDVOMLAvwc6g/HzbqmjqBYrtUoqn5EaPnD89cGTr0=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=rj37OqQNLti3RD2T/wtUKfFq9C6jUZ17fp9mDmSplXm8OKsFVlF9iJkkQXce5Z+ya
	 EXnW8Gmrh7CjVeRFXBOovYhsjJCEJQb1y65N822DFT7nVVVbIVO5s5wju4rJVtN65d
	 kMbaVF7ztQOSDHwKOk4bo8vPwigi+7O7ceW1yNAKu7TAslJKf09Dfk/riJP1uKh1h6
	 iViMlT5yUZvd3uXanuVy+y6lXPYxcOXWum5jhvKRbDXWreScJIDd0ij2uqptHsxkQo
	 F9U6GYYh85RKIfqx+H+2w9NHYGlfZZIq1dW6OeJzOu+a9JB4FRoBFaYerFKGNX/9p8
	 UeoNRQlvkaXiw==
Date: Thu, 29 Feb 2024 14:53:04 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, bertrand.marquis@arm.com, julien@xen.org, 
    Rahul Singh <rahul.singh@arm.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [XEN PATCH 06/10] arm/smmu: address some violations of MISRA C
 Rule 20.7
In-Reply-To: <8bbf294d832b973648e84e5d837b1878a84f7645.1709219010.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402291451390.853156@ubuntu-linux-20-04-desktop>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com> <8bbf294d832b973648e84e5d837b1878a84f7645.1709219010.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 29 Feb 2024, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> ---
>  xen/drivers/passthrough/arm/smmu.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
> index 625ed0e41961..83196057a937 100644
> --- a/xen/drivers/passthrough/arm/smmu.c
> +++ b/xen/drivers/passthrough/arm/smmu.c
> @@ -242,7 +242,7 @@ struct arm_smmu_xen_device {
>  	struct iommu_group *group;
>  };
>  
> -#define dev_archdata(dev) ((struct arm_smmu_xen_device *)dev->iommu)
> +#define dev_archdata(dev) ((struct arm_smmu_xen_device *)(dev)->iommu)
>  #define dev_iommu_domain(dev) (dev_archdata(dev)->domain)
>  #define dev_iommu_group(dev) (dev_archdata(dev)->group)

this is OK


> @@ -627,7 +627,7 @@ struct arm_smmu_master_cfg {
>  };
>  #define INVALID_SMENDX			-1
>  #define for_each_cfg_sme(cfg, i, idx, num) \
> -	for (i = 0; idx = cfg->smendx[i], i < num; ++i)
> +	for (i = 0; idx = (cfg)->smendx[i], (i) < (num); ++(i))

The first i = 0 is missing parentheses?
Also idx misses parentheses?


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 22:54:33 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 22:54:33 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687438.1070892 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpIL-0008BS-5p; Thu, 29 Feb 2024 22:54:29 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687438.1070892; Thu, 29 Feb 2024 22:54:29 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpIL-0008BL-1W; Thu, 29 Feb 2024 22:54:29 +0000
Received: by outflank-mailman (input) for mailman id 687438;
 Thu, 29 Feb 2024 22:54:28 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfpIK-0008BF-Lv
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 22:54:28 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 7c351b00-d755-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 23:54:26 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 39330CE0C50;
 Thu, 29 Feb 2024 22:54:24 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id C074CC433C7;
 Thu, 29 Feb 2024 22:54:21 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 7c351b00-d755-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709247263;
	bh=VkgVlGf55lYhsVQXFsP5IwERQXeaYSut5Ug7KUjjAe8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=T5y2IF386+sr4RcSUt2F3wmvKv2XuytRJM90r+RsksDFJ/maIR7XvUIa2It3piPOm
	 eLNa1UcZYRbpzWreIb+6i2N/XCGSWtxywESglmXz4nBQoWz49enJtRacuI57XgxOLp
	 r0KWBRj8O5VisOu9SGuQ7cW3s1QQPo6GlJvl0L2n7xmRr6BCVw9qLZCHpIjLopW8hd
	 NhlQV4CzjF5/JOG7luB2gnSWdUhvEMDbY8uW3nCBnWou8Sqpqhh2xsmzlSLAC4JiWD
	 mUukxyhimaAX80Qd+CNcdShex+zKdez+j9xtDBAKy71GFHBPQEVXulzqOb8NS9240Q
	 kEdLvjuRUYS4A==
Date: Thu, 29 Feb 2024 14:54:20 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, bertrand.marquis@arm.com, julien@xen.org, 
    Rahul Singh <rahul.singh@arm.com>, 
    Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [XEN PATCH 07/10] xen/arm: smmuv3: address violations of MISRA
 C Rule 20.7
In-Reply-To: <a11cd3f9c3dc64d67df14d8b7527b7467d46f573.1709219010.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402291454110.853156@ubuntu-linux-20-04-desktop>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com> <a11cd3f9c3dc64d67df14d8b7527b7467d46f573.1709219010.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 29 Feb 2024, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/drivers/passthrough/arm/smmu-v3.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
> index c3ac6d17d1c8..b1c40c2c0ae7 100644
> --- a/xen/drivers/passthrough/arm/smmu-v3.c
> +++ b/xen/drivers/passthrough/arm/smmu-v3.c
> @@ -111,7 +111,7 @@
>  #define GFP_KERNEL		0
>  
>  /* Device logger functions */
> -#define dev_name(dev)	dt_node_full_name(dev->of_node)
> +#define dev_name(dev)	dt_node_full_name((dev)->of_node)
>  #define dev_dbg(dev, fmt, ...)			\
>  	printk(XENLOG_DEBUG "SMMUv3: %s: " fmt, dev_name(dev), ## __VA_ARGS__)
>  #define dev_notice(dev, fmt, ...)		\
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 22:55:56 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 22:55:56 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687440.1070902 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpJj-0000Hw-F5; Thu, 29 Feb 2024 22:55:55 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687440.1070902; Thu, 29 Feb 2024 22:55:55 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpJj-0000Ho-CQ; Thu, 29 Feb 2024 22:55:55 +0000
Received: by outflank-mailman (input) for mailman id 687440;
 Thu, 29 Feb 2024 22:55:53 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfpJh-0000He-UL
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 22:55:53 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id af5f1a66-d755-11ee-afd8-a90da7624cb6;
 Thu, 29 Feb 2024 23:55:52 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 0785DCE0C50;
 Thu, 29 Feb 2024 22:55:48 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87690C433F1;
 Thu, 29 Feb 2024 22:55:45 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: af5f1a66-d755-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709247347;
	bh=SDdNPuWlgRxSvWMm8KMSuux7iGtwMQFrncK2pWaQwgE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=hgf7zu1AlZXD0IIG8Czjk84cF7xj2xHD+Um04UwfLdlVLkJIl1L4xZWjKMM2S0kqZ
	 NNK0cVwWWRGZToi6lpkVF8TdmxsoSEKM9C1/csR7pm6JcgCABEPJ2nfje45Ge0bRIy
	 uQMmmhjxI5nNgEhsUJyuggLf5o/hXVv2ejthY0Ceowk/q2SSj7MV/imSAmusbxjnio
	 CGK5z84s3nGguUMZRfd6yTtAbdGx4SMfuEqQpvzThhd+3SDRR8EvyvhyWWyLDOdJNj
	 MODd0lPlAuV3b2ILz9leOSdubkqIEZ/2X2xIhYksRGSF55bGBD3YJKmURcjnsS4mQT
	 lmciX5LtaiXqg==
Date: Thu, 29 Feb 2024 14:55:43 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, bertrand.marquis@arm.com, julien@xen.org, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH 08/10] xen/errno: address violations of MISRA C Rule
 20.7
In-Reply-To: <92dbcd9e5aa2ab45c50f3f567ddece59108568ad.1709219010.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402291454360.853156@ubuntu-linux-20-04-desktop>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com> <92dbcd9e5aa2ab45c50f3f567ddece59108568ad.1709219010.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 29 Feb 2024, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> ---
>  xen/include/xen/errno.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/include/xen/errno.h b/xen/include/xen/errno.h
> index 69b28dd3c6c5..506674701fae 100644
> --- a/xen/include/xen/errno.h
> +++ b/xen/include/xen/errno.h
> @@ -3,7 +3,7 @@
>  
>  #ifndef __ASSEMBLY__
>  
> -#define XEN_ERRNO(name, value) name = value,
> +#define XEN_ERRNO(name, value) name = (value),

I see this and the fact that "name" was not parenthesized and it would
deliberate right? So I guess the left side of an assignment doesn't need
parenthesis?


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 22:56:50 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 22:56:50 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687442.1070912 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpKc-0000nk-OK; Thu, 29 Feb 2024 22:56:50 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687442.1070912; Thu, 29 Feb 2024 22:56:50 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpKc-0000nd-Lg; Thu, 29 Feb 2024 22:56:50 +0000
Received: by outflank-mailman (input) for mailman id 687442;
 Thu, 29 Feb 2024 22:56:49 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfpKb-0000nV-LQ
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 22:56:49 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id d006947f-d755-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 23:56:47 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 0A2F1CE0C50;
 Thu, 29 Feb 2024 22:56:45 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87C92C433C7;
 Thu, 29 Feb 2024 22:56:42 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: d006947f-d755-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709247404;
	bh=WIv2SbTkcmK9Wn3JUKPksz+Ccji2UinT1E7Ev6UMVFI=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=QqzGJhRJnMgjanFj0J6Kit3RpaEAlWXcRZ0dYF2eSJvr3fXtgXjn2NJUhAkYDG9Km
	 ZsqTyGUpWVutpv2I56SEyWrVzIKdoe3pcW7iv1qcKmf29U+ye+aMXWrA/hLuG5fJt+
	 TdyC5nJeT2gcYttNH8DgpSR10hoVUMyPdY1zwBo3HmqZCkeSmkyL9h3FWKYzVzpbGr
	 147YRXFZ3UEx7tJ36aKSojCoSfKssMc+0qg1/DDtHE42BX1JDNfTj1pVh5di/wF9Yf
	 FP6DXuESRq1w5ajNd0s+DFXBsfw+2EASuudgtUgpupDg6kqknszDanKmkQ4m64zO+O
	 dpx3SuBWx506w==
Date: Thu, 29 Feb 2024 14:56:40 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, bertrand.marquis@arm.com, julien@xen.org, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH 09/10] xen/include: tasklet: address violations of
 MISRA C Rule 20.7
In-Reply-To: <952d13bdb830cb1d8d3282e49333323bc8755cd5.1709219010.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402291456150.853156@ubuntu-linux-20-04-desktop>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com> <952d13bdb830cb1d8d3282e49333323bc8755cd5.1709219010.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 29 Feb 2024, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
> ---
>  xen/include/xen/tasklet.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/include/xen/tasklet.h b/xen/include/xen/tasklet.h
> index 593d6a2400fb..78760b694a39 100644
> --- a/xen/include/xen/tasklet.h
> +++ b/xen/include/xen/tasklet.h
> @@ -27,7 +27,7 @@ struct tasklet
>  
>  #define _DECLARE_TASKLET(name, func, data, softirq)                     \
>      struct tasklet name = {                                             \
> -        LIST_HEAD_INIT(name.list), -1, softirq, 0, 0, func, data }
> +        LIST_HEAD_INIT((name).list), -1, softirq, 0, 0, func, data }
>  #define DECLARE_TASKLET(name, func, data)               \
>      _DECLARE_TASKLET(name, func, data, 0)
>  #define DECLARE_SOFTIRQ_TASKLET(name, func, data)       \

In reality this is not required due to "struct tasklet name", but for
uniformity:

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 22:57:17 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 22:57:17 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687445.1070922 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpL2-0001Jg-2v; Thu, 29 Feb 2024 22:57:16 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687445.1070922; Thu, 29 Feb 2024 22:57:16 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpL2-0001JZ-09; Thu, 29 Feb 2024 22:57:16 +0000
Received: by outflank-mailman (input) for mailman id 687445;
 Thu, 29 Feb 2024 22:57:14 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfpL0-0000nV-60
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 22:57:14 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id df7da3ad-d755-11ee-a1ee-f123f15fe8a2;
 Thu, 29 Feb 2024 23:57:12 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 33CF460B84;
 Thu, 29 Feb 2024 22:57:11 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id CA620C433C7;
 Thu, 29 Feb 2024 22:57:08 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: df7da3ad-d755-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709247430;
	bh=wVyKWsKWvr/bMiVJL4heuH8fpTxngbQ0/r+YQkB5hjg=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=JwLOQmWmIkgZI6YOlaCg2BgsQjv7v7C4XmLMJCSkrQ6gUcwDqe/GJKpWKjfro39SH
	 TXILjEVlEWyf/7VZWo+G8+lrtuJ6HsxGl90lnIhm8z7MDCJintAMdmJaIrSZLulJ2f
	 acHTg3j6nRoO0dC2MKrepeScAR95ZhqCl6FVblmpjgW4e5z5MZxbioR4k4gFl5sTK4
	 cE5J8z82r2ZXBnZb5nRSvE8a3Ivc9chc+uFvxe4s0f8JmS8ryljhqgSfL5gJQ+izK2
	 Ir03e7uKcsigsAfHrxK6X/clzZvUJDEUIYIKO8uZldk26NUdaxo5YFvBhEyyyvXvax
	 mLRUaC8FJk5qA==
Date: Thu, 29 Feb 2024 14:57:07 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Nicola Vetrini <nicola.vetrini@bugseng.com>
cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org, 
    michal.orzel@amd.com, xenia.ragiadakou@amd.com, ayan.kumar.halder@amd.com, 
    consulting@bugseng.com, jbeulich@suse.com, andrew.cooper3@citrix.com, 
    roger.pau@citrix.com, bertrand.marquis@arm.com, julien@xen.org, 
    George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: [XEN PATCH 10/10] xen/keyhandler: address violations of MISRA
 C Rule 20.7
In-Reply-To: <2bc4a964f0f2f47488e72237678e944dbdbd7bb7.1709219010.git.nicola.vetrini@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402291457000.853156@ubuntu-linux-20-04-desktop>
References: <cover.1709219010.git.nicola.vetrini@bugseng.com> <2bc4a964f0f2f47488e72237678e944dbdbd7bb7.1709219010.git.nicola.vetrini@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 29 Feb 2024, Nicola Vetrini wrote:
> MISRA C Rule 20.7 states: "Expressions resulting from the expansion
> of macro parameters shall be enclosed in parentheses". Therefore, some
> macro definitions should gain additional parentheses to ensure that all
> current and future users will be safe with respect to expansions that
> can possibly alter the semantics of the passed-in macro parameter.
> 
> No functional change.
> 
> Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  xen/common/keyhandler.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/common/keyhandler.c b/xen/common/keyhandler.c
> index 127ca506965c..4c1ce007870f 100644
> --- a/xen/common/keyhandler.c
> +++ b/xen/common/keyhandler.c
> @@ -42,10 +42,10 @@ static struct keyhandler {
>  } key_table[128] __read_mostly =
>  {
>  #define KEYHANDLER(k, f, desc, diag)            \
> -    [k] = { { .fn = (f) }, desc, 0, diag }
> +    [k] = { { .fn = (f) }, (desc), 0, (diag) }
>  
>  #define IRQ_KEYHANDLER(k, f, desc, diag)        \
> -    [k] = { { .irq_fn = (f) }, desc, 1, diag }
> +    [k] = { { .irq_fn = (f) }, (desc), 1, (diag) }
>  
>      IRQ_KEYHANDLER('A', do_toggle_alt_key, "toggle alternative key handling", 0),
>      IRQ_KEYHANDLER('d', dump_registers, "dump registers", 1),
> -- 
> 2.34.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 23:00:28 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 23:00:28 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687447.1070933 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpO3-0003P6-HN; Thu, 29 Feb 2024 23:00:23 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687447.1070933; Thu, 29 Feb 2024 23:00:23 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpO3-0003Oz-DK; Thu, 29 Feb 2024 23:00:23 +0000
Received: by outflank-mailman (input) for mailman id 687447;
 Thu, 29 Feb 2024 23:00:22 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfpO2-0003Ot-40
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 23:00:22 +0000
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 4e33d270-d756-11ee-a1ee-f123f15fe8a2;
 Fri, 01 Mar 2024 00:00:18 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by dfw.source.kernel.org (Postfix) with ESMTP id 1FB976134B;
 Thu, 29 Feb 2024 23:00:17 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8BDF1C43390;
 Thu, 29 Feb 2024 23:00:15 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 4e33d270-d756-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709247616;
	bh=Nr+2S5hkfscTDCR6fobtHuzA31U3TLZ01xIBOfUECyE=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=grgkGWyGhT8nGhX6dJ7Nrivwz0+0H4h7TeU1omZvb9tW8EiHPTroyJPwmBBLI1d+U
	 WOKrjPXsNCqo6LRScH0OsMk26GfeZm3FwxaMUtBF0ZdZ4KK56xgAZjMyx3N/5Nq1bM
	 dBHN9klN8m27xtEiRGPCUEfCFFtlAYKwrSg+WXgQwshv4UmvaKoXbDootnIq7T0oHg
	 kljtm/yAOBttfXm65crzKQy3OSvN7yx+olzlhfErt6WhywXtq80njSS3G+nairfXw3
	 sdX4MGEp+rddSkT2gXLsqWLgNFV6B5sbiIYoal1eftmw5dLftnjZjhAZlQbBPx/Cl2
	 GZ85xID2moNYQ==
Date: Thu, 29 Feb 2024 15:00:13 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Jan Beulich <jbeulich@suse.com>
cc: Federico Serafini <federico.serafini@bugseng.com>, consulting@bugseng.com, 
    Simone Ballarin <simone.ballarin@bugseng.com>, 
    Doug Goldstein <cardoe@cardoe.com>, 
    Stefano Stabellini <sstabellini@kernel.org>, 
    Andrew Cooper <andrew.cooper3@citrix.com>, 
    George Dunlap <george.dunlap@citrix.com>, Julien Grall <julien@xen.org>, 
    Wei Liu <wl@xen.org>, xen-devel@lists.xenproject.org
Subject: Re: [XEN PATCH] automation/eclair: extend deviations of MISRA C:2012
 Rule 16.3
In-Reply-To: <0e2f5394-6322-4845-954f-ff080626523f@suse.com>
Message-ID: <alpine.DEB.2.22.394.2402291458370.853156@ubuntu-linux-20-04-desktop>
References: <6c96f889ae3fbdf516f0edfe4a5fb207540b6862.1709109662.git.federico.serafini@bugseng.com> <1a00c221-ce1c-45be-9ab3-e8f1a0e5678d@suse.com> <98d82c40-a81a-4c75-8cdf-aaef640382b6@bugseng.com> <0e2f5394-6322-4845-954f-ff080626523f@suse.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Thu, 29 Feb 2024, Jan Beulich wrote:
> On 29.02.2024 09:01, Federico Serafini wrote:
> > On 28/02/24 10:06, Jan Beulich wrote:
> >> On 28.02.2024 09:53, Federico Serafini wrote:
> >>> --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> >>> +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> >>
> >> Comments below apply similarly to text added to this file.
> >>
> >>> --- a/docs/misra/deviations.rst
> >>> +++ b/docs/misra/deviations.rst
> >>> @@ -291,7 +291,14 @@ Deviations related to MISRA C:2012 Rules:
> >>>        - Project-wide deviation; tagged as `deliberate` for ECLAIR.
> >>>   
> >>>      * - R16.3
> >>> -     - Switch clauses ending with continue, goto, return statements are safe.
> >>> +     - Switch clauses ending with an unconditional flow control statement
> >>> +       (i.e., continue, goto, or return) are safe.
> >>> +     - Tagged as `safe` for ECLAIR.
> >>
> >> With this edit (unmentioned in the description, btw) ...
> >>
> >>> +   * - R16.3
> >>> +     - Switch clauses ending with an if-else statement are safe if both
> >>> +       branches consist of a flow control statement (i.e., continue, break,
> >>> +       goto, return).
> >>
> >> ... why is it not also "ending with" here?
> > 
> > Because the allowed pattern is:
> > 
> > if ( cond )
> >   return; /* Or continue / break / goto */
> > else
> >   break;  /* Or continue / goto / return */
> > 
> > See below for more information.
> > 
> >>
> >> Also what about either situation ending with a call to a noreturn function?
> > 
> > This can be added.
> > 
> >>
> >>> @@ -307,6 +314,16 @@ Deviations related to MISRA C:2012 Rules:
> >>>        - Switch clauses ending with failure method \"BUG()\" are safe.
> >>>        - Tagged as `safe` for ECLAIR.
> >>>   
> >>> +   * - R16.3
> >>> +     - On X86, switch clauses ending generating an exception through
> >>> +       \"generate_exception()\" are safe.
> >>> +     - Tagged as `safe` for ECLAIR.
> >>
> >> This macro is limited to the emulator, so shouldn't be deviated globally.
> > 
> > Noted.
> > 
> >> Furthermore - why does the special case need mentioning here? Shouldn't
> >> it be the underlying pattern which is deviated (along the lines of the
> >> earlier ones):
> >>
> >>      if ( true )
> >>      {
> >>          ...
> >>          goto ...; /* Or break / continue / return */
> >>      }
> > 
> > This pattern that involves a compound statement for the true branch
> > is not deviated by this configuration.
> > 
> > See below for more information.
> > 
> >>
> >>> +   * - R16.3
> >>> +     - Switch clauses ending generating a parse error through
> >>> +       \"PARSE_ERR_RET()\" are safe.
> >>> +     - Tagged as `safe` for ECLAIR.
> >>
> >> Again this isn't a global scope macro, so shouldn't be deviated globally.
> > 
> > Noted.
> > 
> >> Plus it ends in "return", so ought to be covered by the earlier clause.
> >> The fact that the return is in a body of do {} while(0) shouldn't matter
> >> at all - that's purely syntactic sugar.
> > 
> > I gather from your comments/questions that you would like to deviate
> > *all* the patterns where an unintentional fall through can not happen.
> > 
> > Rule 16.3 is a purely syntactic rule, and, as a consequence,
> > in the current version of ECLAIR additional "allowed pattern" (aka
> > deviations) for that rule need to be described through AST nodes,
> > meaning that all what you consider as syntactic sugar cannot be ignored.
> > 
> > A deviation that covers all the pattern you are asking for could be
> > done, but it will result in a complex and quite long expression
> > (not easy to read and justify in front of an assessor).
> > 
> > Hence, what I am proposing is to deviate only the the simplest and
> > most readable cases, such as:
> > 
> > if ( cond )
> >    return x;
> > else
> >    return y;
> > 
> > without involving compound statements, fake do-wile and fake if
> > statements but rather deviating the macro inside of which are used
> > (as I did).
> 
> I see. Problem is that this isn't sufficient for the code we have, and
> the seemingly random deviation of certain constructs by name looks to
> me as pretty undesirable.

Yeah, I also think it is not ideal. At the same time among all options,
it is probably the best way forward (better than in-code comments or
better than leaving the violations outstanding).

I think we should go for it.


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 23:10:32 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 23:10:32 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687459.1070941 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpXm-0005Q8-Cj; Thu, 29 Feb 2024 23:10:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687459.1070941; Thu, 29 Feb 2024 23:10:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfpXm-0005Q1-A7; Thu, 29 Feb 2024 23:10:26 +0000
Received: by outflank-mailman (input) for mailman id 687459;
 Thu, 29 Feb 2024 23:10:25 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfpXl-0005Ps-GY
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 23:10:25 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id b6dffbc5-d757-11ee-afd8-a90da7624cb6;
 Fri, 01 Mar 2024 00:10:23 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id E1992CE1B22;
 Thu, 29 Feb 2024 23:10:19 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5DF7C433F1;
 Thu, 29 Feb 2024 23:10:17 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: b6dffbc5-d757-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709248219;
	bh=PGsZG3sU7hajT14NZwdUpq2C3rYv9teT56gyoIJ0P3s=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=p77jp8uduPUDTaJp2v38mcsWqHlje8fZHiub+sCfdF3kN15dy28vdCrk7uNsSiNlq
	 CRZ1MIEVxT/gFrWsOAYLcp6AYcs5E4yz1bF0VxkeTZi4dPDH0IoF7xyZf45t0i6aMd
	 tTAi/zdHOvs8OIRA5ArQtCAc54e0ECWijdri+rDbx6AtNJyxkvt/8ujvDMB3kUiwKX
	 YuZFsk4CnTISF4nqmlUAnuoL/NmwdefePanMk63kkdJXecO4f803pLWVMyomF2Ktnq
	 zOIvrChtbc3EPV6Rjy97ivHEtpUche0/Zfu1AbxWAAiZwwM2+S9xtSrYKSqJGIbChV
	 ELyK/8bVMRD4g==
Date: Thu, 29 Feb 2024 15:10:16 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Vikram Garhwal <vikram.garhwal@amd.com>
cc: qemu-devel@nongnu.org, sstabellini@kernel.org, jgross@suse.com, 
    Anthony Perard <anthony.perard@citrix.com>, Paul Durrant <paul@xen.org>, 
    Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, 
    David Hildenbrand <david@redhat.com>, 
    =?UTF-8?Q?Philippe_Mathieu-Daud=C3=A9?= <philmd@linaro.org>, 
    "open list:X86 Xen CPUs" <xen-devel@lists.xenproject.org>
Subject: Re: [QEMU][PATCH v3 6/7] xen: add map and unmap callbacks for grant
 region
In-Reply-To: <20240227223501.28475-7-vikram.garhwal@amd.com>
Message-ID: <alpine.DEB.2.22.394.2402291510080.853156@ubuntu-linux-20-04-desktop>
References: <20240227223501.28475-1-vikram.garhwal@amd.com> <20240227223501.28475-7-vikram.garhwal@amd.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Tue, 27 Feb 2024, Vikram Garhwal wrote:
> From: Juergen Gross <jgross@suse.com>
> 
> Add the callbacks for mapping/unmapping guest memory via grants to the
> special grant memory region.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
>  hw/xen/xen-mapcache.c | 176 +++++++++++++++++++++++++++++++++++++++++-
>  system/physmem.c      |  11 ++-
>  2 files changed, 182 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
> index 179b7e95b2..2e4c9b4947 100644
> --- a/hw/xen/xen-mapcache.c
> +++ b/hw/xen/xen-mapcache.c
> @@ -9,6 +9,8 @@
>   */
>  
>  #include "qemu/osdep.h"
> +#include "qemu/queue.h"
> +#include "qemu/thread.h"
>  #include "qemu/units.h"
>  #include "qemu/error-report.h"
>  
> @@ -23,6 +25,8 @@
>  #include "sysemu/xen-mapcache.h"
>  #include "trace.h"
>  
> +#include <xenevtchn.h>
> +#include <xengnttab.h>
>  
>  #if HOST_LONG_BITS == 32
>  #  define MCACHE_BUCKET_SHIFT 16
> @@ -377,7 +381,7 @@ uint8_t *xen_map_cache(hwaddr phys_addr, hwaddr size,
>      return p;
>  }
>  
> -ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
> +static ram_addr_t xen_ram_addr_from_mapcache_try(void *ptr)
>  {
>      MapCacheEntry *entry = NULL;
>      MapCacheRev *reventry;
> @@ -588,10 +592,179 @@ uint8_t *xen_replace_cache_entry(hwaddr old_phys_addr,
>      return p;
>  }
>  
> +struct XENMappedGrantRegion {
> +    void *addr;
> +    unsigned int pages;
> +    unsigned int refs;
> +    unsigned int prot;
> +    uint32_t idx;
> +    QLIST_ENTRY(XENMappedGrantRegion) list;
> +};
> +
> +static xengnttab_handle *xen_region_gnttabdev;
> +static QLIST_HEAD(GrantRegionList, XENMappedGrantRegion) xen_grant_mappings =
> +    QLIST_HEAD_INITIALIZER(xen_grant_mappings);
> +static QemuMutex xen_map_mutex;
> +
> +static void *xen_map_grant_dyn(MemoryRegion **mr, hwaddr addr, hwaddr *plen,
> +                               bool is_write, MemTxAttrs attrs)
> +{
> +    unsigned int page_off = addr & (XC_PAGE_SIZE - 1);
> +    unsigned int i;
> +    unsigned int total_grants = 0;
> +    unsigned int nrefs = (page_off + *plen + XC_PAGE_SIZE - 1) >> XC_PAGE_SHIFT;
> +    uint32_t ref = (addr - XEN_GRANT_ADDR_OFF) >> XC_PAGE_SHIFT;
> +    uint32_t *refs = NULL;
> +    unsigned int prot = PROT_READ;
> +    struct XENMappedGrantRegion *mgr = NULL;
> +
> +    if (is_write) {
> +        prot |= PROT_WRITE;
> +    }
> +
> +    qemu_mutex_lock(&xen_map_mutex);
> +
> +    QLIST_FOREACH(mgr, &xen_grant_mappings, list) {
> +        if (mgr->idx == ref &&
> +            mgr->pages == nrefs &&
> +            (mgr->prot & prot) == prot) {
> +            break;
> +        }
> +
> +        total_grants += mgr->pages;
> +    }
> +
> +    if (!mgr) {
> +        if (nrefs + total_grants >= XEN_MAX_VIRTIO_GRANTS) {
> +            qemu_mutex_unlock(&xen_map_mutex);
> +            return NULL;
> +        }
> +
> +        mgr = g_new(struct XENMappedGrantRegion, 1);
> +
> +        if (nrefs == 1) {
> +            refs = &ref;
> +        } else {
> +            refs = g_new(uint32_t, nrefs);
> +            for (i = 0; i < nrefs; i++) {
> +                refs[i] = ref + i;
> +            }
> +        }
> +        mgr->addr = xengnttab_map_domain_grant_refs(xen_region_gnttabdev, nrefs,
> +                                                    xen_domid, refs, prot);
> +        if (mgr->addr) {
> +            mgr->pages = nrefs;
> +            mgr->refs = 1;
> +            mgr->prot = prot;
> +            mgr->idx = ref;
> +
> +            QLIST_INSERT_HEAD(&xen_grant_mappings, mgr, list);
> +        } else {
> +            g_free(mgr);
> +            mgr = NULL;
> +        }
> +    } else {
> +        mgr->refs++;
> +    }
> +
> +    qemu_mutex_unlock(&xen_map_mutex);
> +
> +    if (nrefs > 1) {
> +        g_free(refs);
> +    }
> +
> +    return mgr ? mgr->addr + page_off : NULL;
> +}
> +
> +static void xen_unmap_grant_dyn(MemoryRegion *mr, void *buffer, ram_addr_t addr,
> +                                hwaddr len, bool is_write, hwaddr access_len)
> +{
> +    unsigned int page_off = (unsigned long)buffer & (XC_PAGE_SIZE - 1);
> +    unsigned int nrefs = (page_off + len + XC_PAGE_SIZE - 1) >> XC_PAGE_SHIFT;
> +    unsigned int prot = PROT_READ;
> +    struct XENMappedGrantRegion *mgr = NULL;
> +
> +    if (is_write) {
> +        prot |= PROT_WRITE;
> +    }
> +
> +    qemu_mutex_lock(&xen_map_mutex);
> +
> +    QLIST_FOREACH(mgr, &xen_grant_mappings, list) {
> +        if (mgr->addr == buffer - page_off &&
> +            mgr->pages == nrefs &&
> +            (mgr->prot & prot) == prot) {
> +            break;
> +        }
> +    }
> +    if (mgr) {
> +        mgr->refs--;
> +        if (!mgr->refs) {
> +            xengnttab_unmap(xen_region_gnttabdev, mgr->addr, nrefs);
> +
> +            QLIST_REMOVE(mgr, list);
> +            g_free(mgr);
> +        }
> +    } else {
> +        error_report("xen_unmap_grant_dyn() trying to unmap unknown buffer");
> +    }
> +
> +    qemu_mutex_unlock(&xen_map_mutex);
> +}
> +
> +static ram_addr_t xen_ram_addr_from_grant_cache(void *ptr)
> +{
> +    unsigned int page_off = (unsigned long)ptr & (XC_PAGE_SIZE - 1);
> +    struct XENMappedGrantRegion *mgr = NULL;
> +    ram_addr_t raddr = RAM_ADDR_INVALID;
> +
> +    qemu_mutex_lock(&xen_map_mutex);
> +
> +    QLIST_FOREACH(mgr, &xen_grant_mappings, list) {
> +        if (mgr->addr == ptr - page_off) {
> +            break;
> +        }
> +    }
> +
> +    if (mgr) {
> +        raddr = (mgr->idx << XC_PAGE_SHIFT) + page_off + XEN_GRANT_ADDR_OFF;
> +    }
> +
> +    qemu_mutex_unlock(&xen_map_mutex);
> +
> +    return raddr;
> +}
> +
> +ram_addr_t xen_ram_addr_from_mapcache(void *ptr)
> +{
> +    ram_addr_t raddr;
> +
> +    raddr = xen_ram_addr_from_mapcache_try(ptr);
> +    if (raddr == RAM_ADDR_INVALID) {
> +        raddr = xen_ram_addr_from_grant_cache(ptr);
> +    }
> +
> +    return raddr;
> +}
> +
> +static const struct MemoryRegionOps xen_grant_mr_ops = {
> +    .map = xen_map_grant_dyn,
> +    .unmap = xen_unmap_grant_dyn,
> +    .endianness = DEVICE_LITTLE_ENDIAN,
> +};
> +
>  MemoryRegion *xen_init_grant_ram(void)
>  {
>      RAMBlock *block;
>  
> +    qemu_mutex_init(&xen_map_mutex);
> +
> +    xen_region_gnttabdev = xengnttab_open(NULL, 0);
> +    if (xen_region_gnttabdev == NULL) {
> +        fprintf(stderr, "can't open gnttab device\n");
> +        return NULL;
> +    }
> +
>      memory_region_init(&ram_grants, NULL, "xen.grants",
>                         XEN_MAX_VIRTIO_GRANTS * XC_PAGE_SIZE);
>      block = g_malloc0(sizeof(*block));
> @@ -606,6 +779,7 @@ MemoryRegion *xen_init_grant_ram(void)
>      ram_grants.ram_block = block;
>      ram_grants.ram = true;
>      ram_grants.terminates = true;
> +    ram_grants.ops = &xen_grant_mr_ops;
>      ram_block_add_list(block);
>      memory_region_add_subregion(get_system_memory(), XEN_GRANT_ADDR_OFF,
>                                  &ram_grants);
> diff --git a/system/physmem.c b/system/physmem.c
> index d989e9fc1f..e6fc075d8f 100644
> --- a/system/physmem.c
> +++ b/system/physmem.c
> @@ -2232,13 +2232,16 @@ RAMBlock *qemu_ram_block_from_host(void *ptr, bool round_offset,
>  
>      if (xen_enabled()) {
>          ram_addr_t ram_addr;
> +
>          RCU_READ_LOCK_GUARD();
>          ram_addr = xen_ram_addr_from_mapcache(ptr);
> -        block = qemu_get_ram_block(ram_addr);
> -        if (block) {
> -            *offset = ram_addr - block->offset;
> +        if (ram_addr != RAM_ADDR_INVALID) {
> +            block = qemu_get_ram_block(ram_addr);
> +            if (block) {
> +                *offset = ram_addr - block->offset;
> +            }
> +            return block;
>          }
> -        return block;
>      }
>  
>      RCU_READ_LOCK_GUARD();
> -- 
> 2.17.1
> 


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 23:29:22 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 23:29:22 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687462.1070951 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfppq-0007kW-Rr; Thu, 29 Feb 2024 23:29:06 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687462.1070951; Thu, 29 Feb 2024 23:29:06 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfppq-0007kP-PK; Thu, 29 Feb 2024 23:29:06 +0000
Received: by outflank-mailman (input) for mailman id 687462;
 Thu, 29 Feb 2024 23:29:05 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=nI6L=KG=kernel.org=sstabellini@srs-se1.protection.inumbo.net>)
 id 1rfppp-0007kF-HG
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 23:29:05 +0000
Received: from sin.source.kernel.org (sin.source.kernel.org
 [2604:1380:40e1:4800::1])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 51690d33-d75a-11ee-a1ee-f123f15fe8a2;
 Fri, 01 Mar 2024 00:29:02 +0100 (CET)
Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])
 by sin.source.kernel.org (Postfix) with ESMTP id 05847CE1B64;
 Thu, 29 Feb 2024 23:28:57 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id EAFE7C433F1;
 Thu, 29 Feb 2024 23:28:54 +0000 (UTC)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 51690d33-d75a-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
	s=k20201202; t=1709249336;
	bh=yLkOZYsNkYyeStMQcxfFu62ISwDUw5RGaHksZiUJAX8=;
	h=Date:From:To:cc:Subject:In-Reply-To:References:From;
	b=lRE0z+Ha1P5ip46wgGwvSB1xbNS7vphi7+zWrReIlVE5Do7rYXfJEzQBrPALE2sHu
	 gb455KHfq9jDjCCH499nNBrkVdcJ0fbQ0PIZo7IWsLDjlCUrU7AO3m5beI++306GFC
	 +RTpUXBJTis+fUq/BHPQ+OCEeepr5Jlv/IX27BXpTXifyZV/2MRzrhq5+MJrH0T4EI
	 q5Jh0Q7ORIJ+adU7Wkr1MFLIWrpWLKoB8zDEj9yagneCLccNK//KpVle7q2BzuWbr4
	 yB1V0iYIqk/GoX38p6myveVKmJoRlu3X66VWP5Y8Q0df+sANs6jgJa7YxZUGiPzNey
	 SmRxW7HwpKTJA==
Date: Thu, 29 Feb 2024 15:28:53 -0800 (PST)
From: Stefano Stabellini <sstabellini@kernel.org>
X-X-Sender: sstabellini@ubuntu-linux-20-04-desktop
To: Federico Serafini <federico.serafini@bugseng.com>
cc: Jan Beulich <jbeulich@suse.com>, andrew.cooper3@citrix.com, 
    george.dunlap@citrix.com, julien@xen.org, bertrand.marquis@arm.com, 
    roger.pau@citrix.com, xen-devel@lists.xenproject.org, 
    Stefano Stabellini <stefano.stabellini@amd.com>, 
    roberto.bagnara@bugseng.com
Subject: Re: [PATCH v2 2/3] docs/misra/rules.rst: add rule 5.5
In-Reply-To: <3632611e-61ff-455a-9cc1-990a1f663d83@bugseng.com>
Message-ID: <alpine.DEB.2.22.394.2402291528230.853156@ubuntu-linux-20-04-desktop>
References: <alpine.DEB.2.22.394.2402131431070.1925432@ubuntu-linux-20-04-desktop> <20240213223334.3693410-2-stefano.stabellini@amd.com> <50719397-b053-43e1-9cf7-cc9eae9098ed@suse.com> <40b7465f-4966-43c7-8db3-e28a6cc48445@bugseng.com>
 <2da73259-a86e-4a13-9fd4-4217372f4b76@suse.com> <3632611e-61ff-455a-9cc1-990a1f663d83@bugseng.com>
User-Agent: Alpine 2.22 (DEB 394 2020-01-19)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Wed, 14 Feb 2024, Federico Serafini wrote:
> On 14/02/24 14:15, Jan Beulich wrote:
> > On 14.02.2024 12:27, Federico Serafini wrote:
> > > On 14/02/24 09:28, Jan Beulich wrote:
> > > > On 13.02.2024 23:33, Stefano Stabellini wrote:
> > > > > Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
> > > > > ---
> > > > >    docs/misra/rules.rst | 6 ++++++
> > > > >    1 file changed, 6 insertions(+)
> > > > > 
> > > > > diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
> > > > > index c185366966..931158b354 100644
> > > > > --- a/docs/misra/rules.rst
> > > > > +++ b/docs/misra/rules.rst
> > > > > @@ -181,6 +181,12 @@ maintainers if you want to suggest a change.
> > > > >           headers (xen/include/public/) are allowed to retain longer
> > > > >           identifiers for backward compatibility.
> > > > >    +   * - `Rule 5.5
> > > > > <https://gitlab.com/MISRA/MISRA-C/MISRA-C-2012/Example-Suite/-/blob/master/R_05_05.c>`_
> > > > > +     - Required
> > > > > +     - Identifiers shall be distinct from macro names
> > > > > +     - Clashes between function-like macros and non-callable entities
> > > > > +       are allowed. The pattern #define x x is also allowed.
> > > > 
> > > > Just for me to know what exactly is covered (hence also a question
> > > > to Roberto as to [to be] implemented Eclair behavior): Even when
> > > > the above would be sufficient (and imo better) people frequently
> > > > write
> > > > 
> > > > #define a(x, y) b(x, y)
> > > > 
> > > > which, transformed to the specific case here, would then be
> > > > 
> > > > #define a(x, y) a(x, y)
> > > > 
> > > > I'd assume such ought to also be covered, but that's not clear
> > > > from the spelling above.
> > > 
> > > I list what happens in some different situations,
> > > then we can find the right words for the documentation and/or
> > > refine the configuration:
> > > 
> > > If you
> > > #define x x
> > > and then use `x' as identifier,
> > > the resulting violation is deviated (allowed pattern).
> > > 
> > > If you
> > > #define a(x, y) a(x, y)
> > > and then use `a' as identifier for a non-callable entity,
> > > the resulting violation is deviated (no clash with non-callable
> > > entities).
> > > If you use identifier `a' for a callable entity, the resulting violation
> > > is reported: the allowed pattern covers only macros expanding to their
> > > own name, in this case the macro name is considered to be
> > > `a' only, not a(x, y).
> > > 
> > > If you
> > > #define a(x, y) b(x, y)
> > > and then use `a' as identifier for a non-callable entity,
> > > the resulting violation is deviated (no clash with non-callable
> > > entities).
> > 
> > I'm afraid I don't see what violation there is in this case, to
> > deviate. As a result I'm also not sure I correctly understand the
> > rest of your reply.
> 
> #define a(x, y) b(x, y)
> 
> int a; // Violation of Rule 5.5.
> 
> The macro name `a' that exist before the preprocessing phase,
> still exists after the preprocessing phase as identifier for the integer
> variable and this is a violation.
> 
> > > If you use `a' as identifier for a callable entity,
> > > this is not a violation because after the preprocessing phase,
> > > identifier `a' no longer exists.
> I correct myself:
> if you use `a' as identifier for a *function*,
> it is not a violation because after the preprocessing phase
> the identifier `a' no longer exists, for example:
> 
> #define a(x, y) b(x, y)
> 
> void a(int x, int y); // Ok.

Federico, do you have a better wording suggestion for this rule?

Jan, any further requests here? What would you like to see as next step?


From xen-devel-bounces@lists.xenproject.org Thu Feb 29 23:43:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 23:43:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687468.1070975 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfq3i-0002pn-8U; Thu, 29 Feb 2024 23:43:26 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687468.1070975; Thu, 29 Feb 2024 23:43:26 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfq3i-0002pg-3w; Thu, 29 Feb 2024 23:43:26 +0000
Received: by outflank-mailman (input) for mailman id 687468;
 Thu, 29 Feb 2024 23:43:24 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mnTJ=KG=gmail.com=alexei.starovoitov@srs-se1.protection.inumbo.net>)
 id 1rfq3g-0002pa-SX
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 23:43:24 +0000
Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com
 [2607:f8b0:4864:20::631])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 52aea4a3-d75c-11ee-a1ee-f123f15fe8a2;
 Fri, 01 Mar 2024 00:43:22 +0100 (CET)
Received: by mail-pl1-x631.google.com with SMTP id
 d9443c01a7336-1dc29f1956cso13542095ad.0
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 15:43:22 -0800 (PST)
Received: from localhost.localdomain ([2620:10d:c090:400::5:8f17])
 by smtp.gmail.com with ESMTPSA id
 l8-20020a170903120800b001dcc129cc2esm2079055plh.60.2024.02.29.15.43.18
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Thu, 29 Feb 2024 15:43:20 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 52aea4a3-d75c-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709250201; x=1709855001; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:from:to:cc:subject:date:message-id:reply-to;
        bh=NzHHSXtSLrv/NknHpxtBThbqiIeWhWk4Z2HCRi70who=;
        b=l1lskIJHOW0Pdc6x9Gyl366e03Us3X105se85FCYIUEcI0JVQBBFtCPB/PQHh0YXQz
         7u95N7iylLxQmjsPvHOVIqrbtoZkZGxdHJgrXBUnywkrV7dv8KKzmJylbFuuX99bCVxV
         3BcyeizhrTCKjT8/lCCxuMzdCrewX9Khy3p66uu2gcwiFLB5T38U1jGdxSTt2E9CEhsN
         wBH9KfRniTbGhUPJ+j/JsxXNPtzMEznhyUK6eyhmmNs4C/nRf7jWR1kv0ZnRXEN+QMep
         ubQPphSHBst3oYGmCVMbXVPXh4Ezj7snzIxzRsrsrjpzgWhKtT3LaKVdadLf3xz71Xj8
         0ILg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709250201; x=1709855001;
        h=content-transfer-encoding:mime-version:message-id:date:subject:cc
         :to:from:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=NzHHSXtSLrv/NknHpxtBThbqiIeWhWk4Z2HCRi70who=;
        b=dCq9jaSYqpZKTaxLs2KZwEUcSAPI4ugfiVRTP10V+CUBrcw86FhVNT7EDTepjZOKsp
         XmYpc6qws6o15RrPf/bjqa0CneUjvxcf6bt745C3DeLMe0Jy+mx+fYl+sP70KbKw3fRO
         ljHMeRWOtU00bKf/cVUKHTFzvJLKJMQquse2MACuuDUWF+QbRKf3DJ2pbw+QRedAi/QE
         Vk8M4dxf2nToUW4ifi3seVQYx4rmD9DCuq6kaTYalig4yYh62KOthBStDX5yO9ZCIPSg
         A3T1vPy0jXpRks43PNFkmBYwzWAsB3gf0PMrdKPN91YccJzhV6x1tVj+ZhZCTLY4zImb
         50Jw==
X-Forwarded-Encrypted: i=1; AJvYcCUrkjqzn9D72wUMgGZ7/EL56gyWzgSdylxOTjDAqMyEZG6bvC79XXm25Or8MU5oQEh1q2okE/slw1JsIwxeb+p5uZM9c5yO5b/L5aSXxRw=
X-Gm-Message-State: AOJu0Yyi70JzwqDroHIJ5xLBcQUE+RNQgbTkKQqgUyvK7AYQsut+8Xu9
	bI2wLdw8+kzHOUgiT+PUnG/iUF57yp8ynmXY8MvRFy2FOCrA+Mdc
X-Google-Smtp-Source: AGHT+IGfl0YdKr5QO4H+FovdTc3eTEo8Q94QPX7tglxTnGBDlmv4twi/rgWyLSio1iEhhbNejBslzA==
X-Received: by 2002:a17:902:d511:b0:1db:be69:d037 with SMTP id b17-20020a170902d51100b001dbbe69d037mr22315plg.46.1709250201120;
        Thu, 29 Feb 2024 15:43:21 -0800 (PST)
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: bpf@vger.kernel.org
Cc: daniel@iogearbox.net,
	andrii@kernel.org,
	torvalds@linux-foundation.org,
	brho@google.com,
	hannes@cmpxchg.org,
	lstoakes@gmail.com,
	akpm@linux-foundation.org,
	urezki@gmail.com,
	hch@infradead.org,
	boris.ostrovsky@oracle.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	linux-mm@kvack.org,
	xen-devel@lists.xenproject.org,
	kernel-team@fb.com
Subject: [PATCH v3 bpf-next 0/3] mm: Cleanup and identify various users of kernel virtual address space
Date: Thu, 29 Feb 2024 15:43:13 -0800
Message-Id: <20240229234316.44409-1-alexei.starovoitov@gmail.com>
X-Mailer: git-send-email 2.39.3 (Apple Git-145)
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Alexei Starovoitov <ast@kernel.org>

v2 -> v3
- added Christoph's reviewed-by to patch 1
- cap commit log lines to 75 chars
- factored out common checks in patch 3 into helper
- made vm_area_unmap_pages() return void

There are various users of kernel virtual address space:
vmalloc, vmap, ioremap, xen.

- vmalloc use case dominates the usage. Such vm areas have VM_ALLOC flag
and these areas are treated differently by KASAN.

- the areas created by vmap() function should be tagged with VM_MAP
(as majority of the users do).

- ioremap areas are tagged with VM_IOREMAP and vm area start is aligned
to size of the area unlike vmalloc/vmap.

- there is also xen usage that is marked as VM_IOREMAP, but it doesn't
call ioremap_page_range() unlike all other VM_IOREMAP users.

To clean this up:
1. Enforce that ioremap_page_range() checks the range and VM_IOREMAP flag
2. Introduce VM_XEN flag to separate xen us cases from ioremap

In addition BPF would like to reserve regions of kernel virtual address
space and populate it lazily, similar to xen use cases.
For that reason, introduce VM_SPARSE flag and vm_area_[un]map_pages()
helpers to populate this sparse area.

In the end the /proc/vmallocinfo will show
"vmalloc"
"vmap"
"ioremap"
"xen"
"sparse"
categories for different kinds of address regions.

ioremap, xen, sparse will return zero when dumped through /proc/kcore

Alexei Starovoitov (3):
  mm: Enforce VM_IOREMAP flag and range in ioremap_page_range.
  mm, xen: Separate xen use cases from ioremap.
  mm: Introduce VM_SPARSE kind and vm_area_[un]map_pages().

 arch/x86/xen/grant-table.c         |  2 +-
 drivers/xen/xenbus/xenbus_client.c |  2 +-
 include/linux/vmalloc.h            |  6 +++
 mm/vmalloc.c                       | 75 +++++++++++++++++++++++++++++-
 4 files changed, 81 insertions(+), 4 deletions(-)

-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 23:43:31 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 23:43:31 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687469.1070984 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfq3n-00035r-In; Thu, 29 Feb 2024 23:43:31 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687469.1070984; Thu, 29 Feb 2024 23:43:31 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfq3n-00035k-GD; Thu, 29 Feb 2024 23:43:31 +0000
Received: by outflank-mailman (input) for mailman id 687469;
 Thu, 29 Feb 2024 23:43:29 +0000
Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254]
 helo=se1-gles-sth1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mnTJ=KG=gmail.com=alexei.starovoitov@srs-se1.protection.inumbo.net>)
 id 1rfq3l-00034g-R7
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 23:43:29 +0000
Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com
 [2607:f8b0:4864:20::631])
 by se1-gles-sth1.inumbo.com (Halon) with ESMTPS
 id 5612cf85-d75c-11ee-afd8-a90da7624cb6;
 Fri, 01 Mar 2024 00:43:28 +0100 (CET)
Received: by mail-pl1-x631.google.com with SMTP id
 d9443c01a7336-1dcd6a3da83so11311245ad.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 15:43:28 -0800 (PST)
Received: from localhost.localdomain ([2620:10d:c090:400::5:8f17])
 by smtp.gmail.com with ESMTPSA id
 e12-20020a17090301cc00b001d90a67e10bsm2081312plh.109.2024.02.29.15.43.23
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Thu, 29 Feb 2024 15:43:24 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5612cf85-d75c-11ee-afd8-a90da7624cb6
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709250207; x=1709855007; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=ZYaENkYQ4G5HZw66zLSCBAz4c9QdNSpOW86RMIeSYNk=;
        b=HbvU+1D2788D3v57L/qKiGhMpnoULiHi2FjuIvjPkv/xH6oSvlJvdcmrwJo2wIKAE9
         ANFTbBV+CWnZS94fTdWXmmFPjSZqDQuGpns0hhl3ia6q2REvpm0EFzx1edd9PPCdp8zG
         DwITDRQOJ3XC5x8V/eTys3D4Y8rSF6Ilq1TLoUCLSi71sUeinHP1WY/SjunUp92KHV2t
         cHOi1SpGxdxhR0iWuGhK78b/O8yGWyGJ1ZUbIRSdkuks7ic4oZJQY/dfOlhDJU0aiXC+
         4U0F6efzrC69YNx4xGFvHg/R39HnoMynaySNAbl6KObRnVJyyAzQJpEMwNL2gz8oIXdM
         u0Og==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709250207; x=1709855007;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=ZYaENkYQ4G5HZw66zLSCBAz4c9QdNSpOW86RMIeSYNk=;
        b=k4CL4EaSPCxnYqPIuytArkRV0vmIUlG/aOV6WxIyuWFhXr8ZHVEoE0UTYXrPRER4af
         XWOiAmKizd5o1z9lbuyhwjmFk+bAVSa+pyC+v7uE5Y3V0ywWk9Kaibd+N65x55DW0viK
         CuiBvxJrfLNwJQuvctXeaocGpy3kMz+fkc8KkioQ6OgcFCUfGCSYf5+62qiOjlbTkAvq
         gH0ZUZA1BAhlvWmXHWb43WOn4ax0kpXzr3q37/YKt2tDb8TcmYlOwc34M2fdc3f5yIA0
         MZI9Dm6YOAt9/Ka7n832rXQ4O3hvG6ZDJpvkF6LBATQCzYOa6nOtjRq/sA23yDOQ+Gj1
         OUEA==
X-Forwarded-Encrypted: i=1; AJvYcCWD0sl9aT8m7eHjJq3Eu7Tad5orfvJadv08t36VKU3pmvTU+aHEpAU13L0xEAiK8FohtIi5IEe17hWa+B4rpFGtJN/J7gPsjLRcOX2NMPY=
X-Gm-Message-State: AOJu0Yw9YhT6ygvfZrrSUxGR6mbu+zpfONPV+kfZwLU7rjc4dL2VyeC3
	YttQxTM/VvLr794T1hNcPC6oxXcz7FoHpe1Lefknwgtz/sejluqH
X-Google-Smtp-Source: AGHT+IGkWYefSAJiD6V3UAHJP4h3dcMjAtD7ol2bRxxY1LAVjzWJTomMlC/1P8AOvCsvxgL62Mpd0g==
X-Received: by 2002:a17:902:6847:b0:1dc:8790:6824 with SMTP id f7-20020a170902684700b001dc87906824mr50051pln.15.1709250204830;
        Thu, 29 Feb 2024 15:43:24 -0800 (PST)
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: bpf@vger.kernel.org
Cc: daniel@iogearbox.net,
	andrii@kernel.org,
	torvalds@linux-foundation.org,
	brho@google.com,
	hannes@cmpxchg.org,
	lstoakes@gmail.com,
	akpm@linux-foundation.org,
	urezki@gmail.com,
	hch@infradead.org,
	boris.ostrovsky@oracle.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	linux-mm@kvack.org,
	xen-devel@lists.xenproject.org,
	kernel-team@fb.com
Subject: [PATCH v3 bpf-next 1/3] mm: Enforce VM_IOREMAP flag and range in ioremap_page_range.
Date: Thu, 29 Feb 2024 15:43:14 -0800
Message-Id: <20240229234316.44409-2-alexei.starovoitov@gmail.com>
X-Mailer: git-send-email 2.39.3 (Apple Git-145)
In-Reply-To: <20240229234316.44409-1-alexei.starovoitov@gmail.com>
References: <20240229234316.44409-1-alexei.starovoitov@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Alexei Starovoitov <ast@kernel.org>

There are various users of get_vm_area() + ioremap_page_range() APIs.
Enforce that get_vm_area() was requested as VM_IOREMAP type and range
passed to ioremap_page_range() matches created vm_area to avoid
accidentally ioremap-ing into wrong address range.

Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
 mm/vmalloc.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index d12a17fc0c17..f42f98a127d5 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -307,8 +307,21 @@ static int vmap_range_noflush(unsigned long addr, unsigned long end,
 int ioremap_page_range(unsigned long addr, unsigned long end,
 		phys_addr_t phys_addr, pgprot_t prot)
 {
+	struct vm_struct *area;
 	int err;
 
+	area = find_vm_area((void *)addr);
+	if (!area || !(area->flags & VM_IOREMAP)) {
+		WARN_ONCE(1, "vm_area at addr %lx is not marked as VM_IOREMAP\n", addr);
+		return -EINVAL;
+	}
+	if (addr != (unsigned long)area->addr ||
+	    (void *)end != area->addr + get_vm_area_size(area)) {
+		WARN_ONCE(1, "ioremap request [%lx,%lx) doesn't match vm_area [%lx, %lx)\n",
+			  addr, end, (long)area->addr,
+			  (long)area->addr + get_vm_area_size(area));
+		return -ERANGE;
+	}
 	err = vmap_range_noflush(addr, end, phys_addr, pgprot_nx(prot),
 				 ioremap_max_page_shift);
 	flush_cache_vmap(addr, end);
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 23:43:34 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 23:43:34 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687470.1070994 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfq3q-0003Ms-Pj; Thu, 29 Feb 2024 23:43:34 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687470.1070994; Thu, 29 Feb 2024 23:43:34 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfq3q-0003Mh-Mn; Thu, 29 Feb 2024 23:43:34 +0000
Received: by outflank-mailman (input) for mailman id 687470;
 Thu, 29 Feb 2024 23:43:33 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mnTJ=KG=gmail.com=alexei.starovoitov@srs-se1.protection.inumbo.net>)
 id 1rfq3p-0002pa-JQ
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 23:43:33 +0000
Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com
 [2607:f8b0:4864:20::d2d])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 58391cc9-d75c-11ee-a1ee-f123f15fe8a2;
 Fri, 01 Mar 2024 00:43:32 +0100 (CET)
Received: by mail-io1-xd2d.google.com with SMTP id
 ca18e2360f4ac-7c7476a11e0so85423139f.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 15:43:31 -0800 (PST)
Received: from localhost.localdomain ([2620:10d:c090:400::5:8f17])
 by smtp.gmail.com with ESMTPSA id
 r19-20020aa78453000000b006e4dad633e1sm1850278pfn.177.2024.02.29.15.43.29
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Thu, 29 Feb 2024 15:43:30 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 58391cc9-d75c-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709250210; x=1709855010; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=8fSQQtv9Pdd1jRnQzOfB2+LAkUYgIfVtnLn4umtyxEI=;
        b=NSaokjsAWAl7udivLZ4z2OBEEW7b44fsKBXpvFhP7Y//L9NwnUrHQZK2tTpXSkRWVQ
         qKYmLhGkCCHCFOJ/2QuP+kDKMPWSw4FjkvFOQsjYjuFUfb3bFaB5FiVqfvino21p7n0x
         EIswM0HWCVW9RsRAZZFoei2wjfNwQq+1L2wAsn+JrpDiTRfQLWKLsQbmI7zqFybku8dS
         Qcv+qyDuoeEbMpB8KV6tM4CkVDD2WyoJ9UogJo5mg1cM02SYEeGLyIIM9J0Y/+4lDjxF
         pjaXOlPwjY78r/3flkdM0/T9smh34vG+6pipdO6Cf5Oj5g7iB5lozkijJKXYeDJkvyQJ
         SjNA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709250210; x=1709855010;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=8fSQQtv9Pdd1jRnQzOfB2+LAkUYgIfVtnLn4umtyxEI=;
        b=Tw6FnB5OInUCs7+VDvSxkgaPYdMNV1unmA5Naxi8rHA7WHnxeEZfr8znuuRzKATKUK
         vM2CrfnKnj23dRtfBbj7l1ThQdJzol/0hvTjnIoTtNosj7KfnWjxgGfTAaczuK/HKH53
         fvLjDxQuS6L63P6D7cRUADfio4OPUTFKIwO9B7BIrOsBZ5WcFX9VPZVPUIPK09WSm1Wq
         hkDx16mKIo5CK3qHlz+0TA6qHcv+mYitXFNX+Ek6u6joNvkPW54hUgc9JAn3ubghKTm1
         18eNpkpRWnuImHKUDwB8rn4xQtGNGbSQut0WN1KV82MsPTh4WMTpngxhzEezOHRP8EF7
         QDIQ==
X-Forwarded-Encrypted: i=1; AJvYcCWWlcfg9U6j7ODRz+8JupM5Kiq5mXvhRbWm63GQ9VehpJz6xcFFXiV6ccnbWR38Ii3bm0skFR5TatHZ7a1ghARr9mx8vFXLIp6ddzJ8aOY=
X-Gm-Message-State: AOJu0YxYFMp+WOilbpeNvzH+U+zSOXNfBmfPooUTIXTAKFTBhJVm92KW
	ikyA3a51tjLGGs5dP/c32MI5r8R/7XPlpEDhFzGlRRv8DQNU+r+U
X-Google-Smtp-Source: AGHT+IEjOcNXmJRmdRJJTTrs5fHGw0wy4T92DO/fjxQ1ryFJYxgxzO1wRU4PulyZo16hmDAMpzbQcA==
X-Received: by 2002:a05:6e02:148e:b0:365:a6a2:24b with SMTP id n14-20020a056e02148e00b00365a6a2024bmr273922ilk.17.1709250210665;
        Thu, 29 Feb 2024 15:43:30 -0800 (PST)
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: bpf@vger.kernel.org
Cc: daniel@iogearbox.net,
	andrii@kernel.org,
	torvalds@linux-foundation.org,
	brho@google.com,
	hannes@cmpxchg.org,
	lstoakes@gmail.com,
	akpm@linux-foundation.org,
	urezki@gmail.com,
	hch@infradead.org,
	boris.ostrovsky@oracle.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	linux-mm@kvack.org,
	xen-devel@lists.xenproject.org,
	kernel-team@fb.com
Subject: [PATCH v3 bpf-next 2/3] mm, xen: Separate xen use cases from ioremap.
Date: Thu, 29 Feb 2024 15:43:15 -0800
Message-Id: <20240229234316.44409-3-alexei.starovoitov@gmail.com>
X-Mailer: git-send-email 2.39.3 (Apple Git-145)
In-Reply-To: <20240229234316.44409-1-alexei.starovoitov@gmail.com>
References: <20240229234316.44409-1-alexei.starovoitov@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Alexei Starovoitov <ast@kernel.org>

xen grant table and xenbus ring are not ioremap the way arch specific code
is using it, so let's add VM_XEN flag to separate these use cases from
VM_IOREMAP users. xen will not and should not be calling
ioremap_page_range() on that range. /proc/vmallocinfo will print such
regions as "xen" instead of "ioremap".

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
 arch/x86/xen/grant-table.c         | 2 +-
 drivers/xen/xenbus/xenbus_client.c | 2 +-
 include/linux/vmalloc.h            | 1 +
 mm/vmalloc.c                       | 7 +++++--
 4 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/arch/x86/xen/grant-table.c b/arch/x86/xen/grant-table.c
index 1e681bf62561..b816db0349c4 100644
--- a/arch/x86/xen/grant-table.c
+++ b/arch/x86/xen/grant-table.c
@@ -104,7 +104,7 @@ static int arch_gnttab_valloc(struct gnttab_vm_area *area, unsigned nr_frames)
 	area->ptes = kmalloc_array(nr_frames, sizeof(*area->ptes), GFP_KERNEL);
 	if (area->ptes == NULL)
 		return -ENOMEM;
-	area->area = get_vm_area(PAGE_SIZE * nr_frames, VM_IOREMAP);
+	area->area = get_vm_area(PAGE_SIZE * nr_frames, VM_XEN);
 	if (!area->area)
 		goto out_free_ptes;
 	if (apply_to_page_range(&init_mm, (unsigned long)area->area->addr,
diff --git a/drivers/xen/xenbus/xenbus_client.c b/drivers/xen/xenbus/xenbus_client.c
index 32835b4b9bc5..b9c81a2d578b 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -758,7 +758,7 @@ static int xenbus_map_ring_pv(struct xenbus_device *dev,
 	bool leaked = false;
 	int err = -ENOMEM;
 
-	area = get_vm_area(XEN_PAGE_SIZE * nr_grefs, VM_IOREMAP);
+	area = get_vm_area(XEN_PAGE_SIZE * nr_grefs, VM_XEN);
 	if (!area)
 		return -ENOMEM;
 	if (apply_to_page_range(&init_mm, (unsigned long)area->addr,
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index c720be70c8dd..71075ece0ed2 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -28,6 +28,7 @@ struct iov_iter;		/* in uio.h */
 #define VM_FLUSH_RESET_PERMS	0x00000100	/* reset direct map and flush TLB on unmap, can't be freed in atomic context */
 #define VM_MAP_PUT_PAGES	0x00000200	/* put pages and free array in vfree */
 #define VM_ALLOW_HUGE_VMAP	0x00000400      /* Allow for huge pages on archs with HAVE_ARCH_HUGE_VMALLOC */
+#define VM_XEN			0x00000800	/* xen grant table and xenbus use cases */
 
 #if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
 	!defined(CONFIG_KASAN_VMALLOC)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index f42f98a127d5..d53ece3f38ee 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -3822,9 +3822,9 @@ long vread_iter(struct iov_iter *iter, const char *addr, size_t count)
 
 		if (flags & VMAP_RAM)
 			copied = vmap_ram_vread_iter(iter, addr, n, flags);
-		else if (!(vm && (vm->flags & VM_IOREMAP)))
+		else if (!(vm && (vm->flags & (VM_IOREMAP | VM_XEN))))
 			copied = aligned_vread_iter(iter, addr, n);
-		else /* IOREMAP area is treated as memory hole */
+		else /* IOREMAP | XEN area is treated as memory hole */
 			copied = zero_iter(iter, n);
 
 		addr += copied;
@@ -4415,6 +4415,9 @@ static int s_show(struct seq_file *m, void *p)
 	if (v->flags & VM_IOREMAP)
 		seq_puts(m, " ioremap");
 
+	if (v->flags & VM_XEN)
+		seq_puts(m, " xen");
+
 	if (v->flags & VM_ALLOC)
 		seq_puts(m, " vmalloc");
 
-- 
2.34.1



From xen-devel-bounces@lists.xenproject.org Thu Feb 29 23:43:39 2024
Return-path: <xen-devel-bounces@lists.xenproject.org>
Envelope-to: archives@lists.xen.org
Delivery-date: Thu, 29 Feb 2024 23:43:39 +0000
Received: from list by lists.xenproject.org with outflank-mailman.687471.1071003 (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfq3v-0003gs-3f; Thu, 29 Feb 2024 23:43:39 +0000
X-Outflank-Mailman: Message body and most headers restored to incoming version
Received: by outflank-mailman (output) from mailman id 687471.1071003; Thu, 29 Feb 2024 23:43:39 +0000
Received: from localhost ([127.0.0.1] helo=lists.xenproject.org)
	by lists.xenproject.org with esmtp (Exim 4.92)
	(envelope-from <xen-devel-bounces@lists.xenproject.org>)
	id 1rfq3v-0003gl-0d; Thu, 29 Feb 2024 23:43:39 +0000
Received: by outflank-mailman (input) for mailman id 687471;
 Thu, 29 Feb 2024 23:43:37 +0000
Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50]
 helo=se1-gles-flk1.inumbo.com)
 by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from
 <SRS0=mnTJ=KG=gmail.com=alexei.starovoitov@srs-se1.protection.inumbo.net>)
 id 1rfq3t-0002pa-Qz
 for xen-devel@lists.xenproject.org; Thu, 29 Feb 2024 23:43:37 +0000
Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com
 [2607:f8b0:4864:20::429])
 by se1-gles-flk1.inumbo.com (Halon) with ESMTPS
 id 5a97006e-d75c-11ee-a1ee-f123f15fe8a2;
 Fri, 01 Mar 2024 00:43:36 +0100 (CET)
Received: by mail-pf1-x429.google.com with SMTP id
 d2e1a72fcca58-6e59bbdd8c7so997186b3a.3
 for <xen-devel@lists.xenproject.org>; Thu, 29 Feb 2024 15:43:36 -0800 (PST)
Received: from localhost.localdomain ([2620:10d:c090:400::5:8f17])
 by smtp.gmail.com with ESMTPSA id
 r8-20020aa79ec8000000b006e50cedb59bsm1850608pfq.16.2024.02.29.15.43.32
 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256);
 Thu, 29 Feb 2024 15:43:33 -0800 (PST)
X-BeenThere: xen-devel@lists.xenproject.org
List-Id: Xen developer discussion <xen-devel.lists.xenproject.org>
List-Unsubscribe: <https://lists.xenproject.org/mailman/options/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xenproject.org>
List-Help: <mailto:xen-devel-request@lists.xenproject.org?subject=help>
List-Subscribe: <https://lists.xenproject.org/mailman/listinfo/xen-devel>,
 <mailto:xen-devel-request@lists.xenproject.org?subject=subscribe>
Errors-To: xen-devel-bounces@lists.xenproject.org
Precedence: list
Sender: "Xen-devel" <xen-devel-bounces@lists.xenproject.org>
X-Inumbo-ID: 5a97006e-d75c-11ee-a1ee-f123f15fe8a2
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20230601; t=1709250214; x=1709855014; darn=lists.xenproject.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=GGmuzG0BuzfpsyKgKDkTT6Enikl3D6AjMcJQdJLpWcI=;
        b=l2E3fNaajwujBe9DCGj69BvfC3VHTNVd4Xud9rR9o6BqgGx3OHagpwPaohBqjCWXUg
         jOUrgiYPAT/nFqF305EjfqbpW7e5Kyp3KsqyW2SAcyoCFs0z5roqhrAMiGUtZS8xBvJp
         UBpqzf19MUa8AJidNH6jMAEjh7uMVhe+XVm95rPsiVO3ohKxpAOYeH/r5NJQhc8u4WT2
         kp0rC7UOeuzLxv9lJQtgclwDTce/xuLY7o3OUionhSs1Xw4o52LyiSm6KBuB+ejf+STJ
         ZVEgLii/DOLZjwZcmMqTrxLUlhETuubrKPOF1zgjn+z3ZOAcqVOyTSu63tRQEQ6vnJhL
         13RQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1709250214; x=1709855014;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=GGmuzG0BuzfpsyKgKDkTT6Enikl3D6AjMcJQdJLpWcI=;
        b=xRNgxi74oTeCqH4klNgHBCoR/7P2biFt4g92j39BT8aq1kCd822iQThaD6ng0Xqei6
         2ZJ7lSOUaSODpm/5jnMk5pMBlbtEFzycd5Yo4osrfjlvtgJnVqtkWaSLbv1RplyumgJd
         fbxKDYykOR8M1odpNyCEM3nuvouG7zAGrQJjn0Vk7p1UKkSF8lqu5nnFyTYZdHfB0EYn
         lsL3/BBh6k0pJgntbx++YkruDVFO2VbPvx5nI1dmuEVmun32RDu2bka3o2SQbEUTnh0t
         x/7FmaS6qQsMq5Au580bpiWTaY+Xg8E+CRuKNziM6y1yd1XjNnLesR5bYm2oYJnfUQJp
         W8hA==
X-Forwarded-Encrypted: i=1; AJvYcCVtCKG0FiA1SnBWgZFXOB8Y9b6WSsmpDROZlN2LW/2uGU1BqKiTdlLCzXj6jU484488aFx0P/mBkMttAbHRXIngGNdfk7rvSRvuVrFjZcg=
X-Gm-Message-State: AOJu0Yx231nhX0lzkgIPcctbkTy+x/BqiUBO/xKKHAmXPoeccEz7Hhvs
	MRWyoBUZLtwedQpSO1QlbWKCiEjYT7OzAZfqp6Gz0cax1CCv31jR
X-Google-Smtp-Source: AGHT+IHI1rET3Z0/K8864m6GkWZwYbKsdcM+bbhi3ZQPOLm9p/XFzMkvrTJzDnJ+bGJeFJuwnOye6Q==
X-Received: by 2002:a05:6a00:92a2:b0:6e4:84db:e30e with SMTP id jw34-20020a056a0092a200b006e484dbe30emr186581pfb.32.1709250214399;
        Thu, 29 Feb 2024 15:43:34 -0800 (PST)
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: bpf@vger.kernel.org
Cc: daniel@iogearbox.net,
	andrii@kernel.org,
	torvalds@linux-foundation.org,
	brho@google.com,
	hannes@cmpxchg.org,
	lstoakes@gmail.com,
	akpm@linux-foundation.org,
	urezki@gmail.com,
	hch@infradead.org,
	boris.ostrovsky@oracle.com,
	sstabellini@kernel.org,
	jgross@suse.com,
	linux-mm@kvack.org,
	xen-devel@lists.xenproject.org,
	kernel-team@fb.com
Subject: [PATCH v3 bpf-next 3/3] mm: Introduce VM_SPARSE kind and vm_area_[un]map_pages().
Date: Thu, 29 Feb 2024 15:43:16 -0800
Message-Id: <20240229234316.44409-4-alexei.starovoitov@gmail.com>
X-Mailer: git-send-email 2.39.3 (Apple Git-145)
In-Reply-To: <20240229234316.44409-1-alexei.starovoitov@gmail.com>
References: <20240229234316.44409-1-alexei.starovoitov@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

From: Alexei Starovoitov <ast@kernel.org>

vmap/vmalloc APIs are used to map a set of pages into contiguous kernel
virtual space.

get_vm_area() with appropriate flag is used to request an area of kernel
address range. It'se used for vmalloc, vmap, ioremap, xen use cases.
- vmalloc use case dominates the usage. Such vm areas have VM_ALLOC flag.
- the areas created by vmap() function should be tagged with VM_MAP.
- ioremap areas are tagged with VM_IOREMAP.
- xen use cases are VM_XEN.

BPF would like to extend the vmap API to implement a lazily-populated
sparse, yet contiguous kernel virtual space. Introduce VM_SPARSE flag
and vm_area_map_pages(area, start_addr, count, pages) API to map a set
of pages within a given area.
It has the same sanity checks as vmap() does.
It also checks that get_vm_area() was created with VM_SPARSE flag
which identifies such areas in /proc/vmallocinfo
and returns zero pages on read through /proc/kcore.

The next commits will introduce bpf_arena which is a sparsely populated
shared memory region between bpf program and user space process. It will
map privately-managed pages into a sparse vm area with the following steps:

  // request virtual memory region during bpf prog verification
  area = get_vm_area(area_size, VM_SPARSE);

  // on demand
  vm_area_map_pages(area, kaddr, kend, pages);
  vm_area_unmap_pages(area, kaddr, kend);

  // after bpf program is detached and unloaded
  free_vm_area(area);

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
---
 include/linux/vmalloc.h |  5 ++++
 mm/vmalloc.c            | 59 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 62 insertions(+), 2 deletions(-)

diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index 71075ece0ed2..dfbcfb9f9a08 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -29,6 +29,7 @@ struct iov_iter;		/* in uio.h */
 #define VM_MAP_PUT_PAGES	0x00000200	/* put pages and free array in vfree */
 #define VM_ALLOW_HUGE_VMAP	0x00000400      /* Allow for huge pages on archs with HAVE_ARCH_HUGE_VMALLOC */
 #define VM_XEN			0x00000800	/* xen grant table and xenbus use cases */
+#define VM_SPARSE		0x00001000	/* sparse vm_area. not all pages are present. */
 
 #if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
 	!defined(CONFIG_KASAN_VMALLOC)
@@ -233,6 +234,10 @@ static inline bool is_vm_area_hugepages(const void *addr)
 }
 
 #ifdef CONFIG_MMU
+int vm_area_map_pages(struct vm_struct *area, unsigned long start,
+		      unsigned long end, struct page **pages);
+void vm_area_unmap_pages(struct vm_struct *area, unsigned long start,
+			 unsigned long end);
 void vunmap_range(unsigned long addr, unsigned long end);
 static inline void set_vm_flush_reset_perms(void *addr)
 {
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index d53ece3f38ee..dae98b1f78a8 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -648,6 +648,58 @@ static int vmap_pages_range(unsigned long addr, unsigned long end,
 	return err;
 }
 
+static int check_sparse_vm_area(struct vm_struct *area, unsigned long start,
+				unsigned long end)
+{
+	might_sleep();
+	if (WARN_ON_ONCE(area->flags & VM_FLUSH_RESET_PERMS))
+		return -EINVAL;
+	if (WARN_ON_ONCE(area->flags & VM_NO_GUARD))
+		return -EINVAL;
+	if (WARN_ON_ONCE(!(area->flags & VM_SPARSE)))
+		return -EINVAL;
+	if ((end - start) >> PAGE_SHIFT > totalram_pages())
+		return -E2BIG;
+	if (start < (unsigned long)area->addr ||
+	    (void *)end > area->addr + get_vm_area_size(area))
+		return -ERANGE;
+	return 0;
+}
+
+/**
+ * vm_area_map_pages - map pages inside given sparse vm_area
+ * @area: vm_area
+ * @start: start address inside vm_area
+ * @end: end address inside vm_area
+ * @pages: pages to map (always PAGE_SIZE pages)
+ */
+int vm_area_map_pages(struct vm_struct *area, unsigned long start,
+		      unsigned long end, struct page **pages)
+{
+	int err;
+
+	err = check_sparse_vm_area(area, start, end);
+	if (err)
+		return err;
+
+	return vmap_pages_range(start, end, PAGE_KERNEL, pages, PAGE_SHIFT);
+}
+
+/**
+ * vm_area_unmap_pages - unmap pages inside given sparse vm_area
+ * @area: vm_area
+ * @start: start address inside vm_area
+ * @end: end address inside vm_area
+ */
+void vm_area_unmap_pages(struct vm_struct *area, unsigned long start,
+			 unsigned long end)
+{
+	if (check_sparse_vm_area(area, start, end))
+		return;
+
+	vunmap_range(start, end);
+}
+
 int is_vmalloc_or_module_addr(const void *x)
 {
 	/*
@@ -3822,9 +3874,9 @@ long vread_iter(struct iov_iter *iter, const char *addr, size_t count)
 
 		if (flags & VMAP_RAM)
 			copied = vmap_ram_vread_iter(iter, addr, n, flags);
-		else if (!(vm && (vm->flags & (VM_IOREMAP | VM_XEN))))
+		else if (!(vm && (vm->flags & (VM_IOREMAP | VM_XEN | VM_SPARSE))))
 			copied = aligned_vread_iter(iter, addr, n);
-		else /* IOREMAP | XEN area is treated as memory hole */
+		else /* IOREMAP | XEN | SPARSE area is treated as memory hole */
 			copied = zero_iter(iter, n);
 
 		addr += copied;
@@ -4418,6 +4470,9 @@ static int s_show(struct seq_file *m, void *p)
 	if (v->flags & VM_XEN)
 		seq_puts(m, " xen");
 
+	if (v->flags & VM_SPARSE)
+		seq_puts(m, " sparse");
+
 	if (v->flags & VM_ALLOC)
 		seq_puts(m, " vmalloc");
 
-- 
2.34.1



